words from the senior editor welcome to the first issue of ijahp! today's economy, politics and social relations require that we consider interactions among all possible dimensions. the ijahp is a journal dedicated to the study, development and dissemination of a way of making decisions that departs from traditional approaches based on normative principles that cannot consider interactions among the factors that shape decisions. luis vargas senior editor ijahp preface: mu/words from the editor-in-chief of the ijahp  international journal of the analytic hierarchy process vol. 3, issue 1, 2011 issn 1936-6744   dear ijahp reader: this issue marks the beginning of our third year as a journal and we are glad to report our progress is solid and strong. the articles in this issue have a theoretical blend. von solms’s discussion of ahp/anp validity opens the issue followed by articles dealing with the use of anp in modeling problem situations, discussion of an ahp/anp approach to stability measurement, a survey of methods to make transitive multiple comparison matrices, and an interpretation of ahp synthesis as the eigensolution of an anp supermatrix. in addition, dr. saaty discusses, in our essay section, a common—but wrong—objection to the ahp method when measuring tangibles. we are also adding a news section in this issue for ahp-related news and events that may be of interest to our ijahp readers. enjoy the issue! enrique mu, phd editor-in-chief   rob typewritten text http://dx.doi.org/10.13033/ijahp.v3i1.89 ijahp: mu/ijahp articles are now more accessible than ever: full open access international journal of the analytic hierarchy process 1 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1032 ijahp articles are now more accessible than ever: full open access dear ijahp readers: the journal has made several important changes this year. first, we moved to a rolling article publication mode; that is, articles are published as soon as they are fully reviewed, accepted and copyedited. this allows quicker diffusion of the studies and authors’ work. our second innovation has been removing the need to register as a reader to access the journal contents. the original rationale for this was to assess the number of readers of the journal; however, it is now possible to obtain this information with many other tools. even though registration was free, it was still one more step to access the article and many people preferred to just skip registration and move to any other article in their google search. for these reasons, we have decided to eliminate the registration requirement for readers. there is another practical and important consequence of this action. given that access to the article is totally open and free, authors are allowed (actually encouraged) to upload the full published version of their articles in public repositories such as researchgate, academia, etc. it is expected that the lack of registration requirement and the storing of published articles in authors’ academic repositories will make ijahp authors’ research more accessible than ever; that is, they will be in full open access mode. enjoy our second ijahp issue of 2022! enrique mu, phd ijahp editor-in-chief 9_saaty_jagiellonian_award_dec_23_p_184 ijahp news: saaty/ honorary doctorate awarded   international journal of the analytic hierarchy process 185 vol. 3 issue 2 2011 issn 1936-6744     saaty  honorary  doctorate             thomas   saaty   (l.)   received   an   honorary   doctorate   (doctor  causa  honoris)   from   jagiellonian   university,   krakow,   poland,   in   october,   2011,   for   developing   the   analytic   hierarchy   process.   the   university,   one   of   the   oldest   and   most   distinguished  institutions  of  higher  education  in  europe,  renowned  for  being  the   university   where   the   great   astronomer   copernicus   studied,   and   where   pope   john  paul  ii  studied,  awarded  the  doctorate  to  saaty  in  conjunction  with  its  700th   anniversary   in   october.   he   was   commended   for   being   a   “true   man   of   renaissance...a  scholar  whose  talent  always  leads  him  fromo  theoretical  issues  to   the   human   aspects   and   practical   applications   of   science   for   the   benefit   of   mankind.     wiktor   adamus   (r.),   a   professor   at   jagiellonian   university   and   a   long-­‐‑time   advocate  of  ahp,    carried  out  a  study  in  2007  using  an  analytic  network  model   about   whether   or   not   poland,   a   member   of   the   european   community,   should   adopt   the   euro   currency.   although   a   member   of   the   european   community,   poland  had  never  adopted  the  unified  currency.  the  conclusion  was  that  poland,   at  least  for  the  next  few  years,  should  not  adopt  the  euro.  adamus’  study  won   first  prize  at  the  isahp2007  in  chile;  the  polish  government  did  not  adopt  the   euro;   and   that   appears   today   to   have   been   the   right   decision   in   view   of   the   stresses  today  in  the  euro-­‐‑zone. thomas saaty (l.) and wiktor adamus at award ceremony in krakow, poland, in october rob typewritten text http://dx.doi.org/10.13033/ijahp.v3i2.126 ijahp news and events: mu/2022 conferences suitable for ahp/anp decision-making submissions international journal of the analytic hierarchy process 1 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.990 2022 conferences suitable forahp/anp decision-making submissions there are two upcoming conferences with either tracks or sessions that are suitable to submit decision-making papers. these upcoming conferences are: the 33 rd central european conference on information and intelligent systems, ceciis 2022, will take place from september 21 st to 23 rd in dubrovnik, croatia, with a special emphasis on ‘smart industry in the post covid-19 society’. this conference is organized by the university of zagreb’s faculty of informatics, and there is a track on strategic planning and decision making which is very suitable for ahp/anp papers. dr. enrique mu, ijahp editor-in-chief, is honored to be part of the organizing committee. here is the link to the call for papers for your convenience: https://ceciis.foi.hr/call-forpapers/conference-papers also, the 21 st international conference on economy, finance and management, in its first polish-peruvian edition and entitled “new challenges in economic policy, business, and management,” will take place from october 23 rd to 25 th in wieliczka, poland. this is an opportunity to address new challenges in all these areas using decision-making analytic tools. the conference organizers have said that they are very open to presentations using ahp/anp for this purpose. three well-known members of our ahp ommunity will participate in this conference. professor dr. hab. wiktor adamus of jagiellonian university in cracow, poland is serving as a member of the scientific committee, dr. anna florek-paszkowska, professor of the business department, centrum, of the pontificia universidad católica del perú is the conference chair, and dr. enrique mu, ijahp editor-in-chief, will organize the ahp/anp session(s) and speak at the conference about ijahp publication opportunities. any questions related to ahp/anp presentations can be addressed to dr. mu via email at editor-in-chief@ijahp.org or to dr. florek-paszkowska via email: aflorekpaszkowska@pucp.edu.pe. more information is available on the conference webpage: https://konferencja.jemi.edu.pl/en/ https://ceciis.foi.hr/call-for-papers/conference-papers https://ceciis.foi.hr/call-for-papers/conference-papers mailto:editor-in-chief@ijahp.org mailto:aflorekpaszkowska@pucp.edu.pe https://konferencja.jemi.edu.pl/en/ ijahp news and events: mu/2022 conferences suitable for ahp/anp decision-making submissions international journal of the analytic hierarchy process 2 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.990 wieliczka city and salt mine (unesco) with underground lakes, chapels, and chandeliers made of salt ijahp news and events: cdf sponsoring grants for the 2019 mcdm conference in istanbul international journal of the analytic hierarchy process 507 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.631 cdf sponsoring grants for the 2019 mcdm conference in istanbul the 25th international conference on multiple criteria decision making will take place at istanbul technical university, istanbul, turkey from june 16th to june 21st, 2019 (www.mcdm2019.org). dr. ilker topcu, conference chairman mcdm xxv of istanbul technical university, istanbul, turkey the creative decision foundation (cdf) (www.creativedecisions.net), established by thomas l saaty and rozann w. saaty in 1996, is a non-profit research organization that aims to spread awareness of the analytic hierarchy process (ahp) and the analytic network process (anp) by sponsoring education, research, and software development in advanced methods of decision making. cdf has offered to support a limited number of grants for students, ph.d. candidates, post-docs or young researchers who have worked in ahp/anp to help them attend the mcdm 2019 conference by awarding a grant that will cover their conference fee. http://www.mcdm2019.org/ http://www.creativedecisions.net/ ijahp news and events: cdf sponsoring grants for the 2019 mcdm conference in istanbul international journal of the analytic hierarchy process 508 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.631 thomas l. saaty and ilker topcu at whistler, canada, mcdm conference in 2004 to apply for a grant:  first, the applicant should submit her/his abstract complying with the conference rules before the submission deadline of january 31, 2019, through the conference website submissions page (mcdm2019.org/submissions). the abstract should be related to ahp/anp.  second, the applicant should prepare and submit a manuscript of 6-10 pages extended from that abstract to the creative decisions foundation by emailing it to rozann saaty (email: rozann@creativedecisions.net) by april 1, 2019, along with a brief vita detailing their education, professional accomplishments, and level of familiarity with ahp/anp.  final step, the board of cdf in collaboration with senior representatives from the mcdm organization will review the submissions and by april 8, 2019 notify the applicants if they have received a grant. grantees will be required to affirm by email to the cdf contact and the mcdm contact before april 25 that they will be able to attend the conference. after they confirm they may then register for the conference using the cdf support link on the registration page. their conference fee will then be forwarded to mcdm from creative decisions foundation. grant recipients will have the opportunity to submit their manuscripts for publication in the international journal of the analytic hierarchical process (www.ijahp.com) accompanied by a special recognition message from the board of cdf. the ijahp journal is indexed in scopus, so it is helpful to a young researcher’s future to be published here. the user guide about formatting manuscripts for publication in ijahp can be downloaded at www.ijahp.org/styleguide.docx. of course, all manuscripts will still have to go through the usual two blind reviews process of the journal to be accepted for publication. good luck! https://mcdm2019.org/submissions mailto:rozann@creativedecisions.net http://www.ijahp.com/ http://www.ijahp.org/styleguide.docx ijahp: mu/words from the editor-in-chief international journal of the analytic hierarchy process 144 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.308 words from the editor-in-chief december 1, 2014 dear ijahp readers, the ijahp editorial team is proud to release our second issue for 2014. this was a particularly busy year since our ijahp team was instrumental in the success of the symposium of the analytic hierarchy process 2014 that took place in washington, d. c. valentina ferretti, our news section editor, has prepared a brief but interesting report on this event. those readers who did not have the opportunity to attend the event will be particularly interested in these highlights. this issue opens up with an article on the use of anp for the selection of new retail locations in italy. this is a very rigorous study by lombardi et al, and presents useful ideas for the use of this methodology for any anp-interested practitioner. the next article by guerrero and gomez-limon proposes a value-based approach toward the selection of strategic projects. this article makes us think about what value is, how to decompose it and how to use it as project selection criteria in organizations. jiri franek follows with a review and critique of mcda techniques in business and management. this serves as a useful reference about the different ways in which ahp has been used with business frameworks such as swot analysis, porter’s five forces and balanced scorecard. next, shih et al use a bocr anp modeling approach to analyze the problem of environmental impact on the selection of vendors for electronic firms. more interestingly, shih et al demonstrate the use of this model in a case study in taiwan. finally, rajguru et al report the results from a study that uses ahp to assess options such as chemical, organic and partial organic farming in the maharashtra state in india. we have also included in this issue an essay that may be particularly useful for new ahp/anp researchers. we have taken the opportunity to address the question of how to select a journal for publication of your research. in our experience, young researchers tend to be very anxious to jump at the first publication opportunity that arises without questioning why they want to publish in that particular journal, or even why they want to publish in the first place. in summary, our editorial team has worked very hard to prepare this ijahp issue, and we feel confident that you will find material that will be useful and interesting for our ahp/anp community. very sincerely, enrique mu, phd editor-in-chief ijahp news and events: petrillo, karpak/ theory and applications of ahp/anp in supply chain management session at informs 2021 international journal of the analytic hierarchy process 548 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.955 theory and applications of ahp/anp in supply chain management session at informs 2021 antonella petrillo università degli studi di napoli “parthenope” italy birsen karpak youngstown state university usa the informs annual meeting was in anaheim, ca, october 24-27, 2021 in person as well as virtual. the 2021 informs annual meeting was a unique opportunity to connect and network with the more than 6,500 members, students, prospective employers and employees, and academic and industry experts who comprise the informs community. figure 1 informs 2021 annual meeting the mcdm section of informs organized a stream of sessions chaired by margaret wiecek, serpil sayin, birsen karpak, antonella petrillo, caroline mota, daniele costa morais, adiel teixeira de almeida, samira fallah, firdevs ulus, ozlem karsu, and salvatore greco. in particular, during informs the session: “theory and applications of ahp/anp in supply chain management” took place and was organized by birsen karpak (youngstown state university – usa) and antonella petrillo (università degli studi di napoli “parthenope” italy). ijahp news and events: petrillo, karpak/ theory and applications of ahp/anp in supply chain management session at informs 2021 international journal of the analytic hierarchy process 549 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.955 in this session gregory zacharewicz (imt-mines ales, france) presented, “model-based approaches for interoperability of next generation supply chain management systems: state of the art and future challenges: contributions from ahp/anp”. figure 2 informs 2021 gregory zacharewicz also, sezin güleryüz (bartin university, turkey) presented “selection of sustainable suppliers using an integrated mcdm approach: a case study of turkey”. figure 3 informs 2021 sezin güleryüz in addition, birsen karpak (youngstown state university, usa), ilker topcu (istanbul technical university, turkey), fusun ulengin (sabanci university, turkey) presented ijahp news and events: petrillo, karpak/ theory and applications of ahp/anp in supply chain management session at informs 2021 international journal of the analytic hierarchy process 550 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.955 “analytic hierarchy process/analytic network process applications in big data enabled supply chain management”. figure 4 informs 2021 birsen karpak, ilker topcu and fusun ulengin finally, ozden bayazit (central washington university, usa) and birsen karpak (youngstown state university, usa) presented, “selection of a 3pl service provider for an aerospace company: analytic network process (anp) approach”. figure 4 informs 2021 ozden bayazit ijahp news and events: petrillo, karpak/ theory and applications of ahp/anp in supply chain management session at informs 2021 international journal of the analytic hierarchy process 551 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.955 we hope our presence at the informs 2022 meeting in indianapolis will be even greater and, hopefully, face-to-face. we look forward to seeing you next year! ijahp news and events: topcu/mcdm in turkey a second time international journal of the analytic hierarchy process 505 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.627 mcdm in turkey a second time the preparations for the 25th international conference on multiple criteria decision making (mcdm 2019) are going on. this time it will take place at istanbul technical university, istanbul, turkey from june 16th to june 21st, 2019. it will be the second conference that will be held in turkey; the first one was the 15th international conference which was held in ankara, turkey in 2000. the organizing committee is very pleased to host you again in turkey. www.mcdm2019.org is the official web site of the conference and the web site is up. organizers invite you to bookmark this page, and check it regularly for news, updates, and recent developments. you may also follow the meeting on twitter (twitter.com/mcdm2019) and on instagram (www.instagram.com/mcdm2019) the main theme of the conference will be “mcdm beyond the information age”. there will be several topics and application areas announced on the web site. it is expected that the theme, the topics and the application areas will be of broad interest to academicians and students as well as professionals. the conference will last for five days including a half-day outing (on wednesday afternoon and evening) and a banquet dinner (on thursday night). there may also be a get-together event before the actual conference (sunday evening). the following sessions will be part of the program:  invited plenary talks (4 sessions)  the award talks of the mcdm society award winners  the wiley practice prize session of the award finalists  the mcdm doctoral dissertation award session of the award finalists  invited sessions on special topics/interests and contributed sessions (16 parallel sessions)  opening and closing ceremony there will be plenary speeches from four distinguished speakers: pinar keskinocak (georgia tech, college of engineering) kathrin klamroth (university of wuppertal, school of mathematics and natural sciences) tuomas sandholm (carnegie mellon university, computer science department) luis vargas (university of pittsburgh, katz business school) ijahp news and events: topcu/mcdm in turkey a second time international journal of the analytic hierarchy process 506 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.627 the “outing on wednesday” will be at sakip sabanci museum. sabancı university's sakıp sabancı museum is located in emirgan, at one of istanbul's oldest settlements on the bosphorus. the ahp/anp community will be well represented at the conference with one plenary by vargas on ahp/anp, a tutorial on the subject as well as proposed invited sessions for ahp/anp and applications. we urge you to submit your own or participate either in invited or contributed papers. there is a big ahp/anp community in turkey. ahp/anp researchers as well as practitioners will have a great opportunity to discuss a variety of topics whether on theory or on applications including extensions of ahp/anp. the abstract submission system is now open and can be found via the web site. the deadline for submissions is january 30, 2019. we are also inviting special session organizers at this time. to organize a session or a stream of sessions, send your proposal to mcdm2019@itu.edu.tr by january 9, 2019. you are all most welcome to mcdm2019, the 25 th organization of mcdm conferences. i̇stanbul’da görüşmek üzere! (see you in istanbul!) sincerely, y. ilker topcu professor, istanbul technical university mcdm 2019 conference chair http://www.sakipsabancimuzesi.org/en mailto:mcdm2019@itu.edu.tr ijahp news: ferretti/isahp meeting in london international journal of the analytic hierarchy process 401 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.421 isahp meeting in london, august 4 – august 7, 2016 valentina ferretti london school of economics and political science department of management united kingdom this summer sunny london hosted an inspiring and interdisciplinary international symposium on the analytic hierarchy process/analytic network process (isahp 2016) (http://www.isahp.org/). this very successful meeting was organized by thomas l. saaty (university of pittsburgh, honorary founding chairman), leandro pecchia (university of warwick, conference chairman), andrea genovese (university of sheffield, program co-chair), marjan hummel (university of twente, program co-chair), antonella petrillo (university of naples "parthenope", program co-chair) with the help of the executive committee formed by enrique mu (carlow university university of pittsburgh, president of the isahp executive council, and editor-in-chief of the international journal of the analytic hierarchy process), rozann saaty (vice president creative decisions foundation), elena rokou (chief research officer, creative decisions foundation) and milagros pereyra (president, maestro meetings, executive director, latin american studies association lasa). the meeting saw strong and enthusiastic international participation with 136 paper proposals from 46 countries. to kick off the meeting and warm up the atmosphere with helpful scientific findings, the first day of the event started with a series of workshops dealing with “user-need elicitation via ahp: study design and piloting”, “anp sensitivity”, “overview of the new super decisions software (v3.0) for the analytic hierarchy process (ahp) and analytic network process” and, finally, “simplifying ahp to make it accessible to youth”. to maintain the inspirational character of the conference, the second day started with a video message recorded by tom saaty, who unfortunately could not attend the meeting but still wanted to be present and share some recent lessons learnt for the future of ahp and anp. the conference then hosted amazing keynote speakers: adriana velazquez berumen from the essential medicines and health products department in the world health organization, geneva, switzerland, james g. dolan m.d. from the university of rochester, rochester, ny, usa, edgardo iozia from the european economic and social committee, belgium, brussels, and prasanta kumar dey from aston university, birmingham, uk. for the first time, younger conference attendees at the meeting were given the opportunity to exchange ideas and get feedback from top authors in the ahp/anp field and have one-on-one time with them in our newly introduced mentoring sessions. these sessions allowed attendees to book time and discuss specific topics with highly acclaimed experts and members of the community. in addition, bill adams organized a session for children who demonstrated they could explore interesting questions and decisions using a simplified version of ahp. these initiatives were received with warm enthusiasm and strong participation! finally, the six pillars and highlights of isahp 2016 i were strongly interdisciplinary, ranging from “multi-criteria decision analysis methodology and theory”, to “government policy and decision making”, to “healthcare decision making”, to ijahp news: ferretti/isahp meeting in london international journal of the analytic hierarchy process 402 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.421 “applications in civil engineering and urban management”, to “industrial and manufacturing engineering” and, finally, to “business and innovation system”. we would also like to express our warm congratulations to the winners of the different prizes awarded at the conference. the isahp 2016 has been a truly international event which brought together researchers, teachers, students and users of ahp/anp to share their research and experiences in decision-making. many very interesting publication opportunities are associated with the event. if you are interested in knowing more about this please check the website for up to date highlights (http://www.isahp.org/callforpapers/?publication-opportunities-7). looking forward to seeing you all at isahp 2018! where do you guess it will be? participants celebrating together a great isahp 2016 meeting in london! http://www.isahp.org/callforpapers/?publication-opportunities-7 ijahp: mu/multicriteria decision making as interdisciplinary research international journal of the analytic hierarchy process 1 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.1080 multicriteria decision making as interdisciplinary research enrique mu, phd ijahp editor in chief in their excellent recounting of the development of multiple criteria decision making (mcdm) from its early history to the 21 st century, koksalan et al. (2011) proposed that mcdm is both old and new. it is old because decision makers have always had to make tradeoffs with objectives when making decisions; the authors refer to benjamin franklin’s approach to making decisions by trading off benefits and costs during the 1700’s. however, mcdm as an important sub-field of management science or operations research is rather new and began in the late 1950s. the elements of decision making fundamentally consist of the “decision”, a “decision-maker” and a “decision analysis methodology.” this is the reason that the mcdm field is inherently interdisciplinary. while applied mathematics is used for the decision quantitative analysis; the decision at hand, its content, needs to be addressed with tools of the corresponding discipline. this is further complicated because the decision topic may correspond to fields as diverse as medicine, sociology, or a variety of other fields. the multiple fields tackled by mcdm can easily be appreciated by taking a quick look at the list of articles that comprise the current ijahp issue and noticing the variety of disciplines involved in the target decisions. because of this, wallenius and wallenius (2023) argue in this issue for the mcdm profession to focus on important problems ranging from the risk of future pandemics to climate change. this need to focus on the discipline where the decision takes place is an important contingent aspect of mcdm that must be carefully considered and many times is not, in particular, by emerging scholars. this is one reason why the mcdm section of informs evaluates research candidates for the mcdm junior best paper research award, by allocating 30% value to the relevance of the decision and 20% to its potential impact on society while the remaining 50% considers the theoretical and methodological value of the mcdm analysis. that is, half the value of the research study (50%) focuses on aspects related to the decision itself (slowinski, 2023). in summary, since being an mcdm methodological expert does not make us decisiondiscipline experts, it is necessary to recruit expert participants from the disciplines in which we are focusing. different methodologies offer various ways to incorporate these experts into the decision-making process. in particular, the ahp/anp has proved to be quite accessible as a method to the inclusion of the judgments and preferences of discipline experts and stakeholders who do not need to understand mcdm or the nuts and bolts of a specific method such as ahp/anp (they just state their preferences in pairwise comparisons answering questions such as, “do you prefer a or b? and how strongly?”). still, even the ahp/anp requires that the mcdm analyst pay careful attention to the discipline on which the analysis is focusing. an example of the wide interdisciplinary nature of mcdm and the broad use of discipline experts is the recent international symposium of the analytic hierarchy process with the theme of “decision making in business practice.” presentations at this symposium ranged from topics such as the middle east conflict to the best ways to allocate human organs for transplants (isahp, 2022). https://ijahp.org/index.php/ijahp/issue/view/43 https://isahp.org/ ijahp: mu/multicriteria decision making as interdisciplinary research international journal of the analytic hierarchy process 2 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.1080 another strategy to deal with the dichotomy of “decision content” and “decision-making analysis” is a bottom up approach. this involves giving discipline experts a simple method to learn and understand and the tools to tackle the decision problem with which they are so familiar. ahp/anp creator, thomas l. saaty (1980), dreamed of an mcdm tool so easy that anyone, even those without sophisticated mathematical knowledge, could use. the simplicity of the method is also the reason why disciplinary conferences in the areas of finance, economy and management, for example, are now including the ahp/anp as specific tracks, as in the case of the recent icefm2022 conference in krakov, poland (ujwary-gil et al., 2023). in an ideal world, we would expect many other disciplinary conferences (e.g. medicine, law) to promote the ahp/anp as part of the toolkit of decision-making for the specific discipline. in conclusion, “decision-content” and “decision analysis” are two of the three intertwined but different dimensions of decision-making and can be approached with a mcdm analyst consulting with experts about the decision or with decision experts learning a friendly mcdm methodology, like the ahp/anp, to perform their own analysis. the third aspect of mcdm is the “decision-maker” and this recognition has led to the development of behavioral decision-making in the case of individuals (simon, 1955), group decision-making in the case of groups or teams (saaty & peniwati, 2008) and public decision-making in the case of decisions affecting a large number of stakeholders (gonzalez-urango et al., 2021); however, this constitutes a topic for future discussions. https://konferencja.jemi.edu.pl/en/ ijahp: mu/multicriteria decision making as interdisciplinary research international journal of the analytic hierarchy process 3 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.1080 references gonzalez-urango, h., mu, e., & garcía-melón, m. (2021). stakeholder engagement and anp best research practices in sustainable territorrial and urban strategic planning. in m. doumpos, f. ferreira, & c. zopounidis (eds.), multiple criteria decision making for sustainable development. springer nature. doi: https://doi.org/10.1007/978-3-03089277-7_5 isahp. (2022). decision-making in business practice. international symposium of the analytic hierarchy process. koksalan, m., wallenius, j., & zionts, s. (2011). multiple criteria decision making: from early history to the 21st century. london: world scientific. doi: https://doi.org/10.1142/8042 saaty, t. l. (1980). decision making for leaders: the analytic hierarchy process. mcgraw-hill. saaty, t. l., & peniwati, k. (2008). group decision making: drawing out and reconciling differences. pittsburgh, pa: rws publications. simon, h. a. (1955). a behavioral model of rational choice. quarterly journal of economics, 69, 99-118. slowinski, r. (2023). mcdm junior researcher best paper award. international journal of the analytic hierarchy process, 14(3). https://doi.org/10.13033/ijahp.v14i3.1068. ujwary-gil, a., florek-pazkowska, a., & koziol, a. (2023). ahp/anp applications in economic policy, business, and management with the t. l. saaty decision making hackathon at icefm 2022. international journal of the analytic hierarchy process, 14(3). https://doi.org/https://doi.org/10.13033/ijahp.v14i3.1071 wallenius, h., & wallenius, j. (2023). focusing on important problems. international journal of the analytic hierarchy process, 14(3). https://doi.org/https://doi.org/10.13033/ijahp.v14i3.1066 ijahp essay: gonzalez-urango/how the analytic hierarchy/network process supports more responsible and committed research and innovation international journal of the analytic hierarchy process 553 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.949 how the analytic hierarchy/network process supports more responsible and committed research and innovation hannia gonzalez-urango gonzalezurango@gmail.com 1. introduction research performing and funding organizations worldwide are increasingly calling for better research and innovation (r&i), societal integration with the grand challenges facing society and the creation of sustainable economic growth. in this context, responsible research and innovation (rri) emerged in the previous decade. rri refers to a process of research, development and innovation that takes into account the effects and potential impacts on the environment and society (stilgoe et al., 2013; von schomberg, 2013). furthermore, rri is part of a broader set of ideas and initiatives that address socially accountable and responsible innovation. in effect, several studies show an increasing interest by researchers to align their innovative work with their own values and societal interests (mu & pereyra-rojas, 2015; pereyra-rojas et al., 2017). the rri concept has been studied from various disciplinary perspectives (wiarda et al., 2021), and has gained traction mainly due to its inclusion in european science policies since 2010, which have been an undeniable driving force. the political will to better align research with societal needs and concerns is leading to ambitious policies and processes to transform the r&i system, including rri. to accomplish these goals, it is necessary to use managerial tools that facilitate the engagement process of the different rri stakeholders, and this is where the analytic hierarchy/network process (ahp/anp), developed by saaty (1980, 2001), can play an important role. 2. responsible research innovation approach the rri approach has gained increasing attention since it first appeared. it has been incorporated into the european union's framework programmes, of which it is still part, and has taken form through scientific publications, books, manuals, conferences, and mainly projects. more than 200 projects have been financed by the eu since the approach was incorporated into the framework 1 . the quick and mainstreamed eruption of rri onto the european scientific agenda caused a reaction from the scientific community, which reflects on how it might be articulated and put into practice, how to measure efforts in this context, and how to integrate the efforts made in other areas of knowledge. however, this process is at a stage where rri as such has not yet been systematically defined or integrated into many of the policies that promote r&i (research and 1 among the projects financed, these stand out: great project (2013), resagora (2013), parrise (2014), responsible-industry (2014), morri (2015) and super-morri (2018), rritools (2016), multi-act’s (2019), youcount (2021), and many others. https://res-agora.eu/ http://www.parrise.eu/ http://www.responsible-industry.eu/ https://www.technopolis-group.com/morri/) https://www.super-morri.eu/super-morri/index.php http://www.rri-tools.eu/ http://www.rri-tools.eu/ https://www.multiact.eu/ https://www.youcountproject.eu/ ijahp essay: gonzalez-urango/how the analytic hierarchy/network process supports more responsible and committed research and innovation international journal of the analytic hierarchy process 554 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.949 innovation) at the country or regional level. the term rri is unknown to the general public, to many people working in the fields of science, technology and innovation, and even more unknown beyond the eu's borders. current publications in rri predominantly originate from european countries and are funded by european institutions or agencies. rri involves the following six key areas (strand et al., 2015): 1. ethics: the ethics dimension in the rri framework addresses the need to ensure the social relevance and ethical acceptability of science and innovation results. it considers what “in order to adequately respond to societal challenges, research and innovation must respect fundamental rights and the highest ethical standards” means. 2. public engagement: this area seeks to promote greater interaction and two-way dialogue between diverse actors about science and innovation related issues. it is the “engagement of all the societal actors (research, industry, policy-makers, and civil society) and their joint participation in the research and innovation process”. 3. gender equality: this area promotes an “all actors (women and men) are on board” ideal. it seeks a greater effective presence of women, but also a greater global openness to people who challenge traditional gender stereotypes. 4. open access: this is defined as access to bibliographic information and data, as well as research methods and results. it means “giving free online access to the results of publicly-funded research (publications and data)”. 5. science education: this refers to all activities that are aimed at facilitating greater interaction among citizens, researchers and other social actors. it seeks to promote interest in science and scientific vocations. 6. governance: this refers to any form or coordination designed to foster and mainstream rri within an organization or in the interaction with other stakeholders. it is a key umbrella area that states that policy-makers “have a responsibility to prevent harmful or unethical developments in research and innovation”. some authors also propose the inclusion of the following areas: 7. social justice: this can be defined as the promotion by science and technology of an ideal condition in which all citizens have equal rights, opportunities and access to social resources. 8. environmental sustainability: this involves the diagnosis and management of environmental impacts, both positive and negative. it addresses the relationship between nature and r&i, and how to incorporate it into responsible r&i. as has been clearly expressed, articulating rri requires that societal actors (researchers, citizens, policymakers, business, third sector organizations, etc.) work together during the whole research and innovation process in order to better align the process and its outcomes with the values, needs and expectations of society (strand et al., 2015). articulating rri requires a combination of different strategies and methods, the involvement of different actors (stilgoe et al., 2013) and consideration of the realities of each context (mejlgaard et al., 2018). thus, rri imposes greater complexity for r&i practitioners and policymakers. establishing scientific policies that address rri in a ijahp essay: gonzalez-urango/how the analytic hierarchy/network process supports more responsible and committed research and innovation international journal of the analytic hierarchy process 555 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.949 comprehensive way is necessary in order to be able to analyze the specific characteristics of the different r&i systems at different levels of analysis (macro, meso and micro). hence, the operationalization of rri practices is currently a hot issue. some authors advocate larger and more comprehensive studies to test specific rri theories. this is especially valid in light of the broad, diverse, and still explorative nature of contemporary rri (wiarda et al., 2021). 3. ahp/anp support of more responsible research and innovation using rri as a new paradigm requires a different approach to the design, monitoring and evaluation of research agendas and policies. to establish and implement rri, several metrics or evaluation approaches have been generated. different discourses from different perspectives have taken place to address the practices of industries, governments, and public research institutes. as previously mentioned, case studies are vital to identify potential propositions that, using larger and more comprehensive studies, can lead to the testing of specific rri theories (wiarda et al., 2021). some recent examples of helpful exploratory case studies constitute a magnificent example of the use of the ahp/anp approach to address some complex decisions with regard to monitoring and promoting rri. the ahp/anp are useful tools for prioritizing and defining which metrics contribute the most to rri monitoring and evaluation at different levels. these methods also have the important advantage of allowing the inclusion of the different rri actors in the decisionmaking process even if they are not familiar with the technical intricacies of the actual process. there are some experiences where the ahp/anp has made it possible to contribute to the body of knowledge applied to rri monitoring at different levels. at the national level, in spain, the inperri project (2016-2019) proposed concise lists of rri indicators to monitor spain’s science and innovation policies in the eight rri areas or perspectives (gonzalez-urango, garcía-melón et al., 2020; monsonís-payá et al., 2017; otero-hermida & garcía-melón, 2018). inperri focused on the co-creation of indicators aimed at identifying specific demands in the spanish research and innovation context. the results assist spanish policymakers and research funding organizations in monitoring their policies in accordance with rri principles. the development of these lists was mainly based on the analysis of existing projects and literature, as well as the participation of different experts in participatory sessions organized for each of the rri areas. for science education, a social network analysis was carried out, following the methodology proposed by gonzalez-urango (2020), to detect the most relevant actors within the field of science education in spain. participatory group sessions were conducted to produce the lists of indicators, which were then prioritized using the ahp (vinagre fernandez, 2019). for public engagement, qualitative content analysis techniques were used for the in-depth study of the deliberative process and the generation of indicators, and the ahp was used for the prioritization of the indicators that were developed (garcía-melón et al., forthcoming). ijahp essay: gonzalez-urango/how the analytic hierarchy/network process supports more responsible and committed research and innovation international journal of the analytic hierarchy process 556 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.949 another particular study in the gender area has been developed into a tool for monitoring and assessing the gender gap at academic events (corona-sobrino et al., 2020). the study is based on a combination of qualitative analysis, ahp and ahp sort (ishizaka et al., 2012) to design specific lists of performance indicators and the thresholds required to sort the results with a traffic light signal. these techniques have allowed the indicators to be weighted as well as the generation of some composite indicators. finally, at the micro level (project), ligardo-herrera et al.(2019) assess the stakeholders’ influence in a research project within the context of responsible research and innovation. the anp method allowed ranking and ordering the project’s stakeholders based on their influence on the anticipation of the responsibility of the project and its possible outcomes. the purpose of this assessment was to help a research team more efficiently devote their limited resources to stakeholder management. all these experiences have involved a typical ahp/anp prioritization process. the findings have been concrete results that can be used to implement and promote rri. they have all involved the active participation of different actors and can therefore be applied at national policy levels, and at the project, institutional and/or individual level, allowing for a wide variety of uses. 4. conclusion rather than being an exhaustive explanation of rri, the intention of this essay is to briefly present an interesting framework that the european union has been building around science that is more aware of global challenges. a second intention is to highlight the usefulness of the ahp/anp as valuable tools to handle the complexities derived from the implementation of a multidimensional concept such as rri. the proposed ahp/anp applications are useful in the context of evaluation, learning or comparison. all the experiences emphasize the importance of including different techniques to collect and analyze the opinions of stakeholders (content analysis of documents, interviews and focus groups; participatory sessions, or social network analysis (sna)). these combinations are useful in studies that seek to tackle complex and sometimes controversial and difficult problems. however, they require a high degree of participation of stakeholders. beyond the traditional identification of stakeholders, the ideal objective is to achieve their effective inclusion. this means the highest level of involvement, according to the stakeholder engagement levels proposed by gonzalezurango et. al (2022), in such a way that they can contribute valuable knowledge that complements scientific expertise and enriches the knowledge base. ijahp essay: gonzalez-urango/how the analytic hierarchy/network process supports more responsible and committed research and innovation international journal of the analytic hierarchy process 557 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.949 references corona-sobrino, c., garcía-melón, m., poveda-bautista, r., & gonzalez-urango, h. (2020). closing the gender gap at academic conferences : a tool for monitoring and assessing academic events. plos one, 15(12), 1–23. doi: https://doi.org/10.1371/journal.pone.0243549 garcía-melón, m., gómez-navarro, t., gonzalez-urango, h., & corona-sobrino, c. (forthcoming.). adapting rri public engagement indicators to the spanish scientific and innovation context. a participatory methodology based on ahp and content analysis. central european journal of operations research. gonzalez-urango, hannia garcía-melón, m., monsonís payá, i., otero-hermida, p., garcía lozano, j. f., gómez navarro, t., miralles, c., & jiménez sáez, f. (2020). propuesta de indicadores para impulsar el diseño de una política orientada al desarrollo de investigación e innovación responsable en españa. doi: https://doi.org/10.5281/zenodo.4293484 gonzalez-urango, h. (2020). a participatory multicriteria approach to evaluate strategic projects for local sustainable development [universitat politècnica de valència, valencia, spain (2020)]. doi: https://doi.org/10.5281/zenodo.3992673 gonzalez-urango, h., mu, e., & garcía-melón, m. (2022). stakeholder engagement and anp best research practices in sustainable territorial and urban strategic planning. in m. doumpos, f. a. f. ferreira, & c. zopounidis (eds.), multiple criteria decision making for sustainable development. springer international publishing. ishizaka, a., pearman, c., & nemery, p. (2012). ahpsort: an ahp-based method for sorting problems. international journal of production research, 50(17), 4767–4784. doi: https://doi.org/10.1080/00207543.2012.657966 ligardo-herrera, i., gómez-navarro, t., & gonzalez-urango, h. (2019). application of the anp to the prioritization of project stakeholders in the context of responsible research and innovation. central european journal of operations research, 27(3), 679–701. doi: https://doi.org/10.1007/s10100-018-0573-4 mejlgaard, n., woolley, r., bloch, c., bührer, s., griessler, e., jäger, a., lindner, r., madsen, e. b., maier, f., meijer, i., peter, v., stilgoe, j., & wuketich, m. (2018). a key moment for european science policy. journal of science communication, 17(3), 1–6. doi: https://doi.org/10.22323/2.17030305 monsonís-payá, i., garcía-melón, m., & lozano, j. f. (2017). indicators for responsible research and innovation: a methodological proposal for context-based weighting. sustainability (switzerland), 9(12). doi: https://doi.org/10.3390/su9122168 mu, e., & pereyra-rojas, m. (2015). impact on society versus impact on knowledge: why latin american scholars do not participate in latin american studies. latin american research review, 50(2), 216–238. doi: https://www.jstor.org/stable/43670297 ijahp essay: gonzalez-urango/how the analytic hierarchy/network process supports more responsible and committed research and innovation international journal of the analytic hierarchy process 558 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.949 otero-hermida, p., & garcía-melón, m. (2018). gender equality indicators for research and innovation from a responsible perspective: the case of spain. sustainability (switzerland), 10(9). doi: https://doi.org/10.3390/su10092980 pereyra-rojas, m., mu, e., gaskin, j., & lingham, t. (2017). the higher-ed organizational-scholar tension: how scholarship compatibility and the alignment of organizational and faculty skills, values and support affects scholar’s performance and well-being. frontiers in psychology, 8(mar), 450. doi: https://doi.org/10.3389/fpsyg.2017.00450/bibtex saaty, t. l. (1980). decision-making for leaders: the analytic hierarchy process. mcgraw hill. saaty, t. l. (2001). decision making with dependence and feedback: the analytic network proces. pittsburgh, pa: rws publications. stilgoe, j., owen, r., & macnaghten, p. (2013). developing a framework for responsible innovation. research policy, 42(9), 1568–1580. doi: https://doi.org/http://dx.doi.org/10.1016/j.respol.2013.05.008 strand, r., spaapen, j., bauer, m. w., hogan, e., revuelta, g., stagl, s., paula, l., & guimarães pereira, â. (2015). indicators for promoting and monitoring responsible research and innovation report from the expert group on policy indicators for responsible research and innovation. european commission directorate-general for research and innovation. vinagre fernandez, m. r. (2019). aplicación del análisis de redes sociales y la técnica ahp al desarrollo de indicadores para monitorizar la educación para la ciencia en la política científica española [universitat politècnica de valència]. doi: https://riunet.upv.es/handle/10251/125064 von schomberg, r. (2013). a vision of responsible innovation (r. owen, m. heintz, & j. bessant (eds.)). wiley. doi: https://doi.org/10.1002/9781118551424.ch3 wiarda, m., van de kaa, g., yaghmaei, e., & doorn, n. (2021). a comprehensive appraisal of responsible research and innovation: from roots to leaves. technological forecasting and social change, 172, 121053. doi: https://doi.org/10.1016/j.techfore.2021.121053 microsoft word 3ninaarticle_pp31_44_03212009 ijahp article: begičević, divjak & hunjak: comparing ahp and anp in a case study of e-learning implementation international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 31 comparison between ahp and anp: case study of strategic planning of e-learning implementation nina begičević e-mail: nina.begicevic@foi.hr blaženka divjak e-mail: blazenka.divjak@foi.hr tihomir hunjak e-mail : tihomir.hunjak@foi.hr university of zagreb faculty of organization and informatics pavlinska 2, 42 000 varaždin abstract the specific objectives of the paper are: validation of the theoretical model for strategic planning of e-learning implementation by means of factor analysis, presentation of the structure of ahp and anp models for decision making about e-learning implementation, comparison between developed ahp and anp models, analysis of the results of group decision making supported by the software teamec2000 based on ahp model and the results of decision making supported by the software super decisions 1.6.0., which is based on an anp model. the models can be applied at the course level, for the group of courses (department level), or at the study program (faculty level). at the same time, the proposed models are useful for structuring discussion on strategic decisions on e-learning implementation at the university level. our aim is to analyze and identify advantages and disadvantages of using different models and tools in the process of decision making about e-learning at different organizational levels. keywords: e-learning, ahp, anp, factor analysis rob typewritten text http://dx.doi.org/10.13033/ijahp.v1i1.5 ijahp article: begičević, divjak & hunjak: comparing ahp and anp in a case study of e-learning implementation international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 32 1. introduction e-learning is usually defined as a type of learning supported by information and communication technology (ict) that improves quality of teaching and learning. implementing e-learning contributes to the advancement of higher education (he). an elearning system is a powerful tool for achieving strategic objectives of the university (teaching, research and serving the society) and it contributes to progress on the institutional level as well as at the personal level for teaching staff and students (divjak and begicevic, 2006). it supports collecting, analyzing and applying information appropriately, and comprises different teaching methods, which include information management, creative thinking, critical thinking, problem solving and collaborative learning (bates, 2005). generally speaking, universities in croatia and some other european universities are currently at the stage of strategic planning and deciding about the systematic implementation of e-learning in the existing academic activities. strategic planning and decision making about e-learning implementation is one of the aims of the tempus eqibelt project (eqibelt, 2006), coordinated by the university of zagreb, which provides a useful platform for our research. in our paper, we will present the possibility of using ahp and anp models and statistical techniques in strategic planning and decision-making about e-learning. 2. objectives and research methodology the overall objectives of the research are: • to provide a basis for decision making for members of the eqibelt project team and university strategy teams in the process of creating e-learning vision and strategic documents • to develop the general model for decision making about e-learning implementation in he based on theoretical findings and surveys results • to complete the factor analysis, validate the theoretical model and reduce a large number of variables to a smaller number of factors, i.e. design an improved theoretical model for modeling purposes • to develop ahp and anp models for decision making about e-learning implementation in he • to compare decision models for e-learning implementation in he based on some other research methods or built-in questionnaires including experts from other countries the specific objectives of this paper are: • presentation and analysis of the results of questionnaire performed on expert group • presentation of the theoretical model for decision making about e-learning implementation in he, by means of factor analysis • developed structure of ahp model for strategic planning of e-learning implementation on course and department level ijahp article: begičević, divjak & hunjak: comparing ahp and anp in a case study of e-learning implementation international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 33 • presentation of the results of group decision making on e-learning implementation supported by the software teamec2000 • developed structure of anp model for strategic planning of e-learning implementation on institutional level • presentation and analysis of the results of expert decision making on e-learning implementation supported by the software super decisions we have treated decision-making as consisting of four phases: (1) intelligence, (2) design, (3) choice, and (4) implementation. more details can be found in (begicevic, divjak, hunjak, 2006). the alternatives in the decision making process on e-learning implementation is: • face-to-face learning, • ict supported face-to-face learning, • blended learning, and • fully online learning. in the statistical evaluation of the results we have used factor analysis to validate the theoretical model for decision making about e-learning implementation. we have connected the results of the survey using factor analysis and these results have served as input in the multicriteria decision model (ahp) that we have developed and described in (divjak and begicevic, 2006). in the decision making phase we have solved the problem of choosing the best option for e-learning implementation. this was done with the assistance of an ahp model developed and validated in the process of group decision-making supported by the software teamec2000. we have also developed an anp model, which was used for defining the structure of the strategies for e-learning at the institutional level. the action plan and the monitoring system have followed the decision making phase. 3. questionnaire description and response after we had developed the theoretical model for decision making about e-learning implementation, we created a questionnaire about the importance of the advantages and goals of e-learning implementation and about criteria and subcriteria essential for decision making about e-learning implementation. the alternatives were not included in the questionnaire, but an explanation of each criterion/subcriterion was attached to the questionnaire. we carried out the survey collecting a total of 90 questionnaires. the participants were vice-rectors, vice-deans, members of relevant university bodies, members of government bodies responsible for the implementation of e-learning methodology and technology, members of the eqibelt project team and university strategy teams, university teachers and student representatives involved or interested in e-learning, etc. the criteria for the selection of experts were an expertise in e-learning, and familiarity with the he environment. hence, the survey represented a group of e-learning experts in croatia. ijahp article: begičević, divjak & hunjak: comparing ahp and anp in a case study of e-learning implementation international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 34 4. the results of the survey: ranking of criteria and subcriteria in this section, we present some of the results of the survey from 90 experts on e-learning in he in croatia. the complete results of the survey can be found in the paper “development of ahp based model for decision making on e-learning implementation” (begicevic, divjak, hunjak, 2006). in all questions, the discrete scale used for importance was from 1 to 5. all the proposed criteria were accepted as important, but four of them ranked above the average mark of four: organizational readiness of environment, development of human resources, availability of human resources and availability of basic ict infrastructure. legal and formal readiness of environment and availability of specific ict infrastructure ranked below average (figure 1). this last ranking reflects the stage of development of elearning in croatia, which is generally below the eu level, and therefore the importance of legal framework and appropriate ict infrastructure is not recognized. details about ranking the proposed subcriteria are shown in table 1. the importance of criteria 4,38 4,28 4,27 4,24 3,94 3,74 3,4 3,6 3,8 4 4,2 4,4 4,6 criteria r at in gs organizational readiness of environment development of human resources availability of human resources availability of basic ict infrastructure legal and formal readiness of environment availability of specific ict infrastructure figure 1 results of the survey importance of criteria. table 1 results of the survey importance of subcriteria. organizational readiness of environment faculty strategy for development 4.54 organizational readiness of universities/faculties for elearning implementation 4.42 university framework for development 4.34 financial readiness of universities/faculties for e-learning 4.21 ijahp article: begičević, divjak & hunjak: comparing ahp and anp in a case study of e-learning implementation international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 35 implementation availability of basic ict infrastructure network infrastructure 4.50 teachers and students equipped with computers 4.43 classrooms equipped for e-learning 4.17 integral information system of universities/faculties 3.86 development of human resources continuous training of academic staff 4.63 continuous training of support staff 4.17 training of students for use of e-learning 4.04 legal and formal readiness of environment evaluation and quality control at universities/faculties 4.20 system and criteria for academic staff promotion 4.04 standardization of digital educational materials 4.03 protecting intellectual property rights on state and academic level 3.49 availability of human resources specialized e-learning centers at universities 4.56 availability of technical support staff for e-learning 4.36 availability of support staff for graphic design. animation and video 4.09 availability of support staff for methodology of e-learning 4.08 availability of specific ict infrastructure virtual learning environment 4.31 managed learning environment 4.06 library management system 3.97 production of video and audio materials 3.61 network videoconferencing system 3.60 exam management system 3.57 video and audio streaming 3.49 systems for simulation and virtual environment 3.32 5. results of the factor analysis factor analysis is a statistical data reduction technique that can simultaneously manage over a hundred variables, compensate for random error and invalidity, and disentangle complex interrelationships into their major and distinct regularities (rummel, 1967). it is used to explain variability among observed random variables in terms of fewer unobserved random variables called factors. we have used factor analysis to validate the theoretical model (table 2), to reduce a large number of variables to a smaller number of factors for modelling purposes (ahp modelling), to specify the strength of the relationship between each factor and each variable and to determine which sets of items should be grouped together in the theoretical model. the complete results of the factor analysis are presented in the paper (begicevic and divjak, 2006). ijahp article: begičević, divjak & hunjak: comparing ahp and anp in a case study of e-learning implementation international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 36 the extraction method, which was used in the factor analysis, was principal component analysis (brace et al., 2000) and the rotation method was the orthogonal varimax rotation (brace et al., 2000) with kaiser normalization. the number of factors was specified, m = 5 (5 factors were recognized in the theoretical model). the factor analysis was performed with the support of the statistical program spss (brace et al., 2000). we set the lower boundary for the projection of variable variance on the factor at 0.519 and noticed that 6 variables did not correlate above 0.519 with the principal components of the original correlation matrix and therefore we excluded them from the model. moreover, 5 out of the above mentioned 6 variables related almost equally to two or three factors. finally, the new theoretical model was reduced to 21 variables (begicevic and divjak, 2006). experts did not agree upon the importance of protecting intellectual property rights and standardization of digital educational materials, and in our opinion, it shows that in general the present state of e-learning implementation in he in croatia is at a rather early stage. furthermore, the variables training of students for use of e-learning, integral information system of universities/faculties, virtual learning environment and organizational readiness of universities/faculties for e-learning implementation were excluded because of redundancy with other variables in the theoretical model. the factor analysis results have also confirmed 5 factors of the theoretical model for decision making about e-learning implementation (table 2). not only does the factor analysis confirm the major findings of prior data acquisition and analysis, but it also refines and better restructures our first theoretical model. we assume that there are two reasons for correspondence between the two models: first, the fact that the qualitative analysis in the first part of research was thorough as a considerable sample of strategic documents on e-learning was used, and second, experts were involved in the survey. the latter were essential because of the highly specific area, which requires both familiarity with e-learning and expertise in the he environment. table 2 results of factor analysis (rotated component matrix). f 1 f 2 f 3 f 4 f 5 f 1 human resources availability of support staff for methodology of e-learning .883 3.415e-02 5.202e-02 -1.120e-02 -4.832e-02 availability of technical support staff for e-learning .835 6.881e-02 .119 2.543e-02 .103 availability of support staff for graphic design. animation and video .761 .118 9.200e-02 .105 1.353e-02 continuous training of support staff .709 .146 .164 .196 .106 specialized e-learning centers .652 -1.242e-03 .176 .206 4.064e-02 continuous training of academic staff .610 .175 .139 .238 .156 ijahp article: begičević, divjak & hunjak: comparing ahp and anp in a case study of e-learning implementation international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 37 f 2 specific ict infrastructure for e-learning video and audio streaming -.196 .840 9.800e-02 -1.927e-03 .108 network videoconferencing system -5.610e-02 .806 .176 .204 .154 systems for simulation and virtual environment .265 .784 -9.944e-02 9.253e-02 .153 production of video and audio materials .214 .769 9.195e-02 -9.597e-03 -4.100e-02 exam management system .160 .609 .254 .136 -.101 library management system .242 .603 .179 9.750e-02 -.276 f 3 basic ict infrastructure for e-learning network infrastructure .163 .193 .778 .107 3.312e-02 teachers and students equipped with computers .266 .105 .720 -6.693e-02 -1.287e-02 classrooms equipped for e-learning -3.167e-02 .183 .625 2.887e-02 .564 managed learning environment .268 .233 .528 .417 -.240 f 4 strategic readiness for e-learning implementation faculty strategy for development .191 3.302e-02 5.800e-02 .792 .154 university framework for development 9.796e-02 .282 -.100 .662 -3.168e-02 financial readiness of universities/faculties for elearning implementation .194 -3.291e-02 .397 .558 7.218e-02 f 5 legal and formal readiness for e-learning implementation system and criteria for academic staff promotion .123 -9.182e-02 -4.377e-03 2.484e-02 .807 evaluation and quality control at universities/faculties .340 .251 6.778e-03 .289 .512 6. an ahp based model for decision making at the course level in the choice phase, we developed an ahp based model for decision making on elearning implementation based on the reduced and restructured theoretical model (21 variables). we have built the ahp model using the teamec2000 software (ec 2000) which is specially designed for making group decisions. we take into account that a group can generate a higher number of ideas and usually know more than an individual does. it is also important that a group is more likely to make the riskier decisions, since risk is shared among all group members. in our case of “decision making on the most suitable option for implementing elearning” for the mathematics course on the faculty of organization and informatics, we were using teamec2000 with wireless electronic keypads for 5 decision makers (participants) and top down structuring with numerical judgments mode. the model and the methodology can also be applied for a group of courses (department level). ijahp article: begičević, divjak & hunjak: comparing ahp and anp in a case study of e-learning implementation international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 38 all participants in the group decision-making process have specific knowledge, which makes them competent to assess and judge the most suitable options for e-learning implementation in the course mathematics on faculty of organization and informatics. the participants were given detailed instructions on definitions of criteria and subcriteria and the tool that would be used, a week before the decision-making event, in order to familiarize themselves with the task. the competencies of the group members are as follows: one participant is an associate professor and main lecturer of mathematics, she has a ph.d. in mathematics, and she is familiar with the strategic planning of e-learning at the faculty and university level. the second participant is an assistant in mathematics and has an ma in mathematics. two other participants have mas in information science and they are also phd students. one of them is an assistant at informatics and one is an administrator of the learning management system (lms) at the faculty of organization and informatics. the fifth participant is a student at faculty and a student tutor for mathematics. during their studies and training, they were included several times in lectures where e-learning was used to support traditional classroom teaching. four of them are involved in creating courses that integrate e–learning and traditional classroom teaching. all participants are working on e-learning projects. three out of five are experts in programming and have experience in developing the necessary infrastructure to implement e-learning courses. all participants are authors or co-authors of several scientific and professional papers in the area of e-learning. these experts form a heterogeneous group of decision makers. the group possesses the knowledge and responsibility to initiate and implement decisions about the most suitable option for e-learning implementation at the course level. the results of the group decision making incorporated the knowledge of all stakeholders provided in the process of group decision making, and the process concluded with a recommendation for applying the most suitable option for implementing e-learning. the results of every participant’s decision-making model and the results of the group decision-making model were available after the meeting. the results of the teamec2000 group decision-making session were a hierarchic model, the objectives’ relative significance, and the priorities of the alternatives, obtained by synthesizing the judgments of the active participants shown in figure 2. the legal and formal readiness for e-learning implementation criterion has the highest relative significance – 0,351, which makes it the most important for reaching the goal. the reasons for the enhanced significance of this criterion are efforts in the he system in croatia to establish an academic staff promotion system for implementing e-learning, and for setting and implementing evaluation and quality control at universities and faculties in croatia. the strategic readiness for e-learning implementation criterion was also recognized as very important and was relatively significant – 0,253. the lowest relative significance – 0,092 was for the criterion specific ict infrastructure for e-learning. the alternative blended learning has the highest priority of 0,429, which means the recommendation is to apply a blended-learning (hybrid) model, i.e. to integrate e-learning ijahp article: begičević, divjak & hunjak: comparing ahp and anp in a case study of e-learning implementation international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 39 and traditional classroom lectures as the best way to implement e-learning in mathematics in the faculty of organization and informatics. it is interesting that alternative fully online learning has a higher priority (0,140) than the alternative face to face learning (0,108). figure 2 results of the group decision-making exercise. 7. an anp based model for decision making at the institutional level we developed an anp based model which incorporated feedback for strategic planning and decision making on e-learning implementation at the institutional level. the model is based on the reduced and restructured theoretical model by means of factor analysis, and the connections, interdependences and outer dependences have been reviewed by experts in this domain. ijahp article: begičević, divjak & hunjak: comparing ahp and anp in a case study of e-learning implementation international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 40 we built the anp model in super decisions software and the domain expert evaluated the model. in figure 3 are shown the clusters and nodes of a model for decision making on e-learning implementation at the institutional level. the results of the validation are presented in figure 4. we developed an anp model to use for structuring discussions on strategic decisions on e-learning implementation and the decision making process, and for designing the strategy to implement e-learning at the faculty and/or university level. in our study of “strategic planning and decision making on e-learning implementation on an institutional level,” the domain expert evaluated the model using the superdecisions anp software in the numerical judgment mode. the domain expert is a member of the committee for developing e-learning strategy at the university of zagreb and a member of the committee for e-learning strategy of the faculty of organization and informatics. the results of the decision making process, based on the developed anp model, were used for defining a structure of these strategies for e-learning at the institutional level. there has been a crucial difference in the way we used the ahp and anp models. the ahp model was used for carrying out the recommendation for applying the most suitable option for implementing e-learning at the course or department level, but the anp model has been primarily exploited for defining the structure of strategies for e-learning on the institutional level. these strategies, based upon the anp model we developed, will be analysed in the implementation phase. figure 4 presents the clusters and elements with their priorities, which have been crucial for structuring e-learning strategies, and it can be said that these priorities have been a guideline to create an acceptable e-learning strategy. the evaluation and quality control at universities and faculties are recognized as being very influential elements in planning e-learning implementation. furthermore, the variables exam management system, library management system and systems for simulation and virtual environment were identified as the most important in the framework of a specific ict infrastructure for elearning, and managed learning environment and teachers and students equipped with computers within the framework of a basic ict infrastructure for e-learning. the crucial points which must be taken into account in the process of strategic planning of e-learning implementation are the establishment of the specialized e-learning centres and continuous training of academic staff. the alternative blended learning has the highest priority and the alternative ict supported face-to-face learning follows. finally, there is the alternative fully online learning, and this ranking can be understood if we take into account the fact that the university of zagreb is an old and traditional university which appreciates the face-to-face approach and pedagogies, and considers e-learning as merely one way to improve the quality of teaching and learning. ijahp article: begičević, divjak & hunjak: comparing ahp and anp in a case study of e-learning implementation international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 41 figure 3 overview of anp network “strategic planning and decision making on elearning implementation on an institutional level.” ijahp article: begičević, divjak & hunjak: comparing ahp and anp in a case study of e-learning implementation international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 42 alternatives total normal ideal ranking 1 ict supported face to face learning 0.0314 0.2554 0.4446 2 2 blended learning 0.0706 0.5745 1.0000 1 3 fully online learning 0.0209 0.1701 0.2960 3 figure 4 priorities of the elements and the alternatives. ijahp article: begičević, divjak & hunjak: comparing ahp and anp in a case study of e-learning implementation international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 43 8. conclusion the results of the survey performed by the group of experts on e-learning in he were used as input for two mathematical models. this modeling contributed significantly to institutional planning, management and quality development for online distance education and e-learning. the problem of prioritizing e-learning options was solved with the help of multi-criteria decision making modelling. the ahp model was developed and validated during a group decision-making process. the anp model was developed by the team and then restructured by the domain expert before giving judgments. the results of the decision-making process, based on the developed anp model, have been used for defining and prioritizing the factors and strategies for e-learning on the institutional level. our experience shows that such models for decision making strongly motivate all participants in the process, speed up decision-making, make the process more effective, and indicate a need for systematic e-learning usage in our educational institutions. more details about the ahp model can be found on http://www.projekti.hr/, and the model can be used and tested in new situations just by acknowledging the authors. references bates, t. (2005). strategy and visions of e-learning in higher education. zagreb, croatia, url: http://eqibelt.srce.hr/lectures/bates.html, 10(3).2006. begicevic, n., divjak, b. (2006). validation of theoretical model for decision making about e-learning implementation. journal of information and organizational sciences, faculty of organization and informatics. university of zagreb, 30 (2), 171-185. begicevic, n., divjak, b., hunjak, t. (2006). development of ahp based model for decision making on e-learning implementation. 17th international conference on information and intelligent systems iis 2006, varaždin, faculty of organization and informatics, university of zagreb, 375-383. brace, n., kemp, r., snelgar, r. (2000). spss for psychologists: a guide to data analysis using spss for windows. new york, n.y: palgrave. divjak, b., begicevic, n. (2006). imaginative acquisition of knowledge —strategic planning of e-learning. iti 2006. proceedings of the 28th international conference on information technology interfaces. cavtat, croatia, 47-53. ijahp article: begičević, divjak & hunjak: comparing ahp and anp in a case study of e-learning implementation international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 44 eqibelt, education quality improvement by e-learning technology,. 16(01)2006, url:. http://eqibelt.srce.hr/ rummel, r. j. (1967). understanding factor analysis. the journal of conflict resolution, 20(04)2006 url: www.hawaii.edu/powerkills/ufa.htm ijahp article: vachnadze/prioritization of performance measures using analytic hierarchy process international journal of the analytic hierarchy process 490 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.442 prioritization of performance measures using analytic hierarchy process revaz vachnadze full professor, esm business school, free university of tbilisi; visiting professor, georgian institute of public affairs; tbilisi, georgia r.vachnadze@freeuni.edu.ge, revaz.vachnadze@gipa.ge abstract the purpose of this paper is to provide an integrated approach that prioritizes organizational performance measures and critical success factors towards the strategic objectives and initiatives of a firm. the analytic hierarchy process (ahp) pair-wise comparisons and hierarchic composition technique is used to prioritize the key performance indicators (kpis) and the key result indicators (kris) as well as the critical success factors (csfs) of the organization within the frame of a single hierarchy. the new model presented in the paper will be more suitable for dealing with the problem than the others which are available. the application of the suggested model will enable staff to more closely align their daily activities to the strategic objectives of the firm. the suggested approach allows for a wide applicability to different types of organizations (business, nonprofit, public) and its use could significantly improve resource allocation and the overall performance in organizations. keywords: performance measures; analytic hierarchy process; key performance indicators; key result indicators; critical success factors 1. introduction organizational performance measurements and measures have attracted a great deal of attention among scholars and practitioners in different spheres of human activities. of particular note is that nonfinancial factors have received more importance in recent years in measuring the overall performance of any firm. therefore, the inclusion of nonfinancial factors such as higher customer satisfaction (delight), effective management and leadership, using more advanced technology in operations, etc., makes valuable contributions to the measurement of the overall performance of organizations rather than limiting the measurement to financials alone. in spite of the wide attention given to performance measures, there is a certain misunderstanding and mixing up of the measures. therefore, to avoid these drawbacks this paper uses the approach based on parmenter (2015) where performance measures are clearly classified, thereby considerably improving their applicability. in the presence of a scarcity of resources, which is a common phenomenon, it is necessary to obtain more relevant measures and pay attention to those having a higher priority. so, along with the development and establishment of relevant characteristics, mailto:r.vachnadze@freeuni.edu.ge mailto:revaz.vachnadze@gipa.ge ijahp article: vachnadze/prioritization of performance measures using analytic hierarchy process international journal of the analytic hierarchy process 491 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.442 such as key performance indicators (kpis), key result indicators (kris), and critical success factors (csfs), the prioritization of these characteristics should be strongly emphasized. performance measures and their prioritization have been the subject of interest of many studies (e.g.,chen, 1999; ho & zhu, 2004; salmeron & herrero, 2005; shahin & mahbod, 2007; survadi, 2007; teker et al., 2011; etc.). inspired by kaplan and norton’s balanced scorecard (kaplan & norton, 1996), these studies paid attention not only to financial but to nonfinancial perspectives (customer, internal processes, and learning and growth) as well. the prioritization of kpis, kris and csfs should be viewed as a multi-criteria decision making problem which may be solved using thomas saaty’s analytic hierarchy process (ahp) which is referred to as one of the most powerful and widely used techniques for decision making. the applicability of the ahp here is especially worthy due to the presence of tangible as well as intangible measures. in this paper, the elaboration of the model and a corresponding method are suggested which will enable staff to more closely align their daily activities to the strategic objectives and initiatives of the firm. this will be achieved by properly selected and prioritized characteristics. the suggested approach is preceded by an example of a football team’s preparation for a championship. 2. critical success factors and performance measures according to performance improvement thought leader dean spitzer, measurement is fundamental to high performance, improvement, and, ultimately, success in business or in any other area of human endeavor and it is potentially one of the highest leverage activities any organization can perform (spitzer, 2007).today the most popular measures of organizational performance for many organizations worldwide are deployed using the term: key performance indicator (kpi). unfortunately, in many practical cases performance measurement is failing. this is due to the fact that these organizations are working with the wrong measures, many of which are incorrectly termed key performance indicators (kpis). the measures that have usually been adopted have no link to the critical success factors (csfs) of the organizations (parmenter, 2015). critical success factors can be defined as the list of issues or aspects of organizational performance that on the whole determine ongoing health, vitality and well-being. we should emphasize that the main purpose of performance measures is to ensure that staff members spend their working hours focused primarily on the organization’s csfs. it is the csfs, and performance measures within them, that link daily activities to the organization’s strategies/goals. being aware of the significance of a well thought through and executed strategy is the responsibility of a selected group of senior executives in the organization whereas the critical success factors should be the daily focus of all of the staff in the organization as this will positively impact the strategic initiatives (parmenter, 2015). one of the most important roles of management is to communicate expectations to the workforce. in fact, people will do what management inspects (measures) and not necessarily what management expects (spitzer, 2007). thus, the right measures need to be put in place. kpis are the main things that truly link day-to-day performance in the workplace to the organization’s csfs. ijahp article: vachnadze/prioritization of performance measures using analytic hierarchy process international journal of the analytic hierarchy process 492 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.442 in representing performance measures as a set we can pick out two subsets from within: result indicators (ris) and performance indicators (pis). parmenter (2015) calls these subsets groups. when we use the term result indicators, we emphasize the fact that these measures are a summation or a result of more than one team’s efforts. unfortunately, these indicators usually do not help management fix a problem as it is difficult to bring out which teams were responsible for high or low performance. as an example, financial indicators are a result of different activities and so financial performance measures are result indicators. financial indicators are useful, but they do not reveal the real drivers of performance. to fully understand what to change, we need to look at the activities that created the financial indicator. performance indicators, on the other hand, are measures that can be linked to a team or a cluster of teams working closely together for a common purpose. in this case, high or low performance is now the responsibility of one team. obviously, some measures are more important and so picking out corresponding subsets from ris and pis and using the extra word “key” we come to: key result indicators (kris) measures which give an overall summary of how the organization is performing. kris display a clear picture of whether or not your organization is moving in the right direction and at the right speed. they provide the board of a governing body with a good overview as to progress with regard to the organization’s strategy. separating kris from other measures has an important impact on reporting. this results in a separation of performance measures into those impacting governance (kris) and those impacting management (kpis, pis, ris). key performance indicators (kpis) – measures which show management how the organization is performing on their critical success factors (csfs) and by monitoring them management is able to increase performance dramatically. kpis focus on the aspects of organizational performance that are the most critical for the current and future success of the organization (parmenter, 2015). 3. analytic hierarchy process (ahp) in the 1970s, thomas l. saaty developed an elegant approach to help decision makers in modeling complex problems in a simple way. his textbook, entitled “the analytic hierarchy process” and the pc-based software, entitled “expert choice” helped popularize the process amongst operations research practitioners (saaty, 1980; expert choice, 2002). in the 40 years since the publication of the first papers, books and software, ahp has been used by decision makers all over the world to model problems in diverse areas including resource allocation, strategic planning, public policy, etc. it has been used to rank, select, evaluate and benchmark a wide variety of decision alternatives (saaty, 2008). ahp has been used by organizations in both the public and private sectors to deal with complex problems, and it has had a significant impact on the practice of decision making. operations research practitioners around the world have repeatedly embraced ahp as a methodology that can produce insightful results for difficult, real-world decision problems. for more than two decades, ahp has been taught as an important part of the curriculum covering decision making techniques at business and engineering schools worldwide. it has been incorporated into mainstream operations research collegelevel textbooks (e.g.,anderson et al., 2012; liberatore and nydick, 2003) and commercial software packages (e.g.,criterion decision plus available at www.infoharvest.com). a wide range of ahp applications and software packages have been catalogued, categorized, and annotated in edited volumes and books http://www.infoharvest.com/ ijahp article: vachnadze/prioritization of performance measures using analytic hierarchy process international journal of the analytic hierarchy process 493 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.442 (e.g.,golden et al., 1989; saaty and vargas, 2000), and in journal articles (e.g., forman and gass, 2001; vaidya and kumar, 2006; ishizaka and labib, 2009, 2011; mcginley, 2012). ahp is based on the following three principles: decomposition, comparative judgment, and a synthesis of priorities. it is a theory of measurement for dealing with quantifiable and intangible criteria that has been applied to numerous areas. it generally involves three steps. the first step is to structure the problem into a hierarchical framework with successive levels of goal, criteria and alternatives. the alternatives are placed at the bottom level. such structuring requires some experience with ahp techniques, but the following guidelines are helpful: 1) start structuring top down – specify an overall goal first and then criteria and the alternatives that have an impact on the goal or which will help to achieve that goal. 2) comparison analysis – once the hierarchy has been structured, the second step is to establish ratio priorities for each node of the hierarchy. this is done through pairwise comparisons of the child items below a parent node. the comparisons are done with respect to the importance or contribution of the item to the parent node. hence, this comparison analysis is generally conducted from bottom to top. once sufficient comparisons have been made for a node, the principal eigenvector of the comparison matrix is standardized so that it sums to one and becomes the ratio measure of the relative importance of each item. since these priorities reflect the relative importance of only the items below a parent node, they are called local weights. 3) aggregate the local weights into a composite priority – this is the ahp’s final step and is done through the principle of hierarchic composition that first multiplies local weights by the product of all higher-level priorities. within the hierarchy, this process transforms the local weights into global weights that measure the importance of each node in the total hierarchy (figure 1). figure1. ahp’s three-level hierarchy (level 1 goal, level 2 criteria, level 3 alternatives). here, n and m describe number of criteria and alternatives accordingly. 4. suggested approach let us examine a hypothetical example of the hierarchy with the focus/goal being a football team’s preparation for a championship in order to achieve a better place than in the previous year (figure 2). . . . goal criterion 1 criterion 2 criterion n alternative 1 alternative 2 alternative 3 alternative m . . . ijahp article: vachnadze/prioritization of performance measures using analytic hierarchy process international journal of the analytic hierarchy process 494 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.442 figure 2. four-level hierarchy of the football team’s preparation for a championship critical success factors (criteria) for this focus/goal could be physical (s1), technical (s2), tactical (s3) and psychological (s4) preparation/training. using the eigenvector method of ahp we obtain csf’s local weights towards the goal. it is important to note that the pair-wise comparison matrix should be filled by the manager (head coach) of the team. these judgments reflect his/her professional and personal conception of football and it is not surprising that calculated weights could differ for different coaches. suppose that our manager has the following priorities: s1=0.2, s2=0.2, s3=0.3, s4=0.3. the next level down the hierarchy consists of the two elements which are generalized key result indicator (gkri) and generalized key performance indicator (gkpi).the gkri shows how good or bad the condition of the team is as a result of its pre-season preparation, whereas the gkpi shows the consequences of the preparation later in the course of the championship. the appearance of the gkri and the gkpi here is caused by this specific example. their presence in some cases may cause certain difficulties during the judgment process and so they may not be necessary in these cases. naturally, comparative judgments on the gkri and the gkpi in each specific case can be different. for each csf the manager together with his/her assistants form pair-wise comparison matrices and calculate weights prij of the j-th indicator(j=1,2) towards i-th factor (i=1,2,3,4). suppose that prij–s are as follows: pr11= 0.6, pr12=0.4, pr21=0.4, pr22=0.6, pr31=0.3, pr32=0.7, pr41=0.5, pr42=0.5. taking into account weights of the csfs, for the gkpi with respect to the focus we obtain weight p=∑sj*pr1j=0.44 and for the gkri we obtain r=∑sj*pr2j=0.56. focus/goal: preparation for a championship csf1 physical training gkpi generalized key performance indicator kpi1– fitness kpi2– trainings with the ball, training games kpi3 – theoretical lessons kpi4– creation of the mutual respect and trust in the team. kri1 – mobility, endurance kri2 “intimacy” with the football, sight of the playing field, improvisation kri3 – discipline, carrying out tasks given by the coach kri4 – purposefulness gkri generalized key result indicator csf2 technical training csf3 tactical training csf4 psychological training ijahp article: vachnadze/prioritization of performance measures using analytic hierarchy process international journal of the analytic hierarchy process 495 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.442 at the bottom level there are different kpis and kris, which separately affect the gkpi and the gkri. suppose that they are as follows: for kpis: kpi1(u1) – fitness; kpi2 (u2) – trainings with the ball, training games; kpi3 (u3) – theoretical lessons; kpi4 (u4) –creation of the mutual respect and trust in the team. for kris: kri1(v1) – mobility, endurance; kri2 (v2) –“intimacy” with the ball, sight of the playing field, improvisation; kri3 (v3) – discipline, carrying out tasks given by the coach; kri4 (v4) – purposefulness. both pair-wise comparison matrices are to be filled by the concerted judgments of the manager, his/her assistants, trainer responsible for the team’s physical preparation, and the consultant psychologist. here, the requirement for consensus should be obligatory; you cannot require coordinated actions from the players when the trainers themselves cannot come to unanimity. suppose that we have the following weights for the kpis and the kris: u1=0.15, u2=0.35, u3=0.3, u4=0.2and v1=0.2, v2=0.25, v3=0.3, v4=0.25. finally, we calculate the global weights (composite priorities) wi of the key indicators with respect to the focus. w1=p*u1=0.44*0.15=0.066, w2=p*u2=0.44*0.35=0.154, w3=p*u3=0.44*0.3=0.132, w4=p*u4=0.44*0.2=0.088; w5=r*v1=0.56*0.2=0.112, w6=r*v2=0.56*0.25=0.14, w7=r*v3=0.56*0.3=0.168, w8=r*v4=0.56*0.25=0.14. note that ∑wi=1, and the average is 1/8=0.125 calculated weights allow the manager to realize how the strategy for the team’s preparation plan for the championship corresponds to his/her conception (which is reflected in his/her weights of the csfs) and to check its effectiveness. it should be emphasized that the weight vector w may present the necessary condition for reaching success, but it is far from being sufficient. to fully execute this task, it is necessary to have a more comprehensive model, which would consider such aspects as the financial resources of the team/club, the roster, individual fitness, operative control, medical control, etc. these aspects are beyond the scope of this paper and should be the object of special research. all the numbers obtained above are conditional, as they might be allowable within the framework of an illustrative example. but even here something must be noted; if we settle on calling the sums of the similar elements of the vector w “actual weights”, then we obtain: actual weight of the physical training (process+result): w1+w5=0.066+0.112=0.178; actual weight of the training in technical skills (process+result): w2+w6=0.154+0.14= 0.294; actual weight of the training in tactics (process+result): w3+w7=0.132+0.168=0.3; actual weight of the psychological training (process+result): w4+w8=0.088+0.14=0.228 ijahp article: vachnadze/prioritization of performance measures using analytic hierarchy process international journal of the analytic hierarchy process 496 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.442 naturally, the numbers obtained differ from the csf’s weights, which can be viewed as global towards the whole process of the team’s management, whereas “actual weights” relate only to the stage of the team’s preparation for the season and its actual performance. note that if the results obtained by the team do not meet the manager’s expectations, then he/she should make appropriate corrections in the process of the team’s preparation for the next season. the hierarchy of our example is shown with the obtained local weights in figure 3. figure 3. four-level hierarchy of the football team’s preparation for a championship, with obtained local weights let us turn to the general case of the prioritization of the performance measures. similar to shahin and mahbod (2007), the approach in this paper is based on using saaty’s ahp. shahin and mahbod use the three-level hierarchy with the smart conception’s components (specific, measurable, achievable, relevant and timebased) as criteria and kpis as alternatives. taking into consideration the above mentioned importance of csfs in establishing key performance measures, this paper suggested the addition of a csfs level to form a four-level hierarchy of the following type (see figure 4). focus/goal: preparation for a championship csf1 physical training 0.2 gkpi generalized key performance indicator 0.44 kpi1– fitness 0.15 kpi2– trainings with the ball, training games 0.35 kpi3 – theoretical lessons 0.3 kpi4– creation of the mutual respect and trust in the team. 0.2 kri1 – mobility, endurance 0.2 kri2 “intimacy” with the football, sight of the playing field, improvisation 0.25 kri3 – discipline, carrying out tasks given by the coach 0.3 kri4 – purposefulness 0.25 gkri generalized key result indicator 0.56 csf2 technical training 0.2 csf3 tactical training 0.3 csf4 psychological training 0.3 ijahp article: vachnadze/prioritization of performance measures using analytic hierarchy process international journal of the analytic hierarchy process 497 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.442 figure 4. suggested four-level hierarchy(level 1 goals/strategies, level 2 smart criteria, level 3 – csfs, level 4 kpis and kris) in the suggested model each of the kpi and kri subsets is to be prioritized separately so, essentially we will have a hierarchy with two different sets of alternatives. the csfs, kpis and kris can be found by using the procedures suggested in the references (barr, 2014; parmenter, 2015). best practice recommends limiting the number of csfs, kpis and kris to 10 each (parmenter, 2015). in figure 4, the numbers of csfs, kpis and kris are taken willfully. for any type of organization, each kpi or kri should not necessarily affect every csf and so not every child item (alternative) of the hierarchy should be connected to every parent node in this case. the hierarchy of figure 4 is not complete which is fine because ahp does not require a complete hierarchy. the presence of several connections of each kpi and kri with different csfs reflects the fact that these measures should have a significant impact on the organization (e.g., kpi or kri impacts on more than one of the top csfs). due to the positive impact on performance, kpis encourage appropriate action, whereas poorly thought through measures can lead to dysfunctional behavior. the model of figure 4 can be substantially simplified. in practice, well formulated organizational goals/strategies are usually a priori stated in a smart mode; therefore, the hierarchy of figure 4 can be reduced by the elimination of the level 2 smart criteria, thus transforming to the following three-level hierarchy (figure 5). goals/strategies specific measurable achievable relevant time-based csf 1 csf 2 csf 3 csf 4 kri-1 kri-2 kri-3 kpi-1 kpi-2 kpi-3 kpi-4 ijahp article: vachnadze/prioritization of performance measures using analytic hierarchy process international journal of the analytic hierarchy process 498 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.442 figure 5.suggested three-level hierarchy. in this case, the willfully taken number of csfs is 4 whereas numbers of kris and kpis are 3 and 4 accordingly. the suggested approach is rather easily operationally applicable for different types of problems, as one could notice in the example shown at the beginning of this section. this approach is now applied to the prioritization of performance measures of a higher education institution in georgia. 5. discussions and conclusions in this paper a novel integrated approach is proposed using thomas saaty’s analytic hierarchy process (ahp) to prioritize key performance measures of organizations; namely, key performance indicators (kpis) and key result indicators (kris), as well as critical success factors (csfs) towards the strategic objectives and initiatives of the firm. the prioritization of kpis, kris and csfs allows the organization’s governing bodies and management to focus scarce resources on the top priorities/issues. the separation of kris and kpis might reflect the degree of interest of the governing body (board) and the management in them so that a governance report should consist of a list of prioritized kris whereas kpis are to be of prime interest for ceos, management and staff. the advantages of the proposed approach are listed below. • the proposed approach is practically applicable to any type of organization of any size in any particular area. • the evaluation of kpis and kris by the integration of ahp and csfs goal setting can take both quantitative and qualitative factors into consideration. • kpis and kris can be arbitrary and subjective. they have significant impact on more than one of the top csfs. • the proposed approach makes it possible to involve all informed persons in establishing priorities for the indicators and reaching a dynamic group decision to obtain the final weights. csf 1 csf 2 csf 3 csf 4 kri-1 kri-2 kri-3 kpi-1 kpi-2 kpi-3 kpi-4 goals/strategies ijahp article: vachnadze/prioritization of performance measures using analytic hierarchy process international journal of the analytic hierarchy process 499 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.442 • the ahp approach is well structured and holistic. the pair-wise comparisons make multi criteria decision making possible and ensure the analyzer that comprehensive decision making has been undertaken based on prioritized csfs. the decision making process is rational and consistent and aids objectivity and understanding. • flexibility of the proposed approach does not require a repeat of all of the judgments when a change in the model, such as the addition or removal of the kpi, kri, or csf, is made. at the same time, there are some limitations and subjective factors by which the results obtained using this approach might be influenced. they are as follows: • a variation in the views of the people participating in the judgment process of the prioritization procedure (e.g., consistency ratios) might lead to differing results. • the accuracy of the suggested approach is limited by estimates obtained in the processes of judgments in the ahp and the actual finding of the right performance measures. • it seems that the categorization of the performance measures most likely has to be carried out within the framework of zadeh’s fuzzy sets (dubois &prade, 1988). • the proposed approach helps to determine which dimensions require improvement, but it does not provide guidance on the actions to be taken. these last two issues might be objects for further studies. ijahp article: vachnadze/prioritization of performance measures using analytic hierarchy process international journal of the analytic hierarchy process 500 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.442 references anderson, d., sweeney, d., williams, t., camm, j., and martin, k. (2012). an introduction to management science: quantitative approaches to decision making, revised 13 th edition. cincinnati, oh: south-western cengage learning. barr, s. (2014).practical performance measurement: using the pump blueprint for fast, easy and engaging kpis. the pump press. chen, t. (1999). critical success factors for various strategies in the banking industry. international journal of bank marketing, 17(2), 83-91. doi: http://dx.doi.org/10.1108/02652329910258943 chiu, c.r., et al. (2014). the performance of commercial banks based on a contextdependent range-adjusted measure model. international transactions in operational research, 21(5), 761-778. doi: 10.1111/itor.12069 dubois, d. and prade, h. (1988). fuzzy sets and systems. new york: academic press. expert choice (2002). expert choice 2000.pittsburgh, pa: expert choice inc. forman, e. and gass, s. (2001). the analytic hierarchy process: an exposition. operations research, 49, 469-486. doi: http://dx.doi.org/10.1287/opre.49.4.469.11231 golden, b., wasil, e. and harker, p. (1989).the analytic hierarchy process: applications and studies. berlin: springer. ho, c, and zhu, d. (2004). performance measurement of taiwan’s commercial banks. international journal of productivity and performance management, 53(5), 425-434. doi: http://dx.doi.org/10.1108/17410400410545897 ishizaka, a, and labib, a. (2009). analytic hierarchy process and expert choice: benefits and limitations. or insight, 22, 201-220.doi: 10.1057/ori.2009.10 ishizaka, a, and labib, a. (2011). review of the main developments in the analytic hierarchy process. expert systems with applications, 38, 14336-14345. doi: http://dx.doi.org/10.1016/j.eswa.2011.04.143 kaplan, r.s. and norton d.p. (1996).the balanced scorecard: translating strategy into action. cambridge, ma: harvard business press. doi:10.1016/s0737-6782(97)89410-6 liberatore, m. and nydick, r. (2003). decision technology: modeling, software, and applications. new york, ny: wiley. mcginley, p. (2012).decision analysis software survey. or/ms today, 39. parmenter, d., (2015). key performance indicators: developing, implementing, and using winning kpis, third edition. wiley. doi: 10.1002/9781119019855 saaty, t.l. (1980).the analytic hierarchy process. mcgraw-hill, new york, ny. doi: http://dx.doi.org/10.1080/00137918308956077 http://dx.doi.org/10.1287/opre.49.4.469.11231 http://dx.doi.org/10.1108/17410400410545897 http://dx.doi.org/10.1016/j.eswa.2011.04.143 http://dx.doi.org/10.1016/s0737-6782(97)89410-6 http://dx.doi.org/10.1080/00137918308956077 ijahp article: vachnadze/prioritization of performance measures using analytic hierarchy process international journal of the analytic hierarchy process 501 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.442 saaty, t.l. (2008), decision-making with the analytic hierarchy process. international journal of services science, 1(1), 83-98. doi: 10.1504/ijssci.2008.017590 saaty, t.l. and vargas, l. (2000).models, methods, concepts and applications of the analytic hierarchy process. boston, ma: kluwer. salmeron,j.l. and herrero,i. (2005). an ahp-based methodology to rank critical success factors of executive information systems. journal of computer standards & interfaces, 28(1), 1-12. doi: http://dx.doi.org/10.1016/j.csi.2004.09.002 shahin,a. and mahbod,m. (2007). prioritization of key performance indicators: an integration of analytical hierarchy process and goal setting. international journal of productivity and performance management, 56(3), 226-240. doi: http://dx.doi.org/10.1108/17410400710731437 spitzer, d. (2007).transforming performance measurement: rethinking the way we measure and drive organizational success. new york: amacom. suryadi, k. (2007). key performance indicators measurement model based on ahp and trend-comparative dimension in higher education institution, isahp 2007, vina del mar, chile. teker s., teker d. and kent o. (2011).measuring commercial banks performances in turkey: a proposed model. journal of applied finance & banking, 1(3), 97-112. vaidya, o. and kumar, s. (2006). analytic hierarchy process: an overview of applications. european journal of operational research, 169, 1-29. doi: http://dx.doi.org/10.1016/j.ejor.2004.04.028 http://dx.doi.org/10.1504/ijssci.2008.017590 http://dx.doi.org/10.1016/j.csi.2004.09.002 http://dx.doi.org/10.1016/j.ejor.2004.04.028 ijahp article: madera/interval uncertainty of estimates and judgments of subject in decision making in multi-criteria problems international journal of the analytic hierarchy process 337 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.240 interval uncertainty of estimates and judgments of subject in decision making in multi-criteria problems alexander madera higher school of economics national research university moscow, russian federation e-mail: agmprof@gmail.com abstract in this article, we propose a method of decision making in multi-criteria problems given an interval uncertainty of the estimates given by the subject in reference to the importance of one criterion over another and various alternatives for each criterion. the method is the development of the deterministic process of the analytic hierarchy process, which uses deterministic point estimates of the importance of criteria and alternatives for each criterion for decision making in multi-criteria problems. while in the standard analytic hierarchy process the values of global priorities corresponding to different alternatives are deterministic and unambiguous, in the interval process developed in this article the global priorities and alternatives are interval and uncertain. if in the standard deterministic analytic hierarchy process the best alternative is selected by the maximum value of the global priority, then, to select the best interval alternative, here we introduce a criterion corresponding to the maximum values of the lower and upper boundaries of the intervals of global priorities of the alternatives. the application of the proposed method is demonstrated by a specific example. keywords: interval; uncertainty; estimates; decision making; analytic hierarchy process 1. introduction a human being cannot give accurate estimates of the absolute values of any variables or values of relative superiority (the importance, significance, intensity) of one value over another. therefore, personal subjective estimates are usually of an uncertain interval nature, i.e., are within certain intervals. in other words, instead of an unambiguous and deterministic estimate, the subject can only indicate the boundaries of the intervals within which he believes there will be the true values of the estimated values, and the width of the estimated interval can be quite significant. by indicating the interval within which the expected value of the estimate may be, the subject implicitly assumes that it is equally likely that the true value of the estimate may be anywhere within the interval. this means that the value estimated by the subject essentially is a random variable uniformly distributed within the interval. we will refer to such an uncertainty interval as stochastic and the estimate itself interval stochastic. because the subject's judgments are usually based on his subjective interval stochastic estimates, the subject's conclusions and decisions in connection with one-criterion and ijahp article: madera/interval uncertainty of estimates and judgments of subject in decision making in multi-criteria problems international journal of the analytic hierarchy process 338 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.240 multi-criteria problems will also be uncertain interval stochastic. indeed, in making decisions in multi-criteria problems, the subject makes interval stochastic estimates in reference to various criteria and the values of relative importance (significance) of intercomparable criteria and alternatives. therefore, the final or global criteria specific for various alternatives will not be unambiguous and deterministic either, but uncertain interval stochastic. the specified circumstance makes it significantly difficult to choose the best alternative, as the global criteria intervals may overlap and include one another, forming common areas within which various alternatives become equivalent. therefore, to provide the subject with the opportunity to make the final selection of a compromise alternative in a multi-criteria problem given the uncertainty of subjective estimates, the relevant criteria as discussed below need to be introduced. to make multi-criteria decisions given the uncertainty, three approaches are provisionally applied in the existing literature. in the first approach the uncertainty is simply ignored and it is assumed that all assessments of the criteria, alternatives and the intercomparisons thereof, which are subsequently used to make decisions, are unambiguous and precisely definable, i.e., deterministic. one of the deterministic methods of decision making is the analytic hierarchy process (ahp) (saaty, 2001). under the second approach, the interval uncertainty is specifically admissible; however, it is being reduced to complete certainty, i.e., again back to the deterministic case. for this purpose, various mathematical techniques aimed at the reduction of the uncertainty intervals to certain unambiguous point estimates are applied so that to subsequently re-apply the decision making methods applied under the first deterministic approach (podinovski, 2007; salo, a.a., and hämäläinen r.p., 1995; wang y.-m. et al., 2005). in the third approach, the interval uncertainty is considered by fuzzy theory or stochastic formulation for the ahp (haines, l.m., 1998; deng, h., 1999; lipovetsky, s., and tishler, a., 1999; mikhailov, l., 2004; eskandari, h., and rabelo, l., 2007). the article proposes the method for solving multi-criteria problems given the uncertainty of the estimates given by the subject, which are interval stochastic. the analytic hierarchy process, which is initially deterministic and uses unambiguous estimates of relative importance of both the criteria and alternatives for the criteria, is the basis of multi-criteria optimization(saaty, 2001). to support decision making in multi-criteria problems given the interval uncertainty of the estimates, the article introduces a criterion which corresponds to the maximum values of the upper and lower boundaries of the intervals of global priorities. the application of the interval stochastic analytic hierarchy process developed in this article is considered using a specific example. 2. interval stochastic analytic hierarchy process the normal analytical hierarchy process ahp is applied for decision-making in multicriteria problems, and it is deterministic (saaty, 2001). in other words, it is assumed that all estimates of relative importance of the criteria and alternatives for the criteria are deterministic and unambiguously defined. the analytic hierarchy process is based on the construction of deterministic pairwise comparison matrices for the criteria and the alternatives with respect to each criterion, followed by determination of the eigenvectors corresponding to the maximum eigenvalue of these matrices. the elements of the eigenvectors pairwise comparison matrices are relative importance coefficients of the criteria and alternatives assessed in terms of each criterion. the analytic hierarchy process (ahp) is finalized by the calculation of global priorities for each alternative and ijahp article: madera/interval uncertainty of estimates and judgments of subject in decision making in multi-criteria problems international journal of the analytic hierarchy process 339 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.240 selection of the best of them corresponding to the maximum value of a global priority. the анр assumes that the subject is capable of carrying out pairwise comparisons with sufficient accuracy of any two factors (both quantitative and qualitative), and furthermore also of indicating the precise value of superiority of one factor over another using the fundamental scale (saaty, 2001). meanwhile, the subject in reality is not capable of providing accurate estimates. he can only conclude that in his opinion the degree of superiority of one factor over another is, for example, somewhere between the weak degree of superiority (2 points based on the fundamental scale) and slightly above the average degree (4 points) of superiority. therefore, subjective estimates of the degree of superiority of one factor over another, as well as the values of any variables/factors at all are fundamentally blurred and uncertain. instead of the exact value of some variable, the subject can only specify a subjective interval of its possible variation, implicitly implying that within this interval the estimated value can equally likely take any value. it should be noted that the subject basically has no reason to believe that his subjective interval stochastic estimate has some other probability density (e.g., triangular, normal, log-normal, etc.) different from the uniform probability density, and the judgments of the subject trying to assess a priori the type of probability density of estimates tend to be wrong (kahneman, d. et al., 2001). thus, subjective estimates are interval stochastic. therefore, the pairwise comparison matrices, their eigenvalues and eigenvectors, the coefficients of relative importance of criteria and alternatives, as well as the global priorities of alternatives in the ahp are also interval stochastic. because of this, the final decision will be determined by not strictly defined deterministic estimates of global priorities, as is the case in the deterministic анр, but by their interval stochastic estimates. the mathematical model {x, f, x  , a (ω)} of multi-criteria decision making given the interval stochastic uncertainty of estimates is considered, where(madera, a.g., 2010): x = (x1, x2, … ,xn) – multitude of possible decisions, f = (f1, f2, … ,fm) – vector criteria, x  – the ratio of the preference given on the set of possible solutions x, a(ω) – subjective interval stochastic estimate of the value, which is firstly interval a(ω)  [ aa , ], where a and a – the lowest and highest boundaries of the interval, and secondly a stochastic variable uniformly distributed in the interval [ aa , ] with the probability density p(a) = 1/δ, a [ aa , ] and p(a) = 0, a  [ aa , ], where δ = a – a – the width of the interval, ω ω – elementary events in the space of elementary events ω (ross, s., 1993). decision making based on the interval stochastic ahp method developed in the article is carried out according to the algorithm: (1) the subject specifies the intervals of estimates for the relative importance of the criteria relative to each other and the alternatives for each criterion. it should be noted ijahp article: madera/interval uncertainty of estimates and judgments of subject in decision making in multi-criteria problems international journal of the analytic hierarchy process 340 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.240 that the interval estimates of the values of relative importance of the criteria and each alternative by the criteria are statistically independent (madera, a.g., 2014); (2) interval stochastic pairwise comparison matrices for the criteria and for each alternative of all criteria are constructed; (3) statistical measures of interval stochastic eigenvectors: mathematical expectations (me), the variance (var) and standard deviations (sd) are determined for each constructed interval stochastic pairwise comparison matrix. in the ahp method, the elements of eigenvectors of pairwise comparison matrices for criteria and alternatives for the criteria are also the coefficients of relative importance of the criteria and alternatives for each criterion; (4) statistical measures of global priorities for each alternative are calculated based on the estimated values of statistical measures of eigenvectors of pairwise comparison matrices, (5) the boundaries of the intervals of global priorities are determined, as me ± ε∙sd, where ε determines the width of the interval global priority for this confidence probability рε; (6) the final decision is made based on the location of the boundaries of the intervals of global priorities of alternatives. let's consider the determination of the statistical measures of interval stochastic variables used for the implementation of steps (1) – (6) of the algorithm. 2.1 determining statistical measures for interval stochastic eigenvector of interval stochastic pairwise comparison matrix the main characteristics defined in the deterministic анр are eigenvectors of pairwise comparison matrices compiled for both the criteria and the alternatives with respect to each criterion. the elements of eigenvectors are priorities, or coefficients of importance, criteria and alternatives in terms of each criterion. in the interval stochastic ahp discussed in this article, the eigenvectors are interval stochastic, and methods to determine their statistical measures are required to define their characteristics. the interval stochastic pairwise comparison matrix for the considered mathematical model is as follows: а(ω) =             1 1 1 )ω()ω( )ω()ω( )ω()ω( 21 221 112     nn n n aa aa aa , (1) where )ω( ij a  [ ijij aa , ] – interval estimate of the relative importance of i factor over j factor with their pairwise comparison carried out by the fundamental scale (saaty, 2001); )ω( ji a = 1/ )ω( ij a (i, j = 1, 2,…, n) for each implementation of ω ω, and the variation interval boundaries )ω( ji a are equal to [1/ ija , 1/ ija ]. ijahp article: madera/interval uncertainty of estimates and judgments of subject in decision making in multi-criteria problems international journal of the analytic hierarchy process 341 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.240 in the deterministic анр, the normalized elements wi of the eigenvector w = (w1, w2, … ,wn) the pairwise comparison matrices а can be calculated with sufficient accuracy by the formula (saaty, 2001): wi = nn j ji n i nn j ji aa 1 11 1 1                   . (2) for the considered interval stochastic ahp, the formulas (2) remain valid for each implementation of ω ω, so the random elements wi(ω) of the random eigenvector w(ω) = (w1(ω), w2(ω), … , wn(ω)) in the pairwise comparison matrix а(ω) (1) are calculated based on the following formula: wi(ω) =                   } 1 /1 11 /1 )ω({)ω( n j n ij n i n j n ij aea , (3) where e{ )ω( /1 n ij a } – mathematical expectation of random variable )ω( /1 n ij a . let's find formulas for determining the statistical measures (me, var and sd) of the elements of the interval stochastic eigenvector w(ω). considering that in each i row of the matrix а(ω) (1) random values )ω( 1i a , )ω( 2i a , …, )ω( in a are statistically independent we obtain according to (3) that values mewi = e{wi(ω)}, varwi = e{(wi(ω) – e{wi(ω)}) 2} and sdwi = (varwi) 1/2 of the stochastic elements wi(ω), i = 1, 2,…, n, the eigenvector w(ω) may be calculated based on the following formulas: mewi =                   }} 1 /1 11 /1 )ω({)ω({ n j n ij n i n j n ij aeae , (4) varwi = e{ )ω( 2 i w } ‒ 2 me wi , (5) where e{ )ω( 2 i w } = 2 1 /1 11 /2 }} )ω({)ω({                           n j n ij n i n j n ij aeae . (6) in the formulas (4) – (6), there may be the mathematical expectations of various degrees of variables )ω( ij a , namely, )ω( /1 n ij a , )ω( /1 n ij a  , )ω( /2 n ij a , )ω( /2 n ij a  . since the probability densities of interval stochastic variables )ω( ij a are uniform, the corresponding values of the mathematical expectations will be determined only by the lower and upper boundaries of the intervals [ ijij aa , ], namely: ijahp article: madera/interval uncertainty of estimates and judgments of subject in decision making in multi-criteria problems international journal of the analytic hierarchy process 342 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.240 e{ )ω( /1 n ij a  } =  naa ij n ij n ij /11)()( /11/11         , (7) e{ )ω( /2 n ij a  } =  naa ij n ij n ij /21)()( /21/21         . (8) having calculated the values (7) and (8) and plugging them into (4) – (6), we will obtain the formulas for determining the values of mewi, varwi and sdwi of the interval stochastic elements wi(ω), i = 1, 2,…, n of the eigenvector w(ω) of the interval stochastic matrix а(ω). 2.2 determining statistical measures for vectors of criteria priorities and vectors of priorities of alternatives for each criterion the formulas for calculating the values of me, var and sd of the interval stochastic eigenvector w(ω) of the interval stochastic pairwise comparison matrix а(ω) were obtained above (section2.1). in the deterministic анр, a matrix of pairwise comparisons (аf) for the criteria f = (f1, f2, … ,fm) and m of the pairwise comparisons matrices ( f x a ) for the alternatives x = (x1, x2, … , xn) relative to each criterion from the vector f is compiled. then, for each pairwise comparison matrix, their eigenvectors are determined, which are the vectors of priorities of the criteria wf = (wf 1, wf 2, … , wf m) and the vectors of priorities of the alternatives x1, x2, … , xnrelative to each criterion fk (k = 1, 2, … , m), i.e., kf x w = ( kf nx kf x kf x www ,,, 21  ). in the interval stochastic ahp, the pairwise comparison matrices are interval stochastic, and their eigenvectors wf = wf(ω) and kf x w = kf x w (ω) – interval stochastic. therefore, for the complete characterization of random vectors, one needs to have their statistical measures (me, var and sd = (var)1/2) for: – the criteria mew f k = e{wf k(ω)}, varw f k = e{(wf k(ω) – e{wf k(ω)}) 2}; – alternatives for each criterion fk (k = 1, 2, … , m) kf iwxme = e{ kf xiw (ω)}, kf iwxvar = e{( kf xiw (ω) – e{ kf xiw (ω)}) 2}, which are calculated according to the formulas of the type (4) – (8). 2.3 determining statistical measures for the vector of global priorities of alternatives in the deterministic анр, the values of global priorities (gps) of alternatives are used for the final selection of the best decision in a multi-criteria problem. the gps are calculated as the sum of the multiplications of the priorities of this alternative with respect to each criterion by the priorities of relevant criteria. ijahp article: madera/interval uncertainty of estimates and judgments of subject in decision making in multi-criteria problems international journal of the analytic hierarchy process 343 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.240 in the interval stochastic ahp, the priorities of both the criteria and alternatives for the criteria are interval stochastic values. therefore, the gps of alternatives will also be interval stochastic gxi (ω), which for each alternative xi (i = 1, 2,…,n) will be determined by the following formula: gxi (ω) = 1f xi w (ω)∙wf 1(ω) + 2f xi w (ω)∙wf 2(ω) +  + mf xi w (ω)∙ wf m(ω), (9) where 1f xi w (ω), 2f xi w (ω), … , mf xi w (ω) – interval stochastic values of the priorities of the alternative xi relative to the criteria f1, f2, … , fm; wf 1(ω); wf 2(ω), … , wf m(ω) – interval stochastic values of the criteria priorities. all priorities are statistically independent, pairwise and in combination. since the stochastic values of the priorities of the alternatives and criteria in (9) are mutually independent, for the random variable gxi (ω) the values of the statistical measures megi, vargi and sdgi = (vargi) 1/2 will be determined by the following formulas: megi = }{ )ω()}ω({ 1 kf m k kf xi wwe e  , (10) vargi =    m k kf ik kf ik kf i wxkfwfwwxfwwx 1 22 })me(varmevarvarvar{ . (11) based on the calculated values of the statistical measures (10), (11), ε-intervals are constructed, within which the gps will be with the confidence level of рε, namely gxi(ω)  [ xixi gg , ], for the gps of alternatives. the lower xig and upper xig boundaries of the gp-intervals of alternatives are equal to: xi g = megi – ε∙sdgi , xig = megi + ε∙sdgi. (12) the values gxi (ω) for the alternative xi will be within the interval megi – ε∙sdgi ≤ gxi(ω) ≤ megi + ε∙sdgi. the number ε should be selected as equal to 3 when the probability of finding the gp value outside the interval [megi ± ε∙sdgi] will not exceed 1/9. 2.4 making the best decisions in the interval stochastic ahp the intervals estimating the gps of various alternatives may have varied mutual arrangements (fig. 1). in general, this leads to a multitude of selected decisions which may contain interval decisions incomparable with each other, which considerably complicates the final selection of the best alternative. ijahp article: madera/interval uncertainty of estimates and judgments of subject in decision making in multi-criteria problems international journal of the analytic hierarchy process 344 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.240 in the considered interval stochastic ahp, the obtained intervals of any possible gp values are the basis for determining the best compromise alternative, which is selected based on the relative arrangement of the lower and upper boundaries of the gp intervals for various alternatives. the article assumes that it is reasonable to select such an interval decision, for which the lower and upper boundaries of the gp interval have maximum values of all other boundaries of the gp intervals of alternatives. on fig. 1а, such is the gp interval corresponding to the alternative x', because its lower and upper boundaries have maximum values (shifted to the right, fig. 1а). accordingly, an alternative (x'', fig. 1а) for which the gp interval boundaries are minimal (shifted to the left, fig. 1а) will be the worst one. this is also true for those occasions when either the upper (fig. 1b) or the lower (fig. 1c) boundaries match in two inter-compared gp-intervals. thus, if the upper boundaries of two gp intervals match, the decision with the lower boundary value which is higher (x', fig. 1b) is selected, and, if the lower gp boundaries match, the best decision will the one with the highest value of the upper boundary (x'', fig. 1c). we should especially note the case when the lengths of the gp-intervals are equal and their lower and upper boundaries match in two compared decisions. in this case, both decisions are equivalent by all the criteria, and the adoption of one of them is determined solely by the opinion and preferences of the subject making the decision. cases of "nested" (fig. 1d) gp-intervals [ x g  , xg  ] [ xg  , xg  ] when the length of the gp-interval [ x g  , xg  ] of one decision (x') is less than the length of the gp-interval [ x g  , xg  ] of another decision (x'') are possible as well, and there are disparities between the lower and upper boundaries: x g  < xg  < xg  < xg  . with the nested gpintervals and the boundaries which do not match, the selection of the best alternative is difficult, which is due to the fact that two alternatives x' and x'' with the nested gpintervals are incomparable with each other. indeed (fig. 1d), the gp values of both alternatives x' and x'' from the nested gp-interval [ xg  , xg  ] completely match, making the alternatives x' and x'' indistinguishable from each other in the impacts. at the same figure 1. variants of relative arrangement of gp-intervals corresponding to two compared alternatives x' and x'' ijahp article: madera/interval uncertainty of estimates and judgments of subject in decision making in multi-criteria problems international journal of the analytic hierarchy process 345 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.240 time, the alternative x'' with large gp-interval [ x g  , xg  ] has the gp values which, on the one hand, are higher than the upper boundary xg  of the nested gp-interval [ xg  , xg  ], making the alternative x'' better than the alternative x', and, on the other hand, less than the lower boundary x g  of the nested gp-interval [ xg  , xg  ], making the alternative x'' worse than the alternative x'. for the final selection of the best alternative in the considered case, subjective interval estimates and judgments provided by this subject making the decision to clarify them and narrow uncertainty intervals should be additionally analyzed. the practical calculations show that it is usually possible to reduce alternatives characterized by uncertain gp-intervals to a comparable type in which the best alternative will have the maximal lower and upper gp-interval boundaries. 3. example of application of interval stochastic ahp let's consider the multi-criteria problem of selecting the best place in the region (of the three proposed x1, x2, x3) for the placement of a publicly significant object assessed in terms of the infrastructural maturity (the criterion f1), the solvent demand of the population (the criterion f2), the competitive environment density (the criterion f3), the estimated cost of future construction (the criterion f4). the subject cannot know apriori the exact values of the factors of a multi-criteria problem at the time of the decision, and the subject can only roughly estimate the boundaries of the intervals, within which, in his opinion, certain factors can assume values. moreover, the possible values of the factors in their variation intervals can be located at any point with equal probability. due to the interval stochastic nature of the estimates of the subject, the pairwise comparison matrices of relative importance of the criteria and alternatives in terms of each criterion are interval stochastic matrices, each element of which is an interval stochastic variable uniformly distributed within its variation interval. let's assume that the interval stochastic pairwise comparison matrices in the this example are: – for the criteria f1, f2, f3, f4 аf =             1]5/1,7/1[]6/1,9/1[]5/1,7/1[ ]7,5[1]3/1,5/1[]4/1,6/1[ ]9,6[]5,3[1]2/1,3/1[ ]7,5[]6,4[]3,2[1 , – for the alternatives x1, x2, x3 relative to each criterion f1, f2, f3, f4 1f x a =           1]4/1,5/1[]5/1,7/1[ ]5,4[1]3/1,5/1[ ]7,5[]5,3[1 , 2f x a =           1]4,2[]8,6[ ]2/1,4/1[1]6,3[ ]6/1,8/1[]3/1,6/1[1 , ijahp article: madera/interval uncertainty of estimates and judgments of subject in decision making in multi-criteria problems international journal of the analytic hierarchy process 346 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.240 3f x a =           1]7/1,9/1[]2/1,4/1[ ]9,7[1]6,4[ ]4,2[]4/1,6/1[1 , 4f x a =           1]2/1,3/1[]7,5[ ]3,2[1]8,6[ ]5/1,7/1[]6/1,8/1[1 . the calculations of the statistical measures of interval stochastic eigenvectors for interval stochastic pairwise comparison matrices were carried out according to the formulas obtained above (sections2.1 – 2.3). the results of the calculations of the statistical measures me and sd for the coefficients of relative importance of the criteria and the alternatives relative to the criteria are given in table 1, and for the gps of alternatives in table 2. table 2 also presents the calculated variation gp-intervals for various alternatives. table 1 statistical measures me and sd of the coefficients of relative importance of criteria and alternatives for each criterion criteria f1 f2 f3 f4 me sd me sd me sd me sd 0,507 0,024 0,321 0,018 0,129 0,007 0,043 0,002 alternatives me sd me sd me sd me sd x1 0,676 0,039 0,0759 0,006 0,183 0,014 0,068 0,003 x2 0,246 0,013 0,273 0,025 0,741 0,034 0,614 0,029 x3 0,079 0,003 0,651 0,046 0,076 0,005 0,318 0,016 table 2 statistical measures me and sd and variation gp-intervals of alternatives global priorities of alternatives statistical measures of global priorities variation gpintervals of alternatives me sd gx1 0,394 0,334 0,273 0,026 [0.315, 0.472] gx2 0,015 [0.290, 0.378] gx3 0,019 [0.216, 0.330] figure 2. boundaries of the gp-intervals of alternatives obtained by the interval stochastic ahp method developed in this article ijahp article: madera/interval uncertainty of estimates and judgments of subject in decision making in multi-criteria problems international journal of the analytic hierarchy process 347 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.240 figure 2 shows the calculated boundaries of the gp-intervals of alternatives x1, x2, x3. the resulting intervals of possible gp values (table 2) serve as the basis for selecting the best compromise alternative (section2.4). in this example, the best alternative is x1 (fig. 2) since the lower and upper boundaries of the interval of its gps have maximal values of all the other gp-intervals of alternatives. the worst alternative will be the one for which the gp-interval boundaries are minimal (the alternative x3, fig. 2). 4. conclusion the interval stochastic анр method developed in the article is an extension of the deterministic анр, also applying to the interval stochastic uncertainty, which is closer in nature to subjective evaluation and decision-making. with the obtained formulas, the statistical measures (mathematical expectations me, variances var and standard deviations sd) for both interval stochastic eigenvectors of pairwise comparison matrices and intervals of global priorities gps of various alternatives can be determined. based on the determined values of statistical measures, the intervals of possible gps are measured, namely: me – ε∙sd ≤g(ω) ≤ me + ε∙sd corresponding to various alternatives. the best compromise decision corresponds to the alternative with such a gp-interval whose lower and upper boundaries are the highest among the boundaries of all other gp-intervals of alternatives. this can always be achieved through interactions with the subject making decisions, by specifying his subjective estimates. the interval stochastic ahp method proposed in this article allows one to make the best compromise decision in multi-criteria problems given the interval stochastic uncertainty of subjective estimates, which reflects the real psychology of the subject carrying out the estimation, selection and decisionmaking. ijahp article: madera/interval uncertainty of estimates and judgments of subject in decision making in multi-criteria problems international journal of the analytic hierarchy process 348 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.240 references deng, h. (1999). multicriteria analysis with fuzzy pairwise comparison. international journal of approximate reasoning, 21, 215−231. eskandari, h., and rabelo, l. (2007). handling uncertainty in the analytic hierarchy process: a stochastic approach. international journal of information technology & decision making, 6(01), 177−189. haines, l.m., 1998 haines, l. m. (1998). a statistical approach to the analytic hierarchy process with interval judgements. (i). distributions on feasible regions. european journal of operational research, 110(1), 112−125. kahneman, d., slovic, p., and tversky, a. (2001). judgment under uncertainty: heuristics and biases. cambridge: cambridge university press. lipovetsky, s., and tishler, a. (1999). interval estimation of priorities in the ahp. european journal of operational research, 114(1), 153−164. madera, a.g. (2010). modeling and decision making in management. moscow: lki. madera, a.g. (2014). risks and chances: uncertainty, prediction and evaluation. moscow: krasand. mikhailov, l. (2004). a fuzzy approach to deriving priorities from interval pairwise comparison judgements. european journal of operational research, 159(3), 687−704. mikhailov, l., didehkhani, h., sadi-nezhad, s. (2011). weighted prioritization models in the fuzzy analytic hierarchy process. international journal of information technology and decision making,10(4), 1−14. podinovski, v.v. (2007). interval articulation of superiority and precise elicitation of priorities. european journal of operational research,180, 406 – 417. ross, s. (1993). introduction to probability models, 5 th ed. new york: academic press. saaty, t.l. (2001). decision making with dependence and feedback: the analytic network process. pittsburgh, pa: rws publications. salo, a.a., and hämäläinen r.p. (1995). preference programming through approximate ratio comparisons. european journal of operational research,82, 458 – 475. wang y.-m., yang j.-b., xu d.-l.(2005). interval weight generation approaches based on consistency test and interval comparison matrices. applied mathematics and computation,167, 252 – 273. ijahp article: pal singh, singh/strategic enhancement of workplace safety in small scale manufacturing industries using ahp approach international journal of the analytic hierarchy process 269 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.621 strategic enhancement of workplace safety in small scale manufacturing industries using ahp approach lakhwinder pal singh associate professor department of industrial and production engineering dr b r ambedkar national institute of technology singhl@nitj.ac.in satnam singh phd candidate department of industrial and production engineering dr. b.r. ambedkar national institute of technology jalandhar, and punjab, india assistant professor lovely professional university phagwara, punjab, india. satnam_pisces@yahoo.com abstract despite progress in technology, workplace safety in small scale manufacturing industries (ssmis) still lacks the required attention. occupational safety of workers is one of the major concerns for organizations as it involves a number of factors that affect many direct and indirect costs of the industry. to this end, the current study was conducted in order to determine the key factors that affect workplace safety. a total of nine main factors and twenty-two sub-factors were identified and then prioritized using the analytic hierarchy process (ahp). opinions from experienced and proficient experts were recorded on a framed questionnaire for all the above factors. the effectiveness of the study was ensured by maintaining a consistency ratio of less than 10% for the factors and sub-factors. the factor personal protective equipment was found to have the highest eigen vector of 27.4%, and therefore the highest priority. organizational attributes and hygiene are the factors that demand the next level of priority, whereas, equipment and hand tools safety and machine guarding, and material handling are the factors requiring the least priority. the outcome of this analysis enables the small scale manufacturing industries to effectively implement safety measures by giving priority to the factors in the order specified by the study. this would empower the organizational safety standards and benefit not only the employees but also the employers without considerable costs. keywords: analytical hierarchy process (ahp); small scale manufacturing industry (ssmi); workplace safety; safety factors 1. introduction small scale industries have played a vital role in the development of india, and industrial growth has contributed greatly to the development of the indian economy mailto:singhl@nitj.ac.in mailto:satnam_pisces@yahoo.com ijahp article: pal singh, singh/strategic enhancement of workplace safety in small scale manufacturing industries using ahp approach international journal of the analytic hierarchy process 270 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.621 (wani et al., 2004). it is well-known that industrial production can be enhanced considerably by providing better safety aspects at the workplace (singh et al., 2009). occupational safety is concerned with protecting the safety and welfare of people engaged in work or employment. the goal of any occupational safety program is to develop a safe and healthy work environment which follows a set of laws that have been made to protect people while they work. safety programs also protect, directly or indirectly, the co-workers, family members, employers, customers and many others who might be affected by the workplace environment as they are all related to the employee/worker. safety is a major concern for individuals as well as organizations, as it is a substantial cause of direct and indirect costs (andersson & menckel, 1995). every year, a large number of employees are injured in small scale manufacturing industries because of workplace accidents due to improper implementation or sometimes the absence of safety norms (takala et al., 2014). occupational accidents and injuries not only lead to permanent disabilities or deaths and/or economic losses, but they also affect the efficiency of the victim and other workers (singh, 2018). in this way, occupational accidents result in economic losses for the employee as well as the employer. occupational safety is the prime responsibility of an organization because the workers are the soul of any progressive organization. safety is of great importance in industrial development and productivity of small scale manufacturing industries. therefore, there is a strong necessity to implement necessary safety measures in industrial organizations, as well-organized occupational safety management is the key to any successful industrial establishment. the utmost attention needs to be given to maintaining safety standards at the workplace of ssmis and the employer needs to reasonably ensure the practical safety of an employee from injuries and health risks at the workplace (marhavilas et al., 2011). the growth of small and medium scale manufacturing industries have a large contribution to the economic development of a nation (wani et al., 2004). however, with an increase in the number of small and medium enterprises, occupational injuries have also increased (bhagwat & sharma, 2007; singh et al., 2009). the reason for the increase is primarily due to lack of awareness and training regarding occupational safety, improper workplace design, unstructured jobs, a mismatch between worker abilities and job demands, an adverse working environment and inappropriate management programs (saiyed & tiwari, 2004; shikdar &sawaqed, 2003). it is wellknown that the production rate can be enhanced by providing better safety aspects at the workplace (singh et al., 2009). in order to develop a good safety culture, the attitude of workers needs to be reoriented by adopting best practices such as hazard measurement, training, good housekeeping and use of better personal protective equipment (wilson & corlett, 2005; feyer and williamson, 1991). an employee’s perception reflects the value of safety in the organization. management is not only responsible for the development of safety oriented policies and procedures, but is also accountable for the implementation of safety enhancing systems (vredenburgh, 2002). therefore, it is imperative to identify and prioritize the underlying causes of workplace accidents in small scale manufacturing industries so that effective safety interventions can be designed and implemented. a number of studies demonstrate the effectiveness of workplace safety and health administration in successful business performance (kwon & kim, 2013). training and enforcement of safety practices impact worker behavior and help prevent accidents on the work floor (atsumbe et al., 2012). improvements in engineering controls, personal protective equipment, safer machinery and processes and http://en.wikipedia.org/wiki/safety http://en.wikipedia.org/wiki/quality_of_life http://en.wikipedia.org/wiki/employment http://simple.wikipedia.org/wiki/law ijahp article: pal singh, singh/strategic enhancement of workplace safety in small scale manufacturing industries using ahp approach international journal of the analytic hierarchy process 271 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.621 adherence to regulations and labor inspections have proved to be the key role parameters (brauer, 2016). regardless of the nature of the work, workers should be able to carry out their responsibilities in a safe and secure environment that is free from hazards. occupational safety and health (osh) is generally defined as the science of anticipation, recognition, evaluation and control of hazards arising in or from the workplace that could harm the health and well-being of workers (zwetsloot & leka, 2010). even though the importance of osh has remained the same, the working environment and the overall conditions in society are always in a state of change (peterson, 2005). moreover, a wide range of new issues are constantly being added to osh due to the rapid development of science and technology in industries. studies on safety factors in medium scale industries were reported by singh et al. (2016). these types of studies are essential for small scale manufacturing industries to increase the level of workplace safety, reduce the rate of accidents and increase the production rate. hence, the present study seeks to prioritize the key factors affecting occupational safety in ssmis. therefore, the analytic hierarchy process (ahp) is utilized to prioritize all the factors along with their respective sub-factors related to workplace safety in ssmis. a questionnaire is developed for conducting the study and the opinions of experienced and proficient experts is recorded for different factors and sub-factors with pair-wise comparison. 2. ahp for prioritization of safety factors the analytical hierarchy process (ahp), developed by saaty (1990), is a combination of mathematics and interaction of the intended work (viswanadhan, 2005; wang & wang, 2010). ahp is one of the most successful techniques for solving decision making problems involving goals, alternatives for reaching the goals and criteria for evaluating the alternatives (harker & vargas, 1987). ahp is successfully implemented in various organizations such as integrated manufacturing, layout design (al-harbi, 2001), assessment of technology asset decisions (boucher & macstravic, 1991), flexible industrialized systems and in many other engineering related fields (arbel & orgler, 1990; armacost et al., 1994; cambron & evans, 1991; das et al., 2012; saaty, 1990; shikdar & al-araimi, 2001). ahp is effective in prioritizing the factors to mitigate unforeseen accidents in industries, and the implementation of prioritized factors saves unallocated funds (akarte et al., 2001; alharbi, 2001; badri, 2001). effective implementation of ahp can increase the growth of smes in many aspects (mudavanhu et al., 2013; singh et al., 2016). the approach is based on the ability of mathematical structure of consistent matrices and the associated eigen vectors to generate true or approximate weights. ahp works on an eigen value which is based on pairwise comparisons (bayazit, 2005; boucher & mac stravic, 1991; saaty, 1990). qualitative and quantitative analyses can be performed simultaneously and calibration can be done using the suitable numeric scale (saaty, 1985). the following steps give a detailed procedure for carrying out the ahp analysis (saaty, 1985; saaty, 1990). step-1: hierarchical structuring of a decision problem and selection of criteria. at the topmost level, this is comprised of a goal or focus. at the intermediate and lower levels, the approach deals with criteria or sub-criteria and the available alternatives, respectively. step-2: construction of a pairwise comparison matrix for each level with respect to higher levels. in this step, the relative importance of different alternatives with ijahp article: pal singh, singh/strategic enhancement of workplace safety in small scale manufacturing industries using ahp approach international journal of the analytic hierarchy process 272 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.621 respect to the immediately above sub-criteria is determined. this is followed by rating the relative priority of the criteria by assigning a weight between 1 (equal importance) and 9 (extreme importance) to the more important criterion. in contrast, the reciprocal of this value is assigned to the other criterion in the pair. step-3: application of eigen vector methods to calculate the relative weight for the pairwise comparison of options on each criterion. step-4: check the consistency associated with the comparison matrix. this is achieved using the consistency ratio (cr) of consistency index (ci) with the appropriate value of the random index (ri). step-5: repeat the above steps for all levels in the hierarchy. step-6: evaluate the overall relative value by linear addition function. 3. ahp methodology the methodology of ahp involves five key steps as shown in figure 1. at first, the problem and the concerned factors that affect the problem are identified and selected. then, feedback is recorded from field experts on a developed questionnaire and a consistency test is performed. every step is scrutinized with ample care. the ranking of different factors is based on the 9-point ahp scale shown in table 1. increasing rank indicates the growing importance of the factor for which that rank is given. figure 1 methodology for analytic hierarchy process (ahp) step-2: selection of factors step-4: feedback from experts step-3: framing of questionnaire step-5: consistency test step-1: problems identification and concerned factors ijahp article: pal singh, singh/strategic enhancement of workplace safety in small scale manufacturing industries using ahp approach international journal of the analytic hierarchy process 273 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.621 table 1 9-point scale of analytic hierarchy process ahp it is ensured that the consistency ratio (cr), the ratio between ci and ri, is less than 10%, after the pairwise comparison is completed because only then is the questionnaire considered adequate. if the cr is above 10%, the questionnaire needs to be revised because it might not accommodate the possible factors. accordingly, the random index (ri) is obtained for a different number of factors considered for this study (saaty, 1990; triantaphyllou & mann, 1995) (table 2). table 2 values for random index (ri) for different number of factors considered no. of factors (n) 1 2 3 4 5 6 7 8 9 10 ri 0.00 0.00 0.58 0.90 1.12 1.24 1.32 1.41 1.45 1.49 4. ahp hierarchy model for ssmis based on the above discussions, a hierarchy model for ssmis is presented in figure 2 in the form of nine main factors and 22 sub-factors. the detailed procedure for the ahp method with pairwise comparisons based on the experts’ opinion has been discussed here for one of the main factors f5 and its sub-factors f51,f52,f53and f54 (figure 3). these four sub-factors refer to: f51: provision of fire detection system (pfds); f52: need of fire-fighting training and emergency plan (nft); f53:provision of emergency exit, exit signs and other relevant safety signs (pee) and f54: proper electrical wiring (pew). as part of the questionnaire, the rating given by experts for these four sub-factors is highlighted in figure 3. relative importance definition 1 equally important. 3 moderate importance of one over another. 5 essential or strong importance. 7 demonstrated importance. 9 absolute importance. 2, 4, 6, 8 intermediate values between the two neighbouring scales. ijahp article: pal singh, singh/strategic enhancement of workplace safety in small scale manufacturing industries using ahp approach international journal of the analytic hierarchy process 274 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.621 figure 1 ahp hierarchical model for ssmis figure 2 rating given by experts for the sub-factors f51,f52,f53, f54 of the main factor f5: fire prevention, fire-fighting and electrical safety the fourth order matrix [a] for the pairwise comparison was prepared and represented by matrix (1), in which the diagonally positioned elements are unity, while the upper triangular part of the matrix is filled as described in the following two steps.  step 1. if the judgemental value given by the expert lies on left hand side of 1, then the value is entered as it is in the matrix.  step 2. if the judgemental value given by the expert lies on right hand side of 1, then the reciprocal of that value is entered in the matrix (saaty, 1985). f9f7 f8f6f5 f53 f52 f51 f62 f61 f72 f71 f83 f82 f81 f54 f92 f91 f12 f4 f43 f42 f41 f1 f2 f3 goal f11 f22 f21 f32 f31 9 main factors 22 sub-factors factors pfds nft pfds pee pfds pew nft pee nft pew pee pew 9 7 5 3 1 3 5 9 7 5 3 7 1 3 5 9 7 9 9 7 5 3 1 3 5 7 9 9 7 5 3 1 3 5 7 9 9 7 5 3 1 3 5 7 9 9 7 5 3 1 3 5 7 9 ijahp article: pal singh, singh/strategic enhancement of workplace safety in small scale manufacturing industries using ahp approach international journal of the analytic hierarchy process 275 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.621   1 5 3 1 1 5 1 3 1 3 1 3 1 3 1 1 3 1 3 3 1 pfds nft pee pew pfds nft a pee pew             (1) once the upper triangular matrix is completed, the lower triangular matrix is filled by taking the reciprocal of the values mirroring the diagonal of the matrix in the upper triangular matrix. it is worthwhile to note that only positive values are entered in the matrix in order to prevent miscalculation. next, the eigen values and eigen vector need to be calculated. for this, each element of matrix (1) is divided by its respective column summation to obtain matrix (2). as a check, the sum of each column in the newly generated matrix (2) must be unity.   0.3947 0.535 0.3 0.3751 0.078 0.107 0.3 0.124 0.131 0.036 0.1 0.124 0.394 0.322 0.3 0.375 pfds nft pee pew pfds nft a pee pew             (2) afterwards, the normalized principal eigen vector {w}, also known as the priority vector, is computed from the arithmetic mean of the respective rows of matrix (2). the vector {w} is shown as matrix (3) below. 0.401 0.154 { } 0.098 0.347 w                (3) the values in vector {w} indicate the relative importance of the four sub-factors f51: pfds, f52: nft,f53: pee and f54: pew of the main factors f5. therefore, the highest priority is given to pfds (0.401), followed by pew (0.347), and then nft (0.154). with a value of w=0.098, the factor pee is given the least priority. hence, based on priority matrix {w}, the relative ranking of factors pfds, pew, nft, pee is 40.1%, 34.7%, 15.4%, and 9.8%, respectively. using equation 4, the eigen value of factor f5 is evaluated by taking the sum of the product of column summations of matrix (1) with the respective principle eigen vector element in matrix (3). this is followed by the calculation of consistency index (ci) using equation 5.        max 38 28 8 0.401 0.154 10 0.098 0.347 4.2783 15 3 3       (4) max 4.2783 4 0.069575 4 n ci n       (5) ijahp article: pal singh, singh/strategic enhancement of workplace safety in small scale manufacturing industries using ahp approach international journal of the analytic hierarchy process 276 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.621 since, the number of factors considered in this particular problem are n = 4, (f51, f52, f53, f54), we will use the random index ri = 0.90 corresponding to n = 4 in table 2. the consistency ratio (cr) is then evaluated as the ratio of consistency index and random index which is 7.73%cr ci ri  . since the cr is 7.73%, which is less than 10%, this means the framework is accepted. this has illustrated the ahp technique with one framework comprised of comparisons of four sub-factors f51, f52, f53 and f54. in the same manner, an ahp analysis was carried out over other frameworks considering the remaining factors and the results are presented in table 3. this table gives the respective eigen vector, priority, and consistency ratio after applying ahp on the sub-factors of the remaining main factors. the prioritization of sub-factors within different frameworks was completed and prioritized. in all the cases, the consistency ratio is well below the acceptable value of 10%, which is evidence of the fact that the judgment of experts is rational. ijahp article: pal singh, singh/strategic enhancement of workplace safety in small scale manufacturing industries using ahp approach international journal of the analytic hierarchy process 277 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.621 table 3 eigen vector (w), priority and consistency ratio (cr) for the various sub-factors considered for ssmis fx: main factor fxj: sub-factor w priority cr f1: organizational attributes f11: existence of safety policy 75.0% 1 0% f12: functioning of safety department 25.0% 2 f2: occupational safety service f21: provision of first aid services 83.3% 1 0% f22: records of accident and injury 16.7% 2 f3: work place layout and housekeeping f31: adequate and smooth material flow 75.0% 1 0% f32: provision of proper disposal of waste 25.0% 2 f4: equipment and hand tools safety and machine guarding f41: need of periodic inspection 10.5% 3 6.4% f42: availability of proper machine guards 63.7% 1 f43: provision of training programs for hand tools and equipment use 25.8% 2 f5: fire prevention, firefighting and electrical safety f51: provision of fire detection system 40.1% 1 7.7% f52: need of fire-fighting training and emergency plan 15.4% 4 f53: provision of emergency exit, exit signs and other relevant safety signs 9.8% 3 f54: proper electrical wiring 34.7% 2 f6: material handling and storage f61: need of inspection schedule 66.7% 1 0% f62: provision of safe storage and stacking 33.3% 2 f7: occupational exposures f71: exposure to high thermal conditions 66.7% 1 0% f72: monitoring of occupational exposures 33.3% 2 f8: personal protective equipment f81: adequate provision of ppe 73.1% 1 6.8% f82: proper maintenance of ppe 18.8% 2 f83: adequate training on ppe usage 8.1% 3 f9: hygiene factors f91: availability of safe drinking water 25.0% 2 0% f92: provision of proper lighting and ventilation 75.0% 1 ijahp article: pal singh, singh/strategic enhancement of workplace safety in small scale manufacturing industries using ahp approach international journal of the analytic hierarchy process 278 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.621 5. results and discussion in a similar way, the main factors are also prioritized using ahp and the priority level is tabulated in table 4. with an eigen vector w = 27.4%, factor f8 (personal protective equipment) is highly prioritized, followed by f1 (organizational attributes) with w=16.8% and f9 (hygiene factors) with w =11.4%. however, prioritization of only the main factors is not enough; there is a strong need to prioritize the sub-factors of each of the main factors as well. without proper prioritization of the sub-factors, the least prioritized factor might be given more importance while the essential factor is missed. this could result in a delay in the progress of workplace safety and in the worst circumstances may further degrade the safety at ssmis. accordingly, the sub-factors of all the main factors have been prioritized based on the analytical hierarchy process. for the first main priority factor f8, it is observed that the sub-factor f81: adequate provision of ppe is almost 4 times more essential than f82: proper maintenance of ppe. this implies that while implementing the required safety measures to improve ppe at the workplace, the most importance must be given to adequate provisions of ppe, followed by proper maintenance of ppe and finally training on ppe. the prioritization of main factors along with their sub-factors helps management make efficient decisions about safety implementation in ssmis. eventually, this will lead to considerable savings in time and cost that can be offset for worker welfare. the next three main factors with an intermediate priority of w = 10.1%, 8.4%, 7% are f5 (fire prevention, fire-fighting, and electrical safety), f2 (occupational safety services/documentation), and f3 (workplace layout and housekeeping), respectively. again, the advantage of ahp is to segregate the least prioritized factors from the important ones, which can be established from the prioritization of the four subfactors of factor f5. it should be noted, that among all the main factors considered in this study, factor f5 has the highest number of sub-factors which is four. now, in the absence of any prioritization of these four sub-factors: f51, f52, f53, f54, it would be a complex task for the management to distribute the necessary equipment, money and time required by each factor. from table 3, it is observed that the provision of a fire detection system   51 38.6%fw  is the most important parameter to improve fire prevention, fire-fighting and electrical safety at workplace. moreover, to fully develop the fire prevention, fire-fighting and electrical safety at workplace proper electrical wiring   54 36.6%fw  , provision of an emergency exit, exit signs and other relevant safety signs   53 13.8%fw  and need of fire-fighting training and an emergency plan   52 10.9%fw  are the next three key factors to be taken care of in the same order as they are presented here. finally, the factors: f7 (occupational exposures), f4 (equipment and hand tools safety and machine guarding) and f6 (material handling and storage) have a minimum eigen vector of 7 6.5%fw  , 4 6.4%fw  and 6 5.9%fw  , respectively and hence need the least attention from the management of ssmis. based on the results, the ssmis should focus on personal protective equipment (ppe) and organizational attributes and hygiene of the workers as these are the three most critical factors that need immediate attention to avoid loss of life and cost at the ijahp article: pal singh, singh/strategic enhancement of workplace safety in small scale manufacturing industries using ahp approach international journal of the analytic hierarchy process 279 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.621 workplace. considering the outcome of this study, it is highly recommended that ssmis follow this hierarchy while implementing the safety measures on the workplace. table 4 eigen vector (w) and priority for various main factors of ssmis main factor w priority f1: organizational attributes 16.8% 2 f2: occupational safety services/documentation 8.4% 5 f3: workplace layout and housekeeping 7.0% 6 f4: equipment and hand tools safety and machine guarding 6.4% 8 f5: fire prevention, fire-fighting and electrical safety 10.1% 4 f6: material handling and storage 5.9% 9 f7: occupational exposures 6.5% 7 f8: personal protective equipment 27.4% 1 f9: hygiene factors 11.4% 3 6. conclusion the analytical hierarchy process is used to prioritize the nine main factors and twenty-two sub-factors of occupational safety associated with ssmis. based on the ahp analysis, it is observed that personal protective equipment has the highest priority, followed by organizational attributes and hygiene factors to improve workplace safety. further, it is important to individually prioritize sub-factors of each main factor as well as the main factors in order to effectively enhance workplace safety. the ssmis should follow the hierarchy presented in this work while implementing safety measures and the industry should also give the highest priority to the factors at the top of the ranking list and conduct necessary training programs in order to prevent workplace accidents. it is envisaged that by employing the hierarchy developed in this study, any newly established ssmi can become capable of saving effort, time and money and can also build a safer working environment. ijahp article: pal singh, singh/strategic enhancement of workplace safety in small scale manufacturing industries using ahp approach international journal of the analytic hierarchy process 280 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.621 references akarte, m., surendra, n., ravi, b., and rangaraj, n. (2001). web based casting supplier evaluation using analytical hierarchy process. journal of the operational research society, 52(5), 511-522. doi: https://doi.org/10.1038/sj.jors.2601124 al-harbi, k. m. a. (2001). application of the ahp in project management. international journal of project management, 19(1), 19-27. doi: https://doi.org/10.1016/s0263-7863(99)00038-1 andersson, r., and menckel, e. (1995). on the prevention of accidents and injuries: a comparative analysis of conceptual frameworks. accident analysis & prevention, 27(6), 757-768. doi: https://doi.org/10.1016/0001-4575(95)00031-3 arbel, a., and orgler, y. e. (1990). an application of the ahp to bank strategic planning: the mergers and acquisitions process. european journal of operational research, 48(1), 27-37. doi: https://doi.org/10.1016/0377-2217(90)90058-j armacost, r. l., componation, p. j., mullens, m. a., and swart, w. w. (1994). an ahp framework for prioritizing customer requirements in qfd: an industrialized housing application. iie transactions, 26(4), 72-79. doi: https://doi.org/10.1080/07408179408966620 atsumbe, b., amine, j., umar, i., and salawu, j. (2012). appraisal of occupational safety in manufacturing industries in kaduna and niger states of nigeria. international journal of engineering research and development, 6, 55-62. badri, m. a. (2001). a combined ahp–gp model for quality control systems. international journal of production economics, 72(1), 27-40. doi: https://doi.org/10.1016/s0925-5273(00)00077-3 bayazit, o. (2005). use of ahp in decision-making for flexible manufacturing systems. journal of manufacturing technology management, 16(7), 808-819. doi: https://doi.org/10.1108/17410380510626204 bhagwat, r., and sharma, m. (2007). information system architecture: a framework for a cluster of small-and medium-sized enterprises (smes). production planning & control, 18(4), 283-296. doi: https://doi.org/10.1080/09537280701248578 boucher, t. o., and macstravic, e. l. (1991). multiattribute evaluation within a present value framework and its relation to the analytic hierarchy process. the engineering economist, 37(1), 1-32. doi: https://doi.org/10.1080/00137919108903055 brauer, r. l. (2016). safety and health for engineers. john wiley & sons. ijahp article: pal singh, singh/strategic enhancement of workplace safety in small scale manufacturing industries using ahp approach international journal of the analytic hierarchy process 281 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.621 cambron, k. e., and evans, g. w. (1991). layout design using the analytic hierarchy process. computers & industrial engineering, 20(2), 211-229. doi: https://doi.org/10.1016/0360-8352(91)90026-3 das, m. c., sarkar, b., and ray, s. (2012). a framework to measure relative performance of indian technical institutions using integrated fuzzy ahp and copras methodology. socio-economic planning sciences, 46(3), 230-241. doi: https://doi.org/10.1016/j.seps.2011.12.001 feyer, a.-m., and williamson, a. m. (1991). a classification system for causes of occupational accidents for use in preventive strategies. scandinavian journal of work, environment & health, 302-311. doi: https://doi.org/10.5271/sjweh.1698 harker, p. t., and vargas, l. g. (1987). the theory of ratio scale estimation: saaty's analytic hierarchy process. management science, 33(11), 1383-1403. doi: https://doi.org/10.1287/mnsc.33.11.1383 kwon, o.-j., and kim, y.-s. (2013). an analysis of safeness of work environment in korean manufacturing: the “safety climate” perspective. safety science, 53, 233-239. doi: https://doi.org/10.1016/j.ssci.2012.10.009 marhavilas, p.-k., koulouriotis, d., and gemeni, v. (2011). risk analysis and assessment methodologies in the work sites: on a review, classification and comparative study of the scientific literature of the period 2000–2009. journal of loss prevention in the process industries, 24(5), 477-523. doi: https://doi.org/10.1016/j.jlp.2011.03.004 mudavanhu, n., zhou, t., and dzomba, p. (2013). an assessment of small and medium enterprise owners’ occupational safety and health efforts: the case of southerton, harare, zimbabwe. journal of scientific research & reports, 4(3), 407418. doi: https://doi.org/10.9734/jsrr/2014/6263 peterson, d. (2005). safety improvement: perception surveys can reveal strengths and weaknesses. professional safety, 50(1), 45-48. saaty, t. l. (1985). decision making for leaders. ieee transactions on systems, man, and cybernetics, 3, 450-452. saaty, t. l. (1990). how to make a decision: the analytic hierarchy process. european journal of operational research, 48(1), 9-26. doi: https://doi.org/10.1016/0377-2217(90)90057-i saiyed, h. n., and tiwari, r. r. (2004). occupational health research in india. industrial health, 42(2), 141-148. doi: https://doi.org/10.2486/indhealth.42.141 shikdar, a. a., and al-araimi, s. a. (2001). ergonomic conditions in small manufacturing industries" sultan qaboos university journal for science [squjs], 6(1), 61-70. doi: https://doi.org/10.24200/squjs.vol6iss1pp61-70 ijahp article: pal singh, singh/strategic enhancement of workplace safety in small scale manufacturing industries using ahp approach international journal of the analytic hierarchy process 282 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.621 shikdar, a. a., and sawaqed, n. m. (2003). worker productivity, and occupational health and safety issues in selected industries. computers & industrial engineering, 45(4), 563-572. doi: https://doi.org/10.1016/s0360-8352(03)00074-3 singh, l. p. (2018). work study and ergonomics. oxford: cambridge university press. singh, l. p., bhardwaj, a., deepak, k., and bedi, r. (2009). occupational noise exposure in small scale hand tools manufacturing (forging) industry (ssi) in northern india. industrial health, 47(4), 423-430. doi: https://doi.org/10.2486/indhealth.47.423 singh, s., singh, l. p., and kaur, m. (2016). analytical hierarchy process-based methodology for selection of safety parameters in manufacturing industry. in arezes p. (ed.) advances in safety management and human factors, 491, springer, cham. doi: https://doi.org/10.1007/978-3-319-41929-9_33 singh, s., singh, l. p. (2016). selection of occupational safety factors related to small scale manufacturing industry. 14 th international conference on humanizing work and work environment hwwe-2016. triantaphyllou, e., and mann, s. h. (1995). using the analytic hierarchy process for decision making in engineering applications: some challenges. international journal of industrial engineering: applications and practice, 2(1), 35-44. viswanadhan, k. (2005). how to get responses for multi-criteria decisions in engineering education–an ahp based approach for selection of measuring instrument. financial support, 20. vredenburgh, a. g. (2002). organizational safety: which management practices are most effective in reducing employee injury rates?. journal of safety research, 33(2), 259-276. doi: https://doi.org/10.1016/j.jsr.2013.07.040 wang, d., and wang, x. (2010). research on the green vendor selection index weight of iron & steel enterprises based on ahp and entropy method. soft science,8, 025. wani, v., garg, t., and sharma, s. (2004). effective industry/institute interaction for developing entrepreneurial vision amongst engineers for the sustainable development of smes in india. international journal of technology transfer and commercialisation, 3(1), 38-55. doi: https://doi.org/10.1504/ijttc.2004.003522 wilson, j. r., and corlett, n. (2005). evaluation of human work. crc press. zwetsloot, g., and leka, s. (2010). corporate culture, health, and well-being. occupational health psychology, 21(2) 250-268. ijahp article: alam, ku khalif, jaini/analytic hierarchy process based on the magnitude of znumbers international journal of the analytic hierarchy process 1 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1063 analytic hierarchy process based on the magnitude of z-numbers nik muhammad farhan hakim nik badrul alam 1 centre for mathematical sciences, universiti malaysia pahang gambang, malaysia college of computing, informatics and mathematics, universiti teknologi mara pahang jengka, malaysia farhanhakim@uitm.edu.my ku muhammad naim ku khalif centre for mathematical sciences, universiti malaysia pahang, gambang, malaysia centre of excellence for artificial intelligence and data science univerisiti malaysia pahang gambang, malaysia kunaim@ump.edu.my nor izzati jaini centre for mathematical sciences, universiti malaysia pahang, gambang, malaysia ati@ump.edu.my abstract the analytic hierarchy process (ahp) is a powerful multi-criteria and multi-alternative decision-making model, which assists decision-makers in giving preferences using pairwise comparison matrices. the development of the ahp using fuzzy numbers has received attention from many researchers due to the ability of fuzzy numbers to handle vagueness and uncertainty. the integration of the ahp with fuzzy z-numbers has improved the model since the reliability of the decision-makers is considered, in which the judgment is followed by a degree of certainty or sureness. most of the existing decision-making models based on z-numbers transform the z-numbers into regular fuzzy numbers by integrating the reliability parts into the restriction parts, causing a significant loss of information. hence, this study develops the ahp based on the magnitude of z acknowledgements: all the authors would like to thank universiti malaysia pahang for laboratory facilities as well as financial support under ump postgraduate research grants scheme (pgrs) no. pgrs220301. mailto:farhanhakim@uitm.edu.my mailto:kunaim@ump.edu.my ijahp article: alam, ku khalif, jaini/analytic hierarchy process based on the magnitude of znumbers international journal of the analytic hierarchy process 2 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1063 numbers, which is used to represent the criteria weights. a numerical example of criteria ranking for the prioritization of public services for digitalization is implemented to illustrate the proposed ahp model. keywords: ahp; magnitude; z-numbers; criteria ranking 1. introduction humans tend to describe almost everything with natural language, and the description is mainly based on cognitive and psychological factors. the decision-making process is related to the cognitive process, in which human thinking is involved. in the real world, human preferences are not well-defined (perote-peña & piggins, 2007). according to aliev et al. (2021), human preferences in the decision-making process are imprecise due to the complexity of alternatives, imperfect information, and psychological biases. in the early development of decision-making methods, crisp numbers were used to describe decision-makers’ preferences. however, due to the lack of information, the use of crisp numbers has led to uncertainty (aliev, 2013). the implementation of fuzzy sets introduced by zadeh (1965) can handle the uncertainty of preference degrees. accordingly, many multi-criteria decision-making (mcdm) methods have been developed to help decision-makers select the best alternatives when there are various attributes that need to be considered. the analytical hierarchy process (ahp) is one of the most powerful of these methods. the ahp was proposed by saaty (1980) and uses a pairwise comparison matrix to obtain the evaluation of decision makers. the ahp has been studied extensively due to the fact that it is simple, easy to use, and flexible (emrouznejad & ho, 2017). it has also been implemented to solve decisionmaking problems with many criteria in various fields such as education, management, engineering, manufacturing, and sports (vaidya & kumar, 2006). in 2011, zadeh introduced the concept of z-numbers to deal with partially reliable information. z-numbers are composed of restriction and reliability components; the reliability component describes how certain the preference on the restriction component is made. according to abdullahi et al. (2020), z-numbers are the generalization of real, interval, and fuzzy numbers. moreover, z-numbers are very powerful in describing decision-making information due to their capability of modeling the real-world. in this article, the strength of z-numbers is adopted in developing the ahp based on the magnitude of z-numbers. since the magnitude of fuzzy numbers exhibits visual and natural meaning (abbasbandy & hajjari, 2009), the magnitude of z-numbers is used to process the decision information instead of converting z-numbers into regular fuzzy numbers that lead to information loss (abdullahi et al., 2020; gardashova, 2019; shen & wang, 2018). ijahp article: alam, ku khalif, jaini/analytic hierarchy process based on the magnitude of znumbers international journal of the analytic hierarchy process 3 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1063 2. literature review the ahp is an additive weighting method described in the pairwise comparison matrix. the ahp was first proposed by saaty (1980) and was further extended into fuzzy ahp by many researchers due to its simplicity and flexibility. the fuzzy ahp generally replaces the crisp numbers used in the pairwise comparison matrix with fuzzy numbers to handle imprecision in the evaluation of the decision-makers. subsequently, van laarhoven and pedrycz (1983) extended saaty's (1980) ahp by using a fuzzy logarithmic least square method to process triangular fuzzy numbers in order to obtain the triangular fuzzy weights. meanwhile, ruoning and xiaoyan (1992) used interval numbers to develop the ahp in a fuzzy environment. the extent analysis method was also used in the fuzzy ahp by chang (1996). however, the extent analysis was unable to estimate the exact weights from the fuzzy comparison matrices (wang et al., 2008). furthermore, leung and cao (2000) defined the consistency of the fuzzy ahp. the fuzzy least-square priority method was also integrated with the ahp by xu (2000), which produced an analytic expression for the criteria weights. in the following year, buckley et al. (2001) proposed a direct fuzzification of saaty’s method to obtain the fuzzy weights. additionally, wang et al. (2006) modified van laarhoven and pedrycz's (1983) method using a constrained non-linear optimization model, which can directly derive the fuzzy weights for fuzzy pairwise comparison matrices. azadeh et al. (2013) extended the ahp based on z-numbers to solve the selection of private self-financing technical institutions; however, in their proposed model, the znumbers were converted into regular fuzzy numbers by integrating the second component into the first component using kang et al.'s (2012) transformation method. the transformation of z-numbers into regular fuzzy numbers causes a significant loss of information (abdullahi et al., 2020; gardashova, 2019; shen & wang, 2018). during the transformation, the reliability component of the z-numbers was converted into a crisp value, which was added to the restriction component as a weight. the weighted restriction component was then converted into regular fuzzy numbers, dissipating some information in the z-numbers since the decision information was not well preserved in the form of paired fuzzy numbers. meanwhile, a novel ahp based on the direct calculation of z-numbers was proposed by zeinalova (2018). instead of converting z-numbers into regular fuzzy numbers, the direct calculation using the arithmetic operations of z-numbers proposed by aliev et al. (2015) was used. although this method could avoid the issue of information loss, the method may, however, lead to high computational complexity caused by the extensive use of linear programming to solve simple problems (abdullahi et al., 2020). in the magnitude of z-numbers derived by farzam et al. (2021), the magnitudes of the restriction and reliability components of the z-numbers were combined using a convex compound. since the magnitude of fuzzy numbers is visual and natural (abbasbandy & hajjari, 2009), it is acceptable to develop the ahp based on the magnitude of z-numbers to preserve the decision information and model an efficient decision-making method. ijahp article: alam, ku khalif, jaini/analytic hierarchy process based on the magnitude of znumbers international journal of the analytic hierarchy process 4 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1063 3. preliminaries in general, a fuzzy number is an extension of zadeh’s fuzzy set, which entails a fuzzy subset of the real line, particularly whose maximum membership degrees are clustered around the average value. the prominent shapes of fuzzy numbers are triangular and trapezoidal. since the triangular fuzzy number is the simplest form of a fuzzy number (voskoglou, 2019), its application in developing the decision-making model in this article is, therefore, easier. the triangular fuzzy number is defined as follows: definition 1 (zhang et al., 2014): let  1 2 3, ,a a a  be a triangular fuzzy number. as such, its membership degree is characterized by       1 1 2 2 1 3 2 3 3 2 , , , , 0 , elsewhere. x a x a a a a a x x x a a a a               (1) definition 2 (zhang et al., 2014): suppose that  1 2 3, ,a a a  and  1 2 3, ,b b b  are triangular fuzzy numbers and  is a scalar. accordingly, (i)  1 1 2 2 3 3, ,a b a b a b      (ii)  1 1 2 2 3 3, ,a b a b a b   (iii)  1 2 3, ,a a a    (iv) 1 3 2 1 1 1 1 , , a a a          zadeh (2011) extended the classical fuzzy number into the z-number, which consists of both the restriction and reliability components. definition 3 (zadeh, 2011): a z-number,  ,z a r consists of two components. the first component, a represents the restriction on the value that a variable can take. the second component, r represents the degree of reliability or certainty of the first component. for simplicity, both components of the z-numbers are represented by triangular fuzzy numbers, as shown in figure 1. ijahp article: alam, ku khalif, jaini/analytic hierarchy process based on the magnitude of znumbers international journal of the analytic hierarchy process 5 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1063 figure 1 z-number,  ,z a r recently, farzam et al. (2021) proposed the magnitude of z-numbers by combining the magnitude of the first and second components using the concept of a convex compound. definition 4 (farzam et al., 2021): let  1 2 3 4, , ,a a a a a and  1 2 3 4, , ,r r r r r be the restriction and reliability components of a z-number, respectively. in this regard, the magnitude of a and r are given by    1 2 3 4 1 5 5 12 mag a a a a a    (2) and    1 2 3 4 1 5 5 12 mag r r r r r    , (3) respectively. hence, the magnitude of the z-number,  ,z a r is given by        1mag z mag a mag r    (4) where  0.5,1 to highlight that the first component is more important in representing the z-number. furthermore, farzam et al. (2021) formed some rules to rank the z-numbers based on the magnitude values. definition 5 (farzam et al., 2021): let  1 1 1,z a r and  2 2 2,z a r be two z-numbers having the magnitude defined in equation 4. accordingly, (i) 1 2 z z if          1 2 1 21mag a mag a mag r mag r          ijahp article: alam, ku khalif, jaini/analytic hierarchy process based on the magnitude of znumbers international journal of the analytic hierarchy process 6 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1063 (ii) 1 2 z z if          1 2 1 21mag a mag a mag r mag r          (iii) 1 2 z z if          1 2 1 21mag a mag a mag r mag r          4. proposed ahp based on the magnitude of z-numbers in this section, the ahp based on the z-numbers is proposed. steps 1 to 3 are contingent on the methodology from buckley’s fuzzy ahp (buckley, 1985), but were conducted separately on the restriction and reliability components. step 4 converts the fuzzy weights representing the restriction and reliability components into magnitude values before combining them in step 5. the combined weight is further normalized in step 6 before being ranked. the detailed steps for the proposed z-number-based ahp are as follows: step 1: construct the pairwise comparison matrices to represent the decision maker’s preferences on the restriction and reliability of each criterion. while ij a denotes the degree to which the i-th criterion is preferred to the j-th criterion, ij r represents the degree of reliability when ij a is determined. ij a and ij r are represented by triangular fuzzy numbers. table 1 pairwise comparisons for the restriction of the criteria criterion 1 criterion 2 criterion n criterion 1 11 a 12a 1n a criterion 2 21 a 22a 2n a criterion n 1n a 2n a nna table 2 pairwise comparisons for the reliability of the restriction of the criteria criterion 1 criterion 2 criterion n criterion 1 11 r 12 r 1n r criterion 2 21 r 22 r 2 n r criterion n 1n r 2n r nn r step 2: the pairwise comparison matrices are then aggregated using the following geometric mean:   1 1 n n i ij j g            (5) ijahp article: alam, ku khalif, jaini/analytic hierarchy process based on the magnitude of znumbers international journal of the analytic hierarchy process 7 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1063 where  denotes any arbitrary triangular fuzzy number that satisfies definition 2. note that the aggregated mean is a triangular fuzzy number. step 3: the fuzzy weights representing the restriction and reliability components are calculated. the aggregated triangular fuzzy numbers are summed using the formula below.     1 n ig i s g     (6) for each criterion 1, 2,...,i n , the fuzzy weights are calculated using the formula below.     1 i ig n s g       (7) step 4: calculate the magnitude of the restriction and reliability components, as follows:    1 2 3 1 10 12i mag n n n n     (8) such that  1 2 3, ,in n n n  . this formula is obtained from definition 4, in which the trapezoidal fuzzy numbers are assumed as triangular fuzzy numbers. step 5: calculate the weight of each criterion by combining the magnitudes of the restriction and reliability components using the convex compound, as follows:        1 ii i ra w c mag n mag n    (9) step 6: the weight is finally normalized using the following formula:       i i n i i w c w c w c   (10) such that   1 1 n i i w c   . step 7: the criteria are ranked based on the normalized weights. the above steps of the proposed ahp model based on the magnitude of z-numbers can be summarized as shown in figure 2. ijahp article: alam, ku khalif, jaini/analytic hierarchy process based on the magnitude of znumbers international journal of the analytic hierarchy process 8 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1063 figure 2 proposed ahp based on the magnitude of z-numbers 5. criteria ranking for the prioritization of public services the case study from sergi and sari (2021) regarding the ranking of criteria to prioritize public services for digitalization was adopted to illustrate the proposed ahp based on the magnitude of z-numbers. figure 3 below summarizes the goal and criteria considered in the decision-making problem. figure 3 goal and criteria of the decision-making problem (sergi & sari, 2021) step 1: the decision maker evaluates the criteria using the pairwise comparison matrix. for the model to be able to handle the partially reliable information, two pairwise comparison matrices are constructed, one each for the restriction and reliability of the decision maker’s preferences as shown in tables 3 and 4, respectively. table 3 pairwise comparisons for the restriction of the criteria c1 c2 c3 c4 c5 c6 c1 ei rwi rmi rmi gi wi c2 wi ei rwi rwi gi mi c3 mi wi ei wi ai mi c4 mi wi rwi ei gi mi c5 rgi rgi rai rgi ei rwi c6 rwi rmi rmi rmi wi ei ijahp article: alam, ku khalif, jaini/analytic hierarchy process based on the magnitude of znumbers international journal of the analytic hierarchy process 9 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1063 table 4 pairwise comparisons for the reliability of the criteria c1 c2 c3 c4 c5 c6 c1 ar vwr fr fr vhr vhr c2 vhr ar vwr vwr vhr fr c3 fr vhr ar vhr sr fr c4 fr vhr vwr ar vhr fr c5 vwr vwr su vwr ar vwr c6 vwr fr fr fr vhr ar the decision maker’s opinion in natural language is transformed into z-numbers, in which the restriction and reliability matrices are converted into triangular fuzzy numbers using the linguistic values as shown in tables 5 and 6, respectively. table 5 linguistic values for the restriction matrix (sergi & sari, 2021) linguistic term triangular fuzzy numbers equally important (ei) (1,1,1) weakly important (wi) (1,3,5) moderately important (mi) (3,5,7) greatly important (gi) (5,7,9) absolutely important (ai) (7,9,9) reciprocal weakly important (rwi) (1/5,1/3,1) reciprocal moderately important (rmi) (1/7,1/5,1/3) reciprocal greatly important (rgi) (1/9,1/7,1/5) reciprocal absolutely important (rai) (1/9,1/9,1/7) table 6 linguistic values for the reliability matrix (sergi & sari, 2021) linguistic terms triangular fuzzy numbers absolutely reliable (ar) (1.0,1.0,1.0) strongly reliable (sr) (0.7,0.8,0.9) very highly reliable (vhr) (0.6,0.7,0.8) highly reliable (hr) (0.5,0.6,0.7) fairly reliable (fr) (0.4,0.5,0.6) weakly reliable (wr) (0.3,0.4,0.5) very weakly reliable (vwr) (0.2,0.3,0.4) strongly unreliable (su) (0.1,0.2,0.3) absolutely unreliable (au) (0.0,0.1,0.2) step 2: the pairwise comparison matrices for the restriction and reliability components are then aggregated using equation 5. table 7 below presents the aggregated triangular fuzzy numbers representing the restriction and reliability components for each criterion. ijahp article: alam, ku khalif, jaini/analytic hierarchy process based on the magnitude of znumbers international journal of the analytic hierarchy process 10 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1063 table 7 aggregated pairwise comparison matrix criterion restriction part reliability part c1 (0.523,0.809,1.308) (0.508,0.605,0.698) c2 (0.918,1.506,2.608) (0.485,0.583,0.677) c3 (1.995,3.557,4.718) (0.586,0.679,0.769) c4 (1.442,2.365,3.608) (0.508,0.605,0.698) c5 (0.177,0.218,0.323) (0.305,0.415,0.515) c6 (0.289,0.447,0.755) (0.475,0.572,0.665) step 3: the aggregated triangular fuzzy numbers for the restriction and reliability parts from table 7 are then summed. hence, its inverse is calculated using definition 2. table 8 summation of aggregated pairwise comparison matrix and its inverse restriction part reliability part summation (5.344,8.902,13.32) (2.868,3.459,4.021) inverse (0.075,0.112,0.187) (0.249,0.289,0.349) subsequently, the fuzzy weights are calculated using equation 7. the fuzzy weights obtained are shown in table 9. table 9 fuzzy weights for all criteria criterion restriction part reliability part c1 (0.039,0.091,0.245) (0.126,0.175,0.243) c2 (0.069,0.169,0.488) (0.121,0.169,0.236) c3 (0.150,0.400,0.883) (0.146,0.196,0.268) c4 (0.108,0.266,0.675) (0.126,0.175,0.243) c5 (0.013,0.025,0.061) (0.076,0.120,0.180) c6 (0.022,0.050,0.141) (0.118,0.165,0.232) step 4: the magnitude of each of the restriction and reliability parts of z-numbers is calculated using equation 8. the magnitudes are presented in table 10. table 10 magnitude of triangular fuzzy numbers criterion restriction part reliability part c1 0.0994 0.1766 c2 0.1874 0.1701 c3 0.4190 0.1981 c4 0.2867 0.1766 c5 0.0266 0.1213 c6 0.0554 0.1670 ijahp article: alam, ku khalif, jaini/analytic hierarchy process based on the magnitude of znumbers international journal of the analytic hierarchy process 11 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1063 step 5: the weight of each criterion is then determined by calculating the magnitude of the z-number representing each criterion, in which the magnitudes of the restriction and reliability components are combined using equation 9. table 11 below presents the weights of the criteria using several  values. table 11 criteria weights criterion  0.5 0.6 0.7 0.8 0.9 1.0 c1 0.1380 0.1302 0.1225 0.1148 0.1071 0.0994 c2 0.1788 0.1805 0.1822 0.1839 0.1857 0.1874 c3 0.3085 0.3306 0.3527 0.3748 0.3969 0.4190 c4 0.2316 0.2426 0.2536 0.2646 0.2757 0.2867 c5 0.0740 0.0645 0.0550 0.0455 0.0361 0.0266 c6 0.1112 0.1001 0.0889 0.0777 0.0666 0.0554 step 6: using equation 10, the criteria weights from table 11 are subsequently normalized, as shown in table 12. table 12 normalized criteria weights criterion  0.5 0.6 0.7 0.8 0.9 1.0 c1 0.1324 0.1242 0.1161 0.1082 0.1003 0.0925 c2 0.1716 0.1721 0.1727 0.1733 0.1739 0.1744 c3 0.2961 0.3153 0.3343 0.3531 0.3716 0.3900 c4 0.2223 0.2314 0.2404 0.2493 0.2581 0.2668 c5 0.0710 0.0615 0.0521 0.0429 0.0338 0.0247 c6 0.1067 0.0954 0.0843 0.0732 0.0624 0.0516 step 7: finally, the criteria are ranked based on the normalized weights. for all values of  , the criteria are ranked as 3 4 2 1 6 5c c c c c c . 6. discussion table 13 compares the ranking of the criteria obtained using the proposed model with the ranking obtained by sergi and sari (2021). the parameter 0.5  was used to highlight equal roles of the restriction and reliability components in representing the z-numbers. in fact, the proposed method validates the criteria ranking using multiple  values, satisfying the fact that the z-numbers are majorly represented by the restriction component. ijahp article: alam, ku khalif, jaini/analytic hierarchy process based on the magnitude of znumbers international journal of the analytic hierarchy process 12 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1063 table 13 comparison of criteria ranking with the existing method criterion proposed ( 0.5  ) proposed ( 1.0  ) sergi and sari (2021) weight ranking weight ranking weight ranking c1 0.1324 4 0.0925 4 0.094 4 c2 0.1716 3 0.1744 3 0.172 3 c3 0.2961 1 0.3900 1 0.399 1 c4 0.2223 2 0.2668 2 0.264 2 c5 0.0710 6 0.0247 6 0.019 6 c6 0.1067 5 0.0516 5 0.052 5 the proposed model produced the same ranking ( 3 4 2 1 6 5 c c c c c c ) as the existing method (sergi & sari, 2021) for all values of . however, the weights were almost similar when the value of  was 1.0. for the case of 1.0,  the reliability part was completely omitted from the z-numbers, thus reducing the model to a regular fuzzy ahp, which is unable to handle the partially reliable decision information. when the weights obtained from the proposed method were integrated with the zwaspas model from sergi and sari (2021), the same ranking of alternatives was obtained. table 14 below displays the sensitivity analysis results when the parameter from the utility function, which controls the weightage of the weighted sum and weighted product models, was changed from 0 to 1. table 14 sensitivity analysis for the z-waspas model alternative parameter in utility function 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 a1 0.212 0.212 0.211 0.211 0.211 0.209 0.209 0.208 0.208 0.207 0.207 a2 0.123 0.123 0.124 0.124 0.124 0.125 0.125 0.126 0.126 0.127 0.127 a3 0.154 0.153 0.153 0.153 0.153 0.152 0.152 0.152 0.152 0.151 0.151 a4 0.155 0.155 0.155 0.156 0.156 0.156 0.156 0.157 0.157 0.157 0.157 a5 0.187 0.187 0.186 0.186 0.186 0.185 0.185 0.184 0.184 0.184 0.183 a6 0.169 0.170 0.170 0.171 0.171 0.172 0.172 0.173 0.173 0.174 0.174 in reference to table 14, the weights obtained from the proposed method produced a consistent ranking of alternatives ( 1 5 6 4 3 2 a a a a a a ) when applied to the zwaspas model from sergi and sari (2021).therefore, this sensitivity analysis has shown that the proposed method is applicable for determining the weights of criteria for the application in multi-criteria decision-making problems. both the z-ahp model from sergi and sari (2021) and the current study implemented buckley’s fuzzy ahp in which the geometric mean to aggregate the pairwise comparison matrix was used. however, the ahp model proposed by sergi and sari (2021) transformed z-numbers into regular fuzzy numbers by defuzzifying the reliability parts of the z-numbers using a defuzzification formula defined by tüysüz and kahraman (2020). the defuzzified reliability parts were then added to the restriction parts. it should be ijahp article: alam, ku khalif, jaini/analytic hierarchy process based on the magnitude of znumbers international journal of the analytic hierarchy process 13 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1063 noted that the transformation of z-numbers into regular fuzzy numbers has caused a significant loss of information (abdullahi et al., 2020; gardashova, 2019; shen & wang, 2018), especially since the original decision information in the form of z-numbers was not preserved. on the other hand, the proposed ahp in the current study keeps the znumbers in the form of paired fuzzy numbers, which are mainly in their original form, except for the quantification of their magnitudes to represent the criteria weights. therefore, the implementation of the proposed ahp method in this study has addressed the issue of information loss since the nature of the expert’s preferences is kept as the restriction and reliability components representing the linguistic evaluation of criteria. the magnitude of z-numbers was also integrated to determine the final priority weights, which combines the restriction and reliability parts in the later step instead of converting z-numbers into regular fuzzy numbers from the beginning. moreover, the determination of criteria weight is important since it affects the final ranking of alternatives. in fact, the ranking of alternatives will not be affected as much when a sensitivity analysis is performed and consistent criteria weights are obtained. the criteria weights obtained using the proposed method were embedded in the z-waspas model from sergi and sari (2021) to produce a better ranking of alternatives. table 15 displays the final score values of z-ahp-waspas (sergi & sari, 2021) when each criterion weight was increased by 40%. table 15 sensitivity analysis for the z-ahp-waspas model (sergi & sari, 2021) alternative criterion with increased weight by 40% c1 c2 c3 c4 c5 c6 a1 0.209 1 0.204 1 0.205 1 0.211 1 0.207 1 0.206 1 a2 0.125 6 0.131 6 0.121 6 0.135 6 0.128 6 0.128 6 a3 0.151 5 0.148 5 0.152 5 0.156 4 0.152 5 0.152 5 a4 0.160 4 0.162 4 0.171 4 0.155 5 0.161 4 0.160 4 a5 0.180 2 0.182 2 0.172 3 0.179 2 0.179 2 0.180 2 a6 0.176 3 0.173 3 0.180 2 0.165 3 0.172 3 0.173 3 the ranking of alternatives was changed when the weights of c3 and c4 were increased by 40%, making the performance of the z-ahp-waspas model from sergi and sari (2021) 66.67%. the performance increased to 83.33% when the z-waspas model (sergi & sari, 2021) was integrated with the proposed z-ahp model, which utilized the magnitude of z-numbers in determining the criteria weights instead of conversion into regular fuzzy numbers. the integrated model maintains its consistency, except when the weight of c4 was increased by 40%. the score values for the integrated model are shown in table 16. ijahp article: alam, ku khalif, jaini/analytic hierarchy process based on the magnitude of znumbers international journal of the analytic hierarchy process 14 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1063 table 16 sensitivity analysis for the proposed z-ahp and z-waspas models (sergi & sari, 2021) alternative criterion with increased weight by 40% c1 c2 c3 c4 c5 c6 a1 0.212 1 0.206 1 0.208 1 0.212 1 0.211 1 0.208 1 a2 0.121 6 0.128 6 0.121 6 0.131 6 0.124 6 0.125 6 a3 0.151 5 0.149 5 0.153 5 0.155 4 0.153 5 0.154 5 a4 0.154 4 0.157 4 0.163 4 0.152 5 0.157 4 0.154 4 a5 0.186 2 0.188 2 0.179 2 0.185 2 0.185 2 0.187 2 a6 0.176 3 0.172 3 0.176 3 0.165 3 0.170 3 0.172 3 7. conclusion the implementation of z-numbers in any mcdm method must consider the preservation of the restriction and reliability components to avoid the loss of decision information. the magnitude of z-numbers was integrated with the ahp to produce a consistent criteria ranking. in the proposed model, the z-numbers were not converted into regular fuzzy numbers because the transformation causes a loss of information. hence, the restriction and reliability components of z-numbers were combined using the magnitude formula to determine the priority weights. this method not only preserves the initial information in the form of z-numbers, but also simplifies the calculation involving z-numbers since the meaning of the magnitude of fuzzy numbers is visual and natural. however, since this study is limited to criteria ranking using the proposed ahp model, there is a need to integrate the ahp model with other mcdm methods such as topsis or vikor to assist decision-makers in ranking the alternatives. it is important to preserve the znumbers when integrating these mcdm methods so that the loss of information can be avoided. ijahp article: alam, ku khalif, jaini/analytic hierarchy process based on the magnitude of znumbers international journal of the analytic hierarchy process 15 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1063 references abbasbandy, s., & hajjari, t. (2009). a new approach for ranking of trapezoidal fuzzy numbers. computers and mathematics with applications, 57(3), 413–419. doi: https://doi.org/10.1016/j.camwa.2008.10.090 abdullahi, m., ahmad, t., & ramachandran, v. (2020). a review on some arithmetic concepts of z-number and its application to real-world problems. international journal of information technology and decision making, 19(4), 1091–1122. doi: https://doi.org/10.1142/s0219622020300025 aliev, r. a. (2013). fundamentals of the fuzzy logic-based generalized theory of decisions. springer-verlag. aliev, r. a., alizadeh, a. v. v., & huseynov, o. h. h. (2015). the arithmetic of discrete z-numbers. information sciences, 290(c), 134–155. doi: https://doi.org/10.1016/j.ins.2014.08.024 aliev, r. a., guirimov, b. g., huseynov, o. h., & aliyev, r. r. (2021). a consistencydriven approach to construction of z-number-valued pairwise comparison matrices. iranian journal of fuzzy systems, 18(4), 37–49. azadeh, a., saberi, m., atashbar, n. z., chang, e., & pazhoheshfar, p. (2013). z-ahp: a z-number extension of fuzzy analytical hierarchy process. ieee international conference on digital ecosystems and technologies, 141–147. doi: https://doi.org/10.1109/dest.2013.6611344 buckley, j. j. (1985). fuzzy hierarchical analysis. fuzzy sets and systems, 17(3), 233– 247. doi: https://doi.org/10.1016/0165-0114(85)90090-9 buckley, james j., feuring, t., & hayashi, y. (2001). fuzzy hierarchical analysis revisited. european journal of operational research, 129(1), 48–64. doi: https://doi.org/10.1016/s0377-2217(99)00405-1 chang, d.-y. (1996). applications of the extent analysis method on fuzzy ahp. european journal of operational research, 95(3), 649–655. doi: https://doi.org/10.1016/0377-2217(95)00300-2 emrouznejad, a., & ho, w. (2017). fuzzy analytic hierarchy process. chapman and hall/crc. doi: https://doi.org/10.1201/9781315369884 farzam, m., kermani, m. a., allahviranloo, t., & belaghi, m. j. s. (2021). a new method for ranking of z-numbers based on magnitude value. in progress in intelligent decision science. ids 2020. advances in intelligent systems and computing (pp. 841– 850). springer, cham. doi: https://doi.org/10.1007/978-3-030-66501-2_68 gardashova, l. a. (2019). z-number based topsis method in multi-criteria decision making. in advances in intelligent systems and computing (vol. 896). springer international publishing. doi: https://doi.org/10.1007/978-3-030-04164-9_10 ijahp article: alam, ku khalif, jaini/analytic hierarchy process based on the magnitude of znumbers international journal of the analytic hierarchy process 16 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1063 kang, b., wei, d., li, y., & deng, y. (2012). a method of converting z-number to classical fuzzy number. journal of information and computational science, 9(3), 703– 709. leung, l. c., & cao, d. (2000). on consistency and ranking of alternatives in fuzzy ahp. european journal of operational research, 124(1), 102–113. https://doi.org/10.1016/s0377-2217(99)00118-6 perote-peña, j., & piggins, a. (2007). strategy-proof fuzzy aggregation rules. journal of mathematical economics, 43(5), 564–580. doi: https://doi.org/10.1016/j.jmateco.2006.09.010 ruoning, x., & xiaoyan, z. (1992). extensions of the analytic hierarchy process in fuzzy environment. fuzzy sets and systems, 52(3), 251–257. doi: https://doi.org/10.1016/01650114(92)90236-w saaty, t. l. (1980). the analytic hierarchy process. mcgraw-hill. sergi, d., & sari, u. i. (2021). prioritization of public services for digitalization using fuzzy z-ahp and fuzzy z-waspas. complex & intelligent systems, 7(2), 841–856. doi: https://doi.org/10.1007/s40747-020-00239-z shen, k. w., & wang, j. q. (2018). z-vikor method based on a new comprehensive weighted distance measure of z-number and its application. ieee transactions on fuzzy systems, 26(6), 3232–3245. doi: https://doi.org/10.1109/tfuzz.2018.2816581 tüysüz, n., & kahraman, c. (2020). codas method using z-fuzzy numbers. journal of intelligent & fuzzy systems, 38(2), 1649–1662. doi: https://doi.org/10.3233/jifs-182733 vaidya, o. s., & kumar, s. (2006). analytic hierarchy process: an overview of applications. european journal of operational research, 169(1), 1–29. doi: https://doi.org/10.1016/j.ejor.2004.04.028 van laarhoven, p. j. m., & pedrycz, w. (1983). a fuzzy extension of saaty’s priority theory. fuzzy sets and systems, 11(1–3), 229–241. doi: https://doi.org/10.1016/s01650114(83)80082-7 voskoglou, m. (2019). application of fuzzy numbers to assessment processes. in k.-p. mehdi (ed.), encyclopedia of information science and technology (pp. 407–420). igi global. doi: https://doi.org/10.4018/978-1-5225-7368-5.ch030 wang, y. m., elhag, t. m. s., & hua, z. (2006). a modified fuzzy logarithmic least squares method for fuzzy analytic hierarchy process. fuzzy sets and systems, 157(23), 3055–3071. doi: https://doi.org/10.1016/j.fss.2006.08.010 wang, y. m., luo, y., & hua, z. (2008). on the extent analysis method for fuzzy ahp and its applications. european journal of operational research, 186(2), 735–747. doi: https://doi.org/10.1016/j.ejor.2007.01.050 ijahp article: alam, ku khalif, jaini/analytic hierarchy process based on the magnitude of znumbers international journal of the analytic hierarchy process 17 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1063 xu, r. (2000). fuzzy least-squares priority method in the analytic hierarchy process. fuzzy sets and systems, 112(3), 395–404. doi: https://doi.org/10.1016/s01650114(97)00376-x zadeh, l. a. (1965). fuzzy sets. information and control, 8(3), 338–353. doi: https://doi.org/10.1016/s0019-9958(65)90241-x zadeh, l. a. (2011). a note on z-numbers. information sciences, 181(14), 2923–2932. doi: https://doi.org/10.1016/j.ins.2011.02.022 zeinalova, l. m. (2018). a z-number valued analytical hierarchy process. chemical technology, control and management, 2018(3), 88–94. doi: https://doi.org/https://doi.org/10.34920/2018.4-5.88-94 zhang, x., ma, w., & chen, l. (2014). new similarity of triangular fuzzy number and its application. the scientific world journal, 2014. doi: https://doi.org/10.1155/2014/215047 ijahp article: de souza, costa, de aruajo/prioritizing criteria to evaluate project success: modeling with the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 1 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.913 prioritizing criteria to evaluate project success: modeling with the analytic hierarchy process (ahp) luciano azevedo de souza universidade federal fluminense (uff) brazil lucianos@id.uff.br helder gomes costa universidade federal fluminense (uff) brazil helder.hgc@gmail.com hernando oliveira de araujo universidade federal fluminense (uff) brazil fernandoaraujo@id.uff.br abstract despite numerous attempts to systematize the evaluation of project success, the topic remains unaddressed, mainly because of the lack of appropriate models for dealing with the subjectivity associated with evaluation. this paper aims to contribute to this discussion by proposing a model for determining the relative importance of the criteria based on a multi-criteria technique (ahp). a core feature of the ahp is determining the relative weights of the criteria, considering the subjectivity associated with the problem. the proposed model was applied to a set of data collected through structured interviews from a sample of 54 respondents consisting of managers and project professionals in a given organization. the criteria with the highest priorities were 'learning opportunities' (20.4%), 'scope' (15.8%) and 'innovation' (14.1%). unexpectedly, the criteria ‘cost’, ‘schedule’, and ‘scope’, although widely used in evaluating success, did not rank as most important. this proposed prioritization can be useful to top management when making decisions about the application of resources that contribute to the success of the projects in the organization, as well as to guide project managers as they decide what actions are necessary to address the most relevant aspects in the context of the organization. keywords: project performance; project success criteria; project success; multicriteria decision analysis (mcda); analytic hierarchy process (ahp) 1. introduction one of the problems faced by organizational leaders is how to evaluate the success of their projects. it is necessary for leaders to be able to determine the relative importance of the factors or criteria that influence the success of projects. despite existing efforts to mailto:lucianos@id.uff.br mailto:helder.hgc@gmail.com mailto:fernandoaraujo@id.uff.br ijahp article: de souza, costa, de aruajo/prioritizing criteria to evaluate project success: modeling with the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 2 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.913 address this problem, this question remains, mainly due to the lack of adequate models to deal with the subjectivity present in evaluation. there are several published works that discuss establishing criteria weights in the context of project evaluation including da silva et al. (2021), roy, das, kar and pamučar (2019), šenitková, burdová and vilčeková (2010), yi, li and zhang (2019) and zagonari (2016). however, despite the great advances achieved in these studies, most of them do not use modeling based on trade-offs for the treatment of the data collected regarding the importance of criteria. it also appears that these works are not based on specific techniques that recognize and address subjectivity, despite the existence of multicriteria decision making (mcdm) which is specifically developed for decisions in subjective environments. according to keeney and raiffa (1993) and saaty (1980), eliciting weights for the importance of criteria involves the use of perception discovery methods based on trade-off administration. this work fills the gap in the literature by proposing and applying the analytic hierarchy process (ahp), which is a mcdm method supported by a trade-off algorithm to obtain the relative importance of the weights of the project evaluation criteria. this is based on a multicriteria technique that has at its core the attribution of relative weights of criteria considering the subjectivity inherent to the problem (rocha, barros, silva & costa, 2016). in general, this study answers the following research questions:  what are the criteria for evaluating success in projects reported in the scientific literature?  what is the relative importance attributed to each criterion found in the literature from the perspective of project management specialists, executives and others involved in the projects? in order to answer these questions, some procedures were performed, and the methods applied are described in section 2. in section 3, we present the application of the ahp approach. in section 4, we discuss the results, the practical applications, and limitations of the study, and finally, section 5 presents the conclusion with suggestions of further studies. 2. methods this study was organized in the following stages: literature review, (project success criteria), construction of the data collection instrument, data collection, construction of criteria, hierarchy of criteria, pairwise judgment of value from the data obtained in the survey, evaluation of the consistency of the pairwise evaluations, and prioritization analysis (from eigenvectors). these stages are presented in figure 1. ijahp article: de souza, costa, de aruajo/prioritizing criteria to evaluate project success: modeling with the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 3 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.913 figure 1 procedure flow 2.1 literature review to further understand project evaluation criteria, bibliographic research was carried out through a literature review of papers published in journals indexed in scopus or in web of sciences (wos). according to rodriguez, costa and carmo (2013), these selection criteria avoid the use of non-peer reviewed articles and reduce the probability of taking into account the so called “gray literature” (rothstein & hopewell, 2009) or even predatory sources. 2.2 construction of the data collection instrument to understand the degree of application of each of the criteria identified in the literature, respondents were asked to choose the alternative that would best represent their opinion regarding the use of these criteria in the organization according to the following scale: (1) strongly disagree; (2) disagree; (3) neither agree / disagree; (4) agree; (5) strongly agree. 2.3 data collection based on the theoretical findings, the survey was carried out by direct questioning of those involved in project management in the organization by asking about their experience in the organization related to project performance. the questionnaires contained objective questions about their experience with projects in the organization, specifically about the criteria for evaluating their performance in the projects. 2.4 prioritizing the criteria the ahp was applied to assign weights to each criterion using the eigenvector. as reported in costa and correa (2010) and rocha, barros, silva and costa (2016), the 1.literature review 2.(project success criteria) 1.construction of the data collection instrument data collection 1.construction of criteria hierarchy of criteria 1.pairwise judgment of value from the data obtained in the survey 1.evaluation the degree of consistency 1.prioritization analysis ijahp article: de souza, costa, de aruajo/prioritizing criteria to evaluate project success: modeling with the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 4 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.913 choice of this method is justified when the problem of weight assignment is a tradeoff problem. the ahp algorithm is adequate to deal with this type of problem, where the preference for one criterion leads to the substitution of another with lesser preference. as reported in saaty (1980), when building a model based on ahp, the following steps are applied:  build the hierarchy of criteria  perform the pairwise comparison of criteria according to their relevance regarding the goal  run the ahp prioritization algorithm  evaluate the consistency of the pairwise evaluations  perform a sensitivity analysis 3. an ahp approach to evaluate the criteria prioritization this section shows the outcome of the steps described in the methods section. for better understanding, it has been organized according to the same structure as the methods section. 3.1 criteria definitions in this section, the criteria collected during the literature review and data collection procedure are presented. 3.1.1 criteria collected from the literature review the theme “project success evaluation criteria” was searched in scopus and web of science databases from april 10, 2018 to april 16, 2018. table 1 presents the criteria, a short description of each criterion and references where each criterion was cited. ijahp article: de souza, costa, de aruajo/prioritizing criteria to evaluate project success: modeling with the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 5 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.913 table 1 criteria for evaluating success in projects criterion short description references  schedule  carried out within the initially established period  pinto and slevin (1988), shenhar et al. (1997), de wit (1988), baccarini (1999), judge and müller (2005), abdulah et al. (2010), serrador and turner (2015), albert et al. (2017), koops et al. (2017), pollack et al. (2018), redda and turner (2018)  cost  does not exceed the established budget limits  pinto and slevin (1988), shenhar et al. (1997), de wit (1988), baccarini (1999), judge and müller (2005), andersen et al. (2006), abdulah et al. (2010), serrador and turner (2015), albert et al. (2017), koops et al. (2017), pollack et al. (2018), redda and turner (2018)  scope  carried out in the scope, technical performance and expected quality, according to prescribed requirements and unstated expectations.  pinto and slevin (1988), shenhar et al. (1997), de wit (1988), atkinson (1999), baccarini (1999), judge and müller (2005), abdulah et al. (2010), serrador and turner (2015), albert et al. (2017), koops et al. (2017), pollack et al. (2018), redda and turner (2018)  efficiency  resources rationally applied  de wit (1988), atkinson (1999), baccarini (1999), judge and müller (2005), andersen et al. (2006), serrador and turner (2015)  mission/purpose  the output of the project fulfilled the desired purpose that motivated the realization of the project  pinto and slevin (1988), shenhar et al. (1997), atkinson (1999), baccarini (1999), judge and müller (2005), andersen et al. (2006), serrador and turner (2015), montes-guerra et al. (2015)  organizational benefits  contributes strategically to the organization  de wit (1988), atkinson (1999), baccarini (1999), judge and müller (2005), andersen et al. (2006), serrador and turner (2015), redda and turner (2018)  preparing for the future  creates conditions for continuous organizational development.  shenhar et al. (1997), serrador and turner (2015), redda and turner (2018)  stakeholders satisfaction  meets stakeholders expectations, including  pinto and slevin (1988), shenhar et al. (1997), de wit (1988), atkinson ijahp article: de souza, costa, de aruajo/prioritizing criteria to evaluate project success: modeling with the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 6 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.913 criterion short description references sponsor, end-user and project team, suppliers, etc. (1999), baccarini (1999), judge and müller (2005), andersen et al. (2006), serrador and turner (2015), albert et al. (2017), koops et al. (2017)  political and social aspects  meets motivating political and social expectations  montes-guerra et al. (2015), koops et al. (2017), redda and turner (2018)  legality and compliance  carried out respecting governance rules, observing legality and compliance requirements.  montes-guerra et al. (2015), koops et al. (2017), redda and turner (2018)  safety  carried out safely and offers security to the end user  koops et al. (2017), acheamfour et al. (2019)  sustainability  positively impacts the environment, society and finances.  koops et al. (2017), acheamfour et al. (2019), mansell and philbin (2020) 3.1.2 data collection all criteria identified in the bibliometric study were included in the form used by the researcher for direct contact with the participants of the survey. thus, the participant who was interviewed was asked to give information about the gradation used by the studied organization to evaluate projects using the scale (1) strongly disagree; (2) disagree; (3) neither agree / disagree; (4) agree and (5) strongly agree. the interview was done using a structured form printed for further typing by the researcher using google forms. the researcher contacted potential participants and then performed the interviews with those who agreed to participate during a personal meeting, where it was possible to discuss their questions. the questionnaire was applied over a sample of 54 employees of an organization from the health sector in brazil from august 27october 20, 2018. table 2 shows how the respondents were classified into three different groups according to their organizational role. this table also shows the estimated size of the population and the percentage of respondents. ijahp article: de souza, costa, de aruajo/prioritizing criteria to evaluate project success: modeling with the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 7 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.913 table 2 respondents group estimated population interviewed % a – executives 10 10 100 b project management specialists 6 5 83.3 c project managers or team members 100 39 39.0 ∑ 116 54 46.5 potential participants were contacted by phone, and the objective was explained and a contact with the researcher was scheduled when the terms of confidentiality of the data and purpose of the research were presented. during the interviews, which were performed by direct contact of the researcher with the respondents, some participants spontaneously reported that the organization did not have formal and clear criteria for determining the performance evaluation of projects. the information collected regarding the criteria used by the organization in a non-systematic and informal way can be found in figure 2. according to the respondents interviewed, the organization’s “legality and compliance”, “purpose and mission”, “organizational benefits” and “political and social factors” were perceived by the respondents as being more likely to be considered by the organization for evaluation; on the other hand, “efficiency, “project team satisfaction”, “stakeholder’s satisfaction” and “sustainability” were not perceived by the respondents to be considered. however, the aim of this paper is to determine the relative importance of each criterion using a method to organize the balance of trade-offs, and for this purpose, we employed the ahp method. ijahp article: de souza, costa, de aruajo/prioritizing criteria to evaluate project success: modeling with the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 8 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.913 figure 2 project evaluation criteria 3.2 construction of criteria hierarchy of criteria the ahp method considers pairwise comparisons; however, when there are a large number of criteria it is more difficult to evaluate the nuances between two criteria. thus, a hierarchy was created aggregating criteria in a structure with macro criteria and subcriteria. therefore, the 16 criteria identified in the literature were aggregated into four macro criteria (c1-iron triangle, c2-mean-related, c3-end-related and c4-related to the future). this proposed structure is shown in the figure 3. ijahp article: de souza, costa, de aruajo/prioritizing criteria to evaluate project success: modeling with the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 9 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.913 figure 3 project success criteria hierarchy next, a pairwise comparison was performed between the sub-criteria and then between the criteria. 3.3 pairwise judgment of value from the data obtained in the survey and evaluation of the degree of consistency the first judgment was performed by comparing the answers of the interviewed participants about cost, schedule, and scope. the results of this group of traditional project success criteria also named the “iron triangle” in the related professional literature are shown in the figure 4. figure 4 agreement grade of iron triangle criteria ijahp article: de souza, costa, de aruajo/prioritizing criteria to evaluate project success: modeling with the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 10 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.913 the importance between the facets of success in projects was evaluated in pairs from the data obtained in the empirical survey beginning with the sub-criteria that make up the socalled iron triangle, (cost, schedule and scope) using the scale of preferences (saaty, 1980). the obtained results are shown in the pairwise comparison matrix in table 3. table 3 pairwise comparison matrix: iron triangle the consistency ratio (cr) was 0.01, which is less than the recommended limit of 0.10 or 10% (saaty, 1980). the same procedure was used to evaluate the criteria that compose the group meanrelated criteria, which aggregates criteria that describe the way the project was done (efficiency, legality and compliance, safety, and sustainability). figure 5 shows the results from the interview related to this group. figure 5 agreement grade of mean-related criteria then, a comparison between the sub-criteria was performed and is shown in table 4. table 4 pairwise comparison matrix: mean-related criteria ijahp article: de souza, costa, de aruajo/prioritizing criteria to evaluate project success: modeling with the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 11 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.913 the consistency ratio (cr) was 0.09, which is less than the recommended limit of 0.10 or 10% (saaty, 1980). the end-related criteria are used as an aggregation of “meeting end user expectations”, “meeting political and social factors”, “meeting project team expectations”, “meeting purpose and mission of the project”, “meeting stakeholders’ expectations” and “production of organizational benefits”. this group of criteria are related to the efficacy of the project. the results can be seen in figure 6. figure 6 agreement grade end-related criteria the pairwise comparison of this group of criteria is shown in table 5. table 5 pairwise comparison matrix: end-related criteria the last group of criteria named “related to the future” groups criteria related to nonimmediate impact (learning opportunity, preparing for the future and promotion of innovation). the results are shown in figure 7. figure 7 agreement grade criteria related to the future ijahp article: de souza, costa, de aruajo/prioritizing criteria to evaluate project success: modeling with the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 12 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.913 the evaluation of the data collected from the interviewed participants about this variable is organized in table 6. ijahp article: de souza, costa, de aruajo/prioritizing criteria to evaluate project success: modeling with the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 13 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.913 table 6 pairwise comparison matrix: criteria related to the future the consistency ratio (cr) is 0.09, less than the recommended limit of 0.10 or 10% (saaty, 1980). after all the sub-criteria were evaluated, an evaluation of the criteria was performed. the results from the sub-criteria were summed to compose each top-level criterion as detailed in table 7. the plot compares the percentage frequencies in each criterion by using the likert scale, and the results are presented in figure 8. table 7 consolidation of sub-criteria for each criterion criteria / sub-criteria answers given c1 iron triangle 1 2 3 4 5 sc1.1 schedule 2 2 13 18 19 sc1.2 cost 1 4 10 26 13 sc1.3 scope 0 4 8 20 22  3 10 31 64 54 c2 mean-related 1 2 3 4 5 sc2.1 safety 0 5 13 19 17 sc2.2 legality and compliance 0 0 3 21 30 sc2.3 sustainability 0 8 13 20 13 sc2.4 efficiency 0 11 17 12 14  0 24 46 72 74 c3 end-related 1 2 3 4 5 sc3.1 purpose and mission 0 0 7 29 18 sc3.2 political and social factors 0 1 9 34 10 sc3.5 stakeholders’ satisfaction 1 6 15 26 6 sc3.6 organizational benefits 0 1 6 23 24  1 8 37 112 58 c4 related to the future 1 2 3 4 5 sc.4.1 preparing for the future 0 3 15 20 16 sc.4.2 innovation 0 2 11 21 20 sc.4.3 learning opportunity 1 2 10 24 17  1 7 36 65 53 ijahp article: de souza, costa, de aruajo/prioritizing criteria to evaluate project success: modeling with the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 14 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.913 figure 8 agreement grade of success of project finally, the upper level of the ahp hierarchy was evaluated by aggregating all the responses for each component as follows in table 8. table 8 criteria and sub-criteria criteria (upper level) sub-criteria (lower level) traditional criteria (iron triangle) cost schedule scope mean-related efficiency legality and compliance safety sustainability end-related meeting political and social factors meeting purpose and mission of the project meeting stakeholders’ expectations production of organizational benefits future-related learning opportunity preparing for the future promotion of innovation these data were used to construct the last pairwise comparison matrix shown in table 9. table 9 pairwise comparison matrix: project success the consistency ratio (cr) is 0.03, less than the recommended limit of 0.10 or 10% (saaty, 1980). ijahp article: de souza, costa, de aruajo/prioritizing criteria to evaluate project success: modeling with the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 15 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.913 the pairwise comparisons are now completed, and it is possible to identify the weights supplied by the ahp method. 3.4 prioritization analysis the results are consolidated in table 10 which shows the relative importance (weight) assigned using the eigenvector of the previous pairwise matrices, and the global weight is obtained by the multiplication of each sub criteria by the upper-level criteria, thus providing the relative importance ranking of each. table 10 relative importance (weights) of criteria sub criteria weight global weight c1 iron triangle 29.2% sc1.1 schedule 16.3% 4.8% sc1.2 cost 29.7% 8.7% sc1.3 scope 54.0% 15.8% c2 mean-related 10.8% sc2.1 safety 16.0% 1.7% sc2.2 legality and compliance 65.8% 7.1% sc2.3 sustainability 12.3% 1.3% sc2.4 efficiency 5.9% 0.6% c3 end-related 18.7% sc3.1 purpose and mission 45.1% 8.4% sc3.2 political and social factors 17.3% 3.2% sc3.5 stakeholders satisfaction 3.6% 0.7% sc3.6 organizational benefits 27.0% 5.1% c4 related to the future 41.3% sc.4.1 preparing for the future 16.4% 6.8% sc.4.2 innovation 34.2% 14.1% sc.4.3 learning opportunity 49.3% 20.4% the data in table 10 communicates the goal of this paper which is to elicit the relative importance of each criterion through the application of the ahp, a method that is able to deal with subjectivity, from the results of the empirical study at the selected organization. table 11 shows a list of criteria with their respective relative importance between parenthesis in descending order. ijahp article: de souza, costa, de aruajo/prioritizing criteria to evaluate project success: modeling with the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 16 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.913 table 11 project success criteria ordered by relative importance 4. discussion, practical applications, and limitations one important consequence of this method is that the results could be affected because the amount of sub-criteria for each criterion was not similar. this is because the relative importance of one criterion could be reduced at the product of the eigenvalues of the criteria and sub-criteria depending on the number of sub-criteria that compose the criterion in the hierarchy. thus, it is recommended that the structure be equilibrated. another observation of this study performed at a public organization is that the criterion “legality and compliance” shown in table 1 was the top ranked criterion; however, by using this method it was reduced in importance to the sixth position when the criterion “mean-related” received a lower relative importance because the other sub-criteria that compose it were rated low (efficiency, safety and sustainability). the practical implication of this work lies in the possibility of offering support for the most used criteria for the evaluation of project performance. in this way, it is possible to direct resources towards the criteria with greater sensitivity to project success. even though the study was carried out with a good sample of respondents, they all belong to the same organization that has a specific organizational culture, and may not, therefore, represent all organizations and their individual cultures and environments. it is also important to note that the researcher is connected to institution that was analyzed. 5. conclusion the main contribution of this work is to provide the relative weights of project success criteria that have been identified in the scientific literature using a method that allows the consistency of subjective evaluations to be validated. this contribution fulfills an existing sub criteria weight criterion weight final weight classification sc.4.3 learning opportunity 49.3% c4 related to the f uture 41.3% 20.4% 1 sc1.3 scope 54.0% c1 iron triangle 29.2% 15.8% 2 sc.4.2 innovation 34.2% c4 related to the f uture 41.3% 14.1% 3 sc1.2 cost 29.7% c1 iron triangle 29.2% 8.7% 4 sc3.1 purpose and mission 45.1% c3 end-related 18.7% 8.4% 5 sc2.2 legality and compliance 65.8% c2 mean-related 10.8% 7.1% 6 sc.4.1 preparing f or the f uture 16.4% c4 related to the f uture 41.3% 6.8% 7 sc3.6 organizational benef its 27.0% c3 end-related 18.7% 5.1% 8 sc1.1 schedule 16.3% c1 iron triangle 29.2% 4.0% 9 sc3.2 political and social f actors 17.3% c3 end-related 18.7% 3.2% 10 sc2.1 saf ety 16.0% c2 mean-related 10.8% 1.7% 11 sc2.3 sustainability 12.3% c2 mean-related 10.8% 1.3% 12 sc3.5 stakeholders satisf action 3.6% c3 end-related 18.7% 0.7% 13 sc2.4 ef f iciency 5.9% c2 mean-related 10.8% 0.6% 14 ijahp article: de souza, costa, de aruajo/prioritizing criteria to evaluate project success: modeling with the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 17 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.913 gap in the field of a lack of adequate models to deal with the present subjectivity in this evaluation. we determined the relative importance of the criteria for evaluating success in projects using the ahp method considering the responses of the 54 participants who were interviewed as follows: learning opportunity (20.4%), scope (15.8%), innovation (14.1%), cost (8.7%), purpose and mission (8.4%), legality and compliance (7.1%), preparing for the future (6.8%), organizational benefits (5.1%), schedule (4.0%), political and social factors (3.2%), safety (1.7%), sustainability (1.3%), stakeholders’ satisfaction (0.7%), and efficiency (0.6%). we highlight the unexpected finding that despite the fact that the aspects of cost, schedule and scope are widely used in performance evaluation, they did not rank highly in the study. it is also important to highlight that the consistency ratio (cr) of all the evaluations was less than 10% which indicates a consistent result using the ahp method. for further study, we propose comparative studies with similar organizations in order to identify possible differences in success criteria prioritization between them. we also recommend extending this work by using the anp to see the relationships among the criteria. ijahp article: de souza, costa, de aruajo/prioritizing criteria to evaluate project success: modeling with the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 18 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.913 references abdullah, a.a., rahman, h.a., harun, z., alashwal, a.m., & beksin, a.m. (2010). literature mapping: a bird’s eye view on classification of factors influencing project success. african journal of business management, 4(19), 4174-4182. acheamfour, v.k., kissi, e., & adjei-kumi, t. (2019). ascertaining the impact of contractors pre-qualification criteria on project success criteria. engineering, construction and architectural management, 26(4), 618-632. doi: https://doi.org/10.1108/ecam-03-2018-0110 albert, m., balve, p., & spang, k. (2017). evaluation of project success: a structured literature review. international journal of managing projects in business, 10(4), 796821. doi: https://doi.org/10.1108/ijmpb-01-2017-0004 andersen, e.s., birchall, d., jessen, s.a., & money, a.h. (2006). exploring project success. baltic journal of management, 1(2), 127-147. doi: https://doi.org/10.1108/17465260610663854 atkinson, r. (1999). project management: cost, time and quality, two best guesses and a phenomenon, its time to accept other success criteria. international journal of project management, 17(6), 337-342. baccarini, d. (1999). the logical framework method for defining project success. the project management journal, 30(4), 25-32. costa, h.g., & correa, p. de s. (2010). construction of an ahp-based model to catch criteria weights in post-occupancy evaluation. international journal of the analytic hierarchy process, 2(1), 30-43. doi: 10.13033/ijahp.v2i1.71 da silva, f. f., souza, c. l. m., silva, f. f., costa, h. g., da hora, h. r. m., & erthal junior, m. (2021). elicitation of criteria weights for multicriteria models: bibliometrics, typologies, characteristics and applications. brazilian journal of operations and production management, 18(4). doi: https://doi.org/10.14488/bjopm.2021.014 de wit, a. de. (1988). measurement of project success. project management, 6(3), 165170. jugdev, k., & müller, r. (2005). a retrospective look at our evolving understanding of project success. project management journal, 36(4), 19–31. doi: https://doi.org/10.1177/875697280503600403 keeney, r.l., & raiffa, h. (1993). decisions with multiple objectives : preferences and value tradeoffs. cambridge university press. koops, l., van loenhout, c., bosch-rekveldt, m., hertogh, m., & bakker, h. (2017). different perspectives of public project managers on project success. engineering, https://doi.org/10.1108/ecam-03-2018-0110 https://doi.org/10.1108/ijmpb-01-2017-0004 https://doi.org/10.1108/17465260610663854 https://doi.org/10.14488/bjopm.2021.014 https://doi.org/10.1177/875697280503600403 ijahp article: de souza, costa, de aruajo/prioritizing criteria to evaluate project success: modeling with the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 19 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.913 construction and architectural management, 24(6), 1294-1318. doi: https://doi.org/10.1108/ecam-01-2015-0007 mansell, p., & philbin, s.p. (2020). measuring sustainable development goal targets on infrastructure projects. journal of modern project management, 8(1), 1-22. montes-guerra, m.i., de-miguel, a.r., pérez-ezcurdia, m.a., gimena ramos, f.n., & díez-silva, h.m. 2015. project management in development cooperation. nongovernmental organizations. rev. innovar, 25(56), 52-67. doi: http://dx.doi.org/10.15446/innovar.v25n56.48989 pinto, j. k., & slevin, d.p. (1988). project success: definitions and measurement techniques. newton square, pa: project management institute. pollack, j., helm, j., & adler, d. (2018). what is the iron triangle, and how has it changed?. international journal of managing projects in business, 11(2), 527-547. doi: https://doi.org/10.1108/ijmpb-09-2017-0107 redda, y., & turner, r. (2018). mapping the success dimensions of large oil and gas projects. in spe annual technical conference and exhibition. society of petroleum engineers. rocha, p.m.p.m., barros, a.p.d.a.p., silva, g.b.d.g.b., & costa, h.g. (2016). analysis of the operational performance of brazilian airport terminals: a multicriteria approach with de borda-ahp integration. journal of air transport management, 51, 19–26. doi: https://doi.org/10.1016/j.jairtraman.2015.11.003 rodriguez d.s.s., costa h.g., & carmo l.f.r.r.s. (2013). métodos de auxílio multicritério à decisão aplicados a problemas de pcp: mapeamento da produção em periódicos publicados no brasil. gest e prod, 20, 134–146. doi: https://doi.org/10.1590/s0104-530x2013000100010 rothstein, h.r., & hopewell, s. (2009). grey literature. in h. cooper et al. (eds.), the handbook of research synthesis and meta-analysis (pp. 103-125). new york: russel sage foundation. roy, j., das, s., kar, s., & pamučar, d. (2019). an extension of the codas approach using interval-valued intuitionistic fuzzy set for sustainable material selection in construction projects with incomplete weight information. symmetry, 11(3), 1-24. saaty, t.l. (1980). the analytic hierarchy process: planning, priority setting, resource allocation. mcgraw-hill international book. šenitková, i., burdová, e.k., & vilčeková, s. (2010). building environmental assessment system-weight determination. proceedings of cesb 2010 prague central europe towards sustainable building “from theory to practice”. prague: czech technical university. https://doi.org/10.1108/ecam-01-2015-0007 https://doi.org/10.1108/ijmpb-09-2017-0107 https://doi.org/10.1016/j.jairtraman.2015.11.003 https://doi.org/10.1590/s0104-530x2013000100010 ijahp article: de souza, costa, de aruajo/prioritizing criteria to evaluate project success: modeling with the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 20 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.913 serrador, p., & turner, r. (2015). the relationship between project success and project efficiency. project management journal, 46(1), 30-39. doi: https://doi.org/10.1002/pmj shenhar, a.j., levy, o., & dvir, d. (1997). mapping the dimensions of project success. project management journal, 28(2), 5-13. yi, p., li, w., zhang, d. (2019). assessment of city sustainability using mcdm with interdependent criteria weight. sustainability (switzerland), 11(6), 1-20. doi: https://doi.org/10.3390/su11061632 zagonari, f. (2016). choosing among weight-estimation methods for multi-criterion analysis: a case study for the design of multi-purpose offshore platforms. applied soft computing journal, 39, 1–10. doi: https://doi.org/10.1016/j.asoc.2015.11.003 https://doi.org/10.1002/pmj https://doi.org/10.3390/su11061632 https://doi.org/10.1016/j.asoc.2015.11.003 microsoft word 1bpreface_pp1_3_04172009 words from the editor‐in‐chief  welcome to the first issue of the international journal of the analytic hierarchy process (ijahp). the  mission of the ijahp is to advance the use and development of the analytic hierarchy/network process  (ahp/anp) for decision making and to educate people in using it.  the first issue of a journal always  raises the question of why we need it in the first place.  so i will use this opportunity to address this  issue.     informs, in its recent october 2008 impact award to thomas l. saaty, the creator of the ahp, stated  that “the ahp has revolutionized how we resolve complex decision problems.” although ahp/anp is  currently applied in diverse areas, and articles and papers on this subject are available in journals from  multiple disciplines; ahp/anp researchers, practitioners, and students lack a publication we can call our  own. we already have our biennial conference, the international symposium on the analytic hierarchy  process (isahp),  that constitutes an academic  home for practitioners of the ahp/anp discipline;  however, we need our own publication to address issues that pertain to the ahp/anp, and to keep our  community of practitioners connected in the periods between ahp/anp symposia. i believe this journal  can fill that vacuum and would constitute an effective complement to the isahp meetings. for this  reason, the editors have chosen, for this issue, three papers from the last isahp in chile that constitute  good examples of ahp/anp applications.  there is also an article by tom saaty on the relationship  between ahp/anp and natural law and the role of judgment in science.  i hope that you will enjoy our  first issue!    enrique mu  editor‐in‐chief  rob typewritten text http://dx.doi.org/10.13033/ijahp.v1i1.8 ijahp article: hadiono/a practical implementation of interval arithmetic in ahp international journal of the analytic hierarchy process 256 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.325 a practical implementation of interval arithmetic in ahp kristophorus hadiono 1 department of information technology vincent mary school of science and technology assumption university of thailand 592/3 ramkhamhaeng 24, hua mak, bang kapi, bangkok 10240, thailand e-mail: mail2kristophorus@gmail.com abstract one of the well-established methods that help the decision makers deal with multiple criteria is the analytical hierarchy process (ahp) which utilizes a weighting approach. the process of decision making with multiple criteria is faster if all the weights of the factors related to a particular problem are clearly stated. however, if the weights of said factors are not well defined, or only their lower and upper weight limits are known, then the decision makers face considerable uncertainty because the standard ahp numerical procedure operates with deterministic values. as a result, the corresponding assessment preferences cannot be expressed in the form of a sequence of numerical values and implemented in the ahp evaluation. a practical approach is presented in this work to deal with the data uncertainty by implementing interval arithmetic in the ahp calculations so that the assessment preferences are presented in the form of interval numbers. keywords: ahp; interval arithmetic; interval numbers; decision making 1. introduction decision making can be considered as a human brain process that occurs in almost every human activity. it can be categorized as one of several brain activities which are called cognitive processes which are known as brain activity associated with attention, memorization, language production and understanding, learning, and problem solving which includes decision making. decision making can be described as a process which is activated after having some initial information, and then analyzing this information using one's judgment in order to decide which option is the best from all the available options. the process of one’s 1 the author gratefully acknowledges the directorate general of higher education, ministry of education, republic of indonesia, which provides the bppln scholarships including the support of the stikubank university semarang. ijahp article: hadiono/a practical implementation of interval arithmetic in ahp international journal of the analytic hierarchy process 257 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.325 judgment typically uses several criteria for every available option. the weighting of each criterion for an option will cause each option to have a set of different weights. one can perform subsequent calculations on the basis of the weights that are available and obtain possible solutions. after further comparison, the best solution can be determined by applying some known criteria. the obtained result can be seen as the best solution from all available options. the decision makers can make proper decisions quickly only if the factor weights are clearly stated. however, if the upper and lower limits of these weights are known instead, then the decision making process faces obvious difficulties. human intuition can be used with relative success as even for a small number of options the resultant decisions could be unreliable. the upper and lower limits of the factor weights can be interpreted as an interval number. by definition, an interval number is a set of real numbers in which the members of the set lie between two limiting numbers. the concept of an interval number is the cornerstone of interval arithmetic. by implementing interval arithmetic into the decision making process, especially in the ahp technique, the uncertainty that occurs is expected to be taken into consideration. this raises a few adjustments in the weight calculation of decision making scenarios. the aim of this paper is to demonstrate the implementation of interval arithmetic in the decision making process, especially in the ahp technique. for that purpose, the structure of the paper is organized accordingly. the second section provides a literature review which covers recent developments in the theory of interval arithmetic in relation to the decision making process. the third section gives a description of the methodology for the implementation of interval arithmetic in the ahp technique and the obtained results are discussed in the last section. 2. literature review the decision making methods should be used to solve complex problems when the number of existing criteria or options are beyond the natural human processing ability. at present, there are numerous published studies presenting different aspects of decision making methods used for dealing with a wide range of such complex problems. some methods rely on giving weights for all criteria and options and subsequently processing the weights with a specific technique. one of the well-established techniques for decision making which is known as the analytical hierarchy process (ahp) was proposed by thomas lorie saaty (lane & verdini, 1989; saaty, 1990). the aim of the operation of a decision making technique is to reduce the uncertainty in the decision outcomes. the uncertainty results in hesitation among the decision makers when giving preferences during problem evaluation. there are two types of uncertainties related to decision making. the first is the uncertainty about the occurrence of events which cannot be controlled. the second is the set of judgment values for expressing the preferences that can be used by the decision makers which is related to the available information (saaty & vargas, 1987). obviously, as more reliable information is gathered, the better the assessment of the existing preferences will be. ijahp article: hadiono/a practical implementation of interval arithmetic in ahp international journal of the analytic hierarchy process 258 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.325 an overview of the decision making process using the fuzzy ahp technique, the theory of interval arithmetic, and the link between the ahp and interval arithmetic as a new proposed method is provided below. 2.1 fuzzy analytic hierarchical process the fuzzy analytic hierarchy process (fuzzy ahp) is another method that deals with uncertainty faced by the decision makers. the conventional ahp method is ineffective when applied to the uncertain nature or significance problems. as in the real world the uncertainty problems are many, some researcher extend the capability of conventional ahp with the fuzzy set theory to handle and overcome the limitation of the conventional ahp method (javanbarg, scawthorn, kiyono, & shahbodaghkhan, 2012). the benefit of combining fuzzy set theory with the conventional ahp is the capability of fuzzy set theory to represent uncertain or vague data in a natural form with the purpose to reduce the uncertainty for decision makers. although the fuzzy set theory is combined with ahp in order to expand the capability of conventional ahp, the main ideas of ahp are not changed. the fuzzy ahp still needs to derive the weight of the criterion and according to wang & chin (2011) the fuzzy ahp has two approaches to derive the weight. the first approach derives the weight of the criterion from a set of fuzzy weight pair-wise comparison matrices and the other approach uses a set of crisp weights from a fuzzy pair-wise comparison matrix. the first approach involves more than one method to get the weight of the criterion such as the geometric mean method, fuzzy logarithmic least-squares methods (llsm), lambda-max method, and the linear programming goal (lgp) method. the second method to get the weight of the criterion uses extent analysis and the fuzzy preference programming (fpp) which is based on nonlinear methods. wang & chin (2011) reported that it is not simple to calculate the weight of the criterion using the second approach which causes many researches to adopt the first approach or the simple extent analysis method for deriving fuzzy ahp weights. one example of the simple extent analysis method implementation for fuzzy ahp is by koul & verma (2012). though using the first approach or adopting the simple extent analysis, the fuzzy ahp still has some problems of inconsistency (wang & chin, 2011; koul & verma, 2012). to overcome the inconsistency, javanbarg et al. (2012) reported that the consistency check process will be incorporated in one step of the fuzzy ahp sequence. the fuzzy ahp method steps are like the conventional ahp method and those steps are: first, structuring the decision hierarchy; second, developing the pair-wise fuzzy comparison matrix. in this step, the triangular fuzzy numbers ),,(~ ijijijij umla  are used to develop the fuzzy reciprocal comparison matrix. the judgment scores for the fuzzy ahp that convert the decision maker’s judgments into scores are displayed in table 1. ijahp article: hadiono/a practical implementation of interval arithmetic in ahp international journal of the analytic hierarchy process 259 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.325 table 1 the judgment scores in fuzzy ahp (javanbarg et al. (2012)) fuzzy judgements fuzzy score about equal (1/2, 1, 2) about x times more important (x 1, x, x + 1) about x times less important (1/(x + 1), 1/x, 1/(x 1)) between y and z times more important (y, (y + z)/2, z) between y and z times less important (1/z, 2/(y + z), 1/y) note: x=2, 3, 9; y and z =1, 2, …, 9; y < z the third step conducts a consistency check and derives priorities. in the consistency check process, a fuzzy comparison matrix }~{ ~ ij aa  is called consistent if ijkjik aaa ~~~  where i, j, and k is 1, 2, …, n (  is fuzzy multiplication and  denotes fuzzy equal to). when the fuzzy comparison matrix passes the consistency check then fuzzy priorities i w ~ can be calculated. the last step of fuzzy ahp is priorities aggregations and alternatives ranking. in this step, if there are i alternatives and j criteria, then the final priority of alternative i will be as    n j ijji awa 1 where jw is the j th criterion weight and ija is the alternative evaluation of ia against j criterion. the higher the value of i a , the more preferred the alternative. still, if the priorities result in fuzzy form, an appropriate ranking procedure should be applied to de-fuzzify the rank of alternatives. 2.2 interval arithmetic the interval arithmetic becomes increasingly popular at present in the virtual battle against uncertainties in numerical computations and is widely used in scientific and engineering applications which deal with incomplete data. the concept of interval arithmetic used in the context of uncertainty mitigation is not quite new as it has been studied since the invention of the digital electronic computer and its immediate utilization in control science. the development of the modern theory of interval arithmetic began in 1959 with the technical reports of ramon edgar moore (dawood, 2011). he developed a number system and arithmetic for real intervals by introducing the so-called range numbers and range arithmetic. those terms are the first popular synonyms of modern interval numbers and interval arithmetic. the subsequently developed theory of interval arithmetic has provided a better solution for the problems occurring between computation and applied mathematics when utilizing floating-point arithmetic or traditional numerical approximation methods. the interval arithmetic can be broadly defined as a field of study dealing with real intervals and is also referred to as interval mathematics, interval analysis or interval computations. it provides specific rules for doing arithmetic operations with closed intervals. the interval arithmetic is centered on interval numbers at its foundation. ijahp article: hadiono/a practical implementation of interval arithmetic in ahp international journal of the analytic hierarchy process 260 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.325 an interval number is a set of real numbers in which any number that lies between two limiting numbers is a member of the set. dawood (2011) stated that the concept of an interval number system is straightforward: “each interval number represents some fixed real number between the lower and upper limits of the closed interval”. the exact definition of an interval number is provided below: let rxx , such xx  , then an interval number [ x , x ] is a closed and bounded nonempty real interval that is [ x , x ] ={x∈ r | x ≤ x ≤ x } where x =min([ xx, ]) and x =max([ x , x ]) are called, respectively, the lower and upper bounds (endpoints) of [ x , x ]. (dawood, 2011). the basic algebraic operations such as addition, multiplication, subtraction, and division for real numbers can be extended to interval numbers. the following are the basic arithmetic operations for two intervals [ x , x ] and [ y , y ] that can be performed with interval numbers (hickey, ju, & emden, 2001). (1) addition is given by equation (7), [ x , x ] + [ y , y ] = [ x + y , x + y ] (7) (2) subtraction is given by equation (8), [ x , x ] [ y , y ] = [ x y , x y ] (8) (3) multiplication is given by equation (9), [ x , x ]  [ y , y ] = [min( yxyxyxyx ,,, ), max( yxyxyxyx ,,, )] (9) (4) division is given by equation (10),    yy xx , , =         ),,,max(),,,,min( y x y x y x y x y x y x y x y x , where y and y is not 0. (10) 2.3 interval arithmetic and ahp as an extension of the standard ahp technique, saaty and vargas (1987) investigated the possibility of decision makers being able to express their judgment by using an interval judgment. they proposed the term interval judgment to be used when dealing with the uncertainty during pair-wise comparisons. the interval judgment was implemented by using the monte carlo simulation approach to derive the resultant weights because the conventional ahp technique derives said weights without considering the uncertainty of human intuition (saaty & vargas, 1987; yu, hsiao, & sheu, 2011). although the interval judgment can be used to facilitate one’s intuition during the decision making process, the problem with the uncertainty does not stop here. it raises another issue, a difficulty in measuring the inconsistency of the generated weights from the interval judgment. an additional study attempted to address this issue by proposing the fuzzy preference programming (fpp) technique. the fpp technique initially defines the values of all tolerance parameters for all judgments. ijahp article: hadiono/a practical implementation of interval arithmetic in ahp international journal of the analytic hierarchy process 261 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.325 the lexicographic goal programming for ahp (lgahp) technique was also proposed to address the inconsistency in the interval ahp technique. the lgahp technique generates the weights of interval pair-wise comparison matrices by using deviation variables. by using lgahp, it is not required to initially define tolerance parameters and it is also possible to minimize the summation of the deviation variables through a positive objective value (yu et al., 2011). the rank reversal is another issue of the interval ahp technique which arises due to a misuse of the aggregation methods. when dealing with rank reversal problems, the multiplicative ahp (mahp) technique was proposed in order to maintain the ratio scaling properties. several studies developed and extensively used the mahp technique because of the absence of rank reversal problems. in addition, there are several proposed approaches intended to make the interval ahp technique more efficient such as the logarithmic goal programming (lgp), the integrated logarithmic goal programming, the logarithmic least-squares method (lls), the logarithmic least absolute value method (llav), etc. (yu et al., 2011) when the interval arithmetic is integrated into the ahp technique, the main ahp algorithmic stages remain intact. each element in the standard ahp technique is modified to have two values, namely, a lower limit (l) and an upper limit (u). the basic algebraic operations for interval numbers are used in all computations, thus replacing the standard algebraic operations. the pair-wise matrix of ahp with interval numbers is formed in accordance with equation (11) because each element of said matrix must be presented by a pair of lower and upper values,                                   u ij l ij u i l i u i l i u j l j ulul u j l j ulul ul xxxxxx xxxxxx xxxxxx aa ,,, ,,, ,,, , 2211 2222222121 1112121111     (11) the relative weighs of the pair-wise matrix components are obtained with equation 12,                                   u ij l ij u i l i u i l i u j l j ulul u j l j ulul u norm l norm aaaaaa aaaaaa aaaaaa aa ,,, ,,, ,,, , 2211 2222222121 1112121111     , where         n i u ij l ij u ij l iju ij l ij xx xx aa 1 , , , (12) the weights of the criteria are given by equation (13),     n aa ww n j u ij l ij u i l i    1 , , , where    uili n i u i l i wwww ,, 1  (13) the consistency index (ci) equation of the pair-wise matrix is given by equation (14) which requires the prior calculation of the λmax interval, ijahp article: hadiono/a practical implementation of interval arithmetic in ahp international journal of the analytic hierarchy process 262 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.325   n bb n i u i l iul   1maxmax , ],[  , where        uili u ij l ij n j u i l i u i l i ww aaww bb , ,, , 1     so that     1 , , maxmax    n n cici ul ul  (14) finally, the coherence ratio (cr) is obtained with equation (15) and the value of random index (ri) is determined in accordance with table 2,      ul ul ul riri cici crcr , , ,  (15) table 2 random consistency index table with interval numbers n 1 2 3 4 5 6 7 8 9 10 ri [0, 0] [0, 0] [0.58, 0.58] [0.90, 0.90] [1.12, 1.12] [1.24, 1.24] [1.32, 1.32] [1.41, 1.41] [1.45, 1.45] [1.49, 1.49] 3. results and discussion this section is divided into two parts; the first part summarizes the ahp research results of jamali, samadi, and marthandan (2014) which are used as an example of the standard ahp process. the second part discusses the implementation of interval arithmetic in ahp by using the basic algebraic operations with intervals for ahp computations. the reference data which is used for the second part is taken from jamali et al. (2014) and all calculations are performed by utilizing wolfram mathematica® version 9 software. 3.1 standard ahp process it is commonly known that the adoption of electronic commerce (e-commerce) technology in small and medium-sized enterprises (smes) has significant effects on their competitiveness and performance. therefore, the technology selection becomes important and must meet the smes requirements and uniqueness. jamali et al. (2014) investigated the most relevant spectrum of e-commerce technology which is suitable for iranian family smes. the objective of the case study from jamali et al. (2014) was extracted from the related literature, refined with unstructured interviews and discussed with the expert member team. the obtained results indicated that the e-commerce technology to be selected must meet the main objectives of the iranian family smes such as: (1) sales growth; (2) maximization of profit; (3) increased market share; (4) minimization of family conflicts; and (5) preservation of family independence. the relevant technologies that meet their needs were identified as e-mail, website technology, intranet, extranet, electronic data ijahp article: hadiono/a practical implementation of interval arithmetic in ahp international journal of the analytic hierarchy process 263 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.325 exchange (edi), electronic funds transfer (eft), and barcode technology. the data sample was collected from the iranian sme database (isipo) using non-probabilistic sampling technique. the selection of the respondents for the conducted study was based on the following criteria: must be the ceo of a family sme; must have at least 5-year managerial work experience; or at least a bachelor’s education in management or in a related field of study (jamali et al., 2014). after the ahp computations, the analysis of the weighted criteria indicated that the minimization of family conflicts had the most significant weight among other criteria, and the final weights of the alternatives against the criteria led to the conclusion that the website technology is the most fitted e-commerce technology for iranian family smes (jamali et al., 2014). 3.2 ahp with interval arithmetic the practical implementation of interval arithmetic in ahp is shown below. uniform increments of 0.5 were used for creating the differences between the minimum and maximum values of the interval numbers. the intervals range from 0 to 4 which results in 9 different initial interval ranges for ahp calculations. the implementation of the modified technique does not exceed the ahp rating scale used in the pair-wise matrix. as an example, the base number starts from 8.0 and increases gradually by 0.5. the number increment will stop at 9.0 and then there is a gradual decrease by 0.5 until stopping at 1.0. the criterion pair-wise matrix (table 3) starts from range 0; there is no difference between the minimum and maximum values. equation (11) is used for the pair-wise matrix with the interval numbers (table 3). table 3 pair-wise criteria matrix with interval numbers criteria 1 criteria 2 criteria 3 criteria 4 criteria 5 criteria 1 [1, 1] [1.78, 1.78] [2.31, 2.31] [2.49, 2.49] [1.34, 1.34] criteria 2 [0.56, 0.56] [1, 1] [2.16, 2.16] [2.70, 2.70] [1.74, 1.74] criteria 3 [0.43, 0.43] [0.46, 0.46] [1, 1] [1.14, 1.14] [2.35, 2.35] criteria 4 [0.40, 0.40] [0.37, 0.37] [0.88, 0.88] [1, 1] [1.31, 1.31] criteria 5 [0.75, 0.75] [0.57, 0.57] [0.43, 0.43] [0.76, 0.76] [1, 1] the result of the normalized pair-wise matrix of the criterion table can be seen in table 4. all calculations use basic algebra operations with interval numbers. the interval of λmax of the criterion matrix is [5.255, 5.263] and the interval of the consistency index (ci) is [0.064, 0.066]. the interval of the coherence ratio (cr) of the criterion table is [0.057, 0.059]. here equations (14) and (15) are used for the calculation of λmax , ci, and cr for the criterion matrix with interval numbers. ijahp article: hadiono/a practical implementation of interval arithmetic in ahp international journal of the analytic hierarchy process 264 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.325 table 4 pair-wise criterion matrix normalization with interval numbers criteria 1 criteria 2 criteria 3 criteria 4 criteria 5 criteria 1 [0.318, 0.318] [0.427, 0.427] [0.340, 0.340] [0.308, 0.308] [0.173, 0.173] criteria 2 [0.178, 0.178] [0.240, 0.240] [0.319, 0.319] [0.334, 0.334] [0.225, 0.225] criteria 3 [0.134, 0.134] [0.110, 0.110] [0.147, 0.147] [0.141, 0.141] [0.304, 0.304] criteria 4 [0.127, 0.127] [0.089, 0.089] [0.130, 0.130] [0.124, 0.124] [0.169, 0.169] criteria 5 [0.239, 0.239] [0.134, 0.137] [0.063, 0.063] [0.094, 0.094] [0.129, 0.129] the final weights of the criteria matrix can be seen in figure 1. figure 1 has non-linear behavior and the highest input interval ranges is the prominent point. for the input interval ranges from 1.5 to 4.0, the initial linear growth is replaced with a smooth decrease of the final intervals. 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 v a lu e s intervals criteria 1 criteria 2 criteria 3 criteria 4 criteria 5 figure 1. priority vectors of criteria matrix the third alternative is the final solution in the jamali et al. (2014) research which gives the dominant weight. after the implementation of the interval arithmetic in the ahp, the third alternative is still the final solution that gives the dominant weight among the other alternatives. the weights of the third alternative for the input range interval 0.0 is [0.362, 0.363] and sequentially the weight values of the input range interval from 0.5 to 4.0 are [0.274, 0.483], [0.218, 0.616], [0.178, 0.759], [0.150, 0.913], [0.122, 1.059], [0.099, 1.219], [0.083, 1.400], and [0.067, 1.623]. the other graphical illustrations of the interval arithmetic implementation in the ahp technique are shown in figures 2-7. figures 2-6 show the graphs of the priority vectors from alternative 1 to alternative 7 for criteria 1-5 and figure 7 shows the graphs of the final priority vectors of all alternatives. the graph tendency of the priority vectors from alternative 1 to alternative 7 for criteria 1-5 is linear but for the final priority vectors is saturated. ijahp article: hadiono/a practical implementation of interval arithmetic in ahp international journal of the analytic hierarchy process 265 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.325 the graphs of the priority vectors for criteria 1-5 in figures 2-6 increase steadily. in figures 2-6, some of the linear graphs have similar values which results in the graph lines being close to each other and sometimes they are overlapping. as an example, in figure 4 the differences between the interval values are smaller than in the remaining graphs so that the graph lines between the alternatives slowly fluctuate in a narrow range. in figure 5, the curve of the fifth alternative follows a noticeably different path as compared to the linear paths of the other alternatives. in some intermediate points it is plotted noticeably lower than the other graphs but it is stabilized for the highest input ranges similarly to the graphs of the other alternatives. 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 0.0 1.0 2.0 3.0 4.0 5.0 v a lu e s intervals alternative 1 alternative 2 alternative 3 alternative 4 alternative 5 alternative 6 alternative 7 figure 2. priority vectors of criterion 1 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 0.0 1.0 2.0 3.0 4.0 5.0 v a lu e s intervals alternative 1 alternative 2 alternative 3 alternative 4 alternative 5 alternative 6 alternative 7 figure 3. priority vectors of criterion 2 ijahp article: hadiono/a practical implementation of interval arithmetic in ahp international journal of the analytic hierarchy process 266 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.325 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 0.0 1.0 2.0 3.0 4.0 5.0 v a lu e s intervals alternative 1 alternative 2 alternative 3 alternative 4 alternative 5 alternative 6 alternative 7 figure 4. priority vectors of criterion 3 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 0.0 1.0 2.0 3.0 4.0 5.0 v a lu e s intervals alternative 1 alternative 2 alternative 3 alternative 4 alternative 5 alternative 6 alternative 7 figure 5. priority vectors of criterion 4 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 0.0 1.0 2.0 3.0 4.0 5.0 v a lu e s intervals alternative 1 alternative 2 alternative 3 alternative 4 alternative 5 alternative 6 alternative 7 figure 6. priority vectors of criterion 5 the graphs of the final priority vectors (figure 7) in the last stage of computation are quite different when compared with the intermediate graphs. the graphs in figure 7 have a non-linear behavior which becomes increasingly noticeable for the highest input ranges. for input interval ranges from 1.5 to 2.5, the initial linear growth is replaced with a smooth decrease of the final intervals. interestingly, the smallest graph increments are observed for the largest input intervals in the ranges from 2.5 to 4.0. the observed nonlinearity occurs in the graphs of the final priority vectors for all alternatives. ijahp article: hadiono/a practical implementation of interval arithmetic in ahp international journal of the analytic hierarchy process 267 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.325 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 0.0 1.0 2.0 3.0 4.0 5.0 v a lu e s intervals alternative 1 alternative 2 alternative 3 alternative 4 alternative 5 alternative 6 alternative 7 figure 7. final priority vectors 4. conclusion the battle with uncertainty has been addressed in many ways especially in the decision making field and has resulted in more than one method. those methods are useful and help decision makers when they are faced with uncertainty. ahp as one popular method expands its method to overcome uncertainty by adopting fuzzy set theory which results in fuzzy ahp. the implementation of the fuzzy set theory into ahp is not the final and perfect answer to dealing with uncertainty. the fuzzy ahp still has imperfections concerned with inconsistency, but that does not mean that this inconsistency cannot be resolved. when decision makers use fuzzy ahp, they have to convert judgments into triangular fuzzy numbers to get the weights of the factors. the triangular fuzzy numbers represent numerical crisp data which are used as a foundation when developing the ahp comparison matrix. the process of fuzzy ahp is like conventional ahp except that steps for consistency checking and de-fuzzifying have been added in order to arrive at the final result. another method to deal with uncertainty is to introduce interval numbers into the ahp method. interval numbers have a special form with minimum and maximum limits not represented by conventional numbers and they can be used to represent uncertainty. single numbers used in the ahp represent exact conditions, but by using interval arithmetic the decision makers have the possibility of representing uncertainty in a formal way. the differences between the fuzzy ahp and the method proposed here can be described as follows. when using the fuzzy ahp, the decision makers have to first choose which method they will use to derive the factors weights. once they decide which method they will use, then the judgment conversion process that uses the triangular fuzzy numbers is started. the next step is building the comparison matrix. the decision makers also have to implement the consistency check in a specific step and must de-fuzzify if the priority result is still in the fuzzy form. the final result of the fuzzy ahp is still in the single number form which can ignore the uncertain value of preferences. the use of interval numbers in ahp will not change the process sequence of ahp. in the conventional ahp that uses a single number, the calculation process uses normal algebraic operations. however, normal algebraic operations cannot be used with the ijahp article: hadiono/a practical implementation of interval arithmetic in ahp international journal of the analytic hierarchy process 268 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.325 interval numbers. the calculations with interval numbers have to use interval numbers algebraic operations. with the implementation of interval numbers in ahp, the decision makers do not have to choose any method as in fuzzy ahp from the beginning of the process. they only need to assign values for the upper and the lower limits of their judgments. the form of the pair-wise comparison matrix for ahp with interval numbers is illustrated in table 3 or table 4 and during the calculation process the interval number algebraic operations are used. the final result of ahp with interval numbers will be in the form of interval numbers and uncertainty is still present but it is in an optimum state. in summary, the development and the implementation of interval arithmetic has the advantage of obtaining reliable information about the uncertainty at the final stage of decision making. in this study, the implementation of interval arithmetic is carried out for the ahp technique, one of the well-established decision making techniques. the input interval numbers in all pair-wise table comparisons used in this study have a uniform increment of 0.5 from 0 to 4 in order to systematically investigate the behavior of the output interval values at the end of the computation process. when introducing interval numbers in the pair-wise comparison tables, it must be noted that the values of the reciprocal interval numbers must be properly ordered according to their minimum and maximum values and cannot exceed the ahp rating scale limit. also, the basic algebraic operations for interval numbers must be thoroughly applied with no exceptions for the computation of all equations within the ahp algorithmic sequence. after comparing the computational results between the ahp techniques with and without interval numbers, it can be seen that the result of the ahp technique without interval numbers is within the range of the ahp technique with interval numbers. the implementation of interval numbers in the judgment process provides a more reliable way for the representation of the corresponding judgment values. the judgment process with the use of interval numbers can estimate the uncertainty which is a common issue during decision making. the interval numbers which are applied in the judgment process can be regarded as uncertainty estimators which have distinct minimum and maximum values. the increased distance between the minimum and maximum values is an indicator of the accumulated uncertainty due to the sequential use of basic algebraic operations with interval numbers. the resultant graphs of the ahp technique with interval numbers show the tendency of interval growth during the computational process. in figures 2-6, the trend is linear with slight fluctuations in every increment of the intervals. however, the final priority vector graphs (figure 7) have a tendency of stabilization. the linearity in figure 7 for input ranges from 0.0 to 1.5 is changed to a non-linear reduction of interval growth for input ranges from 1.5 to 2.5 and the interval growth for input ranges from 2.5 until 4.0 is on an even smaller scale. the main observation is that for bigger ranges of input intervals, the final intervals become saturated as can be seen in the final tendency obtained for the sample input ranges from 1.5 to 4.0. this can be explained with the principle of operation of the ahp ijahp article: hadiono/a practical implementation of interval arithmetic in ahp international journal of the analytic hierarchy process 269 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.325 technique which results in a stabilization of weight differences in the final algorithmic stage. this study is limited to the use of a uniform distribution of weights within an interval. further extensions on the use of interval arithmetic may include non-uniform distributions of weights for specific applications. ijahp article: hadiono/a practical implementation of interval arithmetic in ahp international journal of the analytic hierarchy process 270 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.325 references dawood, h. (2011). theories of interval arithmetic: mathematical foundation and applications. saarbrücken, lap lambert academic publishing gmbh & co. kg dudweiler landstr. hickey, t., ju, q., & emden, m. van. (2001). interval arithmetic: from principles to implementation. journal of the acm, 48(5), 1038-1068. doi:10.1145/502102.502106 jamali, s. k., samadi, b., & marthandan, g. (2014). prioritizing electronic commerce technologies in iranian family smes. interdisciplinary journal of contemporary research in business, 6(2), 147-180. javanbarg, m. b., scawthorn, c., kiyono, j., & shahbodaghkhan, b. (2012). fuzzy ahpbased multicriteria decision making systems using particle swarm optimization. expert systems with applications, 39(1), 960–966. doi:10.1016/j.eswa.2011.07.095 koul, s., & verma, r. (2012). dynamic vendor selection: a fuzzy ahp approach. international journal of the analytic hierarchy process, 4(2), 118–136. doi: http://dx.doi.org/10.13033/ijahp.v4i2.25 lahby, m., cherkaoui, l., & adib, a. (2013). a novel ranking algorithm based network selection. journal of networks, 8(2), 263–272. doi:10.4304/jnw.8.2.263-272 lane, e. f., & verdini, w. a. (1989). a consistency test for ahp decision makers. decision sciences, 20(3), 575–590. doi:10.1111/j.1540-5915.1989.tb01568.x. saaty, t. l. (1994). how to make a decision: the analytic hierarchy process. interfaces, 24(6), 19-43. doi:http://dx.doi.org/10.1287/inte.24.6.19 saaty, t. l. (1990). an exposition on the ahp in reply to the paper “remarks on the analytic hierarchy process”. management science, 36(3), 259–268. doi:http://dx.doi.org/10.1287/mnsc.36.3.259 saaty, t., & vargas, l. (1987). uncertainty and rank order in the analytic hierarchy process. european journal of operational research, 32, 107–117. doi:10.1016/03772217(87)90275-x yu, j. r., hsiao, y., & sheu, h. (2011). a multiplicative approach to derive weights in the interval analytic hierarchy process. international journal of fuzzy systems, 13(3), 225–231. wang, y. m., & chin, k. s. (2011). fuzzy analytic hierarchy process: algorithmic fuzzy preference programming methodology. international journal of approximate reasoning, 52(4), 541-553. doi:10.1016/j.ijar.2010.12.004 zahedi, f. (1986). the analytic hierarchy process: a survey of the method and its applications. interfaces, 16(4), 96–108. doi: http://dx.doi.org/10.1287/inte.16.4.96 http://dx.doi.org/10.1145/502102.502106 http://dx.doi.org/10.1016/j.eswa.2011.07.095 http://dx.doi.org/10.13033/ijahp.v4i2.25 http://dx.doi.org/10.1287/inte.24.6.19 http://dx.doi.org/10.1287/mnsc.36.3.259 http://dx.doi.org/10.1016/0377-2217(87)90275-x http://dx.doi.org/10.1016/0377-2217(87)90275-x http://dx.doi.org/10.1016/j.ijar.2010.12.004 http://dx.doi.org/10.1287/inte.16.4.96 ijahp article: karasan/a novel hesitant intuitionistic fuzzy linguistic ahp method and its application to prioritization of investment alternatives international journal of the analytic hierarchy process 127 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.610 a novel hesitant intuitionistic fuzzy linguistic ahp method and its application to prioritization of investment alternatives ali karasan yildiz technical university turkey akarasan@yildiz.edu.tr abstract intuitionistic fuzzy extensions are the most used type of fuzzy extension in the literature because they better represent the decision makers’ strength of commitment on the considered subject in an effective way including membership and non-membership functions. on the other hand, decision makers may assign more than one intuitionistic fuzzy number when they are hesitant about assigning a membership degree and a nonmembership degree. hesitant fuzzy sets, another extension of ordinary fuzzy sets, help decision makers assign different values to the same element in an attempt to reflect the decision makers’ hesitation. utilizing these two types of fuzzy sets captures both the uncertainty and ambiguity of the considered problem and helps eliminate the weaknesses of each fuzzy extension. in this study, hesitant intuitionistic fuzzy linguistic sets (hiflss) are used to extend the analytic hierarchy process (ahp). the developed method is applied to an investment prioritization problem based on relevant risk factors. comparative analyses with intuitionistic fuzzy ahp and hesitant fuzzy ahp methods are conducted in order to validate the proposed method. a sensitivity analysis is also applied to present the stability of the results of the hesitant intuitionistic fuzzy linguistic ahp method. keywords: hesitant fuzzy sets; intuitionistic fuzzy sets; ahp; prioritization; hesitant intuitionistic fuzzy linguistic sets 1. introduction multi-criteria decision making (mcdm) methods are one of the most used decisionmaking tools in the literature to deal with sets of criteria and alternatives simultaneously. the analytic hierarchy process (ahp), which is one of the most useful multi-criteria decision-making methods that was originally developed by saaty, aims to achieve the weighting of values of independent inputs by comparing them in a pairwise comparison matrix (saaty, 1980). these inputs can be quantitative such as cost, profit, and distance and/or qualitative such as judgments, behaviors, and preferences. before the evaluation process starts, the values of the compared inputs must be checked to calculate the matrix’s consistency. since the inconsistencies are in the decision makers’ decisions, the mailto:akarasan@yildiz.edu.tr ijahp article: karasan/a novel hesitant intuitionistic fuzzy linguistic ahp method and its application to prioritization of investment alternatives international journal of the analytic hierarchy process 128 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.610 ahp method is tolerated up to a lower limit of consistency ratio which is 90%. but, it is difficult to construct consistent matrices in systems where information is uncertain and decision makers are hesitant. therefore, the classic ahp method cannot provide effective results and needs some extensions to handle these deficiencies. fuzzy logic is one of the most advantageous ways of handling uncertainty by representing it with the degree of membership function and was introduced by zadeh (1965). it is an invaluable way of finding efficient solutions for researchers in the fields of engineering, mathematics, medical sciences, computer sciences, natural and applied sciences, business analysis, public relations, and human behaviors. in order to increase this efficiency, the fuzzy sets are extended with many types in light of fuzzy logic. these types are as follows: type-2 fuzzy sets, intuitionistic fuzzy sets, neutrosophic fuzzy sets, hesitant fuzzy sets, pythagorean fuzzy sets, and orthopair fuzzy sets (zadeh, 1975; atanassov, 1986; smarandache, 1995; torra, 2010; yager & abbasov, 2013; ciucci, 2016). besides, as many extensions of fuzzy sets have been introduced, many multicriteria decision-making methods that are related to fuzzy logic have kept pace with them. the ahp method has been extended in many forms including buckley’s fuzzy ahp method, chang’s fuzzy ahp method, intuitionistic fuzzy ahp method, type-2 fuzzy ahp method, hesitant fuzzy ahp method, neutrosophic ahp method, and the pythagorean fuzzy ahp method (buckley, 1985; chang, 1996; wu et al., 2013; kahraman et al., 2014; öztaysi et al., 2015; abdel-basset et al., 2017; ilbahar et al., 2018). also, ozdemir & sahin (2018), durao et al. (2018), pipatprapa et al. (2018), yucesan & kahraman (2019), vladeanu & matthews (2019), vyas et al. (2019), xu et al. (2019), arulbalaji et al. (2019) are some studies that applied the ahp method with its extension to different areas. pipatprapa et al. (2018) presented an environmental performance assessment for the food industry in thailand by using the ahp method. durao et al. (2018) presented a process selection for the internet of things by using the ahp method. ozdemir & sahin (2018) studied the ahp method for the selection of a location for a solar photovoltaic power plant. arulbalaji et al. (2019) studied geographical information systems and ahp techniques for the delineation of potential groundwater zones in india. xu et al. (2019) studied an entropy weight modified ahp hierarchy model for the construction of a regional informatization ecological environment. vyas et al. (2019) developed a green building rating system by using an extended ahp model with fuzzy integrals. vladeanu & matthews (2019) presented a risk based asset management study for wastewater pipes by using the ahp method. yucesan & kahraman (2019) studied risk evaluation and prevention in a hydropower plant operation by using the pythagorean fuzzy ahp method. intuitionistic fuzzy sets are the most used type of fuzzy extensions in the literature since they have the ability to better represent the decision makers’ strength of commitment on the considered subject including membership and non-membership functions simultaneously. on the other hand, decision makers may assign more than one intuitionistic fuzzy number when they are hesitant in assigning a membership degree and a non-membership degree. hesitant fuzzy sets, another extension of ordinary fuzzy sets, help decision makers assign different values to the same element in order to reflect the decision makers’ hesitation. utilizing these two types of fuzzy sets captures both the uncertainty and ambiguity of the considered problem and helps eliminate the weaknesses of each fuzzy extension. ijahp article: karasan/a novel hesitant intuitionistic fuzzy linguistic ahp method and its application to prioritization of investment alternatives international journal of the analytic hierarchy process 129 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.610 this paper contributes to the literature as follows:  the proposed method has the ability to reflect both pessimistic and optimistic judgements of the decision maker by using intuitionistic fuzzy sets.  since intuitionistic fuzzy sets offer a larger domain than ordinary fuzzy sets for the decision makers to introduce their judgements, the proposed method enables the representation of the decision makers’ hesitancy by using both membership and non-membership functions of an element to a set.  the proposed method can be a supportive decision making tool for the decision makers who intend to work with mcdm methods with uncertain information. in investment decision making problems, managers aim to minimize adverse effects of risks by using personal assessments, and experiences and mathematical models when they are deciding the appropriate alternatives to invest in. in this paper, we propose a novel hesitant intuitionistic fuzzy linguistic ahp method for the prioritization of investment alternatives to help make the best decision. the paper is organized as follows: in section 2, the steps of our proposed methodology and details of hesitant and intuitionistic fuzzy sets are presented. in section 3, an application of a company that wants to invest to build a factory is demonstrated. the paper ends with a conclusion and suggestions for further research. 2. methodology in this section, the preliminaries of hesitant fuzzy sets and intuitionistic fuzzy sets, and a novel hesitant intuitionistic fuzzy linguistic ahp method are given, respectively. 2.1 hesitant fuzzy sets hesitant fuzzy sets (hfs), initially developed by torra (2010), are the extensions of regular fuzzy sets which handle the situations where a set of values are possible for the membership of a single element (rodrigez et al., 2012). torra (2010) defines hesitant fuzzy sets as follows. let x be a fixed set, a hesitant fuzzy set (hfs) on x is in terms of a function that when applied to x returns a subset of [0, 1]. a mathematical expression for hfs is as follows: 𝐸 = {< 𝑥, ℎ𝐸 (𝑥) > | 𝑥𝜖 𝑋 } (1) where ℎ𝐸 (𝑥) is a set of some values in [0, 1], denoting the possible membership degrees of the element 𝑥𝜖 𝑋 to the set e. xu & xia (2011) call ℎ = ℎ𝐸 (𝑥) a hesitant fuzzy element (hfe). some basic definitions about hesitant sets are given as follows (torra, 2010): 𝜆ℎ = ∪𝛾∈ℎ {1 − (1 − 𝛾) 𝜆} (2) ℎ1⨁ℎ2 = ∪𝛾1∈ℎ1,𝛾2∈ℎ2, {𝛾1 + 𝛾2 − 𝛾1𝛾2} (3) ijahp article: karasan/a novel hesitant intuitionistic fuzzy linguistic ahp method and its application to prioritization of investment alternatives international journal of the analytic hierarchy process 130 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.610 ℎ1⨂ℎ2 = ∪𝛾1∈ℎ1,𝛾2∈ℎ2, { 𝛾1𝛾2} (4) where ℎ1, ℎ2 are hfes and 𝑙 is the number of elements in an hfe, which is called length. 2.2 intuitionistic fuzzy sets intuitionistic fuzzy sets (ifss) were introduced by atanassov (1986). an ifs includes two membership values named membership and non-membership for describing any x in x such that their sum is at most equal to 1, which is demonstrated in definition 1 (atanassov, 1986). definition 1. let 𝑋 ≠ ∅ be a given set. an intuitionistic fuzzy set in x is an object a which is called an intuitionistic fuzzy number (ifn), if it holds the following conditions: �̃� = {〈𝑥, 𝜇�̃�(𝑥), 𝜐�̃�(𝑥)〉; 𝑥 ∈ 𝑋} (5) where 𝜇�̃�(𝑥): 𝑋 → [0,1] and 𝜐�̃�(𝑥): 𝑋 → [0,1] satisfies the condition of 0 ≤ 𝜇�̃�(𝑥) + 𝜐�̃�(𝑥) ≤ 1, for every 𝑥 ∈ 𝑋. (6) besides, 𝜋�̃�(𝑥) = 1 − 𝜇�̃�(𝑥) + 𝜐�̃�(𝑥) is the hesitation degree of element �̃� in set x. it is obvious that we obtain 0 ≤ 𝜋�̃� ≤ 1 for every 𝑥 ∈ 𝑋. definition 2. the operations of addition ⊕ and multiplication ⊗ on ifns were defined by atanassov (1986) as follows. let �̃� = 〈𝜇𝐴, 𝑣𝐴〉 and �̃� = 〈𝜇𝐵 , 𝑣𝐵 〉 be ifns. �̃� ⊕ �̃� = 〈𝜇𝐴 + 𝜇𝐵 − 𝜇𝐴𝜇𝐵 , 𝑣𝐴𝑣𝐵 〉 (7) �̃� ⊗ �̃� = 〈𝜇𝐴𝜇𝐵 , 𝑣𝐴 + 𝑣𝐵 − 𝑣𝐴𝑣𝐵 〉 (8) definition 3. let �̃� = 〈𝜇𝐴, 𝑣𝐴〉 be ifn. the defuzzification operation of an intuitionistic fuzzy number is given as follows: ℌ𝐴 = 𝜇𝐴+(1−𝜗𝐴) 2 (9) 2.3 hesitant intuitionistic fuzzy linguistic (hifl) ahp method step 1. construct the collected pairwise comparison matrices for criteria and alternatives of the expert’s appraisals by using linguistic terms. step 2. the linguistic terms are transformed into ifns using the scale given in table 1. a 10% hesitancy exists in this scale. ijahp article: karasan/a novel hesitant intuitionistic fuzzy linguistic ahp method and its application to prioritization of investment alternatives international journal of the analytic hierarchy process 131 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.610 table 1 linguistic scale for hesitant intuitionistic fuzzy linguistic (hifl) ahp method linguistic term ifn certainly low importance cli <[0.05, 0.85]> very low importance vli <[0.2, 0.7]> low importance li <[0.35, 0.55]> equal importance ei <[0.5, 0.5]> high importance hi <[0.55, 0.35]> very high importance vhi <[0.7, 0.2]> certainly high importance chi <[0.85, 0.05]> step 3. determine the minimum and maximum of the membership and non-membership values which are given in a collected pairwise comparison matrix. step 4. calculate the optimistic and pessimistic weight values of the pairwise comparison matrix by using equation 9. step 5. calculate the mid points of the optimistic and pessimistic weight values. step 6. normalize the mid points and rank them in descending order. 3. illustrative example a company in turkey wants to be a distributor of the components of unmanned aerial vehicles for the defense industry sector. to make a distribution agreement for the turkey market, the company performed preparatory work about the conditions of component producers. through this work, the company determined four producers as the alternatives which are suitable for the budget. it was the consensus of the company to assess these alternatives with respect to the four criteria given below.  investment cost  maintenance cost  average life expectancy of the components  flexibility of the components to the different models 3.1 steps of the method to check the applicability of the proposed method, the hesitant intuitionistic fuzzy linguistic (hifl) ahp method was applied as follows: the collected pairwise comparison matrices are given in table 2. ijahp article: karasan/a novel hesitant intuitionistic fuzzy linguistic ahp method and its application to prioritization of investment alternatives international journal of the analytic hierarchy process 132 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.610 table 2 collected pairwise comparison matrices wrt goal c1 c2 c3 c4 c1 ei ei ei li ei ei li hi c2 h i ei ei ei ei li ei vli vhi hi c3 ei hi hi ei vhi ei ei ei vhi chi c4 li vli li vli cli ei ei ei wrt c1 al1 al2 al3 al4 al1 ei ei ei li ei ei li hi al2 h i ei ei ei ei li ei vli vhi hi al3 ei li hi ei vhi ei ei ei vhi chi al4 li vli li vli cli ei ei ei wrt c2 al1 al2 al3 al4 al1 ei ei ei li hi hi ei hi al2 h i li ei ei ei vli vhi hi ei li al3 li vhi vli ei ei ei vli li al4 ei li li ei hi vhi hi ei ei ei wrt c3 al1 al2 al3 al4 al1 ei ei ei li hi vhi hi li ei hi al2 h i li vli ei ei ei vli hi hi ei li al3 li hi vhi li ei ei ei vli li al4 ei li li ei hi vhi hi ei ei ei wrt c4 al1 al2 al3 al4 al1 ei ei ei li hi hi ei ei hi hi al2 h i li ei ei ei vli li hi hi ei li al3 li ei vhi hi li ei ei ei hi li li al4 ei li li li ei hi li hi hi ei ei ei wrt: with respect to to clarify table 2, the comparison of c2 to c1 with respect to goal is handled. when we check the table, there are two linguistic terms, hi and ei, and the term “-” which means null. the intuitionistic fuzzy numbers that correspond to the hi and ei terms will be used for the calculations. we can consider hi an optimistic value and ei a pessimistic value for this comparison. all comparisons were made according to this logic in table 2. for the next step, linguistic terms were transformed to ifns. table 3 presents the collected pairwise comparison matrices which correspond with the ifns. ijahp article: karasan/a novel hesitant intuitionistic fuzzy linguistic ahp method and its application to prioritization of investment alternatives international journal of the analytic hierarchy process 133 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.610 table 3 collected pairwise comparison matrices that correspond with the ifns goal c1 c2 c3 c4 c1 <[0.5, 0.5]>, <[0.5, 0.5]>, <[0.5, 0.5]> <[0.35, 0.55]>, <[0.5, 0.5]> <[0.5, 0.5]>, <[0.35, 0.55]> <[0.55, 0.35]> c2 <[0.55, 0.35]>, <[0.5, 0.5]> <[0.5, 0.5]>, <[0.5, 0.5]>, <[0.5, 0.5]> <[0.35, 0.55]>, <[0.5, 0.5]>, <[0.2, 0.7]> <[0.7, 0.2]>, <[0.55, 0.35]> c3 <[0.5, 0.5]>, <[0.55, 0.35]> <[0.55, 0.35]>, <[0.5, 0.5]>, <[0.7, 0.2]> <[0.5, 0.5]>, <[0.5, 0.5]>, <[0.5, 0.5]> <[0.7, 0.2]>, <[0.85, 0.05]> c4 <[0.35, 0.55]> <[0.2, 0.7]>, <[0.35, 0.55]> <[0.2, 0.7]>, <[0.05, 0.85]> <[0.5, 0.5]>, <[0.5, 0.5]>, <[0.5, 0.5]> c1 al1 al2 al3 al4 al1 <[0.5, 0.5]>, <[0.5, 0.5]>, <[0.5, 0.5]> <[0.35, 0.55]>, <[0.5, 0.5]> <[0.5, 0.5]>, <[0.35, 0.55]> <[0.55, 0.35]> al2 <[0.55, 0.35]>, <[0.5, 0.5]> <[0.5, 0.5]>, <[0.5, 0.5]>, <[0.5, 0.5]> <[0.35, 0.55]>, <[0.5, 0.5]>, <[0.2, 0.7]> <[0.7, 0.2]>, <[0.55, 0.35]> al3 <[0.5, 0.5]>, <[0.35, 0.55]> <[0.55, 0.35]>, <[0.5, 0.5]>, <[0.7, 0.2]> <[0.5, 0.5]>, <[0.5, 0.5]>, <[0.5, 0.5]> <[0.7, 0.2]>, <[0.85, 0.05]> al4 <[0.35, 0.55]> <[0.2, 0.7]>, <[0.35, 0.55]> <[0.2, 0.7]>, <[0.05, 0.85]> <[0.5, 0.5]>, <[0.5, 0.5]>, <[0.5, 0.5]> c2 al1 al2 al3 al4 al1 <[0.5, 0.5]>, <[0.5, 0.5]>, <[0.5, 0.5]> <[0.35, 0.55]>, <[0.55, 0.35]> <[0.55, 0.35]> <[0.5, 0.5]>, <[0.55, 0.35]> al2 <[0.55, 0.35]>, <[0.35, 0.55]> <[0.5, 0.5]>, <[0.5, 0.5]>, <[0.5, 0.5]> <[0.2, 0.7]>, <[0.7, 0.2]> <[0.55, 0.35]>, <[0.5, 0.5]>, <[0.35, 0.55]> al3 <[0.35, 0.55]> <[0.7, 0.2]>, <[0.2, 0.7]> <[0.5, 0.5]>, <[0.5, 0.5]>, <[0.5, 0.5]> <[0.2, 0.7]>, <[0.35, 0.55]> al4 <[0.5, 0.5]>, <[0.35, 0.55]> <[0.35, 0.55]>, <[0.5, 0.5]>, <[0.55, 0.35]> <[0.7, 0.2]>, <[0.55, 0.35]> <[0.5, 0.5]>, <[0.5, 0.5]>, <[0.5, 0.5]> c3 al1 al2 al3 al4 al1 <[0.5, 0.5]>, <[0.5, 0.5]>, <[0.5, 0.5]> <[0.35, 0.55]>, <[0.55, 0.35]>, <[0.7, 0.2]> <[0.55, 0.35]>, <[0.35, 0.55]> <[0.5, 0.5]>, <[0.55, 0.35]> al2 <[0.55, 0.35]>, <[0.35, 0.55]>, <[0.2, 0.7]> <[0.5, 0.5]>, <[0.5, 0.5]>, <[0.5, 0.5]> <[0.2, 0.7]>, <[0.55, 0.35]> <[0.55, 0.35]>, <[0.5, 0.5]>, <[0.35, 0.55]> al3 <[0.35, 0.55]>, <[0.55, 0.35]> <[0.7, 0.2]>, <[0.35, 0.55]> <[0.5, 0.5]>, <[0.5, 0.5]>, <[0.5, 0.5]> <[0.2, 0.7]>, <[0.35, 0.55]> al4 <[0.5, 0.5]>, <[0.35, 0.55]> <[0.35, 0.55]>, <[0.5, 0.5]>, <[0.55, 0.35]> <[0.7, 0.2]>, <[0.55, 0.35]> <[0.5, 0.5]>, <[0.5, 0.5]>, <[0.5, 0.5]> c4 al1 al2 al3 al4 al1 <[0.5, 0.5]>, <[0.5, 0.5]>, <[0.5, 0.5]> <[0.35, 0.55]>, <[0.55, 0.35]> <[0.55, 0.35]>, <[0.5, 0.5]> <[0.5, 0.5]>, <[0.55, 0.35]>, <[0.55, 0.35]> al2 <[0.55, 0.35]>, <[0.35, 0.55]> <[0.5, 0.5]>, <[0.5, 0.5]>, <[0.5, 0.5]> <[0.2, 0.7]>, <[0.35, 0.55]>, <[0.55, 0.35]> <[0.55, 0.35]>, <[0.5, 0.5]>, <[0.35, 0.55]> al3 <[0.35, 0.55]>, <[0.5, 0.5]> <[0.7, 0.2]>, <[0.55, 0.35]>, <[0.35, 0.55]> <[0.5, 0.5]>, <[0.5, 0.5]>, <[0.5, 0.5]> <[0.55, 0.35]>, <[0.35, 0.55]>, <[0.35, 0.55]> al4 <[0.5, 0.5]>, <[0.35, 0.55]>, <[0.35, 0.55]> <[0.35, 0.55]>, <[0.5, 0.5]>, <[0.55, 0.35]> <[0.35, 0.55]>, <[0.55, 0.35]>, <[0.55, 0.35]> <[0.5, 0.5]>, <[0.5, 0.5]>, <[0.5, 0.5]> ijahp article: karasan/a novel hesitant intuitionistic fuzzy linguistic ahp method and its application to prioritization of investment alternatives international journal of the analytic hierarchy process 134 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.610 for the next step, the minimum and maximum values of the membership and nonmembership values are obtained as shown in table 4. table 4 minimum and maximum vales of the membership and non-membership values 𝝁𝒎𝒊𝒏 𝝁𝒎𝒂𝒙 𝝑𝒎𝒊𝒏 𝝑𝒎𝒂𝒙 𝝁𝒎𝒊𝒏 𝝁𝒎𝒂𝒙 𝝑𝒎𝒊𝒏 𝝑𝒎𝒂𝒙 𝝁𝒎𝒊𝒏 𝝁𝒎𝒂𝒙 𝝑𝒎𝒊𝒏 𝝑𝒎𝒂𝒙 𝝁𝒎𝒊𝒏 𝝁𝒎𝒂𝒙 𝝑𝒎𝒊𝒏 𝝑𝒎𝒂𝒙 go al c1 c2 c3 c4 c1 0.5 0.5 0.5 0.5 0.3 5 0.5 5 0.5 0.5 5 0.3 5 0.5 0.5 0.5 5 0.5 5 0.5 5 0.3 5 0.3 5 c2 0.5 0.5 5 0.3 5 0.5 0.5 0.5 0.5 0.5 0.2 0.5 0.5 0.7 0.5 5 0.7 0.2 0.3 5 c3 0.5 0.5 5 0.3 5 0.5 0.5 0.5 0.2 0.5 0.5 0.5 0.5 0.5 0.7 0.8 5 0.0 5 0.2 c4 0.3 5 0.3 5 0.5 5 0.5 5 0.2 0.7 0.5 5 0.7 0.0 5 0.2 0.7 0.8 5 0.5 0.5 0.5 0.5 c1 al1 al2 al3 al4 al 1 0.5 0.5 0.5 0.5 0.3 5 0.5 5 0.5 0.5 5 0.3 5 0.5 0.5 0.5 5 0.5 5 0.5 5 0.3 5 0.3 5 al 2 0.5 0.5 5 0.3 5 0.5 0.5 0.5 0.5 0.5 0.2 0.5 0.5 0.7 0.5 5 0.7 0.2 0.3 5 al 3 0.3 5 0.5 0.5 0.5 5 0.5 0.5 0.2 0.5 0.5 0.5 0.5 0.5 0.7 0.8 5 0.0 5 0.2 al 4 0.3 5 0.3 5 0.5 5 0.5 5 0.2 0.7 0.5 5 0.7 0.0 5 0.2 0.7 0.8 5 0.5 0.5 0.5 0.5 c2 al1 al2 al3 al4 al 1 0.5 0.5 0.5 0.5 0.3 5 0.5 5 0.3 5 0.5 5 0.5 5 0.5 5 0.3 5 0.3 5 0.5 0.5 5 0.3 5 0.5 al 2 0.3 5 0.5 5 0.3 5 0.5 5 0.5 0.5 0.5 0.5 0.2 0.7 0.2 0.7 0.3 5 0.5 5 0.3 5 0.5 5 al 3 0.3 5 0.3 5 0.5 5 0.5 5 0.2 0.7 0.2 0.7 0.5 0.5 0.5 0.5 0.2 0.3 5 0.5 5 0.7 al 4 0.3 5 0.5 0.5 0.5 5 0.3 5 0.5 5 0.3 5 0.5 5 0.5 5 0.7 0.2 0.3 5 0.5 0.5 0.5 0.5 c3 al1 al2 al3 al4 al 1 0.5 0.5 0.5 0.5 0.3 5 0.5 5 0.2 0.5 5 0.3 5 0.5 5 0.3 5 0.5 5 0.5 0.5 5 0.3 5 0.5 al 2 0.2 0.5 5 0.3 5 0.7 0.5 0.5 0.5 0.5 0.2 0.5 5 0.3 5 0.7 0.3 5 0.5 5 0.3 5 0.5 5 al 3 0.3 5 0.5 5 0.3 5 0.5 5 0.3 5 0.5 5 0.2 0.5 5 0.5 0.5 0.5 0.5 0.2 0.3 5 0.5 5 0.7 al 4 0.3 5 0.5 0.5 0.5 5 0.3 5 0.5 5 0.3 5 0.5 5 0.5 5 0.7 0.2 0.3 5 0.5 0.5 0.5 0.5 c4 al1 al2 al3 al4 al 1 0.5 0.5 0.5 0.5 0.3 5 0.5 5 0.3 5 0.5 5 0.5 0.5 5 0.3 5 0.5 0.5 0.5 5 0.3 5 0.5 al 2 0.3 5 0.5 5 0.3 5 0.5 5 0.5 0.5 0.5 0.5 0.2 0.5 5 0.3 5 0.7 0.3 5 0.5 5 0.3 5 0.5 5 al 3 0.3 5 0.5 0.5 0.5 5 0.3 5 0.5 5 0.2 0.5 5 0.5 0.5 0.5 0.5 0.3 5 0.5 5 0.3 5 0.5 5 al 4 0.3 5 0.5 0.5 0.5 5 0.3 5 0.5 5 0.3 5 0.5 5 0.3 5 0.5 5 0.3 5 0.5 5 0.5 0.5 0.5 0.5 ijahp article: karasan/a novel hesitant intuitionistic fuzzy linguistic ahp method and its application to prioritization of investment alternatives international journal of the analytic hierarchy process 135 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.610 then, the values in table 4 were used to obtain optimistic and pessimistic weights of the criteria and alternatives in the comparison matrices. to obtain an optimistic value of a comparison, the maximum of the membership and minimum of the non-membership values was taken. similarly, to obtain a pessimistic value of a comparison, the minimum of the membership and maximum of the non-membership values was taken. to clarify this step, an example is given based on the comparison of all criteria with c1 with respect to goal as shown in table 5. table 5 optimistic results based on the comparison of all criteria with c1 with respect to goal comparison 𝜇 𝜗 ℌ c1 c1 0.50 0.50 0.50 c2 c1 0.55 0.35 0.60 c3 c1 0.55 0.35 0.60 c4 c1 0.35 0.55 0.40 here, when we examined the values in table 4, 𝜇 and 𝜗 values for the optimistic results were calculated as below, respectively: for c1-c1, 𝜇 value max𝜇 𝐶1−𝐶1(0.5,0.5) = 0.5, and 𝜗 value min𝜗 𝐶1−𝐶1(0.5,0.5) = 0.5. for c2-c1, 𝜇 value max𝜇 𝐶2−𝐶1(0.50,0.55) = 0.55, and 𝜗 value min𝜗 𝐶2−𝐶1(0.35,0.5) = 0.35. the same calculations were performed for every comparison. after finding the membership and non-membership functions, equation 9 was applied to find the defuzzified values. similar logic was applied for the pessimistic results. this time, the minimum of 𝜇 values and maximum of 𝜗 values were selected. the results are given in table 6. table 6 pessimistic results based on the comparison of all criteria with c1 with respect to goal comparison 𝜇 𝜗 ℌ c1 c1 0.50 0.50 0.50 c2 c1 0.50 0.50 0.50 c3 c1 0.50 0.50 0.50 c4 c1 0.35 0.55 0.40 these values were used to obtain the unnormalized weights. for the comparison of criteria with respect to goal, the optimistic and pessimistic values of c1 were found as shown below, respectively: ijahp article: karasan/a novel hesitant intuitionistic fuzzy linguistic ahp method and its application to prioritization of investment alternatives international journal of the analytic hierarchy process 136 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.610 optimistic value= 0.5+0.525+0.5+0.6 4 = 0.531 pessimistic value= 0.5+0.4+0.4+0.6 4 = 0.475 the results of these calculations are given in table 7. table 7 unnormalized weights of the criteria and alternatives goal optimistic pessimistic c1 0.531 0.475 c2 0.588 0.463 c3 0.663 0.563 c4 0.431 0.313 c1 optimistic pessimistic al1 0.531 0.475 al2 0.588 0.463 al3 0.638 0.538 al4 0.431 0.313 c2 optimistic pessimistic al1 0.575 0.5 al2 0.613 0.388 al3 0.513 0.35 al4 0.588 0.475 c3 optimistic pessimistic al1 0.594 0.45 al2 0.575 0.35 al3 0.544 0.388 al4 0.588 0.475 c4 optimistic pessimistic al1 0.575 0.475 al2 0.575 0.388 al3 0.569 0.425 al4 0.55 0.425 finally, mid values and local weights of alternatives and weights of criteria are presented in table 8. ijahp article: karasan/a novel hesitant intuitionistic fuzzy linguistic ahp method and its application to prioritization of investment alternatives international journal of the analytic hierarchy process 137 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.610 table 8 mid values and local weights of alternatives and weights of criteria goal mid-point normalized weight c1 0.503 0.25 c2 0.525 0.261 c3 0.613 0.304 c4 0.372 0.185 c1 mid-point normalized weight al1 0.503 0.253 al2 0.525 0.264 al3 0.588 0.296 al4 0.372 0.187 c2 mid-point normalized weight al1 0.538 0.269 al2 0.5 0.25 al3 0.431 0.216 al4 0.531 0.266 c3 mid-point normalized weight al1 0.522 0.263 al2 0.463 0.233 al3 0.466 0.235 al4 0.531 0.268 c4 mid-point normalized weight al1 0.525 0.264 al2 0.481 0.242 al3 0.497 0.25 al4 0.488 0.245 global weights of the alternatives were calculated as follows: al1=0.262 al2=0.247 al3=0.248 al4=0.243 to clarify the calculations, al1’s normalized weight was obtained by using the values given in table 7 as below. 0.25 ∗ 0.253 + 0.261 ∗ 0.268 + 0.304 ∗ 0.263 + 0.185 ∗ 0.264 = 0.262 similarly, the same calculation was also performed for the other alternatives and the global weights were found. after the calculations, al1 was the best alternative for investment. the worst alternative was determined to be al4. 3.2 sensitivity analysis to determine the robustness of the proposed approach, we conducted a one-at-a-time sensitivity analysis with respect to changes of criteria weights and then observed the alternatives ranks. table 9 represents the pattern of the analysis and results. ijahp article: karasan/a novel hesitant intuitionistic fuzzy linguistic ahp method and its application to prioritization of investment alternatives international journal of the analytic hierarchy process 138 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.610 table 9 pattern for the sensitivity analysis and rank of the alternatives al1 al2 al3 al4 c1 – cli 1 3 4 2 c1 – ei 1 3 2 4 c1 – chi 2 3 1 4 c2 – cli 1 3 2 4 c2 – ei 1 2 4 3 c2 – chi 1 3 4 2 c3 – cli 1 3 2 4 c3 – ei 1 4 3 2 c3 – chi 1 4 3 2 c4 – cli 1 3 4 2 c4 – ei 1 3 2 4 c4 – chi 1 3 2 4 to illustrate with an example, c1’s weight was changed as cli which corresponds to [0.05, 0.85]. then, this intuitionistic fuzzy number was converted to a crisp value by using the defuzzification function. next, new results were obtained by using this new weight. finally, the normalized weights of alternatives were found. figure 1 presents the ranks of alternatives based on changes in criteria weights. figure 1 results of the sensitivity analyses through the calculations, it can be seen that al1 is always the best alternative except for one criterion weight change (when c1 is equal to chi). this verifies that our proposed 1 2 3 4 c1 cli c1 ei c1 chi c2 cli c2 ei c2 chi c3 cli c3 ei c3 chi c4 cli c4 ei c4 chi al1 al2 al3 al4 ijahp article: karasan/a novel hesitant intuitionistic fuzzy linguistic ahp method and its application to prioritization of investment alternatives international journal of the analytic hierarchy process 139 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.610 method’s decisions are robust. the weights also changed in every change of criteria weight. this also verifies that our model is sensitive to the changes in the criteria values. 4. conclusion in this paper, a hesitant intuitionistic ahp method was proposed and applied to an illustrative example to show its applicability. also, a one-at-a-time sensitivity analysis was conducted to show that the proposed method’s decisions are robust. since intuitionistic fuzzy sets offer a larger domain to represent the decision makers’ judgments, it is very useful for the application. hesitant fuzzy sets help decision makers assign different values to the same element. the result of the proposed method shows that the c3 criterion is the most effective criterion for the selection process. as a result of the application, al1 is selected as the best alternative. through the application process, a one-at-a-time sensitivity analysis was applied, and it was revealed that the results of the proposed method were robust. thus, our proposed methodology was an evaluation process which can be used as a decisionmaking tool by managers or researchers to make useful inferences, judgements, and decisions for real-world applications. since the model does not only consider the quantitative data, but also qualitative data, it is very useful for areas that have uncertainty and indeterminacy in the decision making processes. for further research, the data can be extended by using a real case problem’s input for business, energy, and facility location planning problems. also, an integrated decision making process consists of our proposed method and a fuzzy inference system can be used and the obtained results can be compared with other decision making methods such as topsis, vikor, and electre. ijahp article: karasan/a novel hesitant intuitionistic fuzzy linguistic ahp method and its application to prioritization of investment alternatives international journal of the analytic hierarchy process 140 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.610 references abdel-basset, m., mohamed, m., & sangaiah, a. k. (2018). neutrosophic ahp-delphi group decision making model based on trapezoidal neutrosophic numbers. journal of ambient intelligence and humanized computing, 9(5), 1427-1443. doi: https://doi.org/10.1007/s12652-017-0548-7 arulbalaji, p., padmalal, d., & sreelash, k. (2019). gis and ahp techniques based delineation of groundwater potential zones: a case study from southern western ghats, india. scientific reports, 9(1), 2082. doi: https://doi.org/10.1038/s41598-019-38567-x atanassov, k. (1986). intuitionistic fuzzy sets, fuzzy sets and systems, 20(1), 87-96. doi: https://doi.org/10.1016/s0165-0114(86)80034-3 buckley, j. j. (1985). fuzzy hierarchical analysis. fuzzy sets and systems, 17(3), 233247. doi: https://doi.org/10.1016/0165-0114(85)90090-9 chang, d. y. (1996). applications of the extent analysis method on fuzzy ahp. european journal of operational research, 95(3), 649-655. doi: https://doi.org/10.1016/0377-2217(95)00300-2 ciucci, d. (2016). orthopairs and granular computing. granular computing, 1(3), 159170. doi: https://doi.org/10.1007/s41066-015-0013-y durão, l. f. c., carvalho, m. m., takey, s., cauchick-miguel, p. a., & zancul, e. (2018). internet of things process selection: ahp selection method. the international journal of advanced manufacturing technology, 99(9-12), 2623-2634. doi: https://doi.org/10.1007/s00170-018-2617-2 ilbahar, e., karaşan, a., cebi, s., & kahraman, c. (2018). a novel approach to risk assessment for occupational health and safety using pythagorean fuzzy ahp & fuzzy inference system. safety science, 103, 124-136. doi: https://doi.org/10.1016/j.ssci.2017.10.025 kahraman, c., öztayşi, b., sarı, i̇. u., & turanoğlu, e. (2014). fuzzy analytic hierarchy process with interval type-2 fuzzy sets. knowledge-based systems, 59, 48-57. doi: https://doi.org/10.1016/j.knosys.2014.02.001 ozdemir, s., & sahin, g. (2018). multi-criteria decision-making in the location selection for a solar pv power plant using ahp. measurement, 129, 218-226. doi: https://doi.org/10.1016/j.measurement.2018.07.020 öztaysi, b., onar, s. ç., boltürk, e., & kahraman, c. (2015). hesitant fuzzy analytic hierarchy process. ieee international conference on fuzzy systems (fuzz-ieee). doi: https://doi.org/10.1109/fuzz-ieee.2015.7337948 pipatprapa, a., huang, h. h., & huang, c. h. (2018). enhancing the effectiveness of ahp for environmental performance assessment of thailand and taiwan’s food https://doi.org/10.1016/j.measurement.2018.07.020 ijahp article: karasan/a novel hesitant intuitionistic fuzzy linguistic ahp method and its application to prioritization of investment alternatives international journal of the analytic hierarchy process 141 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.610 industry. environmental monitoring and assessment, 190(12), 748. doi: https://doi.org/10.1007/s10661-018-7113-5 rodriguez, r. m., martinez, l., & herrera, f. (2012). hesitant fuzzy linguistic term sets for decision making. ieee transactions on fuzzy systems, 20(1), 109-119. doi: https://doi.org/10.1109/tfuzz.2011.2170076 saaty, t.l. (1980). the analytic hierarchy process: planning, priority setting, resource allocation. mcgraw-hill international book company. smarandache f. (1995). neutrosophic logic and set, mss, http://fs.gallup.unm.edu/neutrosophy.htm. torra, v. (2010). hesitant fuzzy sets. international journal of intelligent systems, 25(6), 529-539. vladeanu, g. j., & matthews, j. c. (2019). consequence-of-failure model for risk-based asset management of wastewater pipes using ahp. journal of pipeline systems engineering and practice, 10(2), 04019005. doi: https://doi.org/10.1061/(asce)ps.19491204.0000370 vyas, g. s., jha, k. n., & patel, d. a. (2019). development of green building rating system using ahp and fuzzy integrals: a aase of india. journal of architectural engineering, 25(2), 04019004. doi: https://doi.org/10.1061/(asce)ae.1943-5568.0000346 wu, j., huang, h. b., & cao, q. w. (2013). research on ahp with interval-valued intuitionistic fuzzy sets and its application in multi-criteria decision making problems. applied mathematical modelling, 37(24), 9898-9906. doi: https://doi.org/10.1016/j.apm.2013.05.035 xu, z., & xia, m. (2011). distance and similarity measures for hesitant fuzzy sets. information sciences, 181(11), 2128-2138. doi: https://doi.org/10.1016/j.ins.2011.01.028 xu, z., & xia, m. (2011). on distance and correlation measures of hesitant fuzzy information. international journal of intelligent systems, 26(5), 410-425. doi: https://doi.org/10.1002/int.20474 xu, s., xu, d., & liu, l. (2019). construction of regional informatization ecological environment based on the entropy weight modified ahp hierarchy model. sustainable computing: informatics and systems, 22, 26-31. doi: https://doi.org/10.1016/j.suscom.2019.01.015 yager, r. r., & abbasov, a. m. (2013). pythagorean membership grades, complex numbers, and decision making. international journal of intelligent systems, 28(5), 436452. doi: https://doi.org/10.1002/int.21584 ijahp article: karasan/a novel hesitant intuitionistic fuzzy linguistic ahp method and its application to prioritization of investment alternatives international journal of the analytic hierarchy process 142 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.610 yucesan, m., & kahraman, g. (2019). risk evaluation and prevention in hydropower plant operations: a model based on pythagorean fuzzy ahp. energy policy, 126, 343351. doi: https://doi.org/10.1016/j.enpol.2018.11.039 zadeh, l. a. (1965). fuzzy sets. information and control, 8(3), 338-353. zadeh, l. a. (1975). the concept of a linguistic variable and its application to approximate reasoning-1, information sciences, 8, 199-249. doi: https://doi.org/10.1016/0020-0255(75)90036-5 ijahp article: beshah, kitaw/ ahp application in a financial institution international journal of the analytic hierarchy process 54 vol.5 issue 1 2013 issn 1936-6744 ahp application in a financial institution1 birhanu beshah2 mechanical engineering department (industrial engineering stream) addis ababa institute of technology, addis ababa university, addis ababa, ethiopia email: birhanu.beshah@aait.edu.et daniel kitaw mechanical engineering department (industrial engineering stream) addis ababa institute of technology, addis ababa university, addis ababa, ethiopia email: daniel.kitaw@aait.edu.et abstract this paper deals with the application of the analytic hierarchy process (ahp) to solve decision making problems in the development bank of ethiopia project financing process. this paper presents an ahp model to select the most feasible project among alternatives. the model has seven criteria including: technical, market and demand, financial, socio-economic, institutional, discounting and non-discounting factors. for these main criteria there are twenty seven sub-criteria. the application of this method in a financial institution ensures reduction of the project evaluation time and increases reliability of the decision made. furthermore, since the decision made can be traced back to the model at any time, the new method will reduce corruption and promote employees’ confidence. key words: decision making, ahp, project financing 1. introduction project financing is one of the most critical roles of financial institutions, despite the risks associated with it. to minimize project failure after disbursing the finances, financiers carry out a rigorous evaluation and appraisal of the projects being considered for support. these decision making processes often take a long period of time, and are also highly susceptible to corruption. in ethiopia the development bank of ethiopia, the commercial bank of ethiopia, and the construction and business bank are the predominant project financing institutions. this research is mainly focused on the development bank of ethiopia (dbe) since it specializes in project financing. in addition, dbe is an instrument the government uses to 1 we acknowledge addis ababa university, research and post-graduate office for its financial support. we are also grateful to the professional support given by prof. thomas saaty, rozann saaty, daniel saaty, abebe abuye, lelisa dabba and mesenbet shenkite. 2 corresponding author mailto:birhanu.beshah@aait.edu.et mailto:daniel.kitaw@aait.edu.et rob typewritten text http://dx.doi.org/10.13033/ijahp.v5i1.135 ijahp article: beshah, kitaw/ ahp application in a financial institution international journal of the analytic hierarchy process 55 vol.5 issue 1 2013 issn 1936-6744 implement its policy and strategy for the country’s development. basically, dbe uses the following process to finance a project: project proposal preparation by the client, prequalification evaluation of the project, pre-approval evaluation, appraisal of the project, decision by committee and finally the financing and follow up of the project. the decision making process, particularly the two stages of project appraisal and decision by a loan committee, take a long time. at the project appraisal stage, the task of the project evaluator is to provide their recommendation for the loan committee after a thorough analysis. based on the evaluators’ recommendation other people, who do not know the project, decide whether or not to fund it. this system is in place to avoid a conflict of interest and to control the members in the appraisal team. however, since the loan committee does not have in-depth knowledge about the projects, it reduces the transparency, accuracy and reliability of the decision making process. it also extremely prolongs the time of the decision making process because the loan committee needs to write a detailed recommendation after understanding the project. another problem in the bank is that there are decision makers with managerial responsibility who try to safeguard themselves as much as possible. it is indeed an irony to see that people are holding managerial positions, and yet they are not willing to make decision and/or take actions. this state of paralysis on the part of the decision makers leaves the economic sector suffering from inefficiencies due to indecision. managers authorized to make decisions at various levels in the bank are observed to either not make decisions at all, not make decisions on time, or not make the best or even the right decisions. in general, the decision makers in the bank always strive to reduce the risk of a financed project’s failure by taking a longer time to perform the analysis and evaluation. currently the total process requires a minimum of four to six months. these delays create a paradoxical situation of underutilized (idle) cash on the one hand, and a cash hungry (starved) economy on the other. the aim of this study is to introduce a multi-criteria decision making approach known as the analytic hierarchy process (ahp) in the dbe project financing system to reduce the decision making time, and at the same time to increase the accuracy of the decisions made. the analytic hierarchy process (ahp) was developed by thomas l. saaty in the 1970s. it provides a flexible and easily understandable way to analyze and decompose a decision problem. in its general form, it is a framework for performing both deductive and inductive thinking. ahp in essence is a procedure for measuring priorities in a hierarchical goal structure. it requires making pairwise comparison judgments about the criteria to derive their relative importance to the goal and pairwise comparison judgments of the alternatives with respect to the criteria for preference. these judgments can be expressed verbally, and enable the decision maker to incorporate subjective experience and knowledge in an intuitive and natural way (ababutain, 2002). according to sipahi (2010), research in the field of ahp is growing exponentially. there are applications of ahp in the areas of manufacturing, environmental management, agriculture, power and energy industries, transportation industry, construction industry, healthcare education, logistics, e-business, it, r&d, telecommunication industry, finance and banking, urban management, defense industry and military, government, marketing, ijahp article: beshah, kitaw/ ahp application in a financial institution international journal of the analytic hierarchy process 56 vol.5 issue 1 2013 issn 1936-6744 tourism and leisure, archaeology, auditing, and the mining industry. it is widely applicable since it is a general process for determining measurements. there has also been some research conducted on applying ahp in evaluating research projects (liang, 2003; wang, 2005; shin, 2007; sun, 2008; huang, 2008). in banking, the ahp has been used in strategic planning (arbel, 1990). however, up until now, no research has been undertaken on the application of ahp in financial institutions allocating money for projects. therefore, this research will be important for financial institutions in general and the dbe in particular. 2. methodology in order to achieve the aforementioned objectives a step-by-step methodology was followed. first, the critical factors and parameters of the success factors for an industrial project were examined and identified. project financing experts from the development bank of ethiopia, construction and business bank, and commercial bank of ethiopia participated in identifying the success factors. furthermore, these financers’ also provided the basic criteria used to evaluate the projects. after the ahp structure was developed, it was evaluated by a group of experts consisting of six members from the development bank of ethiopia including the vice president for loan service, the head of a technical project study and other experts who evaluate projects during the loan provision process. three projects from the development bank of ethiopia were selected for use in validating the ahp model. finally, a general decision making process for financial institutions is proposed along with an implementation strategy. the limitation of this paper is that it does not explain the details of the actual projects that were evaluated since it is bank policy not to disclose project information to a third party. given that different projects have different characteristics with different success and failure factors, ethiopian banks overall, and the development bank of ethiopia in particular, have developed general criteria to evaluate individual projects from a wide perspective in order to select and finance the most viable projects. some drawbacks to their process are that the criteria do not consider all the factors that affect a project, nor is there a systematic way to measure the effect of individual factors on the overall performance of a project. furthermore, the overall performance of the projects on the criteria cannot be aggregated. the other limitation of the current decision making process is essentially the departmental paralysis. in this study, we structure the criteria in a hierarchy so the decision can be made by a group of experts and professionals making judgments. as a result, development bank of ethiopian should be able to reduce the time required to evaluate and decide on the viability of a project, and can measure the overall effects of all the criteria and at the same time increase the reliability of the decision made. ijahp article: beshah, kitaw/ ahp application in a financial institution international journal of the analytic hierarchy process 57 vol.5 issue 1 2013 issn 1936-6744 3. identification of criteria the identification of project evaluation criteria is the most important stage in the application of ahp for project selection. extensive criteria and sub-criteria were identified including all the characteristic aspects of a project that trigger failure (chandra, 2005; commercial bank, 2005; construction bank, 2006; development bank, 2006; harrison, 1992). however, the decision maker can suspend any criteria which are not relevant to the projects under consideration. the main criteria and the sub-criteria we developed are explained below and summarized in table 1. the main criteria are technical, market and demand, finance, socio-economic and institutional. the sub-criteria are discussed for each criterion. 3.1 technical criteria the main factors considered in evaluating a project from a technical point of view include: the availability of raw materials, appropriate location and site, production capacity, production process and technology, machinery and equipment, project charts and layouts, and schedule of project implementation. the sub-criteria for this criterion include: 1. raw materials: the evaluator should assess the availability of raw materials in terms of right amount, place, and quality. the raw material supply program also should be taken into consideration. there is an intimate relationship between the raw materials and the other parts of the project formulation, particularly those concerned with location, technology, and equipment. the greater the availability of raw material for the designed project, the better it is and the higher the project will rank. 2. production process/technology: appropriate technology refers to those methods of production which are suitable to the local economic, social, and cultural conditions. the production process/technology should be evaluated for: (1) utilization of local raw materials, (2) utilization of local man power, (3) whether the goods or services produced cater to the basic needs of the society, (4) whether the technology protects ecological balance, (5) whether the technology is harmonious with social and cultural condition. the greater the feasibility of the production process/technology, the higher the rank. 3. machinery and equipment: requirements for machinery and equipment depend on the production technology, plant capacity and type of project. the machinery and equipment should be evaluated for: (1) its feasibility to the production technology, plant capacity, and the type of project, (2) availability of spare parts, and (3) availability of machinery operators. the greater the feasibility of machinery and equipment design, the higher the rank. 4. plant capacity: plant capacity (production capacity) should be evaluated for: (1) technological appropriateness, (2) availability of the inputs, (3) amount of investment cost and (4) market availability. the greater the feasibility of plant capacity for the designed project, the higher the rank. 5. location and site: location and site should be evaluated for: (1) proximity to raw materials and markets, (2) availability of infrastructures such as electric power, ijahp article: beshah, kitaw/ ahp application in a financial institution international journal of the analytic hierarchy process 58 vol.5 issue 1 2013 issn 1936-6744 transportation, water, communications and other utilities, (3) labor availability, and (4) government policies. the greater the location and site suitability for the designed project, the higher the rank. 6. project charts and layout: evaluation of project charts and layout includes: (1) general functional layout, (2) material flow diagram, (3) production flow diagram, (4) production line diagram, (5) transport layout, (6) utility consumption layout, (7) communication layout, (8) organizational layout, and (9) layout. the greater the feasibility of the project charts and layout design, the higher the rank. 7. schedule of project implementation: evaluation of project schedule and implementation should include: (1) list of possible activities from project planning to commencement of production, (2) the sequence in which various activities have to be performed, (3) the time required for performing the various activities, (4) the resources normally required for performing the various activities. the greater the feasibility of the schedule of project implementation design, the higher the rank. 3.2 market and demand criteria products and services should be marketable. to ascertain this, project market and demand analysis is vital. these include: the availability of customers for the products/services, identification of other suppliers and/or substitutes for the items in demand and a way to penetrate the market (marketing strategy). the sub-criteria for market and demand criteria include: 1. demand: the demand for the product/services should be evaluated for: (1) the availability of customers for the products/services, (2) the total demand for the products/services in a particular period of time and the life time of the products/services. the greater the demand for the products/services that are produced, the higher the rank. 2. supply: the supply of alternative product/services in the market should be evaluated for: (1) availability and nature of the product and services, (2) local production level, (3) import, and (4) the export level of the product or services. the supply analysis is important to determine the gap between the demand and the supply. the lesser the number of alternative suppliers, the higher the rank. 3. marketing strategy: the evaluators of the marketing strategy should assess the distribution channels, promotional strategy, pricing and positioning of the products/services. the greater the feasibility of the strategy designed, the higher the rank. 3.3 financial criteria to measure the affordability and profitability of a project, financial analyses should be conducted that include the total cost of the project, the cost of production, the means of financing and profitability projections. the sub-criteria of project financial criteria include: 1. cost of the project: project cost should include (1) land and site development, (2) building and civil works, (3) plant and machinery, (4) technical know-how and ijahp article: beshah, kitaw/ ahp application in a financial institution international journal of the analytic hierarchy process 59 vol.5 issue 1 2013 issn 1936-6744 engineering fees, (5) expense of foreign technicians and training, (6) preliminary and capital issue expenses, (7) pre-operative expenses, (8) margin money for working capital, (9) initial cash losses. the less the project cost and affordability, the higher the rank. 2. cost of production: the major costs of production such as material cost, utilities cost, labour costs and factory overhead costs should be evaluated. the less the production cost, the higher the rank. 3. means of finance: the evaluator should assess the potential source of finance for the project implementation and operating cost. the greater the feasibility of financing, the higher the rank. 4. profitability projections: given the estimates of sales revenues and costs of production, the next step is to prepare the profitability projections or estimates of working results. the greater the profitability of the designed project, the higher the rank. 3.4 socio-economic criteria at large, any activity should contribute to a society’s well-being and economic development. in order to justify that this will occur, socio-economic analyses is necessary including: project effect on employment, project effect on net foreign exchange of a country, the impact of a project on net social benefits and welfare. the sub-criteria of socio-economic criteria include: 1. employment effect: while assessing the impact of a project on employment, both unskilled and skilled labor has to be taken into account. not only direct employment, but also indirect employment opportunities created within the project have to be taken into account. furthermore, employment opportunities created in the input and output sides of the project should be considered. the greater the potential employment the project will generate for the country, the higher the rank. 2. net foreign exchange effect: a project may be export-oriented or it may reduce reliance on imports. in such analysis the effect of the project on the balance of payments for the country and the potential for import substitution is necessary. the assessment of the project on the country’s foreign exchange could be done in two stages: first, the balance of payments effects and, second, the import substitution effects of a project. net foreign exchange flows need to be calculated. the greater the positive net foreign-exchange effect on the country, the higher the rank. 3. impact of the project on net social benefits or welfare: the evaluation of the sociological aspect of the project should focus on the extent to which it adequately takes into account four main factors: (1) the socio-cultural and demographic characteristics of local beneficiaries, (2) the social organization of productive activities of the population in the project area, (3) the actual acceptability of the project and its compatibility with the behavior and perceived needs of the intended beneficiaries, and (4) the social strategy for project implementation and operation needed to elicit and sustain beneficiary participation. the greater the net social benefits or welfare of the project designed, the higher the rank. ijahp article: beshah, kitaw/ ahp application in a financial institution international journal of the analytic hierarchy process 60 vol.5 issue 1 2013 issn 1936-6744 4. environmental impact: the evaluation of environmental effects should include: (1) existing environmental base line conditions, (2) potential environmental impacts, including opportunities, for environmental enhancement, (3) preventive, mitigatory and compensatory measures, in the form of action plan, (4) monitoring. to the extent possible, capital and recurrent costs, environmental staffing, training and monitoring requirements and the benefits of proposed alternatives and mitigation measures should be evaluated. the less the negative impact on the environment, the higher the rank. 3.5 institutional criteria during project preparation and analysis, the suitability of the organization, the competence and availability of management and manpower should be examined. the objective is to make sure that the project can be adequately carried out. the sub-criteria of institutional analysis include: 1. management: the management of the project should be evaluated to measure its ability to plan, direct, coordinate, motivate, control and implement the overall activity. project management analyses should include: knowledge of the business and experience, financial management, technical management, personnel management, management ability to cope with changing environment. the greater the ability of management, the higher the rank. 2. manpower: project manpower should be evaluated to understand the worker’s capability in knowledge, wisdom, know-how and character to implement a project idea according to the plan. the greater the ability of the workers, the higher the rank. 3. organization: evaluation of the project’s organization is needed to ensure it has the structural ability to integrate the activities and functions of various departments as well as external organizations involved in the project work. the greater the feasibility of project organization the higher the rank. 3.6 discounting criteria evaluating a project by considering time value of money is important in measuring the worthwhileness of a project. evaluation of a project can be done by using discounting criteria that include: net present value, internal rate of return and benefit-cost ratio. the sub-criteria include: 1. net present value: the evaluator of the project should assess the direct linkage to the objective of maximizing the value of the project by using the net present value. the greater the net present value of the project, the higher the rank. 2. benefit-cost ratio: the valuation of a project with benefit-cost ratio determines the amount of benefit that a project sponsor will earn. the greater the benefit-cost ratio, the higher the rank. ijahp article: beshah, kitaw/ ahp application in a financial institution international journal of the analytic hierarchy process 61 vol.5 issue 1 2013 issn 1936-6744 3. internal rate of return: the evaluation of a project based on the internal rate of return provides the rate of return of the project over its lifetime. the greater the internal rate of return the higher the rank. 3.7 non-discounting criteria it is possible to measure the worthwhileness of a project and prioritize projects through non-discounting criteria evaluation. non-discounting criteria include: payback period, urgency, accounting rate of return. the sub-criteria of the non-discounting criterion include: 1. payback period: an evaluation of the payback period is important in measuring the risk of the project in its life time. the less the payback period the higher the rank. 2. urgency: including an evaluation of the project’s importance and urgency can help in prioritizing and selecting projects. the greater the urgency the higher the rank. 3. the accounting rate of return: evaluating a project’s accounting rate of return is important to measure profitability which relates income to investment. the greater the accounting rates of return, the higher the rank. table 1 main selection criteria and their sub-criteria criteria symbol summary technical raw materials 1rm the greater the availability of raw material, the higher the rank production process 2pp the greater the feasibility of production process, the higher the rank machinery and equipment 3me the greater the feasibility of machineries, the higher the rank plant capacity 4pc the greater the feasibility of plant capacity, the higher the rank location and site 5ls the greater location and site suitability, the higher the rank project charts and layout 6pcl the greater the feasibility of the project charts and layout design, the higher the rank schedule of project implementation 7spi the greater the feasibility of the schedule of implementation, the higher the rank market and demand demand analysis 1da the greater the demand, the higher the rank supply analysis 2sa the lesser the alternative supplies of goods/services expected to be produced in the market, the higher the rank marketing strategy 3ms the greater the visibility of the strategy, the higher the rank ijahp article: beshah, kitaw/ ahp application in a financial institution international journal of the analytic hierarchy process 62 vol.5 issue 1 2013 issn 1936-6744 financial analysis cost of the project 1cp the lesser the project cost, the higher the rank production cost 2pc the lesser the production cost, the higher the rank means of finance 3mf the greater the viability of means of finance, the higher the rank profitability projection 4pp the greater the profitability, the higher the rank socio-economic employment effect 1ee the greater the employment effect, the higher the rank net foreign exchange effect 2nfe the greater the plan to export goods manufactured, the higher the rank impact of the project 3ip the greater the net social benefits, the higher the rank environmental impact 4ei the lesser the environmental impact, the higher the rank institutional managerial analysis 1ma the greater the ability of managers, the higher the rank organization 2og the greater the feasibility of project organization, the higher the rank manpower 3mp the greater the skill of man power, the higher the rank discounting net present value 1npv the greater the npv, the higher the rank benefit-cost ratio 2bcr the greater bcr, the higher the rank internal rate of return 3irr the greater irr, the higher the rank non-discounting urgency 1ug the greater the urgency to implement the project, the higher the rank pay back period 2pbp the lesser the pay back period the higher the rank accounting rate of return 3ar the greater the arr, the higher the rank 4. ahp model based on the above criteria and sub-criteria, an ahp model for project selection was developed. the model is made up of four levels of hierarchy (see figure 1).  the top level or the goal is to select the most viable project(s) to finance.  the second level or the criteria includes the main seven criteria: technical, market and demand, financial, socio-economic, institutional, discounting, and nondiscounting criteria.  the third level or the sub-criteria level includes the 27 sub-criteria discussed under each criterion in the section above.  the fourth level or the alternatives includes the available projects being proposed with the objective being to select the best project from among the alternatives, or to rank all the available projects by their viability. in this case, we have three alternative projects with their unique characteristics. according to reports the first alternative, project 1, had sufficient market, excellent social impact, and the project owners were experienced enough to carry out the project design according to the plan. the second alternative, project 2, was an innovative ijahp article: beshah, kitaw/ ahp application in a financial institution international journal of the analytic hierarchy process 63 vol.5 issue 1 2013 issn 1936-6744 business with an untapped market. there was no question on the profitability of the project, but the technical viability was the main issue raised for discussion. the third alternative, project 3, was a mature business. in addition, the project sponsor did not have experience in modern project implementation and operation, but the project was technically excellent. all three alternatives have their own advantages and disadvantages. the ahp model was built using the super decisions software for testing. the pairwise comparisons for the criteria, sub-criteria and alternatives were carried out by experts from the development bank of ethiopia through group discussion followed by consensus, and finally, the judgment was put into the software. each set of pairwise comparisons was checked for consistency and revised if necessary until the maximum inconsistency was below ten percent, which is considered the minimum standard level in the literature. the results were synthesized throughout the model to yield the overall priorities of the alternatives, shown in table 2, and the criteria and sub-criteria, shown in table 3. the priorities of the criteria rank in the following order: discounting, institutional, nondiscounting, market and demand, financial analysis, socio-economic and technical. the priorities of the alternatives rank as follows: alternative 2, alternative 1 and alternative 3. table 2 final results: overall synthesized priorities of the alternatives ijahp article: beshah, kitaw/ ahp application in a financial institution international journal of the analytic hierarchy process 64 vol.5 issue 1 2013 issn 1936-6744 figure 1 ahp model to select project from alternatives select viable project to finance technical market and demand financial socio-economic discounting non-discounting institutional 5ls 3ms 2sa 1da 7spi 6pcl 4pp 3mf 2pc 1cp 1ma 4ei 3ip 2nfe 1ee 1rm 2og 2pp 3me 4pc 3irr 2cba 1npv 3mp 3arr 2pbp 1ug alternative 2 alternative 1 alternative 3 alternatives criteria sub-criteria objectives ijahp article: beshah, kitaw/ ahp application in a financial institution international journal of the analytic hierarchy process 65 vol.5 issue 1 2013 issn 1936-6744 table 3 overall synthesized priorities of the criteria and sub-criteria criteria weights of criteria sub-criteria weights of subcriteria technical 0.002 1rm 0.102 2pp 0.417 3me 0.096 4pc 0.028 5ls 0.258 6pcl 0.096 7spi 0.003 market and demand 0.168 1da 0.291 2sa 0.590 3ms 0.120 financial analysis 0.142 1cp 0.246 2pc 0.083 3mf 0.424 4pp 0.246 socio-economic 0.117 1ee 0.472 2nfe 0.101 3ip 0.149 4ei 0.278 institutional 0.189 1ma 0.387 2og 0.240 3mp 0.373 discounting 0.204 1npv 0.615 2bcr 0.308 3irr 0.077 non-discounting 0.180 1ug 0.200 2pbp 0.683 3ar 0.117 sensitivity analysis was carried out by varying the weights of the criteria and sub-criteria to determine the stability of the decision reached by using the decision model proposed in this study. sensitivity is performed by selecting a criterion (or sub-criterion) and changing its priority, redistributing the change among the other criteria (sub-criteria), and recalculating the priorities of the alternatives to observe if any change occurred in their ranking. a graphical representation of sensitivity for the discounting criterion is shown in figure 2 and figure 3. criterion priorities are read from the x-axis; the alternatives’ priorities are read from the y-axis. in figure 2, the priority of the discounting criterion, indicated by the vertical line, is set to its original priority of 0.20 (from table 2). the priorities of the alternatives are then read from the y-axis at the points where the vertical line crosses the alternatives’ lines (also given in the legend below the graph: 0.366, 0.432, 0.301). these are the original overall synthesized priorities for the alternatives shown in table 1. ijahp article: beshah, kitaw/ ahp application in a financial institution international journal of the analytic hierarchy process 66 vol.5 issue 1 2013 issn 1936-6744 figure 2 sensitivity graph with discounting criterion priority set to 0.20 figure 3 sensitivity graph with discounting criterion priority set to 0.60 sensitivity is performed by varying the priority of the discounting criterion by moving the vertical line and determining the corresponding alternative priorities. in figure 3, it has been moved to the right to a priority of about 0.60 and the order of the alternatives ijahp article: beshah, kitaw/ ahp application in a financial institution international journal of the analytic hierarchy process 67 vol.5 issue 1 2013 issn 1936-6744 has changed from 2 (0.432), 1 (0.366), 3 (0.201) to 1 (0.424), 2 (0.375), 3 (0.201). the ranks of alternatives 1 and 2 have switched places. in fact, by examining figure 3, we can observe that for any priority for the discounting criterion greater than about 0.43, alternative 1 will rank higher than alternative 2. performing sensitivity on the criteria of technical, financial, socio-economic, institutional and non-discounting did not affect the first ranked alternative, but in some cases the second and third ranked alternatives switched places. similarly, sensitivity analyses of the sub-criteria was also conducted and showed that the priorities of the alternatives will not change. therefore, this analysis reveals the fact that alternative 1 and alternative 2 are competing projects. 5. proposed decision making process in order to select the most viable project by using the new model, a project submitted to the development bank of ethiopia will have to pass through the following stages: client prepares a proposal, client submits a proposal to the bank, checking data and price proposal invoices (the client is expected to submit the profile of potential contractures which can implement the project and their quotation with the proposal to the bank), the bank forms a group of experts in the fields related to the client’s proposal, the group makes the pairwise judgments, checks the consistency of their judgments and finally selects the top ranked project. 1. client prepares proposal – while the client prepares the project proposal, the bank should deliver or introduce all the necessary requirements: the feasibility study, curriculum vitae, and borrowers’ detail, among other things. furthermore, the bank should also provide the available data identified related to the project idea. these simplify working time and reduce money spent by both parties (the bank and the borrower). since the bank has a lot of data in the relevant business area through rigorous research; using the large amount of data in a relevant business area that the bank already possesses through rigorous research to prepare the proposal increases the validity of the proposal, reduces the bank’s time in checking the proposal and reduces the effort of the borrower to find data. hence, providing the necessary information and data for the client to prepare the proposal definitely decreases the effort of both the banker and the client in approving the proposal and allows for a decision in a shorter period of time. 2. client submits the proposal to the bank – at this point, the bank checks the content of the proposal to see if it has all the necessary requirements of the bank. if the proposal does not include a necessary document the bank can provide information on the part of the proposal which needs improvement by the client. if the proposal meets all the necessary requirements a disclaimer agreement between the bank and borrower will be created. the agreement at this stage is basically to create a bridge for information flow about the proposal between the bank and the borrower. 3. check the data and price proposal invoices – before the bank goes into a detailed analysis of the project for its validity, the foundational building blocks (the data ijahp article: beshah, kitaw/ ahp application in a financial institution international journal of the analytic hierarchy process 68 vol.5 issue 1 2013 issn 1936-6744 and invoices) should first be valid. if the data and invoices are correct, an agreement between the bank and the borrower will follow, or the bank may return the proposal to the client for further improvements. 4. forming a group – according to the type and character of projects, a group consisting of experts from different disciplines should be formed that includes: experts from the project area, technical experts, marketing experts, socioeconomic experts, and financial experts, both from among the bank’s employees and external experts. 5. make the pairwise judgments – the group first confirms that the criteria are valid and sufficient. that means, if the group believes that the criteria in the model that was developed are not sufficient they can add additional criteria, or if they found some unnecessary criteria they can eliminate them. finally, the group performs pair-wise comparisons throughout the model. 6. check the consistency – the quality of the decision made is supported by checking the degree of consistency and making sure it is sufficiently low for every set of pairwise comparisons. after the group finishes each set of judgments, they should check the consistency ratio, and if they find it greater than 0.1, they should review their judgments. 7. selection – after the bank’s people perform the analysis and their judgments are combined, the projects will be rank ordered either manually or by using the software available. the top ranked project would be selected as the most viable project that would be further investigated using sensitivity analysis. 6. step-by-step implementation guide even though the new approach developed for project financing can solve many of the dbe funding decision problems, if it is not implemented properly the desired objective will not be achieved. in this study, after developing the model, three projects from the bank were selected to test the model. as was explained in the methodology section, a group was formed including experts and decision makers from the dbe. using the ahp approach described above, the priorities of the projects were established. although, the experts who participated in the group were convinced by the model’s usefulness, it is impossible to directly apply the new approach in the day-to-day activities of the bank because it would require major steps including revision of the policies of the bank. in general, the following activities would have to be appropriately dealt with before implementing ahp modeling in the bank: planning, creating awareness, revising policy, deploying policy, providing training, conducting demonstrations, implementing the new approach, and finally monitoring and reviewing results. 1. planning – a detailed plan of the implementation procedures and instructions should be drafted first. in addition, the necessary budget, the time frame required, and the infrastructure necessary to support the program would have to be established the resources necessary to begin implementing the new approach ijahp article: beshah, kitaw/ ahp application in a financial institution international journal of the analytic hierarchy process 69 vol.5 issue 1 2013 issn 1936-6744 would have to be earmarked and secured. these are important basic steps that need to be taken to prevent mix-ups while the implementation is getting started. 2. creating awareness – when the implementation begins, a financial institution needs to create awareness among top management, employees, evaluators of institutional performance and the society at large. the awareness that should be created includes: the need to specifically identify the criteria for selecting projects, how ahp modeling works and the need to follow the procedures detailed above. in general, the society should be aware that funding decisions in the future will be made systematically and scientifically in a way that can be traced back to the model and that can be used to improve the evaluation process itself as the bank’s people gain experience with the system. 3. revising credit policy – currently the banks use control and check as a method of project financing. in order to apply the ahp in the decision making process, the banks’ credit policies and strategies should be revised to accommodate the new methodology which is group multi-criteria decision making. 4. deploying the policy – the macro level objectives set by the leaders of a bank may not necessarily lead to the desired goals unless they are deployed in the daily activity procedures and instructions. deploying the policy is intended to change the decision making process into a transparent one of manageable size. 5. training – training should be given to all employees of the bank on ahp modeling for making project financing decisions. the training should be given continuously for different types of projects and different cases. this training may also trigger employees to use mcdm in solving other organizational problems. 6. demonstration – demonstrating how the process is working is essential for the society and for the employees. the main objective of the demonstration is to change attitudes about the decision making process and to inculcate new principles of awareness and behavior. 7. implementing – after awareness is created and the policy is revised to support the new paradigm, it can be deployed into procedures and instructions. after the employees are well trained, and have practiced the process that was demonstrated, a totally the new method for project financing can be implemented in a financial institution. 8. evaluating – implementation results need to be checked against a plan. then if there is gap between the objectives and the implementation results, corrective action should be taken. this will lead to continuous improvement of the decision making process. as a result, the quality of the decisions will also be improved. 7. conclusion and recommendations the research aim was to solve problems in the dbe project financing process through the application of the ahp to ensure improvements in the decision making process so that ijahp article: beshah, kitaw/ ahp application in a financial institution international journal of the analytic hierarchy process 70 vol.5 issue 1 2013 issn 1936-6744 more reliable decisions could be made. the time required to evaluate one or multiple projects should not be more than one month. in addition to this, the following conclusions may be drawn from our research.  there are many criteria and variables that affect project selection. this research has developed seven criteria and twenty seven sub-criteria.  though the dbe did not fully implement the ahp model developed in this study, it was tested in the bank. furthermore, the bank now actually considers most of the factors that we included in our project evaluation.  the ahp is a convenient tool for selecting projects for financing. since the decision is made by a group, the method provides an opportunity for experts, government officials, and the banker to participate and check their judgments through a consistency ratio. one of the benefits of using the group decision making process of the ahp is that people who rarely speak to each other learn from the other specialists in the group during the discussions that go on as they try to reach consensus judgments. knowledge and information is shared and enhanced.  implementation of this research result can reduce corruption, facilitate development of the country, reduce wastage of time and labor and support entrepreneurs’ efforts in creating new ventures.  if the results of this research can not be implemented in the suggested step-bystep manner supported by additional research, the targeted objective may not be realized.  this research was focused on project financing (long-term loans). the model developed was designed to work for long-term loans only. hence, different models need to be developed for short-term and medium-term loans. ijahp article: beshah, kitaw/ ahp application in a financial institution international journal of the analytic hierarchy process 71 vol.5 issue 1 2013 issn 1936-6744 references ababutain,y. abdulaziz. (2002). a multi-criteria decision making model for selection of bot toll road proposals within the public sector. thesis for doctor of philosophy, university of pittsburgh. arbel, a. and orgler, ye. (1990). an application of the ahp to bank strategic planning: the mergers and acquisitions process. european journal of operation research, 48(1), 27-37. beshah, b. and kitaw, d. (2013). the evaporating cloud diagram: a tool to resolve conflicts in the ethiopian banks. research journal of finance and accounting, 4(1), 144-148. chandra, praasanna. (2005). projects planning, analysis, financing, implementation, and review, fifth ed. new delhi: tata mcgraw hill publishing co. limited. commercial bank of ethiopia (cbe). (2005). credit policy. construction and business bank (cbb). (2006). credit policy. development bank of ethiopia (dbe). (2006). credit policy. huang, cc., chu, py., and chiang, yh. (2008). a fuzzy ahp application in government sponsored r&d project selection. omega, 36(6), 1038-52. liang, wen-yau. (2003). the analytic hierarchy process in project evaluation: an r&d case study in taiwan. benchmarking: an international journal, 10(5), 445 – 456. saaty, l.thomas. (1990). decision making for leaders analytic hierarchy process approach. pittsburgh, pa: rws publications. shin, c. et al. (2007). applying the analytic hierarchy process to evaluation of the national nuclear r&d projects: the case of korea. progress in nuclear energy, 49 (5), 375-84. sipahi, s, timor, m. (2010). the analytic hierarchy process and analytic network process: an overview of applications. management decision, 48(5), 775 – 808. sun, y. et al. (2008). a group decision support approach to evaluate experts for r&d project selection. ieee transactions on engineering management, 55(1), 158-70. wang, k. et al. (2005). analytic hierarchy process with fuzzy scoring in evaluating multidisciplinary r&d projects in china. ieee transactions on engineering management, 52(1), 119-29. ahp application in a financial institution0f abstract this paper deals with the application of the analytic hierarchy process (ahp) to solve decision making problems in the development bank of ethiopia project financing process. this paper presents an ahp model to select the most feasible project among a... 1. introduction 2. methodology 3. identification of criteria 3.1 technical criteria 3.2 market and demand criteria 3.3 financial criteria 3.4 socio-economic criteria 3.5 institutional criteria 3.6 discounting criteria 3.7 non-discounting criteria 4. ahp model 5. proposed decision making process 6. step-by-step implementation guide 7. conclusion and recommendations references ijahp article: ali, haroon, abdullah, khan/the best manufacturing procedure for the commercial production of urea: using ahp based topsis international journal of the analytic hierarchy process 313 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.636 the best manufacturing procedure for the commercial production of urea: using ahp based topsis yousaf ali ph.d assistant professor department of management science & humanities ghulam ishaq khan institute of engineering sciences & technology topi, swabi, kpk, pakistan; email: yousafkhan@giki.edu.pk muhammad haroon department of chemical engineering ghulam ishaq khan institute of engineering sciences and technology email: u2015264@giki.edu.pk muhammad abdullah department of chemical engineering ghulam ishaq khan institute of engineering sciences and technology email: u2015218@giki.edu.pk amin ullah khan master in engineering management department of management sciences ghulam ishaq khan institute of engineering sciences & technology, topi email: gem1847@giki.edu.pk abstract nitrogen is one of the most significant nutrients needed for the proper growth and development of crops and other plants. in synthetic nitrogen fertilizers, solid urea is the largest source of nitrogen (n) as a nutrient. prilling, granulation, and hybrid systems are the commercial processes used for the production of urea. one of the biggest challenges involved in the determination and implementation of those alternatives is rationalized decision making. the objective of this research study is to evaluate these processes by considering some of the significant attributes like profit, environmental friendliness, process flexibility and reliability to determine which process is the most optimal. the results show that the prilling process is the best technology for urea production. it is the most optimal process in terms of profitability and reliability, and is therefore widely used in the fertilizer industry. prilling is not the best option when it comes to the environment when compared to granulation. the granulation process is not the best fit for the commercial production of urea because it is not a reliable process, especially for high agricultural demands and market competition. the results show that it would be very difficult to keep up with the rapid growth of the population using the granulation process. mailto:yousafkhan@giki.edu.pk mailto:u2015264@giki.edu.pk mailto:u2015218@giki.edu.pk mailto:gem1847@giki.edu.pk ijahp article: ali, haroon, abdullah, khan/the best manufacturing procedure for the commercial production of urea: using ahp based topsis international journal of the analytic hierarchy process 314 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.636 if the environmental and urea quality concerns are considered, the hybrid system is the highest priority and may be preferred. keywords: fertilizers; reliability; urea; prilling; granulation; mcdm; ahp; topsis 1. introduction in the past few decades, there has been a big shift from the use of natural manure to synthetic fertilizers in agricultural activities. in the past, people used natural animal manure for better cultivation because animal manure possesses a considerable amount of nutrients for the growth of crops, especially nitrogen. on the other hand, animal manure contributes to land pollution as well (galloway, 2003). animal manure as a source of fertilizer could not keep up with the increasing demand for crops due to the increasing population of the world. hence, there was a dire need for synthetic fertilizer that is readily available. currently, the fertilizer industry is well-established and is the backbone of the agricultural industry. nitrogenous fertilizers are commonly utilized for agricultural purposes. nitrogen is produced from various resources that are listed in table 1. (rahmanian, 2011). in synthetic nitrogen fertilizers, urea is the largest source of nitrogen, which is a nutrient that contributes to better growth of crops and other plants. approximately 46wt% of urea consists of nitrogen, the highest when compared to other fertilizers. after applying urea to the soil, it is either absorbed directly via diffusion or first converted to ammonium nitrate by bacteria through a urease enzyme and then absorbed by plants as a nutrient. furthermore, solid urea is produced in two different forms, namely prills and granules. the process involved for both forms is almost the same with the differentiation happening at the finishing point when either prilling, granulation, or the hybrid system occurs (fertilizers manual, 2015) 1 . globally, all of these different processes are being practiced continuously in synthetic fertilizers like nitrogen phosphate (np), calcium ammonium nitrate (can), & diammonium phosphate (dap). the basic raw materials for the production of urea are natural gas and nitrogen. nitrogen is always taken from the air while h2 and co2 are taken mostly from natural gas. this is why countries that have natural gas in abundance are producing urea in huge quantities. in pakistan, due to the shortage of natural gas, the production of urea and other fertilizers is highly affected. most of the urea produced in pakistan is through the prilling process, and the rest of the processes are not widely used because of the profitability of the prilling process. in prilling, a tall tower with a rotating bucket at the top is used to spray the molten urea, and hot air is introduced from the bottom of the tower. while descending the tower, molten urea takes the form of small round grains (1mm-2mm in diameter) called prills. if the air is not dry, prills will be hollow and will break easily. however, the hot air carries dust, ammonia, co2 and other harmful gases which are released into the environment. these gases are dangerous to the environment and cause global warming, affecting millions of lives. on the other hand, the manufacturing capacity of the prilling tower is very high because there is no reverse loading. due to this simplicity, the prilling process is very economically effective. furthermore, in the granulation process, a large close rotary drum is used to mix molten urea with seeds of granules through some binding 1 https://www.rothamsted.ac.uk/sites/default/files/rb209-fertiliser-manual-110412.pdf https://www.rothamsted.ac.uk/sites/default/files/rb209-fertiliser-manual-110412.pdf ijahp article: ali, haroon, abdullah, khan/the best manufacturing procedure for the commercial production of urea: using ahp based topsis international journal of the analytic hierarchy process 315 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.636 agents. the toxic gases produced during this process are scrubbed in the scrubbing section for environmental protection. the granulation process is environmentally friendly, but has a low capacity, and therefore possesses a high operating cost. the granulation process is very flexible and the quality of urea can easily be improved. the granules are strong and do not easily break as compared to prills. furthermore, nitrogen loss is another drawback that is connected to prills (rose et al., 2016). the combined form of the prilling and granulation processes is called the hybrid system. the hybrid system or combined process is more flexible and reliable than granulation or prilling alone. the only shortcoming of the hybrid process is that the technology has high initial and operating costs. it is also closely dependent on the needs of the market demand and varies accordingly. most of the urea fertilizer industries are focusing on innovative research where consideration of different alternatives and developmental improvements are the key to successful businesses. one of the biggest challenges involved in the determination and implementation of those alternatives is rationalized decision making (chen, 1992). this research project aims to resolve the complexity of the rationalized decision scenario of determining the best finishing technology for the production of urea fertilizer (hodgett, 2013). the selection of the best manufacturing procedure for the production of urea must not be based only on profit, but also on other factors like environment, flexibility, and reliability. the main purpose of this research is to carry out a comparative study of the prilling process, granulation and the hybrid system. the research study will help industrialists and managers decide the best alternative from among the different procedures of urea production based on the same criteria or factors. an approved questionnaire was distributed among experts such as professors of chemical engineering and working engineers in the urea fertilizer field. the collected data were analyzed by mcdm techniques like ahp and then topsis to determine the rank of the alternatives. the article is organized as follows: introduction, literature review, methodology, results and discussions, and finally the conclusion. table 1 sources of nitrogen forms of nitrogen source organic nitrogen animal manure, plants residue urea commercial fertilizer plant nitrogen phosphate (np) commercial fertilizer plant ammonium nitrate (nh4no3) commercial fertilizer plant 2. literature review there would be no life without nitrogen because nitrogen is one of the most significant nutrients for plants along with phosphorous (p) and potassium (k). the combination of these three nutrients forms an npk-value of a fertilizer in which the npk-value of urea is 1-0-0 (fertilizers manual, 2015). solid urea is the largest source of nitrogen produced in the two forms of prills and granules. nitrogen is released from the earth in the form of gas into the atmosphere which results in a deficiency of nitrogen in the soil. as a nitrogen ijahp article: ali, haroon, abdullah, khan/the best manufacturing procedure for the commercial production of urea: using ahp based topsis international journal of the analytic hierarchy process 316 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.636 fertilizer, urea has wide applications in the agriculture sector covering a wide range of areas (rehmanian, 2015). the chemical and fertilizer industries are facing critical problems like increased manufacturing time, environmental regulations, and costs. most of the urea plants incorporate the prilling technology because of its high capacity and low cost. there are other technologies, like granulation and the hybrid system (prilling with granulation) that can also be used, not only based on costs but also the flexibility of the process, environmental friendliness, sustainability, and effectiveness (aber, 1997). the different physical and mechanical properties of prills and granules are distinguishable and make them suitable as a fertilizer or intermediate material for other chemical industries. the granulation process is very flexible for improving the quality of urea as well as reducing the overall cost. similarly, the prilling process requires a prilling tower with a height range of 40m. in the prilling process, the perforated rotating bucket is fed with urea melt which rotates at a high speed and sprays urea melt in the form of droplets. these droplets are solidified by falling against blown air. this process is not very flexible and produces off-gases while in granulation. the urea melt is sprayed on the recycled seeds of urea along with some other agglomerating agents to form granules of high strength and normal size (2mm-4mm) in a granulation drum (day et al., 1984). the granulation process is very flexible and produces good quality urea with less emission of toxic gases (fertilizer manual, 1998). in the combined or hybrid system, prills from the prilling tower are sometimes introduced to the granulation loop to produce granules with some specific properties, but often the granulation and prilling processes work independently. this process is flexible, but still produces a sufficient amount of greenhouse gases. according to rehmanian, n. et al. (2015), the granulation process is preferred over prilling because of the flexibility of the process and the product quality. the chemical properties of both types of urea are the same, but the physical properties of prilled and granulated urea are different. urea granules are strong and do not cake easily, while prilled urea is weak and formulates into cake easily. however, the strength distribution and size of prilled urea are more uniform than granulated urea. on the other hand, the flexibility of the prilling process and the product quality can be sufficiently improved by installing dispersants rvg with a vibration unit. this will help improve the safety of the prilling process and product quality, and reduce emissions of harmful gases to the environment. according to quin et al. (2017), the prilling process can also be improved by increasing the fall height along with improving the process design. alamdari (2000) used a mathematical model to study the urea finishing processes. the mathematical model was based on the design equations of the prilling tower. the experimental data obtained from the model were compared with the actual tower to bring the model closer to the practical one. similarly, in the granulation drum, a liquid binder is used to bind the liquid urea onto the surface of the seeds. therefore, in granulation, the scaling of the drum with the materials is also unavoidable. furthermore, the combined system of granulation and prilling also faces the same problems as the individual processes. since the granulation process is very flexible, any desired material can be added to it to produce the desired quality of urea (emady et al., 2016). ijahp article: ali, haroon, abdullah, khan/the best manufacturing procedure for the commercial production of urea: using ahp based topsis international journal of the analytic hierarchy process 317 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.636 the nitrogen loss of urea is not only economically inefficient, but also creates an environmental burden. the nitrogen that is released moves from the soil down to the water aquifers and pollutes the natural underground water. it has been determined that if a certain amount of coal is mixed with the raw materials it will reduce the loss of nitrogen from the urea. the treated urea that is produced is known as brown coal-urea and shows a high capability of more efficient use of nitrogen in the long term because of reduced nitrogen loss and the environmental benefits of retaining more nitrogen in the soil (emady et al., 2016). considering the work done in previous studies, the research topic of this study revolves around the evaluation of urea producing processes i.e. prilling, granulation and hybrid (prilling plus granulation). these three alternatives are being considered after a vigorous literature review. the alternatives will be analyzed based on four distinct criteria i.e. profit, environmentally-friendly, flexibility, and reliability. the alternatives along with their distinct criteria are shown in figure 1. until now, most of the research that has been conducted on this topic has only considered a specific characteristic of the process or product, and then decided which of the finishing processes is optimal for the commercial manufacturing of urea (jahanmiri, a. et al. 2013). hence, this study addresses the research gap by considering all of the sustainable factors involved in the manufacturing processes of urea along with an in-depth evaluation of these processes. figure 1 hierarchy diagram of alternatives and criteria the current research is mostly concerned with all of the major multiple factors that must be considered when deciding which finishing process is best for the commercial production of urea. the problem must be addressed by considering all of the important and relevant factors which may affect the decision. these decisions involve complex situations that have many criteria factors to be considered for multiple alternatives. these situations display a need for multi-criteria decision-making (mcdm) techniques such as dematel, topsis, vikor, ahp, and anp, to name a few. for this study, the analytic hierarchy process (ahp) and technique for order of preference by similarity to ideal solution (topsis) are incorporated. the ahp technique was developed by prilling profit environmentfriendly flexibilty reliability granulation profit environmentallyfriendly flexibilty reliability hybrid profit environmentfriendly flexibilty reliability ijahp article: ali, haroon, abdullah, khan/the best manufacturing procedure for the commercial production of urea: using ahp based topsis international journal of the analytic hierarchy process 318 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.636 thomas l. saaty in the 1980s (wind & saaty, 1980). the ahp technique helps derive ratio scales from pairwise comparisons. furthermore, the ahp also extends its usefulness into complex decision-making situations and helps decision-makers make a conclusive decision. in this way, the experts can reach a strong agreement while deciding on any situation (saaty, 1989). various studies have incorporated the ahp for decision making, and clearly shown the authenticity of this method in many situations. one such study employed the ahp with grey relational analysis (gra) for integrated supplier selection (wang et al., 2017). this study also showed that the ahp can work in combination with other techniques, thus improving their analytical capacity. furthermore, another study by dweiri et al. (2016) incorporated ahp for supplier selection in the automotive industry. similarly, karim and karmaker (2016) employed ahp in combination with topsis for the most suitable machine for the current industrial era. these studies show that ahp can fulfill the purpose of decision making in a complex situation with utter efficiency. the topsis technique was first developed by hwang and yoon (1981). it offers an efficient decision-making opportunity for researchers in every field. topsis is based on the concept of distance from the ideal solution. an alternative whose distance is closer to the positive ideal solution is considered the best option. comparatively, the alternative that is near the negative ideal solution is considered the worst case scenario. many studies have incorporated the topsis technique, clearly showing that this method is worthy of consideration. jain et al. (2018) carried out a study to make a selection in the indian automotive industry. the study employed a fuzzy ahp and fuzzy topsis technique for decision making. another study by gupta and barua (2017) incorporated fuzzy topsis and bwm for supplier selection among smes. similarly, another study by mavi et al. (2016) also used fuzzy topsis for supplier selection in supply chain risk management. based on the above-mentioned studies, there is enough literature suggesting that ahp and topsis can both prove to be efficient options for decision-making scenarios. 3. methodology the complex problem is to determine and implement the most optimal finishing process for the commercial production of urea with rationalized decision making. this research project uses the ahp and topsis techniques to solve this complex problem. 3.1 data collection an approved questionnaire was used to collect data from experts in the fertilizer industry such as professors, engineers, and industrial managers. there were a total of ten respondents whose breakdown is given in table 2. the ahp technique was used to rate all of the supposed criteria with relative weights. in the second phase, these weights were used as input for the topsis technique to evaluate the designated alternatives of the present research study. ijahp article: ali, haroon, abdullah, khan/the best manufacturing procedure for the commercial production of urea: using ahp based topsis international journal of the analytic hierarchy process 319 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.636 table 2 respondents of the questionnaire-based survey qualification no of respondents professors 3 engineers 4 managers 3 3.2 ahp ahp was first developed by thomas l. saaty in 1980, and is currently used extensively when there are multi-criteria that significantly affect a single decision (saaty, 1990; ali et al, 2017a&b). the ahp has been shown to be the best option of an effective tool in decision-making. it helps researchers and decision-makers make the best decision by setting priorities. all of the alternatives are simultaneously compared to each criterion in the form of a pairwise comparison matrix. a pairwise comparison matrix is established to cross-compare the alternatives as well as criteria. pairwise matrices are formed when the ahp reduces complex situations. the ahp will rank or categorize all of the alternatives from the most to the least optimal. this is why the method is known as the analytic hierarchy process. the ahp can effectively help answer the question ‘which one’ among different alternatives. it helps capture the subjective and objective characteristics of a decision, and assists in evaluating the consistency of a decision that a researcher has made after the analysis. in this way, the inconsistent factors can be ruled out. the ahp is a linear procedure and should be used in problems where the alternatives are distinctive rather than very fuzzy. the steps involved in the ahp process are shown in figure 2.the problem is to determine the most optimal finishing process for the commercial production of urea, and this research study will help managers and industrialist select the best procedure. 1. the finishing procedures for the production of urea that have been designed and developed are prilling, granulation, and the hybrid system. the most relevant and distinguishable factors for these processes are profits, environmentally-friendly, flexibility, and reliability. 2. 3. the linguistic scale used in this research study is a nine-point scale. 4. first, the data for the pairwise comparison matrices are collected with the help of a survey questionnaire. the pairwise comparison matrices are created using this data. the criteria of the pairwise matrix are equally represented as: a=[aij];i,j=1,2,3,4………n (1) the entries of the comparison matrices follow three simple rules as: aij>0, aji = 1/aij ; ajj = aii = 1 ijahp article: ali, haroon, abdullah, khan/the best manufacturing procedure for the commercial production of urea: using ahp based topsis international journal of the analytic hierarchy process 320 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.636 normalized matrices result from the pairwise comparison matrices being made stochastic. then, the equation below is used to calculate the maximum eigenvalue for the normalized matrix; 𝜆𝑚𝑎𝑥 = 1 𝑛 ∑ ( 𝑖𝑡ℎ 𝑒𝑛𝑡𝑟𝑦 𝑖𝑛 𝐴𝑊 𝑖𝑡ℎ 𝑒𝑛𝑡𝑟𝑦 𝑖𝑛 𝑊 ) 𝑛 𝑖=1 (2) to check whether the matrices are consistent or not; we 𝐶𝐼 = (𝜆𝑚𝑎𝑥 − 𝑛) (𝑛 − 1) ⁄ (3) n is the number of alternatives. to calculate ci, random index (ri) values are required which are given in table 3. table 3 random index (ri) for different values of n n 2 3 4 5 6 7 8 9 10 ri 0 0.58 0.90 1.12 1.24 1.32 1.41 1.45 1.49 ijahp article: ali, haroon, abdullah, khan/the best manufacturing procedure for the commercial production of urea: using ahp based topsis international journal of the analytic hierarchy process 321 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.636 figure 2 flow diagram for ahp based model if cr = ci/ri < 0.10, then the degree of consistency is satisfactory. if cr>0.10, serious inconsistencies may exist. 5. finally, the optimum alternative is the one that has the greatest value in the following expression: 𝐴𝐻𝑃𝑖 = ∑ 𝑎𝑖𝑗 ∑ 𝑎𝑖𝑗 𝑚 𝑖=1 × 𝑤𝑗 𝑛 𝑗=1 (4) define problem & its objectives literature review expert’s opinion rank the alternatives by relative weights do serious inconsistencies exist? positive alternatives & criteria set a linguistic scale for ahp calculate priority vectors develop pairwise comparison and normalized matrices chec k cr? ijahp article: ali, haroon, abdullah, khan/the best manufacturing procedure for the commercial production of urea: using ahp based topsis international journal of the analytic hierarchy process 322 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.636 through an ahp analysis, the relative weights of the criteria and the optimal solution of the present problem were calculated. these results are used as an input for the topsis technique to determine the most optimal finishing procedure for the production of urea. 3.3 topsis topsis (technique of order preference by similarity to ideal solution) is one of the classical mcdm techniques developed by hwang and yoon (markovic, 2010). it evaluates alternatives with respect to the ideal solution. the alternative that is closer to the ideal solution is preferred by topsis (ali et al, 2018). therefore, it develops a space for each criterion in which the alternative (ai) is represented by a point. the decision matrix assigns criterion values to the coordinates of those points. next, the ideal and negative ideal solutions are hypothesized, which represents the most optimal points in that defined space. topsis decides the best alternative, the one that has the shortest euclidean distance from the ideal solution, and is simultaneously furthest from the negative ideal solution (ishfaq et al., 2018). the ideal alternative can be defined in terms of the best attribute values i.e. maximum benefit criteria and minimum cost criteria. similarly, the negative ideal solution can be defined in terms of having the worst attributes such as the minimum benefit attributes and the maximum cost attributes (cheng et al., 2018). suppose that m represents the number of alternatives and n the number of criteria. similarly, xij is the score of the i th alternative with respect to the j th criterions. j is assumed to be the set of positive attributes, while j’ is a set of negative attributes. step 1. develop a decision matrix let, a = [xij] be a decision matrix and w = [w1, w2, …., wn ] be a weight vector of the criteria determined from the ahp analysis in the previous section, where xij & wij are real numbers. step 2. determine the normalized decision matrix decision matrices have criteria values that must be normalized into non-dimensional numbers. there are different methods for normalizing decision matrices. the most frequently used formula for normalization is: 𝑛𝑖𝑗 = 𝑋𝑖𝑗 √∑ 𝑋𝑖𝑗 2𝑚 𝑖=1 (5) for i = 1, 2, 3, …., m and j = 1, 2, 3, …., n step 3. calculate the weighted normalized matrix relative importance weights wij are multiplied with the corresponding entries of the normalized matrix; 𝑆𝑖𝑗 = 𝑤𝑖𝑗 × 𝑛𝑖𝑗 (6) step 4. determine the positive and negative ideal solution the positive ideal solution has the largest values of positive attributes and the lowest values for negative (cost) attributes. it maximizes the positive criteria while minimizing ijahp article: ali, haroon, abdullah, khan/the best manufacturing procedure for the commercial production of urea: using ahp based topsis international journal of the analytic hierarchy process 323 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.636 the negative criteria. the negative ideal solution is the reverse of the positive ideal solution. a + is the set of positive ideal alternatives: 𝐴+ = (𝑆1, 𝑆2, … . . , 𝑆𝑛) = [(max 𝑆𝑖𝑗 |𝑗 ∈ 𝐽), (min 𝑆𝑖𝑗 |𝑗 ∈ 𝐽 ′)] (7) ais representing a negative ideal solution in the form of: b 𝐴− = (𝑆1, 𝑆2, … . , 𝑆𝑛) = [(min 𝑆𝑖𝑗 |𝑗 ∈ 𝐽), (max 𝑆𝑖𝑗 |𝑗 ∈ 𝐽 ′)] (8) step 5. calculate the euclidean distance (separation) from the positive and negative ideal solution topsis evaluates each alternative on the basis of its euclidean distance or separation from the ideal solution. the euclidean distance from the positive ideal solution is given as; 𝑑+ = √∑ ( 𝑆𝑖𝑗 − 𝑆𝑗 + )2𝑛𝑗=1 , 𝑖 = 1, 2, … . . , 𝑚. (9) the euclidean distance of each of alternative from the negative ideal solution is 𝑑− = √∑ ( 𝑆𝑖𝑗 − 𝑆𝑗 − )2𝑛𝑗=1 , 𝑖 = 1, 2, … . . , 𝑚. (10) step 6. calculate the relative closeness to the positive ideal solution the relative closeness of an alternative ai with respect to a + is; 𝑅𝐶𝑖 = 𝑑𝑖− 𝑑𝑖 − + 𝑑𝑖 + , 0 ≤ 𝑅𝐶𝑖 ≤ 1 , 𝑖 = 1, 2, … … , 𝑚. (11) step 7. rank the alternatives finally, each alternative has a certain value of rc. the alternative which has an rc value more close to 1 is the most optimal alternative for the given problem. 4. results and discussion the present research study is related to the evaluation of different finishing procedures for the commercial production of urea. the ahp technique with topsis was used to evaluate prilling, granulation, and the hybrid system which are the best current technologies for urea production. according to the ahp analysis, the hybrid or combined system has the top ranking among the other alternatives (table 4.). this is because the hybrid system is a combination of the prilling and granulation processes. therefore, the beneficial properties of both systems combine to give an optimal solution. in this study, we assumed a 50% contribution from both the prilling and granulation process in the hybrid system. if the percentage is skewed more towards prilling or granulation, the results would also change. the hybrid system also closely depends on market demand, and that is why its application varies based on the needs of the market. ijahp article: ali, haroon, abdullah, khan/the best manufacturing procedure for the commercial production of urea: using ahp based topsis international journal of the analytic hierarchy process 324 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.636 table 4 ranking of alternatives in ahp analysis alternative relative weights ranks hybrid system 0.54 1 granulation 0.24 2 prilling 0.22 3 the main processes are prilling and granulation, which are also mostly applied in the fertilizer industry (shrev, 1998). the granulation and prilling processes are almost equally preferred. environmentally, granulation is ranked first based on the relative weights of the ahp, which is the same result that shrev (1998) obtained. the flexibility of the granulation process is also greater than that of the prilling process as shown in table 5. since the quality of urea is strongly dependent on the process flexibility, the quality of the granulated urea will be higher than the prilled urea according to rahmanian (2015). the reliability of granulation is a bit less than prilling because of the high loop loading of the granulation process. due to this problem, granulation is not recommended for higher capacity requirements. table 5 relative importance weights of each alternative alternatives profit environmentfriendly flexibility reliability prilling process 0.1212 0.020 0.0080 0.0634 granulation process 0.040 0.1400 0.02811 0.0310 hybrid system 0.2400 0.0456 0.0587 0.20607 as mentioned above, the relative weights of criteria were used as an input for topsis in order to evaluate the prescribed alternatives. the relative importance weights from the ahp analysis are given in figure 3. ijahp article: ali, haroon, abdullah, khan/the best manufacturing procedure for the commercial production of urea: using ahp based topsis international journal of the analytic hierarchy process 325 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.636 figure 3 relative weights of each criterion from the ahp analysis according to the topsis analysis, prilling is the most optimal finishing process for the commercial production of urea. the hybrid system ranked second followed by granulation which ranked third (table 6.). the rankings were formulated based on the relative closeness of each alternative. in a positive ideal solution, prilling is selected as the ideal with respect to profit and reliability. this means that the prilling process is the most profitable and reliable for the production of urea (k.m. constant, 1992). table 6 ranking of alternatives in topsis analysis alternative relative closeness ranks prilling 0.76 1 hybrid system 0.48 2 granulation 0.40 3 granulation is an ideal process for the commercial production of urea with respect to the environment and process flexibility. since granulation is not as profitable and reliable as the prilling process, it ranked the lowest in the topsis analysis. due to these reasons, the granulation process is rarely used in industry when compared to the prilling process (baba, 2012). the rankings are depicted in table 6. 40% 20% 10% 30% relative weights of criteria profit environment flexiblity relaibility ijahp article: ali, haroon, abdullah, khan/the best manufacturing procedure for the commercial production of urea: using ahp based topsis international journal of the analytic hierarchy process 326 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.636 table 7 positive ideal solution criteria prilling granulation hybrid system profit 0.31 0.1833 0.174 environmental 0.164 0.0356 0.096 flexibility 0.03714 0.0711 0.05975 reliability 0.1455 0.1102 0.2381 table 8 negative ideal solution criteria prilling granulation hybrid system profit 0.31 0.1833 0.174 environmental 0.164 0.0356 0.096 flexibility 0.03714 0.0711 0.05975 reliability 0.1455 0.1102 0.2381 from tables 7 and 8, it is clear that prilling is close to the positive ideal solution in profit and reliability, but closer to the negative ideal solution in environment and flexibility as shown by the colored boxes. the reason for creating two tables was to identify the values for the positive and negative ideal solutions, respectively. the question is, why was prilling still ranked first? the reason is because the ahp analysis gave higher weights to profit and reliability and lower weights to environment and process flexibility. in table 9 the separation of each alternative from the positive (si) and negative ideal solutions (si’) are listed. table 9 euclidean distance from positive & negative ideal solutions alternative si si ’ result prilling 0.162 0.53 0.76 hybrid system 0.18 0.116 0.48 granulation 0.149 0.139 0.40 ijahp article: ali, haroon, abdullah, khan/the best manufacturing procedure for the commercial production of urea: using ahp based topsis international journal of the analytic hierarchy process 327 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.636 this research study used mcdm techniques (ahp and topsis) to determine the most optimal process for the production of urea. the results and methodology of the present study only focus on urea, but almost the same results would be obtained if we focused on other types of production facilities. other methods, such as experimentation and forecasting with historical data may also be used to determine the best process for urea production 5. conclusions urea is most widely used as a nitrogen fertilizer in agricultural activities. factors like profit/cost, environment, process flexibility, and reliability are the most significant decision attributes for determining the optimal technological finishing process (prilling, granulation, and hybrid system) for the commercial production of urea. the ahp and topsis results confirm that prilling is the best finishing process for urea production as a whole. prilling is an ideal process with respect to profitability as well as reliability. furthermore, it has some greater environmental concerns when compared to granulation. granulation is not the best fit for commercial production of urea because of its low profitability and reliability when compared to others. if only the granulation process was used, it would not meet the market demand and therefore may not be able to cope with the rapid growth of the population. one of the positive attributes of this process is that it is the most environmentally safe process when compared to the others. since process flexibility is almost analogous to product quality, the present analysis also confirms that the quality of granulated urea is higher than that of prilled urea. this analysis also demonstrates that granulation is not capable of being widely used as a commercial process, especially for high agricultural demands. if the concerns about environment and process flexibility are high, then the hybrid system may be preferred. although this research study only focused on urea production, the same results may be obtained for other fertilizers because the processes are the same in all cases. however, since only fertilizer (urea) was used as a case study these results might not be exactly in accordance with other fertilizer or chemical facilities. therefore, it can be concluded that the prilling process has been shown to be more reliable for the market demand, and the government should focus on sustainable urea production processes to avoid risk to the general population and environment. ijahp article: ali, haroon, abdullah, khan/the best manufacturing procedure for the commercial production of urea: using ahp based topsis international journal of the analytic hierarchy process 328 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.636 references alamdari, a. jahanmiri, n. rahmaniyan (2000). mathematical modeling of the urea prilling process. chemical engineering communication, 178, 185-198. doi: https://doi.org/10.1080/00986440008912182 aber, j.d. et al. (1997). human alteration of the global nitrogen cycle: sources and consequences, ecological appliations, 7(3), 7-50. ali, y., iftikhar, n. and edwin, a.c. (2017a). assessment of career selection problems in developing countries: a mcdm approach. international journal of the analytic hierarchy process, 9(2), 219-243. doi: https://doi.org/10.13033/ijahp.v9i2.488 ali, y., asghar, a., muhammad, n. and salman, a. (2017b). selection of a fighter aircraft to improve the effectiveness of air combat in the war on terror: pakistan air forcea case in point. international journal of the analytic hierarchy process, 9(2), 244-273. doi: https://doi.org/10.13033/ijahp.v9i2.489 ali, y., aslam, z., dar, h.s. and mumtaz, u., 2018. a multi-criteria decision analysis of solid waste treatment options in pakistan: lahore city—a case in point. environment systems and decisions, 38(4), 528-543. doi: https://doi.org/10.1007/s10669-018-9672-y baba, s. (2012). characterisation of strength and structure of granular and prilled urea fertilizer. petronas: universiti technology petronas. chen, s. (1992). fuzzy multiple attributes decision making: methods and applications. heidelberg: springer-verlag. cheng, j. et al. (2018). structural optimization of a high-speed press considering multisource uncertainties based on a new heterogeneous topsis. applied sciences, 8(1), 126. doi: https://doi.org/10.3390/app8010126 constant, k.m. (1992). world nitrogen survey. world bank technical paper 174. day, j. h., lees, r. e., clark, r. h. & pattee, p. l. (1984). respiratory response to formaldehyde and off-gas of urea-formaldehyde foam insulation. canadian medical association journal, 131(9), 1061. dweiri, f., kumar, s., khan, s. a. & jain, v. (2016). designing an integrated ahp based decision support system for supplier selection in the automotive industry. expert systems with applications,62, 273-283. doi: https://doi.org/10.1016/j.eswa.2016.06.030 emady, h., hapgood, k. & smith, r. (2016). granulation and tabletting. cham: springer. galloway, j. n. et al. (2003). the nitrogen cascade. bioscience, 53(4), 341-356. https://doi.org/10.13033/ijahp.v9i2.489 ijahp article: ali, haroon, abdullah, khan/the best manufacturing procedure for the commercial production of urea: using ahp based topsis international journal of the analytic hierarchy process 329 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.636 gupta, h. & barua, m. k. (2017). supplier selection among smes on the basis of their green innovation ability using bwm and fuzzy topsis. journal of cleaner production, 152, 242-258. doi: https://doi.org/10.1016/j.jclepro.2017.03.125 hodgett, r. (2013). multiple criteria decision making in whole process design. newcastle university publisher. hwang, c.-l. & yoon, k. (1981). methods for multiple attribute decision making. berlin: springer. jahanmiri, a. et al. (2013). mathematical modeling of the urea prilling process. chemical engineering communication, 178(1), 1-19. doi: https://doi.org/10.1080/00986440008912182 jain, v. et al., 2018. supplier selection using fuzzy ahp and topsis: a case study in the indian automotive industry. neural computing and applications, 29(7), 555-564. doi: https://doi.org/10.1007/s00521-016-2533-z karim, r. & karmaker, c. (2016). machine selection by ahp and topsis methods. american journal of industrial engineering, 4(1), 7-13. mavi, r. k., goh, m. & mavi, n. k. (2016). supplier selection with shannon entropy and fuzzy topsis in the context of supply chain risk management. procedia-social and behavioral sciences, 235(24), 216-225. doi: https://doi.org/10.1016/j.sbspro.2016.11.017 doi: https://doi.org/10.1016/j.sbspro.2016.11.017 markovic, z. (2010). modification of topsis method for solving of multi-criteria tasks. journal of operation research, 20, 117-143. quin, b. f., harold, s., spilsbury, s. & bates, g.,(2017). commercialisation of onesystem (wetted, nbpt-treated prilled urea) in newzealand and victoria, australia, s.l.: fertilizer and lime research centre. rahmanian, n. et al. (2011). effect of process parameters on the granule properties made in a high shear granulator. chemical engineering research and design, 89(5), 512-518. doi: https://doi.org/10.1016/j.cherd.2010.10.021 rehmanian, n. et al. (2015). urea finishing process: prilling vs granulation. procedia engineering, 102, 174-181. doi: https://doi.org/10.1016/j.proeng.2015.01.122 rose, m. t. et al. (2016). a slow-release nitrogen fertilizer produced by simultaneous granulation and superheated steam drying of urea with brown coal. chemical and biological technologies in agriculture, 3(1), 10. doi: https://doi.org/10.1186/s40538016-0062-8 ishfaq, s., ali, s. and ali, y., 2018. selection of optimum renewable energy source for energy sector in pakistan by using mcdm approach. process integration and ijahp article: ali, haroon, abdullah, khan/the best manufacturing procedure for the commercial production of urea: using ahp based topsis international journal of the analytic hierarchy process 330 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.636 optimization for sustainability, 2(1), 61-71. doi: https://doi.org/10.1007/s41660-0170032-z saaty, t. l., 1989. group decision making and ahp. berlin: springer, berlin. wang, t.-k., zhang, q., chong, h.-y. & wang, x. (2017). integrated supplier selection framework in a resilient construction supply chain: an approach via the analytic hierarchy process (ahp) and grey relational analysis (gra). sustainability, 9(2), 289. doi: https://doi.org/10.3390/su9020289 wind, y. & saaty, t. l. (1980). marketing applications of the analytic hierarchy process. management science, 26(7), 641-658. doi: https://doi.org/10.1287/mnsc.26.7.641 ijahp article: botelho/analyzing the ahp priority vectors: going beyond inconsistency indexes international journal of the analytic hierarchy process 1 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.922 analyzing the ahp priority vectors: going beyond inconsistency indexes marcelo neto botelho instituto de educação tecnológica brazil marcelo.botelho1@hotmail.com abstract a great concern when utilizing the analytic hierarchy process (ahp) is how the final priority vector, resulting from the inconsistent analysis, behaves when compared with the original priority ranking, resulting from the consistent analysis. the ahp utilizes an inconsistency index to predict rank reversal. in addition to the original inconsistency index of the ahp, several authors have worked on developing alternative inconsistency indexes, with the goal of improving the predictability of rank reversal. however, inconsistency indexes do not help clarify whether a rank reversal is a rejectable outcome or, to some extent, the correct answer. a rank reversal may express the correct priority, particularly when some positions in the original priority rank have small weight differences among them. therefore, it is very important to develop a method to allow a clear and definitive analysis on how disturbed the weights and ranking of the final priority vectors are when compared to their original consistent rankings. such a method is developed here and its utilization is demonstrated by analyzing a corporate governance scenario. keywords: ahp; mcda; rank reversal; inconsistency index; corporate governance 1. introduction the ahp works to support decisions as different criteria and alternatives are considered. these criteria and alternatives can combine objective and subjective parameters. the ahp, as proposed by saaty (1977), generates a priority vector from a pairwise comparison matrix (pcm). such pairwise comparison, made by the decision makers (dm) involved in the respective analysis, carries a certain level of inconsistency derived from the inherent subjectivity of human scrutiny. this ability of the ahp to absorb a certain level of inconsistency is highly valuable. saaty (1997) emphasizes that the ahp places special focus on integrating human judgment into decision-making process and on evaluating the consistency of such judgments. saaty (1977) also comments that a certain level of inconsistency in a pcm does not necessarily affect the ranking of the final priority vector. grybowski and starczewski (2020) introduced the swfr – significantly-wrong-finalranking – as a concept to analyze whether a certain rank reversal should represent the rejection of the final priority vector (fpv). buede and maxwell (1995) state that “through formal and informal discussions about rank reversal, the focus has been on mailto:marcelo.botelho1@hotmail.com ijahp article: botelho/analyzing the ahp priority vectors: going beyond inconsistency indexes international journal of the analytic hierarchy process 2 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.922 assessing whether the problem exists, what the reasons are, as well as whether the rank reversal is the problem or the desired response”. wang and triantaphyllou (2008) investigated the rank reversal that occurred in a case of mcda, not only analyzing the inconsistency index, but analyzing the changes that actually occurred in the ranking positions. different authors have worked on the development of other inconsistency indexes besides the one originally proposed by saaty (2009). saaty’s inconsistency index is defined as 𝐼𝐶 = (λ𝑚𝑎𝑥 − 𝑛)/(𝑛 − 1) where λmax is the maximum eigenvalue of the pcm. bozóki and rapcsák (2015) presented alternative inconsistency indexes that have their particular threshold of acceptability, while grzybowski and starczewski (2020) introduced the so-called ic atia. however, although these are serious contributions to the assessment of the inconsistency in a pairwise comparison, these indexes do not contribute to the assessment of the real impact on final ranking positions. furthermore, it is important to note that even in the case of no rank reversal, an investigation into the disturbance that occurred in the fpv may be of interest. cases, for example, that involve the allocation of resources among different alternatives may imply a deeper evaluation on the weight distribution along the ranking positions, even if no reversal has been observed. finally, a certain magnitude of rank reversal can be accepted, as it can represent proper prioritization analysis. for example, this occurs in cases where the dm made some pairwise comparisons considering a very similar level of relevance (in the limit, a tie). in these cases, the analysis of inconsistencies can lead to some sorting reversal that, in fact, is adequately clarifying and segregating the criteria judgement performed by the dms. these considerations support the need to establish a methodology that goes beyond inconsistency indexes to allow an easy, clear, complete and definitive analysis of the fpv generated via the ahp. to test the method and metrics developed here to evaluate the fpv, an application in a corporate governance scenario is analyzed. a governance maturity analysis has been chosen since, generally, the relevance of the governance criteria do not differ that much. the subjectivity of governance issues is highlighted by ibgc (2015) 1 “in the exercise of corporate governance, the topics dealt with are often subjective and ambiguous, which requires from the governance agents the ability to assess, reason and judge”. the development of the proposed methods and the results regarding the prioritization of the governance theme are presented in this study. the analysis of governance issues, with the support of the methods developed here, also compared the use of the original scale by saaty (oss) (1997) and the generalized balanced scale (gbs), as proposed by goepel (2018). 2. technical background 2.1 the original ahp methodology the ahp considers the perron-frobenius theory, where the maximum autovector (maximum eigenvector) of a matrix containing positive values forms the priority vector. furthemore, saaty (1977) stipulates that a pairwise comparison is the best 1 ibgc: brazilian institute of corporate governance. ijahp article: botelho/analyzing the ahp priority vectors: going beyond inconsistency indexes international journal of the analytic hierarchy process 3 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.922 method that humans can use to compare different criteria or alternatives. saaty (1977) refers to the studies by weber (1846) and fechner (1860), which focus on the evaluation of human response to stimuli, to define the scale adopted in the original ahp methodology. saaty (1977) defines a scale from 1 to 9 (original saaty scale – oss), linked to nine levels of criteria relevance. the pairwise comparison can introduce an inconsistency into the pairwise comparison matrix (pcm). this inconsistency is a result of the fact that dms may not maintain perfect proportionality among their comparisons in their pairwise comparisons. when the comparison maintains proper proportionality, there will be a consistent pcm and when not, an inconsistent one. in principle, the inconsistency embodies the subjectivity of human judgements. due to such inconsistencies, saaty (1977) defined the consistency ratio (cr) with a threshold of 10% 2 as the limit for accepting the ahp autovector. the figures below summarize the main steps and concepts of the ahp methodology. figure 1 original saaty scale (oss) figure 2 pairwise comparison matrix (pcm) the final priority vector (fpv) w(i,j) calculated in the ahp method is defined as: [w1,j] = [ai,j] x [c1,j], where: [w1,j] = final priority vector (fpv); [ai,j] = matrix of normalized priority vectors of the alternatives [c1,j] = criteria eigenvector and the saaty inconsistency index is defined as 𝐶𝐼 = ((λ𝑚𝑎𝑥 − 𝑛))/((𝑛 − 1)), where λmax = pcm autovalue and n = number of criteria. 2 consistency ratio: the relation between ci and ri. ri is the random consistency index as per saaty (1987) 1 2 3 4 5 6 7 8 9 equal importance intermediate moderate importance intermediate strong importance intermediate very strong importance intermediate extreme importance c1 c2 c3 c4 ....... cn c1 1,0 (p1/p1) p2/p1 p3/p1 p4/p1 ....... pf/p1 c2 p1/p2 1,0 (p2/p2) p3/p2 p4/p2 ....... pf/p2 c3 p1/p3 p2/p3 1,0 (p3/p3) p4/p3 ....... pf/p3 c4 p1/p4 p2/p4 p3/p4 1,0 (p4/p4) ...... pf/p4 ....... ....... ....... ....... ....... ....... ...... cn p1/pn p2/pn p3/pn p4/pn ...... 1,0 (pf/pn) pairwise comparison matrix pcm parameters (p1 to pf = weights 1 to 9) of criteria (c1 to cn) diagonal = 1,0 once a criteria is compared with itself first line = consistent comparisons (all criterias compared with first one) region of potential inconsistencies (comparisons not obligatorily proportional to fisrt line ) region of reciprocal comparisons ijahp article: botelho/analyzing the ahp priority vectors: going beyond inconsistency indexes international journal of the analytic hierarchy process 4 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.922 2.2 a case study: corporate governance maturity assessment to test the method and metrics developed here to assess the fpv, an application in a corporate governance scenario was analyzed. to this end, a methodology was structured to assess the maturity of governance based on the proposal by álvares, giacometti and gusso (2008). botelho (2021) presents more details about the choice of this methodology and the tool that was developed to assess the maturity of corporate governance. the evolution of each element is fictitious (ranging from 30% to 100%) and the pairwise comparison of the criteria involved in this governance maturity was obtained from directors invited from seven different companies. the ahp methodology was inserted into the aforementioned governance analysis tool, generating an agenda where, taking into account the pairwise criteria comparison provided by the seven board members, the elements' priorities (the alternatives) were prioritized. therefore, this exercise allowed a comparison between the original ranking (derived from consistent pcm) and the fpv. in addition, a comparison was also made between the fpv and the priority agenda organized via the ahp. last but not least, both sets of comparisons were performed using oss and gbs balances. figure 3 presents a summary of the mentioned tool, and figure 4 shows an example of a priority agenda generated by the ahp. figure 3 hierarchy analysis of governance maturity criterias -> dimensions -> 1.1.1 values 67% 2.1.1 corporate 50% 2.2.1stock exchange 83% 3.1.1right to vote and oversight 33% 3.2.1 responsibility to employees 90% 4.1.1 adm. council (board) 66,7% 4.2.1adm. council (board) 83% 4.3.1 member assembly 100% 5.1.1ceo hiring and succession 67% 5.2.1capital structure 50% 1.1.2mission and vision 88% 2.1.2tax and social security 50% 2.2.2sec & sox 38% 3.1.2profit sharing 50% 3.2.2 responsibility to the market 40% 4.1.2 committe es 30,0% 4.2.2 committees 30% 4.3.2 shareholde r's assembly 92% 5.1.2ceo performance and compensation assessment 100% 5.2.2 conflict resolution mechanism 100% 1.1.3code of ethics 50% 2.1.3labor laws 100% 2.2.3 ibovespa 30% 3.1.3leave the organization 67% 3.2.3 responsibility to society 33% 4.1.3 fiscal committe 41,7% 4.2.3fiscal committe 100% 5.1.3board of directors evaluation 67% 5.2.3 functional policies 83% 1.1.4 shareholders agreeement 83% 3.2.4 responsibility to the 88% 4.1.4manag. director 50,0% 4.2.4manag. director 100% 5.2.4 guidelines and 100% 3.2.5work safety and occupational health 88% 4.1.5 holding 37,5% 4.2.5 holding 50% 5.2.5 strategic management 100% 4.1.6 independ ent auditing council 40,0% 4.2.6independen t auditing council 50% 5.2.6 market relations 50% 5.2.7risk management 30% 5.2ownership (shareholders / partners) / board / ceo 1 -alignment governance maturity elements -> 5processes4structure3sustainability2conformity 1.1corporate and business cohesion 2.2regulatory requirements 2.1legal conformities 3.1responsibility to partners 3.2responsibility to stakeholders 4.1organs 4.2roles and effectiveness 4.3operation and procedures 5.1administ. council ijahp article: botelho/analyzing the ahp priority vectors: going beyond inconsistency indexes international journal of the analytic hierarchy process 5 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.922 figure 4 priority agenda (via ahp over the hierarchy analysis of governance maturity) 2.3 an alternative scale several authors have worked on other scales of development besides the oss. the goal with these alternative scales is to improve the predictability of rank reversal. this study compares the use of the oss and an alternative scale, an ahp applied process of prioritizing corporate governance criteria, as presented by botelho (2021). goepel (2018) compared different weight scales and introduced the generalized balanced scale (gbs). goepel (2018) demonstrated that gbs performed better reaching small weight uncertainty and weight dispersion compared to the others. therefore, botelho (2021) and this study adopted the gbs as the alternative scale to be applied in the aforementioned ahp governance prioritizing exercise. the gbs is defined as: where c = weight value in gbs; n = number of criteria and x = weight value in oss. figure 5 shows gbs weights compared with oss. figure 5 oss & gbs oss gbs n=3 gbs n=4 gbs n=5 gbs n=6 gbs n=7 1 1,0 1,0 1,0 1,0 1,0 2 1,3 1,4 1,4 1,5 1,5 3 1,7 1,8 1,9 2,0 2,1 4 2,1 2,3 2,5 2,6 2,8 5 2,7 3,0 3,2 3,4 3,5 6 3,5 3,9 4,1 4,3 4,5 7 4,6 5,0 5,3 5,5 5,7 8 6,3 6,6 6,8 7,0 7,1 9 9,0 9,0 9,0 9,0 9,0 ijahp article: botelho/analyzing the ahp priority vectors: going beyond inconsistency indexes international journal of the analytic hierarchy process 6 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.922 3. methodology this study can be classified as applied research, as it aims to generate knowledge for the practical application of scenario analysis. in addition, it uses a practical case study to address the proposal. the methodology developed here, as mentioned in the introduction, presents a complete criterion for evaluating the priority vectors generated by ahp independently and not limited to the consistency indices. below, the development of this new approach is demonstrated with respect to a complete analysis of the level of perturbation suffered by the fpvs when compared to the original ordering vectors. 3.1 analyzing final priority vectors without rank reversal occurrences given an initial priority vector (ipv) [w(i)1, w(i)2, w(i)3, .... w(i) [n-1], w(i)n] (the eigenvector of the consistent pcm) and a final one (fpv) [w(f)1, w(f)2, w(f)3,.... w(f)[n-1], w(f)n] (the eigenvector of the inconsistent pcm), the absolute difference between the weights (wf wi), taken in inverse proportion to their positions in the ranking, weights the intensity of the disturbance, providing a no reversion index (nri), as follows: nri= ∑ [𝑤𝑖(𝑖) − 𝑤𝑖(𝑓)]/𝑖 𝑛𝑖=1 (1) see figure 6 for a nri definition and figures 7 and 8 for a nri histogram and nri cumulative frequency, respectively. furthermore, it established a metric to qualify the level of disturbance without rank reversal. for that, a monte carlo analysis was performed evaluating 13.200 random simulations without rank reversal. this monte carlo exercise considered a vector with 5 positions in the ranking (n=5). it has defined the “no reversal quality vector” (nrqv), according to equation 2. figure 6 analysis without rank reversal ipv (consistent) fpv (inconsistent) wi (1) wf(1) [wi (1) wf(1)] / 1 wi (2) wf(2) [wi (2) wf(2)] / 2 wi (3) wf(3) [wi (3) wf(3)] / 3 . . . . . . . . . wi (n-2) wf(n-2) [wi (n-2) wf(n-2)] / (n-2) wi (n-1) wf(n-1) [wi (n-1) wf(n-1)] /(n-1) wi (n) wf(n) [wi (n) wf(n)] / n ijahp article: botelho/analyzing the ahp priority vectors: going beyond inconsistency indexes international journal of the analytic hierarchy process 7 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.922 figure 7 nri histogram figure 8 nri’s cumulative frequency therefore, the ipv disturbance level is reflected via the nri index and classified by the nrqv index, represented in equation 2 below, in line with figure 8 (for n=5). 𝑁𝑅𝑄𝑉(5) = −632. 𝑁𝑅𝐼6 + 1473. 𝑁𝑅𝐼5 − 1307. 𝑁𝑅𝐼4 + 538. 𝑁𝑅𝐼3 − 93. 𝑁𝑅𝐼2 + 𝑁𝑅𝐼 + 1 (2) 3.2 analyzing final priority vectors with rank reversal occurrences similar to the assessment without rank reversal, the impact on the fpv when a rank reversal occurs is more significant the higher in these rankings the positions affected by the reversal are. the higher the original ranking position impacted by the reversal, the higher the level of relevance of this disturbance. therefore, an index to represent the magnitude of this particular case of reversal must consider the impact inversely proportional to the original ranking position. ijahp article: botelho/analyzing the ahp priority vectors: going beyond inconsistency indexes international journal of the analytic hierarchy process 8 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.922 considering that, an index of reversal (ir) is defined as: ir = ∑ [𝑃𝑖(𝑖) − 𝑃𝑖(𝑓)]/𝑖 𝑛𝑖=1 (3) where: pi(i) = position “i” in original ranking e pf(i)= position “i” in final ranking in addition to the ir index, a complementary metric is needed to evaluate how significant the reversal was. to determine this, a minimum and a maximum reversal impact index must be defined. the minimum reversal that can be observed is between the penultimate (n-1) and the last position (n) and only a draw between these positions (n and n-1) represents a lighter impact. this observation is taken to define the so-called index of minimum reversal (imr) (see equations 4 and 5). 𝐼𝑀𝑅 (𝑟𝑒𝑣𝑒𝑟𝑠𝑎𝑙) = [(1/𝑛) + 1/(𝑛 − 1)]  𝐼𝑀𝑅(𝑟𝑒𝑣𝑒𝑟𝑠𝑎𝑙) = [(2𝑛 − 1)/(𝑛(𝑛 − 1))] (4) 𝐼𝑀𝑅 (𝑑𝑟𝑎𝑤) = [(𝑛 − (𝑛 − 1))/(𝑛 − 1)]  𝐼𝑀𝑅 (𝑑𝑟𝑎𝑤) = [1/(𝑛 − 1)] (5) table 1 imr x n (reversal and draw on last ranking positions) regarding the definition of a maximum reversal impact index, an extremely relevant ranking reversal that can be conceived is the complete symmetric reversal of the ranking (the first position is reversed with the last, the second with the penultimate, the third with the antepenultimate, successively). it is a fact that significant reversals are undesirable. therefore, to establish a maximum reversal impact index (mrii) this must be taken into account and a monte carlo simulation carried out to verify the level of confidence of this assumption. figure 9 presents the above concept, and figure 10 shows confidence level check. n 2 3 4 5 6 7 8 9 10 imr reversal of n and (n-1) . 1.50 0.83 0.58 0.45 0.37 0.31 0.27 0.24 0.21 imr draw between (n) and (n-1) 1.00 0.50 0.33 0.25 0.20 0.17 0.14 0.13 0.11 ijahp article: botelho/analyzing the ahp priority vectors: going beyond inconsistency indexes international journal of the analytic hierarchy process 9 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.922 𝑈𝑝𝑤𝑎𝑟𝑑𝑠 = (𝑛 − 1)/1 + (𝑛 − 3)/2 + (𝑛 − 5)/3 + (𝑛 − 7)/4 + ⋯ therefore, 𝑈𝑝𝑤𝑎𝑟𝑑𝑠 = ∑ (𝑛 + 1 − 2𝑖)/𝑖 𝑖𝑛𝑡( 𝑛 2 ) 𝑖=1 𝐷𝑜𝑤𝑛𝑤𝑎𝑟𝑑𝑠 = 𝑛 − 1 𝑛 + 𝑛 − 3 𝑛 − 1 + 𝑛 − 5 𝑛 − 2 + 𝑛 − 7 𝑛 − 3 + ⋯ therefore, 𝐷𝑜𝑤𝑛𝑤𝑎𝑟𝑑𝑠 = ∑ (𝑛 + 1 − 2𝑖)/(𝑛 − 𝑖 + 1) 𝑖𝑛𝑡( 𝑛 2 ) 𝑖=1 figure 9 basis for mrii definition therefore, in total: 𝑀𝑅𝐼𝐼 = ∑ (𝑛 + 1 − 2𝑖). ( 1 𝑖 + 1 𝑛−𝑖+1 ) 𝑖𝑛𝑡( 𝑛 2 ) 𝑖=1 (6) table 2 mrii mrii 0.00 1.50 2.67 4.58 6.30 8.52 10.59 12.84 15.38 18.01 n 1 2 3 4 5 6 7 8 9 10 1 1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00 9.00 2 0.50 1.00 1.50 2.00 2.50 3.00 3.50 3 0.33 0.67 1.00 1.33 1.67 4 0.25 0.50 0.75 5 0.20 6 0.17 7 0.33 0.43 8 0.25 0.40 0.50 0.57 0.63 9 0.33 0.50 0.60 0.67 0.71 0.75 0.78 10 0.50 0.67 0.75 0.80 0.83 0.86 0.88 0.89 0.90 dsc asc ijahp article: botelho/analyzing the ahp priority vectors: going beyond inconsistency indexes international journal of the analytic hierarchy process 10 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.922 figure 10 mrii (n=5) level of confidence 3 figure 10 shows a confidence level of 97.3% for the adopted mrii concept. therefore, based on the above methodologies, an index to qualify the reversal quality vector (rqv), with regard to the relevance of fpv rank reversal, is defined as: 𝑅𝑄𝑉 = 1 − [ 𝐼𝑅−𝐼𝑀𝑅 𝑀𝑅𝐼𝐼−𝐼𝑀𝑅 ]𝑘 (7) since the ranking reversal of higher magnitudes is not desired, the rvq was defined considering the exponential characteristic, with the exponent k of ¼ to induce significant reductions in this quality index. thus, using this methodology, the governance scenarios were analyzed. these comparisons were considered in a theoretical status of governance maturity. two different ahp scales (oss and gbs) were applied. finally, a comparison was also made between ipv (from consistent pcms) and fpv (from inconsistent pcms), as well as between fpv and the priority agenda of elements of governance. the results and discussions are presented in the next sections. 3 19,842 occurrences with rank reverse obtained from 20,000 simulations where, randomly, normalized vectors with 5 positions (n=5) had their initial and final ranking compared. ijahp article: botelho/analyzing the ahp priority vectors: going beyond inconsistency indexes international journal of the analytic hierarchy process 11 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.922 4. results table 3 summary of indexes considering 7 pairwise comparison (between consistent and inconsistent pcms) 4.1 analysis of fpv x ipv (pcm consistent x inconsistent): table 4 nri table 5 nrqv board member escalas isr ir qsr qcr ic saaty 01 ~09 0.057 1.50 0.921 0.326 egb 0.038 1.50 0.987 0.326 01 ~09 0.046 0.00 0.935 1.000 egb 0.031 0.00 0.991 1.000 01 ~09 0.038 0.00 0.967 1.000 egb 0.023 0.00 0.977 1.000 01 ~09 0.0163 0.00 0.9837 1.0000 egb 0.0111 0.00 0.9889 1.0000 01 ~09 0.0383 0.33 0.9726 0.6574 egb 0.0285 0.87 0.9752 0.4350 01 ~09 0.0258 0.00 0.9742 1.0000 egb 0.0047 0.00 0.9953 1.0000 01 ~09 0.0436 0.25 0.9501 1.0000 egb 0.0328 0.25 0.9894 1.0000 c7(*) 0.00% 4.92% pcm inconsistent x consistent 16.97% 11.73% (*): draw in some positions of ipv (out of consistents pcms) c6 8.04% 6.34% c1 (*) c5 (*) 15.92% c2 c3 c4 ijahp article: botelho/analyzing the ahp priority vectors: going beyond inconsistency indexes international journal of the analytic hierarchy process 12 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.922 table 6 ir table 7 rqv 4.2 analysis of fpv x priority agenda of governance maturity: table 8 rqv (inconsistent pcm) table 9 rqv (consistent pcm) 4.3 interpretation of results from the above tables, it can be noted that: a) concerning the comparisons between inconsistent and consistent pcms: i. the gbs, in all cases, generated a lower level of disturbance in the fpv when no rank reversal has occurred. ii. when rank reversal occurred, the gbs generated a higher disturbance in the fpv than oss. b) concerning the comparisons between fpv and the priority governance agenda: i. fpv of inconsistent pcms presented a higher magnitude of rank reversal with gbs than oss. ijahp article: botelho/analyzing the ahp priority vectors: going beyond inconsistency indexes international journal of the analytic hierarchy process 13 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.922 ii. no difference was observed regarding the consistent pcms iii. the average level of disturbance due to rank reversal was higher with the consistent pcm than with inconsistent ones, for both weight scales (average rqv of 0.61 for gbs and 0.87 for oss compared to 0.58 for consistent pcms). 5. conclusion based on these interpretations of the results, it can be concluded that: i. gbs is a more sensitive scale than oss, as far as rank reversal trends are concerned. ii. gbs generates a smaller weight difference among ranking positions. therefore, if the perturbation is not enough to generate rank reversal, gbs provides a fpv with lower weight perturbations. iii. the inconsistent analysis of the invited board members generated fpvs with higher adherence to the governance priority agenda than the consistent one. the inconsistency introduced by the pairwise comparison provided by the board members is an indication that, indeed, such inconsistencies were a refinement/optimization of the process. therefore, it reinforces the ahp methodology as an adequate process for decision making involving corporate governance issues. iv. the metrics developed in this study allowed a deeper understanding of the disturbance that occurred with fpv. these disturbances could not be quantified if only the inconsistency index had been used. the indices developed here made it possible to quantify and compare the fpv ranking disorders. in general, the method and metrics developed in this study went beyond the traditional focus of checking the consistency index (ci). a complete and detailed analysis of rank disturbances was feasible due to this new proposed methodology. this proposed methodology focused on an effective evaluation of the disturbances that occurred with the fpv and brings clarity to the final analysis of the outputs provided by the ahp and similar mcda techniques. the analysis of the proposed nri, ir and rqv indices gives a complete view of the ranking disturbance suffered by the fpv when compared to the original ranking. the analysis of these indices allows the dm to interpret whether a given rank reversal should be rejected or, on the contrary, reflects the expected response. a fpv with some rank reversion and a small nri may represent the appropriate response and, if followed by a high nri, should be rejected. the magnitude of the rqv also supports this analysis. other surveys may carry out further investigations into the limits of acceptance of the fpv based on these proposed new indices. ijahp article: botelho/analyzing the ahp priority vectors: going beyond inconsistency indexes international journal of the analytic hierarchy process 14 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.922 references álvares, e., giacometti, c., gusso, e. (2008) governança corporativa: um modelo brasileiro. 4th ed. rio de janeiro: elsevier editora. isbn 978-85-352-3092-5, 8535230920 2008. amenta, p., lucadamo, a., marcarelli, g. (2018). approximate thresholds for salohamalainen index. ifac papersonline, 51(11), 1655-1659. doi: https://doi.org/10.1016/j.ifacol.2018.08.219. araujo, m., oliveira, e., monteiro, s. (2017). avaliação de maturidade de processos de gestão de riscos de ti: ferramenta de apoio para a qualidade e eficiência do processo. revista brasileira de computação aplicada, 9(2017). doi: https://doi.org/10.5335/rbca.v9i2.6099. atz, f., gerhard, m., freitag, v., vanti, a., et al. (2017). análise do relacionamento existente entre governança da tecnologia da informação, o gerenciamento de risco corporativo e as funções de compliance. in: seminário de administração. são paulo: usp, 1-16. belton, v., gear, t. (1985). the legitimacy of rank reversal a comment. international journal of management science, 13, 143-144. doi: https://doi.org/10.1016/0305-0483(85)90052-0 bernasconi, m. choirat, c., seri, r. (2009). a re-examination of the algebraic properties of the ahp as ratio-scaling technique. journal of mathematical psychology, 55(2), 152-165. doi: https://doi.org/10.1016/j.jmp.2010.11.002 bernasconi, m., choirat, c., seri, r. (2010). the analytic hierarchy process and the theory of measurement. management science, 56(4), 699-711. doi: https://www.jstor.org/stable/27784145. bezerra, p. (2021). proposição de um novo modelo para avaliar a sustentabilidade empresarial. 77 f. dissertação (mestrado em administração) – centro de humanidades, universidade federal de campina grande, campina grande. disponível em: http://dspace.sti.ufcg.edu.br:8080/jspui/handle/riufcg/17903. acesso em: 20 jun. 2021. botelho, m.(2021). a decisão multicritério nos conselhos de administração via análise hierárquica de processos (ahp): análise da inconsistência nas matrizes de comparação par a par nas abordagens de governança corporativa. 106 f. dissertação (mestrado em engenharia e gestão de processos e sistemas) instituto de educação tecnológica. doi: https://doi.org/10.19146/pibic-2017-78788 bozóki, s.; rapcsák, t. (2008). on saaty’s and koczkodaj’s inconsistencies of pairwise comparison matrices. journal of global optimization, 42, 157-17. doi:10.1007/s10898-007-9236-z. https://doi.org/10.1016/j.ifacol.2018.08.219 https://doi.org/10.5335/rbca.v9i2.6099 https://www.jstor.org/stable/27784145 http://dspace.sti.ufcg.edu.br:8080/jspui/handle/riufcg/17903.%20acesso https://doi.org/10.19146/pibic-2017-78788 ijahp article: botelho/analyzing the ahp priority vectors: going beyond inconsistency indexes international journal of the analytic hierarchy process 15 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.922 brunelli, m. (2018). a survey of inconsistency indexes for pairwise comparisons. international journal of general systems, 47(8), 751-771. doi: https://doi.org/10.1080/03081079.2018.1523156. brunelli, m. (2017). studying a set of properties of inconsistency indexes for pairwise comparisons. annals of operations research, 248, 143-161. doi: https://doi.org/10.1007/s10479-016-2166-8. buede, d.m., maxwell, d.t. (1995). rank disagreement: a comparison of multicriteria methodologies. journal of multicriteria decision analysis, 4(1), 1-21. doi: https://doi.org/10.1002/mcda.4020040102 caríssimo, c., moreira, m., ornelas, m., silva, j. (2016). uso da análise hierárquica (ahp) para identificação da preferência de peritos-contadores quanto ao método de avaliação de sociedades em perícias contábeis. revista de educação e pesquisa em contabilidade, 10(1), 44-62. doi: https://doi.org/10.17524/repec.v10i1.1333. cobo, a., vanti, a., rocha, r. (2014). a fuzzy multicriteria approach for it governance evaluation system. journal of information systems and technology management, 11(2), 257-276. doi: 10.4301/s1807-17752014000200003. duarte, b. (2019). maturidade da governança corporativa e gestão dos incentivos fiscais como determinantes da longevidade e desempenho das empresas do polo industrial de manaus, biblioteca face / ufg lvr 040/2020. ellermeier, w., faulhammer, g. (2000). empirical evaluation of axioms fundamental to stevens's ratio-scaling approach: i. loudness production. perception & psychophysics, 62(8), 1505-1511, 2000. doi: https://doi.org/10.3758/bf03212151. emrouznejad, a., marra, m. (2017). the state of the art development of ahp (1979 2017): a literature review with a social network analysis. international journal of production research, 55(22), 6653-6675. doi: https://doi.org/10.1080/00207543.2017.1334976. franeka, j., krestaa, a. (2014). judgment scales and consistency measure in ahp. procedia economics and finance, 12(2014), 164-173. doi: 10.1016/s22125671(14)00332-3. gass, s., rapcsá, k. (2001). models, methods, concepts & applications of the analytic hierarchy process: singular value decomposition in ahp. international symposium on the analytic hierarchy process, 6, berne, switzerland, 129-130. goepel, k. (2018). comparison of judgement scales of the analytical hierarchy process: a new approach. international journal of information technology and decision making, 18(2), 445-463. doi:10.1142/s0219622019500044. gonzáles, p., vanti, a., seiber, t. r., castro, m. (2018). design de hierarquização multicritério para o aumento da competitividade empresarial. international conference on information systems & technology management, 5, 5025-5042. doi:10.5748/9788599693148-15contecsi/doct-5997. https://doi.org/10.1080/03081079.2018.1523156 https://doi.org/10.1007/s10479-016-2166-8 https://doi.org/10.1002/mcda.4020040102 https://doi.org/10.17524/repec.v10i1.1333 https://doi.org/10.3758/bf03212151 https://doi.org/10.1080/00207543.2017.1334976 ijahp article: botelho/analyzing the ahp priority vectors: going beyond inconsistency indexes international journal of the analytic hierarchy process 16 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.922 görener, a. (2012). comparing ahp and anp: an application of strategic decision making in a manufacturing company. international journal of business and social science, 3(11), 194-208. doi: 10.1016/j.sbspro.2012.09.1139. grybowski, a., starczewski, t. (2020). new look at the inconsistency analysis in the pairwise-comparisons: based prioritization problems. expert systems with applications, 159(2020). doi: https://doi.org/10.1016/j.eswa.2020.113549. grybowski, a., starczewski, t. (2016). new results on inconsistency indexes and their relationship with the quality of priority vector estimation. expert systems applications, 43, 197-212. doi: 10.1016/j.eswa.2015.08.049. harker, p., vargas, l. (1987). the theory of ratio scale estimation: saaty's analytic hierarchy process. the institute of management science, 33(11), 1383-1403. doi:10.1287/mnsc.33.11.1383. ibgc (2015a). código das melhores práticas de governança. 5th ed. são paulo: instituto brasileiro de governança corporativa. isbn: 978-85-99645-38-3 1. ibgc (2015b). métrica de governança corporativa empresas de capital fechado. são paulo: instituto brasileiro de governança corporativa. isbn: 978-85-99645-38-3 1. instituto ethos (2013). indicadores ethos para negócios sustentáveis e responsáveis. são paulo: instituto ethos. ishizaka, a., labib, a. (2011). review of the main developments in the analytic hierarchy process. expert systems with applications, 38(11), 14336-14345. doi:10.1016/j.eswa.2011.04.143. januri, s., pozi, m., nadzri, m., zainuddin, m. (2021). factors influencing islamic banking by customers: a case study in northern malaysia. international journal of academic research in business e social sciences, 11(7), 720-735. doi:10.6007/ijarbss/v11-i7/10530 kaiser, r., futami, a., valentina, l., oliveira, m. (2019). development of a managerial tool for prioritization and selection of portfolio projects using the analytic hierarchy process methodology in software companies. gestão & produção, 26(4), 1-13. doi: https://doi.org/10.1590/0104-530x4267-19. kamil, k., ismail, a., shahimi, s. (2013). objectives of islamic banks in the management of asset and liability: a decision process of deriving priority. proceedings of the international symposium on the analytic hierarchy process. kazibudzki, p. (2017). new results of the quality of recently introduced index for a consistency control of pairwise judgments. multi criteria decision making, 12, 90102. doi: 10.22367/mcdm.2017.12.07. kazibudzki, p. (2017). the quality of priority ratios estimation in relation to a selected prioritization procedure and consistency measure for a pairwise comparison matrix. advances in operations research, 2019, 1-24. doi:10.1155/2019/3574263. https://doi.org/10.1016/j.eswa.2020.113549 https://doi.org/10.1590/0104-530x4267-19 ijahp article: botelho/analyzing the ahp priority vectors: going beyond inconsistency indexes international journal of the analytic hierarchy process 17 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.922 klozikova j., dockalikova, i. (2014). corporate governance rating: synthesis of rating models of corporate governance with utilization methods ahp and dematel. european conference on management, leadership & governance. koklu, o., jakubowski, e., servi, t., huang, j. (2016). identifying and interpreting subjective weights for cognitive and performance characteristics of mathematical learning disability: an application of a relative measurement method. the journal of international lingual, social and educational sciences, 2(2), 136-151. lambert-torres g., costa c., barros, m. & martins,h. (2007). processo de análise hierárquica paraconsistente. vi congress of logic applied to technology, paper 132. doi: http://dx.doi.org/10.13140/rg.2.1.2022.6807 loostsma, f. (1990). the french and the american school in multicriteria decision analysis. rairo: recherche opérationnelle, 24(3), 263-285. melo, r.t., jesus, v. m. (2019). avaliação 360º dos membros de um conselho de administração pelo método anp. revista de ciência, tecnologia e inovação, 26(4), 44-53. doi: https://doi.org/10.1590/0104-530x4267-19. neves, r., pereira, v., costa, h. (2015). auxílio multicritério à decisão aplicado ao planejamento e gestão na indústria de petróleo e gás. scientific eletronic library on line -scielo, associação brasileira de engenharia de produção, 25(1). doi: https://doi.org/10.1590/s0103-65132013005000060. odic, d., im, h., eisinger, r., ly, r., halberda, j. (2015). psimle: a maximum likelihood estimation approach to estimating psychophysical scaling and variability more reliably, efficiently, and flexibly. behavior research methods, 48, 445-462. doi: 10.3758/s13428-015-0600-5. paiva, b. (2021). mensuração do grau de maturidade da governança corporativa em cooperativas de crédito pelo método análise hierárquica de processos – ahp. 2018. 154 f. tese (doutorado em administração) – pontifícia universidade católica de são paulo, são paulo, 2018. disponível em: https://tede2.pucsp.br/handle/handle/19566. acessed on june 20, 2021. rodriguez, d. (2016). autoavaliação da governança de ti em uma autarquia federal com base na gestão de riscos. 2016. 228 f. dissertação (mestrado em computação aplicada) instituto de ciências exatas, departamento de ciência da computação, universidade de brasília, brasília. saaty, r. (2016). decision making in complex environments. pittsburgh, pennsylvania, usa: creative decisions foundation. isbn 1-888603-00-3. saaty, r.w. (1987). the analytic hierarchy process: what is and how it is used. mathematical modeling, 9(3-5), 161-176. doi:10.1016/0270-0255(87)90473-8. http://dx.doi.org/10.13140/rg.2.1.2022.6807 https://doi.org/10.1590/0104-530x4267-19 https://tede2.pucsp.br/handle/handle/19566 ijahp article: botelho/analyzing the ahp priority vectors: going beyond inconsistency indexes international journal of the analytic hierarchy process 18 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.922 saaty, t.l. (1977). a scaling method for priorities in hierarchical structures. journal of mathematical psychology, 15(3), 234-281. doi: https://doi.org/10.1016/00222496(77)90033-5. saaty, t.l. transport planning with multiple criteria: the analytic hierarchy process applications and progress review. journal of advanced transportation, 29(1), 81126. doi: https://doi.org/10.1002/atr.5670290109. saaty, t.l., vargas, l.g. (2001). the seven pillars of the analytic hierarchy process. in: models, methods, concepts & applications of the analytic hierarchy process. boston, ma: springer, 27-46. (international series in operations research & management science; 34). doi: https://doi.org/10.1007/978-1-4615-1665-1_2. salo, a., hämäläinen, p. (1997). on the measurement of preferences in the analytic hierarchy process. journal of multi-criteria decision analysis, 6(6), 309-319. doi: https://doi.org/10.1002/(sici)1099-1360(199711)6:6<309::aidmcda163>3.0.co;2-2. salo, a., hämäläinen, p. (1993). preference programming through approximate rate comparisons. european journal of operational research, 82(3), 458-475. santiago, l.p., soares, v.o. (2018). strategic alignment of an r&d portfolio by crafting the set of buckets. iee transactions of engineering management. doi:10.1109/tem.2018.2876408. starczewski, t. (2016). relationship between priority ratios disturbances and priority estimation errors. journal of applied mathematics and computational mechanics, 15, 143-154. doi: 10.17512/jamcm.2016.3.14. starczewski, t. (2018). remarks about geometric scale in the analytic hierarchy process. journal of applied mathematics and computational mechanics, 17(3), 7182. doi: 10.17512/jamcm.2018.3.07. subramanian, n., ramanathan, r. (2012). a review of applications of analytic hierarchy process in operations management. international journal of production economics, 138(2), 215-241. doi: http://dx.doi.org/10.1016/j.ijpe.2012.03.036 trindade, j. (2016). mensuração e avaliação da capacidade inovativa de micro, pequenas e médias empresas: aplicação de métodos multicritério fuzzy de apoio à decisão. 107 f. dissertação (mestrado em qualidade e inovação) – pontifícia universidade católica do rio de janeiro, rio de janeiro. doi: https://doi.org/10.17771/pucrio.acad.29381. tsyganok, v.v., kadenko, s.v., andriichuk, o.v. (2012). significance of expert competence consideration in group decision making using ahp. international journal of production research, 50(17), 4785-4792. doi: https://doi.org/10.1080/00207543.2012.657967. https://doi.org/10.1016/0022-2496(77)90033-5 https://doi.org/10.1016/0022-2496(77)90033-5 https://doi.org/10.1002/atr.5670290109 https://doi.org/10.1007/978-1-4615-1665-1_2 https://doi.org/10.1002/(sici)1099-1360(199711)6:6%3c309::aid-mcda163%3e3.0.co;2-2 https://doi.org/10.1002/(sici)1099-1360(199711)6:6%3c309::aid-mcda163%3e3.0.co;2-2 http://dx.doi.org/10.1016/j.ijpe.2012.03.036 https://doi.org/10.17771/pucrio.acad.29381 https://doi.org/10.1080/00207543.2012.657967 ijahp article: botelho/analyzing the ahp priority vectors: going beyond inconsistency indexes international journal of the analytic hierarchy process 19 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.922 vidal, l.a., marle, f., bocquet, j.c. (2011). using a delphi process and the analytic hierarchy process (ahp) to evaluate the complexity of projects. expert systems with applications, 38(5), 5388-5405. doi: https://doi.org/10.1016/j.eswa.2010.10.016. wang x., triantaphyllou, e. (2008). ranking irregularities when evaluating alternatives by using some electre methods. omega – the international journal of management science, 36(1), 46-63, 2008. doi:10.1016/j.omega.2005.12.003 watson, r., freeling, n. (1982). assessing attribute weights by ratios. omega,20(6), 582-583 1982. doi: http://dx.doi.org/10.1016/0305-0483(82)90061-5. wegner, r., battist, i.a., tontini, j., malheiros, m., rossato, v. (2020). aplicação do método analytic hierarchy process (ahp) na priorização das ações de inovações em serviços em um estudo de multicaso. navus revista de gestão e tecnologia, 10, 0119. doi:10.22279/navus. 2020. v10.p01-19.1006. yrjölä, m. uncovering executive prioritization: evaluating customer value propositions with pairwise comparison method. journal of service and science management, 8, 1-13. doi: 10.4236/jssm.2015.81001. https://doi.org/10.1016/j.eswa.2010.10.016 http://dx.doi.org/10.1016/0305-0483(82)90061-5 microsoft word 2sato_p64-82_vol_i_issue_2_2009.docx ijahp article: sato/how to measure human perception in survey questionnaires international journal of the 64 vol. 1 issue 2 2009 issn 936-6744 analytic hierarchy process how to measure human perception in survey questionnaires yuji sato graduate school of policy science, mie chukyo university matsusaka, japan e-mail: ysatoh@mie-chukyo-u.ac.jp abstract the objective of this study was to examine the effectiveness of multiple-choice method, ranking method, rating method and the application of analytic hierarchy process in measuring human perception. the ahp not only clearly identifies the most important alternative but also the preference for each alternative by each decision maker. therefore, using ahp to analyze the decision-making process may result in a precise clarification of preference for alternatives. based on survey researches on social issues, the results offered some evidence that the application of the ahp was superior to traditional questionnaire methods in representing human perceptions. keywords: survey questionnaire, multiple-choice, ranking, rating, feeling thermometer 1. introduction questionnaire design for survey research, such as public opinion polls, presents one of the biggest challenges for survey researchers in terms of accuracy in measuring respondents’ perceptions (traugott and lavrakas, 2000). consequently, many ways of asking questions have been proposed and much discussion has been generated. the set of categories or range of scores on a variable is called a scale, and the process of assigning scores to objects to yield a measure of a construct is called scaling. when a respondent applies judgment to assign scores to individuals or objects along the scale, a rating method is being used (judd, smith, & kidder, 1991). one rating method—called a feeling thermometer (ft; kabashima, 1998)—was extensively used in survey questionnaires, which ranges from 0, the coldest feeling toward alternatives, to 100, the hottest, with 50, being neutral. in surveys, this method asks respondents to express their perceptions by indicating their “temperature” for each alternative for a given question. although this method helps respondents precisely clarify their judgments for each alternative, consistency among responses to the alternatives is not always satisfactory (sato, 2005). a more traditional method for measuring respondents’ perceptions is the multiple-choice （mc） question format, which has been thought to be well suited to questionnaire formatting because respondents find the questions easy to answer and they allow researchers to easily identify the main concerns of the respondents (jerard, 1995; downing, 2004). this method takes two different forms: one is simple multiple-choice (smc); the other is modified multiple-choice (mmc; sato, 2004). in the smc method, respondents must choose one from among the given alternatives. the smc identifies only the rob typewritten text rob typewritten text http://dx.doi.org/10.13033/ijahp.v1i2.31 ijahp article: sato/how to measure human perception in survey questionnaires international journal of the 65 vol. 1 issue 2 2009 issn 936-6744 analytic hierarchy process most important alternative for each respondent, thus preventing the respondent from expressing his or her preference concerning a selected alternative over the others. moreover, no information regarding the relationship among the non-selected alternatives is derived (sato, 2004). in the mmc method, respondents have the option of indicating their top-two (or more) alternatives. since respondents are allowed to express their preferred alternatives, the mmc can be expected to be an effective way to make up for the lack of information incurred by the smc. nevertheless, the difference in the degree of importance among the selected alternatives is not clarified, nor is the information concerning non-selected alternatives reflected in the results (sato, 2004). let us consider the case asking respondents why they are non-partisan, for example. on a question designed in the smc method, respondents must express their opinion by choosing one from among the reasons given. respondents with a definite reason regarding the issue could choose one alternative without confusion if they found that it exactly represented their perception. this format could be expected to function quite well for these respondents. on the other hand, it might be that some respondents are non-partisan for no particular reason, while others are non-partisan for complex reasons. the mc would not be an effective format for those respondents. another method that has been applied is the ranking method used by ronald inglehart and paul abramson (1993) in their world values survey. this method asks respondents to rank all given alternatives in a question, from the most preferred to the least, thus allowing researchers to identify a respondent’s preference order for all alternatives. the problem with this method, however, is that the more alternatives a questionnaire offers, the more difficult it is for the respondent to answer (inglehart and abramson, 1993). another drawback to this approach is that it does not allow for ties (sato, 2003). for example, let us consider the case asking executive staff members of a prefectural government who have authority for final budget decisions “which governmental projects should be budgeted with high priority for next year?” on a question designed in the ranking method, respondents must express their opinion by ranking all projects given in the question. respondents with definite preferences on the issue could rank all the projects without hesitation. on the other hand, some respondents might have no definite preference concerning the issue while others might have many ties in the priority for projects. one possible option for formatting questionnaires is to apply the ahp, a popular tool for decision-making developed by t. saaty (1977, 1980). since it was released, many individuals and groups in various fields have used the ahp because of its user-friendly interface for multi-criteria decision-making (vargas, 1990). in the ahp, data from a decision-maker’s judgments, called pairwise comparisons, are aggregated, and the degree of importance of each alternative is quantified in the ahp. this procedure identifies not only the most important alternative but also the preference for all alternatives for each decision-maker (crawford and williams, 1985). using the ahp to analyze the decision-making process, therefore, results in a precise clarification of respondents’ preferences for alternatives (sato, 2007). here we summarize the aforementioned four questioning methods by taking the case of buying a fruit as an example. if you were supposed to buy a fruit at a store from among the following four alternatives, apple, banana, melon and orange for instance, your preference should be represented as follows. ijahp article: sato/how to measure human perception in survey questionnaires international journal of the 66 vol. 1 issue 2 2009 issn 936-6744 analytic hierarchy process in a question asking your preference, smc method requires you to choose one alternative from among the four fruits, and mmc, if it allows you to give your second choice, asks you to choose up to two alternatives from among the four fruits. on the other hand, ranking method asks you to rank all fruits from the most preferred to the least by number, and feeling thermometer requires you to assign adequate number between 0 and 100 to each one of the four fruits. the application of ahp asks you to conduct pairwise comparison over possible combinations of the four fruits and then the weights for all alternatives would be calculated. in this study, we compared the answers to four sets of questions on a particular issue, each formatted in a different way. the first two of the four sets consisted of questions formatted using two different types of the mc method and the ahp; the third set consisted of two pairs of questions formatted by the ranking method and the ahp; the fourth set consisted of two pairs of questions formatted by the rating method and the ahp. we then evaluated the four methods in terms of appropriateness for representing each respondent’s perception. first, we focused on the difference of the aggregated ranking of alternatives across all respondents between the mc and ahp methods. the ranking derived from the smc implies aggregated plurality, while that elicited from the ahp suggests aggregated intensity.1 since both rankings reflect the pattern of responses for alternatives, they are likely to produce similar results. in addition, we evaluated the effectiveness of the mmc in terms of its ability to make up for the lack of information incurred by the smc. since the mmc is a type of the mc question method that allows respondents to indicate their second-best alternative, it may reflect each respondent’s preference for alternatives more precisely than does the smc. to compare the two methods, three questions—formatted by the smc, mmc and ahp methods—on the same issue were posed: each asked about the reasons that respondents were non-partisan. details of the data set are shown in sections 2.1 and 2.2. second, we compared the preference orders of alternatives across all respondents between the ranking and the ahp methods, and evaluated these two formats in terms of the appropriateness for representing each respondent’s perception. the aggregated ranking of alternatives across all respondents derived from the ranking method implies preference order of alternatives for respondents. similarly, aggregated weight for alternatives across all respondents elicited from the ahp also implies preference order of alternatives for respondents. since both preference orders reflect the entire trend concerning each alternative of a population, these two methods are likely to produce similar results. to compare the two methods, two different sets of a pair of questions on a particular issue were used: one concerned an abstract issue—the 1 from discussion at the may conference, university of michigan, may 2000. apple banana melon orange smc : x mmc (in case, up to 2 alternatives) : x x ranking : 4 1 3 2 feeling thermometer : 22 95 43 78 ahp : 0.092 0.399 0.181 0.328 ijahp article: sato/how to measure human perception in survey questionnaires international journal of the 67 vol. 1 issue 2 2009 issn 936-6744 analytic hierarchy process basic concepts of refining governmental program policy; the other related to a concrete issue—governmental projects with high priority. details of the data set are shown in section 2.3. third, we focused on the difference of the aggregated weight of alternatives across all respondents between the ft and the ahp methods. the weight derived from ft is based on absolute evaluation, while that elicited from the ahp is based on comparative evaluation. since both aggregated weight reflect the pattern of responses for alternatives, they are likely to produce similar results. in addition, we evaluated the effectiveness of each method in terms of its ability as independent variables in regression analyses. in the above-mentioned comparisons, a data set obtained from a 2004 survey on public opinion was employed. to compare the two methods, two types of questions on a particular issue were employed, each formatted in a different way, one using the ft and the other using the ahp. each asked about respondents’ intention to vote for a party in the next election. details of the data set are shown in section 2.4. 2. outline of surveys in this study, we compared the three above-mentioned methods—the mc method, the ranking method, rating method, and the ahp method—using survey data. the data sets were obtained from the following four surveys. for the ahp, respondents were asked to respond to a series of redundant pairwise comparisons. we thus needed to take into account the possible inconsistency of a pairwise comparison matrix in analyzing elicited weights (webber, apostolou and hassell, 1997). in this paper, however, we do not cut off those samples exceeding the threshold of consistency index (c.i.), such as 0.15, because the respondents whose c.i. exceeded the threshold are social existences, such as constituencies in elections, too. 2.1 survey 1 (january 1999) survey 1 was carried out in january 1999, one month after the coalition cabinet of the liberal democratic party and the liberal party was established in japan. respondents were 834 students at a university in japan. the purpose of the survey was to identify “the political attitude of students when a coalition cabinet was established.” the survey consisted of 30 questions. in q.12, respondents were asked their party identification, and for only the 398 respondents who answered “non-partisan” to the question, the following three sub-questions were posed in three ways to clarify why they were non-partisan. q.13 (hereinafter referred to as q1s) and q.29 (q1m) in survey 1 were respectively formatted by the smc method (see appendix 1), which requires respondents to choose only one from among four given reasons, and by the mmc method (see appendix 2), which gives respondents the option of indicating their second-best alternative. q.26 (q1a) was formatted by the ahp method (see appendix 3), in which respondents are required to conduct pairwise comparisons across all possible combinations of reasons. the reasons offered were: too much political realignment; political apathy; non-confidence with party and politician; and corruption of political ethics. 2.2 survey 2 (april 2001) survey 2 was conducted in april 2001, the month when graduating students usually begin to start job search in japan. in this survey, respondents were 323 students of a university in japan. the intellectual purpose of this survey was to clarify “the main concerns of japanese graduating students as they begin ijahp article: sato/how to measure human perception in survey questionnaires international journal of the 68 vol. 1 issue 2 2009 issn 936-6744 analytic hierarchy process their job search.” the main concerns offered were: job specifications; welfare program of the company; salary; and place of employment. in this survey, three differently formatted questions were posed, each asking the respondents’ main concerns of their job-search activities. the first question (hereinafter referred to as q2s) was formatted by the smc, the second question (q2a) by the ahp and the third question (q2m) by the mmc. we leave out the details of questions q2s, q2a and q2m; suffice it to say that each question had exactly the same format as questions q1s, q1a and q1m in survey 1, respectively (cf. appendices 1, 2 and 3). 2.3 survey 3 (january 2002) survey 3 was carried out in january 2002, the month executive staff members of a local government in japan finalize the budget for the following fiscal year. respondents were 35 executive staff members having authority for final budget decisions on governmental projects. the purpose of the survey was to identify “the main concerns of executive staff members in budgeting.” the survey included two issues: one asked about the basic concepts for refining the public administration (abstract issue); the other asked about actual governmental projects with high priority (concrete issue). the first pair of questions (hereinafter referred to as q3a1 and q3r1) concerned an abstract issue—the basic concepts of refining governmental program policy. they were: concept i—for cultural development of our area; concept ii—for safety of our social life; concept iii—for environmental preservation of our area; concept iv—for economic growth of our area; and concept v—for enhancement of our area. the respondents were asked, in two ways, which of the five basic concepts they thought significant for refining governmental program policy: q3a1 was formatted by the ahp method, in which respondents were asked to conduct pairwise comparisons across all possible combinations of the basic concepts (cf. appendix 3). q3r1 used the ranking method (see appendix 4), in which respondents were asked to rank the entire concept in the given question. the second pair of questions (hereinafter referred to as q3a2 and q3r2) related to a concrete issue—governmental projects with high priority (php, for short). they were: php i—support for school education (for cultural development of our area); php ii—improvements to rivers, mountains and coasts (for safety of our social life); php iii—preservation of water resources (for environmental preservation of our area); php iv—support for entrepreneurs (for economic growth of our area); and php v—construction and repair of roads (for enhancement of our area). the respondents were asked, in two ways, which of the governmental projects should be budgeted with high priority: q3a2 employed the ahp method and q3r2 was formatted by the ranking method. q3a2 and q3r2 were formatted as the same as q3a1 and q3r1 respectively (cf. appendices 3 and 4). 2.4 survey 4 (march 2004) survey 4 was conducted in march 2004, one month after japan self-defense forces was dispatched to iraq. respondents were lay citizens of a local city in japan. the purpose of the survey was to identify “the political attitude of citizens when the self-defense force was dispatched to a country in the state of warfare.” the sample size was 30; each respondent’s political attitude was elicited by interviewing him/her one by one. the survey consisted of 33 questions. in qs.9 and 17, respondents were asked their intention to vote for a party in the next election. the parties were: liberal democratic party of japan (ldp); democratic party of japan (dpj); new komeito (nk); japanese communist party (jcp); and social democratic party (sdp). q.9 (hereinafter referred to as q4w) was formatted by the ft method ijahp article: sato/how to measure human perception in survey questionnaires international journal of the 69 vol. 1 issue 2 2009 issn 936-6744 analytic hierarchy process (see appendix 5), which requires respondents to assign an adequate number between 0 and 100 according to their intention to vote for each party. q.17 (q4a) used the ahp method (cf. appendix 3). the ft gave feeling score for each party, and the ahp gave the weight of each party. we also asked respondents about their political ideology in q.13, and whether they support prime minister and the president of ldp, jyunichiro koizumi, in q.33. both questions were formatted by the smc method, which requires respondents to choose one from among the given alternatives. each of these issues was mutually related to the party electorates would vote for in the house of councilors election. the outputs from these questions would serve as ideal dependent variables in a regression analysis. 3. results in this section, we analyze responses to the smc-formatted questions, the mmc-formatted questions and the ranking-formatted questions of surveys 1, 2, 3 and 4, based on the weight of each alternative elicited from the ahp. 3.1 comparison between smc and ahp first, we focus on the difference of the aggregated ranking of alternatives between the smc method and the ahp method; in particular, the observed ratios of each alternative derived from the smc and the weight of each alternative elicited from the ahp are compared. table 1 summarizes the results of two questions, q1s and q1a, from survey 1. numbers in the first row represent: weights derive from q1a aggregated across the respondents who chose apathy in answering q1s; observed number of the respondents; its ratio among all respondents; and the ratio’s ranking among four reasons. from the second to fourth row show the results in the same way as the first row. the fifth row represents the weight of each reason aggregated across all responses, and total number of the respondents. the last row shows the ranking of the average weight of each reason among all responses. as was done in table 1, table 2 summarizes the results of two questions, q2s and q2a, from survey 2. table 1 reason for being non-partisan (smc and ahp) apathy realignment corruption non-confidence apathy 0.192 0.350 0.223 0.235 108 38.4% 1 realignment 0.184 0.171 0.227 0.418 40 14.2% 4 corruption 0.177 0.199 0.371 0.253 56 19.9% 3 non-confidence 0.298 0.189 0.248 0.265 77 27.4% 2 average 0.217 0.250 0.260 0.273 281 100% ranking 4 3 2 1 obs. ratio rankingsmc (q1s) ahp (q1a): aggregated weight over each answer to smc ijahp article: sato/how to measure human perception in survey questionnaires international journal of the 70 vol. 1 issue 2 2009 issn 936-6744 analytic hierarchy process table 2 main concern of job-search activities (smc and ahp) as can be seen in tables 1 and 2, each diagonal element is not always the maximum in each row; that is, an answer on the smc and the most weighted alternative in the ahp do not necessarily coincide. for example, in table 1, respondents who answered apathy to the smc attached their weight mostly to realignment in the ahp, while respondents who answered realignment to the smc attached their weight mostly to non-confidence in the ahp. furthermore, the highest ratio of the answer to the smc is apathy; the most weighted reason in the ahp, on the other hand, is non-confidence. thus, based on the smc, the survey research would conclude that the most important reason respondents were non-partisan is apathy and the next is non-confidence, and so on. in contrast, based on the ahp, the most important reason is non-confidence and the next is corruption, and so on. clearly, these two methods yield a different aggregated ranking of alternatives. the smc is superior for two reasons. first, it is easy for respondents to fill out the questionnaire, and second, the main concerns of a respondent can be easily identified. however, this method prevents respondents from expressing their preference for a particular alternative over the others. furthermore, no information regarding the relationship among non-selected alternatives can be derived. on the other hand, the ahp makes it possible to reflect the relative importance of alternatives to results, even though it requires respondents to answer complex questions, thus requires much more time than the smc. which of these methods accurately represents respondents’ perceptions is still an open question; verification requires more empirical tests. these results imply, however, that the output of the smc, widely employed in survey research, might conceivably provide erroneous information. 3.2 comparison between mmc and ahp next, we compare the ratio for each alternative derived from the mmc method and the weight of each alternative elicited from the ahp method. the mmc is another type of the mc question format allowing respondents to express their top two alternatives, thus giving them a greater degree of freedom in answering questions. each respondent’s preference for alternatives is likely to be specified more precisely than it is with the smc. in surveys 1 and 2, however, almost half the respondents in each survey chose only one alternative in the mmc. accordingly, we define as “singular” the group of respondents that chose only one alternative despite the fact that they were given the option of indicating their second choice in the mmc (q1m and q2m in surveys 1 and 2, respectively), and we define as “plural” the specifications welfare salary place specifications 0.405 0.097 0.178 0.320 83 47.4% 1 welfare 0.276 0.219 0.184 0.322 28 16.0% 3 salary 0.356 0.179 0.266 0.199 46 26.3% 2 place 0.306 0.066 0.087 0.540 18 10.3% 4 average 0.361 0.134 0.193 0.312 175 100% ranking 1 4 3 2 smc (q2s) ahp (q2a): aggregated weight over each answer to smc obs. ratio ranking ijahp article: sato/how to measure human perception in survey questionnaires international journal of the 71 vol. 1 issue 2 2009 issn 936-6744 analytic hierarchy process group of respondents that chose two alternatives. in this section, we first focus on the difference of aggregated ranking of alternatives among the “singular” and the “plural” of the mmc and of the ahp. table 3 summarizes the results of the two questions, q1m and q1a, for those defined as “singular” in the same way as tables 1 and 2. table 5 similarly summarizes the results of the two questions q2m and q2a. tables 4 and 6 summarize the results concerning those defined as “plural,” where two answers are counted for each respondent. as can be seen in tables 3 and 5, each diagonal element is not always the maximum in each row; that is, an answer of the “singular” to the mmc and the most weighted alternative in the ahp do not necessarily coincide. for example, in table 5, respondents who answered specifications to the mmc attached their weight to place most in the ahp, and respondents who answered welfare to the mmc also attached their weight most to place in the ahp. furthermore, the highest ratio of the answer to the mmc is salary; however, the most weighted concern in the ahp is specifications. as a result, these two methods yield a different aggregated ranking of alternatives. the fact that the choice in the mmc is more compatible than the smc with the most weighted alternative in the ahp is likely to result in a rationale that the mmc functioned well for the “plural.” as shown in table 4, for example, each diagonal element is always the maximum in each row. table 6, however, shows that the relationship between the choice in the mmc and the most weighted alternative is in a state of chaos. furthermore, as was seen in tables 3 and 5, the mmc and the ahp yield a different aggregated ranking of alternatives for both tables 4 and 6, while the rankings elicited from the ahp are robust (see tables 1, 3 and 4 from survey 1 and tables 2, 5 and 6 from survey 2). lastly, we evaluate the effectiveness of the mmc in making up for the lack of information encountered with the smc. since the mmc gives respondents the option of indicating their second-best alternative, whether or not a second-best alternative is chosen may depend on the strength of that alternative vis-à-vis the first alternative. in other words, if the mmc functions well for the “plural,” it could contribute by supplying missing information (e.g., the difference in the degree of importance among selected alternatives and the respondent’s perception concerning non-selected ones). let wp1 and wp2 respectively denote respondent p’s maximum and second-maximum element of the eigenvector corresponding to the frobenius root, then ∆wp≡wp1 wp2 (≥0) represents the discrepancy in the maximum weight and the second maximum weight of alternative for respondent p. therefore, the larger the ∆wp is, the more clearly respondents would distinguish their best alternative from the second-best. table 7 summarizes the number of the “singular” and the “plural” respondents in surveys 1 and 2, the table is stratified by the size of ∆wp. as shown in the table, the difference in the distribution of responses between the “singular” and the “plural” seems small. indeed in chi-square tests, x2(q1m) = 2.979 < x2(6, 0.8) = 3.070 and x2(q2m) = 2.108 < x2(6, 0.9) = 2.204 hold. these results from the chi-square tests imply that there may not be any difference in ∆wp between the “singular” and the “plural”; in other words, whether or not respondents added their second choice in answering the mmc-formatted question may be independent of the discrepancy in the degree of importance between the best and the second-best alternative. thus, we could conclude that the mmc does not succeed in precisely specifying a respondent’s preference of alternatives. the mmc, overall, does seem to be a good option for designing questionnaires because it enhances the degree of freedom in answering questions for respondents. many researchers thus extensively employ the mmc in their survey questionnaires. insofar as our surveys are concerned, however, the mmc may not be effective in filling the information gap between the smc and the ahp. ijahp article: sato/how to measure human perception in survey questionnaires international journal of the 72 vol. 1 issue 2 2009 issn 936-6744 analytic hierarchy process table 3 reason for being non-partisan (mmc “singular” answer and ahp) table 4 reason for being non-partisan (mmc “plural” answers and ahp) table 5 main concern of job-search activities (mmc “singular” answer and ahp) mmc (q1m) "singular" answer apathy realignment corruption non-confidence apathy 0.281 0.207 0.229 0.283 41 31.3% 2 realignment 0.169 0.372 0.230 0.229 48 36.6% 1 corruption 0.154 0.165 0.415 0.266 25 19.1% 3 non-confidence 0.142 0.191 0.219 0.448 17 13.0% 4 average 0.198 0.258 0.263 0.281 131 100% ranking 4 3 2 1 ahp (q1a): aggregated weight over each answer to mmc obs. ratio ranking mmc (q2m) "singular" answer specifications welfare salary place specifications 0.283 0.079 0.162 0.476 19 23.5% 2 welfare 0.309 0.215 0.096 0.381 11 13.6% 4 salary 0.367 0.169 0.314 0.150 38 46.9% 1 place 0.247 0.140 0.082 0.530 13 16.0% 3 average 0.320 0.149 0.212 0.319 81 100% ranking 1 4 3 2 ahp (q2a): aggregated weight over each answer to mmc obs. ratio ranking mmc (q1m) "plural" answer apathy realignment corruption non-confidence apathy 0.275 0.244 0.245 0.236 83 28.0% 1 realignment 0.223 0.322 0.203 0.251 72 24.3% 3 corruption 0.199 0.211 0.324 0.266 78 26.4% 2 non-confidence 0.209 0.216 0.246 0.330 63 21.3% 4 average 0.228 0.248 0.256 0.267 296 100% ranking 4 3 2 1 ahp (q1a): aggregated weight over each answer to mmc obs. ratio ranking ijahp article: sato/how to measure human perception in survey questionnaires international journal of the 73 vol. 1 issue 2 2009 issn 936-6744 analytic hierarchy process table 6 main concern of job-search activities (mmc “plural” answers and ahp) table 7 comparison of ∆wp between “singular” and “plural” 3.3 comparison between ranking method and ahp first, we analyze the difference of the preference order for all alternatives concerning the abstract issue, between the ranking method and the ahp method. specifically, the aggregated ranking of each concept for refining governmental program policy derived from the ranking method and the aggregated weight of each concept elicited from the ahp are compared. table 8 summarizes the results of the two questions, questions q3r1 and q3a1, from survey 3. the numbers in the first row are the aggregated ranking of each concept across all responses obtained from q3r1. the second row represents the average weight for each concept aggregated across all responses derived from q3a1. the last row indicates the change from year 2001 to 2002 in the actual implementation of the budget for each concept, which corresponds to one of the five budgeting categories in the local government. mmc (q2m) "plural" answer specifications welfare salary place specifications 0.445 0.136 0.230 0.189 57 32.4% 1 welfare 0.283 0.225 0.093 0.399 40 22.7% 3 salary 0.426 0.134 0.286 0.154 45 25.6% 2 place 0.328 0.196 0.063 0.413 34 19.3% 4 average 0.381 0.167 0.181 0.271 176 100% ranking 1 4 3 2 ahp (q2a): aggregated weight over each answer to mmc obs. ratio ranking singular plural singular plural 46 45 24 30 ( 0 , 0.1 ] 34 35 22 21 ( 0.1 , 0.2 ] 16 24 9 13 ( 0.2 , 0.3 ] 7 10 6 8 ( 0.3 , 0.4 ] 10 14 5 5 ( 0.4 , 0.5 ] 6 10 6 5 ( 0.5 , 1.0 ] 12 10 9 6 q1m (survey 1) q2m (survey 2) ∆ w p 0 ijahp article: sato/how to measure human perception in survey questionnaires international journal of the 74 vol. 1 issue 2 2009 issn 936-6744 analytic hierarchy process table 8 concepts of refining governmental program policy as shown in table 8, the difference of the preference order for all alternatives between the ranking method and the ahp seems small. for example, the highest ranked concept in the aggregated ranking derived from the ranking method is concept ii; the most weighted concept in the ahp is also concept ii. the case is the same for both the lowest concept in the ranking method and the lightest weighted concept in the ahp: one concept—concept iv—results for both. in addition, the correlation coefficient (cc) between the outputs of q3r1 and q3a1 is -0.87. consequently, the preference order of the basic concepts derived from the ranking method coincides almost exactly with that elicited from the ahp. as a result, these two methods would likely produce similar results concerning respondents’ preference order of alternatives. nevertheless, the preference order of basic concepts for refining governmental program policy does not coincide with the annual change in the actual implementation of the budget. specifically, the concept thought to be the most significant by executive staff members for refining governmental program policy (concept ii) and the category budgeted with the highest priority (corresponding to concept iii) are different. consequently, the preference orders obtained from the ranking method and the ahp do not coincide with the annual change in budget. indeed, ccs between the preference orders of concepts and the actual change in budget are -0.44 and 0.32. these results may imply that both the ranking method (q3r1) and the ahp (q3a1) did not function well in capturing respondents’ perceptions. we would not, however, be able to draw such a conclusion based solely on this survey because the change rates in the budget were so small (between –1.62% and +1.26%) that the correlation between the preference order and the change rate may have been affected by measurement error. next, we focus on the difference of the preference order for all alternatives concerning the concrete issue between the ranking method and the ahp. specifically, the aggregated ranking of each governmental project with high priority obtained from the ranking method and the aggregated weight of each project elicited from the ahp are compared. table 9 summarizes the results of the two questions, q3r2 and q3a2 in the same way as tables 8. as shown in table 9, the preference order for all alternatives between the ranking method and the ahp are different. for example, both the highest php in the aggregated ranking obtained from the ranking method and the most weighted php in the ahp are php i, while the lowest php in the ranking method (php v) and the lightest weighted php in the ahp (php iii) are different. indeed, the cc between the outputs of q3r2 and q3a2 is -0.32. as a result, the ranking method and the ahp would reason differently. q3r1 & q3a1 concept i concept ii concept iii concept iv concept v ranking 2.000 1.250 3.500 4.500 3.750 weight 0.18803 0.36152 0.19979 0.08191 0.16874 annual change in budget 2001→ 2002 -1.11%0.355% -1.62% 1.26% -0.231% ijahp article: sato/how to measure human perception in survey questionnaires international journal of the 75 vol. 1 issue 2 2009 issn 936-6744 analytic hierarchy process table 9 governmental projects with high priority in contrast to the case of the abstract issue, the preference order of phps induced from the ahp are highly correlated with the annual change in the actual implementation of the budget for the case of the concrete issue. as can be seen in the table 9, the preference order elicited from the ahp coincides almost exactly with the annual change in the actual implementation of the budget (cc=0.98), while that generated by the ranking method does not (cc= -0.32). although these results may have been affected by measurement error, these results imply that each respondent’s preference for alternatives elicited from the ahp is likely to be specified more precisely than it is with the ranking method. 3.4 comparison between rating method and ahp first, we focus on the difference of the aggregated ranking of alternatives between the ft method and the ahp method; in particular, the average of feeling score for each party obtained from the ft and that of the weight of each party derived from the ahp are compared. since both rankings reflect the entire trend concerning each party of a population, they are likely to produce similar results. table 10 summarizes the results of two questions, q4w and q4a, from survey 4. the numbers in the first and the second rows are the average of feeling score and average weight of each party aggregated across all responses, derived from q4w and q4a, respectively. the last row represents the correlation coefficients between feeling score and the weight elicited from the ahp of each party among all respondents. as shown in table 10, feeling scores and the weights of the ahp imply that the most and the second most favored party among respondents are dpj and ldp, respectively; both the ft and the ahp clearly identify the top 2 parties. on the other hand, the ranking for the remaining parties, nk, jcp and sdp, are different, even though the difference of feeling score or the weight of each party among those parties is quite small. indeed, the correlation coefficients between feeling score and the weight of each party are high on some level. as a result, answers on the ft and on the ahp do not necessarily coincide, which yields a different aggregated ranking of alternatives. both the ft and the ahp, however, produces similar results, and they overall seem to specify respondents’ preference for parties in the choice of house of councilors election. q3r2 & q3a2 php i php ii php iii php iv php v ranking 2.000 2.750 2.750 3.250 4.250 weight 0.39234 0.14968 0.08169 0.14725 0.22904 annual change in budget 2001→ 2002 -5.87% -0.300%10.3% -7.72% -8.16% ijahp article: sato/how to measure human perception in survey questionnaires international journal of the 76 vol. 1 issue 2 2009 issn 936-6744 analytic hierarchy process table 10 aggregated feeling score and weight of each party next, we evaluate the effectiveness of each method in terms of its ability as independent variables in regression analyses; in particular, we formulate four regression models and compare r2s. in the evaluations, as independent variables, we employ feeling score and not-normalized weight of each party except for those of the social democratic party (sdp), because the party was minority group (the number of the member of house of councilors was; ldp: 116, dpj: 70, nk: 23, jcp: 20 and sdp: 5, as at the survey was carried out. (march 2004)). insofar as dependent variables, based on quantification theory i, we employ two variables; one is respondent’s political ideology (q.13) in regression models 1 and 2, and the other is whether respondent supports prime minister koizumi or not (q.33) in regression models 3 and 4. we offered six alternatives including “don’t know” answer in q.13, and we set “progressive” or “slightly progressive” as 0, and “conservative” or “slightly conservative” as 1, and we omitted “neutral” and “don’t know” responses from the regression analysis a. in the same way as for regression analysis a, we set “support koizumi” as 0, and “do not support koizumi” as 1, and we omitted “don’t know” responses from regression analysis b. thus, the actual regression models can be formulated as follows. regression analysis a dependent variables; political ideology “progressive” or “slightly progressive” = 0, “conservative” or “slightly conservative” = 1 regression equations; political ideology = a1 + a2*fsldp + a3*fsdpj + a4*fsnk + a5*fsjcp (regression model 1) political ideology = b1 + b2*wldp + b3*wdpj + b4*wnk + b5*wjcp (regression model 2) regression analysis b dependent variables; support koizumi or not “support” = 0, “do not support” = 1 regression equations; koizumi = c1 + c2*fsldp + c3*fsdpj + c4*fsnk + c5*fsjcp (regression model 3) koizumi = d1 + d2*wldp + d3*wdpj + d4*wnk + d5*wjcp (regression model 4) q4w & q4a ldp dpj nk jcp sdp feeling score 53.6 66.6 19.2 14.4 14.2 weight 0.34662 0.38969 0.08945 0.08007 0.09417 correlation coefficient 0.81257 0.72848 0.77450 0.77443 0.75042 ijahp article: sato/how to measure human perception in survey questionnaires international journal of the 77 vol. 1 issue 2 2009 issn 936-6744 analytic hierarchy process table 11 summarizes the results of regression analysis a. as can be seen in the table, the r2s of regression models 1 or 2 are not large enough to predict respondents’ political ideology; respondent’s intention for voting does not necessarily correlate with his/her political ideology. by employing the weights derived from the ahp, however, r2 is improved from 0.23755 to 0.42850. furthermore, focusing on the p-values, the degree of precision of the regression model 2 is higher than that of the regression model 1. table 11 results of regression analysis a (models 1 and 2) table 12 summarizes the results of regression analysis b. as shown in the table, r2s are relatively larger that those of the regression analysis a; whether respondent supports prime minister koizumi or not can be predicted on some level based on regression models 3 or 4. by employing the weight derived from the ahp, r2 is again improved from 0.42293 to 0.51299. furthermore, focusing on the p-values, the degree of precision of the regression model 4 is higher than that of the regression model 3. table 12 results of regression analysis b (models 3 and 4) constant 1.48488 ** 1.35099 *** ldp -1.58819 ** -0.00968 *** dpj -0.51939 * -0.00175 * nk -0.45019 * -0.00330 ** jcp -1.62317 * -0.00642 ** n = 25 r 2 = 0.23755 se = 0.48460 r 2 = 0.42850 se = 0.41955 model 1 (feeling score) model 2 (weight) coefficient coefficient political ideology p-value: 0.05 < p <= 0.1: *, 0.01 < p <= 0.05: **, p <= 0.01: *** constant -0.05558 * -4.22262 ** ldp 0.00918 ** 5.29983 *** dpj -0.00148 * 4.09808 ** nk 0.00481 ** 6.46932 *** jcp 0.00125 * 9.04207 *** n = 17 r 2 = 0.42293 se = 0.45130 r 2 = 0.51299 se = 0.41459 p-value: 0.05 < p <= 0.1: *, 0.01 < p <= 0.05: **, p <= 0.01: *** model 3 (feeling score) model 4 (weight) coefficient coefficient support koizumi ijahp article: sato/how to measure human perception in survey questionnaires international journal of the 78 vol. 1 issue 2 2009 issn 936-6744 analytic hierarchy process the ft method overall seems to be a good option for designing questionnaires because it assures respondents expressing their preferences for all alternatives, and it enhances the degree of freedom in answering questions for respondents. therefore, many researchers, especially in political science field, have extensively employed the ft in their survey questionnaires. insofar as our survey is concerned, however, feeling scores may not be effective in predicting respondents’ preferences in comparison with weights elicited from the ahp. either in regression analysis a or b, r2s based on the ft are not large enough to predict respondents’ preferences. on the other hand, the results of this section imply that the ahp could quantify respondents’ preferences in terms of the distribution of the weight of each alternative; by employing weights of the ahp, r2s in regression analyses are improved from the use of feeling scores. since feeling score measures the feeling for parties and the weight of the ahp measures the preference for parties, we cannot simply compare these r2s and conclude that the weight is the better explainer than feeling score. the weights derived from the ahp, however, could be conjectured to function better as independent variables in regression analyses than feeling scores. 4. concluding remarks questionnaire design poses one of the biggest challenges for survey researchers because how respondents are asked questions can have a great effect on results. one political scientist2 remarked that different question formats yielded different results, despite the fact that they were asking about the exact same content. consequently, various ways of eliciting opinions have been proposed and evaluated in order to clarify which best represents each respondent’s perception. in particular, the multiple-choice method, ranking method, and rating method have often been compared, generating a great deal of discussion. among the aforementioned methods, each method has the pros and cons. the multiple-choice method has been most extensively used because of its ease of response and its ease in identifying for the researcher the respondents’ main concerns. in addition, this method provides some formats that enhance a respondent’s degree of freedom in answering questions. meanwhile, no information regarding the non-selected alternatives or the relative importance among selected alternatives can be derived. the ranking method, often considered ideal for designing questionnaires, is also extensively used because it allows researchers to easily identify respondents’ preference orders. this method, however, does not allow ties for alternatives nor can they represent the degree of importance for each alternative. the rating method, such as the feeling thermometer, has also been extensively employed because of its ease in identifying for the researcher the respondents' concerns for all alternatives. on the other hand, this method does not fit the natural way of thinking for humans. in contrast, the ahp, a support system for decision-making, can thus be a possible option for formatting questionnaires. this method makes it possible to reflect the relative importance of alternatives to results, even though it requires respondents to answer complex questions, thus necessitates much more time. in this study we focused on survey research in which the questions that respondents were asked involved issues, we verified the effectiveness of the multiple-choice method, the ranking method, and the rating method by using the weight of alternatives elicited from the ahp method. the results were: (1) the simple or modified multiple-choice and the ahp yielded a different aggregated ranking of alternatives, while the weights elicited from the ahp were robust; (2) whether or not respondents added a second choice in answering the modified multiple-choice formatted questions was irrelevant to the discrepancy in the degree of importance between the best and the second-best alternative; (3) for the abstract issue, the 2 chris achen, university of michigan, february 2000 (personal communication). ijahp article: sato/how to measure human perception in survey questionnaires international journal of the 79 vol. 1 issue 2 2009 issn 936-6744 analytic hierarchy process ranking method and the ahp yielded similar aggregated ranking of alternatives; (4) for the concrete issue, the results from the ahp coincided almost exactly with the annual change in the actual implementation of the budget, unlike those generated by the ranking method; (5) the feeling thermometer method and the ahp yielded a similar aggregated ranking of alternatives; (6) in regression analyses, feeling scores may not be effective in predicting respondents’ preferences, while the weight derived from the ahp could predict their preferences on some level. these results, insofar as our survey is concerned, provide some evidence that the multiple-choice method, the ranking method, and rating method do not succeed in precisely specifying a respondent’s perception for alternatives and thus might not be appropriate for measuring human perception in questionnaires of survey research. the application of the ahp to questionnaire design in survey research, on the other hand, might very well be superior to those traditional methods. nevertheless, several issues remain. the first issue is: “what are the criteria that would be appropriate for evaluating the effectiveness of the questionnaire format?” for example, in survey 3 is the annual change in the actual implementation of a budget really adequate? this issue relates to the criterion evaluating the method: the criterion needs to be sensitive in measuring the correlation between what is in respondents’ minds and their actual behavior. as for the theme of the survey, the annual change in budget is still a functional criterion for the evaluation because the change is one of the distinguishing phenomena representing the priority of executive staff members in the government. on the other hand, other criteria for evaluation could be considered: the number of staff members for a particular project, the frequency of being one of the main agenda items in the executive staff meetings, the number of times a particular project is covered in the public relations material of the government, and so on. further empirical tests are required for verification. the second issue concerns what the results would be if the number of alternatives in a question were other than we used in the surveys. would similar results be obtained if the question offered nine alternatives, for instance? in this study, we analyzed the case for just four or five alternatives; investigations of up to at least ten alternatives would therefore be warranted in order to verify what an adequate format for questionnaire design would be. references downing m.steven. (2004). reliability: on the reproducibility of assessment data, medical education, 38(9), 1006–1012. inglehart, r. & abramson, p. (1993). values and value change of five continents, paper presented at the 1993 annual meeting of the american political science association, washington d.c., september 1-5. jerard, k. (1995). writing multiple-choice test items practical assessment, research and evaluation, 4(9). judd, c. m., smith, e. r. & kidder, l. h.(1991). research methods in social relations. sixth edition. orlando: harcourt brace & co. kabashima, i. (1998). seiken koutai to yuken-shya no taido henyou. tokyo: bokutaku-shya (in japanese). saaty, t.l. (1980). the analytic hierarchy process. new york: mcgraw-hill. ijahp article: sato/how to measure human perception in survey questionnaires international journal of the 80 vol. 1 issue 2 2009 issn 936-6744 analytic hierarchy process saaty, t.l. (1994). highlights and critical points in the theory and application of the analytic hierarchy process, european journal of operational research, 52, 426-447. sato, y. (2003). comparison between ranking method and the analytic hierarchy process in program policy analysis, proceedings of the seventh international symposium on the analytic hierarchy process, 439-447. sato, y. (2004). comparison between multiple-choice and analytic hierarchy process: measuring human perception, international transactions in operational research, 11(1), 77-86. sato, y. (2005). questionnaire design for survey research: employing weighting method, proceedings of the eighth international symposium on the analytic hierarchy process, issn 1556-830x. sato, y. (2007). administrative evaluation and public sector reform: an analytic hierarchy process approach, international transactions in operational research, 14(5), 445-453. traugott, w. mike & lavrakas, j. paul. (2000). the voter’s guide to election polls, 2ed. new york: chatham house publishers, seven bridges press, llc. webber, a., apostolou, b. & hassell, j.m. (1997). the sensitivity of the analytic hierarchy process to alternative scale and cue presentations, european journal of operations research, 96, 351-362. ijahp article: sato/how to measure human perception in survey questionnaires international journal of the 81 vol. 1 issue 2 2009 issn 936-6744 analytic hierarchy process appendix 1 (q1s: q.13 in survey 1, as formatted by the smc method) which of the following was the significant reason you were non-partisan? choose one from among the four reasons and write the letter in the box. appendix 2 (q1m: q.29 in survey 1, as formatted by the mmc method) which of the following was the significant reason you were non-partisan? choose one from among the four reasons and write the letter in the box. you may give an additional reason in the box marked “optional.” appendix 3 (q1a: q.26 in survey 1, as formatted by the ahp method) if you compare each of the following pairs of reasons for being non-partisan, which do you think is more significant? compare each of the following pairs of reasons and mark the place along the segment. a: too much political realignment b: political apathy c: non-confidence with party and politician d: corruption of political ethics a: too much political realignment b: political apathy c: non-confidence with party and politician d: corruption of political ethics optional absolutely equivalent absolutely realignment non-confidence corruption apathy non-confidence corruption realignment corruption apathy realignment non-confidence apathy ijahp article: sato/how to measure human perception in survey questionnaires international journal of the 82 vol. 1 issue 2 2009 issn 936-6744 analytic hierarchy process appendix 4 (q3r1: one of the two first questions in survey 3, as formatted by the ranking method) the five basic concepts for refining our govenmental program policy are as follows: concept i: for cultural development of our area concept ii: for safety of our social life concept iii: for environmental preservation of our area concept iv: for economic growth of our area concept v: for enhancement of our area which of the following projects are more significant than others? rank all projects, from the most preferred (1) to the least (5), and write the number in each box. appendix 5 (q4w: q.9 in survey 4, as formatted by the feeling thermometer method) the next house of councilors election is scheduled this coming july. which party are you willing to vote for the election by a proportiional representation system? indicate your intention of voting for each of the following parties by a number from 0, coldest feeling, to 100, the hottest feeling, with 50 being neutral, and write the number in each box. for cultural development of our area for the safety of our social life for environmental preservation of our area for the economic growth of our area for the attraction of our area liberal democratic party (ldp) democratic party japan (dpj) new komeito (nk) japan communist party (jcp) social democratic party (sdp) 5 rev botteroeg5-282-emu-06-20-12_def m. bottero, v. ferretti, s. pomarico / assessing the sustainability of alternative transport infrastructures international journal of the analytic hierarchy process 61 vol. 4 issue 1 2012 issn 1936-6744 assessing the sustainability of alternative transport infrastructures marta bottero*1 interuniversity department of regional and urban studies and planning (dist) politecnico di torino torino, italy e-mail: marta.bottero@polito.it valentina ferretti interuniversity department of regional and urban studies and planning (dist) politecnico di torino torino, italy e-mail: valentina.ferretti@polito.it silvia pomarico interuniversity department of regional and urban studies and planning (dist) politecnico di torino torino, italy e-mail: silvia.pomarico@polito.it abstract this paper addresses the problem of sustainability assessment of territorial transformation through the use of the anp. the case of a new transport infrastructure in a city located in italy is considered in the study. the project discussed involves further development of the existing ring road in the city which will lead to a radically new multifunctional design of the urban area. in the case study, four alternatives have been identified and compared in order to select the most sustainable option. the model takes into consideration in more detail the different aspects of the decision-making process economic, environmental, social, transport and urban planning aspects) that have been organized according to the benefits, opportunities, costs and risks (bocr) categories. in the present study a focus group was organized with actors from public authorities in order to discuss the general aspects of the problem. also, several experts were questioned in order to elicit the priorities of the aspects under consideration. the application of the anp technique, which was performed using the super decisions software, allowed the most relevant aspects of the decision-making process to be highlighted. keywords: analytic network process; decision-making; territorial transformation; sustainability assessment; transport infrastructures. 1. introduction sustainable development is a multidimensional concept that includes socio-economic, ecological, technical and ethical perspectives and thus leads to issues that are simultaneously characterized by a high degree of conflict, complexity and uncertainty. when speaking about territorial planning, many objectives have to be considered in the decision-making process. these objectives range from the promotion * corresponding author rob typewritten text http://dx.doi.org/10.13033/ijahp.v4i1.101 m. bottero, v. ferretti, s. pomarico / assessing the sustainability of alternative transport infrastructures international journal of the analytic hierarchy process 62 vol. 4 issue 1 2012 issn 1936-6744 of cultural events to the requalification of downgraded urban areas, from the reduction of soil consumption to the optimization of environmental resource use and from the promotion of tourism to the rationalization of the mobility system. it is generally agreed that multicriteria analysis (figueira et al., 2005) is an adequate approach to deal with sustainability assessment of territorial transformations at both micro and macro study levels, and the use of a multicriteria framework is an efficient tool when implementing an inter-disciplinary approach. in the context of mca, the analytic network process plays a very important role (saaty 2005; saaty and vargas, 2006). this technique, which represents the generalization of the more well-known analytic hierarchy process (saaty, 1980) on dependences, is particularly suitable for dealing with complex decision-making processes which are characterized by interrelationships and feedback at stake. among the elements this paper addresses the problem of sustainability assessment of territorial transformation through the use of the anp. the case of a new transport infrastructure in a city located in italy is considered in detail. the project discusses the further development of the existing city ring road which will lead to a radically new multifunctional design of the urban area. as stated by yin (1984), research methods based on case studies can be defined as empirical inquiries that investigate a contemporary phenomenon within its real-life context, when the boundaries between phenomenon and context are not clearly evident. in this sense, it is possible to say that case study research offers an understanding of complex issues and extends the already available knowledge about the topic. scientists have used the case study research method for many years across a variety of disciplines; this is particularly true in the context of social sciences, where this methodology has been used to examine contemporary situations and to provide the basis for the application of ideas and extension of methods. more generally, case study methodology by investigating phenomena in their real-life context can be a very important tool in opening the “black box” of how interventions and program effectiveness are linked. this is an advantage over traditional experimental and quasi-experimental designs which may measure outcomes and some process variables, but fall short in dealing with the dynamic that is inherent in community-based collaborative initiatives (horsch and anderson, 1997). according to this approach, this paper aims to analyze a real case study and investigate the contribution that the anp offers in the field of sustainability assessment and transport infrastructures. transport planning undeniably plays a key role in the economic growth of any region, and has long-term effects on the local community. a number of objectives must be met in order to select an optimal transport route. these objectives can be in conflict with one other, according to the opinions of the different stakeholders involved in the process. in order to support the decision-making process related to the implementation of the aforementioned project, an anp model was developed. the reasons for using an anpbased decision approach in the present analysis include: (i) the assessment of different transport route alternatives is a multicriteria decision-making process; (ii) there are dependencies among groups of criteria and between these and the alternatives to be analyzed, and (iii) the detailed analysis of the inter-relationships between criteria forces the decision makers (dms) to reflect carefully on their project priority approach and on the decision-making problem itself. this helps dms to gain a better understanding of the problem and to make a more reliable final decision. m. bottero, v. ferretti, s. pomarico / assessing the sustainability of alternative transport infrastructures international journal of the analytic hierarchy process 63 vol. 4 issue 1 2012 issn 1936-6744 2. anp: theory overview and state of the art the anp (saaty, 2005) represents a theory of relative measurement on absolute scales of both tangible and intangible criteria based on both the judgement of experts and on existing measurements and statistics needed to make a decision. the anp represents any decision as a network and allows the structure to develop more naturally by freeing us from the burden of ordering the components in the form of a directed chain as in the ahp hierarchy.. the anp therefore represents a better way to faithfully describe what can happen in the real world, and is gaining merit as a useful tool to help technicians make their decision processes traceable and reliable. by including dependences and feedbacks and by cycling their influence by means of the supermatrix approach, the anp is more objective and more likely to capture what happens in the real world, thus providing effective support for the kind of decisions needed to plan for the future (zoffer et al., 2008). from a methodological point of view the anp is based on five fundamental steps (saaty, 2005): (i) structuring of the decision-making problem; (ii) clusters and nodes weighting by means of pairwise comparisons; (iii) supermatrices formation; (iv) elicitation of final priorities and (v) sensitivity analysis. there are two possible ways for structuring the decision-making problem: the simple network or the complex benefitsopportunities-costs-risks (bocr) network (saaty, 2005). a very large and consolidated amount of literature concerning the anp exists in different fields. applications have been made in the sphere of waste management (khan and faisal, 2008; aragonés-beltrán et al., 2010; bottero and ferretti, 2011), strategic policy planning (ulutas, 2005), environmental impact assessment of territorial transformations (bottero et al., 2008; bottero and mondini, 2008; liu and lai, 2009), market and logistics (liang and li, 2008), economics and finance (niemura and saaty, 2004) and civil engineering (neaupane and piantanakulchai, 2006). in the transport planning field in particular, applications of anp models exist for selecting optimal routes and for designing new corridors (piantanakulchai, 2005; tuzkaya and onut, 2008). finally, , a number of interesting works in the transport planning and territorial transformation field exist in recent studies focusing on sustainability assessment (pèti, 2012; bojković et al., 2011; bottero and ferretti, 2010; bottero and lami, 2010; basbas and papanikolaou, 2009; lombardi, 2009). 3. application 3.1 presentation of the case study and description of the alternatives the purpose of the evaluation is to compare the different road infrastructure alternatives that the city is analysing in order to achieve a priority ranking of the alternative projects. in the present application, four alternatives were identified and compared in order to select the most sustainable option. more importantly, the transformation under examination refers to the so-called “undesirable facilities location problems”. in addition, the projects are currently under development and the alternatives refer to a timetable projected to the year 2030. for the aforementioned reasons, it is not possible to provide a detailed description of the alternatives. the full range of alternative options is described in table 1. in the rest of the paper the projects under examination will be denoted as “alternative x” and “alternative y”. m. bottero, v. ferretti, s. pomarico / assessing the sustainability of alternative transport infrastructures international journal of the analytic hierarchy process 64 vol. 4 issue 1 2012 issn 1936-6744 table 1 alternative description alternatives description alternative 0 this alternative represents the situation with no project. alternative x the transformation refers to the implementation of a new north-south highway in the western metropolitan area. this project will lead to a radically new multifunctional design of the city, including new residential and commercial areas and a highly innovative multilevel road project. alternative y this alternative refers to the development of the eastern part of the city ring road in order to strengthen the existing road network. the area to be crossed by the road is characterized by agricultural land and the main concern of the project relates to the amount of land being consumed. alternative z this alternative refers to the development of both the aforementioned projects. 3.2 construction of the bocr network a complex anp model was developed in order to take into account the complexity of this decision-making problem. the use of the simple network, on the contrary, would have been largely unsuitable because the large number of elements and connections would have seriously weighed down the model. the problem has been divided into five clusters (economic aspects, environmental aspects, social aspects, transport aspects and urban planning aspects) that were organized according to the bocr model. with reference to the alternative options previously described (section 3.1), the general objective of the analysis is to rank the alternative projects according to their overall performance. each decision-making problem is characterized by positive and negative aspects that can emerge in different temporal phases. in this anp model, the benefits and costs have been considered, respectively, as positive and negative aspects of the transformation with reference to a short time period, for which detailed previsions are available. the opportunities and the risks have been considered, respectively, as positive and negative aspects of the transformation over a long time period, and they are difficult to anticipate. table 2 represents the anp model according to the bocr structure. there are four subnets which have different clusters and elements, always including a common cluster of alternatives. as an example, figure 1 shows in detail the benefits subnetwork. table 2 the anp model bocr clusters elements denotation benefits environmental aspects environmental quality improvement b1 economic aspects real estate valorization b2 valorization of the local commercial system b3 investment profitability (tolls and rates) b4 social aspects services improvement for the inhabitants b5 adherence to local community expectations b6 urban planning aspects creation of a polycentric system b7 significance of the project for the urban transformation b8 transport aspects increase in accessibility and mobility for both people and goods b9 m. bottero, v. ferretti, s. pomarico / assessing the sustainability of alternative transport infrastructures international journal of the analytic hierarchy process 65 vol. 4 issue 1 2012 issn 1936-6744 table 2 (cont’d) the anp model opportunities economic aspects trade efficiency o1 possible valorization of the neighboring areas o2 environmental aspects environmental mitigation measures o3 social aspects travelling time reduction o4 transport aspects transport and communication means innovation o5 urban planning aspects revitalization of the area o6 improvement in the image of the town o7 costs economic aspects investment costs c1 operating and maintenance costs c2 environmental aspects soil consumption c3 negative impacts of the construction work c4 air and acoustic pollution c5 social aspects duration of construction work c6 transport aspects complexity of the project c7 traffic congestion due to the construction work c8 risks economic aspects lean investment profitability r1 environmental aspects visual impact r2 impacts on groundwater r3 effects on the ecological connections r4 urban sprawl r5 social aspects social opposition to the project r6 cost of injury r7 transport aspects inefficiency of the transport system r8 environmental quality improvement environmental aspects significance of the project for the urban transformation creation of a polycentric system urban planning aspects alternative 0 alternative x alternative y alternative z alternatives services improvement for the inhabitants adhesion to local community expectations social aspects accessibility and mobility increase for both people and goods transport aspects real estate valorization valorization of local commercial system investment profitability (tolls and rates) economics aspects figure 1 benefits subnetwork 3.3 development of the model the next stage in the analysis according to the anp methodology consists of pairwise comparisons in order to establish the relative importance of the different elements, with respect to a certain component of the network. the comparison m. bottero, v. ferretti, s. pomarico / assessing the sustainability of alternative transport infrastructures international journal of the analytic hierarchy process 66 vol. 4 issue 1 2012 issn 1936-6744 and evaluation phase is divided into two distinct levels: the cluster level, which is more strategic, and the node level, which is more specific and detailed. at the cluster level, the numerical judgments used to fill the pairwise comparison matrices were derived by a specific focus group. the focus group was made up of decision-makers and project coordinators from the local public authorities who worked together to evaluate the different aspects that characterized the problem with respect to the overall objective in order to reach a consensus decision on weights and priorities. the result of this phase is represented by the so-called cluster matrix. questions such as “which aspects will lead to the greatest benefits associated with the transformation project?, and to what extent?” were solved by the focus group considering the cluster of the alternatives as a parent node in the benefits subnetwork. the judgments expressed were used to create the related pairwise comparison matrix (table 3). table 3 pairwise comparison matrix at the cluster level for the benefits subnetwork alternatives environmental aspects economic aspects social aspects transport aspects urban planning aspects priorities environmental aspects 1 1/5 1/3 1/7 1/5 0.040 economic aspects 5 1 3 1/3 1 0.186 social aspects 3 1/3 1 1/5 1/4 0.078 transport aspects 7 3 5 1 5 0.508 urban planning aspects 5 1 4 1/5 1 0.188 table 3 shows the pairwise comparison matrix and the main eingenvector which represents the priorities of the different aspects in the benefit subnetwork with respect to the goal. this result highlights that transport aspects are the most important from the benefits point of view. according to anp methodology, the final priority vectors that result from the comparison matrices at the cluster level determine the columns of the cluster matrix. table 4 shows the cluster matrix for the benefits subnetwork. the priorities of the elements that had previously been compared (table 3) are shown. environmental aspects 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 economic aspects environmental aspects 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 social aspects environmental aspects 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 transport aspects environmental aspects 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 urban planning aspects economic aspects 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 social aspects economic aspects 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 transport aspects economic aspects 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 urban planning aspects social aspects 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 transport aspects social aspects 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 urban planning aspects transport aspects 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 urban planning aspects m. bottero, v. ferretti, s. pomarico / assessing the sustainability of alternative transport infrastructures international journal of the analytic hierarchy process 67 vol. 4 issue 1 2012 issn 1936-6744 table 4 cluster matrix for the benefits subnetwork alternatives environmental aspects economic aspects social aspects transport aspects urban planning aspects alternatives 0.000 0.750 0.594 1.000 0.400 1.000 environmental aspects 0.040 0.000 0.000 0.000 0.046 0.000 economic aspects 0.186 0.000 0.249 0.000 0.275 0.000 social aspects 0.078 0.250 0.157 0.000 0.096 0.000 transport aspects 0.508 0.000 0.000 0.000 0.000 0.000 urban planning aspects 0.188 0.000 0.000 0.000 0.184 0.000 once the clusters comparison had been conducted, it was necessary to study the problem in depth through the analysis of the elements. at the nodes level, in order to create the pairwise comparison matrices the values were derived from the judgments expressed by technical experts in the field of environmental assessment, transport infrastructures, social analysis, urban planning and economic feasibility. each expert received a detailed questionnaire containing only questions about his own field of expertise with reference to the specific issue of the decision-making process. for example, a question submitted to a technical expert in the transport field was: with reference to the evaluation of the priority of the considered projects, from the benefits point of view, which alternative satisfies the objective “accessibility and mobility increase for both people and goods” more closely? and how much more? the judgments expressed were used to fill in the related pairwise comparison matrix (table 5). table 5 pairwise comparison matrix at the node level for the benefits subnetwork accessibility and mobility increase for both people and goods alternative 0 alternative x alternative y alternative z priorities alternative 0 1 1/7 1/3 1/8 0.045 alternative x 7 1 5 1/3 0.307 alternative y 3 1/5 1 1/5 0.096 alternative z 8 3 5 1 0.552 once the pairwise comparison matrices had been compiled, all of the related vectors together formed the unweighted supermatrix. in this case, four supermatrices were obtained, one for each subnetwork. table 6 represents the unweighted supermatrix, with reference to the benefits subnetwork. the priorities of the elements that had previously been compared (tables 5) are shown. alternative 0 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 alternative x alternative 0 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 alternative y alternative 0 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 alternative z alternative x 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 alternative y alternative x 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 alternative z alternative y 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 alternative z m. bottero, v. ferretti, s. pomarico / assessing the sustainability of alternative transport infrastructures international journal of the analytic hierarchy process 68 vol. 4 issue 1 2012 issn 1936-6744 table 6 unweighted supermatrix for the benefits sub network environmental aspects transport aspects 0 x y z b1 b4 b3 b2 b6 b5 b9 b7 b8 0 0.000 0.000 0.000 0.000 0.056 0.033 0.036 0.041 0.122 0.061 0.045 0.042 0.066 x 0.000 0.000 0.000 0.000 0.279 0.264 0.170 0.243 0.444 0.302 0.307 0.290 0.292 y 0.000 0.000 0.000 0.000 0.139 0.188 0.170 0.141 0.122 0.124 0.096 0.085 0.084 z 0.000 0.000 0.000 0.000 0.525 0.515 0.625 0.576 0.312 0.513 0.552 0.583 0.558 environmental aspects b1 1.000 1.000 1.000 1.000 0.000 0.000 0.000 0.000 0.000 0.000 1.000 0.000 0.000 b4 0.333 0.714 0.637 0.683 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 b3 0.333 0.143 0.258 0.200 0.000 0.000 0.000 0.000 0.000 0.000 0.750 0.000 0.000 b2 0.333 0.143 0.105 0.117 0.000 0.000 1.000 0.000 0.000 0.000 0.250 0.000 0.000 b6 0.750 0.250 0.500 0.250 1.000 0.000 0.000 0.000 0.000 0.000 0.125 0.000 0.000 b5 0.250 0.750 0.500 0.750 0.000 0.000 1.000 0.000 0.000 0.000 0.875 0.000 0.000 transport aspects b9 1.000 1.000 1.000 1.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 b7 0.500 0.833 0.750 0.875 0.000 0.000 0.000 0.000 0.000 0.000 0.833 0.000 0.000 b8 0.500 0.167 0.250 0.125 0.000 0.000 0.000 0.000 0.000 0.000 0.167 0.000 0.000 urban planning alternatives economic aspects social aspects urban planning aspects alternatives economic aspects social aspects finally, according to the anp methodology, the cluster matrix was applied to the initial supermatrix as a cluster weight. the result was the weighted supermatrix, which was raised to a limiting power in order to obtain the limit supermatrix, where all columns were identical and each column gave the global priority vector. in this case, four limit supermatrices were obtained, one for each subnetwork. 3.4 final results each column of the limit supermatrices obtained from the four subnetworks provides the final priority vector of all the elements being considered (table 7). leaving aside the alternative options, table 7 shows the priorities of the elements of the model. table 7 final priorities of the elements of the model b1 0.027 o1 0.065 c1 0.068 r1 0.050 b2 0.040 o2 0.103 c2 0.014 r2 0.096 b3 0.059 o3 0.015 c3 0.025 r3 0.054 b4 0.052 o4 0.047 c4 0.028 r4 0.055 b5 0.050 o5 0.091 c5 0.016 r5 0.110 b6 0.019 o6 0.166 c6 0.125 r6 0.086 b7 0.098 o7 0.056 c7 0.115 r7 0.033 b8 0.020 c8 0.171 r8 0.036 b9 0.214 benefits opportunities cos ts ris ks the results of the complex anp model highlight that the most important elements in the decision-making problem are: i) increase in accessibility and mobility for both people and goods (transport aspects cluster) for the benefits subnetwork (0.214); ii) the revitalization of the area (urban planning aspects) for the opportunities subnetwork (0.166); iii) the traffic congestion due to the construction m. bottero, v. ferretti, s. pomarico / assessing the sustainability of alternative transport infrastructures international journal of the analytic hierarchy process 69 vol. 4 issue 1 2012 issn 1936-6744 work (transport aspects) for the costs subnetwork (0.171); and iv) the urban sprawl (environmental aspects) for the risks subnetwork (0.110). 3.5 aggregation of the final results by means of the strategic criteria in the case of the bocr model, it is necessary to synthesize the outcome of the alternative priorities in order to obtain an overall ranking. according to the anp technique, the analysis can make use of strategic criteria organized in an additional top layer, which can be very useful for determining the weights of the bocr merits; these weights can be used to aggregate the priority vectors from the four subnetworks and to obtain the final list. the idea of using strategic criteria comes from the necessity of linking the global objectives of the problem to the particular decision. according to this approach, it is possible to link the global invariant strategic criteria to the alternatives in order to evaluate the importance of the bocr elements for the decision, instead of directly comparing each component of bocr against each other with respect to the goal. using strategic criteria in the model leads to a multi-layered structure with benefits, opportunities, costs and risks (bocr) merit nodes at the top level of the network, control criteria within each of the four attached subnetworks and finally bottom level decision subnetworks which contain the alternatives that are in turn attached to the control criteria. the top level network also has an attached ratings component for evaluating the importance of the bocr through the use of the strategic criteria (saaty, 2003). the mechanism for using the strategic criteria in the model can be described as follows: (i) definition of the strategic criteria; (ii) weighting of the strategic criteria with respect to the goal; (iii) determination of the bocr weights with respect to the strategic criteria; (iv) use of the bocr weights in the aggregation formula for the overall synthesis of the alternative priorities. although the weighting of the strategic criteria with respect to the goal is made according to the pairwise comparison approach (saaty, 1980), the determination of the bocr weights may follow a different procedure. it is possible to rate how the highest priority alternative in the four bocr subnetworks affects each strategic criteria by means of the rating process. this process makes use of a matrix, where the columns represent the strategic criteria and the rows represent the bocr elements. keeping in mind the best alternative under benefits, opportunities, costs and risks, it is necessary to rate across the row how this alternative affects each strategic criterion respectively in a beneficial, opportune, costly and risky way. for example, considering the opportunities subnetwork, it is necessary to assess how the best alternative under opportunities impacts the strategic criteria in a positive way. if the rating categories are high, medium and low, selecting high on a strategic criterion means that the alternative is really good and positive for that strategic criterion. turning, for example, to costs, using the same high, medium and low categories, it is possible to assess how much the highest priority alternative costs for each of the control criteria. selecting high on a strategic criterion means that the highest priority alternative costs a lot for that strategic objective (saaty, 2010). there is a large amount of literature pertaining to the use of strategic criteria in anp models (saaty and ozdemir, 2008; www.superdecision.com). in the present application, the strategic criteria reflect the general objectives which have to be pursued in territorial transformation projects, named “population well-being and quality of life”, m. bottero, v. ferretti, s. pomarico / assessing the sustainability of alternative transport infrastructures international journal of the analytic hierarchy process 70 vol. 4 issue 1 2012 issn 1936-6744 “optimization in the use of natural resources” and “equity in the reallocation of the economic benefits”. figure 2 gives a representation of the anp model considering the use of strategic criteria while table 8 shows how the bocr merits were rated on the set of the aforementioned strategic criteria. assessment of the different scenarios population well-being and quality of life optimization in the use of natural resources equity in the reallocation of the economic benefits benefits opportunities riskscosts figure 2 representation of the complete multilayered anp model including the use of strategic criteria for the decision problem under examination in prioritizing the strategic criteria, the aspects related to “population well-being and quality of life” were given the highest importance (0.65 in the final priority vector), followed by the aspects related to the “equity in the reallocation of the economic benefits” (0.25), and finally by the aspects related to the “optimization in the use of natural resources” (0.10). this leads to the following priorities for the bocr merits: 0,47 for benefits, 0.12 for opportunities, 0.35 for costs and 0.06 for risks. table 8 strategic criteria and rating scale for the anp model bocr priorities equity in the reallocation of the economic benefits [0,25] population well-being and quality of life [0,65] optimization in the use of natural resources [0,10] benefits 0.47 high high medium opportunities 0.12 medium medium medium costs 0.35 low high medium risks 0.06 medium low low the aforementioned priorities were used in order to synthesize the outcome of the alternatives for each of the bocr subnetworks and to obtain the overall synthesis. in particular, the global formula bb+oo-cc-rr was applied, where b, o, c and r represent the priorities obtained by rating the bocr with respect to the strategic criteria, and b, o, c and r represent the ideal priorities of the alternatives in the benefits, opportunities, costs and risks subnetworks, respectively (saaty, 2005). the results of the calculations showed that “alternative z” has the highest priority (0.54), followed by “alternative x” (0.32), then by “alternative y” (0.08) and finally by “alternative 0” (0.05). in particular, from the benefits point of view, the objectives related to the “equity in the reallocation of the economic benefits” and to the “population well-being and quality of life” are fulfilled very well by “alternative z”, while it fulfils less well the objective related to the “optimization in the use of natural resources” (due to atmospheric and acoustic emissions associated with the alternative). from the opportunities point of view, m. bottero, v. ferretti, s. pomarico / assessing the sustainability of alternative transport infrastructures international journal of the analytic hierarchy process 71 vol. 4 issue 1 2012 issn 1936-6744 “alternative z” fulfils the strategic objectives moderately well (the assessment is uncertain due to the long term temporal horizon). from the costs point of view, “alternative z” is not expensive with reference to the “equity in the reallocation of the economic benefits”, but rather expensive with reference to “population well-being and quality of life” (significant impacts during the construction phase) and to the “optimization in the use of natural resources” (land consumption). finally, from the risks point of view, “alternative z” is moderately risky with reference to the “equity in the reallocation of the economic benefits” (uncertain profit, possible need of public funding), but less risky in terms of “population well-being and quality of life” and “optimization in the use of natural resources”. the analysis performed thus provides a transparent decision-making structure, making key considerations and values explicit and providing opportunities for stakeholders and community participation. 3.6 sensitivity analysis in order to test the model’s robustness, a sensitivity analysis was performed after obtaining a ranking of the alternatives. a sensitivity analysis is concerned with the “what if” kinds of questions to see if the final answer is stable when the inputs, whether judgments or priorities, are changed. as a matter of fact, it is of special interest to see whether these changes modify the order of the alternatives. in the present paper three different sensitivity analyses were undertaken in order to study the robustness of the model with respect to the components and interdependencies of the network. in the first analysis, the stability of the solution was studied with regard to the control criteria (bocr) priorities. in the second, the analysis explored the modification of the influences of the alternatives on the criteria and of the criteria on the alternatives. finally, in the third study, an attempt was made to verify the rank reversal of the alternatives (saaty, 2006) by eliminating one alternative at a time from each subnetwork of the model and thus studying the resulting final ranking, searching for potential changes. in the first study, while measuring the sensitivity of the alternatives to the bocr weights, an additive formulation was used, since the meaningful changes could not be obtained by a multiplicative formulation (tuzkaya et al., 2007). the sensitivity analysis for the four subnetworks is represented in figure 3, where the x axis represents the changes in the weights of the control criteria and the y axis represents the changes in the weights of the alternatives. when the relationships between the benefits dimension and the road infrastructure alternatives are considered it becomes clear that “alternative z” provides more benefits compared to the other options (fig. 3a). the sensitivity analysis shows that the costs dimension is quite an unstable subnetwork (figure 3b), since both the results and the rank of the alternatives are very sensitive to the changes in the weight of the costs. the ranking of the alternatives changes from “alternative z”“alternative x”“alternative y”“alternative 0” (for 0% cost weight) to “alternative 0”“alternative y”“alternative x”“alternative z” (for 100% cost weight). in the opportunities subnetwork (figures 3c) the road infrastructure alternatives are almost completely insensitive to the changes in the weight of the control criteria. finally, the sensitivity analysis shows that the risks dimension (figure 3d) is the most unstable subnetwork, and five inversions in the ranking of the alternatives can be identified. m. bottero, v. ferretti, s. pomarico / assessing the sustainability of alternative transport infrastructures international journal of the analytic hierarchy process 72 vol. 4 issue 1 2012 issn 1936-6744 0 0,1 0,2 0,3 0,4 0,5 0,6 0 0,2 0,4 0,6 0,8 1 opportunities absence of intervention scena rio x scena rio y scena rio x + scena rio y alternative 0 alternative y alternative x alternative z -0,4 -0,2 0 0,2 0,4 0,6 0,8 0 0,2 0,4 0,6 0,8 1 benefits absence of intervention scena rio x scena rio y scena rio x + scena rio y alternative 0 alternative y alternative x alternative z -0,6 -0,4 -0,2 0 0,2 0,4 0,6 0,8 0 0,2 0,4 0,6 0,8 1 risks absence of intervention scena rio x scena rio y scena rio x + scena rio y alternative 0 alternative y alternative x alternative z -0,8 -0,6 -0,4 -0,2 0 0,2 0,4 0,6 0,8 0 0,2 0,4 0,6 0,8 1 costs absence of intervention scena rio x scena rio y scena rio x + scena rio y alternative 0 alternative y alternative x alternative z (a) (b) (c) (d) figure 3 sensitivity analysis for each subnetwork using the additive (reciprocal) formula in the second sensitivity analysis, the influences between the elements and the alternatives were modified and the resulting final priority list of the alternatives was analyzed in order to see if any changes occurred. in particular, the priorities of the elements of the model resulting from the limit supermatrix were considered and, for each of the four subnetworks, the highest priority alternative and the highest priority element in each cluster were taken into account. in order to perform the analysis, the influences among the aforementioned elements were changed in the unweighted supermatrix. the original values were modified by +/50% in a three-step process (aragonés-beltràn et al., 2010), resulting in several possible combinations which have generated new rankings of alternatives. table 9 represents the results of the analysis where, for each subnetwork, the first iteration represents the original values and the corresponding final priorities of the alternatives. instead the following iterations consider the subsequent modifications of influences and the resulting new priorities. m. bottero, v. ferretti, s. pomarico / assessing the sustainability of alternative transport infrastructures international journal of the analytic hierarchy process 73 vol. 4 issue 1 2012 issn 1936-6744 table 9 sensitivity analysis modifying influences among elements and alternatives in the unweighted supermatrix iteration unweighted supermatrix values limit priorities b e n e f it s b3 vs z b5 vs z b7 vs z z vs b3 z vs b5 z vs b7 0 x y z 1 0.1998 0.7500 0.875 0.625 0.5135 0.583 0.021 0.119 0.051 0.230 2 0.2997 0.9999 0.9999 0.625 0.5135 0.583 0.020 0.118 0.050 0.230 3 0.1998 0.75 0.875 0.937 0.77 0.875 0.015 0.086 0.037 0.283 4 0.0999 0.375 0.438 0.625 0.5135 0.583 0.021 0.121 0.051 0.228 5 0.1998 0.75 0.875 0.312 0.257 0.292 0.026 0.152 0.065 0.178 c o s t s c4 vs z c1 vs z c8 vs z z vs c4 z vs c1 z vs c8 0 x y z 1 0.163 0.833 0.5 0.568 0.658 0.497 0.021 0.132 0.053 0.233 2 0.245 0.9999 0.75 0.568 0.658 0.497 0.021 0.137 0.052 0.237 3 0.163 0.833 0.5 0.853 0.988 0.746 0.013 0.081 0.036 0.318 4 0.082 0.417 0.25 0.568 0.658 0.497 0.020 0.129 0.053 0.229 5 0.163 0.833 0.5 0.284 0.329 0.373 0.027 0.168 0.063 0.181 o p p o r t u n it ie s o2 vs z o6 vs z z vs o2 z vs o6 0 x y z 1 0.5 0.75 0.692 0.471 0.031 0.135 0.061 0.230 2 0.75 0.9999 0.692 0.471 0.030 0.129 0.063 0.238 3 0.5 0.75 0.9999 0.706 0.021 0.102 0.040 0.294 4 0.25 0.375 0.692 0.471 0.033 0.142 0.059 0.220 5 0.5 0.75 0.346 0.235 0.041 0.170 0.084 0.162 r is k s r5 vs z r6 vs z z vs r5 z vs r6 0 x y z 1 0.356 0.75 0.417 0.565 0.046 0.076 0.143 0.215 2 0.535 0.9999 0.417 0.565 0.046 0.074 0.147 0.214 3 0.356 0.75 0.625 0.848 0.040 0.056 0.120 0.262 4 0.178 0.375 0.417 0.565 0.046 0.077 0.139 0.216 5 0.356 0.75 0.208 0.283 0.053 0.094 0.165 0.169 for the opportunities and risks subnetworks, less iterations were performed: this happens in those cases where there is only one element in a cluster to be put in correlation to the highest priority alternative. as can be seen, there are no relevant changes in the new ranking of the alternatives compared to the original one. finally, we tried to investigate the possibility of rank reversal of the alternatives (saaty, 2006) by developing a third sensitivity analysis. this analysis eliminated one alternative at a time from the original model, and evaluated the new results. table 10 thus illustrates, for each subnetwork of the model, the original ranking of the alternatives and the results arising from the elimination of the highest priority alternative. acknowledging that rank can and should reverse under general conditions that have been recognized such as introducing copies or near copies of alternatives and criteria (saaty et al., 2009), the question is not whether rank should be preserved (tversky et al. 1990), but whether or not the assumption of independence applies (saaty et al., 2009). according to the proposed sensitivity analysis the rank reversal of the alternatives does not occur (table 10). m. bottero, v. ferretti, s. pomarico / assessing the sustainability of alternative transport infrastructures international journal of the analytic hierarchy process 74 vol. 4 issue 1 2012 issn 1936-6744 table 10 sensitivity analysis with respect to the rank reversal of the alternatives subnetwork priority of the alternatives original ranking eliminated alternative new priorities new ranking benefits 0: 0.05 x: 0.28 y: 0.12 z: 0.55 z>x>y>0 z 0: 0.08 x: 0.66 y: 0.26 x>y>0 opportunities 0: 0.07 x: 0.30 y: 0.13 z: 0.50 z>x>y>0 z 0: 0.11 x: 0.59 y: 0.30 x>y>0 costs 0: 0.05 x: 0.30 y: 0.12 z: 0.53 z>x>y>0 z 0: 0.09 x: 0.68 y: 0.23 x>y>0 risks 0: 0.10 x: 0.16 y: 0.30 z: 0.45 z>y>x>0 z 0: 0.15 x: 0.35 y: 0.50 y>x>0 the purpose of the sensitivity analysis was to create an explanatory process by which the decision makers achieve a deeper understanding of the structure of the problem. it is helpful to the analyst to learn how the various decision elements interact in order to determine the most preferred alternative and to determine which elements are important sources of disagreement among dms and interest groups. thus the anp is not only aids in selecting the best alternative, but also helps dms to understand why one alternative is preferred over the other options (khan and faisal, 2008). 4. discussion and conclusions this paper illustrates the application of the complex anp method to support the choice between different projects for the further development of the existing ring road of a city in italy. the technique allows the most important elements of the decision to be highlighted through a transparent and traceable decision-making process thus facilitating deliberation. moreover, the technique supports communication with the dms and grants mutual understanding. the results of the analysis that was performed show that the anp bocr model is suitable to represent a real world problem. in fact, the technique provides the means to perform complex trade-offs on multiple evaluation criteria, while taking the dm’s preferences into account. the main drawback in the practical application of the anp is a consequence of the complexity of the decision-making issue being analysed. for example, the anp prescribes a large number of comparisons that occasionally become too complex for dms to understand if they are not familiar with the method. hence, a great deal of attention should be devoted to the writing up of the questionnaires and the comparison process should be helped by a facilitator (aragonés-beltrán et al., 2010). however, there are still a number of opportunities for expanding the study and for validating the obtained results. first, it would be of scientific interest to assess the strategic criteria through a participatory focus group in order to move the collaborative decision processes forward. second, the model could be combined with a costs-benefits analysis in order to develop an overall assessment of the transformation project impacts (tsamboulas and mikroudis, 2000). in conclusion, the adopted methodology was successful in structuring the complex planning context, in m. bottero, v. ferretti, s. pomarico / assessing the sustainability of alternative transport infrastructures international journal of the analytic hierarchy process 75 vol. 4 issue 1 2012 issn 1936-6744 communicating the stakeholders’ perspectives, in improving the stakeholders’ commitment and perception of being involved, in enhancing transparency in the decisionmaking process and thus in increasing acceptance of the proposed solutions. references aragonés-beltrán, p., pastor-ferrando, j.p., garcìa-garcìa, f., & pascual-agullò, a. (2010). an analytic network process approach for siting a municipal solid waste plant in the metropolitan area of valencia (spain). journal of environmental management, 91, 1071-1086. basbas, s., & papanikolaou, a. (2009). evaluation of a sustainable urban transport system through the use of the transecon methodology. international journal of sustainable development and planning 4(1), 18-34. bojković, n., macura, d., pejčić-tarle, & s., bojović, n. (2011). a comparative assessment of transport-sustainability in central and eastern european countries with a brief reference to the republic of serbia. international journal of sustainable transportation, 5(6), 319-344. bottero, m., & ferretti, v. (2010). integrating the analytic network process (anp) and the driving orce-pressure-state-impact-response (dpsir) model for the sustainability assessment of territorial transformations. management of environmental quality, 21(5), 618-644. bottero, m., & ferretti, v. (2011). an analytic network process-based approach for location problems: the case of a waste incinerator plant in the province of torino (italy). journal of multicriteria decision analysis, 17, 63–84. bottero, m., & lami, i.m., (2010). analytic network process and sustainable mobility: an application for the assessment of different scenarios. journal of urbanism 3(3), 275293. bottero, m., lami, i.m., & lombardi, p. (2008). analytic network process. la valutazione di scenari di trasformazione urbana e territoriale. firenze: alinea. bottero, m., & mondini, g. (2008). an appraisal of analytic network process and its role in sustainability assessment in northern italy. international journal of management of environmental quality, 19(6), 642660. figueira. j., greco, s., & ehrgott, m. (eds.). (2005). multiple criteria decision analysis. state of the art survey. new york: springer. horsch k., anderson c. (1997), interview with robert yin. the evaluation exchange, 3(1), 4. http://www.hfrp.org/evaluation/theevaluation-exchange/issuearchive/methodology/interview-with-robertk.-yin khan, s., & faisal, m.n. (2008). an analytic network process model for municipal solid waste disposal options. waste management, 28(9), 1500-1508. m. bottero, v. ferretti, s. pomarico / assessing the sustainability of alternative transport infrastructures international journal of the analytic hierarchy process 76 vol. 4 issue 1 2012 issn 1936-6744 liang, c., & li, q. (2008). enterprise information system project selection with regard to bocr. international journal of project management, 26(8), 810-820. liu, k.f.r., & lai, j.h. (2009). decision-support for environmental impact assessment: a hybrid approach using fuzzy logic and fuzzy analytic network process. expert systems with applications, 36(3), 5119-5136. lombardi, p. (2009). evaluation of sustainable urban redevelopment scenarios. proceeding of the institution of civil engineers: urban design and planning, 162(4), 179-186. neaupane, k.m., & piantanakulchai, m. (2006). analytic network process model for landslide hazard zonation. engineering geology, 85, 281-294. niemura, m.p., saaty, t.l. (2004). an analytic network process model for financialcrisis forecasting. international journal of forecasting, 20(4), 573-587. pèti, m. (2012). a territorial understanding of sustainability in public development. environmental impact assessment review, 32(1), 61-73. piantanakulchai, m. (2005). analytic network process model for highway corridor planning. proceedings of isahp 2005. honolulu, hawaii. saaty, rw. (2003). decision-making in complex environments: the analytic hierarchy process (ahp) for decision-making and the analytic network process (anp) for decision-making with dependence and feedback. pittsburgh, pa: rws publications. saaty, rw. (2010). routing bocr against strategic criteria. (working paper). saaty, t.l. (1980). the analytic hierarchy process. new york: mcgraw hill. saaty, t.l. (2005). theory and applications of the analytic network process: decision making with benefits, opportunities, costs, and risks. pittsburgh, pa: rws publications. saaty, tl. (2006). rank from comparisons and from ratings in the analytic hierarchy/network processes. european journal of operational research, 168, 557–570. saaty, tl, ozdemir, ms. (2008). the encyclicon. a dictionary of complex decisions using the analytic network process. pittsburgh, pa: rws publications. saaty, t.l., & vargas, l.g. (2006). decision making with the analytic network process. new york: springer. saaty, t., vargas, l., whitaker, r. (2009). addressing with brevity criticisms of the analytic hierarchy process. international journal of the analytic hierarchy process, 1(1), 121-134. tsamboulas, d., mikroudis, g. (2000). efet-evaluation framework of m. bottero, v. ferretti, s. pomarico / assessing the sustainability of alternative transport infrastructures international journal of the analytic hierarchy process 77 vol. 4 issue 1 2012 issn 1936-6744 environmental impacts and costs of transport initiatives. transportation research, 5, 283-303. tversky, a., slovic p., kahneman, d. (1990). the causes of preference reversal. the american economic review, 80(1), 204-215. tuzkaya, g., & onut, s. (2008). a fuzzy analytic network process based approach to transportation-mode selection between turkey and germany: a case study. information sciences, 178(15), 3133-3146. tuzkaya, g., onut, s., tuzkaya, u.r., gulsun, b. (2007). an analytic network process approach for locating undesirable facilities: an example from istanbul, turkey. journal of environmental management, 88, 970–983. ulutas, b.h. (2005). determination of the appropriate energy policy for turkey. energy, 30(7), 1146-1161. yin, r. (1984). case study research: design and methods (1st ed.). beverly hills, ca: sage publishing. zoffer, j., bahurmoz, a., hamid, m.k., minutolo m., & saaty t. (2008). synthesis of complex criteria decision making: a case towards a consensus agreement for a middle east conflict resolution. group decision negotiation, 17, 363-385. 4 final_abastante_bottero_lami ijahp article: abastante, f., bottero, m., lami, i.m / using the anp for ranking the critical issues of a transport decision problem international journal of the analytic hierarchy process 41 vol. 4 issue 1 2012 issn 1936-6744 using the analytic network process for addressing a transport decision problem francesca abastante department of regional and urban studies and planning (dist) politecnico di torino e-mail: francesca.abastante@polito.it marta bottero department of regional and urban studies and planning (dist) politecnico di torino e-mail: marta.bottero@polito.it isabella m. lami∗ department of regional and urban studies and planning (dist) politecnico di torino e-mail: isabella.lami@polito.it abstract one of the crucial issues for decision makers when considering transport choices is how to simultaneously optimize several criteria that take into account technical, economic, territorial and environmental constraints., the present paper proposes the application of the analytic network process (anp) methodology for supporting the decision-making process related to the implementation of the railway corridor in italy specifically the trans-european railway axis from rotterdam to genoa (i.e. corridor 24). in particular, the objective of the work is to rank the effects that a delay in the construction of the italian portion of the corridor would have on the territorial system. the full range of possible effects have been identified and grouped into three clusters (socio-economic aspects, environmental aspects and transport aspects). the anp model includes both subjective and objective elements which also have some interdependencies. the complexity of the case under examination made it necessary to consult iterative experts and manage the process through a specific focus group and different questionnaires. the most important aspects of the decision problem were discovered as a result of this application. keywords: analytic network process, transport infrastructures, evaluation, corridor 24. 1. introduction transport policy is a current and serious topic in both industrialized and developing countries. in the past, further transport investment in cities has been supported by arguing ∗ corresponding author rob typewritten text http://dx.doi.org/10.13033/ijahp.v4i1.119 ijahp article: abastante, f., bottero, m., lami, i.m / using the anp for ranking the critical issues of a transport decision problem international journal of the analytic hierarchy process 42 vol. 4 issue 1 2012 issn 1936-6744 on the basis of growth allocation and, subsequently, as the main means to promote economic development and revitalization of depressed areas (banister, 1994). this topic has now been broadened to embrace new aspects, mostly importantly, an, enhanced awareness of the issue of sustainability (whitehead et al., 2006). for these reasons, the necessity of decision support tools which are able to simultaneously consider different aspects of the problems related to transport planning is getting more and more evident (jefferson, 1996). there are several features of transport problems and models which must be taken into account when determining which analytical approach to use (ortuzar and willumsen, 2001). to start with, it is necessary to consider the decision-making context. this element involves the adoption of a particular perspective, and requires a choice of scope or coverage of the system of interest. it also helps to define requirements for the models to be used such as the variables to be included in the model whether given or exogenous. second, the availability of suitable data has to be taken into account. the stability of the data and the difficulty involved in forecasting their future values must be considered in many cases the data available will be the key factor in deciding the modeling approach. moreover, it is necessary to consider the accuracy required, the state of the art in modeling and the resources available for the study, with particular reference to the time involved and the level of communication with the decision makers and the public. finally, the data processing requirements and the levels of training and skill of the analysts must also be taken into account. despite the fact that land use and transport systems are closely intertwined, the integration of planning between these systems is still far off for several reasons which include the controversy of the domain (many institutional and non-institutional stakeholders with divergent values and mandates), the complexity of the issues, and the high level of interdependencies. different institutions that have responsibility in the domain often have competing mandates. “in this regard, in order to create agreement, it is tantamount to asking institutions to act in ways that are not consistent with their mandates and the interests of their immediate stakeholders” (waddel, 2011). this is particularly evident in the case of the project of a corridor, particularly in italy. the difficulties emerge around the definition of the concept of “corridor”. in the field of architecture the corridor concept and its performance characteristics are reasonably clear. its primary purpose is to give access to a variety of different rooms, areas or activities. functionally and economically, it is necessary to build as short a corridor as possible, while providing effective access to all accommodation requests. the corridor can be seen as a dynamic space but at the same time as a product to create a series of experiences. by contrast, corridors of development and infrastructure may need to perform in a variety of different ways that are so divergent as to create conflicts between them (chapman et al., 2003). the corridor concept in this sense is not easy to define. it is not always clear which are the territorial areas included in the corridor and, as a consequence, which are the political parties involved in the process. normally, the corridor is seen as a "multifunctional backbone” that includes transport infrastructure for people and goods, high-level services (research, logistics, etc.), and the creation of spatial and environmental effects all within a framework of cross-regional and local policies. this ijahp article: abastante, f., bottero, m., lami, i.m / using the anp for ranking the critical issues of a transport decision problem international journal of the analytic hierarchy process 43 vol. 4 issue 1 2012 issn 1936-6744 very broad definition of "multifunctional backbone” covers a wide range of services to and from a specific territory where the development of infrastructure must be seen as a strategic driver for the transformation of that territory and not only as a project that meets the needs of a particular sector. the definition of a corridor is not a formalized category nor is it the outcome of a government decision; it is one of the many policies where strategies, clusters and alliances between the various players act (fubini, 2008). the corridors can exist as an axis of infrastructure, economic development, urbanization and institutional development, but these four dimensions should be viewed as both qualitatively and functionally very different elements. the dimensions may coexist but they can also be seen as acting at quite different scales (romein et al., 2000). although the term "corridor" clearly suggests the concept of connection and access, it may fail to adequately represent all aspects (subtle but crucial) related to the above four dimensions. there is also the problem of scale and scope that suggests a natural geographic shape, a linear rather than an institutional structure, and an idea of homogeneity rather than distinctiveness. while in infrastructure and institutional connections it is clearly desirable to be able to perform effectively, it’s less clear how such high levels of consistency are "necessary” in terms of economic development or urbanization (chapman et al., 2003). therefore we think that the use of decision support tools, like the analytic network process (saaty, 2005; saaty and vargas, 2006), which are able to simultaneously consider different aspects of the problem, can help the stakeholders and especially the decision makers to reflect on the effects of a delay in the construction of the rotterdamgenoa section of the italian section of the rotterdam-genoa corridor (i.e. corridor 24). the whole corridor connects areas that are profoundly different when considering the four dimensions mentioned above and the effects they would have on the territorial system. 2. context and objectives of the study the goal of the trans-european railway axis (ten-t) no. 24 from rotterdam to genoa (i.e. corridor 24) is the interconnection of economic development, spatial, transport and ecological planning. from a european point of view, there is the need to strengthen links between countries to facilitate freight transport considering that difficulties arising from geographical context with orographic obstacles, administrative barriers and characteristics of the railway infrastructure are often not compatible with each other. the project area of corridor 24 covers a number of the most important economic regions in europe, crossing the netherlands, germany, switzerland and italy, and linking the north sea port of rotterdam and the mediterranean port of genoa with a catchment area of 70 million inhabitants and operating 50% (700 million tons/year) of the north-south rail freight (figure 1). ijahp article: abastante, f., bottero, m., lami, i.m / using the anp for ranking the critical issues of a transport decision problem international journal of the analytic hierarchy process 44 vol. 4 issue 1 2012 issn 1936-6744 figure 1 the trans-european railway axis corridor 24 (source: www.code-24.eu) ijahp article: abastante, f., bottero, m., lami, i.m / using the anp for ranking the critical issues of a transport decision problem international journal of the analytic hierarchy process 45 vol. 4 issue 1 2012 issn 1936-6744 the european union's objective is to double by 2020 the capacity of rail transport on the axis in order to encourage a modal shift of freight by rail. the main projects relating to this objective are the swiss rail tunnel loetschberg (opened in 2007), the gotthard tunnel (the last wall was torn down in october 2010 and the tunnel will be operational by 2017), and mount ceneri tunnel (which is expected in 2020). the overall objective is to jointly develop and accelerate the transport capacity of the entire corridor by ensuring optimal economic benefits and spatial integration while reducing negative impacts on the environment at both the local and regional levels. by focusing on regional aspects in the corridor area and joint development strategies, the project will strengthen the position of regional actors and stakeholders within the entire corridor. there are still many problems that exist despite of the importance of this connection in terms of freight traffic and passenger transport these problems include infrastructure (as many sections do not have adequate capacity of functioning in the corridor), management, due to the presence of different transport services (freight, long distance, local traffic), and a lack of coordination and interoperability at the trans-regional level. it is a priority to attain state of the art infrastructure and maintain the minimum quality standard along the corridor in order to continue the use of the network however this is currently not completely satisfactory. in italy, the old and poor connection between the port of genoa, the lombardy region and the swiss border makes it necessary to transport goods using the road network with evident environmental and territorial problems. in fact, since many systems are implementing “knots systems”, the increase in speed must be great enough to accommodate the increase in the capacity required to produce benefits to all transport modality. in particular, in italy, the bottlenecks mainly involve the access to lotschberg and gottardo railway tunnels, the doubling of the existing lines and their adaptation to freight, the improvement of some critical nodes (first of all the port of genoa), and the overcoming of the apennines barrier (“terzo valico dei giovi”). the obsolescence of railway lines and the lack of financial resources are not the only critical issues limiting the development of the corridore24 in italy; the capacity utilization of the logistic terminal in milan, in the lombardy region, which is now sufficient will become inadequate for the new european requirements by 2015 and, therefore the development of a north-west italy intermodal transport network will be necessary to remain competitive in the european scenario (figure 2). ijahp article: abastante, f., bottero, m., lami, i.m / using the anp for ranking the critical issues of a transport decision problem international journal of the analytic hierarchy process 46 vol. 4 issue 1 2012 issn 1936-6744 figure 2 north-west italy planned railway infrastructure (source: oti nord ovest, 2010) 1. laveno – luino, doubling works (preliminary design) 2. como – seregno – bergamo, doubling and upgrading works (preliminary design) 3. pavia – milano rogoredo, quadrupling works (preliminary design) 4. tortona – voghera, quadrupling works (design) 5. vignale – oleggio – arona, doubling works (design) 6. nodo di novara, reorganization of the railway node (design) 7. gallarate – rho, upgrading and doubling works (design) 8. variante di gozzano, work in progress from 2009 9. new line torino – lione, in the desing phase, end by 2025 10. pontremolese, line doubling (in contruction) 11. nodo di genova, line quadrupling voltrisanpierdarena and 6-track line between principe and bringnole (in construction) 12. genova – ventimiglia, line doubling between san lorenzo al mare and finale ligure (in costruction) 13. terzo valico dei giovi, works authorized for the first lot, expected work completion after 2019 ijahp article: abastante, f., bottero, m., lami, i.m / using the anp for ranking the critical issues of a transport decision problem international journal of the analytic hierarchy process 47 vol. 4 issue 1 2012 issn 1936-6744 major bottlenecks, a lack of trans-regional coordination, diminishing consent among the involved population, and increasing difficulties because of doubts about financial resources still threaten the potential of the axis in europe, limiting its economic and spatial development. the italian situation however is particularly worrying because italy lags behind in the implementation of the corridor with respect to the rest of europe. the european routes will be finished at the latest in 2020, but in italy the projects that have already obtained financial support risk not being completed by that deadline. according to the studies conducted by the european working group of corridor 24, in an optimistic view, italy may be able to complete the planned works in the alpine zone, but not those of ligurian ports and the south po valley. in this case, the increased permeability of the north italy would not be compensated by a port development, causing the "conquest" of valley and the isolation of the ligurian ports from europe. however, a pessimistic view, which envisions a general delay in the realization of the planned infrastructure (tunnels, railway lines, ports) is unfortunately considered more probable. from this complex panorama the main issues causing the bottlenecks in the corridor emerge including public acceptance, noise, land management and landscape design, functionality of intermodal hubs and operational aspects as well as the management of planning processes and the financial issues related to major infrastructures. for all these reasons, removing existing bottlenecks requires co-operation across political, organisational and technical bodies, focussing on four main topics: i) railway and settlement development; ii) environmental issues; iii) integrated management of logistics; and iv) communication and stakeholder involvement strategies. extensive collaboration along with the use of evaluation methodologies able to support the decision-making process have to be explored and tested in the areas of conflict regarding local planning competencies and over-regional spatial planning issues. the construction of european transport infrastructures is a complex topic where new values have to be taken into account. it is not a specific question of localisms, nor is it merely an issue of moving goods and people. it does not simply affect only environmental, transport-related or town planning aspects. tackling the issue of largescale infrastructures involves dealing with a maze of decision elements which require new trans-disciplinary approaches. currently, the fundamental issue connected to largescale infrastructures seems to be related to the definition of the underlying agreements, rather than to the construction itself (bertolini, 2001; lami and staffelbach, 2008). in this context, the anp could provide a very useful support in the decision-making process because it allows the different elements (both tangible and intangible) of the problem to be represented according to a network model, and it allows the judgements of experts as well as existing measurements and statistics to be considered in the analysis. in territorial transformation processes (characterized by a long term nature), where different actors are associated in a dynamic context, some indefinite issues will need to be negotiated according to future evaluations and attempts made to mediate between opposing positions that can change during the decision process. through the use of the anp, it becomes possible to compare different objectives, interconnected among each ijahp article: abastante, f., bottero, m., lami, i.m / using the anp for ranking the critical issues of a transport decision problem international journal of the analytic hierarchy process 48 vol. 4 issue 1 2012 issn 1936-6744 other, and measurable with different units of measure; furthermore, the analysis makes it possible to identify new definitions of the problem (bottero et al, 2008). this paper proposes the application of the anp methodology for supporting the decisionmaking process related to the critical issues that could arise from a significant delay in the implementation of the italian section of corridor 24. the starting point of the anp application is the idea that the possibility of not having properly upgraded the italian railway lines when the swiss tunnel gotthard and mount ceneri will be working would bring the railway system in piedmont and lombardy to rapid saturation, and the ligurian ports would lose competitiveness in europe. in particular, the lack of a functional link between the port of genoa and the railway network in the hinterland could relegate the ligurian port to a marginal role with respect to the major ports of northern europe, which are organizing an efficient rail connection with their hinterland. in order to investigate this situation, this paper proposes to use the anp not to assess different alternatives, but to examine and compare the main aspects of the problem and to rank them. it is an unusual use of the methodology, but particularly efficient in a case where there is a lack of exhaustive transformation scenarios and, at the same time, detailed information on key aspects of the decision problem is available. this application of the anp is part of an interreg ivb new project, called “code24”, involving 15 partners from 5 countries for 4 years (2010 – 2013). 3. development of the model 3.1 the anp-based approach territorial transformation processes refer to a multidimensional concept that includes socio-economic, ecological, and technical perspectives, and thus leads to issues that are simultaneously characterized by a high degree of conflict, complexity and uncertainty. particularly, when speaking about transport planning, many objectives have to be considered in the decision making process. these objectives range from the rationalization of the mobility system to the reduction of soil consumption, from the promotion of economic activities to the cut of air pollution due to traffic emissions, and from the endorsement of energy efficiency to the increase in the quality of public spaces. the assessment of alternative scenarios of transport planning is therefore a complex decision problem where different aspects need to be considered simultaneously, taking into account both technical elements, which are based on empirical observations, and non-technical elements, which are based on social visions, preferences and feelings. in this context, the anp method provides a very useful aid which allows the different elements of the decision problem to be represented while also considering their interdependent relationships. the network structure of anp makes it possible to model the various aspects at stake without being concerned about what comes first and what comes next. this way of representing the problem, with fewer constraints than the structure imposed by the analytic hierarchy process (saaty, 1980), is more similar to real situations where the elements act in a non-hierarchical way. a very large and consolidated amount of literature concerning the anp exists in different fields. with regards to transport planning, applications of anp model have been used for selecting ijahp article: abastante, f., bottero, m., lami, i.m / using the anp for ranking the critical issues of a transport decision problem international journal of the analytic hierarchy process 49 vol. 4 issue 1 2012 issn 1936-6744 optimal routes and for designing new corridors (piantanakulchai, 2005; tuzkaya and onut, 2008). from a methodological point of view, the anp requires a network structure to represent the problem, as well as pairwise comparisons to establish the relationships within the structure. in order to develop an anp model, it is necessary to carry out five fundamental steps. step i: development of the structure of the decision-making process first the decision-making structure must be defined through the recognition of its main objective. the objective should later be divided into groups (“clusters”) that are made up of various elements (“nodes”), and alternatives or options. second, the relationships between the different parts of the network must be identified. each element can be a “source”, that is, an origin of a path of influence, or a “sink”, that is, a destination of a path of influences. there are two possible structures for an anp model, a “simple” network and a “complex” network. the “simple” network is a free-modeling approach, which is not supported by any guide or pre-determined structure. it consists of a network which has cycles connecting its components and a loop that connects a component to itself. the “complex” network or bocr (benefits, opportunities, costs, risks) network allows one to simplify the problem by classifying issues in traditional categories of positive and negative aspects (saaty and ozdemir, 2008). the favorable concerns are called benefits, while the unfavorable ones are called costs; the uncertain concerns of a decision are the positive opportunities that the decision might create, and the negative risks that it could entail. each of these four concerns utilizes a separate structure for the decision. a full bocr is in some ways similar to a swot analysis. while the bocr model is expected to catch all the aspects (positive and negative) of the decision in the present and future, the swot analysis focuses more on the external and internal elements of the problem (wijnmalen, 2007). step ii: pairwise comparison as in the ahp, a series of pairwise comparisons are made to establish the relative importance of the different elements with respect to a certain component of the network. in the case of interdependencies, components with the same level are viewed as controlling components of each other. the comparisons are made with the saaty’s fundamental scale. the numerical judgments established at each level of the network make up pair matrices. the weighted priority vector is calculated through pairwise comparisons between the applicable elements. this vector corresponds to the main eigenvector of the comparison matrix (saaty, 1980, 2003). the eigenvector method yields a natural measure of consistency. saaty (1980) defined the consistency index (ci) as in equation (1): ( ) ( )1max −−= nnci λ (1) where λmax is the maximum eigenvalue and n is the number of factors in the judgment matrix. accordingly, saaty (1980) defined the consistency ratio (cr) as in equation (2): ijahp article: abastante, f., bottero, m., lami, i.m / using the anp for ranking the critical issues of a transport decision problem international journal of the analytic hierarchy process 50 vol. 4 issue 1 2012 issn 1936-6744 cr=ci/ri (2) where ri is the consistency index of a randomly generated reciprocal matrix from the 9point scale, with forced reciprocals. saaty (1980) has provided average consistencies (ri values) of randomly generated matrixes (up to 11 x 11 size) for a sample size of 500. the consistency ratio cr is a measure of how a given matrix compares to a purely random matrix in terms of the consistency index. a value of the consistency ratio cr < 0.1 is considered acceptable. larger values of cr require the decision-maker to revise his judgments. step iii: supermatrix formation the supermatrix elements consider the interdependencies that exist among the elements of the system and allow a resolution to be made. it is a portioned matrix where each submatrix is composed of a set of relationships between and within the levels, as represented by the dm’s model (step i). the supermatrix obtained in this step is called the initial supermatrix, and it contains all the eigenvectors that are derived from the pairwise comparison matrixes of the model. the eigenvector obtained from a cluster level comparison with respect to the control criterion is applied to the initial supermatrix as a cluster weight. the result is the weighted supermatrix. step iv: final priorities in this step, the weighted supermatrix is raised to a limiting power, as in equation (3), in order to converge and to obtain, as stated in the perron-frobenius theorem, a long-term stable set of weights that represents the final priority vector. k k w ∞→ lim (3) in the case of the complex network, it is necessary to synthesize the outcome of the alternative priorities for each of the bocr structures in order to obtain their overall synthesis; for this operation different aggregation formulas are available (saaty, 2005). step v: sensitivity analysis the fifth and final step involves carrying out the sensitivity analysis on the final outcome of the model in order to test its robustness (saaty r.w., 2003). with particular reference to the application of the anp proposed in this paper, the anp is not used as a method to determine a priority list of the different alternatives in the decision problem, but as a structured procedure that is able to support the analysis in the identification of the principal aspects to consider in order to come to a decision (bottero and lami, 2010). this choice is due to the current issue in the decision process where despite the importance of developing the corridor 24 being officially shared by all government’s levels, the planning choices are struggling to find real implementation. therefore it is not possible to affirm that there are real alternatives of intervention. what is clear instead is a general concern regarding this stalemate in italy compared to the actions in the rest of europe. in order to translate these concerns into concrete elements, a decision has been made to apply the anp methodology in order to achieve real choices for infrastructural ijahp article: abastante, f., bottero, m., lami, i.m / using the anp for ranking the critical issues of a transport decision problem international journal of the analytic hierarchy process 51 vol. 4 issue 1 2012 issn 1936-6744 and track interventions in other words, in the present study the anp model has not been applied with the aim of ranking different options in the context of the development of the corridor 24. instead, the overall object of the analysis is to determine a priority list of the effects that a delay in the construction of the italian portion of the corridor will have on the territorial system. in this sense, the anp consists of a simple network where the different elements and their reciprocal relationships are represented and linked with the aforementioned goal (nekhay et al., 2009). the model has been developed by means of a specific focus group where seven experts in the different fields (most of whom are currently involved in the “code24” european project) worked together to compile the pairwise comparison matrices. 3.2 structure of the network in the case considered in this paper, the development of the model involves a cluster and node comparison. in fact, as already mentioned, the objective of the performed analysis is not to find the best solution or scenario for the construction of the transport network, but to identify what the critical issues are that can arise from a delay (or failure to complete) of the italian section of the european corridor 24. the full range of effects related to the delay in construction have been identified and grouped into three clusters, namely socio-economic aspects, environmental aspects and transport aspects. these aspects represent the main issues of the decision problem under investigation. the first cluster refers to the consequences that a delay in the construction of the infrastructure could have on the social and economic system, including the real estate market, employment, the attractiveness of the region, and the volume of trade. as far as the environmental aspects are concerned, the analysis takes into consideration the problems that could result from a failure to complete construction of the new railway connection which would leave a large part of the traffic on the road network the environmental effects include a variation of the environmental quality (air and noise pollution), energy consumption, and use of the soil. finally, the transport aspects refer to the adverse consequences that could be expected on the infrastructural system, including road congestion, the costs of transports, the freight travel time and the reliability of the services. the nodes are explained in table 1. table 1 clusters and nodes of the model clusters nodes description environmental aspects increase in acoustic emission the increase in noise is due to the passage of a large amount of trucks. increase in air pollution the increase in air pollution is due to the road congestion, particularly in urban areas. increase in energy consumption the energy consumption is related to the use of private motor vehicles. increase in soil consumption the necessity of new roads causes an increase in the consumption of soil. ijahp article: abastante, f., bottero, m., lami, i.m / using the anp for ranking the critical issues of a transport decision problem international journal of the analytic hierarchy process 52 vol. 4 issue 1 2012 issn 1936-6744 table 1 (cont’d) clusters and nodes of the model clusters nodes description socio economic aspects decrease in property values the decrease in market value is due to the peripheral effect due to the lack of good connections. lack of employment effect the reduction of the existing jobs or, at least, the absence of new jobs, is related to less interest by companies for this area. lower level of attractiveness the companies, factories, offices prefer in general a very well connected location and for this reason they will invest in different areas. reduction of trade the reduction is due to the increase in transport costs and to extra time of travel . transport aspects congestion of road network the congestion of the road network is due to the increased passage of the trucks without an effective alternative by rail. increase in costs of transports the increase in costs of transport is expected due to the costs of fuel and toll. increase in freight travel time the increased passage of trucks and t consequently the congestion of roads causes an increase in freight travel time. less reliable service the risk of an incident is greater in the road transport than in the rail transport, so the freight risk of freight damage is increased and the service is less reliable. once the elements of the decision problem were set up, the relationships among them have been established. finally, all the elements in the clusters have been connected to the goal of the evaluation that has been organized in an autonomous cluster. figure 3 represents the decision network of the problem under examination. figure 3 decision network of the problem goal individuation of the most important critical issues due to the delay in the realization of the italian portion of the corridor24 environmental aspects increase in acoustic emission; increase in air pollution; increase in energy consumption; increase in soil consumption. socio-economic aspects decrease in property values; lack of employment effect, lower level of attractiveness; reduction of trade. transport aspects congestion of road network; increase in costs of transport; increase in freight travel time; less reliable service. ijahp article: abastante, f., bottero, m., lami, i.m / using the anp for ranking the critical issues of a transport decision problem international journal of the analytic hierarchy process 53 vol. 4 issue 1 2012 issn 1936-6744 3.3 pairwise comparison according to the anp methodology, after having structured the decision network, the second step of the analysis consists in creating the pairwise comparison matrices. it is important to highlight that there are two levels of pairwise comparisons in the anp, the cluster level, which is more strategic, and the node level, which is more specialized. the model has been developed by a specific focus group where experts in the different fields worked together. the focus group included different experts in the fields of transport infrastructures, environmental assessment, urban planning, economic evaluation and social sciences. the focus group had the dual purpose of helping to structure the decision problem taking into account the feedback and suggestions coming from the experts, and to compile the pairwise comparison matrices in order to come to a coherent result. we first asked every expert to write down their individual judgments for each question. the given judgments were then illustrated and discussed in the focus group until a shared weight was achieved. according to the anp methodology, in pairwise comparisons, a ratio scale of 1-9, that is the saaty’s fundamental scale, is used to compare any two elements translating qualitative variables in numerical values and vice-versa (table 2). the main eigenvector of each pairwise comparison matrix represents the synthesis of the numerical judgements established at each level of the network (saaty, 1980). table 2 saaty’s fundamental scale value definition explanation 1 equally important two decision elements equally influence the parent decision element. 3 moderately more important one decision element is moderately more influential than the other. 5 much more important one decision element has more influence than the other. 7 very much more important one decision element has significantly more influence over the other. 9 extremely more important the difference between influences of the two decision elements is extremely significant. 2, 4, 6, 8 intermediate judgment value judgment values between equally, moderately, much, very much and extremely. in the presented application all the calculations have been implemented using the super decisions software (www.superdecisions.com). the questions that were generated considering the pairwise comparison at the clusters level were similar to the following: among the “critical issues due to the delay in the realization of the italian portion of corridor 24”, which of these two aspects do you think is more important? to what extent? ijahp article: abastante, f., bottero, m., lami, i.m / using the anp for ranking the critical issues of a transport decision problem international journal of the analytic hierarchy process 54 vol. 4 issue 1 2012 issn 1936-6744 environmental aspects socio-economic aspects e x tr em el y m or e im p or ta n t v er y m u ch m or e im p or ta n t m uc h m o re im p or ta n t m od er at el y m or e im p or ta n t e qu al ly im p or ta n t m od er at el y m or e im p or ta n t m uc h m o re im p or ta n t v er y m u ch m or e im p or ta n t e x tr em el y m or e im p or ta n t in this context, the weight that has been assigned to the socio-economic aspects reflects the economic expectations attributed to the construction of the corridor, such as the fear of becoming peripheral in the european context or the decrease in the capacity of attracting new economic activities. these expectations are considered much more important than the environmental issues. another example of a question generated at the clusters level is as follows: among the “critical issues due to the delay in the realization of the italian portion of corridor 24”, which of these two aspects do you think is more important and how much more? environmental aspects transport aspects e x tr em el y m or e im p or ta n t v er y m u ch m or e im p or ta n t m uc h m o re im p or ta n t m od er at el y m or e im p or ta n t e qu al ly im p or ta n t m od er at el y m or e im p or ta n t m uc h m o re im p or ta n t v er y m u ch m or e im p or ta n t e x tr em el y m or e im p or ta n t the weight that has been assigned to transport aspects is much more important than the environmental aspects. this reflects the opinion that the implementation of the links in europe is fundamental. once the cluster comparisons have been conducted, it is necessary to study the problem in depth through the analysis of the elements. the questions that were generated considering the pairwise comparison at the nodes level were similar to the following: with reference to the environmental aspects, among the “critical issues due to the delay in the realization of the italian portion of corridor 24”, which of these two elements do you think is more important and how much more? increase in air pollution increase in acoustic emission e x tr em el y m or e im p or ta n t v er y m u ch m or e im p or ta n t m uc h m o re im p or ta n t m od er at el y m or e im p or ta n t e qu al ly im p or ta n t m od er at el y m or e im p or ta n t m uc h m o re im p or ta n t v er y m u ch m or e im p or ta n t e x tr em el y m or e im p or ta n t ijahp article: abastante, f., bottero, m., lami, i.m / using the anp for ranking the critical issues of a transport decision problem international journal of the analytic hierarchy process 55 vol. 4 issue 1 2012 issn 1936-6744 in this case the weight that has been given to the air pollution reveals the general opinion that the atmospheric quality is more important than the acoustic one, even if, according to the sector literature, the increase in acoustic emissions can be seen as the cause of the main social costs due to the presence of a transport infrastructure. another example of a question generated at the nodes level is as follows: with reference to transport aspects, among the “critical issues due to the delay in the realization of the italian portion of corridor 24”, which of these two elements do you think is more important and how much more? congestion of road network increase in costs of transport e x tr em el y m or e im p or ta n t v er y m u ch m or e im p or ta n t m u ch m o re im p or ta n t m o d er at el y m or e im p or ta n t e qu al ly im p or ta n t m o d er at el y m or e im p or ta n t m u ch m o re im p or ta n t v er y m u ch m or e im p or ta n t e x tr em el y m or e im p or ta n t according to the evaluation performed, the increase in cost of transport is considered much more important than congestion of the road network by the focus group. this is because the increase in cost is related to the congestion of road network. in fact, if there is congestion, there will be also an increase in transport time and therefore an increase in cost of transport due mainly to fuel expenses. in the development of the model, a great deal of attention was given to check the consistency index of the pairwise matrixes. in the full range of cases this index was less than 0,1 in order to ensure the acceptability of the model. 3.4 final results from the cluster level comparison it is possible to derive the cluster matrix, which represents the importance of the general aspects of the decision problem (table 3). in the case under examination, the socio-economic aspects were given the highest importance (0,649), followed by the transport aspects (0,280) and finally by the environmental aspects (0,071). table 3 cluster matrix goal environmental aspects socio-economic aspects transport aspects goal 0,000 0,000 0,000 0,000 environmental aspects 0,071 0,000 0,000 0,152 socio-economic aspects 0,649 1,000 1,000 0,606 transport aspects 0,280 0,000 0,000 0,242 the totality of the eigenvectors that are derived from the pairwise comparison matrixes of the elements of the model forms the unweighted supermatrix (table 4). the abbreviations used in this table refer to figure 3. ijahp article: abastante, f., bottero, m., lami, i.m / using the anp for ranking the critical issues of a transport decision problem international journal of the analytic hierarchy process 56 vol. 4 issue 1 2012 issn 1936-6744 table 4 unweighted supermatrix goal environmental aspects socio-economic aspects transport aspects g e1 e2 e3 e4 s1 s2 s3 s4 t1 t2 t3 t4 goal g 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 environmental aspects e1 0,083 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,100 0,000 0,000 0,000 e2 0,225 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,225 0,000 0,000 0,000 e3 0,619 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,675 0,000 0,000 0,000 e4 0,073 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 socioeconomic aspects s1 0,054 0,000 1,000 0,000 0,000 0,000 0,000 0,000 0,000 0,166 0,000 0,000 0,000 s2 0,168 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 s3 0,389 0,000 0,000 0,000 0,000 0,000 0,000 0,000 1,000 0,834 0,750 0,750 0,000 s4 0,389 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,250 0,250 1,000 transport aspects t1 0,304 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,080 0,000 t2 0,082 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,732 0,000 t3 0,192 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 t4 0,422 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,188 0,000 the cluster matrix is then applied to the unweighted supermatrix as a cluster weight and the result is the weighted supermatrix (table 5). table 5 weighted supermatrix goal environmental a. socio-economic a. transport a. g e1 e2 e3 e4 s1 s2 s3 s4 t1 t2 t3 t4 goal g 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 environ mental e1 0,005 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,020 0,000 0,000 0,000 e2 0,016 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,045 0,000 0,000 0,000 e3 0,044 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,134 0,000 0,000 0,000 e4 0,005 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 socio economic s1 0,034 0,000 1,000 0,000 0,000 0,000 0,000 0,000 0,000 0,133 0,000 0,000 0,000 s2 0,109 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 s3 0,252 0,000 0,000 0,000 0,000 0,000 0,000 0,000 1,000 0,666 0,750 0,535 0,000 s4 0,252 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,250 0,178 1,000 transport t1 0,084 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,023 0,000 t2 0,022 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,208 0,000 t3 0,053 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 t4 0,117 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,000 0,053 0,000 finally, the weighted supermatrix is raised to a limiting power in order to converge and to obtain a long-term stable set of weights that represents the final priority vector (figure 4). figure 4 final priority vector of the elements of the model ijahp article: abastante, f., bottero, m., lami, i.m / using the anp for ranking the critical issues of a transport decision problem international journal of the analytic hierarchy process 57 vol. 4 issue 1 2012 issn 1936-6744 4. discussion of the results and conclusions the results of the anp model as shown in the priority vector of figure 3 highlight some interesting findings that can be summarized as follows. the most important effect that the delay in the realization of the corridor could produce is the “lower level of attractiveness” element (0,443), which belongs to the “socio-economic aspects” cluster. this effect is approximately as important as all the other elements combined. the second effect in the priority list is the “reduction of trade” element (0,229) in the “socio-economic aspects” cluster. it is possible to say that the two aforementioned elements are independent but strongly interrelated in determining the potential threats on the territorial system. the third element in order of importance is the “less reliable service” element (0,070) which is part of the “transport aspects” cluster. it is important to note that the reliability of service has been considered by the experts in the focus group more important than the freight travel time and the costs of transport. it is also important to mention the fact that the elements belonging to the “environmental aspects” cluster have been given a very low importance. this can be explained in two different ways. to start with, it is possible to suppose that the freight volume will decrease because of the segregation effect due to the absence of the corridor axis. secondly, the environmental effects are not a crucial point in the decision problem under examination because they would also be very high in the case of the construction of the corridor, considering the presence of the high speed rail. the aforementioned considerations show that the results of the technical focus group reflect the concerns of the overall political class about the role that italy could play in the international context. in fact, even if the project under examination refers to a transport infrastructure, the most important effects that one could expect from the delay in the construction of the corridor have a socio-economic nature and are not related to transport aspects or spatial planning considerations. this is consistent with other technical transport studies within the “code24 project” that focus on the fact that north-west italy is late compared to the rest of europe in its expected conclusion of the work along the corridor 24 (2020). it is possible to suppose at the strategic level, three extreme and provocative scenarios at that time (carrara and arnone, 2011): 1. work not finished: italy gets ”isolated”, with loss of accessibility to the economic system of the pianura padana; the ligurian ports are just ports of the pianura padana; with loss of competitiveness of italy; 2. “alps” work finished, “ligurian” work not completed: italy gets “conquered”. this could represent an increased permeability of the alps; northern ports enlarge their hinterland in the pianura padana and ligurian ports are isolated from europe; 3. all work finished: italy becomes a”gateway of europe”, with an increased connection between italy and europe and a greater competitiveness of the ligurian ports (well linked to the inland) in europe. in terms of scientific contribution, the developed model has offered a creative way of combining many detailed criteria in an evaluation study and synthesizing them to obtain a ijahp article: abastante, f., bottero, m., lami, i.m / using the anp for ranking the critical issues of a transport decision problem international journal of the analytic hierarchy process 58 vol. 4 issue 1 2012 issn 1936-6744 priority list of the most important effects that a delay in the construction of the infrastructure could produce on the territorial system. moreover, the anp-based application succeeded in bringing together a heterogeneous decision-making group made of specialists coming from different fields of expertise related to transport activities and forcing them to discuss and to evaluate the criteria of the model. the anp methods proved to be suitable in dealing with decision problems related to transport planning for several reasons. to start with, the methodology allowed the most important elements of the decision problem to be highlighted through a transparent and traceable decision-making process thus facilitating deliberation. second, the methodology supported communication with the dms and granted a mutual understanding. more generally, it can be said that the anp can be usefully applied within the context of the stakeholders-driven or institutional approach to transport project evaluation. policy makers can use the findings resulting from the application as input for designing formal decision-making processes geared towards including stakeholder’s objectives in transport project evaluation (de brucker et al., 2004). apart from the aforementioned advantages that are a result of the application of the anp in complex decision environments, one of the most significant strengths of the adopted methodology is represented by the fact that the dm gains more awareness of the elements at stake while structuring the model and thus learns about the problems while solving them (bottero et al., 2010). there are still a number of opportunities for expanding the study and for validating the obtained results. first, it would be of scientific interest to implement the anp model on the whole corridor, including organizing different focus groups with all the partners of the code24 project. second, even though the use of fuzzy logic in ahp and anp methodologies is quite controversial, future research could explore this approach in the development of the model. in particular, the fuzzy logic could be useful in a group decision making context for the aggregation of the different preferences. it could therefore be very interesting to investigate the use of fuzzy numbers in the pairwise comparison process (liu and lai, 2009) in order to tolerate vagueness and ambiguity in the evaluation. finally, the application of the anp model could be enriched by specific visualization tools that are of great importance for presenting and communicating the results of the analysis to decision makers and the interest group (lami, 2011). references banister, d. (1994). transport planning. london: e & fn spon. bertolini, l. (2001). territorializzazione delle infrastrutture nei paesi bassi. in: g. dematteis & f. governa (a cura di), contesti locali e grandi infrastrutture. milano: franco angeli. bond, a.j. & brooks, d.j. (1997). a strategic framework to determine the best practicable environmental option for proposed transport schemes. journal of environmental management, 51, 305-321. ijahp article: abastante, f., bottero, m., lami, i.m / using the anp for ranking the critical issues of a transport decision problem international journal of the analytic hierarchy process 59 vol. 4 issue 1 2012 issn 1936-6744 bottero, m. & lami, i.m. (2010). analytic network process and sustainable mobility: an application for the assessment of different scenarios. journal of urbanism, 3, 275 – 293. bottero, m., ferretti, v. & pomarico, s. (2010). using the analytic network process (anp) for enhancing public participation in urban planning processes. in: proceedings of the 72nd meeting of the european working group "multiple criteria decision aiding", 7-9 october 2010, paris. bottero, m., lami, i.m. & lombardi, p. (2008). analytic network process. la valutazione di scenari di trasformazione urbana e territoriale. firenze: alinea. carrara, m. & arnone, m. (2011). la logistica del nord-ovest: il sistema intermodale. in: xxxii conferenza italiana di scienze regionali, 15-17 september 2011, torino. chapman d., pratt, d., larkham, p. & dickins, i. (2003). concepts and definitions of corridors: evidence from england’s midlands. journal of transport geography, 11, 179– 191. de brucker, k., verbeke, a. & macharis c. (2004). the applicability of multicriteriaanalysis to the evaluation of intelligent transport systems (its). research in transportation economics, 8, 151-179. figueira. j., greco, s. & ehrgott, m. (eds.). (2005). multiple criteria decision analysis. state of the art survey. new york: springer. fubini a. (ed) (2008). corridor policies and territorial development. francoangeli: milano. jefferson, c. (1996). improving access by public transport. landscape and urban planning, 35, 173-176. lami, i.m. & staffelbach, l. (2008). evaluating the urban redevelopment of railway sites: zurich central station and surrounding areas. proceedings of the third kuhmonectar conference transport and urban economics, 3-4 july 2008, amsterdam. lami, i.m., masala e. & pensa s. (2011). analytic network process (anp) and visualization of spatial data: the use of dynamic maps in territorial transformation processes. international journal of the analytic hierarchy process, 4(1), 92-106. liu, k. & lai, j.k. (2009). decision-support for environmental impact assessment: a hybrid approach using fuzzy logic and fuzzy analytic network process. expert systems with applications, 36, 5119–5136. nekhay, o., arriaza, m. & boerboom, l. (2009). evaluation of soil erosion risk using analytic network process and gis: a case study from spanish mountain olive plantations. journal of environmental management, 90, 3091-3104. ijahp article: abastante, f., bottero, m., lami, i.m / using the anp for ranking the critical issues of a transport decision problem international journal of the analytic hierarchy process 60 vol. 4 issue 1 2012 issn 1936-6744 ortuzar, j. & willumsen, l. (2001). modelling transport. chichester: wiley. piantanakulchai, m. (2005). analytic network process model for highway corridor planning. proceedings of isahp 2005. honolulu, hawaii. romein a., trip j.j. & vries j. de (2000). theoretical framework. report within the framework of action 3 of corridesign. otb research institute, delft university of technology, delft. saaty, r.w. (2003). decision-making in complex environments: the analytic hierarchy process (ahp) for decision-making and the analytic network process (anp) for decision-making with dependence and feedback. pittsburgh: rws publications. saaty, t.l. (1980). the analytic hierarchy process. new york: mcgraw hill. saaty, t.l. (2005). theory and applications of the analytic network process. pittsburgh: rws publications. saaty, t.l. & vargas, l.g. (2006). decision making with the analytic network process. new york: springer science. saaty, t.l. & ozdemir, s.o. (2008). the encyclicon: a dictionary of applications of decision making with dependence and feedback based on the analytic network process. pittsburgh: rws publications. stefan a, (2008). supporting multi-stakeholder environmental decision. journal of environmental management, 88, 607-614. tuzkaya, g. & onut, s. (2008). a fuzzy analytic network process based approach to transportation-mode selection between turkey and germany: a case study. information sciences, 178(15), 3133-3146. waddel, p. (2011). integrated land use and transportation planning and modeling: addressing challenges in research and practice. transport reviews, 31(2), 209-229. whitehead, t., simmonds, d. & preston, j. (2006). the effect of urban quality improvements on economic activity, journal of environmental management, 80, 1-12. wijnmalen, d.j.d. (2007). analysis of benefits, opportunities, costs, and risks (bocr) with the ahp–anp: a critical validation. mathematical and computer modelling, 46(7/8), 892-905. www.code-24.eu ijahp article: krstic, mimovic/the impact of the new global reality on the european integration process: the case of the republic of serbia international journal of the analytic hierarchy process 413 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.883 the impact of the new global reality on the european integration process: the case of the republic of serbia 1 ana krstic anakrstic@kg.ac.rs predrag mimovic mimovicp@kg.ac.rs abstract the acquisition of full membership for the republic of serbia in the european union depends on a large number of factors. the combined and synergistic effect of these factors has made the process of serbia's accession to the european union extremely uncertain in terms of date and final outcome. the new reality, including the covid-19 pandemic, u.s. isolationist policies during president trump's tenure, the strengthening of china and russia, and britain's exit from the eu, has made this process even more uncertain. in this context, it is crucial to identify and prioritize key factors that affect or could affect the process of serbia's accession to the european union, in order to evaluate the possible outcomes of this process. to solve problems of this level of complexity and uncertainty, it is necessary to apply an interdisciplinary approach of the appropriate level of complexity. therefore, this paper is an extension of previous research (krstic et al, 2018; mimovic et al, 2019) based on the defined research goal, but incorporates a complex, network model called theanalytical network process, which included new circumstances that determine what is colloquially called the new reality. all calculations in the paper were performed using the software package superdecisions as computer support for the analytical network process. keywords: european integration; global change; forecast; new global reality; date of eu accession; analytical network process; republic of serbia; european union 1. introduction the world is changing at a rapid pace like never before. changes in the environment affect both individual and group preferences, which directly affect the perception and degree of understanding of these changes. entire nations are facing great existential temptations caused by huge changes in the environment. we need, as never before, a global consensus on the key issues of survival, a universal value system, climate change and long-term goals of humanity. we need appropriate forecasting models that will be able to incorporate the complexity of sudden changes and their consequences. 1 the paper is the result of research on the project contemporary challenges of economic development of the republic of serbia, funded by the faculty of economics, university of kragujevac. mailto:anakrstic@kg.ac.rs mailto:mimovicp@kg.ac.rs ijahp article: krstic, mimovic/the impact of the new global reality on the european integration process: the case of the republic of serbia international journal of the analytic hierarchy process 414 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.883 in the past six years, serbia has opened 18 negotiation chapters in the eu accession negotiations and temporarily closed two (chapters 25 and 26). in 2020, serbia did not open a single new chapter, partly because of domestic political reasons, which was serbia’s responsibility and partly due to foreign policy circumstances, which were the eu’s responsibility. in any case, in order for serbia to open a new chapter, all member states must give their consent. however, gaining the status of a full member of the european union does not only depend on how quickly the reforms and harmonization of regulations in the republic of serbia adapt to the legal heritage of the european union, nor on how satisfied other countries are with the progress of the republic of serbia as a candidate. the world is changing rapidly; new geostrategic centers of power are emerging, old ones are disappearing, and the extent to which both the eu and potential members adapt to these changed circumstances, will determine not only the european perspective of serbia and the western balkans, but also the future of the eu itself. the benefits of eu membership are well-known. it provides significant benefits to serbia and other potential member states by encouraging regional cooperation, strengthening democracy, ensuring more rights and freedoms for citizens, contributing to peace and stability, strengthening the position of states in international relations and improving the well-being and living standards of their citizens. for the eu, enlargement offers an opportunity to create a stronger, more secure and larger union, despite the internal and external crises it faces. external factors, such as the migration crisis, the rise of extremism, and the penetration of china and russia's geostrategic use of soft and hard power have a major impact on the western balkans region and how the eu member states view it. the influx of immigrants to european borders, especially the western balkans region, has led to a nonhomogeneous response from some member states and a crisis of confidence. china's rise has been in the making for a long time; china's foreign direct investment rose from 1 billion euros in 2008 to 35 billion euros in 2016, and in 2017, the u.s. national security strategy described china as a "revisionist force." the origins of the pandemic in china and its subsequent attempts to minimize the damage caused at the very beginning of the pandemic have only intensified this existing trend. eu-china relations were delicate even before the pandemic, but european commission president ursula von der leyen announced a new diplomatic era, which she called “challenging”, that may have been inevitable, given the growing hostility in us-china relations. increasingly tense transatlantic relations between the eu and the u.s. have also been further complicated by the pandemic. trump's arrival in office marked the beginning of a turbulent period that culminated in an aggressive u.s. customs policy and the introduction of several tariffs on european exports in 2018. meanwhile, the eurussia relationship has been delicate for some time, with russia considered a partner until 2014. the relationship, although "difficult", became more difficult with the later annexation of parts of ukraine, the eu's military support to the syrian government and accusations of cyber attacks and the increased disinformation campaign in europe. increasingly frequent accusations that russia wants to increase europe's dependence on russian gas with its energy policy have further polarized the european union. it is estimated that the rapprochement between russia and china is likely to have negative implications for the eu. on the other hand, the covid-19 pandemic caused an exogenous shock that has tested the resilience of european integration. analysis of the early stages of the global pandemic showed the following: ijahp article: krstic, mimovic/the impact of the new global reality on the european integration process: the case of the republic of serbia international journal of the analytic hierarchy process 415 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.883 • a new chapter of strategic and systemic rivalry with china has opened up as the eu has struggled with internal solidarity and become prey to a global war of narratives led by china and the u.s.; • the strategic gap in transatlantic relations has widened because the united states has not shown global leadership, further eroding european confidence through the "america first" approach; • strengthening of strategic challenges coming from africa, where the potential health crisis will coincide with a significant economic downturn (koenig & stahl, 2020). the european union is at a crossroads. uncoordinated national responses threaten major european institutions, but the crisis is also an opportunity to advance integration and strengthen eu goals (bongardt & torres, 2020; bonomi et al, 2020). in europe, the pandemic crisis has revealed numerous failures in crisis management, such as a lack of preparedness, solidarity, and slow responses (roloff, 2020). the corona virus crisis has illustrated the fragility of european common goods such as the single market, the schengen agreement, and even economic and monetary union (emu) if national measures are incompatible or insufficiently coordinated. these are all challenges that the eu is currently facing, and the future of the eu will largely depend on the way it will respond to them, including how it handles the enlargement policy, which is primarily focused on the so-called western balkans countries. the eu remains by far the largest trading partner of the western balkans, accounting for 69.4% of total merchandise trade in 2019 (82.9% of total exports and 61.8% of total imports). since 2009, trade has grown by 129.6%. eu companies are the largest investors in the region, accounting for 73% of foreign direct investment (european commission, 2020). they are therefore the main external driver of growth and jobs in the region. on the other hand, as many times before in the past, the balkans are a place where the interests of great countries clash, in this case, the eu, russia, china and the united states. what is the place of serbia, as a part of the western balkans, without whose integration europe cannot be complete, as the president of the nato parliamentary assembly, paolo ali, once emphasized; and, if it still exists, if it is not only declarative, what is the european perspective on serbia? the paper is structured as follows: the second part of the paper includes a review of the literature in which forecasting problems are solved using the analytic network process. the third part of the paper describes the prediction problem, with reference to the research from 2018 and published in 2019, and constructs an appropriate model. the results of the model are presented in the fourth part, with a comparative analysis in relation to the forecasting model from 2018 (mimovic et al, 2019) and with a brief discussion, and the last part includes the concluding observations, work limitations and guidelines for future research. 2. methodology and literature review the anp method (saaty, 2001) has a wide application in solving decision-making problems in conditions of risk, uncertainty, dynamics and complexity. one of the important areas of application of the anp method, in addition to evaluating performance and solving the problem of choice in conditions of uncertainty, is prediction, i.e., ijahp article: krstic, mimovic/the impact of the new global reality on the european integration process: the case of the republic of serbia international journal of the analytic hierarchy process 416 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.883 evaluation of alternative future outcomes based on the distribution of their probabilities. mimovic (2012) showed that the anp can be used very successfully to structure the influence of various factors on the final outcome of the forecasting process with the example of predicting the sale of a new fiat 500l car model. anp sales forecasting was performed by voulgaridou et al. (2009) who predicted the sales volume of a new product (new edition of the book), as well as shih et al. (2012) who investigated the sales volume of printers in taiwan. ozorhon et al. (2007) applied the anp to predict the impact of international construction joint ventures. using the anp model, the authors examined the key impact factors of these joint ventures and concluded that the most important determinants of their success are partner relationships, structural factors, and partner cultural coherence. the anp has proven to be a good model for predicting the probability of failure of construction companies in turkey, including internal and external factors that determine the current business situation of the company and the links between them (dikmen et al, 2010), and as a model for predicting future sustainability specifications (eraki & shoura, 2019), predicting dental diseases (radyshevskaya & andreichicova, 2010), etc. in the field of economic forecasting, the anp method has been used to predict the probability of a financial crisis (niemira & saaty, 2004; saaty & vargas, 2006, 2013), to predict economic recovery (blair et al., 2010; azis, 2010), business forecasting (mu, 2004), etc. interesting research by adamus (2010) and saaty & vargas (2013) was conducted in connection with the forecast of poland's entry into the euro area, which resulted in the optimal alternative for poland being late entry (after 2011), as well as predicting the date of serbia's entry into the european union (mimovic et al., 2019). political problems are also common in the application of the anp method. saaty and vargas (2006, 2013) applied the anp to predict possible ways to overcome the china-taiwan conflict and the u.s. response to north korean nuclear threats. in relation to the ahp, the anp method allows each decision to be presented in the form of a network rather than as a strict hierarchy. hierarchical decisions can be subjective and predetermined due to the imposed structure, while matrix decisions, whose structure includes dependencies and feedback, represent a more realistic world, which makes the anp method a more efficient tool than the ahp for practical decision making (saaty, 2001b). in addition, the anp is successfully applied in many areas of forecasting, as it allows quick inclusion of feedback and easy comparison with actual results. however, it is also necessary to consider the disadvantages of this method, such as high complexity, manifested in a potentially large number of comparisons in general; cluster comparisons can often be vague and confusing and the decision-making process is longer than with the ahp method. 3. problem description and structuring of the anp model for a better understanding of the nature of the research subject, as well as the goal of the research, keeping in mind the dramatically changed circumstances in the environment, we first provide a description of the problem from the perspective of 2019, presented in the form of an anp model (figure 1, mimovic et al., 2019). ijahp article: krstic, mimovic/the impact of the new global reality on the european integration process: the case of the republic of serbia international journal of the analytic hierarchy process 417 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.883 figure 1 anp model for evaluation of alternative outcomes of serbia's eu accession process (2019) table 1 probabilities of achieving alternative outcomes of serbia's eu accession process (2019) alternatives normalized benchmarking rank before 2025 0.425020 1.000000 1 between 2025 and 2030. 0.191682 0.450995 3 between 2030 and 2035. 0.200024 0.470624 2 not entering the european union 0.183274 0.431213 4 source: author's assessments table 1 shows that in 2019 the highest probability of realization, according to the authors, was the entry of serbia into the eu by 2025 (42.5%), followed by entry of serbia into the eu between 2030 and 2035 (20%), between 2025 and 2030 (19.17%), and the probability that serbia will not join the eu was 18.32%. the obtained results indicated a clear commitment to join the eu at that moment, with the assessment that if the accession does not take place by 2025, it will certainly not be in the time immediately after that, but closer to 2030. it should also be emphasized that the authors jointly evaluated and made comparisons of model elements by pairs. the alternative approach was that each of the authors would independently make estimates and then calculate the geometric mean, which is one of the common approaches for calculating final priorities in situations where there are multiple decision makers (saaty & peniwati, 2008; mu & pereyra-rojas, 2018). ijahp article: krstic, mimovic/the impact of the new global reality on the european integration process: the case of the republic of serbia international journal of the analytic hierarchy process 418 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.883 in the meantime, primarily on a global level, there have been changes of a tectonic nature, with unforeseeable consequences for the world as a whole, and thus for the internal conditions in serbia and the countries of the region. we will list the most significant ones, in the author’s opinion, that have an evident influence on the processes of european integration, and which because of their internal and external interdependencies, will be included in the model. 1. brexit. the exit of great britain, the former "empire in which the sun never sets", a permanent member of the un security council and the nuclear power, from the european union, monopolized the eu's attention during a time when it was least desirable due to the covid-19 pandemic, and intensified the eu identity crisis. this diminished the eu’s influence, and thus its attractiveness from the outside, and emphasized the internal and otherwise present aversion to its further expansion and the need to redefine the conditions of accession. in this sense, the scenarios first went from several concentric, homogeneous circles of countries, classified according to the level of economic and political development with the founding countries of the eu in the center and candidate countries in external circles, also classified into multiple circles. due to resistance within the union, the members who joined later and felt discriminated against abandoned the french president macron’s idea to open chapters in clusters. however, after several years of wandering and seeking answers to how to proceed with the eu, the countries came to a consensus that the western balkans must be included in the european union, primarily for strategic, geo-political reasons, in order to prevent the growing influence of china and russia in that area. the process of joining, by opening and closing the prescribed chapters in order to have systemic harmonization, is a very slow process which has led to the strengthening of populism and the formation of soft autocracies, the so-called stabilitocracies, in the candidate countries. the length of the process also created time and space for the influence of other great players in these countries. this created the need for some kind of incentive to speed up the integration process; therefore, at the beginning of 2021, the french president presented the idea of opening chapters by clusters. however, britain’s imperial ambitions, its own long-term global goals (trade, defense, intelligence, etc.) as well as the desire to be present in the balkans have never disappeared and cannot be ignored in this analysis. 2. isolationism in the united states. the european parliament in its resolution of 11-25-2020, stressed the need to consolidate and strengthen eu-us cooperation in an unstable international environment, based on mutual respect and a common agenda in defense of multilateralism, international law and common democratic values.the arrival of donald trump into power in the united states marked america’s return to traditional values and its turning inward in accordance with the mantra "america in the first place." this scenario has already been seen in history at the beginning of the 20 th century, which is otherwise characteristic of a republican administration. trump's unorthodox behavior, including the split from the trans-pacific trade agreement, insistence on a larger allocation of nato budget funds for joint defense, withdrawal from the paris climate agreement, unilateral withdrawal from the nuclear agreement with iran despite opposition from other guarantors, and trade war with china, all led to weakening of international institutions, reduced the role of the u.s. in europe and the middle east, and changed the perspective through which the american administration has viewed regional problems such as syria, libya, the balkans. the results of the last u.s. elections and the ijahp article: krstic, mimovic/the impact of the new global reality on the european integration process: the case of the republic of serbia international journal of the analytic hierarchy process 419 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.883 return of democrats to power, with the election of president joseph biden, suggest a change of course and a return to liberal-globalist values and foreign policy guidelines, which cannot be without influence on events everywhere in the world, even in the western balkans, in light of the perspective of its european integration. on the other hand, whether the united states will understand that the world is no longer the same as it was, unipolar, but multipolar, is a key question that will determine not only their destiny, but the destiny of the whole world. 3. strengthening of china. the european parliament's resolution of 11-25-2020 noted the strengthening of the assertive public diplomacy of the people's republic of china around the world after the outbreak of covid-19 in an effort to fill the political vacuum created by u.s. isolationist policies and to establish itself as a dominant global player with alternative governance. the united states and the people's republic of china are the two largest economies in the world, both in nominal terms (gdp) and purchasing power parity (ppp). the united states is nominally at the top, while china has been at the top in terms of purchasing power parity since 2014 when it overtook the united states. combined, the countries shared 40.75% and 34.27% of the total world gdp in nominal terms in 2019. the gdp of both countries is huge compared to the third-ranked countries of japan (nominally) and india (ppp). the difference in terms of per capita income between china and the united states is very large since china’s population is four times the size of the us. the per capita income of the united states is 6.38, or 3.32 times higher than the chinese nominal, or ppp. the united states is the eighth richest country in the world, while china is in 72 nd place. according to the ppp, the united states is in 12 th place, and china is in 75 th place. however, in the 21 st century, the people's republic of china is emerging as a new global player by expanding its power primarily through the economy with huge, politically unconditional direct investments and favorable and unlimited loans, and strengthening its military and political power. the new silk road, or the silk road, as the most important chinese project in the country's history is called, will enable the creation of two trade routes that will connect china with the rest of the world. the new route, which is land and coastal, will pass through 60 countries from europe, asia and africa. the complete infrastructure which includes the construction of ports, roads and power plants, will cost china around one thousand billion dollars. the new silk road is often compared to the marshall plan, but is also seen as beijing's attempt to win over financially dependent vassal states. criticism of beijing's largest project has focused on the political, economic and strategic effects, while much less attention has been paid to business prospects. the republic of serbia is certainly interested in china's efforts to position itself more firmly in europe, more precisely to penetrate northwestern europe. in this context, we should look at large chinese investments in infrastructure projects in the western balkans as well as earlier in greece (port of piraeus and a 35-year concession) as the soft belly of europe, which will obviously have political consequences and cannot be left out of the analysis of serbia's european integration. china's growing global economic influence and the economic and trade policies it maintains have significant implications for the united states and are therefore of great interest to congress. although china is a large and growing market for u.s. firms, its incomplete transition to a free market economy has resulted in economic policies considered detrimental to u.s. economic interests, such as its industrial policy and the theft of u.s. intellectual property. all this has led to a complete focus on the distant pacific and the identification of china's main strategic rival in the future on a ijahp article: krstic, mimovic/the impact of the new global reality on the european integration process: the case of the republic of serbia international journal of the analytic hierarchy process 420 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.883 global level by the american administration. however, the new american administration has made moves that suggest a return to the kissinger doctrine when it comes to china, a strategic competitor with whom it will have extreme competition, but not conflict. china is the biggest geopolitical test and the only country that has the power to seriously question america's ability to shape a global system of "rules, values and attitudes," according to antony blinken, the us secretary of state, adding "our relationship with "china will be competitive when necessary, partner when possible and rival when necessary." 4. covid-19 pandemic. the european parliament, in its report of 11-25-2020, emphasized that the global outbreak of the covid-19 pandemic in a changing international environment is a risk multiplier and catalyst for change in the global order. it also emphasized the fundamental importance of strengthening the eu's internal resilience, developing new partnerships and strengthening its multilateral vision globally. strengthening geopolitical competition in the new geopolitical environment has farreaching consequences for eu foreign policy, which confirms the need for a more effective eu security policy, strengthening the euro-atlantic partnership to combat future pandemics and other major international challenges, while respecting universal values, such as multilateralism and the rule of law. the end of 2019 marked the beginning of an epidemic of a new corona virus (sarscov-2), first observed in wuhan, china (wang et al., 2020). covid-19, a disease caused by sars-cov 2, represents a spectrum of symptoms ranging from mild to severe, often with asymptomatic presentation (huang et al., 2020). the world health organization (who) declared covid-19 a pandemic on march 11, 2020, and to date, more than 106 million have been infected and two million and three hundred thousand people have died worldwide. this greatest plague that has befallen humanity in the last 100 years, not counting the second world war, has changed the world in several ways, affecting primarily human health, way of life and the economy, both on a micro level and globally. the economic consequences of the pandemic are yet to be assessed, but it is certain that in addition to these material consequences, the psychological and social ones will leave a mark long after the pandemic’s end. the political consequences are reflected in the recomposition of global relations. some of the world's powers, such as china and russia, have used the incompetence of western governments and their ineffectiveness in fighting the corona virus to demonstrate their own superiority. for example, in the firstphase of pandemic relief process they delivered protective masks, gloves and other equipment, developed a corona virus vaccine and distributed it unconditionally. in this way, the pandemic altered the daily political routine, including the traditional promotion and insistence of western countries of liberal-democratic values and their use for criticism and control of global competitors, and instead they became a tool for political struggle and repositioning of global power. particularly at the very beginning of the pandemic, the european union reacted inconsistently and chaotically with mutual distrust among member states, ad hoc closing of borders, and continued long-term lock downs as a way to fight the virus. vaccine procurement demonstrated all of the weaknesses of the complicated mechanism for decision-making, which led to growing dissatisfaction among the population and a wave of protests. the eu has been shown to be slow to act in times of crisis, and its size and complexity often lead to delays in decision-making. hence, the ijahp article: krstic, mimovic/the impact of the new global reality on the european integration process: the case of the republic of serbia international journal of the analytic hierarchy process 421 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.883 covid-19 pandemic is a factor that should certainly be taken into account in the context of the future of the eu enlargement policy. we should also not forget russia as a global player and with whose policy the eu policy is inextricably linked. in its resolution, the european parliament notes that russia's systematic attempts to re-impose itself as a global player have geopolitical and geoeconomic dimensions. russia threatens to undermine the unity of the european union, and to position itself in the western balkans, occupying the vacant space created by the indecision of the eu and its fatigue from further enlargement. strengthened by the viral diplomacy of the covid-19 pandemic, the struggle of narratives and aggressive propaganda, russia is taking assertive steps to promote its geopolitical agenda internationally. with this in mind, in addition to the internal and external interdependencies identified in the 2018 model, the new model in 2021 includes the following links and influences of these factors: a) the alternatives are compared in relation to the above factors because it is evident that each of them has an impact on the status and dynamics not only of serbia's eu accession process, but also on eu enlargement policy in general, as part of its foreign policy; b) serbia's relations with the united states, china, russia and kosovo are compared with each other, given that serbia's foreign policy in recent years in relation to the eu and kosovo is based on a policy of balancing with the world's largest powers. hence, the following dependencies have been identified: serbia's relations with the united states, russia and kosovo are compared in relation to the factor of serbia's relations with china; serbia's relations with china, the united states and kosovo are compared in relation to the factor of serbia's relations with russia; serbia's relations with russia, china and kosovo are compared in relation to the factor of serbia's relations with the united states; serbia's relations with russia, china and the united states are compared in relation to the factor of serbia's relations with kosovo. these are compared by determining which of the two observed factors is considered more important, or has a greater impact, in relation to the factor of serbia's relations with kosovo; c) the covid-19 pandemic and the uk exit are compared in relation to the eu security challenges factor, in terms of answering which of these two factors is more important from the point of view of the eu security challenges factor. the inclusion of these factors, as well as their mutual influences with accompanying implications, led to the need to form a new model, shown in figure 2. ijahp article: krstic, mimovic/the impact of the new global reality on the european integration process: the case of the republic of serbia international journal of the analytic hierarchy process 422 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.883 figure 2 anp model for evaluating alternative outcomes of serbia's eu accession process (2021) 4. model results figure 2 presents the anp model of evaluation of alternative outcomes of the process of serbia's accession to the eu, where three outcomes are defined in terms of entry timing, and the fourth outcome excludes this possibility. the structure of the model consists of clusters, elements and connections within and between the outcomes. table 3 shows the relative importance of the criteria in the model. it is estimated that the criterion of geopolitical factors (weight coefficient is 0.64299), followed by internal factors (0.15291), cost factors (0.12939) and risk factors (0.07471) has the greatest relative importance in relation to the main goal of the model. this result could be interpreted as the final outcome of the process of serbia's accession to the eu is most influenced by the factors on which the republic of serbia has the least influence, and as such have the greatest importance. this significance is greater than three years ago, which is understandable when considering the newly created circumstances on the global level that have been discussed. also, the shift of focus and relative importance of the remaining three factors is noticeable, when compared with the values from three years ago (figure 3). tables 4 7 show the relative importance of sub-criteria within the criteria, i.e., the cluster to which they belong. thus, among the elements of the cluster, internal factors are estimated to have the greatest relative importance, in the context of european integration, with the sub-criterion of relations with kosovo (0.82379), which is significantly higher than the value three years ago (0.31503). china and russia's interest in the western balkans, ijahp article: krstic, mimovic/the impact of the new global reality on the european integration process: the case of the republic of serbia international journal of the analytic hierarchy process 423 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.883 heightened by indecision, disorientation and what could be called fatigue from eu enlargement, and the somewhat more favorable geostrategic circumstances that occurred with the coming to power of the trump administration and could be characterized as a simple shift of focus of american foreign policy, led to the re-actualization of the kosovo issue, which was reflected in the internal political situation in serbia, as an instrument of political struggle. the cluster of geo-political factors, unlike three years ago, included factors that reflect the new geostrategic circumstances through the prism of whose relations it is necessary to observe the process of european integration of serbia. the greatest relative importance in this cluster is the sub-criterion of serbia's relations with the united states (0.24668), followed by serbia's relations with russia (0.23521) and serbia's relations with china (0.22535), which is understandable, given the political balance in this triangle and serbia's support from russia and china in the un security council on the issue of kosovo, which is most closely related to european integration. it is estimated that other factors are not crucial from the point of view of the model goal. among the sub-criteria of the cost criterion, opportunity costs of non-accession to the eu has the highest relative importance (0.60863), which is slightly less than three years ago (0.69). among the risk factors, the sub-criterion alienation and exploitation of natural resources (0.28002) has the highest relative importance, which is a significant shift in focus compared to 2018, when the highest priority was the risk of exacerbation of social conflicts (0.23). this may indicate an awakened and more developed environmental awareness of the citizens of serbia. the obtained priorities of alternatives, shown in table 8 are interpreted in terms of the probability of realization of the selected alternatives. we see that the highest estimated probability of realization in 2021 has an alternative outcome and 2035 (21.64%), compared to 19.16% and 23.21% for serbia's accession to the eu after 2030, compared to 20% in 2018 (figure 6). unlike in 2018, when the obtained results showed a clear commitment to join the eu with the assessment that if accession does not take place by 2025, it will certainly not be in the time immediately thereafter, but closer to 2030, the results obtained by the analysis in 2021 indicate that all outcomes have approximately the same probability of realization. table 2 relative importance of the anp model cluster for the evaluation of alternative outcomes of serbia’s eu accession process clusters relative importance of the clusters cluster of internal factors 0.0647 cluster of geo-political factors 0.5666 costs of eu accession 0.1843 risks of eu accession 0.1843 inconsistency 0.06504 ijahp article: krstic, mimovic/the impact of the new global reality on the european integration process: the case of the republic of serbia international journal of the analytic hierarchy process 424 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.883 table 3 relative importance of primary factors in relation to the main goal of the model criteria relative importance of the criteria geo-political factors 0.64299 cost factors 0.12939 risk factors 0.07471 internal factors 0.15291 inconsistency 0.04435 figure 3 comparative overview of the relative importance of the criteria in 2018 and 2021 table 4 relative importance of the sub-criteria of the criterion internal factors name normalized by cluster relations with kosovo 0.82379 internal political conflicts 0.01748 coherence of the legal-political system 0.08592 coherence of the economic system 0.03735 coherence of the social system 0.03546 source: author's assessments 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 external factors costs factors risk factors internal factors relative importance in 2018 relative importance in 2021 ijahp article: krstic, mimovic/the impact of the new global reality on the european integration process: the case of the republic of serbia international journal of the analytic hierarchy process 425 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.883 figure 4 comparative overview of the relative importance of cluster elements internal factors in 2018 and 2021 table 5 relative importance of the sub-criteria of the criterion geo-political factors name normalized by cluster eu security challenges 0.09685 britain's exit from the eu 0.08754 strengthening the eu conservative right 0.04785 covid-19 pandemic 0.03937 eu internal political factors 0.02115 serbia's relations with china 0.22535 serbia's relations with russia 0.23521 serbia's relations with the united states 0.24668 source: author's assessments figure 5 comparative overview of the relative importance of cluster elements geopolitical factors in 2018 and 2021 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 estimate in 2018 estimate in 2021 0 0.05 0.1 0.15 0.2 0.25 0.3 estimate in 2021 estimate in 2018 ijahp article: krstic, mimovic/the impact of the new global reality on the european integration process: the case of the republic of serbia international journal of the analytic hierarchy process 426 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.883 table 6 relative importance of the sub-criteria of the criterion cost factors name normalized by cluster negative current account due to increased imports 0.20850 opportunity costs of non-accession to the eu 0.60863 reduction of budget revenues due to loss of customs revenues 0.10679 costs of harmonization with eu standards 0.07609 source: author's assessments figure 6 comparative overview of the relative importance of cluster elements cost factors in 2018 and 2021 table 7 relative importance of the sub-criteria of the criterion risk factors name estimated in 2021 alienation and exploitation of natural resources 0.28002 risk of unequal exchange and exploitation 0.04696 risk of disproportions in economic development 0.03231 risk of suffocation of certain industries 0.08902 risk of increased competition 0.17507 risk of rising unemployment 0.19501 risk of exacerbation of social problems 0.18160 source: author's assessments 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 estimate in 2018 estimate in 2021 ijahp article: krstic, mimovic/the impact of the new global reality on the european integration process: the case of the republic of serbia international journal of the analytic hierarchy process 427 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.883 figure 7 comparative overview of the relative importance of cluster elements risk factors in 2018 and 2021 table 8 probabilities of achieving alternative outcomes of the process of serbia's accession to the eu (2021) alternative normalized benchmarking rank accession to the eu by 2025 0.26687 0.9379 2 accession to the eu between 2025 and 2030 0.21643 0.7606 4 accession to the eu after 2030 0.23217 0.8159 3 no accession to the eu 0.28453 1.0000 1 figure 8 comparative presentations of the probabilities of achieving alternative outcomes of the process of serbia's accession to the eu in 2018 and 2021 0 0.05 0.1 0.15 0.2 0.25 0.3 estimate in 2018 estimate in 2021 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 accession to the eu by 2025 accession to the eu between 2025 and 2030 accession to the eu after 2030 no accession to the eu estimate in 2018 estimate in 2021 ijahp article: krstic, mimovic/the impact of the new global reality on the european integration process: the case of the republic of serbia international journal of the analytic hierarchy process 428 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.883 the prediction goal and the obtained results indicate a high level of uncertainty present in the prediction process, which only confirms the need for additional analysis of the sensitivity of the results to changes in the values of key model parameters in order to obtain a more complete and comprehensive target value estimate. a sensitivity analysis of the results by changing the level of significance of higher level elements, i.e., parent elements, can more or less significantly affect the order of importance and evaluation of the observed options, showing the performance of alternatives in terms of each criterion, as well as how sensitive these alternatives are to changes in the importance of criteria. thus, from figure 9 it can be seen that an increase in the relative importance of the kosovo relations criterion from 0.0001 to 0.9999 affects the probabilities of alternative outcomes to some extent. the probability of no eu accession increases from 12.77% to 46.80 % and the probability of expected entry by 2025 slightly decreases from 28.91% to 24.08%. slightly more sensitive to the changes in the relative importance of the subcriteria relations with kosovo are shown by the remaining alternative outcomes, whose probability of realization decreases to a greater extent. the probability of joining the eu decreases between 2025 and 2030 from 26.12% to 16.39%, but from 2030-2035 it decreases from 32.18% to 12.71%. the obtained results differ substantially from those derived from the model predicting serbia's accession to the eu (mimovic, et al., 2018), where the sub-criterion relations with kosovo has essentially no impact on serbia's accession to the eu in terms of deadlines if other assumed interdependencies remain unchanged. we now see that this factor has gained importance for the aforementioned reasons, and reduces the likelihood of other outcomes. however, given the maximum probability of outcomes that there will be no eu accession (46.80%), in the case of maximum relative importance, the factor relations with kosovo may slow down eu integration, but is not key to stopping it (figures 9 and 10). how can these results be explained and understood? first, the observed model is more complex than the 2019 model, including several new factors and feedback networks of influence between them, which are the result of existing interdependencies. in addition, in the meantime, there have been several events and circumstances that have radically affected the importance of relations with kosovo in serbia's eu accession process. this primarily refers to the evidently different view of the american administration, which is more flexible and leans toward solving the problem of kosovo's unrecognized independence. it also refers to the somewhat more relaxed attitude of brussels, i.e., the eu, which resulted in less pressure and deadlines for resolving the kosovo problem. although, lately, there have been noticeable attempts to intensify the resolution of this issue, especially through bilateral negotiations with the strongest european countries. this, as well as the complicated political situation in the world, has given serbia more room for political calculations and bargaining, and left as an option the possibility for serbia to decide not to join the eu in the case of lost outcome, i.e., getting nothing and losing everything regarding kosovo. in any case, when it comes to relations with kosovo, the outcome depends on serbia to a large extent, which is not the case with some other factors and circumstances. thus, if we look at the impact of the change in the relative importance of the relations with the united states sub-criterion, the increase in the relative importance of this sub-criterion leads to a slight increase in the probability that serbia will not join the eu. this can be understood in light of the fact that the united states, as the creator of kosovo's so-called independence, is one of the key actors who create policy in the western balkans and whose influence must be counted on in political planning. however, it is in the interest of ijahp article: krstic, mimovic/the impact of the new global reality on the european integration process: the case of the republic of serbia international journal of the analytic hierarchy process 429 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.883 the united states that serbia is not under the influence of china and russia. analysis of the impact of changes in the relative importance of relations with china and russia gives very similar results, a slight increase in the probability of non-entry into the eu, up to just over 30% and stable trends in the probability of other outcomes, while the impact of changes in the relative importance of the eu internal policy sub criterion is insignificant. similar sensitivity analyses of the solution can be performed for other sub-criteria in the model. since an element can directly and indirectly affect another element, and by its influence also a third element, any such possibility and its influence on the final result should also be considered (azis, 2003). figure 9 impact of the change in the relative importance of the sub-criteria relations with kosovo on the development of the probability of achieving alternatives (2019) figure 10 impact of the change in the relative importance of the sub-criteria relations with kosovo on the development of the probability of achieving alternative outcomes (2021) 0.00 0.20 0.40 0.60 0.80 1.00 1.20 1 2 3 4 5 6 7 relations with kosovo no accession to the eu accession to the eu by 2025 accession to the eu between 2025 and 2030 0.00 0.20 0.40 0.60 0.80 1.00 1.20 1 2 3 4 5 6 7 relations with kosovo no accession to the eu accession to the eu by 2025 ijahp article: krstic, mimovic/the impact of the new global reality on the european integration process: the case of the republic of serbia international journal of the analytic hierarchy process 430 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.883 having the above in mind, it can be concluded that the circumstances in the world have changed dramatically in a short time, and have had unforeseeable consequences, especially for small countries such as serbia. the lack of consensus on key global issues between the world's most powerful countries has dramatic consequences for the policy and positioning of small countries. although the unresolved status of the former province of kosovo has so far been perceived by the public as a key obstacle to serbia's accession to the european union, the results of this and previous research indicate a synergy of complex influences of various internal, regional and global factors. therefore, this analysis has shown that the unresolved status of kosovo cannot be viewed in isolation from other identified factors. within only two years, the perception of the accession process and its dynamics has changed significantly, both within the eu public and the serbian public, with both sides, despite declarative statements, clearly tired of this story. 5. conclusion the paper analyzes whether and in what way global and regional circumstances such as the covid-19 pandemic, brexit, the strengthening of china and russia, as well as political instabilities and the u.s. administration's disorientation in the new global reality, affect the dynamic of serbia's eu accession process to the european union. special attention is paid to relations with the former province of kosovo, as well as the structural weaknesses of the european union. although in theoretical-methodological and applicative terms, the paper is an extension of previous research, the fact that the time dimension is included imposed a conditio sine qua non dynamic, an interdisciplinary approach which is crucial in the context of completely new circumstances on a global scale. observing real problems from different time distances provides a source of new information, and the coordination and synthesis of the preconditions are created for their comparative analysis and better strategic decision making. a complex multi-attribute decision-making methodology known as the analytical network process was used to include and prioritize relevant factors that define serbia's geostrategic position, the mutual influences of the factors, and prioritize alternatives identified in the form of indicative dates for eu accession or inability to complete the process. the results of the model showed significantly different results than the research conducted in 2018. this can be partly explained by the redefinition of the structure of the model resulting from the introduction of new elements and their interdependencies as well as their impact on model alternatives. however, a certain shift in the political focus in the international politics of the republic of serbia is also a reason according to the authors. geostrategic factors still have the highest relative importance (64%); however, this factor has higher relative importance than four years ago when it was just over 50%. the relative importance of the influence of cost factors as well as the influence of internal factors has increased, which is understandable given the political situation in the country. the importance of the influence of risk factors has decreased. it is estimated that the ijahp article: krstic, mimovic/the impact of the new global reality on the european integration process: the case of the republic of serbia international journal of the analytic hierarchy process 431 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.883 relative importance of the sub-criteria relations with kosovo is large (82%) because of the intensifying and updating of the resolution of the status of kosovo, and the determination of the eu and the new u.s. administration to raise this issue ad acta. on the other hand, the outcome of the prediction seems to have changed substantially, but not only in terms of the probability of realization and the rank of alternatives. four years after the first survey, the highest probability of realization (28%) is the outcome no entry into the eu, which is almost identical to the probability of realization for the alternatives entry into the eu by 2025 (26.81%), entry into the eu after 2030 (23%) and eu accession between 2025 and 2030 (21%). therefore, the conclusion in relation to the previous survey, when it stated "that serbia will either join the eu by 2025, or will not join this union at all", could now read that it is certain that it will join the eu because the cumulative probability of achieving joining the eu at any time is 72%, while the probability that it will not join the eu is 28%. at the time of writing, it appears that the new circumstances are in serbia’s favor for the accession process, especially considering that the european commission has adopted a new methodology for the eu enlargement process as suggested by france. this methodology includes strengthening mutual trust and more precise formulation of commitments by both the eu and the western balkans as a whole, stronger political orientation and engagement at the highest levels, grouping negotiation chapters into six thematic clusters and greater clarity on what the eu expects from enlargement countries at different stages of the accession process. a sensitivity analysis also yielded interesting results. the growth of the relative importance of the factor relations with kosovo affects the probability of achieving an alternative outcome. there is the alternative no entry into the eu, but it does not reach 50% (maximum of 48%), while the probability of achieving other alternative outcomes decreases slightly. the sensitivity analysis with the influences of other key factors, such as relations with the u.s., russia and china, leads to a similar conclusion, with the growing relative importance of these factors it is more likely that serbia will not join the eu. however, but although the alternative no entry has the highest probability of realization, that probability is relatively small (less than 50%), so in the long run, there is reason for optimism regarding the final outcome of the accession process. the relative importance of the eu's internal factors shows that this influence is almost non-existent. the conclusion is then that success and timing of entrance into the eu depends primarily on serbia, and their strategic commitments. bearing in mind that a large number of factors and their network of influences had to be taken into account for serious analysis, the theoretical, methodological and practical implications of applying the analytical network process can be considered in order to predict the final outcome of serbia's eu accession process. the theoretical implications are reflected in the fact that the effective ability of the analytical network process to conceptually and creatively include all identified relevant, qualitative and quantitative factors from the prediction context has been confirmed. the presentation of the use of the anp predictive model in the field of european integration, which is extremely complex and uncertain, confirms its ability to be a good support in the forecasting process. in situations characterized by pronounced complexity and uncertainty, prediction methods based on econometric models do not give satisfactory ijahp article: krstic, mimovic/the impact of the new global reality on the european integration process: the case of the republic of serbia international journal of the analytic hierarchy process 432 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.883 results. social justification is reflected in the possibility of using research results to redefine existing or make new strategic decisions in the stabilization and association process. the limitations of the research are related to reduced accuracy, lack of information and increased bias when applying qualitative prediction methods. also, for certain clusters and sub-criteria it is possible to identify and assess more complex networks of influence in the model and analyze in more detail the indirect influences between individual factors and their influence on the final result. given the limited cognitive capacities of the decision makers, this can certainly affect the growth of subjectivity in the model results. significant objectivity could be achieved by including a group of experts in areas relevant to the european integration process in the process of comparing elements of the model by pairs. when the individual rankings of alternatives and their probabilities of realization are obtained, a geometric environment can be used to determine the unique and objectified ranking of alternative outcomes and a better assessment of their probability of realization. also, the fact that clusters cover only some negotiating chapters may have an impact on the results obtained. ijahp article: krstic, mimovic/the impact of the new global reality on the european integration process: the case of the republic of serbia international journal of the analytic hierarchy process 433 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.883 references adamus, w. (2010). when shall poland enter the euro zone? decision making with the anp.the school of business, 1, 31-52. azis, i.j. (2003). analytic network process with feedback influence: a new approach to impact study.prepared for a seminar organized by the department of urban and regional planning, university of illinois at urbana-champaign, in conjunction with the investiture ceremony for professor john kim, 1-22. azis, i. j. (2010). predicting a recovery date from the economic crisis of 2008.socioeconomic planning sciences, 44(3), 122-129. doi: https://doi.org/10.1016/j.seps.2010.03.003 blair, a. r., mandelker, g. n., saaty, t. l., & whitaker, r. (2010).forecasting the resurgence of the us economy in 2010: an expert judgment approach.socio-economic planning sciences, 44(3), 114-121.doi: http://dx.doi.org/10.1016/j.seps.2010.03.002 bongardt, a. & torres, f. (2020). lessons from the coronavirus crisis for european integration. intereconomics, review of european policy, 55(3), 130-131. doi: https://doi.org/10.1007/s10272-020-0883-3 bonomi, m., merja, a., toglhofer, t. & reljić, d. (2020). make or break moment: eu enlargement in southeast europe in pandemic thimes. policy paper, friedrich ebert stiftung, 1-29. coronavirusdisease (covid-19). available at: https://www.who.int/emergencies/diseases/novel-coronavirus-2019 dikmen, i., talat, b., ozorhon, b., & egilmezer sapci, n. (2010). using analytic network process to assess business failure risks of construction firms.engineering, construction and architectural management, 17(4), 369-386. doi: http://dx.doi.org/10.1108/09699981011056574 eraqi, a. & shoura, w. (2019). a model proposed for the prediction of future sustainable residence specifications using analytical network process.international journal on environmental science and sustainable development, 109-126. doi: http://dx.doi.org/10.21625/essd.v4i3.681 european council. (2015). normalisation of relations between serbia and kosovo, chapter 35: other issues. conference on accession to the european union–serbia– brussels, 30 november 2015, 1-8. european commission. (2020). communication from the commission to the european parlament, the council, the european economic and social committee and the committee of the regions. 2020 communication on eu enlargement policy, brussels, 166. https://www.who.int/emergencies/diseases/novel-coronavirus-2019?gclid=eaiaiqobchmi2pme8y3f7givmad7ch1lzg1reaayasaaegi6xpd_bwe ijahp article: krstic, mimovic/the impact of the new global reality on the european integration process: the case of the republic of serbia international journal of the analytic hierarchy process 434 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.883 european parliament resolution of 25 november 2020 on the foreign policy consequences of the covid-19 outbreak. available at: https://www.europarl.europa.eu/doceo/document/ta-9-2020-0322_en.html huang, c., wang, y. & li, x. (2020). clinical features of patients infected with 2019 novel coronavirus in wuhan, china. lancet, 395(10223), 497–506. doi: https://doi.org/10.1016/s0140-6736(20)30183-5 hulle, j., kaspar, r.& moller, k. (2013). analytic network process an overview of applications in research and practice.international journal of operational research, 16(2), 172-213. doi: http://dx.doi.org/10.1504/ijor.2013.051788 jayant, a., paul, v. & kumar, u. (2015). application of analytic network process (anp) in business environment: a comprehensive literature review. international journal of research in mechanical engineering & technology, 5(1), 29-37. kheybari, s., fariba, m. &farazmand, h. (2020). analytic network process: an overview of applications. applied mathematics and computation, 367, 1-35.doi: 10.1016/j.amc.2019.124780 krstic a., savic, j. & p. (2018). forecasting the accession of the republic of serbia to the european union by using the analytic network process, xiii balkan conference on operational research, balcor 2018, 25-28 may, belgrade, proceedings, 262-269 koenig, n.& stahl, a. (2020). how the coronavirus pandemic affects the eu’s geopolitical agenda. policy paper, hertie school, jacques delors centre, 1-16. mimovic, p. (2012). application of analytical network process in forecasting automobile sales of fiat 500 l. economic horizons, 165-176. doi: 10.5937/ekonhor1203165m mimovic, p., krstic, a. & savic, j. (2019). serbia joining the european union: an anp model for forecasting date.international journal of the analytic hierarchy process, 11(1), 2-19. doi: http://dx.doi.org/10.13033/ijahp.v11i1.616 mu, e. (2006).a unified framework for site selection and business forecasting using anp. journal of systems science and systems engineering, 15, 178–188. doi: https://doi.org/10.1007/s11518-006-5006-6 mu, e.& pereyra-rojas, m. (2018). practical decision making using super decisions v3.an introduction to the analytic hierarchy process.springer. niemira, m. p. & saaty, t. l. (2004). an analytic network process model for financialcrisis forecasting. international journal of forecasting, 20(4), 573-587. doi: 10.1016/j.ijforecast.2003.09.013 ozorhon, b., dikmen, i. & birgonul, m. t. (2007).using analytic network process to predict the performance of international construction joint ventures. journal of https://www.europarl.europa.eu/doceo/document/ta-9-2020-0322_en.html ijahp article: krstic, mimovic/the impact of the new global reality on the european integration process: the case of the republic of serbia international journal of the analytic hierarchy process 435 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.883 management in engineering, 23(3), 156-163. doi: https://doi.org/10.1061/(asce)0742597x(2007)23:3(156) radyshevskaya, t.n. & andreichicova, o.n. (2010). use of analytic network process in forecasting occurrence of dental diseases, journal of volgsmu,33(1), 44-47. roloff, r. (2020). covid-19 and no one’s world: what impact for the european union? connections qj, 19(2), 25-36. doi: https://doi.org/10.11610/connections.19.2.02 saaty, t.l. & kearns, k. p. (1985). analytical planning: the organization of systems, the analytic hierarchy process series, vol. iv. pittsburgh, pa: rws publications. saaty, t. l. (1999). fundamentals of the analytic network process.proceedings of the 5th international symposium on the analytic hierarchy process, 12-14. saaty, t. l. (2001a). analytic network process .encyclopedia of operations research and management science, 28-35. saaty, t. l. (2001b). decision making with the analytic network process (anp) and its “super-decisions” software the national missile defense (nmd) example. isahp 2001 berne, switzerland, 365-382. saaty, t. l. (2001c). decision making with dependence and feedback: the anp. pittsburgh, pa: rws publications. saaty, t. l., & vargas, l. g. (2006).decision making with the analytic network process.economic, political, social and technological applications with benefits, opportunities, costs and risks. springer us. saaty, t.l. & peniwati, k. (2008).group decision making: drawing out and reconciling differences. pittsburgh, pa: rws publications. saaty, t. l., & vargas, l. g. (2013). decision making with the analytic network process. economic, political, social and technological applications with benefits, opportunities, costs and risks, 2 nd edition. springer us. shih, h. s., lee, e. s., chuang, s. h. & chen, c. c. (2012). a forecasting decision on the sales volume of printers in taiwan: an exploitation of the analytic network process. computers & mathematics with applications, 64(6), 1545-1556. doi: https://doi.org/10.1016/j.camwa.2011.12.082 vargas, l. g. (1990). an overview of the analytic hierarchy process and its applications. european journal of operational research, 48(1), 2-8. doi: https://doi.org/10.1016/0377-2217(90)90056-h voulgaridou, d., kirytopoulos, k., & leopoulos, v. (2009). an analytic network process approach for sales forecasting. operational research, 9(1), 35-53. doi: http://dx.doi.org/10.1007/s12351-008-0026-2 ijahp article: krstic, mimovic/the impact of the new global reality on the european integration process: the case of the republic of serbia international journal of the analytic hierarchy process 436 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.883 wang, c., horby, p., hayden, f. & gao, g. (2020). a novel coronavirus outbreak of global health concern. lancet, 395(10223), 470–473. doi: https://doi.org/10.1016/s01406736(20)30185-9 ijahp article: shamshol bahri, shariff, yahya/ comparative analysis on decision criteria for port personnel using hybrid analytical hierarchy process (h-ahp) international journal of the analytic hierarchy process 1 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 comparative analysis on decision criteria for port personnel using hybrid analytical hierarchy process (h-ahp) muhamad safuan shamshol bahri malaysia institute of transport (mitrans) universiti teknologi mara, shah alam malaysia msafuansham@gmail.com s.sarifah radiah shariff 1* malaysia institute of transport (mitrans) universiti teknologi mara, shah alam school of mathematical sciences, college of computing, informatic and media universiti teknologi mara, shah alam malaysia shari990@uitm.edu.my nazry yahya johor port skills centre, johor port berhad malaysia nazry@johorport.com.my abstract the demand for talented labor to serve the maritime logistics, particularly in port operations, is growing as the industry expands globally. this requires professional competency in terms of manpower and skills that must be developed effectively. furthermore, considering the harm done to the industry from the covid-19 pandemic, capable professionals in port management are a crucial part to reviving the business and its long-term growth and viability. this study explores the criteria for the needed talent from the perspective of port logistics experts using the following multicriteria decision making (mcdm) approaches: analytical hierarchy process (ahp), technique for order of preference by similarity to ideal solution (topsis) and preference ranking organization system method for enrichment evaluation (promethee). the objective of this study is to identify the important criteria for personnel performance evaluation in the port marine logistics industry. in order to determine the performance evaluation framework for personnel performance evaluation, the study uses the ahp method to calculate the weightage of the criteria. the highest weightage is work attitude (0.560), followed by job performance (0.298) and work ability (0.120). lastly, in order to identify the suitable hybrid mcdm approaches for personnel performance evaluation in the port marine logistics industry, three different mcdm approaches (ahp, topsis and promethee) were used and the results show that the ahp is the best mcdm method to rank the 1 acknowledgements: we would like to acknowledge malaysia institute of transport (mitrans) uitm shah alam for funding this study and all participants who contributed to this study. *corresponding author mailto:msafuansham@gmail.com mailto:shari990@uitm.edu.my ijahp article: shamshol bahri, shariff, yahya/ comparative analysis on decision criteria for port personnel using hybrid analytical hierarchy process (h-ahp) international journal of the analytic hierarchy process 2 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 personnel in the port industry by obtaining the highest kendall’s tau coefficient of 0.619. keywords: analytical hierarchy process; topsis; promethee; hybrid; marine personnel 1. introduction logistics assists the flow of goods within and ahead of national borders and is deemed a significant component of the modern economy. it is a key enabler for economic sectors such as manufacturing, agriculture, and retail. therefore, the logistics industry is important since it has the ability to improve the economic and social opportunities of a nation in delivering positive multiplier effects such as improved market accessibility and job availability. since a good business performance measurement system is a very powerful tool to motivate and monitor employees, especially within the logistics related industry, a well-designed system has gathered interest among researchers. in the current competitive environment, a performance measurement system adopted by companies should include strategic success factors as part of the elements. at the same time, as the logistics industry has become a critical component in the commercial link by adding value, the growing business in sea trade has increased the value of services such as electronic tracking, warehousing, and resources distribution. in addition, 90% of global trade in malaysia is conducted through marine transportation and logistics. the marine logistics industry is expected to grow globally, at a compound annual growth rate (cagr) of 4.5% during the forecasted period of 2021 to 2030 (industry growth insight, 2021). the growth of this market also drives the need for talented manpower to support the industry. key players in the industry in many countries are focusing on producing talented manpower who are innovative and creative with technological advances and able to develop pioneering breakthroughs. however, there is a need to determine the traits of the right individuals for the industry. in malaysia, in order to remain competitive, all industry players need to act fast in line with the development of technology produced through industry 4.0. necessary steps include strengthening the education system in the field of maritime technology and establishing a national shipping and port council (nspc) in order to ensure the continuity of the talent produced for the industry (abdhamid, 2020). also, in malaysia the employment market is rigid in nature, meaning people who have been in a particular industry, namely marine logistics, for many years accumulating specific knowledge and skills are at a loss when they are retrenched because all their hardearned knowledge might not be suitable in other industries (benjamin, 2019; cicek et al., 2019). in this study, several traits or criteria for talented professionals, especially for the marine logistics industry, were considered in an effort to develop a performance evaluation model. since performance evaluation is expected to be comprehensive, it should also be reliable and consistent. the chosen indicators must be selected carefully as most of the time those indicators are weighted subjectively. different levels of working positions as well as different job scopes produce different opinions and views about the indicators. qu et al. (2015) proposed a hybrid model to measure the performance of professionals in the marine logistics industry for innovation and technology. our study adopted the same three main criteria: job performance, work ability and work attitude, followed by several other sub-criteria. in a study by othman et al. (2020), an ijahp article: shamshol bahri, shariff, yahya/ comparative analysis on decision criteria for port personnel using hybrid analytical hierarchy process (h-ahp) international journal of the analytic hierarchy process 3 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 advanced analytical hierarchy process (ahp) was used to identify the main factors that contribute to imbalanced cargo flows at large-scale minor ports in malaysia. extensive adaptation of the ahp can be seen in constructing the performance evaluation metrics within various organizations. in this study, in line with the growing need for talented manpower in maritime logistics, specifically focusing on port personnel (all roles in ports that are able to contribute to smooth operations), criteria for measuring their performance were explored using various combinations of the ahp. therefore, the objective of this study is to adopt various combinations of mcdm approaches to the ahp to evaluate the performance of port personnel and compare the performance of each of the combinations. 2. methodology table 1 shows the proposed criteria to be considered as the indicators for performance. the ultimate goal of the performance of each professional in the industry is to measure their performance while considering three elements: job performance, work ability and work attitude. however, each of the elements is constituted by several criteria as described in table 1. table 1 proposed elements for three level indicators goal indicator level indicator secondary indicators tertiary indicators to determine the criteria with the main priority job performance productivity(𝑉11) job experience (𝑉111) workplace (𝑉112) organization rule (𝑉113) personnel behavior(𝑉12) leadership(𝑉121) work culture(𝑉122) effective communication(𝑉123) work ability social influence(𝑉21) reputation(𝑉211) success rate(𝑉212) referral center(𝑉213) core competence(𝑉22) keen insight and flexibility (𝑉221) logical thinking(𝑉222) innovation ability(𝑉223) work attitude teamwork(𝑉31) sharing knowledge(𝑉311) cooperation spirit(𝑉312) team diversity(𝑉313) job satisfaction(𝑉32) responsibility(𝑉321) enthusiasm for work(𝑉322) discipline(𝑉323) source: (bahri et al., 2020) ijahp article: shamshol bahri, shariff, yahya/ comparative analysis on decision criteria for port personnel using hybrid analytical hierarchy process (h-ahp) international journal of the analytic hierarchy process 4 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 2.1 data setting close-structured questionnaires were distributed to port personnel at the managerial level in several ports in malaysia to gain their feedback regarding the research. the purposive sampling technique was applied to focus on the selection of a group of qualified decision makers based on their experience and expertise within the scope of this study. in this study, there were two parts to the questionnaire (part a and part b). the objective of part a was to determine the elements for the performance indicators of the port personnel and to ensure that they are in line with the experts’ opinions. the respondents were asked to rank the importance of each element and its sub-elements. the responses were tabulated and examined for consistency and validity. in part b, the objective of the questionnaire was to determine the weightage of the indicator of the performance evaluation index. the respondents were asked to compare the importance of each sub-element based on saaty’s level of importance as seen in table 2. table 2 saaty’s level of importance scale meaning explanation 1 equally important two criteria have equal importance 3 weakly important experience and judgement slightly favor one criterion over another 5 strongly important experience and judgement strongly favor one criterion over another 7 very strongly important a criterion is favored very strongly over another 9 extremely important the evidence favoring one criterion over another is of the highest possible affirmative 2,4,6,8 intermediate value between adjacent scales when there is a compromise between the judgement source: (saaty, 1990) considering the consistency and validity of the responses, only five (5) experts’ responses were used for the detailed analysis. those experts are the top managers from different departments of major ports in malaysia, had more than 10 years working experience and hold decision making positions. table 3 summarizes the profile of experts. ijahp article: shamshol bahri, shariff, yahya/ comparative analysis on decision criteria for port personnel using hybrid analytical hierarchy process (h-ahp) international journal of the analytic hierarchy process 5 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 table 3 selected experts respondent organization department position years worked r1 thought partners group consulting group managing partner 30 r2 johor port operations group managing container 27 r3 kuantan port operations chief of operation 24 r4 port tanjung pelepas quality assistant group managing 19 r5 johor port berhad special project senior manager 14 2.2 hybrid multi-criteria decision making (mcdm) there has been a substantial effort in the use of combined mcdm methods or hybrid methods in order to improve decisions. mesghouni et al. (1999), one of the earlier references to a hybrid method in decision-making (hmcdm), considered three approaches in their study. genetic algorithms (gas), constraints logic programming (clp), and mcdm were combined to address a scheduling challenge. shyur and shih (2006) coined the name "hmcdm" to describe the mcdm approach that combined the analytic network process (anp) with topsis. figure 1 shows how mcdm methods can be combined with other methods to calculate the relative significance of criteria. figure 1 composition of hybrid mcdm source: (zavadskas, et al., 2016) ijahp article: shamshol bahri, shariff, yahya/ comparative analysis on decision criteria for port personnel using hybrid analytical hierarchy process (h-ahp) international journal of the analytic hierarchy process 6 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 several flaws in traditional mcdm methods can be addressed by employing the proposed hybrid methods, as follows: (1) choosing the best strategy is a constant issue in any circumstance that necessitates a decision. different mcdm methods can result in different alternative rankings. there is no single method, specific or general, that can be deemed the best for any situation (saaty & ergu, 2015). therefore, it is recommended that more than one mcdm approach be employed and the results combined before making a final decision. (2) the importance of each criterion in the studied situation might have a substantial impact on the ranking order and final conclusion. there are studies available that do not use weighting and assign the same importance to all criteria (dahooie et al., 2018). the hybrid strategy proposes doing two jobs at once, defining criteria weights and values and incorporating them into the multiattribute utility function value. furthermore, incorporating criteria weights, which are derived using a variety of objective and subjective weighing methodologies, helps to more accurately reflect stakeholders' preferences. (3) the decision-making models should be as near to real-world challenges as possible. fuzziness in the decision-making process sometimes arises from unclear management situations where ambiguities and challenges make it difficult to draw informed conclusions. as a result, it is preferable to combine mcdm with fuzzy sets or grey numbers. fuzzy logic could aid in the resolution of uncertainty arising from human qualitative judgments and incomplete preference connections (herrera-viedma et al., 2020). (4) other methods can be used to provide further rationale in the problem formulation. because there are no universally accepted metrics for sustainability assessments (ingwersen et al., 2014; lima-junior & carpinetti, 2020), quantitative and qualitative methods can be used to generalize data, choose sustainability assessment indicators, and derive evaluation criteria for multiple criteria analysis. 2.3 types of mcdm methods there are several mcdm approaches that are commonly used in the areas of operations research and management science. these two areas are focused on making better decisions based on logical approaches. therefore, in order to ensure structured decisions are made, many attempts to consider all possible elements resulted in these approaches. these approaches each have their own strengths in producing structured decisions for any organization. analytical hierarchy process (ahp) according to yadav et al., (2015), the analytical hierarchy process (ahp) was first established in 1977 by thomas saaty at the wharton business school. the ahp is primarily based on pairwise comparison matrices, which are used to establish preferences between alternatives for various criteria and rating systems with the help of a decision maker. the technique analyzes both qualitative and quantitative elements. it breaks down complex issues into tangible and intangible components, then organizes them into hierarchies of criteria and options, ranking them from most to least important (korkmaz, 2019). ijahp article: shamshol bahri, shariff, yahya/ comparative analysis on decision criteria for port personnel using hybrid analytical hierarchy process (h-ahp) international journal of the analytic hierarchy process 7 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 many academics are interested in the ahp because of its solid mathematical approach and the ease in which the essential input data can be obtained. the ahp is a decision-making approach that can be used to solve difficult decisions involving many criteria. objectives, criteria, subcriteria, and options are organized in a multilevel hierarchical structure. a set of paired comparisons is used to acquire relevant data. in terms of each individual decision criterion, this comparison is used to determine the importance of decision criteria and alternative performance indicators. there is also a way to improve consistency if the comparison is not entirely consistent (pamucar et al., 2018; mi et al., 2019) the ahp provides decision-makers with a variety of benefits and advantages. making the selection process transparent, organizing decisions in a structured hierarchical style, offering a framework for reviewing and reconciling decision contradictions, and facilitating information synthesis and sensitivity analysis are all examples of the advantages of using the ahp (sharma & sehrawat, 2020). according to kraujalienė, (2019), the ahp's main feature is pairwise comparison, which is useful for comparing several options in the case of multiple variables and subjective preferences. to derive the priority scales, this method relies on the judgements of selected specialists or experts. technique for order of preference by similarity to ideal solution (topsis) kahraman et al. (2007) were among the first to design topsis, the well-known traditional mcdm approach. topsis finds the solutions that are simultaneously close to the ideal point and far from the anti-ideal point by positioning the options. as a result, the relevant options are chosen. topsis is based on a simple and straightforward concept. it depicts the distances to both the positive and negative ideal solutions (pis and nis) in real time, and preferences are ordered based on their relative closeness, as well as a mixture of these two-distance metrics (solangi et al., 2021). topsis is a method for defining ideal and anti-ideal solutions that is based on the premise that the best alternatives should be the closest to the positive ideal solution (pis) and the furthest away from the negative ideal solution (nis). topsis is a preferred method since it gives better results compared to other considered approaches in ranking the candidates or the alternatives (korkmaz, 2019). the topsis approach is simple to use and can efficiently manage both qualitative and quantitative data. however, it only analyzes a single value, whereas human judgments are frequently confusing and cannot be assessed using fixed numbers. the topsis tool is based on the following rule according to ighravwe et al. (2021). the optimal output should be the farthest point from the negative ideal solution point and the shortest line from the positive ideal solution point. preference ranking organization system for enrichment evaluation (promethee) promethee (preference ranking organization system for enrichment evaluation) is another preferred decision-making method due to its simple concept and application when compared to other methods. it is ideally suited to challenges with a limited number of possible actions to rate based on some, often contradicting, criteria (turcksin et al., 2011). the promethee technique has six primary types of optional functions, allowing the decision maker to define flexible standards based on specific requirements. ijahp article: shamshol bahri, shariff, yahya/ comparative analysis on decision criteria for port personnel using hybrid analytical hierarchy process (h-ahp) international journal of the analytic hierarchy process 8 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 the promethee method is an outranking approach that evaluates alternatives with respect to multiple criteria, which is a highly useful characteristic in this study because there are a few alternatives considered simultaneously. in addition, the promethee method is capable of approaching the methods of expression and synthesis of human mind choices in the face of competing perspectives. the promethee approach involves constructing an outranking relationship by comparing the assessed alternatives pairwise in each individual criterion (kolios et al., 2016). 3. results and discussion the objective of this study is to adopt various combinations of mcdm approaches with the ahp to evaluate the performance of port personnel and compare the performance of each of the combinations. part a of the questionnaire asked the experts to rank the importance of all criteria and subcriteria considered in this study. table 4 shows the results of part a. based on the ranking from 5 to 1 (very strongly agree to disagree), the average score from the experts was 3.88 with a low standard deviation of 0.61. however, the lowest score was 3.00 for code (𝑉212) which is success rate. this means that most of the experts gave a rank of 3 meaning they agree with considering this criterion in the evaluation of personnel. the highest standard deviation was 0.89 for codes (𝑉122), (𝑉123) and (𝑉221) which are work culture, effective communication and keen insight and flexibility, respectively. table 4 shows that the scores given in ranking the indicators were not largely dispersed or were nearly constant. this is due to the low value of the standard deviation (0.61). a low standard deviation means the values are consistent with each other. a low standard deviation indicates that the values tend to be close to the mean while a high standard deviation indicates that the data are more spread out and not consistent. large standard deviation values show that an estimation procedure yields very different parameter estimates when facing equivalent data and its estimations are not precise (bahri et al., 2020). thus, the proposed criteria are acceptable to evaluate the personnel of port logistics. ijahp article: shamshol bahri, shariff, yahya/ comparative analysis on decision criteria for port personnel using hybrid analytical hierarchy process (h-ahp) international journal of the analytic hierarchy process 9 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 table 4 summary of experts’ opinions on level of importance of evaluation criteria criteria ranking by experts code average std dev (𝑉111) 4.40 0.55 (𝑉112) 3.60 0.55 (𝑉113) 3.80 0.45 (𝑉121) 4.60 0.55 (𝑉122) 3.60 0.89 (𝑉123) 4.60 0.89 (𝑉211) 3.20 0.45 (𝑉212) 3.00 0.00 (𝑉213) 3.20 0.45 (𝑉221) 3.60 0.89 (𝑉222) 4.00 0.71 (𝑉223) 4.00 0.71 (𝑉311) 3.80 0.84 (𝑉312) 4.20 0.84 (𝑉313) 3.40 0.55 (𝑉321) 4.00 0.71 (𝑉322) 4.20 0.45 (𝑉323) 4.60 0.55 average 3.88 0.61 after the criteria and subcriteria were finalized, the weightage of the criteria and subcriteria were calculated using the analytical hierarchy process (ahp). as mentioned earlier, a tri-level hierarchical structure of criteria, with three criteria at the first level, six sub-elements at the second level and 18 sub-elements in the third level, with three for each sub-elements of the second level criteria, was proposed. each of the main criteria were compared against each other (pairwise) and the results are presented in the preference matrix in table 5. in the ahp, when two criteria are compared the reciprocal relationship between them is assumed (qu et al., 2015; othman et al., 2020). in table 5, when job performance (𝑉1) is compared to work ability (𝑉2), the value is 8/3 which means job performance (𝑉1) is very extremely important compared to work ability (𝑉2), and work ability (𝑉2) is only 3/8 = 0.375 or 37.5% important compared to job performance (𝑉1). note that the value for comparing work attitude (𝑉3) to work ability (𝑉2) is not exactly its reciprocal of comparison between work ability (𝑉2) to work attitude (𝑉3). as there was more than one decision maker in this study, the preferences of the decision makers were averaged, and the value of ( ) was calculated using the following equation: ijahp article: shamshol bahri, shariff, yahya/ comparative analysis on decision criteria for port personnel using hybrid analytical hierarchy process (h-ahp) international journal of the analytic hierarchy process 10 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 where k = total number of decision-makers (othman et al., 2020). table 5 pairwise comparison matrix for main criteria (𝑽𝟏) (𝑽𝟐) (𝑽𝟑) (𝑉1) 1 8/3 1/7 (𝑉2) 3/8 1 1/7 (𝑉3) 13/2 36/5 1 table 6 shows the calculated weights of each criterion. the weightage for job performance (𝑉1) is 0.161, work ability (𝑉2) = 0.084 and work attitude (𝑉3) = 0.755. the weightage of work attitude (𝑉3) is the highest which means it is ranked as the top priority in measuring performance. job performance (𝑉1) is ranked second and work ability (𝑉2) is ranked third. table 6 calculated weightage (w) of main criteria (𝑽𝟏) (𝑽𝟐) (𝑽𝟑) weightage (𝑉1) 0.13 0.25 0.11 0.161 (𝑉2) 0.05 0.09 0.11 0.084 (𝑉3) 0.83 0.66 0.78 0.755 3.1 consistency check the consistency is calculated in order to ensure the judgements of the experts are consistent. the consistency check of the comparisons is determined by calculating the maximal eigenvalue according to the following equation: ijahp article: shamshol bahri, shariff, yahya/ comparative analysis on decision criteria for port personnel using hybrid analytical hierarchy process (h-ahp) international journal of the analytic hierarchy process 11 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 the value in table 5 is used to calculate the consistency index (ci) for the main criteria. for the main criteria, the ci is calculated as follows: 3.0834 0.0417 lambda(max) is the matrix’s largest eigenvalue. then, the consistency ratio is calculated using the random consistency index (ri) where consistency ratio (cr) = ci/ri = 0.0417/0.52 = 0.0802 x 100% = 8.02 % the pairwise comparisons for the evaluation of the main criteria are consistent because the cr for the comparison matrix is less than 10%. as depicted in figure 2, all the consistency ratios for each of the criteria are less than 10% which means that the judgements of the experts are acceptable. the highest cr is 9.55% for social influence competence (𝑉21) which is close to 10%. this means that the judgement may be inconsistent but is still reliable and acceptable. the lowest cr is 0% for job performance, work ability and work attitude, which means that the judgements are perfectly consistent (rahul et al., 2018; meybodi, 2015). table 7 shows the global weightage for the subcriteria. figure 2 summary of evaluation structure with its consistency ratio value ijahp article: shamshol bahri, shariff, yahya/ comparative analysis on decision criteria for port personnel using hybrid analytical hierarchy process (h-ahp) international journal of the analytic hierarchy process 12 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 table 7 global weightage for subcriteria elements weightage consistency index 𝑉111 0.055 0.0728 𝑉112 0.069 𝑉113 0.116 𝑉121 0.030 0.0955 𝑉122 0.006 𝑉123 0.023 𝑉211 0.023 0.0754 𝑉212 0.029 𝑉213 0.024 𝑉221 0.010 0.0176 𝑉222 0.011 𝑉223 0.044 𝑉311 0.147 0.0147 𝑉312 0.199 𝑉313 0.118 𝑉321 0.038 0.0190 𝑉322 0.023 𝑉323 0.035 3.2 data preparation and application this study attempted to rank the port personnel based on the proposed criteria identified in section 3.1. the ranking was carried out using three multi-criteria decision-making methods, ahp, topsis and promethee. the reason the mcdm methods were used was that they allow the personnel evaluation to be conducted in an easier and faster manner, considering input from experienced management within the organization. the weightage of the calculated criteria using the ahp (as summarized in figure 2) is used in this section in order to rank the personnel. figure 3 shows the flow of the analysis. in this study, the performance of seven personnel (demerci & kılıç, 2019) was used to test the applicability of the proposed approaches. the personnel scores on the seven criteria used to measure their performance are shown in tables 8 and 9. the criteria are education, personality and personal skills, experience, technical skills and requirements, foreign language, vocational flexibility and exam results. ijahp article: shamshol bahri, shariff, yahya/ comparative analysis on decision criteria for port personnel using hybrid analytical hierarchy process (h-ahp) international journal of the analytic hierarchy process 13 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 figure 3 flow of analysis ijahp article: shamshol bahri, shariff, yahya/ comparative analysis on decision criteria for port personnel using hybrid analytical hierarchy process (h-ahp) international journal of the analytic hierarchy process 14 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 table 8 adapted criteria from literature no criteria from demerci and kılıç (2019) proposed criteria (in this study) 1 education quality of personnel 2 personality personnel behavior 3 experience social influence technical skills 4 exam result core competence 5 foreign language team cooperation 6 vocational job satisfaction the scores of the seven criteria in table 9 were then adapted into the new value to be in line with the new proposed criteria as shown in table 10 using the monte carlo simulation method (zhu, tian, & yan, 2020). table 9 scores of seven criteria from the literature education personality experience technical skills foreign language vocational exam results c 1 100 100 100 80 85 100 100 c 2 97 100 80 80 85 100 100 c 3 97 100 80 70 75 100 95 c 4 97 100 90 85 80 100 80 c 5 100 100 80 70 65 100 10 c 6 100 100 80 75 70 100 15 c 7 100 100 50 70 50 100 20 each personnel has his/her own associated score of criteria. the score for each personnel was derived from the literature review (demirci & kılıç, 2019) and was modified by recalculating the probability distribution for scores in the form of the proposed structure of criteria depicted in figure 2. recall that the proposed structure of performance evaluation is based on three main criteria and 18 subcriteria (figure 2). as mentioned earlier, the relevant scores were then generated using the monte carlo simulation method (zhu, tian, & yan, 2020). for example, c1 from the literature scored 100 for education (which is mapped onto quality of personnel, refer to table 8), then using the monte carlo simulation method, the total random score for the subcriteria with respect to quality of personnel must also be 100. the first step is to randomize the number from 0-100; then, the initial value is chosen to be the value ijahp article: shamshol bahri, shariff, yahya/ comparative analysis on decision criteria for port personnel using hybrid analytical hierarchy process (h-ahp) international journal of the analytic hierarchy process 15 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 for the first subcriteria. in the next step, the second value is chosen to be the value for the second subcriteria, and so on. the total for both values must be less than 100. lastly, the value for the third subcriteria was assigned as the balance (to make up the total of 100). the randomization process was done 100 times for all candidates and the average values were calculated and adjusted for all criteria and subcriteria. table 10 scores of proposed criteria based on monte carlo simulation method criteria c1 c2 c3 c4 c5 c6 c7 job performance 𝑉111 55 10 48 21 32 50 41 𝑉112 37 56 23 28 30 17 17 𝑉113 8 31 26 48 38 33 42 𝑉121 65 49 17 28 42 18 46 𝑉122 22 13 51 39 32 43 21 𝑉123 13 38 32 33 26 39 33 work ability 𝑉211 22 86 58 37 54 47 22 𝑉212 50 22 38 78 73 72 46 𝑉213 108 52 54 60 23 37 53 𝑉221 27 15 36 15 0 5 3 𝑉222 34 44 31 47 8 6 6 𝑉223 39 41 28 18 2 4 11 work attitude 𝑉311 10 34 16 21 17 34 12 𝑉312 43 34 25 41 31 18 35 𝑉313 32 17 34 18 17 18 3 𝑉321 34 31 25 47 44 12 40 𝑉322 44 38 38 40 11 59 37 𝑉323 22 31 37 13 45 29 23 3.3 results from the ahp the ahp is primarily based on pairwise comparison matrices, which are utilized by a decision maker to generate preferences between alternatives for various criteria and rating systems. the approach considers both qualitative and quantitative factors while making decisions. there are four steps to calculate the rank of personnel using the ahp: 1) forming the decision matrix (a), 2) normalizing the decision matrix, 3) generating a weighted normalized decision matrix and 4) identifying a performance score. table 11 shows the score of the personnel after adding all the weighted normalized performance values of each personnel. rank is allocated based on the performance ijahp article: shamshol bahri, shariff, yahya/ comparative analysis on decision criteria for port personnel using hybrid analytical hierarchy process (h-ahp) international journal of the analytic hierarchy process 16 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 score. from the table, it can be concluded that c2 is the best with a performance score of 0.167 (the highest) and is ranked first table 11 performance scores of personnel personnel performance score rank c1 0.155 2 c2 0.167 1 c3 0.143 4 c4 0.155 3 c5 0.131 6 c6 0.132 5 c7 0.117 7 3.4 results from topsis the topsis approach is a multi-criteria decision-making tool developed by hwang and yoon (1981). the topsis approach is based on the largest distance between the positive and negative ideal solutions. the topsis method consists of six steps: 1) forming the decision matrix (a), 2) forming the normalized decision matrix (r), 3) forming the weighted standard decision matrix (v), 4) finding the ideal (𝐴+) and negative ideal (𝐴−) solutions, 5) calculating the distance between alternatives, and 6) calculating the relative proximity to the ideal solution. table 12 shows the ranking of personnel based on their ideal solution value. according to the results shown in table 12, the highest scoring personnel is c2 while the lowest is c6. table 12 ranking of personnel using topsis personnel p rank c2 0.631 1 c4 0.593 2 c1 0.530 3 c3 0.507 4 c6 0.488 5 c5 0.466 6 c7 0.397 7 ijahp article: shamshol bahri, shariff, yahya/ comparative analysis on decision criteria for port personnel using hybrid analytical hierarchy process (h-ahp) international journal of the analytic hierarchy process 17 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 3.5 result from promethee outranking methods are well-suited to challenges in which a limited number of alternatives must be ranked based on a set of criteria. preference ranking organization method for enrichment evaluation (promethee) has been determined to be the most stable of the outranking approaches (turcksin et al., 2011; kolios et al., 2016). the information on the weights of the criterion and the decision maker's preference functions are required for promethee to be implemented. there are no explicit standards for determining the criteria weights in promethee. one of the most extensively used mcdm methods for applications such as selection, evaluation, allocations, prioritization, and ranking is the analytic hierarchy process (ahp). macharis et al. (2004) examined the strengths and shortcomings of the ahp and promethee and presented a method to improve promethee by incorporating the positive aspects of the ahp. similar to other methods, the ahp is used to assign the criteria weights and promethee ii is used to obtain the final ranking of the personnel. table 13 shows the net outranking flow values for every personnel and their relative ranking. the best personnel that scored the highest net outranking flow is c2 while the worst personnel who scored the lowest net outranking flow is c7. table 13 net outranking values for every personnel personnel net outranking flow rank c1 0.067 3 c2 0.152 1 c3 -0.033 4 c4 0.113 2 c5 -0.052 5 c6 -0.117 6 c7 -0.131 7 3.6 choosing best mcdm method in order to choose the best mcdm method, a kendall’s tau test was implemented. kendall-tau is a non-parametric correlation coefficient that can be used to evaluate and analyze correlations between ordinal variables that are not scaled in intervals. the kendall tau correlation coefficient is sometimes abbreviated with the greek letter (tau). the spearman rank correlation coefficient and the kendall tau correlation coefficient are thought to be comparable. the spearman rank correlation coefficient is similar to the pearson correlation coefficient, except that it is calculated using rankings, whereas the kendall tau correlation is a probability. table 14 shows the results of the first part of the calculations to choose the best mcdm method from among the ahp, topsis, and promethee. kendall's tau, goodman and kruskal's gamma, and logistic regression all use concordant (c) and discordant (d) pairs. they are used to determine if there is agreement (or disagreement) between scores for ordinal (ordered) variables. the data must be organized and grouped into pairs before concordance or discordance can be ijahp article: shamshol bahri, shariff, yahya/ comparative analysis on decision criteria for port personnel using hybrid analytical hierarchy process (h-ahp) international journal of the analytic hierarchy process 18 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 calculated. concordant pairs are the number of observed ranks below a particular rank which are larger than that particular rank while discordant pairs are the number of observed ranks below a particular rank which are smaller in value than that particular rank. the rankings from table 14 are sorted from the lowest to highest. the data are compared with the literature review (demirci & kılıç, 2019). for example, in the ahp ranking, demirci & kılıç (2019) gave c1 the first rank and the ahp calculation gave it second; this is not perfectly concordant. therefore, to calculate the concordant and discordant values, the number of rankings that are larger in value from the calculation are compared to the rankings given in the literature. thus, with the ahp there are five rankings that are larger than 2, except personnel (c2) which is ranked as number 1. then, the concordant is 5 and discordant is 1. the total number of concordant pairs for ahp is 17 and there are 4 discordant pairs. the same method is used for topsis and promethee where the total value for concordant and discordant are c=16, d=5 and c=15, d=6, respectively. the kendall’s tau is calculated as follows: the kendall’s tau coefficient for the ahp is 0.619 while for topsis and promethee it is 0.524 and 0.429, respectively. these test values range from -1 to +1, with the sign telling the direction of the relationship. minus (-) means that as one increases the other decreases. while plus (+) means that as one goes up so does the other. the closer the value to +1 or -1, the stronger the relationship is. as the correlation coefficient value goes towards 0, the relationship between the two variables will be weaker; therefore, the best alternative between the three mcdm approaches is the ahp method because it has the highest kendall's tau coefficient that is closest to +1 (ramsey, 1989). table 14 kendall’s tau coefficient for mcdm approaches ahp topsis promethee lr rank c d rank c d rank c d c1 1 2 5 1 3 4 2 3 4 2 c4 2 3 4 1 2 4 1 2 4 1 c2 3 1 4 0 1 4 0 1 4 0 c6 4 5 2 1 5 2 1 6 1 2 c5 5 6 1 1 6 1 1 5 1 1 c3 6 4 1 0 4 1 0 4 1 0 c7 7 7 7 7 sum 17 4 sum 16 5 sum 15 6 0.619 0.524 0.429 ijahp article: shamshol bahri, shariff, yahya/ comparative analysis on decision criteria for port personnel using hybrid analytical hierarchy process (h-ahp) international journal of the analytic hierarchy process 19 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 4. conclusion as the marine logistics industry has become a critical component in the commercial link, talented manpower is needed to support the industry and provide a competitive advantage to the nation. this study explores the criteria for needed talents from the perspective of port logistics experts using the following mcdm approaches: ahp, topsis and promethee. the three approaches were selected based on their ability to simultaneously consider many criteria as well as consider the subjective judgement of importance from the participating experts. next, the weightage was calculated using the ahp method and the highest weightage is work attitude, followed by job performance and work ability. work attitude scored more than 50% of weightage compared to job performance and work ability. therefore, the experts believed that work attitude is the most important criterion in selecting and evaluating personnel in the port logistics industry. work attitude refers to the tendencies of personnel in a behavioral evaluation. in recruitment information, sense of responsibility and group spirit are very common, and should be given more emphasis in the logistics industry. furthermore, being cautious and working hard are essential characteristics for logistics professionals. job performance scored the second highest weightage and refers to the goal of performance management. according to the literature, employers prefer individuals who have prior work experience. employees who have worked for more than a year are expected to hold 76.1% of jobs. this indicates that recent graduates are unable to match employers’ demands in terms of practical experience and that more employers are ready to select better workers from a job market brimming with inexperienced workers. the largest need for job experience is in production logistics. work ability is the least preferred criteria by the experts in evaluating the performance of personnel. work ability refers to a person possessing a set of standardized requirements. communication skills, computer applications skills, english skills, and coordination skills have all become vital traits for a logistics expert in the current employment market. furthermore, stress tolerance is a crucial consideration for businesses when hiring logistics personnel. finally, the personnel were ranked based on their performance score using the following different approaches: ahp, topsis and promethee. based on the selected data from the literature, the results show that the best mcdm approach is the ahp method because it has the highest kendall’s tau coefficient compared to topsis and promethee. the same calculation method of kendall’s tau rank correlation was used to reconfirm that the proposed weightage of the criteria agrees with the actual evaluation at the actual organization. the results of kendall’s tau coefficient are 0.490 which means the variables are correlated. ijahp article: shamshol bahri, shariff, yahya/ comparative analysis on decision criteria for port personnel using hybrid analytical hierarchy process (h-ahp) international journal of the analytic hierarchy process 20 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 references abdhamid, a.s. (2020, february 24). plug the gaps to make malaysia a maritime nation, borneo post online. https://www.theborneopost.com/2020/02/24/plug-thegaps-to-make-malaysia-a-maritime-nation/ benjamin, r. (2019, april 9). mismatch of skills and jobs, the star. https://www.thestar.com.my/opinion/letters/2019/04/09/mismatch-of-skills-and-jobs cicek, k., akyuz, e., & celik, m. (2019). future skills requirements analysis in maritime industry. procedia computer science, 158, 270-274. doi: https://doi.org/10.1016/j.procs.2019.09.051 demirci, a. e., & kılıç, h. s. (2019). personnel selection based on integrated multicriteria decision making techniques. international journal of advances in engineering and pure sciences, 31(2), 163-178. heidary dahooie, j., beheshti jazan abadi, e., vanaki, a. s., & firoozfar, h. r. (2018). competency‐based it personnel selection using a hybrid swara and aras‐g methodology. human factors and ergonomics in manufacturing & service industries, 28(1), 5-16. doi: https://doi.org/10.1002/hfm.20713 herrera-viedma, e., palomares, i., li, c. c., cabrerizo, f. j., dong, y., chiclana, f., & herrera, f. (2020). revisiting fuzzy and linguistic decision making: scenarios and challenges for making wiser decisions in a better way. ieee transactions on systems, man, and cybernetics: systems, 51(1), 191-208. doi: https://doi.org/10.1109/tsmc.2020.3043016 hwang, c. l., & yoon, k. (1981). methods for multiple attribute decision making. multiple attribute decision making: methods and applications a state-of-the-art survey. multiple attribute decision making, 1, 58-191. doi: https://doi.org/10.1007/978-3-642-48318-9_3 ighravwe, d. e., babatunde, m. o., mosetlhe, t. c., aikhuele, d., & akinyele, d. (2021). a mcdm-based framework for the selection of renewable energy system simulation tool for teaching and learning at university level. environment, development and sustainability, 24(11), 1-22. doi: https://doi.org/10.1007/s10668021-01981-1 industry growth insights. (2021). global maritime logistics market by type (general cargo maritime logistics, bulk cargo maritime logistics, maritime logistic), by application (port service, coastal service, other) and by region (north america, latin america, europe, asia pacific and middle east & africa), forecast from 2022 to 2030. https://industrygrowthinsights.com/report/maritime-logistics-market/, retrieved november 8, 2022. ingwersen, w., cabezas, h., weisbrod, a. v., eason, t., demeke, b., ma, x., ... & ceja, m. (2014). integrated metrics for improving the life cycle approach to assessing product system sustainability. sustainability, 6(3), 1386-1413. kahraman, c., ateş, n. y., çevik, s., gülbay, m., & erdoğan, s. a. (2007). hierarchical fuzzy topsis model for selection among logistics information https://www.theborneopost.com/2020/02/24/plug-the-gaps-to-make-malaysia-a-maritime-nation/ https://www.theborneopost.com/2020/02/24/plug-the-gaps-to-make-malaysia-a-maritime-nation/ https://www.thestar.com.my/opinion/letters/2019/04/09/mismatch-of-skills-and-jobs https://industrygrowthinsights.com/report/maritime-logistics-market/ ijahp article: shamshol bahri, shariff, yahya/ comparative analysis on decision criteria for port personnel using hybrid analytical hierarchy process (h-ahp) international journal of the analytic hierarchy process 21 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 technologies. journal of enterprise information management, 20(2), 143-168. doi: https://doi.org/10.1108/17410390710725742 kolios, a., mytilinou, v., lozano-minguez, e., & salonitis, k. (2016). a comparative study of multiple-criteria decision-making methods under stochastic inputs. energies, 9(7), 566. doi: https://doi.org/10.3390/en9070566 korkmaz, o. (2019). personnel selection method based on topsis multi-criteria decision making method. uluslararası i̇ktisadi ve i̇dari i̇ncelemeler dergisi, 23, 1-16. doi: https://doi.org/10.18092/ulikidince.468486 lima-junior, f. r., & carpinetti, l. c. r. (2020). an adaptive network-based fuzzy inference system to supply chain performance evaluation based on scor® metrics. computers & industrial engineering, 139, 106191. doi: https://doi.org/10.1016/j.cie.2019.106191 macharis, c., springael, j., de brucker, k., & verbeke, a. (2004). promethee and ahp: the design of operational synergies in multicriteria analysis.: strengthening promethee with ideas of ahp. european journal of operational research, 153(2), 307-317. doi: https://doi.org/10.1016/s0377-2217(03)00153-x mesghouni, k., pesin, p., trentesaux, d., hammadi, s., tahon, c., & borne, p. (1999). hybrid approach to decision-making for job-shop scheduling. production planning & control, 10, 690–706. doi: https://doi.org/10.1080/095372899232768 meybodi, m. z. (2015). consistency of strategic and tactical benchmarking performance measures. benchmarking: an international journal, 22(6), 1019-1032. doi: https://doi.org/10.1108/bij-07-2013-0074 mi, x., tang, m., liao, h., shen, w., & lev, b. (2019). the state-of-the-art survey on integrations and applications of the best worst method in decision making: why, what, what for and what's next?. omega, 87, 205-225. doi: https://doi.org/10.1016/j.omega.2019.01.009 othman, m. k., rahman, n. s. f. a., ismail, a., & saharuddin, a. h. (2020). factors contributing to the imbalances of cargo flows in malaysia large-scale minor ports using a fuzzy analytical hierarchy process (fahp) approach. the asian journal of shipping and logistics, 36(3), 113-126. doi: https://doi.org/10.1016/j.ajsl.2019.12.012 pamučar, d., stević, ž., & sremac, s. (2018). a new model for determining weight coefficients of criteria in mcdm models: full consistency method (fucom). symmetry, 10(9), 393. doi: https://doi.org/10.3390/sym10090393 qu, q., chen, k. y., wei, y. m., liu, y., tsai, s. b., & dong, w. (2015). using hybrid model to evaluate performance of innovation and technology professionals in marine logistics industry. mathematical problems in engineering, 2015. doi: https://doi.org/10.1155/2015/361275 rahul, b., dey, p. k., ghosh, s. k., & konstantinos, p. (2018). strategic maintenance technique selection using combined quality function deployment, the analytic hierarchy process and the benefit of doubt approach. international journal https://doi.org/10.18092/ulikidince.468486 ijahp article: shamshol bahri, shariff, yahya/ comparative analysis on decision criteria for port personnel using hybrid analytical hierarchy process (h-ahp) international journal of the analytic hierarchy process 22 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.974 of advanced manufacturing technology, 94(1-4), 31-44. doi: https://doi.org/10.1007/s00170-016-9540-1 ramsey, p. h. (1989) critical values for spearman’s rank-order correlation. journal of educational statistics, 14(3), 245-253. doi: https://doi.org/10.3102/10769986014003245 saaty, t. l., & ergu, d. (2015). when is a decision-making method trustworthy? criteria for evaluating multi-criteria decision-making methods. international journal of information technology & decision making, 14, 1171–1187. doi: https://doi.org/10.1142/s021962201550025x saaty, t.l., (1990). how to make a decision: the analytic hierarchy process, european journal of operational research, 48, 9-26. doi: https://doi.org/10.1016/0377-2217(90)90057-i sharma, m., & sehrawat, r. (2020). a hybrid multi-criteria decision-making method for cloud adoption: evidence from the healthcare sector. technology in society, 61, 101258. doi: https://doi.org/10.1016/j.techsoc.2020.101258 shyur, h.-j., & shih, h.-s. (2006). a hybrid mcdm model for strategic vendor selection. mathematical and computer modelling, 44, 749–761. doi: https://doi.org/10.1016/j.mcm.2005.04.018 solangi, y. a., longsheng, c., & shah, s. a. a. (2021). assessing and overcoming the renewable energy barriers for sustainable development in pakistan: an integrated ahp and fuzzy topsis approach. renewable energy, 173, 209-222. doi: https://doi.org/10.1016/j.renene.2021.03.141 turcksin, l., bernardini, a., & macharis, c. (2011). a combined ahppromethee approach for selecting the most appropriate policy scenario to stimulate a clean vehicle fleet. procedia-social and behavioral sciences, 20, 954965. doi: https://doi.org/10.1016/j.sbspro.2011.08.104 van der vleuten, c. p., & schuwirth, l. w. (2005). assessing professional competence: from methods to programmes. medical education, 39(3), 309-317. doi: https://doi.org/10.1111/j.1365-2929.2005.02094.x yadav, a., anis, m., ali, m., & tuladhar, s. (2015). analytical hierarchy process (ahp) for analysis: selection of passenger airlines for gulf country. international journal of scientific & engineering research, 6(3), 379-389. zavadskas, e. k., govindan, k., antucheviciene, j., & turskis, z. (2016). hybrid multiple criteria decision-making methods: a review of applications for sustainability issues. economic research-ekonomska istraživanja, 29(1), 857-887. doi: https://doi.org/10.1080/1331677x.2016.1237302 zhu, y., tian, d., & yan, f. (2020). effectiveness of entropy weight method in decision-making. mathematical problems in engineering, 2020. ijahp article: shiwakoti, regmi/strategic analysis of the nepal electricity authority: a swot-ahp analysis based on stakeholders’ perceptions international journal of the analytic hierarchy process 1 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.944 strategic analysis of the nepal electricity authority: a swot-ahp analysis based on stakeholders’ perceptions sagar shiwakoti manager, nepal electricity authority kathmandu, nepal sagar.shiwakoti@nea.org.np suraj regmi* deputy manager, nepal electricity authority kathmandu nepal suraj.regmi@nea.org.np abstract the strength, weakness, opportunities & threat (swot) analysis of an organization is a crucial method for assessing the most relevant internal (strengths and weaknesses) and external (opportunities and threats) factors affecting the organization. the purpose of this study is to rank the identified factors of strengths, weaknesses, opportunities and threats of the nepal electricity authority (nea), an undertaking of the government of nepal mandated for generation, transmission, and distribution of electricity throughout the country. this paper utilizes the analytical hierarchy process (ahp), a multicriteria decision making (mcdm) technique, to rank the swot factors for nea strategic planning. the ahp method in combination with a swot analysis was utilized to conduct pairwise comparison among the identified factors in order to prioritize them using eigen values. a structured questionnaire was used to collect data from the experts of the nea, independent power producers, the ministry of energy, water resources and irrigation, energy companies, and donor agencies. the study reveals that the monopoly business nature, the huge internal demand for energy and a nationwide electricity distribution network are the main strengths of the nea whereas an inability to absorb all energy procured through “take or pay” contracts and an insufficient transmission and distribution network are the main threats and weaknesses. similarly, long-term growth of electricity demand, trade of power and energy banking are the identified opportunities. the findings of the study will help the organization utilize its strengths and opportunities, tackle its weaknesses, and mitigate the threats. keywords: swot; ahp; nepal electricity authority; electricity utility 1. introduction the nepal electricity authority (nea) is a vertically integrated power utility which is an undertaking of the government of nepal. the main objective of the nea is planning and development of generation, transmission and distribution infrastructures for interconnected and isolated power systems for adequate, affordable and reliable mailto:suraj.regmi@nea.org.np ijahp article: shiwakoti, regmi/strategic analysis of the nepal electricity authority: a swot-ahp analysis based on stakeholders’ perceptions international journal of the analytic hierarchy process 2 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.944 power service within nepal. it is also mandated to recommend shortand long-term plans and policies related to the energy sector of the government (nea, 2021). the energy sector plays a very important role in the economic development of any nation. evidence shows that expanding the electricity sector has contributed to economic growth in many countries (gunatilake, wijayatunga, & roland-holst, 2020). the electricity sector in nepal has developed considerably in recent years. the currently installed power generation capacity of the country is about 2000mw as opposed to a mere 1000mw just four years ago. the number of consumers and the demand for energy are also increasing annually as shown in tables 1 and 2. the power sector in nepal is largely dominated by hydro generation with over 99% of the annual generation from hydroelectricity. a small quantity of electricity is generated from other sources such as solar and thermal generation (nea, 2021). nepal’s enormous technically viable hydropower potential of 43gw and its geographic proximity to india and resource-starved bangladesh provide opportunities for cross-border electricity trade with its neighboring countries. nepal is looking to tap into this trading opportunity by constructing cross-border transmission infrastructures and expanding the internal transmission network. however, the internal or domestic demand needs to be fulfilled before electricity can be sold to neighboring countries. despite the huge potential, nepal’s sole state-owned electricity utility, nea, has not been able to cater to the growing domestic demand (dhakal, karki, & shrestha, 2019). on one hand, this is due to the lack of sufficient generation of electricity; while on the other hand, it is because of a lack of sufficient distribution infrastructures. nepal still depends on the energy imported from india to fulfill its own needs in the winter season when the streamflow in rivers is reduced to almost a third of the full capacity. as depicted in table 1, in 2021, nepal fulfilled 31.8% of its total annual energy needs through imports from india (nea, 2021). most of the importing of energy happens in the dry winter season that lasts from october to march. however, during the monsoon season that lasts for around four months, excess energy is generated. this causes valuable energy to be wasted because of a lack of sufficient cross-border transmission infrastructures. reliance on a single source of energy and the resulting mismatch of supply in dry and wet season energy into the national grid poses a great financial challenge to the nea in the coming years. table 1 total energy available and peak demand in the nea system during the last five years particular 2016 2017 2018 2019 2020 2021 generated by nea (gwh) 2133 2305 2308 2548 3021 2811 from ipps (gwh) 1166 1778 2168 2190 2991 3241 imported from india (gwh) 1778 2175 2582 2813 1729 2826 total available energy (gwh) 5077 6258 7058 7551 7741 8878 peak demand (gw) 1.385 1.444 1.508 1.320 1.408 1.482 source: (nea, 2021) the power sector in nepal is practically vertically integrated with the nea acting as the single buyer that purchases electricity from multiple generators or sellers, including the private or independent power producers (ipps). as shown in table 1, the contribution of electricity generation by the ipps has been more than 30% in recent years. the nea is also the single buyer of imported electricity from india. the ijahp article: shiwakoti, regmi/strategic analysis of the nepal electricity authority: a swot-ahp analysis based on stakeholders’ perceptions international journal of the analytic hierarchy process 3 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.944 electricity market has not opened up in a structured and functional manner for other players to enter the market. the nea sells bulk energy to domestic and industrial consumers at a pre-determined rate. the transmission business is also solely operated by the nea. it is mandated to construct, expand and operate the transmission network within the country. the distribution business is also operated by the nea through its seven provincial distribution offices. the distribution offices sell bulk energy to consumers throughout the nation. this vertically integrated and unbundled structure of the energy market in nepal has resulted in several challenges in the energy sector. there is a lack of a competitive electricity market which results in consumers being forced to buy electricity at predetermined and uncompetitive rates. in other words, the consumers have no choice of service provider. however, this is a strength for the nea, as it has no competitors in the market. the growing demand for electricity as shown in table 1 and the growing number of consumers as shown in table 2 are areas of opportunity for the nea to grow its business. however, the growing demand comes with the challenge of curtailing system losses (nea, 2021). table 2 growth of consumers and system loss of the nea particular 2016 2017 2018 2019 2020 2021 growth of consumers (%) 5 10 10 9 8 7 systems loss (%) 25.78 22.90 20.45 15.32 15.27 17.18 source: (nea, 2021) a review of previous works related to the energy sector in nepal in general and about the nea in particular shows that there is a lack of an integrated and detailed analysis of the strengths, opportunities, weaknesses and threats of the nea. a study was conducted regarding the barriers and opportunities in cross border electricity trading opportunities for nepal (dhakal, karki, & shrestha, 2019). several studies have been conducted regarding the need to unbundle the nea and separate it into different business units (necoechea-porras & lopez, 2021). however, there is a lack of an integrated swot analysis that is aimed at identifying the key areas of strength/opportunities and weaknesses/threats of nepal’s energy sector. this study conducted a swot analysis of the nea as a way to find out key decision supporting information that will enable the organization's resources and capabilities to be synchronized with the competitive environment in which it operates. this analysis is combined with the ahp to quantify the relative importance of each element among the groups and identify the prospective strategy crucial to its internal and external environment. the main objective of the research is to determine the key strengths, weaknesses, opportunities and threats of the nea and their relative importance. similar studies on the energy sector have been conducted in other parts of the world. a swot analysis of power utilities in the south african development community (sadc) discovered that political instability, poor water management and corruption pose endemic threats to the south african energy market (tshombe, 2013). the article, cross-border electricity trade for nepal: a swot-ahp analysis of barriers and opportunities based on stakeholders’ perception (dhakal, karki, & shrestha, 2019) identified the untapped hydro potential in nepal, coupled with bbin’s ijahp article: shiwakoti, regmi/strategic analysis of the nepal electricity authority: a swot-ahp analysis based on stakeholders’ perceptions international journal of the analytic hierarchy process 4 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.944 complementary seasonal demand patterns, differences in peak load timing, and rapidly growing electricity demand in bangladesh and india as favorable for regional electricity cooperation, especially for nepal. the article, the analytical hierarchy process (ahp) approach to the challenges of electricity power generation in nigeria (oluchukwu & emmanuel, 2019) revealed a lack of maintenance, continuous use of obsolete equipment, a biased process of staff recruitment, insufficient staff training, shortage of qualified manpower, lack of staff welfare, absence of equipment upgrades, vandalism and community disturbances as the key challenges facing the energy sector in nigeria. 2. swot and ahp model 2.1. swot analysis a swot analysis is a universally used technique for analyzing internal and external environments related to an organization in order to identify a systematic approach for decision making and strategy formulation (kurttila, pesonen, kangas, & kajanus, 2000; osuna & aranda, 2007). it is a tool that supports decision making in the context of formulating strategies for any organization. a swot analysis involves systematic thinking and comprehensive identification of factors related to an organization, its management and planning. a swot framework provides an organized basis for insightful discussion and information sharing, which could improve the quality of the choices and decisions that managers subsequently make (pearce & robinson, 2005). a swot analysis involves the identification and summarization of internal and external environmental factors that are most important to the enterprise’s future (kahraman, demirel, & demirel, 2007). the internal environmental factors are the variables that are within the control of the organization, such as its strengths and weaknesses. the external environmental factors are the variables that are out of the control of the organization, such as the opportunities and threats to the organization arising from external factors. the objective of a swot analysis is to identify these variables in order to develop and adopt a strategy that is a good fit between the internal and external factors, which are also referred to as strategic factors. it helps achieve the strategic objectives that correspond to an organization’s resources and environmental opportunities (pike, 2008). this tool helps utilize the strengths, take advantage of the opportunities, tackle the weaknesses and mitigate the threats. a swot analysis helps an organization decide how it can utilize its strengths, take advantage of the opportunities, overcome weaknesses and deal with existing threats. (lumaksuno, 2008). it is also an effective tool for achieving the strategic objectives that correspond with an organization’s resources and environmental opportunities. 2.2. analytical hierarchy process the ahp is an approach that is widely used in ranking or prioritizing multiple alternatives based on multi-criteria decision analysis. in this process, weights are assigned to compare certain criteria or alternatives, which are identified as factors influencing the decision making (gorener, toker, & ullucay, 2012). this process provides flexibility for decision making as well as for ranking and prioritizing problems. researchers use this tool to manage and formulate the hierarchy model in prioritizing the available alternatives. with this approach, subjective criteria can be quantitatively analyzed. ijahp article: shiwakoti, regmi/strategic analysis of the nepal electricity authority: a swot-ahp analysis based on stakeholders’ perceptions international journal of the analytic hierarchy process 5 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.944 the ahp helps express the decision by decomposing a complicated problem into a multilevel hierarchical structure of objective or goal, criteria/sub-criteria and decision alternatives. in this process, pairwise comparisons of the criteria are done to derive the relative importance of the variable at each level of the hierarchy. the ahp is used to determine relative priorities on absolute scales from both discrete and continuous paired comparisons in multilevel hierarchic structures (saaty & vargas, 1996). the prioritization mechanism is executed by assigning a number from a comparison scale developed by saaty (1980) to represent the relative importance of the criteria. pairwise comparison matrices of these factors are analyzed to determine the importance of the factors (sharma, moon, & bae, 2008). this method uses a reciprocal decision matrix obtained by pairwise comparisons so that the information is given in a linguistic form. the ahp method is conducted in three steps. in the initial step, the ahp model structure is set up as a hierarchy of several levels using research goals, analysis criteria and sub-criteria and the decision alternatives (gorener, toker, & ullucay, 2012). in the second step, a comparative judgment of decision alternatives is done based on the various elements of the criteria/sub-criteria based on the pairwise comparison table. in the third step, a synthesis of the priorities is done to identify the best decision alternative. the research objective, decision criteria and alternatives are arranged in a hierarchical structure similar to a family tree. a complete hierarchy has at least three levels as follows: goal of the problem at the top, multiple criteria based on which the decision is to be made in the middle, and decision alternatives at the bottom (albayrk & erensal, 2004). in this study, we used the ahp for prioritization of the swot elements of the nea (nea, 2018) to recommend the best strategy formulation by identifying the most important element in each group of the criteria. for this, the problem has been decomposed and a hierarchy has been constructed (dagdeviren & yavuz, 2009). the prioritization procedure is carried out in order to determine the relative importance of each element in each set of the criteria and also the relative importance among the different sets of criteria. for each group of the identified criteria (strength, weakness, opportunities and threats), multiple pairwise comparisons of the elements within the criteria were done based on the standardized comparison scale of nine levels as shown in table 3 (albayrk & erensal, 2004). table 3 comparison scale for pairwise comparison matrix under the ahp model importance explanation 1 two criteria contribute equally to the objective 3 importance of criteria i is slightly higher than that of j towards the objective 5 importance of criteria i is strongly higher than that of j towards the objective 7 importance of criteria i is very strongly higher than that of j towards the objective 9 importance of criteria i is absolutely higher than that of j towards the objective 2, 4, 6, 8 used to represent intermediate values reciprocal values are used if criteria j is more important than criteria i. ijahp article: shiwakoti, regmi/strategic analysis of the nepal electricity authority: a swot-ahp analysis based on stakeholders’ perceptions international journal of the analytic hierarchy process 6 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.944 let c = {cj | j = 1, 2, . . . , n} be a criteria set (one of the 4 criteria: strength, weakness, opportunity, threat of the nea), where n is the number of elements in each criteria set. the pairwise comparison obtained from the respondents on n elements of each matrix can be summarized (by averaging the response values from each respondent) in a paired comparison matrix of size n x n. this pairwise comparison can be shown by a square and reciprocal matrix aij as shown in equation 1. 𝐴 = 𝑎𝑖𝑗 = 𝑎11 𝑎12 ⋯ 𝑎1𝑛 𝑎21 𝑎22 … 𝑎2𝑛 𝑎𝑛1 𝑎𝑛2 𝑎𝑛𝑛 (1) the pairwise comparison matrix is normalized by dividing each element in the matrix by the sum of the elements in the corresponding column. normalization of the matrix gives relative weights of each sub-criterion on the basis of which, ranking of the sub-criteria can be done. this normalized relative weight of each element is obtained by calculating the normalized eigen vector of size n x 1 (w), corresponding to the largest eigen value as shown in equation 2. the normalized eigen vector is also known as the priority vector. the sum of all elements of the priority vector is 1. it shows the relative weights among the elements within each criteria set. aw = λmax.w (2) the quality of the output of an ahp analysis is dependent on the consistency of the pairwise comparison judgments. consistency of the pairwise comparison is defined by the relation between the elements of matrix a as shown in equation 3 (dagdeviren & yavuz, 2009). aij x ajk = ai (3) if a matrix is absolutely consistent or if it exists in the ideal case of total consistency, the principal eigenvalue (λmax) is equal to n (alonso & lamata, 2006). for a pairwise comparison, the consistency index (ci) can be calculated using the following formula defined by saaty as shown in equation 4. ci = λmax−n n−1 (4) this means that for an absolutely consistent matrix, ci=0; however, it is unrealistic to obtain such a comparison matrix due to the inherent nature of human responses. if the responses are not absolutely consistent, λmax > n, then the level of inconsistency needs to be measured. for this purpose, saaty defined the consistency ratio (cr) as depicted in equation 5. cr = ci ri (5) in the above equation, ri is the average value of ci for randomly generated matrices using the saaty scale (forman, 1990). for n up to 10, the values of ri as shown in table 4 were used in this study (borajee & yakchali, 2011). ijahp article: shiwakoti, regmi/strategic analysis of the nepal electricity authority: a swot-ahp analysis based on stakeholders’ perceptions international journal of the analytic hierarchy process 7 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.944 table 4 values of random index for n 1-10 n 1 2 3 4 5 6 7 8 9 10 ri 0.00 0.00 0.58 0.90 1.12 1.24 1.32 1.41 1.45 1.49 different values of ri for different values of n have been obtained by various studies (forman, 1990) (golden & wang, 1990) (saaty, 1980) (alonso & lamata, 2006) depending on the simulation method and the number of randomly generated matrices used for the study. in this study, for n > 10, the ri values obtained by using 500,000 randomly generated matrices in a study were used as shown in table 5 (alonso & lamata, 2006). only a matrix with cr < 0.1 is accepted (alonso & lamata, 2006). if the consistency ratio exceeds 0.1, the evaluation procedure needs to be repeated to improve consistency (borajee & yakchali, 2011). table 5 values of random index for n > 10 (alonso & lamata, 2006) n ri 3 0.5247 4 0.8816 5 1.1086 6 1.2479 7 1.3417 8 1.4057 9 1.4499 10 1.4854 11 1.5140 12 1.5365 13 1.5551 14 1.5713 15 1.5838 2.3. swot-ahp model the swot analysis has been widely used in various studies and research for identification of the critical factors in each group (strength, weakness, opportunity and threat) that influences an organization’s strategy. the use of a swot analysis has some limitations. it only helps identify the important elements in each group, but does not prioritize the elements. it also does not suggest which element is most important or critical among all the identified elements. in other words, it does not indicate the weightage of the factors to determine the effect of each factor on the proposed alternatives (yuksel & dagdeviren, 2007). a swot analysis is subjective or qualitative and reflects the biases and experiences of the individuals. therefore, it is impossible to obtain objective or quantitative data relating to the issue. to improve the usefulness of the results obtained from a swot analysis, the swot framework can be restructured into a hierarchic structure and integrated and analyzed using the ahp (kurttila, pesonen, kangas, & kajanus, 2000). the ahp can provide a quantitative measure of the importance of each swot factor obtained during the study. the analysis based on the swot-ahp hybrid method has been used in various areas of study such as transportation, agriculture, tourism, ijahp article: shiwakoti, regmi/strategic analysis of the nepal electricity authority: a swot-ahp analysis based on stakeholders’ perceptions international journal of the analytic hierarchy process 8 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.944 management, manufacturing, etc. (jaroslaw & krzysztof, 2017; wickramasinghe & takano, 2010; yuksek & akin, 2006). in a recent study, the ahp-swot model was used to make an assessment of opportunities and challenges for cross-border electricity trade with bangladesh (haque, dhakal, & mostafa, 2019). in another recent study regarding the electricity sector of eu and ukraine, an ahp-based comparative analysis of electricity generating portfolios was conducted to explain which energy technology best meets the needs of the companies through the mechanism of quantitative assessment (volodymyr & pasichna, n.d.). a swot analysis and the ahp are combined to create a hybrid swot-ahp model that utilizes the advantages of both methods. the steps of swot-ahp analysis are as follows (gallego-ayala & juizo, 2011): 1. swot analysis considering the internal and external factors 2. paired comparison between the elements of each swot group 3. paired comparison between the four swot groups 4. ahp calculation to determine the priority vector for each element within the swot groups and also the relative priority for the four swot groups 5. strategy formulation from the results 3. swot analysis of the nea being the largest state-owned utility in nepal, the nea must deal with strength, weakness, opportunities, and threats factors. it is necessary to conduct a strategic analysis of the organization to fulfill the entire mandate from the government. it would be naive to develop a goal-setting strategy without considering the organization's strengths and shortcomings, as well as the competitive environment (tshombe, 2013). from a review of several works of literature related to the energy sector in nepal, such as plans and policies including periodical plans and annual budgets of the government, occasional mediumand long-term policies published by the ministry of energy, water resources and irrigation, annual reports and the recently published five-year corporate development plan of the nea, several factors were identified as internal and external factors influencing the overall operation and future strategies of the nea. these findings were used to develop the questionnaire used in this research. the following strengths, weaknesses, opportunities and threats of the nea have been referenced from the recent five-year corporate development plan of the nea (nea, 2018). strengths: (a) monopoly (single seller) and monopsony (single buyer) market of an essential service due to the vertically integrated and regulated structure of nepal’s energy sector, the nea is the single buyer of electricity so far in the country. all power purchase agreements (ppas) with electricity generators are done with the nea. similarly, the nea is the only retail supplier of electricity to consumers. all domestic and non-domestic consumers in nepal have no other alternative for purchasing electricity. (b) huge domestic demand for electricity and energy services ijahp article: shiwakoti, regmi/strategic analysis of the nepal electricity authority: a swot-ahp analysis based on stakeholders’ perceptions international journal of the analytic hierarchy process 9 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.944 nepal has very low electricity consumption. the per capita electricity consumption is below 300 kwh per annum. currently, 22% of the nepalese population is living without access to grid connected electricity. the ministry of energy, water resources and irrigation has committed to increase the per capita energy consumption to 700 mw in the next five years (nea, 2019). there is a huge amount of room for industrial and commercial growth in nepal that will induce increased demand for electricity. this is an opportunity for the nea to further expand its business in the future. (c) ability to access relatively cheaper sources of capital from government, public and international multilateral banks/donors the nea is a public entity with 100% shareholding of the government of nepal. this enables the nea to access relatively cheaper sources of capital and funds in non-commercial terms from the government and multilateral banks, such as world bank, adb, aiib, etc. (d) nationwide electricity distribution network one of the main business areas of the nea is the transmission of electricity. for this purpose, the nea has constructed a wide network of high voltage and low voltage transmission lines throughout the country. the government of nepal and the nea have made huge investments in the expansion of the existing network to cater to the increasing generation and electricity demand. (e) network of 4 million customers since it is a monopoly, the nea has a huge customer base of over 4 million customers comprising both domestic and industrial customers. the number of consumers is expected to increase every year, creating further business opportunities for the nea. (f) improved brand name and public good will with recent operational and managerial improvements and the elimination of the chronic load shedding that had been hampering the country’s economy for almost a decade, the public image of the nea has improved. the organization has also gained the confidence of the government and multilateral donor agencies. (g) human resources with strong technical skills the nea recruits its employees through a competitive process to attract highly qualified and skilled human resources. this is a strength of the organization. weaknesses: (a) insufficient and inefficient transmission and distribution network the existing transmission and distribution network operated by the nea is not sufficient to cater to the electricity demand of all domestic and industrial customers in the whole country. even though the electricity generation is sufficient, the nea is not able to supply electricity to all prospective customers because of the lack of sufficient transmission and reliable distribution network. (b) poor and unsatisfactory quality of power particularly in rural areas due to the infrastructure that is poor and old, it is difficult to provide quality and reliable electricity, mainly in rural areas with low electricity demand. (c) lack of focus on customer service and customer experience ijahp article: shiwakoti, regmi/strategic analysis of the nepal electricity authority: a swot-ahp analysis based on stakeholders’ perceptions international journal of the analytic hierarchy process 10 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.944 the nea has a considerable lack of focus on customer service and feedback on customer experience. (d) weak project management, procurement and contract management capacity past records show that most of the projects owned and executed by the nea have not been completed within the stipulated time. there are several managerial and contractual weaknesses that hinder the organization in completing the projects on time and within a prefixed budget. similarly, procurement processes take a long time and there are no procurement specialists within the organization. (e) high internal construction and operation costs the operational costs of the nea are quite high mainly due to the large number of employees (over 12,000). (f) lack of automated data collection and analysis of its operations the nea does not have state of the art it technology in all sectors of its operations. many processes are still handled manually. however, there has been considerable improvement in the use of ict in recent years. (g) mismatch between demand and supply currently, one of the main problems the organization is facing is the mismatch between the supply and demand of electricity. the energy demand in inps is around 7,318 gwh (nea, 2021), but only 60% of the energy is available from domestic generation by ipps and nea hydro plants. the nea has been importing the remaining 40% of the energy from different cross-border connections between nepal and india. the total installed capacity (including nea’s own generation and power purchased from the ipps) is around 2,000 mw, but the peak demand has been recorded to be just over 1,600 mw. (h) traditional operating and management system the nea has adopted the traditional bureaucratic management system. this is considered a weakness since the traditional approach of management causes delays in decision making and involves weak supervision and monitoring. (i) lack of energy storage capacity out of the total installed capacity of around 2,000 mw in nepal, only 92 mw are operated as a reservoir based plant. all the remaining capacity is installed either as runoff river or peaking runoff river power plants. this has severely handicapped the energy storage capacity of the nation. (j) limited transmission capacity with neighboring countries although nepal has a huge prospect of energy trade with neighboring countries, primarily india and bangladesh, the nea has not been able to conduct crossborder energy banking or trading in real commercial terms. this is mainly because of the lack of interconnection infrastructure with india and bangladesh. (k) rural electrification expense to expand network into remote, distant and disperse locations many rural and remote areas of nepal have poor road access and low electricity demand. due to the unfavorable geographic conditions, difficult terrain and low power demand of such places, it is cost intensive for the nea to provide infrastructure for transmission and distribution to those places. ijahp article: shiwakoti, regmi/strategic analysis of the nepal electricity authority: a swot-ahp analysis based on stakeholders’ perceptions international journal of the analytic hierarchy process 11 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.944 opportunities: (a) long-term growth for demand of electricity there is a huge prospect for an increase in electricity demand in nepal, which is expected to grow rapidly with the increase in commercial and industrial activities in the future. (b) reduction of cost of energy through trade and economies of scale as the power system expands the nea has an opportunity to reduce the cost of electricity generation through economies of scale. the new technologies in electricity generation help reduce the cost of generation as the power sector expands. (c) export and trade of power as mentioned in the previous section, nepal has a huge prospect of energy trade with its neighboring countries. this is an opportunity for the nea to expand its business across the national borders and contribute to decreasing nepal’s trade deficit. (d) energy banking to meet deficit demand in dry season given the seasonal variation, power plants in nepal generate electricity above the demand in monsoon season (may-october); however, the generation of energy is reduced to almost a third of that in the dry season (november to april). thus, the energy supply is short of the demand in the dry season. this provides an opportunity for nepal to conduct energy banking with india which experiences an energy surplus in the dry season due to the easy availability of coal and deficit in the monsoon season. (e) improve utility efficiency through automation, digitization and use of centrally integrated software the nea has an opportunity to integrate the processes and data related to all of its business units by implementing efficient enterprise resources planning software. recently, the organization has gained considerable success in automation and digitization of its processes by the implementation of integrated software related to human resources, bookkeeping, revenue collection, etc. (f) improve profits and reduce cost of supply by decreasing aggregate technical and commercial losses due to recent improvement in transmission and the distribution technology and reduction of technical and non-technical losses, the nea has the opportunity to improve its profits and reduce the cost of supply. (g) make energy system more efficient through demand side management tools there is an opportunity for the nea to adopt demand side management tools aimed at increasing energy efficiency, which is an important sector of priority for the organization. (h) expand market by adding 2 million customers due to the increasing commercial activities and prospects for industrialization of the country, the nea has the opportunity to increase its customer base by at least 2 million in the near future. (i) increased operational and financial efficiency through restructuring ijahp article: shiwakoti, regmi/strategic analysis of the nepal electricity authority: a swot-ahp analysis based on stakeholders’ perceptions international journal of the analytic hierarchy process 12 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.944 the government has felt the need to restructure the current vertically integrated structure of the nea and deregulate the energy market in nepal (karmacharya, 2012). there is an opportunity for the nea to increase its operational and financial efficiency in the process of and after complete restructuring. (j) expand and upgrade transmission and distribution operations the nea has the opportunity to expand its transmission and distribution network to new load centers and upgrade the existing transmission and distribution infrastructure. threats: (a) high cost of capital to finance capital expenditure plans hydropower and other energy projects (solar energy, transmission lines, etc.) are highly capital intensive projects and require funding from different national and international sources. the financial feasibility of any project is always a major concern for both debt and equity investors. the volatile nature of the cost of capital and the intrinsically high investment requirement of energy projects poses a threat to the nea in arranging funds to finance such projects. (b) adverse effects of climate change and extreme weather patterns on hydrology and structures due to the rapidly increasing effects of climate change and global warming that cause unpredicted climate patterns and flow patterns in rivers, hydropower projects with which the nea has signed a ppa may not be able to generate the contracted quantum of energy, thus hampering the financial condition of the nea. (c) new regulatory regime and delays in tariff reviews in nepal, electricity tariff review is done annually by the electricity regulatory commission on the basis of recommendations from the nea. any delay caused by the electricity regulatory commission (erc) in reviewing the tariff will cause problems in the nea’s financial operations. similarly, adverse changes in the regulatory regime, which generally happens in nepal, also pose a threat to the nea. (d) distribution and self-generation by customers reduces the quantum and increases variability of demand captive generation of energy by using alternative sources such as diesel generators in the domestic as well as industrial sector makes the forecasting of energy demand difficult and also reduces the quantum of energy that the nea can sell to its consumers. (e) economic slowdown or failure of economy to grow as projected in the demand forecast will result in excess capacity nepal is a developing country with a very low level of industrial activities. there are no large manufacturing plants that can increase the demand of electricity. this failure of growth in the economy as predicted and desired by the government has caused the electricity demand to be less than the actual generation by plants under operation. this has resulted in excess supply and spillage of valuable energy. (f) inability to absorb all new generation that the nea has signed take-or-pay contracts with ipps ijahp article: shiwakoti, regmi/strategic analysis of the nepal electricity authority: a swot-ahp analysis based on stakeholders’ perceptions international journal of the analytic hierarchy process 13 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.944 due to less demand of electricity as compared to generation, and a lack of sufficient transmission and distribution infrastructure, the nea is not in a position to absorb the generation of all the projects with which it has signed a take-or-pay ppa. this will result in the nea continuing to make payments to the ipps without absorbing the power generated by them. (g) new levies and taxes imposed by local, provincial and federal governments the unchecked and unsystematic levies and taxes imposed by the local, provincial and federal governments under various headings are the cause of increasing expenses to energy projects and the nea overall. (h) potential complications due to federal restructuring nepal has recently transitioned to the federal system of government from the centrally controlled unified form. the federal system is still in the process of complete institutionalization. this causes unforeseen complications in the operations of the nea, which is still functioning as a centrally controlled entity of the government. (i) adverse movement in the dollar and other foreign currencies any adverse movement in foreign currencies will affect the financial condition of the nea, especially with regard to international procurement and foreign currency denominated ppas. (j) inability to engage in trade of electricity with neighboring countries due to political and economic reasons. apart from the lack of sufficient cross border transmission infrastructure, the nea’s plan to engage in electricity trading with neighboring countries is hampered by the geo-political complications in the region. for instance, india has not shown sufficient political willpower to purchase electricity generated in nepal and is also reluctant to provide land to connect nepal and bangladesh. (k) theft and leakage of electricity and collection losses pilferage and leakage of electricity and collection losses is another major threat to the nea. the annual losses from this amount to 15-20% of the actual generation of energy. (l) delay in construction of projects due to social and/or legal issues such as resettlement, right of way and local shares the nea faces social issues related to resettlement and compensation during construction of generation and transmission line projects. this causes unwarranted delays in project completion. lengthy and difficult provisions related to forest clearance, lack of timely site availability, and high expectations of local people for the project are the main challenges faced by the nea. (regmi & mandal, 2020). it is a major threat for the organization which hampers its overall business and functions. 4. methodology and application the purpose of utilizing the ahp with a swot analysis in this study is to qualify the swot factors and evaluate their intensities. in this study, a swot analysis and the ahp were combined to create a hybrid swot-ahp model that utilizes the ijahp article: shiwakoti, regmi/strategic analysis of the nepal electricity authority: a swot-ahp analysis based on stakeholders’ perceptions international journal of the analytic hierarchy process 14 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.944 advantages of both methods of research. the steps of the swot-ahp analysis used in this study are as follows: 1. swot analysis of the nea by listing the important internal and external factors for strategic planning (nea, 2018) 2. paired comparison between the elements of each swot group 3. paired comparison between the four swot groups 4. ahp calculation: a. computation of the local priority vector for each element within the swot groups b. computation of group relative priority for the 4 swot groups as a single matrix c. calculation of overall priority of each element (local priority multiplied by the group priority) 5. strategy formulation from the results the ahp-swot combination is carried out in five stages as shown in figure 1 (fadim yavuz, 2014). figure 1 stages of the study in the first stage, a review of the literature relevant to the nea, its business operations and factors influencing its current and future strategies was prepared. based on the review, various factors related to the internal and external environment of the organization were identified. these factors are positive influencers (strengths and opportunities) and negative influencers (opportunities and threats) (explained in detail in section 3). in the second stage, a swot analysis of the nea was done by analyzing the elements identified in the previous stage. table 6 shows the swot matrix or model of the nea used in this study. the elements in each group have been briefly described. 1. swot analysis 2. pairwise comparison of swot factors 3. determining the relative priorities of s, w, o and t 4. evaluation of the strategy alternatives for each swot factors 5. general priority calculations for the strategy alternatives ijahp article: shiwakoti, regmi/strategic analysis of the nepal electricity authority: a swot-ahp analysis based on stakeholders’ perceptions international journal of the analytic hierarchy process 15 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.944 table 6 swot analysis of the nea strengths weaknesses s1: monopoly (single seller) and monopsony (single buyer) market of an essential service s2: huge domestic demand for electricity and energy services s3: ability to access relatively cheaper sources of capital from government, public and international multilateral banks/ donors s4: nationwide electricity distribution network s5: network of 4 million customers s6: improved brand name and public good will s7: human resources with strong technical skills w1: insufficient and inefficient transmission and distribution network w2: poor and unsatisfactory quality of power, particularly in rural areas w3: lack of focus on customer service and customer experience w4: weak project management, procurement and contract management capacity w5: high internal construction and operation costs w6: lack of automated data collection and analysis of its operations w7: mismatch between demand and supply w8: traditional operating and management system w9: lack of energy storage capacity w10: limited transmission interconnection capacity with neighboring countries w11: rural electrification expense to expand network into remote, distant and disperse locations opportunities threats o1: long-term growth for demand of electricity o2: reduce cost of energy through trade and economies of scale as the power system expands o3: export and trade of power o4: energy banking to meet deficit demand in dry season o5: improve utility efficiency through automation, digitization and use of centrally integrated software o6: improve profits and reduce cost of supply via decrease in at&c losses o7: make energy system more efficient through demand side management tools o8: expand market by adding 2 million new customers o9: increased operational and financial efficiency through restructuring o10: expand and upgrade transmission and distribution operations t1: high cost of capital to finance capital expenditure plans t2: adverse effects of climate change and extreme weather patterns on hydrology and structures t3: new regulatory regime and delays in tariff reviews t4: distribution and self-generation by customers reduces quantum and increases variability of demand t5: economic slowdown or failure of economy to grow as projected in the demand forecast will result in excess capacity t6: inability to absorb all new generation that nea has signed takeor-pay contracts with ipps t7: new levies and taxes imposed by local, provincial and federal governments t8: potential complications due to federal restructuring t9: adverse movement in the dollar ijahp article: shiwakoti, regmi/strategic analysis of the nepal electricity authority: a swot-ahp analysis based on stakeholders’ perceptions international journal of the analytic hierarchy process 16 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.944 and other foreign currencies t10: inability to engage in trade of electricity with neighboring countries due to political and economic reasons t11: theft and leakage of electricity and collection losses t12: delay in construction of projects due to social and/or legal issues such as resettlement, right of way and local shares in the third stage of the study, a swot-ahp model was developed. the model used in this study is depicted in figure 2. figure 2 hierarchical structure of the swot matrix the hierarchical ahp structure is divided into 3 levels. the first level is the goal of the study which is to identify the major organizational strategies related to the internal and external environment of the nea that it is recommended they follow. the second level comprises the criteria or factors under study. for the current study, these factors are the swot groups. the third level is the sub-criteria or the elements within each swot group. the ahp was applied to the swot matrix. for this purpose, a set of questions was developed. the questionnaire was distributed to 61 experts working in the energy sector in nepal who are in decision making and managerial positions, as shown in table 7. the questionnaire was completed by 38 respondents, with seven incomplete responses. these seven responses were rejected and only the 31 complete responses were considered for further analysis. the respondents include representatives of nea employees, employees of the independent power producers of nepal (ipps), the ministry of energy, water resources and irrigation (moewri) and donor agencies. the experts were asked to conduct pairwise comparisons of the swot factors using saaty’s comparison scale. it was also used to make pairwise ijahp article: shiwakoti, regmi/strategic analysis of the nepal electricity authority: a swot-ahp analysis based on stakeholders’ perceptions international journal of the analytic hierarchy process 17 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.944 comparison between the four swot groups. after collecting the responses from the experts, all data were input in a ms-excel spreadsheet for further analysis. these pairwise calculations were then analyzed to obtain the following scores (dhakal, karki, & shrestha, 2019) :  relative importance/weight of swot groups  local priority scores (the relative weights of factors within the same swot group)  global priority scores (the overall relative weights of a factor considering the weights of all four swot groups)  strategy formulation the respondents were identified through purposive sampling. the professionals having significant years of relevant experience were selected for the questionnaire survey. table 7 list of respondents for questionnaire survey category participants response received response percentage rejected responses used for further analysis nea 30 21 70 3 18 moewri 10 8 80 2 6 ipps 15 7 46.7 2 5 donors 6 2 33.3 0 2 total 61 38 62.3 7 31 5. results and discussion the results of the pairwise comparison and priority vectors (weightage) of all the elements in each swot group as well as the calculation of the reliability index are presented and discussed in tables 9-20. similarly, the comparison result along with the calculation of priority vectors (weightage) of the swot factors and the reliability index are depicted in tables 21-24. table 6 shows the calculations for validity and reliability in the survey. cronbach's coefficient alpha, “α” was used to test the reliability of the survey as shown in table 8. the acceptable lower limit for cronbach's alpha is usually considered to be 0.7, although values as low as 0.6 are sometimes acceptable for exploratory research (hair et al., 1998). table 8 validity and reliability of factors group factors n of items cronbach’s alpha (𝞪) strength 21 0.850 weakness 55 0.900 opportunity 45 0.940 threats 66 0.960 cronbach’s alpha (𝞪) 0.925 ijahp article: shiwakoti, regmi/strategic analysis of the nepal electricity authority: a swot-ahp analysis based on stakeholders’ perceptions international journal of the analytic hierarchy process 18 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.944 since the average value of coefficient of alpha is more than 0.7, i.e. 0.925, our survey is acceptable and the overall reliability of the questionnaire is found to be good. table 9 pairwise comparison matrix of the elements within the strength criteria factor s1 s2 s3 s4 s5 s6 s7 s1 1.00 2.68 1.64 2.50 2.57 2.58 3.24 s2 0.37 1.00 0.31 1.96 2.50 3.01 3.08 s3 0.61 3.25 1.00 2.72 2.86 2.47 3.69 s4 0.40 0.51 0.37 1.00 3.15 3.55 3.90 s5 0.39 0.40 0.35 0.32 1.00 3.26 3.06 s6 0.39 0.33 0.40 0.28 0.31 1.00 2.43 s7 0.31 0.33 0.27 0.26 0.33 0.41 1.00 sum 3.47 8.50 4.34 9.04 12.72 16.28 20.39 table 10 normalized pairwise comparison matrix of the elements within the strength criteria factor s1 s2 s3 s4 s5 s6 s7 priority vector (pv) rank s1 0.29 0.32 0.38 0.28 0.20 0.16 0.16 0.254 1 s2 0.11 0.12 0.07 0.22 0.20 0.18 0.15 0.149 4 s3 0.18 0.38 0.23 0.30 0.23 0.15 0.18 0.236 2 s4 0.12 0.06 0.08 0.11 0.25 0.22 0.19 0.147 3 s5 0.11 0.05 0.08 0.04 0.08 0.20 0.15 0.101 5 s6 0.11 0.04 0.09 0.03 0.02 0.06 0.12 0.069 6 s7 0.09 0.04 0.06 0.03 0.03 0.03 0.05 0.045 7 sum 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 the results from table 10 show that sub-criteria s1 (monopoly (single seller) and monopsony (single buyer) market of an essential service) account for over 25% of all the strength elements. the respondents perceive this element to be the most important element among the strength elements. ijahp article: shiwakoti, regmi/strategic analysis of the nepal electricity authority: a swot-ahp analysis based on stakeholders’ perceptions international journal of the analytic hierarchy process 19 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.944 table 11 calculation of maximum eigen value (λ max) and consistency ratio for the strength criteria factor s1 (1) s2 (2) s3 (3) s4 (4) s5 (5) s6 (6) s7 (7) sum (8) sum/pv (9) s1 0.25 0.40 0.39 0.37 0.26 0.18 0.15 1.99 7.83 s2 0.09 0.15 0.07 0.29 0.25 0.21 0.14 1.20 8.05 s3 0.16 0.49 0.24 0.40 0.29 0.17 0.17 1.90 8.07 s4 0.10 0.08 0.09 0.15 0.32 0.24 0.18 1.15 7.82 s5 0.10 0.06 0.08 0.05 0.10 0.22 0.14 0.75 7.46 s6 0.10 0.05 0.10 0.04 0.03 0.07 0.11 0.49 7.21 s7 0.08 0.05 0.06 0.04 0.03 0.03 0.05 0.33 7.37 λ max = average of column (9) = 7.7 consistency index ci = (λ max -n)/n-1 = 0.115 for n=7 reliability index (ri) (for n=7) =1.32 consistency ratio (cr) = 𝐶𝐼/𝑅𝐼 = 0.087< 0.1 table 12 pairwise comparison matrix of the elements within the weakness criteria factor w1 w2 w3 w4 w5 w6 w7 w8 w9 w10 w11 w1 1 2.06 2.43 2.18 2.41 2.12 2.85 2.19 2.57 1.04 2.38 w2 0.48 1 2.38 0.94 1.21 1.09 1.63 2.24 2.06 2.44 0.96 w3 0.41 0.42 1 0.46 1.17 1.84 2.16 1.35 1.52 1.13 1.68 w4 0.46 1.06 2.16 1 2.18 2.08 2.24 2.33 2.43 2.31 2.86 w5 0.42 0.83 0.85 0.46 1 2.36 1.99 1.27 1.42 2.29 1.51 w6 0.47 0.92 0.54 0.48 0.42 1 1.87 1.76 1.63 2.24 2.92 w7 0.35 0.61 0.46 0.45 0.50 0.53 1 2.84 3.34 3.41 3.08 w8 0.46 0.45 0.74 0.43 0.79 0.57 0.35 1.00 2.80 3.09 3.55 w9 0.39 0.48 0.66 0.41 0.70 0.62 0.30 0.36 1 2.87 3.18 w10 0.97 0.41 0.88 0.43 0.44 0.45 0.29 0.32 0.35 1.00 1.55 w11 0.42 1.04 0.60 0.35 0.66 0.34 0.33 0.28 0.31 0.64 1 sum 5.82 9.29 12.71 7.60 11.49 13.00 15.00 15.95 19.44 22.47 24.66 ijahp article: shiwakoti, regmi/strategic analysis of the nepal electricity authority: a swot-ahp analysis based on stakeholders’ perceptions international journal of the analytic hierarchy process 20 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.944 table 13 normalized pairwise comparison matrix of the elements within the weakness criteria factor w1 w2 w3 w4 w5 w6 w7 w8 w9 w10 w11 priority vector (pv) rank w1 0.17 0.22 0.19 0.29 0.21 0.16 0.19 0.14 0.13 0.05 0.10 0.168 1 w2 0.08 0.11 0.19 0.12 0.11 0.08 0.11 0.14 0.11 0.11 0.04 0.109 3 w3 0.07 0.05 0.08 0.06 0.10 0.14 0.14 0.08 0.08 0.05 0.07 0.084 7 w4 0.08 0.11 0.17 0.13 0.19 0.16 0.15 0.15 0.12 0.10 0.12 0.135 2 w5 0.07 0.09 0.07 0.06 0.09 0.18 0.13 0.08 0.07 0.10 0.06 0.091 4 w6 0.08 0.10 0.04 0.06 0.04 0.08 0.12 0.11 0.08 0.10 0.12 0.085 6 w7 0.06 0.07 0.04 0.06 0.04 0.04 0.07 0.18 0.17 0.15 0.12 0.091 5 w8 0.08 0.05 0.06 0.06 0.07 0.04 0.02 0.06 0.14 0.14 0.14 0.079 8 w9 0.07 0.05 0.05 0.05 0.06 0.05 0.02 0.02 0.05 0.13 0.13 0.062 9 w10 0.17 0.04 0.07 0.06 0.04 0.03 0.02 0.02 0.02 0.04 0.06 0.052 10 w11 0.07 0.11 0.05 0.05 0.06 0.03 0.02 0.02 0.02 0.03 0.04 0.044 11 sum 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 it can be stipulated that sub-criteria w1 (insufficient and inefficient transmission and distribution network) account for about 17% of the total weakness of the nea. the respondents perceive this element to be the biggest weakness of the nea. table 14 calculation of maximum eigen value (λ max) and consistency ratio for the weakness criteria factor w1 (1) w2 (2) w3 (3) w4 (4) w5 (5) w6 (6) w7 (7) w8 (8) w9 (9) w10 (10) w11 (11) sum (12) sum/pv (13) thw1 0.17 0.22 0.20 0.29 0.26 0.18 0.26 0.17 0.16 0.05 0.11 2.08 12.40 w2 0.08 0.11 0.20 0.13 0.13 0.09 0.15 0.18 0.13 0.13 0.04 1.36 12.56 w3 0.07 0.05 0.08 0.06 0.13 0.16 0.20 0.11 0.09 0.06 0.07 1.08 12.79 w4 0.08 0.12 0.18 0.13 0.24 0.18 0.20 0.18 0.15 0.12 0.13 1.71 12.66 w5 0.07 0.09 0.07 0.06 0.11 0.20 0.18 0.10 0.09 0.12 0.07 1.16 12.67 w6 0.08 0.10 0.05 0.06 0.05 0.09 0.17 0.14 0.10 0.12 0.13 1.08 12.64 w7 0.06 0.07 0.04 0.06 0.05 0.05 0.09 0.22 0.21 0.18 0.14 1.16 12.77 w8 0.08 0.05 0.06 0.06 0.09 0.05 0.03 0.08 0.17 0.16 0.16 0.98 12.48 w9 0.07 0.05 0.06 0.06 0.08 0.05 0.03 0.03 0.06 0.15 0.14 0.76 12.30 w10 0.16 0.04 0.07 0.06 0.05 0.04 0.03 0.03 0.02 0.05 0.07 0.62 11.87 w11 0.07 0.11 0.05 0.05 0.07 0.03 0.03 0.02 0.02 0.03 0.04 0.53 12.00 λ max = average of column (13) = 12.47 consistency index ci = (λ max -n)/n-1 = 0.147 for n=11 reliability index (ri) (for n=11) =1.514 consistency ratio (cr) = 𝐶𝐼/𝑅𝐼 = 0.097< 0.1 ijahp article: shiwakoti, regmi/strategic analysis of the nepal electricity authority: a swot-ahp analysis based on stakeholders’ perceptions international journal of the analytic hierarchy process 21 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.944 table 15 pairwise comparison matrix of the elements within the opportunity criteria factor o1 o2 o3 o4 o5 o6 o7 o8 o9 o10 o1 1 3.45 3.74 3.25 3.28 2.73 2.79 3.32 1.18 3.00 o2 0.29 1 3.94 0.56 0.97 1.53 1.91 0.99 2.44 2.87 o3 0.27 0.25 1 2.44 2.87 2.66 2.71 2.58 2.55 2.84 o4 0.31 1.79 0.41 1 2.62 2.67 2.91 3.05 2.70 2.64 o5 0.31 1.03 0.35 0.38 1 2.34 2.62 2.68 2.76 2.91 o6 0.30 0.66 0.38 0.37 0.43 1 2.28 2.98 3.26 2.68 o7 0.37 0.52 0.37 0.34 0.38 0.44 1 2.14 2.92 2.66 o8 0.36 1.01 0.39 0.33 0.37 0.34 0.47 1 2.12 2.24 o9 0.30 0.41 0.39 0.37 0.36 0.31 0.47 0.47 1 2.73 o10 0.84 0.35 0.39 0.38 0.34 0.37 0.38 0.45 0.37 1 sum 4.35 10.47 11.36 9.42 12.62 14.39 17.54 19.66 21.3 25.57 table 16 normalized pairwise comparison matrix of the elements within the opportunity criteria factor o1 o2 o3 o4 o5 o6 o7 o8 o9 o10 priority vector (pv) rank o1 0.23 0.33 0.33 0.35 0.26 0.19 0.16 0.17 0.06 0.12 0.219 1 o2 0.07 0.10 0.35 0.06 0.08 0.11 0.11 0.05 0.11 0.11 0.114 4 o3 0.06 0.02 0.09 0.26 0.23 0.18 0.15 0.13 0.12 0.11 0.136 2 o4 0.07 0.17 0.04 0.11 0.21 0.19 0.17 0.15 0.13 0.10 0.133 3 o5 0.07 0.10 0.03 0.04 0.08 0.16 0.15 0.14 0.13 0.11 0.101 5 o6 0.07 0.06 0.03 0.04 0.03 0.07 0.13 0.15 0.15 0.10 0.085 6 o7 0.08 0.05 0.03 0.04 0.03 0.03 0.06 0.11 0.14 0.10 0.067 7 o8 0.08 0.10 0.03 0.03 0.03 0.02 0.03 0.05 0.10 0.09 0.056 8 o9 0.07 0.04 0.03 0.04 0.03 0.02 0.03 0.02 0.05 0.11 0.044 10 o10 0.19 0.03 0.03 0.04 0.03 0.03 0.02 0.02 0.02 0.04 0.046 9 sum 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 it can be interpreted that sub-criteria o1 (long-term growth for demand of electricity) account for about 22% importance of all the opportunities for the nea. the respondents perceive this to be the most important opportunity for the nea among all the identified opportunities. ijahp article: shiwakoti, regmi/strategic analysis of the nepal electricity authority: a swot-ahp analysis based on stakeholders’ perceptions international journal of the analytic hierarchy process 22 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.944 table 17 calculation of maximum eigen value (λ max) and consistency ratio for the opportunity criteria factor o1 (1) o2 (2) o3 (3) o4 (4) o5 (5) o6 (6) o7 (7) o8 (8) o9 (9) o10 (10) sum (11) sum/pv (12) o1 0.22 0.38 0.50 0.43 0.33 0.23 0.19 0.19 0.13 0.14 2.60 11.90 o2 0.07 0.11 0.45 0.41 0.28 0.25 0.22 0.19 0.11 0.13 2.08 18.30 o3 0.06 0.03 0.14 0.32 0.29 0.23 0.18 0.15 0.11 0.13 1.51 11.09 o4 0.07 0.04 0.06 0.13 0.27 0.23 0.20 0.17 0.12 0.12 1.27 9.56 o5 0.07 0.04 0.05 0.05 0.10 0.20 0.18 0.15 0.12 0.13 0.95 14.22 o6 0.07 0.04 0.05 0.05 0.04 0.08 0.15 0.17 0.14 0.12 0.80 14.11 o7 0.08 0.03 0.05 0.05 0.04 0.04 0.07 0.12 0.13 0.12 0.60 8.96 o8 0.08 0.03 0.05 0.04 0.04 0.03 0.03 0.06 0.09 0.10 0.46 8.07 o9 0.07 0.05 0.05 0.05 0.04 0.03 0.03 0.03 0.04 0.12 0.38 8.68 o10 0.07 0.04 0.05 0.05 0.03 0.03 0.03 0.03 0.02 0.05 0.35 7.65 λ max = average of column (12) = 11.25 consistency index ci = (λ max -n)/n-1 = 0.139 for n=10 reliability index (ri) (for n=10) =1.49 consistency ratio (cr) = 𝐶𝐼/𝑅𝐼 = 0.094< 0.1 table 18 pairwise comparison matrix of the elements within the threat criteria factor t1 t2 t3 t4 t5 t6 t7 t8 t9 t10 t11 t12 t1 1 2.13 0.45 1.25 1.02 0.24 1.32 2.70 1.77 0.75 2.76 0.30 t2 0.47 1.00 0.55 0.74 0.53 1.59 2.48 2.32 2.25 0.53 2.20 2.15 t3 2.22 1.80 1 3.27 1.00 1.04 2.72 2.15 2.82 1.37 2.61 2.49 t4 0.80 1.35 0.31 1 2.32 0.77 2.21 2.20 2.40 1.85 2.56 1.78 t5 0.98 1.87 1.00 0.43 1 2.52 2.92 2.66 2.58 2.29 2.37 1.69 t6 4.11 0.63 0.96 1.29 0.40 1 3.31 3.04 3.29 4.35 3.27 3.97 t7 0.76 0.40 0.37 0.45 0.34 0.30 1 1.33 1.50 1.42 0.46 1.81 t8 0.37 0.43 0.47 0.45 0.38 0.33 0.75 1 1.20 1.52 1.26 2.69 t9 0.56 0.45 0.35 0.42 0.39 0.30 0.67 0.83 1 2.13 0.57 2.57 t10 1.33 1.89 0.73 0.54 0.44 0.23 0.70 0.66 0.47 1.00 1.21 1.58 t11 0.36 0.45 0.38 0.39 0.42 0.31 2.15 0.79 1.76 0.82 1.00 1.18 t12 3.33 0.47 0.40 0.56 0.59 0.25 0.55 0.37 0.39 0.63 0.85 1 sum 16.30 12.87 6.98 10.81 8.83 8.89 20.79 20.05 21.45 18.67 21.13 23.19 ijahp article: shiwakoti, regmi/strategic analysis of the nepal electricity authority: a swot-ahp analysis based on stakeholders’ perceptions international journal of the analytic hierarchy process 23 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.944 table 19 normalized pairwise comparison matrix of the elements within the threat criteria factor t1 t2 t3 t4 t5 t6 t7 t8 t9 t10 t11 t12 priority vector (pv) r t1 0.06 0.17 0.06 0.12 0.12 0.03 0.06 0.13 0.08 0.04 0.13 0.01 0.085 6 t2 0.03 0.08 0.08 0.07 0.06 0.18 0.12 0.12 0.10 0.03 0.10 0.09 0.088 5 t3 0.14 0.14 0.14 0.30 0.11 0.12 0.13 0.11 0.13 0.07 0.12 0.11 0.136 2 t4 0.05 0.10 0.04 0.09 0.26 0.09 0.11 0.11 0.11 0.10 0.12 0.08 0.105 4 t5 0.06 0.15 0.14 0.04 0.11 0.28 0.14 0.13 0.12 0.12 0.11 0.07 0.124 3 t6 0.25 0.05 0.14 0.12 0.04 0.11 0.16 0.15 0.15 0.23 0.15 0.17 0.145 1 t7 0.05 0.03 0.05 0.04 0.04 0.03 0.05 0.07 0.07 0.08 0.02 0.08 0.050 11 t8 0.02 0.03 0.07 0.04 0.04 0.04 0.04 0.05 0.06 0.08 0.06 0.12 0.054 8 t9 0.03 0.03 0.05 0.04 0.04 0.03 0.03 0.04 0.05 0.11 0.03 0.11 0.051 10 t10 0.08 0.15 0.10 0.05 0.05 0.03 0.03 0.03 0.02 0.05 0.06 0.07 0.061 7 t11 0.02 0.04 0.05 0.04 0.05 0.03 0.10 0.04 0.08 0.04 0.05 0.05 0.050 11 t12 0.20 0.04 0.06 0.05 0.07 0.03 0.03 0.02 0.02 0.03 0.04 0.04 0.052 9 sum 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1 it can be interpreted that the sub-criteria t6 (inability to absorb all new generation that the nea has signed take-or-pay contracts with ipps) account for about 15% of the total threats of the nea. the respondents perceive this to be the biggest threat for the nea. table 20 calculation of maximum eigen value (λ max) and consistency ratio for the threat criteria factor t1 (1) t2 (2) t3 (3) t4 (4) t5 (5) t6 (6) t7 (7) t8 (8) t9 (9) t10 (10) t11 (11) t12 (12) sum (13) sum/pv (14) t1 0.08 0.19 0.06 0.13 0.13 0.04 0.07 0.14 0.09 0.05 0.14 0.02 1.13 13.33 t2 0.04 0.09 0.08 0.08 0.07 0.23 0.13 0.12 0.11 0.03 0.11 0.11 1.20 13.55 t3 0.19 0.16 0.14 0.35 0.12 0.15 0.14 0.12 0.14 0.08 0.13 0.13 1.84 13.58 t4 0.07 0.12 0.04 0.11 0.29 0.11 0.11 0.12 0.12 0.11 0.13 0.09 1.42 13.44 t5 0.08 0.17 0.14 0.05 0.12 0.36 0.15 0.14 0.13 0.14 0.12 0.09 1.68 13.59 t6 0.35 0.06 0.13 0.14 0.05 0.14 0.17 0.16 0.17 0.26 0.16 0.21 1.99 13.76 t7 0.06 0.04 0.05 0.05 0.04 0.04 0.05 0.07 0.08 0.09 0.02 0.09 0.68 13.57 t8 0.03 0.04 0.06 0.05 0.05 0.05 0.04 0.05 0.06 0.09 0.06 0.14 0.72 14.32 t9 0.05 0.04 0.05 0.04 0.05 0.04 0.03 0.04 0.05 0.13 0.03 0.13 0.69 12.88 t10 0.11 0.17 0.10 0.06 0.05 0.03 0.04 0.04 0.02 0.06 0.06 0.08 0.82 16.20 t11 0.03 0.04 0.05 0.04 0.05 0.04 0.11 0.04 0.09 0.05 0.05 0.06 0.66 10.94 t12 0.28 0.04 0.05 0.06 0.07 0.04 0.03 0.02 0.02 0.04 0.04 0.05 0.75 14.99 λ max = average of column (14) = 13.68 consistency index ci = (λ max -n)/n-1 = 0.153 for n=12 reliability index (ri) (for n=12) =1.5365 consistency ratio (cr) = 𝐶𝐼/𝑅𝐼 = 0.099< 0.1 ijahp article: shiwakoti, regmi/strategic analysis of the nepal electricity authority: a swot-ahp analysis based on stakeholders’ perceptions international journal of the analytic hierarchy process 24 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.944 table 21 pairwise comparison matrix of the swot factors factor strengths (s) weaknesses (w) opportunities (o) threats (t) strengths (s) 1.00 2.00 1.06 0.63 weaknesses (w) 0.50 1.00 1.01 1.67 opportunities (o) 0.94 0.99 1.00 0.99 threats (t) 1.59 0.60 1.01 1.00 sum 4.03 4.59 4.08 4.29 table 22 normalized pairwise comparison matrix of the swot factors factor s w o t group priority vector (pv) rank strengths (s) 0.25 0.44 0.26 0.15 0.273 1 weaknesses (w) 0.12 0.22 0.25 0.39 0.245 3 opportunities (o) 0.23 0.22 0.24 0.23 0.231 4 threats (t) 0.39 0.13 0.25 0.23 0.251 2 sum 1.00 1.00 1.00 1.00 1.00 the findings show the following ranking of each swot group priority: strengths (group weight 27.3%), opportunities (24.5%), weaknesses (23.1%) and threats (25.1%). the results indicate that the 4 factors (strength, weakness, opportunities and threats) carry almost equal weightage with regards to future strategy formulation for the nea. table 23 calculation of maximum eigen value (λ max) and consistency ratio for the swot factors factor s (1) w (2) o (3) t (4) sum (5) sum/pv (6) strengths (s) 0.27 0.49 0.25 0.16 1.17 4.27 weaknesses (w) 0.14 0.24 0.23 0.42 1.03 4.23 opportunities (o) 0.26 0.24 0.23 0.25 0.98 4.23 threats (t) 0.43 0.15 0.23 0.25 1.06 4.24 λ max = average of column (6) = 4.24 consistency index ci = (λ max -n)/n-1 = 0.089 for n=4 reliability index (ri) (for n=4) =0.9 consistency ratio cr= 0.13/1.32=0.098 < 0.1 finally, the overall priority scores of the swot factors are calculated as shown in table 24. ijahp article: shiwakoti, regmi/strategic analysis of the nepal electricity authority: a swot-ahp analysis based on stakeholders’ perceptions international journal of the analytic hierarchy process 25 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.944 table 24 calculation of local and global priority scores of the swot factors swot group group priority swot factors factor priority within the group (local priority) overall (global) priority of factor strengths 0.273 monopoly (single seller) and monopsony (single buyer) market of an essential service (s1) 0.254 0.069 huge domestic demand for electricity and energy services (s2) 0.149 0.041 ability to access relatively cheaper sources of capital from government, public and international multilateral banks/ donors (s3) 0.236 0.064 nationwide electricity distribution network (s4) 0.147 0.040 network of 4 million customers (s5) 0.101 0.028 improved brand name and public good will (s6) 0.069 0.019 human resources with strong technical skills (s7) 0.045 0.012 weakness 0.245 insufficient and inefficient transmission and distribution network (w1) 0.168 0.041 poor and unsatisfactory quality of power, particularly in rural areas (w2) 0.109 0.027 lack of focus on customer service and customer experience (w3) 0.084 0.021 weak project management, procurement and contract management capacity (w4) 0.135 0.033 high internal construction and operation costs (w5) 0.091 0.022 lack of automated data collection and analysis of its operations (w6) 0.085 0.021 mismatch between demand and supply (w7) 0.091 0.022 traditional operating and management system (w8) 0.079 0.019 lack of energy storage capacity (w9) 0.062 0.015 ijahp article: shiwakoti, regmi/strategic analysis of the nepal electricity authority: a swot-ahp analysis based on stakeholders’ perceptions international journal of the analytic hierarchy process 26 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.944 limited transmission interconnection capacity with neighboring countries (w10) 0.052 0.013 rural electrification expense to expand network into remote, distant and disperse locations (w11) 0.044 0.011 opportunit ies 0.231 long-term growth for demand of electricity (o1) 0.219 0.051 reduce cost of energy through trade and economies of scale as the power system expands (o2) 0.114 0.026 export and trade of power (o3) 0.136 0.031 energy banking to meet deficit demand in dry season (o4) 0.133 0.031 improve utility efficiency through automation, digitization and use of centrally integrated software (o5) 0.101 0.023 improve profits and reduce cost of supply by decreasing at&c losses (o6) 0.085 0.020 make energy system more efficient through demand side management tools (o7) 0.067 0.015 expand market by adding 2 million new customers (o8) 0.056 0.013 increased operational and financial efficiency through restructuring (o9) 0.044 0.010 expand and upgrade transmission and distribution operations (o10) 0.046 0.011 threats 0.251 high cost of capital to finance capital expenditure plans (t1) 0.085 0.021 adverse effects of climate change and extreme weather patterns on hydrology and structures (t2) 0.088 0.022 new regulatory regime and delays in tariff reviews (t3) 0.136 0.034 distribution and self-generation by customers reduces quantum and increases variability of demand (t4) 0.105 0.026 economic slowdown or failure of economy to grow as projected in the demand forecast will result in excess capacity (t5) 0.124 0.031 inability to absorb all new generation that the nea has signed take-or-pay contracts with ipps (t6) 0.145 0.036 ijahp article: shiwakoti, regmi/strategic analysis of the nepal electricity authority: a swot-ahp analysis based on stakeholders’ perceptions international journal of the analytic hierarchy process 27 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.944 figure 3 perceptual mapping of the swot factors as shown in table 24 and figure 3, the analysis suggests that ‘monopoly (single seller) and monopsony (single buyer) market of an essential service’ (s1) is the most important factor in the swot analysis with an overall priority value of 6.9% among the 40 factors under study. other considerable factors in order are ‘ability to access relatively cheaper sources of capital from government, public and international multilateral banks/donors’ (s3) with a priority value of 6.4%, ‘long-term growth for demand of electricity (o1)’ with a priority value of 5.1%, ‘insufficient and inefficient transmission and distribution network’ (w1) with a priority value of 4.1%, ‘huge domestic demand for electricity and energy services’ (s2) with a priority value of 4.1%, ‘nationwide electricity distribution network’ (s4) with a priority value of 4%, s1 s2 s3 s4 s5 s6 s7 w1 w2 w3 w4 w5 w6 w7 w8 w9 w10 w11 o1 o2 o3 o4 o5 o6 o7 o8 o9 o10 t1 t2 t3 t4 t5 t6 t7 t8 t9 t10 t11 t12 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 -0.17 -0.07 0.03 0.13 0.23 new levies and taxes imposed by local, provincial and federal governments (t7) 0.050 0.013 potential complications due to federal restructuring (t8) 0.054 0.014 adverse movement in the dollar and other foreign currencies (t9) 0.051 0.013 inability to engage in trade of electricity with neighboring countries due to political and economic reasons (t10) 0.061 0.015 theft and leakage of electricity and collection losses (t11) 0.050 0.013 delay in construction of projects due to social and/or legal issues such as resettlement, right of way and local shares (t12) 0.052 0.013 ijahp article: shiwakoti, regmi/strategic analysis of the nepal electricity authority: a swot-ahp analysis based on stakeholders’ perceptions international journal of the analytic hierarchy process 28 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.944 and ‘inability to absorb all new generation that nea has signed take-or-pay contracts with ipps’ (t6) with a priority value of 3.6%. this study shows the strength factors of the nea are stronger than any other factors of swot. therefore, the nea should develop future strategies that reap benefits by utilizing the strength parameters and eliminating the effects of the identified threats and weaknesses. 6. conclusion this study has analyzed the external and internal factors relevant to the strategies and operations of the nea. the internal factors have been categorized as the strengths and weaknesses of the organization while the external factors have been categorized as its opportunities and threats. the entire study was done through a swot–ahp analysis. the findings show that weightage or importance of all groups of factors is almost the same, with the weightage of the strengths criteria being slightly higher (27.3%) than the others. the analysis suggests that ‘monopoly (single seller) and monopsony (single buyer) market of an essential service’ is the most important factor in the swot analysis with an overall priority value of 6.9% among the 40 factors under study. other considerable factors are ‘ability to access relatively cheaper sources of capital from government, public and international multilateral banks/donors’ as a strength and ‘long-term growth for demand of electricity’ as the most important opportunity, ‘insufficient and inefficient transmission and distribution network’ as the most critical weaknesses and ‘inability to absorb all new generation that the nea has signed take-or-pay contracts with ipps’ as the most important threat to the nea. the strength factors of the nea are more dominant than the other factors as perceived by the respondents. it is recommended that the nea formulate its future strategies by considering these important factors. the nea should be able to take advantage of its monopolistic business nature while ensuring delivery of safe and reliable electricity to consumers. it also needs to continue accessing the relatively easily available capital from the government and donor agencies to expand its business and cater to the growing electricity demand in the country. however, it has the challenge of rapidly expanding the existing transmission and distribution network to absorb the growing generation capacity within the country, which is also necessary to decrease nepal’s reliance on imported electricity from india. the approach of integrating a swot analysis with the ahp to rank the criteria or factors is an efficient method in decision making and strategy selection for the organization. future research can improve on this approach by using a fuzzy logic framework with the ahp method to analyze cases with uncertainty. ijahp article: shiwakoti, regmi/strategic analysis of the nepal electricity authority: a swot-ahp analysis based on stakeholders’ perceptions international journal of the analytic hierarchy process 29 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.944 references albayrk, e., & erensal, y. (2004). using ahp to improve human performance: an application of multiple criteria decision making proplem. journal of intelligent manufacturing, 15, 491-503. doi: https://doi.org/10.1023/b:jims.0000034112.00652.4c alonso, j., & lamata, t. (2006). consistency in the analytic hierarchy process: a new approach. international journal of uncertainty, fuzziness and knowledge based systems, 2006, 445-459. doi: https://doi.org/10.1142/s0218488506004114 borajee, m., & yakchali, s. (2011). using the ahp-electre iii integrated method in a competitive profile matrix. international conference of financial management and economics, 2011 proceedings, (pp. 68-72). dagdeviren, m., & yavuz, s. (2009). weapon selection using the ahp and topsis methods under fuzzy environment. expert systems with applications, 36, 8143-8151. doi: https://doi.org/10.1016/j.eswa.2008.10.016 dhakal, s., karki, p., & shrestha, s. (2019). cross-border electricity trade for nepal: a swot ahp analysis of barriers and opportunities based. international journal of water resources development, 37(3) 559-580. doi: https://doi.org/10.1080/07900627.2019.1648240 fadim yavuz, t. b. (2014). application combined analytic hierarchy process and swot for integrated watershed management. international journal of the analytical hierarchy process, 6(1), 1-34. doi: https://doi.org/10.13033/ijahp.v6i1.194 forman, e. (1990). random indices for incomplete pairwise comparison matrices. european journal of operational research, 48, 153-155. doi: https://doi.org/10.1016/0377-2217(90)90072-j gallego-ayala, j., & juizo, d. (2011). strategic implementation of integrated water resources management in mozambique: an a-swot analysis. physics and chemistry of the earth, 36, 1103-1111. doi: https://doi.org/10.1016/j.pce.2011.07.040 golden, b., & wang, q. (1990). an alternative measure of consistency. in analytic hierarchy process: applications aand studies (pp. 68-81). new york: springer verlag. gorener, a., toker, k., & ullucay, k. (2012). application of combined swot and ahp: a case study of a manufacturing firm. 8th international strategic management conference (pp. 1525-1534). barelona. doi: https://doi.org/10.1016/j.sbspro.2012.09.1139 gunatilake, h., wijayatunga, p., & roland-holst, d. (2020). hydropower development and economic growth in nepal. manila, philipines: asian development bank (adb). doi: https://doi.org/10.22617/wps200161-2 ijahp article: shiwakoti, regmi/strategic analysis of the nepal electricity authority: a swot-ahp analysis based on stakeholders’ perceptions international journal of the analytic hierarchy process 30 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.944 haque, h. e., dhakal, s., & mostafa, s. (2019). an assessment of opportunities and challenges for cross-border electricity trade for bangladesh using swot-ahp approach. energy policy, 137(feb 2020), 111118. doi: https://doi.org/10.1016/j.enpol.2019.111118 jaroslaw, w., & krzysztof, k. (2017). multi criteria analysis of electric vans for city logistics. sustainability. 9(8), 1453. doi: https://doi.org/10.3390/su9081453 kahraman, c., demirel, n., & demirel, t. (2007). prioritization of e-government strategies using a swot-ahp anaalysis: the case of turkey. european journal of information systems, 16(3), 284-298. doi: https://doi.org/10.1057/palgrave.ejis.3000679 karmacharya, s. (2012). lessons to be learned from the experience of electricity reforms in india. hydro nepal, (11), 29-36. doi: https://doi.org/10.3126/hn.v11i0.7158 kurttila, m., pesonen, j., kangas, m., & kajanus, m. (2000). utilizing the analytic hierarchy process (ahp) in swot analysis-a hybrid method and its application to a forest certification case. forest policy and economics, 1, 44-52. doi: https://doi.org/10.1016/s1389-9341(99)00004-0 lumaksuno, h. (2008). implementation of swot-fahp method to determine the best strategy on development of traditional shipyard in sumenep. academic research international, 5(5), 56-67. nea. (2018). nepal electricity authority: corporate development plan (2018/192022/23). kathmandu: nepal electricity authority. necoechea-porras, p. d., & lopez, a. (2021). deregulation in the energy sector and its economic effects in the power sector: a literature review. sustainability, 13(6), 3429. doi: https://doi.org/10.3390/su13063429 nepal electricity authority. (2019). nepal electrification statistics. nepal electricity authority. (2021). a year in review fiscal year 2020/2021. kathmandu: nepal electricity authority. doi: https://doi.org/10.3126/pycnjm.v12i1.30597 nepal electricity authority. (2021). about us: nepal electricity authority. retrieved from nepal electricity authority website: https://nea.org.np/aboutus oluchukwu, a., & emmanuel, o. (2019). analytical hierarchy process (ahp) approach to the challenges of electric power generation in nigeria. sumerianz journal of economics and finance, 2(3), 26-36. osuna, e., & aranda, a. (2007). combining swot and ahp techniques for strategic planning. isahp 2007. vina del mar. ijahp article: shiwakoti, regmi/strategic analysis of the nepal electricity authority: a swot-ahp analysis based on stakeholders’ perceptions international journal of the analytic hierarchy process 31 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.944 pearce, j., & robinson, j. (2005). strategic management: formulation, implementation and control. boston: irwin mcgraw-hill. pike, s. (2008). destination marketing: an integrated markeing communication approach. uk: butterworth-heinemann. regmi, s., & mandal, a. (2020, november). challenges for construction of high electricity transmission lines in nepal. international jurnal of engineering, applied sciences and technology, 5(7), 46-52. doi: https://doi.org/10.33564/ijeast.2020.v05i07.009 saaty, t. (1980). the analytic hierarchy process. new york: mcgraw-hill. saaty, t., & vargas, l. (1996). decision making with the analytic network process. springer. sharma, m., moon, i., & bae, h. (2008). analytic hierarchy process to assess and optimize distribution network. applied mathematics and computation, 202, 256-265. doi: https://doi.org/10.1016/j.amc.2008.02.008 shobhakhar dhakal, pratik karki & subina shrestha. (2019). international journal of water resource, 1-22. this is an complete reference. tshombe, l. (2013). swot analysis of power utilities . journal for development and leadership, 91-103. tshombe, l. (2013). swot analysis of power utilities in the sadc. journal for development and leadership, 2, 91-103. volodymyr, z., & pasichna, m. (n.d.). ahp based comparative analysis of electricity generating portfolios for the companies in eu and ukraine: criteria, reliability, safety. wickramasinghe, v., & takano, s. (2010). application of combined swot and ahp for tourism revival strategic marketing planning: a case of sri lanka tourism. journal of the eastern asia society for transportation studies, 8, 954-969. yuksek, i., & akin, a. (2006). determination of strategy in business with ahp. dogus university journal, 2, 254-268. yuksel, i., & dagdeviren, m. (2007). using the analytic network procecss in a swot analysis: a case study for a textile firm. information sciences, 177, 33643382. doi: https://doi.org/10.1016/j.ins.2007.01.001 ijahp article: nguyen, truong/integral swot-ahp-tows model for strategic agricultural development in the context of drought: a case study in ninh thuan, vietnam international journal of the analytic hierarchy process 1 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.890 integral swot-ahp-tows model for strategic agricultural development in the context of drought: a case study in ninh thuan, vietnam nguyen hoang tuan faculty of environment university of science, viet nam national university ho chi minh city, viet nam nhtuansg@gmail.com truong thanh canh faculty of environment university of science, viet nam national university ho chi minh city, viet nam ttcanh@hcmus.edu.vn abstract the study combined a qualitative analysis model and quantitative analysis to rank strategies in agricultural production in ninh thuan province, vietnam. a strength, weakness, opportunity, and threat (swot) analysis is used to evaluate the characteristics of internal and external factors through the tows matrix. the research used the analytical hierarchy process (ahp) to compare and rank the influence of the criteria on agricultural production. the ahp method quantified the weights of the factors in the tows matrix based on a pairwise comparison of the elements. the research results show that the ahp technique can help make the decisions of policymakers easier rather than making decisions based on qualitative uncertainty. the swot-ahp-tows model provides a new approach to drought in vietnam using a quantitative tool based on multicriteria analysis. keywords: swot; ahp analysis; mcdm; tows matrix, drought agriculture, vietnam 1. introduction selecting a strategy for a development plan is an important task to assist managers in achieving their goals. the choice of criteria for developing a process is essential, and it is even more critical to assess the value of each factor. multiple-criteria decision-making (mcdm) is a group technique that assists policy planners in making decisions by reviewing all criteria and goals (büyüközkan et al., 2019; kumar et al., 2017). mcdm techniques are well suited to solve complex problems and situations by analyzing various criteria based on expert selection (khan & ali, 2020). mcdm is both a quantitative and a qualitative tool (khan & ali, 2020); therefore, mcdm can be implemented with many different algorithms (dehghanimohammadabadi & kabadayi, 2020; khan & ali, 2020; lohan et al., 2020). currently, the application of the mcdm model is very diverse, being implemented in many different fields from the economy to society and the environment. many different mailto:nhtuansg@gmail.com ijahp article: nguyen, truong/integral swot-ahp-tows model for strategic agricultural development in the context of drought: a case study in ninh thuan, vietnam international journal of the analytic hierarchy process 2 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.890 algorithms have been used with the mcdm technique, including the analytic hierarchy process (ahp) (bellahcene et al., 2020; dixon-ogbechi & adebayo, 2020; garg & ganesh, 2020), the analytic network process (anp) (arsic et al., 2017; azizi & mansouri, 2021, 2021; kabak et al., 2016; živkovi & mihajlovi, 2015), technique for order of preference by similarity to ideal solution (topsis) (arabzad et al., 2015; ozkaya & erdin, 2020; solangi et al., 2019a), vlse kriterijumska optimizacija i kompromisno resenje (vikor) (büyüközkan et al., 2019; opricovic & tzeng, 2004), stochastic multicriteria acceptability analysis (smaa) (kajanus et al., 2012; kangas et al., 2003), weighted product model (wpm) (goepel, 2018), preference ranking organization method for enrichment of evaluations (promethee) (gul et al., 2018), simple multi-attribute rating technique (smart) (lavik et al., 2020), and many other methods that can be reviewed in the studies of genç et al. (2018) and kumar et al. (2017). this study will provide an ahp analysis to select the most optimal agricultural activities under drought conditions in ninh thuan province. the ahp is a popular mcdm methodology (dehghanimohammadabadi & kabadayi, 2020; haque et al., 2020), a technique used to prioritize attributes that influence decision-making to select an alternative based on the relative importance of the attributes (lohan et al., 2020; saaty, 1980). in addition to preparing the ahp analysis data, the study was combined with the swot analysis using the tows matrix. swot is a popular technique for strategic planning (helms & nixon, 2010; osuna & aranda, 2007). this analysis is responsible for evaluating internal and external factors through a relationship analysis of factors (houben et al., 1999). however, the swot analysis provides only the decision-making information environment without ranking the strategy or elements of the swot analysis (genç et al., 2018). consequently, the research will use mcdm to support decisions through swot and ahp analyses. this combination aims to strengthen and improve the information basis of the decision-making and strategies (kajanus et al., 2012). 2. research method 2.1. swot analysis the swot analysis is used to strategically analyze activities based on their strengths (s), weaknesses (w), opportunities (o), and threats (t). the swot study began in the united states between 1960-70 as part of a research project conducted by albert humphrey and a research team at stanford university. the team developed a tool to analyze strategies and understand the causes of business failure, and initially, this method was called soft (sidharth thakur, 2010). soft analysis was studied with 500 companies and 1,100 organizations, and ended with the efforts of 5,000 executives between 1960-1969 (humphrey, 2005). at a long-range planning workshop in zurich, switzerland in 1964, economists urick and orr proposed replacing the f with w (sidharth thakur, 2010). this version was used until 1966 and improved and blended in 1973 at j.w. french ltd. until 2004, several tweaks were made until the swot analysis was perfected and used in many parts of the world (humphrey, 2005). swot analysis is an analytical method that helps plan the development strategy of decision-making (görener et al., 2012). the construction of the elements in a swot analysis is based on identifying the internal and external aspects of ijahp article: nguyen, truong/integral swot-ahp-tows model for strategic agricultural development in the context of drought: a case study in ninh thuan, vietnam international journal of the analytic hierarchy process 3 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.890 the strategy to be implemented (kangas et al., 2003; yuksel & dagdeviren, 2007; živkovi & mihajlovi, 2015). using a swot analysis helps leaders set goals in their field and achieve the desired performance while in environmental transformation (houben et al., 1999). furthermore, the results of swot selection are the starting platform for identifying and selecting feasible strategies that help stimulate and expand the activity (dyson, 2004). the swot analysis has been widely used in economics, society, and environmental research. for example, it has been used in research on tourist marketing in sri lanka (wickramasinghe & takano, 2010), a management strategy for the turkish dyeing industry (yuksel & dagdeviren, 2007), strategic development of agricultural products (elyaspour & bahmani, 2016), plans to expand the aviation industry (sevkli et al., 2012), analysis of the hospitality industry (hung, 2013), and analysis of the global competitiveness of machine tools (shinno et al., 2006). in the environmental field, research into the development of industrial parks in india (patnaik & poyyamoli, 2015), an application for the management of greek coastal groundwater (kallioras et al., 2010), hydrology research (nathan, 2007), and a sustainable energy sector development strategy (markovska et al., 2009) are some examples of its application. on the other hand, the swot analysis also presents specific gaps in the factor measurement and assessment process. the main limitation of a swot analysis is the non-quantification of the importance of each factor in the decision-making process, and also its failure to evaluate the effect of each swot factor on individual strategies (arsić, 2017; hill & westbrook, 1997). however, the advent of mcdm has made the swot analysis more objective in improving swot, and this method is used in many social fields (görener, 2012; markovska et al., 2009; partani et al., 2013). currently, many studies have highlighted the application of swot analysis to mdcm, such as swotahp, swot-fahp, swot-anp, swot-fanp, and swot-topsis (aghasafari et al., 2020; arsic et al., 2017; karimi et al., 2019; solangi et al., 2019b). 2.2. tows matrix this strategy is constructed by interacting with internal and external factors (weihrich, 1982). this analysis has been widely applied in identifying strategies (asadpourian et al., 2020; gottfried et al., 2018). using the tows matrix leverages the swot analysis of the strengths, weaknesses, opportunities, and threats within and outside the system (chanthawong & dhakal, 2016). the advantage of this strategy is the influence of internal and external factors, which are prioritized for elements included in alternative methods (wickramasinghe & takano, 2010). table 1 presents the tows matrix subdivided into internal (s, o) and external (o, t) factors. the strategy is constructed through internal and external factors (haque et al., 2020; wickramasinghe & takano, 2010) and on a database from a swot analysis (gottfried et al., 2018). the tows matrix study comprises four pairs: so, st, wo, and wt (asadpourian et al., 2020). inside, so is the directional strategy for development based on forces to take advantage of external opportunities, and st is a strategy to mitigate the impacts of external risks through internal forces. the wo strategy uses internal weaknesses to harness external opportunities, and wt is a defensive strategy to reduce the disadvantages of the critical sides to limit external dangers. the tows matrix will be used for agriculture production in the context of drought in ninh thuan (table 1). ijahp article: nguyen, truong/integral swot-ahp-tows model for strategic agricultural development in the context of drought: a case study in ninh thuan, vietnam international journal of the analytic hierarchy process 4 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.890 table 1 tows matrix analysis external factors internal factors strengths (s) s1 s2 sn weaknesses (w) w1 w2 wn opportunities (o) o1 o2 on strategic: so use internal forces to exploit external opportunities. strategic: wo use the weaknesses of the interior to exploit the opportunities of the exterior. threats (t) t1 t2 tn strategic: st apply internal forces to reduce external risks. strategic: wt develop strategies to reduce internal weaknesses to minimize external risks. with the tows matrix, it is possible to see the preference for prioritization of future elements (t;o) over those of the past (s;w) (arsic et al., 2017; weihrich, 1982). therefore, during strategic analysis with the tows matrix, the evaluator needs to consider the choice of time; it is possible to start tows from the past, then move to the present, and then analyze them in the future (figure 1). it is also necessary to focus on different periods to evaluate the matrix. usually, the external environment is dynamic, and any change can lead to other factors (arsic et al., 2017; weihrich, 1982). therefore, it is necessary to prepare a tows matrix and evaluate its performance for future periods. figure 1 performance and time of tows analysis source: (arsic et al., 2017; weihrich, 1982) 2.3. ahp analysis the ahp analysis is a mcdm technique (al-rahbi et al., 2020; dehghanimohammadabadi & kabadayi, 2020; garg & ganesh, 2020; kopytov et al., 2011; kou et al., 2013). it was developed by thomas saaty to make complex decisions (mantogiannis & katsigiannis, 2020; t. l. saaty, 1980, 1996, 2005). the ahp compares ijahp article: nguyen, truong/integral swot-ahp-tows model for strategic agricultural development in the context of drought: a case study in ninh thuan, vietnam international journal of the analytic hierarchy process 5 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.890 pairs of factors to prioritize them using separate calculations for each element (görener et al., 2012). the ahp method approximates the criteria to prioritize them using the eigenvalue computation. there are several outstanding advantages to the method, such as its ability to assess the importance of the requirements through comparing standards, the fact that the process is easy to understand, and that it is a qualitative and quantitative analysis. the ahp method can be easily combined with other methods to quantify the factors in scoring the criteria and can check the consistency of the decision maker's assessment. this approach helps decision-makers choose strategies that match the objective of choice. the hybrid approach's application aims to enhance the quantitative aspect of strategic planning. the ahp analysis model is based on t. l. saaty & vargas (2012) and kou et al. (2013) as shown in figure 2. figure 2 three-hierarchical structure model of ahp analysis the ahp analysis is generally based on five basic steps as follows (kou et al., 2013): (1) analyze and problem-solve; (2) construct a set of pairwise comparison matrices; (3) calculate eigenvalues and eigenvectors; (4) check the consistency of all comparison matrices; (5) summarize the absolute priorities of alternatives to make the decision. in the above steps, attention should be paid to calculating the values and checking the consistency ratio (cr). according to saaty, a 9-point scale (table 2) quantifies intangible attributes or criteria into measurable numbers by arranging the pairs to be compared in a matrix (kou et al., 2013; r.w. saaty, 1987; t.l. saaty, 1978; t.l. saaty & vargas, 2012). ijahp article: nguyen, truong/integral swot-ahp-tows model for strategic agricultural development in the context of drought: a case study in ninh thuan, vietnam international journal of the analytic hierarchy process 6 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.890 table 2 saaty’s rating scale rating scale definition explanation 1 equally important these two values contribute equally to the objective. 3 weak importance experience and judgment are somewhat more susceptible to one activity than to another. 5 more important experience and judgment are more in place in one activity than in others. 7 very important intense activity and influence have been shown in practice. 9 extremely important the evidence for one activity over another is as strong as possible. 2, 4, 6, 8 intermediate between two adjoining judgments. compromise needed between comments. if activity a has one of the above nonzero numbers assigned to it compared to activity b, b has the inverse value compared to a equations 1 and 2 are used to calculate the consistency ratio (cr) and to capture uncertainty in the judgment (dehghanimohammadabadi & kabadayi, 2020; kou et al., 2013). cr= ci ri (1) and ci= 𝜆𝑚𝑎𝑥 − 𝑛 𝑛 − 1 (2) where 𝜆𝑚𝑎𝑥 is the highest eigenvalue in the matrix, n is the number of groups, ci is the consistency index, and ri is the random index. table 3 mean random consistency index (ri) n 1 2 3 4 5 6 7 8 9 10 ri 0 0 0.52 0.89 1.11 1.25 1.35 1.4 1.45 1.49 source: (t.l. saaty & vargas, 2012) the ri value is displayed in table 3. the value of cr below or equal to 0.1 or 10% is acceptable, and if the cr is greater than 10% we must consider the matched comparisons (hussey, 2014; nguyen & liu, 2019; t. l. saaty & vargas, 2012; wilford et al., 2020; yavuz & baycan, 2014). 2.4. hybrid swot-ahp-tows although swot analysis identifies and describes the internal factors, it has shortcomings in the measurement and evaluation of the weight of each inner element of the swot group (yuksel & dagdeviren, 2007). in addition, the swot analysis results are only ijahp article: nguyen, truong/integral swot-ahp-tows model for strategic agricultural development in the context of drought: a case study in ninh thuan, vietnam international journal of the analytic hierarchy process 7 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.890 qualitative, incompletely express value in terms of internal and external factors, and lead to a failure to comprehensively evaluate the strategy (hill & westbrook, 1997; kangas et al., 2003; shinno et al., 2006; yuksel & dagdeviren, 2007). therefore, more tools need to be considered and selected to assess the significance of each internal and external value of the swot analysis. the swot-ahp model is a quantitative strategy that analyzes swot analysis factors (gao & peng, 2011; haque et al., 2020; yavuz & baycan, 2014). the functionality of this method is a paired comparison of internal and external swot factors (shinno et al., 2006). the application of the swot-ahp model is widely used in the multi-criteria analysis, such as the study for improving service at service stations (gonzalez & pradenas, 2019), integrated watershed management (yavuz & baycan, 2014), research on the development of healthcare services (osuna & aranda, 2007), policy planning for the education sector (malik, 2013), sustainability of waste management (felice et al., 2013), agricultural development (nasab & azizi, 2014b, 2014a), manufacturing business development strategy (görener, 2012), support policies for transportation in europe (d'adamo et al., 2020), a strategy to develop the korean satellite and space industry (j. lee et al., 2021), resolution of internal conflicts in wetland conservation (pournabi et al., 2021), conversion of biomass refineries in the pulp and paper industry in europe (brunnhofer et al., 2020), studies on transboundary trade in bangladesh (haque et al., 2020), and the energy planning strategy for pakistan (solangi et al., 2019b). in vietnam, the swot-ahp model has been studied in several fields, including research on business strategies for the shipping industry (thành, 2020), desertification and drought in agriculture production (hương, 2015), research into sustainable development for lobster farming (ton nu hai & speelman, 2020), and a study on the competitiveness of the logistics industry (linh, 2018). this study aims to combine the swot-ahp model with the tows matrix to create a swot-ahp-tows model for selecting a priority strategy. the integrated model is a strategic tool for the bottom-up approach and is widely used for policy formulation (chanthawong & dhakal, 2016; haque et al., 2020). the swot-ahp-tows model is easy to apply (gottfried et al., 2018) and can be easily combined with quantitative and qualitative analysis (chanthawong & dhakal, 2016). it is used for research in many fields, such as the power trade in bangladesh (haque et al., 2020), strategic management of natural resources (cajanus et al., 2012), policy development on biodiesel and bioethanol in thailand (chanthawong & dhakal, 2016), study on strategic options for private investment in the biogas sector in china (gottfried et al., 2018), sustainable ecotourism development in iran (asadpourian et al., 2020), and the sustainability of ecotourism in djercdap national park in serbia (arsic et al., 2017). the above studies are often in regional contexts and are very general (haque et al., 2020). they are not directly related to the problem being studied in vietnam. the strength of this model can be seen in the quantification of each criterion in the swot analysis. this model transfers qualitative variables to quantitative variables and helps the decisionmaker see the order of the criteria. in contrast, if only the swot analysis is used, the results are qualitative and very subjective regarding strategic options. therefore, this combination model is more ijahp article: nguyen, truong/integral swot-ahp-tows model for strategic agricultural development in the context of drought: a case study in ninh thuan, vietnam international journal of the analytic hierarchy process 8 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.890 appropriate and has more objective decision-making than other qualitative methods. the model is shown in figure 3. figure 3 analysis diagram of swot-ahp-tows model figure 3 shows that the swot-ahp-tows model is implemented by the following steps: (1) building the swot analysis information through internal factors (strengths and weaknesses) and external factors (opportunities and threats); (2) comparing each pair to compute the weight of each group in the swot analysis; (3) using ahp analysis to prioritize each factor in the analysis groups; (4) prioritizing and selecting strategies. 3. case study of ninh thuan, vietnam 3.1. ninh thuan province ninh thuan is a province located in south central vietnam (figure 4) that has a humid and semi-arid climate (vinh et al., 2013). according to research by scientists, ninh thuan is considered an extreme drought region and at risk of desertification (thao et al., 2018; truong, 2008; tuan & an, 2016). the region has seen severe droughts in 1988, 1993, 1998, 2001, 2002, 2004, and 2005, causing significant damage to the agricultural sector and socio-economic activities (tuan & canh, 2021). in 2015-2016, ninh thuan experienced its worst period of drought, which was the first time the local government declared a drought emergency in the whole province (thao et al., 2018). according to research, drought has reduced agricultural land area by 83.2%, reduced crop yield by 84.6%, and reduced the quality of agricultural products by 80.3%. (nam & trang, 2019). according to pham quang vinh (2013), drought profoundly impacts socio-economic ijahp article: nguyen, truong/integral swot-ahp-tows model for strategic agricultural development in the context of drought: a case study in ninh thuan, vietnam international journal of the analytic hierarchy process 9 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.890 activities, and agriculture is the most affected (vinh, 2015). therefore, ninh thuan needs to have strategic solutions for agricultural development in the context of drought. figure 4 location map of the research area 3.2. research data the study uses research data from experts in drought and agriculture in vietnam. drought data and data about drought’s effect on agricultural production are used. the drought data includes research using documents about climate change trends and future climate forecasts for ninh thuan province (tuan & canh, 2021), rainfall trends in ninh thuan in the context of climate change by non-parametric methods (tuan & canh, 2020), drought and climate change in ninh thuan province (vinh, 2015), research on climate characteristics of ninh thuan province (nam et al., 2018), and investigation, evaluation and proposals for integrated management measures to limit land degradation due to drought and desertification in ninh thuan (gai et al., 2018). data on the impact of drought on agricultural production has been collected including research on planting techniques of some indigenous tree species with economic value in arid areas of ninh thuan and binh thuan provinces (dung et al., 2012), assessing the effects of agroclimatic conditions on short-term crops in ninh thuan and binh thuan (vinh et al., 2013), and studying desertification and degradation land conversion to agricultural production in ninh thuan province (truong, 2015). 3.3. results and discussion phase 1: swot model in agriculture in ninh thuan province the swot analysis is based on a detailed analysis of ninh thuan province's natural and socio-economic characteristics. the research focuses on terrain, climate, rivers, soil, organisms, temperature, precipitation, and climate change with natural elements. in addition to the socio-economic characteristics, the study focuses on assessing growth rate, socio-economic planning policies for agriculture, and policies to attract scientific research, demographics, markets, and labor qualification. in relation to external factors, the study focuses on the agricultural planning policy of the state, the development policy ijahp article: nguyen, truong/integral swot-ahp-tows model for strategic agricultural development in the context of drought: a case study in ninh thuan, vietnam international journal of the analytic hierarchy process 10 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.890 of neighboring regions, the scientific research and its application in the country, and the climate change situation in vietnam. the internal and external factors and selected factors were identified through a survey with experts asking their opinions on the elements of the swot group. the experts participating in the assessment of the swot analysis framework include local and non-local experts who have studied drought in the ninh thuan area. fourteen experts were selected for the research including ten from research institutes and universities and four from ninh thuan province (table 4). table 4 experts participating in the survey/scoring expert academic titles fields of research/department expert 1 prof., phd water resources and drought management expert 2 assoc. prof., dr earth sciences and climate change expert 3 assoc. prof., phd socio-economic geography expert 4 assoc. prof., phd environmental science, engineering and management expert 5 assoc. prof., phd geographic information systems, remote sensing, and drought management expert 6 assoc. prof., phd drought management expert 7 phd human geography expert 8 phd regional sustainable development expert 9 phd human geography expert 10 phd earth sciences expert 11 master department of natural resources and environment of ninh thuan province expert 12 master department of science and technology of ninh thuan province expert 13 master department of agriculture and rural development of ninh thuan province expert 14 master center of hydro-meteorological forecasting of ninh thuan province a tows matrix was created through a four-step process of building a swot analysis as follows: step 1: synthesize relevant documents mentioned in section 3.1 of the research data. step 2: deploy the survey and consult experts on the contents of the swot analysis through scoring. step 3: re-check the elements in the additional swot analysis and remove the nonconforming factors based on expert judgments. step 4: select the factors in the swot analysis with an average score of 5.0 or higher. these steps helped select the internal factors (strengths, weaknesses) and external factors (opportunities, threats). the elements in the swot analysis are shown through the tows matrix (table 5). ijahp article: nguyen, truong/integral swot-ahp-tows model for strategic agricultural development in the context of drought: a case study in ninh thuan, vietnam international journal of the analytic hierarchy process 11 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.890 table 5 tows matrix for agricultural production strategy of ninh thuan province internal factor external factor strengths (s) weakness (w) (s1). diverse terrain and soils (s2). the number of days and sunny hours is favorable for annual planting (s3). the irrigation system is gradually upgraded and invested in (s4). people have extensive experience in production in drought conditions. (s5). there is a policy to attract and apply science and technology to the agricultural sector (s6). high-tech agricultural cluster planning policy (s7). there are many critical crops of high economic value (w1). water sources in reservoirs and groundwater are increasingly depleted. (w2). the dry season is prolonged, and there is a high potential for evaporation. (w3). average annual precipitation is low and very unevenly distributed. (w4). drought usually occurs in the dry season, and soil degradation is ongoing. (w5). agricultural production greatly depends on nature (w6). people's ability to proactively prevent and mitigate natural disasters is low. (w7). agricultural land is on a downward trend due to industrialization and urbanization. (w8). disaster forecasting and warning work are limited and not close to reality (w9). human resources with expertise in disaster management are lacking. opportunity (o) strategy: so strategy: wo (o1). policies support national crop conversion and agricultural production policies in drought-affected areas. (o2). the trend of applying science and high technology to sustainable agricultural production adapting to drought (so1). focus on investing in the production of critical crops of high economic value. (so2). expand the market for vital agricultural products. (wo1). develop policies and transform farming in drought-affected areas (wo2). call for scientific research projects in the field of agriculture ijahp article: nguyen, truong/integral swot-ahp-tows model for strategic agricultural development in the context of drought: a case study in ninh thuan, vietnam international journal of the analytic hierarchy process 12 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.890 (o3). the policy of restructuring the agricultural sector and forming regional link chains for critical products (o4). near the market of large agricultural products in the country. (o5). domestic and international cooperation, scientific research projects. (o6). ngos' concerns about natural disasters caused by drought and climate change threats (t) strategy: st strategy: wt (t1). the hard-to-assess activity of climate change (t2). dry season flows tend to decrease (t3). drought and desertification are on the rise (t4). the irrigation development policy of neighboring areas (t5). the trend of the labor movement in agriculture (t6). no insurance policy for agriculture (st1). increase investment, expansion, and upgrading of irrigation systems. (st2). support enterprises to invest in the application of science and technology to agricultural production (wt1). adjust the time of planting (wt2). develop of insurance policies in agricultural production (wt3) train human resources, increase investment in disaster warning and monitoring systems ijahp article: nguyen, truong/integral swot-ahp-tows model for strategic agricultural development in the context of drought: a case study in ninh thuan, vietnam international journal of the analytic hierarchy process 13 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.890 based on evaluation of the factors in the tows matrix, the research has combined pairs of so, wo, st, and wt (table 5). the results of the proposed solutions for the couples are as follows. research on the so pair suggests the following two solutions: focus on investing in the production of critical crops of high economic value (so1) and expand the market for vital agricultural products (so2). the study proposes two solutions for the wo pair as follows: develop policies and transform farming in drought-affected areas (wo1) and call for scientific research projects in agriculture (wo2). the st solutions include increasing investment, expansion, and upgrading irrigation systems (st1) and supporting enterprises to apply science and technology to agricultural production (st2). the wt pair solutions include strategies such as adjusting the time of planting (wt1), developing insurance policies in agricultural production (wt2), training human resources, and increasing investment in disaster warning and monitoring systems (wt3). it has been demonstrated that the so strategy maximizes internal resources by making the most of external opportunities. the wo strategy shows that it is necessary to minimize weaknesses and maximize opportunities from external factors. the minimal st strategy involves defining a production strategy based on internal resources while minimizing impacts from external threats; wt is the ultimate strategy for identifying solutions in agriculture to reduce weaknesses and avoid external challenges (arsic et al., 2017; weihrich, 1982). phase 2: calculating the weights of factors in the tows matrix by ahp analysis since the decision-making process is often not easy, objective judgment and subjective evaluation must be considered when making decisions (arsić, 2017). in addition, to establishing criteria, it is necessary to reconcile and accept ambiguous assessments, and different levels of expert interest (yu et al., 2005). a comparison of the weights of the s, w, o, and t factors based on expert selection and scoring are shown in table 6. the comparison matrix of the factor pairs of the analysis, swot, shows that the cr of this comparison is satisfactory and acceptable, with cr = 0.0631 (cr≤0.1). the results of comparing s pairs with weighted wot is 0.416 (rank 1); w with weighted sot gives 0.198 (rank 2); o with weighted swt gives 0.236 (rank 2); and t with a weighted swo yields 0.150 (rank 4). this comparison shows the importance of strengths (s) and opportunities (o) which rank 1 and 2, respectively. table 6 pairwise comparison of the swot factors swot s w o t weight ranking s 1 2 2 3 0.416 1 w 1/2 1 1/2 2 0.198 3 o 1/2 2 1 1 0.236 2 t 1/3 ½ 1 1 0.150 4 cr=0.0631 the process was combined with an ahp analysis where experts calculated the weight of the swot group. the evaluation findings were demonstrated by the internal comparison ijahp article: nguyen, truong/integral swot-ahp-tows model for strategic agricultural development in the context of drought: a case study in ninh thuan, vietnam international journal of the analytic hierarchy process 14 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.890 of factors with the cr of satisfactory swot components with cr < 0.1 (tables 7, 8, 9, and 10). in the group of strengths, local weights of strengths that have factors directly or indirectly related to agriculture in ninh thuan province were calculated. the results show that s3 ranks first, s1 ranks second, and s2 ranks third with weights of 0.263, 0.206, 0.141, respectively (table 7). next, the weighting of the weaknesses group shows that the human resource factor is considered the most important among the weaknesses, ranking first with a weight of 0.226 (w9); next, the problem of forecasting and warning of natural disasters (w8) had a weight of 0.192 (table 8). the weighting results also reflected some of the weaknesses of ninh thuan province. in recent years, human resources with expertise in drought have been greatly reduced, and are therefore unable to handle the changes in drought patterns and the problem of forecasting and warning of natural disasters. table 7 pairwise comparison matrix of the strengths s s1 s2 s3 s4 s5 s6 s7 weight ranking s1 1 ½ 1/2 1/2 3 2 2 0.141 3 s2 2 1 1/2 2 2 3 3 0.206 2 s3 2 2 1 3 3 2 3 0.263 1 s4 2 ½ 1/3 1 1/2 1/2 2 0.109 5 s5 1/3 ½ 1/3 2 1 1/2 1 0.083 6 s6 1/2 1/3 1/2 2 2 1 2 0.122 4 s7 1/2 1/3 1/3 1/2 2 1/2 1 0.077 7 cr=0.0840 table 8 pairwise comparison matrix of the weaknesses w w1 w2 w3 w4 w5 w6 w7 w8 w9 weight ranking w1 1 2 2 4 3 2 ½ 1/3 1/2 0.123 4 w2 1 1 2 4 4 2 1/3 1/3 1/3 0.096 5 w3 1/2 1/2 1 1/2 1/2 2 1/3 ¼ 1/3 0.051 7 w4 1/2 1/2 2 1 3 2 1/3 ½ 1/2 0.081 6 w5 1/4 1/4 2 1/3 1 2 ½ ½ 1/3 0.061 8 w6 1/2 1/2 1/2 1/2 1/2 1 ½ ½ 1/5 0.042 9 w7 2 2 3 3 2 2 1 1/3 1/3 0.128 3 w8 3 3 4 2 2 3 3 1 1/2 0.192 2 w9 3 3 3 2 3 5 3 2 1 0.226 1 cr=0.0833 the results of the opportunities group showed that o1 ranked first with the direct support of the national crop conversion support policy for drought-producing regions (0.326), and o3 ranked second with policy restructuring and the formation of chain links for critical products. in addition, the guidelines for attracting scientific research and collaborating with ngos were ranked at 5 and 6, with local weights of 0.077 and 0.051 (table 9). the evaluation matrix of threat factors shows the priority order from t6 > t5 > t1-t2 > t3 > ijahp article: nguyen, truong/integral swot-ahp-tows model for strategic agricultural development in the context of drought: a case study in ninh thuan, vietnam international journal of the analytic hierarchy process 15 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.890 t4. among the threats, t6 (0.329) is considered a top challenge for ninh thuan and the country. t1 and t2 have equal assessments of the difficult-to-assess activities of climate change, and the volume of dry season flows tends to decrease (table 10). table 9 pairwise comparison matrix of the opportunities o o1 o2 o3 o4 o5 o6 weight ranking o1 1 2 2 3 4 5 0.326 1 o2 1/2 1 1/3 2 3 3 0.161 3 o3 1/2 3 1 3 4 3 0.262 2 o4 1/3 ½ 1/3 1 3 3 0.122 4 o5 1/4 1/3 1/4 1/3 1 3 0.077 5 o6 1/5 1/3 1/3 1/3 1/3 1 0.051 6 cr=0.0747 table 10 pairwise comparison matrix of the threats t t1 t2 t3 t4 t5 t6 weight ranking t1 1 ½ 2 2 1/2 1/2 0.135 3 t2 2 1 2 1/2 1/2 1/3 0.135 3 t3 1/2 ½ 1 2 1/2 1/2 0.107 4 t4 1/2 2 1/2 1 1/3 1/4 0.093 5 t5 2 2 2 3 1 1/3 0.201 2 t6 2 3 2 4 3 1 0.329 1 cr=0.0982 phase 3: determining the weighting of factors in swot at the end of the ahp analysis, the study determined the weight of each factor in the swot group. however, according to experts, it is necessary to calculate the global importance of the individual elements with the group weight of that factor. therefore, the particular factor's global weight is calculated by the local weights x group weights (lee, 2013; wickramasinghe & takano, 2010; yavuz & baycan, 2014). the results of the global weight calculation are detailed in table 11. ijahp article: nguyen, truong/integral swot-ahp-tows model for strategic agricultural development in the context of drought: a case study in ninh thuan, vietnam international journal of the analytic hierarchy process 16 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.890 table 11 global weight score of swot group with ahp analysis swot group group weight swot factors local weight global weight internal s 0.416 s1 0.141 0.059 s2 0.206 0.086 s3 0.263 0.109 s4 0.109 0.045 s5 0.083 0.035 s6 0.122 0.051 s7 0.077 0.032 w 0.198 w1 0.123 0.024 w2 0.096 0.019 w3 0.051 0.010 w4 0.081 0.016 w5 0.061 0.012 w6 0.042 0.008 w7 0.128 0.025 w8 0.192 0.038 w9 0.226 0.045 external o 0.236 o1 0.326 0.077 o2 0.161 0.038 o3 0.262 0.062 o4 0.122 0.029 o5 0.077 0.018 o6 0.051 0.012 t 0.150 t1 0.135 0.020 t2 0.135 0.020 t3 0.107 0.016 t4 0.093 0.014 t5 0.201 0.030 t6 0.329 0.049 phase 4: ranking and selection of priority strategies the priority of the strategies depends on the experts’ choice on a scale of 0 to 5 (table 12). the relational weighting of the swot factors and strategies was calculated from previous dcmd research (fabac & zver, 2011 ; kandakoglu et al., 2007 ; malik, 2013; wickramasinghe & takano, 2010; yavuz & baycan, 2014). ijahp article: nguyen, truong/integral swot-ahp-tows model for strategic agricultural development in the context of drought: a case study in ninh thuan, vietnam international journal of the analytic hierarchy process 17 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.890 table 12 levels of relationship evaluation degree of relationship number none 0 very weak 1 weak 2 medium 3 strong 4 very strong 5 to evaluate the priority selection of the research strategies, equation 3 was used (kandakoglu et al., 2007; wickramasinghe & takano, 2010). 𝑆𝑖 = ∑ 𝐺𝑗 𝑅𝑖𝑗 𝑛 𝑗=1 (3) where sj is the total weight of the i th strategy, gj is the global weight of the j th swot factor, rij is the degree of relationship of the i th strategy and j th swot factors, and n is the number of swot factors. next, the standardized value of strategic weights was calculated by equation 4, ni= 𝑆𝑖 ∑ si m i=1 (4) where ni is the normalized weight of i th strategy and m is the number of strategies. ijahp article: nguyen, truong/integral swot-ahp-tows model for strategic agricultural development in the context of drought: a case study in ninh thuan, vietnam international journal of the analytic hierarchy process 18 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.890 table 13 priority strategy selection matrix swot factors strategy with the degree of relationship so1 so2 wo1 wo2 st1 st2 wt1 wt2 wt3 s1 3 0 1 1 3 0 0 0 0 s2 3 0 4 1 0 2 5 0 1 s3 5 3 4 1 5 1 3 0 1 s4 3 1 2 1 0 0 2 1 0 s5 2 0 2 5 0 4 2 1 2 s6 4 3 1 1 2 1 1 2 3 s7 4 3 3 1 2 0 3 2 2 w1 3 0 2 1 5 0 4 1 2 w2 3 0 2 1 4 2 5 4 2 w3 1 0 4 2 5 1 4 4 2 w4 1 0 4 3 5 1 3 5 2 w5 1 1 3 2 2 3 2 1 1 w6 0 0 1 1 0 0 1 0 0 w7 1 0 1 0 0 0 0 2 0 w8 2 0 3 2 2 0 3 2 0 w9 0 0 1 2 1 0 1 0 5 o1 2 2 4 1 4 3 2 2 2 o2 1 0 2 2 2 1 1 0 1 o3 3 5 2 1 2 2 1 3 0 o4 2 3 0 0 0 0 0 0 0 o5 0 0 0 2 0 0 1 0 2 o6 0 0 0 1 1 1 0 1 1 t1 2 1 4 3 3 0 2 2 1 t2 2 0 3 1 4 0 3 0 1 t3 2 1 2 1 3 1 2 1 1 t4 1 0 1 0 2 0 1 0 0 t5 0 1 0 0 0 0 0 0 1 t6 1 1 1 0 0 0 0 5 0 total weight 2.397 1.299 2.310 1.224 2.099 0.991 1.955 1.260 1.182 weight of relationship 0.163 0.088 0.157 0.083 0.143 0.067 0.133 0.086 0.080 ranking 1 5 2 7 3 9 4 6 8 the weighting of the relationship of the swot factors with the proposed strategies is described in table 13. the results of the calculations identified strategies with different weights. for example, in the strategic group of the tows matrix, the so group (0.126) has the highest priority; the second is wo (0.120), the third is st (0.105), and the last is wt (0.100). however, if we consider the priority of each strategy, there is a division of priority in the specific strategy. for example, figure 4 shows the order of the strategy from the core going counter-clockwise with the priority strategy so1 (0.163), the secondpriority strategy wo1 (0.157), and the last-priority strategy st2 (0.067). with the prioritization of the strategies defined, we needed to prioritize the coordination based on the same time with strategic equal weights. table 14 shows progress in ijahp article: nguyen, truong/integral swot-ahp-tows model for strategic agricultural development in the context of drought: a case study in ninh thuan, vietnam international journal of the analytic hierarchy process 19 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.890 implementing solutions for the agricultural sector in the context of drought. first, the priority strategy was established to select joint or individual strategies. this process shows that the priority strategy must focus on producing crops of high economic value and developing crop conversion policies for drought-stricken areas. crops of high economic value in ninh thuan are mostly those that use water sparingly and are specially adapted to drought conditions. the transformation of the crop structure to drought conditions is the next most important issue because ninh thuan is one of the driest regions in vietnam. to prepare for the development of the agricultural sector, it is necessary to increase investment and expand the irrigation system to adjust the planting time to suit the actual situation. later stages focus on policies in agriculture, such as raising the market for agricultural products, building insurance policies in agriculture, and calling for investment in science and technology in agriculture. in addition to implementing strategies, decision-makers need to define a timeline for strategy implementation (figure 5). since there will be big or small changes depending on the status quo, it is necessary to adjust the strategy. the pdca cycle (plan–do– check–act) can be combined with strategies to check the performance of the implemented strategy and then make adjustments to the subsequent development strategy. table 14 strategic alternatives strategy explanation 1 so1 focus on investing in the production of critical crops of high economic value 2 wo1 develop policies and transform farming in drought-affected areas 3 st1 increase investment, expansion, and upgrading of irrigation systems 4 wt1 adjust the time of planting 5 so2 expand the market for vital agricultural products 6 wt2 develop insurance policies in agricultural production 7 wo2 call for scientific research projects in the field of agriculture 8 wt3 train human resources, increase investment in disaster warning and monitoring systems 9 st2 support enterprises to invest in the application of science and technology to agricultural production ijahp article: nguyen, truong/integral swot-ahp-tows model for strategic agricultural development in the context of drought: a case study in ninh thuan, vietnam international journal of the analytic hierarchy process 20 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.890 figure 5 performance and time of strategies 4. conclusion this study has successfully combined the swot-ahp-tows models to select a priority strategy in agricultural development in ninh thuan province in the context of drought. with an accepted cr of 0.0631, the study determined the local weighting of the factors in the swot group. the order of the weights is as follows: strengths (0.416), weaknesses (0.198), opportunities (0.236), and threats (0.150). the tows matrix is formed with strategy pairs of so, wo, st, and wt. by evaluating the relationship between individual factors within the swot group and strategies, the study calculated and selected the priority strategies for so1, wo1, st1, wt1, so2, wt2, wo3, and st2. the research results show that ninh thuan should implement agricultural development plans in drought conditions. the first proposed task is to increase the planting of high economic value crops based on drought adaptation (so1). in parallel, policies are required to support drought-affected agricultural production areas (wo1). on the other hand, the research findings also demonstrate the role of experts in the selection and assessment of strategies. therefore, selecting experts and researchers who understand the research problem and have a strategic vision is important. for further evaluation, experts may be chosen from local researchers and non-local activists. in addition, priority strategies can only be developed quickly and adjusted for internal and external changes. the research results show that qualitative and quantitative analysis is an effective combination for selecting solutions. these methods have solved the problem of subjectivity in the decision-making of strategic planners, and the application of mcdm in planning is essential. thus, a strength of the research is the improvement of qualitative ijahp article: nguyen, truong/integral swot-ahp-tows model for strategic agricultural development in the context of drought: a case study in ninh thuan, vietnam international journal of the analytic hierarchy process 21 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.890 decisions through quantitative modeling. in addition, the analysis simplified and explained the calculation steps and strategy selection to the reader. the weakness of the research is that decisions are based on individual factors without considering the network of criteria. therefore, it is necessary to conduct further research on the network relationship between the requirements to improve and enhance the reliability of future decisions. in the future, the swot-ahp-tows research model may be applied to agriculture and drought. first, however, it is necessary to combine this model with other mcdm methodologies to increase more effective decision-making accuracy. ijahp article: nguyen, truong/integral swot-ahp-tows model for strategic agricultural development in the context of drought: a case study in ninh thuan, vietnam international journal of the analytic hierarchy process 22 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.890 references aghasafari, h., karbasi, a., mohammadi, h., & calisti, r. (2020). determination of the best strategies for development of organic farming: a swot – fuzzy analytic network process approach. journal of cleaner production, 277, 124039. doi: https://doi.org/10.1016/j.jclepro.2020.124039 al-rahbi, a. k. h., abushammala, m. f. m., & qazi, w. a. (2020). application of the analytic hierarchy process for management of soil erosion in oman. international journal of the analytic hierarchy process, 12(1), 104-116. doi: https://doi.org/10.13033/ijahp.v12i1.683 arabzad, s.m., ghorbani, m., razmi, j., & shirouyehzad, h. (2015). employing fuzzy topsis and swot for supplier selection and order allocation problems. the international journal of advanced manufacturing technology, 76(5–8), 803–818. doi: https://doi.org/10.1007/s00170-014-6288-3 arsić, s. (2017). integralni swot-anp-fanp model za prioritizaciju strategija održivog razvoja ekoturizma u nacionalnom parku đerdap [doktorska disertacije]. univerziteta u beogradu – tehničkog fakulteta u boru. arsic, s., nikolic, d., & zivkovic, z. (2017). hybrid swot-anp-fanp model for prioritization strategies of sustainable development of ecotourism in national park djerdap, serbia. forest policy and economics, 80, 11–26. doi: https://doi.org/10.1016/j.forpol.2017.02.003 asadpourian, z., rahimian, m., & gholamrezai, s. (2020). swot-ahp-tows analysis for sustainable ecotourism development in the best area in lorestan province, iran. social indicators research, 152(1), 289–315. doi: https://doi.org/10.1007/s11205-020-02438-0 azizi, m., & mansouri, h. (2021). analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry. international journal of the analytic hierarchy process, 13(1), 51-91. doi: https://doi.org/10.13033/ijahp.v13i1.813 bellahcene, m., benamar, f.-z., & mekidiche, m. (2020). ahp and wafgp hybrid model for information system project selection. international journal of the analytic hierarchy process, 12(2), 228-253. doi: https://doi.org/10.13033/ijahp.v12i2.761 brunnhofer, m., gabriella, n., schöggl, j.-p., stern, t., & posch, a. (2020). the biorefinery transition in the european pulp and paper industry – a three-phase delphi study including a swot-ahp analysis. forest-based circular bioeconomy: matching sustainability challenges and new business opportunities, 110, 101882. doi: https://doi.org/10.1016/j.forpol.2019.02.006 büyüközkan, g., göçer, f., & karabulut, y. (2019). a new group decision making approach with if ahp and if vikor for selecting hazardous waste carriers. measurement, 134, 66–82. doi: https://doi.org/10.1016/j.measurement.2018.10.041 https://doi.org/10.1016/j.jclepro.2020.124039 https://doi.org/10.13033/ijahp.v12i1.683 https://doi.org/10.1007/s00170-014-6288-3 https://doi.org/10.1016/j.forpol.2017.02.003 https://doi.org/10.1007/s11205-020-02438-0 https://doi.org/10.13033/ijahp.v13i1.813 https://doi.org/10.13033/ijahp.v12i2.761 https://doi.org/10.1016/j.forpol.2019.02.006 https://doi.org/10.1016/j.measurement.2018.10.041 ijahp article: nguyen, truong/integral swot-ahp-tows model for strategic agricultural development in the context of drought: a case study in ninh thuan, vietnam international journal of the analytic hierarchy process 23 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.890 chanthawong, a., & dhakal, s. (2016). stakeholders' perceptions on challenges and opportunities for biodiesel and bioethanol policy development in thailand. energy policy, 91, 189–206. doi: https://doi.org/10.1016/j.enpol.2016.01.008 d’adamo, i., falcone, p. m., gastaldi, m., & morone, p. (2020). res-t trajectories and an integrated swot-ahp analysis for biomethane. policy implications to support a green revolution in european transport. energy policy, 138, 111220. doi: https://doi.org/10.1016/j.enpol.2019.111220 dehghanimohammadabadi, m., & kabadayi, n. (2020). a two-stage ahp multiobjective simulation optimization approach in healthcare. international journal of the analytic hierarchy process, 12(1), 117-135. doi: https://doi.org/10.13033/ijahp.v12i1.701 gai đ. đ., long l. b., thia l. h., & hung p. t. (2018). dieu tra, danh gia va de xuat bien phap quan ly tong hop de han che qua trinh thoai hoa dat do bi kho han, hoang mac hoa tren dia ban tinh ninh thuan (no. 2018-259-04/kqnc). vien khoa học cong nghe va quan li moi truong. dixon-ogbechi, b. n., & adebayo, a. k. (2020). application of the ahp model to determine prefab housing adoption factors for developers in lagos state. international journal of the analytic hierarchy process, 12(2), 297-327. doi: https://doi.org/10.13033/ijahp.v12i2.635 dyson, r. g. (2004). strategic development and swot analysis at the university of warwick. applications of soft o.r. methods, 152(3), 631–640. doi: https://doi.org/10.1016/s0377-2217(03)00062-6 elyaspour, f., & bahmani, p. (2016). design of strategic model of agriculture products of bijar county by hybrid swot-anp model. bioscience biotechnology research communications, 9(4), 783–789. doi: https://doi.org/10.21786/bbrc/9.4/30 fabac, r., & zver, i. (2011). applying the modified swot–ahp method to the tourism of gornje međimurje. tourism and hospitality management, university of rijeka, 17(2), 16. doi: https://doi.org/10.20867/thm.17.2.3 felice, f. d., petrillo, a., autorino, c., & carlomusto, a. (2013, june 23). sustainable decision-making model based on analytical hierarchy process and swot analysis: "sahp" model. the international symposium on the analytic hierarchy process, malaysia. doi: https://doi.org/10.13033/isahp.y2013.080 gao, c.-y., & peng, d.-h. (2011). consolidating swot analysis with nonhomogeneous uncertain preference information. knowledge-based systems, 24(6), 796–808. doi: https://doi.org/10.1016/j.knosys.2011.03.001 garg, a., & ganesh, t. (2020). an analytical hierarchy process approach for covid-19 risk assessment study amid the latest re-open and unlock phase in india. international journal of the analytic hierarchy process, 12(3), 565-576. doi: https://doi.org/10.13033/ijahp.v12i3.814 https://doi.org/10.1016/j.enpol.2016.01.008 https://doi.org/10.1016/j.enpol.2019.111220 https://doi.org/10.13033/ijahp.v12i1.701 https://doi.org/10.13033/ijahp.v12i2.635 https://doi.org/10.1016/s0377-2217(03)00062-6 https://doi.org/10.21786/bbrc/9.4/30 https://doi.org/10.20867/thm.17.2.3 https://doi.org/10.13033/isahp.y2013.080 https://doi.org/10.1016/j.knosys.2011.03.001 ijahp article: nguyen, truong/integral swot-ahp-tows model for strategic agricultural development in the context of drought: a case study in ninh thuan, vietnam international journal of the analytic hierarchy process 24 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.890 genç, t., kabak, m., özceylan, e., & çetinkaya, c. (2018). evaluation of natural gas strategies of turkey in east mediterranean region: a strengths-weaknessesopportunities-threats and analytic network process approach. technological and economic development of economy, 24(3), 1041–1062. doi: https://doi.org/10.3846/20294913.2016.1253043 goepel, k. d. (2018). implementation of an online software tool for the analytic hierarchy process (ahp-os). international journal of the analytic hierarchy process, 10(3), 469-487. doi: https://doi.org/10.13033/ijahp.v10i3.590 gonzalez, f., & pradenas, l. (2019). multi-criteria analysis to improve the service in gas stations. international journal of the analytic hierarchy process, 11(1), 67–90. doi: https://doi.org/10.13033/ijahp.v11i1.601 görener, a. (2012). comparing ahp and anp: an application of strategic decisions making in a manufacturing company. international journal of business and social science, 3(11), 194-208. görener, a., toker, k., & uluçay, k. (2012). application of combined swot and ahp: a case study for a manufacturing firm. 8th international strategic management conference, 58, 1525–1534. doi: https://doi.org/10.1016/j.sbspro.2012.09.1139 gottfried, o., de clercq, d., blair, e., weng, x., & wang, c. (2018). swot-ahptows analysis of private investment behavior in the chinese biogas sector. journal of cleaner production, 184, 632–647. doi: https://doi.org/10.1016/j.jclepro.2018.02.173 gul, m., celik, e., gumus, a. t., & guneri, a. f. (2018). a fuzzy logic based promethee method for material selection problems. beni-suef university journal of basic and applied sciences, 7(1), 68–79. doi: https://doi.org/10.1016/j.bjbas.2017.07.002 haque, h. m. e., dhakal, s., & mostafa, s. m. g. (2020). an assessment of opportunities and challenges for cross-border electricity trade for bangladesh using swot-ahp approach. energy policy, 137, 111118. doi: https://doi.org/10.1016/j.enpol.2019.111118 hill, t., & westbrook, r. (1997). swot analysis: it's time for a product recall. long range planning, 30(1), 46–52. doi: https://doi.org/10.1016/s0024-6301(96)00095-7 houben, g., lenie, k., & vanhoof, k. (1999). a knowledge-based swot-analysis system as an instrument for strategic planning in small and medium sized enterprises. decision support systems, 26(2), 125–135. doi: https://doi.org/10.1016/s01679236(99)00024-x humphrey, a. (2005). swot analysis for management consulting. sri alumni newsletter, sri international. https://www.sri.com/sites/default/files/brochures/dec05.pdf https://doi.org/10.3846/20294913.2016.1253043 https://doi.org/10.13033/ijahp.v10i3.590 https://doi.org/10.13033/ijahp.v11i1.601 https://doi.org/10.1016/j.sbspro.2012.09.1139 https://doi.org/10.1016/j.jclepro.2018.02.173 https://doi.org/10.1016/j.bjbas.2017.07.002 https://doi.org/10.1016/j.enpol.2019.111118 https://doi.org/10.1016/s0024-6301(96)00095-7 https://doi.org/10.1016/s0167-9236(99)00024-x https://doi.org/10.1016/s0167-9236(99)00024-x https://www.sri.com/sites/default/files/brochures/dec-05.pdf https://www.sri.com/sites/default/files/brochures/dec-05.pdf ijahp article: nguyen, truong/integral swot-ahp-tows model for strategic agricultural development in the context of drought: a case study in ninh thuan, vietnam international journal of the analytic hierarchy process 25 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.890 hung, k. (2013). understanding china's hotel industry: a swot analysis. journal of china tourism research, 9(1), 81–93. doi: https://doi.org/10.1080/19388160.2013.756771 huong, b. t. t. (2015). nghien cuu anh huong cua hoang mac hoa den san xuat nong nghiep o tinh binh thuan trong boi canh bien doi khi hau [phd dissertation]. vietnam academy of science and technology. hussey, l. k. (2014). analytic network process (anp) for housing quality evaluation: a case study in ghana. electronic thesis and dissertation repositor.2407, western university. https://ir.lib.uwo.ca/etd/2407 kabak, m., dağdeviren, m., & burmaoğlu, s. (2016). a hybrid swot-fanp model for energy policy making in turkey. energy sources, part b: economics, planning, and policy, 11(6), 487–495. doi: https://doi.org/10.1080/15567249.2012.673692 kajanus, m., leskinen, p., kurttila, m., & kangas, j. (2012). making use of mcds methods in swot analysis—lessons learnt in strategic natural resources management. forest policy and economics, 20, 1–9. doi: https://doi.org/10.1016/j.forpol.2012.03.005 kallioras, a., pliakas, f., diamantis, i., & kallergis, g. (2010). swot analysis in groundwater resources management of coastal aquifers: a case study from greece. water international, 35(4), 425–441. doi: https://doi.org/10.1080/02508060.2010.508929 kandakoglu, a., akgun, i., & topcu, y. i. (2007). strategy development & evaluation in the battlefield using quantified swot analytical method. 9th international symposium on analytic hierarchy process (isahp), 9. doi: https://doi.org/10.13033/isahp.y2007.022 kangas, j., kurttila, m., kajanus, m., & kangas, a. (2003). evaluating the management strategies of a forestland estate—the s-o-s approach. journal of environmental management, 69(4), 349–358. doi: https://doi.org/10.1016/j.jenvman.2003.09.010 karimi, m., niknamfar, a. h., & niaki, s. t. a. (2019). an application of fuzzy-logic and grey-relational anp-based swot in the ceramic and tile industry. knowledge-based systems, 163, 581–594. doi: https://doi.org/10.1016/j.knosys.2018.09.020 khan, a. u., & ali, y. (2020). analytical hierarchy process (ahp) and analytic network process methods and their applications: a twenty year review from 2000-2019: ahp & anp techniques and their applications: twenty years review from 2000 to 2019. international journal of the analytic hierarchy process, 12(3), 369-459. doi: https://doi.org/10.13033/ijahp.v12i3.822 kopytov, e., demidovs, v., & petukhova, n. (2011). application of the analytic hierarchy process in development of train schedule information systems. international journal of the analytic hierarchy process, 3(2), 130-148. doi: https://doi.org/10.13033/ijahp.v3i2.113 https://doi.org/10.1080/19388160.2013.756771 https://ir.lib.uwo.ca/etd/2407 https://doi.org/10.1080/15567249.2012.673692 https://doi.org/10.1016/j.forpol.2012.03.005 https://doi.org/10.1080/02508060.2010.508929 https://doi.org/10.13033/isahp.y2007.022 https://doi.org/10.1016/j.jenvman.2003.09.010 https://doi.org/10.1016/j.knosys.2018.09.020 https://doi.org/10.13033/ijahp.v12i3.822 https://doi.org/10.13033/ijahp.v3i2.113 ijahp article: nguyen, truong/integral swot-ahp-tows model for strategic agricultural development in the context of drought: a case study in ninh thuan, vietnam international journal of the analytic hierarchy process 26 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.890 kou, g., ergu, d., peng, y., & shi, y. (2013). a new consistency test index for the data in the ahp/anp. in g. kou, d. ergu, y. peng, & y. shi, data processing for the ahp/anp (vol. 1, pp. 11–27). springer berlin heidelberg. doi: https://doi.org/10.1007/978-3-642-29213-2_2 kumar, a., sah, b., singh, a. r., deng, y., he, x., kumar, p., & bansal, r. c. (2017). a review of multi criteria decision making (mcdm) towards sustainable renewable energy development. renewable and sustainable energy reviews, 69, 596–609. doi: https://doi.org/10.1016/j.rser.2016.11.191 lavik, m. s., hardaker, j. b., lien, g., & berge, t. w. (2020). a multi-attribute decision analysis of pest management strategies for norwegian crop farmers. agricultural systems, 178, 102741. doi: https://doi.org/10.1016/j.agsy.2019.102741 lee, j., kim, i., kim, h., & kang, j. (2021). swot-ahp analysis of the korean satellite and space industry: strategy recommendations for development. technological forecasting and social change, 164, 120515. doi: https://doi.org/10.1016/j.techfore.2020.120515 lee, y.-h. (2013). application of a swot-fanp method. technological and economic development of economy, 19(4), 570–592. doi: https://doi.org/10.3846/20294913.2013.837111 linh, d. b. (2018). nang cao nang luc canh tranh cua nganh dich vu logistics viet nam trong boi canh hoi nhap kinh te quoc te [phd dissertation]. foreign trade university. lohan, a., ganguly, a., & kumar, c. (2020). "what’s foreign is better”: a fuzzy ahp analysis to evaluate factors that influence foreign product choice among indian consumers. international journal of the analytic hierarchy process, 12(3), 460-487. doi: https://doi.org/10.13033/ijahp.v12i3.743 malik, s. a. (2013). integrating ahp, swot and qspm in strategic planningan application to college of business administration in saudi arabia. international journal of the analytic hierarchy process, 13, 1-13. doi: https://doi.org/10.13033/isahp.y2013.049 mantogiannis, v. a., & katsigiannis, f. a. (2020). assessing real estate investment alternatives: a multi-criteria and multi-stakeholder decision aid tool. international journal of the analytic hierarchy process, 12(1), 136-165. doi: https://doi.org/10.13033/ijahp.v12i1.702 markovska, n., taseska, v., & pop-jordanov, j. (2009). swot analyses of the national energy sector for sustainable energy development. energy, 34(6), 752–756. doi: https://doi.org/10.1016/j.energy.2009.02.006 nam t. h., & trang l. t. h. (2019). the impact of drought in adaptability of farmer's garlic cultivation in ninh hai district, ninh thuan province. journal of forestry science and technology, 6, 170–176. https://doi.org/10.1007/978-3-642-29213-2_2 https://doi.org/10.1016/j.rser.2016.11.191 https://doi.org/10.1016/j.agsy.2019.102741 https://doi.org/10.1016/j.techfore.2020.120515 https://doi.org/10.3846/20294913.2013.837111 https://doi.org/10.13033/ijahp.v12i3.743 https://doi.org/10.13033/isahp.y2013.049 https://doi.org/10.13033/ijahp.v12i1.702 https://doi.org/10.1016/j.energy.2009.02.006 ijahp article: nguyen, truong/integral swot-ahp-tows model for strategic agricultural development in the context of drought: a case study in ninh thuan, vietnam international journal of the analytic hierarchy process 27 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.890 nam t. t., minh p. t., & tuyet b. t. (2018). general characteristic climate in ninh thuan province. vietnam journal of hydrometeorology, 7, 21-29 nasab, m. n., & azizi, m. (2014a). analysis of agricultural mechanization in zanjan province, iran: application of swot-ahp method. international symposium on the analytic hierarchy process, washington, dc, usa. doi: https://doi.org/10.13033/isahp.y2014.034 nasab, m. n., & azizi, m. (2014b). strategy for agricultural development in city of zanjan, iran: application of swot-ahp method. international symposium on the analytic hierarchy process, washington, dc, usa. doi: https://doi.org/10.13033/isahp.y2014.162 nathan, r. (2007). the future: a hydrological swot analysis. australasian journal of water resources, 11(2), 133–144. doi: https://doi.org/10.1080/13241583.2007.11465318 nguyen, t., & liu, c.-c. (2019). a new approach using ahp to generate landslide susceptibility maps in the chen-yu-lan watershed, taiwan. sensors, 19(3), 505. doi: https://doi.org/10.3390/s19030505 opricovic, s., & tzeng, g.-h. (2004). compromise solution by mcdm methods: a comparative analysis of vikor and topsis. european journal of operational research, 156(2), 445–455. doi: https://doi.org/10.1016/s0377-2217(03)00020-1 osuna, e. e., & aranda, a. (2007). combining swot and ahp techniques for strategic planning. 9th international symposium on analytic hierarchy process (isahp), 8. doi: https://doi.org/10.13033/isahp.y2007.012 ozkaya, g., & erdin, c. (2020). evaluation of smart and sustainable cities through a hybrid mcdm approach based on anp and topsis technique. heliyon, 6(10), e05052. doi: https://doi.org/10.1016/j.heliyon.2020.e05052 partani, t., marashi, s. v., & alishahi, m. h. (2013). using fuzzy analytic network process (fanp) in a swot analysis. global journal of computer science and technology, 13(2), 1-18. doi: http://computerresearch.org/index.php/computer/article/view/161 patnaik, r., & poyyamoli, g. (2015). developing an eco-industrial park in puducherry region, india ‒ a swot analysis. journal of environmental planning and management, 58(6), 976–996. doi: https://doi.org/10.1080/09640568.2014.904768 pham the dung, phung van khen, & tran van thanh (2012). research on planting techniques for suitable indigenous tree species for the arid regions of ninh thuan and binh thuan provinces (pp. 171–180) [scientific and technological research results for the period 2006-2010]. south vietnam forest science sub-institute, fsiv pournabi, n., janatrostami, s., ashrafzadeh, a., & mohammadi, k. (2021). resolution of internal conflicts for conservation of the hour al-azim wetland using ahp-swot and https://doi.org/10.13033/isahp.y2014.034 https://doi.org/10.13033/isahp.y2014.162 https://doi.org/10.1080/13241583.2007.11465318 https://doi.org/10.3390/s19030505 https://doi.org/10.1016/s0377-2217(03)00020-1 https://doi.org/10.13033/isahp.y2007.012 https://doi.org/10.1016/j.heliyon.2020.e05052 http://computerresearch.org/index.php/computer/article/view/161 https://doi.org/10.1080/09640568.2014.904768 ijahp article: nguyen, truong/integral swot-ahp-tows model for strategic agricultural development in the context of drought: a case study in ninh thuan, vietnam international journal of the analytic hierarchy process 28 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.890 game theory approach. land use policy, 107, 105495. doi: https://doi.org/10.1016/j.landusepol.2021.105495 saaty, r. w. (1987). the analytic hierarchy process—what it is and how it is used. mathematical modelling, 9(3–5), 161–176. doi: https://doi.org/10.1016/02700255(87)90473-8 saaty, t. l. (1978). modeling unstructured decision problems—the theory of analytical hierarchies. mathematics and computers in simulation, 20(3), 147–158. doi: https://doi.org/10.1016/0378-4754(78)90064-2 saaty, t. l. (1980). the analytic hierarchy process: planning, priority setting, resource allocation. new york: mcgraw-hill. saaty, t. l. (1996). the analytic network process. pittsburgh: rws publications. saaty, t. l. (2005) theory and applications of the analytic network process: decision making with benefits, opportunities, costs, and risks. pittsburgh: rws publications. saaty, t. l., & vargas, l. g. (2012). models, methods, concepts & applications of the analytic hierarchy process (vol. 175). springer us. doi: https://doi.org/10.1007/978-14614-3597-6 sevkli, m., oztekin, a., uysal, o., torlak, g., turkyilmaz, a., & delen, d. (2012). development of a fuzzy anp based swot analysis for the airline industry in turkey. expert systems with applications, 39(1), 14–24. doi: https://doi.org/10.1016/j.eswa.2011.06.047 shinno, h., yoshioka, h., marpaung, s., & hachiga, s. (2006). quantitative swot analysis on global competitiveness of machine tool industry. journal of engineering design, 17(3), 251–258. doi: https://doi.org/10.1080/09544820500275180 sidharth thakur (2010). swot – history and evolution. brighthub project management. http://www.brighthubpm.com/methods-strategies/99629-history-of-the-swot-analysis/ solangi, y. a., tan, q., mirjat, n. h., & ali, s. (2019a). evaluating the strategies for sustainable energy planning in pakistan: an integrated swot-ahp and fuzzy-topsis approach. journal of cleaner production, 236, 117655. doi: https://doi.org/10.1016/j.jclepro.2019.117655 solangi, y. a., tan, q., mirjat, n. h., & ali, s. (2019b). evaluating the strategies for sustainable energy planning in pakistan: an integrated swot-ahp and fuzzy-topsis approach. journal of cleaner production, 236, 117655. doi: https://doi.org/10.1016/j.jclepro.2019.117655 thanh n. k. (2020). combining swot model and analytic hierarchy process (ahp) to choose a business strategy for vinaship transport joint stock company. journal of marine science and technology, 61(1), 8. https://doi.org/10.1016/j.landusepol.2021.105495 https://doi.org/10.1016/0270-0255(87)90473-8 https://doi.org/10.1016/0270-0255(87)90473-8 https://doi.org/10.1016/0378-4754(78)90064-2 https://doi.org/10.1007/978-1-4614-3597-6 https://doi.org/10.1007/978-1-4614-3597-6 https://doi.org/10.1016/j.eswa.2011.06.047 https://doi.org/10.1080/09544820500275180 http://www.brighthubpm.com/methods-strategies/99629-history-of-the-swot-analysis/ https://doi.org/10.1016/j.jclepro.2019.117655 https://doi.org/10.1016/j.jclepro.2019.117655 ijahp article: nguyen, truong/integral swot-ahp-tows model for strategic agricultural development in the context of drought: a case study in ninh thuan, vietnam international journal of the analytic hierarchy process 29 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.890 thao, p. t. p., thu, p. m., giam, n. m., duan, h. d., & long, b. h. (2018). relationship between drought and rainfall due to tropical cyclone and depression in ninh thuan, vietnam. current journal of applied science and technology, 30(1), 1–8. doi: https://doi.org/10.9734/cjast/2018/44857 ton nu hai, a., & speelman, s. (2020). involving stakeholders to support sustainable development of the marine lobster aquaculture sector in vietnam. marine policy, 113, 103799. doi: https://doi.org/10.1016/j.marpol.2019.103799 truong, n. h. (2006). anh huong cua han han—hoang mac hoa va thoai hoa dat den san xuat nong nghiep tai tinh ninh thuan. vietnam journal of hydrometeorology, 546, 133– 139. truong, n. h. (2008). ung dung cac bien phap truyen thong va khoa hoc ky thuat moi nham khai thac, bao ve tai nguyen dat va nuoc, gop phan phong chong han o xa phuoc nam tinh ninh thuan. vietnam journal of hydrometeorology, 45–50. tuan, n. d., & an, n. l. (2016). research on drought factors, criteria, classification in ninh thuan binh thuan provinces and their prevention and mitigation measures. journal of water resources and environmental engineering, 30, 132-137. tuan, n. h., & canh, t. t. (2020). researching trends of rainfall change in ninh thuan in the context of climate change by the non-parametric method. science and technology development journal natural sciences, 5(1), 957-968. doi: https://doi.org/10.32508/stdjns.v5i1.939 tuan, n. h., & canh, t. t. (2021). analysis of trends in drought with the non-parametric approach in vietnam: a case study in ninh thuan province. american journal of climate change, 10(01), 51–84. doi: https://doi.org/10.4236/ajcc.2021.101004 tuan, n. h., & canh, t. t. (2021). research trend of change and future projected climate in ninh thuan province. vietnam journal of hydrometeorology, 722(2), 23–37. doi: https://doi.org/10.36335/vnjhm.2021(722).23-37 vinh, p. q. (2015). han han va van de bien doi khi hau tinh ninh thuan [drought and climate change in ninh thuan]. publishing house of natural science and technology, vietnam vinh, p. q., binh, n. t., & huong, b. t. t. (2013). assessing the impacts of agroclimatic conditions on annual trees in ninh thuan and binh thuan province. vietnam journal of earth sciences, 35(4), 365–373. weihrich, h. (1982). the tows matrix—a tool for situational analysis. long range planning, 15(2), 54–66. doi: https://doi.org/10.1016/0024-6301(82)90120-0 wickramasinghe, v., & takano, s. (2010). application of combined swot and analytic hierarchy process (ahp) for tourism revival strategic marketing planning: a case of sri lanka tourism. journal of the eastern asia society for transportation studies, 8, 954– 969. doi: https://doi.org/10.11175/easts.8.954 https://doi.org/10.9734/cjast/2018/44857 https://doi.org/10.1016/j.marpol.2019.103799 https://doi.org/10.32508/stdjns.v5i1.939 https://doi.org/10.4236/ajcc.2021.101004 https://doi.org/10.36335/vnjhm.2021(722).23-37 https://doi.org/10.1016/0024-6301(82)90120-0 https://doi.org/10.11175/easts.8.954 ijahp article: nguyen, truong/integral swot-ahp-tows model for strategic agricultural development in the context of drought: a case study in ninh thuan, vietnam international journal of the analytic hierarchy process 30 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.890 wilford, a.l., bodin, l.d., & gordon, l.a. (2020). using the analytic hierarchy process to assess the impact of internal control weaknesses on firm performance. international journal of the analytic hierarchy process, 12(2), 203-227. https://doi.org/10.13033/ijahp.v12i2.709 yavuz, f., & baycan, t. (2014). stakeholder-based decision making in integrated watershed management using swot and analytic hierarchy process combination. international journal of the analytic hierarchy process, 6(1), 3-34. doi: https://doi.org/10.13033/ijahp.v6i1.194 yu, h.-c., lee, z.-y., & chang, s.-c. (2005). using a fuzzy multi-criteria decision making approach to evaluate alternative licensing mechanisms. information & management, 42(4), 517–531. doi: https://doi.org/10.1016/j.im.2002.12.001 yuksel, i., & dagdeviren, m. (2007). using the analytic network process (anp) in a swot analysis ? a case study for a textile firm. information sciences, 177(16), 3364– 3382. doi: https://doi.org/10.1016/j.ins.2007.01.001 živkovi, ž., & mihajlovi, i. (2015). analytical network process in the framework of swot analysis for strategic decision making (case study: technical faculty in bor, university of belgrade, serbia). acta polytechnica hungarica, 12(7), 18. https://doi.org/10.13033/ijahp.v12i2.709 https://doi.org/10.13033/ijahp.v6i1.194 https://doi.org/10.1016/j.im.2002.12.001 https://doi.org/10.1016/j.ins.2007.01.001 ijahp article: packeer mohamed, baharom, deraman, tarawneh, yusof/software process assessment and certification: application of the analytic hierarchy process for priority determination international journal of the analytic hierarchy process 1 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.870 software process assessment and certification: application of the analytic hierarchy process for priority determination shafinah farvin packeer mohamed* 1 school of computing, universiti utara malaysia, sintok, 06010, kedah, malaysia, shafinah@uum.edu.my fauziah baharom school of computing, universiti utara malaysia, sintok, 06010, kedah, malaysia, fauziah@uum.edu.my aziz deraman faculty of ocean engineering technology and informatics, universiti malaysia terengganu, kuala terengganu, 21300, terengganu, malaysia, a.d@umt.edu.my omar tarawneh software engineering department, amman arab university, amman, jordan, o.husain@aau.edu.jo yuhanis yusof school of computing, universiti utara malaysia, sintok, 06010, kedah, malaysia, yuhanis@uum.edu.my abstract software certification involves assessing and certifying the quality of the software process based on multiple evaluation criteria where each criterion has different importance values on the quality of the software. however, the different importance values of the evaluation criteria have not been addressed in the existing software process certification models. a systematic technique is needed to ensure that the certification results are consistent, accurate and not made arbitrarily. to address this issue, the 1 corresponding author mailto:shafinah@uum.edu.my mailto:fauziah@uum.edu.my mailto:a.d@umt.edu.my mailto:yuhanis@uum.edu.my ijahp article: packeer mohamed, baharom, deraman, tarawneh, yusof/software process assessment and certification: application of the analytic hierarchy process for priority determination international journal of the analytic hierarchy process 2 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.870 extended software process certification (espac) model was introduced by adopting the analytic hierarchy process (ahp) technique to determine the priorities of the evaluation criteria. there were three main phases in this study: (a) theoretical study, (b) expert review and (c) focus group discussion. ultimately, a mutual agreement was achieved about the evaluation criteria and the ahp was shown to be a suitable technique to be employed in software process assessment and certification. furthermore, the acquired priorities were used as the ideal priorities for the espac model, which can be used by assessors during the assessment and certification process. the outcome of this study benefits researchers in the ahp and software process assessment fields. keywords: software engineering; software certification; espac model; priority; analytic hierarchy process 1. introduction the need for software in today’s world is constantly on the rise and its usage has become more critical than ever in every domain. therefore, software developers must be able to produce high quality software in a shorter time to market and rapidly react to changing requirements in order to compete in today’s business environment. however, although software developers claim that they produce high quality software, customer dissatisfaction still exists. this is evidenced by continuous reports of software failures that have affected various industries like banking, airlines and even social media platforms (computerworldukstaff, 2020). these incidents have caused huge loss and disruption of services. jones and bonsignour (2012) found that the software failure rate is among the highest compared to other products in recorded human history. also, a study conducted by the standish group discovered 71% of software projects were failures or were being challenged in 2015 (meier, 2017). concern about the quality of software has triggered doubt among customers, particularly in terms of investing in such projects. one way to satisfy customers’ requirements on the quality of software is through certification (ferreira et al., 2019; pietrantuono & russo, 2018; voas & laplante, 2018; darwish, 2016; baharom et al., 2011; heck et al., 2010). certification is the process of giving a written assurance that a process, product, or service complies with a criterion and is performed by a third party (rae et al., 1995). through software certification, customers can have greater confidence in the quality of the software that they are going to invest in because certification involves independent assessment; therefore, it is assumed that the risk of failure is reduced (rae et al., 1995; sun-jen & wen-ming, 2006). a study conducted by baharom et al. (2005) with software practitioners in malaysia discovered that software certification is certainly required to confirm the quality of the software. more recently, ferreira et al. (2019) and pietrantuono and russo (2018) highlighted the importance of certification. according to voas (1998), the three approaches to certify software’s quality are process, product, and personnel. studies by heck et al. (2010) and yahya (2007) focused on the product approach. similarly, gualo et al. (2020) worked towards certification of the functional suitability for master data management applications. nevertheless, the quality of newly developed software cannot be immediately determined by using the product approach because the software needs to be used for a period of time before the quality can be determined (baharom et al., 2011; heck et al., 2010). it is different with the ijahp article: packeer mohamed, baharom, deraman, tarawneh, yusof/software process assessment and certification: application of the analytic hierarchy process for priority determination international journal of the analytic hierarchy process 3 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.870 process approach, where the software process is assessed from the beginning of the software development. this enables customers to know the capability of an organization to produce high quality software, which will then help them decide whether to invest or not in any of the software development projects. in essence, as mentioned by voas and laplante (2018), the best way to certify a software product is through its process. the software process is a “set of activities undertaken to manage, develop and maintain software systems in order to produce a software system, executed by a group of people organized according to a given organizational structure and counting on the support of techno-conceptual tools” (acuna et al., 2000, p.1). the underlying idea behind this is that by having a well-defined certified development process, the produced software will be of a guaranteed quality. as highlighted by deming (1982) and humphrey (1989), the quality of the end product is determined by the quality of the process employed to produce the product. therefore, by using the process approach, customers are assured that the software process has been implemented effectively and efficiently. a considerable amount of literature has been published on models and standards that perform assessments on the quality of the software process, for example, the capability maturity model integrated (cmmi) (cmmi institute, 2018) and iso/iec 15504 (mas et al., 2012; galin, 2004). however, as indicated by acuna et al. (2000) the aim of these existing models and standards is more to assess and improve the software process rather than provide a mechanism for certification. therefore, baharom (2008) developed a certification model that assesses the software process known as software process assessment and certification (spac). however, the existing models are more concerned with the conventional software development process, which emphasizes that each phase of software development must be completed before going on to the next. the requirements in the early stage of software development must be completed before moving onto design, coding and testing. this type of software process also focuses on producing documentation (sommerville, 2007). however, to survive in today’s business environment which demands high quality and secured software, incorporating agile and secure software processes has become essential in order to produce higher quality software faster (ansari et al., 2018; pressman, 2010; sommerville, 2007). therefore, the existing software process and the certification model are enhanced by incorporating the agile and secure software development processes. the enhanced model is the extended software process assessment and certification model (espac) (packeer mohamed et al., 2015). moreover, the existing software process certification models and standards also lack an appropriate synthesis technique. baharom (2008) observed that the priorities of evaluation criteria are not considered in the existing software process certification models and standards. priorities of the evaluation criteria need to be considered especially when it has been conclusively shown that an assessment involving multiple criteria will have different importance. therefore, these criteria should be prioritized (saaty & de paola, 2017; saaty, 2008). additionally, the process of assigning priorities to the evaluation criteria is significant, particularly when qualitative information is needed from the decision-makers (triantaphyllou & mann, 1995). despite its importance, little attention has been given to the consideration of priorities for the assessed criteria in existing software process assessment and certification. consequently, this issue was addressed in the espac model in order to produce more consistent and better quality certification results. the software process is assessed based on five main criteria, namely process, people, technology, project constraints, and working ijahp article: packeer mohamed, baharom, deraman, tarawneh, yusof/software process assessment and certification: application of the analytic hierarchy process for priority determination international journal of the analytic hierarchy process 4 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.870 environment. these factors are further decomposed into sub-factors and evaluation criteria. the analytic hierarchy process (ahp) was employed as the synthesis technique to obtain the priorities for the evaluation criteria in the espac model (saaty & de paola, 2017; saaty, 2008; saaty, 1990). the ahp is one of the most outstanding multi-criteria decision-making (mcdm) techniques. it provides a mechanism for decision-makers to break a problem into a hierarchy, prioritize the evaluation criteria and find a suitable solution for the problem. unlike other mcdm techniques, the ahp is rooted in determining the priorities of criteria (saaty & de paola, 2017; saaty, 2008; saaty, 1990). the main benefit of utilizing the ahp technique is that it offers systematic steps to synthesize information through a structured hierarchy. the hierarchy contains the criteria and the sub-criteria, which can help decision-makers understand and simplify a problem by providing better focus during the priority allocation for both the criteria and the sub-criteria (ishizaka & labib, 2011). this is the advantage gained in this study as it involves numerous factors, sub-factors, and evaluation criteria. furthermore, by using the hierarchy, the criteria are systematically organized. in addition, with the use of the ahp, judgment accuracy can be improved because by integrating relative numbers, there should be no or only minimal loss of accuracy (crostack et al., 2007). more importantly, this technique is appropriate for group decision-making because it allows favorable agreement among the group members (marjani et al., 2012; lai et al., 2002). this is important as the study involves group decision-making to determine priorities. above all, the judgments made in the ahp are more accurate and consistent as this technique provides a mechanism to test the consistency of the judgements. considering the aforementioned advantages, this study adopted the ahp as its synthesis technique. the seven steps involved in implementing this technique will be explained later. even though the ahp has been used widely in various areas, this technique has not been applied in the field of software process assessment and certification, and therefore this study contributes to this field of study. this paper describes the ahp in section 2 and the methodology of the study in section 3. a thorough explanation of the priority determination using the ahp is in section 4. the outcomes of the ahp implementation through the focus group discussion is in section 5 and ends with the conclusion. 2. analytic hierarchy process many techniques can be utilized to make decisions that involve multi-criteria. one of the most utilized techniques is the analytic hierarchy process (ahp) (saaty & de paola, 2017; saaty, 2008; saaty, 1990). this technique enables decision-makers to represent decision-making problems, which involve multiple criteria in a hierarchy, commonly comprised of three levels. the first level refers to the overall goal for a problem, the second level represents the evaluation criteria, and the third level is comprised of various alternatives. however, in this study, the hierarchy only contains the goal and several levels of evaluation criteria because it does not involve making decisions from among a number of alternatives. the priorities are obtained through pairwise comparisons, which are performed among the evaluation criteria of each level. then, a normalized ranking is obtained by applying the eigen value method. other simpler methods that can be used include the normalization of row average (nra), normalization of the reciprocal sum of ijahp article: packeer mohamed, baharom, deraman, tarawneh, yusof/software process assessment and certification: application of the analytic hierarchy process for priority determination international journal of the analytic hierarchy process 5 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.870 columns (nrc), average of normalized columns (anc), and normalization of the geometric mean of the rows (ngm) (hsiao, 2002). this study adopted the ngm method. the ahp has been applied extensively in various areas such as selection, ranking, and evaluation. for example, baidya et al. (2018) applied the ahp to select the most appropriate maintenance technique in manufacturing by considering the strategic, planning and operational criteria, while ali et al. (2018) utilized the ahp to rank suitable sites for wind farm installation using multiple criteria. kumari and shylaja (2019) utilized the ahp in the routing protocol. al-tarawneh (2014), kunda (2003), zhou and liang (2013), and chen et al. (2013) evaluated the component-based software, the network course in china, and the potential outsourcing partner, respectively. padumadasa (2009) utilized the technique to select and evaluate tenders, while kumar et al. (2019) applied the ahp to assess the quality of soil. more recently, moradi (2022) applied ahp with balanced scorecard and topsis to evaluate the performance of university faculty. in the field of software process assessment, the ahp has been utilized for software process improvement (jung, 2001). this technique has also been widely used in combination with other multi-criteria decision-making techniques. for example, the ahp has been applied with promethee, topsis and wsm (mokhtar et al., 2017). animah and shafiee (2019) applied the ahp with promethee to select the best strategy for performing maintenance for shipboard machinery systems. moreover, the ahp was combined with topsis to determine the best fresh fruit bunches in hambali and rahman’s (2017) research. more recently, zaidan et al. (2020) applied the ahp with topsis to select a qualified programmer. these studies used the ahp to obtain the priority and continued with other appropriate multi-criteria decision-making techniques. similarly, the current study adopted the ahp to obtain the priorities for the evaluation. 3. methodology this study was conducted in three main phases, namely, theoretical study, expert review, and a focus group discussion. 3.1 theoretical study in this phase, the existing literature including journals, books, proceeding papers and dissertations, were reviewed. the main aim of this phase was to obtain the factors that can influence software quality. the factors were analyzed and classified in order to form a hierarchy that could be used in the ahp implementation. they were classified into five main factors including process, people, project constraints, technology, and environment. each of these factors was further decomposed into sub-factors and evaluation criteria, as discussed in section 4 (step 1). 3.2 expert review in this phase, the identified factors were verified through expert review. the main aim of this phase was to verify the factors, sub-factors and the evaluation criteria derived from the theoretical study. the expert review was used because of its simplicity, low cost, and quick completion. moreover, it is accepted as a significant way to detect and remove defects (komuro & komoda, 2008). the following three steps were executed in the expert review: ijahp article: packeer mohamed, baharom, deraman, tarawneh, yusof/software process assessment and certification: application of the analytic hierarchy process for priority determination international journal of the analytic hierarchy process 6 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.870 i. identify the experts the experts were chosen from among academicians (knowledge experts) by following the characteristics of experts suggested by hallowell and gambatese (2010). the characteristics include: (a) are currently attached to the field of the study under examination, (b) hold an advanced degree (phd.), (c) are faculty members in an accredited university, (d) authorship, and (e) have at least 5 years of experience. additionally, software practitioners were also included as experts to perform the verification. their insight is important since they can give feedback based on their real life experience as domain experts. they were chosen using purposive sampling (liamputtong, 2011). four characteristics were used to choose the domain experts: (a) are agile software practitioners, (b) have experience in secure software process, (d) have more than 3 years of software development experience. ii. determine the verification criteria the factors, sub-factors and evaluation criteria were verified for their comprehensiveness, understandability, accurateness, and organization. these criteria were adapted from previous studies (al-tarawneh, 2014; behkamal et al., 2009; kunda, 2003). the experts provided their feedback via a checklist. iii. collect and analyze the feedback the experts’ feedback was collected and analyzed for further improvement. 3.3 focus group discussion the outcomes of the expert review revealed that the factors, sub-factors and evaluation criteria were acceptable. these factors were used to construct the hierarchy for the ahp implementation. to implement the ahp, a focus group discussion was performed by adapting the guidelines from martakis and daneva (2013), mazza and berre (2007), and kontio et al. (2008). focus group discussions have been applied extensively in the software engineering field for evaluation or gleaning practitioners’ experiences (daneva & ahituv, 2011; mazza & berre, 2007; kontio et al., 2008). the most appropriate participants for this purpose are the software practitioners since they have vast experience with the software process. three main stages were involved, as depicted in figure 1. figure 1 focus group execution 1 : p la n define objectives identify and recruit participants identify and book the meeting place prepare materials remind participants 2 : c o n d u c t informal conversation brief about the espac model and the ahp implement the ahp 3 : r e p o r t in g r e s u lt s prepare technical report email the report to participants get feedback ijahp article: packeer mohamed, baharom, deraman, tarawneh, yusof/software process assessment and certification: application of the analytic hierarchy process for priority determination international journal of the analytic hierarchy process 7 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.870 stage 1. plan proper planning was needed to conduct an effective focus group session. first, the objective of the focus group was determined. the objective was to implement the ahp to determine the priorities of the factors that influence software quality. then, the participants were identified using purposive sampling based on several common characteristics explained in section 3.2. the participants were solicited by phone, email and facebook invitations. next, the meeting venue was identified, which was a hotel in the kuala lumpur area as it was central for all participants. more importantly, the meeting room provided in the hotel was considered neutral since it did not have any influence on the participants, which could affect the discussion. next, the materials needed for the session were prepared which included presentation slides, documents, cards, certificates of appreciation as well as an excel file, which automated the priority calculation. finally, the participants were contacted to remind them about the session. this was to confirm their attendance so that they would not miss the session and let them know that their attendance was important. stage 2. conduct on the planned date and time, the focus group session was organized and seven participants attended; this is an appropriate number of participants needed to conduct a focus group (morgan, 1998). initially, the participants were engaged in an informal conversation to create rapport before the discussion started. this was to make them feel comfortable and relaxed and help the participants get to know each other since they were from different companies. next, the discussion started with some ice-breaking conversation within the group which further facilitated the moderator and the participants getting to know each other better. this was important to build rapport and create group cohesion (liamputtong, 2011). a briefing on the objectives of the focus group as well as the espac model, the ahp technique and its implementation in the focus group discussion, then followed. the participants started to communicate and asked questions about the issues that were not clear to them. then, the ahp was implemented. this study adopted the group ahp technique, in which decisions on pairwise comparisons were made as a group. to simplify the judging process in a group, the planning poker technique was adapted. this technique is widely used in agile software development to perform task estimations among software developers (dyba et al., 2014). the participants were provided with guidelines on the ahp technique and a list of pairwise comparisons. table 1 shows an excerpt of the pairwise comparisons list. it comprises the factors in level one of the hierarchy. the participants were also provided with nine cards that contained numbers from 1-9 used to make pairwise comparisons. ijahp article: packeer mohamed, baharom, deraman, tarawneh, yusof/software process assessment and certification: application of the analytic hierarchy process for priority determination international journal of the analytic hierarchy process 8 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.870 table 1 pairwise comparisons list for factors in level one of the hierarchy factors equality scale process is [more / equally/ less] important than/to people by a factor of 1 2 3 4 5 6 7 8 9 process is [more / equally/ less] important than/to technology by a factor of 1 2 3 4 5 6 7 8 9 process is [more / equally/ less] important than/to project constraint by a factor of 1 2 3 4 5 6 7 8 9 process is [more / equally/ less] important than/to environment by a factor of 1 2 3 4 5 6 7 8 9 people is [more / equally/ less] important than/to technology by a factor of 1 2 3 4 5 6 7 8 9 people is [more / equally/ less] important than/to project constraint by a factor of 1 2 3 4 5 6 7 8 9 people is [more / equally/ less] important than/to environment by a factor of 1 2 3 4 5 6 7 8 9 technology is [more / equally/ less] important than/to project constraint by a factor of 1 2 3 4 5 6 7 8 9 technology is [more / equally/ less] important than/to environment by a factor of 1 2 3 4 5 6 7 8 9 project constraint is [more / equally/ less] important than/to environment by a factor of 1 2 3 4 5 6 7 8 9 to implement the planning poker, the moderator raised each of the pairwise comparisons one by one. the participants discussed and exchanged their experiences about the evaluation criteria that were being compared. then, they used the card to choose the importance value for the pairwise comparison. they kept their chosen value to themselves until everyone had chosen one and after everyone was ready, the cards were revealed concurrently. the value of the pairwise comparison was chosen if a consensus was reached; otherwise, the majority vote was taken. however, a compromise among the group was reached if neither a consensus or majority was reached. if this process was still not successful at choosing a value, then the geometric mean was used to obtain the average. each of the evaluation criteria were compared and the values were entered into the excel file, which had been prepared earlier. therefore, the cr value could be obtained once the pairwise comparisons were made for each pairwise comparison matrix. when there was inconsistency, the judgment process was repeated. stage 3. reporting results after completing the focus group session, a technical report was prepared and sent to the participants via email. they provided the feedback that they were satisfied with the priorities of the factors that influenced software quality obtained during the focus group discussion. the next section provides an example of the ahp implementation to provide a better understanding of the process and uses an example based on the theoretical study conducted and the ahp implementation during the focus group discussion. ijahp article: packeer mohamed, baharom, deraman, tarawneh, yusof/software process assessment and certification: application of the analytic hierarchy process for priority determination international journal of the analytic hierarchy process 9 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.870 4. priority determination using the ahp technique figure 2 illustrates the seven main steps used to obtain the priorities for the espac model evaluation criteria. figure 2 steps of the ahp to determine priorities step 1. identify the factors that influence software process quality first, the factors that influence the quality of the software process were determined through the theoretical study. as mentioned earlier, deming (1982) stated that the quality of the end product is determined by the quality of the process employed to produce the product. therefore, the software will be good quality as a result of a good quality process. according to wang and leung (2001), the software process can be categorized into three perspectives as follows: organization, development and management. significantly, this highlights that the software process is not simply the methodology followed to develop the software but also the environment where the software is developed. as mentioned by akbar et al. (2017), the use of the proper software process model greatly influences the software quality that meets customers’ needs within the stipulated time and budget. the software process is assessed based on its effectiveness and efficiency. moreover, according to gasston (1996), the effectiveness of the software process is based on human, ijahp article: packeer mohamed, baharom, deraman, tarawneh, yusof/software process assessment and certification: application of the analytic hierarchy process for priority determination international journal of the analytic hierarchy process 10 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.870 management, economics, and technology. subsequently, assessing the process alone is not sufficient since the software process is implemented by humans. as argued by destefanis et al. (2016), developers are one of the key success factors of the software process because they are the ones who are involved throughout the software development. apparently, with the existence of agile software development, customers, organizations and developers play an essential role in determining the success of a project in addition to the technology used and the working environment (aldahmash et al., 2017; abd el hameed et al., 2016; chow & cao, 2008). as a result, in this study, other factors than simply the software process that can influence software quality were taken into consideration. the effectiveness of the software process was measured based on the completeness, consistency, and accuracy of the process. the software process must fulfill customers’ expectations by having good quality people, appropriate technology, and a stable working environment. on the other hand, efficiency was determined by the capability of producing the software within the expected time and budget (baharom et al., 2011). overall, there were five factors used to determine the quality of the software process as listed below: 1. software process – the quality of the software process carried out when developing the software. 2. people – the quality of the developers, organizations and customers involved with the software development. 3. technology – the technology used during the software development. 4. environment the safety and comfort provided for staff at the location the software is developed. 5. project constraints – the ability to produce the software within the scheduled time and stipulated budget. these factors were verified by experts that included three academicians and seven software practitioners. the factors, sub-factors and the evaluation criteria were used for the next step, which was to structure the hierarchy for the ahp implementation. step 2. structure factors in hierarchy the five factors identified in step 1 could not be measured directly; therefore, they were decomposed into measurable sub-factors and evaluation criteria as illustrated in figure 3. they were organized in a hierarchical structure adopted from the ahp technique. the goal was positioned at the first level of the hierarchy followed by the factors, sub-factors and the evaluation criteria in the subsequent levels. ijahp article: packeer mohamed, baharom, deraman, tarawneh, yusof/software process assessment and certification: application of the analytic hierarchy process for priority determination international journal of the analytic hierarchy process 11 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.870 software process quality process people project constraint technology environment software development management support re des code test comp cons acc prm comp cons acc comp cons acc comp cons acc chm secm rim comp cons acc comp cons acc comp cons acc comp cons acc dev custorga team comm inv inv sch budg trg sti doc comp acc comp comp acc) wenv stdto&te comp comprem comp ips mgmnt skills tech skills know exper safety comf figure 3 hierarchy for software process quality assessment step 3. construct pairwise comparison matrices to perform the pairwise comparisons, pairwise comparison matrices were constructed for the assessed evaluation criteria. in the pairwise comparison matrix, the sibling criteria at each level were compared in pairs to judge their importance. they were organized in a matrix of two dimensions (square matrix) whereby the compared criteria were sorted vertically in the first column and horizontally in the first row of the matrix. for example, the five factors that influence the quality of the software located in the first level of the hierarchy (see figure 3) were compared using one pairwise comparison matrix. the factors were process (proc), technology (tech), people, project constraints (pc), and environment (env). a similar process was performed for the other factors, subfactors and the evaluation criteria for the entire hierarchy. one pairwise matrix was constructed for each sub-tree with more than one factor/sub-factor/evaluation criteria. there was a total of 18 pairwise comparison matrices as summarized in table 2. table 2 summary of the pairwise comparison matrices for the espac model description level 1 level 2 level 3 level 4 total number of criteria 5 11 26 28 70 number of pair wise comparison matrices 1 4 5 8 18 legend: acc: accuracy budg: budget chm: change management code: coding comf: comfort comp : completeness cons: consistency cust: customer des: design dev: developer doc:documentation exper: experience inv: involvement ips: interpersonal skills know: knowledge sch: schedule secm: security management std: standard & procedure sti: staff intiative team comm:team commitment tech skills: technical skills test: testing to&te: tools & techniques trg: training wenv: working environment mgmnt skills: management skills orga: organization prm: project management re: requirement engineering rem: resource management rim: risk management e v .c ri te ri a s u b -f a c to rs g o a l f a c to rs ijahp article: packeer mohamed, baharom, deraman, tarawneh, yusof/software process assessment and certification: application of the analytic hierarchy process for priority determination international journal of the analytic hierarchy process 12 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.870 step 4. perform judgments in each pairwise comparison matrix judgments must be performed for each two criteria in the matrix. to do this, the relative importance of each two criteria in the matrix was determined using the scale of 1 to 9 created by saaty (1990). for example, the importance was determined by making the following comparison: c1 is more/equally/less important than/to c2 by a factor of 2/3/4/5/6/7/8/9. the number of pairwise comparisons needed for each matrix was determined by equation (1). pairwise comparisons in each matrix = n (n-1)/2 (1) where n is the number of criteria in the matrix. as an example, five factors influence software quality; therefore, n = 5. accordingly, there were ten pairwise comparisons in this pairwise comparison matrix. this number was obtained using equation (1), where 5 (5-1) / 2 = 10 pairwise comparisons. table 3 depicts the pairwise comparison for the first level of the hierarchy, which consists of five criteria. the diagonal elements of the matrix were assigned a value of 1, since aij=1 when i=j. comparisons were made only for the upper triangular matrix (colored columns), since the lower triangular matrix was comprised of the reciprocals of these values. for example, in the first row of the matrix the process was considered five times more important than the technology and the people, while the process was four times more important than the environment. additionally, the project constraints were considered equally important as the process. on the other hand, the people and the project constraints were four times more important than the technology, while environment was three times more important than the technology. these are represented in the second row of the matrix. table 3 pairwise comparison factors in first level of hierarchy factor proc tech people pc env proc 1 5 5 1 4 tech 1/5 1 1/4 1/4 1/3 people 1/5 4 1 1/3 4 pc 1 4 3 1 5 env 1/4 3 1/4 1/5 1 step 5. synthesize pairwise comparisons once the pairwise comparisons were complete, they were then synthesized to determine the priority. for this purpose, the normalization of the geometric mean (ngm) was utilized (hsiao, 2002). the n elements in each row were multiplied, and the n th root was calculated to find the priority. then, the resulting numbers were normalized. this is shown in equation (2). ijahp article: packeer mohamed, baharom, deraman, tarawneh, yusof/software process assessment and certification: application of the analytic hierarchy process for priority determination international journal of the analytic hierarchy process 13 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.870 wi= (∏ aij n j=1 ) 1/n / ∑ (∏ aij n j=1 ) 1 nn i=1 (2) where wi = priority of the evaluation criteria i i = 1,2….,n j = 1,2…..,n aij = pairwise comparison in matrix ij considering the pairwise comparisons made in table 3 and equation 2, the priority of the process was calculated as below: priority for process = (1*5*5*1*4) 1/5 / (2.512+0.334+1.1013+2.268+0.52) = 2.512 / 6.647 = 0.378 similarly, the priority was calculated for the remaining factors. table 4 shows the priorities obtained for the factors in the first level of the hierarchy. table 4 priority for factors in first level of hierarchy factor proc tech people pc env n th root values priority proc 1 5 5 1 4 2.512 0.378 tech 1/5 1 1/4 1/4 1/3 0.334 0.05 people 1/5 4 1 1/3 4 1.013 0.152 pc 1 4 3 1 5 2.268 0.341 env 1/4 3 1/4 1/5 1 0.52 0.078 total 6.647 1.000 step 6. perform consistency analysis to eliminate inconsistency in the judgments, the consistency ratio (cr) was calculated for each of the pairwise comparison matrices. this is the advantage of using ahp, whereby the consistency of the decisions can be revealed. the cr value should be less than 0.1 to indicate consistency (saaty, 1990). equations (3) and (4) are used to obtain the cr value: cr = consistency index (ci) / random index (ri) (3) where ci is calculated using this formula: ci = (λmax – n) / (n-1) (4) where n = number of evaluation criteria in the matrix ijahp article: packeer mohamed, baharom, deraman, tarawneh, yusof/software process assessment and certification: application of the analytic hierarchy process for priority determination international journal of the analytic hierarchy process 14 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.870 λmax = the average value of consistency vectors first, the eigenvalue (λmax) was obtained. the priority of each factor (see table 4) and the pairwise comparisons were arranged in table 5. then, the weighted sum vectors were obtained for each row by multiplying the appropriate priority with the pairwise comparison. the results of the multiplication were then summed to obtain the weighted sum vectors for each row. table 5 calculation of weighted sum vectors prio rity (proc) pwc (proc) prio rity (tech) pwc (tech) prio rity (people) pwc (people) prio rity (pc) pwc (pc) prio rity (env) pwc (env) weighted sum vectors 0.378 1 0.05 5 0.152 5 0.34 1 1 0.078 4 2.041 1/5 1 1/4 1/4 1/3 0.275 1/5 4 1 1/3 4 0.853 1 4 3 1 5 1.765 1/4 3 1/4 1/5 1 0.429 next, the consistency vectors were calculated by dividing the weighted sum vectors with the respective priority and are shown in table 6. table 6 calculation of consistency vectors criteria weighted sum vectors priority consistency vectors proc 2.041 0.378 5.399 tech 0.275 0.05 5.5 people 0.853 0.152 5.612 pc 1.765 0.341 5.176 env 0.429 0.078 5.5 then, the average of the consistency vectors was calculated to obtain λmax. next, the ci value was calculated. equation 4 was used to obtain the ci value. finally, equation 3 was utilized to obtain the cr value. the ri value was determined based on the value of n, as provided by saaty (1990). for the current pairwise comparison matrix, since the n value was 5, and ri was 1.12, the cr value obtained is 0.098. since the cr value was less than 0.1, the pairwise comparison made was considered to be consistent. however, if the cr value was greater than 0.1, then the judgments on the criteria would be considered inconsistent and need to be performed again (return to step 4). the cr value for each pairwise comparison matrix must be consistent before the global priority can be obtained. step 7. obtain the global priority the priorities attained from the preceding steps are the local priorities. the final priorities are known as the global priority, and are acquired by multiplying the local priority of a ijahp article: packeer mohamed, baharom, deraman, tarawneh, yusof/software process assessment and certification: application of the analytic hierarchy process for priority determination international journal of the analytic hierarchy process 15 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.870 child by its parents’ local priority (the calculation starts from the lowest level and goes to the first level of the hierarchy). equation (5) shows how to obtain the global priority. 𝐺𝑊𝑖 = 𝐿𝑊𝑖 ∗ ∏ 𝑃𝑗 𝑛 𝑗=1 (5) where: gwi = global priority for i th evaluation criteria lwi = local priority for i th evaluation criteria pj = local priority for j th parents i = 1,2……,n j = 1,2……,n for example, equation 5 is used to obtain the global priority for the completeness of requirement engineering (gwcompre). the local priority for the completeness of requirement engineering is multiplied with the local priorities of its parents (requirement engineering, software development, and process) as illustrated in figure 4. software process quality process 0.378 software development 0.659 -- requirement engineering 0.703 software design 0.128 completeness 0.691 consistency 0.16 accuracy 0.149 completeness 0.691 consistency 0.16 accuracy 0.149 -- -- figure 4 part of hierarchy with local priority likewise, this calculation is performed to obtain the global priority for other factors/subfactors/evaluation criteria. the complete local priorities and the global priorities are listed in table 8 (section 5). these global priorities are the ideal priorities suggested by the espac model. the global priority for completeness of requirement engineering is calculated as: gwcompre = 0.691 *0.703 * 0.659 * 0.378 = 0.121 5. outcomes from the ahp implementation this section discusses the outcomes obtained from the ahp implementation through the focus group, namely the participants’ background and the priorities obtained. ijahp article: packeer mohamed, baharom, deraman, tarawneh, yusof/software process assessment and certification: application of the analytic hierarchy process for priority determination international journal of the analytic hierarchy process 16 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.870 5.1 participants’ backgrounds the seven participants were agile practitioners from distinct organizations located in the kuala lumpur area. they had different positions, such as team leader and scrum master. the participants had more than three years of experience in software development. four of them worked in large companies with more than 250 employees. most of them seemed to favor scrum and extreme programming (xp). table 7 recapitulates the background of the participants. table 7 participants’ background id positions size of organization experience in software development agile methods used a team leader >250 11-20 fdd b team leader 51-250 6–10 scrum c architect >250 11-20 scrum, xp d programmer 51-250 6–10 scrum, xp e programmer >250 6-10 scrum, xp f application lifecycle manager 20-50 11-20 scrum, tdd g scrum master >250 6-10 scrum, xp, am, lean 5.2 priorities obtained the priorities that were obtained through the planning poker technique are provided in table 8. the table consists of the local priorities for all the factors/sub-factors/evaluation criteria as well as the global priorities. the global priorities are used as the ideal priority for the espac model, which can be used by potential users of the model. however, if the assessors do not agree with those values, they can perform the ahp technique again and obtain their own priorities, which might be more suitable for their organizations. table 8 priority values factors / sub-factors / (local priorities) evaluation criteria global / ideal priorities process software development (0.659) requirement engineering (0.703) completeness (0.691) 0.121 consistency (0.16) 0.028 accuracy (0.149) 0.026 software design (0.128) completeness (0.691) 0.022 consistency (0.16) 0.005 accuracy (0.149) 0.005 coding (0.071) completeness (0.691) 0.012 consistency (0.16) 0.003 accuracy (0.149) 0.003 testing completeness (0.691) 0.017 ijahp article: packeer mohamed, baharom, deraman, tarawneh, yusof/software process assessment and certification: application of the analytic hierarchy process for priority determination international journal of the analytic hierarchy process 17 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.870 factors / sub-factors / (local priorities) evaluation criteria global / ideal priorities (0.378) (0.097) consistency (0.16) 0.004 accuracy (0.149) 0.004 management (0.156) project management (0.25) completeness (0.333) 0.005 consistency (0.333) 0.005 accuracy (0.333) 0.005 change management (0.25) completeness (0.333) 0.005 consistency (0.333) 0.005 accuracy (0.333) 0.005 security management (0.25) completeness (0.333) 0.005 consistency (0.333) 0.005 accuracy (0.333) 0.005 risk management (0.25) completeness (0.333) 0.005 consistency (0.333) 0.005 accuracy (0.333) 0.005 support 0.185 staff initiative (0.499) completeness (1.0) 0.035 documentation (0.067) completeness (1.0) 0.005 resource management (0.249) completeness (1.0) 0.017 training (0.185) completeness (1.0) 0.013 technology 0.05 tools & techniques (0.2) completeness (1.0) 0.01 standard & procedures (0.8) completeness (1.0) 0.04 people 0.152 developers (0.084) interpersonal skills (0.039) 0.00049 management skills (0.054) 0.00069 technical skills (0.169) 0.00216 knowledge (0.166) 0.00212 experience (0.164) 0.00209 team commitment (0.409) 0.00522 organization (0.211) involvement (1.0) 0.03207 customers (0.705) involvement (1.0) 0.10716 project constraint 0.341 budget (0.5) accuracy (1.0) 0.1705 schedule (0.5) accuracy (1.0) 0.1705 ijahp article: packeer mohamed, baharom, deraman, tarawneh, yusof/software process assessment and certification: application of the analytic hierarchy process for priority determination international journal of the analytic hierarchy process 18 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.870 factors / sub-factors / (local priorities) evaluation criteria global / ideal priorities environment 0.078 working environment (1.0) safety (0.8) 0.062 comfort (0.2) 0.016 based on the opinions of the participants, the evaluation criteria utilized in this study were sufficient to assess the software process. according to table 8, the most important factor in producing high quality software is the development process, followed by the project constraints, technology, environment, and people in descending order. even though in the agile software process people are the main contributors for producing high quality software (aldahmash et al., 2017; abd el hameed et al., 2016; chow & cao, 2008), the participants of this study mutually agreed that the other factors were more important. furthermore, software development was rated as the highest priority and was followed by the support and the management processes. this is in line with the work of akbar et al. (2017), where the software process was considered the most important to ensure software quality. for the software development phases, the requirement engineering phase had the highest priority and was followed by the software design and testing. coding had the lowest priority. requirement engineering was considered highly important because the subsequent steps in software development depend on the correctness of the requirements (pressman, 2010). the requirements are considered as the groundwork in software production, which will be used to verify the users’ requirements and act as an indicator to show that the proposed requirements are fulfilled (heikkil¨a et al., 2015). the management processes, which are project management, risk management, security management, and change management, were all rated with equal priority. the management of a project as well as the software development is essential (yaghoobi, 2018; ahimbisibwe et al., 2017) not only in reference to managing the project but also the risk, security and change management. therefore, they are equally prioritized. for the development process, which consists of software development, management and support processes, the completeness criterion obtained the highest priority. this shows that the participants emphasized completing the process while developing the software. moreover, for the support process, the participants gave the highest importance to staff initiatives, followed by resource management, training, and documentation. the least priority was given to documentation. this might be because they were more concerned about creating the end product. one value of the agile manifesto (2001) is “working software over comprehensive documentation”. however, even though agile software development focuses on producing working software, minimal documentation is still needed for further references and maintenance. wagenaar et al. (2018) concluded that documents in the agile software development are essential to providing governance to the team. a means of internal communication could be used for quality control and might be needed by other parties as a reference. additionally, for people who were involved during the software development, the customers were the highest priority, followed by the organization, and the lowest was software practitioners. this shows that the participants’ focus on customer satisfaction aligns with the agile principles (agile manifesto, 2001). similarly, the outcome from a study conducted by sambinelli and borges (2019) revealed that when customers were valued during software development, they observed an improvement in the customers’ communication, satisfaction, and collaboration. this can help increase customer’s ijahp article: packeer mohamed, baharom, deraman, tarawneh, yusof/software process assessment and certification: application of the analytic hierarchy process for priority determination international journal of the analytic hierarchy process 19 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.870 satisfaction and loyalty to the organization. for software practitioners, team commitment was rated as the most important, followed by technical skills, knowledge, experience, management skills, and interpersonal skills. this is because the commitment given by each team member is essential to the production of high quality software since the team produces the software (poth et al., 2020; sjøberg, 2018). on the other hand, technical skills, knowledge, experience, and management skills can be gained over time and can be improved through life-long learning processes. for technology, the completeness of standards and procedures was rated less important than the completeness of tools and techniques. the importance of using tools was described in ciancarini et al. (2019) where the developers were reported to use tools widely to support software development. however, as mentioned by salo and abrahamsson (2008), following proper standards and procedures is essential; otherwise, understanding the requirements might be more difficult. chow and cao (2008) also identified that having coding standards upfront as one of the success factors in the agile software development. nevertheless, when proper tools are collectively used in a team, the standards and procedures might be indirectly implemented in some way. furthermore, accuracy of the budget and the schedule were rated as equally important. both the budget and the schedule are dependent on each other in the production of high quality software and always become a measure of a successful project as widely indicated by researchers (nath et al., 2020; alaidaros, & omar, 2017; lee & xia, 2010). on the other hand, safety was considered more important than comfort in the working environment. the working environment is considered one of the motivational tools to increase software developers’ productivity (machuca-villegas et al., 2020). overall, the majority of the priorities or levels of importance ranked by the participants of the focus group have been supported by previous studies. priorities can differ from one organization to another since they are influenced by the organization’s standard of operation and the experience of the developers. therefore, in the case of implementing the espac model, it is appropriate to let the assessors decide whether to utilize the proposed ideal priorities or to generate new ones by implementing the ahp technique. the priority values are included in the software process assessment for each of the evaluation criteria, which finally produces the certification level and the quality levels (packeer mohamed et al., 2015). 5.3 sensitivity analysis the sensitivity analysis was performed by using a ‘what-if’ analysis in order to analyze the changes that might happen to global priority values if the local priority values of the evaluation criteria are changed. with the aid of a sensitivity analysis, the reliability of the initial decision and factors that influenced the result can be determined such as which criteria influenced the original results (mu & pereyra-rojas, 2017). table 8 shows that the highest local priority value was the process (0.378) factor when compared to the other four factors. within the sub-factors of process, the highest local priority was the software development process (0.0659) while requirement engineering had the highest priority (0.703) compared to the other software development processes. in order to determine if changes in the local priority values for the requirement engineering has an effect on its global priority, a sensitivity analysis on the completeness, consistency and accuracy criteria was performed. ijahp article: packeer mohamed, baharom, deraman, tarawneh, yusof/software process assessment and certification: application of the analytic hierarchy process for priority determination international journal of the analytic hierarchy process 20 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.870 tables 9 to 11 show the changes in the global priority values. note that in table 9, the obtained global priority value for the completeness of requirement engineering is 0.121 while the local priority value is 0.691. in this sensitivity analysis, the local priority value was changed by increasing and decreasing it by 0.005. the result shows that there was only a slight difference (by 0.001) in the global priority value when the local priority value was changed. table 9 changes in global priority values when the local priority for completeness of requirement engineering was changed local prioriy 0.121 0.676 0.118 0.681 0.119 0.686 0.120 0.691 0.121 0.696 0.122 0.701 0.123 0.706 0.124 table 10 depicts the global and local priority values for the consistency of requirement engineering where the initial obtained values are 0.001 and 0.028, respectively. by increasing and reducing the local priority value by 0.005, a small difference in the global priority value was noticed. table 10 changes in global priority values when the local priority for consistency of requirement engineering was changed local priority 0.028 0.001 0.000 0.006 0.001 0.011 0.002 0.016 0.003 0.021 0.004 0.026 0.005 0.031 0.005 table 11 shows the difference of the global priority of the accuracy of requirement engineering when changes were made to the initial local priority. similar to the earlier outcome, only a small difference was found in the global priority, which is 0.001. ijahp article: packeer mohamed, baharom, deraman, tarawneh, yusof/software process assessment and certification: application of the analytic hierarchy process for priority determination international journal of the analytic hierarchy process 21 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.870 table 11 changes in the global priority values when the local priority for accuracy of requirement engineering was changed local priority 0.026 0.134 0.023 0.139 0.024 0.144 0.025 0.149 0.026 0.154 0.027 0.159 0.028 0.164 0.029 6. conclusion this paper presented the technique used to determine the priorities for the evaluation criteria in assessing and certifying the software process. the existing software process assessment and certification models do not consider priorities of the evaluation criteria even though multiple criteria are involved in the assessment. to produce more accurate and consistent certification results, priorities of the evaluation criteria need to be determined in a systematic way. therefore, this study adopted the analytic hierarchy process (ahp) in the extended software process certification model (espac) to determine the priorities. five main factors were used to assess the software process quality, namely process, technology, people, project constraints and environment. these factors were obtained through theoretical study and verified through expert reviews. to obtain the priorities for these criteria, seven steps of the ahp were performed in a focus group discussion with seven software practitioners. the results from the expert reviews and focus group discussion revealed that the experts were satisfied with the evaluation criteria and the ahp technique implemented in this study. additionally, the priorities obtained were used as the ideal priorities in the espac model. among the five factors prioritized, process was rated the most important criterion, followed by the project constraints, technology, environment, and people. with the obtained priorities, potential users of this model may utilize the proposed values, or they can obtain preferred priorities using the ahp if the ideal priority is not suitable to their organization’s environment. at the end of this study, a sensitivity analysis was carried out using a 'what-if' analysis to examine the effect on the global priority values if the local priority values of the evaluation criteria were changed. to perform this analysis, the local priority values for completeness, consistency and accuracy of requirement engineering were changed and observed. this analysis revealed that there was only a small difference in the global priority values when the local priority values were changed. this was predicted, and indicates that the initial priorities are reliable. this study contributes to the body of knowledge on the application of the ahp technique in the field of software process assessment and certification. furthermore, a systematic technique to determine the priorities for the evaluation criteria is provided for the assessors. as for future work, this research can be enhanced by using ai techniques such as fuzzy or firefly algorithms to reduce dependency on human judgment while simultaneously being more transparent. ijahp article: packeer mohamed, baharom, deraman, tarawneh, yusof/software process assessment and certification: application of the analytic hierarchy process for priority determination international journal of the analytic hierarchy process 22 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.870 references abd el hameed, t., latif, m. a. e., & kholief, s. (2016). identify and classify critical success factors of agile software development methodology using mind map. international journal of advanced computer science and applications, 7(5), 83-92. doi: https://doi.org/10.14569/ijacsa.2016.070513 acuna, s. t., antonio, a. d., ferre, x., lopez, m., & mate, l. (2000). the software process: modeling, evaluation and improvement. in chang, s. k. handbook of software engineering and knowledge engineering (pp. 193-237). river edge: world scientific publishing co. pte. ltd. doi: https://doi.org/10.1142/9789812389718_0011 agile manifesto. (2001). retrieved from www.agilemanifesto.org. ahimbisibwe, a., daellenbach, u., & cavana, r. y. (2017). empirical comparison of traditional plan-based and agile methodologies. journal of enterprise information management, 30(3), 400-454. doi: 10.1108/jeim-06-2015-0056 akbar, m. a., sang, j., khan, a. a., shafiq, m., hussain, s., hu, h., & xiang, h. (2017). improving the quality of software development process by introducing a new methodology–az-model. ieee access, 6, 4811-4823. doi: 10.1109/access.2017.2787981 alaidaros, h., & omar, m. (2017). software project management approaches for monitoring work-in-progress: a review. journal of engineering and applied sciences, 12(15), 3851-3857. aldahmash, a., gravell, a. m., & howard, y. (2017). a review on the critical success factors of agile software development. european conference on software process improvement, 504-512. springer, cham. doi: 10.1007/978-3-319-64218-5_41 ali, y., butt, m., sabir, m., mumtaz, u., & salman, a. (2018). selection of suitable site in pakistan for wind power plant installation using analytic hierarchy process (ahp). journal of control and decision, 5(2), 117-128. doi: 10.1080/23307706.2017.1346490 al-tarawneh, f. h. (2014). a framework for cots software evaluation and selection for cots mismatches handling and non-functional requirements. (unpublished doctoral dissertation). universiti utara malaysia, kedah, malaysia. animah, i., & shafiee, m. (2019). maintenance strategy selection for critical shipboard machinery systems using a hybrid ahp-promethee and cost benefit analysis: a case study. journal of marine engineering & technology, 1-12. doi: 10.1080/20464177.2019.1572705 ansari, m. t. j., pandey, d., & alenezi, m. (2018). store: security threat oriented requirements engineering methodology. journal of king saud university-computer and information sciences, 34(2), 191-203. doi: 10.1016/j.jksuci.2018.12.005 baharom, f., yahya, j., deraman, a., & hamdan, a. r. (2011). spqf: software process quality factor for software process assessment and certification. international conference on electrical engineering and informatics, 1-7. doi: 10.1109/iceei.2011.6021526 http://www.agilemanifesto.org/ http://dx.doi.org/10.1109/iceei.2011.6021526 ijahp article: packeer mohamed, baharom, deraman, tarawneh, yusof/software process assessment and certification: application of the analytic hierarchy process for priority determination international journal of the analytic hierarchy process 23 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.870 baharom, f. (2008). a software certification model based on development process quality assessment. (unpublished doctoral dissertation). universiti kebangsaan malaysia, selangor, malaysia. baharom, f., deraman, a., & hamdan, a. r. (2005). a survey on the current practices of software development process in malaysia. journal of ict, 4, 57-76. baidya, r., dey, p. k., ghosh, s. k., & petridis, k. (2018). strategic maintenance technique selection using combined quality function deployment, the analytic hierarchy process and the benefit of doubt approach. the international journal of advanced manufacturing technology, 94(1-4), 31-44. doi: 10.1007/s00170-016-9540-1 behkamal, b., kahani, m., & akbari, m. k. (2009). customizing iso 9126 quality model for evaluation of b2b applications. information and software technology, 51(3), 599-609. doi: 10.1016/j.infsof.2008.08.001 chen, j. k., pham, v. k., & yuan, b. j. (2013). adopting ahp approach on evaluation and selection of outsourcing destination in east and southeast asia. technology management in the it-driven services (picmet), 528-537. retrieved from http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6641715 chow, t., & cao, d. b. (2008). a survey study of critical success factors in agile software projects. journal of systems and software, 81(6), 961-971. doi: 10.1016/j.jss.2007.08.020get ciancarini, p., missiroli, m., & sillitti, a. (2019). preferred tools for agile development: a sociocultural perspective. international conference on objects, components, models and patterns, 43-58. springer, cham. doi: 10.1007/978-3-030-29852-4_3 cmmi institute. (2018). cmmi adoption and transition guidance v2.0. retrieved from https://cmmiinstitute.com/getattachment/5868888b-5f37-4715-bc8bc43250ec0abc/attachment.aspx computerworldukstaff. (2020, feb 17). top software failures in recent history. computerworld. retrieved from https://www.computerworld.com/article/3412197/topsoftware-failures-in-recent-history.html#slide1 crostack, h. a., hackenbroich, i., refflinghaus, r., & winter, d. (2007). investigations into more exact weightings of customer demands in qfd. asian journal on quality, 8(3), 71-80. doi: http://dx.doi:org/10.1108/15982688200700026 daneva, m., & ahituv, n. (2011). what practitioners think of inter-organizational erp requirements engineering practices: focus group results. international journal of information system modeling and design, 2(3), 49-74. doi: 10.4018/jismd.2011070103 darwish s. m. (2016). software test quality rating: a paradigm shift in swarm computing for software certification. knowledge-based systems, 10, 167-175. doi: 10.1016/j.knosys.2016.07.022 deming, w. (1982). out of the crisis. cambridge, ma: mit center for advanced engineering study. destefanis, g., ortu, m., counsell, s., swift, s., marchesi, m., & tonelli, r. (2016). http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6641715 https://cmmiinstitute.com/getattachment/5868888b-5f37-4715-bc8b-c43250ec0abc/attachment.aspx https://cmmiinstitute.com/getattachment/5868888b-5f37-4715-bc8b-c43250ec0abc/attachment.aspx https://www.computerworld.com/article/3412197/top-software-failures-in-recent-history.html#slide1 https://www.computerworld.com/article/3412197/top-software-failures-in-recent-history.html#slide1 http://dx.doi:org/10.1108/15982688200700026 ijahp article: packeer mohamed, baharom, deraman, tarawneh, yusof/software process assessment and certification: application of the analytic hierarchy process for priority determination international journal of the analytic hierarchy process 24 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.870 software development: do good manners matter?. peerj computer science, 2, e73. doi: 10.7717/peerj-cs.73. dyba, t., dingsoyr, t., & moe, n. b. (2014). agile project management. in ruhe, g. & wohlin, c. (eds.), software project management in a changing world (pp. 277-300). springer berlin heidelberg. ferreira, g., kästner, c., sunshine, j., apel, s., & scherlis, w. (2019). design dimensions for software certification: a grounded analysis. arxiv preprint arxiv:1905.09760. galin, d. (2004). software quality assurance. england: pearson education limited. gasston, j. l. (1996). process improvement: an alternative to bpr for software development organizations. software quality journal, 5, 171-183. doi: https://doi.org/10.1007/bf00678582 gualo, f., caballero, i., & rodriguez, m. (2020). towards a software quality certification of master data-based applications. software quality journal, 28, 1-24. doi: 10.1007/s11219-019-09495-w hallowell, m. r., & gambatese, j. a. (2010). qualitative research: application of the delphi method to cem research. journal of construction engineering and management, 136(1), 99-107. doi: 10.1061/_asce_co.1943-7862.0000137 hambali, a., & rahman, m. a. (2017). application of integrated ahp and topsis techniques for determining the best fresh fruit bunches (ffb). journal of telecommunication, electronic and computer engineering (jtec), 9(3), 145-149. heck, p., klabbers, m., & eekelen, m. (2010). a software product certification model. software quality journal, 18(1)37-55. doi: 10.1007/s11219-009-9080-0 heikkilä, v. t., damian, d., lassenius, c., & paasivaara, m. (2015). a mapping study on requirements engineering in agile software development. 41st euromicro conference on software engineering and advanced applications, 199-207. ieee. hsiao, s. w. (2002). concurrent design method for developing a new product. international journal of industrial ergonomics, 29(1), 41-55. doi: 10.1016/s01698141(01)00048-8 humphrey, w. (1989). managing the software process. mass: addison-wesley. ishizaka, a., & labib, a. (2011). review of the main developments in the analytic hierarchy process. expert systems with applications, 38(11), 14336-14345. doi: 10.1016/j.eswa.2011.04.143 jones, c., & bonsignour, o. (2012). the economics of software quality. boston: pearson education. jung, h. w. (2001). rating the process attribute utilizing ahp in spice‐based process assessments. software process: improvement and practice, 6(2), 111-122. doi: 10.1002/spip.139 ijahp article: packeer mohamed, baharom, deraman, tarawneh, yusof/software process assessment and certification: application of the analytic hierarchy process for priority determination international journal of the analytic hierarchy process 25 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.870 komuro, m., & komoda, n. (2008). an explanation model for quality improvement effect of peer reviews. international conference on computational intelligence for modelling control & automation, 1159-1164. doi: 10.1109/cimca.2008.187 kontio, j., bragge, j., & lehtola, l. (2008). the focus group method as an empirical tool in software engineering. in shull, f. et al. (eds.) guide to advanced empirical software engineering (pp. 93-116). springer, london. kumar, u., kumar, n., mishra, v. n., & jena, r. k. (2019). soil quality assessment using analytic hierarchy process (ahp): a case study. in sugumaran, v. (ed.), interdisciplinary approaches to information systems and software engineering (pp. 1-18). igi global. doi: 10.4018/978-1-5225-7784-3.ch001 kumari, n. d., & shylaja, b. s. (2019). amgrp: ahp-based multimetric geographical routing protocol for urban environment of vanets. journal of king saud universitycomputer and information sciences, 31(1), 72-81. doi: 10.1016/j.jksuci.2017.01.001 kunda, d. (2003). stace: social technical approach to cots software evaluation. in cechich, a., piayyini, m., & vallecillo (eds.), a. component-based software quality (pp. 64-84). berlin heidelberg: springer-verlag. lai, v. s., wong, b. k., & cheung, w. (2002). group decision making in a multiple criteria environment: a case using ahp in software selection. european journal of operational research, 137, 134-144. doi: 10.1016/s0377-2217(01)00084-4 lee, g., & xia, w. (2010). toward agile: an integrated analysis of quantitative and qualitative field data on software development agility. mis quarterly, 34(1), 87-114. doi: 10.2307/20721416 liamputtong, p. (2011). focus group methodology principles and practices. london: sage publication. machuca-villegas, l., gasca-hurtado, g. p., tamayo, l. m. r., & puente, s. m. (2020). social and human factor classification of influence in productivity in software development teams. european conference on software process improvement, 717-729. springer, cham. doi: 10.1007/978-3-030-56441-4_54 marjani, m. e., soh, k. g., majid, m., mohd sofian, o. f., nur surayyah, m. a., & mohd rizam, a. b. (2012). usage of group decision making approach in karate agility test selection. international symposium on the analytic hierarchy process, 1-11. retrieved from http://www.isahp.org/uploads/59.pdf martakis, a., & daneva, m. (2013). handling requirements dependencies in agile projects: a focus group with agile software development practitioners. seventh international conference on research challenges in information science, 1-11. doi: 10.1109/rcis.2013.6577679 mas, a., fluxa, b., & amengual, e. (2012). lessons learned from an iso/iec 15504 spi programme in a company. journal of software: evolution and process, 24(5), 493-500. doi: 10.1002/smr.501 mazza, r., & berre, a. (2007). focus group methodology for evaluating information http://ieeexplore.ieee.org.eserv.uum.edu.my/xpl/mostrecentissue.jsp?punumber=5172579 http://ieeexplore.ieee.org.eserv.uum.edu.my/xpl/mostrecentissue.jsp?punumber=5172579 http://dx.doi.org/10.1109/cimca.2008.187 http://link.springer.com/book/10.1007/b11721 http://www.isahp.org/uploads/59.pdf http://dx.doi.org/10.1109/rcis.2013.6577679 ijahp article: packeer mohamed, baharom, deraman, tarawneh, yusof/software process assessment and certification: application of the analytic hierarchy process for priority determination international journal of the analytic hierarchy process 26 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.870 visualization techniques and tools. 11 th international conference information visualization, 74-80. doi: 10.1109/iv.2007.51 meier, s. r. (2017). technology portfolio management for project managers. retrieved from https://www.pmiwdc.org/sites/default/files/presentations/201703/pmiw_localcommunit y_tysons_presentation_2017-02.pdf mu, e., & pereyra-rojas, m. (2017). understanding the analytic hierarchy process. in mu and pereya-rojas (eds.), practical decision making (pp. 7-22). springer, cham. mokhtar, m. r., abdullah, m. p., hassan, m. y., & hussin, f. (2017). comparative study of multiple criteria decision making methods for selecting the best demand side management options. elektrika-journal of electrical engineering, 16(1), 11-16. doi: 10.11113/elektrika.v16n1.15 moradi, n. (2022). performance evaluation of university faculty by combining bsc, ahp, and topsis: from the students’perspective. international journal of the analytic hierarchy process, 14(2), 1-29. doi: https://ijahp.org/index.php/ijahp/article/view/915 morgan, d. l. (1998). planning focus groups. thousand oaks: sage publications. nath, u. k., jagadev, a. k., & pattnaik, p. k. (2020). achieving target schedule time faster through agile simulation model. journal of critical reviews, 7(19), 4080-4088. doi: 10.31838/jcr.07.19.476 packeer mohamed, s. f., baharom, f., & deraman, a. (2015). espac model: extended software process assessment and certification model for agile and secure software processes. arpn journal of engineering and applied sciences, 10(3), 1364-1373. padumadasa, e. u., colombo, s., & rehan, s. (2009). investigation into decision support systems and multiple criteria decision making to develop a web-based tender management system. international symposium on the analytic hierarchy process, 1-17. retrieved from http://www.isahp.org/2009proceedings/final_papers/66_padumadasa_evaluatingtender offers_rev_fin.pdf pietrantuono, r., & russo, s. (2018). robotics software engineering and certification: issues and challenges. ieee international symposium on software reliability engineering workshops (issrew), 308-312. ieee. doi: 10.1109/issrew.2018.00023 poth, a., kottke, m., & riel, a. (2020). evaluation of agile team work quality. international conference on agile software development, 101-110. springer, cham. doi: 10.1007/978-3-030-58858-8_11 pressman, r. s. (2010). software engineering: a practitioner's approach. (7th ed). new york: mcgraw-hill higher education. rae, a., robert, p., & hausen, h. l. (1995). software evaluation for certification principles, practice and legal liability. england: mcgraw-hill. saaty, t. l., & de paola, p. (2017). rethinking design and urban planning for the cities of the future. buildings, 7(3), 76. doi: 10.3390/buildings7030076 http://dx.doi.org/10.1109/iv.2007.51 https://www.pmiwdc.org/sites/default/files/presentations/201703/pmiw_localcommunity_tysons_presentation_2017-02.pdf https://www.pmiwdc.org/sites/default/files/presentations/201703/pmiw_localcommunity_tysons_presentation_2017-02.pdf http://www.isahp.org/2009proceedings/final_papers/66_padumadasa_evaluatingtenderoffers_rev_fin.pdf http://www.isahp.org/2009proceedings/final_papers/66_padumadasa_evaluatingtenderoffers_rev_fin.pdf ijahp article: packeer mohamed, baharom, deraman, tarawneh, yusof/software process assessment and certification: application of the analytic hierarchy process for priority determination international journal of the analytic hierarchy process 27 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.870 saaty, t. l. (2008). decision making with the analytic hierarchy process. international journal of services sciences, 1(1), 83-98. doi: 10.1504/ijssci.2008.01759 saaty, t. l. (1990). how to make a decision: the analytic hierarchy process, european journal of operation research, 48(1), 9–26. doi: 10.1016/0377-2217(90)90057-i sambinelli, f., & borges, m. a. f. (2019). survey on strategies to increase customer value in brazilian agile software development companies. 14 th iberian conference on information systems and technologies (cisti), 1-7. ieee. doi: 10.23919/cisti.2019.8760653 salo, o., & abrahamsson, p. (2008). agile methods in european embedded software development organizations: a survey study of extreme programming and scrum, iet software, 2(1), 58-64. doi: 10.1049/iet-sen:20070038 sjøberg, d. i. (2018). teamwork quality and team performance: exploring differences between small and large agile projects. agile processes in software engineering and extreme programming, 267. sommerville, i. (2007). software engineering (8th ed). harlow: pearson education limited. sun-jen, h., & wen-ming, h. (2006). selection priority of process areas based on cmmi continuous representation. information & management, 43(3), 297-307. doi: 10.1016/j.im.2005.08.003 triantaphyllou, e., & mann, s. h. (1995). using the analytic hierarchy process for decision making in engineering applications: some challenges. international journal of industrial engineering: applications and practice, 2(1), 35-44. retrieved from http://bit.csc.lsu.edu/trianta/journal_papers1/ahpapls1.pdf voas, j., & laplante, p. a. (2018). iot’s certification quagmire. computer, 51(4), 86-89. doi: 10.1109/mc.2018.2141036 voas, j. (1998). the software quality certification triangle: crosstalk. the journal of defense software engineering, 11(11), 12-14. wagenaar, g., overbeek, s., lucassen, g., brinkkemper, s., & schneider, k. (2018). working software over comprehensive documentation–rationales of agile teams for artefacts usage. journal of software engineering research and development, 6(1), 7. doi: 10.1186/s40411-018-0051-7 wang, y., & leung, h. k. (2001). a benchmark-based adaptable software process model. proceedings 27th euromicro conference. 2001: a net odyssey, 216-224. yaghoobi, t. (2018). prioritizing key success factors of software projects using fuzzy ahp. journal of software: evolution and process, 30(1), e1891. doi: 10.1002/smr.1891 yahya, j. (2007). the development of software certification model based on product quality approach. (unpublished doctoral dissertation). universiti kebangsaan malaysia, selangor, malaysia. http://dx.doi.org/10.1016/0377-2217(90)90057-i http://dx.doi.org/10.1049/iet-sen:20070038 http://bit.csc.lsu.edu/trianta/journal_papers1/ahpapls1.pdf ijahp article: packeer mohamed, baharom, deraman, tarawneh, yusof/software process assessment and certification: application of the analytic hierarchy process for priority determination international journal of the analytic hierarchy process 28 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.870 zaidan, a. a., zaidan, b. b., alsalem, m. a., momani, f., & zughoul, o. (2020). novel multiperspective hiring framework for the selection of software programmer applicants based on ahp and group topsis techniques. international journal of information technology & decision making, 19(3), 775-847. doi: 10.1142/s0219622020500121 zhou, z., & liang, k. (2013). network course evaluation system based on ahp theory. in wenjiang, du. informatics and management science ii (pp. 569-575). london: springer london. doi: 10.1007/978-1-4471-4811-1_73 ijahp article: kumar/buyer-supplier relationship selection for a sustainable supply: a case of the indian automobile industry international journal of the analytic hierarchy process 215 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.605 buyer-supplier relationship selection for a sustainable supply chain: a case of the indian automobile industry divesh kumar malaviya national institute of technology jaipur india diveshcms@gmail.com abstract the primary objective of this study is to propose an analytical approach to evaluate dimensions of the buyer-supplier relationships alternative selection in the context of a sustainable supply chain considering benefits, opportunities, costs, and risks of a relationship type. the study uses a fuzzy analytic hierarchy process (fahp) approach to develop a model that explores benefits, costs, opportunities, and risks of a buyersupplier relationship in a sustainable supply chain. a case study of the indian automobile sector is used to determine the appropriate form of the buyer-supplier relationship. this study found that a problem solving relationship is the most preferred form of relationship. long term relationships and joint development ranked 2 and 3, respectively. a sensitivity analysis showed that if the weights of benefits, opportunities, costs and risks are changed the preference for alternatives selection also changes. the most important implication of the study is in providing supply chain managers with a model for development of buyersupplier relationships with their supply chain partners based on the considerations of benefits, opportunities, costs and risks involved in developing such a relationship in a sustainable supply chain. the novelty of the present study rests in the incorporation of sustainability specific criteria for selection of the relationship between two parties in a supply chain context considering benefits, opportunities, costs, and risks of a relationship type. this is the first such model that incorporates considerations of bocr in a sustainable supply chain. keywords: sustainable supply chain; analytical hierarchy process; fuzzy logic; buyersupplier relationship 1. introduction relationships with supply chain partners are developed with the intention of providing assistance for technology development, resource sharing, information dissemination, and developing capacity and capability of supply chain partners (kam & lai, 2018; kumar & rahman, 2016). buyer-supplier relationships have undergone significant changes during the last decade due to increased focus on sustainability (kumar et al., 2017; pagell, & shevchenko, 2014). the focus of the research has shifted from company specific sustainability to buyer-supplier collaborative relationships for managing supply chain sustainability (dekker et al., 2019; gimenez & tachizawa, 2012). few studies have mailto:diveshcms@gmail.com ijahp article: kumar/buyer-supplier relationship selection for a sustainable supply: a case of the indian automobile industry international journal of the analytic hierarchy process 216 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.605 indicated the negative impact of supplier activities on environmental and social sustainability, making it difficult for any firm to claim their products are sustainable without considering their supply chain (kumar & garg, 2017; carter & rogers, 2008; mahler, 2008). this has led to the emergence of 'supplier’s attitude towards sustainability' as one of the most important criteria in industrial purchase and order allocation (kumar et al., 2017). in order to achieve sustainability objectives, firms always look for an appropriate partner to be able to attract a considerable investment of time and resources (hammerschmidt et al., 2018). lee (2009) has highlighted the need for a comprehensive model for buyer-supplier relationship selection. this puts forth a strong case for developing a buyer-supplier relationship model based on the benefits, opportunities, costs and risks (bocr) of a relationship alternative. there is, however, a paucity of studies in the extant literature that consider bocr in buyer-supplier relationships. only one model based on fuzzy ahp proposed by lee (2009) includes bocr to study the buyer-supplier relationship for an electronics company manufacturing lcds. however, this two-step model does not consider sustainability dimensions. lee (2009) further suggested that more complicated alternatives should be selected in order to study supplier specific relationship types based on the bocr. the current study aims to extend the work of lee (2009) by including the sustainability dimensions in the buyer-supplier relationship. a two-stage model is proposed to study the four selection criteria – benefit, opportunity, cost, and risk of a relationship in the supply chain as suggested by the lee (2009). the present study builds upon the proposition that buyer-supplier relationships are based on an analysis of benefit, cost and expected return on the relationship in terms of opportunities and risks. most of the bocr analysis in the extant literature has been done using statistical techniques such as the analytical hierarchy process (ahp) and the analytical network process (anp) (wijnmalen, 2007; ming-chien et al., 2015; hernandez et al., 2016). the present study employs fuzzy logic with the ahp technique for developing a buyer-supplier relationship model for a sustainable supply chain. the use of fuzzy logic with ahp has been advised by many authors in order to remove any vagueness in the responses (kumar & garg, 2017; lee, 2009; chang, 1996). the remainder of the paper has been organized as follows: section 2 discusses the literature on buyer-supplier relationships, followed by a computational procedure in section 3; section 4 will discuss a case study of the indian automobile supply chain and section 5 discusses the conclusions and direction of future research. 2. buyer-supplier relationship in a sustainable supply chain buyer-supplier relationships are influenced by the level of inter-organizational dependency (dekker et al., 2019). a few authors argue that close cooperation with supply chain partners would result in better performance of sustainable supply chains (im et al., 2019; seuring & muller, 2008). however, relationships between all supply chain partners may not be based on the same considerations. the relationships between supply chain partners may be based on varied levels of economic, technological and behavioral considerations (stranieri et al., 2019; hadjikhani & laplaca, 2013). further, developing long-term relationships might not guarantee the expected return with respect to the ijahp article: kumar/buyer-supplier relationship selection for a sustainable supply: a case of the indian automobile industry international journal of the analytic hierarchy process 217 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.605 investment (vishnu et al., 2019). firms are required to optimize their available resources among the supply chain partners for developing relationships. the extant literature on relationship marketing in the industrial environment emphasizes that a relationship should be developed with consideration of the costs and risks involved. the contemporary approach to marketing is witnessing a shift from the traditional product-based transactional approach to a resource-based relational approach (zhou et al., 2018; grönroos, 1996). the relationship approach is based on allocation of resources towards developing relationships with suppliers on the basis of the expected return from the relationship (hong et al., 2018). further, the quality and nature of the buyer-supplier relationship is influenced by inter-organizational dependency and their joint abilities to achieve a common objective (aharonovitz et al., 2018). 2.1 dimensions of a sustainable supply chain carter & roger (2008) defined sustainable supply chain management as “the strategic achievement and integration of an organization’s social, environmental, and economic goals through the systemic coordination of key inter-organizational business processes to improve the long-term economic performance of the individual company and its value network.” similarly seuring & muller (2008) defined management of a sustainable supply chain as “the management of material, information and capital flow as well as cooperation among companies along the supply chain while taking goals from three dimensions of sustainable development, i.e. economic, environmental, and social, into account which is derived from customer and stakeholder requirements.” elkington (1994) defined the three dimensions of sustainability as environmental, social and economic. thus, sustainability should be assessed by the activities of the supplier on the environmental standards, social development for the employees and society, and the economic production of the product. the criteria for supplier selection can be developed based on evaluation by experts and from support of extant literature for specific problems. (kumar et al., 2017; mangla et al., 2018). rejection percentage on a quality basis (walker et al., 2008; matos & hall, 2007), percentage of late delivery items (zsidisin & hendrick, 1998; daugherty, 2011) and cost of the sourcing item (zutshi & sohal, 2004; holt & ghobadian, 2009) are some of the economic and operational criteria used for identifying acceptability of suppliers. social sustainability criteria listed in the topical literature include child labor (joplin et al., 2007; muller et al., 2009), working conditions (pommel, 2010; carter & rogers 2008), rights of employees (rocha et al., 2007; calibers, 2008; ni et al., 2010) and poverty reduction (blitzer et al., 2008; ni et al., 2010). various environmental sustainability criteria identified in the literature includes packaging improvements (souflas & pappies, 2006; hall, 2000), energy efficiency (wu & patel, 2011; nakano & hirao, 2011), pollution and emission minimization (florida, 1996; calibers, 2008), waste minimization (matos & hall, 2007; bitzer et al., 2008), reverse logistics (ni et al., 2010, carter & jenning, 2002), green purchasing (bitzer et al., 2008; ni et al., 2010), green designing (holt & ghobadian, 2009; bai & sarkis, 2010), using renewable energy (smith, 2007; zhu et al., 2007) and disposal (vachon & klassen, 2006; olorunniwo & li, 2010). kumar and garg (2017) prioritized several dimensions of a sustainable chain in the context of the indian automobile industry. kumar et al. (2017) also used dimensions of sustainability for optimizing the order among the suppliers. ijahp article: kumar/buyer-supplier relationship selection for a sustainable supply: a case of the indian automobile industry international journal of the analytic hierarchy process 218 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.605 2.2 evidences of buyer-supplier relationship in a sustainable supply chain there are many studies that have shown evidence that a relationship based supply chain is necessary for the adoption of a particular practice across the supply chain; like iso adoption, new technology and sustainability adoption (kumar et al., 2017; kumar & rahman, 2016). the literature has used many keywords for supply chain relationships, like collaboration, coordination, integration, and cooperation. however, the exact form of the relationship is not clear with these keywords. for example, collaboration is important for product designing, forecasting, planning and increasing sustainability performance (attaran & attaran, 2007; vermeulen & seuring, 2009). collaboration is also vital for creating awareness about sustainability across the supply chain (dangol et al., 2015; zhu & sarkis, 2004). kumar & rahman (2016) used supplier performance, supplier selection and suppliers' performance review to define the construct of the buyer-supplier relationship. zsidisin & hendrick (1998) argued that collaboration with suppliers to provide equipment, material, parts and services is essential for extending environmental sustainability across the supply chain. the authors have found various situations in which the word ‘collaboration’ fits. hence, a single keyword cannot define the nature of a buyer-supplier relationship. it is situational and based on the need of the relationship and areas for improvement. in order to understand the buyer-supplier relationship, a detailed analysis needs to be done. the type of relationship depends upon the capabilities, capacity and commitment of the supplier. besides this, some buyer firms do not want to indulge in any relationship and use their buying power to create pressure on the suppliers (ford, 1980). the channel literature first discussed supply chain relationships that vary from arm's length to vertical integration (golicic et al., 2003; contractor & lorange 1988). relationships were further categorized by many authors based on the scope and magnitude of the relationship. supply chain relationships can be termed as partnerships, alliances, joint ventures, network organizations, franchisees, license agreements, contractual relationships, service agreements, and administered relationships (golicic et al., 2003). cannon & perreautt (1999) argued in favor of characterizing buyer-seller relationships based on a variety of different ways and listed eight types of buyer-seller relationships as basic buying and selling, bare bones, contractual transaction, customer supply, cooperative systems, collaborative, mutually adaptive, and customer is king. hansen (2006) explained four types of relationships in terms of exchange as transactional, collaboration, co-production and co-creation. rinehart et al. (2002), based on empirical data, defined seven types of buyer-supplier relationships based on three dimensions of trust, interaction frequency and commitment. they labeled these seven types of relationships as non-strategic transactions, administered relationships, contractual relationships, specialty contract relationships, partnerships, joint ventures, and strategic alliances. the nature and extent of the relationship with a supplier can be determined by the expected performance of the relationship (zhu & sarkis, 20008). for example, if a supplier is the most sustainable among all available suppliers of a particular input material but lacks in the capacity, the buyer firm can work on developing the capacity of that supplier. firms in the buyer-seller relationship should consider strengthening each other in order to improve the sustainability performance of the relationship. hence, ijahp article: kumar/buyer-supplier relationship selection for a sustainable supply: a case of the indian automobile industry international journal of the analytic hierarchy process 219 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.605 developing a relationship is specific to the capacity, capability and current performance of the supplier. 2.3 benefits, costs, risks and opportunities of buyer-supplier relationship sustainability of the supply chain is fast becoming a mandatory criterion in relationships between supply chain partners. firms are adopting sustainability practices due to external pressure, while others see it as an opportunity for growth and the associated benefits. firms may also be reluctant towards sustainability because of the costs involved in overhauling the process or due to the perception of low economic returns and performance (nakano & hirao, 2011; ageron et al., 2011; cai et al., 2008; bowen et al., 2001; simpson & power, 2005; keatinga et al., 2008, fortes, 2009). the buyer-supplier relationship always depends on the trade-off between costs, risks and benefits, opportunities involved. in the literature on supply chain sustainability, the authors have quoted many benefits, costs, risks and opportunities with respect to relationship development for sustainability adoption in the supply chain (kumar & rahman, 2015) (see table 1). ijahp article: kumar/buyer-supplier relationship selection for a sustainable supply: a case of the indian automobile industry international journal of the analytic hierarchy process 220 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.605 table 1 bocr of buyer-supplier relationship in sustainable supply chain criteria sub criteria definition source benefits financial (b1) 1.1 reduce distribution cost buyer/supplier will have less cost on the distribution due to the relationship ytterhus, 1999; tsoulfas & pappis, 2006; green et al., 1998; eltayeb, 2011; zhu et al., 2008; peters et al., 2011 1.2 low cost on information better information exchange and sharing darnll et al., 2008; tsoulfas & pappis, 2006; eltayeb, 2011; zhu et al., 2008; peters et al., 2011 1.3 reduce inventory a good relationship may help in reducing the inventories of buyer / supplier hong et al., 2009; closs et al., 2010; ageron et al., 2011; zhu et al., 2008; attaran &attaran, 2007 operational (b2) 2.1 improve internal process improving process by providing feedback on supplier's operations. hong et al., 2009; tsoulfas & pappis, 2006; zhu et al., 2008; 2.2 resource optimization understanding each other’s operations and optimization hong et al., 2009; brito et al., 2008; tsoulfas &pappis, 2006 2.3 on time delivery real time supply of items due to relationship eltayeb, 2011; daugherty, 2011; zhu et al., 2008; brito et al., 2008 sustainability adoption (b3) 3.1 improved quality on sustainability standards improving product quality on sustainability dimensions. zhu & sarkis, 2008; ytterhus, 1999; eltayeba et al., 2011; bitzer et al., 2008; ageron et al., 2011 3.2 reduced pressure from various agencies less pressure from various stakeholders. matos & hall, 2007; ageron et al., 2011; cai et al., 2008; muller et al., 2009; zhu & sarkis, 2010 3.3 sustainable supply chain incorporation of sustainability across supply chain zutshi & sohal, 2004; klassen &vachon, 2003; nakano & hirao, 2011; seuring & muller, 2008; smith, 2007 opportunities marketing advantage (o1) 4.1 improve corporate image/ reputation improved reputation due to better sustainability performance. simpson et al., 2007; matos & hall, 2007; rocha et al., 2007; vermeulen & ras, 2006; muller et al., 2009 4.2 premium pricing pricing the product for better sustainability performance ytterhus, 1999; eltayeba et al., 2011; ageron et al., 2011 4.3 product differentiation product differentiation due to sustainability adoption. kogg, 2003 4.4 new market targeting new customers holt & ghobadian, 2009; clemens & douglus, 2006; ijahp article: kumar/buyer-supplier relationship selection for a sustainable supply: a case of the indian automobile industry international journal of the analytic hierarchy process 221 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.605 having positive attitude towards sustainable products. bitzer et al., 2008; markley & devis, 2007 technical capabilities (o2) 5.1 sharing technology and knowledge sharing new and advanced technology with buyers and suppliers. rao, 2002; rocha et al., 2007; muller et al., 2009; vachon, 2007; kogg, 2003; hong et al., 2009; koplin et al., 2007 5.2 educating each other’s employee providing education to buyers and suppliers' employees on sustainability issues. clemens & douglus, 2006; lee, 2008; ytterhus, 1999; seuring& muller, 2008; zhu et al., 2007; ciliberti, 2008 5.3 developing technical standards developing better sustainability enables standards rao & holt, 2005; klassen &vachon, 2003; bai & sarkis, 2010; zhu et al, 2007 mutual growth (o3) 6.1 capacity building and development developing the capacity of buyer/ supplier considering length of relationship klassen & vachon, 2003; wu & pagell, 2011; lee, 2008; ageron et al., 2011; markley & davis, 2007 6.2 sharing resources and information sharing the resources with each other to better utilization. bommel, 2010; wu & pagell, 2011; klassen &vachon, 2003; nakano & hirao, 2011; smith, 2007; lee, 2008 6.3 jointly setting goals buyer and suppliers jointly setting up economic, social and environmental goals. vachon & klassen, 2008; olorunniwo & li, 2010; matos & hall, 2007 costs cost of relationship (c1) 7.1 financial investment for developing relationship financial investments required from the supply chain partners. linton et al., 2007; ageron et al., 2011; cai et al., 2008; muller et al., 2009; bowen et al., 2001 7.2 time required to develop relationship time investment required from the supply chain partners. peters et al., 2011; handfield, 2005; bitzer et al., 2008; zsidisin & hendrick, 1998; carter & jenning, 2002 7.3 responsibility sharing sharing of responsibilities among supply chain partners. zutshi & sohal, 2004; rocha et al., 2007; ni et al., 2010; ageron et al., 2011; cai et al., 2008; daugherty, 2011 impact of relationship (c2) 8.1 perception of relationship success buyers and suppliers are not sure of the success of the relationship nakano & hirao, 2011; ageron et al., 2011; cai et al., 2008; bowen et al., 2001; simpson & power, 2005; fortes, 2009 8.2 no improvement in sustainability performance there is no improvement in sustainability performance. ageron et al., 2011; cai et al., 2008; bowen et al., 2001; simpson & power, 2005; keatinga et al., 2008; fortes, 2009 ijahp article: kumar/buyer-supplier relationship selection for a sustainable supply: a case of the indian automobile industry international journal of the analytic hierarchy process 222 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.605 8.3 poor partner commitment towards sustainability partners are not ready to share the cost of the relationship. diabata & givindanb, 2011; rao & holt, 2005; zutshi & sohal, 2004; lee, 2008; vachon, 2007; ageron et al, 2011 cost of adoption (c3) 9.1 technological changes investment required by supply chain partners for technological changes wu & pagell, 2011; klassen &vachon, 2003; lee, 2008; bitzer et al., 2008; zhu & sarkis, 2004 9.2 process change investment required by supply chain partners for process change. olorunniwo & li, 2010; rocha et al., 2007; cai et al., 2008; attaran & attaran, 2007 9.3 infrastructure development investment required by supply chain partners for infrastructure development bowen et al., 2001; simpson & power, 2005; markley & devis, 2007 risks management (r1) 10.1 lack of trust lack of trust between parties bitzer et al., 2008; senge & prokesch, 2010; diabat & govindan, 2011; rao & holt, 2005 10.2 problem in sharing risk no party is ready to share the risk involved hall, 2000; olorunniwo & li, 2010, simpson & power, 2005 10.3 lack of integration parties are integrating their operational process with respect to others asif et al, 2008; seuring & muller, 2008; vachon, 2007 market (r3) 11.1 dependency on few suppliers dependency on a few available sustainable suppliers handfield, 2005; matos & hall, 2007; rocha et al., 2007 11.2 bargaining power of supplier few suppliers have better bargaining power bitzer et al., 2008; senge & prokesch, 2010; diabat & govindan, 2011; rao & holt, 2005 11.3 competition in future buyer or supplier may do forward or backward integration and create competition bitzer et al., 2008; senge & prokesch, 2010; diabat & govindan, 2011; rao & holt, 2005 investment (r4) 12.1 huge investment required for developing relationship buyer/supplier need to investment in developing relationship linton et al., 2007; ageron et al., 2011; cai et al., 2008; muller et al., 2009; bowen et al., 2001 12.2 unavailability of required technology with partners existing buyer/supplier are not having required technology zutshi & sohal, 2004; lee, 2008; hall, 2000; bai & sarkis, 2010; bitzer et al., 2008; vermeulen & ras, 2006 12.3 breaking partnership in between fear of buyer/supplier breaking relationship diabat & govindan, 2011; rao & holt, 2005 ijahp article: kumar/buyer-supplier relationship selection for a sustainable supply: a case of the indian automobile industry international journal of the analytic hierarchy process 223 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.605 table 1 presents the several criteria and sub-criteria under the benefits, opportunities, costs, and risks. the benefits of the relationship include financial, operational and sustainability adoption. the opportunities include marketing advantage, technical capability development, and mutual growth possibilities. the costs include cost of relationship, impact of relationship, and cost of adoption of sustainability practices. the risks of a relationship include management of relationship, market risk, and investment risk. 3. computational procedure 3.1 fuzzy ahp fuzzy set theory helps analyze the vagueness and fuzziness of uncertain environments (zadeh, 1965). in ahp, the crisp value is taken for the pairwise comparison, but it is not appropriate for making real life decisions where responses are supposed to be uncertain (shaw et al., 2012). to solve this problem, decision models should incorporate a fuzzy theory to deal with uncertainty (lee, 2009; yu, 2002). fuzzy ahp is often used in research for decision-making with various proposed methods for calculating fuzziness (chang, 1996; kumar et al., 2017). there are advantages and disadvantages for each method. considering the simplicity of calculations and advantages of one method over another, chang (1996) used the extent analysis method for fuzzy ahp. this approach deals with the uncertainty of decision making and is more robust in nature (chan & kumar, 2007). wang et al. (2008) published one article "on the extent analysis method for fuzzy ahp and its applications" in the european journal of operation research. in this article the author commented on the feasibility of usage of the extent analysis method in obtaining crisp numbers from fuzzy triangular numbers. though many previous studies have used the extent analysis method due to its simplicity in computation and supported the methodology, the researchers must make an adequate decision before opting for any method. figure 1a triangular fuzzy number µ𝑚 (�̌�) 1 0 b a c ijahp article: kumar/buyer-supplier relationship selection for a sustainable supply: a case of the indian automobile industry international journal of the analytic hierarchy process 224 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.605 figure 1b two triangular fuzzy numbers m1 and m2 (lee, 2009). the triangular fuzzy number m can be represented by (a, b, c), and the membership function for fuzzy number is shown in figure 1a (cheng, 1999; lee et al., 2005). 𝜇𝑚 (𝑥) = { 𝑥−𝑎 𝑏−𝑎 (𝑎 ≤ 𝑥 ≤ 𝑏) 𝑐−𝑥 𝑥−𝑏 (𝑏 ≤ 𝑥 ≤ 𝑐) 0 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 (1) with -∞ < a ≤ b ≤ c ≤ ∞. the strongest grade of membership is the parameter b that is, fm (b) = 1, while a and c are the lower and upper bounds. two triangular fuzzy numbers m1 (𝑚1 −, 𝑚1, 𝑚1 +) and m2(𝑚2 −, 𝑚2, 𝑚2 +) are shown in figure 1 (b). when, 𝑚1 − ≥ 𝑚2 −, 𝑚1 ≥ 𝑚2, 𝑚1 + ≥𝑚2 +) (2) the degree of possibility is represented in equation (3): v (m1 ≥ m2) = 1 (3) otherwise, the ordinate of the highest intersection point is calculated as (chang, 1996; zhu, et al., 1999; lee, 2009; shaw et al, 2012). v (m2 ≥ m1) = hgt (m1∩m2) = µ (d) = 𝑚1 −−𝑚2 + (𝑚2−𝑚2 +)−(𝑚1−𝑚1 −) (4) equations (5) to (11) can be used for the calculation of the fuzzy synthetic extent value (chang, 1996; zhu et al., 1999; lee, 2009). fi = ∑ 𝑀𝑔𝑖 𝑗 𝑚 𝑗=𝑖 ⊗ (∑ ∑ 𝑀𝑔𝑖 𝑗 𝑚 𝑗=1 𝑛 𝑖=1 ) −1 , 𝑖 = 1,2, … … . . 𝑛 (5) 𝑚2 − m2 d 1 µ (d) m1 𝑚1 − 𝑚2 + 𝑚1 + 𝑚1 𝑚2 ijahp article: kumar/buyer-supplier relationship selection for a sustainable supply: a case of the indian automobile industry international journal of the analytic hierarchy process 225 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.605 ∑ 𝑀𝑔𝑖 𝑗 𝑚 𝑗=𝑖 = (∑ 𝑀𝑖𝑗 − 𝑚 𝑗=𝑖 , ∑ 𝑀𝑖𝑗 𝑚 𝑗=𝑖 , ∑ 𝑀𝑖𝑗 + 𝑚 𝑗=𝑖 ) 𝑖 = 1,2, … … . . 𝑛 (6) (∑ ∑ 𝑀𝑔𝑖 𝑗 𝑚 𝑗=1 𝑛 𝑖=1 ) −1 = [ 1 ∑ ∑ 𝑀𝑖𝑗 +𝑚 𝑗=1 𝑛 𝑖=1 , 1 ∑ ∑ 𝑀𝑖𝑗 𝑚 𝑗=1 𝑛 𝑖=1 , 1 ∑ ∑ 𝑀𝑖𝑗 −𝑚 𝑗=1 𝑛 𝑖=1 ] (7) a convex fuzzy number can be defined as, v (f ≥ f1, f2 …fk) = minv (f ≥ fi), i = 1, 2, …… k (8) d (fi) = minv (f ≥ fk) = 𝑊𝑖 , k = 1, 2, ……, n and k ≠ i (9) based on the above procedure, the weights,𝑊𝑖 , of the factors are = ( 𝑊1 ′, 𝑊2 ′, … . . , 𝑊𝑛 ′) t (10) after normalization, the priority weights are as follows =( 𝑊1, 𝑊2, … . . , 𝑊𝑛 ) t (11) 3.3 bocr addition methods in order to deal with the benefits, opportunities, costs and risks, a pair wise comparison has been done to know which option is more beneficial and more opportunistic in nature. the same approach has been done for costs and risks by asking which option is costlier and riskier in nature. the weights calculated from the pair-wise comparison can be added as proposed by saaty (2003): (1) additive 𝑅𝑒𝑙𝑎𝑡𝑖𝑣𝑒 𝑝𝑟𝑖𝑜𝑟𝑖𝑡𝑦 𝑓𝑜𝑟 𝑎𝑙𝑡𝑒𝑟𝑛𝑎𝑡𝑖𝑣𝑒𝑠 = 𝑏𝐵 + 𝑜𝑂 + 𝑐(1/𝐶) + 𝑟(1/𝑅) (2) probabilistic additive 𝑅𝑒𝑙𝑎𝑡𝑖𝑣𝑒 𝑝𝑟𝑖𝑜𝑟𝑖𝑡𝑦 𝑓𝑜𝑟 𝑎𝑙𝑡𝑒𝑟𝑛𝑎𝑡𝑖𝑣𝑒𝑠 = 𝑏𝐵 + 𝑜𝑂 + 𝑐(1 − 𝐶)𝑛𝑜𝑟𝑚𝑎𝑙𝑖𝑧𝑒𝑑 + 𝑟(1 − 𝑅)𝑛𝑜𝑟𝑚𝑎𝑙𝑖𝑧𝑒𝑑 (3) subtractive 𝑅𝑒𝑙𝑎𝑡𝑖𝑣𝑒 𝑝𝑟𝑖𝑜𝑟𝑖𝑡𝑦 𝑓𝑜𝑟 𝑎𝑙𝑡𝑒𝑟𝑛𝑎𝑡𝑖𝑣𝑒𝑠 = 𝑏𝐵 + 𝑜𝑂 − 𝑐𝐶 − 𝑟𝑅 (4) multiplicative priority powers 𝑅𝑒𝑙𝑎𝑡𝑖𝑣𝑒 𝑝𝑟𝑖𝑜𝑟𝑖𝑡𝑦 𝑓𝑜𝑟 𝑎𝑙𝑡𝑒𝑟𝑛𝑎𝑡𝑖𝑣𝑒𝑠 = 𝐵𝑏𝑂𝑜[(1/𝐶)𝑛𝑜𝑟𝑚𝑎𝑙𝑖𝑧𝑒𝑑]𝑐[(1/𝑅)𝑛𝑜𝑟𝑚𝑎𝑙𝑖𝑧𝑒𝑑]𝑟 (5) multiplicative 𝑅𝑒𝑙𝑎𝑡𝑖𝑣𝑒 𝑝𝑟𝑖𝑜𝑟𝑖𝑡𝑦 𝑓𝑜𝑟 𝑎𝑙𝑡𝑒𝑟𝑛𝑎𝑡𝑖𝑣𝑒𝑠 = 𝐵𝑂/𝐶𝑅 ijahp article: kumar/buyer-supplier relationship selection for a sustainable supply: a case of the indian automobile industry international journal of the analytic hierarchy process 226 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.605 where b, o, c and r represent the synthesized results and b, o, c and r are normalized weights of b, o, c and r, respectively. 3.4 methodology and algorithm a systematic fuzzy ahp model for evaluating the forms of buyer–supplier relationship is proposed in this section. the steps are summarized as follows: step 1: identify the experts and clearly state the problem to them. collaboration is not always a good option, and there are various types of relationships based on the degree of collaboration and expected outcomes of the relationship. an identification of relationship type in the industry under consideration is also required. step 2: decompose the problem hierarchically. develop two hierarchies based on the literature and expert opinions. step 3: a nine-point scale questionnaire is developed for pairwise comparison by the experts (table 2). experts are included from the supply chain and operation management department of the company (lee, 2009). table 2 nine point scale (lee, 2009) fuzzy number membership function 1̃ (1,1,2) �̃� (x-1, x, x+1) for x= 2,3,4,5,6,7,8 9̃ (8,9,9) 1/1̌ (2 -1 , 1 -1 , 1 -1 ) 1/𝑥 ((x+1) -1 , x -1 , (x-1) -1 ) for x= 2,3,4,5,6,7,8 1/9̌ (9 -1 , 8 -1 , 8 -1 ) step 4: combine experts’ opinions on the importance weight for each strategic criterion. for a number of s experts, the synthetic set representing the relative importance level between strategic criteria p and q can be generated by geometric average as (lee, 2009): ℎ− = (∏ 𝑙𝑡 𝑠 𝑡=1 ) 1 𝑠 , ∀t = 1, 2 … s. ℎ = (∏ 𝑚𝑡 𝑠 𝑡=1 ) 1 𝑠 , ∀t = 1, 2 … s. ℎ+ = (∏ 𝑛𝑡 𝑠 𝑡=1 ) 1 𝑠 , ∀t = 1, 2 … s. ijahp article: kumar/buyer-supplier relationship selection for a sustainable supply: a case of the indian automobile industry international journal of the analytic hierarchy process 227 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.605 and (lt, mt, ut) is the lower, middle and upper limit of fuzzy response from expert t. step 5: calculate the relative weights, b, o, c and r, for the four merits b, o, c and r (stage 1). step 6: fuzzy extent analysis method developed by chang (1996) is used to obtaining the crisp relative priority of criteria. step 7: stage 2 calculations. calculate the fuzzy ranking of alternatives under each merit (b, o, c and r) by following step 6. step 8: obtain the performances of each alternative under each qualitative criterion by following step 6. step 9: identify the ranking of each alternative under benefits, opportunities, cost and risk. step 10: synthesize and establish the fuzzy ranking of alternatives under each merit (b, o, c and r) by following the five combination ways as discussed in section 3.3. 4. application of the model on an indian automobile supply chain supplier the effectiveness of the model is discussed by a case study on a multinational automobile company (abc), at the corporate office based in noida (india). the company has four manufacturing units, one research center and five sales offices across india. the company is mainly domestic demand oriented and partially export oriented. the company produces electrical, thermal, electronic, as well as power train products for both two wheelers and four wheelers. the company procures products from various suppliers in both semifinished and finished forms. this study has been done for a plastic mold parts supplier. in this case, the company is sourcing material from four different suppliers. the abc company has decided to improve its sustainability performance and wants to convey this to its suppliers. abc has decided to develop a relationship with its suppliers based on the company performance of several strategic sustainability criteria. these criteria have been selected by six experts and the type of relationship that can be made with the plastic mold parts supplier are as follows: one time relationship, foundation relationship, problem solving relationship, long term relationship and joint development. profiles of experts involved in decision making are as follows: 1. production manager 16 years of experience 2. procurement manager 13 years of experience 3. quality manager (supplier) 9 years of experience 4. professor working on sustainable supply chain management: 25 years of experience. 5. general manager (supplier) 21 years of experience 6. production manager (supplier) 17 years of experience experts from the automobile supply chain finalized the following relationship alternatives for the case study. the types of relationships in the industry under consideration and their definition are as follows: ijahp article: kumar/buyer-supplier relationship selection for a sustainable supply: a case of the indian automobile industry international journal of the analytic hierarchy process 228 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.605 1. one time relationship (carter & rogers 2008; seuring & muller, 2008, vachon, 2007). relationship depends upon the current transaction only. 2. foundation relationship (monczkaet et al., 1998; wu & pagell, 2011; klassen & vachon, 2003). a relationship intended to develop basic trust and commitment between the supplier-buyer. in terms of sustainability, it is related to the basic support extended to each other for developing a sustainable product. 3. problem solving relationship (smith, 2007; lee, 2008; bai & sarkis, 2010; vachon, 2007; bitzer et al., 2008, ageron et al., 2011; elkington, 1994). this relationship is intended to know and solve the supplier’s problems. this type of relationship is problem specific and help is provided to the supplier for handling problems of sustainability adoption. 4. long term trust based relationship (monczkaet et al., 1998; sahay, 2003; walker et al., 2008; lee, 2008; seuring & muller, 2008; bai & sarkis, 2010; zsidisin & hendrick, 1998). in this type of relationship, the buyer and supplier enter into long-term business objectives. trust between the buyer and supplier is very important to attract long-term investments in the relationship. 5. mutual development and growth (klassen & vachon, 2003; wu & pagell, 2011; lee, 2008; ageron et al., 2011; markley & devis, 2007). a relationship focused on setting joints goals and developing a program for sustainability adoption. it also concentrates on mutually developing the capacity and capability of each other. due to the increase in sustainability practices across the industry, the abc company is looking to incorporate sustainability related criteria in the procurement processes. the company found that having excellent relationship management makes a supplier more sustainable and loyal. the relationship between the two parties depends on the capability and capacity of the individual suppliers. management has invited experts from the marketing, production, quality, and research departments for the buyer-supplier relationship selection. 4.1 two-stage model in this study, the problem of the best form of relationship selection is divided into two phases as shown in figures 2 and 3. in phase 1, the hierarchal model is to achieve a sustainable supply chain while maintaining the best form of relationship. the criteria for the best form of relationship in a sustainable supply chain are at level 2 and each criterion can be considered as sub-goals that firms want to achieve by developing a buyer-supplier relationship in terms of sustainability. the sub-criteria selected for this research are: energy usage, pollution emission, waste reduction, employee and society welfare, late delivery, rejection on quality, cost of the product and demand (kumar et al., 2017). the four merits, benefits (b), opportunities (o), costs (c), and risks (r), for the evaluation of buyer-supplier relationships are at level 3. the main objective behind developing phase one is to calculate the relative weights (b, o, c & r) of benefits, opportunities, costs, and risks (saaty, 2005). it is obvious that in the real world priorities for all the merits are not equal and hence these should be calculated. however, it is not easy for experts to determine the priorities of these four merits by doing pairwise comparisons. for example, asking a question like, "what is the relative importance of benefits compared with opportunities in achieving the goal of the best form of buyer–supplier relationship?" is very complex and hard to answer. therefore, as suggested by saaty (2005), a control hierarchy can be used to determine the relative weights b, o, c, & r, for the four merits b, o, c, and r. for example, questions can be reframed in the control hierarchy (phase 1) as ijahp article: kumar/buyer-supplier relationship selection for a sustainable supply: a case of the indian automobile industry international journal of the analytic hierarchy process 229 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.605 "what level of 'benefit' do you associate with strategic criterion 'pollution emission'. experts then respond on the level of benefit of that strategic criterion. in phase 2 of this study, the objective is to identify and select the best form of relationship in a supply chain. bocr are considered here to achieve the goal and there are criteria and sub-criteria (table 1). the weights calculated from phase 1 are used as input and used to calculate the overall priority weight of each relationship type alternatives. the following two-stage model has been developed to solve the relationship selection problem. this case study has been done with respect to the particular supplier. in the first stage, the benefits, costs, opportunities, and risks have been compared with respect to the sustainability criteria finalized for the relationship selection. in the second stage, all the relationship alternatives have been compared with respect to the benefits, opportunities, costs, and risks of the relationship. stage 1 calculate the weight of benefits by comparing the sustainability indicators and demand of the product. in our case, eight criteria have been considered for deciding the buyer-supplier relationship. this includes energy use, emission, waste, employee and society, cost of sourcing, quality of input product, on time delivery and product demand. figure 2 illustrates the comparison hierarchy of stage 1. figure 2 control hierarchy (stage 1) stage 2 all the relationship alternatives are compared with respect to the benefits, costs, risks, and opportunities of the buyer-supplier relationship. benefits (b) opportunity (o) cost (c) risk (r) minimize emission reduce waste employee and society welfare cost of sourcing quality of input material timely delivery minimize energy use demand form of buyer supplier relationship in sustainable supply chain ijahp article: kumar/buyer-supplier relationship selection for a sustainable supply: a case of the indian automobile industry international journal of the analytic hierarchy process 230 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.605 figure 3 stage 2 figure 3 illustrates a comparison hierarchy for selecting the best form of buyer-supplier relationship with respect to the benefits, opportunities, costs, and risks of the relationship. the criteria for the model are benefits, opportunities, costs and risks. each criterion has several sub-criteria (table 1). there are five buyer-supplier relationship alternatives: one time relationship, foundation relationship, problem-solving relationship, long-term trust based relationship and mutual development and growth. a pairwise comparison has been done for each level of the model in order to prioritize the final alternatives. 5. findings and discussion the sustainability criteria have been compared to the relationship selection. experts were asked to compare the criteria to determine the final weight for deciding the best form of the buyer-supplier relationship. the response from the experts was added using lee’s (2009) formula discussed in the computation procedure. the final cumulative response of the experts is shown in table 3. a response matrix has been calculated using chang’s fuzzy method (1996). similar steps were used for the calculation of all possible metrics in stage 1 and stage 2 of our model. mutual development and growth long term trust based relationship problem solving relationship foundation relationship one time relationship b1 b2 b3 o1 o2 o3 c1 c2 c3 r1 r2 r3 benefits (b) risk (r) cost (c) opportunity (o) the best form of buyer supplier relationship ijahp article: kumar/buyer-supplier relationship selection for a sustainable supply: a case of the indian automobile industry international journal of the analytic hierarchy process 231 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.605 table 3 comparison of selection criteria ordering cost rejection on quality late delivery social welfare emission energy use per product demand waste generation ordering cost 1.00,1.00,1.00 1.51,1.82,2.94 0.48,0.66,0.93 1.59,2.08,3.17 1.59,2.08,2.83 1.62,1.91,3.05 1.59,2.08,3.17 1.35,1.82,2.62 rejection on quality 0.34,0.55,0.66 1.00,1.00,1.00 1.00,1.44,2.00 1.59,2.08,3.17 1.59,2.08,3.17 1.51,1.82,2.94 1.26,1.44,2.24 2.14,2.85,3.96 late delivery 1.07,1.52,2.08 0.50,0.69,1.00 1.00,1.00,1.00 1.41,1.73,2.52 1.51,2.18,2.94 1.91,2.62,3.70 1.59,2.08,3.17 1.51,2.04,3.14 social welfare 0.31,0.48,0.63 0.31,0.48,0.63 0.40,0.58,0.71 1.00,1.00,1.00 1.00,1.12,2.14 0.46,0.56,0.87 0.92,1.07,1.73 0.41,0.45,0.78 emission 0.35,0.48,0.63 0.31,0.48,0.63 0.34,0.46,0.66 0.47,0.89,1.00 1.00,1.00,1.00 1.26,1.44,2.52 1.41,1.73,2.83 1.00,1.00,1.41 energy use / product 0.33,0.52,0.62 0.34,0.55,0.66 0.27,0.38,0.52 1.15,1.78,2.15 0.40,0.69,0.79 1.00,1.00,1.00 1.26,1.44,2.52 1.12,1.20,1.59 demand 0.31,0.48,0.63 0.45,0.69,0.79 0.31,0.48,0.63 0.58,0.93,1.09 0.35,0.58,0.71 0.40,0.69,0.79 1.00,1.00,1.00 0.89,0.89,1.78 waste generation / product 0.38,0.55,0.74 0.25,0.35,0.47 0.32,0.49,0.66 1.29,2.24,2.42 0.71,1.00,1.00 0.63,0.83,0.89 0.56,1.12,1.12 1.00,1.00,1.00 ∑ ∑ 𝑀𝑔𝑖 𝑗 = (1,1,1) + (1.51,1.82,2.94) + ⋯ + (1,1,1) = (57.94, 74.71, 103.15) 𝑚 𝑗=1 𝑛 𝑖=1 (∑ ∑ 𝑀𝑔𝑖 𝑗 𝑚 𝑗=1 𝑛 𝑖=1 ) −1 = ( 1 103.15 , 1 74.71 , 1 57.94 ) = (0.0097, 0.0134, 0.0173) ∑ 𝑀𝑔1 𝑗𝑚 𝑗=1 = (1,1,1) + (1.51, 1.82, 2.94) + ⋯ + (1.59,2.08,3.17) = (10.72, 13.44, 19.71) ∑ 𝑀𝑔2 𝑗 𝑚 𝑗=1 = (10.43,13.27,19.15), ∑ 𝑀𝑔3 𝑗 𝑚 𝑗=1 = (10.51,13.87,19.55), ∑ 𝑀𝑔4 𝑗 𝑚 𝑗=1 = (4.82,5.74,8.48) ∑ 𝑀𝑔5 𝑗 𝑚 𝑗=1 = (6.15,7.48,10.68), ∑ 𝑀𝑔6 𝑗 𝑚 𝑗=1 = (5.87,7.57,9.86), ∑ 𝑀𝑔7 𝑗 𝑚 𝑗=1 = (4.30,5.757.43), ∑ 𝑀𝑔8 𝑗 𝑚 𝑗=1 = (5.14,7.59,8.30) 𝐹1 = ∑ 𝑀𝑔𝑖 𝑗 𝑚 𝑗=1 ⊗ [∑ ∑ mgi j m j=1 n i=1 ] −1 = (10.72, 13.44, 19.71) ⊗ (0.0097, 0.0134, 0.0173) = (0.10, 0.18, 0.34) ijahp article: kumar/buyer-supplier relationship selection for a sustainable supply: a case of the indian automobile industry international journal of the analytic hierarchy process 232 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.605 𝐹2 = (10.43,13.27,19.15) ⊗ (0.0097, 0.0134, 0.0173) = (0.10, 0.18, 0.33) 𝐹3 = (10.51,13.87,19.55) ⊗ (0.0097, 0.0134, 0.0173) = (0.10, 0.19, 0.34) 𝐹4 = (4.82,5.74,8.48) ⊗ (0.0097, 0.0134, 0.0173) = (0.05, 0.08, 0.15) 𝐹5 = (6.15,7.48,10.68) ⊗ (0.0097, 0.0134, 0.0173) = (0.06, 0.10, 0.18) 𝐹6 = (5.87,7.57,9.86) ⊗ (0.0097, 0.0134, 0.0173) = (0.06, 0.10, 0.17) 𝐹7 = (4.30,5.757.43) ⊗ (0.0097, 0.0134, 0.0173) = (0.04, 0.08, 0.13) 𝐹8 = (5.14,7.59,8.30) ⊗ (0.0097, 0.0134, 0.0173) = (0.05, 0.10, 0.14) v (f 1 ≥ f2) = 1, v (f 1 ≥ f3) = 1, v (f 1 ≥ f4) = 1, v (f 1 ≥ f5) = 1, v (f 1 ≥ f6) = 1, v (f 1 ≥ f7) = 1, v (f 1 ≥ f8) = 1 similarly, v (f 2 ≥ f 1 f 3 f 4 f 5 f 6 f 7 f 8) = (1, 1, 1, 1, 1, 1, 1) v (f 3 ≥ f 1 f 2 f 4 f 5 f 6 f 7 f 8) = (1, 1, 1, 1, 1, 1, 1) v (f 4 ≥ f 1 f 2 f 3 f 5 f 6 f 7 f 8) = (0.375, 0.444, 0.444, 1, 1, 1, 0.857) v (f 5 ≥ f 1 f 2 f3 f 4 f 6 f 7 f 8) = (0.143, 0.25, 0.25, 0.80, 1, 0.833, 0.667) v (f 6 ≥ f 1 f 2 f3 f 4 f 5 f 7 f 8) = (0.286, 0.375, 0.375, 1, 1, 1, 0.833) v (f 7 ≥ f 1 f 2 f3 f 4 f 5 f 6 f 8) = (0.143, 0.25, 0.25, 0.80, 1, 0.833, 0.667) v (f 8 ≥ f 1 f 2 f3 f 4 f 5 f 6 f 7) = (0.429, 0.50, 0.50, 1, 1, 1, 1) the weight vectors are calculated as follows: d (f1) = min v (f 1 ≥ f 2 f 3 f 4 f 5 f 6 f 7 f 8) = min (1, 1, 1, 1, 1, 1, 1) = 1 d (f2) = min v (f 2 ≥ f 1 f 3 f 4 f 5 f 6 f 7 f 8) = min (1, 1, 1, 1, 1, 1, 1) = 1 d (f3) = min v (f 3 ≥ f 1 f 2 f 4 f 5 f 6 f 7 f 8) = min (1, 1, 1, 1, 1, 1, 1) = 1 d (f4) = min v (f 4 ≥ f 1 f 2 f 3 f 5 f 6 f 7 f 8) = min (0.375, 0.444, 0.444, 1, 1, 1, 0.857) = 0.375 d (f5) = min v (f 5 ≥ f 1 f 2 f3 f 4 f 6 f 7 f 8) = min (0.143, 0.25, 0.25, 0.80, 1, 0.833, 0.667) = 0.143 d (f6) = min v (f 6 ≥ f 1 f 2 f3 f 4 f 5 f 7 f 8) = min (0.286, 0.375, 0.375, 1, 1, 1, 0.833) = 0.286 d (f7) = min v (f 7 ≥ f 1 f 2 f3 f 4 f 5 f 6 f 8) = min (0.143, 0.25, 0.25, 0.80, 1, 0.833, 0.667) = 0.143 d (f8) = min v (f 8 ≥ f 1 f 2 f3 f 4 f 5 f 6 f 7) = min (0.429, 0.50, 0.50, 1, 1, 1, 1) = 0.429 𝑊 ′= (d (f1) d (f2) d (f3) d (f4) d (f5) d (f6) d (f7) d (f8)) t = (1,1,1, 0.375, 0.143, 0.286, 0.143, 0.429) t = (0.229, 0.229, 0.229, 0.086, 0.033, 0.065, 0.033, 0.098) the normalized priorities of the sustainability criteria for the relationship selection are 0.229, 0.229, 0.229, 0.086, 0.033, 0.065, 0.033, and 0.98 (figure 2). the final priorities of the benefits, costs, opportunities and risks can be calculated by following the same procedure. the final priority weights for the stage 1 of the model are shown in table 4. the final normalized weights have been calculated by multiplying the weights of sustainability criteria and bocr weight of the control criteria. for example, the weight of benefit is calculated by: 0.229*0.415+0.229*0.401+0.229*0.374+0.033*0.543+0.065*0.396+0.033*0.364+0.098 *0.524 = 0.426 the weights of opportunity, cost and risk are 0.260, 0.226 and 0.088 respectively. ijahp article: kumar/buyer-supplier relationship selection for a sustainable supply: a case of the indian automobile industry international journal of the analytic hierarchy process 233 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.605 table 4 bocr rating ordering cost rejection on quality late delivery social welfare emission energy demand waste normalized weight 0.229 0.229 0.229 0.086 0.033 0.065 0.033 0.098 benefits 0.415 0.401 0.374 0.554 0.543 0.396 0.364 0.524 0.426 opportunities 0.333 0.209 0.291 0.197 0.215 0.244 0.297 0.200 0.260 costs 0.149 0.289 0.200 0.212 0.229 0.349 0.191 0.257 0.226 risks 0.103 0.101 0.135 0.037 0.013 0.010 0.148 0.019 0.088 these priorities are obtained by comparing the bocr with respect to the strategic criteria. the strategic criteria were selected from the literature and expert opinions (figure 3). in stage 2, the relative weight of the criteria and sub-criteria are listed in table 5. the most important control criterion under the benefit category is financial benefits, which have a priority of 0.411. it means a firm keen on developing a relationship with the supplier for developing a more sustainable supply chain should look for the financial benefits. in the sub-criteria under benefits, reducing distribution costs is preferred, having a priority of 0.2198. other major benefits in the sub-criteria are low cost of information (0.1465), improved internal process (0.1278) and reduced pressure from the external agencies (0.1277). the improvement of quality on sustainability standards also makes a significant contribution in the benefits sub-criteria (0.1192). under the opportunity merit, improved corporate image (0.1698) under marketing advantage and sharing technology and knowledge (0.1516) under technical capabilities are the most important criteria. this implies the buyer-supplier relationship in a sustainable supply chain is developed for improving the buyer’s and supplier’s image, and companies look for sharing technology and knowledge about increasing sustainable practices. under the cost merit, all the control costs of a relationship (0.352), impact of relationship (0.343) and the cost of adoption (0.304) have nearly equal priority. this is because each type of relationship has a different cost of developing the relationship. the type of relationship is also dependent upon the type of adoption the supplier needs in terms of sustainability and impact of the relationship in developing sustainability performance. under the opportunity merit, the most important sub-criteria are financial investment for the relationship (0.1598), cost of infrastructure development (0.1373), time required for developing a relationship (0.1299) and perception of the relationship success (0.1209). under risk merit, management of the relationship is the most important control criteria having a priority of 0.815. all the important sub-criteria that come under the relationship management control criteria are lack of trust (0.2485), lack of integration (0.2685) and problem in sharing loss (0.2980). this means that a firm worries the most about the capability of managing the relationship. the relative importance of relationship alternatives is shown in table 6. under the benefits merit, long-term relationship and joint development performed well with a priority of 0.319 and 0.309, respectively. problem solving relationship is also at 0.306. in the opportunity merit, joint development is the best option with a priority of 0.332. longijahp article: kumar/buyer-supplier relationship selection for a sustainable supply: a case of the indian automobile industry international journal of the analytic hierarchy process 234 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.605 term relationship is at 0.323. however, under the cost merit, foundation relationship becomes the best with a weight of 0.626 and one-time relationship is in second place with a weight of 0.272. under the merit of risk, one-time relationship has a high priority (0.747) followed by the foundation relationship (0.150). consistency check of the metrics all the comparison metrics were checked for the consistency. some authors only verify the consistency for crisp matrices whose elements are the middle significant values of the triangular fuzzy numbers from the corresponding fuzzy pair-wise comparison matrix (tesfamariam & sadiq, 2006, pan, 2008, vahidnia et al., 2009). the above matrix (table 3) was transformed into a crisp matrix by considering only the middle value of the triangular fuzzy number. the following values were found: cr= 0.040970119, ri (for n=8) = 1.41, ci=0.029056822, as ci is less than 0.1, the above matrix is considered consistent. a similar approach was used for each comparison matrix. there are other methods to check the consistency of a fuzzy matrix such as zheng et al. (2012), but in this approach, the crisp matrix obtained is not reciprocal, hence its check of consistency is questionable. ijahp article: kumar/buyer-supplier relationship selection for a sustainable supply: a case of the indian automobile industry international journal of the analytic hierarchy process 235 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.605 table 5 relative priorities of control criteria and subcriteria merits control criteria sub criteria normalized priority integrated priority benefits (0.426) financial (b1) (0.411) reduce distribution cost 0.516 0.2198 low cost of information 0.344 0.1465 reduce inventory cost 0.140 0.0596 operational (b2) (0.326) improve internal process 0.392 0.1278 resource optimization 0.301 0.0981 on time delivery 0.307 0.1001 sustainability adoption (b3) (0.263) improved quality on sustainability standards 0.453 0.1192 reduced pressure from various agencies 0.485 0.1277 sustainable supply chain 0.061 0.0161 opportunities (0.260) marketing advantage (o1) (0.454) improve corporate image/ reputation 0.374 0.1698 premium pricing 0.291 0.1321 product differentiation 0.200 0.0908 new market 0.135 0.0613 technical capabilities (o2) (0.369) sharing technology and knowledge 0.411 0.1516 educating each other’s employee 0.326 0.1202 developing technical standards 0.263 0.0972 mutual growth (o3) (0.177) capacity building and development 0.600 0.1063 sharing resources and information 0.257 0.0457 jointly setting goals 0.141 0.0250 costs (0.226) cost of relationship (c1) (0.352) financial investment for relationship 0.454 0.1598 time required to develop relationship 0.369 0.1299 responsibility sharing 0.177 0.0623 impact of relationship (c2) (0.343) perception of relationship success 0.352 0.1209 no improvement in sustainability performance 0.343 0.1178 poor partner commitment 0.304 0.1043 cost of adoption (c3) (0.304) technological changes 0.244 0.0742 process change 0.304 0.0925 infrastructure development 0.452 0.1373 risks (0.088) management (r1) (0.815) lack of trust 0.305 0.2485 problem in sharing risk 0.329 0.2685 lack of integration 0.366 0.2980 market (r2) (0.162) dependency on few suppliers 0.815 0.1320 bargaining power of supplier 0.163 0.0264 competition in future 0.022 0.0036 ijahp article: kumar/buyer-supplier relationship selection for a sustainable supply: a case of the indian automobile industry international journal of the analytic hierarchy process 236 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.605 investment (r3) (0.022) huge investment for relationship 0.233 0.0051 unavailability of required technology with partners 0.145 0.0032 breaking partnership in between 0.623 0.0137 table 6 priorities of alternatives under the four merits merits benefits (0.426) opportunities (0.260) alternatives relative normalized relative normalized one time relationship 0.150 0.048 0.151 0.050 foundation relationship 0.059 0.019 0.090 0.030 problem solving relationship 0.958 0.306 0.800 0.266 long term relationship 1.000 0.319 0.973 0.323 joint development 0.968 0.309 1.000 0.332 merits cost (0.226) risk (0.088) alternatives relative normalized reciprocal normalized relative normalized reciprocal normalized one time relationship 0.073 0.032 31.10 0.272 0.035 0.013 75.28 0.747 foundation relationship 0.032 0.014 71.45 0.626 0.173 0.066 15.12 0.150 problem solving relationship 0.351 0.156 06.41 0.056 0.560 0.214 04.67 0.046 long term relationship 0.800 0.355 02.81 0.024 0.852 0.325 03.07 0.030 joint development 1.000 0.443 02.25 0.019 1.000 0.382 02.62 0.026 ijahp article: kumar/buyer-supplier relationship selection for a sustainable supply: a case of the indian automobile industry international journal of the analytic hierarchy process 237 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.605 table 7 final synthesis of priorities of alternatives synthesis methods additive probabilistic additive subtractive multiplicative priority powers multiplicative alternatives priority rank priority rank priority rank priority rank priority rank one time relationship 0.094985 v 0.3391992 iv 0.024971 iv 0.043697 iv 0.000991 v foundation relationship 0.157577 iv 0.3209759 v 0.006748 v 0.041111 v 0.002649 iv problem solving relationship 0.205222 iii 0.4593121 i 0.145084 i 0.170002 i 0.111408 i long term relationship 0.234654 ii 0.4251118 ii 0.110884 ii 0.162888 ii 0.094357 ii joint development 0.235069 i 0.3980857 iii 0.083857 iii 0.158281 iii 0.088181 iii table 8 sensitivity analysis merit benefits (0.426) opportunities (0.260) change in merit with synthesis method b decreases b increases o decreases o increases b best alternatives b best alternatives o best alternatives o best alternatives additive no change mdg 0.467 ltr, mdg 0.213 ltr, mdg no change mdg probabilistic additive no change psr no change psr no change psr 0.858 ltr, psr subtractive no change psr no change psr no change psr 0.858 ltr, psr multiplicative priority powers no change psr no change psr no change psr 0.479 ltr, psr multiplicative no change psr no change psr no change psr no change psr merit cost (0.226) risk (0.088) change in merit with synthesis method c decreases c increases r decreases r increases c best alternatives c best alternatives r best alternatives r best alternatives additive no change mdg 0.354 ltr (0.311), fr no change psr no change mdg probabilistic additive 0.053 ltr, psr no change psr no change psr 0.688 fr subtractive no change psr no change psr no change psr no change psr multiplicative priority powers 0.173 ltr, psr 0.815 fr no change psr 0.151 mdg multiplicative no change psr no change psr no change psr no change psr ijahp article: kumar/buyer-supplier relationship selection for a sustainable supply: a case of the indian automobile industry international journal of the analytic hierarchy process 215 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.605 the final alternative priority is calculated by using five different combination methods shown in table 7. under all the combination methods, the problem solving relationship ranks first, except in the additive method, while joint development scores first in the additive method and ranks third in the remaining methods. long-term relationship scores second under all methods of combination. under the additive method, long-term relationship ranks second and joint development ranks first with scores of 0.235069 and 0.234653, respectively (insignificant difference of only 0.000415). similarly, the problem solving relationship ranks third, foundation relationship ranks fourth and one-time relationship ranks fifth with the scores of 0.205222, 0.157577 and 0.094985, respectively. under the probabilistic additive method and the subtractive and multiplicative priority powers method, the ranking of all the alternatives is the same. the problem solving relationship ranked first, followed by long-term relationship and joint development which ranked second and third. one-time relationship ranked fourth in all the synthesis methods of combining priorities. in the multiplicative synthesis method, one-time relationship ranked fifth and foundation relationship is at rank four with priorities (0.000991) and (0.002649), respectively. the reason for the good performance of the alternative “problem solving relationship” is the moderate performance of all the merits. other relationship types ranked second and third are more beneficial and have more opportunity, but have more risks and costs. a sensitivity analysis was performed to check the change in the priority level of the alternatives. the sensitivity analysis was done by changing the value of one merit while keeping the others constant. table 8 shows the change in the priority level of the alternatives with the change in the priorities of benefits, opportunities, costs and risks. the sensitivity analysis was performed by changing the value of one strategic factor and keeping the value of the other constant. when b decreases from 0.426, mutual development and growth remain the most preferred alternative in the additive method. figure 4 shows the sensitivity analysis in the additive method with respect to the change in the value of b. in all the other methods of combination, the problem-solving relationship is the most preferred. similarly, when opportunity is decreased to 0.213, long-term relationship is the most preferred. a special case is seen with the cost factor. when the cost priority is increased by more than 0.226, the most preferred alternatives at 0.311 are ltr and mdg and when it is increases further, after 0.354 the foundation relationship is the most suitable alternative in terms of cost. similarly, all the variances can be seen in table 8. in this study, the problem-solving relationship was preferred, since it is focused on a specific problem related to sustainability adoption. it has been found to be moderate on the benefits and opportunities; it also has moderate values on risk and cost compared to the other alternatives. ijahp article: kumar/buyer-supplier relationship selection for a sustainable supply: a case of the indian automobile industry international journal of the analytic hierarchy process 216 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.605 figure 4 sensitivity analysis with respect to the change in the value of b in additive method 6. managerial implications and conclusions the primary aim of this study was to evaluate the forms of buyer-supplier relationships for developing a more sustainable supply chain model using ahp based on benefits, opportunities, costs and risks (bocr). in contrast to most supplier selection models available in the extant literature, the proposed model takes into consideration the benefits, costs, opportunities and risks related to development of buyer-supplier relationships in a sustainable supply chain. firms can apply the learning from this study while choosing an appropriate relationship type with supply chain partners in their supply chains. it is to be noted, that this study has also integrated learning from fuzzy theory in the development of a sustainable buyer-supplier relationship model to address the issues of ambiguity and vagueness faced by managers while making supply chain decisions. managers can use the identified criteria and criteria of bocr for developing relationships with their supply chain partners. for example, managers can understand the benefits of a relationship type in term of sustainability adoption, finance and operations. similarly, managers can compare the relationship type in terms of opportunity of a relationship type considering the marketing advantage, technical capability and mutual growth. at the same time, managers can incorporate several criteria related to cost and risk of a relationship type. for example, cost of developing a relationship (financial investment, time required, and responsibility sharing) with respect to a supplier provides better decision making possibilities to managers. this model also provides information on the most profitable relationship alternative with respect to changes in the priority of benefits, opportunities, costs and risks that are manifested in buyer-supplier relationships in addition to the sustainability indicators for deciding the best form of the buyer-supplier relationship (stage 1). this study provides the supply chain managers with the freedom to include customized sustainability ijahp article: kumar/buyer-supplier relationship selection for a sustainable supply: a case of the indian automobile industry international journal of the analytic hierarchy process 217 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.605 indicators based on the industry and specific problems under consideration. in addition to this, the supply chain managers can use this model as a framework to evaluate their current supply chain relationships. a sensitivity analysis based on the model may provide these managers with the knowledge to decide relationship alternatives with the changing priorities of bocr. this study has also considered five types of addition methods to incorporate all possible decision realities related to the selection of the best form of relationship. the proposed model can further be modified with respect to other industries to cater to their specific needs from their supply chains considering the benefits, opportunities, costs, and risks of developing such buyer-supplier relationships. another contribution of this study includes the identification of several criteria and sub-criteria under benefits, opportunities, costs and risks of relationship type in the context of a sustainable supply chain. this study provides an opportunity in future research to analyze the benefits, opportunities, costs and risks of existing relationship types in industries using empirical study on larger sample. ijahp article: kumar/buyer-supplier relationship selection for a sustainable supply: a case of the indian automobile industry international journal of the analytic hierarchy process 218 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.605 references ageron, b., gunasekaran, a., & spalanzani, a. (2011). sustainable supply management: an empirical study. international journal of production economics, 140(1), 168-182. doi: https://doi.org/10.1016/j.ijpe.2011.04.007 asif, m., bruijn, e. j. d., fisscher, o. a. m., &steenhuis, h. (2008). achieving sustainability three dimensionally. icmit 2008. 4th ieee international conference, 423428. doi: https://doi.org/10.1109/icmit.2008.4654402 attaran, m. &attaran, s. (2007). collaborative supply chain management the most promising practice for building efficient and sustainable supply chains. business process management journal, 13(3), 390-404. doi: https://doi.org/10.1108/14637150710752308 bai, c. & sarkis, j. (2010). green supplier development: analytical evaluation using rough set theory. journal of cleaner production, 18(12), 1200-1210. doi: https://doi.org/10.1016/j.jclepro.2010.01.016 bitzer, v., francken, m., &glasbergen, p. (2008). intersectoral partnership for a sustainable coffee chain: really addressing sustainability or just picking (coffee) cherries. global environmental change, 18(2), 271-284. doi: https://doi.org/10.1016/j.gloenvcha.2008.01.002 bommel, w. v. (2010). a conceptual framework for analyzing sustainability strategies in industrial supply networks from an innovation perspective. journal of cleaner production, 19(8), 895-904. doi: https://doi.org/10.1016/j.jclepro.2010.12.015 bowen, f. e., cousins, p. d., lamming, r. c., & faruk, a. c. (2001). the role of supply management capabilities in green supply. production and operations management, 10(2), 174-189. doi: https://doi.org/10.1111/j.1937-5956.2001.tb00077.x brito, m., p. d., carbone, v., &blanquart, c. (2008). towards a sustainable fashion retail supply chain in europe: organisation and performance. international journal of production economics, 114(2), 534-55. doi: https://doi.org/10.1016/j.ijpe.2007.06.012 cai, s., souza, r. d., goh, m., li, w., lu, q., & sundarakani, b. (2008). the adoption of green supply strategy: an institutional perspective. proceedings of the 2008 ieee icmit, 1044-1049. doi: https://doi.org/10.1109/icmit.2008.4654512 cannon, j. p. & perreault w. d. jr. (1999). buyer–seller relationships in business markets. journal of marketing research, 36(4), 439-460. doi: https://doi.org/10.2307/3151999 carter, c. r. & rogers, d. s. (2008). a framework of sustainable supply chain management: moving towards new theory. international journal of physical distribution & logistics management, 38(5), 360-387. doi: https://doi.org/10.1016/s13665545(01)00008-4 https://doi.org/10.1016/j.ijpe.2011.04.007 ijahp article: kumar/buyer-supplier relationship selection for a sustainable supply: a case of the indian automobile industry international journal of the analytic hierarchy process 219 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.605 carter, c. r. &jenning, m. m. (2002). social responsibility and supply chain relationships. transportation research part e: logistics and transportation review, 38(1), 37–52. doi: https://doi.org/10.1016/s1366-5545(01)00008-4 chae, s., choi, t. y., & hur, d. (2017). buyer power and supplier relationship commitment: a cognitive evaluation theory perspective. journal of supply chain management, 53(2), 39-60. doi: https://doi.org/10.1111/jscm.12138 chan, f. t. s., & kumar, n. (2007). global supplier development considering risk factors using fuzzy extended ahp based approach. omega, 35(4), 417–431. doi: https://doi.org/10.1016/j.omega.2005.08.004 chang, d. y. (1996). applications of the extent analysis method on fuzzy ahp. european journal of operational research, 95(3), 649-655. doi: https://doi.org/10.1016/0377-2217(95)00300-2 chen, s. m. (1996). evaluating weapon systems using fuzzy arithmetic operations. fuzzy sets and systems, 77(3), 265-276. doi: https://doi.org/10.1016/0165-0114(95)00096-8 cheung, y. k. f. & rowlinson, s. (2011). supply chain sustainability: a relationship management approach. international journal of managing projects in business, 4(3), 480-497. doi: https://doi.org/10.1108/17538371111144184 ciliberti, f., pontrandolfoa, p., & scozzi, b. (2008). logistics social responsibility: standard adoption and practices in italian companies. international journal of production economics, 113(1), 88-106. doi: https://doi.org/10.1016/j.ijpe.2007.02.049 clemens, b &douglus, t.j. (2006). does coercion drive firm to adopt 'voluntary' green initiatives? relationship among coercion, superior firm resources and voluntary green initiatives. journal of business research, 59(4), 483-491. doi: https://doi.org/10.1016/j.jbusres.2005.09.016 closs, d. j., speier, c., & meacham, n. (2010). sustainability to support end-to-end value chains: the role of supply chain management. journal of academy of marketing science, 39, 101-116. doi: https://doi.org/10.1007/s11747-010-0207-4 contractor, f. j. &lorange, p. (2002). the growth of alliances in the knowledge-based economy. international business review, 11(4), 485-502. doi: https://doi.org/10.1016/s0969-5931(02)00021-5 cramer, j. m. (2007). organising corporate social responsibility in international product chains. journal of cleaner production, 16(3), 395–400. doi: https://doi.org/10.1016/j.jclepro.2007.01.007 darnall, n., jolley, g. j., & horvath, a. (2008). environmental management system and green supply chain management: complements for sustainability?.business strategy and the environment, 17(1), 30-45. doi: https://doi.org/10.1002/bse.557 https://doi.org/10.1111/jscm.12138 http://onlinelibrary.wiley.com/doi/10.1002/bse.v17:1/issuetoc ijahp article: kumar/buyer-supplier relationship selection for a sustainable supply: a case of the indian automobile industry international journal of the analytic hierarchy process 220 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.605 daugherty, p. j. (2011). review of logistics and supply chain relationship literature and suggested research agenda. international journal of physical distribution & logistics management, 41(1), 16-31. doi: https://doi.org/10.1108/09600031111101402 diabata, a. & govindanb, k. (2011). an analysis of the drivers affecting the implementation of green supply chain management. resources, conservation and recycling, 55(6), 659-667. doi: https://doi.org/10.1016/j.resconrec.2010.12.002 elkington, j. (1994). towards the sustainable corporation: win-win-win business strategies for sustainable development. california management review, 36(winter), 90100. doi: https://doi.org/10.2307/41165746 ellis, n. & higgins, m. (2006). recatechizing codes of practice in supply chain relationships: discourse, identity and otherness. journal of strategic management, 14(4), 387-410. doi: https://doi.org/10.1080/09652540600947870 eltayeb, t. k., zailani, s., &ramayah, t. (2011). green supply chain initiatives among certified companies in malaysia and environmental sustainability: investigating the outcomes. resources, conservation and recycling, 55(5), 495-506. doi: https://doi.org/10.1016/j.resconrec.2010.09.003 florida, r. (1996). lean and greenthe move to environmentally conscious manufacturing. california management review, 39(1), 80-105. doi: https://doi.org/10.2307/41165877 ford, d. (1980). the development of buyer-seller relationships in industrial markets. european journal of marketing, 14(5/6), 339-353. doi: https://doi.org/10.1108/eum0000000004910 fortes, j. (2009). green supply chain management: a literature review. otago management graduate review, 7, 51-62. gimenez, c. &tachizawa, e. m. (2012). extending sustainability to suppliers: a systematic literature review. supply chain management: an international journal, 17(5), 531-543. doi: https://doi.org/10.1108/13598541211258591 golicic, s. l., foggin, j. h., & mentzer, j. t. (2003). relationship magnitude and its role in interorganizational relationship structure. journal of business logistics, 24(1), 57-75. doi: https://doi.org/10.1002/j.2158-1592.2003.tb00032.x green, k., morton, b., & new, s. (1998). green purchasing and supply policies: do they improve companies’ environmental performance? supply chain management, 3(2), 8995. doi: https://doi.org/10.1108/13598549810215405 gronroos, c. (1990). relationship approach to marketing in service contexts: the marketing and organizational behavior interface. journal of business research, 20(1), 311. doi: https://doi.org/10.1016/0148-2963(90)90037-e https://doi.org/10.2307/41165746 https://doi.org/10.1080/09652540600947870 https://doi.org/10.1016/j.resconrec.2010.09.003 https://doi.org/10.1108/eum0000000004910 https://doi.org/10.1108/13598541211258591 https://doi.org/10.1002/j.2158-1592.2003.tb00032.x https://doi.org/10.1016/0148-2963(90)90037-e ijahp article: kumar/buyer-supplier relationship selection for a sustainable supply: a case of the indian automobile industry international journal of the analytic hierarchy process 221 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.605 grönroos, c. (1997). value‐driven relational marketing: from products to resources and competencies. journal of marketing management, 13(5), 407-419. doi: https://doi.org/10.1080/0267257x.1997.9964482 hadjikhani, a. & laplaca, p. (2013). development of b2b marketing theory. industrial marketing management, 41(3), 294-305. doi: https://doi.org/10.1016/j.indmarman.2013.03.011 hall, j. (2000). environmental supply chain dynamics. journal of cleaner production, 8(6), 455-471. hammerschmidt, m., wetzel, h. a., & arnold, t. j. (2018). the burden of rank: the impact of preferred supplier status on excessive buyer requests. (in press) industrial marketing management. doi: https://doi.org/10.1016/j.indmarman.2017.10.005 hamprecht, j., corsten, d., noll, m., &meier, e. (2005). controlling the sustainability of food supply chain. supply chain management: an international journal, 10(1), 7-10. doi: https://doi.org/10.1108/13598540510578315 handfield, r. b., scroufe, r., & walton, s. (2005). integrating environmental management and supply chain strategies. business strategies and the environment, 14(1), 1-9. hansen, j. m. (2009). the evolution of buyer-supplier relationships: an historical industry approach. journal of business & industrial marketing, 24(3/4), 227-236. doi: https://doi.org/10.1108/08858620910939778 holt, d. &ghobadian, a. (2009). an empirical study of green supply chain management practices amongst uk manufacturers. journal of manufacturing technology management, 20(7), 933-956. doi: https://doi.org/10.1108/17410380910984212 hong, p., kwon, h., & roh, j. j. (2009). implementation of strategic green orientation in supply chain: an empirical study of manufacturing firms. european journal of innovation management, 12(4), 512-532. doi: https://doi.org/10.1108/14601060910996945 hsu, c. c., tan, k. c., zailani, s. h. m., &jayaraman, v. (2013). supply chain drivers that foster the development of green initiatives in an emerging economy. international journal of operations & production management, 33(6), 656-688. doi: https://doi.org/10.1108/ijopm-10-2011-0401 hutchins, m. j. & sutherland, j. w. (2008). an exploration of measures of social sustainability and their application to supply chain decisions. journal of cleaner production, 16(15), 1688-1698. doi: https://doi.org/10.1016/j.jclepro.2008.06.001 kam, b. h., & lai, m. k. (2018). buyer-supplier exchange relationship: how do exchange partners behave across the relationship life-cycle?. transportation research part e: logistics and transportation review, 113, 239-257 doi: https://doi.org/10.1016/j.tre.2017.12.007 https://doi.org/10.1016/j.indmarman.2017.10.005 https://doi.org/10.1108/08858620910939778 https://doi.org/10.1108/14601060910996945 https://doi.org/10.1108/ijopm-10-2011-0401 ijahp article: kumar/buyer-supplier relationship selection for a sustainable supply: a case of the indian automobile industry international journal of the analytic hierarchy process 222 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.605 keatinga, b., quazib, a., krizc, a., &coltmana, t. (2008). in pursuit of a sustainable supply chain: insights from westpac banking corporation. supply chain management: an international journal, 13(3), 175-179. doi: https://doi.org/10.1108/13598540810871217 klassen, r. &vachon, s. (2003). collaboration and evaluation in the supply chain: the impact on plant-level environmental investment. production and operations management, 12(3), 336-352. doi: https://doi.org/10.1111/j.1937-5956.2003.tb00207.x kogg, b. (2003). greening the cotton textile supply chain: a case study of the transition towards organic production without a powerful focal company. greener management international, 43, 53-64. doi: https://doi.org/10.9774/gleaf.3062.2003.au.00007 koplin, j., seuring, s., &mesterharm, m. (2007). incorporating sustainability into supply management in the automotive industry-the case of the volkswagen ag. journal of cleaner production, 15(11–12), 1053-1062. doi: https://doi.org/10.1016/j.jclepro.2006.05.024 kumar, d. & rahman, z. (2013). modelling relationship marketing strategies for sustainability adoption. international journal of intercultural information management, 3(4), 277-291. doi: https://doi.org/10.1504/ijiim.2013.059971 kumar, d., & garg, c. p. (2017). evaluating sustainable supply chain indicators using fuzzy ahp: case of indian automotive industry. benchmarking: an international journal, 24(6), 1742-1766. doi: https://doi.org/10.1108/bij-11-2015-0111 kumar, d., & rahman, z. (2015). sustainability adoption through buyer supplier relationship across supply chain: a literature review and conceptual framework. international strategic management review, 3(1), 110-127. doi: https://doi.org/10.1016/j.ism.2015.04.002 kumar, d., & rahman, z. (2016). buyer supplier relationship and supply chain sustainability: empirical study of indian automobile industry. journal of cleaner production. 131. doi: https://doi.org/10.1016/j.jclepro.2016.04.007 kumar, d., rahman, z., & chan, f. t. (2017). a fuzzy ahp and fuzzy multi-objective linear programming model for order allocation in a sustainable supply chain: a case study. international journal of computer integrated manufacturing, 1-17. doi: https://doi.org/10.1080/0951192x.2016.1145813 lee, a. h. i. (2009). a fuzzy ahp evaluation model for buyer–supplier relationships with the consideration of benefits, opportunities, costs, and risks. international journal of production research, 47(15), 4255-4280. doi: https://doi.org/10.1080/00207540801908084 lee, a. h. i., kang, h. y., & wang, w. p. (2005). analysis of priority mix planning for semiconductor fabrication under uncertainty. international journal of advanced manufacturing technology, 28(3–4), 351–361. https://doi.org/10.1108/13598540810871217 https://doi.org/10.9774/gleaf.3062.2003.au.00007 https://doi.org/10.1016/j.jclepro.2006.05.024 https://doi.org/10.1108/bij-11-2015-0111 ijahp article: kumar/buyer-supplier relationship selection for a sustainable supply: a case of the indian automobile industry international journal of the analytic hierarchy process 223 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.605 lee, h. i., kang, h. y., hsu, c. f., & hung, h. c. (2009a). a green supplier selection model for high-tech industry. expert systems with applications, 36(4), 7917-7927. doi: https://doi.org/10.1016/j.eswa.2008.11.052 lee, s. (2008). drivers for the participation of small and medium sized suppliers in green supply chain initiatives. supply chain management: an international journal, 13(3), 185198. doi: https://doi.org/10.1108/13598540810871235 linton, j. d., klassen, r., & jayaraman, v. (2007). sustainable supply chain: an introduction. international journal of operations management, 25(6), 1075-1082. doi: https://doi.org/10.1016/j.jom.2007.01.012 luthra, s., kumar, v., kumar, s., & haleem, a. (2011). barriers to implement green supply chain management in automobile industry using interpretive structural modeling technique-an indian perspective. journal of industrial engineering and management, 4(2), 231-257. doi: https://doi.org/10.3926/jiem.2011.v4n2.p231-257 mahler, d. (2007) the sustainable supply chain. supply chain management review, 5960. markley, m. j. & davis, l. (2007). exploring future competitive advantage through sustainable supply chains. international journal of physical distribution & logistics management, 37(9), 763-774. doi: https://doi.org/10.1108/09600030710840859 matos, s. & hall, j. (2007). integrating sustainable development in the supply chain: the case of life cycle assessment in oil and gas and agriculture biotechnology. journal of operation management, 25(6), 1083-1102. doi: https://doi.org/10.1016/j.jom.2007.01.013 monczka, r. m., petersen, k. j., handfield, r. b., &ragatz, g. l. (1998). success factors in strategic supplier alliances: the buying company perspective. decision sciences, 29(3), 553-577. doi: https://doi.org/10.1111/j.1540-5915.1998.tb01354.x muller, m., dos santos, v. g., &seuring, s. (2009). the contribution of environmental and social standards towards ensuring legitimacy in supply chain governance. journal of business ethics, 89(4), 509-523. doi: https://doi.org/10.1007/s10551-008-0013-9 nakano, k. & hirao, m. (2011). collaborative activity with business partners for improvement of product environmental performance using lca. journal of cleaner production, 19(11), 1189–1197. doi: https://doi.org/10.1016/j.jclepro.2011.03.007 ni, d, li, k. w., & tang, x. (2010). social responsibility allocation in two-echelon supply chains: insights from wholesale price contracts. european journal of operational research, 207(3), 1269-1279. doi: https://doi.org/10.1016/j.ejor.2010.06.026 olorunniwo, f. o. & li, x. (2010). information sharing and collaboration practices in reverse logistics. supply chain management: an international journal, 15(6), 454-462. doi: https://doi.org/10.1108/13598541011080437 https://doi.org/10.1016/j.eswa.2008.11.052 https://doi.org/10.1108/09600030710840859 https://doi.org/10.1016/j.jom.2007.01.013 https://doi.org/10.1007/s10551-008-0013-9 https://doi.org/10.1016/j.ejor.2010.06.026 https://doi.org/10.1108/13598541011080437 ijahp article: kumar/buyer-supplier relationship selection for a sustainable supply: a case of the indian automobile industry international journal of the analytic hierarchy process 224 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.605 pagell, m., & shevchenko, a. (2014). why research in sustainable supply chain management should have no future. journal of supply chain management, 50(1), 4455.doi: https://doi.org/10.1111/jscm.12037 peters, n. j., hofdtetter, j. s., and hoffmann, v. h. (2011). institutional entrepreneurship capabilities for inter-organizational sustainable supply chain strategies. international journal of logistics management, 22(1), 52-86. doi: https://doi.org/10.1108/09574091111127552 preuss, l. (2005). rhetoric and reality of corporate greening-a view from the supply chain management function. business strategy and environment, 14(2), 123-139. doi: https://doi.org/10.1002/bse.435 pullman, m. e., maloni, m. j., & dillard, j. (2010). sustainability practices in food supply chains: how is wine different?. journal of wine research, 21(1), 35-56. doi: https://doi.org/10.1080/09571264.2010.495853 rao, p. & holt, d. (2005). do green supply chain lead to competitiveness and economic performance?.international journal of operation & production management, 25(9), 898-916. doi: https://doi.org/10.1108/01443570510613956 rao, p. (2002). greening the supply chain : a new initiatives in south east asia. international journal of operations & production management, 22(6), 632 – 655. doi: https://doi.org/10.1108/01443570210427668 rinehart, l. m., eckert, j. a., handfield, r. b., page, t. j., & atkin, t. (2004). an assessment of supplier-customer relationships. journal of business logistics, 25(1), 2562. doi: https://doi.org/10.1002/j.2158-1592.2004.tb00169.x rocha, m., searcy, c., &karapetrovic, s. (2007). integrating sustainable development into existing management systems. total quality management & business excellence, 9(2), 33-41. doi: https://doi.org/10.1080/14783360601051594 ronchi, s., luzzini, d., & spina, g., (2007). supply chain coordination: the problem of risk and benefit sharing. supply chain forum an international journal, 8(2), 54-66. doi: https://doi.org/10.1080/16258312.2007.11517182 routroy, s. (2009). antecedents and drivers for green supply chain management implementation in manufacturing environment. icfai journal of supply chain management, 6(1), 20-35. saaty, r.w. (2003). decision making in complex environment: the analytic hierarchy process (ahp) for decision making and the analytic network process (anp) for decision making with dependence and feedback. pittsburgh, pa: super decisions. doi: https://doi.org/10.13033/ijahp.v9i2.496 saaty, t. l. (1990). how to make a decision: the analytic hierarchy process. european journal of operational research, 48(1), 9-26. https://doi.org/10.1111/jscm.12037 https://doi.org/10.1002/bse.435 https://doi.org/10.1080/09571264.2010.495853 https://doi.org/10.1080/14783360601051594 https://doi.org/10.1080/16258312.2007.11517182 https://doi.org/10.13033/ijahp.v9i2.496 ijahp article: kumar/buyer-supplier relationship selection for a sustainable supply: a case of the indian automobile industry international journal of the analytic hierarchy process 225 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.605 sahay, b. s. (2003). supply chain collaboration: the key to value creation. work study, 52(2), 76-83. doi: https://doi.org/10.1108/00438020310462872 salam, m. a. (2008). an empirical investigation of the determinants of adoption of green procurement for successful green supply chain management. ieee conference publication of 4th ieee international conference ieee management of innovation and technology, 1038-1043. doi: https://doi.org/10.1109/icmit.2008.4654511 sarkis, j. (1995). supply chain management and environmentally conscious design and manufacturing. international journal of environmentally conscious design and manufacturing, 4(2), 43-52. doi: https://doi.org/10.1109/ecodim.2001.992305 sarkis, j., gonzalez-torre, p., & adenso-diaz, b. (2010). stakeholder pressure and the adoption of environmental practices: the mediating effect of training. journal of operation management, 28(2), 163–176. doi: https://doi.org/10.1016/j.jom.2009.10.001 senge, p. & prokesch, s. (2010). the sustainable supply chain. harvard business review, 88(10), 70-72. seuring, s. & muller, m. (2008b). from a literature review to a conceptual framework for sustainable supply chain management”, journal of cleaner production, 16(15), 16991710. doi: https://doi.org/10.1016/j.jclepro.2008.04.020 sharma, a., iyer, g. r., mehrotra, a., &krishanan, r. (2010). sustainability and business to business marketing: a framework and implications. industrial marketing management, 39(2), 330-341. doi: https://doi.org/10.1016/j.indmarman.2008.11.005 shaw, k., shankar, r., yadav, s. s., & thakur, l. s. (2012). supplier selection using fuzzy ahp and fuzzy multi-objective linear programming for developing low carbon supply chain. expert systems with applications. 39, 8182-8192. doi: https://doi.org/10.1016/j.eswa.2012.01.149 simpson, d. f. & power, d. j. (2005). use the supply relationship to develop lean and green suppliers. supply chain management: an international journal, 10(1), 60-68. doi: https://doi.org/10.1108/01443570710714529 simpson, d., power, d., & samson, d. (2007). greening the automotive supply chain: a relationship perspective. international journal of operations & production management, 27(1), 28-48. doi: https://doi.org/10.1108/01443570710714529 smith, b. g. (2007). developing sustainable food supply chains. philosophical transactions of the royal. society b: biological sciences, 363(1492), 849–861. doi: https://doi.org/10.1098/rstb.2007.2187 sridharan, r., & simatupang, t. m. (2013). power and trust in supply chain collaboration. international journal of value chain management, 7(1), 76-96. doi: https://doi.org/10.1504/ijvcm.2013.057344 https://doi.org/10.1108/00438020310462872 https://doi.org/10.1109/ecodim.2001.992305 https://doi.org/10.1016/j.jclepro.2008.04.020 https://doi.org/10.1108/01443570710714529 ijahp article: kumar/buyer-supplier relationship selection for a sustainable supply: a case of the indian automobile industry international journal of the analytic hierarchy process 226 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.605 svensson, g. (2007). aspects of sustainable supply chain management (sscm): conceptual framework and empirical example. supply chain management: an international journal, 12(4), 262-266. doi: https://doi.org/10.1108/13598540710759781 svensson, g. (2009). the transparency of scm ethics: conceptual framework and empirical illustrations. supply chain management: an international journal, 14(4), 259269. doi: https://doi.org/10.1108/13598540710759781 teuteberg, f. & wittstruck, d. (2010). a systematic review of sustainable supply chain management research: what is there and what is missing? proceedings of mkwi corporate environmental and sustainability management conference held at göttingen at the georg-august-university, 1001-1015. tsoulfas, g. t. & pappis, c. p. (2006). environmental principles applicable to supply chain design and operation. journal of cleaner production, 14(18), 1593–1602. doi: https://doi.org/10.1016/j.jclepro.2005.05.021 vachon, s. & klassen, r. d. (2006). extending green practices across supply chain: impact of upstream and downstream integration. journal of operation and production management, 26(7), 795-821. doi: https://doi.org/10.1108/01443570610672248 vachon, s. & klassen, r.d., (2008). environmental management and manufacturing performance: the role of collaboration in supply chain. international journal of production economics, 111(2), 299-315. doi: https://doi.org/10.1016/j.ijpe.2006.11.030 vachon, s. (2007). green supply chain practices and the selection of environmental technologies. international journal of production research, 45(18-19), 4357-4379. doi: https://doi.org/10.1080/00207540701440303 vermeulen, w. j. v. & ras, p. (2006). the challenges of greening global product chains: meeting both ends, sustainable development, 14(4), 245-156. doi: https://doi.org/10.1002/sd.270 vermeulen, w. j. v. & seuring, s. (2009). sustainability through the market the impacts of sustainable supply chain management: introduction. sustainable development, 17, 269–273. doi: https://doi.org/10.1002/sd.422 walker, h., sisto, l. d., and mcbain, d. (2008). drivers and barriers to environmental supply chain management: lesson from the public and private sectors. journal of purchasing & supply management, 14(1), 69-85. doi: https://doi.org/10.1016/j.pursup.2008.01.007 walton, s. v., handfield, r. b., & melnyk, s. a. (1998). the green supply chain: integrating suppliers in to environmental management process. international journal of purchasing and material management, 34(2), 2-11. doi: https://doi.org/10.1111/j.1745493x.1998.tb00042.x https://doi.org/10.1016/j.jclepro.2005.05.021 https://doi.org/10.1108/01443570610672248 https://doi.org/10.1002/sd.270 https://doi.org/10.1111/j.1745-493x.1998.tb00042.x https://doi.org/10.1111/j.1745-493x.1998.tb00042.x ijahp article: kumar/buyer-supplier relationship selection for a sustainable supply: a case of the indian automobile industry international journal of the analytic hierarchy process 227 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.605 wu, z. &pagell, m. (2011). balancing priorities-decision-making in sustainable supply chain management. journal of operations management, 29(6), 577-590. doi: https://doi.org/10.1016/j.jom.2010.10.001 ytterhus, b. e. (1999). environmental initiatives in the retailing sector: an analysis of supply chain pressure and partnership. eco-management and auditing, 6(4), 181-188. doi: https://doi.org/10.1002/(sici)1099-0925(199912)6:4%3c181::aidema115%3e3.3.co;2-t yu, c. s. (2002). a gp-ahp method for solving group decision-making fuzzy ahp problems. computers and operations research, 29(14), 1969-2001. doi: https://doi.org/10.1016/s0305-0548(01)00068-5 zadeh, l. a. (1965). fuzzy sets. information and control, 8, 338–353. zhu, q. h. & sarkis, j. (2004). relationship between operational practices and performance among early adopter of green supply chain management practices in chinese manufacturing enterprises. journal of operation management, 22(3), 265–289. doi: https://doi.org/10.1016/j.jom.2004.01.005 zhu, q. h. & sarkis, j. (2007). the moderating effect of institutional pressure on emergent green supply chain practices. international journal of production research, 45(18-19), 4333-4355. doi: https://doi.org/10.1080/00207540701440345 zhu, q., sarkis, j., & lai, k. h., (2008a). green supply chain management implications for “closing the loop". transportation research, 44(1), 1-18. doi: https://doi.org/10.1016/j.tre.2006.06.003 zsidisin, g. a. & hendrick, t. e. (1998). purchasing’s involvement in environmental issues: a multi-country perspective. industrial management & data systems, 98(7), 313320. doi: https://doi.org/10.1108/02635579810241773 zutshi, a. & sohal, a. s. (2004). adoption and maintenance of environmental management systems-critical success factors. management of environmental quality: an international journal, 15(4), 399-419. doi: https://doi.org/10.1108/14777830410540144 https://doi.org/10.1016/j.jom.2010.10.001 ijahp news and events: announcing the 26 th international conference on multicriteria decision making international journal of the analytic hierarchy process 552 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.953 announcing the 26 th international conference on multicriteria decision making we are very pleased to announce that the 26th international conference on multiple criteria decision making will be held at the university of portsmouth from june 26 july 1, 2022. there will be an ahp/anp track co-chaired by enrique mu and rozann saaty which will be sponsored, as in previous years, by the creative decisions foundation. this is an opportunity for ahp/anp scholars to submit their research. abstract submissions (with a maximum of 600 characters) will be opened in early december with a submission deadline of march 1, 2022. important dates event dates: june 26july 1, 2022 call for invited session proposals: october 1, 2021 submission deadline for invited session proposals: november 30, 2021 call for abstracts: december 2, 2021 submission deadline for abstracts: march 1, 2022 notification of acceptance: april 30, 2022 early bird registration deadline: may 30, 2022 organizing committee professor alessio ishizaka (chair) dr. banu lokman (co-chair) professor dylan jones professor ashraf labib professor salvatore greco dr. negar akbari dr. maria barbati dr. salem chakh aerial view of portsmouth and portsmouth harbor source: by photo: la(phot) paul a'barrow./mod, ogl v1.0, https://commons.wikimedia.org/w/index.php?curid=26896810 https://mcdm2021.org/ https://mcdm2021.org/ https://mcdm2021.org/committees/ ijahp: kahraman/special topic articles on fuzzy ahp international journal of the analytic hierarchy process 144 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.600 special topic articles on fuzzy ahp in this issue, three papers on the theoretical development and/or practical usage of fuzzy ahp are presented. the first paper, warehouse risk assessment using interval valued intuitionistic fuzzy ahp by selcuk cebi and esra ilbahar is on the selection of a maintenance strategy involving the prioritization of critical factors. it presents an ahp application based on four maintenance strategies, namely corrective maintenance, preventive maintenance, predictive maintenance, and proactive maintenance, and the decision criteria, which are cost, quality, safety, value added and viability. the contribution of the paper is that fuzzy ahp may be a useful tool to solve the consistency problems in classical ahp applications. the second paper, selection of industrial maintenance strategy: classical ahp and fuzzy ahp applications by robison ohta, valerio a. p. salomon and messias borgia silva, tries to evaluate and categorize the risks in warehouses in terms of occupational health and safety. it provides a new perspective to warehouse risk assessment. an intervalvalued intuitionistic fuzzy ahp is proposed for evaluating and categorizing the risks in warehouses. the third paper, intuitionistic fuzzy originated interval type-2 fuzzy ahp: an application to damless hydroelectric power plants by cengiz kahraman, başar öztayşi, sezi çevik onar and onur doğan, develops an intuitionistic fuzzy originated interval type-2 fuzzy ahp method and applies it to the technology selection problem of a damless hydroelectric power plant. linguistic evaluations and its corresponding triangular intuitionistic fuzzy scale are used in this multi criteria damless technology selection problem. i hope this section will provide a useful resource of ideas, techniques, and methods for research on fuzzy ahp. i am grateful to the referees whose valuable and highly appreciated works contributed to the selection of the papers published in this section. i would like to thank the editor-in-chief, enrique mu, and associate editor, birsen karpak for their supportive roles in this process. cengiz kahraman, guest editor ckahraman department of industrial engineering, istanbul technical university, 34367 besiktas, istanbul, turkey ijahp article: news/editor-in-chief enrique mu uses ahp to help city of pittsburgh move to the cloud international journal of the analytic hierarchy process 88 vol. 3 issue 1 2011 issn 1936-6744   ahp news editor-in-chief enrique mu uses ahp to help city of pittsburgh move to the cloud professor enrique mu, editor-in-chief of the international journal of the ahp (on the right), honored by mayor luke ravenstahl for helping the city of pittsburgh select the best technology to move its information technology functions to the cloud, a tough decision that had to be made by city officials. professor mu used ahp (the analytic hierarchy process for decision-making) to guide the process, facilitating it over a period of months. cloud computing provides resources using a computer network as opposed to reliance on an individual computer’s operating system, software or data. when using “the cloud” the individual computer serves only as a means of displaying information processed on a network of computers far away. the city of pittsburgh required a cloud computing internet technology solution. due to the high cost, complexity and public nature of this requirement it was imperative that the decision-making process be accurate, transparent and open to the views of different city information system stakeholders. for assistance with this high-priority decision, the city of pittsburgh turned to dr. enrique mu, ph.d., carlow university associate professor and specialist in ahp-based decisionmaking and information technology management. the group participating in the decision process consisted of approximately 12 ijahp article: news/editor-in-chief enrique mu uses ahp to help city of pittsburgh move to the cloud international journal of the analytic hierarchy process 89 vol. 3 issue 1 2011 issn 1936-6744   specialists in finance, technical issues and administration from the city information systems department. the ahp model was a combination of the requirements that had been developed by the city for the system and judgments input by the participants with the participants giving judgments in the parts of the hierarchy in which they were expert. each such meeting to collect judgments was finished in about one session. in his recognition of professor mu, mayor ravenstahl said he had made an invaluable analytical contribution in the help he gave the city in the quest to find a suitable cloud computing solution, and he went on to say the success by the decision-making team led to the solution of a real and urgent city problem.   rob typewritten text http://dx.doi.org/10.13033/ijahp.v3i1.92 ijahp: kahraman/special topic articles on fuzzy ahp international journal of the analytic hierarchy process 312 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.628 special topic articles on fuzzy ahp: part-2 in the special topic articles of this issue, three papers on the theoretical development and/or practical usage of fuzzy ahp are presented. the first paper, solar pv power plant location selection using a z-fuzzy number based ahp by irem otay and cengiz kahraman is on the selection of a location for a solar energy pv plant using a four-level hierarchy. the authors consider several main and subcriteria including initial cost, maintenance cost, slope, and distance to highways. the study solves the multi-criteria solar pv power plant location selection problem with a z-fuzzy based ahp method. a z-fuzzy number is a relatively new concept in the fuzzy set theory that considers the reliability assigned to an ordinary fuzzy number and enables one to circumvent the limitations of ordinary fuzzy numbers. the second paper, interval-valued neutrosophic ahp with possibility degree method by eda bolturk and cengiz kahraman is on the development of an interval-valued neutrosophic ahp method by employing the possibility degree method for ranking interval numbers in this method. pairwise comparison matrices are filled in by using linguistic terms and the relative importance degrees of criteria are obtained. an application of the proposed method is given in the area of energy planning. comparative and sensitivity analyses are also presented in the paper. the third paper, prioritization of alternatives based on analytic hierarchy process using interval type-2 fuzzy sets by konstantin yury degtiarev and mikhail yury borisov presents a model assigning weights to the constraints based on expert assessments. an interval type-2 membership functions based fuzzy ahp is presented. the main objective of the paper is to develop an approach which would make it possible to integrate an expert's confidence degree into fahp model. i hope this section will provide a useful resource of ideas, techniques and methods for research on fuzzy ahp. i am grateful to the referees whose valuable and highly appreciated work contributed to the selection of the papers published in this section. i would like to thank the editor-in-chief, enrique mu and associate editor, birsen karpak for their supportive roles in this process. cengiz kahraman guest editor department of industrial engineering istanbul technical university, 34367 besiktas, istanbul, turkey ijahp news and events: petrillo/ world events international journal of the analytic hierarchy process 404 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.906 world events ahp/anp as multidimensional tools ________________________________________________________________________ mas 2021 international conference on modeling and applied simulation krakow, poland september 15-17, 2021 a flexible conference: in person & virtual the 2021 international conference on modeling and applied simulation (mas) will be a platform for knowledge exchange, the review and discussion of theoretical advances, research results, and industrial experiences, among scientists, researchers, decision makers, practitioners and students dealing with topics under the umbrella of multidisciplinary modeling and simulation. figure 1 mas 2021 conference during mas 2021, a session titled “decision support systems applications” organized by fabio de felice (university of cassino and southern lazio italy) and antonella petrillo (università degli studi di napoli “parthenope” italy) will take place. the session will offer the opportunity to present results and critical issues concerning decision support systems, the theory underlying such systems and its applications. the aim is to promote a yearly forum for researchers and practitioners to exchange new ideas and practical experience in the areas of decision support systems, in order to provide a chance for growing in the field of research. topics of interest include:  multi-criteria decision making (ahp method, electre method, etc.)  machine learning ijahp news and events: petrillo/ world events international journal of the analytic hierarchy process 405 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.906  modelling techniques of decision support systems  industrial and engineering applications of decision support systems  decision support systems in e-commerce  group decision support systems  expert systems  soft computing techniques  datawarehousing and datamining  project management techniques and operational research  executive information systems  knowledge-based decision support systems  strategic decision support systems for more info please see: http://www.msc-les.org/mas2021/ ________________________________________________________________________ informs 2021 annual meeting in anaheim, ca october 24-27, 2021 a flexible conference: in person & virtual the 2021 informs annual meeting is a unique opportunity to connect and network with the more than 5,000 informs members, students, prospective employers and employees, and academic and industry experts who compose the informs community. this year it will be possible to attend in-person in anaheim, ca, or participate virtually via online meeting platform. figure 2 informs 2021 annual meeting during informs, a session titled: “theory and applications of ahp/anp in supply chain management” organized by birsen karpak (youngstown state university – http://www.msc-les.org/mas2021/ ijahp news and events: petrillo/ world events international journal of the analytic hierarchy process 406 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.906 usa) and antonella petrillo (università degli studi di napoli “parthenope” italy) will take place. the event will be an opportunity to discuss current issues, including the following topics: model-based approaches for interoperability of next generation supply chain management systems: state of the art and future challenges: contributions from ahp/anp gregory zacharewicz; imt mines ales, france. theory and applications of ahp/anp in supply chain management yousaf ali khan; ghulam ishaq khan institute of engineering sciences & technology, pakistan. selection of sustainable suppliers using an integrated mcdm approach: a case study of turkey sezin güleryüz; bartin university, turkey. analytic hierarchy process/analytic network process applications in big data enabled supply chain management birsen karpak 1 , ilker topcu 2 , fusun ulengin 3 1 youngstown state university, usa 2 istanbul technical university, turkey 3 sabanci university, turkey selection of a 3pl service provider for an aerospace company: analytic network process (anp) approach ozden bayazit 1 , birsen karpak 2 1 central washington university, usa 2 youngstown state university, usa for more information please see: http://meetings2.informs.org/wordpress/anaheim2021/ http://meetings2.informs.org/wordpress/anaheim2021/ ijahp: ferretti/highlights: international symposium on the analytic hierarchy process 2014 international journal of the analytic hierarchy process 252 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.309 highlights: international symposium on the analytic hierarchy process 2014 valentina ferretti this summer washington d.c. hosted a lively and sparkling international symposium on the analytic hierarchy process. isahp 2014 took place between june 29 and july 2 at the very heart of washington’s elegant and fascinating downtown. the general theme of the symposium was “beyond decision making”, which was a reference to the idea that it is the time for ahp/anp to go beyond the toolbox of multi-criteria decision-making experts and become a tool used by everyday decision makers. after all, this broad use was the original intention of ahp/anp creator, professor thomas l. saaty. he envisioned a methodology that would help the world be more rational when making decisions. this year the event attracted the largest number of participants ever! more than 200 paper proposals were submitted, and about 230 participants took part in the symposium, coming from more than 40 countries (the largest number ever!). within this truly international environment, approximately one third of the presenters were students. indeed, the organizers made a special effort to involve master’s-level students, the next generation of ahp/anp advocates, in this symposium. this was the first time that mba students were targeted for participation in the symposium, and several panels were dedicated to this group of students. the meeting felt like an ahp family reunion, and there was a festive mood in the air. this was also evident by the success of the gala dinner river cruise where the largest number of participants ever registered, and participants brought their friends and family with them. we reached the capacity of the glass boat reserved for the evening. ijahp: ferretti/highlights: international symposium on the analytic hierarchy process 2014 international journal of the analytic hierarchy process 253 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.309 indonesia poland us nigeria iran romania algeria turkey pakistan canada ukraine uk spain israel saudi arabia nepal brazil croatia colombia india italy russia mexico hungary chile taiwan czech republic australia china poland cuba venezuela japan korea germany austria egypt palestine france greece figure 1. countries who participated in the isahp 2014 this symposium was an exceptional opportunity to bring together researchers, teachers, students and users of ahp/anp to share their research and experiences in decision making, by fostering a collage of interactions, catching up, discovery, debate, and enjoyment. all the organized sessions had the merit of providing very interesting applications in many different fields, ranging from conflict resolution to supply chain management, thus stimulating debate on further research methods to make better decisions and supplying important cues for future developments. moreover, this symposium also attracted the largest number of full paper submissions for publication opportunities for the 3 main sponsoring journals: the international journal of production economics, the international journal of the analytic hierarchy process, and the international journal of management and decision making. the paper proposals covered the major areas of research in ahp/anp, and provided solutions for current challenges in important areas of decision making. if you were not able to attend the symposium, you can learn about all of its interesting sessions at following this link: http://www.isahp.org/ finally, a warm congratulation to the following award-winning paper proposals: best application paper proposal  “using alignment with corporate strategy for the selection of a project portfolio based on anp” by mónica garcía-melóna, rocío poveda-bautistaa and josé l. del valle m. (academic award) http://www.isahp.org/ ijahp: ferretti/highlights: international symposium on the analytic hierarchy process 2014 international journal of the analytic hierarchy process 254 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.309  “using anp to design a living system like balanced operating model for intangible services” by angela minzoni and eléonore mounoud (academic award) best session student paper proposal  “best plant location for a manufacturing company: an analytic hierarchy process approach” by sayantoni dey, stephen bosela, and jakub waksmundzki from youngstown state university, u.s. (best session proposal award in its session)  “carlow-uganda initiative ii: an ahp bocr decision analysis” by bethany dorney, elizabeth kallenborn, douglas morgan, shannon robertson from carlow university, pittsburgh, pa, u.s. (best session proposal award in its session) the next isahp will take place in 2016, make sure you don’t miss it! professor thomas saaty with dr. birsen karpak (to his left) together with youngstown state university mba students who participated in the isahp 2014 ijahp: ferretti/highlights: international symposium on the analytic hierarchy process 2014 international journal of the analytic hierarchy process 255 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.309 a toast during the gala dinner the organizing committee (from left to right: rozann saaty, birsen karpak, tom saaty, antonella petrillo and enrique mu) valentina ferretti department of regional and urban studies and planning politecnico di torino italy e-mail: valentina.ferretti@polito.it ijahp: ferretti/informs 2014 highlights international journal of the analytic hierarchy process 146 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.330 informs 2014 highlights valentina ferretti the informs 2014 conference took place in sunny and welcoming san francisco, one of the most fascinating cities in beautiful california. this conference was a huge success with more than 5000 participants and many interesting events, including the 50 th anniversary of decision analysis! the conference theme, "bridging data and decisions," reflected the broad scope of what was covered, from data analytics to new optimization and analysis methodologies to models and practical applications pertinent to numerous industries and research fields. in addition, a very distinguished group of keynote and plenary speakers headlined the program. the four days of the conference (november 9-12) provided very interesting applications in many different fields, thus stimulating the debate on further research concerning methods to make better decisions and supplying important cues for future developments. in particular, several interesting studies dealing with the ahp and anp were presented. to name a few inspiring ones, magda gabriela sava, jerrold may and luis vargas proposed a study dealing with sensitivity analysis for analytic network models; daisuke amano and yasutaka kainuma suggested the development of a customer satisfaction function based on kano’s quality model and the ahp; halil cobuloglu and esra buyuktahtakin developed a stochastic ahp model for sustainable biomass crop selection; lin su, bing gou, xia wang and xin zheng evaluated the degree of city-production integration of shanghai high-tech industrial park by applying the fuzzy ahp; william fox proposed modeling and sensitivity analysis to madm overlay and used the ahp to find node influences in the noordin dark network; and fesseha gebremikael, iddrisu awudu and khalid bachkar developed a deterministic model using the ahp for agriculture feed processing supply chain modeling. all the presentations highlighted a wide tool box, extending the ahp into exciting new directions and relevant applications for business and society which bridge the gap between research and practice. in conclusion, this shows that the ahp community had strong presence at the informs annual meeting. let’s start thinking about contributing to the next meeting which will take place in philadelphia this autumn! valentina ferretti in point reyes, heading north from san francisco after the conference ijahp news and events: ferretti/highlights of the informs 2018 meeting international journal of the analytic hierarchy process 502 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.624 highlights of the informs 2018 meeting valentina ferretti department of architecture, built environment and construction engineering politecnico di milano v.ferretti@lse.ac.uk this year, the informs annual meeting took place in sunny phoenix from november 4 7. the theme of the meeting was smart city and sustainable communities, which represents a growing and stimulating domain of both research and application for operational research and management science. smart cities and sustainable communities indeed bring new and interesting challenges for or and ms, which translate into high relevance of the potential societal impact that both disciplines can have in the field. the topicality of the theme together with the welcoming atmosphere of phoenix’s conference venue allowed this annual meeting to achieve record attendance. the total number of attendees soared to 6,195, while the total number of paid attendees reached 5,885. both numbers were all-time records. the program featured 1,320 sessions spread over 79 tracks and four days. among these sessions, the ahp/anp community shared the latest developments of the techniques across different application domains. here are a few highlights from the ahp/anp community from the conference. the joint session mcdm/ practice on the theory and application of the anp chaired by orrin cooper featured the following talks: “coherency and reducing comparisons in the anp” by orrin cooper; “the added value of a team-based model in preventive healthcare services” by gabriela sava, luis vargas, jerrold may, jennifer shang, and james dolan; “analogical evaluation of the urban function combination mode of the integrated transportation hub based on ahp fuzzy comprehensive evaluation method” by siyu tao, feng tao, xinmei chen, anjun li, lisha wang. gabriela sava, luis vargas, and jerrold may also contributed a paper titled “multi-dimensional sensitivity analysis in operations research, and its ijahp news and events: ferretti/highlights of the informs 2018 meeting international journal of the analytic hierarchy process 503 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.624 importance for preventive healthcare services” in the joint session mcdm/practice: mcdm methodology in emerging applications, chaired by serpil sayin. the joint session mcdm/practice: mcdm in practice chaired by lorraine gardiner featured the following ahp/anp related talks: “blockchain's impact on digital supply chain: contributions from mcdm” by birsen karpak and valerio salomon, and “weighting criteria for agriculture planning: an application of the analytical hierarchy process” by jay parsons and kathleen brooks. jesse pietz presented a contribution titled “tradespace analytics for air force acquisition programs” in the session that he chaired on development, acquisition, and maintenance. gazi duman and elif kongar presented a paper titled “a balanced scorecard-based approach to integrate sustainability into the operational performance: a food industry case study” during the session on sustainability chaired by gulver karamemis. abdulhadi nasser al-marri, salem nechi, and omar ben-ayed presented a talk titled “factors affecting labor productivity in turnaround maintenance projects: case of the oil and gas sector in qatar” in the session titled product and project development and management 1 chaired by barbaros yet. finally, valentina ferretti presented a talk titled “cognitive biases in spatial decision analysis” in the session that she chaired on spatial decision analysis. the panorama offered by the ahp/anp related talks during this year’s informs meeting thus provided a stimulating balance between theoretical contributions and real applications, across a variety of domains ranging from healthcare services, transportation, food industry, supply chain, agriculture planning and labour productivity. let’s start planning for the next informs meeting which will take place in seattle next year! ijahp editorial: mu, e./words from the editor-in-chief international journal of the analytic hierarchy process 184 vol. 5 issue 2 2013 issn 1936-6744 december 2013 dear ijahp readers: welcome to this ijahp issue! our opening article by oddershede uses a mixed method approach involving ahp and montecarlo simulation to address electric grid impact assessment taking into account political, economic, societal and technological considerations. next, poveda develops and weights sustainable development indicators for surface mining operations and discusses how ahp can be used to address the different stakeholders needs in this area. in the next two articles we move from sustainability to application. bijan, keramati and salehi apply the analytic network process (anp) to the american customer satisfaction index (acsi) for e-commerce to develop a model for web user satisfaction comparison. sabiruddin, bhattacharya and das apply ahp to the selection of appropriate process parameters in gas metal arc welding. this article is particularly interesting because it uses ahp to assess experimental results in the presence of many different outcome variables. therefore, if you plan to work with experiments you may find this ahp application very interesting, even if you are not in the manufacturing field. we close our article section with a paper by tom saaty about the connection between the eigenvector as the solution for the priority vector of an ahp comparison matrix and perron’s theorem. more precisely, that it is not necessary to have perron’s theorem to prove that the eigenvector is the solution. in this issue, we have started the practice of reviewing recent ahp/anp books on the market. salomon, zinoviev, and zinovieva review ishizaka and nemery’s “multi-criteria decision analysis: methods and software.” we hope that our readers will welcome their colleagues’ book reviews, and you are encouraged to keep us abreast of new book publications in the ahp/anp field. finally, we welcome valentina ferretti, phd to the ijahp editorial team. ferretti will be in charge of the news and events section, covering news related to ahp/anp conference and events. this section provides conference highlights from two academic events in chile (thanks to dr. garuti and dr. salomon for the inside information), an announcement about a university-wide scholarship award obtained by this editor, and ijahp editorial: mu, e./words from the editor-in-chief international journal of the analytic hierarchy process 185 vol. 5 issue 2 2013 issn 1936-6744 most importantly an invitation to attend our upcoming isahp 2014 meeting (www.isahp.org) being held in washington, d .c. from june 29 – july 2, 2014. although this symposium is independent from our journal, rozann saaty, vice president of creative decisions foundation, has been instrumental in getting the ijahp editorial team to become the organizers of the upcoming isahp 2014. therefore, isahp 2014 attendees should be thankful to rozann saaty for making creative decisions foundation resources available to support the symposium. thanks should also be extended to birsen karpak and antonella petrillo whose hard work as program co-chairs is making this symposium possible. i would also like to recognize the work of elena rokou, who has been instrumental in sending out the isahp communications, and milagros pereyra, who as head of maestro meetings inc., has implemented the entire symposium infrastructure (e.g. website development, online submission system). the year ahead promises to be important and productive for the diffusion of ahp/anp at both the theoretical and practical level. enjoy this issue and see you at isahp 2014 in washington d. c. enrique mu, phd editor-in-chief, ijahp isahp 2014, chairman http://www.isahp.org/ rob typewritten text http://dx.doi.org/10.13033/ijahp.v5i2.214 ijahp essay: mu/reporting public multicriteria decision-making applications: a journal editor’s perspective international journal of the analytic hierarchy process 1 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1025 reporting public multicriteria decision-making applications: a journal editor’s perspective the following reflections refer to the development and reporting of public decisionmaking studies. these observations are provided with the hope of improving the quality and validity of multicriteria decision-making (mcdm) studies in the public sector. while my thoughts here are not intended to be a comprehensive study of how to report public mcdm studies, my intention is to highlight some key elements that are often not properly addressed by researchers. these observations are particularly relevant for ahp/anp applications due to their widespread use in public decision-making. the analytic hierarchy/network process (ahp/anp) is commonly used in practical applications in many different areas of decision-making. as a matter of fact, the ahp is one of, if not the most widely used multicriteria decision-making analysis methodologies worldwide. while the ahp calculations are rather simple to perform using a spreadsheet (mu & pereyra-rojas, 2018), there are currently many ahp software packages to facilitate the analysis (goepel, 2018; solutions, 2022; superdecisions, 2020). in situations where either ahp axioms (e.g., criteria independence) do not hold or the situation of mutual interaction (e.g., among the alternatives) arises, a generalization, the analytic network process, is used (saaty, 2004). to date, the only package, to my knowledge, that can help perform anp analysis is superdecisions (2020) developed by the creative decisions foundation (2022). this software also facilitates traditional ahp calculations. the success of ahp/anp in addressing societal issues is due mainly to its intuitive simplicity which allows public participation through the incorporation of multiple stakeholders, who may not need to be trained in the intricacies of mcda, to express their opinions in a way that can be quantitatively incorporated into the mcda process. public participation in the decision-making process requires a systematic “engagement of all the societal actors (research, industry, policy-makers, and civil society) and their joint participation in the research and innovation process” (strand et al., 2015). at the very least, and when public participation is not directly possible, a group of experts is conveyed to represent the various stakeholders’ interests. many published ahp/anp studies use one of these two approaches (gonzalez-urango, 2018; mu & stern, 2014). as an example, let’s consider a public decision among different urban developments. this decision will most likely require decision-making consultation and broader participation from the public. for this purpose, stakeholders will need to be identified and prioritized to ensure that the consideration of their perspectives and interests, as well as their participation, is proportional to their relative importance. there are several ways to engage stakeholders, and selecting the preferred method will be dictated by the specific context of the decision; however, the most important aspect, in this case, is that there is a proper systematic engagement of the stakeholders and that this is reported in the article. unfortunately, many analysts either do not perform a systematic stakeholder analysis or do not report it when submitting their manuscripts for publication (gonzalez-urango, mu, & garcía-melón, 2021). this lack of precision in the research report casts a shadow on the validity of the overall study. ijahp essay: mu/reporting public multicriteria decision-making applications: a journal editor’s perspective international journal of the analytic hierarchy process 2 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1025 in cases where broad participatory decision making is pursued, it is necessary to consult stakeholders using either a survey methodology, or a face-to-face, virtual, or hybrid group decision making approach. in the case of using a survey analysis, proper survey reporting is needed (e.g., response rate, sample of questions). survey research has a long tradition in science, and what needs to be reported and how it is reported is highly standardized. a description of survey development and reliability tests are part of what an editor would expect as reasonable elements to report for this approach (burns & kho, 2015). for this reason, analysts should pay careful attention to follow established practices when reporting mcda survey use. sometimes, during the mcda stage, handling stakeholders directly may be too cumbersome and an mcda team that represents the various stakeholders’ perspectives is used instead. this is what constitutes a group decision-making approach (saaty & peniwati, 2008). in this situation, the analysts are required to report group-decision methodological aspects such as how the group membership was decided (to ensure that specific members represent the needs of specific stakeholders); the group process followed (e.g., consensus, voting); and how the various perspectives were aggregated. in the ahp/anp, the aggregation of perspectives can take place during the individual comparison judgments or at the final priorities. many articles do not provide enough information to assess the group decision-making protocol, again causing doubt about the overall validity of the mcda study (mu & cooper, 2022). in conclusion, while authors must report their mcda analysis according to the established practices in the field, they must also be mindful of the methodological context (e.g., stakeholder engagement, group decision-making) and report according to best practices in the corresponding methodological approaches that complement their central mcda methodology (e.g., ahp/anp). not doing so casts a shadow over the validity of the mcdm study and jeopardizes the opportunities for publication. ijahp essay: mu/reporting public multicriteria decision-making applications: a journal editor’s perspective international journal of the analytic hierarchy process 3 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1025 references burns, k. e. a., & kho, m. e. (2015). how to assess a survey report: a guide for readers and peer reviewers. cmaj, 187(6), e198-e205. doi:10.1503/cmaj.140545 creative_decisions_foundation. (2022). changing the way we measure things. goepel, k. d. (2018). implementation of an online software tool for the analytic hierarchy process (ahp-os). international journal of the analytic hierarchy process, 10(3), 469-487. doi:https://doi.org/10.13033/ijahp.v10i3.590 gonzalez-urango, h. (2018). a combined social network analysis-analytic network process approach to evaluate sustainable tourist strategies. paper presented at the latin american studies association, barcelona, spain. gonzalez-urango, h., mu, e., & garcía-melón, m. (2021). stakeholder engagement and anp best research practices in sustainable territorrial and urban strategic planning. in m. doumpos, f. ferreira, & c. zopounidis (eds.), multiple criteria decision making for sustainable development. switzerland: springer nature. mu, e., & cooper, o. (2022). a contingency approach to multi-criteria decision-making: a search for validity through rigor and relevance. in i. topcu, s. o. ekici, o. kabak, e. aktas, & o. ozaydin (eds.), new perspectives in operations research and management science. london: springer. mu, e., & pereyra-rojas, m. (2018). practical decision making using super decisions v3: springer. mu, e., & stern, h. (2014). the city of pittsburgh goes to the cloud: a case study of cloud strategic selection and deployment. journal of information technology teaching cases, 4, 70-85. saaty, t. l. (2004). fundamentals of the analytic network process dependence and feedback in decision making with a single network. journal of systems science and systems engineering, 13(2), 129-157. doi: https://doi.org/10.1007/s11518-006-0158-y saaty, t. l., & peniwati, k. (2008). group decision making: drawing out and reconciling differences. pittsburgh, pa: rws publications. solutions, t. (2022). decision mentor. retrieved from https://www.decisionmentor.app/ strand, r., spaapen, j., bauer, m. w., hogan, e., revuelta, g., stagl, s., . . . guimarães pereira, a. (2015). indicators for promoting and monitoring responsible research and innovation report from the expert group on policy indicators for responsible research and innovation. superdecisions. (2020). super decisions. retrieved from https://www.superdecisions.com/ https://doi.org/10.13033/ijahp.v10i3.590 https://www.decisionmentor.app/ https://www.superdecisions.com/ ijahp essay: mu/book review: overcoming the retributive nature of the israeli-palestinian conflict international journal of the analytic hierarchy process 559 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.945 book review: overcoming the retributive nature of the israeli-palestinian conflict by thomas. l. saaty, h. j. zoffer, luis g. vargas, amos guiora enrique mu ijahp editor-in-chief thomas saaty continues to amaze us with publications appearing four years posthumously. a book written with his collaborators h. j. zoffer, luis g. vargas and amos guiora has been recently published by springer. there is a rich tradition of the use of the analytic hierarchy process (ahp) in conflict negotiation and resolution, beginning with the classic “conflict resolution: the analytic hierarchy approach” (saaty & alexander, 1989), in which the second half of the book was dedicated to retributive conflicts. these are conflicts in which each party is concerned not only with their own gains, but also with the pain they can inflict on the opposing side to compensate for perceived injustices. saaty and his collaborators have tackled other real conflicts in the past, such as the conflict in south africa, the free trade agreement negotiation between canada and the u.s., and the violent hostage crisis in peru in 1996-1997 (saaty, 1986; saaty, 1988; saaty & mu, 1997). however, no unresolved conflict can match the importance, scale, duration and complexity of the conflict between the palestinians and the israelis in the middle east, and this is the struggle addressed in this book. what i particularly liked about this book and what, in my opinion, makes it different from so many books written about this conflict is that firstly it provides a process, ijahp essay: mu/book review: overcoming the retributive nature of the israeli-palestinian conflict international journal of the analytic hierarchy process 560 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.945 based on the use of a mathematical decision-making method called the analytic hierarchy process (ahp), to address the israeli–palestinian conflict, which is perhaps the most intractable socio-political conflict of our time. secondly, the authors demonstrate that this process is a workable model because they used it to discuss resolving conflict with real israeli and palestinian negotiators who remained anonymous for political and safety reasons. they developed an objective tentative solution to the conflict, or more accurately, a set of tradeoffs on the many issues that would satisfy both parties. for this reason, the proposed results reflect the collective opinion of the negotiators who are subject matter experts and who have engaged in substantive international negotiations in the past. to our knowledge, no one has proposed a tangible objective plan to solve the conflict that includes the extent and level of detail provided by the actual stakeholders as has been done in this book. the book (chapters 1 and 2) begins by laying the groundwork from an historical context by discussing the origins and evolution of the middle east conflict as well as recapping the attempts that have been made to resolve it. the authors recognize that the parties have a different mental model about the conflict which in turn leads to each party having their own different truths (for example, “i was here first”) that they believe about the conflict. the authors make a wise observation by stating that there are not competing truths among the participants, but their perceived truths must be respected. at the request of the palestinian participants, the negotiation focused exclusively on the west bank with hamas and gaza being deliberately excluded from consideration. this provided a very good start because one of the most important aspects of strategic decision-making is the context, including the assumptions, of the decision. in chapters 3 and 4 the authors explain the ahp fundamentals, taking into account that some readers may be hearing about the ahp for the first time. the ahp introduction is followed by a description of what retributive conflicts are and how to address them using the ahp’s ability to quantify both tangible and intangible gain and loss factors for both parties. in conflicts like this, the intangible factors may have far greater importance than the tangible ones. in chapters 5 and 6, the authors show the hierarchy they used to equalize the trade-offs between the parties with calculations that include each party’s knowledge of their own benefits and costs, and what they perceived the benefits and costs would be to the other party. this is followed by a chapter explaining the lessons learned in the process and has a very ambitious title: “lessons learned: the ahp can help achieve peace.” judging from the results of the negotiations, the ahp certainly has the potential to do so, if given a chance. the remaining chapters (7 to 10) address very specific results of the negotiation and their implementation. one of the most important contributions is the ten pittsburgh principles for fair and equitable trade-offs discussed in chapter 7. these principles were developed from the negotiated trade-offs that served as guidelines for their creation. as an example, the first principle states the intention by the two parties to establish “a two-state solution based on the borders of the fourth of june, 1967, with mutually agreed upon land swaps.” (saaty, zoffer et al., 2021, p. 85). the last principle ijahp essay: mu/book review: overcoming the retributive nature of the israeli-palestinian conflict international journal of the analytic hierarchy process 561 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.945 states that “the full implementation of these principles concludes the end of the conflict and claims of the two parties” (p, 86). this is a direct quote from the book and means that upon implementation of these ten principles, both parties will agree that the conflict has ended. while the whole negotiation process and proposed solution to the israeli-palestinian conflict seems very promising, the book ends on a cautionary rather than optimistic note (ch. 10: looking ahead). the authors recognize that resolving the israelipalestinian conflict is now on the back burner of the israeli government due to the emergency created by the covid-19 pandemic and the sense of political power gained by netanyahu because of actions of the trump administration including the recognition of jerusalem as the capital of israel by the u.s., the u.s. and israel’s common concern about iran, and the lack of interest by the u.s. in pressing israel to address the conflict. whether the biden administration will change current policies, and to what extent, remains to be seen. the authors make a very important point that, in retrospect and in light of the recent violent confrontations between hamas and israel, ignoring the gaza strip and hamas in the negotiation process (because of the explicit request of the palestinian participants) was akin to ignoring the elephant in the room. the current split between hamas and the palestinian authority makes it very difficult to see a viable two-state solution. in other words, until this internal conflict among the palestinians is resolved, moving forward to create a palestinian state could be a non-starter. reviewing this book from different disciplinary perspectives (e.g. political science) would probably require a book in itself. the present review focuses on its multi-criteria decision-making (mcdm) aspects and, in particular, the use of the ahp methodology (saaty, 1980). this book will become a classic and the go-to reference handbook for negotiating retributive conflicts using the ahp approach. note: the book is nicely printed by springer and is pleasant to read. however, the details, such as the color maps and large tables in chapter 8, are better appreciated in the electronic version because of the reading zoom capability. the book is available for purchase at the springer link site https://link.springer.com/book/10.1007/978-3-03083958-1 https://link.springer.com/book/10.1007/978-3-030-83958-1 https://link.springer.com/book/10.1007/978-3-030-83958-1 ijahp essay: mu/book review: overcoming the retributive nature of the israeli-palestinian conflict international journal of the analytic hierarchy process 562 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.945 references saaty, t. l. (1980). decision making for leaders: the analytic hierarchy process. mcgraw-hill: new york. saaty, t. l. (1986). resolution of retributive conflicts. ifac proceedings, 19(8), 91-94. doi: https://doi.org/10.1016/b978-0-08-034915-2.50022-6 saaty, t. l. (1988). the negotiation and resolution of the conflict in south africa: the analytic hierarchy process. orion, 4(1), 3-25. doi: http://dx.doi.org/10.5784/4-1-488 saaty, t. l. and j. m. alexander (1989). conflict resolution: the analytic hierarchy approach. new york: praeger publishers. saaty, t. l. and mu, e. (1997). the peruvian hostage crisis of 1996-1997: what should the government do? socio-economic planning sciences, 31(3), 165-172. saaty, t. l., zoffer, h. j., vargas, l. g. and guiora, a. (2021). overcoming the retributive nature of the israeli-palestinian conflict. switzerland: springer. doi: https://doi.org/10.1007/978-3-030-83958-1 ijahp essay: mu/an mcdm reflection on the fifa 2014 world cup golden ball award international journal of the analytic hierarchy process issn 1936-6744 124 vol. 6 issue 2 2014 http://dx.doi.org/10.13033/ijahp.v6i1.241 essay an mcdm reflection on the fifa 2014 world cup golden ball award enrique mu, phd -------------------------------------------------------------------------------------------------- another world cup is over and with that another controversy about the rightful winner of the most prestigious award of the tournament, the golden ball. the former argentinian player and current celebrity diego maradona was most likely right when he argued that his fellow countryman lionel messi should not have accepted the golden ball (“balon de oro”) award (la nacion, 2014). there was even a viral video of a little girl who was crying because she thought that messi should have given the golden ball award to james rodriguez from colombia (youtube, 2014). from a multi-criteria decision making (mcdm) point of view, both maradona and the little girl were right and in the following article we will attempt to explain why. table 1 shows the statistics provided by fifa for the golden ball player candidates. we have taken the fifa numbers and averaged all the statistics with respect to a single game. for example, james rodriguez scored 6 goals in 5 games; therefore, his average goals/game score is 6/5 = 1.2. on the other hand, a player that did not score any goals, such as philipp lahm from germany, has 0/7 = 0 as his score for the criterion goals/game, as shown below. an analytic hierarchy approach the problem of selecting the golden ball award winner from among ten candidates constitutes a classic prioritization problem in the context of multi-criteria decision analysis. the analytic hierarchy process is an ideal methodology to model this decision (saaty, 1980). figure 1 shows the selection problem graphically. the scores from table 1 were converted into proportional ratings on a scale of 1 to 10. for example, the maximum score for goals/game is 1.2 (james); therefore, whoever has a score of 1.2 score has 10 points. the minimum score is 0 (lahm and mascherano); therefore, whoever has this score obtains 0 points. scores that fall between 0 and 1.2 are allocated points from 1 to 10 proportionally. this same procedure was applied to each of the 8 criteria in table 1. ijahp essay: mu/an mcdm reflection on the fifa 2014 world cup golden ball award international journal of the analytic hierarchy process issn 1936-6744 125 vol. 6 issue 2 2014 http://dx.doi.org/10.13033/ijahp.v6i1.241 table 1 fifa 2014 player game average statistics lionel messi toni kroos arjen robben mats hummels thomas muller philipp lahm james rodriguez neymar jr javier mascherano angel di maria country arg ger hol ger ger ger col bra arg arg games played 1 1 1 1 1 1 1 1 1 1 c1 goals scored 0.6 0.3 0.4 0.3 0.7 0.0 1.2 0.8 0.0 0.2 c2 assists 0.1 0.6 0.1 0.8 0.4 0.3 0.4 0.2 0.0 0.0 c3 crosses success ratio * 45.5 31.0 38.0 0.0 9.0 46.2 41.7 21.0 0.0 22.4 c4 passes complete ratio * 34.0 84.8 95.0 81.4 62.5 86.3 69.0 72.2 85.6 65.7 c5 recovered balls 3.7 2.9 2.7 7.8 1.7 5.6 2.0 3.2 7.0 1.8 c6 fouls committed 0.9 1.0 1.4 0.7 2.0 0.4 2.0 1.6 1.3 0.6 c7 yellow cards 0.0 0.0 0.1 0.0 0.0 0.1 0.2 0.2 0.1 0.2 c8 red cards 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 * % remains the same ijahp essay: mu/an mcdm reflection on the fifa 2014 world cup golden ball award international journal of the analytic hierarchy process issn 1936-6744 126 vol. 6 issue 2 2014 http://dx.doi.org/10.13033/ijahp.v6i1.241 figure 1. ahp ratings model for the selection of the golden ball (“balon de oro”) award winner the “balon de oro” evaluation finally, the eight criteria and the ten player candidates were entered into decision lens tm , a commercial software package that operationalizes an analytic hierarchy process ratings model, and we proceeded to consider different scenarios in which the criteria have different weights as follows: attack performance criteria 1 to 4 (goals, assists, crosses, and passes) can be considered player’s attack performance ratings. four scenarios are possible depending on which criteria combination (c1, c1 & c2, c1 – c3, c1 – c4) is considered the most important. figure 2 shows the results. notice that in all the cases, james rodriguez is the clear winner. the only exception is in the last scenario (4) where all the attack criteria were considered important. in this case rodriguez is still the best, but is tied with toni kroos who has a higher rate of completed passes. 1.2 (10 pts) 0.8 (10 pts) 46.2% (10 pts) 7.8 (10 pts) 0.4 (10 pts) 0.0 (10 pts) 0.0 (10 pts) 95% (10 pts) 34% (1 pt) 0.0 (1 pt) 0.0 (1 pt) 0.0 (1 pt) 1.7 (1 pt) 2.0 (1 pt) 0.2 (1 pt) c1 goals c2 assists c3 crosses c4 passes c5 balls recovered c6 fouls c7 yellow cards c8 red cards award winner selectio n ijahp essay: mu/an mcdm reflection on the fifa 2014 world cup golden ball award international journal of the analytic hierarchy process issn 1936-6744 127 vol. 6 issue 2 2014 http://dx.doi.org/10.13033/ijahp.v6i1.241 figure 2a. scenario 1: c1 – goals scored is most important criterion figure 2b. scenario 2: c1 goals and c2 assists are most important criteria figure 2c. scenario 3: c1 goals, c2 assists and c3 crosses are most important criteria ijahp essay: mu/an mcdm reflection on the fifa 2014 world cup golden ball award international journal of the analytic hierarchy process issn 1936-6744 128 vol. 6 issue 2 2014 http://dx.doi.org/10.13033/ijahp.v6i1.241 figure 2d. scenario 4: c1 goals, c2 assists, c3 – crosses, and c4 – recovered balls are most important criteria defense performance criteria 4 and 5 (passes completed and balls recovered) can be considered defense performance criteria (yes, c4 is double dipping since passes completed can be considered important performance for both attack and defense). figure 3 shows that in this case matts hummels (germany) is the clear winner, followed by mascherano (argentina). figure 3a. scenario 5: c5 – recovered balls is the most important criterion ijahp essay: mu/an mcdm reflection on the fifa 2014 world cup golden ball award international journal of the analytic hierarchy process issn 1936-6744 129 vol. 6 issue 2 2014 http://dx.doi.org/10.13033/ijahp.v6i1.241 figure 3b. scenario 6: c4 – passes completed and c5 – recovered balls are the most important criteria fair play performance finally, c6 to c8 (fouls, yellow cards, and red cards) can be considered a measure of fair play performance 1 . when the results were modeled based on c6 – c8 being the most important criteria, the winner is again matts hummels. figure 4. scenario 7: c6 – fouls, c7 – yellow and c8 – red cards are the most important criteria conclusion this analysis suggests that in any circumstance; that is, any combination of criteria, in which attack performance criteria have greater importance than defense and fair-play performance criteria, james rodriguez should have been the winner of the “balon de oro” award! although it is not possible to know how fifa weighs the different criteria, we could not find any scenario, i.e. changing criteria weights, where lionel messi could have been a potential winner for this award. furthermore, given the ahp’s capability to 1 in this case, the less number of fouls, yellow cards, and red cards average per game, the highest the player rating. ijahp essay: mu/an mcdm reflection on the fifa 2014 world cup golden ball award international journal of the analytic hierarchy process issn 1936-6744 130 vol. 6 issue 2 2014 http://dx.doi.org/10.13033/ijahp.v6i1.241 include intangible criteria such as team leadership, public charisma and so forth; we strongly suggest the use of an ahp evaluation approach that can make the award decision as transparent as possible. when half the world is watching this tournament, it is the least that fifa can do for soccer fans worldwide. ijahp essay: mu/an mcdm reflection on the fifa 2014 world cup golden ball award international journal of the analytic hierarchy process issn 1936-6744 131 vol. 6 issue 2 2014 http://dx.doi.org/10.13033/ijahp.v6i1.241 references el comercio (2014). en cifras: lionel messi merecio ganar el balon de oro? [in numbers: did lionel messi deserve the golden ball award?] july 16. retrieved from http://elcomercio.pe/deporte-total/brasil-2014/lionel-messi-balon-oro-cifras-estadisticasnoticia-1743512 la nacion (2014). diego maradona critico el balon de oro para messi. [diego maradona criticized the golden ball award given to messi]. july 14. retrieved from http://canchallena.lanacion.com.ar/1709807-diego-maradona-critico-el-balon-de-oropara-lionel-messi-quieren-hacerle-ganar-algo-que-no-gano saaty, t. l. (1980). decision-making for leaders. new york: mc-graw hill. youtube 2014). nina llora al ver que messi le “robo” el balon de oro a james. little girl cries because she argues that messi stole the golden ball from james. retrieved from https://www.youtube.com/watch?v=obiapiiwz2c http://elcomercio.pe/deporte-total/brasil-2014/lionel-messi-balon-oro-cifras-estadisticas-noticia-1743512 http://elcomercio.pe/deporte-total/brasil-2014/lionel-messi-balon-oro-cifras-estadisticas-noticia-1743512 http://canchallena.lanacion.com.ar/1709807-diego-maradona-critico-el-balon-de-oro-para-lionel-messi-quieren-hacerle-ganar-algo-que-no-gano http://canchallena.lanacion.com.ar/1709807-diego-maradona-critico-el-balon-de-oro-para-lionel-messi-quieren-hacerle-ganar-algo-que-no-gano https://www.youtube.com/watch?v=obiapiiwz2c a style guide for papers presented to the international journal of the analytic hierarchy process ijahp article: garuti, salomon / compatibility indices between priority vectors international journal of the analytic hierarchy process 152 vol. 4 issue 2 2011 issn 1936-6744 compatibility indices between priority vectors claudio garuti a. fulcrum engineering santiago, chile e-mail: claudiogaruti@fulcrum.cl valerio a. p. salomon∗ sao paulo state university guaratingueta, sp, brazil e-mail: salomon@feg.unesp.br abstract in the analytic hierarchy process priorities are derived from judgments. different priority vectors, however, can be obtained in the solution of a problem. the main objective of this article is to present two different compatibility indices between priority vectors that have been suggested. a comparison between the indices is presented, following a mixed qualitative-quantitative research approach. keywords: compatibility index, garuti compatibility index, saaty compatibility index, priority vector, analytic hierarchy process, multi-criteria decision theory 1. introduction as observed by saaty (2011), in the analytic hierarchy process (ahp), for intangibles, “priorities are derived from judgments in a special way. the process involves a composition of priority vectors given as the columns of a matrix according to certain rules”. saaty (1977) first established the consistency index, µ = (λmax – n) / (n –1), “as a measure of the consistency or reliability of information by an individual”. it was also noted that it is desirable to have µ near zero, to obtain consistency. if they were not close, the judgments may be revised and the consistency index may be improved. but, “improving consistency does not mean getting an answer closer to the real life solution”. it only means that judgments are closer to being logically related than randomly chosen. different priority vectors can be obtained in the solution of a multi-criteria decision making (mcdm) problem. as a matter of fact, different mcdm methods may yield different results when applied to the same problem (zanakis, et al. 1998). still, a single method application, such as ahp, can lead to different priorities. this can be a result of different individuals providing judgments or lapses in time when collecting judgments. ∗corresponding author rob typewritten text http://dx.doi.org/10.13033/ijahp.v4i2.130 ijahp article: garuti, salomon / compatibility indices between priority vectors international journal of the analytic hierarchy process 153 vol. 4 issue 2 2011 issn 1936-6744 that is, different priority vectors are often obtained in group decision making (gdm). this situation was previously studied mainly by focusing on the ranks that different priorities imply (emond and mason, 2002). the present article aims to extend the study from the ordinal focus to a cardinal one, where differences in the priorities are considered. even when vectors are not identical, they can sometimes be considered close to each other. according to saaty (2005), “when two vectors are close, we say they are compatible”. the saaty compatibility index, s, was the first developed measure of compatibility between priority vectors. this index uses the concept of the hadamard product, the element-wise product of two matrices. a different index has been proposed by garuti (2007), the garuti compatibility index, g, based on a physical interpretation of the inner product that includes the concept of vector projection. the main objective of this article is to present the two compatibility indices between priority vectors that have been proposed: g and s. a comparison of these indices is presented based on the hypothetical and classical examples of priority vectors obtained with the ahp. that is, we intend to achieve the objective with a non-exhaustive number of examples. this is a mixed qualitative-quantitative research approach (bryman and bell, 2007). compatibility indices are a new theme in ahp theory and practice, and this is the first article about them to be published in a journal. section 2 on theory presents the calculation procedures. section 3 presents some examples of the usage of g and s. section 4 ends this article with concluding remarks and possible uses for compatibility indices to be investigated in future works. 2. theory background the saaty compatibility index, s, between vectors x and y is obtained with equation 1, where n is the number of elements of the vectors, e is a column-matrix with all elements equal to 1,aij = xi/xj, bij = yi/yj, and ⦁ is the hadamard product operator. s = (1/n2)eta⦁bte (1) the calculation procedure of s is explained in saaty and peniwati (2007, p. 148) as: “given two sets of positive numbers, form the matrix of ratios of all the numbers in one set; then also form the matrix of ratios of all the numbers in the second set. take the transpose of the second matrix. multiply the two matrices element-wise (that is perform the hadamard product). add all the numbers and divide by n2.” one desirable property of a consistency index is that it should indicate that a vector is completely compatible with itself. for identical vectors, s = 1. it can be observed that this is true by substituting x = y, in the variables involved in creating equation 1: ijahp article: garuti, salomon / compatibility indices between priority vectors international journal of the analytic hierarchy process 154 vol. 4 issue 2 2011 issn 1936-6744 =                         •             = 1 ... 1 1 /...// ......... /...// /...// /...// ......... /...// /...// )1...11)(/1( 21 22221 11211 11 22212 12111 2 nnnn n n nnnn n n xxxxxx xxxxxx xxxxxx xxxxxx xxxxxx xxxxxx ns                         = 1 ... 1 1 1...11 ......... 1...11 1...11 )1...11)(/1( 2n = (1/n2) (1 + 1 + … + 1) = (1/n2) (n2) = 1 so when the two vectors are the same, s = 1 as shown above. if s ≤ 1.1 the two vectors are said to be compatible; otherwise, not (saaty and peniwati, 2007). table 1 presents three hypothetical priority vectors obtained with judgments provided by three different judges. corresponding elements of vectors 1 and 2 appear to be close to each other based on a cursory examination of the differences between them. elements of vectors1 and 3 do not even appear to be close. so, for vectors 1 and 2, s = 1.03, indicating that they are indeed compatible. this result was expected. for vectors 1 and 3, s = 3.13, indicating incompatibility between them. table 1 examples of compatible and incompatible vectors vector 1 vector 2 vector 3 element 1 0.50 0.52 0.10 element 2 0.40 0.41 0.60 element 3 0.10 0.07 0.30 table 2 presents two more priority vectors that appear, a priori, to be compatible. vectors 4 and 5 have elements that appear close to each other in terms of differences between them, but their compatibility index s = 1.63 indicates incompatibility. this result was not expected since the vectors appeared close to each other. ijahp article: garuti, salomon / compatibility indices between priority vectors international journal of the analytic hierarchy process 155 vol. 4 issue 2 2011 issn 1936-6744 table 2 another example of compatible vectors vector 4 vector 5 element 1 0.45 0.49 element 2 0.30 0.30 element 3 0.20 0.20 element 4 0.05 0.01 the garuti compatibility index, g, between vectors x and y is obtained with equation 2. this index is based on a physical interpretation of the inner product of two vectors, ‹x, y›, given by |x||y| cos α, where α is the angle between vectors x and y. for identical normalized vectors, α = 0 and ‹x, y› = 1. for perpendicular (orthogonal) vectors, α = 90o and ‹x, y› = 0. for identical normalized vectors, g = 1, meaning total compatibility. the minimum possible value is g = 0, indicating total incompatibility. ∑ =       + = n i ii ii ii yx yx yx g 1 2 )( ),max( ),min( (2) if g< 0.9, garuti (2007) proposes that x and y should be considered as not compatible. for vectors 1 and 2 from table 1, g = 0.95; for vectors 1 and 3, g = 0.46. this way, using 0.9 as the lower limit for compatibility, g indicates compatibility between vectors 1 and 2 and incompatibility between vectors 1 and 3. for vectors 4 and 5 (table 2), g = 0.94. that is, contrary to the s index for this case, g indicates compatibility between these vectors. it is possible that the small element of 0.01 in vector 5 may be the cause of the problem with the s index. it seems that g does not have the sensitivity to small element vectors that s does, i.e. g is not affected by small elements in the vector. the sensitivity of s to small elements is due to its calculation procedure. that is, in this procedure one element xi of a vector interacts with all elements from the vector, x. these interactions are the ratios that form the matrix a, aij = xi/xj. so, for instance, for vectors 4 and 5, we have a41 = 0.05/0.45 = 1/9 and b41 = 0.49/0.01 = 49. ijahp article: garuti, salomon / compatibility indices between priority vectors international journal of the analytic hierarchy process 156 vol. 4 issue 2 2011 issn 1936-6744 the multiplication of these elements give a14 b14 = 49/9≈ 5.444 which is relatively greater than 1. as the sum of all elements of a⦁b must be close to n2, if one element is much greater than 1, that forces s to be greater than the upper limit of 1. we also should note that these numbers represent priorities. therefore it should be expected that a small priority, like 0.01 or 0.05, may not have a great influence in the assessment of the index. the better behavior of g cannot be generalized based only on two hypothetical examples, so in section 3 we compute the g and s indices for two classical ahp validation exercises. 3. results table 3 is an estimate of the remoteness of cities from philadelphia; for each pair of cities the judgment that is entered is an estimate of how many times farther the more distant is from philadelphia than the nearer one (saaty, 1977). table 3 distance from philadelphia (adapted from saaty, 1977) cai tyo ord sfo lgw ymx eigenvector distance [miles] cairo (cai) 1 1/3 8 3 3 7 0.263 5,729 tokyo (tyo) 3 1 9 3 3 9 0.397 7,449 chicago (ord) 1/8 1/9 1 1/6 1/5 2 0.033 660 san francisco (sfo) 1/3 1/3 6 1 1/3 6 0.116 2,732 london (lgw) 1/3 1/3 5 3 1 6 0.164 3,658 montreal (ymx) 1/7 1/9 1/2 1/6 1/6 1 0.027 400 the saaty compatibility index between the eigenvector and the vector of distances is s = 1.02. this is an indication that these vectors are compatible. it is interesting to note that the elements of the eigenvector and distance vector in table 3 do not have the same unit. that is, elements from the eigenvector sum to 1, since the normalized eigenvector gives the relative priority of its elements, but the elements from the distance vector sum to 20,628 miles, the total distance from all the cities to philadelphia. as the computation procedure of s is based on the ratios between two elements, the vectors do not need to first be transformed by normalizing to have their compatibility measured by this index. in other words, if the elements of the eigenvector were multiplied by 20,628 miles, one will still obtain s = 1.02 for the index, the same as between the eigenvector and the vector of distances. the compatibility between the eigenvector and the vector of distances can be visually checked. that is, tokyo is the most remote city from philadelphia, followed by cairo, london, and san francisco, and so on, in that order. to use g, the vector of distances, [0.278, 0.361, 0.032, 0.132, 0.177, 0.019], must be normalized. then, the garuti compatibility index between the eigenvector and the normalized vector of distances will be ijahp article: garuti, salomon / compatibility indices between priority vectors international journal of the analytic hierarchy process 157 vol. 4 issue 2 2011 issn 1936-6744 g = 0.92 >0.9 thus, g also shows the two vectors are compatible. table 4 presents comparisons done by students in an electrical engineering class estimating the consumption of electricity of common household appliances (whitaker, 2007). table 4 relative electricity consumption of household appliances (adapted from whitaker, 2007) a b c d e f g eigenvector actual electric range (a) 1 2 5 8 7 9 9 .393 .392 refrigerator (b) 1/2 1 4 5 5 7 9 .261 .242 tv (c) 1/5 1/4 1 2 5 6 8 .131 .167 dishwasher (d) 1/8 1/5 1/2 1 4 9 9 .110 .120 iron (e) 1/7 1/5 1/5 1/4 1 5 9 .061 .047 radio (f) 1/9 1/7 1/6 1/9 1/5 1 5 .028 .028 hairdryer (g) 1/9 1/9 1/8 1/9 1/9 1/5 1 .016 .003 the saaty compatibility index between the eigenvector and the normalized vector of actual consumption is s = 1.46. this is an indication that these vectors are not compatible. the elements of these vectors have the same digit in the first decimal place, and the vectors have the same ordinal rank: [1, 2, 3, 4, 5, 6, 7], so because of this the vectors could be considered to be compatible with each other in spite of the value of s. again s seems not to be a good indicator when one of the vectors has a much smaller element, .003 for the hairdryer in the actual vector. the garuti compatibility index between the eigenvector and the normalized vector of actual consumption gives g = 0.92. thus the garuti compatibility index shows that the vectors are indeed compatible and it does not have any problem handling the small element in the vector. 4. conclusions this article presents two possible compatibility indices: g and s. the saaty compatibility index was developed first and uses the concept of the hadamard product. when two vectors are identical, s = 1. the threshold s ≤ 1.1 was established as an upper limit for compatible vectors. however, we presented two examples with s> 1.1 between two vectors that appeared to be relatively compatible. in the examples we have given, it seems that this compatibility index has a strong sensitivity to vectors with small elements. it is important to remember that those elements represent priorities. so, a small element corresponds to small priority and it should have a small influence over the index. ijahp article: garuti, salomon / compatibility indices between priority vectors international journal of the analytic hierarchy process 158 vol. 4 issue 2 2011 issn 1936-6744 the garuti compatibility index is also equal to 1 for identical priority vectors, and seems to better represent the influence of the element according to its size. that is, a small element will have a small influence over the index, while a big one will have a big influence. this appears to be an advantage of g when compared to s. as the calculation process of s only uses ratios between the elements, the vectors do not need to have the same unit. that advantage of s against g can be particularly useful when comparing results obtained with ahp to real values or values obtained with other mcdm methods. however, priority is a concept that does not require a dimensional meaning so the normalization process required by g should not present a problem. more generally, the normalization process cannot alter the cardinality of a well-defined set of cardinal priorities. the first theme for future research is a more in depth study of the threshold of 10% suggested for both indices of compatibility. perhaps for small element vectors s should have an incompatibility index limit higher than 1.1; however, initial studies show that s may present a trend to divergence in the presence of small numbers. as in g’s computational procedure the min/max ratio between elements is multiplied by their arithmetic mean, and values for this index can only vary from zero to one. thus, the image of the g function is the closed interval [0, 1] and the image of the s function is the left-bounded interval [1, +∞]. a mathematical simulation should be carried out in order to study if g < 0.9 can be tolerated for vectors with higher n, as was done for the consistency index. other interesting themes for research connect the index g with ahp/anp models. for instance, directly using the weights of the criteria obtained from the models to measure compatibility. one can also study the membership of an alternative for a given set, based on the compatibility (closeness) between the alternative and the set, for instance, by assessing patterns of behavior against some level of perturbation. the main practical use of compatibility indices may be for group decision making (gdm). the use of g and s can facilitate the effort to come to a consensus by quantifying and qualifying differences between priority vectors obtained from different experts. since priority vectors are often different for the members of the group in gdm, the use of the compatibility index can numerically express how far, or how incompatible, a priority vector provided by one person is from the aggregated priority vector. so the use of g and s may be considered a scientific way to provide consensus between conflicting parts. there are many possible fields of application for compatibility indices that include: diagnoses pattern recognition in medicine, psychiatry and psychology; measuring the degree of matching, or closeness, between buyers and sellers: does the offer match the customer’s need? how well does it match the need?; and conflict resolution. in gdm, compatibility indices can be used to measure the closeness between different value systems from the participants, assess the compatibility between different ijahp article: garuti, salomon / compatibility indices between priority vectors international journal of the analytic hierarchy process 159 vol. 4 issue 2 2011 issn 1936-6744 ways of thinking, clearly establish where they agree (or disagree) and by how much, making it easier to achieve a consensus. a compatibility index makes it possible to assess the closeness between complex profiles in weighted environments and answer the important question of when close really means close. ijahp article: garuti, salomon / compatibility indices between priority vectors international journal of the analytic hierarchy process 160 vol. 4 issue 2 2011 issn 1936-6744 references bryman, a.,& bell, e. (2007).business research methods. oxford: oxford university press. emond, e. j.,& mason, d. w. (2002) a new rank correlation coefficient with application to the consensus ranking problem. journal of multi-criteria decision analysis,11, 17–28. garuti a., c. (2007). measuring compatibility (closeness) in weighted environments. proceedings of the international symposium on the ahp, vina del mar, chile, august 2– 6. saaty, t. l., (1977). a scaling method for priorities in hierarchical structures. journal of mathematical psychology, 15, 234–281. saaty, t. l., (2005). theory and applications of the analytic network process: decision making with benefits, opportunities, costs, and risks. pittsburgh, pa: rws publications. saaty, t. l., (2011). aligning the measurement of tangibles with intangibles and not the converse. international journal of the analytic hierarchy process, 3(1), 79–87. saaty, t.l., & peniwati, k. (2007).group decision-making: drawing out and reconciling differences. pittsburgh, pa: rws publications. whitaker, r. (2007). validation examples of the analytic hierarchy process and analytic network process. mathematical and computer modelling, 46, 840–859. zanakis, s.h., solomon, a., wishart, n., & dublish, s. (1998) multi-attribute decision making: a simulation comparison of select methods. european journal of operational research, 107(3), 507-529. compatibility indices between priority vectors 1. introduction 2. theory background 3. results 4. conclusions ijahp article: news/three special issues planned for isahp2011 papers international journal of the analytic hierarchy process 90 vol. 3 issue 1 2011 issn 1936-6744   three special issues planned for isahp2011 papers the international symposium on the ahp (isahp2011) will take place in sorrento, italy, june 15-18, 2011. participants from more than 40 countries have registered to attend the meeting to be held at the hilton sorrento palace hotel in sorrento, italy. the best papers in the following areas will be selected for publication in three special issues of this journal. • ahp/anp: multicriteria decision making for sustainable development • ahp/anp: theory & methods • ahp/anp: multidisciplinary approaches & applications more information about the special issues will be emailed to the members of the isahp and ijahp after the meeting.   rob typewritten text http://dx.doi.org/10.13033/ijahp.v3i1.93 7 ed new nina decision ijahp article: saaty, t.l., zoffer, h.j. / nina’s decision: how to make better decisions and resolve conflicts international journal of the analytic hierarchy process 78 vol. 4 issue 1 2012 issn 1936-6744 nina’s decision: how to make better decisions and resolve conflicts an essay for the layman thomas l. saaty distinguished university professor university of pittsburgh saaty@katz.pitt.edu h.j. zoffer dean emeritus and professor of business administration university of pittsburgh zoffer@katz.pitt.edu decision making is a central activity of all people, usually done so automatically that we do not even realize that we are doing it every moment of every day of every year for all our lives. this silent and inarticulate approach worked well when humanity was fragmented and individuals and groups of people didn’t have to think of others very much. today however, the world has become expanded and interdependent and many of its resources are becoming scarce and valuable, including water, land, air, vegetation and minerals. now we have to consciously work together to choose important courses of action, and we have to justify these actions not only to ourselves but also to others, so we can live in harmony and with minimum conflict. nearly all of us have been brought up to believe that clear-headed logical thinking is our only sure way to face and solve problems. but experience suggests that logical thinking is not natural to us. indeed, we have to practice, and for a long time, before we can do it well. since complex problems usually have many related factors, traditional logical thinking leads to sequences of ideas so tangled that often the best solution cannot be easily discerned. our present complex environment calls for a new logic ─ a new way to cope with the myriad of factors that affect the achievement of goals and the consistency of the judgments we use to draw valid conclusions. this approach should be justifiable and appeal to our wisdom and good sense. it should not be so complex that only the educated can use it, but should serve as a unifying tool for thought in general. the lack of a coherent procedure to make decisions is especially troublesome when neither logic nor intuition are helpful in determining which of several options is the most desirable, or the least objectionable. since we are concerned with real-life problems, we must recognize the necessity for tradeoffs to best serve the common interest. to be really useful, ijahp article: saaty, t.l., zoffer, h.j. / nina’s decision: how to make better decisions and resolve conflicts international journal of the analytic hierarchy process 79 vol. 4 issue 1 2012 issn 1936-6744 this process should also assist in building consensus and reaching a compromise. we need a way to determine which objective outweighs another, both in the near and long terms. how can we do this? we describe and discuss a logical approach, the analytic hierarchy process (ahp), that can be used to make such decisions. it involves breaking a problem down into finer and finer parts, so that one is called upon to give a judgment comparing only a pair of issues in each judgment. this avoids mixing too many aspects of a problem, and not knowing what goes with what to obtain the final answer. however, it does call for one to structure the problem hierarchically with broad understanding of the people and their interests and of the issues involved. once one has the structure, it becomes easier to convey to others the influences driving a decision. all possible factors should be included in the structure; all sides involved in the discussion should be able to include whatever factors they feel appropriate, no matter how critical or unfriendly they may be. later, a process of prioritization by the different stakeholders will weed out the unimportant or irrelevant factors. in cases of extreme disagreement, the arguments of outliers can be given appropriate weightings and then combined with the positions of the majority. the ahp has appealed to managers and decision-makers at all levels of decision-making. it enables one to include both the strength of feelings needed to express judgments, and the logic and understanding relating to the issues involved in the decision. it combines the multiplicity of judgments in a systematic way to obtain the best outcome, or mix of actions to be taken. finally, and more significantly, these outcomes are derived in an agreeable way that are in harmony with our intuition and understanding and not forced on us by technical manipulations. we should be able to say that, given the information, we agree on the method of making the decision (if not the outcome of any particular decision). the matter becomes a common concern, not a mystical phenomenon. in summary, the process contributes to solving complex problems by structuring a hierarchy of criteria, stakeholders, and outcomes and by eliciting judgments to develop priorities. it also leads to prediction of likely outcomes according to these judgments. the outcome can be used to rank alternatives, allocate resources, conduct benefit/cost comparisons, exercise control in the system by evaluating the sensitivity of the outcome to changes in judgment, and carry out planning of projected and desired futures. a useful by-product of the process is a measurement of how well the people involved understand the relationships among the factors. although people generally are not consistent, the main concern here is the degree of their inconsistency. is their understanding close to capturing the interactions observed, or is it a random understanding that only hits the target now and then? good decisions must survive the difficulties and hazards of people and environment. we need to make decisions that are both desirable and survivable, rather than simply ones that we like best, without regard to how lasting they may be. predicting outcomes plays an important role in making such choices. to do this well, we can decompose a decision into separate structures involving scenarios of benefits, costs, opportunities and risks and then carefully combine their separate outcomes for the best decision. ijahp article: saaty, t.l., zoffer, h.j. / nina’s decision: how to make better decisions and resolve conflicts international journal of the analytic hierarchy process 80 vol. 4 issue 1 2012 issn 1936-6744 decision-making groups need to formalize their agenda and structure their interactions. group decision making needs a process that can unfold the complexity of the issues or problems. recognizing that perceptions and stakes can vary among group members, such a process should also specify how individuals can bargain on specific differences. finally, it would be desirable to have a measure of the consistency of judgments which individuals give and which the group settles on. the analytic hierarchy process described in this paper is such a process. one might ask: why is it that so many distinguished politicians and negotiators have so often failed to reach consensus on conflict after decades of trying? here are some possible reasons: 1. they had no way to measure the importance and value of intangible factors which can dominate the process. 2. they had no overall unifying structure to organize and prioritize issues and concessions. 3. they had no mechanism to trade off concessions by measuring their worth. 4. they had no way to capture each party’s perception of the other side’s benefits and costs. 5. they had no way to provide assurance to the other party that they, the opposing party, were not gaining more. 6. they had no way to avoid the effect of intense emotions and innuendoes which negatively affect the negotiation process. 7. they had no way to test the sensitivity and stability of the solution to changes in their judgments with respect to the importance of the factors that determined the best outcome. in the analytical hierarchy process, we address all these factors in an integrated framework. its main purpose is not only to include the measurement of tangible factors, but also to measure the myriad of intangibles which can seriously affect the ultimate outcome. the word intangible is most commonly used to describe things that are recognized but not easily quantified or measured. our purpose is to show that all intangibles can be measured in relative terms by using comparisons. a trivial example is that of choosing the best candidate for a husband. • nina is an attractive 27 year old mba graduate with three years of business experience. she is from a middle class family and wants to choose a husband from among three suitors. her criteria are age, looks, intelligence and economic status. these are her suitors. ijahp article: saaty, t.l., zoffer, h.j. / nina’s decision: how to make better decisions and resolve conflicts international journal of the analytic hierarchy process 81 vol. 4 issue 1 2012 issn 1936-6744 • peter is a rugged looking 30 year old engineer with a well-paying job, interested in his career and in raising a family. he is a hard-nosed, no nonsense kind of person, gentle and loving; • david is a 37 year old promising artist who is very romantic but whose career is still developing. his income from day to day is uncertain, but he is a sensitive, imaginative genius with a lot of promise. he is more interested in beauty and spirit than he is in accumulating wealth; • george is a 25 year old handsome, virile and fast moving young man with a brilliant future in a famous and successful family business. he is extremely generous and thoughtful, but he is also temperamental and assertive. figure 1 represents the structure of the problem. at the top is the goal (choosing the best husband for nina). at the second level of the hierarchy, below the goal are the criteria that nina has chosen to judge the possible husbands, and at third level of the hierarchy are the candidate husbands, the alternatives of the decision making process. the numbers in each cell represent the priority or importance of the criteria in table 1, and of the weighted priorities of the alternatives in table 3. choosing the best husband for nina goal age (0.062) looks (0.327) intelligence (0.429) riches (0.182) george (0.232) david (0.573) peter (0.195) figure 1best husband hierarchy nina has to determine the priorities of the four criteria and then judge the three suitors by comparing them on each criterion separately. finally, she must weight or multiply the priorities of the candidates by the importance or priorities of the criteria and add to determine the best candidate. we call these judgments a system of comparisons or better ijahp article: saaty, t.l., zoffer, h.j. / nina’s decision: how to make better decisions and resolve conflicts international journal of the analytic hierarchy process 82 vol. 4 issue 1 2012 issn 1936-6744 pairwise comparisons. in this decision process, nina is busily occupied making the judgments, and is unaffected by the overall impression that each individual candidate might have on her. this is because she is forced to determine priorities by comparing them and then rank the candidates on the basis of these comparisons. thus, the process drains away the bias that her emotions might otherwise cause. the pairwise comparison judgments of the criteria given by nina in this case are shown in table 1. the judgments are entered by comparing a criterion listed on the left of the table with another listed at the top. a criterion compared with itself is always assigned the value of one. the numbers 3, 5, 7, and 9 correspond to the verbal judgments of the comparisons of elements on the left over those at the top: “moderately more dominant”, “strongly more dominant”, “very strongly more dominant”, and “extremely more dominant” (with 2, 4, 6, and 8 for compromise between the previous values). reciprocal values (1/3, 1/5,1/7, 1/9) are automatically entered when the element on the left does not dominate but is dominated by the element at the top of the table. she must make a total of (4 × 3)/2 = 6 pairwise judgments. there are 16 positions in all with four “ones” for comparing the elements with themselves and half of the remaining 12 are the reciprocals of the others, so six judgments need to be entered. the priorities are obtained by raising the matrix to a large power to capture all the interactions, adding the entries in each row and dividing by the total sum of the rows. we are permitted to use decimal values between the integers, such as 2.6, if desired. it has been mathematically demonstrated that it is necessary to use this scale to get meaningful results in practice. it represents the normal range of human sensitivity to phenomena that are homogeneous. when things are widely scattered, they can be grouped into separate clusters with a common element in adjacent clusters, and the scale 1 to 9 is applied to compare the elements in each cluster with the common element serving as a link. when there are actual measurements for pairs being compared, such as money, we can use the ratio of their measurements. it is generally preferable to use the verbal judgment. with some software packages there are several equivalent modes: verbal, matrix mode in which numbers are entered in a matrix, the graphical model allowing graphical comparison of a pair, and the direct mode in which raw data can be entered for all elements in the comparison (not a pairwise comparison mode). judgments entered in any mode are reflected in the others. in table 1, when comparing age on the left with looks on top, she thinks that looks are very strongly more important, and the value 1/7 is entered in the (age, looks) position, and a 7 is automatically entered in the (looks, age) position. similarly, in comparing intelligence with looks, she thinks that intelligence is slightly more important than looks, and a 2 is entered in the (intelligence, looks) position, and a 1/2 is entered in the (looks, intelligence) position and so on. we always compare the criterion on the left as to how much more dominant it is than the criterion at the top. if it is not, the reciprocal value is used. in table 2 nina compares the candidates as to who is better and the strength of their superiority for each criterion. she does this first with respect to age, then looks, then intelligence and finally riches. ijahp article: saaty, t.l., zoffer, h.j. / nina’s decision: how to make better decisions and resolve conflicts international journal of the analytic hierarchy process 83 vol. 4 issue 1 2012 issn 1936-6744 table 1 comparing the criteria for importance with respect to the goal table 2 comparing the suitors with respect to the criteria in table 3 we multiply the weights of the suitors by the weights of the criteria and add to obtain the final ranking. table 3 synthesis of the priorities criteria age looks intelligence riches priority suitors (0.062) (0.327) (0.429) (0.182) synthesis peter .258 .166 .188 .237 .195 david .105 .740 .731 .064 .573 george .637 .094 .081 .699 .232 david wins because he has better looks and is more intelligent. both criteria have high priorities of 0.327 and 0.429 or a total of 0.756 of the assessment. thus, she should marry david the artist. this approach to prioritization provides the opportunity to help focus attention on the important issues and allocate resources to them accordingly. ijahp article: saaty, t.l., zoffer, h.j. / nina’s decision: how to make better decisions and resolve conflicts international journal of the analytic hierarchy process 84 vol. 4 issue 1 2012 issn 1936-6744 other applications and accomplishments of ahp include: • since its early development the ahp has been used to correctly predict, a few months before the election, the next candidate to be elected for president. the factors involved varied from election to election, depending on the domestic and international circumstances prevailing at the time. • in 1986, the institute of strategic studies in pretoria, a government-backed organization, used the ahp to analyze the conflict in south africa and recommended actions ranging from the release of nelson mandela to the removal of apartheid and the granting of full citizenship and equal rights to the black majority. all of these recommended actions were quickly implemented by the white government. • a company used the ahp in 1987 to choose the best type of platform to build to drill for oil in the north atlantic. a platform costs around 3 billion dollars to build, but the demolition cost was an even more significant factor in the decision. • xerox corporation has used the ahp to allocate close to a billion dollars to its research projects. • ibm used the process in 1991 in designing its successful mid-range as 400 computer. ibm won the prestigious malcolm baldrige award for excellence for that effort. the book about the as 400 project has a chapter devoted to how ahp was used in benchmarking. • the ahp has been used since 1992 in student admissions, and prior to that in military personnel promotions, and hiring decisions. • in 1995, the process was applied to the u.s. versus china conflict in the intellectual property rights battle over chinese individuals copying music, video, and software tapes and cd’s. an ahp analysis involving three hierarchies for benefits, costs, and risks showed that it was much better for the u.s. not to sanction china. shortly after the study was complete, the u.s. awarded china most-favored nation trading status, and did not sanction it. • in sports the ahp was used to predict which football team would go to the superbowl and win in 1995 (correct outcome, dallas won over our hometown, pittsburgh). the ahp was applied in baseball to analyze which padres players should be retained. • british airways used the process in 1998 to choose the entertainment system vendor for its entire fleet of airplanes ijahp article: saaty, t.l., zoffer, h.j. / nina’s decision: how to make better decisions and resolve conflicts international journal of the analytic hierarchy process 85 vol. 4 issue 1 2012 issn 1936-6744 • the ford motor company used the ahp in 1999 to establish priorities for criteria that improve customer satisfaction. • in 2001 it was used to determine the best site to relocate the earthquake devastated turkish city of adapazari. • a comprehensive analysis as to whether the unites states should develop an antinuclear missile (estimated in the 1990’s to cost $60 billion and strongly opposed by scientists as technically infeasible) was presented to the national defense university (ndu) in february 2002. in december of that year president bush decided to move forward, andthe us actually developed prototypes and successfully tested them in stages. • an application by professor wiktor adamus of krakow university convinced the prime minister of poland in 2007 not to adopt the euro for currency until many years later. • an ahp application, known to the military at the pentagon, showed that occupying or bombing iran is not the best option for security in the middle east in terms of benefits, opportunities, costs and risks. • the ahp was used to assist the green bay packers in hiring the best players, perhaps partly the reason why they won the superbowl in 2011 by beating the pittsburgh steelers. other sports, including hockey and baseball teams, are also using it. • ahp was used in three studies by economists to determine the turn-around dates of the us economy and the strength of recovery, in 1991, 2001 and 2009. these studies were uncannily accurate. • the ahp is used by many organizations, including the military, to prioritize their projects and allocate their resources optimally according to these priorities. • the latest application of the ahp, made in august 2011, was to the israelipalestinian conflict. five top participants from each side used the process to reach an agreement called the pittsburgh principles. one of them wrote: “i had been in hundreds of meetings between israelis and palestinians where we tried to reach a joint statement but failed because in most of the cases each side was trying to score points and court his own public opinion rather than being objective and trying to be real and responsible”. where there are people with different objectives that cannot coexist, there is potential for conflict; if individuals or groups attempt to satisfy only their own objectives conflict will ijahp article: saaty, t.l., zoffer, h.j. / nina’s decision: how to make better decisions and resolve conflicts international journal of the analytic hierarchy process 86 vol. 4 issue 1 2012 issn 1936-6744 occur. applications of the ahp in conflict resolution have been applied to a variety of conflicts. two of these conflicts, south africa and northern ireland, have been significant, and the ahp had an effect on their outcome. most discussions on conflict start with the premise that there will always be winners and losers in any situation where people have opposing desires. sometimes this is true; however, it is often possible to find a compromise that will work especially in the short run. in the long run, of course, it is usually necessary to remove the underlying source of the conflict, if that is possible. conflict resolution has frequently been defined as the search for an outcome that, at a minimum, represents for some participants an improvement from, and for no participants a worsening of, their present situation. if such outcomes exist, we want to find them, and then need see which of these outcomes is in some sense "best." to the olympian observer, this is obviously the way to go. however, in any conflict the participants will have conflicting objectives and desires, and the so-called best outcome will almost certainly fall short of each party's desired outcome.. how can we persuade each party to cease pursuing its own goals and to accept the compromise solution? the most distinctive attribute of humans is their ability to reason and analyze. it is particularly necessary that people in conflict should use reason, since there may be many interests at stake. to hold one’s ground without the use of reason is to inhibit progress. we need to introduce more reason and less intransigence into our methods of conflict management. in summary, ahp has been used all over the world in a variety of problem settings that would have otherwise been difficult to analyze. it is only now being applied to certain situations, such as the israeli-palestinian conflict, because of the complexity of that set of issues and the fact that it is an extremely retributive conflict (each side wants to inflict pain on the other side), and therefore lacks the advantage of most negotiations where both sides want the best solution without regard to inflicting any kind of pain on the other side. while a description of the process as noted above explains the technicalities of the initiative, only personal participation in an ahp exercise will highlight the advantage of this approach. rob typewritten text http://dx.doi.org/10.13033/ijahp.v4i1.139 ijahp ijahp news and events: vargas/ic4cr wins award from national collaborative on gun violence research international journal of the analytic hierarchy process 407 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.911 international center for conflict resolution (ic4cr) wins award from the national collaborative on gun violence research luis g. vargas professor business analytics and operations the joseph m. katz graduate school of business university of pittsburgh pittsburgh, pa 15260 lgvargas@pitt.edu the international center for conflict resolution (ic4cr) won a $30,000 grant award, given by the national collaborative on gun violence research (ncgvr), to develop a method to estimate the harms and benefits that gun policy and violence prevention interventions have on legal users of firearms. the international journal of the analytic hierarchy process (ijahp) is honored to publish the report titled “an analytic network model (anp) to estimate the benefits, opportunities, costs, and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms” authored by professors amos n. guiora, marcel c. minutolo and luis g. vargas, in this issue. furthermore, the authors are close to securing a contract to write a book about the implementation of their proposal. they are in the process of developing a questionnaire to acquire judgments from gun owners about 13 major laws. the international center for conflict resolution (ic4cr) was created by university of pittsburgh professors luis vargas (ijahp senior editor) and jerry zoffer (dean emeritus) and housed in the katz graduate school of business. the mission of ic4cr is to provide decision makers with an in-depth understanding of the negotiating positions of all parties and recommend implementation guidelines, based on preferences and priorities, to facilitate resolution of otherwise intractable conflicts. https://www.ncgvr.org/ https://business.pitt.edu/centers-and-initiatives/katz-international-center-for-conflict-resolution/ ijahp: mu/ijahp in 2018 international journal of the analytic hierarchy process 311 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.629 ijahp in 2018 dear ijahp reader: welcome to our third and last issue in 2018! this year has been marked by several important changes in our journal and the ahp/anp community at large. most important, this is the first year we are indexed by scopus, signaling a new maturity stage in our journal. this should make submissions to ijahp more appealing for the academic community at large. also, the international symposium of the analytic hierarchy process (isahp)’s focus theme this year was a remembrance of tom saaty and his legacy. many representative members of our ahp/anp worldwide community got together in hong kong to celebrate tom’s legacy and contribution to decision-making. it was an event to remember! several ahp/anp papers were presented at informs 2018 and our news editor, valentina ferretti, provides a thorough report on that in our current issue. this issue also contains four regular articles related to the ethical evaluation of care attendance in hospitals, the prioritization of cities in terms of liveability according to maqasid al-shariah indicators, a framework for local governments in japan to evaluate projects based on outcomes and using ahp to evaluate the performance of middle east airlines. this ijahp issue includes three additional articles related to our special topic on fuzzy ahp. i will let cengiz kahraman, guest editor for this special topic, introduce the papers in this section. the essay section discusses the implementation of an online software tool for ahp analysis by klaus goepel which will certainly be helpful for scholars and practitioners, as well as claudio garuti’s reflections on ahp topics that, although familiar, are still worthy of discussion. finally, the upcoming 25 th international conference on decision making (mcdm 2019) meeting in istanbul, turkey will have several ahp/anp presentations and panels and for this reason we have invited ilker topcu, general chair of the event, to announce these events in a news article. furthermore, the creative decisions foundation (cdf) has agreed to support a limited number of students, ph. d. candidates, post-docs, and young researchers to attend mcdm 2019 via pledging to donate their registration fees to allow them present their studies in the conference. there will also be an opportunity to submit their manuscripts for publication in our journal. enjoy the issue, happy holidays! enrique mu, phd ijahp, editor-in-chief ijahp essay: wallenius and wallenius/focusing on important problems international journal of the analytic hierarchy process 1 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.1066 focusing on important problems 1 hannele wallenius aalto university school of science aalto university po box 11000 fi-00076 aalto, finland jyrki wallenius aalto university school of business aalto university po box 11000 fi-00076 aalto, finland research in multiple criteria decision making (mcdm) and its subfields (multiattribute utility theory or decision analysis, ahp, goal programming, mcda, multi-objective mathematical programming, emo) is very active. however, our general feeling is that bright, young people do not focus their energies sufficiently on solving important problems despite many very important problems that deserve attention. some of the problems are critical to the well-being of the world. the purpose of our essay is to identify these very important problems (or mega trends, as they are sometimes called) and briefly discuss how the research community could help solve or at least alleviate such problems. the reader could interpret our essay as suggestions for a research agenda or program. it does not matter whether we are ahp scholars, multi-objective optimizers, or decision analysts, everyone’s contribution is needed. in 2015, the world economic forum published a report titled deep shift: technology tipping points and societal impact which discussed ongoing technology mega trends. the report, which classifies the technological mega trends into the following six categories, is still relevant today. 1. the internet – the world’s access to the internet will continue improving; people’s interaction with it will become more ubiquitous (at least in the western world). 2. enhancements – further enhancements in computing power, communications technologies, and data storage will take place. 3. the “internet of things” – this will be developed to a fuller extent. 4. big data and artificial intelligence – the ability to access and analyze huge amounts of data; coupled with the ‘ability’ of computers to make decisions based on this data. 5. sharing and platform economies – these will play an even larger role in the future than today. 6. 3d-printing – this will play a more important role in industrial processes than today. 1 our article is a much shorter and updated version of a paper which was recently published in intelligent decision support systems, (eds.) s. greco, v. mousseau, j. stefanowski, and c. zopounidis, springer, 2022. ijahp essay: wallenius and wallenius/focusing on important problems international journal of the analytic hierarchy process 2 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.1066 in addition to technological mega trends, the world is facing other very important “challenges” or even threats to humankind (non-technological). some of them are discussed by the world economic forum. 1. demographic and social change taking place in many countries and many continents (aging populations; decreasing fertility; urbanization; refugee problem) 2. the world population (in particular, in india and africa) continues to increase, causing a growing need for food, clean water, and cheap energy 3. climate change, concern for the environment 4. conflict resolution and disarmament 5. possible future pandemics and economic impacts of wars in our essay, we will discuss technology mega trends 1, 4, and 5 and non-technology mega trends 3, 4, and 5 from the above world economic forum’s lists in more detail. what role can decision analysis/multi-objective optimization, ahp, and preference modeling play to help alleviate the problems? what can we as an mcdm profession do? we argue that the mcdm profession is in an excellent position to help solve problems caused by world mega trends by developing tools and software and generating novel ideas. the world is increasingly driven by software and algorithms, and algorithm development is something we have been involved in for over half a century. the analytics skill set required in this type of work is very natural to mcdm professionals as is human-computer interface development. moreover, the very central concept (in our field) of nondominance (or efficiency or pareto-optimality) is at the core of many challenging decision problems. when resources are limited, it is imperative to understand tradeoffs. matching (of supply and demand and interests) is based on how well preferences or needs of both parties are modeled. it is also important to understand risks and conflict resolution. all of these skills are increasingly needed now and in the future – and mcdm professionals can help. internet searches and recommendations perhaps because of the pandemic, online shopping or multi-channel shopping is growing faster than originally expected. this growth has many implications. the activity of online shopping has generated many interesting research questions for strategy, marketing, and supply chain scholars to name a few. an observation concerning our field is that many consumers who buy online could benefit from some support when making purchasing decisions on the internet. such decision support must be targeted at the masses and must be simple. when people buy online, they often use search engines to help them find products or services. search engines allow users to specify what they want. typically, this is done via (hard) constraints related to the specific search; for example, an upper limit for the price of the product or service, location of the rental house, or the type of car (ride). search engines commonly ignore features other than these constraints and rank order the products/services based solely on price. mcdm offers tools that allow multiple factors (or attributes) to be incorporated. which tools are best suited for the purpose of internet searches is a subject that needs to be researched. for an attempt at improving search engines, see roy et al. (2008). ijahp essay: wallenius and wallenius/focusing on important problems international journal of the analytic hierarchy process 3 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.1066 to combat the information overload phenomenon, many companies (and academics) have developed, so called, recommender systems 2 . wallenius & wallenius (2022) reported the following: recommender systems are extensively used, for example, in choosing movies to watch, music to listen to, news to follow, books to read, and restaurants to visit. recommender systems are based on the “similarity” logic…the human desire for variety is not important in this logic. the assumption is that people lack curiosity and the desire to experiment. we think that the recommender systems should periodically suggest different types of products/services (or news), to broaden the person’s horizon. how to operationalize this idea, calls for research. (p. 63) big data (and artificial intelligence) data is continuously being generated from various sources including individuals (consumers) who constantly generate data by visiting the internet. many corporations are realizing that they should better utilize data to their (strategic) advantage. big data and ai are proving highly valuable in the following problem domains: 1. strategy development and management practices: schildt (2020) provides an in-depth look at how data and algorithms are reshaping management practices, organizational structures, corporate culture, and work roles. 2. advertising: the role of big data in advertising is growing in importance, allowing companies to target individuals and customer segments. 3. medicine and health care: big data will increasingly find uses in medicine and health care. healthcare decisions naturally involve multiple criteria, and complex tradeoffs between cost, the quality of care, even potential loss of life. artificial intelligence (ai) is currently an important field and will continue to develop in importance. however, ai is not only about hi-technology, but involves many complex legal and ethical issues that must be solved. interestingly, in 2019 one of the wallenberg foundations in sweden launched a program to fund research focusing on analyzing potential ethical, cultural, economic, labor market, socio-economic and legal aspects of the technology transition, including ai. we personally hesitate to delegate decision-making powers in important matters to ‘robots’ no matter how ‘intelligent’ they are. humans should be in control of important decisions that will impact their lives. platform and sharing economies concern for the environment is driving sharing and circular economies. a sharing economy needs online platforms where supply meets demand and where some type of matching based on preferences is sought. an example of an online marketplace is an online auction, which has been around since the late 1990’s. newer examples are the 2 we share the criticism voiced in the following quote from e-commerce relevance report 2022 (by coveo). “sky high expectations crushed by frustrating search and irrelevant recommendations.” ijahp essay: wallenius and wallenius/focusing on important problems international journal of the analytic hierarchy process 4 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.1066 san francisco-based taxi companies, uber and lyft, and the online market for housing, airbnb. does matching work optimally in markets like these? in the late 1990’s, we worked with the platform economy concept and developed a multi-attribute reverse (or procurement) auction site, called negotiauction (teich et al., 2001). we realized that price-only auctions were too simplistic and that auctions needed to include other aspects as well, such as quality and terms of delivery. our negotiauction system was based on ‘pricing out’ (or costing out) all other attributes besides cost (see keeney and raiffa, 1976). many commercial multi-attribute auction sites exist today (pham et al., 2015). climate change, concern for environment the concern for the environment is almost universal. environmental studies naturally require the decision-makers to consider multiple criteria and the complex tradeoffs between them (hobbs & meier, 2003). these types of decisions offer many application opportunities for mcdm professionals. from the mcdm’s perspective, an under-researched topic is sustainable (or green or ethical) investing. markowitz (1952)’s original portfolio optimization model had two objectives, maximizing expected returns and minimizing risk (or volatility). now, many investors (including institutional investors) also want to consider how companies perform in terms of a third dimension of sustainability (or environmental, social and corporate governance, for short esg, criteria). however, measuring sustainability is far from a trivial task. sustainability is multi-dimensional, and measuring it requires research in which our community should help. interesting early work has been reported in hallerbach et al. (2004). conflict resolution and disarmament this year, conflict resolution and disarmament has become as important as it was in the 1960’s due to the war in ukraine (see saaty, 1968; saaty et al., 2022). saaty’s ideas from 1968 are as relevant today as ever. in fact, saaty’s work at the state department and in the geneva negotiations with the soviets concerning nuclear arms reduction served as a catalyst for his development of the ahp method (saaty, 1980). saaty et al. (2022) is based on the contributions of the ahp in conflict resolution, in particular the israeli-palestine conflict. more research and real-world implementations are needed. possible future pandemics and impacts of wars the recent covid-19 pandemic and the war in ukraine have taught the world the importance of risk analysis and the necessity of being prepared for future crises. safety-stocks of important (critical) materials need to be large enough for a country (the world) to survive, but this involves a tradeoff with cost. the mcdm field offers tools to gauge risks and decide the appropriate levels of critical materials storage. due to disruptions in foreign supply chains (a case in point is the energy shortages prevalent in europe), each country needs to carefully consider the possibility of improving its self-sufficiency in critical areas. this calls for difficult tradeoff and risk analyses. big data offers tools to combat disinformation campaigns, especially prevalent in times of war. ijahp essay: wallenius and wallenius/focusing on important problems international journal of the analytic hierarchy process 5 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.1066 conclusion the future of mcdm looks bright. many of the world’s mega trends reinforce the role of mcdm by pointing out novel application areas. the mcdm community needs to seize these opportunities. the world is getting more complex and despite the relative affluence of the world, resource constraints still prevail. because of resource constraints, we cannot achieve everything we want, and tradeoffs must be made. the concepts of efficiency (or pareto optimality) and tradeoff are at the core of mcdm. this is a summary of our suggestions in areas where we see great potential for novel contributions of mcdm: 1. develop better recommender systems and search engines. 2. develop better matching algorithms for various situations. 3. promote the use of big data in companies and the public sector. 4. develop better measures (indices) for sustainable investing. 5. develop decision support tools targeted at the masses to help consumers make online purchasing more efficient. 6. produce theoretical and applied work on conflict resolution. 7. work on risk analysis, and improving the preparedness of organizations, countries, and regions for eventual crises. ijahp essay: wallenius and wallenius/focusing on important problems international journal of the analytic hierarchy process 6 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.1066 references brechbuhl, h. world economic forum. (2015). https://www3.weforum.org/docs/wef_gac15_technological_tipping_points_repo rt_2015.pdf ecommerce relevance report by coveo. (2022). https://www.coveo.com/en/resources/reports/relevance-report-ecommerce hallerbach, w., ning, h., soppe, a., and spronk, j. (2004). a framework for managing a portfolio of socially responsible investments. european journal of operational research, 153(2), 517-529. doi: https://doi.org/10.1016/s03772217(03)00172-3 hobbs, b. and p. meier (2003). energy decisions and the environment: a guide to the use of multi-criteria methods. boston: kluwer academic. keeney, r. and h. raiffa (1976). decisions with multiple objectives: preferences and value tradeoffs. new york: wiley. markowitz, h. (1952). portfolio selection, journal of finance, 7(1), 77-91. doi: https://doi.org/10.1111/j.1540-6261.1952.tb01525.x pham, l., teich, j., wallenius, h., and wallenius, j. (2015). multi-attribute online reverse auctions: recent research trends. european journal of operational research, 242(1), 1-9. doi: https://doi.org/10.1016/j.ejor.2014.08.043 roy, a., mackin, p., wallenius, j., corner, j., keith, m., schymik, g., and h. arora (2008). an interactive search method based on user preferences. decision analysis, 5(4), 203-22. doi: https://doi.org/10.1287/deca.1080.0125 saaty, t. l. (1968). mathematical models of arms control and disarmament: application of mathematical structures in politics. john wiley & sons. saaty, t.l. (1980). the analytic hierarchy process: planning, priority setting, resource allocation. new york: mcgraw-hill. saaty, t.l., zoffer, h. j., vargas, l.g., and guiora, a. (2022). overcoming the retributive nature of the israeli-palestinian conflict. switzerland: springer nature. doi: https://doi.org/10.1007/978-3-030-83958-1 schildt, h. (2020). the data imperative: how digitalization is reshaping management, organizing and work. oxford: oxford university press. doi: https://doi.org/10.1093/oso/9780198840817.001.0001 teich, j., wallenius, h., wallenius, j. and a. zaitsev (2001). designing electronic auctions: an internet-based hybrid procedure combining aspects of negotiations and auctions. electronic commerce research, 1, 301-314. doi: https://doi.org/10.1109/dexa.2000.875150 https://www3.weforum.org/docs/wef_gac15_technological_tipping_points_report_2015.pdf https://www3.weforum.org/docs/wef_gac15_technological_tipping_points_report_2015.pdf https://www.coveo.com/en/resources/reports/relevance-report-ecommerce http://www.sciencedirect.com/science/article/pii/s0377221714007292 http://www.sciencedirect.com/science/article/pii/s0377221714007292 http://www.sciencedirect.com/science/article/pii/s0377221714007292 http://www.sciencedirect.com/science/article/pii/s0377221714007292 javascript:void(0) ijahp essay: wallenius and wallenius/focusing on important problems international journal of the analytic hierarchy process 7 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.1066 wallenius, h. and wallenius, j. (2022). how can decision sciences and mcdm help solve challenging world problems. in s. greco, v. mousseau, j. stefanowski, and c. zopounidis (eds), intelligent decision support systems (pp 59-71). springer. doi: https://doi.org/10.1007/978-3-030-96318-7_4 ijahp: mu/ tom saaty’s spirit is alive in his personal archive international journal of the analytic hierarchy process 410 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.954 tom saaty’s spirit is alive in his personal archive while preparing this ijahp issue for publication, i received an email from rozann saaty, tom’s widow and collaborator, to inform me that the hundreds of papers and notes from tom’s lifetime of work had been organized and were ready to be moved to their final home at the university of pittsburgh, where tom worked the last part of his life. rozann’s news and her invitation to show me tom’s private collection of papers was irresistible, and i quickly went to her home to admire tom’s collection of papers and mementos that were neatly arranged in a big room in the house. what was particularly interesting was the thoroughness of tom’s notes. at a time when handwriting was the common form of taking notes and putting down ideas, and fountain pens were the standard writing tool, his notes were very neatly made and only had a few scratches and corrections. also, it was possible to follow the development of his ideas, through the different notes and papers that led to the creation of his magnum opus, the analytic hierarchy process (ahp), and later its generalization, the analytic network process (anp). i had the fortune of having an office almost across the hall from tom’s at the university of pittsburgh and his door was always open and he was always eager to engage in intellectual conversations. he usually carried pieces of paper with some notes in his pocket. he would use these notes when we would talk about diverse topics such as the major crises the world would face or the lamb menu for the day at the middle eastern restaurant across from our building. it was obvious that he kept these notes with great care. finally, it was also nice to remember tom, the man, through an exploration of his life as a boy and young man through the pictures of himself and his family. tom was born in mosul, iraq, where he grew up before coming to the u.s. to study at different high-level institutions until he obtained his phd in mathematics from yale university in 1953. after graduation, he worked for fifteen years for the u.s. government where he realized there was a need for a new decision-making methodology that would be both so easy and rigorous that “my grandmother in iraq could understand and my yale thesis collection of tom’s papers tom on the isahp2007 poster (his creativity book is visible at bottom right) more of tom’s papers ijahp: mu/ tom saaty’s spirit is alive in his personal archive international journal of the analytic hierarchy process 411 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.954 advisor would accept it as mathematically valid” (in his own words, according to my recollection). in 1969, he joined the faculty of the wharton business school’s graduate groups of operations research and social systems sciences at the university of pennsylvania where he developed the ahp/anp methodology in the 1970s. after ten years there, he moved to the university of pittsburgh, joseph m. katz graduate school of business, where he held the chair of distinguished university professor at the time of his death in 2017. i was fortunate to meet him in 1996 while i was a visiting lecturer at the katz graduate school of business and he recruited me into his work and vision for better decision-making in the world. family photos. bottom center, tom saaty as a young man i feel indebted to rozann saaty, president of the creative decisions foundation and a key member of our ijahp editorial team for the opportunity to peek into this treasure trove of tom’s documents. as a result of this visit, i feel more energized to continue tom’s work to improve decision making in the world. enrique mu and rozann saaty at the end of the visit ijahp: mu/ tom saaty’s spirit is alive in his personal archive international journal of the analytic hierarchy process 412 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.954 speaking of the world, this final issue for 2021 includes research articles from different regions of the world and covers a wide variety of topics ranging from the sustainable development of the gem industry in brazil to financial decisions made by young adults in north macedonia, among many others. also, this issue includes a report on the ahp/anp track presentations at informs 2021 by track chairs birsen karpak and antonella petrillo and the announcement of the 26 th mcdm conference at the university of portsmouth, uk, from june 26 to july 1, 2022 . in addition, there is a book review of tom’s posthumous publication overcoming the retributive nature of the israelipalestinian conflict written with his collaborators h. j. zoffer, luis g. vargas and amos guiora. finally, this issue concludes with an essay by hannia gonzalez-urango from the polytechnic university of valencia, spain, about how the analytic hierarchy/network process supports a more responsible and committed research and innovation. tom’s spirit is quite alive in the work of ahp disciples! enrique mu ijahp editor-in-chief 6_lipovetsky_eigenvectors_dec_23_pp_171-178 ijahp article: lipovetsky/ priority eigenvectors in analytic hierarchy/network processes with outer dependence between alternatives and criteria international journal of the analytic hierarchy process 172 vol. 3 issue 2 2011 issn 1936-6744 priority eigenvectors in analytic hierarchy/network processes with outer dependence between alternatives and criteria stan lipovetsky gfk custom research north america minneapolis, mn, usa e-mail: stan.lipovetsky@gfk.com abstract this work continues consideration of the relations between analytic hierarchy and network processes. it shows that in the case of a simple network with outer dependence between alternatives and criteria, the priority vectors can be constructed not only via the powered supermatrix but also by the eigenvectors of the supermatrix. the relationship to the ahp least squares approach and other methods of priority estimation are considered as well. an anp matrix of local eigenvectors includes priorities for all the compared items in the whole network. here we interpret the complex ahp/anp connections and show clearly how they result in priority estimations useful for applied decision making. keywords: analytic hierarchy/network process, outer dependence, supermatrix’ eigenvectors. “you can choose where to spend eternity” thomas saaty, the thinking man’s new millennium joke book 1. introduction this paper continues the consideration of the relationship between the analytic hierarchy process and the analytic network process started in two previous papers in this journal (lipovetsky, 2010, 2011). in the current article we consider a simple network with outer dependence between the alternatives compared with respect to the criteria on the one hand and the criteria compared with respect to each of the alternatives on the other hand. we can show that in this case the priority vectors can be constructed not only via the powered supermatrix approach but also by determining the eigenvectors of the anp matrix. the relations to the ahp least squares approach and other methods of priority estimation are considered. it is noted that this problem can be related to the system of the so called geary-khamis equations known in international statistics for comparing currencies and volumes of various goods produced by different countries (geary 1958, khamis 1972, kravis et al. 1975, rao and selvanathan 1992). a similar approach has also been used for finding preferences evaluated both by ranks and rates in marketing research (lipovetsky 2007). rob typewritten text rob typewritten text http://dx.doi.org/10.13033/ijahp.v3i2.123 ijahp article: lipovetsky/ priority eigenvectors in analytic hierarchy/network processes with outer dependence between alternatives and criteria international journal of the analytic hierarchy process 173 vol. 3 issue 2 2011 issn 1936-6744 starting with the anp matrix of local eigenvectors, the eigenvectors of the matrix itself give the priorities for all the compared items in the whole network. 2. anp solution and interpretation let us consider the example of the simple network with outer dependence between alternatives compared with respect to the criteria on one hand, and the criteria compared for each of the alternatives on the other hand. for an explicit illustration, we will use a network system with feedback given in saaty’s book (1994) where various water levels in a dam (low, middle, or high) are evaluated for best serving the purposes of flood control, recreation, or power generation. table 1 shows the local eigenvectors for the three water levels compared with respect to each of the purposes of the dam in the lower left block, and for the purposes compared with respect to each of the levels in the upper right block. the levels might be considered to be the alternatives in this example as the model is for choosing the best one, and the purposes of the dam are the criteria. table 1 local priority vectors for “management of water reservoir” (saaty, 1994) compared characteristics flood control recreation electric power low dam middle dam high dam overall priority flood control 0 0 0 .637 .200 .060 .241 recreation 0 0 0 .258 .600 .231 .374 electric power 0 0 0 .105 .200 .709 .385 low dam .722 .072 .058 0 0 0 .223 middle dam .205 .649 .207 0 0 0 .372 high dam .073 .279 .735 0 0 0 .405 the last column at the right in table 1 presents the overall priority vector of the purposes of the dam, and the levels obtained using the standard anp process of raising the matrix of local eigenvectors to powers until it converges. that is, the elements of the overall priority vectors obtained by summing each row and normalizing the results are sufficiently close for two successive powers of the matrix. note that the local eigenvectors, the criteria part of the overall vector and the alternatives part of the overall vector are normalized to one; that is, the totals equal one in every case. consider such an outer dependent model for the general case of m alternatives and n criteria. let the local eigenvectors of alternatives and of criteria be stacked into two matrices a and b, of the order m by n, and n by m, respectively. the supermatrix consists of the non-diagonal blocks a (vectors of the alternatives under each criterion) and b (vectors of the criteria compared for each alternative), and two diagonal zero-blocks of nth and m-th orders, as shown in saaty (1994): ⎟⎟ ⎠ ⎞ ⎜⎜ ⎝ ⎛ 0 0 a b (1) ijahp article: lipovetsky/ priority eigenvectors in analytic hierarchy/network processes with outer dependence between alternatives and criteria international journal of the analytic hierarchy process 174 vol. 3 issue 2 2011 issn 1936-6744 instead of raising the matrix to powers to find the overall priority vector, the same solution can be obtained by solving the eigenproblem for the combined vector )( ʹ′βα of criteria and alternatives overall priorities: ⎟⎟ ⎠ ⎞ ⎜⎜ ⎝ ⎛ =⎟⎟ ⎠ ⎞ ⎜⎜ ⎝ ⎛ ⎟⎟ ⎠ ⎞ ⎜⎜ ⎝ ⎛ β α λ β α 0 0 a b (2) multiplying the supermatrix in (1) by the equation (2) yields the following eigenproblem for the squared supermatrix: ⎟⎟ ⎠ ⎞ ⎜⎜ ⎝ ⎛ =⎟⎟ ⎠ ⎞ ⎜⎜ ⎝ ⎛ ⎟⎟ ⎠ ⎞ ⎜⎜ ⎝ ⎛ β α λ β α 2 0 0 ab ba (3) where ba and ab are square blocks of n-th and m-th order, respectively. in the supermatrices in both (2) and (3) each column equals one, so they are columnstochastic matrices. the main eigenvectors in (2) have eigenvalues with an absolute value of one. the eigenproblems (2) and (3) produce essentially the same main eigenvectors as becomes clear after the overall priorities for the criteria and alternatives are normalized to one. the supermatrix in table 1 gives the main solutions of the eigenproblem (2) and (3) shown in table 2. table 2 local and global priorities obtained as the main eigenvectors for the supermatrix in the “management of water reservoir” example the original eigenvectors (2) and (3) in table 2 seem to be rather different, but after normalizing to one each segment of the vector separately for the alternatives and criteria, the results obtained from table 2 coincide with the overall priority shown in the table 1. let us consider how to interpret the eigenproblem solutions for equations (2) and (3). the matrix equation in (2) can be represented by the two equations: λβαλαβ == ab , (4) compared characteristics supermatrix eigenproblem (2) supermatrix eigenproblem (3) original eigenvectors each part normalized original eigenvectors each part normalized vector 1 vector 2 vector 1 vector 2 vector 1 vector 2 vector 1 vector 2 flood control .393 .385 .241 .241 0 .497 .241 recreation .611 .598 .374 .374 0 .773 .374 electric power .628 .615 .385 .385 0 .794 .385 low dam -.364 .357 .223 .223 .476 0 .223 middle dam -.607 .595 .372 .372 .794 0 .372 high dam -.661 .647 .405 .405 .864 0 .405 eigenvalue λ -1 1 1 1 ijahp article: lipovetsky/ priority eigenvectors in analytic hierarchy/network processes with outer dependence between alternatives and criteria international journal of the analytic hierarchy process 175 vol. 3 issue 2 2011 issn 1936-6744 substituting one of these equations into another yields two separate eigenproblems: βλβαλα 22 , == abba (5) which can be expressed by the combined eigenproblem in (3). all the matrices in equations (4) and (5) are column-stochastic, so the absolute value of all the main eigenvalues equals one. such kinds of equations had been also considered in relation to bayes theorem and independence effects in a network in saaty (1994), and for the multidimensional matrices eigenproblem in lipovetsky and tishler’s article (1994). the relations (4) show clearly that using an initial identity vectors leads to the next approximation for vector β as the average of the elements in the rows of the a matrix of alternatives, and for α the average of the elements in the rows of the b matrix of criteria. eventually the converged solution for the system (4), or the solution for the supermatrix (2), can be interpreted explicitly as follows: the vector of alternative priorities β is the average of the alternatives with weights of the mean criteria; and vice versa – the vector of criteria priorities α is the average of the criteria with weights of the mean alternatives. this simple and evident interpretation for a supermatrix solution can be completed by the following observation. for the matrices a and b of the alternatives and criteria priority vectors (as in table 1 or in the problem (2)) consider a system of the following equations: . 1 1 , 1 1 1 1 1 1 ∑ ∑ ∑ ∑ = = = = == m i iq m j jqj qn i ik n j jkj k a m b m b n a n β α α β (6) the elements kβ of the alternatives’ preference are defined by the quotient of the alternatives averaged with weights of the criteria (the first numerator) and the mean level of each alternative in the criteria matrix (in the first denominator). similarly, the elements qα of the criteria preference are defined by the quotient of the criteria averaged with weights of the alternatives (the second numerator) and the mean level of each criterion in the alternatives matrix (in the second denominator). all the local priority vectors are normalized so their total equals one, thus the denominators in the formulae (6) equal 1/n or 1/m, respectively. then, in matrix form, the relations (6) are similar to the equations in (4). it is interesting to note that the system (6) of mutual weighting of two data matrices is known from the socalled geary-khamis equations used in international statistics for comparisons of the currencies and volumes of various goods produced by different countries (see the references in the introduction). for transposed stochastic matrices connecting both vectors, the main eigenvalues equal one and the corresponding vectors give the shares of priorities. ijahp article: lipovetsky/ priority eigenvectors in analytic hierarchy/network processes with outer dependence between alternatives and criteria international journal of the analytic hierarchy process 176 vol. 3 issue 2 2011 issn 1936-6744 another ahp technique can be mentioned in relation to the priority estimation in equations (2) to (5) for the anp with outer dependence. it is based on the least squares (ls) approximation method of finding the priority vector solution of a matrix. let us assume there is such a solution, and name it αi. since such a vector exists, we can form a consistent matrix from ratios of its entries. since it is a consistent matrix we have not only the priority vector αi (which can be obtained in a number of equivalent ways: summing across the rows and normalizing the result, summing down the columns and normalizing the result, using least squares, or by finding its principal eigenvector) but also an anti-priority vector βi which we can obtain from the vector αi by inverting it. to invert a priority vector replace each entry αi by 1/αi, sum the entries and divide each entry by the sum. since we have a consistent matrix the assumed solution vector αi is a right eigenvector and the anti-priority vector βi is a left eigenvector. these are still unknown vectors, so we need to solve for them, and we shall do this using the method of least squares. the least squares approximation of the matrix of priority vectors in (1) by two vectors can be written as follows: ∑ −= ij jiijals 2)( βλα , (7) where α is a vector of priorities, β can be understood as a vector of reciprocal priorities, or anti-priorities, and their outer product corresponds to pairwise ratio elements of the matrix (1). differentiating the ls objective in (7) by the vectors' elements yields a system of equations: ∑∑ =−−= ∂ ∂ =−−= ∂ ∂ i ijiij jj jjiij i a ls a ls 0)()2(,0)()2( αβλαλ β ββλαλ α . (8) normalizing the eigenvectors by their euclidean norm, ∑∑ == i i j j 1,1 22 αβ (9) we can represent the system (8) as follows: ∑∑ == i jiij j ijij aa λβαλαβ , . (10) in matrix form the equations (10) are: λβαλαβ =ʹ′= aa , (11) ijahp article: lipovetsky/ priority eigenvectors in analytic hierarchy/network processes with outer dependence between alternatives and criteria international journal of the analytic hierarchy process 177 vol. 3 issue 2 2011 issn 1936-6744 and substituting one of these relations into another yields two eigenproblems: βλβαλα 22 , =ʹ′=ʹ′ aaaa (12) the vectors corresponding to the maximum eigenvalue 2λ define the priorities α and anti-priorities β of the compared items. the equations in (7) to (12) describe the singular value decomposition (svd) for the matrix (eckart and young 1936; lipovetsky and tishler 1994; gass and rapcsak 2004). the system of equations in (11) can be represented in a block supermatrix ⎟⎟ ⎠ ⎞ ⎜⎜ ⎝ ⎛ =⎟⎟ ⎠ ⎞ ⎜⎜ ⎝ ⎛ ⎟⎟ ⎠ ⎞ ⎜⎜ ⎝ ⎛ ʹ′ β α λ β α 0 0 a a (13) with the combined α and β vector. multiplying the matrix in (13) by this relation itself yields the eigenproblem: ⎟⎟ ⎠ ⎞ ⎜⎜ ⎝ ⎛ =⎟⎟ ⎠ ⎞ ⎜⎜ ⎝ ⎛ ⎟⎟ ⎠ ⎞ ⎜⎜ ⎝ ⎛ ʹ′ ʹ′ β α λ β α 2 0 0 aa aa (14) which represents the equations in (12) in one combined matrix form. comparison of the ahp problem given in (14) with the anp problem given in (3) shows that the anp is a generalization of the ahp problem to the case of two matrices in place of only one. but essentially both problems produce the same interpretation: the priority vectors represent the weighted averages of each item giving its preference over the other items under consideration. 4. summary this work has shown that the priority vector for a simple network with outer dependence between the alternatives and the criteria can be represented in the standard framework of the supermatrix eigenproblem solution which has the main eigenvectors related to the largest by modulo eigenvalues. the relations with other methods are also considered such as the least squares approach applied in ahp and the system so-called geary-khamis equations used in international statistics for currency rates, and for finding preferences evaluated both by ranks and ratios. the elements of the anp priority eigenvector have the same meaning as they do in the ahp – they give the mean preferences of each alternative or criterion (that is, element in the problem) over all the others involved in pairwise comparisons. the interpretation of ahp and anp priority vectors as representing the mean prevalence among the compared elements is useful for practical purposes by managers and decision makers. ijahp article: lipovetsky/ priority eigenvectors in analytic hierarchy/network processes with outer dependence between alternatives and criteria international journal of the analytic hierarchy process 178 vol. 3 issue 2 2011 issn 1936-6744 references eckart, c., & young, g. (1936). the approximation of one matrix by another of lower rank. psycometrika, 1, 211-218. gass, s.i., & rapcsak, t. (2004). singular value decomposition in ahp. european journal of operational research, 154, 573-584. geary, r.c. (1958). a note on the comparison of exchange rates and purchasing power between countries. journal of royal statistical society, a 121, 97-99. khamis, s.h. (1972). a new system of index numbers for national and international purposes. journal of royal statistical society, a 135, 96-121. kravis, i.b., kenessey, z., heston, a.w., & summers, r. (1975). a system of international comparisons of gross product and purchasing power. baltimore, md: johns hopkins university press. lipovetsky, s., & tishler, a. (1994). linear methods in multimode data analysis for decision making. computers and operations research, 21, 169-183. lipovetsky, s. (2007). priority and choice probability estimation by ranking, rating, and combined data, journal of statistical theory and practice, 1, 265-278. lipovetsky s. (2010). an interpretation of the ahp eigenvector solution for the lay person. international journal of the analytic hierarchy process, 2, 158-162. lipovetsky s. (2011). an interpretation of the ahp global priority as the eigenvector solution of an anp supermatrix. international journal of the analytic hierarchy process, 3, 70-78. rao, p., & selvanathan, e.a. (1992). computation of standard errors for geary-khamis parities and international prices: a stochastic approach. journal of business & economic statistics, 10, 109-115. saaty, t.l. (1994). fundamentals of decision making and priority theory with the analytic hierarchy process. rws publications, pittsburgh, pa. additional bibliography lipovetsky s. and conklin m. (2005). singular value decomposition in additive, multiplicative, and logistic forms. pattern recognition, 38, 1099-1110. lipovetsky, s. (2009). pca and svd with nonnegative loadings. pattern recognition, 42, 68-76. saaty, t.l. (1996). decision making with dependence and feedback: the analytic network process. rws publications, pittsburgh, pa. ijahp article: lipovetsky/ priority eigenvectors in analytic hierarchy/network processes with outer dependence between alternatives and criteria international journal of the analytic hierarchy process 179 vol. 3 issue 2 2011 issn 1936-6744 saaty, t.l. (2008). the analytic hierarchy and analytic network measurement processes: applications to decisions under risk. european journal of pure and applied mathematics, 1,122-196. saaty, t.l. (2009). extending the measurement of tangibles to intangibles, international journal. of information technology & decision making, 8, 7-27. saaty, t.l. (2010). principia mathematica decernendi: mathematical principles of decision making: generalization of the analytic network process to neural firing and synthesis. rws publications, pittsburgh, pa. saaty t.l., the thinking man’s new millennium joke book, part i, p.55. ijahp: saaty/about a hundred years of creativity in decision making international journal of the analytic hierarchy process 138 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.321 about a hundred years of creativity in decision making thomas l. saaty distinguished university professor university of pittsburgh saaty@katz.pitt.edu can all the methods that people have invented to do decision making be right all the time? this may be the case for trivial decisions, but not for decisions that involve dependence and feedback and that need complete analysis of benefits, opportunities, costs and risks to get an overall justifiable outcome. according to the great philosopher karl popper, a theory is scientific only if it can provide grounds for its own falsifiability. in decision making, a theory would not be scientific if it could not provide examples of where it would be false if it does not work. a theory cannot universally explain everything, and must define the limits of its applicability, or give examples of where it would not work. for example, for the ahp (analytic hierarchy process) we could say that just because it is a decision making theory that can be used on intangibles, the question still remains about whether the procedures of that theory can provide a correct estimate when we have tangible measurements to check against. the ahp would not work well in the hands of a neophyte who does not understand the problem being addressed. a decision making theory that claims to be valid for intangibles but not for tangibles is more like garbage in and garbage out. a decision theory should also be able to make tradeoffs possible, and thus be useful in resolving conflicts. following is a list of only a few of the decision making methods that are circulating. the kinds of numbers they use and how they combine these numbers is a challenge. what follows is taken from a section in a forthcoming paper (saaty & ergu, 2015). over the past few decades a number of mcdm methods have been developed to deal with the measurement of tangible/intangible (and conflicting) criteria and with the measurement of the alternatives of a decision with respect to these criteria (kou et al 2013, 2014a). some of the most popular methods include: the analytic hierarchy process (ahp), the analytic network process (anp) (saaty 1972; saaty 1980; saaty 2005), additive ratio assessment (aras) (turskis and zavadskas, 2010), complex proportional assessment of alternatives (copras) (zavadskas and kaklauskas, 1996), compromise programming (cp) (zeleny, 1973), decision making trial and evaluation laboratory (dematel) (gabus and fontela, 1972), dominance based rough set approach (drsa) (greco, matarazzo et al., 1999; greco, matarazzo et al., 2002), elimination et choice translating reality (electre) (roy, benayoun et al., 1966; roy, 1990), evidential reasoning (er) (lowrance and garvey, 1982), guess method (buchanan, 1997), goal programming (gp) (lee, 1972), grey relational analysis (gra) (deng, 1989), inner product of vectors (ipv), measuring attractiveness by a categorical based evaluation technique (macbeth) (bana e costa and vansnick, 1994), multi-attribute global inference of quality (magiq) (mccaffrey, 2009), multi-attribute utility theory (maut) (raiffa and keeney, 1976; humphreys 1977), multi-attribute value theory (mavt) (raiffa and keeney, 1976), maximal entropy ordered weighted averaging mailto:saaty@katz.pitt.edu file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_49 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_50 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_52 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_62 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_68 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_69 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_12 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_14 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_15 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_48 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_47 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_31 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_5 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_5 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_29 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_9 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_9 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_2 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_33 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_46 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_23 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_46 ijahp: saaty/about a hundred years of creativity in decision making international journal of the analytic hierarchy process 139 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.321 (me-owa) (yager, 1988; filev and yager, 1995), new approach to appraisal (nata), nonstructural fuzzy decision support system (nsfdss) (tam, tong et al., 2002; tam, tong et al., 2006), potentially all pairwise rankings of all possible alternatives (paprika) (hansen and ombler, 2008), preference ranking organization method for enrichment of evaluations (promethee) (brans, vincke et al., 1986; brans and mareschal, 2005), simple additive weighting (saw) (churchman and ackoff, 1954), superiority and inferiority ranking method (sir) (xu, 2001), technique for order preference by similarity to ideal solution (topsis) (hwang and yoon, 1981), utility additive (uta) (jacquet-lagreze and siskos, 1982), value analysis (va) (keen, 1981), value engineering (ve) (zimmerman and hart, 1982; parker, 1985; mudge, 1989), vikor method (opricović, 1980; opricovic and tzeng, 2007), weighted product model (wpm) (triantaphyllou and mann, 1989),weighted sum model (wsm) (fishburn, 1967; von winterfeldt and edwards, 1986) and on and on ad infinitum. many of the original mcdm methods have also been extended or adapted by the creators of those theories and by researchers on these methods. with these variations we have more than a hundred mcdm methods. this wide variety of available methods bewilders potential users, resulting in the difficulty of selecting an appropriate method (hobbs, 1986).that is, the existence of many mcdm methods itself becomes a decision problem! references aczel, j. and f. s. roberts (1989). on the possible merging functions. mathematical social sciences,17(3), 205-243. bana e costa, c. a. and j.-c. vansnick (1994). macbeth—an interactive path towards the construction of cardinal value functions. international transactions in operational research,1(4), 489-500. brans, j.-p. and b. mareschal (2005). promethee methods. multiple criteria decision analysis: state of the art surveys. new york: springer. brans, j.-p., p. vincke, et al. (1986). how to select and how to rank projects: the promethee method. european journal of operational research, 24(2), 228-238. buchanan, j. t. (1997). a naive approach for solving mcdm problems: the guess method. journal of the operational research society,48(2), 202-206. cho, k. t. (2003). multicriteria decision methods: an attempt to evaluate and unify. mathematical and computer modelling, 37(9), 1099-1119. churchman, c. w. and r. l. ackoff (1954). an approximate measure of value. operations research,2(2), 172-187. currim, i. s. and r. k. sarin (1984). a comparative evaluation of multi-attribute consumer preference models. management science,30(5), 543-561. file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_66 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_10 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_58 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_59 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_59 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_20 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_4 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_3 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_3 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_7 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_65 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_24 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_25 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_27 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_70 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_43 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_34 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_37 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_39 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_61 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_11 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_63 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_21 file:///c:/users/saaty/desktop/2015%20papers/saaty-ergu-mcdm%20paper-2-26-2015-revised%20version.doc%23_enref_21 ijahp: saaty/about a hundred years of creativity in decision making international journal of the analytic hierarchy process 140 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.321 deng, j.-l. (1989). introduction to grey system theory. the journal of grey system, 1(1), 1-24. filev, d. and r. r. yager (1995). analytic properties of maximum entropy owa operators. information sciences,85(1), 11-27. fishburn, p. c. (1967). letter to the editor—additive utilities with incomplete product sets: application to priorities and assignments. operations research,15(3), 537-542. gabus, a. and e. fontela (1972). world problems, an invitation to further thought within the framework of dematel. geneva, switzerland: battelle geneva research center. geldermann, j. and a. schöbel (2011). on the similarities of some multi‐criteria decision analysis methods. journal of multi‐criteria decision analysis, 18(3-4), 219230. greco, s., b. matarazzo, et al. (1999). rough approximation of a preference relation by dominance relations. european journal of operational research,117(1), 63-83. greco, s., b. matarazzo, et al. (2002). rough approximation by dominance relations. international journal of intelligent systems,17(2), 153-171. guba, e. g. (1981). criteria for assessing the trustworthiness of naturalistic inquiries. ectj, 29(2), 75-91. guitouni, a. and j.-m. martel (1998). tentative guidelines to help choosing an appropriate mcda method. european journal of operational research, 109(2), 501521. hajkowicz, s. and a. higgins (2008). a comparison of multiple criteria analysis techniques for water resource management. european journal of operational research,184(1), 255-265. hanne, t. (2001). intelligent strategies for meta multiple criteria decision making. dordrecht: kluwer academic publishers. hansen, p. and f. ombler (2008). a new method for scoring additive multi‐attribute value models using pairwise rankings of alternatives. journal of multi‐criteria decision analysis,15(3‐4), 87-107. hobbs, b. f. (1986). what can we learn from experiments in multiobjective decision analysis?, ieee transactions on systems, man and cybernetics, 16(3), 384-394. hobbs, b. f., v. chankong, et al. (1992). does choice of multicriteria method matter? an experiment in water resources planning. water resources research, 28(7), 1767-1779. humphreys, p. (1977). application of multi-attribute utility theory. decision making and change in human affairs. new york: springer. ijahp: saaty/about a hundred years of creativity in decision making international journal of the analytic hierarchy process 141 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.321 hwang, c.-l. and k. yoon (1981). multiple attribute decision making. new york: springer. jacquet-lagreze, e. and j. siskos (1982). assessing a set of additive utility functions for multicriteria decision-making, the uta method. european journal of operational research, 10(2), 151-164. karni, r., p. sanchez, et al. (1990). a comparative study of multiattribute decision making methodologies. theory and decision, 29(3), 203-222. keen, p. g. (1981). value analysis: justifying decision support systems. mis quarterly, 5(1), 1-15. kou, g., lu,y.,peng,y.,and shi, y.(2012), evaluation of classification algorithms using mcdm and rank correlation. international journal of information technology & decision making, 11(1),197-225. doi: http://dx.doi.org/10.1142/s0219622012500095 kou, g. ergu, d., shang, j., (2013). enhancing dataconsistency in decision matrix: adapting hadamard model to mitigate judgement contradiction, european journal of operational research, 236 (1), 261-271. doi: http://dx.doi.org/10.1016/j.ejor.2013.11.035 kou, g.,lin, c. (2014a). a cosine maximization method for the priority vector derivation in ahp, european journal, 235, 225–232. kou, g., peng, y., wang, g., (2014 b). evaluation of clustering algorithms for financial risk analysis using mcdm methods, journal of the operational research society, 1-12. khairullah, z. y. and s. zionts (1980). an experiment with some algorithms for multiple criteria decision making. in fandel, g., gal, t. (eds), multiple criteria decision making theory and application, (150-159). berlin: springer. lee, s. m. (1972). goal programming for decision analysis. philadelphia: auerbach. leshan, l. l. and h. margenau (1982). einstein's space and van gogh's sky: physical reality and beyond. new york: macmillan. lowrance, j. d. and t. d. garvey (1982). evidential reasoning: a developing concept. proceedings on the international conference of cybernetics and society, 6-9. maccrimmon, k. r. (1973). an overview of multiple objective decision making. in cochrane, j.l. and zeleny m. (eds), multiple criteria decision making (18-44). columbia, south carolina: the university of south carolina press. mccaffrey, j. d. (2009). using the multi-attribute global inference of quality (magiq) technique for software testing. information technology: new generations, 2009. itng'09. sixth international conference. ijahp: saaty/about a hundred years of creativity in decision making international journal of the analytic hierarchy process 142 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.321 mudge, a. e. (1989). value engineering: a systematic approach. pittsburgh, pa: j. pohl associates. olson, d. l. (2001). comparison of three multicriteria methods to predict known outcomes. european journal of operational research,130(3), 576-587. olson, d. l. (2004). comparison of weights in topsis models. mathematical and computer modelling, 40(7-8), 721-727. opricović, s. (1980). an extension of compromise programming to the solution of dynamic multicriteria problem. in iracki, k., malanowski, k., walukiewicz, s. (eds), optimization techniques, (508-517), springer. opricovic, s. and g.-h. tzeng (2004). compromise solution by mcdm methods: a comparative analysis of vikor and topsis. european journal of operational research, 156(2), 445-455. opricovic, s. and g.-h. tzeng (2007). extended vikor method in comparison with outranking methods. european journal of operational research, 178(2), 514-529. ozernoy, v. m. (1987). a framework for choosing the most appropriate discrete alternative multiple criteria decision-making method in decision support systems and expert systems. in sawaragi, y. (ed), toward interactive and intelligent decision support systems (56-64). berlin: springer. ozernoy, v. m. (1992). choosing the “best” multiple criteria decision-making method. infor, 30(2), 159-171. park, c. w. (1978). a conflict resolution choice model. journal of consumer research, 5 124-137. parker, d. e. (1985). value engineering theory. washington d.c.: lawrence d miles value foundation. peniwati, k. (2007). criteria for evaluating group decision-making methods. mathematical and computer modelling, 46(7), 935-947. raiffa, h. and r. keeney (1976). decisions with multiple objectives: preferences and value tradeoffs. new york: wiley. roy, b. (1990). the outranking approach and the foundations of electre methods. in c.a. bana e costa (ed.) readings in multiple criteria decision aid (155-183). berlin: springer-verlag. roy, b., r. benayoun, et al. (1966). electre. paris: société d’economie et de mathématique appliquées. saaty, t. l. (1980). the analytical hirarchical process. new york: wiley. ijahp: saaty/about a hundred years of creativity in decision making international journal of the analytic hierarchy process 143 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.321 saaty, t. l. (2004). fundamentals of the analytic network process—dependence and feedback in decision-making with a single network. journal of systems science and systems engineering, 13(2), 129-157. saaty, t. l. (2005). the analytic hierarchy and analytic network processes for the measurement of intangible criteria and for decision-making. in multiple criteria decision analysis: state of the art surveys (345-405). new york: springer. saaty, t. l. (2008). relative measurement and its generalization in decision making why pairwise comparisons are central in mathematics for the measurement of intangible factors the analytic hierarchy/network process (to the memory of my beloved friend professor sixto rios garcia). revista de la real academia de ciencias exactas fisicas y naturales serie a-matematicas, 102(2), 251-318. saaty, t. l., & ergu, d. (2015). when is a decision making method trustworthy? criteria for evaluating multi-criteria decision making methods. 14. saaty, t. l. and n. begicevic (2010). the scope of human values and human activities in decision making. applied soft computing, 10(4), 963-974. salminen, p., j. hokkanen, et al. (1998). comparing multicriteria methods in the context of environmental problems. european journal of operational research,104(3), 485-496. saltelli, a., m. ratto, et al. (2008). global sensitivity analysis: the primer. new york: wiley-interscience. saltelli, a., s. tarantola, et al. (1999). a quantitative model-independent method for global sensitivity analysis of model output. technometrics, 41(1), 39-56. tam, c., t. k. tong, et al. (2002). non-structural fuzzy decision support system for evaluation of construction safety management system. international journal of project management, 20(4), 303-313. tam, c., t. k. tong, et al. (2006). comparing non-structural fuzzy decision support system and analytical hierarchy process in decision-making for construction problems. european journal of operational research,174(2), 1317-1324. tam, c., t. k. tong, et al. (2002). site layout planning using nonstructural fuzzy decision support system. journal of construction engineering and management, 128(3), 220-231. triantaphyllou, e. and s. h. mann (1989). an examination of the effectiveness of multidimensional decision-making methods: a decision-making paradox. decision support systems, 5(3), 303-312. turskis, z. and e. k. zavadskas (2010). a new additive ratio assessment (aras) method in multicriteria decision-making. technological and economic development of economy,16(2), 159-172. ijahp: saaty/about a hundred years of creativity in decision making international journal of the analytic hierarchy process 144 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.321 von winterfeldt, d. and w. edwards (1986). decision analysis and behavioral research. cambridge: cambridge university press. wagner, h. m. (1995). global sensitivity analysis. operations research,43(6), 948-969. xu, x. (2001). the sir method: a superiority and inferiority ranking method for multiple criteria decision making. european journal of operational research,131(3), 587-602. yager, r. r. (1988). on ordered weighted averaging aggregation operators in multicriteria decisionmaking. systems, man and cybernetics, ieee transactions 18(1), 183-190. zanakis, s. h., a. solomon, et al. (1998). multi-attribute decision making: a simulation comparison of select methods. european journal of operational research,107(3), 507529. zavadskas, e. and a. kaklauskas (1996). multicriteria evaluation of building (pastatų sistemotechninis įvertinimas). vilnius: technika. zeleny, m. (1973). compromise programming. in cochrane j.l. and zeleny m. (eds), multiple criteria decision making (262-301). columbia, south carolina: university of south carolina press. zimmerman, l. w. and g. d. hart (1982). value engineering: a practical approach for owners, designers, and contractors. new york: van nostrand reinhold. 3 final_giordano_lombardi ijahp article: lombardi, p., giordano, s. / evaluating the european smart cities visions of the future international journal of the analytic hierarchy process 27 vol. 4 issue 1 2012 issn 1936-6744 evaluating the european smart cities visions of the future patrizia lombardi politecnico di torino interuniversity department of regional and urban studies & planning torino, italy patrizia.lombardi@polito.it silvia giordano siti innovation research centre of politecnico di torino torino, italy silvia.giordano@polito.it abstract this paper illustrates a novel application of an analytic network process (anp) in the field of smart cities with the aim of investigating the interrelations between the components and sectors of smart cities. although there is no agreement on the exact definition of a smart city, a number of main dimensions can be identified through literature review and these include: smart economy, smart mobility, smart environment, smart people, smart living, and smart governance. these six dimensions are used to evaluate four different policy visions of the city of the future as derived from the joint programme initiatives “urban europe” (jpi-eu). the results of the evaluation show that the entrepreneurial city is the policy vision with higher priorities in all the sectors considered in the model, i.e. universities, government, industry and civil society. some relevant urban planning and policy implications of this vision are provided in the conclusion. keywords: analytic network process, smart cities performances, triple helix approach. 1. introduction in the last fifty years the world population has grown exponentially, at an average rate of 1.2% per year, and has recently reached 7 billion people. this is accompanied by an urban explosion, with 1.25 million inhabitants joining urban areas every week. since 2008 over half the world’s population, that is to say 3.3 billion people, is now living in an urban environment. this figure will reach around 5 billion in 2030, and in 2050, 65% of people will live in urban areas. according to united nations, by 2020 about 60 million people will migrate from sub-saharan regions to north africa and europe, a flow which will increase the current high migration trend. the urban population issue is perhaps the most complex and crucial aspect of the sustainable human development problem. in many regions of the world, the natural increase in population has a greater effect on internal urban growth than the rural exodus. the future of europe will be an urbanized future. urbanization faces the grand challenges our society is confronted with, and simultaneously contributes to their scale and scope. european urban areas have to respond properly and urgently to avoid becoming less attractive to creative talents and firms and degrading in ‘liveability’. rob typewritten text rob typewritten text http://dx.doi.org/10.13033/ijahp.v4i1.108 ijahp article: lombardi, p., giordano, s. / evaluating the european smart cities visions of the future international journal of the analytic hierarchy process 28 vol. 4 issue 1 2012 issn 1936-6744 this leads to a new ‘urban imperative’ and raises the question of long-term strategies for sustainable development. in this context, the european commission has recently launched the joint programme initiatives “urban europe” (jpi-eu) with the goal of developing innovative approaches to adequately address the above mentioned challenges and to create urban places of vitality, liveability and accessibility. to reach these goals jpi-ue provides a systemic long term and strategic approach which takes advantage of emerging technologies, assessing their potential and socioeconomic impacts and utilising them in fundamentally new urban policies and design strategies (nijkamp, and koutrik, 2011). the jpi-ue research programme is based on four interlinked cornerstones: economic vitality, smart logistics and sustainable mobility, social participation and social capital, and ecological sustainability. to extract a systematic and coherent research programme from these cornerstones, the jpi-ue has developed four thematic urban images based on stylized appearances of urban agglomerations in the year 2050: • smart logistics and sustainable mobility – connected city 2050 • economic vitality – entrepreneurial city 2050 • ecological sustainability – liveable city 2050 • social participation and social capital – pioneer city 2050 all the developed city visions are connected to the concept of “smart city” which has been quite fashionable in the policy arena in recent years. a smart city is usually understood as a city with a high urban quality and a capacity to innovate by developing integrated actions regarding all aspects of economy, environment, quality of living, governance, transport and ict. therefore, smart urban development is based on a strategic vision and new approaches to policies and urban planning encompassing both efficient management of territorial resources and cultural identity through the use of advanced technologies (komninos, 2002; shapiro, 2008; deakin, 2010). this paper aims to offer an evaluation of the jpi-eu policy visions on the basis of an analysis of the interrelations between “smart” cities components and sectors of innovation. it adopts a novel framework for understanding smart city relationships called the triple helix approach. the triple helix model has emerged as a reference framework for the analysis of knowledge-based innovation systems, and relates the multiple and reciprocal relationships between the three main agencies in the process of knowledge creation and capitalization: university, industry and government (etzkowitz, 2008). this analysis of the triple helix is supported and augmented using the analytic network process (anp) in order to model, cluster and begin measuring the performance of smart cities. the model obtained allows interactions and feedbacks within and between clusters, providing a process to derive ratio scales priorities from elements (saaty, 2005). this offers a more truthful and realistic representation on which to support policy making. the model is applied by using a full list of indicators, available at the urban level, and a utilizing a focus group that supported judgments for deriving priorities. ijahp article: lombardi, p., giordano, s. / evaluating the european smart cities visions of the future international journal of the analytic hierarchy process 29 vol. 4 issue 1 2012 issn 1936-6744 2. development of a framework for evaluating smart cities performances the application of information and communication technology (ict) in the context of future cities is often indicated by the notion of a smart city. compared to the concept of a digital city or an intelligent city (lombardi et al., 2009), the main focus of a smart city is not limited to the role of ict infrastructure, but instead focuses on the role of human capital/education, social and relational capital and environmental issues. these are considered important drivers of urban growth. although the term “smart city” is not very widely used yet in spatial planning literature or regional and urban studies, it is still possible to identify various aspects of the concept as a basis for further elaboration. however, it should be noted that, in the literature, the term is not used in a holistic way; rather, it is often related to various aspects of urban life which range from economy to education. in association with economy, the term smart city is used to describe a city based on smart jobs or smart industries, i.e. the adoption of a modern business lifestyle and culture in a digital economy. this term recognizes that modern dynamic sectors of the economy, in particular the ict sector, have the potential to generate high returns. in addition, access to knowledge and information is considered a key entrepreneurial factor for success in an uncertain dynamic urban business environment (wakelin, 1990; acs et al., 2002; giffinger et al., 2007; fusco girard et al., 2009; caragliu et al., 2011). the term smart city is also used in relation to the training and education of citizens. a smart city therefore has smart inhabitants in terms of talent, skills and formative level (benner, c. 2003; florida, 2002). this concept is also linked to the influence and role that the university can have on local economy by creating living labs and innovation. such living labs benefit from interactions among companies, universities and research institutes as well as governmental institutions and organisations because these shape the urban innovation system and highlight their role as centres of excellence (torres, 2005). furthermore, the term smart city is used to illustrate the use of modern technology in everyday urban life. this includes both the relationship between the city government administration and its citizens as well as modern transport technologies. on one side, good governance or smart governance, often refers to the usage of new channels of communication for the citizens, e.g. “egovernance” or “e-democracy” (rosenthal and strange, 2001; lombardi et al., 2009). on the other side, logistics as well as new transport systems are “smart” systems which improve urban traffic and the inhabitants’ mobility. various other aspects are mentioned in the literature in connection with the term smart city like security/safety, green, efficient and sustainable, energy etc. (benner, 2003, komninos, 2007; giffinger et al., 2007; caragliu et al., 2011). in summary, there are several fields of activity and a number of main dimensions described in literature in relation to a smart city (giffinger et al., 2007; van soom, 2009; fusco girard et al., 2009). these dimensions include smart economy, smart mobility, smart environment, smart people, smart living, and smart governance. these six dimensions connect with traditional regional and neoclassical theories of urban growth and development. in particular, the dimensions are based, respectively, on theories of regional competitiveness, transport and ict economics, natural resources, human and social capital, quality of life, and participation of citizens in the governance of cities. in order to explore the concept of a smart city, an innovative conceptual framework has been suggested in this paper which is based on the triple helix approach (etzkowitz, 2008). this model has recently emerged as a reference for the analysis of knowledge-based innovation systems, and relates the multiple and reciprocal ijahp article: lombardi, p., giordano, s. / evaluating the european smart cities visions of the future international journal of the analytic hierarchy process 30 vol. 4 issue 1 2012 issn 1936-6744 relationships between the three main agencies in the process of knowledge creation and capitalization: university, industry and government (leydesdorff and deakin, 2011). in the triple helix innovation model, university-industry-government work together to achieve regional or national innovations in science and technology, forming a mutually beneficial relationship. the triple helix model is composed of three basic elements (1) a more prominent role for the university in innovation, on a par with industry and government in a knowledge –based society; (2) a movement toward collaborative relationships among the three major institutional spheres in which innovation policy is increasingly an outcome of interaction rather than a prescription from government; (3) in addition to fulfilling their traditional functions, each institutional sphere “takes the role of the other” in some regard. for instance, universities, traditional providers of human resources and knowledge, are now critical socio-economic development actors, performing a “third mission”, in addition to research and teaching (etzkowitz and leydesdorff, 2000). although the above mentioned model is able to generally explain innovation, it is clearly missing a relevant actor in the discussion of sustainable urban development, alongside the university, the industry and the government. the civil society plays a key role in innovation next to these three institutional spheres this was first highlighted by etzkowitz and zhou (2006) who explain, “university–industry– government relations, based on reciprocal principles, focus on the positive aspect of s&t development. when issues arise, the public starts to play a distinct role, directed toward redressing the negative consequences of s&t development or innovation”. in the innovative university–industry–government triple helix model, three institutional spheres interact to achieve innovation. thus, the university–industry–government triple helix is basically in alignment. however, there may be some conflicts between innovation and application of new ict solutions in society. only the inclusion of civil society in the model can guarantee the achievement of innovation in cities. this advanced model presupposes that the four helices operate in a complex urban environment, where civic involvement along with cultural and social capital endowments shape the relationships between the traditional helices of university, industry and government. the interplay between these actors and forces determines the success of a city in moving on a smart development path. the final framework includes both the above mentioned sectors or helices of innovation, i.e. university, industry, government and civil society, and the identified clusters or main components of a smart city, i.e. smart governance (related to participation); smart human capital (related to people); smart environment (related to natural resources); smart living (related to the quality of life) and smart economy (related to competitiveness). this framework has been used for classifying a number of smart city performance indicators, as shown in table 1. the sources of this data include both a detailed and focused literature review, including eu projects’ reports and urban audit dataset and indicators selected from statistics of european commission, european green city index, tissue, trends and indicators for monitoring the eu thematic strategy on sustainable development of urban environment and smart cities ranking of european medium-sized cities (giffinger et al., 2007; van soom, 2009; deakin, 2010; caragliu et al., 2011) and a focus group with specialists and professionals. the final list includes more than 60 indicators classified in the five clusters mentioned previously. for instance, an indicator such as “public expenditure on r&d” has been identified as “smart economy” and linked to university. another example ijahp article: lombardi, p., giordano, s. / evaluating the european smart cities visions of the future international journal of the analytic hierarchy process 31 vol. 4 issue 1 2012 issn 1936-6744 states that the number of lectures or courses which can be downloaded from the internet is a university performance indicator belonging to a “smart governance” category. this classification has been used for structuring an anp (analytic network process) exercise (saaty, 2005) with the aim of investigating the relations between smart cities components, actors and strategies. this exercise was conducted within a focus group, involving a number of experts in different disciplines as described in the next section. table 1 smart cities’ components, revised triple helix sectors and performance indicators smart governance smart economy smart human capital smart living smart environment u n iv e r s it y n. of universities, research centres in the city public expenditure on r&d % of gdp per head of city % of population aged 15-64 with secondary level education (urban audit) % of professors & researchers involved in international projects and exchange an assessment of the ambitiousness of co2 emissions reduction strategy n. courses entirely downloadable from the internet / total no. courses public expenditure on education % of gdp per head of city % of population aged 15-64 with high education (urban audit) number of grants for international mobility per year an assessment the extensiveness of city energy efficiency standards for buildings number of research grants funded by international projects % of inhabitants working in education and in research & development sector % of accessible courses g o v e r n e m e n t e-government on-line availability (% of the 20 basic services which are fully available online) gross domestic product per head voter turnout in national and eu parliamentary elections proportion of the area in recreational sports and leisure use total annual energy consumption, in gigajoules per head debt of municipal authority per inhabitant share of female city representatives green space (m2) to which the public has access, per capita efficient use of electricity (use per gdp) percentage of households with computers median or average disposable annual household income city representatives per resident number of public libraries total annual water consumption, in cubic metres per head unemployment rate number of theatres & cinemas efficient use of water (use per gdp) percentage of households with internet access at home energy intensity of the economy gross inland consumption of energy divided by gdp health care expenditure % of gdp per head of city area in green space (m2) greenhouse gas emissions intensity of energy consumption tourist overnight stays in registered accommodation in per year per resident an assessment of the comprehensiveness of policies to contain the urban sprawl and to improve and monitor environmental performance urban population exposure to air pollution by particulate matter micrograms per cubic metre c iv il s o c ie t y e-government usage by individuals (% individuals aged 16 to 74 who have used the internet, in the last 3 months, for interaction with public authorities) % of projects funded by civil society foreign language skills total book loans and other media per resident the total percentage of the working population travelling to work on public transport, by bicycle and by foot participation in life-long learning (%) museums visits per inhabitant an assessment of the extent to which citizens may participate in environmental decision-making individuals' level of computer skills theatre & cinema attendance per inhabitant an assessment of the extensiveness of efforts to increase the use of cleaner transport individuals' level of internet skills % of citizens engaged in environmental and sustainability oriented activity table 1 (cont’d) smart cities’ components, revised triple helix sectors and performance indicators ijahp article: lombardi, p., giordano, s. / evaluating the european smart cities visions of the future international journal of the analytic hierarchy process 32 vol. 4 issue 1 2012 issn 1936-6744 smart governance smart economy smart human capital smart living smart in d u s t r y number of research grants funded by companies, foundations, institutes / no annual scholarships employment rate in: high tech & creative industries renewable energy & energy efficiency systems financial intermediation and business activities culture & entertainment industry commercial services transport and communication hotels and restaurants patent applications per inhabitant number of enterprises adopting iso 14000 standards the percentage of total energy derived from renewable sources, as a share of the city's total energy consumption combined heat and power generation % of gross electricity generation all companies (total n.) employment rate in knowledge-intensive sectors rate of people undertaking industry based training rate of recycled waste per total kg of waste produced number of local units manufacturing high tech & ict products companies with hq in the city quoted on national stock market total co2 emissions, in tonnes per head (2) components of domestic material consumption % of new buildings and renovation, sustainability certification 3. assessing the smart city’s visions of the future in order to adequately address urban challenges and to create urban places of vitality, liveability and accessibility, the jpi-ue research programme has identified the following four interconnected urban images: the connected city (smart logistic and sustainable mobility): the image of a connected city refers to the fact that in an interlinked (from local to global) world, cities can no longer be economic islands (‘no fortresses’), but have to seek their development opportunities in the development of advanced transportation infrastructures, smart logistic systems and accessible communication systems through which cities become nodes or hubs in polycentric networks (including knowledge and innovation networks). the entrepreneurial city (economic vitality): this image assumes that in the current and future global and local competition, europe can survive only if it is able to maximize its innovative and creative potential in order to gain access to emerging markets outside europe. cities are then spearheads of europe’s globalization policy. the liveable city (ecological sustainability): this vision addresses the view that cites are not only energy consumers (and hence environmental polluters), but may through smart environmental and energy initiatives (e.g., recycling, waste recuperation) act as engines for ecologically‐benign strategies this allows cities to act as climate‐neutral agents in a future space‐economy, and cities in europe are then attractive places to live and work. the pioneer city (social participation and social capital): this image refers to the innovative ‘melting pot’ character of urban areas in the future, which will show unprecedented cultural diversity and fragmentation of lifestyles in european cities. this will prompt not only big challenges, but also great opportunities for smart and ijahp article: lombardi, p., giordano, s. / evaluating the european smart cities visions of the future international journal of the analytic hierarchy process 33 vol. 4 issue 1 2012 issn 1936-6744 creative initiatives in future cities, through which europe can become a global pioneer. these four thematic urban images on stylized appearances of urban agglomerations in the year 2050 are evaluated using the framework described in section 2 with the support of the anp model. this consists of five clusters of smart cities components, the cluster of the four urban visions, and the sixty selected indicators. a structured anp model has been developed which involves the clusters of a smart city, i.e. smart governance (related to participation), smart human capital (related to people), smart environment (related to natural resources), smart living (related to the quality of life) and smart economy (related to competitiveness). the relationships between indicators (and clusters) has been identified by using a “control hierarchy”, composed of the four axes of the adopted triple helix, i.e. university, industry, government, civil society, as shown in figure 1. figure 1the main network each axis is organised by a sub-network consisting of: • the five clusters representing the above mentioned smart cities component/activities including the relative selected indicators; • a cluster of alternatives composed of four policy visions (or prototypes) of smart cities in 2050, as derived from the “urban europe” joint programme initiatives (p. nijkamp, k. kourtik, 2011): connected city, entrepreneurial city, liveable city and the pioneer city. figure 2 shows an example of the civil society sub-network. bidirectional relationships have been recognized between “smart human capital” and “smart living” by means of indicators such as “museums visit per inhabitant”, “theatre and cinema attendance per inhabitant”, and “total book loans and other media per resident”. “smart economy” and “smart environment” have a bidirectional relationship by means of indicators such as “percentage of projects funded by civil society” and “relationship to percentage of citizens engaged in environmental and sustainability oriented activities”. in addition, a number of mono-directional relations are recognized between: “smart governance” and “smart human capital”, “smart economy” and “smart human capital”, and “smart human capital” and “smart environment”. ijahp article: lombardi, p., giordano, s. / evaluating the european smart cities visions of the future international journal of the analytic hierarchy process 34 vol. 4 issue 1 2012 issn 1936-6744 figure 2 the civil society sub-network the subsequent step of the anp application requires the development of pair wise comparisons of both elements (or nodes) and clusters. this assessment exercise was conducted within a focus group composed of a group of stakeholders belonging to all the identified sectors: university, industry, government, and civil society. focus group members had heterogeneous backgrounds and expertise in the following areas: environmental engineering, evaluation of the built environment sustainability, design, planning and micro-economy. although all the performance indicators included in the anp model are quantitative and measurable, this assessment exercise was conducted through judgment attribution not by using statistical data. the reason for this is that quantitative data were not available for the investigated areas, i.e. the four city visions. this was a pilot assessment exercise with the goal of comparing different urban images of the future. however, a quantitative evaluation is possible and feasible if one wishes to evaluate and rank different cities on the basis of their performance in the present. during the exercise a pair wise comparison was carried out both between clusters and nodes (indicators). in each pair wise comparison matrix a ratio scale of 1-9 was used. figure 3 shows one of the several pair matrices used to derive weighted priority vectors of elements (saaty, 2001). in particular, the figure shows the cluster comparison matrix for the alternatives. figure 3 pair wise cluster comparison using saaty’s fundamental scale ijahp article: lombardi, p., giordano, s. / evaluating the european smart cities visions of the future international journal of the analytic hierarchy process 35 vol. 4 issue 1 2012 issn 1936-6744 judgments/assessments have been done for the performance indicators in each subnetwork. the performance indicators (children nodes) are connected from the same cluster of smart city components (parent node), and are assessed with respect to how they influence that node, or how that node influences them. for instance, in the civil society sub-network, it was asked whether the “percentage of projects funded by civil society” is more influential than the “relationship to percentage of citizens engaged in environmental and sustainability oriented activities” in relation to smart economy. as recommended by adams and saaty (2003), influence has been treated consistently (how the parent influences the children, or vice versa) but the flow direction has been kept the same throughout the network and the model. the software used for making comparisons and for deriving priorities allows one to know in real time the comparisons coherence level. inconsistent answers were re-submitted to the participants until an acceptable compromise was found. nonsense questions, such as “with respect to the alternatives, which is more important (or is more influencing)?, the alternative cluster or the smart economy cluster?”, were left unanswered; in this case, the system, by default, assigns a 1 point judgment score to them. the final priorities have been derived from each sub-network. table 2 shows the final priorities (%) of the four urban visions in each sub-network. i in all the four rankings the preferred alternative is the entrepreneurial city. the second position is different only in the civil society sub-network. here, the pioneer city assumes a high priority while, in all the other sub-networks, the livable city is the second preferred vision. the ‘worst’ alternative is the connected city for both university and civil society while it is the pioneer city for both government and industry. table 2 final priorities (%) assigned to the alternatives related to each sub-network university government civil society industry connected city 14 16 16 16 entrepreneurial city 47 47 36 48 livable city 21 23 18 22 pioneer city 18 14 30 14 in addition to ranking the alternatives, a synthesis of the priorities for each node (performance indicator) has been derived in each sub-network. for instance, figure 4 shows the performance indicators with the highest priorities in the civil society. both “the e-government usage by individuals” and “the percentages of projects funded by civil society” are the two performance indicators with the highest priorities. these indicators belong respectively to the “smart governance” cluster and “smart economy” cluster. this sufficiently explains why the entrepreneurial city and the pioneer city are the two preferred alternatives. figure 4 illustrates the most preferred indicators in each sub-network. ijahp article: lombardi, p., giordano, s. / evaluating the european smart cities visions of the future international journal of the analytic hierarchy process 36 vol. 4 issue 1 2012 issn 1936-6744 figure 4 final priorities of both alternatives and nodes (indicators) in the civil society sub-network the results obtained in each sub-network are useful information for a decision-maker as they highlight the most preferred performance indicators, identifying a short-list of key performance indicators, as it is shown in table 3. table 3 the smart cities’ key performance indicators smart governance smart economy smart human capital smart living smart environment u n iv e r s it y n. of universities, research centres in the city public expenditure on education % of gdp per head of city % of inhabitants working in education and in research & development sector an assessment the extensiveness of city energy efficiency standards for buildings g o v e r n e m e n t percentage of households with internet access at home c iv il s o c ie t y e-government usage by individuals (% individuals aged 16 to 74 who have used the internet, in the last 3 months, for interaction with public authorities) % of projects funded by civil society participation in lifelong learning (%) total books and other media loan per resident in d u s t r y number of research grants funded by companies, foundations, institutes / no annual scholarships employment rate in: high tech & creative industries patent applications per inhabitant employment rate in knowledge-intensive sectors ijahp article: lombardi, p., giordano, s. / evaluating the european smart cities visions of the future international journal of the analytic hierarchy process 37 vol. 4 issue 1 2012 issn 1936-6744 finally, a ranking of the four urban visions has been obtained by synthesizing the priorities of the alternatives from all the sub networks. in order to derive the final priorities, the following combining formula has been used: $normalnet(civil society) * $smartalt (civil society) + $normalnet (government) * $smartalt (government) + $normalnet (industry) * $smartalt (industry) + $normalnet (university) *$smartalt (university). the overall priorities of the alternatives obtained are as follows: 1. entrepreneurial city (48%) 2. pioneer city (20%) 3. livable city (17%) 4. connected city (13%) 4. conclusions and further steps this paper has illustrated a study in the field of smart cities’ evaluation. the analysis began with a revised notion of the triple helix approach. this revision is based on the consideration that civil society usually plays a prominent role toward the realization of sustainable development in cities (etzkowitz and zhou, 2006). in order to assess the connections between smart city development and this institutionalization of the triple helix approach, an anp model has been developed. the development of this model, as well as the assessment exercise, is the result of a participative process, involving a number of people with urban expertise such as, urban planning, sustainable development evaluation, urban sociology and urban economy. this model has been used to investigate the relationship between smart cities components, actors and visions, or strategies. the anp model that was developed is not only able to underline the complexity of an urban system, but it also shows the relationships and the inter-connections between all the constituting elements of a smart cities vision. the main innovative features of the model are: • the introduction of the civil society as a crucial stakeholder that empowers the classical triple helix model composed by university-government-industry. • a more truthful and realistic city model representation based on a network system with the expression of relationships between elements. • the use of the four helices, representing the main stakeholders operating in a smart urban development, as control criteria for modelling the decision making problem. • a measurement of a “smart city” policy vision, considered as an holistic, interrelated, multistakeholders concept, which requires both quantitative indicators as well as experts’ opinions. this interrelated model has been used for both assessing four urban images as derived from the joint programme initiatives “urban europe” (jpi-eu) and selecting the keyperformance indicators for a smart city. the most relevant indicators are related to “smart governance” and include the following: • number of universities, research centres in the city • percentage of households with internet access at home ijahp article: lombardi, p., giordano, s. / evaluating the european smart cities visions of the future international journal of the analytic hierarchy process 38 vol. 4 issue 1 2012 issn 1936-6744 • e-government usage by individuals (% of individuals aged 16 to 74 who have used the internet, in the last 3 months, for interaction with public authorities) • number of research grants funded by companies, foundations, institutes / number of annual scholarships the most relevant indicators belonging to “smart economy” are: • public expenditure on education % of gdp per head of city • % of projects funded by civil society • employment rate in high-tech and creative industries finally, “smart environment” and “smart living” include, respectively, “an assessment of the extensiveness of city energy efficiency standards for buildings” and “total books and other media loan per resident”. a second relevant result of this exercise is the ranking of the urban visions. the final results show that the entrepreneurial city is the policy vision with higher priorities in all the sectors considered in the model, i.e. universities, government, industry and civil society. some relevant urban planning and policy implications of this vision are as follows: • a high degree of entrepreneurial activities and a constant flow of new firm creation is a prerequisite for finding a role within the new global economic landscape. innovation and creativity are thus the necessary ingredients for entrepreneurial cities in europe. • special emphasis has to be given to new architectures, building technologies, intra‐urban mobility solutions, public space management, e.g. for lighting or citizen information management, integrated urban energy planning and management and ict‐based solutions that offer various opportunities for new urban design and management. • new requirements for efficient, effective and reliable infrastructures (such as energy, ict, water, waste treatment and management etc) may occur. since an appropriate infrastructure is essential for a cities´ attractiveness to companies and people and therefore to their economic development, emphasis has to be given to the determination of these requirements within the scope of cities as complex systems. in conclusion, the results obtained from this exercise are interesting, but clearly the model requires further implementation and improvement. this assessment exercise is a pilot study, and still requires the development of a testing exercise with the participation of main city stakeholders, offering a reflexive learning opportunity for the cities to measure what options exist to improve their performances. the author’s plan is to develop this in the future. references abler, r., adams, j. s., gould p. (1971). spatial organization: the geographer's view of the world. michigan: prentice-hall. acs, z.j., groot, h.l.f. de and nijkamp p. (2002). the emergence of the knowledge economy, a regional perspective. berlin: springer. ijahp article: lombardi, p., giordano, s. / evaluating the european smart cities visions of the future international journal of the analytic hierarchy process 39 vol. 4 issue 1 2012 issn 1936-6744 adams w.j. and saaty r., (2003). super decisions software guide. retrieved from http://www.superdecisions.com benner, c. (2003). learning communities in a learning region: the soft infrastructure of cross-firm learning networks, environment and planning a, 35, 1809-1830. caragliu, a, del bo, c. & nijkamp, p. (2011). smart cities in europe. journal of urban technology, 18(2), 65-82. deakin, m. (2010). scran: the smart cities (inter) regional academic network supporting the development of a trans-national comparator for the standardisation of egovernment services, in reddick, c. comparative e-government: an examination of e-government across countries, (pp. 425-446) berlin: springer press. eu (2007-2013). smart cities is a north sea intereg 4b project. retrieved from http://www.northsearegion.eu/ivb/projects/details/&tid=84 etzkowitz h. and zhou c. (2006). triple helix twins: innovation and sustainability, science and public policy, 33(1), 77–83. ezkowitz, h. (2008). the triple helix: university, industry and government. london: routledge. etzkowitz h. and leydesdorf l. (2000). the dynamics of innovation: from national systems and ‘‘mode 2’’ to a triple helix of university–industry–government relations, research policy, 29, 109–123. http://www.elsevier.nlrlocatereconbase florida, r. (2002). the rise of the creative class. new york: basic books. fusco girad, l., lombardi, p., nijkamp, p. (2009). creative urban design and development (special issue), international journal of services technology and management, 13( 2/3/3,)111-115. giffinger, r., fertner, c., kramar, h., kalasek, r., pichler-milanovic, n., meijers, e. (2007). smart cities – ranking of european medium-sized cities, centre of regional science, vienna ut, october 2007. retrieved november 25, 2010, from http://www.smart-cities.eu komninos, n. (2002). intelligent cities: innovation, knowledge systems and digital spaces. london: spon press. leydesdorff, l. and deakin, m. (2011). the triple helix of smart cities: a neoinstitutional perspective, journal of urban technology, 18 (2), 53-63. lombardi p., giordano s., caragliu a., del bo c., deakin m., nijkamp p., kourtit k. and farouh h. (2012). an advanced triple-helix network model for smart cities performance, in y. ozge (ed.), green and ecological technologies for urban planning: creating smart cities, igi global, hershey, pp. 59-73. lombardi p., giordano s., farouh h. and yousef w. (2012). modelling the smart cities performances, the european journal of social science research, 25 (2), 133 145. ijahp article: lombardi, p., giordano, s. / evaluating the european smart cities visions of the future international journal of the analytic hierarchy process 40 vol. 4 issue 1 2012 issn 1936-6744 lombardi p., cooper i, paskaleva k & deakin m. (2009). the challenge of designing user-centric e-services: european dimensions, in c.reddick strategies for local egovernment adoption and implementation: comparative studies, 461-478. hershey, pa: idea group publishing. lombardi p. (2011). new challenges in the evaluation of smart cities. network industries quarterly, 13 (3), 8-10. nijkamp, p. and kourtik, k. (2011). joint programming initiative (jpi) on urban europe. global challenges and local responses in the urban century. a scoping document, 27, vu university amsterdam, 2011. rosenthal, s.s., and strange, w.c. (2001). the determinants of agglomeration. journal of urban economics, 50, 191-229. saaty, t.l. (2005). theory and applications of the analytic network process. pittsburgh: rws publications. shapiro, j. m. (2008). smart cities: quality of life, productivity, and the growth effects of human capital, the review of economics and statistics, 88 (2), 324-335. torres, l., vicente, p. and basilio, a. (2005). e-government developments on delivering public services among eu cities, government information quarterly, 22, 217238. van soom, e. (2009). measuring levels of supply and demand for e-services and egovernment: a toolkit for cities. smart cities research brief, n. 3, retrieved february 25, 2009, from http://www.smartcities.info/research-briefs. wakelin m. (1990). globalization of regional development. in gibson, d. v., kozmetsky, g., smilor, r. w. the technopolispphenomenon. smart cities, fast systems, global networks. usa: rowman & littlefield publishing. ijahp article: özdemir m., özdemir ms/ the use of the analytic hierarchy process for admission to an air traffic control department international journal of the analytic hierarchy process 64 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.503 the use of the analytic hierarchy process for admission to an air traffic control department mustafa özdemir* 1 department of air traffic control, anadolu university eskişehir, turkey mozdemir26@anadolu.edu.tr müjgan sağır özdemir department of industrial engineering, eskisehir osmangazi university eskişehir, turkey mujgan.sagir@gmail.com abstract meeting the need for qualified staff in the field of air traffic control depends primarily on selection of appropriate candidates among many applicants. the selection of competent air traffic controllers, who play a vital role in the air traffic system, is of critical importance and is usually conducted through multi-stage examinations. it is also important to use correct methods in the selection process in order to identify the most suitable candidates. in particular, the application of subjective examinations, such as interviews in a standard way, and determination and standardization of criteria can assist in selecting the right candidates. within this context, the student selection process in an air traffic control department is investigated, with the aim of supporting the interview examination through the analytic hierarchy process. the criteria weighting is determined by expert opinion. thirty-nine candidates are ranked according to the ahp, with the current and proposed rankings being compared. as a result, the ranking has changed significantly with the proposed approach. keywords: analytic hierarchy process; air traffic control; interview 1. introduction expertise and ability play an important role in the aviation industry. this role increases the responsibility of institutions providing education on the related subject. in particular, qualified staff in areas such as pilotage or air traffic control (atc) have a direct impact on the efficiency of education, as well as work in this area, and the successful fulfillment of responsibilities that are undertaken. the selection of staff for the intended purpose is the first and one of the most important processes in the training of qualified staff. therefore, it is necessary to comply with new conditions and to develop the selection acknowledgements: the authors would like to express their gratitude to prof. aydan cavcar and prof. oznur usanmaz for their help granting the permission for and providing the expertise to conduct the surveys at air traffic control department, faculty of aeronautics and astronautics. *corresponding author ijahp article: özdemir m., özdemir ms/ the use of the analytic hierarchy process for admission to an air traffic control department international journal of the analytic hierarchy process 65 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.503 process. in this study, the air traffic controller (atco) candidate selection process that was applied in 2014 is investigated and a new methodology for the interview examination is proposed, based on the analytic hierarchy process (ahp). the aim of the study is to select the most appropriate candidates for the atc department by minimizing any shortcomings of the interview process and to compare the proposed solution with the current one. there have been a number of studies concerning personnel and student selection using the ahp. altunok at al. (2010) suggest multi-criteria decision-making (mcdm) methods for postgraduate student selection. they compare results obtained by the ahp, weighted product (wp) and the technique for order preference by similarity to ideal solution (topsis). in order to determine the most appropriate method, they compute the total deviation between student’s cumulative grade point average (cgpa) and scores calculated by each method. as a result, they state that the ahp is the most appropriate method among the investigated techniques, as it has the least total deviation from the cgpa (altunoluk et al, 2010). gibney and shang (2007) propose the ahp method for a dean selection process, which consists of qualitative criteria, such as candor and honesty, trustworthiness, inclusive style of leadership and ability to develop vision. they state that the ahp can be applied for personnel selection in academia since it is a powerful tool for evaluating intangible criteria, and it can reflect judgments about feelings and emotions (gibney& shang, 2007). fuzzy ahp is also applied for similar problems. taşkın et al. (2013) use the fuzzy ahp to score and rank students during oral examinations for erasmus student mobility, with the results obtained from the fuzzy ahp being compared with the results of the rubric, which is a scoring tool that lists criteria for a piece of work. the authors state that the rank obtained by the fuzzy ahp is more satisfactory for decision-makers because it is more flexible, i.e., the criteria weightings can easily be changed (taşkın, üstün & deliktaş, 2013). in another study, kasim et al. (2012) use the ahp method to determine the importance and prioritization of each criterion for the teacher candidate selection process. they identify three main criteria; namely, content of knowledge, communication skill and personality, based on the literature and expert opinion. the weights of the criteria were determined by a group of experts using pairwise comparisons (kasim et al, 2012). taylor et al. (1998) conducted a study to research a college dean at texas a&m university using the ahp method. a committee, comprising faculty, administrators, and community representatives, identified four attributes including experience with aacsb (the association to advance collegiate schools of business) accreditation processes, experience in an administrative position, a good publication record and proven ability at fundraising. in their study, there were thirty-three candidates, and they divided the candidates into three groups with respect to each attribute in order to reduce the number of pairwise comparisons. they assumed that within these groups the candidates were approximately equal with respect to that attribute. this significantly reduces the evaluation time of decision-makers, especially when there are a large number of alternatives (taylor, ketcham & hoffman, 1998). norddin et al. (2012) investigated a new lecturer’s selection model to overcome shortcomings of the current selection process, such as difficulties in making comparisons among candidates and correctly evaluating their performance at all stages of the process before a final decision. they state that they chose the ahp method, since it possesses advantages over other mcdms, and that the criteria were identified as a mock teaching performance, a face-to-face interview, and academic qualifications (norddin, ibrahim & aziz, 2012). ijahp article: özdemir m., özdemir ms/ the use of the analytic hierarchy process for admission to an air traffic control department international journal of the analytic hierarchy process 66 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.503 most decision problems involving qualitative criteria have difficulty combining with quantitative criteria. when quantitative techniques are used, intangibles may have to be ignored. however, the ahp is a decision-making theory of measurement through pairwise comparisons which helps decision-makers to select the best alternatives or to rank the alternatives by taking into account qualitative and quantitative aspects of decisions (bogdanovic & miletic, 2014; saaty, 2008). it is especially useful for evaluating complex multi-attribute alternatives involving qualitative criteria (kwong &bai, 2002; saaty & sagir, 2009). in addition, when there are many alternatives, rating is a more effective and practical method to evaluate the alternatives. in this study, an ahp model based on a rating scale is proposed to be used in the interview stage where candidates are evaluated based on their qualitative characteristics. the ratings mode is explained in greater detail later. the use of the ahp is not a new approach in student selection problems. however, it has not yet been implemented in the area of atc to standardize and minimize the shortcomings of interviews. it is also believed that this study will improve the student selection process in the related department, and that the criteria and prioritization identified in the study will contribute to personnel selection problems with similar criteria. 2. problem statement and a decision support model for the student selection process in civil and military aviation, a great majority of accidents are caused by human error (wiegmann & shappell, 2001). for this reason, the selection of qualified atcos is especially important in order to improve safety in flight operations. one measure that can be taken to achieve this is to develop and implement an appropriate selection process or to overcome deficiencies in the existing system. interviews are frequently used as a method for assessing the suitability of candidates for a particular profession by questionnaire and giving points to evaluate their discernment, comprehension ability, overall appearance, appropriateness, competence, and suitability of behavior and reactions. selection of candidates by interview only can prevent selection of the most appropriate candidate. this can lead to one being skeptical of the results because interviews are considerably open to subjective evaluation and are often far from objective. furthermore, appealing interview results are common, and it becomes a burden for the interviewers and institutions conducting the interview. therefore, conducting the interview examination in a more systematic way can avoid possible inconveniences and lead to a more effective selection procedure. 2.1 current system the department of atc admits its students according to scores achieved in a special talent exam which is held every year. it consists of four stages (four examinations) including mathematics, aptitude, interview and simulations. the admissions process is organized by the department management. the candidates are evaluated based on a method of elimination. on the other hand, due to the elimination method, candidates who are going to be evaluated by interviewers are those who have already maintained their chances of being selected; i.e. candidates passing the first two examinations can ijahp article: özdemir m., özdemir ms/ the use of the analytic hierarchy process for admission to an air traffic control department international journal of the analytic hierarchy process 67 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.503 participate in the interview procedure. the interview panel consists of the decisionmakers at this phase, but their task is limited to this stage. in addition, we can say that each candidate is evaluated with respect to different criterion at each step. in the study, the special talent examination applied in 2014 is investigated. the candidates need to meet certain conditions in order to participate in this examination. they are required to be under the age of 21 years and have received at least 250 points (increased to 280 in 2016) in the university entrance examination, conducted by the student selection and placement center (osym), in order to apply for the department. in addition, the candidates should have no physical disabilities, speech disorders, hearing loss, or color blindness. 194 candidates fulfilling these requirements were entitled to participate in the examinations. figure 1 illustrates the entire selection and elimination process. the number of candidates eligible to take the mathematics examination was 194. following the mathematics examination, 94 were eliminated and 100 candidates took the aptitude examination. following this, 60 of the 100 candidates were eliminated. the remaining candidates went on to interview. finally, 20 remaining candidates were ranked according to their scores. at the end, the first 15 of the 20 candidates were determined as principals with the next 5 candidates being determined as alternates. figure 1 current selection process a commission consisting of five members conduct the interview stage. every member evaluates candidates based on various criteria and gives a score in order to measure a candidate's ability for the profession. each candidate achieves five scores from every member of the commission. under the current system, the interview grade is calculated by arithmetic mean, with 40 candidates being graded by this method. all candidates math exam (194 candidates) aptitude exam (100 candidates) interview exam (40 candidates) simulation exam (30 candidates) principal: 15 alternate: 5 94 candidates eliminated 60 candidates eliminated 10 candidates eliminated 10 candidates eliminated ijahp article: özdemir m., özdemir ms/ the use of the analytic hierarchy process for admission to an air traffic control department international journal of the analytic hierarchy process 68 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.503 2.2 proposed approach the proposed approach is applied to the interview stage, since it has qualitative factors and subjective assessment. the other stages could be considered as being straightforward, and are based on quantitative and objective assessment. the study is conducted in two stages; firstly, criteria and intensity levels are determined, and their weights are calculated by pairwise comparison of the ahp, then the alternatives are evaluated by the rating method using expert choice software. 2.3 the analytic hierarchy process the ahp is a tool developed by thomas saaty to help decision-makers in complex decision problems involving many attributes of varying degrees of subjectivity (taylor, ketcham & hoffman, 1998). the ahp is applied in different decision problems, such as planning, resource allocation, resolving conflict, optimization, and so on (vaidya & kumar, 2006). when applying the ahp in a decision problem, the factors affecting the decision are transformed into a hierarchy consisting of a goal, criteria and alternatives. the goal of the decision problem is stated at the top of the hierarchy, and criteria are placed at the bottom of the goal. alternatives are at the lowest level of the hierarchy (an, kim & kang, 2007). after the hierarchy has been created, decision-makers determine the relative importance of the main and sub-criteria by way of a pairwise comparison process. in this process, the criteria at the same level of the hierarchy are compared and scored using saaty's 1-9 scale of pairwise comparisons shown in table 1. for example, suppose that two of the criteria in a decision problem are called a and b. if criterion a has strong importance over criterion b with respect to the goal, then it will be shown by the number 5 in the pairwise comparison matrix. if criterion b has strong importance over criterion a, it will be shown by 1/5. in this way, pairwise comparison matrices are obtained. next, each element of the pairwise comparison matrix is divided by the sum of its corresponding column elements. finally, the relative weights of the criteria are calculated by taking the average of each row (al-harbi, 2001). ijahp article: özdemir m., özdemir ms/ the use of the analytic hierarchy process for admission to an air traffic control department international journal of the analytic hierarchy process 69 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.503 table 1 the fundamental scale of absolute numbers (saaty, 2008). intensity of importance definition explanation 1 equal importance two activities contribute equally to the objective 2 weak or slight 3 moderate importance experience and judgment slightly favor one activity over another 4 moderate plus 5 strong importance experience and judgment strongly favor one activity over another 6 strong plus 7 very strong or demonstrated importance an activity is favored very strongly over another, its dominance demonstrated in practice 8 very, very strong 9 extreme importance the evidence favoring one activity over another is of the highest possible order of affirmation for the final step, on evaluating the alternatives, each alternative is compared with the other alternatives with respect to each criterion and sub‐criterion. the pairwise comparison process allows decision-makers to measure the inconsistency ratio at each level of the hierarchy. if this exceeds 0.10, judgments should be re-examined. if the inconsistency is at an acceptable level, the alternatives are ranked from the most preferred to the least preferred (saaty, 1987; swiercz & ezzedeen, 2001). the use of pairwise comparison in this final step becomes difficult as the number of alternatives increases, because the number of pairwise comparisons reaches an unreasonable level at large numbers of alternatives. if each candidate is compared to every other candidate, the total number of comparisons that must be performed is: 𝑇 = 𝑛(𝑛−1) 2 (1) where t is the total number of comparisons, and n is the number of alternatives (taylor, ketcham & hoffman, 1998). in our case, n is 39. in the case of using relative mode, the total number of comparisons required would be 741, and this process must be applied for nine criteria. in such cases, the use of the rating method would be more appropriate. in the rating method, alternatives are not compared pairwise, but are evaluated independently using intensity levels for each criterion, and this significantly reduces the number of pair-wise comparisons. the two methods, relative and rating modes, do not deliver the same priorities exactly. the relative method where alternatives are compared with each other under the various criteria is more accurate. the ratings method has the advantage that one can rate large numbers of alternatives rather quickly, and the results are adequately close (saaty, 2008). in light of the above explanations, we prefer using the rating mode to evaluate the alternatives for this study. ijahp article: özdemir m., özdemir ms/ the use of the analytic hierarchy process for admission to an air traffic control department international journal of the analytic hierarchy process 70 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.503 2.4 determination of the criteria atcos are responsible for ensuring the safe, regular and rapid flow of aircraft traffic in airspace and at airports (icao, 2007). their tasks are based mainly on cognitive skills, such as knowledge, attention, memory, judgment and evaluation, reasoning and computation, problem-solving, fast and accurate decision-making under stress, quick verbal communication skills and so on. (eurocontrol, 1998). they consistently carry out their duties in coordination and cooperation with pilots, technical personnel, management and other supervisors. in such interactions, accurate and rapid communication becomes crucial. in particular, an error that may occur during communication between a pilot and an air traffic controller may jeopardize flight safety. therefore, the communication skills and diction of the candidates are vital for a proper understanding of instructions. non-verbal communication skills make verbal communication more effective and are considered to be important during an interview in accordance with expert opinion. an air traffic controller’s job is a profession which comes with a high level of stress. air traffic controllers need to make and apply a lot of decisions within a limited time in an intensive and isolated working environment. such factors cause physical and mental stress on controllers. therefore, fatigue and pressure from high workplace stress negatively impact professional skills, responsiveness and prudence of atcos. as a result, one of the most important characteristics of those wishing to enter this profession is the ability to cope with stress and fulfill their duty under heavy stress (jou, kuo & tang, 2013). the air traffic system is a dynamic environment with only short periods of time available for control; any lack of attention may cause serious consequences. it is also difficult to maintain attention while traffic is being monitored over a long period. a controller should not confuse tasks when doing two different things simultaneously, such as monitoring traffic and following a written procedure. consequently, candidates are expected to have a high level of attention (eurocontrol, 2010). the job interest criterion aims to test whether candidates really want the job and have a basic knowledge of atc. self-reliance refers to the confidence that a candidate has in his/her ability to manage the work and the impression gained by interviewers in this regard. previously, five items of criteria have been used to rank candidates. these can be defined as communication skills, stress control, attention level, job interest and self-reliance. in the proposed approach, the criteria have been revised and expanded by noting the opinions of interview commission members. we suggest that the communication skill criterion should be divided into sub-criteria, because it contains many different qualifications. therefore, five main criteria and eight sub-criteria have been determined. after the new criteria set was identified, thirteen experts were surveyed in order to determine the criteria’s relative importance. an example of the survey is given in appendix 1. the professions and years of experience of the experts are given in table 2. the importance level of the criteria according to each other are obtained by geometric mean of the scores given by the experts. ijahp article: özdemir m., özdemir ms/ the use of the analytic hierarchy process for admission to an air traffic control department international journal of the analytic hierarchy process 71 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.503 table 2 professions and years of experience of the experts experts area of expertise years of experience 1 licensed atco, atc instructor, and member of interview commission 25 2 licensed atco, atc instructor, and member of interview commission 19 3 licensed atco, and atc instructor 13 4 licensed atco, and atc instructor 9 5 licensed atco, and atc instructor 7 6 licensed atco, actively works as an atco 10 7 licensed atco, actively works as an atco 5 8 academician in atc, and member of the interview commission 29 9 academician in atc, and member of the interview commission 27 10 academician in atc, and member of the interview commission 17 11 academician in atc 5 12 academician in atc 4 13 academician in atc 3 2.5 the proposed ahp model in order to implement the rating method, intensity levels should be created for each criterion. the criterion with higher weightings are divided into further levels in order to achieve more precise assessments. after the identification of the criteria and the intensities for each criterion, an ahp model is developed, as shown in figure 2. the model consists of the goal, the criteria and the weightings obtained from paired comparisons, and the intensities for each criterion. the rating categories are prioritized by pair-wise comparison. for example, the rating categories for the manner of speaking criterion in figure 2, is decided as very good, good, fair and poor; we compare these for preference using a pair-wise comparison matrix in the usual way, as given in table 3. the rating categories for all the covering criteria and their priorities are established in a similar way. next, the idealized values are obtained by dividing by the largest value in the vector of priorities, derived from their paired comparisons matrix. the idealized priorities are always used for ratings, and this allows the outstanding to receive the full criterion weighting. alternatives are then evaluated by selecting the appropriate rating category on each criterion. ijahp article: özdemir m., özdemir ms/ the use of the analytic hierarchy process for admission to an air traffic control department international journal of the analytic hierarchy process 72 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.503 table 3 deriving priorities for ratings on manner of speaking manner of speaking very good good fair poor priorities idealized priorities very good 1 2 3 4 0.467 1.000 good 1/2 1 2 3 0.277 0.593 fair 1/3 1/2 1 2 0.160 0.343 poor 1/4 1/3 1/2 1 0.095 0.204 in the model, there are four different intensity levels, with the highest inconsistency ratio among them being obtained as 0.02. a final matrix inconsistency ratio for the criteria is obtained equal to 0.006, which indicates that the model is well-structured. of the five main criteria selected for the ahp model, the most important is stress control, and the least important is job interest. a candidate should simultaneously satisfy a number of requirements to receive a high score in communication skills, since it consists of several sub-criteria. ijahp article: özdemir m., özdemir ms/ the use of the analytic hierarchy process for admission to an air traffic control department international journal of the analytic hierarchy process 73 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.503 goal: student selection for air traffic control department attention level (0.254) self-reliance (0.173) job interest (0.100) communication skills (0.217) stress control (0.256) poor excellent good above average average below average 1.000 0.655 0.417 0.263 0.168 0.112 1.000 0.655 0.417 0.263 0.168 0.112 1.000 0.593 0.343 0.204 1.000 0.593 0.343 0.204 1.000 0.627 0.382 0.232 0.148 1.000 0.627 0.382 0.232 0.148 1.000 0.550 0.303 1.000 0.550 0.303 1.000 0 0.593 0.343 0.204 average excellent good above average below average poor very good good fair poor very good good fair poor verbal communication (0.798) non-verbal communication (0.202) manner of speaking (0.424) appearance (0.330) body language (0.670) tone of voice (0.364) pronunciation (0.636) diction (0.576) very good good fair poor very good good acceptable poor very poor very good good acceptabl e poor very poor good fair bad good fair bad figure 2 ahp model for student selection problem for air traffic control department 2.6 application of rating method during the interview phase, a number of procedures are carried out to assess a candidate according to each criterion. for example, candidates are asked basic questions about the atc in order to determine their job interest, or they are asked to read a specified text that can reveal their level of communication skills, such as pronunciation and tone of voice. following the same procedure described above, based on the rating method, the candidates are re-evaluated using the criteria weights and the intensity levels associated with each criterion. in applying the rating method, there are two parts in our individual score calculation for each candidate, and we divide the criteria in figure 2 into two sets as follows: ijahp article: özdemir m., özdemir ms/ the use of the analytic hierarchy process for admission to an air traffic control department international journal of the analytic hierarchy process 74 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.503  the criteria set-1: it consists of the criteria that have already been used in the current interview process, namely stress control, attention level, job interest and self-reliance.  the criteria set-2: it consists of the newly added criteria, i.e. sub-criteria of communication skills. we already have the score (1-100) of each candidate for each item of the criteria set-1. we also define the intensity levels for these criteria. in the first part, in order to transform the numerical score of a candidate for the criteria set-1 into the intensity levels, we use the following methodology. if we assume that a candidate’s score is 60 for job interest in 2014, we determine the intervals for this criterion depending on the number of the intensity levels. job interest includes four levels, so the intervals are defined as 0-25 (poor), 26-50 (fair), 51-75 (good) and 76-100 (very good). the intervals for the other criteria in the criteria set-1 are established in a similar way. therefore, since this candidate has a score of 60 in the job interest criterion, then, this candidate's score category is determined as ‘good’ according to the uniform category above. the calculation is then conducted by using the numerical equivalence of ‘good’, which is 0.593 for the related criterion (see figure 2). this way we are able to transform the previous scores into our categories. since we obtain the priority for each intensity level by paired comparisons (i.e. 0.593 for ‘good’ here), these adjusted values are more appropriate to use. we use this simple transformation to protect the original judgements of the experts for the criteria set-1, instead of asking them their opinions again. the second part is related to the rating of the alternatives in terms of each item of the criteria set-2. in this part, the evaluations are carried out by the experts through watching the videos recorded during the interview. these new criteria were not used before so the evaluation scores of the candidates for these criteria are not available. finally, the score of each candidate for each criterion is obtained by multiplying the related criterion weighting by the corresponding intensity level. a candidate's final score is calculated by totaling these scores. for example, if the evaluations of candidate 1 (c1), with respect to manner of speaking, pronunciation, tone of voice, body language, appearance, stress control, attention level, job interest and self-reliance are good, good, acceptable, bad, good, poor, poor, poor and fair, respectively, the final score for this candidate would be obtained as follows: (0.073×0.593) + (0.064×0.627) + (0.036×0.382) + (0.029×0.303) + (0.014×1.00) + (0.256×0.112) + (0.254×0.112) + (0.1×0.204) + (0.173×0.343) = 0.257 therefore, a new ranking based on the ahp is obtained, and the evaluation of other candidates is given in appendix 2. 2.7 results and sensitivity analysis the rankings of candidates according to interview and ahp grade are listed in table 4. it can be seen that the rankings of nine candidates have not changed while the rankings of five candidates have changed the most (c3, c16, c26, c35 and c36). it is believed that the main reason for the differences between the interview and the ahp ranking is the differences in the criteria weightings used by both methods. in the proposed method, for instance, the weighting of stress control is 25.6% while the weighting of job interest is ijahp article: özdemir m., özdemir ms/ the use of the analytic hierarchy process for admission to an air traffic control department international journal of the analytic hierarchy process 75 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.503 10%. however, these criteria were equally important in the current process. in addition, we propose to use more criteria by constructing an ahp model for the problem under consideration. as previously stated, the first 30 candidates according to the ranking obtained in the interview phase proceed to the next stage, and the remaining candidates are eliminated. the candidates who are eliminated in the interview process are c4, c18, c20, c23 c24, c26, c34, c38 and c39. with the ahp, not only is the ranking changed, but also two of the candidates to be eliminated are changed. if the ahp method was applied, c10 and c36 would be eliminated instead of c18 and c26. although only two eliminated candidates are changed according to these two methods, the grades obtained from the interview and ahp are quite different from each other. for example, in the interview process, c36 is the latest candidate with a right to pass to the next stage ranking 30 th with 26.40 points while, in the ahp method, this candidate is c32 ranking 30 th with 23.96 points. this situation has created a number of differences in the rankings. in addition, these grades are also important, since they are taken into account in a final assessment to determine successful candidates. the criteria with a high level of importance have more influence on the ranking of candidates than those with low levels of importance. for instance, a candidate can be placed among the top fifteen if he/she gets a full score in one item of criteria including stress control, attention level and communication skills; however, self-reliance and job interest are insufficient by themselves. ijahp article: özdemir m., özdemir ms/ the use of the analytic hierarchy process for admission to an air traffic control department international journal of the analytic hierarchy process 76 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.503 table 4 comparison of the candidate’s interview and ahp grades candidate number interview grade ranking based on interview ahp grade ranking based on ahp difference c1 33.12 22 25.74 20 2 c2 77.80 3 73.97 3 0 c3 53.40 8 33.11 13 -5 c4 26.12 32 21.87 36 -4 c5 32.80 23 26.67 19 4 c6 35.68 19 27.43 18 1 c7 67.80 5 47.49 5 0 c8 49.04 10 36.07 10 0 c9 34.32 21 25.39 22 -1 c10 27.24 29 23.54 33 -4 c11 28.88 26 24.30 27 -1 c12 80.00 2 77.02 2 0 c13 30.40 24 24.96 25 -1 c14 53.20 9 37.70 8 1 c15 85.00 1 87.59 1 0 c16 42.60 12 31.43 17 -5 c17 41.44 14 32.90 14 0 c18 26.28 31 24.18 29 2 c19 53.92 6 43.29 7 -1 c20 24.88 34 23.84 31 3 c21 70.80 4 55.68 4 0 c22 53.84 7 44.80 6 1 c23 22.80 37 22.81 34 3 c24 24.56 36 23.66 32 4 c25 37.08 17 32.67 15 2 c26 25.44 33 24.29 28 5 c27 30.00 25 25.63 21 4 c28 34.60 20 25.07 24 -4 c29 41.64 13 35.28 11 2 c30 46.60 11 37.03 9 2 c31 40.80 15 33.37 12 3 c32 28.80 27 23.96 30 -3 c33 39.80 16 32.24 16 0 c34 24.80 35 20.02 39 -4 c35 36.40 18 25.31 23 -5 c36 26.40 30 22.47 35 -5 c37 28.12 28 24.31 26 2 c38 16.00 38 20.91 38 0 c39 14.00 39 21.75 37 2 ijahp article: özdemir m., özdemir ms/ the use of the analytic hierarchy process for admission to an air traffic control department international journal of the analytic hierarchy process 77 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.503 a sensitivity analysis allows decision-makers to test the robustness of the rankings for the alternatives. to perform a sensitivity analysis, it is necessary to change the weighting of a criterion and see how they affect the overall priorities of the alternatives. increasing or decreasing a criterion weighting changes the weightings of the other criteria, proportionally. the sensitivity analysis is conducted in three scenarios: (a) when all the main criteria have the same weighting, i.e. 0.20 each; (b) increasing the weighting of stress control by 10%; and (c) increasing the weighting of job interest by 10%. when the weightings of the main criteria are taken as equal, the weightings of stress control, attention level and communication skills are decreased, while the others are increased. in figures 3 and 4, bars above and below the axis represent the increase and decrease in the ranking, respectively. for example, c20 is ranked as 31st in normal ranking while his raking is raised to 28th when we assume that all the main criteria have an equal weighting, and the difference is 3. candidates such as c11, c13 and c20 are raised in the ranking as can be seen in figure 3. on the other hand, the rankings of certain candidates, such as c25, c26 and c28, who are categorized as ‘poor’ in job interest and self-reliance, are reduced. as a result, the rankings of twelve candidates are changed. figure 3 sensitivity analysis when all the main criteria have equal weighting increasing the weighting of stress control by 10% changes the rankings of 10 candidates as shown in figure 4. however, changing the weighting of job interest by 10% does not make any changes in the rankings, since it is a criterion with low weighting, and thus its reflection on the scores is limited. -4 -3 -2 -1 0 1 2 3 4 c 1 c 2 c 3 c 4 c 5 c 6 c 7 c 8 c 9 c 1 0 c 1 1 c 1 2 c 1 3 c 1 4 c 1 5 c 1 6 c 1 7 c 1 8 c 1 9 c 2 0 c 2 1 c 2 2 c 2 3 c 2 4 c 2 5 c 2 6 c 2 7 c 2 8 c 2 9 c 3 0 c 3 1 c 3 2 c 3 3 c 3 4 c 3 5 c 3 6 c 3 7 c 3 8 c 3 9 d if fe re n c e ijahp article: özdemir m., özdemir ms/ the use of the analytic hierarchy process for admission to an air traffic control department international journal of the analytic hierarchy process 78 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.503 figure 4 sensitivity analysis of alternatives when stress control is increased by %10 3. discussion and conclusions the main concern about the interview process that consists of many qualitative factors is the objectivity of the process. it is not easy to evaluate qualitative factors objectively because an interviewer's overall impression of a candidate may influence the interviewer's feelings and thoughts about that candidate's character. to prevent such situations, it is necessary to determine the criteria and criteria weightings by appropriate methods, and to systematize the interview process. in the study, a method based on the ahp is proposed for use in the interview examination of an air traffic control department. as a result, thirty-nine candidates are re-ranked according to this method and compared with the current ranking. in addition, a sensitivity analysis is conducted for different scenarios. in the current interview process, each criterion is considered to be equally weighted. however, in this study, the weightings of the criteria are determined by expert opinions, and experts have noted that, for atcos, certain criteria, such as stress control and attention level, are far more important than others. therefore, the criteria with higher levels of importance have greater influence on the ranking of candidates than those with lower levels of importance. as a result, the use of the ahp at the interview phase may ensure that more appropriate candidates are selected. -4 -3 -2 -1 0 1 2 3 4 c 1 c 2 c 3 c 4 c 5 c 6 c 7 c 8 c 9 c 1 0 c 1 1 c 1 2 c 1 3 c 1 4 c 1 5 c 1 6 c 1 7 c 1 8 c 1 9 c 2 0 c 2 1 c 2 2 c 2 3 c 2 4 c 2 5 c 2 6 c 2 7 c 2 8 c 2 9 c 3 0 c 3 1 c 3 2 c 3 3 c 3 4 c 3 5 c 3 6 c 3 7 c 3 8 c 3 9 d if fe re n c e ijahp article: özdemir m., özdemir ms/ the use of the analytic hierarchy process for admission to an air traffic control department international journal of the analytic hierarchy process 79 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.503 appendix i the comparison of the criteria according to the importance scale from 1 to 9 is shown in the following form. example a 9 8 7 6  4 3 2 1 2 3 4 5 6 7 8 9 b the criterion a is five times more important than the criterion b a 9 8 7 6 5 4 3 2 1 2  4 5 6 7 8 9 b the criterion b is three times more important than the criterion a a 9 8 7 6 5 4 3 2  2 3 4 5 6 7 8 9 b the criteria a and b have equal importance paired comparisons communication 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 stress control skills communication 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 attention level skills communication 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 job interest skills communication 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 self-reliance skills stress control 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 attention level stress control 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 job interest stress control 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 self-reliance attention level 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 job interest attention level 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 self-reliance job interest 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 self-reliance verbal 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 nonverbal communication communication manner of 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 diction speaking pronunciation 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 tone of voice body language 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 appearance ijahp article: özdemir m., özdemir ms/ the use of the analytic hierarchy process for admission to an air traffic control department international journal of the analytic hierarchy process 80 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.503 appendix ii the values in the column titled “total” indicate the total score that the candidates have gained from the all criteria. l and g indicate, respectively, local and global weights of the criteria. ijahp article: özdemir m., özdemir ms/ the use of the analytic hierarchy process for admission to an air traffic control department international journal of the analytic hierarchy process 81 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.503 references al-harbi, k.m.a.s. (2001). application of the ahp in project management. international journal of project management, 19(1), 19-27. doi: https://doi.org/10.1016/s0263-7863(99)00038-1 altunoluk, t., özpeynirci, ö., kazançoğlu, y., & yılmaz, r. (2010). comparative analysis of multicriteria decision making methods for postgraduate student selection. eurasian journal of educational research, 40, 1-15. an, s., kim, g., & kang, k. (2007). a case-based reasoning cost estimating model using experience by analytic hierarchy process. building and environment, 42(2007), 25732579. doi: https://doi.org/10.1016/j.buildenv.2006.06.007 bogdanovic, d., & miletic, s. (2014). personnel evaluation and selection by multicriteria decision making method, economic computation and economic cybernetics studies and research, 48(3), 179-196. european organization for the safety of air navigation (eurocontrol). (1998). integrated task and job analysis of air traffic controllers, belgium, brussels: eurocontrol. european organization for the safety of air navigation (eurocontrol). (2010). human performance in air traffic management safety. belgium, brussels: eurocontrol. gibney, r., & shang, j. (2007). decision making in academia: a case of the dean selection process. mathematical and computer modelling, 46, 1030-1040. doi: https://doi.org/10.1016/j.mcm.2007.03.024 international civil aviation organization (icao). (2007). procedures for air navigation services air traffic management (doc 4444). canada, montreal: icao. jou, r.c., kuo, c.w., & tang, m.l. (2013). a study of job stress and turnover tendency among air traffic controllers: the mediating effects of job satisfaction, transportation research part e, 57 (2013), 95-10. doi: https://doi.org/10.1016/j.tre.2013.01.009 kasim, m., ramli, r., ibrahim, h., ghazali, m., kamal, f., & vikneswari s. (2012). prioritization of criteria in teacher-candidate selection process by a pairwise comparison method, knowledge management international conference. johor bahru, malaysia. kwong, c.k., & bai, h. (2002). a fuzzy ahp approach to the determination of importance weights of customer requirements in quality function deployment. journal of intelligent manufacturing, 13(5), 367-377. norddin, n.i., ibrahim, k., & aziz, a.h. (2012). selecting new lecturers using the analytical hierarchy process (ahp). 2012 international conference on statistics in science, business and engineering (icssbe). langkawi, malaysia. doi: 10.1109/icssbe.2012.6396585 ijahp article: özdemir m., özdemir ms/ the use of the analytic hierarchy process for admission to an air traffic control department international journal of the analytic hierarchy process 82 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.503 saaty, t.l. (2008). decision making with the analytic hierarchy process. international journal of services sciences, 1(1), 83-98. doi: 10.1504/ijssci.2008.017590 saaty, t.l. (1987). the analytic hierarchy process-what it is and how it is used. mathematical modelling, 9(3-5), 161-176. doi: https://doi.org/10.1016/02700255(87)90473-8 saaty, t.l., & sagir, m. (2009). extending the measurement of tangibles to intangibles. international journal of information technology & decision making, 8(01), 7-27. doi: https://doi.org/10.1142/s0219622009003247 swiercz, p., & ezzedeen, s. (2001). from sorcery to science: ahp, a powerful new tool for executive selection. hr. human resource planning, 24(3), 15-26. taşkın, h., üstün, ü., & deliktaş, d. (2013). fuzzy mcdm approach for oral examination in erasmus student selection process. dpü fen bilimleri enstitüsü dergisi, 32, 21-40. taylor, f., ketcham, a., & hoffman, d. (1998). personnel evaluation with ahp. management decision, 36, 679-685. doi: https://doi.org/10.1108/00251749810245336 vaidya, o.s., & kumar, s. (2006). analytic hierarchy process: an overview of applications. european journal of operational research, 169(1), 1-29. doi: https://doi.org/10.1016/j.ejor.2004.04.028 wiegmann, d.a., & shappell, s.a. (2001). human error analysis of commercial aviation accidents: application of the human factors analysis and classification system (hfacs). aviation, space, and environmental medicine, 72(11), 1006-1016. 3_amponsah_dec23_p__106-128-2 ijahp article: amponsah, c.t. / critical success factors for procurement of capital projects under public-private partnerships international journal of the analytic hierarchy process 107 vol. 3 issue 2 2011 issn 1936-6744 application of multi-criteria decision making process to determine critical success factors for procurement of capital projects under publicprivate partnerships christian tabi amponsah faculty of business and management university canada west, vancouver, bc, canada email: chris_tabi@hotmail.com abstract investigation about project success has attracted the interest of many researches and practitioners. determining the critical success factors for procurement of capital projects is a contemporary phenomenon. this paper presents the outcome of an investigation into the critical success factors in public-private-partnerships (p-p-p) for procurement of capital projects using the multi-criteria decision making process. drawing from the results of responses to a survey of 705 experts involved in p-p-p projects worldwide, the paper presents the critical success factors (csf) from a list of 47 factors, identified as contributing to the successful delivery of capital projects. the study revealed that owner satisfaction with the delivered project, adherence to schedules/budget/quality/ safety/environmental controls, and appropriate funding mechanisms were predictable factors while lack of legal encumbrances, clearly defined project mission and adequate planning and control techniques were less commonly expected factors. key words: analytic hierarchy process, critical success factors, multi-criteria decision making, infrastructure, public-private partnerships 1. introduction despite well-known research results, volumes of words written about procurement of capital projects using the public-private-partnerships (p-p-p) model, and decades of individual and collective experience managing p-p-p projects, results continue to disappoint stakeholders. in the past, research has focused only on success factors for the procurement of projects, and best practices in dealing with success in project development. a wide array of research methods have been used to determine the critical success factors of project success. some of these factors are the realization of case studies (e.g. sumner, 1999), group interviews (e.g. khandewal and miller, 1992), structured interviews (rockart and van bullen, 1986), as well as analysis of relevant literature (e.g. esteves and pastor, 2000). according to shah and siddiqui, (2002) the most frequently used method to identify success factors is the realization of a questionnaire. although the number of studies examining the relevance of csf in regard to the individual phases of the project lifecycle has increased, most studies still remain limited to the sole identification of these csf and do not address their individual degree of relevance at all. rob typewritten text http://dx.doi.org/10.13033/ijahp.v3i2.121 ijahp article: amponsah, c.t. / critical success factors for procurement of capital projects under public-private partnerships international journal of the analytic hierarchy process 108 vol. 3 issue 2 2011 issn 1936-6744 two different approaches can be found from comprehensive studies addressing both identification and the relevance. the approach implemented by pinto and prescott (1988), for instance, is based upon the same set of csf at all times, while examining their individual degree of criticality along the different project phases. in contrast, other studies have chosen to define different sets of csf for each project phase. although differently executed, both concepts generally tend to refer to the same set of csf. presently, a pragmatic study is required to identify critical success factors for procurement of projects by incorporating a comprehensive approach. the multi-criteria decision making process (mcdm) has been found to be significant in decision making when an extensive number of factors are involved. the mcdm as a methodology provides a precise language regarding the components of the problem and the relationship between these components. one such method of the mcdm is the analytic hierarchy process (ahp) developed by professor thomas saaty in the 1970s. this paper presents the outcome of an investigation into the csf in p-p-p for procurement of capital projects using multi-criteria decision making process. the subject of csf was selected for detailed research because it represents areas or functions where events and actions occur to ensure successful competitive performance for an organization. 2. background since the end of world war ii, economic development has become a key policy concern worldwide. changes in the social economic structure over the past several decades have led to radical responses toward the economic development policies of many governments. policy officials at different levels of government have discovered that greater economic development and sustainability might be reached if a more active approach toward attracting investment is made. rather than passively waiting for business interests to seize on new incentives in the tax code, public officials proactively court businesses in an attempt to secure contractual agreements. this policy has come to be known as the public-private partnership, abbreviated variously as; p-p-p, 3ps, p3, p3. p-p-p assumes that the public and private sectors can cooperate and create new value and benefit for all concerned parties. whereas the traditional approach is to procure separate project roles for the public and private sectors, p-p-ps combine the forces of public and private sectors to create added value for projects. proponents of p-p-p claim that the public and private sectors benefit immensely under the p-p-p approach (pinto & slevin, 1987). the successful procurement of capital projects is very important to many developers because large sums of investment capital are expended annually on developmental projects. enough information is required by developers to be able to determine the right procedures required to make the best procurement processes. the capital projects industry (i.e. the industry that executes the planning, engineering, procurement, construction and operation of predominantly large-scale buildings, plants, facilities and infrastructure) greatly lags behind other sectors in exploiting technological advances (fiatech, 2006). the capital project industry is faced with huge disparities in business practices and application of technology. it is evident from looking at different companies in this industry and across their supply chain that many different tools and technologies are used (fiatech, 2006). known emerging procurement strategies such as public finance initiative (pfi) and public-private partnership (ppp) are being used, but they are not a panacea (confederation of british industry, 1996). a search for proper strategies ijahp article: amponsah, c.t. / critical success factors for procurement of capital projects under public-private partnerships international journal of the analytic hierarchy process 109 vol. 3 issue 2 2011 issn 1936-6744 which can contribute to successful procurement and implementation of capital projects (confederation of british industry, 1996; private finance panel, 1996) is required. public-private partnerships (p-p-p) are increasingly being adopted as a procurement strategy for delivery of capital (and other privatized) projects all over the world. several factors have been mentioned as making contributions to the success or failure of capital projects in terms of their objectives. this research explores the critical factors that can contribute to successful procurement and implementation of capital projects by developing, administering and analyzing results of a p-p-p survey. the findings can influence policy development toward p-p-ps, and the way in which those involved go about developing public-private partnerships. identifying contributing factors to successful procurement of capital projects is one of the many management practices that facilitate corporate success. attempts to study problems related to p-p-p procurement have discovered issues such as high cost in tendering, complex negotiation, cost restraints on innovation, and differing or conflicting objectives among the project stakeholders. despite this, many p-p-p projects are regarded as successful. many studies have developed differing lists of success factors (sf) for p-p-p projects, and similarities occur among them (hardcastle, edwards, akintoye & li, 2004). less information exists about the relative importance of csf associated with p-p-p projects in many nations. different types of public-private partnerships are being practiced in worldwide infrastructure development with diverse results and a variety of problems are being encountered. this study sought to identify the critical factors that can successfully facilitate procurement of capital projects under the p-p-p projects. by identifying the csf the study thus seeks to contribute to the knowledge base in the construction industry by strengthening the theoretical understanding of methodologies used for procurement and development of p-p-p projects and analyzing the processes using relevant theories. second, the study provides a new methodology that enables a rigorous examination of capital projects under p-p-p concepts. third, the study provides evidence as to whether p-p-p projects deliver valuefor-money for the client and profit for organizations by way of improvements in economic efficiency reached. fourth, the study identifies the forces in the p-p-p development process that either encourage or discourage implementing capital projects. fifth, it helps to develop potential solutions to the problematic issues identified in the procurement of p-p-p projects, and clarifies the future research agenda on p-p-p. in addition, the research findings can help a wide range of individuals, from people concerned with the state of the public services to those involved in public spending whose duties include the delivery of capital projects. as pp-p type of procurement is rapidly gaining popularity around the world, the findings can be of value in countries that are still experimenting with this procurement processes. 3. critical success factors the term, critical success factors (csf), was first used in the context of information systems and project management by rockart (1982). his definition states, “those few key areas of activity in which favorable results are absolutely necessary for a particular manager to reach his or her own goals...those limited number of areas where ‘things must go right’” (rockart, 1982, p.2). since then, some publications have cited the csf methodology in research and the definitions, following that of rockart. boynton and zmund (1984) define critical success factors as “those few things that must go well to ijahp article: amponsah, c.t. / critical success factors for procurement of capital projects under public-private partnerships international journal of the analytic hierarchy process 110 vol. 3 issue 2 2011 issn 1936-6744 ensure success for a manager or an organization” (p. 17). sanvido, parfitt, grobler, guvenis and coyle (1992) also define critical success factors as “those factors predicting success on projects and events or circumstances that require the special attention of managers” (p. 99). tiong, khim-teck, yeo and mccarthy (1992) identify csf as “those characteristics...that when properly sustained and managed have a significant impact upon winning...those things that must be given special and continued attention and must go well to increase the...chances of success” (p. 220). according to a. j. smith and c. walker (1994) csfs are “those factors in which success is necessary in order that each of the major project participants in a...project has the maximum chance of achieving the goals” (p. 247). lim and mohamed (1999) wrote that csfs are “those [things] needed to produce the desired deliverables for the customer” (p. 244). csf measure end results. ghosh, liang, meng and chan (2001) see csf as key success factors which are critical for excellent performance of the company, rather than just survival. of interest is the pattern formed by each of the above definitions. three clear sections to each definition become apparent. for example, in rockart’s quote the three sections of the definition are, “those few key areas”, “favorable results are absolutely necessary”, and “to reach his or her own goals”. using this method to segregate and analyze the identified definitions, a new definition of critical success factors related to p-p-p projects is proposed. csf are those few factors which, when judiciously applied to a p-p-p scenario, can lead to, and/or actively contribute to, a profitable conclusion for one or more of the parties involved” (owen 1997, p. 55). this definition is used throughout this paper to describe critical success factors. 4. methodology the multi-criteria decision method (mcdm) was leveraged for the study because of its significance in decision making when an extensive number of factors are involved. mcdm as a methodology has a precise language regarding the components of the problem and the relationship between them. one such method of the mcdm is the analytic hierarchy process (ahp) developed by professor thomas saaty in 1970. since its development, the ahp has been successfully applied to solve a wide range of multicriteria decision making problems. some areas where ahp has been applied are: location analysis (min, 1994), resource allocation (e.w.l. cheng & li, 2001; ramanathan & ganesh, 1995), outsourcing (udo, 2000), and evaluation (c.h. cheng, 1997; chin, chui, & tummala, 1999; davis & williams, 1994; liang, 2003), and ahp application review (vaidya, o., s., & kumar, s, 2006). the ahp involves four steps: 1. constructing a decision hierarchy by breaking down the decision problem into a hierarchy of inter-related elements 2. performing pair-wise comparisons of the inter-related elements 3. estimating the weights of the decision elements by using the eigen -value method 4. aggregating the relative weights of the decision elements to provide a set of ratings for the decision alternatives (canada, sullivan, and white, 1996) ijahp article: amponsah, c.t. / critical success factors for procurement of capital projects under public-private partnerships international journal of the analytic hierarchy process 111 vol. 3 issue 2 2011 issn 1936-6744 three major principles of analytic thought associated with ahp are the construction of hierarchy, establishment of priorities and logical consistency (saaty, 1990). one of the software packages available that incorporates the ideas and methodology of the ahp is expert choice (ec). it is "designed for the analysis, synthesis, and justification of complex decisions and evaluations" and can be used in either individual or group situations (quick start guide and tutorials, 2001, p. 6). all necessary steps involved in the ahp are captured in the ec software. the ahp and ec software engage decision makers in structuring a decision into smaller parts, proceeding from the goal to objectives to sub-objectives down to the alternative courses of action. decision makers then make simple pair-wise comparison judgments throughout the hierarchy to arrive at priorities for the alternatives. the decision problem may involve social, political, technical, and economic factors. the ahp helps people cope with the intuitive, the rational and the irrational, and with risk and uncertainty in complex settings. it can be used to predict likely outcomes, plan projected and desired futures, facilitate group decision making, exercise control over changes in the decision making system, allocate resources, select alternatives, do cost/benefit comparisons, evaluate employees and allocate wage increases. expert choice is intuitive, graphically based and structured in a user-friendly fashion so as to be valuable for conceptual and analytical thinkers, novices and category experts. because the criteria are presented in a hierarchical structure, decision makers are able to drill down to their level of expertise, and apply judgments to the objectives deemed important to achieving their goals. at the end of the process, decision makers are fully aware of how and why the decision was made and have results that are easy to communicate and are actionable. for this study, seven hundred and five participants were enlisted from around the world and invited to participate in a survey. these participants included 267 from canada, 175 from us, 82 from uk, 48 from australia, 47 from the middle east, 33 from hong kong, 18 from africa, 18 from new zealand, 10 from china, and 7 from germany. the 705 experts were comprised of owners, project managers, consultants/contractors, financiers, and operators for procurement of capital projects. using these experts, a model based on the analytical hierarchy process was developed to investigate the csf. the hierarchical model developed was used for detailed analysis of the findings, using software based on the analytical hierarchy process (ahp). the analysis helped to point out the csf considered by each project participant in the construction industry. agreement between different project participants has been calculated using spearman’s and kendall’s technique. a pair-wise comparison was performed on 47 success factors derived from existing literature to determine the csf. out of the 705 experts that were recruited world-wide, 140 agreed to participate in the survey. ninety-three participants responded to the survey invitation producing a response rate of 70.0%. since this is an exploratory study, the researcher gathered data only from experts in the construction industry who were involved in capital projects procurement and who have experience with public-private partnerships. this is called judgment sampling, a type of purposive sampling. according ijahp article: amponsah, c.t. / critical success factors for procurement of capital projects under public-private partnerships international journal of the analytic hierarchy process 112 vol. 3 issue 2 2011 issn 1936-6744 to cooper and schindler (2003), “judgment sampling occurs when a researcher selects sample members to conform to some criterion” (p. 201). according to cooper and schindler (2006), research questions are the choices that best state the purpose of the research study. a number of questions were posed for this research with the main research question being, “what are the csf for procurement of capital projects using public-private partnership arrangements?” more specific questions such as those enumerated below were asked in light of the issues raised in order to provide the information needed to make decisions: 1. what are the critical success factors for delivering p-p-p projects? 2. how can critical success factors which are particular to the successful realization and delivery of p-p-p projects be identified? the following investigative questions were formulated in order to reach a satisfactory conclusion about the research question: 1. what are the factors that contribute to the successful delivery of p-p-p projects? 2. how can critical issues be identified to provide successful delivery of projects under the pp-p projects? 3. how can the factors that contribute to the successful delivery of p-p-p projects be assessed? 4. what factors contribute to the failure of the p-p-p projects? 5. what factors have been applied to the delivery of p-p-p projects and have contributed profitably to one or more of the parties involved? 6. how can the transfer of risk be reduced to the private sector in p-p-p project delivery? 7. how can value-for-money be provided to the taxpayer for the contributions made in p-p-p project delivery? 8. what scheme can be applied to ensure the successful delivery of projects under the p-p-p arrangement? in addition to the questions indicated above, the following hypotheses were tested; ho: the owners, project managers, consultants/contractors, financiers and operators are mutually independent in the ranking of major portion of the success factors. h1: the owners, project managers, consultants/contractors, financiers and operators are not mutually independent in the ranking of major portion of the success factors. a literature review shows earlier studies focused on the csf for different project objectives. for example, asif (2003), concluded after a pair-wise comparison and analysis that adequate planning, and control technique, owner satisfaction with the delivered project and clearly defined project mission, and objective and scope are the most important csf. similar conclusions have been identified in the literature survey; however asif (2003) proposed that further research be carried out for specific project delivery such as public-private partnership and project finance initiatives to provide a ijahp article: amponsah, c.t. / critical success factors for procurement of capital projects under public-private partnerships international journal of the analytic hierarchy process 113 vol. 3 issue 2 2011 issn 1936-6744 better understanding of the developing construction environment and the ability to consistently achieve outstanding project outcomes. in other research, factor analysis was used to identify factor groupings that can be used to represent relationships among sets of many inter-related variables (kleinbaum, kupper, & muller, 1988; norusis, 1992). saaty and vargas (1991) identify two philosophical foundations for grouping success-related factors under separate sub-hierarchies. first, factors of similar nature should logically be grouped into one cluster to facilitate pair-wise comparison during the survey. second, it is known that an individual cannot simultaneously compare more than 7± 2 elements with satisfactory consistency, and hence hierarchical decomposition is desirable (saaty & vargas, 1991). hierarchical decomposition uses a simple but powerful notation by first dividing the system into top-level subsystems. next, every top-level sub-system is divided into second-level sub-systems, and the sub-systems are identified with a hierarchical numbering. the technique has been applied in listing success factors in this study to explore the groupings that might exist among the success factors. morledge and owen (1998) used six principal factors to group success factors for the investigations on csf in public finance initiatives and the principal factors groupings have been adapted for this study. based on the criterion above, the 47 factors identified in the literature review were grouped into six categories: project participants, effective procurement, project implementation/characteristics, government guarantee, favorable economic conditions and available financial market. a questionnaire developed from the research questions indicated above was then sent to participants requesting that they identify, from a list, those factors which they agreed were critical in procurement of p-p-p projects. yin defines the case study research method as an empirical inquiry that investigates a contemporary phenomenon within its real-life context when the boundaries between phenomenon and context are not clearly evident and in which multiple sources of evidence are used (yin, 1984, p. 23). a case study of a typically procured public-private partnership capital project was used as the basis of the research. the sea-to-sky road project linking vancouver to whistler in the province of british columbia, canada, was used for this study. the $600-million capital project involved upgrading the highway to increase its safety, reliability and capacity for the 2010 winter olympic games. the research methodology constructively distributed into various phases of the research program is shown in figure 1. ijahp article: amponsah, c.t. / critical success factors for procurement of capital projects under public-private partnerships international journal of the analytic hierarchy process 114 vol. 3 issue 2 2011 issn 1936-6744 figure 1 methodology flowchart to comply with the requirement of the ec software, the success factors identified for this study were decomposed into a hierarchy as shown in figure 2 with four levels of hierarchy. at the top (level 1) of the hierarchy is the goal or overall objective of the research i.e. success factors. level 2 of the hierarchy shows the principal factor groupings of the success factors. the major factors are at the third level of the hierarchy followed by the success related factors or the alternatives at the bottom or last (level 4) of the hierarchy. the model as developed in expert choice (ec) software is a multi-objective decision support tool based on the analytical hierarchy process (ahp), a powerful and comprehensive methodology designed to facilitate sound decision making by using both empirical data as well as subjective judgments of the decision-maker (www.expertchoice.com). development of a list of success factors from literature review development of a comprehensive questionnaire and questionnaire testing data collection through interviews data analysis conclusion and recommendations ijahp article: amponsah, c.t. / critical success factors for procurement of capital projects under public-private partnerships international journal of the analytic hierarchy process 115 vol. 3 issue 2 2011 issn 1936-6744 figure 2 hierarchy of success factors (csf # 1 to csf # 24) level 1 level 2 level 3 level 4 ijahp article: amponsah, c.t. / critical success factors for procurement of capital projects under public-private partnerships international journal of the analytic hierarchy process 116 vol. 3 issue 2 2011 issn 1936-6744 figure 2 (cont’d) hierarchy of success factors (csf # 25 to csf # 47) level 1 level 2 level 3 level 4 ijahp article: amponsah, c.t. / critical success factors for procurement of capital projects under public-private partnerships international journal of the analytic hierarchy process 117 vol. 3 issue 2 2011 issn 1936-6744 5. data collection, analysis and results survey data were collected from september 01, 2009 to november 20, 2009. the data were collected through interviews and were separated by the major factors and screened for different project participants. the target population was experts in the construction industry who were involved in procurement of capital projects using the public-private partnership arrangement. at least twenty (20) experts from each of the identified groups (owners, project managers, consultants/ contractors, financiers and operators) were recruited world-wide in the survey process. the analytical hierarchy process (ahp) was used for detailed analysis of the data in the expect choice theory software. the analysis helped to point out the critical csf considered by each project participant for the construction industry. agreement between different project participants was calculated using spearman’s and kendall’s technique. in order to provide a definite measure of the primary objectives for the research, it was necessary to collect the data by conducting surveys. semi-structured interviews and electronic self-administered questionnaires were used. personal semi-structured interviews have the advantage of being more formal than “unstructured” interviews and yet are more flexible than structured interviews. they are constructed using specific topics and both open and closed-ended questions. the purpose of conducting the semistructured interviews was to “scope” the experience of the target groups for capital project p-p-p procurement in order to ascertain the relative issues involved. the topic areas for the semi-structured interviews were sent to the interviewees after they had initially been contacted by email to confirm their interest and willingness to be involved in the research. due to the disparate geographic locations of participants and the limited time available for this study, it was difficult to conduct face-to-face interviews with the experts. consequently, a “tele-interview” was held as required and the responses were coded and recorded on a blank copy of the topic document by the interviewer. an introductory note explaining the objectives of the research and a question about their interest in participating in the research was sent to the selected groups through email. meetings were arranged with 5 members of project group of the sea-to-sky project who showed interest in the study. the face-to-face method of qualitative study was explored with the sea-to-sky project as a case study. a case study yields deep narrow results (fellows & liu, 1997). the case study project in question served to test the validity of the csf identified from the related literature, and the established framework as a sound foundation applicable to p-p-p project. yin (1984) notes that the single case study method is appropriately applied where the case in question represents an extreme or unique case or if the situation has not previously been selected as the most appropriate means for the research reported in this paper. evidence was collected for the case study through the review of project documents from the centric project web site and an informal three-stage semi-structured interview process with the participants. brief definitions of the factors were developed and taken to all the interviews to help the experts better understand the meaning of the factors. the interviewer also used the contributions of the participants in refining the definitions of the success factors. ijahp article: amponsah, c.t. / critical success factors for procurement of capital projects under public-private partnerships international journal of the analytic hierarchy process 118 vol. 3 issue 2 2011 issn 1936-6744 the factors considered for the research are, the major factors at the third level of the hierarchy followed by the success related factors or the alternatives at the bottom or last (level 4) of the hierarchy. figure 3 shows the hierarchy model in expert choice 11.5 software. figure 3 hierarchy model developed in expert choice, 11.5 software figure 4 shows an example of the pair-wise comparison data grid in which factors for project characteristics are being compared for the cost objective. weights were assigned for the pair-wise comparison from the 1-9 scale scale and varied from 1 to 5. in the questionnaire at the top of figure 4 judgments marked in the top row indicated the row element for the pair in the matrix was dominant, judgments marked in the bottom row indicated the column element was dominant and the inverse would have been entered in the matrix. ijahp article: amponsah, c.t. / critical success factors for procurement of capital projects under public-private partnerships international journal of the analytic hierarchy process 119 vol. 3 issue 2 2011 issn 1936-6744 figure 4 example of pair-wise comparison data grid built in expert choice 11.5 software the results from ec for all the participants are shown in figure 5 for the major factors. combined results of all the participants show that the project participant’s role is of great importance for the construction industry. project characteristics/implementation is followed by effective procurement and available financial market. these are followed by government guarantee and favorable economic considerations. ijahp article: amponsah, c.t. / critical success factors for procurement of capital projects under public-private partnerships international journal of the analytic hierarchy process 120 vol. 3 issue 2 2011 issn 1936-6744 figure 5 priorities with respect to goal: most important csf 5.1 validity and reliability of study although perfect consistency is hard to achieve especially when considering multiple conflicting criteria, analytical hierarchy process provided a mechanism for measuring the consistency of the decision made and allowed for revisions of the decision to reach an acceptable level of consistency. in the ahp a measure of consistency of judgment is derived by means of consistency ratio (cr). if the value of the ratio is 0.1 or less, the decision is “good”. if the value exceeds 0.1, the judgment may somehow be random and should be revised (saaty, 1990). calculating the cr starts with multiplying each entry of the pair-wise comparison matrix by the relative priority (the average) corresponding to the column, and then totaling the row entries. next, the row totals are divided by the corresponding entry from the priority vector. the average of those entries is the eigenvalue λmax. consistency index (ci) was measured using the formula: ci = (λmax – n) (n-1), where n is the number of elements (factors) being compared in the matrix. the ci was then divided by its random index (ri) to get the consistency ratio, which indicated a measure of how much variation is allowed. spearman correlation (rs) was primarily used to assess the correlation of the factors. spearman r is the regular pearson product-moment correlation coefficient (pearson r); that is in terms of the proportion of variability accounted for, except that spearman r is computed from ranks. spearman correlation coefficient can range from -1.00 to + 1.00. on the lower side, -1.0 represents a perfect negative correlation, +1.00 represent a perfect positive correlation and 0.00 represents a lack of correlation. the spearman correlation was used to find and compare how well any two participants agree while ignoring the third participant completely. the spearman correlation was calculated by the following formula (thondike, 1978): rs = 1 – 6 σd2/ (n3-n), where: rs = the spearman correlation d = the difference between ranking for each group of judges n = number of factors to be ranked ijahp article: amponsah, c.t. / critical success factors for procurement of capital projects under public-private partnerships international journal of the analytic hierarchy process 121 vol. 3 issue 2 2011 issn 1936-6744 to assess the degree of association or agreement among sets of rankings, the kendall coefficient of concordance was measured. the kendell coefficient of concordance (τ) is a measure of degree of association or agreement among sets of rankings. range of the coefficient of concordance is from zero to one. one indicates a perfect agreement and zero indicates no agreement. to calculate the kendell coefficient, the data was first arranged into a ‘k x n’ matrix. each row (n) represents ranks assigned by a particular judge (k) to (n) factors or aspects of a concept or problem. kendell coefficient was calculated using the following formula (thondike, 1978): ( ) 12/1 )( 2 1 2 − − = ∑ = nn rr k i i τ (1) ri = average of the ranks assigned by an individual r = average of the ranks assigned to the nth variable factor (sum of nr /1 ) k = number of judgments n = the number of aspect of a problem or factor being ranked in this study, 47. n (n2 -1) / 12 = the maximum possible squared deviations; i.e. the numerator which occurs if a perfect agreement among k set ranks and the average ranking are 1, 2, 3,……n. 5.2 level of agreement between all participants recruited for the survey in order to assess the level of agreement between participants for the study, hypotheses were designed. hypothetical testing relating to the level of agreement present between all the project participants i.e. owners, project manager, consultants/contractor, financier and operators groups were performed. using kendall coefficient of concordance (τ) depends upon the sample size (n). for values of n greater than 10, standard error of (τ) is calculated using the following equation: )1(9 )52(2 − + = nn n sr (2) a test of statistical significance is calculated using the following equation: sr z τ = (3) calculations are presented in the table 1 for (ri – r), which were used to calculate kendall’s coefficient of concordance. ijahp article: amponsah, c.t. / critical success factors for procurement of capital projects under public-private partnerships international journal of the analytic hierarchy process 122 vol. 3 issue 2 2011 issn 1936-6744 table 1 level of agreement between all participants kendall‟s data correlation ranking by mean ri success factor owner project manager consultant/ contractors financier operato r ri-r (ri-r) 2 csf # 1 owner enthusiasm 6 7 11 6 7 7.4 0.77 0.5929 csf # 2 owner commitment to establishing budget and schedules 6 9 4 6 9 6.8 0.17 0.0289 csf # 3 owner satisfaction with the delivered project 9 13 12 9 13 11.2 4.57 20.8849 csf # 4 project manager‟s competency and authority 8 10 11 8 10 9.4 2.77 7.6729 csf # 5 project manager‟s commitment to establish budget and schedule 6 8 2 6 8 6 -0.63 0.3969 csf # 6 nature of the projects managers authority 9 4 21 9 4 9.4 2.77 7.6729 csf # 7 capability of contractor/consultant‟s key persons to establish budget and schedule 8 8 16 8 8 9.6 2.97 8.8209 csf # 8 contractor/consultants team commitment to budget and schedule 6 7 2 6 7 5.6 -1.03 1.0609 csf # 9 experience of contractor/consultant‟s team in p-p-p 10 9 16 10 9 10.8 4.17 17.3889 csf # 10 transparency in procurement process 5 14 10 5 14 9.6 2.97 8.8209 csf # 11 competitive procurement process 4 7 3 4 7 5 -1.63 2.6569 csf # 12 thorough and realistic assessment of the cost and benefits 10 10 19 10 10 11.8 5.17 26.7289 csf # 13 absence of bureaucracy 6 3 10 6 3 5.6 -1.03 1.0609 csf # 14 meeting design goals 4 3 1 4 3 3 -3.63 13.1769 csf # 15 efficient pre-contract activities 9 4 14 9 4 8 1.37 1.8769 csf # 16 satisfactory budget management (profit and loss) 6 5 13 6 5 7 0.37 0.1369 csf # 17 proper design construction interface management 11 2 15 11 2 8.2 1.57 2.4649 csf # 18 effective communication throughout the project 8 7 14 8 7 8.8 2.17 4.7089 csf # 19 effective communication throughout the project 10 6 12 10 6 8.8 2.17 4.7089 csf # 20 proper coordination between project professionals 3 3 5 3 3 3.4 -3.23 10.4329 csf # 21 monitoring and feedback of project activities 7 4 17 7 4 7.8 1.17 1.3689 csf # 22 regular construction control meetings 4 6 10 4 6 6 -0.63 0.3969 ijahp article: amponsah, c.t. / critical success factors for procurement of capital projects under public-private partnerships international journal of the analytic hierarchy process 123 vol. 3 issue 2 2011 issn 1936-6744 csf # 23 design and control meetings 1 5 3 1 5 3 -3.63 13.1769 csf # 24 adherence to schedules, budget, quality, safety and environmental controls 11 4 18 11 4 9.6 2.97 8.8209 csf # 25 site limitation and location 6 2 11 6 2 5.4 -1.23 1.5129 csf # 26 favorable legal framework 2 4 3 2 4 3 -3.63 13.1769 csf # 27 project technical feasibility 6 4 18 6 4 7.6 0.97 0.9409 csf # 28 appropriate risk allocation and risk sharing 9 5 19 9 5 9.4 2.77 7.6729 csf # 29 strong private consortium 12 4 11 12 4 8.6 1.97 3.8809 csf # 30 clearly defined project mission, objective and scope definitions 9 6 15 9 6 9 2.37 5.6169 csf # 31 adequacy of plans and specifications 6 8 5 6 8 6.6 -0.03 0.0009 csf # 32 formal dispute resolution process 2 3 4 2 3 2.8 -3.83 14.6689 csf # 33 contractual motivation/incentives 4 5 21 4 5 7.8 1.17 1.3689 csf # 34 accurate initial cost estimates 9 5 12 9 5 8 1.37 1.8769 csf # 35 adequate planning and control techniques 2 6 0 2 6 3.2 -3.43 11.7649 csf # 36 minimal start-up difficulties 3 4 17 3 4 6.2 -0.43 0.1849 csf # 37 the perceive value of the project 6 6 14 6 6 7.6 0.97 0.9409 csf # 38 lack of legal encumbrances 9 3 17 9 3 8.2 1.57 2.4649 csf # 39 minimized number of public/government agencies involved 2 3 0 2 3 2 -4.63 21.4369 csf # 40 constraints imposed by end-users 3 2 20 3 2 6 -0.63 0.3969 csf # 41 government involvement in providing a guarantee 8 5 10 8 5 7.2 0.57 0.3249 csf # 42 multi-benefit objective 5 3 1 5 3 3.4 -3.23 10.4329 csf # 43 stable macroeconomic conditions 4 3 13 4 3 5.4 -1.23 1.5129 csf # 44 sound economic policy 5 3 15 5 3 6.2 -0.43 0.1849 csf # 45 availability of a suitable and adequate financial market 7 5 15 7 5 7.8 1.17 1.3689 csf # 46 appropriate funding mechanisms 8 8 15 8 8 9.4 2.77 7.6729 csf # 47 confidence in project funding agencies 12 6 9 12 6 9 2.37 5.6169 total ∑ 331.6 280.078 mean r 6.63 ijahp article: amponsah, c.t. / critical success factors for procurement of capital projects under public-private partnerships international journal of the analytic hierarchy process 124 vol. 3 issue 2 2011 issn 1936-6744 hypotheses: hypotheses are as follows: ho: the owners, project managers, consultants/contractors, financiers and operators are mutually independent in the ranking of major portion of the success factors. h1: the owners, project managers, consultants/contractors, financiers and operators are not mutually independent in the ranking of major portion of the success factors. test statistic:. the test statistics were calculated using equation 1. and values of (ri-r) are obtained from table 1. the sum (ri-r) is 331.6, the summation of (ri-r)2 is 280.078, and the mean of r is 6.63. with the value obtained, the test statistics was calculated as follows: ( ) 12/14747 )0783.280( 2 1 − = ∑ = k iτ (4) 032.0=τ decision rule: significance statistic was calculated using equations 2 and 3 as follows: )147(479 )5472(2 − + = x x sr (5) 1.0=sr z = 0.032 /0.1 z = 0.32 to accept ho, the test statistic should be less than the significance statistic; otherwise the null hypothesis is rejected. the null hypothesis is accepted since 0.1 < 0.32. at a significance level (α) of 0.05 or 95% confidence interval, it can be deduced that owners, project managers, consultants/contractors, financiers and operators do not agree on the rankings of a major portion of the success factors. one reason for this is that each group is working towards their own interest in the procurement of public-private partnership projects. 6. conclusions, implications and recommendations the main purpose of the study was to determine the critical success factors that could be used to successfully procure capital projects under p-p-p based on accumulated knowledge and judgment of experts including owners, project managers, consultants/ contractors, financiers and operators. the research helped in achieving the following objectives: ijahp article: amponsah, c.t. / critical success factors for procurement of capital projects under public-private partnerships international journal of the analytic hierarchy process 125 vol. 3 issue 2 2011 issn 1936-6744 1. investigate the csf, and delineate into factor groupings for analysis purposes 2. investigate the csf according to different project participants (each participant provided a set of most important csf based on their objectives) the findings are clearly supported by the outcome of the hypothesis testing which was conducted using the project participants on all the identified success factors.” the top ten factors that were considered the csf are listed as follows: 1. owner satisfaction with the delivered project 2. clearly defined project mission, objective and scope definitions 3. adequacy of plans and specifications 4. lack of legal encumbrances 5. appropriate funding mechanisms 6. adequate planning and control techniques 7. experience of contractor/consultant’s team in p-p-p 8. adherence to schedules, budget, quality, safety and environmental controls 9. project manager’s commitment to establish budget and schedule 10. effective communication throughout the project the factors are listed in order of importance based on the analysis of the survey response in expert choice software. the top five on the list may be considered the most critical success factors. the critical success factors which are particular to the successful realization and delivery of p-p-p project based on the responses are as follows: 1. perceived need identified with a well-defined purpose and objective 2. early identification and selection of a viable project by the consortium is critical for project delivery 3. adequate and accurate risk assessment by all parties involved, with the responsibility of managing each risk placed with the party most able to control them 4. detailed guidelines based on past experience to explain the risk accepted by each party is critical for project delivery 5. good communication between all team members through established links is a critical factor in p-p-p 6. setting of objectives by all parties, and an agreement made before a contract is signed is a critical success factor for p-p-p project 7. detailed guidelines based on past experience to explain the risk accepted by each party is a critical success factor 8. an unerring commitment from public sector management and civil servants with an appreciation for the private sector 9. a multi-disciplinary team with an experienced, skilled leader 10. client with sufficient financial strength and ability to pay for all services being provided for the duration of the contract from a practical and professional standpoint, these findings should influence policy development towards p-p-ps and the manner in which partners go about the development of p-p-p projects. since the utmost factor is owner satisfaction with the delivered project, this has important implications for the development of p-p-p at the conceptual stage. ijahp article: amponsah, c.t. / critical success factors for procurement of capital projects under public-private partnerships international journal of the analytic hierarchy process 126 vol. 3 issue 2 2011 issn 1936-6744 parameters that are of ultimate concern to the owner must be given the greatest attention. as long as the utmost factor is linked with the perceived need identified and there is a well-defined purpose and objective for the project, the project is likely to be successful. the study involved experts in the construction industry belonging to five major groups, namely owners, project managers, consultants/contractors, financiers and operators, with at least ten years’ experience in the construction industry. one of the issues that emerged was that the greatest proportion of participants was from the consultant/contractor category followed closely by the financiers. some of the issues that emerged from the findings relate specifically to the structuring of the financing and legislation for the procurement of p-p-p. as a result, construction management organizations can use those csf to evaluate whether or not they should embark on development of capital projects. this research study has focused on the procurement of capital projects for public-private partnerships in general. further research can be carried out on different projects with specific delivery systems, e.g. success factors for design-built, built operate transfer, construction management, design-management, and research and development projects. by collecting data from different project delivery systems, researchers may unearth factors that are unique to the specific project delivery. the factor groupings used for the study include project participants, effective procurements, project implementation/characteristic, government guarantee, favorable economic conditions and available financial markets. in future investigations it might be possible to use different project objectives grouping the factors into project objectives including cost, time and quality. the methodological approach to this research was the analytical hierarchy process based on the multi-critical decision method. other multivariate techniques like regression can be used in further research using only the top few factors to gain further insight into critical success factors for procurement of capital projects. ijahp article: amponsah, c.t. / critical success factors for procurement of capital projects under public-private partnerships international journal of the analytic hierarchy process 127 vol. 3 issue 2 2011 issn 1936-6744 references asif, m.a. (2003). critical success factors for different project objectives. unpublished master’s thesis, king fahd university of petroleum and minerals, dhahran, saudi arabia. boynton, a. c., & zmund, r. w. (1984). an assessment of critical success factors. sloan management review, 25(4), 17-27. canada, j. r., sullivan w.g., & white j. a. (1996). capital investment analysis for engineering and management, 2nd edition, new jersey: prentice hall. cheng, e.w. l., & li, h. (2001). information priority-setting for better resource allocation using analytic hierarchy process. information management and computer security, 9(2), 61–70. cheng, c.h. (1997). evaluating naval teaching missile systems by fuzzy ahp based on the grade value of membership function. european journal of operational research, 96(2), 342–350. chin, k.s., chiu, s., & tummala, v.m.r. (1999). an evaluation of success factors using the ahp to implement iso 14001-based ems. international journal of quality and reliability management, 16(4), 341–361. confederation of british industry. (1996). private skills in public service tuning the project finance initiative. london: cbi. cooper, d.r., & schindler, p.s. (2006). business research methods, (9th ed.). boston, new york: mcgraw-hill. davis, l., & williams, g. (1994). evaluating and selecting simulation software using the analytic hierarchy process. integrated manufacturing system, 5 (1), 23–32. esteves, j. and pastor, j. (1999). an erp lifecycle-based research agenda. paper presented at the first international workshop on enterprise management resource and planning systems emrps, venice, italy, 359-371. expert choice inc (2001). advanced decision support software quick start guide and tutorials. expert choice inc., pennsylvania. fellows, r., & liu, a. (1997). research methods for construction. london: blackwell science. fiatech. (2006). capital projects technology roadmap. retrieved january 10, 2008, from http://www. fiatech.org/projects /roadmap/cptri.htm. ijahp article: amponsah, c.t. / critical success factors for procurement of capital projects under public-private partnerships international journal of the analytic hierarchy process 128 vol. 3 issue 2 2011 issn 1936-6744 ghosh, b. c., liang, t. w., meng, t. t., & chan, b. (2001). the key success factors, distinction capabilities and strategic thrusts of top smes in singapore. journal of business research, 51, 209-221. hardcastle, c., edwards, p. j., akintoye, a., & li, b. (2005). critical success factors for ppp/pfi projects in the uk construction industry. journal of construction management and economics, 23(5), 459 – 471. khandewal, v. and miller, j. (1992). information system study. in: opportunity management program, ibm corporation, new york. kleinbaum, d. g., kupper, l.l., & muller, k. e. (1988). applied regression analysis and other multivariable methods. boston: pws-kent. liang, w.y. (2003). the analytic hierarchy process in project evaluation: an r & d case study in taiwan. benchmarking, 10(5), 445–456. lim, c. s., & mohamed, m. z. (1999). criteria of project success and exploratory reexamination. international journal of project management, 17(4), 243-248. min, h. (1994). location analysis of international consolidation terminals using the analytic hierarchy process. journal of business logistics, 15(2), 25–44. morledge, r., & owen, k. (998). critical success factors in pfi projects. in hughes, w (ed.), 14th annual arcom conference, 9-11, university of reading. association of researchers in construction management, 2, 565-74. norusis, m.j. (1992). spss for windows, professional statistics, release 5. chicago: spss inc. owen, k. j. (1997). critical success factors of pfi. unpublished doctoral dissertation. the nottingham trent university, uk.   pinto, j.k., & slevin, d.p. (1987). critical factors in successful project implementation. ieee transactions on engineering management, 34(1), 22-27.   pinto, j., & prescott, j. (1988). variations in critical success factors over the stages in the project lifecycle. journal of management, 14(1), 5-18. private finance panel. (1996). private finance initiative guidelines for smoothing the procurement process. her majestic treasury and the private finance panel. ramanathan, r., & ganesh, l.s. (1995). using ahp for resource allocation problems. european journal of operational research, 80, 410–417. rockart, j. f. (1982). the changing role of the information system executive: a critical success factor perspective. sloan management review, 24 (1), 3-13. ijahp article: amponsah, c.t. / critical success factors for procurement of capital projects under public-private partnerships international journal of the analytic hierarchy process 129 vol. 3 issue 2 2011 issn 1936-6744 rockart, j. and van bullen, c. (1986). a primer on critical success factors. in rockart, j. and van bullen, c. (ed.), the rise of management computing. homewood: irwin. saaty, t. l., & vargas, l. g. (1991). prediction, projection and forecasting. boston: kluwer academic publishers. saaty, t. l. (1990). the analytical hierarchy process: planning, priority setting and resources allocation. london: mcgraw-hill. sanvido, v., parfitt, k., grobler, f., guveris m., & coyle, m. (1992). critical success factors for construction projects. journal of construction engineering and management, 11 (1), 94-111. shah, m. and siddiqui, f. (2002). a survey of research methods used to investigate critical factors. paper presented at the european conference on research methodology for business and management studies, reading, uk..     smith, a. j., & walker, c. t. (1994). bot: critical factors for success. journal of construction engineering and management, 120 (9), 247-254.   sumner, m. (1999). critical success factors in enterprise wide information management systems projects. paper presented at the americas conference on information systems, milwaukee, usa. thondike, r. m. (1978). correlation procedures for research. new york: gardner press inc. tiong, r. l. k., khim-teck, y., & mccarthy, s. c. (1992). critical success factors in winning bot contracts. journal of construction engineering and management, 118(2), 217-228. udo, g.g. (2000). using analytic hierarchy process to analyze the information technology outsourcing decision. industrial management and data systems, 100(9), 421– 429. vaidya, o., & s. kumar, s. (2006). analytic hierarchy process: an overview of applications, european journal of operational research, 169, 1–29. yin, r. k. (1994) .case study research: design and methods (2nd ed.). thousand oaks, ca: sage publications. ijahp essay: kahraman, cebi, cevik onar, öztayşi / recent developments on fuzzy ahp and anp under vague and imprecise data: evidence from infus conferences international journal of the analytic hierarchy process 1 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1033 recent developments on fuzzy ahp and anp under vague and imprecise data: evidence from infus conferences cengiz kahraman 1 , selçuk çebi 2 , sezi çevik onar 1 , başar öztayşi 1 1 istanbul technical university, department of industrial engineering, 34367 macka, istanbul, türkiye 2 yildiz technical university, department of industrial engineering, 34349, yildiz, istanbul, türkiye abstract when a decision maker is not sure about assigning a value for a pairwise comparison, he/she should assign a fuzzy value such as ‘between 3 and 4’ or ‘around 5’. these expressions can be represented by fuzzy numbers, which make the use of fuzzy ahp methods possible. each new fuzzy set extension has caused a new fuzzy ahp method to be developed by researchers. the infus conference is a scientific arena where researchers can present their new fuzzy ahp extensions and benefit from the discussions during the sessions. this paper summarizes the latest fuzzy ahp/anp extensions that have been developed in the recent years at infus conferences. keywords: fuzzy extensions of ahp; intuitionistic fuzzy ahp; picture fuzzy ahp; neutrosophic ahp; spherical fuzzy ahp; pythagorean fuzzy ahp; q-rung orthopair fuzzy ahp 1. introduction the ahp and anp are the most popular multi-attribute decision making methods all over the world. fuzzy set theory has caused almost all multi-attribute methods to be extended under vague and imprecise data where exact and complete data are hard to obtain. recent extensions of ordinary fuzzy sets such as intuitionistic fuzzy sets, picture fuzzy sets, and neutrosophic sets provided the opportunity for a new extension of each method to be developed. thus, intuitionistic fuzzy ahp, neutrosophic ahp, spherical fuzzy ahp, picture fuzzy ahp, pythagorean fuzzy ahp, fermatean fuzzy ahp, q-rung orthopair fuzzy ahp have appeared in the literature. sub-extensions of each of these based on the used fuzzy numbers have also been developed such as interval-valued intuitionistic fuzzy ahp, triangular intuitionistic fuzzy ahp, trapezoidal intuitionistic fuzzy ahp and single-valued intuitionistic fuzzy ahp. infus is an acronym for intelligent and fuzzy systems. it is an international research forum with the goal of advancing the foundations and applications of intelligent and fuzzy systems, computational intelligence, and soft computing for applied research in general and for complex engineering and decision support systems. the principal mission of infus is bridging the gap between fuzzy and ijahp essay: kahraman, cebi, cevik onar, öztayşi / recent developments on fuzzy ahp and anp under vague and imprecise data: evidence from infus conferences international journal of the analytic hierarchy process 2 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1033 intelligent systems and real complex systems via joint research between universities and international research institutions, encouraging interdisciplinary research and bringing multidisciplinary researchers together. infus is an arena for the newly developed fuzzy extensions of the ahp to be presented and discussed. many ahp papers have been submitted to infus and reviewed by a peer review process each year. these papers have been presented, discussed and then published in springer proceedings having scopus and wos indexing. the anp has rarely been used in fuzzy set approaches since super matrices and limit matrices are not well-defined issues and it is hard to obtain the corresponding equivalent fuzzy operations. hence, there are almost no papers on fuzzy anp in the following list of fuzzy ahp/anp reviews. the rest of the paper is organized as follows. section 2 presents ahp/anp papers from infus 2019. section 3 presents ahp/anp papers from infus 2020. section 4 summarizes ahp/anp papers from infus 2021. section 5 summarizes ahp/anp papers from infus 2022. finally, the paper is concluded in section 6. 2. infus 2019 ahp/anp papers at infus 2019, a total of 17 papers on fuzzy ahp were presented. these papers developed some new fuzzy extensions of the ahp and applied them to popular problems such as construction company selection, home health care vehicle routing, social suitable development factor evaluation, airline flight carbon election performance assessment, suitability evaluation of blockchain technologies and others. among these ahp-based papers, many studies combined fuzzy ahp with other multi-criteria decision making methods. one of the most utilized approaches was to combine fuzzy ahp with fuzzy topsis to solve various types of decision making problems. kabayir et al. (2019) developed a framework for construction company selection by combining the fuzzy ahp and fuzzy topsis methods. karasan et al. (2019) used a combined ahp and topsis method for evaluating the challenges of sustainable cities. the fuzziness was represented by neutrosophic fuzzy sets. it is not only topsis, but also the copras method that has been widely combined with fuzzy ahp. seker et al. integrated the fuzzy ahp and fuzzy copras methods for solar power plant location selection. in this study, intuitionistic fuzzy sets were used. montazer et al. (2019) combined the ahp and copras methods to select a strategic partner for an airline. at infus 2019, many papers used ahp modifications where ahp is modeled with the extensions of fuzzy sets to represent fuzziness. the fuzzy sets where both membership and non-membership can be defined separately, such as intuitionistic fuzzy sets, pythagorean fuzzy sets and spherical fuzzy sets are the most commonly used extensions of fuzzy sets utilized in ahp modification. peker et al. (2019) evaluated home health care vehicle routing methods using intuitionistic fuzzy ahp. ozkan and aydin (2019) introduced a supplier selection model that combines intuitionistic fuzzy ahp and goal programming. karasan et al. (2019a) used the pythagorean fuzzy ahp method for clean energy technology selection. karasan et al. (2019b) performed a risk analysis of autonomous vehicle driving systems using pythagorean fuzzy ahp. gundogdu and kahraman (2019) modified the ahp by ijahp essay: kahraman, cebi, cevik onar, öztayşi / recent developments on fuzzy ahp and anp under vague and imprecise data: evidence from infus conferences international journal of the analytic hierarchy process 3 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1033 using spherical fuzzy sets for industrial robot selection. hesitant fuzzy sets, which enable multiple evaluations to be assigned to represent the hesitancy in decision making, have also been used to modify the ahp. hesitant ahp methods usually use hesitant linguistic term sets in which the linguistic expressions such as “more than”, “between”, or “at least” can be modelled. öztayşi et al. (2019a) created an innovative teaching feedback system design using hesitant fuzzy ahp. öztayşi et al. (2019b) weighted performance indicators of debt collection offices using hesitant fuzzy ahp. other fuzzy extensions such as z-numbers and neutrosophic sets have also been used for the extension of ahp. yildiz and kahraman (2019) used a z-numbers-based ahp for evaluating social sustainable development factors. the ahp is used with other methods such as qfd to define the criteria weights to satisfy customer needs. seker et al. (2019) used a combined fuzzy ahp and qfd methodology for retail chain evaluation. cinar and cebi (2019) assessed risks of the mining industry using a fuzzy logic-based combined qfd and ahp approach. at infus 2019, even though fuzzy anp is more complex and has some computational indistinctness, it was used as well as the ahp. three papers utilized fuzzy anp to evaluate various multi-criteria decision-making methods. maden (2019) used fuzzy anp to evaluate the suitability of blockchain–based systems. this approach was applied to evaluate the systems in a logistics company. aydogan and bereketli zafeirakopoulos (2019) evaluated the leg base airline flight carbon emission performance using fuzzy anp. kılıç and kabak (2019) analyzed the relationship between human development and competitiveness using fuzzy anp and data envelopment analysis (dea). 3. infus 2020 ahp/anp papers at infus 2020, a total of 16 papers on fuzzy ahp were presented. these papers developed some new fuzzy extensions of ahp and applied them to some popular problems such as assessment of freight transportation alternatives, beacon technology selection, food waste management, sustainable supplier selection and others. jaller and otay (2020) proposed ahp and topsis based on spherical fuzzy sets to evaluate the sustainable vehicle technology alternatives for freight transportation. the authors evaluated the alternatives using the following five criteria: financial; business and market-related; environmental and legal; maintenance and repair availability; and safety and vehicle performance factors. twenty-one sub-criteria were also evaluated. öztayşi et al. (2020) tackled a beacon technology selection problem which enables applications providing location-based services such as notifications, tracking, and navigation via smart mobile devices. one of the most commonly adopted business models based on beacon technology is location-based advertisements. in their paper, target customers were defined, and the most appropriate advertisement was sent to the customer based on his/her location. while the conversion rates of this model are high, there is a need for a dynamic pricing model that calculates the value of an advertisement to an advertiser in a given location. they proposed an advertisement pricing model using the spherical fuzzy ahp scoring method with four main criteria, namely the content quality, daily potential, frequency of communication and user ijahp essay: kahraman, cebi, cevik onar, öztayşi / recent developments on fuzzy ahp and anp under vague and imprecise data: evidence from infus conferences international journal of the analytic hierarchy process 4 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1033 interest. buyuk and temur (2020) proposed a spherical fuzzy ahp method for food waste management since this decision involves multi-faceted dimensions simultaneously under a linguistic and uncertain environment. a numerical example was conducted in order to indicate how the model can be used for food waste management. with the help of the proposed model, a supportive tool for decision makers was created in related institutions such as municipalities. unal and temur (2020) used the spherical fuzzy ahp method for the selection of sustainable suppliers. the proposed method takes into consideration the following four main criteria: traditional, economic, social and environmental. interviews were conducted with three experts who have valuable experiences in this field of study. otay and yildiz (2020) studied a cloud service provider selection problem under a fuzzy environment. in the proposed model, a hierarchical structure with six criteria (financial, performance, security and privacy, assurance, agility and usability), and four alternatives, was designed based on a comprehensive literature review. the weights of the criteria were obtained using pythagorean fuzzy ahp. to rank the alternatives, the pythagorean fuzzy vikor method was used. bakioglu and atahan (2020) obtained the weights of factors affecting the adoption of self-driving vehicles using interval-valued pythagorean fuzzy ahp. the study considered psychological, safety, driving-related and external factors. alem (2020) assessed four different types of agriculture including traditional farming, artificial intelligence aided farming, vertical farming and plant-based meat. in the study, cost, ecology and customers were selected as the three main criteria. no dependency could be found between these criteria; therefore, interval type-2 fuzzy ahp was selected to determine the weight of the factors. after the weights were found, hesitant fuzzy topsis was used to rank the alternatives. hasgul and aytore (2020) solved a road selection problem for autonomous trucks in turkey. although self-driving trucks have not yet been introduced, the aim of this study was to be a pioneer in the field by examining truck roads between industrial areas to decide which road would be the best fit to invest in for autonomous trucking. since there are many uncertainties in this decision, the fuzzy ahp method was used. cost and maintenance, traffic, demands in the region, trade potential, and weather conditions were the considered criteria. otay and atik (2020) concentrated on a location evaluation and selection problem of an oil station. the location was comprised of several quantitative and qualitative criteria such as environmental, economic, and traffic-related factors. the spherical fuzzy ahp and spherical fuzzy waspas methods were used to provide a solution for the problem. karayazi and bereketli (2020) proposed a multi-criteria decision model to assist a global logistics company on the blockchain software selection problem based on buckley’s fuzzy ahp. six main criteria and 13 sub-criteria were used in the hierarchy. sahin et al. (2020) presented an application in which companies within the country can be individually categorized according to the logistic performance index (lpi). using classical ahp and fuzzy ahp, the most important criteria were also found. these results provided a practical application that shows which areas are behind the lpi index and which criteria can improve their performances. demir and koca (2020) compared the applications of intuitionistic multi-criteria supplier selection methods under a fuzzy environment. the intuitionistic fuzzy ahp and topsis methods were thoroughly discussed and utilized to complete a green supplier evaluation. three alternative green suppliers and 10 sub-criteria under the three maincriteria, including supplier performance, environmental protection and supplier risk ijahp essay: kahraman, cebi, cevik onar, öztayşi / recent developments on fuzzy ahp and anp under vague and imprecise data: evidence from infus conferences international journal of the analytic hierarchy process 5 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1033 were considered. the intuitionistic fuzzy topsis and ahp methods were used to rank alternatives from best to worst. keivanpour (2020) proposed a model for benchmarking and visualization of the ergonomic risks in manufacturing. the joint application of the fuzzy ahp and treemap was applied for benchmarking and mapping these risks. ilbahar et al. (2020) identified risks in research and development projects and evaluated them using interval-valued intuitionistic fuzzy ahp and a fuzzy inference system. alkan (2020) examined the risks of the advancement of technology more closely on sustainable supply chain environments. the aim was to create a model that evaluated the risks that are encountered under the digitalization perspective of the sustainable supply chain. the main and sub-criteria of the risk assessment model were determined by a comprehensive literature review and experts’ opinions. subsequently, the interval-valued pythagorean fuzzy ahp method was used to evaluate the related risks arising in the sustainable supply chain with digitalization. mizrahi et al. (2020) developed the prioritization-based taxonomy of the accounting information systems (ais)’s information quality factors using the fuzzy ahp approach. the information production processes of ais directly affect the quality of the information produced. for this reason, critical factors affecting the information quality were determined by focusing on the information production processes of ais. nineteen information quality factors were extracted from the literature. then, fuzzy ahp was used to prioritize and develop the taxonomy of the identified factors. 4. infus 2021 ahp/anp papers at infus 2021, a total of 13 papers on fuzzy ahp and one paper on fuzzy anp were presented. these papers developed some new fuzzy extensions of ahp and applied them to popular problems such as a software selection problem, green hospital attribute prioritization, clean energy evaluation, sustainable supply chain development. konyalıoğlu et al. (2021) proposed an assessment technique to develop an innovative multi-criteria decision-making approach that incorporates the anp method under a fuzzy environment. the authors provided a decision model with four criteria and 11 sub-criteria which are dependent on each other. the authors prioritized the criteria and the results revealed that the highest weight belongs to energy-related factors as criteria and as sub-criteria, the highest weight belongs to renewable material usage. altıntaşlı et al. (2021) focused on prioritization of significant criteria of service quality and evaluation of different business-to-customer (b2c) websites using fuzzy ahp and fuzzy topsis. the authors used eight criteria namely, agility, responsiveness towards customers, website appearance, easy-to-use interface, variety of payment options, security of payment options, product variety and product prices. the authors determined the weights of the criteria using fuzzy ahp and evaluated the alternatives using fuzzy topsis. miloševic et al. (2021) focused on the problem of internet and communication technologies in smart cities and provided a comparative study using fuzzy ahp and interval type-2 fuzzy sets. the decision model used for comparison involved six criteria, namely, governance, environment, buildings, mobility, education, and healthcare. the results of the study revealed that the methods are significantly similar. yel et al. (2021) focused on selecting a project ijahp essay: kahraman, cebi, cevik onar, öztayşi / recent developments on fuzzy ahp and anp under vague and imprecise data: evidence from infus conferences international journal of the analytic hierarchy process 6 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1033 management process based on technical and managerial criteria by considering the skills of the teams. the authors found the weights of the criteria using fuzzy ahp and then utilized the fuzzy edas and waspas methods for ranking. mukul et al. (2021) proposed a framework for evaluating clean energy alternatives using hesitant fuzzy sets. the evaluation criteria weights were found by hesitant ahp and then, clean energy alternatives were assessed by the hesitant edas method. the authors proposed a decision model involving five criteria, namely, environment, social, security and governance, economic, and technical and 15 sub-criteria. by using hesitant fuzzy sets, the decision makers could express their hesitance using linguistic terms such as between or at least. cubukcu and cantekin (2021) proposed using fuzzy ahp for determining the best firewall selection problem. the decision model involved three criteria, namely, cost, capacity, and productivity and nine sub-criteria. the model was solved using fuzzy ahp and five firewall alternatives were ranked. cevik onar and ibil (2021) focused on personal treatment methods for type-2 diabetes. the authors proposed a model for determining the best oral anti-diabetic combination for patients in different risk groups. spherical fuzzy ahp was used in the study to solve the decision model which included six criteria and five alternatives. menekşe and akdag (2021) studied an information technology governance problem as a multicriteria decision making problem and proposed using the spherical fuzzy ahp and electre methods. the decision model involved five criteria and four alternatives. in the proposed solution, the criteria weights were determined with the spherical fuzzy ahp technique and the alternatives were ranked with electre by constructing outranking relationships utilizing score and accuracy functions developed for spherical fuzzy sets. ariöz et al. (2021) concentrated on barriers to the big data driven fight against the covid-19 pandemic. after a literature survey, the authors provided a decision model with four criteria and 17 sub-criteria. the criteria involved in the study are socio-economic structure, technological incompetence, datarelated characteristics, and governmental policies. the authors used spherical fuzzy ahp to solve the problem and the results revealed that data related characteristics are the most important barrier. buran and erçek (2021) applied spherical fuzzy ahp to evaluate the public transportation business model taking into account internal and external perspectives with a political, economic, social, technological, legal, and environmental (pestle) analysis. the authors solved the problem with a decision model involving two main criteria, nine sub-criteria and 26 sub-sub-criteria. cizmecioglu et al. (2021) focused on a software project management model selection problem. in order to evaluate the different models such as waterfall, prototype, spiral, incremental, iterative and agile they used an interval-valued neutrosophic ahp methodology. the decision model involved five criteria, namely, requirement determination, cost, success guarantee, making changes, and time. the results of the case study showed that an agile project management methodology was the best alternative for the given case. ilbahar et al. (2021) concentrated on a social acceptability problem, and the factors affecting social acceptance were listed first. the authors used a bocr model to categorize the factors as benefits, opportunities, costs and risks. the decision model involved four criteria and 10 sub-criteria. the social acceptance assessment problem was solved using interval-valued intuitionistic fuzzy (ivif) ahp. yıldızbaş et al. (2021) emphasized the importance of a circular economy and focused on key challenges of the lithium-ion battery recycling process. the authors approached the problem with a decision model involving four criteria, ijahp essay: kahraman, cebi, cevik onar, öztayşi / recent developments on fuzzy ahp and anp under vague and imprecise data: evidence from infus conferences international journal of the analytic hierarchy process 7 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1033 namely, technical, economic, social, and environmental and 30 sub-criteria. the results of the study showed that economic challenges have relatively more importance during the lithium-ion battery recycling process. 5. infus 2022 ahp/anp papers at infus 2022, a total of 200 papers were presented and 17 of these papers utilized the ahp method to solve the problems. eleven of these ahp-based papers preferred extensions of fuzzy sets, while four articles included the application of ordinary fuzzy sets. only two of these papers were related to the classical ahp method. these papers developed some new fuzzy extensions of the ahp and applied them to some popular problems such as supply chain evaluation, site selection, risk assessment, human resources, and it technology. yalcinkaya and cebi (2022) used pythagorean fuzzy ahp to handle risks in the pharmaceutical supply chain under four main criteria (delivery reliability, quality, operational, communication and technology) and 28 sub-criteria. in the study, the pythagorean fuzzy ahp method was used to determine the importance degrees of the criteria. tunc and tasdemir (2022) proposed a study to predict the daily amount of money that should be in atm devices. in the study, the fuzzy ahp method was used to determine importance degrees of the parameters affecting the cash flow. coban and cevik onar (2022) used an integrated method including the fuzzy ahp and topsis methods to determine the appropriate site for grid-connected photovoltaic power plants. in the paper, fuzzy ahp was used to define importance degrees of the five criteria including social, technical, economic, environmental, and political factors. oturakci and konyalioglu (2022) proposed an objective risk assessment method and prioritized precautionary actions that must be taken. in the study, a fuzzy ahp-based topsis method was used to prioritize actions which were developed for the hazards defined by a fuzzy failure mode effect analysis. yel et al. (2022) proposed a study to determine the competencies of system analysts at a participating bank using fuzzy ahp-based neutrosophic z-number sets. in the study, fuzzy ahp was used to determine the importance weights of the competencies while rankings of system analysts were made using neutrosophic z-number sets. bal and ucal (2022) discussed the importance degrees of decision criteria on remote-working preferences of companies using picture fuzzy ahp. ilhan, gundogdu, and karasan (2022) considered a spaceport site selection problem using spherical fuzzy ahp. in the study, five main criteria and twenty sub-criteria and the method were applied to determine the importance degrees of these criteria. aslan and tolga (2022) conducted a study to determine the most appropriate area in which artificial intelligence (ai) technology can be used in aviation maintenance-repair-overhaul activities and to detect the most suitable ai tool. for the first aim, eleven criteria were determined and an ahp-based vikor method was used, while for the second aim, twelve criteria were determined and an ahp-based topsis method was used. in the study, ahp was utilized to determine the importance degrees of the criteria. cakir and tas (2022) conducted a study to present the challenges of remote working for businesses using circular intuitionistic fuzzy ahp. alkan and kahraman (2022) proposed a spherical fuzzy z-ahp method and the method was applied to a supplier selection problem. kose et al. (2022) used a spherical fuzzy ahp method to prioritize the factors causing ijahp essay: kahraman, cebi, cevik onar, öztayşi / recent developments on fuzzy ahp and anp under vague and imprecise data: evidence from infus conferences international journal of the analytic hierarchy process 8 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1033 rheumatoid arthritis and to select the best treatment alternative. in the study, three main criteria and a total of twenty-three sub-criteria are evaluated. seker (2022) utilized interval-valued pythagorean fuzzy ahp to determine importance degrees of the agility factors considering the fuel oil supply chain. in the paper, a total of nine criteria were considered. ilbahar et al. (2022) proposed an integrated method including interval-valued intuitionistic fuzzy (ivif) ahp and safety and critical effect analysis (scea) to evaluate and prioritize the risk in waste-to-energy (wte) plants. in the study, after risks in a wte plant were categorized, ivif ahp was used to obtain the probability, severity, frequency, and detectability weights of these risks, and then scea was utilized to prioritize the risk. soygüder et al. (2022) studied a supplier selection problem under the selected seven criteria using fuzzy ahp. yaman and yaylali (2022) used twelve criteria and a spherical fuzzy ahp-based topsis method for the selection of relevant locations of cargomatics. zakieh et al. (2022) developed a decision support system for investments in cryptocurrencies. in the study, the ahp method was used to understand the individual preferences of the investors. dalyan et al. (2022) utilized an interval-valued pythagorean fuzzy (ivpf) ahp-based topsis method for the selection of an enterprise resource planning system under four main criteria and twenty-two sub-criteria. in the study, ivpf ahp was utilized to determine weights of the criteria while the topsis method was used to evaluate alternatives. 6. conclusion figure 1 illustrates the distributions of ahp papers at each infus conference. it is clear that ahp papers using extensions of ordinary fuzzy sets have significantly increased from year to year (8 to 11), whereas ordinary fuzzy ahp papers have decreased (9 to 4). classical fuzzy ahp has only been used 2 times at infus 2022. the usage of vague data in ahp/anp methods has caused new scales to be defined for pairwise comparisons including a scale for interval-valued intuitionistic fuzzy ahp, a scale for interval-valued spherical fuzzy ahp, and a scale for interval-valued picture fuzzy sets. usage of different extensions and different fuzzy numbers in ahp made it necessary to develop a consistency measurement for each fuzzy ahp method. this approach has received a large amount of criticism in the literature since the consistency measurement is made by a defuzzification method, leaving the fuzziness very early, as in the classical ahp. consistency measurement is an important problem area of fuzzy ahp methods. conducting research on this problem provides important opportunities for researchers. since the membership degrees are between 01, it is necessary to investigate which value the fuzzy numbers correspond to on the 1-9 scale and to develop solutions for this. ijahp essay: kahraman, cebi, cevik onar, öztayşi / recent developments on fuzzy ahp and anp under vague and imprecise data: evidence from infus conferences international journal of the analytic hierarchy process 9 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1033 figure 1 distributions of ahp papers at infus conferences 0 2 4 6 8 10 12 2019 2020 2021 2022 extension of fuzzy ahp fuzzy ahp ahp ijahp essay: kahraman, cebi, cevik onar, öztayşi / recent developments on fuzzy ahp and anp under vague and imprecise data: evidence from infus conferences international journal of the analytic hierarchy process 10 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1033 references alem, s. (2021). special agriculture production selection using interval type-2 fuzzy ahp. in kahraman, c., cevik onar, s., oztaysi, b., sari, i., cebi, s., tolga, a. (eds) intelligent and fuzzy techniques: smart and innovative solutions, infus 2020. advances in intelligent systems and computing, 1197. springer, cham. doi: https://doi.org/10.1007/978-3-030-51156-2_64 alkan, n. (2021). risk analysis for digitalization oriented sustainable supply chain using interval-valued pythagorean fuzzy ahp. in kahraman, c., cevik onar, s., oztaysi, b., sari, i., cebi, s., tolga, a. (eds) intelligent and fuzzy techniques: smart and innovative solutions, infus 2020. advances in intelligent systems and computing, 1197. springer, cham. doi: https://doi.org/10.1007/978-3-030-511562_160 alkan, n., kahraman, c. (2022). fuzzy analytic hierarchy process using spherical z-numbers: supplier selection application. in kahraman, c., tolga, a.c., cevik onar, s., cebi, s., oztaysi, b., sari, i.u. (eds) intelligent and fuzzy systems, infus 2022, lecture notes in networks and systems, (504). springer, cham. doi: https://doi.org/10.1007/978-3-031-09173-5_81 altıntaşlı, a., temiz, m., şenvar, ö. (2022). a case study for evaluating e-commerce websites using integrated fuzzy ahp and fuzzy topsis. in kahraman, c., cebi, s., cevik onar, s., oztaysi, b., tolga, a.c., sari, i.u. (eds) intelligent and fuzzy techniques for emerging conditions and digital transformation, infus 2021, lecture notes in networks and systems, (307). springer, cham. doi: https://doi.org/10.1007/978-3-030-85626-7_37 arıöz, y., yılmaz, i., yıldızbaşı, a., öztürk, c. (2022). big data-driven in covid19 pandemic management system: evaluation of barriers with spherical fuzzy ahp approach. in kahraman, c., cebi, s., cevik onar, s., oztaysi, b., tolga, a.c., sari, i.u. (eds) intelligent and fuzzy techniques for emerging conditions and digital transformation, infus 2021, lecture notes in networks and systems, (308). springer, cham. doi: https://doi.org/10.1007/978-3-030-85577-2_94 aslan, m.e., tolga, a.c. (2022). evaluation of artificial intelligence applications in aviation maintenance, repair and overhaul industry via mcdm methods. in kahraman, c., tolga, a.c., cevik onar, s., cebi, s., oztaysi, b., sari, i.u. (eds) intelligent and fuzzy systems, infus 2022, lecture notes in networks and systems, (504). springer, cham. doi: https://doi.org/10.1007/978-3-031-09173-5_94 aydogan, f., zafeirakopoulos, i.b. (2020). leg base airline flight carbon emission performance assessment using fuzzy anp. in kahraman, c., cebi, s., cevik onar, s., oztaysi, b., tolga, a., sari, i. (eds) intelligent and fuzzy techniques in big data analytics and decision making, infus 2019, advances in intelligent systems and computing, (1029). springer, cham. doi: https://doi.org/10.1007/978-3-030-237561_97 bakioglu, g., atahan, a.o. (2021). evaluating the influencing factors on adoption of self-driving vehicles by using interval-valued pythagorean fuzzy ahp. in kahraman, c., cevik onar, s., oztaysi, b., sari, i., cebi, s., tolga, a. (eds) intelligent and fuzzy techniques: smart and innovative solutions, infus 2020, advances in intelligent https://doi.org/10.1007/978-3-030-51156-2_64 https://doi.org/10.1007/978-3-030-51156-2_160 https://doi.org/10.1007/978-3-030-51156-2_160 https://doi.org/10.1007/978-3-031-09173-5_81 https://doi.org/10.1007/978-3-030-85626-7_37 https://doi.org/10.1007/978-3-030-85577-2_94 https://doi.org/10.1007/978-3-031-09173-5_94 https://doi.org/10.1007/978-3-030-23756-1_97 https://doi.org/10.1007/978-3-030-23756-1_97 ijahp essay: kahraman, cebi, cevik onar, öztayşi / recent developments on fuzzy ahp and anp under vague and imprecise data: evidence from infus conferences international journal of the analytic hierarchy process 11 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1033 systems and computing, (1197). springer, cham. doi: https://doi.org/10.1007/978-3030-51156-2_58 bal, m., ucal sari, i. (2022). working environment selection after pandemic using picture fuzzy sets. in kahraman, c., tolga, a.c., cevik onar, s., cebi, s., oztaysi, b., sari, i.u. (eds) intelligent and fuzzy systems, infus 2022, lecture notes in networks and systems, (504). springer, cham. doi: https://doi.org/10.1007/978-3031-09173-5_58 buran, b., erçek, m. (2022). public transportation business model assessment with spherical fuzzy ahp. in kahraman, c., cebi, s., cevik onar, s., oztaysi, b., tolga, a.c., sari, i.u. (eds) intelligent and fuzzy techniques for emerging conditions and digital transformation, infus 2021, lecture notes in networks and systems, (308). springer, cham. doi: https://doi.org/10.1007/978-3-030-85577-2_87 buyuk, a.m., temur, g.t. (2021). a framework for selection of the best food waste management alternative by a spherical fuzzy ahp based approach. in kahraman, c., cevik onar, s., oztaysi, b., sari, i., cebi, s., tolga, a. (eds) intelligent and fuzzy techniques: smart and innovative solutions, infus 2020, advances in intelligent systems and computing, (1197). springer, cham. doi: https://doi.org/10.1007/978-3030-51156-2_19 çakır, e., taş, m.a. (2022). circular intuitionistic fuzzy analytic hierarchy process for remote working assessment in covid-19. in kahraman, c., tolga, a.c., cevik onar, s., cebi, s., oztaysi, b., sari, i.u. (eds) intelligent and fuzzy systems, infus 2022, lecture notes in networks and systems, (504). springer, cham. doi https://doi.org/10.1007/978-3-031-09173-5_68 cinar, u., cebi, s. (2020). a hybrid risk assessment method for mining sector based on qfd, fuzzy logic and ahp. in kahraman, c., cebi, s., cevik onar, s., oztaysi, b., tolga, a., sari, i. (eds) intelligent and fuzzy techniques in big data analytics and decision making, infus 2019, advances in intelligent systems and computing, (1029). springer, cham. doi: https://doi.org/10.1007/978-3-030-23756-1_141 cizmecioglu, n., kilic, h.s., kalender, z.t., tuzkaya, g. (2022). selection of the best software project management model via interval-valued neutrosophic ahp. in kahraman, c., cebi, s., cevik onar, s., oztaysi, b., tolga, a.c., sari, i.u. (eds) intelligent and fuzzy techniques for emerging conditions and digital transformation, infus 2021,lecture notes in networks and systems, (308). springer, cham. doi: https://doi.org/10.1007/978-3-030-85577-2_46 çoban, v., onar, s.ç. (2022). site selection of grid-connected photovoltaic power plants with fuzzy hybrid method. in kahraman, c., tolga, a.c., cevik onar, s., cebi, s., oztaysi, b., sari, i.u. (eds) intelligent and fuzzy systems, infus 2022, lecture notes in networks and systems, (504). springer, cham. doi: https://doi.org/10.1007/978-3-031-09173-5_37 cubukcu, c., cantekin, c. (2022). using a fuzzy-ahp decision model for selecting the best firewall alternative. in kahraman, c., cebi, s., cevik onar, s., oztaysi, b., tolga, a.c., sari, i.u. (eds) intelligent and fuzzy techniques for emerging conditions and digital transformation, infus 2021, lecture notes in networks and systems, (307). springer, cham. doi: https://doi.org/10.1007/978-3-030-85626-7_50 https://doi.org/10.1007/978-3-030-51156-2_58 https://doi.org/10.1007/978-3-030-51156-2_58 https://doi.org/10.1007/978-3-031-09173-5_58 https://doi.org/10.1007/978-3-031-09173-5_58 https://doi.org/10.1007/978-3-030-85577-2_87 https://doi.org/10.1007/978-3-030-51156-2_19 https://doi.org/10.1007/978-3-030-51156-2_19 https://doi.org/10.1007/978-3-031-09173-5_68 https://doi.org/10.1007/978-3-030-23756-1_141 https://doi.org/10.1007/978-3-030-85577-2_46 https://doi.org/10.1007/978-3-031-09173-5_37 https://doi.org/10.1007/978-3-030-85626-7_50 ijahp essay: kahraman, cebi, cevik onar, öztayşi / recent developments on fuzzy ahp and anp under vague and imprecise data: evidence from infus conferences international journal of the analytic hierarchy process 12 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1033 dalyan, t., otay, i., gülada, m. (2022). interval-valued pythagorean fuzzy ahp & topsis for erp software selection. in kahraman, c., tolga, a.c., cevik onar, s., cebi, s., oztaysi, b., sari, i.u. (eds) intelligent and fuzzy systems, infus 2022, lecture notes in networks and systems, (505). springer, cham. doi: https://doi.org/10.1007/978-3-031-09176-6_78 demir, e., koca, g. (2021). green supplier selection using intuitionistic fuzzy ahp and topsis methods: a case study from the paper mills. in kahraman, c., cevik onar, s., oztaysi, b., sari, i., cebi, s., tolga, a. (eds) intelligent and fuzzy techniques: smart and innovative solutions, infus 2020, advances in intelligent systems and computing, (1197). springer, cham. doi: https://doi.org/10.1007/978-3030-51156-2_77 hasgul, z., aytore, c. (2021). road selection for autonomous trucks in turkey with fuzzy ahp. in kahraman, c., cevik onar, s., oztaysi, b., sari, i., cebi, s., tolga, a. (eds) intelligent and fuzzy techniques: smart and innovative solutions, infus 2020,. advances in intelligent systems and computing, (1197). springer, cham. doi: https://doi.org/10.1007/978-3-030-51156-2_67 ilbahar, e., cebi, s., kahraman, c. (2021). risk assessment of r&d activities using intuitionistic fuzzy ahp and fis. in kahraman, c., cevik onar, s., oztaysi, b., sari, i., cebi, s., tolga, a. (eds) intelligent and fuzzy techniques: smart and innovative solutions, infus 2020, advances in intelligent systems and computing, (1197). springer, cham. doi: https://doi.org/10.1007/978-3-030-51156-2_155 ilbahar, e., cebi, s., kahraman, c. (2022). risk assessment of wte plants by using a modified fuzzy scea approach. in kahraman, c., tolga, a.c., cevik onar, s., cebi, s., oztaysi, b., sari, i.u. (eds) intelligent and fuzzy systems, infus 2022, lecture notes in networks and systems, (505). springer, cham. doi: https://doi.org/10.1007/978-3-031-09176-6_27 ilbahar, e., cebi, s., kahraman, c. (2022). social acceptability assessment of renewable energy policies: an integrated approach based on ivpf bocr and ivif ahp. in kahraman, c., cebi, s., cevik onar, s., oztaysi, b., tolga, a.c., sari, i.u. (eds) intelligent and fuzzy techniques for emerging conditions and digital transformation, infus 2021, lecture notes in networks and systems, (308). springer, cham. doi: https://doi.org/10.1007/978-3-030-85577-2_11 i̇lhan, m., gündoğdu, f.k., karaşan, a. (2022). analyzing critical criteria of spaceport site selection based on spherical fuzzy ahp method. in kahraman, c., tolga, a.c., cevik onar, s., cebi, s., oztaysi, b., sari, i.u. (eds) intelligent and fuzzy systems, infus 2022,. lecture notes in networks and systems, (504). springer, cham. doi: https://doi.org/10.1007/978-3-031-09173-5_80 jaller, m., otay, i. (2021). evaluating sustainable vehicle technologies for freight transportation using spherical fuzzy ahp and topsis. in kahraman, c., cevik onar, s., oztaysi, b., sari, i., cebi, s., tolga, a. (eds) intelligent and fuzzy techniques: smart and innovative solutions, infus 2020, advances in intelligent systems and computing, (1197). springer, cham. doi: https://doi.org/10.1007/978-3-030-511562_15 https://doi.org/10.1007/978-3-031-09176-6_78 https://doi.org/10.1007/978-3-030-51156-2_77 https://doi.org/10.1007/978-3-030-51156-2_77 https://doi.org/10.1007/978-3-030-51156-2_67 https://doi.org/10.1007/978-3-030-51156-2_155 https://doi.org/10.1007/978-3-031-09176-6_27 https://doi.org/10.1007/978-3-030-85577-2_11 https://doi.org/10.1007/978-3-031-09173-5_80 https://doi.org/10.1007/978-3-030-51156-2_15 https://doi.org/10.1007/978-3-030-51156-2_15 ijahp essay: kahraman, cebi, cevik onar, öztayşi / recent developments on fuzzy ahp and anp under vague and imprecise data: evidence from infus conferences international journal of the analytic hierarchy process 13 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1033 karabayir, a.n., botsali, a.r., kose, y., cevikcan, e. (2020). supplier selection in a construction company using fuzzy ahp and fuzzy topsis. in kahraman, c., cebi, s., cevik onar, s., oztaysi, b., tolga, a., sari, i. (eds) intelligent and fuzzy techniques in big data analytics and decision making, infus 2019, advances in intelligent systems and computing, (1029). springer, cham. doi: https://doi.org/10.1007/978-3-030-23756-1_60 karaşan, a., bolturk, e., kahraman, c. (2020). an integrated interval-valued neutrosophic ahp and topsis methodology for sustainable cities’ challenges. in kahraman, c., cebi, s., cevik onar, s., oztaysi, b., tolga, a., sari, i. (eds) intelligent and fuzzy techniques in big data analytics and decision making, infus 2019, advances in intelligent systems and computing, (1029). springer, cham. doi: https://doi.org/10.1007/978-3-030-23756-1_79 karaşan, a., gündoğdu, f.k., kahraman, c. (2020). pythagorean fuzzy ahp method for the selection of the most appropriate clean energy technology. in kahraman, c., cebi, s., cevik onar, s., oztaysi, b., tolga, a., sari, i. (eds) intelligent and fuzzy techniques in big data analytics and decision making, infus 2019, advances in intelligent systems and computing, (1029). springer, cham. doi: https://doi.org/10.1007/978-3-030-23756-1_105 karasan, a., kaya, i̇., erdoğan, m., budak, a. (2020). risk analysis of the autonomous vehicle driving systems by using pythagorean fuzzy ahp. in kahraman, c., cebi, s., cevik onar, s., oztaysi, b., tolga, a., sari, i. (eds) intelligent and fuzzy techniques in big data analytics and decision making, infus 2019, advances in intelligent systems and computing, (1029). springer, cham. doi: https://doi.org/10.1007/978-3-030-23756-1_110 karayazi, f., bereketli, i. (2021). criteria weighting for blockchain software selection using fuzzy ahp. in kahraman, c., cevik onar, s., oztaysi, b., sari, i., cebi, s., tolga, a. (eds) intelligent and fuzzy techniques: smart and innovative solutions, infus 2020, advances in intelligent systems and computing, (1197). springer, cham. doi: https://doi.org/10.1007/978-3-030-51156-2_70 keivanpour, s. (2021). benchmarking and visualization of the ergonomic risks in a manufacturing plant: joint application of fuzzy ahp and treemap. in kahraman, c., cevik onar, s., oztaysi, b., sari, i., cebi, s., tolga, a. (eds) intelligent and fuzzy techniques: smart and innovative solutions, infus 2020, advances in intelligent systems and computing, (1197). springer, cham. doi: https://doi.org/10.1007/978-3030-51156-2_80 kılıç, h., kabak, ö. (2020). analysis of relation between human development and competitiveness using fuzzy anp and dea. in kahraman, c., cebi, s., cevik onar, s., oztaysi, b., tolga, a., sari, i. (eds) intelligent and fuzzy techniques in big data analytics and decision making, infus 2019, advances in intelligent systems and computing, (1029). springer, cham. doi: https://doi.org/10.1007/978-3-030-237561_103 konyalıoğlu, a.k., beldek, t., camgöz-akdağ, h. (2022). green hospital attributes’ prioritization using a fuzzy analytic network process (f-anp)-based method. in kahraman, c., cebi, s., cevik onar, s., oztaysi, b., tolga, a.c., sari, i.u. (eds) intelligent and fuzzy techniques for emerging conditions and digital transformation, https://doi.org/10.1007/978-3-030-23756-1_60 https://doi.org/10.1007/978-3-030-23756-1_79 https://doi.org/10.1007/978-3-030-23756-1_105 https://doi.org/10.1007/978-3-030-23756-1_110 https://doi.org/10.1007/978-3-030-51156-2_70 https://doi.org/10.1007/978-3-030-51156-2_80 https://doi.org/10.1007/978-3-030-51156-2_80 https://doi.org/10.1007/978-3-030-23756-1_103 https://doi.org/10.1007/978-3-030-23756-1_103 ijahp essay: kahraman, cebi, cevik onar, öztayşi / recent developments on fuzzy ahp and anp under vague and imprecise data: evidence from infus conferences international journal of the analytic hierarchy process 14 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1033 infus 2021, lecture notes in networks and systems, (307). springer, cham. doi: https://doi.org/10.1007/978-3-030-85626-7_36 köse, r.e., rençber, n., beldek, t., konyalıoğlu, a.k. (2022). a decision support system for rheumatoid arthritis (ra) treatment selection and factor prioritization by using spherical fuzzy sets. in kahraman, c., tolga, a.c., cevik onar, s., cebi, s., oztaysi, b., sari, i.u. (eds) intelligent and fuzzy systems, infus 2022, lecture notes in networks and systems, (504). springer, cham. doi: https://doi.org/10.1007/978-3031-09173-5_83 kutlu gündoğdu, f., kahraman, c. (2020). spherical fuzzy analytic hierarchy process (ahp) and its application to industrial robot selection. in kahraman, c., cebi, s., cevik onar, s., oztaysi, b., tolga, a., sari, i. (eds) intelligent and fuzzy techniques in big data analytics and decision making, infus 2019, advances in intelligent systems and computing, (1029). springer, cham. doi: https://doi.org/10.1007/978-3-030-23756-1_117 maden, a. (2020). suitability evaluation of blockchain–based systems using fuzzy anpa case study in a logistics company. in kahraman, c., cebi, s., cevik onar, s., oztaysi, b., tolga, a., sari, i. (eds) intelligent and fuzzy techniques in big data analytics and decision making, infus 2019, advances in intelligent systems and computing, (1029). springer, cham. doi: https://doi.org/10.1007/978-3-030-237561_50 menekşe, a., camgöz akdağ, h. (2022). information technology governance evaluation using spherical fuzzy ahp electre. in kahraman, c., cebi, s., cevik onar, s., oztaysi, b., tolga, a.c., sari, i.u. (eds) intelligent and fuzzy techniques for emerging conditions and digital transformation, infus 2021, lecture notes in networks and systems, (308). springer, cham. doi: https://doi.org/10.1007/978-3030-85577-2_89 milošević, d., milošević, m., simjanović, d. (2022). a comparative study of fahp with type-1 and interval type-2 fuzzy sets for ict implementation in smart cities. in kahraman, c., cebi, s., cevik onar, s., oztaysi, b., tolga, a.c., sari, i.u. (eds) intelligent and fuzzy techniques for emerging conditions and digital transformation, infus 2021, lecture notes in networks and systems, (308). springer, cham. doi: https://doi.org/10.1007/978-3-030-85577-2_97 mizrahi, r., aygülen, b.t., karakul, k. (2021). fuzzy ahp based prioritization and taxonomy of information quality factors in accounting information systems. in kahraman, c., cevik onar, s., oztaysi, b., sari, i., cebi, s., tolga, a. (eds) intelligent and fuzzy techniques: smart and innovative solutions, infus 2020, advances in intelligent systems and computing, (1197). springer, cham. doi: https://doi.org/10.1007/978-3-030-51156-2_163 montazer, g.a., nourianfar, k., shayganmehr, m. (2020). selecting strategic partner for iranian official airline based on mixed fuzzy ahp and copras methods. in kahraman, c., cebi, s., cevik onar, s., oztaysi, b., tolga, a., sari, i. (eds) intelligent and fuzzy techniques in big data analytics and decision making, infus 2019, advances in intelligent systems and computing, (1029). springer, cham. doi: https://doi.org/10.1007/978-3-030-23756-1_90 https://doi.org/10.1007/978-3-030-85626-7_36 https://doi.org/10.1007/978-3-031-09173-5_83 https://doi.org/10.1007/978-3-031-09173-5_83 https://doi.org/10.1007/978-3-030-23756-1_117 https://doi.org/10.1007/978-3-030-23756-1_50 https://doi.org/10.1007/978-3-030-23756-1_50 https://doi.org/10.1007/978-3-030-85577-2_89 https://doi.org/10.1007/978-3-030-85577-2_89 https://doi.org/10.1007/978-3-030-85577-2_97 https://doi.org/10.1007/978-3-030-51156-2_163 https://doi.org/10.1007/978-3-030-23756-1_90 ijahp essay: kahraman, cebi, cevik onar, öztayşi / recent developments on fuzzy ahp and anp under vague and imprecise data: evidence from infus conferences international journal of the analytic hierarchy process 15 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1033 mukul, e., güler, m., büyüközkan, g. (2022). evaluation of clean energy alternatives with hesitant fuzzy linguistic mcdm methods. in kahraman, c., cebi, s., cevik onar, s., oztaysi, b., tolga, a.c., sari, i.u. (eds) intelligent and fuzzy techniques for emerging conditions and digital transformation, infus 2021, lecture notes in networks and systems, (307). springer, cham. doi: https://doi.org/10.1007/978-3-030-85626-7_39 onar, s.c., ibil, e.h. (2022). a decision support system proposition for type-2 diabetes mellitus treatment using spherical fuzzy ahp method. in kahraman, c., cebi, s., cevik onar, s., oztaysi, b., tolga, a.c., sari, i.u. (eds) intelligent and fuzzy techniques for emerging conditions and digital transformation, infus 2021, lecture notes in networks and systems, (308). springer, cham. doi: https://doi.org/10.1007/978-3-030-85577-2_88 otay, i., atik, s. (2021). multi-criteria oil station location evaluation using spherical ahp & waspas: a real-life case study. in kahraman, c., cevik onar, s., oztaysi, b., sari, i., cebi, s., tolga, a. (eds) intelligent and fuzzy techniques: smart and innovative solutions, infus 2020, advances in intelligent systems and computing, (1197). springer, cham. doi: https://doi.org/10.1007/978-3-030-51156-2_68 otay, i., yıldız, t. (2021). multi-criteria cloud computing service provider selection employing pythagorean fuzzy ahp and vikor. in kahraman, c., cevik onar, s., oztaysi, b., sari, i., cebi, s., tolga, a. (eds) intelligent and fuzzy techniques: smart and innovative solutions, infus 2020, advances in intelligent systems and computing, (1197). springer, cham. doi: https://doi.org/10.1007/978-3-030-511562_49 oturakçı, m., konyalıoğlu, a.k. (2022). action selection based on fuzzy ahp-based topsis method in fuzzy fmea-based risk assessment: a case study. in kahraman, c., tolga, a.c., cevik onar, s., cebi, s., oztaysi, b., sari, i.u. (eds) intelligent and fuzzy systems, infus 2022, lecture notes in networks and systems, (504). springer, cham. doi: https://doi.org/10.1007/978-3-031-09173-5_45 ozkan, o., aydin, s. (2020). supplier selection with intuitionistic fuzzy ahp and goal programming. in kahraman, c., cebi, s., cevik onar, s., oztaysi, b., tolga, a., sari, i. (eds) intelligent and fuzzy techniques in big data analytics and decision making, infus 2019, advances in intelligent systems and computing, (1029). springer, cham. doi: https://doi.org/10.1007/978-3-030-23756-1_100 oztaysi, b., onar, s.c., kahraman, c. (2020). innovative teaching feedback system design using hesitant fuzzy ahp approach. in kahraman, c., cebi, s., cevik onar, s., oztaysi, b., tolga, a., sari, i. (eds) intelligent and fuzzy techniques in big data analytics and decision making, infus 2019, advances in intelligent systems and computing, (1029). springer, cham. doi: https://doi.org/10.1007/978-3-030-237561_119 oztaysi, b., onar, s.c., kahraman, c. (2021). a fuzzy pricing model for mobile advertisements by using spherical fuzzy ahp scoring. in kahraman, c., cevik onar, s., oztaysi, b., sari, i., cebi, s., tolga, a. (eds) intelligent and fuzzy techniques: smart and innovative solutions, infus 2020, advances in intelligent systems and computing, (1197). springer, cham. doi: https://doi.org/10.1007/978-3-030-511562_18 https://doi.org/10.1007/978-3-030-85626-7_39 https://doi.org/10.1007/978-3-030-85577-2_88 https://doi.org/10.1007/978-3-030-51156-2_68 https://doi.org/10.1007/978-3-030-51156-2_49 https://doi.org/10.1007/978-3-030-51156-2_49 https://doi.org/10.1007/978-3-031-09173-5_45 https://doi.org/10.1007/978-3-030-23756-1_100 https://doi.org/10.1007/978-3-030-23756-1_119 https://doi.org/10.1007/978-3-030-23756-1_119 https://doi.org/10.1007/978-3-030-51156-2_18 https://doi.org/10.1007/978-3-030-51156-2_18 ijahp essay: kahraman, cebi, cevik onar, öztayşi / recent developments on fuzzy ahp and anp under vague and imprecise data: evidence from infus conferences international journal of the analytic hierarchy process 16 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1033 oztaysi, b., yücel, m., onar, s.c., öztürk, e., kahraman, c. (2020). weighting performance indicators of debt collection offices by using hesitant fuzzy ahp. in kahraman, c., cebi, s., cevik onar, s., oztaysi, b., tolga, a., sari, i. (eds) intelligent and fuzzy techniques in big data analytics and decision making, infus 2019, advances in intelligent systems and computing, (1029). springer, cham. doi: https://doi.org/10.1007/978-3-030-23756-1_120 peker, i., caybasi, g., buyukozkan, g., gocer, f. (2020). evaluation of home health care vehicle routing methods by intuitionistic fuzzy ahp. in kahraman, c., cebi, s., cevik onar, s., oztaysi, b., tolga, a., sari, i. (eds) intelligent and fuzzy techniques in big data analytics and decision making, infus 2019, advances in intelligent systems and computing, (1029). springer, cham. doi: https://doi.org/10.1007/978-3030-23756-1_74 şahin, e., pala, i̇., denizhan, b. (2021). logistics performance index (lpi) analysis with using ahp and fuzzy ahp. in kahraman, c., cevik onar, s., oztaysi, b., sari, i., cebi, s., tolga, a. (eds) intelligent and fuzzy techniques: smart and innovative solutions, infus 2020, advances in intelligent systems and computing, (1197). springer, cham. doi: https://doi.org/10.1007/978-3-030-51156-2_73 seker, s. (2020). fuzzy ahp-qfd methodology and its application to retail chain. in kahraman, c., cebi, s., cevik onar, s., oztaysi, b., tolga, a., sari, i. (eds) intelligent and fuzzy techniques in big data analytics and decision making, infus 2019, advances in intelligent systems and computing, (1029). springer, cham. doi: https://doi.org/10.1007/978-3-030-23756-1_140 seker, s. (2020). site selection for solar power plants using integrated two-stage hybrid method based on intuitionistic fuzzy ahp and copras approach. in kahraman, c., cebi, s., cevik onar, s., oztaysi, b., tolga, a., sari, i. (eds) intelligent and fuzzy techniques in big data analytics and decision making, infus 2019, advances in intelligent systems and computing, (1029). springer, cham. doi: https://doi.org/10.1007/978-3-030-23756-1_75 seker, s. (2022). evaluation model for supply chain agility in a fuel oil supply company. in kahraman, c., tolga, a.c., cevik onar, s., cebi, s., oztaysi, b., sari, i.u. (eds) intelligent and fuzzy systems, infus 2022, lecture notes in networks and systems, (505). springer, cham. doi: https://doi.org/10.1007/978-3-031-09176-6_24 soygüder, s., rouyendegh, b.d., tan, a. (2022). hybrid models for vendor selection problem in software industry: a pilot case study. in kahraman, c., tolga, a.c., cevik onar, s., cebi, s., oztaysi, b., sari, i.u. (eds) intelligent and fuzzy systems, infus 2022, lecture notes in networks and systems, (505). springer, cham. doi: https://doi.org/10.1007/978-3-031-09176-6_34 tunç, a., taşdemir, s. (2022). fuzzy based weighted, arithmetic optimization algorithm (aoa) for cash management optimization on automatic teller machines (atm). in kahraman, c., tolga, a.c., cevik onar, s., cebi, s., oztaysi, b., sari, i.u. (eds) intelligent and fuzzy systems, infus 2022, lecture notes in networks and systems, (504). springer, cham. doi: https://doi.org/10.1007/978-3-031-09173-5_36 https://doi.org/10.1007/978-3-030-23756-1_120 https://doi.org/10.1007/978-3-030-23756-1_74 https://doi.org/10.1007/978-3-030-23756-1_74 https://doi.org/10.1007/978-3-030-51156-2_73 https://doi.org/10.1007/978-3-030-23756-1_140 https://doi.org/10.1007/978-3-030-23756-1_75 https://doi.org/10.1007/978-3-031-09176-6_24 https://doi.org/10.1007/978-3-031-09176-6_34 https://doi.org/10.1007/978-3-031-09173-5_36 ijahp essay: kahraman, cebi, cevik onar, öztayşi / recent developments on fuzzy ahp and anp under vague and imprecise data: evidence from infus conferences international journal of the analytic hierarchy process 17 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1033 unal, y., temur, g.t. (2021). using spherical fuzzy ahp based approach for prioritization of criteria affecting sustainable supplier selection. in kahraman, c., cevik onar, s., oztaysi, b., sari, i., cebi, s., tolga, a. (eds) intelligent and fuzzy techniques: smart and innovative solutions, infus 2020, advances in intelligent systems and computing, (1197). springer, cham. doi: https://doi.org/10.1007/978-3030-51156-2_20 yalcinkaya, i., cebi, s. (2022). using fuzzy set based model for pharmaceutical supply chain risks assessment. in kahraman, c., tolga, a.c., cevik onar, s., cebi, s., oztaysi, b., sari, i.u. (eds) intelligent and fuzzy systems, infus 2022, lecture notes in networks and systems, (504). springer, cham. doi: https://doi.org/10.1007/978-3-031-09173-5_32 yaman, t.t., yaylalı, s. (2022). pilot location selection for cargomatics: contactless parcel pick-up points. in kahraman, c., tolga, a.c., cevik onar, s., cebi, s., oztaysi, b., sari, i.u. (eds) intelligent and fuzzy systems, infus 2022, lecture notes in networks and systems, (505). springer, cham. doi: https://doi.org/10.1007/978-3031-09176-6_56 yel, i̇., sarucan, a., baysal, m.e. (2022). an application of fuzzy ahp, edas and waspas for the selection of process method in software projects. in kahraman, c., cebi, s., cevik onar, s., oztaysi, b., tolga, a.c., sari, i.u. (eds) intelligent and fuzzy techniques for emerging conditions and digital transformation, infus 2021, lecture notes in networks and systems, (307). springer, cham. doi: https://doi.org/10.1007/978-3-030-85626-7_42 yel, i̇., sarucan, a., baysal, m.e. (2022). determination of competencies with fuzzy multi-criteria decision making methods for determining the development program for analyst position in a participation bank. in kahraman, c., tolga, a.c., cevik onar, s., cebi, s., oztaysi, b., sari, i.u. (eds) intelligent and fuzzy systems, infus 2022, lecture notes in networks and systems, (504). springer, cham. doi: https://doi.org/10.1007/978-3-031-09173-5_51 yildiz, n., kahraman, c. (2020). evaluation of social sustainable development factors using buckley’s fuzzy ahp based on z-numbers. in kahraman, c., cebi, s., cevik onar, s., oztaysi, b., tolga, a., sari, i. (eds) intelligent and fuzzy techniques in big data analytics and decision making, infus 2019, advances in intelligent systems and computing, (1029). springer, cham. doi: https://doi.org/10.1007/978-3030-23756-1_92 yıldızbaşı, a., öztürk, c., yılmaz, i̇., arıöz, y. (2022). key challenges of lithium-ion battery recycling process in circular economy environment: pythagorean fuzzy ahp approach. in kahraman, c., cebi, s., cevik onar, s., oztaysi, b., tolga, a.c., sari, i.u. (eds) intelligent and fuzzy techniques for emerging conditions and digital transformation, infus 2021, lecture notes in networks and systems, (308). springer, cham. doi: https://doi.org/10.1007/978-3-030-85577-2_66 zakieh, a.r., utku, s., amroush, f. (2022). evaluation of cryptocurrencies dynamically based on users’ preferences using ahp. in kahraman, c., tolga, a.c., cevik onar, s., cebi, s., oztaysi, b., sari, i.u. (eds) intelligent and fuzzy systems, infus 2022, lecture notes in networks and systems, (505). springer, cham. doi: https://doi.org/10.1007/978-3-031-09176-6_62 https://doi.org/10.1007/978-3-030-51156-2_20 https://doi.org/10.1007/978-3-030-51156-2_20 https://doi.org/10.1007/978-3-031-09173-5_32 https://doi.org/10.1007/978-3-031-09176-6_56 https://doi.org/10.1007/978-3-031-09176-6_56 https://doi.org/10.1007/978-3-030-85626-7_42 https://doi.org/10.1007/978-3-031-09173-5_51 https://doi.org/10.1007/978-3-030-23756-1_92 https://doi.org/10.1007/978-3-030-23756-1_92 https://doi.org/10.1007/978-3-030-85577-2_66 https://doi.org/10.1007/978-3-031-09176-6_62 ijahp article: marcarelli/evaluating healthcare organizations with a network model which integrates anp with a revised-bsc international journal of the analytic hierarchy process 86 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.443 evaluating healthcare organizations with a network model which integrates anp with a revised-bsc gabriella marcarelli university of sannio benevento, italy gabriella.marcarelli@gmail.com abstract the evaluation of healthcare organizations is complex due to the trade-offs between all healthcare features, benefits and costs to consider and the great number of stakeholders involved. each healthcare policy is multidimensional and its criteria are both objective and subjective in nature, so multicriteria methods are appropriate decision-making tools for a general manager in the evaluation process. this paper proposes to evaluate the strategic policies of an italian healthcare organization by means of the analytic network process (anp) integrated with a revised balanced scorecard (r-bsc) approach. this integrated network model allows one to take into account the micro-macro organizational setting opportunities and some normative constraints and incentives, typical of the italian health context. so, it may be useful not only to rank strategic policies and derive priorities for perspectives, objectives and metrics involved in a strategic map, but also to assign a performance index to each element of the network model. https://doi.org/10.13033/ijahp.v9i1.443 keywords: healthcare organization; strategic management; analytic network process; balanced scorecard; organizational decision making 1. introduction in recent years, in the health sector in italy, as in other economically developed countries, deep changes have occurred as a result of the gradual aging of the population, the consequent growth of health care spending, the continuous progress of scientific and technological knowledge, increasing competition, the emergence of a greater awareness and attention for the quality of health care by users, and the chronic scarcity of available resources. since the nineties, italian laws have intended to promote a more efficient and effective public health service management through decentralization and the introduction of a managerial vision. new organizational, managerial and accounting methods, typical of a company, have been introduced in healthcare organizations (hcos) for translating strategies into operational terms. healthcare organizations have paid attention to the https://doi.org/10.13033/ijahp.v9i1.443 https://doi.org/10.13033/ijahp.v9i1.443 ijahp article: marcarelli/evaluating healthcare organizations with a network model which integrates anp with a revised-bsc international journal of the analytic hierarchy process 87 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.443 balanced scorecard (bsc), which allows them to define and enforce business strategies through an integrated and balanced system of qualitative and quantitative metrics: economic and financial metrics for measuring efficiency and quality and performance metrics for evaluating effectiveness and social sustainability. the bsc has the great advantage of aligning the strategy to operational management by bridging the gap between formulation and implementation of the strategy (riccaboni, busco, bacci & del medico, 2008). hcos need to adopt some tools and criteria to enable them to act effectively and efficiently and, therefore, to evaluate the overall results, in terms of consistency between resources employed and the results achieved in terms of protection of health. the bsc has been applied as a tool for linking healthcare organization’s long-term objectives and local operations and enhancing strategic linkage between central and local units. in ramanathan and ramanathan (2011) the bsc integrated with dea (data envelopment analysis) was applied to balanced performance evaluation of health authorities in the uk. in a comprehensive review, zelman et al (2003) show that the bsc has been introduced across all areas related to healthcare. some limitations and implications linked to the use of the bsc approach in the healthcare system have been explored by chang (2007) and bentes, carneiro, ferreira da silva & kimura (2012). first of all, the four dimensions of the bsc do not take into account the interests of some stakeholders (competitors, suppliers, community and regulators). the bsc was designed for profit-motivated organizations with the aim of maximizing the profit for the shareholder’s of an organization. however, an organization operating in the public sector such as a healthcare facility, which has multiple objectives to deliver, may have difficulty incorporating a complex organizational and political context, institutional pressures and all causal relationships and conflicts within a simplified bsc framework (chang 2007). moreover, no standard mathematical method supports the bsc; thus, perspectives, objectives and metrics seldom have equal importance and, finally, relations among objectives and metrics are unidirectional (bentes et al. 2012). in the healthcare context, the number of perspectives may be greater than four, so the number of performance objectives and metrics could be high and the evaluation could be more complex to deal with. multicriteria methods are appropriate for facing these limits. hcos and their policies are multidimensional and criteria involved are both objective and subjective in nature, so multicriteria methods would appear to be useful tools to handle the general manager (gm). in particular, the analytic hierarchy process (ahp) and the analytic network process (anp) analyze complex problems by combining both qualitative and quantitative aspects in a single framework (saaty & vargas 1982; vargas 1990). several applications of the ahp in the healthcare context exist (forman & gass, 2001; liberatore & nydick, 2008; marcarelli, 2016; sloane et al., 2003). it has been applied for defining the medical priorities of a community in order to determine hospital requirements, supporting the hospitals purchasing negotiation and evaluating organizational performance within the bsc approach (saaty & vargas 1982; sloane et al., 2003; bentes et al., 2012). the anp, used for coping with dependence and feedback, has been applied for modeling supply chains or evaluating different alternatives for end-of-life computers (agarwal, shankar, & tiwari, 2006; pramod & banwet, 2010; ravi et al., 2005). in the literature, some approaches that integrate the anp with the bsc have been proposed for measuring management performance, evaluating the performance of a university in taiwan, https://doi.org/10.13033/ijahp.v9i1.443 ijahp article: marcarelli/evaluating healthcare organizations with a network model which integrates anp with a revised-bsc international journal of the analytic hierarchy process 88 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.443 defining an it outsourcing strategy, and determining the weights for bsc perspectives (chen, huang & cheng, 2009; tzeng, 2010; tjader, may, shang, vargas & gao 2014; tjader, shang & vargas, 2009; thakkar, deshmukh, gupta & shankar, 2007). however, in regards to a healthcare system, only a modified bsc has been applied for highlighting the necessity to consider more than 4 perspectives or a multicriteria method (such as ahp and anp) has been suggested for deriving the weights to assign to perspectives or objectives/metrics. this paper provides the gm of a hco (local units or hospitals) with a strategic tool to analyze and select healthcare policies and strategies by integrating, within a single framework, a revised-balanced scorecard (r-bsc) with anp. this model allows one to consider clinical and social implications, micro and macro organizational setting opportunities, and some normative constraints and incentives typical of the italian health context (ovretveit 1998; cinelli, 2015; hu, wen, & yan, 2015; verzola et al., 2009). the main contribution of this paper lies in the development of a comprehensive model (which incorporates anp rbsc) for analyzing a healthcare context (in particular, italian healthcare organizations) characterized by a complex decision making process with many specificities. this model provides the analytical tool for ranking healthcare policies and measuring relative and global weight of each network element considered in the analysis (perspectives, objectives and metrics) by taking into accout the relationships among all the elements. 2. the italian healthcare system 2.1 an overview of the inhs the institution of the italian national health service (inhs), founded in 1978, realizes the constitutional principle of health protection as a fundamental right of the individual and collective interest. the inhs differs from two health systems adopted in previous decades, that is, voluntary insurance and social security (of illness). the previous insurance system was suitable for achieving disease protection, thereby insuring at the moment when the disease had occurred, but could not be used to prevent. while in previous models, insurers simply pay back health costs incurred by clients, in the inhs the coverage provides uniform healthcare access to citizens by means of taxation, following a model similar to the beveridge model developed by the british nhs (nuti & seghieri, 2013). the inhs is organized into three levels: national the central government, regional and local health authorities, and hospitals (cinelli, 2015; balduzzi & carpani, 2013). in the nineties, some legislative reforms of the inhs (see legislative decrees 502/1992 and 517/1993) transferred administrative and organizational responsibilities from the central government to the regions, and transformed local health units and hospitals into public companies giving them broad management autonomy. the local health authorities have a purchasing function and the hospital units have a function of providing specialized health services. the central government is in charge of the general functions of planning, coordinating and control and the determination of essential assistance levels (eal), that is, benefits and services that the inhs has to provide to all citizens, free of charge or for a fee (ticket). regions are responsible for institutional structure, organizational model of service delivery and funding criteria. local health authorities, directed by public managers appointed by the regions, manage services in the territory of competence and https://doi.org/10.13033/ijahp.v9i1.443 ijahp article: marcarelli/evaluating healthcare organizations with a network model which integrates anp with a revised-bsc international journal of the analytic hierarchy process 89 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.443 are organized into districts to supply health services of primary assistance and first aid. health federalism focuses on the passage from a financing policy based on historic spending to one based on standard costs and requirements for each region. transfers to the regions occur according to standard costs, that is, per capita spending supported by the most efficient regions (benchmark) in supplying the eal. regions provide for the financing of local healthcare companies and hospital units according to different criteria. the former are financed according to the number of citizens residing in the territory of competence (by taking into account epidemiological and demographic factors), the latter are financed under the services actually provided (diagnosis related groups drg) and also receive a lump sum funding for highly specialized services. 2.2 the role of the general manager the management of health services is well-structured with regards to its functions (strategic and professional trend and management), the complexity of the organizational structures, and the different professional roles (medical, health, technical, administrative). at the top there is management including the gm, selected by the region, and the chief administrative and medical officers, selected by the gm; at a lower level there is the operational management that is divided into the roles of medical, health and technicaladministrative personnel (fragale, 2013; balduzzi & carpani, 2013). the gm plays a key role because he or she attempts to find an internal balance between the need to ensure effective performance to users (quality of care and wellbeing) and the need to optimize the operational results, by taking as reference the centrality of the patient (marcarelli, 2016). he is responsible for the results achieved by the company. by comparing the points of view of the main actors of the health system, the top manager, on the one hand, and the medical officer, on the other hand, it has been shown that common purposes and objectives do not always exist. is it better to pay attention to health by improving the quality of care or to drg? the gm should ensure that the mix of services and procedures provided is that which will give the greatest benefit for the population served at lowest risk and cost: rationalization of input and efficiency of output and outcome (table 1). the future sustainability of the inhs is influenced by the increasing demand for health services by the population and the reduction of resources involved in financing the national health service (toth, 2014). financial measures undertaken in recent years by the italian government have caused a curtailment of the national health fund. the spending review has imposed a reduction of the number of hospital beds by regions. therefore, on the one hand, there are additional costs due to the aging population and the increase in chronic diseases, and, on the other hand, the new frontiers of medical science and the complexity of health services are constantly growing. central and local government should consider these constraints and the need to cut both the expenditure and taxes for the low economic growth. the costs of health systems are determined by the offerings, such as new therapies and technologies, and the growing expectations of the population in terms of protection against risks to health and access to high quality healthcare. indeed, the aim of the last reforms of the inhs was to control expenditures and promote quality, efficiency and citizen satisfaction. health conditions and longevity are influenced by many factors and conditions including hereditary characteristics (genetic heritage), social and physical environment, socio-economic status, and lifestyle (marmot & wilkinson, 2006; gay et al., 2011; figueras & mckee, 2012). in 2012, the member states of the european region of the world health organization agreed with a new common https://doi.org/10.13033/ijahp.v9i1.443 ijahp article: marcarelli/evaluating healthcare organizations with a network model which integrates anp with a revised-bsc international journal of the analytic hierarchy process 90 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.443 model of politics which includes improving the health and wellbeing of populations, reducing inequalities in health, strengthening public health, and ensuring that health systems focus on the person, are universal, equitable, sustainable, high quality. this requires a new approach to healthcare systems in order to give priority to the prevention of disease, promote continuous improvement in the quality and supply of integrated services, ensure continuity of care, encourage autonomy in the care by patients and aim to provide support as close as possible to the patients home to protect the safety and costeffectiveness (world health organization, 2013). since the financing depends on the quality of services provided by the local health authority and its ability to attract users living outside the regional territory, in order to obtain greater financing, the gm of a local health organization could select policies that increase the quality of services and contain costs by also taking into account the healthcare offerings and demand in the regional and local territories by cooperating with other health organizations present locally (clerico, 2015). table 1 input, output and outcome input output outcome patients technologies material assets and services employees available spaces hospitalizations/drugs outpatient services laboratory tests patient safety life quality/wellbeing 2.3 macro and micro organizational setting: territorial integration and lean organization through information and communication technologies changes that occurred in the inhs in the 1990s involved hcos getting a shift from bureaucratic-functional forms of organization (with an internal subdivision based on technical and operational homogeneity) to forms more oriented to resource management and results (divisional or matrix models). traditional bureaucratic-functional models made horizontal integration hard to realize. a lean approach emerges in service organizations, as hcos: it is characterized by flexibility, monitoring of inefficiencies and waste, customer value, and provision for the patient’s centrality and the involvement of each operator in the improvement of performance (costa et al., 2014). in regards to the organizational structure, at the local level, the keystone is represented by the local health structures with their districts and departments, focused on the results. local health facilities are organized into departments. the department is a sub-structure of the hco and represents a managerial tool to coordinate clinical and organizational aspects of health services. attention to territorial assistance has modified services geography. the legislative decree no. 229, 1999, has extended the functions of districts. they organize primary assistance, day surgery and home help. in coordination with hospital assistance, the districts are responsible for managing social and healthcare activities delegated by municipalities and for supporting families and people in difficulty. they represent a place of social and health integration at the institutional, managerial and professional level. in order to https://doi.org/10.13033/ijahp.v9i1.443 ijahp article: marcarelli/evaluating healthcare organizations with a network model which integrates anp with a revised-bsc international journal of the analytic hierarchy process 91 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.443 supply territorial health services, the districts use several structures such as clinic, laboratory, hospice, and private hospital (rehab centre, nursing home). the concept of integration includes both the involvement of people in the health system and cooperation between different services with the aim of achieving common objectives. inner integration is functional to outer cooperation. several dimensions of integration exist: institutional, organizational, managerial, operational, and professional (gosetti & la rosa, 2006). the integration may follow two directions: vertical, concerning different levels of government and responsibility within a given organization, and horizontal, concerning the relations among subjects (as professionals), organizational units or organizations, at the same level. in a context in which regionalization has resulted in widely differing situations, a particular role is played by information and its management in hcos. relations between data, information and knowledge have given rise to studies on health knowledge management. technology, knowledge and human capital play an important role in organizations that take care of citizen health. the gm of a hco plays a complex role because he has to balance managerial and organizational logistics with those connected to investments, such as technological ones. socio-demographic changes and the need to balance available resources (economic, professional and organizational) and quality of healthcare assistance give rise to new modes of health services delivery that enable one to clearly trace a patient’s path from the first moment they interact with the assistance network (reina, 2012). the hta is a strategic tool to meet the needs and expectations of citizens. technologies entail an improvement in the level of satisfaction and the relationship with the patients, and facilitate communication and access to information and health resources. furthermore, together with economic, professional and organizational resources, they improve the performance of a healthcare system. e-health may involve several benefits:  access to information about the patient’s life history through the patient summary;  improvement of service to citizens and support medical decisions;  innovation in primary care through electronic health records, digitalization and electronic transmission of prescriptions and medical certificates;  public identification by sharing useful information for health planning;  improvement in access to health services by allowing citizens to reserve health services on the entire national territory. digital innovation of healthcare processes is a critical step to improve the cost-quality ratio of health services, limiting waste and inefficiencies, and reducing the differences between regions. information and communication technologies (ict) are crucial to managing the interdependencies and coordination requirements. 3. evaluating strategies and policies of healthcare organizations 3.1 the balanced scorecard in health context: limits and opportunities the balanced scorecard is a useful strategic measurement system, introduced by norton and kaplan (1992, 1996). it allows one to evaluate organizational performance from four perspectives (financial, customer, internal business processes and innovation learning), https://doi.org/10.13033/ijahp.v9i1.443 ijahp article: marcarelli/evaluating healthcare organizations with a network model which integrates anp with a revised-bsc international journal of the analytic hierarchy process 92 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.443 providing a balance between short-term and long-term objectives, financial and nonfinancial measures, lagging and leading indicators and internal and external perspectives. the bsc translates an organization strategy into four perspectives providing feedback around both internal business processes and external outcomes in order to continuously improve strategic performance and results. for each perspective a number of performance measures are considered. the bsc enables the manager to link the financial budget of a company with its strategic objectives and communicate strategic priorities of the gm to all levels of the hierarchy. two main approaches to bsc may be used: one emphasizing a balance between perspectives, leading to focus on stakeholder satisfaction, and the other considering a hierarchy of perspectives, via strategy maps, in which some perspectives may be functional to the other. in regards to the latter, there is no single map, but the map depends on the organization features (bisbe & barrubés, 2012). in the hierarchy of perspectives there are cause-effect relationships between objectives which are simple and unidirectional, non-cyclical and do not involve conflict and compromises (no perspective or stakeholder has priority over the other). the health system presents a dichotomy between the financial vision of the management and the clinical vision of healthcare professionals, so the bsc represents a good measurement system to integrate the two visions and control health costs, providing greater managerial flexibility. however, the use of bsc in healthcare exhibits some limits (ramanathan & ramanathan, 2011; chang, 2007; bentes et al. 2012):  no standard mathematical method supports the bsc;  perspectives and metrics seldom have equal importance;  more than four perspectives are usually considered but, with increasing the number of perspectives, the number of performance measures increases and the evaluation may be more complex to deal with. 3.2 outline of steps of the anp the anp is a multicriteria method developed by saaty for coping with problems that present dependence and feedback. the first step of the anp is the decomposition of the problem into a set of decision elements (objectives, criteria, subcriteria and alternatives) which form a network of clusters and interactions between and within these clusters. in the simplest case there exists a single network, where all clusters can influence or can be influenced by the other clusters and their elements can influence or be influenced by some other elements within the cluster (inner dependence) or belonging to some other clusters (outer dependence). relative weights reflect the relative importance of the elements belonging to each cluster with respect to the elements within the cluster or belonging to another cluster (saaty, 2005). to each pair of elements (xi, xj) of a cluster, a positive number (aij) is assigned expressing how much (xi) is preferred to (xj) with regards to a given criterion. saaty has proposed a nine point scale for comparing two elements: 1 implies indifference (equal impact) while 9 implies stronger impact (preference, or influence) of a row element than a column element; the reciprocal number from 1 to 9 expresses the inverse preference. in a more complex case, multiple networks with benefits, opportunities, costs and risks (bocr) exist. control criteria for bocr merits are determined and their priorities are then derived by pairwise comparisons using a supermatrix approach. for each control criterion a subnet of clusters is determined and the pairwise comparisons of the elements https://doi.org/10.13033/ijahp.v9i1.443 ijahp article: marcarelli/evaluating healthcare organizations with a network model which integrates anp with a revised-bsc international journal of the analytic hierarchy process 93 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.443 within the clusters and among the clusters are performed according to outer and inner dependence. judgments are expressed for answering two types of questions: 1) what element is preferred to another with respect to a given criterion? and 2) given two elements, who influences a 3-th element with respect to a given criterion? the weights of the clusters are derived by pairwise comparisons of them with respect to the control criterion and these weights are then used to weight the elements of the supermatrix and obtain the weighted column stochastic supermatrix. final priorities of stochastic supermatrix are calculated and syntesized by weighting each limit vector by the weight of its control criterion. the final ranking of the alternatives is given by a vector the components of which are obtained by the ratio bo/cr. anp may help managers understand the effect of their decisions and prioritize the areas requiring improvement. it is user friendly thanks to the superdecisions software, that allows for defining influence and feedback relationships between the elements of the map and performing a sensitivity analysis to verify whether results are reasonably stable: that is, there is preservation of the ranking among the alternatives when some other alternatives or criteria are added. 4. the anp integrated with the r-bsc after discussing the limits of the bsc (section 3.1), the opportunities offered by the anp (section 3.2), and the characteristics of the inhs (section 2), this paper proposes a network model which integrates the anp with a revised-bsc for analyzing hco performance and deriving priorities for interdependent (strategic or operational) objectives. instead of a traditional bsc approach, the revised-bsc is characterized by the analysis of six interdependent perspectives which allows for the incorporation of the specificities of the healthcare context; moreover, it makes it possible to define every type of relationship (among perspectives, objectives and metrics) that may be analyzed by the anp. thus, the analytical network process provides a proven way of eliciting and quantifying the relationships necessary to implement the bsc. in the r-bsc model proposed in this paper, the following perspectives have been considered: financial, customer, internal business processes, innovation and learning, socio-environmental and clinical. the socio-environmental perspective is particularly important in healthcare because it contextualizes the organization within the territory of competence and pays specific attention to the institutional system, identifying constraints and opportunities. this dimension includes the characteristics of the population (incidence of chronic diseases), territory of competence (population incidence by age and lifestyle, i.e. obesity rate) and institutional context (regional funding availability). hcos have to operate as a team to enhance synergies and ensure access and equity to all citizens of the region by enforcing regional strategic plans. interdependent relationships between relative objectives have been analyzed including: lack of skills that may affect some processes, low performance that may affect the quality perceived by customers, and decreasing customer satisfaction that may lead to inadeguate financial results. once the hco mission is fixed, the gm defines strategic areas (macro-objectives) by which the mission may be achieved. figure 1 shows the steps of the integrated model. https://doi.org/10.13033/ijahp.v9i1.443 ijahp article: marcarelli/evaluating healthcare organizations with a network model which integrates anp with a revised-bsc international journal of the analytic hierarchy process 94 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.443 figure 1. steps of integrated model let us consider three strategic objectives (macro-objectives):  improving the quality of healthcare services for citizens (e.g. reducing wait times for outpatientperformance as determined by the regional plan);  rationalization of use of resources (reorganize the health-care system in the interior of the hospital and arrange staff relating to health professions based on regional indications; stabilize the temporary staff; improve the training of health personnel and create a dedicated database);  strengthening of territorial healthcare (e.i. increase the seats of extensive rehabilitation beds in relation to the quota allocated by the region; increase the number of cases of home care). within the six perspectives of the r-bsc, the gm defines operational objectives and their metrics that allow them to realize their strategic objectives. tables 2-3 itemize the elements of the r-bsc that represent the first three steps of the integrated model in figure 1: goal (mission), strategic objectives, perspectives, operational objectives and metrics. https://doi.org/10.13033/ijahp.v9i1.443 ijahp article: marcarelli/evaluating healthcare organizations with a network model which integrates anp with a revised-bsc international journal of the analytic hierarchy process 95 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.443 table 2 clinical, socio-territorial and customer perspectives and relative operational objectives and metrics https://doi.org/10.13033/ijahp.v9i1.443 ijahp article: marcarelli/evaluating healthcare organizations with a network model which integrates anp with a revised-bsc international journal of the analytic hierarchy process 96 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.443 table 3 financial, internal process and innovation and learning perspectives and relative operational objectives and metrics the integrated model proposed in this paper is a network consisting of clusters and interactions between and within these clusters. there are some nodes representing decision criteria (perspectives, objectives and metrics) and alternatives, and some arcs depicting relationships among the clusters and their components. an arc connects pairs of nodes when there is significant interaction (with a two way arrow) or impact from one to the other (with a one way arrow). one-way arrows can also represent subordinate relationship between nodes. after all the required connections are made, the criteria are pairwise-compared, both within and between clusters. the proposed model may be applied for deriving the weights to assign to strategic objectives in order to realize the mission, or for obtaining the ranking among alternative actions (different policies) once a strategic objective has been fixed. the alternatives are pairwise compared with respect to all metrics; each metric has a weight depending on the operational objectives to which it is connected. for the sake of simplicty, figure 2 shows the clusters of perspectives, operational objectives and some clusters of metrics (in particular, all the metrics related to the objectives within the socio-territorial perspective and some metrics related to an objective of the innovation and learning perspective). it highlights some interdependencies among the clusters and/or the nodes. when comparing the elements within the cluster of “internal process objectives” with respect to the objective “enhance technological innovation” in the innovation and learning https://doi.org/10.13033/ijahp.v9i1.443 ijahp article: marcarelli/evaluating healthcare organizations with a network model which integrates anp with a revised-bsc international journal of the analytic hierarchy process 97 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.443 perspective, we capture the relative importance of internal process objectives when “enhance technological innovation” is concerned. in figure 2, an arc connects the technologies obsolescence rate to the time efficiency objective (within the “internal process perspective”). this means that there is a dependence relationship among these two elements. there is also a dependence relationship among the number of agreements with public and private organizations and time efficiency. this implies that, if the technologies obsolescence rate is high then it would be useful to increase both the number of agreements with public or private organizations and the percentage of sharing or outsourcing technologies (improving horizontal, vertical and territorial integration) in order to preserve the time efficiency; furthermore, if the percentage of sharing or outsourcing technologies increases then the time efficiency and the clinical appropriateness may be enhanced. figure 2. a network model. the integrated model may be useful for deriving performance indices in order to evaluate the performance of each element of the model both with respect to operational objectives and different analysis perspectives. by assuming that a strategic objective has been fixed, in the case of a single network (with one control criterion and 6 perspectives), the performance index of the i-th alternative may be determined as follows: perf (i) = p s i( )vs s=1 6 å where https://doi.org/10.13033/ijahp.v9i1.443 ijahp article: marcarelli/evaluating healthcare organizations with a network model which integrates anp with a revised-bsc international journal of the analytic hierarchy process 98 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.443 p s i( ) = o k s m j s ,k( ) w j s ,k( ) i( ) j=1 n k å k=1 m l å indicates the performance of the i-th alternative with respect to the s-th perspective and vs represents the weight of the perspective s. wj (s,k) (i), mj (s,k) and ok s represent, respectively, the weight of the i th alternative with respect to the j th metric within the objective k of the perspective s; the weight of the j th metric within the k th objective of perspective s; the weight of the k th objective of the perspective s. finally, ml and nk indicate the number of objectives within a given perspective and the number of metrics of the k th objective. some opportunities and advantages provided by the proposed integrated model are shown in figure 4. figure 4. advantages and opportunities of proposed network model 5. an example of decision problem let us consider a simple decision problem (figure 3). once the gm of a local hco has fixed a strategic objective, i.e. “rationalize the use of resources”, he has to select the best policy to realize that objective. the overall goal of the model is to derive a numerical score for each alternative. https://doi.org/10.13033/ijahp.v9i1.443 ijahp article: marcarelli/evaluating healthcare organizations with a network model which integrates anp with a revised-bsc international journal of the analytic hierarchy process 99 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.443 he has to choose between two alternative policies: closing some departments or reducing their functions and, simultaneously, enhancing territorial cooperation (equipment sharing); outsourcing it and human resources in order to offer the same services and, simultaneously, enhancing excellent departments. figure 3. an example of decision problem under the assumption that there are no interdependent relationships between and within the clusters, the score (performance) of alternative a1 with respect to the “financial” perspective is given by: perf fp a 1( ) = ok m j kw j k a 1( ) j=1 n 5 å k=1 5 å where wj (a1) represents the weight of the alternative a1 with respect to the j th metric (within the objective k of the financial perspective), m k j is the weight of the j th metric within the k th objective and ok represents the weight of the k th objective. tables 4-6 show some pairwise comparison matrices. https://doi.org/10.13033/ijahp.v9i1.443 ijahp article: marcarelli/evaluating healthcare organizations with a network model which integrates anp with a revised-bsc international journal of the analytic hierarchy process 100 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.443 table 4 pairwise comparisons between operational objectives with respect to the “financial perspective” financial perspective increase redditivity increase productivity improve competiveness reduce costs increase research funding increase redditivity 1 ½ 1/3 ¼ ½ increase productivity 2 1 ½ 1/3 2 improve competiveness 3 2 1 ½ 3 reduce costs 4 3 2 1 4 increase research funding 2 ½ 1/3 ¼ 1 the eigenvector of the above matrix provides the relative weights of the operational objectives: (0.077, 0.152, 0.256, 0.412, 0.102). table 5 pairwise comparisons between metrics with respect to the objective “increase redditivity” increase redditivity earnings before interest and taxes index of financial liquidity time to cash in earnings before interest and taxes 1 1/3 1/5 index of financial liquidity 3 1 ½ time to cash in 5 2 1 table 6 pairwise comparisons between metrics with respect to the objective “improve competiveness” improve competiveness brand image attraction rate retention rate incidence of passive mobility brand image 1 ½ 1/3 2 attraction rate 2 1 ½ 3 retention rate 3 2 1 4 incidence of passive mobility ½ 1/3 ¼ 1 https://doi.org/10.13033/ijahp.v9i1.443 ijahp article: marcarelli/evaluating healthcare organizations with a network model which integrates anp with a revised-bsc international journal of the analytic hierarchy process 101 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.443 the pairwise comparison matrices of metrics related to “reduce costs” and “increase productivity” have not been reported. table 7 shows the relative weights of metrics with regard to their related operational objective. table 7 the relative weights of metrics with respect to the related objective increase redditivity relative weights earnings before interest and taxes 0.109 index of financial liquidity 0.309 time to cash in 0.582 increase productivity relative weights operating income 0.109 profit margin 0.309 economic value added 0.582 improve competiveness relative weights brand image 0.160 attraction rate 0.277 retention rate 0.467 incidence of passive mobility 0.096 reduce costs relative weights structural intervention 0.1666 costs of production 0.1666 supply costs 0.1666 mean time for suppliers payment 0.1666 incidence of costs for drugs consumption 0.1666 incidence of costs for diagnostic materials 0.1666 increase research fund relative weight course of fund raising 1 table 8 shows the relative weights of the alternatives a1 and a2 with respect to the metrics considered in the financial perspective. https://doi.org/10.13033/ijahp.v9i1.443 ijahp article: marcarelli/evaluating healthcare organizations with a network model which integrates anp with a revised-bsc international journal of the analytic hierarchy process 102 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.443 table 8 the relative weights of the alternatives with respect to the metrics metrics a1 a2 earnings before interest and taxes 0.5 0.5 index of financial liquidity 0.5 0.5 time to cash in 0.5 0.5 operating income 0.4 0.6 profit margin 0.45 0.55 economic value added 0.5 0.5 brand image 0.4 0.6 attraction rate 0.45 0.55 retention rate 0.5 0.5 incidence of passive mobility 0.5 0.5 structural intervention 0.5 0.5 costs of production 0.55 0.45 supply costs 0.4 0.6 mean time for suppliers payment 0.5 0.5 incidence of costs for drugs consumption 0.7 0.3 incidence of costs for diagnostic materials 0.6 0.4 course of fund raising 0.5 0.5 thus, the performance of the alternative a1 with respect to the financial perspective may be obtained as follows: perf fp a 1( ) = 0.5 * 0.109 * 0.077 + 0.5 * 0.309 * 0.077 + 0.5 * 0.582* 0.077 + +0.4 * 0.109 * 0.152+ ... + 0.5*1* 0.102 = 0.6717 perf fp a 2( ) = 0.6606 this means that, if only the financial perspective is considered, then policy a1 is preferred to a2. of course, the global evaluation of the alternatives will result from a weighted linear combination depending on the weights assigned to objectives and perspectives. the performance of each metric may be derived too. 6. conclusion the proposed model which integrates the anp with the r-bsc approach has several advantages. first of all, this model is mathematically rigorous because of the anp framework; then, it allows for handling subjective, qualitative judgments as well as quantitative data and using different metrics (financial and non financial). moreover, it allows one to prioritize perspectives, objectives and metrics and to conduct the benefit/cost analysis, e.g. for direct resource allocation issues. it is an effective procedure even in group decision making, and is applicable to a dynamic changing organization. https://doi.org/10.13033/ijahp.v9i1.443 ijahp article: marcarelli/evaluating healthcare organizations with a network model which integrates anp with a revised-bsc international journal of the analytic hierarchy process 103 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.443 the proposed integrated model may allow the general manager: to set network relations between the elements of the strategic map (perspectives, objectives and metrics); to determine their priorities and derive performance indices in order to evaluate the performance of each element of the model both with respect to strategic and operational objectives and the different analysis perspectives; to modify the model structure according to changes in strategic direction and operational tactics; to assess the robustness of the final solution through anp sensitivity analysis; to compare the performance of different healthcare organizations. by taking into account the importance of each element for achieving the mission (by means of some strategic objectives), the integrated network model allows the gm to individuate the more appropriate element for improving the performance of hco, realizing the maximum benefit with the minimum effort in terms of resources employed. of course, the decision-making power of the gm is not absolute, but moves within the limits established at the national and regional level by strategic planning. the need to contain costs has led hcos to adopt practices usually used by private companies such as outsourcing and public-private cooperation. integration and cooperation are strategic elements on which the hco should rely. cooperation would entail a decrease in costs and and an increase in benefits and make it possible to follow the whole process of taking charge of the patient. the integrated model may be useful for many decision making problems; for example, the problem of selecting the best strategic policy in order to modify the business plan of a health facility according to the regional guidelines, or evaluating whether it is better to close or reduce some departments, or their functions, and enhance their excellence or open new departments. obviously, this choice presupposes preliminary analysis of the demographic and epidemiological context of the territory of competence and is closely related to the offering of health facilities in the local and regional territory. https://doi.org/10.13033/ijahp.v9i1.443 ijahp article: marcarelli/evaluating healthcare organizations with a network model which integrates anp with a revised-bsc international journal of the analytic hierarchy process 104 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.443 references agarwal, a., shankar, r., & tiwari m.k. (2006). modeling the metrics of lean, agile and leagile supply chain: an anp-based approach. european journal of operational research, 173, 211-225. doi: http://doi.org/10.1016/j.ejor.2004.12.005. balduzzi, r. & carpani, g. (2013). manuale di diritto sanitario. il mulino, bologna. bentes, a.v., carneiro, j., ferreira da silva, j., kimura h. (2012). multidimensional assessment of organizational performance: integrating bsc and ahp, journal of business research, 65, 1790-1799. doi: http://doi.org/10.1016/j.jbusres.2011.10.039. bisbe, j. & barrubés, j. (2012). the balanced scorecard as a management tool for assessing and monitoring strategy implementation in health care organizations. rev. esp. cardiol. 65(10), 919-927. doi: http://doi.org/10.1016/j.rec.2012.05.011. chang, l. (2007). the nhs performance assessment framework as a balanced scorecard approach. limitations and implications. international journal of public sector management, 20(2), 101-117. doi: http://dx.doi.org/10.1108/09513550710731472. chen m.y., huang m.j. & cheng y.c. (2009). measuring knowledge management performance using a competitive perspective: an empirical study. expert systems with applications, 36(4), 8449-8459. doi: http://doi.org/10.1016/j.eswa.2008.10.067. cinelli, m. (2015). diritto della previdenza sociale. giappichelli, torino. clerico, g. (2015). la governance della sanita pubblica. la coesistenza fra efficienza e qualità. giuffré. costa, g., gubitta, p. & pittino, d. (2014). organizzazione aziendale. italy: mcgraw hill education. figueras, j. & mckee, m. (2012). health systems, health, wealth and societal well-being, assessing the case for investing in health systems. mcgraw hill, open university press. forman, e.h. & gass, s.i. 2001. the analytic hierarchy process-an exposition, operations research, 49(4), 469-486. doi: http://dx.doi.org/10.1287/opre.49.4.469.11231. fragale, e.n. (2013). la dirigenza delle aziende sanitarie dopo la c.d. riforma balduzzi. istituzioni del federalismo, 2(2013), note e commenti. gay, j.g., paris, v., devaux, m. & de looper, m. (2011). mortality amenable to health care in 31 oecd countries, paris, oecd health working papers, n.55. doi: 10.1787/5kgj35f9f8s2-en. https://doi.org/10.13033/ijahp.v9i1.443 http://doi.org/10.1016/j.ejor.2004.12.005 http://doi.org/10.1016/j.jbusres.2011.10.039 http://doi.org/10.1016/j.rec.2012.05.011 http://doi.org/10.1016/j.eswa.2008.10.067 http://dx.doi.org/10.1287/opre.49.4.469.11231 ijahp article: marcarelli/evaluating healthcare organizations with a network model which integrates anp with a revised-bsc international journal of the analytic hierarchy process 105 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.443 gosetti, g. & la rosa, m. (2006). sociologia dei servizi. elementi di organizzazione e programmazione. milano, italy: franco angeli. hu, y., wen, j. & yan, y. (2015). measuring the performance of knowledge resources using a value perspective: integrating bsc an anp. journal of knowledge management, 19(6),1250-1272. doi: http://dx.doi.org/10.1108/jkm-10-2014-0431. kaplan, r.s. & norton, d.p. (1996). using the balanced scorecard as a strategic management system. harvard business review, 74(1), 75–85. kaplan, r.s. & norton, d.p. (1992). the balanced scorecard-measures that drive performance. harvard business review, 70(1), 71-79. liberatore, m.j. & nydick, r.l. (2008). the analytic hierarchy process in medical and health care decision making: a literature review. european journal of operational research, 189, 194-207. doi: http://doi.org/10.1016/j.ejor.2007.05.001. marcarelli, g. (2016). multi-criteria decision making for evaluating healthcare policies: the benefit/cost analysis by the analytic hierarchy process. international journal of medical engineering and informatics, 8(2), 163-181. doi: http://dx.doi.org/10.1504/ijmei.2016.075754. marmot, m. & wilkinson, r.g. (2006). using the balanced scorecard as a strategic management system, social determinants of health. oxford: oxford university press. ovretveit, j. (1998). evaluating health interventations. maintenhead: open university press. nuti, s. & seghieri, c. (2013). is the variation management included in regional healthcare governance systems? some proposals from italy. health policy, 114, 71-78. doi: http://dx.doi.org/10.1016/j.healthpol.2013.08.003. pramod, v.r. & banwet, d.k. (2010). analytic network process analysis of an indian telecommunication service supply chain: a case study. service science, 2(4), 281-293. doi: http://dx.doi.org/10.1287/serv.2.4.281. ramanathan, r. & ramanathan, u. (2011). a performance measurement framework combining dea and balanced scorecard for the uk health sector. international journal of operational research, 12(3), 257-278. doi: http://dx.doi.org/10.1504/ijor.2011.042916. ravi, v., shankar, r. & tiwari, m.k. (2005). analyzing alternatives in reverse logistics for end-of-life computer: anp and balances scorecard approach. computers industrial engineering, 48, 327-356. doi: http://doi.org/10.1016/j.cie.2005.01.017. reina, r. (2012). organizzare i sistemi informativi per la sanita. analisi e esperienze giuffre ed. https://doi.org/10.13033/ijahp.v9i1.443 http://doi.org/10.1016/j.ejor.2007.05.001 http://dx.doi.org/10.1016/j.healthpol.2013.08.003 http://dx.doi.org/10.1287/serv.2.4.281 http://doi.org/10.1016/j.cie.2005.01.017 ijahp article: marcarelli/evaluating healthcare organizations with a network model which integrates anp with a revised-bsc international journal of the analytic hierarchy process 106 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.443 riccaboni, a., busco, c., bacci, a. & del medico, g. (eds) (2008). la balanced scorecard per l’attuazione della strategia nelle pubbliche amministrazioni. teoria, casi ed esperienze. knowita. saaty, t.l. (2005). theory and applications of the analytic network process: decision making with benefits, opportunities, costs and risks. pittsburgh: rws publications. saaty, t.l. & vargas, l.g. (1982). the logic of priorities: applications in business, energy, health, and transportation. boston: kluwer-nijhoff. reprinted in paperback (1991), pittsburgh: rws publications. sloane, e.b., liberatore, m.j., nydick, r.l., luo, w. & chung, q.b. (2003). using the analytic hierarchy process as a clinical engineering tool to facilitate an iterative, multidisciplinary, microeconomic health technology assessment. computers and operations research, 30, 1447-1465. doi: http://doi.org/10.1016/s0305-0548(02)001879. toth, f. (2014). la sanità in italia. il mulino. thakkar j., deshmukh s. g., gupta a. d. & shankar r. (2007). development of a balanced scorecard an integrated approach of interpretive structural modeling (ism) and analytic network process (anp). international journal of productivity and performance management, 56(1), 22-59. doi: http://dx.doi.org/10.1108/17410400710717073. tjader y., may j.h., shang, j., vargas l.g. &gao n. (2014). firm-level outsourcing decision making: a balanced scorecard-based analytic network process model. international journal of production economics, 147, 614-623. doi: http://doi.org/10.1016/j.ijpe.2013.04.017. tjader y., shang, j. & vargas l. (2009). offshore outsourcing decision making: a policy-maker's perspective. working paper, university of pittsburgh. doi: http://doi.org/10.1016/j.ejor.2010.03.042. tzeng m.l. (2010). implementation and performance evaluation using the fuzzy network balanced scorecard. computers & education, 55, 188-201. doi: http://doi.org/10.1016/j.compedu.2010.01.004. vargas, l.g. (1990). an overview of the analytic hierarchy process and its applications, european journal of operational research, 48(1), 2-8. doi: https://doi.org/10.1016/0377-2217(90)90056-h. verzola, a., bentivegna, r., carandina, g., trevisani, l., gregorio, p. & mandini, a. (2009). multidimensional evaluation of performance: experimental application of the balanced scorecard in ferrara university hospital. cost effectiveness and resourse allocation, 7(15), 1-8. doi: 10.1186/1478-7547-7-15. zelman, w., pink, g., matthias, c. (2003). use of the balanced scorecard in health care. journal of health care finance, 29(4), 1-16. https://doi.org/10.13033/ijahp.v9i1.443 http://doi.org/10.1016/s0305-0548(02)00187-9 http://doi.org/10.1016/s0305-0548(02)00187-9 http://doi.org/10.1016/j.ijpe.2013.04.017 http://doi.org/10.1016/j.ejor.2010.03.042 http://doi.org/10.1016/j.compedu.2010.01.004 https://doi.org/10.1016/0377-2217(90)90056-h ijahp article: marcarelli/evaluating healthcare organizations with a network model which integrates anp with a revised-bsc international journal of the analytic hierarchy process 107 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.443 world health organization (regional office for europe) (2013). health 2020: a european policy framework supporting action across government and society for health and well-being. https://doi.org/10.13033/ijahp.v9i1.443 ijahp article: shahwan, sapian, ibrahim, mohammed, agha, kanesan/ranking the compliance of islamic banking home financing products with maqasid shariah using ahp international journal of the analytic hierarchy process 1 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.891 ranking the compliance of islamic banking home financing products with maqasid shariah using ahp syahidawati shahwan* 1 islamic finance and wealth management institute (ifwmi) faculty of economics and muamalat (fem) universiti sains islam malaysia (usim) syahidawati@usim.edu.my safeza mohd sapian faculty of economics and muamalat (fem) universiti sains islam malaysia (usim) safeza@usim.edu.my norhazlina ibrahim faculty of economics and muamalat (fem) universiti sains islam malaysia (usim) norhazlina@usim.edu.my mustafa omar mohammed college of economics and management (coem) alqasimia university, sharjah, uae momar@alqasimia.ac.ae ehsanullah agha standard chartered bank, uae ehsanaagha@gmail.com thavamaran kanesan proofreading by a uk ph.d. drthava@proofreadingbyphd.com abstract compliance of islamic banking products with maqasid shariah has been a neverending debate. however, most discussions are merely explorative, descriptive and perceptive in nature. this article aims to evaluate two types of home financing products in malaysia based on the maqasid shariah of the product development process in islamic banks. to provide a more coherent and concrete decision, the analytic hierarchy process (ahp) has been used. bay’ bithaman ajil (sale with deferred payment) and tawarruq (monetization) have been chosen to represent salebased financing and musharakah mutanaqiṣah (diminishing partnership) represents * corresponding author 1 acknowledgements: the outcome of this paper is part of research activities funded by the ministry of higher education malaysia under the fundamental research grant scheme (frgs) with code frgs/1/2016/ss01/usim/02/6. mailto:syahidawati@usim.edu.my mailto:safeza@usim.edu.my mailto:%20norhazlina@usim.edu.my mailto:momar@alqasimia.ac.ae mailto:ehsanaagha@gmail.com mailto:drthava@proofreadingbyphd.com ijahp article: shahwan, sapian, ibrahim, mohammed, agha, kanesan/ranking the compliance of islamic banking home financing products with maqasid shariah using ahp international journal of the analytic hierarchy process 2 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.891 partnership-based financing. the overall results demonstrated that partnership-based financing complied with the social requirements of mspdf (maqasid shariah-based product development framework) at 87.5%, while sale-based financing was rated at 12.5%. the findings will assist shariah advisors, islamic finance product developers, economic analysts, financial engineers and policymakers to incorporate the relevant shariah objectives in islamic financial product development. keywords: islamic finance; product development; maqasid shariah; sale-based product; partnership-based product 1. introduction the shariah-compliant banking industry in malaysia has made remarkable progress over the last three decades. the industry has been able to cater to the wider financial needs of its customers, particularly muslims, by providing various interest-free products. one of these products is home financing, which is offered by islamic banks through the concepts of bay bithaman ajil (bba-sale with deferred payment), tawarruq (monetization) and musharakah mutanaqiṣah (mm or diminishing partnership) (shahwan, 2017). currently, these products constitute 9%, 46% and 9% of total financing in 2022 in malaysia, respectively. other contracts like ijarah (leasebased), istiṣna’ (manufacturing) and others make up 36% of the total amount of financing based on islamic concepts (bnm, 2022). figure ‎1 bank negara malaysia statistical report on concepts used in financing facilities source: bank negara malaysia statistical report, 2022 over the years, islamic banking has been criticized for its overreliance on sale-based financing products for home financing, particularly bay bithaman ajil (bba) and tawarruq (t). critics argue that bba is structured based on debt, replicating conventional loan products that overburden customers in terms of price (agha, 2016; alziyadat & ahmed, 2019). in contrast, mm, which is a partnership-based product, has been proposed as just, equitable and closer to the objectives of shariah (asadov et al., 2018). nevertheless, after mm was introduced into the market, it has also been criticized for failing to meet maqasid shariah (objectives of islamic law) (shahwan, ijahp article: shahwan, sapian, ibrahim, mohammed, agha, kanesan/ranking the compliance of islamic banking home financing products with maqasid shariah using ahp international journal of the analytic hierarchy process 3 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.891 2017; shahwan, mohammad, rahman, 2013). this raises the question of whether these products do actually fail to achieve maqasid shariah. if they do, then at what stage in the process of development does this failure occur? most of the previous relevant studies are only explorative and descriptive. the present study contributes to the literature of islamic finance by empirically examining the compliance of the home financing products with the objectives of shariah. the objective of this study is to evaluate the extent to which sale-based home financing (represented by bba & t) and partnership-based home financing products (represented by musharakah mutanaqisah) in malaysia conform to shariah objectives by using the maqasid shariah-based product development framework (mspdf) proposed by shahwan (2017). 2. literature review an examination of shariah rulings discloses that islamic law is based on two philosophical foundations: ensuring human wellbeing (falah) by safeguarding public interest (maṣlaḥah) and preventing harm (mafṣadah) in social, political, economic and religious aspects of human life (agha, 2016; mohammad and shahwan, 2013). this philosophy is known in islamic discourse as maqasid shariah (hurayra, 2015). in islamic finance literature, maqasid shariah refers to the overall objectives that the shariah aims to achieve from its principles and guidelines, particularly those prescribed for financial affairs. these objectives provide a direction for arranging financing activities in an islamic economic system (soualhi, 2015). 2.1 maqasid shariah-based product development framework (mspdf) for islamic financial institutions shahwan (2017) proposed the maqasid shariah-based product development framework (mspdf) for islamic financial institutions. the mspdf is based on the theory of maqasid shariah adopted by al-ghazali (1997) and ʿatiyyah (2010), and the current study will use it to examine both sale-based products (bba & t) and the mm partnership-based product due to its comprehensiveness and relevance to modern financial transactions. figure ‎2 social requirements of maqasid shariah-based product development framework (mspdf) based on shahwan (2017) ijahp article: shahwan, sapian, ibrahim, mohammed, agha, kanesan/ranking the compliance of islamic banking home financing products with maqasid shariah using ahp international journal of the analytic hierarchy process 4 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.891 ‘atiyyah (2010) divided human essential needs (daruriyyat) into various angles instead of the five categories proposed by classical jurists (auda, 2015). he divided maqasid shariah into 24 objectives under four headings: i. individuals: (a) preservation of human life, (b) consideration of mind, (c) preservation of personal piety, (d) preservation of honor, and (e) preservation of material wealth. ii. family: (a) ordering the relationship between the sexes, (b) preservation of progeny, (c) achieving harmony, affection and compassion, (d) preservation of lineage, (e) preservation of personal piety within the family, (f) ordering the institutional aspect of the family, and (g) ordering the financial aspect of the family. iii. ummah: (a) institutional organization of the ummah (muslim nation), (b) maintenance of security, (c) establishment of justice, (d) preservation of religion and morals, (e) cooperation, solidarity and shared responsibility, (f) dissemination of knowledge and preservation of reason in the ummah, and (g) developing the earth and preserving the ummah’s wealth. iv. humanity: (a) mutual understanding, cooperation and integration, (b) realizing humanity’s viceregency on earth, (c) achieving peace with justice, (d) protection of human rights, and (e) dissemination of the islamic message. the new proposed division is an attempt to promote the idea that maqasid shariah should not be limited to only five objectives but should be extended based on circumstances and situations (mustafa, 2007). adapting the theory of al-ghazali (1997) and ʿatiyyah (2010) by including social requirements in the development of islamic banking products, shahwan (2017) divided the framework into two main sections as follows: product development process and incorporated social requirement of maqasid elements. the product development process adopts the three stages of ahmed’s (2011a) product development model including conceptualization, product development, and commercialization. 2.1.1 social requirements in the objective of shariah in islamic banks the social aspect of mspdf is the requirement that is able to be modified and amended since it does not yet have a specific regulation endorsed by regulators. the behavioral science method of sekaran and bougie (2016) is used to operationalize ‘aṭiyyah’s theory of maqasid shariah into several dimensions and elements observed in the framework. apart from ‘aṭiyyah’s theory, some elements were added to the model from the service quality model (servqual) (haywood-farmer, 1988), human development index (hdi) (chakravarty, 2003), malaysian quality of life report (government of malaysia, 2011), performance measurement and management systems (taticchi et al., 2012), and bnm product guidelines (bnm, 2014). the elements of maqasid shariah under the social requirements will be observed in all three stages of product conceptualization, development and commercialization. accordingly, this paper aims to evaluate the two selected types of islamic banking home financing based on their observance of maqasid shariah. hence, shahwan’s (2017) framework is selected for two reasons: ijahp article: shahwan, sapian, ibrahim, mohammed, agha, kanesan/ranking the compliance of islamic banking home financing products with maqasid shariah using ahp international journal of the analytic hierarchy process 5 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.891 i. it covers the social aspects of islamic finance; and ii. it combines finance-relevant and operational maqasid shariah elements from various models proposed by contemporary and classical scholars in the discipline. 2.1.2 product development stages of islamic banks there is limited literature on product development (pd) in islamic banking. most of the research focuses on shariah processes and discusses selected product development activities of islamic banks (hassan & lewis, 2007; lahsasna & hassan, 2011; muda & abdullaah, 2007). also, the literature conceptually mentions several stages of the shariah approval process rather than discussing the overall process of product development in islamic banks. ahmed (2011a, 2011b) addressed the limitations in the literature by developing a product development (pd) model adapted from cooper and brentani (1991) as shown in figure 3. figure 3 product development model based on habib ahmed (2011) rusni (2012) also provided general guidelines for shariah compliant practices related to the product development process in the shariah governance framework (2010) shown in figure 4. ijahp article: shahwan, sapian, ibrahim, mohammed, agha, kanesan/ranking the compliance of islamic banking home financing products with maqasid shariah using ahp international journal of the analytic hierarchy process 6 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.891 figure 4 product development process of shariah governance framework, bank negara malaysia, 2010 based on rusni (2012) 2.2 sale-based home financing: bba and tawarruq bba refers to bay’ bithaman ajil. the shariah advisory council (sac) of bank negara malaysia (bnm) (2010) defined bba home financing as “an islamic house financing facility which is based on the shariah concept of ‘bay’ bithaman ajil’ (bba)” (p. 162). it is a sale with deferred payment. the term was first used by the bank islam malaysia berhad (bimb) in 1983 with the inception of islamic banking (mohd yassin, 1997). tawarruq refers to “an arrangement that involves the purchase of an asset based on musawamah (supplier does not disclose its cost or markup) or murabaḥah (supplier sells at a markup and discloses it), and the subsequent sale of the same asset to a third party in order to raise cash” (isra, 2016, p. 337). the literature reveals that bba and tawarruq-based bba home financing products are criticized due to the involvement of (1) bay’ ‘inah (sale and buy back contract), (2) gharar (uncertainty) in the case of incomplete property, (3) non-possession in the event of project abandonment, and (4) overpricing. these issues have caused the customers of islamic banks to be dissatisfied with these products (meera & razak, 2005; nor, 2008; osmani and abdullah, 2010; saiful azhar, 2010; asni and sulong, 2018). to avoid these issues in islamic home-financing, musharakah mutanaqiṣah (diminishing partnership)-based home financing was suggested by scholars and economists (shahwan, 2017; ali & hassan, 2020). 2.3 partnership-based home financing: musharakah mutanaqiṣah musharakah in arabic refers to joint membership in one particular activity or asset, and mutanaqiṣah means diminishing. technically, aaoifi (2017) in its shariah standard no. 12 defines the mm contract as “a form of partnership in which one of the partners promises to buy the equity share of the other partner gradually until the title to the equity is completely transferred to him.” the product is deemed one of the innovative products in the islamic finance industry (asadov et al., 2018). earlier literature argued that mm is more compliant with the maqaṣid compared to bba since it resembles equity-based financing while the latter represents debt-based financing (asadov et al., 2018). however, after the application of mm in the market, despite its theoretical and conceptual superiority, it raised some concerns (haneef & furqani, 2011; kashi & mohamad, 2017). some of the concerns are: (1) sensitivity to shariah non-compliance risk, (2) pricing, (3) complicated operation, and (4) less ijahp article: shahwan, sapian, ibrahim, mohammed, agha, kanesan/ranking the compliance of islamic banking home financing products with maqasid shariah using ahp international journal of the analytic hierarchy process 7 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.891 adherence in practice to maqaṣid shariah (meera & razak, 2005; osmani & abdullah, 2010; mansoori, 2011; nor et al., 2019). the above analysis reveals that mm-based home financing as an alternative to bba & t has also become a focus of debate. however, there is no empirical research to validate the alignment of mm with the maqaṣid shariah. therefore, the present study aims to investigate whether mm-based islamic home-financing products actually fulfill maqaṣid shariah at the various stages of the product development process. 3. research methodology 3.1 decision analysis based on multiple criteria the goal of this paper is to evaluate the two types of home financing products by providing rankings of their compliance to the social requirements of maqasid shariah in the three stages of product development. the authors have benefited from thomas l. saaty’s decision-making tool, popularly known as the analytic hierarchy process (ahp), which is a multi-attribute decision making instrument for various fields including intangible criteria (huu phuong & kar yin, 2000). the steps of the ahp used in this study are shown in figure 5. figure 5 ahp model used in this study adapted from naseer et. al. (2022) 3.2 analytic hierarchy process (ahp) to evaluate the extent to which the two products conform to the maqasid shariahbased product development framework (mspdf), the ahp has been adopted. the ahp is a decision-making tool for selecting a right choice in a decision making ijahp article: shahwan, sapian, ibrahim, mohammed, agha, kanesan/ranking the compliance of islamic banking home financing products with maqasid shariah using ahp international journal of the analytic hierarchy process 8 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.891 problem (saaty, 1990). the ahp is considered the best tool for dealing with complex decision-making processes. previous research using the ahp in various banking and finance-related studies include sharma et al. (2022), abduh & omar (2012), ecer (2014), yana (2022), kristina & atmojo w.t. (2022), and qian (2022). the current study similarly adopted the ahp according to saaty's (2008) guidelines as shown in table 1. table 1 saaty (2008) guidelines incorporated in the current study steps saaty (2008) current study step 1 define the problem and determine the kind of knowledge sought. the problem is to investigate which of the two types of islamic home financing (sale-based or partnership-based) achieve the highest level of maṣlaḥah (welfare) based on maqasid shariah (the developed framework of mspdf) and at what stage of their product development. step 2 structure the decision hierarchy from the top with the goal of the decision, then the objectives from a broad perspective, through the intermediate levels (criteria on which subsequent elements depend) to the lowest level (which is usually a set of alternatives). the goal is to rank the two types of islamic home-financing at their stages of product development based on maqasid shariah. the criteria are the stages of product development in islamic banks; and the sub-criteria are the elements of maqasid shariah in mspdf. the hierarchy is structured based on the newly developed framework explained in the literature. the alternatives are the two types of home-financing products; sale-based (bba & t or partnership-based (musharakah mutanaqisah). step 3 construct a set of pair-wise comparison matrices (pcm). each element in the upper level is used to compare the elements in the level immediately below with respect to it. the pcm for all three criteria (product development stages), sub-criteria (maqasid elements) and two alternatives is constructed. face-to-face structured interviews with selected experts from the islamic banking industry were conducted. step 4 use the priorities obtained from the comparison to weigh the priorities in the level immediately below. then, for each element in the level below add its weighed value and obtain its overall or global priority. continue this process of weighing and adding until the final priorities of the alternatives in the bottom-most level are obtained. synthesis of the weights was obtained using the expert choice ii software. similarly, the overall priorities of the alternatives were obtained based on their individual importance ranking with respect to the criteria (product development stages) and sub-criteria (maqasid elements) compared to the two alternatives (islamic banking products). ijahp article: shahwan, sapian, ibrahim, mohammed, agha, kanesan/ranking the compliance of islamic banking home financing products with maqasid shariah using ahp international journal of the analytic hierarchy process 9 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.891 the framework developed and presented in figure 2 has been utilized in the hierarchy structure to complete step 2 of the ahp process. the commonly used four hierarchical levels; goal, criteria, sub-criteria and alternatives are shown in figure 6. figure 6 hierarchy structure based on mspdf for ahp 3.3 semi-structured interviews data were collected through structured interviews that were conducted over three months. five experts (pe1-pe5) were selected for interviews based on their expertise in product development and their shariah knowledge. all of the experts are officers in islamic banks and have substantial experience in the product development process of these banks in malaysia. their profiles lend credibility to the researchers’ choice. table 2 experts’ profile no. profiles pe1 pe2 pe3 pe4 pe5 1 educational background bachelor’s degree bachelor’s degree bachelor’s degree professional certificate bachelor’s degree master’s degree (phd candidate) 2 work experience in islamic banks 5 years 5 years 12 years 5 years 8 years 3 work experience in the current islamic bank 5 years 5 years 3 years 5 years 3 years 4 current designated bank locallyowned islamic bank locallyowned islamic bank foreignowned islamic bank locallyowned islamic bank development islamic financial institution 5 current designated department personal financing contract centre product manager shariah review product development 6 current designated position deputy manager team leader senior manager acting head of department not revealed 7 experience in product development yes yes yes yes yes ijahp article: shahwan, sapian, ibrahim, mohammed, agha, kanesan/ranking the compliance of islamic banking home financing products with maqasid shariah using ahp international journal of the analytic hierarchy process 10 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.891 the interview questionnaire was designed in three sections. section a required the interviewees to provide their profile as reported in table 1. section b provides guidelines for answering the questionnaire and a sample on how to structure the hierarchy for the two alternatives. this section was a response to previous pilot testing conducted by the researcher in which most of the participants did not know how to provide ratings based on the ahp technique. a glossary explaining the subcriteria was also provided to the interviewees since the mspdf was new to them. the glossary provided a contextual link between the alternatives and the criteria in the framework. section c in the questionnaire was based on the framework. a summary of the guidelines was also given to ensure that the experts fully understood the ranking procedure. a sample of section c is shown in figure 6. figure 6 sample questionnaire given to the experts the interview sessions were conducted in the offices of the interviewees and some were conducted via online teleconference. each interview lasted an average of sixty minutes. the interviewees were given introductory notes on the objectives of the study and the purpose of conducting the session. they were repeatedly reminded that their views as experts in the field of study were extremely important. the technique to rate the two alternatives based on mspdf was also explained. 4. findings the current study involved multiple experts which required aggregation of the responses. according to johnson (1980) and zahedi (1986), the last step after collecting pairwise comparisons data and assigning weights for the elements using the “eigenvalue” method is to aggregate the relative weights of decision elements from the multiple respondents to obtain a set of rankings for the decision alternatives (or ijahp article: shahwan, sapian, ibrahim, mohammed, agha, kanesan/ranking the compliance of islamic banking home financing products with maqasid shariah using ahp international journal of the analytic hierarchy process 11 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.891 outcomes). forman and peniwati (1998) stated that there are two approaches frequently used in the ahp to aggregate group decision making; aggregation of individual judgements (aij); and aggregation of the individual priorities (aip). aij normally uses the geometric mean and aip uses the arithmetic mean. in the present study, the geometric mean of aij was chosen to aggregate the group judgements. expert choice 11 software was utilized to compute the priorities of the bba & t and mm from the pcms based on the mspdf elements at the three stages in their product development. before computation of the priorities, a check of the consistency in the allocation of judgement was conducted. according to saaty (1987), the computation of a consistency index (ci) and its consistency ratio (cr) are as follows: where ri (random index) is the average of the ci values associated with several randomly generated pcm (of size n) (saaty & vargas, 1982). if the cr is less than 0.1, the matrix may be considered to have acceptable consistency; otherwise the judgments have to be revised. the consistency ratio (cr) for all the pair-wise comparison matrices (pcms) computed for the current study is 0.0458 or 4.58%. according to saaty (cited by marcarelli, g. and mancini, p (2022) and islam r. (2009)), if the cr value is less than 0.10, then the pcm may be considered to have acceptable consistency; however, if cr > .10, then the amount of consistency is not acceptable and revised judgements are required. since the current study corresponds to the former value, its shows that the respondents have been consistent in their judgements and the data were considered reliable (saaty & vargas, 2001). the ranking of bba & t and mm was calculated for three aspects: (1) ranking of the two types of products with reference to mspdf elements in three stages of product development; (2) ranking of sale-based home financing and partnership-based home financing with reference to mspdf elements, and (3) ranking the three stages of product development with reference to mspdf elements in sale-based home financing and partnership-based home financing. 4.1 ranking of the two types of products with reference to mspdf elements in three stages of product development the output from expert choice 11 software of the global weights of the two islamic home financing products of bba & t and mm with regards to their relation to mspdf is shown in table 3. ijahp article: shahwan, sapian, ibrahim, mohammed, agha, kanesan/ranking the compliance of islamic banking home financing products with maqasid shariah using ahp international journal of the analytic hierarchy process 12 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.891 table 3 ranking sale-based home financing (bba & t) and partnership-based home financing (musharakah mutanaqisah) of mspdf in three stages of product development no. types of product ranking 1 sale-based home financing (bba & t) 2 [0.125] 2 partnership-based home financing (musharakah mutanaqisah) 1 [0.875] * the number in [ ] shows the weight for each type of product with respect to mspdf elements as shown in table 3, the experts have ranked partnership-based home financing (musharakah mutanaqisah) higher than sale-based home financing (bba & t) in terms of achieving the 18 elements of maqasid shariah in mspdf in the three stages of their product development. this means, in the two stages (product commercialization and development), partnership-based home financing is able to preserve the standard of living, support financial literacy and commitment, promote spiritual and mental wellness, support social security, socio-economic justice, social solidarity and participation, and promote environmental quality compared to its alternative sale-based home financing (bba & t). in the stage of product commercialization, partnership-based home financing (musharakah mutanaqisah) is ranked higher than sale-based home financing (bba & t) in matters pertaining to promotion of customer product knowledge and awareness, customer physical and mental satisfaction, institutional (bank) physical readiness and institutional (staff) product training and knowledge. these results support similar findings by previous studies (mydin meera ahamed kameel & dzuljastri, 2009; mydin meera ahamed kameel & dzuljastri, 2005; dzuljastri & abduh, 2012; dzuljastri abdul & fauziah md, 2011; dzuljastri abdul et al., 2008; fauziah et al., 2008; md_dahlan, 2010; noreeta, 2008; dahlan nuarrual hilal & sharifah zubaidah, 2011; rosly saiful azhar, 2010, 2011; smolo & hassan, 2011). this implies that the industry should prioritize offering partnership-based home financing compared to sale-based home financing. furthermore, the potential of partnership-based home financing to foster maqasid shariah as per mspdf in the overall product development process in islamic banks will promote the execution of the maqasid theory on a larger scale along with the agenda of the sustainable development goal (sdg). 4.2 ranking of sale-based home financing and partnership-based home financing with reference to mspdf elements table 4 shows the ranking of sale-based home financing (bba & t) and partnershipbased home financing (musharakah mutanaqisah) with reference to mspdf elements in malaysia. at the stages of product conceptualization and product development, the first priority is to ensure that the products preserve shariah compliance which had been redefined in the framework by the term “spiritual and mental wellness” of muslims (which are symbolized by the letter a ). the second priority is to maintain “socioeconomic justice” which is symbolized by the letter b . “spiritual and mental wellness” is explained as the customer being able to adhere to the religious obligation especially in the muamalat with the new product offering in the market. it also means that by offering the new product, the customer can improve their confidence towards muamalat activity in islam. additionally, the new product promotes better livelihood and satisfaction. the new product also promotes better counseling and advice in the event of default. these are the aspects elaborated by the ijahp article: shahwan, sapian, ibrahim, mohammed, agha, kanesan/ranking the compliance of islamic banking home financing products with maqasid shariah using ahp international journal of the analytic hierarchy process 13 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.891 experts in relation to prioritizing spiritual and mental wellness when developing a new product of islamic bank. “socio-economic justice” is a situation where by offering a new product, the customer will be able to be involved in a profit and loss sharing (pls) activity. the customer will also be able to be involved in a cooperative activity and could experience just product price. they could then experience product price stability. in addition, when introducing a new product, in the conceptualization stage, setting this as a priority will reduce bias towards customers based on religion and gender. it also promotes continuous protection towards an existing customer and his/her future generation through any means (e.g. takaful protection, etc.). similarly in the product development stage, “socio-economic justice” is also given priority due to its proximity to philanthropic values. activities like micro financing, waqf and sadaqah will enhance the socio-economic justice of the society. in terms of product commercialization, the first priority of the experts was “institutional physical readiness” which is symbolized by the letter c , and refers to the situations where islamic financial institutions provide sufficient facilities (it, physical) related to the new product (existing or new facilities). the institution is also expected to provide sufficient promotion of the new product. the second priority based on the experts’ opinions was “institution staff training and development” which is symbolized by the letter d where the institution is expected to provide sufficient staff training, such as the technical input and the end to end product operation, prior to product launch. the contents of the training should also include conceptual input of the product to ensure sustainable product knowledge and delivery to end-users (banking customers). ijahp article: shahwan, sapian, ibrahim, mohammed, agha, kanesan/ranking the compliance of islamic banking home financing products with maqasid shariah using ahp international journal of the analytic hierarchy process 14 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.891 table 4 ranking of sale-based home financing (bba & t) and partnership-based home financing (musharakah mutanaqisah) with reference to mspdf elements * the number in [ ] shows the weight for each mspdf elements with respect to the types of home financing the above findings are in line with the findings of prior research (meera & razak, 2005; nor, 2008; osmani & abdullah, 2010; saiful azhar, 2010; zabri & mohammed, 2018; nor et al., 2019). these findings support the view that islamic banks should prioritize the mm home financing facility (mydin meera & abdul razak, 2009; mohammad & shahwan, s., 2013; zaaba & hassan, 2019; juliyanti & wibowo, 2021). 4.3 ranking the three stages of product development with reference to mspdf elements the software ranked the three stages of product development with respect to the selected islamic home financing products including product conceptualization, product development and product commercialization, as shown in table 5. table 5 rankings of the pd stages no. pd stages ranking 1 product conceptualisation 1 [0.662] 2 product development 2 [0.191] 3 product commercialisation 3 [0.146] * the number in [ ] shows the weight for each stage of product development elements with respect to mspdf maqasid elements bba&t mm p r o d u c t c o n c e p tu a li z a ti o n standard of living 4 [0.125] 3 [0.125] financial literacy and commitment 6 [0.064] 6 [0.064] spiritual and mental wellness a 1 [0.266] 1 [0.400] social security 5 [0.118] 5 [0.096] socio economic justice b 2 [0.205] 2 [0.167] social participation 3 [0.165] 4 [0.097] environmental quality 7 [0.070] 7 [0.051] p r o d u c t d e v e lo p m e n t standard of living 7 [0.019] 5 [0.087] financial literacy and commitment 3 [0.144] 3 [0.125] spiritual and mental wellness a 1 [0.207] 1 [0.325] social security 4 [0.132] 6 [0.082] socio economic justice b 2 [0.188] 2 [0.195] social participation 5 [0.125] 3 [0.125] environmental quality 6 [0.085] 7 [0.061] p r o d u c t c o m m e r c i a li sa ti o n customer product knowledge and awareness 4 [0.105] 4 [0.085] customer physical and mental satisfaction 3 [0.123] 3 [0.089] institution physical readiness c 1 [0.387] 1 [0.472] institution staff training and development d 2 [0.385] 2 [0.353] ijahp article: shahwan, sapian, ibrahim, mohammed, agha, kanesan/ranking the compliance of islamic banking home financing products with maqasid shariah using ahp international journal of the analytic hierarchy process 15 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.891 table 5 shows that of all three product development stages, product conceptualization should be given the highest priority when making a decision to comply with maqasid shariah as per the mspdf framework, followed by both product development and commercialization. the experts agreed that the product conceptualization stage was the most critical stage because it sets a foundation for any product to be a better product. one of the experts said, “in product conceptualization which is more or less similar to product development in terms of its process, idea generation which could be derived from the product development department or from the shariah department possesses critical value in ensuring that the product from its root to its fruit (end product) follow exactly what has been prescribed in the shariah regulations.” 4.4 summary of findings in summary, this study concluded that in developing a new product, during the three stages of product development used in islamic banks, partnership-based home financing products must be given priority over sale-based home financing products. considering the stages of product development, in the first two stages of product conceptualization and development, “spiritual and mental wellness” and “socioeconomic justice” should be given priority when planning a new product. these two aspects represent the preservation of wealth objective of shariah according to ibn ashur (2006) which supports that the objective in wealth preservation is to support and position the spiritual state of humans. when the product reaches the stage of commercialization, institutional and staff readiness should be taken into consideration. in summary, all the elements in mspdf should be given attention when developing a new product with a different priority. these elements are consistent with the current strategy of the united nation’s sustainable development goals where the ultimate aim is a call for a global response to end poverty, protect the planet, and ensure that by 2030 all people enjoy peace and prosperity. figure 7 shows the overall mspdf hierarchy model based on the goals of conducting the ahp in ranking islamic banking home financing products in each stage of product development. ijahp article: shahwan, sapian, ibrahim, mohammed, agha, kanesan/ranking the compliance of islamic banking home financing products with maqasid shariah using ahp international journal of the analytic hierarchy process 1 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.891 figure 7 maqasid shariah-based product development framework (mspdf) hierarchy model ijahp article: shahwan, sapian, ibrahim, mohammed, agha, kanesan/ranking the compliance of islamic banking home financing products with maqasid shariah using ahp international journal of the analytic hierarchy process 1 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.891 5. conclusion the results showed that the partnership-based home financing product ranked at 87.5% in conforming to the social requirements of maqasid while sale-based home financing had a ranking score of 12.5%. the score also indicated that both products complied with the social requirements of maqasid shariah with different rankings. these findings support the view of previous researchers who argued that partnership-based home financing represented by mm is theoretically superior to sale-based home financing represented by bba & t in terms of observing maqasid shariah in practice. however, this study also found that by using the ahp to give the rankings, the experts did not totally reject sale-based home financing as not complying with the social requirements of maqasid shariah in all the stages of product development, but only to a lesser degree than partnership-based home financing. this is a strength of the ahp, its ability to give a rank rather than a total rejection of the product. the use of the ahp helped settle the long-debated issue about sale-based and partnership-based financing with regards to maqasid compliance. countless future studies could benefit from this framework. future researchers may use it for other financial instruments like vehicle financing, personal financing and corporate financing. prioritization also could be crafted from customers’ experience and perception with regard to fulfilling maqasid requirements. . ijahp article: shahwan, sapian, ibrahim, mohammed, agha, kanesan/ranking the compliance of islamic banking home financing products with maqasid shariah using ahp international journal of the analytic hierarchy process 2 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.891 references aaoifi (2017). aaoifi shariah standards. manama: the accounting and auditing organization for islamic financial institutions. abduh, m., & omar, m. a. (2012). islamic-bank selection criteria in malaysia: an ahp approach. business intelligence journal, 5(2), 271-281. agha, e. (2015). current practice of islamic finance: a trade-off between maqāṣid al-sharīʿah orientation and profit maximization. majallah al-iqtisad al-islami al-alami, 43(5), 113–120. ahmed, habib. (2011a). product development in islamic banks. edinburgh: edinburgh university press. ahmed, habib. (2011b). maqasid al-shari'ah and islamic financial products: a framework for assessment. isra international journal of islamic finance, 3(1), 149-160 al-ghazali, abu hamid muhammad bin muhammad. (1997). al-mustasfa min 'ilm al-usul (1st ed. vol. 1). beirut: muassasah al-risalah ali, m.m. and hassan, r. (2020). survey on sharīʿah non-compliant events in islamic banks in the practice of tawarruq financing in malaysia. isra international journal of islamic finance, 12(2), 151–169. doi: https://doi.org/10.1108/ijif-07-2018-0075 al-raysuni, a. (2006), imam al-shatibi’s theory of the higher objectives and intents of islamic law. herndon, virginia: islamic book trust. doi: https://doi.org/10.2307/j.ctvkjb1w9 al-yubi, m. (2008). maqāṣid al-sharīʿah al-islāmiyyah wa ʿalāqatuhā bi al-adillah alsharʿiyyah (1st ed.). damascus: dar ibn al-jawzi. doi: https://doi.org/10.2307/j.ctvkc67vz.4 alziyadat, n. and ahmed, h. (2019). ethical decision-making in islamic financial institutions in light of maqasid al-sharia: a conceptual framework. thunderbird international business review, 6(5), 707–718. doi: https://doi.org/10.1002/tie.22025 asadov, a., sori, z.b.m., ramadilli, s.m., anwer, z. and shamsudheen, s.v. (2018). musharakah mutanaqisah home financing: issues in practice. journal of islamic accounting and business research, 13(6), 89–99. doi: https://doi.org/10.1108/jiabr-08-2015-0036 asni, f. and sulong, j. (2018). non-ethical elements in tawarruq munazzam product for islamic housing financing in malaysia. international journal of academic research in business & social science, 11(4), 45–55. doi: https://doi.org/10.6007/ijarbss/v8-i12/5149 ʿatiyyah, g.e. (2010). towards realization of the higher intents of islamic law. herndon, virginia: islamic book trust. auda, j. (2015). what are maqāṣid al-shariah? herndon, virginia: islamic book trust. bnm (2014). regulator document: introduction of new products (bnm/rh/std 028-5). kuala lumpur: bank negara malaysia. bnm (2020). islamic banking bulletin, kuala lumpur: bank negara malaysia. bnm (2010). shariah resolutions in islamic finance, 2nd ed. kuala lumpur: bank negara ijahp article: shahwan, sapian, ibrahim, mohammed, agha, kanesan/ranking the compliance of islamic banking home financing products with maqasid shariah using ahp international journal of the analytic hierarchy process 3 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.891 malaysia. chakravarty, s.r. (2003). a generalized human development index. review of development economics, 7(1), 99–114. cooper, r. and de brentani, u. (1991). new industrial financial services: what distinguishes the winners. journal of product innovation and management, 8(2), 75-90. ecer, f. (2014). a hybrid banking websites quality evaluation model using ahp and copras-g: a turkey case. technological and economic development of economy, 20(4), 758-782. doi: https://doi.org/10.3846/20294913.2014.915596 government of malaysia (2011). malaysian quality of life 2011. economic planning unit prime minister’s department. haneef, m.a. and furqani, h. (2011). methodology of islamic economics: overview of present state and future direction. international journal of economics, management and accounting, 19(1), 67–77. hassan, m. kabir, & lewis, mervyn k. (2007). product development and shariah issues in islamic finance. thunderbird international business review, 49(3), 281-284. haywood-farmer, j. (1988). a conceptual model of service quality. international journal of operations & production management, 41(6), 123–134. hurayra, a. (2015). achievement of maqasid al-shari`ah in islamic banking: an evaluation of islami bank bangladesh limited. global journal of computer science and technology: a hardware & computation, 15(1), 34–44. ibn ʿāshūr, m. al-ṭāhir (2006). maqāṣid al-sharīʿah al-islāmiyyah. riyadh: dār al-saḥnūn li al-nashr wa al-tawzīʿ. doi: https://doi.org/10.1163/1573-3912_ei3_com_30709 ibn ashur, muhammad altahir. (2006). treatise on maqasid al-shari'ah (mohammad eltahir el-mesawi, trans.). washinghton: the international institute of islamic thought (iiit) and islamic book trust (south east asia publisher). doi: https://doi.org/10.2307/j.ctvkc673b ibn manẓūr, m. (2002). lisān al-ʿarab. aleppo: dār iḥyāʾ al-turāth al-ʿarabī. isra (2016). islamic financial system: principles & operations, 2nd ed. kuala lumpur: international shari’ah research academy for islamic finance. doi: https://doi.org/10.32350/ibfr.2014.01.01 juliyanti, w. and wibowo, y.k. (2021). literature review: implementation of musharakah mutanaqisah partnership over the world. bukhori: kajian ekonomi dan keuangan islam, 1(1), 1–10. doi: https://doi.org/10.35912/bukhori.v1i1.196 kashi, a. and mohamad, a. (2017). does musharakah mutanaqisah converge with bai bithamin ajil and conventional loans?. international journal of law and management, 59(5), 740-755. doi: https://doi.org/10.1108/ijlma-04-2016-0044 kristina, e., & atmojo, w. t. (2022). decision support system selecting money market mutual fund using ahp method. jurteksi (jurnal teknologi dan sistem informasi), 8(3), 343350. doi: https://doi.org/10.33330/jurteksi.v8i3.1618 https://doi.org/10.3846/20294913.2014.915596 ijahp article: shahwan, sapian, ibrahim, mohammed, agha, kanesan/ranking the compliance of islamic banking home financing products with maqasid shariah using ahp international journal of the analytic hierarchy process 4 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.891 lahsasna, ahcene, & hassan, m. kabir. (2011). the shariah process in product development and approval in icm. in m. kabir hassan & m. mahlknecht (eds.), islamic capital markets: products and strategies (pp. 23-68). west sussex: john wiley & sons. mansoori, m.t. (2011). use of hiyal in islamic finance and its shari'ah legitimacy. journal of islamic business and management, 1(1), 152–176. marcarelli, g., & mancini, p. (2022). school and academic performance for ranking high schools: some evidence from italy. international journal of the analytic hierarchy process, 14(2), 1-20. doi: https://doi.org/10.13033/ijahp.v14i2.948 meera, m. and razak, d. (2005). islamic home financing through musharakah mutanaqisa and al bai bithman ajil: a comparative analysis. review of islamic economics, 9(2), 23–35. mohd yassin, n. (1997). islamic banking: case commentaries involving al-bay’ bithaman ajil. kuala lumpur: center for research and training (cert). mustafa, o.m. (2007). objectives of islamic banking: maqasid approach. international conference on jurisprudence, kuala lumpur. mohammad, m.o. and shahwan, s., (2013). the objective of islamic economics and islamic banking in light of maqasid al-shariah: a critical review. middle east journal of scientific research, 5(2), 23–34. muda, muhamad, & abdullaah, jalil. (2007). islamic financial product development: shariah analysis. paper presented at the iium international conference on islamic banking and finance, crown plaza hotel, kuala lumpur. mydin meera, a.k. and abdul razak, d. (2009). home financing through the musharakah mutanaqisah contracts: some practical issues. journal of king abdulaziz university: islamic economics, 22(1), 56–66. doi: https://doi.org/10.4197/islec.22-1.1 naseer, s., song, h., aslam, m. s., abdul, d., & tanveer, a. (2022). assessment of green economic efficiency in china using analytical hierarchical process (ahp). soft computing, 26(5), 2489-2499. nor, n.m. (2008). musharakah mutanaqisah as an islamic financing alternative to bba. malaysian islamic finance monthly, 3(9), 22–24. nor, m.z.m., mohamad, a.m., azhar, a., latif, h.m., khalid, a.-h.m. and yusof, y. (2019). legal challenges of musharakah mutanaqisah as an alternative for property financing in malaysia. journal of legal, ethical and regulatory issues, 22(3), 1–8. osmani, n.m. and abdullah, m.f. (2010). musharakah mutanaqisa home financing: a review of litratures and practices of islamic banks in malaysia. international review of business research papers, 6(2), 32–40. qian, s. (2022). construction of financial credit risk evaluation system model based on analytic hierarchy process. in the international conference on cyber security intelligence and analytics (pp. 488-496). springer, cham. doi: https://doi.org/10.1007/978-3-030-969080_61 ijahp article: shahwan, sapian, ibrahim, mohammed, agha, kanesan/ranking the compliance of islamic banking home financing products with maqasid shariah using ahp international journal of the analytic hierarchy process 5 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.891 saaty, t.l. (1990). how to make a decision: the analytic hierarchy process. european journal of operational research, 48(8), 12–23. doi: https://doi.org/10.1016/0377-2217(90)90057-i saaty, t.l. (2008). decision making with the analytic hierarchy process. international journal of services sciences, 1(1), 45–55. saiful azhar, r. (2010), critical issues on islamic banking and financial markets. kuala lumpur: dinamas publishing. sekaran, u. and bougie, r. (2016). research methods for business: a skill building approach. new york: john wiley & sons. shahwan, s., mohammad, m. o., and rahman, z. a. (2013). home financing pricing issues in the bay’ bithaman ajil (bba) and musharakah mutanaqisah (mmp). global journal althaqafah, 3(2), 23-36. shahwan, s. (2017). evaluation of islamic banking product development process based on maqāṣid al-sharīʿah: the case of sale and partnership-based home financing in malaysia. phd thesis, international islamic university malaysia, kuala lumpur. doi: https://doi.org/10.33826/afmj/v4i7.01 sharma, j., arora, m. sonia and alsharef, a. (2022). an illustrative study on multi criteria decision making approach: analytical hierarchy process. 2022 2nd international conference on advance computing and innovative technologies in engineering (icacite). 2000-2005. doi: 10.1109/icacite53722.2022.9823864. soualhi, y. (2015). application of shariah contracts in contemporary islamic finance: a maqasid perspective. intellectual discourse, 23, 333–354. tarique, k. m., islam, r., mohamed, m. o., razak, d. a., & bin hamdan, h. (2020). constructing a maqasid (objective) based performance measurement index for islamic banks. international journal of the analytic hierarchy process, 12(2), 328-354. doi: https://doi.org/10.13033/ijahp.v12i2.770 taticchi, p., balachandran, k. and tonelli, f. (2012). performance measurement and management systems: state of the art, guidelines for design and challenges. measuring business excellence, 16(2), 41-54. doi: https://doi.org/10.1108/13683041211230311 yana, a. a. (2022). application of ahp method in determining bank selection for account opening. jurnal mandiri it, 11(1), 26-33. ijahp article: el hadidi, el-dash, besiouny, meshref/evaluation of building life cycle cost (lcc) criteria in egypt using the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 1 vol. 14 issue 2 2022 issn 19366744https://doi.org/10.13033/ijahp.v14i2.958 evaluation of building life cycle cost (lcc) criteria in egypt using the analytic hierarchy process (ahp) omia el hadidi omaya1411@gmail.com human_ib@yahoo.com karim el-dash professor, department of project management, civil engineer, misr university for science and technology, egypt, karim.eldash@must.edu.eg mohamed besiouny dean sinai faculty of engineering, professor, department of environmental and sanitary, civil engineer, sinai university, egypt, mohamed.basiouny@su.edu.eg ahmed meshref doctor, department of project management, civil engineer, benha university, egypt, dr.ahmednouh@bhit.bu.edu.eg abstract recently, life cycle cost (lcc) has gained a wide acceptance in the field of industrial building construction, where it is categorized under economic sustainability in the overall sustainability of buildings. hence, it is necessary to think about the categories and criteria that affect the building’s cost over its lifespan. in this study, the analytical hierarchy process (ahp), a multi-criteria decision-making methodology, is employed to evaluate and weight four categories which are building blocks of the lcc of industrial building construction. the assessment model applies seventeen criteria which are distributed under the following four categories: initial cost, operating or maintenance cost, environmental impact cost, and the end of life. these are evaluated by thirty-seven civilian experts responding to a pair wise questionnaire. the results are significant as they reflect the viewpoints of the civilian experts and can aid in the development of a building's economic sustainability by illuminating the impact factors of the life cycle cost of buildings. to the best of our knowledge, this is the first study to handle criteria evaluation of lcc for sustainable building using the ahp multi-criteria decision-making (mcdm) methodology. keywords: life cycle cost (lcc); lcc categories; lcc criteria; sustainable building in egypt; economic sustainability; analytic hierarchy process (ahp) 1. introduction there has been a growing need to consider building costs and develop methods to evaluate life cycle costs (lcc). prior to 1970, the initial capital cost was the only mailto:omaya1411@gmail.com mailto:dr.ahmednouh@bhit.bu.edu.eg ijahp article: el hadidi, el-dash, besiouny, meshref/evaluation of building life cycle cost (lcc) criteria in egypt using the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 2 vol. 14 issue 2 2022 issn 19366744https://doi.org/10.13033/ijahp.v14i2.958 investment choice considered for many clients. a number of reports have upheld the necessity of thinking through the long-term cost of project choices, which is called a life cycle cost assessment. presently, sustainable projects support the implementation of lcc methods since they can deliver an evaluation of the long-term cost of industrial building construction. this study will launch an initiative to apply economic sustainability of industrial buildings construction using a life cycle cost assessment. a systematic methodology is applied to evaluate lcc selection criteria of industrial buildings construction in egypt by establishing an ahp questionnaire to achieve high performance in economic sustainability for the new buildings. civilian experts evaluated the selected criteria from the previous literature. the relative importance weights were calculated by the analytic hierarchy process (ahp), a pair-wise comparison method. these criteria were developed under the following categories of lcc (nrm 1, 2012; nrm 3, 2014; rics, 2016): initial cost, operating and maintenance cost, environmental impact cost, and end of life cost. the questionnaire was divided into three sections; the first section provided the selected life cycle costs criteria and definitions; the second section provided the calculated relative weights based on the pair-wise matrix and the scale ranges between one and nine provided by civilian experts; and the third section determined the consistency of the responses. the main problem was to determine the lcc categories and criteria and evaluate them in order to make a decision from amongst the long-term cost of industrial buildings construction choices. therefore, this study presents an ahp model to evaluate and weight the selected categories and criteria of the lcc of industrial buildings construction by civilian experts responding to a pair-wise questionnaire. the ahp is a systemic technique that helps make complex decisions. the method supports decision makers in choosing the optimal option for their needs based on their comprehension of the problem. it represents an organized and thorough framework for structuring a problem, representing and quantifying its aspects, connecting the identified aspects to goals, and evaluating alternate solutions. the objective of this research is to develop a multi-criteria decision-making (mcdm) methodology by creating an ahp model of the four lcc categories, and seventeen criteria for industrial building construction. this will allow the discussion of mechanisms for achieving sustainable economic development dimensions, and the application of the true concept of lcc management for the construction of industrial projects to achieve economic recovery in the real estate market. 2. literature review 2.1 life cycle cost concept and analysis of the construction of industrial buildings in the process of creating a building, early decisions have the greatest impact, hence the need for life cycle costing (nrm 1., 2012; nrm 3., 2014; rics, 2016; iso/tc59, b.c. iso, 2017)). life cycle costing is an economic quantitative estimation method (iso/tc59, b.c. iso, 2017; reisinger et al., 2022). this method evaluates the entire cost of a building over its operating life (zhang et al., 2018; noshadravanet al., 2017). the operating life includes initial capital costs, maintenance costs, operating costs and the ultimate disposal of the asset at the end of its life (noshadravanet al., 2017). ijahp article: el hadidi, el-dash, besiouny, meshref/evaluation of building life cycle cost (lcc) criteria in egypt using the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 3 vol. 14 issue 2 2022 issn 19366744https://doi.org/10.13033/ijahp.v14i2.958 determination of the economic effects of the alternatives is an important step in a lcc analysis study. a literature review was conducted to extract and coordinate common independent variables related to lcc analysis (tu& huang, 2013; tu, taur & lin c., 2016; ahmed & arocho, 2021; marreroet al., 2022). the variables that are applied for lcc are building area, floor height, number of floors, structure and envelope type, building age, and year built. a previous study in egypt aimed to evaluate the effects of façade retrofit measures in residential buildings in cairo using life-cycle cost analysis (medhat et al., 2021). in egypt, despite the high initial cost, the high-performance glazing systems (hpgss) are supposed to be economically feasible in the lcc. youssef (2022) investigated and compared the economic feasibility of three hpgss for an office building in new cairo with the help of mhuc (ministry of housing, utilities & urban communities). in terms of life cycle cost (lcc), the low-e, electro chromic, and photovoltaic glazing systems were compared to clear double-glazing as a conventional system (youssef et al., 2021). in the operation phase of lcc, the rapid and continuous rise in energy costs dictates significant cost control efforts despite the fact that achieving remarkable energy cost reductions is highly governed by the decisions made during earlier phases of the industrial construction project (yussra et al., 2019). 2.2 objectives of the ahp process thomas l. saaty, a mathematician, developed the ahp method (saaty,1990; saaty, 2000; saaty, 2013; ferretti, & saaty, 2014). this method is a framework for operative decision-making of complex problems by simplifying the decision-making process while dividing the issues into sections. these sections or variables are arranged in a hierarchical order with the importance of each variable assigned numerical values which are used to determine which variable has the highest priority and ability to influence the outcome (saaty, 2005; ferretti, & saaty, 2014; alitaneh, 2019). the ahp method helps solve complicated problems by developing a framework hierarchy of criteria. the ahp also integrates the strengths of the various issues of reasoning, and then aggregates the results that are consistent with our estimates as previously presented (salgado, et al., 2012). saaty solved the problem of how to stimulate and discuss the key characteristics of multicriteria analysis techniques and work through the conceptual lens of decision processes in the field of urban and territorial transformations with the ahp according to three principles: the framework of the hierarchy, the principle of prioritization, and the logical consistency principle (ferretti & saaty, 2014). the ahp sets up a hierarchy of issues to be resolved, taking into account the criteria that support the accomplishment of the objectives (garuti& salomon, 2012; akman & dagdeviren, 2018; karaman & akman, 2018). it is important to carefully choose the criteria in the ahp objectives process to avoid problems. 3. methodology we used a two-phase research methodology in this study to determine lcc criteria priorities of sustainable buildings. in phase i, we identified the categories and criteria of lcc that are more effective in sustainable buildings with the help of a comprehensive literature review. then, we formed an expert panel (ep). the lcc categories and criteria were determined through discussions and the support of the ep. in phase ii, the priorities of the sustainable building’s lcc categories and criteria ijahp article: el hadidi, el-dash, besiouny, meshref/evaluation of building life cycle cost (lcc) criteria in egypt using the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 4 vol. 14 issue 2 2022 issn 19366744https://doi.org/10.13033/ijahp.v14i2.958 were evaluated via the widely recognized ahp methodology by surveying civil professionals. we present a systematic research methodology in figure 1. figure 1 systematic two-phase research methodology 3.1 phase i criteria selection during the criteria selection stage, it is important to ensure that all aspects utilized in making the decision are covered in order to obtain the required objectives. each of these criteria should be defined in order to help decision-makers appreciate the suggested alternatives. criterion that have the same meaning should be avoided which is why we set disciplined standards based on the desired goal (garuti & salomon, 2012; an, 2014). the defined categories and criteria for lcc of industrial building construction included in this research are a result of a combination of information from the existing literature and civil experience. detailed information about what is included under each category of selected criteria is given. the descriptions and references of the selected lcc categories and criteria are summarized in table 1. determine the weights of the the sustainable buildings lcc categories and criteria ahp method implement the questionnaire by the civil professionals phase ii finalize lcc categories and criteria of the sustainable buildings with the support of the ep form an expert panel (ep) identify the sustainable building's lcc categories and criteria phase i ijahp article: el hadidi, el-dash, besiouny, meshref/evaluation of building life cycle cost (lcc) criteria in egypt using the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 5 vol. 14 issue 2 2022 issn 19366744https://doi.org/10.13033/ijahp.v14i2.958 table 1 categories and criteria description and references related to building life cycle cost category selected criteria descriptions references initial cost construction industrial cost tu k. j. & huang y. w. (2013); medhat k., sherif e.& osama t., (2021); tu k., taur y. &lin c. (2016); domenico a.n., hitapriya s. & tri joko w. a. (2020); kaming peter f. (2017); youssef o., mona g. i., koji t.& ahmed m. a. (2021). building area the total area of building which is one of the basics of the building footprint floor height the height of each building story in unit length which is one of the basics of the building footprint numbers of floors number of stories (e.g. 1,2,3….) which is one of the basics of the building footprint structure & envelope type includes steel, concrete, wood, and precast concrete in various combinations building age the study period in years (e.g. 15, 25, 30 years) location city cost index of a location varies in different cities year of industrial construction this parameter includes the projects that are built within the study period operating and maintenance refers to hard facilities management costs including operating costs such as cleaning and energy costs, maintenance costs, and other costs. iso/tc59, b.c. iso (2017); rics (2016); nrm 1, (2012); gao x. & pishdadbozorgi p. (2018); ibrahim a. (2001); medhat k., sherif e.& osama t., (2021); nrm 3 (2014); yussra m. r., ibrahim n. , khaled n., islam a. m.&meshary g. (2019); kaming peter f. (2017); yussra m. r., ibrahim n. , khaled n., islam a. m.& meshary g. (2019). energy cost energy used for heating and lighting. catering and services general support services, communications and security services, letting fees, facilities management fees, caretaker and janitorial services, service transport, it services, and laundry and linen services, e.g. internal deliveries. cleaning waste management and disposal. major repairs redecoration, renovation, rehabilitation, replacement. periodic maintenance the cost of contractors who perform skilled jobs, such as sanitation and hvac services. rent and insurance insurance rates and other local taxes and charges. environmental impact cost environmental cost is a reference to the cost of greenhouse gas (ghg) emissions, which are produced during industrial construction of concrete and which has effects on the environment. iso/tc59, b.c. iso (2017); nrm 3, (2014); iso 15686 part 5, (2013); rics, (2016); kaming ijahp article: el hadidi, el-dash, besiouny, meshref/evaluation of building life cycle cost (lcc) criteria in egypt using the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 6 vol. 14 issue 2 2022 issn 19366744https://doi.org/10.13033/ijahp.v14i2.958 category selected criteria descriptions references structure and envelope material waste cost waste gathered from all stages such as production of raw materials, manufacturing concrete, placing concrete in the location, and demolition. (2017); zhang, c., cao, l. w., & romagnoli, a, (2018); point carbon website. market price of resultant co2 cost of controlling gas emissions. end of life this includes disposal and demolition, but specifically includes the worth of alternatives at the end period of lcc analysis. iso/tc59, b.c. iso (2017); yussra m. r., ibrahim n. , khaled n., islam a. m.&meshary g. (2019); rics, (2016); domenico a.n., hitapriya s. & tri joko w. a. (2020); noshadravan a., miller t. r. & gregory j. g., (2017). salvage and recycling recycling, the conversion of building waste into new objects. demolition cost building demolition waste such as materials, aggregate, concrete, wood, and metal. life cycle cost components for buildings can be broken down into several elements in a hierarchy structure, as shown in table 1. the first level contains the four cost categories of initial costs, operating and maintenance costs, and end of life cost (salvage value & demolition cost). since parties in the kyoto protocol committed to reach their targets through reducing ghg emissions over the 2008-2012 commitment period and since the protocol allows countries that have emission units to sell the extra capacity to countries that are over their targets (united nations, 2011), the environmental impact costs are added as future costs based on the prices and quantities of ghg converted into co2 and structure and envelope material waste costs. the second level consists of the seventeen criteria, which can be broken down into the four categories. for example, initial costs include building area, floor height, numbers of floors, structure & envelope type, building age, location city, and year of industrial construction, while operating and maintenance costs includes energy cost, catering and services, cleaning, major repairs, periodic maintenance, and rent and insurances. 3.2 phase ii – ahp comparative judgment means building judgments about the proportional weight of two criteria at a given level in relation to the above levels. this evaluation is the core of the ahp, as it will influence the preference of the criteria. we used a pair wise comparison matrix to establish the evaluation results (an, 2014). to achieve effective metrics when comparing two criteria, we needed to understand the general purpose. the benchmark reference in table 1 was used to complete the scale of comparative importance in pairs according to saaty. we applied an ahp survey to develop the selected criteria of lcc for buildings; this survey proposed the effective execution of lcc implementation for new buildings. therefore, we looked for the most effective lcc criteria from the latest literature. in ijahp article: el hadidi, el-dash, besiouny, meshref/evaluation of building life cycle cost (lcc) criteria in egypt using the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 7 vol. 14 issue 2 2022 issn 19366744https://doi.org/10.13033/ijahp.v14i2.958 order to choose the best priorities of the selected criteria, thirty-seven civilian experts evaluated the chosen criteria using saaty’s 1-9 scale. the main objective of the applied survey was to collect the decision makers’ opinions and then calculate the relative weights for the selected criteria by using the ahp pairwise comparison method (ferretti, & saaty, 2014; alitaneh, 2019). in this context, the ahp hierarchy for the effective categories and criteria of industrial building construction lcc are given in figure 2. figure 2 hierarchy of the ahp model 3.2.1 calculations of sample size we can statistically calculate the required sample size according to equation (1) (montgomery, 1998). where, n is sample size, zα/2 is a critical value from statistical tables, p is a percentage of the target sample population to the total population, and d is the accepted error percentage. criteria categories goal hierarchical grouping of the effective criteria of building construction llc initial cost building area floor height numbers of floors structure & envelope type building age location city year of construction operation and maintenance energy consumption cost catering and services cleaning major repairs periodic maintenance rent and insurance environmental impact cost structure and envelope material waste cost market price of resultant co2 end of life cost salvage and recycling demolition cost ijahp article: el hadidi, el-dash, besiouny, meshref/evaluation of building life cycle cost (lcc) criteria in egypt using the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 8 vol. 14 issue 2 2022 issn 19366744https://doi.org/10.13033/ijahp.v14i2.958     2 2 2 * * 1  z p p n d  (1) for a target sample population of 10,000 and 22,729 for contractors and consultants respectively, and a total population of 182,703 civil engineers (all registered civil engineers in all departments of the egyptian engineers syndicate), the assumed accepted error percentage in this method is 10% ; zα/2 = 1.645 and the minimum sample size is calculated to be 15. thirty-seven civilian experts participated in the research. 3.2.2 survey study a web-based survey was applied based on pilot study feedback. it was distributed to about tens experts including building managers, consultants, academics, and contractors both in and out of egypt. this study was conducted in english. we received 37 responses and excluded five of them. the responses were collected electronically through the web-based system and the egyptian engineers syndicate. the questionnaires were scanned many times and discussed with certain civilian experts. 3.2.3 evaluation of selected criteria by the civilian experts once the hierarchy of a problem has been established, the decision makers must evaluate and compare its different elements to each another. in making the comparisons, the decision makers can use their judgments about the elements or they can use real data, or a combination of both. the main attribute of the ahp is that human judgments, and not just the underlying information, can be used to perform the evaluations. we started by asking the experts to fill out half of an ×n matrix a of criteria,𝑖 and 𝑗 about a preference numerical value,𝑎𝑖𝑗from the 1 to 9scale as in table 2.the matrix a is written below in equation (2). table 2 saaty’s fundamental rating scale (saaty, 1990, 2000) scale definition explanation 1 equal importance two activities contribute equally to the objective 3 moderate importance of one over another experience and judgment moderately favor one activity over another 5 essential or strong importance experience and judgment strongly favor one activity over another 7 very strong importance activity is very strongly favored and its dominance demonstrated in practice 9 extreme importance evidence favors one activity over another is of the highest possible order of affirmation 2, 4, 6, 8 intermediate values between the two adjacent judgments compromise is needed ijahp article: el hadidi, el-dash, besiouny, meshref/evaluation of building life cycle cost (lcc) criteria in egypt using the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 9 vol. 14 issue 2 2022 issn 19366744https://doi.org/10.13033/ijahp.v14i2.958 (2) other entries of matrix 𝐴 are determined by 𝑎𝑖𝑖=1, 𝑎𝑗𝑖 =1/ 𝑎𝑖, assuming 𝑎𝑖𝑗≠0. after that, we reversed the values in the other half of matrix. this method evaluates and quantifies the relative weights of each matrix 𝐴 dividing the entries into the columns to the respective column sums for the selected categories and criteria. the relative weights are normalized and calculated based on the pair-wise matrix and the scales provided by experts as below in equation (3). 𝑨𝒘=𝝀𝒎𝒂𝒙𝒘 (3) where 𝑤 is the principal eigenvector corresponding to the largest eigenvalue 𝜆𝑚𝑎𝑥. tables 3 to 7 represent the average of the pair-wise comparison matrix for 32 experts of the selected categories and criteria. table 3 pair-wise comparison matrix of lcc categories lcc categories initial costs operating and maintenance environmental impact cost end of life cost initial costs 1.00 2.00 2.00 3.00 operating and maintenance 0.50 1.00 2.00 2.00 environmental impact cost 0.50 0.5 1.00 4.00 end of life cost 0.33 0.50 0.25 1.00 ijahp article: el hadidi, el-dash, besiouny, meshref/evaluation of building life cycle cost (lcc) criteria in egypt using the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 10 vol. 14 issue 2 2022 issn 19366744https://doi.org/10.13033/ijahp.v14i2.958 table 4 pair-wise comparison matrix of initial cost criteria table 5 pair-wise comparison matrix of operating and maintenance cost criteria operating and maintenance criteria energy consumption cost catering and services cleaning major repairs periodic maintenance rent and insurance energy cost 1 3 1 2 3 4 catering and services 0.33 1 1 0.33 0.5 4 cleaning 1 1 1 0.5 0.33 2 major repairs 0.5 3 2 1 2 4 periodic maintenance 0.33 2 3 0.5 1 4 rent and insurance 0.25 0.25 0.5 0.25 0.25 1 table 6 pair-wise comparison matrix of environmental impact cost criteria environmental impact cost criteria structure and envelope material waste cost market price of resultant co2 structure and envelope material waste cost 1.00 2.00 market price of resulted co2 0.50 1.00 initial cost criteria building area floor height no. of floors structure & envelope type building age location city year built building area 1 2 1 2 6 5 3 floor height 0.5 1 0.5 2 3 4 2 no. of floors 1 2 1 3 4 6 3 structure & envelope type 0.5 0.5 0.33 1 4 5 2 building age 0.167 0.33 0.25 0.25 1 0.33 0.5 location city 0.2 0.25 0.167 0.2 3 1 0.5 year built 0.33 0.5 0.33 0.5 2 2 1 ijahp article: el hadidi, el-dash, besiouny, meshref/evaluation of building life cycle cost (lcc) criteria in egypt using the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 11 vol. 14 issue 2 2022 issn 19366744https://doi.org/10.13033/ijahp.v14i2.958 table 7 pair-wise comparison matrix of the end of life cost criteria end of life cost criteria salvage value demolition cost salvage and recycling 1.00 3.00 demolition cost 0.33 1.00 the ahp utilizes five pair-wise comparison matrices consisting of various factors. the previous pair-wise comparison matrices provided the importance ratio for each pair of alternatives. each matrix is a mutual matrix in which the main diagonal elements are ‘one’ and the values above the diagonal are the reciprocal of those below. the relative importance of each category and sub category are based on a 1-9 scale with the interpretations presented in table 2. the ahp converts each different evaluation to a numerical value that can be easily processed and compared over the whole range of the problem. in the next step, a numerical weight is determined for each element of the hierarchy, which often allows incommensurable and varied elements to be compared to each other in a rational and consistent way. this feature distinguishes the ahp from other techniques. next, numerical priorities are estimated for each alternative in the final step of the process. these numbers represent the alternatives' relative ability to achieve the main goal, which allows a simple consideration of the various courses of action. 3.2.4 consistency analysis of the responses all responses are submitted to a consistency test. the consistency ratio is calculated by dividing the consistency index value ci by the random consistency ri index value as below in equation (4). we calculated the ci using equation (5), while the ri value was obtained from table 8, and this value depends on a size n matrix. if ci equals 0, the matrix is consistent. where the cr value of any matrix is less than 10%, the inconsistency of the responses is still considered acceptable (ergu, kou, peng & shi, 2011). we obtained 37 responses, and excluded five due to their high consistency ratio. table 9 represents the value of λmax, calculated by dividing the vector weight by the relative weight of each criterion, the consistency index value ci, and consistency ratio cr for all the previous matrixes. cr = ci / ri (4) ci = (λmax – n)/ (n -1) (5) table 8 random inconsistency index (ri) for n=1, 2…10 n 1 2 3 4 5 6 7 8 9 10 ri 0 0 0.58 0.90 1.12 1.27 1.32 1.41 1.54 1.49 there are different consistency ratios cr, and the value of λmax and ci for all the previous matrices are shown in table 9. when the consistency ratio of the responses for lcc categories, initial cost criteria, operating and maintenance criteria are 0.082, 0.042, 0.077, respectively, which less than 0.1 (ergu, kou, shi & shi, 2011), the ijahp article: el hadidi, el-dash, besiouny, meshref/evaluation of building life cycle cost (lcc) criteria in egypt using the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 12 vol. 14 issue 2 2022 issn 19366744https://doi.org/10.13033/ijahp.v14i2.958 consistency is considered acceptable. when the cr was decreased to zero, the comparison matrix was completely consistent as in the environmental impact cost criteria and end of life cost criteria comparison matrix. table 9 value of λmax, ci, cr for all the previous matrices matrix name lcc categories initial cost criteria operating & maintenance criteria environmental impact criteria end of life criteria λmax 4.22 7.33 6.49 2 2 ci 0.074 0.55 0.098 0 0 cr 0.082 0.042 0.077 0 0 4. results and discussion 4.1 results of ahp weights and ranking lcc is a sustainable economic tool to evaluate the cost performance of industrial building construction. it aims to make choices between different categories to achieve the client's goals when those categories are not only in their initial costs but also in their subsequent operation and maintenance costs, environmental impact costs, and end of life costs. the ahp allows the weighting and ranking of the alternatives of categories and criteria to be compared on the same basis. it is used for option appraisal by the weighting of lcc categories and criteria. the first comparison of the study (shown in table 10) includes initial costs, operation and maintenance cost, environmental impact cost, and end of life cost. table 10 ahp weights and rank of the lcc categories lcc categories weight rank initial costs 0.40 1 operating and maintenance 0.27 2 environmental impact cost 0.23 3 end of life cost 0.10 4 sum 1.00 the second ahp ranking of the study was applied for the paired matrix criteria of each category. as shown in table 11, the initial cost criteria were weighted and ranked. the civilian experts ranked number of floors and building area at the top of the initial cost criteria with 27% and 26%, respectively. whereas, floor height and structure & envelope type ranked lower with 16% and 14%, respectively. at the bottom of the ranking, was year built, followed by location (city) then building age. ijahp article: el hadidi, el-dash, besiouny, meshref/evaluation of building life cycle cost (lcc) criteria in egypt using the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 13 vol. 14 issue 2 2022 issn 19366744https://doi.org/10.13033/ijahp.v14i2.958 table 11 ahp weights and rank of the initial cost criteria initial cost criteria weight rank building area 0.26 2 floor height 0.16 3 no. of floors 0.27 1 structure & envelope type 0.14 4 building age 0.04 7 location city 0.05 6 year built 0.08 5 sum = 1.00 in table 12, weighting and ranking of the operating and maintenance cost criteria reveal that the energy cost (30%) has the top priority in sustainable building followed by major repairs (23%), periodic maintenance (18%), cleaning (12.4%), catering and services (11%), and rent and insurance (4.8%). rics (2016) discusses that lower expenditure on building fabric or insulation might lead to higher energy expenditure and a more expensive cladding system might lead to savings on frame and foundation costs; however, this will also cost more to renew and using a cheaper component might be less durable and require replacement or maintenance more frequently. the logic behind our results may be a result of the emphasis on the professional civilian experts and the value of the collected data. ijahp article: el hadidi, el-dash, besiouny, meshref/evaluation of building life cycle cost (lcc) criteria in egypt using the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 14 vol. 14 issue 2 2022 issn 19366744https://doi.org/10.13033/ijahp.v14i2.958 table 12 ahp weights and rank of the operating and maintenance cost criteria operating and maintenance criteria weight rank energy cost 0.30 1 catering and services 0.11 5 cleaning 0.124 4 major repairs 0.23 2 periodic maintenance 0.18 3 rent and insurance 0.048 6 sum = 1.00 the experts evaluated structure and envelope material waste cost as 67% of the environmental impact cost criteria, whereas market price of the resultant co2 was 33% as shown in table 13. this means that the cost of controlling co2 gas emissions is about 7% of all the lcc for sustainable buildings, and waste gathered from all stages such as production of raw materials, manufacturing concrete, placing concrete at the location, renewal and replacement and demolition is about 11% of all the lcc for sustainable building. table 13 ahp weights and rank of the environmental impact cost criteria environmental impact cost criteria weight rank structure and envelope material waste cost 0.67 1 market price of resultantco2 0.33 2 sum = 1.00 the weight of salvage value and recycling at the end of life cycle criteria is about 75% (table 14). as shown in table 10 the end of life cost takes 10% of the lcc. therefore, it is logical that it would be 7.5% yield cost of the lcc (salvage and recycling) because it has 75% of the end of life cost (table 14). building demolition waste costs, such as materials, aggregate, concrete, and wood, are 2.5% of the lcc because it has 25% of the end of life cost (table 14). ijahp article: el hadidi, el-dash, besiouny, meshref/evaluation of building life cycle cost (lcc) criteria in egypt using the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 15 vol. 14 issue 2 2022 issn 19366744https://doi.org/10.13033/ijahp.v14i2.958 table 14 ahp weights and rank of the end of life cost criteria end of life cost criteria weight rank salvage and recycling 0.75 1 demolition cost 0.25 2 sum = 1.00 4.2 discussion since the lcc models in previous studies were varied in terms of categories of lcc and methods, figure 3 presents the lcc categories including initial cost, operation & maintenance cost, and energy cost that were obtained and compiled from green government office buildings in indonesia (domenico, hitapriya & tri joko, 2020). kaming (2015, 2016, 2017) presented three types of buildings which are hostels, universities, and commercial buildings in three different studies with the same categories including initial construction industrial cost, operational cost, and maintenance and replacement costs. figure 4 shows the distribution of lcc categories of commercial buildings found in his studies. figure 3 distribution of lcc categories of green government office buildings in indonesia (domenico, hitapriya & tri joko, 2020) initial cost 44% maintenance and replacement cost 39% energy cost 17% ijahp article: el hadidi, el-dash, besiouny, meshref/evaluation of building life cycle cost (lcc) criteria in egypt using the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 16 vol. 14 issue 2 2022 issn 19366744https://doi.org/10.13033/ijahp.v14i2.958 figure 4 distribution of lcc categories of commercial buildings (kaming, 2017) in this study, the civilian experts viewed initial cost as important with 40% of all of the life cycle cost followed by operation and maintenance cost and environmental impact cost, then end of life cost with 27%, 23%, and 10%, respectively as shown in figure 5. this means that the running costs are significantly more important, since they are more than 50% of the lcc of industrial building construction in egypt. whereas the operation and maintenance cost criteria revealed that the energy cost (30%) had a top priority in sustainable building followed by major repairs (23%), periodic maintenance (18%), cleaning (12.4%), catering and services (11%), and rent and insurance (4.8%), respectively (figure 6).the expert’s evaluated the structure and envelope material waste cost as 67% of the environmental impact cost criteria, whereas market price of resultant co2 was 33% as shown in figure 7. salvage value and recycling at the end of life cycle criteria is about 75% and building demolition waste cost is about 25% as shown in figure 8. figure 5 distribution of lcc categories of industrial building construction in egypt operational cost 39% maintenance and replacement cost 19% construction cost 42% initial cost 40% operation and maintenance cost 27% environmental impact cost 23% end of life cost 10% ijahp article: el hadidi, el-dash, besiouny, meshref/evaluation of building life cycle cost (lcc) criteria in egypt using the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 17 vol. 14 issue 2 2022 issn 19366744https://doi.org/10.13033/ijahp.v14i2.958 figure 6 distribution of the operation and maintenance cost criteria of industrial building construction in egypt figure 7 distribution of the environmental impact cost criteria of industrial building construction in egypt figure 8 distribution of the end of life cycle cost criteria of industrial building construction in egypt energy cost 30% major repairs 23% periodic maintenance 18% cleaning 13% catering and services 11% rent and insurance 5% structure and envelope material waste cost 67% market price of resultant co2 33% salvage value and recycling 75% demolition waste cost 25% ijahp article: el hadidi, el-dash, besiouny, meshref/evaluation of building life cycle cost (lcc) criteria in egypt using the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 18 vol. 14 issue 2 2022 issn 19366744https://doi.org/10.13033/ijahp.v14i2.958 5. conclusion  the ahp methodology was proposed to rank life cycle cost (lcc) categories and criteria of industrial buildings in egypt. this methodology is a multicriteria decision-making process. in this study, we collected data from previous reviews, and then formed an experts’ panel (ep). next, we developed a hierarchical structure with lcc categories and criteria of construction buildings.  a web-based ahp questionnaire was distributed to civilian experts inside and outside of egypt. only 37 responses were received with five responses excluded due to their high consistency ratio.  the consistency was checked for each paired matrix by dividing the consistency index value ci by the random consistency ri index value; the indication of consistency ratios (cr) for lcc categories, initial cost criteria, operating and maintenance criteria matrices are 0.082, 0.042, 0.077, respectively, which are all less than 0.1. the comparison matrix for environmental impact cost criteria and end of life cost criteria is completely consistent, as the crs equal zero.  the assessments made so far reflect that the experts' opinions to the lcc categories and criteria in general define a roadmap for building construction. the civilian expert evaluations clarified that the running costs are significantly important, as this criterion is more than 50% of the lcc, whereas initial cost is about 40% of lcc.  the civilian experts provided responses for the paired matrix criteria under each category. in the matrix of initial cost criteria, the number of floors and buildings are at the top of the initial cost criteria ranked 27% and 26%, respectively. whereas, floor height and structure & envelope type are less important with 16% and 14%, respectively. at the bottom of the ranking, we find year built, followed by location (city) then building age.  in the operating and maintenance cost criteria ahp matrix, energy cost (30%) has a top priority in sustainable building followed by major repairs (23%), periodic maintenance (18%), cleaning (12.4%), catering and services (11%), and rent and insurance (4.8%), respectively.  the assessment of environmental impact cost criteria clarified that structure and envelope material waste cost is rated 67% and market price of resultant co2 is 33%.  the ahp rank shows that the salvage value and recycling of the end of life cycle criteria is about 75%, accordingly demolition cost is 25% of the end of life cost criteria. the experts' views can enable clients or researchers to assess the current building’s lcc, and plan for economic sustainability of buildings in coming projects and investments. the study limitations are as follows:  the developed ahp model and selection framework are limited to industrial building construction in egypt.  the gathered data used are from previous studies related to lcc categories and criteria.  the results are based on civilian expert’s views, such as civil engineers in the egyptian engineers syndicate and engineering civilian services that were gathered by an electronic survey. ijahp article: el hadidi, el-dash, besiouny, meshref/evaluation of building life cycle cost (lcc) criteria in egypt using the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 19 vol. 14 issue 2 2022 issn 19366744https://doi.org/10.13033/ijahp.v14i2.958 recommendations for future research:  collect additional data points for life cycle components such as initial costs of business practices and more criteria related to energy costs, environmental impact cost criteria, and end of life cycle cost criteria.  gather additional experts’ opinions regarding the weighting of selection criteria and the preferred range of each criterion.  provide another approach for a future study such as modeling the historical costs and forecasting costs of industrial buildings in egypt, based on an artificial intelligence model. ijahp article: el hadidi, el-dash, besiouny, meshref/evaluation of building life cycle cost (lcc) criteria in egypt using the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 20 vol. 14 issue 2 2022 issn 19366744https://doi.org/10.13033/ijahp.v14i2.958 references ahmed s., arocho i. (2021). analysis of cost comparison and effects of change orders during construction industrial: study of a mass timber and a concrete building project. journal of building engineering, 33,101856. doi: https://doi.org/10.1016/j.jobe.2020.101856 ahp (analytic hierarchy process) calculation software by cgi http://www.isc.senshu-u.ac.jp/~thc0456/eahp/ahpweb.html akman, e., & dagdeviren, m. (2018). discovering what makes a sme website good for international trade. technological and economic development of economy, 24(3), 1063-1079. doi: https://doi.org/10.3846/20294913.2016.1266709 alitaneh s. (2019). theories on coefficient of variation scales triangle and normalization of different variables: a new model in development of multiple criteria decision analysis. international journal of the analytic hierarchy process, 11(2), 283-295. doi: https://doi.org/10.13033/ijahp.v11i2.565 an m., (2014).ahp approach to aircraft selection process. transportation research procedia, 3, 165-174.doi: https://doi.org/10.1016/j.trpro.2014.10.102 domenico a.n., hitapriya s.& tri joko w. a. (2020), life cycle cost structure of a government office green building – case of the ministry of public works and housing main building, journal of infrastructure and facility asset management, 2(1),1-20.doi: https://doi.org/10.12962/jifam.v2i0.8407 egyptian engineers syndicate. available online: https://eea.org.eg/ ergu d., kou g., peng y. & shi y. (2011).a simple method to improve the consistency ratio of the pairwise comparison matrix in anp. european journal of operational research, 213(1), 246.doi:10.1016/j.ejor.2011.03.014. ergu d., kou g., shi y. & shi y., (2011).analytic network process in risk assessment and decision analysis. computers and operations research, 42, 5874. doi:10.1016/j.cor.2011.03.005 ferretti, v., & saaty, t. (2014). questions and answers: valentina ferretti interviews tom saaty. international journal of the analytic hierarchy process, 6(2), 132-143. doi: http://dx.doi.org/10.13033/ijahp.v6i1.235 garuti, c. & salomon, v. a. p., (2012). compatibility indices between priority vectors. international journal of the analytic hierarchy process, 4(2), 152-160. doi: https://doi.org/10.13033/ijahp.v4i2.130 goh, b.h. & sun, y. (2016).the development of life-cycle costing for buildings. building research & information, 44(3), 319–333. doi: https://doi.org/10.1080/09613218.2014.993566 http://www.isc.senshu-u.ac.jp/~thc0456/eahp/ahpweb.html https://doi.org/10.3846/20294913.2016.1266709 https://doi.org/10.13033/ijahp.v11i2.565 http://dx.doi.org/10.13033/ijahp.v6i1.235 ijahp article: el hadidi, el-dash, besiouny, meshref/evaluation of building life cycle cost (lcc) criteria in egypt using the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 21 vol. 14 issue 2 2022 issn 19366744https://doi.org/10.13033/ijahp.v14i2.958 gao, x., & pishdad-bozorgi, p. (2018). past, present, and future of bim-enabled facilities operation and maintenance. paper presented at the construction industrial research congress. doi: https://doi.org/10.1061/9780784481295.006 ibrahim, a. (2001). life cycle cost analysis: a computer aided tool for the egyptian construction industrial industry. thesis, american university in cairo. iso 15686 part 5. (2013). life cycle costing for buildings and constructed assets. bcis and the british standards institute. date accessed 7 january 2013, http://www.ciria.org/service/membership_options/am/contentmanagernet/cont ent iso/tc59, b.c. iso 15686-5 (2017). buildings and constructed assets service life planning part 5: life cycle costing. iso international organization for standardization: geneva, switzerland. available online: https://www.iso.org/standard/61148.html (july 2017). marrero m. rivero-camacho c., martínez-rocamora a., alba-rodríguez, m.d.&solísguzmán j. (2022).life cycle assessment of industrial building construction industrial and recovery potential. case studies in seville. processes,10, 76. doi: https://doi.org/10.3390/pr10010076. kaming peter f. &mardiansyah, j. (2015). implementation of life cycle costing: a case of hostel building in kediri, eastern jawa, indonesia. applied mechanics and materials, 845, 326-331. doi: https://doi.org/10.4028/www.scientific.net/amm.845.326. kaming, p.f. (2016). implementation of life cycle costing for a university building. indonesian journal of life cycle assessment and sustainability, 1(1), 29-38. doi: https://doi.org/10.52394/ijolcas.v1i1.5 kaming, p.f. (2017). implementation of life cycle costing for a commercial building: case of a residential apartment at yogyakarta. matec web of conferences, 138, 05008. doi: 10.1051/matecconf/201713805008 karaman, a. s., & akman, e. (2018).taking-off corporate social responsibility programs: an ahp application in airline industry. journal of air transport management, 68, 187-197. doi: https://doi.org/10.1016/j.jairtraman.2017.06.012 medhat k., sherif e. & osama t., (2021). life-cycle cost analysis for façade retrofit measures of residential buildings in cairo. indoor and built environment, 31(4), 913-928. 26. doi: https://doi.org/10.1177/1420326x211040242 mhuc (2020). building materials price list for june 2020. cairo, egypt: ministry of housing, utilities & urban communities, 2020. montgomery, j. (1998). making a city urbanity, vitality and urban design. journal of urban design, 3, 93-116. doi: https://doi.org/10.1080/13574809808724418 noshadravan, a., miller, t. r., & gregory, j. g. (2017). a lifecycle cost analysis of residential buildings including natural hazard risk. journal of construction http://www.ciria.org/service/membership_options/am/contentmanagernet/content http://www.ciria.org/service/membership_options/am/contentmanagernet/content https://www.iso.org/standard/61148.html https://doi.org/10.1016/j.jairtraman.2017.06.012 https://doi.org/10.1177/1420326x211040242 ijahp article: el hadidi, el-dash, besiouny, meshref/evaluation of building life cycle cost (lcc) criteria in egypt using the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 22 vol. 14 issue 2 2022 issn 19366744https://doi.org/10.13033/ijahp.v14i2.958 industrial engineering and management, 143(7), 10.doi: https://doi.org/10.1061/(asce)co.1943-7862.0001286 point carbon website, https://www.refinitiv.com/en/products/point-carbon-prices reisinger, j., kugler, s., kovacic, i., knoll, m. (2022). parametric optimization and decision support model frame work for life cycle cost analysis and life cycle assessment of flexible industrial building structures integrating production planning. buildings, 12, 162. doi: https://doi.org/10.3390/buildings12020162. rics (2012).nrm 1order of cost estimating and cost planning for capital building works, (2nd edition). rics (2014).nrm 3order of cost estimating and cost planning for building maintenance works (1st edition). rics (2016). guidance note, uk 1st edition. saaty, t. l. (1990). how to make a decision: the analytic hierarchy process. european journal of operational research, 48, 9-26. doi: https://doi.org/10.1016/0377-2217(90)90057-i saaty, t. l. (2000).decision making for leaders – the analytic hierarchy process for decisions in a complex world. pittsburgh, pa, usa: rws publications. saaty, t. l. (2005).the analytic hierarchy and analytic network processes for the m of intangible criteria and for decision-making. in figueira, j, greco, s, ehrgott, m, (eds). multiple criteria decision analysis: state of the art surveys, new york: springer, pp. 345-407.doi: https://doi.org/10.1007/0-387-23081-5_9 saaty, t. l. (2013). the analytic hierarchy process without the theory of oskar perron. international journal of the analytic hierarchy process, 5(2), 268-293. doi: http://dx.doi.org/10.13033/ijahp.v5i2.191 salgado, e., salomon, v. a. p. & mello, c. (2012). analytic hierarchy prioritization of new product development activities for electronics manufacturing. international journal of production research, 50, 4860-4866. doi: https://doi.org/10.1080/00207543.2012.657972 tu, k j. & huang y. w. (2013).predicting the operation and maintenance costs of condominium properties in the project planning phase: an artificial neural network approach. international journal of civil engineering, (4a), 242-250. tu, k., taur, y. & lin, c. (2016).integrating building information modeling technology, facility management system and maintenance cost database in predicting building life cycle. in saari, a. and huovinen, p. (eds). wbc16 proceedings: volume iii. building up business operations and their logic. shaping materials and technologies. tampere: tampere university of technology. united nations (2011). kyoto protocol framework convention on climate change. available online at: http://unfccc.int/kyoto_protocol/items/2830.php, accessed [22 dec 2011]. https://www.refinitiv.com/en/products/point-carbon-prices https://doi.org/10.3390/buildings12020162 http://dx.doi.org/10.13033/ijahp.v5i2.191 http://unfccc.int/kyoto_protocol/items/2830.php http://unfccc.int/kyoto_protocol/items/2830.php ijahp article: el hadidi, el-dash, besiouny, meshref/evaluation of building life cycle cost (lcc) criteria in egypt using the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 23 vol. 14 issue 2 2022 issn 19366744https://doi.org/10.13033/ijahp.v14i2.958 youssef o., mona g. i., koji t. & ahmed m. a. (2021). life cycle cost analysis on three high-performance glazing systems for an office building in new cairo, egypt. architectural engineering and design management, 17(1-2), 131-145. doi:https://doi.org/10.1080/17452007.2020.1788500 yussra m. r., ibrahim n. , khaled n., islam a. m.& meshary g. (2019). a bimbased life cycle cost (lcc) method to reduce the operation energy costs in buildings, proceedings of the 16th ibpsa conference, rome, italy, sept. 2-4. doi: https://doi.org/10.26868/25222708.2019.210616 zhang, c., cao, l. w. & romagnoli, a. (2018). on the feature engineering of building energy data mining. sustainable cities and society, 39, 508-518. https://doi.org/10.26868/25222708.2019.210616 ijahp article: el hadidi, el-dash, besiouny, meshref/evaluation of building life cycle cost (lcc) criteria in egypt using the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 24 vol. 14 issue 2 2022 issn 19366744https://doi.org/10.13033/ijahp.v14i2.958 appendix ahp questionnaire survey for sustainable industrial buildings construction based on life cycle cost please perform a pair-wise comparison of importance using the following 1 to 9 scale: the information extracted from this questionnaire will be compiled and will be used as a part of the research. your contribution towards this study is greatly appreciated since it will significantly add to the value of this research. section 1: general information this section contains general information about you and your firm to identify the various responses received.  name:  phone number:  email address:  experience in green and sustainable construction industrial industry:  company name:  position:  how would you classify your firm? □ owner representative □ consultant firm □ contractor □ other, please specify: section 2: definition of selected dimensions selected dimensions definition initial cost construction industrial cost building area the total area of building which is one of the basics of building foot print floor height the height of each building story in unit length which is one of the basics of building foot print numbers of floors no. of building stories (e.g. 1,2,3….) which is one of the basics of building foot print structure & envelope type these include steel, concrete, wood, and precast concrete in various combinations building age the study period in years (e.g. 15, 25, 30, ….. year ) ijahp article: el hadidi, el-dash, besiouny, meshref/evaluation of building life cycle cost (lcc) criteria in egypt using the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 25 vol. 14 issue 2 2022 issn 19366744https://doi.org/10.13033/ijahp.v14i2.958 location city the cost index of a location varies in different cities year of construction industrial this parameter includes the projects that are built within the study period operating and maintenance referred to as hard facilities management costs, these costs include operating costs such as cleaning and energy costs, maintenance costs, and other costs. energy cost energy used for heating and lighting. catering and services general support services, communications and security services, letting fees, facilities management fees, caretaker and janitorial services, service transport, it services, and laundry and linen services, e.g. internal deliveries. cleaning waste management and disposal. major repairs redecoration, renovation, rehabilitation, replacement. periodic maintenance the cost of contractors who perform skilled jobs, such as sanitation and hvac services. rent and insurance insurance rates and other local taxes and charges. environmental impact cost environmental cost references the cost of greenhouse gas (ghg) emissions, which are produced during the industrial construction of concrete and which has effects on the environment. structure and envelope material waste cost waste gathered from all stages such as production of raw materials, manufacturing concrete, placing concrete in the location, and demolition. market price of resultant co2 cost of controlling gas emissions. end of life this includes disposal and demolition, but specifically includes the worth of alternatives at the end period of lcca. salvage and recycling recycling, the conversion of building waste into new objects. demolition cost building demolition waste such as materials, aggregate, concrete, wood, and metal. section 3: ahp questionnaire the pair-wise comparison matrix of lcc categories 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 initial costs operating and maintenance initial costs environmental impact cost initial costs end of life operating and maintenance environmental impact cost operating and maintenance end of life environmental impact cost end of life the pair-wise comparison matrix of initial cost criteria 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 building area floor height building area no. of floors ijahp article: el hadidi, el-dash, besiouny, meshref/evaluation of building life cycle cost (lcc) criteria in egypt using the analytic hierarchy process (ahp) international journal of the analytic hierarchy process 26 vol. 14 issue 2 2022 issn 19366744https://doi.org/10.13033/ijahp.v14i2.958 building area structure & envelope type building area building age building area location city building area year of built floor height no. of floors floor height structure & envelope type floor height building age floor height location city floor height year of built no. of floors structure & envelope type no. of floors building age no. of floors location city no. of floors year of built structure & envelope type building age structure & envelope type location city structure & envelope type year of built building age location city building age year of built location city year of built the pair-wise comparison matrix of operating and maintenance cost criteria 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 energy cost catering and services energy cost cleaning energy cost major repairs energy cost periodic maintenance energy cost rent and insurances catering and services cleaning catering and services major repairs catering and services periodic maintenance catering and services rent and insurance cleaning major repairs cleaning periodic maintenance cleaning rent and insurance major repairs periodic maintenance major repairs rent and insurances periodic maintenance rent and insurances the pair-wise comparison matrix of environmental impact cost criteria 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 structure and envelope materialwaste cost market price of resulted co2 the pair-wise comparison matrix of the end of life cost criteria 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 salvage and recycling demolition cost ijahp article: karmaker, halder, ahmed/customer driven quality improvement of a specific product through ahp and entropy-based qfd: a case study international journal of the analytic hierarchy process 389 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.606 customer driven quality improvement of a specific product through ahp and entropy-based qfd: a case study chitra lekha karmaker assistant professsor jashore university of science and technology bangladesh k.chitroleka@gmail.com pobitra kumar halder phd student school of engineering, rmit university s3634814@student.rmit.edu.au s.m. tazim ahmed lecturer jashore university of science and technology bangladesh tazim100705@gmail.com abstract understanding the voice of the customers (vocs) and properly incorporating their preferences and perceptions into the conceptual design process is the core step of customer-driven product development. to improve customer satisfaction and market profitability, the design team should have a customer-driven quality management and product development system. quality function deployment (qfd) is an important customer-driven quality management tool that helps identify customer requirements and translate them into proper technical measures. this paper focuses on the application of the ahp and an entropy-based qfd approach on a manufacturing company to improve the quality of its product (blender) and determine the priorities for further improvement. the paper shows how customer requirements can be identified and applied to prioritize the design requirements for improving the quality of a blender. the analytic hierarchy process (ahp) is integrated to determine the final importance of the weights of the customer needs, and entropy is used to determine the set of priority ratings. this integrated framework can help achieve an effective evaluation of the final design solution for product development by overcoming the pitfalls of the traditional qfd approach. an application in a bangladeshi company that produces blenders is presented to illustrate the performance of the proposed approach. keywords: integrated qfd; product development; customer satisfaction; ahp; entropy mailto:k.chitroleka@gmail.com https://scholar.google.com/citations?view_op=view_org&hl=en&org=8943956419179841881 mailto:s3634814@student.rmit.edu.au ijahp article: karmaker, halder, ahmed/customer driven quality improvement of a specific product through ahp and entropy-based qfd: a case study international journal of the analytic hierarchy process 390 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.606 1. introduction in today’s global markets with the fierce competition and rapid changes in customer orientation, companies need to deliver products and services that are responsive to the customers’ expectations in order to enhance corporate profit and competitiveness. to improve customer satisfaction, the design team should have a customer-driven quality management and product development system. quality function deployment (qfd) is a structured total quality management tool which can translate customer requirements into specific technical or engineering characteristics. this customer-driven design and manufacturing approach originated in the late 1960s in japan and is now widely used in all sectors such as banking, educational institutions and the garment industry. it helps the quality improvement team by identifying customer needs (performance needs) and converting them into design requirements. the proper use of qfd can help a company evaluate its design requirements, and as a result make the product more responsive to the customers. the conventional qfd methodology involves four sets of matrices called the house of quality (hoq), namely product planning, part planning, process planning, and operations/production planning. with the help of these four matrices, the qfd translates customer requirements into engineering characteristics, and subsequently into parts characteristics, process plans, and production requirements. the customer requirement planning matrix is fundamentally and strategically important in the qfd system. it is the communication platform in the investigation of what customers want and their relative position in the market. the matrix starts with the identification of the “voice of the customer” (voc) which is obtained from an interview with the customer, a market study and past data. since customer needs vary, the relative importance of the whats is articulated by allowing the customers to state their perceptions on the relative importance of the whats. then, a list of measurable engineering characteristics is specified and used to convert the customer requirements. next, the product development team develops a relationship matrix between the customer requirements and the engineering characteristics, and performs the competitive analysis and the correlations between the engineering characteristics. finally, the importance of the engineering characteristics is calculated using the information obtained from the house of quality (cohen, 1995; curcic & milunovic, 2007; durga prasad et al., 2014). qfd is used to improve the components, to accelerate the improvement rate, and to determine the effects of the design changes. the customers’ requirements and satisfaction should be considered the priority for every product design and specification. therefore, qfd is an important and suitable tool for successful new product development. it is used in the early phase of a new or improved product/service design process and can support the process from problem identification to design specification. since 2000, many researchers have applied qfd to present a new product or to improve a product design, and some of these uses are explained in table 1. ijahp article: karmaker, halder, ahmed/customer driven quality improvement of a specific product through ahp and entropy-based qfd: a case study international journal of the analytic hierarchy process 391 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.606 table 1 examples of research on qfd authors/developer research nature remarks prasad (2000) a concurrent function development procedure has been applied for a workgroup based engineering design process. it alters qfd. herrmann et al. (2000) market-driven product & service design. they tried to bridge the gap between quality improvement and customer requirements & satisfaction through qfd. harding et al. (2001) market-driven design system has been implemented and tested by helping the design team analyze and use the market information throughout the design process. qfd is adopted to present and analyze the quality of the product. pullman et al. (2002) compare two product design approaches, qfd, and conjoint analysis. qfd highlighted the importance of starting explicitly with customer requirements. kwong&bai (2003) qfd process was compared with the conventional ahp for a hair dryer design. improve the imprecise ranking of customer requirements. lai et al. (2004) a combination of the kano model & qfd is proposed to meet customer requirements. a new way to optimize the product design. iranmaresh et al. (2005) an integrated approach is presented to optimize product cost. it respects the customer perception of a product where the modified qfd method is used. an illustrative example is given to demonstrate the use of the method. lin et al. (2006) explains a novel procedure to effectively link customer requirements with design characteristics for product design based on the concept of qfd. the procedure is validated using a case study on the design of functional clothes. sakao (2007) proposes a general design methodology to effectively the proposed methodology effectively supports the ijahp article: karmaker, halder, ahmed/customer driven quality improvement of a specific product through ahp and entropy-based qfd: a case study international journal of the analytic hierarchy process 392 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.606 support the environmentally consciousness design of products by using three tools; lca, qfde, and triz. wide range of product planning and conceptual design stages in the upper stream of eco-design. das & mukherjee (2008) developed an ahp-qfd framework for designing a tourism product. the design of a tourism product incorporating the diverse needs of tourists. zhai et al. (2009) proposes a rough set based qfd approach to managing the aforementioned imprecise design information in product development. a case study on a bicycle design is used to illustrate the proposed approach and effectively manage the imprecise design information and facilitate decision-making in product development. felice & petrillo (2010) proposed a new methodological approach to define customer specifications through the employment of an integrated qfd – ahp model. the approach has been validated in a real case study about the filter in ceramic material production. liu (2011) integrates fuzzy qfd and the prototype product selection model to develop a product design and selection approach. the proposed method provides product developers with more useful information and precise analysis results. sharma (2012) attempts to merge these diverse tools of customer-orientation, financial consideration, and value creation, thus integrating target costing and value engineering into qfd framework. a case study has been discussed and issues of implementation of this cross-disciplinary approach from the perspective of an entrepreneur by selecting a consumer product are highlighted. bereketli & genevois (2013) proposed a multi-aspect qfd for environment method to identify improvement strategies in ecodesign. the method applied for the product “hand blender”, a member of electrical and electronic equipment family. vinodh et al. (2014) propose a model that integrates environmentally conscious qfd, the theory of inventive problemsolving, and ahp for innovative the voice of the customer was captured and translated to engineering characteristics using ijahp article: karmaker, halder, ahmed/customer driven quality improvement of a specific product through ahp and entropy-based qfd: a case study international journal of the analytic hierarchy process 393 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.606 and sustainable product development of automotive components. environmentally conscious qfd. muda and roji (2015) propose a framework based on qfd approach for determining employer’s selection criteria. the authors tried to identify the gaps in the curriculum based on the requirements from the industry. onar et al. (2016) proposed a new fuzzy quality function deployment (qfd) approach to effectively determine the design requirements (drs) of a computer workstation. the proposed model is more efficient than the existing qfd. carpinetti et al. (2018) proposed a group decision model based on qfd and hesitant fuzzy to select metrics for supply chain sustainability management. the model focuses on the selection and weighting of the metrics as a group decision process. yazdani et al. (2019) proposed a fuzzy mcdm framework having integration of qfd and grey analysis. this model can facilitate decision making process. during the complex decision process, determining the final importance rating of the customer requirements is a crucial step (wang, 1999; armacost et al., 1994; chan et al., 2002; kim et al., 2007). a proper estimation of the final importance ratings of the customer requirements helps the planning team design and develops appropriate design characteristics to match or exceed the customer satisfaction of all of the competitors in the target market, and therefore leads to more competitive advantages. generally, four steps are required to derive the final importance ratings of customer requirements in the house of quality model (chan & wu, 2005). the steps are as follows: step 1. identify potential customers and acquire their requirements. step 2. determine the fundamental importance ratings of each requirement. step 3. identify competitors and conduct a competitive analysis to know the market position. step 4. determine the final importance ratings of the customer requirements. since customer requirements are the crucial factor in the hoq model, considerable effort should be taken to capture those requirements (lu et al., 1994). in the literature, there are numerous methods available for collecting the voice of customers (vocs), including ijahp article: karmaker, halder, ahmed/customer driven quality improvement of a specific product through ahp and entropy-based qfd: a case study international journal of the analytic hierarchy process 394 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.606 personal interviews, warranty data, feedback, affinity diagrams, analyzing complaints, field reports and rough set, but all of the approaches have some pitfalls and are not capable of properly acquiring the customer requirements. various techniques exist to determine the fundamental importance ratings of the customer requirements. the point scoring scale and conjoint analysis method are the simplest methods that are used to calculate the relative importance of the customer requirements. several researchers have focused on the use of the analytic hierarchy process (ahp) to estimate the importance weights (armacost et al., 1994; lin et al., 2008; li et al 2009; vinodh et al., 2014). hoet al. (1999) described a group decisionmaking approach in the quality function deployment model to estimate the weights of the customer requirements. a number of scholars have proposed the integration of the fuzzy set theory or integrated fuzzy set approaches into the house of quality to overcome the pitfalls of the traditional model. different integrated fuzzy set approaches include triangular fuzzy numbers, fuzzy ahp, fuzzy arithmetic, and the fuzzy analytic network process. chen et al. (2006) rated the technical attributes in fuzzy qfd by integrating the fuzzy weighted average method and fuzzy expected value operator. chan et al. (1999) combined triangular fuzzy numbers and entropy methods to determine the final importance of the customer requirements. several researchers have developed fuzzy ahp based on the qfd model to prioritize customer requirements (kabir & hasin, 2011; liu; 2011; vanegas & labib, 2001; lin, 2003; chan & wu, 2005; kahraman et al., 2006). chan et al. (1999) used a fuzzy arithmetic approach to determine the importance of each requirement. the fuzzy analytic network process was used by several researchers to determine the fundamental importance ratings (tan & shen, 2000). another important factor for determining the final importance of the customer requirements is the competitive priority ratings of a company that are obtained through the analysis of the company’s relative positions. the traditional method used to determine this is the sales point concept. currently, different methods are suggested to more objectively and convincingly analyze company performance ratings. wang et al. (2015) suggested the entropy method to measure the competitive priority ratings of each customer requirement. kano’s model was incorporated into the competitive analysis in order to properly capture the voice of the customers (voc) (madzík, 2018). from the discussion, it is quite obvious that an accurate determination of the customer requirements is very crucial to prioritize the design characteristics. an improper priority analysis of the customer requirements leads to an inappropriate decision-making product improvement and ultimately hurts customer satisfaction. after reviewing the literature, the authors think it is appropriate to integrate the analytic hierarchy process (ahp) and the entropy methods to determine the final importance of the customer requirements. this research aims to develop a framework to improve the quality of a product and find the priorities for further developments. to achieve this goal, ahp and entropy-based qfd are used. after the extraction of the customer requirements, the kano model was used to understand the nature of these requirements. then, ahp was used to assess the relative weight of each requirement. entropy was applied to measure the competitive priority rating of each requirement. finally, a hoq matrix was developed to prioritize the design requirements through a quantitative analysis. from the customers’ requirement ijahp article: karmaker, halder, ahmed/customer driven quality improvement of a specific product through ahp and entropy-based qfd: a case study international journal of the analytic hierarchy process 395 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.606 identification to design characteristics, each step of this framework underwent a quantitative analysis to avoid the vagueness of the subjective judgment. therefore, this integrated framework can result in a more realistic and promising decision than a standalone qfd. the proposed framework was applied to improve the quality of a renowned company in bangladesh for the validation of the framework. 2. research methodology the basic conceptual structure coordinates the concepts of the analytic hierarchy process (ahp) and the entropy method into the house of quality (hoq) model to present an effective method for converting customer requirements into design requirements. the suggested approach begins by using the ahp method to calculate the rating of the relative importance of the customer needs. the entropy method is introduced to measure the companies’ current and target performance in terms of whats to drive the final importance rating of the customer needs. finally, the customer satisfaction management strategy, qfd, is applied to translate the customer desires into an actual technical requirement to satisfy the customer needs. a schematic diagram in figure 1 shows the outline of the proposed methodology. figure 1 outline of the proposed methodology identify the customer requirements (whats) determination of relative importance ratings of whats identify design requirements (hows) develop relationship between whats & hows measure competitive priority ratings prioritize design requirements ahp entropy qfd ijahp article: karmaker, halder, ahmed/customer driven quality improvement of a specific product through ahp and entropy-based qfd: a case study international journal of the analytic hierarchy process 396 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.606 3. description of a 7-step hoq model in this study, a 7-step hoq model is proposed to identify customer needs and prioritize the technical measures to satisfy their needs. the elements of the hoq are illustrated in figure 2. a 7-step hoq model can be described as follows: step 1. identify the customer and their requirements (whats) the first step in a qfd is to analyze the market segments during the process and identify who the ultimate customers are and what their needs are. the potential customer is the main focus when designing or modifying the product. the customer of the product should be concerned about the product and the company producing the product. necessary data from customers were collected through interviews, questionnaires, and investigations. the only way to satisfy customers is through the realization of the customers’ needs regarding a product. figure 2 elements of the house of quality (hoq) the presence of appropriate design characteristics can create more customer satisfaction if they are capable of fulfilling certain customer needs when they are made known before designing or manufacturing the product. therefore, it is very important to capture all of the important customer requirements and integrate them into the product in the form of technical measures. this process is not easy because customers cannot always express all of their desired product attributes. it is essential to use some techniques to identify all of the relevant customer requirements. the kano model is one effective technique for step 6 generate technical measures (hows) step 1 identify customer & their requirements (whats) step 2 relative importance ratings of whats step 5 develop relationship between whats & hows step 3 identify competitors, conduct customer competitive analysis & set customer performance goals for whats step 4 final importance ratings of whats step 7 determine technical ratings of hows ijahp article: karmaker, halder, ahmed/customer driven quality improvement of a specific product through ahp and entropy-based qfd: a case study international journal of the analytic hierarchy process 397 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.606 categorizing customer requirements. according to this model, there are three categories of customer requirements (figure 3) which can influence user satisfaction. these categories are: 1) basic attributes (unsatisfied or must-have) these meet the primary needs of the customers. the presence of these requirements does not expand the customers’ satisfaction, but their exemption creates a high level of frustration. customers see these attributes as prerequisites. for example, ‘proper blending’ is considered a minimum feature that customers naturally expect from a blender. figure 3 the kano model 2) performance requirements the presence or absence of these attributes affects the customers. these attributes produce both satisfaction and dissatisfaction depending on the performance levels, and if they are present, the customer will be satisfied and vice versa. ‘less energy consumption’ is one type of performance attribute. 3) attractive or excitement requirements these attributes are the key factors for improving customer satisfaction. when these are offered, customers become excited, but their absence does not cause dissatisfaction. high performance of these attributes has a positive impact on the overall satisfaction level. providing a ‘multiple operations’ feature is considered an attractive need. step 2. compute the fundamental importance of the customer requirements the needs of the customer vary depending on the different degrees of importance, and companies focus on that particular requirement which is relatively more important than others. the relative importance of the customer requirements is articulated by allowing the customers to state their perceptions on the relative importance of the requirements. in ijahp article: karmaker, halder, ahmed/customer driven quality improvement of a specific product through ahp and entropy-based qfd: a case study international journal of the analytic hierarchy process 398 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.606 this research, ahp, one of the well-known multi-criteria decision making (mcdm) methods, has been used to obtain the customers’ perceptions as well as to determine the importance weight of the customer requirements. suppose the customers’ requirements are represented by wn (where n=7), and then a 7×7 comparison matrix is formed. the ahp method is applied to the matrix to derive the degree of the weight of each what. step 3. identify competitors and conduct a customer competitive analysis the company needs to identify its competitors who produce similar products. to keep pace with competitors in a competitive business environment, a company needs to know the strengths and constraints of all aspects of a product with respect to its main competitors. this is done by having customers express their opinions and rate the relative performance of the company and its competitors for each customer requirement. there is an unending need for an aggregation of expert opinions that prevents bias and diminishes unfairness in the decision process. therefore, a group decision should be adopted to improve the customer competitive priority ratings for the customer requirements in the evaluation process. step 4. determine the final importance ratings of the customer requirement the final importance ratings of customer requirements are calculated through the multiplication of the relative importance perceived by customers, the competitive priority and the improvement ratio obtained from step 3. companies must give more attention to customer requirements with higher final ratings which indicate both higher importance and potential business benefits to the company. 𝐹𝑖𝑛𝑎𝑙 𝑖𝑚𝑝𝑜𝑟𝑡𝑎𝑛𝑐𝑒 𝑟𝑎𝑡𝑖𝑛𝑔𝑠 𝑜𝑓 𝑐𝑢𝑠𝑡𝑜𝑚𝑒𝑟 𝑟𝑒𝑞𝑢𝑖𝑟𝑒𝑚𝑒𝑛𝑡= 𝑅𝑒𝑙𝑎𝑡𝑖𝑣𝑒 importance × competitive priority rating × improvement ratio step 5. develop technical or engineering characteristics (hows) after the customer requirements are identified, the next task is to generate a set of design requirements (hows) from the company’s technicians or product development team to translate customer requirements into meaningful engineering characteristics. step 6. develop the interrelationship matrix between whats and hows the interrelationship matrix, an essential part of the house of quality (hoq), is produced by analyzing to what extent the customer requirement is technically related and influenced by the engineering characteristics. the accuracy of the matrix depends on how carefully and collectively the relationship is developed. step 7. determine the technical ratings of the engineering characteristics the technical ratings of the design requirements are calculated by the multiplication of two factors, which are the final importance ratings of the customer ratings and the relationships between the engineering characteristics and the customer requirements. 𝑇𝑒𝑐ℎ𝑛𝑖𝑐𝑎𝑙 𝑟𝑎𝑡𝑖𝑛𝑔𝑠 𝑜𝑓 𝑒𝑛𝑔𝑖𝑛𝑒𝑒𝑟𝑖𝑛𝑔 𝑐ℎ𝑎𝑟𝑎𝑐𝑡𝑒𝑟𝑖𝑠𝑡𝑖𝑐𝑠 = final importance ratings of customer requirements × interrelationsip matrix ijahp article: karmaker, halder, ahmed/customer driven quality improvement of a specific product through ahp and entropy-based qfd: a case study international journal of the analytic hierarchy process 399 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.606 the actual design ratings of the hows are determined by two factors, which are the final importance ratings of the whats and the relationship between the hows and the whats. this rating shows the basic importance of the hows progress in relation to the whats. 3.1 proposed scales scale 1: to measure the relative importance of the whats: in this study, saaty’s fundamental scale for pair-wise comparison was used to measure the relative degree of each customer requirement (what) (saaty, 2005). table 2 represents the fundamental scale used for pair-wise comparison. table 2 fundamental scale of saaty (saaty, 2005) relative importance (𝑎𝑖𝑗 ) description 1 equal importance of 𝑖 and 𝑗 3 moderate importance of 𝑖 over 𝑗 5 strong importance of 𝑖 over 𝑗 7 very strong importance of 𝑖 over 𝑗 9 absolute importance of 𝑖 over 𝑗 2,4,6,8 intermediate values scale 2: to measure the companies’ current and target performance in terms of the whats: to measure the companies’ current and target performance in terms of the whats by the entropy method, a 9-point scale is used. in the proposed scale, there are five linguistic terms along with corresponding numerical values. a more practical approach is to capture customers’ opinions using linguistic assessments. for example, rather than using numbers 1 or 5, “very low” or “very high” are used to capture customers’ perception. scale 3: to measure the relationship between each what and each how: to measure the relationship between each what and each how, a 1-3-5 point scale is used where the value ‘5’ indicates a strong relationship, ‘3’ means a moderate relationship and ‘1’ denotes a weak relationship between the what and how. the suggested scale is presented below. ijahp article: karmaker, halder, ahmed/customer driven quality improvement of a specific product through ahp and entropy-based qfd: a case study international journal of the analytic hierarchy process 400 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.606 3.2 the stepwise procedure of ahp the analytic hierarchy process (ahp) is a multiple criteria decision-making tool for organizing and analyzing complex decisions. this widely applied method was developed by thomas l. saaty to make complex decisions and rank different alternatives (saaty, 1986). this method is used to model an unstructured problem into hierarchical forms of elements to make decisions. the ahp has many application areas such as product development, project management, supply chain, business and research (rao & pawar, 2018; anjomshoae et al., 2019; yap et al., 2018). in this study, ahp has been used to calculate the relative importance of the customers’ requirements which are necessary to develop a qfd model. the stepwise procedure of the ahp is as follows: step 1: construct the structural hierarchy. step 2: construct the pair-wise comparison matrix. assuming n attributes, the pair-wise comparison of attribute i with attribute j yields a square matrix nn a  where aij denotes the comparative importance of attribute i with respect to attribute j. in the matrix, aij = 1 when i = j and aji = 1/aij. j 1 2 ……… k ……… n 1 2 : : k i : : n 1 𝑎12 …. 𝑎1𝑘 …. 𝑎1𝑛 𝑎21 1 …. 𝑎2𝑘 …. 𝑎2𝑛 : : 𝑎𝑘1𝑎𝑘2 …. 1 …. 𝑎𝑘𝑛 : : 𝑎𝑛1𝑎𝑛2 …. 𝑎𝑛𝑘 …. 1 sum= 𝑦1𝑦2 …. 𝑦𝑘 ….. 𝑦𝑛 step 3: calculate the geometric mean from elements of the row. 𝑏𝑘 = [(𝑎𝑘1).(𝑎𝑘2)……(𝑎𝑘𝑛)] 1 𝑛⁄ ( 3 ) http://en.wikipedia.org/wiki/mcda ijahp article: karmaker, halder, ahmed/customer driven quality improvement of a specific product through ahp and entropy-based qfd: a case study international journal of the analytic hierarchy process 401 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.606 step 4: calculate the normalized weights. 𝑥𝑘 = 𝑏𝑘 ∑ 𝑏𝑘 𝑛 𝑘=1 ( 4 ) step 5: calculate the eigenvector & row matrix.   rootvaluen rootvaluene th th 1 1 j n j ij earowmatrix    step 6: calculate the maximum eigen value max  . e rowmatrix max  step 7: calculate the consistency index & consistency ratio.    1 max    n n ci  ri cicr  where n and ri denote the order of the matrix and the randomly generated consistency index, respectively. if cr ≤10%, the criteria or alternative are accepted. otherwise, the criteria or alternative is rejected. 3.3 the stepwise procedure of the entropy method entropy can be defined as a measure of the number of difference or vagueness. the stepwise procedure of the entropy method is presented as follows: step 1: construct a customer comparison matrix of all whats.                       nlnnn l l l n ln aaaa aaaa aaaa aaaa w w w w whats a ........ ........................ ........................ ........ ........ ........ .... .... 321 3333231 2232221 1131211 3 2 1 where 𝑎𝑛𝑙 denotes the performance of company l’s product on the customer need wn. ijahp article: karmaker, halder, ahmed/customer driven quality improvement of a specific product through ahp and entropy-based qfd: a case study international journal of the analytic hierarchy process 402 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.606 step 2: determine the probability distribution of each what by dividing the score of the evaluation needs with the total score. pnl = anl ∑ anl l l=1 ⁄ step 3: calculate the entropy of each what. 𝐸(𝑊𝑛 ) = −∅𝐿 ∑ 𝑝𝑛𝑙 𝐿 𝑙=1 ln (𝑝𝑛𝑙 ) step 4: determine customer competitive priority ratings. 𝑒𝑛 = 𝐸(𝑊𝑛 ) ∑ 𝐸(𝑊𝑛 ) 𝑁 𝑛=1 ⁄ 4. an illustrative example in this section, a case study of the quality improvement of a blending machine produced by the walton group is presented to illustrate the concepts and computations of the proposed ahp and the entropy based hoq model for prioritizing technical measures. the authors chose this company because it is the highest exporting bangladeshi enterprise in the field of electronics and delivers versatile products like refrigerators, blenders, freezers, air conditioners, led/lcd televisions, motorcycles and smart phones. the blender by walton has gained popularity among bangladeshi customers due to its low cost and high performance. the company wants to make an improvement in the proposed product to raise its market share. the basic idea is to (i) identify what the needs of the customer (whats) are and determine the final importance using the analytic hierarchy process (ahp) and entropy, and (ii) satisfy the needs of the customer with the appropriate technical measures (hows) and prioritize the important ones for further developments. 4.1 acquiring the customer requirements for the hoq first, the company must know who their potential customers are. twelve focus groups were selected through a market survey and the company’s sales network to help identify the customer requirements. the focus groups were interviewed personally and their expectations were captured using the customers’ words. eight requirements were identified from the field survey and the internet to represent the largest concern of the customers (zikrillah). the features are ‘proper blending’ (cr1), ‘easy to clean’ (cr2), ‘less vibration’ (cr3), ‘easy to use’ (cr4), ‘less energy consumption’ (cr5), ‘less heat’ (cr6), ‘specific measurements’ (cr7), and ‘multiple operations’ (cr8). 4.2 measuring the relative importance of the customer requirements (whats) through ahp it is not likely that the selected eight customer requirements (whats) have the same importance to the customers. once the customer needs are identified, the next task is to determine the level of importance of the customer expectations in order to develop the hoq model. in this research, the ahp has been used to measure the relative importance ijahp article: karmaker, halder, ahmed/customer driven quality improvement of a specific product through ahp and entropy-based qfd: a case study international journal of the analytic hierarchy process 403 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.606 of these eight customer requirements. twelve focus groups expressed their opinions about these eight customer requirements using saaty’s scale, and pair-wise comparison matrices were constructed based on these opinions. then, an aggregate pair-wise comparison matrix was developed from the geometric means and normalized weights for all of the crs that were estimated using the necessary formulas. the author was satisfied with the result as the value of the consistency ratio (6.9%) was below the value (10%) suggested by saaty. all of the calculations are presented in table 3.the results revealed that the normalized weights ranged between 0.03-0.28. the most demanded attribute was ‘proper blending’ with a fundamental importance weight of 0.28 which was followed by ‘less energy consumption’. the ‘multiple operations’ requirement was the least expected. ijahp article: karmaker, halder, ahmed/customer driven quality improvement of a specific product through ahp and entropy-based qfd: a case study international journal of the analytic hierarchy process 404 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.606 table 3 the fundamental importance of the crs by ahp method attributes cr1 cr2 cr3 cr4 cr5 cr6 cr7 cr8 geometric mean normaliz ed weight cr1 1.00 5.00 3.33 3.33 2.00 3.67 6.00 3.00 3.04 0.28 cr2 0.20 1.00 0.33 0.29 0.20 0.33 3.00 0.20 0.41 0.04 cr3 0.31 3.00 1.00 0.33 0.27 1.33 3.33 0.24 0.74 0.07 cr4 0.31 3.67 3.00 1.00 0.78 3.33 4.67 0.44 1.44 0.13 cr5 0.67 5.00 4.00 1.67 1.00 3.33 5.00 1.00 2.09 0.20 cr6 0.29 3.00 0.89 0.31 0.31 1.00 3.33 0.29 0.72 0.06 cr7 0.17 0.33 0.31 0.22 0.20 0.31 1.00 0.20 0.29 0.03 cr8 0.31 5.00 4.33 2.67 1.00 3.67 5.00 1.00 2.06 0.19 total 3.25 26.00 17.19 9.81 5.75 16.97 31.33 6.37 10.78 1.00 consistency ratio= 6.9 % < 10% 4.3 measuring the improvement ratios of the crs the main competitors of the walton company in bangladesh (represented as co1 for easy understanding) which produce a similar type of blending machine were identified. to preserve confidentiality, the names of the companies have been kept secret and they are referenced as co2, co3, co4, and co5. the company asked twelve focus groups to rate the satisfactory estimation of its own product and the four competitors’ similar products in terms of eight whats using a 1-9 scale to understand the market and relative market position of the company, and to determine the priority ratings for further improvements. according to the customers’ assessment of the relative performance of all of the companies’ similar products in terms of the eight requirements, a 5×8 customer comparison matrix was formed by averaging the assessments of the twelve customers. the elements of the matrix are shown in table 4. based on the available resources and the relative performances of the five companies on the eight crs, the company can set satisfactory estimation goals of those crs for further improvement. after various considerations, future goals were set which are shown in the sixth column of the customer comparison matrix. note that all of the performance goals are higher than company co1’s current performance level listed in the first column of the matrix. the aggregated comparison matrix is now the input of the entropy approach. using the necessary formulas, the probability distribution and entropy of each customer requirement were estimated. after getting all entropies of the eight whats, a set of competitive priority ratings of these crs were determined and shown in the last column of table 4. ijahp article: karmaker, halder, ahmed/customer driven quality improvement of a specific product through ahp and entropy-based qfd: a case study international journal of the analytic hierarchy process 405 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.606 table 4 competitive comparison matrix of satisfactory estimation co1 co2 co3 co4 co5 satisfactory estimation goals set of priority ratings cr1 6.83 5.83 7.83 8.08 4.58 7.50 0.125 cr2 6.50 6.58 7.25 6.92 4.83 7.25 0.126 cr3 5.75 5.58 7.08 7.42 4.42 6.50 0.125 cr4 6.33 5.50 6.33 5.67 6.67 7.00 0.124 cr5 5.58 6.17 7.33 7.17 5.17 6.50 0.125 cr6 5.75 5.33 6.83 7.75 4.08 6.50 0.124 cr7 6.25 6.17 6.17 6.17 6.17 7.00 0.125 cr8 6.67 5.75 7.67 7.83 4.00 7.50 0.125 based on the current performance level and satisfactory estimation goals of the eight crs, the improvement ratios were determined. the ratios are 1.097, 1.115, 1.130, 1.105, 1.164, 1.130, 1.120, and 1.125. the analysis revealed that ‘less energy consumption’ (cr5) was the highest improvement ratio for the walton group followed by ‘less vibration’ (cr3), and ‘less heat’ (cr6). it indicates that the current performance of company co1 with respect to cr5, cr3, cr6 is much poorer than the performance of most of its competitors. 4.4 measuring the final importance of the customer requirements after determining the fundamental importance of the crs, a set of priority ratings and improvement ratios, the final importance of the crs was calculated. a graphical representation is shown in figure 4. from the diagram, it is quite obvious that the ‘proper blending’ feature should be the most important concern of the company to meet customer demands. the second highest requirement is ‘less energy consumption’ which is followed by ‘multiple operations’. ijahp article: karmaker, halder, ahmed/customer driven quality improvement of a specific product through ahp and entropy-based qfd: a case study international journal of the analytic hierarchy process 406 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.606 figure 4 schematic diagram of final importance ratings of crs 4.5 establishing the design requirements this part of determining the appropriate design characteristics is the most time consuming as well as the most challenging. this process involves the experts using their knowledge and experience to identify the measures. after careful consideration, the design team of the company proposed nine technical measures that could help translate whats into hows. table 5 summarizes the proposed design requirements that could help meet the customer requirements. table 5 competitive comparison matrix of satisfactory estimation serial no design requirements 1 proper design of the blender 2 glass jar material 3 electrical properties 4 mechanical properties 5 visible indicator 6 depth of the container 7 ergonomics 8 speed level 9 jug capacity 4.6 building a qfd matrix at this stage, an interrelationship matrix between each cr and ec was formed using scale 3. the above steps complete the hoq matrix to improve the quality of the product. 28% 4% 7% 13% 20% 6% 3% 19% final importance ratings of crs proper blending easy to clean less vibration easy to use less energy consumption less heat specific measurements multiple operation ijahp article: karmaker, halder, ahmed/customer driven quality improvement of a specific product through ahp and entropy-based qfd: a case study international journal of the analytic hierarchy process 407 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.606 the corresponding tables of results, after appropriate arrangement, formed a hoq that could link customer needs to technical considerations. figure 5 shows the complete hoq matrix for the blender. from the hoq matrix, the final importance weights of the technical measures were determined. the last row of figure 4 shows the importance weights of the hows (percentage score) which range from 3.304 to 26.657. the highest priority was the ‘electrical properties’ attribute, and the second priority was the ‘speed level’. the design requirement ‘depth of the container’ had the lowest weight. figure 5 qfd matrix for blending machine 5. results and discussion in this research, the objective of the integrated qfd framework is to identify the design requirements which are most important to meet the customers’ expectations of a specific product. the results from the above analysis revealed the final importance weights of the technical measures (hows) and also showed which how the design team should focus on to improve customer satisfaction and market share. the percentage score of the technical attributes (hows) is graphically shown in figure 6 and reveals their rank. figure 6 shows that the most important attribute is ‘electrical properties’ which is followed by ‘mechanical properties’. therefore, the design team should increase their focus on these attributes to improve the current product. ijahp article: karmaker, halder, ahmed/customer driven quality improvement of a specific product through ahp and entropy-based qfd: a case study international journal of the analytic hierarchy process 408 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.606 figure 6 ratings of the technical attributes 6. managerial and practical implications this research contributes to the literature by developing a structured framework that helps identify customers’ requirements and uses them to prioritize the design requirements to improve the quality of a product. the decision makers at the product design stage can use this prioritization to refine the product which can help them attract and retain customers. the managerial implications of this research are summarized below:  developing a strategic policy for product quality at the walton company, bd: to increase the market share as well as gain loyalty from customers, it is essential to formulate a strategic policy to incorporate the prioritized design requirements into the existing product. this research can help a product designer concentrate on these design requirements and adopt them into their current practice of service.  arranging different training programs: customer satisfaction is the key driver for any successful business. to be sustainable in a competitive market, different training programs must be offered to increase the skills of employees. this study will help managers raise funds as well as arrange training programs to capture the customer needs and translate them into design needs. 16% 4% 27% 16% 7% 3% 8% 16% 3% ratings of technical measures design blade glass jar material electrical properties mechanical properties visible indicator depth the container ergonomics speed level jug capacity ijahp article: karmaker, halder, ahmed/customer driven quality improvement of a specific product through ahp and entropy-based qfd: a case study international journal of the analytic hierarchy process 409 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.606  developing organizational vision and managerial policy to develop technology: a clear and structured organizational vision is very crucial. this research will help managers formulate a company vision and managerial policy to implement the proposed framework. 7. conclusion in this paper, an integrated framework based on a qfd that combined the ahp and entropy methods was proposed to identify customer requirements and design characteristics and help achieve an effective evaluation of the final design solution for product development. to improve the conventional hoq prioritization process, the proposed approach helps by (1) determining the degree of importance of customer requirements through the ahp multi-attribute decision process, (2) assigning the customers’ priority rating using the entropy method rather than a conventional approach, (3) determining the company’s goals and improvement ratios using entropy rather than a the conventional approach. to validate the proposed methodology, the integrative decision approach was applied to a blending machine produced by the walton group who wants to improve the quality of the current product through a systematic customer-driven approach. this proposed framework has revealed 8 customer requirements, and there are 9 design characteristics that were considered against these 8 requirements. five companies were considered for entropy. the final importance of the customer requirements obtained from the ahp approach revealed that the criteria ‘proper blending’ was the most demanding customer need among the eight proposed requirements. among the nine technical measures, ‘electrical properties’ had the highest priority weight and therefore should be focused on to satisfy the customer demands. the analysis allowed the walton group to measure their current market position and find ways to plan for the future that could help increase market share. it also allowed the company to determine the design requirements that they should focus on in order to keep their customers satisfied. in the future, fuzzy logic could be incorporated into the hoq matrix to capture the customers’ expectations. 7.1 limitations of the research this study had some limitations which could be mitigated in future. for example, in this study, only the eight most appealing customer requirements were considered for evaluation. second, the feedback was collected from 12 focus groups which might not reveal the real picture of the product. third, a non-fuzzy mcdm method (ahp) was used to measure the importance of the customers’ needs. last, only four leading competitors were considered for the evaluation process. the limitations can provide a framework for future research. 7.2 direction of future research in the future, more customer needs could be considered. the impact and interaction among these factors could be assessed using other mcdm techniques like vikor, promethee, etc. also, in the future a fuzzy environment could be considered. more competitors could be considered to measure the improvement ratios of the customer ijahp article: karmaker, halder, ahmed/customer driven quality improvement of a specific product through ahp and entropy-based qfd: a case study international journal of the analytic hierarchy process 410 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.606 requirements. the proposed method could be applied in other industries such as furniture, spinning mills, ship building, and pharmaceutical. ijahp article: karmaker, halder, ahmed/customer driven quality improvement of a specific product through ahp and entropy-based qfd: a case study international journal of the analytic hierarchy process 411 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.606 references anjomshoae, a., hassan, a. & wong, k. y. (2019). an integrated ahp-based scheme for performance measurement in humanitarian supply chains. international journal of productivity and performance management, 68(5), 938-957. doi: https://doi.org/10.1108/ijppm-04-2018-0132 armacost, r. l., componation, p. j., mullens, m. a. & swart, w. w. (1994). an ahp framework for prioritizing customer requirements in qfd: an industrialized housing application. iie transactions, 26(4), 72–79. doi: https://doi.org/10.1080/07408179408966620 chan, l. k. & wu, m. l. (2002). quality function deployment: a literature review. european journal operational research, 143, 463–497. doi: https://doi.org/10.1016/s0377-2217 (02)00178-9 chan, l. k., & wu, m. l. (2005). a systematic approach to quality function deployment with a full illustrative example. omega, 33, 119–139. doi: https://doi.org/10.1016/j.omega.2004.03.010 chan, l. k., kao, h. p., ng, a. & wu, m. l. (1999). rating the importance of customer needs in quality function deployment by fuzzy and entropy methods. international journal production research, 37(11), 2499–2518. doi: https://doi.org/10.1080/002075499190635 chen, y. z., fung, r. y. k. & tang, j. f. (2006). rating technical attributes in fuzzy qfd by integrating fuzzy weighted average method and fuzzy expected value operator. european journal operational research, 174, 1553–1566. doi: https://doi.org/10.1016/j.ejor.2004.12.026 cohen, l. (1995). quality function deployment: how to make qfd work for you. prentice hall. curcic, s. & milunovic, s. (2007). product development using quality function deployment (qfd). international journal for quality research udk-005.6 professional paper, 1(3), 243-247. durga prasad, k. g., venkata subbaiah, k. & narayana rao, k. (2014). supply chain design through qfd-based optimization. journal of manufacturing technology management, 25(5), 712-733. doi: https://doi.org/10.1108/jmtm-03-2012-0030 felice, f. d. & petrillo, a. (2010). a multiple choice decision analysis: an integrated qfd – ahp model for the assessment of customer needs. international journal of engineering, science and technology, 2(9), 25-38. doi: https://doi.org/10.4314/ijest.v2i9.63849 harding, j.a., popplewell, k., fung, r.y.k. & omar, a.r. (2001). an intelligent information framework relating customer requirements and product characteristics. https://doi.org/10.1108/ijppm-04-2018-0132 ijahp article: karmaker, halder, ahmed/customer driven quality improvement of a specific product through ahp and entropy-based qfd: a case study international journal of the analytic hierarchy process 412 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.606 computers in industry, 44(1), 51–65. doi: https://doi.org/10.1016/s0166-3615 (00)00074-9 herrmann, a., huber, f., braunstein, c. (2000). market-driven product and service design: bridging the gap between customer needs, quality management, and customer satisfaction. international journal of production economics, 66(1), 77–96. doi: https://doi.org/10.1016/s0925-5273 (99)00114-0 ho, e., lai, y. j. & chang, s. i. (1999). an integrated group decision-making approach to quality function deployment. iie transactions, 31, 553–567. doi: https://doi.org/10.1080/07408179908969858 kabir, g., hasin, m. a. a., (2011). customer perceived quality improvement of synthetic fiber using fuzzy qfd: a case study. international journal for quality research, 5(2), 75-87. kahraman, c., ertay, t. & buyukozkan, g. (2006). a fuzzy optimization model for qfd planning process using analytic network approach. european journal operational research, 171, 390–411. doi: https://doi.org/10.1016/j.ejor.2004.09.016 kim, k. j., kim, d. h., & min, d. k. (2007). robust qfd: framework and a case study. quality and reliability engineering international, 23, 31–44. doi: https://doi.org/10.1002/qre.821 kwong, c. k. & bai, h. (2003). determining the importance weights for the customer requirements in qfd using a fuzzy ahp with an extent analysis approach. iie transactions, 35, 619–626. doi: https://doi.org/10.1080/07408170304355 li, y., tang, j., luo, x. & xu, j. (2009).an integrated method of rough set, kano’s model and ahp for rating customer requirements’ final importance. expert systems with applications 36, 7045–7053. doi: https://doi.org/10.1016/j.eswa.2008.08.036 lin, c.t. (2003). a fuzzy logic-based approach for implementing quality function deployment. international journal of smart engineering systems design, 5, 55–62. doi: https://doi.org/10.1080/10255810305037 lin, m. c., wang, c. c., chen, m. s., chang, c. a. (2008). using ahp and topsis approaches in customer-driven product design process. computers in industry, 59, 17–31. doi: https://doi.org/10.1016/j.compind.2007.05.013 liu, h.-t. (2011). product design and selection using fuzzy qfd and fuzzy mcdm approaches. applied mathematical modeling, 35, 482–496. doi: https://doi.org/10.1016/j.apm.2010.07.014 lu, m., madu, c. n., kuei, c. & winokur, d. (1994). integrating qfd, ahp, and benchmarking in strategic marketing. journal business and industrial market, 9(1), 41– 50. doi: https://doi.org/10.1108/08858629410053470 ijahp article: karmaker, halder, ahmed/customer driven quality improvement of a specific product through ahp and entropy-based qfd: a case study international journal of the analytic hierarchy process 413 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.606 madzík, p. (2018). increasing accuracy of the kano model–a case study. total quality management & business excellence, 29(3-4), 387-409. doi: https://doi.org/10.1080/14783363.2016.1194197 muda, n. & roji, n.s.m. (2015). a quality function deployment (qfd) approach in determining the employer’s selection criteria. journal of industrial engineering, 1-10. doi: https://doi.org/10.1155/2015/789362 onar, s. ç., büyüközkan, g., öztayşi, b. & kahraman, c., (2016). a new hesitant fuzzy qfd approach: an application to computer workstation selection. applied soft computing, 46, 1-16. doi: https://doi.org/10.1016/j.asoc.2016.04.023 osiro, l., rodrigues lima-junior, f. & carpinetti, l. c. r., (2018). a group decision model based on quality function deployment and hesitant fuzzy for selecting supply chain sustainability metrics. journal of cleaner production, 118, 964-978. doi: https://doi.org/10.1016/j.jclepro.2018.02.197 prasad, b. (2000). a concurrent function deployment technique for a workgroup-based engineering design process. journal of engineering design, 11(2), 103–119. doi: https://doi.org/10.1080/09544820050034213 rao, m. s. & pawar, p. j. (2018). application of ahp for process parameter selection and consistency verification in secondary steel manufacturing. materials today: proceedings, 5(13), 27166-27170. doi: https://doi.org/10.1016/j.matpr.2018.09.027 saaty, t. l. (1986). axiomatic foundation of the analytic hierarchy process. management science, 32(7), 841-855. doi: https://doi.org/10.1287/mnsc.32.7.841 saaty, t. l. (2005). analytic hierarchy process. encyclopedia of biostatistics. john wiley and sons. doi: https://doi.org/10.1002/0470011815.b2a4a002 sakao, t. (2007). quality engineering for early stage of environmentally conscious design. international journal of engineering, science and technology, 3(9), 45-58. sharma, j. (2012). development of efficient fuzzy integrated quality function deployment software a conceptual analysis. international journal of computer science, 1(1), 15-24. tan, k. c. & shen, x. x. (2000). integrating kano’s model in the planning matrix of quality function deployment. total quality management, 11(8), 1141–1151. doi: https://doi.org/10.1080/095441200440395 vanegas, l.v.,labib, a.w. (2001). a fuzzy quality function deployment model for deriving optimum targets. international journal of product research, 39, 99–120. doi: https://doi.org/10.1080/00207540010005079 vinodh, s., kamala, v. & jayakrishna, k., (2014). integration of ecqfd, triz, and ahp for innovative and sustainable product development. applied mathematical modeling, 38, 2758–2770. doi: https://doi.org/10.1016/j.apm.2013.10.057 https://www.sciencedirect.com/science/article/pii/s1568494616301715#! https://www.sciencedirect.com/science/article/pii/s1568494616301715#! https://www.sciencedirect.com/science/article/pii/s1568494616301715#! https://www.sciencedirect.com/science/article/pii/s1568494616301715#! https://www.sciencedirect.com/science/article/pii/s0959652618305171#! https://www.sciencedirect.com/science/article/pii/s0959652618305171#! https://www.sciencedirect.com/science/article/pii/s0959652618305171#! https://www.sciencedirect.com/science/journal/09596526 ijahp article: karmaker, halder, ahmed/customer driven quality improvement of a specific product through ahp and entropy-based qfd: a case study international journal of the analytic hierarchy process 414 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.606 wang, c. h., shih, c. h. (2013). integrating conjoint analysis with quality function deployment to carry out customer-driven concept development for ultra-books. computer standards & interfaces, 36, 89–96. doi: https://doi.org/10.1016/j.csi.2013.07.010 wang, j. (1999). fuzzy outranking approach to prioritize design requirements in quality function deployment. international journal production research, 37(4), 899–916. doi: https://doi.org/10.1080/002075499191599 wang, q., wu, c. & sun, y. (2015). evaluating corporate social responsibility of airlines using entropy weight and grey relation analysis. journal of air transport management, 42, 55-62. doi: https://doi.org/10.1016/j.jairtraman.2014.08.003 yap, j. y., ho, c. c. & ting, c. y. (2018). analytic hierarchy process (ahp) for business site selection. aip conference proceedings, 2016(1), 020151. doi: https://doi.org/10.1063/1.5055553 yazdani, m., kahraman, c., zarate, p. & onar, s. ç. (2019). a fuzzy multi attribute decision framework with integration of qfd and grey relational analysis. expert system with applications, 115, 474-485. doi: https://doi.org/10.1016/j.eswa.2018.08.017 https://www.sciencedirect.com/science/article/pii/s0957417418305256#! https://www.sciencedirect.com/science/article/pii/s0957417418305256#! https://www.sciencedirect.com/science/article/pii/s0957417418305256#! https://www.sciencedirect.com/science/article/pii/s1568494616301715#! ijahp article: moradi/performance evaluation of university faculty by combining bsc, ahp and topsis: from the students’ perspective international journal of the analytic hierarchy process 1 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 performance evaluation of university faculty by combining bsc, ahp and topsis: from the students’ perspective nima moradi 1 faculty of engineering and natural sciences, sabanci university, istanbul, turkey nimamoradi@sabanciuniv.edu abstract today, the university plays an important role in establishing a relationship between industry and academia by training a specialized workforce. due to the important role of the university in the development of a country, evaluating the performance of the faculty or research centers of universities is one of the vital issues in the quality management of universities. in this paper, a performance evaluation method is presented for three faculty of a university located in istanbul, turkey (the name of university is kept in confidential due to the request of the university’s expert). the proposed method is based on the combination of balanced scorecard (bsc), analytic hierarchy process (ahp), and technique for order of preference by similarity to ideal solution (topsis). first, bsc and ahp are integrated, and then the strategies and measures are introduced for each perspective of bsc. then, by implementing the topsis method, a comprehensive performance evaluation approach was proposed and discussed with the university management. the proposed methodology was validated by a real case study based on the judgments of students and verified by sensitivity analysis. finally, several managerial insights, conclusions, and suggestions for future studies are presented. keywords: university performance evaluation; bsc; ahp; topsis 1. introduction currently, the role of the university or higher education institution as one of the main pillars of development and progress in any country is well known. the university plays an important role in establishing a relationship between industry and academia by training a specialized workforce for the industry. also, in addition to training specialists, the university has an important role in promoting community culture. due to the acknowledgements the authors of this project would like to acknowledge the anonymous students of the engineering department (4 students), social sciences department (3 students) and business school (3 students), which presented their judgments for our discretion. mailto:nimamoradi@sabanciuniv.edu ijahp article: moradi/performance evaluation of university faculty by combining bsc, ahp and topsis: from the students’ perspective international journal of the analytic hierarchy process 2 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 important role of the university in the development of a country, evaluating and analyzing the performance of the faculty or research centers of the university will be one of the vital issues in their quality management. some of the current questions that arise are as follows: what is an effective way to evaluate the performance of the faculty of a university?, what measures and factors should be considered to evaluate the performance of the faculty of a university? and what method can comprehensively and appropriately evaluate the performance of the university when considering the domestics factors? these are the questions this paper will address. moreover, the main motivation of this paper is to study the strengths and weaknesses of each faculty at a university located at istanbul, turkey (the name of university is kept in confidential due to a request from the university’s expert), which has not been done before by multi-criteria decision-making methods such as the analytic hierarchy process (ahp) and technique for order of preference by similarity to ideal solution (topsis). also, the results are all obtained from the students’ perspective which shows their opinions about the quality of each faculty; thus, this can be a useful method for a university to check its performance. therefore, the main purpose of this paper is to introduce a performance evaluation method for faculty of a university located in istanbul, turkey based on the combination of balanced scorecard (bsc), ahp, and topsis. first, bsc is integrated with ahp. then, by modifying the topsis method, the ahp is integrated with topsis, which makes the proposed approach a comprehensive performance evaluation tool. finally, the methodology is validated by implementing it on a real case study. as a result, not only is the newly introduced method able to theoretically evaluate university performance, but it is also practical. to summarize, the contributions of the present work can be presented as follows:  proposing a performance evaluation method for three main faculty of the university including faculty of engineering and natural sciences (fens), faculty of art and social sciences (fass), and the business school based on the bsc, ahp and topsis.  defining the strategies and measures using the four perspectives of bsc for three main faculty of the university, although the proposed strategies and measures can be used for a performance evaluation of any other university.  calculating the weights of the measures and strategies and ranks of each faculty based on the judgments of students from the fens, fass and business school. in the next section, the literature is reviewed briefly. in the third section, bsc, ahp and topsis are explained, and the methodology of this work is presented in detail. in the fourth section, the results of the methodology are given. in the fifth section, managerial insights are provided with analysis and discussion of the results. finally, in the sixth section, the conclusion and suggestions for future studies are presented. ijahp article: moradi/performance evaluation of university faculty by combining bsc, ahp and topsis: from the students’ perspective international journal of the analytic hierarchy process 3 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 2. literature review historically, many methods for performance evaluation have been introduced, from traditional methods such as expert opinions and meetings, which are generally qualitative methods, to new methods based on real data and statistics and various quantitative measurements. in addition to choosing a performance evaluation method, it is more important to create a hybrid performance evaluation system based on experts’ opinions. in this way, if university administrators only pay attention to one method of performance evaluation, they will no longer be able to identify the strengths and weaknesses of the faculties of university. 2.1 balanced scorecard (bsc) before introducing bsc, only financial aspects were important to managers who practically ignored other aspects of the organization or enterprise. because these financial-specific methods were not extensive, they were not effective in evaluating the performance of the organization from an overall perspective. in the 1980s, a novel fourdimensional model for performance management known as bsc was introduced by kaplan and norton (2001). bsc is a performance management tool that helps organizations practically reach their goals, vision, and strategies (kaplan & norton, 2001). the bsc approach has four main perspectives as follows (kaplan & norton, 1996):  growth and learning: in this perspective, an organization tries to find the strategies that lead to improvement and long-term growth. in addition, an enterprise or company tries to work toward value creation and innovation and doing innovative activities to create new services or ideas.  internal business processes: in this perspective, an organization tries to define the critical internal processes which are important for an organization’s success. by implementing the right internal processes, an organization can find ways to satisfy the customers’ expectations and financial objectives.  customer: in this perspective, the organization tries to satisfy the customer’s expectations. moreover, identifying new customers and customer retention are considered as critical factors.  financial: in this perspective, the organization tries to reach the profitability and financial objectives. in other words, the organization emphasizes the financial performance such as profit, income, cost, etc. 2.2 analytic hierarchy process (ahp) & the technique for order of preference by similarity to ideal solution (topsis) the ahp method was first introduced by thomas l. saaty in the 1980s. this method is one of the most-used multi-criteria decision-making (mcdm) methods. the ahp is based on the hierarchical structure and considers both quantitative and qualitative criteria in the model. moreover, it finds the consistency and inconsistency of the comparison between alternatives. as an important point, inputs of the ahp are pairwise comparison matrices, which are filled by the judgments of experts. for interested readers, steps of the ahp method are explained in saaty (2008). the ahp consists of the following steps: ijahp article: moradi/performance evaluation of university faculty by combining bsc, ahp and topsis: from the students’ perspective international journal of the analytic hierarchy process 4 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915  hierarchical tree formation: including objectives, criteria, sub-criteria and alternatives hierarchically, so that the relationship of each level with its upper and lower levels is known.  formation of pairwise comparison matrices: for each criterion or sub-criterion, the alternatives in the problem are compared in a matrix.  calculating the weights for criteria and sub-criteria.  calculating the final weight of each alternative by adding the multiplication of each weight related to each of the sub-criteria. it is noteworthy to say, in this paper, among mcdm methods, multi-attribute decision making (madm) methods are proper for our case study since comparisons between different alternatives are made according to the different criteria. also, among madm methods, topsis is chosen since "compensatory methods such as topsis allow tradeoffs between criteria, where a poor result in one criterion can be negated by a good result in another criterion" (greene, et al., 2011). topsis was first introduced by ching-lai hwang and yoon in 1981 and like ahp, it is one of the most popular mcdm methods. in topsis, “the best alternative should have the least distance from the positive ideal solution and the greatest distance from the negative ideal solution” (hwang, lai, & liu, 1993). for interested readers, steps of topsis are explained briefly in hwang & yoon (1981). 2.3 combination of bsc, ahp and topsis the bsc is known as one of the most extensive strategic management tools. the ahp and topsis both have their own strengths and weaknesses. for example, topsis did not consider any weights or preferences between the criteria, so the ahp could support topsis for finding the weights of the criteria in comparison to each other with a quantitative analysis. a combination of bsc, ahp and topsis can lead to finding a comprehensive method which has the strengths of all three tools in one place. a bscahp-topsis approach has been studied by several researchers in recent years (table 1). according to table 1, there are only two papers that used bsc, ahp and topsis to evaluate the performance of faculty of engineering education and determine a strategic plan for higher education; as a result, there is no similar work in the literature to the present work. ijahp article: moradi/performance evaluation of university faculty by combining bsc, ahp and topsis: from the students’ perspective international journal of the analytic hierarchy process 5 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 table 1 recent articles on bsc-ahp-topsis approach ref. year case methodology bsc ahp topsis (ertuğrul & karakaşoğlu, 2008) 2008 facility location selection    (lee, chen, & chang, 2008) 2008 evaluating performance of it department in the manufacturing industry in taiwan    (seçme, bayrakdaroğlu, & kahraman, 2009) 2009 performance evaluation in turkish banking sector    (ertuğrul & karakaşoğlu, 2009) 2009 performance evaluation of turkish cement firms    (gumus, 2009) 2009 evaluation of hazardous waste transportation firms    (azar, olfat, khosravani, & jalali, 2011) 2011 supplier selection strategy    (manian, fathi, zarchi, & omidian, 2011) 2011 performance evaluating of it department    (bentes, carneiro, da silva, & kimura, 2012) 2012 multidimensional assessment of organizational performance    (shojaee & fallah, 2012) 2012 strategic planning    (bhutia & phipon, 2012) 2012 supplier selection problem    (önder, taş, & hepsen, 2013) 2013 performance evaluation of turkish banks    (sundharam, sharma, & stephan thangaiah, 2013) 2013 sustainable growth of manufacturing industries    (fallah shams lialestanei, raji, & khajeh poor, 2013) 2013 evaluate the performance of organization branches in tehran    (vinodh, prasanna, & prakash, 2014) 2014 selecting the best plastic recycling method    (aly, attia, & mohammed, 2014) 2014 prioritizing faculty of engineering education performance    (graham, freeman, & chen, 2015) 2015 green supplier selection    (sehhat, taheri, & sadeh, 2015) 2015 ranking of insurance companies in iran    (yudatama & sarno, 2016) 2016 priority determination for higher education strategic    ijahp article: moradi/performance evaluation of university faculty by combining bsc, ahp and topsis: from the students’ perspective international journal of the analytic hierarchy process 6 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 ref. year case methodology bsc ahp topsis planning (pramanik, haldar, mondal, naskar, & ray, 2017) 2017 resilient supplier selection    (hájek, stříteská, & prokop, 2018) 2018 innovation performance evaluation    (moradi, malekmohammad, & jamalzadeh, 2018) 2018 performance evaluation of digital game industry    (yılmaz & nuri i̇ne, 2018) 2018 assessment of sustainability performances of banks    (chou, yen, dang, & sun, 2019) 2019 assessing the human resource in science and technology for asian countries    (chatterjee & stević, 2019) 2019 supplier evaluation in manufacturing environment    (guru & mahalik, 2019) 2019 performance measurement of indian public sector banks    (ban, ban, bogdan, popa, & tuse, 2020) 2020 performance evaluation model of romanian manufacturing listed companies    (yildiz, ayyildiz, taskin gumus, & ozkan, 2020) 2020 atm site selection problem    (yucesan & gul, 2020) 2020 hospital service quality evaluation    (moradi & moradi, 2021) 2020 performance evaluation of a project-based growth and entrepreneurship organization in iran    present work 2021 performance evaluation of faculties at the university    2.4 university performance evaluation according to the literature, there are several works which have studied the performance evaluation of a university or a higher education institution. chen et al. (2006) used bsc as a performance evaluation tool for the taiwanese higher education sector. by implementing the proposed method on a real case study, they constructed five major strategic themes such as an adequate financial structure, an accord with customer expectations, an excellent learning environment, organizational learning and management, and high-quality staff. farid et al. (2008) used bsc as a strategic management and powerful measurement tool in universities and higher education institutes. finally, the performance measures have been introduced for the real case study to validate the proposed bsc. taylor and baines (2012) implemented bsc in uk ijahp article: moradi/performance evaluation of university faculty by combining bsc, ahp and topsis: from the students’ perspective international journal of the analytic hierarchy process 7 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 universities to evaluate their performance. the real case study included four uk universities and interviews with senior managers. also, the results provided insight into the application of new management tools within higher education in uk universities. al-zwyalif (2012) used bsc to evaluate the performance of jordanian private universities. to reach the goals of the study, data were collected from the jordanian private universities through a questionnaire for faculty deans, deputy deans, heads of scientific departments, financial managers, and administrative managers. the results showed that “the jordanian private universities are aware of the importance of implementing the bsc in performance evaluation”. cugini and michelon (2007) proposed and developed a performance evaluation approach which is suitable for the specific features of an academic department. their case study is the university of padua, italy, where data were collected. wu and li (2009) extracted the performance measure indicators (pmis) for higher education based on bsc. in addition to bsc, they used drf (data reduction factor) and dea (data envelopment analysis) tools to complete the evaluation performance process. in their case study, 15 science and technology universities of the moe (ministry of education) were selected. özdemir and tüysüz (2017) proposed a fuzzy decision making based bsc model for performance evaluation of universities. their decision-making approach includes a fuzzy decision-making trial and evaluation laboratory (dematel) and fuzzy analytic network process (anp) methods. the fuzzy dematel method is used for showing the relationship among the perspectives and the strategies of the bsc. finally, by applying fuzzy anp, the weights of perspectives and strategies are obtained. ramasamy et al. (2016) proposed a performance evaluation tool based on bsc. they also used the ahp to prioritize the performance measures of higher-level academic institutions over bsc perspectives. using the ahp, the weights of evaluation indexes were obtained and a real case study, a university in south india, was studied to validate the proposed method. yousif and shaout (2018) presented a fuzzy logic computational model based on a survey to measure and classify the performance of sudanese universities and academic staff. also, they used ahp and topsis to determine the criteria weights and overall evaluation of sudanese universities and academic staff. in recent works, mu and nicola (2019) developed a model for rank and tenure (r&t) decisions using ahp. they used a case method approach for the development of the model and the demonstration of its use. they concluded that the proposed model rendered objectivity, transparency, and customization for r&t committee decisions in higher-education institutions (mu & nicola, 2019). moreover, there are several papers which have studied performance evaluation at the university or in higher education by presenting various methodologies such as big data analytics (job, 2018), dea (majidi, fallah lajimi, & safaei ghadikolaei, 2021; navas et al., 2020; soummakie & wegener; villegas, castañeda, & castañeda-gómez, 2020), bsc (anuforo, ayoup, mustapha, & abubakar, 2019; doh, 2015; gamal & soemantri, 2017; ilyasin, 2017; nazari-shirkouhi et al., 2020; peris-ortiz, garcía-hurtado, & devece, 2019; ruggiero, 2004; h.-y. wu, lin, & chang, 2011; zolfani & ghadikolaei, 2013), and review of bsc (al-hosaini & sofian, 2015). mu and pereya-rojas (2017) is a nice work on ahp and its applications. as a result, the proposed methodology is a unique ijahp article: moradi/performance evaluation of university faculty by combining bsc, ahp and topsis: from the students’ perspective international journal of the analytic hierarchy process 8 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 topic in the literature and the aim is to fill this research gap which has both theoretical and practical implications. 3. research methodology the steps used for research methodology in this paper are as follows (mu, cooper, & peasley, 2020):  introducing strategies for each perspective of the bsc based on the university's mission and goals (based on literature, judgments of experts and the website).  introducing measures for each strategy of the bsc perspectives (based on the literature and experts’ opinions).  calculating the weights of the measures and perspectives of bsc using the ahp (based on the scores given by students on a questionnaire).  calculating the weights and ranks of the faculty including the fens, fass, and business school using topsis (based on the scores given by students on a questionnaire). 3.1 data collection tools as mentioned in the first step of methodology, the contents of the literature and available references have been consulted. in addition to these resources, the questionnaire has been used to gather the judgments of students of the various faculty of the university (see appendix). these faculty are located at the university and the number of students is given in the acknowledgments. also, all of the calculations related to the ahp and topsis were done in excel microsoft office. 3.2 strategies for the perspectives of bsc first, the strategies for each perspective of bsc were extracted using the papers in the literature for bsc perspectives (beard, 2009; chen et al., 2006; kaplan & norton, 2015), strategies (alani, khan, & manuel, 2018; aslam, 2011; cugini & michelon, 2007; farid et al., 2008) and turkish higher education (mizikaci, 2003; özdemir & tüysüz, 2017; soummakie & wegener) and confirmed by experts at the university who have more than 10 years teaching and research experience and are presented in table 2. ijahp article: moradi/performance evaluation of university faculty by combining bsc, ahp and topsis: from the students’ perspective international journal of the analytic hierarchy process 9 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 table 2 strategies for each perspective of bsc perspective strategy 1. growth and learning 1.1 student literacy development 1.2 faculty development 1.3 increase the motivation of students 1.4 improve work environment-faculty and staff 1.5 development of organizational culture and civilization 1.6 increase the competence and ability of staff 1.7 improve research quality 1.8 promote online learning applications 2. internal business processes 2.1 transfer of learning 2.2 curriculum excellence 2.3 information technology development 2.4 establish high quality service process 2.5 complete teaching facility 2.6 provide excellent teaching quality 2.7 establish coordination among all parts of the university 3. customer 3.1 customer satisfaction-students, faculty and staff 3.2 community satisfaction 3.3 consistent with customer's expectations 4. financial 4.1 sufficient generation of funds 4.2 increase asset usage rate 4.3 reduce redundant costs 4.4 investment in research and development (r&d) 4.5 budget management 3.3 measures for the strategies and perspectives of bsc in the next step, the measures for each strategy are extracted using the papers and other useful resources (al-zwyalif, 2012; beard, 2009; chen et al., 2006; cugini & michelon, 2007; doh, 2015; navas et al., 2020; nazari-shirkouhi et al., 2020; özdemir & tüysüz, 2017; ruggiero, 2004; taylor & baines, 2012; y. wu & li, 2009; zolfani & ghadikolaei, 2013), in which the proposed measures were verified. in table 3, the measures for the strategies of bsc perspectives are provided. there are 23 extensive strategies and 56 measures for bsc perspectives which were used to evaluate the performance of each faculty comprehensively. ijahp article: moradi/performance evaluation of university faculty by combining bsc, ahp and topsis: from the students’ perspective international journal of the analytic hierarchy process 10 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 table 3 measures for the strategies of bsc perspectives strategy measures 1.1 student literacy development 1.1.1 number of licenses owned by students 1.1.2 number of seminars held for students 1.1.3 number of graduate students with gpa higher than 3.5 1.2 faculty development 1.2.1 number of licenses owned by faculty members 1.2.2 number of conferences held for faculty members 1.2.3 ratio of the citations for each faculty member 1.3 increase the motivation of students 1.3.1 ratio of the graduated students to total number of students 1.3.2 number of students who continue study for their phd 1.3.3 number of students participating in conferences and seminars 1.4 improve work environment-faculty and staff 1.4.1 modernization of equipment/facilities 1.4.2 upgrading of teaching methodology 1.5 development of organizational culture and civilization 1.5.1 ratio of the number of hours of seminars for strengthening communication skills to total number of staff 1.6 increase the competence and ability of staff 1.6.1 organization active rate 1.6.2 internal promotion rate 1.7 improve research quality 1.7.1 number of papers published 1.7.2 national science conference rate 1.7.3 faculty obtaining qualification and patent rate 1.7.4 faculty writing teaching materials or books ratio 1.7.5 number of tübi̇tak projects 1.8 promote online learning applications 1.8.1 number of distant teaching applications 1.8.2 familiarity of staff using computers 2.1 transfer of learning 2.1.1 number of reports about learning experiences during each year 2.2 curriculum excellence 2.2.1 number of non-conflict courses with each other 2.2.2 number of new courses presented during each semester 2.2.3 adequate budget on course development 2.2.4 automated process on updating courses 2.3 information technology development 2.3.1 ratio of administration computerized training 2.3.2 customer satisfaction level of administration computerized 2.3.3 teaching facility use rate 2.3.4 ratio of administration computerized 2.4 establish high quality 2.4.1 student/staff ratio ijahp article: moradi/performance evaluation of university faculty by combining bsc, ahp and topsis: from the students’ perspective international journal of the analytic hierarchy process 11 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 strategy measures service process 2.4.2 full-time staff rate 2.5 complete teaching facilities 2.5.1 teaching facility renewal rate 2.6 provide excellent quality education 2.6.1 library availability and facility ratio 2.6.2 number of areas available to everyone for use 2.6.3 international scholar academic exchange rate 2.7 establish coordination among all parts of the university 2.7.1 library availability and facility ratio 3.1 customer satisfaction 3.1.1 staff satisfaction level 3.1.2 student satisfaction level 3.1.3 faculty member satisfaction level 3.2 community satisfaction 3.2.1 satisfaction level of external partners 3.3 consistent with customers' expectations 3.3.1 number of modifications made due to customers' expectations 3.3.2 numbers of customer complaints 3.3.3 numbers participating in public charity activities 4.1 sufficient funds generation 4.1.1 tuition income 4.1.2 education promotion rewards 4.1.3 amount of cooperation between education and business 4.1.4 business donation 4.1.5 ministerial grants and research grants 4.1.6 allowance amount 4.2 increase asset usage rate 4.2.1 assets and facilities recycle rate 4.2.2 assets and facilities return rate 4.2.3 library resources and facilities usage rate 4.3 reduce redundant costs 4.3.1 human resources expense rate 4.3.2 elimination rate of unsuitable staff 4.4 investment in research and development (r&d) 4.4.1 r&d expense rate 4.5 budget management 4.5.1 gross profit 3.4 combination of bsc and ahp to combine the bsc with the ahp, the measures and bsc perspectives are considered as the alternatives and criteria in the ahp, respectively. therefore, the measures in the previous section are compared with each other in accordance with the related bsc perspective. for example, there are 21 measures in the growth & learning perspective, which are given in a pairwise comparison matrix and assigned a score by students according to their importance in the growth & learning perspective. this process is repeated for the other measures of the other three perspectives. however, due to the huge pairwise comparison matrices, the strategies of each perspective are compared with each other in the pairwise comparison matrices and then the weight of each strategy is divided evenly among its measures to obtain the weight of each measure. ijahp article: moradi/performance evaluation of university faculty by combining bsc, ahp and topsis: from the students’ perspective international journal of the analytic hierarchy process 12 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 3.5 calculation of the weight of the three main faculty with topsis in this section, we implement topsis in the proposed model. as described before, the ahp is based on a pairwise comparison matrix, while topsis is based on the decision matrix (comparison between alternatives according to the different criteria). with topsis, in the decision matrix, we use three groups of faculty (fens, fass and business school) and measures as the alternatives and criteria, respectively (table 4). thus, the weights and ranks of each faculty can be calculated based on the judgments of the students according to the following algorithm: start: consider matrix 𝑅𝑛,𝑚 as the decision matrix in topsis (input). for all 𝑖, 𝑗 do: { 𝑛𝑖𝑗 = 𝑟𝑖𝑗 ∑ 𝑟𝑖𝑗 𝑛 𝑖=1 ⁄ (normalization step) } for all 𝑖, 𝑗 do: { 𝑉𝑛,𝑚 = 𝑊𝑛,𝑛 × 𝑁𝑛,𝑚 (𝑊𝑛,𝑛 is the diagonal matrix with the weights of the measures in its main diagonal and 𝑁𝑛,𝑚 is the normalized matrix) } for all 𝑖 do: { 𝑑𝑖 + = √∑ (𝑣𝑖𝑗 − 𝑣𝑗 +) 2𝑚 𝑗=1 and 𝑑𝑖 − = √∑ (𝑣𝑖𝑗 − 𝑣𝑗 −) 2𝑚 𝑗=1 (𝑣𝑗 +, positive ideal solution (pis), and 𝑣𝑗 −, negative ideal solution (nis), are the maximum value of the j-th column and the minimum value of the j-th column of the matrix 𝑉𝑛,𝑚, respectively) } for all 𝑖 do: { 𝐶𝐿𝑖 ∗ = 𝑑𝑖 − (𝑑𝑖 − + 𝑑𝑖 +) ⁄ (𝐶𝐿∗is the closeness coefficient of each alternative) } rank the alternatives (faculty) in descending order according to their 𝐶𝐿∗. end table 4 decision matrix in the proposed ahp-topsis model decision matrix in topsis criteria measure 1.1.1 … measure 4.5.1 alternatives fens 𝑟1,1 … 𝑟1,56 fass 𝑟2,1 … 𝑟2,56 business school 𝑟3,1 … 𝑟3,56 ijahp article: moradi/performance evaluation of university faculty by combining bsc, ahp and topsis: from the students’ perspective international journal of the analytic hierarchy process 13 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 4. results 4.1 case study to verify our methodology, a questionnaire survey was used to obtain the weights of the measures and faculty. the questionnaires were distributed to 10 students; 4 students from fens, 3 students from fass and 3 students from the business school, located at the university to aggregate their judgments. the aggregation took place through a designed google form and placing the form on social networks such as telegram and whatsapp groups. the profile of the individual student is anonymous since we promised that these judgments would remain confidential, but the statistics of all participants are given in table 5. we continued the questionnaire survey until the inconsistency of the ahp matrices became acceptable; in other words, since the pairwise comparison matrices of the ahp were inconsistent due to the inconsistency rate in the first round of gathering the scores given by the students, we continued gathering the judgments of the students until the inconsistency rate of the ahp matrices became an acceptable inconsistency rate (less than 0.1) (saaty, 2008). in addition, these faculty have not been evaluated by the students before, so these judgments will be helpful for our model verification. table 5 statistics of all participants in the aggregation process participant age education level faculty department 1 30 phd fens industrial engineering 2 29 phd fens industrial engineering 3 33 m.sc. fens industrial engineering 4 34 phd fens industrial engineering 5 30 phd fass economics 6 27 m.sc. fass turkish studies 7 26 m.sc. fass turkish studies 8 31 phd business general business 9 28 m.sc. business mba 10 27 m.sc. business mba 4.2 pairwise comparison matrices (ahp input) in this section, as the inputs of the ahp, the preferences of each of the four perspectives of the bsc and strategies of each perspective were determined by the questionnaire survey; here, the average of the scores was rounded to the nearest integer number (tables 6-10). in these pairwise comparison matrices, the inconsistency of each is less than 0.1, so these matrices can be used as inputs for calculating the weights of the measures. also, tables 7-10 show the weight of each strategy. ijahp article: moradi/performance evaluation of university faculty by combining bsc, ahp and topsis: from the students’ perspective international journal of the analytic hierarchy process 14 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 table 6 pairwise comparison matrix for the perspectives of the bsc bsc 1 2 3 4 weights 1 1.00 2.00 3.00 4.00 0.45 2 0.70 1.00 2.00 3.00 0.29 3 0.40 0.30 1.00 2.00 0.15 4 0.10 0.60 0.40 1.00 0.09 inconsistency rate 0.02 table 7 pairwise comparison matrix for the strategies of the growth & learning perspective growth & learning 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 weights 1.1 1.00 1.00 0.50 0.70 0.30 1.00 0.10 0.90 0.06 1.2 0.90 1.00 2.00 1.00 2.00 2.00 1.00 3.00 0.16 1.3 2.00 0.50 1.00 2.00 2.00 1.00 0.30 0.90 0.10 1.4 3.00 2.00 0.50 1.00 3.00 1.00 0.20 1.00 0.12 1.5 2.00 0.30 0.40 0.50 1.00 0.50 0.10 0.40 0.05 1.6 1.00 0.50 1.00 1.00 3.00 1.00 0.50 1.00 0.10 1.7 7.00 1.00 4.00 5.00 5.00 2.00 1.00 3.00 0.29 1.8 0.50 1.00 2.00 0.50 1.00 1.00 0.30 1.00 0.09 inconsistency rate 0.09 table 8 pairwise comparison matrix for the strategies of the business internal processes business internal processes 2.1 2.2 2.3 2.4 2.5 2.6 2.7 weights 2.1 1.00 2.00 2.00 3.00 2.00 1.00 3.00 0.23 2.2 1.00 1.00 2.00 2.00 1.00 2.00 3.00 0.20 2.3 0.50 1.00 1.00 0.70 0.40 0.50 1.00 0.09 2.4 0.30 1.00 1.00 1.00 2.00 1.00 2.00 0.13 2.5 0.50 1.00 2.00 1.00 1.00 0.50 2.00 0.12 2.6 0.90 0.40 0.30 0.50 3.00 1.00 3.00 0.14 2.7 0.50 0.40 0.50 0.60 1.00 0.30 1.00 0.07 inconsistency rate 0.09 table 9 pairwise comparison matrix for the strategies of the customer perspective customer 3.1 3.2 3.3 weights 3.1 1.00 2.00 1.00 0.42 3.2 0.30 1.00 2.00 0.30 3.3 1.00 0.40 1.00 0.26 inconsistency rate 0.08 ijahp article: moradi/performance evaluation of university faculty by combining bsc, ahp and topsis: from the students’ perspective international journal of the analytic hierarchy process 15 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 table 10 pairwise comparison matrix for the strategies of the financial perspective financial 4.1 4.2 4.3 4.4 4.5 weights 4.1 1.00 2.00 2.00 3.00 1.00 0.31 4.2 0.40 1.00 1.00 2.00 1.00 0.18 4.3 0.50 1.00 1.00 2.00 2.00 0.22 4.4 0.50 0.30 0.20 1.00 1.00 0.10 4.5 1.00 1.00 0.50 1.00 1.00 0.17 inconsistency rate 0.01 4.3 weights of the measures (ahp output/topsis input) after the ahp calculations, the weights of the measures and perspectives of the bsc are presented in table 11. here, the weight of each measure is calculated by dividing the weight of each strategy by the number of its measures. these weights are now the output of the ahp, which can be considered as the inputs of the topsis in the next step. table 11 weights of the measures and perspectives of the bsc (m: measure, w: weight) m 1.1.1 1.1.2 1.1.3 1.2.1 1.2.2 1.2.3 1.3.1 1.3.2 1.3.3 1.4.1 1.4.2 1.5.1 w 0.010 0.010 0.010 0.025 0.025 0.025 0.015 0.015 0.015 0.028 0.028 0.023 m 1.6.1 1.6.2 1.7.1 1.7.2 1.7.3 1.7.4 1.7.5 1.8.1 1.8.2 2.1.1 2.2.1 2.2.2 w 0.023 0.023 0.027 0.027 0.027 0.027 0.027 0.020 0.020 0.068 0.015 0.015 m 2.2.3 2.2.4 2.3.1 2.3.2 2.3.3 2.4.1 2.4.2 2.5.1 2.6.1 2.6.2 2.6.3 2.7.1 w 0.015 0.015 0.006 0.006 0.006 0.006 0.019 0.019 0.038 0.020 0.020 0.021 m 3.1.1 3.1.2 3.1.3 3.2.1 3.3.1 3.3.2 3.3.3 4.1.1 4.1.2 4.1.3 4.1.4 4.1.5 w 0.021 0.021 0.021 0.047 0.013 0.013 0.013 0.004 0.004 0.004 0.004 0.004 m 4.1.6 4.2.1 4.2.2 4.2.3 4.3.1 4.3.2 4.4.1 4.5.1 1 2 3 4 w 0.004 0.005 0.005 0.005 0.009 0.009 0.009 0.015 0.459 0.296 0.153 0.090 4.4 weights of the faculty (topsis output) in this section, the weight of each faculty is calculated which gives their rank. the initial scores given by students for the decision matrix of topsis are not given in this paper due to its large dimension. hence, by the given scores and topsis calculations, the distance of each project phase from pis and nis, the closeness coefficient (weight), and ranking of each faculty (fens, fass, business school) are presented in table 12. ijahp article: moradi/performance evaluation of university faculty by combining bsc, ahp and topsis: from the students’ perspective international journal of the analytic hierarchy process 16 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 table 12 distance from pis and nis, the closeness coefficient (weight) and rank of each faculty (topsis output) faculty fens fass business school 𝑑+ 0.007135 0.009788 0.006501 𝑑− 0.007049 0.00498 0.008966 𝐶𝐿∗ 0.496938 0.337234 0.579686 rank 2 nd 3 rd 1 st 5. discussion of results this section includes two sub-sections. first, the sensitivity analysis with 9 scenarios was performed to verify the robustness of topsis method. after the sensitivity analysis, we provided the managerial insights according to the obtained results where precise analysis was done over the computational results. 5.1 sensitivity analysis 5.1.1 scenario 1 here, we examined the following scenario: what if we do not use the ahp to weigh the measures and instead use the same weights for each measure (or apply solo topsis)? this scenario is equal to removing the ahp from the methodology and just applying topsis with the same weights for the measures. the results of this scenario are given in table 13. as seen in table 13, changing weights, and considering them as the same value did not impact the final ranking although adding weights by ahp gives us better and more precise results. table 13 results for scenario 1 faculty fens fass business school 𝐶𝐿∗ (with ahp) 0.496938 0.337234 0.579686 𝐶𝐿∗ (without ahp) 0.472355 0.280902 0.641319 rank (with ahp) 2 nd 3 rd 1 st rank (without ahp) 2 nd 3 rd 1 st 5.1.2 scenario 2 here, we examined the following scenario: what if we replace the highest weight among the measures with the lowest weight among the measures (measure 2.1.1 with measure 4.1.1)? the results of this scenario are given in table 14. as seen in table 14, replacing the highest weight among the measures with the lowest weight among the measures did not impact the final ranking. ijahp article: moradi/performance evaluation of university faculty by combining bsc, ahp and topsis: from the students’ perspective international journal of the analytic hierarchy process 17 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 table 14 results for scenario 2 faculty fens fass business school 𝐶𝐿∗ (old) 0.496938 0.337234 0.579686 𝐶𝐿∗ (new) 0.517615 0.327338 0.592475 rank (old) 2 nd 3 rd 1 st rank (new) 2 nd 3 rd 1 st 5.1.3 scenario 3 here, we examined the following scenario: what if we replace the two highest weights among the measures with the two lowest weights among the measures (measures 2.1.1 and 3.2.1 with measures 4.1.1 and 4.1.2, respectively)? the results of this scenario are given in table 15. as seen by table 15, replacing the two highest weights among the measures with the two lowest weights among the measures did not impact the final ranking although the weights of the fens and business school are very close. table 15 results for scenario 3 faculty fens fass business school 𝐶𝐿∗ (old) 0.496938 0.337234 0.579686 𝐶𝐿∗ (new) 0.531702 0.374632 0.546032 rank (old) 2 nd 3 rd 1 st rank (new) 2 nd 3 rd 1 st 5.1.4 scenario 4 here, we examined the following scenario: what if we use the same weight for the highest and lowest weight among the measures (the same weight is their average weight)? the results of this scenario are given in table 16. as seen in table 16, using the same weight for the highest and lowest weight among the measures did not impact the final ranking. table 16 results for scenario 4 faculty fens fass business school 𝐶𝐿∗ (old) 0.496938 0.337234 0.579686 𝐶𝐿∗ (new) 0.503139 0.334331 0.583451 rank (old) 2 nd 3 rd 1 st rank (new) 2 nd 3 rd 1 st 5.1.5 scenario 5 here, we examined the following scenario: what if we use the same weight for the two highest and two lowest weights among the measures (the same weight is their average ijahp article: moradi/performance evaluation of university faculty by combining bsc, ahp and topsis: from the students’ perspective international journal of the analytic hierarchy process 18 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 weight)? the results of this scenario are given in table 17. as seen in table 17, using the same weight for the two highest and two lowest weights among the measures did not impact the final ranking. table 17 results for scenario 5 faculty fens fass business school 𝐶𝐿∗ (old) 0.496938 0.337234 0.579686 𝐶𝐿∗ (new) 0.507613 0.359702 0.557367 rank (old) 2 nd 3 rd 1 st rank (new) 2 nd 3 rd 1 st 5.1.6 scenario 6 here, we examined the following scenario: what if we replace the three highest weights among the measures with the three lowest weights among the measures (measures 2.1.1, 3.2.1 and 2.5.1 with measures 4.1.1, 4.1.2 and 4.1.3, respectively)? the results of this scenario are given in table 18, in which replacing the three highest weights among the measures with the three lowest weights among the measures did not impact the final ranking, although there is no change for the weight of the fens. table 18 results for scenario 6 faculty fens fass business school 𝐶𝐿∗ (old) 0.496938 0.337234 0.579686 𝐶𝐿∗ (new) 0.497359 0.262501 0.625372 rank (old) 2 nd 3 rd 1 st rank (new) 2 nd 3 rd 1 st 5.1.7 scenario 7 here, we examined the following scenario: what if we replace the four highest weights among the measures with the four lowest weights among the measures (measures 2.1.1, 3.2.1, 2.5.1 and 1.4.1 with measures 4.1.1, 4.1.2, 4.1.3 and 4.1.4, respectively)? the results of this scenario are given in table 19. according to table 19, replacing the four highest weights among the measures with the four lowest weights among the measures did not impact the final ranking, although there is no significant change for the weight of the fens. ijahp article: moradi/performance evaluation of university faculty by combining bsc, ahp and topsis: from the students’ perspective international journal of the analytic hierarchy process 19 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 table 19 results for scenario 7 faculty fens fass business school 𝐶𝐿∗ (old) 0.496938 0.337234 0.579686 𝐶𝐿∗ (new) 0.495644 0.263232 0.65448 rank (old) 2 nd 3 rd 1 st rank (new) 2 nd 3 rd 1 st 5.1.8 scenario 8 here, we examined the following scenario: what if we replace the five highest weights among the measures with the five lowest weights among the measures (measures 2.1.1, 3.2.1, 2.5.1, 1.4.1 and 1.4.2 with measures 4.1.1, 4.1.2, 4.1.3, 4.1.4 and 4.1.5, respectively)? the results of this scenario are given in table 20. as seen in table 20, replacing the five highest weights among the measures with the five lowest weights among the measures did not impact the final ranking. table 20 results for scenario 8 faculty fens fass business school 𝐶𝐿∗ (old) 0.496938 0.337234 0.579686 𝐶𝐿∗ (new) 0.500713 0.265453 0.651497 rank (old) 2 nd 3 rd 1 st rank (new) 2 nd 3 rd 1 st 5.1.9 scenario 9 here, we examined the following scenario: what if we remove fass from the alternatives to see the competition between the fens and business school in the absence of the fass? the results of this scenario are given in table 21. as seen in table 21, after removing the fass from the alternatives, the business school is still better than the fens, so topsis is robust in this scenario. table 21 results for scenario 9 faculty fens fass business school 𝐶𝐿∗ (old) 0.496938 0.337234 0.579686 𝐶𝐿∗ (new) 0.868203 0.909646 rank (old) 2 nd 3 rd 1 st rank (new) 2 nd 1 st after examining different scenarios, we can see that after sensitivity analyses, the topsis method is robust and changing the weights using the ahp has not had a significant impact on topsis. therefore, topsis is independent of the ahp, and works ijahp article: moradi/performance evaluation of university faculty by combining bsc, ahp and topsis: from the students’ perspective international journal of the analytic hierarchy process 20 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 with the scores given to the decision matrix. in other words, changing the weights of the measures has not had a major impact on the final ranking according to the sensitivity analysis; as a result, topsis is robust. 5.2 managerial insights and implications according to the results and sensitivity analysis, some managerial insights can be elicited as follows:  inconsistency rates of all ahp pairwise comparison matrices are lower than 0.1, so ahp is verified (according to tables 6-10).  changing weights by ahp and removing the fass from the alternatives has not had a significant impact on the final ranking by topsis, so the topsis method is robust and verified (according to tables 13-21).  the business school has a higher rank in comparison with the fens and fass from the students’ perspective; this shows that students at the business school at the university view the performance of their faculty more satisfactorily (see table 12).  among all of the measures, measures 1.4.1 modernization of equipment/facilities, 1.4.2 upgrading teaching methodology, 2.1.1 number of reports about learning experiences during each year, 2.6.1 everyone could use library and facilities ratio, 3.2.1 satisfaction level of external partners have higher weights which shows the high importance of satisfaction level and teaching technology in the students’ opinion (see table 11).  from the students’ perspective, among bsc perspectives, growth & learning and business internal processes have higher weights in comparison to customer and financial factors; this shows that students emphasize learning and business internal processes and they are more satisfied if these two sections are improved (see table 6).  in the growth & learning perspective, among its strategies, strategy 1.7 improve research quality has the highest weight in comparison to other strategies of the growth & learning section; this shows that research quality such as number of articles or conference participation is important to students (see table 7).  in the business internal processes perspective, among its strategies, strategy 2.1 transfer of learning has the highest weight in comparison to other strategies of the business processes section; this shows that an exchange program or sending students to the other universities as an additional activity is important for students of the fens, fass and business although there may be some biases according to the opinions of only ten students (see table 8).  in the customer perspective, among its strategies, strategy 3.1 customer satisfaction has the highest weight in comparison to other strategies of the customer section; this shows that satisfaction level including students and faculty members’ satisfaction is one of the most important factors among the other factors (see table 9).  in the financial perspective, among its strategies, strategy 4.1 sufficient funds generation has the highest weight in comparison to other strategies; this shows that funds generation is important for students since they want to support their education costs (see table 10). ijahp article: moradi/performance evaluation of university faculty by combining bsc, ahp and topsis: from the students’ perspective international journal of the analytic hierarchy process 21 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 6. conclusions and suggestions for future studies in this paper, a performance evaluation method is presented for faculty of a university located in istanbul, turkey. the proposed method is based on the combination of balanced scorecard (bsc), ahp, and topsis. first, we integrated bsc with ahp. then, by modifying the topsis method, we integrated the ahp with topsis, which makes our approach a comprehensive performance evaluation tool. finally, we validated our methodology by implementing it for a real case study and based on the judgments of students. also, this method can use the opinions of students at the university to extract strategies and performance measures and to obtain the weights of each strategy and measure. finally, we conclude that since the inconsistency rates of all ahp pairwise comparison matrices are lower than 0.1, the ahp is verified. also, changing weights by ahp and removing the fass from the alternatives did not significantly impact the final ranking by topsis; therefore, the topsis method is robust and verified. in addition, the business school has a higher rank in comparison to the fens and fass from the students’ perspective; this shows that students at the business school at the university are more satisfied with the performance of their faculty. for future studies, we suggest that the opinions of faculty members and staff can be added to the model to obtain more precise results. ijahp article: moradi/performance evaluation of university faculty by combining bsc, ahp and topsis: from the students’ perspective international journal of the analytic hierarchy process 22 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 references al-hosaini, f. f., & sofian, s. (2015). a review of balanced scorecard framework in higher education institution (heis). international review of management and marketing, 5(1), 26. al-zwyalif, i. m. (2012). the possibility of implementing balanced scorecard in jordanian private universities. international business research, 5(11), 113. doi: https://doi.org/10.5539/ibr.v5n11p113 alani, f. s., khan, m. f. r., & manuel, d. f. (2018). university performance evaluation and strategic mapping using balanced scorecard (bsc): case study–sohar university, oman. international journal of educational management. doi: https://doi.org/10.1108/ijem-05-2017-0107 aly, m., attia, h., & mohammed, a. m. (2014). prioritizing faculty of engineering education performance by using ahp-topsis and balanced scorecard approach. international journal of engineering science and innovative technology, 3(1), 11-23. anuforo, p. u., ayoup, h., mustapha, u. a., & abubakar, a. h. (2019). the implementation of balance scorecard and its impact on performance: case of universiti utara malaysia. international journal of accounting & finance review, 4(1), 1-16. doi: https://doi.org/10.46281/ijafr.v4i1.226 aslam, h. d. (2011). performance evaluation of teachers in universities: contemporary issues and challenges. journal of educational and social research, 1(2), 11-11. azar, a., olfat, l., khosravani, f., & jalali, r. (2011). a bsc method for supplier selection strategy using topsis and vikor: a case study of part maker industry. management science letters, 1(4), 559-568. doi: https://doi.org/10.5267/j.msl.2011.05.005 ban, a. i., ban, o. i., bogdan, v., popa, d. c. s., & tuse, d. (2020). performance evaluation model of romanian manufacturing listed companies by fuzzy ahp and topsis. technological and economic development of economy, 1-29. doi: https://doi.org/10.3846/tede.2020.12367 beard, d. f. (2009). successful applications of the balanced scorecard in higher education. journal of education for business, 84(5), 275-282. doi: https://doi.org/10.3200/joeb.84.5.275-282 bentes, a. v., carneiro, j., da silva, j. f., & kimura, h. (2012). multidimensional assessment of organizational performance: integrating bsc and ahp. journal of business research, 65(12), 1790-1799. doi: https://doi.org/10.1016/j.jbusres.2011.10.039 ijahp article: moradi/performance evaluation of university faculty by combining bsc, ahp and topsis: from the students’ perspective international journal of the analytic hierarchy process 23 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 bhutia, p. w., & phipon, r. (2012). application of ahp and topsis method for supplier selection problem. iosr journal of engineering, 2(10), 43-50. doi: https://doi.org/10.9790/3021-021034350 chatterjee, p., & stević, ž. (2019). a two-phase fuzzy ahp-fuzzy topsis model for supplier evaluation in manufacturing environment. operational research in engineering sciences: theory and applications, 2(1), 72-90. chen, s. h., yang, c. c., & shiau, j. y. (2006). the application of balanced scorecard in the performance evaluation of higher education. the tqm magazine. doi: https://doi.org/10.1108/09544780610647892 chou, y.-c., yen, h.-y., dang, v. t., & sun, c.-c. (2019). assessing the human resource in science and technology for asian countries: application of fuzzy ahp and fuzzy topsis. symmetry, 11(2), 251. doi: https://doi.org/10.3390/sym11020251 cugini, a., & michelon, g. (2007). performance evaluation in research departments: from the balanced scorecard to the strategy map. paper presented at the 4th conference on performance measurement and management control, measuring and rewarding performance. nice, france. doh, b. t. s. (2015). evaluating the strategic objectives of cameroonian higher education: an application of the balanced scorecard. doctoral dissertation. research unit for the sociology of education. ertuğrul, i̇., & karakaşoğlu, n. (2008). comparison of fuzzy ahp and fuzzy topsis methods for facility location selection. the international journal of advanced manufacturing technology, 39(7-8), 783-795. doi: https://doi.org/10.1007/s00170-0071249-8 ertuğrul, i̇., & karakaşoğlu, n. (2009). performance evaluation of turkish cement firms with fuzzy analytic hierarchy process and topsis methods. expert systems with applications, 36(1), 702-715. doi: https://doi.org/10.1016/j.eswa.2007.10.014 fallah shams lialestanei, m., raji, r., & khajeh poor, k. p. (2013). performance evaluation by using hybrid method: bsc, topsis and ahp. industrial management journal, 5(1), 81-100. farid, d., nejati, m., & mirfakhredini, h. (2008). balanced scorecard application inuniversities and higher education institutes: implemntation guide in an iranian context. annals of the university of bucarest, the economic & administrative series, 2. gamal, a., & soemantri, a. i. (2017). the effect of balanced scorecard on the private college performance (case study at the university of wr supratman surabaya). archives of business research, 5(5). doi: https://doi.org/10.14738/abr.55.3093 ijahp article: moradi/performance evaluation of university faculty by combining bsc, ahp and topsis: from the students’ perspective international journal of the analytic hierarchy process 24 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 graham, g., freeman, j., & chen, t. (2015). green supplier selection using an ahpentropy-topsis framework. supply chain management: an international journal, 20(3), 327-340. doi: https://doi.org/10.1108/scm-04-2014-0142 gumus, a. t. (2009). evaluation of hazardous waste transportation firms by using a two step fuzzy-ahp and topsis methodology. expert systems with applications, 36(2), 4067-4074. doi: https://doi.org/10.1016/j.eswa.2008.03.013 guru, s., & mahalik, d. (2019). a comparative study on performance measurement of indian public sector banks using ahp-topsis and ahp-grey relational analysis. opsearch, 56(4), 1213-1239. doi: https://doi.org/10.1007/s12597-019-00411-1 hájek, p., stříteská, m., & prokop, v. (2018). integrating balanced scorecard and fuzzy topsis for innovation performance evaluation. pacis 2018 proceedings. hwang, c.-l., lai, y.-j., & liu, t.-y. (1993). a new approach for multiple objective decision making. computers & operations research, 20(8), 889-899. hwang, c.-l., & yoon, k. (1981). methods for multiple attribute decision making. in multiple attribute decision making (pp. 58-191): springer. doi: https://doi.org/10.1016/0305-0548(93)90109-v ilyasin, m. (2017). balanced scorecard: a strategy for the quality improvement of islamic higher education. dinamika ilmu, 17(2), 223-236. doi: https://doi.org/10.21093/di.v17i2.703 job, m. a. (2018). an efficient way of applying big data analytics in higher education sector for performance evaluation. international journal of computer applications, 180(23), 25-32. doi: https://doi.org/10.5120/ijca2018916434 kaplan, r. s., & norton, d. p. (1996). linking the balanced scorecard to strategy. california management review, 39(1), 53-79. doi: https://doi.org/10.2307/41165876 kaplan, r. s., & norton, d. p. (2001). transforming the balanced scorecard from performance measurement to strategic management: part 1. accounting horizons, 15(1), 87-104. doi: https://doi.org/10.2308/acch.2001.15.1.87 kaplan, r. s., & norton, d. p. (2015). balanced scorecard success: the kaplan-norton collection (4 books): harvard business review press. lee, a. h., chen, w.-c., & chang, c.-j. (2008). a fuzzy ahp and bsc approach for evaluating performance of it department in the manufacturing industry in taiwan. expert systems with applications, 34(1), 96-107. doi: https://doi.org/10.1016/j.eswa.2006.08.022 majidi, s., fallah lajimi, h., & safaei ghadikolaei, a. (2021). the application of data envelopment analysis in evaluating the performance of universities and higher education ijahp article: moradi/performance evaluation of university faculty by combining bsc, ahp and topsis: from the students’ perspective international journal of the analytic hierarchy process 25 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 institutions: a systematic review of the literature. journal of industrial management perspective, 11, 53-80. manian, a., fathi, m. r., zarchi, m. k., & omidian, a. (2011). performance evaluating of it department using a modified fuzzy topsis and bsc methodology (case study: tehran province gas company). journal of management research, 3(2), 1. doi: https://doi.org/10.5296/jmr.v3i2.640 mizikaci, f. (2003). quality systems and accredition in higher education: an overview of turkish higher education. quality in higher education, 9(1), 95-106.doi: https://doi.org/10.1080/13538320308160 moradi, n., malekmohammad, h., & jamalzadeh, s. (2018). a model for performance evaluation of digital game industry using integrated ahp and bsc. journal of applied research on industrial engineering, 5(2), 97-109. moradi, n., & moradi, s. (2021). a method for project performance evaluation by combining the project golden triangle, bsc, ahp, and topsis. international journal of supply and operations management, 8(1), 81-95. mu, e., cooper, o., & peasley, m. (2020). best practices in analytic network process studies. expert systems with applications, 159, 113536. doi: https://doi.org/10.1016/j.eswa.2020.113536 mu, e., & nicola, c. b. (2019). managing university rank and tenure decisions using a multi-criteria decision-making approach. international journal of business and systems research, 13(3), 297-320. doi: https://doi.org/10.1504/ijbsr.2019.100374 mu, e., & pereyra-rojas, m. (2017). understanding the analytic hierarchy process. in practical decision making (pp. 7-22): springer.doi: https://doi.org/10.1007/978-3-31933861-3_2 navas, l. p., montes, f., abolghasem, s., salas, r. j., toloo, m., & zarama, r. (2020). colombian higher education institutions evaluation. socio-economic planning sciences, 71, 100801. doi: https://doi.org/10.1016/j.seps.2020.100801 nazari-shirkouhi, s., mousakhani, s., tavakoli, m., dalvand, m. r., šaparauskas, j., & antuchevičienė, j. (2020). importance-performance analysis based balanced scorecard for performance evaluation in higher education institutions: an integrated fuzzy approach. journal of business economics and management, 21(3), 647-678. doi: https://doi.org/10.3846/jbem.2020.11940 önder, e., taş, n., & hepsen, a. (2013). performance evaluation of turkish banks using analytical hierarchy process and topsis methods. journal of international scientific publication: economy & business, 7(part 1), 470-503. ijahp article: moradi/performance evaluation of university faculty by combining bsc, ahp and topsis: from the students’ perspective international journal of the analytic hierarchy process 26 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 özdemir, a., & tüysüz, f. (2017). an integrated fuzzy dematel and fuzzy anp based balanced scorecard approach: application in turkish higher education institutions. journal of multiple-valued logic & soft computing, 28. peris-ortiz, m., garcía-hurtado, d., & devece, c. (2019). influence of the balanced scorecard on the science and innovation performance of latin american universities. knowledge management research & practice, 17(4), 373-383. doi: https://doi.org/10.1080/14778238.2019.1569488 pramanik, d., haldar, a., mondal, s. c., naskar, s. k., & ray, a. (2017). resilient supplier selection using ahp-topsis-qfd under a fuzzy environment. international journal of management science and engineering management, 12(1), 45-54. doi: https://doi.org/10.1080/17509653.2015.1101719 ramasamy, n., rajesh, r., pugazhendhi, s., & ganesh, k. (2016). development of a hybrid bsc-ahp model for institutions in higher education. international journal of enterprise network management, 7(1), 13-26. doi: https://doi.org/10.1504/ijenm.2016.075174 ruggiero, j. (2004). performance evaluation in education. in handbook on data envelopment analysis (pp. 323-346): springer. saaty, t. l. (2008). decision making with the analytic hierarchy process. international journal of services sciences, 1(1), 83-98. seçme, n. y., bayrakdaroğlu, a., & kahraman, c. (2009). fuzzy performance evaluation in turkish banking sector using analytic hierarchy process and topsis. expert systems with applications, 36(9), 11699-11709. doi: https://doi.org/10.1016/j.eswa.2009.03.013 sehhat, s., taheri, m., & sadeh, d. h. (2015). ranking of insurance companies in iran using ahp and topsis techniques. american journal of research communication, 3(1), 51-60. shojaee, m., & fallah, m. (2012). a hybrid topsis-bsc method for strategic planning. management science letters, 2(8), 2845-2850. doi: https://doi.org/10.5267/j.msl.2012.09.029 soummakie, b., & wegener, m. (2021). a two-stage research performance assessment of turkish higher education institutions using data envelopment analysis and beta regression. preprint. sundharam, v., sharma, v., & stephan thangaiah, i. (2013). an integration of bsc and ahp for sustainable growth of manufacturing industries. international journal of business excellence, 6(1), 77-92. doi: https://doi.org/10.1504/ijbex.2013.050577 taylor, j., & baines, c. (2012). performance management in uk universities: implementing the balanced scorecard. journal of higher education policy and management, 34(2), 111-124. doi: https://doi.org/10.1080/1360080x.2012.662737 ijahp article: moradi/performance evaluation of university faculty by combining bsc, ahp and topsis: from the students’ perspective international journal of the analytic hierarchy process 27 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 villegas, j. g., castañeda, c., & castañeda-gómez, e. (2020). planning and performance measurement in higher education: three case studies of operational research application. revista facultad de ingeniería universidad de antioquia, 100, 97-112. doi: https://doi.org/10.17533/udea.redin.20210526 vinodh, s., prasanna, m., & prakash, n. h. (2014). integrated fuzzy ahp–topsis for selecting the best plastic recycling method: a case study. applied mathematical modelling, 38(19-20), 4662-4672. doi: https://doi.org/10.1016/j.apm.2014.03.007 wu, h.-y., lin, y.-k., & chang, c.-h. (2011). performance evaluation of extension education centers in universities based on the balanced scorecard. evaluation and program planning, 34(1), 37-50. doi: https://doi.org/10.1016/j.evalprogplan.2010.06.001 wu, y., & li, c. (2009). research on performance evaluation of higher education based on the model of bsc-drf-dea. paper presented at the 2009 16th international conference on industrial engineering and engineering management. doi: https://doi.org/10.1109/icieem.2009.5344267 yildiz, a., ayyildiz, e., taskin gumus, a., & ozkan, c. (2020). a modified balanced scorecard based hybrid pythagorean fuzzy ahp-topsis methodology for atm site selection problem. international journal of information technology & decision making, 19(02), 365-384. yılmaz, g., & nuri i̇ne, m. (2018). assessment of sustainability performances of banks by topsis method and balanced scorecard approach. international journal of business and applied social science (ijbass), 4(1). yousif, m. k., & shaout, a. (2018). fuzzy logic computational model for performance evaluation of sudanese universities and academic staff. journal of king saud universitycomputer and information sciences, 30(1), 80-119. doi: https://doi.org/10.1016/j.jksuci.2016.08.002 yucesan, m., & gul, m. (2020). hospital service quality evaluation: an integrated model based on pythagorean fuzzy ahp and fuzzy topsis. soft computing, 24(5), 32373255.doi: https://doi.org/10.1007/s00500-019-04084-2 yudatama, u., & sarno, r. (2016). priority determination for higher education strategic planning using balanced scorecard, fahp and topsis (case study: xyz university). paper presented at the iop conference series: materials science and engineering. doi: https://doi.org/10.1088/1757-899x/105/1/012040 zolfani, s. h., & ghadikolaei, a. s. (2013). performance evaluation of private universities based on balanced scorecard: empirical study based on iran. journal of business economics and management, 14(4), 696-714. doi: https://doi.org/10.3846/16111699.2012.665383 ijahp article: moradi/performance evaluation of university faculty by combining bsc, ahp and topsis: from the students’ perspective international journal of the analytic hierarchy process 28 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 ijahp article: moradi/performance evaluation of university faculty by combining bsc, ahp and topsis: from the students’ perspective international journal of the analytic hierarchy process 29 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.915 appendix during the research process six questionnaires, including five related to the ahp and one related to topsis, were distributed among 10 students. since five ahp questionnaires were similar, the only difference being their rows and columns, only one of them (pairwise comparison matrices of bsc perspectives) is included in the appendix as table 22. next, the topsis questionnaire is given in table 23. table 22 questionnaire of the pairwise comparison matrices of bsc perspectives education level: faculty: department: age: instruction for filling out the questionnaire: the table (matrix) below reflects your opinions and preferences towards the perspectives of bsc, which are given as the rows and columns of the matrix. to express your preference in a correct way, you should obey the following rule: if you prefer element x of a row over element y of a column, then use integer numbers ranging 2 to 9 at the blank place, the greater the number is, the stronger your preference is; if you prefer element y of a column over element x of a row, then choose a number in the set {1/9,1/8,1/7,1/6,1/5,1/4,1/3,1/2}, the lower the number is, the stronger your preference is. number 1 shows indifference! bsc 1 2 3 4 1 ■ 2 ■ ■ 3 ■ ■ ■ 4 ■ ■ ■ ■ inconsistency rate ■ table 23 questionnaire of the decision matrix of topsis education level: faculty: department: age: instruction for filling out the questionnaire: the table (matrix) below reflects your opinions on the score of each measure for each faculty, which are given as the rows and columns of the matrix. to express your preference in a correct way, you should obey the following rule: choose an integer number ranging 1 to 9 to give a score for the performance of each faculty in each measure, the greater the number is, the stronger your score is. decision matrix measures 1.1.1 1.1.2 … 4.5.1 fens … fass … business school … ijahp article: tecle, perez-verdin/analytic hierarchy process application for multiple purpose forest resources management budget allocation in durango, mexico international journal of the analytic hierarchy process 39 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.422 analytic hierarchy process application for multiple purpose forest resources management budget allocation in durango, mexico aregai tecle 1 professor, college of engineering, forestry and natural sciences, northern arizona university flagstaff, arizona aregai.tecle@nau.edu gustavo perez-verdin associate professor, instituto politecnico nacional, sigma 119 durango, mexico abstract a very important aspect of natural resources management is determining optimal budget allocation to satisfy the needs and aspirations of multiple stakeholders. this is especially the case in developing countries like mexico where budgetary funds are in short supply. there has been an increasing debate in durango, mexico, for example, about determining the most efficient way of allocating a budget for multi-purpose forest management. the debate has been triggered by a growing number of interests and stakeholders, which in addition to optimal timber production, have the desire to improve environmental conditions, water resource development, range and other non-timber resources production, and to provide better amenity values and expanded recreational opportunities. conafor (comisión nacionale forestal), the mexican agency in charge of allocating funds to promote sustainable forest resources development, has been implementing four national programs: developments of forest resources, tree plantations, non-timber products, and water resources. in addition to these programs, the forest resources management decision-making process involves four interest groups and six management objectives independently connected in a hierarchical framework. accordingly, the most suitable multi-objective/multi-criterion decision-making (modm/mcdm) technique for optimal allocation of scarce budgetary funds among the four natural resources development programs is the analytic hierarchy process (ahp). the two programs that receive the most funds are forest resources development and water resources/ environmental services development. in this way, the ahp can be used to optimally distribute scarce financial resources among competing programs to improve regional economic development and better satisfy the needs of various interest groups. acknowledgements: the authors wish to express their gratitude to dr. ciro hernandez-diaz of silviculture and wood research institute and ramon silva-flores, a resource planner with conafor-durango, for their invaluable help in getting the data used in the pairwise comparison matrices and making the conafor budget available. this project was partially funded by the state of arizona’s prop 301 funding and mexico's conacyt. mailto:aregai.tecle@nau.edu ijahp article: tecle, perez-verdin/analytic hierarchy process application for multiple purpose forest resources management budget allocation in durango, mexico international journal of the analytic hierarchy process 40 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.422 keywords: multi-objective forest management; conafor; forest budget allocation; mexican community forestry; ejido; utility model 1. introduction in the past, mexican forest management decision-makers and consultants focused their allocation of funds primarily on agroforestry-related resource management activities. the intention was to support forestland farming communities who depended on timber production, cattle grazing and crop farming for their livelihood. recently, however, there is a growing interest among stakeholders on improving other land-based goods and services such as environmental protection, water production, natural resources conservation, as well as promoting biodiversity, and cultural and recreational values. this in principle is multi-objective forest resources management. in this regard, the concept of multiple-use forestry is being revised by mexican federal and local governments to more efficiently allocate scarce funds. the revision entails adopting a forest resources management approach that incorporates the views and aspirations of multiple, some of them conflicting, stakeholders. such stakeholders in a forest ecosystem management may include community and individual landowners, the logging industry, wildlife enthusiasts, forest-related resources managers, non-government organizations (ngos), the general public, and federal and local institutions (tecle et al., 1995, 1998; jenkins, 2005; niemela et al., 2005; hossain & robak, 2010). a considerable portion of the necessary funds for managing mexican natural resources comes from the national forest commission (or conafor which stands for its spanish equivalent acronym). conafor is a relatively recent mexican federal agency created by a presidential decree on april 4, 2001. its main objectives are to promote efficient forest management and restoration activities and to enforce and monitor sustainable forest development policies. to achieve these objectives, conafor has recently been engaged in projects that increase forest stock levels. the projects include development of plantation forestry and improved management of native forests, improvement of landowner’s resource management skills, development of forest system management for non-timber products such as resins, oregano and mushrooms, and rewarding landowners whose efforts have improved water quality and environmental protection (perez-verdin et al., 2011). for instance, recently combined federal and state funds equivalent to us $4.1 million were issued in the budget to support 1,200 projects in the state of durango, mexico (see figure 1). the aims of the projects were grouped into nine categories: (1) managing timber, (2) developing plantation forestry, (3) increasing non-timber products, (4) expanding recreational opportunities and ecotourism, (5) protecting water quality and quantity, (6) enhancing environmental quality, (7) fostering skilled manpower development, (8) providing technical assistance, and (9) reducing operational costs. ijahp article: tecle, perez-verdin/analytic hierarchy process application for multiple purpose forest resources management budget allocation in durango, mexico international journal of the analytic hierarchy process 41 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.422 figure 1 location of the state of durango in mexico (rhoda and burton, 2010) allocating a budget to such a complex problem consisting of different resources with non-commensurable values, and involving numerous interested parties some with incompatible and conflicting objectives like those stated above is a multi-objective problem (tecle et al., 1995; zanakis et al., 1995; schmoldt et al., 2001a, 2001b). such a problem tends to become more complex as it usually involves evaluating the performances of numerous alternative management programs using a set of criteria to arrive at the most preferred decision outcome(s) (tecle & duckstein, 1994; duckstein & tecle, 2002; vaidya & kumar, 2006). in the problem under study, the alternatives are the varying ejido forest management budget allocation levels, and the preferred outcome is the most optimal budgetary allocation to achieve the various management objectives that can produce the most satisfying mix of forest resources products and services. in general, there are numerous types of multi-objective/multi-criterion decision-making (modm/mcdm) techniques that have been used to solve various types of multiobjective problems (tecle, 1992; tecle & duckstein, 1994; triantaphyllou, 2000; duckstein & tecle, 2002; hajkowicz & higgins, 2006; vaidya & kumar, 2006; moseley et al., 2009; sadeghi et al., 2012; mardani et al., 2015). a typology of the many different available techniques is summarized in duckstein and tecle (1993a), while their solution approaches are described in duckstein and tecle (1993b). there are many modm/mcdm techniques that have specifically been used to analyze and solve varying types of multi-objective forest resources management problems (kangas, 1992; tecle et al., 1995, 1998; kangas et al., 2001; jenkins, 2005; krcmar et al., 2005; niemela et al., 2005; phua & minowa, 2005; hajkowicz & higgins, 2006; balteiro & romero, 2008; ananda & herath, 2009; balteiro et al., 2009; šporčić et al., 2010; šporčić, 2012, to mention some). very few of these problems involve costs and budget allocations in multiple objective forest resources management projects; yet, to the best of our knowledge, this study is the only one specifically dedicated to optimally allocate scarce budgetary funds to a multi-objective community level forest resources management mexico ijahp article: tecle, perez-verdin/analytic hierarchy process application for multiple purpose forest resources management budget allocation in durango, mexico international journal of the analytic hierarchy process 42 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.422 (kangas, 1992; tecle et al., 1998; balteiro et al., 2009; chiou et al., 2009; poff et al., 2010, 2012; bradford & d'amato, 2012; white & bordoloi, 2015). as noted previously, there are numerous multi-criterion decision-making approaches that have been used to evaluate and solve various types of multi-objective forest resources management problems (mendoza & prabhu, 2000; kangas et al., 2001; balteiro & romero, 2008; poff et al., 2010, 2012; šporčić et al., 2010; šporčić, 2012; tecle & jibrin, 2014). however, the formulation of the different parts of the problem in an interconnected hierarchical structure makes this particular problem a candidate for evaluation using either the analytic hierarchy process (ahp) or the analytic network process (anp). but, since the different levels in an analytic network process show interactions and dependence among themselves as described, for example, in cheng and li (2004), ozdemir et al. (2011), thangamani (2012), sadeghi et al. (2012), napoli and schilleci (2014), and saaty (2017), the anp is excluded in favor of the ahp. the latter is so because the different levels in the budget allocation problem are independent of each other to justify and favor the use of the ahp (leskinen, 2000). the ahp evaluates the multi-objective forest resources management budget allocation problem by incorporating public values to indicate preferences in the decision making process (saaty, 1977, 1980, 1988, 1990; schmoldt et al., 2001a; niemela et al., 2005; proctor, 2005; saaty & vargas, 2006; ishizaka & labib, 2011). furthermore, we found the ahp to be a relatively simple and effective method with an ability to deal with both quantitative and qualitative criteria to determine the most efficient budget allocation for the multi-objective ejido forest management problem under consideration. 2. problem description in the past, mexican planners and resource managers rarely focused on developing strategies that strengthen multiple use and sustainable land resource development practices. hence, federal programs resulted in poor resource management scenarios that jeopardized sustainable development, which eventually led to a decrease in resource productivity and an increase in environmental degradation as stated herewith. the lack of a consistent policy to strengthen ejidos [common properties] has grave social and economic implications that result in the degradation of natural resources, which consequently prevents rural communities from making sustainable use of land resources such as forests leading to a decreased quality of life. this creates the vicious cycle of degradation, poverty and poor quality of life. (semarnat, 2000, p. 43) recent changes have improved the management of natural resources by developing new forest policies and their enforcement to promote not only sustainable resource management but also improve important environmental services. these in turn have led ijahp article: tecle, perez-verdin/analytic hierarchy process application for multiple purpose forest resources management budget allocation in durango, mexico international journal of the analytic hierarchy process 43 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.422 to improved research, and more efficient decision-making. such changes are necessary to alleviate the disparities between farmers and forest landowners. previously, crop farmers received a government subsidy to cultivate their lands, while forest landowners did not receive any funds to help them improve management of their natural resources; instead, they were blamed for the deterioration in land productivity and the poor quality of water coming from the forested areas. to ameliorate the situation, the mexican government through conafor has decided to manage the ejido forest system by developing four different forest resources/service programs to improve and optimize sustainable forest resources productivity. the programs are forest development (fd), plantations development (pd), non-timber products development (ntp), and water resources/environmental services development (wh). table 1 shows a one year u.s. $4.1 million mexican government budget allocation to these four different forest resources management programs. table 1 a one-year federal budget allocation to manage the four forest resources/service development programs. programs budget (mill $) % forest development (fd) 2.16 53 plantations development (pd) 1.45 35 non-timber products development (ntp) 0.49 12 water resources/ environmental services development (wh) 0 0 total $4.10 100 the main objectives for developing the four different forest resources /services programs are: (1) to promote the sustainability of primarily timber-related products, (2) to improve landowner’s resource management skills and (3) to reduce operational costs. of the four programs, forest resources development (fd) was considered the most important one receiving the largest share of the budget with the largest number of individual projects funded. the plantation development program (pd) was designed to increase forest stocks through establishing new commercial plantations, by producing seedlings and reforesting burned areas and former agricultural or pasture lands. the non-timber products development program (ntp) was not given as much importance as the first two programs, and was meant to focus on the production of items such as oregano and mushrooms and the development of fisheries and improved ecotourism projects. though these kinds of multiple programs development plans were designed to operate in only six wetter and more forested states of mexico, the state of durango was included because of its enormous potential to generate similar products in its temperate and semi-arid forested areas. the fourth program, water resources/ environmental services development (wh), was the latest program created by the federal government to protect and enhance amenity values, improve water quality and quantity, increase carbon sequestration, promote recreation, and reduce the pressure on timber-based products. it was not funded during ijahp article: tecle, perez-verdin/analytic hierarchy process application for multiple purpose forest resources management budget allocation in durango, mexico international journal of the analytic hierarchy process 44 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.422 the above budget year, but both federal and state authorities expect its importance to grow quickly as the public becomes more aware of and familiar with it. the mandate to develop these programs involves providing monetary resources directly to landowners to help them achieve sustainable forest management. the fundamentals of these mexican national programs are fully described in the “2001-2006 national forest program”. the main aspects of the program consist of seven national strategies and 14 objectives which are designed to promote sustainable forest management in mexico. to satisfy the objectives, the four national programs (fd, pd, ntp, and wh) were developed as the feasible programs to compete for the limited funds. but, first the 14 objectives and nine project categories are reduced into six management objectives specifically designed to: (1) improve economic benefits, (2) increase water yield, (3) increase recreational or other environmental benefits, (4) increase forest stocks, (5) reduce operational costs, and (6) increase yield of non-timber products. note that some important biological objectives such as maintaining biodiversity or protecting wildlife habitat which are not within conafor’s areas of responsibility are not included in the above list. on the other hand, all the interested parties in durango who are involved in multi-purpose forest resources management are considered part of the study. those parties are grouped into those members of the public or government sector (pub), landowners (lnd), those who belong to the private sector (prv), and members of nongovernmental organizations (ngos). the latter includes environmental groups and nonprofit forest-interest organizations. landowners, the most dominant group in durango, consist of both common and private land owners. the common properties or ejidos are collectively owned expanses of land, which occupy up to 70 percent of the forestlands in the state of durango (alcorn & toledo, 1998). thus, the essence of this study is to determine the most efficient budgetary distribution among the four national programs in ejido forest lands, taking into account the views and aspirations of the four interest groups and the achievement of the six forest resources management objectives. 3. methodology 3.1 the analytic hierarchy process our interest in this paper is to optimally allocate a limited budgetary resource among four competing programs to achieve six ejido forest resources management objectives. a convenient multi-objective decision-making technique that can handle such a hierarchically-structured level of independent elements is the analytic hierarchy process (saaty, 1977, 1980, 1988, 1990, 1997, 2008b). this is a well-known technique, which has been successfully used to solve numerous multi-objective management problems (kangas, 1992, 1994, 1999; saaty, 2001; duke & aull-hyde, 2002 to mention a few). the analytic hierarchy process formulation of a problem involves describing the elements in a lower level in terms of some or all of the elements in the next higher level of the hierarchy (saaty, 1980, 1988). altogether, the process consists of the following steps: (1) describing an overarching objective in terms of an overall and ultimately desired direction (e.g. maximizing the overall utility); (2) identifying the interested parties, stakeholders, or decision makers involved; (3) defining the specific objectives that represent the wishes and aspirations of every interested party (or parties) (e.g. maximizing desired objectives, minimizing undesired outcomes and project costs, improving economic benefits and environmental conditions, etc.); and (4) articulating ijahp article: tecle, perez-verdin/analytic hierarchy process application for multiple purpose forest resources management budget allocation in durango, mexico international journal of the analytic hierarchy process 45 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.422 feasible alternatives that can be used to achieve the desired objectives (saaty, 1998, 2008a, 2008b, 2010; ansah 2015). criteria and sub-criteria are used to articulate the specific objectives in step (3) especially when the objectives represent complex problems that need to be expressed in more detailed and ordinal forms (duckstein & tecle, 2002). the theoretical foundation behind the ahp is the utility theory of value. zahedi (1987) showed that the process of selecting alternatives is consistent with maximizing a decision maker’s either single or multi-attribute utility functions (saaty, 1998, 2003). results implied that the ahp and the utility maximization process can be combined to solve decision problems. the problems can be multi-objective forest resource management approaches such as the problem under study that involves multi-attribute utility functions. the utility functions are usually described in terms of weights and relative rankings of alternatives that can be evaluated using ahp (saaty & vargas, 1982; duke & aull-hyde, 2002; proctor, 2005; balteiro & romero, 2008; ford et al., 2017). the overarching objective of this multi-objective forest resources management planning problem is maximizing the overall utility u of the system. such maximization of the overall utility occupies the highest level in the hierarchical formulation of an ahp approach of solving a multi-objective problem. a utility model is a mathematical tool that describes a problem in terms of features such as goals or objectives that express the wishes and aspirations of individuals and/or groups. a very simple utility model can be described in terms of the overall utility value u, which is the sum of the products of the individual objective weights (ai) and the decision variable xi. algebraically, this can be expressed as  i ii xau (1) in which i stands for an individual alternative. in a multi-objective problem, the alternative that produces the highest utility value becomes the most preferred management option (tecle, 1992; saaty, 1998, 2003; tecle et al., 1998; schmoldt et al., 2001a; poff et al., 2012). the solution process includes arrangement of pairwise comparisons of the relative weights of the different objectives. this would allow the decision maker to determine a preferred management alternative where the weights represent the decision maker’s preference structure on the objectives. saaty (1988) used weights of 1,3,5,7, and 9 to respectively represent equal (or the preference of an objective weighted against itself), moderately dominant, strongly dominant, very strongly dominant, and extremely dominant of one objective over the other in a pairwise comparison. the weights of 2, 4, 6, and 8 represent the intermediate values between two consecutive pairs of the above values. reciprocal values are entered in the transpose position and values between integers are permitted when desired. use of this methodology is based on extensive research on the psychology of human preference behavior, which demonstrated that an individual cannot simultaneously compare more than five objects fairly and without having some confusion (tversky & kahneman, 1981; kangas, 1994; ford et al., 2017). to do a pairwise comparison, the relative weights (importance / preference) of elements at each hierarchical level are determined (saaty, 2001). the analyst uses this to create the ijahp article: tecle, perez-verdin/analytic hierarchy process application for multiple purpose forest resources management budget allocation in durango, mexico international journal of the analytic hierarchy process 46 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.422 matrix of pairwise comparisons, a, in equation 2 from which an eigenvector is computed (saaty, 1998, 2003). a a w w w w w w w w w w w w ij n n n n                 1 1 1 1 2 1 2 1 2 1 2 / ... / / ... / : / : / : (2) here aij represents the pairwise comparisons of elements i and j; when i = j, then aij =1. the ratio wi/wj is the relative importance or weight of element i over element j, while n is the total number of elements being compared. because the ahp involves subjective assignment of values to utilities, the process may lead to some inconsistencies. to deal with this problem, saaty (1980, 2003) proposed an eigenvector method for testing inconsistencies. for example, the inconsistency in the matrix a can be estimated using a consistency index (ci), which can be described in the form of equation 3, which is )1/()( max  nnci  (3) where m ax  is the largest eigenvector in matrix a. this value ( m ax  ) is obtained using equation 4, which involves multiplying the matrix aij by the vector of relative importance or weights, wi. 𝜆𝑚𝑎𝑥 = ∑ ∑ 𝑎𝑖𝑗 𝑗𝑖 𝑤𝑖 (4) the vector of relative weights, wi, is obtained by first normalizing each column and then each row in the matrix a (saaty, 1988). such a matrix has to be estimated for each decision variable at all levels of the system. in each level, the eigenvector is scaled to add up to one to obtain level-wise priorities. we developed a basic approach using a spreadsheet to calculate both the relative weights and the eigenvectors. if the pairwise comparisons in the n × n matrix include no inconsistencies, then m ax  = n; otherwise a is simply the reciprocal of the matrix (saaty, 2001). the more consistent the comparisons are the closer the value of the computed m ax  becomes to n. a consistency ratio (cr) determined using equation 5 indicates the coherence of the pairwise comparisons (alonso & lamata, 2006): cr = 100 (ci / aci) (5) where aci is the average consistency index for randomly generated comparisons, which varies with the size of the developed matrix (saaty, 1988). kangas (1994) considers the appropriate value for cr to be 0.10 or less. ijahp article: tecle, perez-verdin/analytic hierarchy process application for multiple purpose forest resources management budget allocation in durango, mexico international journal of the analytic hierarchy process 47 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.422 3.2 data acquisition and problem formulation a questionnaire was used to obtain the basic data for this study. twenty representative individuals involved in forest resources management were identified and asked to answer an online questionnaire to reveal their personal preferences on the way the budget for forest resources management in durango should be allocated. the identified individuals belong to each one of the four interested groups described above. however, seven of the 20 refused to participate in the study. hence, we used only the responses from the 13 consenting respondents to construct the decision matrix used in the pairwise comparisons (see appendix ai). of the 13 individuals, four were from the public sector, three were landowners, three were members of non-government organizations, and three were from the private sector. the questionnaire started with self-identification of the respondents, followed by questions on their knowledge of the four national programs and others related to the desired management objectives. to use the information for the intended decision making process, the data were averaged and then formulated into a matrix of preferences for each interested party. this was followed by calculating the eigenvectors and determining the consistency indexes to check for potential inconsistencies (saaty, 2003). a similar process was followed for the management objectives. but, in this case, we made pairwise comparisons of each management objective with each interested party and for which a consistency test was also made. the arrangement of the data in the form of a matrix, calculations of the eigenvector and consistency indexes, making the pairwise comparisons for alternative preferences and arriving at optimal budgetary allocation are done step by step in a hierarchical framework. formulation of the problem in an ahp framework follows four hierarchical steps (see figure 2). the first step consists of identifying an overarching objective of the study. the purpose is to allocate a one year forest management budget in the state of durango, mexico to achieve optimal resources productivity. the second step is to determine the relative importance or weights of all interested parties that represent the various resources and arrange them as level two in the hierarchically structured framework. kangas (1994) suggested that the assignments and pairwise comparisons of the weights may be done by the office staff administering the resource management since its members may have a better understanding of the relative importance of each resource type and the interested parties involved (saaty & vargas, 1982). as such, pairwise comparisons of the weights of the parties and the resources involved were based on personal knowledge of the area and the role the interested parties play in the management of natural resources as in proctor (2005). we also considered the scope of the forest law, which gives more importance to forest landowners than to any other groups in the management of forest resources. ijahp article: tecle, perez-verdin/analytic hierarchy process application for multiple purpose forest resources management budget allocation in durango, mexico international journal of the analytic hierarchy process 48 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.422 figure 2 an ahp flow chart of optimal budget allocation for multi-objective forest resources management in durango, mexico the third step consisted of describing the six management objectives and their relative weights. the latter came from the responses to the survey questionnaires. the last one in the hierarchy, level 4, consists of arranging the national forest resources/service development alternative programs. starting from the bottom, the elements in each level are usually linked with the elements in the level above it. however, we also find that not every element in a lower level is linked with every element in the level above it. for example, the plantation development program is not linked with the improve economic benefits objective in the level above it (see figure 2). this usually takes place when an element in the lower level is not related to one or more elements in the upper level as described in saaty (1988). in any case, the hierarchical structure in an ahp links one level with the immediate levels above and below it (if there are any), thereby mathematically tying the entire decision-making process together (saaty, 1988, 2010, 2017). 4. results optimal allocation of the available budget in this study is made by evaluating the four national resources development programs with respect to their performances in achieving the six desired management objectives. normally, expert consultants and actual management decision-makers are involved in assigning the preference structures or weights to the objectives (saaty & vargas, 1982; zanakis et al., 1995; duckstein & tecle, 2002; ford et al., 2017). in this study, the authors first assumed the roles of both the expert consultants and the decision-makers and calculate the weights using kangas’ ijahp article: tecle, perez-verdin/analytic hierarchy process application for multiple purpose forest resources management budget allocation in durango, mexico international journal of the analytic hierarchy process 49 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.422 (1994) methodology. the matrix of interested parties’ responses and the relative weights assigned to each interested party are shown in appendix ai. the relative weights are 0.18 for the public sector (pub), 0.51 for the landowners (lnd), 0.17 for the private sector (prv), and 0.14 for the non-government organizations (ngo). in this arrangement, the relative importance given to landowners is much higher than that assigned to the other stakeholders (see the table in appendix ai). this makes the values of m ax  = 4.25, aci = 0.90, ci = 0.08, and cr = 0.09. the cr value here is within the standards recommended by kangas (1994), i.e., cr ≤ 0.1. the next step consists of determining the relative importance of each objective to the various interested parties. here, the responses to the questionnaires are first averaged, then normalized and arranged in a matrix format to indicate objective-interested party relationships as shown in table 2. the table also shows the relative importance of each objective where the objective that receives the highest overall weight of 0.97 is increasing forest stocks. table 2 objective values and weights given by each interested party involved in the multiobjective forest management decision process objectives interested parties pub lnd prv ngo overall weights improving economic benefits 0.10 0.26 0.18 0.09 0 .63 increasing water yield 0.25 0.12 0.12 0.24 0 .73 increasing recreation or other environmental benefits 0.23 0.07 0.19 0.22 0 .71 increasing forest stocks 0.24 0.28 0.24 0.21 0 .97 reducing operational costs 0.06 0.18 0.13 0.07 0 .44 increasing non-timber products yield 0.12 0.09 0.14 0.17 0 .52 m ax 6.50 6.37 6.60 6.16 cr 0.08 0.06 0.10 0.03 on the other hand, reducing operational costs receives the lowest overall weight of 0.44. in a similar manner, the relationships between the four alternative forest resources development programs and the desired objectives are given in table 3. using values from table 2, table 3 and the weights in appendix ai, the global utility (or budget allocation) value with respect to a particular management alternative program is determined using equation 6.  i gp               )( 6 1 4 1 k ikjk j j lpmslpolpig (6) where gpi is the desired global budgetary ratio (or global utility value) obtained using management alternative program i, lpigj is the specific utility level or relative weight ijahp article: tecle, perez-verdin/analytic hierarchy process application for multiple purpose forest resources management budget allocation in durango, mexico international journal of the analytic hierarchy process 50 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.422 assigned to interest group j, lpojk is the local utility or relative weight of objective k from the point of view of interest group j, and lpmsik is the specific utility level or relative weight of management alternative i (national program) with respect to objective k. note that a special attention should be paid in determining gpi because all management objectives do not have links with all management alternatives. hence, there are no lpmsik values reported for some of the cells in table 3. in such a situation, saaty (1988) recommends to follow a procedure similar to that we have done in this paper. in any case, the sum of all gpi's must be equal to one. table 3 local priorities of management alternatives (national programs) with respect to decision objectives (level 3 to 4) objectives management alternatives (national programs)* m ax  cr fd pd ntp wh improving economic benefits 0.44 ----0.17 0.39 3.03 0.02 increasing water yield 0.20 0.20 -----0.60 3.00 0.00 increasing recreation or other environmental benefits 0.17 0.39 -----0.44 3.03 0.02 increasing forest stocks 0.16 0.42 0.14 0.27 4.07 0.03 reducing operational costs 0.67 ----0.33 -----2.00 0.00 increasing non-timber products yield 0.30 0.16 0.54 -----3.01 0.01 * dotted cells represent the management objectives that do not have costs associated with a particular national program. also, note that equation 7 is the numerical application of equation 6 that determines the global utility value (0.307) for the forest development program (fd). gpfd = [0.18 × {(0.10 × 0.44) + (0.25 × 0.20) + (0.23 × 0.17) + (0.24 × 0.16) + (0.06 × 0.67) + (0.12 × 0.30)}+ 0.51 × {(0.26 × 0.44) + (0.12 × 0.20) + (0.07 × 0.17) + (0.28 × 0.16) + (0.18 × 0.67) + (0.09 × 0.30)} + 0.17 × {(0.18 × 0.44) + (0.12 × 0.20) + (0.19 × 0.17) + (0.24 × 0.16) + (0.13 × 0.67) + (0.14 × 0.30)}+ 0.14 × {(0.09 × 0.44) + (0.24 × 0.20) + (0.22 × 0.17) + (0.21 × 0.16) + (0.07 × 0.67) + (0.17 × 0.30)}] = 0.307 (7) the global utility values for the other three national resources development programs are similarly determined to be 0.214 for pd, 0.177 for ntp, and 0.303 for wh. these show that the resource development program with the highest global utility value is forest development (fd), very closely followed by water harvesting (wh). the calculated values are based on the wishes and aspirations (or utilities) of representative groups of community members (zanakis et al., 1995; chiou et al., 2009). based on this, the government would allocate the budget for forest resources management among the four national resources development programs to best achieve the desired management objectives. ijahp article: tecle, perez-verdin/analytic hierarchy process application for multiple purpose forest resources management budget allocation in durango, mexico international journal of the analytic hierarchy process 51 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.422 in the past, decisions on budget allocations followed no systematic approach. for instance, the available funds for the particular budget year considered in this paper were allocated as 53% for forest development, 35% for plantations development, 12% for nontimber products, and zero to water harvesting. however, if the budget were distributed among all four national resources development programs water harvesting and environmental service development program would take some part of the budget. such an arrangement would proportionally reduce the funds that would be allocated to forest development and the other two programs. figure 3 compares this revised budget allocation with the actual budget distribution above. the hypothetical allocations of the annual budget of $4.10 million among the four resources development programs (fd, pd, ntp and wh) for one year were obtained using equation 6 and in a manner similar to the calculation for fd using equation 7. figure 3 comparing actual (current) and hypothetical budget allocations to the four resources development programs in the state of durango, mexico; the actual budget allocation does not include funding to the water harvesting program (wh) in the year 5. sensitivity analysis an important aspect of any modeling effort is testing the reliability and robustness of the model in performing as desired under varying input variables/parameters. this is sensitivity analysis and it can be done by testing the effects of changes in one or more of the utilities of the interested parties on the desired budget allocation among the four forest resources management programs (see figure 3). performing sensitivity analyses on all the decision variables would be repetitive, very tedious, and unnecessarily time consuming. hence, we only performed a sensitivity analysis on the effect of changes in the utilities of one interested party (or decision-maker), the landowners, on the global utility values (see figure 4). in this analysis, the utilities of the other three decisionmakers are kept constant. 0.50 1.00 1.50 2.00 2.50 fd pd ntp wh alternatives (national programs) m il li o n s o f d o ll a rs .. current desired ijahp article: tecle, perez-verdin/analytic hierarchy process application for multiple purpose forest resources management budget allocation in durango, mexico international journal of the analytic hierarchy process 52 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.422 the results of the sensitivity analysis show that the analytic hierarchy process is very reliable for optimal budget allocation among the different and competing forest resources management programs in durango, mexico. this reliability is indicated by the small variation in the global utility values of each natural resources development program with changes in the landlord's utility values (see figure 4). as shown in the figure, the change in the global utility value for water harvesting is very little, ranging from 0.350 to 0.370, as the landlord's utility value decreases from 0.5 to 0.0. on the opposite side, when the landowner’s utility value is between 0.5 and 1.0, there is a slight rise, from 0.350 to 0.385, in the global utility for forest development. likewise, the global utility values for development of plantation and the non-timber products development programs are lower than those of the above and do not change significantly with changes in the landlord's utility values. also the basis for providing the necessary budget is related to the land tenure system prevailing in the state of durango, mexico, where new forest policies are formulated to help landowners. hence, it makes sense that the preference for budget allocation favors landowners, albeit slightly, compared to the other interested parties involved. in spite of the latter, however, there is little or no sensitivity in the derived utility levels of the resource development programs with changes in the decision-maker’s utility values. hence, for the little difference observed between the outcomes of the two most desirable forest resources management alternatives, forest development and water harvesting. figure 4 effect of changes in landowner’s utility values on global priority values. in this test, the utility values of the other interested parties are assumed to be equal to corroborate the sensitivity analysis results, we performed a t-test, along with a test for homogeneity of variances, by dividing the landowner’s utility values into two classes: 0.0 0.1 0.1 0.2 0.2 0.3 0.3 0.4 0.4 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 landowners' utility values g lo b a l p ri o ri ty v a lu e s fd pd ntp wh ijahp article: tecle, perez-verdin/analytic hierarchy process application for multiple purpose forest resources management budget allocation in durango, mexico international journal of the analytic hierarchy process 53 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.422 those having utility values less than 0.50 as class 1, and those with utility values greater than or equal to 0.50 as class 2. in the process, the four resources development programs of fd, pd, ntp and wh are considered to be dependent variables while their utility values are classes or factors of independent variables. in the end, a result of the t-test indicates existence of no significant differences between the two classes. this means that changing the utility values of landowners does not have a significant effect on the overall preference or on the prioritization of one resource development program over the others under consideration. 6. conclusions and recommendations this study used the multi-criterion decision-making technique, ahp, to optimally allocate a scarce forest management budget among the different forest resources development programs in the state of durango, mexico. we analyzed the attitudes of four representative decision-making groups toward an optimal allocation of a scarce budgetary fund among four national programs to achieve six management objectives. two of the four programs, forest development and water harvesting/environmental services, received the highest budget allocation in accordance with their higher global utility values. however, since water harvesting/environment services constitutes a new program, its funding has to come by reducing the budgetary share of the other programs, especially that of forest development. this should lead to a fair and optimal allocation of the entire budget among the four national forest resources management programs in durango, mexico. there are at least three major reasons for the need for optimal forest resources management budget allocation. first, since the money comes from taxes, its distribution should make it possible to provide the greatest amount of goods and services to the largest number of people in the state. second, an optimal budgetary distribution process can help resources managers put their scarce budgetary funds where they are most needed and efficiently used. third, the developed approach must be a convenient technique for simultaneous management of multiple forest resources to achieve multiple objectives as in tecle (1992), balteiro and romero (2008), saaty (2008b), and perez-verdin et al. (2009). the multi-objective approach is more advantageous than a traditional and inefficient, single objective and single resources management approach (tecle, 2007). using the multi-objective decision-making approach in this study water harvesting and environmental services can be handled simultaneously with forest and non-timber development programs to benefit not only landowners but the entire ejido community who uses the resources as a whole. hence, we recommend the use of the ahp not only because it enables optimal allocation of scarce resources to do the greatest good for the largest number of people, but because it also engenders active participation of the public in resolving natural resources management-related conflicts (balteiro et al., 2009; nordstrom, 2010; groselj et al., 2016; nilsson et al., 2016). this is possible because ahp is capable of handling numerous types of qualitative as well as quantitative information such as ordinal data from public opinions and cardinal and ratio data types generated using all sorts of research endeavors (saaty, 2008a; šporčić, et al. 2010; šporčić, 2012). one method such as that of duke and aull-hyde (2002), for example, can be used to gather a large amount of public opinions. an additional strength of the ahp is its ability to normalize non-commensurable data and express it in the form of ratio scale to make ijahp article: tecle, perez-verdin/analytic hierarchy process application for multiple purpose forest resources management budget allocation in durango, mexico international journal of the analytic hierarchy process 54 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.422 handling of a complex multiobjective problem analysis easy (tecle, 1992; mendoza & prabhu, 2000; saaty, 2008a). the ahp is widely used to evaluate and solve many different kinds of multi-objective decision problems (saaty, 1990, 2001, 2008a, 2008b; schmoldt et al., 2001a, 2001b; duke & aull-hyde, 2002; macharis et al., 2004; hajkowicz & higgins, 2006), in general, and various types of forest resources management problems such as those in balteiro and romero (2008), ananda and herath (2009), perez-verdin et al. (2009, 2011), šporčić et al. (2010), poff et al. (2010, 2012), to mention some. but, to the best of our knowledge, this is the first time the ahp has specifically been used to optimally distribute scarce budget resources among four ejido-based forest resources management programs designed to achieve six objectives of four interested parties. a sensitivity analysis was conducted on the effects of varying the utility values of landowners on the global utility values that determined the budget distribution among the four competing forest resources management programs. the analysis results show the reliability and robustness of the ahp in allocating the budget. another benefit of using the ahp is its integrative assessment of numerous interacting and at times competing forest resources components. it takes advantage of not only the synergistic relationships among the various components of a multi-objective decision problem, but also incorporates the risks and uncertainties inherent in such a complex problem to reach a realistic and acceptable decision outcome (macharis et al., 2004). weights and public preferences are heavily used to hierarchically interconnect various decision-makers, their objectives and the different alternative management schemes involved. these aspects of the ahp are the reasons for its desirability, wide use and success to solve various types of multi-objective forest resources management problems. ijahp article: tecle, perez-verdin/analytic hierarchy process application for multiple purpose forest resources management budget allocation in durango, mexico international journal of the analytic hierarchy process 55 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.422 appendix a ai. pairwise comparisons (a) of stakeholder’s utilities and vector of weights (w) a w                  12/13/11 213/12/1 3314 124/11 ng pr ld pu ngprldpu aaw ik ×             14.0 17.0 51.0 18.0 ; m ax =4.25; cr=0.09 pu = public sector; ld = landowners; pr = private sector; and ng = non-government organizations aii. pairwise comparisons of stakeholders’ management objective values and vector of weights public sector preferences ee wy rec fs opc ntp ee 1.00 0.33 0.33 0.33 3.00 1.00 wy 3.00 1.00 2.00 0.50 3.00 3.00 rec 3.00 0.50 1.00 2.00 4.00 1.00 fs 3.00 2.00 0.50 1.00 3.00 2.00 opc 0.33 0.33 0.25 0.33 1.00 0.50 ntp 1.00 0.33 1.00 0.50 2.00 1.00 ×                     12.0 06.0 24.0 23.0 0.25 0.10 ; m ax =6.50; cr=0.08 landowners preferences ee wy rec fs opc ntp ee 1.00 3.00 3.00 1.00 2.00 2.00 wy 0.33 1.00 2.00 0.50 0.33 2.00 rec 0.33 0.50 1.00 0.25 0.33 1.00 fs 1.00 2.00 4.00 1.00 3.00 2.00 opc 0.50 3.00 3.00 0.33 1.00 2.00 ntp 0.50 0.50 1.00 0.50 0.50 1.00 ×                     09.0 18.0 28.0 07.0 0.12 0.26 ; m ax =6.37; cr=0.06 private sector preferences ee wy rec fs opc ntp ee 1.00 3.00 1.00 1.00 0.50 1.00 wy 0.33 1.00 1.00 0.50 1.00 1.00 rec 1.00 1.00 1.00 0.50 3.00 2.00 fs 1.00 2.00 2.00 1.00 3.00 1.00 opc 2.00 1.00 0.33 0.33 1.00 1.00 ntp 1.00 1.00 0.50 1.00 1.00 1.00 ijahp article: tecle, perez-verdin/analytic hierarchy process application for multiple purpose forest resources management budget allocation in durango, mexico international journal of the analytic hierarchy process 56 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.422 non-governmental organizations preferences ee wy rec fs opc ntp ee 1.00 0.33 0.33 0.33 1.00 1.00 wy 3.00 1.00 1.00 1.00 3.00 2.00 rec 3.00 1.00 1.00 1.00 4.00 1.00 fs 3.00 1.00 1.00 1.00 3.00 1.00 opc 1.00 0.33 0.25 0.33 1.00 0.33 ntp 1.00 0.50 1.00 1.00 3.00 1.00 ×                     17.0 07.0 21.0 22.0 0.24 0.09 ; m ax =6.16; cr=0.03 ee = improve economic benefits; wy = improve water yield; rec = improve recreation/ environmental services; fs = increase forest stocks; opc = reduce operational costs; and ntp = increase non-timber products yield. aiii. pairwise comparisons of forest resources management programs, and corresponding vector of weights (w) and other parameters related to each objectives. improve economic benefits fd ntp wh w fd 1.00 3.00 1.00 0.44 ntp 0.33 1.00 0.50 0.17 wh 1.00 2.00 1.00 0.39 m ax  =3.03; cr=0.02 improve water yield fd pd wh w fd 1.00 1.00 0.33 0.20 pd 1.00 1.00 0.33 0.20 wh 3.00 3.00 1.00 0.60 m ax =3.00; cr=0.00 improve recreation and environmental services fd pd wh w fd 1.00 0.50 0.33 0.17 pd 2.00 1.00 1.00 0.39 wh 3.00 1.00 1.00 0.44 m ax =3.03; cr=0.02 increase forest stocks fd pd ntp wh w fd 1.00 0.50 1.00 0.50 0.16 pd 2.00 1.00 3.00 2.00 0.42 ntp 1.00 0.33 1.00 0.50 0.14 wh 2.00 0.50 2.00 1.00 0.27 m ax =4.07; cr=0.02 reduce operational costs m ax =2.00; cr=0.00 fd ntp w fd 1.00 2.00 0.67 ntp 0.50 1.00 0.33 increase non-timber products fd pd ntp w fd 1.00 2.00 0.50 0.30 pd 0.50 1.00 0.33 0.16 ntp 2.00 3.00 1.00 0.54 m ax =3.01; cr=0.01 fd = forest development; pd = plantations development; ntp = non-timber products; wh = water harvesting ijahp article: tecle, perez-verdin/analytic hierarchy process application for multiple purpose forest resources management budget allocation in durango, mexico international journal of the analytic hierarchy process 57 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.422 references alcorn, j.b. and toledo, v.m. (1998). resilient resource management in mexico's forest ecosystems: the contribution of property rights. in f. berkes and c. folke (eds), linking social and ecological systems: management practices and social mechanisms for building resilience (216-249). cambridge, u.k: cambridge university press. http://www.consecol.org/vol4/iss2/art5/ alonso, j.a., and lamata, m.t. (2006). consistency in the analytic hierarchy process: a new approach. international journal of uncertainty, fuzziness and knowledge-based systems, 14(4). doi: 10-1142/s0218488506004126. ananda, j. and herath, g. (2009). a critical review of multi-criteria decision making methods with special references to forest management and planning. ecological economics, 68(10), 2535-2548. doi: https://doi.org/10.1016/j.ecolecon.2009.05.010 ansah, r.h., sorooshian, s. and bin mustafa, s. (2015). analytic hierarchy process decision making algorithm. global journal of pure and applied mathematics, 11(4) 2015. http///ssrn.com/abstract=2683706. balteiro, l.d., and romero, c. (2008). making forestry decisions with multiple criteria: a review and assessment. forest ecology and management, 255(2008), 3222-3241. doi: https://doi.org/10.1016/j.foreco.2008.01.038 balteiro, l.d., gonzalez-pachon, j. and romero, c. (2009). forest management with multiple criteria and multiple stakeholders: an application to two public forests in spain. scandinavian journal of forest research, 24(1), 87-93. doi: https://doi.org/10.1080/02827580802687440 balteiro, l.d. romero, c. (2001). combined use of goal programming and the analytic hierarchy process in forest management. in d.l. schmoldt, j. kangas, g.a. mendoza, and m. pesonen (eds), the analytic hierarchy process in natural resource and environmental decision-making (81-95). norwell, ma : kluwer academic publishers. doi: https://doi.org/10.1007/978-94-015-9799-9_6 bradford, j.b and d’amato, a.w. (2012). recognizing trade-offs in multiple-objective land management. frontiers in ecology and the environment, 10(4), 210-216. doi: 10.1890/110031. doi: 10.1890/110031 cheng, e.w.l., and li, h. (2004). contractor selection using the analytic network process. construction management and economics, 2004(22), 1021-1032. doi: https://doi.org/10.1080/0144619042000202852 chiou, c.w., chen, c.c. and chiou, s.c. (2009). a decision-making model of budget allocation for the restoration of traditional settlement buildings. international conference on management and service science on 20-22 sept. 2009 in wuhan, china. doi: 10.1109/icmss.2009.5303265 https://doi.org/10.1016/j.ecolecon.2009.05.010 https://doi.org/10.1016/j.foreco.2008.01.038 https://doi.org/10.1080/02827580802687440 https://doi.org/10.1080/0144619042000202852 http://ieeexplore.ieee.org/xpl/mostrecentissue.jsp?punumber=5300802 http://ieeexplore.ieee.org/xpl/mostrecentissue.jsp?punumber=5300802 https://doi.org/10.1109/icmss.2009.5303265 ijahp article: tecle, perez-verdin/analytic hierarchy process application for multiple purpose forest resources management budget allocation in durango, mexico international journal of the analytic hierarchy process 58 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.422 duckstein, l. and a. tecle. (1993a). multiobjective analysis in water resources: part ii a new typology of mcdm techniques. in: marco, harboe and salas (eds.), stochastic hydrology and its use in engineering. kluwer academic publishers, dordrecht, the netherlands. pp. 333-344. doi: https://doi.org/10.1007/978-94-0111697-8_21 duckstein, l., and tecle, a. (1993b). multiobjective analysis in water resources: part i – numerical versus non-numerical and discrete versus continuous alternatives. in: marco, harboe and salas (eds.), stochastic hydrology and its use in engineering (319-332). dordrecht, the netherlands: kluwer academic publishers. doi: https://doi.org/10.1007/978-94-011-1697-8_20 duckstein, l. and tecle, a. (2002). multicriterion analysis in water resources management. in h.h.g. savenije and a.y. hoeksta (eds.), water resources management, encyclopedia of life support systems (eolss), oxford, uk: unesco, eolss publishers. doi: https://doi.org/10.1061/(asce)0733-9496(1990)116:6(840) duke, j.m., and aull-hyde, r. (2002). identifying public preferences for land preservation using the analytic hierarchy process. ecological economics, 42, 131-145. doi: https://doi.org/10.1016/s0921-8009(02)00053-8 ford, r.m., anderson, n,m, nitschke, c., bennet, l.t. and williams, k.j.h. (2017). psychological values and cues as a basis for developing socially relevant criteria and indicators for forest management. forest policy and economics, 78(2017), 141-150. doi: https://doi.org/10.1016/j.forpol.2017.01.018 forman, e. and , peniwati, k. (1998). aggregating individual judgments and priorities with the analytic hierarchy process. european journal of operational research, 108(1),165-169. doi: https://doi.org/10.1016/s0377-2217(97)00244-0 groselj, p., hodges, d.g. and stirn, l.z. (2016). participatory and multi-criteria analysis for forest (ecosystem) management: a case study of pohorje, slovenia. forest policy and economics, 71(2016), 80–86. doi: https://doi.org/10.1016/j.forpol.2015.05.006 hajkowicz, s., and higgins. a. (2006). a comparison of multiple criteria analysis techniques for water resource management. european journal of operational research, 184(2008), 255–265. doi: https://doi.org/10.1016/j.ejor.2006.10.045 hossain, s.m. y., and robak. e.w. (2010). a forest management process to incorporate multiple objectives: a framework for systematic public input. forests, 1, 99-113. doi: 10.3390/f1030099 ishizaka, a., and labib. a. (2011). review of the main developments in the analytic hierarchy process. expert systems with applications, 38(11), 14336–14345. doi: https://doi.org/10.1016/j.eswa.2011.04.143 jenkins, t. (2005). multi-objective forest planning. forestry, 78(4),457-458. doi:https://doi.org/10.1093/forestry/cpi051. https://doi.org/10.1061/(asce)0733-9496(1990)116:6(840) https://doi.org/10.1016/s0921-8009(02)00053-8 https://doi.org/10.1016/j.forpol.2017.01.018 http://www.sciencedirect.com/science/journal/03772217 https://doi.org/10.1016/s0377-2217(97)00244-0 http://www.sciencedirect.com/science/article/pii/s1389934115300058 http://www.sciencedirect.com/science/article/pii/s1389934115300058 http://www.sciencedirect.com/science/journal/13899341 http://www.sciencedirect.com/science/journal/13899341 http://www.sciencedirect.com/science/journal/13899341/71/supp/c https://doi.org/10.1016/j.forpol.2015.05.006 https://doi.org/10.1016/j.ejor.2006.10.045 http://www.sciencedirect.com/science/journal/09574174 http://www.sciencedirect.com/science/journal/09574174/38/11 https://doi.org/10.1016/j.eswa.2011.04.143 ijahp article: tecle, perez-verdin/analytic hierarchy process application for multiple purpose forest resources management budget allocation in durango, mexico international journal of the analytic hierarchy process 59 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.422 kangas, j. (1992). multiple-use planning of forest resources by using the analytic hierarchy process. scandinavian journal of forest research, 7, 259-268. doi: https://doi.org/10.1080/02827589209382718 kangas, j. (1994). an approach to public participation in strategic forest management planning. forest ecology and management, 70, 75-88. doi: https://doi.org/10.1016/03781127(94)90076-0 kangas, j. (1999). the analytic hierarchy process (ahp): standard version, forestry applications and advances. in f. helles, p. holten-andersen, and l. wichmann (eds), multiple use of forests and other natural resources (96-105). norwell, ma: kluwer academic publishers. doi: https://doi.org/10.1007/978-94-011-4483-4_8 kangas, j., kangas, a., leskinen, p. and pykalainen, j. (2001). mcdm methods in strategic planning of forestry on state-owned lands in finland: applications and experiences. journal of multi-criteria decision analysis, 10(5), 257-271. doi: 10.1002/mcda.306 krcmar, e, van kooten, g.c., vertinsky, i. (2005). managing forest and marginal agricultural land for multiple trade-offs: compromising on economic, carbon and structural diversity objectives. ecological modelling, 185(2), 451-468. doi: https://doi.org/10.1016/j.ecolmodel.2004.12.014 leskinen, p. (2000). measurement scales and scale independence in the analytic hierarchy process. journal of multicriteria decision analysis, 9(4), 163. doi: 10.1002/1099-1360(200007)9:4<163::aid-mcda274>3.0.co;2-l macharis, c., springael, j., de brucker, k. and verbeke, a. (2004). promethee and ahp: the design of operational synergies in multicriteria analysis. strengthening promethee with ideas of ahp. european journal of operational research, 153, 307−317. doi: https://doi.org/10.1016/s0377-2217(03)00153-x mardani, a., jusoh, a., nor, k., khalifah, z., zakwan, n. & valipour, a. (2015). multiple criteria decision-making techniques and their applications – a review of the literature from 2000 to 2014. economic research-ekonomska istraživanja, 28(1), 516571. doi: https:/doi/full/10.1080/ 1331677x.2015.1075139. mendoza, g., and prabhu, r. (2000). multiple criteria decision making approaches to assessing forest sustainability using criteria and indicators: a case study. forest ecology and management, 131(1-3), 107-126. doi: https://doi.org/10.1016/s03781127(99)00204-2 moseley, k.r., ford, w.m., edwards, j.w. and strager, m.p. (2009). a multi-criteria decision making approach for management indicator species selection on the monongahela national forest, west virginia. usda forest service northern research station, research paper nrs-12. doi: https://doi.org/10.2737/nrs-rp-12 https://doi.org/10.1080/02827589209382718 https://doi.org/10.1016/0378-1127(94)90076-0 https://doi.org/10.1016/0378-1127(94)90076-0 https://doi.org/10.1016/j.ecolmodel.2004.12.014 https://doi.org/10.1016/s0377-2217(03)00153-x http://www.tandfonline.com/author/mardani%2c+abbas http://www.tandfonline.com/author/jusoh%2c+ahmad http://www.tandfonline.com/author/md+nor%2c+khalil http://www.tandfonline.com/author/khalifah%2c+zainab http://www.tandfonline.com/author/zakwan%2c+norhayati http://www.tandfonline.com/author/valipour%2c+alireza http://www.tandfonline.com/toc/rero20/28/1 https://doi.org/10.1016/s0378-1127(99)00204-2 https://doi.org/10.1016/s0378-1127(99)00204-2 https://doi.org/10.2737/nrs-rp-12 ijahp article: tecle, perez-verdin/analytic hierarchy process application for multiple purpose forest resources management budget allocation in durango, mexico international journal of the analytic hierarchy process 60 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.422 napoli, g., and schilleci, f. (2014). application of an analytic network process in the planning process: the case of an urban transformation in palermo (italy). a conference paper. doi: 10.1007/978-3-319-09150-1_22. nilsson, h., nordstrom, e.m. and ohman, k. (2016). decision support for participatory forest planning using ahp and topsis. forests, 7(5), 100. doi: 10.3390/f7050100. niemela, j., j. young, d. alard, m. askasibar, k. henle, r. johnson, m. kurttila, t.b. larsson, s. matouch, p. nowicki, r. paiva, l. portoghesi, r. smilders, a. stevenson, u. tartes, and a. watt. (2005). identifying, managing and monitoring conflicts between forest biodiversity conservation and other human interests in europe. forest policy and economics, 7(2005), 877-890. doi: https://doi.org/10.1016/j.forpol.2004.04.005 nordstrom, e.v. (2010). integrating multiple criteria decision analysis into participatory forest planning. a doctoral thesis for the department of forest resources management, swedish university of agricultural sciences, umea, sweden. doi: https://doi.org/10.1016/j.forpol.2010.07.006 ozdemir, y., basligil, h. and karaca, m. (2011). aircraft selection using analytic network process: a case for turkish airlines. proceedings of the world congress on engineering, ii, london, u.k. perez-verdin g., navar-chaidez, j.j., kim, y-s and silva-flores, r. (2011). valuing watershed services in mexico's temperate forests. modern economy, 2(5),769-779. doi: 10.4236/me.2011.25085 pérez-verdín, g., hernández-díaz, j.c., márquez-linares , m.a. and tecle, a. (2009). aplicación de técnicas multicriterio en el manejo integral forestal en durango, méxico. madera y bosques, 15(1), 1405-0471. doi: http://dx.doi.org/10.21829/myb.2009.1511195 phua, m.h., and minowa, m. (2005). a gisbased multi-criteria decision making approach to forest conservation planning at a landscape scale: a case study in the kinabalu area, sabah, malaysia. landscape and urban planning, 71(2-4), 207-222. doi: https://doi.org/10.1016/j.landurbplan.2004.03.004 poff, b., tecle, a. neary, d.g. and geils, b. (2010). compromise programming in forest management. journal of the arizona-nevada academy of science, 42(1), 44-60. doi: https://doi.org/10.2181/036.042.0107 poff, b., tecle, a. neary, d. and geiles, b. (2012). spatio-temporal multi-objective decision making in forest management., in c. van riper iii, m. villarreal, c. van riper, and m. johnson (eds.), the colorado plateau v: research, environmental planning, and management for collaborative conservation (121-129). tuscon, az: the university of arizona press. doi: 10.2307/j.ctt183pc7f proctor, w. (2005). valuing forest resources using multi-criteria decision analysis and stakeholder participation. in. michael getzner, m., c. spash, and s. stagl (eds.), https://doi.org/10.1016/j.forpol.2004.04.005 https://doi.org/10.1016/j.forpol.2010.07.006 http://www.scielo.org.mx/scielo.php?script=sci_serial&pid=1405-0471&lng=es&nrm=iso https://doi.org/10.1016/j.landurbplan.2004.03.004 https://doi.org/10.2181/036.042.0107 ijahp article: tecle, perez-verdin/analytic hierarchy process application for multiple purpose forest resources management budget allocation in durango, mexico international journal of the analytic hierarchy process 61 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.422 developing alternatives for valuing nature: part ii: taking multiple criteria into account. london, uk: routledge. rhoda, r. and burton, t. (2010). geo-mexico: the geography and dynamics of modern mexico. canada: sombrero books. saaty, t.l. (2017). the analytic network process. download from iors.ir at 0:40+0430 on wednesday april 26, 2017. https://iors.ir/journal/article-1-27-en.pdf. saaty, t. l. (2010). principia mathematica decernendi: mathematical principles of decision making. pittsburgh, pa: rws publications. isbn 978-1-888603-10-1. saaty, t. l. (2008a). measurement and its generalization in decision making: why pairwise comparisons are central in mathematics for the measurement of intangible factors the analytic hierarchy/network process. racsam (review of the royal spanish academy of sciences, series a, mathematics), 102 (2), 251–318. doi: https://doi.org/10.1007/bf03191825 saaty, t. l. (2008b). decision making with the analytic hierarchy process. international journal services sciences, 1(1), 83-98. doi: 10.1504/ijssci.2008.017590 saaty, t.l. (2003). decision-making with the ahp: why is the principal eigenvector necessary. european journal of operational research, 145(1), 85-91. doi: https://doi.org/10.1016/s0377-2217(02)00227-8 saaty, t. l. (2001). fundamentals of the analytic hierarchy process. in d.l schmoldt, j. kangas, g.a. mendoza, and m. pesonen (eds), the analytic hierarchy process in natural resource and environmental decision making (15-35). norwell, ma: kluwer academic publishers. doi: https://doi.org/10.1007/978-94-015-9799-9_2 saaty, t.l. (1998). ranking by eigenvector versus other methods in the analytic hierarchy process. applied mathematics letters, 11(4),121-125. doi: https://doi.org/10.1016/s0893-9659(98)00068-8 saaty, t.l. (1997). that is not the analytic hierarchy process: what the ahp is and what it is not. journal of multi-criteria decision analysis, 6(6), 324-335. doi: 10.1002/(sici)1099-1360(199711)6:6<324::aid-mcda167>3.0.co;2-q saaty, t.l. (1990). how to make a decision: the analytic hierarchy process. european journal of operational research, 48, 9-26. doi: https://doi.org/10.1016/03772217(90)90057-i saaty, t. l. (1988). multicriteria decision making: the analytic hierarchy process. pittsburgh, pa: rws publications. saaty, t. l. (1980). the analytic hierarchy process. planning priority setting, resource allocation. new york, ny: mcgraw-hill inc. doi: https://doi.org/10.1016/03772217(82)90022-4 http://en.wikipedia.org/wiki/thomas_l._saaty http://en.wikipedia.org/wiki/international_standard_book_number http://en.wikipedia.org/wiki/special:booksources/978-1-888603-10-1 http://en.wikipedia.org/wiki/thomas_l._saaty http://dx.doi.org/10.1504/ijssci.2008.017590 http://www.lamsade.dauphine.fr/mcda/biblio/author/saaty-tl.html http://www.lamsade.dauphine.fr/mcda/biblio/author/saaty-tl.html https://doi.org/10.1016/s0377-2217(02)00227-8 http://en.wikipedia.org/wiki/thomas_l._saaty http://www.sciencedirect.com/science/journal/08939659 https://doi.org/10.1016/s0893-9659(98)00068-8 http://www.lamsade.dauphine.fr/mcda/biblio/author/saaty-tl.html http://www.lamsade.dauphine.fr/mcda/biblio/author/saaty-tl.html https://doi.org/10.1016/0377-2217(90)90057-i https://doi.org/10.1016/0377-2217(90)90057-i http://en.wikipedia.org/wiki/thomas_l._saaty http://en.wikipedia.org/wiki/thomas_l._saaty https://doi.org/10.1016/0377-2217(82)90022-4 https://doi.org/10.1016/0377-2217(82)90022-4 ijahp article: tecle, perez-verdin/analytic hierarchy process application for multiple purpose forest resources management budget allocation in durango, mexico international journal of the analytic hierarchy process 62 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.422 saaty, t.l. (1977). a scaling method for priorities in hierarchical structures. journal of mathematical psychology, 15, 234-281. doi: https://doi.org/10.1016/00222496(77)90033-5 saaty, t. l.. and vargas, l.g. (2006). decision making with the analytic network process: economic, political, social and technological applications with benefits, opportunities, costs and risks. new york: springer. saaty, t. and vargas, l.g. (1982). the logic of priorities. boston: kluwer-nijhoff. doi: https://doi.org/10.1007/978-94-017-3383-0_1 sadeghi, m., rashidzadeh, m.a. and soukhakian, m.a. (2012). using analytic network process in a group decision-making for supplier selection. informatica, 23(4), 621-643. schmoldt, d.l., kangas, j. and mendoza, g.a. (2001a). basic principles of decision making in natural resources and the environment. in d.l schmoldt, j. kangas, g.a. mendoza, and m. pesonen (eds),the analytic hierarchy process in natural resource and environmental decision making (1-13). dordrecht, the netherlands: kluwer academic publishers. doi: https://doi.org/10.1007/978-94-015-9799-9_1 schmoldt, d.l., kangas, j. and mendoza, g.a. and pesonen, m. (2001b). strategic and tactical planning for managing national park resources. in schmoldt et al. (eds.), the analytic hierarchy process in natural resource and environmental decision-making (67-79) dordrecht, the netherlands : kluwer academic publishers. doi: https://doi.org/10.1007/978-94-015-9799-9_5 semarnat. (2000). strategic forest program 2025. (secretaria de medio ambiente y recursos naturales). available at http://www.conafor.gob.mx/documentos_conafor/ eng/pdf/19.pdf (last time visited june 7, 2004). šporčić, m. (2012). application of multi-criteria methods in natural resources management a focus on forestry: chapter 22. in jorge martin garcia and julio javier diez (eds.), sustainable forest management current research. casero, doi: 10.5772/23667 šporčić, m., landekić, m. lovrić, m. bogdan, s. and šegotić, k. (2010). multi-criteria decision-making in forestry methods and experiences. sumarski list, 134(5-6), 275-284. tecle, a. (2007). sustainable natural resources management in an era of global climate change. forum on public policy, 3(4), 443-454. tecle, a. (1992). selecting a multicriterion decision making technique for watershed resources management. water resources bulletin, 28(1), 129-140. doi: 10.1111/j.17521688.1992.tb03159.x http://www.lamsade.dauphine.fr/mcda/biblio/author/saaty-tl.html https://doi.org/10.1016/0022-2496(77)90033-5 https://doi.org/10.1016/0022-2496(77)90033-5 http://en.wikipedia.org/wiki/thomas_l._saaty http://www.conafor.gob.mx/documentos_conafor/eng/pdf/19.pdf http://www.conafor.gob.mx/documentos_conafor/eng/pdf/19.pdf ijahp article: tecle, perez-verdin/analytic hierarchy process application for multiple purpose forest resources management budget allocation in durango, mexico international journal of the analytic hierarchy process 63 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.422 tecle, a., and duckstein, l. (1994). concepts of multicriterion decisionmaking, chapter 3. in h.p. nachtnebel (ed.), decision support system in water resources management (33-62) paris, france: unesco press. tecle, a., and jibrin, s. (2012). incorporating fuzzy logic and stochastic processes in multiobjective forest management. hydrology and water resources in arizona and the southwest, 41, 41-45. tecle, a., shrestha, b. and duckstein, l. (1998). multiobjective decision support system for multiresource forest management. group decision and negotiation, 7, 23-40. doi: https://doi.org/10.1023/a:1008671129325 tecle, a., szidarovszky, f. and duckstein, l. (1995). conflict analysis in multi-resource forest management with multiple decision-makers. nature & resources, 31(3), 8-17. thangamani, g. (2012). technology selection for product innovation using analytic network process (anp) a ccase study. international journal of innovation, management and technology. 3(5), 560-565. doi: 10.7763/imt.2012.v3.298. triantaphyllou, e. (2000). multi-criteria decision making methods: a comparative study. boston, ma: springer. doi: https://doi.org/10.1007/978-1-4757-3157-6 tversky, a., and kahneman, d. (1981). the framing of decisions and the psychology of choice. science, 211,453-458. doi: 10.1126/science.7455683 vaidya, o.s., and kumar. s. (2006). analytic hierarchy process: an overview of applications. european journal of operational research, 169(1), 1-29. doi: https://doi.org/10.1016/j.ejor.2004.04.028 white, s.w., and bordoloi, s.k. (2015). a review of dea-based resource and cost allocation models: implications for services. international journal of service and operations management, 20(1). 86-101. doi: 10.1504/ijsom.2015.065973 zahedi, f. (1987). a utility approach to the analytic hierarchy process. mathematical modeling, 9, 387-395. doi: https://doi.org/10.1016/0270-0255(87)90497-0 zanakis, s.h., mandakovic, t., gupta, s.k., sahay, s. and hong, s. (1995). a review of program evaluation and fund allocation methods within the service and government sectors. socio-economic planning sciences, 29(1), 59-79. doi: https://doi.org/10.1016/0038-0121(95)98607-w https://doi.org/10.1016/j.ejor.2004.04.028 http://dx.doi.org/10.1504/ijsom.2015.065973 https://doi.org/10.1016/0270-0255(87)90497-0 https://doi.org/10.1016/0038-0121(95)98607-w ijahp article: v, radha ramanan/identification and ranking of competencies that positively influence customer service: an indian case study international journal of the analytic hierarchy process 28 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.637 identification and ranking of competencies that positively influence customer service: an indian case study shanujas v research scholar national institute of technology calicut 673601, india shanujas.v@gmail.com thiyagarajan radha ramanan 1 radha_ramanan@nitc.ac.in abstract this study identifies and ranks the competencies that positively influence customer service and are therefore helpful in creating customer satisfaction. the study is conducted in a cooperative bank in a district of south india. the competencies were identified after direct observation of the employee’s jobs, interviews with the employees, a review of the job descriptions and specifications, a review of the literature, the checklist method, and administration of a questionnaire. using the delphi method, a consensus was achieved. the analytic hierarchy process was used to rank the competencies. there were five job competencies that were identified and were then ranked. three distinct groups of employees that included managers, accountants, and clerical staff were identified, and their responses were collected using a 1 to 9 pairwise comparison scale. normalization of the data was achieved by computing the geometric mean of the responses. the ranking revealed that the relationship management competency had the top priority, and the teambuilding and technical competencies had the least priority in providing customer service. the major finding of the study was the identification of the cognitive competency as the most important for managers and accountants and the relationship management competency for the clerical staff to achieve customer satisfaction. these findings will be helpful in identifying the training needs of these three categories of employees to help improve their customer service performance through the job competencies. keywords: customer service performance; employee competencies; cooperative banks; ahp approach 1. introduction at the heart of any successful business is a group of competent employees. ever since the dawn of liberalization, privatization and globalization, the world has become very competitive. it has become particularly important to manage and retain the competency level and performance of a business’ human resources in order to gain a competitive advantage in the market. this competition is present in the financial acknowledgements: we would like to thank the anonymous reviewer whose suggestions have improved the presentation and understanding of this paper greatly. we would also like to thank them for the detailed review which helped us improve the article. mailto:shanujas.v@gmail.com mailto:radha_ramanan@nitc.ac.in ijahp article: v, radha ramanan/identification and ranking of competencies that positively influence customer service: an indian case study international journal of the analytic hierarchy process 29 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.637 services sector, and banks have searched for ways to attract more customers and increase the volume of business. even though product differentiation is not possible in the banking sector, organizations should focus on improving the quality of services that are delivered to customers. modern banking organizations have become aware of the role of employees in attracting, retaining and satisfying customers by providing superior customer service. indian banks have secured a better position in the world because of their remarkable performance with assets and return on assets. the banking and financial sector plays a significant role in the economic progression of india. the indian banking sector comprises scheduled banks (commercial banks and cooperative banks) and non-scheduled banks. cooperative banks have attained many landmarks since their formation and play a crucial role in the life of ordinary indians. ordinary people are the primary customers of cooperative banks, and they seek the support of employees for the completion of banking transactions. even though the cooperative banks have widespread branches and a good number of employees, they are far behind many nationalized and private banks with regard to their functioning and other innovative employee related customer service operations. modern banking organizations are more concerned with the customer service domain and have initiated separate wings and programs to improve employee competencies and the quality of customer service, and therefore increase customer satisfaction and profit. cooperative banks must also emphasize finding competent employees and initiate programs to improve the competencies of their current employee’s abilities to provide superior services to customers. the development of an employee competency framework would enable the cooperative banks to identify the critical competencies that would enhance the customer service performance of their employees. this would allow the cooperative banks to make strategic decisions on technological applications, recruitment and selection, and training and development programs based on the identified critical employee competencies. the present study aims to identify the job competencies needed to improve the quality of customer service, ascertain their relative weights and rank the competencies. in order to identify different job competencies, the study adopted methods such as direct observation of employee activities, a semi-structured interview with the employees, scrutiny of the job description and specifications of the managers, accountants and clerical staff, a literature review, the checklist method, a questionnaire survey and the delphi method. further, the analytic hierarchy process (ahp) method was applied to understand the priority and rank of the job competencies in enhancing customer service performance from the managers, accountants and the clerical staff separately. the ahp is a multi-criteria decisionmaking technique that was developed by saaty (1980) to understand the priority of things based on pairwise comparison. the study employed a 1 to 9 pairwise comparison scale proposed by saaty to understand the priority of each job competency over the others in providing excellent customer services. the geometric mean method was used to aggregate the individual judgments of the sample into a single representative judgment of the sample group. the ahp method is the most efficient tool for ranking the alternatives when decisions are made based on several criteria and sub-criteria (mardani et al. 2015; tahriri et al., 2008). the paper is further organized as follows: the second section is devoted to a literature review of job competencies and ahp approaches employed in previous studies. the third part includes the research methodology with variables identification phases including observation, checklist, delphi method and questionnaire survey, sample characteristics, and description of the ahp method. the fourth section is devoted to the results and discussions, and the final sectiondiscusses the conclusions and managerial implications of the study. ijahp article: v, radha ramanan/identification and ranking of competencies that positively influence customer service: an indian case study international journal of the analytic hierarchy process 30 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.637 2. literature review customer service has been defined as the firm’s ability to perform all of the necessary tasks for understanding the customer needs, meeting customer demands, solving problems and providing excellent service (kotler, 2000). customer service is an important topic becasuse of it’s strong linkage to customer satisfaction and customer loyalty which then leads to the financial performance of the firms (duncan & elliott, 2002, 2004; abu-elsamen et al., 2011). all of the employees who have direct contact with customers are responsible for satisfying them. drucker (1968) claimed that customer service is not an assigned or specified job, and that every employee in the organization should take the initiative to satisfy the customer. employee competencies play a crucial role in the delivery of superior service to customers (abu-elsamen et al., 2011). competencies enable the employees to perform better when compared with an intelligence quotient and personality tests (mcclelland, 1973). spencer and spencer (1993) described competencies as the fundamental characteristics or traits of an individual which enable him to display superior performance in his job. there have been widespread studies on the identification of employee competencies in different work contexts (jena & sahoo, 2014; lakshminarayanan et al., 2016; trivellas et al., 2015 etc.). kunnanatt (2008) found that achievement orientation is essential for bank managers to perform well at work. the survey conducted by hopkins and bilimoria (2008) among top-level executives of financial institutions in the united states concluded that competencies such as change catalyst, selfconfidence, inspirational leadership and achievement orientation are the four social and emotional competencies that distinguish the most successful managers. sharma (2012) investigated the level of social and emotional competencies of employees in india and north america. competencies such as emotional self-control, selfmanagement, emotional awareness, self-confidence, accurate self-assessment, trustworthiness, initiative, conscientiousness, achievement orientation, empathy, adaptability, organizational knowledge, social awareness, service orientation, inspirational leadership, relationship management, change catalyst, developing others, influence, conflict management, and teamwork and collaboration were identified as critical for employees of banking and financial sectors in the indian context. similar competencies were identified for bank managers in the kingdom of saudi arabia (alferaih, 2017). multiple-criteria decision-making (mcdm) or multiple-criteria decision analysis (mcda) is a sub-discipline of operations research that clearly assesses the multiple complex criteria in decision making in both quantitative and qualitative factors. numerous mcdm methods have been proposed by many researchers in different periods for making ideal decisions. most of the prominent studies on operations research adopted the recently developed hybrid and modular techniques such as topsis (hwang & yoon, 1981), saw (maccrimmon & rand, 1968), ahp (saaty, 1980), anp (saaty, 1996), vikor (opricovic, 1998), dematel (fontela & gabus, 1976), dea (charnes, cooper, & rhodes, 1978), promethee (mareschal, brans, & vincke, 1984), electre (roy, 1971) and their modification of fuzzy and grey number theory. in addition to this, grip, copras, aras, ruta, moora, utadisgms, swara, and waspas have also been recently developed for solving decision problems. numerous decision-making tools are available for an operations researcher, however, choosing the best technique remains a challenge. mardani et al. (2015) systematically studied the methodologies of the mcdm tools by reviewing 393 research articles published from 2000 to 2014 in more than 120 prominent journals, and investigated which method is used most by researchers. the ijahp article: v, radha ramanan/identification and ranking of competencies that positively influence customer service: an indian case study international journal of the analytic hierarchy process 31 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.637 results indicated that the ahp method was ranked first. tahriri et al. (2008) also found that the ahp method is the most efficient tool for ranking the alternatives when decisions are made based on several criteria and sub-criteria. although nursikuwagus, melian and permatasari (2018) employed the fuzzy topsis approach to predict the level of student competence, and sekhar, patwardhan and vyas (2015) adopted the delphi-ahp-topsis method to rank the indicators of intellectual capital including employee competencies, most of the prominent studies in the literature employed the analytic hierarchy process (ahp) technique to explore and rank competencies in many domains. islam and bin mohdrasad (2006) applied the ahp approach to evaluate employee performance. hafeez and essmail (2007) developed an integrated framework to define the association between organizational core competencies and related employee competencies using the ahp approach. fang, chang and chen (2010) used the ahp method to develop a competency framework for middle managers in taiwan. liu, lin & chen (2011) conducted a study to explore and rank the professional competencies needed for marketing managers in taiwanese banks using the ahp. the results found that the professional attitude competency which includes observing marketing ethics and law, initiative, responsibility, tolerance, and teamwork are ranked respectively. demirtas et al. (2015) employed the analytic network process to determine the weight and ranks of job competencies for a state bank in turkey. wang et al. (2016) developed a competency evaluation index system for operating room nurses (orn) in china using the ahp approach. chao, chou and lai (2017) also applied the ahp method to identify competencies for service sector managers using three factors and fifteen subfactors. the present study aims to understand the priority or ranks of the main criteria and sub-criteria only; there is no dependence among sub-criteria. therefore, application of the anp and topsis is not required and the ahp is considered the appropriate tool for the present study. even though various competency studies are available with the ahp approach, studies that aim to identify and understand the priority of job competencies in improving customer service performance of employees are rare, and cooperative banks are still an untouched area of research. against this backdrop, the present study aims to identify and rank the job competencies that help improve customer service performance of cooperative bank employees using the ahp and has enormous relevance. 3. research methodology 3.1 design multiple methods were used for the study. the model was developed using a literature review, direct observation, a semi-structured interview, the checklist method, a questionnaire survey and the delphi technique. the ahp method was also employed to understand the priority and relative weights of the competencies. 3.2 sample the first level data regarding employee competencies that are required for superior customer service was collected from fifty employees in different job positions including managers, accountants and clerks at the headquarters of a prominent district cooperative bank in kerala, south india. the expert opinions and variable confirmations were collected from three academic experts and two professional experts. ahp data was collected separately from three different groups of employees which included ten managers, ten accountants and five clerical staff working in the ijahp article: v, radha ramanan/identification and ranking of competencies that positively influence customer service: an indian case study international journal of the analytic hierarchy process 32 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.637 fifty-three branches of the same district cooperative bank in kerala, south india to understand the differences in their priorities. 3.3 variable identification: procedure and measures 3.3.1 stage i: direct observation, interview, job description and specification, literature review, and checklist method the variable identification was conducted in three stages. initially, the activities of the employees who have direct customer contact in the headquarters of one of the district cooperative banks were observed directly and the competency requirements were noted. a semi-structured interview was conducted with fifty employees and they were asked to list the competencies they thought were needed to better serve customers. qualitative data analysis techniques such as transcribing, reading, categorizing, and extracting themes were used with the interview data. the job description and specification of the managers, accountants and clerks were reviewd and the common competencies that were required for the role were determined. based on the observation, interview and job description, a long list of matching job competencies was identified from the mcber competency dictionary (spencer & spencer, 1993) and the literature (boyatzis & ratti, 2009; sharma, 2012). the list of competencies was abridged by eliminating the competencies that had similar meanings, and this helped avoid duplication. then, a checklist of 21 competencies was prepared and provided to the fifty employees of the bank. they were asked to identify the competencies which they thought were indispensable in the provision of quality customer service, and also asked to suggest any important competencies that were not included in the checklist. based on the feedback from the employees, four more competencies related to the technical aspect of the jobs such as computer proficiency, accounting knowledge, inspection skills, and use of technology were added to the checklist. the identified competencies and their sources are given in table 1. ijahp article: v, radha ramanan/identification and ranking of competencies that positively influence customer service: an indian case study international journal of the analytic hierarchy process 33 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.637 table 1 details of variables identified and their origin competencies source negotiation spencer and spencer (1993) problem solving spencer and spencer (1993) decision making spencer and spencer (1993) influence spencer and spencer (1993) change catalyst spencer and spencer (1993) communication spencer and spencer (1993) efficiency orientation boyatzis and ratti (2009) planning boyatzis and ratti (2009) empathy boyatzis and ratti (2009) social objectivity boyatzis and ratti (2009) use of concepts boyatzis and ratti (2009) emotional self-control spencer and spencer (1993) self confidence spencer and spencer (1993) assertiveness spencer and spencer (1993) initiative spencer and spencer (1993) attention to detail spencer and spencer (1993) adaptability sharma (2012) team work and collaboration sharma (2012) committed to responsibilities sharma (2012) conflict management sharma (2012) trustworthiness sharma (2012) computer proficiency suggested by employees and experts accounting knowledge suggested by employees and experts inspection skill suggested by employees and experts use of technology suggested by employees and experts 3.3.2 stage ii: questionnaire survey after identifying the important competencies that aid in customer service from the employees, a five-point likert-type scale was prepared with response values ranging from 5 (very important) to 1 (not important) and used to measure the extent to which the competencies are important in better serving the customer (see questionnaire in appendix i). the questionnaire was distributed to the same fifty employees of the headquarters who had answered the checklist. all of the competencies scored more than 3.5, and had less than 20% in the coefficient of variation and were therefore in consensus with the literature (wang et al., 2016; williams & webb, 1994). the profile of the employees is given in table 2. ijahp article: v, radha ramanan/identification and ranking of competencies that positively influence customer service: an indian case study international journal of the analytic hierarchy process 34 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.637 table 2 profile of the fifty employees summary designation manager accountant clerk 14 17 19 gender male employees 27 female employees 23 age group male employees female employees 28 to 50 years 29 to 52 years education qualification graduated employees 31 post graduated employees 19 work experience 6 months to 30 years source: primary data 3.3.3 stage iii: delphi method the delphi method was used in the next stage of the study. the delphi method is interactive and consists of a panel of independent experts, which allows a consensus to be reached with the expert’s opinions on a specified topic following the questionnaire and feedback. in the delphi method, experts are allowed to keep their anonymity because it is not necessary for them to interact with each other (okoli & pawlowski, 2004). the delphi method is a useful tool when the knowledge about a topic is incomplete (skulmoski, hartman & krahn, 2007). this method is very prevalent for forecasting, prioritizing, decision making etc. linstone and turoff (2002) defined the delphi method as a “method for structuring a group communication process so that the process is effective in allowing a group of individuals, as a whole, to deal with a complex problem.” there are four phases involved in the delphi process (hsu & sandford, 2007). in the first phase, the researcher or investigator prepares an open-ended questionnaire and distributes it to the experts. the open-ended questionnaire is about the content area from the delphi subject. after receiving the completed responses from the expert, a researcher prepares a survey questionnaire based on the responses of experts. in the second phase, a survey questionnaire is distributed to the experts who are asked to rate the items or rank the order of priority of the items as per the delphi subject. the agreement or disagreement of the experts on the topic is identified in round two. the experts are also asked to state the reason for their answers in this phase. in the third round, the researcher gives a questionnaire to each expert. the questionnaire includes the items and ratings assigned by the experts in the previous stage and they are asked to specify the reason for the dissimilarity in the consensus. the fourth round begins after further clarification from the experts and achieving consensus. the items are distributed to the experts and they are given a final chance to review their judgements. the number of rounds in the delphi method depends on the degree of consensus achieved. the delphi method was used in this study for two purposes. the first was to confirm the importance rating given by the fifty employees in the second stage of variable identification concerning the importance of each competency in providing better ijahp article: v, radha ramanan/identification and ranking of competencies that positively influence customer service: an indian case study international journal of the analytic hierarchy process 35 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.637 customer service. for that purpose, the interview results, checklist, questionnaire and the results were cross-checked with three academic experts whose specializations were in the areas of human resource management, marketing management, and operations management, and two professional experts including the human resource manager and general manager of the bank. the profile of the experts is given in table 3. a questionnaire with the same five-point likert-type scale (see in appendix i) was given to the five experts and they were asked to rate the importance of each competency. in this second round of the questionnaire survey, the result was almost the same and they reached a consensus. the second purpose of the delphi method was to classify the identified job competencies into criteria and to develop the ahp model. the twenty-five competencies were classified into five criteria based on suggestions from the five experts. the five criteria of the job competencies are the relationship management competency, the cognitive competency, the self-management competency, the teambuilding competency, and the technical competency. the relationship management competency has six sub-criteria that include negotiation, problemsolving, decision making, influence, change catalyst, and communication. the cognitive competency has five sub-criteria, namely, efficiency orientation, planning, empathy, social objectivity, and the use of concepts. the self-management competency has six sub-criteria that include emotional self-control, self-confidence, assertiveness, initiative, attention to detail and adaptability. the teambuilding competency has four sub-criteria, namely, teamwork and collaboration, commitment to responsibilities, conflict management, and trustworthiness. the technical competency has four sub-criteria that include computer proficiency, accounting knowledge, inspection skill, and use of technology. the definition of each competency is given in the questionnaire in appendix i. an ahp model was developed based on the opinions of the experts. the conceptual framework of the study is given in figure 1. ijahp article: v, radha ramanan/identification and ranking of competencies that positively influence customer service: an indian case study international journal of the analytic hierarchy process 36 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.637 table 3 profile of the experts summary professional position (n) human resource manager of bank 1 general manager of bank 1 academician – hrm 1 academician – marketing management 1 academician – operations management 1 average working experience (year) more than 10 years average age 42 education level (n) master's degree 5 doctoral degree 3 source: primary data figure 1 conceptual framework of the study 3.4 pairwise comparison questionnaire and data collection after receiving approval from the experts, a new questionnaire was prepared to understand the rank or priority of each competency needed for better customer service from the employees. a scale of 1-9 as proposed by saaty (2008) measured the perception of how many times more important or dominant one competency is over another competency to serve customers well. table 4 shows the details of the pairwise comparison scale used for the study. the data collection began with the distribution of the pairwise comparison questionnaire to three different groups of employees that included ten managers, ten accountants and five clerks in the fifty-three branches of the district cooperative bank competencies improve the customer service performance relationship management competency negotiation problem solving decision making influence change catalyst communication cognitive competency efficiency orientation planning empathy social objectivity use of concepts self-management competency emotional selfcontrol self-confidence assertiveness initiative attention to detail adaptability teambulding competency teamwork & collaboration committed to responsibilities conflict management trustworthiness technical competency computer proficiency accounting knowledge inspection skills use of technology ijahp article: v, radha ramanan/identification and ranking of competencies that positively influence customer service: an indian case study international journal of the analytic hierarchy process 37 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.637 in the state of kerala, south india. the study adopted a convenience sampling method for data collection. based on the responses from the separate samples, a pairwise comparison of each competency was made.the questionnaire is given in appendix ii. table 4 fundamental scale for pairwise comparison or ranking intensity of importance of scale definition explanation 1 equally important two measures contribute equally to the problem 3 moderately important experience and judgement slightly favor one measure over another 5 strongly important experience and judgement strongly favor one measure over another 7 very strongly important a measure is favored very strongly over another; its dominance demonstrated in practice 9 extremely important the evidence favoring one measure over another is of the highest order 2, 4, 6, 8 intermediate values intermediate values between two adjacent judgements source: saaty (2008) 3.5 group decision aggregation the data was collected separately from three different groups of employees that included ten managers, ten accountants and five clerks, and different judgements were obtained. in order to aggregate the individual judgements into a single representative judgement for the three separate groups, the geometric mean method (gmm) was used. the gmm is considered the best method when compared with the arithmetic mean method (aczel & saaty,1983; saaty & kearns, 1985; saaty, 2008; willet & sharda, 1991; benjamin et al., 1992). krejci and stoklasa (2018) revealed the advantage of the weighted geometric mean aggregation over the weighted arithmetic mean aggregation in ahp to develop global priorities of alternatives. most of the other newest prominent studies employed the gmm for aggregation of individual judgements (anish, dhanish & sridharan, 2017; de luca, 2014; srdjevic, lakicevic & srdjevic, 2013). ramanathan and ganesh (1994) found that pareto optimality is not satisfied with the gmm and suggested the eigen vector-based method for group aggregation. similarly, amenta et al. (2019) advised the use of the common priority vector procedure for group aggregation instead of the gmm. on the other hand, van den honert and lootsma (1997) argued that violations that result from the gmm are typical in pairwise calculations and that the results can be compromised. nevertheless, the present study adopted the gmm for group aggregation for appropriateness. microsoft excel was used to compute the geometric mean of the judgements. ijahp article: v, radha ramanan/identification and ranking of competencies that positively influence customer service: an indian case study international journal of the analytic hierarchy process 38 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.637 the geometric mean is defined as “the n th root of the product of n numbers.” for a set of numbers {xi}ni=1, the geometric mean is: n n n n i aaaxi ..... 21 /1 1         (1) where, n represents number of items a represents individual judgments 3.6 ahp method the analytic hierarchy process (ahp), developed by thomas l. saaty (1980) is one of the tools for multi-criteria decision-making based on the pairwise comparison of factors using a nine-point scale. complex multi-criteria problems can be solved using the ahp. the ahp involves developing a graphical representation of the problem in a hierarchical structure in terms of goal, main criteria, sub-criteria and decision alternatives. the judgments of experts or decision-makers about the relative importance or preference for each criterion are collected. the output of the ahp provides the priorities or ranks of the criterion based on the overall preferences expressed by the expert or decision-maker. the phases of the ahp method are as follows: 1. development of the hierarchical structure the first step of the ahp is to develop a graphical representation of the problem in terms of goal, main-criteria, sub-criteria, and the decision alternatives in the hierarchical structure. figure 1 shows the conceptual framework of the study as discussed above with the delphi method showing the hierarchical structure of the problem of the present study. 2. designing the pairwise comparison scale the second phase begins with the preparation of the pairwise comparison questionnaire with a nine-point scale developed by saaty (1980). the scale measures the perception of how many times more important or dominant one criteria is over another. table 4 given above in section 3.4 shows the fundamental scale for pairwise comparison. the experts or decision-makers are required to express their priority or preference for the criteria. 3. construction of the pairwise comparison matrix the third step is concerned with the construction of a matrix of the pairwise comparison ratings. the pairwise comparison matrix will consist of rows and columns as below. a[i,j]               ... ... ............ ... 21 22221 1121 nnnn n n aaa aaa aaa the matrix indicates the relative importance of the i th element over the j th element (saaty, 1980). ijahp article: v, radha ramanan/identification and ranking of competencies that positively influence customer service: an indian case study international journal of the analytic hierarchy process 39 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.637 1. synthesization this phase includes calculating the priority of each criterion in terms of its overall goal, which is referred to as synthesization. the following steps are required. i. calculate the sum of each column of the pairwise comparison matrix. ii. divide each element in the pairwise comparison matrix by its column total and the resulting matrix is called the normalized pairwise comparison matrix. iii. calculate the average of the elements in each row of the normalized pairwise comparison matrix and these averages are the priorities for the criteria. 2. consistency the present phase provides a measure of consistency for the pairwise comparisons by calculating a consistency ratio. if the ratio is 0.10 or less, the inconsistency of the pairwise comparisons is considered rational. if the value is higher than the acceptable value of 0.10, this indicates an inconsistency in the judgment and the decision-maker needs to revise the pairwise comparisons. in order to understand the consistency of the matrices, the sum of the normalized values and the weighted sum must be found. to do this, one must multiply each value in the first column of the pairwise comparison matrix by the priority of the first item; then, multiply each value in the second column of the pairwise comparison matrix by the priority of the second item; this process is continued for all columns of the pairwise comparison matrix. the weighted sum can be found by dividing each sum value by its normalized average weight. then, find the average value of weighted sum (aw), and this average is denoted as λmax. aw = λmaxw λmax is the largest eigen value of matrix a w is the related eigen vector or vector of weight of the comparison matrix. consistency index, ci= λmax − n n −1 where n is the number of items being compared. inconsistency ratio, cr = consistency index value (ci) random imdex number (ri) the random index (ri) is the consistency index of a randomly generated pairwise comparison matrix (alonso & lamata, 2006). the value of ri depends on the number of items being compared and the random indices employed by saaty (1977) is given as follows: number of factors, n 1 2 3 4 5 6 7 8 9 10 random index, ri 0.00 0.00 0.58 0.90 1.12 1.24 1.32 1.41 1.45 1.49 source: saaty(1977) ijahp article: v, radha ramanan/identification and ranking of competencies that positively influence customer service: an indian case study international journal of the analytic hierarchy process 40 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.637 3.6.1. calculation of weights or priority using the ahp method the analytic hierarchy process (ahp) is applied to rank the job competencies that improve the customer service performance of employees. the study used microsoft excel instead of expert choice for the ahp calculations because of convenience. a) ranking the job competencies (main criteria) there are five main criteria related to job competencies that help determine which competency is most important in customer service. the rows and columns 1, 2, 3, 4, 5 of the pairwise comparison matrix denote the options; relationship management competency (rmc), cognitive competency (cc), self-management competency (smc), teambuilding competency (tbc) and technical competency (tc), respectively. the values obtained from the managersare given below. pairwise comparison matrix for managersx1 𝑅𝑀𝐶 𝐶𝐶 𝑆𝑀𝐶 𝑇𝐵𝐶 𝑇𝐶 x1 = 𝑅𝑀𝐶 𝐶𝐶 𝑆𝑀𝐶 𝑇𝐵𝐶 𝑇𝐶 𝑪𝒐𝒍𝒖𝒎𝒏 𝒔𝒖𝒎 [ 1 1/2 2 2 2 2 1 1 1 1 1/2 1 1 2 1 1/2 1 1/2 1 2 1/2 1 2 1/2 1 𝟗/𝟐 𝟗/𝟐 𝟏𝟏/𝟐 𝟏𝟑/𝟐 𝟕] the normalized matrix (n1) that corresponds to the pairwise comparison matrix x1 is given below. 𝑅𝑀𝐶 𝐶𝐶 𝑆𝑀𝐶 𝑇𝐵𝐶 𝑇𝐶 n1 = 𝑅𝑀𝐶 𝐶𝐶 𝑆𝑀𝐶 𝑇𝐵𝐶 𝑇𝐶 [ 2/9 1/9 1/3 1/3 2/7 4/9 2/9 1/5 1/6 1/7 1/9 2/9 1/5 1/3 1/7 1/9 2/9 0 1/6 2/7 1/9 2/9 1/5 0 1/7] the normalized average weights (priority) are calculated as follows: w1 / n = (2/9 + 1/9 + 1/3 + 1/4 + 1/3) / 5 = 0.26 w2 / n = (4/9 + 2/9 + 1/5 + 1/8 + 1/5) / 5 = 0.23 w3 / n = (1/9 + 2/9 + 1/5 + 1/4 + 1/5) / 5 = 0.19 w4 / n = (1/9 + 2/9 + 0 + 1/6 + 2/7) / 5 = 0.17 w5 / n = (1/9 + 2/9 + 1/5 + 0 + 1/7) / 5 = 0.15 the weighted average matrix 𝑊1, 𝑊1 = ( 0.26 0.23 0.19 0.17 0.15) sum can be found by ijahp article: v, radha ramanan/identification and ranking of competencies that positively influence customer service: an indian case study international journal of the analytic hierarchy process 41 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.637 sw1= 0.26 [ 1 2 1/2 1/2 1/2] + 0.23 [ 1/2 1 1 1 1 ] + 0.19 [ 2 1 1 1/2 2 ] + 0.17 [ 2 1 2 1 1/2] + 0.15 [ 2 1 1 2 1] = [ 1.40 1.26 1.04 0.92 0.78] the weighted sum can be found by s𝑊1, 1.40 0.26 = 5.4 1.26 0.23 = 5.5 1.04 0.19 = 5.4 0.92 0.17 = 5.3 0.78 0.15 = 5.3 λmax = (5.4 + 5.5+ 5.4 + 5.3 + 5.3) / 5 = 5.4 ci= λmax – n n −1 = 5.4 – 5 5 −1 = 0.10 inconsistency ratio, cr = ci ri = 0.10 1.12 = 0.089 because the inconsistency ratio is less than the threshold value of 0.1, the pairwise comparison matrix is considered to be consistent (saaty, 1980). b) ranking the relationship management competencies there are six options for the relationship management competency, namely, negotiation (n), problem-solving (ps), decision making (dm), influence (in), change catalyst (cc), and communication (c). the calculations are given below. pairwise comparison matrix for managers x2 𝑁 𝑃𝑆 𝐷𝑀 𝐼𝑁 cc c x2 = 𝑁 𝑃𝑆 𝐷𝑀 𝐼𝑁 𝐶𝐶 𝐶 𝑪𝒐𝒍𝒖𝒎𝒏 𝒔𝒖𝒎 [ 1 1/3 1/4 1 1 1/3 3 1 2 2 2 2 4 1/2 1 1 1 2 1 1/2 1 1 1 1 1 1/2 1 1 1 1 3 1/2 1/2 1 1 1 𝟏𝟑 𝟏𝟎/𝟑 𝟐𝟑/𝟒 𝟕 𝟕 𝟐𝟐/𝟑] ijahp article: v, radha ramanan/identification and ranking of competencies that positively influence customer service: an indian case study international journal of the analytic hierarchy process 42 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.637 the normalized matrix (n2) that corresponds to the pairwise comparison matrix x2is given below. 𝑁 𝑃𝑆 𝐷𝑀 𝐼𝑁 cc c n2 = 𝑁 𝑃𝑆 𝐷𝑀 𝐼𝑁 𝐶𝐶 𝐶 [ 0 0 0 1/7 1/7 0 1/4 1/3 1/3 2/7 2/7 1/4 1/3 1/7 1/6 1/7 1/7 1/4 0 1/7 1/6 1/7 1/7 1/7 0 1/7 1/6 1/7 1/7 1/7 1/4 1/7 0 1/7 1/7 1/7] table 5 priority calculations of matrix two factors average weight sum weighted sum λmax= 6.28 ci = 0.06 cr = 0.05 n 0.09 0.56 6.10 ps 0.29 1.80 6.29 dm 0.20 1.28 6.46 in 0.14 0.86 6.24 cc 0.14 0.86 6.24 c 0.15 0.94 6.35 source: primary data the cr value for the groups of managers is 0.05, which is less than 0.1, and therefore matrix two is considered to be consistent. the managers suggested that problemsolving skills are the most important for serving customers better when compared with the other five relationship management competencies. c) ranking the cognitive competencies the options for the cognitive competencies are efficiency orientation (eo), planning (p), empathy (e), social objectivity (so), and use of concepts (uc). the calculations are given below. pairwise comparison matrix for managers x3 𝐸𝑂 𝑃 𝐸 𝑆𝑂 uc x3 = 𝐸𝑂 𝑃 𝐸 𝑆𝑂 𝑈𝐶 𝑪𝒐𝒍𝒖𝒎𝒏 𝒔𝒖𝒎 [ 1 1 1/2 1 2 1 1 1/2 1/2 1 2 2 1 2 2 1 2 1/2 1 1 1/2 1 1/2 1 1 𝟏𝟏/𝟐 𝟕 𝟑 𝟏𝟏/𝟐 𝟕] the normalized matrix (n3) that corresponds to the pairwise comparison matrix x3 is given below. 𝐸𝑂 𝑃 𝐸 𝑆𝑂 uc n3 = 𝐸𝑂 𝑃 𝐸 𝑆𝑂 𝑈𝐶 [ 1/5 1/7 1/6 1/5 2/7 1/5 1/7 1/6 0 1/7 1/3 2/7 1/3 1/3 2/7 1/5 2/7 1/6 1/5 1/7 0 1/7 1/6 1/5 1/7] ijahp article: v, radha ramanan/identification and ranking of competencies that positively influence customer service: an indian case study international journal of the analytic hierarchy process 43 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.637 table 6 priority calculations of matrix three factors average weight sum weighted sum λmax = 5.12 ci = 0.03 cr = 0.03 eo 0.19 0.98 5.12 p 0.15 0.74 5.11 e 0.33 1.67 5.13 so 0.19 0.98 5.12 uc 0.15 0.74 5.11 source: primary data the consistency of matrix three is confirmed since the cr value is 0.03. the option ‘empathy’ is determined to be the most important competency for customer service. d) ranking the self-management competency the self-management competency has six sub-criteria including emotional self-control (emc), self-confidence (sc), assertiveness (a), initiative (i), attention to detail (ad), and adaptability (adp). pairwise comparison matrix for managers x4 𝐸𝑀𝐶 𝑆𝐶 𝐴 𝐼 ad adp x4 = 𝐸𝑀𝐶 𝑆𝐶 𝐴 𝐼 𝐴𝐷 𝐴𝐷𝑃 𝑪𝒐𝒍𝒖𝒎𝒏 𝒔𝒖𝒎 [ 1 2 3 1/2 1 1 1/2 1 1 1/2 1 1 1/3 1 1 1/2 1/3 1/4 2 2 2 1 3 1 1 1 3 1/3 1 1 1 1 4 1 1 1 𝟑𝟓/𝟔 𝟖 𝟏𝟒 𝟐𝟑/𝟔 𝟐𝟐/𝟑 𝟐𝟏/𝟒] the normalized matrix (n4) that corresponds to the pairwise comparison matrix x4 is given below. 𝐸𝑀𝐶 𝑆𝐶 𝐴 𝐼 ad adp n4 = 𝐸𝑀𝐶 𝑆𝐶 𝐴 𝐼 𝐴𝐷 𝐴𝐷𝑃 [ 1/6 1/4 1/5 1/8 1/7 1/5 0 1/8 0 1/8 1/7 1/5 0 1/8 0 1/8 0 0 1/3 1/4 1/7 1/4 2/5 1/5 1/6 1/8 1/5 0 1/7 1/5 1/6 1/8 2/7 1/4 1/7 1/5] table 7 priority calculations of matrix four factors average weight sum weighted sum λmax = 6.3 ci = 0.06 cr = 0.05 emc 0.18 1.15 6.31 sc 0.12 0.78 6.30 a 0.08 0.50 6.25 i 0.27 1.69 6.36 ad 0.15 0.98 6.37 adp 0.19 1.24 6.35 source: primary data the cr value (0.05) of matrix four shows its consistency. the ‘initiative’ compentency is found to be the most important for serving customers better. ijahp article: v, radha ramanan/identification and ranking of competencies that positively influence customer service: an indian case study international journal of the analytic hierarchy process 44 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.637 e) ranking the teambuilding competencies the priority options for the teambuilding competency are teamwork and collaboration (tc), committed to responsibilities (cr), conflict management (cm), and trustworthiness (tr). pairwise comparison matrix for managers x5 𝑇𝐶 cr cm tr x5 = 𝑇𝐶 𝐶𝑅 𝐶𝑀 𝑇𝑅 𝐶𝑜𝑙𝑢𝑚𝑛 𝑠𝑢𝑚 [ 1 1 1 1/3 1 1 2 1/3 1 1/2 1 1/4 3 3 4 1 𝟔 𝟏𝟏/𝟐 𝟖 𝟐 ] the normalized matrix (n5) that corresponds to the pairwise comparison matrix x5 is given below. 𝑇𝐶 cr cm tr n5 = 𝑇𝐶 𝐶𝑅 𝐶𝑀 𝑇𝑅 [ 1/6 1/5 1/8 1/6 1/6 1/5 1/4 1/6 1/6 0 1/8 1/8 1/2 1/2 1/2 1/2 ] table 8 priority calculations of matrix five factors average weight sum weighted sum λmax = 4.1 ci = 0.02 cr = 0.02 tc 0.16 0.66 4.05 cr 0.19 0.78 4.06 cm 0.13 0.52 4.02 tr 0.52 2.09 4.05 source: primary data matrix five is considered to be consistent with a cr value of 0.02. ‘trustworthiness’ is the most important factor for serving customers better when compared with the other three teambuilding competencies. f) ranking the technical competencies the technical competency consists of computer proficiency (cp), accounting knowledge (ak), inspection skills (is), and use of technology (ut). pairwise comparison matrix for managers x6 𝐶𝑃 ak is ut x6 = 𝐶𝑃 𝐴𝐾 𝐼𝑆 𝑈𝑇 𝐶𝑜𝑙𝑢𝑚𝑛 𝑠𝑢𝑚 [ 1 2 1 2 1/2 1 1 2 1 1 1 2 1/2 1/2 1/2 1 𝟑 𝟗/𝟐 𝟕/𝟐 𝟕] the normalized matrix (n5) that corresponds to the pairwise comparison matrix x5 is given below. ijahp article: v, radha ramanan/identification and ranking of competencies that positively influence customer service: an indian case study international journal of the analytic hierarchy process 45 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.637 𝐶𝑃 ak is ut n6 = 𝐶𝑃 𝐴𝐾 𝐼𝑆 𝑈𝑇 [ 1/3 4/9 2/7 2/7 1/6 2/9 2/7 2/7 1/3 2/9 2/7 2/7 1/6 1/9 1/7 1/7 ] table 9 priority calculations of matrix six factors average weight sum weighted sum λmax = 4.1 ci = 0.02 cr = 0.02 cp 0.34 1.38 4.09 ak 0.24 0.97 4.05 is 0.28 1.14 4.05 ut 0.14 0.57 4.05 source: primary data the results confirm the consistency of matrix six because the cr value is 0.02, which is less than 0.1. among the other three options ‘computer proficiency’ is ranked first. using the above calculations, an ahp analysis was performed on the main criteria and each sub-criterion for the managers, accountants and clerks. the results are given in the table 10. ijahp article: v, radha ramanan/identification and ranking of competencies that positively influence customer service: an indian case study international journal of the analytic hierarchy process 46 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.637 table 10 weights of different job competencies competency weight/ priority manager accountant clerks relationship management competency 0.26 0.31 0.36 cognitive competency 0.23 0.18 0.15 self-management competency 0.19 0.25 0.19 teambuilding competency 0.17 0.12 0.15 technical competency 0.15 0.14 0.15 λmax 5.4 5.15 5.15 ci 0.10 0.04 0.04 cr 0.089 0.03 0.03 relationship management competency 26% 31% 36% negotiation 0.09 0.13 0.13 problem solving 0.29 0.23 0.26 decision making 0.20 0.16 0.18 influence 0.14 0.17 0.13 change catalyst 0.14 0.14 0.14 communication 0.15 0.17 0.16 λmax 6.28 6.30 6.22 ci 0.06 0.06 0.04 cr 0.05 0.05 0.04 cognitive competency 23% 18% 15% efficiency orientation 0.19 0.21 0.20 planning 0.15 0.14 0.13 empathy 0.33 0.45 0.41 social objectivity 0.19 0.11 0.16 use of concepts 0.15 0.10 0.11 λmax 5.12 5.20 5.36 ci 0.03 0.05 0.09 cr 0.03 0.04 0.08 self-management competency 19% 25% 19% emotional self-control 0.18 0.20 0.16 self confidence 0.12 0.13 0.15 assertiveness 0.08 0.11 0.13 initiative 0.27 0.25 0.23 attention to detail 0.15 0.13 0.15 adaptability 0.19 0.18 0.18 λmax 6.32 6.14 6.29 ci 0.06 0.03 0.06 cr 0.05 0.02 0.05 teambuilding competency 17% 12% 15% team work and collaboration 0.16 0.19 0.19 committed to responsibilities 0.19 0.23 0.27 conflict management 0.13 0.15 0.12 trustworthiness 0.52 0.43 0.42 λmax 4.05 4.05 4.08 ci 0.02 0.02 0.03 cr 0.02 0.02 0.03 technical competency 15% 14% 15% computer proficiency 0.34 0.35 0.30 accounting knowledge 0.24 0.20 0.21 inspection skill 0.28 0.20 0.25 ijahp article: v, radha ramanan/identification and ranking of competencies that positively influence customer service: an indian case study international journal of the analytic hierarchy process 47 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.637 use of technology 0.14 0.25 0.25 λmax 4.06 4.06 4.06 ci 0.02 0.02 0.02 cr 0.02 0.02 0.02 source: primary data 4. results and discussion after considering the priorities of the three groups of employees namely, managers, accountants, and clerical staff separately, the results of the ahp indicate that of the five main criteria, relationship management competency was the most critical criteria for better customer service. the inconsistency ratio was less than 0.1, and the pairwise comparison matrixes for the three groups are considered to be consistent. the option ‘relationship management competency’ has the highest weight in the three groups of samples when compared with the other four competencies, and this indicates that it has the highest priority for serving customers better. the technical and teambuilding skill was determined to be a less critical skill for customer service by the three sample groups. table 10 clearly shows the competency requirements for employees. figure 2 shows the graphical representation of the competency requirements of the three categories of employees. the managers need cognitive competency (23%) more than accountants (18%) and clerks (15%) for better customer service. at the same time, accountants (31%) and clerical staff (36%) require more relationship management competency to deliver superior customer service. in figure 2, the amount of relationship management competency that is required for managers is less, and begins to increase for accountants and clerks. similarly, the cognitive competency is highly required for managers, and starts to diminish withaccountants and clerks. the identification of the most critical competency for the three categories of employees is a significant finding of the study. figure 2 competency requirements of employees the overall priorities of the competencies are the same across all of the groups, and some differences in the rankings arise only when considering the sub-criteria. the weight of all of the classifications of employees is assumed to be the same. this result is not surprising since the present study disregarded the role-based competency level requirement and considered customer service to be the responsibility of all of the employees who have direct contact with customers in a service sector like a bank, relationship management competency cognitive competency manager clerk accountant ijahp article: v, radha ramanan/identification and ranking of competencies that positively influence customer service: an indian case study international journal of the analytic hierarchy process 48 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.637 irrespective of their position (drucker, 1968). from the results, we can infer that the employees recognize the importance of establishing a relationship with the customers to ensure their satisfaction.the employees’ involvement in understanding the customer’s problems and helping the customers make decisions are the factors that have the most influence in customer service. employees believe that customers always expect good behavior from the employees, desire a good welcome, expect employees to show initiative in learning their needs and fulfilling them, and need empathetic and trustworthy staff with technical knowledge to hear and resolve their problems. customers can be satisfied and retained by maintaining a good relationship with them and providing quality customer service. the manager’s job demands more cognitive skills since it involves planning, use of concepts, social consideration, empathy, and efficiency, while the accountant and clerks have technical interactions with the clients. all three groups of employees suggested that problem-solving skills are the most critical skill in the relationship management competency cluster to serve customers better, and empathy received the highest weight in the cognitive competency criteria in the three sample groups. initiative has the top priority among the self-management competencies. trustworthiness has more weight in the teambuilding competency, and computer proficiency is the priority in the technical competency criteria. all of the three sample groups suggested that the technical competency is less critical for customer service. this indicates that the employees are not aware of the importance of technological advancements in customer service. even though the cooperative bank has the largest number of banks and customers, the bank’s technical aspect needs to improve. the number of atm counters, internet banking facilities, and mobile banking technologies can still develop. most of the customers of the bank are ordinary people and employees could meet the demands of customers with limited facilities. nevertheless, the growing competition in the financial sector demands more sophisticated technologies to attract customers and a make transactions and the life of the customer hassle-free. cooperative banks need to adopt new technological advancements to be competitive and win customers. the fact that the technical competency ranked last highlights the importance of adopting technological advancements in the bank and the need to train the employees to improve their technical competencies including computer proficiency, computerized accounting knowledge, use of office automation technologies, and inspection skills. 5. conclusions and managerial implications the study identified the critical job competencies which improve customer service in the cooperative banking sector using the opinions of employees. managers and accountants ranked the relationship management competency as crucial to customer service, and the teambuilding competency as the least important. at the same time, the technical competencies ranked least for the clerical staff. even though the three groups of employees priorities were almost identical, the difference in their priority percentage was different. problem-solving skills, empathy, initiative and trustworthiness were found to be the most important for providing superior service to customers. these findings have significant implications for the management of the bank for different policy decisions. this could help the management of the bank modify the recruitment policy and seek to hire people with those specific skills. these competencies can also be used as performance criteria in the performance appraisal ijahp article: v, radha ramanan/identification and ranking of competencies that positively influence customer service: an indian case study international journal of the analytic hierarchy process 49 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.637 process. the organization may also plan and design various trainings and development programs that focus on those competencies for employees. the ranking of technical skills as the lowest by the employees in this technologically advanced era indicates the need for sophisticated technologies in banking activities and indicates the importance of making employees aware of techniques and their applications. training and development programs could be planned and designed to improve the technical skills of the employees. ijahp article: v, radha ramanan/identification and ranking of competencies that positively influence customer service: an indian case study international journal of the analytic hierarchy process 50 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.637 references aczel, j. and saaty, t.l. (1983). procedures for synthesizing ratio judgements. journal of mathematical psychology, 27, 93-102. doi: https://doi.org/10.1016/0022-2496(83)90028-7 alferaih, a. (2017). conceptual model for measuring saudi banking managers’ job performance based on their emotional intelligence (ei). international journal of organizational analysis, 25(1), 123-145. doi: https://doi.org/10.1108/ijoa-102014-0807 alonso, j. a., and lamata, m. t. (2006). consistency in the analytic hierarchy process: a new approach. international journal of uncertainty, fuzziness and knowledge-based systems, 14(04), 445-459. doi: https://doi.org/10.1142/s0218488506004114 amenta, p., ishizaka, a., lucadamo, a., marcarelli, g., & vyas, v. (2019). computing a common preference vector in a complex multi-actor and multigroup decision system in analytic hierarchy process context. annals of operations research, 1-30.doi: https://doi.org/10.1007/s10479-019-03258-3 anish, m. n., dhanish, p. b., & sridharan, r. (2017). swot-ahp analysis in medical tourism area of kerala. international journal of society systems science, 9(3), 256-276. doi: https://doi.org/10.1504/ijsss.2017.10008325 benjamin, c.o., ehie, i.c. and omurtag, y. (1992). planning facilities at the university of missouri-rolla. interfaces, 22/4, 95-105. doi: https://doi.org/10.1287/inte.22.4.95 chao, y. p., chou, y. c. & lai, w. h. (2017). ahp competency model in the service chain industry. advances in management and applied economics, 7(3), 1. charnes, a., cooper, w. w. & rhodes, e. (1978). measuring the efficiency of decision making units. european journal of operational research, 2, 429–444. doi: https://doi.org/10.1016/0377-2217(78)90138-8 de luca, s. (2014). public engagement in strategic transportation planning: an analytic hierarchy process based approach. transport policy, 33, 110–124. doi:https://doi.org/10.1016/j.tranpol.2014.03.002 demirtas, e. a., buruk, y. & sağır, m. (2015).a multi-criteria job evaluation method for a state bank. international journal of the analytic hierarchy process, 7(2), 188-200. doi: https://doi.org/10.13033/ijahp.v7i2.264 drucker, p. (1968). the practice of management. london, uk: pan books ltd. fang, c. h., chang, s. t. & chen, g. l. (2010). competency development among taiwanese healthcare middle manager: a test of the ahp approach. african journal of business management, 4(13), 2845-2855. fontela, e. & gabus, a. (1976). the dematel observer, dematel 1976 report. geneva, switzerland: battelle geneva research center. ijahp article: v, radha ramanan/identification and ranking of competencies that positively influence customer service: an indian case study international journal of the analytic hierarchy process 51 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.637 hafeez, k. & essmail, e. a. (2007). evaluating organisation core competences and associated personal competencies using analytical hierarchy process. management research news, 30(8), 530-547. doi: https://doi.org/10.1108/01409170710773689 hopkins, m. m. & bilimoria, d. (2008). social and emotional competencies predicting success for male and female executives. journal of management development, 27(1), 13-35. doi: https://doi.org/10.1108/02621710810840749 hsu, c. c., &sandford, b. a. (2007). the delphi technique: making sense of consensus. practical assessment,research & evaluation, 12(10), 1-8. hwang, c., & yoon, k. (1981). multiple attribute decision making: methods and applications, a state of the art survey. new york, ny: sprinnger-verlag. islam, r. & bin mohdrasad, s. (2006). employee performance evaluation by the ahp: a case study. asia pacific management review, 11(3), 163-176. jena, s. & kumar sahoo, c. (2014). improving managerial performance: a study on entrepreneurial and leadership competencies. industrial and commercial training, 46(3), 143-149. doi: https://doi.org/10.1108/ict-10-2013-0066 kotler, p. (2000). marketing management: the millennium edition. marketing management, 23(6), 188-193. krejčí, j. & stoklasa, j. (2018). aggregation in the analytic hierarchy process: why weighted geometric mean should be used instead of weighted arithmetic mean. expert systems with applications, 114, 97-106. doi: https://doi.org/10.1016/j.eswa.2018.06.060 kunnanatt, j. t. (2008). strategic question in indian banking sector: are indian bank managers achievement oriented? journal of management development, 27(2), 169-186. doi: https://doi.org/10.1108/02621710810849317 lakshminarayanan, s., pai, y. p. & ramaprasad, b. s. (2016). competency need assessment: a gap analytic approach. industrial and commercial training, 48(8), 423-430. doi: https://doi.org/10.1108/ict-04-2016-0025 linstone, h. & turoff, m. (2002). in h.a. linstone & m. turoff (eds.), the delphi method: techniques and applications. liu, s. n., lin, y. t., & chen, y. c. (2011). professional competencies for marketing managers in taiwan: an application of the analytic hierarchy process (ahp). world transactions on engineering and technology education, 9(04). maccrimmon, k. r., & rand, c. (1968). decision making among multipleattribute alternatives: a survey and consolidated approach. santa monica, ca: rand corp. mardani, a., jusoh, a., nor, k., khalifah, z., zakwan, n. &valipour, a. (2015). multiple criteria decision-making techniques and their applications–a review of the literature from 2000 to 2014. economic research-ekonomskaistraživanja, 28(1), 516-571. doi:https://doi.org/10.1080/1331677x.2015.1075139 ijahp article: v, radha ramanan/identification and ranking of competencies that positively influence customer service: an indian case study international journal of the analytic hierarchy process 52 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.637 mareschal, b., brans, j. p. &vincke, p. (1984). promethee: a new family of outranking methods in multicriteria analysis. ulb institutional repository, ulb–universite libre de bruxelles, brussels. mcclelland, d. c. (1973). testing for competence rather than for intelligence. american psychologist, 28(1), 1. nursikuwagus, a., melian, l. & permatasari, d. (2018).computational model of student competency analysis in fuzzy topsis method. in iop conference series: materials science and engineering,407(1),012095. doi: https://doi.org/10.1088/1757-899x/407/1/012095 okoli, c. & pawlowski, s. d. (2004). the delphi method as a research tool: anexample, design considerations and application. information and management,42, 15–29. doi: https://doi.org/10.1016/j.im.2003.11.002 opricovic, s. (1998). multicriteria optimization of civil engineering systems (in serbian). belgrade: faculty of civil engineering. ramanathan, r. & ganesh, l. s. (1994). group preference aggregation methods employed in ahp: an evaluation and an intrinsic process for deriving members' weightages. european journal of operational research, 79(2), 249-265. doi: https://doi.org/10.1016/0377-2217(94)90356-5 roy, b. (1971). problems and methods with multiple objective functions. mathematical programming, 1, 239–266. saaty, t. (1977). a scaling method for priorities in hierarchical structures. journal of mathematical psychology, 15, 234-281. doi: https://doi.org/10.1016/0022-2496(77)90033-5 saaty, t. (1980). the analytic hierarchy process. new york, ny: mcgraw-hill. saaty, t. l. (1996). decision making with dependence and feedback: the analytic network process: the organization and prioritization of complexity. pittsburgh: rws publications. saaty, t. l. (2008). decision making with the analytic hierarchy process. international journal of services sciences, 1(1), 83-98. saaty, t.l. & kearns, k.p. (1985). analytical planning: the organization of systems. oxford: pergamon press. sekhar, c., patwardhan, m. & vyas, v. (2015). a delphi-ahp-topsis based framework for the prioritization of intellectual capital indicators: a smes perspective. procedia-social and behavioral sciences, 189, 275-284. doi: https://doi.org/10.1016/j.sbspro.2015.03.223 sharma, r. (2012). measuring social and emotional intelligence competencies in the indian context. cross cultural management: an international journal, 19(1), 30-47. doi: https://doi.org/10.1108/13527601211195619 skulmoski, g., hartman, f. & krahn, j. (2007). the delphi method for graduate research. journal of information technology education research, 6(1), 1-21. ijahp article: v, radha ramanan/identification and ranking of competencies that positively influence customer service: an indian case study international journal of the analytic hierarchy process 53 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.637 spencer, l. m. & spencer, p. s. m. (2008). competence at work models for superior performance. john wiley & sons. srdjevic, z., lakicevic, m. & srdjevic, b. (2013). approach of decision making based on the analytic hierarchy process for urban landscape management. environmental management, 51(3), 777–785. doi: https://doi.org/10.1007/s00267-012-9990-7 tahriri, f., osman, m.r., ali, a. and yusuff, r.m. (2008). a review of supplier selection methods in manufacturing industries. suranaree journal of science and technology, 15( 3), 201-208. trivellas, p., akrivouli, z., tsifora, e. & tsoutsa, p. (2015).the impact of knowledge sharing culture on job satisfaction in accounting firms.the mediating effect of general competencies. procedia economics and finance, 19, 238-247. doi: https://doi.org/10.1016/s2212-5671(15)00025-8 van den honert, r. & lootsma, f. (1997). group preference aggregation in the multiplicative ahp: the model of the group decision process and pareto optimality. european journal of operational research, 96(2), 363-370. doi: https://doi.org/10.1016/0377-2217(95)00345-2 wang, y. m., xiong, l. j., ma, y., gao, x. l. & fu, w. f. (2016). construction of competency evaluation measures for operating room nurses. chinese nursing research, 3(4), 181-184.doi: https://doi.org/10.1016/j.cnre.2016.07.001 willet, k.& sharda, r. (1991). using the analytic hierarchy process in water resources planning: selection of flood control projects.socio economic planning sciences, 25/2, 103-112.doi: https://doi.org/10.1016/0038-0121(91)90008-f williams, p. l. & webb, c. (1994). the delphi technique: a methodological discussion. journal of advanced nursing, 19(1), 180-186. ijahp article: v, radha ramanan/identification and ranking of competencies that positively influence customer service: an indian case study international journal of the analytic hierarchy process 54 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.637 appendix i questionnaire for measuring the importance of competencies in customer service criteria definition very important important moderately important slight important not important 1. relationship management competencies ability to make a relationship with others. negotiation working with others to find a mutually agreeable outcome. problem solving working with others to identify, define and solve problems. decision making exploring and analyzing options to make sound decisions. influence ability to encourage subordinates to work toward an organizational goal. change catalyst ability to plan, bring and implement change initiatives. communication ability to convey information to individuals or groups to ensure that they understand the message. 2. cognitive competencies ability to think or analyze information and situations. efficiency orientation ability to do things faster with lower resources (with less time and less materials). planning ability to plan things properly and avoid uncertainties in the job and bank. empathy ability to positively think from others perspective while dealing with others. social objectivity always doing the best to help others and thinking about societal benefits and growth. use of concepts using concepts to understand things and doing the job ijahp article: v, radha ramanan/identification and ranking of competencies that positively influence customer service: an indian case study international journal of the analytic hierarchy process 55 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.637 correctly. 3. selfmanagement competencies ability to recognize, understand, and use emotional information about oneself. emotional selfcontrol deal with emotional stress and strain that arises as a consequence of work situations of authority, leadership, power, targets and deadlines. self confidence aware of one’s own beliefs, goals, values, feeling, behavior and the part they play in influencing one’s own actions. assertiveness able to state one’s opinion firmly and positively. initiative respond to the needs of urgent situations voluntarily. attention to detail explore the necessary details, so as to get the exact information needed. adaptability conform and adjust to changing circumstances and work environments. 4. teambuilding competencies ability to cooperate with others and team building. team work and collaboration enjoy working in groups and able to contribute to and learn from the group. committed to responsibilities exercise responsibilities with trust, sincerity and commitment. conflict management ability to manage and resolve disagreements constructively. trustworthiness gain trust and confidence by interacting with fairness, honesty and truthfulness. 5. technical competencies knowledge in all technical and ijahp article: v, radha ramanan/identification and ranking of competencies that positively influence customer service: an indian case study international journal of the analytic hierarchy process 56 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.637 procedural aspects such as knowledge of equipment and machinery, knowledge of concepts, ideas etc. required for the job. computer proficiency have excellent computer knowledge and be able to apply this knowledge to the work for the effective performance of the job. accounting knowledge ability to maintain and audit accounts and prepare reports on assets, liabilities etc. of a business and to analyzeits financial status and operating results. inspection skill ability to find defects, fraud, misrepresentation, irregularities of the job. use of technology ability to utilize equipment, office software (i.e. word, excel, power point, internet access) and to prepare correspondence, reports, forms, mailings etc. ijahp article: v, radha ramanan/identification and ranking of competencies that positively influence customer service: an indian case study international journal of the analytic hierarchy process 57 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.637 appendix ii pairwise comparison questionnaire for undertsanding the importance of competencies in customer service 1. how important is the criterion a in providing superior services to customers in comparison with criterion b? criterion a 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 criterion b relationship management competency cognitive competency relationship management competency self-management competency relationship management competency teambuilding competency relationship management competency technical competency cognitive competency self-management competency cognitive competency teambuilding competency cognitive competency technical competency selfmanagement competency teambuilding competency selfmanagement competency technical competency teambuilding technical competency negotiation problem solving negotiation decision making negotiation influential negotiation change catalyst negotiation communication problem solving decision making problem influential ijahp article: v, radha ramanan/identification and ranking of competencies that positively influence customer service: an indian case study international journal of the analytic hierarchy process 58 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.637 solving problem solving change catalyst problem solving communication decision making influential decision making change catalyst decision making communication influential change catalyst influential communication change catalyst communication efficiency orientation planning efficiency orientation empathy efficiency orientation social objectivity efficiency orientation use of concepts planning empathy planning social objectivity planning use of concepts empathy social objectivity empathy use of concepts social objectivity use of concepts emotional self-control self-confidence emotional self-control assertiveness emotional self-control initiative emotional self-control attention to detail emotional self-control adaptability self assertiveness ijahp article: v, radha ramanan/identification and ranking of competencies that positively influence customer service: an indian case study international journal of the analytic hierarchy process 59 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.637 confidence selfconfidence initiative selfconfidence attention to detail selfconfidence adaptability assertiveness initiative assertiveness attention to detail assertiveness adaptability initiative attention to detail initiative adaptability attention to detail adaptability teamwork and collaboration committed to responsibilities teamwork and collaboration conflict management teamwork and collaboration honesty committed to responsibilitie s conflict management committed to responsibilitie s honesty conflict management honesty computer proficiency accounting knowledge computer proficiency inspection skill computer proficiency use of technology accounting knowledge inspection skill accounting knowledge use of technology inspection skill use of technology ijahp article: ohoitimur, krejci, raco, raton, jamlean, welerubun, tanod/strategic management study based on buckley’s fuzzy-ahp and swot: example of the congregation of the missionaries of the sacred heart international journal of the analytic hierarchy process 329 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.871 strategic management study based on buckely’s fuzzy-ahp and swot: example of the congregation of the missionaries of the sacred heart johanis ohoitimur johoitimur@unikadelasalle.ac.ied james krejci krijcija@lewisu.edu jozef richard raco* 1 jraco@unikadelasalle.ac.ied yulius raton yraton@unikadelasalle.ac.id anselmus jamlean anseljamleanmsc@yahoo.id ignasius welerubun igomsc@yahoo.com rafael h.m tanod revimario@yahoo.com abstract strategic priorities are commonly implemented by both profit and non-profit institutions, but rarely employed by religious congregations. the congregation of the missionaries of the sacred heart (msc), an institution in the catholic church, has undertaken an empirical study, using a combination of swot and fuzzy-analytic hierarchy process (ahp) methodologies. four strategies were employed. first, the strategy s-o (30.1%), aimed at intensifying and improving our human resources in order to enrich the lives of all our members. second, we adopted the strategy s-t (25.8%) that focused on our 1 corresponding author acknowlegements: the authors wish to thank father samuel maranresy, provincial of msc indonesian province, fr diarmuid o’murchu of msc ireland province, father benny salomber of yayasan universitas katolik de la salle manado, anonimous reviewers and editor for their helpful comments and suggestions. mailto:johoitimur@unikadelasalle.ac.ied mailto:krijcija@lewisu.edu mailto:jraco@unikadelasalle.ac.ied mailto:yraton@unikadelasalle.ac.id mailto:anseljamleanmsc@yahoo.id mailto:igomsc@yahoo.com mailto:revimario@yahoo.com ijahp article: ohoitimur, krejci, raco, raton, jamlean, welerubun, tanod/strategic management study based on buckley’s fuzzy-ahp and swot: example of the congregation of the missionaries of the sacred heart international journal of the analytic hierarchy process 330 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.871 sacred heart spirituality to empower the fraternal community. third, we adopted the strategy w-t (24.1%) that aimed at strengthening the sacred heart spirituality throughout the entire msc family. finally, we used the strategy w-o (20.1%) in order to strengthen networking in the congregation while also revitalizing family ministry. a sensitivity analysis confirmed the so alternative as the main strategy. internally, the congregation is quite strong but more needs to be done to deepen the sacred heart spirituality among all our members. the researchers recommend conducting further study of the congregation about how they perceive the msc pastoral ministry and understand the vision. keywords: swotfuzzy ahp; msc; strategic management; catholic, decision making; religious 1. introduction the word ‘strategy’ or strategic planning, which is commonly used in management, organization and business, is defined as the art of creating specific business and management strategies, and implementation and evaluation of the achievements. the application of strategic planning and the achievement of goals always integrate the involvement of all units in the organization. the focus is on the results or outcomes, rather than products or outputs (candy & gordon, 2011). gyampoh and asare (2019) added that a strategy is a unified, comprehensive and integrated plan that relates the strategic advantages of the firm to the challenges of the environment. it is designed to ensure that the basic objectives of the enterprise are achieved through proper execution by the organization. the benefits of strategic planning are as follows: helps formulate the goals and objectives of organization, helps identify strategic issues and the main priorities of the organization, helps allocate the distribution of resources, assists human resource development, and assists leaders in making correct decisions. strategic planning will drive the efficiency and effectiveness of the work of an organization's management. thus, strategic planning is an indicator of effective management and at the same time the strength of organizational leadership. the church as an organization that serves the community must understand the context of its ministry, namely the local people. an important question that must be answered is what are the important issues for the missionaries of the sacred heart congregation (msc) now? this is where the importance of context and strategic management are seen. (oosthuizen & lategan, 2016). the msc as a religious organization within the catholic church cannot be separated from the implementation of strategic management. although msc is not a business institution, the application of management principles in the organization is important (gyampoh & asare, 2019). as an organization with universal values, msc needs a management system that provides a guide to assist in the execution of its mission according to the context of the times and culture. ijahp article: ohoitimur, krejci, raco, raton, jamlean, welerubun, tanod/strategic management study based on buckley’s fuzzy-ahp and swot: example of the congregation of the missionaries of the sacred heart international journal of the analytic hierarchy process 331 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.871 the msc, which was founded in 1854 in issoudun, france by jules chevalier, is an international religious congregation of priests and brothers, serving the catholic church in 48 countries. in indonesia, the congregation is growing. the religious vocation is increasing, training centers have been established, and services have expanded, not only in parishes but also abroad. support from the laity, especially the lay msc members, gives hope for the msc to move forward into a new future. the organizational structure and its administration are essentially no different from other congregations in the catholic church. elements such as leadership position and councils, vision and mission, and spiritual values as well as governance and norm systems are fundamental components of every religious congregation. the congregation also faces a number of challenges. extensive media coverage of the sexual abuse crisis in the catholic church has tarnished the image of the church and damaged the reputation of the priesthood. for young people, the pleasures, enjoyments, and allurements offered by the modern world have diminished a sense of calling to both the priesthood and religious life. internally, msc indonesia faces leadership and management challenges. leadership that relies on authority and position are no longer viewed as acceptable. efficient leadership that is in tune with the needs of the community while maintaining its core values is required. as a social and organizational entity, the msc is called to adopt modern methods of strategic planning to manage their assets and people, to contextualize their spirituality and mission, to introduce innovation in their pastoral services, and to strengthen their internal environment (miller, 2002). just as all organizations, the institution has stakeholders, intended and emergent strategies, structures, and processes. they are open systems depending on external resources (miller & kent, 2016). the msc must determine which strategic priorities the congregation should apply to achieve their vision and meet their goals. the importance of implementing strategic management in church organizations such as msc is apparent, but the literature on this topic is very rare or even absent. this study will contribute to developing the congregation’s future initiatives and at the same time address the lack of literature on this topic. in this study, the researchers aim to establish the most appropriate strategic priorities for the msc congregation in its service of ministry for the new millennium. the researchers plan to do this by developing an effective analytic hierarchy process (ahp), combining fuzzy-ahp and swot to develop an integrated process, with a clearer and a more effective set of priorities for ministry today. this study uses a combination of the swot and fuzzy-ahp methods. the researchers believe that a combination of these two methods is very powerful for the kind of strategic planning we are pursuing. thus far, the relevant literature indicates that no religious congregation has used this combination of swot and fuzzy-ahp. this is a pioneering strategic study of a catholic institution, in this case the msc congregation. although this research is a case study of the msc congregation, the results of this research will undoubtedly be useful as a model for any religious organization with similar characteristics. even organizations outside the catholic church can draw inspiration from the results of this research for the implementation of its vision and mission. ijahp article: ohoitimur, krejci, raco, raton, jamlean, welerubun, tanod/strategic management study based on buckley’s fuzzy-ahp and swot: example of the congregation of the missionaries of the sacred heart international journal of the analytic hierarchy process 332 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.871 this study is organized into seven sections. the first part of the study discusses the background, problems and objectives of the study. the second part is the literature review which includes an overview of previous studies. the third part investigates the methodology of swot and buckley’s fuzzy-ahp and their combination. the fourth part of the research discusses the results of the study which includes consistency tests, results of analysis of criteria, sub-criteria, alternatives, and global values. global value is the multiplication of the results of the criteria, sub-criteria and alternative strategies. the fifth part is the sensitivity analysis. the sixth part of the research discusses the results of the analysis. the seventh part highlights the conclusions and recommendations. 2. literature review 2.1. the missionary of the sacred heart (msc) like other church institutions, the msc has members, structures, and rules, and is involved in the management of properties. this congregation is a social entity, non-profit in nature, and strategic planning techniques will provide a more effective and efficient plan to accomplish the ministry of pastoral work for human salvation (ohoitimur, krejci, raco, raton, & taroreh, 2019). the msc congregation is an international group of priests and brothers in the catholic church, and the number of its members is growing. the congregation owns training centers and properties. their members are working as parish priests both in the country and abroad, some are acting as professionals mostly in educational institutions as lecturers or teachers. the institution manages assets and runs small businesses. the members practice a participatory style of leadership, supported by a strong fraternal community from within and by msc lay communities (kwakman, 2011). the church, as a service industry needs to have people with competence, credibility, reliability, responsiveness, trust, security, and an understanding of their customers (santos & mathews, 2001). excellent hospitality is demanded from church personnel such as msc members. they should be acting like good samaritans, giving hope and compassion, especially to the poor and destitute (dries, 2015). this is in line with the charism of the msc, namely special care for all people, especially the poor, the oppressed, the marginalized, through charity, gentleness, compassion and kindness. the essence of the msc charism is the belief that an act of love (agape) can change a human life. in light of that charism, the members live the spirituality of the heart as a way of life. the concrete manifestations of this charisma and spirituality can be seen in various aspects of msc's mission, such as special care for all people, especially the poor, the oppressed, the marginalized, through charity, gentleness, compassion and kindness (cuskelly, 1978). initially, this charism was a response to the prevailing social ills of the time in 19 th century france, namely, egoism and religious indifference (nguyen, 2014). jules chevalier, the founder of the msc congregation, advocated that all who are miserable must be touched and supported with a compassionate heart. according to chevalier, first of all, compassion must be lived by members of the congregation as a core part of spiritual identity establishing a religious brotherhood (kwakman, 2011). ijahp article: ohoitimur, krejci, raco, raton, jamlean, welerubun, tanod/strategic management study based on buckley’s fuzzy-ahp and swot: example of the congregation of the missionaries of the sacred heart international journal of the analytic hierarchy process 333 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.871 it is realized that religious identity and all the fundamental values contained in charisma, mission, and spirituality are dynamically related to one’s life. this means that religious identity, community life, ways of carrying out missions, and other forms of spirituality are always changing and require renewal. the msc congregation constantly faces aspects of renewal in religious life (cuskelly, 1978). these aspects include authority and leadership, community life and work, rules, and identity. in this context, strategic planning and management becomes an essential element in the future success of the msc congregation. one of the major challenges facing the church today, including the msc congregation, is the issue of clerical sex abuse, which has tarnished the good image of the church and its priesthood. it has caused extensive and profound trauma to the victims who are mostly children, often altar boys (warner, 2019). victims often feel hopeless, losing positive vision for their future, losing inner peace, and feeling no meaning in life. they encounter ethical upheaval and a disturbed relationship with god (demasure, 2016). the honor, respect, and trust they held for the priest, as god’s representative in the world, has been seriously damaged (warner, 2019). church personnel, particularly priests, are now considered a dangerous threat for the victims. as a result, faith in the church has declined significantly (donnelly, 2015). often the target of public mockery, the church, as an institution, is no longer trusted. (vela-mcconnell, 2018). the priests also experience many problems. priests are expected to have a lifetime commitment to the church through their service, loyalty, trust, and obedience to their superiors. they are required to live a holy life and prioritize common interests (community) rather than personal ambitions. many priests feel under pressure, exhibiting frustration and boredom. some priests feel a lack of support, from both fellow priests and superiors, thus affecting their motivation, mental health, and ultimately the quality of their work (mcdevitt, 2010). priests can also be discouraged by the fact that young people today, lured by the distractions of the modern world, no longer show interest in the church or priesthood (stark & finke, 2000). in addition, a growing number of people are leaving the church and no longer believing in god (brown, 2014). against this complex background, the msc in the region of sulawesi, now wish to look deeper into both their internal and external circumstances cited from the constitutions and statutes of the missionaries of the sacred heart of jesus (braun, 2000) and develop strategies to move forward in their ministry. braun mentioned several important aspects, namely the inspiration of spirituality of the heart, participatory leadership, member development and coaching, asset management, regional mission services, and the participation of the msc lay people-professed. 2.2. strategic management in the church there are three factors that highlight the importance of implementing strategic management in church organizations, including msc. first, the leadership has a commitment to develop suitable content and focus on proper use of resources. second, church organizations are tasked with serving a diverse group of people, and church ministers must have multiple strengths and weaknesses because they need to master the right strategy in ministry. third, achieving the vision and mission requires coordination ijahp article: ohoitimur, krejci, raco, raton, jamlean, welerubun, tanod/strategic management study based on buckley’s fuzzy-ahp and swot: example of the congregation of the missionaries of the sacred heart international journal of the analytic hierarchy process 334 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.871 of all levels of the organization. all of these activities are at the heart of sound management (chatira & mwenje, 2018). according to shah and david, the application of strategic management in church organizations has biblical basis (shah & david, 2004) such as: “where there is no vision, the people will perish….” (proverbs 29:18) ” without counsel, plans go awry, but in the multitude of counselors, they are established”. (proverbs 15:22) “listen to counsel and receive instruction, that you may be wise in your latter days” (proverbs 19:20) “plans are established by counsel: by wise counsel wage war”. (proverbs 20:18). the church, as a social entity and a non-profit organization, must start considering strategic planning techniques in the planning of pastoral services. by utilizing strategic planning methodologies, the church’s programs will become more efficient by having specific, focused targets and goals that can be measured, evaluated and improved over the years (ohotimur et al., 2019). 2.3. the swot analysis swot stands for strengths, weaknesses, opportunities, and threats. it is a very popular method, related to strategy analysis (taghavifard et al., 2018). the swot method is employed to analyze both internal and external conditions of an institution, organization, or company as a first step in developing a research strategy (kahraman, birgun, & yenen, 2008). the outline of the swot method is as follows. first, researchers and experts determine the main factors of both internal swot, namely strength (s) and weaknesses (w); and the external factors, namely opportunity (o) and threats (t). these factors are determined based on documents, literature, and previous studies. the next step is to determine an alternative strategy that is a combination of internal and external factors which will produce four categories which are s-o, w-t, s-t, and w-o (taghavifard et al., 2018). the s-o strategy uses the internal strengths to take advantage of external opportunities (ideal case). the combination of s-o is often called a maximaxi strategy. w-t strategies are called 'mini-mini strategies'. these alternative strategies are aimed at reducing internal weaknesses and avoiding threats. s-t or “maxi-mini strategies” are used to reduce the impact of external threats. w-o is adopted to minimize weaknesses by utilizing opportunities. this strategy is also called “mini-maxi” (mousavi & behzadi, 2019). the weakness of this method is that the identification of relevant factors only happens at a qualitative level. the swot method does not quantify or rank the factors identified. however, the swot method remains useful for identifying the state of the institution or organization as a first step in determining an appropriate strategy (koca & behdioğlu, 2017). swot can then be integrated with other methods such as ahp and fuzzy-ahp, so that quantification of relevant factors can be made and their importance can be calculated for further use. (tavana, zareinejad, di caprio, & kaviani, 2016). ijahp article: ohoitimur, krejci, raco, raton, jamlean, welerubun, tanod/strategic management study based on buckley’s fuzzy-ahp and swot: example of the congregation of the missionaries of the sacred heart international journal of the analytic hierarchy process 335 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.871 saaty (2008) explained that there are three basic principles to the ahp method, namely decomposition, comparison of judgments, and priority synthesis (ivanco, hou, & michaeli, 2017). decomposition describes solving or dividing a problem into a hierarchical structure. breaking down the problems into a hierarchy will help the researcher unravel the complexities (raco et al., 2020). decomposition is one of the ways to circumvent the limitations of human thought in dealing with a large amounts of information, allowing structured assessment of the trade-offs between conflicting criteria (promentilla et al., 2018). the structure consists of three levels, namely the first level (goal), the second (criteria) and the third (alternatives) (fabjanowicz, bystrzanowska, namie, tobiszewski, & justyna, 2018). the analysis in strategic planning starts with environmental scanning of the organization, including the religious institution. in this analysis the strengths, potentials, challenges, and threats are observed. this can be done using the swot method (oreski, 2012). swot analysis is a good tool used to dissect an organization or institution. the right choice of strategy can be obtained through understanding the relationship between internal and external factors (roghanian, ghorbani, & alipour, 2018). swot analysis has become a very broad reference used for an instrument that analyzes an institution (ghazinoory, esmail zadeh, & memariani, 2007). oreski added that swot is the most widely used method by managers to carry out strategic planning (oreski, 2012). the greatest benefit of swot is its simplicity as it can be used and understood by anyone (roghanian et al., 2018). 2.4. fuzzy-ahp the main input of the ahp method is the perception of experts, so there is a factor of subjectivity in the decision-making (alzoubi, mohammad, & abu-salma, 2015). this method also takes into account data validity due to inconsistency limits. however, considerable uncertainty and doubt in making an assessment will have an impact on the accuracy of the data and the results obtained, requiring the further use of a strategy known as the fuzzy analytic hierarchy process. fuzzy analytic hierarchy process (fahp) is the ahp method developed in conjunction with fuzzy logic theory, specifically triangular fuzzy. the steps of solving a problem using the fuzzy ahp method are almost the same as the ahp method. the fuzzy ahp method changes the ahp scale into a triangular fuzzy scale to establish priorities. a disadvantage of the ahp method is its limitation in eliminating subjective factors and inaccuracies because this method uses crisp (single) numbers (esmaeili, kahnali, rostamzadeh, & sepahvand, 2014). inevitably, there will be nuances of gray (vagueness) and uncertainty in the respondents' perceptions. therefore, the combination of swotahp with fuzzy is the right solution. fahp is widely used in research related to prioritization. researchers use fahp because fuzzy ratios help respondents provide answers related to imprecise and vague elements in pairwise comparisons. additionally, respondents’ uncertainty over their choices is reduced (lohan, ganguly, & kumar, 2020) and it avoids the use of crisp numbers that are used in the ahp (ganguly & kumar, 2019). ijahp article: ohoitimur, krejci, raco, raton, jamlean, welerubun, tanod/strategic management study based on buckley’s fuzzy-ahp and swot: example of the congregation of the missionaries of the sacred heart international journal of the analytic hierarchy process 336 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.871 the fahp used in this study is buckley's fahp which presents a three-step decisionmaking process, namely finalize the weights, then normalize weights for all the attributes/factors and finally rank the alternatives (lohan et al., 2020). 3. method one of the limitations of the swot method is its inability to provide quantitative data that would be helpful to decision makers. to overcome this, the swot method is usually combined with other methods such as ahp and fahp. the combination of swot, ahp and fahp assists the decision maker by establishing a priority indicated by the quantitative data. current conditions, marked by rapid change in contemporary organizations or institutions are difficult to predict, due to prevailing uncertainties. therefore a combination of these methods is very appropriate to use (taghavifard, mahdiraji, alibakhshi, zavadskas, & bausys, 2018), and have been adopted for the present study. this research applies a hybrid method of swot, ahp and fahp utilizing five steps. the first step is a literature search to identify factors or criteria and sub-criteria regarding the management of religious congregations. the second step is to discuss the factors obtained through the literature in focused group discussions with msc leaders to identify, discuss and explore in more depth the internal and external conditions of the msc congregation using the swot method. the criteria, sub-criteria and the strategic alternatives of the swot analysis were later put into a hierarchical structure as shown in figure 1 (explanation of the criteria and sub-criteria is provided in the appendix). ijahp article: ohoitimur, krejci, raco, raton, jamlean, welerubun, tanod/strategic management study based on buckley’s fuzzy-ahp and swot: example of the congregation of the missionaries of the sacred heart international journal of the analytic hierarchy process 337 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.871 figure 1 hierarchy structure of the msc development in the third step, the researchers quantify the results of the swot analysis using the ahp method. the assessment is done by comparing the criteria in pairs using a numeric scale from 1 to 9. the scores of the assessment obtained are then arranged into a pairwise comparison matrix and a synthesis process is carried out to obtain the value of each criterion. the value of each criterion is obtained by calculating priority vectors (eigenvectors) from pairwise comparison matrices. for example, there are n criteria 𝑎1, 𝑎2, 𝑎…, 𝑎𝑛 with 𝑤𝑖 𝑤𝑗 (𝑖 = 1,2, … , 𝑛, 𝑗 = 1,2, … , 𝑛) the weight of the pairwise comparison, then a pairwise comparison matrix can be arranged based on equation 1 as follows. 𝐴 = [𝑎𝑖𝑗 ], 𝑎𝑖𝑗 = 𝑤𝑖 𝑤𝑗⁄ , 𝑎𝑗𝑖 = 1 𝑎𝑖𝑗 ⁄ 𝑎𝑖𝑖 = 1 (1) normalization of pairwise comparison matrices based on equation 2 below: 𝑏𝑖𝑗 = 𝑎𝑖𝑗 ∑ 𝑎𝑖𝑗 𝑛 𝑖=1 (2) ijahp article: ohoitimur, krejci, raco, raton, jamlean, welerubun, tanod/strategic management study based on buckley’s fuzzy-ahp and swot: example of the congregation of the missionaries of the sacred heart international journal of the analytic hierarchy process 338 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.871 priority weights are obtained by averaging the rows in the normalized pairwise comparison matrix with the following equation 3. 𝑤𝑖 = ∑ 𝑏𝑖𝑗 𝑛 𝑗=1 𝑛 (3) the ahp method expects some inconsistency in the results. many questions are interrelated making it necessary to calculate the extent to which the results of the questionnaire are consistent (mu & pereyra-rojas, 2018). if the opposite happens, then the process of filling out the questionnaire needs to be repeated. a consistency test is performed to determine the consistency of the respondent's perceptions (mu & stern, 2014). the basic principle of consistency testing is if a> b, and b > c, then it should be a > c. the consistency test calculation steps are as follows: calculate the maximum eigenvalue (λ_max) using equation 4. 𝜆𝑚𝑎𝑥 = ∑ (𝐴𝑤)𝑖 𝑛𝑤𝑖 𝑛 𝑖=1 (4) next, calculate the consistency index (ci) using equation 5. 𝐶𝐼 = 𝜆𝑚𝑎𝑥 − 𝑛 𝑛 − 1 (5) the final step is to calculate the consistency ratio (cr). the consistency ratio value is the comparison between the consistency index (ci) and the random index (ri). the ratio consistency value is calculated according to the following equation 6. 𝐶𝑅 = 𝐶𝐼 𝑅𝐼 (6) saaty (2008) compiled a random index obtained from an average of 500 matrix consistency indexes. the ri value is the ri value every n object with 2 ≤ 𝑛 ≤ 15 table 1 random index (ri) n 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 r.i 0 0 0.58 0.90 1.12 1.24 1.32 1.41 1.45 1.49 1.51 1.48 1.56 1.57 1.59 if the consistency ratio value (𝐶𝑅) ≤ 0,1 (10%), the respondent's perception is considered consistent and the results of other calculations are considered valid. ijahp article: ohoitimur, krejci, raco, raton, jamlean, welerubun, tanod/strategic management study based on buckley’s fuzzy-ahp and swot: example of the congregation of the missionaries of the sacred heart international journal of the analytic hierarchy process 339 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.871 in theory, the respondent's perception is valid (perfect) if the value (𝐶𝑅) = 0. however, in practice this is difficult to achieve, given the complexity of the problems that influence the decision maker in making decisions. therefore, the value of cr according to equation 6 is the tolerance limit of the inconsistency of respondents' permitted perceptions. when consistent conditions are met using the ahp method, the researchers than proceed to the next step. in the fourth step, the researchers proceeded with weighing the respondents' perceptions using buckley’s fahp method (hsieh, lu, & tzeng, 2004). the initial step of the buckley’s fahp is to convert the perception of respondents on the ahp scale that has been completed on the questionnaire into the form of a fuzzy triangle number (tfn) on the fahp scale (table 2) (cebeci, 2009) table 2 function members of linguistic scale linguistic scales scale of fuzzy number equally important (1, 1, 1) weakly important (1, 3, 5) fairly important (3, 5, 7) very strongly important (5, 7, 9) absolutely important (7, 9, 9) the steps to determine respondents' perception weights using the fahp method according to buckley as mentioned by hsieh (2004) are as follows: step 1. design pairwise comparison matrix for criteria, subcriteria and alternatives with respect to each sub-criteria. �̃� = [ 1 �̃�12 ⋯ �̃�1𝑛 �̃�21 1 ⋯ �̃�2𝑛 ⋮ ⋮ ⋱ ⋮ �̃�𝑛1 �̃�𝑛2 … 1 ] = [ 1 �̃�12 ⋯ �̃�1𝑛 1/�̃�12 1 ⋯ �̃�2𝑛 ⋮ ⋮ ⋱ ⋮ 1/�̃�1𝑛 1/�̃�2𝑛 ⋯ 1 ] (7) with, �̃�𝑖𝑗 = { 1̃, 3̃, 5̃, 7̃, 9,̃ 𝑐𝑟𝑖𝑡𝑒𝑟𝑖𝑜𝑛 𝑖 𝑖𝑠 𝑟𝑒𝑙𝑎𝑡𝑖𝑣𝑒 𝑖𝑚𝑝𝑜𝑟𝑡𝑎𝑛𝑐𝑒 𝑡𝑜 𝑐𝑟𝑖𝑡𝑒𝑟𝑖𝑜𝑛 𝑗 1, 𝑖 = 𝑗 1̃−1, 3̃−1, 5̃−1, 7̃−1, 9 ̃−1, 𝑐𝑟𝑖𝑡𝑒𝑟𝑖𝑜𝑛 𝑖 𝑖𝑠 𝑟𝑒𝑙𝑎𝑡𝑖𝑣𝑒 𝑙𝑒𝑠𝑠 𝑖𝑚𝑝𝑜𝑟𝑡𝑎𝑛𝑐𝑒 𝑡𝑜 𝑐𝑟𝑖𝑡𝑒𝑟𝑖𝑜𝑛 𝑗 step 2. calculate the geometric mean of fuzzy comparison value of criterion i to each criterion using the following formula, ijahp article: ohoitimur, krejci, raco, raton, jamlean, welerubun, tanod/strategic management study based on buckley’s fuzzy-ahp and swot: example of the congregation of the missionaries of the sacred heart international journal of the analytic hierarchy process 340 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.871 �̃�𝑖 = (�̃�𝑖1⨂ �̃�𝑖2⨂ ⋯ ⨂ �̃�𝑖𝑛 ) 1/𝑛 (8) where, �̃�𝑖𝑛 is fuzzy comparison value of criterion i to criterion n. step 3. determine fuzzy weight of each criterion indicated by triangular fuzzy number �̃�𝑖 = �̃�𝑖 ⨂(�̃�1⨁ ⋯ ⨁ �̃�𝑛 ) −1 (9) where, �̃�𝑖 adalah fuzzy weight of the ith criterion and can be indicated using triangular fuzzy number, �̃�𝑖 = (𝐿𝑤𝑖 , 𝑀𝑤𝑖 , 𝑈𝑤𝑖 ). 𝐿𝑤𝑖 , 𝑀𝑤𝑖 𝑎𝑛𝑑 𝑈𝑤𝑖 mean is lower, middle and upper value of the fuzzy weight of the ith criterion. step 4. the process of defuzzification using the centre of area method to get the weight of best nonfuzzy performance (bnp) applying equation 10, 𝐵𝑁𝑃𝑤𝑖 = [(𝑈𝑤𝑖 − 𝐿𝑤𝑖 ) + (𝑀𝑤𝑖 − 𝐿𝑤𝑖 )]/3 + 𝐿𝑤𝑖 (10) step 5. calculate the weight and rank of the alternatives. the alternatives’ weight with respect to each criteria is calculated using the equations 8 10. the final value for each alternative is obtained by multiplying the alternative weight with the weight of the corresponding sub-criteria (cebeci, 2009). then, the calculation of bnp of each alternative is performed applying equation 10. the fifth step consisted of a sensitivity analysis. a sensitivity analysis or "what-if" analysis is performed to determine whether there will be a change in the results if the value of the criteria changes. a sensitivity analysis aims to determine how robust the results are. at the same time, a sensitivity analysis provides information about the main driver (mu & pereyra-rojas, 2018). a sensitivity analysis will guarantee the stability of the results against the various assessments (mu & stern, 2014). 4. results 4.1. swot factors researchers and experts identified swot factors and sub-factors, later known as criteria and sub-criteria, guided by seven elements which are as follows: identity, leadership, personnel, structure, mission, environment, facilities or infrastructures (table 3). ijahp article: ohoitimur, krejci, raco, raton, jamlean, welerubun, tanod/strategic management study based on buckley’s fuzzy-ahp and swot: example of the congregation of the missionaries of the sacred heart international journal of the analytic hierarchy process 341 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.871 table 3 swot factors and sub-factors swot factors (criteria) swot sub factors (sub criteria) elements strength s1. inspired by heart spirituality identity s2. participative and subsidiarity style of leadership leadership s3. increasing number of members; members are more productive; parish ministry; professional and expertise members personnel s4. national and international ministry; 100 years of service; fraternity of community life s5. community-based congregation structure s6. having formation centers; own property assets facilities and infrastructure weakness w1. heart spirituality is not yet comprehensive; identity w2. having limited qualified leaders; having limited integrated-program of heart spirituality promotion leadership w3. lack of assistance to elderly members; lack of professional assistance personnel w4. having less categorized ministry; having limited productive work mission w5. weak fighting spirit of the young members environment w6. protocol to handle problems in congregation is not yet available. structure w7. there is no nursing facility; it-based ministry not yet available; having limited funding for ministry development and promotion; regulation of assets use is not available facilities and infrastructure opportunities o1. huge opportunities to strengthen heart spirituality identity o2. congregation can develop a good corporate governance; strengthening cooperation with government, dioceses in the country and abroad; chances to upgrade visitation program and job description of msc leaders at provincial level. leadership o3. regeneration programs for young members; improve quality and professionalism of training; opportunities to upgrade education of the brothers personnel o4. develop family ministry; catechism for people; develop the accompaniment of chevalier’s families; managing the faithful’s support mission o5. expand the charity movement of the faithful environment o6. improve the function and role of community structure o7. chances to boost revenues and assets; improve financial management; investment in credible ijahp article: ohoitimur, krejci, raco, raton, jamlean, welerubun, tanod/strategic management study based on buckley’s fuzzy-ahp and swot: example of the congregation of the missionaries of the sacred heart international journal of the analytic hierarchy process 342 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.871 swot factors (criteria) swot sub factors (sub criteria) elements financial institution; develop management of inheritance; intensify the fund raising threats t1. ministry not based on it and strategic planning leadership t2. for young people, priesthood is not first choice; freedom inspired by modernity threaten identities identity t3. problems in family are threatening the vocation; diocesan priests overtake ministry of the msc mission t4. post-modern fluidity and hedonism; family numbers getting smaller; spirit of the age weakening the militancy struggles of young members; extensive negative publication due to clergy sex abuse. environment 4.2. factors of strategic alternatives development strategies of the msc were structured by researchers and experts using a combination of the identified internal and external factors of swot. there were four strategic alternatives which were s-o, s-t, w-t and w-o. the elements of each strategic alternative were as follows: so 1: intensifying the improvement of human resources so 2: enhancing the msc structure to be more dynamic and accessible to all members st 1: disseminating heart spirituality st 2: empowering fraternal community wo1: strengthening networking wo2: revitalizing family ministry wt1: strengthening the heart spirituality at all levels of the community 4.3. data analysis 4.3.1. consistency test the respondents in this study are 16 members of the msc region of sulawesi. they met the criteria to be respondents (raco & tanod, 2014) because they are the decision makers who have been deeply involved with msc, holding and sharing all the relevant information needed by the researchers. the respondents completed the questionnaires that were in pairwise comparison matrix form. then, the researchers aggregated the data using a geometric mean formula. afterwards, the data was analyzed using the ahp method by applying equations 16 to determine the consistency of the respondents’ answers. based on the consistency test on criteria, sub-criteria and alternatives, the results showed that the consistency ratio of all elements was less than 10% or (cr) < 0.1. the researchers concluded that the results were valid and accepted. 4.3.2. perception weighting using fuzzy-ahp once the consistency of the data was approved, the research process was extended by applying the fahp. the respondents’ assessments were then aggregated along with the structure in fuzzy pairwise comparison matrix for each criterion, sub-criteria and ijahp article: ohoitimur, krejci, raco, raton, jamlean, welerubun, tanod/strategic management study based on buckley’s fuzzy-ahp and swot: example of the congregation of the missionaries of the sacred heart international journal of the analytic hierarchy process 343 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.871 alternatives. the fuzzy pairwise comparison matrix of the level of importance for the criteria is shown in table 4 below. table 4 fuzzy pairwise comparison matrix of the level of importance for the criteria criteria s w o t s (1, 1, 1) (1.47, 2.11, 2.73) (0.91, 1.28, 1.66) (1.27, 1.87, 2.64) w (0.37, 0.47, 0.68) (1, 1, 1) (0.37, 0.51, 0.91) (0.83, 1.05, 1.38) o (0.60, 0.78, 1.10) (1.10, 1.96, 2.73) (1, 1, 1) (1.38, 2.39, 3.16) t (0.38, 0.53, 0.79) (0.73, 0.95, 1.21) (0.32, 0.42, 0.73) (1, 1, 1) determining the geometric mean of fuzzy comparison value of criteria, strength, using equation 8 as follows: �̃�𝑆 = (�̃�11⨂ �̃�12⨂ �̃�13 ⨂ �̃�14) 1/4 = ((1 × 1.47 × 0.91 × 1.27)1/4, (1 × 2.11 × 1.28 × 1.87)1/4, (1 × 2.73 × 1.66 × 2.64)1/4) = (1.142, 1.499, 1.861) using the same method to obtain the geometric mean of fuzzy comparison value of the other criteria as follows: �̃�𝑊 = (0.577, 0.710, 0.959) �̃�𝑂 = (0.978, 1.384, 1.755) �̃�𝑇 = (0.543, 0.679, 0.912) next, determine the weight of the criteria, strength, based on equation 9 as follows: �̃�𝑆 = �̃�𝑆⨂(�̃�𝑆⨁ �̃�𝑊 ⨁ �̃�𝑂 ⨁ �̃�𝑇 ) −1 = (1.142, 1.499, 1.861 )⨂(1/(1.861 + 0.959 + 1.755 + 0.912), 1/(1.499 + 0.710 + 1.384 + 0.679) + 1/(1.142 + 0.577 + 0.978 + 0.543)) = (0.208, 0.351, 0.574) we also get the weight of other criteria as follows: �̃�𝑊 = (0.105,0.166,0.296) �̃�𝑂 = (0.178,0.324,0.542) �̃�𝑇 = (0.099,0.159,0.281) furthermore, the best nonfuzzy performance (bnp) value for the strength criteria is calculated based on equation 10 as follows: 𝐵𝑁𝑃𝑤𝑆 = [(𝑈𝑤𝑆 − 𝐿𝑤𝑆) + (𝑀𝑤𝑆 − 𝐿𝑤𝑆)]/3 + 𝐿𝑤𝑆 = [(0,574 − 0,208) + (0,351 − 0,208)]/3 + 0,208 = 0,378 in the same way the bnp value can be calculated for other criteria and the results as follows: ijahp article: ohoitimur, krejci, raco, raton, jamlean, welerubun, tanod/strategic management study based on buckley’s fuzzy-ahp and swot: example of the congregation of the missionaries of the sacred heart international journal of the analytic hierarchy process 344 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.871 𝐵𝑁𝑃𝑤𝑊 = 0,189 𝐵𝑁𝑃𝑤𝑂 = 0,348 𝐵𝑁𝑃𝑤𝑇 = 0,180 the weights for each sub-criterion are calculated in the same way using equations 8, 9 and 10 and the results are listed in the following table. table 5 local and overall weight of criteria and sub-criteria criteria and sub criteria local weight overall weight fuzzy bnp and normalized bnp and normalized strength (0.21, 0.35, 0.57) 0.345 s1 (0.15, 0.28, 0.48) 0.268 0.093 s2 (0.12, 0.23, 0.43) 0.228 0.079 s3 (0.07, 0.14, 0.27) 0.145 0.050 s4 (0.05, 0.09, 0.20) 0.103 0.035 s5 (0.09, 0.15, 0.27) 0.152 0.052 s6 (0.06, 0.10, 0.19 0.105 0.036 weakness (0.11, 0.17, 0.30) 0.173 w1 (0.10, 0.21, 0.41) 0.205 0.035 w2 (0.08, 0.17, 0.35) 0.172 0.030 w3 (0.08, 0.15, 0.29) 0.149 0.026 w4 (0.05, 0.10, 0.20) 0.101 0.017 w5 (0.07, 0.12, 0.25) 0.126 0.022 w6 (0.06, 0.11, 0.23) 0.115 0.020 w7 (0.07, 0.13, 0.26) 0.131 0.023 opportunities (0.18, 0.32, 0.54) 0.318 o1 (0.10, 0.18, 0.31) 0.178 0.057 o2 (0.09, 0.15, 0.25) 0.146 0.046 o3 (0.09, 0.16, 0.28) 0.161 0.051 o4 (0.09, 0.15, 0.25) 0.149 0.047 o5 (0.06, 0.10, 0.21) 0.108 0.034 o6 (0.07, 0.13, 0.24) 0.135 0.043 o7 (0.07, 0.12, 0.21) 0.124 0.039 threats (0.10, 0.16, 0.28) 0.164 t1 (0.14, 0.23, 0.42) 0.238 0.039 t2 (0.14, 0.28, 0.52) 0.282 0.046 t3 (0.13, 0.22, 0.41) 0.225 0.037 t4 (0.15, 0.26, 0.43) 0.254 0.042 the pairwise comparison, geometric mean of fuzzy comparison value, and the weights of each alternative with respect to the corresponding sub-criterion were calculated in the same way using equations 8 and 9 and the results are listed in the following tables. the final alternative value is obtained by multiplying the weight of each alternative with the corresponding sub-criterion weight, the results are listed in table 6. ijahp article: ohoitimur, krejci, raco, raton, jamlean, welerubun, tanod/strategic management study based on buckley’s fuzzy-ahp and swot: example of the congregation of the missionaries of the sacred heart international journal of the analytic hierarchy process 345 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.871 table 6 final alternative value sub crit. weight (bnp) alternative sub crit local overall local weight (bnp) global weight (bnp) so st wo wt so st wo wt a b c d e f g 𝐻 = 𝐷 ∗ 𝐶 𝐼 = 𝐸 ∗ 𝐶 𝐽 = 𝐹 ∗ 𝐶 𝐾 = 𝐺 ∗ 𝐶 s1 0.268 0.093 0.314 0.278 0.216 0.249 0.029 0.026 0.020 0.023 s2 0.228 0.079 0.352 0.288 0.166 0.277 0.028 0.023 0.013 0.022 s3 0.145 0.050 0.359 0.285 0.165 0.256 0.018 0.014 0.008 0.013 s4 0.103 0.035 0.322 0.289 0.188 0.283 0.011 0.010 0.007 0.010 s5 0.152 0.052 0.325 0.305 0.188 0.255 0.017 0.016 0.010 0.013 s6 0.105 0.036 0.339 0.291 0.205 0.240 0.012 0.011 0.007 0.009 w1 0.205 0.035 0.356 0.271 0.178 0.299 0.013 0.010 0.006 0.011 w2 0.172 0.030 0.366 0.281 0.197 0.261 0.011 0.008 0.006 0.008 w3 0.149 0.026 0.316 0.315 0.196 0.302 0.008 0.008 0.005 0.008 w4 0.101 0.017 0.298 0.288 0.299 0.232 0.005 0.005 0.005 0.004 w5 0.126 0.022 0.328 0.286 0.210 0.305 0.007 0.006 0.005 0.007 w6 0.115 0.020 0.321 0.280 0.201 0.281 0.006 0.006 0.004 0.006 w7 0.131 0.023 0.361 0.278 0.199 0.281 0.008 0.006 0.005 0.006 o1 0.178 0.057 0.258 0.342 0.195 0.312 0.015 0.019 0.011 0.018 o2 0.146 0.046 0.399 0.271 0.244 0.230 0.018 0.013 0.011 0.011 o3 0.161 0.051 0.429 0.266 0.188 0.238 0.022 0.014 0.010 0.012 o4 0.149 0.047 0.270 0.240 0.321 0.271 0.013 0.011 0.015 0.013 o5 0.108 0.034 0.232 0.274 0.346 0.266 0.008 0.009 0.012 0.009 o6 0.135 0.043 0.325 0.295 0.172 0.286 0.014 0.013 0.007 0.012 o7 0.124 0.039 0.348 0.291 0.215 0.265 0.014 0.011 0.008 0.010 t1 0.238 0.039 0.265 0.279 0.256 0.270 0.010 0.011 0.010 0.011 t2 0.282 0.046 0.368 0.238 0.290 0.226 0.017 0.011 0.013 0.010 t3 0.225 0.037 0.316 0.272 0.282 0.231 0.012 0.010 0.010 0.009 t4 0.254 0.042 0.310 0.261 0.236 0.253 0.013 0.011 0.010 0.011 sum 0.329 0.282 0.219 0.264 the final results appear in table 7. table 7 final results of alternatives, bnp after normalization and ranking alternative bnp and normalized ranking so 0.301 1 st 0.258 2 wo 0.201 4 wt 0.241 3 5. sensitivity analysis a sensitivity analysis was carried out to understand the robustness of the decision priorities obtained as a result of the analysis in determining the overall priority. in addition, the sensitivity analysis helped determine which criteria are very influential in the original results. it is understood that the overall priorities of criteria are influenced by the weight of each criterion. for this reason, a sensitivity analysis was carried out using a "what-if" analysis to determine the change in the final results if the weight of the criteria changed. ijahp article: ohoitimur, krejci, raco, raton, jamlean, welerubun, tanod/strategic management study based on buckley’s fuzzy-ahp and swot: example of the congregation of the missionaries of the sacred heart international journal of the analytic hierarchy process 346 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.871 various scenarios were assumed in the sensitivity analysis, and the results are listed in table 8 below. table 8 scenarios in sensitivity analysis scenario weight of criteria (bnp) overall priority of alternative (bnp and normalized) s w o t so st wo wt original 0.345 0.173 0.318 0.164 0.301 0.258 0.201 0.241 sc1 0.250 0.250 0.250 0.250 0.300 0.255 0.205 0.240 sc2 0.100 0.300 0.300 0.300 0.298 0.252 0.210 0.240 sc3 0.300 0.100 0.300 0.300 0.299 0.255 0.208 0.238 sc4 0.300 0.300 0.100 0.300 0.302 0.255 0.203 0.240 sc5 0.300 0.300 0.300 0.100 0.302 0.258 0.197 0.244 in scenario 1, the criteria weights are made equal, each as 0.25. the results show that alternative so remains the first priority and there is no significant change in the value of the alternative. the alternative rankings do not change from the original, which are so, st, wt, wo going from largest to smallest. scenarios 2 to 5 weigh three (3) criteria equal to 30% each, while one criterion is 10%. as a result, alternative so remains first, and the alternative ranking order remains, respectively from the largest so, st, wt, wo. these results indicate that changes in the criterion weights do not affect the alternative rankings. thus, it can be concluded that the decisions made by the decision maker are very solid. 6. discussion swot analysis is very useful in assessing our situation (wickramasinghe & takano, 2010). the combination of swot and fahp was helpful in formulating strategic planning quantitatively that can reveal the priorities that need to be addressed. even though the respondents’ sample is small, a combination of the two methods can be done based on the knowledge, lived-experience, and rich information provided by the respondents. through the calculation of the consistency ratio, the researchers found that all the data are consistent, so the analysis is acceptable. the results showed that the alternative strategy s-o received 30.1%, which was the highest. the alternative strategy of s-o is about the intensification and improvement of human resources, thus enhancing the msc structure to be more dynamic and accessible to all members. it means that the msc should give priority to creating a program to improve their human resources and strengthen their structure to be more dynamic and accessible to all members. according to wickramasinge & takano (2009), the position of s-o was the ideal position because it means that the institution has a strong internal ijahp article: ohoitimur, krejci, raco, raton, jamlean, welerubun, tanod/strategic management study based on buckley’s fuzzy-ahp and swot: example of the congregation of the missionaries of the sacred heart international journal of the analytic hierarchy process 347 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.871 environment. therefore, internally the msc is strong. this is also evident in the calculation of criteria where this strength was at the highest level (34.5%). the strength was shown in sub-criterion s1 that was the identity of the msc, which is inspired by sacred heart spirituality. other strengths of the congregation were demonstrated in sub criterion s2 (7.9%) highlighting the participative and subsidiarity style of leadership. through the s-o strategy, the msc must maximize its internal strengths to take greater advantage of its external opportunities. this is called a maxi-maxi strategy. the second strategy is s-t (25.8%), which refers to disseminating sacred heart spirituality and empowering fraternal community. the findings required the msc to optimize its strengths to eliminate its external threats. the data showed the external threat of sub criterion t2 (4.6%),which was about the modern era that offers attractive facilities and unlimited freedom for people, to a degree that has made the priesthood not interesting especially for the younger generation. however, sub criterion s1 (9.3%) is greater than sub criterion t2 (4.6%). the results recommended that msc place strategy s-t as the second priority in their program. the third strategy is w-t (24.1%) which is to strengthen the sacred heart spirituality in all levels of the community. the data showed that sub criterion w1 (3.5%), which is the spirituality of the heart is not yet comprehensive. the findings recommended that the msc should reduce its internal weaknesses to eliminate its external threats. in other words, the msc has to do its best to diminish the percentage of w1 to eliminate its threats. the fourth strategy is w-o (20.1%), which is strengthening the congregation’s capacity for networking and revitalizing the family ministry. the w-o strategy required the msc to reduce its weaknesses by taking advantages of its opportunities. the sub criterion o1 (5.7%) indicated that the msc has huge opportunities to strengthen sacred heart spirituality which can be maximized to minimize its weaknesses. putting all the above into a program for action, the first priority strategy of s-o should become a short-term program and is very urgent. the second strategy of s-t then becomes the middle-term program, and the third (w-t) and fourth strategies (w-o) can be a long-term undertaking. 7. conclusion this study aimed to determine the priorities for strategic planning of the msc using a combination of swot and fahp analysis. the combination of these two methods is adequate to determine the priorities quantitatively. the swot analysis is used to scan the internal and external elements of an organization as an initial step of strategic planning. the fahp is useful to minimize the vagueness and uncertainties of human thoughts and perceptions. the research findings are congruent because all the data analysis of the criteria, subcriteria and alternatives are consistent. the findings indicate that the s-o (30.1%) was the highest priority. this strategy was related to intensifying the improvement of human resources and enhancing the msc structure to be more dynamic and accessible to all members. the second was s-t (25.8%), which is related to the dissemination of sacred ijahp article: ohoitimur, krejci, raco, raton, jamlean, welerubun, tanod/strategic management study based on buckley’s fuzzy-ahp and swot: example of the congregation of the missionaries of the sacred heart international journal of the analytic hierarchy process 348 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.871 heart spirituality and empowering fraternal community among the members. w-t (24.1%) is the third strategy which aims at strengthening the sacred heart spirituality at all levels of the community. finally, w-o (20.1%) aims at strengthening the congregation’s capacity for networking and revitalizing the family ministry. the results of the sensitivity analysis reinforce the so strategy as the main strategy that must be a priority in preparing the msc congregation's future development program. the findings reveal that the members of the congregation believe that the spirituality of the heart is the primary incentive for animating the members. it also distinguishes them from other religious congregations. this is the internal strength proudly possessed by the msc. internally, this congregation is strong and its future looks quite promising. however, further work is required to deepen the sacred heart spirituality amongst the members of the congregation. the main contribution of this study is that it objectifies and quantifies the strategic priorities for the future renewal of the msc congregation. the combination of methodologies involved is very helpful for msc leaders to determine strategic priorities for the future. as already stated, the researchers claim that this is a pioneering study adopted for the first time in the catholic church – by the msc congregation using a combination of swot and fuzzy-ahp analysis. there are some limitations to such a study. some respondents prefer other options, to honor a variety of tastes and experiences. the researchers also recommend that further studies be carried out on the expectations of young people interested in joining the msc congregation, or becoming involved in its various ministries. ijahp article: ohoitimur, krejci, raco, raton, jamlean, welerubun, tanod/strategic management study based on buckley’s fuzzy-ahp and swot: example of the congregation of the missionaries of the sacred heart international journal of the analytic hierarchy process 349 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.871 references alzoubi, h., mohammad, s., & abu-salma, a. (2015). evaluating strategic management dimensions using analytic hierarchy process (ahp) and its impact on organizational success. international journal of research in management, 5(1), 137-150. brown, n. (2014). spirituality and the new atheism. in compass, 48(4), 26. braun, c. (2000). constitutions and statutes of the missionaries of the sacred heart of jesus. italy. candy, v., & gordon, j. (2011). the historical development of strategic planning theories. international journal of management & information systems, 15(4), 71–90. doi: https://doi.org/10.19030/ijmis.v15i4.5803 cebeci, u. (2009). fuzzy ahp-based decision support system for selecting erp systems in textile industry by using balanced scorecard. exper system with applications, 36, 8900–8909. doi: https://doi.org/10.1016/j.eswa.2008.11.046 chatira, f., & mwenje, j. (2018). the development of management skills for effective church management in pastoral preparation programs in zimbabwe. african journal of business management, 12(5), 103–120. doi: https://doi.org/10.5897/ajbm2018.8500 cuskelly, e. . (1978). a new heart and a new spirit. rome: grafiche citta’ del castello. ganguly, a., & kumar, c. (2019). evaluating supply chain resiliency strategies in indian pharmaceutical sector: a fahp process approach. international journal of analytic hierarchy process, 11(2), 153–180. doi: https://doi.org/10.13033/ijahp.v11i2.620 gyampoh, r. k. a.-, & asare, a. o. (2019). assessing the impact of good governance , church management and structure on the growth and development of the church. international journal of business and management, 14(4), 99–106. doi: https://doi.org/10.5539/ijbm.v14n4p99 hsieh, t., lu, s., & tzeng, g.-h. (2004). fuzzy mcdm approach for planning and design tenders selection in public office building. international journal project management, 22, 573–584. doi: https://doi.org/10.1016/j.ijproman.2004.01.002 kwakman, h. (2011). jules chevalier’s charism and the identity of the chevalier family. manado: cahaya pineleng. lohan, a., ganguly, a., & kumar, c. (2020). what’s foreign is better: a fuzzy-ahp analysis to evaluate factors that influence foreign product choice among indian consumers. international journal of analytic hierarchy process, 12(3), 460–487. doi: https://doi.org/10.13033/ijahp.v12i3.743 miller, kent, d. (2016). a case for including religious organizations in management research. journal of management, spirituality & religion, 3(3), 214–230. doi: ijahp article: ohoitimur, krejci, raco, raton, jamlean, welerubun, tanod/strategic management study based on buckley’s fuzzy-ahp and swot: example of the congregation of the missionaries of the sacred heart international journal of the analytic hierarchy process 350 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.871 https://doi.org/10.1080/14766080609518626 miller, k. d. (2002). competitive strategies of religious organizations. strategic management journal, 23, 435–456. doi: https://doi.org/10.1002/smj.234 mu, e., & pereyra-rojas, m. (2018). practical decision making using super decisions v3. an introduction to the analytic hierarchy process. pittsburgh: springer. mu, e., & stern, h. a. (2014). the city of pittsburgh goes to the cloud: a case study of cloud solution strategic selection and deployment. journal of information technology teaching cases, 4(2), 70–85. doi: https://doi.org/10.1057/jittc.2014.5 ohoitimur, j., krejci, j., raco, j. r., raton, y., & taroreh, f. (2019). pastoral strategic planning priorities for the church: case study of the vicariate episcopal of tonsea of the diocese manado, lewis university, illinois usa college of business. international journal of the analytic hierarchy process, 11(3), 415–434. doi: https://doi.org/10.13033 oosthuizen, a., & lategan, l. (2016). strategic management of the church as challenge and reality : can the habitat of expectation solve this dilemma? journal for christian scholarship, 277–297. raco, j., ohoitimur, j., krejci, j. v, raton, y., rottie, r., paseru, d., … rachmadi, r. (2020). the dominant factor of lectureres' research productivity using the ahp: case study of catholic university of de la salle manada. international journal of analytic hierarchy process, 12(3), 546–564. doi: https://doi.org/10.13033/ijahp.v12i3.819 saaty, t. l. (2008). decision making with the analytic hierarchy process. international journal services sciences, 1(1), 83–98. doi: http://dx.doi.org/10.1504/ijssci.2008.017590 shah, a., & david, f. (2004). does strategic planning help churches?: an exploratory study. the coastal business journal, 2(1), 28–35. stark, r., & finke, r. (2000). catholic religious’ vocations: decline and revival. review of religious research, 125-145. doi: https://doi.org/10.2307/3512525 taghavifard, m. t., amoozad mahdiraji, h., alibakhshi, a. m., zavadskas, e. k., & bausys, r. (2018). an extension of fuzzy swot analysis: an application to information technology. information, 9(3), 46. doi: https://doi.org/10.3390/info9030046 tavana, m., zareinejad, m., di caprio, d., & kaviani, m. a. (2016). an integrated intuitionistic fuzzy ahp and swot method for outsourcing reverse logistics. applied soft computing, 40, 544-557. doi: https://doi.org/10.1016/j.asoc.2015.12.005 vela-mcconnell, j. a. (2017). behind closed doors: organizational secrecy, stigma, and abuse within the catholic church. oppression and resistance: structure, agency, transformation, 48, 19à49. doi: https://doi.org/10.1108/s0163-239620170000048005 ijahp article: ohoitimur, krejci, raco, raton, jamlean, welerubun, tanod/strategic management study based on buckley’s fuzzy-ahp and swot: example of the congregation of the missionaries of the sacred heart international journal of the analytic hierarchy process 351 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.871 warner, c. m. (2019). the politics of sex abuse in sacred hierarchies: a comparative study of the catholic church and the military in the united states. religions, 10(4), 281. doi: https://doi.org/10.3390/rel10040281 wickramasinghe, v. s. k., & takano, s. e. (2009). application of combined swot and analytic hierarchy process (ahp) for tourism revival strategic marketing planning. in proceedings of the eastern asia society for transportation studies vol. 7 (the 8th international conference of eastern asia society for transportation studies, 2009) (pp. 189-189). eastern asia society for transportation studies. ijahp article: ohoitimur, krejci, raco, raton, jamlean, welerubun, tanod/strategic management study based on buckley’s fuzzy-ahp and swot: example of the congregation of the missionaries of the sacred heart international journal of the analytic hierarchy process 352 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.871 appendix model element definitions of the goal, criteria and sub-criteria. these definitions refer to the elements of the model in figure 1. 1. goal: the goal is to determine the strategic priorities of the msc using swot and fuzzy-ahp. those priorities’ factors will form the basis of designing the congregation’s future programs. the assumptions are: a. the respondents are really experts who know about the congregation and are acting as decision makers. b. the internal and external factors were shared by the experts based on the constitutions and statutes of the missionaries of the sacred heart of jesus and the results of the congregation chapter. c. information provided by the experts is true and trustworthy. whenever there were questions, experts were contacted and required to reply in writing. 2. criteria: the key criteria below will be used to decide the strengths, opportunities, weaknesses and threats. each of criteria has some sub-criteria that will be used in the assessment of strategic priorities of the msc. 2.1. strength: strength is the characteristic that adds value, advantages that make this congregation special, more effective and efficient internally than other organizations or congregations. this criterion refers to most supporting internal conditions of the congregation that inspired their ministries. there are seven subcriteria: (inspired by heart spirituality, participative and subsidiarity style of leadership, increasing number of members who are more productive, either as parish priests of as professionals, having national and international ministry supported by fraternity community living and already in service for 100 years in the diocese of manado, community based congregation, having formation centers and own property assets). 2.1.1. inspired by heart spirituality that animates all members and their activities. 2.1.2. participative and subsidiarity style of leadership 2.1.3. increasing number of members who are more productive and serve some dioceses 2.1.4. emphasis on fraternity amongst members 2.1.5. community living 2.1.6. having formation centers; own property assets 2.2. weaknesses: something that is internally owned by the msc which is more disadvantageous, negative and unfavorable when compared to something else. in other words, the aspects or activities of msc that are less effective and efficient compared to its competitors. a weakness is something an organization lacks or does poorly -in comparison to othersor a condition that puts it at a disadvantage. there are seven sub-criteria of weaknesses: ijahp article: ohoitimur, krejci, raco, raton, jamlean, welerubun, tanod/strategic management study based on buckley’s fuzzy-ahp and swot: example of the congregation of the missionaries of the sacred heart international journal of the analytic hierarchy process 353 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.871 2.2.1. the spirit of the heart that is not yet deep 2.2.2. having limited qualified leaders 2.2.3. lack of assistance to elderly members; lack of professional assistance 2.2.4. having less categorized ministry; 2.2.5. weak fighting spirit of the young members. 2.2.6. protocol to handle problems in congregation is not yet available 2.2.7. it based ministry not yet available; 2.3. opportunities: opportunity means a situation or condition (external) suitable for an activity. opportunity is an advantage and the driving force for an activity to take place. for this reason, it has a positive and favorable characteristic. for organizational management, an opportunity is the convenient time or situation that the environment presents to the organization to achieve its goals. opportunities are those that would yield positive results for the organization determined as a result of the analysis of its environment. competition and the intense work present organizations big opportunities. in fact, “opportunities are conditions in the external environment that allow an organization to take advantage of organizational strengths, overcome organizational weaknesses or neutralize environmental threats. there are seven sub-criteria of opportunities. 2.3.1. huge opportunities to strengthen heart spirituality 2.3.2. offer from other diocese abroad for msc to do pastoral service in the country 2.3.3. there some vocation to be msc brother. 2.3.4. pastoral service for family still open to msc 2.3.5. expand the charity movement of the faithful 2.3.6. improve the function and role of community 2.3.7. chances to boost revenues and assets; improve financial management; investment in credible financial institution; develop management of inheritance; intensify the fund raising 2.4. threats: a threat is a situation or condition (dari luar) that jeopardizes the actualization of an activity. it refers to a disadvantageous situation. for this reason, it has a negative characteristic that should be avoided. for organizational management, a threat is the element that makes it difficult or impossible to reach organizational goals. threats are the situations that come out as a result of the changes in the distant or immediate environment that would prevent the organization from maintaining its existence or lose its superiority in competition, and that are not favorable for the organization. four sub-criteria of threats: 2.4.1. there is no measurable strategic planning 2.4.2. modernism and challenges to celibacy especially for young people. for young people, priesthood is not a first choice; freedom inspired by modernity threatens self-identity. ijahp article: ohoitimur, krejci, raco, raton, jamlean, welerubun, tanod/strategic management study based on buckley’s fuzzy-ahp and swot: example of the congregation of the missionaries of the sacred heart international journal of the analytic hierarchy process 354 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.871 2.4.3. the number of diocesan priests is increasing and overtaking the ministry of the msc. 2.4.4. government-sponsored family planning produces small family (husband and wife plus two children). post-modern fluidity of hedonism; family sizes getting smaller; spirit of the age (instant culture) weakening the militancy struggles of young members; extensive negative publication due to clergy sex-abuse 3. s-o identified as using the internal strength to take advantage of external opportunities (ideal case). the combination of so is often called a ‘maxi-maxi’ strategy. (s1; s2; o1; o3; o4) 3.1. intensifying the important of human resources 3.2. enhance the msc structure to be dynamic and accessible to all members 4. s-t identified as ‘maxi-mini’ strategy: increase the strength to reduce the impact of external threats 4.1. dissemination of heart spirituality 4.2. empowering fraternal community 5. w-o: (mini-maxi) to minimize weaknesses by utilizing opportunities 5.1. strengthening networking 5.2. revitalizing family ministry 6. w-t: called ‘mini-mini’, aimed at reducing internal weaknesses and avoiding threats 6.1. strengthening the heart spirituality at all levels of the community ijahp news and events: our news editor in the news international journal of the analytic hierarchy process 504 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.625 our news editor in the news dr. valentina ferretti we are pleased to inform you that our news editor, dr. valentina ferretti, has been appointed as associate professor in project appraisal at the politecnico of milano, italy. in addition, valentina has been invited to continue her collaboration with the london school of economics (uk) by being appointed as a visiting fellow and lecturer for the next two years. she has previously been a visiting lecturer at the university of oxford and école centrale paris. she will be globetrotting over europe to fulfill her academic responsibilities in both institutions for the next two years. also, she again be offering her popular guardian masterclass on how to make better decisions in spring 2019 in london. kudos to our news editor! politecnico di milano (milan, italy) https://www.theguardian.com/guardian-masterclasses/2017/nov/09/how-to-make-better-decisions-lse-fellow-decision-science-dr-valentina-ferretti-life-work-business-management-lifestyle-biases-course ijahp article: akman, boyacı, kurnaz/selecting the suitable e-commerce marketplace with neutrosophic fuzzy ahp and edas methods from the seller’s perspective in the context of covid-19 international journal of the analytic hierarchy process 1 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.994 selecting the suitable e-commerce marketplace with neutrosophic fuzzy ahp and edas methods from the seller’s perspective in the context of covid-19 gülşen akman kocaeli university, turkiye akmang@kocaeli.edu.tr ali i̇hsan boyacı kocaeli university, turkiye ali.ihsan@kocaeli.edu.tr sinem kurnaz kocaeli university, turkiye snm.kurnaz@gmail.com abstract e-commerce, which is defined as making commercial transactions in an electronic environment, is becoming widespread with the increase of the use of internet and mobile devices. covid-19 has greatly changed the consumption habits of individuals, increasing interest in electronic sales channels. regardless of their size, most companies and retailers are currently looking for ways to engage their customers through electronic channels due to the effect of covid-19. in this process, the rapidly increasing trend of electronic commerce raises an important question for companies, "in which e-marketplace should we sell?” in this study, five criteria that are important in the choice of the right e-marketplace were determined and eight online alternative e-marketplaces were evaluated. the study was carried out using the neutrosophic fuzzy ahp and edas methods, which are multi-criteria decision making techniques, and a framework was established for choosing the right ecommerce marketplace for sellers. keywords: e-commerce; e-marketplace; pandemic; covid-19; multi-criteria decision making; neutrosophic fuzzy ahp; edas 1. introduction coronavirus disease (covid-19) began in china and spread all over the world in a short time causing a global health crisis in 2020. during this crisis, countries have tried to reduce the number of cases and deaths and control the social impact of the pandemic with various national and international measures. the implementation of social distancing rules, personal isolation and quarantine practices have caused sociomailto:akmang@kocaeli.edu.tr mailto:ali.ihsan@kocaeli.edu.tr mailto:snm.kurnaz@gmail.com ijahp article: akman, boyacı, kurnaz/selecting the suitable e-commerce marketplace with neutrosophic fuzzy ahp and edas methods from the seller’s perspective in the context of covid-19 international journal of the analytic hierarchy process 2 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.994 economic changes in societies around the world. one of these changes has been in the shopping habits of people from all walks of life. the covid-19 pandemic has increased online spending, impacting investments in e-commerce and digital advertising. the pandemic almost forced individuals to meet their product and service needs on the internet (erdoğan, 2020). developing technologies create innovations on a global scale. the internet, which becomes more and more a part of our lives every day, causes people's habits to change. the use of social media platforms has increased the most, followed by online video platforms and online sales channels (taşdemir, 2018). the intense use of the internet has led to changes in the purchasing habits of consumers. retailers, as well as consumers, have turned to online commerce because profit margins increase by selling at lower costs on internet platforms. these buying and selling transactions made through the internet are called e-commerce (gürpınar, 2007). e-commerce is constantly growing worldwide, and the largest share of this growth belongs to huge online marketplaces. many sellers today use existing online marketplaces because building and growing their own website is risky and time consuming. online marketplaces such as amazon and alibaba, which are increasing in volume every day, are important platforms for sellers to grow their business in ecommerce. online sellers have been known to increase their retail sales by 1250% when using these large e-marketplaces (karlson, 2021). an e-marketplace is defined as an inter-organizational information system that provides a ‘virtual space' where multiple buyers and sellers can collaborate (e.g., exchange information on product/service offerings, either generic or industryspecific, and their prices) and transact (e.g., sell and buy products/services and pay for them), very often supported by a variety of services (e.g. financial, transport, logistic, etc.) (loukis et al., 2011). the covid-19 pandemic has influenced traditional trade since the beginning of 2020. traditional trade shares have become volatile and declined sharply. in 2020, the global gross domestic product (gdp) declined by 4.3%. global commerce in products has decreased by 9%, while global trade in services has decreased by 15% (sirimanne, 2021). the covid-19 situation caused significant growth in ecommerce and a rise in the use of the internet to purchase services and products (abraham, 2021). the world trade organization (wto) stated that this is the time for e-commerce to save the global economy, and that it should intervene with vigor and vitality to demonstrate the importance and success of e-commerce in the field of e-trade and online purchasing (abdelrhim and elsayed, 2020). thus, from 2019 to 2020, the shares of e-commerce of global retail climbed from 14% to 17%. in china, for example, between august 2019 and august 2020, the online retail share climbed from 19.4% to 24.6%. over the same time period, kazakhstan's internet retail share climbed from 5% to 9.4%. between february and march 2020, the number of shopping applications downloaded in thailand climbed by 60% (sirimanne, 2021). consistent with this situation, covid-19 has been a strong motivator for each of the traditional market traders to move to internet trade in order to preserve their https://www.emerald.com/insight/content/doi/10.1108/imds-11-2020-0651/full/html#ref024 ijahp article: akman, boyacı, kurnaz/selecting the suitable e-commerce marketplace with neutrosophic fuzzy ahp and edas methods from the seller’s perspective in the context of covid-19 international journal of the analytic hierarchy process 3 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.994 remaining shares and maintain their commercial field and market success (abdelrhim and elsayed, 2020). therefore, the aim of this study is to show how sellers/retailers should select an e-marketplace to sell their products or services. an e-marketplace selection literature investigation shows that most of the studies were performed from the buyer’s viewpoint (duan et al., 2010; ozkok and pappalardo, 2013; kahraman et al., 2018; wibowo and yunianto, 2019; lubis et al., 2022; yunianto and taryadi, 2022). they explain how buyers evaluate emarketplaces and determine where to buy products. in the e-marketplace literature, there is very little research about evaluation of e-marketplaces from the seller’s perspective (schu and morschett, 2017; hidayat et al., 2021).. therefore, another aim of this study is to fill this gap by evaluating e-marketplaces from the seller’s perspective. multi-criteria decision making (mcdm) is the process of choosing the best option among alternatives by considering multiple criteria. mcdm is widely used in the literature in various fields to evaluate alternatives on the basis of criteria. the analytic hierarchy process (ahp) is a widely used method in the literature to weight evaluation criteria. the evaluation based on distance from average solution (edas) method is very useful in decision problems with conflicting criteria and is widely used in various problems in the literature such as evaluating steam boiler alternatives (kundakcı, 2019), evaluating the barriers to renewable energy adoption (asante et al., 2020), assessment of solid waste management performance (behzad et al., 2020), supplier selection (keshavarz-ghorabaee et al., 2016; karaşan and kahraman, 2017), airline evaluation (keshavarz-ghorabaee et al., 2017e), hospital selection (gündoğdu et al., 2018), inventory evaluation (ilieva et al., 2018), evaluation of bank branches (keshavarz-ghorabaee et al., 2017c), erp deployment strategy selection (erkayman et al., 2018), solid waste disposal site selection (kahraman et al., 2017), and financial performance evaluation of a food and drink index (aldalou and perçin, 2020). the edas method is used in various mcdm problems; however, we did not find any study evaluating e-marketplaces with the edas method. thus, another aim of this study is to expand the use of this method. in this study, a framework was created using neutrosophic fuzzy analytical hierarchy process (ahp) and the evaluation based on distance from average solution (edas), which are multi-criteria decision-making methods, so that retailers who want to sell via e-commerce marketplaces can choose the right e-marketplace from among the alternatives. various e-commerce websites from all over the world were evaluated according to 5 criteria and a sample application study was presented. this paper is constructed as follows; section 2 includes background information and a literature review about e-marketplace selection. section 3 contains a hybrid methodology consisting of the neutrosophic fuzzy ahp and edas methods and explains neutrosophic numbers, neutrosophic fuzzy adp and edas. section 4 includes the application steps for e-marketplace ranking and selection. section 5 discusses the findings of the study. the last section includes the conclusions which https://www.sciencedirect.com/science/article/abs/pii/s0969593117300070?casa_token=pbo1t44e6uqaaaaa:yigs_edz43adxis8dmxxe-hbw7vipmtxn-aaasjtozxvvrun82um1zptzh4qq8xq09u3adxfeso#! https://www.sciencedirect.com/science/article/abs/pii/s0969593117300070?casa_token=pbo1t44e6uqaaaaa:yigs_edz43adxis8dmxxe-hbw7vipmtxn-aaasjtozxvvrun82um1zptzh4qq8xq09u3adxfeso#! ijahp article: akman, boyacı, kurnaz/selecting the suitable e-commerce marketplace with neutrosophic fuzzy ahp and edas methods from the seller’s perspective in the context of covid-19 international journal of the analytic hierarchy process 4 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.994 explain the contributions, managerial implications, limitations, and further research ideas. 2. background and literature review 2.1. e-commerce and e-marketplaces according to the definition of the world trade organization, e-commerce is the production, advertising and distribution of goods/services over telecommunication networks (canbaz, 2007). according to ibm, e-commerce means making commercial transactions over the internet (gürpınar, 2007). the japan electronic commerce center defines e-commerce as the fulfillment of all activities such as product design, production and promotion, commercial transactions and payment of accounts over all kinds of computer networks (arslandere, 2010). while the turkish ministry of commerce defines e-commerce as commercial transactions which are made online. it can also be defined as the purchase of products or services over the internet. selling or buying anything on the internet is within the scope of e-commerce activities (ticaret, 2021). the e-commerce model, which brings together individuals who want to sell or buy products/services from all over the world, differs from traditional trade methods by eliminating the concept of space and time. the scope of electronic commerce is purchasing and selling products and services, direct marketing of products and services to consumers, online tenders, design and production, shipment of goods, contracts with institutions and businesses, advertising, promotion and information, banking transactions carried out on the internet, commercial keeping and tracking records, post-purchase support, using private and public services (i̇çigen and kutlu, 2012). e-commerce is gradually replacing classical commerce and its volume is increasing each year. figure 1 shows the top 10 countries in order of retail e-commerce sales growth. figure 2 illustrates the actual and estimated worldwide retail e-commerce sales, percent change, and percent of total retail sales from 2019 to 2025. according to this figure, it is predicted that retail e-commerce sales will reach 7385 trillion dollars in 2025. ijahp article: akman, boyacı, kurnaz/selecting the suitable e-commerce marketplace with neutrosophic fuzzy ahp and edas methods from the seller’s perspective in the context of covid-19 international journal of the analytic hierarchy process 5 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.994 figure 1 top 10 countries, ranked by retail e-commerce sales growth, 2021 (% change) (emarketer, 2021a) types of e-commerce in terms of the relationship between the parties are intercompany (b2b) e-commerce, business-to-consumer e-commerce (b2c), consumer-to-government e-commerce (b2g), business-to-government e-commerce (b2g), citizen-to-government e-commerce (c2g) and consumer-to-consumer ecommerce (c2c) (bucaklı, 2007). figure 2 retail e-commerce sales worldwide, 2019-2025 (trillions, % change, and % of total retail sales) (emarketer, 2021b). according to kollmann and lomberg (2010), an e-marketplace is a digital network that allows for electronic trade of products and/or services. this refers to the use of cutting-edge information and communication technology to support and conclude the 16.80% 17.90% 18% 18.50% 20% 20.50% 21.10% 26% 26.10% 26.80% 27% 0.00% 5.00% 10.00% 15.00% 20.00% 25.00% 30.00% worldwide us vietnan china phliphines uk mexico argentina russia brasil india 3351 4213 4921 5545 6169 6773 7385 [value] [value] [value] [value] [value] [value] [value] [value] [value] [value] [value] [value] [value] [value] -5.0% 5.0% 15.0% 25.0% 35.0% 45.0% 55.0% 65.0% 75.0% 0 1000 2000 3000 4000 5000 6000 7000 8000 2019 2020 2021 2022 2023 2024 2025 retail e-commerce sales ($) % change %of total retail sales ijahp article: akman, boyacı, kurnaz/selecting the suitable e-commerce marketplace with neutrosophic fuzzy ahp and edas methods from the seller’s perspective in the context of covid-19 international journal of the analytic hierarchy process 6 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.994 supply and demand matching process, respectively. patel (2010) defined an emarketplace as a virtual arena that allows buyers and sellers to meet and perform commercial transactions through mobile agent systems. according to tan and macaulay (2008), an e-marketplace is a web-based system that promotes and stimulates purchasing and selling in order to foster collaboration among trading partners from various industries. according to kahraman et al. (2018), emarketplaces, which are electronic spaces where vendors and buyers interact and perform various types of transactions such as buying, selling, and information exchange, are an alternative e-commerce route. e-marketplaces can be physical, virtual, or conceptual (wang & archer, 2007). while an e-marketplace performs the same functions as a physical one, digital systems are more efficient because they give more up-to-date information, a variety of support services, and simple transaction execution. 2.2. literature review in parallel with the general increasing interest, many research studies have been carried out on e-commerce. for e-market definitions and classifications, wang and archer (2007) conducted a review of the e-marketplace literature in order to clarify and explain facts concerning electronic markets that had previously been published. boyacı and baynal (2016) proposed a classification for e-commerce research and applications and conducted a literature review. in the electronic business area, fuzzy multi-criteria decision-making approaches are widely used. these are mostly concerned with analyzing website quality, assessing website usability, and tracking consumer satisfaction. because the focus of this study is on the evaluation and selection of e-marketplaces, the literature review was conducted in these areas. bhatti et al. (2020) examined the effects of the covid-19 pandemic on e-commerce trends. sharifi et al. (2006) proposed a categorization and selection methodology for e-marketplaces to improve supply chain alignment. stockdale and standing (2002) proposed a framework for the selection of electronic marketplaces via a content analysis approach. this framework supports electronic marketplace-related decision making which is based within the contexts of business drivers, internal company issues and e-marketplace facilitators. büyüközkan (2004) proposed that fuzzy logic based on multi-criteria evaluation be used to improve the efficiency of decision making for e-marketplace selection in uncertain situations. this evaluation approach incorporates both fuzzy analytic hierarchy process and fuzzy delphi methodologies. guo (2007) claimed that as integration technologies advance, organizations will utilize more private, community, and public e-marketplaces and select their appropriate type of e-marketplace when evaluating the function-cost ratios of business interoperability on different types of b2b e-marketplace. duan et al. (2010) presented a multi-criteria analysis approach for effectively evaluating and selecting the most appropriate electronic market (e-market) in electronic business by extending the topsis method. ozkok and pappalardo (2013) provided an introduction of the most well-known fuzzy ahp techniques and their applications, as well as a case https://www.emerald.com/insight/search?q=rosemary%20stockdale https://www.emerald.com/insight/search?q=craig%20standing https://www.emerald.com/insight/content/doi/10.1108/10662240210430900/full/html https://www.emerald.com/insight/content/doi/10.1108/10662240210430900/full/html ijahp article: akman, boyacı, kurnaz/selecting the suitable e-commerce marketplace with neutrosophic fuzzy ahp and edas methods from the seller’s perspective in the context of covid-19 international journal of the analytic hierarchy process 7 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.994 study for selecting an e-marketplace for a company that manufactures and sells computer electronic parts in türkiye. kolomvatsos et al. (2014) discussed the fundamental elements of the negotiation and outlined a decision-support system for sellers in e-marketplaces. they proposed a decision-making mechanism based on fuzzy logic (fl) in order to handle uncertainty in the negotiation process. schu and morschett (2017) identified, tested and explained factors influencing the foreign market selection behavior of european online retailers. they utilized a rankordered logistic regression model to see how different features affect overall judgments of market attractiveness, assuming that online retailers try to optimize the usefulness of markets for their unique interests based on the criteria they observed. kahraman et al. (2018) used a modified version of hesitant fuzzy analytic hierarchy process to model e-marketplace selection for buyers. thitimajshima et al. (2019) performed exploratory research to identify the elements that influence the functionality of third-party b2b e-marketplaces from a seller's perspective. they presented ten criteria and mapped them into the domain-specific model for emarketplaces. wibowo and yunianto (2019) conducted a study about the selection of e-marketplace with the fuzzy ahp and vikor methods. the alternative emarketplaces selected were the top three that were frequently visited in indonesia in 2019. jaikumar (2019) argued that the selection of a seller from an e-marketplace is influenced by both assimilation-contrast-related evaluations of the display price of the goods and anchoring-related evaluations of the volume of seller reviews. he found that buyers prefer high-display-price vendors with many reviews over low-displayprice sellers with few reviews. arif et al. (2020) prioritized the goods sellers alternatives in an e-marketplace by combining the vikor with the smarter methods using five selection criteria which are product price, number of products sold, seller score rating, number of reviews with five stars, and location distance. yuianto and wibowo (2020) described the e-marketplace selection approach using fuzzy ahp and the fuzzy moora approach in the alternative assessment and the fuzzy ahp method to weight the criteria kumar et al. (2021) aimed to determine registered vendors'/sellers’ experiences with online marketplaces. they defined major dimensions of the seller experience using exploratory factor analysis (efa) and confirmatory factor analysis (cfa). they found that the most important factors for sellers to evaluate e-marketplaces are “registration,”; “product listing”; “pricing autonomy”; “ease of pick-up and delivery”; “credit of receivables” and “vendor assistance.” xu et al. (2021) investigated a collaborating mode selection problem for a manufacturer who distributes its products through an offline channel and an online platform under cap-and-trade regulations. the platform can be used as a marketplace or for reselling. they looked into the manufacturer's best operational decisions and mode selection for the platform in the face of demand disruptions. hidayat et al. (2021) conducted their study to determine the e-marketplace that is suitable for use in terms of service quality, system quality, information quality, and vendor quality https://www.sciencedirect.com/science/article/abs/pii/s0969593117300070?casa_token=pbo1t44e6uqaaaaa:yigs_edz43adxis8dmxxe-hbw7vipmtxn-aaasjtozxvvrun82um1zptzh4qq8xq09u3adxfeso#! https://www.sciencedirect.com/science/article/abs/pii/s0969593117300070?casa_token=pbo1t44e6uqaaaaa:yigs_edz43adxis8dmxxe-hbw7vipmtxn-aaasjtozxvvrun82um1zptzh4qq8xq09u3adxfeso#! ijahp article: akman, boyacı, kurnaz/selecting the suitable e-commerce marketplace with neutrosophic fuzzy ahp and edas methods from the seller’s perspective in the context of covid-19 international journal of the analytic hierarchy process 8 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.994 owned by the e-marketplace. they conducted their study with ornamental plant traders and their choice of the right e-marketplace for their online business activities using the ahp method. lubis, erdiansyah and ramadhan (2022) investigated the use of decision support systems to choose an online marketplace. in this instance, they evaluated a number of websites or programs that are typically utilized by the public in online buying activities using the vikor and rank order centroid methodologies. yuianto and taryadi (2022) developed a fuzzy decision making method for selecting the best emarketplace using the integral total value method because choosing an e-marketplace has an effect on increasing income. 3. methodology every day many decisions have to be made and the decision process includes various uncertainties and hesitations. this undeniable uncertainty is widely known and accepted in the decision-making process, especially in mcdm problems. a hybrid approach including the neutrosophic fuzzy ahp and edas methods was used in this study. the research methodology is shown in figure 3. the motivation behind the use of these two methods together is that the ahp method is very useful in determining the criterion weights by compiling the opinions of various experts and the edas method is quite capable at evaluating alternatives. after deciding on the evaluation criteria for e-marketplaces based on expert opinions and a literature review, these criteria were weighted using the neutrosophic fuzzy ahp method and the pairwise evaluation matrices of the experts. next, an assessment was made for alternative e-marketplaces using the criteria’s weights and the edas method. figure 3 methodology of research 3.1. neutrosophic fuzzy ahp the ahp was developed by saaty (1980) and is a technique used by decision makers to evaluate the weight of their judgments and rank alternatives (liu et al., 2020). this ijahp article: akman, boyacı, kurnaz/selecting the suitable e-commerce marketplace with neutrosophic fuzzy ahp and edas methods from the seller’s perspective in the context of covid-19 international journal of the analytic hierarchy process 9 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.994 method is based on multiple criteria and subjective marginal inputs. as each criterion is evaluated, these inputs are converted into scores (pourghasemi et al., 2012). the ahp approach is a widely used decision-making tool that can assist decision makers in obtaining a solution based on a hierarchy of criteria and sub-criteria, with the top and bottom goals being examined as potential solutions (stević et al., 2016). despite its widespread use, the ahp technique has been criticized for its inability to deal with ambiguities and imprecision, making it difficult for decision makers to obtain a precise number. fuzzy sets only consider the membership function (truth degree) and ignore non-membership (falsity degree) and indeterminacy degrees, resulting in a failure to reflect uncertainty and indeterminacy (abdel-basset et al., 2018b). the classical ahp does not include the concept of uncertainty and the fuzzy ahp does not fully and accurately reflect the thoughts of the decision makers (abdel-basset et al., 2018a). real-world situations, on the other hand, entail a great deal of ambiguity and uncertainty, necessitating the use of fuzzy numbers. as a result, the ahp and neutrosophic sets were integrated and transformed into the neutrosophic fuzzy ahp model (nf-ahp) (aydın et al., 2019). 3.1.1. definition of a neutrosophic set although fuzzy theory is a powerful tool, it has been found to be inadequate because it only characterizes linguistic terms by degree of membership (kokoç and ersöz, 2021). a neutrosophic set is a combination of a classical set, a fuzzy set, and an intuitionistic fuzzy set that effectively models real-world situations by taking into account all aspects of a choice scenario (i.e., truthiness, indeterminacy, and falsity) (abdel-basset et al., 2008a). because membership was the symmetric equivalent of non-membership with respect to indeterminacy, the neutrosophic set served as a symmetric tool in the suggested method. some important definitions of neutrosophic sets are introduced as follows: definition 1 (wang et al., 2010; abdel-basset et al., 2008a): the neutrosophic set n is characterized by three membership functions of truth 𝑇�̃�(x), indeterminacy 𝐼�̃� (x) and falsity 𝐹�̃�(x), where x ∈ x and x are a space of points. also, 𝑇(𝑥):𝑋 → ]0 −,1+[ , (𝑥):𝑋 → 𝐼]0−,1+[ and 𝐹(𝑥):𝑋 → ]0−,1+[ . there is no restriction on the sum of t (x), i (x), and 𝐹(𝑥), therefore 0− ≤ 𝑠𝑢𝑝 𝑇(𝑥) + 𝑠𝑢𝑝 𝐼 (𝑥) + 𝑠𝑢𝑝 𝐹(𝑥) ≤ 3 (1) in order to apply the neutrosophic set theory to real-life problems, wang et al. (2010) developed a single-valued neutrosophic set theory, a subclass of neutrosophic sets. definition 2 (wang et al., 2010; abdel-basset et al., 2008a): the following equation presents the form of a single-valued neutrosophic set n over x: 𝐴 = {〈𝑥,𝑇(𝑥),𝐼(𝑥),𝐹(𝑥)〉:𝑥 ∈ 𝑋} (2) ijahp article: akman, boyacı, kurnaz/selecting the suitable e-commerce marketplace with neutrosophic fuzzy ahp and edas methods from the seller’s perspective in the context of covid-19 international journal of the analytic hierarchy process 10 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.994 where 𝑇�̃�(𝑥):𝑋 → [0,1], 𝐼�̃�(𝑥):𝑋 → [0,1] and 𝐹�̃�(𝑥):𝑋 → [0,1], with 0 ≤ 𝑇�̃�(𝑥)+ 𝐼�̃�(𝑥) + 𝐹�̃�(𝑥) ≤ 3 for all x ∈ x. the single valued neutrosophic (svn) number is symbolized by n = (𝑡�̃�, 𝑖�̃�, 𝑓�̃�), where 𝑡�̃�, 𝑖�̃�, 𝑓�̃�∈ [0,1] and 𝑡�̃� + 𝑖�̃� + 𝑓�̃� ≤ 3. definition 3 (yücesan, 2020; abdel-basset et al., 2008a): the single valued triangular neutrosophic number, �̃� = 〈(𝑎1,𝑎2,𝑎3;𝑡�̌�, 𝑖�̌�,𝑓�̌�〉, is a neutrosophic set on the real line set r, where 𝑇�̃�(𝑥):𝑋 → [0,1], 𝐼�̃�(𝑥):𝑋 → [0,1] and 𝐹�̃�(𝑥):𝑋 → [0,1] as sum of 𝑇�̃�(𝑥), 𝐼�̃�(𝑥) and 𝐹�̃�(𝑥) are between 0 and 3. truth, indeterminacy and falsity membership functions are as follows: 𝑇�̃�(𝑥) = { 𝑡�̃� ( 𝑥−𝑎1 𝑎2−𝑎1 ) (𝑎1 ≤ 𝑥 ≤ 𝑎2) 𝑡�̃� (𝑥 = 𝑎2) 𝑡�̃� ( 𝑎3−𝑥 𝑎3−𝑎2 ) (𝑎2 < 𝑥 ≤ 𝑎3 0 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 (3) 𝐼�̃�(𝑥) = { ( 𝑎2−𝑥+𝑖𝑎(𝑥−𝑎1 𝑎2−𝑎1 ) (𝑎1 ≤ 𝑥 ≤ 𝑎2) 𝑖�̃� (𝑥 = 𝑎2) ( 𝑥−𝑎2+𝑖𝑎(𝑎3−𝑥) 𝑎3−𝑎2 ) (𝑎2 < 𝑥 ≤ 𝑎3 1 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 (4) 𝐹�̃�(𝑥) = { ( 𝑎2−𝑥+𝑓𝑎(𝑥−𝑎1 𝑎2−𝑎1 ) (𝑎1 ≤ 𝑥 ≤ 𝑎2) 𝑓�̃� (𝑥 = 𝑎2) ( 𝑥−𝑎2+𝑓𝑎(𝑎3−𝑥) 𝑎3−𝑎2 ) (𝑎2 < 𝑥 ≤ 𝑎3 1 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 (5) definition 4 (yücesan, 2020; abdel-basset et al., 2008a): let �̃� = 〈(𝑎1,𝑎2,𝑎3;𝑡�̌�, 𝑖�̌�,𝑓�̌�〉 and �̃� = 〈(𝑏1,𝑏2,𝑏3;𝑡�̌�, 𝑖�̌�,𝑓�̌�〉 two single-valued triangular neutrosophic numbers and γ ≠ 0 be any real number. then, the addition of two triangular neutrosophic numbers is in eq. (6), the subtraction of two triangular neutrosophic numbers is in eq. (7), the inverse of a triangular neutrosophic number is in eq. (8), the multiplication of two triangular neutrosophic numbers is in eq. (9), and the division of two triangular neutrosophic numbers is in eq. (10), �̃� + �̃� = 〈(𝑎1 + 𝑏1,𝑎2 + 𝑏2,𝑎3 + 𝑏3);𝑡�̌�⋀𝑡�̌�, 𝑖�̌�⋁𝑖�̌�,𝑓�̌�⋁𝑓�̌�〉 (6) �̃� − �̃� = 〈(𝑎1 − 𝑏1,𝑎2 − 𝑏2,𝑎3 − 𝑏3);𝑡�̌�⋀𝑡�̌�, 𝑖�̌�⋁𝑖�̌�,𝑓�̌�⋁𝑓�̌�〉 (7) �̃�−1 = 〈( 1 𝑎3 , 1 𝑎2 , 1 𝑎1 );𝑡�̌�, 𝑖�̌�,𝑓�̌�〉 𝑤ℎ𝑒𝑟𝑒 �̃� ≠ 0 (8) ijahp article: akman, boyacı, kurnaz/selecting the suitable e-commerce marketplace with neutrosophic fuzzy ahp and edas methods from the seller’s perspective in the context of covid-19 international journal of the analytic hierarchy process 11 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.994 �̃� ∗ �̃� = { 〈(𝑎1𝑏1,𝑎2𝑏2,𝑎3𝑏3);𝑡�̌�⋀𝑡�̌�, 𝑖�̌�⋁𝑖�̌�,𝑓�̌�⋁𝑓�̌�〉 𝑖𝑓 (𝑎3 > 0,𝑏3 > 0) 〈(𝑎1𝑏3,𝑎2𝑏2,𝑎3𝑏1);𝑡�̌�⋀𝑡�̌�, 𝑖�̌�⋁𝑖�̌�,𝑓�̌�⋁𝑓�̌�〉 𝑖𝑓 (𝑎3 < 0,𝑏3 > 0) 〈(𝑎3𝑏3,𝑎2𝑏2,𝑎3𝑏3, );𝑡�̌�⋀𝑡�̌�, 𝑖�̌�⋁𝑖�̌�,𝑓�̌�⋁𝑓�̌�〉 𝑖𝑓 (𝑎3 < 0,𝑏3 < 0) (9) �̃� �̃� = { 〈( 𝑎1 𝑏1 , 𝑎2 𝑏2 , 𝑎3 𝑏3 );𝑡�̌�⋀𝑡�̌�, 𝑖�̌�⋁𝑖�̌�,𝑓�̌�⋁𝑓�̌�〉 𝑖𝑓 (𝑎3 > 0,𝑏3 > 0) 〈( 𝑎1 𝑏3 , 𝑎2 𝑏2 , 𝑎3 𝑏1 );𝑡�̌�⋀𝑡�̌�, 𝑖�̌�⋁𝑖�̌�,𝑓�̌�⋁𝑓�̌�〉 𝑖𝑓 (𝑎3 < 0,𝑏3 > 0) 〈( 𝑎1 𝑏1 , 𝑎2 𝑏2 , 𝑎3 𝑏3 );𝑡�̌�⋀𝑡�̌�, 𝑖�̌�⋁𝑖�̌�,𝑓�̌�⋁𝑓�̌�〉 𝑖𝑓 (𝑎3 < 0,𝑏3 < 0) (10) 3.1.2. neutrosophic fuzzy ahp the application steps of neutroscophic fuzzy ahp method are as follows: (yücesan, 2020; junaid et al., 2019) step 1: select a group of experts. step 2: structure the hierarchy of the problem. step 3: structure the neutrosophic pair-wise comparison matrix of factors, subfactors and alternatives, through the linguistic terms shown in table 1. table 1 linguistic terms and identical triangular neutrosophic numbers (ahmad et al., 2019) saaty scale explanation symbol neutrosophic fuzzy scale 1 equally influential ei 1̃ = 〈(1,1,1);0.50,0.50,0.50〉 3 moderately influential mi 3̃ = 〈(2,3,4);0.60,0.35,0.40〉 5 strongly influential si 5̃ = 〈(4,5,6);0.70,0.30,0.30〉 7 very strongly influential vsi 7̃ = 〈(6,7,8);0.80,0.25,0.20〉 9 absolutely influential ai 9̃ = 〈(8,9,9);0.90,0.10,0.10〉 2 sporadic values between two close scales ei-mi 2̃ = 〈(1,2,3);0.55,0.40,0.45〉 4 mi-si 4̃ = 〈(3,4,5);0.65,0.30,0.35〉 6 si-vsi 6̃ = 〈(5,6,7);0.75,0.25,0.25〉 8 vsi-ai 8̃ = 〈(7,8,9);0.80,0.25,0.20〉 according to expert judgment, the neutrosophic scale is accomplished. the neutrosophic pair-wise comparison matrices of factors will have the form shown in eq. (11). 𝐴 ̃𝑘 = [ �̃�11 𝑘 �̃�12 𝑘 … �̃�1𝑛 𝑘 ⋮ ⋱ ⋮ �̃�𝑛1 𝑘 �̃�𝑛2 𝑘 �̃�𝑛𝑛 𝑘 ] (11) ijahp article: akman, boyacı, kurnaz/selecting the suitable e-commerce marketplace with neutrosophic fuzzy ahp and edas methods from the seller’s perspective in the context of covid-19 international journal of the analytic hierarchy process 12 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.994 where �̃�𝑗𝑖 = �̃�𝑖𝑗 −1 and is the a triangular neutrosophic number that measures the decision makers vagueness. the triangular neutrosophic fuzzy number is presented as where �̃�𝑖𝑗 𝑘 is the preference relation of i th criterion over j th criterion according to k th decision maker. step 4: combine decision makers’ evaluations with the geometric mean. by having more than one decision maker in the estimation process, the aggregated �̃�𝑗𝑖 of all the decision makers is calculated as in eq. (12) for obtaining the final comprehensive preference values via taking average values of all decision makers preferences. evaluations made by more than one expert are combined with the geometric mean. for the combined neutrosophic decision matrix, the calculation is made using eq. (12) by means of eq. (9). �̃�𝑖𝑗 = √∏ 〈(𝑙𝑖𝑗 𝑘 ,𝑚𝑖𝑗 𝑘 ,𝑢𝑖𝑗 𝑘 );𝑇𝑖𝑘 𝑘 , 𝐼𝑖𝑘 𝑘 ,𝐹𝑖𝑘 𝑘〉𝐾𝑘=1 𝐾 (12) where 𝑇𝑖𝑗 = 𝑚𝑖𝑛(𝑇𝑖𝑘) 1/𝑘 ; 𝐼𝑖𝑗 = (1 − (1 − max(𝐼𝑖𝑘)) 1/𝑘 ; 𝐹𝑖𝑗 = (1 − (1 − max(𝐹𝑖𝑘)) 1/𝑘 the aggregated pair-wise comparison matrix according to the averaged preference values has the following form: �̃�𝑘 = [ �̃�11 �̃�12 … �̃�1𝑛 ⋮ ⋱ ⋮ 𝑟𝑛1 �̃�𝑛2 �̃�𝑛𝑛 ] (13) step 5: calculate the weight of the criteria, sub-criteria and alternatives from the neutrosophic pair-wise comparison matrix, by transforming it to a deterministic matrix using the following equations, eq. (15, 16). thus, crispy values of neutrosophic pairwise comparison evaluation matrix are obtained. let �̃�𝑖𝑗 = 〈(𝑎1,𝑎2,𝑎3), 𝑡𝑎, 𝑖𝑎, 𝑓𝑎〉 is a single valued triangular neutrosophic number and then score (s) of �̃�𝑖𝑗 is calculated as follows: 𝑆(�̃�𝑖𝑗) = 1 8 [𝑎1 + 𝑎2 + 𝑎3]𝑥(2 + 𝑇 − 𝐼 − 𝐹) (14) 𝑆(�̃�𝑗𝑖) = 1 𝑆(�̃�𝑖𝑗) (15) the following deterministic matrix is obtained by compensating for the score value of each triangular neutrosophic number in the neutrosophic pair-wise comparison matrix. ijahp article: akman, boyacı, kurnaz/selecting the suitable e-commerce marketplace with neutrosophic fuzzy ahp and edas methods from the seller’s perspective in the context of covid-19 international journal of the analytic hierarchy process 13 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.994 𝐴 = [ 1 𝑎12 … 𝑎1𝑛 ⋮ ⋱ ⋮ 𝑎𝑛1 𝑎𝑛2 1 ] (16) step 6: determine weights of criteria after the crispy values are found, the total value of each column is calculated and divided by the sum of the columns in which each matrix element is located. this process is done until the last matrix element. 𝑎𝑖𝑗 ′ = 𝑎𝑖𝑗 ∑ 𝑎𝑖𝑗 𝑛 𝑖=1 , 𝑖, 𝑗 = 1,2,…,𝑛 (17) in the matrix obtained by dividing by the column total, the average of each row is taken and thus neutrosophic criterion weights are found by the following equation. 𝑤𝑖 = 1 𝑛 ∑ 𝑎𝑖𝑗 ′ , 𝑖, 𝑗 = 1,2,…,𝑛𝑛𝑖=1 (18) step 7: verify the weighted matrix consistency the consistency of the matrix should be evaluated based on the expert's judgment. by dividing the consistency index (ci) by the random index (ri), the consistency can be determined. the result should be less than 0.1. the consistency index is calculated using the following equations. 𝐶𝐼 = 𝑚𝑎𝑥−n 𝑛−1 (19) 𝑚𝑎𝑥 = ( 1 𝑛 )∑ ∑ 𝑎𝑖𝑗𝑤𝑗 𝑛 𝑗=1 𝑤𝑖 𝑛 𝑖=1 (20) as shown in the eq. (21), the value of the consistency ratio (cr) is found by dividing ci by the value of ri. this is the consistency ratio of the comparison matrix. 𝐶𝑅 = 𝐶𝐼 𝑅𝐼 (21) 3.2. edas method the edas method, developed by ghorabaee et al. (2015), uses mean solutions to evaluate alternatives. in this method, two measures called positive distance from the average (pda) and negative distance from the average (nda) are considered to evaluate alternatives. an alternative with higher pda values and lower nda values is evaluated as better (kahraman et al., 2017). the algorithm of the edas method for a decision-making problem with 𝑚 criteria and 𝑛 alternatives is as follows (stanujkic et al., 2017): step 1: criteria and alternatives for decision-making problem are determined and decision matrix is constructed as follows: ijahp article: akman, boyacı, kurnaz/selecting the suitable e-commerce marketplace with neutrosophic fuzzy ahp and edas methods from the seller’s perspective in the context of covid-19 international journal of the analytic hierarchy process 14 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.994 𝑋 = [𝑋𝑖𝑗]𝑛𝑥𝑚 = [ 𝑋11 𝑋21 ⋯ 𝑋1𝑚 ⋮ ⋮ ⋮ ⋮ 𝑋𝑛1 𝑋𝑛2 ⋯ 𝑋𝑛𝑚 ] (22) step 2: average solutions (𝐴𝑉) for each criterion are calculated using the following equations. 𝐴𝑉 = [𝐴𝑉𝑗]1𝑥𝑚 , 𝑗 = 1,…,𝑚 (23) 𝐴𝑉𝑗 = (∑ xij m i=1 ) m , 𝑗 = 1,…,𝑚 (24) where 𝐴𝑉𝑗 denotes the average solution of the j th criteria. step 3: pda and nda matrices are calculated according to the type of criteria (benefit and cost). 𝑃𝐷𝐴 = [𝑃𝐷𝐴𝑖𝑗]𝑛𝑥𝑚 (25) 𝑁𝐷𝐴 = [𝑁𝐷𝐴𝑖𝑗]𝑛𝑥𝑚 (26) 𝑃𝐷𝐴𝑖𝑗 = { max (0,(𝑋𝑖𝑗−𝐴𝑉𝑗) 𝐴𝑉𝑗 , 𝑗 ∈ 𝐶𝑚𝑎𝑥 max (0,(𝐴𝑉𝑗−𝑋𝑖𝑗) 𝐴𝑉𝑗 , 𝑗 ∈ 𝐶𝑚𝑖𝑛 (27) 𝑁𝐷𝐴𝑖𝑗 = { max (0,(𝐴𝑉𝑗−𝑋𝑖𝑗) 𝐴𝑉𝑗 , 𝑗 ∈ 𝐶𝑚𝑎𝑥 max (0,(𝑋𝑖𝑗−𝐴𝑉𝑗) 𝐴𝑉𝑗 , 𝑗 ∈ 𝐶𝑚𝑖𝑛 (28) where 𝐶𝑚𝑎𝑥 and 𝐶𝑚𝑖𝑛 denote the benefit and cost criteria sets respectively step 4: sum of weighted pda and nda for all alternatives are calculated using the following equations. 𝑆𝑃𝑖 = ∑ 𝑤𝑗𝑃𝐷𝐴𝑖𝑗 𝑚 𝑗=1 (29) 𝑆𝑁𝑖 = ∑ 𝑤𝑗𝑁𝐷𝐴𝑖𝑗 𝑚 𝑗=1 (30) where 𝑊 = (𝑤1,𝑤2,…,𝑤𝑗) denotes the vector of weights. step 5: the sp and sn values for all alternatives are normalized using eq. (31) and eq. (32), respectively. ijahp article: akman, boyacı, kurnaz/selecting the suitable e-commerce marketplace with neutrosophic fuzzy ahp and edas methods from the seller’s perspective in the context of covid-19 international journal of the analytic hierarchy process 15 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.994 𝑁𝑆𝑃𝑖 = 𝑆𝑃𝑖 max (𝑆𝑃𝑖) (31) 𝑁𝑆𝑁𝑖 = 1 − 𝑆𝑁𝑖 max(𝑆𝑁𝑖) (32) step 6: assessment scores (as) for all alternatives are calculated using eq. (33). here, as values take values between 0 and 1 (0 ≤ 𝐴𝑆𝑖 ≤ 1). 𝐴𝑆𝑖 = 1 2 (𝑁𝑆𝑃𝑖 + 𝑁𝑆𝑁𝑖) (33) step 7: the alternatives are sorted in ascending order according to the as obtained. the alternative with the highest as value is the best among the other alternatives. 4. case study this study is performed in a turkish sme in a woman’s apparel industry that desires to join an online marketplace abroad to strengthen its position in the market. 4.1. determination of criteria as in classical commerce, it is very important to determine the market potential, understand consumer trends and pricing, and determine the right e-marketplace in national or international marketing studies in e-commerce. correctly identifying target customers and the e-marketplace are key to successful marketing. for this reason, it is very important to choose an e-marketplace using an analytical approach by evaluating the e-marketplaces in terms of various criteria. online marketplaces, which are increasing in size every day, are very useful platforms for sellers. instead of dealing with building and growing their own websites, many retailers are making successful sales and increasing their business volume through these e-marketplaces. however, in order to achieve these successes, it is important to choose the right e-marketplace on the basis of various criteria. each online marketplace has its own unique requirements, product categories, listing fees, and audiences. different strategies should be researched and analyzed in order to sell in e-marketplaces, and it is necessary to determine in advance which strategies can provide the most profit for the products to be sold (bigcommerce, 2021). in order to determine the right e-marketplaces in e-commerce, sellers should seek answers to the following questions and make a comprehensive evaluation: (1) what is the popularity of the chosen online marketplace in the world? (2) what policies does the e-marketplace apply to sellers? (3) what is the average number of visits and traffic to the e-marketplace? (4) what are the commission rates that the e-marketplace charges sellers? (5) what is the total business volume of the e-marketplace? an effective and systematic evaluation of these issues in the selection of the emarketplace will increase the market share of the company's products in the national and international market and contribute positively to profitability (yıldız, 2019). ijahp article: akman, boyacı, kurnaz/selecting the suitable e-commerce marketplace with neutrosophic fuzzy ahp and edas methods from the seller’s perspective in the context of covid-19 international journal of the analytic hierarchy process 16 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.994 according to kahraman et al. (2018), the most important factors affecting users selection of e-marketplaces are the number of buyers on the marketplace, shipping options, and fees. in this context, an evaluation model consisting of 2 main criteria and 8 sub-criteria was created by experts to choose the right e-marketplace. eight different emarketplaces from europe, asia and north america were evaluated with this model. as a result of the literature review and brainstorming with the expert team, the two main criteria were determined to be (1) general features and (2) sellers’ specific features. the first main criterion includes four sub-criteria which are the average number of visits, number of members, e-marketplace annual revenue and popularity. the second main criterion includes four sub-criteria which are commission rate, policies applied to sellers, supported channel features and monthly payment. general features are adapted from the literature as seen in table 2. first, the two sub-criteria of the sellers’ specific features are adapted from the literature and the last two subcriteria were developed by the author after investigating the webpages of online markets such as amazon and trendyol to see what they provide to sellers and what they require from sellers. the evaluation criteria are presented in table 2. 4.2. determination of alternative e-marketplaces some important e-marketplaces around the world were selected as alternatives to be evaluated in the case study. these e-marketplaces are bol, allegro, and trendyol from the european continent; alibaba, flipkart, rakuten from the asian continent; and amazon and etsy from the americas. amazon: amazon.com is an international e-commerce company that offers online retail, computing services, consumer electronics, digital content, as well as other local services such as daily deals and groceries. amazon is the leading us e-retailer and emarketplace with close to $386 billion in net sales in 2020 according to recent industry figures. due to amazon’s global reach, it is also recognized as one of the most valuable brands worldwide (statista, 2021; amazon, 2021). etsy: etsy is an online marketplace that acts as an intermediary between customers and artists, artisans of handmade goods, or collectors of vintage items. items sold and purchased through etsy range from clothing, jewelry and other decorative objects to arts and crafts supplies. after the launch of etsy, the company had almost immediate success, reaching one million sales within a few years. the parent company of the platform generated revenue of approximately $1.72 billion in 2020. this corresponds to a growth of approximately 110% (statista, 2021; etsy, 2021). alibaba: alibaba group is china’s leading online commerce platform, providing a wide range of b2b, b2c and c2c e-commerce services, mobile payments and logistics services. alibaba.com was founded by jack ma and his team in april 1999 and is one of the ten most valuable companies worldwide. for the fiscal year ending in march 31, 2020, the alibaba group had an annual revenue of approximately us $72 billion and a net income of approximately us $19.6 billion (statista, 2021; alibaba, 2021). ijahp article: akman, boyacı, kurnaz/selecting the suitable e-commerce marketplace with neutrosophic fuzzy ahp and edas methods from the seller’s perspective in the context of covid-19 international journal of the analytic hierarchy process 17 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.994 table 2 evaluation criteria criteria explanation resources general factors average number of visits average number of customers who visit each e-marketplace in a month kahraman et al. (2018), yıldız (2019) number of members number of companies that sell their products in each e-marketplace kahraman et al. (2018), yıldız (2019), kolomvatsos (2014) e-marketplace annual revenue total sales of each e-marketplace within a year yıldız (2019) popularity preferred by customers and offers a convenient way to compare prices and products from a single source. yıldız (2019), büyüközkan (2004), kahraman et al. (2018) factors for sellers commission rate user is charged a fee for each transaction. this by far the most popular online marketplace business model. when the customer pays the supplier, the marketplace charges a percentage or a fixed fee for its services. kolomvatsos et al. (2014), kahraman et al. (2018) policies applied to sellers negotiation policies, customer’s tactics and coalition tools as a value-added services in e-marketplaces. yıldız (2019) supported channel features services such as product offerings (ean matching), automatic relationships, product content, pick-up point delivery, product status import, orders, cancellations, shipments (including carrier mapping), returns (channel and merchant), external fulfillment (lvb/fbb), repricing, fee reductions, product variations (sizes/colors) provided by e-marketplaces defined by authors monthly payment monthly fee which sellers have to pay to use e-marketplace defined by authors flipkart: flipkart is an indian e-commerce company headquartered in bangalore, karnataka, india and incorporated as a private limited company in singapore. the company initially focused on online book sales before moving into other product categories such as consumer electronics, fashion, home supplies, groceries, and lifestyle products. flipkart, along with amazon india, is one of india’s largest online retailers and marketplaces (statista, 2021; flipkart, 2021). bol: bol.com is a web-shop based in the netherlands and offers general merchandise in categories such as music, movies, electronics, toys, jewelry, watches, baby products, gardening and diy. the store serves 11 million active customers in the netherlands and belgium as of 2020 and offers more than 23 million products. since ijahp article: akman, boyacı, kurnaz/selecting the suitable e-commerce marketplace with neutrosophic fuzzy ahp and edas methods from the seller’s perspective in the context of covid-19 international journal of the analytic hierarchy process 18 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.994 2011, bol.com has also opened its web-shop for retailers to sell, effectively becoming a platform. since the launch of “sell through bol.com”, more than 30,000 retailers have sold on the site (emarketer, 2021; bol, 2021). allegro: allegro was founded in 1999 as an online auction website. it has been part of the renowned south african e-commerce group naspers for years, but in october 2016 naspers sold it to an investor funds alliance. more than 125,000 merchants and retailers from smes have registered on the site to sell a total of over 1 million products. they are required to pay a listing fee, sales commission, and a percentage of all commissions based on the number of successful sales. in 2020, the polish emarketplace allegro generated a net revenue of approximately $1.07 million, up 54.2% compared to the same period the previous year (statista, 2021; allegro, 2021). rakuten: rakuten is a multinational e-commerce company headquartered in tokyo, japan. it is a company founded in 1997 with the launch of the business-to-consumer (b2c) e-marketplace rakuten ichiba. through its worldwide operating subsidiaries, rakuten is involved in communications services, fintech and video distribution services, among others. the groups company’s continued growth and successful efforts to expand its business portfolio are reflected in a decade-long upward trend in consolidated net income (statista, 2021; rakuten, 2021). trendyol: trendyol, founded by demet mutlu in 2010, is türkiye’s largest ecommerce platform and is headquartered in istanbul. as the company grew, it expanded its products and services and switched to the market model. it offers customers more than 300 million products each year covering many categories such as fashion, electronics, home and furniture, food, mother and baby, and cosmetics. the platform’s mobile apps have more than 15 million customers every year, an average of 180 million mobile visits per month, and 45 million downloads so far. trendyol, which defines itself as a technology company, uses the technology it has developed to digitize shopping (bigcommerce, 2021; trendyol, 2021). 4.3. determination of criteria weights three experts working in e-marketplaces were identified to perform pairwise evaluations. the characteristics of experts are presented in table 3. ijahp article: akman, boyacı, kurnaz/selecting the suitable e-commerce marketplace with neutrosophic fuzzy ahp and edas methods from the seller’s perspective in the context of covid-19 international journal of the analytic hierarchy process 19 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.994 table 3 characteristics of experts/decision makers expert sex education work experience in e-commerce sector position expert 1 (dm1) male bachelor;s degree 5 years e-commerce product manager of a company selling products on ecommerce web site expert 2 (dm2) male master’s degree 8 years e-commerce supplier manager expert 3 (dm3) female master’s degree 10 years e-commerce owner of an e-commerce site the ahp hierarchy structure was created for the appropriate e-commerce marketplace selection for suppliers. the hierarchy is shown in figure 4. in the hierarchy structure, there are goals, criteria and alternatives, respectively, from top to bottom. figure 4 ahp hierarchy structure for e-commerce marketplace selection selection of the best e marketplace to sell products average number of visits number of members bol commissio n rates popularity emarketplace annual revenue allegro flipkart alibaba amazon rakuten etsy policies applied to seller monthly payment supported channel features general features of emarketplace features of emarketplace related with sellers trendyol ijahp article: akman, boyacı, kurnaz/selecting the suitable e-commerce marketplace with neutrosophic fuzzy ahp and edas methods from the seller’s perspective in the context of covid-19 international journal of the analytic hierarchy process 20 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.994 in the ahp method, the criteria are evaluated by experts and their importance levels are scored relative to each other. the evaluation results performed by three different decision makers (dms) are shown in tables 4, 5 and 6 as pairwise comparison matrices of importance. table 4 linguistic pairwise comparison matrix of main criteria decision makers features of emarketplace related with sellers general features of e-marketplace features of emarketplace related with sellers dm1 ei mi-si a dm2 ei si-vsi dm3 ei mi general features of a e-marketplace dm1 ei b dm2 ei dm3 ei table 5 linguistic pairwise comparison matrix for features of e-marketplace related with sellers commission rates policies applied to seller supported channel features monthly payment a1 a2 a3 a4 commission rates dm1 ei si mi-si ei a1 dm2 ei ei-mi mi mi dm3 ei mi 1/mi mi policies applied to seller dm1 ei ei 1/mi a2 dm2 ei ei 1/mi-si dm3 ei 1/mi 1/si supported channel features dm1 ei 1/mi a3 dm2 ei 1/si dm3 ei 1/mi-si monthly payment dm1 ei a4 dm2 ei dm3 ei ijahp article: akman, boyacı, kurnaz/selecting the suitable e-commerce marketplace with neutrosophic fuzzy ahp and edas methods from the seller’s perspective in the context of covid-19 international journal of the analytic hierarchy process 21 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.994 table 6 linguistic pairwise comparison matrix for general features of a e-marketplace average number of visits number of members e-marketplace annual revenue popularity b1 b2 b3 b4 average number of visits dm1 ei si 1/mi-si ei b1 dm2 ei mi-si 1/mi-si 1/mi dm3 ei mi 1/mi 1/ei-mi number of members dm1 ei 1/si 1/mi b2 dm2 ei 1/si-vsi 1/mi-si dm3 ei 1/si 1/si e-marketplace annual revenue dm1 ei ei b3 dm2 ei mi dm3 ei mi popularity dm1 ei b4 dm2 ei dm3 ei linguistic pairwise matrices were transformed to a neutrosophic fuzzy pairwise evaluation matrix with the neutrosophic fuzzy numbers in table 1. the geometric average of the expert evaluations was computed, and then aggregated evaluation matrices for the main criteria and sub-criteria are provided in tables 7, 8 and 9. table 7 aggregated neutrosophic pairwise evaluation matrix for main criteria a b a1 a2 a3 ta ia fa b1 b2 b3 tb ib fb a 1.00 1.00 1.00 0.50 0.50 0.50 5.65 6.65 7.65 0.78 0.25 0.22 b 1.00 1.00 1.00 0.50 0.50 0.50 the neutrosophic values in tables 7, 8 and 9 were transformed to crisp values using equation 13 as seen in tables 10, 11 and 12. similarly, the neutrosophic values in table 7 were transformed to crisp values as seen in tables 9, 10 and 11. ijahp article: akman, boyacı, kurnaz/selecting the suitable e-commerce marketplace with neutrosophic fuzzy ahp and edas methods from the seller’s perspective in the context of covid-19 international journal of the analytic hierarchy process 22 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.994 table 8 aggregated evaluation matrix for features of e-marketplace related with sellers policies applied to seller supported channel features monthly payment to e marketplace average profitability rate of seller a1 a2 a3 ta ia fa b1 b2 b3 tb ib fb c1 c2 c3 tc ic fc d1 d2 d3 td ld fd a1 1.00 1.00 1.00 0.50 0.50 0.50 2.52 3.56 4.58 0.63 0.33 0.36 0.53 0.69 0.94 0.49 0.35 0.49 1.59 2.62 3.63 0.53 0.38 0.46 a2 1.00 1.00 1.00 0.50 0.50 0.50 1.26 1.82 2.29 0.55 0.41 0.45 0.22 0.28 0.38 0.35 0.32 0.65 a3 1.00 1.00 1.00 0.50 0.50 0.50 0.44 0.53 0.69 0.42 0.33 0.57 a4 1.00 1.00 1.00 0.50 0.50 0.50 table 9 aggregated neutrosophic pairwise evaluation matrix for general features of e-marketplace average number of visits number of members e-marketplace annual revenue popularity a1 a2 a3 ta ia fa b1 b2 b3 tb ib fb c1 c2 c3 tc ic fc d1 d2 d3 td ld fd b1 1.00 1.00 1.00 0.50 0.50 0.50 2.52 3.56 4.58 0.63 0.00 0.36 0.23 0.30 0.44 0.38 0.42 0.62 0.48 0.63 1.00 0.47 0.43 0.53 b2 1.00 1.00 1.00 0.50 0.50 0.50 0.17 0.20 0.25 0.30 0.30 0.70 0.19 0.24 0.31 0.33 0.32 0.66 b3 1.00 1.00 1.00 0.50 0.50 0.50 0.87 1.22 1.59 0.48 0.33 0.48 b4 1.00 1.00 1.00 0.50 0.50 0.50 ijahp article: akman, boyacı, kurnaz/selecting the suitable e-commerce marketplace with neutrosophic fuzzy ahp and edas methods from the seller’s perspective in the context of covid-19 international journal of the analytic hierarchy process 23 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.994 table 9 crisp values of neutrosophic numbers in table 7 a b features of e-marketplace related with sellers a 0.56 3.24 general features of a emarketplace b 0.31 0.67 sub 0.87 3.90 table 10 crisp values of neutrosophic numbers in table 8 commission rates policies applied to seller supported channel features monthly payment commission rates a1 0.563 2.578 0.448 1.586 policies applied to seller a2 0.388 0.563 1.132 0.184 supported channel features a3 2.233 0.883 0.563 0.346 monthly payment a4 0.631 5.446 2.887 0.563 sum 9.53 21.54 2.14 1.85 table 11 crisp values of neutrosophic numbers in table 9 average number of visits number of members e-marketplace annual revenue popularity average number of visits b1 0.563 3.021 0.164 0.396 number of members b2 0.331 0.563 0.100 0.125 e-marketplace annual revenue b3 6.110 9.979 0.563 0.764 popularity b4 2.523 7.980 1.309 0.563 sum 3.81 9.47 5.03 2.68 the total value of each column is calculated and then each matrix element is divided by the sum of the columns as seen in tables 9, 10 and 11. average values of each line were calculated and criteria weights were obtained as shown in tables 12, 13 and 14. as seen in table 12, the most important criterion among the main criteria is features of e-marketplace related with sellers. ijahp article: akman, boyacı, kurnaz/selecting the suitable e-commerce marketplace with neutrosophic fuzzy ahp and edas methods from the seller’s perspective in the context of covid-19 international journal of the analytic hierarchy process 24 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.994 table 12 weights of main criteria a b weights features of e-marketplace related with sellers a 0.65 0.83 0.740 general features of a emarketplace b 0.35 0.17 0.260 1.00 as seen in table 13, the most important criterion among features of the emarketplace related to sellers is monthly payment. this criterion is followed by commission rates, supported channel features and policies applied to sellers’ criteria respectively. table 13 weights of sub criteria for features of e-marketplace related with sellers commission rates policies applied to seller supported channel features monthly payment weights commission rates 0.147 0.272 0.089 0.592 0.275 policies applied to seller 0.102 0.059 0.225 0.069 0.114 supported channel features 0.585 0.093 0.112 0.129 0.230 monthly payment 0.165 0.575 0.574 0.210 0.381 1.000 as seen in table 14, the most important criterion among general features of the emarketplace is e-market annual revenue. this criterion is followed by popularity, average number of visits and number of members, respectively. table 14 weights of sub-criteria for general features of e-marketplace average number of visits number of members e-marketplace annual revenue popularity weights average number of visits 0.059 0.140 0.077 0.214 0.123 number of members 0.035 0.026 0.047 0.068 0.044 e-marketplace annual revenue 0.641 0.463 0.263 0.413 0.445 popularity 0.265 0.370 0.613 0.304 0.388 sum 1.000 lastly, the weights of the sub-criteria are multiplied by the weights of main criteria, and the global weights of all criteria are obtained as shown in table 15. ijahp article: akman, boyacı, kurnaz/selecting the suitable e-commerce marketplace with neutrosophic fuzzy ahp and edas methods from the seller’s perspective in the context of covid-19 international journal of the analytic hierarchy process 25 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.994 table 15 weights of criteria main criteria weights sub-criteria weights global weights features of emarketplace related with sellers 0.740 commission rates a1 0.275 0.204 policies applied to seller a2 0.114 0.084 supported channel features a3 0.230 0.170 monthly payment a4 0.381 0.282 general features of emarketplace 0.260 average number of visits b1 0.123 0.032 number of members b2 0.044 0.011 e-marketplace annual revenue b3 0.445 0.116 popularity b4 0.388 0.101 4.4 evaluation of e-marketplace alternatives after determining the weights of the criteria with the ahp method, the eight big ecommerce websites operating as retail e-marketplaces around the world were evaluated using the edas method on the basis of these weighted criteria. the decision matrix seen in table 16 was created by data provided from the webpages of the e-marketplaces (statista, 2021; amazon, 2021; etsy, 2021; alibaba, 2021; flipkart, 2021; bol, 2021; allegro, 2021; rakuten, 2021; trendyol, 2021). the average values (av) of the criteria were calculated using equation 9. table 16 data of alternative e-marketplaces on the basis of each criterion criteria a1 a2 a3 a4 b1 b2 b3 b4 min max max min max max max max bol 15 6 11 0 80,630,000 13,000,000 5,179,780,000 366 allegro 10 7 7 300 197,390,000 21,000,000 849,000,000 114 etsy 25 7 15 6 405,640,000 90,600,000 1,724,000,000 61 amazon 15 9 19 39.99 2,550,000,000 300,000,000 386,000,000,000 12 alibaba 4 8 13 208.33 117,400,000 1,180,000,000 71,985,000,000 293 flipkart 18 6 8 0 176,930,000 100,000,000 5,914,160,994 163 rakuten 13 8 8 39 580,080,000 111,000,000 13,800,000,000 42 trendyol 18 7 8 0 221,430,000 19,300,000 791,800,000 120 av 15 7 11 74 541,187,500 229,362,500 60,780,467,624 146 two matrices were calculated as positive distance to mean (pda) matrix and negative distance to mean (nda) matrix. the pda and nda matrices are presented in table 17 and table 18, respectively. https://www.edesk.com/blog/selling-allegro-europes-5th-largest-marketplace/ ijahp article: akman, boyacı, kurnaz/selecting the suitable e-commerce marketplace with neutrosophic fuzzy ahp and edas methods from the seller’s perspective in the context of covid-19 international journal of the analytic hierarchy process 26 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.994 table 17 pda matrix a1 a2 a3 a4 b1 b2 b3 b4 bol 0 0 0 1 0 0 0 1.5004 allegro 0.3220 . 0 0 0 0 0 0 etsy 0 0 0.3483 0.9191 0 0 0 0 amazon 0 0.2115 0.7079 0.4608 3.7119 0.3079 5.3507 0 alibaba 0.7288 0.0769 0.1685 0 0 4.1447 0.1843 1.0017 flipkart 0 0 0 1 0 0 0 0.1136 rakuten 0.1186 0.0769 0 0.474 0.072 0 0 0 trendyol 0 0 0 1 0 0 0 0 weights 0.2037 0.0841 0.1702 0.2820 0.0319 0.0114 0.1158 0.1009 table 18 nda matrix a1 a2 a3 a4 b1 b2 b3 b4 bol 0.0169 0.0000 0.0112 0 0.8510 0.9433 0.9148 0 allegro 0 0.3220 0.3708 3.0450 0.6353 0.9084 0.9860 0.2212 etsy 0.6949 0 0 0 0.2505 0.6050 0.9716 0.5833 amazon 0.0169 0 0 0 0 0 0 0.9180 alibaba 0 0.7288 0 1.8090 0.7831 0 0 0 flipkart 0.2203 0 0.2809 0 0.6731 0.5640 0.9027 0 rakuten 0 0.1186 0.2809 0 0 0.5160 0.7730 0.7131 trendyol 0.2203 0 0.2809 0 0.590844 0.9159 0.9870 0.1802 weights 0.2037 0.0841 0.1702 0.2820 0.0319 0.0114 0.1158 0.1009 the pda and nda matrices were multiplied by the weights in table 15 obtained by the neutrosophic fuzzy ahp method, and then the weighted nda matrix and weighted pda matrices were obtained as shown in table 19 and table 20, respectively. the weighted sum of the positive distances from the mean (sp) and the weighted sum of the negative distances from the mean (sn) values were obtained for each alternative by row sums from the weighted pda and weighted nda matrices given in table 19 and table 20, respectively. ijahp article: akman, boyacı, kurnaz/selecting the suitable e-commerce marketplace with neutrosophic fuzzy ahp and edas methods from the seller’s perspective in the context of covid-19 international journal of the analytic hierarchy process 27 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.994 table 19 weighted pda matrix a1 a2 a3 a4 b1 b2 b3 b4 total bol 0 0 0 0.2820 0 0 0 0.1514 0.4335 allegro 0.0656 0 0 0 0 0 0 0 0.0656 etsy 0 0 0.0593 0.2592 0 0 0 0 0.3185 amazon 0 0.0178 0.1205 0.1300 0.1183 0.0035 0.6195 0 1.0096 alibaba 0.1484 0.0065 0.0287 0 0 0.0473 0.0213 0.1011 0.3533 flipkart 0 0 0 0.2820 0 0 0 0.0115 0.2935 rakuten 0.0242 0.0065 0 0.1337 0.0023 0 0 0 0.1667 trendyol 0 0 0 0.2820 0 0 0 0 0.2820 table 20 weighted nda matrix a1 a2 a3 a4 b1 b2 b3 b4 total bol 0.0035 0 0.0019 0 0.0271 0.0108 .1059 0 0.1492 allegro 0 0.0271 0.0631 0.8588 0.0203 0.0104 0.1142 0.0223 1.1161 etsy 0.1415 0 0 0 0.0080 0.0069 0.1125 0.0589 0.3278 amazon 0.0035 0 0 0 0 0 0 0.0926 0.0961 alibaba 0 0.0613 0 0.5102 0.0250 0 0.5965 flipkart 0.0449 0 0.0478 0 0.0215 0.0064 0.1045 0 0.2251 rakuten 0 0.0100 0.0478 0 0 0.0059 0.0895 0.0720 0,2251 trendyol 0.0449 0 0.0478 0 0.0188 0.0105 0.1143 0.0182 0,2544 as seen in table 21, normalized sp (nsp) and normalized np (nsn) values were obtained by normalizing sp and sn values. then, the averages of these values were taken and the evaluation scores (as) were obtained for each alternative. table 21 edas results alternatives sp sn nsp nsn as rank bol 0.4335 0.1492 0.4293 0.8663 0.6478 2 allegro 0.0656 1.1161 0.0650 0.0000 0.0325 8 etsy 0.3185 0.3278 0.3155 0.7063 0.5109 5 amazon 1.0096 0.0961 1.0000 0.9139 0.9569 1 alibaba 0.3533 0.5965 0.3500 0.4656 0.4078 7 flipkart 0.2935 0.2251 0.2907 0.7983 0.5445 3 rakuten 0.1667 0.2251 0.1651 0.7983 0.4817 6 trendyol 0.2820 0.2544 0.2794 0.7720 0.5257 4 the rank values formed by ordering the as values from largest to smallest are also shown in table 20. accordingly, amazon is the most preferred e-marketplace with ijahp article: akman, boyacı, kurnaz/selecting the suitable e-commerce marketplace with neutrosophic fuzzy ahp and edas methods from the seller’s perspective in the context of covid-19 international journal of the analytic hierarchy process 28 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.994 the current criteria and weights. it is followed by bol, flipkart, trendyol, etsy, rakuten, alibaba and allegro, respectively. 5. discussion in this study, a framework was created for sellers to determine the right emarketplace for selling their products. after a literature review and experts’ assesments were conducted, two main-criteria and eight sub-criteria were determined to be the evaluation criteria, and these criteria were weighted using the neutrosophic fuzzy ahp method. using these weighted criteria and the edas method, eight online marketplaces operating worldwide were determined as alternatives including bol, allegro, trendyol, alibaba, flipkart, rakuten, amazon and etsy. then, using weighted criteria from the neutrosophic fuzzy ahp, these alternatives were evaluated using the edas method. as a result of the study, the most important three criteria for an e-marketplace selection problem are monthly payment demanded by the emarketplace, commission rates and supported channel features, respectively. another result of the study is that among the eight e-marketplaces, the most preferred emarketplace is amazon, followed by bol, flipkart, trendyol, etsy, rakuten, alibaba and allegro, respectively. when the literature was reviewed, most of the studies about e-marketplace evaluation and selection had been performed from the viewpoint of buyers. for the buyers, the most appropriate e-marketplace for purchasing a product via online shopping was determined (wibowo and yunianto, 2019; lubis et al., 2022; guo, 2007). there is very little research about the e-marketplace selection problem from the seller’s perspective (schu and morschett, 2017; hidayat et al., 2021). this study can fill that gap. when the edas literature was reviewed, no studies were found that used the edas method for the e-market selection problem. this study expands the usage of edas. the e-marketplace selection problem is a mcdm problem. to solve this problem, some mcdm techniques are used; for example, ahp (hidayat et al., 2021), fuzzy ahp (büyüközkan, 2004; ozkok and pappalardo, 2013), modified hesitant fuzzy ahp (kahraman et al., 2018), fuzzy ahp and vikor (wibowo and yunianto, 2019), fuzzy ahp and fuzzy moora (yuianto and wibowo, 2020), extended topsis (duan et al., 2010), vikor and smarter (arif et al., 2020), vikor and rank order centroid (lubis et al., 2022). this study is one of the first studies to use the integrated neutrosophic ahp and edas methods to evaluate e-marketplaces. 6. conclusion the internet has now become an indispensable part of our lives and its audience is growing every day. with the widespread use of the internet, the consumption and purchasing habits of individuals have also changed. individuals are increasingly turning to online commerce for almost all their needs. e-commerce, which has https://www.sciencedirect.com/science/article/abs/pii/s0969593117300070?casa_token=pbo1t44e6uqaaaaa:yigs_edz43adxis8dmxxe-hbw7vipmtxn-aaasjtozxvvrun82um1zptzh4qq8xq09u3adxfeso#! https://www.sciencedirect.com/science/article/abs/pii/s0969593117300070?casa_token=pbo1t44e6uqaaaaa:yigs_edz43adxis8dmxxe-hbw7vipmtxn-aaasjtozxvvrun82um1zptzh4qq8xq09u3adxfeso#! ijahp article: akman, boyacı, kurnaz/selecting the suitable e-commerce marketplace with neutrosophic fuzzy ahp and edas methods from the seller’s perspective in the context of covid-19 international journal of the analytic hierarchy process 29 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.994 accelerated its already increasing popularity with the effects of the covid-19 epidemic, has become indispensable for many businesses and individuals. e-commerce platforms, which destroy the concept of place and time, bring together sellers and customers, creating a wide e-marketplace, that is, a platform. e-commerce marketplaces, which allow sellers to grow their reach worldwide, also offer customers the opportunity to obtain their needs from all over the world. it is very important for sellers to determine the right e-marketplace from among many e-commerce marketplaces, to reach their target market and increase their profitability. this study contributes to the literature by creating a practical model to explain the preference of e-marketplaces from the seller’s perspective. it determines important factors in the e-marketplace literature that make them preferred. these factors will add to discussions on how and to what extent the features of e-marketplaces affect their preference by sellers. these findings have implications for practitioners. this study can be a guide for sellers who decide to move their selling activities online by helping them evaluate alternatives of online marketplaces and choose the most appropriate from among them to sell their products. sellers can increase their profitability by evaluating the criteria according to their own priorities and choosing the most suitable e-marketplace for them. this study can provide a scientific method for decision makers in companies who want to move their selling activities online. another implication of the study is for e-marketplaces. from this study, e-marketplaces can understand which factors are important for sellers to perform in an online marketplace, and they can develop their channel features to be attractive for possible new sellers. this study has several limitations. first, the respondents were limited to turkish companies who want to sell their products in e-commerce marketplaces. the results cannot be generalized for other countries because they have different cultures or industry systems. for future studies, to generalize results, analyses can be performed in different countries. mostly, the ahp and its different configurations such as fuzzy ahp, hesitant fuzzy ahp are used to weight evaluation criteria for e-market selection problem. in future studies, different weighted methods such as smart or critic can be used. to evaluate alternatives, other mcdm techniques such as additive ratio assessment (aras), brown-gibson model (bgm), best-worse method (bwm), or deep ranking analysis by power eigenvectors (drape) can be used to extend the use of these new techniques. ijahp article: akman, boyacı, kurnaz/selecting the suitable e-commerce marketplace with neutrosophic fuzzy ahp and edas methods from the seller’s perspective in the context of covid-19 international journal of the analytic hierarchy process 30 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.994 references abdel-basset m., mohamed m., & sangaiah a.k. (2018a). neutrosophic ahpdelphi group decision making model based on trapezoidal neutrosophic numbers. journal of ambient intelligence and humanized computing, 9, 1427-1443. doi: https://doi.org/10.1007/s12652-017-0548-7 abdel-basset, m., mohamed, m., & smarandache, f. (2018b). an extension of neutrosophic ahp–swot analysis for strategic planning and decisionmaking. symmetry, 10(4), 116. doi: https://doi.org/10.3390/sym10040116 abdelrhim, m. and elsayed, a. (2020). the effect of covid-19 spread on the ecommerce market: the case of the 5 largest e-commerce companies in the world (june 7, 2020). doi: http://dx.doi.org/10.2139/ssrn.3621166 abraham, j. (2021). ecommerce and covid-19 – the good, the bad, and the ugly, the paypers, february 5, 2021. https://thepaypers.com/interviews/ecommerce-andcovid-19-the-good-the-bad-and-the-ugly--1247083. ahmad, n., mohamad, h., zainol, h., & yusof z.m. (2019). decision making using neutrosophic and analytical hierarchy process (n-ahp) in choosing the qualified candidate to enter the teacher training institutions. international journal of advanced trends in computer science and engineering, 8(1), 56-59. doi: https://doi.org/10.30534/ijatcse/2019/1281.52019 aldaloui, e., & perçin, s. (2020). financial performance evaluation of food and drink index using fuzzy mcdm approach. international journal of economics and innovation, 6(1), 1-19. alibaba (2021). https://turkish.alibaba.com/ (date of access: november 6, 2021) allegro (2021). https://allegro.pl/ (date of access: november 4, 2021) amazon (2021). https://www.amazon.com/ (date of access: november 6, 2021) arif, m., suseno, j. e. and isnanto, r. r. (2020). multi-criteria decision making with the vikor and smarter methods for optimal seller selection from several emarketplaces. e3s web of conferences 202, 5 th international conference on energy, environmental and information system (icenis 2020), 14002. doi: https://doi.org/10.1051/e3sconf/202020214002 arslandere, m. (2010). elektronik ticaret ve karaman’daki kobi’ler üzerine bir araştırma [a research on electronic commerce and smes in karaman]. master’s thesis, karamanoğlu mehmetbey university. https://doi.org/10.1007/s12652-017-0548-7 https://doi.org/10.3390/sym10040116 https://dx.doi.org/10.2139/ssrn.3621166 https://thepaypers.com/interviews/ecommerce-and-covid-19-the-good-the-bad-and-the-ugly--1247083 https://thepaypers.com/interviews/ecommerce-and-covid-19-the-good-the-bad-and-the-ugly--1247083 http://dx.doi.org/10.30534/ijatcse/2019/1281.52019 https://allegro.pl/ https://www.amazon.com/ https://doi.org/10.1051/e3sconf/202020214002 ijahp article: akman, boyacı, kurnaz/selecting the suitable e-commerce marketplace with neutrosophic fuzzy ahp and edas methods from the seller’s perspective in the context of covid-19 international journal of the analytic hierarchy process 31 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.994 asante, d., he, z., adjei, n.o., & asante, b. (2020). exploring the barriers to renewable energy adoption utilizing multimooraedas method. energy policy. 142, 111479. doi: https://doi.org/10.1016/j.enpol.2020.111479 aydın, s., aktas, a., & kabak, m. (2019). neutrosophic fuzzy analytic hierarchy process approach for safe cities evaluation criteria. in aliev, r., kacprzyk, j., pedrycz, w., jamshidi, m., sadikoglu, f. (eds) 13th international conference on theory and application of fuzzy systems and soft computing, 896, 956-965. behzad, m., zolfani, s.h., pamucar, d., & behzad, m. (2020). a comparative assessment of solid waste management performance in the nordic countries based on bwm-edas. journal of cleaner production, 266, 122008. doi: https://doi.org/10.1016/j.jclepro.2020.122008 bhatti, a., akram, h., basit, h.m., khan, a.u., naqvi, s.m.r., & bilal, m. (2020). e-commerce trends during covid-19 pandemic, international journal of future generation communication and networking, 13(2), 1449-1452. bigcommerce. (2021) https://www.bigcommerce.com/ (date of access: september 27, 2021) bol (2021). https://www.bol.com/ (date of access: november 6, 2021). boyaci, a.i̇., & baynal, k. (2016). e-commerce research and applications: a proposal for classification and an updated literature review. international refereed journal of marketing and market researches, 9(1), 23-41. doi: https://doi.org/10.17369/uhpad.2016923644 bucaklı, a.t. (2007). elektronik ticaret [electronic commerce]. master’s thesis, school of management, i̇stanbul kadir has university (2007). buyukozkan, g. (2004). multi-criteria decision making for e-marketplace selection. internet research-electronic networking applications and policy, 14(2), 139-154. doi: https://doi.org/10.1108/10662240410530853 canbaz, s. (2007). türkiye’de elektronik ticaret uygulamasında karşılaşılan muhasebe sorunları [accounting problems encountered in electronic commerce practice in turkey]. master’s thesis, trakya university (2007). duan, x., deng, h., & corbitt, b. (2010). a multi-criteria analysis approach for the evaluation and selection of electronic market in electronic business in small and medium sized enterprises. in wang, f.l., gong, z., luo, x., lei, j. (eds), web information systems and mining. wism 2010. lecture notes in computer science, 6318. springer. doi: https://doi.org/10.1007/978-3-642-16515-3_17 emarketer. (2021a). https://www.emarketer.com/chart/247776/top-10-countriesranked-by-retail-ecommerce-sales-growth-2021-change (date of access: december 27, 2021). https://doi.org/10.1016/j.enpol.2020.111479 https://doi.org/10.1016/j.jclepro.2020.122008 http://dx.doi.org/10.17369/uhpad.2016923644 https://doi.org/10.1108/10662240410530853 https://doi.org/10.1007/978-3-642-16515-3_17 ijahp article: akman, boyacı, kurnaz/selecting the suitable e-commerce marketplace with neutrosophic fuzzy ahp and edas methods from the seller’s perspective in the context of covid-19 international journal of the analytic hierarchy process 32 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.994 emarketer. (2021b). https://www.emarketer.com/chart/c/t11547/retail-ecommercesales-worldwide-2019-2025-trillions-change-of-total-retail-sales-1 [1] (date of access: december 27, 2021). erdoğan, g.m. (2020). covid-19 döneminde e-ticaret ve dijital reklam yatırımları [e-commerce and digital advertising investments in the covid-19 era]. selçuk i̇letişim dergisi, 13(3), 1296-1318 (2020). doi: https://doi.org/10.18094/josc.776605 erkayman, b., khorshidi, m., & usanmaz, b. (2018). an integrated fuzzy approach for erp deployment strategy selection under conflicting criteria. atatürk üniversitesi i̇ktisadi ve i̇dari bilimler dergisi, 32(3), 807-823. etsy (2021). https://www.etsy.com/ (date of access: november 6, 2021). flipkart (2021). https://seller.flipkart.com/( date of access: november 4, 2021). ghorabaee, k.m., zavadskas, e.k., olfat, l., & turskis, z. (2015). multi-criteria inventory classification using a new method of evaluation based on distance from average solution (edas). informatica, 26(3), 435–451. doi: http://dx.doi.org/10.15388/informatica.2015.57 guo, j. (2007). business-to-business electronic marketplace selection. enterprise information systems, 1(4), 383-419. doi: https://doi.org/10.1080/17517570701648109 gündoğdu, f.k., kahraman, c., & civan, h.n. (2018). a novel hesitant fuzzy edas method and its application to hospital selection. journal of intelligent and fuzzy systems, 35, 6353-6365. gürpınar, b. (2007). elektronik ticaretin hukuksal boyutu üzerine. akademik bilişim, kütahya, turkiye. hidayat, s., susanti, m., & rahmawati, m. (2021). selection of the right marketplace for sale of ornamental plants using analytical hierarchy process (ahp) method. techno nusa mandri: journal of computing and i̇nformation technology, 18(1), 110. doi: https://doi.org/10.33480/techno.v18i1.2054 ilieva, g., yankova, t. & klisarova-belcheva, s. (2018). decision analysis with classic and fuzzy edas modifications. computational & applied mathematics, 37, 5650–5680. doi: https://doi.org/10.1007/s40314-018-0652-0 i̇çigen, t.e., & kutlu, u. (2012). elektronik ticaret: tüketicilerin bakış açılarını saptamaya yönelik bir araştırma [electronic commerce: a research to determine consumers' perspectives]. duzce university journal of social sciences institute, 2(1), 54-70 (2012). https://doi.org/10.18094/josc.776605 https://seller.flipkart.com/ http://dx.doi.org/10.15388/informatica.2015.57 http://dx.doi.org/10.1080/17517570701648109 https://doi.org/10.33480/techno.v18i1.2054 https://doi.org/10.1007/s40314-018-0652-0 ijahp article: akman, boyacı, kurnaz/selecting the suitable e-commerce marketplace with neutrosophic fuzzy ahp and edas methods from the seller’s perspective in the context of covid-19 international journal of the analytic hierarchy process 33 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.994 junaid, m., xue, y., syed, m.w., li, j.z., & ziaullah, m. (2019). a neutrosophic ahp and topsis framework for supply chain risk assessment in automotive industry of pakistan. sustainability, 12(1), 154-180. doi: https://doi.org/10.3390/su12010154 kahraman, c., ghorabaee, m.k., zavadskas, e.k., onar, s.c., yazdani, m., & oztaysi, b. (2019). intuitionistic fuzzy edas method: an application to solid waste disposal site selection. journal of environmental engineering and landscape management, 25(1), 1-12. doi: https://doi.org/10.3846/16486897.2017.1281139 kahraman, c., onar, s.ç., öztayşi, b. (2018). b2c marketplace prioritization using hesitant fuzzy linguistic ahp. international journal of fuzzy systems, 20, 22022215. doi: https://doi.org/10.1007/s40815-017-0429-4 karaşan, a., and kahraman, c. (2017). interval-valued neutrosophic extension of edas method. advances in intelligent systems and computing, 343–357. doi: https://doi.org/10.1007/978-3-319-66824-6_31 karlson, a.w. (2021). elements of marketing on global online marketplaces a new conceptual framework for b2b companies. master thesis, copenhagen business school. denmark keshavarz-ghorabaee, m., zavadskas, e., amiri, m. and turskis, z. (2016). extended edas method for fuzzy multi-criteria decision-making: an application to supplier selection. international journal of computers communications & control, 11(3), 358-371. doi: https://doi.org/10.15837/ijccc.2016.3.2557 keshavarz-ghorabaee, m., amiri, m., zavadskas, e.k., turskis, z. and antucheviciene, j. (2017a). a new hybrid simulation-based assignment approach for evaluating airlines with multiple service quality criteria. journal of air transport management, 63, 45-60. doi: https://doi.org/10.1016/j.jairtraman.2017.05.008 keshavarz-ghorabaee, m., amiri, m., zavadskas, e.k., turskis, z., and antucheviciene, j. (2017b). stochastic edas method for multi-criteria decisionmaking with normally distributed data. journal of intelligent and fuzzy systems, 33, 1627-1638. kokoç m., & ersöz s., (2021). aralık değerli sezgisel bulanık ortamda entropinin çok kriterli karar verme problemlerinde kullanılmasına ilişkin eleştirel analiz. politeknik dergisi, 25 (2), 799-811. doi: https://doi.org/10.2339/politeknik.764082 kollmann, t., & lomberg, c. (2010). web 1.0, web 2.0 and web 3.0: the development of e-business. in lee, in. (ed.) encyclopedia of e-business development and management in the global economy. igi global. kolomvatsos k., anagnostopoulos c., & hadjiefthymiades, s. (2014). sellers in emarketplaces: a fuzzy logic based decision support system. information sciences, 278, 267-284, doi: https://doi.org/10.1016/j.ins.2014.03.052. https://doi.org/10.3390/su12010154 https://doi.org/10.3846/16486897.2017.1281139 https://doi.org/10.1007/s40815-017-0429-4 https://doi.org/10.1007/978-3-319-66824-6_31 http://dx.doi.org/10.15837/ijccc.2016.3.2557 https://www.sciencedirect.com/journal/journal-of-air-transport-management https://www.sciencedirect.com/journal/journal-of-air-transport-management https://doi.org/10.1016/j.jairtraman.2017.05.008 https://doi.org/10.2339/politeknik.764082 https://doi.org/10.1016/j.ins.2014.03.052 ijahp article: akman, boyacı, kurnaz/selecting the suitable e-commerce marketplace with neutrosophic fuzzy ahp and edas methods from the seller’s perspective in the context of covid-19 international journal of the analytic hierarchy process 34 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.994 kumar, a., sikdar, p., & saha, r. (2021). seller experience assessment in online marketplace: a scale development study. benchmarking: an international journal, 28(7), 2315-2342. doi: https://doi.org/10.1108/bij-06-2020-0305 kundakcı, n. (2019). an integrated method using macbeth and edas methods for evaluating steam boiler alternatives. journal of multi-criteria decision analysis, 26, 27-34. doi: https://doi.org/10.1002/mcda.1656 liu, y., eckert, c.m., & earl, c. (2020). a review of fuzzy ahp methods for decision-making with subjective judgements. expert systems with applications, 161(1), 1-30. doi: https://doi.org/10.1016/j.eswa.2020.113738 loukis, e., spinellis, d., & katsigiannis, a. (2011). barriers to the adoption of b2b e-marketplaces by large enterprises: lessons learned from the hellenic aerospace industry. information systems management, 28(2), 130-146. doi: https://doi.org/10.1080/10580530.2011.562129 lubis, a.i., erdiansyah, u., & ramadhan, m. (2022). kombinasi metode vikor dan rank order centroid dalam pemilihan e-marketplace. jurnal medıa informatıka budıdarma, 6(1), 236-242. doi: https://doi.org/10.30865/mib.v6i1.3376 ozkok, b.a., & pappalardo, e. (2013). fuzzy analytic hierarchy process and its application to e-marketplace selection. enterprise business modeling, optimization techniques, and flexible information systems. italian national research council. patel, a. (2010). concept of mobile agentbased electronic marketplace safety measures. in lee, (ed.) encyclopedia of e-business development and management in the global economy. igi global. pourghasemi, h.r., pradhan, b., & gokceoglu, c. (2012). application of fuzzy logic and analytical hierarchy process (ahp) to landslide susceptibility mapping at haraz watershed, iran. natural hazards, 63(1), 965–996. doi: https://doi.org/10.1007/s11069-012-0217-2 rakuten (2021). https://www.rakuten.com/ (date of access: november 6, 2021). saaty, t.l. (1980). the analytic hierarchy process. new york: mcgraw-hill. schu, m., & morschett, d. (2017). foreign market selection of online retailers a path-dependent perspective on influence factors. international business review, 26(4), 710-723. doi: https://doi.org/10.1016/j.ibusrev.2017.01.001 sharifi, h., kehoe, d.f., & hopkins, j. (2006). a classification and selection model of e‐marketplaces for better alignment of supply chains. journal of enterprise information management, 19(5), 483-503. doi: https://doi.org/10.1108/17410390610703639 https://www.emerald.com/insight/search?q=amresh%20kumar https://www.emerald.com/insight/search?q=pallab%20sikdar https://www.emerald.com/insight/search?q=raiswa%20saha https://www.emerald.com/insight/publication/issn/1463-5771 https://doi.org/10.1108/bij-06-2020-0305 https://doi.org/10.1002/mcda.1656 https://doi.org/10.1016/j.eswa.2020.113738 http://dx.doi.org/10.1080/10580530.2011.562129 https://doi.org/10.30865/mib.v6i1.3376 https://www.igi-global.com/book/enterprise-business-modeling-optimization-techniques/72364 https://www.igi-global.com/book/enterprise-business-modeling-optimization-techniques/72364 https://doi.org/10.1007/s11069-012-0217-2 https://doi.org/10.1016/j.ibusrev.2017.01.001 https://www.emerald.com/insight/search?q=h.%20sharifi https://www.emerald.com/insight/search?q=d.f.%20kehoe https://www.emerald.com/insight/search?q=j.%20hopkins https://www.emerald.com/insight/publication/issn/1741-0398 https://www.emerald.com/insight/publication/issn/1741-0398 https://doi.org/10.1108/17410390610703639 ijahp article: akman, boyacı, kurnaz/selecting the suitable e-commerce marketplace with neutrosophic fuzzy ahp and edas methods from the seller’s perspective in the context of covid-19 international journal of the analytic hierarchy process 35 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.994 sirimanne, s.n. (2021). covid-19 and e-commerce: a global review. new york: unctad, united nations publications. united nations. https://unctad.org/webflyer/covid-19-and-e-commerce-global-review stanujkic, d., zavadskas, e.k., ghorabaee, m.k., & turskis, z. (2017). an extension of the edas method based on the use of interval grey numbers. studies in informatics and control, 26(1), 5-12. doi: https://doi.org/10.24846/v26i1y201701 statista (2021). https://www.statista.com/ (date of access: november 27, 2021) stević, ž., tanackov, i., vasiljević, m., novarlić, b., & stojić, g. (2016). an integrated fuzzy ahp and topsis model for supplier evaluation. serbian journal of management, 11(1), 15-27. doi: https://doi.org/10.5937/sjm11-10452 stockdale, r., & standing, c. (2002). a framework for the selection of electronic marketplaces: a content analysis approach. internet research, 12(3), 221-234. doi: https://doi.org/10.1108/10662240210430900 tan, y.l., & macaulay, l. (2008). small business collaboration through electronic marketplaces. in lee (ed.) encyclopedia of e-business development and management in the global economy. igi global. taşdemir, n. (2008). i̇hracatta internet ve e-ticaretten faydalanma yolları ve bir uygulama. master’s thesis, i̇stanbul ticaret university. thitimajshima, w., esichaikul, v. & krairit, d. (2018). a framework to identify factors affecting the performance of third-party b2b e-marketplaces: a seller’s perspective. electronic markets, 28, 129–147. doi: https://doi.org/10.1007/s12525017-0256-3 ticaret (2021). https://www.ticaret.gov.tr/ (date of access: september 4, 2021) trendyol (2021). https://www.trendyol.com/ (date of access: november 6, 2021) wang, h., smarandache, f., zhang, y., & sunderraman, r. (2010). single valued neutrosophic sets. review of the air force academy, 1(10), 410-413. wang, s., & archer, n.p. (2007). electronic marketplace definition and classification: literature review and clarifications. enterprise information systems, 1(1), 89-112. doi: https://doi.org/10.1080/17517570601088380 wibowo, a.p., & yunianto, e. (2019). pemilihan e-marketplace berkualitas dengan metode fuzzy ahp dan fuzzy vikor. seminar nasional edusainstek, 3, 76-87. doi: https://doi.org/10.47775/ictech.v14i2.65 xu, x., chen, y., he, p., yu, y., & bi, g. (2021). the selection of marketplace mode and reselling mode with demand disruptions under cap-and-trade regulation. https://unctad.org/webflyer/covid-19-and-e-commerce-global-review http://dx.doi.org/10.24846/v26i1y201701 https://www.statista.com/ http://dx.doi.org/10.5937/sjm11-10452 https://doi.org/10.1108/10662240210430900 https://doi.org/10.1007/s12525-017-0256-3 https://doi.org/10.1007/s12525-017-0256-3 https://doi.org/10.1080/17517570601088380 https://doi.org/10.47775/ictech.v14i2.65 ijahp article: akman, boyacı, kurnaz/selecting the suitable e-commerce marketplace with neutrosophic fuzzy ahp and edas methods from the seller’s perspective in the context of covid-19 international journal of the analytic hierarchy process 36 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.994 international journal of production research, 1(1), 1-20. doi: https://doi.org/10.1080/00207543.2021.1897175 yıldız, y.m. (2019). uluslararası hedef pazar seçiminde rekabet istihbaratının rolü ve önemi, solar panel sektörü özelinde bir değerlendirme. master’s thesis, school of management, başkent university. yunianto, e., & taryadi, t. (2022). implementasi fuzzy decision making untuk pemilihan marketplace. smart comp: jurnalnya orang pintar komputer, 11(1), 100-104. doi: http://dx.doi.org/10.30591/smartcomp.v11i1.3253 yuianto, e. & wibowo, a.p. (2020). marketplace selectıon strategy wıth ıntegratıon of fuzzy ahp and fuzzy moora methods. prosiding seminar edusainstech, fmipa unimus 2020, 508-517. yücesan, m. (2020). fmea analysis in mechanical installation project based on best worst and neutrosophic ahp integrated model. electronic social sciences journal, 19(73), 363-382. doi: https://doi.org/10.17755/esosder.569291 https://doi.org/10.1080/00207543.2021.1897175 https://ejournal.poltektegal.ac.id/index.php/smartcomp/issue/view/223 http://dx.doi.org/10.30591/smartcomp.v11i1.3253 https://doi.org/10.17755/esosder.569291 ijahp article: nahavandi, homayounfar, daneshvar/a fuzzy analytical hierarchy process for evaluation of knowledge management effectiveness in research centers international journal of the analytic hierarchy process 1 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.978 a fuzzy analytical hierarchy process for evaluation of knowledge management effectiveness in research centers bijan nahavandi assistant professor of industrial management faculty of management and economics, science and research branch islamic azad university tehran, iran bijan.nahavandi@srbiau.ac.ir mahdi homayounfar department of industrial management rasht branch, islamic azad university rasht, iran homayounfar@iaurasht.ac.ir amir daneshvar department of information technology management electronic branch islamic azad university tehran, iran a_daneshvar@iauec.ac.ir abstract measuring knowledge management (km) effectiveness is a very important issue in organizations today. this study aims to develop a method to evaluate the effectiveness of km under uncertainty in research centers (rcs) in iran. to develop an evaluation, the relevant literature was reviewed to identify km effectiveness criteria. next, the judgments of the experts, specialists, scholars, and professionals in it and km systems and senior managers of iran’s rcs were determined using a pairwise comparison questionnaire. because linguistic terms are an integral part of human judgments that will influence the results of the research, a fuzzy analytic hierarchy process (ahp) called the extent analysis (ea) method was used for data analysis and weight determination. accordingly, 34 subcriteria extracted from the literature that are important in the evaluation of km effectiveness were categorized into six main criteria as follows: human resources, leadership and center structure, knowledge creation and acquisition, knowledge storage and security, knowledge sharing, and knowledge utilization and updating. the findings indicate that human resources is the most important criterion based on both the ahp and fuzzy ahp methods. the other five criteria in descending order of importance are knowledge sharing, leadership and center structure, knowledge utilization and updating, knowledge creation and acquisition and knowledge storage and security. finally, to test the validity and reliability of the proposed framework, we evaluated the effectiveness of km system in nine of iran’s rcs. mailto:bijan.nahavandi@srbiau.ac.ir mailto:homayounfar@iaurasht.ac.ir ijahp article: nahavandi, homayounfar, daneshvar/a fuzzy analytical hierarchy process for evaluation of knowledge management effectiveness in research centers international journal of the analytic hierarchy process 2 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.978 keywords: knowledge management; effectiveness; fuzzy set theory (fst), ahp; research centers 1. introduction in recent decades, organizations have shifted their focus from capital, human resources, and technology to knowledge (wang & chang, 2007). moreover, managers need knowledge and information to make appropriate decisions. therefore, the knowledge and information that comes from human resources must be organized and accessible for decision-makers whenever it is needed. as a logical result, each organization should develop a comprehensive and effective system to manage knowledge and information. knowledge management (km) is an appropriate tool for this purpose (barão et al., 2017; ogiela, 2015). currently, km has become increasingly important because of the growing awareness of its significance for an organization's prosperity and survival (byukusenge & munene, 2017). in summary, km allows an organization to meet its needs and expectations, enhance its performance and competitiveness in the market, and develop new ideas or ways of conducting business. km links information supply and demand with a learning management system and consequently, improves organizational performance (schniederjans et al., 2019). integrating organizational knowledge from theory into practical actions is an essential function of km. the successful implementation of km in organizations can increase the organization's productivity by more than 30% (kim et al., 2021). an appropriate km system enables an organization to efficiently create, acquire, share, transform, and use knowledge, which leads to better organizational performance (kim et al., 2021; yap et al., 2022). the most cited document on the topic of km was written and published by alavi and leidner (2001). they indicate that companies’ difficulties in maintaining, locating, and applying knowledge have led them to develop systematic procedures to manage it. their framework of km processes encompasses knowledge creation, knowledge storage/retrieval, knowledge transfer, and knowledge application to create value from the knowledge assets the firm possesses. simultaneously, the establishment of km by research centers (rcs) has been growing in recent years (ermine, 2010). numerous rcs expect to achieve the capability of managing their intellectual and valuable resources and reinforce existing advantages by initiating km. the strength of any rc highly depends on the knowledge that is in the hands of their researchers. an organization will be exposed to risk when a researcher leaves that organization. indeed, tacit knowledge of employees at rcs is one of the most essential factors that affect business performance (adachi, 2009; smits & de moor, 2004). however, the reduction in strength is preventable for rcs, like any other business, by establishing an effective km system. measuring the effectiveness of km operations is one of the most challenging issues for rcs in iran, and is also an extremely complicated matter in other countries around the world. one limitation of the classic analytical hierarchy process (ahp) approach is the use of deterministic values as the equivalent of verbal values, which removes the subjective nature of the comparisons from the calculation process. therefore, researchers have used ijahp article: nahavandi, homayounfar, daneshvar/a fuzzy analytical hierarchy process for evaluation of knowledge management effectiveness in research centers international journal of the analytic hierarchy process 3 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.978 fuzzy ahp because fuzzy ratios help respondents include imprecise and ambiguous answers in pairwise comparisons in weight calculations which helps reduce respondents’ uncertainty about their choices (ohoitimur et al., 2021). fuzzy ahp is one of the most popular and famous mcdm methods that has been used in various fields such as dynamic vendor selection (koul & verma, 2009), warehouse location selection (kahraman et al., 2016), industrial maintenance strategy (ohta et al., 2018), strategic management (ohoitimur et al., 2021), software requirements selection (nazim et al., 2022), and public transportation (buran & erçek, 2022). the aim of this study is to provide a framework for evaluating km in rcs. nevertheless, models for evaluating and measuring the effectiveness of km in the literature are very general and include a wide range of businesses and organizations. in this article, a specific framework for measuring the effectiveness of km at rcs is presented. then, both fuzzy and crisp analytical hierarchy process (ahp) methods are used to prioritize the criteria and subcriteria of the proposed framework and their results are compared. the applied methods are able to assess quantitative and qualitative criteria. among the other weight determining methods in the literature, the ahp is the most frequently applied method which indicates its critical importance. it is believed that the proposed evaluation framework is very useful for research-based organizations. this study adds precise and practical knowledge to the available literature on the evaluation and measurement of km effectiveness by providing experts with suitable criteria to evaluate and measure km effectiveness. these criteria would be useful especially in rcs with a fuzzy standpoint for dealing quantitatively with imprecision and uncertainty. it also helps to obtain a fuzzy prioritization of km effectiveness measurement indicators. rcs, which are knowledge producers, have a greater need to establish a km system at different stages in their work. the existence of a system that creates synergy in conducting research projects and optimal use of the resources of knowledge-based organizations is a matter that is directly related to the efficiency and effectiveness of conducting research projects in these centers. therefore, in this study an attempt has been made to evaluate the influence factors of km in rcs by presenting a model to improve the performance of the organization. 2. literature review 2.1 knowledge and knowledge management rapid changes in today’s environment have led organizations to adjust and update the knowledge they possess to maintain their competitive advantage (ale et al., 2014). knowledge is recognized as a core competency and a primary source of value creation for organizations around the world (liu et al., 2020). it is the only asset in an organization that completes technology, strategy, process, and structure as a whole (breznik, 2018). knowledge has an inherent value that needs to be managed, applied, developed, and exploited. it can be seen as an asset that raises traditional asset questions to management, such as when, how much, and what to invest in (mardani et al., 2018). knowledge has different meanings and concepts and is referred to by different terms, such as expertise, experience, skill, intelligence, and insight, depending on the subject area. the km literature has not reached a consensus on the definition of knowledge (anjaria, 2020). as the combination of experiences, values, contextual information and expert insights, knowledge provides a framework for combining experiences and ijahp article: nahavandi, homayounfar, daneshvar/a fuzzy analytical hierarchy process for evaluation of knowledge management effectiveness in research centers international journal of the analytic hierarchy process 4 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.978 inferring new knowledge (donate & sánchez de pablo, 2015; kusumastuti et al., 2021; kokkaew et al., 2022). in the literature, knowledge is divided into tacit and explicit forms. tacit knowledge is produced by information processing in an individual’s mind and acquired through experience (ale et al., 2014; schniederjans et al., 2020). it is deeply rooted in action, commitment, and involvement and thus is very difficult to formalize, communicate, and share with other people. in contrast, explicit knowledge can be expressed in words and numbers. it is produced by the articulation and communication of tacit knowledge and captured in a code or language that facilitates its communication in libraries, archives, and databases (schniederjans et al., 2020). this knowledge is codified and transmittable in formal, systematic language and thus is able to be captured (ale et al., 2014). there is a permanent interaction between tacit and explicit knowledge that moves across individuals, groups, organizations, and back to individuals (nonaka & toyama, 2003). the theory of knowledge creation proposes that knowledge is created over an endless cycle and, thus, increases organizational knowledge via externalization (conversion of tacit to explicit knowledge), socialization (creation of tacit knowledge from shared tacit knowledge), combination (creation of knowledge through the combination and exchange of explicit knowledge), and internalization (conversion of explicit to tacit knowledge) (ale et al., 2014; curado & bontis, 2011). in the current knowledge economy, knowledge is an essential strategic resource that enables firms to sustain a competitive advantage in a dynamic market environment. because it is intangible, knowledge is a complex concept to understand and share among the departments of the organization. using knowledge effectively and consistently is an important way to succeed (rabeea et al., 2019; martins et al., 2019). as the necessary intangible resource for any organization, knowledge should be effectively managed (mardani et al., 2018). in the literature, there are many authors who point out the role played by km as an increasingly important capability for an organization to be successful in both public and private sectors (al ahbabi et al., 2018; gaviria-marin et al., 2019; gonzaga de albuquerque et al., 2018; martins et al., 2019). duhon (1998) defined km as a discipline for promoting the identifying, capturing, evaluating, retrieving, and sharing of the organizations’ information, systematically. to this aim, some assets such as procedures, policies, expert and experienced workers, documents, databases and computerized systems must be used. since its introduction in 1990s, the concept of km has become an important area of research in modern management and leadership for academics and practitioners. there is a consensus among researchers that km can be seen as a collaborative and integrated approach that facilitates the creation, capture, organization, access, and use of the intellectual asset for long-term sustainability and strategic advantage in organizations (al saifi, 2015; hussinki et al., 2017; peng et al., 2007; prusak, 2014; martins et al., 2019). km bridges information demand and supply on behalf of learning processes, and consequently, organizational performance improvement (curado & bontis, 2011; schniederjans et al., 2020). 2.2 knowledge management effectiveness in the 21 st century, business environments have become increasingly knowledge-based, leveraging this knowledge to innovate (urban & matela, 2022). knowledge is essential to the organization because it is essential to its survival in the competitive market. it is the ijahp article: nahavandi, homayounfar, daneshvar/a fuzzy analytical hierarchy process for evaluation of knowledge management effectiveness in research centers international journal of the analytic hierarchy process 5 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.978 main engine of economic growth and the catalyst for technological progress and productivity (abusweilem & abualous, 2019). knowledge creates value only when it is applied by organizations to create capabilities and take effective actions. therefore, support and enhancement of knowledge use in organizations should be one of the major focuses of km initiatives (yun, 2013). km refers to the range of practices and techniques used by organizations to identify and distribute knowledge, know-how, expertise, intellectual capital and other forms of knowledge for leveraging, reusing and transferring knowledge and learning across the organization. it can be defined as the practice of using prior knowledge to make decisions that affect the current and future effectiveness of the organization (ale et al., 2014). indeed, km directs organizational innovation processes toward a competitive advantage (mardani et al., 2018; vaio et al., 2021) and helps organizations improve their effectiveness by strengthening their decision-making capabilities. the view that regards knowledge as a source of competitive advantage comes from the resource-based perspective of organizations, which stems primarily from the theory of internal resources and capabilities. scholars have argued for the relevance of km in increasing organizational effectiveness. indeed, km plays a significant role in organizations that are seeking ways to reach their goals and strategic plans more efficiently and effectively. therefore, it is a key factor for survival in the turbulent international market because km makes knowledge delivery at all layers of an organization possible (ping et al., 2009; albassam, 2019). innovative firms such as rcs need sophisticated km that gives close attention to the special requirements of interactive knowledge. particularly in knowledge-based organizations such as rcs, effectiveness is highly dependent on how well knowledge is shared between individuals, teams, and units (alavi & leidner, 2001). the incentive for the search, absorption, and sharing of knowledge has contributed considerably to the achievement of organizational goals (martin et al., 2019). it has been argued that knowledge-sharing behaviors contribute to the generation of various organizational capabilities such as innovation, which is vital to a firm's performance (mardani et al., 2018). according to wu and lee (2007) and kamara et al. (2002), km is the organizational optimization of knowledge to achieve superior productivity through a variety of techniques. km is a systematic approach to managing knowledge in organizations for competitiveness (wu & lee, 2007). it is useful to consider the relationship between data, information, knowledge, and wisdom to have a clear view of km. data and information are the foundation of knowledge and have hierarchical relationships based on the studies of arthur anderson business consultant in 1999 (wen, 2009). dóci et al. (2022) studied km in transfer management, and the results showed that the problem is due to the lack of knowledge integration. wen (2009) established a model with five major constructs including staff, information, data, knowledge, wisdom, and 30 indices to measure the effectiveness of the km process. according to the research, several problems would be solved by establishing an assessment model for km effectiveness such as multiple objectives, evaluation difficulties, and fuzzy behavior of km. in other research, tseng (2008) proposed a new metric measure for assessing the performance of a km system based on financial and ijahp article: nahavandi, homayounfar, daneshvar/a fuzzy analytical hierarchy process for evaluation of knowledge management effectiveness in research centers international journal of the analytic hierarchy process 6 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.978 non-financial criteria. the study presented the critical factors to improve the quality of the km system. furthermore, the study claimed that appropriate it can significantly enhance the effectiveness and efficiency of implementing a km system. as previous studies indicate, many factors affect the prosperity of km projects with leadership and center structure being the most important (chang et al., 2009; cabrera & cabrera, 2005; hislop, 2003). moreover, among these factors, human resources is regarded as a key lever of competitive advantage in the current global, dynamic, and complex business environment, particularly in the context of km (e.g., chen & huang, 2009; oltra, 2005; hislop, 2003; martinsons, 1997; davis & botkin, 1994; ulrich, 1998; winch & schneider, 1993). ode and ayavoo (2020) investigated the relationship between km practices and firm innovation by the mediating role of knowledge application. their findings showed that knowledge generation, storage, and application have significant effects on firm innovation. kim et al. (2021) studied the impact of km strategies on firm performance by categorizing km portfolios into four patterns. the findings indicate that unrelated diversity portfolio strategies show substitutable effects on firm performance, while the effects of related diversity portfolios strategies are complementary. abbas and sağsan (2019) studied the impact of km practices on green innovation and corporate sustainable development. based on the results, km significantly impacts green innovation and corporate sustainable development activities. na et al. (2017) developed a knowledge-based advisory expert system using the ahp to investigate the late-life structural ambiguity of fixed jacket platforms in the selection of the best practicable decommissioning method. in their study, the effects and ranking of key factors on the decommissioning planning process were numerically computed. castrogiovanni et al. (2016) determined the sources of knowledge that have the greatest effect on financial entities’ knowledge acquisition and management using the analytic hierarchy process (ahp). the results showed that human resources and new technology adoption are the most effective sources of knowledge acquisition and management. patil and kant (2014) prioritized the solutions of km adoption in an indian hydraulic valve manufacturing organization supply chain to overcome its barriers. they proposed a framework based on fuzzy ahp and fuzzy topsis to identify and rank the solutions for km adoption in the supply chain and overcome its barriers. the ahp was used to determine the weights of the barriers as criteria, and the fuzzy topsis method was used to obtain the final ranking of the solutions of km adoption in supply chain. fan et al. (2009) proposed a fuzzy linguistic method based on a 2-tuple linguistic representative model with seven attributes. they stated that knowledge creation, knowledge acquisition, knowledge storage, security, and knowledge utilization have a strong effect on km capability (wen, 2009). also, the capability of km depends on two other factors: infrastructure capability and process capability (fan et al., 2009; chuang, 2004; gold et al., 2001). cui et al. (2005) also mentioned that km capabilities consist of the following three interrelated processes: knowledge acquisition, knowledge conversion, and knowledge application (liao & wu, 2010). some important studies in km effectiveness measurement with their proposed criteria are mentioned in table 1. ijahp article: nahavandi, homayounfar, daneshvar/a fuzzy analytical hierarchy process for evaluation of knowledge management effectiveness in research centers international journal of the analytic hierarchy process 7 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.978 table 1 research on km effectiveness measurement source proposed criteria tseng (2010) technology; structure; culture; acquisition; conversion; application; security; market share; new green product competitiveness; monitoring market forces; specialized market units; export percentage; success rate of r&d green products; self-generated innovative products; number of patents; r&d intensity; percentage of researchers to overall employees; degree of innovation of r&d green products; intensity of collaboration with others; r&d knowledge sharing ability; forecasting and evaluation of technological innovation; entrepreneurial innovation initiatives. tan and nasurdin, (2011) acquisition; sharing; application; innovation. fan et al. (2009) technology; structure; culture; acquisition (obtaining knowledge); conversion (make existing knowledge useful); application (actual use of the knowledge); security. chang and wang (2009) strategy; employee traits; superintendent traits; audit and assessment; organizational culture; operation procedure; information technology. hsieh et al. (2009) km strategy; km promotion; km assessment; intellectual capital; knowledge identification and classification; knowledge sharing; knowledge capture; knowledge store; knowledge application; knowledge creation and innovation; knowledge protection; knowledge learning and training; best practices; communities of practice (cops); it infrastructure; km system. wen (2009) human resources; information; data; knowledge; wisdom. oltra (2005) motivation; participation; human resources; integration it infrastructure; technical customization; past experiences. smits and de moor (2004) use of knowledge resources; development of knowledge resources; level of km; level of maintenance km. hoy and miskel (2001) knowledge adaptation effectiveness; knowledge achievement effectiveness; knowledge integration effectiveness; knowledge potential effectiveness mehdibeigi et al. (2016) customer km; organizational agility; organizational effectiveness adaileh et al. (2020) knowledge capturing; knowledge sharing; firm performance; knowledge acquisition; knowledge application kavalić et al. (2021) organizational culture; t-shaped skills: human resources; information technology; knowledge acquisition process; knowledge conversion process; process of applying knowledge; knowledge protection process; competition advantage; organizational culture darvishi and darvishi (2019) knowledge creation; knowledge accumulation; knowledge sharing; knowledge utilization; knowledge internalization 2.3 knowledge management in research centers (rcs) in the most common classification of knowledge, the concept is divided into two categories, tacit and explicit (li & chandra, 2007). since encouraging employees to share their tacit knowledge is crucial in a research environment, the role of km would be ijahp article: nahavandi, homayounfar, daneshvar/a fuzzy analytical hierarchy process for evaluation of knowledge management effectiveness in research centers international journal of the analytic hierarchy process 8 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.978 more obvious in these types of organizations (joseph et al., 2005; okemwa, 2006; smits and de moor, 2004; astrid, 2003). rcs face varied and challenging projects which require knowledgeable and experienced staff to manage. in most cases, the greater portion of knowledge and experience that staff gain during their work and research is tacit not explicit. due to the strategic role of knowledge in rcs, measuring the effectiveness of km in these centers has been recognized by academics and practitioners (chin et al., 2010). tacit knowledge is a valuable resource for future projects and highly depends on the presence of staff. if researchers leave the rc, the tacit knowledge will disappear; this issue is the most important concern of rcs. indeed, this situation illustrates the critical role of effective km. undoubtedly, to have effective km criteria must be customized to measure the km effectiveness of rcs (okemwa, 2006). because of the importance of rcs in producing and developing knowledge, researchers focus on different aspects of them. for instance, adachi, (2009) attempted to answer the four following questions in his studies about rcs:  how is knowledge transferred to each task in the planning stage?  what are the problems in the planning stage and how do you eliminate them?  how does km work in the planning stage of a public research organization?  what kinds of knowledge are created, shared, utilized, and accumulated in the planning stage of a public research organization? smits and de moor (2004) asked “when is km effective?” and “are measurements necessary to realize the effectiveness of km?” the answers to these questions were “it depends on the level of km” and “for a successful km, sufficient attention must be paid to the selection of key aspects, instead of trying to measure everything”, respectively. joseph et al. (2005) studied the need for and impact of a km system in a research organization and how it would help the researcher’s work/output and institutional values. moreover, research was done by the international atomic energy agency (iaea) of australia (2006) to determine the essential criteria for an effective km system. agostino et al. (2012) presented a system for performance measurement of the r&d activities in public research centers in italy. due to the multiple stakeholders with different needs, they concluded that managers should balance the multiple goals and integrate their different performance indicators. several researchers also studied the main performance indicators for assessing r&d performance in public sector research centers (leitner & warden, 2004; chu et al., 2006; secundo et al., 2010; agostino et al., 2012). this research stream is based on intellectual capital (ic) reporting models. wang et al. (2016) found that the consistency of the three dimensions of ic (i.e. human capital, structural capital, and relational capital) and km strategies facilitates both operational and financial performance of high-tech firms. similarly, knowledge strategy was found to moderate the relationship between ic and organizational performance (asiaei et al., 2018). there are also two research streams in the private sector; one focuses on the choice of performance indicators and assessment of the r&d department (bremser & barsky, 2004; schiuma & neely, 2004; mettänen, 2005) and the other concentrates on the design of the performance measurement system in r&d firms and their characteristics, objectives and contextual variables (chiesa & frattini, 2007). based on these considerations, some key performance indicators were defined to measure the ijahp article: nahavandi, homayounfar, daneshvar/a fuzzy analytical hierarchy process for evaluation of knowledge management effectiveness in research centers international journal of the analytic hierarchy process 9 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.978 effectiveness of the rcs. the effectiveness indicator makes it possible to evaluate the rcs’ outputs based on the level of achievement of their research objectives (agostino et al., 2012). as steiner and nixon (1997) mentioned, this aspect is related to a key issue of “what is the output of a research center” and “how do you measure it”. according to okemwa (2006)’s study of the international livestock research institute (ilri), establishing km for a research organization is necessary to determine how knowledge should be generated, shared, transferred and integrated into their day-to-day operations. although different studies have been done in the field of km in rcs, the studies did not pay enough attention to determining and prioritizing criteria of km effectiveness. therefore, this article fills this gap in the literature. many rcs exist that are very important for the production of knowledge in various fields of science. km helps improve performance and processes in organizations by discovering, collecting, and using technical knowledge. in this research, considering the importance and role of km in rcs, the influence factors of km were evaluated using the fuzzy ahp method. by evaluating the influence factors of km in rcs, the gap in knowledge production in this sector is filled. 3. research methodology the present study was conducted in three main phases. the first stage included an exploratory literature review on km effectiveness criteria through the study of relevant articles in scholarly journals. the results are summarized in table 1. these criteria were investigated by conducting interviews with km experts and senior managers of nine research centers in iran (phase 1). in this phase, the experts were asked to judge the main criteria and sub-criteria of the hierarchical structure until a consensus was reached. the second phase involved collecting data through a pairwise comparison questionnaire, which was analyzed by the ahp using simple additive weighting techniques and fuzzy ahp (chang's method). finally, in the third phase, the validity of the proposed framework was tested by measuring the km effectiveness of nine rcs (research institute of petroleum industry, iranian research institute for information science and technology, center of strategic research, institute for cognitive science studies, ipm institute for research in fundamental sciences, institute for management and planning studies, iran standard and industrial research institute, chemistry & chemical engineering research center of iran, and iran management & productivity study centre (impsc)). although the population of the study included all of tehran's rcs, only nine agreed to take part in the research. it should be noted that in each rc, the manager of the r&d department or the assistant director was selected as the organization's representative. madm methods, especially the ahp/fuzzy ahp, do not have any limitation on the number of experts, according to the consistency rate of each expert's responses. ijahp article: nahavandi, homayounfar, daneshvar/a fuzzy analytical hierarchy process for evaluation of knowledge management effectiveness in research centers international journal of the analytic hierarchy process 10 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.978 figure 1 research phases as the analytical part of the research, a multiple attribute decision making (madm) method was applied. most madm techniques resort to either the ordinal scale or the interval scale as an easy way to transform qualitative criteria into the numerical scale (hatefi, 2021). there are several techniques for dealing with madm problems in the literature. a comprehensive literature review was conducted by toloie-eshlaghy and homayonfar (2011) who reviewed mcdm methodologies and applications from 1999 to 2009. based on their study, the ahp is the most widely used and popular method among all of the madm methods. the ahp method, which was first introduced by saaty (1980), divides a complex system into a hierarchical structure, where decision elements are placed at the final (bottom) level, decision criteria and sub-criteria are placed at intermediate levels, and the goal is placed at the first level (ozan, 2008; he et al., 2022). according to lee et al. (2008), the ahp has six fundamental steps: 1. identify the problem and determine the objectives and outcomes. 2. divide the compound problem into a hierarchical structure with decision elements (criteria, subcriteria, and alternatives). 3. conduct pairwise comparisons among decision elements and form comparison matrices. 4. use the eigenvalue method to calculate the relative weights of the decision elements approximately. 5. check the consistency of the matrices to ensure that the judgments of the decisionmakers are consistent. 6. aggregate the relative weights of the decision elements to gain an overall rating of the alternatives. in order to solve the ambiguity and subjectivity of human judgments in the decisionmaking process and to express linguistic variables, fuzzy numbers are replaced with linguistic values in fuzzy ahp, as the second method of data analysis. using the fuzzy ahp method reduces vague judgments and increases accuracy in the calculations (liu et al., 2020). many fuzzy ahp methods have been proposed by researchers such as van laarhoven and pedrycz, (1983); buckley (1985); boender et al. (1989); chang (1996). in this study, chang’s ea method (chang, 1996) is preferred because its steps are relatively easier than the other fuzzy ahp approaches and are similar to the conventional ahp (bozbura et al., 2007). phase 2 prioritizing km effectiveness criteria using: ahp fuzzy ahp and testing validity using: consistency rate (cr) wen (2009) approach phase 1 determining main criteria and sub-criteria of km effectiveness in rcs using: literature review interviews with experts of it and km system and senior managers of rcs phase 3 evaluating the validity of the proposed framework using: case study ijahp article: nahavandi, homayounfar, daneshvar/a fuzzy analytical hierarchy process for evaluation of knowledge management effectiveness in research centers international journal of the analytic hierarchy process 11 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.978 chang’s extent analysis method can be represented as below (rezaee kelidbari et al., 2016): let x = {x1, x2, . . ., xn} be an object set, and g={g1, g2, …. , gn} be a goal set. then, the fuzzy synthesis extent values for each object are as the following: m 1 gi , m 2 gi , …, m m gi ; i =1, 2, …, n (1) in the above equation, m j gi (j = 1, 2 . . . m) are triangular fuzzy numbers (tfns) whose parameters are l, m, and n, which show pessimistic, the most likely, and optimistic values, respectively. the steps to measure chang’s fuzzy synthesis extent can be given as follows: step 1. the value of fuzzy synthetic extent with respect to the i th object is defined as: si =∑mgi j m j=1 ⊗[∑∑mgi j m j=1 n i=1 ] −1 (2) to obtain∑ mgi jm j=i , the fuzzy addition operation of m extent analysis values for a particular matrix is performed such that ∑ mgi jm j=i =(∑ lj m j=1 ,∑ mj m j=1 ,∑ uj m j=1 ) i =1, 2, …, n (3) to obtain[∑ ∑ mgi jm j=1 n i=1 ], the fuzzy addition operation of mjgi (j = 1, 2, ... , m) values is performed such that ∑∑mgi j m j=1 n i=1 =(∑lj m j=i ,∑mj m j=i ,∑uj m j=i ) (4) then, the inverse of the vector in equation 4 is calculated such that [∑∑mgi j m j=1 n i=1 ] −1 =( 1 ∑ ui n i=1 , 1 ∑ mi n i=1 , 1 ∑ li n i=1 ) (5) step 2. the possibility degree of m2 = (l2, m2, u2) ≥ m1 = (l1, m1, u1) is defined as v(m2 ≥m1 =sup⌊min⁡(μm1(x),μm2(x)⌋ (6) and can be equivalently expressed as follows: v(m2 ≥m1)=hgt(m1 ⊗m2)=μm2(d)= (7) ijahp article: nahavandi, homayounfar, daneshvar/a fuzzy analytical hierarchy process for evaluation of knowledge management effectiveness in research centers international journal of the analytic hierarchy process 12 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.978 { 1⁡⁡ ⁡⁡⁡⁡if⁡m2 ≥m1 0⁡ ⁡if⁡⁡l1 ≥u2 l1−u2 (m2−u2)−(m1−l1) ⁡⁡otherwise where d is the ordinate of the highest intersection point, d, between μm1 and μm2 (see figure 2). to compare m1 and m2, we need both the values of v (m1 ≥ m2) and v (m2 ≥ m1). figure 2 intersection between �̃�1 and �̃�2 step 3. the possibility degree for a convex fuzzy number to be greater than k convex fuzzy numbers mi (i = 1, 2 . . . k) can be defined by 𝑉(𝑀 ≥𝑀1,𝑀2,…,𝑀𝑘)= 𝑉[(𝑀 ≥𝑀1)⁡𝑎𝑛𝑑⁡(𝑀 ≥𝑀2)𝑎𝑛𝑑…𝑎𝑛𝑑(𝑀 ≥𝑀𝑘) =min𝑉(𝑀 ≥𝑀𝑖) i= 1, 2, 3, …, k (8) assuming that �́�(𝐴𝑖)=𝑚𝑖𝑛𝑉(𝑆𝑖 ≥ 𝑆𝑘) (9) for k= 1, 2, …, n; k≠i then the weight vector is given by 𝑊 =́ (�́�(𝐴1́ ), �́�(𝐴2́ ),…, �́�(𝐴𝑛́ )) 𝑇 (10) where ai (i = 1, 2 . . . n) are n elements. step 4. the normalized weight vectors are 𝑊 =(𝑑(𝐴1),𝑑(𝐴2),…,𝑑(𝐴𝑛)) 𝑇 (11) where w is a non-fuzzy number. measuring consistency in the fuzzy ahp method is the same as in the traditional ahp; however, it consists of two parallel procedures. according to the gogus and boucher (1998), each fuzzy matrix should be divided into two matrices: (1) the matrix formed from the geometric mean of the upper (optimistic) and lower (pessimistic) values (a g ), and (2) the matrix formed from the most likely value (a m ). then, the consistency ratio of both of them must be calculated based on saaty’s method. the steps for calculating the consistency ratio are as follows: l2 d u1 m2 0 m v (m2≥m1) 𝑀1 𝑀2 1 l1 m 1 u2 𝜇�̃� https://www.sciencedirect.com/topics/mathematics/geometric-mean ijahp article: nahavandi, homayounfar, daneshvar/a fuzzy analytical hierarchy process for evaluation of knowledge management effectiveness in research centers international journal of the analytic hierarchy process 13 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.978 step 1: as mentioned above, in the first step, the fuzzy triangular matrix was divided into two matrices a m and a g . suppose the fuzzy triangular number as almu , then 𝐴𝑚 =⁡[𝑎𝑖𝑗𝑚]⁡⁡⁡;⁡⁡⁡⁡𝐴 𝑔 =√𝑎𝑖𝑗𝑢.𝑎𝑖𝑗𝑙 (12) step 2: the weight vector of each matrix should be calculated using equation 13: 𝑤𝑖 𝑚 =⁡ 1 𝑛 ⁡∑ ⁡⁡ 𝑎𝑖𝑗𝑚 ∑ 𝑎𝑖𝑗𝑚 𝑛 𝑖=1 𝑛 𝑗=1 ⁡𝑤𝑚 =⁡[𝑤𝑖 𝑚] 𝑤𝑖 𝑔 =⁡ 1 𝑛 ⁡∑ ⁡⁡ √𝑎𝑖𝑗𝑢.𝑎𝑖𝑗𝑙 ∑ √𝑎𝑖𝑗𝑢.𝑎𝑖𝑗𝑙 𝑛 𝑖=1 𝑛 𝑗=1 ⁡𝑤𝑔 =⁡[𝑤𝑖 𝑔 ] (13) step 3: in the next step, the highest eigen value for each matrix was calculated using equation 14: 𝜆𝑚𝑎𝑥 𝑚 =⁡ 1 𝑛 ⁡∑ ⁡⁡∑ ⁡⁡𝑎𝑖𝑗𝑚 𝑛 𝑗=1 𝑛 𝑖=1 ⁡( 𝑤𝑗 𝑚 𝑤𝑖 𝑚) 𝜆𝑚𝑎𝑥 𝑔 =⁡ 1 𝑛 ⁡∑ ⁡⁡∑ ⁡⁡√𝑎𝑖𝑗𝑢.𝑎𝑖𝑗𝑙 𝑛 𝑗=1 𝑛 𝑖=1 ⁡( 𝑤𝑗 𝑔 𝑤 𝑖 𝑔) (14) step 4: then, the consistency ratio was calculated using equation 15: 𝐶𝐼𝑚 = (𝜆𝑚𝑎𝑥 𝑚 −𝑛) (𝑛−1) ⁡⁡⁡⁡;⁡⁡⁡⁡𝐶𝐼𝑔 = (𝜆𝑚𝑎𝑥 𝑔 −𝑛) (𝑛−1) (15) step 5: the consistency index (ci) was divided by the random index (ri) to calculate the consistency rate (cr) using equation 16. 𝐶𝑅𝑚 = 𝐶𝐼𝑚 𝑅𝐼𝑚 ⁡⁡⁡⁡;⁡⁡⁡⁡𝐶𝑅𝑔 = 𝐶𝐼𝑔 𝑅𝐼𝑔 ⁡ (16) if the cr index is lower than 0.1, the matrix is considered consistent and the experts’ judgments are reliable. in fuzzy ahp, since the numerical values of fuzzy comparisons are not always integers, and the geometric mean generally converts them to non-integer numbers even if the 1 to 9 scale was used, the ri table provided by saaty is not applicable. table 2 shows the random index of both fuzzy ahp and ahp. https://www.sciencedirect.com/topics/computer-science/triangular-matrix https://www.sciencedirect.com/topics/computer-science/triangular-matrix ijahp article: nahavandi, homayounfar, daneshvar/a fuzzy analytical hierarchy process for evaluation of knowledge management effectiveness in research centers international journal of the analytic hierarchy process 14 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.978 table 2 consistency ratio in ahp and fuzzy ahp size of the matrix ri n ri o ri 1 0 0 0 2 0 0 0 3 0.4890 0.1796 0.58 4 0.7937 0.2627 0.90 5 1.0720 0.3597 1.12 6 1.1996 0.3818 1.24 7 1.2874 0.4090 1.32 8 1.3410 0.4164 1.41 9 1.3793 0.4348 1.45 10 1.4095 0.4455 1.49 11 1.4181 0.4536 1.51 12 1.4462 0.4776 1.54 13 1.4555 0.4691 1.56 14 1.4913 0.4804 1.57 15 1.4986 0.4880 1.58 the process of measuring the consistency ratio in this method is the same as the in classic ahp method, with the only difference being that two rates of consistency, namely, consistency ratio of the matrix of mean values (crn) and the consistency ratio of the matrix of geometric means of lower and upper bounds (cro) should be measured for each fuzzy pairwise comparison matrix. the matrix is consistent if both cr values are 0.1 or less than 0.1. 4. km effectiveness evaluation framework 4.1. hierarchical structure to prioritize km effectiveness measures in rcs as mentioned in the previous section, in phase 1 of the research 34 sub-criteria were finalized for measuring the km effectiveness in rcs through a literature review and interviews with experts. these subcriteria were then divided into six main criteria, as follows: human resources, leadership and center structure, knowledge creation and acquisition, knowledge sharing, knowledge storage and security, and knowledge utilization and updating. it should be noted that, by investigating the technical documents of the rcs, some subcriteria were added to this list. figure 3 illustrates the decision tree of the extracted factors. ijahp article: nahavandi, homayounfar, daneshvar/a fuzzy analytical hierarchy process for evaluation of knowledge management effectiveness in research centers international journal of the analytic hierarchy process 15 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.978 figure 3 decision making hierarchy human resources leadership and center structure knowledge creation and acquisition knowledge sharing knowledge utilization and updating knowledge storage and security knowledge management effectiveness evaluation r e se a rc h b u d g e t o f r c t o p m a n a g e r su p p o rt o f k m c o m p a tib le stru c tu re w ith k m s p a rtic ip a tio n in d e c isio n m a k in g h e lp fu l c re a tiv e a tm o sp h e re in r c l e a rn in g a tm o sp h e re in r c n u m b e r o f d o m e stic p a te n ts n u m b e r o f in te rn a tio n a l p a te n ts n u m b e r o f p a p e rs p u b lish e d in sc ie n tific jo u rn a ls a n d c o n fe re n c e s n u m b e r o f sc ie n tific p u b lish e d b o o k s n u m b e r o f sc ie n tific p u b lish e d jo u rn a ls n u m b e r o f sc ie n tific w e b site s o f w h ic h r c is a m e m b e r c o m p re h e n siv e d a ta / in fo rm a tio n a n d v a rio u s a n d q u a lita tiv e in fo rm a tio n re so u rc e s it b u d g e t o f r c q u a lita tiv e a n d h e lp fu l it in fra stru c tu re s d a ta b a se fo r p e rso n n e l's k n o w le d g e a n d id e a s d e g re e o f sta n d a rd iz a tio n in d o c u m e n ta tio n n u m b e r o f e x e c u te d id e a s (e x e c u te d id e a s/t o ta l id e a s ra tio ) d e g re e to w h ic h p e rso n n e l u se th e la te st (th e n e w e st) k n o w le d g e t im e d u ra tio n fo r a n sw e rin g c u sto m e rs c o m p la in s e sta b lish in g a c u sto m e rs’ d a ta b a se (c r m ) o rg a n iz e d in te rv a l e v a lu a tio n n u m b e r o f p ro fe ssio n a l c e rtific a te s a n d sc ie n tific a w a rd s g a in e d b y p e rso n n e l n u m b e r o f re se a rc h e rs in th e r c n u m b e r o f it e x p e rts in th e r c n u m b e r o f p e rso n n e l w h o a re d ire c tly in v o lv e d in k m s n u m b e r o f e m p lo y e e s’ p ro p o sa ls fo r im p ro v e m e n t c re a tiv e c a p a b ility o f h u in te g ra te d in te rn a l in fo rm a tio n t ra in in g b u d g e t o f r c g e n e ra l tra in in g h o u rs p ro fe ssio n a l tra in in g h o u rs n u m b e r o f ta sk s a n d d u tie s th a t p e rso n n e l c o m p le te v ia c o m p u te rs a m o u n t o f d a ta / in fo rm a tio n th a t p e rso n n e l a re a llo w e d to u se v ia th e w e b fo r p e rso n n e l ijahp article: nahavandi, homayounfar, daneshvar/a fuzzy analytical hierarchy process for evaluation of knowledge management effectiveness in research centers international journal of the analytic hierarchy process 16 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.978 table 3 km effectiveness measures for research centers sym. sub-criteria ‍criteria h1 number of researchers in the rc(researchers/total staff ratio) human resources h2 number of it experts in the rc(it experts/total staff ratio) h3 number of personnel who are directly involved in kms (km personnel/total staff ratio) h4 creative capability of hr (creative human resources) h5 number of employees’ proposals for improvement – (improving/non improving proposals ratio) l1 research budget of rc (research budget/total budget ratio) leadership and center structure l2 top manager support for km l3 compatible structure with kms l4 helpful and creative atmosphere in rc l5 participation in decision making l6 learning atmosphere in rc (incentives for motivating personnel to learn more) c1 number of domestic patents (per capita domestic patents) knowledge creation and acquisition c2 number of international patents (per capita international patents) c3 number of papers published in scientific journals and conferences (per capita papers) c4 number of scientific published journals c5 number of scientific published books c6 number of scientific web sites of which rc is a member c7 comprehensive data and information and various and qualitative information resources s1 it budget of rc (it budget/total budget ratio) knowledge storage and security s2 qualitative and helpful it infrastructures s3 database for personnel's knowledge and ideas s4 degree of standardization in documentation t1 training budget of rc (training budget/total budget ratio) knowledge sharing t2 general training ratio (general training hours/total training hours) t3 professional training hours (professional training hours/total training hours ratio) t4 amount of data and information that personnel are allowed to use via the web (intranet, extranet and internet). t5 number of tasks and duties that personnel complete via computers (e-tasks and duties) t6 integrated internal information u1 degree to which personnel use the latest (the newest) knowledge knowledge utilization and updating u2 time to answer customers’ complains u3 establishing a customer database (crm) u4 number of professional certificates and scientific awards gained by personnel u5 organized interval evaluation u6 number of executed ideas (executed ideas/total ideas ratio) in past studies about prioritizing km effectiveness criteria, the same weights were attributed to all criteria; however, in this study, the criteria do not have the same weights. weights can be derived by pairwise comparison of the criteria based on the ahp methodology. in this research, decision-makers used linguistic terms to demonstrate the significant weights of the factors given in table 4. ijahp article: nahavandi, homayounfar, daneshvar/a fuzzy analytical hierarchy process for evaluation of knowledge management effectiveness in research centers international journal of the analytic hierarchy process 17 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.978 table 4 linguistic terms for importance weights of factors triangular fuzzy reciprocal scale triangular fuzzy scale linguistic scale for importance (1,1,1) (1,1,1) just equal (2/3,1,2) (1/2,1,3/2) equally important (1/2,2/3,1) (1,3/2,2) weakly important (2/5,1/2,2/3) (3/2,2,5/2) strongly more important (1/3,2/5,1/2) (2,5/2,3) very strongly more important (2/7,1/3,2/5) (5/2,3,7/2) absolutely more important to achieve the results, the experts (managers of r&d department or their assistant directors) were asked to fill in the ahp pairwise comparisons matrix based on the linguistic values in table 4. then, the nine resulting individual matrices were fuzzified and merged based on the arithmetic mean. table 5 illustrates the resulting group matrix for the main criteria. table 5 group pairwise comparisons matrix of km effectiveness criteria knowledge utilization and updating knowledge storage and security knowledge sharing knowledge creation and acquisition leadership and center structure human resources km effectiveness (1.26,1.84,2.38) (1.78,2.62,2.8) (0.72,1.26,1.76) (1.96,2.47,2.97) (1.36,1.96,2.51) (1,1,1) human resources (0.79,1.36,1.89) (1.82,2.32,2.82) (1,1.55,2.08) (1.84,2.38,2.9) (1,1,1) (0.4,0.51,0.74) leadership and center structure (1,1.36,1.82) (0.64,0.79,1) (0.56,0.69,0.87) (1,1,1) (0.42,0.54,0.34) (0.34,0.41,0.51) knowledge creation/ acquisition (1.96,2.47,2.97) (2.32,2.82,3.32) (1,1,1) (1.14,1.44,1.8) (0.48,0.64,1) (0.56,0.79,1.39) knowledge sharing (1.19,0.67,0.87) (1,1,1) (0.3,0.35,0.43) (1,1.26,1.55) (0.35,0.43,0.55) (0.36,0.38,0.56) knowledge storage and security (1,1,1) (0.84,1.14,1.49) (0.34,0.41,0.51) (0.55,0.74,1) (0.53,0.74,1.26) (0.42,0.54,0.79) knowledge utilization and updating the results of the implementation of the extent analysis with respect to the main criteria were calculated and summarized. the value of fuzzy synthetic extent was calculated for each criterion using equation 2. [∑ ∑ 𝑀𝑔𝑖 𝑗𝑚 𝑗=𝑖 𝑛 𝑖=1 ] −1 = (33.63, 42.96, 53.4) -1 = (0.0187, 0.0232, 0.0297) then: s1= (8.08, 11.15, 13.42) × (0.0187, 0.0232, 0.0297) = (0.151, 0.258, 0.398) s2= (6.85, 9.12, 11.43) × (0.0187, 0.0232, 0.0297) = (0.128, 0.211, 0.339) s3= (3.88, 4.67, 5.74) × (0.0187, 0.0232, 0.0297) = (0.0725, 0.108, 0.170) s4= (7.46, 9.16, 11.48) × (0.0187, 0.0232, 0.0297) = (0.139, 0.212, 0.340) s5= (3.68, 4.29, 5.28) × (0.0187, 0.0232, 0.0297) = (0.068, 0.099, 0.156) s6= (3.68, 4.57, 6.05) × (0.0187, 0.0232, 0.0297) = (0.068, 0.106, 0.179) ijahp article: nahavandi, homayounfar, daneshvar/a fuzzy analytical hierarchy process for evaluation of knowledge management effectiveness in research centers international journal of the analytic hierarchy process 18 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.978 next, the degree of possibility for each criterion was calculated. for each pairwise comparison, the minimum degree of possibility was as follows using the equation 8: v (s1 ≥ si) = 1; v (s2 ≥ si) = 0.801; v (s3 ≥ si) = 0.114; v (s4 ≥ si) = 0.914; v (s5 ≥ si) = 0.035, v (s6 ≥ si) = 0.155 after computing the primary weight vector of the main criteria (equations 9-10) and normalizing them, the final weights of the main criteria (wi) were calculated using equation 11. w = (0.34, 0.27, 0.04, 0.28, 0.02, 0.05) since the use of the weight values directly depended on the consistency of fuzzy pairwise comparison matrix, the gogus and boucher (1998) method was used to examine the validity of the results. for the fuzzy comparison matrix of the main criteria, the crn and cro indexes were calculated as 0.03 and 0.05, respectively which indicates the consistency of the fuzzy pairwise comparison matrix of the main criteria. for each of the fuzzy pairwise comparison matrices of the subcriteria for which this ratio was computed, the consistency ratio was equal to or less than 0.1. to conduct the classic ahp, the pairwise comparisons matrices which were previously filled in by the experts using the linguistic values of table 4, were quantified based on saaty’s scale and merged based on the arithmetic mean. due to the simplicity of the ahp method, its steps are not discussed in detail; however, the weights of the criteria and inconsistency ratio were calculated using the super decision software and illustrated in figure 4. figure 4 results of ahp method table 6 shows the results of the classic and fuzzy ahp methods based on the weights of the six main criteria. the findings indicate that the weights of all criteria are relatively close in the two methods. figure 5 also illustrates the radar chart for weights of the main criteria. ijahp article: nahavandi, homayounfar, daneshvar/a fuzzy analytical hierarchy process for evaluation of knowledge management effectiveness in research centers international journal of the analytic hierarchy process 19 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.978 table 6 importance weights of the six criteria with using ahp and fuzzy ahp fuzzy ahp ahp criteria 0.34 0.31 human resources 0.27 0.26 knowledge sharing 0.04 0.06 leadership and center structure 0.28 0.25 knowledge utilization and updating 0.02 0.06 knowledge creation and acquisition 0.05 0.07 knowledge storage and security figure 5 radar chart for weights of the six criteria based on ahp and fuzzy ahp methods the weights of the subcriteria in both the ahp and fuzzy ahp were calculated and are summarized in table 6. human resources knowledge sharing leadership and center structure knowledge utilization and updating knowledge creation acquisition knowledge storage and security ahp weights fahp weights ijahp article: nahavandi, homayounfar, daneshvar/a fuzzy analytical hierarchy process for evaluation of knowledge management effectiveness in research centers international journal of the analytic hierarchy process 20 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.978 table 6 importance weights of subcriteria using ahp and fuzzy ahp fuzzy ahp ahp subcriteria no. fuzzy ahp ahp subcriteria no. 0.12 0.10 c7 18 0.14 0.22 h1 1 0.12 0.03 t1 19 0.13 0.17 h2 2 0.20 0.16 t2 20 0.24 0.18 h3 3 0.15 0.27 t3 21 0.23 0.18 h4 4 0.16 0.16 t4 22 0.26 0.25 h5 5 0.21 0.22 t5 23 0.13 0.17 l1 6 0.16 0.16 t6 24 0.19 0.29 l2 7 0.21 0.29 s1 25 0.19 0.14 l3 8 0.23 0.23 s2 26 0.12 0.17 l4 9 0.26 0.21 s3 27 0.14 0.09 l5 10 0.29 0.27 s4 28 0.23 0.14 l6 11 0.14 0.17 u1 29 0.19 0.18 c1 12 0.12 0.19 u2 30 0.20 0.18 c2 13 0.20 0.10 u3 31 0.14 0.17 c3 14 0.17 0.28 u4 32 0.16 0.16 c4 15 0.20 0.12 u5 33 0.09 0.11 c5 16 0.16 0.14 u6 34 0.09 0.09 c6 17 4.2 measuring the rcs’ km effectiveness an approach presented by wen (2009) was used to acquire scores of km effectiveness. according to this approach, after weighting 34 sub-criteria to obtain the standard z value (which is between 0 and 1), we normalized the weights. standardization makes it possible to cope with problems that arise from the fact that the measurement units of these 34 criteria are different from each other. the standard z is used to normalize the weights. the sum of each indicator multiplied by the related weight was the standardized value of the respective criterion. the sum of each criterion multiplied by the related weight created the final score of km effectiveness. to acquire the score of km effectiveness in the mentioned method, the results of the fuzzy ahp were combined with the saw method. therefore, by using equation 12, the score of the influence evaluation model of km was calculated. ai =∑wij ×⁡zij (12) where zij is the normalized value of the i th criterion and the j th index; wij is the relative weight of the i th criterion and the j th index, and ai is the score of the i th criterion (equation 13). ∑wij 5 i=1 ×ai (13) where ai is the standardized value of the i th criterion; wij is the relative weight of the i th criterion, and e is the total score of km effectiveness of iran’s rcs. to execute the introduced framework, we have attempted to rank and compare nine iranian rcs. the final results are summarized in table 7. ijahp article: nahavandi, homayounfar, daneshvar/a fuzzy analytical hierarchy process for evaluation of knowledge management effectiveness in research centers international journal of the analytic hierarchy process 21 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.978 table 7 results of ranking nine rcs in iran code of rcs effectiveness score rank rc2 0.007226 1 rc5 0.006598 2 rc7 0.006446 3 rc1 0.006435 4 rc6 0.006429 5 rc4 0.006236 6 rc3 0.005710 7 rc8 0.005341 8 rc9 0.005264 9 5. conclusion although km effectiveness leads to an increase in efficiency in organizational performance and gives a competitive advantage, in order to manage it successfully, effectiveness measures and indicators must be identified and defined. few systematic studies have been done regarding km effectiveness in rcs. the ahp is the most appropriate method for this problem because of the subjective and intangible nature of the attributes that are used in the evaluation. the opinions of experts and specialists in areas like it and at different management levels (km researchers, etc.) should be used to present a comprehensive model for assessing km effectiveness in rcs. since experts prefer natural language expressions rather than sharp numerical values in their assessments, in the present research triangle fuzzy numbers (tfns) were used for comparison in the form of the extent analysis method. this article presents a framework with six criteria and 34 sub-criteria to evaluate the km effectiveness in rcs. with respect to the results derived from the fuzzy ahp, the criterion human resources (0.34) ranked first. the other five criteria in descending order of importance are knowledge sharing (0.28), leadership and center structure (0.27), knowledge utilization and updating (0.05), knowledge creation and acquisition (0.04) and knowledge storage and security (0.01). furthermore, the number of employees’ proposals for improvement subcriterion (0.26) within the criterion human resources and the learning atmosphere in rc subcriterion (0.23) within the criterion leadership and center structure, the number of international patents subcriterion (0.20) within the criterion knowledge creation and acquisition, the amount of tasks and duties that personnel complete via computers subcriterion (0.21) within the criterion knowledge sharing, the degree of standardization in documentation subcriterion (0.29) within the criterion knowledge storage and security, the establishing customers' database (0.20) and organized interval evaluation subcriteria (0.20) within the criterion knowledge utilization and updating received the highest level of importance among all of the subcriteria. this framework can be used in any country; however, the obtained results reflect the situation of km in iranian rcs. moreover, such a model could be used for a wide variety of research organizations in future research. finally, we suggest developing this type of ijahp article: nahavandi, homayounfar, daneshvar/a fuzzy analytical hierarchy process for evaluation of knowledge management effectiveness in research centers international journal of the analytic hierarchy process 22 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.978 framework in the future using a combination of fuzzy methods and other madm methods. the results of the research provide some implications to consider. from a practical point of view, this research contributes to theory by providing a framework for measuring km effectiveness in iranian rcs. despite the considerable attention to the evaluation of km effectiveness, little is known about the importance of a systematic and comprehensive criteria determination and professionals’ judgments, particularly in rcs. this study suggests the suitable criteria of both a qualitative and quantitative nature which promotes the validation of the km effectiveness measurement. the applied classic and fuzzy ahp methods make it possible to analyze the sensitivity of the results and make better assessments. from the theoretical point of view, the main criteria to measure rcs effectiveness are human resources, leadership and center structure, knowledge creation and acquisition, knowledge storage and security and knowledge sharing. organizations that produce knowledge, especially rcs, need to measure km performance at different stages in their life cycle. the effectiveness of the km system strongly relies on these criteria which create synergy in conducting research projects and optimal use of the resources. therefore, in this study an attempt was made to evaluate the influence factors of km in rcs by presenting a model to improve the performance of this organization. concerning limitations, we propose conducting further analysis to identify additional variables regarding the effectiveness of the rcs performance. in addition, it may be interesting to determine rcs performance through alternative variables (e.g., number of research projects) or even study the performance as a latent variable related to multiple measured variables. for future research, we recommend defining a framework for structuring and evaluating rcs’ activities according to the new approaches. ijahp article: nahavandi, homayounfar, daneshvar/a fuzzy analytical hierarchy process for evaluation of knowledge management effectiveness in research centers international journal of the analytic hierarchy process 23 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.978 references abbas, j. and sağsan, m. (2019). impact of knowledge management practices on green innovation and corporate sustainable development: a structural analysis, journal of cleaner production, 229, 611-620. doi: https://doi.org/10.1016/j.jclepro.2019.05.024 abusweilem, m.a., and abualous, s. (2019). the impact of knowledge management process and business intelligence on organizational performance, management science letters, 9, 2143–2156. doi:10.5267/j.msl.2019.6.020 adachi, e. (2009). knowledge management in a public research organization: reform of a planning section service by action research, picmet proceedings, portland, oregon usa. adaileh, m.j., alrawashdeh, m., elrehail, h., and aladayleh, k.j (2020). assessing the nexus between knowledge management and firm performance: a data article. data in brief, 32, 106283 doi: https://doi.org/10.1016/j.dib.2020.106283 agostino, d., arena, m., azzone, g., dal molin, m., masella, c. (2012). developing a performance measurement system for public research centres, international journal of business science and applied management, 7(1), 40-60. al ahbabi, s.a., singh, s., and balasubramanian, s. (2018). employee perception of impact of knowledge management processes on public sector performance. journal of knowledge management, 21(1), 351-373. doi: http://dx.doi.org/10.1108/jkm-08-20170348 alavi, m., leidner, d.e. (2001). knowledge management and knowledge management systems: conceptual foundations and research issues. mis quarterly, 25(1), 107-136. doi: https://doi.org/10.2307/3250961 albassam, b. a. (2019). building an effective knowledge management system in saudi arabia using the principles of good governance, resources policy, 64, 101531. doi: https://doi.org/10.1016/j.resourpol.2019.101531 ale, m.a., toledo, c.m., chiotti, o., and galli. m.r. (2014). a conceptual model and technological support for organizational knowledge management, science of computer programming, 95(1), 73–92. doi: https://doi.org/10.1016/j.scico.2013.12.012 anjaria, k. (2020). negation and entropy: effectual knowledge management equipment for learning organizations, expert systems with applications, 157(1), 113497. https://doi.org/10.1016/j.eswa.2020.113497 asiaei, k., jusoh, r., and bontis, n. (2018). intellectual capital and performance measurement systems in iran, journal of intellectual capital, 19(2), 294-319. doi: http://dx.doi.org/10.1108/jic-11-2016-0125 https://doi.org/10.1016/j.jclepro.2019.05.024 http://dx.doi.org/10.5267/j.msl.2019.6.020 https://doi.org/10.1016/j.dib.2020.106283 http://dx.doi.org/10.1108/jkm-08-2017-0348 http://dx.doi.org/10.1108/jkm-08-2017-0348 https://doi.org/10.2307/3250961 https://doi.org/10.1016/j.resourpol.2019.101531 https://doi.org/10.1016/j.scico.2013.12.012 https://doi.org/10.1016/j.eswa.2020.113497 http://dx.doi.org/10.1108/jic-11-2016-0125 ijahp article: nahavandi, homayounfar, daneshvar/a fuzzy analytical hierarchy process for evaluation of knowledge management effectiveness in research centers international journal of the analytic hierarchy process 24 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.978 barão, a. vasconcelos, j.b.d. rocha, á. and pereira, r. (2017). a knowledge management approach to capture organizational learning networks, international journal of information management, 37(6), 735–740. doi: http://dx.doi.org/10.1016/j.ijinfomgt.2017.07.013 boender, c.g.e. de grann, j.g. and lootsma, f. a. (1989). multicriteria decision analysis with fuzzy pairwise comparison, fuzzy sets and systems, 29(2), 133–143. https://doi.org/10.1016/0165-0114(89)90187-5 bozbura, f.b. beskese, a. and kahraman, c. (2007). prioritization of human capital measurement indicators using fuzzy ahp, expert systems with applications, 32(4), 1100-1112. doi: https://doi.org/10.1016/j.eswa.2006.02.006 bremser, w.g., and barsky, n.p. (2004). utilizing the balanced scorecard for r&d performance measurement. r & d management, 34(3), 229-238. doi: https://doi.org/10.1111/j.1467-9310.2004.00335.x breznik, k. (2018). knowledge management – from its inception to the innovation. 14 th international symposium in management. procedia social and behavioral sciences, 238, 141-148. doi: https://doi.org/10.1016/j.sbspro.2018.03.017 buckley, j.j. (1985). fuzzy hierarchical analysis, fuzzy sets and systems, 17(3), 233– 247. doi: https://doi.org/10.1016/0165-0114(85)90090-9 buran, b., and erçek, m. (2022). public transportation business model evaluation with spherical and intuitionistic fuzzy ahp and sensitivity analysis, expert systems with applications, 204(15), 117519. doi: https://doi.org/10.1016/j.eswa.2022.117519 byukusenge, e., and munene, j.c. (2017). knowledge management and business performance: does innovation matter?, cogent business & management, 4(1), 1368434. doi: 10.1080/23311975.2017.1368434 cabrera, e. and cabrera, a. (2005). fostering knowledge-sharing through people management practices, international journal of human resource management, 16(5), 720–735. doi: http://dx.doi.org/10.1080/09585190500083020 castrogiovanni, g. ribeiro-soriano, d. mas-tur, a. and roig-tierno, n. (2016). where to acquire knowledge: adapting knowledge management to financial institutions, journal of business research, 69(5), 1812-1816. doi: https://doi.org/10.1016/j.jbusres.2015.10.061 chang, d.y. (1996). applications of the extent analysis method on fuzzy ahp, european journal of operational research, 95(3), 649–655. doi: https://doi.org/10.1016/0377-2217(95)00300-2 chang, m.y. hung, y.c. yen, d.c. and tseng, p.t.y. (2009). the research on the critical success factors of knowledge management and classification framework project in http://dx.doi.org/10.1016/j.ijinfomgt.2017.07.013 https://doi.org/10.1016/0165-0114(89)90187-5 https://www.sciencedirect.com/science/article/abs/pii/s0957417406000868#! https://www.sciencedirect.com/science/article/abs/pii/s0957417406000868#! https://doi.org/10.1016/j.eswa.2006.02.006 https://doi.org/10.1111/j.1467-9310.2004.00335.x https://doi.org/10.1016/j.sbspro.2018.03.017 https://doi.org/10.1016/0165-0114(85)90090-9 https://doi.org/10.1016/j.eswa.2022.117519 http://dx.doi.org/10.1080/23311975.2017.1368434 http://dx.doi.org/10.1080/09585190500083020 https://doi.org/10.1016/j.jbusres.2015.10.061 https://doi.org/10.1016/0377-2217(95)00300-2 ijahp article: nahavandi, homayounfar, daneshvar/a fuzzy analytical hierarchy process for evaluation of knowledge management effectiveness in research centers international journal of the analytic hierarchy process 25 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.978 the executive yuan of taiwan government, expert systems with applications, 36(3), 5376–5386. doi: https://doi.org/10.1016/j.eswa.2008.06.060 chang, t.h. and wang, t.c. (2009). using the fuzzy multi criteria decision making approach for measuring the possibility of successful knowledge management, information sciences, 179(4), 355–370. doi: https://doi.org/10.1016/j.ins.2008.10.012 chen, c.j. and huang, j.w. (2009). strategic human resource practices and innovation performancethe mediating role of knowledge management capacity, business research, 62(1), 104-114. doi: https://doi.org/10.1016/j.jbusres.2007.11.016 chiesa, v., and frattini, f. (2007). exploring the differences in performance measurement between research and development: evidence from a multiple case study, r&d management, 37(4), 283-301. doi: https://doi.org/10.1111/j.1467-9310.2007.00476.x chin, k.s. lo, k.c. and leung, j.p.f. (2010). development of user-satisfaction-based knowledge management performance measurement system with evidential reasoning approach, expert systems with applications, 37(1), 366–382. doi: https://doi.org/10.1016/j.eswa.2009.05.051 chuang, s.h. (2004). a resource-based perspective on knowledge management capability and competitive advantage: an empirical investigation, expert systems with applications, 27(3), 459–465. doi: https://doi.org/10.1016/j.eswa.2004.05.008 cui, a.s. griffith, d.a. and cavusgil, s.t. (2005). the influence of competitive intensity and market dynamism on knowledge management capabilities of mnc subsidiaries, international marketing, 13(3), 32–53. doi: http://dx.doi.org/10.1509/jimk.13.3.32 darvishi, d., and darvishi, h. (2019). assessing the effectiveness of knowledge management using analytic network process, international journal of applied operational research, 9(1), 31-40. doi: http://ijorlu.liau.ac.ir/article-1-588-en.html davis, s. and botkin, j. (1994). the coming of knowledge-based business, harvard business review, 72(5), 193-70. dóci, g. rohracher, h. and kordas, o. (2022). knowledge management in transition management: the ripples of learning, sustainable cities and society, 78, 103621. doi: https://doi.org/10.1016/j.scs.2021.103621 donate, m.j. and sánchez de pablo, j.d. (2015). the role of knowledge-oriented leadership in knowledge management practices and innovation, journal of business research, 68(2), 360–370. doi: http://dx.doi.org/10.1016/j.jbusres.2014.06.022 duhon, b. (1998). it's all in our heads. inform, 12, 8-13. https://doi.org/10.1016/j.eswa.2008.06.060 https://doi.org/10.1016/j.ins.2008.10.012 http://www.sciencedirect.com/science?_ob=articleurl&_udi=b6v7s-4rs9sp4-3&_user=5700344&_coverdate=01%2f31%2f2009&_alid=1406861279&_rdoc=1&_fmt=high&_orig=search&_cdi=5850&_sort=r&_st=0&_docanchor=&_ct=3&_acct=c000068166&_version=1&_urlversion=0&_userid=5700344&md5=d81f98abf7be5c69d61465388ce3b7d9 http://www.sciencedirect.com/science?_ob=articleurl&_udi=b6v7s-4rs9sp4-3&_user=5700344&_coverdate=01%2f31%2f2009&_alid=1406861279&_rdoc=1&_fmt=high&_orig=search&_cdi=5850&_sort=r&_st=0&_docanchor=&_ct=3&_acct=c000068166&_version=1&_urlversion=0&_userid=5700344&md5=d81f98abf7be5c69d61465388ce3b7d9 https://doi.org/10.1016/j.jbusres.2007.11.016 https://doi.org/10.1111/j.1467-9310.2007.00476.x https://doi.org/10.1016/j.eswa.2009.05.051 https://doi.org/10.1016/j.eswa.2004.05.008 http://dx.doi.org/10.1509/jimk.13.3.32 http://ijorlu.liau.ac.ir/article-1-588-en.html https://doi.org/10.1016/j.scs.2021.103621 http://dx.doi.org/10.1016/j.jbusres.2014.06.022 ijahp article: nahavandi, homayounfar, daneshvar/a fuzzy analytical hierarchy process for evaluation of knowledge management effectiveness in research centers international journal of the analytic hierarchy process 26 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.978 ermine, j-l. (2010). methods and tools for knowledge management in research centers, electronic journal of knowledge management, 8(3), 293-306. fan, z.p. feng, b. sun, y.h., and ou, w. (2009). evaluating knowledge management capability of organizations: a fuzzy linguistic method, expert systems with applications, 36(2), 3346–3354. doi: https://doi.org/10.1016/j.eswa.2008.01.052 gaviria-marin, m., merigo, j.m., and popa, s. (2018). twenty years of the journal of knowledge management: a bibliometric analysis, journal of knowledge management, 22(6), 16551687. doi: http://dx.doi.org/10.1108/jkm-10-2017-0497 gold, a.h. malhotra, a. and segars, a.h. (2001). knowledge management: an organizational capabilities perspective, management information systems, 18(1), 185– 214. doi: https://doi.org/10.1080/07421222.2001.11045669 gogus, o., and boucher, t.o. (1998). a consistency test for rational weights in multicriteria decision analysis with pairwise comparisons, fuzzy sets and systems, 86(2), 129138. doi: https://doi.org/10.1016/0165-0114(95)00410-6 hatefi, m.a. (2021). braw: block-wise rating the attribute weights in madm, computers & industrial engineering, 156, 107274. doi: https://doi.org/10.1016/j.cie.2021.107274 hislop, d. (2003). linking human resource management and knowledge management via commitment: a review and research agenda, employee relations, 25(2), 182-202. doi: https://doi.org/10.1108/01425450310456479 hsieh, p.j. lin, b. and lin, c. (2009), the construction and application of knowledge navigator model (knmtm): an evaluation of knowledge management maturity, expert systems with applications, 36(2), 4087–4100. doi: https://doi.org/10.1016/j.eswa.2008.03.005 joseph, j.v.m. soundararajan, e. jayakumar, c. and somasekharan, m. (2005). knowledge management in a research organization, scientific information resource division, indira gandhi centre for atomic research, kalpakkam: india. kahraman, c., boltürk, e., çevik onar, s., öztay?i, b., and göztepe, k. (2016). multiattribute warehouse location selection in humanitarian logistics using hesitant fuzzy ahp, international journal of the analytic hierarchy process, 8(2), 271-298. doi: https://doi.org/10.13033/ijahp.v8i2.387 kamara, j.m. anumba, c.j. and carrillo, p.m. (2002). a clever approach to selecting a knowledge management strategy, international journal of project management, 20(3), 205–211. doi: https://doi.org/10.1016/s0263-7863(01)00070-9 kavalić, m., nikolić, m., radosav, stanisavljev, s. (2021). influencing factors on knowledge management for organizational sustainability, sustainability, 13(3), 1-18.doi: http://dx.doi.org/10.3390/su13031497 https://doi.org/10.1016/j.eswa.2008.01.052 http://dx.doi.org/10.1108/jkm-10-2017-0497 https://doi.org/10.1080/07421222.2001.11045669 https://doi.org/10.1016/0165-0114(95)00410-6 https://doi.org/10.1016/j.cie.2021.107274 https://doi.org/10.1108/01425450310456479 https://doi.org/10.1016/j.eswa.2008.03.005 https://doi.org/10.13033/ijahp.v8i2.387 https://doi.org/10.1016/s0263-7863(01)00070-9 http://dx.doi.org/10.3390/su13031497 ijahp article: nahavandi, homayounfar, daneshvar/a fuzzy analytical hierarchy process for evaluation of knowledge management effectiveness in research centers international journal of the analytic hierarchy process 27 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.978 kim, t.h., choi, b., lee, j.n., and sambamurthy, v. (2021). portfolio effects of knowledge management strategies on firm performance: complementarity or substitutability? information & management, 58(4), 103468. https://doi.org/10.1016/j.im.2021.103468. kokkaew, n. jokkaw, n. peansupap, v. and wipulanusat, w. (2022). impacts of human resource management and knowledge management on non-financial organizational performance: evidence of thai infrastructure construction firms, ain shams engineering journal, 13(6), 101750. doi: https://doi.org/10.1016/j.asej.2022.101750 koul, r., and verma, s. (2009). dynamic vendor selection; a fuzzy ahp approach, international journal of the analytic hierarchy process, 4(2), 118-136. doi: https://doi.org/10.13033/ijahp.v4i2.25 kusumastuti, r.d. arviansyah, a. nurmala, n. and wibowo, s.s. (2021). knowledge management and natural disaster preparedness: a systematic literature review and a case study of east lombok, indonesia, international journal of disaster risk reduction, 58, 102223. doi: https://doi.org/10.1016/j.ijdrr.2021.102223 lee, a.h.i a. chen, w.c. and chang, c.j. (2008). a fuzzy ahp and bsc approach for evaluating performance of it department in the manufacturing industry in taiwan, expert systems with applications, 34(1), 96–107. doi: https://doi.org/10.1016/j.eswa.2006.08.022 li, x. and chandra, c. (2007). a knowledge integration framework for complex network management, industrial management and data systems, 107(8), 1089-1109. doi: http://dx.doi.org/10.1108/02635570710822769 liao, s.h. and wub, c. c. (2010). system perspective of knowledge management, organizational learning, and organizational innovation, expert systems with applications, 37(2),1096–1103. doi: https://doi.org/10.1016/j.eswa.2009.06.109 liu, y., eckert, c.m., and earl, c. (2020). a review of fuzzy ahp methods for decisionmaking with subjective judgements, expert systems with applications, 161, 113738. doi: https://doi.org/10.1016/j.eswa.2020.113738 mardani, a. nikoosokhan, s. moradi, m. and doustar, m. (2018). the relationship between knowledge management and innovation performance, journal of high technology management research, 29(1), 1226. doi: https://doi.org/10.1016/j.hitech.2018.04.002 martins, v.w.b., rampasso, i.s., anholon, r., quelhas, o.l.g. and leal filho, w. (2019). knowledge management in the context of sustainability: literature review and opportunities for future research, journal of cleaner production, 229, 489-500. doi: https://doi.org/10.1016/j.jclepro.2019.04.354 martinsons, m. (1997). human resource management applications of knowledge-based systems, international journal of information management, 17(1), 35-53. doi: https://doi.org/10.1016/s0268-4012(96)00041-2 https://doi.org/10.1016/j.im.2021.103468 https://doi.org/10.1016/j.asej.2022.101750 https://doi.org/10.13033/ijahp.v4i2.25 https://doi.org/10.1016/j.ijdrr.2021.102223 https://doi.org/10.1016/j.eswa.2006.08.022 http://dx.doi.org/10.1108/02635570710822769 https://doi.org/10.1016/j.eswa.2009.06.109 https://doi.org/10.1016/j.eswa.2020.113738 https://doi.org/10.1016/j.hitech.2018.04.002 https://doi.org/10.1016/j.jclepro.2019.04.354 http://www.sciencedirect.com/science?_ob=articleurl&_udi=b6vb4-3swt3sn-4&_user=5700344&_coverdate=02%2f28%2f1997&_alid=1406861279&_rdoc=2&_fmt=high&_orig=search&_cdi=5916&_sort=r&_st=0&_docanchor=&_ct=3&_acct=c000068166&_version=1&_urlversion=0&_userid=5700344&md5=89db6d6091bc13bb0463246b72f49b5c http://www.sciencedirect.com/science?_ob=articleurl&_udi=b6vb4-3swt3sn-4&_user=5700344&_coverdate=02%2f28%2f1997&_alid=1406861279&_rdoc=2&_fmt=high&_orig=search&_cdi=5916&_sort=r&_st=0&_docanchor=&_ct=3&_acct=c000068166&_version=1&_urlversion=0&_userid=5700344&md5=89db6d6091bc13bb0463246b72f49b5c https://doi.org/10.1016/s0268-4012(96)00041-2 ijahp article: nahavandi, homayounfar, daneshvar/a fuzzy analytical hierarchy process for evaluation of knowledge management effectiveness in research centers international journal of the analytic hierarchy process 28 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.978 mehdibeigi, n., dehghani, m., and yaghoubi, n.m. (2016). customer knowledge management and organization's effectiveness: explaining the mediator role of organizational agility, procedia social and behavioral sciences, 230, 94 – 103. doi: https://doi.org/10.1016/j.sbspro.2016.09.012 mettänen, p. (2005). design and implementation of a performance measurement system for a research organization, production planning and control, 16(2), pp. 178-188. doi: http://doi.org/10.1080/09537280512331333075 nazim, m., mohammad, c.w., and sadiq, m. (2022). a comparison between fuzzy ahp and fuzzy topsis methods to software requirements selection, alexandria engineering journal, 61(12), 10851-10870. doi: https://doi.org/10.1016/j.aej.2022.04.005 nonaka, i. and toyama, r. (2003). the knowledge-creating theory revisited: knowledge creation as a synthesizing process, knowledge management research & practice, 1(1), 2-10. doi: https://doi.org/10.1057/palgrave.kmrp.8500001 ode, e. and ayavoo, r. (2020). the mediating role of knowledge application in the relationship between knowledge management practices and firm innovation, journal of innovation & knowledge, 5(3), 210-218. doi: https://doi.org/10.1016/j.jik.2019.08.002 ogiela, l. (2015). advanced techniques for knowledge management and access to strategic information, international journal of information management, 35(2), 154–159. doi: http://dx.doi.org/10.1016/j.ijinfomgt.2014.11.006 ohoitimur, j., krejci, j., raco, j. r., raton, y., jamlean, a., welerubun, i., and tanod, r. (2021). strategic management based on buckley’s fuzzy ahp and swot: example of the congregation of the missionaries of the sacred heart, international journal of the analytic hierarchy process, 13(2), 329-354. doi: https://doi.org/10.13033/ijahp.v13i2.871 ohta, r., salomon, v.a.p., and silva, m.b. (2018). selection of the industrial maintenance strategy: classic ahp and fuzzy ahp applications, international journal of the analytic hierarchy process, 10(2) 254-265. doi: https://doi.org/10.13033/ijahp.v10i2.551 okemwa, e.o. (2006). knowledge management in a research organization: international livestock research institute (ilri), libri, 56, 63–72. oltra, v. (2005). knowledge management effectiveness factors: the role of hrm knowledge management, journal of knowledge management, 9(4), 70-86. doi: https://doi.org/10.1108/13673270510610341 ozan, c. (2008). on the order of the preference intensities in fuzzy ahp, computers & industrial engineering, 54(4), 993–1005. doi: https://doi.org/10.1016/j.cie.2007.11.010 http://doi.org/10.1080/09537280512331333075 https://doi.org/10.1016/j.aej.2022.04.005 https://doi.org/10.1057/palgrave.kmrp.8500001 https://doi.org/10.1016/j.jik.2019.08.002 http://dx.doi.org/10.1016/j.ijinfomgt.2014.11.006 https://doi.org/10.13033/ijahp.v13i2.871 https://doi.org/10.13033/ijahp.v10i2.551 https://www.emerald.com/insight/publication/issn/1367-3270 https://doi.org/10.1108/13673270510610341 https://doi.org/10.1016/j.cie.2007.11.010 ijahp article: nahavandi, homayounfar, daneshvar/a fuzzy analytical hierarchy process for evaluation of knowledge management effectiveness in research centers international journal of the analytic hierarchy process 29 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.978 patil, s.k. and kant, r. (2014). a fuzzy ahp-topsis framework for ranking the solutions of knowledge management adoption in supply chain to overcome its barriers, expert systems with applications, 41(2), 679-693. doi: https://doi.org/10.1016/j.eswa.2013.07.093 rezaee kelidbari, h.r., homayounfar, m., and alavi foumani, s.f. (2016). a combined group ea-promethee method for a supplier selection problem. iranian journal of optimization, 8(2), 87-100. saaty, t.l. (1980). the analytic hierarchy process. mcgraw-hill, new york. schniederjans, d.g., curado, c., and khalajhedayati, m. (2020). supply chain digitisation trends: an integration of knowledge management, international journal of production economics, 220, 107439. doi: https://doi.org/10.1016/j.ijpe.2019.07.012 smits, m. and de moor, a. (2004). effective knowledge management in knowledgeintensive organizations, proceedings of organizational knowledge, learning and capabilities, innsbruck. tan, c. l., and nasurdin, a. m. (2011). human resource management practices and organizational innovation: assessing the mediating role of knowledge management effectiveness, electronic journal of knowledge management, 9(2).155–167. toloie-eshlaghy, a. and homayonfar, m. (2011). mcdm methodologies and applications: a literature review from 1999 to 2009, research journal of international studies, 21, 86-137. tseng, s.m. (2008). knowledge management system performance measure index, expert systems with applications, 34(1), 734–745. doi: https://doi.org/10.1016/j.eswa.2006.10.008 tseng, s-m. (2010). the correlation between organizational culture and knowledge conversion on corporate performance, journal of knowledge management, 14(2), 269284. doi: http://dx.doi.org/10.1108/13673271011032409 ulrich, d. (1998). intellectual capital = competence × commitment, sloan management review, 39(2), 15-26. urban, b. and matela, l. (2022). the nexus between innovativeness and knowledge management: a focus on firm performance in the hospitality sector, international journal of innovation studies, 6(1), 26-34. doi: https://doi.org/10.1016/j.ijis.2021.12.002 vaio, a.d. palladino, r. pezzi, a. and kalisz, d.e. (2021). the role of digital innovation in knowledge management systems: a systematic literature review, journal of business research, 123, 220-231. doi: https://doi.org/10.1016/j.jbusres.2020.09.042 van laarhoven, p.j.m. and pedrycz, w. (1983). a fuzzy extension of saaty’s priority theory, fuzzy sets and systems, 11(1-3), 229–241. doi: https://doi.org/10.1016/s0165-0114(83)80082-7 https://doi.org/10.1016/j.eswa.2013.07.093 https://doi.org/10.1016/j.ijpe.2019.07.012 https://scholar.google.com/scholar?oi=bibs&cluster=121448902355503322&btni=1&hl=en https://scholar.google.com/scholar?oi=bibs&cluster=121448902355503322&btni=1&hl=en https://doi.org/10.1016/j.eswa.2006.10.008 https://doi.org/10.1016/j.ijis.2021.12.002 https://doi.org/10.1016/j.jbusres.2020.09.042 https://doi.org/10.1016/s0165-0114(83)80082-7 ijahp article: nahavandi, homayounfar, daneshvar/a fuzzy analytical hierarchy process for evaluation of knowledge management effectiveness in research centers international journal of the analytic hierarchy process 30 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.978 wang, t.c., and chang, t.h. (2007). forecasting the probability of successful knowledge management by consistent fuzzy preference relations, expert systems with applications, 32(3), 801–813. doi: https://doi.org/10.1016/j.eswa.2006.01.021 wen, y.f. (2009). an effectiveness measurement model for knowledge management, knowledge-based systems, 22(5), 363–367. doi: https://doi.org/10.1016/j.knosys.2009.02.007 winch, g. and schneider, e. (1993). managing the knowledge-based organization: the case of architectural practice, management studies, 30(6), 923–37. doi: https://doi.org/10.1111/j.1467-6486.1993.tb00472.x wu, w.w. and lee, y.t. (2007). selecting knowledge management strategies by using the analytic network process, expert systems with applications, 32(3), 841–847. doi: https://doi.org/10.1016/j.eswa.2006.01.029 yap, j.b.h. lim, b.l. and skitmore, m. (2022). capitalizing knowledge management (km) for improving project delivery in construction, ain shams engineering journal, 13(6) 101790. doi: https://doi.org/10.1016/j.asej.2022.101790 yun, e.k. (2013). predictors of attitude and intention to use knowledge management system among korean nurses, nurse education today, 33(12), 1477–148. doi: https://doi.org/10.1016/j.nedt.2013.05.018 https://doi.org/10.1016/j.eswa.2006.01.021 https://doi.org/10.1016/j.knosys.2009.02.007 https://doi.org/10.1111/j.1467-6486.1993.tb00472.x https://doi.org/10.1016/j.eswa.2006.01.029 https://doi.org/10.1016/j.asej.2022.101790 https://doi.org/10.1016/j.nedt.2013.05.018 ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 228 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 integrations and applications of analytic hierarchy process with data envelopment analysis – a literature review saurabh pradhan birla institute of management technology, greater noida, uttar pradesh, india saurabh.pradhan_fpm17@bimtech.ac.in maryam olfati school of mathematics and information science, guangzhou university, guangzhou, china maryam.olfati@vsb.cz prof. gokulananda patel* birla institute of management technology, greater noida, uttar pradesh, india gn.patel@bimtech.ac.in abstract the scope of the ahp (analytic hierarchy process) and the dea (data envelopment analysis) is widened when they are integrated to create the deahp. this method has boundless flexibility and varied applicability. furthermore, the scope, flexibility, and applicability are also increased when deahp is integrated with other approaches and methods. this paper reviews the literature that discusses this integration with deahp. on the basis of a defined search criterion, related articles appearing in international journals from 2000 to 2018 were collected and analyzed so that the following five questions could be answered: (1) which approach out of the various integrated-deahp approaches was most prevalent and got the utmost attention?, (2) which of the areas of application were most often studied and found their application with integrated deahp approaches?, (3) which of the specific problems were most frequently applied to integrated deahp approaches?, (4) what are the publication trends for the papers considering the integrated deahp approaches?, and (5) which jourmals of international standards published what number of journal articles and among them which has the highest number of articles on the approach of integrated deahp? this literature review shows that the integrated deahps are able to tackle the inadequacies of the stand-alone deahps. this article allows the researcher to think about new application areas and specific problem areas where an integrated deahp approach could be effectively and efficiently applied. the novelty of this paper lies in the fact that this is a one of a kind literature review that considers the integrated deahp approach along with stand-alone deahp with its applications. mailto:saurabh.pradhan_fpm17@bimtech.ac.in mailto:maryam.olfati@vsb.cz mailto:gn.patel@bimtech.ac.in ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 229 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 keywords: integrated data envelopment analysis-analytic hierarchy process; ahp; dea; deahp; literature review 1. introduction lately, there have been many application-based papers developed in the field of integrated approaches of the analytic hierarchy process and data envelopment analysis (dea/ahp). in this globalized society, changing business scenarios demand the most efficient and effective ways to manage an organization’s business operations in every application area related to multi-criteria decision making (mcdm). this paper will aid academic and industry researchers and decision makers in utilizing the integrated deahp approach for various business problems in different application areas. this literature review identifies advances and new trends of the stand-alone and integrated deahp and its applications and problems from the business perspective between 2000 to 2018. this integration of ahp/dea with other approaches is widely applied in the field of performance and efficiency measurement and optimization. besides these, it also finds its application in quality measurement and ranking of different products or services in every sector namely education, engineering, manufacturing, government, management, industry, social, political, personal and even sports. for every organization, performance is a multi-dimensional phenomenon that addresses the quality, efficiency, cost, delivery, and flexibility aspects of the organization which relate to its improved performance. organizational performance needs to be flexible, robust and competitive enough for the organization to survive and grow and gain an edge over competitors in the dynamic and vibrant scenario of the global and national economy. for instance, measurements of efficiency, quality, performance, and rank have been addressed using numerous approaches by a number of different researchers with diversified levels and dimensions for consideration. a various number of tools and techniques for the aforementioned indicators have been used which include the analytic network process (anp), analytic hierarchy process (ahp), data envelopment analysis (dea), goal programming (gp), linear programming (lp), slacks-based measure (sbm), assurance region (ar), grey relational analysis (gra), technique for ordering preference by similarity to ideal solution (topsis), total cost of ownership (tco) quality deployment function (qfd), balanced scorecard (bsc), and suppy chain operations reference (scor) model, to name a few. some structural modeling approaches include interpretive structural modelling (ism), decision making trial and evaluation laboratory (dematel), and structural equation modelling (sem). benchmarking of performance emphasizes both the inter and intra-organizational levels. recently, integrated approaches (ahpdea model, delphi-ahp-topsis, fuzzy-ahp-fuzzy-topsis, bsc-anpdematel, bsc-ahp, bsc-ism-anp, scor-bsc, dematelanpvikor, and dependence-based interval-valued evidential reasoning (dier)-bsc) have also been propounded for the measurement of performance and analysis. the most popular set of tools for managers engaged in multi-criteria decision making are dea, ahp, and anp. there have been quite a few examples in the past literature of integrated approaches with deahp, such as deahp-fuzzy, deahp-lp, deahpgp, deahp-simulation, deahp-abc, eahp-sbm, deahp-tco, dea-vahp, deahp-ar, deahp-gower plot, deahp-gra, delphi-deahp-lp, deahpijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 230 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 fuzzy-qfd-lp, deahp-topsis-fuzzy, and 2-level som-deahp-ar, to name a few. before the integrated dea/ahp was introduced, the stand-alone ahp was sufficient for a wide variety of fields which are related to mcdm (multi-criteria decisionmaking). thomas l. saaty introduced ahp in the 1970s. the ahp was developed by many researchers such as sinuany-stern, mehrez, and hadad (sinuany-stern et al., 2000) and thomas l. saaty (saaty, 2004). the ahp is a subjective method which helps analyze both qualitative and quantitative criteria to generate important weights of the decision criterias and the relative performance measures of the alternatives in terms of each individual’s decision criteria. ho (2008) noticed that when the ahp is integrated with other approaches, it leads to a far more practical and trustworthy decisions than with the stand-alone ahp. hence, the attention shifted in the direction of integrated-ahp and its realistic applications. there have been only 5 publications which reviewed the literature about ahp in the last decade (2007-2016), which is quite limited in number. ho (2008) reviewed 66 papers from 1997 to 2006, and concluded that approaches like integrated-ahp-qfd and integrated-ahp-gp were the most extensively applied approaches. the paper also observed that the logistics sector was the most diverse area of application for the integrated-ahp approach. ho & ma (2018) followed the previous work by comparing the papers in the 2008 article with the 88 papers published between 2007 and 2016. according to ho (2008), the ahp is one of the most widespread mcdm approaches. however, the ahp has been criticized to a great extent for the problem of rank reversal. the eigenvector method, which is used for determing the priorities in ahp, has been criticized by bana e costa and vansnick (2008) because it violates the condition of order-preservation (cop). the numerical examples given by bana e costa and vansnick were examined by wang, chin, and luo (2009). they reexamined the concept of oveall judgements, and it was clearly shown that the criticisms were not justified. the criticism of bana e costa and vansnick has also been argued by kulakowski (2015). this article argued that the problem of the prioritization procedure followed in the ahp was not inherent, rather the issue arose because of the inconsistencies which were found between the elements of the pairwise-comparison matrix. the inconsistency measurement which was taken for the matrix as a wholes does not detect the inconsistencies that are present between the elements of the pairwise-comparison matrix (kulakowski, 2015). however, on the contrary, the dea is a data-oriented approach which is in a way based on mathematical programming. charnes et. al. (1978) introduced dea, and it is one of the most efficient methods that is primarlily used for evaluating the relative efficiency of dmus (decision making units). the performance evaluations of scale efficiency, technical efficiency, allocative efficiency, and cost efficiency are usually carried out using this approach. the objective function value of a linear programming model is basically efficiency. there are many bibliographies in the area of dea in the existing literature, e.g. emrouznejad et al. (2008), emrouznejad and thanassoulis (1996a, 1996b, 1997), seiford (1994, 1997), gattoufi et al. (2004a, 2004b), and tavares (2002). from 1978 to 2016, there were 9881 dea-related articles which were published only in journals (excluding any articles published as a chapter, book or working paper). 94% of the ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 231 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 papers related to dea were published by less than four authors, and about 34% of the papers have two authors. energy, industry, banking, education, and healthcare were found to be the most popular application areas of dea (emrouznejad & yang, 2017). sinuany-stern et al. (2000) claimed that the key objective of the ahp-dea model is to eliminate the subjective evaluation of ahp and to overcome the ranking inefficiency of dea. with the ahp-dea approach, the judgment matrix is formed using basic ahp models and then dea is used to rank the dmus (decision making units). as per the aforementioned paper, this method eliminates the subjective evaluation of ahp and overcomes the ranking inefficiency of dea. first, the dea is run separately for each of the units. second, the pairwise-evaluation matrix which was generated in stage one is utilized to rank-scale the units using the ahp. both the ahp and dea are commonly used and have limitations. the evolution of the ahp was comprehensively detailed in ho (2008). similarly, the literature review in seiford (1996) outlines the evolution of the dea. even though ho (2008) and seiford (1996) have made a significant contribution to the ahp and dea literature as stand-alone approaches, there are two significant knowledge gaps that were the motivation for conducting this study. first, all of the works mentioned in the two previous studies include approaches that are integrated with ahp and dea in a stand-alone manner. with the onset of dea/ahp (deahp), which overcomes the limitations of ahp, several other approaches seem to fit with deahp streamlined to various problem areas and application areas. second, both of the review articles mentioned previously primarily emphasize the commonly studied ahp with the integration of other approaches and the stand-alone ahp approach during a particular time period. also, these review articles do not show a relative comparative analysis and do not analyze the integrated approaches application-wise, problem-wise, trendwise, journal-wise, and even popularity-wise. in order to fill these two gaps, an exhaustive review of highly relevant journal articles between 2000 to 2018 have been presented by this paper to analyze the applications and integrations of deahp. in this paper, we consider the literature concerning dea/ahp between 2000 and 2018. therefore, 72 papers have been studied with varied approaches and applications that include a diverse set of practical problems. the application area is very wide because of the flexibility and ease of use with a variety of specific problems. this combination of different approaches with deahp (also referred to as integrated deahp approaches) is able to make a decision with more accurately than the stand-alone deahp. these approaches were selected because of their wide applicability, popularity, and accomplishment in making decisions. the mechanism of the deahp is described briefly in the next section. the tools integrated with the deahp and their applications are critically explained in the subsequent sections. analysis of the stand-alone-deahp approaches and integrateddeahp approaches, suggestions which are applicable for future work, and conclusions for this work on literature are discussed in the last few sections. 2. deahp the deahp consists of six stages, which are developed into a flowchart as shown in figure 1. stage 1 breaks down a complex mcdm problem into its constituent parts. in these parts, every criterion, sub-criteria, and the alternatives are arranged into ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 232 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 multiple hierarchical levels. subsequently, in stage 2, each cluster is compared in the same level in a pairwise manner which may be based on the decision-maker’s knowledge and experience. some degree of inconsistency may occur in it since the comparisons are carried out through personal or subjective judgments. stage 3 constructs the dea problem where the efficiency is calculated for each criterion taken as dmu. stage 4 identifies if all the hierarchical levels (criteria, sub-criteria, and alternatives) have finished comparision. if the answer is “no”, then a pairwise comparison matrix is developed again. if the answer is “yes”, then it moves to stage 5 of synthetization, where the efficiency of the alternatives is calculated. finally, stage 6 develops the overall priority ranking where the judgments can then be synthesized to determine the priority ranking of each criterion and its alternatives. figure 1 flowchart of the deahp (data envelopment analysis – analytic hierarchy process) ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 233 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 3. research methodology since this literature review focuses on the recently created integrated approach of the stand-alone deahp, all the journal articles are published between 2000 to 2018. the following illustration in figure 2 represents the research methodology. figure 2 flowchart of the research methodology published by ho & ma (2018) this paper followed a very lucid search, inclusion, exclusion, shortlisting and review given by ho & ma (2018). initially, the search terms were defined, and then keywords/abstracts/titles were searched using the terms “integrated data envelopment analysis-analytic hierarchy process” (shortened to keywords like deahp, ahp/dea, integrated deahp, integrated ahp/dea, integrated dea/ahp) which were utilized in the process of searching within the databases. second, a wide variety of academic databases were used to find relevant journal articles. these databases included emerald, ebscohost, elsevier, sciencedirect, jstor, taylor & francis, proquest, world scientific, scientific research, ieeexplore, mdpi, metapress, ingenta, wiley, and springer. only prps (peerreviewed papers) which were printed and published in english and in international level journals were chosen to ensure an elevated relevance level. conference papers, book chapters, textbooks, dissertations of doctoral and master’s level, and notes were not considered for this process of review. in the third step, the journal articles were screened on the basis of a determined criterion. using the criterion, all the abstracts of the articles along with relevant informative portions of the papers were scrutinized to identify whether the article addressed the integrated-deahp approach or the stand-alone-dea/ahp approach with its applications rather than addressing only ahp or dea or integrated ahp without dea or integrated dea without ahp. however, it must be kept in mind that ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 234 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 the journal articles which proposed approaches for re-modifying deahp, like a linear programming method for the generation of weights from a pairwisecomparison matrix which tends to be the most favorable (wang et. al., 2008); weight derivation in the case of dea and aggregation in the case of ahp (ramanathan, 2006); and ranking decision-making units (dmus) using approach of integrated fuzzy-dea-fuzzy-ahp (alem et. al., 2013) were also considered as these papers were well-tested and evaluated on a few hypothetical examples which did not follow any specific applications. the authors successfully demonstrated their work based on these hypothetical examples. also, an application-based literature review paper by ho (2008) was considered for this paper. we tried to include the maximum number of papers because this paper deals with the focused approach of deahp. the state-ofthe-art and prominent integrated-deahp approach and its varied applications can be studied by enabling this filtering criterion. the criteria for filtration screened out those journal articles which did not achieve even one of these criteria of filtration. in the fourth step, a careful evaluation of all the lists of references of these shortlisted articles was carried out to make sure that any other related and highly relevant articles were not missed in the search process. finally, an exhaustive review of the content of each journal article was done to make sure that the paper was aligned with the context and was fit in the context of the integrated-deahp approaches and stand-alone deahp approach. this analysis resulted in 72 journal articles for this study. five questions were asked of the 72 collected articles, which include: (i) which of the integrated-deahp approaches were most prevalent and widely appreciated?, (ii) in which fields/areas did the integrated-deahp approaches find a varied number of applications?, (iii) which kind of specific problems were addressed using integrateddeahp approaches?, (iv) what are the publication trends for the papers considering the integrated deahp approaches?, and (v) which jourmals of international standards published how many journal articles and among them which have the highest number of articles on the approach of integrated deahp? 4. stand-alone deahp approaches and applications there are many papers that demonstrate the method and more specifically the applications of the deahp in its stand-alone form. it should be noted that even the stand-alone deahp is a combination of ahp and dea. in the deahp, the dea part removes the limitations of ahp which are rank reversal, element dominance, and row dominance issues. therefore, this method produces better output weights that can be applied to different business scenarios. forty-one journal articles were identified that applied the stand-alone deahp without the integration of any other approaches with their categorization of application areas and specific problems as shown in table 1. these journal articles focus on specific problem statements such as supplier evaluation and selection (sevkli et al, 2007; moon & kang, 2015; aji & hariga, 2013; yadav & sharma, 2015), economic performance assessment (lin et al., 2011) that comes under performance evaluation and efficiency measurement, and the problem of evaluation and selection of a warehouse (korpela et al., 2007) that comes under business unit evaluation. the results of dea and ahp have been combined by ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 235 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 sueyoshi et al. (2009) to tackle the problem of critical business-unit selection and evaluation in a rental car business. table 1 the stand-alone deahp approach and its application areas and specific problems approach authors application areas specific problems dea-ahp (general) william ho (2008) literature review literature review antonio et al. (2012) higher education performance evaluation & efficiency measurement pakkar (2015) banking multiplicative aggregation of financial ratios ahmad (2006) manufacturing business unit evaluation aji & hariga (2013) e-commerce supplier evaluation and section pakkar (2014) iron / steel performance evaluation & efficiency measurement amin & toloo (2007) manufacturing facility layout selection azadeh et al. (2011) banking / mining personal productivity evaluation cai & wu (2001) financial process evaluation omid & zegordi (2015) manufacturing performance evaluation & efficiency measurement moon & kang (2015) defence industry supplier evaluation and section giokas & pentzaropoulos (2008) telecommunication performance evaluation & efficiency measurement hadad & hanani (2011) manufacturing product evaluation and selection banwet & deshmukh (2008) r & d performance evaluation & efficiency measurement kasap & kiriş (2013) energy business unit evaluation kavurmaci & üstün (2016) environment quality evaluation kengpol & tuammee (2015) logistics business unit evaluation korpela et al. (2007) process business unit evaluation lin et al. (2011) government performance evaluation & efficiency measurement lozano & villa (2009) logistics business unit evaluation zarei et al. (2012) manufacturing performance evaluation & efficiency measurement foroughia & esfahani (2012) airlines business unit evaluation ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 236 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 pakkar (2016) government performance evaluation & efficiency measurement ramanathan (2006) multiple industries performance evaluation & efficiency measurement saleeshya & babu (2012) manufacturing business unit evaluation sevkli et al. (2007) manufacturing supplier evaluation and section shirouyehzad et al. (2013) hotel business unit evaluation singh & aggarwal (2014) human resource performance evaluation & efficiency measurement sinuany-stern et al. (2000) manufacturing performance evaluation & efficiency measurement sueyoshi et al. (2009) rental cars business unit evaluation tseng & lee (2009) electronics process evaluation wang et al. (2008) manufacturing risk assessment yadav & sharma (2015) manufacturing supplier evaluation and section yang & kuo (2003) electronics facility layout selection zhang & fu (2012) logistics performance evaluation & efficiency measurement ertay et al. (2006) manufacturing facility layout selection saen et al. (2005) government performance evaluation & efficiency measurement takamura & tone (2003) government location evaluation wang et al. (2008) higher education expert evaluation and selection wang & chin (2009) higher education expert evaluation and selection mirhedayatian & saen (2011) higher education expert evaluation and selection total 41 it was clearly shown by wang et al. (2008), mirhedayatian and saen (2011), and wang and chin (2009) that the approach of integrated-deahp for derivation and aggregation of weights in the ahp was erroneous. sometimes, the method resulted in counter-intuitive priority vectors for inconsistent pairwise-comparison matrices which highly restricted its application. therefore, to address these drawbacks, the above researchers reassessed the integrated deahp approach by using examples such as research fellow recruitment (wang et al., 2008), and university president recruitment (wang & chin, 2009; mirhedayatian & saen, 2011), all of which fall under expert evaluation and selection. the other problems studied included facility layout design selection (yang & kuo, 2003; amin & toloo, 2007; ertay et al., 2006), government agency location selection ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 237 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 and evaluation (takamura & tone, 2003), a supplier efficiency and productivity measurement problem that falls under performance evaluation and efficiency measurement (sinuany-stern et al., 2000; zarei et al., 2012; omid & zegordi, 2015; giokas & pentzaropoulos, 2008), assessment and optimization of personal productivity (azadeh, 2011), financial performance assessment (pakkar, 2014), agility-enablers identification and agility measurement that falls under business unit evaluation (saleeshya & babu, 2012), bridge risk assessment (wang et al., 2008), emerging logistics performance evaluation and efficiency measurement (zhang & fu, 2012), evaluation and selection of business firm unit (ahmad et al., 2006; kasap & kiris, 2013), institutional efficiency measurement that falls under performance evaluation and efficiency measurement (antonio et al., 2012), service units evaluation as in the case of business unit evaluation (shirouyehzad & lofti, 2013), ground water quality evaluation (kavurmaci & üstün, 2016), comparison of appropriate decision support of hr practices on organizational performance which falls under process evaluation (tseng & lee, 2009), manpower performance evaluation and efficiency measurement (singh & aggarwal, 2014), multiplicative aggregation of financial ratios (pakkar, 2015), port logistics performance as in the case of business unit evaluation (lozano & villa, 2009), quantitative risk assessment in multi-modal green logistics (kengpol & tuammee, 2015), relative efficiency measurement (foroughia & esfahani, 2012), road safety performance (pakkar, 2016), selection of best alternative for forklifts purchase (hadad & hanani, 2011), synthetic financial evaluation (cai & wu, 2001) and performance evaluation (saen et al., 2005; banwet & deshmukh, 2008). ramanathan (2006) demonstrated performance evaluation and efficiency measurement on a few hypothetical examples where dea (data envelopment analysis) was propounded. here, the dea was used for the generation of local weights of alternatives from the pair-wise comparison judgment matrices which is used in the ahp. a literature review paper, ho (2008) concentrated on literature from 1997 to 2006 with a wide view and huge number of diversified applications of ahp/dea. 5. integrated deahp approaches and applications 5.1 integrated-deahp and fuzzy-set theory the concept of fuzzy-set theory empowers decision-makers to consider uncertainty or fuzziness by rendering interval judgments. according to table 2, the fuzzy (deahpfuzzy) approach has been applied in 14 articles. the table shows their categorization of application areas and specific problems. ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 238 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 table 2 the integrated deahp-fuzzy set theory approach and its applications approach authors application areas specific problems deahp-fuzzy li et al. (2016) logistics business unit evaluation loron et al. (2015) construction project/program evaluation and selection alem et al. (2013) multiple industries performance evaluation & efficiency measurement huang & chi (2015) tourism location evaluation che et al. (2010) financial process evaluation çalik et al. (2018) government performance evaluation & efficiency measurement gupta et al. (2018) logistics project/program evaluation and selection hadi-vencheh & mohamadghasemi (2011) manufacturing inventory control (multiple criteria abc inventory classification) kumar et al. (2014) telecommunication expert evaluation and selection kuo et al. (2010) manufacturing supplier evaluation and section lee et al. (2010) energy process evaluation lee et al. (2011) energy technological evaluation lee et al. (2013) energy resource allocation parameshwaran et al. (2009) manufacturing business unit evaluation total 14 the integrated deahp-fuzzy approach has been applied to efficiency measurement and performance evaluation (lee et al., 2010; lee et al., 2011; alem et al., 2013; çalik, 2018) along with benchmarking (parameshwaran, 2009). alem et al. (2013) demonstrated his work using a few hypothetical examples. several articles applied the integrated dea/ahp-fuzzy approach for efficiency allocation (lee et al., 2013), customers’ preference analysis and relative efficiency measurement (kumar et al., 2014), efficiency measurement (li et al., 2016), supplier evaluation and selection/performance evaluation (kuo et al., 2014), inventory control (multiple criteria abc inventory classification) (hadi-vencheh & mohamadghasemi, 2011), efficient loaning decision making (che et al., 2010), intelligent building assessment (loron et al., 2015), competitiveness evaluation (huang & chi, 2015) and extended capacitated sustainable transportation problem (gupta et al., 2018). 5.2 integrated deahp and mathematical programming the integrated deahp approach combined with mathematical programming approaches was used in four articles from 2008 to 2018 as shown in table 3 with their categorization of application areas and specific problems. these mathematical programming approaches were lp (linear programming) and gp (goal programming). in these approaches, the relative importance of criteria and alternatives was evaluated and quantified by the deahp. the objective function or constraints of a mathematical programming model were then incorporated with the deahp-derived weights. apart from these, there are other mathematical ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 239 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 programming approaches like molp (multi-objective linear programming), dp (dynamic programming) and, milp (mixed-integer linear programming) which have not been integrated with deahp. table 3 the integrated deahp-mathematical programming approaches and their applications approach authors application areas specific problems deahp-gp jatuphatwarodom et al. (2018) manufacturing supplier evaluation and section deahp-gp kumar & babu (2012) manufacturing supplier evaluation and section deahp-lp falsini et al. (2012) logistics supplier evaluation and section deahp-lp wang et al. (2008) multiple industries multiple problem areas total 4 5.2.1 integrated deahp-gp approach kumar and babu (2012) used the integrated deahp-gp approach. this framework makes use of a multi-stage process which is comprised of approaches, namely dea, ahp and pgp (pre-emptive goal programming) applied to the problem of supplier selection and evaluation in it service and general manufacturing sectors. first, this framework makes use of an ahp-based methodology to decide whether to choose outsourcing or not and then from whom to outsource. after this, a dea-based method was followed, where the relative efficiencies of the priority sources, identified by ahp were determined. finally, the best source from the given number of sources for the given targets of the outsourcer was identified by an ahp-pgp formulation. jatuphatwarodom et al. (2018) exhibited extended goal programming (egp) along with the combined approach of ahp and dea so that managerial decision support could be provided. initially, the dea was used in a descriptive sense to provide information regarding the efficiency for a set of units. after that, the ahp was used to determine the importance of criteria which arises from decision problem(s) which are in turn related to the improvement of unit-efficiency. in the final stage, in order to improve unit-efficiency, egp was used in a prescriptive sense for the selection of a set of specific actions. two specific multi-objective situations in the thai silk industry were used as case studies for the proposed methodology. these involved supplier selection and inventory management with multiple conflicting goals and objectives. in the aforementioned case studies, the dea was used to provide efficiency estimates of current suppliers and processes. after that, the ahp was used to determine the relative importance of criteria for the improvement of efficiency in the supply chain processes. in order to deal with high levels of inconsistency, adaptations were made to an automated inconsistency reduction algorithm. the relationship between the decision maker’s confidence and consistency was then investigated. in the final stage, an egp model was built to recommend actions for improvement to the supply chain processes. ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 240 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 5.2.2 integrated deahp-lp approach falsini et al. (2012) applied the integrated deahp-lp approach in supplier evaluation and selection which proposed an evaluation method in the logistics sector that aims at providing an efficient and effective decision support system to select suppliers. this system is easy to use, avoids limitations in the application field and is able to effectively manage multi-criterion complexity. an integrated-deahp-lp method for generating the most favorable weights from a pairwise-comparison matrix was demonstrated by wang et al. (2008). a lp (linear programming method) for the generation of the most favorable weights (lp-gfw) from pairwise-comparison matrices was given by this article. the variable-weight concept of dea into the priority scheme of the ahp was incorporated by this method to generate the most favorable weights for the underlying criteria and alternatives based on a crisp pairwise comparison matrix. precise weights for perfectly consistent pairwise comparison matrices and approximate weights for inconsistent pairwise comparison matrices were generated by this lp-gfw method. these weights are not significantly different than saaty’s principal right eigenvector weights. in the article, rank preservation methods and the issue of aggregation of local but the most favorable weights are also discussed. to demonstrate its significant advantages and potential applications over some of the previously prevailing priority methods, four hypothetical numerical examples were examined using the lp-gfw method. 5.3 integrated deahp and simulation one article was found that applied the integrated deahp and simulation (deahpsimulation) approach for performance evaluation and efficiency measurement in the railway sector as shown in table 4 with the categorization of application area and specific problem. table 4 the integrated deahp-simulation approach and application approach authors application areas specific problems deahpsimulation azadeh et al. (2008) railways performance evaluation & efficiency measurement total 1 to improve and optimize the railway system by means of efficiency assessment and performance optimization an integrated simulation, multiple decision analysis, and multivariate analysis were presented by azadeh et al. (2008). moreover, the integrated model was based on the ahp and dea which was integrated with computer simulation. the optimum alternatives can be selected by considering multiple qualitative and quantitative outputs and inputs when using the integrated ahp and dea model with simulation. to begin with, the system being studied was model verified and validated by the computer simulation. after that, the weight of any qualitative criteria (outputs or input) was determined by the ahp methodology. in the end, the multi-objective model was solved by using the dea model. this allows the best alternative(s) as well as the mechanism for optimizing the current system to be identified. 5.4 integrated deahp and other approaches seven papers were found that applied the integrated deahp with other approaches as shown in table 5 with their categorization of application areas and specific problems, such as abc (activity based costing), sbm (slacks-based measure), ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 241 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 tco (total cost of ownership), vahp (voting ahp), assurance region (ar), grey relational analysis (gra), and gower plot. table 5 the integrated deahp-other approaches and their applications approach authors application areas specific problems deahp-abc zhang et al. (2012) electronics supplier evaluation and section deahp-sbm yuan et al. (2013) environment resource allocation deahp-tco ramanathan (2007) supply chain management supplier evaluation and section dea-vahp soltanifar & lotfi (2011) healthcare performance evaluation & efficiency measurement deahp-ar lai et al. (2015) airlines performance evaluation & efficiency measurement deahp-gower plot li & ma (2008) business administration location evaluation deahp-gra prasad et al. (2017) iron / steel supplier evaluation and section total 7 the first paper by zhang et al. (2012) applied the integrated deahp-abc for supplier evaluation and selection in the electronics sector. this article suggested a hybrid and integrated model, based on deahp and abc (activity-based costing) so that the overall efficiency of suppliers and costs (which includes indirect costs and direct costs) could be evaluated. comprehensive decision support for the selection and evaluation of suppliers can be derived with the help of this model. the order quantity from the preferred suppliers can be integrated with the decision regarding the selection of suppliers by using this hybrid model. simultaneously, in a single model, a few factors of relevance like the past delivery performance of suppliers, the purchasing budget of the buyer and capacity constraints were considered. the second article by yuan et al. (2013) applied integrated deahp-sbm which focused on disaster vulnerability assessment and resource allocation. this article developed an integrated index which contained sensitivity, exposure, and adaptive capacity so that regional vulnerability to the drought in china could be measured. the same indicator was calculated by the approach which integrated the sbm (slacksbased measure) model with the dea along with ahp which was in turn based on a genetic algorithm. the reasons behind drought vulnerability were provided by the calculated integrated index with the integrated-deahp approach. consequently, the proposal for available mitigation strategies for different drought-vulnerable areas was given. the third article discussed the outcomes of combining ahp and tco (total cost of ownership) along with the application of dea as a way to combine subjective and objective information. this article has been suggested by ramanathan (2007) which integrates the dea-tco-ahp model and applies it to the supplier selection problem. this model is the first work to find its application in the problem of supplier selection. ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 242 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 the fourth article by soltanifar & lotfi (2001) applied dea-voting ahp for benchmarking, efficiency assessment and performance optimization in healthcare (nursing home) and commercial banks. this article presents a method for ranking efficient dmus by means of the voting-analytic hierarchy process (vahp). a few ranking models in dea were reviewed in this article along with their strengths and weaknesses. next, a method for ranking efficient dmus by vahp was presented. the fifth journal article by lai et al. (2015) applied integrated deahp-ar for evaluating the performance efficiency of airports. this article used the ahp and mcdm method for integrating the weightings of output and input variables into ar (assurance region) and dea. in the empirical analysis, 24 major international airports were taken into consideration. this article concluded that the discriminatory power in the suggested model of deahp-ar is greater and better than the basic dea model while the efficiency of airports was being measured. the comparison of operational efficiency between airports can be done effectively by practitioners and policy-makers using this approach. this will help in the generation of more informed decisions. the sixth paper by li & ma (2008) used deahp and the gower plot technique to rank decision alternatives in space/location selection and evaluation. with the integration of ahp, dea, and gower plot techniques, an iterative method of ranking decision alternatives was developed. to narrow the range of preferences of the decision maker, a dea model with modification was first used by this developed method. then, in order to demonstrate the ordinal and cardinal inconsistencies of these preferences, the decision alternatives’ tentative ranks which were computed by embedding the preferences of the decision maker were depicted by means of gower plots. after that, until the inconsistencies come within the tolerance, the decision maker keeps adjusting the preferences in an iterative manner. the seventh paper by prasad et al. (2017) applied deahp-gra for supplier evaluation and selection in the steel manufacturing sector. this article offered a hybrid methodology using ahp, dea and grey relational analysis (gra) for assessing suppliers and for picking the best supplier for an organization. in this work, the suppliers’ performance scores and efficiencies were discovered using the dea methods of cross-efficiency and super-efficiency, respectively. the total list of efficient suppliers showing high-performance levels was based on cross-efficiency scores and super-efficiency scores. the best supplier was discovered using gra methodology. 5.5 integrated deahp and multiple approaches five articles found their applications with the integration of deahp with more than one approach or method (deahp-multiple approach), as illustrated in table 6 with their categorization of application areas and specific problems. two of the papers combined the fuzzy-set theory and the deahp with other different methods or approaches like topsis (rezaei-shouroki et al., 2017) and qfd-lp (kamvysi et al., 2014). there are three other journal articles that combined the deahp with multiple approaches, but have not considered fuzzy-set theory for integration with deahp. moreover, these three other papers have integrated deahp with 2-level som & ar (yu & lee, 2013), delphi-lp (raut, 2014) and qfd-deanp (kamvysi et al., 2010). hence, two papers by kamvysi et al. (2014) and raut (2014) used the integrated deahp approach and lp approach along with other approaches. again, there are ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 243 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 two papers about qfd-deanp that used the integrated deahp approach and qfd approach along with other approaches (kamvysi et al., 2010; kamvysi et al. 2014). table 6 the integrated deahp-multiple approaches and their applications approach authors application areas specific problems deahp, fuzzy, qfd, lp kamvysi et al. (2014) higher education course design evaluation and selection deahp, delphi, lp raut (2014) pharmaceuticals supplier evaluation and section deahp, 2-level som, ar yu & lee (2013) nano technology technological evaluation deahp, topsis, fuzzy rezaei-shouroki et al. (2017) energy location evaluation deahp, qfd, deanp kamvysi et al. (2010) banking service evaluation and selection total 5 yu & lee (2013) applied the integrated deahp with 2 level som & ar for the evaluation and selection of nano-technology. a method of ahp-rating was used for assessing the importance of emerging technology alternatives rather than pairwise comparison since there are a large number of alternatives that are independent to each other. the importance value was termed as the ahp score within this paper. finally, the cumulative score of each emerging technology alternative was calculated using the suggested method by means of the addition of the scores of ahp and of dea of the alternatives. the emerging technology alternatives’ cumulative scores indicate the most promising emerging technology. rezaei-shouroki et al. (2017) used integrated deahp with topsis for evaluating and selecting locations for wind farm/wind energy. the method of dea and a list of technical, economic and geological factors were utilized in this paper for prioritization and ranking of thirteen cities in the fars’ province with respect to their appropriateness for the building of a wind farm. the validity of the obtained results was evaluated using ftopsis and ahp. kamvysi et al. (2014) applied the integrated deahp-fuzzy-qfd-lp in higher education to evaluate and select course design. the usage of a substitute alternate framework to prioritize the requirements of students was suggested and tested using qfd (quality density function). more explicitly, to account for the inherent subjectivity of human judgments, a fuzzy-ahp and lp (lp-gw-ahp) method which is based on dea was integrated with qfd (qfd-lp-gw-fuzzy-ahp). capturing and prioritizing the requirements of students with respect to the learning outcomes of courses within the academic course design’s process was evaluated on the effectiveness of the proposed framework. the robustness of the prioritization solution was assessed with a sensitivity analysis. the implications were discussed for the specifications of the design of the course. kamvysi et al. (2010) applied the integrated deahp as qfd-deahp-deanp in the greek banking sector to evaluate the banking services so that the best selection could be presented to help customers choose their bank according to their preference of banking services mix. this article used deahp so that the relationship-matrix of ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 244 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 hoq (house of quality) was completed. also, a methodological framework to complete the roof matrix of hoq was developed while using the deahp approach, and it was based on the dea and anp techniques which gave rise to deanp. the implementation of qfd along with the abovementioned quantitative approaches was studied utilizing data from the greek banking sector and a comparative analysis was conducted. the paper by raut (2014) was applied to the pharmaceutical sector where the deahp was combined with the delphi-lpp multi-criteria decision-making approach to select a supplier and to allocate a system for order quantity. the methodology which was based on integrated-delphi-ahp-dea-lpp (dadlp) was used. this methodology was organized into a four phases. the initial phase was the delphimethod. in the delphi method, a new and consistent scale for measurement was developed after identifying, synthesizing and prioritizing important factors and subfactors of performance. the ahp and dea-lp based mcdm was developed for both quantitative and qualitative factors in the selection of suppliers. the product characteristics were matched with supplier characteristics (using supplier ratings derived from pair wise comparisons) by the ahp method to determine the strategy of the supply-chain qualitatively. the efficient suppliers were evaluated and determined from the selected suppliers, mathematically using dea. the variations of pair-wise comparisons in ahp will affect the final order quantity because ahp ratings are taken as input by lp and dea. 6. observations and discussions this literature review looked at 72 journal articles published between 2000 and 2018 that studied the stand-alone deahp approach and integrated deahp with other approaches. 31 articles were identified that applied the integrated deahp approach with other approaches, and the rest of the 41 articles concentrated on the stand-alone dea/ahp approach without being associated with a third approach. methods such as simulation, mathematical programming, fuzzy-set theory, and others were found to have been combined with the integrated deahp approach. some of the articles used more than one approach (multiple approaches). the stand-alone deahp approach and integrated deahp along with other approaches are summarized in tables 1 – 6 with their categorization of application areas and specific problems. based on the categorization of these 72 journal articles, five questions can be answered: (1) which integrated-deahp approach was most prevalent and received the utmost attention?, (2) which areas of application were most often studied and found their application with integrated deahp approaches?, (3) which of the specific problems were most frequently applied to integrated deahp approaches?, (4) what are the publications trends for the papers considering the integrated deahp approaches?, and (5) which jourmals of international standards published how many journal articles and among them which have the highest number of articles on the approach of integrated deahp? over and above answering these queries, this journal article also seeks to reiterate the work in ho (2008) and, in some manner to further extend its work from integrated ahp to integrated deahp. 6.1 the most prevalent integrated-deahp approach the foremost query seeks to identify the most prevalent approach of integrateddeahp. as tabulated in tables 2–5, 26 articles out of a total of 72 used the integrated deahp approach with only one method. however as shown in table 6, in 5 of the articles two or more methods were integrated with the deahp approach ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 245 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 (also known as integrated deahp-multiple approaches). as observed in tables 2–5, the fuzzy-set theory (14 out of 26 articles) was the most prevalent method integrated with the deahp, followed by mathematical programming (4 articles), simulation (1 article), and some others (7 articles), including vahp (voting ahp), tco (total cost of ownership), gower plot, abc (activity based costing), sbm (slacks-based measure), assurance region (ar), and grey relational analysis (gra). it can be seen from table 6 that a varied number of integrated deahp-multiple approaches exist. a major part of these selected studies integrated the deahp with 2 methods (4 out of 5 articles), followed by 3 methods (1 article). as no single approach overpowers another in the combination, the most prevalent method applied in the integrated deahp-multiple approaches was determined by developing table 7. the top three methods were fuzzy-set theory (2 out of 5 articles), lp which is a subset of mathematical programming (2 articles) and qfd (2 articles) which is followed by topsis (1 article), ar (1 article), delphi (1 article), deanp (1 article) and 2-level som (1 article) as shown in table 7. table 7 summary of methods (other than deahp) used in the integrated deahp-multiple approaches methods number of articles authors fuzzy set theory 2 kamvysi et al. (2014), rezaei-shouroki et al. (2017) lp 2 kamvysi et al. (2014), raut (2014) qfd 2 kamvysi et al. (2010), kamvysi et al. (2014) topsis 1 rezaei-shouroki et al. (2017) ar 1 yu & lee (2013) delphi 1 raut (2014) deanp 1 kamvysi et al. (2010) 2-level som 1 yu & lee (2013) based on the written and tabulated observations above, it can be seen that the integrated deahp-fuzzy approach gained major attraction and attention between 2009 and 2018. furthermore, linear mathematical programming (lp), quality density function (qfd) and fuzzy-set theory were found to be commonly integrated with the deahp in the integrated deahp-multiple approaches. from table 2, fuzzy set theory was integrated with deahp in 14 out of 31 journal articles. the key rationale is that the crisp pairwise-comparison in conventional deahp is unable to bear ambiguity or vagueness. thus, in an uncertain pairwise-comparison environment, fuzzy-numbers and linguistic terms have been extensively used (lee, 2009; liang, 2015). unambiguously, the fuzzy-set theory has the additional benefit of being similar to human-reasoning in its use of approximate information and uncertainty to address mcdm problems when information is incomplete and imprecise (chan & kumar, 2007). 6.2 the most frequently studied area of application the next query aimed to find out the most frequently studied area of application. as illustrated in table 8, there are 28 application areas and 1 literature review that has ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 246 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 been applied to different fields. among the application areas mentioned in table 8, manufacturing has attracted the maximum attention, followed by logistics, government, energy, higher education, electronics, multiple industries, airlines, banking, environment, iron/steel, financial, telecommunications, business administration, construction, defence industry, e-commerce, healthcare, hotel industry, human resource, nano-technology, pharmaceuticals, process, r & d, railways, rental cars, supply chain management and tourism sectors. one paper was a literature review paper. among 16 articles dedicated to the manufacturing sector, the approach of the fuzzyset theory was extensively used in combination with deahp, including 3 integrated deahp-fuzzy approaches and 2 integrated deahp-goal programming approach (deahp-mathematical programming). table 8 summary of application areas application areas approaches number of articles sub-total authors airlines dea-ahp (general) 1 2 foroughia & esfahani (2012) deahp-ar 1 lai et al. (2015) banking dea-ahp (general) 2 3 pakkar (2015) omid & zegordi (2011) deahp-qfddeanp 1 kamvysi et al. (2010) business administration deahp-gower plot 1 1 li & ma (2008) construction deahp-fuzzy 1 1 loron et al. (2015) defence industry dea-ahp (general) 1 1 moon & kang (2015) e-commerce dea-ahp (general) 1 1 aji & hariga (2013) electronics dea-ahp (general) 2 3 tseng & lee (2009) yang & kuo (2003) deahp-abc 1 zhang et al. (2011) energy dea-ahp (general) 1 5 kasap & kiriş (2010) deahp-fuzzy 3 lee et al. (2011) lee et al. (2013) lee et al. (2017) deahptopsis-fuzzy 1 rezaei-shouroki et al. (2013) ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 247 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 environment dea-ahp (general) 1 2 kavurmaci & üstün (2016) deahp-sbm 1 yuan et al. (2013) financial dea-ahp (general) 1 2 cai & wua (2001) deahp-fuzzy 1 che et al. (2010) government dea-ahp (general) 4 5 lin et al. (2011) pakkar (2016) saen et al. (2018) takamura & tone (2005) deahp-fuzzy 1 çalik et al. (2003) healthcare dea-vahp 1 1 soltanifar & lotfi (2011) higher education dea-ahp (general) 4 5 antonio et al. (2014) wang et al. (2008) wang & chin (2009) mirhedayatian & saen (2011) deahp-fuzzyqfd-lp 1 kamvysi et al. (2012) hotel dea-ahp (general) 1 1 shirouyehzad et al. (2013) human resource dea-ahp (general) 1 1 singh & aggarwal (2014) iron / steel dea-ahp (general) 1 2 pakkar (2014) deahp-gra 1 prasad et al. (2017) literature review dea-ahp (general) 1 1 ho (2008) logistics dea-ahp (general) 3 6 kengpol & tuammee (2015) lozano & villa (2009) zhang & fu (2012) deahp-fuzzy 2 li et al. (2016) gupta et al. (2018) deahp-lp 1 falsini et al. (2012) manufacturing dea-ahp (general) 11 16 ahmad et al. (2006) amin & toloo (2007) omid & zegordi (2015) ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 248 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 hadad & hanani (2011) zarei et al. (2012) saleeshya & babu (2012) sevkli et al. (2007) sinuany-stern et al. (2000) wang et al. (2008) yadav & sharma (2015) ertay et al. (2011) deahp-fuzzy 3 hadi-vencheh & mohamadghasemi (2010) kuo et al. (2009) parameshwaran et al. (2012) deahp-gp 2 jatuphatwarodom et al. (2018) kumar & babu (2012) multiple industries dea-ahp (general) 1 3 ramanathan (2006) deahp-fuzzy 1 alem et al. (2013) deahp-lp 1 wang et al. (2008) nano technology deahp-ar-2 level som 1 1 yu & lee (2013) pharmaceuticals deahpdelphi-lpp 1 1 raut (2014) process dea-ahp (general) 1 1 korpela et al. (2007) r & d dea-ahp (general) 1 1 banwet & deshmukh (2008) railways deahpsimulation 1 1 azadeh et al. (2008) rental cars dea-ahp (general) 1 1 sueyoshi et al. (2009) supply chain management deahp-tco 1 1 ramanathan (2007) telecommunication dea-ahp (general) 1 2 giokas & pentzaropoulos (2008) deahp-fuzzy 1 kumar et al. (2014) tourism deahp-fuzzy 1 1 huang & chi (2015) total 72 72 ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 249 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 a graphical representation of the application areas and corresponding number of articles is summarized below. figure 3 summary of application areas and their corresponding number of articles 6.3 the most frequently studied specific problem the third query aimed to discover the most frequently studied specific problem. table 9 shows that 1 literature review and 19 problems were studied. the problem of performance evaluation & efficiency measurement gained the maximum attention, followed by evaluation and supplier selection, evaluation of a business unit, evaluation of a process, evaluation and selection of an expert, evaluation of a location, selection of a facility layout, allocation of resources, evaluation and selection of a project/program, evaluation and selection of a product, evaluation and selection of a service, technological evaluation, multiplicative aggregation of financial ratios, course design evaluation and selection, inventory control (multiple criteria abc inventory classification), multiple problem areas, personal productivity evaluation, quality evaluation and risk assessment. one paper was developed as a literature review. 18 of 72 articles addressed performance evaluation and efficiency measurement out of which 5 articles applied integrated deahp which includes 2 papers with integrated deahp-fuzzy approach, 1 paper with integrated deahp-ar, 1 paper with integrated dea-vahp and 1 paper with integrated deahp-simulation. 2 3 1 1 1 1 3 5 2 2 5 1 5 1 1 2 1 6 16 3 1 1 1 1 1 1 1 2 1 0 2 4 6 8 10 12 14 16 18 a ir li n e s b a n k in g b u si n e ss a d m in is tr a ti o n c o n st ru ct io n d e fe n ce i n d u st ry e -c o m m e rc e e le ct ro n ic s e n e rg y e n v ir o n m e n t f in a n ce g o v e rn m e n t h e a lt h ca re h ig h e r e d u ca ti o n h o te l h u m a n r e so u rc e ir o n / s te e l li te ra tu re r e v ie w lo g is ti cs m a n u fa ct u ri n g m u lt ip le i n d u st ri e s n a n o t e ch n o lo g y p h a rm a ce u ti ca ls p ro ce ss r & d r a il w a y s r e n ta l ca rs s u p p ly c h a in m a n a g e m e n t t e le co m m u n ic a ti o n t o u ri sm n u m b e r o f a rt ic le s application areas ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 250 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 table 9 summary of specific problems specific problems approaches number of articles sub-total authors business unit evaluation dea-ahp (general) 9 11 foroughia & esfahani (2012) ahmad et al. (2006) korpela et al. (2007) kasap & kiriş (2013) shirouyehzad et al. (2013) kengpol & tuammee (2015) lozano & villa (2009) saleeshya & babu (2012) sueyoshi et al. (2009) deahpfuzzy 2 li et al. (2016) parameshwaran et al. (2009) course design evaluation and selection deahpfuzzy-qfdlp 1 1 kamvysi et al. (2014) expert evaluation and selection dea-ahp (general) 3 4 wang et al. (2008) wang & chin (2009) mirhedayatian & saen (2011) deahpfuzzy 1 kumar et al. (2014) facility layout selection dea-ahp (general) 3 3 yang & kuo (2003) amin & toloo (2007) ertay et al. (2006) inventory control (multiple criteria abc inventory classification) deahpfuzzy 1 1 hadi-vencheh & mohamadghasemi (2011) literature review dea-ahp (general) 1 1 ho (2008) location evaluation dea-ahp (general) 1 4 takamura & tone (2003) deahpfuzzy 1 huang & chi (2015) deahptopsisfuzzy 1 rezaei-shouroki et al. (2017) ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 251 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 deahpgower plot 1 li & ma (2008) multiple problem areas deahp-lp 1 1 wang et al. (2008) multiplicative aggregation of financial ratios dea-ahp (general) 1 1 pakkar (2015) performance evaluation & efficiency measurement dea-ahp (general) 13 18 lin et al. (2011) pakkar (2016) saen et al. (2005) antonio et al. (2012) singh & aggarwal (2014) pakkar (2014) zhang &fu (2012) omid & zegordi (2015) zarei et al. (2012) sinuany-stern et al. (2000) ramanathan (2006) banwet & deshmukh (2008) giokas & pentzaropoulos (2008) deahpfuzzy 2 çalik et al. (2018) alem et al. (2013) deahp-ar 1 lai et al. (2015) dea-vahp 1 soltanifar & lotfi (2011) deahpsimulation 1 azadeh et al. (2008) personal productivity evaluation dea-ahp (general) 1 1 azadeh et al. (2011) process evaluation dea-ahp (general) 2 4 tseng & lee (2009) cai & wua (2001) deahpfuzzy 2 lee et al. (2010) che et al. (2010) product evaluation and selection dea-ahp (general) 1 1 hadad & hanani (2011) ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 252 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 service evaluation and selection deahpqfddeanp 1 1 kamvysi et al. (2010) project/program evaluation and selection deahpfuzzy 2 2 loron et al. (2015) gupta et al. (2018) quality evaluation dea-ahp (general) 1 1 kavurmaci & üstün (2016) resource allocation deahpfuzzy 1 2 lee et al. (2013) deahpsbm 1 yuan et al. (2013) risk assessment dea-ahp (general) 1 1 wang et al. (2008) supplier evaluation and selection dea-ahp (general) 4 12 moon & kang (2015) aji & hariga (2013) sevkli et al. (2007) yadav & sharma (2015) deahpfuzzy 1 kuo et al. (2010) deahp-lp 1 falsini et al. (2012) deahp-gp 2 jatuphatwarodom et al. (2018) kumar & babu (2012) deahpdelphi-lpp 1 raut (2014) deahpabc 1 zhang et al. (2011) deahpgra 1 prasad et al. (2017) deahptco 1 ramanathan (2007) technological evaluation deahpfuzzy 1 2 lee et al. (2011) deahpar-2 level som 1 yu & lee (2013) total 72 72 ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 253 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 a graphical representation of specific problems and corresponding number of articles is summarized below. figure 4 summary of specific problems and their corresponding number of articles 6.4 publication trends table 10 presents the distribution of the 72 journal papers over the last two decades (19 years to be precise) for both stand-alone deahp and integrated deahp with other approaches. 11 1 4 3 1 1 4 1 1 18 1 4 1 1 2 1 2 1 12 2 0 2 4 6 8 10 12 14 16 18 20 b u si n e ss u n it e va lu a ti o n c o u rs e d e si g n e v a lu a ti o n a n d s e le ct io n e xp e rt e v a lu a ti o n a n d s e le ct io n f a ci li ty l a y o u t s e le ct io n in v e n to ry c o n tr o l li te ra tu re r e v ie w lo ca ti o n e va lu a ti o n m u lt ip le p ro b le m a re a s m u lt ip li ca ti v e a g g re g a ti o n o f fi n a n ci a l ra ti o s p e rf o rm a n ce e v a lu a ti o n & e ff ic ie n cy m e a su re m e n t p e rs o n a l p ro d u ct iv it y e v a lu a ti o n p ro ce ss e v a lu a ti o n p ro d u ct e v a lu a ti o n a n d s e le ct io n s e rv ic e e v a lu a ti o n a n d s e le ct io n p ro je ct /p ro g ra m e v a lu a ti o n a n d s e le ct io n q u a li ty e va lu a ti o n r e so u rc e a ll o ca ti o n r is k a ss e ss m e n t s u p p li e r e v a lu a ti o n a n d s e le ct io n t e ch n o lo g ic a l e v a lu a ti o n n u m b e r o f a rt ic le s specific problems ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 254 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 table 10 distribution of the number of journal articles since 2000 number of articles standalone dea/ahp approach integrated deahp approach years integration with no method integration with one method integration with multiple methods total 2000 1 0 0 1 2001 1 0 0 1 2002 0 0 0 0 2003 2 0 0 2 2004 0 0 0 0 2005 1 0 0 1 2006 3 0 0 3 2007 3 1 0 4 2008 5 3 0 8 2009 4 1 0 5 2010 0 3 1 4 2011 4 4 0 8 2012 5 2 0 7 2013 3 3 1 7 2014 2 1 2 5 2015 5 3 0 8 2016 2 1 0 3 2017 0 1 1 2 2018 1 3 0 2 total 26 5 72 41 31 there are 41 articles that considered stand-alone deahp with the integration of no other approach to be sufficient to address the paper’s problem definition. however, there are 31 journal articles that specifically applied integrated deahp approaches. out of these 31 articles which have applied integrated deahp, there are 26 articles which have integrated deahp with only one method. 5 papers found the need for more than one approach to be integrated with deahp to satisfy the problem definition. the articles for the stand-alone deahp were published between 2000 to 2018. there are 31 journal articles for the integrated deahp approach from 2007 to 2018. based on table 10, it is clearly visible that stand-alone deahp has been equally used in terms of numbers of articles published in both decades. it cannot be overlooked that there were no papers published before 2006 for the integrated deahp approach, but this fact is substantiated with the increase in the number of publications on the subject after 2007. also, in 2010, papers with the integration of multiple approaches to deahp began to be published. papers with integrated deahp approaches have attracted more attention since 2007 and it is estimated that it will continue to do the same after 2018 with the changing needs of the competitive business environment. ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 255 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 a graphical representation of publication trends and corresponding number of articles is summarized below. figure 5 summary of publication trends and their corresponding number of articles 1 1 2 1 3 3 5 4 4 5 3 2 5 2 1 3 1 3 4 2 3 1 3 1 1 3 1 1 2 1 0 1 2 3 4 5 6 2 0 0 0 2 0 0 1 2 0 0 2 2 0 0 3 2 0 0 4 2 0 0 5 2 0 0 6 2 0 0 7 2 0 0 8 2 0 0 9 2 0 1 0 2 0 1 1 2 0 1 2 2 0 1 3 2 0 1 4 2 0 1 5 2 0 1 6 2 0 1 7 2 0 1 8 n u m b e r o f a rt ic le s year of publication standalone dea/ahp approach (integration with no method) integration with one method ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 256 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 6.5 the most frequently published journal table 11 lists all of the international journals in which integrated deahp and standalone deahp approaches were published. table 11 summary of journals journals number of articles authors african journal of business management 1 antonio et al. (2012) american journal of operations research 1 pakkar (2014) annals of operations research 1 jatuphatwarodom et al. (2018) applied mathematics and computation 2 saen et al. (2005), azadeh et al. (2008) computers & industrial engineering 3 amin & toloo (2007), wang et al. (2008), soltanifar & lotfi (2011) computers & operations research 3 ramanathan (2006), wang et al. (2008), lozano & villa (2009) decision science letters 1 omid & zegordi (2015) decision support systems 1 wang et al. (2008) energy sources, part b: economics, planning, and policy 1 kasap & kiriş (2013) environmental monitoring and assessment 1 kavurmaci & üstün (2016) european journal of industrial engineering 1 parameshwaran et al. (2009) european journal of operational research 6 yang & kuo (2003), ho (2008), sueyoshi et al. (2009), wang & chin (2009), lin et al. (2011), kamvysi et al. (2014) expert systems with applications 5 tseng & lee (2009), che et al. (2010), hadi-vencheh & mohamadghasemi (2011), azadeh et al. (2011), yu & lee (2013) information sciences 1 ertay et al. (2006) international journal of applied decision sciences 1 aji & hariga (2013) international journal of business and systems research 1 saleeshya & babu (2012) international journal of hydrogen energy 3 lee et al. (2010), lee et al. (2011), rezaei-shouroki et al. (2017) international journal of information technology & decision making 2 ahmad et al. (2006), li & ma (2008) international journal of logistics systems and management 2 hadad & hanani (2011), raut (2014) international journal of operational research 1 alem et al. (2013) ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 257 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 international journal of production economics 1 korpela et al. (2007) international journal of production research 4 sevkli et al. (2007), zhang et al. (2011), falsini et al. (2012), kengpol & tuammee (2015) international journal of productivity and performance management 1 banwet & deshmukh (2008) international journal of productivity and quality management 1 shirouyehzad et al. (2013) international journal of services and operations management 1 kumar & babu (2012) international journal of simulation: systems, science & technology 1 huang & chi (2015) international journal on advances in information sciences and service sciences 1 zhang & fu (2012) international transactions in operational research 2 sinuany-stern et al. (2000), cai & wua (2001) journal of applied operational research 1 pakkar (2015) journal of enterprise information management 1 yadav & sharma (2015) journal of information and communication convergence engineering 1 moon & kang (2015) journal of the operational research society 1 mirhedayatian & saen (2011) journal of the operations research society of china 1 singh & aggarwal (2014) journal of traffic and transportation engineering 1 li et al. (2016) management science letters 2 foroughia & esfahani (2012), zarei et al. (2012) mitigation and adaptation strategies for global change 1 yuan et al. (2013) production planning & control: the management of operations 1 kuo et al. (2010) renewable and sustainable energy reviews 1 lee et al. (2013) resources policy 1 gupta et al. (2018) socio-economic planning sciences 1 takamura & tone (2003) supply chain management 1 ramanathan (2007) systems 1 pakkar (2016) technological and economic development of economy 1 çalik et al. (2018) telecommunications policy 1 giokas et al. (2008) telematics and informatics 1 kumar et al. (2014) the journal tehnički vjesnik – technical gazette (tv-tg) 1 loron et al. (2015) thetqm journal 1 kamvysi et al. (2010) transport policy 1 lai et al. (2015) uncertain supply chain management 1 prasad et al. (2017) total 72 ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 258 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 of the existing 49 journals, 26 have published articles that applied integrated deahp and 31 have published articles that applied stand-alone deahp. the top three highlighted journals for publication of works related to deahp and integrated deahp are european journal of operational research, followed by expert systems with applications, and international journal of production research. there is a clear indication that stand-alone deahp and integrated deahp are spreading their wings in journals of different fields while exploring different application areas. 7. future work there is still space for integrated-deahp approaches, mainly when the integrations are with multiple methods. there are many areas where integrated deahp can be applied to the marketing sector which have not yet been touched by the integrated deahp approach. hence, some recommendations can be pushed for newer applied working areas of the integrated deahp approaches. for example, risk management in the supply-chain is considered one of the trending research topics in the field of supply-chain management/operations and it has gained a substantial amount of attention (ho et al., 2015). yet, the amalgamation of both downstream and upstream strategies of risk-mitigation in the selection of suppliers has not been studied to a great extent (ho et al., 2015). hence, we propose to apply the integration of deahp with other approaches in unexplored application areas. for instance, integrated deahp approaches could be explored in retail space optimization and allocation for better and more efficiently performing stores. the ultimate aim is to maximize the utilization of the total space available in the planning period and minimize the total cost. furthermore, we propose the application of the integration of deahp to tco (total cost of ownership), green initiatives, and csr (corporate social responsibility) for developing a sustainability performance management model which will be holistic in nature for the assessment of suppliers of sustainable smes (small and mediumsized enterprise) in emerging economies. additionally, a procurement contract could be designed by a game model so that the mutual benefits of both the suppliers and the manufacturers could be maximized. 8. conclusion this paper reviewed the literature on the approaches with integrated-deahp and stand-alone deahp along with their applications which were published between 2000 and 2018. this paper uses the criteria for search and filtration which was previously used in ho (2008). researchers were able to gain insights about the noteworthy and state-of-the-art integrated-deahp approaches and their respective applications from this criterion. moreover, they can be equipped with valuable acumen for the application, development and publishing of novel integrated-deahp approaches in high-ranking journals. the summarized findings of this article, in a nutshell, can be outlined as follows. first, the integrated-deahp-fuzzy approach became the most prevalent between 2000 and 2018. second, the most widespread area of application is the manufacturing sector. third, the problem area of performance evaluation and efficiency measurement is the most commonly studied problem. this is predictable, but also logical that integration of deahp with more than one method will gain more academic attraction and attention. in the the end, it is recommended that a few nascent areas of applications for newer integrations of ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 259 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 deahp be explored. these areas and integrations could assist researchers in filling research gaps in the available literature. ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 260 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 references ahmad, n., berg, d., & simons, g. r. (2006). the integration of analytical hierarchy process and data envelopment analysis in a multi-criteria decisionmaking problem. international journal of information technology & decision making, 5(2), 263-276. doi: 10.1142/s0219622006001940 aji, y., & hariga, m. (2013). an ahp-dea-based vendor selection approach for an online trading platform. international journal of applied decision sciences, 6(1), 2013. doi: https://doi.org/10.1504/ijads.2013.052633 alem, s. m., jolai, f., & shirkouhi, s. n. (2013). an integrated fuzzy dea-fuzzy ahp approach: a new model for ranking decision-making units. international journal of operational research, 17(1), 2013. doi: https://doi.org/10.1504/ijor.2013.053187 amin, g. r. & toloo, m. (2007). finding the most efficient dmus in dea: an improved integrated model. computers and industrial engineering, 52(2), 71-77. doi: https://doi.org/10.1016/j.cie.2008.07.014 antonio, a. c., domingo, g. m., humberto, b. o., alvaro, l. l., alvaro, l. r. & rebeca del rocío, p. v. (2012). measuring the institutional efficiency using data envelopment analysis and analytic hierarchy process: the case of a mexican university. african journal of business management, 6(50), 11923–11930. doi: https://doi.org/10.5897/ajbm10.770 azadeh, a., ghaderi, s. f., & izadbakhsh, h. (2008). integration of dea and ahp with computer simulation for railway system improvement and optimization. applied mathematics and computation, 195(2), 775–785. doi: https://doi.org/10.1016/j.amc.2007.05.023 azadeh, a., ghaderi, s. f., mirjalili, m., & moghaddam, m. (2011). integration of analytic hierarchy process and data envelopment analysis for assessment and optimization of personnel productivity in a large industrial bank. expert systems with applications, 38(5), 5212-5225. doi: https://doi.org/10.1016/j.eswa.2010.10.038 bana e costa, c. a., & vansnick, j. c. (2008). a critical analysis of the eigenvalue method used to derive priorities in ahp. european journal of operational re search, 187(3), 1422–1428. banwet, j. d. k., deshmukh, s. g. (2008). evaluating performance of national r&d organizations using integrated dea-ahp technique. international journal of productivity and performance management, 57, 405–419. doi: https://doi.org/10.1108/17410400810881836 cai, y., & wu, w. (2001). synthetic financial evaluation by a method of combining dea with ahp. international transactions in operational research, 8, 603-609. doi: http://dx.doi.org/10.1111/1475-3995.00336 çalik, a., yapici pehlivan, n., & kahraman, c. (2018). an integrated fuzzy ahp/dea approach for performance evaluation of territorial units in turkey. technological and economic development of economy, 24(4), 2018. doi: ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 261 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 https://doi.org/10.3846/20294913.2016.1230563 chan, f. t. s. & kumar, n. (2007). global supplier development considering risk factors using fuzzy extended ahp-based approach. omega, 35(4), 417–431. doi: https://doi.org/10.1016/j.omega.2005.08.004 charnes, a., cooper, w. w., & rhodes, e. (1978). measuring the efficiency of decision-making units. european journal of operational research, 2(6), 429-444. doi: https://doi.org/10.1016/0377-2217(78)90138-8 che, z. h., wang, h. s., & chuang, c. l. (2010). a fuzzy ahp and dea approach for making bank loan decisions for small and medium enterprises in taiwan. expert systems with applications, 37(10),7189-7199. doi: https://doi.org/10.1016/j.eswa.2010.04.010 emrouznejad, a., parker, b.r., tavates, g. (2008). evaluation of research in efficiency and productivity: a survey and analysis of the first 30 years of scholarly literature in dea. socio-economic planning sciences, 42, 151–157. doi: https://doi.org/10.1016/j.seps.2007.07.002 emrouznejad a, thanassoulis e. (1996a). an extensive bibliography of data envelopment analysis (dea), vol. i: working paper 244, 1–55. emrouznejad a, thanassoulis e. (1996b). an extensive bibliography of data envelopment analysis (dea), vol. ii: working paper 245, 1–21. emrouznejad a, thanassoulis e. (1997). an extensive bibliography of data envelopment analysis (dea), vol. iii: supplement 1, working paper 258, 1–24. emrouznejad, a., & yang, g. l., (2017). a survey and analysis of the first 40 years of scholarly literature in dea: 1978–2016. socio-economic planning sciences, 61(2018), 4-8. doi: http://dx.doi.org/10.1016/j.seps.2017.01.008 ertay, t., ruan, d., & tuzkaya, u. r. (2006). integrating data envelopment analysis and analytic hierarchy for the facility layout design in manufacturing systems. information sciences, 176(3), 237-262. doi: https://doi.org/10.1016/j.ins.2004.12.001 falsini, d., fondi, f., & schiraldi, m. m. (2012). a logistics provider evaluation and selection methodology based on ahp, dea and linear programming integration. international journal of production research, 50(17), 4822-4829. doi: https://doi.org/10.1080/00207543.2012.657969 foroughi, a., & esfahani, m. j. (2012). a robust ahp-dea method for measuring the relative efficiency: an application of airport industry. management science letters, 2(2012), 93-100. doi: https://doi.org/10.5267/j.msl.2011.09.018 gattoufi, s., oral, m.,& reisman, a. (2004a). a taxonomy for data envelopment analysis. socio-economic planning sciences, 38(2–3), 141–158. doi: https://doi.org/10.1016/s0038-0121(03)00022-3 gattoufi, s., oral, m., & reisman, a. (2004b). data envelopment analysis literature: a bibliography update (1996–2001). socio-economic planning sciences, 38(2–3), ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 262 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 122–159. doi: https://doi.org/10.1016/s0038-0121(03)00023-5 giokas, d. i., & pentzaropoulos, g. c. (2008). efficiency ranking of the oecd member states in the area of telecommunications: a composite ahp/dea study. telecommunications policy, 32(9-10), 672-685. doi: https://doi.org/10.1016/j.telpol.2008.07.007 gupta, p., mehlawat, m. k., aggarwal, u., & charles, v. (2018). an integrated ahp-dea multi-objective optimization model for sustainable transportation in mining industry. resources policy. doi: https://doi.org/10.1016/j.resourpol.2018.04.007 hadad, y., & hanani, m. z. (2011). combining the ahp and dea methodologies for selecting the best alternative. international journal of logistics systems and management, 9(3), 251-267. doi: https://doi.org/10.1504/ijlsm.2011.041687 hadi-vencheh, a., & mohamadghasemi, a. (2011). a fuzzy ahp-dea approach for multiple criteria abc inventory classification. expert systems with applications, 38(4), 3346-3352. doi: https://doi.org/10.1016/j.eswa.2010.08.119 ho, w. (2008). integrated analytic hierarchy process and its applications a literature review. european journal of operational research, 186(1), 211-228. doi: https://doi.org/10.1016/j.ejor.2007.01.004 ho, w., & ma, x. (2018). the state-of-the-art integrations and applications of the analytic hierarchy process. european journal of operational research, 267(2), 399414. doi: https://doi.org/10.1016/j.ejor.2017.09.007 ho, w., zheng, t., yildiz, h., & talluri, s. (2015) supply chain risk management: a literature review. international journal of production research, 53(16), 5031-5069. doi: https://doi.org/10.1080/00207543.2015.1030467 huang, d., & chi, x. (2015). application of delphi-ahp-dea-fce model in competitiveness evaluation of sports tourism destination. international journal of simulation: systems, science and technology, 16(1b), 20.1-20.5. doi: https://doi.org/10.5013/ijssst.a.16.1b.20 jatuphatwarodom, n., jones, d. f., & ouelhadj, d. (2018). a mixed-model multiobjective analysis of strategic supply chain decision support in the thai silk industry. annals of operations research, 267(1-2), 221-247. doi: https://doi.org/10.1007/s10479-018-2774-6 kamvysi, k., gotzamani, k., georgiou, a. c., & andronikidis, a. (2010) integrating deahp and deanp into the quality function deployment. the tqm journal. 22 (3), 293-316. doi: https://doi.org/10.1108/17542731011035532 kamvysi, k., gotzamani, k., andronikidis, a., & georgiou, a. c. (2014). capturing and prioritizing students’ requirements for course design by embedding fuzzy-ahp and linear programming in qfd. european journal of operational research, 237(1), 177-195. doi: https://doi.org/10.1016/j.ejor.2014.02.041 kasap, y., & kiriş, ş. (2013). an ahp-dea approach for evaluating electricity ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 263 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 generation firms of oecd countries. energy sources, part b: economics, planning and policy, 8(2), 200-208. doi: https://doi.org/10.1080/15567249.2011.647241 kavurmaci, m., & üstün, a. k. (2016). assessment of groundwater quality using dea and ahp: a case study in the sereflikochisar region in turkey. environmental monitoring and assessment, 188, 258. doi: https://doi.org/10.1007/s10661-016-52596 kengpol, a., & tuammee, s. (2015). the development of a decision support framework for a quantitative risk assessment in multimodal green logistics: an empirical study. international journal of production research, 54(4), 1020-1048. doi: https://doi.org/10.1080/00207543.2015.1041570 korpela, j., lehmusvaara, a., & nisonen, j. (2007). warehouse operator selection by combining ahp and dea methodologies. international journal of production economics, 108(1-2), 135-142. doi: https://doi.org/10.1016/j.ijpe.2006.12.046 kulakowski, k. (2015). notes on order preservation and consistency in ahp. european journal of operational research, 245(1), 333–337. doi: https://doi.org/10.1016/j.ejor.2015.03.010 kumar, a., debnath, r. m., shankar, r., & prabhu, j., (2014). analyzing customer preference and measuring relative efficiency in telecom sector: a hybrid fuzzy ahp/dea study. telematics and informatics, 32(3), 447-462. doi: https://doi.org/10.1016/j.tele.2014.10.003 kumar, k. s., & babu, a. s. (2012). an integrated method using ahp, dea and gp for evaluating supply sources. international journal of services and operations management, 11(2), 123-150. doi: https://dx.doi.org/10.1504/ijsom.2012.045196 kuo, r. j., lee, l. y., & hu, t. l. (2010). developing a supplier selection system through integrating fuzzy ahp and fuzzy dea: a case study on an auto lighting system company in taiwan. production planning and control: the management of operations, 21(5), 468-484.doi: https://doi.org/10.1080/09537280903458348 lai, p. l., potter, a., beynon, m., & beresford, a. (2015). evaluating the efficiency performance of airports using an integrated ahp/dea-ar technique. transport policy, 42(2015), 75-85. doi: https://doi.org/10.1016/j.tranpol.2015.04.008 lee, a. h. i. (2009). a fuzzy ahp evaluation model for buyer–supplier relationships with the consideration of benefits, opportunities, costs and risks. international journal of production research, 47(15), 4255–4280. doi: https://doi.org/10.1080/00207540801908084 lee, s. k., mogi, g., & hui, k. s. (2013). a fuzzy analytic hierarchy process (ahp)/data envelopment analysis (dea) hybrid model for efficiently allocating energy r&d resources: in the case of energy technologies against high oil prices. renewable and sustainable energy reviews, 21(2013), 347355. doi: https://doi.org/10.1016/j.rser.2012.12.067 lee, s. k., mogi, g., lee, s. k., hui, k. s., kim, j. w. (2010). econometric analysis of the r&d performance in the national hydrogen energy technology development ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 264 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 for measuring relative efficiency: the fuzzy ahp/dea integrated model approach. international journal of hydrogen energy, 35(6), 2236-2246. doi: https://doi.org/10.1016/j.ijhydene.2010.01.009 lee, s. k., mogi, g., li, z., hui, k. s., lee s. k., hui, k. n., park, s. y., ha, y. j., & kim, j. w. (2011). measuring the relative efficiency of hydrogen energy technologies for implementing the hydrogen economy: an integrated fuzzy ahp/dea approach. international journal of hydrogen energy, 36(20), 1265512663. doi: https://doi.org/10.1016/j.ijhydene.2011.06.135 li, h. l., & ma, l. c. (2008). ranking decision alternatives by integrated dea, ahp and gower plot techniques. international journal of information technology & decision making,7(2), 241-258. doi: https://doi.org/10.1142/s0219622008002922 li, x., liu, y., wang, y., & gao, z. (2016). evaluating transit operator efficiency: an enhanced dea model with constrained fuzzy-ahp cones. journal of traffic and transportation engineering (english edition, 3(3), 215-225. doi: https://doi.org/10.1016/j.jtte.2016.05.004 liang, y. h. (2015). performance measurement of interorganizational information systems in the supply chain. international journal of production research, 53(18), 5484–5499. doi: https://doi.org/10.1080/00207543.2015.1026614 lin, m. i., lee, y. d., & ho, t. n. (2011). applying integrated dea/ahp to evaluate the economic performance of local governments in china. european journal of operational research, 209(2), 129-140. doi: https://doi.org/10.1016/j.ejor.2010.08.006 loron, a. s., loron, m. s., & peyvandi, g. (2015) an integrated fuzzy analytic hierarchy process-fuzzy data envelopment analysis (fahp-fdea) method for intelligent building assessment. the journal tehnički vjesnik – technical gazette (tv-tg), 2(2015), 383-389. doi: https://doi.org/10.17559/tv-20140418223907 lozano, s., & villa, g. (2009). multiobjective target setting in data envelopment analysis using ahp. computers & operations research, 36(2), 549-564. doi: https://doi.org/10.1016/j.cor.2007.10.015 mirhedayatian, s. m., & saen, r. f. (2011). a new approach for weight derivation using data envelopment analysis in the analytic hierarchy process. journal of the operational research society, 62, 1585–1595. doi: https://doi.org/10.1057/jors.2010.105 moon, j. & kang s. (2015). an integrated dea -ahp model for the acquisition of weapon system: selection of a next-generation fighter system in korea. journal of information and communication convergence engineering, 13(2), 97-108. doi: https://doi.org/10.6109/jicce.2015.13.2.097 omid, a., & zegordi, s. h. (2015). integrated ahp and network dea for assessing the efficiency of iranian handmade carpet industry. decision science letters, 4(2015), 477-486. doi: https://doi.org/10.5267/j.dsl.2015.6.002 pakkar, m. s. (2014). using dea and ahp for ratio analysis. american journal of ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 265 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 operations research, 4, 268–279. doi: https://doi.org/10.4136/ajor.2014.44026 pakkar, m. s. (2015). using data envelopment analysis and analytic hierarchy process for multiplicative aggregation of financial ratios. journal of applied operational research, 7(1), 23–35. pakkar, m. s. (2016). a hierarchical aggregation approach for indicators based on data envelopment analysis and analytic hierarchy process. systems, 4(1), 6. doi: https://doi.org/10.3390/systems4010006 parameshwaran, r., srinivasan, p. s. s., punniyamoorthy, m., charunyanath, s. t., & ashwin, c. (2009). integrating fuzzy analytical hierarchy process and data envelopment analysis for performance management in automobile repair shops. european journal of industrial engineering, 3(4), 450-467. doi: https://doi.org/10.1504/ejie.2009.027037 prasad, k. g. d., subbaiah, k. v., & prasad, m. v. (2017). supplier evaluation and selection through dea-ahp-gra integrated approach-a case study. uncertain supply chain management, 5, 369–382. doi: https://doi.org/10.5267/j.uscm.2017.4.001 ramanathan, r. (2006). data envelopment analysis for weight derivation and aggregation in the analytic hierarchy process. computers and operations research, 33(5), 1289-1307. doi: https://doi.org/10.1016/j.cor.2004.09.020 ramanathan, r. (2007). supplier selection problem: integrating dea with the approaches of total cost of ownership and ahp. supply chain management : an international journal, 12(4), 258-261. doi: https://doi.org/10.1108/13598540700759772 raut, r. d. (2014). an integrated delphi-ahp-dea-lpp multi criteria decision making approach for supplier selection and order quantity allocation system. international journal of logistics systems and management, 18(3), 366-393. doi: https://doi.org/10.1504/ijlsm.2014.062823 rezaei-shouroki, m., mostafaeipour, a., & qolipour, m. (2017). prioritizing of wind farm locations for hydrogen production: a case study. international journal of hydrogen energy, 42(15), 9500-9510. doi: https://doi.org/10.1016/j.ijhydene.2017.02.072 saen, r. f., memariani, a., & lotfi, f. h. (2005). determining relative efficiency of slightly non-homogeneous decision making units by data envelopment analysis: a case study in irost. applied mathematics and computation, 165, 313–328. doi: https://doi.org/10.1016/j.amc.2004.04.050 saleeshya, p. g., babu, a. s. (2012). a combined ahp-and dea-based approach to measure agility of manufacturing systems. international journal of business and systems research, 6, 431-455. doi: https://doi.org/10.1504/ijbsr.2012.049472 saaty, t. l. (2004) decision making: the analytic hierarchy and network processes (ahp/dea), journal of systems science and systems engineering, 1-34. ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 266 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 seiford l. m. (1994). a dea bibliography (1978–1992). in a. charnes, w. w. cooper, a. y. lewin and l. m. seiford (eds.). data envelopment analysis: theory, methodology and applications (437-469) boston, usa: kluwer. seiford, l. m. (1996). data envelopment analysis: the evolution of the state of the art (1978-1995). the journal of productivity analysis, 7(2/3), 99-137. doi: https://doi.org/10.1007/bf00157037 seiford l. m. (1997). a bibliography for data envelopment analysis (1978–1996). annals of operations research, 73, 393–438 sevkli, m., lenny koh, s. c., zaim, s., demirbag, m., & tatoglu, e. (2007). an application of data envelopment analytic hierarchy process for supplier selection: a case study of beko in turkey. international journal of production research, 45(9), 1973-2003. doi: https://doi.org/10.1080/00207540600957399 shirouyehzad, h., lotfi, f. h., arabzad, s. m., & dabestani, r. (2013). an ahp/dea ranking method based on service quality approach: a case study in hotel industry. international journal of productivity and quality management, 11(4), 434445. doi: https://doi.org/10.1504/ijpqm.2013.054169 singh, s., & aggarwal, r. (2014). deahp approach for manpower performance evaluation. journal of the operations research society of china, 2(3), 317-332. doi: https://doi.org/10.1007/s40305-014-0050-2 sinuany-stern, z., mehrez, a., & hadad, y. (2000). an ahp/dea methodology for ranking decision making units. international transactions in operational research, 7(2), 109-124. doi: https://doi.org/10.1111/j.1475-3995.2000.tb00189.x soltanifar, m., & hosseinzadeh lotfi, f. (2011). the voting analytic hierarchy process method for discriminating among efficient decision making units in data envelopment analysis. computers and industrial engineering, 60(4), 585-592. doi: https://doi.org/10.1016/j.cie.2010.12.016 sueyoshi, t., shang, j., & chiang, w. c. (2009). a decision support framework for internal audit prioritization in a rental car company: a combined use between dea and ahp. european journal of operational research, 199(1), 219-231. doi: https://doi.org/10.1016/j.ejor.2008.11.010 takamura, y., & tone, k. (2003). a comparative site evaluation study for relocating japanese government agencies out of tokyo. socio-economic planning sciences, 37(2), 85-102. doi: https://doi.org/10.1016/s0038-0121(02)00049-6 tavares g. (2002). a bibliography of data envelopment analysis (1978–2001). rutcor, rutgers university. retrieved from http://rutcor.rutgers.edu/pub/rrr/reports2002/1_2002.pdf. tseng, y. f., & lee, t. z. (2009). comparing appropriate decision support of human resource practices on organizational performance with dea/ahp model. expert systems with applications, 36(3), 6548-6558. doi: https://doi.org/10.1016/j.eswa.2008.07.066 ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 267 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 wang, y. m., & chin, k. s. (2009). a new data envelopment analysis method for priority determination and group decision making in the analytic hierarchy process. european journal of operational research, 195(1), 239-250. doi: https://doi.org/10.1016/j.ejor.2008.01.049 wang, y. m. , chin, k. s. , & luo, y. (2009). aggregation of direct and indirect judgments in pairwise comparison matrices with a re-examination of the criticisms by bana e costa and vansnick. information sciences, 179(3), 329–337. wang, y. m., chin, k. s., & poon, g. k. k. (2008). a data envelopment analysis method with assurance region for weight generation in the analytic hierarchy process. decision support systems, 45(4), 913-921. doi: https://doi.org/10.1016/j.dss.2008.03.002 wang, y. m., liu, j., & elhag, t. m. s. (2008). an integrated ahp-dea methodology for bridge risk assessment. computers and industrial engineering, 54(3), 514-525. doi: https://doi.org/10.1016/j.cie.2007.09.002 wang, y. m., parkan, c., & luo, y. (2008). a linear programming method for generating the most favorable weights from a pairwise comparison matrix. computers and operations research, 35(12), 3918-3930. doi: https://doi.org/10.1016/j.cor.2007.05.002 yadav, v., & sharma, m. k. (2015), an application of hybrid data envelopment analytical hierarchy process approach for supplier selection. journal of enterprise information management, 28(2), 218 242 yang, t., & kuo, c. (2003). a hierarchical ahp/dea methodology for the facilities layout design problem. european journal of operational research, 147(1),128-136. doi: https://doi.org/10.1016/s0377-2217(02)00251-5 yu, p., & lee, j. h. (2013). a hybrid approach using two-level som and combined ahp rating and ahp/dea-ar method for selecting optimal promising emerging technology. expert systems with applications, 40(1), 300-314. doi: https://doi.org/10.1016/j.eswa.2012.07.043 yuan, x. c., wang, q., wang, k., wang, b., jin, j. l., wei, y. m. (2013) china’s regional vulnerability to drought and its mitigation strategies under climate change: data envelopment analysis and analytic hierarchy process integrated approach. mitigation and adaptation strategies for global change, 20(3), 341-359. doi: https://doi.org/10.1007/s11027-013-9494-7 zarei, a., mehdiabadi, a., & javidnia, m. (2012). measuring the units efficiency using an integrated dea–ahp method: a case study of rivet producer. management science letters, 2(2012), 189-196. doi: https://doi.org/10.5267/j.msl.2011.09.006 zhang, j., & fu, s. (2012). an effective dea-ahp algorithm for evaluation of emergency logistics performance. international journal on advances in information sciences and service sciences, 14(12) https://doi.org/10.4156/aiss zhang, x., lee, c. k. m., & chen, s. (2012). supplier evaluation and selection: a hybrid model based on deahp and abc. international journal of production ijahp article: pradhan, olfati, patel/integrations and applications of analytic hierarchy process with data envelopment analysisa literature review international journal of the analytic hierarchy process 268 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.632 research, 50(7), 1877-1889. doi: https://doi.org/10.1080/00207543.2011.560908 analisís multicriterio para mejorar el servicio en gasolineras ijahp article: gonzález, pradenas/multi-criteria analysis to improve the service in gas stations international journal of the analytic hierarchy process 67 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.601 multi-criteria analysis to improve the service in gas stations francisco gonzález industrial engineering, university of concepción, chile 4070043, casilla 160-c, correo 3, concepción, chile fgonzalezj@udec.cl lorena pradenas* industrial engineering, university of concepción 4070043, casilla 160-c, correo 3, concepción, chile lpradena@udec.cl * contact for correspondence abstract in developing countries, oil consumption corresponds to 56% of total energy consumption. this generates competition between supply points, which are gas stations. given the scarce differentiation between these supply points and low margins for sales, the strategy adopted by these service stations depends on the correct identification of both external and internal factors. in the present study, six multi-criteria techniques and a “strengths, weaknesses, opportunities and threats” (swot) analysis are proposed to quantitatively evaluate the factors that affect a specific network of gas stations. a total of two sets of results are obtained and it was determined for the chosen set of analysis that the recommended alternative is the improvement of training for personnel and image of the brand. the factor with the greatest weight is the low operational risk of compliance with emergency regulations. the differences in the results cause some factors to be more important than others and the proposed implementation to be contrary to expectations. the contribution of this study is the analysis of the performance of different multi-criteria tools in an actual case using the same data source. keywords: ahp; anp; swot; gas stations; fuzzy logic; topsis 1. introduction fuel has a large impact on the productivity of a country. because of the competition between different fuel suppliers, each of them adapt their resource allocations as efficiently as possible. these resource allocations depend, to a large extent, on the strategy adopted by each company. in a developing country, net consumption of oil derivatives can be close to 56%, and one part of the supply chain of this industry is the gas stations. the gas stations are facilities that supply fuel for vehicular use and heating to the public (chima, 2007). these facilities offer various additional services that include car washing, motor lubricants, maintenance, convenience stores, tourist guides and electric motor recharging in addition to selling fuel. therefore, those in charge must mailto:fgonzalezj@udec.cl ijahp article: gonzález, pradenas/multi-criteria analysis to improve the service in gas stations international journal of the analytic hierarchy process 68 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.601 manage each of these services efficiently by designating positions according to the capabilities of the staff through a chart replicated in each of the locations. one of the challenges of this type of company is defining a commercial strategy that is differentiated from the competition (bello & cavero, 2008). strategic administration corresponds to all activities that allow for the long-term development of an organization with decisions based on future actions and the ability to identify both the internal and external environments of the organization (bartusková & kresta, 2015). corporations are developed well with correct business administration in accordance with the environment (gorëner et al., 2012a). therefore, these strategies can improve both the position of the company itself within the sector and generate benefits for all companies of the sector. in order to face the market efficiently, gas stations must evaluate their internal and external factors before defining the strategies to be implemented. the environment of a company can be identified through a strengths, weaknesses, opportunities and threats (swot) analysis, which intensifies the factors in different strategic criteria for greater support of the future decisions of the company (gorëner et al., 2012b). it is necessary to correctly define the main objective (goal) of the company, the evaluation criteria and subcriteria, and the alternatives or strategies to meet the objective (wind & saaty, 1980). unfortunately, this method is not capable of quantifying the importance of the factors, thus making it difficult to assess their impact on the strategic decision (mehmood et al., 2014). however, a quantitative evaluation method should be taken into account, such as a multi-criteria decision-making method (mcdm). the selection of criteria is a multiple decision problem used in various industrial companies (erdil & erbiyik, 2015). the factors that impact organizational performance are quantified by comparison, and thus a strategy of continuous improvement is reached (dulange et al., 2014). this method makes it possible to solve problems with a limited number of alternatives that require human participation because it depends on the knowledge of experts in a subject or sector (kubler et al., 2016). however, the uncertainty of the selection of alternatives in a network of gas stations is because of the vagueness with which they are presented, which is based on subjective interpretations according to what is required by the specific company (yussuff & poh yee, 2001). this allows the evaluator’s points of view to affect the final results (tavana et al., 2016). therefore, it is required to consider the uncertainty as fuzzy. 2. multi-criteria analyses 2.1 analytic hierarchy process (ahp) the ahp method is used for decision making based on paired comparisons between different alternatives that reflect the relative differences according to decision makers (saaty, 1977). its wider applications can be found in strategic planning, resource localization and conflict resolution (saaty, 1987). with the comparisons of each pair of factors, a matrix of comparisons is constructed and the values indicate the importance with which each element dominates the other with respect to an established criterion. this forms a matrix 𝐴 of 𝑚 × 𝑛 dimensions (see equation 1), where 𝑎𝑖𝑗 represents the priority between factor 𝑖 and factor 𝑗, and the ijahp article: gonzález, pradenas/multi-criteria analysis to improve the service in gas stations international journal of the analytic hierarchy process 69 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.601 reciprocal values of the lower half with respect to the diagonal correspond to the inverse values of the upper half (𝑎𝑗𝑖 = 1/𝑎𝑖𝑗 ) are the values of the diag 𝑎𝑖𝑗 = 1 when 𝑖 = 𝑗. 𝐴 = 𝑎𝑖𝑗 = [ 1 𝑎21 ⋮ 𝑎𝑚1 𝑎12 1 ⋮ 𝑎𝑚2 … 𝑎1𝑛 … 𝑎2𝑛 ⋮ … 1 ] (1) first, the criteria comparison matrix (ccm) that must be normalized by means of equations 2 and 3 is constructed. then, the normalized comparison matrices of subcriteria are obtained for each criterion (scm) and the normalized comparison matrices of alternatives (acm). then, the vectors of the local weights are multiplied by the vector of weights of the criteria in order to obtain the vector of global weights of the sub-criteria, the vectors of weights of the alternatives for the vector of weights of the corresponding sub-criterion and the vectors of weights of the alternatives for the corresponding vector of weights of the sub-criterion in order to obtain the normalized local vectors of the alternatives according to each criterion (local nva). finally, the matrix of the local nva is multiplied by the vector of weights of criteria to obtain the global normalized vector of alternatives (global nva). 𝑋𝑖𝑗 = 𝑎𝑖𝑗 ∑ 𝑎𝑖𝑗 𝑛 𝑖=1 (2) 𝑊𝑖 = ∑ 𝑋𝑖𝑗 𝑛 𝑗=1 𝑛 (3) 2.2 technique for order performance by similarity to ideal solution (topsis) the topsis method states that the best alternative must have the shortest distance with respect to the ideal positive solution (ips) that minimizes the costs and maximizes the benefits, and have the greatest distance to the ideal negative solution (ins) that maximizes the costs and minimizes the benefits (hwang & yoon, 1981; zhao & fang, 2016). to determine the weight of each criterion, the normalized vector 𝑟𝑖𝑗 = 𝑎𝑖𝑗 ∑ (𝑎𝑖𝑗 ) 2𝑛 𝑖=1⁄ is calculated from the elements 𝑎𝑖𝑗 of the 𝐴 comparison matrix. then, the entropy method 𝑒𝑗 = − ∑ 𝑟𝑖𝑗 ln 𝑟𝑖𝑗 𝑛 𝑗=1 ln 𝑛⁄ is used to decrease the effects of subjectivity, where 0 ≤ 𝑒𝑗 ≤ 1 and n are the number of alternatives (kim, 2016). the vector of weights 𝑤𝑗 = 1 − 𝑒𝑗 ∑ (1 − 𝑒𝑗 ) 𝑚 𝑗=1⁄ indicates the weighting of the global weights of all sub-criteria of the study, where m is the number of total criteria. a standardized weight matrix is constructed according to 𝑣𝑖𝑗 = 𝑤𝑗 × 𝑟𝑖𝑗 . then, an ideal positive solution a* (4) and an ideal negative solution (equation 5) are constructed, where 𝐽1 is the set of criteria of costs, 𝑣𝑗 ∗ is the distance between the index j and the closest value to the optimum, and 𝑣𝑗 − is the distance between the index j and the value farther from the optimum. the strengths and opportunities can be criteria of benefits and the weaknesses can be the criteria of costs. ijahp article: gonzález, pradenas/multi-criteria analysis to improve the service in gas stations international journal of the analytic hierarchy process 70 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.601 𝐴∗ = {(maxi 𝑎𝑖𝑗 |𝑗 ∈ 𝐽1), (mini 𝑣𝑖𝑗 |𝑗 ∈ 𝐽2), 𝑖 = 1,2, … , 𝑚} = 𝑣1 ∗, 𝑣2 ∗, … , 𝑣𝑛 ∗ (4) 𝐴− = {(mini 𝑎𝑖𝑗 |𝑗 ∈ 𝐽1), (maxi 𝑣𝑖𝑗 |𝑗 ∈ 𝐽2), 𝑖 = 1,2, … , 𝑚} = 𝑣1 −, 𝑣2 −, … , 𝑣𝑛 − (5) then, the distance between the objective and the positive ideal solution (𝑑∗ = √∑ (𝑣𝑖𝑗 − 𝑣𝑗 ∗) 2𝑛 𝑗=1 ) and the ideal negative solution (𝑑 − = √∑ (𝑣𝑖𝑗 − 𝑣𝑗 −) 2𝑛 𝑗=1 ) is calculated. finally, the relative closeness with respect to the ideal solution 𝐶𝐶𝑖 (6) where 0 ≤ cci ≤ 1. the higher the value of cci, the greater preference a certain alternative has. 𝐶𝐶𝑖 = ∑ 𝑑𝑖 − ∑ 𝑑𝑖 ∗ + ∑ 𝑑𝑖 − (6) 2.3 analytic network process (anp) the anp allows one to generalize the processes of paired comparisons (saaty, 1996). according to this method, all elements can be determined by intertwined interactions in the same hierarchical level (cluster) or with others. as a result, a supermatrix is constructed. this is a matrix divided into segments where each segment represents a relation between two clusters in a system. considering the fact that the swot analysis includes four clusters (goal, criteria, sub-criteria and alternatives), a supermatrix is proposed (equation 7) which is similar to that proposed in the study by yüksel & dagdeviren (2007). the way in which the swot factors are interrelated can be observed in figure 1 (živković et al., 2015). 𝑊 = goal factors subfactors alternatives ∶ [ 0 0 𝑤1 𝑊2 0 0 0 0 0 𝑊3 0 0 0 0 𝑊4 𝐼 ] (7) figure 1 interdependence between swot factors (živković et al., 2015) in equation 7, w1 is the vector that represents the impact of the general goal, w2 is the matrix that represents the interdependence of the factors, w3 is the matrix that denotes the impact of the factors on each of the sub-factors, and w4 is the matrix that denotes the impact of the sub-factors in each alternative. first, the matrix of comparisons of the swot factors (w1) should be determined by assuming that there is no dependence among them and the interdependence of each ijahp article: gonzález, pradenas/multi-criteria analysis to improve the service in gas stations international journal of the analytic hierarchy process 71 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.601 swot factor with respect to other factors (w2). then, the interdependent priorities of the factors must be calculated by multiplying w2 by w1 in order to determine the degrees of local importance of the sub-factors 𝑤𝑠𝑢𝑏𝑓𝑎𝑐𝑡𝑜𝑟(𝑙𝑜𝑐𝑎𝑙) and to determine the degrees of global importance of the sub-factors 𝑤𝑠𝑢𝑏𝑓𝑎𝑐𝑡𝑜𝑟(𝑔𝑙𝑜𝑏𝑎𝑙) by multiplying 𝑤𝑠𝑢𝑏𝑓𝑎𝑐𝑡𝑜𝑟(𝑙𝑜𝑐𝑎𝑙) by 𝑤𝑓𝑎𝑐𝑡𝑜𝑟. finally, the importance degrees of the alternatives (w4) with respect to each sub-criterion are determined and the global priorities of the alternatives are calculated by multiplying w4 by 𝑤𝑠𝑢𝑏𝑓𝑎𝑐𝑡𝑜𝑟(𝑔𝑙𝑜𝑏𝑎𝑙). 2.4 fuzzy ahp (fahp) fuzzy logic allows uncertain information to be presented through a triangular fuzzy number (tfn) (zadeh, 1965). it is denoted as (a, b, c) so that (a < b < c), where a is the lowest possible value, b is the most promising value, and c is the highest possible value. the fahp method provides a range of values that incorporate the indecision of the decision makers. according to recent research, it is the second most widely used independent technique after ahp and its applications, and its applications have been diverse in various sectors such as manufacturing, industry, planning and resource allocation (kubler et al., 2016; rouyendegh & erkan, 2012). the degree of possibility of each alternative is based on the extensive method of chang (1996) and is used to evaluate fuzzy paired comparisons. the matrix of fuzzy comparisons �̃� of dimensions 𝑚 × 𝑛 that contain all the paired comparisons 𝑎𝑖�̃� is constructed between elements 𝑖 and 𝑗. then, the geometric mean of the fuzzy comparative values of each criterion �̃�𝑖 = (∏ 𝑎𝑖�̃� n j=1 ) 1/n is constructed. to determine the weights of the criteria and evaluate the alternatives, it is necessary to calculate the fuzzy weights of each criterion according to equation 8. 𝑟1̃ = [(l11 ∗ l12 ∗ l13) 1 3; (m11 ∗ m12 ∗ m13) 1 3; (u11 ∗ u12 ∗ u13) 1 3] = [a1; b1; c1] (8) then, the sum vector is calculated from each �̃�𝑖 so that (∑ 𝑎𝑖 , ∑ 𝑏𝑖 , ∑ 𝑐𝑖 ). thus, the inverse of each reordered element is determined so that the lowest value is in the first position and the highest value is in the last one. thus, the fuzzy weight of each criterion 𝑖 (𝑤�̃�) is obtained by multiplying each �̃�𝑖 by its reverse vector. since 𝑤�̃� corresponds to fuzzy values, it is necessary for it to be “defuzzified” and converted into unique values (𝑀𝑖 = 𝑎𝑖𝑤1+𝑏𝑖𝑤2+𝑐𝑖𝑤3 3 ). finally, the values are normalized. then, by multiplying each weight of the alternatives to which they belong, the global scores of each alternative are obtained. alternatives with higher scores suggest better results for the decision maker. 2.5 fuzzy topsis (ftopsis) the ftopsis method is used to solve localization problems such as the selection of suppliers and the aspects of renewable energies and sustainability (chen & huang, 1992; nadaban, 2016). it is usually combined with fahp in the problems of the selection and evaluation of strategies (nadaban, 2016). assuming that there are k members in the group, the fuzzy score of the k-th decision maker about alternative ai with respect to ijahp article: gonzález, pradenas/multi-criteria analysis to improve the service in gas stations international journal of the analytic hierarchy process 72 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.601 criterion cj is denoted as �̃�𝑖𝑗 𝑘 = (𝑎𝑖𝑗 𝑘 , 𝑏𝑖𝑗 𝑘 , 𝑐𝑖𝑗 𝑘 ) (equation 9). the weight of the criterion cj is denoted as �̃�𝑖𝑗 𝑘 = (𝑤𝑗1 𝑘 , 𝑤𝑗2 𝑘 , 𝑤𝑗3 𝑘 ). then, the “aggregate” fuzzy ranking (𝑎𝑖𝑗 , 𝑏𝑖𝑗 , 𝑐𝑖𝑗 ) of the j-th criterion must be calculated (equation 10). (𝑎𝑖𝑗 , 𝑏𝑖𝑗 , 𝑐𝑖𝑗 ) = (min 𝑘 {𝑎𝑖𝑗 𝑘 } , 1 𝐾 ∑ 𝑏𝑖𝑗 𝑘 𝐾 𝑘=1 , max 𝑘 {𝑐𝑖𝑗 𝑘 }) (9) (𝑤𝑗1, 𝑤𝑗2, 𝑤𝑗3) = (min 𝑘 {𝑤𝑗1 𝑘 } , 1 𝐾 ∑ 𝑤𝑗2 𝑘 𝐾 𝑘=1 , max 𝑘 {𝑤𝑗3 𝑘 }) (10) then, the normalized fuzzy decision matrix is calculated as �̃� = [�̃�𝑖𝑗 ]. if the criterion is a benefit, equation 11 will be used. if the criterion is a cost, equation 12 would be used. �̃�𝑖𝑗 = ( 𝑎𝑖𝑗 𝑐𝑗 ∗ , 𝑏𝑖𝑗 𝑐𝑗 ∗ , 𝑐𝑖𝑗 𝑐𝑗 ∗ ) ; 𝑐𝑗 ∗ = max 𝑘 {𝑐𝑖𝑗 } (11) �̃�𝑖𝑗 = ( 𝑎𝑗 − 𝑐𝑖𝑗 , 𝑎𝑗 − 𝑏𝑖𝑗 , 𝑎𝑗 − 𝑎𝑖𝑗 ) ; 𝑎𝑗 − = min 𝑘 {𝑎𝑖𝑗 } (12) with the above, the standardized fuzzy decision weight matrix is calculated. this matrix is denoted with �̃� = (�̃�𝑖𝑗 ), where �̃�𝑖𝑗 = �̃�𝑖𝑗 × �̃�𝑗. then, the fuzzy positive ideal solution (fpis) (equation 13) and the fuzzy negative ideal solution (fnis) (14) are calculated. 𝐴∗ = (�̃�1 ∗, �̃�2 ∗, … , �̃�𝑛 ∗ ), �̃�𝑗 ∗ = max 𝑖 {𝑣𝑖𝑗3} (13) 𝐴− = (�̃�1 −, �̃�2 −, … , �̃�𝑛 −), �̃�𝑗 − = min 𝑖 {𝑣𝑖𝑗1} (14) let x̃ = (a1, b1, c1) and ỹ = (a2, b2, c2). the distance between x̃ and ỹ is calculated according to equation 15. 𝑑(�̃�, �̃�) = √ 1 3 [(𝑎1 − 𝑎2) 2 + (𝑏1 − 𝑏2) 2 + (𝑐1 − 𝑐2) 2] (15) from each alternative 𝐴𝑖 , the distance to fpis and fnis (𝑑𝑖 ∗ = ∑ 𝑑(�̃�𝑖𝑗 , �̃�𝑗 ∗)𝑛𝑗=1 and 𝑑𝑖 − = ∑ 𝑑(�̃�𝑖𝑗 , �̃�𝑗 −)𝑛𝑗=1 ) and the proximity coefficient of each alternative 𝐶𝐶𝑖 = 𝑑𝑖 − (𝑑𝑖 − + 𝑑𝑖 ∗)⁄ , which indicates which of all alternatives is the most preferred, are calculated. to “defuzzify” the weights of the sub-criteria, the procedure from the study by kacprzak (2017) is used. to represent the benefit criteria, the triangular positive values (𝑎𝑖𝑗 , 𝑏𝑖𝑗 , 𝑐𝑖𝑗 ) where (𝑎𝑖𝑗 ≤ 𝑏𝑖𝑗 ≤ 𝑐𝑖𝑗 ) are used. for the values of the cost criteria, the triangular negative values (𝑎𝑖𝑗 , 𝑏𝑖𝑗 , 𝑐𝑖𝑗 ) where (𝑎𝑖𝑗 ≥ 𝑏𝑖𝑗 ≥ 𝑐𝑖𝑗 ) are used. the steps to obtain the weights are described as follows. first, build a fuzzy decision matrix with ijahp article: gonzález, pradenas/multi-criteria analysis to improve the service in gas stations international journal of the analytic hierarchy process 73 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.601 “aggregate” fuzzy scores while considering if they are criteria of benefits or costs. then, normalize the matrix of the previous step with the elements (𝑎𝑖𝑗 𝑁 , 𝑏𝑖𝑗 𝑁 , 𝑐𝑖𝑗 𝑁 ). after that, construct the entropy fuzzy vector 𝑒𝑗 (equation 16), where m is the number of alternatives and calculate the fuzzy vector of weights of each file as a value of "the fuzzy vector of weights of criteria 𝑤𝑗” (equation 17), where n is the number of sub-criteria. finally, select the central value of the fuzzy vector of weights in each row as a value of the “defuzzified” weight of each criterion. 𝑒𝑗 = (− ∑ 𝑎𝑖𝑗 𝑁𝑚 𝑖=1 ln 𝑚 ln 𝑎𝑖𝑗 𝑁 = 𝑒𝑗(0), − ∑ 𝑏𝑖𝑗 𝑁𝑚 𝑖=1 ln 𝑚 ln 𝑏𝑖𝑗 𝑁 = 𝑒𝑗(1), − ∑ 𝑐𝑖𝑗 𝑁𝑚 𝑖=1 ln 𝑚 ln 𝑐𝑖𝑗 𝑁 = 𝑒𝑗(2)) (16) 𝑤𝑗 = ( 1 − 𝑒𝑗(0) ∑ (1 − 𝑒𝑗(0)) 𝑛 𝑖=1 , 1 − 𝑒𝑗(1) ∑ (1 − 𝑒𝑗(1)) 𝑛 𝑖=1 , 1 − 𝑒𝑗(2) ∑ (1 − 𝑒𝑗(2)) 𝑛 𝑖=1 ) (17) 2.6 fuzzy anp (fanp) fanp analysis includes both the interdependence of criteria and the internal dependence of criteria within the matrix of comparison, which corresponds to a combination of the anp and ahp methods, with fuzzy parameters (reza et al., 2016). in the model suggested for the analysis in this study, the following steps are included.  determine the local weights of the criteria and sub-criteria of each alternative by assuming that there are no internal dependencies from the saaty fuzzy scale.  determine the fuzzy scale of the interdependence matrix of each factor with respect to the other factors in each alternative. this matrix is multiplied by the local weights to obtain the interdependence weights of the factors.  calculate the total weights of the sub-factors in each alternative. the global steps of sub-factors are calculated by multiplying the local weights of the sub-factors with the interdependence weight of the factor to which they belong.  determine the importance of each alternative through a ranking. 3. literature review there are several studies that combine multi-criteria techniques with a swot matrix. in görener et al. (2012), an ahp-swot matrix is proposed to determine the priorities of both internal and external factors of a kitchen hood company in turkey. in the study by mehmood et al. (2014), they determined the most effective factors to adopt a specific cell phone technology in an italian company. erdil & erbiyik (2015) determined the best development strategy for a dairy company in turkey. moreover, the study by yüksel & dagderiven (2007) showed a process to quantify the swot matrix with dependence among other strategic factors in a textile company in turkey. görener (2012) proposed an improvement to the swot matrix with the ahp-anp method in a manufacturing company also in turkey and used both methods in the prioritization of swot factors and the analysis of the differences in their results with the indicated interdependencies. the study by živković et al. (2015) used the anp-swot method to generate swot strategies in the case of a technical college in serbia. zare et al. (2015) presented a ijahp article: gonzález, pradenas/multi-criteria analysis to improve the service in gas stations international journal of the analytic hierarchy process 74 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.601 swot matrix to evaluate the supply chain of electricity in a region of iran. the ahp method was integrated with ftopsis as a proposal to prioritize the swot factors. the results show that the proposed method can be used to determine the strategic plan with high prioritization for planning and decision making in the supply chain. tavana et al. (2016) proposed an fahp method to identify the decision criteria in the selection of the best suppliers for an outsourcing company through a programmatic model in order to produce local and global weights and thus create a global ranking. islam et al. (2017) implemented the swot analysis together with topsis in order to find the best strategy inside a pharmaceutical company in bangladesh and the ranking of each of the criteria in the swot matrix was performed. by using combined methods, ervural et al. (2017) proposed a hybrid method for the analysis of the energy sector in turkey with swot, anp and topsis to formulate and analyze the alternatives of energy strategies and their priorities. this method makes it possible to identify the relevant criteria and sub-criteria by using swot, then anp to determine the weights of each factor, and finally ftopsis to prioritize the alternatives. shahba et al. (2017) applied an ahp-topsis method and swot factors and strategies for waste management iron mines in iran. the ranking of factors was qualitatively determined, where ahp was used to calculate the weight of the criteria and topsis was used to take advantage of their ability to use both negative and positive criteria. from the literature review, it can be concluded that fuzzy logic allows one to eliminate or diminish the problems of classical logic by reducing the uncertainty of decisions and the imprecision in the companies due to the number of experts involved in the lifecycle of the product (from designers to final sellers) (kubler et al., 2016). this also increases the complexity of the judgments of the experts. in addition, multi-criteria decision methods are very versatile. they are used in various types of industries globally or locally, in small or large companies, and in manufacturing or services, and they can be applied together to determine the order of preference according to the conditions of the company. for this reason, this work aimed to identify the most important factors of a network of gas stations through swot analysis and quantitatively analyzed different multi-criteria techniques, such as classical logic and fuzzy logic, in order to compare the results of their weightings and to determine the most appropriated method for global evaluations. 4. methodology 4.1 swot analysis the swot analysis is the most popular method used in strategic analysis (tavana et al. 2016). it identifies the internal and external factors of an organization, which are known as strengths, weaknesses, opportunities and threats. with this analysis, it is possible to build the swot matrix and define the strategies (alternatives) with each pair of factors: sos (strengths-opportunities), which require a good use of the opportunities by using the existing strengths in the organization; wos (weakness-opportunities), which obtain the benefits from external opportunities by considering the weaknesses of the organization; sts (strengths-threats), which use the strengths of the organization and remove or reduce the effects of the threats; and wts (weaknesses-threats), which consider the efforts of the company to reduce the effects of threats by considering the weaknesses. ijahp article: gonzález, pradenas/multi-criteria analysis to improve the service in gas stations international journal of the analytic hierarchy process 75 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.601 4.2 surveys a survey was prepared for a population of executives (7 in total), and it contained three sections. the first section was for ahp and fahp and contained two parts and 210 questions, including the comparison of each criterion/sub-criterion with respect to the others and a comparison of swot alternatives considering each sub-criterion. the second section was for topsis and ftopsis. it also had two parts and 120 questions to evaluate the importance of each sub-criterion with respect to the global goal and to evaluate the importance of each alternative with respect to each sub-criterion. finally, the third section of the survey was for anp and fanp and considered 12 questions to evaluate the internal dependencies among the factors and consider the importance of each factor over another. the respondents marked some of the cells indicated in the numbered columns (1 to 5). the more distant it was from the center of the table, the greater the degree of importance that criterion had over the other. if there was any doubt, the mark could be placed between two adjacent columns. the meaning of each value that corresponds to the reciprocal value and the values converted to the saaty scale and fuzzy scale can be seen in table 1. all of the executives responded to the survey, where (dei) is the population of executives surveyed and i is the number of administrators of the gas station network. table 1 saaty survey scale (saaty, 1987) survey significance saaty reciprocal fuzzy reciprocal 1 equally important 1 1 1 1 3 1/3 1 1 1.5 between 1 and 2 2 ½ 1 2 4 1/4 1/2 1 2 relative importance 3 1/3 1 3 5 1/5 1/3 1 2.5 between 2 and 3 4 ¼ 2 4 6 1/6 1/4 ½ 3 strong importance 5 1/5 3 5 7 1/7 1/5 1/3 3.5 between 3 and 4 6 1/6 4 6 8 1/8 1/6 ¼ 4 very strong importance 7 1/7 5 7 9 1/9 1/7 1/5 4.5 between 4 and 5 8 1/8 6 8 9 1/9 1/8 1/6 5 absolute importance 9 1/9 7 9 9 1/9 1/9 1/7 4.3 multi-criteria analysis since the ratings are subjective (and there are inconsistencies in the chosen decisions) from the consistency vector per the obtained matrix (equation 18), the measure of consistency is calculated as 𝜆𝑚𝑎𝑥 = ∑ 𝐶𝑣𝑖𝑗 /𝑊𝑖 𝑛 𝑖=1 . [ 𝑎11 𝑎12 𝑎13 𝑎21 𝑎22 𝑎23 𝑎31 𝑎32 𝑎33 ] ∗ [ 𝑊1 𝑊2 𝑊3 ] = [ 𝐶𝑣1 𝐶𝑣2 𝐶𝑣3 ] (18) then, the consistency index is calculated by 𝐶𝐼 = 𝜆𝑚𝑎𝑥−𝑛 𝑛−1 , which reflects the consistency of the judgments of each decision maker. finally, the consistency ratio 𝐶𝑅 = 𝐶𝐼/𝐼𝐴 is ijahp article: gonzález, pradenas/multi-criteria analysis to improve the service in gas stations international journal of the analytic hierarchy process 76 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.601 calculated. in it, if cr = 0, the matrix is consistent; if cr ≤ 0.1, the matrix has an acceptable consistency; and if cr > 0.1, the matrix has an inadmissible consistency and the paired comparisons must be reevaluated. ia is the consistency index of a random matrix of order n, which is obtained depending on the size of the matrix (n). data consistency was performed for each of the comparison matrices and obtained from each respondent for the ahp and anp analyses, except for topsis in which no paired assessments are considered. once all of the experts had delivered their results (k = 1…m experts), each provided its own result (𝑎𝑖𝑗 𝑘 ) on the relative importance of a criterion over another. the global preference of that specific comparison was obtained with equation 19, where i corresponded to the question, j to the respondent and n to the total number of respondents. for the values of fuzzy logic, equation 20 is used. then, it is possible to calculate the weights of the criteria and the scores of the alternatives according to the steps of each multi-criteria analysis. 𝑎𝑖𝑗 𝑔𝑙𝑜𝑏𝑎𝑙 = ∏(𝑎𝑖𝑗 𝑘 )1/𝑚 𝑚 𝑘=1 (19) (𝑙𝑖𝑗 , 𝑚𝑖𝑗 , 𝑢𝑖𝑗 )𝑔𝑙𝑜𝑏𝑎𝑙 = (∏(𝑙𝑖𝑗 𝑘 )1/𝑚 𝑚 𝑘=1 , ∏(𝑚𝑖𝑗 𝑘 )1/𝑚 𝑚 𝑘=1 , ∏(𝑢𝑖𝑗 𝑘 )1/𝑚 𝑚 𝑘=1 ) (20) 4.4 analysis of alternatives and sub-criteria a ranking of alternatives and global weights of sub-criteria was constructed. the analysis of alternatives involves comparing the obtained results, determining sets of analysis with similar results and observing which alternatives have the greatest preference. the analysis of global weights is demonstrated in a graph where the line in each quadrant represents the global importance of each group and each point represents the global priorities of each individual factor (mehmood, et al., 2014). if there are more points close to the final edge of the line, the criterion that includes them is considered more influential. to determine the influence of each sub-criterion in the sequence of alternatives of each set, a sequence table was defined, and obtained from the alternative scores for each subcriterion, which is compared with the sequence of alternatives of each set. the greater the similarity of the preferred alternatives, the greater influence the sub-criterion has in the average sequence of each set, and therefore greater consideration should be given to that sub-criterion. ijahp article: gonzález, pradenas/multi-criteria analysis to improve the service in gas stations international journal of the analytic hierarchy process 77 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.601 5. results both the criteria and sub-criteria are defined in table 2 based on the indicators defined by the decision makers. table 2 criteria and sub-criteria criteria initial sub-criteria strengths (s) s1 predictable long-term revenue stream. s2 integrated management between administrators and owners. s3 accurate control of fuel sales. s4 brand support, technical and operational. s5 staff has years of experience and low turnover. s6 low operational risks. weaknesses (w) w1 very variable fuel order volume. w2 external financing for the payment of credits. w3 low profit margins. w4 lack of staff training of the attendants. w5 possibility of loss of money by staff. w6 high dependence on suppliers. opportunities (o) o1 high demand for fuel. o2 reduced conflicts of workers’ union. o3 station location in crowded places. o4 generation of other types of businesses in addition to fuel. o5 generates great circulation of people. o6 field sales to companies outside the station. threats (t) t1 high external and internal competition (same brand). t2 low differentiation of service with respect to the competition. t3 needs of vehicles with alternative energy sources. t4 direct sales from the oil providers. t5 assault risk. t6 price increases and customer decisions. with the sub-criteria, it was possible to define four alternatives to guide the decisions of the company. these are shown in the swot matrix (table 3). table 3 swot matrix and alternatives criteria strengths weaknesses opportunities so: company expansion through alternative business other than automobile services wo: improve personnel training (attention and image) threats st: improve facility security to prevent theft wt: differentiation of service for all types of vehicles ijahp article: gonzález, pradenas/multi-criteria analysis to improve the service in gas stations international journal of the analytic hierarchy process 78 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.601 according to the consistency analysis for both the ahp and anp methods, all the decision makers obtained an average consistency ratio of less than 0.1, thus verifying that all decision makers comply with an acceptable inconsistency. in table 6 (see appendix), the cr values of each decision maker (𝐷𝐸𝑖 ) are displayed. the results of each methodology (table 7, see appendix) are classified into two groups. in the first group, the analyses ahp-fahp-anp-fanp (hierarchical order analysis: set 1) agree that the order of alternatives is wo (0.295) > so (0.281) > st (0.252) > wt (0.172). in the second group, the topsis-ftopsis analyses (preferential order analysis: set 2) agree that the order of alternatives is so (0.288) > wo (0.258) > st (0.246) > wt (0.208). the average standard deviation of set 1 is 0.003 and is 0.095 for set 2. table 8 (see appendix) shows the sub-criterion weight values for each applied method. 6. discussion figure 2 shows the results of the global weights of the sub-criteria calculated according to each set in order to reduce the data’s standard deviation from table 8. the main difference in the results of the scores of the alternatives is the position of the alternative wo, which ranks first in set 1 and second in set 2. while analyses ahp-fahp-anpfanp (hierarchical order: set 1) compare each possible paired combination, analyses topsis-ftopsis (preferential order: set 2) only require one to know the general preference of a factor and/or alternative without considering the others. given this, some factors are more important for a certain set and less important in others. if decision makers consider only set 1, the best alternative would be wo, but if they only consider set 2, the alternative would be so. in figure 2a, the maximum lengths in each quadrant are s6 (0.089), w3 (0.045), o1 (0.083) and t5 (0.032). both the strengths and opportunities have more weight than weaknesses and threats because each length of the benefit criteria is greater and its points are better distributed towards the outer end of the line. in figure 2b, the maximum lengths in each quadrant are s3 (0.095), w5 (0.082), o2 (0.064) and t5 (0.091). it is not clear whether the benefit criteria outweigh the cost criteria since the strengths have points that are distributed close to the central axis and the opportunities have the shortest length of all quadrants. each set can generate different distributions of preferences among themselves despite corresponding to the same population of respondents. ijahp article: gonzález, pradenas/multi-criteria analysis to improve the service in gas stations international journal of the analytic hierarchy process 79 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.601 a) b) figure 2 graphical representations of the average global weights of the sub-criteria according to each set: a) results obtained for set 1, and b) results obtained for set 2 the selection of a multi-criteria analysis must be made with caution because the strategic guidelines in the company are defined from these sub-criteria. this is important because in practice this could lead to different effects than those considered. in addition, according to the chosen set, there are some sub-criteria with sequences similar to the average sequence of alternatives. those with greater similarity could exert a greater influence in the average sequence of alternatives of each set and could be determinants when defining strategies apart from the sub-criteria with the highest global and local scores. table 4 shows, for each sub-criterion, which set obtains the greatest global and local percentage and which set has the greatest influence in the sequence of average alternatives. it indicates which alternatives have coincidences in each set, the positions (locations) of these alternatives and which set has the greatest importance in the average sequence. s1, s2, s3, s6, w2, o1, o3, o4, o6 and t4, despite having greater global weights using a certain set, have importance in the sequence of alternatives of the opposite set. in these cases, the order of alternatives of the same set is not influenced by a greater global s1 s2s3 s4 s5 s6 w1 w2 w3w4 w5 w6 o1 o2o3 o4 o5 o6 t1 t2t3t4 t5 t6 -0.120 -0.090 -0.060 -0.030 0.000 0.030 0.060 0.090 0.120 -0.100 -0.075 -0.050 -0.025 0.000 0.025 0.050 0.075 0.100 strenghts weaknesses opportunities threats s1 s2 s3 s4 s5 s6 w1 w2w3 w4w5 w6 o1 o2 o3 o4o5 o6 t1 t2 t3 t4 t5 t6 -0.120 -0.090 -0.060 -0.030 0.000 0.030 0.060 0.090 0.120 -0.100 -0.075 -0.050 -0.025 0.000 0.025 0.050 0.075 0.100 strenghts weaknesses opportunities threats ijahp article: gonzález, pradenas/multi-criteria analysis to improve the service in gas stations international journal of the analytic hierarchy process 80 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.601 percentage. t2, t3 and t6, despite having greater global weights using a certain set, generate no greater influence in the average order. in the rest of the sub-criteria, the set with the greatest global percentage coincides with the average order: s4, s5 and o2 using set 1, and w3, w4, w5, w6, o5, t1 and t5 using set 2. these last sub-criteria should be considered when selecting some set for this problem because they have greater agreement with the global preferences of the alternatives of each set. the last row corresponds to data in table 9 (see appendix), and shows the set with greater similarity of all subcriteria with respect to average sequences. ijahp article: gonzález, pradenas/multi-criteria analysis to improve the service in gas stations international journal of the analytic hierarchy process 81 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.601 table 4 for each sub-criteria, the set with the highest global and local percentage, and the set with the greatest influence on the average sequence of alternatives subcriteria set 1 (%) set 2 (%) set with highest global % set with highest local % set with the greatest influence on alternatives (table 9) rl rg rl rg s1 12.8 4.8 9.8 2.2 1 2 2 s2 18.7 7.0 21.7 4.8 1 2 2 s3 20.2 7.5 43.3 9.5 2 2 1 s4 8.7 3.2 12.7 2.8 1 2 1 s5 15.7 5.8 1.8 0.4 1 1 similar s6 23.9 8.9 10.6 2.3 1 1 2 w1 4.8 0.8 1.4 0.4 1 1 neither w2 11.0 1.9 11.1 3.0 2 2 1 w3 26.0 4.5 11.7 3.2 1 1 1 w4 23.9 4.2 27.3 7.5 2 2 2 w5 21.3 3.7 30.0 8.2 2 2 same w6 13.0 2.3 18.6 5.1 2 2 2 o1 24.0 8.3 2.2 0.4 1 1 2 o2 20.5 7.1 40.6 6.4 1 2 1 o3 21.6 7.5 3.1 0.5 1 1 2 o4 16.8 5.8 23.3 3.7 1 2 2 o5 6.1 2.1 25.7 4.1 2 2 2 o6 11.0 3.8 5.0 0.8 1 1 2 t1 19.7 2.1 13.1 4.6 2 1 similar t2 11.4 1.2 16.5 5.7 2 2 neither t3 8.9 0.9 24.5 8.6 2 2 neither t4 12.5 1.3 12.4 4.3 2 1 1 t5 30.0 3.2 26.1 9.1 2 1 2 t6 17.5 1.9 7.4 2.6 2 1 neither in table 5, the main disadvantages of each of the analyses are detailed from the data in table 10 (see appendix). based on this, for this case it is not recommended to use the methodologies from set 2 because several sub-criteria have similar values to each other (figure 2b), thus preventing the clear identification of any criterion being superior to the rest, and the deviation of the results compared with set 1. in addition, the definition of the swot sub-criteria may not be evident due to the uncertainty of the administrators themselves. this could require reevaluating a new set of factors and verifying new values of alternatives of greater proximity between set 1 and set 2. the global ranking of sub-criteria according to the sub-criteria of set 1 is shown in figure 3 and is defined as the set of analyses used for decision making in the study. ijahp article: gonzález, pradenas/multi-criteria analysis to improve the service in gas stations international journal of the analytic hierarchy process 82 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.601 table 5 main disadvantages of each analysis analysis main disadvantages ahp-fahp it requires paired comparisons by each decision-maker on the criteria, sub-criteria and alternatives and compliance with a maximum consistency ratio of 10%. the preferences are more restricted. anp-fanp the anp increased the threats (21%), while the rest only slightly changed (approximately 4%). the fanp increased the strengths (14%), reduced the weaknesses (40%), increased the opportunities (11%) and reduced the threats (53%). however, the changes do not generate significant deviations in the analyses of set 1. thus, they are a complement to ahpfahp in this study. topsisftopsis they only require preferences of each sub-criterion (only applies in ftopsis) and preferences of each alternative over each sub-criterion, thereby being the least complex technique in this study. in addition, the standard deviation of the global alternatives scores is higher. this causes the averaged weight of sub-criteria to be very different from set 1 (table 10). for example, differences greater than 200% are seen in the threats, which may be triggered by the subjectivity. figure 3 final global ranking of the sub-criteria the distribution of factors indicates a high tendency for strengths and opportunities and a low tendency for weaknesses and threats. the sub-criteria with the greater weight for each criterion are: s6, o1, w3 and t5; whereas the sub-criteria with the lowest weight for each criterion are: s4, o5, t3 and w1. in addition, s4 (14 th place), s5 (8 th place) and o2 (5 th place) are the factors with the greatest similarity in the sequence of alternatives (according to table 9). although the strengths and opportunities are stronger than the weaknesses and opportunities, the wo was the one that obtained a greater preference. then, weaknesses and opportunities should be globally stronger than the rest. finally, the 0.000 0.010 0.020 0.030 0.040 0.050 0.060 0.070 0.080 0.090 0.100 s6 o1 s3 o3 o2 s2 o4 s5 s1 w3 w4 o6 w5 s4 t5 w6 o5 t1 w2 t6 t4 t2 t3 w1 n o rm a li ze d w e ig h ts subcri teri a ijahp article: gonzález, pradenas/multi-criteria analysis to improve the service in gas stations international journal of the analytic hierarchy process 83 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.601 contribution of this study is the analysis of how the different multi-criteria analyses perform in an actual case using the same population of interviewees where the results are very different from each other. 7. conclusions the consumption of oil in a developing country generates high competition within the last link of the supply chain which is gas stations. the allocation of the resources of each gas station depends on the strategy adopted by the company, and one of the challenges of these companies is to select the appropriate strategy. several multi-criteria decisionmaking techniques (mcdm) are proposed in this study in order to improve the service provided by these companies. ahp, topsis, anp, fahp, ftopsis and fanp were used with foda analysis to determine the preferences of the alternatives and weights of sub-criteria in a company that manages a network of gas stations. through a service, the administrators of this company evaluated these factors comparatively. with six multi-criteria analyses, differences in the sequence of alternatives were observed. in the ahp-fahp-anp-fanp analyses (hierarchical order, set 1), the following sequence was obtained: wo > sw > wt > da. in the topsisftopsis analyses (preferential order, set 2) the sequence was sw >wo > wt > da. the difference in how the analyses collect information is as follows. in set 1, each possible paired combination is compared, and in set 2, it is only the intrinsic preference of each factor and/or alternative that must be known, which causes some factors to be more important for one set of analyses than others. regarding sub-criteria in set 1, the strengths and opportunities have more weight than the weaknesses and threats. in set 2, it has not been determined if the benefit criteria have a greater weight than the cost criteria or vice versa. each set generates a distribution of preferences, which is different despite originating in the same population of respondents. the choice of a multi-criterion analysis should be made with caution because when implementing the results in reality the effects are often different than expected. in addition, according to the chosen set, some sub-criteria have sequences that closely resemble the average sequence of alternatives. those with greater similarity would exert a greater influence in the average sequence of alternatives of each set that could be determined when defining strategies. specifically, this occurs with s4, s5 and o2 (using set 1) and w3, w4, w5, w6, o5, t1 and t5 (using set 2). when considering choosing one of these sets for the problem, they have greater concordance with the global preference of alternatives of each set. in relation to the advantages of the studied methods, it is not recommended to use the methodologies of set 2 for this case because they are less precise about the sub-criteria. therefore, set 1 was defined for decision making and the preferred alternative is wo (the improvement of training for the staff and the brand’s image). the sub-criteria with the greatest weight of each criterion are: s6, o1, w3 and t5; whereas the sub-criteria with the lowest weight for each criterion are: s4, o5, t3 and w1. in addition, s4 (14 th place), s5 (8 th place) and o2 (5 th place) are the factors with the greatest similarity in the sequence of alternatives. despite that, the strengths and weaknesses were stronger than the weaknesses and threats. wo was the most selected alternative. then, weaknesses and ijahp article: gonzález, pradenas/multi-criteria analysis to improve the service in gas stations international journal of the analytic hierarchy process 84 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.601 opportunities should be globally stronger than the rest. in addition, the contribution of this study is the analysis of how the different multi-criteria analyses perform in a real case from the same population of interviews where the results are very different from each other. ijahp article: gonzález, pradenas/multi-criteria analysis to improve the service in gas stations international journal of the analytic hierarchy process 85 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.601 references albert, g., musicant, o., oppenheim, i. & lotan, t. (2016). which smartphone’s apps may contribute to road safety? an ahp model to evaluate experts’ opinions. transport policy, 50, 54-62. doi: https://doi.org/10.1016/j.tranpol.2016.06.004 bartusková, t. & kresta, a. (2015). application of ahp method in external strategic analysis of the selected organization. procedia economics and finance, 30, 146-154. doi: https://doi.org/10.1016/s2212-5671(15)01278-2 bello, a. y cavero, s. (2008). estructura y estrategia competitiva en el mercado español de carburantes. economía industrial, 365, 97-112. bethel, s. (2015). valuing, buying & selling. gas station operations. united states: mattatall press. chang, d.y. (1996). applications of the extent analysis method on fuzzy ahp. european journal of operational research, 95, 649-655. doi: https://doi.org/10.1016/03772217(95)00300-2 chen, s.j. & hwang, c.i. (1992). fuzzy multiple attribute decision-making, methods and applications. lecture notes in economics and mathematical systems, 375. berlin heidelberg: springer-verlag. doi: https://doi.org/10.1007/978-3-642-46768-4_5 chima, c. (2007). supply-chain management issues in the oil and gas industry. journal of business & economics research, 5, 27-36. dulange, s., pundir, a. & ganapathy, l. (2014). prioritization of factors impacting on performance of power looms using ahp. journal of industrial engineering international, 10, 217-227. doi: https://doi.org/10.1007/s40092-014-0080-8 erdil, a. & erbiyik, h. (2015). selection strategy via analytic hierarchy process: an application for a small enterprise in milk sector. procedia social and behavioral sciences, 195, 2618-2628. doi: https://doi.org/10.1016/j.sbspro.2015.06.463 ervural, b.c., zaim, s. demirel, o., aydin, z. & delen, d. (2017). an anp and fuzzy topsis-based swot analysis for turkey's energy planning. renewable and sustainable energy reviews, 82(1), 1538-1550. doi: http://dx.doi.org/10.1016/j.rser.2017.06.095 gorëner, a., toker, k. & uluçay, k. (2012a). application of combined swot and ahp: a case study for a manufacturing firm. procedia social and behavioral sciences, 58, 1525-1534. doi: https://doi.org/10.1016/j.sbspro.2012.09.1139 gorëner, a. (2012b). comparing ahp and anp: an application of strategic decisions making in a manufacturing company. international journal of business and social science, 3(11), 194-208. hwang, c.l. & yoon, k. (1981). multiple attribute decision making: methods and applications. heidelberg: springer. doi: http://dx.doi.org/10.1007/978-3-642-48318-9 ijahp article: gonzález, pradenas/multi-criteria analysis to improve the service in gas stations international journal of the analytic hierarchy process 86 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.601 islam, a., sikder, s. & uddin, s. (2017). implementing swot-ftopsis methods for selection of the best strategy: pharmaceutical industry in bangladesh. journal of modern science and technology, 5(1), 110-124. kacprzak, d. (2017). objective weights based on ordered fuzzy numbers for fuzzy multiple criteria decision-making methods. entropy, 19(7), 373. doi:10.3390/e19070373 kim, a.r. (2016). a study on competitiveness analysis of ports in korea and china by entropy weight topsis. the asian journal of shipping and logistics, 32(4), 187-194. doi: https://doi.org/10.1016/j.ajsl.2016.12.001 kubler, s., robert, j., derigent, w., voisin, a. & le traon, y. (2016). a state-of the-art survey & testbed of fuzzy ahp (fahp) applications. expert systems with applications, 65, 398-422. doi: https://doi.org/10.1016/j.eswa.2016.08.064 mehmood, f., hassannezhad, m. & abbas, t. (2014). analytical investigation of mobile nfc adaption with swot-ahp approach—a case of italian telecom. procedia technology, 12, 535-541. doi: https://doi.org/10.1016/j.protcy.2013.12.526 nadaban, s., dzitac, s. & dzitac, i. (2016). fuzzy topsis: a general view. procedia computer science, 91, 823-831. doi: https://doi.org/10.1016/j.procs.2016.07.088 ramik, j. (2009). consistency of pair-wise comparison matrix with fuzzy elements. ifsa/eusflat conference, 98-101. reza, m. & ebrahim, m. (2011). development of a model for evaluating and ranking gas stations using analytic hierarchical process. australian journal of basic and applied sciences, 5(12), 1536-143. rouyendegh, b.d. & erkan, t.e. (2012). selection of academic staff using the fuzzy analytic hierarchy process (fahp): a pilot study. technical gazette, 19, 923-929. saaty, t.l. (1977). a scaling method for priorities in hierarchical structures. journal of mathematical psychology, 15, 234-281. doi: https://doi.org/10.1016/00222496(77)90033-5 saaty, t.l. (1996). decision making with dependence and feedback: the analytic network process. pittsburgh: rws publications. saaty, r.w. (1987). the analytic hierarchy process—what it is and how it is used. mathematical modelling, 9(3-5), 161-176. doi: https://doi.org/10.1016/02700255(87)90473-8 shahba, s., arjmandi, r., monovari, m. & ghodusi, j. (2017). application of multiatribute decision-making methods in swot analysis of mine waste management (case study: sirjan's golgohar iron mine, iran). resources policy, 51 (2017), 67-76. doi: https://doi.org/10.1016/j.resourpol.2016.11.002 ijahp article: gonzález, pradenas/multi-criteria analysis to improve the service in gas stations international journal of the analytic hierarchy process 87 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.601 tavana, m., zareinejad, m. & di caprio, d. (2016). an integrated intuitionistic fuzzy ahp and swot method for outsourcing reverse logistics. applied soft computing, 40, 544-557. doi: https://doi.org/10.1016/j.asoc.2015.12.005 wind, y. & saaty, t.l. (1980). marketing applications of the analytic hierarchy process. management science, 26(7), 641-658. doi: https://doi.org/10.1287/mnsc.26.7.641 yüksel, i. & dagdeviren, m. (2007). using the analytic network process (anp) in a swot analysis a case study for a textile firm. information sciences, 177, 3364-3382. doi: https://doi.org/10.1016/j.ins.2007.01.001 yussuf, r.d. & poh yee, k. (2001). a preliminary study on the potencial use of the analytical hierarchical process (ahp) to predict advanced manufacturing technology (amt) implementation. robotics and computer integrated manufacturing, 17, 421-427. doi: https://doi.org/10.1016/s0736-5845(01)00016-3 zadeh, i. (1965). fuzzy sets. information and control, 8, 338-353. zare, k., mehri-tekmeh, j. & karami, s. (2015). a swot framework for analyzing the electricity supply chain using an integrated ahp methodology combined with fuzzytopsis. international strategic management review, 3, 66-80. doi: https://doi.org/10.1016/j.ism.2015.07.001 zhao, j. & fang, z. (2016). research on campus bike path planning scheme evaluation based on topsis method: wei’shui campus bike path planning as an example. procesia engineering, 137, 858-866. doi: https://doi.org/10.1016/j.proeng.2016.01.326 živković, ž., nikolić, d., djordjević, p., mihajlović, i. & savić, m. (2015). analytical network process in the framework of swot analysis for strategic decision making (case study: technical faculty in bor, university of belgrade, serbia). acta polytechnica hungarica, 12-7, 199-216. doi: https://doi.org/10.12700/aph.12.7.2015.7.12 ijahp article: gonzález, pradenas/multi-criteria analysis to improve the service in gas stations international journal of the analytic hierarchy process 88 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.601 appendix table 6 average consistency ratio (cr) and standard deviation (ahp, anp) of each decision maker 𝐷𝐸𝑖 decision maker cr (ahp) 𝜎 cr (anp) 𝜎 de1 0.083 0.019 0.030 0.020 de2 0.082 0.022 0.060 0.041 de3 0.080 0.012 0.092 0.006 de4 0.077 0.019 0.055 0.028 de5 0.091 0.005 0.052 0.049 de6 0.080 0.029 0.048 0.044 de7 0.082 0.017 0.085 0.011 table 7 final results, arithmetic means and standard deviations according to each analysis alter. ahp fahp topsis ftopsis anp fanp �̅� 𝜎 so 0.281 0.281 0.275 0.302 0.280 0.284 0.284 0.009 wo 0.291 0.298 0.265 0.251 0.291 0.297 0.282 0.020 st 0.252 0.250 0.252 0.240 0.251 0.253 0.250 0.005 wt 0.176 0.171 0.208 0.208 0.178 0.166 0.184 0.019 ijahp article: gonzález, pradenas/multi-criteria analysis to improve the service in gas stations international journal of the analytic hierarchy process 89 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.601 table 8 global sub-criterion weights, means and standard deviations according to each analysis sub-criteria ahp fahp topsis ftopsis anp fanp �̅� 𝜎 s1 0.045 0.047 0.018 0.021 0.022 0.054 0.038 0.014 s2 0.063 0.072 0.040 0.048 0.047 0.083 0.060 0.016 s3 0.072 0.075 0.080 0.095 0.095 0.085 0.077 0.005 s4 0.031 0.032 0.023 0.028 0.028 0.037 0.029 0.005 s5 0.057 0.057 0.003 0.004 0.004 0.065 0.040 0.026 s6 0.089 0.085 0.020 0.023 0.023 0.097 0.066 0.033 w1 0.009 0.009 0.003 0.004 0.004 0.006 0.007 0.003 w2 0.019 0.022 0.025 0.030 0.030 0.016 0.021 0.004 w3 0.050 0.047 0.027 0.032 0.032 0.033 0.039 0.011 w4 0.045 0.044 0.107 0.074 0.076 0.031 0.064 0.032 w5 0.040 0.039 0.117 0.082 0.082 0.028 0.064 0.038 w6 0.024 0.024 0.110 0.051 0.051 0.017 0.052 0.042 o1 0.081 0.083 0.003 0.004 0.004 0.092 0.057 0.038 o2 0.070 0.070 0.054 0.065 0.064 0.078 0.066 0.009 o3 0.074 0.074 0.004 0.005 0.005 0.082 0.051 0.034 o4 0.060 0.055 0.031 0.037 0.037 0.062 0.049 0.013 o5 0.021 0.021 0.034 0.041 0.041 0.023 0.026 0.006 o6 0.040 0.035 0.007 0.008 0.008 0.039 0.028 0.015 t1 0.021 0.022 0.038 0.046 0.046 0.014 0.027 0.009 t2 0.012 0.013 0.049 0.058 0.057 0.008 0.024 0.017 t3 0.009 0.010 0.072 0.086 0.085 0.007 0.030 0.029 t4 0.014 0.014 0.036 0.043 0.043 0.009 0.021 0.011 t5 0.034 0.031 0.076 0.090 0.091 0.021 0.047 0.022 t6 0.020 0.018 0.022 0.026 0.026 0.012 0.020 0.004 ijahp article: gonzález, pradenas/multi-criteria analysis to improve the service in gas stations international journal of the analytic hierarchy process 90 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.601 table 9 sequence of alternatives in each set according to each sub-criteria and global orden subcriteria global s1 s2 s3 s4 s5 s6 set 1 2 1 2 1 2 1 2 1 2 1 2 1 2 so 2 1 1 3 3 3 3 3 4 4 3 1 1 1 wo 1 2 3 2 2 2 1 1 1 1 1 3 2 2 st 3 3 2 1 1 1 2 2 2 2 2 1 3 4 wt 4 4 4 4 4 4 4 4 3 3 4 4 4 3 subcriteria global w1 w2 w3 w4 w5 w6 set 1 2 1 2 1 2 1 2 1 2 1 2 1 2 so 2 1 1 3 2 3 2 4 3 1 4 1 1 3 wo 1 2 4 1 4 1 4 1 1 4 1 3 3 2 st 3 3 2 4 3 2 3 2 2 3 2 4 4 1 wt 4 4 3 2 1 4 1 3 4 2 3 2 2 4 subcriteria global o1 o2 o3 o4 o5 o6 set 1 2 1 2 1 2 1 2 1 2 1 2 1 2 so 2 1 4 4 2 2 1 1 1 1 2 1 1 1 wo 1 2 2 2 1 1 3 4 3 2 4 4 2 3 st 3 3 1 1 3 3 2 2 2 3 1 2 4 4 wt 4 4 3 3 4 4 4 3 4 4 3 3 3 2 subcriteria global t1 t2 t3 t4 t5 t6 set 1 2 1 2 1 2 1 2 1 2 1 2 1 2 so 2 1 3 1 1 3 1 4 1 3 3 1 1 2 wo 1 2 1 4 3 4 3 3 4 1 2 3 2 4 st 3 3 4 2 4 1 4 1 3 2 1 4 4 1 wt 4 4 2 3 2 2 2 2 2 4 4 2 3 3 table 10_ percentage difference of the averaged weights of sub-criteria of each swot category and comparison between different analyses sub-criteria ahp anp % fahp fanp % ahp/fahp anp/fanp topsis/ ftopsis % s 0.059 0.057 -4.3% 0.061 0.070 14.4% 0.060 0.037 -38.3% w 0.031 0.032 4.2% 0.031 0.022 -28.5% 0.031 0.046 48.4% o 0.058 0.055 -4.6% 0.056 0.063 11.0% 0.057 0.026 -54.4% t 0.018 0.022 21.2% 0.018 0.012 -34.5% 0.018 0.058 222.2% comparability, decision theory and the ahp ijahp article: von solms/comparability, decision theory and the ahp international journal of the analytic hierarchy process 513 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.313 comparability, decision theory and the ahp sibs von solms department of business management university of zululand richards bay, south africa email: sibsvonsolms@gmail.com abstract saaty (2011) briefly discusses the three basic laws of aristotelian logic and suggests a fourth, which he calls the law of comparisons. he argues that comparison is both relevant and essential to the other three laws and, in fact, precedes them. this view comparativism is however, not without criticism. here we present a more comprehensive discussion of various problems regarding comparability, focusing on three aspects; (i) the problem of a proper scale; (ii) the problem of a proper aggregation of conflicting criteria and (iii) the debate whether values are subjective or objective. the debate regarding incomparability is varied and intense making a perfunctory or uncritical acceptance of comparativism wrong. however, saatian comparativism will be shown to be a solution to the major issues raised by incomparativists. two conclusions are reached; (i) saaty’s (2011) view is confirmed and (ii) the work of saaty is not reflected in the incomparability or incommensurability literature and this debate stands to be enriched by seriously considering saatian comparativism. keywords: analytic hierarchy process; covering value; incomparability; incommensurability; value realism 1. introduction in a recent article, saaty (2011) briefly discusses the three basic laws of aristotelian logic and suggests a fourth, which he calls the law of comparisons. he argues that comparison is both relevant and essential to the other three laws and, in fact, precedes them. saaty (2011) states that rational thinking involves the basic assumption that things are implicitly comparable. the view that things are implicitly and generally comparable is called comparativism, and a large amount of literature exists on the subject. authors argue both for and against comparativism from a philosophical as well as a decision theoretical perspective. what is, however, missing from this debate is references to and arguments about ahp/anp, while on the other hand there are also few references to comparability in the ahp/anp literature. these two bodies of literature largely exist separately, without overlap or inter-literature critique. although not explicit, saaty (2011) seems to present the ahp/anp as a decisionmaking methodology that properly applies these four laws and, hence, comparativism. here we will come to the same conclusion after a more comprehensive discussion of various problems with comparability highlighted in the comparability literature. the discussion will highlight general critiques of comparativism found in the comparability literature and point out how saatian comparativism answers these issues, in an attempt to stimulate useful debate between these two bodies of literature. mailto:sibsvonsolms@gmail.com ijahp article: von solms/comparability, decision theory and the ahp international journal of the analytic hierarchy process 514 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.313 2. importance of the incomparability argument saaty’s (2011) discussion purely states the importance of comparisons, but does not allude to the debate regarding the possibility of not always being able to compare items which is the view in opposition to comparativism (chang, 1997). the first question is whether a detailed discussion of this comparability/incomparability debate is of any real importance. the consensus amongst pluralist philosophers is that pluralism entails value incomparability (kekes, 1992; mason, 2011). several authors point out that it is important to settle the argument regarding the comparability or incomparability of options because of the critical role comparisons play in practical decision-making, particularly in rational choice theory (pildes & anderson, 1990; broome, 1997, 2000; eriksson, 2003; gert, 2004; hsieh, 2005b; okapal, 2007, 2010; kelly, 2008; da silva, 2011 inter alia). it is difficult to see how one can choose rationally if one cannot compare and say which of the available options are best, i.e. optimal. the importance of comparability is not only explicit in rational choice and decision theory, but is also implicit in most forms of consequentialism (chang, 1998, 2013). rauschmayer (2001) points out that due to the problem of incomparability – which he argues is much more prevalent than often thought – anybody involved in decision aiding has both a scientific as well as a social responsibility to make all assumptions, including the possibility of incomparability, clear when designing or using decision-making tools. the importance of clarifying the incomparability issue is also clear when aldred (2006) points out that comparability is so entrenched in economics and decision theory that it would shock (sic) economists and decision theorists to learn how widely disputed the comparability view is amongst philosophers. 3. introducing the incomparability argument the question is whether it is always possible to compare any two items a and b or whether there are cases when a and b are incomparable. chang (2002a) points out that philosophers typically have one of three reactions to a decision situation in which it is difficult to decide on comparisons, for example: who is more creative mozart or michelangelo or which career is best, accounting or skydiving? first, epistemicists insist that, although it may be difficult or even impossible to determine how the items compare, all things considered, one must be better than the other or they must be equally good. second, incomparabilists, however, insist that even omniscience will not yield a true comparison in terms of all the relevant considerations; hard cases are difficult precisely because there is no comparison of them – neither is better than the other nor are they equally good. third, the semantic indeterminists argue that it is indeterminate how the items compare all things considered – it is indeterminate whether one is better than the other or whether they are equally good. on this view the indeterminacy arises because the terms, “better than” or “equal to” are vague and hard cases are borderline applications of one of these comparatives. wasserman (2004) is one who relates indeterminism and vagueness. chang (1997) sums it up by pointing out that the epistemicists think that it is true that one item is always better than the other or they are equally good; incomparabilists think it is false that that one item is always better than the other or they are equally good; semantic indeterminists think it is neither true nor false that one item is always better than the other or they are equally good. incidentally, a second distinction regards in-comparability and non-comparability of which harris (2001) says that non-comparability relates to situations where the two items being compared do not share a covering value, e.g. comparing an apple and a concept as to taste – because concepts do not possess the value taste they cannot be ijahp article: von solms/comparability, decision theory and the ahp international journal of the analytic hierarchy process 515 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.313 compared to apples in this regard. incomparability, on the other hand, relates to difficult comparisons where both items exhibit the covering value, e.g. comparing mozart to michelangelo as to creativity. non-comparability will not be dealt with here. traditional decision theory bases preference modelling on two fundamental preference relations, i.e. preference and indifference. if two items a and b are compared three possible relationships may be obtained; either a is preferred to b (apb or a>b); or b is preferred to a (bpa or b>a); or neither is preferred – a situation of indifference expressed as aib or a~b. these relationships are often modelled as three fundamental relations, i.e. “better than” (equivalent to apb); “worse than” (bpa) and “equal to” (a~b). the thesis, that if two items are evaluatively comparable, then a must be better or worse than b; or a and b must be equally good, is called the trichotomy thesis (chang, 2002a). according to this thesis the conceptual space of comparability between two items is spanned by the trichotomy of relations “better than”, “worse than” and “equal to”. part of the appeal of this thesis is in a tidy assimilation of evaluative (e.g. kindness, beauty) comparisons to non-evaluative (e.g. mass, length) comparisons that proceed in terms of “more”, “less” or “equal” amounts of some attribute. the trichotomy thesis (tt) is almost universally accepted and is assumed by all the major writers on the topic of measurement, decision and game theory, but it is argued that in many cases none of these relations hold between two items a and b, and that, in such cases, the items should be deemed incomparable (klocksiem, 2010; harris, 2001; aldred, 2002). chang (1997, 2002a) notices problems with the tt in its simple form but still rejects the possibility of incomparability. she believes that two items can be evaluatively compared without any of the trichotomy relations necessarily holding between them, and suggests that there is conceptual space in the intuitive notion of evaluative comparability for a fourth relation to hold between two items, i.e. parity. providing a definition of parity or rough equality these terms are often taken as similar is not straightforward, given the persistent controversy over the notion (griffin, 1997; wasserman, 2004; aldred, 2006). chang (1997, 2002a) maintains that parity is a positive value relation, that is, another relation that requires, or defines, comparability, in addition to “better”, “worse” and “equal”. she terms parity the relation of options being ‘on a par’, and points to examples such as where an agent must choose between a career as a lawyer (l) and one as a clarinettist (c). it may be that the two are comparable, but neither is better than the other, and yet a small improvement in one, a slight improved legal career (l + ), does not make it better that the other (chang, 1997). in this formulation she does not specify which relation holds between l + and c; but she simply denies one, thus it appears that l + bl, lrec but not that l + bc. wasserman (2004) says that chang is wrong to reject the tt and argues that by acknowledging the vagueness inherent in some comparisons the need for relations like rough equality or parity disappears. in qizilbash’s (2000, 2005) view, vagueness of predicates is not only widespread but also leads inevitably to situations of difficult comparability, but accepts that parity is a comparative relation and, hence, incomparability does not follow if none of the standard trichotomy relations hold. gert (2004) argues that it is not necessary to define the fourth relation of parity if the terms “better than” or “worse than” are redefined in terms of majority consensus. in his opinion, choosing a over b (ie a>b) would be justified if the majority would not be surprised by that choice. if no majority exists, i.e. neither choosing a over b nor choosing b over a would be deemed irrational by a majority of observers, the items a and b should be seen as “equal”. the notion that community consensus represents truth is a general and long standing view in philosophy and is discussed elsewhere (von solms, 2011). harris (2001), on the other hand, argues against chang’s (1997, 2002a) views, but not by trying to ijahp article: von solms/comparability, decision theory and the ahp international journal of the analytic hierarchy process 516 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.313 show how the tt can be defended but rather that restricting relations to comparable ones (be that three or four) is wrong and that the existence of incomparability must be accepted. aldred (2002) mentions five mutually exclusive value relations between two items a and b: abb bba aeb areb aicb where b implies “better than”, e implies “equal to”, re implies “roughly equal to” and ic implies “incomparability”. the first four are equivalent to the relationships of traditional decision theory and constitute the area of comparability. aldred (2002) uses ic as strict incomparability. consequently, the possibility of incomparability, in contrast to saaty’s (2011) general comparability view, warrants closer evaluation. this evaluation will proceed via three routes: (i) the lack of a common measurement scale; (ii) the lack of a proper aggregation method when multiple criteria are involved and (iii) the debate whether values and scales are objectively given or subjectively derived. 4. comparability and commensurability: the problem of a scale the dictionary definitions of comparable and commensurable are confusing, but the preponderance of evidence indicates that comparable refers to the examination of items to determine how they may be similar, different, better or worse; while commensurable implies measurable by the same standard or scale of values in terms of size, degree, importance or quality. however, roget’s thesaurus, a widely-used english language thesaurus, gives comparable and commensurable as alternatives (synonyms) of each other. a wide ranging debate exists in the literature in which these two terms are defined as similar, related or different, making any discussion on the topic confusing. aldred (2002) mentions incomparability along with incommensurability and indicates that these two terms are often confused and while he argues for keeping incomparability and incommensurability distinct, qizilbash (2000, 2005), on the other hand, equates these two terms. this begs the question as to how the relationship between incommensurability and incomparability should be understood. on one hand the two terms are conflated. raz (1986, 1997), for example, defines incommensurability in terms of incomparability when he says that a and b are incommensurate if it is neither true that one is better than the other nor true that they are of equal value. warner (1998) represents a view similar to raz arguing that reasons are incommensurable when (and only when) they are not comparable as better, worse or equally good. heuer (2004), too, points out that the razian account of incommensurability relies on the impossibility of comparison and that incommensurability itself is established by failure of comparison, by the fact that neither of two options is equal in value to, or better than the other. broome (2000) is another author who sees incommensurability as existing when none of the trichotomy relations are obtained between two items or values. grimm (2003) states that conflating incomparability and incommensurability is widespread and points to griffin (1997) who states that what nearly everyone, on reflection, means by the incommensurability of values is their incomparability – that there are values that cannot be acquired on any scale, that they cannot even be compared as to greater, less or equal. on the other hand, some authors find the conflation of incomparability and incommensurability mistaken. grimm (2003), for instance, does not believe that incommensurability necessarily implies incomparability. chang (1997) states that things are incommensurable, when they cannot be precisely measured along some ijahp article: von solms/comparability, decision theory and the ahp international journal of the analytic hierarchy process 517 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.313 common cardinal scale of units of value, and incomparable when no positive value relation holds between them – they cannot even be ranked on an ordinal scale. she says that comparability is a necessary but not sufficient condition for commensurability. pildes & anderson (1990) agree with this changian definition of incommensurability, while da silva (2011) states that incommensurability is not equal to incomparability and that it is specifically the latter that is very important in legal decision-making. sunstein (1997) makes a very clear distinction between the terms incomparability and incommensurability in opposition to the views of raz (1986, 1997). aldred (2006), too, defines incommensurability and incomparability in changian terms, with the former as measurement and the latter as ranking. hsieh (2007) uses these terms in an interestingly different way. for him, incomparability ordinarily describes two or more concrete bearers of value of which no positive comparative evaluative judgment is true, while he uses the term incommensurability to describe the way in which two or more abstract values stand in relation to one another. considering in greater detail the relationship between the incomparability of bearers of value and the incommensurability of abstract values, will be done later (in section 5) because it is relevant to multi-criteria problems. it is often argued that commensurability is taken to imply measurement along a common cardinal scale often monetary (chang, 1997; aldred, 2002, 2006; gowdy & erickson, 2005). sunstein (1997) makes it clear that commensurability requires cardinal metrics and that ordinal rankings – which he calls comparability – are not cases of commensurability, but nevertheless important in decision-making. rauschmayer (2001) defines incomparability in terms of the relations of preference and indifference but quotes the definition of sunstein (1997) for incommensurability. his view is, however, not exactly the same as that of sunstein’s as rauschmayer (2001) allows commensurability to be based on both ordinal and cardinal scalable values, while sunstein (1997) requires cardinal metrics exclusively. chang (2013) also argues that commensurability requires a cardinal (interval or ratio) scale, while comparability requires only an ordinal scale. similarly, luban (2001) defines two types of commensurabilities; r-commensurability and o-commensurability. he says that formally items are commensurable with respect to a certain property (distance, fame, etc) when that property induces a function – more precisely, an isomorphism – placing the items in one-to-one correspondence with elements in some ordered set. this ordered set provides the common measure of the items. the ordered set, in turn, can possess whatever mathematical structure one wishes to impose on it. for purposes of economic analysis it is common to assume the set to possess all the properties of the real numbers and, along with the usual axioms, the function assigning a real number to each item of interest is regarded as a utility function. luban (2001) defines r-commensurability as: items are r-commensurable if and only if they can be assigned real-valued utilities. if no such utility function exists, the items are r-incommensurable. r-commensurability is what is usually meant by commensurability and correspondingly, the incommensurability-of-values thesis is typically taken as the denial that disparate goods are r-commensurable (luban, 2001). but he points out that it is important to realize that items may be commensurable in a weaker sense, if their common measure (musical complexity, creative talent, etc.) does not possess the rich mathematical structure of the real numbers. such goods may be ordinally commensurable, for example, if they are isomorphic to some set whose structure consists of an orderor partial-order relation. this can be called o-commensurability leading to an ordinal ranking of the items. aldred (2002, 2006) clearly distinguishes commensurability as measurement by a cardinal scale from comparability as comparisons on an ordinal scale. mather (2002) distinguishes between cardinal measurement and ordinal ranking but labels both as ijahp article: von solms/comparability, decision theory and the ahp international journal of the analytic hierarchy process 518 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.313 forms of commensurability – metrical and ordinal commensurability, respectively. eriksson (2003) states that, to practical reason, (cardinal) incommensurability is bad enough, but worse is (ordinal) incomparability, since this possesses a clear threat to reason-based decision-making. griffin (1997) captures this idea when he says: “(w)hen many of us insist, for instance, that complex decisions about the environment cannot be reduced to cost-benefit analysis because some of the clashing values are incommensurable, we do not just mean that those values cannot be got on to additive cardinal scales, but they cannot be got even on to the ordinal scales that economists are by and large content to work with. … the serious threat to practical reason comes not from, say, a mere breakdown in addition or from the appearance of a lexicographical ordering, but from a breakdown in ranking. that threat is the most important one to confront.” in summary, the relationship between incommensurability and incomparability is complex with the literature either conflating or differentiating the terms. the consensus that is developing however seems to be that commensurability is cardinal measurement while comparability is ordinal ranking of alternatives. if we accept these as working definitions, we are faced with the question of whether rational decision-making can be based on either. from the arguments above it seems as if the answer may be yes; preferred alternatives can be identified either through some form of cardinal measurement or through ordinal ranking and rational decision-making is only under threat if neither process is possible. this solution however, is only superficially valid because at least three objections threaten its validity; (i) a measurement theory objection that cardinal measurement is only available for selected variables and, hence, has a very limited application in decision-making; (ii) the social choice objection that ordinal aggregation – either across multiple criteria or multiple decision-makers – is subject to arrowian impossibility and (iii) the philosophical argument that pluralism implies that no scales – be they cardinal or ordinal – exist to commensurate or compare diverse options. we return to these points in sections 5, 6 and 7 respectively. 5. measurement theory: the problem of quantification the first threat, the measurement theory objection regarding cardinal measurement, can be dealt with briefly. the view that only quantitatively measurable data is of real scientific value is promoted within the traditional measurement theory (tmt) in opposition to the representational measurement theory (rmt) (barrett, 2003; michell, 1997, 2003, 2008; acton, 2003). von solms (2013) discusses this debate and reaches the conclusion that not only must the general critique of the tmt aimed at rmt be rejected but, more importantly, the insistence of the tmt proponents that only tangibles (quantitative variables) can be measured must be rejected because it excludes immediately the possibility of any real world decision-making in which intangibles always play an important role (forman & selly, 2001; saaty, 2001, 2010, 2013; jackson, 2003; saaty & sagir, 2009). 6. all-things-considered judgments: the problem of aggregation the social choice objection requires a more comprehensive discussion. over and above the problem of a scale there is a second problem in the incomparability debate relating to the complexity of choice situations which manifests in two aspects (ellis, 2008). first, difficult situations are characterized by comparisons where the evaluations of options need to be based on multiple aspects of evaluation. second, a related insight is that evaluations, on such multiple criteria, require some form of aggregation to achieve a single overarching relation amongst alternatives. although ijahp article: von solms/comparability, decision theory and the ahp international journal of the analytic hierarchy process 519 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.313 both these themes are found in the comparability literature, solutions are either not discussed or those offered are not satisfactory. some philosophers are skeptical of the integration (i.e. to aggregate multiple criteria evaluations into an overarching result) and the best developed form of this argument is that some values are incommensurable, they resist integration and so they cannot be brought together to form a single assessment (ellis, 2008, 2012). ellis (2008) acknowledges that in principle incomparability may have nothing to do with composite (multi-criteria) values, but that in practice the existence of multiple irreducible ends is at the heart of the incomparability issue. rauschmayer (2001), too, links the problem of incomparability to that of mcdm when he says that generally speaking, the value system of the deciding individual will contain different values (criteria, points of view), which cannot be reduced to a single measure and, hence, is one of the reasons for the incomparability of decision options. several authors argue that, particularly in environmental and sustainability decisions, the evaluation procedure cannot rely on a single numeraire (commensurability) but must be based on weak comparability operationalized through multi-criteria evaluations because the world is characterized by deep complexity and that neither the problem situation description nor the viewpoints of decision-makers can be expressed using a single perspective (o’neill, 1997; martinez-alier et al, 1998; geldermann et al, 2000; munda, 2004). o’neill (1997) provides a clear definition of weak comparability by saying that it is based on the idea that the same value can, at the same time, be a good x and a bad y, i.e. different descriptions of the issue may lead to different outcomes because a unique single comparative term is not available for the comparison. his example of weak comparability is that the same area can be described both as a pretty scene and as a bad habitat because the invader plants growing there may make the area pretty to look at but not a suitable habitat because of the replacement of natural indigenous vegetation. any comparison must proceed with respect to an evaluative covering consideration. x cannot be better than y, tout court, but can be better than y only with respect to, say, well-being, beauty or morality (martinez-alier et al, 1998; chang, 2004a; hsieh, 2005a). just as it makes no sense to say that one stick is greater than another, tout court, it makes no sense to say that one item is better than another, tout court. one stick can be greater than another with respect to length but not mass, and one item be can better than another with respect to beauty but not morality. comparability is a three-place relation: x is comparable with y with respect to c, where c is the covering consideration. this covering consideration can be simple (uni-dimensional) or complex (multi-dimensional). in the complex case x is better than y, all things considered, if there is some set of values (c1,c2,...,cn), which together comprises c, that are the things to be considered (chang, 2013). the complex case includes the view that values can form a hierarchical structure. for example, grimm (2003, 2007) defines two different values, general and contributory – general values represent overarching values with several contributory values which together constitute each general value. he presents two examples. first, in selecting a basketball team the general value could be ‘excellence at basketball’ with contributory values ‘defence’, ‘shooting’ and ‘passing’. second, for awarding a prize in painting, the general value could be ‘artistic ability’ with its contributing values ‘mastery of perspective’, ‘use of colour’ and ‘expressiveness’. boot (2007) distinguishes between covering values and contributing values as follows: covering value is a value with respect to which options are compared while a contributory value is a value contributing to a covering value (e.g. ‘analytical skill’ or ‘originality’ contributing to the covering value ‘philosophical talent’). ijahp article: von solms/comparability, decision theory and the ahp international journal of the analytic hierarchy process 520 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.313 covering values often contain more than one contributory value (harris, 2001). in such complex (multi-criteria) cases, the fact that one alternative (say a) can be better than another (say b) in terms of some and worse in terms of other criteria (contributing values), is used to identify the alternatives a and b as incomparable (munda et al, 1994; chang, 1997; geldermann et al, 2000; rabinowicz, 2008). rauschmayer (2001) calls this the problem of bi-directionality and points out that it is well known in the literature of the mcda (brans & mareschal, 2005), while ellis (2008) refers to such cases as conatively mixed. boot (2007) argues that not all cases of multi-criteria comparisons are problematic, and indicates that if (i) the covering value is one-dimensional or (ii) the covering value is multi-dimensional provided there is no bi-directionality of contributory values, multi-dimensionality does not pose a problem. seung & bonevac (1992) state that when two items are compared using a single value-measure (e.g. comparing apples and oranges according to sweetness) the resultant ranking is an algorithmic ranking. if, on the other hand, two items need to be compared using multiple criteria (e.g. comparing apples and oranges on sweetness, juiciness and nutritional content) an algorithmic ranking can only be obtained in the case that one alternative is ranked above the other on all criteria. if, however, apples are ranked above oranges on sweetness; oranges above apples on juiciness and apples and oranges are rank similar on nutrition no algorithmic ranking can be obtained because no super-measure exists that can be used to aggregate the disparate rankings and the only alternative in such cases is indeterminate ranking, which is based on intuition (seung & bonevac, 1992). the question is now, how can the problem of bi-directionality be resolved or avoided? the first approach to address the aggregation problem is philosophical in nature. a necessary feature of comparativist theories is to describe how all the factors are put together because this unity is demanded by the notion of an all-things-considered (atc) judgment. for atc judgments to be possible they have to have a unity in virtue of which their components (the various factors) have the normative relations they do (okapal, 2007, 2010). chang (2004a) clearly feels the need for some form of aggregation to move from individual covering values to an atc judgment and suggests the more comprehensive value (mcv) as the integrative factor. she, however, does not discuss or present any practical ways in which the atc can be obtained (okapal, 2007, 2010). on the contrary, she identifies the unifying attribute of mcvs as a profound mystery. chang (2004a) explains the nature of comprehensive factors using a jigsaw puzzle metaphor according to which the correct solution relies on the picture that is created once all the pieces are in place. her mcv view is like a jigsaw puzzle where the picture is what tells how the pieces go together and comprehensive values would be, metaphorically, this picture (chang 2004b). it is insightful to listen to chang’s (2004a) own presentation: “the ‘picture’ that puts values together is the unity of a more comprehensive value. although it is hard to explain just what this ‘picture’ is, it is important to emphasize that the mystery of what makes values hang together is not peculiar to the more comprehensive values approach. any normative theory that recognizes values is saddled with the problem of explaining their unity…. we have no account of what it is about such values in virtue of which their components (sic) values hang together in the way that they do.” mcv judgments are multi-criteria judgments in which several values together make up the final judgment, which is not found via an aggregation across criteria but via the intrinsic normative structure of the covering value. chang (2004a) argues that there seems to be no practical way to establish the unifying principle except that she ijahp article: von solms/comparability, decision theory and the ahp international journal of the analytic hierarchy process 521 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.313 explicitly argues against it being in subjective criteria weighting 1 . okapal (2007, 2010) agrees that the changian approach lacks clarity in the details and presents – what he calls – a sophisticated orthodox approach. but this approach also provides little practical guidance as to how aggregation is achieved via normative-level criteria, interaction principles and judgment. mason (2011) argues that there is a problem in understanding quite what a synthesizing category or covering value is, and how the covering value determines the relative weightings of the constituent values. one possibility is that it does it by pure stipulation as a martini just is a certain proportion of gin and vermouth. however, stipulation does not have the right sort of explanatory power (mason, 2011). tiberius (2008), too, criticizes the morecomprehensive-value approach by indicating that it is unclear what kind of explanation the mcv can offer for the respective weights of different conflicting values. although such mysterious attributes may be philosophically interesting, this is an extremely unsatisfactory notion for practical real world decision-making and relies implicitly on the assumption that covering values are objectively given and not subjectively derived by the decision-makers, a theme we return to in section 7. the second approach lies in the attempt at aggregating ordinal rankings. as mentioned above, the suggestion was that the threat of (cardinal) incommensurability could be avoided by using ordinal ranking to achieve comparability. this is, however, problematic. first, ordinal comparability also runs afoul of bi-directionality as demonstrated by examples in roy & vincke (1984) and seung & bonevac (1992). one approach to decision aiding, trying to accommodate incomparability rather than avoid it, is outranking (bisdorff, 2004). the american maut tradition is based on the principle of general comparability, thus representing an epistemicist tradition while the outranking methods of the french school of mcdm accept that not all items are necessarily comparable, hence, representing a more incomparabilist tradition, within which incomparability is accepted as inevitable and honoured as representing decision situations more realistically (simpson, 1996; coello coello, 2000; brans & mareschal, 2005; geldermann et al, 2000; vincke, 2000; bouyssou, 2001). within the outranking methods two candidates a and b are deemed incomparable in cases where the problems are multi-criteria comparisons and contradictory (bi-directional) evidence of which candidate is preferred emanates from different criteria (roy & vincke, 1984). the significance of the notion of incomparability lies – as in the seung & bonevac (1992) case not in the multi-criteria aspects of the problem, but rather in the aggregation aspect, i.e. in trying to find an overall preference from the conflicting information. the infrequent existence of a dominance relationship and the resulting high incidence of incomparability necessitate the search for additional information to help reduce the number of incomparabilities. different outranking methods use different sets of additional information to achieve this (brans & mareschal, 2005). aggregation of ordinal rankings can be attempted either using a dominance relationship as in the outranking methods, or a different approach is to attempt to integrate diverse options based on ordinal rankings and some associated decision rules (greco et al, 2005; dembczyński et al, 2007). second, ordinal aggregation has serious problems of its own. in a multi-criteria environment, rauschmayer (2001) argues, incommensurability can occur at two levels; first, when comparing options in terms of criteria – called criterial 1 here the focus is on the lack of clarity as to how the covering value facilitates aggregation, the objection to subjective criteria weighing will be discussed below in section 7. ijahp article: von solms/comparability, decision theory and the ahp international journal of the analytic hierarchy process 522 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.313 incommensurability and second, aggregate incommensurability when attempting to find the relative importance of the criteria. if incommensurability is assumed to be a serious issue in rational decision-making, and the decision maker can only make use of ordinal data at both these levels, decision between options will not be possible. from this rauschmayer (2001) emphasizes that the way in which mcdm aggregation is done critically affects the success of decision outcomes and concludes that (i) both criterial as well as aggregate incommensurability leads to incomparability; (ii) ordinal aggregate commensurability leads at best to limited comparability and (iii) only cardinal criterial and aggregate commensurability guarantee general comparability. because the weighting of criteria before comparing options on the different criteria is necessary, but an ordinal weighting process is problematic ordinal weighting makes no sense other that providing a ranking of the criteria (vargas, 1994). ordinal rankings are informationally poor and cannot provide an adequate basis for a proper comparison process (sen, 1995). these problems emanate from social choice theory and include arrow’s impossibility theorem and the condorcet and borda voting paradoxes (list, 2013; arrow, 1951). both outranking methods and methods using ordinal ranking plus decision rules are linked to social choice theory (greco et al, 2005; munda, 2012). these problems are exacerbated in the group decision-making (gdm) environment. nooteboom (1984) shows that intransitivity flows from preference orderings when multiple (conflicting) criteria are involved and points out that this is similar to arrowian impossibility in social choice theory. the latter deals with diverging inter-personal preference orderings while the former relates to intra-personal preference conflicts. may (1954) suggests an intra-personal analogue to arrow's impossibility theorem showing that an individual's response to a plurality of values will, given certain additional assumptions, lead to intransitive preference orderings. hurley (1985) challenges may's (1954) assumptions – specifically, universal domain and independence of irrelevant alternatives as implausibly strong in the intra-personal case; but her work does not exclude the possibility that values may disobey the canon of rationality that insists on transitivity (edmundson, 2009). pildes & anderson (1990) identify radical incommensurability as resulting when individual or collective value judgments fail to converge on a confident, complete ranking of the options at stake. this indicates that incommensurability is not only a phenomenon – as it commonly appears in the incomparabilist literature – of an individual that cannot compare items or options, but clearly also a problem due to incomparability being caused by multiple judges not agreeing. so, even if individuals could compare options but as a group cannot come to a substantive consensus – either on which values to apply or how options rate on these values (criteria) – this could be seen as instances of incomparability. kornhauser (1998) states that arguments concerning incommensurability have a formal structure that parallels the structure of arguments concerning the appropriate aggregation of interests of different individuals and, incidentally, calls the inability of an individual to come to a consolidated all-thingsconsidered judgment in a multi-criteria situation radical incommensurability. the structure of the problem of incommensurability thus parallels the problem of collective choice. bouyssou et al. (2000) agree saying that aggregating the opinion or the preferences of voters or individuals of a community into collective or social preferences is a problem quite similar to devising comprehensive preferences of a decision-maker from a set of conflicting criteria in mcda. the discord amongst the views of different group members is analogous to the discord amongst the perceptions of a single decision maker and similarly leads to indecision – only in gdm both intraand well as inter-member indecision could be present simultaneously. if ordinal ranking is applied to multi-criteria problems the arrowiantype inconsistencies are magnified (arrow & raynaud, 1986). the complication in ijahp article: von solms/comparability, decision theory and the ahp international journal of the analytic hierarchy process 523 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.313 group decision-making is that both the aggregation of group members’ individual ordinal rankings and the aggregation of different criteria rankings (if done ordinally) are subject to arrowian inconsistencies (hurley, 1985, 1989; coello coello, 2000; boot, 2009). as long as ordinal scales are used in multi-criteria or gdm situations the demon of incomparability will remain. an attempt to avoid the problem of ordinal ranking is to use a lexicographic approach. assigning lexical priorities means that one value gets absolute weight in relation to another and always receives priority in case of value conflicts (boot, 2007). the lexical priority approach avoids the problem of determining equivalence relations between values because a lexically prior value, however small its amount, always gets priority to another value, however large its amount. boot (2007) argues that a scheme of lexical priorities seems more capable of resolving value conflicts than the relative weight approach because the former does not require a (subjective) equivalence relation which is problematic in cases of heterogeneous values. however, the lexical priority approach has its limitations because, as boot (2007) demonstrates, there are few, if any, values that earn an unconditional lexical priority to other values and we have to fall back on balancing the relevant values by assigning relative weights in order to be capable of rationally resolving value conflicts. chapman (1998) states that a lexical priorities approach honours incommensurability better because it does not try to commensurate criteria that do not have a common measure. he suggests a method where the criteria are used singly and in a particular order, called the conceptually sequenced argument. this method, however, results in different selections depending on the order in which the criteria are used, indicating that although no relative weight is explicitly allotted to criteria, the order acts as if the first criterion is allocated more weight than the criteria used subsequently. chang (2001) points out that the view of raz (1986) on constitutive incommensurability and that of anderson (1997) on hierarchical incommensurability are similar to each other. both views are similar to boot’s (2007) lexical incommensurability, arguing that certain values are, by nature, of a higher status than others and that this makes any attempt at comparison between higher and lower values wrong. the paradigm example involves friendship and money. comparing friends to monetary gain bars the comparer from being a (true) friend because it is constitutive of friendship that friends are to be considered more important than mere monetary gain (raz, 1986; anderson, 1997). chang (2001) says that the arguments of many authors (including raz and anderson) in favour of constitutive incommensurability can be answered with an account of emphatic comparability in place. she concludes that if a certain token of one value is seen as lexically (or constitutively) higher than a token of another value, instead of regarding this as a case of incommensurability, it could more accurately be seen as a case of comparability with the higher value regarded as emphatically better than the lower value (chang, 2001, 2013). in summary, the impracticality of the mcv approach ; the limited applicability of the lexical priority approach and the arrowian impossibility problem in ordinal aggregation approaches brings us back full circle to the use of cardinal scales and the incommensurability issue (tiberius, 2008; okapal, 2007, 2010; mason, 2011; chang, 2001, 2013; boot, 2007; hurley, 1985, 1989; vargas, 1994; coello coello, 2000; forman & selly, 2001; saaty, 2001, 2010). ijahp article: von solms/comparability, decision theory and the ahp international journal of the analytic hierarchy process 524 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.313 7. incomparability, objective or subjective values and scales? we have come to an impasse. on the one hand, using cardinal scales to commensurate objects or actions was found to run afoul of the incommensurability thesis; while on the other hand, ordinal comparisons were shown to falter on aggregation into all-things-considered analysis due to arrowian impossibility. this leaves rational decision-making in a quandary because without some form of rating or ranking the best alternatives cannot be identified making rational choice impossible. one aspect of the incommensurability debate that is relevant here but not normally explicitly discussed is whether values are to be seen as objectively given or subjectively found. the answer significantly impacts if and how comparisons are performed and comparison scales developed. in our earlier discussions, questions of what incommensurability or incomparability involves were discussed along with questions as to how these issues affect choice, but exactly why incommensurability and incomparability exist was not evaluated. in the debate regarding objectivism or subjectivism in value, the possibility of identifying a source for the problems of incommensurability and incomparability presents itself. before turning to a possible solution for the incommensurability/incomparability impasse some discussion of the objectivism/subjectivism debate will prove fruitful. in general ethics, philosophers actively and explicitly debate whether good is objective or subjective. on the one hand are arguments for value anti-realism, i.e. subjectivism about values (mackie, 1977; thomson, 1997; fehige, 2006; heathwood, 2014), while on the other hand strong arguments are raised for value realism (railton, 1986; arneson, 1999, 2010; oddie, 2005; bradley, 2014). arneson (1999) says that in this philosophical debate, different questions have been asked under these two descriptions, that of good as objective or subjective. subjective theories of human good are sometimes taken to be those that make welfare depend at least in part on some mental state. the intended contrast is with objective theories of well-being which make the well-being of an agent depend entirely on states of the world apart from the state of mind of the agent whose well-being is under review. this, arneson (1999), claims is a coherent usage, but potentially confusing. he prefers to let the contrast between objective and subjective mark the contrast between (i) views which hold that claims about what is good can be correct or incorrect and that the correctness of a claim about a person's good is determined independently of that person's volition, attitudes, and opinions and (ii) views which deny this. oddie (2005) defines value realism as the thesis that value claims; (i) can be literally true or false; (ii) that some such claims are indeed true; (iii) that their truth is not simply a matter of any individual's subjective attitudes or even of the attitudes of some larger collective. the subjective theory of value, on the other hand, is a theory of value which advances the idea that the value of a good is not determined by any inherent property of the good, nor by the amount of labour required to produce the good, but instead value is determined by the importance an acting individual places on a good for the achievement of their desired ends (rorty, 1991; von mises, 1998; heathwood, 2014). linking this debate with the one regarding incomparability, rauschmayer (2001) indicates that the philosophical debate about incomparability is marked by three oppositions of which the second is about the origin of value(s); i.e. whether they are objective or subjective values are somehow given and not subject to conscious change (objective), or they are constructed by practical reason in the decision process (subjective). mather ([2002) distinguishes between real and rational incommensurability where real incommensurability results due to objective ijahp article: von solms/comparability, decision theory and the ahp international journal of the analytic hierarchy process 525 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.313 evaluative facts in reality outside of (and beyond) human knowledge and belief, while rational incommensurability implies that the ranking of items cannot be made by human judges according to the rational principles. boot (2007) concludes that it is often impossible to rationally and objectively (instead of intuitively and subjectively) weigh the different types of values involved in a decision situation and to rationally and objectively resolve their conflicts. he argues that this impossibility is due to incomplete comparability of the relevant options, which in its turn, is largely caused by incommensurability of the relevant values. however, different authors’ objectivist/subjectivist stances are not always explicit in the incommensurability/incomparability literature. often the case for (or against) commensurability is phrased in terms that imply an objective view of value. so, for example, is incommensurability often defined as the case where two values (a and b) are incommensurable because no measure exists by which both a and b can be measured (raz, 1986, 1997), or it is also seen in, for example, the question ‘was mozart more creative than michelangelo?’ (laitenin, 2008; klocksiem, 2010). in both these cases the impression is created (albeit not explicitly stated) that the measurement (or comparison) of objects must rely on a pre-existing objective fact as to their relative value a fact that must be discovered from the structure of reality rather than established via the preferences of decision-makers. boot (2007) is more explicit than most, indicating that comparisons can be done either in a subjective or objective way, but identifies subjective comparisons as cases of incomplete comparison. to the extent that judgments are not merely based on objective amount comparisons but also on subjective importance comparisons, they are not the result of a completely rational comparison. that any plausible theory of practical reason must be comparativist in form is a frequent point made, over a number of years, by chang (1997, 2004c, 2012, 2013). it is no wonder then that okapal (2010) believes chang to have developed the most detailed comparativist view. okapal (2007, 2010) criticizes this view, not because of its comparativism, but rather in terms of its insistence on covering values at the expense of other more orthodox views. for our purposes, however, it is the light that this critique sheds on the debate about the objectivity or subjectivity of value that is of importance. ruth chang not only publishes widely within the incommensurability/incomparability field but she is also frequently critiqued by other authors (and her views can fruitfully be considered as a starting point when discussing the objectivity/subjectivity issue (chang, 1997, 2001, 2012, 2013; anderson, 1997; broome, 1997, 2000; gert, 2004; boot, 2007, 2009; okapal, 2007, 2010) . okapal (2007, 2010) reads chang (1997, 2002b, 2004c) as arguing for a situation that values are somehow given and fixed in decision-making situations. the changian mcv approach is a case in point as aggregation is found via the intrinsic normative structure of the covering value and not via any subjective criteria weighting because the criteria (covering values) possess an inherent, objective unifying principle even if individual judges do not know what it is (chang, 2004a). okapal (2007, 2010) characterizes the core idea in the changian system as that for any given choice situation a single, comprehensive factor exists that determines choice. interpreting the jigsaw puzzle metaphor of chang (2004a, 2004b) in a literal (strong) way, okapal (2007, 2010) reaches the conclusion that this implies that the changian comprehensive value approach accepts these factors as static, complete and given to the decision-maker, and that this precludes any genuine disagreement, because in any disagreement (at least) one of the parties must be wrong. the impression that chang (2002b) proposes an objectivist approach also seems clear from her own statement that by ‘the justification of choice’ she has in mind the all-in, ijahp article: von solms/comparability, decision theory and the ahp international journal of the analytic hierarchy process 526 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.313 normative, undefeated, objective ground of or reason or warrant for choice. this, however, remains obscure because, a few paragraphs later, she says that although she focuses on the objective justification of choice the same arguments can be adjusted to apply to justification subject to an agent’s foibles (sic). she mixes statements of objects having value with statements regarding agents having reasons to value these objects in a way that makes it difficult to see whether she is arguing for intrinsic (objective) value or more subjective value instilled by the agent’s valuation (chang, 2004c). for example, she often refers to ‘evaluative facts’ about objects and actions but it is not clear as to where the facts are located. an example she uses is the choice to eat ice cream – it is not clear whether she sees the value of this as based on the physical and chemical properties of the ice cream (objective) or on the agent’s (subjective) valuing the taste and pleasure of enjoying an ice cream. chang (2004c) identifies desire-based accounts, in which all practical reasons are grounded in the present desires of the agent; justification has its source in the fact that an agent wants it. in opposition to these she identifies value-based accounts, reasons for acting are provided by facts about the value of something, where being valuable is not simply a matter of being desired. it is not the fact that an agent wants something that makes having it valuable – it would be valuable even if the agent does not desire it. 2 chang (2004c) feels that the conflict between these two stories is striking because it is so stark; desire-based theorists think that all reasons are grounded in desires while their value-based opponents tend to think that none are, and as with many such conflicts, the truth may lie somewhere in the middle – the position she argues for. chang (2013) is concerned that evaluation is often done in the abstract rather than in terms of substantive considerations. she says that there is good reason to think that values do not rank themselves in the abstract but are rather ranked by substantive covering values. this, however, still sounds as if the covering values do the ranking independent of the decision-maker’s preferences or goals and it seems that grimm (2007) reads the changian view as objective, i.e. that measures exist and rank options vis-á-vis humans measuring and ranking. in a more explicit discussion, hsieh (2007) states that values are incommensurable if and only if there is no true general overall ranking of the realization of one value against the realization of the other value. this (as mentioned above) is the definition of value realism the thesis that value claims can be literally true or false and that their truth is not simply a matter of any individual's subjective attitudes (arneson, 1999, 2010; oddie, 2005). richardson (1994) defines strong commensurability as the thesis that there is a true ranking of the realization of one value against the realization of the other value in terms of one common value across all conflicts of value. a denial of such a singular common value, however, does not rule out what richardson (1994) calls weak commensurability the thesis that in any given conflict of values, there is a true ranking of the realization of one value against the realization of the other value in terms of some value. another conception of value incommensurability denies both strong and weak commensurability claiming that in some conflicts of values – in cases where the gain in one factor cannot compensate for the loss in another factor there is no true ranking of values. hsieh (2007) points out that several authors argue for different external (non-comparativist) sources of commensurating value. he concludes, however, that even if these are external 2 chang (2004c) identifies versions of the value-based view, according to which it is not strictly the evaluative fact that provides the reason but the facts upon which the evaluative fact supervenes. the supervenience debate is vast and complex (dorsey, 2012) but will be ignored here, because our argument stands whether objectivism is based on evaluative facts or on the facts on which they supervene. ijahp article: von solms/comparability, decision theory and the ahp international journal of the analytic hierarchy process 527 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.313 sources of commensurating value, it does not follow that they provide a systematic resolution to value conflicts because two individuals can still resolve the same value conflict differently. the fact that it may be rational to resolve the same value conflict in different ways, points to the possibility of value incommensurability, i.e. that no true ranking of options exists. the aspect of truth is not always clear within the incommensurability/incomparability discussions but is central to the general ethics debate. value realism relies on the idea of truth, specifically, truth as correspondence with facts. truth, however, is an elusive concept and particularly the realist interpretation is widely criticized (rescher, 1973; checkland & scholes, 1990; rorty, 1991; jackson, 2003; bowden & swartz, 2004; mingers, 2008). mather (2002) points out that the concept of truth is ambiguous, and particularly the realist version is probably beyond the reach of human decision-makers. mather (2002) tries to circumvent the problems the realist version of truth presents to decision-making by distinguishing between real and rational truth; real truth resides in moral realism while rational truth implies that the truth of propositions are determined by human judges in deliberative processes applying rational principles. arneson (1999) mentions that one criticism of the objective list theories is a skeptical doubt that there is no rational way to determine what putative goods qualify as entries on such a list. although he does not discuss it any further, he acknowledges that this doubt is a genuine worry. crisp (2013), on the other hand, does criticize objective list theories for, inter alia, this exact reason. the problem of how to understand truth leads to the view that mistaken decisions are often based on a lack of full information or the presence of cognitive bias and several authors, consequently, argue for a full information account of the good (railton, 1986; arneson, 1999; carson, 2013). railton (1986) defines a person’s non-moral good as consisting in what he would want himself to want, or to pursue, were he to contemplate his present situation from a standpoint fully and vividly informed about himself and his circumstances, and entirely free of cognitive error. these full information accounts are not without critics (rosati, 1995; murphy, 1999). the criticisms vary but what must be accepted here is that such an idealized decision-maker is of no use in practical decision-making because, given that the real truth is guaranteed only by ideal cognitive circumstances – by optimal coherence with a perfect data base that we do not have, rather than by apparent coherence with the sub-optimal data base we actually do have, we have no categorical assurance of the actual correctness of our inquiries, and no unqualified guarantee that they provide the real truth (rescher, 1973). the problem and limitations of a realist view of truth in the context of decision-making were identified and discussed elsewhere (von solms, 2009, 2011). the debate regarding objective/subjective incommensurability or incomparability is important also regarding the establishment of scales. are scales given by the structure of reality, i.e. do different values exhibit fixed, predetermined amounts of value derived from a universal hierarchy of values? can values be measured on fixed, predetermined scales or failing that must they be deemed incommensurable in principle because no universal scales exist in nature? alternatively, the subjective side of the debate argues for measurement or comparisons that rely on human preference and choice. the implication of this is that scales must be developed by decision-makers using logical and rational principles – mather’s (2002) rational commensurability. in summary, incommensurability or incomparability based on an objectivist comparativism leads to serious problems because accepting that measurement or comparisons must proceed via existing scales or hierarchies and accepting that, in many cases, such scales or hierarchies are not available, practical decision-making is ijahp article: von solms/comparability, decision theory and the ahp international journal of the analytic hierarchy process 528 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.313 seriously compromised. this problem is arguably attested to by authors in the incommensurability/incomparability literature when attempts are made to escape the dilemma. the first line of argument is to avoid comparativism by substituting other bases of choice, e.g. agency or practical reason and a second line of argument is to include indeterminate choices into comparativism via terms like parity; rough equality; clumpiness or vagueness (raz, 1986, 1997; anderson, 1997; chang, 2002a; griffin, 1997; hsieh, 2005a; broome, 1997, 2000; harris, 2001). a subjectivist solution, discussed above, is social multi-criteria evaluation (but this is based on weak comparability which involves ordinal multi-criteria evaluations which was shown to falter on arrowian impossibility. in the light of these problems, saaty’s (2010, 2011) form of comparativism should now be considered as an alternative solution (munda, 2004; o’neill, 1997). 8. a solution to the problems of incomparability – saaty’s comparativism the first problem identified above was that of scale; the debate regarding incommensurability vs. incomparability. working definitions emanating from this debate resulted in using incommensurability in cases where no cardinal scale exists to measure options while incomparability exists when no ordinal ranking can be achieved amongst options. by using cardinal scales (ratio/absolute) based on comparisons, saatian comparativism bridges this incommensurability/incomparability gap. by realizing the possibility of establishing cardinal scales through a process of pairwise comparisons and appreciating the mathematical requirements of this process, ahp achieves commensurability via comparability. saaty (2010) argues that many people think that measurement needs a physical scale with a zero and a unit to apply to objects or phenomena. he points out that this is not true. surprisingly, he says, accurate and reliable relative scales that do not have a zero or a unit – can be derived by using human understanding and judgments. decision-makers can apply their minds and understanding to make comparisons; and these comparisons can be made on a meaningful scale. the second issue discussed was the argument – mainly from traditional measurement theory – that we cannot measure intangibles and that true measurement is only possible for objects that can be shown to be quantitative. saatian comparability does not only argue against the view that intangibles cannot be measured but also provides a mathematically sound alternative in which intangibles are included along with tangible variables in measurement via comparisons and, hence, in decision-making (saaty & sagir, 2009; saaty, 2010, 2013). the third problem was that of aggregation in culti-criteria decision-making situations. the problem of multiple criteria decisions was shown to emanate from attempts to combine evaluations on diverse (conflicting) criteria. this problem, named bidirectionality, is a result of the use of ordinal scales where proper weighing of criteria importance is not done adequately. the problem – arrowian impossibility identified in social choice theory presented by voting-style ordinal evaluations, is conclusively removed in saatian comparativism by the use of absolute scale values (saaty & vargas, 2005; saaty & peniwati, 2008). gdm presents a similar problem when preferences of a single decision-maker are combined into an overall group preference structure, but here too saatian comparativism provides for aggregation via consensus or geometric mean calculations or via a combination approach in the 3phase application of the ahp (forman & peniwati, 1998; saaty & peniwati, 2008; saaty, 2010; von solms & peniwati, 2001). ijahp article: von solms/comparability, decision theory and the ahp international journal of the analytic hierarchy process 529 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.313 the last, and possibly the most important, aspect of the incomparability debate was that of whether values and preferences are subjective or objective. the search for an objective, eternal and universal hierarchy of values applicable to all decision-makers in all decision situations leads inevitably to an acknowledgement of incomparability as we find that such a hierarchy does not exist, or is at least not available to human decision-makers. in contrast, saatian comparativism is based on the acceptance of the futility of searching for ‘pre-determined’ cardinal scales, ordinal hierarchies or lexical priorities as a general method. on the contrary, the human need for and ability to compare is not taken seriously enough and we must accept the innate ability of human agents to express preferences through comparisons. instead of seeking for universal ranking of values the ahp focuses on providing decision-makers with a way of prioritizing issues in a specific decision situation relevant to the specific environment of the particular situation, the objectives of the people involved in that decision and the issues at the heart of this decision (saaty & peniwati, 2008). the ahp depends on the knowledge and experience of people and varies from problem to problem. saaty (2010) says that the ahp relies on judgments and is therefore subjective because judgments can differ from one person to another, while saaty & vargas (2012) say that the ahp provides the objective mathematics to process the inescapably subjective and personal preferences of an individual or a group in making a decision. in saaty’s (2000) own words: “what does it mean to be objective? if all reality is a matter of interpreting stimuli and information according to our needs and goals, then it is meaningless to speak of universal objectivity. facts, numbers and other stimuli are regarded by each person according to a certain purpose. unless purpose and the experience behind it are identical for all people, it is futile to insist that they all look at a datum with the same sense of priority or urgency. what we need is to persuade each other of the usefulness for all concerned to see things from the standpoint of some interpretation. ... we are all conditioned and biased by our family, environment, the teachers who teach us and the books we read.” no wonder then that vargas (1994), when comparing ahp to maut and outranking methods, specifically mentions that the composition principles are just as subjective as the concept of independence on which they are based. he claims that the basic distinction with the ahp is that from the start, it assumes that all is relative and subjective. 9. conclusion the debate regarding incomparability was seen to be varied and intense making a perfunctory or uncritical acceptance of comparativism wrong. many reasons for incommensurability or incomparability were highlighted. the first conclusion reached from our discussion is that saaty’s (2011) view that comparison forms an important aspect of logic is confirmed and although incomparability seems a serious problem, a solution seems to exist in saatian comparativism. a second issue that became apparent during our review of the literature is that the work of tom saaty is not reflected in the incomparability or incommensurability literature as far as the current author could establish. the only references to ahp are found in (i) the outranking literature arguing that the ahp is based on general comparability while the orms honour and incorporate incomparability and (ii) the decision theoretical literature where mostly the mathematical or axiomatic aspects of ahp that are discussed. the incommensurability/incomparability literature is void of any mention of the philosophical and logical contributions made by saaty to the comparability debate and to the sound mathematical methodology underpinning his views. the ijahp article: von solms/comparability, decision theory and the ahp international journal of the analytic hierarchy process 530 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.313 comparativism debate can only be enriched if, on one hand, the ahp/anp literature recognizes the problems and concerns regarding comparativism raised by the incommensurability/incomparability authors. on the other hand, the comparability literature should seriously consider saatian comparativism and the solutions it proposes. this paper is an attempt to help initiate a fruitful dialogue between these groups. ijahp article: von solms/comparability, decision theory and the ahp international journal of the analytic hierarchy process 531 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.313 references acton, g.s. (2003). what is good about rasch measurement? rasch measurement transactions, 16(4), 902-903. aldred, j. (2002). cost-benefit analysis, incommensurability and rough equality. environmental values, 11(1), 27-47. doi: 10.3197/096327102129340966 aldred, j. (2006). incommensurability and monetary valuation. land economics, 82(2), 141-16. anderson, e. (1997). practical reason and incommensurable goods. in chang, r. (ed), incommensurability, incomparability and practical reason. cambridge, ma: harvard university press. arneson, rj. (1999). human flourishing versus desire satisfaction. social philosophy and policy, 16(1), 113-142. arneson, rj. (2010). good, period. analysis, 70(4), 731-744. doi: 10.1093/analys/anq019 arrow, kj. (1951). social choice and individual values. new york, ny: john wiley & sons. arrow, kj & raynaud, h. (1986). social choice and multicriterion decision-making. cambridge, ma: mit press. barrett, pt. (2003). beyond psychometrics: measurement non-quantitative structure and applied numerics. journal of managerial psychology, 18(5), 421-439. doi: 10.1108/02683940310484026 bisdorff, r. (2004). concordant outranking with multiple criteria of ordinal significance. 4or, 2(4), 293-308. doi: 10.1007/s10288-004-0053-7 boot, m. (2007). incommensurability, incomplete comparability and the scales of justice. dphil dissertation; balliol college, oxford, uk: university of oxford. boot, m. (2009). parity, incomparability and rationally justified choice. philosophical studies, 146 (1), 75-92. doi: 10.1007/s11098-008-9245-x bouyssou, d. (2001). outranking methods. in floudas, c.a. & pardalos, p.m. (eds), encyclopedia of optimization. london, uk: kluwer academic publishers. bouyssou, d., marchant, t., pirlot, m., et al. (2000). evaluation and decision model: a critical perspective. dordrecht, the netherlands: kluwer academic publishers. bowden, b & swartz, n., (2004). truth. the internet encyclopedia of philosophy. online at http://www.iep.utm.edu/t/truth.htm bradley, b. (2014). objective theories of well-being. in eggleston, b. & miller, d.e. (eds), the cambridge companion to utilitarianism. cambridge, uk : cambridge university press. http://philpapers.org/s/richard%20j.%20arneson http://www.informatik.uni-trier.de/~ley/db/journals/4or/4or2.html#bisdorff04 http://www.iep.utm.edu/t/truth.htm ijahp article: von solms/comparability, decision theory and the ahp international journal of the analytic hierarchy process 532 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.313 brans, j-p. & mareschal, b. (2005). promethee methods. in figueira, j. greco, & ehrgott, m. (eds), multiple criteria decision analysis: state of the art surveys. new york, ny: springer science & business media inc. broome, j. (1997). is incommensurability vagueness? in chang, r (ed), incommensurability, incomparability and practical reason. cambridge, ma: harvard university press. broome, j. (2000). incommensurable values. in crisp, r. & hooker, b. (eds); wellbeing and morality: essays in honour of james griffin. oxford, uk: clarendon press. chang, r.e. (1997). introduction. in chang, r.e. (ed), incommensurability, incomparability and practical reason. cambridge, ma: harvard university press. chang, r.e. (1998). comparison and the justification of choice. university of pennsylvania law review, 146(5), 1569-1598. chang, r.e. (2001). against constitutive incommensurability or buying and selling friends. philosophical issues, 11(1), 33-60. doi: 10.1111/0029-4624.35.s1.2 chang, r.e. (2002a). the possibility of parity. ethics, 112(4), 659-688. doi: 10.1086/339673 chang, r.e. (2002b). making comparisons count. london, uk: routledge. chang, r.e. (2004a). all things considered. philosophical perspectives, 18(1), 1-22. chang, r.e. (2004b). putting together morality and well-being. in baumann, p. & betzler, m. (eds), practical conflicts. cambridge, uk: cambridge university press. chang, r.e. (2004c). can desires provide reasons for action? in wallace, r.j., pettit, p., scheffler, s. & smith, m. (eds), reasons and value: themes from the philosophy of joseph raz. oxford, uk: oxford university press. chang, r.e. (2012). value pluralism. in wright, j.d. (ed), international encyclopedia of the social and behavioral sciences, 2 nd edition. oxford, uk: elsevier publishers. chang, r.e. (2013). incommensurability (and incomparability). in lafollette, h. (ed); international encyclopedia of ethics. hoboken, nj: blackwell-wiley. checkland, p.b. & scholes, j. (1990). soft systems methodology in action. chichester, uk: john wiley & sons. coello coello, c.a. (2000). handling preferences in evolutionary multiobjective optimization: a survey. 2000 congress on evolutionary computation, 30-27. piscataway, nj; ieee service center. crisp, rb (2013)b well-being; in zalta, e.n. (ed); the stanford encyclopedia of philosophy. on-line at http://plato.stanford.edu/archives/sum2013/entries/well-being http://philpapers.org/go.pl?id=chatpo-5&proxyid=&u=http%3a%2f%2fdx.doi.org%2f10.1086%2f339673 http://plato.stanford.edu/archives/sum2013/entries/well-being ijahp article: von solms/comparability, decision theory and the ahp international journal of the analytic hierarchy process 533 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.313 da silva, va. (2011). comparing the incommensurable: constitutional principles, balancing and rational decision. oxford journal of legal studies, 31(2), 273-301. doi: 10.1093/ojls/gqr004 dembczyński, k., kotlowski, w. & slowiński, r. (2007). ordinal classification with decision rules. in raś, z.w., tsumoto, s. & zighed, d. (eds), proceedings of the 3 rd international workshop, mining complex data; warsaw, poland. dorsey, d. (2012). intrinsic value and the supervenience principle. philosophical studies, 157(2), 267-285. doi: 10.1007/s11098-010-9636-7 edmundson, w.a. (2009). pluralism, intransitivity, incoherence. in white, m.d. (ed), theoretical foundations of law and economics. cambridge, uk: cambridge university press. ellis, s. (2008). the main argument for value incommensurability (and why it fails). southern journal of philosophy, 46(1), 27-43. doi: 10.1111/j.20416962.2008.tb00068.x ellis, s. (2012). prioritizing multiple objectives: feeling our way. working paper; department of philosophy, norman, ok: university of oklahoma. available on-line at www.ou.edu/ouphil/faculty/ellis/prioritizing.pdf eriksson, l., (2003). completeness and incomparability: limits of rct. annual meeting of the american political science association, philadelphia, pa. on-line at www.allacademic.com/meta/p62882_index.html forman, e.h. & peniwati, k. (1998). aggregating individual judgments and priorities with the analytic hierarchy process. european journal of operational research, 108(1), 165-169. forman, e.h. & selly m.a. (2001). decision by objectives: how to confine others that you are right. singapore: world scientific publications. geldermann, j., spengler, t. & rentz, o. (2000). fuzzy outranking for environmental assessment: case study: iron and steel making industry. fuzzy sets and systems, 115(1), 45-65. doi:10.1016/s0165-0114(99)00021-4 gert, j. (2004). value and parity. ethics, 114(3), 492-510. gowdy, j. & erickson, j.d. (2005). the approach of ecological economics. cambridge journal of economics, 29(2), 207-222. doi: 10.1093/cje/bei033 greco, s., matarazzo, b. & slowiński, r. (2005). decision rule approach. in figueira, j., greco, s. & ehrgott, m. (eds), multiple criteria decision analysis: state of the art surveys. new york, ny: springer science & business media inc. griffin, j. (1997). incommensurability: what’s the problem?. in chang, r. (ed), incommensurability, incomparability, and practical reason. 35 cambridge, ma: harvard university press. http://ojls.oxfordjournals.org/search?author1=virg%c3%adlio+afonso+da+silva&sortspec=date&submit=submit http://philpapers.org/s/dale%20dorsey http://philpapers.org/go.pl?id=doriva&proxyid=&u=http%3a%2f%2fdx.doi.org%2f10.1007%2fs11098-010-9636-7 http://www.ou.edu/ouphil/faculty/ellis/prioritizing.pdf http://www.allacademic.com/meta/p62882_index.html http://dx.doi.org/10.1016/s0165-0114(99)00021-4 ijahp article: von solms/comparability, decision theory and the ahp international journal of the analytic hierarchy process 534 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.313 grimm, s.r. (2003). is incommensurability a problem for practical reason? paper presented at columbia/nyu graduate conference in philosophy, new york, ny: new york university. grimm, s.r. (2007). easy cases and value incommensurability. ratio, 20(1), 26-44. doi: 10.1111/j.1467-9329.2007.00344.x harris, g.w. (2001). value vagueness, zones of incomparability and tragedy. american philosophical quarterly, 38(2), 155-176. heathwood, c. (2014). subjective theories of well-being. in eggleston, b. & miller, d.e. (eds), the cambridge companion to utilitarianism. cambridge, uk: cambridge university press. hsieh, n-h.; (2005a). equality, clumpiness and incomparability. utilitas, 17(2), 180204. doi: http://dx.doi.org/10.1017/s0953820805001512 hsieh, n-h. (2005b). value maximization, incomparability and managerial choice. 3 rd biennial global conference on business ethics. santa clara, ca: santa clara university. hsieh, n-h. (2007). incommensurable values. in zalta, e.n. (ed), the stanford encyclopedia of philosophy; on-line at http://plato.stanford.edu/entries/valueincommensurable hurley, s.l. (1985). supervenience and the possibility of coherence, mind, 94(376), 501-525. doi:10.1093/mind/xciv.376.501 hurley, s.l. (1989). natural reasons, personality and polity. new york, ny: oxford university press. jackson, m.c. (2003). systems thinking: creative holism for managers. chichester, uk: john wiley & sons. kekes, j. (1992). pluralism and conflict in morality. journal of value inquiry, 26(1), 37-50. doi: 10.1007/bf00136589 kelly, c. (2008). the impossibility of incommensurable values. philosophical studies, 137(3), 369-382. doi: 10.1007/s11098-006-0005-5 klocksiem, j. (2010). in defense of the trichotomy thesis. acta analytica, 25(3), 317327. doi: 10.1007/s12136-009-0067-z kornhauser, l.a. (1998). no best answer?, university of pennsylvania law review, 146(5), 1599-1637. laitenin, a. (2008). strong evaluation without moral sources. berlin, germany: walter de gruyter gmbh & co. list, c. (2013). social choice theory. in zalta, e.n. (ed), the stanford encyclopedia of philosophy. on-line at http://plato.stanford.edu/archives/win2013/entries/socialchoice http://dx.doi.org/10.1017/s0953820805001512 http://plato.stanford.edu/entries/value-incommensurable http://plato.stanford.edu/entries/value-incommensurable http://philpapers.org/go.pl?id=hursat&proxyid=&u=http%3a%2f%2fdx.doi.org%2f10.1093%2fmind%2fxciv.376.501 http://plato.stanford.edu/archives/win2013/entries/social-choice http://plato.stanford.edu/archives/win2013/entries/social-choice ijahp article: von solms/comparability, decision theory and the ahp international journal of the analytic hierarchy process 535 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.313 luban, d. (2001). value pluralism and rational choice, georgetown university law center working paper no 264335, washington, dc: georgetown university. martinez-alier, j. munda, g. & o’neill, j. (1998). weak comparability of values as a foundation for ecological economics. ecological economics, 26(3), 277-286. mason, e. (2011). value pluralism. in zalta, e.n. (ed); the stanford encyclopedia of philosophy, published on-line at http://plato.stanford.edu/entries/value-pluralism mather, h.s. (2002). law-making and incommensurability. mcgill law journal, 47(2), 345-388. may, k.o. (1954). intransitivity, utility and the aggregation of preference patterns. econometrica, 22(1), 1-13. michell, j. (1997). quantitative science and the definition of measurement in psychology. british journal of psychology, 88(3), 355-383. michell, j. (2003). the quantitative imperative positivism, naive realism and the place of qualitative methods in psychology. theory and psychology, 13(1), 531.doi: 10.1177/0959354303013001758 michell, j. (2008). is psychometrics pathological science?, measurement, 6(1-2), 724. doi: 10.1080/15366360802035489 mingers, j.c. (2008). management knowledge and knowledge management: realism and forms of truth. knowledge management research & practice, 6(1), 62-76. doi: 10.1057/palgrave.kmrp.8500161 munda, g. (2004). social multi-criteria evaluation (smce): methodological foundations and operational consequences. european journal of operational research, 158(3), 662-677. doi:10.1016/s0377-2217(03)00369-2 munda, g. (2012). choosing aggregation rules for composite indicators. social indicators research, 109(3), 337-354. doi:10.1007/s11205-011-9911-9 munda, g. nijkamp, p. & rietveld, p. (1994). qualitative multicriteria evaluation for environmental management, ecological economics 10(2), 97-112. doi: 10.1016/0921-8009(94)90002-7 murphy, m.c. (1999). the simple desire-fulfillment theory. nous, 33(2), 247–272. nooteboom, b. (1984). intransitive preferences in retailing. services industries journal, 4(1), 82-92. okapal, j.m. (2007). comparative choice without comprehensive factors. 104 th annual meeting of the american philosophical association, chicago, il central division. okapal, j.m. (2010). problematic arguments for comprehensive values. florida philosophical review, 10(1), 43-74. http://plato.stanford.edu/entries/value-pluralism ijahp article: von solms/comparability, decision theory and the ahp international journal of the analytic hierarchy process 536 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.313 o’neill, j. (1997). value pluralism, incommensurability and institutions. in foster, j. (ed), valuing nature? ethics, economics and the environment. london, uk: routledge. pildes, r.h. & anderson, e.s. (1990). slinging arrows at democracy: social choice theory, value pluralism and democratic politics. columbia law review, 90(8), 21212214. qizilbash, m. (2000). comparability of values, rough equality and vagueness: griffin and broome on incommensurability. utilitas, 12(2), 223-240. qizilbash, m. (2005). the mere addition paradox, incompleteness and vagueness, conference on broome’s weighing lives. london school of economics; university of london. rabinowicz, w. (2008). value relations. theoria, 74(1), 18-49. doi: 10.1111/j.17552567.2008.00008.x railton, p. (1986). moral realism. philosophical review, 95(2), 163-207. doi: 10.2307/2185589 rauschmayer, f. (2001). philosophical aspects of incommensurability and incomparability, informatica, 12(1), 119-132. raz, j. (1986). the morality of freedom. oxford, uk: clarendon press. raz, j. (1997). incommensurability and agency. in chang, r.e. (ed), incommensurability, incomparability and practical reason. cambridge, ma: harvard university press. rescher, n. (1973). the coherence theory of truth. oxford, uk: clarendon press. richardson, h.s. (1994). practical reasoning about final ends. cambridge, uk: cambridge university press. rorty, r. (1991). objectivity, relativism and truth: philosophical papers. cambridge, uk: cambridge university press. rosati, c.s. (1995). persons, perspectives and full information accounts of the good. ethics, 105(1), 296-325. doi: 10.1086/293702 roy, b. & vincke, p. (1984). relational systems of preference with one or more pseudo-criteria: some new concepts and results. management science, 30(11), 13231335. http://dx.doi.org/10.1287/mnsc.30.11.1323 saaty, t.l. (2000). fundamentals of decision making and priority theory with the analytic hierarchy process. pittsburgh, pa: rws publications. saaty, t.l. (2001). decision-making with dependence and feedback: the analytic network process; 2 nd edition. pittsburgh, pa: rws publications. saaty, t.l. (2010). principia mathematica decernendi: mathematical principles of decision-making. pittsburgh, pa: rws publications. http://philpapers.org/go.pl?id=raimr&proxyid=&u=http%3a%2f%2fdx.doi.org%2f10.2307%2f2185589 http://philpapers.org/go.pl?id=rosppa-4&proxyid=&u=http%3a%2f%2fdx.doi.org%2f10.1086%2f293702 http://dx.doi.org/10.1287/mnsc.30.11.1323 ijahp article: von solms/comparability, decision theory and the ahp international journal of the analytic hierarchy process 537 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.313 saaty, t.l. (2011). the serious omission of comparisons in aristotle’s laws of thought. international journal of the analytic hierarchy process, 3(2), 180-181. doi: http://dx.doi.org/10.13033/ijahp.v3i2.124 saaty, t.l. (2013). on the measurement of intangibles: a principal eigenvector approach to relative measurement derived from paired comparisons. notices of the american mathematical society, 60(2), 192-208. saaty, t.l. & peniwati, k. (2008). group decision making: drawing out and reconciling differences. pittsburgh, pa: rws publications. saaty, t.l. & sagir, m. (2009). extending the measurement of tangibles to intangibles, international journal of information technology and decision making, 8(1), 7-27. doi: 10.1155/2012/873710 saaty, t.l. & vargas, l.g. (2005). the possibility of group welfare functions. international journal of information technology & decision making, 4(2), 167-176. doi:10.1007/s00355-011-0541-6 saaty, t.l. & vargas, l.g. (2009). decision-making with the analytic network process: economical, political, social and technological applications with benefits, opportunities, cost and risks, 2 nd edition; new york, ny: springer science & business media. saaty, t.l. & vargas, l.g. (2012). models, methods, concepts and applications of the analytic hierarchy process, 2 nd edition. new york, ny: springer science & business media. sen, a.k. (1995). rationality and social choice. american economic review, 85(1), 1-24. seung, t.k. & bonevac, d. (1992). plural values and indeterminate rankings. ethics, 102(4), 799-813. simpson, l. (1996). do decision makers know what they prefer?: mavt and electre ii. journal of the operational research society, 47(7), 919-929. sunstein, c.r. (1997). incommensurability and kinds of valuation: some applications in law. in chang, r.e. (ed), incommensurability, incomparability and practical reason. cambridge, ma: harvard university press. thomson, j.j. (1997). the right and the good. journal of philosophy 94(6), 273-298. doi: jphil199794637 tiberius, v. (2008). the reflective life: living wisely with our limits. oxford, uk: oxford university press. vargas, l.g. (1994). comparison of three multicriteria decision making methodologies: the analytic hierarchy process, multiattribute utility theory and outranking methods, proceedings of the 3 rd international symposium on the analytic hierarchy process, washington, dc. http://dx.doi.org/10.1155/2012/873710 http://philpapers.org/go.pl?id=thotra&proxyid=&u=http%3a%2f%2fdx.doi.org%2fjphil199794637 ijahp article: von solms/comparability, decision theory and the ahp international journal of the analytic hierarchy process 538 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.313 vincke, p. (2000). {p,q,i,j} – preference structures. in fodor, j. de baets, b. & perny, p. (eds), preference and decisions under incomplete knowledge. new york, ny: physica-verlag. von mises, l. (1998). human sction: a treatise on rconomics, scholar’s edition. auburn, al: ludwig von mises institute. von solms, s.h. (2009). homogeneity and choice aggregation in the analytic hierarchy process. proceedings 10 th international symposium on the analytic hierarchy process, pittsburgh, pa: university of pittsburgh. von solms, s.h. (2011). validity of the ahp/anp: comparing apples and oranges, international journal of the analytic hierarchy process, 3(1), 2-27. doi: http://dx.doi.org/10.13033/ijahp.v3i1.60 von solms, s.h. (2013). the fiction of a factual approach to decision-making. proceedings 12 th international symposium on the analytic hierarchy process, kuala lumpur, malaysia. von solms, s.h. & peniwati, k. (2001). to agree or not to agree, that is the question: choice aggregation in the ahp. proceedings 6 th international symposium on the analytic hierarchy process, bern, switzerland. wasserman, r. (2004). indeterminacy, ignorance and the possibility of parity. philosophical perspectives, 14(1), 391-403. ijahp article: mancini, marcarelli/high school choice: how do parents make their choice? international journal of the analytic hierarchy process 91 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.633 high school choice: how do parents make their choice? paola mancini university of sannio italy paola.mancini@unisannio.it gabriella marcarelli university of sannio italy gabriella.marcarelli@unisannio.it abstract location, well-qualified teachers, leaving score and academic performance are the main factors associated with parents' high school choices. this paper aims to provide students and their parents with a helpful tool for synthesizing these elements. by focusing on a small italian town, we analyze eduscopio and scuolainchiaro’s data concerning high schools’ characteristics and students’ performances, and apply the analytic hierarchy process (ahp) in order to derive the ranking of high schools taking into account three criteria: the students' performance at school, their academic performance and the school’s characteristics (such as the number of students per class and per teacher). the results from the ahp procedure using only school performances and characteristics highlight that the classic lyceum has the best performance and the scientific lyceums rank second, albeit rather close to the other lyceums. entering the academic performance factor into the model changes the ranking of preferences in favor of the scientific lyceum, whose value is slightly higher than the classic one, and decreases the values of the other lyceums and technical high schools. this is due to the excellent academic performance of those who leave scientific schools, mostly in terms of credits at the end of the first year and average exam scores. keywords: school choice; school performance; academic performance; ahp; eduscopio 1. introduction wespieser et al. (2015) analyzed the factors that parents consider when choosing a school: location, well-qualified teachers, discipline and exam results to cite a few. the importance of these factors may vary depending on the parents’ household income. the authors highlight that, in order to decide (or help them decide) which school their child should attend parents discuss with other parents, undertake their own research on possible schools and/or select the school attended by the child’s siblings. the international literature about this topic is vast. the literature covers aspects dealing with educational choice ranging from the effect of educational reforms in favor of mailto:mancini.paola.64@gmail.com mailto:gabriella.marcarelli@unisannio.it ijahp article: mancini, marcarelli/high school choice: how do parents make their choice? international journal of the analytic hierarchy process 92 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.633 improving equity and social justice, to the influence of parents’ involvement and family engagement on the students’ achievements, and last but not the least, the role of socioeconomic status in the choice among public and private school (anderson & minke, 2007; hanushek et al., 2007; goldhaber, 1999; seitsinger, 2019; seitsinger & brand 2011; taylor, 2018). by considering preferences for schools, burgess et al. (2014) showed that parents’ choices are influenced by the academic performance of the school, but recognized the great importance of socio-economic factors and location (proximity to home) too. lauer (2003) analyzed the impact of family background and cohort on educational achievement. with regard to the italian school system, agasisti and murtinu (2012) highlighted that the main factors influencing students’ achievements are their socio-economic status, geographical area of residence and the type of school attended. a number of scholars have analyzed the effect of school size and other significant characteristics, such as students’ features and school management and resources, on italian students’ achievements (giambona & porcu, 2018; masci et al., 2018). in recent decades, schools have achieved more autonomy in organizing their activities and introducing innovative teaching methods and changes in their educational programs. nevertheless, masci et al. (2018) showed that managerial practices have little influence on students’ school performance. investigating the main factors behind the parents’ choice between public and private school, pandolfini (2013) underlined, besides the influence of parents’ educational level and socio-economic status, the importance of other latent variables like family involvement and parents’ motivation. in order to provide a tool for parents to be involved in a school choice for their children, we propose the application of the analytic hierarchy process (ahp) to rank preferences among different types of high schools, considering three criteria: the students' school and academic performance and the school characteristics. since in italy only 4.24% of students attend a private high school (i.stat 2014), we consider the socio-economic factor negligible in our analysis. annually, the eduscopio portal provides parents with a tool to identify the best school within their area of residence. the ranking of schools is based only on students’ academic performance. in this paper, we performed the ahp procedure in two steps. first (model 1), we considered only 2 criteria accounting for school characteristics and performance in order to capture the "school effect", and then in (model 2) we included the students' academic performance in the model in order to identify the "mixed school/academic effect". this paper is organized as follows: after a brief introduction on the school choice problem, section 2 describes data sources and provides an exposition on the methodology used to support parents in that choice; then, a discussion of results is performed (section 3); finally some concluding remarks are provided (section 4). ijahp article: mancini, marcarelli/high school choice: how do parents make their choice? international journal of the analytic hierarchy process 93 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.633 2. data & methods 2.1 data collection in italy, the eduscopio portal has provided parents with a tool to identify the best school within their area of residence. the ranking of schools is based only on students’ academic performance, as measured by the fga (giovanni agnelli foundation) index, taking into account academic credits and average exam scores. for more details, see bordignon et al. (2017). in order to support parents’ school choice, we performed a pilot analysis based on 28 high schools located in the province of benevento, campania, grouped into 4 typologies: 8 classic lyceums, 10 scientific lyceums, 4 linguistic and human sciences lyceums and 6 technical (economic or technological) high schools. the eduscopio (giovanni agnelli foundation) and scuolainchiaro (ministry of education) portals provided the data. with reference to the 2017/18 school year, data from eduscopio (https://eduscopio.it/) deals with some students' school and academic performance (henceforth called criterion 1 and criterion 2); data from scuolainchiaro (http://cercalatuascuola.istruzione.it/ cercalatuascuola/) provide information on the characteristics of the school (henceforth called criterion 3) and the invalsi 1 test score. in order to define the sub-criteria associated with each criterion, we consider a number of basic indicators, as proposed directly by eduscopio, and new indicators, as can be seen in table 1. the intuitive significance of the sub-criteria accounts for students' performance through the invalsi's test score, the hs leaving score, the proportion of academic enrolled students and the percentage of those passing the first year specifically their credits and average exam scores. regarding school characteristics, it is worth pointing out that the students per class and per teacher sub-criteria are considered in order to take into account the support (assistance provided) in the educational path, whilst the percentage of teachers employed in the school for more than 6 years may represent an indicator of educational continuity. 1 the national institute for the evaluation of the educational system ijahp article: mancini, marcarelli/high school choice: how do parents make their choice? international journal of the analytic hierarchy process 94 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.633 table 1 the elements of the problem and data sources portal criteria sub-criteria description e d u s c o p io s c h o o l p e rf o rm a n c e high school leaving score calculated as the weight average between the high school leaving score of academic enrolled and not enrolled students proportion of academic enrolled students basic indicator relative difference (compared to regional average) of academic enrolled students calculated as the ratio between regional average minus school value (in the numerator) and corresponding regional value (in the denominator) for each school; values higher than 0 indicate a better performance than campania's regional average; vice versa for values lower than 0 a c a d e m ic p e rf o rm a n c e percentage of students passing the first year calculated as the proportion of academic enrolled students relative difference (compared to regional average) of students passing the first year calculated as the ratio between regional average minus school value (in the numerator) and corresponding regional value (in the denominator) for each school; values higher than 0 indicate a better performance than campania's regional average; vice versa for values lower than 0 percentage of academic credits at the end of the first year in reference to total credits expected at the end of the first year average exam score basic indicator s c u o l a i n c h ia r o s c h o o l c h a ra c te ri st ic s students per class average number of students per class basic indicator students per teacher average number of students per teacher basic indicator percentages of teachers employed for more than 6 years in the school basic indicator s c h o o l p e rf o rm a n c e invalsi's test score computed as the average between individual students' math, reading and foreign language test scores basic indicator ijahp article: mancini, marcarelli/high school choice: how do parents make their choice? international journal of the analytic hierarchy process 95 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.633 our dataset consists of the average values of the indicators for each type of school (table 2). as can be expected, the data highlight the best school and academic performances of lyceums, particularly classic and scientific lyceums. the high school leaving score exceeds 80 out of 100, and the invalsi's test score is very close to 5. more than 85% of school leavers continue their studies, achieving approximately a third of their total academic credits at the end of the first year, with an average exam score of more than 25 out of 30. on the opposite side, there are the technical high school students. their leaving score is nearly 8 points lower than the classic lyceum students, the invalsi's test score is 1.2 points lower than scientific lyceum students; interestingly, only 1 in 3 school leavers decide to continue their studies, achieving less than half of their credits during the first academic year, with an average exam score 2 points lower than scientific lyceum students. table 2 average values of the sub-criteria c ri te ri a sub-criteria classic lyceum scientific lyceum other lyceum technical hs s c h o o l p e rf o rm a n c e high school leaving score 83.9 81.2 78.1 75.7 proportion of academic enrolled students 87.2 85.6 64.3 38.2 relative difference (compared to regional average) of academic enrolled students -4.8 0.1 -0.6 -6.5 invalsi's test score 4.7 4.8 4.1 3.6 a c a d e m ic p e rf o rm a n c e percentage of students passing the first year 88.0 88.6 83.7 79.4 relative difference of students passing the first year -2.9 1.8 -2.0 8.0 percentage of academic credits at the end of the first year 61.8 66.0 50.5 48.7 average exam score 25.3 25.9 23.2 23.8 s c h o o l c h a ra c te ri st ic s students per class 18.7 20.9 20.4 19.2 students per teacher 10.2 10.7 10.6 8.4 percentage of teachers employed in the school for more than 6 years 51.7 52.4 57.4 50.8 compared to the regional average (for the same type of school), scientific lyceums seem to almost always achieve the best score (values higher than 0) in the students' academic enrollments and the overcoming of the first year factors, whilst classic lyceums almost always achieve the worst score. nevertheless, even if the percentage of academic enrollments for technical high school students is much lower than the regional value, they ijahp article: mancini, marcarelli/high school choice: how do parents make their choice? international journal of the analytic hierarchy process 96 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.633 perform much better with regard to first year university careers. regarding school characteristics, the lowest value of students per teacher is in the technical high schools and extended teacher employment in other lyceums should be noted. 2.2 methodology the ahp, developed by saaty at the end of the 1970s, is a multicriteria method which provides the decision maker with a tool to analyze problems involving several conflicting factors and stakeholders. it allows the decision maker to combine both qualitative and quantitative aspects in a single framework and then develop priorities for alternatives based on the judgments of experts. by breaking the problem down into simpler decisions, the ahp provides the decision maker with an analytical tool that can help him/her solve a more complex problem. the aim of this study is to apply the ahp in order to assist parents and their children in the selection of a high school. the ahp involves 4 steps: (i) the decomposition of the decision problem into a multilevel hierarchy; (ii) data collection by means of pairwise comparisons; (iii) determination of the relative weights, reflecting the relative importance of the elements belonging to each hierarchical level considered with respect to the elements of the level immediately above; (iv) aggregation of the relative weights to obtain the overall priorities, expressing the importance of alternatives with respect to the overall objective of the evaluation (by applying the principle of hierarchical composition). this priority vector provides the ranking of alternatives (saaty & vargas, 1982). in order to choose among the 4 types of high schools, we propose the consideration of 3 criteria: the students' performance during secondary school (high school leaving score, percentage of academic enrolled students, relative difference of academic enrolled students and invalsi test score), their academic performance (percentage of students passing the first year, relative difference of students passing the first year, percentage of academic credits at the end of first year and average exam score) and the characteristics of that school (the number of students per class and students per teacher and the percentage of teachers employed by the school for more than 6 years). figure 1 shows the structure of the problem and its 4 hierarchical levels. we performed the ahp procedure in two steps. first (model 1), we only considered the 2 criteria accounting for school characteristics and performance in order to capture the "school effect", then (model 2), we included the students' academic performance in order to identify the "mixed school/academic effect". ijahp article: mancini, marcarelli/high school choice: how do parents make their choice? international journal of the analytic hierarchy process 97 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.633 figure 1 hierarchy of a school choice problem 2 3. findings and discussion the results from the ahp procedure using only 2 criteria highlight the classic lyceum as the best performer (table 3 and figure 2), particularly for leaving score (0.483), academic enrollment (0.400) and students per class (0.520). scientific lyceums rank second (value=0.268), albeit close to linguistic and human science lyceums (value=0.217), both of which achieved a better performance with regards to the percentage of enrolled students than other hs in the region (0.437 and 0.396, respectively). 2 acroyms: high school leaving score: hs-ls; proportion of academic enrolled students: %aest; relative difference (compared to the regional average) of proportion of academic enrolled students: reldiff-%aest; invalsi test score: invalsi tsc; percentage of students passing the first year: %stpfy; relative difference (compared to the regional average) of percentage of students passing the first year: reldiff%stpfy; percentage of academic credits at the end of first year: %acrfy; exams' average score: e'sasc; students per class: st/class; student per teacher: st/teach; percentages of teachers with more than 6 years of service in the school: %teach6+. choosing the type of high school providing the best performance school performances classic lyceum technical hs hs-ls modello ahp other lyceum academic performances goal criteria sub criteria alternatives scientific lyceum school characteristics %aest reldiff %aest invalsi tsc %stpfy reldiff %aest %acrf y e'sasc ore st/class st/teach %teach 6+ ijahp article: mancini, marcarelli/high school choice: do parents get an idea? international journal of the analytic hierarchy process 98 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.633 table 3 relative and global priorities of the alternatives, results obtained by applying the ahp model 1 c ri te ri a sub-criteria criteria weight subcriteria weight alternatives classic lyceum scientific lyceum other lyceum technical hs s c h o o l p e rf o rm a n c e s high school leaving score 0 .6 0 0 0,213 0.483 0.331 0.158 0.028 proportion of academic enrolled students 0,213 0.400 0.381 0.215 0.004 relative difference (compared to regional average) of academic enrolled students 0,059 0.136 0.437 0.396 0.031 invalsi's test score 0,115 0.328 0.361 0.206 0.106 s c h o o l c h a ra c te ri st ic s students per class 0 .4 0 0 0,160 0.520 0.095 0.119 0.266 students per teacher 0,160 0.140 0.116 0.125 0.619 percentages of teachers employed by the school for more than 6 years 0,080 0.125 0.190 0.641 0.045 final ranking 1 1 0.349 0.268 0.217 0.166 ijahp article: mancini, marcarelli/high school choice: do parents get an idea? international journal of the analytic hierarchy process 99 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.633 figure 2 relative and global ranking of school types for model 1 once the results were obtained, a sensitivity analysis was conducted in order to verify the robustness of the preference ranking among the alternatives with respect to changes in the criteria weights. the sensitivity analysis, according to both school characteristics and school performances, highlights that at first, classic lyceums are the best alternative if the weight assigned to school performance is greater than 0.2; in other words, if the weight of school characteristics is lower than 0.8 (see figure 3). indeed, in the 2d 3 plot in figure 4 the classic lyceum is in the upper right box. a) b) figure 3 ranking of school types with regard to the goal on varying the weight assigned to school performance and school characteristics (model 1) 3 the 2-dimensional plot simultaneously illustrates the weight of the alternatives with respect to 2 criteria. the best alternatives are in the upper box on the right; the worst alternatives are in the lower box on the left; the alternatives in the upper box on the left or in the lower box on the right highlight a conflict between the two criteria considered on the axis. ijahp article: mancini, marcarelli/high school choice: how do parents make their choice? international journal of the analytic hierarchy process 100 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.633 figure 4 school performances vs. school characteristics (model 1) the technical hs alternative is very sensitive to the weight system, becoming the best choice only if school characteristics are taken into account, in particular with regard to students per teacher (see figure a2 in appendix), and the worst for school performances (see figure a1 in appendix). in fact, in the 2d plot in figure 4 this type of hs is in the upper box on the left. furthermore, other lyceums always assume the same global score regardless of the weight system (close to 0.20-0.25), and are the third choice. indeed, figure 4 shows other lyceum type next to the barycentre. considering the sub criterion accounting for the length of service of the teachers, the relative ranking of the alternatives strictly depends on the weight assigned; other lyceum being the best choice in ensuring teaching continuity with increasing weight (figure a2 in appendix). when entering the academic performances (model 2) into the model, the ranking of preference changes in favor of the scientific lyceum, whose value is slightly greater than the classic lyceums, and decreases the values of other lyceums and technical high schools (table 4 and figure 5). this is due to the excellent academic performance of scientific school-leavers, mostly in terms of the percentage of credits at the end of the first year and average exam score. the sensitivity analysis obviously confirms the results of model 1 with regard to school characteristics and performance, and highlights that regardless of the weight system scientific lyceums are always the better choice over classical ones for both academic performance criterion (figure 6) and for each sub-criterion associated with it (figure a3 in appendix). figure 7 confirms the above results by comparing the performance of the 4 school types when simultaneously considering 2 criteria: school performance vs. academic performance (fig. 7a) and school characteristics vs academic performance (figure 7b). ijahp article: mancini, marcarelli/high school choice: do parents get an idea? international journal of the analytic hierarchy process 101 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.633 table 4 relative and global priorities of the alternatives, obtained by applying the ahp model 2 c ri te ri a sub-criteria criteria weight subcriteria weight alternatives classic lyceum scientific lyceum other lyceum technical hs s c h o o l p e rf o rm a n c e s high school leaving score 0 .2 8 5 0.101 0.483 0.331 0.158 0.028 proportion of academic enrolled students 0.101 0.396 0.387 0.213 0.004 relative difference (compared to regional average) of academic enrolled students 0.028 0.136 0.437 0.396 0.031 invalsi's test score 0.054 0.328 0.361 0.206 0.106 a c a d e m ic p e rf o rm a n c e percentage of students passing the first year 0 .4 9 8 0.149 0.379 0.402 0.198 0.021 relative difference of students passing the first year 0.050 0.027 0.281 0.074 0.618 percentage of academic credits at the end of first year 0.149 0.397 0.521 0.067 0.015 average exam score 0.149 0.355 0.424 0.066 0.155 s c h o o l c h a ra c te ri st ic s students per class 0 .2 1 7 0.087 0.520 0.098 0.116 0.266 students per teacher 0.087 0.163 0.153 0.174 0.510 percentage of teachers employed in the school for more than 6 years 0.043 0.125 0.19 0.641 0.045 final ranking 1 1 0.346 0.350 0.166 0.138 ijahp article: mancini, marcarelli/high school choice: how do parents make their choice? international journal of the analytic hierarchy process 102 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.633 figure 5 relative and global ranking of school types for model 2 the results obtained by applying the above models show that, with regard to academic and scholastic performances of the high school students, the classic and scientific lyceums are preferred to other lyceums and technical schools. the ranking depends on the weight that parents assign to these criteria. if parents base the choice only on school performance and on the school characteristics, then they should favour a classic lyceum; if instead they take into consideration academic performance then the choice should be a scientific lyceum. a) b) c) figure 6 ranking of school types with regard to the goal on varying the weight assigned to school performance, school characteristics, and academic performance (model 2) ijahp article: mancini, marcarelli/high school choice: how do parents make their choice? international journal of the analytic hierarchy process 103 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.633 a) b) figure 7 school performance vs. academic performance and school characteristics vs. academic performance (model 2) 4. concluding remarks this paper deals with the problem of choosing a high school in italy. in order to assist parents with this multicriteria choice, we proposed the application of the ahp. we considered 2 models. one model represented the problem by using only 2 criteria, school characteristics and students’ school performance, and the second took into account academic student performance. relative priority vectors represent the weights of each hierarchical element; that is, the importance that parents devote to each criterion and subcriterion. the global priority vector provides the ranking of preferences between 4 types of high schools. the results obtained by our analysis demonstrate that there are 2 separate types of schools. on one side there are the classic and the scientific lyceums, and on the other side there are lyceums and technical schools, in particular with reference to the academic and scholastic performances of the students. furthermore, other things, not considered in our analysis being equal (such as income, distance from home and attitude towards certain disciplines), if parents base the choice of high school only on academic performance and on the characteristics of the school they should favor a classical lyceum. if instead they take into consideration academic performance, as eduscopio suggests, then the choice should be a scientific lyceum. finally, the sensitivity analysis highlights that the ranking of preferences between classic and scientific lyceums largely depends on the importance (weight) assigned to each criterion considered in the choice problem. due to the specificity of the italian secondary education system (mainly based on public high schools), the results of our analysis (a pilot study) cannot be extended to some international contexts in which private schools are more widespread for example, the us and uk. as underlined by many authors, in italy there are differences across the geographical areas, so it may be interesting to extend our study to other clusters of italian towns possessing the same characteristics but located in a different geographical area (agasisti et al., 2012; masci et al., 2016; 2018). if there was more data, further analysis could be carried out using the dematel (decision making trial and evaluation laboratory) ijahp article: mancini, marcarelli/high school choice: how do parents make their choice? international journal of the analytic hierarchy process 104 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.633 method, in order to study the interrelationships between the criteria and also among their respective sub-criteria (mumtaz et al., 2018). ijahp article: mancini, marcarelli/high school choice: how do parents make their choice? international journal of the analytic hierarchy process 105 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.633 references agasisti t., bonomi f., sibiano p. (2014). measuring the “managerial” efficiency of public schools: a case study in italy. international journal of educational management, 28(2), 120–140. doi: https://doi.org/10.1108/ijem-02-2013-0032. agasisti t., vittadini g. (2012). regional economic disparities as determinants of students' achievement in italy. research in applied economics, 4(2), 33–54. doi: https://doi.org/10.5296/rae.v4i2.1316 
 agasisti t., murtinu s. (2012). perceived competition and performance in italian secondary school: new evidence from oecd-pisa 2006. british educational research journal, 38(5), 841–858. doi: https://doi.org/10.1080/01411926.2011.588314 anderson, k. j., & minke, k. m. (2007). parent involvement in education: toward an understanding of parents' decision making. the journal of educational research, 100(5), 311-323. doi: https://doi.org/10.3200/joer.100.5.311-323 bauch, p. a., & goldring, e. b. (1996). parent involvement and teacher decision making in urban high schools of choice. urban education, 31(4), 403-431. doi: https://doi.org/10.1177/004208599603100404 bordignon m., carapella p. and turati g. (2017). eduscopio: le determinanti dell'accesso all'informazione, programma education fga working paper, n.56, fondazione giovanni agnelli. burgess s., greaves e., vignoles a. and wilson d. (2014). what parents want: school preferences and school choice. the economic journal, 125, 1262–1289. doi: https://doi.org/10.1111/ecoj.12153 giambona f. and porcu m. (2018). school size and students’ achievement. empirical evidences from pisa survey data. socio-economic planning sciences, 64(c), 66-77. doi: https://doi.org/10.1016/j.seps.2017.12.007 goldhaber, d.d. (1999). school choice: an examination of the empirical evidence on achievement, parental decision making, and equity. educational researcher, 28(9), 1625. doi: https://doi.org/10.3102/0013189x028009016 hanushek, e.a., kain, j. f., rivkin, s. g., & branch, g. f. (2007). charter school quality and parental decision making with school choice. journal of public economics, 91(5-6), 823-848. doi: https://doi.org/10.1016/j.jpubeco.2006.09.014 lauer c. (2003). family background, cohort and education: a french-german comparison based on a multivariate ordered probit model of educational attainment. labour economics, 10, 231–251. doi: https://doi.org/10.1016/s0927-5371(03)00007-1 masci c., witte k. d., agasisti t. (2018). the influence of school size, principal characteristics and school management practices on educational performance: an efficiency analysis of italian students attending middle schools. socio-economic https://doi.org/10.1108/ijem-02-2013-0032 https://doi.org/10.1016/j.seps.2017.12.007 ijahp article: mancini, marcarelli/high school choice: how do parents make their choice? international journal of the analytic hierarchy process 106 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.633 planning sciences, 61, 52–69. doi: https://doi.org/10.1016/j.seps.2016.09.009 masci c, ieva f, agasisti t, paganoni am. (2016). bivariate multilevel models for the analysis of mathematics and reading pupils’ achievements. journal of applied statistics, 44(7), 1296-1317. doi: http://dx.doi.org/10.1080/02664763.2016.1201799. mumtaz u., ali y., petrillo a., fabio de felice f. (2018). identifying the critical factors of green supply chain management: environmental benefits in pakistan, science of the total environment, 640-641, 144-152. doi: https://doi.org/10.1016/j.scitotenv.2018.05.231 pandolfini v. (2013). public or private education? parents' choice between actual and potential pluralism, italian journal of sociology of education, 5(2), 189-211. saaty t.l. and vargas l.g. (1982). the logic of priorities: applications in business, energy, health, and transportation. boston: kluwer-nijhoff. reprinted in paperback. (1991). pittsburgh, pa: rws publications. shneider b. and coleman j.s. (2018). parents, their children and schools. new york: routledge, new york. seitsinger, a. m. (2019). examining the effect of family engagement on middle and high school students’ academic achievement and adjustment. the wiley handbook of family, school, and community relationships in education. john wiley and sons inc. doi: https://doi.org/10.1002/9781119083054.ch8 seitsinger a. and brand s. (2011). the effect of home-school-community partnership on us high school students: a large scale examination of the effects of family engagement in education on us high school students’ academic adjustment and achievement, proceedings of 8th ernape conference, milan. doi: https://doi.org/10.1002/9781119083054.ch8 taylor, c. (2018). geography of the 'new' education market: secondary school choice in england and wales. new york: routledge. saaty t.l. and vargas l.g. (1982). the logic of priorities: applications in business, energy, health, and transportation. boston: kluwer-nijhoff. reprinted in paperback. (1991). pittsburgh, pa: rws publications. wespieser k., durbin b, and sims d. (2015). school choice: the parent view. slough, uk: national foundation for educational research. https://doi.org/10.1016/j.scitotenv.2018.05.231 ijahp article: mancini, marcarelli/high school choice: how do parents make their choice? international journal of the analytic hierarchy process 107 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.633 appendix a) b) c) d) figure a1 relative ranking of school types with regard to school performance on varying the weight assigned to high school leaving score, percentage of academic enrolled students, relative difference of academic enrolled students and invalsi test score (model 1) ijahp article: mancini, marcarelli/high school choice: how do parents make their choice? international journal of the analytic hierarchy process 108 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.633 a) b) c) figure a2 relative ranking of school types with regard to school characteristics on varying the weight assigned to student per class, student per teacher and the percentage of teachers employed in the school for more than 6 years (model 1) ijahp article: mancini, marcarelli/high school choice: how do parents make their choice? international journal of the analytic hierarchy process 109 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.633 a) b) c) d) figure a3 relative ranking of school types with regard to academic performance on varying the weight assigned to percentage of students passing the first year, relative difference of students passing the first year, percentage of academic credits at the end of first year and average exam score (model 2) ijahparticle: miri, omidvari, sadeghi, haleh/developing anp to rank the branches of an insurance company based on servqual international journal of the analytic hierarchy process issn 1936-6744 58 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.218 developing anp to rank the branches of an insurance company based on servqual mohsen miri graduate student, industrial and mechanical engineering faculty islamic azad university, qazvin branch, qazvin, iran e-mail: mohsen_miri54@yahoo.com manouchehr omidvari professor assistant, industrial and mechanical engineering faculty, islamic azad university, qazvin branch, qazvin, iran e-mail: omidvari88@yahoo.com ahmad sadeghi professor assistant, industrial and mechanical engineering faculty, islamic azad university, qazvin branch, qazvin, iran e-mail: a.sadeghi@qiau.ac.ir hasan haleh professor assistant, industrial and mechanical engineering faculty, islamic azad university, qazvin branch, qazvin, iran e-mail: hhaleh@qiau.ac.ir abstract the purpose of this paper is to show how the servqual scale and anp model can be used to effectively and scientifically rank strategies within the insurance service quality sector, instead of traditional models. this approach is used to show how the ranking decision changes as a function of service quality utilizing the anp model. this study evaluated the quality of services of the dana insurance company branches by measuring policy holders’ views toward the current and the expected levels of quality. the standard questionnaire “servqual”, which emphasizes measuring the gap between the level of current and expected quality, was used. the proposed algorithm utilized the analytic network process (anp), which allows measurement of the dependency among the quality dimensions, to rank the branches. the results showed a difference between policy holders’ expectations and the current level of quality of services in all dimensions of service quality. according to the anp limit super matrix, there was a significant difference between the proposed ranking approach and the traditional one. keywords: anp, servqual, service industry. mailto:mohsen_miri54@yahoo.com mailto:omidvari88@yahoo.com mailto:a.sadeghi@qiau.ac.ir mailto:hhaleh@qiau.ac.ir rob typewritten text http://dx.doi.org/10.13033/ijahp.v6i1.218 ijahparticle: miri, omidvari, sadeghi, haleh/developing anp to rank the branches of an insurance company based on servqual international journal of the analytic hierarchy process issn 1936-6744 59 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.218 1. introduction rational decision-making is a talent that must be encouraged if we want to be more effective in implementing our ideas in the real world with its risks and resistance to change. there are two types of decisions, normative and descriptive. the first, normative decision making, determines what we prefer the most, and in these circumstances it is easy to see why one does not wish anything to happen that could undermine the best choice that is made. the second, descriptive decision making, is used to make the best choice given all the influences in the world around us that could affect the optimality of any choice that is made (whitaker, 2007). in reality the quality of any choice that is made depends on how well we know our alternatives as compared with each other and with others outside the collection in order that the alternatives may be ranked. decision making involves prioritizing ideas according to current or future circumstances. a fundamental problem in service quality lies in one’s ability to measure its intangible criteria and interpret these criteria sensibly. the anp is fundamentally a way to measure intangibles factors by using pair wise comparisons with judgments that represent the dominance of one element over another with respect to a property that they share (saaty & khouja, 1976). the anp has been found useful in decision making which involves numerous intangible criteria such as insurance industry service quality. it is a process of laying out a structure of all the essential factors that influence the outcome of a decision. then numerical pair wise comparison judgments are elicited to express people’s understanding of the importance, performance or likely influence of these elements on the final outcome. the final outcome is obtained by synthesizing the priorities derived from different sets of pair wise comparisons, and finally a sensitivity analysis is performed to determine the stability of the outcomes to wide perturbations in the judgments (saaty, 1980). undoubtedly all organizations seek to attain increased service quality; however this issue is of greater importance in service organizations because their share of the macro economy is about seventy percent (foroughi et al., 2011). service quality is increasingly becoming a critical determinant of business performance and a strategic tool for gaining a competitive advantage, so measuring its level has been a matter of grave concern for both practitioners and researchers during the past two decades (gale et al. 1994). notwithstanding, the most popular measure of service quality is servqual, which was developed by parasuraman in 1988. (parasuraman et al. 1988) a number of applications of servqual have been reported in a variety of settings (ladhari, 2009). the servqual instrument is comprised of five dimensions with 22 items. analysis of the data can take several forms such as item-by-item analysis, dimension-by-dimension analysis and computation of the single measure of overall service quality (cooper et al. 2000). ijahparticle: miri, omidvari, sadeghi, haleh/developing anp to rank the branches of an insurance company based on servqual international journal of the analytic hierarchy process issn 1936-6744 60 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.218 services play an increasingly important role in the economy of the islamic republic of iran (i.r.i) and the global economy. services have in fact become more important than goods, for example, services produced by iranian insurance companies accounted for 1% of the gross domestic product (gdp) in 2011, and are projected increase to 1.45% by 2015 (bmi, 2011). services differ from goods in several important ways. for example, while goods are tangible and can be stored, services are often intangible and must be produced and consumed simultaneously (capar & kotabe, 2003). to the best of our knowledge, studies utilizing anp to rank insurance branches based on their service quality are lacking. to fill this gap, we scientifically ranked the branches of firms operating in the financial service industry, specifically the insurance industry, utilizing the anp. these branches were ranked based on their service quality level. our decision to study the insurance industry is motivated by several factors. first, the insurance industry has experienced an acceptable volume of domestic direct and indirect investment in recent years in iran mainly due to new technological advancements and governmental liberalization policies which have created many opportunities for private insurance companies to become active in the financial market. this has made the insurance industry one of the largest service industries in iran. second, since insurance policy holders develop a long-term relationship with their insurance company by paying a premium over a long period of time, and are likely to have a specific interest in the performance of their insurers (parente, 2010). these reasons along with the lack of international management research on insurance companies and their branches, especially their ranking methods, make the insurance industry interesting to study. 2. servqual several definitions of service quality exist in the research literature. some define the quality of the perceived service as the result of the assessment of the clients’ expectations and the perceived services. service quality is a stable criterion that indicates how the presented services correspond with the clients expectations. others define service quality as the presentation of services in a way that exceeds the client’s expectations (gaglianoo & hathcot, 1994, jabonoun & al tamimi, 2003). the most comprehensive and widely accepted definition belongs to parasuraman et al (1985). according to this definition, service quality is related to satisfaction, but not equal to that. it is attained by measuring the difference between clients’ expectations and their perceptions of the service. parasuraman et al (1985, 1991, 1994) identified 10 dimensions in their studies for the identification of service quality: facilities, reliability, responsibility, communication, credit, security, qualification, politeness, understanding of the client, and availability. later, they summarized these into five dimensions. servqual can also be defined as a multiple-item scale composed of five dimensions and 22 items for measuring consumer perceptions of service quality (hakyeon le, 2012). table 1 presents the five dimensions of servqual. this survey instrument includes 22 items for measuring expectations (e), and the corresponding 22 items for measuring perceptions (p) of the clients (policy holders). a seven point likert scale from “strongly disagree (1) to strongly agree (7)” can be used for measurement. for each item, a difference score gap (g) is obtained by ijahparticle: miri, omidvari, sadeghi, haleh/developing anp to rank the branches of an insurance company based on servqual international journal of the analytic hierarchy process issn 1936-6744 61 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.218 determining the difference between the ratings for perception (p) and expectation (e) that is; g = p –e (parasuraman et al. 1994). table1 dimensions of servqual (parasuraman et al., 1994) dimensions definition number of item tangible physical facilities, equipment, appearance of personnel and organization accommodations 4 reliability ability to perform the promised service dependably, accurately and precisely 5 responsiveness willingness to help customers and provide prompt service, and disposition to quickly serve the clients 4 assurance/ guarantee knowledge and courtesy of employees and their ability to inspire trust and confidence 4 empathy/ sympathy caring, individualized attention the firm provides to its customers / personal attention to each client 5 3. literature review the anp is a mathematical theory that can deal with all kinds of dependence systematically, and has been successfully applied in many fields (chung et al., 2005). anp has a systematic approach to set priorities and trade-offs among goals and criteria, and can also measure all tangible and intangible criteria. many decision problems cannot be structured hierarchically because they involve the interaction and dependence of higher-level elements with lower-level elements. not only does the importance of the criteria determine the importance of the alternatives as in a hierarchy, but also the importance of the alternatives themselves determines the importance of the criteria. also feedback enables us to determine what must be done to attain a desired future. the analytic network process (anp) is a generalization of the analytic hierarchy process (ahp), and its’ feedback structure looks more like a network, with cycles connecting components of elements with loops that connect a component to it (figueira et al., 2005). multiple criteria decision making (mcdm) methods are based on the additive concept along with the independence assumption, but an individual criterion is not always completely independent for solving the interactions among elements. the anp, a relatively new mcdm method, was proposed by saaty (1996) and its greatest advantage is that it can handle intangible factors based on individual or collective judgments of the situation (saaty & vargas, 2006). therefore, based on these advantages, the anp was used for modeling and comparing in this research. this study involves numbers of pair wise comparisons for deriving the priorities of branches of insurance companies’ evaluation and ranking. it synthesizes experts’ opinions to determine compliance with the geometric mean method (buckley, 1985). the valuation scales used in the study are those recommended by saaty (1980, 1996, 2005) where 1 represents equal importance, 3 moderate importance, 5 strong importance, 7 very ijahparticle: miri, omidvari, sadeghi, haleh/developing anp to rank the branches of an insurance company based on servqual international journal of the analytic hierarchy process issn 1936-6744 62 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.218 strong or demonstrated importance, and 9 represents extreme importance. even numbered values will fall in between importance levels. reciprocal values (e.g. 1/3, 1/5, etc.) mean less importance, even less importance, etc. saaty (1980) proved that for the consistent reciprocal matrix, the λ max value is equal to the number of comparisons, or λ max = n. a measure of consistency, called the consistency index, is a deviation or degree of consistency using the following formula. if the value of i.i. ratio [i.i. = (λ max − n)/ (n−1)] is smaller or equal to 10%, the inconsistency is acceptable. if the i.i. ratio is greater than 10%, the subjective judgment needs to be revised. n in the mentioned formula denotes the number of elements that have been compared. when λ max = 0, the complete consistency exists within judgment procedures and then λ max = n. the consistency ratio (i.r.) of i.i. to the mean random consistency index (i.i.r) is expressed as i.r. (i.r. = i.i. / i.i.r) less than 0.1. by using the process above, an un-weighted supermatrix is composed. its columns contain the priorities derived from the pairwise comparisons of the elements. in an un-weighted supermatrix, the columns may not be column stochastic to obtain a stochastic matrix, i.e., each column sums to one. the blocks of the un-weighted supermatrix should be multiplied by the corresponding cluster priority. to derive the overall priorities of elements, this method involves multiplying sub matrices numerous times in turn until the columns stabilize and become identical in each block of sub-matrices. the weighted supermatrix can then be raised to limiting powers to calculate the overall priority weights. the anp employs the limiting process method lim k→∞ wk of the powers of the supermatrix (saaty, 1996, meade & sarkis, 1998, sekitani & takahashi, 2001, tseng et al., 2008). for synthesizing overall priorities for the alternatives, the un-weighted supermatrix requires adjusting in order to keep it column stochastic (sarkis 1999). the strength of the anp lies in its use of ratio scales to capture all kinds of interactions, formulate accurate predictions, and make better decisions (saaty, 2003). the traditional annual ranking model considers only the annual sale (portfolio) of a branch, and this method could unfortunately lead to much dishonesty. falsification of accounts and quality level ignorance in branches are the greatest deficiencies of this method as opposed to our proposed algorithm servqual utilizing anp. the proposed algorithm would ensure the elimination of account falsification and the establishment of quality management. this is true because of the mathematical approaches property. 4. research method in this research, we aimed to clarify the current state of the service quality level of the insurance service industry in iran. we used five servqual dimensions as the research criteria provided by the branches as the research alternatives in both underwriting and loss adjusting sections in order to rank them by anp. figure 1 shows the proposed algorithm using the anp in order to determine the overall priorities of the criteria identified with the servqual analysis in order to rank the alternatives. ijahparticle: miri, omidvari, sadeghi, haleh/developing anp to rank the branches of an insurance company based on servqual international journal of the analytic hierarchy process issn 1936-6744 63 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.218 figure1. algorithm of branches ranking by anp based on service quality the research method was a descriptive-survey which was selected on the basis of the nature of this research. the population of this research included all the policy holders who were living in zone 1(this zone contains the policy holders who are in tehran and will be described later in this paper) from 2012 to 2013. 376 policy holders were sampled based on the volume sample assessment formula as suggested by parasuraman (1985) and foroughi abari (2011). since the population of the research was all policy holders in tehran, access to all of them was time consuming and costly, so the volume sample assessment “formula no. 1” was adopted to estimate the number of sample elements. the instrument was the standard servqual-parasuraman questionnaire which was designed on the basis of a likert seven point scale and distributed among the participants (parasuraman et al. 1994). to analyze the data, spss software was used at the two levels of descriptive and analytical statistics. at the level of descriptive statistics, frequency, percentage, mean and standard deviation were used, and at the level of analytical statistics dependent t-tests were used to investigate the policy holders’ opinions. the number of sample elements was calculated according to the following formula: (lapin, 1990). 2 22 d z n   (1) d=desired precision (or maximum error) 2 =assumed population variance 2/αz =critical normal deviate for specified reliability1-α identify branches (alternatives) by cluster sampling identify servqual criteria and determine their weights by questionnaire determine with 1-9 scale the inter dependence matrix of each servqual factor with respect to other factor determine the importance degrees of the servqul factors determine the importance degrees of the alternatives with respect to each servqual factor with a 1–9 scale determine the overall priorities of the alternatives &criteria ranking the branches based on the results of supermatrix ijahparticle: miri, omidvari, sadeghi, haleh/developing anp to rank the branches of an insurance company based on servqual international journal of the analytic hierarchy process issn 1936-6744 64 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.218 to calculate the variance and reliability 40 questionnaires were distributed among policy holders. the variance was equal to 0.5789, and previous studies indicated a desired precision of d=0.077 with a reliability probability of 1-α=.95. from the critical normal deviate values table we know that z0.025=1.96, thus the required sample size was 376 (rounded). the reliability was estimated via cronbach's alpha (perceptions 92% and expectations 90%) by spss software which was more than the acceptable percentage (70%), so the reliability of the questionnaire was accepted. figure 2 shows the network on which the algorithm was proposed. this network was obtained from a group of experts consisting of three assistant professors at azad islamic university and two top managers of the dana insurance company. it was based on the assumption that each branch as an element of the cluster should be compared with respect to its importance with the five servqual criteria and vice versa. it contains 4 alternatives (four branches) and five criteria (servqual factors), and the goal of the model was to find the superior branch according to policy holders’ opinions. goal:superior branch tangibles c5 reliability –c4 vahdat a1 assurance c1 responsiveness c2 empathy c3 7 tir branch a3 azadi a2 sadeghiyeh a4 figure 2. network used to identify the superior branch of an insurance company based on servqual the anp is composed of four major steps (saaty, 2003): step 1: model construction and problem structuring: the problem should be stated clearly and be decomposed into a rational system, like a network. this network structure can be obtained by decision-makers through brainstorming or other appropriate methods. step 2: pairwise comparison matrices and priority vectors: similar to the comparisons performed in ahp, pairs of decision elements at each cluster are compared with respect to their importance towards their control criteria. the clusters themselves are also compared with respect to their contribution to the objective. decision-makers are asked to respond to a series of pairwise comparisons of two elements or two clusters to be evaluated in terms of their contribution to their particular upper level criteria. in addition, interdependencies among elements of a cluster must also be examined pairwise; the influence of each element on other elements can be represented by an eigenvector. the relative importance values are determined with saaty’s 1–9 scale (table 2), where a score ijahparticle: miri, omidvari, sadeghi, haleh/developing anp to rank the branches of an insurance company based on servqual international journal of the analytic hierarchy process issn 1936-6744 65 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.218 of 1 represents equal importance between the two elements and a score of 9 indicates the extreme importance of one element (row cluster in the matrix) compared to the other one (column cluster in the matrix) (meade,1999). a reciprocal value is assigned to the inverse comparison, that is, aij ¼ 1=aji, where aij (aji) denotes the importance of the ith (jth) element. pairwise comparison is performed in the framework of a matrix, and a local priority vector can be derived as an estimate of the relative importance associated with the elements (or clusters) being compared by solving the following equation: wwa max . where a is the matrix of pair-wise comparison, w is the eigenvector, and λ max is the largest eigenvector value. saaty (1980) proposes several algorithms to approximatew . in this paper, super decisions software is used to compute the eigenvectors from the pair-wise comparison matrices and to determine the consistency ratios. table 2 saaty’s 1–9 scale for ahp & anp preference (saaty, 1996) intensity of importance definition 1 equal importance 3 moderate importance 5 strong importance 7 very strong importance 9 absolute importance 2,4,6,8 intermediate importance reciprocal of above non-zero numbers if activity i has one of the above non-zero numbers assigned to it when compared with activity j, then j has the reciprocal value when compared with i each matrix should be normalized by the following formula:    m i ij ij ij a a a 1 1 1 (2) the consistency index that was chosen is as below: (figueira et al., 2005) 1   n n ii max  .. (3) and the rate of inconsistency was calculated according to the following formula: rii ii ri .. . ..  (4) ijahparticle: miri, omidvari, sadeghi, haleh/developing anp to rank the branches of an insurance company based on servqual international journal of the analytic hierarchy process issn 1936-6744 66 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.218 table 3 shows where i.i.r random index is chosen from random index table. table 3 random index table (figueira et al., 2005) 9 8 7 6 5 4 3 2 1 order 1.45 1.41 1.32 1.24 1.12 0.9 0.58 0 0 i.i.r step 3: supermatrix formation: the supermatrix concept is similar to the markov chain process (saaty, 1996). to obtain global priorities in a system with interdependent influences, the local priority vectors are entered in the appropriate columns of a matrix. as a result, a supermatrix is actually a partitioned matrix, where each matrix segment represents a relationship between two clusters in a system. the local priority vectors obtained in step 2 are grouped and placed in the appropriate positions in a supermatrix based on the flow of influence from one cluster to another, or from a cluster to itself, as in the loop. a standard form for a supermatrix is as shown below. note that any zero value in the supermatrix can be replaced by a matrix if there is an interrelationship of the elements within a cluster or between two clusters. since there is usually interdependence among clusters in a network, the columns of a supermatrix may sum to more than one. however, the supermatrix must be modified so that each column of the matrix sums to unity. an approach recommended by saaty (1996) involves determining the relative importance of the clusters in the supermatrix, using the column cluster as the controlling cluster. that is, row clusters with non-zero entries in a given column cluster are compared according to their impact on the cluster of that column cluster. an eigenvector is obtained from the pairwise comparison matrix of the row clusters with respect to the column cluster, which in turn yields an eigenvector for each column cluster. the first entry of the respective eigenvector for each column cluster is multiplied by all the elements in the first cluster of that column; the second by all the elements in the second cluster of that column and so on. in this way, the cluster in each column of the supermatrix is weighted, and the result, known as the weighted supermatrix, is stochastic. raising a matrix to exponential powers gives the long-term relative influences of the elements on each other. to achieve convergence on the importance weights, the weighted supermatrix is raised to the power of 2k + 1, (w=lim w 2k+1 ) where k is an arbitrarily large number; the new matrix is called the limit supermatrix (saaty, 1996). the limit supermatrix has the same form as the weighted supermatrix, but all the columns of the limit supermatrix are the same. the final priorities of all elements in the matrix can be obtained by normalizing each cluster of this supermatrix. additionally, the final priorities can be calculated using matrix operations, especially where the number of elements in the model is relatively few. matrix operations              nn kn n nk kk k n k ij w w w w w w w w w w 11 1 2 11 ..... ..... ..... ijahparticle: miri, omidvari, sadeghi, haleh/developing anp to rank the branches of an insurance company based on servqual international journal of the analytic hierarchy process issn 1936-6744 67 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.218 are used in order to easily convey the steps of the methodology and how the dependencies are worked out. step 4: selection of the best alternatives: if the supermatrix formed in step 3 covers the whole network, the priority weights of the alternatives can be found in the column of alternatives in the normalized supermatrix. on the other hand, if a supermatrix only comprises clusters that are interrelated additional calculations must be made to obtain the overall priorities of the alternatives. the alternative with the largest overall priority should be selected, as it is the best alternative as determined by the calculations made using matrix operations. 5. case study this section presents an illustration of the proposed approach summarized in the previous sections. in the following case study, the servqual method utilizing the anp analysis was performed on the dana insurance company. dana insurance company is one of the largest iranian insurance companies among the forty active insurance companies in the country, and controls one fifth of the total insurance market. it has more than 40 branches throughout the country which are divided in to 8 zones. zone 1 is in tehran and consists of 4 branches. despite the huge market share and portfolio of this firm, it uses the traditional ranking system to assess the efficiency of its branches. it has three types of branches (superior branch, level 1 branch and level 2 branch) whose portfolios are annually assessed by the traditional method (annual sale level), then ranked by the level of the annual sale level of the portfolio. this ineffective kind of assessment and ranking system causes many internal problems for branches such as falsification of accounts, property wastefulness, oppression of staff, and infringing on policy holders’ rights. many other damages to society and the economy result because of this traditional method. in this paper we want to eradicate these deficits by our proposed model which is a scientific approach utilizing anp. the data for the five dimensions of servqual for service quality units were generated through the standard servqual questionnaire for both perceptions and expectations in the four branches as our alternatives. these questionnaires were sent to policy holders via their e-mail, post address or in person when they visited the branches. then, anp was conducted with the data set of servqual. table 4 presents the generated data and results of the servqual questionnaires. the statistical analysis revealed that the mean for expectations was 22.056 and was 19.203 for perceptions. the test result was 23.22 at p = .001, so the data analysis of the policy holders’ expectations and perceptions in all dimensions of service quality showed that they were significant at p<0.05. there was a gap between their expectations and their perceptions which showed that the insurer could not satisfy the policy holders’ expectations. ijahparticle: miri, omidvari, sadeghi, haleh/developing anp to rank the branches of an insurance company based on servqual international journal of the analytic hierarchy process issn 1936-6744 68 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.218 table4 generated data of servqual questionnaires on insurance branches total c5 c4 c3 c2 c1 criteria alternatives 3.4312 0.5268 0.9488 0.7046 0.7475 0.5025 a1 3.627 0.60825 0.6928 0.6474 1.1375 0.541 a2 3.554 0.725 0.832 0.5666 0.7225 0.7075 a3 3.69 0.4 0.926 0.284 1.1925 0.8875 a4 5.1 anp application on servqual data set in this section, anp is applied to the data generated from the servqual questionnaires’ results and analyses which are summarized in table 4. first, each servqual criterion is compared with respect to alternatives (a), and then each alternative with respect to criteria (b). a: comparison of criteria with all alternatives: five service quality dimensions are compared with respect to four insurance branches. b: comparison of alternatives with all criteria: four insurance branches which are considered as alternatives are compared with respect to five servqual dimensions. the results of these comparisons are shown in table 5 as a super matrix and table 6 as the limit super matrix. 5-1-acriteria comparison with alternatives 5-1-a-1assurance criterion (c1) comparison with alternatives: 1.00017.0 9.0 0015.0 ..0015.0 3 40045.4 . 176. 223. 289. 312.                 riiiiw 5-1-a-2responsiveness criterion (c2) comparison with alternatives: 1.00055.0 9.0 0005.0 ..0005.0 3 40015.4 . 189. 312. 198. 301.               riiiiw ijahparticle: miri, omidvari, sadeghi, haleh/developing anp to rank the branches of an insurance company based on servqual international journal of the analytic hierarchy process issn 1936-6744 69 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.218 5-1-a-3empathy criterion (c3) comparison with alternatives: 1.00003.0 9.0 0025.0 ..0025.0 3 40075.4 . 427. 214. 187. 172.               riiiiw 5-1-a-4reliability criterion (c4) comparison with alternatives: 1.00000.0 9.0 0000.0 ..0000.0 3 40000.4 . 226. 251. 302. 221.               riiiiw 5-1-a-5tangibles criterion (c5) comparison with alternatives: 1.000046.0 9.0 00042.0 ..0042.0 3 400125.4 . 337. 186. 221. 256.               riiiiw 5-1-b-alternatives comparison with criteria: 5-1-b-1vahdat branch (a1) comparison with all servqual criteria: 1.000009.0 12.1 0001.0 ..0001.0 4 50004.5 . 154. 277. 205. 218. 146.                 riiiiw 5-1-b-2azadi branch (a2) comparison with all servqual criteria: 1.001.0 12.1 1125.0 ..1125.0 4 5045.5 . 168. 191. 179. 313. 149.                 riiiiw 5-1-b-3seven tir branch (a3) comparison with all servqual criteria: 1.00033.0 12.1 0365.0 ..00365.0 4 50146.5 . 204. 234. 159. 203. 199.                 riiiiw 5-1-b-4sadeghiyeh branch (a3) comparison with all servqual criteria: 1.000018.0 12.1 0002.0 ..0002.0 4 50008.5 . 108. 251. 077. 323. 241.                 riiiiw ijahparticle: miri, omidvari, sadeghi, haleh/developing anp to rank the branches of an insurance company based on servqual international journal of the analytic hierarchy process issn 1936-6744 70 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.218 table 5 the super matrix of insurance branches ranking                             0000337.0226.0427.0189.0176.0 0000186.0251.0214.0312.0223.0 0000221.0302.0187.0198.0289.0 0000256.0221.0172.0301.0312.0 108.0204.0168.0154.000000 251.0234.0191.0277.000000 077.0159.0179.0205.000000 323.0203.0313.0218.000000 241.0199.0149.0146.000000 4 3 2 1 5 4 3 2 1 432154321 a a a a c c c c c aaaaccccc table 6 the limit super matrix of insurance branches ranking                              00002556396.00255504.25575823.0255416433.0255427991.0 00002456764.0245676.0245625035.0245688855.0245708829.0 00002414222.0241436.0241387689.0241424701.0241457809.0 00002566763.0256714.0256607967.0256736739.0256763133.0 177818801.0177743204.0177891411.0177914046.000000 238948024.0238707585.0238939172.0238944804.000000 174540533.0174518175.0174663932.0174698925.000000 222245302.0222043687.0222833677.0222924826.000000 184074242.0183801168.0183985235.0183964317.000000 4 3 2 1 5 4 3 2 1 5 432154321 a a a a c c c c c w aaaaccccc 6. results the aim of this research was to assess the service quality of the branches of insurance companies by applying the servqual model and combining the results with anp in order to propose a scientific ranking model. the participants included policy holders who were randomly selected by cluster sampling method, and the instrument used was the standard servqual questionnaire with five dimensions (assurance, empathy, tangibles, responsiveness and reliability). the reliability of the instrument was measured by cronbach's alpha (perceptions 0.92 and expectations 0.90). data analysis was conducted and revealed that there was a gap in the policy holders’ expectations ( 22.056) and perceptions ( 19.203) which was significant at p<0.05. results showed that alternatives ranked in this order: 1-vahdat, 2-sadeghiyeh, 3-seven tir, 4azadi and were not in accordance with the traditional assessment ranking system where they were ranked as: 1-azadi, 2seven tir, 3-sadeghiyeh, 4-vahdat. figure 3 shows the traditional assessment of the portfolios, and table 7 shows the priorities of criteria. table 8 shows the ranking of alternatives based on the proposed model which proved that there was a significant difference between the traditional ranking model and the engineering proposed model. ijahparticle: miri, omidvari, sadeghi, haleh/developing anp to rank the branches of an insurance company based on servqual international journal of the analytic hierarchy process issn 1936-6744 71 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.218 figure 3. traditional assessment and ranking system of insurance branches (portfolio billion of rials) table 7 prioritization of criteria (servqual dimensions) by the proposed engineering model from the limit super matrix ranking element 1 c4 2 c2 3 c1 4 c5 5 c3 table 8 ranking of alternatives (branches) by the proposed engineering model from the limit super matrix ranking element 1 a1 2 a4 3 a3 4 a2 7. conclusion successful management starts with a proper assessment system that is chosen through a robust evaluation method. akhlaghi et al (2012) assessed the quality of educational services in ahvaz technical college via the servqual model. also, hakyeon et al (2012) proposed a data envelopment analysis (dea) approach to computation of a measure of overall service quality and benchmarking when measuring service quality with servqual. foroughi et al (2011) evaluated the quality of services at a postgraduate school in a non-governmental university. students’ views toward the current and expected levels of quality via servqual scale were measured at the islamic azad university-khorasgan branch. ramon et al (2012) explored the problem of integrating semantically heterogeneous data (natural language included) from various websites with opinions about e-financial services. they developed an extension of the fuzzy model based on semantic translation (fmst) under the perspective of the service quality (servqual) stream of research. in 2011, yucenur et al proposed a model for selecting of the global supplier by the analytical hierarchy process (ahp) and the 170 144 95 16 0 50 100 150 200 a2 a3 a4 a1 ijahparticle: miri, omidvari, sadeghi, haleh/developing anp to rank the branches of an insurance company based on servqual international journal of the analytic hierarchy process issn 1936-6744 72 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.218 analytical network process (anp) based on linguistic variable weight and then fuzzy ahp and fuzzy anp results were compared. the anp is a relatively new mcdm method which can deal with many interactions systematically, unlike traditional mcdm methods which are based on the independence assumption. the anp can be used not only as a way to handle the inter-dependences within a set of criteria, but also as a way of producing more valuable information for decision-making. this paper proposes a solution based on combining anp and the servqual in a management assessment system. this approach helps the decisionmaking team to have a proper solution in the management and ranking system. the results of this study showed that there was a significant difference between the traditional assessment of branches and the mathematically proposed model. so, after conducting the servqual model, it was revealed that service quality functioning of branches in the five dimensions was slightly below the mean and that there was a gap between the perception of the present situation and expectations from the viewpoint of the policy holders. therefore, if the insurance companies want to be efficient and effective so as not to lose their market shares their annual assessment system should be converted to the scientific model. fuzzy numbers can be introduced in the anp method to more effectively analyze cases having greater uncertainty in the pair wise comparison matrices for future research. based on the general findings of the research, and in order to continuously improve the process of service quality, it is necessary to repeat this research every year to examine the changes in expectations and perceptions of the policy holders and to identify new needs and trends. benchmarking can be a useful tool in this industry when considering expectations and perceptions. finally, decision-makers should leave behind the traditional methods of assessment, and move to the scientific approach, such as the engineering model, which is proposed in this paper in order to effectively and efficiently cope with future. ijahparticle: miri, omidvari, sadeghi, haleh/developing anp to rank the branches of an insurance company based on servqual international journal of the analytic hierarchy process issn 1936-6744 73 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.218 references akhlaghi e., amini s., akhlaghi h.(2012). evaluating educational service quality in technical and vocational colleges using servqual model. procedia social and behavioral sciences, 46, 5285 – 5289. bmi annual report (2011). rate of insurance industry in iri gnp. journal of economics, 2324 (12). (in persian) buckley, j. j. (1985). fuzzy hierarchical analysis. fuzzy sets and systems, 17, 233–247. capar, n., kotabe, m., (2003).the relationship between international diversification and performance in service firms. journal of international business studies, 34, 345–355. chung s.h., lee a.h.l., pearn w.l., (2005). analytic network process (anp) approach for product mix planning in semiconductor fabricator. international journal of production economics, 96, 15–36. cooper ww, seiford lm, tone k. (2000). data envelopment analysis: theory, methodology, and applications, references and dea-solver software. boston: kluwer academic publishers. figueira j., greco s., ehrogtt m, (2005). multiple criteria decision analysis: state of the art surveys. boston, dordrecht, london: springer verlag. foroughi abari a.a, yarmohammadian m.h, esteki.m, (2011). assessment of quality of education a non-governmental university via servqual model. procedia social and behavioral sciences, 15, 2299–2304. gaglianoo, k. b. & hathcot, j. (1994). customer expectation and perceptions of service quality in apparel retailing .journal of service marketing, 8(1), 66-69. gale bt. (1994). managing customer value: creating quality and service that customers can see. new york: the free press. hakyeon leea, chulhyun kim, (2012). a dea-servqual approach to measurement and benchmarking of service quality. procedia social and behavioral sciences, 40, 756 – 76. jabonoun, n. & al tamimi, a. h. (2003). measuring perceived service quality at uae commercial banks. international journal of quality &reliability management, 20, 458472. kang, h. & bradley, g. (2002). measuring the performance of it services an assessment of servqual. international of accounting information system, 3(4), 151-164. ladhari r. (2009), a review of twenty years of servqual research. international journal of quality service science, 1, 172-198. ijahparticle: miri, omidvari, sadeghi, haleh/developing anp to rank the branches of an insurance company based on servqual international journal of the analytic hierarchy process issn 1936-6744 74 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.218 lapin l. (1990). probability and statistics for modern engineering. boston: pws-kent publishing company. lee j.w., kim s.h., (2000). using analytic network process and goal programming for interdependent information system project selection. computers and operations research, 27, 367–382. meade, l. m., & sarkis, j. (1998). strategic analysis of logistics and supply chain management systems using the analytic network process. logistics and transportation review, 34(2), 201–215. meade l.m., sarkis j., (1999). analyzing organizational project alternatives for agile manufacturing processes: an analytical network approach. international journal of production research, 37, 241–261. parasuraman a, zeithaml va, berry ll. (1988). servqual: a multiple item scale for measuring consumer perceptions of service quality. journal of retailing, 65, 12-40. parasuraman a, zeithaml va, berry ll. (1985). a conceptual model of service quality and its implications for future research. journal of marketing, 49, 41-50. parasuraman, a., zeithaml, v. & berry, l. l. (1991). refinement and reassessment of the servqual scale. journal of retailing, 67(4), 420450. parasuraman, a., zeithaml, v. & berry, l. l. (1994). alternative scale for measuring service quality: a comparative assessment based on psychometric and diagnostic criteria. journal of retailing, 70, 207-230. parente .r, byeongyong p.c, arjen h.l, slangen s.k, (2010). distribution system choice in a service industry: an analysis of international insurance firms operating in the united state. journal of international management, 16, 275–287. ramon a. c., munoz-l.f., sanchez .j.-fernandez b, francisco j. liebana c. (2012). a model for the integration of e-financial services questionnaires with servqual scales under fuzzy linguistic modeling. expert systems with applications, 39, 11535–11547. saaty, t.l., khouja, m.w., (1976). a measure of world influence. journal of peace science, (spring). saaty. t. l, (1980). the analytic hierarchy process. new york: mcgraw-hill. saaty, t. l. (1996). the analytic network process-decision making with dependence and feedback. pittsburgh, pa: rws publications. saaty, r. w. (2003). decision making in complex environment: the analytic hierarchy process (ahp) for decision making and the analytic network process (anp) for decision making with dependence and feedback. pittsburgh: super decisions. ijahparticle: miri, omidvari, sadeghi, haleh/developing anp to rank the branches of an insurance company based on servqual international journal of the analytic hierarchy process issn 1936-6744 75 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.218 saaty, t. l. (2005). theory and applications of the analytic network process: decision making with benefits, opportunities, costs and risks. pittsburgh: rws publications. saaty, t. l., & vargas, l. g. (2006). decision making with the analytic network process: economic, political, social and technological applications with benefits, opportunities, costs and risks. new york: springer science+ business media, llc. sarkis, j. (1999). a methodological framework for evaluating environmentally conscious manufacturing programs. computers & industrial engineering, 36(4), 793–810. sekitani, k., & takahashi, i. (2001). a unified model and analysis for ahp and anp. journal of the operations, research society of japan, 44(1), 67–89. tseng, m. l., lin, y. h., chiu, a. s. f., & liao. h.(2008). using fanp approach on selection of competitive priorities based on cleaner production implementation: a case study in pcb manufacturer, taiwan. clean technology and environmental policy, 10(1), 17–29. whitaker, r., (2007), validation examples of the analytic hierarchy process and analytic network process. mathematical and computer modeling, 46, 840–859. yucenur. g.n. & vayvay. o. & demirel n.ç, (2011). supplier selection problem in global supply chains by ahp and anp approaches under fuzzy environment. international journal of advanced manufacturing technology, 56, 823–833. words from the creator of the ahp words from the creator of the ahp let me take this occasion to wish this journal and its contributors my very best. it is a joyful occasion to have our own journal for decision making with the ahp/anp. we have a brilliant, dedicated and responsible editor-in-chief in enrique mu. we also have a sizeable community of scholars and users who can enjoy the diversity of theory, applications and practical ideas that will appear here and will advance the cause of decision making. i want to express my thanks to enrique and to senior editor luis vargas and to managing editor rozann saaty, my wife, and to all of you who actually do the work. i feel personally proud to have received the impact prize for “seminal work on the analytic hierarchy process, and for its deployment and extraordinary impact” from the informs society in 2008. the recognition that led to the award came from the many applications and involvement of other people which created the impact over the past 30 years. this field would not be where it is if it were not for the efforts of all of us. over the coming years i will be looking forward to enjoying reading the contributions from our growing community. we must not be too theoretical and lose touch with reality, nor too practical to lose the rigor and beauty of theory and inventive ideas. i wish every reader and every contributor the very best in creating this journal and contributing to its growth and widespread recognition. thomas l. saaty creator of the ahp http://dx.doi.org/10.13033/ijahp.v1i1.9 2_saaty_r_eigenvector_challenge_vol2_issue2_2010_p78 ijahp preface: saaty, rozann/introduction to the eigenvector challenge international journal of the 78 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 at the isahp 2009 meeting in pittsburgh, thomas l. saaty offered a challenge to the attendees to explain the eigenvector solution of the pairwise comparison judgment matrix that is the basis of the analytic hierarchy process so that it would be understandable to any intelligent layman. he wrote an essay himself on it during the meeting, and submitted it as his offering. a second essay on the subject, also during the meeting, was written by stan lipovetsky. these two papers appear in this issue of the journal. the competition remains open, and we, the editors of the ijahp, are soliciting more papers on the topic for publication in this journal. tom saaty has offered a prize for a truly outstanding paper. we remind you that three things count in a good paper: technical accuracy, originality of approach, and excellence in exposition, not necessarily in order of priority for this special topic. since this is an open-ended competition it is not quite sure when the decision will be made to award the prize; it depends on the quality of the papers submitted. with regards, rozann saaty ijahp managing editor rob typewritten text http://dx.doi.org/10.13033/ijahp.v2i2.86 rob typewritten text ijahp: mu/the ijahp road ahead international journal of the analytic hierarchy process 1 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.322 the ijahp road ahead april 15, 2015 welcome to our first issue in 2015! this first issue is being published earlier than our traditional mid-year deadline because we are increasing our publication frequency from twice a year to three times a year. more articles are being submitted to our journal and if we want to publish the selected ones on a timely basis, we need to increase our frequency. this suggests that our journal is becoming more mainstream in our seventh year of publication. as a way to thank our reviewers who have made this growth possible we have decided to recognize those reviewers who have been exemplary in terms of commitment and quality of their reviews during 2014-2015. the winners will be announced in our last issue this year. also, as you may have noticed from the new logos we proudly show on the cover page of this issue, we are now indexed by google scholar, microsoft academic search and the directory of open access journals (doaj). we are also applying to academic onefile (gale), ebsco, international bibliography of the social sciences (ibss), general onefile (gale) and copernicus. we expect to be indexed in most, if not all of these databases, within the next two years. in addition, as previously announced, we are using dois for all our published articles, ensuring that ijahp articles persist in the digital world. in summary, we intend to keep ijahp growing not only in number of articles but also in quality and recognition. however, this is only possible if we can count on your collaboration as reader, reviewer and author. thank you for your assistance, and we are looking forward to our seventh year of publication serving the ahp/anp community! very sincerely, enrique mu, phd ijahp editor-in-chief ijahp news and events: kahraman/infus conferences international journal of the analytic hierarchy process 1 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1034 infus conferences http://infus.itu.edu.tr infus (http://infus.itu.edu.tr) is an acronym for intelligent and fuzzy systems. it is an international research forum to advance the foundations and applications of intelligent and fuzzy systems, computational intelligence, and soft computing for applied research in general and for complex engineering and decision support systems. the principal mission of infus is bridging the gap between fuzzy and intelligence systems and real complex systems via joint research between universities and international research institutions, encouraging interdisciplinary research and bringing researchers together from multiple disciplines. theme infus has been organized annually since 2019, and this year the fifth conference will be organized on august 22-24, 2023. each year, a theme is selected, and special sessions are organized based on the theme. this year, our theme is intelligence and sustainable future. the themes of the previous conferences are as follows:  2019: intelligent and fuzzy techniques in big data analytics and decision making  2020: smart and innovative solutions  2021: emerging conditions and digital transformation  2022: digital acceleration and the new normal ijahp news and events: kahraman/infus conferences international journal of the analytic hierarchy process 2 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1034 keynote speakers each year, infus invites world-renowned researchers on fuzzy logic and intelligent systems as keynote speakers. the keynote speakers attending the next infus are as follows:  ahmet fahri ozok  ajith abraham  habib zaidi  janusz kacprzyk  krassimir t. atanassov  okyay kaynak  oscar castillo  vicenc torra  vilem novak publication opportunities all accepted papers are double peer blind reviewed, and the accepted papers are published in the web of science indexed proceedings by springer. furthermore, the selected papers have the opportunity to be published in conference associated journals. up to now, hundreds of articles have been published in the following journals:  journal of intelligent and fuzzy systems (sci-e)  symmetry (sci-e)  international journal of computational intelligence systems (sci-e)  journal of multiple-valued logic and soft computing (sci-e)  journal of information assurance and security (esci)  international journal of risk analysis and crisis response (doaj)  journal of fuzzy logic and modeling in engineering (scopus)  international journal of analytic hierarchy process (scopus) ijahp news and events: kahraman/infus conferences international journal of the analytic hierarchy process 3 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1034 x` program and social events the conference includes a social program that is organized to include top sights in the cities where the infus conference is held. the following pictures show some wonderful memories from these programs. izmir pergamon red basilica hagia sophia bosphorus ephesus cruise tour 7_saaty_essay_laws_of_thought_dec_23_179-180 ijahp article: saaty/ the serious omission of comparisons in aristotle’s laws of thought international journal of the analytic hierarchy process 180 vol. 3 issue 1 2011 issn 1936-6744       saaty  lecturing  at  isahp2011   in  sorrento,  italy.   the serious omission of comparisons in aristotle’s laws of thought thomas l. saaty university of pittsburgh pittsburgh, pennsylvania 15260 usa saaty@katz.pitt.edu essay comparisons are the bedrock of all human thinking no matter where it is done on this earth in every culture. we compare things with one another and with themselves through memory to know if they are what we are looking for. we need memory and time to do that regardless of how brief and spontaneous a comparison may be. thus thought is a dynamic process in its minutest form and needs time to be connected and unfolded to draw meaning as we compare it to other thoughts. the original three laws of thought due to aristotle are fundamental axiomatic rules on which western philosophy rational discourse through reasoning has been primarily based for more than two millennia. they have been used in our logic, mathematics and scientific thinking, and they implicitly assume an individual can compare a thing against his memory of a thing to determine if what he sees or perceives is or is not the same as his memory of it; if not, then it is something else, or if it does not have the properties it is remembered to have, then it is not the same thing. here we will discuss the three laws and show that a fourth law should be added, the law of comparisons, which in fact should precede the other three. being able to compare is a talent we have that is necessary for us to recognize and make distinctions among things; and it must come first because without it we would not know how to use the other three laws. 1. the law of identity a necessary and sufficient condition that two entities x and y be identical (or the same) is that they share the same properties, and thus can be said to be indiscernible. according to leibnitz the law of identity is the first primitive truth of reason which is affirmative. but absolute identity has no meaning unless one thinks of relative identity, which means relative to other things. this involves the even more basic assumption that things are implicitly comparable. we need to compare x and y to see if they are identical or not. there is no other way to know. ijahp article: saaty/ the serious omission of comparisons in aristotle’s laws of thought international journal of the analytic hierarchy process 181 vol. 3 issue 1 2011 issn 1936-6744       2. the law of non-contradiction either x or not x is true and there is no other possibility. this means that opposite assertions cannot be true at the same time. here again one must compare x to not x to determine which case one is dealing with. 3. the law of the excluded middle an entity x has or does not have a property p. conversely, a property p is either possessed by an entity x or it is not possessed by x. here x is compared with itself having the property p and not having the property p to determine the case. the significant conclusion is that to know if anything is identical with itself, it is necessary to compare it with itself and with other things with respect to a certain property p and also compare p with other properties as to which is possessed by x and which is not. thus comparisons are intertwined with all three laws of thought. all meaning derives from comparison in some way. in light of this we suggest that the zeroth law of thought, one of the basic definitions in the analytic hierarchy process, should come first. 0. the law of comparisons let ! be a finite set of n elements called alternatives. let ℭ be a set of properties or attributes with respect to which elements in ! are compared. a property is a feature that an object or individual possesses even if we are ignorant of this fact, whereas an attribute is a feature we assign to some object: it is a concept. here we assume that properties and attributes are interchangeable, and we generally refer to them as criteria. a criterion is a primitive concept. when two objects or elements in ! are compared according to a criterion c in ℭ , we say that we are performing binary comparisons. let > c be a binary relation on ! representing "more preferred than" or "dominates" with respect to a criterion c in ℭ. let ~ c be the binary relation "indifferent to" with respect to a criterion c in ℭ. hence, given two elements, ia , ja ε !, either i jca a> or j ica a> or ~i jca a for all c∈  ℭ. we use i ja ac> % to indicate either more preferred or indifferent. a given family of binary relations with respect to a criterion c in ℭ is a primitive concept. we shall use this relation to derive the notion of priority or importance both with respect to one criterion and also with respect to several. in passing we note that induction has also been proposed as another law of thought. thomas saaty, pittsburgh, pennsylvania, 2011 rob typewritten text http://dx.doi.org/10.13033/ijahp.v3i2.124 ijahp essay: kahraman/a brief literature review for fuzzy ahp international journal of the analytic hierarchy process 293 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.599 a brief literature review for fuzzy ahp cengiz kahraman department of industrial engineering istanbul technical university 34367 besiktas, istanbul, turkey ahp is an effective tool for dealing with complex hierarchical decision making problems. it divides a large complex problem into small and easily solvable subproblems and then combines the solutions of these small problems in order to obtain the solution of the larger problem. consistency measurement of the pairwise comparison matrices is another useful component of ahp. it uses a 1 to 9 scale whose linguistic expressions are defined accordingly. however, the representation of a linguistic term by an exact number may not fully reflect a decision maker’s judgments in his/her mind. fuzzy numbers and scales have been developed to consider this issue in the literature. after ordinary fuzzy sets have been extended to new fuzzy types of sets such as type-2 fuzzy sets, intuitionistic fuzzy sets, hesitant fuzzy sets, pythagorean fuzzy sets, and neutrosophic sets, new extensions of fuzzy ahp have been proposed by various authors accordingly. a brief literature review on fuzzy ahp studies is given. the scopus database was utilized for this literature review. when the term “fuzzy ahp” was searched in article title, abstract, and keywords, 2,066 documents were found. figure 1 illustrates the number of documents by their publication years. after 2003, the number of documents on fuzzy ahp sharply increases. figure 2 presents the countries publishing fuzzy ahp papers and the numbers of publications. china, iran, turkey, india, and taiwan are the five leading countries in this area. figure 3 shows the types of documents written about fuzzy ahp. articles on fuzzy ahp and conference papers are by far the leading types of documents written about fuzzy ahp. figure 4 presents the classification of fuzzy ahp documents by their subject areas. engineering and computing sciences are the leading subject areas followed by business, management and account, mathematics, and decision sciences. in the special topic articles section, three papers on the theoretical development and/or practical usage of fuzzy ahp are presented. i hope these papers will provide a useful resource of ideas, techniques, and methods for research on fuzzy ahp. ijahp essay: kahraman/a brief literature review for fuzzy ahp international journal of the analytic hierarchy process 294 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.599 figure 1 number of publications on fuzzy ahp by their year figure 2 countries publishing fuzzy ahp papers 0 50 100 150 200 250 1988 1993 1998 2003 2008 2013 2018 publication on fuzzy ahp by years 0 100 200 300 400 500 600 700 c h in a ir a n t u rk e y in d ia t a iw a n u n it e d s ta te s m a la y si a u n it e d k in g d o m s o u th k o re a c a n a d a in d o n e si a h o n g k o n g it a ly ja p a n a u st ra li a f ra n c e s e rb ia m o ro c c o t h a il a n d l it h u a n ia s p a in g e rm a n y d e n m a rk g re e c e p o la n d v ie t n a m b a n g la d e sh q a ta r s a u d i a ra b ia b ra z il p h il ip p in e s s in g a p o re n e th e rl a n d s u n it e d a ra b e m ir a te s b e lg iu m c o lo m b ia c z e c h r e p u b li c n ig e ri a s o u th a fr ic a t u n is ia number of documents by their countries ijahp essay: kahraman/a brief literature review for fuzzy ahp international journal of the analytic hierarchy process 295 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.599 figure 3 types of documents on fuzzy ahp 0 200 400 600 800 1000 1200 1400 article conference paper book chapter review conference review article in press erratum book note types of documents on fuzzy ahp ijahp essay: kahraman/a brief literature review for fuzzy ahp international journal of the analytic hierarchy process 296 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.599 figure 4 classification of fuzzy ahp documents by their subject areas 0 200 400 600 800 1000 1200 classification of fuzzy ahp documents by their subject areas ijahp essay: kahraman/a brief literature review for fuzzy ahp international journal of the analytic hierarchy process 297 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.599 references atanassov, k.t. (1986). intuitionistic fuzzy sets, fuzzy sets and systems, 20(1), 8796. doi: https://doi.org/10.1016/s0165-0114(86)80034-3 saaty, t. (1980). the analytic hierarchy process. new york: mcgraw-hill. scopus database, elsevier, 2018. smarandache, f. (2003). neutrosophic set-a generalization of the intuitionistic fuzzy set. international journal of pure and applied mathematics, 24(3), 287-297. torra, v. (2010). hesitant fuzzy sets. international journal of intelligent systems 25(6), 529-539. doi: https://doi.org/10.1002/int.20418 yager, r. and abbasov, a. (2013). pythagorean membership grades, complex numbers, and decision making. international. journal of intelligent systems, 28(5), 436-452. doi: https://doi.org/10.1002/int.21584 zadeh, l.a. (1965). fuzzy sets, information and control, 8(3), 338-353. doi: https://doi.org/10.1016/s0019-9958(65)90241-x zadeh, l.a. (1975). the concept of a linguistic variable and its application to approximate reasoning, information sciences 8(3), 199-249. doi: https://doi.org/10.1016/0020-0255(75)90036-5 https://doi.org/10.1016/s0165-0114(86)80034-3 https://doi.org/10.1002/int.20418 https://doi.org/10.1002/int.21584 https://doi.org/10.1016/s0019-9958(65)90241-x ijahp news: karpak, shang/five papers presented at isahp2014 are published in the international journal of production economics international journal of the analytic hierarchy process 547 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.445 five papers presented at isahp 2014 are published in the international journal of production economics birsen karpak associate editor, ijahp, guest editor, ijpe williamson college of business youngstown state university bkarpak@ijahp.org jennifer shang guest editor, ijpe area head, business analytics and operations katz graduate school of business university of pittsburgh eleven papers, out of 91 submitted, presented at the 13th international symposium on the analytic hierarchy process (isahp), held in washington dc, usa, from june 29july 2, 2014 were selected as candidate papers for the international journal of production economics (ijpe) by the guest editors drs. birsen karpak and jennifer shang. the following 5 out of 11 were able to go through the double blind review process handled by dr. peter kelle, north-american editor of ijpe, and are published.  akyildiz, b., kadaifci, c. and topcu, i., a decision framework proposal for customer order prioritization: a case study for a structural steel company  büyüközkan, g., güleryüz, s., an integrated dematel-anp approach for renewable energy resources selection in turkey  dangol, r., bahl, m. and karpak, b., timing cooperative relationships with sequential capability development process to reduce capability development trade-offs  garcía-melón, m., poveda-bautista, r., del valle, m.j.l., using the strategic relative alignment index for the selection of portfolio projects application to a public venezuelan power corporation  mu. e., carroll j., development of a fraud risk decision model for prioritizing fraud risk cases in manufacturing firms the dangol et al. paper was not handled by the guest editors, karpak and shang, but was directly submitted to the ijpe editorial office by the author. this article is included in the list since it was also presented at isahp 2014. we are very pleased that all of the papers went through the regular review process of the journal instead of publishing them as a special issue. we thank the authors for preparing high quality papers that were publishable in the ijpe, and are very happy to be able to serve the ahp/anp community! https://www.journals.elsevier.com/international-journal-of-production-economics/ https://www.journals.elsevier.com/international-journal-of-production-economics/ an interpretation of the ahp eigenvector solution for the lay person ijahp article: lipovetsky/the ahp eigenvector solution for the lay person international journal of the 158 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 an interpretation of the ahp eigenvector solution for the lay person1 stan lipovetsky stan.lipovetsky@gfk.com gfk custom research north america 8401 golden valley rd., minneapolis, mn 55427, usa abstract an ahp priority vector represents the importance, preference, or likelihood of its elements with respect to a certain property or criterion and here we examine how that priority vector can be derived through an iterative process applied to the pairwise comparison matrix. further, we show that the vector obtained in this way satisfies the definition for an eigenvector of the original judgment matrix. practical managers using ahp in decision making would most likely be better able to appreciate this approach than they would a process phrased in the language of linear algebra. the overall priority vector for the alternatives in a hierarchy and, further, in a network, can be obtained in the same way by applying the iterative process to the supermatrix of the anp. this claim is examined in depth in a forthcoming paper that will appear in this journal. keywords: analytic hierarchy process, ahp, eigenvector “everything is foreseen, yet freedom of choice is granted” the wisdom of the fathers. 1. introduction the analytic hierarchy process (ahp) and analytic network process (anp) are the methodologies and methods of multiple criteria decision making in complex hierarchies and networks, and were originated and developed by thomas saaty (1977, 1980, 1994, 1996, 2000, and 2010). the ahp and anp have been expanded by numerous authors and widely used for solving various multi-criteria decision problems (saaty and kearns, 1985; saaty and vargas, 1994; golden, wasil, and harker, 1989; wasil and golden, 2003). some works contributed to ahp techniques in relation to optimizing procedures, group decisions, robust and statistical evaluations (for example, lipovetsky and tishler, 1999; lipovetsky and conklin, 2002; lipovetsky, 1996, 2005, 2008, 2009a, b; and references within). 1 acknowledgment: my special thanks to rozann saaty for the great help in editing and improving the paper. rob typewritten text http://dx.doi.org/10.13033/ijahp.v2i2.42 ijahp article: lipovetsky/the ahp eigenvector solution for the lay person international journal of the 159 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 this paper describes how to interpret ahp solutions. in ahp, the elements of the priority vector represent the mean values of the preference of each alternative over the others. in addition to using a regular synthesizing process to obtain the ahp global vector, it is possible to construct it by raising the supermatrix of the hierarchy to powers until it converges, or by finding the principal eigenvector of this matrix. in an anp model with feedback, the solution can also be interpreted as a mean preference of alternatives weighted by the criteria, and vice versa – the means of the criteria weighted by the alternatives. the anp solution can be obtained from the limit power of the supermatrix, or by solving the eigenproblem for this matrix. the approach suggested here offers a clear description of complex ahp/anp interrelations and an explicit interpretation of the priority results for managerial decision making. in this paper, we offer an interpretation of the priority vector for an ahp pairwise comparison matrix that supports the use of the principal eigenvector as a solution. in the next paper, part ii, which will appear in a later issue of this journal, we will show that similar reasoning can be applied to interpret the priority vectors that are either a global synthesis of the priorities of the alternatives in an ahp model or of an anp model with feedback derived from the supermatrix. 2. the ahp solution and its interpretation for each group of compared items, classic ahp considers judgments elicited from an expert arranged in a pairwise comparison matrix as shown below: (1)             = 1... ........................... ...1 ...1 21 221 112 nn n n aa aa aa a , where each element ija is the judge’s estimate of the quotient of the i-th and j-th priorities, respectively. the solution of the eigenproblem (see saaty, 1980, 1994, 1996, 2000, and 2010) (2) λαα =a for the maximum eigenvalue λ yields the principal eigenvector α which serves as the ahp estimate for the priorities. a numerical estimate of the principal eigenvector can be obtained by iteratively solving the equation: (3) )1()1()( ++= ttta αλα , with t = 1, 2, 3…. the initial vector )1(α usually taken as the uniform vector e divided by n, so multiplying a times this initial vector in the first step with t = 1 yields the following: ijahp article: lipovetsky/the ahp eigenvector solution for the lay person international journal of the 160 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 (4)                 =                           =                 ⋅ ∑ ∑ ∑ = = = nn j nj n j j n j j m m m a n a n a n n a . . 1 . . 1 1 1 . . 1 1 1 2 1 1 1 2 1 1 , where jm is the mean of the elements in the jth row of the matrix a. the vector on the the right-hand side of equation (4) serves as the second approximation vector α(2) in (3), and it is proportional to the simple means above: (5) ( )′= nmmm ,...,, 21)2(α , where prime denotes transposition. eigenvectors are usually normalized by the euclidean metric: (6) 1 1 2 ==′ ∑ = n i iααα . but ahp priority vectors are conventionally normalized so the total equals one: (7) 1 1 ==′ ∑ = n i ie αα , so the vector (5) becomes proportional to relative mean values, (8) ( )′= nsss ,...,, 21)2(α , defined as: (9) .1, 1 1 == ∑ ∑ = = n j jn k k j j s m m s using the vector in (8) in place of the initial uniform vector in (3) yields the next approximation (4) to the eigenvector which again equals the weighted mean value: ijahp article: lipovetsky/the ahp eigenvector solution for the lay person international journal of the 161 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 (10)                 =                       =                 ∑ ∑ ∑ = = = n n j njj n j jj n j jj n m m m as as as s s s a . . . . . . 2 1 1 1 2 1 1 2 1 . now the new mj are from the weighted means in the rows of the matrix a, and the weights are again proportional to the previous means in equation (9). this process continues until the solutions in two consecutive iterations coincide with the needed precision. thus, elements of an ahp eigenvector equal the weighted mean values of the elements of the matrix rows and the weights are proportional to these means themselves. recalling that the elements in the i-th row of matrix (1) express the prevalence of the i-th item over each of the others, we get the conclusion that the ahp eigenvector consists of weighted averages of the judgments in each row that express the i-th item’s preference over all the others. a forthcoming paper will describe how the solution of an anp supermatrix can be characterized in a similar way. references golden, b.l., wasil, e.a., & harker, p.t. (eds.), (1989). the analytic hierarchy process: applications and studies. berlin-heidelberg: springer. lipovetsky, s. (1996). the synthetic hierarchy method: an optimizing approach to a obtaining priorities in the ahp. european journal of operational research, 93, 550-564. lipovetsky, s., & tishler, a. (1999). interval estimation of priorities in the ahp. european journal of operational research, 114, 153-164. lipovetsky, s., & conklin, m. (2002). robust estimation of priorities in the ahp. european journal of operational research, 137, 110-122. lipovetsky, s. (2005). analytic hierarchy processing in chapman-kolmogorov equations. international j. of operations and quantitative management, 11, 219-228. ijahp article: lipovetsky/the ahp eigenvector solution for the lay person international journal of the 162 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 lipovetsky, s. (2008). comparison among different patterns of priority vectors estimation methods. international j. of mathematical education in science and technology, 39, 301-311. lipovetsky, s. (2009a). global priority estimation in multiperson decision making. journal of optimization theory and applications, 140, 77-91. lipovetsky, s. (2009b). optimal hierarchy structures for multi-attribute-criteria decisions. journal of systems science and complexity, 22, 228-242. saaty, t.l. (1977). a scaling method for priorities in hierarchical structures. mathematical psychology, 15, 234-281. saaty, t.l. (1980). the analytic hierarchy process. new york: mcgraw-hill. saaty, t.l. (1994). fundamentals of decision making and priority theory with the analytic hierarchy process. pittsburgh, pa: rws publications. saaty, t.l. (1996). decision making with dependence and feedback: the analytic network process. pittsburgh, pa: rws publications. saaty, t.l. (2000). decision making for leaders. pittsburgh, pa: rws publications. saaty, t.l. (2010). principia mathematica decernendi: mathematical principles of decision making: generalization of the analytic network process to neural firing and synthesis. pittsburgh, pa: rws publications. saaty, t.l., & kearns, k.p. (1985). analytical planning. new york: pergamon press. saaty, t.l., & vargas, l.g. (1994). decision making in economic, political, social and technological environment with the analytic hierarchy process. pittsburgh, pa: rws publications. the wisdom of the fathers, and its classical commentaries (1960). translated by y. goldin, new york: the heritage press. wasil, e.a., & golden, b.l. (2003). celebrating 25 years of ahp-based decision making. computers and operations research, 30, 1419-1420. ijahp news and events: theory and applications of ahp/anp in mcdm 2022 international journal of the analytic hierarchy process 1 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1027 theory and applications of ahp/anp at mcdm 2022 birsen karpak youngstown state university participants of the 26 th international conference on multiple criteria decision making (mcdm2022) at the university of portsmouth, uk june 26 july 1, 2022 we had a wonderful welcome from the conference organizers alessio ishizaka (chair), banu lokman (co-chair), dylan jones (program chair), ashraf labib, salvatore greco, negar akbari, maria barbati and salem chakhar of the 26 th international conference on multiple criteria decision making (mcdm2022) which was held at the university of portsmouth, uk from june 26 july 1, 2022. the purpose of the mcdm conference is to bring together the mcdm community every two years. this year however, there had been a gap of three years since our last meeting because of covid. conference organizers did not want us to miss the opportunity to experience face-to-face interaction with students and colleagues in portsmouth, uk, so they rescheduled mcdm 2021 to 2022. i am sure most of the participants benefited from the discussions after the talks, and ijahp news and events: theory and applications of ahp/anp in mcdm 2022 international journal of the analytic hierarchy process 2 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1027 enjoyed the social program that brings our community closer together and inspires future collaboration. mcdm 2022 brought together academics, researchers, practitioners, and other professionals from around the world. it was a great opportunity for the delegates to present, discuss, network, and hear about the recent advances in mcdm. the distinguished plenary speakers included professor salvatore greco from the university of catania, professor murat köksalan from the university of michigan, and professor luis martínez from the university of jaén. the scientific program committee consisted of matthias ehrgott, lancaster university, uk; carlos coello coello, cinvestav-ipn, mexico; kerstin daechert, fraunhofer institute itwm, germany; michalis doumpos, technical university of crete, greece, josé rui figueira, technical university of lisbon, portugal; salvatore greco, university of catania, italy; birsen karpak, youngstown state university, usa; kathrin klamroth, bergische universität wuppertal, germany; murat köksalan, michigan university, usa; banu lokman, university of portsmouth, uk; alessio ishizaka, university of portsmouth, uk; dylan jones, university of portsmouth, uk; caroline mota, university of pernambuco, brazil; francisco ruiz, university of malaga, spain; hsu-shih shih, tamkang university, taiwan; johannes siebert, management center innsbruck, austria; theo stewart, university of cape town, south africa; i̇lker topçu, i̇stanbul technical university, turkey; and last, but not least, jyrki wallenius, aalto university, finland. the scientific program committee contributed to the success of the conference in a variety of ways including chairing award committees, reviewing papers, and being instrumental in getting many invited participants to the conference. the conference consisted of the following eleven streams: analytic hierarchy/network processes chaired by enrique mu and rozann saaty; evolutionary multiobjective optimization by carlos fonseca; multiobjective optimization by serpil sayin; goal programming by fouad ben abdelaziz; outranking methods by josé figueira, salvatore corrente and yves de smet; rule-based methods and artificial intelligence by salvatore greco; value and utility models by milosz kadzinski; data envelopment analysis by ali emrouznejad; best-worst method by jafar rezaei; interactive mcdm by jyrki wallenius and applications of mcda by sarah ben amor and sachin mangla. theory and applications of the analytic hierarchy and network processes (ahp/anp) and extensions were presented in the ahp/anp streams as well as in the applications of mcda stream. i will provide a summary of some of them that i was able to attend. interested readers can find all of the presentations in the program and book of abstracts. antonella petrillo and i invited quite a few participants from the ahp/anp community to a special session titled “ahp/anp and extensions: applications in supply chain management and industrial engineering.” though it was difficult to get a timely visa for the uk, and quite a few of our invited ahp/anp delegates had to cancel their trip at the last minute (some of them received their uk entrance visas a month after the conference!), we still had ahp/anp and extensions’ presentations every day of the conference. https://mcdm2021.org/teddy/wp-content/uploads/2022/06/mcdm2022-conferenceprogrambook.pdf ijahp news and events: theory and applications of ahp/anp in mcdm 2022 international journal of the analytic hierarchy process 3 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1027 in the following paragraph, i will provide a summary of the sessions i was able to attend. samah alnuaimi from sheffield hallam university, uk, (coauthor sameh saad from the same university) presented “an innovation framework for the digital era an ahp comparison approach between the uk and germany.” cecilia montt from universidad de santiago de chile, (coauthors luz martinez, luis quezada and astrid oddershede from the same university) presented method for characterizing risk management using multi-criteria. the most influential factors in determining the risk are identified using the analytic hierarchy process (ahp). it was great to see my friend astrid oddershede from universidad de santiago de chile, who i had met at thomas and rozann saaty’s home many years ago. miguel angel ortiz-barrios from universidad de la costa cuc, colombia (coauthors armando pérez-aguilar and pablo pancardo both from juarez autonomous university of tabasco, colombia) presented “multi-criteria hybrid approach for prioritizing suspected covid-19 patients within the public emergency departments: a case study.” valerio salomon from sao paulo state university, brazil, (coauthor antonio lombardi netto from the same university) presented “including environment, social and governance (esg) into the multicriteria analysis of investments.” valerio salomon explained how they combined fuzzy anp with a monte carlo simulation in credit risk analysis and asset selection. luis vargas from university of pittsburgh, usa, (coauthors marcel minutolo from robert morris university, usa and amos guiora from university of utah, usa) presented “how to measure the impact of gun laws on lawful owners of guns.” birsen karpak from youngstown state university, usa presented “a framework for prioritizing sustainable third-party logistics service providers: a case study,” (coauthors ozden bayazit from central washington university, usa and tacibaht turel from youngstown state university) and “ahp/anp applications of big data analytics in supply chain management” (coauthors ilker topcu from istanbul technical university, turkey and fusun ulengin from sabanci university, turkey) since both ozden bayazit as well as fusun ulengin could not make the conference though both had registered and were planning to come. “ahp/anp applications of big data analytics in supply chain management” was an update about the study that the authors presented at informs 2021. in this new study, the authors first searched the web of science (wos) core collection database considering terms related to big data analytics (bda) and supply chain management (scm) and found 563 articles published before the end of 2021. then, they explored ahp/anp and extensions applications in big data enabled supply chains as of may 2022. after comparing and summarizing the reviewed approaches, they identified the primary research opportunities where the deployment and use of ‘big data’ is likely to have a significant impact on supply chain management. they came up with some future challenges of bda in scm for collecting data, storing data, cleaning and verifying data, summarizing data, and modeling with big data. big data enabled digital supply chains and big data adoption risks involved in sustainable supply chains are among the new research directions they are considering. ram kumar dhurkari from indian institute of management sirmaur, india presented “improving the predictive power of ahp-based decision support systems.” yuji sato from graduate school of management, japan presented “an improvement of the consistency index in pairwise comparison.” he proposes a new inconsistency index ijahp news and events: theory and applications of ahp/anp in mcdm 2022 international journal of the analytic hierarchy process 4 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1027 based on the mathematical property of a pairwise comparison matrix and further validated based on the conformity and the goodness-of-fit of weight. he asserts that the proposed index detects inconsistency among real-world pairwise comparisons (pcs) more sensitively than the consistency ratio. vanessa assumma from polytechnic university of turin, italy (coauthors marta bottero from the same university and alessio ishizaka from neoma business school, france) presented “ganp-fuzzy sorting ii method for the environmental planning and management of the susa valley.” this study aims to support both decision makers and policy makers in the definition and prioritization of complex and uncertain strategies and recommendations. particularly, this work focuses on mcda sorting classification to deal with complex spatial systems with interdependences, such as biocultural landscapes. the contribution starts from an existing sorting method called group analytic network process sorting ii method (ganpsort ii), which was developed in the context of vanessa assumma’s ph.d. thesis and based on the previous extension of anp sort (ishizaka & pereira, 2017). the ganp-fuzzy sorting ii is proposed as an extension by integrating the fuzzy logic in the evaluation stage. konrad kułakowski from agh ust, poland (coauthor michal strada from the same institution) presented “single expert manipulations in the cardinal pairwisecomparisons method.” astrid oddershede from the university of santiago of chile, (coauthors luis quezada, miguel angel rodriguez and cecilia montt veas from the same university) presented “ahp model for assessing critical factors impacting stadium security.” last, but not least, we heard an ahp/anp and extensions presentation from luis quezada, (university of santiago of chile) (coauthors astrid oddershede, pedro palominos, cecilia montt veas and miguel angel gonzalez, all from the same university). luis quezada presented “a method for designing a strategy map with scenarios using anp/ahp.” in addition to the academic presentations, we also had the opportunity to socialize with friends and colleagues after the long hiatus caused by the pandemic and to enjoy the beautiful views in portsmouth. an afternoon boat cruise and an authentic british pub experience was held on wednesday, june 29, 2022. delegates enjoying the boat tour ijahp news and events: theory and applications of ahp/anp in mcdm 2022 international journal of the analytic hierarchy process 5 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1027 delegates enjoying the boat tour ijahp news and events: theory and applications of ahp/anp in mcdm 2022 international journal of the analytic hierarchy process 6 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1027 how many new ideas are born during discussions onboard the boat, and how many new connections established! conference co-organizer banu lokman with her son and husband on the boat tour with jyrki wallenius ijahp news and events: theory and applications of ahp/anp in mcdm 2022 international journal of the analytic hierarchy process 7 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1027 jyrki wallenius, awards committee chair of the society, presented the conference chair award to alessio ishizaka (chair) during the banquet dinner at a beautifully restored historic boathouse on june 30, 2022. he also recognized banu lokman (co-chair) and dylan jones (program chair). watch jyrki and his sense of humor, presenting the award in the following video! https://www.youtube.com/watch?v=zq_utkucqsm conference awardees with jyrki wallenius and matthias ehrgott (president of the society) at the closing session, future conferences related to mcdm were presented. alessio ishizaka announced the 7 th dyses conference to be held october 5-7, 2022 in rouen (france); birsen karpak from youngstown state university, presented isahp 2022, a web conference, to be held december 15-18, 2022; fouad ben abdelaziz from neoma business school announced mcdm 2024 to be held in person in hammamet, https://www.youtube.com/watch?v=zq_utkucqsm https://neoma-bs.com/faculty-and-research/research/dyses2022-conference/ https://www.isahp.org/ https://www.isahp.org/ ijahp news and events: theory and applications of ahp/anp in mcdm 2022 international journal of the analytic hierarchy process 8 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1027 tunisia in late may or early june. the mcdm community has many opportunities to exchange ideas in france and tunisia as well as over the web. isahp promises you a different web conference; we hope you will enjoy it. looking forward to seeing you at future conferences. ijahp article: tsyganok/about one approach to ahp/anp stability measurement international journal of the analytic hierarchy process 46 vol. 3 issue 1 2011 issn 1936-6744 about one approach to ahp/anp stability measurement vitaliy tsyganok national academy of sciences of ukraine the institute for information recording kyiv, ukraine e-mail: vitaliy.tsyganok@gmail.com abstract ahp/anp stability measurement methods are described. in this paper we define the method’s stability as the measure of its results dependence on the expert’s errors, made during pair comparisons. ranking stability (order preservation in alternative ranking under natural expert’s errors, made during expert estimation) and estimating stability (maintaining alternative weights within the specified maximal relative inaccuracy range) are considered. targeted genetic algorithm search procedure is used for possible stability violation detection. then division-in-half (dichotomy) method is applied to calculate stability metric of a given criteria hierarchy. keywords: ahp/anp stability measurement, ranking/estimate stability, genetic algorithm 1. introduction in the decision making systems (dss), particularly, while using ahp/anp (saaty, 1980; saaty, 2008) for getting decision variants, the question of the decision’s stability degree arises. in other words, the question is: what is the extent of dependence between expert’s errors (made during criteria hierarchy\network model building and during alternatives estimation according to a chosen set of criteria) and decision’s stability? for example, in case of criterion hierarchy, used for choosing the best alternative from the set, it is undesirable for the object’s place in the rating to depend upon a single slight difference in pair-wise comparisons of two sub-criteria influence coefficients (like difference between “very, very strong” or “extreme” domination). it is obvious that decision stability depends on both criteria hierarchy structure and numerical values reflecting experts’ priorities. having analyzed the essentials of ahp method, we can come to a conclusion that complexity of criteria hierarchy structure, and therefore, stability of decisions made on its basis, depends on lengths of ways from any alternative in the hierarchy to the main criterion\goal, because, the longer the ways are, the more ratios characterizing criteria influence degree “contribute” to alternative weights’ calculation. rob typewritten text http://dx.doi.org/10.13033/ijahp.v3i1.50 ijahp article: tsyganok/about one approach to ahp/anp stability measurement international journal of the analytic hierarchy process 47 vol. 3 issue 1 2011 issn 1936-6744 2. development of decision stability estimation method 2.1 problem classes let’s consider two main classes of problems solved by ahp. these are: alternatives’ ranking and their estimation. in the case of alternative ranking-associated decisions we shall define decision stability as the property of alternatives’ rank order preservation under natural expert’s errors, made during expert estimation. we suggest using the probability of rank order preservation as stability metric. in order to ensure decisions’ stability in case of alternative weights’ calculation, obtained estimates are required to lie within the range, limited by specified relative error. in this case, the probability of keeping the alternative weight within the range may be chosen as stability metric. it should be stressed that the solution procedure for decision stability-related problem is the same for both aforementioned cases. so, let’s formulate the problem statement. 2.2 problem statement what is given: 1) a ready-built criteria hierarchy; 2) expert priorities, given in the form of pair-wise comparison matrices (pcm) in fundamental scale. we suppose that experts have already built pcm-s for calculating criterion and alternative weights (it is presumed that alternatives have been compared according to each of bottom-level criteria). what is to be obtained: stability metric value under given conditions. 2.3 stability metrics let’s model expert’s errors, made in pair-wise comparisons ki, i∈{1..n} by setting a specified relative deviation level of dominance values (in order to avoid complications with indices we define n as the total number of pair-wise comparison values in all the matrices built for the given hierarchy). in our model the deviation value is considered the same for all pair-wise comparisons of criterion weights and of all alternatives according to each criterion of the given hierarchy. hereinafter, deviation is the boundary random deflection value of every pair-wise comparison ki for the given criteria hierarchy measured as percentage of its current value. a hierarchy is considered subject to influence of deviation δ when every pair comparison is varied according to the following law: ki* = ki + ri ⋅ki ⋅δ / 100, where ri is a random value, evenly distributed in the [-1;1] range. it is significant, that, for modeling purposes, while evaluating stability metric the fundamental scale will be considered continual. we shall define the maximal deviation δ, under which ahp result remains stable (in ijahp article: tsyganok/about one approach to ahp/anp stability measurement international journal of the analytic hierarchy process 48 vol. 3 issue 1 2011 issn 1936-6744 other words, no rank reversals occur, or, in case of another problem class, obtained alternative weights remain within the given range) as stability metric. 2.4 solution process let’s obtain maximal alternative weight deviation for a given criteria hierarchy as relative value measured in percentage, through division-in-half (dichotomy) method. in order to do this let’s specify a certain arbitrary deviation value within the search range between 0% and 100% and subject the hierarchy to the influence of the deviation. for the specified deviation value let’s check whether ahp result keeps stable in comparison with the result under deviation 0%. if the result remains stable then the search for maximal deviation is continued in the range limited by the value used in previous calculation and upper bound of the range (i.e., 100%) (to be exact, the middle of the range is considered). otherwise, i.e. when the result is not stable, the middle of another range (lying to the left of the value used in the previous calculation step) is considered while checking. the searching range converges with every such iteration. this iterative process ends when desired accuracy of deviation is achieved. it means that the process stops when the middle value of the search range is different from one of the boundary values of the range, limited by desired accuracy. for example, if the initial relative value deviation value is 10% and after all pair comparisons have been subjected to such deviation ahp results remained stable, the next deviation value will be 55% (the average of 10 and 100%). otherwise (i.e. in case of stability breach), the next deviation value will constitute 5%, which is the average of 0 and 10%. 2.5 minimizing the possibility of undetected stability breach it should be noted that since the process of expert’s errors modeling is stochastic, it is possible that stability breach won’t be detected at some current deviation level. while applying the specified searching method it is very important to minimize this possibility, because unlike in sequential search when we are using division-in-half (dichotomy) method, though the searching process is faster, any omission of stability violation switches the search to the wrong range, which often leads to incorrect results and conclusions. for example, suppose it is known for sure, that rank reversal in some hierarchy occurs under the deviation of more then 3%, but at some dichotomy step, when the value of deviation amounted to 5%, rank reversal wasn’t detected. then, even in the case when during all the following tests of the rest of deviation values existing rank reversals are detected, the deviation value obtained by division-in-half (dichotomy) method will be greater than 5%. one of the obvious ways to avoid non-detected stability violations under specified deviation is the complete enumeration of possible variants of paired comparisons ki at this deviation level. under the obvious assumption, that dependence of stability violation’s probability from deviation value is a monotonously increasing function, we can conclude that, in the first place, stability violations begin to occur under extreme deflections of pair comparisons values (i.e., on the boundaries of deviation range). so, in order to reduce the search range significantly and to detect possible stability violations as fast as possible, let’s specify only the outermost values of the deflection range during enumeration: ki* = ki ± ki ⋅ δ / 100. ijahp article: tsyganok/about one approach to ahp/anp stability measurement international journal of the analytic hierarchy process 49 vol. 3 issue 1 2011 issn 1936-6744 in this case a number of pair comparison values’ variants that should be considered during exhaustive search is 2n, where n is a number of pair comparisons that were carried out for a given criteria hierarchy. this kind of enumeration makes sense only under small values of n, when n does not exceed 15-20. in order to detect ahp results’ stability breaches in a real criteria hierarchy, we suggest applying genetic algorithm (holland, 1994). 2.6 specific problem let’s consider the problem statement, which, in author’s opinion, can be most easily solved by applying classic genetic algorithm. the problem could be formulated in the following way. what is given: 1) a ready-built criteria hierarchy, based on n pair comparisons, conducted by the expert; 2) deviation of the pair comparisons δ. we should find: a set of pair comparisons’ values obtained under influence of deviation δ, on which maximal stability violation of ahp results is observed. 2.7 formulating genetic algorithm’s utility function when the last formulated condition is satisfied, detection of stability violation can be guaranteed (i.e., maximal stability breach will certainly be detected). utility function, realized in genetic algorithm, will be based on this condition. this utility function should possess larger values while more drastic stability violations are observed. in the case under consideration the function will be specified algorithmically by means of eigenvector method realization for each set of objects, whose pairs are compared, followed by ahp method application and calculation of maximal result values’ deflection from stable values (i.e. values calculated under zero deviation –δ = 0). in order to calculate maximal deflection of ahp method’s results from stable values, let’s consider each of the two aforementioned problems’ classes separately. while considering alternative ranking-related problems, it seems logical to use kemeny’s length or distance (kemeny & snell, 1972) as ahp results’ deflection measure. this value is determined on two relations a and b, where a characterizes ranking α – in the absence of deviation, b characterizes ranking β – based on a specified set of pair comparisons’ results under deviation δ: ( ) ∑ −= ji jijibad , ),(),(, βα . (1) but after further analysis it was found, that kemeny’s distance did not fully satisfy the requirements to utility functions, used in genetic algorithm, because it was not sensible to such changes of ahp results, which didn’t cause rank reversals. that is, on various input data sets (various pair comparisons’ sets) utility function value remains the same in case ijahp article: tsyganok/about one approach to ahp/anp stability measurement international journal of the analytic hierarchy process 50 vol. 3 issue 1 2011 issn 1936-6744 there is no rank reversal. and in this case, criteria for targeted search, applied in genetic algorithm, are absent. therefore, for this class of decision making problems, concerning alternatives’ ranking, the following empiric function was suggested as utility function to be used in genetic algorithm: ∑ − = + + − − = 1 1 1 1 1 k i ii ii aa bb f , (2) where ai is a cardinal estimate of alternative with i-th rank, obtained under zero deviation; bi is a cardinal estimate of alternative with i-th rank, obtained on a given set of pair comparison results, that were subjected to deviational influence δ; k is a number of alternatives in the hierarchy. as for utility function for a problem class, where hierarchies are analyzed as to keeping the results of alternatives’ estimations within the bounds of the maximal relative error, the following function for calculating that maximal relative error corresponds to the specificity of the genetic algorithm-based search: ⎟⎟ ⎠ ⎞ ⎜⎜ ⎝ ⎛ − = i ii i a ba f max2 , (3) where ai and bi are cardinal estimates of the i-th alternative, obtained without and under deviation of pair comparison results, respectively. 2.8 adapting genetic algorithm to the specified problem “population” concept, used in genetic algorithm, corresponds to a set of fixed potency, consisting of sets of pair comparison results in criteria hierarchy, i.e. decision variants (in genetic algorithm they are called “individuals” in population). the difference between the specified procedure and classical implementation of genetic algorithm is that during the search process, stability violation checks are performed, and the process stops if such violation is detected. in case genetic algorithm has completed its work and stability violation is not detected during the search, it is deduced that decisions made under given deviation δ are stable. this feature of genetic algorithm realization allows to discard the possible issue of function (2) domain, namely its behavior while ai ≤ ai+1 (when zero or a negative value can appear in the fraction’s denominator). under such values decision stability violation (i.e. rank reversal) is detected, so genetic algorithm ends its work and, therefore, further calculation of utility function is unnecessary. thus, the specified genetic algorithm is a single step in the solution procedure of the more general problem of obtaining ahp stability metric, stated in section 2.2 of this paper. ijahp article: tsyganok/about one approach to ahp/anp stability measurement international journal of the analytic hierarchy process 51 vol. 3 issue 1 2011 issn 1936-6744 2.9 software used in the experiment in order to analyze criteria hierarchies using the aforementioned technique, the software applications were developed and integrated into an existing dss. a new mode called “decisions stability analysis”, envisioning the input of data for calculations, was added to the dss. during calculations, the subsystem of dss allows detection of two types of decision stability violations: alternative rank reversals and cases when calculated alternative estimate goes beyond the bounds of permissible deflection. in order to verify genetic algorithm’s efficiency under specified parameters, namely “power (cardinality, potency) of population”, “mutation probability” and “number of generations with the same results”, exhaustive search method was applied for smaller criteria hierarchies. screenshots, illustrating the software used in the experiment, are shown on figures 1 and 2. figure 1. dss interface with a given hierarchy structure. figure 2. search parameters input window. ijahp article: tsyganok/about one approach to ahp/anp stability measurement international journal of the analytic hierarchy process 52 vol. 3 issue 1 2011 issn 1936-6744 3. example what is given: 1) criterion hierarchy with main criterion (denoted as 0) end sub-criteria (denoted as 111); 2) set of alternatives, whose weights are being calculated (denoted as 12-15). (see figure 3). figure 3. graph of criterion hierarchy. all experts’ priorities are given in the matrices below. pcm, where influences of sub-criteria 1, 2, 3 and 4 upon criterion 0 are compared in fundamental scale: ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ 1 21 221 4321 . pcm, where influences of sub-criteria 5, 6 and 7 upon criterion 1 are compared: ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎣ ⎡ 1 21 321 . pcm, where influences of sub-criteria 8 and 9 upon criterion 3 are compared: ⎥ ⎦ ⎤ ⎢ ⎣ ⎡ 1 2/11 . pcm, where influences of sub-criteria 10 and 11 upon criterion 9 are compared: ⎥ ⎦ ⎤ ⎢ ⎣ ⎡ 1 61 . pcm for comparing alternatives (nodes 12-15) according to criterion ijahp article: tsyganok/about one approach to ahp/anp stability measurement international journal of the analytic hierarchy process 53 vol. 3 issue 1 2011 issn 1936-6744 2: ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ 1 21 2/12/11 4/13/12/11 ; according to criterion 4: ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ 1 21 22/11 212/11 ; 5: ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ 1 21 221 4321 ; 6: ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ 1 21 221 4321 ; 7: ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ 1 4/11 2/111 2/1221 ; 8: ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ 1 21 221 5231 ; 10: ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ 1 11 5/16/11 2/12/131 ; 11: ⎥ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎢ ⎣ ⎡ 1 21 4/13/11 2251 . it should be noted here that given matrices are consistent enough to apply eigenvector method for alternative weights’ calculation. what is to be obtained? maximal relative deflection of pair comparisons from their given values, under condition that relative deflection of calculated weights does not exceed 5%. using ahp, we have calculated weights of alternatives 12-15 without deviation. they are: {0.2845873027, 0.20527370971, 0.2767528361, 0.23338615149}; let’s call them reference weights. during the targeted search we calculate alternative weights, under condition, that some expert priorities have changed (increased or decreased) by some specific value measured in percentage. for a start, let’s define this value as, say, 10%. it will be the starting point of dichotomy search. we should note that during targeted genetic algorithm-based search, the search range includes all possible variants of expert priorities’ (i.e. pcm elements) change. in the given example all pcm elements (except diagonal ones) were changed by ±10%. the total number of such non-diagonal pcm elements in our example is 59. let’s define this set as k={ki}, i={1..59}. thus, k={2, 3, 4, 2, 2, 2, 2, 3, 2, 1/2, 6, 1/2, 1/3, 1/4, 1/2, 1/2, 2, 1/2, 1, 2, 1/2, 2, 2, 2, 3, 4, 2, 2, 2, 2, 3, 4, 2, 2, 2, 2, 2, 1/2, 1, 1/2, 1/4, 3, 2, 5, 2, 2, 2, 3, 1/2, 1/2, 1/6, 1/5, 1, 5, 2, 2, 1/3, 1/4, 2} so, at present stage, genetic algorithm-based search is targeted at defining whether the weight of any alternative can change by more than 5% as a result of changing any of 59 pcm elements by ±10%. in the given example while k={1.8, 2.7, 4.4, 1.8, 2.2, 1.8, 2.2, 3.3, 1.8, 0.45, 5.4, 0.55, 0.3, 0.225, 0.55, 0.55, 2.2, 0.45, 0.9, 1.8, 0.55, 1.8, 1.8, 1.8, 3.3, 4.4, 2.2, 2.2, 1.8, 2.2, 3.3, 3.6, 2.2, 2.2, 1.8, 2.2, 1.8, 0.55, 1.1, 0.55, 0.275, 3.3, 2.2, 4.5, 1.8, 2.2, 2.2, 2.7, 0.45, 0.55, 0.18(3), 0.18, 0.9, 5.5, 1.8, 2.2, 0.367, 0.275, 1.8}, alternative weights are {0.31069680293, 0.2068284784, 0.26215412441, ijahp article: tsyganok/about one approach to ahp/anp stability measurement international journal of the analytic hierarchy process 54 vol. 3 issue 1 2011 issn 1936-6744 0.22032059426}, so the 1st alternative’s weight exceeds the reference value by 8.40353%, which is more than 5%. so, at this point the further search under given deviation value (10%) stops. to continue the search the new deviation value of (0%+10%)/2=5% is set. under this deviation value (5%) the procedure finds another variant of k, on which calculated alternative weights differ from reference values by more than 5%. under deviation value of (0%+5%)/2=2.5% genetic algorithm doesn’t find any variants of k, on which alternative weights differ from reference values by more than 5%, so the next deviation value is (2.5%+5%)/2=3.75%. under this deviation value there exists a variant of k, on which stability violation is witnessed (hereafter we shall mark such variants with “+”, and negative search results with “–“). the search process continues with the following results: under deviation (2.5%+3.75%)/2=3.13% (value 3.125 is approximated to the second decimal digit) – “+”, under deviation (2.5%+3.13%)/2=2.82% – “+”, 2.66% – “+”, 2.58% – “+”, 2.54% – “–“, 2.56% – “+”, 2.55% – “+”. so, we have found the maximal relative deviation of pair comparison values. it is 2.54%, and under such deviation the absolute relative difference between reference alternative weights and their calculated values does not exceed 5%. 4. conclusions a technique for calculating decisions’ stability measure is suggested. we consider ahpbased decisions, which are made using expert data, input into an existing dss. the developed decision stability estimation methodology can also be applied to obtaining stability measure in anp method, and integrated into other systems, where expert information is used. the described stability metric calculation technique can be used for stability breach detections, resulting from deviations of criterion weights (influence coefficients) and alternative estimates. in cases when stability metric values do not satisfy posed requirements it is possible to develop methods for improving decision stability using the suggested technique. references holland, j.h. (1994). adaptation in natural and artificial systems. an introductory analysis with application to biology, control, and artificial intelligence. london: bradford book edition. kemeny, j.g., & snell, j.l. (1972). mathematical models in the social sciences. cambridge, ma: mit press. saaty, t.l. (1980). the analytic hierarchy process. new york, ny: mcgraw-hill. ijahp article: tsyganok/about one approach to ahp/anp stability measurement international journal of the analytic hierarchy process 55 vol. 3 issue 1 2011 issn 1936-6744 saaty, t.l. (2008). relative measurement and its generalization in decision making. why pairwise comparisons are central in mathematics for the measurement of intangible factors. the analytic hierarchy/network process. statistics and operations research, 102 (2), 251–318. 7_ saaty_eigenvector_in_lay_language_vol2_issue2_p163-169 ijahp article: saaty/the eigenvector in lay language international journal of the 163 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 the eigenvector in lay language thomas l. saaty university of pittsburgh pittsburgh, pa, usa e-mail: saaty@katz.pitt.edu abstract decision making depends on identifying a structure of criteria and alternatives of a decision. it also depends on experience and judgments to select the best alternative. in the analytic hierarchy process (ahp) for decision making the criteria and alternatives are prioritized by forming matrices of judgments and from these judgments priorities are derived for each matrix in the form of the principal eigenvector. an eigenvector is a technical mathematical idea that would benefit from a simplifying explanation. that is what this note does in two ways. keywords: ahp, decision making, principal eigenvector 1. motivation an eigenvector is the solution of a problem (a system of equations) that depends on several factors. in the case of decision problems, one is concerned with the identification of the best alternative among several alternatives and the solution represents the best decision. in geometry we know that to determine the position of a point in space three cartesian coordinates are required. therefore, the factors on which the solution depends are only three. generally, however, in decision problems the parameters to be taken into account to reach a decision may be many. for example, when we choose the city where to live we need to examine and identify the criteria that qualify the best city for us to live in. therefore, criteria such as employment opportunities, entertainment, good schools for our children, not too high a standard of living are our system of equations. alternatives such as orlando, pittsburgh, washington are our system of unknowns to find the best among them. the decision about which city is best (the goal) is our solution. at this point, once we understand what an eigenvector is, it is essential to understand how the assessment process occurs to identify the best solution. the process is based on the comparison judgments that we express in terms of preference for a solution rather than for another. here we have introduced the concepts of many criteria and many alternatives in a decision and the need for an eigenvector in each set of comparisons. thus we have many systems of equations each having its own eigenvector. one then also needs to talk about synthesis. this synthesis can be obtained from the supermatrix, a stochastic matrix with largest eigenvalue equal to one and whose corresponding principal eigenvector is the synthesis of all other eigenvectors. rob typewritten text http://dx.doi.org/10.13033/ijahp.v2i2.87 ijahp article: saaty/the eigenvector in lay language international journal of the 164 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 2. what we learn when we have measurement the following explanation of what an eigenvector is begins with a consideration of comparing items (apples) in pairs whose weight is known. we then show that the same process can be used to derive relative weights for apples where their weights are not known, but we can guess as we compare them in pairs. the latter way of estimating the pairwise comparisons can be used to compare items that are not measurable, for example on critical issues where judgments must be made as to which issue is more important with a certain goal in mind. in this case, too, the measurements are unknown in advance and must be derived from the quantitative judgments. notice that all the solutions are presented as relative numbers. suppose we have three apples, a, b, and c, whose weights in ounces are known to be 6, 3, and 1; to derive their relative weights we compare them in pairs and enter the comparison as shown in figure 1 below. to form a comparison judgment the size of the apple at the left of the row is written in the numerator and the size of the apple at the top of the column is written in the denominator as shown in figure 1. figure 1 pairwise comparison matrix with known weights the weights of these apples are given in ounces but if the weights are transformed into pounds or kilograms the absolute values would be different but the proportions in the comparisons and hence the fractions would be the same. it does not matter what scale is used to make the measurements if proportions are used rather than absolute measurements. to get the proportionate measures, we add the numbers obtained from using any scale of measurement and divide each value by the total. for example the respective relative weights of the apples a, b, c using their measurements in ounces are: 6 3 1 .6, .3, .1. 6 3 1 6 3 1 6 3 1 = = = + + + + + + ijahp article: saaty/the eigenvector in lay language international journal of the 165 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 these fractions give us the part of the whole for each apple, regardless of the scale used to make the measurements. the decimals .6, .3, and .1 are referred to as the priorities for apples a, b, and c according to weight. the table of comparisons can be rewritten using these priorities instead of the original weights as shown in figure 2. figure 2 matrix of pairwise comparisons using priorities 2. what to do if we have no measurements if the weights of the apples are unknown, we estimate their relative sizes by using judgments; apple a is 2 times the size of apple b; apple b is 1/2 the size of apple a and so on. each apple is compared in its row with all the apples using the smaller apple as the unit and estimating the larger as a multiple of that unit. the priority vector on the right in figure 3 is any of the column entries of the comparisons matrix normalized to one or the sum of each of the the three rows normalized using their total sum of all rows. figure 3 comparison according to volume ijahp article: saaty/the eigenvector in lay language international journal of the 166 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 how do we then obtain the relative weights of the apples from our judgments? in that case we follow what we did when we have measurements above, but because we don’t have priorities for the apples we assume they are unknowns and we need to find the true values. thus we multiply by three unknowns in place of the three priority measurements .6, .3, .1 . we note that the one in the first row and first column in this table represents the comparison of the large apple with itself, but the one in the middle position represent comparison of the medium apple with itself and the one in the third row and third column represents comparison of the small apple with itself. these ones do not convey the information about the size of the apple being compared. we must introduce the weights of the apples to make the different ones mean what they say. but as we just said, the weights are unkown and must be indicated for now by unknown variables x, y, z. thus we multiply the numbers in the first column by x, those in the second second column by y and those in the third column by z. now we can add the weighted numbers in each row. following our example above we require that the solution should be a constant multiple a (whose value in the example above was equal to 3). for this example, we have for the sum of the weighted rows respectively: 2 6 3 2 6 3 and for the sum of the weighted columns we have 10 10 10 6 3 we must solve a system of four equations in four unknowns , , , . x y z ax x y z ay x y z az x y z a x y z a + + = + + = + + = + + = here is how we get that solution. if we multiply the second equation by two and subtract the resulting equation from the first equation we get 2 x y = . if we multiply the third eqaution by six and subtract the resulting equation from the first equation, we get . 6 x z = if we now normalize by dividing each of x, y, z by the sum of their values we get: 6 10 2 6 3 2 10 1 6 10 and from the fourth eqaution we have on substituting these values for x,y and that 3. x x x xx y z x x y x z z a = = + + + + = = = = = ijahp article: saaty/the eigenvector in lay language international journal of the 167 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 the three variables together (.6, .3, .1) define what is known in mathematics as an eigenvector of the matrix of the numerical coefficients found in the first three equations, and the constant a is called an eigenvalue of that matrix. note: (saaty, 1996) it is known in mathematics that an n x n matrix has n eigenvalues and n corresponding eigenvectors. the eigenvalues may not be distinct and in general are often complex conjugates. when a matrix has positive entries, perron’s theory assures us that the matrix has a real positive eigenvalue that dominates all other eigenvalues in modulus and a corresponding eigenvector that has positive entries. these are respectively called the principal eigenvalue and the principal eigenvector of that matrix. perron-frobenius theory extends the idea to non-negative matrices with slight modification on the dominance of the principal eigenvalue over the other eigenvalues and the entries of the eigenvector, some of which may now be zero. frobenius further showed that when a matrix is “irreducible” and thus has a block of zeros of the form:       cb a 0 the principal eigenvalue need not strictly dominate the moduli of the other eigenvalues but may be equal to them. 3. a second interpretation the priority of the importance of each apple according to its relative is the sum of all the numbers that represent the judgments in its row of the comparisons. these numbers indicate how much it dominates every other element. but all the elements are not equally important. if we know how important they are we would use that priority of importance to weight each judgment by it and then add the weighted numbers in each row and get these priorities back. not knowing the priorities, we assume that all the elements are equally important and use the same constant number to weight the judgments in each row and add over that row. doing that, we get a first estimate of the priorities. this estimate is the exact priorities when we have measurements. we use this first estimate of the priorities to weight the judgments in each row and add the weighted numbers in each row to get a new estimate of the priorities of the elements. we stop if the first set of priorities is identical to the second set. otherwise, again we use this second set of priorities to weight the judgments in each row and add the weighted numbers to get a third estimate. we continue the process until the last estimate of the priorities is close enough for our need of accuracy to the one before it. now we have the priorities we are looking for. computing the eigenvector does exactly what we just described above. 4. ratio scales and interval scales as utilities some researchers of multiple-criteria decision-making prefer to obtain the priorities using the utility values instead of the eigenvector. some of them are critics of the eigenvector, because of a misunderstanding according to harker & vargas (1987) of the theoretical foundation of the ahp or reluctance to move away from traditional methods used in multi-attribute utility theory which uses interval instead of the customary ratio scales used for measurement in science. to convert measurements from a ratio scale to an interval scale, take any column of priorities and identify the smallest among the three ijahp article: saaty/the eigenvector in lay language international journal of the 168 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 values, subtract this value from each of the three values, then divide by the maximum value among the resulting three values as illustrated in the three columns of figure 4. several ratio scale values can lead to the same interval scale value resulting in loss of information. weight [in ounces, a ratio scale measurement] normalized eigenvector utility apple a 6 0.6 (0.6 0.1)/0.5 = 1 apple b 3 0.3 (0.3 0.1)/0.5 = 0.4 apple c 1 0.1 (0.1 0.1)/0.5 = 0 figure 4 converting ratio scale measurements to interval scale measurements let us suppose that apple a is rotten and someone replaces it with apple d, weighing 4 oz; the new values for the eigenvector and utility priorities are shown in figure 5. weight [ounces] normalized eigenvector utility apple d 4 0.500 1 apple b 3 0.375 0.667 apple c 1 0.125 0 figure 5 replacing apple a with apple d ijahp article: saaty/the eigenvector in lay language international journal of the 169 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 as we can see, only the utility of the weight of apple b will be changed from 0.4 to 0.667, so the heaviest and the lightest apples will have their utility unchanged, even though the heaviest now weighs 50% less. the eigenvector is more sensitive to capture and portray this kind of change that happens often in real world decisions where factors are measured in days, dollars, people and so on. references saaty, t. l., (1996). decision making with dependence and feedback: the analytic network process. pittsburgh, pa: rws publications. harker, p.t. & l.g. vargas (1987). the theory of ratio scale estimation: saaty’s analytic hierarchy process. management science, v. 33, n. 11, 1383-1403. ijahp: ferretti/university of zagreb awards enrique mu friend of the faculty of organization and informatics international journal of the analytic hierarchy process 145 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.324 university of zagreb awards enrique mu, ijahp editor-in-chief, a friend of the faculty of organization and informatics valentina ferretti it is both a pleasure and an honor to announce and celebrate an important award that the editor-in-chief of the international journal of the analytic hierarchy process recently received. on december 18, 2014 enrique mu, associate professor at the carlow university, co-director of the mba program and editor-in-chief of the ijahp, was honored by the university of zagreb in a live ceremony using a special video link as a "friend of the faculty of organization and informatics". this award is a charter that was established 5 years ago to recognize people whose relationship with the university exceeds the academic or research realm, and who become a friend of the institution. this is the first time the award has been given to a u.s.-based scholar. “we find you a very important person in the process of the faculty of organization and informatics (foi) development, especially in research and international project cooperation and networking. you were ready to host our researchers, you were a devoted mentor of our phd students, you supported mobility of our students, you did a lot on international affirmation of our faculty, you inspire us when we think about our strategy, some of us were kindly received at your university...”, said vjeran strahonja, the dean of foi at the university of zagreb, in an e-mail announcing the award. “i was nicely surprised when i received the notice of this award from the university of zagreb. i have worked on different research projects with faculty and students there for several years, and this has led to a mutual respect and friendship, which is what this recognition is about. it was an honor to receive this award”, said enrique. warm congratulations to enrique from all of us! pictures from the award ceremony that took place on december 18, 2014, at the croatian national theatre in varazdin, croatia 2_lamifinal_dec_23_p_91-105 ijahp article: lami, e. masala, s. pensa/ analytic network process (anp) and visualization of spatial data: the use of dynamic maps in territorial transformation processes. international journal of the analytic hierarchy process vol. 3 issue 2 2011 issn 1936-6744 92 analytic network process (anp) and visualization of spatial data: the use of dynamic maps in territorial transformation processes isabella m. lami department of housing and city politecnico di torino torino, italy e-mail: isabella.lami@polito.it elena masala department of technology communication and society università degli studi guglielmo marconi roma, italy e-mail: elena.masala@polito.it stefano pensa department of architectural and industrial design, politecnico di torino, torino, italy e-mail: stefano.pensa@polito.it abstract multiple criteria decision analysis (mcda) is generally used to support planning and decision making processes, but the sharing of information is often limited by two main factors. first, many users have difficulty reading output data, especially tables, matrixes or databases. second, the presence of many actors introduces different disciplines, knowledge, interests and languages. this paper shows the contribution that the visualization of spatial data can give to the analytic network process (anp) (saaty, 2005; 2001; saaty, vargas, 2006). this study aims to use visualization techniques to create a common grammar among actors involved, and a shared basis for generating discussion. to do this we have implemented on-going research on a modelling system which is able to visualize various kinds of data in real time. this modelling system works on generative and parametric features applied to datasets, and is based on mcneel’s “rhinoceros” software and its free plug-in "grasshopper". the starting point of the work described here is an application on a german section of corridor 24, genoa-rotterdam, part of an interreg ivb new project, called “code24”. the goal of this anp application is to rank three scenarios of improvement of the "betuwe" railway line connecting oberhausen to the dutch borders. the research resulted in an academic internal application which allowed us to improve the system for use in real-world focus groups which included the main stakeholders of these territorial transformation projects. keywords: geo-visualization, analytic network process, dynamic maps, parametric and generative modelling rob typewritten text http://dx.doi.org/10.13033/ijahp.v3i2.109 ijahp article: lami, e. masala, s. pensa/ analytic network process (anp) and visualization of spatial data: the use of dynamic maps in territorial transformation processes. international journal of the analytic hierarchy process vol. 3 issue 2 2011 issn 1936-6744 93 1. introduction the use of anp to study alternative planning solutions in real decision-making arenas highlighted areas for potential improvement. first, the assignment of weights to clusters and nodes requested by anp could create misunderstandings due to a lack of ability of non-expert users to comprehend their meaning. second, a more visual grammar can generate a common basis for sharing information and allowing discussions. therefore, the indexes, weights and rankings from anp must be an object of discussion as well as their results. they are not part of a black box, (latour, 1987) but also need to be assessed by the decision-makers (dm). third, the large quantity of data to manage during the decision-making process has highlighted the necessity to filter items in order to better identify and isolate core features. in the last twenty years many authors have studied new methodologies for improving the readability of multi-criteria analysis through integration with gis (geographic information system) technologies (malczewski, 2006). in 1988 diamond and wright addressed this topic which opened an international debate that transcends various disciplines. in particular, this research refers to the integration of mcda with tools for the visualization of spatial data, such as gis data and cad drawings. the visualization of spatial data, also known as geo-visualization, is a branch of cartographic science and is defined as a technique for the exploration of spatial and spatio-temporal data through the use of interactive tools (andrienko et al, 2007). in its research agenda for 2011-2015, the commission on geovisualization of ica (international cartographic association) recommends the use of interactive tools to improve knowledge discovery (http://geoanalytics.net/ica/). in the international landscape, interaction between user and model is a common goal, but only a few studies come close to obtaining a result able to combine data exploration, multicriteria analysis and geovisualization. one example of a paper that accomplished this goal is the “geoviz toolkit” (http://www.geovista.psu.edu/geoviztoolkit/index.html) proposed by the geovista center of pennsylvania state university. it offers a useful system for analyzing and mining data on the basis of their geographical position, but it works only on flat maps. meanwhile, literature on geovisualization provides different examples of threedimensional models coming from gis data. these representations are mainly based on generative modelling (van den brink, et al., 2007), which results in the automatic production of three-dimensional volumes directly from databases, model libraries or through the extrusion of specific database attributes. many tools provide these types of spatial data visualization such as “community viz” (orton family foundation and placeways, llc) and “metroquest” (envision sustainability tools inc.), but use it primarily for project presentation, instead of data exploration during the planning process. there is a lack of systems able to integrate the generation of 3d volumes and tools which make use of parametric functions. to address these issues, a new study has begun to implement on-going research on a modelling system which visualizes various kinds of data in real time. information and communication technologies (ict) can provide a large number of tools of different ijahp article: lami, e. masala, s. pensa/ analytic network process (anp) and visualization of spatial data: the use of dynamic maps in territorial transformation processes. international journal of the analytic hierarchy process vol. 3 issue 2 2011 issn 1936-6744 94 typologies which offer the possibility to visualize numerous data and relate them to specific features. since most of these tools are underused, existing software has been investigated in order to find those best fitted to multidisciplinary processes, and no new tools have been created. this modelling system is based on mcneel’s “rhinoceros” software and its plug-in "grasshopper", and works by applying generative and parametric features to datasets. although this plug-in has been created to support industrial production and generative architectures, this research aims to use it as a support to decision making processes on large areas. in particular, the grasshopper plug-in offers new opportunities in planning. it implements generative and parametric functions, which allows the system to work in real time, relating shapes with behavioural formulas, and supporting large microsoft excel databases. additionally it can be implemented with plug-ins and scripts in visual basic language, which are available for free on the web. 2. combining anp with geo-visualization in order to successfully merge the anp technique and three-dimensional model technology, three key issues needed to be solved: • the translation of qualitative information into spatial form; • the building of a framework of weighted relations among the spatial items which represent the nodes, clusters and subnets of anp structure; • the interaction between the model and its users. during planning processes, many aspects of decision-making are referred to specific areas and have a well-defined spatial form and localization. these elements and their influence on land can easily be represented in a three-dimensional model. for example, a new road or a residential zone or the geometric features included in gis data can be characterized in a three-dimensional model. on the other hand, economic or social issues do not have this direct connection with spatial form, and still need to be represented coherently. to tackle this problem, geo-visualization offers many techniques (tufte, 2001) which make use of symbolic features applied to spatial areas. in this instance, geo-visualization is used to support mcda, by building an intuitive language that can enhance the cognitive process of the actors involved (maceachren, et al., 2004). data knowledge process is then implemented by two key factors: the localization of information, which defines the spatial position of data; and the cause-effect relationship that results between the actor’s decisions and the spatial forms that mutate in real-time, in accordance with the changing intent of the decision makers. anp works on weights assigned to relations among nodes, clusters and subnets, therefore, this modelling system must be implemented in order to generate spatial forms on the basis of weighted relations among different components. for this reason, the same framework of relations and weights among anp elements has been reproduced to set up the 3d model. one map is created for every anp node, where the node is represented by ijahp article: lami, e. masala, s. pensa/ analytic network process (anp) and visualization of spatial data: the use of dynamic maps in territorial transformation processes. international journal of the analytic hierarchy process vol. 3 issue 2 2011 issn 1936-6744 95 a symbolic mode of visualization. mathematical formulas, which can be complex, can be applied to visualize spatial behaviours since not all factors have a direct form of representation. in this way the influence of an element on the form of space can be determined with specific equations derived from different fields. each map changes according to the weights assigned to its relation with the other nodes, clusters and scenarios. this structure of relations is very complex, but allows the visualization of the spatial effect of each node. furthermore, the modelling system can provide a map for each level of anp structure, allowing them to overlap and be compared. in this system, users can interact with the model, which can visually update its form on the basis of dm's input. if during a focus group, weights and/or mathematical formulas of spatial behaviours are changed, the system will update its outputs in real time, offering an important tool for explaining the cause-effect relationships between decisions and their spatial results. 3. application to the study case 3.1 presentation of the case anp and geovisualization tools have been used to study a german section of corridor 24, genoa-rotterdam. the connectivity of the german railway system between the german city of oberhausen and the dutch border in the ruhr region needs to be upgraded. in the framework of corridor 24 development, the changes in the transport system across the border of the netherlands and germany imply a new spatial configuration for the areas of northwest germany. therefore, the case study analyzes various possibilities for upgrading the connectivity in this area. the entire spatial dimension is very large, so the study analysis will be conducted by proposing identical scenarios for smaller areas related to focus groups. this paper shows the application of anp and geovisualization tools beginning with a study of the area of wesel, in which four partners of the "code24 project" were involved: siti politecnico di torino (italy), eth of zurich (ch), university of duisburg-essen (d) and universiteit of utrecht (nl). three alternatives have been considered in applying anp (table 1). ijahp article: lami, e. masala, s. pensa/ analytic network process (anp) and visualization of spatial data: the use of dynamic maps in territorial transformation processes. international journal of the analytic hierarchy process vol. 3 issue 2 2011 issn 1936-6744 96 table 1 alternatives for improving the rail line in the ruhr region alternatives characteristics scenario 0 improvement of existing "betuwe" railway line: no new railway is created; railway signalling is increased in order to augment the number of trains. scenario 1 addition of a third track in the german stretch of betuwe railway line, which runs through many towns, making noise barriers for the passage of high speed traffic and the elimination of many grade crossings. scenario 2 freight transport abandons railway passage through venlo, and uses the betuwe railway line to wesel, where an existing but unused by-pass has to be restored in order to connect wesel with both its port on the rhine and oberhausen through the countryside. 3.2 structure of network according to the literature review and problem analysis (saaty, 2005; bottero et al, 2008), the decision problem has been divided into four clusters (environmental aspects, economic aspects, transport aspects and urban planning aspects) that have been organized according to the benefits, opportunities, costs, risks (bocr) model. to obtain spatial visualizations of bocr, a model was built using grasshopper based on drawings, databases and gis data such as infrastructures, built areas and geographical features. the model creates relationships among all the elements in order to reproduce the anp structure as a “flow chart”. it determines the rules of the parametric model, and assigns spatial behaviour to each node. this presents many questions that need to be answered drawing from the knowledge of experts from several fields. in this test case, we do not aim to obtain a fully functional model, but to investigate how the merging of anp with this modelling system can effectively support participative and collaborative processes. therefore, even if the modelling system can make use of complex mathematical formulas, the initial phase of research will only employ simple rules given by the symbolic representation of phenomena, as indicated by an internal survey (table 2). to simplify this test case, a symbolic map, acting on a 50m x 50m grid, has been assigned to each node of anp framework. nodes with an identified spatial behaviour generate an influence depending on distance, while "non-spatial" nodes (such as most of the economic nodes) produce constant maps which cover the whole area. this test case makes use of two kinds of maps: a bi-dimensional one in which colour gradient is the indicator of the weight and importance of each node, and a 3d visualization one based on the extrusion of symbolic maps. anp inquiries place the weight of a node’s importance on a numerical scale. each map changes according to the weights assigned to its relation with other nodes, clusters and scenarios. the resulting representation is a deformation of land, acting in real time that creates a 3d diagram which expresses the weights assigned to each node. in order to increase the number of ways for communicating information, other visualizations that combine 3d models with, for example, colour and buffer areas, are also possible. ijahp article: lami, e. masala, s. pensa/ analytic network process (anp) and visualization of spatial data: the use of dynamic maps in territorial transformation processes. international journal of the analytic hierarchy process vol. 3 issue 2 2011 issn 1936-6744 97 table 2 clusters and nodes and symbolic modes of visualization used in the test case bocr cluster elements map benefits environment reduction in traffic emissions in urban areas linear buffer along main roads economic improving the economic role of the region constant value new employment due to the improvement of transport constant value transport increase in frequency of connection radial buffer on railway stations creation of a freight hub, by intersecting rail, road and river transport, connected to wesel port land use increase in the capacity of freight transport linear buffer along main roads and railways opportunities economic possible creation of new jobs indirectly related to the improvement of the regional transport system constant value increase in trade constant value transport possible river connections to berlin for freight transport linear buffer along main waterways and radial buffer centred on port increase in the capacity of the transport of people radial buffer on railway stations urban planning promotion of new forms of settlement along the enhanced track linear buffer along railways elimination of grade crossings radial buffer on grade crossings costs economic investment costs constant value acquisition/expropriation of areas for the insertion of new track linear buffer along railways environment noise and vibration impacts linear buffer along railways and main roads urban planning visual impact of trains linear buffer along railways possible creation of large barriers in landscape linear buffer along railways risks economic possible extensions of implementation time due to conflicts arising with the local population constant value decrease in property values linear buffer along railways environment soil consumption/land consumption linear buffer along railways 3.3 resulting visualizations for this first application, visualizations have been used to show the anp results referring to subnets and scenarios. the bocr model allowed users to visualize the behaviours and tendencies for each scenario. in particular, it proves to be effective when used with a slicing plane which eliminates the lowest values and displays output with more potential. the visualization of each subnet describes the effects on space, and highlights the characteristics of each scenario. figure 1 shows the benefits network. ijahp article: lami, e. masala, s. pensa/ analytic network process (anp) and visualization of spatial data: the use of dynamic maps in territorial transformation processes. international journal of the analytic hierarchy process vol. 3 issue 2 2011 issn 1936-6744 98 figure 1 benefits network according to the anp methodology, after having structured the decision network, the second step of the analysis consists in filling in the pairwise comparison matrices for each subnet. it is important to highlight that there are two levels of pairwise comparisons in the anp: the cluster level, which is more strategic, and the node level, which is more specialized. in pairwise comparisons, a ratio scale of 1-9, that is the saaty’s fundamental scale, is used to compare any two elements. the main eigenvector of each pairwise comparison matrix represents the synthesis of the numerical judgements established at each level of the network (saaty, 1980). alternatives scenario 0 scenario 1 scenario 2 transport aspects creation of a freight hub increase in frequency of connection increase in the capacity of freight transport environmental aspects reduction in traffic emissions in urban areas economic aspects improving the economic role of the region creation of employment directly related to the transport improvement upgrading connectivity between essen and the dutch borders opportunities s costs risks s benefits ijahp article: lami, e. masala, s. pensa/ analytic network process (anp) and visualization of spatial data: the use of dynamic maps in territorial transformation processes. international journal of the analytic hierarchy process vol. 3 issue 2 2011 issn 1936-6744 99 considering the pairwise comparison at the cluster level, the following questions were generated: with respect to the choice of alternatives (i.e. the three options to upgrade the connectivity between essen and the dutch border), in your opinion, what aspect is more important, and how much more?” economic aspects 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 environmental aspects economic aspects 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 transport aspects environmental aspects 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 transport aspects from the cluster level comparison in subnet benefits, it is possible to derive the cluster matrix (table 3). in the case under examination, economic aspects are given the highest importance (0,740), followed by the transport aspects (0,166) and finally by the environmental aspects (0,094). table 3 cluster matrix alternatives economic aspects environmental aspects transport aspects alternatives 0,000 0,500 1,000 1,000 economic aspects 0,740 0,000 0,000 0,000 environmental aspects 0,094 1,000 0,000 0,000 transport aspects 0,166 0,500 0,000 0,000 once the clusters comparison has been conducted, it is necessary to study the problem in depth through the analysis of the elements. considering the pairwise comparison at the node level, the following questions were generated: “with reference to scenario 2 (freight transports leave the railway passing by venlo and use the betuwe rail line until wesel, where an existing but unused by-pass has to be restored in order to connect wesel with both its port on rhein and oberhausen through the countryside), in your opinion, what transport aspect is more important and how much more? ijahp article: lami, e. masala, s. pensa/ analytic network process (anp) and visualization of spatial data: the use of dynamic maps in territorial transformation processes. international journal of the analytic hierarchy process vol. 3 issue 2 2011 issn 1936-6744 100 creation of a freight hub 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 increase in frequency of connections creation of a freight hub 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 increase in capacity of freight transport increase in frequency of connections 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 increase in capacity of freight transport the weight assigned shows that the most appreciated characteristic of scenario 2 is the creation of a freight hub, by intersecting rail, road and river transport, connected to wesel port. this score is consistent with the current trends which highlight intermodal transport. the increase in capacity of freight transport is considered more important, however slightly, than the frequency. the totality of the eigenvectors, which are derived from the pairwise comparison matrixes of the model elements, forms the unweighted supermatrix. the cluster matrix is then applied to the unweighted supermatrix as a cluster weight and the result is the weighted supermatrix (table 4). table 4 weighted supermatrix (benefits) alternatives economic aspects environmental aspects transport aspects option 0 option 1 option 2 employment economic role traffic emission reduction freight hub connections frequency alternatives option 0 0,000 0,000 0,000 0,055 0,055 0,072 0,077 0,200 option 1 0,000 0,000 0,000 0,290 0,154 0,279 0,231 0,200 option 2 0,000 0,000 0,000 0,655 0,290 0,650 0,692 0,600 economic aspects employment 0,123 0,185 0,123 0,000 0,000 0,000 0,000 0,000 economic role 0,616 0,555 0,616 0,000 0,000 0,000 0,000 0,000 environmental aspects traffic emission reduction 0,094 0,094 0,094 0,000 0,000 0,000 0,000 0,000 transport aspects freight hub 0,011 0,015 0,124 0,000 0,000 0,000 0,000 0,000 connections frequency 0,078 0,076 0,018 0,000 0,000 0,000 0,000 0,000 figure 2 shows the benefits subnet visualisation, according to the weighted matrix. the heavy weight of the economic cluster, represented by constant values, is visible in the large difference which occurs in the level height among scenarios. scenario 1 is more useful in improving people mobility: the maps of stations, which symbolize the increase in frequency of connection, and maps of the road network, which represent the reduction ijahp article: lami, e. masala, s. pensa/ analytic network process (anp) and visualization of spatial data: the use of dynamic maps in territorial transformation processes. international journal of the analytic hierarchy process vol. 3 issue 2 2011 issn 1936-6744 101 of traffic emissions, are the weightiest. on the other hand, scenario 2 shows its propensity to trade and economic development due to the importance of the creation of a logistic hub. figure 2 visualization of benefits by 3d diagrams: scenario comparison. the same tendencies of benefits maps are visible in the subnet of opportunities (figure 3). the elimination of the grade crossing in scenario 1 and the river connection in scenario 2 confirms that the latter is more indicated for people mobility while the former is more indicated for economic development. figure 3 visualization of opportunities by 3d diagrams: scenario comparison ijahp article: lami, e. masala, s. pensa/ analytic network process (anp) and visualization of spatial data: the use of dynamic maps in territorial transformation processes. international journal of the analytic hierarchy process vol. 3 issue 2 2011 issn 1936-6744 102 the costs subnet shows that scenario 1 is more expensive than scenario 2 (figure 4). scenario 0 has a smaller investment cost, but it is characterized by the lack of noise barriers shown by a sequence of peaks along the railway due to visual and vibration impact of trains. figure 4 visualization of costs by 3d diagrams: scenario comparison. it is important to emphasize that the modelling system permits the visualization of the assessed impact of each cluster, considering the three scenarios separately. visualisation allows the perception of the impact problem components to be considered during a focus group and facilitates understanding by all stakeholders. for example, figure 5 illustrates for the subnet of costs the perception of the distribution of economic costs (investment costs and acquisition/expropriation of areas for the insertion of new track) arising on the territory in the three scenarios. in this specific visualisation, when only the financial costs are taken into account, it would seem that scenario 0 has no costs, while, in figure 4 where all the aspects of the problem are considered, it becomes apparent that there are costs along the railway, due to noise and visual impact. ijahp article: lami, e. masala, s. pensa/ analytic network process (anp) and visualization of spatial data: the use of dynamic maps in territorial transformation processes. international journal of the analytic hierarchy process vol. 3 issue 2 2011 issn 1936-6744 103 figure 5 visualization of the economic cluster in costs subnet by 3d diagrams: scenario comparison also in the risks subnet (figure 6), scenario 0 is strongly influenced by the lack of noise barriers. in both scenarios 1 and 2, the extension of implementation time determines the increase of risks, but in scenario 2, the use of an old track and the bypass of the urban area could reduce the possibility of local conflicts. figure 6 visualization of risks by 3d diagrams: scenario comparison the modelling system also provides the ability to compare scenarios using different combined views of subnets. for example, the switching of benefits and costs maps (figure 7) shows that the lack of barriers in scenario 0 causes the costs to outweigh the benefits. in scenario 1, the benefits are greater than the costs, but there is a clear conflict ijahp article: lami, e. masala, s. pensa/ analytic network process (anp) and visualization of spatial data: the use of dynamic maps in territorial transformation processes. international journal of the analytic hierarchy process vol. 3 issue 2 2011 issn 1936-6744 104 along the rail tracks as a consequence of the weight of the urban planning cluster. only in scenario 2 do benefits completely outweigh costs. figure 7 comparison of the visualization of benefits and costs finally, according to the anp methodology, the weighted supermatrix was raised to a limiting power to obtain the limit supermatrix, where all columns were identical and each column gave the global priority vector. in this case, four limit supermatrices were obtained, one for each subnetwork. the analysis development for the four subnets leads to the identification of the final priorities of all the elements of the model. in order to obtain the ranking of the alternatives, it is necessary to synthesize the raw priorities obtained from the limit supermatrix by normalizing them. in the case of the bocr network structure, it is necessary to synthesize the outcome of the alternative priorities for each of the four subnets in order to obtain an overall synthesis. different aggregation formulas are available; figure 8 shows the final ranking of the alternative scenarios according to additive (reciprocal) formula. ijahp article: lami, e. masala, s. pensa/ analytic network process (anp) and visualization of spatial data: the use of dynamic maps in territorial transformation processes. international journal of the analytic hierarchy process vol. 3 issue 2 2011 issn 1936-6744 105 figure 8 final ranking of the alternatives the model shows that the best scenario for upgrading the connectivity of the areas of northwest germany is the one which exploits the betuwe railway line to wesel, where an existing but unused by-pass has to be restored in order to connect wesel with both its port on the rhine and oberhausen through the countryside. 4. conclusion this paper presents a new approach to the integration of visualization with the analytic network process. the anp methodology is capable of taking both tangible and intangible criteria into consideration without sacrificing their relationships, and it can deal with various interdependencies systematically. this is particularly important for evaluating urban and regional transformation processes, as in the real-world case presented here. the paper tries to take a further step towards facilitating the dm in handling all of the data collected during the decision process, and to analyze the influences among the different elements of the system as perceived by the same dm, using specific 2d and 3d maps created by the modeling system. the application of this evaluation tool shows that no other new software is needed, but rather, a more creative use the existing ones. the results obtained are sufficiently promising, even if many questions remain to be answered. the spatial visualization of the anp application is coherent with input choices. parametric and generative features of the modelling system for symbolic spatial visualization conform well to the use of weights in anp. the weights assigned to the elements of bocr analysis were related to spatial forms and represented through dynamic maps. this means that the spatial model correlates quantitative and qualitative values with their effects on spatial form, providing a 3d spatial localization of planning choices. it constantly updates its form reflecting the effects that weights assigned in anp have on spatial form. in particular, the ability to change the weights and relations among bocr elements in real time makes this tool an interesting resource for generating discussion in multi-actor processes, such as territorial and urban contexts. ijahp article: lami, e. masala, s. pensa/ analytic network process (anp) and visualization of spatial data: the use of dynamic maps in territorial transformation processes. international journal of the analytic hierarchy process vol. 3 issue 2 2011 issn 1936-6744 106 references andrienko, g., andrienko, n., jankowski, p., keim, d., kraak, m.j., maceachren, a.m., & wrobel, s. (2007). geovisual analytics for spatial decision support: setting the research agenda. international journal of geographical information science, 21(8), 839857. bottero, m., lami, i. m., & lombardi, p. (2008). analytic network process. la valutazione di scenari di trasformazione urbana e territoriale. firenze: alinea. diamond, j.t., & wright j. (1988). design of an integrated spatial information system for multiobjective land use planning. environment and planning b: planning and design,15, 205–214. lami, i. m., masala e., & pensa s. (2011). la visualizzazione spaziale delle analisi multicriteria: un caso applicativo nel processo decisionale, xxxii conferenza italiana di scienze regionali, torino, 15-17 settembre. latour, b. (1987). science in action: how to follow scientists and engineers through society. cambridge, ma: harvard university press. maceachren, a.m., gahegan, m., pike, w., brewer, i. (2004). geovisualization for knowledge construction and decision-support. computer graphics & applications, 24(1), 13-17. malczewski, j. (2006). gis-based multicriteria decision analysis: a survey of the literature. international journal of geographical information science, 20(7), 703-726. saaty t.l. (2005). theory and applications of the analytic network process. pittsburgh, pa: rws publications. saaty, t. l. (2001). the analytic network process: decision making with dependence and feedback. pittsburgh, pa: rws publications. saaty, t l, vargas, l g (2006). decision making with the analytic network process. new york: springer science. tufte, e.r. (2001). the visual display of quantitative information. cheshire, ct: graphics press. van den brink, a. et al. (2007). geo-visualization for participatory spatial planning in europe: imaging the future, wageningen academic publishers, mansholt publication series vol. 3, wageningen (nl). http://geoanalytics.net/ica/ http://www.geovista.psu.edu/geoviztoolkit/index.html ijahp article: dave, desai, raval/a decision support system for tool electrode selection international journal of the analytic hierarchy process 89 vol. 4 issue 2 2012 issn 1936-6744 a decision support system for tool electrode selection for electro discharge machining process using the analytic hierarchy process harshit k. dave department of mechanical engineering sardar vallabhbhai national institute of technology surat, india email: harshitkumar@yahoo.com keyur p. desai department of mechanical engineering sardar vallabhbhai national institute of technology surat, india harit k. raval department of mechanical engineering sardar vallabhbhai national institute of technology surat, india abstract the shape and accuracy of any part machined using the electro discharge machining process depends primarily on the shape and accuracy of the tool or the cutting electrode. hence, the selection of tool electrodes in electro discharge machining process becomes an important task. theoretically, any material that is a good conductor of electricity can be used as a tool with varying advantages. it is very difficult to find a single material that possesses all the desirable characteristics, and therefore the selection becomes a difficult task. the present paper proposes a methodology based on the analytic hierarchy process (ahp) for selection of a tool electrode for the electro discharge machining process. based on the ahp method, an electrode selection index (esi) is found for all the alternatives that are considered in the present study. this esi helps to evaluate and rank any given number of alternative materials. the results of the present study suggest that graphite and copper are the best materials for fabricating a tool electrode in edm process. keywords: tool electrodes, analytic hierarchy process, electrode selection index mailto:harshitkumar@yahoo.com rob typewritten text http://dx.doi.org/10.13033/ijahp.v4i2.131 ijahp article: dave, desai, raval/a decision support system for tool electrode selection international journal of the analytic hierarchy process 90 vol. 4 issue 2 2012 issn 1936-6744 1. introduction in an electro discharge machining (edm) process the shape and accuracy of the machined part depends largely on the shape and accuracy of the tool electrode. no matter how accurate the machine is, the work produced by edm process can never be more accurate than the electrode that machines the work. also, the tool electrode is the means of providing electrical energy to the work material, and therefore should be good conductor of electricity. the heat that the tool receives must be dissipated away quickly in order to reduce surface temperature and tool wear. hence, it is necessary that the tool electrode is made of highly thermal conductive material. it is also evident that the tool electrode should have high melting point to reduce its wear rate. it has been observed that during edm process, both the work piece and tool electrode get eroded. hence, the accuracy of the machined part depends on electrode wear. the amount of erosion suffered by the electrode as compared to that of work material is referred to as the wear ratio. the wear ratio depends on the physical and chemical properties of the electrode and work material as well as environmental conditions. hence, the basic desirable characteristics of a tool electrode can be summarized as: (i) electrical conductivity (ii) thermal conductivity (iii) melting temperature (iv) mrr (v) wear ratio (vi) cost. (pandey and shan, 2003; mishra, 2007). a good amount of research work has been carried out in the past on materials selection [3-16]. the selection of an optimal material for an engineering design from among two or more alternative materials on the basis of two or more attributes is a multiple attribute decision-making problem. various approaches have been proposed in the past to help address the issue of material selection. liao (1996) presented a fuzzy multi criteria decision-making method for material selection. however, the method is complicated and requires a large amount of computation. farag (1997) proposed a simple mathematicsbased weighted properties method that can be used when several properties should be taken into consideration. giachetti (1998) described a prototype material and manufacturing process selection system that integrates a formal multiple attribute decision model with a relational database. ashby (2000) proposed multi-objective optimization in materials design and selection, using ‘utility’ functions. ashby et al. (2004) provided a comprehensive review of the strategies or methods for materials selection. a framework to represent and deal with the relationships between design variables of both materials parameters and system-level parameters was proposed by raj (2000) and raj et al. (2000). deng and edwards (2007) presented an overview of recent research in materials identification and materials selection. shanian and savadogo (2006) had presented a material selection model using an madm method known as electre. chan and tong (2006) proposed a multi criteria weighted average method using gray relational analysis to rank the materials. rao (2006) presented a material selection model using graph theory and the matrix approach. a ‘material suitability index’ was proposed ijahp article: dave, desai, raval/a decision support system for tool electrode selection international journal of the analytic hierarchy process 91 vol. 4 issue 2 2012 issn 1936-6744 that evaluates and ranks the materials for a given engineering component. kumar and singh (2006) presented an intelligent system for selection of materials for progressive die components. cheng et al. (2008) used the fuzzy ahp method for selection of technological forecasting methods for prediction of new materials development. rao and davim (2007) used the topsis method for selection of materials for a given application. ahp method has also been applied for selection of phase change material using fuzzy concept (2011). despite this volume of research on material selection, there is still a need for a simple, systematic, and logical scientific method or mathematical tool to guide user organizations in making a proper material selection decision with respect to tool electrode material selection in electro discharge machining process. from purely technical considerations, it is possible to specify a specific material as the most efficient tool electrode material providing high mrr with least tool wear. however, the cost of such an electrode under most conditions is very high. hence, it becomes a very difficult task to select a material based on any specific characteristics. it is necessary to make the proper choice regarding material since the tool electrode cost represents more than 50% of total machining cost. the selection of the best tool material primarily aims to achieve better material utilization. however, the tool electrode with least tool wear may not be the best selection; indeed the tool fabrication may become more complex which could offset the savings due to material economy. whatever the chosen procedure for obtaining the best alternative, it is desirable to make an optimum choice among the available materials. ideally, it is essential to choose the factors that are relevant to the particular problem at hand, and determine the relative importance of one factor over another. the actual identification of evaluating factors may involve discussions with the experts working in the areas of production and tool design. very few published studies are available that address the selection of tool material especially for electro discharge machining process. the objective of a tool electrode selection procedure is to identify the influencing factors and obtain the most appropriate combination of the factors in conjunction with the real requirement of the edm process. thus, efforts need to be extended to determine factors which influence tool electrode selection for edm process, using a logical approach, to ensure the selection of a proper tool electrode to strengthen the existing selection procedure. this objective is considered in this paper using analytic hierarchy process (ahp). ahp is a logical approach, and is proved to be useful for modeling and analyzing various types of decision making situations in many fields of science and technology (saaty, 1980; saaty, 2000). ahp deals with the problem of choosing an alternative from a set of candidate alternatives which are characterized in terms of some factors. the selection of a right tool electrode for the edm process amongst the available tool ijahp article: dave, desai, raval/a decision support system for tool electrode selection international journal of the analytic hierarchy process 92 vol. 4 issue 2 2012 issn 1936-6744 electrodes is clearly a decision making situation and hence in the present work the application of ahp, in selecting the right tool electrode, is considered. 2. analytic hierarchy process saaty (1980, 2000) has evolved the ahp which enables decision makers to represent the interaction of multiple factors in complex situations. the process requires the decision makers to develop a hierarchical structure for the factors which are explicit in the given problem, to provide judgments about the relative importance of each of these factors, and specify a preference for each decision alternative with respect to each factor. it provides a prioritized ranking order indicating the overall preference for each of the decision alternatives. an ahp hierarchy can have as many levels as needed to fully characterize a particular decision situation. a number of functional characteristics make ahp a useful methodology. these include the ability to handle decision situations involving subjective judgments, multiple decision makers, and the ability to provide measures of consistency of preference (triantaphyllou, 2000). designed to reflect the way people actually think, ahp continues to be the most highly regarded and widely used decision-making method. an advantage of the ahp over other multi criteria decision making methods is that ahp is designed to incorporate tangible as well as non-tangible factors especially where the subjective judgments of different individuals constitute an important part of the decision process. the main procedure of ahp using geometric mean method has been explained by rao (2000). the procedure is explained briefly as follows: step 1: determine the objective and the evaluation attributes. develop a hierarchical structure with a goal or objective at the top level, the attributes at the second level and the alternatives at the third level. step 2: find out the relative importance of different factors with respect to the goal or objective: 1) construct a pair-wise comparison matrix using a scale of relative importance. the judgments are entered using the fundamental scale of the ahp (liao, 1996; farag, 1997) as given in table 1. assuming n factors, the pair-wise comparison of factor i with factor j yields a square matrix a1n×n where aij denotes the relative importance of factor i with respect to factor j. in the matrix, aij = 1 when i = j and aji = 1/aij ijahp article: dave, desai, raval/a decision support system for tool electrode selection international journal of the analytic hierarchy process 93 vol. 4 issue 2 2012 issn 1936-6744 table 1 relative importance of factors relative importance (aij) description 1 equal importance of i and j 3 moderate importance of i over j 5 strong importance of i over j 7 very strong importance of i over j 9 absolute importance of i over j 2, 4, 6, 8 intermediate values 2) find the relative normalized weight (wj) of each attribute by (i) calculating the geometric mean of the ith row, and (ii) normalizing the geometric means of rows in the comparison matrix. this can be represented as: 1 1 nn i ij j gm a =   =     ∏ and a. 1 i i n i j gm w gm = = ∑ 3) calculate matrix a3 and a4 such that 3 1 2a a a= × and 4 3 / 2a a a= where [ ]1 22 , , ,.... t i na w w w w= 4) find the maximum eigenvalue λmax which is the average of matrix a4. 5) calculate consistency index, max 1 n ci n λ − = − the smaller the value of ci, the smaller the deviation from the consistency. 6) obtain the random index (ri) for the number of factors used in decision making. table 2 helps the users determine the ri value. (liao, 1996; farag, 1997; ashby, 2000). 7) calculate the consistency ratio, cr ci ri= . usually, a cr of 0.1 or less is considered as acceptable as it reflects an informed judgment which could be attributed to the knowledge of the analyst about the problem under study. ijahp article: dave, desai, raval/a decision support system for tool electrode selection international journal of the analytic hierarchy process 94 vol. 4 issue 2 2012 issn 1936-6744 table 2 random index (ri) values attributes ri 3 0.52 4 0.89 5 1.11 6 1.25 7 1.35 8 1.4 9 1.45 10 1.49 step 3: the next step is to compare the candidate alternatives pair-wise with respect to how much better each is at (more dominant) satisfying each of the factors. this is simply ascertaining how well each candidate alternative serves each factor. if there are m numbers of candidate alternatives, then there will be n number of m ×m matrices of judgments since there are n factors. construct pair-wise comparison matrices using a scale of relative importance. the judgments are entered using the fundamental scale of the ahp (liao, 1996; farag, 1997). the steps are same as that suggested under step 2. step 4: the next step is to obtain the composite weights for the alternatives by multiplying the relative normalized weight (wi) of each factor (obtained in step 2) with its corresponding normalized weight value for each alternative (obtained in step 3) and making summation over all the factors for each alternative. 3. tool electrode selection using ahp the authors propose using ahp to work on tool wear monitoring in the electro discharge machining process. various electrodes materials are available, and it is a difficult task to select any one or few from the choices. hence, ahp is applied to rank these tool electrode materials and based on the electrode selection index (esi) so obtained materials are selected to fabricate electrodes for experimentation. the materials considered are electrolyte copper (cu), copper tungsten (cuw), tungsten (w), brass (b), stainless steel (ss), aluminum (al) and graphite (g). six selection factors are identified that are relevant to the present case: cost (c), material removal rate (mrr), tool wear (tw), electrical resistivity (er), thermal conductivity (tc) and melting temperature (mt). table 3 presents the estimated quantitative values of er, tc, mt and qualitative values of c, mrr, tw. ijahp article: dave, desai, raval/a decision support system for tool electrode selection international journal of the analytic hierarchy process 95 vol. 4 issue 2 2012 issn 1936-6744 table 3 attribute data for different alternatives c mrr tw er tc mt cu high very high average 1.7 385 1083 cuw very high low very low 6.2 150 3410 w very high low very low 5.7 163 3370 b high high high 5.4 136 966 ss average average low 72 16 1400 al average low high 5.9 210 660 g average high average 6000 24 3650 cols: c= cost; mrr = material removal rate (volume/time); tw = tool wear; er = electrical resistivity ( x 10-6 ohm. cm); tc = thermal conductivity (w/mk); mt = melting temperature (oc). rows: cu = copper; cuw = copper tungsten; w = tungsten; b = brass; ss = stainless steel; al = aluminum; g = graphite now, various steps of the proposed procedure are carried out to obtain an electrode selection index (esi) as mentioned below: 3.1 calculation and consistency check of relative normalized weights step 1: the objective is to select the best material to fabricate a tool electrode from amongst a number of available materials. various selection factors are identified and these are: cost (c), material removal rate (mrr), tool wear (tw), electrical resistivity (er), thermal conductivity (tc) and melting temperature (mt). the objective, selection factors, and the decision alternatives (i.e. tool electrode materials) are arranged in a hierarchy as shown in fig. 1. figure 1 decomposition of tool electrode material problem into a hierarchy material selection factors c mrr tw er tc mt alternatives objective cu, cuw, w, b, ss, al, g ijahp article: dave, desai, raval/a decision support system for tool electrode selection international journal of the analytic hierarchy process 96 vol. 4 issue 2 2012 issn 1936-6744 to make comparative judgments, the relative importance of all possible pairs of factors with respect to the overall objective of selecting the right material for fabricating tool electrodes, is decided by consensus judgment for each pair, and their judgments are arranged into a matrix. the questions to ask when comparing two factors are: which is more important and how much more important is it with respect to the overall objective? the matrix, a16×6, of pair-wise comparison judgments on the factors is shown below. the judgments are entered using the fundamental scale of the ahp as described in table 1. c mrr tw er tc mt 6 6 1 3 3 5 5 5 1 3 1 1 2 2 3 2 1 3 2 1 3 3 3 1 1 5 1 2 1 3 1 2 1 1 5 1 3 1 3 1 2 1 1 2 1 5 1 2 1 3 1 2 1 a ×         =           tw is moderately more important than mt. decreasing tw is comparatively moderately more important than increasing mt. in order to reduce the total cost, the value of tw should be decreased. so, a relative importance value of 3 is assigned to tw over mt (i.e. a35 = 3) and a relative importance value of 1/3 is assigned to mt over tw. (i.e. a53 = 1/3). mt and er are considered equally important and a relative importance of 1 is assigned to mt over er (i.e. a46 = 1) and relative importance of 1/1 is assigned to er over mt (i.e. a64 = 1). similarly, the relative importance among other factors can be explained. it may be added that these values can be decided by the user/experts depending on the requirements. these values are to be arrived at judiciously after careful analysis. the assigned values in this paper are specifically applicable to the problem under discussion. the next step is to find out the relative normalized weight (wi) of each factor by calculating the geometric mean of the ith row and normalizing the geometric means of rows in the comparison matrix. ( ) 1 6 1 1 3 3 5 5 5 3.225gm = × × × × × = ( ) 1 6 2 1 11 2 3 2 1.12253 2gm = × × × × × = ( ) 1 6 3 1 2 1 3 3 3 1.61893gm = × × × × × = c mrr tw er tc mt ijahp article: dave, desai, raval/a decision support system for tool electrode selection international journal of the analytic hierarchy process 97 vol. 4 issue 2 2012 issn 1936-6744 ( ) 1 6 4 1 1 1 1 2 1 0.63685 2 3gm = × × × × × = ( ) 1 6 5 1 1 1 1 11 0.42085 3 3 2 2gm = × × × × × = ( ) 1 6 6 1 1 1 1 2 1 0.63685 2 3gm = × × × × × = now, 5 1 i i i i gm w gm = = ∑ gives w1 = 0.421, w2 = 0.1465, w3 = 0.2113, w4 = 0.0831, w5 = 0.0549, w6 = 0.0831 these relative normalized weights are written in matrix form as follows: 6 1 0.421 0.1465 0.2113 2 0.0831 0.0549 0.0831 a ×         =           matrix 6 13a × is calculated as 6 1 6 6 6 13 1 2a a a× × ×= × . 6 1 2.5999 0.8896 1.3079 3 0.5039 0.3415 0.5039 a ×         =           matrix 6 14a × is calculated as 6 1 6 1 6 1 3 4 2 a a a × × × = . 6 1 6.1755 6.0724 6.1898 4 6.0638 6.2204 6.0638 a ×         =           ijahp article: dave, desai, raval/a decision support system for tool electrode selection international journal of the analytic hierarchy process 98 vol. 4 issue 2 2012 issn 1936-6744 now, the maximum eigenvalue maxλ which the average of matrix a4 is determined as shown below: ( )6 6 1 max 1 4 6 i i a λ × =   =     ∑ 6.1755 6.0724 6.1898 6.0638 6.2204 6.0638 6 + + + + +  =    6.131= now, consistency index (ci) is calculated as shown below: ( ) ( ) ( ) ( ) max 6.131 6 0.0262 1 6 1 m ci m λ − − = = = − − the value of random index (ri) for 6 factors is taken from table 2 which is 1.25 in the current case. the consistency ratio (cr) is calculated as 0.0262 1.25 0.02096cr ci ri= = = this value is less than the allowed cr of 0.1 and hence the value is acceptable. thus, there is a good consistency in the judgments made in the current example. 3.2 normalization of attribute data of different alternatives mrr, tc and mt are beneficial factors and higher values of these factors are desirable. c, tw and er are non-beneficial factors and lower values of these factors are desirable. c, mrr and tw are assigned qualitative values. these qualitative values are assigned values as mentioned in table 4. ijahp article: dave, desai, raval/a decision support system for tool electrode selection international journal of the analytic hierarchy process 99 vol. 4 issue 2 2012 issn 1936-6744 table 4 conversion of qualitative to quantitative value of factors qualitative measure assigned value very low 0.1 low 0.3 average 0.5 high 0.7 very high 0.9 hence, table 5 is obtained that has all the attributes in quantitative form. table 5 attribute data in quantitative form c mrr tw er tc mt cu 0.7 0.9 0.5 1.7 385 1083 cuw 0.9 0.3 0.1 6.2 150 3410 w 0.9 0.3 0.1 5.7 163 3370 b 0.7 0.7 0.7 5.4 136 966 ss 0.5 0.5 0.3 72 16 1400 al 0.5 0.3 0.7 5.9 210 660 g 0.5 0.7 0.5 6000 24 3650 normalization of beneficial attributes is done by assigning the value ‘1’ to the highest value, and then the rest of the normalized values are obtained by dividing the respective values in table 5 by the highest value. similarly, for non-beneficial attributes, the lowest value is assigned ‘1’ and then the rest of the normalized values are obtained by dividing the least value by respective values in table 5. hence, a normalized table is obtained as given in table 6. ijahp article: dave, desai, raval/a decision support system for tool electrode selection international journal of the analytic hierarchy process 100 vol. 4 issue 2 2012 issn 1936-6744 table 6 normalized attribute data c mrr tw er tc mt cu 0.7143 1.0 0.2 1 1 0.2967 cuw 0.5555 0.3333 1 0.2742 0.3896 0.9343 w 0.5555 0.3333 1 0.2982 0.4234 0.9233 b 0.7143 0.7777 0.1429 0.3148 0.3532 0.2647 ss 1.0 0.5555 0.3333 0.0236 0.0416 0.3836 al 1.0 0.3333 0.1429 0.2881 0.5455 0.1808 g 1.0 0.7777 0.2 0.0003 0.0623 1 3.3 calculation of electrode selection index (esi) the overall performance scores, termed electrode selection index (esi) in the current case, for all the alternatives have to be calculated. this score is obtained by multiplying the relative normalized weight (wj) of each attribute (obtained in section 3.1) with its corresponding normalized weight value for each alternative (as obtained in table 6) and summing over the attributes for each alternative. the alternative with the highest value of esi is considered as the best alternative. ( ) 1 m j ij normal j esi w m =  = × ∑ for the alternative 1 i.e. cu, esi is calculated as follows: [(0.421 0.7143) (0.1465 1) (0.2113 0.2)cuesi = × + × + × (0.0831 1) (0.0549 1) (0.0831 0.2967)]+ × + × + × 0.6521= similarly, esi for all alternatives are calculated as shown in table 7. they are ranked in descending order. ijahp article: dave, desai, raval/a decision support system for tool electrode selection international journal of the analytic hierarchy process 101 vol. 4 issue 2 2012 issn 1936-6744 table 7 esi and rank of all alternatives alternative esi rank cu 0.6521 ii cuw 0.6155 iv w 0.6187 iii b 0.5124 vii ss 0.6089 v al 0.5689 vi g 0.6637 i the highest esi is noted for graphite followed by copper, tungsten, copper tungsten, stainless steel, aluminum and brass. hence, it is concluded that graphite is the best material for fabricating tool electrode followed by copper, tungsten, copper tungsten, stainless steel, aluminum and brass. 4. conclusion a logical procedure based on the analytic hierarchy process (ahp) has been suggested that helps in the selection of a suitable material from amongst a large number of available materials for fabricating tool electrodes for electro discharge machining process. the methodology is capable of taking into account various desirable characteristics of the material and it strengthens the existing procedure by proposing a logical and rational method of material evaluation and selection for tool electrode fabrication. for all the alternatives, an electrode selection index (esi) is evaluated and ranked in descending order. this resulted in the selection of graphite as the best material for tool electrode fabrication. ijahp article: dave, desai, raval/a decision support system for tool electrode selection international journal of the analytic hierarchy process 102 vol. 4 issue 2 2012 issn 1936-6744 references pandey and shan, 2003. modern machining processes. new delhi, india: tata mcgraw hill publishing co. ltd. mishra p. k., 2007. nonconventional machining. new delhi, india: narosa publishing house. liao tw, 1996. a fuzzy multicriteria decision-making method for material selection. journal of manufacturing systems, 15:1–12. farag m, 1997. materials selection for engineering design. new york: prentice-hall. giachetti re, 1998. a decision support system for material and manufacturing process selection. journal of intelligent manufacturing, 9:265–276. ashby mf, 2000. multi-objective optimization in material design and selection. materials & design, 48:359–369. ashby mf, brechet yjm, cebon d, salvo l, 2004. selection strategies for materials and processes. materials & design, 25:51–67. raj r, 2000. an interdisciplinary framework for the design and life prediction of engineering systems. trans asme, journal of engineering materials technology, 122:348–354. raj r, enright mp, frangopol dm, 2000. a system level partitioning approach for analyzing the origins of variability in life prediction of tungsten filaments for incandescent lamps. materials & design, 21:9–18. deng ym, edwards kl, 2007. the role of materials identification and selection in engineering design. materials & design, 28:131–139. shanian a, savadogo o, 2006. a material selection model based on the concept of multiple factor decision making. materials & design, 27:329–337. chan jwk, tong tkl, 2006. multi-criteria material selections and end-of-life product strategy: grey relational analysis approach. materials & design, 28 (5): 15391546. rao rv, 2006. a material selection model using graph theory and matrix approach. materials science and engineering: a, 431:248–255. ijahp article: dave, desai, raval/a decision support system for tool electrode selection international journal of the analytic hierarchy process 103 vol. 4 issue 2 2012 issn 1936-6744 kumar s, singh r, 2006. a short note on an intelligent system for selection of materials for progressive die components. journal of materials processing technology, 182 (13):456-461. cheng ac, chen cj, chen cy, 2008. a fuzzy multiple criteria comparison of technology forecasting methods for predicting the new materials development. technological forecasting and social change, 75:131-141. rao rv, davim jp, 2007. a decision making framework model for material selection using a combined multiple attribute decision making method. international journal of advanced manufacturing technology, 35(7): 751-760. rathod m k, kanzaria h v, 2011. a methodological concept for phase change material selection based on multiple criteria decision analysis with and without fuzzy environment. materials and design, 32(6): 3578-3585. saaty t.l., 1980. the analytic hierarchy process. new york: mcgraw hill. saaty, t.l., 2000. fundamentals of decision making with the analytic hierarchy process. pittsburgh, pa: rws publications. triantaphyllou e, 2000. multi-criteria decision making methods: a comparative study. dordrecht: kluwer academic publishers. rao r. v., 2007. decision making in the manufacturing environment: using graph theory and fuzzy multiple attribute decision making methods. london: springer-verlag. 1. introduction 2. analytic hierarchy process 3. tool electrode selection using ahp 4. conclusion ijahp article: repetski, sarkani, mazzuchi / applying the analytic hierarchy process (ahp) to expert documents international journal of the analytic hierarchy process 1 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.919 applying the analytic hierarchy process (ahp) to expert documents edward repetski george washington university edward_repetski@gwu.edu shahram sarkani george washington university sarkani@gwu.edu thomas mazzuchi george washington university mazzu@gwu.edu abstract this paper explores an innovative technique to elicit prioritization from expert documents using the analytic hierarchy process (ahp). practitioners within domains that have a large quantity of expert literature can utilize these prior efforts to answer new questions. this method can solve some of the challenges of the ahp which include dealing with inconsistencies amplified by large numbers of options, locating experts with the availability and commitment to undergo an iterative opinion refinement process, and the speed of obtaining priorities. by extracting comparisons through rules applied to expert literature, this paper shows how prioritization for a number of options larger than 15 can be achieved using the ahp. using the literature creation process as a means to gain consistency from a panel of experts, the extracted priorities are absolutely consistent. further, the process can be accomplished without iterative engagement of experts which helps reduce time and cost. overall, this unique application of the ahp provides those with access to a set of expert literature a method to quantify expert opinions in a consistent and cost-effective manner. keywords: expert opinion; opinion extraction 1. introduction this paper details a technique to elicit expert opinions from documents. while this is a defense-related example, this technique could be used with any well-documented domain. various domains require prioritization of lists of requirements, constraints, conditions, etc. while this technique was used to prioritize eighteen linkages for operational integration optimization, the method could easily prioritize a large number of requirements for any project. the authors wished to find a way to weight the eighteen linkages as priorities for integration efforts in a separate model. the analytic hierarchy process (ahp) has been mailto:edward_repetski@gwu.edu mailto:sarkani@gwu.edu mailto:mazzu@gwu.edu ijahp article: repetski, sarkani, mazzuchi / applying the analytic hierarchy process (ahp) to expert documents international journal of the analytic hierarchy process 2 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.919 used successfully to weight priorities of sets; however, using the tool, in this case, presented a few challenges including the number of elements to prioritize, access to a large field of experts and the time needed to solicit opinions in the traditional manner from those experts. commercially available ahp software typically addresses cases with fifteen or fewer options, so the authors built an ahp tool that could accommodate a larger number of options instead of adding more layers to the hierarchy. further, access to experts for the extended amount of time required to extract these many paired comparisons and to iteratively engage until the opinions about priorities were complete and adequately consistent was not available. the authors decided to address this challenge by using the doctrinal literature as experts. fortunately, the area of this research was well covered by detailed documents. the authors treated the doctrinal manuals as experts and used metrics to elicit the opinions from those manuals. this technique proved useful in three ways; it was effectively broader and faster than the typical ahp elicitation process, and provided absolutely consistent responses. this method is broader as the study used five doctrinal manuals, each created, edited and approved by an extended set of experts. the manuals could be evaluated in a few days once a set of metrics was determined and thus this is faster than any survey tool. lastly, the use of metrics provided absolutely consistent results. systems thinking, through model-based systems engineering (mbse) is expanding into broader areas of application. one such area of potential application is military readiness. the case study involved the operational integration and testing of unstabilized gunnery crews composed of us army soldiers. this type of integration has long been the domain of expert opinion. the authors were attempting to model the various integration tools and the efficacy of their application to the integration of humans and equipment sets into effective teams. such a model would advance the application of mbse into the operational life of systems and incorporate humans into those models. the potential impact is significant given the sheer quantity of teams that fall into this category. a part of that model, shown in figure 1, illustrates a combined integration matrix (cim). this matrix shows all the potential linkages between crew members and with each crew member and parts of the kit of a mounted crew. the integration matrix is derived from a multi-domain architecture design structure matrix (mda-dsm) of a three-soldier crew. the mechanics of that model aside, this matrix needed to be weighted for that model by prioritizing the 18 linkages. the left side of figure 1 shows those 18 cells as they are abstracted from a larger dsm with the linkages of interest labeled a-r. on the right side of figure 1 is the model with the weighted values (in percentages) that were determined by this proposed technique applied to the ahp. figure 1 left: combined integration matrix (cim); letters identify the 18 linkages to be prioritized. right: weighting determined by this method ijahp article: repetski, sarkani, mazzuchi / applying the analytic hierarchy process (ahp) to expert documents international journal of the analytic hierarchy process 3 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.919 to validate this method, a group of qualified us army master gunners were engaged for their expert opinions to determine if these were consistent with the results of the doctrinal survey process. four master gunners were engaged and provided their input to the model. this paper illustrates the extraction of priorities from a doctrinal literature variant of the ahp and validates that method using the ahp by traditional means. 2. literature review in this increasingly complex world, governments and industry have pursued ways to combine large multitudes of expert opinions to find optimal recommendations for complex situations. the ahp, created by saaty, is a recognized tool for decision making in complex situations (saaty, 1988). his essential text on this subject even mentions “resource allocation” in its title. the goal of this work is indeed seeking to best allocate resources. the ahp has been described and examined in the literature and a brief synopsis is provided below. the major components of the ahp involve determining a hierarchy and then prioritizing the elements at any given level using a method to translate the opinions of experts into numeric values. early work in ahp involved the use of additive and multiplicative formats which were logarithmically related (lootsma, 1999). multiplicative ahp achieves the same results mathematically and more intuitively as fewer practitioners work directly with logarithms with the growth of available computing power. therefore, this work uses multiplicative ahp. in either processing method, the practitioner needs to extract opinions from experts, iteratively refine those opinions to deal with inconsistencies, and combine those opinions to reach a recommended decision or prioritization. we commend lootsma’s paper on saaty for a strong synopsis of the mathematics of this process (lootsma, 1980). eliciting opinions in multiplicative ahp involves using adjectives or a small range of numbers to quantify opinions by doing pairwise comparisons (saaty, 2013). for example, if a is more important than b, then a is 2b; if much more important than b, then a is 4b. by focusing on each pair, the expert isolates themselves from the multitude of judgements and makes a determination. this can be a daunting process. in the given example with 18 options, a staggering 306 opinions are required to complete a table. this assumes the diagonal of ones where each element is as important as itself. this is a monotonous task that often discourages experts from participating. the many comparisons can be daunting, but also unnecessary. if a is 2b, then b is ½ a. two cells can be completed with one judgement. further, if a is 2b and b is 2c, then a is 4c in a perfectly consistent situation. these assumptions of consistency reduce the number of comparisons that need to be elicited from users. reciprocity alone reduces the 306 opinions needed in the example to 153. another third or more opinions can be eliminated with assumptions of consistency. this balancing act is complicated. the fewer pairwise comparisons elicited, the more pronounced the impact of inconsistencies will be. also, the selection of the pairs could vary between experts. the practitioner doing the iterative work can complete the entire matrix and determine the inconsistencies during further engagements; however, garnering data iteratively is challenging. lootsma provides an interesting example as he works to extract data from practitioners in one of his papers ijahp article: repetski, sarkani, mazzuchi / applying the analytic hierarchy process (ahp) to expert documents international journal of the analytic hierarchy process 4 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.919 (lootsma, 1999a). in this case, he found it challenging to obtain voluntary responses to his questions from experts even in the very field of applying the ahp. multiplicative ahp is based on absolute ratio scales; therefore, zeroes are not necessary (garuti, 2018). the opinions gathered here are comparisons and a ratio scale is essential to this process. experts may be constrained in their judgements depending on how the opinion-gathering instrument is constructed (for example, only able to use adjectives, numbers or even intermediate values). even so, challenges will be faced with consistency. the two areas of consistency are in terms of intransivity and cyclic judgements (lootsma, 1999b). intransivity is the idea that aij is the inverse of aji or if a is 2b and b is 2c then a should be 4c. if this is violated, consistency suffers. cyclic judgements where a>b and b>c, but c>a severely compromises consistency. relative comparisons require a ratio scale. with a ratio scale, geometric means rather than arithmetic means are preferred for averaging to obtain results that have mean the same thing for each opinion. this is shown in equation 1. the n th root of the n multiples provides a means of combining the n opinions while maintaining the relationship of aij always equals 1/aji (lootsma, 1999b). critical relationships between cells are lost if this is not observed. 𝐺𝑒𝑜𝑚𝑒𝑡𝑟𝑖𝑐 𝑀𝑒𝑎𝑛 = √(𝐴1𝐴2 … 𝐴𝑛) 1/𝑛 (1) even after a cyclic process of opinion solicitation is complete, the opinions must be evaluated for consistency. the intransitives and logical breaks inside an expert’s opinion or across the set of experts’ opinions must be sufficiently small to allow the consolidated opinion to be seen as acceptable. here, the standard is the consistency ratio (saaty, 1988). the consistency ratio (cr) is the consistent index (ci) divided by the random consistency index (ri) as shown in equation 2. 𝞴max is the maximum eigenvalue, n the number of options in the set, all divided by (n-1). ri is obtained from saaty’s text and shown in figure 2. note that ri was created by comparing consistency against randomly created matrices. the scale ranges from 3 to 15 for values of n. a cr of less than 0.1 is acceptably consistent with 0 representing perfectly consistent and higher values being less and less consistent. ri is not defined for situations like the one at hand when n = 18. 𝐶𝑅 = 𝐶𝐼 𝑅𝐼 = 𝞴𝒎𝒂𝒙−𝒏 𝒏−𝟏 𝑹𝑰 (2) figure 2 random consistency index (ri) (saaty, 1988) to find, or at least approximate, the maximum eigenvalue, 𝞴max the practitioner must define the consolidated opinion matrix and use the right eigenvector. the right eigenvector can be computed in a number of ways. all the methods agree in the case where the matrix is absolutely consistent; therefore, the simplest method works as well as n 3 4 5 6 7 8 9 10 11 12 13 14 15 … ri 0.58 0.9 1.12 1.24 1.32 1.41 1.45 1.49 1.51 1.53 1.55 1.57 1.59 … ijahp article: repetski, sarkani, mazzuchi / applying the analytic hierarchy process (ahp) to expert documents international journal of the analytic hierarchy process 5 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.919 any other. the combined opinion matrix is summed by rows, then the values are normalized to one. that is, the sum of each row is divided by the total of row sums. these represent the priority ranking of the options in each row. to find the eigenvalue, the opinion matrix is multiplied by the eigenvector. this produces a matrix of the same form as the eigenvector. this new matrix is now divided by n times the eigenvector to achieve a new matrix that will be close to a column of ones (a column of ones in the perfectly consistent case) and their sum will be the maximum eigenvalue. we recommend lootsma’s text to understand the calculation in detail (lootsma, 1999b). another benefit of the eigenvalue calculations is that the practitioner now has a ready means to check their work. the maximum eigenvalue cannot be less than n. as this process allows for a spread of resources over a large range of options, the matrices involved grow very difficult to manually check for errors. therefore, the assumption of consistency must be checked both to confirm consistency and to check for errors in logic. a large amount of literature has focused on the level of inconsistency that can be tolerated and how to extract sufficiently consistent data from available experts (russo & camanho, 2015). there are challenges involved with identifying the experts, soliciting their opinions, and ensuring that those opinions are consistent. first, the decision maker or their agents must identify the experts. this is not a trivial decision. the practitioner must define the requirements for experts and define what mix of experts is needed from differing domains (garuti & sandoval, 2006). then, the experts need to be engaged. large organizations with internal experts can task those experts to participate in the gathering of opinions, but this is rarely the case. most often, the practitioner must solicit voluntary participation from the experts. several mechanisms exist to address inconsistencies. as expected, iteratively engaging experts to conform their opinions into consistent matrices is time and cost prohibitive (basak, 2020). inconsistency grows as the number of options increase (saaty, 1988). as the decision set increases, inconsistencies grow faster. a solution to this problem is to add a level to the hierarchy to create more pools with fewer decisions (lootsma, 1999b). saaty and sagir (2009) make a case for this layered approach. if such a bifurcation of the problem is not practical, the reality of the slow increase of the ri has greater impact. as shown in figure 2, the value of ri increases very slowly after n = 8. excessive inconsistency invalidates the use of precious data; therefore, various techniques have been proposed to deal with this challenge. some have recommended using bootstrapping to improve inconsistent data (basak, 2020). others have applied fuzzy comparisons to refine the data (boender, de graan, & lootsma, 1989; liu, xu, & liao, 2016). lastly, at least one practitioner has used rules to apply the ahp in cloud computing (ergu, kou, peng, shi, & shi, 2013). it is valuable to discuss the scale that is used. saaty firmly supported the idea that a scale of 1 to 9 was ideal (saaty, 1988). others have postulated different scales (lootsma, 1999b). saaty was looking for numbers that could be easily interpreted by the experts providing the opinions and he found, through trial and error, that using 1-9 was most effective. lootsma shows both logarithmic and multiplicative scales as options. he further posits that any ratio scale works mathematically (lootsma, 1999b). the fundamental issue with a relatively short scale is that intransitivity can quickly appear ijahp article: repetski, sarkani, mazzuchi / applying the analytic hierarchy process (ahp) to expert documents international journal of the analytic hierarchy process 6 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.919 when a is four times as important as b and b is three times as important as c and c is more important than d. while not a cyclic intransitive case, the math quickly explodes the range of 1 to 9. saaty shows how these issues can be returned to experts for refined opinions. in this example, because the numbers are not linked to adjectival ratings by experts, the option was used to apply a larger scale than normally used for the ahp. the validation tool, however, does use a 1 to 9 scale. in summary, the ahp is a recognized tool that uses experts to weight priorities and combine their opinions. the shortcomings of this method, especially when accounting for a large value of n, the undefined value of ri for n greater than 15, include the difficulty of gathering expert opinions, the risk of losing those opinions through iteration and the increasing inconsistencies that must be addressed. the methodology applied addresses those issues successfully. 3. methodology the proposed technique addresses all of the challenges of the ahp including large n values and limited access to iteratively work with experts. successful application of this technique requires that the subject be addressed in a significant body of technical literature. our case study meets this requirement with multiple us army doctrinal publications that directly deal with the topic of integrating and testing three-person crews of vehicle-mounted, unstabilized weapons. the technique treats each of the documents as an expert or a panel of experts. this approach addresses the challenges of the ahp which include a dearth of experts, the time-consuming impacts of iteration, the potential for increased inconsistencies, and problems resulting from a large value of n. it shifts the benefit of opinion refinement from the ahp interaction with experts to those opinions being refined in the writing of doctrine. each army doctrinal publication is a summary of the work of multiple experts, often the most qualified, across a very large organization. each document represents not a single expert, but a panel of selected experts. further, since the manuals are written at different times, these writing teams effectively represent a different pool of experts each time. the metrics that are used must clearly relate to the topic of the document. in this example the manuals deal with the process of crew integration and the chosen metrics are focused on that area. once the metrics are chosen, they are drawn from the documents by a procedure. perhaps most importantly, once the metrics are selected, because they are drawn by a set of rules, they provide absolutely consistent results. these panels of experts, in the writing and editing of these publications, act as the iterative event which drives the panel of experts to be consistent. thus, the prior event, the publication writing process, acts as the iterative, consistency-improving, time-consuming process from which the practitioner can profit. in this case, the army’s doctrinal library was consulted for related technical publications. five manuals were found that addressed the case study. then, a process was followed whereby the relevant section or sections of the documents were examined and a count was taken of each reference to each linkage as shown in figure 3. each sentence that addressed the interaction between two elements and thus representing a linkage to be integrated was counted against that linkage. for example, the sentence “the gunner ijahp article: repetski, sarkani, mazzuchi / applying the analytic hierarchy process (ahp) to expert documents international journal of the analytic hierarchy process 7 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.919 orients the weapon…” would be counted against letter i. similarly, in diagrams and illustrations, bullet comments that addressed a linkage were counted. note, in figure 3 the variance between sources both in the volume of comments and in their dispersion across topics. figure 3 raw data drawn from five doctrinal publications for each expert document, a matrix was created with the 18 elements (a-r) along the left and top margins. before simply dividing the row by the column value, some preprocessing was necessary to ensure that the matrices were compatible, consistent and easily usable. first, the decision was made to set all five to a common scale to ensure that the weights of each opinion were the same. because the data was integer in nature it was easier to convert all ratios to a common base. each value was multiplied by 256 and divided by the strongest opinion in that source. therefore, the strongest opinion always had a value of one. this was not an opinion scale such as 1 to 9, but a way to balance the opinions between each other. we could have converted to any base number, but settled on 256. adding one to each value removed zeroes which simplified the use of ratios while keeping the distance between the counts consistent. some variance in the opinions was seen when the common factor was small. this rapidly smoothed out at a larger scale and a very stable set of opinions was achieved with a common factor of 256. ijahp article: repetski, sarkani, mazzuchi / applying the analytic hierarchy process (ahp) to expert documents international journal of the analytic hierarchy process 8 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.919 figure 4 shows the details of one of the opinion matrices; specifically, the top left corner. imagine the columns running from a to r as well as the rows. focusing on this corner of one matrix nicely illustrates the process. the raw data is shown in the column and row labeled cnt or count. the row and column labeled 1 shows the values as one is added to each count to convert to a ratio scale. the column labeled norm has the largest value converted to 256, that is multiplied by 256 and divided by the largest value. the largest data point is 256 while all others are scaled similarly. next, the opinions are created by dividing row by column. note that the appearance of the expected diagonal of ones as items are compared to themselves. also, the relationships are as expected as aij always equal 1/aji; for example, the two circled cells are the inverse of each other. of course, these numbers are truncated to two decimal places for legibility. figure 4 matrix of opinion comparisons for one of the expert documents more verbose manuals had more points to distribute to more linkages and addressed more detail while less wordy texts often omitted some linkages of lesser importance. the normalization to a common number allowed the five expert documents to be weighted the same in the ensuing combination of tables and allowed for easier comparison between opinions. further, adding unity before normalizing allowed for less impact of the omitted factors in the less wordy texts, where omission had more impact on ranking in longer writings. because they were built by a rule, aij always equals 1/aji and a > 2b and b > 2c always means a > 4c. this is, absolutely, the definition of consistency. once the five opinions were moved into these standard matrices, the five matrices were combined into a unified opinion matrix (see figure 5). this is the matching highlight of the combined opinions to the single opinion in figure 4. this was combined using the geometric mean, the 5 th root of the product of the five ratios. note again the inverse relationships across the main diagonal and highlighted in the circled squares with 2.11 equal to 1/0.47. this is the main reason to use the geometric mean as it retains reciprocity. from here, two essential functions are possible. first, the eigenvector can be extracted to see the consolidated opinion of priorities for the 18 linkages. as discussed, this is most easily obtained from a consistent matrix by summing the rows into a column vector on the right of the matrix and then normalizing the values by dividing each by the ijahp article: repetski, sarkani, mazzuchi / applying the analytic hierarchy process (ahp) to expert documents international journal of the analytic hierarchy process 9 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.919 sum of the column matrix providing a new matrix that sums to unity. this priority matrix was the answer to the question that had prompted the effort. figure 5 matrix of combined opinions the second function is to use this eigenvector to find the maximum eigenvalue and test for consistency. 𝞴max was extracted as described in the literature review and found to be equal to 18 which is the definition of a consistent 18 x 18 matrix. this reduces the consistency index and thus the consistency ratio to zero meaning absolutely consistent data. 4. results figure 6 shows the right eigenvector (transposed to horizontal) which is the prioritization of the 18 linkages. further, note how they loop back to figure 1 where they are shown as percentages. this was a way to extract the initial weighting of the larger model by using readily available data in the form of a large body of expert documentation. from here, the larger model was successfully tested. this is as expected with a perfectly consistent matrix. figure 6 right eigenvector (transposed) or the list of the priority for each linkage 5. validation this method applied to the work at hand demands to be validated. a sample of experts was surveyed using traditional ahp processes to confirm the variation as described. an ahp survey of four us army master gunners was conducted using a pair-wise comparison tool. the tool asked them to respond to 153 pair-wise comparisons allowing responses from 1 to 9. these pairs were randomized in order to prevent long sections focusing on only one linkage. the total matrix has 324 cells. the diagonal of all ones representing items compared to themselves were automatically valued at one. because aij = 1/aji, the remaining 306 cells were divided in half. the 153 pairs were, perhaps ranking i a k b h c e o g f p d j m r n l q priority 0.215 0.197 0.097 0.093 0.048 0.043 0.040 0.040 0.032 0.027 0.024 0.024 0.022 0.020 0.020 0.019 0.018 0.018 ijahp article: repetski, sarkani, mazzuchi / applying the analytic hierarchy process (ahp) to expert documents international journal of the analytic hierarchy process 10 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.919 obviously, the most daunting number of comparisons required. this created a longer tool that attempted to reduce iteration by being more intense at the beginning. these pairwise matrices were entered into the same tool as the literature described earlier with their comparisons being entered into the tables directly as ratios. each was independently exposed to a consistency test and all four achieved cr values below 0.1. further, these used a ri value of 1.59 which is the value of ri for n = 15. given the slowly increasing value of ri, this was reasonably close but still a conservative approach. all four expert opinions proved sufficiently consistent on the first iteration to be usable. the value of the cr for the four sources and the combined opinion matrix is shown in figure 7. figure 7 cr values in validation the combined results are shown in figure 8. the percentage values and ordinals show variance between the two surveys, but the graph in figure 9 tells the complete story. the most important connection, linkage i, is the same in both surveys confirming that the model for the larger study remained unchanged. however, some values make large ordinal changes; b drops from 4 to 11, h drops from 5 to 14, o climbs from 8 to 2, while q climbs from 18 to 12. these appear to be wild swings; however, the graph helps to clarify the data. figure 9 graphs the two sets of results; the dashed line represents the doctrinal ahp results and the solid line represents the validation survey results. notice how flat the ahp survey results are compared to the doctrinal extract ahp. the range is more than doubled. the graphs suggest an amplified signal. the similar shapes of the two validate the doctrinal method, even endorse it. the issue with the survey results is that they are much flatter, a small fluctuation in a value moves it wildly up or down the rank ordering. this may be due to the difference between using a 1 to 9 ratio scale for the survey instead of the 1 to 256 scale for the doctrinal extraction method. this appears to have amplified the results producing greater clarity. the smaller scale allowed for many elements to be in a very narrow band in the middle of the rankings. the larger scale spread those elements out while tightening the values toward the bottom. the consistent shape of the two curves allows the doctrinal extraction method to be used for this model. ijahp article: repetski, sarkani, mazzuchi / applying the analytic hierarchy process (ahp) to expert documents international journal of the analytic hierarchy process 11 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.919 figure 8 table of results for the doctrinal tool and the validation surveys ijahp article: repetski, sarkani, mazzuchi / applying the analytic hierarchy process (ahp) to expert documents international journal of the analytic hierarchy process 12 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.919 figure 9 graph of ahp values for doctrinal literature and validation surveys 6. conclusions as systems thinking moves into domains where a large body of expert documentation exists, the ahp can be usefully applied to literature to extract embedded expert opinions. while the practitioner must be careful to select appropriate methods to measure opinions, well-selected metrics can readily convert those opinions into priorities or quantitative recommendations. a formal proof of this process should be pursued. applications in the area of this study, operational integration and operational integration testing of human and equipment organizations, are legion. these are domains where expert opinion has long held sway and those opinions have been memorialized in texts, reviews, studies, etc. the ability to quantify the opinions on the most important issues in after-action reports or prioritizing steps in a process provides the ability to better focus time and other resources to more effectively, even optimally, address issues. the proposed process enables studies to exploit the corpus of existing documentation to inform more reasoned applications to solving problems with a labor and time saving tool. further, the validation process suggests that the literature extraction method could be used in other domains where large bodies of documents written by experts can be put to use. ijahp article: repetski, sarkani, mazzuchi / applying the analytic hierarchy process (ahp) to expert documents international journal of the analytic hierarchy process 13 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.919 references basak, i. (2020). estimation of priority weights based on a resampling technique and a ranking method in analytic hierarchy process. journal of multi-criteria decision analysis, 27(1–2), 61–64. doi: https://doi.org/10.1002/mcda.1664 boender, c. g. e., de graan, j. g., & lootsma, f. a. (1989). multi-criteria decision analysis with fuzzy pairwise comparisons. fuzzy sets and systems, 29(2), 133–143. doi: https://doi.org/10.1016/0165-0114(89)90187-5 ergu, d., kou, g., peng, y., shi, y., & shi, y. (2013). the analytic hierarchy process: task scheduling and resource allocation in cloud computing environment. journal of supercomputing, 64(3), 835–848. doi: https://doi.org/10.1007/s11227-011-0625-1 garuti, c. (2018). reflections on common misunderstanding when using ahp and a response to criticism of saaty’s consistency index. international journal of the analytic hierarchy process, 10(3), 488–501. doi: https://doi.org/10.13033/ijahp.v10i3.573 garuti, c., & sandoval, m. (2006). the ahp: a multicriteria decision making methodology for shiftwork prioritizing. journal of systems science and systems engineering, 15(2), 189–200. doi: https://doi.org/10.1007/s11518-006-5007-5 liu, h., xu, z., & liao, h. (2016). the multiplicative consistency index of hesitant fuzzy preference relation. ieee transactions on fuzzy systems, 24(1), 82–93. doi: https://doi.org/10.1109/tfuzz.2015.2426315 lootsma, f. a. (1980). saaty’s priority theory and the nomination of a senior professor in operations research. european journal of operational research, 4(6), 380–388. doi: https://doi.org/10.1016/0377-2217(80)90189-7 lootsma, f. a. (1999a). letter to the editor: the expected future of mcda. journal of multi-criteria decision analysis, (8), 59–60. retrieved from https://search.proquest.com/docview/220297257?accountid=12834 lootsma, f. a. (1999b). multi-criteria decision analysis via ratio and difference judgement. springer. russo, r. d. f. s. m., & camanho, r. (2015). criteria in ahp: a systematic review of literature. procedia computer science, 55(itqm), 1123–1132. doi: https://doi.org/10.1016/j.procs.2015.07.081 saaty, t. l. (1988). the analytic hierarchy process: planning, priority setting and resource allocation (2nd ed.). doi: https://doi.org/0-07-054371-2 saaty, t. l. (2013). fundamentals of decision making and priority theory with the analytic hierarchy process (ebook seco). pittsburgh, pa, usa: rws pubications. saaty, t. l., & sagir, m. (2009). extending the measurement of tangibles to intangibles. ijahp article: repetski, sarkani, mazzuchi / applying the analytic hierarchy process (ahp) to expert documents international journal of the analytic hierarchy process 14 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.919 international journal of information technology and decision making, 8(1), 7–27. doi: https://doi.org/10.1142/s0219622009003247 ijahp article: gawlik/enhancing the work-life balance through ahp modeling of early career decision making international journal of the analytic hierarchy process 181 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.626 enhancing the work-life balance through ahp modeling of early career decision making 1 remigiusz gawlik cracow university of economics poland remigiusz.gawlik@uek.krakow.pl abstract the aim of this paper is to present the ranking of quality of life determinants in the evaluation of university students that are starting their professional activities. the research methodology was composed of five levels. first, a review of the scientific literature on the quality of life and work-life balance, as well as on multi-criteria decision-making (mcdm) was performed. then, an ahp decision-making model for early career decision-making was elaborated. it was followed by a two-stage expert selection process, from which significance rankings for all four parent criteria, 16 subcriteria and four decision alternatives were obtained. finally, a graphical and descriptive presentation of the obtained results was presented. the research sample was composed of 14 experts extracted from an initial sample of almost 200 university students. the research findings show that university students who are on the verge of entering so-called “adult life” most likely choose a career-oriented approach. at the same time, they seem to most value the “safety, stability and certainty” parent criterion. the implications of the presented research could help students better adapt to the labor market trends and lead to a higher life satisfaction of future employees. the main research limitation comes from narrowing the group of experts to only university students. the value added by this research is derived from the fact that employees with a satisfactory level of work-life balance will contribute to a general increase in the overall satisfaction level in society. keywords: analytic hierarchy process; determinants of quality of life; work-life balance; human resources; decision-making 1 this paper was presented at the international symposium on the analytic hierarchy process in hong kong, hk on july 12-15, 2018. the author’s most sincere gratitude goes to ms. rozann w. saaty from the creative decisions foundation for her guidance and support when preparing and submitting this manuscript. this research was supported by the national science centre of poland (decision no.: dec 2013/11/d/hs4/04070) within a research project entitled “the application of analytic hierarchy process for analyzing material and non-material determinants of life quality of young europeans” led by the author between 2014 and 2017. ijahp article: gawlik/enhancing the work-life balance through ahp modeling of early career decision making international journal of the analytic hierarchy process 182 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.626 1. introduction this paper focuses on the ranking of significance of quality of life determinants obtained in a research task targeted at university students that are on the verge of starting their professional activities. the aim of the presented research was to identify which determinants of quality of life play the most important role for university students when choosing their future lifestyle. lifestyle was understood as the composition of professional and private activities, which constitute one’s relation of work-to-leisure ratio, referred to as the work-life balance. the objective was to incorporate both material and non-material determinants of quality of life into the scope of the research. the individual preferences of the respondents were crucial for understanding the rationale behind young peoples’ decision-making. their inclination towards a certain lifestyle was expressed by choosing one of the decision alternatives, i.e. career-, incomeor family-oriented, or a complete opt-out from the socio-economic system (outsider’s attitude). the motivation for the research came from the observation that early career decisionmaking of young people is often artificial and unstructured. meanwhile, the inclusion of quality of life determinants into this process could enhance their choice of the most appropriate professional development strategy, taking into account their work-life balance preferences. the research methodology was composed of a literature review, conceptual, methodological, exploratory and explanatory research. the main research method is the analytic hierarchy process (ahp). the sections of the article will contain a brief review of recent scientific literature on the matter (section 2), an introduction to research methodology and sample (section 3), a presentation and discussion of obtained results (section 4) and a conclusion (section 5). 2. literature review the literature review will be divided into two sections. the first will deal with quality of life studies and work-life balance from the perspective of economics, and the second will briefly discuss the utility of mcdm methods in economic and managerial applications. 2.1 quality of life studies and work-life balance the literature review of this section was carried out on october 18, 2018 in the web of science online database. the search on the “topic=quality of life” inquiry, after limitation to four categories (business, economics, management and operations research management science) returned 11272 results, out of which 4930 have been published in past five years. the selection of quoted literature was performed on a twoway basis: (i) usefulness of the explanation of the quality of life and work-life balance concepts; (ii) finding examples of qol studies relevant for research in economics and management. ijahp article: gawlik/enhancing the work-life balance through ahp modeling of early career decision making international journal of the analytic hierarchy process 183 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.626 first, the appearance of quality of life (qol) and well-being studies in economics can be traced back to smith (1759). this author discussed qol determinants such as health, wealth and conscience. learmonth et al. (2015) describes qol as a global psychological construct that takes into account the weighting or importance individuals place on particular areas of life. lau et al. (2015) stated that qol is how well people are able to perform daily activities, and how they feel about the physical, social, and psychological functioning of their lives. work-life balance is a part of qol studies that refers to the work-to-leisure time ratio. balance is understood as a configuration of time use that maximizes positive emotional and developmental outcomes. it depends on an array of normative, situational, demographic, and psychological factors which defy ‘linear’ interpretation and complicate traditional statistical analyses (zuzanek, 2009). nevertheless, this ratio is crucial for qol perception by individuals (hansen, 2015). qol research in economics has gained momentum in the past decades with the works of researchers such as maslow (1954, theory of needs), abel-smith & townsend (1965, distribution of welfare), atkinson (1983, social inequality), sen (1985, wellbeing and capabilities), schuessler and fisher (1985, qol theory), layard (2005, happiness), şerban-oprescu (2012, qol sustainability), graafland & compen (2015, life satisfaction), ulman & šoltés (2015, poverty measurement), somarriba arechavala, zarzosa espina & pena trapero (2015, qol measurement), adame, caplliure & miquel (2016, gender studies), russo, shteigman & carmeli (2016, work efficiency), gawlik & jacobsen (2016, work-life balance) and others. 2.2 multicriteria decision-making the aim of this part of the literature review was to provide an introduction on the utility of mcdm methods in the field of management and economics. the web of science online database (accessed on october 19, 2018) returned 1513 answers to the “topic=multicriteria decision making” enquiry, limited to the same four categories (business, economics, management and operations research management science). 541 of these publications have been published in past five years. the selection of quoted literature was aim-oriented and sought to present the sense and utility of mcdm in a compact way. mcdm is one field of decision-making theory. the main purpose of mcdm is to support decision-makers when facing multi-criteria problems (sałabun, 2014). the theoretical framework on aiding mcdm processes has been presented in zopounidis & doumpos (2013). rezaei (2015) states that mcdm problems are generally divided into two classes with respect to the solution space of the problem: continuous and discrete. to handle continuous problems, multi-objective decision-making (modm) methods are used. discrete problems are solved using multi-attribute decision-making (madm) methods; although, in the scientific literature they are commonly referred to as mcdm. ivlev, vacek & kneppo (2015) point out the complexity of decision-making criteria, he high degree of the decision maker’s responsibility and the uncertainty at every stage of ijahp article: gawlik/enhancing the work-life balance through ahp modeling of early career decision making international journal of the analytic hierarchy process 184 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.626 the decision-making process as specific features of mcdm. the last feature, uncertainty, is often due to interfering aims of involved or affected parties, their various policies, different economic, social, technical and organizational environment and consequences of decisions. this internal and external uncertainty becomes an important determinant of mcdm and results in low predictability of the final effects of the decision-making (durbach & stewart, 2012). teixeira de almeida et al. (2016) observed that the crucial issue in using mcdm models is the evaluation of weights of criteria (or attributes) in the aggregation procedure. in ahp, the method applied in the presented research, the problem persists. ben amor, jabeur & martel (2007) stated that conciliating the results of the pair comparisons according to the criteria could be difficult due to the heterogeneity of the measurement scales and the nature of the evaluations. another problem arises when the differences between the alternatives are inherently close together or when the number of alternatives increases (pomerol & barba-romero, 2000). cabello et al. (2014) observed that from a strictly mathematical point of view, all efficient solutions of a mcdm problem are equally optimal. therefore, the preferences of the decision maker are crucial in determining which decision alternative is the most preferred solution. this feature gains more importance in multi-objective optimization tasks of mcdm problems. therefore, the choice of an appropriate mcdm method is of crucial importance in order to assure a possibly optimal effect of decision-making. varmazyar, dehghanbaghi & afkhami (2016) proposed the application of a combination of various mcdm methods as a way to enhance the precision of the final decision. in such cases, the most common aggregation procedure was a simple averaging function, although pomerol & barbaromero (2000) suggested employing borda and copeland rules. whereas borda selects the highest valued alternatives, copeland ranks them as the result of the number of pairwise victories minus the number of pairwise defeats between the alternatives (varmazyar, dehghanbaghi & afkhami, 2016). various methods of enhancing mcdm have been discussed in sałabun (2014) and gawlik (2016a). the main difference of the presented study from others of this kind is the application of an mcdm method, more specifically the ahp, to research career planning with regard to quality of life. the web of science search on the enquiry “topic=multicriteria decision making and quality of life and career” returned only 1 answer (accessed on march 28, 2019). section 3 focuses on the choice of the applied research methodology and its justification. 3. material and methods the research was designed in five stages: 1) literature review; 2) conceptual research (elaboration of an ahp decision-making model); 3) methodological research (two-stage expert selection); 4) exploratory research (significance rankings by experts); 5) explanatory research (graphical and descriptive presentation of obtained results). although research in economics is mostly based on quantitative data, the description of socio-economic reality should also encompass qualitative factors. quantitative indexes ijahp article: gawlik/enhancing the work-life balance through ahp modeling of early career decision making international journal of the analytic hierarchy process 185 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.626 provide researchers with comparative knowledge on the analyzed occurrence, whereas the qualitative features explain its context and environment. therefore, the use of a methodology that allows incorporating qualitative measures into quantitative research is advised. in fact, ahp allows including both quantitative and qualitative criteria into the decision-making process by assigning the qualitative criteria a number. therefore, a quantification of preference of criterion a over criterion b can be expressed. such mathematical notation allows the decision maker to pick one of the decision alternatives as the possible optimal solution. therefore, ahp was chosen as the research method. developed by saaty (1980), the ahp can be used for complex hierarchical decision problems when the optimal solution has to be chosen from a set of alternatives on a subjective basis (saaty, 1999). the method consists of three steps: (i) formulating the main goal of the decision-making process; (ii) building a hierarchy of decision criteria, sub-criteria and their indicators; (iii) identifying decision alternatives (saaty, 1996). saracoglu et al. (2015) provided a compilation of the advantages of the ahp method from various scientific sources: (i) the ability to model daily real life problems with ease and simplicity; (ii) the ability to reflect the reality of the problems in a true and easy manner; (iii) giving experts the ability to express their thoughts in a free, correct and almost perfect manner due to their experience; (iv) giving people with little or no knowledge about decision-making methods the opportunity to understand the method; (v) having simple and easy mathematical calculations, which helps the experts concentrate on the problems rather than the difficult mathematical calculations; (vi) having the pairwise comparisons, which help the experts and decision makers compare each criteria and alternative one by one with respect to the goal and with respect to the alternatives. the practical ahp application consists of building a hierarchy of independent criteria. then, pairwise comparisons of alternatives, criteria, sub-criteria and their indicators are performed (each-with-each, based on the fundamental comparison scale). this results in the dominant factor from the pair below being linked with the dominant factor from the pair straight above, which gives a ranking of importance of different criteria in the form of a pairwise comparison matrix. finally, a consistency check of the obtained comparisons is performed (saaty, 1996). several works that are critical of the ahp methodology have been published and address such problems as the lack of a theoretical basis for the construction of hierarchies (belton & gear, 1983), subjectivity of the final rankings (dyer, 1990) and a low research repetitiveness (barzilai, 2001). most of these critical remarks have been answered in a satisfying manner in saaty, vargas & whitaker (2009). the above discussion justifies the methodological correctness of the ahp application for the construction of a model that encompasses work-life balance into early career decision-making. the elaboration of the model was carried out in three independent phases. first, exploratory research was performed on a group of 31 young people from european countries. it took the form of a two-week long intensive research project composed of in-depth interviews, group discussions and peer assessment sessions, followed by preparation of individual essays. it resulted in the elaboration of the set of criteria and identification of possible work-life balance strategies relevant for early career ijahp article: gawlik/enhancing the work-life balance through ahp modeling of early career decision making international journal of the analytic hierarchy process 186 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.626 decision-making of young people (gawlik, 2013). second, a self-administered, webbased questionnaire with single-answers and limited choice answers of a qualitative and quantitative nature was introduced to a sample of almost 1000 young respondents in order to eliminate non-relevant criteria and strategies from the formerly identified set (gawlik, titarenko & titov, 2015). third, the model was tested on a group of norwegian students. additionally, explanatory and exploratory research was carried out in order to identify its possible imperfections (gawlik & jacobsen, 2016). the respondents (experts) were university students who were in the process of commencing their professional activities. the specificity of the ahp methodology allows the limitation of direct evaluators to a smaller number, which is possible due to their high level of expertise in the field. following the prescriptions of newman et al. (2015), a two-stage expert selection process consisted of: (i) preliminary selection, based on the assessment of written assignments on the candidate’s understanding of socio-economic occurrences; (ii) final selection through structured direct individual in-depth interviews with candidates. the final set of evaluators was composed of 14 carefully chosen international experts from a sample of almost 200 university students. the judgments of each evaluator have been attributed equal weight. the expert selection process was discussed in gawlik (2016b). figure 1 presents the scheme of the constructed model. ijahp article: gawlik/enhancing the work-life balance through ahp modeling of early career decision making international journal of the analytic hierarchy process 187 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.626 figure 1 ahp-based decision-making model for early career decision-making of youth section 4 presents the obtained research results, whereas section 5 summarizes them. 4. results and discussion expert significance rankings were collected online with the use of an expert choice inc. comparison™ suite academic license. each evaluator obtained a personalized link via an e-mail message that was sent by the software. aggregated and normalized research results were presented (figure 2). the aggregation of individual judgments (aij) through the weighted geometric mean method (wgmm) was applied. this was possible because the group structure was homogenous and individual respondents did not show any conflicts of interest (ossadnik, schinke & kaspar, 2016). maximization of individual perception of quality of life finance ability to save money for the future cost of living level of income financial investment risk safety, stability & certainty geopolitical safety and stability keeping contact with family and friends living without fear about the future predictability of consequences of our actions freedom & society being useful to the society free and safe travelling in an open world having access to credible information high legal and societal standards worklife balance combining private and professional life professional and individual development free time working in one's qualifications and interests area main goal criteria decision alternatives income oriented career oriented family oriented opting-out from the system sub-criteria ijahp article: gawlik/enhancing the work-life balance through ahp modeling of early career decision making international journal of the analytic hierarchy process 188 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.626 figure 2 aggregated ahp evaluation results with prioritization of parent criteria (%) the results in figure 2 prove that the assessments of significance of all criteria and subcriteria of the model (see figure 1 and table 1) show the respondents’ strongest preference towards a career-oriented life strategy (28.4%). the second preferred life strategy was income-oriented (23.4%), with an almost similar preference for a familyoriented one (22.8%). a significantly lower attractiveness was attributed to time-oriented (16.1%) and opt-out (9.3%) life strategies. it seems rational that young people on the verge of starting their professional life show a predominant interest in their future career and income. family values and free time, although still important, are dominated by the need for independence, which is also understandable. most interesting, is the wish of almost 10% of youth to opt-out entirely from the socio-economic system which apparently does not sufficiently answer their needs and expectations within any of the other four life strategies. the different colors in figure 2 represent the relevance of respective parent criterion in the assessment of a given life strategy. figure 3 aggregated prioritization of parent criteria (%) figure 3 shows the aggregated prioritization of parent criteria in the obtained responses, i.e. their importance for early-career decision-making of young people. the highest rank has been attributed to the group of criteria named safety, stability and certainty. the respondents perceived its relevance in the maximization of their overall life satisfaction at the level of 31.1% (out of 100%). work-life balance came second (24.4%), freedom and society came in third (23.5%), and finance ranked fourth (20.9%). these results stand in opposition to those presented on figure 2. several explanations are possible, e.g. the difference between internal motivations and those declared publicly by the respondents, the pressure for success from their environment, and the wish to combine colliding life finance safety, stability & certainty freedom & society work-life balance ijahp article: gawlik/enhancing the work-life balance through ahp modeling of early career decision making international journal of the analytic hierarchy process 189 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.626 strategies, etc. this issue definitely needs further research, as it could also shed light on the unexpectedly high attractiveness of the opt-out strategy. table 1 presents local and global prioritizations of decision criteria and sub-criteria that are a result of the discussed research project. the local priorities are the ratio-scale weights of a sub-criteria node with respect to the parent criterion. they add up to 100% inside one parent criterion. global priorities are the ratio-scale weights of any parent criterion with respect to the main goal. global priorities of all the lowest level sub-criteria sum to 100%. table 1 local and global prioritization of decision criteria and sub-criteria (%) criteria & sub-criteria prioritization (%) local global finance 25.29% 25.29% ability to save money and future retirement pension level 29.46% 7.45% cost of living 23.62% 5.97% level of income 35.94% 9.09% level of risk related to financial investments 10.98% 2.78% safety, stability and certainty 32.38% 32.38% geopolitical safety and stability 22.52% 7.29% keeping contact with family and friends 28.14% 9.11% living without fear about the future 24.44% 7.92% predictability of consequences of our actions 24.89% 8.06% freedom and society 25.10% 25.10% being useful to the society 19.38% 4.86% free and safe travelling in an open world 22.69% 5.70% having access to credible information 15.06% 3.78% living accordingly to high legal and societal standards 42.88% 10.76% work-life balance 17.22% 17.22% being able to combine private and professional life 28.04% 4.83% being able to develop professionally and pursue selfdevelopment 30.31% 5.22% free time 9.68% 1.67% working accordingly to your qualifications and interests 31.96% 5.50% a consistency check built in to the software was performed after each round of evaluations, when all pairwise comparisons for one parent criterion were finalized. an abbreviated consistency report was presented to the evaluators, who were asked to reassess their evaluations each time the inconsistency of their preference statements was higher than 10% (consistency ratio ≥ 0.1). due to their low consistency, the preference statements of two evaluators out of the initial 14 have not been included in the final results. ijahp article: gawlik/enhancing the work-life balance through ahp modeling of early career decision making international journal of the analytic hierarchy process 190 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.626 5. conclusion the outcome of the presented research is an aggregation of evaluation results of quality of life determinants provided by university students who are considering their professional path and future work-life balance. the created decision-making tool can be used for modeling the life satisfaction of future employees as a function of their individual assessments of significance of particular determinants of quality of life. the proposed model can be optimized with regard to its levels, e.g. ahp main goal (maximizing life satisfaction in general), ahp parent criteria and particular sub-criteria and ahp decision alternatives. the cognitive value of this research arises from three areas: (i) it identifies and helps understand the relationship between social, economic and psychological determinants of early career decisions of future employees; (ii) it supports the recent trend in economic research that forces researchers to reassess traditional rationales of decision-making processes of individuals (i.e. the paradigm of rationality of human behavior); (iii) it promotes an interdisciplinary approach to science, which should result in a more and more frequent inclusion of phenomena traditionally belonging to other scientific disciplines into socio-economic studies. the main research limitation comes from narrowing the group of experts to university students. in future research, the experts’ sample should be extended to people with a nonacademic background. moreover, similar research should be performed with groups of employers and employees that have been active on the job market between 5–10 years, 10–20 years, above 20 years, and up to 5 years before their retirement. a separate research should be devoted to a deeper understanding of the motivations of young people who were attracted to the opt-out life strategy. these research tasks can be performed by other researchers. the author will concentrate his future research on an in-depth insight into individual motivations of early career decision-making of future employees. it could result in a more accurate adaptation to trends in the labor market. another interesting question to explore is the reason for the low consistency of evaluations of two of the experts which resulted in excluding them from the final results. this issue will be explored in the author’s future research, along with a sensitivity analysis of the expert evaluations that were presented in this paper. the value added by the presented research comes from the increase of knowledge on the nature of one of the most important decisions in human life – the choice of a career path in accordance with one’s individual preferences on work-life balance. companies will gain more focused and better-motivated employees, who will be able to more closely follow their own development paths, leaving less opportunity for frustration and professional burnout. moreover, a satisfactory level of work-life balance indirectly contributes to an increase in the overall satisfaction level in society. newman et al. (2015) supports this by stating that initiatives by organizations to foster enhanced worklife balance would be expected to reap benefits not only to individuals and to organizations, but also to communities. more economists incorporating qualitative studies into their research and applying decision-making models would also add value. ijahp article: gawlik/enhancing the work-life balance through ahp modeling of early career decision making international journal of the analytic hierarchy process 191 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.626 references abel-smith, b., & townsend, p. (1965). the poor and the poorest. occasional papers on social administration, 17, london: bell & sons. adame, c., caplliure, e-m., & miquel, m-j. (2016). work–life balance and firms: a matter of women? journal of business research, 69(4), 1379–1383. doi: https://doi.org/10.1016/j.jbusres.2015.10.111 atkinson, a.b. (1983). the economics of inequality (2 nd ed). oxford: clarendon press. barzilai, j. (2001). notes on the analytic hierarchy process. in: proceedings of the 2001 nsf design, service & manufacturing grantees & research conference (1-6). tampa, fl: national science foundation. belton, v., & gear, a.e. (1983). on a shortcoming of saaty’s method of analytic hierarchies. omega, 11(3), 228–230. doi: https://doi.org/10.1016/0305-0483(83)90047-6 ben amor, s., jabeur, k., & martel, j-m. (2007). multiple criteria aggregation procedure for mixed evaluations. european journal of operational research, 181, 1506–1515. doi: https://doi.org/10.1016/j.ejor.2005.11.048 cabello, j.m., luque, m., miguel, f., ruiz, a.b., & ruiz, f. (2014). a multiobjective interactive approach to determine the optimal electricity mix in andalucía (spain). top, 22(1), 109–127. doi: https://doi.org/10.1007/s11750-011-0236-2 durbach, i.n., & stewart, t.j. (2012). modeling uncertainty in multicriteria decision analysis. european journal of operational research, 223(1), 1–14. doi: https://doi.org/10.1016/j.ejor.2012.04.038 dyer, j.s. (1990). remarks on the analytic hierarchy process. management science, 36(3), 249–258. gawlik, r. (2016a). methodological aspects of qualitative-quantitative analysis of decision-making processes. management and production engineering review, 7(2), 3– 11. doi: https://doi.org/10.1515/mper-2016-0011 gawlik, r. (2016b). stratification of research target group and selection of experts for ahp-based decision making model (stratyfikacja próby badawczej i dobór ekspertów na przykładzie modelu decyzyjnego opartego na metodzie ahp, in polish). studia i materiały "miscellanea oeconomicae", 20(3), 193–200. doi: https://doi.org/10.18559/soep.2017.1.4 gawlik, r. (2013). material and non-material determinants of european youth’s life quality. in: delener, n., fuxman, l., lu, f.v., rodrigues, s., rivera, l.e. (eds.) globalizing businesses for the next century: visualizing and developing contemporary approaches to harness future opportunities. (339-346). new york, ny: global business and technology association. http://apps-1webofknowledge-1com-1wos.han.uek.krakow.pl/full_record.do?product=wos&search_mode=generalsearch&qid=18&sid=q1sfvbltnyzjwrf5thc&page=1&doc=1 http://apps-1webofknowledge-1com-1wos.han.uek.krakow.pl/full_record.do?product=wos&search_mode=generalsearch&qid=18&sid=q1sfvbltnyzjwrf5thc&page=1&doc=1 ijahp article: gawlik/enhancing the work-life balance through ahp modeling of early career decision making international journal of the analytic hierarchy process 192 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.626 gawlik, r., & jacobsen, g. (2016). work-life balance decision-making of norwegian students: implications for human resources management. entrepreneurial business and economics review, 4(4), 153–170. doi: https://doi.org/10.15678/eber.2016.040410 gawlik, r., titarenko, r., titov, s. (2015). perception of quality of life and its components between russian students and its implications for university lecturers. horizons of politics, 6(16), 127–150. graafland, j., & compen, b. (2015). economic freedom and life satisfaction: mediation by income per capita and generalized trust. journal of happiness studies, 16(3), 789– 810. doi: https://doi.org/10.1007/s10902-014-9534-3 hansen, k.b. (2015). exploring compatibility between “subjective well-being” and “sustainable living” in scandinavia. social indicators research, 122(1), 175–187. doi: https://doi.org/10.1007/s11205-014-0684-9 ivlev, i., vacek, j., & kneppo, p. (2015). multi-criteria decision analysis for supporting the selection of medical devices under uncertainty. european journal of operational research, 247, 216–228. doi: https://doi.org/10.1016/j.ejor.2015.05.075 lau, e.y.y., cheung, s-h., lam, j. hui, c.h., cheung, s-f., & mok, d.s.y. (2015). purpose-driven life: life goals as a predictor of quality of life and psychological health. journal of happiness studies, 16(5), 1163–1184. doi: https://doi.org/10.1007/s10902014-9552-1 layard, r. (2005). happiness: lessons from a new science. new york, ny: penguin press. learmonth, y.c., alwick, e.a., mcauley, e., & motl, r.w. (2015). quality of life and health-related qol over 1 year in older women: monitoring stability and reliability of measurement. social indicators research, 123(1), 267–279. doi: https://doi.org/10.1007/s11205-014-0729-0 maslow, a. (1954). motivation and personality. new york, ny: harper. newman, a., nielsen, i., smyth, r., & hooke, a. (2015). examining the relationship between workplace support and life satisfaction: the mediating role of job satisfaction. social indicators research, 120(3), 769–781. doi: https://doi.org/10.1007/s11205-0140613-y ossadnik, w., schinke, s., & kaspar, r.h. (2016). group aggregation techniques for analytic hierarchy process and analytic network process: a comparative analysis. group decision and negotiation, 25(2), 421–457. doi: https://doi.org/10.1007/s10726015-9448-4 pomerol, j.c., & barba-romero, s. (2000). multicriterion decision in management: principles and practice. international series in operations research & management science, 25, new york, ny: springer. doi: https://doi.org/10.1007/978-1-4615-4459-3_5 mailto:ycl@illinois.edu mailto:russell.smyth@buseco.monash.edu.au ijahp article: gawlik/enhancing the work-life balance through ahp modeling of early career decision making international journal of the analytic hierarchy process 193 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.626 rezaei, j. (2015). best-worst multi-criteria decision-making method. omega, 53, 49–57. doi: https://doi.org/10.1016/j.omega.2014.11.009 russo, m., shteigman, a., & carmeli, a. (2016). workplace and family support and work–life balance: implications for individual psychological availability and energy at work. the journal of positive psychology, 11(2), 173–188. doi: https://doi.org/10.1080/17439760.2015.1025424 saaty, t.l. (1999). decision making for leaders: the analytic hierarchy process for decisions in a complex world. pittsburgh, pa: rws publications. saaty, t.l. (1996). multicriteria decision making. pittsburgh, pa: rws publications. saaty, t.l. (1980). the analytic hierarchy process. new york, ny: mcgraw-hill. saaty, t.l., vargas, l., & whitaker, r. (2009). addressing with brevity criticisms of the analytic hierarchy process. international journal of the analytic hierarchy process, 1(2), 121–134. doi: https://doi.org/10.13033/ijahp.v1i2.53 sałabun, w. (2014). reduction in the number of comparisons required to create matrix of expert judgment in the comet method. management and production engineering review, 5(3), 62–69. doi: https://doi.org/10.2478/mper-2014-0028 saracoglu, b.o., mahallesi, o., caddesi, t., & sokak, g. (2015). an ahp application in the investment selection problem of small hydropower plants in turkey. international journal of the analytic hierarchy process, 7(2), 211–239. doi: https://doi.org/10.13033/ijahp.v7i2.198 schuessler, k.f., & fisher, g.a. (1985). quality of life research and sociology. annual review of sociology, 11, 129–149. doi: https://doi.org/10.1146/annurev.soc.11.1.129 sen, a.k. (1985). commodities and capabilities. oxford: oxford university press. smith, a. (1759). the theory of moral sentiments. strand: a. millar; edinburgh: a. kincaid & j. bell. şerban-oprescu, g. (2012). economic approaches to sustainability and quality of life – an epistemological study. the romanian economic journal, 46 bis, 79–96. somarriba arechavala, n., zarzosa espina, p., & pena trapero, b. (2015). the economic crisis and its effects on the quality of life in the european union. social indicators research, 120(2), 323–343. doi: https://doi.org/10.1007/s11205-014-0595-9 teixeira de almeida, a., araujo de almeida, j., cabral seixas costa, a.p., & teixeira de almeida-filho, a. (2016). a new method for elicitation of criteria weights in additive models: flexible and interactive tradeoff. european journal of operational research, 250, 179–191. doi: https://doi.org/10.1016/j.ejor.2015.08.058 ijahp article: gawlik/enhancing the work-life balance through ahp modeling of early career decision making international journal of the analytic hierarchy process 194 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.626 ulman, p., & šoltés, e. (2015). the monetary and non-monetary aspects of poverty in poland and slovakia. entrepreneurial business and economics review, 3(1), 61–73. doi: https://doi.org/10.15678/eber.2015.030105 varmazyar, m., dehghanbaghi, m., & afkhami, m. (2016). a novel hybrid mcdm model for performance evaluation of research and technology organizations based on bsc approach. evaluation and program planning, 58, 125–140. doi: https://doi.org/10.1016/j.evalprogplan.2016.06.005 zopounidis, c., & doumpos, m. (2015). multicriteria decision systems for financial problems. top, 21(2), 241–261. doi: https://doi.org/10.1007/s11750-013-0279-7 zuzanek, j. (2009). time use imbalances: developmental and emotional costs. in: k. matuska & c. christiansen, c. (eds.), life balance: biological, psychological and sociological perspectives on lifestyle and health (207–222). bethesda: aota press. ijahp article: lipovetsky, conklin/ ahp priorities and markov steady-states probabilities international journal of the analytic hierarchy process 349 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.243 ahp priorities and markov-chapman-kolmogorov steady-states probabilities stan lipovetsky gfk north america, minneapolis, mn, usa e-mail: stan.lipovetsky@gfk.com w. michael conklin gfk north america, minneapolis, mn, usa e-mail: mike.conklin@gfk.com abstract an ahp matrix of the quotients of the pair comparison priorities is transformed to a matrix of shares of the preferences which can be used in markov stochastic modeling via the chapman-kolmogorov system of equations for the discrete states. it yields a general solution and the steady-state probabilities. the ahp priority vector can be interpreted as these probabilities belonging to the discrete states corresponding to the compared items. the results of stochastic modeling correspond to robust estimations of priority vectors not prone to influence of possible errors among the elements of a pairwise comparison matrix. keywords: ahp; markov stochastic modeling; chapman-kolmogorov equations. 1. introduction we consider a modified ahp for finding the robust preference estimation by a transformed matrix of the items shares. we show that this approach can be obtained from the markov stochastic processing in the form of chapman-kolmogorov equations using the pair comparison data for intensity of transitions among the items in a system of differential equations. solving this system yields the dynamic and the eventually reached steady-state probabilities. comparison shows that the results of markov modeling in the modified ahp and classical ahp priorities are very close. this means that the regular ahp preferences can be interpreted as the probabilities of belonging to the states corresponding to the compared items. the presented techniques have been studied in more detail in several works (lipovetsky, 1996; lipovetsky and tishler, 1999; lipovetsky and conklin, 2002, 2003). the considered methods enrich the possibilities of priority estimation and its application for various practical problems, particularly, in the marketing research field. the approach described in this paper is based on the initial transformation of the pairwise ratios matrix into the matrix of priority shares. this transformation makes the opposite ratios more balanced. for instance the pairwise reciprocal values, 9/1 and 1/9 differ by 81 times, but after the transformation they become 0.9 and 0.1, respectively, so they then differ by 9 times. there is also a problem because of the pairwise ratio’stransitivity and ijahp article: lipovetsky, conklin/ ahp priorities and markov steady-states probabilities international journal of the analytic hierarchy process 350 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.243 the limited scale. the theory says that for consistency in a pairwise ratio matrix the condition aijajk=aik should hold because of the theoretical relation between the priorities w: if aij=wi/wj and ajk=wj/wk, then (wi/wj)(wj/wk)=wi/wk=aik. but it is often impossible to remain within the bounded scale for the judgments, for instance, if aij=3 and ajk=5, then the consistent estimate should be aik=15. however, the scale is limited to a maximum value of 9, so an expert can only assign aik=9, and that would cause inconsistency in the matrix. thus, consistency can decrease not only because of the actual inconsistency in the expert judgments but also because of the bounded scale. a transformation of the shares, however, makes all the pairwise judgments bij and bji to belong to the 0-1 range of values where they are equidistant from the diagonal elements bii=0.5, so that decreases inconsistency. 2. ahp in a modified model a theoretical saaty matrix (saaty, 1980) of pair comparisons for n items defines each ijth element as a ratio of unknown priorities wi and wj:            nnnn n wwwwww wwwwww w /... // /...// 21 12111 . (1) multiplying matrix (1) by the vector )',....,,( 21 nwwww  we get the identical relation nwww  , (2) elicited from a judge, an empirical pair comparison matrix of priority ratios is                1... ........................... ...1 1 ...1 21 221 12 nn n n aa aa aa a . (3) this is a saaty matrix with transposed-reciprocal elements 1 jiij aa . (4) similarly to (2), priorities in the ahp are estimated by the eigenproblem for a matrix (3):  a , (5) where the maximum eigenvalue corresponds to the term n in (2), and the principal eigenvector α estimates the vector of priorities w. let us introduce a theoretical matrix of shares ijahp article: lipovetsky, conklin/ ahp priorities and markov steady-states probabilities international journal of the analytic hierarchy process 351 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.243               )/(... )w/()/( )/(...)/()/( 21 11211111 nnnnnnn n wwwwwwww wwwwwwwww u (6) each element uij is defined as i-th priority in the sum of i-th and j-th theoretical priorities: j w i w j w i w j w i w i w ij u /1 /     . (7) to estimate the priority vector using the matrix (6) we write identical equalities:                       nnn nn n n n n n n n n n nwww ww w ww ww w ww ww w nwww ww w ww ww w ww ww w )(...)()( )(...)()( 2 2 1 1 11 1 1 21 21 1 11 11 1 . (8) then using notation (7) we present the system (8) as:               nn n j njnnnn nn j j nwwuuwuwu nwwuwuw n uu )(... ...)( 1 2211 112121 1 111 (9) in the matrix form the system (9) is:   nwwuediagu  )( , (10) where u is the matrix (6), e denotes a uniform vector of n-th order, and diag(ue) is a diagonal matrix of totals in each row of matrix u. relations (8)-(10) for the theoretical matrix of shares (6) are derived similarly to the problem (2) for the matrix (1). in classical ahp, pair ratios wi/wj (1) are estimated by elicited values aij (3). using aij in (7) we obtain empirical estimates bij of the pairs’ shares: ij ij ij a a b   1 (11) this transformation of the elements of a matrix a (3) yields a pairwise share matrix b with elements (11). the elements of such a matrix (11) are positive, less than one, and have a property of symmetry: 1 jiij bb . (12) ijahp article: lipovetsky, conklin/ ahp priorities and markov steady-states probabilities international journal of the analytic hierarchy process 352 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.243 this means that the transposed elements bij and bji are equidistant from the diagonal elements bii=0.5, so jiiiiiij bbbb  . elements of a saaty matrix (3) with large or small values are transformed in (11) to the values closer to one or zero, respectively. in the ahp, for the empirical saaty matrix a (3) we have the eigenproblem (5) in place of the theoretical relations (2). by the same pattern, using empirical matrix b (11) in place of theoretical matrix u, we represent the system (10) as an eigenproblem     )( ebdiagb , (13) where α as a vector of priority. multiplying the matrix in (13) by the uniform vector and using (12) shows that this matrix has a property such that the total in its each column equals n:   enbeebeebdiagb  )( , (14) where prime denotes transposition. dividing both sides of equations (13) by the term n, we obtain an eigenproblem of a positive matrix with totals in the columns equal to one, which is an eigenproblem of the transposed stochastic matrix. such a matrix has the maximum eigenvalue equal to one. due to the perron-frobenius theory for a positive matrix, its main eigenvector always exists, is a unique one, and has all positive elements. thus, the maximum eigenvalue in (13) equals n, and a solution for the main eigenvector exists and is unique, which ensures in the desired properties of the priority vector. 3. ahp modeling in markov-chapman-kolmogorov equations the eigenproblem (13) has a matrix that is of a transposed stochastic kind that relates it to matrices known in markov modeling. consider a discrete state and continuous time markov model presented via chapman-kolmogorov differential equations describing a stochastic process of transitions among the states. this model is based on properties of a finite set of the elements (alternatives compared within a criterion) that are tied by the constant transition probabilities of each alternative’s prevalence over the others. the prevalence of one item over another one in the ahp corresponds to probability of the former item is preferred over the latter one in the eliciting process. the chapmankolmogorov equations express the change in probability to be found in any of n states as a linear combination of these probabilities with the coefficients of the transition intensities. taking a pair of the elements bij and bji of the share matrix (11) we notice that each element can be interpreted in terms of probability to prefer one of the items over another one, due to the meaning of the theoretical shares (7). the preference of an i-th item over a j-th item corresponds to transition between them with intensity bij. the share matrix b can be presented as a connected oriented graph with n nodes of states/alternatives and two edges between each of pair of nodes, one going to state i from state j corresponding to the transition intensity bij and the other going from state i to state j corresponding to the transition intensity bji. an example of such a network is shown in figure 1. ijahp article: lipovetsky, conklin/ ahp priorities and markov steady-states probabilities international journal of the analytic hierarchy process 353 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.243 figure 1. ahp network of the transition shares the system of chapman-kolmogorov equations can be presented as follows:            ) , ...() , ...( )...()...( 111111 111211212 1 nnnnnnnnn n nnn pbpbpbpb dt dp pbpbpbpb dt dp (15) where pi denotes probability to belong to an i-th state, and coefficients bij are the values (11). items with positive signs at the right-hand side (15) define influx to each state from all the others, and those with negative signs define departure from a state to all the other states. if canceling items 0.5pi are added to both positive and negative inputs in each i-th equation (15), this system can be represented in matrix form as:   pebdiagbp )(  , (16) where p is a vector of the probabilities pi for all the states, p denotes the vector of their derivatives, b is the same matrix with elements (11), b’ is its transposition, and e is the identity vector. using property (14) that the sum of totals in i-th column and row of the matrix b equals n, we can rewrite (16) as:   pniebdiagbp  )( , (17) where i denotes the identity matrix of the n-th order. considering the solution of the chapman-kolmogorov equations (17) for the steady-state probabilities when the process is stabilized, we put the derivatives in the left-hand side equal to zero, and (17) reduces to: b21 b12 b31 b13 b32 1 2 b23 3 ijahp article: lipovetsky, conklin/ ahp priorities and markov steady-states probabilities international journal of the analytic hierarchy process 354 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.243   pnpebdiagb  )( . (18) but (18) is nothing else but the same eigenproblem (13) with the largest eigenvalue n and a unique positive main eigenvector, as it was discussed in relation to equality (14). so the results of the ahp priority evaluation (13) can be interpreted from the point of view of the stochastic process steady-state solution (18) as follows: the ahp priority vector corresponds to the eventual probabilities of belonging to the discrete states, or alternatives, and these probabilities define the preferences among the compared items. finding a general dynamic solution for system (17) can be also implemented for some problems in the ahp. for instance, a researcher can be interested in differences among the preferences and in their specific behavior (monotonic increase, decrease, or oscillation) before the process stabilizing. the solution of a homogeneous linear system of differential equations with constant coefficients can be presented as: ct j diagptp ))(exp ()(  , (19) where c is a vector of constants, λj are the eigenvalues and p is a corresponding matrix of columns pj of eigenvectors obtained in the problem:   ppniebdiagb  )( . (20) this is the eigenproblem with the matrix at the right-hand side of chapman-kolmogorov system (17), and its solution coincides with the solution for the ahp problem (13) up to reducing the latter eigenvalues by n. for the initial moment t=0 the solution (19) is reducing to p(0)=pc, and solving this linear system with the known vector of initial conditions p(0) yields the vector of constants c=p -1 p(0). the general solution of the differential system is )0())(exp()( 1 pptdiagptp j    , (21) the expression 1))(exp( ptdiagp j in (21) is known as the matrix exponent. each component of the vector p(t) is a linear combination of the exponents in (21), and functions )exp ( tj behave in accordance with the specific values of λj obtained in the eigenproblem (20). as was mentioned above, the main eigenvalue in (20) is less by n than the main eigenvalue in (13), or (18), so it equals zero, λ1 = 0, that corresponds to the constant part of (21) behavior. the other eigenvalues (20) are real numbers or conjugated pairs of complex numbers. as we know from the perron–frobenius theory, all other eigenvalues have a less real value than the main eigenvalue, so all real eigenvalues or real parts in complex eigenvalues are negative. thus, the general behavior of solution (21) is defined by a constant part (λ1 = 0), by diminishing exponents (real negative eigenvalues), and by oscillating diminishing exponents (complex eigenvalues giving sine and cosine parts of the functions). there also can be polynomial items corresponding to equal eigenvalues, although in practical numerical evaluations such cases are rare. the eigenvectors p of the complex eigenvalues are complex, but the total expression (21) yields real values. ijahp article: lipovetsky, conklin/ ahp priorities and markov steady-states probabilities international journal of the analytic hierarchy process 355 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.243 4. numerical comparisons table 1 presents an example of a matrix a (3) of pair comparison among eight criteria used for the problem of “choosing the best home” – a classical ahp problem described in several articles (saaty, 1996; saaty and kearns, 1985; saaty and vargas, 1994). this matrix was also used for testing some new techniques in lipovetsky (1996), lipovetsky and tishler (1999), lipovetsky and conklin (2002). the items of comparison in table 1 are: 1 – size of house, 2 – location to bus, 3 – neighborhood, 4 – age of house, 5 – yard space, 6 – modern facilities, 7 – general condition, 8 – financing. the transformed matrix b (11) is presented in table 2. this transformation makes all the elements belong to 0-1 interval and diminishes the influence of any possible errors in pair comparisons. the row and column totals in table 2 correspond to be and b’e vectors related due to (14). the grand total can be used for checking – it equals n 2 /2. table 1 example of ahp pair comparison matrix a item 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 1/5 1/3 1/7 1/6 1/6 3 4 5 1 3 1/5 1/3 1/3 5 7 3 1/3 1 1/6 1/3 1/4 1/6 5 7 5 6 1 3 4 7 8 6 3 3 1/3 1 2 5 6 6 3 4 1/4 1/2 1 5 6 1/3 1/5 6 1/7 1/5 1/5 1 2 1/4 1/7 1/5 1/8 1/6 1/6 1/2 1 ijahp article: lipovetsky, conklin/ ahp priorities and markov steady-states probabilities international journal of the analytic hierarchy process 356 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.243 table 2 pair shares matrix b item 1 2 3 4 5 6 7 8 total 1 2 3 4 5 6 7 8 .500 .167 .250 .125 .143 .143 .750 .800 .833 .500 .750 .167 .250 .250 .833 .875 .750 .250 .500 .143 .250 .200 .143 .833 .875 .833 .857 .500 .750 .800 .875 .889 .857 .750 .750 .250 .500 .667 .833 .857 .857 .750 .800 .200 .333 .500 .833 .857 .250 .167 .857 .125 .167 .167 .500 .667 .200 .125 .167 .111 .143 .143 .333 .500 5.123 3.542 4.931 1.621 2.536 2.869 5.101 6.278 total 2.877 4.458 3.069 6.379 5.464 5.131 2.89 1.722 32 table 3 presents several methods of priority estimation. each vector is normalized so that the total equals one. the matrix a from table 1 is used to obtain three vectors shown as “regular methods” in table 3. the first of these vectors is obtained in the classic ahp eigenproblem (5), the next by the least squares approach also known in ahp, and the third vector is estimated by the multiplicative ahp technique (saaty and vargas, 1984, 1994; lootsma, 1999). the last column in table 3 contains the results of the modified estimation obtained by the transformed matrix b from table 2 in the eigenproblem (13) or the chapman-kolmogorov equations for the steady-states (18). the ranks of the items are shown in parentheses after the elements. the solutions yield different priority ordering that could indicate inconsistent relations in the pairwise matrix. table 3 priority vector in several estimations i eigenvector least squares multiplicative chapmankolmogorov 1 2 3 4 5 6 7 8 .173 (6) .054 (4) .188 (7) .018 (1) .031 (2) .036 (3) .167 (5) .333 (8) .199 (7) .100 (4) .148 (5) .017 (1) .045 (2) .065 (3) .184 (6) .242 (8) .175 (7) .063 (4) .149 (5) .019 (1) .035 (2) .042 (3) .167 (6) .350 (8) .150 (6) .054 (4) .141 (5) .022 (1) .037 (2) .041 (3) .163 (7) .392 (8) ijahp article: lipovetsky, conklin/ ahp priorities and markov steady-states probabilities international journal of the analytic hierarchy process 357 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.243 the eigenvalues in the problem (20) approximately equal the following values: .1.6 8 ,3.5 7 ,8.4 6 ,0.4 5 ,5.09.2 4,3 ,0.2 2 ,0 1   i the behavior of the solution (21) is mostly defined by the several first exponents with the bigger absolute value of the eigenvalues – those are the functions ),5.0sin( 9.2 ),5.0cos( 9.2 , 2 ,1 t t et t e t e  and the other functions’ decay is much steeper. with equal initial conditions pi(0)=1/n=0.125 in (21), the total behavior of the chapmankolmogorov solution is shown in figure 2. iterations p r i o r i t y 0 1 2 3 4 0.0 0.1 0.2 0.3 0.4 1 2 3 4 5 6 7 8 figure 2. ahp priority in chapman-kolmogorov solution the numbers of the alternatives are given at the right of the curves. beginning from about the third iteration, the process is stabilized. eventually, all the curves reach priority levels coinciding with those presented in the table 3 last column. 5. inconsistency and robust estimation as it was shown in lipovetsky and conklin (2002) on the same example, the appropriate ranks are given by the results in the last column of table 3, which presents the robust solution not prone to the possible inconsistencies in the data matrix. let us describe how to find such inconsistencies. returning to (1), we see that the theoretical saaty matrix equals the outer product 'vww  , (22) of a vector-column w of priorities )',....,,( 21 nwwww  (23) and a vector-row v' of element-wise reciprocal priorities (or anti-priorities): ijahp article: lipovetsky, conklin/ ahp priorities and markov steady-states probabilities international journal of the analytic hierarchy process 358 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.243 ),....,,(' 11 2 1 1   n wwwv . (24) multiplying the matrix (1) by the vector (23) from the right side, or by the vector (4) from the left side, we get identically true relations nvvwnwww  , (25) that correspond to the right and the left eigenproblems for the w matrix. notation w’ is used for the transposed w matrix. using vectors (23) and (24), consider the theoretical matrix (1)-(2) as the structure in the following table 2. table 4 theoretical saaty matrix as a contingency table anti-priority priority 1 2 … n row total 1 2 … n w1v1 w1v2 … w1vn w2v1 w2v2 … w2vn - - - - - - - - - - - - wnv1 wnv2 … wnvn w1jvj w2jvj ---- wnjvj column total v1iwi v2iwi …. vniwi (iwi )(jv j) each element wij of this matrix can be seen as product of i-th row sum and j-th column sum divided by grand total: totalgrand totalcolumntotalrow vww ji jiij )()(  (26) this type of theoretical (or expected) structure of the table is well-known in statistical analysis as a contingency table. ahp theoretical matrix (1) corresponds to an n by n contingency table of two vectors – priority (in rows) and anti-priority (in columns) (see table 2). the difference is that the ahp matrix does not describe counts, or frequencies, as a regular contingency table does. however, taking product p of all priorities wi in denominators of theoretical matrix (1) p = w1 w2 … wn (27) and multiplying each element of theoretical saaty matrix by this term results in integer numbers in place of the pairwise ratios (1): wij = (wi / wj ) p (28) these integer numbers (12) make sense of the proportions which describe how each item is preferred or failed in pairwise comparison with other items in consideration. relation ijahp article: lipovetsky, conklin/ ahp priorities and markov steady-states probabilities international journal of the analytic hierarchy process 359 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.243 (28) can be seen as a connection between ahp priority ratios and relative proportions of preference among the items. similarly to table 4, an elicited saaty matrix (3) can be presented as a contingency table in the following table 5. table 5 empirical saaty matrix as a contingency table anti-priority priority 1 2 … n row total 1 2 … n a11 a12 … a1n a21 a22 … a2n - - - - - - - - - - - - an1 an2 … ann j a1j j a2j ---- j anj column total i ai1 i ai2 …. i ain ij aij expected values of elements in this contingency table are defined similarly to the expression (26), using the margins of table 5: eij = (k aik )( t at j ) / ( k t ak t  to estimate the agreement between observed (empirical) and expected (theoretical) composition of numbers in cells, we use the chi-squared objective:     n ji ij ijij e ea 1, 2 2 )(  (30) if an elicited data matrix (3) corresponds exactly to the theoretical matrix (1) then all deviations equal zero and the value (30) is zero as well. the bigger the discrepancy between elicited data and theoretical ahp structure (1) the higher the value (30). for instance, using the same data from the example given in table 1, the total value (30) equals   = 32.7. the mean value of these deviations is 0.51, and the standard deviation is 1.51, and all the items of the chi-squared sum (30) are presented in table 6. the biggest deviations of empirical pairwise ratios from their theoretical values identify the coordinates of the inconsistent data in a saaty matrix. in our example, the chisquared item of one element a37 equals 11.91 which is far beyond a reasonable confidence interval for a mean value of 0.51 with a standard deviation of 1.51. the reciprocally symmetrical element a73 could be also considered as an outlier (both a37 and a73 are marked by asterisk in table 6). the input of these two elements among all 64 elements in sum (30) is about 41% of its total value. ijahp article: lipovetsky, conklin/ ahp priorities and markov steady-states probabilities international journal of the analytic hierarchy process 360 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.243 table 6 items of the chi-squared sum j i 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 0.32 0.44 0.85 0.00 0.10 0.22 1.14 1.08 0.13 0.44 0.07 0.07 0.31 0.63 0.27 0.25 0.51 0.36 0.25 0.00 0.01 0.18 1.54* 1.91 0.12 0.55 0.05 0.18 1.25 1.33 0.03 0.77 0.15 0.21 0.35 0.02 0.00 0.20 0.01 0.14 0.20 0.24 0.00 0.07 0.25 0.13 0.02 0.10 1.35 0.53 11.91* 0.00 0.09 0.23 0.38 0.17 0.12 0.03 0.11 0.17 0.05 0.01 0.00 0.15 criterion (30) uses the expected values estimated as the margins given in (29). more exact estimation of the expected values could be performed using the first pair of the vectors in the spectral decomposition of a matrix a. but a more convenient approach good for all practical needs can be the following one. when the theoretical saaty matrix is considered as a contingency table, all the rows are proportional one to another, and the columns as well. this means that for a more consistent empirical matrix the correlations between rows (and between columns) become closer to one. thus, instead of chi-squared we can find the matrix of correlations between rows and the matrix of correlations between columns – let us denote them as rr and rc. for each of these two matrices, calculate vectors of mean correlations in each row, and the results are as follows: mean (rr) = (.78 .85 .57* .80 .80 .83 .73 .73), mean (rc) = (.73 .83 .69 .78 .78 .79 .32* .73). the asterisk here marks the minimum values among the others, when the average correlation suddenly falls. we see that items 3 and 7 have the lowest mean correlations, so elements a37 and a73 of the saaty matrix in table 1 could be considered outliers. instead of averaging the elements in rows of the correlation matrices rr and rc we can apply principal component analysis to see the relations between the items. the first vectors of these two matrices are as follows: pc (rr) = (.35 .39 .25* .37 .37 .38 .36 .33), pc (rc) = (.36 .41 .34 .38 .39 .39 .12* .36), ijahp article: lipovetsky, conklin/ ahp priorities and markov steady-states probabilities international journal of the analytic hierarchy process 361 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.243 where the asterisk denotes positions of elements with the weights noticeably below the mean level of 35.08/1  . again, the results show that a37 and a73 are probable outliers in the saaty matrix in table 1, and they reduce consistency of the data. identification of the outliers can be found by means of the regular ahp measures. let us take the matrix in table 1 and solve the ahp problem (5). the maximum eigenvalue there is 9.67, and the consistency index ci=(-n)/(n-1) =0.24. excluding one item at a time from the set of eight and solving the ahp problem for each set of seven items, we get the following values of consistency index: ci = (0.20 0.27 0.12* 0.27 0.28 0.28 0.12* 0.26). we see that without the 3 rd and 7 th items, the consistency index falls lower, which indicates an improved consistency of the matrix. again, we localize a37 and a73 as the probable outliers in the data in table 1. in this assumption, we change the value of a37 (and take a73 as the reciprocal value) in a wide range of possible values, and each time solve the ahp problem (3) for finding maximum eigenvalue and the consistency index ci. the results are given in table 7. table 7 eigenvalue and consistency index of subsequently adjusted matrix a37 6 4 2 1 1/2 1/4 1/5 max 9.669 9.369 9.037 8.868 8.812 8.847 8.879 ci .238 .196 .148 .124 .116 .121 .126 table 7 shows that the best consistency (the minimum value of ci) is attained for the value a37 =1/2 (and a73 =2, respectively). adjusting by these values makes the matrix of table 1 become highly consistent, so any method yields very similar priority estimations and the same priority ordering. ranks of priorities for the adjusted matrix coincide with the ranks given in the last column of table 3. so we see that the results of the eigenproblem (13) or the chapman-kolmogorov equations for the steady-states (18) are very robust to possible inconsistencies in the data. being applied to the original matrix in table 1 with the inconsistent elements, the robust methods yield the same priority ordering as estimations by the regular ahp approach applied to the adjusted matrix with the corrected elements. these results were observed in numerous calculations with different data. finally, it is interesting to note that the eigenproblem (13) or (18) can be described in terms of the errors in the elicited data as follows. for the elements (11), consider a model with relative errors bij ( i + j ) = i (1+ ij ) (31) ijahp article: lipovetsky, conklin/ ahp priorities and markov steady-states probabilities international journal of the analytic hierarchy process 362 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.243 summing the relations (31) by index j yields a system of equations:      n j i n j ijjiji n j ij ninbb 1 11 ,...,1,)()(  . (32) in the assumption of equal sums of errors for all i, denote nin n j ij ,...,1, 1     , (33) then the system (32) reduces to the eigenproblem (13). thus, the eigenproblem (13) corresponds to the assumptions of multiplicative errors (31) and the equalized total errors in rows (33). 6. summary transformation of a pairwise ratio ahp matrix to the pairwise share matrix and solving the corresponding eigenproblem is considered. this approach can be obtained in chapman-kolmogorov modeling of transitions among the discrete states of the alternatives. coincidence of these results for ahp priority evaluation and of the stochastic steady-state solution suggests a useful interpretation: the ahp priorities have a meaning of the eventual probabilities of belonging to the discrete states of the compared items. the priorities expressed as the choice probabilities are useful in theoretical consideration and practical applications for various multiple criteria decision making problems. ijahp article: lipovetsky, conklin/ ahp priorities and markov steady-states probabilities international journal of the analytic hierarchy process 363 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.243 references lipovetsky, s. (1996) the synthetic hierarchy method: an optimizing approach to obtaining priorities in the ahp, european journal of operational research, 93, 550564. lipovetsky, s. (2005) analytic hierarchy processing in chapman-kolmogorov equations, international journal of operations and quantitative management, 11, 219-228. lipovetsky, s. & tishler, a. (1999) interval estimation of priorities in the ahp, european journal of operational research, 114, 153-164. lipovetsky, s., & conklin, w.m. (2002) robust estimation of priorities in the ahp, european journal of operational research, 137, 110-122. lipovetsky, s., & conklin, w.m. (2003). priority estimations by pair comparisons: ahp, thurstone scaling, bradley-terry-luce, and markov stochastic modeling, proceedings of the asa joint statistical meeting, jsm’03, 2473-2478, san francisco, ca. lootsma, f. (1999) multi-criteria decision analysis via ratio and difference judgement. doredrecht: kluwer academic publishers. saaty, t.l., (1980). the analytic hierarchy process. new york: mcgraw-hill. saaty, t.l. (1996). decision making with dependence and feedback: the analytic network process. pittsurgh, pa: rws publications. saaty, t.l. & kearns, k.p. (1985). analytical planning. new york: pergamon press. saaty, t.l. & vargas, l.g. (1984). comparison of eigenvalue, logarithmic least squares and least squares methods in estimating ratios, mathematical modelling, 5, 309-324. saaty, t.l. & vargas, l.g. (1994). decision making in economic, political, social and technological environment with the analytic hierarchy process. pittsburgh, pa: rws publications. | il modello multicriteriale a network, anp: ijahp article: lombardi, giordano, brunetta, caldarice/applying the analytical network process to identify new retail scenarios in the province of trento (italy) international journal of the analytic hierarchy process 145 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.221 applying the analytical network process to identify new retail scenarios in the province of trento (italy) patrizia lombardi politecnico di torino, interuniversity department of regional & urban studies and planning, viale mattioli, 39 – 10125 torino (italy) patrizia.lombardi@polito.it silvia giordano ex politecnico di torino, interuniversity department of regional & urban studies and planning, viale mattioli, 39 – 10125 torino (italy) silvia.giordano@polito.it grazia brunetta politecnico di torino, interuniversity department of regional & urban studies and planning, viale mattioli, 39 – 10125 torino (italy) grazia.brunetta@polito.it ombretta caldarice politecnico di torino, interuniversity department of regional & urban studies and planning, viale mattioli, 39 – 10125 torino (italy) ombretta.caldarice@polito.it abstract in the italian trentino area, the spatial planning institutional context is continuously evolving, especially in the field of retail planning tools. new emerging concepts and processes related to ex-ante evaluation of new retail settlements are highlighted in the regional decision process named territorial integrated evaluation (tie). this envisages a framework of criteria, strategies and action for the design of retail policies that can support the institutional innovation process of the valley communities. starting from this theoretical framework, this paper aims to illustrate a novel application of the analytic network process (anp) in the field of spatial planning with the purpose of investigating the interrelations between retail components and spatial development retail scenarios within the province of trento (italy). although the anp model was tested on the main municipalities of the trentino area (trento, rovereto, riva del garda, pergine valsugana and arco), this study presents the rovereto case study. the results obtained assist in the development of the operational evaluation system, that is easy to replicate to other spatial scenarios and that can support the policy makers in a participative process. keywords: analytic network process, decision support system, spatial planning, territorial development policy, territorial integrated evaluation (tie), retail location. mailto:patrizia.lombardi@polito.it mailto:silvia.giordano@polito.it mailto:grazia.brunetta@polito.it mailto:ombretta.caldarice@polito.it ijahp article: lombardi, giordano, brunetta, caldarice/applying the analytical network process to identify new retail scenarios in the province of trento (italy) international journal of the analytic hierarchy process 146 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.221 1. introduction despite institutional innovations in italian retail policy, the application of these innovations in most of the italian regions has only been half-heartedly pursued or completely disregarded until recently. italian retail policies do not generally consider territorial aspects or the impacts of retail settlements on the landscape. therefore, it is necessary to rethink this approach to the retail settlement processes’ management (brunetta and caldarice, 2014). this paper develops the theoretical framework for the experimentation of territorial integrated evaluation (tie) in the italian trentino area. in this region – the cradle of italy’s planning tradition – the institutional context is continuously evolving. for instance, in light of the indications of the recent trentino district plan, the newly formed valley communities are now setting up retail planning tools. this exceptionally dynamic context has allowed experimentation with some emerging concepts in spatial planning such as multi-disciplinarity. multi-disciplinarity is the connection between the territorial, environmental and economical dimensions of analysis and evaluation. the example of the trentino area has allowed us to apply and redesign the tie methodology with the aim of setting up local development scenarios which meet both the need for economic growth (in the retail and tourist sector) and the need for the conservation of exceptional landscape values. the outcomes of tie – which have been converted into local legislation in conformity with recent european and italian liberalization decrees – envisage a framework of criteria, strategies and action for the design of retail policies that can support the valley communities in their institutional innovation process. this paper reports a novel application of the analytic network process (anp) in the field of spatial planning with the aim of investigating the interrelations between retail components and spatial development retail scenarios. in particular, the paper focuses on the rovereto municipality as a relevant case study with the purpose of identifying key spatial development themes related to sustainable land use and improving policies for the positioning of new retail areas within the province of trento. the paper is structured as follows. after the introduction, the next section introduces the technical background and the role of territorial integrated evaluation (tie) to: (i) analyze territorial features, needs and requirements, (ii) define metrics to evaluate development scenarios and (iii) devise strategies for the localization of new retail areas. section 3 illustrates the results of the analytic network process application, which was conducted with the participation of a relevant number of province of trento’s stakeholders. finally, in section 4 the outcomes are discussed and final conclusions reached. 2. the territorial integrated evaluation (tie): a background territorial integrated evaluation (tie) is a technical method used for the ex-ante evaluation of the dynamics of new retail settlements in the regional decision process. formally, tie is the outcome of a research programme coordinated by the politecnico di torino from 2004 to 2007 for the retail department of the piedmont region. tie does not intend to replace the assessment procedures which are compulsory by law (such as the environmental impact assessment and the strategic environmental assessment), but aims to be a voluntary evaluation tool to help in the decision-making processes and to design the territorial transformation and development. therefore, the primary aim of tie is to support the development of spatial programming scenarios (brunetta, 2013) and to overcome the limitations of one-dimensional approaches to ijahp article: lombardi, giordano, brunetta, caldarice/applying the analytical network process to identify new retail scenarios in the province of trento (italy) international journal of the analytic hierarchy process 147 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.221 regional retail planning policies. the tie method necessarily requires a multidimensional and multidisciplinary approach in which territory (such as the land use and infrastructure system), landscape (such as the status and the evolution of the natural and cultural resources) and economic systems are integrated. this evaluation process has been closely linked to regional policy decision-making, and thus has been defined and proposed as the governance model for settlement of gross leasable areas (brunetta, 2008). it should be emphasised that tie is both a ‘voluntary’ procedure and a ‘holistic’ technical process for institutional learning. more generally, tie is configured as a consistent, logical and rational process with specific tools and techniques (themes, scales and indicators) of analysis and evaluation for the design of territorial development scenarios. because of these factors, the tie methodological procedure was applied to regional policy in planning new retail settlements in the trentino district. it was applied as cognitive support for the planning and design of new territorial development retail scenarios. tie proposes an alternative approach to consolidated strategic evaluation procedures in regional planning and identifies a technical proposal for innovation in institutional decision-making planning. finally, tie is a systematic process and a rational and consistent evaluation action based on a holistic evaluation framework. 2.1. themes, indicators and swot analysis to organise the structure of the evaluation, tie adopts and adapts some of the elements of the multi-criteria and swot analysis methods (strengths, weakness, opportunities, threats) while making a number of adjustments and changes. in accordance with the theoretical evaluation logic described above, the tie operational dimension is characterised by both a descriptive and a normative nature (brunetta, 2006; guba and lincoln, 1989; miller and patassini, 2005). with this in mind, the structure of the tie process relates to two interwoven scales of evaluation which correspond to two levels of territorial governance, both with their own precise mandate, purposes and resources. these two levels are as follows: (i) the macro-territorial level of evaluation, which is aimed at defining the programming scenario for trentino’s territories (valley communities); and (ii) the micro-territorial level of evaluation, which is aimed at defining town planning guidelines (municipalities). the tie process model defines five strategic themes of evaluation, outlined for the construction of territorial scenarios. these themes are as follows: (i) territorial retail supply, that describes quantitative and qualitative aspects of spatial retail dynamics; (ii) tourism, that describes the socioeconomic impact of tourism; (iii) urban settlement, that describes the land use of the areas; (iv) landscape, that describes the values and risks of landscape; and (v) local initiatives, that describe the valley communities’ planning ability. subsequently, the five strategic themes have been specified by using 34 indicators at the micro-territorial level and 36 indicators at the macro-territorial level (see table 1). table 1 strategic themes and indicators ijahp article: lombardi, giordano, brunetta, caldarice/applying the analytical network process to identify new retail scenarios in the province of trento (italy) international journal of the analytic hierarchy process 148 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.221 strategic themes macro-territorial level (valley communities) micro-territorial level (municipalities) territorial retail supply a.01m retail consistency a.01m retail consistency a.02m number of retail personnel a.02m retail realisation index a.03m number of retail personnel variation a.03m retail typology a.04m retail consistency variation a.04m retail specialisation a.05m retail density a.06m retail integration a.07m retail typology a.08m retail specialisation a.09m retail choice index a.10m retail evasion index tourism b.01m tourism intensity b.01m arrival b.02m tourism intensity (winter) b.02m tourist number b.03m tourism intensity (summer) b.04m tourist number variation b.05m tourist accommodation b.06m number of tourism personnel b.07m number of tourism personnel variation b.08m cycle path use urban settlement c.01m population variation c.01m retail spatial configuration c.02m urbanised areas incidence c.02m increase of urbanised areas index c.03m retail areas incidence c.03m agricultural areas incidence c.04m rural areas incidence c.04m reuse of disused area index c.05m grazing areas incidence c.05m retail areas incidence c.06m infrastructural density c.06m retail spatial distribution c.07m retail spatial diversification landscape d.01m protected areas incidence d.01m protected areas incidence d.02m agricultural areas incidence d.02m heritage incidence d.03m heritage incidence d.04m visual diversity consistency d.05m hydrogeological risk incidence d.06m hydrogeological danger incidence d.07m pollution d.08m air quality d.09m increase of urbanised areas index d.10m land consumption index local initiatives e.01m district financial helps for retail e.01m old town centre identification e.02m consortium financial helps for retail e.02m old shops identification e.03m district financial helps for retail e.04m consortium financial helps for retail e.05m reuse of disused area projects e.06m events once the indicators have been identified and analysed, the tie methodology uses the swot analysis to specify and develop, at the macro-level, the sub-regional policy framework guidelines and, at the micro-level, the strengths and weaknesses (or benefits and costs) of the dynamics already underway at the local municipality level. this allows for implementation of the dominant and prospective scenarios while paying particular attention to retail settlement (figure 1). ijahp article: lombardi, giordano, brunetta, caldarice/applying the analytical network process to identify new retail scenarios in the province of trento (italy) international journal of the analytic hierarchy process 149 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.221 figure 1. opportunities and threats the results have shown the different development scenarios in which retail – with the tourist industry and natural resources plays a role which varies according to local revitalisation perspectives. two scenarios – ‘dominant’ and ‘prospective’ – and key strategies, objectives and actions have been identified for each territorial context (micro and macro levels). in order to effectively implement the scenarios outlined by tie, the research has identified a number of conditions and criteria that affect the territories in which the implementation of retail policies are related to the different scenarios. 3. analytic network process support the tie methodology was reinforced and augmented using the analytic network process (anp) in order to model, cluster and begin evaluating possible spatial scenarios according to a ‘what-if’ framework. in the tie procedure, the anp method is developed at the micro-level (municipality level) comparing indicator priorities to the relative control criterion for a better definition of territorial scenarios (retail, marketing and cultural identity). in particular, the anp method enables the most relevant indicators defining the territorial scenario to be selected. it represents an advanced version of the analytic hierarchy process, theorised by saaty (1980; 2008), which describes the interaction between elements belonging to heterogeneous clusters in a complex system. the anp allows interactions and feedback within and between clusters providing a process from which ratio scale priorities from elements can be derived (saaty, 2005, 2013). this offers a more accurate and realistic representation for policy-making support (bottero et al., 2008; lombardi, 2007; saaty and vargas, 2006; saaty and ozdemir, 2008). relations can be bidirectional (i.e. cause-effect) or mono-directional (i.e. simple influence without reciprocity). the anp exercise was supported by super decisions 1.6.0 software 1 , adopting a structured complex model (see figures 2 and 3). the relationships between elements were identified by a focus group exercise organized in october of 2011 that involved 1 www.superdecisions.com ijahp article: lombardi, giordano, brunetta, caldarice/applying the analytical network process to identify new retail scenarios in the province of trento (italy) international journal of the analytic hierarchy process 150 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.221 all the politecnico research team members. relation identification defined a subnetwork for each control criteria previously set by swot analysis (see section 2.1). a scenario and indicator priority was provided for each sub-network. 3.1. application of the anp model at the urban level: the rovereto case study rovereto is a city of 38,244 inhabitants which is located in the autonomous province of trento in the vallagarina valley. the municipality is crossed by highway a22 and by state road ss12 fostering straightforward communications between trento and the rest of the province. accessibility is, therefore, one of the territory’s main characteristics. on the basis of prior spatial analysis (see section 2), a decision-making model made up of a main and secondary network (or sub-network), was developed. the methodological steps were as follows: a. structuring the decision-making model. the aim is to choose the most appropriate scenario at the micro level (urban/municipality) in order to define local planning guidelines. b. setting up the main network. the model is made up of a main network and a number of sub-networks. as outlined in figure 2, the main network has been developed starting from a swot analysis where strengths (i.e. benefits) and weaknesses (i.e. costs) are conceived as control criteria for a complex network model. these criteria are weighted on the basis of two spatial conditions: accessibility and spatial integration. this allows control criteria to be prioritized (or weighted) in relation to the strategic requirements previously identified by the trentino district plan. figure 2. main network at the spatial micro level c. defining the sub-network. the development of the sub-networks requires identifying: (i) a cluster of alternatives (the three aforementioned territorial scenarios of retail, cultural identity and tourism); (ii) a variable number of indicators grouped in homogeneous clusters (in relation to the swot analysis) and filtered down using related control criteria; (iii) the relationships between the sub-network nodes leading to the relationships between clusters. the micro level swot control criteria: strengths (or benefits) and weaknesses (or costs). in each one a sub-network is developed. goal: to weight the benefits and costs control criteria the spatial conditions for the control criteria evaluation: accessibility and spatial integration ijahp article: lombardi, giordano, brunetta, caldarice/applying the analytical network process to identify new retail scenarios in the province of trento (italy) international journal of the analytic hierarchy process 151 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.221 figure 3. costs sub-network d. pairwise comparison questionnaires. for each sub-network, using super decisions software, a number of pairwise comparisons were developed starting from the relations identified. the evaluation process required the participation of different stakeholders during a focus group. each node (indicator) and cluster is compared to another referring to a specific goal which is another element of the network. each network element could be a ‘parent node’ or a ‘child node’ in rotation. a binary relation is established between two child nodes in relation to the parent node (or comparison criterion). the possible question could be: “how important is node a compared to node b considering c as a goal?”. the evaluation process uses saaty’s fundamental scale (see figure 4) to assess the priority of the elements. this is a nine-point scale in which 2, 4, 6 and 8 are used as intermediate values (compromise). figure 4. saaty’ fundamental scale (saaty, 2008) the vector of priorities is the principal eigenvector of the matrix as derived from a set of pairwise comparison matrices. this vector gives the relative priorities of the criteria measured on a ratio scale. if one ensures that they sum to one, they are then unique and belong to a scale of absolute numbers (saaty, 2008). e. defining priorities. the expected results from the paired comparisons are an identification of: ijahp article: lombardi, giordano, brunetta, caldarice/applying the analytical network process to identify new retail scenarios in the province of trento (italy) international journal of the analytic hierarchy process 152 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.221  the importance (or weight) of the clusters (as tie strategic themes) in relation to the relative control criterion;  indicator priorities (as actions) in relation to the relative control criterion;  alternatives or spatial scenarios sorted by priority (weight) at the subnetwork level;  the most relevant spatial scenario obtained through the following synthesis formula (equation 1) $normalnet(benefits)*$smartalt(benefits)+$normalnet(costs) *$smartalt (costs) (1) where:  $smartalt(node name) refers to the best alternative in relation to the current network;  $normalnet(node name) refers to the normalized value of the node in relation to the current network. as recommended by adams and saaty (2003), influence was treated consistently how the parent influences the children, or vice versa but the flow direction was kept the same throughout the network and throughout the model. the super decisions software used for making comparisons and deriving priorities makes it possible to discover comparison coherence levels which are directly evaluated by the software. finally, the overall priorities for the rovereto municipality were derived (table 2). table 2 rovereto anp overall priorities subnet scenario actions nor lim criteria accessibility = 0,75 integration = 0,25 weight benefits = 0,37 costs = 0,63 b e n e f it s retail marketing identity 0,53 0,27 0,20 a.01m 0,39 0,18 b.01m 0,22 0,10 e.06m 0,55 0,09 a.06m 0,14 0,06 a.02m 0,13 0,05 c o s t s marketing retail identity 0,19 0,48 0,33 b.02m 0,90 0,25 d.02m 1,00 0,23 c.05m 0,77 0,09 e.01m 0,22 0,03 a.07m 0,09 0,02 result aggregation retail +0,56 marketing +0,32 identity +0,12 here, indicators are ordered starting from the most important ones, which are also the most interrelated, such as a.01m retail consistency for the benefit and b.02m tourist number for the cost. furthermore, table 2 shows the five indicators with the major priority in the model, both with regard to their weights on the clusters (see column ‘nor’) and in regard to their weight on the whole model (see column ‘lim’). ijahp article: lombardi, giordano, brunetta, caldarice/applying the analytical network process to identify new retail scenarios in the province of trento (italy) international journal of the analytic hierarchy process 153 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.221 3.2. discussion of the results obtained in the costs sub-network, the retail scenario has relevant weight (0.48) because of the landscape value preservation and the efforts made in trade-tourism system integration investments. however, within the benefits sub-network, the retail scenario is more relevant (0.53) because of the high incidence of the indicators relating to the distributive structure of commercial supply (a.01m a.06m). the marketing (0.27) and cultural identity (0.20) scenarios are definitely weaker in terms of benefits because in rovereto tourism is characterized by a low stay length in relation to good historical-cultural supply/demand and good local planning (e.06m). analyzing the results aggregation, the retail scenario prevails (0.56), underlining local strength in terms of retail consistency, retail variety and retail specialization. the marketing (0.32) and cultural identity (0.12) scenarios assign high scenic and ecological value to the territories together with a necessary strategy of integration with cultural resources (i.e. the cultural pole of the mart museum) in order to increase not only tourist numbers but also length of stay. indicator prioritization supports rovereto’s commercial vocation. the retail demand indicator, indeed, overbears and its importance is equal to 0.18 of the average of the whole model values. the impact of the spatial theme/analysis categories defined by the tie (see section 2) within the benefits sub-network is as follows: i. territorial retail supply, 47% ii. local initiatives, 21% iii. urban settlements, 14% iv. tourism, 13% v. landscape, 5% the cost sub-network indicators highlight the unsuitability of tourist investments as compared to city characteristics. as a matter of fact, transforming rovereto into a tourist destination does not appear the most suitable strategy. promoting and incentivizing commercial diversification would be a better strategy for implementing development and business opportunities. within this sub-network the most relevant (costly) spatial categories are tourism (40%) and landscape (37%); urban settlement (19%), and local initiatives (4%) have less affect than the other categories. 3.3. testing the results using stakeholders participation a local workshop that involved a group of technical stakeholders of the autonomous province of trento was organized in december 2011. the aim of this group was to test the rovereto anp model application and to compare the results obtained from the one developed by the politecnico research team with a new one developed with the support of local actors. the meeting was conducted as a focus group, and was composed of fifteen province of trento’ technical stakeholders with retail and tourism management skills. participants were asked to define priorities within the rovereto anp decision-making model by answering about 150 pairwise comparisons questions according to their knowledge of the case study. a comparison between the results obtained by the two anp applications (the one developed by the politecnico illustrated in section 3.1 and the new one developed with the support of local stakeholders) denoted a net discord ijahp article: lombardi, giordano, brunetta, caldarice/applying the analytical network process to identify new retail scenarios in the province of trento (italy) international journal of the analytic hierarchy process 154 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.221 both among the priorities of each sub network, and the synthesis priorities, as reported in figure 5. figure 5. comparison between the results obtained by the local actors focus group and those of the politecnico research group. the opposite trend highlighted in these results can be explained by the different approach adopted by the two groups in answering the pairwise comparisons. while the politecnico research group used quantitative data for the calculation of each indicator, following an objective evaluation, the local stakeholders answered in a subjective way, without any guidance or support other than their own technical skills and local knowledge. in the first application, the priority (0.56) attributed by the politecnico group to the commerce scenario corresponds to the value of the retail demand for the municipality of rovereto 64% and is considerably above the average of the other municipalities as shown in figure 6. figure 6. territorial retail supply of the main municipalities of the pat compared with the medium value. the objectivity of these data does not allow us to further interpret how elements and spatial dynamics are perceived by local people and stakeholders or give us future trends and perspectives. in the second application, retail implementation was perceived more as a problem than an opportunity. this is because of marked traffic congestion and parking issues that would worsen with further retail scenario development. moreover, limited knowledge of rovereto’s territorial characteristics and spatial dynamics unbalanced ijahp article: lombardi, giordano, brunetta, caldarice/applying the analytical network process to identify new retail scenarios in the province of trento (italy) international journal of the analytic hierarchy process 155 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.221 the evaluation process in favour of landscape preservation (as stated in the regional plan of vallagarina valley). the only element of coherence and confirmation between the two evaluations was related to the tourist sector. in general, both groups viewed the ‘arrivals’ (b05) indicator positively and as beneficial to the local economy. any attempt to lengthen tourist stay is perceived as an opportunity for both the municipality and the vallagarina valley (figure 7). figure 7. arrival (dark grey) and tourist number (light grey) compared to the medium value. 4. conclusions this paper illustrates an analytical network process application in the field of tie spatial planning and retail localization. primarily, this tie experimentation has provided the opportunity to discuss some general planning issues, as the link between theory and action in spatial planning. in this research this was realised through the scenario design of the sixteen valley communities of the trentino district. the spatial scenarios are not ‘closed’ visions, but are more similar to ‘open routes’ to territorial enhancement. these are linked to a set of shared criteria which act as the basis of the evaluation matrix as well as to a socially inclusive process of local planning action. therefore, this study is relevant to the current debate on development considering the fundamental synergy between university and government institutions (brandon and lombardi, 2011; lombardi et al., 2012). finally, this paper has demonstrated the flexibility of the anp approach in supporting different ways of reasoning and decision-making platforms in the field of land resource management. as already highlighted by the scientific literature (saaty and vargas, 2006; lombardi, 2010; ferretti, 2011), an application of anp methodology can support:  strategy definition for land use planning and management increasing (i) territory value through land marketing policies and (ii) considering crucial requirements (such as, in the present case-study, accessibility, land availability, disused areas, and environmental constriction).  indicator priority definition for the integration and social and environmental sustainability of the design/planning proposals. the test carried out on the municipality of rovereto was useful as it allowed us to verify the solidity and internal coherence of the anp decision-making model adopted in both the two applications. in addition, it confirmed the importance of involving ijahp article: lombardi, giordano, brunetta, caldarice/applying the analytical network process to identify new retail scenarios in the province of trento (italy) international journal of the analytic hierarchy process 156 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.221 local stakeholders in multicriteria anp evaluation related to public decision-making. finally, it has been useful for highlighting the importance of the tourism issue in a strategy of local economy enhancement. this anp application has a further more crucial added value. it represents the first concrete successful attempt to include anp modelling in a structured policy framework. from this perspective, it is relevant to highlight that the recent retail legislation of the province of trento (law 1339/2013) has adopted a framework of criteria, strategies and actions for the design of retail policies based on this anp model for supporting decision making in the valley communities institutional innovation process. ijahp article: lombardi, giordano, brunetta, caldarice/applying the analytical network process to identify new retail scenarios in the province of trento (italy) international journal of the analytic hierarchy process 157 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.221 references adams w.j. and saaty r. (2003). super decisions software guide, http://www.superdecisions.com (accessed 15 january 2013). bottero, m., lami, i. and lombardi, p. (2008). analytic network process: la valutazione di scenari di trasformazione urbana e territoriale. firenze: alinea editrice. brandon, p. s. and lombardi, p. (2011). evaluating sustainable development in the built environment. oxford: wiley-blackwell science. brunetta, g. and caldarice, o. (2014). self-organisation and retail-led regeneration: new territorial governance within the italian context. local economy, 29(4-5), 334344. brunetta, g. (2013). territorial integrated evaluation in spatial planning. scienze regionali italian journal of regional science, 12(2), 71-91. brunetta, g. (2008). valutazione territoriale integrata degli insediamenti commerciali [territorial integrated evaluation for retail settlements]. alinea, firenze. brunetta g. (2006).valutazione e pianificazione. verso l’integrazione? [evaluation in planning. which integration?]. scienze regionali italian journal of regional science, 5(3), 119-126. ferretti, v. (2011). a multicriteriaspatial decision support system (mc-sdss) development for siting a landfill in the province of torino (italy). journal of multicriteria decision analysis, 18, 231–252. guba, e. g. and lincoln, y. s. (1989). fourth generation evaluation, california: sage publication. lombardi, p., giordano, s., farouh, h. and yousef, w. (2012). modelling the smart city performance. innovation, 25(2), 137-149. lombardi p. (2010). strategic management plan evaluation of a river basin district. management of environmental quality, 21, 32-44. lombardi p. (2007). analytical hierarchy process. in deakin m., mitchell g., nijkamp p., vreeker r. (eds), sustainable urban development: the environmental assessment methods, vol. 2. (pp. 209-222). routledge: oxon. miller, d. and patassini, d. (eds.) (2005). beyond benefit cost analysis. aldershot: ashgate. saaty, t.l. (1980). the analytical hierarchy process. new york: mcgraw-hill. saaty, t.l. (2005). theory and applications of the analytic network process. pittsburgh, pa: rws publications. saaty, t. l. and m. s. ozdemir (2008). the encyclicon. a dictionary of complex decisions using the analytic network process. pittsburgh, pa: rws publications. saaty tl. and l.g. vargas (2006). decision making with the analytic network process. new york, usa: springer. saaty, t.l. (2008). decision making with the analytic hierarchy process. international journal of services sciences, 1(1), 83-98. http://porto.polito.it/2278658/ http://porto.polito.it/view/publication/management_of_environmental_quality.html ijahp article: lombardi, giordano, brunetta, caldarice/applying the analytical network process to identify new retail scenarios in the province of trento (italy) international journal of the analytic hierarchy process 158 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.221 saaty, t.l. (2013). the modern science of multicriteria decision making and its practical applications: the ahp/anp approach. operations research, 61(5), 11011118. ijahp article: de castro-pardo, de la fuente-cabrero, laguna-sánchez, pérez-rodriguez/ combining ahp and goal programming in the context of the assessment of e-learning international journal of the analytic hierarchy process 301 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.630 combining ahp and goal programming in the context of the assessment of e-learning mónica de castro-pardo* 1 universidad rey juan carlos monica.decastro@urjc.es concepción de la fuente-cabrero universidad rey juan carlos concepcion.delafuente@urjc.es pilar laguna-sánchez universidad rey juan carlos pilar.laguna@urjc.es fernando pérez-rodríguez fora forest technologies sll fernando.perez@fora.es *corresponding author abstract the analytical hierarchy process is a very common method used in multi-criteria decision making (mcdm) to analyze participative assessments. however, due to the qualitative nature of this methodology, a high percentage of inconsistencies need to be addressed when analyzing user preferences. this work analyzes the efficiency of the goal programming model in order to reduce inconsistencies with pairwise comparisons when working with inexpert participants and time limitations. a case study has been carried out that assesses online courses in higher education with the analytical hierarchy process in order to understand the usefulness and feasibility of the method. evaluation of four e-learning tools (collaboration tools, content tools, tutorial sessions and evaluation tools) used in an online business degree were collected from 72 students through a ‘saaty-type’ survey, and the model was applied to improve the consistency of these results. this model has been able to minimize the inconsistencies of individual preferences while avoiding the loss of primary information. keywords: goal programming; analytical hierarchy process; inconsistencies; elearning; participative decision making 1. introduction effective quality measures for e-learning have been described as being “urgently required” (martínez-caro et al., 2014). in this sense, it is important to remark on the this work was supported by the government of spain, department of economy, industry and competitiveness under the torres quevedo contract ptq-16-08633. mailto:monica.decastro@urjc.es mailto:concepcion.delafuente@urjc.es mailto:pilar.laguna@urjc.es mailto:fernando.perez@fora.es ijahp article: de castro-pardo, de la fuente-cabrero, laguna-sánchez, pérez-rodriguez/ combining ahp and goal programming in the context of the assessment of e-learning international journal of the analytic hierarchy process 302 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.630 importance of assessment in e-learning environments (strother, 2002; garcía-peñalvo & seoane-pardo, 2015). the participative processes oriented to distance learning assessment have been broadly studied (bozkurt et al., 2016). nevertheless, participating stakeholders could show inconsistencies, which could be circular or undefined preferences (brunelli, 2017). in such a situation, special attention must be given to the methodology of these processes in order to maintain objectivity and representation without losing usefulness or efficacy. for some participants, such as students, it is difficult to define individual preferences in the early stages of the decision-making process which makes it necessary to incorporate a high degree of iteration in certain phases of the evaluation process (owen, 2015). this can be tedious and adds complexity to the process, consuming additional resources (belton & steward, 2002). moreover, reviewing responses or asking for a repetition of responses from the same participant does not guarantee the reduction of inconsistencies. furthermore, on many occasions it is not possible to do this because of time limitations. some studies have utilized pairwise comparisons to assess e-learning systems (jeong & yeo, 2014; de castro et al., 2017). in fact, one of most used methods when designing participative processes is a methodology based on paired comparisons, the analytical hierarchy process (ahp) (saaty, 1990). the ahp has been applied to educational environments and has been applied to participative processes with users in order to assess e-learning (ho, 2008; lin, ho & chang, 2014). recently, studies have been published by anggrainingsih et al. (2018) that use ahp to evaluate elearning criteria such as “quality of design and material”, and by mohammed et al. (2018) that apply the same method with more technical criteria. nevertheless, it is common to obtain a high number of inconsistent primary observations because of the subjective nature of the human mind. to solve this problem, inconsistent responses are generally removed, or the valuations are repeated until they generate results with an acceptable consistency level (shee & wang, 2008; li & ma, 2007; lin et al., 2014). the first option, discarding inconsistent results, causes valuable information to be lost and may negatively impact the reliability of the result as it reduces the sample size. the second option, iterating the evaluation process, requires more resources and increases complexity. some studies evidence positive results for iteration in the participative process as it reduces conflicts and increases consensus. on the other hand, this option is only practical for small groups that are easily managed, with plenty of time, and an intimate knowledge of the evaluation process. however, this iterative process is generally unpractical due to the limited availability of some students. in this study, we propose the use of weighted goal programming to correct inconsistencies in the primary results and thus avoid information loss without modifying the data collection process (chen, kou & li, 2018). this model allows researchers to obtain consistent results that are as similar as possible to the original results. therefore, the objective of this work is to evaluate the applicability of using a goal programming model to reduce inconsistencies in participative evaluation systems which collect the preferences of higher education students that study business administration through online courses. 2. methodology “ahp is a multi-criteria decision making approach in which factors are arranged in a hierarchic structure” (saaty, 1990). ahp can measure preferences through pairwise ijahp article: de castro-pardo, de la fuente-cabrero, laguna-sánchez, pérez-rodriguez/ combining ahp and goal programming in the context of the assessment of e-learning international journal of the analytic hierarchy process 303 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.630 comparisons to derive priority scales. it is these scales that measure intangibles in relative terms (saaty, 2001). for these reasons, it has a wide area of applicability and has been successfully used to solve a wide variety of public and private sector decision making problems that require group consensus (belton & steward, 2002). ahp measures individual preferences through judgment evaluations on the relative importance of different paired criteria that are being considered. the decision maker can express the intensity of their preference on a 9-point scale. if two criteria are equally important, they receive a score of 1. a score of 9 indicates that a criterion is extremely preferable over another (saaty, 2001). through it, pairwise comparison scores are used to build reciprocal matrices. from these, the relative weights of each attribute are measured. based on these weights, the different alternatives are ranked. pairwise comparison matrices must be reciprocal, homogeneous and consistent (saaty, 2001). let m=(𝑚𝑖𝑗 )𝑖𝑗 a pairwise comparison matrix, m verifies the reciprocity condition when 𝑚𝑖𝑗 𝑚𝑗𝑖 = 1 ∀𝑖 ≠ 𝑗, and verifies the consistency condition when 𝑚𝑖𝑗 𝑚𝑗𝑘 − 𝑚𝑖𝑘 = 0 ∀𝑖 ≠ 𝑗 𝑗 ≠ 𝑖 (gonzález-pachón & romero, 2004). notwithstanding, in the decision making processes the consistency condition is usually not accomplished. participants in a decision-making process usually provide inconsistent results because the judgment calls have innate subjectivity. the level of consistency can be measured with the consistency index (ci), the cumulative average of matrix inconsistencies. the consistency ratio (cr) is the comparison between the ci and the random consistency index (ri). an acceptable cr is equal to or less than 0.10 (saaty, 2001). ahp inconsistency reduction has been studied in depth using different approaches (kulakowski, 2018). khatwani and kumar (2017) used a stochastic method to define the cosine consistency index. this method is based on a cosine maximization that uses an iterative basis to achieve the most consistent solution. in that case, ahp can be used iteratively until it achieves a consistent ratio. some studies have been oriented to deal with inconsistencies to improve the group decision making processes. fuhua et al. (2010) used two qualitative strategies allocating a weight vector based on the “expert’s experience value”. the main limitations of these methods are twofold: first, the loss of information is important and second, the participants may not feel the decision-making process and final result is their own. srdjevic et al. (2013) proposed a model to assign the weights to the users in order to obtain consistent results. however, the problem related to the lost information remained. moreover, the qualitative approach involves subjectivity and bias on the part of the user who is determining the weights. ivanco et al. (2017) used sensitivity analysis to improve the consistency of the ahp matrices, taking into account the consensus of the group solution. this method presents more flexibility in order to obtain a consensual solution, however it proposes to address users’ disparities without quantifying them. benitez et al. (2014) proposed a linear approach to obtain the closest consistent matrix through a suitable orthogonal projection expressed in terms of a fourier-like expansion. this method achieves the proposed goal, however, modeling the problem is very complex. ijahp article: de castro-pardo, de la fuente-cabrero, laguna-sánchez, pérez-rodriguez/ combining ahp and goal programming in the context of the assessment of e-learning international journal of the analytic hierarchy process 304 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.630 in this paper, we apply a simpler linear approach to optimize the consistency of the ahp matrices based on the goal programing method in order to improve the group decision making processes with inexpert participants. goal programming (gp) is a versatile multi-criteria technique used to resolve complex problems. in addition, it has been applied in other management science techniques (tamiz et al., 1998). gp finds compromise solutions that may not fully satisfy all the goals but do reach certain satisfaction levels set by the decision-maker. for this, an objective function and some constraints are defined. the constraints of the model are formed by the relationship between the objectives of the achievement level for each attribute with these attributes linking themselves through negative and positive deviations. gp can be modelled with different approaches: minmax gp, lexicographic gp and weighted gp. weighted gp is a linear model that minimizes the weighted sum of the deviations from each goal and provides the most balanced solution. minmax gp minimizes the maximum deviation between all possible deviations. lexicographic gp seeks to minimize an achievement function based on a pre-emptive or non-archimedean priorities approach (romero, 2014). in this specific case, we applied an archimedean gp model as laid out by gónzalez-pachón and romero (2004). with a n=4 matrix the model is as follows: min ∑ (𝑛𝑙 (1) + 𝑝𝑙 (1) )𝑝𝑙 + ∑ (𝑛𝑠 (2) + 𝑝𝑠 (2) )𝑝𝑠 + ∑ (𝑛𝑡 (2) + 𝑝𝑡 (2) )𝑝𝑡 (1) s.t. 𝑤𝑖𝑗 − 𝑚𝑖𝑗 + 𝑛𝑙 (1) − 𝑝𝑙 (1) = 0, l=1, 2,…, n(n-1), (2) 𝑤𝑖𝑗 𝑤𝑗𝑖 + 𝑛𝑠 (2) − 𝑝𝑠 (2) =1, 𝑠 = 1,2, … , 𝑛(𝑛−1) 2 , (3) 𝑤𝑖𝑗 𝑤𝑗𝑘 − 𝑤𝑖𝑘 + 𝑛𝑡 (3) − 𝑝𝑡 (3) =0, t=1, 2,…., n(n-1)(n-2), (4) 0.11≤ 𝑤𝑖𝑗 ≤ 9 ∀ 𝑖, 𝑗. (5) where: 𝑛𝑙 (1) and 𝑝𝑙 (1) are the negative and positive deviations of the goal, respectively, for constraints that ensure the condition of similarity in the position l, 𝑛𝑠 (2) and 𝑝𝑠 (2) are the negative and positive deviations of the goal, respectively, for constraints that ensure the condition of reciprocity in the position s, and 𝑛𝑡 (3) and 𝑝𝑡 (3) are the negative and positive deviations of the goal, respectively, for constraints that ensure the condition of consistency in the position t. 𝑚𝑖𝑗 are the components of the matrix m for each pair of criteria. 𝑤𝑖𝑗 are the components of the matrix w, formed by the weights that represent the most similar weights to the components of the original matrix m for each pair of criteria ij. these are the results of the model. let m=(𝑚𝑖𝑗 )𝑖𝑗 a general matrix given by a student, there exists a set of positive numbers, (𝑤𝑙 … 𝑤𝑛 ), such that 𝑚𝑖𝑗 = 𝑤𝑖 𝑤𝑗 for every 𝑖, 𝑗 = 1, … , 𝑛 this model uses a distance-based framework approach to inconsistencies in pairwise comparison matrices. the goal is to obtain a matrix that is as similar as possible to the ijahp article: de castro-pardo, de la fuente-cabrero, laguna-sánchez, pérez-rodriguez/ combining ahp and goal programming in the context of the assessment of e-learning international journal of the analytic hierarchy process 305 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.630 one generated by the decision maker while meeting saaty’s conditions of similarity, reciprocity, and consistency (gonzález-pachón & romero, 2004). after correcting inconsistencies in individual pairwise comparison matrices, we aggregated their values by calculating their geometric mean. the resulting matrix represents the collective evaluation of all participants. from this aggregated matrix, we generated the matrix of weights indicating the priorities of each tool in the achievement of the competency under study using the eigenvector method. 3. application the process was organized into two steps. first, we collected the evaluations obtained through a survey designed according to saaty’s (2001) guidelines and corrected the inconsistent matrices. second, conjoint results were assessed by the students in order to identify the consensus between individual preferences and aggregated results. the high levels of agreement suggest that this method was effective, which improves the entire process by making it more flexible, efficient, and practical. students were asked to evaluate four e-learning tools (collaboration tools, content tools, tutorial sessions and evaluation tools) used in an online business degree course, based on how well they helped them acquire a specific competency (table 1).the competency being evaluated was ‘ability to work autonomously’, which is especially relevant in online courses. table 1 description of e-learning tools analyzed as criteria in an inquiry e-learning tools objective collaborative facilitate the interaction with the professor and among students through chat, messages and a forum. contents providing the courses theoretic and practical assignments. evaluation allow students to follow the continuous evaluation process through tasks and test type exams. tutorial sessions to solve course doubts and questions at a one-to-one level. online or in person. the gp model was applied in the inconsistent matrices collected, as shown in the example below. example: let p={𝑐𝑙, 𝑐𝑙, 𝑡, 𝑒} be a set of pairwise comparisons that represent the individual preferences of one student about the importance of each type of e-learning tool (collaboratives (cl), contents (ct), tutorials (t) and evaluations (e)), for the achievement of the competency “ability to work autonomously” explained in the table 1. all these preferences were collected using saaty’s scale. also, the matrix m formed by the cardinal pairwise comparisons over p is: m=( 1 1/7 1 7 1 7 1 1 1/7 1/7 1 1 7 7 1 1 ) ijahp article: de castro-pardo, de la fuente-cabrero, laguna-sánchez, pérez-rodriguez/ combining ahp and goal programming in the context of the assessment of e-learning international journal of the analytic hierarchy process 306 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.630 so, the matrix m can be approximated by a reciprocal and consistent matrix, by using the following gp model: min ∑ (𝑛𝑙 (1) + 𝑝𝑙 (1) ) + ∑ (𝑛𝑠 (2) + 𝑝𝑠 (2) ) + ∑ (𝑛𝑡 (3) + 𝑝𝑡 (3) ) ,24𝑡=1 6 𝑠=1 12 𝑙=1 s.t. 𝑤12 − 1 7 + 𝑛1 (1) − 𝑝1 (1) = 0, 𝑤13 − 1 + 𝑛2 (1) − 𝑝2 (1) = 0, 𝑤14 − 7 + 𝑛3 (1) − 𝑝3 (1) = 0, 𝑤21 − 7 + 𝑛4 (1) − 𝑝4 (1) = 0, 𝑤23 − 7 + 𝑛5 (1) − 𝑝5 (1) = 0, 𝑤24 − 7 + 𝑛6 (1) − 𝑝6 (1) = 0, 𝑤31 − 1 + 𝑛7 (1) − 𝑝7 (1) = 0, 𝑤32 − 1/7 + 𝑛8 (1) − 𝑝8 (1) = 0, 𝑤34 − 1 + 𝑛9 (1) − 𝑝9 (1) = 0, 𝑤41 − 1 + 𝑛10 (1) − 𝑝10 (1) = 0, 𝑤42 − 1 7 + 𝑛11 (1) − 𝑝11 (1) = 0, 𝑤43 − 1 + 𝑛12 (1) − 𝑝12 (1) = 0; 𝑤12𝑤21 + 𝑛1 (2) − 𝑝1 (2) =1, 𝑤13𝑤31 + 𝑛2 (2) − 𝑝2 (2) =1, 𝑤14𝑤41 + 𝑛3 (2) − 𝑝3 (2) =1, 𝑤23𝑤32 + 𝑛4 (2) − 𝑝4 (2) =1, 𝑤24𝑤42 + 𝑛5 (2) − 𝑝5 (2) =1, 𝑤34𝑤43 + 𝑛6 (2) − 𝑝6 (2) =1; 𝑤13𝑤32 − 𝑤12 + 𝑛1 (3) − 𝑝1 (3) =0, 𝑤14𝑤42 − 𝑤12 + 𝑛2 (3) − 𝑝2 (3) =0, 𝑤12𝑤23 − 𝑤13 + 𝑛3 (3) − 𝑝3 (3) =0, 𝑤14𝑤43 − 𝑤13 + 𝑛4 (3) − 𝑝4 (3) =0, 𝑤12𝑤24 − 𝑤14 + 𝑛5 (3) − 𝑝5 (3) =0, 𝑤13𝑤34 − 𝑤14 + 𝑛6 (3) − 𝑝6 (3) =0, 𝑤23𝑤31 − 𝑤21 + 𝑛7 (3) − 𝑝7 (3) =0, 𝑤24𝑤41 − 𝑤21 + 𝑛8 (3) − 𝑝8 (3) =0, 𝑤21𝑤13 − 𝑤23 + 𝑛9 (3) − 𝑝9 (3) =0, 𝑤24𝑤43 − 𝑤23 + 𝑛10 (3) − 𝑝10 (3) =0, 𝑤21𝑤14 − 𝑤24 + 𝑛11 (3) − 𝑝11 (3) =0, 𝑤23𝑤34 − 𝑤24 + 𝑛12 (3) − 𝑝12 (3) =0, 𝑤32𝑤21 − 𝑤31 + 𝑛13 (3) − 𝑝13 (3) =0, 𝑤34𝑤41 − 𝑤31 + 𝑛14 (3) − 𝑝14 (3) =0, 𝑤31𝑤12 − 𝑤32 + 𝑛15 (3) − 𝑝15 (3) =0, 𝑤34𝑤42 − 𝑤32 + 𝑛16 (3) − 𝑝16 (3) =0, 𝑤31𝑤14 − 𝑤34 + 𝑛17 (3) − 𝑝17 (3) =0, 𝑤32𝑤24 − 𝑤34 + 𝑛18 (3) − 𝑝18 (3) =0, 𝑤42𝑤21 − 𝑤41 + 𝑛19 (3) − 𝑝19 (3) =0, 𝑤43𝑤31 − 𝑤41 + 𝑛20 (3) − 𝑝20 (3) =0, 𝑤41𝑤12 − 𝑤42 + 𝑛21 (3) − 𝑝21 (3) =0, 𝑤43𝑤32 − 𝑤42 + 𝑛22 (3) − 𝑝22 (3) =0, 𝑤41𝑤13 − 𝑤43 + 𝑛23 (3) − 𝑝23 (3) =0, 𝑤42𝑤23 − 𝑤43 + 𝑛24 (3) − 𝑝24 (3) =0; 0.11≤ 𝑤𝑖𝑗 ≤ 9 ∀ 𝑖, 𝑗. as a result of the application of the model in the matrix m, we obtained a consistent matrix w: w=( 1 1/7 1 7 1 7 1 1 1/7 1/7 1 1 𝟏 7 1 1 ) , where the only corrected component of the original matrix was (𝑚14). this change only permitted one to obtain a matrix with a cr=0 when the index of the original matrix was cr=0.4192. this process was applied to improve the consistency of all the matrices that represented the students' preferences, that were obtained using the saaty survey, with a cr>0.10. ijahp article: de castro-pardo, de la fuente-cabrero, laguna-sánchez, pérez-rodriguez/ combining ahp and goal programming in the context of the assessment of e-learning international journal of the analytic hierarchy process 307 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.630 4. results and discussion individual opinions were collected from the 72 students through a ‘saaty-type’ survey with a 1 to 9 scale. in it, they were asked to perform pairwise comparisons of collaboration, content, tutorial sessions, and evaluation tools as a means of acquiring the ‘ability to work autonomously’ competency. from the resulting 72 pairwise comparison matrices, 7 were excluded for being incomplete or incorrectly completed. of the remaining 65 valid matrices, 8 had a consistency ratio of less than 0.1. the inconsistent results were corrected by modeling a goal programming function using lingo 17.0. we obtained improvements in the consistency of 57 matrices. as a result, 65 n=4 matrices were obtained with a consistency ratio under 0.1. differences between the weights obtained with corrected inconsistencies and the weights obtained with the original results removing inconsistent answers were not relevant, but these differences changed the final priorities over each e-learning tool (table 2). table 2 results and differences between results with corrected inconsistencies and results with original results removing inconsistent answers e-learning tools consistent results without corrections results with corrected inconsistencies (de castro et al., 2017) differences in percentages collaborative 20.33% 18.91% 1.42% contents 32.23% 32.53% -0.30% evaluation 16.84% 21.89% -5.05% tutorial sessions 30.59% 26.66% 3.93% table 3 results and differences between results with corrected inconsistencies and results with original results considering inconsistent answers e-learning tools results with inconsistent and consistent matrices results with corrected inconsistencies (de castro et al., 2017) differences in percentages collaborative 19.05% 18.91% 0.14% contents 34.67% 32.53% 2.14% evaluation 25.39% 21.89% 3.5% tutorial sessions 20.87% 26.66% -5.79% the results after correcting the inconsistencies give more weight to evaluation tools and less to collaborative tools. collaborative tools are prioritized in the same order with just 1.42% less relative importance than the original results. content tools are similarly ranked in the original and corrected versions (table 2). ijahp article: de castro-pardo, de la fuente-cabrero, laguna-sánchez, pérez-rodriguez/ combining ahp and goal programming in the context of the assessment of e-learning international journal of the analytic hierarchy process 308 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.630 furthermore, when the original inconsistent and consistent results were compared with the results, after correcting the inconsistencies, differences were found. tutorial sessions showed the highest divergences when the original inconsistencies remained (table 3). this shows the effect of the inconsistent results over the group solution. table 4 ranking provided by the inconsistent and consistent original results, the original results removing inconsistent answers and the results with corrected inconsistencies e-learning tools ranking inconsistent and consistent results ranking only consistent results ranking with corrected inconsistencies collaborative 4 3 4 contents 1 1 1 evaluation 2 4 3 tutorial sessions 3 2 2 these differing prioritizations between the results after discarding the inconsistent preferences and the results with corrected inconsistences, illustrate the effect of information loss on results (table 4). a sensitivity analysis performed on the aggregated results shows the importance of evaluation tools in acquiring the competency under study, providing the most robust prioritization (figure 1). figure 1 graphic of sensitivity analysis with variation of one positive point in each pair of aggregated comparison matrices 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 sensitivity analisys collaborative contents tutorial sessions evaluation ijahp article: de castro-pardo, de la fuente-cabrero, laguna-sánchez, pérez-rodriguez/ combining ahp and goal programming in the context of the assessment of e-learning international journal of the analytic hierarchy process 309 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.630 furthermore, the differing prioritizations between the results considering inconsistencies, the results with corrected inconsistencies and the results after discarding inconsistent preferences show the effect of inconsistent responses on distorting global results (table 4). the goal programming model proposed in this work consolidates the weights of content and evaluation tools, thus reducing the sensitivity of the overall results. the data suggests that information loss can distort evaluation results and diminish the quality of the process. after this phase, students were asked to complete an online, likert-scale survey that focused on their level of agreement or disagreement with the results, where 1 represented the minimum agreement and 5 represented the maximum agreement. here, 87.5% of respondents agreed highly or very highly with the priority ranking generated by the aggregated matrices corrected for inconsistencies. this high level of agreement demonstrated the effectiveness of the proposed model to treat inconsistencies in pairwise comparison matrices of e-learning tools for acquiring competencies. further validation was provided by high levels of participant satisfaction with the aggregated results. this would seem to suggest that the changes carried out to diminish inconsistencies did not significantly alter the opinion of the group. finally, the model allowed researchers to recover 90.47% of the missing information while maintaining the flexibility of the evaluation process; thus, making it more practical. in addition, the high level of agreement from the participants with the results validates the effectiveness of this method. 5. conclusion global results are different when consistency is improved using the proposed gp model. thus, e-learning tools received different weights when inconsistencies were corrected. both results, corrected and primary, agree with the assessment that content and tutorial sessions are the most important elements, even though tutorial sessions received a lower weight with the corrected matrices. notably, the corrected model prioritizes evaluation tools over collaborative tools. gp is an effective technique when correcting inconsistencies in pairwise comparison matrices as applied to higher education evaluation systems. by correcting the primary results, both the quality and the agility of the evaluation process are improved. the gp model has improved the performance of the ahp method in order to reduce the inconsistencies of the pairwise comparison matrices, solving some of the limitations of previously proposed methods. first, the loss of information has been avoided. second, the preferences of all the participants have been considered in the decisionmaking process. third, the applicability of the process has remained, thus avoiding the use of iterations. finally, rigor has been maintained throughout the process. ultimately, the use of gp in the proposed model efficiently improved the analytical hierarchy process in the context of working with inexpert users such as those who might evaluate an online business degree course. ijahp article: de castro-pardo, de la fuente-cabrero, laguna-sánchez, pérez-rodriguez/ combining ahp and goal programming in the context of the assessment of e-learning international journal of the analytic hierarchy process 310 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.630 references anggrainingsih r, umam mz, setiadi h. (2018). determining e-learning success factor in higher education based on user perspective using fuzzy ahp. matec web of conferences, 154. doi: 10.1051/matecconf/201815403011 belton, v., stewart, t. (2002). multiple criteria decision analysis: an integrated approach. boston: kluwer springer science and business media. benítez, j., izquierdo, j., pérez-garcía, r., & ramos-martínez, e. (2014). a simple formula to find the closest consistent matrix to a reciprocal matrix. applied mathematical modelling, 38(15-16), 3968-3974. doi: https://doi.org/10.1016/j.apm.2014.01.007 bozkurt, a., akgun-ozbek, e., yilmazel, s. et al. 2015. trends in distance education research: a content analysis of journals 2009-2013. the international review of research in open and distributed learning, 16(1), 330-363. doi: http://dx.doi.org/10.19173/irrodl.v16i1.1953 brunelli, m. (2017). studying a set of properties of inconsistency indices for pairwise comparisons. annals of operations research, 248(1-2), 143-161. doi: 10.1007/s10479-016-2166-8 chen, k., kou, g., li, c. (2018). a linear programming model to reduce rank violations while eliciting preference from pairwise comparison matrix. journal of the operational research society, 69(1-12). doi: 10.1080/01605682.2017.1409156 cho, y. g., & cho, k. t. (2008). a loss function approach to group preference aggregation in the ahp. computers & operations research, 35(3), 884-892. doi: https://doi.org/10.1016/j.cor.2006.04.008 de castro, m., de la fuente-cabrero, c., laguna sánchez, mdp (2017). assessment of autonomous learning skill through multi-criteria analysis for online ade students in moodle. in: m. peris-ortiz, j. gómez, j. merigó-lindahl, and c. rueda-armengot (eds). entrepreneurial universities: exploring the academic and innovative dimensions of entrepreneurship in higher education. (197-2013). washington, dc, usa: springer. garcía-peñalvo, fj, seoane-pardo, am (2015). una revisión actualizada del concepto de elearning. décimo aniversario education in the knowledge society, 16(1), 119144. doi: 10.14201/eks2015161119144 gonzález-pachón j, romero c (2004) a method for dealing with inconsistencies in pairwise comparisons. european journal of operational research, 158(2), 351-361. doi: 10.1016/j.ejor.2003.06.009 ho, w. (2008). integrated analytic hierarchy process and its applications–a literature review. european journal of operational research, 186(1), 211-228. doi: 10.1016/j.ejor.2007.01.004 https://doi.org/10.1016/j.apm.2014.01.007 http://dx.doi.org/10.19173/irrodl.v16i1.1953 https://doi.org/10.1016/j.cor.2006.04.008 ijahp article: de castro-pardo, de la fuente-cabrero, laguna-sánchez, pérez-rodriguez/ combining ahp and goal programming in the context of the assessment of e-learning international journal of the analytic hierarchy process 311 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.630 ivanco, m., hou, g., & michaeli, j. (2017). sensitivity analysis method to address user disparities in the analytic hierarchy process. expert systems with applications, 90, 111-126. doi: https://doi.org/10.1016/j.eswa.2017.08.003 jeong, hy, yeo, ss (2014). the quality model for e-learning system with multimedia contents: a pairwise comparison approach. multimedia tools and applications, 73(2), 887-900. doi: 10.1007/s11042-013-1445-5 khatwani, g., & kar, a. k. (2017). improving the cosine consistency index for the analytic hierarchy process for solving multi-criteria decision making problems. applied computing and informatics, 13(2), 118-129. doi: https://doi.org/10.1016/j.aci.2016.05.001 kułakowski, k. (2018). inconsistency in the ordinal pairwise comparisons method with and without ties. european journal of operational research, 270(1), 314-327. doi: https://doi.org/10.1016/j.ejor.2018.03.024 li, hl, ma, lc (2007). detecting and adjusting ordinal and cardinal inconsistences through a graphical and optimal approach in ahp models. computers and operations research, 34(3), 780-798. doi: 10.1016/j.cor.2005.05.010 lin, tc, ho, hp, chang, ct (2014). evaluation model for applying an e-learning system in a course: an analytic hierarchy process-multi-choice goal programming approach. journal of educational computing research, 50(1), 135-157. doi: https://doi.org/10.2190/ec.50.1.g martínez-caro, e., cegarra-navarro, jg, cepeda-carrion, g. (2015). an application of the performance-evaluation model for e-learning quality in higher education. total quality management and business excellence, 26(5-6), 632-647. doi: 10.1080/14783363.2013.867607 mohammed, hj, kasim, mm, shaharanee, in (2018). evaluation of e-learning approaches using ahp-topsis technique. journal of telecommunication, electronic and computer engineering (jtec,) 10(1-10), 7-10. owen, d. (2015). collaborative decision making. decision analysis, 12(1), 29-45. doi: 10.1287/deca.2014.0307 romero, c. (2014). handbook of critical issues in goal programming. elsevier. saaty, tl (1990). how to make a decision: the analytic hierarchy process. european journal of operational research, 48(1), 9-26. doi: 10.1016/0377-2217(90)90057-i saaty, tl, vargas, l. (2001). models, methods, concepts and applications of the analytic hierarchy process. london, united kingdom: kluwer academic publishers. shee, dy, wang, ys (2008). multi-criteria evaluation of the web-based e-learning system: a methodology based on learner satisfaction and its applications. computers and education, 50(3), 894-905. doi: 10.1016/j.compedu.2006.09.005 srdjevic, b., srdjevic, z., blagojevic, b., & suvocarev, k. (2013). a two-phase algorithm for consensus building in ahp-group decision making. applied https://doi.org/10.1016/j.eswa.2017.08.003 https://doi.org/10.1016/j.aci.2016.05.001 https://doi.org/10.1016/j.ejor.2018.03.024 https://doi.org/10.2190/ec.50.1.g https://doi.org/10.1287/deca.2014.0307 https://doi.org/10.1016/0377-2217(90)90057-i ijahp article: de castro-pardo, de la fuente-cabrero, laguna-sánchez, pérez-rodriguez/ combining ahp and goal programming in the context of the assessment of e-learning international journal of the analytic hierarchy process 312 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.630 mathematical modelling, 37(10-11), 6670-6682. doi: https://doi.org/10.1016/j.apm.2013.01.028 strother, jb (2002). an assessment of the effectiveness of e-learning in corporate training programs. the international review of research in open and distributed learning, 3(1). tamiz, m., jones, d., romero, c. (1998). goal programming for decision making: an overview of the current state-of-the-art. european journal of operational research, 111(3), 569-581. doi: 10.1016/s0377-2217(97)00317-2 https://doi.org/10.1016/j.apm.2013.01.028 ijahp article: koyun yilmaz, ozkir /extended consistency analysis for pairwise comparison method international journal of the analytic hierarchy process 117 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.506 extended consistency analysis for pairwise comparison method sahika koyun yılmaz yildiz technical university turkey skoyun@yildiz.edu.tr vidan ozkir yildiz technical university turkey cvildan@yildiz.edu.tr abstract pairwise comparison (pc) is a widely used scientific technique to compare criteria or alternatives in pairs in order to express the decision maker’s judgments without the need for a unique common measurement unit between criteria. the method constructs a pc matrix by requesting the assessments of the decision maker(s) in the judgment acquisition phase and calculates an inconsistency measure to determine whether the judgments are adequately consistent with each other before subsequent phases. although the method requires the decision maker to make all judgments in a pc matrix, it does not force him/her to make a judgment for each element of the matrix. if any judgment in a pc matrix is absent, for this reason, the judgment acquisition phase yields an incomplete pc matrix rather than a complete one. missing judgments are calculated by multiplication of the judgments made by the decision maker. if the judgements of the decision maker are transitive and well-proportioned, missing judgments will not disturb the consistency of the resulting pc matrix. in other words, consistency of a pc matrix relies on the judgments made by the decision maker. since the current consistency analysis procedure is designed for complete pc matrices, the suitability for evaluating the inconsistency of incomplete pc matrices is questionable. probability density functions of random pc matrices with altering numbers of missing judgments show distinct features, indicating an incomplete pc matrix and a complete pc matrix do not come from the same probability function, and their mean consistency index (ri) is different. consequently, we propose an extended consistency analysis procedure to evaluate the consistency of incomplete pc matrices. keywords: consistency analysis; decision support systems; pairwise comparison; random index 1. introduction multiple attribute decision-making problems are frequently encountered in real life decision making, and they search for the best alternative among a set of feasible alternatives regarding a set of predefined criteria. these problems require examining a number of feasible alternatives with respect to a number of criteria in order to determine ijahp article: koyun yilmaz, ozkir /extended consistency analysis for pairwise comparison method international journal of the analytic hierarchy process 118 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.506 the best decision. as the number of criteria increases, the problem becomes more complex. the construction of the attribute scales and their associated weights challenge decision makers because of the absence of a common unit of measurement. multiple criteria decision making often requires tradeoffs between money, environmental quality, health and similar entities (thurstone, 1994). in order to get relative weights of importance, the pc method processes both objective assessments and subjective judgments of decision makers together without the need of a common unit of measurement. the consistency analysis procedure is used to determine whether the judgments in a pc matrix are appropriate for use or not. if the inconsistency of judgments is unacceptable, the decision maker is asked to revise his/her judgments. the pc method cannot automatically eliminate inconsistency caused by the initial judgments. therefore, the viability of the final decision substantially depends on the judgments of the decision maker. current consistency analysis procedure consists of two steps as follows: the first step evaluates if the given judgments are consistent within, and the second step evaluates if the matrix is consistent in comparison to randomly generated matrices. if a pc matrix satisfies both steps, then it is accepted as consistent. moreover, the procedure does not address the case of missing information. it assumes that the decision maker can provide all initial judgments in the pc matrix. saaty (1980) explained a computational procedure which preserves the mathematical consistency of a pc matrix; acquired judgments can be utilized in order to determine any of these missing judgments. regarding the judgment acquisition process, pcs can be classified into two groups: direct pairwise comparisons (direct pcs) and indirect pairwise comparisons (indirect pcs). direct pcs are directly given by decision maker(s), while indirect pcs are derived through direct pcs. indirect pcs enable the decision maker to not evaluate some of the pairwise comparisons (harker, 1987a, 1987b). considering the successive judgment capability of a human being, it is difficult to obtain consistent pc matrices when a large set of comparisons is present. the use of indirect pcs assists decision makers by allowing less comparison among pairs of pc matrices with higher dimensions. however, indirect pcs may cause false negatives and false positives during consistency analysis. since indirect pcs are calculated based on at least two distinct direct pcs, the mathematical consistency of pc matrix increases. the pc method is generally used with the analytical hierarchy process technique which is a systematized method for handling multiple criteria in a hierarchy structure. the ahp takes advantage of using the pc method while transforming subjective judgments into analytical information. the pc method preserves the inherent characteristics of judgments; the conclusion of analytical hierarchy process technique is highly dependent on initial judgments and their consistency levels. therefore, completeness and consistency of initial judgments have a fundamental role in the formation of the final decision. this study proposes a two-dimensional consistency analysis for evaluating the consistency of an incomplete pc matrix. ijahp article: koyun yilmaz, ozkir /extended consistency analysis for pairwise comparison method international journal of the analytic hierarchy process 119 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.506 2. pairwise comparison method thurstone (1994) formulated the law of comparative judgment, defined a psychological scale and introduced the pc method in 1927. the method has been adapted for assisting several decision-making methods to assess the relative importance of criteria and alternatives (siraj, mikhailov, & keane, 2015). despite the fact that studies on pcs are increasing rapidly, the pc literature is still intact. dede, kamalakis et al (2016) present a methodology to extend the pairwise comparison framework in order to provide some information on the credibility of its outcome. elliot (2010) investigates how the final decision is effected when different numerical scales are utilized in pc. the use of different numerical scales in the pc method yields significant effects on the attribute weights that are calculated from the judgments supplied and potentially on the preferred decisions that these judgments imply (elliott, 2010). let us consider n criteria ci, (i = 1,2,3,…,n). a pc matrix is a = [aij]n×n where aij reflects the relative importance of criterion i over criterion j. a is an n th order square matrix including aii = 1 for all self-comparisons and aji = 1/ aij for all reciprocal pcs. the acquired set of pcs constructs the pc matrix. before the extraction of the weights from a pc matrix, it should be confirmed by consistency analysis. further explanation of the consistency analysis is given in section 3.1. 𝐶𝑜𝑚𝑝𝑙𝑒𝑡𝑒 𝑃𝐶𝑀 = 𝑏1 𝑏2 𝑏3 𝑏4 𝑏1 𝑏2 𝑏3 𝑏4 [ 1 1 2⁄ 1 4⁄ 2 1 1 2⁄ 4 8 2 4 1 2 1 8⁄ 1 4⁄ 1 2⁄ 1 ] (1) figure 1 graph of complete pairwise comparison matrix (eq. 1) a pc matrix can be depicted as a graph (figure 1). in such a graph, vertices represent the criteria and edges represent the comparative judgments between criteria. any pairwise comparison can be computed straightforwardly if the graph of the incomplete pc matrix satisfies the minimum requirement of being a spanning tree. figure 1 illustrates the graph representation of the pc matrix given in equation 1. here bi (i={1,2,3,4}) are arbitrary criteria and aij (i,j = {1,2,3,4}) are the pairwise comparison of criterion bi over bj. figure 1a is the representation of the upper triangle of the pc matrix while figure 1b is a representation of the lower triangle. the reciprocity property allows us to represent the pc matrix as two distinct graphs whether it is complete or not. ijahp article: koyun yilmaz, ozkir /extended consistency analysis for pairwise comparison method international journal of the analytic hierarchy process 120 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.506 considering existing judgments constitutes at least a sub-tree of the pc matrix then missing judgments can be calculated as illustrated in figure 2. let m be the number of missing judgments in the lower or upper triangle of a n th order pc matrix. if existing judgments constitute at least a spanning tree of the pc matrix then (n-2) number of missing judgments can be calculated through a single node as in equation 2 while the remaining can be calculated through multiple nodes. figure 2 illustrates the graph representation of calculating a missing judgment through a single node. therefore, an incomplete pc matrix with the order of n should at least be a connected graph containing no loop with (𝑛 − 1) direct pcs. this condition verifies that at least two direct pcs are required in order to determine an associated indirect pc. eq. 1 represents the calculation of an indirect pc by using two direct pcs, which are the associated two edges in the path. if these direct pcs are consistent with each other, the path, used for determining the value of indirect pc, does not yield any variation. if not, the value of indirect pc can be assigned to the geometric mean of corresponding links. 𝑎𝑖𝑗 = 𝑤𝑖 𝑤𝑗 , 𝑎𝑗𝑘 = 𝑤𝑗 𝑤𝑘 → 𝑎𝑖𝑘 = 𝑎𝑖𝑗 × 𝑎𝑗𝑘 (2) figure 2 graph representation of calculation process for a14 and a41 3. consistency analysis in real life decision problems, pc matrices are rarely consistent. nevertheless, decision makers are interested in the level of consistency of the judgments, which somehow expresses the goodness or harmony of pairwise comparisons totally, because inconsistent judgments may lead to senseless decisions (bozóki & rapcsák, 2008). in the literature, several approaches can be found for evaluating the consistency of pc matrices such as consistency ratio using an eigenvector method, the least squares method, the χ squares method, the singular value decomposition method, koczkodaj's inconsistency index, the logarithmic least squares method and geometric consistency index (saaty, 1980; chu, kalaba, & spingarn, 1979; jensen, 1983; gass & rapcsák, 2004; w.w. koczkodaj, 1993; waldemar w. koczkodaj, herman, & orłowski, 1997; crawford & williams, 1985; aguarón & moreno-jiménez, 2003; bozóki & rapcsák, 2008). rahmami et al. (2009) propose a method that will improve consistency with less cost and better results than other methods in practice. a generalization of the purcell ijahp article: koyun yilmaz, ozkir /extended consistency analysis for pairwise comparison method international journal of the analytic hierarchy process 121 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.506 method is used to solve a system of homogeneous linear equations. dadkhah et al. (1993) assume inconsistency is a measurement error, and propose a revision method similar to saaty’s. the proposed method is also based on locating the elements that will contribute the most to increasing consistency. the difference from saaty’s revision method is that the elements which will decrease the maximum eigenvalue are investigated; in order to do so, the first derivation of λa of aij is used. the element with the largest absolute value for δλa δaij is revised; if it is negative the decision maker is instructed to increase aij, if it is positive the decision maker is instructed to decrease aij. benitez et al. (2011) use a linearization technique that provides the closest consistent matrix to a given inconsistent matrix using orthogonal projection in a linear space. in order to measure the closeness of two given matrices, the frobenius norm is utilized. the proposed method improves consistency in a direct and straightforward way differently from iterative methods. gomez et al. (2009) propose a multi-layer perceptron-based model to improve the consistency of a given pcm. the given model is capable of computing missing values while also improving consistency in this study, to investigate the consistency of incomplete pc matrices, we analyzed saaty's (1980) consistency analysis which defines the inconsistency ratio as an index for the deviation from randomness. 3.1 consistency of complete pc matrices in ahp consistency analysis presents a systematic approach to assess the consistency of judgments in a pc matrix. a pairwise comparison matrix a is consistent if it satisfies the transitivity property: 𝑎𝑖𝑗 × 𝑎𝑗𝑘 = 𝑎𝑖𝑘 𝑖, 𝑗, 𝑘 = 1,2, … , 𝑛 it was shown by saaty (1980) that a pairwise comparison matrix is consistent if and only if it is of rank one. when a pairwise comparison matrix a is consistent, the normalized weights computed from a are unique. otherwise, an approximation of a by a consistent matrix (determined by a vector) is needed (bozóki & rapcsák, 2008). a pc matrix is said to be perfectly consistent if its maximum eigenvalue is equal to its order n. on the contrary, the deviation of λmax from n is utilized as a measure of inconsistency. when a pc matrix deviates from perfect consistency with an acceptable degree, it is said to be consistent. if the deviation exceeds the acceptable degree, then it is an inconsistent pc matrix. if there are small variations of aij and the transitivity condition holds, then the maximum eigenvalue of a become close to n. this results in a ci value close to 0, and the matrix is accepted as consistent (harker, 1987a, 1987b). there are some reasons for inconsistency, namely, a decision maker’s mistakes while using the 9-point scale, the scale is unsuitable for the specific criteria, transitivity issues, psychological dependence, etc (kwiesielewicz & van uden, 2004). when dealing with an increasing number of pcs the possibility of consistency error also increases (franek & kresta, 2014). saaty implied that psychological dependence between comparisons causes higher inconsistency; hence, higher levels of consistency can mostly be achieved for a smaller number of objects (ishizaka & lusti, n.d.; saaty, 1980; wedley, schoner, & ijahp article: koyun yilmaz, ozkir /extended consistency analysis for pairwise comparison method international journal of the analytic hierarchy process 122 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.506 tang, 1993). since decreasing the number of criteria (or alternatives) to a relatively small number is not always possible, the use of incomplete pairwise matrices can help eliminate the side effects of higher dimensions. additionally, with lack of a common unit of measurement among the elements of a, deviation from the exact pc is inevitable. subjective judgments are a kind of estimation of reality and are expected to converge to the exact pc. small variations of aij keep the maximum eigenvalue close to n, and remaining eigenvalues close to zero (saaty, 1980). 3.2 consistency of incomplete pc matrices in ahp missing information is encountered due to various reasons. the criteria pair may not be suitable for pairwise comparison, the decision maker may not be eligible for assessing this pair, or the decision maker may not have a reasonable time or adequate motivation for evaluating all pcs. in such cases, decreasing the number of pcs assists the decision maker by providing flexibility and preserving motivation (harker, 1987a, 1987b). any pc matrix is investigated in terms of consistency and reliability before it is integrated into the decision-making process. a direct pc is a representation of human judgment, which is collected directly from the decision maker(s). therefore, the number of direct pcs in a pc matrix affects the consistency of the matrix. as the number of direct pcs in a pc matrix increases, the consistency index also generally increases. let us consider a two-level analytical hierarchy model with n number of criteria and n number of alternatives and suppose that each criterion has n number of sub-criteria. eq. 3 represents the total number of pcs, which increases rapidly as n increases n(n2 + n + 1)(n − 1) 2 (3) figure 3 total number of pcs vs the size of pcms with the increase in the number of direct pc judgments, the time required to complete the evaluation process also increases rapidly. this leads to negative effects due to ijahp article: koyun yilmaz, ozkir /extended consistency analysis for pairwise comparison method international journal of the analytic hierarchy process 123 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.506 psychological dependence, and therefore higher ci values. regarding saaty's 9-point scale, we generated random pc matrices to examine the relationship between ci values and the number of direct pcs. the consistency of an incomplete pc matrix relies on the consistency of the existing judgements. if the judgments of the decision maker do not create loops in the graph i.e. do not violate transitivity of judgments and are mathematically consistent, consistency of the pc matrix does not deteriorate. in section 3.3 the influence of indirect judgments on the consistency of a pc matrix is further investigated. 3.3 an experimental research for understanding consistency behavior this section presents a numerical study to illustrate that the presence of indirect pc(s) affects the consistency of a pc matrix. the following example also implies that a change in the number of indirect pcs causes a deviation in the consistency of the pc matrix. let r be the randomly generated pc matrix being investigated. in order to generate the pc matrix r, the following steps are applied, correspondingly. the right upper half of the matrix is filled by random evaluations using saaty's 9-point scale, the main diagonal of r is filled with only values of 1, and finally, the lower left half of the matrix can thus be completed with the corresponding reciprocals. the generated complete pc matrix r and corresponding consistency measures are given in eq. 4. 𝐑 = [ 1 1/4 1 4 1 7 1 1/7 1 6 9 1 1/4 1 7 3 1 1 1/6 4 1/3 1/9 1 1 1 1/7 1 1 1 3 1 1 1/5 1/3 5 1 ] 𝜆_ max = 9.5499 𝐂𝐈 = 0.7099 𝐂𝐑 = 0.5726 (4) suppose r1 is an incomplete pc matrix, which is derived from r by removing any 4 judgments. the pc matrix r1, calculated indirect pcs (a12, a16, a25, a35) and corresponding consistency measures are given in eq 5. 𝐑𝟏 = [ 1 − 1 − 1 7 1 1/7 1 6 9 − 1/4 − 7 3 1 1 1/6 4 1/3 1/9 − 1 − 1/7 1 1 − 3 − 1 1/5 1/3 5 1 ] 𝑎12 = 1 6 7 𝑎16 = 3 1 5 𝑎25 = 15 2 3 𝑎35 = 15 𝜆_ max = 8.6643 𝐂𝐈 = 0.5329 𝐂𝐑 = 0.4297 (5) now, suppose r2 is also an incomplete pc matrix, which is derived from r by removing any 6 judgments. the pc matrix r2, calculated indirect pcs (a13, a15, a23, a36, a45, a56) and corresponding consistency measures are given in eq 6. ijahp article: koyun yilmaz, ozkir /extended consistency analysis for pairwise comparison method international journal of the analytic hierarchy process 124 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.506 𝐑𝟐 = [ 1 1/4 − 4 1 − − − 1 6 − 1 1/4 1 7 3 1 − 1/6 4 1/3 − 1 1 1 1/7 − 1 − 3 − 1 1 1/3 − 1] 𝑎13 = 2 𝑎15 = 1 4 𝑎23 = 2 7 𝑎36 = 9 𝑎45 = 1 1 6 𝑎56 = 7 𝜆_ max = 8.5358 𝐂𝐈 = 0.5072 𝐂𝐑 = 0.4090 (6) the maximum eigenvalue of pc matrices r, r1 and r2 are 9.5499, 8.6643 and 8.5358 respectively. consistency index (ci) and consistency ratio (cr) are dependent on the maximum eigenvalue of a pc matrix since ri is the expected value of a consistency index and is constant. figure 4 illustrates the decrease in consistency indicators, i.e the pc matrix becomes more consistent while the number of indirect judgments increases. as a result, the consistency of a pc matrix does not deteriorate with the number of indirect judgments instead it becomes more consistent. figure 4 changes in consistency indicators while the number of indirect judgments increases this example straightforwardly indicates that any increase in the number of indirect pcs corresponds to higher consistency in judgments. the differences between the original judgments and the calculated indirect pcs for removed elements can be considered as the divergence of the decision maker's judgments from mathematical consistency. an increase in the number of indirect pcs corresponds to a decrease in the number of direct pcs; r2 is arithmetically more consistent than others are. for investigating the effects of direct pcs on consistency in detail, we generated 10,000 samples of random pc matrices in a similar manner. let m denote the number of direct pcs. figure 5a shows the mean and standard deviation of ci for an increasing number of direct pcs, and figure 5b shows probability density functions of ci for three different m values. probability density functions for different m values show that the random index (ri) e.g the expected value of the consistency index of random pc matrices is dependent on the number of indirect judgments. in other words, the probability distribution changes with the number of indirect judgments. ijahp article: koyun yilmaz, ozkir /extended consistency analysis for pairwise comparison method international journal of the analytic hierarchy process 125 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.506 simulated results imply that the mean of the ci is robust, but the variance is clearly unstable to the changes in the number of direct pcs. (a) mean and standard deviation (b) probability density function figure 5 ci behaviour while the number of direct pcs changes 4. material and methods the main purpose of this section is to analyze random index (ri) values in case of missing information. ris are the consistency indices of a randomly generated pc matrix from scale 1 to 9, with reciprocals forced (saaty, 1980). let a be an incomplete pc matrix with the order of n and m denotes the number of direct pairwise comparisons. in order to attain missing elements in a pc matrix, the existing direct pcs must construct a spanning tree. eq.7 represents the interval for m. [(𝑛 − 1), 𝑛(𝑛 − 1) 2 ) = {𝑚 ∈ ℤ+|(𝑛 − 1) ≤ 𝑚 > 𝑛(𝑛 − 1) 2 } (7) similar to the instance given in section 3.3, complete and incomplete pc matrices are generated randomly and the corresponding consistency levels are investigated. for pc matrices with the order of 4 through 11, we investigate the mean and the standard deviation of ris for each level of m using the sample size of 10,000 simulated data. figure 6 shows the types and the ranges of simulated pc data. ijahp article: koyun yilmaz, ozkir /extended consistency analysis for pairwise comparison method international journal of the analytic hierarchy process 126 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.506 figure 6 simulated dataset the pseudo code for construction of random pc matrices and calculation of corresponding ris is given as in figure 7. ijahp article: koyun yilmaz, ozkir /extended consistency analysis for pairwise comparison method international journal of the analytic hierarchy process 127 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.506 figure 7 pseudo code for random index generation 4.1 computational experiments and results for each order of n, we generated a set of 10,000 random pc matrices for each m number of direct pcs. next, calculated ci values are accumulated for each set and corresponding ris are examined. the aim of our simulation is to analyze the empirical distributions of the maximum eigenvalues λmax of randomly generated pc matrices for varying direct pc levels. the elements aij were chosen randomly from the scale: [ 1 9 , 1 8 , 1 7 , … , 1,2, … ,7,8,9] and aji is defined as 1/aij. by investigating probability density functions of ci values, we find that the number of direct pcs plays a major role in the consistency analysis. the results of the computational analysis can be summarized as follows: 1. while the number of direct pcs increases, the expected value of random index also increases and the variance decreases. 2. while the number of direct pcs increases, the consistency index (ci) of a random pc matrix converges to the expected value of the random index. 3. while the number of direct pcs increases, the probability density function of random index converges to a normal distribution. let cin m and rin m denote the corresponding cis and ris of an n th order pc matrix with m number of direct pcs, respectively. for distinctive threshold levels, table 1 shows the number of consistent pc matrices comparatively for m-specific and non-specific ri values. ijahp article: koyun yilmaz, ozkir /extended consistency analysis for pairwise comparison method international journal of the analytic hierarchy process 128 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.506 table 1 the number of pc matrices below different levels of cr threshold for any cr threshold, the total number of acceptable matrices decreases as the number of direct pcs increases, as in table 1. when saaty's (1980) proposal cr≤10% is employed to evaluate whether the judgments are deemed to have the acceptable consistency or not, the decrease in the total number of acceptable pc matrices is more drastic comparatively to other thresholds. when non-specific ri values are utilized in the accept/reject decision process, only three of 10,000 complete pc matrices (n=15) can be accepted as consistent, while 4,353 of 10,000 incomplete pc matrices with six direct pcs can be accepted as consistent. if m-specific ri value is utilized for assessing the consistency of 10,000 incomplete pc matrices with six direct pcs, only 2,844 of them will be accepted. as mentioned earlier, the number of indirect pcs can generate false positives and false negatives in the decision making process. for any incomplete n th order pc matrix, false positives are mostly encountered in the matrices that have a lower number of direct pcs, and the false negatives are mostly encountered in the matrices that have a higher number of direct pcs. here, table 1 illustrates the case for sixth order pc matrices. the difference between 4,353 and 2,844 implies that false positives exist in the consistency analysis. about 1,500 pc matrices could be inadvertently accepted during the consistency evaluation process. conversely, for the case with 11 direct pcs and 0.5 threshold level, the acceptable number of pc matrices increases to 2,118 when m-specific ri value is utilized, whereas it is 2,050 for nonspecific ri computation. this instance illustrates the case for false negatives. depending on the sensitivity of the problem, the decision maker can determine particular threshold level(s) regarding the number of direct pcs. for example, a lower threshold level and a higher number of pc judgments can be required for the problems with high sensitivity and vice versa. ijahp article: koyun yilmaz, ozkir /extended consistency analysis for pairwise comparison method international journal of the analytic hierarchy process 129 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.506 5. experimental study and results we conducted an experimental study to investigate the effects of incomplete pc matrix on consistency. the survey experiment was carried out in cooperation with 104 participants. each participant was given six differently sized zeros shown in figure 8 and asked to judge their sizes relatively. participants are required to make 10 randomly chosen pcs and observed the durations during the judgment process. average completion time was about 12 minutes, and participants spent about one minute for each evaluation. additionally, we noticed that the average duration of a previous judgment is generally lower than that of the subsequent one. as we have not previously set up a hypothesis on this resulting observation, we have not logged a separate data for each of the evaluation processes. however, this observation can be tested by a more extensive experiment in the future. after the data was collected from participants and transferred to a data file, we removed four randomly chosen pcs from each matrix in order to construct new incomplete pc matrices. therefore, this manipulation provided us two different sets of incomplete pc matrices for each participant. we have analyzed their consistency with non-specific and m-specific ri values (table 1). figure 8 survey example the consistency measures, ci and cr, were investigated for two sets of incomplete pc matrices. we depict the frequency distribution of consistency measures, which are uniformly grouped by 10% percentiles in figure 9. in figure 9a, the frequencies of cr values change for two ri values. assuming that the duration needed for any pc judgment is uniformly distributed, the time needed for 15 pcs is nearly 2.5 times greater than the time needed for six pcs. for the first percentile, only six of 102 incomplete pc matrices fall under 0.10 when a non-specific ri value is used for analysis. in figure 9b, the frequency distributions of cr values are identical for m-specific and nonspecific ris. however, the time needed for 10 direct pcs is expected to be shorter than that of 15 direct pcs. ijahp article: koyun yilmaz, ozkir /extended consistency analysis for pairwise comparison method international journal of the analytic hierarchy process 130 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.506 (a) n=6, m=6 (b) n=6, m=10 figure 9 cr intervals for ri6 6 , ri6 10 and ri6 15 consequently, the required time for the acquisition process of pc judgment can be reduced by the use of m-specific ris without incurring a loss of information. 6. conclusions and discussions in decision-making problems, experts generally express the information, evaluations, preferences and weights linguistically. since the data is inherently non-numeric, decisionmaking methods require a pre-procedure to transform the non-numerical data into numeric values. the pc method is one of the popular methods for transforming subjective judgments into analytical information. it plays a significant role in multiple criteria decision-making methods, especially with the analytic hierarchy process and analytic network process methods. the pc method is mathematically capable of dealing with large sets of criteria and helps experts focus on only two elements one at a time. however, successive judgment capability of a human being is limited. hence, it can be puzzling for experts to provide consistent pairwise judgments when a large set of comparisons is available. in these cases, the judgment matrix fails in conformity to the transitivity requirement and exhibits some inconsistency. the use of incomplete pc matrices assists decision makers by enabling a substantial decrease in the number of pcs when a higher number of criteria are available. however, the use of indirect pcs can cause false positives or false negatives throughout consistency analysis. namely, for an incomplete pc matrix, the use of non-specific (original) ri with a consistency test can indicate the presence of consistency, when it is not consistent in reality. the obtained weights from a pc matrix are highly dependent on initial judgments and its level of consistency and completeness has a fundamental role in the formation of the final decision. in this paper, a two-dimensional consistency analysis approach was presented for evaluating the consistency of incomplete pc matrices. the computational experiment reveals that the number of direct pcs is the significant factor while evaluating ijahp article: koyun yilmaz, ozkir /extended consistency analysis for pairwise comparison method international journal of the analytic hierarchy process 131 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.506 consistency. moreover, indirect judgments do not deteriorate the consistency ratio given that the judgments of the decision maker are mathematically consistent and transitive. our study demonstrates that if there are missing judgments in a pc matrix the probability distribution is different from the case of a complete pc matrix. as a result, the expected value of the consistency index (ri) is changed. in order to avoid false-positive results for a consistency ratio, an incomplete pc matrices consistency ratio should be calculated with the respective probability distribution. for future study, the approach presented could be extended to fuzzy approaches and group decision making. a more extensive experiment could be conducted for examining the required time for successive evaluations in the future. furthermore, the relationship between the sensitivity of decision problem and the number of direct pcs can be investigated. ijahp article: koyun yilmaz, ozkir /extended consistency analysis for pairwise comparison method international journal of the analytic hierarchy process 132 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.506 appendix ijahp article: koyun yilmaz, ozkir /extended consistency analysis for pairwise comparison method international journal of the analytic hierarchy process 133 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.506 references aguarón, j., & moreno-jiménez, j. m. m. (2003). the geometric consistency index: approximated thresholds. european journal of operational research, 147(1), 137–145. doi: https://doi.org/10.1016/s0377-2217(02)00255-2 benítez, j., delgado-galván, x., izquierdo, j., & pérez-garcía, r. (2011). achieving matrix consistency in ahp through linearization. applied mathematical modelling, 35(9), 4449–4457. doi: https://doi.org/10.1016/j.apm.2011.03.013 bozóki, s., & rapcsák, t. (2008). on saaty’s and koczkodaj’s inconsistencies of pairwise comparison matrices. journal of global optimization, 42(2), 157–175. doi: https://doi.org/10.1007/s10898-007-9236-z chu, a. t. w., kalaba, r. e., & spingarn, k. (1979). a comparison of two methods for determining the weights of belonging to fuzzy sets. journal of optimization theory and applications, 27(4), 531–538. doi: https://doi.org/10.1007/bf00933438 crawford, g., & williams, c. (1985). a note on the analysis of subjective judgment matrices. journal of mathematical psychology, 29(4), 387–405. doi: https://doi.org/10.1016/0022-2496(85)90002-1 dadkhah, k. m., & zahedi, f. (1993). a mathematical treatment of inconsistency in the analytic hierarchy process. mathematical and computer modelling, 17(4–5), 111–122. doi: https://doi.org/10.1016/0895-7177(93)90180-7 dede, g., kamalakis, t., & sphicopoulos, t. (2016). theoretical estimation of the probability of weight rank reversal in pairwise comparisons. european journal of operational research, 252(2), 587–600. doi: https://doi.org/10.1016/j.ejor.2016.01.059 elliott, m. a. (2010). selecting numerical scales for pairwise comparisons. reliability engineering & system safety, 95(7), 750–763. doi: https://doi.org/10.1016/j.ress.2010.02.013 franek, j., & kresta, a. (2014). judgment scales and consistency measure in ahp. procedia economics and finance, 12, 164–173. doi: https://doi.org/10.1016/s22125671(14)00332-3 gass, s. i., & rapcsák, t. (2004). singular value decomposition in ahp. european journal of operational research, 154(3), 573–584. doi: https://doi.org/10.1016/s03772217(02)00755-5 gomez-ruiz, j. a., karanik, m., & peláez, j. i. (2009). improving the consistency of ahp matrices using a multi-layer perceptron-based mode. lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics), 5517 lncs(part 1), 41–48. doi: https://doi.org/10.1007/978-3-64202478-8_6 harker, p. t. (1987a). alternative modes of questioning in the analytic hierarchy process. ijahp article: koyun yilmaz, ozkir /extended consistency analysis for pairwise comparison method international journal of the analytic hierarchy process 134 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.506 mathematical modelling, 9(3–5), 353–360. doi: https://doi.org/10.1016/02700255(87)90492-1 harker, p. t. (1987b). incomplete pairwise comparisons in the analytic hierarchy process. mathematical modelling, 9(11), 837–848. doi: https://doi.org/10.1016/02700255(87)90503-3 ishizaka, a., & lusti, m. (n.d.). an expert module to improve the consistency of ahp matrices. retrieved from http://www.ishizaka-dev.myweb.port.ac.uk/itor.pdf jensen, r. (1983). comparison of eigenvector, least squares, chi square and logarithmic least square methods of scaling a reciprocal matrix. retrieved from https://scholar.googleusercontent.com/scholar.bib?q=info:bb3lrd5maokj:scholar.googl e.com/&output=citation&scisig=aagbfm0aaaaawxdvkq5u779tdq1abrgupb7cu ozwcf61&scisf=4&ct=citation&cd=-1&hl=tr&scfhb=1 koczkodaj, w. w. (1993). a new definition of consistency of pairwise comparisons. mathematical and computer modelling, 18(7), 79–84. doi: https://doi.org/10.1016/08957177(93)90059-8 koczkodaj, w. w., herman, m. w., & orłowski, m. (1997). using consistency-driven pairwise comparisons in knowledge-based systems. in proceedings of the sixth international conference on information and knowledge management cikm ’97 (pp. 91–96). new york, new york, usa: acm press. doi: https://doi.org/10.1145/266714.266867 kwiesielewicz, m., & van uden, e. (2004). inconsistent and contradictory judgements in pairwise comparison method in the ahp. computers & operations research, 31(5), 713–719. doi: https://doi.org/10.1016/s0305-0548(03)00022-4 rahmami, m., & navidi, h. (2009). a new approach to improve inconsistency in the analytical hierarchy process. applications and applied mathematics: an international journal, 4(1), 40–51. saaty, t. l. (1980). the analytic hierarchy process. new york: mcgraw-hill. siraj, s., mikhailov, l., & keane, j. a. (2015). contribution of individual judgments toward inconsistency in pairwise comparisons. european journal of operational research, 242(2), 557–567. doi: https://doi.org/10.1016/j.ejor.2014.10.024 thurstone, l. l. (1994). a law of comparative judgment. psychological review, 101(2), 266–270. retrieved from https://brocku.ca/meadproject/thurstone/thurstone_1927f.html wedley, w. c., schoner, b., & tang, t. s. (1993). starting rules for incomplete comparisons in the analytic hierarchy process. mathematical and computer modelling, 17(4–5), 93–100. doi: https://doi.org/10.1016/0895-7177(93)90178-2 ijahp article: ferretti, saaty/ questions and answers: valentina ferretti interviews tom saaty international journal of the analytic hierarchy process issn 1936-6744 132 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.235 questions and answers: valentina ferretti interviews tom saaty the following interview aims to stimulate the discussion about the key characteristics of multicriteria analysis techniques. in particular, the interview will work through the conceptual lens of decision processes taking place in the field of urban and territorial transformations. valentina ferretti: multicriteria analysis was introduced during the 1960s in the field of operation research. more recently it went through an independent evolution, and it has now become a point of reference for many disciplines which are concerned with the analysis of decision processes. which elements of multicriteria analysis make it possible for many disciplines to find consensus in its use? what is the relationship between multicriteria analysis and operation research? has this relationship changed during the years? tom saaty: let me answer this question by recalling my personal experience. in 1959 i wrote the first book about the mathematics used in operation research 1 . about 10 years later, i was consulting for the pentagon in washington and the presence of trade-offs among the criteria in the project naturally led me to multicriteria analysis. since operation research deals with optimization of one criterion, it is thus based on single criterion decision analysis. since the 1970s, multiple criteria decision making (mcdm) research has developed rapidly, and has become a hot research topic because many complex practical decision problems involve multiple and conflicting criteria as well as multiple objectives. therefore, over the past few decades a number of mcdm methods have been developed to deal with the measurement of tangible/intangible conflicting criteria and with the measurement of the alternatives of a decision with respect to these criteria. mcdm methods now have their own societies and their own journals and represent a consolidated field of research. both the ahp and anp, the fields of multi-criteria decision making that i developed, are descriptive approaches to decision-making. they evolved out of my experience at the arms control and disarmament agency (acda) in the united states department of state during the kennedy and johnson years. the reason why i was invited to join acda was probably because of the work i had done for the military using the mathematics of operations research. a key aspect for the increasing the use of mcdm methods is that decision making is a central activity of all people; it is usually done so automatically that we do not even realize that we are doing it every moment of every day of every year for all our lives. this silent and inarticulate approach worked well for us when humanity was fragmented and individuals and groups of people could act on their thoughts without having to think of others very much. 1 saaty t. 1959. mathematical methods of operation research (dover phoenix edition). new york: dover publications. ijahp article: ferretti, saaty/ questions and answers: valentina ferretti interviews tom saaty international journal of the analytic hierarchy process issn 1936-6744 133 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.235 today, the world has become extended and interdependent and many of its resources are becoming scarce and valuable, including water, air, land, grains and vegetables and minerals. so now we have to work conscientiously together to choose our important courses of action. we have to justify these actions not only to ourselves but also to others, so we can live in harmony and with minimum conflict (saaty and sagir, 2012) 2 . we need to think and decide in bigger ways than ever before and in very general ways. decision making today depends much on intuition, but it needs to be transformed into a dependable science. nearly all of us have been brought up to believe that clear-headed logical thinking is our only sure way to face and solve problems. but experience suggests that logical thinking is not natural to us. indeed, we have to practice, and for a long time, before we can do it well. since complex problems usually have many related factors, traditional logical thinking leads to sequences of ideas so tangled that often the best solution cannot be easily discerned. since we are concerned with real-life problems, we must recognize the necessity for tradeoffs to best serve the common interest. to be really useful, this process should also assist in building consensus and reaching compromises. we need to make decisions that are both desirable and survivable, rather than simply ones that we like best, without regard to how effective or lasting they may be. predicting outcomes plays an important role in making such choices. i would suggest the following elements contribute to the increased use of mcdm methods: it is simple to construct; it is adaptable to both groups and individuals; it is natural to our intuition and general thinking; it encourages compromise and consensus building; it does not require inordinate specialization to master and communicate; the details of the processes leading up to the decision making outcome are easy to review. valentina ferretti: evaluation criteria and elicitation of weights are issues that have been the focus of a broad debate within the scientific international literature, and numerous approaches have been proposed. in your opinion, what are the most critical aspects and drawbacks with regards to these issues? does a trend line exist for research in this context? tom saaty: the first, and fundamental, step needed to develop a multicriteria model is structuring the mcdm model. . to structure is to design and put together smaller components into larger ones and these again into still larger ones. structure involves three ideas: wholeness, transformation and selfregulation. structures give rise to the idea of formalization that concerns flows in the structure to fulfill certain functions designed to meet certain objectives of varying priorities. while the structure exists regardless of the 2 saaty t, sagir m. 2012. global awareness, future city design and decision making. systems engineering society of china & springer-verlag berlin heidelberg. 21(3): 337-355 ijahp article: ferretti, saaty/ questions and answers: valentina ferretti interviews tom saaty international journal of the analytic hierarchy process issn 1936-6744 134 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.235 flows within it, the flows themselves are dependent on the particular structure and need mathematics to describe them. their description depends on the choice of the theoreticians involved (saaty and begicevic, 2010) 3 . it is a highly demanding process of creative thinking and understanding to structure our problems and piece together the different ideas about what must be done, and is at least as important as (and can be far more complex than) performing the operations within a structure. we tend to think of structures as physical objects around us, but most structures are built in our minds for our own convenience to understand occurrences around us that are both physical and mental. fundamentally structures are no less mathematical in nature than are operations and transformations that take place within them, but as yet we are not as sophisticated in dealing with them (saaty and shih, 2009) 4 . in the field of decision making, creating a structure is the first step in organizing, representing and solving a problem. a structure is a model, an abstraction of a problem. it helps us visualize and understand the relevant elements within it that we know from the real world, and then use our understanding to solve the problem represented in the structure with greater confidence. determining the factors that are the criteria and the way they are put together into a structure that influence the decision is a very demanding task. when we have a problem, and feelings and ideas that need to be expressed in a certain way, we often have difficulty with how to put them all together and connect them in an appropriate way to represent the causes and effects of the problem. to formulate an initially unstructured decision problem, we attack it by assembling its elements into sub-hierarchies, each sub-hierarchy dealing with a part of the problem. then we arrange them into a single overall hierarchy in decreasing order of influence. a hierarchy is a structure for organizing influences from sources to sinks, so that each element in a level of the hierarchy, except for the single top element known as the goal of the decision, is subordinate or is a sub-criterion of an element immediately above. a major purpose of structuring hierarchies in decision making is to make it possible to compare the importance of the elements (criteria and alternatives) in a given level with respect to the elements in the level above, and to derive priorities from the judgments expressed numerically. a hierarchy is a special case of a network with connections that go only in one direction. a network has clusters of elements, with the elements in one cluster being connected to elements in another cluster (outer dependence) or the same cluster (inner dependence). hierarchies and networks occur abundantly in personal life, in businesses and corporations, and in government strategy, public policy, the health care 3 saaty t. and begicevic n. 2010. the scope of human values and human activities in decision making. applied soft computing. 10(4): 963-974. 4 saaty t. and shih h.s. 2009. structures in decision making: on the subjective geometry of hierarchies and networks. european journal of operational research. 199, 867-872. ijahp article: ferretti, saaty/ questions and answers: valentina ferretti interviews tom saaty international journal of the analytic hierarchy process issn 1936-6744 135 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.235 industry, military strategy, non-profit organization strategy, planning and so on (saaty and begicevic, 2010) 5 . there are two important requirements that a structure must fulfill: it must be logical in identifying and grouping similar things together. it must be complete, with nothing left out that has an important influence. no matter how a structure is validated, group participation with knowledgeable people is a good way to ensure its logicality and completeness. this type of thinking is a complex mental process involving cognition, pattern matching, associative memory and knowledge, judgment, comparisons, and imagination. to find a good structure and to identify a new alternative which is a possible way to act imagination, one of our most treasured attributes, can be extremely helpful to overcome our psychological inertia and expand the solution space so that some creative ideas can be generated. creative thinking and decision making thus work together very closely. to make a decision one needs creative thinking, at least to design a structure of the factors in the decision. and to think creatively one needs to make a variety of decisions to be able to proceed in depth and breadth, knowing what to include and how and where to include it. for greater details about such considerations the reader might consult my book about creative thinking and problem solving 6 . following this reasoning, establishing the structure may be considered more of an art than a science. once one has the structure, it becomes easier to convey to others the influences which drive that decision. all possible factors should be included in the structure. all sides to the discussion should be able to include the factors they feel appropriate. later, the process of prioritization by the different stakeholders will weed out the unimportant or irrelevant factors and can be given appropriate weightings and then combined with the positions of the majority. because decision-making is the most frequent activity of all people all the time, the techniques used today to help people make better decisions should probably remain closer to the biology and psychology of people than to the techniques conceived and circulated at a certain time. these are likely to become obsolete, as all knowledge does, even though decisions go on and on forever. this suggests that methods offered to help make better decisions should be closer to being descriptive and considerably transparent (saaty, 2005) 7 . trade-offs will always exist between economic development, in the material sense, and the welfare of society and environment. 5 saaty t. and begicevic n. 2010. the scope of human values and human activities in decision making. applied soft computing. 10(4), 963-974. 6 saaty t. 2001. creative thinking, problem solving and decision making. pittsburgh, pa: rws publications. 7 saaty t. 2005. the analytic hierarchy and analytic network processes for the measurement of intangible criteria and for decision-making. chapter 9, pp.345-407 in multiple criteria ijahp article: ferretti, saaty/ questions and answers: valentina ferretti interviews tom saaty international journal of the analytic hierarchy process issn 1936-6744 136 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.235 comparative or relative judgments made on pairs of elements to ensure accuracy are a great help in this direction. in paired comparisons, the smaller or lesser element is used as the unit, and the larger or greater element is estimated as a multiple of that unit with respect to the common property or criterion for which the comparisons are made. this approach involves breaking the problem down into finer and finer parts, so that one is called upon to give a judgment comparing only a pair of issues in each judgment. this avoids mixing too many aspects of the problem and not knowing what goes with what to obtain the final answer. valentina ferretti: a further element of discussion with reference to the structuring phase of a decision problem and to the definition of a coherent family of criteria, concerns the possibility of interaction among the criteria being identified. this issue is the subject of a broad debate from which very different points of view seem to emerge. could you share with us your vision with reference to this issue? tom saaty: interdependence is a very important subject in decision making. you should use methods that take it into account. if we think about it carefully, everything can be seen to influence everything else including itself, directly or indirectly, according to many criteria. the world is far more interdependent than we know. the concept of a network is useful in helping us to portray the complex relations of real-world problems. there are many examples of networks in transportation, computer science, neurology, operations research, flow problems, business, and marketing, and in human society. the anp which uses network structures is a logical way to deal with dependence. by freeing us from the burden of ordering the components in the form of a directed chain as in a hierarchy, the anp frees us from the burden of ordering the components in the form of a directed chain as in a hierarchy and rather represents any decision as a directed network. this allows the structure to develop more naturally and closer to what actually happens in the real world (saaty and vargas, 2006) 8 . valentina ferretti: since currently the trend focuses on participative design and planning processes, a crucial element for decision processes taking place in the context of territorial transformation assessment is the presence of multiple stakeholders with different values and conflicting objectives. what relationship exists between the need to include different stakeholders in the decision aiding process and the available mathematical tools? what are the most suitable modalities of participation to facilitate an effective interaction decision analysis: state of the art surveys, edited by j. figueira, s. greco, and m ehrgott, springer (2005). 8 saaty t. and vargas l.g. 2006. decision making with the analytic network process: economic, political, social and technological applications with benefits, opportunities, costs and risks. new york, new york: springer's international series. ijahp article: ferretti, saaty/ questions and answers: valentina ferretti interviews tom saaty international journal of the analytic hierarchy process issn 1936-6744 137 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.235 between the analyst and the stakeholders, and to generate collective learning effects? tom saaty: if the mcdm method satisfies the requirements that i suggested while answering the first question, then one of the most important applications of multicriteria analysis is to support participative processes and conflict resolution problems. to be applied to conflict resolution a method must provide a way for each conflicting party to evaluate the costs and risks of giving up some of what it has, in return for the benefits and opportunities for getting what it wants from the other party. let me recall in this context my professional experience with the israeli palestinian conflict 9 . this conflict is not a single isolated problem to be solved, but a system of people with conflicting aspirations. physically, the problem is geographic with two parties desiring the same piece of land, but its origins are deeply rooted in people’s history and beliefs and in their attachment to a land consecrated by their great religions. despite the best efforts of diplomats and world leaders over a period of more than sixty years, a satisfactory resolution has not emerged. what we suggested and used is a holistic model that explores feedbacks from various criteria and input from key constituents. over a 3-day period, the panel, comprised of three to five representatives from each side, structured the problem, defined the constituents and developed several potential alternatives. the process was not without conflict and negotiation of its own. at times, the panel differed on various definitions, on the structure of the model, and on the potential solutions. however, there was nearly always unanimous agreement on the nature of the conflict, with little debate within either side about the underlying concerns or where the power and influence belonged that could bring about termination of a 66-year old confrontation. similarly, there was practically no problem in identifying the key constituents. in the context of participative decision processes, the following aspects are of fundamental importance: detailed and advanced planning of resources (in terms of time, budget, places, number of actors, etc.); inclusion of a moderator to facilitate the process and ensure that all parties agree before moving on to the next step in the process; a knowledge that language and understanding matter; use of visualization tools (to ensure mutual understanding); interpretation of the result, not just as a set of numbers produced by the model, but as a roadmap and learning process for all the actors. valentina ferretti: another crucial issue for the development of a multicriteria analysis approach concerns the modeling and management of uncertainty. this issue is particularly undeniable for applications within the 9 saaty t. and zoffer h.j. 2013. principles for implementing a potential solution to the middle east conflict. notices of the ams, 60(10), 13301322. ijahp article: ferretti, saaty/ questions and answers: valentina ferretti interviews tom saaty international journal of the analytic hierarchy process issn 1936-6744 138 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.235 field of territorial transformation projects where both quantitative and qualitative data are characterized by high levels of uncertainty. within our discipline, we usually use sensitivity analysis more often than robustness analysis. could you share with us your vision with reference to the different aspects of this issue? tom saaty: i propose three approaches to deal with uncertainty which are not mutually exclusive. 1. in any decision one expects to consider favorable and unfavorable concerns. some concerns are sure things while others are less certain but have a likelihood of materializing. the sure concerns are called benefits and costs, while the uncertain concerns are called opportunities and risks. we refer to the four concerns collectively as bocr. for each of the four bocr merits we have a system of control criteria that we use to assess influence. the result is that such control criteria and/or their subcriteria serve as the basis for all comparisons made under them, both for the components and for the elements in these components. 2. sensitivity analysis is the way to test the stability of the outcome. this is a widely used method for quantitative model assessment in many disciplines in order to validate the feasibility, robustness, and reliability of a model or a method. sensitivity analysis refers to the study of how the uncertainty in the output of a mathematical model or system (numerical or otherwise) can be apportioned to different sources of uncertainty in its inputs. to practitioners, it is most important to understand the sensitivity of a model’s outputs to simultaneous variations in several parameters. sensitivity analysis is thus a way to predict the outcome of a decision if a situation turns out to be different compared to the key prediction(s). therefore, an mcdm method should be capable of assessing the stability and validity of outputs to imprecise values for the variant values of some of the model’s parameters. 3. when we create a structure to make a decision we assume that the decision maker needs to know all the important factors that go into that decision, but that may not always be true. in making a decision we learn that there can be factors inadvertently left out that could have led to a different outcome. there are many factors that influence outcomes in decision making, and these factors may straddle the spectrum of possibilities from very low to very high. we often impatiently assume that we can reduce the diversity of factors to only a few—what we at a given time consider to be the important ones. but in real life, there may be numerous not-so-important determinants of an outcome, and these low-priority determinants could collectively be very influential in shaping a decision. a serious weakness in decision making to date is the mixing and reduction of all factors into the few that one habitually assumes to be the important ones. we propose the use of the concept of “other” as a http://en.wikipedia.org/wiki/uncertainty http://en.wikipedia.org/wiki/mathematical_model http://en.wikipedia.org/wiki/uncertainty ijahp article: ferretti, saaty/ questions and answers: valentina ferretti interviews tom saaty international journal of the analytic hierarchy process issn 1936-6744 139 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.235 criterion to complete a set of criteria with an expert giving the judgments that would help give closure to the question of the uniqueness of ranking 10. valentina ferretti: it is interesting to think about the historical evolution of multicriteria analysis techniques. these methods were introduced in businessoriented contexts characterized by a data-driven approach, while recently they have started to attract increasing interest in the context of public decision processes, which are usually more value-driven. in your opinion, how is multicriteria analysis trying to adapt in order to cope with this growing need? tom saaty: multi-decision problems bring us head-on to the question of ordering our values, a subject in order topology and not in metric topology. values are enduring guides for establishing priorities for activities to enable us to best apply our mental and physical resources to obtain the greatest overall benefit. we use values to relate and interpret everything that we learn and experience. they are the focus around which our thinking revolves. value is an anchor that binds our energies, our thoughts, and our actions. in a sense, our values are us. they are not something abstract and eternal. many people believe that value theory, which is so fundamental to decision-making, is the most important area in philosophy. all religions and most philosophical movements have been concerned with value theory to some degree. in philosophy, value theory, or axiology, concerns itself with the notion of goodness. there are two categories of goodness: ethics, concerning the morally good, and aesthetics, concerning the artistically good, or the beautiful. value theory also concerns social goodness, and considerations that are of such great importance in economics and political science. value defines “good” and “bad” for a community or society. it affects everyone’s life—maybe all life forms and not just people (saaty, 2007) 11 . the interested reader is referred to k. baier and n. rescher for a comprehensive list of values 12 . valentina ferretti: traditionally, the assessment of public policies and projects has developed based on a mono-criterion approach (for instance, the cost benefit analysis). more recently, assessment procedures have started to make use of multicriteria analysis techniques that are able to take into account the multidimensionality of the systems under evaluation. what is, in your opinion, the relationship between these two approaches? does a complementarity exist? does a substitutability exist? is it a matter of merging one tool into the other? tom saaty: with reference to the relation between multicriteria analysis (mca) and cost benefit analysis (cba), i would like to say that both 10 saaty t. and ozdemir m. 2006. the unknown in decision making: what to do about it. european journal of operational research, 174, 349-359. 11 saaty t. 2007. multi-decisions decision-making: in addition to wheeling and dealing, our national political bodies need a formal approach for prioritization. mathematical and computer modelling, 46, 1001–1016. 12 k. baier, n. rescher (1969). values and the future. new york: the free press. ijahp article: ferretti, saaty/ questions and answers: valentina ferretti interviews tom saaty international journal of the analytic hierarchy process issn 1936-6744 140 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.235 approaches have pros and cons. the direction of research in this context should move towards possible integrated approaches that are capable of fostering synergies and stimulating innovation. with particular reference to territorial transformation evaluation, mcda presents some key advantages over cba: 1. environmental decision making is characterized by multiple conflicting objectives. in this context, mcdm methods support multi-stakeholder interactions in making trade-offs between different objectives explicit. mcdm thus supports negotiation and interactive learning processes while reducing the problem to a single monetary value as is done in cba which loses important information. 2. mcdm avoids the necessity of placing a monetary value on every aspect involved in the decision. as a result, it avoids a range of theoretical and practical shortcomings (i.e. mental account biases such as the elicitation of the willingness to pay from one environmental good in isolation from other goods and aggregation biases such as the choice of the appropriate population and time period as well as discount rate). valentina ferretti: within the field of environmental sustainability assessments, it is often necessary to take into account qualitative criteria that are not easily monetized. we can think, for instance, about the assessment of the quality of the landscape, the historical/architectural value of a public good, etc. usually, the evaluation of these aspects makes use of symbolic matrices that give rise to strong critiques due to the inherent subjectivity of the evaluation approach. what is, in your opinion, the best way to deal with intangible aspects? tom saaty: the word intangible is most commonly used to describe things that are recognized but not easily quantified or measured. the challenge is how to deal with the measurement of intangible factors that arise in order to make tradeoffs with the other tangible factors when both benefits and potential benefits, costs and potential costs are involved. the solution is to make pairwise judgments, using a 9 point fundamental scale 13 , which represents the normal range of human sensitivity to phenomena that are homogeneous. the numbers used in this scale are absolute and not ordinal numbers. to say that one thing is five times more important than another, which is what it means to use the number 5 from the fundamental scale, is a much stronger and more meaningful statement than to just assign an arbitrary number to it. a number from the fundamental scale can thus not be changed to another number and still convey the same sense. to deal with intangibles scientifically, we thus have to pairwise compare them to derive the relative priorities. making comparisons is our biological heritage. it was there long before measurement scales were invented with their 13 saaty t, sagir m. 2012. global awareness, future city design and decision making. systems engineering society of china & springer-verlag berlin heidelberg. 21(3), 337-355. ijahp article: ferretti, saaty/ questions and answers: valentina ferretti interviews tom saaty international journal of the analytic hierarchy process issn 1936-6744 141 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.235 arbitrary units, and gradually used in science to develop scientific theories about the physical universe. the intuitive idea behind the ahp 14 is exactly this. the fundamental scale ahp uses is comprised of absolute numbers derived from stimulus-response theory to quantify judgments in making reciprocal pairwise comparisons of elements in a matrix as to dominance with respect to a given property. valentina ferretti: the evaluation procedure that should accompany the design/planning process after the very first structuring and judging steps, in order to support defining objectives to be pursued and generating alternative options is another very important aspect in our discipline. how can such a procedure be structured, in your opinion, in order to cope with this need and be integrated in a virtuous design/planning cycle? tom saaty: in my opinion, there are four fundamental aspects to which we should pay particular attention when structuring the decision aiding process: 1. detailed and advanced planning of the whole process with reference to the people involved (who to invite? how many people?,) the resources that are needed (who is going to fund the process?), the allocation of time, the location, the means to support the activities, etc. 2. a balanced group of experts (not only technicians, but also people from society and politics). 3. promoting the engagement of the participants by always keeping them updated and periodically verifying that everybody has the same understanding of the problem under consideration. 4. ensuring justification (of consequences, of the procedure and of the approach) valentina ferretti: given the increasing interest in multicriteria analysis from different disciplines, which aspects should young researchers and practitioners pay more attention to in the near future? what are, in your opinion, the new frontiers of research in decision science? tom saaty: speaking about future directions for research i have two interesting suggestions for young researchers and practitioners. most of the factors that determine the structure of a decision depend largely on the feelings and memories of the decision makers, and that leaves room for doubt about the completeness of most decisions. decision makers would be helped by having a general well-researched list of factors available for some of their complex decisions in order to have greater assurance that their 14 saaty, t.l., 1990. how to make a decision: the analytic hierarchy process. european journal of operational research, 48 (1), 9–26. ijahp article: ferretti, saaty/ questions and answers: valentina ferretti interviews tom saaty international journal of the analytic hierarchy process issn 1936-6744 142 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.235 decisions are comprehensive and right. to do that, they need wide exposure and familiarity with the full spectrum of human values and environmental opportunities 15 . there have been two major attempts to document the structure of more than a thousand kinds of decisions in the ahp/anp classified according to their area of application. there are two dictionaries of hierarchies. the first, the hierarchon 16 and the new forthcoming hierarchon of 2014 are books of many examples of hierarchically structured decisions that descend from a goal through criteria, sub-criteria, actors, diverse kinds of influences wielded by the actors, groups affected, their objectives and the alternatives of the decision. the two-volume encyclicon 17 deals with decisions with dependence and feedback. but these are only a beginning because we need lists of attributes and areas of human enterprise so that governmental, regional, group and individual decision makers can have greater assurance about the completeness of their factors. although what we have here has been the subject of a great deal of research by numerous people over a period of 80 years, it is certain that it still needs further elaboration and debate to broaden its scope. there is thus still much more work to be done on the subject of structures in decision making and what should go into them. moreover, there remains the question of the classification of policies and alternatives, a concern that has not yet been dealt with systematically in the field of decision making. the second line of research that i foresee concerns the environment of the future, our future cities. over the next thirty years, two thirds of the world’s projected close-to-10-billion population is expected to live in urban locations. this mass migration to cities creates a significant challenge for city planners as they work to create a sustainable infrastructure to support the vast population growth, whilst being sensitive to the preservation of cultural heritage and historic landmarks as well as existing structures already shaping the development of the dense conurbation. ensuring that environmental awareness and protection, economic growth and social wellbeing also remain at the heart of a city’s urban strategy is also paramount to success. the future thus requires better planning and design of our world and a scientific way of making decisions that includes all of us in the judgment process and not simply the elite among us. why do we need to design our future cities in harmony with the changes that occur and the necessary sustainability to meet the requirements of a better future? what responsibility do we have to future generations? we have the responsibility to leave the environment in as healthy a condition as that which we inherited it. we need to respect the balance of nature’s ecosystems when 15 saaty t. and begicevic n. 2010. the scope of human values and human activities in decision making. applied soft computing. 10(4), 963-974. 16 saaty t. and forman e.h. 1996. the hierarchon: a dictionary of hierarchies. expert choice. 17 saaty t. and cillo b. (2007). the encyclicon volume 2. pittsburgh, pa: rws publications. ijahp article: ferretti, saaty/ questions and answers: valentina ferretti interviews tom saaty international journal of the analytic hierarchy process issn 1936-6744 143 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.235 we use its resources, and also consider ending illegal and destructive logging, destructive fishing and polluting industries. we also need to take a look at our lifestyles and consider the things which may contribute to climate change. the past we inherit; the future we create, the saying goes. to create a credible future that is founded on our global values and priorities we need to learn how to deal with the immense variety of factors and with the expanse of the many dimensions of this variety. our progress depends on our ability to make effective decisions that depend on one another in their causes and effects, on inputs and outputs and on being able to reach out to the smallest and largest causes and effects of which we are aware. in this context a fundamental role is played by the design of alternative future cities. the research trend that i foresee is thus about mcdm in supporting the design of better cities and a better environment for the future, in the effort to shrink the world to building the collective mind of the inventive species, homosapiens. ijahp article: shih/subjective geometry for problem modeling in the analytic hierarchy/network processes international journal of the analytic hierarchy process 1 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1012 subjective geometry for problem modeling in the analytic hierarchy/network processes hsu-shih shih 1 department of management sciences, tamkang university taiwan, r.o.c. e-mail: hshih@mail.tku.edu.tw abstract this paper is in memory of professor thomas l. saaty, who died on august 14, 2017. in a problem-solving process, the first step is to model a real-world problem. the modeling effort is one of the hardest parts of the process, and there is no universal way to do it. the initial step involves structuring the problem and obtaining a simplified problem so that any multi-criteria or statistical models can be applied. this paper intends to explain the modeling process through subjective geometry. although creative thinking is critical in developing the hierarchies and networks in the analytic hierarchy or network process (ahp/anp), models from business management and information theories can supply a verified ready-to-use relationship among elements and their clusters. we hope the exploration offers a guideline for utilizing the ahp/anp in more detail. keywords: structure problem; subjective geometry; analytic hierarchy process; analytic network process; decision making; problem modeling 1. introduction to solve a problem, one first needs to identify the problem. a problem is a question that results in inquiry, consideration, or a solution and generally appears when humans feel there is a difference or discrepancy between the current condition and the intended situation. people may feel uncomfortable if the problem remains unsolved or an unwanted situation exists. we may struggle to recognize the real issue and address the incorrect problem. it is always important to carefully recognize the real problem, which can often be difficult to see at first (clemen, 1996). aside from knowing the environment well, specific domain knowledge from experienced experts is helpful. management tools such as 5w1h (what? who? where? when? why? how?) (robertson, 1946), brain storming (osborn, 1953), an ishikawa diagram or fishbone diagram (ishikawa, 1968), triz (a russian acronym for theory of the resolution of invention-related tasks) acknowledgments: the author wishes to thank professor david l. olson at the university of nebraska, u.s.a. for his valuable help with this study. ijahp article: shih/subjective geometry for problem modeling in the analytic hierarchy/network processes international journal of the analytic hierarchy process 2 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1012 (altshuller & shapiro, 1956), or other creative techniques, could also help to discover the true problem. in addition to quantitative data illustrating the problem clearly, qualitative data or even a figure can provide an image of the possible problems that exist. some graphic tools, such as the fishbone diagram, help structure the problem and understand the nature of the problem by roughly capturing its system components (clement, 1996). these methods are beneficial, especially in the early stages of decision making. in addition, a hierarchy or network could assist in visualizing the problem and its relevant elements through a geometric representation (saaty & shih, 2009). after identifying a problem, the decision maker (dm) reacts to it and attempts to solve it with his or her conscious and aware mind. this response starts the problem-solving process. simon (1977) proposed an idea about the decision-making process, related to the problem-solving process, which includes the following phases: (i) intelligence, (ii) design, and (iii) choice. he later added another phase, (iv) implementation. from the perspective of management, the process usually involves a monitoring phase so that the consequences of implementation can be examined and that is can be confirmed that the consequences were as expected; otherwise, we need another cycle of the process for problem solving. some experts have examined the cognitive role of the dm. rowe and boulgarides (1992) included a stimulation phase before the above phases. the cognitive stage views outside factors, such as potential advantages, dangers, or feedback from the current system, which are the root cause of the issue. decision theory emphasizes preference, which is a technical term for selecting one option over another, in addition to the dm’s cognition (einhorn & hogarth, 1981). it has a direct relation to desires and includes two contrasting options before choosing one. the general process for solving problems is outlined in figure 1. real-world problem solution simplified problem model verification validation implementation ijahp article: shih/subjective geometry for problem modeling in the analytic hierarchy/network processes international journal of the analytic hierarchy process 3 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1012 figure 1 problem-solving process (shih & olson, 2022) the paper is organized as follows. section 2 discusses hierarchies and networks in detail and provides many different forms of hierarchies and networks in decision making. section 3 illustrates the relationships of these geometric structures. section 4 discusses the use of these hierarchies and networks. section 5 provides some conclusions. 2. geometric structures geometry is concerned with properties of space that are related to distance, shape, size, and relative position of figures (tabak, 2014). the shape or figure of the elements in relation to each other could help solve a problem. in a traditional managerial meeting that is promoting teamwork, the participants can use a whiteboard to draw figures to focus on the issue being addressed. once the figures are drawn, even if they are not represented precisely, the participants can discuss the related elements analogous to the problem, stimulating critical thinking and modifying the figures as needed. an example of this process could be brainstorming or brainwriting (hwang & lin, 1987). the cognitive process is to identify the problem and realize its relationship via a visualized shape or figure. this is called “geometric cognition” or “spatial/visual reasoning in mathematics”, which offers the clearest and often unavoidable expression in mathematical cognition (whiteley, 2015). though numerous figures could be considered in the process, hierarchies or networks are common types of figures used for this purpose. saaty (1980, 1996) developed the analytic hierarchy process (ahp) and the analytic network process (anp) to solve problems and provided solid theories to support the decisions. both the ahp and anp are structured frameworks for multi-criteria decision making (mcdm). however, we will concentrate on the modeling of the problem. 2.1 hierarchical and networked structures saaty and vargas (1994) defined a hierarchy as the ordering of parts or elements of a whole from the highest to the lowest, in a top-to-bottom fashion, and with connected relationships among them to fulfill the function of an organization (saaty & vargas, 1994). the simplest hierarchical structure consists of a goal, criteria, and alternatives. after the levels and their elements are chosen, pair-wise comparisons of the strengths of preferences are executed from the top-down, and the synthesis process is carried out from the bottom-up for the final decision (saaty, 1980). saaty (1980) also provided a benefitcost analysis, in which benefit and cost hierarchies are involved in the modeling process. he originally suggested the ratio of benefit to cost (i.e., b/c) on alternatives as the standard to rank them. however, a few debates arose because the cutoff value for the ratio is the critical factor for the selection. readers interested in this discussion can refer to saaty (1996) and shih (2008) for details. to manage a complex problem, saaty and özdemir (2003) further introduced four merits of a decision: benefit (b), opportunity (o), cost (c), and risk (r). the basic idea is to establish four hierarchies for modeling the problem and synthesize them at the level of alternatives. they proposed four formulae for the synthesis which are as follows: (i) ijahp article: shih/subjective geometry for problem modeling in the analytic hierarchy/network processes international journal of the analytic hierarchy process 4 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1012 bo/cr, (ii) bb+oo+c(1/c)+r(1/r), (iii) bb+oo+c(1–c)+r(1–r), and (iv) bb+oo–cc– rr, where b, o, c, and r are the weights assigned to the corresponding merits. though bo/cr is directly extended from benefit-cost analysis, they preferred the use of bb+oo–cc–rr. these four weights are obtained from a pair-wise comparison. wijnmalen (2007) verified that the additive synthesis with properly weighted factor priorities based on relative magnitudes produces sound results. a network is a system of connected travel routes or communication lines, where some elements are connected to influence others, be influenced by others, mutually influence each other, or have a feedback with a connected cycle. its geometry does not have the simple linear top-to-bottom form of a hierarchy. network representation is rather flexible and relies on the purpose of a system or what an organization is trying to achieve. besides its flexibility, the network can formulate relationships of dependence and feedback or with interactions among elements. hence, the figure could be any shape in geometry. because of its flexibility, network formulation is challenging and needs a control hierarchy to guide the thought process whereby the function of the goal/focus is at the top of the hierarchy. fortunately, saaty (1996) provided the supermatrix from the priority vectors of positive reciprocal matrices, to deal with the problem under a control hierarchy, and suggested four measures to manage complex problems, opportunity, cost, and risk along with benefit. 2.2 simplified structures a network has various advantages in modeling real-world problems. in reality, the anp suffers from a large number of similar questions given to experts if the structure is complex. to avoid this judgmental confusion, we need to take a very cautious step to avoid the phenomenon of “garbage in, garbage out.” one option is to apply statistical methods to eliminate unimportant relationships in the network. saaty and takizawa (1986) suggested a simplified structure, including three steps, to deal with the dependence on elements. the first step, as in traditional ahp, only processes the influences without considering dependence. the second step considers dependence, and the final step combines the information from the first two to determine the final priority with dependence. despite the fact that the proposal has limited capability for handling dependence, it is a tradeoff between network complexity and the ease of applicable procedure. for example, shyur and shih (2006) utilized a hierarchical structure with dependent relationships to solve the problem of vendor selection, in which there were four criteria among the seven encounter interactions. the simplified structure or this problem can be seen in figure 2. ijahp article: shih/subjective geometry for problem modeling in the analytic hierarchy/network processes international journal of the analytic hierarchy process 5 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1012 on-time delivery product quality price/cost facility and technology responsive -ness to customer needs professionalism of salepersons quality of relationship with vendors vendor selection vendor 1 vendor 2 vendor n ● ● ● ● ● ● ● ● ● ● ● ● figure 2 simplified structure (shyur & shih, 2006) the decision making trial and evaluation laboratory (dematel) technique by gabus and fontela (1972) has drawn much attention in recent decades. ou yang et al. (2008) combined dematel with anp, naming it dematel-based anp, to solve problems with dependence and feedback. since dematel also takes advantage of limiting operations on matrices, dematel and anp share some common characteristics so their combination does not have much effect. gölcük and baykasolu (2016) organized four types of combinations of dematel and anp. however, dematel can only identify cause-and-effect relationships in a complicated system and is incapable of handling the structural dependence of the anp. hence, these combinations could be ambiguous in designing problem-solving processes. interested readers could read gölcük and baykasolu (2016) to learn more about the four classes of dependence: criteria dependency (structural dependency, causal dependency, and preferential dependency) and criteria interactivity. 2.3 other structures as anp has been prominent over the past two decades, various studies have applied it to business management under existing structures. by taking advantage of existing or modified structures, the anp is able to formulate the influence of dependence and feedback in the analyses. for instance, yüksel and dadeviren (2007) utilized the anp to measure the dependence among strategic factors under the structure of strengths, weaknesses, opportunities, and threats (swot). they used a four-level hierarchical structure with dependence only on the swot aspects and observed the change in strategy priorities for a textile firm. nguyen and truong (2022) also considered swot for modeling vietnam’s strategic agricultural development in the context of a drought. hsu et al. (2012) referred to the e-s-qual model (parasuraman et al., 2005) to assess the electronic service quality (e-sq) of online travel websites using the anp. considering the interdependence among its criteria and sub-criteria, the study sheds light on the truly important criteria and sub-criteria of e-sq for future improvement. povedaijahp article: shih/subjective geometry for problem modeling in the analytic hierarchy/network processes international journal of the analytic hierarchy process 6 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1012 bautista et al. (2012) exploited the anp to prioritize 17 company competitiveness indicators (ccis) under the structure of a balanced scorecard (bsc) with four aspects and for three companies. managing the dependence and feedback on ccis and the aspects, the research obtained the relative positions of the companies in the sector and the priorities of ccis. wu et al. (2012) employed the anp to evaluate the current strategy by choosing the most important criteria under the structure of porter’s five forces model. the 30 criteria under five forces were evaluated to identify the important criteria under the current strategy for the case company. in the area of information theories, there are also numerous structures. in particular, chen and shih (2014) examined the acceptance of wearable technology for consumers under the structure of the unified theory of adoption and use of technology (utaut) model (venkatesh et al., 2003) through an anp analysis. because four constructs that include performance expectancy, effort expectancy, social influence, and facilitating conditions, are direct determinants of behavioral intention and use behavior, the anp assessed 14 factors for the four constructs and three each for behavioral intention and use behavior, as shown in figure 3. nilashi et al. (2016) determined the importance of hospital information system adoption factors under the structure of the human, organization, and technology (hot) fit model (yusof et al., 2008) using the anp. the 17 sub-factors under four factors (i.e., technological, organizational, environmental, and human) were weighted and ranked to determine the influences on adoption. there are innumerable theories in the areas of business management and information management. these existing structures or models can indeed provide fast and ready-to-use structures to apply the anp. ijahp article: shih/subjective geometry for problem modeling in the analytic hierarchy/network processes international journal of the analytic hierarchy process 7 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1012 figure 3 utaut model using the anp (chen & shih, 2014) it is obvious that the models from business management and information theories can supply a verified ready-to-use relationship among elements and their clusters. hence, the anp does not exert much effort on the relationship and concentrates on the intensity of the relationship. therefore, the process saves time sorting out dependence and feedback. more networks can be found in saaty and özdemir (2005). 3. structure relationships geometry is a branch of mathematics that deals with the measurement, properties, and relationships of points, lines, angles, surfaces, and solids. its origins are in ancient greece goal: the acceptance of wearable technology facilitating conditions perceived behavioral control compatibility facilitating conditions use behavior low medium high use intention intend to use plan to use predict to use effort expectancy perceived ease of use ease of use complexity social influence subjective norm image social factors performance expectancy perceived usefulness extrinsic motivation job-fit relative advantages outcome expectations ijahp article: shih/subjective geometry for problem modeling in the analytic hierarchy/network processes international journal of the analytic hierarchy process 8 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1012 and it is one of the oldest branches of its field. the scope of geometry has been greatly expanded and the field has been split into many subfields (de risi, 2015). in euclidean geometry, which is the closest to our discussion, the elements begin with plane or twodimension geometry. plane geometry is taught in secondary or high school as the first axiomatic system and the first example of a mathematical proof. it then goes on to solid geometry which involves three dimensions. much of the element’s states are the result of what is now known as algebra and number theory, which are explained in geometrical language (eves, 1972). for most of the decision-making process, only the simple elements, line segments and shapes are enough to establish relationships in a twodimensional plane. for instance, in the fishbone diagram, the causes of imperfection are the sources of variation. the causes of the variation are usually grouped into major categories to identify and classify the sources (ishikawa, 1968). we may question the process that lacks a rigorous mathematical proof. nevertheless, it is the first step to focusing our cognition on the targeted problem. this is the reason why it is called “subjective geometry.” although the question of mathematical proof is not answered, statistical tools or observations on brain responses could be helpful to ensure the relationships under the problem. for instance, shih et al. (2014) organized criteria by the content validity ratio to delete the minor criteria and executed factor analysis to determine groups of the criteria and clusters to which they belong. thus, the experts can manage a simplified questionnaire on the relationships between dependence and feedback in the network. due to these helpful tools, the burden of cognition of dms decreases so that right judgments can be generated. note that though there are no perfect dms, better decisions can be made through more structure and guidance (clemen, 1996). within decision analysis, we are always concerned about whether we are dealing with the right problem. validation is critical in guaranteeing that the true problem is being solved, but there is no universal way to do it. saaty (2005a) discussed validation by comparing solutions to real world facts, and both outcomes are very close in numerous listed cases. this means that the analytic results of the ahp/anp can solve real world problems if the judgments of experts are carefully handled. saaty (2005a) also supported the idea that group decision-making is better than any wise individual because it can obtain benefits from multiple sources of knowledge and experiences so that the bias of the individual can be alleviated (surowiecki, 2004). a sensitivity analysis allows dms to examine the effect of varying the influences on the stability of the outcome in the ahp/anp (saaty, 2005b), but it is a verification process, as shown in figure 1, and does not relate to dealing with the right problem. 4. discussion in section 2, we discussed many types of networks and hierarchies. using existing networks for business management and information theories is beneficial because the models have been verified and are ready to be applied. nonetheless, the approaches appear to be lacking in innovation without incorporating our human potential (saaty, 2001). in addition, there is a difference in the networks from business management and information theory compared to the anp. they generally supply the relationships of the networks, but not the intensity of the relationships. however, the anp could help ijahp article: shih/subjective geometry for problem modeling in the analytic hierarchy/network processes international journal of the analytic hierarchy process 9 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1012 prioritize the influences of the elements and their clusters/levels and provide more decision insights regardless of how it exploits networks, with or without the two areas. 5. conclusions subjective geometry is a useful tool for extracting our cognition during the decision making process through the use of some common management tools. it introduces an initial structure for making a decision. we can thus make judgments about the ahp/anp procedure through the fundamental scale. in spite of the fact that subjective geometry lacks a rigorous mathematical representation, it is a rapid form of approximation to the real problem. creative thinking is critical in developing these hierarchies and networks. in addition, some statistical tools can help alleviate the burden of the cognitive process, which group decision making also does. the proposed merits of bocr also play the same role in the cognitive process. ijahp article: shih/subjective geometry for problem modeling in the analytic hierarchy/network processes international journal of the analytic hierarchy process 10 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1012 references altshuller, g., shapiro, r. (1956). about technical creativity. questions of psychology, 6, 37-49. (in russian) chen, c.-c., shih, h.-s. (2014), a study of the acceptance of wearable technology for consumers an anp perspective. 13th international symposium on the analytic hierarchy process, isahp 2014. washington, dc, usa. doi: https://doi.org/10.13033/isahp.y2014.011 clemen, r. t. (1996). making hard decisions: an introduction to decision analysis. pacific grove, california: brooks/cole publishing, second edition. isbn: 9780534260347. de risi, v. (2015)(ed.). mathematizing space: the objects of geometry from antiquity to the early modern age. cham, switzerland: birkhäuser, springer. doi: https://doi.org/10.1086/687011. einhorn, h. j., hogarth, r. m. (1981). behavioral decision theory: processes of judgment and choice. annual review of psychology, 32, 53-88. doi: https://doi.org/10.1146/annurev.ps.32.020181.000413. eves, h. (1972). a survey of geometry (volume one). white plains, new york, allyn & bacon, longman, revised edition. isbn: 9780205032266. gabus, a., fontela, e. (1972). world problems, an invitation to further thought within the framework of dematel, 1-8. battelle geneva research center, geneva, switzerland. gölcük, i., baykasoğlu, a. (2016). analysis of dematel approaches for criteria interaction handling within anp. expert systems with applications, 46, 346-366. doi: https://doi.org/10.1016/j.eswa.2015.10.041. hsu, t.-h., huang, l.-c., tang, j.-w. (2012). a hybrid anp evaluation model for electronic service quality. applied soft computing, 12(1), 72-81. doi: https://doi.org/10.1016/j.asoc.2011.09.008. hwang, c.-l., lin, m.-j. (1987). group decision making under multiple criteria. springer-verlag, berlin. doi: https://doi.org/10.1007/978-3-642-61580-1. ishikawa, k. (1968). guide to quality control. tokyo: juse. isbn: 978-9283310365. nguyen, h.t., truong, t.c. (2022). integral swot-ahp-tows model for strategic agricultural development in the context of drought: a case study in ninh thuan, vietnam. international journal of the analytic hierarchy process, 14(1), 1-30. doi: https://doi.org/10.13033/ijahp.v14i1.890. nilashi, m., ahmadi, h., ahani, a., ravangard, r., ibrahima, o.b. (2016). determining the importance of hospital information system adoption factors using fuzzy analytic ijahp article: shih/subjective geometry for problem modeling in the analytic hierarchy/network processes international journal of the analytic hierarchy process 11 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1012 network process (anp). technological forecasting and social change, 111, 244-264. doi: https://doi.org/10.1016/j.techfore.2016.07.008. osborn, a. f. (1953). applied imagination: principles and procedures of creative thinking. new york: charles scribner's sons. isbn: 9781447417101 ou yang, y.-p., shieh, h.-m., leu, j.-d., tzeng, g.-h. (2008), a novel hybrid mcdm model combined with dematel and anp with applications. international journal of operations research, 5, 160-168. parasuraman, a., zeithaml, v.a., malhotra, a. (2005). e-s-qual: a multiple-item scale for assessing electronic service quality. journal of service research, 7(3), 213-233. doi: https://doi.org/10.1177/1094670504271156. poveda-bautista, r., baptista, d.c., garcía-melón, m. (2012). setting competitiveness indicators using bsc and anp. international journal of production research, 50, 47384752. doi: https://doi.org/10.1080/00207543.2012.657964. robertson, jr., d. w. (1946). a note on the classical origin of "circumstances" in the medieval confessional. studies in philology, 6-14. rowe, a. j., boulgarides, j. d. (1992). managerial decision making. new york: macmillan. isbn: 9780024041111. saaty, t. l. (1980). the analytic hierarchy process. new york: mcgraw-hill. isbn: 9780070543713. saaty, t.l. (1996). decision making with dependence and feedback: the analytic network process. pittsburgh: rws publications. isbn: 9780962031793. saaty, t.l. (2001), creative thinking, problem solving and decision making. pittsburgh, pa: rws publications. isbn: 9781888603033. saaty, t.l. (2005a). theory and applications of the analytic network process: decision making with benefits, opportunities, costs and risks. pittsburgh, pa: rws publications. isbn: 9781888603064. saaty, t.l. (2005b). making and validating complex decisions with the ahp/anp. journal of systems science and systems engineering, 14, 1-36. doi: https://doi.org/10.1007/s11518-006-0179-6. saaty, t.l., özdemir, m.s. (2003). negative priorities in the analytic hierarchy process. mathematical and computer modelling, 37(9-10), 1063 -1075. doi: https://doi.org/10.1016/s0895-7177(03)00118-3. saaty, t.l., özdemir, m.s. (2005). the eecyclicon: a dictionary of decisions with dependence and feedback based on the analytic network process. pittsburgh, pa: rws publications. isbn: 9781888603057. ijahp article: shih/subjective geometry for problem modeling in the analytic hierarchy/network processes international journal of the analytic hierarchy process 12 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1012 saaty, t.l., takizawa, m. (1986). dependence and independence − from linear hierarchies to nonlinear networks. european journal of operational research, 26, 229237. doi: https://doi.org/10.1016/0377-2217(86)90184-0. saaty, t.l., shih, h.-s. (2009). structures in decision making: on the subjective geometry of hierarchies and networks. european journal of operational research, 199(3), 867-872. doi: https://doi.org/10.1016/j.ejor.2009.01.064. saaty, t.l., vargas, l.g. (1994). decision making in economic, political, social and technological environments: the analytic hierarchy process. pittsburgh, pa: rws publications. isbn: 9780962031779. shih, h.-s. (2008). incremental analysis for mcdm with an application to group topsis. european journal of operational research, 186(2), 720-734. doi: https://doi.org/10.1016/j.ejor.2007.02.012. shih, h.-s., cheng, c.-b., chen, c.-c., lin, y.-c. (2014), environmental impact on the vendor selection problem in electronics firms – a systematic analytic network process with bocr. international journal of the analytic hierarchy process, 6(2), 1-25. doi: https://doi.org/10.13033/ijahp.v6i2.256. shih, h.-s., olson, d.l. (2022). topsis and its extensions: a distance-based mcdm approach. cham, switzerland: springer nature. isbn: 9783031095764. shyur, h.-j., shih, h.-s. (2006), a hybrid mcdm model for strategic vendor selection. mathematical and computer modelling, 44(7-8), 749-761. doi: https://doi.org/10.1016/j.mcm.2005.04.018. simon, h.a. (1977). the new science of management decision. 3rd ed., englewood cliffs, new jersey: prentice hall. isbn: 9780136161363. surowiecki, j. (2004). the wisdom of crowds: why the many are smarter than the few and how collective wisdom shapes business, economies, societies and nations. new york: random house. doi: https://doi.org/10.4324/9781912453184-22. tabak, j. (2014). geometry: the language of space and form. new york: infobase publishing. isbn 9780816049530. venkatesh, v., morris, m.g., davis, g.b., and davis, f.d, (2003). user acceptance of information technology: toward a unified view. mis quarterly, 27(3), 425-478. doi: https://doi.org/10.2307/30036540. whiteley, w. (2015). geometric cognition. in v danesi, m. (ed.), interdisciplinary perspectives on math cognition. 247-262. cham, switzerland: springer. doi: https://doi.org/10.1007/978-3-030-22537-7_13 ijahp article: shih/subjective geometry for problem modeling in the analytic hierarchy/network processes international journal of the analytic hierarchy process 13 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.1012 wijnmalen, d.j.d. (2007). analysis of benefits, opportunities, costs, and risks (bocr) with the ahp–anp: a critical validation. mathematical and computer modelling, 46(78), 892-905. doi: https://doi.org/10.1016/j.mcm.2007.03.020. wu, k.-j., tseng, m.-l., chiu, a.s.f. (2012). using the analytical network process in porter’s five forces analysis – case study in philippines. procedia social and behavioral sciences, 57, 1-9. doi: https://doi.org/10.1016/j.sbspro.2012.09.1151. yusof, m.m., kuljis, j., papazafeiropoulou, a., stergioulas, l.k. (2008). an evaluation framework for health information systems: human, organization and technology-fit factors (hot-fit). international journal of medical informatics, 77(6), 386-398. doi: https://doi.org/10.1016/j.ijmedinf.2007.08.011. yüksel, i̇, dağdeviren, m. (2007). using the analytic network process (anp) in a swot analysis – a case study for a textile firm. information sciences, 177(16), 3364-3382. doi: https://doi.org/10.1016/j.ins.2007.01.001. 1_dear_ijahp_readerenrique_dec23_p_90 ijahp preface: mu/words from the editor-in-chief         international journal of the analytic hierarchy process vol. 3 issue 2 2011 issn 1936-6744                                        december  2011   dear  ijahp  reader:   with  this  issue,  ijahp  celebrates  its  third  year  of  publication.  we  have  been  very  active   during  this  period  and  have  started  publishing  special  issues;  the  first  one  was  dedicated   to  the  status  of  ahp  practice  in  brazil  and  we  are  currently  working  on  another  special   issue  on  the  topic  of  sustainability,  scheduled  for  spring  2012,  thanks  to  the  initiative  of   fabio  defelice  and  the  valuable  assistance  of  antonella  petrillo.  more  such  special   issues  are  planned  for  the  future.  a  new  section  called  news  and  events  has  been  added   to  keep  our  ijahp  community  abreast  of  events  of  interest  and  here  you  will  find  fabio   defelice’s  report  on  the  success  of  the  isahp  2011  conference  in  june  in  sorrento,  italy,   and  news  about  jagiellonian  university  of  krakow,  poland,  awarding  thomas  saaty  an   honorary  doctorate.    the  current  issue  has  five  articles,  including  the  last  installment  on   the  topic  of  the  eigenvectors  of  the  ahp/anp  by  stan  lipovetsky  and  an  essay  on  the   serious  omission  of  comparisons  from  aristotle’s  laws  of  thought  by  thomas  saaty.     enjoy  the  holiday  issue!   enrique  mu   ijahp  editor-­‐in-­‐chief           rob typewritten text rob typewritten text http://dx.doi.org/10.13033/ijahp.v3i2.59 ijahp news: karpak/mcdm 2017 is in ottawa, canada, july 10-14, 2007 international journal of the analytic hierarchy process 546 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.446 mcdm 2017 is in ottawa, canada, july 10-14, 2017 birsen karpak associate editor, ijahp youngstown state university bkarpak@ijahp.org the 24th international conference on multiple criteria decision making will be organized in ottawa, canada, from july 10th to july 14th, 2017. the deadline for the abstracts is february 1, 2017. one of the leading conferences on multiple criteria decision-making (mcdm), the international mcdm conference is hosted by the telfer school of management at the university of ottawa. papers on theory and applications of mcdm on variety of subject are expected. organizers invite you ottawa for mcdm 2017 during canada’s 150th birthday celebration! http://sites.telfer.uottawa.ca/mcdm2017/ 8_defelice_ijahp_report_isahp_2011_dec_23_pp_181-183 ijahp news: de felice/ special report on isahp2011 meeting   international journal of the analytic hierarchy process 181 vol. 3 issue 2 2011 issn 1936-6744     fabio  de  felice  –  executive  organizer  of   isahp2011,  university  of  cassino,  italy antonella  petrillo  –  executive   secretary  of  isahp2011,   university  of  cassino   special  report  on  isahp2011                                       dear  colleagues  and  friends,     i  am  proud  and  honored  to  contribute  to  this  issue  of  the  international  journal  on   the   analytic   hierarchy   process.   it   is   the   first   issue   since   the   international   symposium  on  the  analytic  hierarchy  process  2011  (isahp  2011)  was  held  in   sorrento  (naples  -­‐‑  italy)  from  june  15-­‐‑18,   2011.  pictures  from  the  three  days  and  gala   dinner  are  posted  here:   www.isahp.org/italy2010/index.php      the   conference   organizing   and   program   committees   represented   all   five   continents,   and   they   were   able   to   bring   together   the   research,   applications   and   perspectives   of   their   areas   of   the   world   to   this   truly   international  forum.     the   papers   presented   in   this   conference   reflect   the   critical   areas   of   research   in   the   ahp   field.   some   of   the   topics   presented   include  but  are  not  limited  to:   ijahp news: de felice/ special report on isahp2011 meeting   international journal of the analytic hierarchy process 182 vol. 3 issue 2 2011 issn 1936-6744      environmental  and  sustainability  applications,  disaster  management,  decision   support  system  aids,  group  decision  making,  conflict  resolution,  purchasing   and   supply   chain   decisions,   medical   decision   making,   civil   engineering   and   urban   management   applications,   quality   management   in   organizations,   marketing   decisions,   human   resource   management,   and   applications   in   the   field  of  education.     i  believe  that  the  symposium  and  its  collected  papers  are  an  effort  to  stimulate   debate  on  the  theme  of  supplying  people  with  methods  to  make  better  decisions,   allowing  discussions  and  even  confrontation  in  order  to  freely  propose  ideas  and   solutions.     in  the  changing  social,  political  and  technological  environment  we  have  today,  i   believe  that  we  must  encourage  people  to  learn  how  to  make  the  right  decisions   at  the  right  time!     the   international   symposium   on   the   analytic   hierarchy   process   and   the   international  journal  of  the  ahp  (  ijahp)  provide  a  forum  for  discussion  about   the  ahp,  and  a  place  to  publish  quality  articles  with  useful  information  about   the  ahp  as  it  is  applied  in  social,  political,  economic,  and  other  fields.               this  year   there  were  160  papers  presented  at   isahp2011,   and   it  was   attended  by   almost  200  researchers,  scholars  and  visitors  from  96  cities  in  42  countries.   ijahp news: de felice/ special report on isahp2011 meeting   international journal of the analytic hierarchy process 183 vol. 3 issue 2 2011 issn 1936-6744     isahp   represents   the   chance   for   many   researchers   and   scientists   around   the   world   to   get   together   and   present   their   own   studies   and   research   in   this   important  field.  in  this  way,  the  conference  has  been  a  chance  to  encourage  the   integration   of   the   international   competencies   in   the   ahp   field,   and   to   supply   important  cues  for  future  researchers.     finally  i  would  like  to  thank  all  the  authors  who  submitted  papers  for  isahp   2011   and   to   give   special   thanks   to   the   international   committee,   the   local   scientific  committee  and  to  prof.  thomas  l.  saaty.     enjoy  yourself  with  this  issue  while  waiting  for  isahp  2013  in  kuala  lumpur,   malaysia!     prof.  fabio  de  felice   isahp  2011  executive  organizer   university  of  cassino  –  italy       rob typewritten text rob typewritten text rob typewritten text http://dx.doi.org/10.13033/ijahp.v3i2.125 ijahp article: bahurmoz/measuring corporate social responsibility performance: a comprehensive ahp based index international journal of the analytic hierarchy process 20 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.608 measuring corporate social responsibility performance: a comprehensive ahp based index asma mohammed bahurmoz bahurmoz consult asma@bahurmoz.com abstract although there is no agreed upon universal definition of corporate social responsibility (csr), organizations are often ranked in terms of their csr performance. however, two glaring gaps have been identified in the csr literature. first, evaluation methodologies are questionable and often lack a scientific basis and second, stakeholder representation is not made explicit or is missing altogether. this paper contributes to the csr literature by constructing a csr index based on the analytic hierarchy process (ahp), as well as ensuring that stakeholder judgments are an integral part of the constructed index. furthermore, the developed index is implemented to measure csr performance in a business setting. an ahp-based csr index is developed for the services sector in saudi arabia to serve as a case study. the developed index is used to measure csr performance in over forty corporations. the paper adds to the existing literature by providing insight into how the saudi corporations perceive and practice csr. it concludes that a systematic usage of the developed ahp-based csr index would facilitate corporations to adopt a more responsible and measurable behavior, while it offers government institutions the option to rank corporations in terms of their csr practices in a scientific and transparent manner. keywords: ahp; corporate social responsibility (csr); csr performance; group decision making; saudi arabia; services sector 1. introduction tead and metcalf (1933) stated that “institutions exist not for their own sake or for the benefit of some small group which controls them. they exist to minister to the life of the entire community”. hoffman (2007), in his historical study, concluded that the concept of corporate social responsibility (csr) goes back to the 1920s. it has grown in recognition as exemplified by initiatives like the global reporting initiative in 2002, and the more recent directive of the european parliament and council of 2013 that require a csr disclosure in annual financial reporting. however, in spite of the accelerating rise of the csr concept in recent decades, and its popularity as a research topic, it has no exact definition to date and lacks a universally accepted framework (govindan et al., 2018; luning, 2012; 2010; devinney, 2009; mcwilliams et al., 2006). bowen (1953, 1975) provides the first modern definition of the concept by stating that businesses are responsible for their actions beyond profit and loss statements. carroll (1979) defines social responsibility as “the economic, legal, and discretionary expectation that society has of organizations”. in 2001, two definitions were proposed by the ijahp article: bahurmoz/measuring corporate social responsibility performance: a comprehensive ahp based index international journal of the analytic hierarchy process 21 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.608 european commission's green paper: 1) csr is a concept whereby companies decide voluntarily to contribute to a better society and a cleaner environment, 2) csr is a concept whereby companies integrate social and environmental concerns in their business operations and in their interaction with their stakeholders on a voluntary basis (commission of the european communities, 2001). the lack of a clear-cut definition of csr has resulted in a wide variation of its practical use (oberseder et al. 2013). some managers view csr as an obligation, some define it as a considered proactive behavior, and still others believe it is nothing more than a reactive action or simply charity (fontaine, 2013; chen et al. 2018). this divergence in managerial perceptions is partly due to the heterogeneous factors that influence csr behavior and practices. these factors include globalization, governmental and intergovernmental bodies, advances in communication technologies, growing demand for more transparency on the policies companies are following in managing environmental and social issues, corporate governance, and finally the limitation of governments to regulate all aspects of csr. mosgaller (2012) states that the three pillars of performance (purpose, process, and people) are essential if csr is to evolve from merely a passing fad to an integral part of organizational practice. the basic argument is that if csr is to be a sustainable proposition, the purpose of csr should be clear to employees, processes should be in place to implement csr effectively and stakeholders should engage in and commit to the csr practices implemented within their organizations. trapp (2014) stated that involving stakeholders in the decision to adopt csr strategies would increase the benefits to a company. related research indicates that there is a correlation between a company’s csr practices and stakeholder responses and attitudes towards that company (tsourvakas & yfantidou, 2018). baric (2017) demonstrates that “the concept of corporate social responsibility has gone, in its several decades of existence, from the ‘unnecessary dependency’ phase to the ‘critical business model phase’”. therefore, there is an urgent need to develop a robust system to measure corporate performance with respect to csr, and this system must address all stakeholders’ interests. 2. objectives and scope of the study although there is abundant research addressing csr in the last decade, it would not be an exaggeration to state that confusion, measurement challenges, and transparency are only a few of the many problems facing the practice of csr worldwide, (pérez & rodríguez del bosque, 2013). it appears that there is no systematic implementation and/or adaptation of csr practices, and as result, the effectiveness of these practices remains ambiguous at best. recently published research shows that the measurement of csr performance is a key objective especially to help funders and investors decisions (indre slapikaite, 2016). accordingly, there is a clear need to formulate a systematic scientific methodology that will not only help corporations identify their social, environmental and economic responsibilities, but one that would also facilitate stakeholders in identifying and prioritizing which factors, in particular, effectively deliver these responsibilities in a transparent and measurable manner. against this background, the aim of this research is to construct a comprehensive csr index that reflects and represents the priorities of stakeholders and that can be utilized to evaluate their csr performances against their own established csr goals. the proposed index is illustrated by constructing a csr ijahp article: bahurmoz/measuring corporate social responsibility performance: a comprehensive ahp based index international journal of the analytic hierarchy process 22 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.608 index for the services sector in saudi arabia. the developed index is implemented to rank corporations in the services sector with respect to their csr performance as prioritized by their stakeholders. 3. emergence of csr index as previously discussed, there is a growing recognition by businesses that csr is, and should be, an integral part of their strategic vision. on the one hand, this agenda is dictated by the greater society, which now demands that businesses be more socially responsible in their decisions and actions, and on the other hand, this focus is partly attributable to greater awareness on the part of the businesses themselves. reflecting this trend, a number of international institutions set out to evaluate market performance of socially responsible firms that gave rise to the so-called csr index and launched csr as a new dimension to measure corporate value. the csr index is defined as a “management and benchmarking tool that enables companies to effectively measure, monitor, report and improve their impacts on society and the environment” (net balance foundation, www.newbalancefoundation.org). such evaluations have been particularly popular in international capital markets as institutions have sought to evaluate the value addition of csr to the corporate value of a firm’s socially responsible investments (sri). in 1999, the first csr index in the world was created by the dow jones stocks and sustainability asset management co., known as the dow jones sustainability world index (djsi world), with the intent to value stock performance of socially responsible firms with reference to expectations of the greater society (wang, 2011). the subsequent rise of csr indices has been fueled by the observation that, on a global level, indices based on csr or environmental, social, and governance (esg) themes have outperformed the benchmark indices. following the lead of the united states, many of the disclosure efforts and the related csr indices that have emerged are from stock exchanges around the world as they attempt to establish a reflective market mechanism that assesses a firm’s efforts in fulfilling its social responsibilities. the djsi world and csr indices in other countries were examined with the intent to identify relevant dimensions and criteria that could be incorporated in constructing a scientific comprehensive csr index to evaluate corporations in saudi arabia. a survey of the related literature reveals 22 csr indices worldwide (see appendix) and shows that construction of csr indices is a relatively recent phenomenon. furthermore, the literature suggests that most countries do not even have any form of informal government regulations to encourage csr disclosure let alone any form of formalized index to monitor disclosure. the middle east is not an exception, as only egypt and saudi arabia support a csr-based index. this is consistent with the csr philosophy that is based on voluntarism. in the absence of government regulations, a scientific based index is necessary to encourage organizations to engage in strategic and transparent csr practices. accordingly, the construction of a scientific based csr index for the saudi corporate world would not only add value to the csr evaluation practices, but it would also set a precedent within middle eastern countries in particular. http://www.newbalancefoundation.org/ ijahp article: bahurmoz/measuring corporate social responsibility performance: a comprehensive ahp based index international journal of the analytic hierarchy process 23 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.608 4. research methodology this research was conducted in two phases. first, a comprehensive csr index was constructed. then, it was implemented to rank service corporations in terms of their csr performance. to construct a comprehensive csr index that includes all stakeholders’ perspectives would entail measuring every single csr indicator. this task would not only be impossible, but it risks confusing rather than clarifying the objectives, especially since at least some, if not all, of the selection is based on normative judgements. therefore, the development of a csr index will be addressed as a multi-criteria group decision-making problem. the methodology that easily lends itself to a task like this is the rating model of the ahp (saaty, 2001). the ahp offers an advantage over other techniques as it focuses on the relative importance of one csr indicator compared to another, and it does not require direct measurement of each indicator for comparison. (saaty, 2008). the published research reveals few applications of the ahp methodology in relation to the field of csr studies. tafti, hosseini and emami (2012) developed a fuzzy ahp model to assess csr practice in a bank. costa and menichini (2013) pointed out the importance of including the stakeholder’s perception and developed a fuzzy multi-criteria model to measure the company csr as perceived by its stakeholders. the proposed research is different in that it builds a comprehensive index representing all stakeholders in the service sector and implements it to evaluate csr performance in the sector. also, it can be easily generalized to cover all other industries. this model also implements the original and simple ahp which has proven to be a robust mathematical model. the ahp methodology aggregates judgments in a way that satisfies the reciprocal relation in comparing two elements. it combines the outcomes of the experts’ judgments using the geometric mean of the judgments. (saaty & peniwati, 2008) the strength of the ahp lies in its capability to compare qualitative and quantitative criteria simultaneously and in integrating the subjective judgments of the decision maker with the objectivity of the alternatives criteria in a robust mathematical model. it follows that as csr indicators often comprise competing conflicting criteria with competing attributes, the ahp offers a logical format to quantify their selection attributes, which can be evaluated systematically, unlike traditional csr index construction methods (bahurmoz et al. 2015). 4.1 constructing the ahp-based csr index constructing an ahp-based csr index as perceived by experts in the private sector and the local community consists of the following stages: 1selection of csr indicators (structuring the hierarchy). 2selecting the group decision makers. 3eliciting experts’ judgements (pairwise comparisons). 4establishing priorities (calculating the principal eigenvector). ijahp article: bahurmoz/measuring corporate social responsibility performance: a comprehensive ahp based index international journal of the analytic hierarchy process 24 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.608 4.1.1 selection of csr indicators a comprehensive csr index must reflect economic, legal, environmental, social, and ethical corporate responsibilities. furthermore, for such an index to be acceptable to a corporation and for it to be ‘owned’ by its management, stakeholders should have the opportunity to set priorities for each of its constituent elements from their personal perspectives. however, as previously stated, there is no specific definition for the csr concept, neither is there agreement on its constituent elements. therefore, as a first step in developing the index, the most frequent elements from the 22 indices studied (section 3 and appendix) were selected and clustered to construct the criteria for the proposed csr index. it is worth noting that each element was carefully assessed with respect to its suitability within the saudi corporate context. the selected indicators are defined herein. 1. legislation: this reflects the extent to which the corporations respect the government’s laws that address the needs of society. these criteria are divided into two sub-criteria, governance and obligations. governance indicates the existence of a system that governs relations among all of the actors who influence the performance of a corporation, such as stockholders rights (equity), stakeholder involvement and employment opportunities for the locals (nationalization). obligations refers to those activities, such as combating corruption, that reflect corporate obligations towards serving the society in which it operates. notably, this sub-criterion is most commonly repeated in international csr indices previously mentioned. 2. social development: this includes communication through increasing public awareness and investing in individuals, an activity that could be termed ‘intelligent giving’ as it encompasses initiatives such as sponsoring talented individuals and minorities in the society. 3. employment: this criterion covers every aspect related to the rights of corporate employees. employment includes aspects such as guaranteeing equal opportunity in recruiting, promoting employee morale and substantial rights, enhancing career development and occupational health and safety. 4. environment: this criterion reflects the extent to which the corporation protects the environment when designing and processing its goods and services. it includes three sub-criteria: the efficient use of resources, environmental consideration and anti-pollution efforts. 5. production efficiency: this refers to the policies the corporation follows in its product/service supply chain, and as such, it covers a wide spectrum of factors. the most important and frequently repeated in most indices are quality and integrity of the product/service, customer relations management (crm), supplier standards, innovation strategies and production costs. these five main indicators and their sub-criteria represent the hierarchy for the proposed ahp-based csr index, as depicted in figure 1. ijahp article: bahurmoz/measuring corporate social responsibility performance: a comprehensive ahp based index international journal of the analytic hierarchy process 25 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.608 figure 1 proposed ahp based index 4.1.2 selecting the group decision makers to construct a proposed csr index that is representative of the priorities of all the stakeholders, the opinions of executive managers from the three service sector categories were sought, i.e. private hospitals, banks and hotels. to remove any industry specific biases and to be able to make ‘like with like’ comparisons, corporations from one business sector, namely, services, were selected. furthermore, to remove any potential bias across the three categories within the service sector, it was decided to combine judgements collected from the service executive managers with judgements from another independent csr expert group. such external validation is a valuable instrument in constructing a robust csr index. this independent csr expert group was categorized as the wider local community, and it was comprised of purposefully selected individuals including academics, mba students and managers from other service sectors. these individuals were chosen to represent the local community on the basis that they would be knowledgeable and possess expertise about csr on par with the executive managers from the service sector organizations. 4.1.3 eliciting expert judgements a questionnaire was designed to ascertain the judgements and opinions of the respondents since it is not feasible to have all of the groups in one setting. the questionnaire was based on a google platform that was adjusted to facilitate the ahp the best company legislation governance equity nationalisation stakeholder involvement obligations (combating corruption) social development public awareness (communication) intelligent giving (investment) employment recruitment fairness employee morale career development occupational health & safety environment efficient use of resources environmental considerations anti -pollution efforts production efficiency quality and integrity of product crm suppliers standards innovation & development strategies cost of production ijahp article: bahurmoz/measuring corporate social responsibility performance: a comprehensive ahp based index international journal of the analytic hierarchy process 26 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.608 pairwise comparisons. using saaty's absolute scale, the following two questions were posed for each element in the ahp hierarchy (saaty, 2001). (1) which of the two criteria do you consider to be more important (dominant) with respect to its upper level criterion? (2) to what degree is the dominant element more important than the subordinate element? acknowledging the fact that some survey participants may not be familiar with such a questionnaire and/or its format, and to remove potential bias and error, the survey was followed up with personal phone calls to the respondents. furthermore, face-to-face meetings were conducted with a random sample of participants to ensure judgement reliability. of the 400 questionnaires posted, 255 were completed and returned. of these, 37 were discarded because they were not complete. thus, the sample comprised 218 completed questionnaires, reflecting a response rate of over 50%. the distribution of the respondents is illustrated in table 1. for illustration, figure 2 shows pairwise comparison of legislation with social development with respect to goal (rank corporate with respect to their csr performance) table 1 distribution of respondents questionnaires local community services corporations total academic students managers banks hospitals hotels distributed 50 100 50 90 60 50 400 received 23 77 18 58 47 32 255 ijahp article: bahurmoz/measuring corporate social responsibility performance: a comprehensive ahp based index international journal of the analytic hierarchy process 27 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.608 figure 2 pairwise comparison of legislation with social development with respect to goal (rank corporate with respect to their csr performance) 4.1.4 establishing priorities based on the judgements given by the respondents, priorities of every element were derived mathematically using the principal eigenvector of a matrix of pairwise comparisons of the main criteria and sub-criteria. ahp based software provides the mathematical calculation of the eigenvalue. it analyzes the priorities showing the relationship between the multi-layered stratification of criteria and sub-criteria to demonstrate a multitude of elements that were pairwise compared so as to determine their relative importance to the goal. the prioritization ranking of the five csr elements that comprise the ahp criteria by the two groups of respondents (i.e., the local community and services corporate sector) are illustrated in table 2. global priorities for all the subcriteria of the proposed csr index are given in table 3. both corporate priorities and community priorities are combined implementing the geometric mean in tables 2 and 3 (saaty, 2001). table 2 priorities of the main csr index criteria by the local community and the services sector criteria corporate priorities community priorities combined priorities employment 0.323 0.200 0.25 production efficiency 0.199 0.251 0.22 environment 0.200 0.174 0.19 legislation 0.171 0.168 0.17 social development 0.107 0.207 0.15 model name: banks hospitals hotels modified asma numerical assessment legislation social development 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 compare the relative importance with respect to: goal: the best company legislation social developmentemploymentenvironmentproduction efficiency legislation 1.76133 (2.40593) (1.20133) 1.01895 social development (2.62324) (1.61653) (2.26973) employment 1.58118 1.48335 environment 1.12709 production efficiency incon: 0.01 page 1 of 114/3/14 23:31:15 ambamb ijahp article: bahurmoz/measuring corporate social responsibility performance: a comprehensive ahp based index international journal of the analytic hierarchy process 28 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.608 table 3 global priorities as judged by all stakeholders corporate sector and local community sub criteria corporate priorities public priorities combined priorities occupational health and safety 0.119 0.077 0.096 intelligent giving (investments) 0.049 0.130 0.080 product quality, integrity 0.073 0.081 0.077 anti-corruption strategies 0.048 0.094 0.067 efficient use of resources 0.074 0.060 0.067 public awareness (communications) 0.058 0.076 0.066 material and moral rights 0.091 0.036 0.057 anti-pollution efforts 0.052 0.063 0.057 fair opportunities in recruitment 0.061 0.049 0.055 environmental considerations 0.054 0.051 0.053 innovation and development strategies 0.047 0.050 0.049 career development 0.052 0.044 0.048 equity 0.055 0.030 0.041 crm 0.027 0.054 0.038 cost of production 0.033 0.036 0.035 stakeholder involvement 0.037 0.025 0.030 standards suppliers 0.020 0.030 0.025 nationalisation 0.030 0.019 0.024 4.2 implementing the developed csr index an intentional sample of the private sector corporations is selected to represent the alternatives for testing the proposed csr index. forty-two corporations were selected mostly from banks, hospitals and hotels. interviews were conducted to collect information about their csr practice. table 4 summary of the rubric and intensity of its levels corporate performance level level intensity priority corporate does not address csr concept in its management practice. 0 .01 top management has the intension but nothing has been done 1 .07 csr is addressed in its strategic plan, systems are developed. 2 .19 csr is addressed in its strategic plan, systems are developed and in the process of implementation 3 .80 fully committed to csr practice. an annual report is publicly published 4 1.00 ijahp article: bahurmoz/measuring corporate social responsibility performance: a comprehensive ahp based index international journal of the analytic hierarchy process 29 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.608 in order to systemize the interviews, a rubric was developed. for each sub-criteria a set of questions was designed to address four levels of performance: leadership, systems, implementation and achievement. a fifth level (none) was added; a value of (0) was assigned wherever a corporation did not address that specific criterion or was not practicing the csr concept in managing its business, (table 4). the resulting rubric from the interview was converted into numbers and intensity priorities were developed (figure 3). the rating model of the ahp is implemented to rank the performance of the 42 surveyed corporations. figure 3 intensity priorities of the level of csr practiced by an alternative corporation 5. data analysis findings show that corporations from the service sector give top priority to employment (table 2). this may reflect the companies’ response to the current drive towards job nationalization by the government. currently, companies are under pressure to develop tangible policies to attract the indigenous population to join the private sector. it is worth noting that traditionally saudis prefer to work for the public sector. statistics show that less than 15% of saudis work in the private sector and almost 85% are employed in the public sector, while the underlying unemployment exceeds 10% (general authorities of saudi statistics, 2017). not surprisingly, employment is a top priority for the local community. this conclusion is reinforced when the combined priorities of the two groups of csr experts, corporate and community, are taken into account. the combined results also rank employment as the first csr priority. the second top criterion is production efficiency as it is vital for corporate sustainability. it is worth pointing out that since the community sample was mostly drawn from the faculty and students in the school of business, it is only natural to see production efficiency come first in the community priorities and second when their priorities are combined with the corporate priorities. global priorities of all sub-criteria in table 3 show that occupational health and safety, intelligent giving, product quality and integrity and anti-corruption strategies rank high, and when they are combined score slightly over 33% of the global priorities. when examining the global priorities for all of the csr index sub-criteria, it is interesting to observe that although priorities vary between community and corporate judgements, the ranking of the csr criteria does not change dramatically. this makes a compelling case for the corporate sector to meet its csr commitments as determined by its own judgements. ijahp article: bahurmoz/measuring corporate social responsibility performance: a comprehensive ahp based index international journal of the analytic hierarchy process 30 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.608 the resulting index was implemented on 42 local corporations mostly from the service sector. members of top management or csr managers were interviewed. the interview rubric that resulted was converted into numbers according to table 4. a summary of corporate performance is given in figure 4, where series 1 stands for level 4 i.e. complete csr performance and series 5 stands for no performance at all. figure 5 exhibits the corporate performance with respect to the top two criteria, namely, employment and product efficiency. the results shows that 75% of the surveyed corporations are at levels beyond merely intention, 53% of the total 42 corporations are fully committed to the employment criterion and 44% are fully committed to the production efficiency criterion. these results emphasize that corporations are committed to their own judgments when their judgments are elicited to prioritize index criteria and when preferences of other stakeholders (community) are taken into account. meanwhile, it reflects the robust design of the proposed index. figure 6 presents the ranking of the surveyed corporations with respect to their csr performance reported by the interviews and rated against the criteria prioritized by the stakeholders. names of surveyed corporations are hidden for confidentiality purpose. figure 4 csr corporate performance 0 5 10 15 20 25 30 35 # o f a lt e rn a ti ve s index subcriteria csr corporate performance series1 series2 series3 series4 series5 ijahp article: bahurmoz/measuring corporate social responsibility performance: a comprehensive ahp based index international journal of the analytic hierarchy process 31 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.608 figure 5 percentage of corporate performance with employment and productivity figure 6 ranking the surveyed corporations according to their csr performance as measured by the proposed ahp based index level 4: 44.25% level 3: 33.75% level 2: 13.5% level4: 53.2% level 3: 15.6% level 2: 6.8% level 1&0: 24.4% persentage of alternatives performance with respect to employment&productivity criteria production efficiency employment ijahp article: bahurmoz/measuring corporate social responsibility performance: a comprehensive ahp based index international journal of the analytic hierarchy process 32 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.608 6. managerial implications of using ahp based csr index 1. the ahp permits a holistic approach whereby corporate csr judgements can be ranked and integrated with judgements from the greater society. thus, a csr index constructed in this manner integrates what the wider society expects from the corporate sector, on the one hand, and what the corporate sector perceives as its own prime csr priority, on the other hand. 2. the ahp ensures that the judgements of all stakeholders are taken into account and are prioritized in a scientific and transparent manner. stakeholder’s judgements can be assigned different weights according to certain criteria such as faculty versus students or top management employees versus third line employees (saaty, 2001). however, such categorization is not applicable to this work. all stakeholders’ judgements are weighted equally. 3. the ahp model helps to determine the comparative strength of the corporate responsibility program. with the number of ratings and rankings in existence, it can be overwhelming to determine which to pursue. the ahp methodology makes this comparative analysis and ranking of priorities relatively easier without compromising the robustness of the model and/or the scientific basis of the model. 4. the ahp model does not require measurement precision for an alternative, which is usually not available in the practice of csr. rather than applying the ahp methodology, the emphasis shifts to the criteria used to weigh and synthesize the measurement of the alternatives as they have a greater impact on the outcome. 5. the ahp-based csr index has a distinct advantage over other indices as it is not only dependent on data as it merges subjective priorities of the stakeholders with objective data provided by the corporations. 6. the ahp-based csr index developed in this study for evaluating corporate csr performance restores trust between the corporate sector and the greater society. however, further research to explore variations across organizations within a sector or across sectors would help uncover the underlying motivations that determine and explain the csr decision-making process. 7. having such a compact csr index would facilitate corporations in formulating a balanced strategy as well as help the greater community evaluate the performances of corporations with respect to their declared csr commitments. furthermore, government institutions can use it to evaluate and compare csr corporate performance. 8. identifying csr obligations and committing to them in a transparent manner would support a sustainable csr-based business environment where sustainability is defined as conducting business without endangering the activities of future generations. this is supported by the underlying thesis of the ahp technique as its inherent ranking process promotes awareness and encourages ownership of the process. 9. the ahp methodology is not overly complex. it legitimately aggregates across scales and addresses consistency in judgements from multiple participants. it also formalizes the selection process, reduces time commitments, creates a process-oriented selection method ijahp article: bahurmoz/measuring corporate social responsibility performance: a comprehensive ahp based index international journal of the analytic hierarchy process 33 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.608 and results in better selection of csr indicators (bahurmoz et al, 2015). furthermore, it can be replicated in similar situations and ultimately reduces costs and effort related to the selection process and to the occurrence of selection errors post-selection. 7. conclusions very few studies have attempted to formulate a framework of analysis that systematically documents or prioritizes csr practices. this research set out to construct a comprehensive csr index that aimed to understand and analyze csr practices within saudi corporations. it takes into account stakeholders’ judgments and facilitates meaningful rankings and comparisons of their csr priorities. most previous csr literature fell short in addressing this issue. given the shortcomings of the existing indices, the study proposed and constructed a csr index based on the ahp methodology. expert judgements were collected through a survey of people from the corporate sector as well as the wider community. the proposed index has been verified by implementing it in the same business environment. local corporations in the service sector were investigated and their performance was evaluated and finally they were rated by the proposed index. extant csr studies are usually one-dimensional, tending to focus on environmental and community issues and using secondary data sources, both of which are considered shortcomings. this research overcomes these shortcomings by collecting original data and by using the ahp model that extends beyond the restrictions of previous approaches. the ahp makes it possible to analyze csr practices in a multi-dimensional context. the findings demonstrate that saudi companies do not view csr mainly in terms of philanthropy given that employment and production efficiency emerge as the most highly ranked corporate priorities. this confirms the view that saudi businesses are moving towards adopting csr practices as part of their corporate strategy. this further highlights the need for a robust csr index for the saudi corporate sector. using an ahp based index helped analyze csr practice in a multi-dimensional fashion and identify csr obligations in a transparent manner. the proposed model can be easily replicated or modified in similar situations not only inside saudi arabia but in other countries around the world. furthermore, the findings contribute to facilitating csr best practices across borders. ijahp article: bahurmoz/measuring corporate social responsibility performance: a comprehensive ahp based index international journal of the analytic hierarchy process 34 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.608 references bahurmoz a, mukhtar, al-sharqi l. (2015). ahp as an effective consensus-based selection tool: a case of personnel selection for the ministry of foreign affairs in saudi arabia. journal for global business advancement, 8(2), 138-156. doi: https://dx.doi.org/10.1504/jgba.2015.069527 barić a (2017). corporate social responsibility and stakeholders: review of the last decade (2006-2015). business systems research 8 (1), 133-146. doi: https://doi.org/10.1515/bsrj-2017-0011 bowen h r (1953). social responsibilities of the businessman. new york, harper & row. doi: 10.2307/j.ctt20q1w8f bowen eh & haire m (1975). a strategic posture towards corporate social responsibility. california management review, 18, 49-58. doi: https://doi.org/10.2307%2f41164638 carroll, ab (1979). a three-dimensional concept model of corporate performance, academy of management review, 4(4), 497-505. doi: 10.2307/257850 chen x, huang r, yang z, dube l. (2018). csr types and the moderating role of corporate competence. european journal of marketing, 52 (7/8), 1358-1386. doi: https://doi.org/10.1108/ejm-12-2016-0702 commission of the european communities. (2001). promoting a european framework for corporate social responsibility. com, 366, brussels. doi: http://dx.doi.org/10.5040/9781472559425.ch-005 costa, menichini (2013). a multidimensional approach for csr assessment: the importance of the stakeholder perception. expert systems with applications, 40(1), 150– 161. doi: https://doi.org/10.1016/j.eswa.2012.07.028 devinney t. (2009). is the socially responsible corporation a myth? the good, the bad, and the ugly of corporate social responsibility. academy of management perspectives, 23, 44-56. doi: https://doi.org/10.5465/amp.2009.39985540 fontaine m. (2013). corporate social responsibility and sustainability: the new bottom line. international journal of business and social science, 4(40), 110-119. general authority of statistics, kingdom of saudi arabia: https://www.stats.gov.sa/en/820 hemat h, yuksel u (2014). a critical review of corporate social responsibility practices from a marketing perspective: is cause-related marketing really a ‘win–win– win’ situation. corporate social responsibility in the global business world. berlin, heidelberg: springer. doi: https://doi.org/10.1007/978-3-642-37620-7_1 ijahp article: bahurmoz/measuring corporate social responsibility performance: a comprehensive ahp based index international journal of the analytic hierarchy process 35 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.608 hoffman r c (2007). corporate social responsibility in the 1920s: an institutional perspective. journal of management history, 13(1), 55-65. doi: https://doi.org/10.1108/17511340710715179 indre slapikaite (2016). practical application of csr complex evaluation system. intellectual economics, 10, 101–107. doi: https://doi.org/10.1016/j.intele.2017.03.002 govindan, k., shankar, m., kannan, d. (2018). supplier selection based on corporate social responsibility practices. international journal of production economics, 200, 353379. doi: https://doi.org/10.1016/j.ijpe.2016.09.003 kèyù zhü (2014). fuzzy analytic hierarchy process: fallacy of the popular methods, european journal of operational research, 236(1), 209-217. doi: https://doi.org/10.1016/j.ejor.2013.10.034 luning, s. (2012. corporate social responsibility (csr) for exploration: consultants, companies and communities in processes of engagements. resources policy, 37(2), 205211. doi: https://doi.org/10.1016/j.resourpol.2011.02.004 mcwilliams a, siegel ds, wright pm (2006). corporate social responsibility: strategic implications. journal of management studies, 43(1), 1–18. mosgaller t (2012). putting social responsibility into practice. the journal for quality & participation, cincinnati, 34(2), 39-41. öberseder, m., schlegelmilch, bb., murphy, pe. (2013). csr practices and consumer perceptions. journal of business research, 66 (10), 1839–1851. doi: https://doi.org/10.1016/j.jbusres.2013.02.005 pérez, a. & del bosque, r. (2013). measuring csr image: three studies to develop and to validate a reliable measurement tool. international journal of business ethics, 118 (2), 265-286. doi: https://doi.org/10.1007/s10551-012-1588-8 saaty, t. (2001). decision making for leaders: the analytic hierarchy process for decisions in a complex world, new edition. pittsburgh, pa: rws publications. saaty, t., peniwati, k. (2008). group decision-making: drawing out and reconciling differences. pittsburgh pa: rws publications. saaty, t. (2008). decision making with the analytic hierarchy process. international journal of services sciences, 1(1), 83-98. doi: https://dx.doi.org/10.1504/ijssci.2008.017590 saaty, t. (2011). on the measurement of intangibles. a principal eigenvector approach to relative measurement derived from paired comparisons. notices of the ams, 60(2), 192208. doi: http://dx.doi.org/10.1090/noti944 https://www.sciencedirect.com/science/journal/03772217 https://www.sciencedirect.com/science/journal/03772217/236/1 https://econpapers.repec.org/article/eeejrpoli/ ijahp article: bahurmoz/measuring corporate social responsibility performance: a comprehensive ahp based index international journal of the analytic hierarchy process 36 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.608 tafti, hosseini, emami (2012). assessment the corporate social responsibility according to islamic values (case study: sarmayeh bank) procedia social and behavioral sciences, 58, 1139-1148. doi: https://doi.org/10.1016/j.sbspro.2012.09.1095 tead, o., metcalf, h. (1933). personnel administration: its principles and practice, 2nd ed., new york, ny: mcgraw-hill. trapp, n. (2014) .stakeholder involvement in csr strategy making? clues from sixteen danish companies. public relations review, 40 (1), 42–49. doi: https://doi.org/10.1016/j.pubrev.2013.11.005 tsourvakas g, yfantidou, i. (2018). corporate social responsibility influences employee engagement. social responsibility journal, 14(1), 123-137. doi: https://doi.org/10.1108/srj-09-2016-0153 wang, y. (2011). corporate social responsibility and stock performance—evidence from taiwan. modern economy, 2, 788-799. doi: 10.4236/me.2011.25087 https://www.stats.gov.sa/en/820 http://www.corporate-citizenship.com/knowledge-series/navigating-djsi/ http://www.ftse.com/indices/ftse4good_index_series/downloads/f4g_criteria.pdf http://www.aha.org/research/reports/13econimpact.shtml http://www.asianhhm.com/healthcare_management/corporate_privatehospitals.htm http://www.cgg.com http://www.google.com.sa/url?sa=t&rct=j&q=&esrc=s&frm=1&source=web&cd=3&cad =rja&ved=0cdwqfjac&url=http%3a%2f%2fwww.cgg.com%2fdata%2f%2f1%2fr ec_docs%2f1683_aspi_factsheet_201009pdf.pdf&ei=lmkuuzzaoihnoe3pgbab&us g=afqjcng4vxnaxkrbzdofducgjhvzqu_mzg http://csrmiddleeast.org/profiles/blogs/the-good-hospital-guide-a-sustainability-guide-forhospitals-by http://www.ourcommunity.com.au/business/business_main.jsp iso 26000, www.iso.org/iso/home/standards/iso26000.htm net balance foundation, www.newbalancefoundation.org https://www.sciencedirect.com/science/journal/18770428 https://www.sciencedirect.com/science/journal/18770428 https://www.sciencedirect.com/science/journal/18770428/58/supp/c https://www.emeraldinsight.com/author/tsourvakas%2c+georgios https://www.stats.gov.sa/en/820 http://www.corporate-citizenship.com/knowledge-series/navigating-djsi/ http://www.ftse.com/indices/ftse4good_index_series/downloads/f4g_criteria.pdf http://www.aha.org/research/reports/13econimpact.shtml http://www.asianhhm.com/healthcare_management/corporate_privatehospitals.htm http://www.cgg.com/ http://www.google.com.sa/url?sa=t&rct=j&q=&esrc=s&frm=1&source=web&cd=3&cad=rja&ved=0cdwqfjac&url=http%3a%2f%2fwww.cgg.com%2fdata%2f%2f1%2frec_docs%2f1683_aspi_factsheet_201009pdf.pdf&ei=lmkuuzzaoihnoe3pgbab&usg=afqjcng4vxnaxkrbzdofducgjhvzqu_mzg http://www.google.com.sa/url?sa=t&rct=j&q=&esrc=s&frm=1&source=web&cd=3&cad=rja&ved=0cdwqfjac&url=http%3a%2f%2fwww.cgg.com%2fdata%2f%2f1%2frec_docs%2f1683_aspi_factsheet_201009pdf.pdf&ei=lmkuuzzaoihnoe3pgbab&usg=afqjcng4vxnaxkrbzdofducgjhvzqu_mzg http://www.google.com.sa/url?sa=t&rct=j&q=&esrc=s&frm=1&source=web&cd=3&cad=rja&ved=0cdwqfjac&url=http%3a%2f%2fwww.cgg.com%2fdata%2f%2f1%2frec_docs%2f1683_aspi_factsheet_201009pdf.pdf&ei=lmkuuzzaoihnoe3pgbab&usg=afqjcng4vxnaxkrbzdofducgjhvzqu_mzg http://www.google.com.sa/url?sa=t&rct=j&q=&esrc=s&frm=1&source=web&cd=3&cad=rja&ved=0cdwqfjac&url=http%3a%2f%2fwww.cgg.com%2fdata%2f%2f1%2frec_docs%2f1683_aspi_factsheet_201009pdf.pdf&ei=lmkuuzzaoihnoe3pgbab&usg=afqjcng4vxnaxkrbzdofducgjhvzqu_mzg http://csrmiddleeast.org/profiles/blogs/the-good-hospital-guide-a-sustainability-guide-for-hospitals-by http://csrmiddleeast.org/profiles/blogs/the-good-hospital-guide-a-sustainability-guide-for-hospitals-by ijahp article: bahurmoz/measuring corporate social responsibility performance: a comprehensive ahp based index international journal of the analytic hierarchy process 37 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.608 saudi rci (sarci) http://csrmiddleeast.org/video/saudi-responsible-competitivenessindex-rci s&p esg india index http://mena.spindices.com/documents/methodologies/methodology-sp-esg-indiaindex.pdf?force_download=true s&p/egx esg index http://www.standardandpoors.com/indices/articles/en/us/?articletype=pdf&assetid=12 45207095955 http://csrmiddleeast.org/video/saudi-responsible-competitiveness-index-rci http://csrmiddleeast.org/video/saudi-responsible-competitiveness-index-rci http://mena.spindices.com/documents/methodologies/methodology-sp-esg-india-index.pdf?force_download=true http://mena.spindices.com/documents/methodologies/methodology-sp-esg-india-index.pdf?force_download=true http://www.standardandpoors.com/indices/articles/en/us/?articletype=pdf&assetid=1245207095955 http://www.standardandpoors.com/indices/articles/en/us/?articletype=pdf&assetid=1245207095955 ijahp article: bahurmoz/measuring corporate social responsibility performance: a comprehensive ahp based index international journal of the analytic hierarchy process 38 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.608 appendix csr indices worldwide 1. the dow jones sustainability index (djsi), as the first set of global sustainability indices, was created in 1999. it recognized companies for their outstanding economic, environmental and social performances. the screening standards of the djsi world were defined to reflect the carport's contribution to the economy, the society, and the environment. in may 2013, s&p dow jones indices and robeco sam launched a new range of diversified sustainable indices. the eight new indices target investors who measure performance against standard benchmarks but want to add sustainable companies to their portfolios. in 2013, dow jones launched the dow jones sustainability emerging markets, the first index to measure sustainability performance from emerging markets. the index has a market capitalization of $680 billion and evaluates sustainability performance based on the esg criteria. other notable indices launched in the us include the launch of the thomson reuter corporate responsibility indices developed in conjunction with s-network global index. these indices rate companies’ csr investments through an assessment of their esg practices (http://www.corporate-citizenship.com) 2. the london stock exchange created the “financial times stock exchange for good index series (ftse4good)” in 2001. it consists of global firms dedicated to a sustainable environment, corporate governance, and international human rights. ftse4good provides a tool for responsible investors to identify and invest in companies that meet globally recognized corporate responsibility standards, and it contributes to the development of responsible business practices around the world. the index concentrates on environmental and human rights criteria in addition to supply chain labor standards, countering bribery and climate change criteria. in 2009, the uk-based social stock exchange (sse) was launched, and in 2010, companies that used more than 6,000 mwh per year were to start reporting on all emissions related to energy use (http://www.ftse.com/indices/ftse4good_index_series) 3. the advanced sustainability performance eurozone index (aspi eurozone®) is recognized as one of the leading sustainability indices. it is used by a growing community of responsible investors to define sustainable investment universes, to benchmark their investment performances and to create index-linked products. it consists of six main criteria: environment, community involvement, human rights, business behaviour, human resources and corporate governance. in 2013, the s&p nordic low volatility index was created from a selection of the 30 least volatile stocks on the s&p nordic broad market index, while the nyse euronext and vigeo partnered to create a range of indices that focus on esg ijahp article: bahurmoz/measuring corporate social responsibility performance: a comprehensive ahp based index international journal of the analytic hierarchy process 39 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.608 issues. these indices include the major listed companies in europe, the asiapacific region, and north america. 4. the morningstar socially responsible investment index (ms-sri) was developed in 2003 by morningstar japan k.k. it is japan’s first stock price index to focus on csr. morningstar japan selects the top 150 publicly listed companies with respect to their csr activities and calculates an index based on stock prices. it is based on five criteria, namely, governance, accountability, markets, working environment, and social contributions. in 2009, environmental etf japan green chip 35 (1347) was launched. 5. the s&p esg india index provides investors with exposure to 50 of the best performing stocks in the indian market as measured by environmental, social, and governance (esg) factors. the index represents the first of its type to measure esg practices based on financial rules and environmental and social criteria. in 2012, the bombay stock exchange (bse) launched the bse carbonex, the first carbon-based thematic index in the country. it tracks the performance of the constituent companies of the bse-100 index and their commitment to greenhouse gas emissions reduction. bse also launched its green index. more recently, the indian institution of corporate affairs (iica) and the bombay stock exchange ltd (bse) collaborated to develop a corporate social responsibility (csr) index. the proposed iica-bse csr index will assess the impact and performance of companies listed on the bse with respect to their csr activities. the index will also examine the performances of companies regarding their mandatory csr spent as per the new companies act 2013 as one of the important and objective criteria (http://www.mena.spindices.com) 6. the s&p/egx esg index for egypt was created by the index egyptian institute of directors, s&p indices and crisil. the purpose of the index is to raise the profile of those companies that perform well with respect to their environmental, social and corporate governance responsibility when compared to their market peers registered on the egyptian stock market (http://www.standardandpoors.com) 7. the saudi responsible competitiveness index (sarci, htttp://csrmiddleeast.org) was developed by the sagia and the king khaled foundation and accountability in 2008. it aims to promote good csr practices within the saudi corporate sector. participation in the index is voluntary for companies. 8. in canada, the jantzi social index was launched in 2000, and in 2007, the ishares launched a socially responsible etf. ijahp article: bahurmoz/measuring corporate social responsibility performance: a comprehensive ahp based index international journal of the analytic hierarchy process 40 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.608 9. following north america and the u.k, in 2001 the corporation act in australia required the disclosure of violations of environmental legislation in listed companies. in 2010, australia introduced its new ethical disclosure requirements under the financial services reform act (fsra) annual reports whereby companies listed on the australian stock exchange (asx) must disclose whether they have developed a code of conduct on environmental risks and controls (http://www.ourcommunity.com.au) 10. in germany, deutsche borse established the dax global alternative energy index, which includes international companies whose revenues are based on technology and services designed to promote and generate alternative energy sources in an effort to highlight growth trends towards alternative energy. in 2007, deutsche borse established the daxglobal sarasin sustainability germany index and the daxglobal sarasin sustainability switzerland index, which follow companies that meet the sustainability requirements of the sarasin sustainability matrix. the german council for sustainable development (gcsd) developed a german sustainability code in 2011 that includes 20 criteria and 27 gri performance indicators. 11. the bolsa de madrid exchange (bme) sustainability-related investment indexftse4goodibex was launched in spain. 12. the swedish stock exchange (omx) launched the omx ges nordic sustainability index in 2008. 13. in 2009, the warsaw stock exchange launched the region’s first stock index of responsible companies in central and eastern europe. in 2012, the warsaw stock exchange launched the respect index, which lists companies with a high reporting quality and an advanced level of investor relations or information governance. 14. in 2005, bm&f bovespa created the ico2 carbon efficient index in brazil. in december 2010, bovespa and development bank bndes launched the ico2 carbon efficient index at the united nations climate talks in cancun, mexico. 15. in 2011, bolsa mexicana de valores launched a sustainability index in mexico. 16. the sse and china securities index company launched the sse social responsibility index in 2009, while the hang seng corporate sustainability index series was launched in 2010. the shanghai stock exchange launched a new environmental protection industry index in september 2012. the new index screens for stocks that obtain more than 25% of their revenue from resource management, clean technology, or pollution management. ijahp article: bahurmoz/measuring corporate social responsibility performance: a comprehensive ahp based index international journal of the analytic hierarchy process 41 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.608 17. in 2012, the taiwan stock exchange (twse) launched an index that focuses on corporate governance and corporate social responsibility. 18. the keji index in korea, introduced in 1991, was the first comprehensive evaluation scheme for corporate business ethics and social responsibility developed and implemented in korea. each year, the keji selects annual economic justice award winners based on quantitative and qualitative evaluations. one of the distinctive features of the keji index is that it is a product of an independent rating service. it focuses on the evaluation of multidimensional corporate social performances and yields a score on seven individual categories of csr: soundness, fairness, contribution to society, consumer protection, environmental protection, employee satisfaction, and contribution to economy (choi et al. 2010) 19. the oww responsibility™ malaysia sri index, which was launched in 2006, aims to help the sri community access up-to-date information on the social performance of companies in malaysia and singapore and to open up the malaysian market to socially responsible investors. in 2012, bursa malaysia (malaysia's stock exchange) launched its environmental, social and corporate governance (esg) index to attract more socially responsible investment (sri) funds to malaysia and to raise the profile of malaysia's listed companies that perform well on the esg indicators compared to their peers. 20. the kehati-sri index was launched in 2009 to track indonesian corporations with sustainable business practices. 21. in 2010, the istanbul stock exchange sustainability index (ise si) was launched. 22. in 2004, the johannesburg stock exchange (jse) launched its sri index of companies. in 2012, jse announced that more than 70% of the listed companies met the base requirements to become constituents of the 2012 socially responsible index. ijahp article: shrestha, acharya, karmacharya, sapkota, paudyal, basnet, timilsina, shakya/application of ahp and gis for optimal solar site identification in madhesh province, nepal international journal of the analytic hierarchy process 1 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.986 application of ahp and gis for optimal solar site identification in madhesh province, nepal ravi raj shrestha *1 nea engineering company, nepal raviraj.shrestha@gmail.com anushilan acharya graduate school of environmental science, hokkaido university, japan anushilanacharya@gmail.com niroj karmacharya nea engineering company, nepal nirojkarma@gmail.com mira sapkota nea engineering company, nepal sapkotameera5@gmail.com binay paudyal nea engineering company, nepal vinaypaudyal@gmail.com prasant basnet nea engineering company, nepal basnetprashant691@gmail.com sushil timilsina nea engineering company, nepal gyanshil07@gmail.com hitendra dev shakya nepal electricity authority, nepal hitendradev@hotmail.com abstract solar-based renewable energy adoption is in its early stage in the power system of nepal complying with its commitment to carbon neutrality. the government of nepal has declared a goal of setting up solar power plants of at least 200 mw in madhesh province, but the selection of optimal sites will be the decisive factor in achieving this goal. 1* corresponding author mailto:raviraj.shrestha@gmail.com mailto:anushilanacharya@gmail.com mailto:nirojkarma@gmail.com mailto:sapkotameera5@gmail.com mailto:vinaypaudyal@gmail.com mailto:basnetprashant691@gmail.com mailto:gyanshil07@gmail.com mailto:hitendradev@hotmail.com ijahp article: shrestha, acharya, karmacharya, sapkota, paudyal, basnet, timilsina, shakya/application of ahp and gis for optimal solar site identification in madhesh province, nepal international journal of the analytic hierarchy process 2 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.986 therefore, rigorous investigation is necessary for optimal site selection. a three-step framework combining the analytic hierarchy process (ahp) and the geographical information system (gis) has been adopted to identify the optimal location for solar power plant installation in madhesh province. the framework includes the creation of weighted individual raster images for the different criteria, images of restricted areas for solar installation, and a combination of all the rasters into a single raster using gis-based software. the weights of the criteria and sub-criteria have been evaluated using the ahp model, which is an expert judgments-based model. the criteria considered were solar irradiance, annual mean temperature, distance from the road, distance from the substation, distance from the urban area, elevation, aspect, and land use. the results were graded from the least preferred area to the most highly suitable area. the most highly suitable areas were in the saptari, siraha, dhanusa, and mahottari districts, while the districts in the western region of the province had less suitable areas. thus, the province’s eastern region is most suitable for installing solar power plants. keywords: ahp; gis; renewable energy; solar power plant 1. introduction with increasing concern about the detrimental impacts of fossil fuels on the sustainability of the environment, focus has shifted towards cleaner sources of energy with solar power being the most promising candidate among all the available resources. countries around the globe have formulated their policies to promote the integration of solar power into their power network (international energy agency, 2021). the international energy agency (2021) estimates that global solar power generation in 2020 has increased by an astounding value of 23% from the previous year with an annual generation of 820 twh and is estimated to reach 6970 twh by 2030. in the nepalese context, solar power is making its way into the hydro-dominated power system. historically, the energy demands of the country have been fulfilled by hydropower and imports from the neighboring country of india. no significant changes in the energy mix have yet been achieved. most of the hydropower plants in nepal are run of river (ror) types whose capacity reduces by more than 50% during the dry season. the resulting deficit of energy is fulfilled by heavy imports from the indian power market. accounting for these factors, the government has proposed the introduction of alternative sources of energy into the energy mix of the country. the current installed capacity of the grid-connected solar power plant is around 54.6 mw, accounting for less than 2.5% of the total installed capacity, while 1239.69 mw have received a survey license, 137.56 mw have received a construction license, 197.4 mw have applied for a survey license and 15 mw have applied for a generation license, respectively (department of electricity development, 2022; nepal electricity authority, 2022). most of the solar power plants that have been installed or are in different phases of being established are located in the northern belt of the country. the government of nepal has formulated a plan to establish a minimum of 200 mw solar power plants in nepal (water and energy commission secretariat, 2010). these data indicate the aggressive approach that has been adopted to promote solar power plants in nepal. with increasing interest and investment in solar photovoltaics, it is necessary to have an overview for developers during the initial stage of planning for suitable sites to establish a solar power plant. there have been limited previous studies for optimal solar site identification in the nepalese context. this study was conducted to fill in the missing piece of the puzzle. the ijahp article: shrestha, acharya, karmacharya, sapkota, paudyal, basnet, timilsina, shakya/application of ahp and gis for optimal solar site identification in madhesh province, nepal international journal of the analytic hierarchy process 3 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.986 ahp and gis have been used to create a suitability map displaying the best possible sites for solar power installation. a proper site for plant location must not be decided solely based on the level of solar irradiation available in the area. a single criterion is insufficient to make a decision; multiple criteria must be considered for site selection. 2. literature review the ahp can be applied for tasks such as planning of renewable energy, allocation of energy resources, management of building energy, and planning of electricity utilities (xin, chen, yang, miao & li, 2019; ahmad & tahar, 2014). it has been applied in the determination of installation sites for power plants. studies have combined ahp, one of the methods within multicriteria decision analysis (mcda), and geographical information system (gis) to determine the location of solar power plant installation (choi, suh, & kim, 2019). different methods of mcda, namely, ahp, electre, topsis, and vikor have been applied to determine the optimal location of solar power plants in turkey with three different criteria, i.e., solar irradiation, surface slope, and feeder capacity (akkas, erten, cam, & inanc, 2017). watson and hudson (2015) conducted similar work in southern england using a combination of ahp and geospatial data constrained to solar radiation, distance from residential areas, distance from wildlife, distance from transportation link and distance from network connections. the ahp in combination with gis has provided the optimal locations for concentrated solar power plants in west africa and morocco with solar irradiation, slope, distance from the road, and distance from the urban area as the criteria set forth for ahp (yushchenko, de bono, chatenoux, patel & ray, 2018; tahri, hakdaoui & maanan, 2015). sánchez-lozano, teruel-solano, soto-elvira & socorro garcía-cascales (2013) considered two different methods of mcda combined with gis to evaluate the location of solar power in southeastern spain. a case study was conducted in mauritius to determine the location of solar power plants using the ahp and gis with proximity to transmission lines and roads, slope, elevation, aspect, global solar radiation, sunshine duration, air temperature, and relative humidity as the evaluation criteria (doorga, rughooputh, & boojhawon, 2019). a site suitability assessment for west kalimantan province, indonesia merging ahp and gis constrained to global horizontal irradiation (ghi), temperature, relative humidity, elevation, slope, aspect, proximity to power grid, road infrastructure, and major settlements concluded that only 0.03-0.07% of the total area of the particular province was optimal for solar plant installation (ruiz, sunarso, ibrahim-bathis, murti, & budiarto, 2020). likewise, nebey, taye & workineh (2020) investigated the site suitability for south gondar zone, ethiopia using a blend of ahp and gis with similar criteria considered by ruiz, sunarso, ibrahim-bathis and budiarto (2020) for indonesia and concluded that 86% of the total land within the south gondar zone was suitable for solar plant installation. a similar strategy was applied for mongolia (munkhbat & choi, 2021), saudi arabia (al garni & awasthi, 2017), thailand (ali, taweekun, techato, waewsak, & gyawali, 2019), and tanzania (aly, jensen, & pedersen, 2017) using the ahp and gis with similar criteria considered by the studies mentioned above. only one study found in the available literature has been conducted in the nepalese context. the solar sites around kathmandu valley, the capital city of nepal have been identified using ahp and gis (paneru, 2016) with slope, aspect, and, proximity to the road as the evaluation criteria. ijahp article: shrestha, acharya, karmacharya, sapkota, paudyal, basnet, timilsina, shakya/application of ahp and gis for optimal solar site identification in madhesh province, nepal international journal of the analytic hierarchy process 4 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.986 table 1 provides a summary of the literature survey considered for the application of the ahp and gis for optimal site identification for solar plants. table 1 summary of literature considered for a combination of ahp and gis for optimal solar site identification no. mcda technique location reference 1 ahp turkey (akkas, erten, cam, & inanc, 2017) 2 ahp southern england (watson & hudson, 2015) 3 ahp west africa (yushchenko, de bono, chatenoux, patel, & ray, 2018) 4 ahp morocco (tahri, hakdaoui, & maanan, 2015) 5 ahp south eastern europe (sánchez-lozano, teruel-solano, soto-elvira, & socorro garcíacascales, 2013) 6 ahp mauritius (doorga, rughooputh, & boojhawon, 2019) 7 ahp indonesia (ruiz, sunarso, ibrahim-bathis, murti, & budiarto, 2020) 8 ahp ethiopia (nebey, taye, & workineh, 2020) 9 ahp mongolia (munkhbat & choi, 2021) 10 ahp saudi arabia (al garni & awasthi, 2017) 11 ahp thailand (ali, taweekun, techato, waewsak, & gyawali, 2019) 12 ahp nepal (paneru, 2016) 13 ahp tanzania (aly, jensen, & pedersen, 2017) 3. material and methods 3.1 study area nepal is a landlocked country that lies between 26° and 31°n latitude, and 80° and 89°e longitude. nepal stretches to a length of about eight hundred kilometers and a breadth of two hundred kilometers with an area of 147,516 km2. terai is the southern belt of nepal distributed over the entire length of the country and occupies 56% of the total area (timilsina & tiwari, 2019). nepal has an estimated potential of 2100 mw of solar power generation as it has an average solar radiation varying from 3.6-6.2kwh/m2day with 300 days of sunshine within a year (water and energy commission secretariat, 2010). figure 1 shows the solar power potential of nepal (the world bank, 2017). ijahp article: shrestha, acharya, karmacharya, sapkota, paudyal, basnet, timilsina, shakya/application of ahp and gis for optimal solar site identification in madhesh province, nepal international journal of the analytic hierarchy process 5 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.986 energy consumption in nepal grew at an average annual growth rate of 10% during the last decade with an annual consumption of 8851gwh in the fiscal year 2020/2021 (nepal electricity authority, 2022). the total installed capacity of the integrated nepal power system is 2189.918 mw, out of which 2081.788 mw is from hydropower plants, 53.41 mw from thermal (diesel and multi-fuel), and 54.6 mw from solar. hydropower accounts for about 95% of the generation capacity in nepal, while the remaining 5% of the installed capacity is thermal and solar (nepal electricity authority, 2022). among the seven provinces, madhesh province (previously called province 2) was considered for this study. only the terai belt is within this province. the study areas lie in the southern part of the country, which receives ample solar radiation for the location of a solar power plant. madhesh province has eight districts, namely, saptari, siraha, dhanusa, mahottari, sarlahi, rautahat, bara, and parsa as depicted in figure 2. it has an area of 9,661 sq. km and occupies 6.56% of the total area of the country. madhesh province has sunny days with temperatures up to 40 degrees celsius, and in winter temperatures drop below 5 degrees celsius. madhesh has an average rainfall between 1100-2100 mm (nepal outlook, 2020). in the fiscal year 2020/21, the total energy consumed within madhesh province was 1253 gwh which was 0.9 % less than the previous fiscal year. the industries occupied the major source of consumption with 54.6 %, followed by domestic and commercial sectors with 34.45% and 3.17%, respectively. the province has no hydropower in operation, but has one 10 mw solar plant in operation (development, 2021). as per license status for madhesh province, a total of eight solar projects with a cumulative capacity of 60.7 mw have received survey licenses, four projects with a cumulative capacity of 18 mw have received a construction license, nine projects with a cumulative capacity of 109 mw have applied for a survey license and two projects with a cumulative capacity of 5 mw have applied for a generation license (department of electricity development, 2022). among the survey license applications, mahottari district has the highest number of applications with six projects of a cumulative capacity of 90 mw. figure 1 solar photovoltaic potential of nepal ijahp article: shrestha, acharya, karmacharya, sapkota, paudyal, basnet, timilsina, shakya/application of ahp and gis for optimal solar site identification in madhesh province, nepal international journal of the analytic hierarchy process 6 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.986 figure 2 location map of madhesh province in nepal 3.2 geographical information system a geographic information system (gis) creates, manages, analyzes, and maps all types of data (esri, 2021). it is an organized collection of computer hardware, software, geographic data, and personnel designed to efficiently capture, store, update, manipulate, analyze, and display all forms of geographically referenced information (kanichakra, 2018). there are several applications of gis in environment and natural resource management, street networks, land information systems, planning, and engineering. there are several studies that combine gis with multicriteria decision analysis techniques to determine the optimal location for solar plant installation (choi, suh, & kim, 2019). 3.3 analytical hierarchical process the analytical hierarchical process (ahp) was developed by saaty in 1980 (saaty, 1980), and is one of the most popular tools for multi-criteria decision analysis. the ahp helps determine the importance of different tangible and intangible factors in a multicriteria decision problem by providing a relative scale of importance in numerical value as shown in table 2. ijahp article: shrestha, acharya, karmacharya, sapkota, paudyal, basnet, timilsina, shakya/application of ahp and gis for optimal solar site identification in madhesh province, nepal international journal of the analytic hierarchy process 7 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.986 table 2 pairwise comparison scale (gašparović, i., & gašparović, 2019; saaty, 1980) importance degree definition description 1 equally preferred both the activities have equal contributions towards achieving the objective 3 moderately preferred one of the activities is slightly favored over the other. 5 strongly preferred one of the activities is strongly favored over the other. 7 very strongly preferred one of the activities is very strongly favored over the other. 9 extremely preferred one of the activities is favored over the other of the highest possible degree. 2,4,6,8 intermediate values between the degrees of importance the weight or the importance of the factor is then calculated through a pairwise decision matrix based on the relative priority between different criteria and sub-criteria. the formation of the pairwise matrix includes the following steps (munkhbat & choi, 2021; al garni & awasthi, 2017): a) for an “n” number of criteria with their preference score as per saaty (1980), a pairwise matrix “m” of size n*n is created such that if mij is the entry in the i th row and j th column, the product of mij and mji must be unity. b) then, a normalized pairwise matrix is prepared by dividing each entry of the column by the total sum of the elements of the same column. c) the average of each entry of the row provides the relative weight of the criteria. after the computation of the weights, a factor called the consistency ratio (cr) is employed to check the consistency of the ahp and the weights. it is the ratio of the consistency index and the random index. the consistency index is calculated using equation 1 as follows: cr= αmax-n n-1 (1) where, αmax is the maximum eigenvalue of the pairwise comparison matrix. the random index as defined by saaty (1980) and depends on the number of criteria being considered as shown in table 3. table 3 random index for different number of criteria n 2 3 4 5 6 7 8 9 10 11 12 ri 0 0.58 0.90 1.12 1.24 1.32 1.41 1.45 1.49 1.51 1.48 ijahp article: shrestha, acharya, karmacharya, sapkota, paudyal, basnet, timilsina, shakya/application of ahp and gis for optimal solar site identification in madhesh province, nepal international journal of the analytic hierarchy process 8 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.986 if the consistency index is less than 0.10, the results obtained are considered to be satisfactory; otherwise, recalculation must be performed due to inconsistencies in the pairwise comparison (al garni & awasthi, 2017). weights of each criterion and sub-criterion are calculated based on the decision matrix as described above. the global weight of an attribute is then determined by the product of weights of criteria and sub-sub-criteria; the total score of the weightage is determined by the summation of the global weight of each sub-criterion. 3.4 criteria selection the criteria for the selection of an optimal site for solar pv power plants must include all the factors. several studies have considered criteria for evaluation (uyan, 2013; prieto amparán, et al., 2021; albraheem & alabdulkarim, 2021; gašparović & gašparović, 2019; arán carrión, et al., 2008). they all had common criteria of slope, irradiation, land use, distance from the road, distance from an urban area, and distance from power lines. the criteria can be broadly divided into two categories, technical and socio-economic. technical criteria include solar irradiation and annual mean temperature while the socioeconomic criteria include land use/land cover, distance from main roads, distance from an urban area, distance from substations, elevation, slope, and aspects. for this study, the technical and socio-economic criteria stated above have been considered including sub-criteria. table 4 lists the criteria considered for the study along with a description. table 5 lists the personnel involved in determining the importance of each criterion for this particular study. table 4 description of criteria criteria description solar radiation sites with higher solar irradiation are preferred as they yield more energy. annual mean temperature energy production decreases as temperature increases. areas with lower temperatures are preferred. distance from roads increased distance from roads increases transportation costs. sites closer to roads are preferred. distance from substation sites near substations reduce the cost of transmission infrastructures and thus are preferred. distance from urban areas solar sites farther from an urban area are preferred as their adverse effect on urban growth is limited. elevation sites with higher elevation are preferred as they receive more solar irradiation. slope sites with smaller slopes are preferred as such sites get more solar irradiation. aspect (orientation) south-facing sites are preferred as they receive more solar irradiation. land use sites on barren land are preferred. ijahp article: shrestha, acharya, karmacharya, sapkota, paudyal, basnet, timilsina, shakya/application of ahp and gis for optimal solar site identification in madhesh province, nepal international journal of the analytic hierarchy process 9 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.986 table 5 information on personnel involved designation affiliation senior manager nea engineering company senior manager nepal electricity authority stakeholder independent power producer stakeholder independent power producer research student hokkaido university research student texas a&m university engineer nea engineering company engineer nepal electricity authority along with the criteria, some constraints or restrictions must be applied to eliminate the unsuitable areas for installation of solar power plants. the constraints as considered by albraheem and alabdulkarim (2021) were studied while those shown in table 6 have been adopted for the study. table 6 constraints for solar site selection 1 forest areas 2 bodies of water 3 flooded vegetation 3.5 data collection the raw data for this study were gathered from several sources. data for solar irradiation was derived from a repository of the global solar atlas while the temperature was purchased from the department of hydrology and meteorology, nepal. the data for the road networks were obtained from the road network of the hindu kush himalayan region created by the international centre for integrated mountain development (icimod) while mapruzin was used to obtain information regarding the buildings. land usage has been retrieved from the land use/land cover map created by esri. for slopes, distance from an urban area, elevation, slope, and land aspects, the data were derived from aster dem. the location of the substation was obtained from the nepal electricity authority. the sources of data are shown in table 7. ijahp article: shrestha, acharya, karmacharya, sapkota, paudyal, basnet, timilsina, shakya/application of ahp and gis for optimal solar site identification in madhesh province, nepal international journal of the analytic hierarchy process 10 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.986 table 7 source of dataset dataset source solar radiation global solar atlas (https://globalsolaratlas.info/download/nepal) annual mean temperature department of hydrology and meteorology, nepal (http://www.dhm.gov.np/) distance from roads road network of hindu kush himalayan (hkh) region (https://rds.icimod.org/home/datadetail?metadataid=398 ) distance from substation nepal electricity authority (www.nea.org.np) distance from urban areas aster gdem v2 ( https://earthexplorer.usgs.gov) elevation aster gdem v2 ( https://earthexplorer.usgs.gov) slope aster gdem v2 ( https://earthexplorer.usgs.gov) aspect (orientation) aster gdem v2 ( https://earthexplorer.usgs.gov) land use esri land use (https://www.arcgis.com/home/item.html?id=d6642f8a4f6 d4685a24ae2dc0c73d4ac) 3.6 methodology to create land suitability map the methodology adopted for the study is shown in figure 2. all the datasets were rasterized and reprojected into utm 45 and resampled into 10m and then applied for further processing in the mcda-ahp model the in gis environment to layer for each criteria map. each thematic layer or the criteria has then been reclassified to create subcriteria. after the calculation of the weights of criteria and sub-criteria, the weighted sum overlay tool in gis was applied to determine the potential sites for solar plant installation. the following procedure was followed to obtain the desired map (al garni & awasthi, 2017): 1. nine layers, one for each criterion with their corresponding weights and the weight of the sub-criteria were created. each criterion was brought to a common scale as it contained different values and ranges. values of input maps were then reclassified into a common preference scale of suitability range. 2. each criteria layer along with its sub-criteria was multiplied by its respective weight and the resulting cell values were added together to generate the ultimate combined layer in arc gis. 3. a restriction layer consisting of the factors was created to isolate unsuitable areas for solar installation. https://globalsolaratlas.info/download/nepal http://www.dhm.gov.np/) https://rds.icimod.org/home/datadetail?metadataid=398 file:///c:/users/neaec/desktop/www.nea.org.np https://earthexplorer.usgs.gov)/ https://earthexplorer.usgs.gov)/ https://earthexplorer.usgs.gov)/ https://earthexplorer.usgs.gov)/ https://www.arcgis.com/home/item.html?id=d6642f8a4f6d4685a24ae2dc0c73d4ac https://www.arcgis.com/home/item.html?id=d6642f8a4f6d4685a24ae2dc0c73d4ac ijahp article: shrestha, acharya, karmacharya, sapkota, paudyal, basnet, timilsina, shakya/application of ahp and gis for optimal solar site identification in madhesh province, nepal international journal of the analytic hierarchy process 11 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.986 figure 3 methodology 4. results and discussion nine criteria were considered to identify the optimal site for solar plant installation using a combination of the ahp and gis. table 8 shows the pairwise comparison matrix for the criteria considered for the present study while table 9 depicts the weight of each criterion. the procedure for computation of the pairwise matrix has been described in section 3.3 the calculated consistency ratio is well below 0.1 which indicates that the pairwise comparison is consistent. solar irradiation was assigned with the highest weight followed by annual mean temperature, while land use was assigned the minimum weightage. figure 4 shows the map of each criterion that was considered for the study (a-i), a map for restricted area (j), and a land suitability map (k). ijahp article: shrestha, acharya, karmacharya, sapkota, paudyal, basnet, timilsina, shakya/application of ahp and gis for optimal solar site identification in madhesh province, nepal international journal of the analytic hierarchy process 12 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.986 table 8 pairwise comparison matrix using ahp for optimal solar sites identification si amt dfr dfs dfu el sl as lu si 1.000 3.000 5.00 5.00 5.00 5.00 5.00 7.00 7.0 amt 0.333 1.000 5.00 5.00 5.00 5.00 5.00 5.00 5.0 dfr 0.200 0.200 1.00 3.00 3.00 3.00 3.00 3.00 3.0 dfs 0.200 0.200 0.33 1.00 3.00 3.00 3.00 3.00 3.0 dfu 0.200 0.200 0.33 0.33 1.00 3.00 3.00 3.00 3.0 el 0.200 0.200 0.33 0.33 0.33 1.00 2.00 3.00 3.0 sl 0.200 0.200 0.33 0.33 0.33 0.50 1.00 3.00 3.0 as 0.143 0.200 0.33 0.33 0.33 0.33 0.33 1.00 3.0 lu 0.143 0.200 0.33 0.33 0.33 0.33 0.33 0.33 1.0 si: solar irradiance, mt: annual mean temperature, dfr: distance from the road, dfs: distance from substation, dfu: distance from an urban area, el: elevation, as: aspect, lu: land use table 9 weight of each criterion criteria weight si 0.33 amt 0.24 dfr 0.11 dfs 0.09 dfu 0.07 el 0.05 sl 0.04 as 0.03 lu 0.03 inconsistency check αmax 10.141 consistency index, ci 0.143 random index, ri 1.450 consistency ratio, cr 0.098 each of the criteria has its own sub-criteria. the weights of the sub-criteria were calculated using the ahp and the consistency ratio was kept below 0.1. the combined weights of criteria and sub-criteria are shown in table 10. . ijahp article: shrestha, acharya, karmacharya, sapkota, paudyal, basnet, timilsina, shakya/application of ahp and gis for optimal solar site identification in madhesh province, nepal international journal of the analytic hierarchy process 13 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.986 table 10 summary of weights of criteria and sub-criteria using ahp for optimal solar sites identification criteria weight sub criteria weight final weight si (kwh/m 2 ) 0.332 4.785-4.860 0.677 0.225 4.731-4.785 0.172 0.057 4.669-4.731 0.097 0.032 4.308-4.669 0.054 0.018 amt( 0 c) 0.241 28.3-28.7 0.731 0.176 28.7-29.2 0.188 0.045 29.2-29.6 0.081 0.020 dfr (meter) 0.113 <5 0.638 0.072 5-10 0.212 0.024 10-15 0.099 0.011 15-20 0.050 0.006 dfs (meter) 0.089 <4 0.638 0.057 4-8 0.212 0.019 8-12 0.099 0.009 >12 0.050 0.004 dfu (meter) 0.070 <4 0.638 0.044 4-8 0.212 0.015 8-12 0.099 0.007 >12 0.050 0.004 el (m.a.s.l) 0.052 <100 0.638 0.033 100-150 0.212 0.011 150-200 0.099 0.005 >200 0.050 0.003 ijahp article: shrestha, acharya, karmacharya, sapkota, paudyal, basnet, timilsina, shakya/application of ahp and gis for optimal solar site identification in madhesh province, nepal international journal of the analytic hierarchy process 14 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.986 criteria weight sub criteria weight final weight sl (degree) 0.045 <1 0.638 0.029 1-5 0.212 0.009 5-10 0.099 0.004 >10 0.050 0.002 as 0.032 flat/south 0.638 0.021 south east 0.212 0.007 east or west 0.099 0.003 other 0.050 0.002 lu 0.025 bare ground 0.743 0.019 grass/shrub 0.194 0.005 agricultural land 0.063 0.002 figure 4a maps of solar irradiation figure 4b map of mean summer temperature ijahp article: shrestha, acharya, karmacharya, sapkota, paudyal, basnet, timilsina, shakya/application of ahp and gis for optimal solar site identification in madhesh province, nepal international journal of the analytic hierarchy process 15 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.986 figure 4c map of aspect figure 4d map of distance from major road figure 4e map of distance from substation figure 4f map of distance from urban area figure 4g map of elevation figure 4h map of land use land cover ijahp article: shrestha, acharya, karmacharya, sapkota, paudyal, basnet, timilsina, shakya/application of ahp and gis for optimal solar site identification in madhesh province, nepal international journal of the analytic hierarchy process 16 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.986 figure 4i map of slopes figure 4j map of restriction layers figure 4k map of land suitability index for madhesh province a land suitability index (lsi) defined as the degree to which each site is suitable for the placement of pv plants according to the associated criteria and excluding all restrictions (al garni & awasthi, 2017) was employed to differentiate the land suitable for solar power plant installation with less suitable land. index one (1) represents the least preferred area while five (5) represents the most suitable area. the highly suitable sites were found in the saptari, siraha, dhanusa, and mahottari districts while the district in the western region of the province had less suitable areas. although the entire region from saptari to mahottari has high solar irradiation, these districts are also densely populated urban areas thereby reducing the amount of land suitable to set up the solar power plant. solar irradiation had a major impact on the suitability of the land for solar power plant placement. table 11 shows the land suitability index of each of the districts of madhesh province in terms of the total area of that district. saptari district has 61.5% of the land with an index of five, followed by siraha with 56.7% and dhanusa with ijahp article: shrestha, acharya, karmacharya, sapkota, paudyal, basnet, timilsina, shakya/application of ahp and gis for optimal solar site identification in madhesh province, nepal international journal of the analytic hierarchy process 17 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.986 35.4%. bara, parsa, and rautahat districts lacked a suitability index of five but had an index ranging from 2 to 4. districts on the western side of the province have national parks and reserves that have reduced the area suitable for the installation of solar power plants. figure 5 depicts the suitability index of each district of madhesh province. table 11 summary of suitability index for districts of madhesh province district % of land with lsi 1 2 3 4 5 saptari 0.0% 0.0% 0.2% 0.8% 61.5% siraha 0.0% 0.3% 0.5% 5.6% 56.7% dhanusa 0.0% 0.1% 1.8% 19.6% 35.4% mahottari 0.0% 0.1% 2.2% 36.7% 16.9% sarlahi 0.0% 1.5% 7.6% 49.6% 0.5% rautahat 0.6% 16.2% 12.8% 25.3% 0.0% bara 2.8% 35.8% 15.9% 0.0% 0.0% parsa 4.7% 25.6% 8.1% 0.0% 0.0% ijahp article: shrestha, acharya, karmacharya, sapkota, paudyal, basnet, timilsina, shakya/application of ahp and gis for optimal solar site identification in madhesh province, nepal international journal of the analytic hierarchy process 18 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.986 ijahp article: shrestha, acharya, karmacharya, sapkota, paudyal, basnet, timilsina, shakya/application of ahp and gis for optimal solar site identification in madhesh province, nepal international journal of the analytic hierarchy process 19 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.986 figure 3 land suitability index of different districts of madhesh province 5. conclusion nepal has adopted the strategy of integrating renewable sources of energy like solar power plants to achieve the goal of carbon neutrality. the nepal electricity authority, the sole buyer of the energy, provides an attractive feed-in tariff (fit) for the solar power plants, which attracts investors. despite many of the benefits of solar power plants, a research gap during the initial planning phase to determine the optimal areas for setting up the power plants still exists. identification of the optimal location is by far the most crucial factor in setting up a power plant. this study can be a guide for the planners, developers, and the government regarding the optimal sites for utility-scale solar power plants. a more realistic and practical approach has been adopted in this study by considering the criteria and sub-criteria that covered the technical, environmental, geographical, and economic aspects of setting up a solar power plant. the application of the ahp combined with gis has given a better overview of land that is suitable for the installation of solar power plants. the method indicated that the areas on the eastern side of the province are more highly suitable to establish solar power plants while the western side has the least suitable areas to establish solar power plants. saptari district has the highest percentage of land that is suitable for solar power plants while the parsa district has the lowest. the major limitation of this study was the use of secondary data on solar irradiation rather than data available from weather stations. more reliable data from weather stations can provide more accurate results for site suitability. it is recommended for future studies ijahp article: shrestha, acharya, karmacharya, sapkota, paudyal, basnet, timilsina, shakya/application of ahp and gis for optimal solar site identification in madhesh province, nepal international journal of the analytic hierarchy process 20 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.986 to include spatial data with higher accuracy and integrate data available from weather stations in various locations throughout the country. ijahp article: shrestha, acharya, karmacharya, sapkota, paudyal, basnet, timilsina, shakya/application of ahp and gis for optimal solar site identification in madhesh province, nepal international journal of the analytic hierarchy process 21 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.986 references ahmad, s., & tahar, r. m. (2014). selection of renewable energy sources for sustainable development of electricity generation system using analytic hierarchy process: a case of malaysia. renewable energy, 63, 458–466. doi:10.1016/j.renene.2013.10.001 akkas, o. p., erten, m. y., cam, e., & inanc, n. (2017). optimal site selection for a solar power plant in the central anatolian region of turkey. international journal of photoenergy, 2017, 1-13. doi:10.1155/2017/7452715 al garni, h. z., & awasthi, a. (2017). solar pv power plant site selection using a gisahp based approach with application in saudi arabia. applied energy, 206, 1225–1240. doi:10.1016/j.apenergy.2017.10.024 albraheem, l., & alabdulkarim, l. (2021). geospatial analysis of solar energy in riyadh using a gis-ahp based technique. isprs international journal of geo-information, 10(5), 291. doi:10.3390/ijgi10050291 ali, s., taweekun, j., techato, k., waewsak, j., & gyawali, s. (2019). gis based site suitability assessment for wind and solar farms in songkhla, thailand. renewable energy, 132, 1360–1372. doi:10.1016/j.renene.2018.09.035 aly, a., jensen, s. s., & pedersen, a. b. (2017). solar power potential of tanzania: identifying csp and pv hot spots through a gis multicriteria decision making analysis. renewable energy, 113, 159-175. doi:https://doi.org/10.1016/j.renene.2017.05.077 arán carrión, j., espín estrella, a., aznar dols, f., zamorano toro, m., rodríguez, m., & ramos ridao, a. (2008). environmental decision-support systems for evaluating the carrying capacity of land areas: optimal site selection for grid-connected photovoltaic power plants. renewable and sustainable energy reviews, 12(9), 2358–2380. doi:10.1016/j.rser.2007.06.011 choi, y., suh, j., & kim, s. m. (2019). gis-based solar radiation mapping, site evaluation, and potential assessment: a review. applied sciences (switzerland), 9(9), 127. doi:10.3390/app9091960 department of electricity development. (2021). license status. retrieved from www.doed.gov.np/license development, m. o. (2021). energy profile: province 2, lumbini and karnali. retrieved from https://www.nrepnepal.com/wp-content/uploads/2021/04/energy-profiles-province2-lumbini-province-and-karnali-province.pdf doorga, j. r., rughooputh, s. d., & boojhawon, r. (2019). multi-criteria gis-based modelling technique for identifying potential solar farm sites: a case study in mauritius. renewable energy, 133, 1201–1219. doi:10.1016/j.renene.2018.08.105 ijahp article: shrestha, acharya, karmacharya, sapkota, paudyal, basnet, timilsina, shakya/application of ahp and gis for optimal solar site identification in madhesh province, nepal international journal of the analytic hierarchy process 22 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.986 esri. (2021). what is gis? retrieved from https://www.esri.com/en-us/what-isgis/overview gašparović, i., & gašparović, m. (2019). determining optimal solar power plant locations based on remote sensing and gis methods: a case study from croatia. remote sensing, 11(12), 1-18. doi:10.3390/rs11121481 international energy agency. (2021). global energy review 2021. retrieved from https://www.iea.org/reports/global-energy-review-2021 kanichakra, l. (2018). geographic information systems (gis) defined. geographic information systems (gis) defined. retrieved from https://www.arcweb.com/blog/geographic-information-systems-gis-defined munkhbat, u., & choi, y. (2021). gis-based site suitability analysis for solar power systems in mongolia. applied sciences (switzerland), 11(9), 1-13. doi:10.3390/app11093748 nebey, a. h., taye, b. z., & workineh, t. g. (2020). site suitability analysis of solar pv power generation in south gondar, amhara region. journal of energy, 2020, 1–15. doi:10.1155/2020/3519257 nepal electricity authority. (2021). a year in review: fiscal year 2020/2021. nepal outlook. (2020). retrieved from https://nepaloutlook.com/about-us/ paneru, g. r. (2016). gis-based identification of potential sites for installing solar energy collection plants in kathmandu valley, nepal. prieto-amparán, j. a., pinedo-alvarez, a., morales-nieto, c. r., valles-aragón, m. c., álvarez-holguín, a., & villarreal-guerrero, f. (2021). a regional gis-assisted multicriteria evaluation of site-suitability for the development of solar farms. land, 10, 1–19. doi:10.3390/land10020217 timilsina, r.h., ojha, g.p., nepali, p.b. & tiwari, u. (2019). agriculture and land use in nepal: prospects and impacts on food security. journal of agriculture and forestry university, 3, 1-9. ruiz, h. s., sunarso, a., ibrahim-bathis, k., murti, s. a., & budiarto, i. (2020). gisahp multi criteria decision analysis for the optimal location of solar energy plants at indonesia. energy reports, 6, 3249–3263. doi:10.1016/j.egyr.2020.11.198 sánchez-lozano, j. m., teruel-solano, j., soto-elvira, p. l., & socorro garcía-cascales, m. (2013). geographical information systems (gis) and multi-criteria decision making (mcdm) methods for the evaluation of solar farms locations: case study in south-eastern spain. renewable and sustainable energy reviews, 24, 544–556. doi:10.1016/j.rser.2013.03.019 ijahp article: shrestha, acharya, karmacharya, sapkota, paudyal, basnet, timilsina, shakya/application of ahp and gis for optimal solar site identification in madhesh province, nepal international journal of the analytic hierarchy process 23 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.986 saaty. (1980). the analytic process: planning, priority setting, resources allocation. mcgraw hill. tahri, m., hakdaoui, m., & maanan, m. (2015). the evaluation of solar farm locations applying geographic information system and multi-criteria decision-making methods: case study in southern morocco. renewable and sustainable energy reviews, 51, 1354– 1362. doi:10.1016/j.rser.2015.07.054 the world bank. (2017). solar resource and photovoltaic potential of nepal. tech. rep. uyan, m. (2013, december 1). gis-based solar farms site selection using analytic hierarchy process (ahp) in karapinar region, konya/turkey. renewable and sustainable energy reviews, 28, 11–17. doi:10.1016/j.rser.2013.07.042 water and energy commission secretariat (2010). energy sector synopsis report. watson, j. j., & hudson, m. d. (2015). regional scale wind farm and solar farm suitability assessment using gis-assisted multi-criteria evaluation. landscape and urban planning, 138, 20–31. doi:10.1016/j.landurbplan.2015.02.001 xin, z., chen, l., yang, y., miao, k., & li, z. (2019). application of ahp-dea model in comprehensive evaluation of distribution network planning reliability. ispec 2019 2019 ieee sustainable power and energy conference: grid modernization for energy revolution, proceedings, 290–294. doi:10.1109/ispec48194.2019.8975139 yushchenko, a., de bono, a., chatenoux, b., patel, m. k., & ray, n. (2018). gis-based assessment of photovoltaic (pv) and concentrated solar power (csp) generation potential in west africa. renewable and sustainable energy reviews, 81, 2088–2103. doi:10.1016/j.rser.2017.06.021 ijahp article: ağaç, şimşir/ optimal site selection of a pandemic hospital using multi-criteria decision-making approach international journal of the analytic hierarchy process 1 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.946 optimal site selection of a pandemic hospital using multi-criteria decision-making approach gökhan ağaç department of health management, sakarya university of applied sciences, sakarya, turkey gokhanagac@subu.edu.tr i̇smail şimşir ismailsimsir@subu.edu.tr abstract the covid-19 pandemic has spread rapidly and affected the whole world. during the process of combating the pandemic, it has become apparent that some physical infrastructures such as intensive care units have been insufficient to meet the current demand. the aim of this study is to select the optimal location for a pandemic hospital and determine the critical factors affecting the selection. the analytical hierarchy process approach is implemented in the study with 27 criteria used to evaluate the alternatives. the data of the study are collected from 23 experts. the result of the analysis proposes the optimal location to establish a pandemic hospital. in addition, the three most important criteria that affect the location selection of the pandemic hospital are risk, accessibility, and opportunities and threats, respectively. moreover, the results of the sensitivity analysis show that the outputs of the proposed model are robust. a location selection problem for a pandemic hospital was addressed and the factors affecting the location selection are discussed in this study. the proposed model is expected to be a guide for health policymakers, healthcare managers, and public and private investment decision-makers as a decision support system. keywords: pandemic hospital; location selection; covid-19; analytical hierarchy process; multi-criteria decision-making 1. introduction human beings have faced with many kinds of disasters whether man-made or natural throughout history including earthquakes, tsunamis, volcanic eruptions, landslides, nuclear disasters and epidemics. these types of disasters affect limited areas; however, pandemics affect wide geographic areas. in addition to being a disaster, the novel covid-19 outbreak, which was declared a pandemic by the who on march 11, 2020 (who, 2020a), can be characterized as the biggest multifaceted crisis ever faced by the modern world resulting in more than 263 million cases and 5.2 million deaths as of december 6, 2021 (who, 2020b). mailto:gokhanagac@subu.edu.tr mailto:ismailsimsir@subu.edu.tr ijahp article: ağaç, şimşir/ optimal site selection of a pandemic hospital using multi-criteria decision-making approach international journal of the analytic hierarchy process 2 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.946 the novel covid-19 pandemic has created an "overwhelming burden" on most of the world’s health systems. governments across the world have pulled out all the stops to ameliorate that overwhelming burden, but have experienced challenges due to the deficiency of available physical resources, especially icu’s (intensive care units). therefore, beyond allocation of resources, increased icu and monitorized (semi-icu) bed capacity and decreased contamination risk are of utmost importance when it comes to building a pandemic hospital. hospitals should also be kept safe and functional considering emergency and disaster conditions. therefore, just like with other disaster conditions, the proper selection of the location of a pandemic hospital would have a direct effect on the survival and/or rapid recovery of an affected population; having a building with sufficient capacity in pandemic conditions will have the same effect. the location and capacity would also have a direct effect on cost and other benefits for its post-pandemic usage. on the other hand, decisions regarding the selection of a hospital location could depend on personal accounts instead of objective analyses (soltani & marandi, 2011). however, an improper decision without conducting an analytical process would increase investment and operational costs and affect the lives of the affected population in the pandemic management cycle. such a decision might also stymie future usage in the post-pandemic era. an appropriate decision made after an analytical process regarding hospital site selection would however have positive repercussions on different parties such as optimizing the allocation of medical resources by matching the provision of health care with the social and economic demands, coordinating the urban and rural health service development, and easing social contradictions on the aspect of government; and access to health care by reducing the time of rescue, satisfying people’s medical needs as well as enhancing the quality of life of the citizens; and cost savings for investors and operators of the hospital (sen, 2017). a decision-making process regarding facility location involves identification, analysis, assessment and selection of a number of alternatives. after recognition of a need for additional capacity, a decision is made about the “best” possible location (yang & lee, 1997). in such a process, determining the criteria that should be considered and their possible level of impact on site selection is of primary importance. in other words, the selection of the location of urban facilities is a strategic issue due to the several side effects and multiple conflicting criteria involved in such a decision (oppio et al., 2016). therefore, appropriate site selection for a hospital is an important determinant in the success or failure of the facility (senvar et al., 2016). different multi-criteria decision-making (mcdm) tools have been applied in health care. the analytical hierarchy process (ahp) is a mcdm tool that decomposes a complex multi-criteria decision problem into a hierarchy (saaty, 1980). based on a review of the existing literature regarding hospital site selection, there is scant research utilizing ahp or its extended forms in this field. yap et al. (2019) performed a systematic review of multi-criteria decision-making methods for location selection applications. they reported ijahp article: ağaç, şimşir/ optimal site selection of a pandemic hospital using multi-criteria decision-making approach international journal of the analytic hierarchy process 3 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.946 that the analytic hierarchy process was the most used approach for location selection problems. in addition, gul and guneri (2021) conducted a literature review on the selection of a hospital location. they also found that the ahp is the most preferred mcdm method. as such, there are a number of studies in the literature using the ahp to find the best hospital location. some of them use the ahp separately (chatterjee & mukherjee, 2013; chiu & tsai, 2013; jalaliyoon et al., 2015; wu et al., 2007) and others combine it with various approaches (al-shabeeb et al., 2016; şen & demiral, 2016; triantono & susetyarto, 2017; vahidnia et al., 2009). there are only a few studies on the location selection of a pandemic hospital (alkan & kahraman, 2021; aydin & seker, 2021; boyacı & şişman, 2021; zolfani et al., 2020). the present study contributes to eliminating this deficiency in the relevant literature. the originality of the present paper primarily comes from its application of the ahp to problem of location selection for a pandemic hospital. second, pandemic conditions are considered when defining criteria, sub-criteria and alternatives. third, 27 criteria were included (8 main and 19 sub-criteria) which adds richness to existing knowledge on a subject that has limited research. lastly, the opinions of top actors in relevant provinces who are of utmost importance when considering building a pandemic hospital were solicited. these actors include urban and regional planners in a metropolitan municipality, engineers in the health facilities department of the provincial health directorate, top managers of hospitals, academics from medicine faculty and attending physicians (especially from the departments of infectious diseases and clinical microbiology and diseases of the chest), academics from nursing and health management departments of universities in the same province, and finally top managers of the provincial health directorate. this study also provides policy makers with a tool to use in their decision-making process to choose a hospital location in both current and possible future pandemic cases. the remainder of this study is organized as follows. section 2 explains the analytical hierarchy process methodology. section 3 presents an implementation of the proposed approach for solving the location problem of a pandemic hospital. section 4 discusses the results of the research and finally, conclusions are discussed in the last section. 2. analytical hierarchy process methodology the analytical hierarchy process is one of the multi-criteria decision-making techniques that is capable of solving large, dynamic, and complex real-world problems (yang & lee, 1997). the ahp was first proposed by thomas l. saaty in the 1970s as a quantitative decision-making approach that converts qualitative judgments into numerical values (soltani & marandi, 2011). there are three basic principles for the implementation of the ahp as follows: (i) identifying the problem and building a hierarchy, (ii) forming a comparative decision-making preference matrix, and (iii) determining factor weights (colak et al., 2020). therefore, the outputs of the implementation provide support to the decision-maker, who is facing a complex problem, during the decision phase. let 𝐴 be 𝑛 × 𝑛 matrix, 𝑤 = ( 𝑤1, 𝑤2, 𝑤3, … , 𝑤𝑛 ) a weight vector of matrix 𝐴. then, the ahp approach steps can be explained as follows (bhushan & rai, 2004; saaty, 1984, 1990): ijahp article: ağaç, şimşir/ optimal site selection of a pandemic hospital using multi-criteria decision-making approach international journal of the analytic hierarchy process 4 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.946 step 1: the problem is decomposed into a hierarchy including goal, criteria, sub-criteria, and alternatives. the first, second, third, and last level of the hierarchy represent the goal, criteria, sub-criteria, and alternatives, respectively. step 2: the pairwise comparison matrices are established. first, the relative importance for the particular element is collected from experts or decision-makers according to the hierarchy of the problem using the fundamental scale (table 1). then, collected data are imported into a pairwise comparison matrix. a pairwise comparison matrix 𝐴𝑛×𝑛 = (𝑤𝑖 𝑤𝑗⁄ ) (𝑖, 𝑗 = 1, 2, 3, … , 𝑛) is composed as follows: 𝐴 = 𝐶𝑖 𝐴1 𝐴2 𝐴1 𝑤1 𝑤1⁄ 𝑤1 𝑤2⁄ 𝐴2 𝑤2 𝑤1⁄ 𝑤2 𝑤2⁄ 𝐴3 ⋯ 𝐴𝑛 𝑤1 𝑤3⁄ ⋯ 𝑤1 𝑤𝑛⁄ 𝑤2 𝑤3⁄ ⋯ 𝑤2 𝑤𝑛⁄ 𝐴3 𝑤3 𝑤1⁄ 𝑤3 𝑤2⁄ ⋮ ⋮ ⋮ 𝐴𝑛 𝑤𝑛 𝑤1⁄ 𝑤𝑛 𝑤2⁄ 𝑤3 𝑤3⁄ ⋯ 𝑤3 𝑤𝑛⁄ ⋮ ⋱ ⋮ 𝑤𝑛 𝑤3⁄ ⋯ 𝑤𝑛 𝑤𝑛⁄ where 𝐶𝑖 (𝑖 = 1, 2, 3, … , 𝑛) and 𝐴𝑖 (𝑖 = 1, 2, 3, … , 𝑛) represent criteria and element/alternative, respectively. let 𝑎𝑖𝑗 = 𝑤𝑖 𝑤𝑗⁄ . then, 𝑎𝑖𝑗 = 1/𝑎𝑗𝑖 and 𝑎𝑖𝑖 = 1 (𝑖, 𝑗 = 1, 2, 3, … , 𝑛). thus, the pairwise comparison matrix transforms into the matrix 𝐴𝑛×𝑛 = (𝑎𝑖𝑗 ) as follows: 𝐴 = 𝐶𝑖 𝐴1 𝐴2 𝐴1 𝑎11 𝑎12 𝐴2 𝑎21 𝑎22 𝐴3 ⋯ 𝐴𝑛 𝑎13 ⋯ 𝑎1𝑛 𝑎32 ⋯ 𝑎2𝑛 𝐴3 𝑎31 𝑎32 ⋮ ⋮ ⋮ 𝐴𝑛 𝑎𝑛1 𝑎𝑛2 𝑎33 ⋯ 𝑎3𝑛 ⋮ ⋱ ⋮ 𝑎𝑛3 ⋯ 𝑎𝑛𝑛 table 1 fundamental scale importance explanation (between 𝐴𝑖 and 𝐴𝑗 alternatives) 1 𝐴𝑖 and 𝐴𝑗 have equal importance 3 𝐴𝑖 is moderately more important than 𝐴𝑗 5 𝐴𝑖 is strongly more important than 𝐴𝑗 7 𝐴𝑖 is very strongly more important than 𝐴𝑗 9 𝐴𝑖 is extremely more important than 𝐴𝑗 2,4,6,8 intermediate values step 3: the relative weights of the pairwise comparison matrix are calculated. to determine the relative weights, each of the column entries of the matrix are first normalized by dividing by their column sum, and then each row is averaged. ijahp article: ağaç, şimşir/ optimal site selection of a pandemic hospital using multi-criteria decision-making approach international journal of the analytic hierarchy process 5 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.946 step 4: the consistency of the pairwise comparison matrix is evaluated. let 𝜆𝑚𝑎𝑥 be the maximum eigenvalue of the comparison matrix. then, the consistency of matrix 𝐴 is calculated using the following equation: 𝐶𝑅 = 𝐶𝐼 𝑅𝐼⁄ , where 𝐶𝐼 (consistency index) = (𝜆𝑚𝑎𝑥 − 𝑛) (𝑛 − 1)⁄ , 𝑅𝐼 (random index) is given in table 2, and 𝐴𝑤 = 𝜆𝑚𝑎𝑥 𝑤. in addition, the consistency ratio (cr) should be less than 0.1 (saaty, 1980). table 2 random index (ri) 𝑛 2 3 4 5 6 7 8 𝑅𝐼 0.00 0.58 0.90 1.12 1.24 1.32 1.41 step 5: global weights (also known as synthesis value) are calculated. local weights become global weights by multiplying them by the weight of the corresponding criterion/alternative and then adding over all scores with respect to which the comparison is made. 3. implementation of case study this section consists of four-phases. in the first phase, the study identified the alternatives, and the main and sub-criteria related to optimal location selection for a pandemic hospital. in the second phase, it decomposed the problem into a hierarchy including goal, criteria, sub-criteria, and alternatives. in the next phase, the pairwise comparison matrices were established by collecting data from experts and used to obtain their local weights, consistency, and global weights. in the last phase, a sensitivity analysis was performed to observe the robustness of the proposed model. 3.1 identification of alternatives, criteria, and sub-criteria 3.1.1 study area sakarya is a province located in northwestern turkey. the province's total area is 4,824 square kilometers (gdm, 2021) and population at the end of 2019 was 1,029,650 (turkstat, 2021). the province is located in the midst of turkey's crowded provinces and has 16 districts. there are three central districts among these districts. as a result of face-to-face interviews with the managers of health institutions in the province, the central districts were specified as potential locations for a pandemic hospital. these districts are adapazarı, erenler, and serdivan. the location of these provinces on a map is shown in figure 1. the area, population, and density information about the provinces obtained from the turkey statistical institute (turkstat, 2021) and general directorate of mapping (gdm, 2021) are presented in table 3. according to this data, whereas adapazarı is first in terms of size and population with 324 square kilometers area and a population of 276,385, serdivan has the highest population density (1,135 per square kilometer). adapazarı, erenler, and serdivan have elderly populations of 9.34%, 8.31%, and 6.54%, respectively. in addition, the number of the population with a university level education is 37,733, 10,396, and 28,601, respectively. ijahp article: ağaç, şimşir/ optimal site selection of a pandemic hospital using multi-criteria decision-making approach international journal of the analytic hierarchy process 6 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.946 ijahp article: ağaç, şimşir/ optimal site selection of a pandemic hospital using multi-criteria decision-making approach international journal of the analytic hierarchy process 7 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.946 table 3 statistical information for potential locations at the end of 2019 alternatives area (square kilometres) current population population density elderly population (≥65 years) population with a university education adapazarı 324 276,385 853 25,826 37,733 erenler 136 89,128 655 7,407 10,396 serdivan 130 147,500 1,135 9,650 28,601 figure 1 map of the study area 3.1.2 main and sub-criteria in this section, the main and sub-criteria that affect the selection of the optimal location for a pandemic hospital are determined. first, a literature review related to hospital location selection was performed. then, as a result of the literature review, eight main criteria and nineteen sub-criteria were determined (table 4). later, two infectious disease experts were asked for their views on the appropriateness of the specified criteria. they confirmed that the specified criteria are suitable for the selection of a pandemic hospital location, and stated that they would not add or remove any criteria. in addition, they stated that meeting the environmental risk criteria would have a positive effect on patient's health. adapazarı serdivan e re n le r turkey sakarya ijahp article: ağaç, şimşir/ optimal site selection of a pandemic hospital using multi-criteria decision-making approach international journal of the analytic hierarchy process 8 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.946 table 4 criteria and sub-criteria to determine optimal location selection for a pandemic hospital reference criteria sub-criteria wu et al. (2007) vahidnia et al. (2009) soltani and marandi (2011) kumar et al. (2016) senvar et al. (2016) adalı and tuş (2019) zolfani et al. (2020) cost investment cost     labor cost   land cost      demographics current population     population density     age profile   education profile  accessibility travel time  transfer modes     risk disaster risk  environmental risk    parcel characteristics ground conditions  capacity expansion    park area   government incentives    opportunities and threats market conditions     economic conditions  human resources availability of skilled staffs   availability of other staffs   3.2 hierarchical representation of the problem the problem is decomposed into a four-level hierarchical structure (figure 2). level 1 includes the problem goal that is the optimal location selection for a pandemic hospital. level 2 represents the main criteria of the problem that are cost, demographics, accessibility, risk, parcel characteristics, government, opportunities and threats, and human resources. below the main criteria, nineteen sub-criteria are determined as the next level. the lowest level of the hierarchy, level 4, shows the alternatives for the problem. ijahp article: ağaç, şimşir/ optimal site selection of a pandemic hospital using multi-criteria decision-making approach international journal of the analytic hierarchy process 9 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.946 figure 2 decomposition of the problem into a hierarchy 3.3 analysis of data 3.3.1 collection of data the data of the study were collected between december 2, 2020 and january 7, 2021 from a total of 23 people who have expertise in their field. in addition, the titles of the experts who are in the positions of health manager, administrative manager, and healthcare academician are presented in table 5. the study also uses secondary data that are available and accessible for the alternatives according to the sub-criteria of demographics. the secondary data were obtained from the turkey statistical institute (turkstat, 2021) and general directorate of mapping (gdm, 2021). however, considering the up-to-dateness of the data, the rate of data to be used in the calculation is determined as 70% for data collected from experts and 30% for secondary data. optimal location selection for a pandemic hospital opportunities and threats cost demographics accessibility parcel characteristics risk human resources governmentlevel 2: criteria level 1: goal level 3: sub-criteria level 4: alternatives investment cost labor cost land cost current population population density age profile travel time transfer modes education profile disaster risk environmental risk ground conditions capacity expansion park area incentives market conditions economic conditions availability of skilled staffs availability of other staffs adapazarı erenler serdivan ijahp article: ağaç, şimşir/ optimal site selection of a pandemic hospital using multi-criteria decision-making approach international journal of the analytic hierarchy process 10 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.946 table 5 distribution of the expert titles title of expert position number of people health directorate, financial services specialist 3 health directorate, public health facilities planner 3 health directorate, support services manager 1 hospital, administrative financial manager 1 hospital, chest diseases specialist 1 hospital, health services manager 1 hospital, technical services manager 1 municipality, city and regional planner 6 municipality, city planning branch manager 1 municipality, development and urban planning department, geophysical engineer 1 university, healthcare management, academician 2 university, infectious diseases specialist, academician 2 total 23 the consistency of each comparison matrix collected from the experts is tested. some pairwise comparison matrices have consistency ratios greater than 10%. these comparison matrices need to be re-evaluated by the expert making the comparison. after this improvement process, when the consistency ratio values are examined, the main criteria vary between 0.0673 and 0.0993, the sub-criteria vary between 0.000 and 0.0941, and the alternatives vary between 0.000 and 0.0960. 3.3.2 findings from the pairwise comparison matrices in this section, the aggregate pairwise comparison matrices, relative weights, and ranking for criteria, sub-criteria, and alternatives are given. the aggregate pairwise comparison matrices are obtained from the aggregated values which are calculated using the geometric mean. the findings are presented in table 6 for criteria, tables 7-13 for subcriteria, and tables 14-21 for alternatives. in addition, the global weights and ranking of the elements in these three levels are shown in tables 21-22. according to table 6, the relative weights of criteria with regard to goal are cost (0.110), demographics (0.059), accessibility (0.201), risk (0.206), parcel characteristics (0.073), government (0.075), opportunities and threats (0.193), and human resources (0.0839). ijahp article: ağaç, şimşir/ optimal site selection of a pandemic hospital using multi-criteria decision-making approach international journal of the analytic hierarchy process 11 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.946 table 6 aggregate pairwise comparison matrix, weights, and consistency values for criteria g o a l c o st d e m o g ra p h ic s a c c e ss ib il it y r is k p a rc e l c h a ra c te ri st ic s g o v e rn m e n t o p p o rt u n it ie s a n d t h re a ts h u m a n re so u rc e s w e ig h ts cost 1.000 2.433 0.431 0.399 1.434 2.038 0.519 1.405 0.110 demographics 0.411 1.000 0.242 0.315 1.009 0.978 0.242 0.761 0.059 accessibility 2.318 4.128 1.000 1.133 2.560 2.614 0.787 2.091 0.201 risk 2.508 3.173 0.882 1.000 2.686 2.521 0.944 3.244 0.206 parcel characteristics 0.698 0.991 0.391 0.372 1.000 0.985 0.483 0.731 0.073 government 0.491 1.022 0.383 0.397 1.015 1.000 0.556 0.919 0.075 opportunities and threats 1.925 4.128 1.271 1.060 2.068 1.798 1.000 2.015 0.193 human resources 0.712 1.314 0.478 0.308 1.369 1.088 0.496 1.000 0.083 𝜆𝑚𝑎𝑥 = 8.1263902; 𝐶𝐼 = 0.0180557; 𝑅𝐼 = 1.41; 𝐶𝑅 = 0.0128055 ≤ 0.1 considering the findings from the sub-criteria, the relative weights for the three subcriteria of cost criteria are investment cost (0.640), labor cost (0.181), and land cost (0.180). for the four sub-criteria of the demographics criterion, the relative weights are current population (0.199), population density (0.355), age profile (0.340), and education profile (0.107). the relative weights of the travel time and transfer modes sub-criteria according to the accessibility criterion are 0.553 and 0.447, respectively. for the risk criterion, the relative weights of sub-criteria are disaster risk (0.682) and environmental risk (0.318). according to the parcel characteristics criterion, the relative weights for ground conditions is 0.535, for capacity expansion is 0.236, and park area is 0.229. the relative weights for the two sub-criteria of the opportunities and threats criterion are market conditions (0.339) and economic conditions (0.661). lastly, the relative weights of availability of skilled staff and availability of other staff according to the human resources criterion are 0.867 and 0.133, respectively. table 7 aggregate pairwise comparison matrix, weights, and consistency values for the subcriteria of cost cost investment cost labor cost land cost weights investment cost 1.000 3.487 3.617 0.640 labor cost 0.287 1.000 0.992 0.181 land cost 0.276 1.008 1.000 0.180 𝜆𝑚𝑎𝑥 = 3.0002210; 𝐶𝐼 = 0.0001105; 𝑅𝐼 = 0.58; 𝐶𝑅 = 0.0001906 ≤ 0.1 ijahp article: ağaç, şimşir/ optimal site selection of a pandemic hospital using multi-criteria decision-making approach international journal of the analytic hierarchy process 12 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.946 table 8 aggregate pairwise comparison matrix, weights, and consistency values for the subcriteria of demographics demographics current population population density age profile education profile weights current population 1.000 0.596 0.486 2.086 0.199 population density 1.679 1.000 1.120 3.338 0.355 age profile 2.056 0.893 1.000 2.830 0.340 education profile 0.479 0.300 0.353 1.000 0.107 𝜆𝑚𝑎𝑥 = 4.0174169; 𝐶𝐼 = 0.0058056; 𝑅𝐼 = 0.90; 𝐶𝑅 = 0.0064507 ≤ 0.1 table 9 aggregate pairwise comparison matrix and weights for the sub-criteria of accessibility accessibility travel time transfer modes weights travel time 1.000 1.239 0.553 transfer modes 0.807 1.000 0.447 table 10 aggregate pairwise comparison matrix and weights for the sub-criteria of risk risk disaster risk environmental risk weights disaster risk 1.000 2.143 0.682 environmental risk 0.467 1.000 0.318 table 11 aggregate pairwise comparison matrix, weights, and consistency values for the subcriteria of parcel characteristics parcel characteristics ground conditions capacity expansion park area weights ground conditions 1.000 3.321 2.361 0.535 capacity expansion 0.301 1.000 2.012 0.236 park area 0.424 0.497 1.000 0.229 𝜆𝑚𝑎𝑥 = 3.0000365; 𝐶𝐼 = 0.0000182; 𝑅𝐼 = 0.58; 𝐶𝑅 = 0.0000315 ≤ 0.1 table 12 aggregate pairwise comparison matrix and weights for the sub-criteria of opportunities and threats opportunities and threats market conditions economic conditions weights market conditions 1.000 0.512 0.339 economic conditions 1.951 1.000 0.661 ijahp article: ağaç, şimşir/ optimal site selection of a pandemic hospital using multi-criteria decision-making approach international journal of the analytic hierarchy process 13 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.946 table 13 aggregate pairwise comparison matrix and weights for the sub-criteria of human resources human resources availability of skilled staff availability of other staff weights availability of skilled staff 1.000 6.535 0.867 availability of other staff 0.153 1.000 0.133 according to tables 14-21, the alternative which ranks first with regard to investment cost, labor cost, and land cost is erenler with relative weights of 0.437, 0.362, and 0.483, respectively. considering the sub-criteria of demographics, the alternatives that rank first are adapazarı (0.428) for current population, serdivan (0442) for population density, adapazarı (0.471) for age profile, and serdivan (0.449) for education profile. erenler (0.433) for travel time and serdivan (0.352) for transfer modes between the sub-criteria of accessibility are more dominate than the other alternatives. the safest alternatives with respect to disaster risk and environmental risk are erenler (0.359) and adapazarı (0.399). the optimal alternatives with regard to parcel characteristics are adapazarı (0.422) for ground conditions, erenler (0.509) for capacity expansion, and erenler (0.415) for park area. according to the incentives sub-criteria, the most advantageous alternative is erenler with a relative weight of 0.489. erenler is the most attractive alternative in terms of both market conditions and economic conditions with relative weights of 0.392 and 0.454 respectively. lastly, the highest access to human resources in terms of availability of skilled staff and availability of other staff is erenler with relative weights of 0.357 and 0.453, respectively. ijahp article: ağaç, şimşir/ optimal site selection of a pandemic hospital using multi-criteria decision-making approach international journal of the analytic hierarchy process 14 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.946 table 14 aggregate pairwise comparison matrix, weights, and consistency values of the subcriteria of cost for alternatives investment cost adapazarı erenler serdivan weights adapazarı 1.000 0.694 1.092 0.297 erenler 1.441 1.000 1.671 0.437 serdivan 0.916 0.598 1.000 0.266 𝜆𝑚𝑎𝑥 = 3.0003983; 𝐶𝐼 = 0.0001991; 𝑅𝐼 = 0.58; 𝐶𝑅 = 0.0003433 ≤ 0.1 labor cost adapazarı erenler serdivan weights adapazarı 1.000 0.978 1.340 0.362 erenler 1.022 1.000 1.289 0.362 serdivan 0.746 0.776 1.000 0.276 𝜆𝑚𝑎𝑥 = 3.0004192; 𝐶𝐼 = 0.0002096; 𝑅𝐼 = 0.58; 𝐶𝑅 = 0.0003614 ≤ 0.1 land cost adapazarı erenler serdivan weights adapazarı 1.000 0.721 2.055 0.348 erenler 1.386 1.000 2.852 0.483 serdivan 0.487 0.351 1.000 0.169 𝜆𝑚𝑎𝑥 = 3.0000002; 𝐶𝐼 = 0.0000001; 𝑅𝐼 = 0.58; 𝐶𝑅 = 0.0000002 ≤ 0.1 ijahp article: ağaç, şimşir/ optimal site selection of a pandemic hospital using multi-criteria decision-making approach international journal of the analytic hierarchy process 15 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.946 table 15 aggregate pairwise comparison matrix, weights, and consistency values of the subcriteria of demographics for alternatives current population adapazarı erenler serdivan weights of comparison matrix (70%) weights of numerical data (30%) aggregate weight (100%) adapazarı 1.000 1.682 0.966 0.381 0.539 0.428 erenler 0.594 1.000 0.590 0.229 0.174 0.212 serdivan 1.035 1.694 1.000 0.391 0.288 0.360 𝜆𝑚𝑎𝑥 = 3.0000836; 𝐶𝐼 = 0.0000418; 𝑅𝐼 = 0.58; 𝐶𝑅 = 0.0000721 ≤ 0.1 population density adapazarı erenler serdivan weights of comparison matrix (70%) weights of numerical data (30%) aggregate weight (100%) adapazarı 1.000 1.645 0.729 0.339 0.323 0.334 erenler 0.608 1.000 0.495 0.214 0.248 0.224 serdivan 1.372 2.019 1.000 0.448 0.429 0.442 𝜆𝑚𝑎𝑥 = 3.0013926; 𝐶𝐼 = 0.0006963; 𝑅𝐼 = 0.58; 𝐶𝑅 = 0.0012005 ≤ 0.1 age profile adapazarı erenler serdivan weights of comparison matrix (70%) weights of numerical data (30%) aggregate weight (100%) adapazarı 1.000 1.462 1.370 0.414 0.602 0.471 erenler 0.684 1.000 1.042 0.294 0.173 0.257 serdivan 0.730 0.959 1.000 0.292 0.225 0.272 𝜆𝑚𝑎𝑥 = 3.0012637; 𝐶𝐼 = 0.0006318; 𝑅𝐼 = 0.58; 𝐶𝑅 = 0.0010894 ≤ 0.1 education profile adapazarı erenler serdivan weights of comparison matrix (70%) weights of numerical data (30%) aggregate weight (100%) adapazarı 1.000 1.817 0.627 0.326 0.492 0.375 erenler 0.550 1.000 0.432 0.193 0.135 0.176 serdivan 1.595 2.314 1.000 0.481 0.373 0.449 𝜆𝑚𝑎𝑥 = 3.0056356; 𝐶𝐼 = 0.0028178; 𝑅𝐼 = 0.58; 𝐶𝑅 = 0.0048583 ≤ 0.1 ijahp article: ağaç, şimşir/ optimal site selection of a pandemic hospital using multi-criteria decision-making approach international journal of the analytic hierarchy process 16 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.946 table 16 aggregate pairwise comparison matrix, weights, and consistency values of the subcriteria of accessibility for alternatives travel time adapazarı erenler serdivan weights adapazarı 1.000 0.476 0.622 0.213 erenler 2.101 1.000 1.180 0.433 serdivan 1.608 0.848 1.000 0.354 𝜆𝑚𝑎𝑥 = 3.0011626; 𝐶𝐼 = 0.0005813; 𝑅𝐼 = 0.58; 𝐶𝑅 = 0.0010023 ≤ 0.1 transfer modes adapazarı erenler serdivan weights adapazarı 1.000 1.148 1.020 0.350 erenler 0.871 1.000 0.823 0.297 serdivan 0.981 1.215 1.000 0.352 𝜆𝑚𝑎𝑥 = 3.0006487; 𝐶𝐼 = 0.0003243; 𝑅𝐼 = 0.58; 𝐶𝑅 = 0.0005592 ≤ 0.1 table 17 aggregate pairwise comparison matrix, weights, and consistency values of the subcriteria of risk for alternatives disaster risk adapazarı erenler serdivan weights adapazarı 1.000 0.937 1.222 0.347 erenler 1.067 1.000 1.184 0.359 serdivan 0.819 0.844 1.000 0.294 𝜆𝑚𝑎𝑥 = 3.0010319; 𝐶𝐼 = 0.0005160; 𝑅𝐼 = 0.58; 𝐶𝑅 = 0.0008896 ≤ 0.1 environmental risk adapazarı erenler serdivan weights adapazarı 1.000 1.548 1.148 0.399 erenler 0.646 1.000 0.861 0.271 serdivan 0.871 1.161 1.000 0.331 𝜆𝑚𝑎𝑥 = 3.0024901; 𝐶𝐼 = 0.0012450; 𝑅𝐼 = 0.58; 𝐶𝑅 = 0.0021466 ≤ 0.1 ijahp article: ağaç, şimşir/ optimal site selection of a pandemic hospital using multi-criteria decision-making approach international journal of the analytic hierarchy process 17 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.946 table 18 aggregate pairwise comparison matrix, weights, and consistency values of the subcriteria of parcel characteristics for alternatives ground conditions adapazarı erenler serdivan weights adapazarı 1.000 1.434 1.487 0.422 erenler 0.698 1.000 1.042 0.295 serdivan 0.673 0.959 1.000 0.283 𝜆𝑚𝑎𝑥 = 3.0000027; 𝐶𝐼 = 0.0000014; 𝑅𝐼 = 0.58; 𝐶𝑅 = 0.0000024 ≤ 0.1 capacity expansion adapazarı erenler serdivan weights adapazarı 1.000 0.590 1.680 0.306 erenler 1.695 1.000 2.711 0.509 serdivan 0.595 0.369 1.000 0.185 𝜆𝑚𝑎𝑥 = 3.0002702; 𝐶𝐼 = 0.0001351; 𝑅𝐼 = 0.58; 𝐶𝑅 = 0.0002329 ≤ 0.1 park area adapazarı erenler serdivan weights adapazarı 1.000 0.906 1.934 0.383 erenler 1.104 1.000 2.019 0.415 serdivan 0.517 0.495 1.000 0.202 𝜆𝑚𝑎𝑥 = 3.0003421; 𝐶𝐼 = 0.0001710; 𝑅𝐼 = 0.58; 𝐶𝑅 = 0.0002949 ≤ 0.1 table 19 aggregate pairwise comparison matrix, weights, and consistency values of the subcriteria of government for alternatives incentives adapazarı erenler serdivan weights adapazarı 1.000 0.514 1.137 0.265 erenler 1.944 1.000 1.892 0.489 serdivan 0.879 0.529 1.000 0.246 𝜆𝑚𝑎𝑥 = 3.0027120; 𝐶𝐼 = 0.0013560; 𝑅𝐼 = 0.58; 𝐶𝑅 = 0.0023379 ≤ 0.1 ijahp article: ağaç, şimşir/ optimal site selection of a pandemic hospital using multi-criteria decision-making approach international journal of the analytic hierarchy process 18 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.946 table 20 aggregate pairwise comparison matrix, weights, and consistency values of the subcriteria of opportunities and threats for alternatives market conditions adapazarı erenler serdivan weights adapazarı 1.000 0.699 0.908 0.284 erenler 1.430 1.000 1.166 0.392 serdivan 1.101 0.858 1.000 0.324 𝜆𝑚𝑎𝑥 = 3.0013008; 𝐶𝐼 = 0.0006504; 𝑅𝐼 = 0.58; 𝐶𝑅 = 0.0011213 ≤ 0.1 economic conditions adapazarı erenler serdivan weights adapazarı 1.000 0.623 1.228 0.295 erenler 1.606 1.000 1.732 0.454 serdivan 0.815 0.577 1.000 0.251 𝜆𝑚𝑎𝑥 = 3.0018702; 𝐶𝐼 = 0.0009351; 𝑅𝐼 = 0.58; 𝐶𝑅 = 0.0016122 ≤ 0.1 table 21 aggregate pairwise comparison matrix, weights, and consistency values of the subcriteria of human resources for alternatives availability of skilled staffs adapazarı erenler serdivan weights adapazarı 1.000 0.818 0.835 0.292 erenler 1.223 1.000 1.021 0.357 serdivan 1.198 0.980 1.000 0.350 𝜆𝑚𝑎𝑥 = 3.0000000; 𝐶𝐼 = 0.0000000; 𝑅𝐼 = 0.58; 𝐶𝑅 = 0.0000000 ≤ 0.1 availability of other staffs adapazarı erenler serdivan weights adapazarı 1.000 0.663 1.277 0.305 erenler 1.509 1.000 1.843 0.453 serdivan 0.783 0.543 1.000 0.242 𝜆𝑚𝑎𝑥 = 3.0002177; 𝐶𝐼 = 0.0001088; 𝑅𝐼 = 0.58; 𝐶𝑅 = 0.0001876 ≤ 0.1 considering table 22, risk is the most important criterion in the selection of a pandemic hospital location. following this, accessibility and opportunities and threats are ranked second and third, respectively, with a slight difference. the cost factor also ranks fourth among the other factors. the remaining factors are human resources, government, parcel characteristics, and demographics, respectively. considering global weights and rankings for the sub-criteria, disaster risk (0.140), which is one of the risk factors, is the most important sub-criterion for evaluation of the optimal selection problem. the subcriterion that ranks second is economic conditions (0.128). the travel time sub-criterion is ranked third with a global weight of 0.111. the rank order of the remaining sub-criteria with regard to their global weights are transfer modes (0.090), incentives (0.075), availability of skilled staff (0.072), investment cost (0.070), environmental risk (0.066), market conditions (0.065), ground conditions (0.039), population density (0.021), age profile (0.020), labor cost (0.020), land cost (0.020), capacity expansion (0.017), park area (0.017), current population (0.012), availability of other staff (0.011), and education profile (0.006), respectively. ijahp article: ağaç, şimşir/ optimal site selection of a pandemic hospital using multi-criteria decision-making approach international journal of the analytic hierarchy process 19 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.946 table 22 global weights and rankings level 2: main criteria level 3: sub-criteria criteria weights ranking sub-criteria local weights local ranking global weights global ranking cost 0.110 4 investment cost 0.640 1 0.070 7 labor cost 0.181 2 0.020 13 land cost 0.180 3 0.020 14 demographics 0.059 8 current population 0.199 3 0.012 17 population density 0.355 1 0.021 11 age profile 0.340 2 0.020 12 education profile 0.107 4 0.006 19 accessibility 0.201 2 travel time 0.553 1 0.111 3 transfer modes 0.447 2 0.090 4 risk 0.206 1 disaster risk 0.682 1 0.140 1 environmental risk 0.318 2 0.066 8 parcel characteristics 0.073 7 ground conditions 0.535 1 0.039 10 capacity expansion 0.236 2 0.017 15 park area 0.229 3 0.017 16 government 0.075 6 incentives 1.000 1 0.075 5 opportunities and threats 0.193 3 market conditions 0.339 2 0.065 9 economic conditions 0.661 1 0.128 2 human resources 0.083 5 availability of skilled staff 0.867 1 0.072 6 availability of other staff 0.133 2 0.011 18 according to table 23, the synthesis values that are the final relative weights of the alternatives are adapazarı (0.317), erenler (0.382), and serdivan (0.300). the optimal location selection for a pandemic hospital with regard to the synthesis values of the alternatives is erenler. the rank order of the other remaining alternatives is adapazarı and serdivan, respectively. ijahp article: ağaç, şimşir/ optimal site selection of a pandemic hospital using multi-criteria decision-making approach international journal of the analytic hierarchy process 20 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.946 table 23 synthesis value and ranking for alternatives level 4: alternatives weights sub-criteria global adapazarı erenler serdivan investment cost 0.070 0.297 0.437 0.266 labor cost 0.020 0.362 0.362 0.276 land cost 0.020 0.348 0.483 0.169 current population 0.012 0.428 0.212 0.360 population density 0.021 0.334 0.224 0.442 age profile 0.020 0.471 0.257 0.272 education profile 0.006 0.375 0.176 0.449 travel time 0.111 0.213 0.433 0.354 transfer modes 0.090 0.350 0.297 0.352 disaster risk 0.140 0.347 0.359 0.294 environmental risk 0.066 0.399 0.271 0.331 ground conditions 0.039 0.422 0.295 0.283 capacity expansion 0.017 0.306 0.509 0.185 park area 0.017 0.383 0.415 0.202 incentives 0.075 0.265 0.489 0.246 market conditions 0.065 0.284 0.392 0.324 economic conditions 0.128 0.295 0.454 0.251 availability of skilled staff 0.072 0.292 0.357 0.350 availability of other staff 0.011 0.305 0.453 0.242 synthesis value 0.317 0.382 0.300 ranking 2 1 3 3.4 sensitivity analysis a sensitivity analysis is a useful approach to measure the response of the model under changing environmental conditions. in this study, a sensitivity analysis was performed for alternatives according to the main criteria using superdecisions (version 2.10) software. the analysis results are shown in figure 3. in the graph, adapazarı, erenler, and serdivan are presented with red, blue, and black lines, respectively. in addition, relative weights corresponding to the intersection points of the lines are given in parentheses. according to the figure, the erenler region, which ranked first for the establishment of a pandemic hospital, does not change its ranking despite the changes in the relative weights of most criteria. however, the ranking order changes for the following three criteria: demographics, risk, and parcel characteristics. considering the three criteria, when the relative weight of demographics reaches 0.293, the erenler region is replaced by the adapazarı region. if the relative weight of the risk criterion exceeds 0.731, adapazarı ranks first among the alternatives. lastly, when the relative weight of parcel characteristics is equal to or more than 0.850, adapazarı becomes the most optimal location to establish a pandemic hospital instead of erenler. taking into account the current relative weights of the criteria, the demographics, risk, and parcel characteristics criteria must be increased by at least 497%, 355%, and 1164%, respectively, for these changes to occur. also, sensitivity performance is observed after ijahp article: ağaç, şimşir/ optimal site selection of a pandemic hospital using multi-criteria decision-making approach international journal of the analytic hierarchy process 21 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.946 the relative weights of each main criterion are changed by 50% (figure 4). however, despite this increase, there is no change in the rank order of all alternatives according to their relative weights. therefore, the outputs of the decision-making model are robust. (a) sensitivity analysis for cost (b) sensitivity analysis for demographics (c) sensitivity analysis for accessibility (d) sensitivity analysis for risk (e) sensitivity analysis for parcel characteristics (f) sensitivity analysis for government (g) sensitivity analysis for opportunities and threats (h) sensitivity analysis for human resources figure 3 sensitivity analysis of alternatives according to main criteria ijahp article: ağaç, şimşir/ optimal site selection of a pandemic hospital using multi-criteria decision-making approach international journal of the analytic hierarchy process 22 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.946 (a) all main criteria are decreased by 50% (b) all main criteria are increased by 50% figure 4 sensitivity analysis of alternatives when all main criteria are changed by 50% 4. discussion the covid-19 pandemic has infected more than a hundred million people all over the world, and approximately two million people have died (csse, 2021). many countries were not prepared for such a large-scale pandemic, particularly concerning the physical resources of hospitals. this unpreparedness was witnessed in the insufficiency of icu’s for patients with severe conditions and the difficulty of prevention of contact with other patients and the risk of contamination. in order to overcome this challenge, there is a need for pandemic hospitals that can provide a suitable environment for infected patients. the problem of location selection for a pandemic hospital is a critical decision because it is strategic. this study focuses on determining the optimal location selection for a pandemic hospital. the proposed model is a useful tool that will provide decision support to health managers and decision-makers in evaluating location alternatives. the results of the study show that the most effective criterion in the selection of a pandemic hospital is the risk factor. risk is very important in terms of hospital establishment. if the risk of disaster is high, the costs required to eliminate the risk will increase. in addition, risks such as air, water, noise, and even traffic will adversely affect patients receiving services from the hospital. disaster risk, which is a sub-criterion of the risk criterion, has become the most important among all the sub-criteria. this may be due to the increase in cost, as well as the fact that sakarya province is located on a fault line and is in the first-degree earthquake zone (afad, 2021). on the other hand, the environmental risk sub-criterion ranks eighth. when the criteria were determined, the experts were asked for their opinions about the criteria, and they stated that the environmental risk factor is very important for a pandemic hospital. however, the environmental risk sub-criteria ranked in the middle of the criteria. this is an interesting result, which could be because the experts think that the environmental risk factor does not have a very negative impact on the region. this idea is supported by a study conducted by the ministry of environment and urbanization on the air pollution rate, 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 cost demographics accessibility risk parcel characteristics government opportunities and threats human resources adapazarı erenler serdivan 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 cost demographics accessibility risk parcel characteristics government opportunities and threats human resources adapazarı erenler serdivan ijahp article: ağaç, şimşir/ optimal site selection of a pandemic hospital using multi-criteria decision-making approach international journal of the analytic hierarchy process 23 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.946 where it was determined that the air quality index value for sakarya province was good (aqi, 2021). this and similar evidence may explain the situation. the second interesting finding is that the opportunities and threats criterion ranks third which is above the cost criterion. in addition, whereas economic conditions ranked second among the sub-criteria, the market conditions criterion ranked in the middle. in fact, this situation should be analyzed from two perspectives. will the pandemic hospital be built by the public or the private sector? the situation can be explained according to the answer given to the question. the opportunities and threats may rank at the top if the hospitals are built by the private sector because the private sector wants to see a return on its investment in the medium or long-term. the situation depends on the correct analysis of the opportunities and threats of the region. it is expected that the sub-criteria of this criterion will also rank at the top; however, it is observed that market conditions does not meet this requirement. this may be because there are no pandemic hospitals that can compete in the region and the opening of a second pandemic hospital is not foreseen in the near future. if the plan is for the hospital to be built by the public sector, opportunities and threats cannot be expected to rank first. however, whereas the opportunities and threats refer directly to the market, a pandemic hospital established by the public sector is not built to gain profit. the pandemic hospital is already established for an extraordinary situation and has no intention of producing a market opportunity. since the priority is to control the pandemic, a pandemic hospital can be opened via the public sector even if it has negative market or economic conditions. another interesting finding is that the demographics criterion and sub-criteria ranked last. the demographic structure of the region where a hospital will be established under normal conditions is an important factor. for example, the high population will increase the number of potential patients. however, the experts stated that the demographic characteristics of the region are not important when it comes to the pandemic hospital. the underlying reason may be that there is no pandemic hospital in sakarya and neighboring provinces and the infected patients have no alternative. lastly, the criteria rankings obtained from the study were compared for both a general hospital and a pandemic hospital. first, when looking at the studies addressing the location selection problem for general hospitals, in studies conducted in taiwan and china, respectively (wu et al., 2007; lin & tsai, 2010), the government criterion was ranked first; however, it ranked last in the current study. in a similar study conducted in turkey (şahin et al., 2019), the government criterion ranked last as in the current study. this situation can be explained by the fact that the incentives that are given for the location selection of a pandemic hospital in turkey are not sufficient. the demographics criterion, which is also called the demand criterion, took the first two places in different studies (kumar et al., 2016; lin & tsai, 2010; wu et al., 2007). however, the criterion ranks last in the current study. this situation may be caused by the special situation of the pandemic hospital in which the pandemic hospital is not open to every region like general hospitals. therefore, the patient's visit to the hospital is considered independent of the demographics dimension. patients can even be transferred to this region from neighboring regions because it will be the only available pandemic hospital in the region. considering zolfani et al. (2020)'s study on a pandemic hospital, the first criterion in the study was "distance from industrial areas". in the current study, this criterion, which ijahp article: ağaç, şimşir/ optimal site selection of a pandemic hospital using multi-criteria decision-making approach international journal of the analytic hierarchy process 24 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.946 corresponds to the environmental risk factor, ranked in the middle. it should come as no surprise that the criterion ranked first because their study was carried out in istanbul which is turkey's most populous city and has advanced industrialization. in addition, whereas the land cost criterion ranked third in their study, it is in the last place in the current study. this situation can be explained by the fact that the land costs in istanbul are quite high compared to sakarya province. the remaining criteria of their study have a similar order of importance to the corresponding criteria in the current study. 5. conclusion the whole world has made intense efforts while struggling with the covid-19 pandemic to minimize its damage. during the fight against the pandemic, the physical infrastructure of general hospitals such as icu’s has been shown to be insufficient. the aim of this study is to determine the optimal location selection for a pandemic hospital and reveal the factors that affect the decision. for this purpose, the analytical hierarchy process approach was used. the application of the approach was planned as a case study. in order to score the criteria, pairwise comparison questionnaires were completed by 23 experts. the inconsistency of the completed questionnaires was checked and they were validated. the findings of the study show that the erenler district is the most optimal location for a potential pandemic hospital. in addition, the most important three main criteria that affect the location selection of the pandemic hospital are risk, accessibility, and opportunities and threats, respectively. a sensitivity analysis was also conducted in order to observe the reaction of the model in changing environmental conditions. the results of the sensitivity analysis show that the importance order of the model outputs is robust against radical changes. consequently, the study proposed a valid analytical hierarchy processbased decision support framework approach for selecting a pandemic hospital location. the proposed model will provide decision support to healthcare managers and decisionmakers. for further studies, researchers can reconsider the problem by using integrated multicriteria decision-making approaches such as ahp-vikor and ahp-topsis. moreover, it can be integrated with systems using real data such as geographic information systems. also, a fuzzy ahp approach can be used with fuzzy sets for uncertainty. in addition, integrated multi-criteria decision-making such as hesitant fuzzy ahp and spherical fuzzy ahp approaches can be applied using hesitant fuzzy or spherical fuzzy sets. ijahp article: ağaç, şimşir/ optimal site selection of a pandemic hospital using multi-criteria decision-making approach international journal of the analytic hierarchy process 25 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.946 references adalı, e. a., & tuş, a. (2019). hospital site selection with distance-based multi-criteria decision-making methods. international journal of healthcare management, 0(0), 1–11. doi: https://doi.org/10.1080/20479700.2019.1674005 afad. (2021). disaster maps. retrieved february 1, 2021, from https://www.afad.gov.tr/afet-haritalari al-shabeeb, a. r., al-adamat, r., & mashagbah, a. (2016). ahp with gis for a preliminary site selection of wind turbines in the north west of jordan. international journal of geosciences, 7(10), 1208. doi: https://doi.org/10.4236/ijg.2016.710090 alkan, n., & kahraman, c. (2021). circular intuitionistic fuzzy topsis method: pandemic hospital location selection. journal of intelligent & fuzzy systems, preprint(preprint), 1– 22. doi: https://doi.org/10.3233/jifs-219193 aqi. (2021). national air quality monitoring network. retrieved february 1, 2021, from http://index.havaizleme.gov.tr/index?stations=71,158,159 aydin, n., & seker, s. (2021). determining the location of isolation hospitals for covid-19 via delphi-based mcdm method. international journal of intelligent systems, 36(6), 3011–3034. doi: https://doi.org/10.1002/int.22410 bhushan, n., & rai, k. (2004). strategic decision making: applying the analytic hierarchy process (1st ed.). london: springer-verlag london. boyacı, a. ç., & şişman, a. (2021). pandemic hospital site selection: a gis-based mcdm approach employing pythagorean fuzzy sets. environmental science and pollution research 2021, 1–13. doi: https://doi.org/10.1007/s11356-021-15703-7 chatterjee, d., & mukherjee, b. (2013). potential hospital location selection using ahp: a study in rural india. international journal of computer applications, 71(17). doi: https://doi.org/10.5120/12447-9144 chiu, j.-e., & tsai, h.-h. (2013). applying analytic hierarchy process to select optimal expansion of hospital location: the case of a regional teaching hospital in yunlin. in 2013 10th international conference on service systems and service management (pp. 603–606). ieee. doi: https://doi.org/10.1109/icsssm.2013.6602588 colak, h. e., memisoglu, t., & gercek, y. (2020). optimal site selection for solar photovoltaic (pv) power plants using gis and ahp: a case study of malatya province, turkey. renewable energy, 149, 565–576. doi: https://doi.org/10.1016/j.renene.2019.12.078 csse. (2021). center for systems science and engineering at jhu. retrieved may 4, 2020, from https://systems.jhu.edu/ ijahp article: ağaç, şimşir/ optimal site selection of a pandemic hospital using multi-criteria decision-making approach international journal of the analytic hierarchy process 26 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.946 gdm. (2021). general directorate of mapping. retrieved january 10, 2021, from https://www.harita.gov.tr/english/ gul, m., & guneri, a. f. (2021). hospital location selection: a systematic literature review on methodologies and applications. mathematical problems in engineering, 2021, 1–14. doi: https://doi.org/10.1155/2021/6682958 jalaliyoon, n., arastoo, a., & pirouti, a. (2015). land selection: using multiple criteria decision making. international journal of academic research in management (ijarm), 4(1), 14–23. kumar, p., singh, r. k., & sinha, p. (2016). optimal site selection for a hospital using a fuzzy extended electre approach. journal of management analytics, 3(2), 115–135. doi: https://doi.org/10.1080/23270012.2016.1152170 lin, c. t., & tsai, m. c. (2010). location choice for direct foreign investment in new hospitals in china by using anp and topsis. quality and quantity, 44(2), 375–390. doi: https://doi.org/10.1007/s11135-008-9199-2 oppio, a., buffoli, m., dell’ovo, m., & capolongo, s. (2016). addressing decisions about new hospitals’ siting: a multidimensional evaluation approach. annali dell’istituto superiore di sanita, 52(1), 78–87. doi: https://doi.org/10.4415/ann_16_01_14 saaty, t. l. (1980). the analytic hierarchy process: planning, priority setting, resource allocation. new york: mcgraw-hill. saaty, t. l. (1984). the analytic hierarchy process: decision making in complex environments. in quantitative assessment in arms control : mathematical modeling and simulation in the analysis of arms control problems (pp. 285–308). new york: plenum press. doi: https://doi.org/10.1007/978-1-4613-2805-6_12 saaty, t. l. (1990). how to make a decision: the analytic hierarchy process. european journal of operational research, 48(1), 9–26. doi: https://doi.org/10.1016/03772217(90)90057-i şahin, t., ocak, s., & top, m. (2019). analytic hierarchy process for hospital site selection. health policy and technology, 8(1), 42–50. doi: https://doi.org/10.1016/j.hlpt.2019.02.005 sen, h. (2017). hospital location selection with aras-g. the eurasia proceedings of science technology engineering and mathematics, 1(1), 359–365. şen, h., & demiral, m. f. (2016). hospital location selection with grey system theory. european journal of economics and business studies, 5(1), 66. doi: https://doi.org/10.26417/ejes.v5i1.p66-79 senvar, o., otay, i., & bolturk, e. (2016). hospital site selection via hesitant fuzzy topsis. ifac-papersonline, 49(12), 1140–1145. doi: ijahp article: ağaç, şimşir/ optimal site selection of a pandemic hospital using multi-criteria decision-making approach international journal of the analytic hierarchy process 27 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.946 https://doi.org/10.1016/j.ifacol.2016.07.656 soltani, a., & marandi, e. z. (2011). hospital site selection using two-stage fuzzy multicriteria decision making process. journal of urban and environmental engineering, 5(1), 32–43. doi: https://doi.org/10.4090/juee.2011.v5n1.032043 triantono, h. b., & susetyarto, m. b. (2017). technical use of analytical hierarchy process and delphi method in determining terminal location multi function of merak port. in 2017 international conference on information management and technology (icimtech) (pp. 350–355). ieee. turkstat. (2021). turkey statistical institute. retrieved january 10, 2021, from https://www.tuik.gov.tr/ vahidnia, m. h., alesheikh, a. a., & alimohammadi, a. (2009). hospital site selection using fuzzy ahp and its derivatives. journal of environmental management, 90(10), 3048–3056. doi: https://doi.org/10.1016/j.jenvman.2009.04.010 who. (2020a). virtual press conference on covid-19 – 11 march 2020. who. (2020b). who coronavirus disease (covid-19) dashboard. retrieved december 06, 2021, from https://covid19.who.int/table wu, c. r., lin, c. t., & chen, h. c. (2007). optimal selection of location for taiwanese hospitals to ensure a competitive advantage by using the analytic hierarchy process and sensitivity analysis. building and environment, 42(3), 1431–1444. doi: https://doi.org/10.1016/j.buildenv.2005.12.016 yang, j., & lee, h. (1997). an ahp decision model for facility location selection. facilities, 15(9/10), 241–254. doi: https://doi.org/10.1108/02632779710178785 yap, j. y. l., ho, c. c., & ting, c.-y. (2019). a systematic review of the applications of multi-criteria decision-making methods in site selection problems. built environment project and asset management, 9(4), 548–563. doi: https://doi.org/10.1108/bepam-052018-0078 zolfani, s. h., yazdani, m., torkayesh, a. e., & derakhti, a. (2020). application of a gray-based decision support framework for location selection of a temporary hospital during covid-19 pandemic. symmetry, 12(6). doi: https://doi.org/10.3390/sym12060886 ijahp article: pradhan, patel, priya /measuring customer lifetime value: application of analytic hierarchy process in determining relative weights of ‘lrfm’ international journal of the analytic hierarchy process 526 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.892 measuring customer lifetime value: application of analytic hierarchy process in determining relative weights of ‘lrfm’ saurabh pradhan birla institute of management technology greater noida india saurabh.pradhan_fpm17@bimtech.ac.in gokulananda patel birla institute of management technology greater noida india gn.patel@bimtech.ac.in pankaj priya birla institute of management technology greater noida india pankaj.priya@bimtech.ac.in abstract the limited availability of resources drives retailers to tailor their resources to identified profitable customers. in the present scenario, when the roi of marketing is being questioned, the satisfaction of the profitable customers is of utmost importance as it drives their loyalty towards the retailer and the retailer’s brand. this research has considered length of association with customers (l), apart from variables like recency (r), frequency (f) and monetary-value of the purchase (m) in measuring customers’ relative-worth based on the calculation of customer lifetime-value (clv). the contribution of this article lies in calculating weights of these variables – l, r, f, and m utilizing ahp and demonstrating the calculation of clv using weighted lrfm based on data collected from a leading apparel retailer in india. the obtained results for the customer base using the proposed approach is more reliable when compared with traditional non-weighted approaches of rfm based clv. this methodology will provide a new and better option to retailers for measuring clv of their customers, thus aiding their decision making about customer-friendly profitable marketing strategies and attaining optimum returns on their investments. keywords: customer lifetime value (clv); ahp; lrfm, rfm 1. introduction neck-to-neck competition among retail firms has led them to think about ways to stay ahead in the market against their respective competitors. this competition is pushing the firms to innovate their methods to identify the right set of profitable customers and fulfill their needs from the limited available resources. this will improve customer satisfaction mailto:saurabh.pradhan_fpm17@bimtech.ac.in mailto:gn.patel@bimtech.ac.in mailto:pankaj.priya@bimtech.ac.in ijahp article: pradhan, patel, priya /measuring customer lifetime value: application of analytic hierarchy process in determining relative weights of ‘lrfm’ international journal of the analytic hierarchy process 527 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.892 and retention, and in the long term, will lead to profitability and sustainability. considering these issues in the indian market, there is a pressing need to stay ahead in the competitive market. to fulfill the aspirations of indian consumers towards their respective brands and thereby achieve optimal profitability, the retailers and marketers need to provide the right set of products to the right segment of customers in alignment with proper allocation of resources (ayoubi, 2016; jain & singh, 2002). it is important to understand the value of customers and essential to identify the most profitable customers in order for retailers to retain them (hawkes, 2000). calculation of clv has many practical applications. several authors have developed various models for these applications such as performance measurement (rust et al., 2004), marketing resources allocation (reinartz et al. 2005; ma et al., 2008), targeting customers (haenlein et al., 2006), pricing (hidalgo et al., 2008), product offering (shih & liu, 2008), and customer segmentation (rosset et al., 2002; haenlein et al., 2007; benoit & van den poel, 2009). one of the methods of segmentation could be based on the relative worth of individual customers to the retailers. relative worth of the customers is expressed through customer life time value (clv). rfm has been a popular value employed in the marketing discipline to evaluate lifetime value (ltv) or customer lifetime value (clv) for decades (gupta et al., 2006). these three variables/factors of the rfm model namely, r (recency), f (frequency) and m (monetary value) may have different impacts on various types of industries. liu and shih (2005) presented these three variables/factors with variable weights for each. an extended approach by liu and shih (2005) captured the idea of lrfm (alvandi et al., 2012; hosseini et al., 2010; lin et al., 2011; parvaneh et al., 2012; wu et al., 2014) by calculating the weight of another variable namely, length (l) or period of activity which is the entire time period for which the customer is associated with the retailer. to calculate the relative weights of l, r, m, and m (i.e., weighted lrfm or w-lrfm), the analytic hierarchy process (ahp) is used to determine the clv. the use of lrfm as a modeling tool for measuring loyalty and later for clv is based on the premise that the variables r, f, and m and later l (length) are critical in deciding the optimal clv. interpreting the results of this approach on these aforementioned variables will give a new and novel dimension to sales and marketing strategies by the company for the targeted set of customers by assigning ranks to the customers according to their relative worth to the retail organization. constrained resources with retailers demand that they invest judiciously in the most profitable customers. therefore, identification of those customers is vital for the sustenance of the retailer. given the plethora of data being captured across the customers’ journey and the availability of various analytical tools to identify the customers’ segments, identification and development of the most appropriate tool is the first step. therefore, the question arises whether the present analytical tools are effective enough to carry out the above exercise. if not, then what improvisations are required? this study has identified the gaps in the existing models of lrfm and suggested incorporation of varying weights to all variables, i.e., l, r, f, and m. ijahp article: pradhan, patel, priya /measuring customer lifetime value: application of analytic hierarchy process in determining relative weights of ‘lrfm’ international journal of the analytic hierarchy process 528 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.892 this study has been conducted with customers that have recorded repeat purchase incidence for a specific product category, i.e., apparel in a given economic, business/ store and social environment, with the assumption that any decision to focus the resources on identified profitable customers would be entirely based on the results from this analysis. 2. literature review this section familiarizes the reader with two concepts which are also discussed further in this paper in later sections. particularly, this section focuses on discussing the concept of an extended approach of clv calculation using a weighted lrfm model and the use of the ahp to calculate the weights of these said variables. 2.1 lrfm for clv calculation the value of customers to a firm or organization is determined by customer lifetime value during the life cycle of the customers (tukel & dixit, 2013). customer lifetime value helps firms and organizations allocate limited available resources to their customers by categorizing them and assigning a specific weight to each customer (greenberg, 2001). a wide variety of marketing strategies can be identified for each customer by suitable calculation of customer lifetime value which can help an organization categorize and classify its customers based on rankings of clv (hiziroglu & sengul, 2012). the rankings of clv are evaluated by one of the popular methods namely, the rfm model for emphasis on the customers who are profitable to the firm (hu & yeh, 2014). according to gupta et al. (2006), the rfm model is the most extensively used method and has been applied in the direct marketing area for more than 30 years. the rfm (recentcy of purchase, frequency of purchase and monetary value of purchase) has emerged as a more potent metric to measure the clv (safari et al., 2016; zhang et al., 2015; kumar et al., 2008). the rfm aids retailers in managing their customers in a profitable manner as resources can be allocated according to various segments of customers based on their relative worth. this ensures their sustainability over a period in the face of uncertain and volatile market conditions. this rfm model takes three major factors into account: a) recentcy of last purchase (r) which refers to the duration of time between the last customer purchase and the present time, b) frequency of the purchase (f) which refers to the total number of purchases made by the customer during their life time, and monetary value of the purchases (m) which refers to the amount of money consumed in a specific period of time (coussement et al., 2014; goodman, 1992). the rfm based clv method provides a reliable base for measuring lifetime worth of customers and understanding market segmentation with different values of recentcy, frequency and monetary value (yoseph & heikkila, 2018). there are numerous scoring methods for these three variables. in the first ever study, arthur hughes (who was the founder of the rfm model) considered a method of scoring rfm where customers were separated into five equal groups. according to him, these three variables were given the same weights to calculate a composite score (hughes, 1994). whereas, stone (1994) debated that different businesses have different natures and assigned different variable weights to the rfm measures according to the nature of the business. for example, he proposed the order as f, r, and m to analyze the value offered by customers who have used a credit card since frequency of use matters more in that case of business. more ijahp article: pradhan, patel, priya /measuring customer lifetime value: application of analytic hierarchy process in determining relative weights of ‘lrfm’ international journal of the analytic hierarchy process 529 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.892 recently, a few researchers have claimed that usually recentcy is a major deciding factor since recent customers are comparatively more capable and responsible for growth and development in a given and specified period of the time. miglautsch (2000) expressed the order of the rfm model as r, f, and m. though rfm has emerged as a significant tool for measuring clv over the years, it is plagued by some gaps. the major objection for these methods of giving scores is that the weight of each variable is determined subjectively and is primarily based on prior knowledge about the business (hong & kim, 2012). therefore, with the purpose of applying a more systematic approach, the analytic hierarchy process (ahp) was used to determine the relative weights of rfm variables to calculate clv (liu & shih, 2005). furthermore, the extended approach of liu and shih (2005) adds another variable namely, length (l) or period of activity which is the entire time period for which the customer is associated with the retailer, while capturing the idea of weighted-lrfm (hosseini et al., 2010; lin et al., 2011; parvaneh et al., 2012; wu et al., 2014). therefore, chang and tsay (2004) added this additional variable, ‘length’ into the original model of rfm, thereby extending it as a lrfm (length, recency, frequency, and monetary) model where length measures the difference in time period between the first visit and the last visit of a particular customer to a retail store. also, reinartz and kumar (2000) stated that just the rfm model alone cannot segment and explain which of the customers have short-term or long-term relationship with the company or retail firm. with the introduction of length, the relationship between the customer and the company can be determined numerically, thus removing the above gaps. it is worthy to note that the data set includes a parameter, customers’ ids (masked contact information) to identify repetitive customers of a firm that affects predictive accuracy of the research positively (malthouse & blattberg, 2005). 2.2 analytic hierarchy process the analytical hierarchy process (ahp) is a structured technique to organize and analyze complex decisions based on mathematics and psychology. it is a process of using a hierarchy to carry out a wide-ranging evaluation and final selection of one of the alternative solutions for a particular problem. it provides the objective mathematics to process the inescapably subjective and personal preferences of an individual or a group in making a decision. the technique serves as a connecting link between qualitative and quantitative data. it can also be explained in a general manner as a theory of measurement using both qualitative and/or quantitative data. taliscali and ercan (2006) demonstrated the “user-friendly” nature and the application of quantitative and qualitative factors in the evaluation of a problem as one of the fundamental advantages of the ahp, in comparison with other mcdm (multi-criteria decision making) methods. according to saaty (1990a), the ahp is a technique which is designed to help with the multiple criteria decision making (mcdm) process. the ahp is comprised of three significant components which are: (1) structuring the problem into a hierarchy consisting of objective/goal and subordinate features (decomposition), (2) pair-wise comparisons between elements at each level (evaluation), and (3) propagation of level-specific, local priorities to global priorities (synthesis). the subordinate levels of a hierarchy may consist of objectives, events, scenarios, actions, outcomes, and alternatives. ijahp article: pradhan, patel, priya /measuring customer lifetime value: application of analytic hierarchy process in determining relative weights of ‘lrfm’ international journal of the analytic hierarchy process 530 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.892 saaty (1990a) developed the mathematical model for this procedure which has two phases. the hierarchical structure of the system is prepared in the first phase. in more common terms, this involves identifying the elements that are involved in the problem and categorizing these elements into a hierarchical tree structure. all the elements which are situated at a higher hierarchical level act on the elements which are located at a level lower. in the second phase, all individual elements are evaluated and the consistency of the evaluated comparison is checked separately. the evaluation works so that all pairs of elements are compared at a given level from the point of view of each element that is located a level higher in the hierarchical structure. the end result of the comparisons is a set of matrices which, after normalization and examination of consistency, serve as the basis for the final evaluation of the problem. let 𝑛 elements 𝐶1, 𝐶2 … 𝐶𝑛 in the same level of hierarchy be considered for comparison, and let the relative weight (or significance or priority) of 𝐶𝑖 with respect to 𝐶𝑗 be denoted by 𝑎𝑖𝑗 and let it be formed into a square matrix 𝐴 = (𝑎𝑖𝑗 ) of order 𝑛 with the constraints that 𝑎𝑖𝑗 = 1 𝑎𝑗𝑖⁄ , for 𝑖 ≠ 𝑗, and 𝑎𝑖𝑖 = 1, all 𝑖. this kind of matrix is called a reciprocal matrix. if the element 𝐶𝑖 is preferred to 𝐶𝑗, then 𝑎𝑖𝑗 > 1. correspondingly, the reciprocal property, 𝑎𝑗𝑖 = 1 𝑎𝑖𝑗⁄ , for 𝑖, 𝑗 = 1, 2, 3, … . , 𝑛 , always holds true. here, 𝑎𝑖𝑗 varies from 1/9 to 9 based on satty’s fundamental scale of comparison (saaty, 2008). each set of comparisons for a level with 𝑛 elements requires n c2 = (𝑛(𝑛 − 1))/2 judgements. a positive reciprocal matrix of pair-wise comparisons 𝐴 = [𝑎𝑖𝑗 ] ∈ ɍ 𝑛×𝑛 is constructed to accommodate all the comparisons. then, the priority vector 𝜔 = (𝜔1, 𝜔2, 𝜔3, … . , 𝜔𝑛 ) may be derived from this matrix. all the elements 𝑎𝑖𝑗 have perfect values when the decision-maker is perfectly consistent in his/her judgements. so, 𝑎𝑖𝑗 = 𝜔𝑖 / 𝜔𝑗. thus, there is consistency in the weights if they are transitive in nature, that is 𝑎𝑖𝑘 = 𝑎𝑖𝑗 ∗ 𝑎𝑗𝑘 for all 𝑖, 𝑗, and 𝑘 = 1, 2, 3, … . , 𝑛. such a matrix is truly ideal and might exist only if the 𝑎𝑖𝑗 are calculated from exactly measured data. then, the pair-wise comparison matrix 𝐴 is said to be consistent and can be represented as 𝐴𝑐 = [𝜔𝑖 / 𝜔𝑗 ] . the consistent priorities are unique and readily available by taking the average of the elements in any column of the comparison matrix a, and then dividing each of them by the sum of all elements of the column. however, the evaluations of the decision-maker, 𝑎𝑖𝑗 are not perfect in real practical scenarios. these serve as estimations of the exact ratios 𝜔𝑖 / 𝜔𝑗 only. such inconsistency in comparison and judgements is more common in practical business scenarios. in that case, matrix 𝐴 is an inconsistent matrix, which can be taken as a perturbation of the consistent one 𝐴𝑐 . moreover, the inconsistent priorities are not unique and these can be derived using some error estimation technique. 3. methodology 3.1 weighted lrfm based clv using the analytic hierarchy process (ahp) 3.1.1 clv calculation the proposed extended approach of liu and shih (2005) calculates the weights of the variables of clv using the ahp while considering the fact that the relative importance of ijahp article: pradhan, patel, priya /measuring customer lifetime value: application of analytic hierarchy process in determining relative weights of ‘lrfm’ international journal of the analytic hierarchy process 531 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.892 these four variables – recency (r), frequency (f), monetary value (m) and l (length) can vary depending upon the business scenario and a causal relationship may exist among two or more variables. the relative weights along with calculated values of the variables l, r, f, and m are used to develop the clv model in accordance with the extension of liu and shih (2005) as shown in the following mathematical model below. if 𝐶𝐼 𝑗 is the integrated rating of cluster 𝑗, the mathematical equation to calculate customer lifetime value is as follows: 𝐶𝐼 𝑗 = 𝑤𝐿 𝐶𝐿 𝑗 + 𝑤𝑅 𝐶𝑅 𝑗 + 𝑤𝐹 𝐶𝐹 𝑗 + 𝑤𝑀 𝐶𝑀 𝑗 (1) where, 𝑤𝐿 = 𝑤𝑒𝑖𝑔ℎ𝑡 𝑜𝑓 𝑙𝑒𝑛𝑔𝑡ℎ, 𝐿 𝐶𝐿 𝑗 = 𝑙𝑒𝑛𝑔𝑡ℎ 𝑟𝑎𝑡𝑖𝑛𝑔 𝑓𝑜𝑟 𝑐𝑙𝑢𝑠𝑡𝑒𝑟, 𝑗 𝑤𝑅 = 𝑤𝑒𝑖𝑔ℎ𝑡 𝑜𝑓 𝑟𝑒𝑐𝑒𝑛𝑐𝑦, 𝑅 𝐶𝑅 𝑗 = 𝑟𝑒𝑐𝑒𝑛𝑐𝑦 𝑟𝑎𝑡𝑖𝑛𝑔 𝑓𝑜𝑟 𝑐𝑙𝑢𝑠𝑡𝑒𝑟, 𝑗 𝑤𝐹 = 𝑤𝑒𝑖𝑔ℎ𝑡 𝑜𝑓 𝑓𝑟𝑒𝑞𝑢𝑒𝑛𝑐𝑦, 𝐹 𝐶𝐹 𝑗 = 𝑓𝑟𝑒𝑞𝑢𝑒𝑛𝑐𝑦 𝑟𝑎𝑡𝑖𝑛𝑔 𝑓𝑜𝑟 𝑐𝑙𝑢𝑠𝑡𝑒𝑟, 𝑗 𝑤𝑀 = 𝑤𝑒𝑖𝑔ℎ𝑡 𝑜𝑓 𝑚𝑜𝑛𝑒𝑡𝑎𝑟𝑦 𝑣𝑎𝑙𝑢𝑒, 𝑀 𝐶𝑀 𝑗 = 𝑚𝑜𝑛𝑒𝑡𝑎𝑟𝑦 𝑣𝑎𝑙𝑢𝑒 𝑟𝑎𝑡𝑖𝑛𝑔 𝑓𝑜𝑟 𝑐𝑙𝑢𝑠𝑡𝑒𝑟, 𝑗 3.1.2 weights calculation using ahp according to saaty (1977), a major component of the ahp is the estimation of priorities from pairwise comparison matrices. t.l. saaty gave the eigenvector method (ev) in 1998 (saaty & hu, 1998). it has been proven by saaty that the principal eigen-vector of the comparison matrix can be used as a priority vector for consistent and inconsistent preferences (saaty, 2003). most of the alternative methods for obtaining priorities in the ahp are based on some optimization approaches. this means that they consist of an objective function or goal, which measures the distance between an actual solution and an “ideal” solution. then the problem of priority derivation comes which is to minimize this goal (or objective function) subjected to some additional constraints. one such method of optimization is the direct least squares method (dls), which tries to minimize the euclidean distance from the given comparison matrix under additive normalization constraints. another such method of optimization is the weighted least squares method (wls), which uses a modified euclidean norm as an objective function (chu et al., 1979). the logarithmic least squares method (lls) makes use of the multiplicative properties of the pairwise comparison matrices and applies a procedure of optimization which minimizes a logarithmic objective function, subject to multiplicative constraints (crawford & williams, 1985). this method gives an explicit solution, which is rather simple and convenient from a computational point of view. quite a few authors use a goal programming (g.p.) approach for solving the prioritization problem. byson (1995) describes the logarithmic goal programming method (gp), which tends to minimize a linear logarithmic function subject to some linear constraints. ijahp article: pradhan, patel, priya /measuring customer lifetime value: application of analytic hierarchy process in determining relative weights of ‘lrfm’ international journal of the analytic hierarchy process 532 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.892 currently, the most common and popular techniques for prioritization in the ahp are the ev and lls. numerous researchers have attempted to compare these two methods by evaluating their performance to determine which is best. however, their conclusions often remain contradictory. barzilai (1997), crawford & williams (1985), and zahedi (1986) assert that the lls outperforms the ev. other researchers claim that the ev is inferior to the lls (saaty, 1990b; kumar & ganesh, 1996). takeda et al. (1987) applied a greater number of criteria for comparison and tested the major prioritization techniques with a greater number of randomly generated pairwise matrices. their findings suggest that the lls is superior to the ev in some of the cases and equal in many other cases. golany & kress (1993) carried out a brilliant comparative analysis among the commonly used methods for deriving priorities. however, they inferred that there is no prioritization method that is superior to the others in any case. all the methods have their own drawbacks and advantages. moreover, the choice of method for prioritization should only be dictated by the objective of the analysis. this conclusion could defend our study in this area of clv and our efforts to develop and test using the ev approach for prioritization in the ahp. when the problem definition demands that the clv variable should be given a priority weight so that the effective clv can be calculated, the same can be identified by applying the ahp. this would help the researcher identify the factors that are comparatively influencing more towards the main decision objective. this may allow the researcher to identify the most and least important factors in driving the decision and act accordingly in the required business scenario. for the same to be effectively realized, the eigenvector method (ev) of the analytic hierarchy process (ahp) is followed. the first step before the ahp is to standardize the response data taken from the respondents. this must be done to remove the outliers in the data set. the cumulative value for each comparison is taken by taking geometric mean (g.m.) rather than the arithmetic mean (a. m.). the g.m. removes the outliers and hence, g. m. ≤ a. m. the ahp technique primarily consists of three major operations including hierarchy construction, priority analysis, and consistency verification. following the steps mentioned in ho’s flowchart (2008), initially, the complex mcdm problems are broken down into their small associated component parts. these criteria are then arranged into multiple hierarchical levels. the hierarchy is comprised of various levels, ranging from the goal (or the objective) to a variety of criteria, sub-criteria, and alternatives situated at the lowest level of the hierarchy. the highest level of the hierarchy has the decisionmaking objective. the intermediate levels are presented with the criteria that influence the decision. the alternatives occupy the bottom level of the hierarchy (if required by the research question). the first step is to develop the hierarchy tree with the overall goal objective, criteria and alternatives (if required by the research question) which is as follows: ijahp article: pradhan, patel, priya /measuring customer lifetime value: application of analytic hierarchy process in determining relative weights of ‘lrfm’ international journal of the analytic hierarchy process 533 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.892 figure 1 hierarchy tree of ahp next, each cluster is compared in the same level in a pairwise fashion based on the decision maker’s own experience and knowledge or by considering the experts’ opinion(s). for instance, every two criteria (for one of the examples, recency and frequency) in the second level are compared at each time with respect to the goal/ objective. the comparison matrix, consisting of elements (criteria) in one level in relation to elements at a higher level (goal) is constructed using individual comparisons translated into scale-values. saaty’s fundamental scale is used to quantify the preferences of decision makers (saaty, 2008). some degree of inconsistency may be present in the comparisons since they are carried out through personal or subjective judgments. a final step called a consistency check or verification is done in order to guarantee that the judgments are consistent enough, which is regarded as one of the advantages of the ahp when 𝑛 ≥ 2. it is incorporated in order to measure the degree of consistency among the pairwise comparisons by computing the consistency ratio. here, 𝑛 is the number of criteria/alternatives. the consistency ratio (c.r.) evaluates the validity of comparisons. before calculation of the c.r., it is necessary to calculate a consistency index (ci) of an 𝑛 × 𝑛 matrix, which is defined as the ratio 𝐶. 𝐼. = (𝜆𝑚𝑎𝑥 ‐ 𝑛)/(𝑛‐ 1). the value 𝜆𝑚𝑎𝑥 is the maximum eigenvalue of the matrix, and n is the matrix dimension. the consistency ratio is calculated as 𝐶. 𝑅. = 𝐶. 𝐼./𝑅. 𝐼. the r.i. value refers to the random consistency index. if the c.r. value is less than or equal to the specified value, the evaluation within the matrix is acceptable and close to ideal values. however, if it is found that the consistency ratio exceeds the limit (is greater than 10% or 0.1), the decision makers will need to review and revise the pairwise comparisons and therefore improve the evaluation process. a relatively ranked matrix for each level of the hierarchy is synthesized by the pairwise comparison. the number count of the matrix depends on the number count of elements in each level. the vector of relative weight and maximum eigenvalue (𝜆𝑚𝑎𝑥) for each matrix is calculated after all matrices have been created. once all pairwise comparisons are carried out at every level and are proven to be consistent, the judgments can then be synthesized to determine the priority ranking of every criterion and of its sub-criteria to finally calculate the global weights. 3.1.2.1 eigenvector method (ev) the eigen vector method is the original saaty approach to derive the priorities in the ahp (saaty, 1977). the ev is based on the premise that small perturbations of the clv (goal) l (criteria 1) r (criteria 2) f (criteria 3) m (criteria 4) ijahp article: pradhan, patel, priya /measuring customer lifetime value: application of analytic hierarchy process in determining relative weights of ‘lrfm’ international journal of the analytic hierarchy process 534 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.892 elements aij from the perfect ratios 𝜔𝑖 / 𝜔𝑗 will lead to small perturbations of the eigenvalues of the comparison matrix a around the eigenvalues of the consistent one 𝐴𝑐 . saaty proves that the principal eigenvector of a can be used as the desired priority vector using the frobenius theorem. the mathematical derivation of the ev method according to saaty (1977) is as mentioned. let a vector 𝜔 of order n be found such that = 𝜆𝜔 . for such a matrix, 𝜔 is said to be an eigenvector (of the order 𝑛) and 𝜆 is an eigenvalue. for a consistent matrix, 𝜆 = 𝑛. for matrices involving human judgment, the condition 𝑎𝑖𝑘 = 𝑎𝑖𝑗 ∗ 𝑎𝑗𝑘 does not hold as human judgments and are inconsistent to a greater or lesser degree. in such a case, the 𝜔 vector satisfies the equation 𝐴𝜔 = 𝜆𝑚𝑎𝑥 𝜔 and 𝜆𝑚𝑎𝑥 ≥ 𝑛. the difference, if any, between 𝜆𝑚𝑎𝑥 and 𝑛 is an indication of the inconsistency of the judgments. if 𝜆𝑚𝑎𝑥 = 𝑛, then the judgements are consistent. finally, a consistency index can be calculated from (𝜆𝑚𝑎𝑥 ‐ 𝑛)/(𝑛‐ 1). that needs to be assessed against completely random judgments and saaty has calculated large samples of random matrices of increasing order and the consistency indices of those matrices. a true consistency ratio is calculated by dividing the consistency index (c.i.) for the set of judgments by the index for the corresponding random matrix – random consistency index (r.i.). saaty suggests that if that ratio exceeds 0.1 the set of judgments may be too inconsistent to be reliable. in practice, crs of more than 0.1 have to be accepted sometimes. if cr equals 0, then that means that the judgments are perfectly consistent. mathematically, the ev method is based on solving the equation: 𝐴𝜔 = 𝜆𝑚𝑎𝑥 𝜔 and 𝜆𝑚𝑎𝑥 ≥ 𝑛 (2) this approach gives a reasonably good approximation of the priorities vector for small deviations around the perfect evaluations. however, the solutions are not that satisfactory when the inconsistency in the preferences of the decision-maker is large. ∑ 𝑎𝑖𝑗 𝜔𝑗 𝑛 𝑗=1 = 𝜆𝑚𝑎𝑥 𝜔𝑖 (3) for 𝑗 = 1, 2, 3, … . 𝑛 taking the summation over 𝑖, ∑ ∑ 𝑎𝑖𝑗 𝜔𝑗 𝑛 𝑗=1 𝑛 𝑖=1 = 𝜆𝑚𝑎𝑥 ∑ 𝜔𝑖 𝑛 𝑖=1 (4) for 𝑖 = 1,2,3, … . 𝑛 and 𝑗 = 1,2,3, … . 𝑛 this approach is built on grounds which assume small perturbations of the element 𝑎𝑖𝑗 from the perfect ratios of 𝜔𝑖 𝜔𝑗⁄ . these lead to small perturbations of the eigenvalues of the comparison matrix 𝐴. saaty proved that the principal eigenvector of 𝐴 can be used as ijahp article: pradhan, patel, priya /measuring customer lifetime value: application of analytic hierarchy process in determining relative weights of ‘lrfm’ international journal of the analytic hierarchy process 535 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.892 the desired priority vector. the ev method is based on solving the equation. it is given by the formula: 𝐴𝜔 = 𝜆𝑚𝑎𝑥 𝜔, 𝑒 𝑇 𝜔 = 1 (5) the principal eigenvector 𝜆𝑚𝑎𝑥 of a is determined by solving the characteristic equation, |𝐴 − 𝜆𝑚𝑎𝑥 𝐼| = 0 (6) then using the value of 𝜆𝑚𝑎𝑥, the eigenvector 𝜔 = (𝜔1, 𝜔2, 𝜔3 … . 𝜔𝑛 ) is determined from: (𝐴 − 𝜆𝑚𝑎𝑥 𝐼)𝜔 = 0 (7) a summary of the methodology behind the ahp involves three basic steps: analysis where the problem is defined and decomposed into a list of related elements and sub-problems (structure the complexity) which helps the decision maker focus on understanding each element according to its importance and effect on the overall process (islam & rasad, 2006). hierarchy where a structure is organized into levels of criteria and sub-criteria in relation to a given goal. a matrix of pairwise comparisons at each level is made. the lowest level of different alternatives is compared with respect to each element. processmathematical methods are used to obtain weights at each level of the hierarchy from the goal to the alternative. a consistency ratio is used to check the consistency of the judgments. to determine the importance of a criterion against another, fundamental scale is applied. this scale quantifies the degree of importance of one element to another such as a criterion against a criterion, or sub-criterion against another sub-criterion (saaty, 2008). in a nutshell, the process of developing a model to solve an ahp problem is as follows: i. identifying the decision problems, ii. listing every evaluation element, iii. setting up hierarchical relationship, iv. pair-wise comparison, v. establishing pair-wise comparison matrix, vi. calculating priority weights using any specific (here ev) method, vii. performing the consistency check if 𝑛 > 2. 3.2 research design this study has been carried out for the apparel retail business in india. the population included all customers purchasing apparel from one specific retailer in india. the data are taken across multiple stores with similar types of input to remove any specific storerelated errors in the data and align the whole data at the same coherence level. the sample had 10 identified industry experts to provide ratings through a structured ijahp article: pradhan, patel, priya /measuring customer lifetime value: application of analytic hierarchy process in determining relative weights of ‘lrfm’ international journal of the analytic hierarchy process 536 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.892 questionnaire to the variables for developing the pcm matrix, whereas purchase data from 1650 random customers were analyzed to calculate the clv. the analysis was conducted on microsoft excel using the eigen vector method of ahp (satty & hu, 1998). this study was initially based on a data set covering 25,938 transactional data points from 6581 unique customers’ pos (point of sale data) spread over 23 months. as the crm can be established, and clv can be meaningful only if a customer makes a repeat purchase, all the unique customers are filtered for those who have made repeated purchases within the time period of this study. this resulted in 1908 unique customers having repeated purchase data (length, 𝐿 ≠ 0). the final selection of data takes into account 1650 unique customers and 685 transaction dates over a span of 23 months after filtering and cleaning the data by removing 258 outliers. this data follows a few basic assumptions: (i) all the customers are exposed to nearly similar input variables, i.e., similar store environment, store format, in-store service, after sales service, to name a few. (ii) all the customers come with the intent to purchase and have actually made a purchase. as a corollary to the above, each customer will be equally valuable to the retailer in the initial stage, i.e., during initial purchase. 4. a business scenario – the case of a leading indian apparel retailer the business scenario considered for this study is based on the empirical data collected from a leading apparel retailer that has good brand equity in india. this retailer has a well-established retail chain across india and deals with all types of formal and informal apparel as well as accessories. as the data were collected overall a long period of time, seasonal purchase variations are encompassed in the data set. the national economy was also stable and brand salience was constant during the duration of the study. therefore, it is assumed that no extraneous variables impacted the results. apparel as a category exhibits frequent purchase incidence at largely regular intervals with considerable level of customer involvement and customers have the option of repeat purchases from the same retailer or switching to another retailer. therefore, this category is an appropriate selection for this study. 4.1 data and variables after calculating the values of the variables for each of the unique customers from raw pos data, the data in the variables are standardized as done in a general lp (linear programming) approach by dividing data in the variables by a common denominator. table 1 presents the data variables selected for this study along with their descriptive statistical characteristics of the standardized data. ijahp article: pradhan, patel, priya /measuring customer lifetime value: application of analytic hierarchy process in determining relative weights of ‘lrfm’ international journal of the analytic hierarchy process 537 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.892 table 1 data variables and their characteristics variables length (l) recency (r) frequency (f) monetary value (m) variable type output (o1) output (o2) output (o3) output (o4) mean 0.320878373 0.52615193 0.207234596 0.326928395 standard error 0.007946301 0.006735343 0.009214604 0.017340027 median 0.189041096 0.516438356 0.103204615 0.12121885 mode 0.002739726 0.556164384 0.24333333 0.11753 standard deviation 0.322780285 0.273590932 0.374298981 0.704355223 sample variance 0.104187113 0.074851998 0.140099727 0.496116281 kurtosis 0.357764686 -0.297473777 43.26282144 38.15094791 skewness 1.190880956 0.441257975 5.714256912 5.528247068 range 1.24109589 1.276712329 4.677325185 7.306507431 minimum 0.002739726 0.01369863 0.015531915 0.002674569 maximum 1.243835616 1.290410959 4.6928571 7.309182 sum 529.449315 868.1506849 341.9370842 539.4318518 count 1650 1650 1650 1650 maximum 1.243835616 1.290410959 4.6928571 7.309182 minimum 0.002739726 0.01369863 0.015531915 0.002674569 confidence level (95.0%) 0.015585904 0.013210727 0.018073558 0.034010792 the overall regression accuracy is determined by r 2 (coefficient of determination) and adjusted r 2 for the data containing 1650 data points. the correlation coefficient is observed to be 0.96 which is closer to 1 and suggests a linear relationship. since a count of independent variables is done more than once in this case, the adjusted r 2 (0.919171) is a better measure than r 2 (0.919367) in terms of accuracy (black, 2019). considering the aforementioned value of adjusted r 2 , it is observed that the approximately 92 % variance in the efficiencies (dependent variable) is explained by the independent variables (l, r, f and m). the adjusted r 2 adjusts for the number of terms in the model and increases with the number of independent variables whenever the predictive power of the model increases positively. considering a 95% confidence interval for the data, the pvalue for the f-statistics is so small which signifies that there is evidence that at least one of the independent variables has a linear relationship with the calculated efficiencies. considering the t-statistics, the p-values for all the independent variables (l, r, f and m) have a significant relationship with the dependent variable (calculated efficiencies of the customer) to be considered for the study. ijahp article: pradhan, patel, priya /measuring customer lifetime value: application of analytic hierarchy process in determining relative weights of ‘lrfm’ international journal of the analytic hierarchy process 538 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.892 5. analysis and results the clv for every customer is calculated using the extended approach of liu and shih (2005) where the relative weights of all four variables are calculated. this business scenario demands variable relative weights. 5.1 problem definition solved by ahp – eigenvector method the analytic hierarchy process (ahp) was used to determine the relative weights of lrfm variables using microsoft excel. table 2 pcm matrix of ahp l r r m l 1 0.53 0.63 0.48 r 1.88 1 0.66 0.71 f 1.59 1.52 1 1.29 m 2.10 1.40 0.78 1 the relative weights of the l, r, f, and m in accordance with the extension of liu and shih (2005) are as shown below in table 3. the consistency ratio is 0.01492. table 3 weights of l, r, f, and m l r f m 0.152653401 0.232611525 0.32048563 0.294249444 5.1.1. sensitivity analysis in ahp a sensitivity analysis gives insight into how the optimal solution changes when coefficients are changed in the model. to evaluate the sensitivity in the ahp, an free online software, superdecisions, was used which was developed thomas saaty’s team. it is necessary to set an independent variable to see a meaningful sensitivity graph. there is one line for each alternative in the sensitivity window. the optimum weights w1, w2, w3 and w4 are 0.152653185, 0.232611563, 0.320485744 and 0.294249508 for l, r, f and m, respectively. in this study, frequency (f) which the most significant criteria and has the highest weight comparatively, is set as an independent variable to obtain the sensitivity graph at all nine different data points as shown below. similarly, sensitivity graph for remaining three criteria can be determined in the same manner. the change in priority weights for the criteria ‘frequency’ can make the weights of the alternatives vary accordingly. to visualize the sensitivity graph, only the top four customers with the best clv has been taken into consideration as an example. ijahp article: pradhan, patel, priya /measuring customer lifetime value: application of analytic hierarchy process in determining relative weights of ‘lrfm’ international journal of the analytic hierarchy process 539 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.892 figure 1 sensitivity graph of the goal with respect to the criteria “frequency” for four alternatives figure 2 sensitivity analysis of the goal with respect to the criteria “frequency” for four alternatives at nine different data points in figure 1, the priority of the criteria ‘frequency’ is plotted on the x-axis and the priorities of the alternatives (all four customers) are plotted on the y-axis. click on the blue vertical line and drag it to change the priority of activities. sensitivity analysis freqency paramenter (actual value) freqency paramenter (rounded off value) customer 1 customer 2 customer 3 customer 4 0.108 0.1 0.2 0.191 0.285 0.324 0.206 0.2 0.194 0.177 0.288 0.341 0.304 0.3 0.188 0.163 0.29 0.359 0.402 0.4 0.182 0.149 0.293 0.376 0.5 0.5 0.175 0.135 0.296 0.394 0.598 0.6 0.169 0.121 0.299 0.411 0.696 0.7 0.163 0.107 0.301 0.428 0.794 0.8 0.157 0.093 0.304 0.446 0.892 0.9 0.15 0.079 0.307 0.463 matrix: customer selection frequency change in weights ijahp article: pradhan, patel, priya /measuring customer lifetime value: application of analytic hierarchy process in determining relative weights of ‘lrfm’ international journal of the analytic hierarchy process 540 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.892 at the point frequency = 0.5, weight of customer 4 is 0.394, weight of customer 3 is 0.296, weight of customer 2 is 0.135, and weight of customer 1 is 0.175. therefore, this graph shows that as the priority of ‘frequency’ increases, customer 4 is always the preferred choice of alternative. however, customer 2 remains the least preferred alternative out of the four alternatives. 5.2 calculated clv results based on the values for each of the four variables l, r, f and m, and their respective relative weights, the clv was calculated for each of the customers to obtain the lifetime worth of each of the customers individually. taking the mean of clv for all the customers, the average clv for all customers is 0.33398639. the standard deviation is also in the acceptable range. the clv calculation for all 1650 customers is depicted in table 4. the clv calculation for the customers is available in the mendeley dataset (http://dx.doi.org/10.17632/48ngxh788s.4#file-823d78cc-45e7-4815-b77a9347e0f34e7c). a snapshot of the highest and lowest clv is shown below. table 4 clv calculation for the customers customer_id_code l r f m clv 2620 0.939726 0.024658 4.461111 7.023006 3.645425613 3542 0.70411 0.019178 4.692857 5.951064 3.367036304 435 0.405479 0.219178 2.874375 7.309182 3.184799788 1979 1.169863 0.021918 2.28125 7.21954 3.039135377 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6002 0.027397 0.30137 0.066364 0.027242 0.103569003 811 0.030137 0.276712 0.072277 0.038415 0.103434499 4965 0.043836 0.210959 0.094805 0.031475 0.095408426 4570 0.010959 0.241096 0.082955 0.033099 0.094079657 the higher clv of a customer expresses higher profitability from these customers. the highest clv is 3.64542561 with a customer id code2620. the lowest clv is 0.09407966 with a customer id code4570. this indicates that the retail firm should invest a significant amount of resources specifically tailored for the customers having higher profitability to enhance the customer experience (cx). this will result in longterm retention of customers with the firm. on the other hand, the customers having a lower clv have a lower profitability. these customers have the potential of being associated with the existing firm provided the http://dx.doi.org/10.17632/48ngxh788s.4#file-823d78cc-45e7-4815-b77a-9347e0f34e7c http://dx.doi.org/10.17632/48ngxh788s.4#file-823d78cc-45e7-4815-b77a-9347e0f34e7c ijahp article: pradhan, patel, priya /measuring customer lifetime value: application of analytic hierarchy process in determining relative weights of ‘lrfm’ international journal of the analytic hierarchy process 541 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.892 retailers enhance their engagement with these customers to identify their need and render greater service. 6. suggestions for further discussion these calculated clvs of customers can be used in a productive manner to segregate those customers who are more profitable for the firm from those customers who are not profitable for the firm comparatively. consequently, the customers can be further divided into segments where segments that have similar clvs are clustered in order to tailor the limited available resources according to the taste, preferences and needs of these customers in each segment as a single entity rather than tailoring the resources for individual customers. retailers planning to apply this proposed model for efficient relationship marketing must identify whether this kind of model will do justice to their customer database because the strategies developed to exploit the customers may not be based entirely on this model, but also incorporate extraneous managerial decisions of the firm. for instance, the thought of investing a disproportionate amount of specific resources in specific customers’ segment or clusters makes undeniable sense when their future behavior can be perfectly predicted, but makes no sense when future behavior is unpredictable (𝑅2 = 0). consequently, should organizations invest discretionary marketing resources on the identified best and profitable segment of customers or increase the promotional cost for the less efficient and low profitable customers to initiate a robust relationship exercise? a corollary of the above would be to decide what would be the boundary value for customers with high clv compared to low clv. these are a few research questions that need to be addressed in future research. ijahp article: pradhan, patel, priya /measuring customer lifetime value: application of analytic hierarchy process in determining relative weights of ‘lrfm’ international journal of the analytic hierarchy process 542 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.892 appendix a. supplementary data supplementary data to this article can be found online from ‘mendeley data’ at http://dx.doi.org/10.17632/48ngxh788s.4 http://dx.doi.org/10.17632/48ngxh788s.4 ijahp article: pradhan, patel, priya /measuring customer lifetime value: application of analytic hierarchy process in determining relative weights of ‘lrfm’ international journal of the analytic hierarchy process 543 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.892 references alvandi, m., fazli, s., & abdoli, f. s. (2012). k-mean clustering method for analysis customer lifetime value with lrfm relationship model in banking services. international research journal of applied and basic sciences, 3(11), 2294-2302. ayoubi, m. (2016). customer segmentation based on clv model and neural network. international journal of computer science issues (ijcsi), 13(2), 31. doi: https://doi.org/10.20943/01201602.3137 barzilai, j. (1997). deriving weights from pairwise comparison matrices. journal of the operational research society, 48(12), 1226-1232. doi: https://doi.org/10.1038/sj.jors.2600474 benoit, d. f., & van den poel, d. (2009). benefits of quantile regression for the analysis of customer lifetime value in a contractual setting: an application in financial services. expert systems with applications, 36(7), 10475-10484. doi: https://doi.org/10.1016/j.eswa.2009.01.031 black, k. (2011). business statistics: for contemporary decision making. john wiley & sons. byson, n. (1995). a goal programming method for generating priorities vectors. journal of operational research society, palgrave macmillan ltd., houndmills, basingstoke, hampshire, rg21 6xs, england, 641-648. chang, h. h., & tsay, s. f. (2004). integrating of som and k-mean in data mining clustering: an empirical study of crm and profitability evaluation. journal of information management, 11, 161-203. chu, a. t. w., kalaba, r. e., & spingarn, k. (1979). a comparison of two methods for determining the weights of belonging to fuzzy sets. journal of optimization theory and applications, 27(4), 531-538. doi: https://doi.org/10.1007/bf00933438 coussement, k., van den bossche, f. a., & de bock, k. w. (2014). data accuracy's impact on segmentation performance: benchmarking rfm analysis, logistic regression, and decision trees. journal of business research, 67(1), 2751-2758. doi: 10.1016/j.jbusres.2012.09.024 crawford, g., & williams, c. (1985). a note on the analysis of subjective judgment matrices. journal of mathematical psychology, 29(4), 387-405. doi: https://doi.org/10.1016/0022-2496(85)90002-1 golany, b., & kress, m. (1993). a multicriteria evaluation of methods for obtaining weights from ratio-scale matrices. european journal of operational research, 69(2), 210-220. doi: https://doi.org/10.1016/0377-2217(93)90165-j https://doi.org/10.20943/01201602.3137 https://doi.org/10.1038/sj.jors.2600474 https://doi.org/10.1016/j.eswa.2009.01.031 https://doi.org/10.1007/bf00933438 https://doi.org/10.1016/0022-2496(85)90002-1 https://doi.org/10.1016/0377-2217(93)90165-j ijahp article: pradhan, patel, priya /measuring customer lifetime value: application of analytic hierarchy process in determining relative weights of ‘lrfm’ international journal of the analytic hierarchy process 544 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.892 goodman, j. (1992). leveraging the customer database to your competitive advantage. (retail/database). direct marketing, dec, 1, 4. greenberg, p. (2001). crm at the speed of light: capturing and keeping customers in internet real time. mcgraw-hill professional. gupta, s. et al. (2006). modeling customer lifetime value. journal of service research, 9, 139-150. doi: http://dx.doi.org/10.1177/1094670506293810 haenlein, m., kaplan, a. m., & beeser, a. j. (2007). a model to determine customer lifetime value in a retail banking context. european management journal, 25(3), 221234. doi: http://dx.doi.org/10.1016/j.emj.2007.01.004 haenlein, m., kaplan, a. m., & schoder, d. (2006). valuing the real option of abandoning unprofitable customers when calculating customer lifetime value. journal of marketing, 70(3), 5-20. hawkes, v. a. (2000). the heart of the matter: the challenge of customer lifetime value. crm forum resources. hidalgo, p., manzur, e., olavarrieta, s., & farias, p. (2008). customer retention and price matching: the afps case. journal of business research, 61(6), 691-696. doi: http://dx.doi.org/10.1016/j.jbusres.2007.06.046 hiziroglu, a., & sengul, s. (2012). investigating two customer lifetime value models from segmentation perspective. procedia-social and behavioral sciences, 62, 766-774. doi: http://dx.doi.org/10.1016/j.sbspro.2012.09.129 ho, w. (2008). integrated analytic hierarchy process and its applications–a literature review. european journal of operational research, 186(1), 211-228. doi: https://doi.org/10.1016/j.ejor.2007.01.004 hong, t., & kim, e. (2012). segmenting customers in online stores based on factors that affect the customer’s intention to purchase. expert systems with applications, 39(2), 2127-2131. doi: http://dx.doi.org/10.1016/j.eswa.2011.07.114 hosseini, s. m. s., maleki, a., & gholamian, m. r. (2010). cluster analysis using data mining approach to develop crm methodology to assess the customer loyalty. expert systems with applications, 37(7), 5259-5264. doi: http://dx.doi.org/10.1016/j.eswa.2009.12.070 hu, y. h., & yeh, t. w. (2014). discovering valuable frequent patterns based on rfm analysis without customer identification information. knowledge-based systems, 61, 7688. doi: https://doi.org/10.1016/j.knosys.2014.02.009 hughes, a. (1994). strategic database marketing: the masterplan for starting and managing a profitable, customer-based marketing program. 1st ed. chicago, il: probus publishing co., chicago, il. http://dx.doi.org/10.1177/1094670506293810 http://dx.doi.org/10.1016/j.emj.2007.01.004 http://dx.doi.org/10.1016/j.jbusres.2007.06.046 http://dx.doi.org/10.1016/j.sbspro.2012.09.129 https://doi.org/10.1016/j.ejor.2007.01.004 http://dx.doi.org/10.1016/j.eswa.2011.07.114 http://dx.doi.org/10.1016/j.eswa.2009.12.070 https://doi.org/10.1016/j.knosys.2014.02.009 ijahp article: pradhan, patel, priya /measuring customer lifetime value: application of analytic hierarchy process in determining relative weights of ‘lrfm’ international journal of the analytic hierarchy process 545 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.892 islam, r., & bin mohd rasad, s. (2006). employee performance evaluation by the ahp: a case study. asia pacific management review, 11(3). jain, d., & singh, s. s. (2002). customer lifetime value research in marketing: a review and future directions. journal of interactive marketing, 16(2), 34. doi: http://dx.doi.org/10.2224/sbp.2012.40.7.1057 kumar, n. v., & ganesh, l. s. (1996). a simulation-based evaluation of the approximate and the exact eigenvector methods employed in ahp. european journal of operational research, 95(3), 656-662. kumar, v., venkatesan, r., bohling, t., & beckmann, d. (2008). practice prize report— the power of clv: managing customer lifetime value at ibm. marketing science, 27(4), 585-599. lin, s. y., wei, j. t., weng, c. c., & wu, h. h. (2011). a case study of using classification and regression tree and lrfm model in a pediatric dental clinic. international proceedings of economic development and research—innovation, management and service, 14, 131-135. liu, d. r., & shih, y. y. (2005). integrating ahp and data mining for product recommendation based on customer lifetime value. information & management, 42(3), 387-400. doi: 10.1016/j.im.2004.01.008 ma, m., li, z., & chen, j. (2008). phase-type distribution of customer relationship with markovian response and marketing expenditure decision on the customer lifetime value. european journal of operational research, 187(1), 313-326. doi: http://dx.doi.org/10.1016/j.ejor.2007.03.018 malthouse, e. c., & blattberg, r. c. (2005). can we predict customer lifetime value?. journal of interactive marketing, 19(1), 2-16. doi: https://doi.org/10.1002/dir.20027 miglautsch, j. r. (2000). thoughts on rfm scoring. journal of database marketing & customer strategy management, 8(1), 67-72. doi: https://doi.org/10.1057/palgrave.jdm.3240019 parvaneh, a., abbasimehr, h., & tarokh, m. j. (2012). data mining application in retailer segmentation based on lrfm variables: case study. global journal on technology, 1. pradhan, s. (2021). excel calculations & tabular data for the article titled "measuring customer lifetime value: application of analytic hierarchy process in determining relative weights of ‘lrfm’" (mendeley data; version v4) [data set]. mendeley data. doi: http://dx.doi.org/10.17632/48ngxh788s.4. reinartz, w. j., & kumar, v. (2000). on the profitability of long-life customers in a noncontractual setting: an empirical investigation and implications for http://dx.doi.org/10.2224/sbp.2012.40.7.1057 http://dx.doi.org/10.1016/j.ejor.2007.03.018 https://doi.org/10.1002/dir.20027 https://doi.org/10.1057/palgrave.jdm.3240019 http://dx.doi.org/10.17632/48ngxh788s.4 ijahp article: pradhan, patel, priya /measuring customer lifetime value: application of analytic hierarchy process in determining relative weights of ‘lrfm’ international journal of the analytic hierarchy process 546 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.892 marketing. journal of marketing, 64(4), 17-35. doi: https://doi.org/10.1509%2fjmkg.64.4.17.18077 reinartz, w., thomas, j. s., & kumar, v. (2005). balancing acquisition and retention resources to maximize customer profitability. journal of marketing, 69(1), 63-79. doi: http://dx.doi.org/10.1509/jmkg.69.1.63.55511 rosset, s., neumann, e., eick, u., vatnik, n., & idan, y. (2002, july). customer lifetime value modeling and its use for customer retention planning. in proceedings of the 8th acm sigkdd international conference on knowledge discovery and data mining (pp. 332-340). doi: https://doi.org/10.1145/775047.775097 rust, r. t., lemon, k. n., & zeithaml, v. a. (2004). return on marketing: using customer equity to focus marketing strategy. journal of marketing, 68(1), 109-127. doi: http://dx.doi.org/10.1509/jmkg.68.1.109.24030 saaty, t. l. (1977). a scaling method for priorities in hierarchical structures. journal of mathematical psychology, 15(3), 234-281. doi: https://doi.org/10.1016/00222496(77)90033-5 saaty, t. l. (1990a). how to make a decision: the analytic hierarchy process. european journal of operational research, 48(1), 9-26. doi: https://doi.org/10.1016/03772217(90)90057-i saaty, t. l. (1990b). eigenvector and logarithmic least squares. european journal of operational research, 48(1), 156-160. doi: https://doi.org/10.1016/03772217(90)90073-k saaty, t. l. (2003). decision-making with the ahp: why is the principal eigenvector necessary. european journal of operational research, 145(1), 85-91. saaty, t. l. (2008). decision making with the analytic hierarchy process. international journal of services sciences, 1(1), 83-98. saaty, t. l., & hu, g. (1998). ranking by eigenvector versus other methods in the analytic hierarchy process. applied mathematics letters, 11(4), 121-125. doi: https://doi.org/10.1016/s0893-9659(98)00068-8 safari, f., safari, n., & montazer, g. a. (2016). customer lifetime value determination based on rfm model. marketing intelligence & planning, 4(4), 446-461. doi: https://doi.org/10.1108/mip-03-2015-0060 shih, y. y., & liu, d. r. (2008). product recommendation approaches: collaborative filtering via customer lifetime value and customer demands. expert systems with applications, 35(1-2), 350-360. stone, b. (1994). successful direct marketing methods. new york, ny: ntc business books, mcgraw-hill education. https://doi.org/10.1509%2fjmkg.64.4.17.18077 http://dx.doi.org/10.1509/jmkg.69.1.63.55511 https://doi.org/10.1145/775047.775097 http://dx.doi.org/10.1509/jmkg.68.1.109.24030 https://doi.org/10.1016/0022-2496(77)90033-5 https://doi.org/10.1016/0022-2496(77)90033-5 https://doi.org/10.1016/0377-2217(90)90057-i https://doi.org/10.1016/0377-2217(90)90057-i https://doi.org/10.1016/0377-2217(90)90073-k https://doi.org/10.1016/0377-2217(90)90073-k https://doi.org/10.1016/s0893-9659(98)00068-8 https://doi.org/10.1108/mip-03-2015-0060 ijahp article: pradhan, patel, priya /measuring customer lifetime value: application of analytic hierarchy process in determining relative weights of ‘lrfm’ international journal of the analytic hierarchy process 547 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.892 takeda, e., cogger, k. o., & yu, p. l. (1987). estimating criterion weights using eigenvectors: a comparative study. european journal of operational research, 29(3), 360-369. taslicali, a. k., & ercan, s. (2006). the analytic hierarchy and the analytic network processes in multicriteria decision making: a comparative study. journal of aeronautics and space technologies, 2(4), 55-65. tukel, o. i., & dixit, a. (2013). application of customer lifetime value model in maketo-order manufacturing. the journal of business and industrial marketing, 28(6), 468474. wu, h. h., lin, s. y., & liu, c. w. (2014). analyzing patients’ values by applying cluster analysis and lrfm model in a pediatric dental clinic in taiwan. the scientific world journal, 2014. yoseph, f., & heikkila, m. (2018, december). segmenting retail customers with an enhanced rfm and a hybrid regression/clustering method. in 2018 international conference on machine learning and data engineering (icmlde) (108-116). ieee. doi: https://doi.org/10.1109/icmlde.2018.00029 zahedi, f. (1986). a simulation study of estimation methods in the analytic hierarchy process. socio-economic planning sciences, 20(6), 347-354. https://doi.org/10.1109/icmlde.2018.00029 ijahp article: mu/ahp conflict resolution in action: the peruvian hostage crisis of 1996-1997 revisited international journal of the analytic hierarchy process 362 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.517 ahp conflict resolution in action: the peruvian hostage crisis of 1996-1997 re-visited enrique mu carlow university business administration 3333 fifth ave, pittsburgh, pa 15213 emu@carlow.edu abstract the ahp analysis of the peruvian hostage crisis of 1996-1997 is particularly significant because it was the first, and to my knowledge, only time that an ahp analysis of a hostage situation was done while the events were unfolding (saaty and mu, 1997). it shows the power of hierarchical thinking in terms of summarizing a complex crisis into two simple hierarchies of 3 to 4 criteria and 4 alternatives. also, this study shows the power of sensitivity analysis to compensate for the lack of experts to estimate the true criteria weights of the parties and, finally, the analysis was completed within the fiveweek period of the crisis duration and on time to potentially influence the outcome. keywords: conflict resolution; hostage crisis; negotiation; ahp conflict; ahp crisis 1. introduction i still recall the morning of december 18, 1996, when following my daily routine of stopping by tom’s office to chat prior to beginning my work as program director at the university, i asked him, “have you read the news? do you know what has happened in peru?” he told me he had not and i proceeded to tell him that the night before, 22 members of the revolutionary movement tupac amaru (mrta) had stormed a diplomatic reception at the japanese ambassador’s residence in lima, peru. they had taken hundreds of hostages and, among them, cabinet members and even a close relative of the president at the time, alberto fujimori. they were asking for the release of their jailed comrades, in particular their jailed leader victor polay. next, i made the comment that would change my life. “i bet the president would welcome an analysis about what to do using your famous ahp methodology”, i said with a smile. tom jumped off his chair and said, “you are right. let’s do it!” i must confess, i was caught off guard, but managed to react. i said, “tom, i just know that you are the creator of the ahp method, but i have never either studied or used it. i am not the research partner you need.” i thought i was safe, but this was not possible with tom. he replied, “you are right.” i felt relieved for a moment, but then he added, while handing me a book, “here is my book ‘decision making for leaders’ that explains all there is to know about the method (saaty, 1980). read it and let’s meet tomorrow at 9 am to start working on this conflict analysis”. he added, “don’t fail me. i need you because the news on this crisis is all in spanish (my native language)”. i realized i had enrolled un-voluntarily into a crash course on ahp and its applications. i did not know at the time that this event would drastically change my professional life. this paper summarizes the study we did, emphasizing the mailto:emu@carlow.edu ijahp article: mu/ahp conflict resolution in action: the peruvian hostage crisis of 1996-1997 revisited international journal of the analytic hierarchy process 363 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.517 methodology and lessons learned in the process, which i believe, may be useful to many readers facing the task of analyzing similar conflicts (saaty and mu, 1997). 2. literature review from the onset, we decided to develop two hierarchies reflecting the points of view of the two parties in conflict: the government and the mrta guerrillas. this approach was consistent with previous ahp studies on conflict resolution (bennett and saaty, 1979). how could we know their objectives? the mrta members were talking to the press and asking for the release of their 400 hundred or so jailed comrades as well as their leader victor polay. the government was unresponsive; however, the press in both peru and the rest of the world were speculating about the government’s possible objectives and actions. therefore, i started collecting all the news from the major publications in peru and the u.s. that were dealing with the crisis and i used three colors of markers to highlight: possible government objectives (green), possible mrta objectives (yellow) and possible alternatives or actions (blue). given that the mrta guerrilla had stated what they wanted (release of their jailed comrades) and were waiting, then the immediate action or alternative would be that of the government. development of objectives our review of the press clips provided several dozen of the government’s objectives and a few of mrta’s; however, it became evident after a while that the government’s objectives being discussed could be categorized mainly into three major themes: saving hostage lives, keeping the government’s image (the current government had been elected based on its iron hand policy toward rebel guerrillas and terrorism) and the obvious police action of preventing terrorism (“peru terror”, 1996; caretas,1997). similarly, an identification of the proposed objectives for the mrta guerrillas as reported by the media yielded the following: release of jailed comrades, publicizing their cause, destabilizing the government and escaping unharmed (“peru stalemate”, 1997a; “perua month gone”, 1997b). concerning this last objective, there had not been any suicide attack in spite of the long history of violence, so it was reasonable to assume that the mrta members expected to escape unharmed (oviedo, 1989). development of alternatives the development of alternatives required not only reading newspapers and listening to expert opinions in the media but also obtaining some from key literature on previous hostage crisis. in effect, two famous former hostage crises had been the munich olympics where all the israeli olympic delegation had been kidnapped and killed and the entebbe rescue where israeli commandos had successfully rescued kidnapped passengers from an air france flight (miller, 1980). from this review of the literature, we concluded the following: first, it was possible to categorize the possible government actions on a continuum that ranged from not negotiating to storming the place to accepting all the mrta demands. second, the expert literature indicated that rescuing about 70 hostages (the mrta guerrillas had released the remaining ones due to the physical impossibility of keeping hundreds of hostages in the building) would most likely translate into great loss of life, mainly among the hostages (miller, 1980). the result of the literature review allowed us to diagram the two hierarchies for the government and mrta respectively as shown in table 1. ijahp article: mu/ahp conflict resolution in action: the peruvian hostage crisis of 1996-1997 revisited international journal of the analytic hierarchy process 364 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.517 table 1 government and mrta’s objectives* government’s goal defuse crisis terrorist’s goal end crisis successfully key objectives 1 keep government image 2 prevent terrorism 3 save hostage lives key objectives 1 release comrades 2 escape unharmed 3 publicize their cause 4 destabilize government government alternatives 1 do not negotiate 2 negotiate possible government actions (1) police action storm the building (2) isolate terrorists and demand unconditional surrender (3) request terrorists release hostages assuring them safe passage to leave the country (4) request terrorists release hostages granting them safe passage out of the country, and other concessions such access to media, money, political recognition, etc. (5) safe passage, money and releasing key mrta leaders out of the country (6) safe passage, money and releasing key mrta leaders and other members (not involved in bloody actions) out of the country (7) safe passage, money and releasing all mrta leaders out of the country (8) safe passage, money and releasing key mrta leaders in the country (9) safe passage, money and releasing key mrta leaders and other members (not involved in bloody actions) in the country (10) safe passage, money and releasing key mrta leaders (not involved in bloody actions) in the country and becoming a political party (11) safe passage, money and releasing all mrta leaders and other members in the country (12) safe passage, money and releasing all mrta leaders in the country and making mrta a political party. * source: saaty and mu (1997). a review of the alternatives in table 1 suggests that we can further categorize them into four different types: storm the building (1), demand unconditional surrender (2), provide concessions without releasing mrta jailed members (3-4) and finally, release jailed mrta prisoners (5-12). based on this and our previous discussion in this section we proposed the hierarchies below to analyze the conflict (figures 1 and 2). ijahp article: mu/ahp conflict resolution in action: the peruvian hostage crisis of 1996-1997 revisited international journal of the analytic hierarchy process 365 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.517 figure 1. government hierarchy source: saaty and mu (1997) figure 2.mrta hierarchy source: saaty and mu (1997) 3. methodology the pairwise comparisons to obtain the priorities shown in figures 1 and 2 for both criteria and alternatives in the government and mrta’s hierarchy respectively were done by the authors. the results corresponding to both perspectives as well as their product aggregation are shown in table 2. ijahp article: mu/ahp conflict resolution in action: the peruvian hostage crisis of 1996-1997 revisited international journal of the analytic hierarchy process 366 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.517 table 2 government and mrta hierarchies source: saaty and mu (1997) notice that for our original analysis, all the government objectives were considered equally important (0.333 each) because there was no way to guess any particular preference although one would expect hostage lives to be high in the list. on the other hand, release comrades (0.680) was considered to be at the top of the list of mrta’s objectives. based on this preliminary analysis, releasing mrta guerrillas (relterp) in one way or another is the most desirable alternative. sensitivity analysis the obvious argument to the preliminary analysis is the question of how valid comparisons judgments made by the two authors of this study could be to model this problem. the answer is that there is no way to know. a solution could be to perform a sensitivity analysis, taking into account all possible combinations. the sensitivity analysis for the case of the government and mrta’s perspectives are shown in tables 4a and 4b respectively. government’s perspective mrta’s perspective key objectives priority key objectives priority govimag 0.333 relcom 0.680 hostlive 0.333 unharmd 0.171 prevterr 0.333 publicit 0.076 govimag 0.072 potential government actions product of the two vectors for compromise uncsurr 0.387 (1) uncsurr 0.084 (4) 0.387 x 0.084 = 0.03 safpasm 0.228 (2) safpasm 0.242 (2) 0.228 x 0.242 = 0.06 relterp 0.208 (3) relterp 0.590 (1) 0.208 x 0.590 = 0.12 stmbldg 0.178 (4) stmbldg 0.085 (3) 0.178 x 0.085 = 0.02 ijahp article: mu/ahp conflict resolution in action: the peruvian hostage crisis of 1996-1997 revisited international journal of the analytic hierarchy process 367 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.517 table 4a sensitivity analysis from the government’s perspective major objectives: all equal government image hostages lives preventing terrorism both government image and preventing terrorism both hostages lives and preventing terrorism both government image and hostages lives govima g 0.333 0.703 0.076 0.243 0.414 0.102 0.442 hostliv e 0.333 0.149 0.703 0.051 0.160 0.448 0.453 prevter r 0.333 0.149 0.221 0.706 0.426 0.450 0.105 desirable government action: demand unconditional surrender demand unconditional surrender demand unconditional surrender safe passage or release of jailed terrorists demand unconditional surrender demand unconditional surrender demand unconditional surrender uncsur r 0.387 (1) 0.446 (1) 0.265 (3) 0.520 (1) 0.457 (1) 0.352 (1) 0.337 (1) safpas m 0.228 (2) 0.198 (3) 0.303 (2) 0.136 (3) 0.183 (3) 0.245 (3) 0.264 (2) relter p 0.208 (3) 0.135 (4) 0.314 (1) 0.118 (4) 0.150 (4) 0.251 (2) 0.236 (3) stmbld g 0.178 (4) 0.221 (2) 0.117 (4) 0.226 (2) 0.210 (2) 0.152 (4) 0.163 (4) ijahp article: mu/ahp conflict resolution in action: the peruvian hostage crisis of 1996-1997 revisited international journal of the analytic hierarchy process 368 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.517 table 4b sensitivity analysis from mrta’s perspective major objectives: release comrades escape unharmed and publicity escape unharmed and government image release comrades and government image release comrades and publicity release comrades and escape unharmed relco m 0.680 0.048 0.048 0.454 0.458 0.449 unhar md 0.170 0.452 0.454 0.041 0.041 0.457 publici t 0.076 0.455 0.043 0.047 0.455 0.049 govim ag 0.072 0.045 0.455 0.458 0.046 0.045 desirable government action: in all cases, the most convenient government action is the release of jailed mrtas. the second best government action is safe passage for the hostages. uncsur r 0.084 (4) 0.100 (3) 0.093 (4) 0.066 (4) 0.073 (4) 0.101 (3 ) safpas m 0.242 (2) 0.340 (2) 0.304 (2) 0.190 (2) 0.246 (2) 0.314 (2 ) relter p 0.590 (1) 0.480 (1) 0.497 (1) 0.618 (1) 0.587 (1) 0.511 (1 ) stmbld g 0.085 (3) 0.080 (4) 0.105 (3) 0.125 (3) 0.094 (3) 0.074 (4 ) ijahp article: mu/ahp conflict resolution in action: the peruvian hostage crisis of 1996-1997 revisited international journal of the analytic hierarchy process 369 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.517 4. conclusions our conclusions can be divided into two parts: those corresponding to the original study and those derived from re-visiting this study later. conclusions from the original study (saaty and mu, 1997) the key conclusions were: a. the three driving issues in any peruvian government decision are hostage’s lives, government image and prevention of terrorism; b. the government’s course of action is driven by the priority assigned to each of these three issues (figure 4-a); c. negotiating, by releasing some terrorists, and, perhaps, giving mrta political negotiation, is the best course of action if hostage lives have greater priority (70.3%) than the other two objectives (figure 4-a, third column). d. if either government image or prevention and punishment of terrorism is the main objective, then the best course of action is to demand unconditional surrender (figure 4-a, second and fourth columns), though it could lead to dire consequences. conclusions from re-visiting the study in hindsight and following our declared interest to discuss the lessons we learned from this study, we can state that: a. developing a model of a conflict requires a qualitative analysis of the existing literature plus gathering expert opinions. in a sense, interviewing experts should follow techniques similar to those used in the development of grounded theory and similar qualitative expert elicitation techniques (corbin and strauss, 2008; charmaz, 2006). in addition, identifying objectives and alternatives out of secondary literature sources (e.g. newspaper clips) requires the use of theme identification techniques used in qualitative studies (saldaña, 2016). b. the previous observation suggests the importance of qualitative research training for researchers and practitioners using ahp/anp and similar mcdm methodologies that are strongly based on decision maker’s opinions. to my knowledge, not even basic qualitative research is currently considered part of the training of mcdm researchers and professionals. the emphasis is still on the quantitative aspects of the discipline. c. never underestimate the power of simple models to provide insights into complex problems. the hierarchies used in this study were relatively simple: 3 levels, no more than 4 criteria and alternatives; however, this simple structure provided us with the insights necessary to analyze a complex conflict. d. sensitivity analysis is key to providing insights about a decision even when the proper criteria weights cannot be accurately determined. in the original study, it was not possible for the researchers to interview either the government or mrta actors during the crisis to elicit comparison judgments on the criteria and alternatives. however, the sensitivity analysis (figures 4a and 4b) clearly delineated the possible courses of action based on different weightings of the actors’ objectives. the crisis aftermath on april 22, 1997, peruvian commandos stormed the japanese embassy through an underground tunnel and rescued the hostages, with the loss of two commandos and one ijahp article: mu/ahp conflict resolution in action: the peruvian hostage crisis of 1996-1997 revisited international journal of the analytic hierarchy process 370 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.517 hostage (who had previous health issues) while all 14 mrta guerrillas were killed in the attack (the new york times, 1997). while the attack was a military success, it was possible to know later on that at least one of the guerrillas had had the opportunity to kill most or all of the hostages but hesitated because of what has been labeled stockholm reverse syndrome (association for diplomatic studies and training, 2017). in other words, attacking instead of negotiating could have had disastrous results. in any case, our ahp analysis did not recommend a specific action per se but provided rather a set of actions which could be chosen based on the government’s own view of the importance of its objectives. my personal aftermath this study was sent to the peruvian media at the end of february 1997, and also discussed by mainstream newspapers. an article indicated that the study had been analyzed by the government (el sol, 1997). while there is no way to know if, and to what extent, this study may have influenced the events, it piqued my interest in the field of decision making. ahp and tom became part of my life ever since! ijahp article: mu/ahp conflict resolution in action: the peruvian hostage crisis of 1996-1997 revisited international journal of the analytic hierarchy process 371 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.517 references association for diplomatic studies and training (2017). moments in u.s. diplomatic history: the dramatic hostage crisis and daring rescue at the japanese embassy in peru. retrieved from: http://adst.org/2013/12/the-dramatic-hostage-crisis-and-daring-rescue-atthe-japanese-embassy-in-peru/#.wfstoxnsxbx bennet, j.p. and saaty, t.l. (1979). terrorism: patterns for negotiation. a case study using hierarchies and holarchies. in kupperman, r. and trent, d. (ed). terrorism: threat, reality, response. hoover press. stanford university, stanford, ca. caretas. (1997, january, 23). las opinions a negociar, 24. charmaz, k. (2006). constructing grounded theory. first edition. new york: sage publications inc. corbin, j. and strauss, a. (2007). basics of qualitative research: techniques and procedures for developing grounded theory. third edition. new york: sage publications. doi: http://dx.doi.org/10.4135/9781452230153 el sol. (1997, february, 24). alternativas a la crisis de los rehenes. lima – peru. miller, abraham h. (1980). terrorism and hostage negotiation. boulder, co: westview press. doi: https://doi.org/10.2307/1962278 o’neill, michael j. (1986). terrorist spectaculars : should tv coverage be curbed? twentieth century fund. new york, ny: priority press. oviedo, carlos v. (1989). prensa y subversion: una lectura de la violencia en el peru. mass comunicacion ed. peru-terror. (1996, december, 21). the economist, 52. peru-stalemate. (1997a, january, 4). the economist (1997a), 44. perua month gone and still waiting. (1997b, january, 14). the economist (1997b), 44. saaty, t. l. (1980). the analytic hierarchy process. new york: mcgraw-hill. saaty, t. l. and mu, e. (1997). the peruvian hostage crisis of 1996-1997: what should the government do?. socio-economic planning sciences, 31(3), 165-72. doi: https://doi.org/10.1016/s0038-0121(97)00015-3 saldaña, j. (2015). the coding manual for qualitative researchers, third edition. new york: sage publications. doi: https://doi.org/10.1108/qrom-08-2016-1408 schemo, dj. (1997, april, 26) how peruvian hostage crisis became trip into the surreal. the new york times (1997). http://adst.org/2013/12/the-dramatic-hostage-crisis-and-daring-rescue-at-the-japanese-embassy-in-peru/#.wfstoxnsxbx http://adst.org/2013/12/the-dramatic-hostage-crisis-and-daring-rescue-at-the-japanese-embassy-in-peru/#.wfstoxnsxbx http://dx.doi.org/10.4135/9781452230153 https://doi.org/10.2307/1962278 https://doi.org/10.1016/s0038-0121(97)00015-3 https://doi.org/10.1108/qrom-08-2016-1408 ijahp news and events: mu/ahp/anp community transitions: twice distinguished professor, birsen karpak, retires from youngstown state university, usa international journal of the analytic hierarchy process 408 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.908 ahp/anp community transitions: twice distinguished professor birsen karpak retires from youngstown state university, usa our beloved ijahp editorial board member and associate editor, dr. birsen karpak, has announced her retirement from youngstown state university (ysu), usa, where she has worked for the past 36 years. fortunately for us, she will remain active with the ijahp journal. dr. karpak’s recent research has been mainly in ahp/anp applications in supply chain management. she has taught enterprise resource management systems and is a certified sap professional. she plans to continue teaching sap integrated courses at ysu and helping her colleagues integrate sap into their classes. during her career, she published in a variety of prestigious journals, including the international journal of production economics, the european journal of purchasing and supply management, the journal of multi‐criteria decision analysis, the transportation journal, the international journal of logistics systems and management, and, of course, our own journal, the international journal of the analytic hierarchy process. https://scholar.google.com/citations?user=kyxdfryaaaaj at her retirement, dr. birsen karpak was the second most cited faculty member at the williamson college of business (wcba), youngstown state university. she loves deep teaching/learning, preparing “business ready” students, providing meaningful service, doing high-quality research with her colleagues at ysu and her international colleagues, and has taught hundreds of students. she has always been happiest when her students get jobs at fortune 500 companies, especially when they tell her they got those jobs because https://scholar.google.com/citations?user=kyxdfryaaaaj ijahp news and events: mu/ahp/anp community transitions: twice distinguished professor, birsen karpak, retires from youngstown state university, usa international journal of the analytic hierarchy process 409 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.908 of her classes or the enterprise resource planning (erp) certification they received while studying at wcba. while this is not the place to list all the different honors and awards dr. karpak received during her career, it is worth noting that she has twice had the unique distinction of being named a distinguished professor at her institution. ysu awards the title of distinguished professor to faculty members who are outstanding in teaching, research, and service. dr. karpak was the first professor at wcba to be given the title of distinguished professor for the second time in april this year. birsen karpak receiving the first title of distinguished professor from provost martin abraham in april 2015 what is particularly important for us is that she has been a close member and significant contributor to our ahp/anp community since the early days. as ijahp colleagues and friends, we wish her the best in her new professional and personal stage of life. she hopes to be even better in her professional activities and continue contributing to her beloved ahp/anp community from now on. ijahp article: ganguly, merino/ an integrated ahp-qfd approach for evaluating competing technological processes international journal of the analytic hierarchy process 539 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.315 an integrated ahp-qfd approach for evaluating competing technological processes anirban ganguly * assistant professor department of management and entrepreneurship hofstra university, ny abhroganguly@gmail.com donald merino professor of economics of engineering school of systems & enterprise, stevens institute of technology, hoboken, nj donald.merino@stevens.edu abstract a major challenge for decision makers in business organization is making appropriate choices among competing high-tech projects. the objective of this paper is to explore a multi-criteria analytical model that can be used for the selection and management of competing manufacturing technologies. the model uses an integrated approach combining analytic hierarchy process (ahp) and quality function deployment (qfd) as the basis for selecting a preferred alternative from a set of competing projects. integration of the two techniques helps to provide a more effective selection process. two competing chemical processes to produce drugs are used as a case study to demonstrate and validate the ahp – qfd model. the policy makers of pharmaceutical and chemical organizations can use this model as a part of their strategic planning and decision-making process. keywords: analytical hierarchy process (ahp); quality function deployment (qfd); multi-criteria decision making; technology evaluation; technology assessment; technology management 1. introduction policy makers in manufacturing organizations are often involved in the selection of competing manufacturing technologies and processes. all selection mechanisms somehow reflect the selection of what creates the most net value to the buyer. * acknowledgement the authors’ would like to express their sincere gratitude to late prof. ajay k. bose and prof. s. n. ganguly for their invaluable support and advice during the research process. a special acknowledgment also goes to prof. arijit bhattacharya, prof. bijan sarkar and prof. sanat kumar mukherjee for inspiring this research. finally, the authors’ want to thank the anonymous reviewers for their valuable comments, which helped to improve the quality of the paper mailto:donald.merino@stevens.edu ijahp article: ganguly, merino/ an integrated ahp-qfd approach for evaluating competing technological processes international journal of the analytic hierarchy process 540 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.315 technology is viewed as one of the major factors that determine the competitiveness of an industry (huang et al., 2008). as a result, the selection of a proper manufacturing process and the technology behind it generally proves to be a key aspect of a firm’s strategic decision-making process. an improper technology selection may adversely affect the manufacturing process of an organization. this may result in reduction of productivity and profitability of the organization. choosing among competing technologies is driven by the need to fulfill customer requirements in the targeted market segment (partovi, 2007). there are also many other factors, ranging from the availability of resources to the specific technology constraints that might prevail in an industry that impact the decision. hence, the selection of technologies is not dependent upon a single criterion, but on a variety of factors. selection and assessment of technology often involves decision-making tools and techniques that are vital to the growth and profitability of the organization and involves the analysis of a large number of tangible and intangible factors (georgakellos, 2005). therefore, using approaches that take into account wide ranging attributes – both subjective and objective is an important aspect in the selection of an alternative from a set of competing technologies (atthirawong & maccarthy, 2002). the purpose of the paper is to evaluate an integrated multi-criteria approach for the selection of competing manufacturing processes. the integrated analytical approach combines analytical hierarchy process (ahp) and quality function deployment (qfd). ahp allows the decision-makers to successfully accommodate both objective and subjective judgments of the evaluators involved in order to make trade-offs and determine priorities among them (allama et al., 2012). ahp, due to its flexibility and ease of use, can be integrated with other decision-making techniques like qfd, anp, and so on. this combination allows the consolidation of both qualitative as well as quantitative factors – thereby providing the policy makers with a more accurate and realistic decision (vaidya & kumar, 2006; ho, 2008). in the context of the current research, ahp is used to determine the pair-wise relationship among the customer requirements, and qfd is used to assess the relationship among the customer and technical requirements. based on the relationship coefficients among the customer and technical requirements, ahp is once again used to determine the best alternative. a more in-depth discussion about the research process is provided later. the integrated ahp-qfd approach used in this paper has been utilized previously in the literature. however, a review of literature on this technique clearly indicated that this has not been utilized to determine the selection criteria among alternative capital projects in the pharmaceutical drug manufacturing sector. furthermore, the ahp-qfd technique used in this paper is simple to conduct and attempts to stay away from complicated mathematics, while making sure that the technical and customer requirements are taken care of. therefore, through this research technique, the authors develop a simple, yet novel technique for selection of alternative techniques for pharmaceutical drug manufacturing and thereby contribute to the body of knowledge. the current research tries to shed some light on utilization of an integrated ahp-qfd model to determine the preferred choice of alternative in the chemical and pharmaceutical sector. the integrated ahp-qfd methodology, coupled with the illustrative case study in this paper, can serve as valuable tool for managers to make decisions regarding the choice of competing projects in the pharmaceutical domain. ahp is effective for ijahp article: ganguly, merino/ an integrated ahp-qfd approach for evaluating competing technological processes international journal of the analytic hierarchy process 541 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.315 quantifying qualitative knowledge through the idea of multi-criteria decision-making, thus harboring subjective as well as objective factors in the process, thereby allowing intangible dimensions such as subjective preferences and comfort to be measured (mustafa & al-bahar, 1991; pecchia et al., 2013). on the other hand, qfd is a tool under the umbrella of total quality management (tqm) that incorporates the voice of the customer in the decision-making process (hauser & clausing, 1988). thus, the use of a decision-making model combining ahp with qfd provides management with a more rational basis upon which decisions can be made, while taking into account the quantitative as well as the qualitative factors of decision-making. the paper begins with a review of the literature on analytic hierarchy process (ahp) and quality function deployment (qfd) along with an overview of the processes compared as a part of the illustrative example. next, a flowchart depicting the overall research process is presented. this is followed by an illustrative case example used to explain the various stages of the decision process and data analysis along with the final selection decision. section 5 makes a critique of the integrated ahp-qfd approach, and the article ends discussing the relevance of the integrated model in the domain of operational management and management science, its usefulness to management practitioners and direction for future research. 2. theoretical background of the concepts 2.1. overview of ahp the analytical hierarchy process (ahp), which was developed by t. l. saaty, was created to deal with models that have intangible criteria or both tangible and intangible criteria (saaty, 1980, 2011). it is considered one of the most widely used multiple criteria decision-making techniques. ahp is a decision-making process for prioritizing alternatives attributes when multiple criteria are considered as a part of the decisionmaking process. ahp uses properties of reciprocal matrices to achieve consistency in pair-wise judgments leading to a cardinal ranking of actions, objectives, attributes and criteria relevant to the decision situation (hughes, 2009). it allows the decision maker to structure the problem through establishing priorities by means of a hierarchic breakdown of the problem, while taking into account the consistencies of the emitted judgments (melon et al., 2008). the process of ahp starts with the construction of a hierarchy that describes the problem to be tackled. an ahp hierarchy can have as many levels as needed to fully characterize a particular decision situation (dave et al., 2012). while constructing the hierarchy, the overall objective of the project is always placed at the top of the hierarchical tree and the main attributes a level below it. the sub-attributes are placed on subsequent levels of the hierarchy and the last level consists of the alternatives from which the selection is to be made. after constructing the hierarchy, the next step in developing an ahp model is to derive the weights of the lowest level of attributes through a series of pair–wise comparisons where each attribute of that particular hierarchical level is compared with its sibling with respect to their relative importance to each other. the pair-wise comparisons are generally made relative to their importance/ desirability, and are normally based on a numerical scale. the pair-wise comparisons are denoted in terms of the relative importance of an attribute with respect to the final alternative decisions. table 1 shows the nine-point ahp scale along with an explanation of each of the scale levels. ijahp article: ganguly, merino/ an integrated ahp-qfd approach for evaluating competing technological processes international journal of the analytic hierarchy process 542 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.315 table 1 scale for pair-wise comparison using ahp relative intensity definition explanation 1 equally preferred the two attributes in question (i and j) are of equal importance 3 slightly more preferred one variable is a little more important than the other 5 moderately preferred one variable is much more important than the other 7 highly preferred one variable is very much more important than the other 9 extremely preferred one variable is extremely more important than the other reciprocal (1/3, 1/5, 1/7, 1/9) if attribute i has one of the above numbers assigned to it when compared with attribute j, then j has the value 1/number assigned to it when compared with i. more formally if nij = x then nji = 1/x, where nji = the pair-wise comparison between the i th and the j th attribute (adapted from saaty, 1980; lang and merino, 1993) after the comparisons are made, they are converted into a numeric scale and entered into a matrix. the resulting data is normalized so that it adds up to one. after the comparison has been completed, the results are combined into a composite score that expresses how well each of the alternatives fits the overall objective (focus) of the decision-making process. in this context, it is worthwhile to mention that the ahp has two options of performing an operation – the distributive mode and the ideal mode (dolan 2000; liberatore &nydick, 1993). the ideal mode of synthesis, which is used more frequently than the distributive mode, is followed in this research. after the final composite score has been calculated and the overall value of the alternatives has been deduced, the last step of the ahp process is that of making the actual decision based on the overall values of the alternatives. the alternative yielding the highest ahp value is chosen. one can conclude from the above that an ahp analysis helps the decision maker to gain valuable insight into the relative merits of the available decision options. the ahp is a structured method that can elicit more information from target respondents (usually experts or decision makers) (cheng & lee, 2001). another important advantage of the ahp is that pair wise decisions can be tested for consistency (through a consistency ratio) to ensure results that are more rational in nature. in addition, several available techniques of sensitivity analysis demonstrate how changes in the pair-wise comparisons of the criteria weights might affect the result. 2.2. overview of qfd qfd is defined as “a method for developing a design quality aimed at satisfying the customer and then translating the customer’s demands into design targets and major quality assurance points to be used throughout the production phase’’ (akao, 1995). it is a systematic process that is often used to focus the attention of an enterprise towards its customers (chakraborty & dey, 2007). the advantage of qfd lies in the fact that it takes ijahp article: ganguly, merino/ an integrated ahp-qfd approach for evaluating competing technological processes international journal of the analytic hierarchy process 543 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.315 into account the “voice of the customers” and tries to integrate it with the planning, design and development process of a particular product or service (cardoso et al., 2015). qfd helps any product development team to specify the customer requirements and evaluate each proposed product systematically in terms of its impact in meeting those requirements (bhattacharya et al., 2005; hauser and clausing, 1988; wasserman, 1993). the basic tool for qfd is the house of quality (hoq), which, in simple terms, is a conceptual map that provides the means of inter-functional planning and communication (hauser and clausing, 1988). hoq is a matrix of matrices and the diagram generally looks like a house where the customer requirements and the resulting technical requirements are ranked and prioritized according to their relationship with each other in order to arrive at an optimal system design (prasad & chakraborty, 2013; reid &hermann, 1989; hauser & clausing, 1988; chakraborty & dey, 2007). a qfd exercise begins with gathering the customer needs and requirements and subsequently translating them into design/technical specifications. this ultimately allows an organization to make better decisions, thereby streamlining its design and manufacturing process (bahil & chapman, 1993). this, in turn, increases the overall growth of an organization and assists it in gaining a competitive advantage in the industry. in a market driven by customer needs and requirements, the primary task of a qfd team is to gather the critical technical/process parameters for a particular process/technology that are most likely to affect the customer requirements. determining the prioritized importance of these requirements often forms the next step of the process. the subsequent steps consist of drawing a correlation among the customer requirements and the technical parameters in order to determine (or rank) the important technical parameters for the process. finally, the operating conditions set the process parameters and control limits in such a way that the product standards and customer requirements are met before ultimately arriving at a preferred technology for the manufacturing process (chakraborty & dey, 2007). combining the two powerful decision-making techniques in ahp and qfd into an integrated model can prove to be a very important decision tool for an organization. the integrated technique prioritizes customer requirements, while at the same time dealing with the actual performance measurement of a specific technology or process. this, therefore, helps the decision makers to arrive at a more accurate and rational decision regarding the choice of a particular alternative among a set of alternatives. furthermore, the model incorporates both the technical aspects of a manufacturing process as well as the customer requirements that are essential for the decision-making process. as a result, the integrated technique covers a broader spectrum of evaluation criteria, thereby making the selection process more robust and accurate in nature, as will be illustrated by this research. 2.3. overview of the chemical processes to be compared a case study evaluates two competing chemical processes in the domain of drug discovery and development. the first is a conventional heating (conductive heating) method of chemical reaction. this method has proven successful for over 150 years, but has lately been challenged by an emerging technology i.e., the microwave assisted method of organic synthesis (maos). pharmaceutical companies and biotech firms are increasingly exploring this technology and applying it for library synthesis and medicinal ijahp article: ganguly, merino/ an integrated ahp-qfd approach for evaluating competing technological processes international journal of the analytic hierarchy process 544 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.315 chemistry, for lead discovery and optimization, and even for scale-up (marx, 2004). this is mainly due to its ability to reduce reaction time, which increases the speed of the chemical reaction and the drug discovery process (kappe & dallinger, 2006). reactions that required hours using the conventional method were completed in minutes using maos, usually with higher yield and purer products. microwave chemistry leads to rapid (minutes instead of hours) reactions, higher yield and purer and more environmentally friendly products. apart from that, microwave chemistry can generate special products that are not easily produced under traditional chemistry procedures. the conductive method, which has been prevalent in the industry for over a century, has certain advantages over the maos, a technology that has challenged its existence over the last two decades or so. the first and foremost advantage that the conductive method has over the maos is its ability to conduct large scale reactions. this is primarily due to the fact that although improving rapidly, there is still a dearth of industrial microwave reactors at present. additionally, it is not possible to carry out chemical reactions at room temperature under microwave irradiation which is very much possible to do by conventional method, thereby often serving as a stumbling block for maos. finally, a chemical reaction in which one of the reactants is to be added drop wise in the reaction mixture at a certain temperature is not possible with microwave irradiation, where as there is no difficulty carrying out the same reaction by the conventional method. the benefits of microwave irradiation are finding an increased role in process chemistry, especially in cases where conventional methods require forcing conditions or prolonged reaction times (wathey et al., 2002). over the last few years, maos has managed to gain a firm foothold in the area of drug discovery and development (wathey et al., 2002; kappe, 2003; collins, 2010). several pharmaceutical organizations, who were initially skeptical of the maos technique, are gradually adapting the technology into their manufacturing logistics. although microwave synthesis has a high initial capital cost, it allows a high return on investment (roi) in a very short period of time, thus making it even more popular with the pharmaceutical sector (kappe, 2003). as a result, maos serves as a key component of it currently being executed by a plethora of pharmaceutical, agrochemical and biotech companies as a frontline methodology in their chemistry programs (kappe, 2003; collins, 2010). 3. the integrated ahp-qfd research model this paper explores an integrated ahp/qfd model that is used to choose between two competing technology alternatives. figure 1 provides a basic flowchart of the generalized decision model using the integrated ahp-qfd approach. ijahp article: ganguly, merino/ an integrated ahp-qfd approach for evaluating competing technological processes international journal of the analytic hierarchy process 545 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.315 figure 1. research model as seen in figure 1, ahp is used to prioritize the customer and the technical attributes while qfd is combined with ahp to explore the relationship between the customer demands and the technical attributes. the importance of qfd in this model is based on the fact that it helps to determine the relationship between customer and technical requirements. therefore, having an integrated evaluative model which combines both ahp and qfd will enable the decision-maker/engineering managers to arrive at a more authentic and robust decision taking into account both customer and technical aspects of the process. the subsequent sections of the paper discuss in detail the above stages along with the techniques of data gathering and analysis that were followed in each of the stages. 4. data analysis and research results to validate the integrated ahp/qfd model, a case study in the research consisted of evaluating two alternative drug development procedures – the conventional vis-à-vis the microwave method of drug manufacturing. an integrated multi-criteria technique combining the analytical hierarchy process (ahp) along with quality function deployment (qfd) was used to arrive at a decision regarding the preferred technology. based on these analyses a decision was made about the competing technologies. 4.1. identification of customer and technical requirements customer and technical requirements were identified through surveys and in-depth discussions with experts in the field of synthetic chemistry and drug development. even though the literature on synthetic chemistry revealed the presence of a large number of attributes, the empirical evidence indicated that not all of the attributes were of equal importance in the selection process (ganguly & merino, 2007). hence, the first step in identifying the customer and technical requirements was an in-depth interview with subject matter experts. since one of the selected chemical processes was a part of the illustrative case example, the number of subject matter experts chosen was not a very large pool. a set of twenty five experts were surveyed to obtain the data for the research. the experts surveyed were eminent research scientists in the field of microwave chemistry and drug development who had produced over 500 published journal and conference articles. they also served as consultants for various major pharmaceutical organizations over the last 25 years (ganguly &merino, 2007). a survey involving a ijahp article: ganguly, merino/ an integrated ahp-qfd approach for evaluating competing technological processes international journal of the analytic hierarchy process 546 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.315 group of experts was used to narrow down the number of attributes from the initially selected list. the purpose of the survey was to convert the large set of initially listed requirements into a smaller subset comprising only the critical attributes. the survey consisted of a structured questionnaire where the respondents were asked to rate the importance of each of the attributes on a scale of 1 – 5, with 1 being least important and 5 being extremely important. based on their responses, the results were aggregated and the top five attributes chosen for final analysis. condensing the set of attributes helps to minimize the large number of attributes which can make it increasingly difficult for the survey respondents to make pair-wise comparisons (tam & tummala, 2001). the top five customer requirements and six critical technical requirements that were thought to be most influential in determining the final preferred alternative were identified based on the respondents’ feedback. tables 2 and 3 list the finalized set of customer and technical requirements that were used in the integrated ahp-qfd model along with their operational definition. ijahp article: ganguly, merino/ an integrated ahp-qfd approach for evaluating competing technological processes international journal of the analytic hierarchy process 547 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.315 table 2 customer requirements and their operational definition customer requirements operational definition 1 reaction time time taken to complete the chemical reaction 2 yield the total yield of the final product 3 environmental benefits the negative effect that the energy and other by-products generated from the chemical reaction has on the environment 4 cost of the reaction the total cost associated with the chemical reaction 5 revenue generated the revenue generated from selling the final manufactured product table 3 technical requirements and their operational definition 4.2. prioritization of customer requirements the first stage of customer requirements (cr) prioritization consisted of designing a questionnaire in conformity with the decision hierarchy. the vital customer requirements relevant to the decision-making process (table 2) were listed to form a matrix for pairwise comparisons. the pair-wise comparisons were performed with respect to the customer requirements, given the overall objective of the decision-making process. the next step involved inviting experts in the field of drug development and synthetic chemistry to complete the designed questionnaire. the pair-wise comparison matrix obtained from the set of evaluators was then combined to determine the consensus pair-wise comparison matrix (saaty, 1980). expert choice 2000® (http://www.expertchoice.com/) was the tool used to perform the ahp analysis and the results subsequently recorded. as stated previously, based on the expert panel input, five attributes emerged as the most important. thus, only 10 judgments [(n (n-1)/2] were required from each of the participants. in the case of the proposed research, the small number of attributes considered kept the analysis and the pair-wise comparison to a manageable proportion. the matrix was subsequently normalized in order to obtain the unique priority weights for each of the attributes (saaty, 1980; tam & tummala, 2001). technical requirements operational definitions 1 controlled reaction condition the ability to control and monitor the reaction conditions and environment 2 minimal solvent use the amount of solvent required to conduct the reaction successfully 3 minimal waste materials the amount of waste materials in the form of chemicals, solvents and other by-products generated as a part of the reaction process 4 energy saving reaction the amount of energy that is saved as a result of using a particular manufacturing process 5 upward scalability of the reaction the ability of the reaction to be conducted on a large scale basis from an laboratory basis 6 reproducibility of the reaction the ability of the reaction to be reproduced / repeated over and over again with the same level of success http://www.expertchoice.com/ ijahp article: ganguly, merino/ an integrated ahp-qfd approach for evaluating competing technological processes international journal of the analytic hierarchy process 548 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.315 table 4 provides a pair-wise comparison among the selected customer requirements along with their mean normalized weights and the consistency ratio. table 4 pair-wise comparison among the customer requirements and their normalized weights raw weights attributes yield reaction time env. ben. revenue cost normalized weights of customer requirements consistency ratio yield 1 5 3 1/7 1/5 0.107 0.14* reaction time 1/5 1 3 1/7 1/5 0.053 env. ben. 1/3 1/3 1 1/9 1/7 0.032 revenue 7 7 9 1 5 0.564 cost 5 5 7 1/5 1 0.244 * the consistency ratio indicated a slight variation from the acceptable range. the explanation for this is provided later. the experts surveyed as a part of the research analysis were asked to evaluate the customer requirements based on the scale provided in table 1. for example, if one of the customer requirements, for example yield, was somewhat preferred over reaction time, the intersection cell between yield (on the row) and reaction time (in the column) would have a value of 5 while the intersection cell between yield (on the column) and reaction time (on the row) would have a value of 1/5. furthermore, it should be mentioned here that the above table depicts the pair-wise comparison among the attributes as provided by one of the experts surveyed and is not a composite mean of the complete survey feedback. the feedback received from the other evaluators was analyzed in a similar fashion and the results obtained were used as part of the final research results. 4.3. construction and analysis of the qfd correlation matrix the next stage of the research process consisted of developing the qfd correlation matrix, which forms the backbone of the qfd analysis. a correlation matrix was constructed to determine the relative importance of the technical requirements (tr). as stated earlier, the critical technical requirements were determined based on the interviews with the experts and through literature reviews. a correlation matrix template was subsequently developed where the survey respondents were asked to map the effect of the listed technical requirements on the customer requirements using a scale of zero to five (0 – 5) where zero signified no correlation and five a very high correlation. the degree of correlation provided by the respondents was used (with the previously determined prioritized customer requirements weights to determine the relative importance of the technical requirements with respect to the customer requirements. this was determined using equation 1 given below: ijahp article: ganguly, merino/ an integrated ahp-qfd approach for evaluating competing technological processes international journal of the analytic hierarchy process 549 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.315 𝑊𝑗 = ∑ 𝑋𝑖𝑗 ∗ 𝑌𝑖 𝑛 𝑖=1 (1) where, wj = relative importance of the j th technical requirement x ij = correlation between the i th customer requirement and j th technical requirement in the qfd matrix and y i = prioritized weights of the i th customer requirement also, since all of the crs were considered while determining the overall prioritized weight of the trs, the equation is summed over the entire set of customer requirements, thereby indicating i = 1 – n in the equation. the final step in this process was to normalize the degree of importance of the technical requirements determined through equation 1. table 5 depicts the correlation matrix and the overall importance of the technical requirements along with their normalized value. table 5 the qfd matrix denoting the relative importance of the technical requirements with respect to customer requirements c o n tr o ll e d r e a c ti o n c o n d it io n m in . s o lv e n t u se m in . w a st e e n e r g y s a v in g r e a c ti o n u p w a r d s c a la b il it y r e p r o d u c ib il it y prioritized weights of customer requirements (from table 4) yield 2 4 4 1 3 3 0.107 reaction time 5 3 3 3 2 2 0.053 environmental benefit 3 4 4 4 1 2 0.032 revenue generated 1 3 2 1 5 5 0.564 cost incurred 4 4 1 1 3 4 0.244 degree of importance of the tr ( wj) 2.115 3.383 2.087 1.202 4.011 4.287 normalized wj (kj) 0.124 0.198 0.122 0.070 0.235 0.251 once again it should be mentioned here that the above table only depicts the relative importance of the technical requirements with respect to the customer requirements as provided by one of the experts surveyed and not a composite mean of the complete survey feedback. since the emphasis was on understanding the degree of association ijahp article: ganguly, merino/ an integrated ahp-qfd approach for evaluating competing technological processes international journal of the analytic hierarchy process 550 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.315 among the customer and technical requirements and not the direction of the relationship, the signs of the correlation coefficient were ignored at this stage. 4.4. prioritization of the technical requirements with respect to the alternatives the next stage of the decision process involved ranking two alternative chemical processes based upon the six trs. a different survey was constructed and the respondents were asked to compare the two alternative chemical processes based upon the trs that were already enlisted and ranked. tables 6 – 11 provide the reader with the result of the analysis. table 6 pair-wise comparison among the alternatives with respect to controlled reaction condition attributes raw weights mean normalized value c-ratio conventional microwave conventional 1 1 0.500 0.00 microwave 1 1 0.500 total 1.000 table 7 pair-wise comparison among the alternatives with respect to minimum solvent use attributes raw weights mean normalized value c-ratio conventional microwave conventional 1 1/5 0.167 0.00 microwave 5 1 0.833 total 1.000 table 8 pair-wise comparison among the alternatives with respect to minimum waste materials attributes raw weights mean normalized value c-ratio conventional microwave conventional 1 1/5 0.167 0.00 microwave 5 1 0.833 total 1.000 ijahp article: ganguly, merino/ an integrated ahp-qfd approach for evaluating competing technological processes international journal of the analytic hierarchy process 551 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.315 table 9 pair-wise comparison among the alternatives with respect to energy saving reaction attributes raw weights mean normalized value c-ratio conventional microwave conventional 1 1/7 0.125 0.00 microwave 7 1 0.875 table 10 pair-wise comparison among the alternatives with respect to upward scalability of the reaction attributes raw weights mean normalized value c-ratio conventional microwave conv. chem. 1 5 0.833 0.00 mic. chem. 1/5 1 0.167 total 1.000 table 11 pair-wise cmparison among the alternatives with respect to reproducibility of the reaction attributes raw weights mean normalized value c-ratio conventional microwave conventional 1 1 0.500 0.00 microwave 1 1 0.500 total 1.000 although mentioned previously, it should be pointed out again that the above set of tables shows the pair wise comparison among the attributes as provided by one of the experts surveyed and not a composite mean of all the survey feedback. the survey feedback from the other respondents was analyzed in similar fashion and the overall composite results from all the surveys were used as the guiding factor for the final decision regarding the process choice. 4.4. computation of the final value for the alternatives the final stage of the decision framework consists of computing the overall ahp-qfd values of the alternatives and ranking these alternatives in their order of importance. the final overall value was calculated using equation 2 given below: 𝐴𝑥 = ∑ 𝑘𝑗 ∗ 𝑏𝑖𝑗 𝑛 𝑗=1 (2) ijahp article: ganguly, merino/ an integrated ahp-qfd approach for evaluating competing technological processes international journal of the analytic hierarchy process 552 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.315 where, ax = overall score for the alternative chemical processes kj = normalized weights of the j th technical requirement bij = value of the j th alternative on the i th technical requirements 4.5. selecting the preferred alternative the technology with the highest overall value was selected as the preferred choice based on the integrated ahp-qfd model. the final overall value of the alternatives as provided by one of the experts is given in table 12. table 12 final values of the alternatives overall weights for the alternatives with respect to the technical requirements technical requirements prioritized weights for the technical requirements conventional chemistry microwave chemistry controlled reaction condition 0.124 0.500 0.500 minimal solvent use 0.198 0.167 0.833 minimal waste materials generated 0.122 0.167 0.833 energy saving reaction 0.070 0.125 0.875 upward scalability of the reaction process 0.235 0.833 0.167 reproducibility of the reaction 0.251 0.500 0.500 final overall score 0.4454 0.5545 as can be seen from table 12, the final overall value of the microwave method was higher than the conventional heating method for drug development. hence, it can be concluded that when using the integrated ahp-qfd technique for process selection, the maos method is clearly the preferred choice for the decision makers. as mentioned repeatedly, the values provided so far comprised the feedback from one of the experts. the survey responses from the other experts were analyzed in a similar fashion and the mean of all the overall scores was used to arrive at the conclusion. the mean overall scores along with the standard deviation and range for the two alternatives based on all the responses are shown in table 13. ijahp article: ganguly, merino/ an integrated ahp-qfd approach for evaluating competing technological processes international journal of the analytic hierarchy process 553 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.315 table 13 mean overall values for the alternatives conventional method microwave method mean overall values 0.4436 0.5564 standard deviation 0.0130 0.0130 results from table 13 indicated that the microwave method for chemical process is a better alternative because it had the highest overall value. furthermore, the low standard deviation values indicated that the experts fairly agreed with each other in spite of having been surveyed separately, thus adding robustness to the research. 4.6. are the pair-wise inputs consistent? the consistency ratio analysis indicated a small variation. the mean of all consistency ratios for pair-wise comparison of the customer requirements came out to be 0.15, which was slightly more than the acceptable range (≤ 0.10). the primary reason is that pair-wise comparison among the attributes selected was not transitive. for example, the relative importance of reaction time being greater than the yield and the relative importance of yield being greater than the environmental benefit does not necessarily signify that reaction time will hold a position of more importance than its environmental benefit. as a result, the final ahp judgment values were kept intact and were not revised to lower the consistency ratio to within the permissible range. according to saaty (2001), evaluators often make tradeoffs that violate transitivity but, overall, are accurate in their judgment since they take into account the relative importance of the criteria themselves. there are times when an evaluator cannot make a clear decision because the tradeoffs among several activities are the same and are not related to some other pair-wise judgment (saaty, 1980). also, tam et al. (2006) states that the root of this problem stems from the 9-point scale of relative importance proposed by saaty. the scale assumes that the decision-makers understand well the relationship and the magnitude of differences among various decisions under consideration. however, in practice, using such a complicated scale makes it extremely difficult to achieve an absolute consistency in the evaluation process (tam et al., 2006). they recommend the use of “non-structural fuzzy decision support system” (nsfdss) as an alternative tool in order to reduce (or even eliminate) the problem of inconsistency, something that should definitely be an important part of future research (tam et al., 2006). 5. critique of the integrated ahp-qfd approach the integrated ahp-qfd approach is a comparatively new technique that can serve as a valuable addition to the toolbox of any decision maker. it is a powerful approach in the selection of alternative manufacturing/ engineering projects. while the case example was directed towards the pharmaceutical sector in the context of this research, this integrated approach can also be applied to a plethora of other engineering decisions. this technique can also be particularly useful in evaluating and assessing a new technology, especially in comparison to an existing technology. the integrated qfd/ahp approach applies two multiple (group) consensus management techniques of decision-making and allows the policy makers to arrive at more robust ijahp article: ganguly, merino/ an integrated ahp-qfd approach for evaluating competing technological processes international journal of the analytic hierarchy process 554 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.315 decisions. it also allows for organization through integrating both customer and technical requirements, to look at the selection process from both an internal as well as an external point of view. the developed model can help any manufacturing /engineering organization to minimize the risks associated with embracing a new technology over an existing one. furthermore, this model can aid in a better allocation of an organization’s technological resources along with obtaining a group based decision-making policy – thereby resulting in an effective strategic planning framework for the selection/evaluation of alternative technologies. despite the flexibility of the integrated approach, it is not devoid of limitations. the major limitation of this approach is its implicit assumption of transitivity among the pair wise judgments. the ahp implicitly assumes a logical transitivity among the pair wise judgments, which is not always the case in a real-life decision-making process. thus, the actual value of the consistency ratio (cr) in ahp often ends up being higher than the desired value (i.e., ≤ 0.10) since the pair wise judgments of the decision maker are not always transitive in nature for all practical purposes. for example, in the present research, the consistency ratio was slightly greater than 0.10 when performing pairwise comparisons between the customer requirements. furthermore, the large number of ahp and qfd comparisons that have to be drawn among the attributes often proves to be a tiring and lengthy process, thereby consuming experts’ valuable time. 6. relevance of the research to decision makers this research dealt with the problem of choosing between two alternative hi-tech capital projects. the proper selection of a manufacturing process is complicated and requires detailed analysis before committing to huge capital investments. hence, choosing among competing technologies for any manufacturing process is key to effectively managing technology projects. this is a typical problem for the managerial policy makers and technology managers. the case presented demonstrates to the decision-makers, especially in the pharmaceutical industry, how to use various techniques to choose among alternative technological projects. even though the case study used in this article involves a chemical process, this technique can also be applied in various other areas pertaining to decision-making and technology management. the insight of the techniques discussed in this paper can considerably aid the policy makers to gain more knowledge about the techniques and practical applications of decision-making. this, in turn, could lead to a more effective decision-making process and hence a better selection of technological projects. additionally, using non-economic decision-making tools in tandem with economic tools like sensitivity analysis and after tax analysis provides the decision-makers with the knowledge of how to combine these tools to make a more robust decision regarding the choice of a project. the decisionmaking techniques as illustrated in this research article can be used either independently or in conjunction with the economic analysis in order to arrive at a more robust and accurate decision, and therefore serve as an important element of the total decision process. 7. conclusion and future research as seen from the research results, the ahp-qfd analysis indicates that the chemical process involving the microwave irradiation method should be the preferred alternative as it yielded a higher overall value. the purpose of this paper was an attempt to show the ijahp article: ganguly, merino/ an integrated ahp-qfd approach for evaluating competing technological processes international journal of the analytic hierarchy process 555 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.315 effectiveness of an integrated ahp – qfd model for the selection of capital projects. while the technique of qfd was pivotal in determining the relationship among the customer and the technical requirements, ahp was used to determine the relationship between the variables in the model, especially the customer requirements and alternative selection of the technical requirements. thus, an effective integration of the two techniques, as shown in the research, allows the decision makers to arrive at a more rational decision while taking into account the idea of multilevel hierarchy. additionally, the proposed methodology might be particularly well suited to a multi-criteria environment where the data is often unstructured and the information complex – an environment that describes the domain of drug development. in such a situation, a proper evaluation of the requirements, both customer as well as technical, can prove to be very important in the final decision-making process for the appropriate selection of an engineering project. also, the integrated model analysis used in this research can be applied to almost any other technological decision-making processes. furthermore, although the proposed illustrative case example discusses only two alternatives, this model can be used for a situation that is concerned with more than two alternatives. using more than two alternatives does not change the structure of the model, but only introduces one (or more, depending on the situation) extra alternative into the decisionmaking process. future research could be directed towards using the ahp analysis at a more detailed level by constructing another level of hierarchy comprising the sub-factors of the attributes selected for pair-wise comparison. in addition, the technical requirements could be further subdivided in order to arrive at an even more detailed correlation between the customer and the technical requirements. finally, using the integrated ahp-qfd model in the selection of various engineering and technology projects would definitely aid in enhancing the overall robustness of the model. this would provide the decision makers with a more in-depth result and thus a more accurate conclusion regarding the choice of alternatives. ijahp article: ganguly, merino/ an integrated ahp-qfd approach for evaluating competing technological processes international journal of the analytic hierarchy process 556 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.315 references akao, y. (1995). quality function deployment: integrating customer requirements into product design. new york: productivity press inc. allama, m. m., parvez, m. s., ali, s. m. & azeem, a. (2012). using the analytical hierarchical process to prioritise safety management elements in manufacturing in bangladesh. international journal of the analytic hierarchy process, 4(2), 104 – 117. doi: http://dx.doi.org/10.13033/ijahp.v4i2.78 atthirawong, w. and b. maccarthy, b. (2002). an application to analytical hierarchy process to international location decision-making. the proceedings of the 7 th cambridge research symposium on international manufacturing, cambridge, uk: centre for international manufacturing. bahil, a. t. and chapman, w. l. (1993). a tutorial on quality function deployment. engineering management journal, 5(3), 24 – 35. doi: http://dx.doi.org/10.1080/10429247.1993.11414742 bhattacharya, a., sarkar, b. and mukherjee, s. k. (2005). integrating ahp with qfd for robot selection under requirement perspective. international journal of production research, 43(17), 3671 – 3685. doi: 10.1080/00207540500137217 cardoso, j. f., filho, n. c., miguel, p. a. c. (2015). application of quality function deployment for the development of an organic product. food quality and preference, 40(a), 180-190. doi: http://dx.doi.org/10.1016/j.foodqual.2014.09.012 chakraborty, s. and dey, s. (2007). qfd-based expert system for non-traditional machining processes selection. expert systems with applications, 32(4), 1208-1217. doi: 10.1016/j.eswa.2006.02.010 cheng, e. w. l. and li, h. (2001). information priority-setting for better resource allocation using analytic hierarchy process (ahp). information management and computer security, 9(2/3), 61-70. doi: http://dx.doi.org/10.1108/09685220110388827 chuang, p. t. (2001). combining the analytic hierarchy process and quality function deployment for a location fecision from a requirement perspective. international journal of advanced manufacturing technology, 18(11), 842-849. doi: 10.1007/s001700170010 collins, m. j. (2010). drug discovery at the speed of light. r&d, available from the internet at < http://www.cem.de/documents/pdf/microwave_synthesis_theory.pdf>, retrieved on april 15, 2010. dave, h. k., desai, k. p., & raval, h. k. (2012). a decision support system for tool electrode selection for electro discharge machining process using the analytic hierarchy process. international journal of the analytic hierarchy process, 4(2), 89 103. doi: http://dx.doi.org/10.13033/ijahp.v4i2.131 http://dx.doi.org/10.13033/ijahp.v4i2.78 http://dx.doi.org/10.1080/10429247.1993.11414742 http://dx.doi.org/10.1016/j.foodqual.2014.09.012 http://dx.doi.org/10.1016/j.eswa.2006.02.010 http://dx.doi.org/10.1108/09685220110388827 http://www.cem.de/documents/pdf/microwave_synthesis_theory.pdf ijahp article: ganguly, merino/ an integrated ahp-qfd approach for evaluating competing technological processes international journal of the analytic hierarchy process 557 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.315 dolan, j. g. (2000). involving patients in decisions regarding preventive health interventions using the analytic hierarchy process. health expectations, 3(1), 37 – 45. doi: 10.1046/j.1369-6513.2000.00075.x ganguly a. and merino, d. n. (2007). applying analytical hierarchy processing in selection among alternative chemical process. proceedings of the 28 th american society of engineering management (asem) national conference proceedings, chattanooga, tn, usa. georgakellos, d. a. (2005). technology selection from alternatives: a scoring model for screening candidates in equipment purchasing. international journal of innovation and technology management, 2(1), 1-18. doi: 10.1142/s0219877005000393 hanumaiah, n., ravi, b. and mukherjee, n. p. (2006). rapid hard tooling process selection using qfd-ahp methodology. journal of manufacturing technology management, 17(3), 332 -350. doi: http://dx.doi.org/10.1108/17410380610648290 hauser, j. r. and clausing, d. (1988). the house of quality. harvard business review, 6(3), 63 – 73. ho, w. (2008). integrated analytic hierarchy process and its applications – a literature review. european journal of operational research, 186(1), 211-228. doi:10.1016/j.ejor.2007.01.004 huang, c. c., chu, p.y. and chiang, y. h. (2008). a fuzzy ahp application in government-sponsored r&d project selection. omega, 36(6), 1038-1052. doi: http://dx.doi.org/10/1016/j.omega.2006.05.003 hughes, w. r. (2009). a statistical framework for strategic decision making with ahp: probability assessment and bayesian revision, omega, 37(2), 463-470. doi:10.1016/j.omega.2007.07.002 kappe, c. o. (2003). microwave-enhanced chemistry – enabling technology revolutionising organic synthesis and drug discovery, a report in business briefings: future drug discovery 2003, touch briefings, 42 – 45. kappe, c. o. and dallinger, d. (2006). the impact of microwave synthesis on drug discovery. nature reviews drug discovery, 5(1), 51 – 63. doi:10.1038/nrd1926 lang, h. j. and merino, d. n. (1993). selection process for capital projects. hoboken, nj: john wiley & sons, inc. liberatore, m. j. and nydick, r. l. (1993). decision technology: modeling, software, and applications. new york: john wiley and sons, inc. madu, c. n., kuei, c. and madu, i. e. (2002). a hierarchic metric approach for integration of green issues in manufacturing: a paper recycling application, journal of environmental management, 64(3), 261 – 272. doi: 10.1006/jema.2001.0498 http://dx.doi.org/10.1108/17410380610648290 http://dx.doi.org/10.1016/j.omega.2007.07.002 ijahp article: ganguly, merino/ an integrated ahp-qfd approach for evaluating competing technological processes international journal of the analytic hierarchy process 558 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.315 marx, v. (2004). riding the microwave, chemical engineering news, 82(50), 14 – 19. doi: http://dx.doi.org/10.1021/cen-v082n050.p014 melón, m. g., beltran, p. a. and cruz, m. c. g. (2008). an ahp-based evaluation procedure for innovative educational projects: a face-to-face vs. computer-mediated case study, omega, 36, 754-765. doi: http://dx.doi.org/10.1016/j.omega.2006.01.005 mustafa, m. a. and al-bahar, j. f. (1991). project risk assessment using the analytic hierarchy process, ieee transactions on engineering management, 38(1), 46 – 52. doi: 10.1109/17.65759 pecchia, l., martin, j. l., ragozzino, a., vanzanella, c., scognamiglio, a., mirarchi, l., & morgan, s. p. (2013). user needs elicitation via analytic hierarchy process (ahp). a case study on a computed tomography (ct) scanner, bmc medical informatics and decision making, 13(2), 1-11. doi:10.1186/1472-6947-13-2 partovi, f. y. (2007). an analytical model of process choice in the chemical industry, international journal of production economics, 105(1), 213-227. doi:10.1016/j.ijpe.2006.03.006 prasad, k. & chakraborty, s. (2013). a quality function deployment-based model for materials selection. materials & design, 49, 525-535. doi: 10.1016/j.matdes.2013.01.035 reid, r. p. and hermann, m. r. (1989). qfd . . . the voice of the customer. the journal for quality and participation, 12(4), 44 – 46. saaty, t. l. (1980). analytic hierarchy process. new york: the mcgraw-hill companies. saaty, t. l. (2001). the analytic hierarchy process: planning, priority setting, resource allocation. pittsburgh, pa: rws publication. saaty t. l. (2011). aligning the measurement of tangibles with intangibles and not the converse. international journal of the analytic hierarchy process, 3(1), 79-87. doi: http://dx.doi.org/10.13033/ijahp.v3i1.91 tam, m. c. y. and tummala, v. m. r. (2001). an application of the ahp in vendor selection of a telecommunications system, omega, 29(2), 171-182. doi: 10.1016/s03050483(00)00039-6 tam, c. m., tong, t. k. l. and chiu, g. w. c. (2006). comparing non-structural fuzzy decision support system and analytical hierarchy process in decision-making for construction problems, european journal of operational research, 174(2),1317-1324. doi: 10.1016/j.ejor.2005.03.013 vaidya, o. s. and kumar, s. (2006). analytic hierarchy process: an overview of applications, european journal of operational research, 169(1), 1-29. doi: 10.1016/j.ejor.2004.04.028 http://dx.doi.org/10.1021/cen-v082n050.p014 http://dx.doi.org/10.1109/17.65759 http://dx.doi.org/10.1016/j.ijpe.2006.03.006 ijahp article: ganguly, merino/ an integrated ahp-qfd approach for evaluating competing technological processes international journal of the analytic hierarchy process 559 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.315 wasserman, g. s. (1993). on how to prioritize design requirements during the qfd planning process, iie transactions, 25(3), 59-65. doi:10.1080/07408179308964291 wathey, b., tierney, j., lidstörm p. and westman, j. (2002). the impact of microwaveassisted organic chemistry on drug discovery, drug discovery today, 7(6), 373 – 380. doi: http://dx.doi.org/10.1016/s1359-6446(02)02178-5 zakarian, a. and kusiak, a. (1999). forming teams: an analytic approach, iie transactions, 31(1), 85-97. doi:10.1080/07408179908969808 http://dx.doi.org/10.1016/s1359-6446(02)02178-5 ijahp preface: mu/ijahp embraces rolling article publication international journal of the analytic hierarchy process 1 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.989 ijahp embraces rolling article publication dear ijahp readers: it is our pleasure to complete the publishing of our first issue this year. as you may have noticed, ijahp has moved to a rolling article publication format. this means that articles are published as they become accepted during the respective issue publication period. the main benefit of this approach is that authors can get their articles published much quicker because there is no need to wait for all the papers of a specific issue to be accepted. this makes it possible to take advantage of the unique opportunities open access publication offers. also, it allows meeting current authors’ need for quick publication and societal expectations of faster access to newly produced knowledge. a second change in our publication format is that going forward readers will not need to register on the ijahp website to access the published articles. while access to the journal has always been (and still is) free, we suspect that many potential readers may be put off by the need to register on the journal website. this is of particular concern as currently the web is plagued with scams and fake sites. we believe that making the articles directly accessible without any specific publication date will drastically increase the number of readers of our published articles. finally, in addition to our customary blend of ahp/anp international scholarly work, we report on the first t. l. saaty hackathon at the university of pittsburgh and inform you of upcoming conferences in croatia and poland where there are opportunities to present your ahp/anp work and get to know new theory and applications in this area. in the future, we intend to increase our coverage of activities and opportunities in our ahp/anp communities worldwide. enjoy this issue! enrique mu ijahp editor-in-chief microsoft word 2saatyarticle_pp4_30_03212009 ijahp article: saaty/decisions, structure, judgment, and natural law international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 4 decisions: structure, judgment, and natural law thomas l. saaty university of pittsburgh pittsburgh, pa 15260 saaty@katz.pitt.edu abstract the object here is to show that our thinking processes and our physical forms and those of all things that exist, are a result of response in nature to influences as stimuli, brought about by natural occurrences. the ideas are developed through a generalization of the role judgment plays in decision making. judgment serves as the basic link between our conscious awareness and the stimuli of the natural world. the mathematics used to represent natural laws is derived from stimulus-response theory and this in turn from the representation of judgment as it is used in decision-making. the representation of discrete judgment as a principal eigenvalue problem is generalized to the continuous case through fredholm theory. solving the resulting fundamental functional equation, which is a necessary condition for the existence of a solution, gives rise to damped periodic oscillation. the fourier transform of the real valued solution has a perturbed inverse square representation that poses a question raised on occasion in science about the full accuracy of exact inverse square laws of gravitation, optics and of electric charges. the fourier transform of the complex valued solution is a linear combination of dirac type distribution of impulsive functions representing how the brain must operate to respond to external stimuli. a generalization is made to a functional equation in operator form with its solution. these solutions describe all forms that exist in nature as anything that responds to influences. these considerations that originate in the mathematics of judgment, serve as a unifying approach to our understanding and to creating tools for modeling and solving complex physical and behavioral problems. keywords: judgment, stimulus-response, natural law, response function, operator, fundamental equation 1. introduction there are two basic characteristics of a decision; the structure that represents the factors and the alternatives of that decision and their connections, and the judgments and their syntheses that determine the best outcome of the decision. we can think of the world and of human experience in terms of internal and external influences to which we respond in our conscious experience. our response characterizes the fundamental internal nature of how our brains must operate, and the law they satisfy to respond to external stimuli. in addition, in nature itself influences are responded to in physical and biological ways that create forms like particles, stones, plants, animals and people. they are characterized as particular forms of response to influences. it appears that stimulus-response theory, an rob typewritten text http://dx.doi.org/10.13033/ijahp.v1i1.13 ijahp article: saaty/decisions, structure, judgment, and natural law international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 5 offshoot of the judgments process and of decision making, serves to unify our understanding of the world. to structure a decision requires use of the various steps normally recognized to belong to creative thinking. it requires understanding of the problem and of specialized thinking and expert knowledge in the area in which the decision is made. a structure can be a hierarchy whose levels and elements are arranged by the experts according to their understanding of the downward flow of influence. it can also be a more general network whose interdependencies capture the forward and backward directions of flows, their branching and confluence that work together to shape the outcome. a single decision, depending on its complexity can involve a number of structures whose outcomes must be brought together into a single overall outcome. many people think that measurement demands a physical scale with a zero and a unit of measure. that is not true. surprisingly, we can derive accurate and reliable relative scales that do not have a zero or a unit by using our understanding and judgments which are, after all, the fundamental determinants of why we want to measure something. in reality we do this all the time without thinking about it. physical scales are useful for the things we know how to measure. even after we obtain readings from a physical scale, they still need to be interpreted with our judgment. and the number of things we do not know how to measure, the so-called intangibles, is infinitely larger than the things we know how to measure. judgment followed by decision and action is the link between our inner understanding and the physical world that we depend on to satisfy our needs. not only all measurement needs to be interpreted in terms of our value system, but especially readings on a linear scale like the meter and the yard that are the simplest kind of measurements need interpretation to determine what the numbers actually mean. it is true that the readings denote quantity represented by a number, but how important that number is depends on the purpose we intend to use it for. thus measurement provides information that is the basis of different kinds of judgments in different decisions. in the end it is judgment that is essential for our understanding. only when judgment is quantitative that we are able make subjective measurements and tradeoffs. how do we do that in a meaningful way that can be validated in practice? judgment is always contextual, relating things among themselves or to standards we store in memory. it is expressed as comparisons among things. to be meaningful and relevant comparisons need to be reduced to pairwise comparisons. pairwise comparisons give rise to priorities that is a cardinal way of determining rank order among things and representing that order numerically. the study of order belongs to the field of order topology in mathematics and is in contrast to metric topology used to determine the closeness of measurements in science. an important property of order is transitivity that is often imposed as an axiom. but when we need judgments, transitivity may be violated and order becomes intransitive. when cardinal numbers are used, we can have transitivity but still be inconsistent. to derive priorities requires that we solve the principal eigenvalue problem to derive priorities. to derive priorities from inconsistent judgments order and transitivity require that we solve the principal eigenvalue problem to derive priorities. ijahp article: saaty/decisions, structure, judgment, and natural law international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 6 in decision making comparisons are discrete and can be represented by matrices which is not true when we automatically and without thinking take in a lot of sensory information that also requires comparison of a continuous kind. we learn from the solution to the generalization of the eigenvalue representation that our minds are constantly responding to stimuli in a way represented by a damped periodic function with two parameters. the fourier transform of the complex valued solution is the sum of functions that are dirac distributions implying that our responding mechanism, that is our brain, must respond with impulsive firings as the brain actually does. we also learn from the fourier transform of the real solution that what we think natural law is, our response always takes the form of an inverse square law that is not precisely a square as assumed in science, but sufficiently close that it is difficult to detect small values of the parameters. let us note that the human mind cannot respond to stimuli in linear proportion to their intensity. as the intensity of the stimulus increases, our senses become duller and gradually level off to a point where we cannot distinguish between an explosion and a much larger explosion, a distant hill and a mountain or among very large numbers ranging from the millions to the billions. we can describe this dampening effect of response by a mathematical function that we derive from an equation that describes the relationship between for example the response at a distance from a stimulus that must be proportional to what that response would be at the origin of the stimulus. we learn from generalizing the continuous formulation to operators, that all forms of creation that respond to influences of which sense stimuli are only one kind have only certain possible general forms with which they can make their response. 2. structuring—the need for creative thinking this section is dedicated to observations we have made about the relationship between decision making and creative thinking as a process. a purposeful system is generally characterized in terms of its purpose or goal; its physical or geometric abstract structure of elements collected into subsystems; the functions of these subsystems carried out to fulfill the goal; and the flows that take place through the connections of the subsystems to perform the functions. in decision making the purpose of a system serves the satisfaction of human needs and values structured and measured in terms of benefits (b), opportunities (o), costs (c) and risks (r) referred to collectively as bocr. they are a critical part of decision making. it takes little effort to realize that creative thinking is an essential part of decision making. one can relate the four tenets of creativity to the steps taken in making a decision. they are to brainstorm all the factors that go into a decision. they are then related (synectics) by putting them into groups clusters or components of homogeneous (close identity) elements with respect to the property or criterion being considered (one axiom of the ahp) and pairwise compared with reciprocal values (another axiom of the ahp). the factors are then arranged into a hierarchic or network structures (morphological analysis in creativity that is also the third axiom of the ahp) that can include the goal, objectives, criteria, influential actors and their objectives and the alternatives of the decision (see later). finally these structures are constantly polished, revised and expanded as needed in the process of arriving at the final best decision (lateral thinking which is the fourth axiom of the ahp with regard to expectations of the outcome and its reliability). we ijahp article: saaty/decisions, structure, judgment, and natural law international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 7 have applied these four operations of creativity on numerous occasions in individual decisions and in groups gathered for a special purpose and even by correspondence with the parties involved. but even with creative thinking there is no better substitute to expert knowledge and understanding except through the use of the prioritization process. to create priorities needs creativity. conversely, to be effective with creativity needs priorities. two seemingly different occupations of the mind are fundamentally interdependent. one would like to think that making decisions needs creativity much more than being creative involves decision making, but that is a subject that awaits more exploration. creative thinking and decision making are linked together with a feedback relation. our representation of the relationship between decision making and creative thinking is roughly represented in the alternative forms of figures 1 and 2. figure 1 relationship among creative thinking, decision making and problem solving decision making precedes creative thinking to determine which area to brainstorm first it also follows creative thinking to determine which avenues to specialize in first for implementation needs creative thinking precedes decision making tobrainstorm, connect and organize the criteria needed to make a decision creative thinking in turn needs prioritization problem solving creative thinking and decision making are the corner stones of problem solving ijahp article: saaty/decisions, structure, judgment, and natural law international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 8 creative thinking what to brainstorm? how to structure? imagination foresighted thinking lateral thinking morphological thinking find opportunity help to add value solve problem remove obstacles decision making present and likely future? problem to solve? opportunity to seize? best action? resource allocation? solution strategy implementation • design, planning and action • physical (engineering) and behavioral (management) ju dg m en ts p ri or it iz at io n figure 2 relationship between creative thinking and decision making there are many creativity techniques that can be used to generate alternatives for an ahp/anp model of a decision. the process of structuring the model and making the factors explicit can trigger thinking about what the alternatives should be. thus with the ahp/anp the very process of defining and structuring the problem is integrated with designing a solution. after the process is completed, reflection may lead the group back to refining the problem’s definition. the ahp/anp does not impose limits on how groups structure their thinking. a decision making method is essentially about eliciting tacit preferences from the decision makers. the ahp/anp does not require physical measurements as inputs although such information can be used if it is available. not only is devising a set of alternatives essential, but also encouraging creativity makes a breakthrough decision more likely. brainstorming enables a group to generate more alternatives. brainstorming means that any judgment which may inhibit creativity must be deferred. despite its wide use, the technique does have limitations and has been modified over the years. one of its modifications is brain writing or idea writing because the use of writing is considered to be better than presenting ideas orally as there is less danger of domination by certain participants. it also encourages people to participate who have trouble expressing their ideas orally. participants have a chance to phrase their ideas clearly in writing beforehand or allow them to be recorded. the method will not work, however, if people are unwilling to express their ideas in writing. it works best with small groups, so big groups need to be broken into smaller groups in parallel sessions. after a proper introduction is given and a stimulating question is asked, group members write their initial response on a given form. they then react in writing to each other’s forms. after each participant reads the comments, the small group discusses the ijahp article: saaty/decisions, structure, judgment, and natural law international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 9 principal ideas that emerge from the written interactions and summarizes the discussion in writing. other modifications of brainstorming include bug lists and negative brainstorming (generating complaints to identify weaknesses), the crawford blue slip method (brainstorming in response to a number of questions that are related to a problem), and free discussion among group participants. brainstorming has been used in complex problems to generate questions rather than solutions. the outcome is a list of questions that the group decides to pursue to move the process forward. synthesizing priorities in networks and in hierarchies the priority vectors derived from pairwise comparison matrices are each entered as a part of some column of a supermatrix. the supermatrix represents the influence priority of an element on the left of the matrix on an element at the top of the matrix. a supermatrix along with an example of one of its general entry (i,j) block is shown in figure 3. the component ci alongside the supermatrix includes all the priority vectors derived for nodes that are “parent” nodes in the ci cluster. figure 4 gives the structure of a hierarchy along with its supermatrix. the entry in the last row and column of the supermatrix of a hierarchy is the identity matrix i. ijahp article: saaty/decisions, structure, judgment, and natural law international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 10 feedback network with components having inner and outer dependence among their elements loop in a component indicates inner dependence of the elements in that component with respect to a common property. arc from component c4 to c2 indicates the outer dependence of the elements in c2 on the elements in c4 with respect to a common property. c4 c1 c2 c3 feedback c1 c2 cn e11e12 e1n1 e21e22 e2n2 en1en2 ennn w11 w12 w1n w21 w22 w2n wn1 wn2 wnn c1 c2 cn e11 e12 e1n1 e21 e22 e2n2 en1 en2 ennn w = c1 c2 cn e11e12 e1n1 e21e22 e2n2 en1en2 ennn w11 w12 w1n w21 w22 w2n wn1 wn2 wnn c1 c2 cn e11 e12 e1n1 e21 e22 e2n2 en1 en2 ennn c1 c2 cn e11e12 e1n1 e21e22 e2n2 en1en2 ennn w11 w12 w1n w21 w22 w2n wn1 wn2 wnn c1 c2 cn e11 e12 e1n1 e21 e22 e2n2 en1 en2 ennn w = wi1 wi1 wi1 wij = (j1) (j2) (jnj) (j1) (j2) (jnj) wi2 wi2 wi2 wini wini wini (j1) (j2) (jnj) figure 3 structure and supermatrix of a network and detail of a matrix in it a holarchy, illustrated in figure 5 is a hierarchy of two or more levels in which the goal is eliminated and what was the second level that used to depend on the goal, now depends on the bottom level of alternatives, thus as a whole the hierarchy is a cycle of successively dependent levels. we have encountered such a form in the analysis of the turn-around of the us economy in which the importance of the primary factors is determined in terms of the time periods at the bottom. ijahp article: saaty/decisions, structure, judgment, and natural law international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 11 21 32 n-1, n-2 n,n-1 0 0 0 0 0 0 0 0 0w 0 0 0 0w w = w 0 0 0 iw •⎡ ⎤ ⎢ ⎥•⎢ ⎥ ⎢ ⎥• ⎢ ⎥ ⎢ ⎥ ⎢ ⎥• • • • • ⎢ ⎥ •⎢ ⎥⎣ ⎦ k k k m m m m m m m k k figure 4 structure and supermatrix of a hierarchy figure 5 the u.s. holarchy of factors for forecasting turnaround in economic stagnation in a network, the system of components may be regarded as elements that interact and influence each other with respect to a criterion or attribute with respect to which the influences occurs and control the thinking and judgments about them whether they are economic influences, political influence or social influences, for example. each of these influences is an attribute in the decision that must be of a higher order of complexity than ijahp article: saaty/decisions, structure, judgment, and natural law international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 12 the components and a fortiori of higher order than the elements contained in the components. we call such an attribute a control criterion. thus even in a network, there is a hierarchic structure that lists control criteria above the networks. for each of the four bocr merits we have a system of control criteria that we use to assess influence. the result is that such control criteria and/or their subcriteria serve as the basis for all comparisons made under them, both for the components and for the elements in these components. in a hierarchy one does not compare levels according to influence because they are arranged linearly in a predetermined order from which all influence flows downwards. in a network, the effect of the influence of different clusters of elements can differ from cluster to another cluster and hence they need to be weighted to incorporate the proportionality of their contributions. the criteria for comparisons are either included in a level, or more often implicitly replaced by using the idea of “importance, preference or likelihood” with respect to the goal, without being more finely detailed about what kind of importance it is. the control criteria for comparisons in a network are intended to be explicit about the importance of influence that they represent. finally, we consider how to combine the four ranking outcomes of the alternatives for the bocr. the top ranked alternative for each of the bocr priorities is rated rather than compared with respect to strategic criteria. these are the overall criteria that serve the decision maker’s goals in making any decision. the outcome of this rating is four priorities, one for each of the bocr. these priorities are then used to weight the overall priorities of the alternatives for the corresponding bocr and adding the result for the benefits and opportunities and subtracting the sum for the costs and the risks. the final priorities of some or all the alternatives can be negative. 3. judgments are our unique conscious link to the physical world an intuitive factual observation how we respond to stimuli is closely related to decision making. we note that in order to sense objects accurately in the environment, our brains miniaturize them within our system of neurons so that we have a proportional relationship between what we perceive and what is out there. without proportionality we cannot coordinate our thinking with our actions with the accuracy needed to control the environment. proportionality with respect to a single stimulus requires that our response to a proportionately amplified or attenuated stimulus should be proportional to what our response would be to the original value of that stimulus. if w(s) is our response to a stimulus of magnitude s, then the foregoing observation leads us to consider the functional equation ( ) ( )w as bw s= . this equation can also be obtained more rigorously as the necessary condition for solving fredholm’s equation of the second kind as we shall see next. formal derivation from judgments we know from the analytic hierarchy process (ahp) that because of inconsistency in judgments and their possible intransitivity, priorities need to be derived by solving for the principal eigenvalue waw maxλ= . in the continuous case this is written as ijahp article: saaty/decisions, structure, judgment, and natural law international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 13 )()()( max s w = dt t wts,k b a λ∫ where, instead of the positive reciprocal matrix a in the principal eigenvalue problem, we have a positive kernel, k(s, t) > 0, with k(s, t) k(t, s) = 1 that is also consistent; that is, k(s, t) k(t, u )= k(s, u), for all s, t, and u . our problem of extracting the principal eigenvector takes the familiar form of fredholm’s equation of the second kind ∫ = b a swdttwtsk )()(),( maxλ classically treated by using the form ∫ = b a swdttwtsk )()(),(λ with the normalization condition ∫ = b a dssw 1)( . it is easy to show that a consistent kernel has the form k(s, t) = k(s)/k(t) from which what i call the “response” eigenfunction w(s) solution can be shown to be ∫ = s dssk sk sw )( )( )( since the denominator is a constant, we can write )()( saksw = . theorem 1 k(s,t) is consistent if and only if it is separable of the form: k(s,t)=k(s)/k(t) proof: (necessity) k(t, u0)≠0 for some u0εs, otherwise k(t, u0)=0 for all u0 would contradict k(u0, u0)=1 for t=u0. we obtain k(s,t) k(t, u0)=k(s, u0) 0 0 k(s, ) k(s)uk(s, t) = = k(t, ) k(t)u for all u0εs and the result follows. (sufficiency) if k(s,t)=k(s)/k(t) holds, then it is clear that k(s,t) is consistent. ijahp article: saaty/decisions, structure, judgment, and natural law international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 14 we now prove that as in the discrete case of a consistent matrix, where the eigenvector is given by any normalized column of the matrix, an analogous result is obtained in the continuous case. theorem 2. if k(s,t) is consistent, the solution of ∫ = b a swdttwtsk )()(),(λ is given by ( ) ( ) s k s w(s) = k s ds∫ generalizing on the discrete approach in which the consistent matrix a has rank one, we assume that the kernel ( , )k s t is homogeneous of order 1. thus, we have: ( , ) ( , ) ( ) / ( )k as at ak s t k as k at= = = [ ( ) / ( )]a k s k t . it follows that ( ) ( ) ( ) ( ) ( )w as k as ak s k s bw sα α β= = = = . to prove that ( ) ( )w as bw s= from ( ) ( ) ( ) k s w s k s ds = ∫ and ( ) ( ) ( ) ( ) k as k s a k at k t = , we first show that ( ) ( ) ( ) ( ) w as w s w at w t = . integrating both terms of ( ) ( ) ( ) ( ) k as k s a k at k t = first over s, we have ( ) ( ) ( ) ( ) k as ds k s ds a k at k t =∫ ∫ . next, we rearrange the terms and integrate over t, to obtain ( ) ( ) ( ) ( )k as ds k t dt a k s ds k at dt=∫ ∫ ∫ ∫ and this implies that ( ) ( ) 1 ( ) ( ) k s ds k at dt a k as ds k t dt =∫ ∫ ∫ ∫ . thus, ( ) / ( ) ( ) / ( ) ( ) ( ) ( ) / ( )( ) ( ) ( ) ( )( ) / ( ) ( ) / ( ) ( ) ( ) ( ) / ( ) k as k as ds k s k s ds k at dt k s ds k s k s dsw as w s a w at w tk at k at dt k t k t dt k as ds k t dt k t k t dt = = = =∫ ∫ ∫ ∫ ∫ ∫ ∫ ∫ ∫ ∫ . assuming that the domain of integration is bounded or at least measurable, by integrating ( ) ( ) ( ) ( ) w s w as w at w t = over t we have ( ) ( ) ( ) ( ) w at w as dt w s dt w tω ω =∫ ∫ and letting 1 ( ) ( ) w at b dt w tdt ω ω = ∫ ∫ , we have the fundamental equation ( ) ( )w as = bw s . ijahp article: saaty/decisions, structure, judgment, and natural law international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 15 the solution of this functional equation in the real domain was derived for the purposes of this author by janos aczel, a leading mathematician in the field of functional equations: ( ) ⎛ ⎞ ⎜ ⎟ ⎝ ⎠ logs logb loga logsw s = ce p loga where p is a periodic function of period 1 and p(0) = 1. one of the simplest such examples with log / logu s a= is p(u) = cos (u/2β) for which p(0) = 1. it is a basic from that we use for p(u). 4. stimulus-response —how the human mind works in response to stimuli a consequence of the foregoing solution is the well-known logarithmic law of response to stimuli which can be obtained as a first-order approximation to this solution through series expansions of the exponential and the cosine functions as: 321 log)()( cscupecuv u +≈= −β where 0,log >−≡ ββab . the expression on the right is known as the weber-fechner law of logarithmic response, 0,log ≠+= absam , to a stimulus of magnitude s. this law was empirically established and tested in 1860 by gustav theodor fechner who used a law formulated by ernest heinrich weber regarding discrimination between two nearby values of a stimulus. we have now shown that that fechner’s version can be derived by starting with our functional equation for stimulus response. the integer-valued fundamental scale of response used in making paired-comparison judgments in the ahp can be derived from the logarithmic response function as follows. for a given value of the stimulus, the magnitude of response remains the same until the value of the stimulus is increased sufficiently large in proportion to the value of the stimulus, thus preserving the proportionality of relative increase in stimulus for it to be detectable for a new response. this suggests the idea, well known in psychology, of just noticeable differences (jnd). thus, starting with a stimulus s0, successive magnitudes of the new stimuli take the form )1(00 0 0 0001 rsss s ssss += δ +=δ+= 2 2 2 1 1 1 0 0(1 ) (1 )s s s s r s r s α= + δ = + = + ≡ m ,...)2,1,0(01 === − nsss n nn αα we consider the responses to these stimuli to be measured on a ratio scale (b = 0). a typical response has the form ,,...,1,log niam ii == α or one after another they have the form .log,...,log2;log 21 ααα namamam n === we take the ratios ,,...,1,/ 1 nimm i = of these responses in which the first is the smallest and serves as the unit of comparison, thus we obtain the integer values 1, 2,…,n of the fundamental scale of the ahp. it appears that numbers are intrinsic to our ability to make comparisons, and were not just an invention by our primitive ancestors. ijahp article: saaty/decisions, structure, judgment, and natural law international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 16 the upshot of this approach is to observe that our responses fall into categories involving just noticeable differences from one category into another. within each category we are unable to tell the difference between a certain value and a slightly larger value. at the very beginning we can compare an object with itself and obtain the value 1 for its dominance over itself with respect to a property. we then compare it with an object that is a little larger. because of the jnd syndrome, we would decide whether the slightly larger object is equal to it or falls in the next category, which is twice its size, and so on, thus obtaining the numbers 1, 2, 3, and so on. it would seem that if fuzziness has any real justification it lies in this psychophysical phenomenon and hence the ahp does not need further theoretical “fuzzifying”. but as we said before we cannot go on with very large numbers because we are unable to compare the object with something that is too large. if we do, we will make such an error that our estimate in the comparison will be very inconsistent and therefore inaccurate and our result will be unreliable. essentially it amounts to assigning values from the positive integers, by dividing our ability to sense things into high, medium, and low and then dividing each one into three categories so we would get for the largest value (high, high), followed by (high, medium), (high, low), (medium, high), (medium, medium), (medium, low), (low, high), (low, medium), (low, low). the numerical values we assigned to them would range from 9 for the (high, high) pair and so on down to 1 for the (low, low) pair. when we compare the smaller object, with the larger object we use the reciprocal value. if the large apple is three times bigger than the small orange, then the orange is automatically one-third as large as the apple. one can prove mathematically that small changes in the numbers lead to small changes in the final answers that we call priorities. 5. the inverse square law of physics in optics, gravity and electricity the solution of fredholm’s equation derived above is defined in the frequency domain or transform domain in fourier analysis as it is based on the flow of electric charge in the brain. we must now take its transform to derive the solution in the spatial or time domain. thus our solution of fredholm’s equation here is given as the fourier transform, ( ) ( ) ( ) + -2 i x f = f x dx = pe ceπ ω βωω ω ∞ ∞ ∫ whose inverse transform is the inverse fourier transform of a convolution of the two factors in the product. we have: ( ) ( ) + 2 i x f x = f de π ωω ω ∞ ∞ ∫ since our solution is a product of two factors, the inverse transform can be obtained as the convolution of two functions, the inverse fourier transform of each of which corresponds to just one of the factors. now the inverse fourier transform of ue β− is given by ijahp article: saaty/decisions, structure, judgment, and natural law international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 17 2 2 (2/ ) + π β β ξ also because a periodic function has a fourier series expansion we have ( ) 2 ikuk k = p u = e πα ∞ ∞ ∑ this has the inverse fourier transform: ( )k 2 kk = α δ ξ π ∞ ∑ ∞ the product of the transforms of the two functions is equal to the fourier transform of the convolution of the two functions themselves which we just obtained by taking their individual inverse transforms. we have, to within a multiplicative constant: ( ) 2 2 ( ) + k k = 2 k d + x α β δ ξ π ξ β ξ ∞ ∞ ∞∞ ∑∫ 2 2 ( ) k k = + = x-2k β α β π ∞ ∞ ∑ let us consider the simple case where the periodic part of our solution is given by p(u)= cos u/2π=(1/2)(eiu/2π + e-iu/2π). the inverse fourier transform of w(u)= c e -βu cos u/2π, β > 0 is: 2 2 2 2 1 1 c + 2 1 1 + + + 2 2 β π ξ β ξβ π π ⎡ ⎤ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎛ ⎞ ⎛ ⎞ ⎢ ⎥⎜ ⎟ ⎜ ⎟ ⎝ ⎠ ⎝ ⎠⎣ ⎦ when the constants in the denominator are small relative to ξ we have c1/ξ2 which we believe is why experiments show that in optics, gravitation, electric charge, and sound intensity, forces act according to inverse square laws. this is the same law of nature in which an object responding to a force field must decide to follow that law by comparing infinitesimal successive states through which it passes. if the stimulus is constant, the exponential factor in the general response solution is constant, and the solution in this particular case would be periodic of period one. when the distance ξ is very small, the result varies inversely with the parameter β>0. now a word about problems concerning the exactness of the inverse square law observed in studies using gravitation. ijahp article: saaty/decisions, structure, judgment, and natural law international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 18 precession is a change in the direction of the axis of a rotating object. precession happens to orbits of many planets but its effect is most noticeable for planets and other objects near the sun with highly elliptical orbits. seen from the earth, mercury's orbit appears to have a precession of 5600 seconds of arc per century (one second of arc=1/3600 degrees). taking into account the effects of the other planets and the sun, newton's equations predict a precession of 5557 seconds of arc per century, a discrepancy of 43 seconds of arc per century. this discrepancy was thought to be resolved in 1915, when the theory of general relativity predicted an additional perihelion precession of exactly 43 arc seconds per year. however, james constant, in his 2006 article on the internet, precession of perihelia; le verrier's and einstein's predictions compared, refers to the works of louis brillouin (relativity reexamined, academic press 1970 page 99) and steven weinberg (gravitation and cosmology, john wiley & sons, inc. 1972 page 198) to conclude that there are “doubts and caveats that relativity can predict correct values for mercury and other planets that meet requirements of elliptic orbits set by newtonian theory.” elsewhere he demonstrates in a book that einstein's theory of gravitation, expressed as riemannian geometry, can not be reconciled with newton's theory of gravitation. so the story is by no means settled. some researchers, notably the harvard astronomers asaph hall and simon newcomb in the 1800s, thought that perhaps newtonian theory was at fault, and gravity isn't exactly an inverse square law. hall thought that he could account for the precession of mercury if instead of using the power 2 in 21 / r one were to use 2.00000016. some people find this proposal in conflict with basic conservation laws. the may 19, 2007, issue of the economist magazine reported that “either newton or einstein was wrong, or there is something missing from the universe. the reason for this is that galaxies do not behave as the laws of gravity predict they should.” in this article it is reported that dark matter and relativity are used to account for what happens after enormous collisions take place, but the fact that physicists sometimes question the form of the law of gravity makes one wonder about its universal accuracy as a strictly inverse square law. 6. the brain as a system of firing neurons solution to the fundamental equation in the complex domain has the form: ln / ln( ) (ln / ln )b aw z z p z a= here p(u) with u = ln z/ln a, is an arbitrary multivalued periodic function in u of period 1. even without p being multivalued, the function w(z) could be multivalued because ln b/ln a is generally a complex number. if p is single-valued and ln b/ln a turns out to be an integer or a rational number, then w(z) is a single-valued or finitely multivalued function, respectively. this generally multivalued solution is obtained in a way analogous to the real case. we now show that the space-time fourier transform of the complex valued solution is a combination of dirac distributions. our solution of fredholm's equation here is given as the fourier transform, ijahp article: saaty/decisions, structure, judgment, and natural law international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 19 with its inverse transform given by: where ∗(2βn+2(b)-x) is the dirac delta function. thus response is expressed as a combination of impulsive functions. the fourier transform of the analytic solution ∑ <= + kqzazw qjkkj ,)( is given by: 1 ( ) 2 ( 1) jk q jk q j jk q j d x a dx δ π +∞ + + = −∑ again involving impulsive functions. we note that the number of nonzero terms in this sum is finite. 7. synthesis of stimuli from different senses we need to synthesize different responses that have the form of the eigenfunction solution: [log| |/[log| |]( ) ( ) ([log | | / log | |]), 1,...,k kz ak k k k k kw z b p z a k n= = where k refers to different neural response dimensions such as sound, “feeling” (which is a mixture of sensations), and so on. their product is a function of several complex variables and is the solution of the equation 1 1 ( ) ( ) n n k k k k k k k k w a z b w z = = =∏ ∏ the product of solutions of )()( kkkkkk zwbzaw = satisfies such an equation with the new .kbb = since the product of periodic functions of period 1 is also a periodic function of period one, the result of taking the product has the same form as the original function: a damping factor multiplied by a periodic function of period 1. if we multiply n solutions in the same variable z, in each of which b and w are allowed to be different we obtain: [log| |/ log| |] [log| |/ log| |] [log| |/ log| |] [log| |/ log| |] [log| |/ log| |] 1 1 ( / ) ( ... ) ( / )... ( / ) ( / ) z a z a z a z a z a n n z a b b w z a w z a b v z a = π2/cos u is the generic form we could adopt for the periodic component of period one. we also have the functional equation w(a1 z1 , . . . , an zn) = b w(z1 , . . . , zn) ( ) ( ) ( ) + -2 i x f = f x dx = pe ceπ ω βωω ω ∞ ∞ ∫ ( ( ) ) ( ) log ( ( ) ) log ( ( ) ) n 2 n + b x 1/2 a a i 2 n + b x’ a | b | + 2 n + b x π θ π δ π θ π θ ∞ ∞ ⎡ ⎤ ⎢ ⎥ ⎣ ⎦ ∑ ijahp article: saaty/decisions, structure, judgment, and natural law international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 20 whose solution for the real variable case with b > 0, ak > 0, and zk > 0, (k = 1, . . . , n), is given by 1 log / log ) / 1 1 1 loglog ( ,..., ) ( ,..., ) log log n k k k z a n n n n zz w z z b p a a = ∑ = where p is an arbitrary periodic function of period one of n variables, that is, 1 1( 1,..., 1) ( ,..., )n np t t p t t+ + = . for a continuum number of stimuli let ( , )k x y be a compact reciprocal kernel i.e. ( , ) ( , ) 1, for all and k x y k y x x yε ε= x y , where x and y are compact subsets of the reals. we have the equation ( ) ( ; ) ( )w k wλ ω = ∫x x y y . formally, we write the general solution in the form log ( ) [exp (log / log ) ] ( ) log w x a dx p aω = ∫ x x which involves a product integral. we now give a fascinating theorem that illuminates the difference between mathematical functions of several variables and the firing of neurons each described by a function of a single variable. 8. approximations and density our thoughts and representations are linear combinations of the response functions derived above and therefore, everything can be represented as accurately as desired by linear combinations of these functions or their transforms both the real and complex valued. approximation is about how general functions can be decomposed into more simple building blocks: polynomials, splines, wavelets, and the like. one needs to guarantee specified rates of convergence when the smoothness of the kind of functions being approximated is specified, such as in sobolev or lipschitz spaces (sobolev space is the space of distributions in the sense of schwartz, in )(wlp whose derivatives of order k also belong to the space )(wlp , where w is an open subset of nr . a function )(xf belongs to a lipschitz class lip[ α ] if there exists a constant c > 0 such that α yxcyfxf −≤− )()( . the subset a in a metric space x is dense in x if every x in x is a limit of a sequence of elements in a. the celebrated theorem of weierstrass proved in 1885 asserts that a set of algebraic polynomials is dense in the space of continuous functions on a compact set ijahp article: saaty/decisions, structure, judgment, and natural law international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 21 in dr . a continuous function in a finite close interval can be approximated with any desired accuracy by polynomials and often also by trigonometric polynomials, and some mathematicians have recently considered approximation by nonlinear homogeneous polynomials on star-like origin-symmetric surfaces (of which convex surfaces are a special case by a pair of homogeneous polynomials as a conjecture). their work is closely related to the findings of the general solution for the operator formulation of the fundamental equation examined in the next section. • in 1937 marshall stone generalized the idea of approximations and density in what is known as the stone–weierstrass theorem: if k is a compact hausdorff space (a set a of functions defined on k is said to separate points – is a hausdorff space – if, for every two different points x and y in k there is a function f in a with f(x) not equal to f(y), and a is a sub algebra of c(k,r) which contains a non-zero constant function, then a is dense in c(k,r) if and only if it separates points. the original statement of weierstrass is a special case because polynomials on [a,b] form a sub algebra of c[a,b] which separates points. runge's approximation theorem in complex analysis says that: if k is a compact subset of c (the set of complex numb)rs), a is a set containing at least one complex number from every bounded connected component of c\k, and f is a holomorphic function on k, then there exists a sequence )( nr of rational functions with poles in a such that the sequence )( nr approaches the function f uniformly on k. the world is made up of many stimuli of different kinds. our brain responds to each kind that it detects with different neurons each specialized for only one aspect of a stimulus. these responses are then synthesized into a complete response. both real and complex valued solutions are dense in very general spaces because we respond to everything we sense or think about with such functions. we expect those spaces to be the most general conceivable. the fact that we think mathematically of the world in terms of functions of several variables is only an abstraction of what actually happens with our brain constructions. in 1957, the russian mathematician a. n. kolmogorov in responding to hilbert’s 13th problem (that not only one cannot express the solution of higher order algebraic equations in terms of basic algebraic operations, but no matter what functions of one or two variables we add to these operations, we still would not be able to express the general solution) actually showed hilbert to have been wrong and proved that an arbitrary continuous function 1( ,..., )nf x x on an n-dimensional cube (of arbitrary dimension n) can be represented as a superposition and composition of continuous functions of only one variable. here is the remarkable theorem of kolmogorov: theorem (kolmogorov, 1957): for every integer dimension 2,d ≥ there exist continuous real functions ( )ijh x defined on the unit interval [0,1]u , such that for every continuous ijahp article: saaty/decisions, structure, judgment, and natural law international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 22 real function 1( ,..., )df x x defined on the d-dimensional unit hypercube du , there exist real continuous functions ( )ig x such that 2 1 1 1 1 ( ,..., ) ( ( )). d d d i ij j i j f x x g h x + = = = ∑ ∑ because of their relation to the brain, we conjecture that the resulting one dimensional functions always have the same form as the solution to the fundamental equation involved in the firing of neurons. now we come to the most important concern we have that also derives from a generalization of judgments and it is about the mathematical form that response to influences in nature takes. for that purpose we use a general operator form of our fundamental equation. 9. the operator equation for stimulus-response julian huxley wrote, “something like the human mind might exist in lifeless matter.” huxley suggested that all natural occurrences involve mental activity, although the mental happenings are at such a low level of intensity that they cannot be detected. in higher animals mental activity is reinforced through an organized system like the brain to reach a high level of intensity; therefore we become aware of it. all nature has a degree of awareness and solves problems. let us now see what we can learn about the characteristics of things, all things in general respond to stimuli. our assertion here is that all quantitative formulas used in science to study nature, physical or biological, including the brain itself, take the form of one of the few solutions of our operator equation. we map the normed linear space e to another normed linear space g over k, k is either r (real) or c (complex), and and α β are given scalars in k with ( )α βw x w x( ) = replace x by α/x to get 2 2 2 2 ( 1) ( 1) ( ) ( ) or ( ) ( ) ( ) ... ( ) ... ( ) ... ( ) p p p p n p n p x x x x w x w w w w x w β β β β α α αα β α − − = = = = = = = = = x 1 w w x α β the equation has solutions derived by nicole brillouet-belluot of nantes, france, in response to an inquiry we made in 1998 about this operator equation to janos aczel who lectured on the solution of the simpler equation given above at a meeting on functional ijahp article: saaty/decisions, structure, judgment, and natural law international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 23 equations. the solutions are shown in table 1 below. it is clear that a non-zero solution exists depending on whether and α β are or are not roots of one. table 1 different solutions of ( ) ( )w ax bw x= . :w e g→ general solution α root of 1 of order n 1nβ ≠ 1) 0w ≡ 1nβ = 2) b subset of e α not root of 1 same as 2) continuous solutions α root of 1 of order n 1nβ = same as 2) with limit conditions α not root of 1, | α|=1 0w ≡ where b = 0, if β ≠ 1, is an arbitrary element of g, if β = 1, w (x) pβ w 0 (α -p x), if x ∈ pα b and p ∈ {0, .., n − 1}, b, if x = 0. ijahp article: saaty/decisions, structure, judgment, and natural law international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 24 | | 1β ≠ | | 1β = root of 1 and not equal to one β 0w ≡ root of 1 and equal to one β w (x) = 0w ( π (x)) where 0w is an arbitrary continuous mapping and ( ) is the continuous natural mappingxπ p not root of 1 and for all p β β α= and if e is a separable inner product space, by: ( ) ( ) ( ) p i x i x x w x x ⎛ ⎞ ⎜ ⎟= ⎜ ⎟ ⎝ ⎠ ( )i xg ( ) ( ) ( ) ( ) ( )( ) ( ), 0 i x i x i x i x i x x x x x e x x ⎛ ⎞ ⎜ ⎟− ≠ ⎜ ⎟⎜ ⎟ ⎝ ⎠ and w(0) = 0, where the functions gn are continuous and subject to some conditions. | α|>1 | | 1β ≤ 1β ≠ 0w ≡ 1β = constant functions | | 1β > same as in 2) above with b=0 and p = ln ln x α ⎡ ⎤ ⎢ ⎥ ⎢ ⎥⎣ ⎦ and 0w arbitrary continuous function satisfying a limit condition. differentiable solutions α root of 1 of order n 1nβ = in this case, there exists a unique p in {0, 1, . . . , n − 1} such that pβ α= ( ) ( ) 1 1... 1 1 1 ,..., ,..., ... q q q nn q n n q n n j w x x a x x ∈ = ∑ where j ={(n1, .., nq) ∈nq : n1 + .. + if e = c, w (x)= ( )0 , p x w x x ⎛ ⎞ ⎜ ⎟⎜ ⎟ ⎝ ⎠ if x ≠ 0, 0, if x = 0 where 0 (0, )w g+∞ → ijahp article: saaty/decisions, structure, judgment, and natural law international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 25 nq = p + j n for some j ∈n} and 1 ... qn n a are arbitrary elements of g α not root of 1, | α|=1 | | 1β ≠ 0w ≡ | | 1β = if is root of 1 and 1β β ≠ 0w ≡ if is root of 1 and 1β β = the solution is the constant functions if p is not a root of 1 and for all nonnegative integers pβ β α≠ 0w ≡ if p is not a root of 1 and for all nonnegative integers pβ β α= , the homogeneous polynomials of degree p | α|>1 | | 1β ≤ 1β ≠ 0w ≡ 1β = constant functions | | 1β > kβ α≠ for all positive integers k 0w ≡ for some k kβ α= w(x) = l(x, x, . . . , x) (x ∈ cq), where l is an arbitrary k-linear symmetric continuous mapping from (cq)k into g. analytic solution ( ) =1, = jk q k qj j w z c z a βα ∞ + =−∞ = ∑ a mapping is a correspondence. it is said to be a natural mapping when it transforms a structure through arithmetic operations; that is, it applies an arithmetic operation to make the correspondence such as in multiplying the positive integers by two to get the even numbers and thus obtaining a 1-1 correspondence rather than by removing the odd numbers which would not be an arithmetic operation. a multilinear form is a map : nf v k→ where v is a vector space over the field k that is linear in each of its n variables. the term multilinear map is used when the map is linear in all its arguments. for n = 2, f is called a bilinear form. a klinear from is a multilinear form with k arguments. an important type of multilinear forms is alternating multilinear forms which change their sign under exchange of two arguments. they are called symmetric multilinear (k-linear) forms when unchanged on interchanging two arguments. ijahp article: saaty/decisions, structure, judgment, and natural law international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 26 homogeneous polynomials of degree p ( ) ( ) 1 1... 1 1 1 ,..., ,..., ... q q q nn q n n q n n j x x a x xφ ∈ = ∑ where j = { (n1, .., nq) ∈nq : n1 + .. + nq = p + j n for some j ∈n} and a 1 ... qn n a are arbitrary elements of g. in his book a mathematician grappling with his century, springer, 2001, laurent schwartz writes on page 134, “i tried to define fourier transforms of operators, and that is where i encountered a total obstacle. the very special role of convolution in the definition of operators leaves absolutely no room for a fourier transform.” but there are works on fourier transforms of operators. we are not sure whether the transform can be defined and used for practical purposes. 10. geometric representations our purpose has been to show that the expression we obtain for neural firing can be validated in practice by showing that linear combinations are dense in the most general spaces that can be used to represent nearly everything we do with electric firings of neurons. but how can we do that? while a function of a real variable depends on a single argument, a function of a complex variable depends on two independent variables one real and one imaginary and thus cannot be drawn in the plane. further, complex valued functions cannot be drawn as one does ordinary functions in three dimensions because they depend on an imaginary variable. nevertheless, one can make a plot of the modulus or absolute value of such a function. the basic assumption we make to represent the response to a sequence of individual stimuli is that all the layers in a network of neurons are identical, and each stimulus value is represented by the firing of a neuron in each layer to synthesize in stages specialized information from each receptor neuron into a more complex package that characterizes more closely what is perceived. this representation it is not invariant with respect to the order in which the stimuli are fed into the network. it is known in the case of vision that the eyes do not scan pictures symmetrically if they are not symmetric, and hence our representation must satisfy some order invariant principle. taking into account this principle would allow us to represent images independently of the form in which stimuli are input into the network. for example, we recognize an image even if it is subjected to a rotation, or to some sort of deformation. thus, the invariance principle must include affine and similarity transformations. this invariance would allow the network to recognize images even when they are not identical to the ones from which it recorded a given concept, e.g., a bird. the next step would be to use the network representation given here with additional conditions to uniquely represent patterns from images, sounds and perhaps other sources of stimuli such as smell. our representation focuses on the real part of the magnitude rather than the phase of the fourier transform. tests have been made to see the effect of phase and of magnitude on the outcome of a representation of a complex valued function. there is much more blurring due to change in magnitude than there is to change in phase. thus we focus on representing responses ijahp article: saaty/decisions, structure, judgment, and natural law international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 27 in terms of dirac functions, sums of such functions, and on approximations to them without regard to the coefficients in the linear combination. the rest of this section on geometric representations in the plane to give an idea how density works was developed in collaboration with my colleague luis vargas. the most significant observation about the brain, which consists of many individual neurons, is that it is primarily a synthesizer of the firings of individual neurons into clusters of information and these in turn into larger clusters and so on, leading to an integrated whole. due to their sequential nature, the firings of a neuron that precede other neurons would be lost unless there is something like a field in which all the firings fit together to form a cohesive entity which carries information. is there a field in the brain? no. we believe that the process of analytic continuation in the theory of functions of a complex variable provides insight into how neurons seem to know one another. on page 373 in their book from neuron to brain,, kuffler and nicholls say, “the nervous system appears constructed as if each neuron had built into it an awareness of its proper place in the system.” that is what analytic continuation does. it conditions neurons to fall on a unique path to continue information that connects with information processed by adjacent neurons with which it is connected. the uniqueness of analytic continuation has the striking consequence that something happening on a very small piece of a connected open set completely determines what is happening in the entire set, at great distances from the small piece. by raising the hypermatrix to powers one obtains transitive interactions. this means that a neuron influences another neuron to fire or not through intermediate neurons. all such two step interactions are obtained by squaring the matrix. three step interactions are obtained by cubing the matrix and so on. by raising the matrix to sufficiently large powers, the influence of each neuron on all the neurons with which one can trace a connection, yields the transient influence of neurons in the original hypermatrix. multiplying the hypermatrix by itself allows for combining the functions that represent the influence from pre-to postsynaptic neurons to accumulate all the transitive influences from one neuron to another and allow for feedback. the fourier transform that takes place as a result of firing and the density of the resulting firings give us the desired synthesis. depending on what parts of the brain are operational and participating in the synthesis, different physical and behavioral attributes are observed to take place, including consciousness related to the fourier transform of the single valued sensory functions. the functions result from modeling the neural firing as a pairwise comparison process in time and are used to approximate pulses. it is assumed that a neuron compares neurotransmittergenerated charges in increments of time. we created a 2-dimensional network of neurons consisting of layers. for illustrative purposes, we assume that there is one layer of neurons corresponding to each of the stimulus values. thus, if the list of stimuli consists of n numerical values, we created n { }0 , ,et t≥βαβα ijahp article: saaty/decisions, structure, judgment, and natural law international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 28 layers with a specific number of neurons in each layer. under the assumption that each numerical stimulus is represented by the firing of one and only one neuron, each layer of the network must also consist of n neurons with thresholds varying between the largest and the smallest values of the list of stimuli. we also assumed that the firing threshold of each neuron had the same width. thus, if the perceptual range of a stimulus varies between two values θ 1 andθ 2 , and each layer of the network has n neurons, then a neuron in the ith position of the layer will fire if the stimulus value falls between and sound experiment in the sound experiment we first recorded with the aid of the mathematica software the first few seconds of haydn's symphony no.102 in b-flat major and mozart's symphony no. 40 in g minor. the result is a set of numerical amplitudes between -1 and 1. each of these amplitudes was used to make neurons fire when the amplitude falls within a prescribed threshold range. under the assumption that each neuron fires in response to one stimulus, we would need the same number of neurons as the sample size, i.e., 117,247 in haydn's symphony and 144,532 in mozart's symphony. our objective was to approximate the amplitude using one neuron for each amplitude value, and then use the resulting values in mathematica to play back the music. a small sample of the numerical data for mozart’s symphony is displayed in figure 6. figure 6 mozart symphony no. 40 this task is computationally demanding even for such simple geometric figures as the bird and the flower shown in figures 7 and 8. for example, for the bird picture, the stimuli list consists of 124 values, and we would need 1242=15376 neurons, arranged in 124 layers of 124 neurons each the network and the data sampled to form the picture given in figure 6, were used to create a 124 by 124 network of neurons consisting of 124 layers with 124 neurons in each layer. each dot in the figure is generated by the firing of a neuron in response to a stimulus falling within the neuron's lower and upper thresholds. the sound experiment required 1000 times more data points than the picture experiment. once the (x, y) coordinates of the points were obtained, the x-coordinate was used to represent time and the y-coordinate to represent response to a stimulus picture experiment in the graphics experiment the bird and rose pictures required 124 and 248 data points, respectively. the numerical values associated with the drawings in figures 7 and 8 were 1 n 1) (i + 121 θθ θ 1-n i + 121 θθ θ ijahp article: saaty/decisions, structure, judgment, and natural law international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 29 tabulated and the numbers provided the input to the neurons in the networks built to represent the bird and the rose. figure 7 bird figure 8 rose 11. conclusions there is still much work to be done on the detailed interpretation of the solutions of the operator equation as they correspond to responsive forms in the real world. ijahp article: saaty/decisions, structure, judgment, and natural law international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 30 i am grateful to three people for their help in the development of some of the ideas in this work. i mention them in the order and extent of help: my friend luis vargas for our close friendship and for early work on fredholm’s equation (first derived in a paper i coauthored with my student hassan ait-kaci) and for the numerical representation of sounds and images; my friend janos aczel for his help with the solution of the functional equation i derived as necessary condition for the existence of a solution to fredholm’s equation and to nicole brillouet-belluot for solving the generalization of that equation to operator form. references aczél, j. d. and m. kuczma. generalizations of a folk-theorem. vol. 19, results in mathematics 5-21. basel: birkhäuser verlag (1991). brillouet-belluot, n. “on a simple linear functional equation on normed linear spaces.” ecole central de nantes, (1999). huxley, j.: man in the modern world, new york: mentor books, 1953. kolmogorov, a.n. (1957). “on the representation of continuous functions of many variables by superposition of real functions of one variable and addition. american marthematical society translations issue series 2, 28:55-59 (1963). saaty, t.l.: the brain, unraveling the mystery of how it works, the neural network process, pittsburgh, pennsylvania: rws publications, 4922 ellsworth avenue, pittsburgh, pa 15213 (2000). saaty, t.l. and l.g. vargas: a model of neural impulse firing and synthesis. j. of math. psych. 200-219, 2 (1993). saaty, t.l., and l.g. vargas: representation of visual response to neural firing. math. and comp. mod. 17-23, 18/7 (1993). ijahp article: yalcin, ozturk/determining the area sizes of each product category in a department store using multi-criteria decision making methodologies international journal of the analytic hierarchy process 3 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.602 determining the area sizes of each product category in a department store using multicriteria decision making methodologies gulcin dinc yalcin eskisehir technical university turkey gdinc@eskisehir.edu.tr zehra ozturk turkey zkamisli@eskisehir.edu.tr abstract in a department store, customers have the opportunity to reach a wide range of consumer goods from different product categories within a single store area. store layouts generally show the size and location of each department, any permanent structures, fixture locations, and customer traffic patterns. determining the area sizes to be allocated to each product category and the layout of these areas in the department store is a strategic planning decision problem. the layout problem has been studied in the literature with different approaches where the sizes of the areas are known. the first purpose of this paper is to determine the area sizes of each product category. customers decide to go to a department store for several reasons including the quality of products, services, location, etc. these reasons have been studied in the literature. however, “for which product categories do customers decide to go to a department store” is an open question. the second purpose of this paper is to find the frequency of product categories from the viewpoint of the customers. therefore, our aim is to obtain the required results in a systematic way with multi-criteria decision making methodologies. for this purpose, we perform the analytic network process (anp) and the analytic hierarchy process (ahp) from the viewpoints of department managers and customers, respectively. in the anp model, several tangible and intangible criteria such as product costs, the demands of customers, sales history, overall inventory, floor space and relationship with suppliers are chosen, and the intersections between them are specified. pairwise comparisons are made by department store managers. the anp outcome is the weight of each product category, and these weights are considered the percentage of the area size within the store from the viewpoint of the department stores. in the ahp model, a simple model is constructed to define the customers’ preference for each product category. pairwise comparisons between product categories are made by the customers. therefore, the outcome of the ahp model is the weight of each product category, and this is the preference of each product category from the viewpoint of the customers. the outcomes show that these weights may be different. this is an expected situation since even if a mailto:gdinc@eskisehir.edu.tr mailto:zkamisli@eskisehir.edu.tr ijahp article: yalcin, ozturk/determining the area sizes of each product category in a department store using multi-criteria decision making methodologies international journal of the analytic hierarchy process 4 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.602 product category is preferred by some as the driver to visit a department store, the footprint of that category in the actual store may be small. the outcome from customers provides feedback to department store managers on which product category should be diversified as well as the area sizes of those categories. keywords: department store; space allocation; multi-criteria decision making; analytical network process; analytical hierarchy process; product category; layout 1. introduction the retail industry is one of the largest and most diversified operations in the world. the structure of this industry connects manufacturers to consumers by providing products and services from the producer to the customer. retailers are dynamic in nature and need to keep their strategies competitive and profit-oriented. product assortment, the collection of goods and/or services a retail store offers to customers, is a factor that affects profitability and has a high priority for retailers as they work with a limited budget, store size and shelf space (kumar et al. 2017). department stores are a kind of retail establishment and provide a wide range of product categories to customers. many studies have been conducted from different perspectives about department stores in the literature. gardner et al. (2002) and park (2012) analyzed the sampoong department store collapse. location evaluation of department stores in a major metropolis was studied by doucet (2003). an accounting technique in a department store was investigated by walsh and jeacle (2003). later on, wargocki et al. (2004) studied the sensory pollution loads of department stores. miller (2006) discussed strategic human resource management in department stores. kernsom (2010) and sahachaisaeree (2012) studied the importance of windows in department stores. a building-energy load model in department stores was investigated by chung and park (2012). while eckert et al. (2015) analyzed location patterns near department stores, ratanatamskul and siritiewsri (2015) studied the development of compact anaerobic treatment of department store wastewater. the creation of a compelling brand meaning by orchestration in a department store in scandinavia was investigated by hjelmgren (2016), and investigation of indoor air quality of department store buildings was discussed by cheng et al. (2017). in addition to the subjects of these studies, layout is a critical factor driving consumer elaboration and response in retailing. as behera et al. (2017) mentioned, store location and layout are essential variables that influence shopper conduct and are a basic determinant of the overall image of a store. well composed store layouts are critical because they impact in-store movement designs, shopping environment, shopping conduct, and operational productivity. this problem has often been studied in the literature (peters et al., 2004; bostani and peters, 2005; yapicioglu and smith, 2012a, 2012b; cil, 2012; ozcan and esnaf, 2013; ballester et al., 2014), and the objective is to maximize revenue or adjacency satisfaction and minimize traffic density or traffic distance. the common assumption in these studies is that the area sizes for product categories are known. ijahp article: yalcin, ozturk/determining the area sizes of each product category in a department store using multi-criteria decision making methodologies international journal of the analytic hierarchy process 5 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.602 furthermore, multi-criteria decision making techniques are used for retail stores. akalin et al. (2013) solved the retail store location selection problem for a clothing store using an analytical hierarchy process (ahp) model with the preferences of the retailers. eroglu (2013) also used ahp to determine the consumer preferences for product attributes for retailer selection. as seen in the literature, there has not been much attention paid to the determination of the area sizes of product categories. this study contributes to the literature by filling this gap. the managers whose job it is to work through the decision making process of these area sizes consider certain criteria such as customer needs, depot space, suppliers, etc. in other words, managers have alternatives and conflicting criteria. on the other hand, customers decide to go to a department store for one or more of these product categories. some product categories may be preferred more than others. namely, customers are making a comparison among the product categories. multi-criteria decision making techniques use the general theory of measurement to derive relative properties. the analytic hierarchy process (ahp) and the analytic network process (anp) are two multi-criteria decision making techniques. the ahp has a hierarchical structure and the anp has a network structure to model a decision problem, but both of these methods can include quantitative and qualitative criteria. the decision model for customers is considered a simple model with paired comparison between product categories. the decision model for the managers is a network model where each criterion influences the others. therefore, in this study, the anp and the ahp were chosen to construct the outcomes from the viewpoints of department managers and customers, respectively. first, several tangible and intangible criteria (product costs, demand of customers, sales history, overall inventory, floor space and relationship with suppliers, etc.) were specified by the department store managers. interactions among these criteria were also presented (e.g. the demands of customers influences sales history). then, an anp model was constructed since anp is a suitable technique to manage inner and outer dependencies between criteria. in addition to department store managers, the customers’ preferences were considered in this study because customers may buy goods from one product category more frequently than goods from any other product category (e.g. outfits may be purchased six times more than cosmetics by a customer). product categories are determined as the criteria, and then, the ahp is used to evaluate the paired comparisons among the product categories to obtain the order preference of each product category from the customers’ perspective. this paper is organized as follows: the anp model that was developed to determine the area sizes of each product category from the viewpoint of the department store managers is given in detail in section 2. then, the ahp model that was developed to determine the customer preferences of each product category is discussed in section 3. finally, the conclusions are given in section 4. ijahp article: yalcin, ozturk/determining the area sizes of each product category in a department store using multi-criteria decision making methodologies international journal of the analytic hierarchy process 6 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.602 2. multi-criteria decision making model for a department store in this section, a brief description of the multi-criteria decision making techniques used in this study is given. also, the model that was developed for product categories in department stores is introduced. 2.1 methodology the analytic hierarchy process (ahp) developed by saaty (1980) is a general theory of measurement to derive relative priorities on absolute scales from paired comparisons within a hierarchic structure (saaty & vargas, 2006). decision makers who use the ahp method structure the problem into a hierarchy with the top level as the overall objective, and the bottom level that includes the action alternatives or the alternatives that would contribute positively or negatively to the main objective through their impact on the criteria in the intermediate levels of the hierarchy (sagir & ozturk, 2010). when using the ahp or its generalization on feedback networks, the analytic network process (anp), to model a problem, one needs a hierarchic or a network structure to represent that problem, as well as pairwise comparisons to establish relationships within the structure. many decision problems cannot be structured hierarchically because they involve interaction and dependence on higher-level elements. not only does the importance of the criteria determine the importance of the alternatives, but the alternatives themselves also determine the importance of the criteria (saaty & vargas, 2006). the ahp is applicable to individual and group decision settings. in group decision making, the individuals’ judgments can be calculated by the geometric mean which is the most common approach used in ahp with group decision making (lai et al., 2002; garcia et al., 2006). the anp represents a decision making problem as a network of criteria and alternatives grouped in clusters. all of the elements in the network can be related in a possible way, i.e., a network can incorporate feedback and interdependence relationships within and between clusters. this provides a way to more accurately model complex decisions (sagir & ozturk, 2010). there are two kinds of influence: outer and inner. in the first, one compares the influence of elements in a cluster on elements in another cluster with respect to a control criterion. in inner influence, one compares the influence of elements in a group to each one in the group. if we think about it carefully, everything may be seen to influence everything including itself according to many criteria. the world is far more interdependent than we know how to deal with using our existing ways of thinking and acting. the anp is a logical way to deal with dependence. the fundamental scale of values to represent the intensities of judgments is a 1-9 scale. the scale represents importance as follows: 1 is equal importance, 3 means moderate importance, 5 means strong importance, 7 means very strong importance, 9 means absolute importance, and 2, 4, 6 and 8 are used to express intermediate values. the priorities derived from the pairwise comparison matrices are entered as a part of the columns of a super matrix. the super matrix represents the influence priority of an element on the left of the matrix on an element at the top of the matrix with respect to a particular control criterion. in the anp, steady-state priorities from a limit super matrix are investigated. to obtain the limit, the matrix is raised to powers. each power of the matrix captures all of the transitivity of an order that is equal to that power. the limit of these powers, according to cesaro summability, is equal to the limit of the sum of all of ijahp article: yalcin, ozturk/determining the area sizes of each product category in a department store using multi-criteria decision making methodologies international journal of the analytic hierarchy process 7 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.602 the powers of the matrix. all order transitivity is captured by this series of powers of the matrix. the outcome of the anp is nonlinear and rather complex. the limit may not converge unless the matrix is column-stochastic, that is, each of its columns sums to one. if the sum of the columns is one, then because the principal eigenvalue of a matrix lies between its largest and smallest column sums, it is known that the principal eigenvalue of a stochastic matrix is equal to one (saaty, 2001). let 𝐶ℎ, where ℎ = 1, … , 𝑁 be a component of a decision network, and it has 𝑛ℎ elements, where 𝑒ℎ1, 𝑒ℎ2, … , 𝑒ℎ𝑛ℎ is denoted. thus, the elements for 𝐶1 are 𝑒11, 𝑒12, … , 𝑒1𝑛1, for 𝐶2 are 𝑒21, 𝑒22, … , 𝑒2𝑛2, and finally for 𝐶𝑁 are 𝑒𝑁1, 𝑒𝑁2, … , 𝑒𝑁𝑛𝑁. given this, the priority vector from the paired comparisons is derived. these comparisons depict the influences of a given set of elements in a component on any element in the system (sengupta, gupta & dutta, 2017). the influence of the elements in the network may be represented in the super matrix, and a typical entry 𝑊𝑖𝑗 in the super matrix is called a block of the super matrix. figure 1 shows these matrices (saaty, 2001; saaty & vargas, 2006). each column of 𝑊𝑖𝑗 is a principle eigenvector of the influence (importance) of the elements in the 𝑖th component of the network on an element in the 𝑗th component. figure 1 super matrix of a network and details of a component briefly, the anp has four major steps (girginer et al., 2007; meade and sarkis, 1998; saaty, 1996): 1. model construction and problem structuring: the problem is modeled as a network. 2. pairwise comparison matrices and priority vectors: pairwise comparison matrices between criteria and alternatives are conducted by judgments which are made with a 1-9 scale. 3. super matrix formation: an eigenvector for each column block is calculated by the row components with respect to the column components of the pairwise comparison matrix. the blocks in each column of the matrix are weighted, known as the weighted super matrix. the final priorities of all of the elements are determined by normalizing each block of the super matrix (see fig. 3). 4. obtaining weights and selection of best alternatives: the priority weights of alternatives are computed in the column of alternatives in the normalized superijahp article: yalcin, ozturk/determining the area sizes of each product category in a department store using multi-criteria decision making methodologies international journal of the analytic hierarchy process 8 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.602 matrix. the alternative with the largest overall priority may be selected as the best option among alternatives or the weights may be used for various purposes. 2.2 anp model for the department managers first, the clusters, factors, and alternatives are explained, and the model is given. then, the outcomes of the anp model are discussed. 2.2.1 clusters, factors, and alternatives department store managers determine the size of space that will be reserved for each product category in their department stores. based on this fact, the alternatives and criteria were defined based on the literature and opinions of four department store managers. these managers are responsible for formulating policies and operating procedures for the store, planning and organizing store activities, etc. in order to obtain a wide range of opinions, the selected managers (four) were from different companies or different branches of the same company. the product categories were determined as the alternatives and five clusters, and ten factors were defined. 1. alternatives: the alternatives are outfits, shoes, bags and accessories, underwear, cosmetics, baby and kid’s wear, and sports goods. these were the existing product categories of the department stores that were part of this study. 2. budget: the budget cluster contains the factors cost of product and markups of product (bahng & kincade, 2014) and overall inventory (silver et al., 1998; bahng & kincade, 2014). product costs and markups are the basis for the initial price of stock-keeping units (sku) and they form a foundation for the calculation of net sales, gross margins, and other profitability measures. stock is the term used for merchandise that is on the floor, in back rooms and on order (bahng & kincade, 2014). budget was chosen as a cluster since the product categories defined in the alternatives are bought from vendors. therefore, the cost of the purchasing process is naturally affected by the cost of the products and markups of products. the overall inventory is checked before a decision is made about the quantity to be purchased. the factors in the budget cluster only influence the alternatives. 3. product: the product cluster contains the sales history factor (bahn & kincade, 2014). product and sales history were chosen as a cluster and a factor since sales forecasting is a significant factor involved in making the purchase process efficient. therefore, sales history is needed for forecasting. sales history is obtained from the previous sales for each product category, and only influences the alternatives. 4. customer: the customer cluster contains the factors that are the characteristics of target customers and demands of the target customers (duncan, 1972; mantrala et al., 2009; bahng and kincade, 2014) and the customers’ disposable income (mantrala et al., 2009; bahng and kincade, 2014). the characteristics of the target customers (such as age, marriage status, and educational level) influence the sales history and alternatives. for example, the more married couples with children who choose a department store, the higher the amount of baby and kid’s wear is sold. the demands of the customers influence the overall inventory, sales ijahp article: yalcin, ozturk/determining the area sizes of each product category in a department store using multi-criteria decision making methodologies international journal of the analytic hierarchy process 9 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.602 history, and alternatives. the more demand for a product category, the more the inventory will be sold out. if customers’ disposable income is high, the area size for the more expensive product categories may be increased in the department store. 5. store: the store cluster has the factors floor space and economic conditions of the store’s region (mantrala et al., 2009, bahng & kincade, 2014) and depot space. in pursuance of floor space, the area sizes are determined. if the department store has a large floor space, then a larger area size may be considered for each product category. floor space influences the alternatives. the economic condition of the store’s region also influences floor space, sales history, characteristics of the target customers and alternatives. if the region is home to high-income workers, then the floor space is expected to be bigger, and the amount of products sold larger. depot space influences the overall inventory and alternatives. the more depot space the department store has, the more inventory is kept. 6. suppliers: the supplier cluster has the factor relationship with suppliers (duncan, 1972; wagner et al., 1989; silva et al., 2002; kannan & tan, 2006; bahng & kincade, 2014). relationship with suppliers influences product costs and overall inventory. a good relationship may ensure lower product costs and the ability to purchase fewer products at one purchase, or in other words, less inventory. the clusters, factors, their related literature and influencing factors that are mentioned above are summarized in table 1. also, the anp model with all of the clusters, factors, alternatives and inner and outer dependencies is shown in figure 2. ijahp article: yalcin, ozturk/determining the area sizes of each product category in a department store using multi-criteria decision making methodologies international journal of the analytic hierarchy process 10 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.602 table 1 clusters, factors and related literature of the department store model cluster factors literature influences budget product costs bahng and kincade (2014) alternatives markups of product bahng and kincade (2014) alternatives overall inventory silver et al. (1998) and bahng and kincade (2014) alternatives product sales history bahng and kincade (2014) alternatives customer characteristics of target customer duncan (1972), mantrala et al. (2009) and bahng and kincade (2014) sales history, alternatives demand of target customer duncan (1972), mantrala et al. (2009) and bahng and kincade (2014) overall inventory, sales history, alternatives customers’ disposable income mantrala et al. (2009) and bahng and kincade (2014) sales history, characteristics of target customers, demand of target customers, alternatives store floor space mantrala et al. (2009) and bahng and kincade (2014) alternatives economic condition of the store’s region mantrala et al. (2009) and bahng and kincade (2014) floor space, sales history, characteristics of target customers, alternatives depot space overall inventory and alternatives suppliers relationship with suppliers silva et al. (2002), duncan (1972), kannan and tan (2006), wagner et al. (1989) and bahng and kincade (2014) product costs, overall inventory ijahp article: yalcin, ozturk/determining the area sizes of each product category in a department store using multi-criteria decision making methodologies international journal of the analytic hierarchy process 11 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.602 figure 2 anp model for the product category of the department store the relationships of the anp model are as follows: the budget cluster influences the alternatives cluster. the customer cluster influences the product since all of the factors (characteristic of target customers, demand of target customers, customers’ disposable income) affect sales history. additionally, it influences the budget cluster since the demands of the customers affects the overall inventory. it also influences the alternatives cluster and itself. the product cluster influences the alternative clusters. the store cluster influences the budget since the depot space affects the overall inventory. also, it influences the customer cluster since the economic conditions of the store’s region affects the characteristics of the target customers. further, it influences the product cluster since the economic condition of the store’s region affects the sales history. it also influences the alternatives cluster and itself. the supplier cluster influences the budget cluster since the relationship with suppliers affects the overall inventory and product cost. the relationships are shown in the relation matrix (fig. 3) where rows and columns represent the factors of the anp model. “+” means that i th row factor influences the j th column factor. the numbers are taken from table 1 and the alternatives are shown as 0 (zero). ijahp article: yalcin, ozturk/determining the area sizes of each product category in a department store using multi-criteria decision making methodologies international journal of the analytic hierarchy process 12 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.602 𝐹𝑎𝑐𝑡𝑜𝑟𝑠 1.1 1.2 1.3 2.1 3.1 3.2 3.3 4.1 4.2 4.3 5.1 [ 1.1 1.2 1.3 2.1 3.1 3.2 3.3 4.1 4.2 4.3 5.1 0 − − − − − − − − − − − + − − − − − − − − − − − + − − − − − − − − − − − + − − − − − − − − − − − + − − − + − + + − − − − + − − + + + − + − − − − + − − − + + + − − − − − + − − − − − − − − + + − + − − − + + − − + − + − + − − + − − − − + + − − + + − + − − − − − − − − −] 1.1 product costs, 1.2 markups of product, 1.3 overall inventory, 2.1 sales history, 3.1 characteristics of target customers, 3.2 demand of target customers, 3.3 customers’ disposable income, 4.1 floor space, 4.2 economic condition of the store’s region, 4.3 depot space, 5.1 relationship with suppliers. 0. alternatives figure 3 relation matrix of the anp model four department store managers in turkey performed the paired comparisons among the factors, clusters and the percentage of space given to the product categories (alternatives). a question similar to “with respect to markups of a product, how much space do you give to baby and kid’s wear rather than cosmetics?” was asked for each factor during a face-to-face interview. also, a question similar to “with respect to customers, how much more important are alternatives than budget?” was asked for the related clusters. these judgments are made based on a 1-9 scale that was discussed in section 2.1, and the geometric mean is calculated. 2.2.2 outcome of the anp model figure 4 shows a screen view for the paired comparison of factors and priorities that were produced from the judgment and inconsistency of the judgment matrix. baby and kid’s wear should have 38% of the total store area with respect to the markups of product. likewise, the inconsistency of the matrix is 0.08304. figure 5 shows a screen view for the paired comparison of clusters and priorities that was produced from the judgment and inconsistency of the judgment matrix. the customer cluster is the most important cluster with a value of 0.54 with respect to customer. this comparison is consistent with a 0.08687 inconsistency ratio. after each comparison related to each factor was made, the priorities for alternatives and factors were obtained and are shown in table 2. the product category, which has the biggest percentage of space in the department store, may be different for each factor. “outfits” should have the biggest percentage of space with respect to eight of the ten factors, while “baby and kid’s wear” has the biggest percentage of space with respect to markups of products, and “outfits” and “sports goods” have the biggest percentage of space with respect to customers’ disposable income. ijahp article: yalcin, ozturk/determining the area sizes of each product category in a department store using multi-criteria decision making methodologies international journal of the analytic hierarchy process 13 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.602 figure 4 screen view of paired comparison of factors and priorities produced from the judgment and inconsistency of the judgment matrix figure 5 screen view of the paired comparison of clusters and priorities produced from the judgment and inconsistency of the judgment matrix ijahp article: yalcin, ozturk/determining the area sizes of each product category in a department store using multi-criteria decision making methodologies international journal of the analytic hierarchy process 14 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.602 table 2 priority of alternatives and/or factors related to each factor clusters 1.budget 2.product 3.customer 4.store 5.supplier s alternative s/ factors 1.1 1.2 1.3 2.1 3.1 3.2 3.3 4.1 4.2 4.3 5.1 outfits 0.3 9 0.3 1 0.3 4 0.4 0 0.3 7 0.4 4 0.2 2 0.6 0 0.3 9 0.4 4 shoes, bags and accessories 0.0 7 0.0 9 0.0 9 0.1 0 0.1 0 0.1 1 0.2 0 0.0 9 0.1 1 0.1 4 underwear 0.0 9 0.1 2 0.0 5 0.0 3 0.0 3 0.0 2 0.0 6 0.0 2 0.0 6 0.0 4 cosmetics 0.0 4 0.0 3 0.0 3 0.0 5 0.0 4 0.0 8 0.1 1 0.0 5 0.0 5 0.0 3 baby and kid’s wear 0.3 1 0.3 8 0.1 2 0.0 9 0.0 7 0.0 5 0.2 0 0.0 6 0.0 8 0.1 0 sport goods 0.0 9 0.0 6 0.3 6 0.3 2 0.3 8 0.3 1 0.2 1 0.1 8 0.3 1 0.2 6 characterist ic of target customers 0.2 5 demand of target customers 0.7 5 overall inventory 0.25 product costs 0.75 1.1 product costs, 1.2 markups of product, 1.3 overall inventory, 2.1 sales history, 3.1 characteristics of target customers, 3.2 demand of target customers, 3.3 customers’ disposable income, 4.1 floor space, 4.2 economic condition of the store’s region, 4.3 depot space, 5.1 relationship with suppliers. after each judgment related to customer and store clusters was determined, the priorities for the clusters were obtained and are shown in table 3. the “customer” cluster has the largest priority with respect to both the customer and store clusters. ijahp article: yalcin, ozturk/determining the area sizes of each product category in a department store using multi-criteria decision making methodologies international journal of the analytic hierarchy process 15 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.602 table 3 priority of clusters related to clusters cluster inconsistency priority customer 0.087 alternative 0.13 budget 0.25 customer 0.54 product 0.08 store 0.067 alternatives 0.14 budget 0.18 customer 0.35 product 0.10 store 0.23 from all of the judgments that were given during the anp comparisons, the weights of the factors were obtained as normalized by cluster and limiting, which are shown in table 4. furthermore, the unweighted, weighted and limit matrices are given in the appendix. the most important factor is the overall inventory using the limiting weights. then, the factors were put in order as product costs, characteristic of target customer, sales history, demand of target customers and floor space. if all of the clusters are evaluated on their own, the overall inventory is more important than product costs for budget, and characteristics of the target customer is more important than the demands of the target customer. ijahp article: yalcin, ozturk/determining the area sizes of each product category in a department store using multi-criteria decision making methodologies international journal of the analytic hierarchy process 16 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.602 table 4 weight of factors cluster factors normalized by cluster limiting budget product costs 0.38 0.075 markups of product 0 0 overall inventory 0.62 0.123 product sales history 1 0.049 customer characteristics of target customer 0.55 0.059 demand of target customer 0.45 0.048 customers’ disposable income 0 0 store floor space 1 0.028 economic condition of the store’s region 0 0 depot space 0 0 suppliers relationship with suppliers 0 0 the outcome of the anp for the alternatives is shown in table 5. in the view of the department store managers, “outfits” has the largest percentage space among the product categories with 37% of total store space. “baby and kid’s wear” and “sports goods” follow outfits with 22% and 21%, respectively. thereafter, “shoes, bags and accessories”, “underwear” and “cosmetics” come after “baby and kid’s wear” and “sports goods” with 10%, 7% and 4%, respectively. therefore, the obtained order of importance is “outfits”, “baby and kid’s wear”, “sports goods”, “shoes, bags and accessories”, “underwear” and “cosmetics”. table 5 outcome of the anp model alternatives normalized by cluster limiting outfits 0.37 0.228 shoes, bags and accessories 0.10 0.059 underwear 0.07 0.045 cosmetics 0.04 0.025 baby and kid’s wear 0.22 0.133 sports goods 0.21 0.127 ijahp article: yalcin, ozturk/determining the area sizes of each product category in a department store using multi-criteria decision making methodologies international journal of the analytic hierarchy process 17 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.602 2.2.3 ahp model for customers flemming (1989) stated that in strategic planning, team planning should be given the highest importance. the plan enables the organization to gain a broad understanding and commitment to the strategic plan by including those who are affected. in this study, those who are affected are the customers. the customers decide to go to a department store for some specific product categories. here, a simple ahp model is developed to determine how often the product categories are preferred in a department store by the customers. in this part of the study, we did not develop an anp model as in the previous section since only the product categories are evaluated to determine the customers’ preferred percentages. the criteria determined for the model are the product categories, and there are no alternatives here. with this model, the aim is to prioritize each product category. in figure 6, the proposed ahp model is shown. the customers were asked to scale the product categories with each other to obtain the percentage of preference for each product category (criteria) in turkey. a question similar to “how often do you go to a department store for baby and kid’s wear rather than cosmetics?” was asked. the judgment was made with a 1-9 scale. to determine the number of experts in a group, saaty and sagir ozdemir (2014) indicated that “the most important aspect is the point at which the weighted sum of errors is least”. this point is reached somewhere between six and eight experts, and the nearest whole number is seven. therefore, seven customers who came into the store during a one hour timeframe on a weekday were randomly selected. the geometric mean was calculated for each of the paired comparisons. the screen view on the paired comparison of the criteria and priorities produced from the judgment and inconsistency of the judgment matrix is shown in figure 7. the comparison was consistent with a 0.09778 inconsistency ratio. figure 6 ahp model ijahp article: yalcin, ozturk/determining the area sizes of each product category in a department store using multi-criteria decision making methodologies international journal of the analytic hierarchy process 18 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.602 figure 7 screen view of the paired comparison of criteria and priorities produced from the judgment and inconsistency of the judgment matrix table 6 outcomes of ahp model and anp models alternatives weights from ahp weights from anp outfits 0.32 0.37 shoes, bags, and accessories 0.22 0.10 underwear 0.07 0.07 cosmetics 0.12 0.04 baby and kid’s wear 0.13 0.22 sports goods 0.14 0.21 the outcomes of the ahp model are summarized in table 6, and the outcomes of the anp model are also included to compare the results of both of the models. in both models, “outfits” was the most preferred product group, while the ranking of the other product groups differed. according to the customers, “outfits” had the biggest preference with 32%, and “shoes, bags, and accessories” followed with 22% among the product categories. next, “sports goods”, “baby and kid’s wear”, and “cosmetics” followed with 14%, 13% and 12%, respectively. finally, “underwear” had a priority of 7%. therefore, the order of priority according to the department store managers is “outfits”, “shoes, bags and accessories”, “sports goods”, “baby and kid’s wear”, “cosmetics”, and “underwear” while the order according to the customers is “outfits”, “baby and kid’s wear”, “sports goods”, “shoes, bags and accessories”, “underwear” and “cosmetics”. ijahp article: yalcin, ozturk/determining the area sizes of each product category in a department store using multi-criteria decision making methodologies international journal of the analytic hierarchy process 19 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.602 3. conclusions and further suggestions in this study, we discussed decision problems related to department stores which offer a wide range of customer goods in different product categories. determination of the area sizes of the product categories is a decision problem from the viewpoint of the department store managers. the improvement of the organization and management of the stores is crucial for them. we chose the anp, which represents a decision making problem as a network of criteria and alternatives, grouped in clusters. all of the elements in the network can be related. first, the product categories were defined as “outfits”, “shoes, bags, and accessories”, “underwear”, “cosmetics”, “baby and kid’s wear” and “sports goods”. these were the existing product categories of the department stores that were in the study. then, we performed a literature review to determine the criteria that may affect the decision about area sizes and discussed them with the department store managers. budget, product, customer, store, and suppliers were defined as the clusters in the decision problem. we also specified the factors within each cluster and defined the intersections. next, pairwise comparisons were made with the department store managers during face-to-face interviews. the area sizes of the product categories were obtained and ranked as “outfits”, “baby and kid’s wear”, “sports goods”, “shoes, bags and accessories”, “underwear” and “cosmetics” with 37%, 22%, 21%, 10%, 7% and 4% of the total store space, respectively. we also determined the preferences for the product categories of the department store customers. we constructed a simple ahp model, and the customers made pairwise comparisons. the preference of product categories were obtained and ranked as “outfits”, “shoes, bags and accessories”, “sports goods”, “baby and kid’s wear”, “cosmetics”, and “underwear” which had 32%, 22%, 14%, 13%, 12% and 7% preference to go to a department store, respectively. the department store managers ranked “outfits” as the highest area size, and the customers ranked “outfits” as their first preference for going to a department store. this was an expected result since “outfits” contains sub-categories such as tops, dresses, jeans, jackets, suits, etc. consequently, the customers had more needs in the “outfits” product category. the department store managers’ ranked “cosmetics” as the least area size, while customers ranked this as fifth in preference. this result was compatible because cosmetic products are smaller than other products. therefore, a wide variety of cosmetics products can be placed in one area. the department store managers ranked “baby and kid’s wear” as the second largest area size, while customers ranked this category fourth. this area size was acceptable since the “baby and kid’s wear” product category has a variety of product sizes that change from month to month for babies and from age to age for kids. if the customers ranked this fourth, the area size seems appropriate. the analysis helps retailers increase their income per square foot and appropriately evaluate the good qualities and shortcomings in their retail strategy. note that, as the selected customers may vary, so may the outcomes of the ahp. in this paper, the customers were randomly chosen, and it was shown that the preferences for both store management and the customers can be achieved by the ahp multi-criteria decision making methodology. store managers may choose the customers who shop the most to use the ahp model. ijahp article: yalcin, ozturk/determining the area sizes of each product category in a department store using multi-criteria decision making methodologies international journal of the analytic hierarchy process 20 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.602 in future research, ahp and anp models with fuzzy set theory could be constructed, and the outcomes from crisp numbers and fuzzy numbers could be compared and discussed. the differences and/or similarities will be demonstrated. ijahp article: yalcin, ozturk/determining the area sizes of each product category in a department store using multi-criteria decision making methodologies international journal of the analytic hierarchy process 21 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.602 references akalin, m., turhan, g. & sahin, a. (2012). the application of ahp approach for evaluating location selection elements for retail store: a case of clothing store. international journal of research in business and social science, 2(4), 1-20. doi: https://doi.org/10.20525/ijrbs.v2i4.77 bahng,y. & kincade, d.h. (2014). retail buyer segmentation based on the use of assortment decision factors. journal of retailing and consumer services, 21(4), 643-652. doi: https://doi.org/10.1016/j.jretconser.2013.12.004 ballester, n., guthrie, b., martens, s., mowrey, c., parikh, p. j. & zhang, x. (2014). effect of retail layout on traffic density and travel distance. proceedings of the 2014 industrial and systems engineering research conference. behera, m. p., & mishra, v. (2017). impact of store location and layout on consumer purchase behavior in organized retail. anvesha, 10(1), 10-21. botsali, a. & peters, b. a. (2005). a network based layout design model for retail stores. proceedings of the 2005 industrial engineering research conference, 1-6. cheng, l., li, b., cheng, q., baldwin, a.n. & shang, y. (2017). investigations of indoor air quality of large department store buildings in china based on field measurements. building and environment, 118,128-143. doi: https://doi.org/10.1016/j.buildenv.2017.03.031 chunk, m. & park, h.c. (2012). building energy demand patterns for department stores in korea. applied energy, 90,241-249. doi: https://doi.org/10.1016/j.apenergy.2011.05.008 cil, i. (2012). consumption universes based supermarket layout through association rule mining and multidimensional scaling. expert systems with applications, 39(10), 86118625. doi: https://doi.org/10.1016/j.eswa.2012.01.192 doucet, m.j. (2003). the department store shuffle: a study of rationalization and location change on a large metropolitan market. progress in planning, 60, 93-110. doi: https://doi.org/10.1016/s0305-9006(02)00093-4 duncan, r.b. (1972). characteristics of organizational environments and perceived environmental uncertainty. administrative science quarterly, 17, 313–327. doi: https://doi.org/10.2307/2392145 eckert, a., he, z. & west, d. s. (2015). an empirical analysis of tenant location patterns near department stores in planed regional shopping centers. journal of retailing and consumer services, 22, 61-70. doi: https://doi.org/10.1016/j.jretconser.2014.09.007 ijahp article: yalcin, ozturk/determining the area sizes of each product category in a department store using multi-criteria decision making methodologies international journal of the analytic hierarchy process 22 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.602 eroglu, e. (2013). factors affecting consumer preferences for retail ındustry and retailer selection using analytic hierarchy process. kafkas university journal of economics and administrative sciences faculty, 4(6), 43-57. flemming, p. (1989). strategic planning: fact sheet. ontario: ministry of agriculture and food, rural organizations and services branch. garcia, m.m., aragones, b., & carmen, c.m. (2008). an ahp-based evaluation procedure for innovative educational projects: a face to face vs. computer-mediated case study. omega, 36(5),754-765. gardner, n.j, huh, j. & chung, l. (2002). lessons from the sampoong department store collapse. cement & concrete composites, 24, 523-529. doi: https://doi.org/10.1016/j.omega.2006.01.005 girginer, n., kamisli ozturk, z., ozturk, g. & ozkul, a.e. (2007). using anp for courseware development platform selection. 21th aaou annual conference, kuala lumpur, malaysia. hjelmgren, d. (2016). creating a compelling brand meaning by orchestrating stories: the case of scandinavia’s largest department store. journal of retailing and consumer services, 32, 210-217. doi: https://doi.org/10.1016/j.jretconser.2016.06.015 kannan, v.r. & tan, k.c. (2006). buyer-supplier relationships: the impact of supplier selection and buyer-supplier engagement on relationship and firm performance. international journal of physical distribution and logistics management, 36,755–775. doi: https://doi.org/10.1108/09600030610714580 kernsom, t. & sahachaisaeree, n. (2010). determining of design elements and compositional setting of window display on the corporate strategic merchandising of large scale department store: a case of central world department store. procedia social and behavioral sciences, 5,1351-1356. doi: https://doi.org/10.1016/j.sbspro.2010.07.286 kernsom, t. & sahachaisaeree, n. (2012). strategic merchandising and effective composition design of windows display: a case of large scale department store in bangkok. procedia social and behavioral sciences, 42,422-428. doi: https://doi.org/10.1016/j.sbspro.2012.04.206 kumar, v., anand, a. & song, h. (2017). future of retailer profitability: an organizing framework. journal of retailing, 93(1), 96-119. doi: https://doi.org/10.1016/j.jretai.2016.11.003 lai, v.s., wong, b.k., & cheung, w. (2002). group decision making in a multiple criteria environment: a case using the ahp in software selection. european journal of operational research, 137, 134–144. doi: https://doi.org/10.1016/s0377-2217(01)000844 ijahp article: yalcin, ozturk/determining the area sizes of each product category in a department store using multi-criteria decision making methodologies international journal of the analytic hierarchy process 23 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.602 meade, l.m., & sarkis, j., (1999). analyzing organizational project alternatives for agile manufacturing processes: an analytical network approach. international journal of production research, 37(2), 241–261. doi: https://doi.org/10.1080/002075499191751 mantrala, m.k., levy,m., kahn, b.e., fox, e.j., gaidarev, p., dankworth, b. & shah, d. (2009). why is assortment planning so difficult for retailers? a framework and research agenda. journal of retailing, 85,71–83. doi: https://doi.org/10.1016/j.jretai.2008.11.006 miller, d. (2006). strategic human resource management in department stores: an historical perspective. journal of retailing and consumer services, 13:99-109. doi: https://doi.org/10.1016/j.jretconser.2005.08.012 ozcan, t., & esnaf, s. (2013). a discrete constrained optimization using genetic algorithms for a bookstore layout. international journal of computational intelligence systems, 6(2), 61-278. doi: https://doi.org/10.1080/18756891.2013.768447 park, t.w. (2012). inspection of collapse cause of sampoong department store. forensic science international, 21,119-126. doi: https://doi.org/10.1016/j.forsciint.2011.10.039 ratanatamskul, c. & siritiewsri, t. (2015). a compact on-site uasb-egsb system for organic and suspended solid digestion and biogas recovery from department store wastewater. international biodeterioration & biodegradation, 102, 24-30. doi: https://doi.org/10.1016/j.ibiod.2015.04.002 sagir, m. & ozturk, z. k. (2010). exam scheduling: mathematical modelling and parameter estimation with the analytic network process approach. mathematical and computer modelling, 52, 930-941. doi: https://doi.org/10.1016/j.mcm.2010.05.029 saaty, t.l. (1980). the analytic hierarchy process. new york: mcgraw-hill. saaty, t.l., (1996). decision making with dependence and feedback: the analytic network process. pittsburgh, pa, usa: rws publications. saaty, t.l. (2001). decision making with dependence and feedback the analytic network process. pittsburgh, pa, usa: rws publications. saaty, t. l. (2004a). decsion making-the analiytic hierarchy and network process (ahp/anp). journal of systems science and systems engineering, 13(2),129-157. saaty, t. l. (2004b) fundamentals of the analytic network process-dependence and feedback in decision-making with a single network. journal of systems science and systems engineering, 13(1),1-35. saaty, t.l. & özdemir, m.s. (2014). how many judges should there be in a group?. annals of data science, 1(3-4), 359-368. doi: https://doi.org/10.1007/s40745-014-0026-4 ijahp article: yalcin, ozturk/determining the area sizes of each product category in a department store using multi-criteria decision making methodologies international journal of the analytic hierarchy process 24 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.602 saaty, t.l. & vargas, l. g. (2006). decision making with the analytic network process: economic, political, social and technological applications with benefits, opportunities, costs and risks. springer science+business media, llc. sengupta, r. n., gupta a. & dutta, j. (2017). decision sciences: theory and practice. crc press, taylor & francis group. silver, e.a., pyke, d.f. & peterson, r. (1998). inventory management and production planning and scheduling. third ed. new york: johnwiley&sons. silva, d. r., davies, g. & naude, p. (2002). assessing the influence of retail buyer variables on the buying decision-making process. european journal of marketing, 36,1327–1343. doi: https://doi.org/10.1108/03090560210445209 su, x., chen, y., hipel, k. w. & kilgour, d. m. (2005). comparison of the analytical network process and the graph model for conflict resolution. journal of systems science and systems engineering, 14(3), 308-325. doi: https://doi.org/10.1007/s11518-006-01965 yapicioglu, h., & smith, a. e. (2012a). retail space design considering revenue and adjacencies using a racetrack aisle network. iie transactions, 44(6),446-458. doi: https://doi.org/10.1080/0740817x.2011.635177 yapicioglu, h., & smith, a. e. (2012b). a bi-objective model for the retail spatial design problem. engineering optimization, 44(3),243-266. doi: https://doi.org/10.1080/0305215x.2011.647812 wagner, j., etteson, r. & parrish, j. (1989). vendor selection among retail buyers: an analysis by merchandise division. journal of retailing, 65, 58–79. walsh, e.j & jeacle, i. (2003). the taming of the buyer: the retail inventory method and the early twentieth century department store. accounting, organizations and society, 28, 73-791. doi: https://doi.org/10.1016/s0361-3682(02)00032-6 wargocki, p, fanger, p.o., krupicz, p & szczecinski, a. (2004). sensory pollution loads in six office buildings and a department store. energy and buildings, 36(10), 995-1001. doi: https://doi.org/10.1016/j.enbuild.2004.06.006 ijahp article: yalcin, ozturk/determining the area sizes of each product category in a department store using multi-criteria decision making methodologies international journal of the analytic hierarchy process 25 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.602 appendix unweighted matrix of the anp model ijahp article: yalcin, ozturk/determining the area sizes of each product category in a department store using multi-criteria decision making methodologies international journal of the analytic hierarchy process 26 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.602 weighted matrix of the anp model ijahp article: yalcin, ozturk/determining the area sizes of each product category in a department store using multi-criteria decision making methodologies international journal of the analytic hierarchy process 27 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.602 limiting the matrix of the anp model ijahp article: damayanti, saputra, wijayanto, mudzakir, rudiyanto/management strategy of 2 nd and 3 rd class fish auction place in pati regency international journal of the analytic hierarchy process 1 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.960 management strategy of 2 nd and 3 rd class fish auction places in pati regency herna octivia damayanti department of aquatic resources, faculty of fisheries and marine science, diponegoro university octivia_oc@yahoo.co.id suradi wijaya saputra 1 department of aquatic resources, faculty of fisheries and marine science, diponegoro university suradiwsaputra@yahoo.co.id dian wijayanto department of aquatic resources, faculty of fisheries and marine science, diponegoro university dianwijayanto94@gmail.com abdul kohar mudzakir department of aquatic resources, faculty of fisheries and marine science, diponegoro university akohmud@gmail.com alin fithor rudiyanto department of aquatic resources, faculty of fisheries and marine science, diponegoro university afithor@gmail.com abstract fisheries management policies are needed to achieve the prosperity of society in indonesia. the auction process at fisheries in indonesia does not run optimally; therfore, the impact of the role and function of fish auctions is not yet the best it could be in some areas. this study aimed to formulate a strategy for managing 2 nd and 3 rd class fish auction places in pati regency. the research locations were the puncel, banyutowo, alasdowo, sambiroto, margomulyo, and pecangaan fish auctions in the pati regency. the research was conducted from january 2021 to november 2021. the research respondents, namely a sample of critical figures (key persons), were determined by purposive sampling of as many as 15 people. data was analyzed using the analytical hierarchy process and expert choice 11.0 software. the study results prioritize the most critical management strategies, namely fish production with a weight of 0.309 (30.9%), infrastructure with a weight of 0.295 (29.5%), institutions with a weight of 0.206 (20.6%), and human resources with a weight 0.190 (19.0%). the priority of alternative management strategies is as follows: (1) maintaining the existence of businesses run by fishermen, with a weight of 0.172 (17.2%), (2) improving facilities and infrastructure to support fish auction place operations, 1 corresponding author mailto:octivia_oc@yahoo.co.id mailto:suradiwsaputra@yahoo.co.id mailto:dianwijayanto94@gmail.com mailto:akohmud@gmail.com mailto:afithor@gmail.com ijahp article: damayanti, saputra, wijayanto, mudzakir, rudiyanto/management strategy of 2 nd and 3 rd class fish auction place in pati regency international journal of the analytic hierarchy process 2 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.960 with a weight of 0.164 (16.4%),and (3) increasing the number of catches landed, with a weight 0.145 (14.5%). keywords: analytical hierarchy process; fish auction place; strategy; management 1. introduction all fishermen can utilize marine fisheries because they are common property (a shared resource) (walden & mcguire, 2011). teh and sumaila (2011) stated that marine fishing efforts contribute to the global economy. the utilization of fisheries contributes significantly to food, nutrition, income, and traditions (okafor-yarwood, 2019). fish marketing requires a sound management system so that several objectives can be achieved, namely increasing income (oktafiani et al., 2019). in an effort to help fishermen sell their catch fish auction places have been built (nurlinda & saharruddin, 2013). pati regency regional government regulation no. 19 of 2009 states that the definition of a fish auction place (tpi) is a place provided explicitly by the regional government to conduct fish auctions. a fish auction invovles the public sale of fish utilizing increased bidding. hikmah and nasution (2017) also mention that fishermen land their caught fish at fish landing sites and then auction them at fish auction places. the auction mechanism has been applied to the fish market for a long time and has a long history (marszalec, 2018). kong et al. (2018) stated that one of the most creative ways to sell fish is by auction. fishermen use auctions to obtain the maximum price from buyers (fluvià et al., 2012; sogn-grundvåg et al., 2020). auction markets with many sellers and buyers are generally more competitive and able to obtain the maximum possible price (sogn-grundvåg et al., 2019; sogngrundvåg et al., 2020). the problem with fish auction places is that the auction process does not operate optimally or sometimes does not even run at all. the fish auction place cannot perform its functions properly because the fish landed through the fish auction place are only weighed there because the fishermen are conducting sales transactions directly with traders. transactions are carried out directly between fishermen and traders without an auction officer (febrianti et al., 2013). a fish auction place is one of the essential activities in a fishing port that requires optimum management because the activities that occur there affect the price of fish, which directly determines fishermens' income (lubis & pane, 2012). one of the potential areas for developing small fisheries is pati regency in indonesia. the capture fisheries in pati regency need attention due to a decline in production. in 2012, capture fisheries production was 47,576,648 kg and by 2018 it had decreased by 11.59% to 23,517,489 kg (statistics central bureau of pati regency, 2013 and 2019). this decrease is partly due to the less than optimal role of fish auction places in holding auctions. the buying and selling activities at 2 nd class fish auction places still do not running optimally. at the 3 rd class fish auction place, the auction process does not exist, rather the transactions are carried out directly between fishermen and traders without intermediaries this has resulted in a less than optimal role and function of the 2 nd class and 3 rd class fish auction places. damayanti (2018) mentions that one of the priority strategies for developing a small-scale/traditional fishery is the marketing trade system for fish catches through the auction process at a fish auction. ijahp article: damayanti, saputra, wijayanto, mudzakir, rudiyanto/management strategy of 2 nd and 3 rd class fish auction place in pati regency international journal of the analytic hierarchy process 3 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.960 based on this background, it is necessary to improve the management of the 2 nd and 3 rd class fish auction place in pati regency. this study aimed to formulate a management strategy for 2 nd class and 3 rd class fish auction places in pati regency. 2. background 2.1. small fishermen retnowati (2011) states that small fishermen come from traditional fishermen;however, they utilize modernization program/boat motorization and fishing gear and no longer rely solely on traditional boats and conventional fishing gear. they also use diesel or motors whose range of coverage is extended or far. traditional fishermen work to catch fish using simple (traditional) boats and fishing gear. the reach of the restricted area is only six nautical miles from the shoreline; usually, these fishermen fish to make ends meet. small fishermen are generally the poorest in the community (the poorest of the poor). the economy relies on the weather for social and routine activities (shaffril et al., 2017). small fishermen have a very large dependence on their livelihoods even though the income earned is uncertain (garcía-lorenzo et al., 2019; surísregueiro and santiago, 2014). 2.2. fish auction place pati regency regional government regulation no. 19 of 2009 states that the definition of a fish auction place (tpi) is a place provided explicitly by the regional government to conduct fish auctions. efforts to help market the fisherman's catch, then build a fish auction place (nurlinda & saharruddin, 2013) will indirectly reduce the function of mediators so that fishermens' losses are reduced, which will then affect the level of income and welfare of the fishermen. the function of a fish auction place (dianto et al., 2015) is as follows:  means of coaching fishing communities on the habit of saving and insurance.  assist in correctly collecting data on the volume of fish, types of fish, and prices.  provision of materials and equipment needed by fishermen and credit payments.  service center for fishing communities with business and business improvement.  source of income for local governments the fish auction place cannot function properly because the fish circulating through the fish auction place are only weighed there. then, the fishermen and traders carry out the sales transaction directly without an auction officer (febrianti et al., 2013). kistanto (2019) stated that one of the reasons that the transactions were carried out directly with the mediators/traders was the small number of catches. 2.3. fish auction peterson and georgianna (1988) highlight that auctions are divided into the following three types: (1) closed bid auctions, where the price is the highest bid and the highest bidder is designated as the buyer, (2) auctions in the uk, buyers actively bid in ascending order of price, and (3) auctions in the netherlands, the auctioneer starts at a very high price and decreases until the first bid, which is the winning bid. the uk type of auction includes about 75% of the world's auctions. ijahp article: damayanti, saputra, wijayanto, mudzakir, rudiyanto/management strategy of 2 nd and 3 rd class fish auction place in pati regency international journal of the analytic hierarchy process 4 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.960 fishermen use auctions to obtain the maximum price from buyers (fluvià et al., 2012; sogngrundvåg et al., 2020). auction markets with multiple sellers and buyers are generally more competitive than individual sales (sogn-grundvåg et al., 2020; sogn-grundvåg et al., 2019). guillotreau and jiménez-toribio (2011) mention the following two ways to increase and expand the fish market/tpi; first, by attracting new entrants to the local trade, and second, by allowing long-distance bidders to participate. 2.4. fish auction place classification a fish auction place is one of the facilities available at the fish landing base (khussurur, 2014). the fish landing base (ppi) is needed to support fishery activities in the management and utilization of fish resources, pre-production activities, production, processing, fish marketing, and supervision of fish resources (solihin et al., 2016). the department of fisheries and marine affairs of central java province (2005) divided fish auction places into classes based on the production value (raman). first class has a production value of >50 billion, 2 nd class has a production value of 25-50 billion, and 3 rd class has a production value of 10-25 billion. 2.5. strategy strategy or "strategos or strategia" comes from the greek word which means "general or generalship" or is defined as something related to management in an organization (suci, 2015). taufiqurokhman (2016) defines strategy as a coordinated commitment and action to exploit these competencies and competitive advantages. the strategy has meaning as a plan, jump (ploy), pattern, taking a position, and perception. a strategy has the following characteristics: (1) is a long-term plan, (2) is general and applicable, (3) is comprehensive, involves all parts of the organization, (4) integrates, cannot see the views of the whole or organization, (5) external, considering the external environment of the company or organization, both stakeholders and the macro environment, (6) can adapt to the environment. research on strategic management, especially for 2 nd and 3 rd class fish auction places using the criteria of human resources, fish production, infrastructure, and institutions has never been carried out, especially in pati regency. febrianto et al. (2015) conducted previous research on strategies for improving fishing port facilities, and sabana et al. (2016) examined the development of fish auction place strategies from service performance. mahyudin (2016) examined the management of fishing ports in terms of facilities and activities. lubis and pane (2017) performed research on functional fish auction places from an institutional aspect, port management, fishermen's socio-culture, and biotechnical ports. alfons et al. (2020) studied fishing port management strategies regarding facilities, human resources, and regulations. 3. methodology the selection of research sites focused on the boat berth and the fish auction place where small fishermen obtain their catch, namely 2 nd and 3 rd class fish auction places in pati regency. the research locations were puncel fish auction, banyutowo fish auction, alasdowo fish auction, sambiroto fish auction, margomulyo fish auction, and pecangaan fish auction. the research was conducted from january 2021 to november 2021. as susilowati and kirana (2008) applied, purposive sampling was used to obtain research respondents in the field which consisted of 15 key persons including academics, business people, ijahp article: damayanti, saputra, wijayanto, mudzakir, rudiyanto/management strategy of 2 nd and 3 rd class fish auction place in pati regency international journal of the analytic hierarchy process 5 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.960 government officials, and members of society. the academic is an expert in capture fisheries. there were six fish traders/mediators representing each fish auction place. the two government officials were representatives of the local government (technical department); in this case, the department of marine affairs and fisheries of pati regency. the community was represented by five members of the community and 1 ngo. ijahp article: damayanti, saputra, wijayanto, mudzakir, rudiyanto/management strategy of 2 nd and 3 rd class fish auction place in pati regency international journal of the analytic hierarchy process 6 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.960 table 1 research respondents for formulation of 2 nd and 3 rd class fish auction place management strategy respondent rule stakeholder represent respondent 1 key person in an interview expertin the field of capture fisheries academia respondents 2, 3, 4, 5, 6, 7 key person in an interview fish traders business respondents 8, 9 key person in an interview department of marine affairs and fisheries of pati regency local government respondents 10, 11, 12, 13, 14 key person in an interview community around the fish auction place community respondent 15 key person in an interview ngo community in-depth interviews were conducted with 15 key people as mentioned in table 1. the research questions were in the form of interest comparison questions for each research aspect to determine the priority criteria and research questions in the form of comparative questions to determine the alternative priorities of each criterion. an example of a question is the comparison of interest between human resources and fish production, with a level of 1 to 9. this key person only plays a role in answering research questions to build the ahp model.the results of in-depth interviews with key figures determined aspects of policy formulation for policy recommendations. primary data were collected by observation and documentation. research activity was documented in photos as authentic evidence of research implementation. saaty (2008) stated that the results of in-depth interviews can be placed into a problem decomposition or problem hierarchy. a key principle in a hierarchical arrangement is to possess detailed knowledge. the human mind organizes complex realities into their main elements, and then into parts again, and so on hierarchically (tiered). the author conducted the research interviews. performance ratings and attribute weights in the ahp were generated from a series of comparative assessments at the same hierarchical level, and assigned a number from a discrete scale of 1 to 9 (bui et al., 2017; brandao et al., 2021). table 2 discrete scale of pairwise comparison weights scale degree of importance 1 equal importance 3 moderate importance 5 high importance 7 very high importance 9 extreme importance 2,4,6,8 intermediate values source : (saaty & shang, 2011; brandao et al., 2021) ijahp article: damayanti, saputra, wijayanto, mudzakir, rudiyanto/management strategy of 2 nd and 3 rd class fish auction place in pati regency international journal of the analytic hierarchy process 7 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.960 the steps of the ahp analysis were as follows: step 1: study the current situation in the 2 nd and 3 rd class fish auction environment. the current conditions were used to select alternative strategies for managing the 2 nd and 3 rd class fish auctions in pati regency. step 2: create a hierarchy of criteria and selected alternatives that are considered to implement ahp. the hierarchical structure of the problem can be seen in figure 1. figure 1 problem hierarchy framework by data analysis (2021) key: a1: quality of fish auction place officers a2: quantity of fish auction place officers a3: existence of businesses run by fishermen a4: catch landed at the fish auction place a5: fish auction place operational support facilities and infrastructure a6: access to fish auction place either by land or sea a7: participation of fish auction place officers/existence of fish auction place a8: participation/existence of the community, traders/intermediaries, and local government step 3: describe the criteria and alternatives chosen to develop the problem hierarchy. the management aspects of 2 nd and 3 rd class fish auctions place in pati regency are the ahp criteria formulated based on the management model's indicators (widayati, 2008; dianto et al., 2015; solihin et al. al., 2016). ijahp article: damayanti, saputra, wijayanto, mudzakir, rudiyanto/management strategy of 2 nd and 3 rd class fish auction place in pati regency international journal of the analytic hierarchy process 8 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.960 table 3 description of criteria in the problem hierarchy criteria description human resources aspect the capacity of a second and third person in pati regency. the human resources aspect consists of 2 alternative criteria, namely, the quality of fish auction place officers and the quantity of fish auction place officers fish production aspect the number of fish managed and the involvement of fishermen in these activities in 2 nd and 3 rd class fish auctions places in pati regency. the aspect of fish resources consists of 2 alternative criteria, namely, the business run by fishermen and the catches that are distributed at the fish auction place infrastructure aspect the availability of facilities and infrastructure.this aspect consists of 2 alternatives, namely, facilities and infrastructure supporting fish auction place operations, as well as fish auction place access both by land and sea institutions aspect the role of institutions interested in the existence of fish auctions places. the institutional aspect consists of 2 alternatives, namely, the participation of fish auction place officers and the participation/existence of the community, traders, and local governments. table 4 description of alternatives in the hierarchy problem alternatives description criteria reference quality of fish auction place officers fulfillment of job responsibilities, the quality of work results human resources yuniar (2016) suherman et al. (2020) quantity of fish auction place officers description of the fish auction place personnel, the level of productivity achieved, the quantity of work, and effectiveness at work human resources pramitasari et al. (2006) yuniar (2016) suherman et al. (2020) existence of businesses run by fishermen the number of fishermen and traders fish production pramitasari et al. (2006) widayati (2008) catch landed at the fish auction place the number of ships fish production pramitasari et al. (2006) widayati (2008) fish auction place operational support facilities and infrastructure the auction floor area, the number of baskets, and the number of scales infrastructure pramitasari et al. (2006) widayati (2008) access to fish auction places either by land/sea the length of the pier infrastructure pramitasari et al. (2006) widayati (2008) participation of fish auction place officers/the existence of fish auction place weighing activities, auction continuity, retribution, and fish auction place institutions institutions perda kabupaten pati no. 22/2009 dianto et al. (2015) participation/existence of the community, the traders/intermediaries, and the local government description of the fishermen’s group, trader’s group, implementation of regulations, and supervision of activities at the fish auction place institutions dianto et al. (2015) idajati et al. (2016) solihin et al. (2016) raodah (2017) ijahp article: damayanti, saputra, wijayanto, mudzakir, rudiyanto/management strategy of 2 nd and 3 rd class fish auction place in pati regency international journal of the analytic hierarchy process 9 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.960 table 5 contribution of criteria and alternatives to the management strategy of the 2 nd and 3 rd class fish auction places in pati regency criteria alternatives contribution to the management strategy of the 2 ndv and 3 rd class fish auctions human resources quality of fish auction place officers human resources (fish auction place officers) can improve the performance of fish auction place management by carrying out their duties according to the instructions and procedures that are given. quantity of fish auction place officers human resources (quantity or number of fish auction place officers) can speed up the implementation of auctions at the fish auction place. fish production existence of businesses run by fishermen fish production (existence of fishing business, number of fishermen and traders) determines the running of management activities at the fish auction place. catch landed at the fish auction place. fish production (landed catch) is the main commodity in the auction process. infrastructure fish auction place operational support facilities and infrastructure infrastructure (availability of fish auction place facilities and infrastructure) can guarantee and be a supporting factor for the ongoing auction process access to fish auction places either by land or sea infrastructure (access to a fish auction place either by land or by sea) is a supporting factor and can attract fishermen and traders to make transactions at the fish auction place institution participation of fish auction place officers/the existence of fish auction place institution of fish auction place officers who play a role and actively participate in implementing activities at fish auction places can support the smooth management of a fish auction place. participation/existence of the community, the traders/intermediaries, and the local government the community, traders, and local governments must actively participate in monitoring the continuity of activities at the fish auction place step 4: after compiling the hierarchy problem, the next step is to give weighting by doing pairwise comparisons on each element. next, evaluate the level of relevance and compare pairs between criteria and alternatives, following a weight scale consisting of three people. the first party is the diponegoro university fishery academics. the second party is a researcher in public policy, which the government of pati regency owns. the third party is the fish auction section of the department of marine affairs and fisheries of pati regency. we explained the concepts that were applied in tables 3 and 4 and emphasized evaluating the importance of the criteria and alternatives for managing 2 nd and 3 rd class fish auction places in pati regency. the weighting results were then processed using the ahp and expert choice 11.0 as a tool to help determine strategic priorities. marsono (2020) explained that the expected consistency must be close to ijahp article: damayanti, saputra, wijayanto, mudzakir, rudiyanto/management strategy of 2 nd and 3 rd class fish auction place in pati regency international journal of the analytic hierarchy process 10 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.960 perfect in order to produce a decision that is valid. the consistency ratio should be less than or equal to 10%. the data assessment must be corrected if the value is more than 10% or 0.1. furthermore, through discussion, the author and the three parties describe the activities that can be applied as policy implications for each of the resulting alternative strategies. the policy implications create a formula to make it easier to execute the resulting alternative strategies so that policymakers, namely the pati regency government, can help them be carried out through related agencies. figure 2 research analysis process (brandao et al., 2021; periaiah et al.,2021) table 6 interpretation of the relationship between criteria and alternatives criteria alternatives scope human resources fish production infrastructure institution human resources quality of fish auction place officers improve the performance of fish auction place management knowledge of fisheries management and business ability to utilize facilities/ infrastructure according to their function carry out activities according to duties and functions quantity of fish auction place officers speed up the implementation of auctions at the fish auction place improve services at the fish auction place increase utilization of facilities and infrastructure carry out activities according to duties and functions fish production existence of businesses run by fishermen knowledge about fisheries business management determine the running of management activities in the ensure the smooth running of the fishery business ensure the existence and conveying of aspirations to ijahp article: damayanti, saputra, wijayanto, mudzakir, rudiyanto/management strategy of 2 nd and 3 rd class fish auction place in pati regency international journal of the analytic hierarchy process 11 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.960 criteria alternatives scope human resources fish production infrastructure institution fish auction place the government catch landed at the fish auction place knowledge about fisheries business management main commodity in the auction process ensure the continuity of auction activities at the fish auction place ensure the performance of fish auction place for management institutions infrastructure fish auction place operational support facilities/ infrastructure availability of facilities to manage fish auction place ensure the continuity of auction activities at the fish auction place guarantee and be a supporting factor for the ongoing auction process ensure the performance of fish auction place for management institutions access to fish auction places either by land or sea smooth access in and out of the fish auction place organize auction activities at the fish auction place supporting factor and can attract fishermen and traders to make transactions at the fish auction place support the smooth management of the fish auction place institution participation of fish auction place officers/the existence of fish auction place professionalism of officers in fish auction placemanageme nt auction organizer at the fish auction place supporting factors for organizers to conduct auctions at the fish auction place support the smooth management of fish auction place participation/exi stence of the community, the traders/intermed iaries, and the local government supervision following the knowledge possessed supervision of the auction process at the fish auction place supervision of the availability of the fish auction place facilities and infrastructure monitor the continuity of activities at the fish auction place 4. results and discussions puncel fish auction place and banyutowo fish auction place are 2 nd class fish auctions and hold regular auctions, while alasdowo fish auction place, sambiroto fish auction place, margomulyo fish auction place, and pecangaan fish auction place are 3 rd class and do not hold auctions. buying and selling transactions occur at 3 rd class fish auction places directly between traders and fishermen without intermediaries. ships that dock at 2 nd class fish auction places are danish seine ships under 30 gt, and ships arriving at 3 rd class fish auction places are gill net ships below 10 ijahp article: damayanti, saputra, wijayanto, mudzakir, rudiyanto/management strategy of 2 nd and 3 rd class fish auction place in pati regency international journal of the analytic hierarchy process 12 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.960 gt, with an average of 10 ships per day. one problem faced at the 2 nd class fish auction places is the inadequate floor area which becomes overloaded when the number of fish is abundant (peak season).the main problems of the 3 rd class fish auction places are the inadequate number of personnel and minimal facilities. 4.1. management strategy of 2 nd class and 3 rd class fish auction place in pati regency the results of the ahp consideration of the criteria for the 2 nd and 3 rd class auction places are shown in figure 3. the inconsistency ratio value of 0.01< 0.1 means the analysis results are acceptable. the fish production aspect has the highest priority in management of 2 nd and 3 rd class fish auction places, with a weight of 0.309 (30.9%). fish is the primary commodity and the principal object available at the fish auction. without the availability of fish, there would be no transactions since one of the main functions of the fish auction place is to organize the marketing and auctioning of fish caught by fishermen (dianto et al., 2015). the second priority is the infrastructure aspect, with a weight of 0.295 (29.5%). the fish auction place infrastructure or facilities play a role in supporting the smooth running of activities or transactions at the fish auction place. inadequate facilities, infrastructure, and access disrupt organizing auction activities at the fish auction place. siburian et al. (2018) stated that the existing infrastructure or facilities and equipment of the fish auction place is critical to support all auction activities. the third priority is the institutional aspect, with a weight of 0.206 (20.6%). institutional aspects at fish auction places or structure functions regulate the roles and functions of managment personnel so that there is no overlap in the implementation of activities. solihin et al. (2016) stated that the fish auction place institution creates price certainty and stability due to the auction process. if there is an institutional failure by a fish auction place, it will result in fishermens' poverty due to market price injustice and a failure of fishery management due to the weakness of the fish landing data. the fourth priority is the aspect of human resources, with a weight of 0.190 (19.0%). the fish auction place management of human resources becomes the fourth priority because findings in the field show that auctioneers at fish auction places expect management personnel to be qualified people. in several fish auction places, especially 3 rd class fish auction places, personnel have a minor role in transactions carried out by fishermen and traders. if fish auction place personnel played a greater role in activities at the fish auction place, satisfaction with services would increase; therefore, increasing the opportunity to maintain the loyalty of bidders and attract new bidders (dewi & kholil, 2019). an auction process that is carried out professionally and transparently will guarantee that the auction participants get a fair result which then fosters trust in the auction organizer (fish auction place officers) and will lead to the auction participants (fishermen and traders) carrying out their buying and selling transactions through auctions at the fish auction place. figure 3 priority criteria for 2 nd and 3 rd class fish auction places in pati regency ijahp article: damayanti, saputra, wijayanto, mudzakir, rudiyanto/management strategy of 2 nd and 3 rd class fish auction place in pati regency international journal of the analytic hierarchy process 13 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.960 the overall priority scale analysis results of the alternative criteria for managing 2 nd and 3 rd class fish auction places are shown in figure 4. the resulting alternative priorities indicate that the existence of businesses run by fishermen is the priority with the greatest weight of 0.172 (17.2%). the catch landed in the fish auction place becomes the second priority with a weight of 0.164 (16.4%). the fish auction place operational support facilities and infrastructure is the third priority with a weight of 0.145 (14.5%). if viewed in more detail, the alternative priorities of these criteria are following the priority criteria that place the human resources and infrastructure aspects as the first and second priorities. the inconsistency ratio value of 0.01< 0.1 means that the analysis results are acceptable. figure 4 alternative priority criteria for 2 nd and 3 rd class fish auction places in pati regency 4.2. management strategy of 2 nd and 3 rd class fish auction placein pati regency based on the human resources aspect the management of 2 nd and 3 rd class fish auction places based on the hr aspect includes improving the quality and increasing the quantity of fish auction place officers. the results of the alternative analysis of the hr aspect criteria can be seen in figure 5. the inconsistency ratio value of 0.00 < 0.1 means the analysis is acceptable. alternative criteria a1, namely improving the quality of fish auction place officers, received a weight of 0.611 (61.1%); while a2, which is the addition of the quantity of fish auction place officers has a weight of 0.389 (38.9%). the quality of officers is prioritized over the quantity of fish auction place officers. based on the results of observations and in-depth interviews, we obtained information that the participants of activities in fish auction places, namely fishermen and traders, prefer the presence of qualified officers compared to a large number of officers. if fish auction place officers are qualified and understand their duties, transactions will be faster and easier. however, if the number of fish auction place officers is sufficient, it will help smooth transactions at fish auction places because it avoids overlapping work. bayyinah et al. (2016) mention that fish auction place officers who can carry out their duties well will affect the quality of service at fish auction places, therefore having an impact on the level of satisfaction of fishermen who use the services. thus, qualified fish auction place officers should be able to carry out their duties properly so that services at fish auction places run well. bayyinah et al. (2016) also mention that if the number of officers is adequate, they will be able to handle transaction services at fish auction places. ijahp article: damayanti, saputra, wijayanto, mudzakir, rudiyanto/management strategy of 2 nd and 3 rd class fish auction place in pati regency international journal of the analytic hierarchy process 14 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.960 if the two alternatives in the human resources aspect are able to be fulfilled, then it can be followed up by making standard operational procedures (sops) at the fish auction place. the sops' impact on the services provided will follow the promised time. services can be carried out quickly and following procedures so that transactions that occur at fish auction places run well and smoothly. figure 5 alternative priority criteria for 2 nd and 3 rd class fish auction places in pati regency based on thr hr aspect the current condition of the 2 nd class fish auction place in pati regency as seen from the hr aspect shows that auction activities and fish auction place officers carry out their functions and duties well. however, the 2 nd class fish auction place in pati regency has not been able to accommodate all the fishermens' catches, so it often exceeds capacity. the current condition gave rise to the idea of adding more auction activities. additional fish auction places will be required if additional auction activities, namely auction officers, weighing, and recording notes are included. the management strategy of 2 nd class fish auction place in pati regency based on the recommended hr aspect is shown in table 5. ijahp article: damayanti, saputra, wijayanto, mudzakir, rudiyanto/management strategy of 2 nd and 3 rd class fish auction place in pati regency international journal of the analytic hierarchy process 15 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.960 table5 management strategy of 2 nd class fish auction place in pati regency based on the hr aspect alternative strategies policy implications actors improve the quality of fish auction place officers increase the quantity of fish auction place officers the type of training adapts to each task, for example, auction procedures recruitment/acceptance of new competent personnel through open and transparent selection department of marine affairs and fisheries of pati regency department of marine affairs and fisheries of pati regency the current condition of the 3 rd class fish auction places in pati regency from the hr aspect shows that no auction activities are occurring, the fish auction places are not operating and fish auction place officers only make retribution withdrawals. fish buying and selling transactions are carried out without an auction process directly by fishermen and traders without involving fish auction place officers. in addition, transactions are conducted outside the fish auction place area. the current conditions gave rise to to the strategy of enforcing the established rules that all fish buying and selling transactions must be carried out at the fish auction place. in addition, it is necessary for the fish auction place manager to start pioneering the implementation of auctions, especially during the peak fishing season. fish auctions can reduce the occurrence of direct transactions between fishermen and traders/middlemen because all fish buying and selling activities will be carried out through an auction process and are held directly by the fish auction place. in this scenario, the traders/middlemen only function as buyers through the auction process at the fish auction place.the management strategy of the 3 rd class fish auction place in pati regency based on the recommended hr aspectis shown in table 6. table 6 management strategy of 3 rd class fish auction place in pati regency based on the hr aspect alternative strategies policy implications actors improve the quality of fish auction place officers increase the quantity of fish auction place officiers training on auction procedures recruitment/acceptance of new competent personnel to fill positions at the fish auction place institutional/organizational structure make sops for auction procedures and non-auction transactions within the fish auction place department of marine affairs and fisheries of pati regency department of marine affairs and fisheries of pati regency department of marine affairs and fisheries of pati regency, fish auction placeofficer ijahp article: damayanti, saputra, wijayanto, mudzakir, rudiyanto/management strategy of 2 nd and 3 rd class fish auction place in pati regency international journal of the analytic hierarchy process 16 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.960 4.3. management strategy of 2 nd and 3 rd c lass fish auction place in pati regency based on the fish production aspect the management of 2 nd and 3 rd class fish auction places in pati regency based on the fish production aspect includes maintaining the existence of businesses run by fishermen and increasing the number of catches landed at fish auction places. the results of the alternative analysis of the fish production aspect criteria can be seen in figure 6. the inconsistency ratio value of 0.00 < 0.1 means the analysis is acceptable. figure 6 alternative priority criteria for 2 nd and 3 rd class fish auction places in pati regency based on the fish production aspect alternative criteria a3, which maintains the existence of fishermen-run businesses has a weight of 0.542 (54,2%), while a4, which increases the number of catches landed at the fish auction place has a weight of 0.458 (45.8%). maintaining the existence of fishermen-run businesses has a higher priority than increasing the catch that lands at the fish auction place. currently, fish auction place management prioritizes that the fishermens’ catch landed at the fish auction place will continue to run smoothly; however, there must be a balance in efforts to increase the number of landed catches with the provision of infrastructure for mooring ships, which is a greater investment of time and resources. damayanti (2017) states that the fishing business of fishermen is closely related to their income. furthermore, damayanti (2018) states that one way to maintain the existence of fishing businesses of small/traditional fishermen is to increase the productivity of these businesses by using modern fishing gear. an increase in the number of catches landed at the fish auction place is closely related to the number of ships that landed at the fish auction place pier, which is in accord with nurhayati and atika (2018) who state that the number of fish landed at the fish auction place increases with the increase in the number of ship visits. in contrast, the number of fish landed at the fish auction decreases when ship visits decrease. the current condition of 2 nd class fish auction places seen from the fish production aspect shows that fishermen land their catch at the fish auction place, and auctions are carried out there every day. however, this has not been the same with the inadequate number of traders in pati regency based on the recommended fish production aspect presented in table 7. ijahp article: damayanti, saputra, wijayanto, mudzakir, rudiyanto/management strategy of 2 nd and 3 rd class fish auction place in pati regency international journal of the analytic hierarchy process 17 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.960 table 7 management strategy of 2 nd class fish auction place in pati regency based on the fish production aspect alternative strategies policy implications actors maintain the existence of businesses run by fishermen increase the number of catches landed in the fish auction place efforts to bring new traders to the fish auction place maintain the continuity of auctions at fish auction places regularly and consistently by increasing the number of ships that land at fish auction places by increasing the length of the pier department of marine affairs and fisheries of pati regencydepartment of marine affairs and fisheries of pati regency, fish auction place officer, fisherman department of marine affairs and fisheries of pati regency, department ofpublic works and spatial planningof pati regency the current condition of the 3 rd class fish auction places as seen from the fish production aspect shows that the fish auction place is not operating and transactions occur directly between fishermen and traders. the 3 rd class fish auction place in pati regency does not hold auctions, and fishermen operating in the vicinity of the fish auction place have a contract with traders that causes them to sell their catch to the trader at a price that the trader has determined. the current conditions must receive attention and intervention from the local government, especially the implementation of transactions between fishermen and traders and determination of the selling price of fish in pati regency based on the recommended fish production aspect shown in table 8. ijahp article: damayanti, saputra, wijayanto, mudzakir, rudiyanto/management strategy of 2 nd and 3 rd class fish auction place in pati regency international journal of the analytic hierarchy process 18 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.960 table 8 management strategy of 3 rd class fish auction place in pati regency based on the fish production aspect alternative strategies policy implications actors maintain the existence of businesses run by fishermen increase the number of catches landed in the fish auction place efforts to maintain the stability of the selling price of fish by making price agreements with traders or determining the lowest selling price for fish promote and empower the village unit cooperative (kud) of fishermen, specifically for savings and loans (capitalization) implementation of strict regulations and sanctions so that ships land and unload at fish auction place docks pati regency local government, department of marine affairs and fisheries of pati regency, traders, fishermen department of marine affairs and fisheries of pati regency; department of cooperatives, micro, small and medium enterprises of pati regency pati regency local government, department of marine affairs and fisheries of pati regency, fish auction place officers 4.4. management strategy of 2 nd and 3 rd class fish auction places in pati regency based on the infrastructure aspect the management of 2 nd and 3 rd class fish auction places based on the infrastructure includes increasing the facilities and infrastructure supporting fish auction place operations and increasing access to fish auction places either by land or sea. the alternative analysis of the infrastructure aspect criteria results can be seen in figure 6. the inconsistency ratio value of 0.00 < 0.1 means the analysis is acceptable. ijahp article: damayanti, saputra, wijayanto, mudzakir, rudiyanto/management strategy of 2 nd and 3 rd class fish auction place in pati regency international journal of the analytic hierarchy process 19 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.960 figure 7 alternative priority criteria for 2 nd and 3 rd class fish auction places in pati regency based on the infrastructure aspect alternative criteria a5, namely improving the facilities and infrastructure supporting fish auction place operations, received a weight of 0.579 (57.9%); while a6, namely increasing access to the fish auction place both by land and sea, weighs 0.421 (42.1%). the supporting facilities and infrastructure for fish auction place operations need to be improved because they are insufficient, especially for 3 rd class fish auction places. in 2 nd class fish auction places, facilities and infrastructure need to added because they are not sufficient, causing auction process to be divided into several sessions which hinders the smooth running of the auction and risks the fish starting to rot. for 3 rd class fish auction places, facilities and infrastructure are still minimal and unavailable because the fish auction place is not operating. febrianto et al. (2015) stated that one of the strategies for improving the performance and governance of fisheries development is to pay attention to the facilities around the fish auction place. in addition to improving facilities and infrastructure, the access to fish auction places either by land or sea needs attention. access from land appears to be quite good. the obstacle that fishermen often face is access from the sea because the pier area is often shallow. the occurrence of siltation certainly hampers the process of docking ships, making it difficult for fish to land in the fish auction place. siry (2011) states that the silting of the port-channel makes it difficult for ships to get in and out and increases operational and maintenance costs. the current condition of the 2 nd class fish auction place as seen from the infrastructure aspect shows that the facilities and infrastructure are already available but not adequate, especially when the number of fish is abundant. facilities that require improvement are the length of the pier, the area of the auction floor, the number of baskets, and the number of scales in pati regency based on the recommendations of the infrastructure aspectas shown in table 9. ijahp article: damayanti, saputra, wijayanto, mudzakir, rudiyanto/management strategy of 2 nd and 3 rd class fish auction place in pati regency international journal of the analytic hierarchy process 20 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.960 table 9 management strategy of 2 nd class fish auction placein pati regency based on the infrastructure aspect alternative strategies policy implications actors improve fish auction place supporting facilities and infrastructure increase access to fish auction places both by land and sea efforts to increase the length of the dock efforts to expand the auction floor efforts to add facilities and infrastructure to support auction activities pier groove dredging repair and maintenance of roads to the fish auction place department of marine affairs and fisheries of pati regency department of marine affairs and fisheries of pati regency the current condition of 3 rd class fish auction places as seen from the infrastructure aspect shows that facilities and infrastructure are still minimal and unavailable. the length of the pier is not sufficient, the auction floor is not spacious and requires repairs, and baskets and scales that are not yet available require special attention because the facilities and infrastructure are very minimal. therefore, it will be able to operate properly again if the fish auction place is equipped with the infrastructure aspects recommended in table 10. table 10 management strategy of 3 rd class fish auction places in pati regency based on the infrastructure aspect alternative strategies policy implications actors improve fish auction place supporting facilities and infrastructure increase access to fish auction places both by land and sea efforts to increase the length of the dock and expand the auction floor efforts to add facilities and infrastructure to support auction activities such as basket and scales dredging of grooves and around moorings repair and maintenance of roads to the fish auction place department of marine affairs and fisheries of pati regency department of public works and spatial planningof pati regency 4.5. management strategy of 2 nd and 3 rd class fish auction places in pati regency based on the institutional aspect the management of 2 nd and 3 rd class fish auctions places based on the institutional aspect includes increasing the participation of fish auction place officials/fish auction place existence and ijahp article: damayanti, saputra, wijayanto, mudzakir, rudiyanto/management strategy of 2 nd and 3 rd class fish auction place in pati regency international journal of the analytic hierarchy process 21 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.960 increasing the participation/existence of the community, traders, and local government. the results of the alternative analysis of institutional aspect criteria can be seen in figure 7. the inconsistency ratio value of 0.00 < 0.1 means the analysis is acceptable. figure 8 alternative priority criteria for 2 nd and 3 rd class fish auction places in pati regency based on the institutional aspect alternative criteria a7, namely increasing the participation of fish auction place officers/existence of fish auction places, received a weight of 0.494 (49.4%). in contrast, a8, which is increasing the participation/existence of the community, the traders/intermediaries, and local governments received a weight of 0.506 (50.6%). the participation of the community, traders, and local governments is prioritized, especially as supervisors of activities at fish auction places. based on observations in the field, auction activities at 2 nd class fish auction places can run well because the community and village government supervise auction activities. the village government issues a rule that fish auctions must be at the fish auction place. this is different from 3 rd class fish auction places, where the community and village government tend to be passive so that the buying and selling of fish occurs without supervision. rianto (2017) stated that the village government plays an essential role in conveying fish auction places. in addition, people who obey the regulations affect the sustainability of auction activities at fish auction places. the role of fish auction place officers is to be the organizer of auction activities. auction activities at fish auction places are entirely the responsibility and primary task of fish auction place officers. maulidya et al. (2014) stated that the work effectiveness of fish auction place employees/officers dramatically influences the performance of fish auction places. the effectiveness of the work of fish auction place officers is significant because it affects the trust of users of fish auction place services, namely fishermen and traders. the current condition of 2 nd class fish auction place as seen from the institutional aspect shows that the fish auction place and fishermens’ institutions are running well based on the regular holding of auctions. the institution that does not yet exist is a trader institution. the existence of a trader institution is essential as a connector for the aspirations of the traders; it would be easier if there was an institution that housed the traders rather than to having to deal with the traders directly. the recommended institutional aspect for the 2 nd class fish auction place in table 11. ijahp article: damayanti, saputra, wijayanto, mudzakir, rudiyanto/management strategy of 2 nd and 3 rd class fish auction place in pati regency international journal of the analytic hierarchy process 22 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.960 table 11 management strategy of 2 nd class fish auction place in pati regency based on the institutional aspect alternative strategies policy implications actors increase the participation of fish auction place officers/fish auction place existence increasing the participation/existence of the community, traders, and local government. efforts to improve the performance of fish auction place officers, especially in auctions, reporting, and documentation of activities at the fish auction place efforts to increase the participation of the community, traders, and local governments in supervising activities at fish auctions place through establishing a supervisor forum. department of marine affairs and fisheries of pati regency, fish auction place officer department of marine affairs and fisheries of pati regency, fishers, traders, community, village government the current condition of the 3 rd class fish auction places as seen from the institutional aspect shows that the institution is not functioning because the fish auction places are not operating. some fishermen are not proactive in fish auction activities, but there is no trader group. thus, local governments need to start pioneering and revitalizing activities at fish auction places in pati regency based on the recommended institutional aspects presented in table 12. table 12 management strategy of 3 rd class fish auction place in pati regency based on the institutional aspect alternative strategies policy implications actors increase the participation of fish auction place officers/fish auction place existence increase the participation/existence of the community, traders, and local government. efforts to hold auctions at fish auction places during peak season intervention of fish auction place officers in the sale and purchase of fishers and traders efforts to establish a supervisor forum department of marine affairs and fisheries of pati regency, fish auction place officer department of marine affairs and fisheries of pati regency, fishers, traders, community, village government ijahp article: damayanti, saputra, wijayanto, mudzakir, rudiyanto/management strategy of 2 nd and 3 rd class fish auction place in pati regency international journal of the analytic hierarchy process 23 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.960 5. conclusion this study concludes that the management strategy of 2 nd and 3 rd class fish auction places in pati regency consists of the following four aspects: human resources, fish production, infrastructure, and institutions. the order of priority for the management strategies of 2 nd and 3 rd class fish auction places in pati regency from highest to lowest are fish production, infrastructure, institutions, and human resources. the priority of alternative management strategies for 2 nd and 3 rd class fish auction places in pati regency from highest to lowest is (1) maintaining the existence of businesses run by fishermen. exisitng fishing businesses can sustain ongoing transactions and auctions at fish auction places. (2) improving facilities and infrastructure to support fish auction place operations. fish auction place facilities and infrastructure support the ongoing auction process at the fish auction place. (3) increasing the number of catches landed at fish auction places. ensure the continuity of auction activities because fish are the main commodity in the fish auction. (4) increasing access to fish auction places by land and sea. access to a fish auction place is a supporting factor and can attract fishermen and baskets to make transactions at a fish auction place. (5) increasing the participation of the community, traders/intermediaries, and local governments. communities, traders/middlemen, and local governments supervise the continuity of activities at fish auction places. (6) increasing the participation of fish auction place officers. the role of fish auction place officers is a supporting factor for holding auctions at fish auction places. (7) improving the quality of fish auction place officers and qualified fish auction place officers who can carry out activities according to their duties and functions. (8) increasing the number of officers. a sufficient number of fish auction place officers can improve services at fish auction places. 6. policy recommendations policy recommendations include government efforts through activities or programs based on the researched and reviewed priority aspects, namely fish production, infrastructure, institutions, and human resources. the aspects of fish production include supporting fishermen's buying and selling activities through professional and transparent auctions at the fish auction place. the infrastructure aspect includes sufficient facilities and infrastructure used to manage and implement auctions at the fish auction place. the institutional aspects include increasing the role and function of fish auction place institutions, fishermen, traders, and the community in implementing and supervising the fish auction place management. the aspects of human resources include improving the quality and quantity of fish auction place officers to work professionally according to their respective functions. ijahp article: damayanti, saputra, wijayanto, mudzakir, rudiyanto/management strategy of 2 nd and 3 rd class fish auction place in pati regency international journal of the analytic hierarchy process 24 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.960 references alfons, j. m., tupamahu, a. and abrahamz, j. (2020). evaluasi peran dan strategi pengelolaan pangkalan pendaratan ikan erie di kota ambon. jurnal triton,14(2), 66–75. bayyinah, a. al, solihin, i. and wisudo, s. h. (2016). kepuasan nelayan terhadap pelayanan pelabuhan perikanan nusantara (ppn) kejawanan cirebon. marine fisheries,7(1), 33–43.doi: https://doi.org/10.29244/jmf.7.1.33-43 . brandao, l., naas, i. d. a. and neto, p. l. de o. c. o. (2021). sustainable development of gem production: an analysis using the analytic hierarchy process. international journal of the analytic hierarchy process, 13(3), 483–509. doi: https://doi.org/10.13033/ijahp.v13i3.880 . bui, n. t., kawamura, a., kim, k. w., prathumratana, l., kim, t. h., yoon, s. h., jang, m., amaguchi, h., bui, d. d. and truong, n. t. (2017). proposal of an indicator-based sustainability assessment framework for the mining sector of apec economies. resources policy, 52, 405– 417. doi: https://doi.org/10.1016/j.resourpol.2017.05.005. damayanti, h. o. (2017). struktur usaha penangkapan ikan oleh nelayan tradisional di desa pecangaan kecamatan batangan kabupaten pati. jurnal litbang:media informasi penelitian, pengembangan dan iptek, xiii(2), 80–92. doi: https://doi.org/10.33658/jl.v13i2.96. damayanti, h. o. (2018). strategi pengembangan usaha penangkapan ikan tradisional : studi kasus di desa pecangaan, kecamatan batangan, kabupaten pati. jurnal sosial ekonomi kelautan perikanan, 8(1), 13–26. doi: http://dx.doi.org/10.15578/jksekp.v8i1.6873. dewi, d. a. n. n. and kholil, h. m. (2019). tingkat kepuasan peserta lelang terhadap pelayanan tempat pelelangan ikan (tpi) morodemak. akuatik jurnal sumberdaya perairan, 13(2), 122–130. doi: https://doi.org/10.33019/akuatik.v13i2.1123. dianto, m. w., harahab, n. and ismadi (2015). evaluasi kinerja tempat pelelangan ikan (tpi) dalam menunjang kesejahteraan nelayan di popoh, desa besole, kecamatan besuki, kabupaten tulungagung, jawa timur. ecsofim, 3(1), 1–14. doi: https://doi.org/10.21776/ub.ecsofim.2016.003.01.01. department of fisheries and marine affairs of central java province. (2005). determination of fish auction place class based on production value (raman). department of fisheries and marine affairs of central java province, semarang. febrianti, s. s., boesono, h. andhapsari, t. d. (2013). analisis faktor-faktor yang mempengaruhi harga ikan manyung (arius thalassinus) di tpi bajomulyo juwana pati. journal of fisheries resources utilization management and technology, 2(3), 162–171. febrianto, r. k., mudzakir, a. k. dan rosyid, a. (2015). peningkatan kapasitas fasilitas fungsional melalui program minapolitan di pelabuhan perikanan nusantara (ppn) pekalongan, jawa tengah. jurnal kebijakan sosial ekonomi kelautan dan perikanan, 5(2), 89–100. doi: http://dx.doi.org/10.15578/jksekp.v5i2.1020. https://doi.org/10.29244/jmf.7.1.33-43 https://doi.org/10.13033/ijahp.v13i3.880 https://doi.org/10.1016/j.resourpol.2017.05.005 https://doi.org/10.33658/jl.v13i2.96 http://dx.doi.org/10.15578/jksekp.v8i1.6873 http://dx.doi.org/10.33019/akuatik.v13i2.1123 https://doi.org/10.21776/ub.ecsofim.2016.003.01.01 http://dx.doi.org/10.15578/jksekp.v5i2.1020 ijahp article: damayanti, saputra, wijayanto, mudzakir, rudiyanto/management strategy of 2 nd and 3 rd class fish auction place in pati regency international journal of the analytic hierarchy process 25 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.960 fluvià, m., garriga, a., rigall-i-torrent, r., rodríguez-carámbula, e. and saló,a. (2012). buyer and seller behavior in fish markets organized as dutch auctions: evidence from a wholesale fish market in southern europe. fisheries research, 127–128(2012), 18–25. doi: https://doi.org/10.1016/j.fishres.2012.04.010. garcía-lorenzo, i., cabaleiro-casal, m. j. and varela-lafuente, m. m. (2019). fishermen’s associations of the small-scale fisheries: study applied to the participation in the cofradías of galicia (nw spain). ocean and coastal management, 178, 1–11. doi: https://doi.org/10.1016/j.ocecoaman.2019.104841. guillotreau, p. and jiménez-toribio, r. (2011). the price effect of expanding fish auction markets. journal of economic behavior and organization, 79(3), 211–225. doi: https://doi.org/10.1016/j.jebo.2011.01.031. hikmah and nasution, z. (2017). upaya perlindungan nelayan terhadap keberlanjutan usaha perikanan tangkap. jurnal sosek kementerian kelautan dan perikanan, 7(2), 127–142. doi: http://dx.doi.org/10.15578/jksekp.v7i2.6464. hosnan, a. (2016). peran tempat pelelangan ikan (tpi) dan persepsi nelayan serta saluran pemasaran hasil tangkap ikan di tpi pondok mimbo kabupaten situbondo.agricultural economics study program,agricultural faculty jember university, jember. idajati, h., pamungkas, a. and kukinul s., v. (2016). the level of participation in mangrove ecotourism development, wonorejo surabaya.procedia social and behavioral sciences227(november 2015), 515–520. doi: https://doi.org/10.1016/j.sbspro.2016.06.109. khussurur, f. (2014). pengembangan tempat pelelangan ikan dan prasarana perikanan samudera di brondong kabupaten lamongan: tema analogi biological. universitas islam negeri maulana malik ibrahim, malang. kistanto, n. h. (2019). sistem sosial budaya masyarakat pesisir nelayan dan bakul ikan di kampung tambak lorok, kota semarang. sabda, 14(1), 67–81. doi: https://doi.org/10.14710/sabda.14.1.67-81. kong, x., huang, george q., luo, h. and yen, b. (2018). physical-internet-enabled auction logistics in perishable supply chain trading: state-of-the-art and research opportunities. industrial management & data systems, 118(8), 1671–1694. doi: https://doi.org/10.1108/imds-10-20170486. lubis, e. and pane, a. b. (2012). an optimum model of fish auction in indonesian fishing ports following the characteristics of fisherman. journal of coastal development, 15(3), 282–296. lubis, e. and pane, a. b. (2017). institutional model of fish auction refunctionalization in indonesia fishing ports. aquaculture, aquarium, conservation & legislation international journal of the bioflux society, 10(6), 1456–1465. mahyudin. (2016). optimalisasi pemanfaatan dan pengelolaan pangkalan pendaratan ikan (ppi) klaligi kota sorong propinsi papua barat. program pascasarjana, universitas terbuka, jakarta. http://dx.doi.org/10.1016/j.fishres.2012.04.010 http://dx.doi.org/10.15578/jksekp.v7i2.6464 https://doi.org/10.14710/sabda.14.1.67-81 http://dx.doi.org/10.1108/imds-10-2017-0486 http://dx.doi.org/10.1108/imds-10-2017-0486 ijahp article: damayanti, saputra, wijayanto, mudzakir, rudiyanto/management strategy of 2 nd and 3 rd class fish auction place in pati regency international journal of the analytic hierarchy process 26 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.960 marsono. (2020).penggunaan metode analytical hierarchy process (ahp) dalam penelitian. in media, bogor. marszalec, d. (2018). auctions for quota: lives for the future primer and perspect. fisheries research, 203(july), 84–92. maulidya, r. n., riniwati, h. and harahap, n. (2014). efektivitas pegawai tempat pelelangan ikan (tpi) dalam menunjang keberdayaan di tpi pelabuhan perikanan nusantara (ppn) prigi desa tasikmadu, kabupaten trenggalek, propinsi jawa timur. ecsofim, 2(1), 1–10. nurhayati, d. and atika, d. (2018). analisis kinerja operasional pelabuhan perikanan pantai (ppp) eretan, indramayu. jurnal sains perikanan dan kelautan, 1(2), 33–45. doi: https://doi.org/10.47685/barakuda45.v1i1.18 . nurlinda and saharruddin. (2013). peranan tpi (tempat pelelangan ikan) ponrang dalam meningkatkan pendapatan masyarakat setempat (studi kasus di desa tirowali kecamatan ponrang). jurnal equilibrium, 3(2), 249–254. doi: http://dx.doi.org/10.35906/je001.v3i2.86. okafor-yarwood, i. (2019). unreported and unregulated fishing, and the complexities of the sustainable development goals (sdgs) for countries in the gulf of guinea. marine policy, 99(may 2017), 414–422. doi: https://doi.org/10.1016/j.marpol.2017.09.016. oktafiani, a. h., nurhayati, a., rizal, a. and nurruhwati, i. (2019). manesar village unit cooperative role to improve fishermen welfare in pangandaran regency, west java province. global scientific journals, 7(6), 944–960. pati local goverment. (2009). peraturan daerah kabupaten pati nomor 19 tahun 2009 tentang tempat pelelangan ikan. pati local government, pati. pati local goverment. (2009). peraturan daerah kabupaten pati nomor 22 tahun 2009 tentang retribusi tempat pelelangan ikan. pati local government, pati. periaiah, n., islam, r. and abdullah, m. f. (2021). environmental impact assessment for malaysian bauxite mining industry. international journal of the analytic hierarchy process, 13(1), 1–26. doi: https://doi.org/10.13033/ijahp.v13i1.851 . peterson, s. and georgianna, d. (1988). new bedford fish auction : a study in auction method and market power. human organization, 47(3), 235–241. pramitasari, s. d., anggoro, s. and susilowati, i. (2006). analisis efisiensi tpi (tempat pelelangan ikan) kelas 1,2 dan 3 di jawa tengah dan pengembangannya untuk peningkatan kesejahteraan nelayan. jurnal pasir laut, 1(2), 12–21. raodah. (2017). peran pappalele pada masyarakat nelayan di desa tamasaju kabupaten takalar. walasuji, 8(2), 369–380. doi: https://doi.org/10.36869/wjsb.v8i2.127. retnowati, e. (2011). nelayan indonesia dalam pusaran kemiskinan struktural (perspektif sosial, ekonomi dan hukum). perspektif, xvi(3), 149–159. doi: http://dx.doi.org/10.30742/perspektif.v16i3.79. https://doi.org/10.47685/barakuda45.v1i1.18 http://dx.doi.org/10.35906/je001.v3i2.86 https://doi.org/10.1016/j.marpol.2017.09.016 https://doi.org/10.13033/ijahp.v13i1.851 https://doi.org/10.36869/wjsb.v8i2.127 http://dx.doi.org/10.30742/perspektif.v16i3.79 ijahp article: damayanti, saputra, wijayanto, mudzakir, rudiyanto/management strategy of 2 nd and 3 rd class fish auction place in pati regency international journal of the analytic hierarchy process 27 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.960 rianto. (2017). peran kepala desa dalam pengelolaan tempat pelelangan ikan milik desa di desa penaso kecamatan pinggir kabupaten bengkalis tahun 2015. jom fisip, 4(1), 1–12. saaty, t. l. (2008). decision making with the analytic hierarchy process.international journal of services sciences, 1(1), 83–98. saaty, t. l. and shang, j. s. (2011). an innovative order of magnitude approach to multicriteria decision making based on ahp: prioritizing divergent intangible humanitarian acts. european journal of operational research, 214, 703–715. sabana, c., madusari, b. d. and pratikwo, s. (2016). kajian strategi pengembangan tempat pelelangan ikan (tpi) kota pekalongan. jurnal litbang kota pekalongan, 11,117–131. doi: https://doi.org/10.54911/litbang.v11i0.11. shaffril, h. a. m., samah, a. a. and d’silva, j. l. (2017). adapting towards climate change impacts: strategies for small-scale fishermen in malaysia. marine policy, 81, 196–201. doi: http://doi.org/10.1016/j.marpol.2017.03.032. siburian, a. a., syaifuddin and rengi, p. (2018). studi pemanfaatan fasilitas tempat pelelangan ikan di pelabuhan perikanan nusantara pekalongan provinsi jawa tengah. jom bidang perikanan dan ilmu kelautan, 5(1), 1–11. siry, h. y. (2011). studi sebaran sedimen dasar dan pendangkalan di pelabuhan minyak (oil wharves) pt caltex pacific indonesia dumai, riau pasca pengerukan 1990. widyariset, 14(3), 643–650. sogn-grundvåg, g., zhang, d. and dreyer, b. (2020). competition in a fish auction : the case of atlantic cod in northern norway. fisheries research, 235(december 2020), 1–10. doi: https://doi.org/10.1016/j.fishres.2020.105826. sogn-grundvåg, g., zhang, d. and iversen, a. (2019). large buyers at afish auction : the case of the norwegian pelagic auction. marine policy, 104(february 2018), 232–238. doi: https://doi.org/10.1016/j.marpol.2018.06.011. solihin, a., alamin, m. a. and isdahartatie. (2016). penguatan kelembagaan tpi dalam mewujudkan perikanan berkelanjutan dan berkeadilan.risalah kebijakan pertanian dan lingkungan, 3(3), 205–215. statistics central bureau of pati regency (2013). pati in numbers 2012. statistics central bureau of pati regency, pati. statistics central bureau of pati regency (2020). pati in numbers 2019. statistics central bureau of pati regency, pati. suci, r. p. (2015).esensi manajemen strategi.sidoarjo. zifatama publisher. suherman, a., kurohman, f. and jayanto, b. b. (2020).kinerja operasional pelabuhan perikanan nusantara (ppn) palabuhanratu, sukabumi, jawa barat. jurnal perikanan dan kelautan, 10(1), https://doi.org/10.1016/j.fishres.2020.105826 https://doi.org/10.1016/j.marpol.2018.06.011 ijahp article: damayanti, saputra, wijayanto, mudzakir, rudiyanto/management strategy of 2 nd and 3 rd class fish auction place in pati regency international journal of the analytic hierarchy process 28 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.960 87–101. surís-regueiro, j. c. and santiago, j. l. (2014). characterization of fisheries dependence in galicia (spain). marine policy, 47, 99–109. doi: http://doi.org/10.1016/j.marpol.2014.02.006. susilowati, i. and kirana, m. (2008). pemberdayaan masyarakat pada usaha mikro kecil di sektor perikanan. badan penerbit universitas diponegoro. semarang. taufiqurokhman. (2016).manajemen strategik. fakultas ilmu sosial dan ilmu politik, universitas prof. dr. moestopo beragama, jakarta. teh, l. c. l. and sumaila, u. r. (2011). contribution of marine fisheries to worldwide employment, fish and fisheries, 14(1), 77-88. doi: https://doi.org/10.1111/j.14672979.2011.00450.x. walden, j. b., and mcguire, c. j. (2011). who owns the fish? moving from the commons to the federal ownership of our national fisheries.marine resources committee newsletter, 14(2), 3–7. widayati, t. (2008). analisis efisiensi teknis tempat pelelangan ikan dan tingkat keberdayaan pengelola tempat pelelangan ikan serta strategi pemberdayaannya di wilayah pantai utara jawa tengah.postgraduate program, diponegoro university, semarang. yuniar, c. w. (2016). pengaruh faktor keadilan kompensasi, kebutuhan untuk berprestasi, kebutuhan untuk berafiliasi, dan kebutuhan untuk berkuasa terhadap kinerja pegawai negeri sipil pada unit pelaksana teknis kementerian sosial di d.i. yogyakarta. universitas muhammadiyah yogyakarta, yogyakarta. ijahp article: go, promentilla, aviso, yu/an ahp-based composite index for sector prioritization international journal of the analytic hierarchy process 42 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.638 an ahp-based composite index for sector prioritization daryn joy go school of economics de la salle university daryn_go@dlsu.edu.ph michael angelo promentilla chemical engineering department de la salle university michael.promentilla@dlsu.edu.ph kathleen aviso chemical engineering department de la salle university kathleen.aviso@dlsu.edu.ph krista danielle yu  school of economics de la salle university krista.yu@dlsu.edu.ph abstract economic sectors are highly interdependent, allowing them to promote sustainable development and inclusive growth by generating positive spillover effects from small investments in the economy. however, this interdependent nature can also generate negative spillover effects that lead to widespread inoperability and unemployment. while interdependence and the problem of scarce resources have led to the development of multiple sector prioritization tools, none of these tools have been able to wholly measure sector significance based on its multiple dimensions. hence, this paper develops a composite sector prioritization index that identifies the key sectors based on five criteria of sector significance: degree of influence, structural significance, degree of interconnectedness, dependence on domestic economy, and contribution to risk of inoperability. the index is constructed using the analytical hierarchy process, which shows the economy’s priorities and primary concerns in order to aid policymakers in  acknowledgements: this work was supported in part by the philippine commission on higher education via the phernet sustainability studies program and the philippine department of science and technology via the grants for outstanding achievements in science and technology program. the authors are grateful to the respondents for sharing their expertise in answering our survey.  corresponding author ijahp article: go, promentilla, aviso, yu/an ahp-based composite index for sector prioritization international journal of the analytic hierarchy process 43 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.638 investing in sectors that would generate the highest positive spillover effects to the economy. a case study from the philippines is considered and the results show that much of the economy’s resources must be allocated towards the manufacturing, trade, and private services sectors. keywords: input-output modelling, analytic hierarchy process, sector prioritization, sector interdependence 1. introduction sustainable development and inclusive growth have increasingly been the central goals of many world economies. the rising trend towards sustainability stems from the shift towards achieving the “economic, environmental, and social dimensions of development” without endangering future potential development (organisation for co-operation and development, 2011). inclusive growth, on the other hand, has been increasingly popular as nations have seen the importance of striving for economic prosperity that can be felt by its individual members. as one of the united nations’ sustainable development goals, sustainable and inclusive economic growth promotes economic prosperity as well as improvements in individual well-being, stable employment, and employment opportunities (organisation for co-operation and development, 2011). the priority given to these goals highlights the importance of economic sectors because of their contribution to inclusive growth, development, and economic activity. economic sectors serve as the cogs of the economic machinery, each possessing a unique role in the overall development of the economy while also working together to produce raw materials and intermediate outputs for other sectors and consumers to use. economic sectors play a large role in generating output for each country and are instrumental in providing a livelihood for its members. additionally, economic sectors greatly impact the overall state of the economy. because economies consist of sectors that are naturally interdependent, a negative or positive change in one sector can cascade and cause a corresponding negative or positive impact in other sectors, thus affecting the entire economy (dejuán, lenzen, & cadarso, 2017). this means that while sector interdependence can allow small investments in one sector to generate positive spillover effects such as an increase in production level, income, and employment opportunities, this relationship can lead to negative spillover effects as well. the interconnectedness of economic sectors can thus hamper the economy’s ability to achieve growth and development. the initial impact experienced by one sector from negative exogenous shocks, such as disasters, can cascade to other sectors that were previously unaffected by said shocks. this means that if one sector experiences a sudden loss in production level, the sectors that depend on it as consumers and producers will experience losses from this as well (resurreccion & santos, 2012a). the compounded impact then impedes the economy’s overall production, which in turn degrades individual welfare as negative shocks or production losses increase unemployment and poverty incidence. there is also the problem of scarce resources, which gives rise to the need for efficient resource allocation in order to minimize the potential losses and maximize the potential ijahp article: go, promentilla, aviso, yu/an ahp-based composite index for sector prioritization international journal of the analytic hierarchy process 44 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.638 gains of the economy. hence, countries must aim to understand the underlying structure and system of their respective economies in order to strategically allocate resources for each sector. this highlights the importance of understanding the specific roles and characteristics of each sector, as well as identifying the economy’s most crucial sectors. recently, there has been a growing interest in identifying key economic sectors using various measures developed from input-output (i-o) modelling. input-output modelling’s ability to account for the interrelationships among sectors makes it a powerful tool in studying the structure of the economy (hewings, 1982). the studies include the use of forward and backward linkages to measure each sector’s demandand supply-side dependence on other sectors for the bangladesh, indian, and japanese economies (alauddin, 1986, dhawan & saxena, 1992, zuhdi, 2017b). the power of dispersion index, which measures sector influence, and the sensitivity of dispersion index, which measures sector sensitivity to exogenous changes, were also used as a basis for sector ranks in the japanese economy and growth factor decomposition, which was used by ballester et al. (2013) to identify the key sector in the philippine economy from 1961 to 2006 (zuhdi, 2017a). however, these studies were only able to rank sector importance based on individual sector significance measures. hence, they were unable to wholly account for the various criteria that collectively contribute to a sector’s overall significance to the economy. there has also been a growing field of literature that develops sector prioritization tools for resource allocation decisions. these include tsekeris’s (2017) work, where he used network analysis measures such as degree centrality (sector interactions) and closeness centrality (economic distance) for the greek economy, and several studies that have been proposed to base the allocation of resources on a sector’s sensitivity to disruptions by developing risk-based sector prioritization techniques. these include barker and santos’ (2010) work which uses lian and haimes’ (2006) dynamic inoperability input-output model to identify the sectors that are highly sensitive to inventory delays, and resurreccion and santos’ (2012b) work which uses the dynamic cross-prioritization plot to determine key sectors based on their sensitivity to risk and their role in meeting the primary goals of the economy. while there have been numerous studies which aim to identify the key sectors in an economy, the previous literature lacks studies that focus on developing composite indices that capture various dimensions of sector importance. one of the key studies that developed a composite index is that of yu et al. (2014), who introduced the vulnerability index, a risk-based sector prioritization tool that identifies critical sectors based on three aspects that contribute to vulnerability: economic impact, sector size, and the propagation length of each sector. however, since this study was solely focused on vulnerability and post-disaster techniques, its index only ranked sectors in terms of measures that contribute to these issues. thus, there is still a need to develop a multi-dimensional composite index composed of various measures (or criteria) that collectively contribute to the overall significance of a sector to the economy. hence, this study fills the research gap by developing a composite multi-criterion index that measures overall sector significance. in addition, this study introduces the use of the ijahp article: go, promentilla, aviso, yu/an ahp-based composite index for sector prioritization international journal of the analytic hierarchy process 45 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.638 analytic hierarchy process as a weighting procedure for the different index components based on experts’ assessments rather than choosing weights arbitrarily. yu et al. (2014) state that a systematic approach to weighing each of the index components, such as saaty’s analytic hierarchy process (ahp), is needed to better mirror the economy’s preference assignments of the sector significance criteria. for these reasons, the construction of the composite index may benefit various stakeholders, especially policymakers, in identifying which sectors to prioritize in policymaking and resource allocation decisions. this methodology is widely used in a variety of fields and has been increasingly used in the development of various composite indices. this systematic approach to generating weights is widely used in a variety of studies including industrial engineering decision making, supplier selection problems, swot analysis, research and development project prioritization, human capital indicator rankings, and economic studies such as i-o modelling and forecasting the foreign exchange rates and the recovery of the united states’ economy (triantaphyllou & mann, 1995; nydick & hill, 1992; barbarosoglu & yazgac, 1997; liu & hai, 2005; kurttila, kangas, & kajanus, 2000; silva, belderrain & pantoja, 2010; abdullah, jaafar & che taib, 2013; saaty & vargas, 1979; saaty, 2010). the ahp has been increasingly used in the development of composite indices such as the environmental performance index (dedeke, 2013). it has been widely used by corporations and institutions in measuring the impact of their policy decisions on the environment, the cyclical economic performance indices (niemira, 2001). it has also been used as an alternative to traditional-nber 1 methods, the composite industry sustainability performance index that accounts for the different dimensions of sustainability such as its social and economic aspects, and the disaster-resilience index for local communities in the philippines (pandian, jawahar & nachiappan, 2013; orencio & fujii, 2013). overall, these studies illustrate the versatility and sophistication of the ahp framework used in the study. this study is set apart from the growing pool of related literature because it is the first or one of the first composite indices that uses ahp in measuring multi-criterion sector significance. given this, the study develops a composite multi-criterion index through ahp to identify the critical sectors to prioritize in an economy. the index ranges between 0 and 1, where higher values imply that the sector has a larger degree of contribution and vulnerability to risks. these higher value sectors must be given more priority in resource allocation decisions. the composite index analyzes the sector’s role and relative importance in an economy through five criteria of sector significance, namely the power of dispersion index, sector size, average propagation length, sectoral purchase coefficient, and inoperability multiplier. first, the power of dispersion index measures how a change in one sector’s final demand impacts other sectors. this measures the sector’s relative influence over other sectors in the economy. thus, sectors with higher power of dispersion indices must be given more priority (zuhdi, 2017b). second, the sector size measures the structural significance of a sector with respect to the entire economy, which in turn contributes to 1 national bureau of economic research ijahp article: go, promentilla, aviso, yu/an ahp-based composite index for sector prioritization international journal of the analytic hierarchy process 46 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.638 its overall significance (yu et al., 2014). third, the average propagation length measures a sector’s degree of interconnectedness as both a producer for and a consumer of other sectors. higher degrees of interconnectedness will indicate that a sector plays a relatively larger role in the overall economy (yu et al., 2014). fourth, the sectoral purchase coefficient measures a sector’s dependence on the domestic economy. sectors with higher sectoral purchase coefficients are given more importance as they are more dependent on the inputs of domestic sectors for their production. thus, these sectors are more vulnerable to domestic shocks that will impede the production of the sectors they are dependent on (resurreccion & santos, 2012a). this, in turn, will also contribute to a greater scale of economic loss and inoperability in the economy. last, the inoperability multiplier measures a sector’s contribution to the economy’s risk of inoperability. this contributes to sector significance because the additional exogenous shock experienced by the sector will lead to a higher degree of inoperability on the entire economy (yu et al., 2014). because of these features, the composite index is considered a useful tool to aid policymakers in identifying which sectors to prioritize given the economy’s varied priorities and preferences. the creation of this tool may benefit various stakeholders in their resource allocation decisions as it identifies the key economic sectors while accounting for a wider range of factors the economy may consider. since this tool grants policymakers a broader view of an economy’s various concerns, it allows them to determine which sectors are critically in need of investment and should be given the most priority. the remainder of the paper is organized as follows. section 2 presents the data used in the case study and the mathematical foundations of the sector significance criteria, which will serve as the components of the composite index. section 3 presents the formulation of the composite index. section 4 presents the case study where the index was used to identify the critical sectors in the philippine economy. lastly, section 5 summarizes the paper and elaborates on the recommendations for future research. 2. mathematical foundations the i-o tables are used to generate the five criteria of sector significance mentioned earlier: (1) the power of dispersion index, which measures each sector’s degree of influence, (2) the sector size, which measures each sector’s structural significance, (3) the average propagation length, which measures each sector’s number of inter-industry interactions, (4) the sectoral purchase coefficient, which measures each sector’s dependence on the domestic economy, and (5) the inoperability multiplier, which measures each sector’s contribution to the risk of inoperability caused by exogenous shocks. the mathematical foundations of these five criteria are demonstrated below. 2.1 basic leontief input-output model the basic i-o model was first established by leontief (1936) to illustrate the interdependent nature of economic sectors. this system of linear equations was developed to represent the interindustry transactions in the economy, where each sector produces homogeneous goods and services and uses a fixed amount of inputs in its production of outputs (miller & blair, 2009). this model is formally defined as: ijahp article: go, promentilla, aviso, yu/an ahp-based composite index for sector prioritization international journal of the analytic hierarchy process 47 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.638 𝐱 = 𝐀𝐱 + 𝐟, (1) where 𝐱 is the total output vector, 𝐀 is the technical coefficients matrix with each element 𝑎𝑖𝑗 representing the proportion of sector j‘s input requirement contributed by sector i, and 𝐟 is the final demand vector. alternatively, this can be expressed as: 𝐱 = (𝐈 − 𝐀)−𝟏𝐟, (2) where (𝐈 − 𝐀)−𝟏 is commonly known as the leontief inverse matrix. 2.2 power of dispersion index the power of dispersion index was established by rasmussenís (1957) as a measure for the nature of inter-sectoral relationships. this measure is widely used in studying economic linkages and sector roles. rasmussenís (1957 in drejer, 2002) defines the power of dispersion index as the impact of an increase in the final demand of one sector on the output of the rest of the economy. this powerful measure determines a sector’s degree of relative influence on the economy and is notably defined as: 𝑝𝑖 = ∑ (i−a)−1𝑖𝑗 𝑛 𝑖=1 1 𝑛 ∑ ∑ (i−a)−1𝑖𝑗 𝑛 𝑖=1 𝑛 𝑗=1 , (3) where ∑ (i − a)−1 𝑖𝑗 𝑛 𝑖=1 represents the jth column sum of the leontief inverse matrix and 1 𝑛 ∑ ∑ (i − a)−1 𝑖𝑗 𝑛 𝑖=1 𝑛 𝑗=1 is the average of the total column sums of the leontief inverse matrix. 2.3 sector size yu et al. (2014) define each sector’s size relative to the economy as: 𝑧𝑖 = 𝑥𝑖 ∑ 𝑥𝑖 𝑛 𝑖=1 , (4) where 𝑥𝑖 is sector i’s output level and 𝑧𝑖 is a value that ranges from 0 to 1. given this, 𝑧𝑖 is defined as the ratio of sector i’s output relative to the total output of the economy. a higher 𝑧𝑖 indicates that sector i is relatively more important to the economy. 2.4 average propagation length the average propagation length (apl) was derived from the i-o model to measure the economic distance between two sectors while accounting for the size of the linkages between them (dietzenbacher, romero & bosma, 2005). it was formally defined as “the average number of steps it takes to transmit a cost-push (demand-pull) from one sector to the other.” each element of the apl matrix is defined as follows: 𝑠𝑖𝑗 = { 𝑖𝑛𝑡(𝑣𝑖𝑗 ) if 𝑓𝑖𝑗 ≥ 𝑎 0 if 𝑓𝑖𝑗 < 𝑎 , (5) ijahp article: go, promentilla, aviso, yu/an ahp-based composite index for sector prioritization international journal of the analytic hierarchy process 48 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.638 where 𝑓𝑖𝑗 is the size of the linkage or the average of the cost-push (forward) and demandpull (backward) effect from sector i to j, 𝑎 is an arbitrary threshold level chosen to guarantee that only apls of sectors with sufficiently large linkages are retained, and 𝑣𝑖𝑗 is the average number of industry interactions it takes to pass on a cost-push or demandpull from sector i to j. 𝑣𝑖𝑗 is specifically defined as: 𝑣𝑖𝑗 = { ℎ𝑖𝑗 𝑔𝑖𝑗−𝛿𝑖𝑗 if 𝑔𝑖𝑗 − 𝛿𝑖𝑗 > 0 0 if 𝑔𝑖𝑗 − 𝛿𝑖𝑗 < 0 , (6) or: 𝑣𝑖𝑗 = { 𝑘𝑖𝑗 𝑙𝑖𝑗−𝛿𝑖𝑗 if 𝑙𝑖𝑗 − 𝛿𝑖𝑗 > 0 0 if 𝑙𝑖𝑗 − 𝛿𝑖𝑗 < 0 , (7) where ℎ𝑖𝑗 is an element from the matrix 𝐇 = 𝐆(𝐆 − 𝐈), 𝐆 is the ghosh inverse matrix – which measures the change in output as a result of a change in primary costs, 𝐈 is the identity matrix, 𝑔𝑖𝑗 is an element of the ghosh inverse matrix, 𝛿𝑖𝑗 is the kronecker delta which is equal to 1 if 𝑖 = 𝑗 and equal to 0 if 𝑖 ≠ 𝑗, 𝑘𝑖𝑗 is an element from the matrix 𝐊 = 𝐋(𝐋 − 𝐈) where 𝐋 is the leontief inverse matrix (𝐈 − 𝐀)−𝟏, and 𝑙𝑖𝑗 is an element of the leontief inverse matrix. by definition, equations 6 and 7 yield the same matrix. lastly, 𝑓𝑖𝑗 represents each element of the 𝐅 matrix, which is defined as: 𝐅 = 1 2[(𝐋−𝐈)+(𝐆−𝐈)] , (8) where 𝐋 − 𝐈 and 𝐆 − 𝐈 represent the demand-pull and cost-push effect, respectively. 2.5 sectoral purchase coefficient the sectoral purchase coefficient was adapted from okuyama and yu’s (2018) regional purchase coefficient. this stems from the following competitive i-o model: 𝐱 = 𝐀𝐱 + 𝐟𝐝 + 𝐞 − 𝐦, (9) where 𝐱 is the total output vector, 𝐀 is the technical coefficients matrix, 𝐟𝐝 is the final domestic demand vector, 𝐞 is the export vector and 𝐦 is the import vector. equation 9 can be transformed into the following non-competitive i-o model: 𝐱 = 𝐀𝐱 + 𝐟𝐝 + 𝐞 − 𝐌(𝐀𝐱 + 𝐟𝐝) (10) through the import coefficient 𝑀𝑖 expressed as: ijahp article: go, promentilla, aviso, yu/an ahp-based composite index for sector prioritization international journal of the analytic hierarchy process 49 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.638 𝑀𝑖 = 𝑚𝑖 ∑ 𝑎𝑖𝑗𝑥𝑗+𝑓𝑖 𝑑𝑛 𝑗=1 , (11) where 𝑚𝑖 measures sector i’s imports while 𝑀𝑖 measures sector i’s dependence on imports for its output production. alternatively, sector i’s dependence on domestic inputs (or the domestic economy) to produce its output can be measured through the following sectoral purchase coefficient: 𝐷𝑖 = 1 − 𝑀𝑖 . (12) 2.6 inoperability input-output model one key extension of leontief’s i-o model was that of disaster modelling initially developed by haimes and jiang (2001) called the physical inoperability input-output model (iim). this model was developed to measure each economic sector’s inoperability – defined as the “inability of a system to perform its intended function” – as a result of a perturbation or external shock. santos and haimes (2004) further improved on this model by integrating the leontief i-o model. this then led to the development of a demandbased iim that defines inoperability as the normalized production loss caused by perturbations. 2 the model is expressed as: 𝐪 = 𝐀∗𝐪 + 𝐜∗ (13) or: 𝐪 = (𝐈 − 𝐀∗)−𝟏𝐜∗, (14) where 𝐪 is the inoperability vector and each element 𝑞𝑖 is defined as the ratio of the output loss (the difference between the ideal output level and the degraded output level) over the ideal output level. a * is the interdependency matrix, where each element 𝑎𝑖𝑗 ∗ represents the inoperability sector i additionally contributes to sector j. c* is the perturbation vector defined as the reduction of final demand, normalized according to the ideal output level. each element 𝑐𝑖 ∗ represents the demand-side perturbation. lastly, the inoperability multiplier is defined as the column sums of (𝐈 − 𝐀∗)−𝟏. this measures the rate of increase in the level of inoperability caused by an additional perturbation. 3. composite sector prioritization index 3.1 normalization of the five sector significance measures to ensure that the composite index generated by this study will meet the standards of a good index as defined by eichhorn (1976), we adopt the following normalization process for each sector significance criteria, which will serve as the components of the index: �̈�𝑖 = 𝑒𝑖 ∑ 𝑒𝑖 𝑛 𝑖=1 , (20) 2 santos and haimes (2004) provides a more extensive discussion of the iim. ijahp article: go, promentilla, aviso, yu/an ahp-based composite index for sector prioritization international journal of the analytic hierarchy process 50 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.638 where �̈�𝑖 represents the element of the normalized criteria vector and 𝑒𝑖 represents the element of a specific criterion’s vector. since all five sector significance measures have positive values, �̈�𝑖 will range from 0 to 1. �̈�𝑖 values approaching one will indicate that sector i must be prioritized in resource allocation decisions. given this, we normalize the five criteria as follows: criterion 1: power of dispersion index applying the normalization process to the power of dispersion index (equation 3) generates the following: �̈�𝑖 = 𝑝𝑖 ∑ 𝑝𝑖 𝑛 𝑖=1 , (21) where �̈�𝑖 represents the relative influence of each sector on the rest of the economy, which also contributes to sector significance. a higher �̈�𝑖 indicates that a final demand increase experienced by the respective sector generates a relatively greater impact on the output of other sectors in the economy. hence, this implies that allocating more resources to these influential sectors will generate greater positive spillover effects on the rest of the economy. criterion 2: sector size given that sector size is already normalized, we re-express equation 4 as follows: �̈�𝑖 = 𝑥𝑖 ∑ 𝑥𝑖 𝑛 𝑖=1 , (22) as mentioned earlier, a higher �̈�𝑖 also indicates that sector i plays a larger role in the economy, specifically in terms of output production. hence, this sector must be given priority with resource allocation. criterion 3: average propagation length adopting the process of yu et al. (2014), the average propagation length component, which measures a sector’s number of inter-industry interactions, will be normalized as follows: �̈�𝑖 = ∑ 𝑠𝑗𝑖+∑ 𝑠𝑖𝑗−2𝑠𝑖𝑖 𝑛 𝑗=1 𝑛 𝑗=1 ∑ (∑ 𝑠𝑗𝑖+∑ 𝑠𝑖𝑗−2𝑠𝑖𝑖 𝑛 𝑗=1 𝑛 𝑗=1 ) 𝑛 𝑖=1 , (23) where ∑ 𝑠𝑗𝑖 𝑛 𝑗=1 represents the impact of a change in final demand of sector i or the backward apl, ∑ 𝑠𝑖𝑗 𝑛 𝑗=1 represents the impact of a change in the primary cost of sector i or the forward apl, and 2𝑠𝑖𝑖 represents the initial impact of a change in either the primary cost or final demand of sector i on itself. this is eliminated in order to ensure that we are only measuring how many sectors sector i interacts with once, whether as a producer or a consumer. ijahp article: go, promentilla, aviso, yu/an ahp-based composite index for sector prioritization international journal of the analytic hierarchy process 51 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.638 a higher �̈�𝑖 indicates that the sector has a greater degree of interconnectedness as both a producer and consumer in the economy. this means that the sector is a key player in the economy and injecting more funds towards this sector will increase the reach or scope of the positive spillover effect on the economy. criterion 4: sectoral purchase coefficient next, normalizing the sectoral purchase coefficient (equation 12) generates the following: �̈�𝑖 = 𝐷𝑖 ∑ 𝐷𝑖 𝑛 𝑖=1 , (24) where �̈�𝑖 is the ith element of the normalized sectoral purchase coefficient matrix. intuitively, a higher �̈�𝑖 indicates that sector i is relatively more dependent on domestic inputs for its output production. hence, exogenous shocks that hamper the production of domestic goods and services may severely impact sector i’s output levels. this indicates that these sectors must be prioritized and given more resources to address the sector’s vulnerability to the effects caused by shocks experienced by other sectors. criterion 5: inoperability multiplier we denote the inoperability multiplier generated from equation 14 as 𝑚𝑖 and apply the normalization process to generate the following: �̈�𝑖 = 𝑚𝑖 ∑ 𝑚𝑖 𝑛 𝑖=1 , (25) where �̈�𝑖 represents how much risk each sector contributes to the whole economy as a result of an exogenous shock. a higher �̈�𝑖 indicates that the sector contributes to a larger degree of widespread inoperability as a result of perturbations, thus contributing to the sector’s overall significance and prioritization level in the economy. 3.2 composite index through analytic hierarchy process 3.2.1 composite index given that each sector significance criterion has been formally defined, we now define the composite index as: 𝑅𝑖 ∗ = 𝑊�̈��̈�𝑖 + 𝑊�̈� �̈�𝑖 + 𝑊�̈� �̈�𝑖 + 𝑊�̈��̈�𝑖 + 𝑊�̈� �̈�𝑖, (26) where 0 ≤ 𝑊�̈�, 𝑊�̈� , 𝑊�̈� , 𝑊�̈� , 𝑊�̈� ≤ 1 and 𝑊�̈�+ 𝑊�̈� + 𝑊�̈� + 𝑊�̈� + 𝑊�̈� = 1. each weight 𝑊�̈�, 𝑊�̈� , 𝑊�̈� , 𝑊�̈� , 𝑊�̈� will be generated using ahp explained below. 𝑅𝑖 ∗ ranges between 0 and 1. a higher 𝑅𝑖 ∗ indicates that the sector plays a key role in the overall economy. ijahp article: go, promentilla, aviso, yu/an ahp-based composite index for sector prioritization international journal of the analytic hierarchy process 52 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.638 3.2.2 analytic hierarchy process the ahp was developed by saaty (1980) as a type of multi-criteria decision analysis (mcda) technique that involves gathering respondents’ pairwise comparisons among different components to determine the relative importance of each criterion. the following steps, defined by saaty (2008), were adopted in the study: step 1: identifying the problem or goal of the study. as mentioned earlier, the goal of the study is to identify the key sectors in the economy based on five criteria of sector significance: power of dispersion index (degree of influence), sector size (structural significance), average propagation length (degree of interconnectedness), sectoral purchase coefficient (dependence on domestic economy), and inoperability multiplier (contribution to risk of inoperability). the key sectors will be identified from the following list of alternatives: agriculture, fishery and forestry (agr), mining and quarrying (min), manufacturing (man), construction (cons), electricity, gas and water (egw), transportation, communication and storage (tcs), trade (trd), finance (fin), real estate and ownership of dwellings (real), private services (privsrv), and government services (govt). the study’s ahp framework is summarized in figure 1. figure 1 sector prioritization index framework step 2: construct the pairwise comparison matrix. given n number of criteria, the pairwise comparison matrix is defined as: 𝑶 = [𝑜𝑖𝑗 ] = ( 1 𝑜12 𝑜13 1 𝑜12⁄ 1 𝑜23 1 𝑜13⁄ 1 𝑜23⁄ 1 ⋯ 𝑜1𝑛 ⋯ 𝑜2𝑛 ⋯ 𝑜3𝑛 ⋮ ⋮ ⋮ 1 𝑜1𝑛⁄ 1 𝑜2𝑛⁄ 1 𝑜3𝑛⁄ ⋱ ⋮ ⋯ 1 ), (27) where 𝑜𝑖𝑗 is greater than zero and represents the average of the respondents’ preference for criterion i over criterion j for 𝑖, 𝑗 = 1, … , 𝑛. it is important to note that 𝑜𝑗𝑖 = 1 𝑜𝑖𝑗⁄ . the respondents’ pairwise comparison ratings range from 1 (equal preference for both alternatives goal criteria identify key sectors economic power of dispersion index (pdi) sector size (ss) average propagation length (apl) sectoral purchase coefficient (spc) inoperability multiplier (im) agr min man cons egw tcs trd fin real priv srv govt ijahp article: go, promentilla, aviso, yu/an ahp-based composite index for sector prioritization international journal of the analytic hierarchy process 53 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.638 measures) to 9 (extremely strong preference for one measure relative to another) as specified by saaty (1980). table 1 presents saaty’s scale of relative importance. table 1 saaty’s scale of relative importance rating preference 1 equal importance 3 moderate importance 5 strong importance 7 very strong importance 9 extreme importance 2,4,6,8 intermediate values since this study makes use of five criteria for the composite index, the matrix from equation 27 now becomes: 𝑶 = [𝑜𝑖𝑗 ] = ( 1 𝑜12 𝑜13 1 𝑜12⁄ 1 𝑜23 1 𝑜13⁄ 1 𝑜23⁄ 1 𝑜14 𝑜15 𝑜24 𝑜25 𝑜34 𝑜35 1 𝑜14⁄ 1 𝑜24⁄ 1 𝑜34⁄ 1 𝑜15⁄ 1 𝑜25⁄ 1 𝑜35⁄ 1 𝑜45 1 𝑜45⁄ 1 ). (28) step 3: determine each criterion’s level of priority or weight 𝑤𝑖. the priority level 𝑤𝑖 of criterion 𝑖 is calculated through the following equation: 𝑶𝒘 = 𝝀𝒎𝒂𝒙𝒘, (29) where 𝒘 is the vector of weights 𝑤𝑖 and 𝜆𝑚𝑎𝑥 is the maximum eigenvalue. step 4: evaluate the consistency of the pairwise comparisons. to check whether the respondents provide a set of consistent preference ratings from the survey results, the consistency ratio (cr), defined by equation 30 as the ratio between the consistency index (ci) of the pairwise comparison matrix and the consistency index of a randomly generated matrix (ri) was calculated. if cr is greater than 0.10, the respondents were asked to reconsider their initial preference rating to improve their judgments in case they inadvertently did not communicate their judgments well. 𝐶𝑅 = 𝐶𝐼 𝑅𝐼 , (30) where ci is defined as: 𝐶𝐼 = 𝜆𝑚𝑎𝑥−𝑛 𝑛−1 (31) and the ri values were generated by saaty (2012) for various sizes of matrices as shown in the table below: ijahp article: go, promentilla, aviso, yu/an ahp-based composite index for sector prioritization international journal of the analytic hierarchy process 54 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.638 table 2 consistency indices for random matrices n 3 4 5 ri 0.58 0.9 1.12 4. case study this section features a case study to demonstrate the use of the composite sector prioritization index in the case of the philippines, an archipelago comprised of 7,100 islands located along southeast asia. its proximity to the pacific ring of fire makes it susceptible to an array of natural hazards including typhoons, tsunamis, earthquakes, and drought. this makes the country prone to widespread output and economic loss. this geographic location allowed the philippines to rank 5 th in the global climate risk index and 3 rd in the most high-risk developing countries in asia in terms of disaster-driven economic loss (eckstein, künzel, & schäfer, 2017; asian development bank, 2013). because of this, the philippines, as well as other disaster-prone countries, need efficient sector prioritization tools in order to manage sector vulnerability in times of crisis. additionally, policymakers are also faced with the problem of scarce resources thus highlighting the need to efficiently allocate these to maximize gains and minimize losses in the economy. hence, policymakers must identify and invest resources towards key sectors that are able to generate the largest positive spillover effects from the limited amount of resources given to them. since an economy has various priorities and concerns, policymakers must also be able to account for their respective economy’s preferences when allocating resources. given this, we adopted the ahp by selecting three key experts in the field of structural economics and the philippine economy to properly mirror the country’s preference assignments of the five sector significance criteria considered in the study. three philippine economy experts, academicians, and policymakers 3 were selected to answer the ahp survey found in appendix 1. additionally, table 3 specifies that the respondents’ pairwise comparisons collectively generated a consistency ratio of 0.009 4 . given this, the geometric means of the respondents pairwise comparisons were generated and used to determine the respective weights of the five sector significance criteria (see 3 these experts include: (1) the assistant director of the national economic development authority, (2) a professor from de la salle university’s school of economics, and (3) a professor from the university of the philippines’s school of economics. 4 each experts’ pairwise comparisons were also found to be consistent as each expert yielded individual consistency ratios (cr) ranging from 0.0003 to 0.089, which are less than the 0.1 threshold. as expected, a consistency index of less than 0.1 has always been obtained from real experts who sincerely gave their judgments as pointed out by the reviewer in his/her over 24 years of experience implementing ahp/anp on real decision making. ijahp article: go, promentilla, aviso, yu/an ahp-based composite index for sector prioritization international journal of the analytic hierarchy process 55 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.638 table 3) 5 . these weights were then multiplied to their corresponding sector significance criteria values. after which, the weighted values were added to obtain each sector’s composite index value, which will be compared to identify the key sectors in the economy. for this study, we used the 2012 philippine input-output table 6 published by the philippine statistics authority to demonstrate the use of the index. table 3 presents the summary of the pairwise comparison matrix and the ahp weights generated for 2012. table 3 analytic hierarchy process weights indicator pdi ss apl spc im final weight pdi 1.0000 1.5286 0.5228 1.9129 0.3293 0.1507 ss 0.6542 1.0000 0.3420 0.7114 0.2311 0.0880 apl 1.9128 2.9240 1.0000 2.0801 0.6300 0.2538 spc 0.5228 1.4057 0.4807 1.0000 0.3029 0.1101 im 3.0367 4.3271 1.5873 3.3014 1.0000 0.3973 consistency ratio 0.0090 as seen in table 3, the economy gives a higher priority to sectors with higher inoperability multipliers or sectors, which largely contribute to the economy’s risk of inoperability. this is possibly due to the philippines’ vulnerability to disaster-driven shocks making it prone to cyclical production disruptions and economic loss. hence, this may cause policymakers to prefer to divert resources towards sectors which contribute to a greater amount of disaster-driven risk, in order to mitigate widespread inoperability and loss. additionally, the country is also seen to prioritize sectors which have higher average propagation lengths or more interactions with the other sectors in the economy. this is understandable, especially since policymakers may opt to invest in sectors that can widen the scope of positive spillover effects through their large interconnections with the rest of the economy. 4.1 sector prioritization results given the economy’s preferences, table 4 presents a comprehensive list of the composite sector prioritization indices and rankings per sector for the year 2012. 7 as shown in table 4, the key sectors of the philippine economy are the manufacturing, private services, and trade sectors. this means that more financial aid and resources must be allocated towards the manufacturing sector as it has the ability to generate widespread inoperability and loss as a result of disruptions in the sector’s operations. since the 5 for ease, the ahp weights were calculated using promentilla’s ahp calculator (2006) in microsoft excel 2016. 6 the 2012 philippine input-output table is found in appendix 2. 7 the breakdown of the sector rankings with respect to each sector significance criteria is summarized in appendix 3. ijahp article: go, promentilla, aviso, yu/an ahp-based composite index for sector prioritization international journal of the analytic hierarchy process 56 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.638 manufacturing sector is a key consumer and producer of goods and services, disruptions experienced by this sector may cause production delays for sectors that rely on manufactured inputs and conversely, may also cause a decline in the manufacturing sector’s demand for inputs from the rest of the economy. overall, these may lead to demandand supply-driven shocks that disrupt the operations of the rest of the economy. additionally, since this sector is heavily interconnected with other economic sectors, investments or negative shocks experienced by the sector can easily be propagated to the rest of the economy. these, coupled with the sector’s large output contribution or size, dependence on the domestic economy, and degree of relative influence, makes manufacturing a key sector to invest in as it can potentially generate higher positive spillover effects due to the magnitude and reach of its influence and interconnections. conversely, these characteristics also make the sector highly vulnerable to large-scale loss and disruptions. hence, these highlight the importance of prioritizing this sector in terms of resource allocation and disaster-risk management decisions. table 4 composite index results (2012) sector composite index rank agriculture, fishery and forestry 0.0825 6 mining and quarrying 0.0596 11 manufacturing 0.1871 1 construction 0.0820 7 electricity, gas and water 0.0755 8 transportation, communication and storage 0.0852 4 trade 0.1070 3 finance 0.0840 5 real estate and ownership of dwellings 0.0598 10 private services 0.1125 2 government services 0.0647 9 moreover, the private services sector, which has been known to create services needed to support and sustain industrial sectors, like manufacturing, in their day-to-day operations was found to have the second highest ranking in the roster. this implies that much of the economy’s financial aid must be allocated towards this sector to ensure that the country’s industries remain afloat. otherwise, disruptions experienced by this sector may lead to large-scale inoperability and loss because of its large contribution to risk and high degree of interindustry relations. additionally, its output is highly dependent on the operations of industrial sectors, thus making it vulnerable to shocks experienced by the domestic economy. hence, much of the economy’s resources must be allocated towards this sector because of the risks it may contribute due to the disruptions that the philippine economy may experience. furthermore, the trade sector also had a high priority ranking. more investment must then be directed towards this sector, as the philippines heavily depends on its operations to fuel its economy. the country’s dependence on this sector also implies that disruptions experienced in the industry may cause production delays and disruptions to proliferate ijahp article: go, promentilla, aviso, yu/an ahp-based composite index for sector prioritization international journal of the analytic hierarchy process 57 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.638 towards the rest of the economy. aside from the trade sector’s structural significance and contribution to risk, its interdependence with other sectors in the economy makes it a crucial sector especially since many sectors depend on the sales or demand of their goods and services to fuel their production. conversely, this also means that the trade sector relies on these other sectors’ output to continue its operations. these make the trade sector an ideal industry to invest in as it is highly vulnerable to shocks and has the ability to potentially propagate small investments into larger positive spillover effects across the entire economy. additionally, we can observe that the following sectors received mid-priority rankings: (1) transportation, communication and storage, (2) finance, and (3) construction because all three sectors have less interindustry interactions and lower sector sizes relative to the other economic sectors. hence, these sectors are less likely to contribute to large-scale disruptions, making them less crucial to invest in as these allocated large amounts of resources may only lead to minimal returns. it is also interesting to note that the agriculture, fishery and forestry sector only received a mid-priority ranking despite its perceived vulnerability to calamities. this is possibly because the sector also has less interindustry interactions and influence on the rest of the economy, thus despite the sector’s susceptibility to disruptions and output contribution, it is not deemed a priority due to its inability to generate large-scale spillover effects in the economy. aside from these, the electricity, gas and water, real estate and ownership of dwellings sectors, and mining and quarrying sectors received low-priority rankings among the other sectors in the roster because of their small contribution to the country’s total output and risk of inoperability. these sectors are also known to have minimal interactions with other sectors in the economy due to the nature of their operations. hence, this further contributes to their low dependence on the domestic economy and their low potential to propagate negative spillover effects towards the rest of the economy. given this, policymakers must not prioritize these sectors when allocating financial aid and resources as they experience and contribute to less vulnerability risk. lastly, table 3 also highlights the government sector’s low-priority rankings. the structure of the philippine economy illustrates that, while the government sector is usually given more priority in terms of resource allocation decisions, this may not necessarily be the case for all countries. our findings affirm that the philippine government sector has almost always taken on the role of a consumer rather than a producer of goods and services. this is because the philippines is a developing and relatively young country, thus making its government less established and influential in propagating spillover effects towards other sectors in the economy due to its minimal interindustry interactions. hence, allocating more financial aid towards this sector may only lead to minimal impacts on the economy. 5. model limitation and future research the current model assumes mutual independence among criteria in the computation of their importance weights that would influence the prioritization of the economic sectors. ijahp article: go, promentilla, aviso, yu/an ahp-based composite index for sector prioritization international journal of the analytic hierarchy process 58 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.638 our proposed framework, which was built from an ahp model, can thus be extended to the analytic network process (anp) model wherein the interdependence among criteria are made explicit and considered in sector prioritization (saaty, 2006). moreover, further research may explore the possibility of adding other sector significance criteria such as export contribution, employment contribution and import reliance to account for other possible concerns of the researcher’s economy of choice. this study only makes use of five sector significance criteria to make the index more concise and specific to the primary needs of an economy. other sector significance measures were also not included in the study due to lack of data but may be explored in future works. nevertheless, the components used in this study can adequately provide insightful observations about each sector’s role and relative importance in the economy. in addition, integrating alternative index aggregation techniques and weighting procedures is recommended to account for possible nonlinear relationships among the index components. 6. conclusion this paper developed a composite sector prioritization index that identifies the key sectors in the economy based on multiple dimensions of sector significance. this index is comprised of five key measures or criteria developed from i-o modelling to account for each sector’s role and relative importance in the economy. first, the power of dispersion index measures a sector’s relative influence or positive spillover effect in the economy. second, the sector size measures a sector’s output share or structural significance with respect to the entire economy. third, the average propagation length measures a sector’s number of interindustry interactions as both a consumer and producer in the economy. fourth, the sectoral purchase coefficient measures a sector’s dependence on the domestic economy for its output production, or vulnerability to domestic shocks. lastly, the inoperability multiplier measures the sector’s negative spillover effects or contribution to the economy’s risk of inoperability. additionally, this index was constructed using the analytic hierarchy process, a systematic approach that allows us to reflect an economy’s preference assignments for each sector significance criterion. because of these features, the composite index is considered a useful tool to aid policymakers in identifying which sectors to prioritize given the economy’s varied priorities and preferences. this tool can benefit various stakeholders in their resource allocation decisions as it identifies the key economic sectors while accounting for a wider range of factors the economy may consider. since this tool grants policymakers a broader view of an economy’s various concerns, this allows them to determine which sectors are critically in need of investment and should be given the most priority in policy-related decisions. through this, policymakers can effectively reduce the risk and production losses these critical sectors may propagate to the entire economy. reducing widespread risk will then prevent other sectors from experiencing negative spillover effects thus promoting stable employment and economic prosperity, which in turn fosters sustainable and inclusive economic growth. using the case of the philippines, we demonstrated the tool’s ability to identify the key sectors to prioritize in order to efficiently manage widespread risk and maximize returns on investment. we were able to show that the policymakers give a higher priority to sectors that largely contribute to the economy’s risk of inoperability. given this, we ijahp article: go, promentilla, aviso, yu/an ahp-based composite index for sector prioritization international journal of the analytic hierarchy process 59 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.638 found that the manufacturing, private services, and trade sectors received high priority rankings, indicating that these sectors’ large contribution to the inoperability risk and significant industry interactions makes it highly vulnerable to and influential in propagating risk. hence, more financial aid and resources must be allocated towards these sectors to prevent the widespread propagation of inoperability and to generate the highest possible impact on the rest of the economy. prioritizing these sectors may also significantly improve the country’s resilience and preparedness measures in the event of disruptions experienced by the economy. overall, the case study was able to demonstrate the advantages of using this tool as it considers the priorities of the economy when identifying the key economic sectors to invest in. while this study highlights the application of the tool in the case of the philippines, this sector prioritization index can easily be applied to various economies, which may grant us more insight into the possible similarities and differences among the key sectors of different nations. ijahp article: go, promentilla, aviso, yu/an ahp-based composite index for sector prioritization international journal of the analytic hierarchy process 60 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.638 references abdullah, l., jaafar, s., & che taib, c.m.i. (2013). ranking of human capital indicators using analytic hierarchy process. procedia social and behavioral sciences, 107. doi: 10.1016/j.sbspro.2013.12.394. alauddin, m. (1986). identification of key sectors in the bangladesh economy: a linkage analysis approach. applied economics, 18(4), 421-442. doi: https://doi.org/10.1080/00036848600000039 asian development bank. (2013). the rise of natural disasters in asia and the pacific: learning from adb's experience, 1-9. retrieved from https://www.adb.org/sites/default/files/evaluation-document/36114/files/rise-naturaldisasters-asia-pacific.pdf. doi: https://doi.org/10.1787/9789264041363-en ballester, r. e. d., granadillos, j. r. g., quintos, m. a., & cruz, m. d. (2013). evolution of the philippine economy as seen through the time-series input-output tables (1961 to 2006). national policy and planning staff-national economic and development authority (npps-neda). barbarosoglu, g., & yazgac, t. (1997). an application of the analytic hierarchy process to the supplier selection problem. production and inventory management journal, 38(1), 14. barker, k., & santos, j. r. (2010). a risk‐based approach for identifying key economic and infrastructure systems. risk analysis, 30(6), 962-974. doi: https://doi.org/10.1111/j.1539-6924.2010.01373.x dedeke, n. (2013). estimating the weights of a composite index using ahp: case of the environmental performance index. british journal of arts & social sciences, 11, 199221. dejuán, ó., lenzen, m., & cadarso, m. á. (eds.). (2017). environmental and economic impacts of decarbonization: input-output studies on the consequences of the 2015 paris agreements. routledge. doi: https://doi.org/10.4324/9781315225937 dhawan, s., & saxena, k. (1992). sectoral linkages and key sectors of the indian economy. indian economic review, 27(2), 195-210. dietzenbacher, e., romero luna, i., and bosma, n.s. (2005) using average propagation lengths to identify production chains in the andalusian economy. estudios de economia aplicada, 23, 405-422. drejer, i. (2002). input-output based measures of interindustry linkages revisited-a survey and discussion. in 14th international conference on input-output techniques, montreal, canada. https://www.adb.org/sites/default/files/evaluation-document/36114/files/rise-natural-disasters-asia-pacific.pdf https://www.adb.org/sites/default/files/evaluation-document/36114/files/rise-natural-disasters-asia-pacific.pdf ijahp article: go, promentilla, aviso, yu/an ahp-based composite index for sector prioritization international journal of the analytic hierarchy process 61 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.638 eckstein, d., künzel, v., & schäfer, l. (2017). global climate risk index 2018. germanwatch, 33. retrieved from http://germanwatch.org/en/download/20432.pdf eichhorn, w. (1976) fisher’s tests revisited. econometrica: journal of the econometric society, 44, 247-256. doi: https://doi.org/10.2307/1912721 haimes, y.y. and jiang, p. (2001). leontief based model of risk in complex interconnected
infrastructures. journal of infrastructure systems, 7, 1-12. doi: https://doi.org/10.1061/(asce)1076-0342(2001)7:1(1) hewings, g. j. (1982). the empirical identification of key sectors in an economy: a regional perspective. the developing economies, 20(2), 173-195. doi: https://doi.org/10.1111/j.1746-1049.1982.tb00444.x kurttila, m., pesonen, m., kangas, j., & kajanus, m. (2000). utilizing the analytic hierarchy process (ahp) in swot analysis—a hybrid method and its application to a forest-certification case. forest policy and economics, 1(1), 41-52. doi: https://doi.org/10.1016/s1389-9341(99)00004-0 leontief, w. (1936). quantitative input and output relations in the economic system of the united states. review of economics and statistics, 18, 105-125. doi: https://doi.org/10.2307/1927837 lian, c. and haimes, y. y. (2006). managing the risk of terrorism to interdependent infrastructure systems through the dynamic inoperability input–output model. systems engineering, 9, 241–258. doi:10.1002/sys.20051 liu, f. h. f., & hai, h. l. (2005). the voting analytic hierarchy process method for selecting supplier. international journal of production economics, 97(3), 308-317. doi: https://doi.org/10.1016/j.ijpe.2004.09.005 miller, r.e. and blair, p.d. (2009) input-output analysis: foundations and extensions. 2nd ed. cambridge, uk: university press. niemira, m. p. (2001). an ahp-based composite cyclical-performance index. indian economic review, 36(1), 241-250. nydick, r. l., & hill, r. p. (1992). using the analytic hierarchy process to structure the supplier selection procedure. journal of supply chain management, 28(2), 31-36. doi: https://doi.org/10.1111/j.1745-493x.1992.tb00561.x okuyama, y. & yu, k.d. (2018). return of the inoperability. economic systems research. doi:10.1080/09535314.2018.1510383. orencio, p. m., & fujii, m. (2013). a localized disaster-resilience index to assess coastal communities based on an analytic hierarchy process (ahp). international journal of disaster risk reduction, 3, 62-75. doi: https://doi.org/10.1016/j.ijdrr.2012.11.006 ijahp article: go, promentilla, aviso, yu/an ahp-based composite index for sector prioritization international journal of the analytic hierarchy process 62 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.638 organisation for economic co-operation and development. (2011). oecd work on sustainable development [brochure]. france. rasmussenís, p.n. (1957), studies in inter-sectoral relations. amsterdam, northholland. pandian, g. s., jawahar, n., & nachiappan, s. p. (2013). composite performance index for sustainability. iosr journal of environmental science, toxicology and food technology, 3(1), 91-102. doi: https://doi.org/10.9790/2402-03191102 promentilla, m.a. (2006). ahp calculator [microsoft excel spreadsheet]. de la salle university, manila, philippines. resurreccion, j.z., & santos, j.r. (2012a). integrated stochastic inventory and inputoutput models for enhancing disaster preparedness of disrupted interdependent sectors. paper presented at the 20th international input-output conference and the 2nd edition of the international school of input-output analysis, bratislava, slovakia. doi: https://doi.org/10.1111/risa.12002 resurreccion, j., & santos, j. r. (2012b). multiobjective prioritization methodology and decision support system for evaluating inventory enhancement strategies for disrupted interdependent sectors. risk analysis, 32(10), 1673-1692. doi: https://doi.org/10.1111/j.1539-6924.2011.01779.x saaty, t. l., & vargas, l. g. (1979). estimating technological coefficients by the analytic hierarchy process. socio-economic planning sciences, 13(6), 333-336. doi: https://doi.org/10.1016/0038-0121(79)90015-6 saaty, t.l. (1980). analytic hierarchy process. new york: mcgraw-hill. saaty t.l. (2006) the analytic network process. in decision making with the analytic network process. (international series in operations research & management science, vol. 95), 1-26. boston, ma: springer. doi: https://doi.org/10.1007/0-387-33987-6_1 saaty, t. l. (2008). decision making with the analytic hierarchy process. international journal of services sciences, 1(1), 83-98. saaty, t. (2010). economic forecasting with tangible and intangible criteria: the analytic hierarchy process of measurement and its validation. economic horizons, 1, 5-45. saaty, t.l. (2012). decision making for leaders: the analytic hierarchy process for decisions in a complex world (3rd revised edition). pittsburgh: rws publications santos, j.r. and haimes, y.y. (2004). modeling the demand reduction input-output (i-o) inoperability due to terrorism of interconnected infrastructures. risk analysis, 24, 14371451. doi: https://doi.org/10.1111/j.0272-4332.2004.00540.x ijahp article: go, promentilla, aviso, yu/an ahp-based composite index for sector prioritization international journal of the analytic hierarchy process 63 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.638 silva, a. c., belderrain, m. c. n., & pantoja, f. c. m. (2010). prioritization of r&d projects in the aerospace sector: ahp method with ratings. journal of aerospace technology and management, 2(3), 339-348. doi: https://doi.org/10.5028/jatm.2010.02039110 triantaphyllou, e., & mann, s. h. (1995). using the analytic hierarchy process for decision making in engineering applications: some challenges. international journal of industrial engineering: applications and practice, 2(1), 35-44. tsekeris, t. (2017). network analysis of inter-sectoral relationships and key sectors in the greek economy. journal of economic interaction and coordination, 12(2), 413-435. doi: https://doi.org/10.1007/s11403-015-0171-7 yu, k.d.s., tan, r.r., aviso, k.b., promentilla, m.a.b., & santos, j.r. (2014). a vulnerability index for post-disaster key sector prioritization. economic systems research, 26(1), 81-97. doi: 10.1080/09535314.2013.872603 zuhdi, u. (2017a). the ranks of japanese industrial sectors: 2005-2011. journal of physics: conference series, 820(1), 012030. doi: https://doi.org/10.1088/17426596/820/1/012030 zuhdi, u. (2017b). an analysis of the characteristics of japanese industrial sectors from 2005 through 2011. iop conference series: earth and environmental science, 88(1), 012027. doi: https://doi.org/10.1088/1755-1315/88/1/012027 ijahp article: go, promentilla, aviso, yu/an ahp-based composite index for sector prioritization international journal of the analytic hierarchy process 64 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.638 appendix appendix 1 analytic hierarchy process questionnaire in 2012, which of the attributes contributes more to the significance of a sector to the entire economy and by how much? power of dispersion index vs. sector size 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 e x tr e m e ly h ig h e r v e r y s ig n if ic a n tl y h ig h e r s ig n if ic a n tl y h ig h e r m o d e r a te ly h ig h e r e q u a l c o n tr ib u ti o n m o d e r a te ly h ig h e r s ig n if ic a n tl y h ig h e r v e r y s ig n if ic a n tl y h ig h e r e x tr e m e ly h ig h e r power of dispersion index vs. average propagation length 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 e x tr e m e ly h ig h e r v e r y s ig n if ic a n tl y h ig h e r s ig n if ic a n tl y h ig h e r m o d e r a te ly h ig h e r e q u a l c o n tr ib u ti o n m o d e r a te ly h ig h e r s ig n if ic a n tl y h ig h e r v e r y s ig n if ic a n tl y h ig h e r e x tr e m e ly h ig h e r power of dispersion index vs. sectoral purchase coefficient 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 e x tr e m e ly h ig h e r v e r y s ig n if ic a n tl y h ig h e r s ig n if ic a n tl y h ig h e r m o d e r a te ly h ig h e r e q u a l c o n tr ib u ti o n m o d e r a te ly h ig h e r s ig n if ic a n tl y h ig h e r v e r y s ig n if ic a n tl y h ig h e r e x tr e m e ly h ig h e r power of dispersion index vs. inoperability multiplier 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 e x tr e m e ly h ig h e r v e r y s ig n if ic a n tl y h ig h e r s ig n if ic a n tl y h ig h e r m o d e r a te ly h ig h e r e q u a l c o n tr ib u ti o n m o d e r a te ly h ig h e r s ig n if ic a n tl y h ig h e r v e r y s ig n if ic a n tl y h ig h e r e x tr e m e ly h ig h e r sector size vs. average propagation length 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 e x tr e m e ly h ig h e r v e r y s ig n if ic a n tl y h ig h e r s ig n if ic a n tl y h ig h e r m o d e r a te ly h ig h e r e q u a l c o n tr ib u ti o n m o d e r a te ly h ig h e r s ig n if ic a n tl y h ig h e r v e r y s ig n if ic a n tl y h ig h e r e x tr e m e ly h ig h e r ijahp article: go, promentilla, aviso, yu/an ahp-based composite index for sector prioritization international journal of the analytic hierarchy process 65 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.638 sector size vs. sectoral purchase coefficient 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 e x tr e m e ly h ig h e r v e r y s ig n if ic a n tl y h ig h e r s ig n if ic a n tl y h ig h e r m o d e r a te ly h ig h e r e q u a l c o n tr ib u ti o n m o d e r a te ly h ig h e r s ig n if ic a n tl y h ig h e r v e r y s ig n if ic a n tl y h ig h e r e x tr e m e ly h ig h e r sector size vs. inoperability multiplier 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 e x tr e m e ly h ig h e r v e r y s ig n if ic a n tl y h ig h e r s ig n if ic a n tl y h ig h e r m o d e r a te ly h ig h e r e q u a l c o n tr ib u ti o n m o d e r a te ly h ig h e r s ig n if ic a n tl y h ig h e r v e r y s ig n if ic a n tl y h ig h e r e x tr e m e ly h ig h e r average propagation length vs. sectoral purchase coefficient 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 e x tr e m e ly h ig h e r v e r y s ig n if ic a n tl y h ig h e r s ig n if ic a n tl y h ig h e r m o d e r a te ly h ig h e r e q u a l c o n tr ib u ti o n m o d e r a te ly h ig h e r s ig n if ic a n tl y h ig h e r v e r y s ig n if ic a n tl y h ig h e r e x tr e m e ly h ig h e r average propagation length vs. inoperability multiplier 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 e x tr e m e ly h ig h e r v e r y s ig n if ic a n tl y h ig h e r s ig n if ic a n tl y h ig h e r m o d e r a te ly h ig h e r e q u a l c o n tr ib u ti o n m o d e r a te ly h ig h e r s ig n if ic a n tl y h ig h e r v e r y s ig n if ic a n tl y h ig h e r e x tr e m e ly h ig h e r sectoral purchase coefficient vs. inoperability multiplier 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 e x tr e m e ly h ig h e r v e r y s ig n if ic a n tl y h ig h e r s ig n if ic a n tl y h ig h e r m o d e r a te ly h ig h e r e q u a l c o n tr ib u ti o n m o d e r a te ly h ig h e r s ig n if ic a n tl y h ig h e r v e r y s ig n if ic a n tl y h ig h e r e x tr e m e ly h ig h e r ijahp article: go, promentilla, aviso, yu/an ahp-based composite index for sector prioritization international journal of the analytic hierarchy process 66 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.638 appendix 2 2012 philippine input-output table (in millions of philippine pesos) appendix 3 breakdown of rankings for each sector significance criteria agr min man cons egw tcs trd fin real privsrv govt f x agriculture, fishery and forestry 264,256,364.48 1,314,816.73 750,262,030.24 33,212,393.32 3,785,050.61 9,042,326.73 50,023,052.70 6,217,225.93 11,686,543.00 104,440,578.55 2,748,526.65 908,331,258.51 2,145,320,167.46 mining and quarrying 73,872.21 301,459.72 310,676,086.17 49,078,614.72 14,235,501.06 188,947.91 4,253,668.66 44,581.53 1,420,463.38 1,317,117.82 16,748.83 -198,431,894.53 183,175,167.46 manufacturing 130,410,148.74 8,334,407.42 1,803,375,804.27 317,366,034.08 81,126,761.82 242,411,790.82 438,051,559.22 40,455,469.23 31,138,668.88 449,751,132.48 15,753,399.88 2,809,121,723.25 6,367,296,900.09 construction 238,415.80 4,672,076.66 54,447,332.78 2,609,838.03 8,542,123.38 19,021,239.56 26,526,666.61 5,729,345.82 9,506,262.57 32,880,346.96 195,278.41 861,736,567.13 1,026,105,493.70 electricity, gas and water 4,362,862.51 1,459,164.97 110,616,825.02 5,265,797.90 85,371,755.51 36,342,768.04 98,902,785.75 7,556,734.22 8,277,136.80 123,197,064.59 3,122,662.59 336,584,575.01 821,060,132.90 transportation, communication and storage 15,542,905.71 1,363,400.97 175,545,943.70 7,048,505.10 1,786,259.39 113,246,607.89 118,357,972.74 25,826,586.64 5,323,419.63 88,117,979.11 34,688,454.55 867,908,894.99 1,454,756,930.42 trade 158,873,898.28 15,571,097.27 683,355,097.67 113,548,388.74 50,834,142.30 158,027,834.31 468,050,239.55 121,692,807.02 78,903,283.01 452,855,350.73 26,845,941.28 837,001,853.81 3,165,559,933.97 finance 15,839,566.37 1,051,685.48 103,000,998.58 22,689,415.77 9,620,010.58 85,378,476.31 196,718,491.77 306,646,251.49 14,994,724.22 156,084,712.07 29,852,682.08 512,084,509.29 1,453,961,524.00 real estate and ownership of dwellings 27,117.15 1,679,124.91 17,946,614.77 883,939.15 1,596,924.54 6,355,796.37 8,084,416.59 2,103,912.51 3,348,457.65 10,221,245.06 43,945.37 674,754,278.24 727,045,772.31 private services 62,200,488.74 6,791,469.35 246,965,842.61 47,775,511.13 7,886,466.04 72,322,147.13 101,400,021.43 81,428,321.28 23,960,268.39 435,791,494.77 68,407,674.80 2,497,170,641.45 3,652,100,347.11 government services 429.83 62,092.09 3,079,103.35 306,784.60 69,369.49 3,603,818.22 21,994,203.52 5,001,537.31 1,225,414.95 17,415,068.85 60,330,065.13 568,550,196.44 681,638,083.77 sector pdi pdi rank ss ss rank apl apl rank spc spc rank im im rank agriculture, fishery and forestry 0.0774 9 0.0990 4 0.1000 4 0.0983 7 0.0653 7 mining and quarrying 0.0725 11 0.0084 11 0.1070 1 0.0300 11 0.0440 11 manufacturing 0.1157 1 0.2937 1 0.1023 3 0.0695 10 0.2773 1 construction 0.1097 2 0.0473 7 0.0767 9 0.1044 3 0.0763 4 electricity, gas and water 0.0800 8 0.0379 8 0.1000 4 0.1046 1 0.0585 8 transportation, communication and storage 0.1015 3 0.0671 5 0.0930 8 0.0944 8 0.0754 5 trade 0.0975 5 0.1460 3 0.0977 6 0.0997 6 0.1100 3 finance 0.0881 6 0.0671 6 0.1047 2 0.1013 5 0.0682 6 real estate and ownership of dwellings 0.0747 10 0.0335 9 0.0558 11 0.1045 2 0.0503 10 private services 0.1001 4 0.1685 2 0.0977 6 0.0900 9 0.1205 2 government services 0.0827 7 0.0314 10 0.0651 10 0.1033 4 0.0543 9 ijahp essay: mu/selecting a journal for your ahp/anp publication: a guide for beginners international journal of the analytic hierarchy process 249 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.307 selecting a journal for your ahp/anp publication: a guide for beginners enrique mu some time ago, a colleague who had submitted a paper to our ijahp journal asked me about the status of his paper. he told me he needed to publish something as soon as possible because he was about to complete his doctoral studies and needed to show that he had publications. i asked him where he planned to submit job applications, and whether ijahp was a suitable journal for this purpose. he had a vague idea about the first part of my question, but he was clueless about the second part. this conversation constitutes the motivation for this essay, although i must warn the reader that it is from a mainly u.s. perspective due to the nature of the author’s experience. when selecting a suitable journal to submit our ahp/anp scholarly work, we may use the same analytic hierarchical approach we use to do our research. at the top level you should decide what the goal of publishing the paper is. are you publishing to get an academic position, or publishing to divulge an important finding? this distinction may not be obvious to young academics who often jump at the first publishing opportunity, but your top goal makes all the difference when choosing where to submit for publication. given that this essay is aimed at young academicians, let’s assume that you want to publish a paper based on your dissertation and that you expect your publication to improve your chances of landing your dream academic job upon doctoral graduation. your criteria should be based on your intended goal. i do not expect to develop a decision model within the boundaries of an essay, but it is possible to discuss some important criteria, in no particular order, to select the most suitable journals in which to submit your research. first, you must consider the journal citation indexing requirements of your target institution. a citation index is a bibliographic database. the top of the line publications for beginners 1 are those indexed in the web of science. next in the pecking order are scopus indexed publications. these two bibliographic databases are the main competitors, and to be listed in one or both will give a journal a good reputation. in addition to these two indices, there are many other indices such as copernicus, citeseer, doaj and google scholar. the bottom line is that you must be aware of the indexing requirements of your target institution, otherwise, you risk having a publication that “does not count” for your job application purposes. second, the journal impact factor, that is, the measure of the average number of citations of recently published papers, also constitutes a common criterion. basically, this factor is used for assessing the impact of a journal. a journal with a higher impact factor should be a “better” journal to publish in because it will expose your paper to more people than a journal with a lower impact factor. however, impact factor is not as reliable as it seems. the impact factor can be affected by different variables, including editorial policy. for example, if a journal publishes a large number of review articles, it will be certain to have 1 in many institutions, a senior basket of journals is defined for their senior faculty should they want to keep moving up in the ranks beyond associate professor. ijahp essay: mu/selecting a journal for your ahp/anp publication: a guide for beginners international journal of the analytic hierarchy process 250 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.307 a higher impact factor because review articles are more often read and cited than original research articles. another questionable approach is the practice (by some editors) of coercing its authors into adding spurious citations from the same journal to increase its impact factor. for these reasons, universities (at least in the u.s.) rarely use impact factor as a criterion to judge a faculty publication, although journals promote it to attract authors. the lesson here is that even though a journal with a higher impact factor may be preferable; do not put too much importance on this factor. third, you must honestly assess your article’s publication potential; in other words, what are the real chances that your paper will be published in a given journal. while we may aim at the best fruit, realistically it may be too high in the tree. to assess this you should evaluate the target journal, its rejection rate, and your own self-assessment of the paper. some elite journals are clearly aimed at senior scholars and may have a rejection rate as high as 95% 2 . therefore, the probability of being published in such a journal is very remote unless you are really breaking new ground with your research. the selfassessment of your paper can prove to be a bit trickier. most young researchers confuse the quality of their research, which may be very high, and the quality of their papers which tends to be lower for the simple reason that young researchers still lack the experience of writing in an engaging and convincing way. a paper may have very little chance of being published in journal a, while having a great chance of being published in journal b, and deciding which journal to submit your paper to is a judgment call. if you are not sure, you can always ask the advice of a more veteran colleague or a mentor. an unfortunate practice, that i certainly do not support, is that of writing similar papers with different titles and cleverly written abstracts to make them appear different, and submitting them to two or more journals at once. nowadays, there are internet tools available to journal editors which make it relatively easy to detect this practice, and i can assure you there is nothing that irritates an editor more than finding out that the paper he/she has been working on is being submitted somewhere else. even without internet tools, i have had the frustrating situation of a trusted reviewer telling me that he revised the same paper for two other journals (true story!). this practice also becomes obvious when an author decides to withdraw a paper later in the editorial process, arguing for example that there is a need to update the references (really?, do you just realize this now?) as a way of avoiding the embarrassment of having the same paper being published simultaneously in two different journals. in summary, be ethical, be professional, and make sure you know where you want to get your paper published. fourth, journal fit or how well your topic fits with the aims of the target journal and, most importantly, its fit with past published papers of a journal. does your application (e.g. supply chain) and methodology (e.g. ahp) fit with previous published papers of a journal? while there is always a first time for everything, if the journal has previously published papers related to your topic, it indicates your scholarship fits the interests of the journal. similarly, if the journal has previously published articles using ahp/anp methodology, you will not have to spend the effort explaining what the methodology is and why it is suitable. 2 some journals report a high rejection rate as a badge of honor, but authors can usually inquire about an estimated rejection rate of a particular journal. ijahp essay: mu/selecting a journal for your ahp/anp publication: a guide for beginners international journal of the analytic hierarchy process 251 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.307 fifth, you must also consider the disciplinary scope of the journal. if your main research area is information technology, you may be expected by your department to publish in journals in this specific area rather than in other nonspecific operational research journals. on the other hand, some departments may not make a distinction between these types of journals. similarly, some departments may encourage inter-disciplinary work while other departments may be more rigid in terms of what constitutes a valid publication in their area. you can always check with your department chair about what they value most in this particular area. you should also do some research into where senior professionals in your school have published, and read their articles to educate yourself. after all, writing and publishing research papers also involves experience. you should also consider what kind of professionals you would like as readers. for example, if my ahp/anp study is about how to prioritize environmentally vulnerable forests, i would probably want a journal that is mainly read by environmental scientists and forestry experts rather than professional decision-making experts. on the other hand, if my approach is breaking new ground in the use of ahp/anp, i would consider a decision-making journal as a suitable outlet. in summary, ask yourself who and where you would like your readers to be. sixth, journal coverage, that is, where most of the journal readers are located is also very important. i can see my readers shaking his/her heads and saying, “c’mon! most journals have international coverage nowadays.” this may be true in theory; after all, how many journals have the word “international” in their names? open access journals are accessible everywhere, however, if you look more carefully, you may notice that an international journal in mongolia is read mainly by people in the region (i know i am using an extreme example, but i want you to get the idea). one simple way to check a journal’s coverage is to look for your target journal in the library of the academic institution you are interested in joining, or to mention the journal to the people that have the potential of becoming your colleagues and see what response you get. finally, you must also take into account your degree of fulfillment. which journal would you like to see your name associated with? i know, this may be in contradiction to your current or future department preferences, but this is why ahp/anp deals with multicriteria decision analysis. after working so hard to complete a study, and even harder to write your paper (believe me, doing the study is the easiest part many times), where do you want to see your paper published? imagine a conversation with a colleague in which you say, “i have recently published a paper in…”. what journal would you like to see your name associated with? this is quite subjective, and you as an author must make this decision. i realize that i have not been comprehensive in my discussion of journal selection criteria for your ahp/anp academic output, but i hope that i will have fulfilled my goal of making you reflect on why and where you want to publish your paper! ijahp article: dixon-ogbechi, adebayo /application of the ahp model to determine prefab housing adoption factors for developers in lagos state international journal of the analytic hierarchy process 297 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.635 application of the ahp model to determine prefab housing adoption factors for developers in lagos state bolajoko nkemdinim dixon-ogbechi university of lagos, lagos, nigeria bdogbechi@gmail.com anthony kayode adebayo university of lagos, lagos, nigeria aadebayo@unilag.edu.ng abstract research has shown that there is presently a housing deficit of about 17 million housing units in nigeria, yet it appears that the concerted efforts that have been put in place by the government to address this issue have not yielded the expected results. although the nigerian government in its vision 2020 policy document recognized that one of the ways of resolving the issue of providing adequate housing in nigeria is to change from the traditional brick and mortar system to modern construction methods like prefab housing, not much has been done to promote the adoption of this new method by the relevant stakeholders, among whom are the developers. this study employed a survey research approach using the ahp (analytic hierarchy process) to investigate the most important factors that developers take into consideration when they build a house. there are several different types of prefab housing and knowledge of what the developers’ consider key factors and priorities would make it possible to determine the type of prefab housing that best fits their needs. this information could provide the basis to design a government marketing strategy to promote prefab housing to both the developers and their customers. since the judgments will be provided by developers in lagos state, they will be customized for the conditions and preferences of the developers and citizens there. therefore, the government and organizations will be able to give priority to these identified factors when formulating relevant strategies to promote the building of more prefab housing. keywords: developers; prefab housing; strategy; ahp; lagos state 1. introduction a shortage of housing has been a major problem in nigeria in recent years. research has shown that there is presently a housing deficit of over 17 million housing units in nigeria, yet it appears that the concerted efforts that have been put in place to address this issue by the government have not yielded the expected results (adebayo & dixon-ogbechi, 2017a). although the nigerian government, in the vision 2020 policy document, recognized that one of the ways to resolve the issue of the provision of adequate housing in nigeria is to move from the traditional brick and mortar system to modern construction methodology like the prefab housing system, not much may have been done to promote mailto:bdogbechi@gmail.com ijahp article: dixon-ogbechi, adebayo /application of the ahp model to determine prefab housing adoption factors for developers in lagos state international journal of the analytic hierarchy process 298 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.635 the adoption of this new methodology by relevant built environment stakeholders among whom are developers (report of the vision 2020 national technical working group on housing, 2009; adebayo & dixon-ogbechi, 2017a). this is in support of johnson’s (2006) view that although the prefab housing system is one of the solutions to a nation’s housing needs, there is still little scholarly research on this topic especially in developing countries. in fact, to the best of the researchers’ knowledge, not much research has been done in this area in nigeria. this study seeks to bridge this gap in knowledge by investigating the factors that can promote the adoption of the prefab housing technology by developers in nigeria using the analytic hierarchy process model (ahp). this is of significance given that research by scholars such as wei, gibb and dainty (2008) on house builders' utilization of offsite construction methods revealed that although the level of usage of offsite modern method of construction (mmc) by large house builders is low, the level is likely to increase, given the pressures to improve quality, time cost, productivity, and health and safety. 1.1 statement of problem developers are private corporate bodies some of whom are registered with the real estate developers association of nigeria (redan), who engage in the business of building houses for sale/rent to the public. aside from the government, developers are the main stakeholders that engage in medium to large scale building of houses in lagos state. however, they may not have been able to make as much of an impact as possible in ameliorating the housing shortage problem because they mostly adopt the traditional brick and mortar system of construction (adebayo & dixon-ogbechi, 2017a). in their study, adebayo and dixon-ogbechi (2017a) sought to investigate the perception of developers on the use of the prefab methodology to deliver housing in lagos state with the aim of determining their willingness to accept this modern method of construction (mmc). they discovered that although developers are aware and quite knowledgeable about the prefab construction methodology, not many of them have used it extensively in their business despite their belief that it is marketable. nevertheless, the majority of the developers who were contacted in this study believed that they would adopt the prefab construction methodology to deliver housing if its advantages were effectively communicated. adebayo and dixon-ogbechi (2017a) further discovered that developers believe that 16 factors could promote their adoption of the prefabricated methodology for housing delivery in lagos state. the most prominent of these factors are faster project completion time (fpt), greater quality control (gct), usage of specialized production labor (uspl), reduced site labor (rsl), minimal environmental impact (mei), less site material (lsm), and cost effectiveness (ce). therefore, there is a need to determine the relative importance of these factors to enable the government to formulate appropriate policies and strategies to promote the adoption of the prefab housing system by developers. we used the ahp to determine these priorities by administering questionnaires to a sample of developers in the built environment in lagos state. http://www.tandfonline.com/author/pan%2c+wei http://www.tandfonline.com/author/gibb%2c+alistair+gf ijahp article: dixon-ogbechi, adebayo /application of the ahp model to determine prefab housing adoption factors for developers in lagos state international journal of the analytic hierarchy process 299 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.635 2. literature review 2.1 the concept of prefab the concept of prefabrication refers to the manufacturing processes in factories whereby various materials are combined to form a component part of a final installation which is then transported to the site (mcgraw hill construction, 2011). prefabrication is based on off-site technology; off-site technologies are structures that are manufactured in different locations than the location of usage with the aim of saving cost and time (panjehpour & abang ali, 2013). these offsite technologies are also called modern methods of construction or mmc (wei, gibb & dainty, 2008). this construction technique uses assembly line manufacturing strategies to produce the components of a house prior to its arrival at a construction site in order to produce prefab homes (tam, 2013). in johnson (2016), the us department of commerce defined prefab homes as homes made up of components such as walls, floors, roofs, panels, etc., which have been assembled in a factory prior to erection on the building foundation. prefab homes differ from traditional brick and mortar homes because of the factory assembly-line production method used in their construction (johnson, 2006). 2.2 types of prefab systems scholars such as tam (2013) and smith, asiz and gupta (2007) identified the following types of prefab systems which vary in terms of degree of flexibility, labor and transportation costs, namely: manufactured systems, modular systems, panelized systems, and pre-engineered systems. smith, asiz and gupta (2007) opined that manufactured systems are the prefab homes that are fully completed before arriving on the building site; modular systems are made up of multiple modules which are assembled together on the site and are also mostly produced in the factory; panelized systems are those highly flexible prefab systems made up of a prefab floor, roof, and wall panels that are assembled on the building site; and pre-engineered systems are the simplest types of prefab systems made up of individual components produced in the factory but whose components are easily assembled at the site without the need for any additional work. on the other hand, johnson (2006) categorized the types of prefab systems as precut, panelized, sectional, and preassembled. according to johnson, the precut system entails incorporating an assembly-line method of production by cutting/processing the lumber at the factory; the panelized system is similar to the precut system but its framing members are cut to specification and then the individual pieces are assembled into larger units, called panels, that are connected with bolts at the factory (carr, 1947); the sectional prefab system entails manufacturing the building in units of three dimensional modules which are completed in the factory and subsequently sent to the building site for assembly into a complete house (graff, 1947); and the preassembled prefab system is where prefab houses are fully constructed at the factory and then transported to be permanently installed at the building site. 3. objectives of study the main objective of this study is to add to the body of existing knowledge by employing the ahp as a tool to investigate the priorities of the factors and types of prefab housing to help the government plan a strategy to promote the adoption of the prefab housing technology by developers in lagos state. precisely, it sought to: http://www.tandfonline.com/author/pan%2c+wei http://www.tandfonline.com/author/gibb%2c+alistair+gf ijahp article: dixon-ogbechi, adebayo /application of the ahp model to determine prefab housing adoption factors for developers in lagos state international journal of the analytic hierarchy process 300 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.635 i. determine the relative importance of factors that determine the adoption of prefab housing by developers in lagos state. ii. identify the prefab system that is the most likely to be adopted by developers in lagos state. iii. recommend appropriate strategy(ies) that will promote the adoption of the prefab system by developers in lagos state. 4. research design/methodology research design: the survey research method using a quantitative approach mainly based on the analytic hierarchy process (ahp) model was adopted for this study. population: the population is made up of developers who engage in the business of building houses for sale/rent to the public in lagos state. the size is unknown because it is a hidden population with no official record of membership. sample size and procedure: a sample of 50 developers in the building environment in lagos state was selected using a combination of purposive and snowball sampling techniques. sources of data: primary and secondary data sources were used. primary data was obtained through the administration of questionnaires to developers in lagos state while secondary data was obtained from a literature review. the literature review enabled us to identify the most popular evaluative criteria for the adoption of the prefab system as well as to identify the alternatives (prefab housing systems that are marketable to consumers in lagos state). these data were subsequently used to construct the ahp model as well as develop the questionnaire. research instrument: a questionnaire was used to gather data (see appendix 1). the questionnaire was divided into two parts. part i sought information on the respondents’ characteristics and was designed using multiple choice and open-ended response questions. part ii was designed using the relative importance scale of the ahp (saaty, 1980; 2001) based on variables identified from previous studies by adebayo and dixonogbechi (2017a; 2017b). in order to ensure that the respondents had the proper knowledge to answer the survey, the purposive sampling procedure was adopted. in addition, we used trained research assistants who interacted with the respondents and explained what was required of each of them before the instrument/survey was administered. apart from this, each survey instrument had an explanation of the criteria and the alternatives as well as instructions to guide the respondents in filling it out. the questionnaire was given to a respondent by a research assistant who explained the process and retrieved it later at a mutually agreed upon date. limitations: most of the challenges faced in the course of data collection revolved around the difficulty in getting respondents to fill out the survey during the visit to the respective companies. also, most senior staff at the companies that were visited seemed to be too busy to fill out the questionnaires, some were very busy working at different ijahp article: dixon-ogbechi, adebayo /application of the ahp model to determine prefab housing adoption factors for developers in lagos state international journal of the analytic hierarchy process 301 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.635 project site locations. at different times, the research assistants had to wait for hours for some of the staff to finish their jobs before they could be persuaded to fill in the questionnaires. however, with the help of insiders at the various companies who assisted in monitoring the retrieval of the instruments, an overall response rate of 76% was achieved. data analytical procedure: the data that was gathered was edited, coded, and analyzed using descriptive statistics from the spss (statistical package for social sciences) version 21, excel and the superdecisions software for the ahp/anp version 3.0. the spss version 21 was used to analyze the demographic data while microsoft excel was used to obtain the geometric means of the priorities that were obtained based on the individual assessments of developers. these were then used as inputs for the superdecisions software. ahp model for the study: the elements used in the ahp model (saaty, 2001) are presented in figure 1 as follows: goal: to determine the most important preferred factors to promote the adoption of prefab housing by developers in lagos state. criteria: the most popular evaluative criteria for the adoption of the prefab system as identified in a study by adebayo and dixon-ogbechi (2017a) are as follows: faster project completion time (fpct), greater quality control (gqc), usage of specialized production labor (uspl), less site material (lsm), reduced site labor (rsl), minimal environmental impact (mei), and cost effectiveness (ce). alternatives: the prefabricated housing systems that are marketable to consumers in lagos state as identified in previous studies by adebayo and dixon-ogbechi, 2017b) are as follows: panelized prefab system (pps), hybrid prefab system (hps), other prefab system (ops), preassembled completed prefab system (pcps) and volumetric prefab system (vps). these alternatives are described in detail below: a panelized prefab system (pps) is prefab units produced in a factory and assembled on-site to produce a three dimensional structure. they could be open or closed panels. a hybrid prefab system (hps) is a combination of both volumetric/pod and panel systems. other prefab systems (ops) are prefab units that include floor cassettes, roof cassettes, pre-cast concrete foundation assemblies, pre-formed wiring looms and mechanical engineering composites. a preassembled completed prefab system (pcps) is completed prefab houses that are fully constructed at the factory. a volumetric prefab system is prefab houses in which the building is manufactured in units of three dimensional modules in controlled factory conditions prior to transport to the site. these include a toilet pod, kitchen pod, electrical pod, and lift pod. ijahp article: dixon-ogbechi, adebayo /application of the ahp model to determine prefab housing adoption factors for developers in lagos state international journal of the analytic hierarchy process 302 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.635 figure 1 analytic hierarchy process model of the most important preferred factors to promote the adoption of prefab housing by developers in lagos state 5. data/model analysis and discussion 5.1 response rate of the 50 questionnaires administered to developers in lagos state, 38 were correctly filled out and returned resulting in a response rate of 76%. 5.2 respondent characteristics the characteristics of the respondents are shown in table 1. determine the most important preferred factors to promote the adoption of prefab housing by developers in lagos state fpct gqc uspl lsm rsl mei ce panelized hybrid others preassembled volumetric alternatives criteria goal ijahp article: dixon-ogbechi, adebayo /application of the ahp model to determine prefab housing adoption factors for developers in lagos state international journal of the analytic hierarchy process 303 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.635 table 1 demographic distribution of respondents frequency valid percentage (%) age 20 to below 30 years 30 to below 40 years 40 to below 50 years total 14 13 11 38 36.8 34.2 28.9 100 highest educational qualification wasc/gce/neco b.sc. or equivalent m.sc. total 1 21 16 38 2.6 55.3 42.1 100 what is the staff strength (number of employees) in your organization? less than 100 above 100 total 8 30 38 21.1 78.9 100 what is the volume (number of houses) your company builds per annum? total 16 9 11 10 1 37 16.2 24.3 29.7 27 2.7 100 years of work experience less than a year 1 to less than 5 years 5 to less than 10 years 10 years and above total 1 7 14 16 38 2.6 18.4 36.8 42.1 100 our findings, in table 1 show that the respondents were primarily between 20 to 40 years of age (71.1%), graduates with at least a b.sc. or equivalent (97.4%), and that the organizations have a staff strength of above 100 (78.9%). furthermore, they have built from 5 to less than 20 houses per annum (81.1%), and have at least five years of work experience (78.9%). 5.3. model analysis: part ii 5.3.1 consistency index (ci) the consistency ratios of all of the pairwise comparison matrices were below 0.1, therefore the judgments of the respondents were all consistent and acceptable. 5.3.2 composite priorities the priorities for the relative importance of the factors for the adoption of prefab housing by developers in lagos state and the prefabricated housing systems that are preferred by developers in lagos state are presented in tables 2, 3 and 4. the geometric mean composite priorities were obtained by first, inputting the individual judgment of the ijahp article: dixon-ogbechi, adebayo /application of the ahp model to determine prefab housing adoption factors for developers in lagos state international journal of the analytic hierarchy process 304 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.635 developers into microsoft excel, and then computing the geometric mean of the priorities that were obtained based on the individual assessments of the developers. second, the geometric mean of the priorities was input into and analyzed with the superdecisions software (see screenshots in appendix 2). the individual assessments of the developers were obtained from the judgments that were recorded in the relative importance scaled section of the questionnaires that were administered (see appendix 1). the relative importance scaled questionnaire items were obtained from the judgment model on the superdecisions software. table 2 geometric mean composite priorities and relative importance ranking of preferred factors to promote the adoption of prefab housing by developers in lagos state criteria fpct gqc uspl lsm rsl mei ce pooled composite priority 0.248 0.202 0.110 0.110 0.110 0.110 0.110 relative importance ranking 1 2 5 5 5 5 5 table 2 shows the relative importance ranking of the factors that determine the adoption of prefab housing by developers in lagos state. these values were the results that were obtained from the criteria node comparisons with respect to the goal as calculated by the superdecisions software. it reveals that the respondents believe that the factor that most determines the adoption of prefab housing by developers in lagos state is faster project completion time (fpct) which had the highest ranking and a pooled composite value of 0.248. this is followed by greater quality control (gqc) with a pooled composite priority of 0.202. interestingly, after these two factors, they considered all of the other five factors of equal importance, namely usage of specialized production labor (uspl), and less site material (lsm), reduced site labor (rsl), minimal environmental impact (mei), and cost effectiveness (ce) of equal importance, with pooled composite priorities of 0.110. ijahp article: dixon-ogbechi, adebayo /application of the ahp model to determine prefab housing adoption factors for developers in lagos state international journal of the analytic hierarchy process 305 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.635 table 3 geometric mean composite priorities of the prefab housing systems by developers in lagos state with respect to criteria criteria prefab housing system fpct gqc uspl lsm rsl mei ce pps 0.427 0.422 0.236 0.333 0.330 0.425 0.264 hps 0.218 0.212 0.234 0.167 0.248 0.213 0.201 ops 0.109 0.120 0.169 0.167 0.140 0.121 0.193 pcps 0.130 0.140 0.169 0.167 0.140 0.121 0.174 vps 0.116 0.106 0.192 0.167 0.140 0.121 0.168 inconsistency 0.051 0.026 .049 0.00 0.013 0.013 0.04 table 3 gives the geometric mean composite priorities of the preferred prefab housing system by developers in lagos state with respect to each criterion. these values were obtained by inputting the geometric mean of the judgements of the developers into the superdecisions software and then analyzing/synthesizing them. it can be seen that when the alternatives, that is the prefab housing systems, are compared with respect to each criteria, the pps (panelized prefab system) is ranked highest across all of the criteria, and this is followed by hps (hybrid prefab system) which is ranked second highest across all of the criteria. table 4 geometric mean composite priorities and relative importance ranking of the preferred prefab housing system by developers in lagos state prefab housing system pps hps ops pcps vps pooled composite priority 0.366 0.214 0.138 0.145 0.137 relative importance ranking 1 2 4 3 5 table 4 reveals the relative importance ranking of the preferred prefab housing system by developers in lagos state. these values were obtained from the synthesized final output of the data that was analyzed with the superdecisions software. it reveals that the respondents believe that the most preferred prefab housing system by developers in lagos state is panelized prefab system (pps) which had the highest ranking and a pooled geometric mean composite priority of 0.366. this is almost twice the pooled composite priority for the hybrid prefab system (hps) of 0.214, which is the next most preferred prefab housing system by developers in lagos state. this is followed by preassembled completed prefab system (pcps) with a pooled composite priority of 0.145. this is closely followed by other prefab systems (ops) and volumetric prefab systems (vps), with pooled composite values of 0.138 and 0.137, respectively. ijahp article: dixon-ogbechi, adebayo /application of the ahp model to determine prefab housing adoption factors for developers in lagos state international journal of the analytic hierarchy process 306 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.635 5.3.3 sensitivity analysis a sensitivity analysis was done for all of the seven criteria, and the results revealed that the panelized prefab system (pps) and the hybrid prefab system (hps) were always the best. however, as the importance of cost effectiveness (ce) increases, the prefab systems became almost equally preferred with only the panelized prefab system (pps) remaining as the most preferred. figures 2 and 3 show the sensitivity graphs for two criteria. the screen shots of the sensitivity analysis graphs for all of the criteria are presented in appendix 3. figure 2 sensitivity graph for fpct (faster project completion time) criterion ijahp article: dixon-ogbechi, adebayo /application of the ahp model to determine prefab housing adoption factors for developers in lagos state international journal of the analytic hierarchy process 307 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.635 figure 3 sensitivity graph for ce (cost effectiveness) criterion 6. conclusions based on our findings, it can be concluded that the relative importance of factors that can promote the adoption of the prefab housing technology by developers in lagos state is as follows: faster project completion time (fpct 0.248) ranks first, greater quality control (gqc 0.202) ranks second, and other factors such as usage of specialized production labor (uspl 0.110), less site material (lsm 0.110), reduced site labor (rsl 0.110), minimal environmental impact (mei 0.110), and cost effectiveness (ce 0.110) all have an equal rank. also, the most adoptable prefab system by developers in lagos state is the panelized prefab system (pps). the strategy that will promote the adoption of prefab housing by developers in lagos state is to allocate 24.8% of the resources to ensure that the project is completed on time, 20.2% of resources should be used to ensure greater quality control and the remaining 55% of the resources should be shared equally (11% each) to ensure that specialized labor is used, less site materials are used, reduction in site labor, minimal environmental impact and cost effectiveness. the conclusions from our study further buttress the fact that the ahp methodology can be successfully applied to solve a wide variety of complex decision making problems in the built environment and other contexts. ijahp article: dixon-ogbechi, adebayo /application of the ahp model to determine prefab housing adoption factors for developers in lagos state international journal of the analytic hierarchy process 308 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.635 therefore, to promote the adoption of the prefab housing by developers in lagos state it is recommended that: a. the lagos state government should put in place adequate infrastructure to ensure that developers who adopt the prefab system can complete their projects on time. b. developers should invest in systems that will enable them to complete their projects on time and also have better quality control strategies. c. the lagos state government should encourage entrepreneurial development in panelized prefab system components building by putting in place relevant policies. d. the lagos state government should also develop an infrastructure that enables panelized prefab system developments both at the state and national levels. e. financial institutions should also make facilities available to encourage investment in the prefab housing industry. f. tertiary educational institutions in lagos state should include prefab housing systems in their architecture curriculum at the undergraduate and postgraduate levels. in addition, for future research, the scope of this study could be broadened by replicating it and performing a comparative study across all built environment professionals. ijahp article: dixon-ogbechi, adebayo /application of the ahp model to determine prefab housing adoption factors for developers in lagos state international journal of the analytic hierarchy process 309 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.635 references adebayo, a.k. & dixon-ogbechi, b.n. (2017a). developers’ perception of prefabricated housing methodology in nigeria: a study of lagos state, journal of strategic and international studies, xii(4), 54-65. adebayo, a.k. & dixon-ogbechi, b.n. (2017b). urban mass housing in lagos metropolis, nigeria – the potentials and marketability of prefabricated housing systems, journal of strategic and international studies, xii(1), 25-30. carr, a.l. (1947). a practical guide to prefabricated houses. new york: harper. graff, r.k. (1947) in johnson (2006). johnson e. cynthia (2006). house in a box: prefabricated housing in the jackson purchase cultural landscape region, 1900 to 1960. kentucky heritage council. retrieved from http://heritage.ky.gov/nr/rdonlyres/69811bb7-b64c-43e7-ac2b-c7a83390e 09d/0/houseinabox.pdf. johnson, e.c. (2006). house in a box: prefabricated housing in the jackson purchase cultural landscape region, 1900 to 1960. kentucky heritage council. retrieved from https://heritage.ky.gov/documents/houseinabox.pdf mcgraw hill construction (2011). prefabrication and modularization: increasing productivity in the construction industry – smartmarket report. retrieved from https://www.nist.gov/system/files/documents/el/economics/prefabricationmodularization-in-the-construction-industry-smr-2011r.pdf panjehpour, m. & abang ali, a. (2013). a review of prefab home and relevant issues. constructii, 1, 53-60. saaty, t.l. (1980). the analytic hierarchy process, planning, priority setting, resource allocation, new york: mcgraw-hill. saaty, t.l. (2001). decision making for leaders. (3 rd ed.). pittsburgh, pa: rws publications. smith, i., asiz, a. & gupta, g. (2007). high performance modular wood construction systems. final report no unb5. ottawa, ontario, canada: value-to-wood program, natural resources, 151. tam, r. (2013). analysis of growth opportunities of pre-fabricated housing in vietnam. a report submitted in partial fulfillment of the requirements for degree of bachelor of science in wood products processing. retrieved from https://pdfs.semanticscholar.org/fe4d/5c671f878e17bdb8b3127daa1a92f9b1d5c6.pdf vision 2020 national technical working group on housing (2009). retrieved 7/9/17 from http://www.ibenaija.org/uploads/1/0/1/2/10128027/housing_ntwg_report.pdf. http://heritage.ky.gov/nr/rdonlyres/69811bb7-b64c-43e7-ac2b-c7a83390e https://heritage.ky.gov/documents/houseinabox.pdf https://www.nist.gov/system/files/documents/el/economics/prefabrication-modularization-in-the-construction-industry-smr-2011r.pdf https://www.nist.gov/system/files/documents/el/economics/prefabrication-modularization-in-the-construction-industry-smr-2011r.pdf https://pdfs.semanticscholar.org/fe4d/5c671f878e17bdb8b3127daa1a92f9b1d5c6.pdf http://www.ibenaija.org/uploads/1/0/1/2/10128027/ ijahp article: dixon-ogbechi, adebayo /application of the ahp model to determine prefab housing adoption factors for developers in lagos state international journal of the analytic hierarchy process 310 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.635 wei p., gibb, a. & dainty, a. (2008). leading uk housebuilders' utilization of offsite construction methods, building research & information, 36, 56-67. doi: https://doi.org/10.1080/09613210701204013 http://www.tandfonline.com/author/gibb%2c+alistair+gf http://www.tandfonline.com/author/dainty%2c+andrew+rj ijahp article: dixon-ogbechi, adebayo /application of the ahp model to determine prefab housing adoption factors for developers in lagos state international journal of the analytic hierarchy process 311 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.635 appendix 1 questionnaire university of lagos akoka-yaba, lagos date: dear respondent, request for information we are academics at the university of lagos conducting a study on: determining a strategy for the adoption of prefabricated housing by developers in lagos state: an ahp approach. prefabrication is the production of building components or even the whole building in a specialized facility usually located off-site and later transported to the building site for installation. the ahp (analytic hierarchy process) is an operations research model that enables decision makers to rank items in order of relative importance, and it results in both quantitative and qualitative outcomes. this model is based on the following: goal = determine prefab housing adoption factors for developers in lagos state criteria = the most popular evaluative criteria for the adoption of the prefab system as identified in a study by adebayo and ogbechi (2017a) are: faster project completion time (fpct), greater quality control (gqc), usage of specialized production labor (uspl), less site material (lsm), reduced site labor (rsl), minimal environmental impact (mei), and cost effectiveness (ce). alternatives = the prefabricated housing systems that are marketable to consumers in lagos state as identified in previous studies by adebayo and dixon-ogbechi, 2017b) are: panelized prefab system (pps): these are prefab units produced in a factory and assembled on-site to produce a three dimensional structure. hybrid prefab system (hps): this is a combination of both the volumetric/pod and panel systems. other prefab systems (ops): these are also called casements. they are prefab units that include floor cassettes, roof cassettes, pre-cast concrete foundation assemblies, pre-formed wiring looms and mechanical engineering composites. preassembled completed prefab system (pcps): these are prefab systems in which the panels come with fittings and in some cases, other finishing like paint. they are assembled on site to form complete buildings. volumetric prefab system (vps): these are prefab houses in which the building is manufactured in units of three dimensional modules (pod) in controlled factory conditions prior to transport to site which include: toilet pod, kitchen pod, electrical pod, lift pod. this study is being undertaken for purely academic and applied research purposes. kindly assist by objectively completing the following questionnaire. the information obtained will be treated with utmost confidentiality. while thanking you in anticipation of your co-operation and sincere responses, we remain, yours faithfully, dixon-ogbechi, bolajoko n. (08077525263) adebayo anthony kayode (08096592878) ijahp article: dixon-ogbechi, adebayo /application of the ahp model to determine prefab housing adoption factors for developers in lagos state international journal of the analytic hierarchy process 312 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.635 research questionnaire part 1: respondents’ characteristics instruction: kindly tick (√) the appropriate option or fill in your response in the spaces provided. 1. sex: a. male ( ) b. female ( ) 2. age: a. below 20 years ( ) b. 20 to below 30 years ( ) c. 30 to below 40 years ( ) d. 40 to below 50 years ( ) e. 50 years and above ( ) 3. marital status: a. single ( ) b. married ( ) c. divorced ( ) d. separated ( ) e. widow/er ( ) f. others (please specify)………….… 4. religion: a. christian ( ) b. muslim ( ) c. others (please specify)………………... 5. highest educational qualification. a. wasc/gce/neco ( ) b. b.sc. or equivalent ( ) c. m.sc. ( ) d. ph.d. ( ) e. others ( ) (please specify)…………………………… 6. what staff strength (number of employees) does your organization have? ………………………………………………………………………………………………… ………………………………………………………………………………… 7. what is the volume (number of houses) your company builds per annum? a. less than 5 ( ) b. 5 to less than 10 ( ) c. 10 to less than 15 ( ) d. 15 to less than 20 ( ) e. 20 and above. ( ) 8. position held in your organization: …………………………………………………. 9. years of working experience: a. less than a year ( ) b. 1 to less than 5 years ( ) c. 5 to less than 10 years ( ) ijahp article: dixon-ogbechi, adebayo /application of the ahp model to determine prefab housing adoption factors for developers in lagos state international journal of the analytic hierarchy process 313 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.635 verbal scale & numerical values verbal scale & numerical values equally important = 1 equally important = 1 fairly moderately more important = 2 fairly moderately less important = -2 moderately more important = 3 moderately less important = -3 fairly strongly more important = 4 fairly strongly less important = -4 strongly more important = 5 strongly less important = -5 fairly very strongly more important = 6 fairly very strongly less important = -6 very strongly more important = 7 very strongly less important = -7 fairly extremely (absolutely) more important = 8 fairly extremely (absolutely) less important = -8 extremely (absolutely) more important = 9 extremely (absolutely) less important = -9 d. 10 years and above. ( ) part ii: data on relevant variables relative importance scaled questions the following questions are aimed at investigating the relative importance of factors that can promote the adoption of the prefab housing technology by developers in lagos state using the analytical hierarchy process. they require your input, and we sincerely desire your opinion in assigning values to the questions asked based on the explanation box below. explanation box note: when the elements are placed in matrix form, by convention the comparison of strength is always of an activity appearing in the column on the left against an activity appearing in the column on the right. criteria explanation the most popular evaluative criteria for the adoption of the prefab system as identified in a study by adebayo and ogbechi (2017a) are: fpct: faster project completion time; gqc: greater quality control. uspl: usage of specialized production labor; lsm: less site material. rsl: reduced site labor; ce: cost effectiveness mei: minimal environmental impact. alternatives (types of prefabricated housing systems) explanation the prefabricated housing systems that are marketable to consumers in lagos state as identified in previous studies by adebayo and dixon-ogbechi, 2017b) are: pps: panelized prefab system; hps: hybrid prefab system, ops: other prefab systems; pcps: preassembled completed prefab system vps: volumetric (pod) prefab system instruction: given the above explanations, kindly indicate the relative importance of the factors/systems listed below by kindly ticking (√) the value you choose in the empty space in each cell. ijahp article: dixon-ogbechi, adebayo /application of the ahp model to determine prefab housing adoption factors for developers in lagos state international journal of the analytic hierarchy process 314 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.635 a. prefab adoption factors for developers criteria with respect to goal instruction: kindly tick (√) your value on the relative importance of factors that can promote the adoption of the prefab housing technology by developers in lagos state by comparing the factors listed on the left side against the ones on the right side where 9.5 is the highest and -9.5 is the lowest as indicated on the explanation box given earlier. >= 9.5 9 8 7 6 5 4 3 2 1 -2 -3 -4 -5 -6 -7 -8 -9 >=9.5 fpct gqc fpct uspl fpct lsm fpct rsl fpct mei fpct ce gqc uspl gqc lsm gqc rsl gqc mei gqc ce uspl lsm uspl rsl uspl mei uspl ce lsm rsl lsm mei lsm ce rsl mei rsl ce mei ce ijahp article: dixon-ogbechi, adebayo /application of the ahp model to determine prefab housing adoption factors for developers in lagos state international journal of the analytic hierarchy process 315 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.635 b. prefab adoption factors for developers criteria with respect wrt faster project completion time (fpct) kindly tick (√) your rating of the importance of faster project completion time (fpct) in the selection of prefab housing system in lagos state by comparing the prefab housing system listed on the left side against the ones on the right side where 9.5 is the highest and -9.5 is the lowest as indicated on the explanation box given earlier. >= 9.5 9 8 7 6 5 4 3 2 1 -2 -3 -4 -5 -6 -7 -8 -9 >=9.5 pps hps pps ops pps pcps pps vps hps ops hps pcps hps vps ops pcps ops vps pcps vps c. prefab adoption factors for developers criteria with respect wrt greater quality control (gqc) kindly tick (√) your rating of the importance of greater quality control (gqc) in the selection of prefab housing system in lagos state by comparing the prefab housing system listed on the left side against the ones on the right side where 9.5 is the highest and -9.5 is the lowest as indicated on the explanation box given earlier. >= 9.5 9 8 7 6 5 4 3 2 1 -2 -3 -4 -5 -6 -7 -8 -9 >=9.5 pps hps pps ops pps pcps pps vps hps ops hps pcps hps vps ops pcps ops vps pcps vps ijahp article: dixon-ogbechi, adebayo /application of the ahp model to determine prefab housing adoption factors for developers in lagos state international journal of the analytic hierarchy process 316 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.635 d. prefab adoption factors for developers criteria with respect wrt usage of specialised production labour (uspl) kindly tick (√) your rating of the importance of usage of specialized production labor (uspl) in the selection of prefab housing system in lagos state by comparing the prefab housing system listed on the left side against the ones on the right side where 9.5 is the highest and -9.5 is the lowest as indicated on the explanation box given earlier. >= 9.5 9 8 7 6 5 4 3 2 1 -2 -3 -4 -5 -6 -7 -8 -9 >=9.5 pps hps pps ops pps pcps pps vps hps ops hps pcps hps vps ops pcps ops vps pcps vps e. prefab adoption factors for developers criteria with respect wrt less site material (lsm) kindly tick (√) your rating of the importance of less site material (lsm) in the selection of prefab housing system in lagos state by comparing the prefab housing system listed on the left side against the ones on the right side where 9.5 is the highest and -9.5 is the lowest as indicated on the explanation box given earlier. >= 9.5 9 8 7 6 5 4 3 2 1 -2 -3 -4 -5 -6 -7 -8 -9 >=9.5 pps hps pps ops pps pcps pps vps hps ops hps pcps hps vps ops pcps ops vps pcps vps ijahp article: dixon-ogbechi, adebayo /application of the ahp model to determine prefab housing adoption factors for developers in lagos state international journal of the analytic hierarchy process 317 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.635 f. prefab adoption factors for developers criteria with respect wrt reduced site labour (rsl) kindly tick (√) your rating of the importance of reduced site labor (rsl) in the selection of prefab housing system in lagos state by comparing the prefab housing system listed on the left side against the ones on the right side where 9.5 is the highest and -9.5 is the lowest as indicated on the explanation box given earlier. >= 9.5 9 8 7 6 5 4 3 2 1 -2 -3 -4 -5 -6 -7 -8 -9 >=9.5 pps hps pps ops pps pcps pps vps hps ops hps pcps hps vps ops pcps ops vps pcps vps g. prefab adoption factors for developers criteria with respect wrt minimal environmental impact (mei) kindly tick (√) your rating of the importance of minimal environmental impact (mei) in the selection of prefab housing system in lagos state by comparing the prefab housing system listed on the left side against the ones on the right side where 9.5 is the highest and -9.5 is the lowest as indicated on the explanation box given earlier. >= 9.5 9 8 7 6 5 4 3 2 1 -2 -3 -4 -5 -6 -7 -8 -9 >=9.5 pps hps pps ops pps pcps pps vps hps ops hps pcps hps vps ops pcps ops vps pcps vps ijahp article: dixon-ogbechi, adebayo /application of the ahp model to determine prefab housing adoption factors for developers in lagos state international journal of the analytic hierarchy process 318 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.635 h. prefab adoption factors for developers criteria with respect wrt cost effectiveness (ce) kindly tick (√) your rating of the importance of cost effectiveness (ce) in the selection of prefab housing system in lagos state by comparing the prefab housing system listed on the left side against the ones on the right side where 9.5 is the highest and -9.5 is the lowest as indicated on the explanation box given earlier. >= 9.5 9 8 7 6 5 4 3 2 1 -2 -3 -4 -5 -6 -7 -8 -9 >=9.5 pps hps pps ops pps pcps pps vps hps ops hps pcps hps vps ops pcps ops vps pcps vps thank you. ijahp article: dixon-ogbechi, adebayo /application of the ahp model to determine prefab housing adoption factors for developers in lagos state international journal of the analytic hierarchy process 319 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.635 appendix 2 screenshots of superdecisions software analysis ijahp article: dixon-ogbechi, adebayo /application of the ahp model to determine prefab housing adoption factors for developers in lagos state international journal of the analytic hierarchy process 320 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.635 ijahp article: dixon-ogbechi, adebayo /application of the ahp model to determine prefab housing adoption factors for developers in lagos state international journal of the analytic hierarchy process 321 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.635 ijahp article: dixon-ogbechi, adebayo /application of the ahp model to determine prefab housing adoption factors for developers in lagos state international journal of the analytic hierarchy process 322 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.635 ijahp article: dixon-ogbechi, adebayo /application of the ahp model to determine prefab housing adoption factors for developers in lagos state international journal of the analytic hierarchy process 323 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.635 ijahp article: dixon-ogbechi, adebayo /application of the ahp model to determine prefab housing adoption factors for developers in lagos state international journal of the analytic hierarchy process 324 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.635 appendix 3 sensitivity graphs 1. fpct 2. gqc ijahp article: dixon-ogbechi, adebayo /application of the ahp model to determine prefab housing adoption factors for developers in lagos state international journal of the analytic hierarchy process 325 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.635 3. uspl 4. lsm ijahp article: dixon-ogbechi, adebayo /application of the ahp model to determine prefab housing adoption factors for developers in lagos state international journal of the analytic hierarchy process 326 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.635 5. rsl 6. mei ijahp article: dixon-ogbechi, adebayo /application of the ahp model to determine prefab housing adoption factors for developers in lagos state international journal of the analytic hierarchy process 327 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.635 7. ce a style guide for papers presented to the international journal of the analytic hierarchy process ijahp article: lipovetsky/the supermatrix eigenproblem: an interpretation of the priority vectors in the analytic network process international journal of the analytic hierarchy process 105 vol.5 issue 1 2013 issn 1936-6744 the supermatrix eigenproblem: an interpretation of the priority vectors in the analytic network process stan lipovetsky gfk custom research north america minneapolis, mn, usa e-mail: stan.lipovetsky@gfk.com abstract continuing the previous articles on interpretation of the solutions obtained in the analytic hierarchy process (ahp) and in the analytic network process (anp) with outer dependence, the current work considers a general anp problem and shows that the solution for its supermatrix, obtained by raising the supermatrix to powers is equivalent to solving the eigenproblem for this matrix. thus, the global priority solution for an anp model is an eigenproblem of its supermatrix, and the principal eigenvector of the supermatrix itself gives the mean priorities for the whole network of the compared items. this approach provides an easy way to describe complex anp interconnections, and gives an explicit interpretation of the priority results convenient for practical managerial decisions. keywords: analytic network process, supermatrix eigenvectors. do not become the slave of your model – vincent van gogh 1. introduction this paper continues the consideration of the relationship between the analytic hierarchy process (ahp) and the analytic network process (anp) started in three previous papers in this journal (lipovetsky, 2010, 2011a, b). in an anp problem there are two kinds of matrices to consider: the pairwise comparison matrices and the supermatrix created by combining the eigenvectors from all the pairwise comparison matrices. in the current article, we consider a general anp problem and show that its solution can be obtained by raising its supermatrix to powers or by solving the eigenproblem for its supermatrix. the solution of an anp problem can be obtained by solving the ejgenproblem for the supermatrix, and its principal eigenvector presents the mean priorities for the elements in the whole network. the eigenproblem approach offers a way to easily describe the complex anp interconnections and explicitly interpret the priority results in a way that is convenient for practical managerial decisions. mailto:stan.lipovetsky@gfk.com rob typewritten text http://dx.doi.org/10.13033/ijahp.v5i1.132 ijahp article: lipovetsky/the supermatrix eigenproblem: an interpretation of the priority vectors in the analytic network process international journal of the analytic hierarchy process 106 vol.5 issue 1 2013 issn 1936-6744 2. anp eigenproblem solution and interpretation for purposes of this exposition, let us consider an example of a general supermatrix for a complicated network with a holarchy structure for estimating the turnaround of the us economy in 1992, described by saaty (1996, pp.171-184). this holarchy has links from the primary factors to the subcriteria to the alternatives, and the alternatives are linked back to the primary factors. in this example, the alternatives represent time periods during which the turnaround will have occurred as measured from the time the prediction was made, and their priorities represent the likelihood that the turnaround happened during that time period. there are 15 nodes in the network structure, grouped into several blocks as listed below. a visual representation of the network containing three clusters of nodes is shown in figure 1 and the derived priorities resulting from the pairwise comparison matrices are shown in the columns in table 1. primary factors 1) conventional adjustment, 2) economic restructuring, subfactors conventional adjustment 3) consumption 4) export 5) investment 6) confidence 7) fiscal policy 8) monetary policy economic restructuring 9) financial sector 10) defense posture 11) global competition, alternatives 12) three months of adjustment 13) six months of adjustment 14) one year of adjustment 15) two years and more of adjustment the colored windows in figure 1 represent the clusters that contain the nodes that are the factors in the problem. clusters may be thought of as logical collections of issues being considered in the decision problem. the arrow from one cluster to another merely indicates that some node(s) in the beginning cluster are linked to some node(s) in the terminal cluster. there are no actual links connecting clusters; links go from node to node(s). the actual details of the connections can be seen in table 1. the blocks of entries in table 1 are color-coded to match the clusters in figure 1. in table 1 the parent node of a comparison set is listed at the top of the column and its children nodes are listed in the rows. children nodes for a given comparison set must all ijahp article: lipovetsky/the supermatrix eigenproblem: an interpretation of the priority vectors in the analytic network process international journal of the analytic hierarchy process 107 vol.5 issue 1 2013 issn 1936-6744 be in the same cluster. for example, the conventional adjustment node number 1, at the top of the table, is the parent of a comparison set and its children are the nodes in rows 3 to 8: consumption, exports, investment, confidence, fiscal policy and monetary policy. the numbers in column1 are the local priority vector obtained from the pairwise comparison matrix. the question is posed like this: which factor (child node) of conventional adjustment is more important in causing an economic turnaround? figure 1 us economy holarchy of factors (saaty, 1996)∗ the column-stochastic supermatrix containing all the local priority vectors obtained from pairwise comparison matrices in the network are in the columns in table 1. the background color of the clusters in figure 1 is the same color as the associated cells in table 1. the supermatrix in table 1 is a square matrix with a row and column for each of the nodes in figure 1. raising a supermatrix s to powers is the simplest way to find its principal eigenvector and eigenvalue. usually it converges after a few iterations. sometimes, in cases of circular priorities insolvency occurs (bar niv and lipovetsky, 1995) as the matrix cycles among several solutions; however, in this case it does not. the maximum eigenvalue of a ∗ the screenshot is of the network if from the superdecisions software, available free to educators and researchers from www.superdecisions.com. ijahp article: lipovetsky/the supermatrix eigenproblem: an interpretation of the priority vectors in the analytic network process international journal of the analytic hierarchy process 108 vol.5 issue 1 2013 issn 1936-6744 column-stochastic matrix is 1=λ ; its left eigenvector is the identity vector, and raising it to powers until it reaches a stable state leads to the limiting matrix shown in table 2. each column in the limiting matrix is the same, the normalized limiting vector w, obtained by raising the supermatrix to powers. table 1 supermatrix of local priority vectors for “turnaround in the us economy” (saaty, 1996) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 1 0 0 0 0 0 0 0 0 0 0 0 .833 .833 .500 .167 2 0 0 0 0 0 0 0 0 0 0 0 .167 .167 .500 .833 3 .118 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4 .029 0 0 0 0 0 0 0 0 0 0 0 0 0 0 5 .058 0 0 0 0 0 0 0 0 0 0 0 0 0 0 6 .334 0 0 0 0 0 0 0 0 0 0 0 0 0 0 7 .118 0 0 0 0 0 0 0 0 0 0 0 0 0 0 8 .343 0 0 0 0 0 0 0 0 0 0 0 0 0 0 9 0 .584 0 0 0 0 0 0 0 0 0 0 0 0 0 10 0 .281 0 0 0 0 0 0 0 0 0 0 0 0 0 11 0 .135 0 0 0 0 0 0 0 0 0 0 0 0 0 12 0 0 .043 .083 .078 .517 .099 .605 .049 .049 .089 0 0 0 0 13 0 0 .113 .083 .078 .305 .088 .262 .085 .085 .089 0 0 0 0 14 0 0 .310 .417 .305 .124 .383 .042 .236 .236 .209 0 0 0 0 15 0 0 .534 .417 .539 .054 .432 .091 .630 .630 .613 0 0 0 0 the matrix in table 2 can be represented by the following outer product: ( )1....11... 1 ... ........................... ... ... 2 1 222 111               =             nnnn w w w www www www (1) this is the eigenvector decomposition of the rank one matrix. the vector w in table 1 is constructed of an element from each row of the limiting matrix, shown in table 2, and is the right eigenvector of the original supermatrix. the vector w can be obtained by solving the eigenproblem of the supermatrix: wws λ= (2) ijahp article: lipovetsky/the supermatrix eigenproblem: an interpretation of the priority vectors in the analytic network process international journal of the analytic hierarchy process 109 vol.5 issue 1 2013 issn 1936-6744 table 2 limiting supermatrix for the 15 elements in “turnaround in the us economy” example 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 1 .161 .161 .161 .161 .161 .161 .161 .161 .161 .161 .161 .161 .161 .161 .161 2 .172 .172 .172 .172 .172 .172 .172 .172 .172 .172 .172 .172 .172 .172 .172 3 .019 .019 .019 .019 .019 .019 .019 .019 .019 .019 .019 .019 .019 .019 .019 4 .005 .005 .005 .005 .005 .005 .005 .005 .005 .005 .005 .005 .005 .005 .005 5 .009 .009 .009 .009 .009 .009 .009 .009 .009 .009 .009 .009 .009 .009 .009 6 .054 .054 .054 .054 .054 .054 .054 .054 .054 .054 .054 .054 .054 .054 .054 7 .019 .019 .019 .019 .019 .019 .019 .019 .019 .019 .019 .019 .019 .019 .019 8 .055 .055 .055 .055 .055 .055 .055 .055 .055 .055 .055 .055 .055 .055 .055 9 .101 .101 .101 .101 .101 .101 .101 .101 .101 .101 .101 .101 .101 .101 .101 10 .048 .048 .048 .048 .048 .048 .048 .048 .048 .048 .048 .048 .048 .048 .048 11 .023 .023 .023 .023 .023 .023 .023 .023 .023 .023 .023 .023 .023 .023 .023 12 .075 .075 .075 .075 .075 .075 .075 .075 .075 .075 .075 .075 .075 .075 .075 13 .051 .051 .051 .051 .051 .051 .051 .051 .051 .051 .051 .051 .051 .051 .051 14 .067 .067 .067 .067 .067 .067 .067 .067 .067 .067 .067 .067 .067 .067 .067 15 .141 .141 .141 .141 .141 .141 .141 .141 .141 .141 .141 .141 .141 .141 .141 the results of the eigenproblem solution for the 15th order supermatrix v in table 1 are given in table 3. the principal eigenvector corresponding to the maximum real eigenvalue 1=λ is given in column 2 of table 3 and shown normalized to 1 in column 3. the normalized principal eigenvector in column 3 is the same as the vector obtained by raising the supermatrix to powers (as shown by any one of the columns of table 2). but the eigenproblem solution for the supermatrix reveals more fascinating features. the next two eigenvalues, i866.0500.0 ±−=λ , shown in the bottom row of table 3, are complex conjugates, but the modulo (absolute value) of each equals one, that is 1866.05.0|| 22 =+=λ . the modulus values obtained from the pairs in column 4 are given in column 5 and normalized to 1 in column 6. the values in columns 6 from the complex conjugate eigenvectors are the same as the normalized eigenvector in column 3. so we have shown that all three of these eigenvectors can be reduced to the same unique solution coinciding with the priorities in table 2. in column 7 the values in each cluster are normalized to 1. ijahp article: lipovetsky/the supermatrix eigenproblem: an interpretation of the priority vectors in the analytic network process international journal of the analytic hierarchy process 110 vol.5 issue 1 2013 issn 1936-6744 table 3 eigenvectors of the supermatrix for the “turnaround in the us economy” example elements principal eigenvector of supermatrix in table 1 second and third complex conjugated eigenvectors of supermatrix priorities (from table 2) eigenvector normalized to total =1 (priorities) original vectors modulo modulo normalized to total =1 (priorities) priorities are normalized to 1within each block 1 .565 .161 .594 ± .544i .806 .161 .484 2 .603 .172 .634 ± .581i .860 .172 .516 3 .067 .019 -.091 ± .029i .095 .019 .057 4 .016 .005 -.022 ± .007i .023 .005 .014 5 .033 .009 -.045 ± .014i .047 .009 .028 6 .189 .054 -.257 ± .081i .269 .054 .162 7 .067 .019 -.091 ± .029i .095 .019 .057 8 .194 .055 -.264 ± .083i .276 .055 .166 9 .352 .100 -.479 ± .151i .502 .100 .301 10 .169 .048 -.230 ± .073i .242 .048 .145 11 .081 .023 -.111 ± .035i .116 .023 .070 12 .261 .074 .081  .363i .372 .074 .223 13 .177 .051 .055  .247i .253 .051 .152 14 .235 .067 .072  .327i .335 .067 .201 15 .495 .141 .153  .689i .706 .141 .424 eigen value 1 -.500 ± .866i to summarize, the supermatrix contains three non-zero blocks shown in table 1. the variables in these blocks are the priorities obtained from pairwise comparison matrices by raising them to powers (saaty, 1996, p.180-182). let us take the real or imaginary part of the second or third original eigenvector and normalize them separately for the three blocks, the first having two elements, the next nine elements, and the last four elements. these normalized-by-block values are in the last column in table 3. for the first block, for instance, the sum 0.484+0.516=1, the sum of the next nine elements in rows 3-11 is one, and the sum of the last four elements in the rows 12-15 is one as well. but normalizing the entire vector so it sums to one, we obtain the vector presented in the second from the last column of table 3and this vector coincides with the principal eigenvector from table 2. thus, all three eigenvectors (corresponding to modulo one eigenvalues) give the same solution which is the same as that obtained by raising the matrix to powers as shown in table 2. by defining the solution of a supermatrix in terms of its principal eigenvector we can easily and explicitly interpret the priority vector elements. the elements of an eigenvector correspond to the mean preferences of each alternative, or criterion, etc., over all the other compared items. global vectors in anp are the eigenvectors of the ijahp article: lipovetsky/the supermatrix eigenproblem: an interpretation of the priority vectors in the analytic network process international journal of the analytic hierarchy process 111 vol.5 issue 1 2013 issn 1936-6744 combined local eigenvectors, and they represent the mean priorities of the compared entries in the whole network. anp can also be described as a stochastic model consisting of the priority vectors for each element of the network. a supermatrix is columnstochastic, so totals in each column equal one, and the elements of supermatrix sij are already the eigenvector preferences of each item over the others in each block of the whole network structure. it means that the transposed stochastic supermatrix itself can be considered as a matrix of transitional probabilities pi and described in a system of chapman-kolmogorov equations:        −++= −−−−−−−−−− −++= −− nnnnn n nn ppsps dt dp ppsps dt dp 11,11 11212 1 ... ... (3) these equations are written in matrix form as follows: ppsp −= (4) for the steady-state priorities it reduces to the eigenproblem of the supermatrix: pps λ= , (5) in which the maximum eigenvalue of the column-stochastic supermatrix is 1=λ , which is associated with a unique positive eigenvector (bar niv and lipovetsky, 1995; lipovetsky and conklin, 2003; lipovetsky, 2005). as shown above, the solution by raising the supermatrix to powers coincides with its principal eigenvector obtained in the eigenproblem equation (2). thus, the anp priority vectors can be described in terms of the probability of choice among all compared items. it is important to know that dynamic equations for a time dependent supermatrix was introduced originally by saaty (1994, ch. 12). 3. summary together with the previous articles (lipovetsky, 2010, 2011a, b) this work has shown that ahp/anp solutions for local and global priority vectors can be derived from the standard framework of an n × n matrix and solved in terms of its principal eigenvector. the elements of an eigenvector correspond to the mean preferences of each alternative, or criterion, over all the other compared items. global vectors in ahp and anp are the eigenvectors of the combined local eigenvectors, and they represent the mean priorities of the compared entries in the whole network. this interpretation of ahp and anp priority vectors as the mean prevalence among the compared elements can be easily accepted by managers and decision makers. ijahp article: lipovetsky/the supermatrix eigenproblem: an interpretation of the priority vectors in the analytic network process international journal of the analytic hierarchy process 112 vol.5 issue 1 2013 issn 1936-6744 references bar niv, m., and lipovetsky, s. (1995). deciding circular priorities in insolvency situations, international journal of operations and quantitative management, 1, 183-195. lipovetsky, s. and conklin, m. (2003). priority estimations by pair comparisons: ahp, thurstone scaling, bradley-terry-luce, and markov stochastic modeling. proceedings of the joint statistical meeting, the american statistical association, august 2003, san francisco, ca, 2473-2478. lipovetsky, s. (2005). analytic hierarchy processing in chapman-kolmogorov equations. international journal of operations and quantitative management, 11, 219-228. lipovetsky, s. (2010). an interpretation of the ahp eigenvector solution for the lay person. international journal of the analytic hierarchy process, 2, 158-162. lipovetsky. s. (2011a). an interpretation of the ahp global priority as the eigenvector solution of an anp supermatrix. international journal of the analytic hierarchy process, 3, 70-78. lipovetsky. s. (2011b). priority eigenvectors in analytic hierarchy/network processes with outer dependence between alternatives and criteria. international journal of the analytic hierarchy process, 3, 172-179. saaty, t.l. (1994). fundamentals of decision making and priority theory with the analytic hierarchy process. pittsburgh, pa: rws publications. saaty, t.l. (1996). decision making with dependence and feedback: the analytic network process. pittsburgh, pa: rws publications. appendix i. additional bibliography gogh, vincent van, the complete letters, published by the van gogh museum, letter to his brother theo, 1882. lipovetsky, s. (1996). the synthetic hierarchy method: an optimizing approach to obtaining priorities in the ahp. european journal of operational research, 93, 550 564. lipovetsky, s., tishler, a., dvir, d., & shenhar, a. (1997). the relative importance of project success dimensions. r&d management, 27, 97-106. lipovetsky, s., & tishler, a. (1999). interval estimation of priorities in the ahp. european journal of operational research, 114, 153-164. lipovetsky, s., & lootsma f. a. (2000). generalized golden section, repeated bisections, and aesthetic preference. european journal of operational research, 121, 213-216. lipovetsky, s., & conklin, m. (2001). dual priority-antipriority thurstone scales as ahp eigenvectors. engineering simulation, 18, 631-648. ijahp article: lipovetsky/the supermatrix eigenproblem: an interpretation of the priority vectors in the analytic network process international journal of the analytic hierarchy process 113 vol.5 issue 1 2013 issn 1936-6744 lipovetsky, s., & conklin, m. (2002). robust estimation of priorities in the ahp. european journal of operational research, 137, 110-122. lipovetsky, s. (2008a). bradley–terry choice probability in maximum likelihood and eigenproblem solutions. international j. of information technology & decision making, 7, 395-405. lipovetsky, s. (2008b). comparison among different patterns of priority vectors estimation methods. international j. of mathematical education in science and technology, 39, 301-311. lipovetsky, s. (2009a). global priority estimation in multiperson decision making. journal of optimization theory and applications, 140, 77-91. lipovetsky, s. (2009b). optimal hierarchy structures for multi-attribute-criteria decisions. journal of systems science and complexity, 22, 228-242. magnan, s., lipovetsky, s. (2010). key driver analysis with very small samples using the analytic hierarchy process method. joint statistical meeting, the american statistical association, august 2010, vancouver, bc, canada. saaty, t.l. (1977). a scaling method for priorities in hierarchical structures. mathematical psychology, 15, 234-281. saaty, t.l. (1980). the analytic hierarchy process. new york: mcgraw-hill. saaty, t.l. (2000). decision making for leaders. pittsburgh, pa: rws publications. saaty, t.l. (2010). principia mathematica decernendi: mathematical principles of decision making: generalization of the analytic network process to neural firing and synthesis. pittsburgh, pa: rws publications. saaty, t.l., & kearns, k.p. (1985). analytical planning. new york: pergamon. saaty, t.l., & vargas, l.g. (1994). decision making in economic, political, social and technological environment with the analytic hierarchy process. pittsburgh, pa:rws publications. wasil, e.a., & golden, b.l. (2003). celebrating 25 years of ahp-based decision making. computers and operations research, 30, 1419-1420. whitaker, r. (2007). validation examples of the analytic hierarchy process and analytic network process. mathematical and computer modelling, 46(7-8), 840-859. whitaker, r., (2007). criticisms of the analytic hierarchy process: why they often make no sense. mathematical and computer modelling, 46(7-8), 948-961. the supermatrix eigenproblem: an interpretation of the priority vectors in the analytic network process 1. introduction 3. summary ijahp article: raton, raco, krejci, ohoitimur, soputan, tumewu, korompis, taroreh, rachmadi, ngenget /soft skills of higher education in industry 4.0 era using buckley’s fuzzy-ahp international journal of the analytic hierarchy process 1 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.943 soft skills of higher education in industry 4.0 era using buckley’s fuzzy-ahp jozef richard raco management department, universitas katolik de la salle manado, indonesia jraco@unikadelasalle.ac.id yulius christian raton* industrial engineering department, universitas katolik de la salle manado, indonesia yraton@unikadelasalle.ac.id james v. krejci lewis university, illinois, usacolle of business krejcija@lewisu.edu johanis ohoitimur philosophy department, sekolah tinggi filsafat seminari pineleng, indonesia johoitimur@unikadelasalle.ac.id jeanette soputan faculty of animal husbandry, universitas sam ratulangi manado, indonesia jeanette@unsrat.ac.id tryadi wilhelmus tumewu industrial engineering department, universitas katolik de la salle manado, indonesia ttumewu@unikadelasalle.ac.id merry jeanned’arc korompis management department, universitas katolik de la salle manado, indonesia mkorompis@unikadelasalle.ac.id frankie j. h. taroreh accounting department, universitas katolik de la salle manado, indonesia ftaroreh@unikadelasalle.ac.ied ronald a. rachmadi industrial engineering department, universitas katolik de la salle manado, indonesia rrachmadi@unikadelasalle.ac.id stevanus ngenget hospitality and tourism department, universitas katolik de la salle manado, indonesia sngenget@unikadelasalle.ac.id * corresponding author mailto:jraco@unikadelasalle.ac.id mailto:yraton@unikadelasalle.ac.id mailto:krejcija@lewisu.edu mailto:johoitimur@unikadelasalle.ac.id mailto:jeanette@unsrat.ac.id mailto:ttumewu@unikadelasalle.ac.id mailto:mkorompis@unikadelasalle.ac.id mailto:ftaroreh@unikadelasalle.ac.ied mailto:rrachmadi@unikadelasalle.ac.id mailto:sngenget@unikadelasalle.ac.id ijahp article: raton, raco, krejci, ohoitimur, soputan, tumewu, korompis, taroreh, rachmadi, ngenget /soft skills of higher education in industry 4.0 era using buckley’s fuzzy-ahp international journal of the analytic hierarchy process 2 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.943 abstract industry 4.0 is characterized by the digitalization of systems and processes in service and manufacturing industries and has changed the way people live. education plays a significant role in preparing the future workforce with the necessary technological skills and competencies required by industries and institutions. studies have shown that soft skills improve a student’s ability to learn, increase their potential for success, and typically increase future economic benefits. this study aims to determine the dominant soft skills that university students in manado should possess. the perceptions of twentyfour lecturers about four criteria and twelve sub-criteria were compared using both the analytic hierarchy process (ahp) and fuzzy analytical hierarchy process (f-ahp) methods. from this, the researchers found teamwork to be the dominant skill (26%). global analysis uncovered that integrity was the dominant factor overall (10.5% with ahp or 10.3% with f-ahp). the findings were provided to university leaders with recommendations to incorporate the elements of teamwork and integrity into their teaching materials, teaching methods, and curriculum. students need to understand that these elements are essential to their future. this research proved that both the ahp and fuzzy-ahp methods were effective tools in analyzing and determining the dominant factors of soft skills in the industry 4.0 era. this research contributes to determining the priority factors related to soft skills needed by higher education graduates in the industry 4.0 era using a combination of ahp and fuzzy-ahp. the researchers recommended that other scholars conduct future studies using entrepreneurs or business practitioners as respondents. keywords: ahp; fuzzy-ahp; industry 4.0; soft skills; la salle; higher-education; sensitivity analysis 1. introduction since the term industry 4.0 appeared in 2011, many studies have emerged discussing this topic as it relates to different fields (meindl et al., 2021). the fast-paced advancement of technology exhibited by industry 4.0 has significantly changed the environment in which we live by improving the connectivity between humans, machines, and other objects (dombrowski, wullbrandt, & fochler, 2019a). real-time data is now available, globally, and to everyone online, often in excessive quantities. as a result, significant changes to the entire industrial system will be required. change will also be required in the world of education. pedagogy, teaching philosophy, educational models, and learning methods will require that the exchange and transfer of information and knowledge become faster and more efficient, accessible and flexible (miranda et al., 2021). innovation in the field of education will improve the teaching and learning process. distance learning, for example, is becoming more prevalent due to advances in connectivity, digitization and virtual platforms. these changes will require that the educational sector, particularly higher education institutions, identify, rethink and address the skills and competencies required for future employees and entrepreneurs. it is through education that students will prepare to take ijahp article: raton, raco, krejci, ohoitimur, soputan, tumewu, korompis, taroreh, rachmadi, ngenget /soft skills of higher education in industry 4.0 era using buckley’s fuzzy-ahp international journal of the analytic hierarchy process 3 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.943 advantage of the opportunities as well as the challenges of industry 4.0. it is predicted that industries and companies will be looking for employees who have a high degree of technological-based expertise. to meet these needs, higher education will need to redefine itself, develop its systems, improve its internal management, and enhance its networking. industry 4.0 companies will still require people who have hard skills, but there will be an increasing need for a workforce with soft skills, or non-technical skills, such as teamwork, critical thinking, communication, systems thinking, and emotional intelligence to truly take advantage of these process improvements. (fitsilis, tsoutsa, & gerogiannis, 2018). this study focuses on analysis and determination of the dominant soft skills that graduates of higher education will need to thrive in industry 4.0. the primary research question is as follows: using ahp and fuzzy-ahp, what are the dominant soft skills and cognitive skills that will enable students to become lifelong learners? secondarily, the researchers seek to determine if there is a significant difference between using the ahp and fuzzy-ahp for data analysis. the goal will be to determine what skills graduates of higher education should possess to thrive in industry 4.0. the study uses the analytic hierarchy process (ahp) and fuzzy-ahp methods to evaluate the perceptions of the lecturers collected through a questionnaire. both methods have been effectively used in decision making studies with complex and multiple variables. using a hierarchical structure, the ahp is able to simplify the analysis of the problem making it easier to understand. fuzzy-ahp allows researchers to deal with vague and uncertain perceptions, commonly referred to the “gray area”. since there is minimal statistical data available for analysis, both methods are used to evaluate the experts’ opinions. the respondents of this study are university lecturers who each have more than twenty years of experience and doctoral degrees. the respondents understand the context of the educational system of the universities in manado and are involved in student activities, comprehend the current situation of the university and meet the essential criteria to be considered as respondents to the questionnaire (raco & tanod, 2014). the researchers acknowledge that there are a number of articles, studies, discourses and commentaries regarding industry 4.0. however, the existing literature focuses on descriptive, assumptive and qualitative analyses that are theoretical in nature and do not adequately consider the necessary skill sets for worker employability (azmi, et al., 2018). the experts of this study identified four criteria and twelve sub-criteria that had been used in previous studies for the research analysis. the criteria include communication skills, teamwork, critical thinking, and entrepreneurial skills. the research findings will be used to enhance the management of higher education in manado. for the universitas katolik de la salle (de la salle catholic university) of manado-indonesia, the results of the study will be considered as key inputs in curriculum review, reformulation of the teaching-learning systems, and processes of the school. the findings will improve the school facilities, and cooperation and networking with other schools or industries. ijahp article: raton, raco, krejci, ohoitimur, soputan, tumewu, korompis, taroreh, rachmadi, ngenget /soft skills of higher education in industry 4.0 era using buckley’s fuzzy-ahp international journal of the analytic hierarchy process 4 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.943 the structure of the study is as follows. first, the background of the study, problem formulation, objectives and limitations were developed. second a literature review was conducted to identify findings and theories of previous studies on industry 4.0 and its impact on higher education. third, the methodologies were reviewed, the reasons for using each methodology were identified, and the benefits and drawbacks of each methodology were explored. fourth, a research questionnaire was developed. fifth, the data was evaluated. sixth, the meaning and significance of the results as well as the limitations were discussed. finally, the conclusions and recommendations were prepared. 2. literature review 2.1 characteristics of industry 4.0 industry 4.0 is often associated with the intelligent, digital integration of people-machineobjects, advanced computing power, augmented reality, big data analysis, horizontal and vertical system integration, autonomous robots, internet of things, cloud computing, and cyber-physical systems for management of business process and value creating networks (dombrowski, wullbrandt, & fochler, 2019b). it serves to integrate intelligent machines, human actors, physical objects, manufacturing lines and processes into every organizational level to create systematic technical data in near real-time. new technologies are developing at an exponential rate. a beginning to the revolution cannot be identified, rather it has had an evolutionary growth (hussin, 2018). industry 1.0 was characterized by the use of mechanical production assets based on water and steam power, then expanded to industry 2.0, which was identified by the introduction of mass production techniques centered on the division of labor and the use of electrical energy. industry 3.0 focused on the introduction of information technology and highly automated production. industry 4.0 is identified by self-optimizing and real-time connectivity of systems. (aulbur, arvind, & bigghe, 2016). technology will continue to develop and result in new products and services that cause disruption to the workplace and workforce which require new skills and competencies (aulbur et al., 2016). the emergence of organizational supply chains resulting in a change from a linear and sequential model to an interconnected, open system, known as a digital supply network will require a new organizational structure and employees with a new skill set to manage them. this digitalization of the integration of vertical and horizontal value-added steps in the supply chain allows the optimization of customer integration and data access resulting in increased productivity. smart factories using smart devices are able to self-optimize production and therefore increase productivity (fitsilis et al., 2018). digitalization reduces waste and promotes a circular economy and more sustainable patterns of production and consumption (paravizo, chaim, braatz, muschard, & rozenveld, 2018). additionally, customization increases the creation of flexible markets that are customer-oriented and can satisfy consumers’ needs faster since the gap between the manufacturer and the customer is significantly reduced. communication will take place seamlessly and require no intermediaries resulting in faster delivery of products. industry 4.0 will create new markets such as industrial robotics design, build and installation, cyber security, internet of things, and 3d printing. ijahp article: raton, raco, krejci, ohoitimur, soputan, tumewu, korompis, taroreh, rachmadi, ngenget /soft skills of higher education in industry 4.0 era using buckley’s fuzzy-ahp international journal of the analytic hierarchy process 5 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.943 in 2016, these markets were valued at $66.67 billion us, and by 2022, they are expected to reach $152.31 billion us. industry 4.0 has also negatively impacted industry in several ways. first, it eliminates the need for many old professions and skills (fitsilis et al., 2018). additionally, security risks have risen exponentially with online integration. data leaks or loss of data, in addition to data security costs, have resulted in significant financial costs. many organizations are reluctant to implement new digital technologies because of these risk/cost factors. workers are not being taught the new skills and competencies that will be required in the future such as digital communication, digital content creation, and digital problem solving (durisova, kucharcikova, & tokarcikova, 2015). the development of technology has grown faster than schools are able to recognize and implement necessary training and education. 2.2 skills in industry 4.0 education is very important for young people and is the key to preparing present and future generations for success in a highly competitive world (rauch, linder, & dallasega, 2019). certain skills will be imperative to function in the industry 4.0 environment. generally, there are two kinds of skills or competencies, namely, soft skills or non-technical skills and hard skills. examples of jobs requiring hard skills are big data analysts, software engineers, domain experts, network engineers, information technology architects, cyber security analysts, and location tracking technology experts. soft skills include communication skills, ability to collaborate with others, complex problem solving, emotional intelligence, creativity, systems thinking, people management, judgement and decision making, cognitive flexibility, and teamwork. heckman and kautz (2012) identified soft skills as crucial for learning and success in the labor market. cognitive skills are also shown to increase when facing more complex tasks. soft skills can predict success as strongly as cognitive abilities. a report detailing the economic returns resulting from soft skills in mexico and sweden found that soft skills can be cultivated throughout one’s lifetime (fitsilis et al., 2018). soft skills can also contribute to an employee’s economic return (hanushek & woessmann, 2008). humans become more mature as they develop their cognitive skills, and they are required from the earliest stages of one’s work life (hanushek et al., 2015). the world bank states that tertiary education is a good opportunity for people to acquire higher orders of cognitive skills. soft skills influence a person’s ability to learn (ra et al., 2019). neuroscience studies show that triggering one’s general curiosity enables the brain to enhance learning (gruber, gelman & ranganath, 2014). heckmann and kautz (2012) found that children who are motivated and curious tend to learn more and score higher on standardized tests. soft skills also intensify the progress of one’s cognitive abilities that further improve learning (cunha & heckman, 2007). industry 4.0 is forcing the education system to change from being facts and proceduresbased to one that actively applies knowledge to collaborative problem solving in the real world. just as the world in constantly changing, innovation and change in the education system is inevitable. the goal is to improve the quality and inclusiveness of the education ijahp article: raton, raco, krejci, ohoitimur, soputan, tumewu, korompis, taroreh, rachmadi, ngenget /soft skills of higher education in industry 4.0 era using buckley’s fuzzy-ahp international journal of the analytic hierarchy process 6 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.943 system and these changes need to happen in pedagogy and teaching methodology (umeda et al., 2019). digital technology should be incorporated in both the content and process of teaching and learning activities. educational management needs to change from deliverable-focused project management to outcome-focused product management. the educational culture has to focus on the recognition of culture’s central role in digital product delivery effectiveness. 2.3 education 4.0 in higher education technological developments have a major impact on the world of education. according to miranda et al. (2021), the emergence of the industrial revolution at the end of the 18 th century had a major impact on the world of education, namely through the creation of paper making machines, mechanical printing, the graphic pencil, the ballpoint pen and the typewriter. this stage in the world of education is called education 1.0; at this stage the teacher was still the center of the education system and their job was to determine and disseminate information that students must know and learn. in the early 20 th century, industrial machines were invented, resulting in mass production, industrialization and electricity. this development penetrated the world of education where electronic devices were introduced into the teaching and learning process. printers, calculators and computers began to enter the classroom. at this stage, teachers were still the center of knowledge development, but students began to play a role in improving their knowledge with the assistance of these electronic tools. with these electronic facilities, students began to develop study groups and peer assessments. this stage is known as education 2.0 (miranda et al., 2021). at the end of the 20 th century, computerization, automation and control grew rapidly and had an impact on the world of education. the teaching and learning process started to be supported by multimedia and went online. learning resources began to be available online, and teaching and learning activities started to be carried out virtually and were able to reach more students. teachers were no longer considered information centers because learning materials could be obtained by students online. study materials could be prepared in advance and utilized by students through online resources. collaboration between teachers and students became a key component in the teaching and learning process. this stage is known as education 3.0 (miranda et al., 2021). the beginning of the 21 st century saw digitalization enter both the industrial world and the world of education so that the digitalization of learning took place. teaching and learning can no longer be separated from computers and the internet, and learning is no longer confined to the classroom. physical form is replaced by digitization and there are no time limits so that teaching and learning can be carried out anywhere and anytime. the learning process changed from teaching and learning to learning and tutoring. there was massive innovation in the world of education. this is called education 4.0 (miranda et al., 2021). there are 4 core components in higher education at this stage. the first component is soft skills competencies such as critical thinking, cooperation, collaboration, communication, creation and innovative. these skills are in the form of training and development of functional, technical and technological knowledge and skills for successful workplace performance; promoting the capacity to research, design, creation and implementation of new technologies; and promoting the use of emerging ijahp article: raton, raco, krejci, ohoitimur, soputan, tumewu, korompis, taroreh, rachmadi, ngenget /soft skills of higher education in industry 4.0 era using buckley’s fuzzy-ahp international journal of the analytic hierarchy process 7 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.943 technologies and adoption of best practices to promote technology-based solutions. the second component consists of the learning delivery modalities such as face-to-face-active learning, online distance learning, synchronous and asynchronous, and hybrid-blended learning needs to be combined and adapted to provide more accessible and flexible programs. strategies to implement challenge-based learning, problem-based learning and learning by doing must be adapted to best utilize these various modalities. the third component is the use of information and communication technologies such as the internet of things, artificial intelligence, machine learning, cloud computing, cyber-physical systems, data science and data analytics, and mixed reality. the tools/platforms for this are web-conference platforms, learning management systems, collaborative virtual platforms, massive online open courses (moocs), remote and cyber-physical labs, robot teaching assistants, hologram teachers and others. the fourth component is the innovative use of infrastructure. at the classroom level, this includes the use of innovative furniture, tools, devices and equipment, and the use of specific architecture, colors, illuminations, sounds and temperature, and connected rooms. at the institutional level, this involves virtual and digital universities, sustainable universities, open innovation laboratories, and smart learning environments. digitalization in the modern world is changing the concept of higher education. its pedagogical orientation leads to learning innovation to meet the needs of the technological community. generation 4.0 knowledge goes beyond pedagogy and andragogy and leads to a combination approach between heutagogy which is the promotion of self-learning, peeragogy which emphasizes collaborative learning, and cybergogy which is a learning strategy using ict that offers learning experiences that go beyond the limits of time and space (miranda et al., 2021). an important feature of digitalization is the concept of the ‘digital triplet’ consisting of the physical world, the cyber world and the intelligent human (umeda et al., 2019). previously, we studied the ‘digital twins’ consisting of the physical world and the cyber world. education 4.0 will need to emphasize outcome-focused management rather than delivery-focused education (fitsilis et al., 2018). hussin (2018) stated that education 4.0 requires several things. first, it requires problemsolving such as introducing non-routine and practical problems and challenging students to solve problems collaboratively. second, it must focus on critical reflection to reconstruct the meaning of experiences, promote responsive guidance through mentoring, and knowing/learning to value experiences whether good or bad. third, it requires the student to learn from errors, learn something new about their own and other’s practices where peers are very significant to their learning. fourth, students need to learn together and from each other while teachers need to assume the role of facilitator (hussin, 2018). learning practices need to change from being classroom-based to being able to be implemented any place and anytime. students will determine how, when and what they want to learn. they need to be exposed to all potential fields of employment, industries or manufacturers. internships and collaborative projects will become more relevant for learning, and assessment methods will need to change. conventional assessment will become both irrelevant and insufficient. assessment will need to be performed during the learning process, while the application of knowledge will need to be tested when students ijahp article: raton, raco, krejci, ohoitimur, soputan, tumewu, korompis, taroreh, rachmadi, ngenget /soft skills of higher education in industry 4.0 era using buckley’s fuzzy-ahp international journal of the analytic hierarchy process 8 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.943 are working on their projects in the field. industries will become a more important place of learning (nyemba et al., 2019). there are problems with the implementation of education 4.0. first, there is a lack of digital culture and training. second, there is a lack of a clear digital operational vision and support from top management. third, the economic benefits of digital investment are unclear and the implementation of digitalization in some institutions, particularly educational institutions, is costly. fourth, technologies are constantly changing (glas & kleemann, 2016). 2.4 the analytic hierarchy process analysis of decision-making of multifaceted and complex problems is continuously improving and researchers, decision makers and managers are now recognizing the benefit of using various methods (javanbarg et al., 2012). one well-known method is the analytic hierarchy process (ahp) that was introduced by thomas saaty in the 1970s. advantages of the ahp include the ability to quantitatively measure subjective topics and reconstruct complex problems into a hierarchical structure to make the problem easy to solve (ohoitimur et al., 2019). questionnaires are designed to perform pairwise comparisons which make it easier for the respondents to determine their preferences. this method is an effective combination of quantitative and qualitative approaches (javanbarg et al., 2012). it has proven useful for decision makers to formulate a business’ management policies and is used by many researchers for scientific studies. the ahp sorts the problem and arranges it in the form of a hierarchy to reduce the complexity of the problem which greatly facilitates the decision maker’s ability to make a decision and determine the criteria to be used and the alternatives to be evaluated (mu & pereyra-rojas, 2018). it is also able to handle intangible criteria such as experience, subjective preferences (ishizaka & labib, 2009) and intuition originating from multiperson respondents with multi-criteria input (vrana, 2008). this method can handle qualitative and quantitative data based on individual perceptions. finally, the mathematical formulas are not difficult, but easy to understand and use (forbes, hebb, & mu, 2018). the ahp does however have some limitations. it uses discrete numbers and does not adequately address uncertainties. anticipating this drawback, the researchers also applied fuzzy-ahp which can calculate and address vagueness. one of the objectives of this study was to compare the findings provided by both the ahp and anp methods. the steps of the ahp are as follows. it begins with determining the research goal, then criteria and sub-criteria are determined, including alternatives. next, the criteria and alternatives are structured in a hierarchy. 2.5 fuzzy-ahp it has been determined that when the preferences are uncertain and not easily determined using exact numerical values, the ahp is insufficient (javanbarg et al., 2012). human understanding of certain complex issues is imprecise (wang & chen, n.d.) because the ijahp article: raton, raco, krejci, ohoitimur, soputan, tumewu, korompis, taroreh, rachmadi, ngenget /soft skills of higher education in industry 4.0 era using buckley’s fuzzy-ahp international journal of the analytic hierarchy process 9 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.943 real world is highly ambiguous and difficult to understand quantitatively (javanbarg et al., 2012). to address these problems zadeh (1965) introduced a fuzzy method to rationalize uncertainties in relation to vagueness and thus make them applicable to human thought. fuzzy methods continue to develop, and today there are many fuzzy methods, one of which is fuzzy-ahp which this study applies based on buckley’s method. in fuzzy-ahp, the pairwise comparison matrices are formed with triangular fuzzy numbers (tfn) and obtained by appropriate fuzzification of saaty’s scale (lavic et al., 2018). the fuzzy-ahp used in this study is buckley's fuzzy-ahp which presents a three-step decision-making process. this process involves finalizing the weights, then normalizing the weights for all the attributes/factors and finally ranking the alternatives (lohan, ganguly, & kumar, 2020). application of the fuzzy ahp method makes decisions possible by taking into account the importance of criteria and their relative priority that is needed in the study of determining soft skills (zavadskas et al., 2020). under certain conditions, the fuzzy set formed in the real numbers is called the fuzzy number. due to the uncertainty of information and the complexity of the decision-making problem, it is difficult for decision makers to express their preferences using exact numbers. in these cases, we can use fuzzy numbers to reduce the complexity. fuzzy numbers plays a vital role in many decision making applications. triangular fuzzy numbers can not only be used to express the vagueness and uncertainty of information, but can also be used to represent fuzzy terms in information processing. besides being integrated into decision-making, triangular fuzzy numbers have been applied in many disciplines such as performance evaluation, forecast, and matrix games (zhang, ma, & chen, 2014). a triangular fuzzy number not only covers interval questions, but also medians and shows the most probable relationship between indicators. the triangular fuzzy number reflects the evaluator's subjective understanding of the important relationships between indicators (lu & zhu, 2018). 2.6 sensitivity analysis a sensitivity analysis is a fundamental step in multi-criteria decision-making (mcdm) methods to measure stability, consistency, and robustness in the selection of the optimal solution. it is used in the event of a change in policy or additional information that requires the decision maker to change a policy that results in a change in the priority order. the sensitivity analysis is also called a 'what if analysis', meaning that the final result will change if there is a change in the criterion weights (mu & pereyra-rojas, 2018). the more sensitive a parameter, both criteria and sub-criteria, the poorer the criteria or subcriteria will be because it will affect the order of priority. sensitivity is determined based on the smallest range value of several criteria. the decision maker can make better decisions if he or she can determine how critical each criterion is. in other words, it is ijahp article: raton, raco, krejci, ohoitimur, soputan, tumewu, korompis, taroreh, rachmadi, ngenget /soft skills of higher education in industry 4.0 era using buckley’s fuzzy-ahp international journal of the analytic hierarchy process 10 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.943 important to know how sensitive the actual ranking of the sub-criteria is to changes in the weights of the current decision criteria. a sensitivity analysis is a dynamic element of a hierarchy. this means that the initial assessment is maintained for a certain period of time and then a change in policy or sufficient action is altered and the sensitivity analysis determines the effects that occur. a sensitivity analysis helps decision makers understand the strength of the decision (raco et al., 2021). this is an important part of the decision-making process and no final decision should be made without conducting a sensitivity analysis (mu & pereyra-rojas, 2018). therefore, to ensure the robustness of the criteria, the researcher conducted a sensitivity analysis. 3. methodology this research was conducted by following several steps that appear in figure 1. the first step was to determine the research objective, namely the determination of the dominant soft skills that graduates of higher education will require in industry 4.0 using the ahp and fuzzy-ahp. the second objective of this research is to determine if there is a significant difference between using the ahp and fuzzy-ahp for the data analysis. ijahp article: raton, raco, krejci, ohoitimur, soputan, tumewu, korompis, taroreh, rachmadi, ngenget /soft skills of higher education in industry 4.0 era using buckley’s fuzzy-ahp international journal of the analytic hierarchy process 11 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.943 figure 1 research flowchart the objective of the research is displayed in a hierarchical form as shown in figure 2. ijahp article: raton, raco, krejci, ohoitimur, soputan, tumewu, korompis, taroreh, rachmadi, ngenget /soft skills of higher education in industry 4.0 era using buckley’s fuzzy-ahp international journal of the analytic hierarchy process 12 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.943 figure 2 hierarchy structure the second step was to determine the criteria and sub-criteria based on the results of the literature review. there were 5 criteria used in this study, namely communication skills, teamwork, critical thinking and entrepreneurship skills. communication skills are defined as the ability to speak english fluently, the ability to write well and the ability to express ideas or thoughts. teamwork is defined as the ability to work under pressure, the ability to work with others at all levels and intra/extra personal skills. critical thinking is defined as skills in problem solving, innovative and creative thinking, and the ability to make good decisions. entrepreneurship skills are defined as being achievement-oriented, customer-oriented and having integrity. the third step was to develop a questionnaire in the form of pairwise comparisons. this questionnaire facilitated the respondents ability to choose because there were only two choices given for each question (raco et al., 2020). the formulation of the pairwise questionnaire used saaty’s comparative scale (1987) shown in table 1. ijahp article: raton, raco, krejci, ohoitimur, soputan, tumewu, korompis, taroreh, rachmadi, ngenget /soft skills of higher education in industry 4.0 era using buckley’s fuzzy-ahp international journal of the analytic hierarchy process 13 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.943 table 1 saaty’s comparative scale intensity of importance on an absolute scale definition explanation 1 equal importance two activities contribute equally to the objective 3 moderate importance of one over another experience and judgment strongly favor one activity over another 5 essential or strong importance experience and judgment strongly favor one activity over another 7 very strong importance an activity is strongly favored and its dominance is demonstrated in practice 9 extreme importance the evidence favoring one activity over another is of the highest possible order of affirmation 2, 4, 6, 8 intermediate values between the two adjacent judgments when compromise is needed the next step involved expert judgments consisting of 24 experienced educators in manado city. in this study, researchers used the aggregation of individual judgments and calculated them manually using the geometric mean (equation 1). according to basak & saaty (1993) and mu & pereyra-rojas (2018), the geometric mean is the correct way to synthesize judgments given by the experts as reciprocal matrices. next, the data processing was carried out using the ahp and fuzzy ahp, and the results were compared. finally, the conclusions were drawn. the steps of data analysis using the ahp are as follows: the questionnaire was completed by the experts and aggregated applying equation 1. 𝐺𝑀 = √(𝑥1)(𝑥2) … (𝑥𝑛 ) 𝑛 (1) the aggregated results were then arranged in the matrix of pairwise comparisons utilizing equation 2. the pairwise comparison matrix was normalized using equation 3. 𝐴 = [𝑎𝑖𝑗 ], 𝑎𝑖𝑗 = 𝑤𝑖 𝑤𝑗⁄ , 𝑎𝑗𝑖 = 1 𝑎𝑖𝑗 , 𝑎𝑖𝑖 = 1 ⁄ (2) 𝑏𝑖𝑗 = 𝑎𝑖𝑗 ∑ 𝑎𝑖𝑗 𝑛 𝑖=1 (3) ijahp article: raton, raco, krejci, ohoitimur, soputan, tumewu, korompis, taroreh, rachmadi, ngenget /soft skills of higher education in industry 4.0 era using buckley’s fuzzy-ahp international journal of the analytic hierarchy process 14 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.943 the priority weight was established using equation 4. 𝑤𝑖 = ∑ 𝑏𝑖𝑗 𝑛 𝑗=1 𝑛 (4) the researchers set up the consistency index as follows:  calculate the maximum (principal) eigenvalue using equation 5. 𝜆𝑚𝑎𝑥 = ∑ (𝐴𝑤)𝑖 𝑛𝑤𝑖 𝑛 𝑖=1 (5)  calculate the consistency index applying equation 6. 𝐶𝐼 = 𝜆𝑚𝑎𝑥 − 𝑛 𝑛 − 1 (6)  then, calculate the consistency of ratio utilizing equation 7. 𝐶𝑅 = 𝐶𝐼 𝑅𝐼 (7) the ratio index for each n object is shown in table 2. table 2 ratio index n 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 ri 0.00 0.00 0.58 0.90 1.12 1.24 1.32 1.41 1.45 1.49 1.51 1.48 1.56 1.57 1.59 the conversion of the ahp to the fuzzy-ahp scale is shown in table 3. ijahp article: raton, raco, krejci, ohoitimur, soputan, tumewu, korompis, taroreh, rachmadi, ngenget /soft skills of higher education in industry 4.0 era using buckley’s fuzzy-ahp international journal of the analytic hierarchy process 15 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.943 table 3 scale ahp and fuzzy-ahp linguistic variables ahp scale fuzzy ahp scale tfns reciprocal tfns equal importance 1 (1, 1, 1) diagonal (1, 1, 1) intermediate 2 (1, 2, 3) (1/3, 1/2, 1) moderately more important 3 (2, 3, 4) (1/4, 1/3, 1/2) intermediate 4 (3, 4, 5) (1/5, 1/4, 1/3) strongly more important 5 (4, 5, 6) (1/6, 1/5, 1/4) intermediate 6 (5, 6, 7) (1/7, 1/6, 1/5) very strongly more important 7 (6, 7, 8) (1/8, 1/7, 1/6) intermediate 8 (7, 8, 9) (1/9, 1/8, 1/7) extremely more important 9 (8, 9 ,9) (1/9, 1/9,1/8) the steps to determine the weight of respondents' perceptions using fuzzy ahp according to buckley’s ahp are as follows: step 1. compile a pairwise comparison matrix of criteria and sub-criteria as follows: �̃� = [ 1 �̃�12 ⋯ �̃�1𝑛 �̃�21 1 ⋯ �̃�2𝑛 ⋮ ⋮ ⋱ ⋮ �̃�𝑛1 �̃�𝑛2 … 1 ] = [ 1 �̃�12 ⋯ �̃�1𝑛 1/�̃�12 1 ⋯ �̃�2𝑛 ⋮ ⋮ ⋱ ⋮ 1/�̃�1𝑛 1/�̃�2𝑛 ⋯ 1 ] (8) with, �̃�𝑖𝑗 = { 1̃, 3̃, 5̃, 7̃, 9,̃ 𝑐𝑟𝑖𝑡𝑒𝑟𝑖𝑜𝑛 𝑖 𝑖𝑠 𝑟𝑒𝑙𝑎𝑡𝑖𝑣𝑒 𝑖𝑚𝑝𝑜𝑟𝑡𝑎𝑛𝑐𝑒 𝑡𝑜 𝑐𝑟𝑖𝑡𝑒𝑟𝑖𝑜𝑛 𝑗 1, 𝑖 = 𝑗 1̃−1, 3̃−1, 5̃−1, 7̃−1, 9 ̃−1, 𝑐𝑟𝑖𝑡𝑒𝑟𝑖𝑜𝑛 𝑖 𝑖𝑠 𝑟𝑒𝑙𝑎𝑡𝑖𝑣𝑒 𝑙𝑒𝑠𝑠 𝑖𝑚𝑝𝑜𝑟𝑡𝑎𝑛𝑐𝑒 𝑡𝑜 𝑐𝑟𝑖𝑡𝑒𝑟𝑖𝑜𝑛 𝑗 step 2. calculate the geometric mean of the fuzzy comparison value of criterion i to each criterion using equation 9. �̃�𝑖 = (�̃�𝑖1⨂ �̃�𝑖2⨂ ⋯ ⨂ �̃�𝑖𝑛 ) 1/𝑛 (9) where, �̃�𝑖𝑛 is the fuzzy comparison value of criterion i to criterion n. step 3. determine the fuzzy weight of each criterion indicated by the triangular fuzzy number. �̃�𝑖 = �̃�𝑖 ⨂(�̃�1⨁ ⋯ ⨁ �̃�𝑛 ) −1 (10) where, �̃�𝑖 is the fuzzy weight of the i th criterion and can be indicated using a triangular fuzzy number, �̃�𝑖 = (𝐿𝑤𝑖 , 𝑀𝑤𝑖 , 𝑈𝑤𝑖 ). 𝐿𝑤𝑖 , 𝑀𝑤𝑖 𝑎𝑛𝑑 𝑈𝑤𝑖 are the lower, middle and upper values of the fuzzy weight of the i th criterion. ijahp article: raton, raco, krejci, ohoitimur, soputan, tumewu, korompis, taroreh, rachmadi, ngenget /soft skills of higher education in industry 4.0 era using buckley’s fuzzy-ahp international journal of the analytic hierarchy process 16 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.943 step 4. the process of defuzzification used the center of area method to obtain the weight of the best nonfuzzy performance (bnp) applying equation 11. 𝐵𝑁𝑃𝑤𝑖 = [(𝑈𝑤𝑖 − 𝐿𝑤𝑖 ) + (𝑀𝑤𝑖 − 𝐿𝑤𝑖 )]/3 + 𝐿𝑤𝑖 (11) several studies show that fuzzy ahp, in comparison to the ahp with crisp numbers, gives more complete, flexible and realistic results. unlike with other mcdm methods, it is not necessary to know the exact numerical values of the factors being considered, but it is enough to assess a good value of comparisons. this is important for application in the construction industry, where in the first phase of a construction project that includes realization and preparation of preliminary feasibility studies, many important data concerning costs, time of work execution and others, are not precisely known, but the values of comparison of important factors could be better assessed. since these values cannot be expressed precisely by crisp numbers, it is necessary to use fuzzy numbers. the usage of verbal judgements ("equal", "equal/moderate", "moderate" to "extreme") for mutual comparison of criteria, subcriteria and alternatives is more accurate than comparison with integers or crisp numbers. at each level, the comparisons may be expressed numerically or linguistically. non numerical values are transformed to numerical ones according to the corresponding scale. the absence of units in comparison values is an important advantage since these values are quotients of two quantities of the same kind. application of fuzzy numbers instead of crisp numbers gives more realistic results and better ranking of alternatives. (praščević & praščević, 2016). the sensitivity analysis calculation is used to assess the robustness of the priority factors in the event of a change in the criteria. if there is a change in the criteria and the priority factors do not change, it can be said that these priority factors can be used in policy making. however, if there is a change in the criteria and the priority factors do change, then policymakers must be careful when using these priority factors. it is important to always pay attention if there is a change. 4. results the goal of this research was to determine the dominant soft skills that graduates should possess in the industry 4.0 era using the ahp and fuzzy-ahp methods. another objective of this study was to compare the results analysis using both methods. the respondents, who were considered the experts in this study, were lecturers in manado who have more than twenty years’ experience teaching in university and hold doctoral degrees. based on previous studies, four criteria and twelve sub-criteria were included. the criteria were communication skills, teamwork, critical thinking, and entrepreneurship skills. each criterion had three sub-criteria; therefore, the total number of sub-criteria was twelve. the goal, criteria and sub-criteria were structured in a hierarchy form (figure 2). 4.1 weighting of criteria and sub-criteria using ahp method weighting of criteria and sub-criteria in the ahp method was calculated using equations 1 – 7. following a consistency test, the pairwise comparison matrix and the weight of criteria and sub-criteria are presented in table 4. ijahp article: raton, raco, krejci, ohoitimur, soputan, tumewu, korompis, taroreh, rachmadi, ngenget /soft skills of higher education in industry 4.0 era using buckley’s fuzzy-ahp international journal of the analytic hierarchy process 17 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.943 the opinion of each respondent, according to saaty, was used as the opinion of the group by combining these opinions using the geometric mean (saaty, t.l., 2013). moreover, this method must satisfy each individual's opinion (saaty, r., 1987). saaty (2008) added that the geometric mean is the best way to combine the opinions of each individual. table 4 matrix of pairwise comparison and priority weight of the criteria c1 c2 c3 c4 priority weight c1 1.000 1.056 1.105 0.727 0.238 c2 0.947 1.000 1.352 0.957 0.260 c3 0.905 0.739 1.000 1.607 0.257 c4 1.375 1.045 0.622 1.000 0.246 𝜆𝑚𝑎𝑥 = 4.104, 𝐶𝐼 = 0.035, 𝐶𝑅 = 0.039 criteria symbols c1: communication skills c2: teamwork c3: critical thinking c4: entrepreneurship skills the priority weights of the criteria are shown in figure 3. figure 3 priority weight of the criteria from the ahp calculation of the criteria, the results were as follows. first the consistency index (𝐶𝐼 = 0.035) and consistency ratio (𝐶𝑅) = 0.039 ˂ 0.1, which means the results were consistent. the results, shown in table 4 and figure 3, show that the criteria teamwork (0.260) was ranked the highest, followed by the criteria critical thinking (0.257), then entrepreneurship skills (0.246), and lastly, communication skills (0.238). ijahp article: raton, raco, krejci, ohoitimur, soputan, tumewu, korompis, taroreh, rachmadi, ngenget /soft skills of higher education in industry 4.0 era using buckley’s fuzzy-ahp international journal of the analytic hierarchy process 18 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.943 the pairwise comparison matrix and priority weights of the sub-criteria of communication skills are displayed in table 5. table 5 pairwise comparison matrix and priority weights of the communication skill sub-criteria sc1.1 sc1.2 sc1.3 priority weight sc1.1 1.000 1.997 0.851 0.386 sc1.2 0.501 1.000 0.714 0.231 sc1.3 1.175 1.401 1.000 0.383 𝜆𝑚𝑎𝑥 = 3.030, 𝐶𝐼 = 0.015, 𝐶𝑅 = 0.026 sub-criteria communication skills symbols sc1.1: fluent english sc1.2: writing skills sc1.3: expressing/presenting ideas the pairwise comparison matrix and priority weights of the sub-criteria of teamwork are shown in table 6. table 6 pairwise comparison matrix and priority weights of the teamwork sub-criteria sc2.1 sc2.2 sc2.3 priority weight sc2.1 1.000 0.891 0.486 0.250 sc2.2 1.122 1.000 1.420 0.381 sc2.3 2.058 0.704 1.000 0.369 𝜆𝑚𝑎𝑥 = 3.103, 𝐶𝐼 = 0.051, 𝐶𝑅 = 0.089 sub-criteria teamwork symbols sc2.1: able to work under pressure sc2.2: able to work with people from all levels sc2.3: inter & intra personal skills the pairwise comparison matrix and priority weights of the sub-criteria of critical thinking are shown in table 7. ijahp article: raton, raco, krejci, ohoitimur, soputan, tumewu, korompis, taroreh, rachmadi, ngenget /soft skills of higher education in industry 4.0 era using buckley’s fuzzy-ahp international journal of the analytic hierarchy process 19 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.943 table 7 pairwise comparison matrix and priority weights of the critical thinking sub-criteria sc3.1 sc3.2 sc3.3 priority weight sc3.1 1.000 0.918 1.016 0.322 sc3.2 1.089 1.000 1.596 0.396 sc3.3 0.984 0.627 1.000 0.281 𝜆𝑚𝑎𝑥 = 3.015, 𝐶𝐼 = 0.007, 𝐶𝑅 = 0.013 sub-criteria critical thinking symbols sc3.1: problem solving skills sc3.2: creative & innovative thinking sc3.3: good decision making the pairwise comparison matrix and priority weights of the sub-criteria of entrepreneurship skills are found in table 8. table 8 pairwise comparison matrix and priority weights of the entrepreneurship skills subcriteria sc4.1 sc4.2 sc4.3 priority weight sc4.1 1.000 0.788 1.034 0.305 sc4.2 1.269 1.000 1.772 0.428 sc4.3 0.967 0.564 1.000 0.267 𝜆𝑚𝑎𝑥 = 3.010, 𝐶𝐼 = 0.005, 𝐶𝑅 = 0.009 sub-criteria entrepreneurship skills symbols sc4.1: achievement-orientated sc4.2: integrity sc4.3: customer-oriented calculations using the ahp method for each of the sub-criteria showed that the experts’ assessments were consistent because the consistency ratio (cr) of each of the subcriteria and were < 0.1 as shown in tables 5-8. 4.2 weighting of criteria and sub-criteria using fuzzy-ahp method the weighting of criteria and sub-criteria in the fuzzy-ahp method was performed using the equations 8-11. the results of the twenty-four experts were transferred into a pairwise comparison matrix as shown in table 9. the assessment of twenty-four experts was aggregated using the arithmetic mean and the results are shown in a fuzzy pairwise comparison matrix in table 9. ijahp article: raton, raco, krejci, ohoitimur, soputan, tumewu, korompis, taroreh, rachmadi, ngenget /soft skills of higher education in industry 4.0 era using buckley’s fuzzy-ahp international journal of the analytic hierarchy process 20 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.943 table 9 fuzzy pairwise comparison matrix for criteria crite ria c1 c2 c3 c4 l m u l m u l m u l m u c1 1.000 1.000 1.000 0.764 1.056 1.421 0.928 1.105 1.341 0.612 0.727 0.891 c2 0.704 0.947 1.310 1.000 1.000 1.000 1.029 1.352 1.646 0.837 0.957 1.091 c3 0.745 0.905 1.078 0.607 0.739 0.972 1.000 1.000 1.000 1.260 1.607 1.986 c4 1.122 1.375 1.634 0.917 1.045 1.195 0.504 0.622 0.794 1.000 1.000 1.000 determine the geometric mean of fuzzy comparison value of criteria using equation 9, �̃�𝑐1 = (�̃�11⨂ �̃�12⨂ �̃�13 ⨂ �̃�14) 1/4 = ((1 × 0.764 × 0.928 × 0.612)1/4, (1 × 1.056 × 1.105 × 0.727)1/4, (1 × 2.421 × 1.341 × 0.891)1/4) = (0.812, 0.960, 1.142) obtain the value of the geometric means for other criteria using the same method, �̃�𝑐2 = (0.882, 1.052, 1.238) �̃�𝑐3 = (0.869, 1.018, 1.201) �̃�𝑐4 = (0.848, 0.973, 1.116) determine the weight of each criterion based on equation 10, �̃�𝐶1 = �̃�𝐶1⨂(�̃�𝐶1⨁ �̃�𝐶2 ⨁ �̃�𝐶3 ⨁ �̃�𝐶4) −1 = (0.812,0.960, 1.142) ⨂(1/(1.142 + 1.238 + 1.201 + 1.116), 1/(0.960 + 1.052 + 1.018 + 0.973) + 1/(0.812 + 0.882 + 0.869 + 0.848)) = (0.173, 0.240, 0.335) the weights of the other criteria are obtained using the same method, �̃�𝐶2 = (0.188, 0.263, 0.363) �̃�𝐶3 = (0.185, 0.254, 0.352) �̃�𝐶4 = (0.181, 0.243, 0.327) the next step is to determine the best nonfuzzy performance (bnp) value using equation 11, 𝐵𝑁𝑃𝐶1 = [(𝑈𝑤𝐶1 − 𝐿𝑤𝐶1) + (𝑀𝑤𝐶1 − 𝐿𝑤𝐶1)]/3 + 𝐿𝑤𝐶1 = [(0,335 − 0,173) + (0,240 − 0,173)]/3 + 0,173 = 0,249 the weights of the other criteria are obtained using the same method, 𝐵𝑁𝑃𝐶2 = 0.271 𝐵𝑁𝑃𝐶3 = 0.264 𝐵𝑁𝑃𝐶2 = 0.250. the results of the calculations of the fuzzy-ahp for the criteria are shown in table 10. ijahp article: raton, raco, krejci, ohoitimur, soputan, tumewu, korompis, taroreh, rachmadi, ngenget /soft skills of higher education in industry 4.0 era using buckley’s fuzzy-ahp international journal of the analytic hierarchy process 21 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.943 table 10 results of fuzzy ahp for criteria criteria weight fuzzy bnp and normalized communication skills c1 (0.173, 0.240, 0.335) 0.241 teamwork c2 (0.188, 0.263, 0.363) 0.262 critical thinking c3 (0.185, 0.254, 0.352) 0.255 entrepreneurship skills c4 (0.181, 0.243, 0.327) 0.242 the resulting calculations of the fuzzy-ahp for criteria showed that the criteria teamwork received the highest value (0.262), followed by critical thinking (0.255), then entrepreneurship skills (0.242), and lastly, communication skills (0.241). the fuzzy pairwise comparison matrix and the weights for each sub-criteria are listed in the following tables. table 11 pairwise comparison matrix for fuzzy-ahp of sub-criteria of communication skills subcriteria sc1.1 sc1.2 sc1.3 l m u l m u l m u sc1.1 1.000 1.000 1.000 1.542 1.997 2.390 0.827 0.851 0.972 sc1.2 0.418 0.501 0.648 1.000 1.000 1.000 0.609 0.714 0.933 sc1.3 1.029 1.176 1.209 1.072 1.401 1.642 1.000 1.000 1.000 table 12 weight of sub-criteria of communication skills sub-criteria weight fuzzy bnp and normalized fluent in english sc1.1 (0.316, 0.387, 0.481) 0.389 writing skills sc1.2 (0.185, 0.230, 0.307) 0.237 expressing/presenting ideas sc1.3 (0.302, 0.383, 0.457) 0.374 ijahp article: raton, raco, krejci, ohoitimur, soputan, tumewu, korompis, taroreh, rachmadi, ngenget /soft skills of higher education in industry 4.0 era using buckley’s fuzzy-ahp international journal of the analytic hierarchy process 22 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.943 table 13 pairwise comparison matrix of fuzzy-ahp for sub-criteria of teamwork subcriteria sc2.1 sc2.2 sc2.3 l m u l m u l m u sc2.1 1.000 1.000 1.000 1.542 1.997 2.390 0.827 0.851 0.972 sc2.2 0.418 0.501 0.648 1.000 1.000 1.000 0.609 0.714 0.933 sc2.3 1.029 1.176 1.209 1.072 1.401 1.642 1.000 1.000 1.000 table 14 weight of sub-criteria of teamwork sub-criteria weight fuzzy bnp and normalized able to work under pressure sc2.1 (0.197, 0.248, 0.324) 0.251 able to work with people sc2.2 (0.304, 0.382, 0.473) 0.379 inter & intra personal skills sc2.3 (0.280, 0.370, 0.484) 0.370 table 15 pairwise comparison matrix of fuzzy-ahp for sub-criteria of critical thinking sub criteria sc3.1 sc3.2 sc3.3 l m u l m u l m u sc3.1 1.000 1.000 1.000 0.780 0.918 1.078 0.718 1.016 1.421 sc3.2 0.928 1.089 1.281 1.000 1.000 1.000 1.204 1.596 2.079 sc3.3 0.704 0.985 1.384 0.481 0.626 0.831 1.000 1.000 1.000 table 16 weight of sub-criteria of critical thinking sub-criteria weight fuzzy bnp and normalized problem solving skills sc3.1 (0.230, 0.322, 0.450) 0.322 creative & innovative sc3.2 (0.289, 0.397, 0.542) 0.394 good decision making sc3.3 (0.194, 0.281, 0.410) 0.284 ijahp article: raton, raco, krejci, ohoitimur, soputan, tumewu, korompis, taroreh, rachmadi, ngenget /soft skills of higher education in industry 4.0 era using buckley’s fuzzy-ahp international journal of the analytic hierarchy process 23 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.943 table 17 pairwise comparison matrix of fuzzy-ahp for sub-criteria entrepreneurship skills subcriteria sc4.1 sc4.2 sc4.3 l m u l m u l m u sc4.1 1.000 1.000 1.000 0.675 0.788 0.941 0.851 1.034 1.263 sc4.2 1.062 1.269 1.480 1.000 1.000 1.000 1.303 1.772 2.239 sc4.3 0.792 0.967 1.175 0.447 0.564 0.767 1.000 1.000 1.000 table 18 weight of sub-criteria of entrepreneurship skills sub-criteria weight fuzzy bnp and normalized achievement-oriented sc4.1 (0.236, 0.305, 0.399) 0.305 integrity sc4.2 (0.317, 0.428, 0.562) 0.424 customer-oriented sc4.3 (0.201, 0.267, 0.364) 0.270 4.3 comparison analysis of the calculated results of ahp and fuzzy-ahp methods a comparison of the calculated results of the ahp and fuzzy-ahp for the criteria are shown in table 19 and figure 4. table 19 comparison analysis results of the ahp and fuzzy ahp for criteria criteria weight (ahp) weight (f-ahp) communication skills c1 0.238 0.241 teamwork c2 0.260 0.262 critical thinking c3 0.257 0.255 entrepreneurship skills c4 0.246 0.242 ijahp article: raton, raco, krejci, ohoitimur, soputan, tumewu, korompis, taroreh, rachmadi, ngenget /soft skills of higher education in industry 4.0 era using buckley’s fuzzy-ahp international journal of the analytic hierarchy process 24 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.943 figure 4 comparison analysis results of the ahp and the fuzzy-ahp for criteria from the results in table 19 and figure 4, we concluded that there were no differences in the results between the two methods. teamwork was ranked first and received the highest weight, followed by critical thinking, then entrepreneurship skills and lastly, communication skills. the differences in the results of the ahp and fuzzy-ahp were quite small. for example, the difference in the results of the teamwork criteria between the ahp and fuzzy-ahp was only 0.0439; for the critical thinking, the difference was 0.0075. this was also true for the comparison between the criteria. for example, the difference between the teamwork criteria (fuzzy-ahp 0.2456) and the critical thinking criteria (fuzzy-ahp 0.2224) was only 0.0232. the comparison of the results of analysis for each of the sub-criteria is shown in the following tables and figures. table 20 comparison analysis of sub-criteria of communication skills criteria weight (ahp) weight (f-ahp) fluent in english sc1.1 0.386 0.389 writing skills sc1.2 0.231 0.237 expressing/presenting ideas sc1.3 0.383 0.374 ijahp article: raton, raco, krejci, ohoitimur, soputan, tumewu, korompis, taroreh, rachmadi, ngenget /soft skills of higher education in industry 4.0 era using buckley’s fuzzy-ahp international journal of the analytic hierarchy process 25 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.943 figure 5 comparison of the results analysis of the ahp and fuzzy ahp for sub-criteria communication skills table 21 comparison of the results analysis of the sub-criteria of teamwork criteria weight (ahp) weight (f-ahp) able to work under pressure sc2.1 0.250 0.251 able to work with people from all levels sc2.2 0.381 0.379 inter & intra personal skills sc2.3 0.369 0.370 figure 6 comparison of the results analysis of ahp and fuzzy ahp for sub-criteria of teamwork ijahp article: raton, raco, krejci, ohoitimur, soputan, tumewu, korompis, taroreh, rachmadi, ngenget /soft skills of higher education in industry 4.0 era using buckley’s fuzzy-ahp international journal of the analytic hierarchy process 26 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.943 table 22 comparison of the results analysis for sub-criteria of critical thinking criteria weight (ahp) weight (f-ahp) problem solving skills sc3.1 0.322 0.322 creative & innovative thinking sc3.2 0.396 0.394 good decision making sc3.3 0.281 0.284 figure 7 comparison of the results analysis of sub-criteria of critical thinking table 23 comparison of the results analysis of sub-criteria of entrepreneurship skills criteria weight (ahp) weight (f-ahp) achievement-orientated sc4.1 0.305 0.305 integrity sc4.2 0.428 0.424 customer-oriented sc4.3 0.267 0.270 ijahp article: raton, raco, krejci, ohoitimur, soputan, tumewu, korompis, taroreh, rachmadi, ngenget /soft skills of higher education in industry 4.0 era using buckley’s fuzzy-ahp international journal of the analytic hierarchy process 27 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.943 figure 8 comparison of the results analysis of the sub-criteria of entrepreneurship skills 4.4 global calculation results the global results or global weight of the dominant factor of soft skills that the graduates at the university in manado should possess based on the assessment of 24 experts was obtained by multiplication between the criteria and each sub-criterion. based on the calculation, the results of the global weights are shown in table 24. ijahp article: raton, raco, krejci, ohoitimur, soputan, tumewu, korompis, taroreh, rachmadi, ngenget /soft skills of higher education in industry 4.0 era using buckley’s fuzzy-ahp international journal of the analytic hierarchy process 28 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.943 table 24 global weights criteria/sub-criteria local weight global weight ahp fahp ahp fahp communication skills c1 0.238 0.241 teamwork c2 0.260 0.262 critical thinking c3 0.257 0.255 entrepreneurship skills c4 0.246 0.242 sum 1.000 1.000 fluent in english sc1.1 0.386 0.389 0.092 0.094 writing skills sc1.2 0.231 0.237 0.055 0.057 expressing/presenting ideas sc1.3 0.383 0.374 0.091 0.090 sum 1.000 1.000 0.238 0.241 able to work under pressure sc2.1 0.250 0.251 0.065 0.066 able to work with people from all levels sc2.2 0.381 0.379 0.099 0.099 inter & intra personal skills sc2.3 0.369 0.370 0.096 0.097 sum 1.000 1.000 0.260 0.262 problem solving skills sc2.1 0.322 0.322 0.083 0.082 creative & innovative thinking sc3.2 0.396 0.394 0.102 0.100 good decision making sc3.3 0.281 0.284 0.072 0.072 sum 1.000 1.000 0.257 0.255 achievement-orientated sc4.1 0.305 0.305 0.075 0.074 integrity sc4.2 0.428 0.424 0.105 0.103 customer-oriented sc4.3 0.267 0.270 0.066 0.065 sum 1.000 1.000 0.246 0.242 the global weights revealed that integrity was the soft skills dominant factor. the result of the calculations for integrity was the highest for both the ahp (10.5%) and fuzzyahp (10.3%). a sensitivity analysis was carried out with the help of superdecisions software using the steps in mu & pereyra-rojas (2018). the weight of the criteria and sub-criteria for the original results, scenario 1 and scenario 2, and the results of the sensitivity analysis are listed in table 25. the original results show that the largest criterion weight is teamwork in the ahp (26%) and in fuzzy-ahp (26.2%). however, the greatest weight for the sub-criteria is integrity in the ahp (10.5%) and in fuzzy-ahp (10.3%). ijahp article: raton, raco, krejci, ohoitimur, soputan, tumewu, korompis, taroreh, rachmadi, ngenget /soft skills of higher education in industry 4.0 era using buckley’s fuzzy-ahp international journal of the analytic hierarchy process 29 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.943 the first scenario in the sensitivity analysis involved making the criteria weights equal, in this case each criterion was given a weight of 0.25. the results of this first scenario show that the largest sub-criteria weight remains integrity (10.6%). table 25 weight of criteria and sub-criteria of sensitivity analysis criteria / sub-criteria weight original scenario 1 scenario 2 communication skills 0.23754 0.25 0.238 teamwork 0.26014 0.25 0.240 critical thinking 0.25782 0.25 0.278 entrepreneurship skills 0.24451 0.25 0.244 fluent in english 0.09192 0.09674 0.09210 writing skills 0.05467 0.05754 0.05478 expressing/presenting ideas 0.09094 0.09572 0.09112 able to work under pressure 0.06440 0.06189 0.05941 able to work with people from all levels 0.09942 0.09555 0.09173 inter & intra personal skills 0.09632 0.09257 0.08886 problem solving skills 0.08312 0.08060 0.08962 creative & innovative thinking 0.10229 0.09919 0.11030 good decision making 0.07241 0.07021 0.07808 achievement-oriented 0.07460 0.07627 0.07444 integrity 0.10464 0.10699 0.10442 customer-oriented 0.06527 0.06674 0.06514 the second scenario involved changing the weights of the two highest criteria, namely teamwork and critical thinking. the weight value of the teamwork criteria was reduced by 2% and the weight of the critical thinking criteria was increased by 2%. the value for the criteria for communication skills and entrepreneurship skills remained the same as in the original results. this resulted in a change in the weight of the sub-criteria, in this case the highest sub-criteria weight became creative and innovative thinking with .1103. ijahp article: raton, raco, krejci, ohoitimur, soputan, tumewu, korompis, taroreh, rachmadi, ngenget /soft skills of higher education in industry 4.0 era using buckley’s fuzzy-ahp international journal of the analytic hierarchy process 30 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.943 5. discussion the ahp and fuzzy-ahp methods were used to determine the soft skills dominant factors that the graduates in manado need to possess to be able to compete in the industry 4.0 era. the research findings will be used to improve the curriculum and the teachinglearning systems of higher institutions, particularly at universitas katolik de la salle manado (de la salle catholic university of manado-indonesia). the criteria and subcriteria were determined by the experts based on previous studies. the researchers designed questionnaires in the form of a pairwise comparison matrix. the analysis of consistencies verified that the results were consistent and considered scientifically acceptable. the results proved, as shown in table 24, that teamwork was the dominant factor, followed by critical thinking, then entrepreneurship skills and lastly, communication skills. the research findings for both ahp and fuzzy-ahp were the same. the differences in the results between the criteria in the ahp and fuzzy-ahp were small. for example, the gap between the ahp and fuzzy-ahp for both the teamwork and critical thinking criteria was only 0.002. this was also true for the resulting gap between the criteria themselves. for example, the difference between the teamwork criteria (fuzzyahp 26.2%) and critical thinking criteria (fuzzy-ahp 25.5%) was only 0.07%. the differences are understandable because the fuzzy-ahp uses triangular fuzzy numbers unlike the ahp which uses a single value. the global analysis (table 24) shows that integrity was the dominant factor. the findings recommend that in the industry 4.0 era, the higher institutions in manado need to provide students with teamwork skills. teamwork skills should appear in the curriculum and the teaching-learning system, and faculty and teaching methods must focus on providing teamwork skills. the sectoral ego will not be effective in the industry 4.0 era. everyone needs to work together to be successful. however, other skills such as critical thinking, entrepreneurship and communication must also be considered because these criteria also support the future success of the graduates. the combination of these criteria is larger than the teamwork criteria. failure to acknowledge the importance of them would cause the graduates’ competencies to suffer. the study advocates that whatever the profession or job chosen by the graduates, teamwork must be given priority during their education. the interconnection and interdependency between human beings, machines and objects in the industry 4.0 era requires workers to be team players. compatibility of products, systems and services are only possible if teamwork exists. the overall result or global weight shows that integrity has the highest score in the ahp (10.5%) and fuzzy-ahp (10.3%) which means that the superior skill in industry 4.0 is integrity. to determine if integrity is a robust result, a sensitivity analysis was conducted by changing the criteria. the value of the original result criteria was changed by 2%. in this case, the weight of the teamwork criteria was reduced by 2% and the weight of the critical thinking criteria was increased by 2%. meanwhile, the weights of the criteria for ijahp article: raton, raco, krejci, ohoitimur, soputan, tumewu, korompis, taroreh, rachmadi, ngenget /soft skills of higher education in industry 4.0 era using buckley’s fuzzy-ahp international journal of the analytic hierarchy process 31 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.943 communication skills and entrepreneurship skills did not change. as a result, with a weight change of 2%, the weight of the teamwork criteria becomes 24% and the weight of the critical thinking criteria becomes 27.8% while the weights of the sub-criteria change with the largest weight being the creative and innovative thinking sub-criteria at 11.03% and the integrity sub-criteria being 10.4%. therefore, a 2% decrease in the value of the teamwork criteria and a 2% increase in weight of the critical thinking criteria will cause a change in the position of the highest sub-criteria, namely, creative and innovative thinking. this means that the integrity sub-criteria is very unstable, not robust or fragile. this study shows that integrity is the most important factor in higher education in the industry 4.0 era. integrity is defined as conformity between words and action, rather than manipulation. integrity is a key to success in industry 4.0 and must be incorporated into the curriculum, teaching-learning materials, studying system and educational processes. integrity means to behave in an honest, fair, and ethical manner. mondal (2015) mentioned that integrity is the ability to act with honesty and be consistent in whatever it is one is doing based on one’s own particular moral, value or belief compass. covey defined integrity as “honestly matching words and feelings with thoughts and actions, with no desire other than for the good of others” (pillay, 2014). the latin word “integritas” denotes wholeness or unity. it means that to attain integrity, someone must be whole and undivided. in the scholarly discourses, this position is called “integrated-self view” and implies that “integrity is a matter of persons integrating various parts of their personality into a harmonious, intact whole (schottl, 2015). these findings are in line with the lasallian expected qualities known as elga (expected lasallian graduate attributes). these attributes include being an effective communicator, critical and creative thinker, lifelong learner, service driven-citizen, steward of the environment and entrepreneurial spirit. the ahp and fuzzy-ahp methods were appropriate for this study. the rankings produced by both of these methods were the same even though there were numerical differences between them. this is because ahp uses discrete numbers while fuzzy-ahp applies triangular fuzzy numbers so it can capture the uncertainties or vagueness of the perceptions of the experts. it was important to keep in mind that the ahp and fuzzyahp were not competing with each other. the ahp was used if the evaluation or information was definite. however, if the information or evaluation was blurred and uncertain fuzzy-ahp was used. the ahp is a good methodology to use when there is a lack of statistical data and researchers have to rely on the experts’ choice. for the experts or respondents, the ahp questionnaire, in the form of a pairwise comparison matrix is quite helpful since they only need to compare two options. one of the limitations of this study was the respondents’ unfamiliarity with the ahp method. there was some confusion about answering the questionnaire which required the researchers to assist the respondents in making their choices. most of the experts felt indifferent about the options and thought that there was little difference between the criteria or sub-criteria. previous studies have used pareto (raco et al., 2020) to explore the capabilities of the fuzzy method paired with the ahp. ijahp article: raton, raco, krejci, ohoitimur, soputan, tumewu, korompis, taroreh, rachmadi, ngenget /soft skills of higher education in industry 4.0 era using buckley’s fuzzy-ahp international journal of the analytic hierarchy process 32 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.943 one limitation of the ahp is that it cannot evaluate vagueness since it uses crisp numbers, while in the real-world problems are not always represented by crisp numbers. in reality, all things are not black and white. 6. conclusions the study aimed to determine the soft skills dominant factors that the graduates of higher education should possess to be successful in industry 4.0 and to compare the results of the ahp and fuzzy-ahp. the results of the study showed that the teamwork criteria had the highest priority of 26% in ahp and 26.2% in f-ahp, followed by critical thinking with a priority of 25.7% in ahp and 25.5% in f-ahp, entrepreneurial skills at 24.6% in the ahp and 24.2% in fahp and communication skills at 23.8% in the ahp and 24.1% in f-ahp. moreover, the global weight calculation showed that the element of integrity was the highest factor, followed by the ability to work with people of all levels, then having intra-extra personal skills, followed by creative and innovative thinking and lastly, fluency in english (table 24). the results of the sensitivity analysis show that the integrity factor is fragile because when the teamwork criteria is reduced by 2% and the critical thinking criteria is increased by 2%, the highest sub-criteria value then becomes creative innovative thinking (11.03%) followed by integrity (10.4%). while teamwork was the most dominant and integrity had the top global weight, other criteria and elements should not be ignored. teamwork and integrity will be effective and successful when combined with other criteria and elements. moreover, the numerical differences between the criteria were quite small, which was also true with other elements of the global weights. failing to consider other criteria and sub-criteria could jeopardize the findings because if those criteria with smaller numerical results were combined they would yield a significant number and hold greater weight. the results of this research must be conveyed to students by qualified lecturers to prepare for their future (lazarević, 2019). the findings might be different than other research that is conducted in different situations. the researchers suggest caution when implementing the results. factors such as lack of knowledge and undistinguished opinions about the options among the respondents should be seriously taken into consideration. the calculation of the fuzzy-ahp method took longer, but had higher accuracy than the ahp. it is able to deal with the vagueness in human thinking and effectively solve multicriteria decision making problems. the results of the study have implications for management. first, the dominant criteria should be used as guidelines for decisions by management when making changes. according to this study, the greatest attention should be given to teamwork and integrity. second, there were only small numerical differences between the criteria and the global weight results. the study suggests a need to do comparative research with other ijahp article: raton, raco, krejci, ohoitimur, soputan, tumewu, korompis, taroreh, rachmadi, ngenget /soft skills of higher education in industry 4.0 era using buckley’s fuzzy-ahp international journal of the analytic hierarchy process 33 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.943 institutions to determine whether there would be different results. since changes are happening fast, this research needs to be conducted regularly. third, every researcher has different interpretations, so a narrative of the background and ranking methodologies is necessary. this study was done in indonesia with indonesian subjects. cross cultural analysis in different countries is required to ensure generalizability of the results. the researchers recommend future studies using the same or different soft skills criteria using entrepreneurs, owners or business management as respondents. ijahp article: raton, raco, krejci, ohoitimur, soputan, tumewu, korompis, taroreh, rachmadi, ngenget /soft skills of higher education in industry 4.0 era using buckley’s fuzzy-ahp international journal of the analytic hierarchy process 34 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.943 references aulbur, a., arvind, c., & bigghe, r. (2016). skill development for industry 4.0. azmi, a. n., kamin, y., noordin, m., & nasir, a. (2018). towards industrial revolution 4.0: employers’ expectations on fresh engineering graduates. international journal of engineering and technology (uae), 7(4), 267–272. doi: https://doi.org/10.14419/ijet.v7i4.28.22593 basak, i., & saaty, t. (1993). group decision making using the analytical hierarchy process. mathematics and computer modelling, 17(415), 101-109as. cunha, f., & heckman, j. (2007). the technology of skill formation (nber working paper no. 12840). nber working paper series. massachusetts. dombrowski, u., wullbrandt, j., & fochler, s. (2019a). center of excellence for lean enterprise 4.0. procedia manufacturing, 31, 66–71. doi: https://doi.org/10.1016/j.promfg.2019.03.011 dombrowski, u., wullbrandt, j., & fochler, s. (2019b). center of excellence for lean enterprise 4.0. procedia manufacturing, 31(2019), 66–71. doi: https://doi.org/10.1016/j.promfg.2019.03.011 durisova, m., kucharcikova, a., & tokarcikova, e. (2015). assessment of higher education teaching outcomes (quality of higher education), procedia-social and behavioral sciences, 174(2015), 2497–2502. doi: https://doi.org/10.1016/j.sbspro.2015.01.922 fitsilis, p., tsoutsa, p., & gerogiannis, v. (2018). industry 4.0 : required personel competencies. international scientific journal “industry 4.0,” 133(3), 130–133. forbes, j., hebb, a., & mu, e. (2018). ethical decision making in action: evaluation hositpal attendance approaches international journal of analytic hierarchy process, 10(3), 313–347. doi: https://doi.org/10.13033/ijahp.v10i3.592 gallon, r., & gales, n. (2017). hybrid communication for industry 4.0: nemetic models. france: the transformation society. glas, a. h., & kleemann, p. f. c. (2016). the impact of industry 4.0 on procurement and supply management: a conceptual and qualitative analysis. international journal of business and management invention. hanushek, e. a., schwerdt, g., wiederhold, s., & woessmann, l. (2015). returns to skills around the world : evidence from piaac. european economic review, 73, 103– 130. doi: https://doi.org/10.1016/j.euroecorev.2014.10.006 hanushek, e. a., & woessmann, l. (2008). the role of cognitive skills in economic development. journal of economic literature, 46(3), 607–668. ijahp article: raton, raco, krejci, ohoitimur, soputan, tumewu, korompis, taroreh, rachmadi, ngenget /soft skills of higher education in industry 4.0 era using buckley’s fuzzy-ahp international journal of the analytic hierarchy process 35 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.943 heckman, j. j., & kautz, t. d. (2012). hard evidence of soft skills. cambridge, ma: national bureau of economics research. hussin, a. a. (2018). education 4.0 made simple : ideas for reaching. international journal of education & literacy studies, 6(3), 92–98. ishizaka, a., & labib, a. (2009). analytic hierarchy process and expert choice : benefits and limitations. or insight, 22(4), 201–220. doi: https://doi.org/10.1057/ori.2009.10 javanbarg, m.b., scawthorn, c., kiyono, j., & shahbodaghkhan, b. (2012). expert systems with applications fuzzy ahp-based multicriteria decision making systems using particle swarm optimization. expert systems with applications, 39(1), 960–966. doi: https://doi.org/10.1016/j.eswa.2011.07.095 lavic, z., vucijak, b., pasic, m., & dukic, n. (2018). consistency check of fuzzy pairwise comparison matrices of dimesions larger than 3x3. 29 th daam international symposium on intelligent manufacturing and automation. doi: https://doi.org/10.2507/29th.daaam.proceedings.102 lazarević, i. (2019). using the electre mlo multi-criteria decision making method in stepwise benchmarking application in higher education. operational research in engineering sciences: theory and applications, 2(3), 77–91. doi: https://doi.org/10.31181/oresta1903077l lohan, a., ganguly, a., & kumar, c. (2020). “what’s foreign is better”: a fuzzy-ahp analysis to evaluate factors that influence foreign product choice among indian consumers. international journal of analytic hierarchy process, 12(3), 460–487. doi: https://doi.org/10.13033/ijahp.v12i3.743 lu, m., & zhu, k. (2018). improved analytic hierarchy process based on triangular fuzzy number. advances in social science, education and humanities research, 182, 314–317. meindl, b., ayala, n. f., mendonça, j., & frank, a. g. (2021). the four smarts of industry 4.0 : evolution of ten years of research and future perspectives. technological forecasting & social change, 168(2020), 1–13. doi: https://doi.org/10.1016 miranda, j., navarrete, c., noguez, j., molina-espinosa, j., ramirez-montoya, m., navarro-tuch, s., … molina, a. (2021). the core components of education 4.0 in higher education: three case studies in engineering education. computers and electrical engineering, 93(2020), 1–13. doi: https://doi.org/10.1016/j.compeleceng.2021.107278 mondal, u. (2015). integrity, competency and ethical behaviour. ijmt, 3(4), 188-196. mu, e., & pereyra-rojas, m. (2018). practical decision making using super decisions v3. an introduction to the analytic hierarchy process. pittsburgh: springer. ijahp article: raton, raco, krejci, ohoitimur, soputan, tumewu, korompis, taroreh, rachmadi, ngenget /soft skills of higher education in industry 4.0 era using buckley’s fuzzy-ahp international journal of the analytic hierarchy process 36 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.943 nyemba, w., carter, k., mbohwa, c., & chinguwa, s. (2019). a systems thinking approach to collaborations for capacity building and sustainability in engineering education and sustainability in engineering education. procedia manufacturing, 33, 732– 739. doi: https://doi.org/10.1016/j.promfg.2019.04.092 ohoitimur, j., krejci, j., raco, j. r., raton, y., & taroreh, f. (2019). pstoral strategic planning priorities for the church: case study of the vicarate episcopal of tonsea of the diocese of manado. international journal of the analytic hierarchy process, 11(3), 415– 434. doi: https://doi.org/10.13033 paravizo, e., chaim, o. c., braatz, d., muschard, b., & rozenveld, h. (2018). exploring gamification to support manufacturing education on industry 4.0 as an enabler for innovation and sustainability. procedia manufacturing, 21, 438–445. doi: https://doi.org/10.1016/j.promfg.2018.02.142 pillay, p. (2014). integrity leadership. african journal of public affairs, 7(2), 177–179. praščević, n., & praščević, ž. (2016). application of fuzzy ahp method based on eigenvalues for decision making in construction industry. tehnički vjesnik, 23(1), 57–64. doi: https://doi.org/10.17559/tv-20140212113942 ra, s., shrestha, u., khatiwada, s., & yoon, s.w. (2019). the rise of technology and impact on skills. international journal of training research, 17(1), 26–40. doi: https://doi.org/10.1080/14480220.2019.1629727 raco, j. ., krejci, j. v, ohoitimur, j., adrian, a. ., raton, y., rottie, r., … sumakud, e. (2021). priority sector of small and medium enterprises using ahp: a case study of yamaru enterprise. international journal of analytic hierarchy process, 13(2), 220–239. doi: https://doi.org/10.13033/ijahp.v13i2.862 raco, j., ohoitimur, j., krejci, j. v, raton, y., rottie, r., paseru, d., … rachmadi, r. (2020). the dominant factor of lecturers' research productivity using the ahp: case study of catholic university of de la salle manado-indonesia. international journal of analytic hierarchy process, 12(3), 546–564. doi: https://doi.org/10.13033/ijahp.v12i3.819 raco, j. r., & tanod, h. (2014). the phenomenological method in entrepreneurship. international journal of entrepreneurship and small business, 22(3), 276–285. rauch, e., linder, c., & dallasega, p. (2019). anthropocentric perspective of production before and within industry 4 . 0. computers & industrial engineering, 139(2020), 1–15. doi: https://doi.org/10.1016/j.cie.2019.01.018 saaty, r. . (1987). the analytic hierarchy process-what and how it is used. math modelling, 9(3), 161–176. saaty, t. l. (2008). relative measurement and its generalization in decision making. ijahp article: raton, raco, krejci, ohoitimur, soputan, tumewu, korompis, taroreh, rachmadi, ngenget /soft skills of higher education in industry 4.0 era using buckley’s fuzzy-ahp international journal of the analytic hierarchy process 37 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.943 why pairwise comparisons are central in mathematics for the measurement of intangible factors. the analytic hierarchy/network process. racsam, 102(2), 251–318. saaty, t. l. (2013). on the measurement of ingangibles. a principal eigenvector approach to relative measurement derived from paired comparisons. notices of the american mathematical society, 60(2), 192–208. schottl, l. (2015). the concept of moral integrity and its implications for business. kicg-forschungspapiere (vol. 9). umeda, y., ota, j., kojima, f., saito, m., matsuzawa, h., sukekawa, t., … shirafuji, s. (2019). development of an education program for digital manufacturing system engineers based on ‘ digital triplet ’ concept. procedia manufacturing, 31, 363–369. doi: https://doi.org/10.1016/j.promfg.2019.03.057 vrana, s, a. (2008). evaluating the knowledge, relevance and experience of expert decision makers utilizing the fuzzy-ahp. agricultural economics, 54(11), 529–535. doi: https://doi.org/10.17221/264-agricecon wang, t., & chen, y. (n.d.). some issues on consistency of fuzzy analytic hierarchy process. zavadskas, e., turskis, z., stević, ž., & mardani, a. (2020). modelling procedure for the selection of steel pipe supplier by applying the fuzzy ahp method. operational research in engineering sciences: theory and applications, 3(2), 39–53. doi: https://doi.org//10.31181/oresta2003034z zhang, x., ma, w., & chen, l. (2014). new similarity of triangular fuzzy number and its application. the scientific world journal, (215047), 1–7. doi: https://doi.org/10.1155/2014/215047 ijahp news: ferretti/software announcement: multichoice as new software for decision making with analytic network process international journal of the analytic hierarchy process 388 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.413 software announcement: multichoice as new software for decision making with analytic network process 1 maria milkova central institute of economics and mathematics of the russian academy of science m.a.milkova@gmail.com olga andreichikova central institute of economics and mathematics of the russian academy of science alexandrol@mail.ru abstract this paper describes the main features of multichoice, a new version of software for multi-criteria decision analysis with anp/ahp. multichoice has been developed by authors at the central institute of economics and mathematics of the russian academy of science in moscow in 2016, and is the first software for anp in russia (available in russian and english). the paper outlines the main steps of anp technology: model constructing, relative and absolute evaluating, synthesizing, visualizing and importing the results, and sensitivity analysis. the authors also discuss further development of implementation of anp-algorithms. keywords: software development; analytic network process; analytic hierarchy process; decision-making; multichoice 1. introduction the appropriate software is invaluable in modern decision-making. nowadays, there are many programs based on the analytic hierarchy process (ahp) on the software market. generally, they can be divided into three types: independent software, add-ins for microsoft excel, and online calculators. some of them are trivial and permit working with very simple structures; others allow one to deal with complex problems and give many opportunities to decision-makers. this variety allows the appropriate software to be chosen depending on the user’s goal. actually many decision problems cannot be structured hierarchically because they involve interaction and dependence between elements. thus, to deal with real life problems we should apply the analytic network process (anp) – a generalization of ahp. anp allows both interaction and feedback, 1 this work is supported by russian foundation of humanities, project no. 16-02-00743 “multicriteria analysis and forecasting the techno-economic state and trends for leading aerospace companies”. mailto:m.a.milkova@gmail.com mailto:alexandrol@mail.ru ijahp news: ferretti/software announcement: multichoice as new software for decision making with analytic network process international journal of the analytic hierarchy process 389 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.413 which best captures the complex effects of interplay in human society, especially when risk and uncertainty are involved. worthy implementation of anp is much more complex than implementation of ahp, so there is only one version of software for anp applications available on the market today. this is the super decisions software, written by the anp team, working for the creative decisions foundation. the superdecisions software is available for free (www.superdecisions.com/~saaty) on the internet (saaty, 2009). because one of our key interests includes the area of decision making associated with anp, we have tried to develop an anp technology through creating a new version of software that meets all requirements related to modern decision making. it is with respect to the author of anp/ahp, dr. saaty and his team, that we have created the multichoice software as an instrument, where we can implement all things we faced in the anp: model constructing, relative and absolute evaluating, synthesizing, visualizing and importing the results, sensitivity analysis. in this paper, we discuss the main features of the multichoice software, and propose further development of implementation of anp-algorithms. we will not focus on the theory of anp/ahp, assuming that readers are familiar with its concept. 2. multichoice overview 2.1 base principles multichoice is a new software product created to support multi-criteria decision analysis based on the anp/ahp. the multichoice software has been developed by authors at the central institute of economics and mathematics of the russian academy of science, in moscow in 2016, and is the first software for anp in russia. the software is certificated by the russian federal service for intellectual property. multichoice software is based on the following concepts:  user friendly interface  common workspace for all networks and hierarchies for the model  distinct tool kits for network and hierarchy  handy instruments for data export and visualization  opportunity for expanding software functionality  russian and english software version multichoice has been tested in the educational process by solving different decisionmaking problems (andreichikova, milkova, 2016). further, we describe the main steps of anp/ahp technology realized in the first version of the multichoice software. 2.2 model construction the first step of decision making is specifying the decision problem and constructing the structures to represent the problem. ijahp news: ferretti/software announcement: multichoice as new software for decision making with analytic network process international journal of the analytic hierarchy process 390 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.413 thus, at the start window of multichoice the main goal and the base structure of the model are defined: networks (or hierarchies) under control hierarchy or a single network (or hierarchy). further, one can specify all its elements (subnetworks, clusters, nodes, and relations between them) at the main window. the main window contains workspace, a menu, the area with the model structure to navigate, and an area for any comments to specify the network (hierarchy), cluster or node (see figures 1 and 2). figure 1. main window of multichoice: network ijahp news: ferretti/software announcement: multichoice as new software for decision making with analytic network process international journal of the analytic hierarchy process 391 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.413 figure 2. main window of multichoice: hierarchy network and hierarchy have different diagrams. hierarchy looks like the straight hierarchy structure with the main goal at the top and levels arranged in a descending order of importance (see fig 2). in a network, the components are not arranged in any particular order, but are connected as appropriate in pairs with directed lines – arrows, forming loops if the element has connection with itself (see figure 1). depending on whether the network or hierarchy is defined as a problem structure, the appropriate menu tab is activated (see also figures 1 and 2). the tab for networks contains options for creating connections, displaying matrices of connections, a supermatrix and its limit forms, and limit vectors. the tab for hierarchy is simpler and does not have these options. 2.3 evaluating multichoice software permits the use of a relative or absolute type of evaluation for expert’s judgments. relative judgment is performed by paired comparisons of elements according to which element influences a third element more and how strongly more with respect to a control criterion (saaty, 2013). the first version of multichoice has been developed for researchers only, so we have not included visual types of evaluation, questionnaires and provide the main form of elements evaluation – the comparison matrix. these can be added into future versions, as could the possibility for setting the priority vector directly. screenshots for pairwise comparisons of the elements within a network are shown in the figures 3 and 4. the panel for navigation through all comparison matrices is arranged at the left of the screen. a checkbox near the comparison matrix shows if the matrix is consistent. an appropriate consistency ratio (cr) for the model is set in the box “limit value of c.r.” ijahp news: ferretti/software announcement: multichoice as new software for decision making with analytic network process international journal of the analytic hierarchy process 392 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.413 by an expert and must be less than 0.2. if the cr for any matrix is higher than the given limit value, multichoice will not calculate the supermatrix of the network. figure 3. pairwise comparisons: cluster comparisons figure 4. pairwise comparisons: nodes comparisons ijahp news: ferretti/software announcement: multichoice as new software for decision making with analytic network process international journal of the analytic hierarchy process 393 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.413 absolute judgment is performed by establishing rating categories for each covering criterion (see figure 5). figure 5. alternatives absolute evaluation elements are evaluated by assigning the appropriate rating category for each criterion. in multichoice, by default, there are the following rating categories: “very high”, “high”, “middle”, “low”, “very low”. categories are prioritized by pairwise comparing them for preference (see figure 6). ijahp news: ferretti/software announcement: multichoice as new software for decision making with analytic network process international journal of the analytic hierarchy process 394 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.413 figure 6. linguistic scale settings 2.4 obtaining results priorities of network nodes derived from the paired comparisons form the columns of the supermatrix (see figure 7). figure 7. limit supermatrix ijahp news: ferretti/software announcement: multichoice as new software for decision making with analytic network process international journal of the analytic hierarchy process 395 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.413 cluster’s priorities are used to weight the elements of the corresponding column blocks of the supermatrix. limit priorities are computed in the limit supermatrix. limit priority vector, normalized by cluster priority vector and limit priority vector for clusters are also shown in the multichoice window for supermatrices. all results can be quickly exported to ms excel. any results may be visualized by a column chart (see figures 8 and 9) and then saved to *.png format. figure 8. chart for the results: multiple column charts for supermatrix ijahp news: ferretti/software announcement: multichoice as new software for decision making with analytic network process international journal of the analytic hierarchy process 396 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.413 figure 9. chart for the results: 3d-column chart for limit normalized priorities to synthesize the results we select networks whose priorities should be inverted (or select nothing if there are no priorities to invert) and then obtain the results with additive and multiplicative types of composition. in multichoice we use the following formulas to obtain the results (for bocr model): (1) , (2) where – priorities of in benefits, opportunities, costs and risks networks respectively; – weights of networks benefits, opportunities, costs and risks respectively. the results of synthesis are shown in figure 10. ijahp news: ferretti/software announcement: multichoice as new software for decision making with analytic network process international journal of the analytic hierarchy process 397 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.413 figure 10. results of synthesis 2.5 sensitivity analysis sensitivity of the results to any changes of the initial priorities is realized through the line graph. firstly, we choose where priorities will be changing (in what hierarchy, network or control hierarchy) and then choose appropriate clusters or nodes. the selected priority will be changed from 0 to 1, and global priorities of the alternatives will be calculated. for the obtained points the line graph will be drawn. vertical lines on the graph mark an interval (5% on default) which illustrates actual increments of the initial priority. an example of sensitivity analysis is shown in figure 11. ijahp news: ferretti/software announcement: multichoice as new software for decision making with analytic network process international journal of the analytic hierarchy process 398 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.413 figure 11. sensitivity analysis 3. discussion anp and ahp are widely used for decision analysis in economic, political, social and technological applications (saaty &vargas, 2006). forman and gass (2001) believe that the real essence of ahp is not generally understood and ahp is more than just a methodology for choice. thus, the best way we can develop the anp/ahp technology is to implement all capabilities of the theory using modern software. the software must not only satisfy all of the user’s needs, but also it should be constantly evolving. thus, further implementation of anp/ahp algorithms in multichoice is essential: other types of elements evaluation, incomplete pairwise comparisons, various visualization capabilities (e.g. graphs for visualizing the results of linguistic estimation). a new round of development of anp/ahp technology can be implemented by creating a global decision-making web portal. a portal will accumulate and aggregate theory and best practice of anp/ahp and, mostly importantly, have a web interface for analyzing decision problems online. ijahp news: ferretti/software announcement: multichoice as new software for decision making with analytic network process international journal of the analytic hierarchy process 399 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.413 4. conclusions multichoice software has been created to support decision making with anp/ahp. the first version of the software has a user-friendly interface and contains all instruments to implement base functions of anp/ahp: model constructing, elements evaluation, different ways of to obtain, synthesize, visualize and export results, sensitivity analysis. future versions of the software may contain advanced functions related to modern decision making, such as different types of estimation (e.g. incomplete pairwise comparisons), different graphs for best data visualization, and anything that is needed by anp/ahp researchers. we also propose the idea of creating a web portal as a global resource for aggregating theory and practice of anp/ahp and making decisions online. the authors are open to all proposals and ideas for further development of the anp/ahp technology. ijahp news: ferretti/software announcement: multichoice as new software for decision making with analytic network process international journal of the analytic hierarchy process 400 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.413 references andreichikova o., milkova m. (2016). application of the analytic network process for comparative evaluation of the startups. journal of economy and entrepreneurship, 3(1), 785-792. forman eh, gass si. (2001). the analytic hierarchy process: an exposition. operations research, 49(4), 469-486. doi: http://dx.doi.org/10.1287/opre.49.4.469.11231 saaty tl, vargas lg. (2006). decision making with the analytic network process: economic, political, social and technological applications with benefits, opportunities, costs and risks. new york: springer. saaty tl. (2009). applications of analytic network process in entertainment. iranian journal of operations researc, 1(2), 41-55. saaty tl (2013) theory and applications of the analytic network process: decision making with benefits, opportunities, costs and risks. pittsburgh: rws publications. http://dx.doi.org/10.1287/opre.49.4.469.11231 ijahp article: saaty/aligning the measurement of tangibles with intangibles and not the converse international journal of the analytic hierarchy process 79 vol. 3 issue 1 2011 issn 1936-6744 aligning the measurement of tangibles with intangibles and not the converse thomas l. saaty university of pittsburgh saaty@katz.pitt.edu it is known in the ahp that when dealing with intangible alternatives they are compared with respect to the criteria and the resulting priorities are multiplied by the priorities of the respective criteria and then synthesized in either the distributive or the ideal mode. when the criteria are tangible the alternatives need to be measured on ratio scales. since ratio scales of measurement differ, they must be standardized and then weighted by the priorities of the criteria to trade off a unit of one scale against a unit of another. of course ratio scales suffer from the defect that they use a unit of \measurement uniformly whether the measurements are large or small. but the meaning and significance of these measurements may not reflect our actual preference for them because our ability to appreciate values differs when they are very large or very small. we need to obtain these measurements in relative form through prioritization and comparisons. if we insist on using them as they are, we must convert them to relative priorities by dividing each value by the total values. but we cannot do that without noting that the weights of the criteria in this case depend on the measurements of the alternatives with respect to these criteria. the analytic hierarchy process was developed to deal with models with intangible criteria or with both tangible and intangible criteria. sometimes critics, in trying to show that the ahp is not a valid theory, use a model with only tangible criteria, and, because they need this for their validation against some real world result obtained by applying an arithmetic formula, they choose tangible criteria with the same ratio scale. they then normalize the results for the alternatives under each criterion, assume the weights of the criteria are equal and get the wrong answer. how to combine normalized measurements of tangibles when the alternatives are tangible they can be dealt with in the manner described below. the eventual concern is to combine them with intangibles in a mathematically plausible way. if there are several criteria all measured on the same tangible scale, these criteria depend on the measurements of the alternatives for their priority and cannot be treated as if they are independent and assigned priority weights in advance. to reduce the overall weights of the alternatives to relative priority form, first the criteria are given relative weights computed from the sum of the readings under each one to the total readings (on the same scale) under the other criteria. these weights are used to weigh the normalized readings under each criterion. then the weighted relative readings are summed for each alternative with respect to all the criteria to obtain the overall priority of that alternative. in this manner the several criteria with different measurements for the alternatives under each are reduced to a single overall criterion with the alternatives having a normalized overall priority. that overall criterion is then compared in the usual way with other intangible criteria or overall tangible ones obtained in a similar way to the given overall ijahp article: saaty/aligning the measurement of tangibles with intangibles and not the converse international journal of the analytic hierarchy process 80 vol. 3 issue 1 2011 issn 1936-6744 tangible criterion. first here is an involving two tangible criteria c1 and c2 and three alternatives a1, a2 and a3 with respect to a single scale using dollar values. table 1 unnormalized criteria and alternative weights from measurements in the same dollar scale for both criteria alternatives criterion c1 unnormalized weight = 1.0 criterion c2 unnormalized weight = 1.0 weighted sum unnormalized normalized or relative values a1 200 150 350 350/1300=.269 a2 300 50 350 350/1300=.269 a3 500 100 600 600/1300=.462 totals 1000 300 1300 1 relative values require that criteria be examined as to their relative importance with respect to each other (on the average or on the whole). what is the relative importance of a criterion, or what numbers should the tangible criteria be assigned that reflect their relative importance? in the ahp when the values of the alternatives are measured on the same scale for several criteria, it is necessary that these criteria have priorities that reflect the proportion of the sum of the values under them to the total under all criteria. multiplying the relative values of the alternatives by the relative values of the criteria and adding gives the final column of table 2. the outcome coincides with the last column of table 1. table 2 normalized criteria weights and normalized alternative weights from measurements on the same ratio scale (additive synthesis) alternatives criterion c1 normalized weight = 1000/1300=0.7692 criterion c2 normalized weight = 300/1300=0.2308 weighted sum a1 200/1000 150/300 350/1300=.269 a2 300/1000 50/300 350/1300=.269 a3 500/1000 100/300 600/1300=.462 ijahp article: saaty/aligning the measurement of tangibles with intangibles and not the converse international journal of the analytic hierarchy process 81 vol. 3 issue 1 2011 issn 1936-6744 it is clear that the following computations are involved in the process of finding the necessary weights for the criteria. we must solve a system of simultaneous equations ∑ ∑ ∑ ∑∑= j i j i j ijijijjij aaaxa // from which we get ∑ ∑∑= j i j ijijj aax / comparing tangibles that require arithmetic formulas this example of expecting to rank rectangles by perimeter length by making pairwise comparisons of the respective lengths and widths, and not getting the right answer, is due to claudio garuti of chile. the problem is that one expects the ahp hierarchy to know the formula p = l × w for the perimeter of rectangles as they relate to the length l and width w without providing the right data. in a case like this, ahp can give the right answer, but information must be provided by setting up criteria for length and width and weighing the criteria by the proportion of the total linear measure they control. how to combine tangibles with intangibles assume that a family is considering buying a house and there are three houses to consider. four factors dominate their thinking as in the hierarchy of figure 1: the price of the house, the remodeling costs, the size of the house as reflected by its footage and the style of the house which is an intangible. they have looked at three houses with data shown in table 3 below on the quantifiables. goal choosing the best house úäääääâääáääâääää¿ criteria price remodeling size style alternatives a1 a2 a3 figure 1 three level hierarchy table 3 values of the alternatives with respect to the criteria; the first two are in thousands of dollars, the third in square feet and the forth in house styles to be prioritized ijahp article: saaty/aligning the measurement of tangibles with intangibles and not the converse international journal of the analytic hierarchy process 82 vol. 3 issue 1 2011 issn 1936-6744 price remodeling size style a 200 150 3000 colonial b 300 50 2000 ranch c 500 100 5500 split level first we normalize each of the columns of quantifiable factors as shown in table 4. table 4 normalized numerical columns price remodeling size style a 200/1000 150/300 3000/10500 colonial b 300/1000 50/300 2000/10500 ranch c 500/1000 100/300 5500/10500 split level next we combine the two factors with the common dollar scale by multiplying the numbers in the first column by 1000/1300 (the weight of c1) and those in the second by 300/1300 (the weight of c2) and adding as in table 5. table 5 combined first two columns that were measured in dollars cost size style a 350/1300 3000/10500 colonial b 350/1300 2000/10500 ranch c 600/1300 5500/10500 split level next we establish priorities for the criteria and also for the three different styles through paired comparisons as in the usual way. the resulting priorities are used to weigh the normalized values of the alternatives as they are given here. observation: we can alternatively, and better, perform paired comparisons on the measurements of the alternatives themselves and then weigh the four criteria through comparisons. the detailed approach illustrated above, would be probably used by a disinterested party for whom the dollar costs are simply numbers of no personal ijahp article: saaty/aligning the measurement of tangibles with intangibles and not the converse international journal of the analytic hierarchy process 83 vol. 3 issue 1 2011 issn 1936-6744 significance, although they might have substantially different significance to the family. to treat dollars (or any other scale measurements) directly without comparisons can yield misleading results. in this example, tangible and intangible factors had to be compared. their priorities are used to weigh the priorities of the alternatives. these priorities are obtained by converting measurements to priorities directly through normalization (seldom justified) or by interpreting their relative importance through judgment (essential where no measurements are possible). there are two other cases to consider. the first is when gambles and expected values are involved between criteria measured on the same scale which are then combined accordingly into a single criterion for that scale. the other may involve relations between subcriteria from a common scale and intangible criteria. the tangible subcriteria must be normalized with the other subcriteria as above and then combined with the intangibles accordingly. the other subcriteria from the given scale would be combined into a single scale criterion as above and one proceeds in the usual way to complete the prioritization process. in justifying the theory in a mathematical context, the invariance principle is the most important idea to remember. priorities are derived from judgments in a special way. the process involves a composition of priority vectors given as the columns of a matrix according to certain rules. for example, in additive composition, priority vectors are weighed (multiplied) by the priority of the element with respect to which they were derived and then summed over all such elements to obtain an overall composite priority vector. in particular, the columns of a judgment matrix are themselves priority vectors with respect to each element represented at the top of the matrix, and their composition, by multiplying by the priority of that element and adding across the elements, yields the composite which can be written as priority vector of all the elements. in the additive case, what we just described above 1 i=1,...,n n ij j i j a w cw = =∑ where c > 0 (which may be taken as c = 1) is a constant of proportionality so that the derived vector is a similarity transformation of the original vector and thus both belong to the same ratio scale. this is the principle of invariance of priorities. the foregoing problem has a unique solution in eigenvectors. ahp model and anp model for perimeter of rectangles the implicit assumption the critics make is that somehow the hierarchical structure is sufficient to give an expected answer regardless of the criteria weights. having the proper criteria weights is essential in ahp for it is in weighing the alternative priorities by the criteria priorities that the priorities are converted to commensurate absolute scales so they can be combined using addition in the synthesis process. to get the right weights for the criteria one must make the weight of each criterion proportionate to the amount of the total resource it controls, or alternatively, combine the measures using whatever arithmetic formula is being used to combine separate measures in the validation example, before normalizing, thus ending up with a single set of measures which can then be normalized. the example given here shows how to reproduce priority results for measures of tangibles involving formulas, even when it is not a simple add and weight ijahp article: saaty/aligning the measurement of tangibles with intangibles and not the converse international journal of the analytic hierarchy process 84 vol. 3 issue 1 2011 issn 1936-6744 formula as in the example below involving perimeters of rectangles that was suggested by claudio garuti of chile. validating ahp and anp results in situations involving tangibles and arithmetic formulas we have four alternative rectangles with dimensions of length and width given as shown in table 6. we wish to prioritize the rectangles by the lengths of their perimeters. this is an example of tangibles that are being combined using formulas and the question is whether ahp results match what would be expected using arithmetic. we show here that both ahp and anp give valid answers. table 6 four rectangles and their perimeters alternatives length width perimeter perimeter normalized rectangle 1 9 1 20 .25 rectangle 2 8 2 20 .25 rectangle 3 7 3 20 .25 rectangle 4 6 4 20 .25 sum 30 10 20 in attempting to validate the ahp people often set up a model of criteria and alternatives where measurements of the alternatives are made on both criteria using some existing ratio scale. they then give the criteria default equal priorities, reasoning that as both measurements on both criteria are made in the same ratio scale, they should be equally weighed. they then normalize the alternative measurements under each criterion, wrongly synthesize by weighing the normalized measures by the criteria weights and adding and do not get the expected priorities as shown in table 7. this example will show why. the expected answers are that the rectangles should have equal priority for perimeter length because we have an arithmetic formula that says p = 2l + 2w and applying this formula gives the answers for the perimeter, which we can then normalize and note that this is equivalent to priority and as all the rectangles have the same perimeter measurement, they should have equal priority. table 7 priorities obtained by normalizing then synthesizing with equal criteria weights ijahp article: saaty/aligning the measurement of tangibles with intangibles and not the converse international journal of the analytic hierarchy process 85 vol. 3 issue 1 2011 issn 1936-6744 alternatives criterion 1 length .5 criterion 2 width .5 priorities rectangle 1 9/30 1/10 =9/30×.5+1/10×.5=.2 rectangle 2 8/30 2/10 =8/30×.5+2/10×.5=.2333 rectangle 3 7/30 3/10 =7/30×.5+3/10×.5=.2666 rectangle 4 6/30 4/10 =6/30×.5+4/10×.5=.3000 sum 30 10 the correct way to set up the problem is to give the criteria their proper importance by weighing them by the proportion of the resource they control as shown in table 8. the total linear measure involved is 40 units. criterion 1 controls 30/40 or .75 of the total resource, the number of units, and criterion 2 controls 10/40 or .25, so these are the proper weights to use and the priorities are correct as shown below. table 8 priorities obtained by weighing criteria proportionately to the total resource controlled alternatives criterion 1 length 30/40=.75 criterion 2 width 10/40=.25 priorities rectangle 1 9/30 1/10 9/30×.75+1/10×.25=.25 rectangle 2 8/30 2/10 8/30×.75+2/20×.25=.25 rectangle 3 7/30 3/10 7/30×.75+3/10×.25=.25 rectangle 4 6/30 4/10 6/30×.75+4/10×.25=.25 sum 30 10 a second method that gives one the right answer is to use an anp model that involves dependence, feedback and input of the raw data directly in the supermatrix as shown in table 9. the superdecisions software can be used to set up a supermatrix as follows, first using the direct data, then normalizing the columns that are the rectangle measures with respect to the criteria and the criteria values with respect to the rectangles. table 9 anp supermatrix with raw data ijahp article: saaty/aligning the measurement of tangibles with intangibles and not the converse international journal of the analytic hierarchy process 86 vol. 3 issue 1 2011 issn 1936-6744 c1 (length) c2 (width) rectangle1 rectangle2 rectangle3 rectangle4 c1 (length) 0 0 9 8 7 6 c2 (width) 0 0 1 2 3 4 rectangle1 9 1 0 0 0 0 rectangle2 8 2 0 0 0 0 rectangle3 7 3 0 0 0 0 rectangle4 6 4 0 0 0 0 the superdecisions software for the anp will automatically convert the raw input data into priorities by normalizing it as shown in table 10. table 10 anp supermatrix of priorities c1 (length) c2 (width) rectangle1 rectangle2 rectangle3 rectangle4 c1 (length) 0 0 0.9 0.8 0.7 0.6 c2 (width) 0 0 0.1 0.2 0.3 0.4 rectangle1 0.3 0.1 0 0 0 0 rectangle2 0.26666667 0.2 0 0 0 0 rectangle3 0.23333333 0.3 0 0 0 0 rectangle4 0.2 0.4 0 0 0 0 raising the matrix to powers until it stabilizes in the limit supermatrix is shown in table 11. in this example, the powers of the matrix are actually cycling between two steady states that give the same final priorities. the supermatrix automatically gives the priorities of both the criteria and the rectangles without any special intercession on the part of the user. table 11 the limit supermatrix of the anp model with final priorities ijahp article: saaty/aligning the measurement of tangibles with intangibles and not the converse international journal of the analytic hierarchy process 87 vol. 3 issue 1 2011 issn 1936-6744 c1 (length) c2 (width) rectangle1 rectangle2 rectangle3 rectangle4 c1 (length) 0.0000 0.0000 0.7500 0.7500 0.7500 0.7500 c2 (width) 0.0000 0.0000 0.2500 0.2500 0.2500 0.2500 rectangle1 0.2500 0.2500 0.0000 0.0000 0.0000 0.0000 rectangle2 0.2500 0.2500 0.0000 0.0000 0.0000 0.0000 rectangle3 0.2500 0.2500 0.0000 0.0000 0.0000 0.0000 rectangle4 0.2500 0.2500 0.0000 0.0000 0.0000 0.0000 as shown in table 11, the priorities are 0.75 for the length criterion, 0.25 for width and 0.25 for each of the rectangles meaning their perimeter lengths are equal. these results match the expected results in table 6. rating scales rating scale values of different alternatives must be treated as ratio scale readings of tangibles because their already derived priorities belong to a scale of normalized values of an absolute scale. rating scale numbers may be the same for two criteria but their interpretation is different as priorities and they cannot be combined directly like numbers from the same ratio scale like money can. they must be weighed by the importance of the criteria, then combined. summarizing as a final comment we note that scales of measurement are very recent in human history and that before the last millennium people used their biological talent to make comparisons and that talent will not disappear just because we invented scales. in fact, even with scales, comparisons are used to determine how good the results are. in addition, no matter what scheme people use to evaluate alternatives with respect to criteria by rating or by comparison, comparisons will always be needed to prioritize criteria because their importance varies from one decision to another (they are of essence intangible) and also because their parent criterion or goal is an intangible. thus, our thinking needs to always make the particular or tangible a special case of the general or intangible and not the converse as some people have been inclined to do. rob typewritten text http://dx.doi.org/10.13033/ijahp.v3i1.91 ijahp news: petrillo, mu/ahp book announcements international journal of the analytic hierarchy process 385 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.419 ahp book announcements antonella petrillo parthenope university of naples department of mechanical engineering italy a.petrillo@unicas.it enrique mu carlow university department of business administration united states emu@carlow.edu two more books have been added to the ahp bibliography this month. the first is a book titled applications and theory in analytic hierarchy process decision making for strategic decisions which was edited by fabio de felice, thomas l. saaty and antonella petrillo. this book offers a very diverse compendium of ahp applications and some very interesting theoretical discussions. this book will constitute a valuable reference for the ahp scholar and practitioner. it is published by intech and is available both in open access (free) format and as a printed book. the second book is titled practical decision making: introduction to the analytic hierarchy process using super decisions v2 and is written by enrique mu and milagros pereyra-rojas and published by springer. this book is aimed at the novice who has a minimal mathematical background. the authors indicate that it was conceived to allow the reader to learn ahp on their own using the freely available software package super decisions v2. ahp scholars may also benefit from using this book for teaching purposes particularly because there are videos available to support the hands-on examples provided in the text. the flyer and general information for both books are provided below. ijahp news: petrillo, mu/ahp book announcements international journal of the analytic hierarchy process 386 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.419 applications and theory in analytic hierarchy process decision making for strategic decisions edited by fabio de felice, thomas l. saaty and antonella petrillo, isbn 978-953-512561-7, print isbn 978-953-51-2560-0, 320 pages, publisher: intech, chapters published august 31, 2016 under doi: 10.5772/61387 this book is about making decisions the natural way which we call the analytic hierarchy process (ahp). it involves assumptions about what people are observed to do with their biological equipment. they should not need to steep themselves for long in technical training to organize their thinking and discover what judgments they hold. they should be able to approach a decision problem by posing and answering the right kind of questions. the motivation beyond the book is the consciousness that we make important decisions every day, simple choices and hard choices. our lives are the sum of our decisions, whether in business or in personal spheres. often, when we decide is as important as what we decide. deciding too quickly can be hazardous; delaying too long can mean missed opportunities. in any case it is important to decide. decision making is fundamental to furthering our goal of survival and ensuring the quality of our life. to be a person is to be a decision maker. thus, the purpose of this book is to provide an introduction to the theory and applications in the field of decision making, especially focused on analytic hierarchy process. the idea of the book is to expand the reader’s consciousness to deal with problems regarding the decision making. this book presents some application examples of analytic hierarchy process. it contains original research and application chapters from different perspectives, and covers different areas such as supply chain, environmental engineering, safety, and social issues. this book is intended to be a useful resource for anyone who deals with decision making problems. the book is intended to be a useful resource for anyone who deals with decision making problems. furthermore, we hope that this book will provide useful resources ideas, techniques and methods for further research on analytic hierarchy process. link to download http://www.intechopen.com/books/applications-and-theory-of-analytic-hierarchyprocess-decision-making-for-strategic-decisions http://www.intechopen.com/books/editor/applications-and-theory-of-analytic-hierarchy-process-decision-making-for-strategic-decisions http://www.intechopen.com/books/applications-and-theory-of-analytic-hierarchy-process-decision-making-for-strategic-decisions http://www.intechopen.com/books/applications-and-theory-of-analytic-hierarchy-process-decision-making-for-strategic-decisions http://www.intechopen.com/books/applications-and-theory-of-analytic-hierarchy-process-decision-making-for-strategic-decisions ijahp news: petrillo, mu/ahp book announcements international journal of the analytic hierarchy process 387 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.419 ijahparticle: kohara, sugiyama/simulating tsunami evacuation with multi-agents and determining a countermeasure with ahp international journal of the analytic hierarchy process 430 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.406 simulating tsunami evacuation with multi-agents and determining a countermeasure with ahp kazuhira kohara chiba institute of technology japan k-kohara@jcom.zaq.ne.jp takuya sugiyama chiba institute of technology japan kohara.kazuhiro@p.chibakoudai.ac.jp abstract we propose an integration method that uses agent-based modeling to simulate tsunami evacuation and the analytic hierarchy process (ahp) to make a decision on a countermeasure. first, we created multiagent coast models that include a tsunami agent, shelter agents, road agents and evacuee agents. second, we divided the coast into several districts and predicted the tsunami evacuation success/failure number of each district by using a computer simulation with multiagent coast models. third, we considered several countermeasures (adding a shelter, adding an evacuation route) using that prediction. fourth, we estimated the effects of each countermeasure. finally, we use ahp to determine the best countermeasure against a tsunami disaster. keywords: agent-based modeling; tsunami disaster; countermeasure decision making 1. introduction multiagent-based social simulations have been extensively investigated, and various attempts have been made to apply them to the layout design of supermarkets, stock markets, sales prediction and tsunami evacuation (yamane et al. 2012; panayi et al. 2012; kohara et al. 2014; saito et al. 2005). the analytic hierarchy process (ahp) has been widely used for economic, political, social and corporate decision making (saaty, 1980; saaty et al. 1994; saaty, 2001; ginda et al. 2016). previously, we investigated the real-world problem of predicting sales for stores and using that prediction to determine where to locate a new store (kohara et al. 2014). we proposed an integrated method that uses agent-based modeling and the ahp to predict sales and to choose a new store location. first, we created multiagent town models that included store agents and consumer agents. we then estimated the predicted sales for each store by using a computer simulation based on multiagent town models. finally, we used ahp to determine the location of a new store. mailto:k-kohara@jcom.zaq.ne.jp mailto:kohara.kazuhiro@p.chibakoudai.ac.jp ijahparticle: kohara, sugiyama/simulating tsunami evacuation with multi-agents and determining a countermeasure with ahp international journal of the analytic hierarchy process 431 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.406 in this paper, we investigate another real-world problem of predicting the tsunami evacuation success/failure number and using that prediction to determine countermeasures against a tsunami disaster. we propose an integrated method that uses agent-based modeling and the ahp to predict the evacuation success/failure number and determine the countermeasure. first, we created multiagent coast models that consist of a tsunami agent, shelter agents, road agents and evacuee agents. then, we estimated the predicted evacuation success/failure number by using a computer simulation with multiagent coast models. finally, we used ahp to decide the best countermeasure against a tsunami disaster. the main features of our method are as follows: (1) we introduced a tsunami into the coast models as an agent. (2) we divided the coast into several districts, predicted the tsunami evacuation success/failure number of each district and considered several countermeasures (adding a shelter, adding an evacuation route) using that prediction. (3) we estimated the effects of each countermeasure. (4) we determined the countermeasure using the results of a multiagent simulation and ahp. 2. multiagent coast models we created a multiagent coast model based on a popular coast in shizuoka prefecture where it is predicted that a large earthquake will occur in the near future. the width of the coast is 2 km. since the size of the coast model is 100 cells by 200 cells, one cell corresponds to 10 m in each direction. evacuees move one cell per step and 100 m per minute, so a minute corresponds to 10 steps. evacuees move one cell per two steps on a sloping road. the tsunami moves two cells per step. here, we assumed that a 10 meter high tsunami arrives 10 minutes after the earthquake, and that 60% of evacuees start to evacuate immediately, 30% of evacuees start at 5 minutes after the earthquake, and 10% of evacuees start at the time of the arrival of the tsunami. these assumptions are based on results from a questionnaire carried out after the large earthquake that occurred in japan on march 11, 2011. there are four kinds of agents: a tsunami agent, shelter agents, road agents and evacuee agents. the number of evacuee agents is 3000, based on the published number of people bathing at the coast. the number of shelters is 10, based on the actual information. evacuee agents move to a higher location in the same way as in the related work (saito et al., 2005). when evacuees arrive at the intersection, they move according to traffic signs. if there is a shelter, they go to the shelter. otherwise, they move to a higher location. figures 1 and 2 show our multiagent coast model. in figure 1, green lines show flat roads, orange lines show sloping roads and light blue shows the sea. ijahparticle: kohara, sugiyama/simulating tsunami evacuation with multi-agents and determining a countermeasure with ahp international journal of the analytic hierarchy process 432 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.406 figure 1. multiagent coast model in figure 2, dark blue shows tsunami agents. the tsunami moves two cells per step. figure 2. multiagent coast model 3. predicting evacuation success/failure number first, we divided the coast into five districts (a, b, c, d and e) as shown in figure 3. we generated 600 people in each district and estimated the evacuation success number of each district. we examined an average number of 50 trials. table 1 shows the results. the evacuation success number of district c is comparatively small. ijahparticle: kohara, sugiyama/simulating tsunami evacuation with multi-agents and determining a countermeasure with ahp international journal of the analytic hierarchy process 433 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.406 figure 3. dividing the coast into five districts table 1 estimated evacuation success number of each district districts estimated evacuation success number district a 472 district b 475 district c 452 district d 480 district e 483 second, we generated 3,000 people on the coast and estimated the evacuation success number. again, the number of shelters is ten. we assumed that a 10 meter high tsunami arrives at 10 minutes after the earthquake and leaves at 15 minutes after the earthquake. we also examined an average number of 50 trials. table 2 shows the results. the number of evacuees who succeeded in reaching shelters was 1,397 and number of evacuees who succeeded in reaching high places was 1,076. therefore, the total number of successes was 2,473 and total number of failures was 527. table 2 results of tsunami evacuation simulation of the current state estimated number success number to reach shelters 1,397 success number to reach high places 1,076 total success number 2,473 total failure number 527 third, we added a shelter (shelter k in district b, shelter l in district c, or shelter m in district d) or an evacuation route (route x in district a, route y in district b, or route z in district c) based on the above results and the actual map, as shown in figures 4 and 5. ijahparticle: kohara, sugiyama/simulating tsunami evacuation with multi-agents and determining a countermeasure with ahp international journal of the analytic hierarchy process 434 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.406 figure 4. adding a shelter figure 5. adding an evacuation route table 3 shows the results when a shelter is added. the average failure number for the 50 trials was 449 when adding shelter k, 372 when adding shelter l, and 343 when adding shelter m. adding shelter m was the most effective, and adding shelter l was the second most effective. table 3 results of tsunami evacuation simulation in case of adding a shelter success to shelters success to high places total number of success total number of failure current state 1,397 1,076 2,473 527 adding shelter k 1,842 709 2,551 449 adding shelter l 1,962 667 2,628 372 adding shelter m 2,024 633 2,657 343 table 4 shows the results when an evacuation route was added. the average failure number for the 50 trials was 236 when adding route x, 230 when adding route y, and 164 when adding route z. adding route z was most the effective and adding route y was the second most effective. ijahparticle: kohara, sugiyama/simulating tsunami evacuation with multi-agents and determining a countermeasure with ahp international journal of the analytic hierarchy process 435 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.406 table 4 results of tsunami evacuation simulation in case of adding an evacuation route success to shelters success to high places total number of success total number of failure current state 1,397 1,076 2,473 527 adding route x 2,075 690 2,764 236 adding route y 1,874 897 2,770 230 adding route z 2,392 443 2,836 164 4. determining a countermeasure by using ahp figure 6 shows the relative measurement ahp model created for the task of deciding a tsunami evacuation measure. here, we used the following four criteria: feasibility, evacuation success rate, cost, and time required to realize the countermeasure. in feasibility, a countermeasure whose feasibility is high is important. in success rate, a countermeasure whose success rate is high is important. in cost, a countermeasure whose cost is low is important. in required time, a countermeasure whose required time is short is important. figure 6. determining a countermeasure with ahp here, we used the following four alternatives: (1) adding shelter l, (2) adding shelter m, (3) adding evacuation route y, and (4) adding evacuation route z. table 5 shows four alternatives for a countermeasure against a tsunami disaster. table 5 four alternatives for a countermeasure against tsunami disaster success to shelters success to high places total number of success total number of failure adding shelter l 1,962 667 2,628 372 adding shelter m 2,024 633 2,657 343 adding route y 1,874 897 2,770 230 adding route z 2,392 443 2,836 164 ijahparticle: kohara, sugiyama/simulating tsunami evacuation with multi-agents and determining a countermeasure with ahp international journal of the analytic hierarchy process 436 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.406 table 6 shows pairwise comparisons of four criteria when feasibility and success rate are most important. in this case, the weights of feasibility and success rate are the highest (their weights = 0.342). consistency index means whether a pair comparison matrix is consistent or not. when the index is lower than 0.10, we judge that the pair matrix is consistent. here, the consistency index is 0.041 and the pairwise comparisons are consistent. table 6 pairwise comparisons of four criteria when feasibility and success rate are most important feasibility success rate measures cost required time weight feasibility 1 1 2 3 0.342 success rate 1 1 2 3 0.342 measures cost 1/2 1/2 1 4 0.226 required time 1/3 1/3 1/4 1 0.091 consistency index = 0.041 table 7 shows pairwise comparisons of alternatives with respect to predicted feasibility. we will use existing hotels for shelters l and m, and construct new evacuation routes for y and z. therefore, the feasibility of shelters l and m is higher than that of routes y and z. route z is shorter than route y; therefore, the feasibility of route z is higher than that of route y. the weights of shelters l and m were the highest. table 7 pairwise comparisons of alternatives with respect to feasibility shelter l shelter m route y route z weight shelter l 1 1 6 2 0.368 shelter m 1 1 6 2 0.368 route y 1/6 1/6 1 1/5 0.054 route z 1/2 1/2 5 1 0.211 consistency index = 0.011 table 8 shows pairwise comparisons with respect to success rate. the success rate is 0.945 (2836/3000) when adding route z, 0.923 (2770/3000) when adding route y, 0.886 (2657/3000) when adding shelter m, and 0.876 (2628/3000) for shelter l. the weight of shelter z is the highest. table 8 pairwise comparisons of alternatives with respect to success rate shelter l shelter m route y route z weight shelter l 1 1/2 1/4 1/5 0.078 shelter m 2 1 1/3 1/4 0.125 route y 4 3 1 1/2 0.306 route z 5 4 2 1 0.492 consistency index = 0.016 ijahparticle: kohara, sugiyama/simulating tsunami evacuation with multi-agents and determining a countermeasure with ahp international journal of the analytic hierarchy process 437 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.406 table 9 shows pairwise comparisons with respect to the measure cost. we will use existing hotels for shelters l and m; therefore, the measure cost is comparatively low. shelter m is smaller than shelter l. route z is shorter than route y. the weight of shelter m is the highest. table 9 pairwise comparisons of alternatives with respect to measures cost shelter l shelter m route y route z weight shelter l 1 1 8 3 0.317 shelter m 1 1 9 4 0.499 route y 1/8 1/9 1 1/6 0.043 route z 1/3 1/4 6 1 0.146 consistency index = 0.012 table 10 shows pairwise comparisons with respect to required time. as route z is short, required time to construct route z is short. as shelter l is larger than shelter m, comparatively many rooms can be used for evacuees immediately. the weight of route z is the highest. table 10 pairwise comparisons of alternatives with respect to required time shelter l shelter m route y route z weight shelter l 1 2 5 1/2 0.289 shelter m 1/2 1 4 1/3 0.176 route y 1/5 1/4 1 1/6 0.059 route z 2 3 6 1 0.476 consistency index = 0.022 table 11 shows the final results when feasibility and success rate are most important. in this case, the weight of adding route z is highest because the weights of adding route z with respect to feasibility and success rate are comparatively high. table 11 final results of ahp when feasibility and success rate are most important alternatives results adding shelter l 0.250 adding shelter m 0.297 adding route y 0.138 adding route z 0.317 5. additional studies 5.1 additional study on changing tsunami height the tsunami height is based on the expected information which is between 5 and 10 meters. table 12 shows an additional study on tsunami height. first, we assumed the ijahparticle: kohara, sugiyama/simulating tsunami evacuation with multi-agents and determining a countermeasure with ahp international journal of the analytic hierarchy process 438 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.406 tsunami height is 10 meters. then, we changed the tsunami height to 5 meters. the evacuation failure number was 169. the tsunami height is crucial for tsunami evacuation; however, a countermeasure against a 10 meter high tsunami is important. table 12 additional study: changing tsunami height tsunami height success to shelters success to high places total number of success total number of failure 5 meters 1,420 1,411 2,831 169 10 meters 1,397 1,076 2,473 527 5.2 additional study on changing tsunami arrival time the tsunami arrival time is based on actual information. tsunamis have arrived at 10 minutes after the earthquake on average over the past hundred years. table 13 shows an additional study on tsunami arrival time. first, we assumed the tsunami arrives 10 minutes after the earthquake. then, we changed tsunami arrival time to 20 minutes or 30 minutes after the earthquake. the evacuation failure number was 508 for 20 minutes or 461 for 30 minutes. anyway, evacuate immediately! table 13 additional study: changing tsunami arrival time tsunami arrival time success to shelters success to high places total number of success total number of failure 10 minutes 1,397 1,076 2,473 527 20 minutes 1,860 632 2,492 508 30 minutes 1,903 636 2,539 461 5.3 additional study on changing tsunami speed table 14 shows an additional study on tsunami speed. first, we assumed the tsunami moves 2 times faster than people’s speed. then, we changed it so that the tsunami moves 4 times faster than people’s speed. the evacuation failure number was 922 when the tsunami moves 4 times faster than people’s speed. anyway, evacuate immediately! table 14 additional study: changing tsunami speed tsunami speed success to shelters success to high places total number of success total number of failure 2 times faster 1,397 1,076 2,473 527 4 times faster 1,347 731 2,078 922 5.4 additional study on changing percentage of evacuation consciousness table 15 shows an additional study on changing the percentage of evacuation consciousness. first, we assumed that 60% of evacuees start to evacuate immediately, 30% of evacuees start at 5 minutes after the earthquake, and 10% of evacuees start at the time of arrival of the tsunami, based on questionnaire results. then, we changed the ijahparticle: kohara, sugiyama/simulating tsunami evacuation with multi-agents and determining a countermeasure with ahp international journal of the analytic hierarchy process 439 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.406 percentage of evacuation consciousness into 70%:25%:5% and 70%:29%:1%. the evacuation failure number was 296 for 70%:25%:5% and 123 for 70%:29%:1%. again, evacuate immediately! table 15 additional study: changing percentage of evacuation consciousness percentage success to shelters success to high places total number of success total number of failure 60%:30%:10% 1,397 1,076 2,473 527 70%:25%:5% 1,584 1,120 2,704 296 70%:29%:1% 1,571 1,306 2,877 123 5.5 additional study on changing pairwise comparisons of four criteria table 16 shows an additional study on changing pairwise comparisons of the four criteria when required time is most important. in this case, the weight of required time is 0.549. table 16 pairwise comparisons of four criteria when required time is most important feasibility success rate measures cost required time weight feasibility 1 2 4 1/3 0.239 success rate 1/2 1 3 1/4 0.147 measures cost 1/4 1/3 1 1/6 0.067 required time 3 4 6 1 0.549 consistency index = 0.020 table 17 shows the final results when required time is the most important. in this case, the weight of route z is highest because the weights of route z with respect to the success rate and required time are comparatively high. table 17 final results of ahp when required time is most important alternatives results adding shelter l 0.279 adding shelter m 0.236 adding route y 0.093 adding route z 0.394 table 18 shows another additional study on changing pairwise comparisons of the four criteria when cost is most important. in this case, the weight of cost is 0.495. ijahparticle: kohara, sugiyama/simulating tsunami evacuation with multi-agents and determining a countermeasure with ahp international journal of the analytic hierarchy process 440 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.406 table 18 pairwise comparisons of four criteria when measures cost is most important feasibility success rate measures cost required time weight feasibility 1 3 1/2 5 0.310 success rate 1/3 1 1/4 3 0.134 measures cost 2 4 1 6 0.495 required time 1/5 1/3 1/6 1 0.061 consistency index = 0.026 table 19 shows the final results when cost is most important. in this case, the weight of shelter m is highest because the weights of shelter m with respect to feasibility and measures cost are comparatively high. table 19 final results of ahp when measures cost is most important alternatives results adding shelter l 0.299 adding shelter m 0.389 adding route y 0.083 adding route z 0.233 6. conclusion we proposed integrating agent-based modeling with the ahp for predicting a tsunami evacuation success/failure number and making decisions about countermeasures against a tsunami disaster. first, we created multiagent coast models that include a tsunami agent, shelter agents, road agents and evacuee agents. second, we divided the coast into five districts and estimated the evacuation success/failure number of each district by using a computer simulation with multiagent coast models. third, we added a shelter or an evacuation route and estimated the failure number. finally, we applied the ahp with four criteria. we applied our method to an actual coast and showed its effectiveness. we also reported additional studies on changing tsunami height, changing tsunami arrival time, changing tsunami speed, changing percentage of evacuation consciousness, and changing pairwise comparisons of four criteria. in future work, we will apply our method to other cases, other coasts and other types of ahp and anp for decision making (saaty, 1996; saaty et al. 2013). ijahparticle: kohara, sugiyama/simulating tsunami evacuation with multi-agents and determining a countermeasure with ahp international journal of the analytic hierarchy process 441 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.406 references ginda, g., & dytczak, m. (2016). a survey of ahp and anp applications in civil engineering and urban management. isahp 2016. kohara, k., & sekigawa, d. (2014). sales prediction with multiagent town models and deciding stores location with ahp. isahp 2014. panayi, e., et al. (2012). agent-based modeling of stock markets using existing order book data. mabs’12 (13th international workshop on multi-agent based simulation). doi: 10.1007/978-3-642-38859-0_8 saaty, t. (1980). the analytic hierarchy process. new york: mcgraw-hill. doi: http://dx.doi.org/10.1080/00137918308956077 saaty, t., & vargas, l. g. (1994). decision making in economic, political, social and technological environments with the analytic hierarchy process. pittsburgh: rws publications. doi:10.1016/0898-1221(95)90109-4 saaty, t. (1996). the analytic network process. arlington: expert choice. saaty, t. (2001). decision making for leaders: the analytic hierarchy process for decisions in a complex world, pittsburgh: rws publications. saaty, t., & vargas, l. g. (2013). decision making with the analytic network process: economic, political, social and technological applications with benefits, opportunities, costs and risks (second edition). springer verlag. doi: 10.1007/978-1-4614-7279-7_1 saito, t., & kagami, h. (2005). simulation of evacuation behavior from tsunami utilizing multiagent system. journal of architecture and planning, transactions of aij, 597, 229234. (in japanese) doi: http://doi.org/10.3130/aija.74.627 yamane, s. et al. (2012). agent-based social simulation for a checkout layout design of a specific supermarket. mabs’12 (13th international workshop on multi-agent based simulation). http://dx.doi.org/10.1080/00137918308956077 http://dx.doi.org/10.1016/0898-1221(95)90109-4 http://doi.org/10.3130/aija.74.627 ijahp article: bagheri fard sharabiani, gholamian, ghannadpour/a new solution for incomplete ahp model using goal programming and similarity function international journal of the analytic hierarchy process 1 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1003 a new solution for incomplete ahp model using goal programming and similarity function maryam bagheri fard sharabiani maryambagherifard4@gmail.com school of industrial engineering, iran university of science and technology mohammad reza gholamian gholamian@iust.ac.ir school of industrial engineering, iran university of science and technology seyed farid ghannadpour ghannadpour@iust.ac.ir school of industrial engineering, iran university of science and technology abstract the pairwise comparison matrix (pcm) is a crucial element of the analytic hierarchy process (ahp). in many cases, the pcm is incomplete and this complicates the decisionmaking process. hence, the present study offers a novel approach for dealing with incomplete information in group decision-making. we present a new model of incomplete ahp using goal programming (gp) and the similarity function. the minimization of this similarity function reduces errors in decision-making. the proposed model will be able to estimate the unknown elements in the pairwise comparison matrix and calculate the weight vectors obtained from the matrices. several examples are implemented to elaborate on the estimation of unknown elements and weight vectors in the proposed model. the results show that the unknown elements have an acceptable value with an appropriate consistency rate. keywords: multi-criteria decision-making (mcdm); analytic hierarchy process (ahp); incomplete ahp; goal programming (gp); similarity function 1. introduction in the multi-criteria decision-making (mcdm) process, the decision-maker (dm) attempts to use information and their knowledge of the subject under study to employ their opinions and professional background to make decisions. therefore, the information available for decision-making is the feedback provided by the decision-makers. when this information is not provided or is unavailable, it causes problems in decision-making and a failure to achieve managerial goals. therefore, it is vital to attain efficient methods to deal with decision problems in the case of incomplete information. mailto:maryambagherifard4@gmail.com mailto:gholamian@iust.ac.ir mailto:ghannadpour@iust.ac.ir ijahp article: bagheri fard sharabiani, gholamian, ghannadpour/a new solution for incomplete ahp model using goal programming and similarity function international journal of the analytic hierarchy process 2 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1003 the analytic hierarchy process (ahp) is one of the essential tools for multi-criteria decision-making (mcdm). this method helps measure the importance of several options relative to each other using pairwise comparisons when objective data for decisionmaking is not available. sometimes the data provided by decision-makers are incomplete, and there are various reasons for this, including the following (harker, 1987): (1) lack of enough time for decision-making (2) unwillingness to express an opinion (3) uncertainty about the opinion the application of pairwise comparison matrices (pcm) with complete information is of paramount importance when using the ahp technique. incomplete pairwise comparison matrices (ipcms) disrupt the decision-making process. due to the significance of this matter, numerous studies have been conducted on this topic. the most common and classic method in response to this issue is the revised geometric mean (rgm) method, which was proposed by harker (1987) based on the concept of "connecting path". the rgm method enables the decision-maker to achieve the priority vector gained from the pairwise comparison matrix even if there is incomplete information. simply put, in this approach there is no estimate for the unknown elements in the pairwise matrix, and only the weight priority vector can be reached. one of the other impressive studies in this domain is a study conducted by takahashi which proposed “the two-step method”. in the two-step method, the unknown elements in the pairwise comparison matrix are estimated and then the priority vector obtained from the matrix is calculated. however, in many cases, the priority vector gained from this method gives the same priority to options that do not have the same priority (takahashi, 1990). therefore, multiple studies have been conducted in connection with incomplete comparison matrices that attempt to solve the listed problems or develop these primary methods. some of these studies make decisions with incomplete information by utilizing optimization problems, among which the study of shiraishi et al. (1998) can be noted. in this research, polynomials constructed by the eigenvalues of the matrix (λ) were first extracted. the results indicated that if the value of the coefficient 𝐶3 = 0 is considered in a polynomial with n-3 degree, then the pairwise comparisons matrix will be consistent. since this property is not established for higher-order polynomials, we can reduce the inconsistency of matrices by exploiting this feature. ultimately, the value of unknown elements in incomplete matrices can be estimated. this technique is only applicable to detect an unknown element and is problematic for a more significant number of unknown elements. in another study, an approach to determine the priority of options from the incomplete pairwise comparison matrix is provided (jianqiang, 2006). the ternary ahp was used to make comparisons and obtain the prioritization vector of options. in this method, the order of priority of the options is obtained by solving two linear programming models. the ultimate priority of options is achieved by comparing the ascending and descending order obtained from these two linear programming models. in another study, bozoki et al. (2011) designed a linear system that monitors the matrix consistency ratio. thus, if the optimal value obtained from this model is more than a predetermined inconsistency threshold in their proposed model, there is no need to continue filling the matrix with unknown elements. instead, we must determine the elements that caused the inconsistencies in the matrix. moreover, dopazo and ruiz-tagle (2011) proposed a method for obtaining the group priority vector in an incomplete ijahp article: bagheri fard sharabiani, gholamian, ghannadpour/a new solution for incomplete ahp model using goal programming and similarity function international journal of the analytic hierarchy process 3 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1003 comparison matrix. they utilized a multi-objective optimization problem (mop) to approximate the priority vector achieved from the pairwise comparison matrix. then, the logarithmic goal programming formulation was adopted to convert the multi-objective planning problem to a single-objective optimization problem, and a weighted priority vector was obtained. in another study, benitez et al. (2015) used the ahp, a dynamic decision-making model, and added and removed a criterion to the pairwise comparison matrix, which turned the static data input mode into a dynamic one and completed the incomplete pairwise comparison matrix. in another study, faramondi et al. (2020) presented an approach to ranking options in the incomplete pairwise comparative matrix by extending the logarithmic least squares method in the information scatter mode. other studies that investigate decision-making under conditions of incomplete information are available. for example, van uden et al. (2002) tried to complete the ahp pairwise comparison matrix by employing the concept of the geometric mean. the estimation of unknown elements was complex for more than two-elements. nishizawa (2005) solved this problem by repeating the geometric mean method. furthermore, employing graph theory is one of the effective methods to deal with the incomplete pairwise comparison matrix, which can be observed in numerous studies. bernroider et al. (2010) suggested a method based on the k-walk procedure from graph theory to obtain the weighted priority in the incomplete pairwise comparison matrix. in another study, srdjevic et al. (2014) proposed a novel approach to complete incomplete ahp pairwise comparison matrices based on the connecting path method. also, chen et al. (2015) proposed a way to enhance the consistency of matrices. the connecting path method is used both to complete incomplete matrices and improve their consistency. benitez et al. (2019) attempted to perform consistent completion of unknown information in the incomplete comparison matrix by adopting graph theory. the results from the study of graph theory concerning pairwise comparison matrices suggest that if the graph resulting from the matrix is unique after completion, the best elements to complete the incomplete pairwise comparison matrices are selected. uncertainty management conducted by hua et al. (2008) proposed the priority vector in the incomplete pairwise comparison matrix, a combination of the ahp method and the dempster-shafer theory (ds-ahp). also, a ranking procedure by incomplete pcms using information entropy and the ds-ahp method was applied by pan et al. (2014). the framework defined in the ds-ahp method can perform decision-making in different modes, including complete and incomplete, fuzzy, uncertain information, etc. shiau (2012) applied the dempster-shafer method in evaluating sustainable transport, and hu et al. (2016) in train control information systems (tcis). hsu at al. (2011) found that the incomplete information in the pairwise comparison matrix could cause problems in the decision-making process. therefore, fuzzy methods can easily express the intuitions of decision-makers compared to classical ahp techniques. mcdm with incomplete linguistic preference relations is applied the same way. dong et al. (2015) applied incomplete information in group decision-making based on power geometric operators and triangular fuzzy ahp. moreover, jandova et al. (2016) estimated interval weights in incomplete pcms based on a weak consistency. one beneficial and applicable investigation on incomplete pairwise comparison matrices ijahp article: bagheri fard sharabiani, gholamian, ghannadpour/a new solution for incomplete ahp model using goal programming and similarity function international journal of the analytic hierarchy process 4 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1003 is presented by zhou et al. (2018). in this study, the unknown elements are completed in the pairwise comparison matrix based on the decision-making trial and evaluation laboratory (dematel) method. this algorithm transforms the direct-relation matrix to the total-relation by the dematel technique and can estimate unknown elements with a good consistency ratio. the application of neural networks to make decisions on incomplete information is discussed. hu and tsai (2006) designed an algorithm with the aid of a neural network with a backpropagation approach. the input for this algorithm is an incomplete pairwise comparison matrix, and its output is a complete pairwise comparison matrix. one of the weaknesses expressed in this model is that we can only identify an unknown element of the incomplete pairwise comparison matrix in the proposed model. thus, in another study, gomez-ruiz et al. (2010) estimated unknown values in the pairwise comparison matrix by designing an algorithm based on the most famous model of an artificial neural network named multi-layer perceptron. one of the critical features of this algorithm is that it is not merely limited to identifying an unknown element and can improve the matrix consistency while completing the pairwise comparison matrix. according to the latest applied ahp studies, a heuristic method to rate the alternatives in the ahp was applied in lin and kou (2020). a new parsimonious ahp methodology is introduced in abastante et al. (2019). in other studies, ruiz et al. (2020) proposed the gis-ahp method, amenta et al. (2021) dealt with the issue of aggregating judgments in non-negotiable ahp, and maleki et al. (2020) proposed a combination method of incomplete ahp and choquet integral. based on the studies reviewed, we conclude that most studies conducted in this area have dealt with the direct determination of the priority vector to accelerate the achievement of the vital goal of decision-making, which is the selection of choices and determination of their priority. simultaneous determination of weight vectors and unknown elements in the pairwise comparison matrix has not been well-studied. therefore, the current study presents a new approach to completing the pairwise comparison matrix in the analytic hierarchy process (ahp). this study aims to calculate the weight vector gained from incomplete pairwise comparison matrices in the ahp by employing the similarity function and goal programming (gp) techniques. additionally, we will strive to identify the unknown elements in the incomplete pairwise comparison matrix with the intended model. the gp technique was inspired by dopazo and ruiz-tagle (2011) and the total weights estimated in this study. the main difference between the proposed study from previous studies is that the unknown elements are estimated here. also, the weight vector is calculated in each group separately and the total weight is estimated by considering the impact coefficient. the results of this study have been compared to the results of zhou et al. (2018). 2. preliminaries definition 1. a matrix m is called a pairwise comparison matrix if the condition 𝑎𝑖𝑗 = 1 𝑎𝑗𝑖⁄ for all, j (saaty, 1998). ijahp article: bagheri fard sharabiani, gholamian, ghannadpour/a new solution for incomplete ahp model using goal programming and similarity function international journal of the analytic hierarchy process 5 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1003 definition 2. if 𝑀𝑛×𝑛 is a pairwise comparison matrix, 𝜆𝑚𝑎𝑥 is the maximum eigenvalue of this matrix, and i is an identity matrix; the weight vector (w) can be calculated using the following equation (saaty, 1998). (1) (𝑀 − 𝜆𝑚𝑎𝑥 × 𝐼) 𝑊 = 0 𝜆𝑚𝑎𝑥≥ n definition 3. to calculate the consistency ratio (cr) in the pairwise matrix 𝑀𝑛×𝑛, the consistency index must be first calculated as follows: (2) 𝐶. 𝐼. = 𝜆𝑚𝑎𝑥−𝑛 𝑛−1 thus, the consistency ratio is as follows: (3) 𝐶. 𝑅. = 𝐶. 𝐼. 𝑅. 𝐼. in equation (3), the random index r.i. is obtained from table 1. if the consistency ratio is less than 0.1, the matrix is consistent, and the resulting weights are acceptable. otherwise, the decision-makers must correct the pairwise comparison matrix (saaty, 1998). table 1 random index (saaty, 1998) 𝑵 1 2 3 4 5 6 7 8 9 10 𝑹𝑰 0 0 0.52 0.89 1.12 1.26 1.36 1.41 1.46 1.49 definition 4. the structure of the incomplete pcms is as below, where the * indicates unknown elements (harker, 1987). 𝑴=[ 1 ∗ 𝑎13 ∗ 1 𝑎23 𝑎31 𝑎32 1 ] definition 5. group decision-making (gdm) can be defined by a finite set of alternatives x = {𝑥1, 𝑥2, . . ., 𝑥𝑛} and a group of experts {𝐸1,..., 𝐸𝑚} in which each expert sorts alternatives through a set of criteria c = {𝑐1, 𝑐2, . . ., 𝑐𝑙} (dopazo & ruiz-tagle, 2011) . 3. proposed model in this section, the proposed model is elaborated. therefore, the symbols used in this section are presented in the table 2. ijahp article: bagheri fard sharabiani, gholamian, ghannadpour/a new solution for incomplete ahp model using goal programming and similarity function international journal of the analytic hierarchy process 6 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1003 table 2 introduction of symbols definition parameter definition variable index of decisionmaking groups k unknown pairwise matrix elements in each decisionmaking group m𝑖𝑗 𝑘 impact coefficient of each decision-making group 𝛼𝑘 weight of each matrix in each decision-making group 𝑤𝑖 𝑘 number of pairwise matrix elements in each decision-making group 𝐵𝑘 final weight 𝑤𝑖 coefficient (weight) of objective functions 𝜆𝑓 estimation error of pairwise matrix elements in each decision-making group 𝑡𝑘 maximum error d in this study, group decision-making was performed, in which the symbol k with the value of k=1, 2,…, m is defined for introducing the groups, and the coefficient 𝛼𝑘 is considered as the impact factor of each group. the introduced impact factor is based on the importance of the groups, and the preference of the opinions presented will have different weight values in the interval of 0 ˂ 𝛼𝑘 ˂ 1 and ∑ 𝛼𝑘 𝑚 𝑘=1 = 1. therefore, it is evident that if opinions are in a group with more importance, the value of the coefficient will be closer to one, and if is a group has less importance, this value will be closer to 0. the pairwise comparison matrices are presented to the decision-making groups, and the unknown values will be estimated by the developed model in each group. therefore, there are two types of weights in this study; one is the weight of each matrix in each decision-making group (𝑤𝑖 𝑘), and the other is the overall weight (𝑤𝑖), which is achieved from the weighted average of the groups. concerning the listed cases, a function to minimize the error value of decision-making is introduced called the similarity function 𝑓𝑘(𝑤). (4) sim (𝑚𝑘,𝑤𝑘) = 1 𝐵𝑘 ∑ |𝑚𝑖𝑗 𝑘 − 𝑤𝑖 𝑘 𝑤𝑗 𝑘| = 𝑓𝑘(𝑤)𝑖,𝑗 the first and second objective functions to minimize this expression are as follows (dopazo & ruiz-tagle, 2011): (5) min∑ 𝛼𝑘. 𝑓𝑘(𝑤) 𝑚 𝑘=1 (6) min 𝑚𝑎𝑥𝑘=1,…,𝑚 𝛼𝑘. 𝑓𝑘(𝑤) equations (5) and (6) are associated with the values 𝑓1(𝑤), 𝑓2(𝑤), … , 𝑓𝑘(𝑤); these functions indicate similarity functions in the first to k-th decision-making groups. therefore, the first objective function ∑ 𝛼𝑘. 𝑓𝑘(𝑤) 𝑚 𝑘=1 represents the weighted sum of deviations or errors obtained in the decision groups. it is clear that the lowering of this ijahp article: bagheri fard sharabiani, gholamian, ghannadpour/a new solution for incomplete ahp model using goal programming and similarity function international journal of the analytic hierarchy process 7 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1003 value reduces errors, and a more accurate estimation of problem variables is achieved. also, using the weighted sum of errors causes errors in less important decision-making groups to have less impact on the total value. in the second objective function, 𝑚𝑎𝑥𝑘=1,…,𝑚 𝛼𝑘. 𝑓𝑘(𝑤) indicates the maximum deviation. therefore, the maximum possible error value is minimized by considering the whole system's performance. the two objective functions introduced by equations (5) and (6) will be minimized. moreover, for the consistent estimation of unknown elements in each group, an error value called t k is considered. therefore, regarding the consistency ratio, we will use the following equations to consider the error t k : (7) 𝑚𝑖𝑗 𝑘 . 𝑚𝑗𝑠 𝑘 𝑚𝑖𝑠 𝑘 ≤ 𝑡 𝑘 (8) 𝑚𝑖𝑠 𝑘 𝑚𝑖𝑗 𝑘 𝑚𝑗𝑠 𝑘 ≤ 𝑡 𝑘 by minimizing this error, due to the elements of each matrix, the consistency ratio between the elements can be minimized. thus, the unknown elements can be estimated with a lower consistency ratio in each group. hence, to minimize 𝑡𝑘, another objective function is as follows: (9) min∑ 𝑡𝑘𝑚𝑘=1 thus, the overall objective function of the problem is composed of three goals. these objectives will be weighted by considering the coefficients 𝜆𝑓 and will be considered as an overall objective. in this case, given the number of the listed objectives, the sum of the coefficients 𝜆𝑓 will be as 𝜆𝑓1 + 𝜆𝑓2 + 𝜆𝑓3 = 1. subsequently, the presented planning will be modelled as goal programming (gp). the purpose of planning problems with goal programming is to attain the goal intended in the study. in the case of establishing this goal, we will obtain the best solution to the problem; otherwise, positive deviation and negative deviation from the goal are taken into account. if 𝑛𝑖𝑗 𝑘 and 𝑝𝑖𝑗 𝑘 are a negative deviation from the goal and a positive deviation from the goal by equations (10) and (11), (10) 𝑛𝑖𝑗 𝑘 = 1 2 [|𝑚𝑖𝑗 𝑘 − 𝑤𝑖 𝑘 𝑤𝑗 𝑘| + (𝑚𝑖𝑗 𝑘 − 𝑤𝑖 𝑘 𝑤𝑗 𝑘)] (negative) (11) 𝑝𝑖𝑗 𝑘 = 1 2 [|𝑚𝑖𝑗 𝑘 − 𝑤𝑖 𝑘 𝑤𝑗 𝑘| − (𝑚𝑖𝑗 𝑘 − 𝑤𝑖 𝑘 𝑤𝑗 𝑘)] (positive) respectively, by placing equation (4) in the objective function and considering the maximum error d, we will have: ijahp article: bagheri fard sharabiani, gholamian, ghannadpour/a new solution for incomplete ahp model using goal programming and similarity function international journal of the analytic hierarchy process 8 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1003 (12) d = 𝑚𝑎𝑥𝑘=1,…,𝑚 𝛼𝑘 𝐵𝑘 ∑ |𝑚𝑖𝑗 𝑘 − 𝑤𝑖 𝑘 𝑤𝑗 𝑘|𝑖,𝑗 accordingly, it can be concluded: (13) 𝛼𝑘 𝐵𝑘 ∑ |𝑚𝑖𝑗 𝑘 − 𝑤𝑖 𝑘 𝑤𝑗 𝑘|𝑖,𝑗  d k =1,2,…m moreover, considering the values of positive deviation (𝑝𝑖𝑗 𝑘 ) and negative deviation (𝑛𝑖𝑗 𝑘 ), the following statement will be obtained. corollary 3.1: for each pairwise comparison matrices in the goal programming model, we will have: (14) |𝑚𝑖𝑗 𝑘 − 𝑤𝑖 𝑘 𝑤𝑗 𝑘| = 𝑛𝑖𝑗 𝑘 + 𝑝𝑖𝑗 𝑘 proof 𝑛𝑖𝑗 𝑘 + 𝑝𝑖𝑗 𝑘 = 1 2 [|𝑚𝑖𝑗 𝑘 − 𝑤𝑖 𝑘 𝑤𝑗 𝑘| + (𝑚𝑖𝑗 𝑘 − 𝑤𝑖 𝑘 𝑤𝑗 𝑘)] + 1 2 [|𝑚𝑖𝑗 𝑘 − 𝑤𝑖 𝑘 𝑤𝑗 𝑘| − (𝑚𝑖𝑗 𝑘 − 𝑤𝑖 𝑘 𝑤𝑗 𝑘)] = 1 2 |𝑚𝑖𝑗 𝑘 − 𝑤𝑖 𝑘 𝑤𝑗 𝑘| + 1 2 (𝑚𝑖𝑗 𝑘 − 𝑤𝑖 𝑘 𝑤𝑗 𝑘) + 1 2 |𝑚𝑖𝑗 𝑘 − 𝑤𝑖 𝑘 𝑤𝑗 𝑘| 1 2 (𝑚𝑖𝑗 𝑘 − 𝑤𝑖 𝑘 𝑤𝑗 𝑘) = |𝑚𝑖𝑗 𝑘 − 𝑤𝑖 𝑘 𝑤𝑗 𝑘| □ accordingly, the following constraint will be added to the model according to corollary 3.1. 𝛼𝑘 𝐵𝑘 ∑ (𝑛𝑖𝑗 𝑘 + 𝑝𝑖𝑗 𝑘 ) ≤ d𝑖,𝑗 k =1,2,…m (15) thus, the transformation of the proposed model into goal programming is as follows: (16) min {(1 − (𝜆𝑓1 + 𝜆𝑓2)) ∑ . 𝛼𝑘 𝐵𝑘 ∑ |𝑛𝑖𝑗 𝑘 + 𝑝𝑖𝑗 𝑘 | + 𝜆𝑓1. 𝐷𝑖,𝑗 + 𝜆𝑓2. ∑ 𝑡 𝑘𝑚 𝑘=1 𝑚 𝑘=1 } s.t 𝛼𝑘 𝐵𝑘 ∑(𝑛𝑖𝑗 𝑘 + 𝑝𝑖𝑗 𝑘 ) ≤ d 𝑖,𝑗 k=1,2,…,m (17) 𝑚𝑖𝑗 𝑘 − 𝑤𝑖 𝑘 𝑤𝑗 𝑘 𝑛𝑖𝑗 𝑘 + 𝑝𝑖𝑗 𝑘 = 0 i=1,2,…,n j=1,2,…,n k=1,2,…,m (18) ∑ 𝑤𝑖 𝑘𝑛 𝑖=1 =1 k=1,2,…,m (19) 𝑚𝑖𝑗 𝑘 .𝑚𝑗𝑠 𝑘 𝑚𝑖𝑠 𝑘 ≤ 𝑡 𝑘 1˂i˂j˂s≤n k=1,2,…,m . (20) 𝑚𝑗𝑠 𝑘 𝑚𝑖𝑗 𝑘 𝑚𝑗𝑠 𝑘 ≤ 𝑡 𝑘 1˂i˂j˂s≤n k=1,2,…,m (21) ijahp article: bagheri fard sharabiani, gholamian, ghannadpour/a new solution for incomplete ahp model using goal programming and similarity function international journal of the analytic hierarchy process 9 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1003 𝑤𝑖 = ∑ 𝛼𝑘. 𝑤𝑖 𝑘 𝑚 𝑘=1 i=1,2,…,n (22) ∑ 𝑤𝑖 𝑛 𝑖=1 =1 (23) 𝑡𝑘˃0 , 𝑚𝑖𝑗 𝑘 ˃0 , 𝑊𝑖˃0 , 𝐷˃0 , 𝑤𝑖 𝑘˃0 , 𝑛𝑖𝑗 𝑘 ≥ 0 , 𝑝𝑖𝑗 𝑘 ≥ 0 (24) equation (18) is the goal constraint, aiming to achieve a zero-error rate of decisionmaking. 4. numerical example in this section, examples of incomplete pairwise comparison matrices in different dimensions, taken from the study of zhou et al. (2018) are solved. in the mentioned study, ipcms were introduced in two categories including with a full consistency ratio and without a full consistency ratio (as shown in the appendix). then, using the dematel method, a new estimation for unknown values of ipcms is proposed. this study was used in our work in such a way that we considered these categories as the responses of two groups of decision makers in solving the problem, and therefore, the results of zhou et al. (2018) can be considered as a comparison benchmark to the results of this study. considering 𝜆𝑓1, 𝜆𝑓2 = 0.1 and 𝛼1, 𝛼2 = 0.5, the results are reported in tables 3-7. many questions have been raised to validate the proposed model:  which matrices can be solved by the proposed model?  what is the effect of the proposed model on matrices with perfect consistency (cr = 0) and incomplete consistency?  will the final weight priority vector provide the decision-maker with different and acceptable weights? to respond to the above questions, the following two steps were taken: step 1: the proposed model is employed to estimate the unknown elements in incomplete comparison matrices. to this end, several matrices are placed into two decision-making groups. the first group of matrices has a consistency ratio, and the second group has matrices with a perfect consistency ratio (cr = 0). the unknown elements are estimated, and the results are reported in the top two rows of tables 3-7. to evaluate the consistency ratio, the same value is considered for the impact factor in each group. also, the matrix dimension and the number of unknown elements increased. the missing values are estimated by the proposed model and compared with the dematel as a benchmark method. step 2: in this step, the ipcms completed by the dematel and the weight priority vector were calculated using the eigenvector method. then, the weight vector obtained from the developed model is compared with the eigenvector as a benchmark method and the results are reported in the following sections. ijahp article: bagheri fard sharabiani, gholamian, ghannadpour/a new solution for incomplete ahp model using goal programming and similarity function international journal of the analytic hierarchy process 10 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1003 4.1. matrix collection in this section, the ipcms in the study of zhou et al. (2018) are exploited for validation. these ipcms have different dimensions which are presented in two groups. 4.2. validation result and discussion the results gained from the first and second validation stages are reported in tables 3-7. table 3 validation results of 4 × 4 matrix m23=1.93 m24=1.25 𝒎𝟑𝟐=0.516 𝒎𝟑𝟒=0.333 𝒎𝟒𝟐=0.8 𝒎𝟒𝟑=1.55 group 1 estimation of unknown elements with research model 𝒎𝟏𝟐=2 𝒎𝟏𝟒=8 𝒎𝟐𝟏=0.5 𝒎𝟐𝟒=4 𝒎𝟒𝟏=0.125 𝒎𝟒𝟐=0.25 group 2 𝒎𝟐𝟑=2 𝒎𝟐𝟒=1.25 𝒎𝟑𝟐=0.52 𝒎𝟑𝟒=0.65 𝒎𝟒𝟐=0.8 𝒎𝟒𝟑=1.54 group 1 estimation of unknown elements with dematel model 𝒎𝟏𝟐=2 𝒎𝟏𝟒=8 𝒎𝟐𝟏=0.5 𝒎𝟐𝟒=4 𝒎𝟒𝟏=0.13 𝒎𝟒𝟐=0.25 group2 cr = 0 group 1 consistency ratio cr = 0.001 group 2 w = [0.395 0.294 0.149 0.162] _ calculation of weight priority vector with the research model w = [0.393 0.310 0.158 0.139] _ calculation of weight priority vector by eigenvector method and complete dematel matrix ijahp article: bagheri fard sharabiani, gholamian, ghannadpour/a new solution for incomplete ahp model using goal programming and similarity function international journal of the analytic hierarchy process 11 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1003 table 4 validation results of 5 × 5 matrix 𝒎𝟏𝟐=0.889 𝒎𝟏𝟑= 1.600 𝒎𝟐𝟏=0.5 𝒎𝟐𝟓= 9 𝒎𝟑𝟏=0.625 𝒎𝟑𝟒=1.69 𝒎𝟒𝟑=0.36 𝒎𝟓𝟐=0.11 group 1 estimation of unknown elements with research model 𝒎𝟐𝟑=1 𝒎𝟐𝟓=4 𝒎𝟑𝟐=1 𝒎𝟑𝟓=4 𝒎𝟒𝟓=2 𝒎𝟓𝟐=0.25 𝒎𝟓𝟑=0.25 𝒎𝟓𝟒=0.5 group 2 𝒎𝟏𝟐=0.889 𝒎𝟏𝟑= 1.69 𝒎𝟐𝟏=1.125 𝒎𝟐𝟓=9 𝒎𝟑𝟏=0.625 𝒎𝟑𝟒=2.77 𝒎𝟒𝟑=0.36 𝒎𝟓𝟐=0.11 group 1 estimation of unknown elements with dematel model 𝒎𝟐𝟑=1 𝒎𝟐𝟓=4 𝒎𝟑𝟐=1 𝒎𝟑𝟓=4 𝒎𝟒𝟓=2 𝒎𝟓𝟐=0.25 𝒎𝟓𝟑=0.25 𝒎𝟓𝟒=0.5 group2 cr=0.014 group 1 consistency ratio cr=0.002 group 2 w= [0.372 0.287 0.206 0.089 0.046] _ calculation of weight priority vector with the research model w= [0.375 0.292 0.196 0.091 0.044] _ calculation of weight priority vector by eigenvector method and complete dematel matrix ijahp article: bagheri fard sharabiani, gholamian, ghannadpour/a new solution for incomplete ahp model using goal programming and similarity function international journal of the analytic hierarchy process 12 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1003 table 5 validation results of 6 × 6 matrix 𝒎𝟏𝟑=3.02 𝒎𝟏𝟔= 1.27 𝒎𝟐𝟒=0.39 𝒎𝟐𝟔=0.26 𝒎𝟑𝟏=0.33 𝒎𝟑𝟓=3.38 𝒎𝟒𝟐=2.56 𝒎𝟒𝟔=0.80 𝒎𝟓𝟑=0.3 𝒎𝟔𝟏=0.79 𝒎𝟔𝟐=3.87 𝒎𝟔𝟒=1.25 group 1 estimation of unknown elements with research model 𝒎𝟏𝟐=1.502 𝒎𝟏𝟒= 3.38 𝒎𝟐𝟏=0.67 𝒎𝟐𝟒=2.25 𝒎𝟑𝟒=1.5 𝒎𝟒𝟏=0.30 𝒎𝟒𝟐=0.44 𝒎𝟒𝟑=0.6 𝒎𝟒𝟓=1.5 𝒎𝟓𝟒=0.67 𝒎𝟔𝟓=0.67 group 2 𝒎𝟏𝟑=3.02 𝒎𝟏𝟔= 1.27 𝒎𝟐𝟒=0.39 𝒎𝟐𝟔=0.26 𝒎𝟑𝟏=0.33 𝒎𝟑𝟓=3.38 𝒎𝟒𝟐=2.56 𝒎𝟒𝟔=0.8 𝒎𝟓𝟑=0.3 𝒎𝟔𝟏=0.79 𝒎𝟔𝟐=3.87 𝒎𝟔𝟒=1.25 group 1 estimation of unknown elements with dematel model 𝒎𝟏𝟐=1.5 𝒎𝟏𝟒= 3.38 𝒎𝟐𝟏=0.67 𝒎𝟐𝟒=2.25 𝒎𝟑𝟒=1.5 𝒎𝟒𝟏=0.30 𝒎𝟒𝟐=0.44 𝒎𝟒𝟑=0.67 𝒎𝟒𝟓=1.5 𝒎𝟓𝟒=0.67 𝒎𝟔𝟓=0.67 group2 cr= 0.048 group 1 consistency ratio cr= 0.001 group 2 w= [0.339 0.153 0.133 0.158 0.062 0.155] _ calculation of weight priority vector with the research model w= [0.380 0.139 0.1494 0.1497 0.057 0.119] _ calculation of weight priority vector by eigenvector method and complete dematel matrix ijahp article: bagheri fard sharabiani, gholamian, ghannadpour/a new solution for incomplete ahp model using goal programming and similarity function international journal of the analytic hierarchy process 13 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1003 table 6 validation results of 7 × 7 matrix 𝒎𝟏𝟒=0.259 𝒎𝟏𝟕= 0.778 𝒎𝟐𝟑=6.11 𝒎𝟐𝟓=0.873 𝒎𝟐𝟕=2.037 𝒎𝟑𝟐=0.164 𝒎𝟑𝟔=0.214 𝒎𝟒𝟏=4.13 𝒎𝟒𝟓=1.286 𝒎𝟓𝟐=2.39 𝒎𝟓𝟒=0.99 𝒎𝟔𝟑=5.62 𝒎𝟔𝟕=1.556 𝒎𝟕𝟏=1.286 𝒎𝟕𝟐=0.491 𝒎𝟕𝟔=0.643 group 1 estimation of unknown elements with research model 𝒎𝟏𝟐 𝒎𝟏𝟑= 1 𝒎𝟏𝟒=2 𝒎𝟏𝟔=8 𝒎𝟏𝟕=8 𝒎𝟐𝟏=1 𝒎𝟐𝟑=1 𝒎𝟐𝟒=2 𝒎𝟐𝟓=4 𝒎𝟑𝟏=1 𝒎𝟑𝟐=1 𝒎𝟒𝟏=0.5 𝒎𝟒𝟐=0.5 𝒎𝟓𝟐=0.25 𝒎𝟔𝟏=0.125 𝒎𝟕𝟏=0.125 group 2 𝒎𝟏𝟒=0.24 𝒎𝟏𝟕= 1.2 𝒎𝟐𝟑=6.11 𝒎𝟐𝟓=0.42 𝒎𝟐𝟕=1.10 𝒎𝟑𝟐=0.16 𝒎𝟑𝟔=0.18 𝒎𝟒𝟏=4.13 𝒎𝟒𝟓=1.01 𝒎𝟓𝟐=2.39 𝒎𝟓𝟒=0.99 𝒎𝟔𝟑=5.62 𝒎𝟔𝟕=1.59 𝒎𝟕𝟏=0.83 𝒎𝟕𝟐=0.91 𝒎𝟕𝟔=0.63 group 1 estimation of unknown elements with dematel model 𝒎𝟏𝟐=1 𝒎𝟏𝟑= 1 𝒎𝟏𝟒=2 𝒎𝟏𝟔=8 𝒎𝟏𝟕=8 𝒎𝟐𝟏=1 𝒎𝟐𝟑=1 𝒎𝟐𝟒=2 𝒎𝟐𝟓=4 𝒎𝟑𝟏=1 𝒎𝟑𝟐=1 𝒎𝟒𝟏=0.5 𝒎𝟒𝟐=0.5 𝒎𝟓𝟐=0.25 𝒎𝟔𝟏=0.125 𝒎𝟕𝟏=0.125 group 2 cr= 0.072 group 1 consistency ratio cr= 0 group 2 w= [0.160 0.217 0.140 0.198 0.137 0.086 0.061] calculation of weight priority vector with the research model w= [0.168 0.208 0.146 0.224 0.098 0.090 0.063] calculation of weight priority vector by eigenvector method and complete dematel matrix ijahp article: bagheri fard sharabiani, gholamian, ghannadpour/a new solution for incomplete ahp model using goal programming and similarity function international journal of the analytic hierarchy process 14 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1003 table 7 validation results of 8 × 8 matrix 𝒎𝟏𝟐=2 𝒎𝟏𝟑= 2 𝒎𝟏𝟕=0.667 𝒎𝟏𝟖=0.75 𝒎𝟐𝟏=0.5 𝒎𝟐𝟓=2.67 𝒎𝟐𝟕=0.33 𝒎𝟑𝟏=0.52 𝒎𝟐𝟑=1 𝒎𝟑𝟐=0.93 𝒎𝟑𝟖=0.38 𝒎𝟒𝟔=1 𝒎𝟓𝟐=0.33 𝒎𝟓𝟕=0.11 𝒎𝟔𝟒=1 𝒎𝟕𝟏=1.5 𝒎𝟕𝟐=3 𝒎𝟕𝟓=9 𝒎𝟕𝟖=1.125 𝒎𝟖𝟏=1.3 𝒎𝟖𝟑=2.667 𝒎𝟖𝟕=0.889 group 1 estimation of unknown elements with research model 𝒎𝟏𝟓=0.5 𝒎𝟏𝟖= 2 𝒎𝟐𝟑=0.25 𝒎𝟐𝟕=0.25 𝒎𝟐𝟖=1 𝒎𝟑𝟐=4 𝒎𝟑𝟓=1 𝒎𝟑𝟖=4 𝒎𝟒𝟔=1 𝒎𝟒𝟕=0.25 𝒎𝟓𝟏=2 𝒎𝟓𝟑=1 𝒎𝟔𝟒=1 𝒎𝟔𝟖=1 𝒎𝟕𝟐=4 𝒎𝟕𝟒=4 𝒎𝟕𝟖=4 𝒎𝟖𝟏=0.5 𝒎𝟖𝟐=1 𝒎𝟖𝟑=0.25 𝒎𝟖𝟔=1 𝒎𝟖𝟕=0.25 group 2 𝒎𝟏𝟐=1.78 𝒎𝟏𝟑= 1.93 𝒎𝟏𝟕=0.69 𝒎𝟏𝟖=0.81 𝒎𝟐𝟏=0.56 𝒎𝟐𝟑=1.08 𝒎𝟐𝟓=2.67 𝒎𝟐𝟕=0.40 𝒎𝟑𝟏=0.52 𝒎𝟑𝟐=0.93 𝒎𝟑𝟖=0.42 𝒎𝟒𝟔=0.81 𝒎𝟓𝟐=0.37 𝒎𝟓𝟕=0.1 1 𝒎𝟔𝟒=1.23 𝒎𝟕𝟏=1.45 𝒎𝟕𝟐=2.52 𝒎𝟕𝟓=9 𝒎𝟕𝟖=1.29 𝒎𝟖𝟏=1.23 𝒎𝟖𝟑=2.37 𝒎𝟖𝟕=0.78 group 1 estimation of unknown elements with dematel model 𝒎𝟏𝟓=0.5 𝒎𝟏𝟖= 2 𝒎𝟐𝟑 = 𝟎. 𝟐𝟓 𝒎𝟐𝟕=0.25 𝒎𝟐𝟖=1 𝒎𝟑𝟐=4 𝒎𝟑𝟓=1 𝒎𝟑𝟖=4 𝒎𝟒𝟔=1 𝒎𝟒𝟕 = 𝟎. 𝟐𝟓 𝒎𝟓𝟏=2 𝒎𝟓𝟑=1 𝒎𝟔𝟒=1 𝒎𝟔𝟖=1 𝒎𝟕𝟐=4 𝒎𝟕𝟒=4 𝒎𝟕𝟖=4 𝒎𝟖𝟏=0.5 𝒎𝟖𝟐=1 𝒎𝟖𝟑=0.25 𝒎𝟖𝟔=1 𝒎𝟖𝟕=0.25 group 2 cr= 0.021 group 1 consistency ratio cr= 0 group 2 w= [0.149 0.075 0.158 0.043 0.127 0.044 0.252 0.153] _ calculation of weight priority vector with the research model w= [0.159 0.079 0.156 0.042 0.091 0.047 0.287 0.135] _ calculation of weight priority vector by eigenvector method and complete dematel matrix we infer that the proposed model can be employed in all pairwise comparison matrices from the results achieved from this validation. the elements estimated by the estimation model provide a desirable consistency ratio, which means we can trust the results and ijahp article: bagheri fard sharabiani, gholamian, ghannadpour/a new solution for incomplete ahp model using goal programming and similarity function international journal of the analytic hierarchy process 15 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1003 accept the obtained priority vector. moreover, the elements estimated by the research model were equal to the ones obtained by the dematel technique, in many cases representing an acceptable estimation of unknown data. the consistency ratio (cr) in a complete pcm with estimated data is less than 0.1 and is considered a good consistency ratio. the priority vector gained from the model does not have equal or inverse weights relative to the vector obtained by the eigenvector method. hence, we can rely on the results of the proposed model. in matrices whose consistency ratio is 0 (cr = 0), the estimation of the unknown elements by the proposed model is precisely equal to that of the dematel technique. therefore, when the consistency ratio for a matrix is 0, the value of the decision error will be 0, and the estimation of unknown elements will be accurate and unique. 4.3. verification results and discussion in order to verify the proposed model, the following question will be answered: how will the matrices' consistency ratio vary with an increasing number of unknown elements in a fixed dimension? in this section, by fixing the dimensions of the matrices we deal with increasing the number of unknown elements and their relationship with the consistency ratio. the matrices employed in this section are the matrices used in zhou et al. (2018), which were examined in different dimensions. the row corresponding to the unknown element pair represents the number of unknown elements in the pairwise matrix. (for example, if an element 𝑚12 is unknown, its inverse element 𝑚21 is also unknown). the first group includes matrices with a consistency ratio less than 0.1, and the second group comprises matrices with a 0-consistency ratio. figures 2-6 illustrate the relationships between the number of unknown elements and their related consistency ratio (cr). microsoft excel was used to illustrate these diagrams. figure 2 variations of cr in 4 × 4 matrix 0 0.02 0.04 0.06 1 2 3 c r unknown pairs variations of cr group 1 group 2 ijahp article: bagheri fard sharabiani, gholamian, ghannadpour/a new solution for incomplete ahp model using goal programming and similarity function international journal of the analytic hierarchy process 16 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1003 figure 3 variations of cr in 5 × 5 matrix figure 4 variations of cr in 6 × 6 matrix figure 5 variations of cr in 7 × 7 matrix 0 0.02 0.04 0.06 0.08 1 2 3 4 c r unknown pairs variation of cr group 1 group 2 0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 1 2 3 4 5 6 c r unknown pairs variation of cr group 1 group 2 0 0.01 0.02 0.03 0.04 0.05 1 2 3 4 5 6 7 8 c r unknown pairs variation of cr group 1 group 2 ijahp article: bagheri fard sharabiani, gholamian, ghannadpour/a new solution for incomplete ahp model using goal programming and similarity function international journal of the analytic hierarchy process 17 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1003 figure 6 variations of cr in 8 × 8 matrix as inferred from the plotted diagrams in the figures above, the consistency ratio (cr) will remain 0, enhancing the number of unknown values for matrices with perfect consistency. furthermore, the blue line shows that for matrices with inconsistency, by enhancing the number of unknown elements, the consistency ratio (cr) decreases. the relation (𝐶𝑅)ʹ < cr holds through all points of the diagrams, where cr is the initial consistency of the matrix and (𝐶𝑅)ʹis the rate of consistency in the incomplete matrix. the obtained results are in accordance with zhou et al. (2018). moreover, the model proposed here could solve matrices with less than 30% unknown elements. 5. conclusion considering that the core of the analytic hierarchy process (ahp) is the availability of an information matrix for the decision-making process, an incomplete matrix resulting from a lack of information disrupts this process. therefore, in this study, by concerning the incomplete information in the ahp and defining the variables and their relationship with each other, the mathematical model is investigated, and these relationships are provided in the form of a multi-objective mathematical programming model. this study aims to present a model to estimate unknown matrix elements and the resulting weight priority vector by using goal programming and the similarity function. the developed model in this investigation can estimate 30% of the unknown elements. in zhou et al.'s (2018) study, only the unknown elements are estimated. however, unknown elements and pcm weights were simultaneously calculated in this study. moreover, the total weight priority vector was easily obtained. this study can be used in mcdm problems when there is the possibility of incomplete information from the experts. it can be beneficial in emergencies, where time is critical and access to complete information and application of decision-making procedures is time-consuming. also, the state of uncertainty and interaction between factors are important to achieve better results in the mcdm technique. therefore, we can make a compressive decision in future studies about these elements. 0 0.02 0.04 0.06 0.08 1 2 3 4 5 6 7 8 c r unknown pairs variation of cr group 1 group 2 ijahp article: bagheri fard sharabiani, gholamian, ghannadpour/a new solution for incomplete ahp model using goal programming and similarity function international journal of the analytic hierarchy process 18 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1003 appendix table a ipcm benchmark matrices used from zhou et al. (2018) group 1: group 2: 4  4 matrix [ 1 0.8 1.55 1 1.25 1 ∗ ∗ 0.65 ∗ 1 ∗ 1 ∗ ∗ 1 ] [ 1 ∗ 4 ∗ ∗ 1 2 ∗ 0.25 0.5 1 2 ∗ ∗ 0.5 1 ] 5  5 matrix [ 1 ∗ ∗ 5 8 ∗ 1 3 5 ∗ ∗ 0.33 1 ∗ 5 0.2 0.2 ∗ 1 3 0.13 ∗ 0.2 0.33 1] [ 1 2 2 4 8 0.5 1 ∗ 2 ∗ 0.5 ∗ 1 2 ∗ 0.25 0.5 0.5 1 ∗ 013 ∗ ∗ ∗ 1] 6  6 matrix [ 1 5 ∗ 3 6 ∗ 0.2 1 0.33 ∗ 3 ∗ ∗ 3 1 0.5 ∗ 0.33 0.33 ∗ 2 1 5 ∗ 0.17 0.33 ∗ 0.2 1 0.2 ∗ ∗ 3 ∗ 5 1 ] [ 1 ∗ 2.25 ∗ 5.06 7.6 ∗ 1 1.5 ∗ 3.38 5.06 0.44 0.67 1 ∗ 2.25 3.38 ∗ ∗ ∗ 1 ∗ 2.25 0.2 0.3 0.44 ∗ 1 ∗ 0.13 0.2 0.3 0.44 ∗ 1 ] 7  7 matrix [ 1 0.25 5 ∗ 0.33 0.5 ∗ 4 1 ∗ 0.33 ∗ 0.25 ∗ 0.2 ∗ 1 0.14 0.14 ∗ 0.33 ∗ 3 7 1 ∗ 2 3 3 ∗ 7 ∗ 1 2 3 2 4 ∗ 0.5 0.5 1 ∗ ∗ ∗ 3 0.33 0.33 ∗ 1 ] [ 1 ∗ ∗ ∗ 4 ∗ ∗ ∗ 1 ∗ ∗ ∗ 8 8 ∗ ∗ 1 2 4 8 8 ∗ ∗ 0.5 1 2 4 4 0.25 ∗ 0.25 0.5 1 2 2 ∗ 0.13 0.13 0.25 0.5 1 1 ∗ 0.13 0.13 0.25 0.5 1 1] 8  8 matrix [ 1 ∗ ∗ 7 6 6 ∗ ∗ ∗ 1 ∗ 5 ∗ 3 ∗ 0.14 ∗ ∗ 1 4 3 3 0.17 ∗ 0.14 0.2 0.25 1 1 ∗ 0.11 0.13 0.17 ∗ 0.33 1 1 1 ∗ 0.11 0.17 0.33 0.33 ∗ 1 1 0.11 0.17 ∗ ∗ 6 9 ∗ 9 1 ∗ ∗ 7 ∗ 8 9 6 ∗ 1 ] [ 1 2 0.5 2 ∗ 2 0.5 ∗ 0.5 1 ∗ 1 0.25 1 ∗ ∗ 2 ∗ 1 4 ∗ 4 1 ∗ 0.5 1 0.25 1 0.25 ∗ ∗ 1 ∗ 4 ∗ 4 1 4 1 4 0.5 1 0.25 ∗ 0.25 1 0.25 ∗ 2 ∗ 1 ∗ 1 4 1 ∗ ∗ ∗ ∗ 1 0.25 ∗ ∗ 1] ijahp article: bagheri fard sharabiani, gholamian, ghannadpour/a new solution for incomplete ahp model using goal programming and similarity function international journal of the analytic hierarchy process 19 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1003 references abastante, f., corrente, s., greco, s., ishizaka, a., & lami, i.m. (2019). a new parsimonious ahp methodology: assigning priorities to many objects by comparing pairwise a few reference objects. expert systems with applications, 127, 109-120. doi: https://doi.org/10.1016/j.eswa.2019.02.036 amenta, p., lucadamo, a., & marcarelli, g. (2021). on the choice of weights for aggregating judgments in non-negotiable ahp group decision making. european journal of operational research, 288(1), 294-301. doi: https://doi.org/10.1016/j.ejor.2020.05.048 benitez, j., delgado-galván, x., izquierdo, j., & pérez-garcía, r. (2015). consistent completion of incomplete judgments in decision making using ahp. journal of computational and applied mathematics, 290, 412-422. doi: https://doi.org/10.1016/j.cam.2015.05.023 benitez, j., carpitella, s., certa, a., & izquierdo, j. (2019). characterization of the consistent completion of analytic hierarchy process comparison matrices using graph theory. journal of multi-criteria decision analysis, 26(1-2), 3-15. doi: https://doi.org/10.1002/mcda.1652 bernroider, e.w., maier, k., & stix, v. (2010). incomplete information within relative pairwise comparisons as utilized by the ahp. international conference on business information systems, 85, 39-50. doi: https://doi.org/10.1007/978-3-642-15402-7_9 bozoki, s., fülöp, j., koczkodaj, w.w. (2011). an lp-based inconsistency monitoring of pairwise comparison matrices. mathematical and computer modeling, 54(1-2), 789793. doi: https://doi.org/10.1016/j.mcm.2011.03.026 chen, k., kou, g., tarn, jm, & song, y. (2015) bridging the gap between missing and inconsistent values eliciting preference from pairwise comparison matrices. annals of operations research, 235, 155-175. doi: https://doi.org/10.1007/s10479-015-1997-z dong, m., li, s., & zhang, h. (2015). approaches to group decision-making with incomplete information based on power geometric operators and triangular fuzzy ahp. expert systems with applications, 42(21), 7846-7857. doi: https://doi.org/10.1016/j.eswa.2015.06.007 dopazo, e., ruiz-tagle, m. (2011). a parametric gp model deals with incomplete information for group decision-making. applied mathematics and computation, 218(2), 514-519. doi: https://doi.org/10.1016/j.amc.2011.05.094 faramondi, l., oliva, g., & bozóki, s. (2020). incomplete analytic hierarchy process with minimum ordinal violations. international journal of general systems, 49(6), 574601. doi:https://doi.org/10.1080/03081079.2020.1786380 https://doi.org/10.1080/03081079.2020.1786380 ijahp article: bagheri fard sharabiani, gholamian, ghannadpour/a new solution for incomplete ahp model using goal programming and similarity function international journal of the analytic hierarchy process 20 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1003 gomez-ruiz, j., karanik, m., & peláez, j.i. (2010). estimation of missing judgments in ahp pairwise matrices using a neural network-based model. applied mathematics and computation, 216(10), 2959-2975. doi: https://doi.org/10.1016/ j.amc.2010.04.009 harker, p.t. (1987). incomplete pairwise comparisons in the analytic hierarchy process. mathematical modeling, 9(11), 837-848. doi: https://doi.org/10.1016/02700255(87)90503-3 hua, z., gong, b., & xu, x. (2008). a ds–ahp approach for multi-attribute decisionmaking problems with incomplete information. expert systems with applications, 34(3), 2221-2227. doi: https://doi.org/10.1016/j.eswa.2007.02.021 hsu, s., wang, t. (2011). solving multi-criteria decision-making with incomplete linguistic preference relations. expert systems with applications, 38(9), 10882-10888. doi: https://doi.org/10.1016/j.eswa.2011.02.123 hu, y.c., tsai, j.f. (2006). backpropagation multi-layer perceptron for incomplete pairwise comparison matrices in the analytic hierarchy process. applied mathematics and computation, 180(1), 53-62. doi: https://doi.org/10.1016/j.amc.2005.11.132 hu, x., li, x., huang, y. (2016). urban rail transit risk evaluation with incomplete information. procedia engineering, 137(2016), 467-477. doi: https://doi.org/10.1016/j.proeng.2016.01.282 jianqiang, w. (2006). multi-criteria decision-making approach with incomplete certain information based on ternary ahp. journal of systems engineering and electronics, 17(1), 109-114. doi: https://doi.org/10.1016/s1004-4132(06)60020-0 jandova, v., krejčí, j., stoklasa, j., & fedrizzi, m. (2016). computing interval weights for incomplete pairwise-comparison matrices of large dimension: a weak-consistencybased approach. ieee transactions on fuzzy systems, 25(6), 1714-1728. doi: https://doi.org/10.1109/tfuzz.2016.2633364 lin, c., kou, g. (2020). a heuristic method to rank the alternatives in the ahp synthesis. applied soft computing, 100(2021), 106916. doi: https://doi.org/10.1016/ j.asoc.2020.106916. maleki, n., bagherifard, m., & gholamian, m. (2020). application of incomplete analytic hierarchy process and choquet integral to select the best supplier and order allocation in the petroleum industry. international journal of engineering, 33(11), 22992309. doi: https://doi.org/10.5829/ije.2020.33.11b.20 nishizawa, k. (2005). estimation of unknown comparisons in incomplete ahp and its compensation. report of the research institute of industrial technology, nihon university, japan. pan, d., lu, x., liu, j. &, deng, y. (2014). a ranking procedure by incomplete pairwise comparisons using information entropy and dempster-shafer evidence theory. the scientific world journal, 2014, 1-11. doi: https://doi.org/10.1155/2014/904596. https://doi.org/10.1155/2014/904596 ijahp article: bagheri fard sharabiani, gholamian, ghannadpour/a new solution for incomplete ahp model using goal programming and similarity function international journal of the analytic hierarchy process 21 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1003 ruiz, h., sunarso, a., ibrahim-bathis, k., murti, s., & budiarto, i. (2020). gis-ahp multi-criteria decision analysis for the optimal location of solar energy plants in indonesia. energy reports, 6(2020), 3249-3263. doi: https://doi.org/10.1016/ j.egyr.2020.11.198. saaty, t. (1988). what is the analytic hierarchy process? in mitra, g., greenberg, h.j., lootsma, f.a., rijkaert, m.j., zimmermann, h.j. (eds) mathematical models for decision support. nato asi series vol 48 (109-121). springer. doi: https://doi.org/10.1007/978-3-642-83555-1_5 shiraishi, s., obata, t., & daigo, m. (1998). properties of a positive reciprocal matrix and their application to ahp. journal of the operations research society of japan, 41(3), 404-414. doi: shiau, t.a. (2012). evaluating sustainable transport strategies with incomplete information for taipei city. transportation research part d: transport and environment, 17(6), 427-432. doi: https://doi.org/10.1016/j.trd.2012.05.002 srdjevic, b., srdjevic, z., & blagojevic, b. (2014). first-level transitivity rule method for filling incomplete pairwise comparison matrices in the analytic hierarchy process. applied mathematics & information sciences, 8(2), 459-467. doi: https://doi.org/10.12785/amis/080202 takahashi, i. (1990). ahp applied to binary and ternary comparisons. journal of the operations research society of japan, 33(3), 199-206. doi: https://doi.org/10.15807/jorsj.33.199 van uden, e., (2002). estimating missing data in pairwise comparison matrices. in: bubnicki, z., hryniewicz, o., & kulikowski, r. (eds.) operational and systems research in the face to challenge the xxi century, methods and techniques in information analysis and decision making. warsaw: academic printing house. zhou, x., hu, y., deng, y., chan, f., & ishizaka, a. (2018). a dematel-based completion method for incomplete pairwise comparison matrix in ahp. annals of operations research, 271, 1045-1066. doi: https://doi.org/10.1007/s10479-018-2769-3 https://doi.org/10.1007/978-3-642-83555-1_5 ijahp article: milkova, andreichikova, andreichikov/venture capitalists decision making: applying analytic network process to the startups evaluation international journal of the analytic hierarchy process 1 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.511 venture capitalists decision making: applying analytic network process to the startups evaluation maria milkova central institute of economics and mathematics of the russian academy of science m.a.milkova@gmail.com olga andreichikova central institute of economics and mathematics of the russian academy of science alexandrol@mail.ru alexander andreichikov 1 central institute of economics and mathematics of the russian academy of science andreichickov@mail.ru abstract the idea of synthesizing analytical and heuristic approaches is proposed in order to join different approaches to the venture capitalists (vcs) decision making. the research applies analytic network process (anp) methodology to the comparative evaluation of four e-commerce startups. the proposed anp model represents the decision problem as a structure of benefits-opportunities and risks networks with dependences and feedbacks between decision criteria and alternatives. based on vcs judgments that are checked for consistency, the anp approach helps choose the best startup for funding or to estimate the target startup versus other startups. the ratings that are obtained may be used as weights for determining a startups valuation. in the model, heuristics is used without reducing the complexity of the task and thus helps avoid the systematic error. moreover, the idea of applying anp to the vcs decision making serves to make the decision process transparent and understandable. to implement the anp model, multichoice software has been developed. keywords: venture capital; decision making; heuristics; analytic network process; startups evaluation 1 this work is supported by russian foundation of humanities, project no. 16-02-00743 “multicriteria analysis and forecasting the techno-economic state and trends for leading aerospace companies” mailto:m.a.milkova@gmail.com mailto:alexandrol@mail.ru mailto:andreichickov@mail.ru ijahp article: milkova, andreichikova, andreichikov/venture capitalists decision making: applying analytic network process to the startups evaluation international journal of the analytic hierarchy process 2 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.511 1. introduction the startup market is highly competitive and the percentage of companies that prosper is small. in the early stage of development, startups often experience various difficulties including a financial gap that limits the company’s ability to innovate and commercialize its products (hudson & khazragui, 2013). the success or failure of a new business is often dependent on overcoming a series of potential barriers, e.g. limited human capital management capabilities, high uncertainty in terms of product and market, volatile development process, weak partnership ties (fielden, davidson, & makin, 2000). there is enough research to support the idea that start-ups that are supported by venture capital (vc) generally tend to be more successful than those that do not receive vc support (gompers & lerner, 2004; chemmanur, krishnan & nandy, 2008; bertoni, colombo & grilli, 2011). undoubtedly, venture capitalists (vcs) play the most crucial role in identifying and financing new and highly innovative firms (monika & sharma, 2015). moreover, savaneviciene, venckuviene and girdauskienea, (2015) conclude that vc is a catalyst for startups to overcome the "valley of death". while some researchers suggest using statistics to make better vcs decisions, others mention the importance of intuition, arguing that most business decisions are made in the face of uncertainty and these uncertain situations have too many unknowns and complexities to lend themselves to statistical analysis (miloud, aspelund & cabrol, 2012; narayanasamy, hashemoghli & rashid, 2012; mousavi & gigerenze,r 2014). zacharakis and meyer (2000) suggest that although the expert vc’s intuition is valuable it often leads to biased results. in this article, in order to join different approaches to vcs decision making, we synthesize analytical and heuristic approaches through applying analytic network process (anp) methodology. an example of comparative evaluation of four russian ecommerce startups is considered. the proposed anp model represents problem complexity as a network structure with dependences and feedbacks between decision criteria and alternatives. based on vcs judgments that are checked for consistency, the anp approach helps choose the best startup for funding or estimate the target startup versus other startups. anp makes it possible to make decisions under risks as it allows examining the problem from different angles, e.g. benefits, opportunities, and risks (saaty, 2008a). to implement the anp model multichoice software has been developed. 2. venture capitalists decision making in the area of vc investment decision methods are used both as tools to evaluate startups, and as tools to analyze in order to identify the factors that drive financial decisions. vcs decision criteria have faced numerous challenges with identifying the economic value of a new venture. a number of studies have produced empirically derived lists of the principle evaluation criteria. the earlier vc research mostly agreed on six criteria: management skill and experience, venture team, product attributes, market growth and size, and expected returns (macmillan et al., 1987; robinson, 1987; hall and hofer, 1993). subsequent works have also acknowledged the importance of passion in entrepreneurship (cardon et al., 2009). ijahp article: milkova, andreichikova, andreichikov/venture capitalists decision making: applying analytic network process to the startups evaluation international journal of the analytic hierarchy process 3 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.511 initial research used post hoc surveys and interviews to collect data on vcs’ self-reported decision policy for decisions made in the past. this reliance on retrospective and selfreported data may have generated biased results. zacharakis and meyer (1998) support the fact that people are poor at introspection and often suffer from recall and post-hoc rationalization biases among others. therefore, real-time methods such as verbal protocols and conjoint analysis are more appropriate and eliminate these biases. verbal protocols are real-time “think aloud” observations of vcs screening a potential deal (landström et al., 2007). different research studies have used verbal protocols to understand information in the actual decision process (hall & hofer, 1993; zacharakis & meyer, 1995). conjoint analysis is a technique that assesses decision criteria (attributes) and their significance in the judgment, and how these attributes affect the judgment and the relative importance of each attribute in the decision process (shepherd & zacharakis, 1999). conjoint analysis has been used in many studies and gains a deeper understanding of the vc decision process (hsu, et al., 2014; zacharakis & shepherd, 2005). another attempt at accomplishing some improvement in the vcs decision process was proposed by zacharakis and meyer (2000). they introduced actuarial decision aides that are models that decompose a decision into component cues and recombine those cues to predict the potential outcome. actuarial models include environmental and bootstrapping models, where the former employ discriminant or regression analysis on actual decision data. shepherd and zacharakis (2002) proposed that bootstrapping models hold considerable potential for improving vcs decision accuracy. in any case, for evaluating new ventures not all the vcs are able to follow the same investment decision process (monika & sharma, 2015). vcs are individuals with their own unique experience, perspective and business priorities, so they do not evaluate startups the same way. some vcs give more importance to the entrepreneur’s characteristics, while others are more intrigued with financial and marketing perspectives. monika and sharma (2015) highlight that vcs follow the multi-criteria perspective for taking investment decision. 2.1 using heuristics it is well recognized in the decision-making literature that decision makers are not perfectly rational, but “boundedly rational”, which means that when individuals make decisions, their rationality is limited by the tractability of the decision problem, the cognitive limitations of their minds, and the time available to make the decision (simon, 1957). tversky and kahneman (1974) showed that people making decisions under uncertainty rely on a limited number of heuristic principles, which leads to systematic errors. zacharakis and meyer (2000) suggest that a vc is apt to assess the success of a current venture prospect by how similar it is to a past success when analyzing vcs decision making. in this assessment, vcs use a representativeness heuristic, which may lead to severe errors (tversky & kahneman, 1974). likewise, if a vc utilizes a satisfying heuristic it may eliminate potentially profitable investments (zacharakis & meyer, 2000). ijahp article: milkova, andreichikova, andreichikov/venture capitalists decision making: applying analytic network process to the startups evaluation international journal of the analytic hierarchy process 4 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.511 while assessing decision criteria, vcs may tend to underweight the more important criteria and overweight the less important criteria (zacharakis & meyer, 1998). thus, biases and heuristics significantly affect the behavior of vcs. bias factors include risk perception, overconfidence, inconsistency and habit and framing (dimov et al., 2007; mitteness et al., 2012, zacharakis & shepherd, 2001). all considerations about applying heuristic rules and intuition in vcs decision making imply simplification of the decision process and inconsistent of human judgments. woike, hoffrage, and petty (2015) compared simple heuristics with machine learning and regression models and showed that simple heuristics is competitive with more complex vcs decision strategies. however, is it possible to synthesize analytic and heuristic approaches? could we apply heuristics without sufficient reduction of the complexity of the problem? this article contributes to answering these questions. 2.2 multi-criteria decision analysis due to the complex nature of vcs decision, we suggest that multi-criteria decision analysis (mcda) methods can help find the best investment strategy. mcda is devoted to supporting and aiding vcs in situations in which multiple conflicting decision factors (objectives, goals, criteria, etc.) must be considered simultaneously. there is a vast body of literature on the use of multi-criteria methodologies in financial decision-making, such as project financing, financial performance evaluation, investment selection, extension of credit, and foreign direct investment; however much less is reported on applications of mcda to vc portfolio selection (beshah & kitaw, 2013; bhandari & nakarmi, 2016; saracoglu, 2015; beim & lévesque, 2004) . in recent research, pakizeh and hosseini (2015) propose promethee method; afful-dadzie, oplatková, and nabareseh (2015) apply fuzzy promethee for selecting startup businesses; beim and lévesque (2004) consider mavt. lu and shen (2011), su, jiang, and ma (2009) and gui-lan (2011) evaluate investment risks of vc company based on analytic hierarchy process (ahp). shijian and yinyan (2015) apply ahp-fuzzy evaluation methods to evaluate vc project. wiratno, latiffianti, and wirawan (2015) apply anp for selection of business funding proposals. 2.3 why anp? among the existing methods, anp is one that considers dependences between decisions criteria (saaty, 1996). in light of research on mcda in making vcs decisions, we consider anp as a decision aid for vcs in understanding the complexities of the decisions they face. although ahp/anp have been widely used in solving different decision problems, the application of ahp/anp in the articles related to vc has been very few. however, over the past two decades much research has considered comparative analysis of conjoint analysis (ca) and ahp (helm et. al., 2002; helm et. al., 2004; schol et. al., 2005; meißner & decker, 2009; ijzerman et. al., 2010). most of them reported that not only “both methods are equivalent with regard to convergent validity”, but also ahp is “the better choice for the special decision situation considered” (meißner & decker, 2009; schol et. al., 2005). this is somewhat astonishing considering the similarity of ahp and ijahp article: milkova, andreichikova, andreichikov/venture capitalists decision making: applying analytic network process to the startups evaluation international journal of the analytic hierarchy process 5 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.511 ca approaches, and the fact that ca is quite popular for measuring vcs preferences, while anp/ahp is not. in order to increase the practical relevance of anp in vcs decision making, we consider an example of evaluation of four e-commerce startups. based on neuroeconomic results that people compare choices within a set rather than assigning separate utilities, the comparative method is appropriate for vcs decision-making (camerer, loewenstein, & prelec, 2004). in a high-risk area such as vc financing, selecting the right candidate can be very challenging and complex since most of the criteria involved are subjective and hold uncertain data. due to lack of the quantitative information on a startup’s activities and the high level of uncertainty, the appropriate method for vcs decision-making must be based on subjective judgments, which not only measure preferences, but also reflect knowledge about influences between decision criteria and the strengths with which these influences occur. zacharakis and shepherd (2004) showed that interactions between decision criteria affect vcs decision making, so it is the anp model that may be applied at startups evaluation process. thus, the anp approach is considered suitable for vcs decision making because, (1) the model allows including a mix of quantitative and qualitative (or only quantitative) factors and implies interconnections between decision criteria; (2) decision problem may be examined not only from the benefits side, but also from the opportunities and risks sides; (3) the integrity of the measurement system is verified for consistency because of subjective of human judgments. 3. model construction and results 3.1 anp model the anp model may be constructed after the screening stage of investment where the number of initially available alternatives has been reduced. in our study, four russian ecommerce startups (let us identify them as a, b, c, and d) are subjected to deep analysis with anp. startup a is a coffee service for drivers, a small chain of stores that offers snacks, tea and coffee to go. drivers can make an order and pay in advance via a mobile app, and then just pick up in order to save time. startup b is a time bank, a reciprocity-based work trading system in which hours are used as currency. with time banking, a person with one skill set can bank and trade hours of work for equal hours of work in another skill set instead of paying or being paid for services. startup c is an organic food delivery online service. the service integrates products from different stores to help people buy any organic food without an extra charge for delivery from different shops. startup d is an online store for renting sporting goods and equipment (skiing, skates, snowboards, bicycles, etc.). the service allows people to rent goods for the whole season ijahp article: milkova, andreichikova, andreichikov/venture capitalists decision making: applying analytic network process to the startups evaluation international journal of the analytic hierarchy process 6 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.511 and thereby solves the problem of storing goods in the apartment at the time of the year when they are not being used. in order to make a comprehensive decision, an investor should take into account the startups performance and development prospects, as well as current and expected risks. the model included two network structures: 1. benefitsopportunities networkthis combines all criteria of efficiency and potential of the startups. we consider benefits and opportunities within one network structure as benefits criteria have always had an impact on opportunities criteria. 2. risks networkthis includes the most important current and expected risk factors. these network structures are combined in the control hierarchy for evaluating the networks contribution to the final decision. each network contains selection criteria, relationship among criteria, and the submitted funding proposals. decision criteria of the networks were chosen based on macmillan et al. (1987), robinson (1987), hall and hofer (1993) and on criteria used in the most popular russian startups competition – school of a young billionaire, organized by forbes russia magazine (forbes, 2017). the network of benefits-opportunities (figure 1) includes six decision clusters which include the following: growth for the last year, society, team, promotion, prospects, production. each of the clusters has its own specified criteria (nodes). figure 1 benefits-opportunities network the network of risks (figure 2) includes four clusters which include the following: competition, commercial risks, operational risks, and other risks. ijahp article: milkova, andreichikova, andreichikov/venture capitalists decision making: applying analytic network process to the startups evaluation international journal of the analytic hierarchy process 7 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.511 figure 2 risks network to combine and evaluate a network’s contribution to the final decision, the control hierarchy is built (figure 3). considered networks are evaluated in terms of the company’s profit, the company’s competitiveness, and improving societal well-being. figure 3 control hierarchy the result of the anp algorithm is to find a startup with the highest ratings in terms of benefits-opportunities and risks priorities ratio. as a result, the selected startup will be the most attractive to receive funding. 3.2 results the model is built in multichoice, which is a new software for mcda based on the anp/ahp (milkova & andreichikova, 2016). all pairwise comparison matrixes are filled by one expert, the investor concerned (for more details on the methodology of ijahp article: milkova, andreichikova, andreichikov/venture capitalists decision making: applying analytic network process to the startups evaluation international journal of the analytic hierarchy process 8 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.511 building pairwise comparisons see saaty (2008b)). in this study, we do not consider the case of multiple experts, although this may be done (for more details see saaty, 2010). for the network of benefits-opportunities, thirty-seven pair comparison matrixes are filled: 6 for cluster comparisons, and 31 for nodes comparisons. the results of evaluating the startups by each decision criteria of benefits-opportunities network are shown in figure 4. figure 4 results of pairwise comparisons of the startups by each node of the benefitsopportunities network further relative measurements of the influence of elements within the risks network are considered. eleven pair comparison matrixes for nodes are filled by the very same expert. the results of evaluating the startups by each decision criteria of the risks network are shown in figure 5. figure 5 results of pairwise comparisons of the startups by each node of the risks network ijahp article: milkova, andreichikova, andreichikov/venture capitalists decision making: applying analytic network process to the startups evaluation international journal of the analytic hierarchy process 9 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.511 the priority vectors that are obtained are then combined in a supermatrix and weighted in a weighed supermatrix. after the limit supermatrix is calculated, cluster limit vectors are normalized. limit normalized by cluster priorities of startups in benefits-opportunities network is shown in figure 6. figure 6 limit normalized by cluster priorities of the startups in benefits-opportunities network to analyze the obtained priorities of alternatives within the benefits-opportunities network (figure 6), one should take into account that according to the network’s structure (figure 1) clusters prospects, growth for the last year and team have the highest weights because they accumulate their significance through all links coming into them. as shown in figure 4, startup d has the highest node priorities of the “significant” clusters: expansion (cluster growth for the last year) – 0.43, financial soundness (cluster prospects) – 0.51. the b startup also has the highest node priorities of the “significant” clusters: team professional satisfaction (cluster team) – 0.56, market share (cluster prospects) – 0.51. therefore, these startups are the best in terms of benefitsopportunities that is shown in figure 6. limit normalized by cluster priorities of startups in the risks network is shown in figure 7. ijahp article: milkova, andreichikova, andreichikov/venture capitalists decision making: applying analytic network process to the startups evaluation international journal of the analytic hierarchy process 10 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.511 figure 7 limit normalized by cluster priorities of the startups in risks network one always should take into account a network’s structure for analyzing the obtained priorities. according to the structure of the risks network (figure 2), only the competition cluster accumulates the importance. therefore, the priorities of the startups by nodes of the competition cluster are more significant in making the final result. as shown in figure 7, startup c has the highest priority (0.45) in the risks network due to its highest nodes priorities of the “significant” cluster competition. to obtain priority ratings for the benefits-opportunities and risks networks of the model, they are evaluated by selecting the appropriate rating category from a defined linguistic scale on each criterion of the control hierarchy. in the study, a “high”, “middle”, and “low” linguistic scale is used. the results of networks linguistic evaluation in control hierarchy (figure 3) are shown in table 1. table 1 linguistic estimation of networks by the criteria of control hierarchy criterion benefitsopportunities risks company's profit high high company's competitiveness high middle improving society well being high low preferences for linguistic categories obtained by pairwise comparisons and equal: ijahp article: milkova, andreichikova, andreichikov/venture capitalists decision making: applying analytic network process to the startups evaluation international journal of the analytic hierarchy process 11 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.511 , , . priorities of the criteria of control hierarchy are obtained from pairwise comparisons and equal: , , . thus, the final weights of benefits-opportunities network ( ) and risks network ( ) are: , . the results of multiplicative and additive composition are shown in table 2. table 2 synthesis results alternatives benefitsopportunities (0,7262) risks (0,2738) multiplicative additive a 0,2188 0,1136 0,2658 0,2824 b 0,2707 0,1236 0,3031 0,3597 c 0,2142 0,4549 0,179 0,0685 d 0,2964 0,3079 0,2522 0,2894 according to the synthesis results, startup b seems to be the most attractive for funding. the startup is an example of a good balanced alternative that is placed second in terms of benefits-opportunities and risks and first in overall ranking. all alternatives have positive priority at additive composition, which means that they carry benefitsopportunities higher than risks. sensitivity analysis shows that the final startup’s priorities are stable to the 5% changes of elements priorities in the networks and to the 5% changes of networks weights. 4. discussion 4.1 practical implementation issues the goal of this paper is to improve vcs decision making by synthesizing analytical and heuristic approaches. the proposed anp methodology is a useful decision aid for vcs that helps to valuate selected ventures. the commonly used valuation techniques in corporate finance (e.g. discounted cash flow method, earning multiple method and net asset method, etc.) depend on strict assumptions and require information that new ventures typically cannot provide (such as accounting information). hence, their applicability is severely limited in valuating earlystage new ventures and both venture capitalists and entrepreneurs are frustrated by huge variance of valuations computed from the extant methods for the same new venture ijahp article: milkova, andreichikova, andreichikov/venture capitalists decision making: applying analytic network process to the startups evaluation international journal of the analytic hierarchy process 12 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.511 (miloud, aspelund, & cabrol, 2012). uncertainty and risk related to product creation and commercialization, human resource management issues, lack of technological knowledge etc. are inherent features for startups thus, vcs must make their decision under uncertainty and risk without a sufficient amount of financial records. all these facts are major requisites for using heuristic rules in evaluation, and as a result in valuation of startups. on the other hand, although the expert vc’s intuition is valuable, it is often biased resulting in suboptimal decisions (zacharakis & meyer, 2000). the proposed anp methodology makes it possible to assess all criteria that are valuable for vcs without sufficient simplification of the problem. anp startups ratings may be used as weights for determining startups valuation. for example, a target startup may be included in an anp model with an already funded similar start up in order to determine an appropriate valuation of the target. the valuation of the considered startup will be determined through final weights of the alternatives. thus, the anp approach may considerably extend the scope of comparative valuation methods. furthermore, the problem of evaluating startups arises not only in the case of their valuation, but also occurs in the selection of winners at startup competitions. at some competitions, the startups may be very different from each other, so the right evaluation of the competitors may be a very complicated task. since winners experience a positive effect of visibility and reputation, a fair and impartial selection of a winner is the best guarantee for adequate funding. the proposed anp model uses one expert who is responsible for model building and making comparisons. however, the anp may be also applied in the case where a group of experts dealing with framing a constructed network structure. aczel and saaty (1983) proved that the unique way to combine reciprocal individual judgments into a corresponding reciprocal group judgment is by using their geometric mean. 4.2 limitations as with any methodology, anp/ahp has its possible limitations. the first one concerns the number of included criteria and alternatives. anp/ahp does not work optimally in the case where the number of alternatives is large; therefore, it cannot be applied at the screening stage of investment. therefore, at first, vcs must screen the hundreds of proposals by using, for example, actuarial decision aides or simple heuristic rules (zacharakis & meyer, 2000; woike, hoffrage, & petty, 2015). those ventures that survive the initial stage can then be subjected to deep analysis with anp. another approach that works with a large number of items, for example, is to use an extension of ahp structuring by incorporating it into another method of prioritization known as best-worst scaling (lipovetsky, 2016). zacharakis and meyer (2000) also conclude that as more information is available to the decision, the vc’s predictive accuracy substantially decreases. although this statement relates to heuristic decision making, it partly concerns anp too. in spite of the fact that anp helps to analyze complex vc decision problems through including as many network structures, decision criteria and interactions between them as needed, when the ijahp article: milkova, andreichikova, andreichikov/venture capitalists decision making: applying analytic network process to the startups evaluation international journal of the analytic hierarchy process 13 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.511 number of criteria is large, the amount of time needed to complete the pairwise comparison will be considerably long. furthermore, pairwise comparison value might be inconsistent due to this massive number of comparisons. as a result, quality of perception of the comparisons will be reduced. to solve this problem, different ways to improve the consistency of judgements have been discussed (saaty, 2003; koczkodaj and szybowski, 2016). another solution is to use incomplete pairwise comparisons (fedrizzia and giove 2007; bozóki, fülöp, & rónyai, 2010). therefore, it is the expert’s responsibility to determine the degree of the model’s complexity that would make anp more applicable. in spite of the requirement that an allowable consistency ratio must be not more than about .10, the requirement of 10% cannot be made smaller, such as 1% or 0.1%, without trivializing the impact of inconsistency. saaty (2013) noticed that inconsistency itself is important because without it new knowledge that changes, preference cannot be admitted. assuming that all knowledge should be consistent contradicts experience, which requires continued revision of understanding. the second limitation of building effective anp models involves the fact that feedback on the quality of vc’s decision is slow in coming (zacharakis & meyer, 2000). it generally takes 7 years to identify the portfolio winners, and 2 to 3 years to identify the losers (timmons & spinelli, 2004). thus, slow feedback makes it difficult to adjust anp approach for vcs in their decision processes. 5. conclusion the research was aimed at applying anp to evaluate and select startup businesses for funding. we proposed the idea that to be transparent and understandable, vcs should not collapse the complexity of the decision process into a simplistic scheme. vcs should decompose judgments through elaborate structures and organize their reasoning and calculations in sophisticated ways. experience indicates that it is not very difficult to do this although it takes more time and effort. indeed, we must use feedback networks to arrive at the kind of decisions needed to cope with the future (saaty & vargas, 2006). thus, anp deliberately synthesizes heuristic and analytic approaches and considerably extends the idea of making business decisions under uncertainty. anp allows considering the complexity of the problem and uses expert’s pairwise comparisons based on heuristics. in anp, heuristics is used without reducing the complexity of the task and thus helps to avoid the systematic error. on the other hand, anp is not a heuristic method; it is a mathematical theory that makes it possible to deal with all kinds of dependence and feedback between decision criteria and alternatives and examines the problem from different angles (in our study these are benefits-opportunities and risks). despite the potential benefits of applying the anp methodology in vcs decision-making process, shepherd and zacharakis (2002) mentioned that, “vcs rarely use decision aids and thus may be missing an opportunity. we hope that the proposed example of applying the anp in vcs decision making and the developed multichoice decisions software encourages researchers to further explore anp in the area of vc investment decisions. we expect that applying heuristics as part of the analytical process will lead to other results than using heuristics per se. however, this statement must be proved in future research. ijahp article: milkova, andreichikova, andreichikov/venture capitalists decision making: applying analytic network process to the startups evaluation international journal of the analytic hierarchy process 14 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.511 references aczel, j., saaty, t. (1983). procedures for synthesizing ratio judgments. journal of mathematical psychology, 27(1), 93-102. doi: https://doi.org/10.1016/00222496(83)90028-7 afful-dadziea e, oplatkováa, z.k., nabareseh, s. (2015). selecting start-up businesses in a public venture capital financing using fuzzy promethee. 19th international conference on knowledge based and intelligent information and engineering systems. procedia computer science, 60, 63-72. doi: https://doi.org/10.1016/j.procs.2015.08.105 beim, g., lévesque, m. (2004). selecting projects for venture capital funding : a multiple criteria decision approach. technical memorandum number 791, weatherhead school of management, case western university. bertoni, f., colombo, m.g., grilli, l. (2011). venture capital financing and the growth of high-tech start-ups: disentangling treatment from selection effects. research policy, 40(7), 1028-1043. doi: https://doi.org/10.1016/j.respol.2011.03.008 beshah, b., kitaw, d. (2013). ahp application in a financial institution. international journal of the analytic hierarchy process, 5(1), 54-71. doi: http://dx.doi.org/10.13033/ijahp.v5i1.135 bhandari, a., nakarmi, a. (2016). a financial perfomance evaluation of commercial banks in nepal using ahp model. international journal of the analytic hierarchy process, 8(2), 318-333. doi: http://dx.doi.org/10.13033/ijahp.v8i2.368 bozóki, sándor, jános, fülöp, rónyai, l. (2010). on optimal completion of incomplete pairwise comparison matrices. mathematical and computer modelling, 52 (1-2), 318333. doi: https://doi.org/10.1016/j.mcm.2010.02.047 camerer c. f., loewenstein, g., prelec, d. (2004). neuroeconomics: why economics needs brains. scandinavian journal of economics, 106 (3), 555-579. doi: 10.1111/j.0347-0520.2004.00377.x chemmanur, t., krishnan, k., debarshi k. nandy. (2008). how does venture capital financing improve efficiency in private firms? a look beneath the surface. review of financial studies, 24(12), 4037-4090. doi: https://doi.org/10.1093/rfs/hhr096 dimov, d., shepherd, d.a., sutcliffe, k.m. (2007). requisite expertise, firm reputation, and status in venture capital investment allocation decisions. journal of business venturing, 22(4), 481-502. doi: https://doi.org/10.1016/j.jbusvent.2006.05.001 fielden, s. l., davidson, m.j., makin, p.j. (2000). barriers encountered during micro and small business start-up in north-west england. journal of small business and enterprise development, 7(4), 295-304. doi: https://doi.org/10.1108/eum0000000006852 https://doi.org/10.1016/0022-2496(83)90028-7 https://doi.org/10.1016/0022-2496(83)90028-7 https://doi.org/10.1016/j.procs.2015.08.105 https://doi.org/10.1016/j.respol.2011.03.008 http://dx.doi.org/10.13033/ijahp.v8i2.368 https://doi.org/10.1016/j.mcm.2010.02.047 https://doi.org/10.1093/rfs/hhr096 https://doi.org/10.1016/j.jbusvent.2006.05.001 https://doi.org/10.1108/eum0000000006852 ijahp article: milkova, andreichikova, andreichikov/venture capitalists decision making: applying analytic network process to the startups evaluation international journal of the analytic hierarchy process 15 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.511 fedrizzia, m., giove,s. (2007). incomplete pairwise comparison and consistency optimization. european journal of operational research, 183(1), 303-313. doi: https://doi.org/10.1016/j.ejor.2006.09.065 forbes, r. (2017). school of a young billionaire. http://www.forbes.ru/school gompers, p. a., lerner, j. (2004). the venture capital cycle (second edition). cambridge, ma: mit press. gui-lan, hu. (2011). an improved ahp-based evaluation study on the investment risk of venture capital company. fourth international symposium on knowledge acquisition and modeling, 2011, 17-20. doi:10.1109/kam.2011.12 hall, j. hofer, c.w. (1993). venture capitalists’ decision criteria in new venture evaluation. journal of business venturing, 8(1), 25–42. doi: https://doi.org/10.1016/0883-9026(93)90009-t helm, r., manthey, l., scholl, a. steiner,m. (2002). preference measurement with conjoint analysis and ahp: an empirical comparison. in ulrike leopold-wildburger et. al. (eds), operations research proceedings, 517-524. heidelberg: springer berlin. doi: https://doi.org/10.1007/978-3-642-55537-4_84 helm, r., steiner, m. scholl, a. manthey, l. (2004). a comparative empirical study on common methods for measuring preferences. jenaer schriften zur wirtschaftswissenschaft : arbeitsund diskussionspapiere der wirtschaftswissenschaftlichen fakultät der friedrich-schiller-universität jena. doi: 10.1504/ijmdm.2008.017408. doi: 10.1504/ijmdm.2008.017408 hsu, d. k, haynie, j.m., simmons, s.a., mckelvie, a. (2014). what matters, matters differently: a conjoint analysis of the decision policies of angel and venture capital investors. venture capital, 16(1), 1-25. https://doi.org/10.1080/13691066.2013.825527 hudson, j. khazragui, h.f. (2013). into the valley of death: research to innovation. drug discovery today, 18(13-14), 610-613. doi: https://doi.org/10.1016/j.drudis.2013.01.012 ijzerman, m. j., van til, j.a., bridges, j.f.. (2010). a comparison of analytic hierarchy process and conjoint analysis mmethods in assessing treatment alternatives for stroke rehabilitation. the patient patient-centered outcomes research, 5(1), 45-56. doi: https://doi.org/10.2165/11587140-000000000-00000 koczkodaj, w.w., szybowski. j. (2016). the limit of inconsistency reduction in pairwise comparisons. international journal of applied mathematics and computer science, 26 (3), 721–729. doi: https://doi.org/10.1515/amcs-2016-0050 landström, h. (2007). handbook of research on venture capital. edward elgar publishing. lipovetsky, s. (2016). ahp structuring in best-worst scaling and the secretary problem. international journal of the analytic hierarchy process, https://doi.org/10.1016/j.ejor.2006.09.065 https://doi.org/10.1109/kam.2011.12 https://doi.org/10.1016/0883-9026(93)90009-t http://dx.doi.org/10.1504/ijmdm.2008.017408 https://doi.org/10.1080/13691066.2013.825527 https://doi.org/10.1016/j.drudis.2013.01.012 https://doi.org/10.1515/amcs-2016-0050 ijahp article: milkova, andreichikova, andreichikov/venture capitalists decision making: applying analytic network process to the startups evaluation international journal of the analytic hierarchy process 16 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.511 8(3), 502-513. doi: http://dx.doi.org/10.13033/ijahp.v8i3.332 lu, z., shen, y. (2011). the study on venture capital project appraisal using ahp-fuzzy comprehensive evaluation methods. international journal of advancements in computing technology, 3(8), 50-56. macmillan, i.c., zeman, l., subbanarasimha. p.n. (1987). criteria distinguishing unsuccessful ventures in the venture screening process. journal of business venturing, 2(2), 123-137. doi: https://doi.org/10.1016/0883-9026(87)90003-6 meißner, m., decker, r. (2009). an empirical comparison of cbc and ahp for measuring consumer preferences. in t. tjader (ed). proceedings of tenth international symposium on analytic hierarchy process (isahp 2009),university of pittsburgh, usa: creative decisions foundation. milkova, m., andreichikova, o. (2016). software announcement: multichoice as new software for decision making with analytic network process. international journal of the analytic hierarchy process, 8(2), 388-400. doi: http://dx.doi.org/10.13033/ijahp.v8i2.413 miloud, t., aspelund, a., cabrol, m. (2012). startup valuation by venture capitalists: an empirical study. venture capital: an international journal of entrepreneurial finance, 14 (2-3), 151-174. doi: https://doi.org/10.1080/13691066.2012.667907 mitteness, c., sudek, r., cardon, m.s. (2012). angel investor characteristics that determine whether perceived passion leads to higher evaluations of funding potential. journal of business venturing, 27(5), 592-606. doi: https://doi.org/10.1016/j.jbusvent.2011.11.003 monika, d., sharma, a.k. (2015). venture capitalists’ investment decision criteria for new ventures: a review.” xviii annual international conference of the society of operations management (som-14). procedia social and behavioral sciences, 189, 465-470. doi: https://doi.org/10.1016/j.sbspro.2015.03.195 mousavi, s., gigerenzer, g. (2014). risk, uncertainty, and heuristics. journal of business research, 67, 1671-1678. doi: https://doi.org/10.1016/j.jbusres.2014.02.013 narayanasamy, c., hashemoghli, a., rashid, r.m.. (2012). venture capital preinvestment decision making process: an exploratory study in malaysia. global journal of business research, 6(5), 49-63. pakizeh, k., hosseini, m. (2015). venture capital investment selection based on promethee. applied mathematics in engineering, management and technology, 3(1), 566-572. robinson, r.b. (1987). emerging strategies in the venture capital industry. journal of business venturing, 2(1), 53–77. doi: https://doi.org/10.1016/0883-9026(87)90019-x saaty, t. l. (1996). decision making with dependence and feedback: the http://dx.doi.org/10.13033/ijahp.v8i3.332 https://doi.org/10.1016/0883-9026(87)90003-6 http://dx.doi.org/10.13033/ijahp.v8i2.413 https://doi.org/10.1080/13691066.2012.667907 https://doi.org/10.1016/j.jbusvent.2011.11.003 https://doi.org/10.1016/j.sbspro.2015.03.195 https://doi.org/10.1016/j.jbusres.2014.02.013 https://doi.org/10.1016/0883-9026(87)90019-x ijahp article: milkova, andreichikova, andreichikov/venture capitalists decision making: applying analytic network process to the startups evaluation international journal of the analytic hierarchy process 17 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.511 analytic network process. pittsburgh: rws publications. saaty, t.l. (2003). decision-making with the ahp: why is the principal eigenvector necessary. european journal of operational research, 145, 85-91. doi: https://doi.org/10.1016/s0377-2217(02)00227-8 saaty, t. l. (2008a). the analytic hierarchy and analytic network measurement processes: applications to decisions under risk. european journal of pure and applied mathematics, 1(1), 122-196. saaty, t. l. 2(008b). relative measurement and its generalization in decision making – why pairwise comparisons are central in mathematics for the measurement of intangible factors: the analytic hierarchy/network process. racsam revista de la real academia de ciencias exactas, fisicas y naturales. serie a. matematicas, 102(2), 251–318. doi: https://doi.org/10.1007/bf03191825 saaty, t.l. (2010). principia mathematica decernendi: mathematical principles of decision making: generalization of the analytic network process to neural firing and synthesis. pittsburgh, pa: rws publications. saaty, t.l. (2013). on the measurement of intangibles. a principal eigenvector approach to relative measurement derived from paired comparisons. notices of the ams, 60(2), 192-208. saaty, t.l., vargas. l.g. (2006). decision making with the analytic network process: economic, political, social and technological applications with benefits, opportunities, costs and risks. new york: springer. saracoglu, b.o. (2015). an ahp application in the investment selection of small hydropower plants in turkey. international journal of the analytic hierarchy process, 7(2), 211-239. doi: http://dx.doi.org/10.13033/ijahp.v7i2.198 savaneviciene, a., venckuviene, v., girdauskienea, l. (2015). venture capital a catalyst for start-ups to overcome the "valley of death": lithuanian case. 4th world conference on business, economics and management, wcbem. procedia economics and finance, 26, 1052-1059. doi: https://doi.org/10.1016/s2212-5671(15)00929-6 schol, a., manthey, l., helm, r., steiner, m. (2005). solving multiattribute design problems with analytic hierarchy process and conjoint analysis: an empirical comparison. european journal of operational research, 164, 760-777. doi: https://doi.org/10.1016/j.ejor.2004.01.026 shepherd, d.a., zacharakis. a. (1999). conjoint analysis: a new methodological approach for researching the decision policies of venture capitalists. venture capital, 1(3), 197-217. doi: https://doi.org/10.1080/136910699295866 shepherd, d.a., zacharakis, a. (2002). venture capitalists' expertise: a call for research into decision aids and cognitive feedback. journal of business venturing, 17(1), 1-20. doi: https://doi.org/10.1016/s0883-9026(00)00051-3 https://doi.org/10.1016/s0377-2217(02)00227-8 http://dx.doi.org/10.13033/ijahp.v7i2.198 https://doi.org/10.1016/s2212-5671(15)00929-6 https://doi.org/10.1016/j.ejor.2004.01.026 https://doi.org/10.1080/136910699295866 https://doi.org/10.1016/s0883-9026(00)00051-3 ijahp article: milkova, andreichikova, andreichikov/venture capitalists decision making: applying analytic network process to the startups evaluation international journal of the analytic hierarchy process 18 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.511 shijian, f. yinyan, c. (2015). ahp-fuzzy comprehensive evaluation model of venture investment and financing system: based on the case of incubation base in anhui. canadian social science, 11(1), 148-153. simon, h. a. (1957). models of man. new york: wiley & sons. su, h., jiang, r. ma, x. (2009_. risk evaluation of venture capital based on ahp and grey relational analysis methods. international conference on information management, innovation management and industrial engineering, 2009, 316-320. doi: 10.1109/iciii.2009.536 timmons, j.a. spinelli, s. (2004). new venture creation: entrepreneurship for the 21st century. boston: mcgraw-hill/irwin. tripathi, a., vidyarthi, d.p. (2015). task allocation on cloud resources using analytic network process. proceedings of international conference on advances in computer engineering and applications (icacea), 971-978. doi: 10.1109/icacea.2015.7164847 tversky, a. kahneman, d. (1974). judgment under uncertainty: heuristics and biases. science, new series, 185(4157), 1124-1131. doi: 10.1126/science.185.4157.1124 wiratno, s., latiffianti, e., wirawan, k.k. (2015). selection of business funding proposals using analytic network process: a case study at a venture capital company.” industrial engineering and service science. procedia manufacturing 4: 237-243. doi: https://doi.org/10.1016/j.promfg.2015.11.037 woike, j.k., hoffrage, u., petty, j.s.. (2015). picking profitable investments: the success of equal weighting in simulated venture capitalist decision making. journal of business research, 68(8), 1705–1716. doi: https://doi.org/10.1016/j.jbusres.2015.03.030 zacharakis, a.l., meyer, g.d. (1998). a lack of insight: do venture capitalists really understand their own decision process?” journal of business venturing, 13(1), 57-76. doi: https://doi.org/10.1016/s0883-9026(97)00004-9 zacharakis, a.l., meyer, g.d. (2000). the potential of actuarial decision models: can they improve the venture capital investment decision?” journal of business venturing, 15(4), 323-346. doi: https://doi.org/10.1016/s0883-9026(98)00016-0 zacharakis, a., shepherd, d.a. (2001). the nature of information and overconfidence on venture capitalists' decision making. journal of business venturing, 16(4), 311-332. doi: https://doi.org/10.1016/s0883-9026(99)00052-x zacharakis, a., shepherd, d.a. (2004). a non-additive decision-aid for venture capitalists investment decisions. european journal of operational research, 162, 673– 689. doi: https://doi.org/10.1016/j.ejor.2003.10.028 https://doi.org/10.1109/iciii.2009.536 https://doi.org/10.1109/icacea.2015.7164847 https://doi.org/10.1016/j.promfg.2015.11.037 https://doi.org/10.1016/j.jbusres.2015.03.030 https://doi.org/10.1016/s0883-9026(97)00004-9 https://doi.org/10.1016/s0883-9026(98)00016-0 https://doi.org/10.1016/s0883-9026(99)00052-x https://doi.org/10.1016/j.ejor.2003.10.028 ijahp article: akhrouf, derghoum/use of analytic hierarchy process model for selection of health infrastructure projects international journal of the analytic hierarchy process 1 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1040 use of analytic hierarchy process model for selection of health infrastructure projects mohamed akhrouf ecole supérieure de commerce algeria m_akhrouf@esc-alger.dz mahfoudh derghoum ecole supérieure de commerce algeria m_derghoum@esc-alger.dz abstract this article aims to develop a methodology in which alternative projects are prioritized and selected using appropriate methods of multi-criteria decision-making in organizations responsible for managing and developing health infrastructure. this study addresses a particular gap in implementing a systematic methodology for prioritizing and selecting projects in the health sector. the methodology developed proposes an approach based on the ahp multi-criteria decision support method for decision-makers and stakeholders to prioritize and select projects in the health sector. the problem was modeled using the expert choice software which allows a good “integration of the decision-maker” in the decision-making process making it possible to intervene in the research and decisions to identify the most efficient and potentially profitable health projects that are useful to the community keywords: project selection; health infrastructure projects; multi-criteria decision support; ahp; analytical hierarchy process 1. introduction the selection of projects is considered a crucial component of project portfolio management. project managers, who are tasked with overseeing projects, face the challenge of having limited resources to carry out a multitude of project ideas. as a result, they must select the most promising projects from the pool of candidates. project selection is viewed as an evaluation process, where each project concept is assessed and the one with the highest priority is chosen. to make this determination, managers can utilize a single criterion, such as the cost of implementation, to rank the projects. the least expensive projects can then be selected, resulting in a simple single-criteria decision. however, decision-makers often use multiple criteria, such as technical, environmental, social, etc., in their decision-making process. these criteria can be conflicting and subjective, including both tangible and intangible factors. this is referred to as multi-criteria decision-making. mailto:m_akhrouf@esc-alger.dz ijahp article: akhrouf, derghoum/use of analytic hierarchy process model for selection of health infrastructure projects international journal of the analytic hierarchy process 2 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1040 the selection of projects is a complex decision-making problem that requires a multicriteria analysis and the use of appropriate methods. it is important to note that there is generally no alternative that is best from all perspectives, and the concept of optimization is not applicable in this context. the development of multi-criteria methods has moved away from aggregating into a single criterion and towards more flexible methods that incorporate interactivity. several multi-criteria decision making (mcdm) methods have been developed to support decision-making in this context (e.g., ahp, electre, macbeth, smart, promethee, uta, etc.), see also ishizaka & labib (2010) and ishizaka & nemery (2013). the literature presents several studies on the selection of projects using multi-criteria analysis. for example, chatterjee and al. (2018) applied the ahp in a fuzzy environment to help companies set priorities in terms of investment. one study describes an application of the ahp method for the problem of selecting investments in small hydroelectric power stations (saracoğlu, 2015). ciptomulyono (2000) proposed a model integrating the ahp and multi-objective programming for the selection of electrical projects. an application based on the ahp for the selection of a production project according to the criteria of sustainable development was developed by jurı´k, horňáková, šantavá, cagáňová, & sablik (2022). a study on project selection using multi-criteria decision support indicated that among many existing techniques, ahp, anp, and topsis were the most popular methods (sadinezhad, 2017). khan and ali (2020) concluded that the ahp method is more widely preferred by researchers in almost all fields and applications. based on a review of the existing literature on the selection of health infrastructure projects, little research exists that uses the ahp or its extensive forms in this area. moreover, the covid-19 outbreak, declared a pandemic by the world health organization (who) on march 11, 2020, can be described as the largest multifaceted crisis ever faced by the modern world (ağaç & şimşir, 2022). globally, as of may 25, 2022, 524,339,768 confirmed cases of covid-19, including 6,281,260 deaths, have been reported to the who (oms, 2022). this pandemic revealed the vital importance of health infrastructure, such as hospitals, health centers, and laboratories, in preserving human life. countries worldwide, including algeria, have launched or relaunched health infrastructure programs and projects to address the shortcomings in the care of covid-19 patients and other patients affected by communicable and noncommunicable diseases. however, decision-makers in the health sector have more potential projects than available resources to implement them. the economic crisis caused by the covid-19 pandemic has also resulted in a decrease in financial resources, forcing countries to reduce their budgets. this has a direct impact on public facilities budgets, including health infrastructure projects. policymakers must adopt policies to rationalize spending and prioritize projects that are most beneficial to the community. this study aims to develop a methodology for prioritizing and selecting health infrastructure project alternatives using appropriate methods of multi-criteria decision-making in organizations responsible for their management and development. ijahp article: akhrouf, derghoum/use of analytic hierarchy process model for selection of health infrastructure projects international journal of the analytic hierarchy process 3 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1040 2. methodology the selection of a multi-criteria method to deal with a particular problem is a multicriteria decision problem with no obvious solution. opting for one method over another is often done arbitrarily, due to the lack of standard rules (al-shemmeri, 1997). the analytical hierarchy process (ahp) has been used to prioritize and select health infrastructure projects. it is recommended for this kind of ranking problem with scores (ishizaka & nemery, 2013). the ahp method has aroused the interest of many researchers mainly because of the good mathematical properties of the method and the fact that the required input data are rather easy to obtain (triantaphyllou & mann, 1995). its simplicity is characterized by comparing pairs of alternatives according to specific criteria (vargas, 2010). in addition, the method has been popularized and made easy to use by the development of several software programs. the most well-known are expert choice (saaty & forman, 2022) and super decisions (saaty & saaty, 2022; mu & pereyra-rojas, 2018). in this study, we used expert choice software version 11, which is a decision-making software that is based on multi-criteria decision-making and implements the ahp by computing the relative weight instead of calculating by hand. 2.1 presentation of the ahp method the fundamental problem with decision-making is choosing the best alternative from a set of competing alternatives that are evaluated according to contradictory criteria. the analytical hierarchy process (ahp) provides a comprehensive framework for solving such problems. it is a systematic procedure to represent the elements of any problem. it organizes basic rationality by breaking down a problem into its smaller constituent parts and then using simple pairwise comparison judgments to develop priorities in each hierarchy (saaty, 1986). the ahp method is one of the most widely used multi-criteria decision-making tools available to decision-makers and researchers (kumar & vaidya, 2006). many notable works have been published based on the ahp including applications in different areas such as planning, selection of a better alternative, resource allocations, conflict resolution, optimization, etc. (kumar & vaidya, 2006). kumar and vaidya (2006) predicted that the ahp will be widely used for decision-making and that the use of ad-hoc software applications will further address the complexity of integrated applications of ahp and other techniques to represent real-world situations. this generalization is certainly due to the method’s ease of application and the structure of the ahp that follows the intuitive way that managers solve problems. hierarchical modeling of the problem, the ability to adapt verbal judgments, and consistency checking are major assets of the ahp (ishizaka & labib, 2011). 2.2 ahp fundamentals three principles can be recognized in problem-solving: decomposition, comparative judgments, and synthesis of priorities (saaty, 1986). the ahp is a rigorous methodology that is divided into a series of important steps including structuring the hierarchy, setting priorities, and verifying the logical consistency of the analysis. ijahp article: akhrouf, derghoum/use of analytic hierarchy process model for selection of health infrastructure projects international journal of the analytic hierarchy process 4 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1040 2.2.1 establishing the hierarchical structure as in all decision-making processes, the facilitator works for a long time with the decision-maker(s) to structure and model the problem. the ahp has the advantage of allowing hierarchical structuring of criteria (figure 1) which allows users to better focus on specific criteria and sub-criteria when assigning weights. the number of components in each level generally ranges from five to nine (saaty, 1984). figure 1 hierarchy modeling the global diagnostic problem of companies (akhrouf, 2007) 2.2.2 prioritization the first step in prioritizing the different elements that are part of a decision problem is to make binary comparisons of elements at the same level of the hierarchy two by two against a given criterion while combining logical thinking with experience. the matrix presents the most effective framework for making such comparisons. to approach the process of binary comparisons, one must start at the top of the hierarchy and select the c or property criterion that will be used to perform the first comparison. then, from the level immediately below, we must consider the elements to compare: a1, a2, a3, etc. suppose we are dealing with seven elements, we would arrange them on a matrix as shown in figure 2. social performance overall business performance financial performance commercial performance technical performance organizational performance target (goal) company a company b company c c r it e r ia a l t e r n a t iv e s ijahp article: akhrouf, derghoum/use of analytic hierarchy process model for selection of health infrastructure projects international journal of the analytic hierarchy process 5 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1040 figure 2 binary comparison matrix to fill the matrix of binary comparisons, numbers are used to represent the relative importance of one element to another as a function of ownership. table 1 describes the scale used to make binary comparisons. cij represents the relative importance of ai compared to aj according to criterion c and cji is its reciprocal (cji=1/cij). table1 binary comparison scale of the ahp method (saaty, 1984) degree of importance definition explanation 1 equal importance of both elements two elements contribute equally to the property 3 low importance of one item over another personal experience and appreciation slightly favor one element over another 5 strong or decisive importance of one element over another personal experience and appreciation strongly favor one element over another 7 demonstrated importance of one element over another an element is strongly favored and its dominance is attested in practice 9 absolute importance of one element over another evidence favoring one element over another is as convincing as possible 2, 4, 6, 8 intermediate values between two neighboring assessments a compromise is necessary between the two assessments reciprocal if element i is assigned one of the previous digits cij when compared to element j, then cji has the reciprocal value1/cij when compared to i (the inverse of the number). the scale defines and explains the values from 1 to 9 assigned to ratings in comparing pairs of similar elements at each level of a hierarchy against a criterion at the next higher level. for example, let the comparison matrix of three elements a1, a2, and a3 concerning criterion c be: c 𝐴1 𝐴2 𝐴3 𝐴1 1 9 5 𝐴2 1/9 1 1/2 𝐴3 1/5 2 1 figure 3 example of a binary comparison matrix c 𝐴1 𝐴2 𝐴3    𝐴7 𝐴1 1 c12 c13    c17 𝐴2 c21 1 c23    c27 𝐴3 c31 c32 1    c37                         𝐴7 c71 c72 c73    1 ijahp article: akhrouf, derghoum/use of analytic hierarchy process model for selection of health infrastructure projects international journal of the analytic hierarchy process 6 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1040 in line 1 of the above matrix (figure 3), if 𝐴1 is an element that has absolute importance over 𝐴2, then c12=9 and c21=1/9. in the case that 𝐴2 is a more important element than 𝐴1, then c12=1/9 and c21=9. finally, if both are of equal importance, the rank would be 1, or c12=c21=1. 2.2.2.1 calculation of priorities calculating the priorities of the elements is equivalent to finding the normalized eigenvector of the matrix as shown in table 2. table 2 calculation of priorities c a1 a2 a3 a1 a2 a3 sum of rows sum of rows divided by the number of items (n=3) a1 1.00 9.00 5.00 (1/1.31) = 0.76 (9/12) = 0.75 (1/6.5) = 0.77 (0.76 +0.75 +0.77) = 2.28 (2.28/3) = 0.76 (76%) a2 0.11 1.00 0.50 (0.11/1.31) = 0.08 (1/12) = 0.08 (0.5/6.5) = 0.08 (0.08 +0.08 +0.08) = 0.25 (0.25/3) = 0.08 (8%) a3 0.20 2.00 1.00 (0.2/1.31) = 0.15 (2/12) = 0.17 (1/6.5) = 0.15 (0.15 +0.17 +0.15) = 0.47 (0.47/3) = 0.16 (16%) 1.31 12.00 6.50 1.00 1.00 1.00 sum of each column (s) standardized matrix sum of each column = 1 eigenvector normalized eigenvector => priority vector the priority vector is given by the normalized eigenvector (0.76; 0.08; 0.16). the final classification of the three elements a1, a2, and a3 concerning criterion c is as follows: a1 with a weighting of 0.76 (76%); a3 with a weighting of 0.16 (16%) and finally, a2 with a weighting of 0.08 (8%). 2.2.2.2 consistency of judgments the ahp method evaluates the overall consistency of assessments using a consistency ratio. the value of the consistency ratio must be equal to or less than 10%. if it is greater than 10%, the assessments may be somewhat random and require some revisions (saaty, 1984). saaty proposed a coherence index (ci) which is linked to the method of the eigenvalues of matrices and is calculated by equation (1): 𝐶𝐼 = (𝜆𝑚𝑎𝑥 − 𝑛) (𝑛 − 1) (1) where n is the dimension of the matrix and max (lambda max) is the maximum eigenvalue. ijahp article: akhrouf, derghoum/use of analytic hierarchy process model for selection of health infrastructure projects international journal of the analytic hierarchy process 7 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1040 the consistency ratio (cr) is calculated by equation (2): 𝐶𝑅 = 𝐶𝐼 𝑅𝐼 (2) where ri is the random coherence index (ri = means ci of 500 randomly filled matrices). random indices are presented in table 3. if the cr is less than 10%, then the matrix can be considered to have acceptable consistency. table 3 random consistency indices (saaty, 1984) matrix dimension 1 2 3 4 5 6 7 8 9 10 random consistency 0.00 0.00 0.58 0.90 1.12 1.24 1.32 1.41 1.45 1.49 for the matrix in figure 3, the maximum eigenvalue is calculated as shown in table 4. table 4 calculation of the eigenvalue the dimension of the matrix n=3; max=3; then, 𝐶𝐼 = (𝜆𝑚𝑎𝑥−𝑛) (𝑛−1) = (3−3) (3−1) = 0; thus, ri=0.58; ci=0; cr=0; the cr is less than 10%; therefore, the matrix can be considered to have acceptable consistency. 3. application of the ahp for the selection of health infrastructure projects the prototyping approach inspired by the delphi method was used in the process of designing and building the multicriteria decision support model. the ideas on the composition and structuring of the model are primarily based on documentary analysis of literature in the research field, personal reflections, and on a practical case study. in the case study, we conducted on-site investigations through interviews and questionnaires with a group of decision-makers and experts representing stakeholders in the planning and management of health infrastructure projects. primarily, officials from the directorate of studies and planning (dep) at the algerian ministry of health, population, and hospital reform (msprh) were interviewed. we also worked with officials from the health and population directorates of the district of tipaza and bouira. interviews were organized with officials in charge of the health eigenvector a1 (0.76) a2 (0.08) a3 (0.16) sum (s) 1.31 12.00 6.50 𝑀𝑎𝑥𝑖𝑚𝑢𝑚 𝑒𝑖𝑔𝑒𝑛𝑣𝑎𝑙𝑢𝑒 𝜆𝑚𝑎𝑥 [(1.31 x 0.76) + (12 x 0.08) + (6.5 x 0.16)] = 2.9956~3 ijahp article: akhrouf, derghoum/use of analytic hierarchy process model for selection of health infrastructure projects international journal of the analytic hierarchy process 8 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1040 sector at the general directorate of budget (dgb) in the ministry of finance. experts from the national fund for equipment and development (cned) attached to the ministry of finance guided us through the provision of project maturity guides. some judges from the court of auditors (cc) responsible for the health sector were also solicited, and they enriched the study from the perspective of auditors controlling public spending. interviews were also conducted with researchers from the higher school of commerce (esc) and the school of health management (enmas). the ahp model of this study is applied to the selection of health infrastructure projects. it is structured in four levels with seven main criteria, eighteen basic subcriteria, and three alternating projects. the result of the hierarchical breakdown for the selection of the projects envisaged is summarized in table 5. table 5 structuring the problem of selecting health infrastructure projects 3.1 objective (subject) the desired objective was to select the best health infrastructure projects from a set of candidate projects while respecting several criteria (political, sociodemographic, epidemiological, etc.). 3.1.1 criteria and sub-criteria we established the list of criteria based on the objectives set by the national health plan and the suggestions of the various actors in the decision-making process for the selection of algerian health projects. we also used some criteria from the literature (vargas, 2010; şahin, 2019; vahidnia, 2009; chatterjee, 2013; ağaç & şimşir, 2022; chatterjee, 2018). given the large number of factors to be taken into account, we selected seven main criteria as follows: political, geographical, sociodemographic, epidemiological, technical, economic and financial, and environmental. these criteria were broken down into 18 sub-criteria. the stakeholders recognized the relevance and completeness of the family of criteria we used to evaluate and select health projects from the community's point of view. 3.1.2 alternatives in our case, the alternatives represent the different possible health infrastructure projects. three different projects have been identified and must be prioritized.  project 1: realization of a polyclinic  project 2: realization of an anti-cancer center (acc)  project 3: construction of a 60-bed hospital purpose (objective) selection of proposed health infrastructure projects criteria political; geographical; sociodemographic; epidemiological; technical; economic and financial; environmental sub-criteria each criterion was broken down into the corresponding sub-criteria alternatives alternatives represent the various health infrastructure projects (hospitals) candidates for selection. https://www.researchgate.net/profile/tezcan-sahin ijahp article: akhrouf, derghoum/use of analytic hierarchy process model for selection of health infrastructure projects international journal of the analytic hierarchy process 9 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1040 4. results and discussion 4.1 establishing the hierarchical structure the criteria to be taken into account are schematized hierarchically in figure 4. to present this hierarchy, which is broken down into eighteen sub-criteria, the subcriteria were grouped into seven main criteria. the criteria, the sub-criteria, and the alternatives are defined in detail in table 6. ijahp article: akhrouf, derghoum/use of analytic hierarchy process model for selection of health infrastructure projects international journal of the analytic hierarchy process 10 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1040 . figure 4 hierarchical structure for the selection of health infrastructure projects level 1 goal level 2 criteria level 3 sub-criteria level 4 alternatives pupose (objective): selection of health infrastructure projects political criterion equity and territorial balance (great south programs, grey areas, ...) public health policy projections of the sectoral health master plan in accordance with the snat guidelines international commitments (who, sdg3,..) geographic criterion relief of the territory climate of the territory sociodemographic criterion total population of the territory the age structure of the population epidémiological criterion communicable diseases noncommunicable diseases technical criterion location of the project (construction site) hospital capacity (number of beds) economic and financial criteria planned budget envelope impacts of the project on the territory job creation environnemental criterion noise pollution on the territory risk of pollution treatment of liquid and solid waste project 1 project 2 project 3 ijahp article: akhrouf, derghoum/use of analytic hierarchy process model for selection of health infrastructure projects international journal of the analytic hierarchy process 11 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1040 table 6 descriptive table of criteria, sub-criteria, and alternatives criteria sub-criteria definition political criterion political criteria are related to political and strategic considerations to be considered before integrating studied projects into the national or regional economy. international commitments (who, mdgs, odd3, rsi, etc.) this criterion is evaluated on a qualitative scale reflecting the degree of contribution of a health project to achieving these strategic objectives, particularly sustainable development goals sdg3 relating to health. health public policy this criterion is evaluated on a qualitative scale reflecting the degree of contribution of the project to the implementation of the public health policy in algeria. equity and territorial balance (great south programs, shadow zones, etc.) this criterion is evaluated on a qualitative scale reflecting the significance of its potential for economic-social-sanitary growth effects throughout the territory. projections of the health sector master plan in accordance with the snat guidelines this criterion is evaluated on a qualitative scale reflecting the coherence of the project with the national master plan for the development of health services and infrastructure. geographical criterion geographical criteria are related to climatic considerations and the terrain of the areas where the studied projects are to be located. relief of the territory the terrain of a project's location influences the choice of new health infrastructure projects. it can determine the nature and type of infrastructure to be developed and the health services that will be integrated. territory climate the climatic conditions of a project's location also influence the choice of a new health facility project. sociodemographic criterion sociodemographic criteria are related to sociological and demographic considerations of the areas where the studied projects are to be located. they are considered as factors of demand for health care, as defined by world health organization. territory total population the total population of the area is a determining factor for the number of hospital beds, size, and capacity of a health facility project. this sub-criterion can be easily evaluated quantitatively (numerical value) from demographic studies on the population basin by the project owner. population age structure the population’s age structure also determines the nature of health services to be integrated into a project. this sub-criterion can be easily evaluated quantitatively (numerical value) from demographic studies on the population basin conducted by the project owner concerned. epidemiological criterion epidemiological criteria are related to epidemiological considerations that characterize the areas where the studied projects are to be located, particularly in preventing and controlling communicable and non-communicable diseases. communicable diseases it is difficult to determine the contribution to the fight against communicable diseases for health projects in the algerian context, which is why it is evaluated qualitatively by expert estimation. non-communicable diseases it is difficult to determine the contribution to the fight against non-communicable diseases for health projects in the algerian context, which is why it is evaluated qualitatively by expert estimation. ijahp article: akhrouf, derghoum/use of analytic hierarchy process model for selection of health infrastructure projects international journal of the analytic hierarchy process 12 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1040 criteria sub-criteria definition technical criterion technical criteria are related to technical considerations that characterize the studied projects, particularly in terms of hospital bed capacity and the project's location. project location (construction site) the choice of project location is made by comparing identified advantages and disadvantages. the geographical location is important as perceived by decision-makers and health sector planners. hospital capacity (number of beds) hospital capacity refers to the number of hospital beds planned for the health infrastructure project, which can vary from ten beds for a polyclinic to a hundred beds for a university hospital center. this sub-criterion can be easily evaluated quantitatively (numerical value) from technical feasibility studies conducted by the project owner concerned. economic and financial criterion economic and financial criteria are related to the economic and financial effects caused by the integration of the studied projects into the national or regional economy. planned budget envelope the importance of investment projects included in the state budget implies measuring the impact on public finances of the initial investment costs (first establishment expenses) as well as the operating costs of the infrastructure to be carried out. this sub-criterion can be easily evaluated quantitatively (monetary value) based on the financial feasibility studies carried out by the project owner concerned project impacts on the territory this criterion measures the expected impact on the economic development of a territory and the implementation of a hospital equipment location. job creation this criterion concerns the evaluation of the number of direct, indirect, and secondary jobs created by a health project. environmental criterion considering the importance in the context of a comparative environmental assessment of proposed projects, we have only referred to this criterion qualitatively. noise pollution on the territory noise pollution can have harmful effects on citizens' health. for this reason, "noise nuisance" is included as an environmental sub-criterion for the selection of healthcare infrastructure projects pollution risk this criterion concerns any project activity that has a potential impact on air pollution and the climate conditions of the planet. they are mainly caused by greenhouse gas emissions. the measures affected by this criterion are determined by environmental experts. liquid and solid waste processing in order to neutralize the effects of harmful waste on the environment, it is necessary to verify the proposed systems for storage, removal, treatment, and discharge for each project. alternatives the alternatives represent the various health infrastructure projects (hospitals) candidates for selection project 1: realization of a polyclinic a polyclinic is a healthcare facility that provides a range of medical services, including primary care, specialized care, and diagnostic services. it is usually larger than a traditional clinic and offers more comprehensive healthcare services. project 2: realization of an anti-cancer center (acc) an anti-cancer center is a specialized medical facility that provides diagnosis, treatment, and care for individuals with cancer. these centers often offer comprehensive services, including chemotherapy, radiation therapy, and surgery, as well as support services such as nutrition counseling and psychological support. project 3: construction of a 60-bed hospital a general hospital with 60 beds is a medical facility that provides a range of basic and specialized healthcare services to patients. it typically has a variety of departments and staff, including physicians, nurses, and other healthcare professionals, who work together to provide comprehensive care to patients. ijahp article: akhrouf, derghoum/use of analytic hierarchy process model for selection of health infrastructure projects international journal of the analytic hierarchy process 13 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1040 4.2 prioritization each of the criteria and sub-criteria belonging to the same level of hierarchical decomposition do not have the same importance in their ability to contribute to the resolution of the project selection problem. evaluators or experts were asked to compare the relative importance of all elements of the criteria with a pairwise comparison. primarily, officials from the directorate of studies and planning (dsp) at the algerian ministry of health, population, and hospital reform (mhphr) were interviewed, but we also worked with officials from the health and population directorates of the district of tipaza. two judges from the court of auditors (ca) responsible for the health sector were also solicited, and they enriched our study from the perspective of auditors controlling public spending. 4.2.1 prioritization of criteria and sub-criteria respondents or decision-makers provided judgments on the relative importance of the primary criteria of the first hierarchical level and the sub-criteria of the second level to assess their contribution to the selection of projects. these judgments collected on a numerical scale can be in the form of matrices of comparisons. from there, we calculated the weights of the relative importance or the percentages of the relative importance to each of these elements in the first hierarchical level. the expert choice software automatically calculated the relative importance of the first-level criteria according to the overall objective pursued (see figure 5). figure 5 priorities and weighting of criteria (data provided by expert choice software) since the demand for healthcare and services comes mainly from the population, which is a necessary determinant of any health policy, it is logical that the sociodemographic criterion would be the most important for evaluating a health project. thus, the sociodemographic criterion obtained a weight (relative importance) of 32.4%. the ijahp article: akhrouf, derghoum/use of analytic hierarchy process model for selection of health infrastructure projects international journal of the analytic hierarchy process 14 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1040 epidemiological criterion is also considered another determining factor in the demand for the healthcare of the population and therefore ranked second in the order of priority with a value of 0.3 or 30% relative importance. the technical criterion ranked last with a value of 0.025 or 2.5%. these rankings currently express the preference of the criteria used for the evaluation of health infrastructure projects in algeria. the overall relative priorities assigned to the criteria and sub-criteria considered by the relevant respondents were adjusted to obtain the overall relative priorities of the selection criteria set for each proposed project as presented in table 7. table 7 overall relative priorities of all project selection sub-criteria (data provided by expert choice software) ranking project selection criteria overall relative priorities 1 total population of the territory 0.162 2 age structure of the population 0.162 3 communicable diseases 0.150 4 noncommunicable diseases 0.150 5 impacts of the project on the territory 0.078 6 planned budget envelope 0.078 7 relief of the territory 0.046 8 climate of the territory 0.046 9 noise pollution on the territory 0.017 10 risk of pollution 0.017 11 treatment of liquid and solid waste 0.017 12 location of the project (construction site) 0.013 13 hospital capacity (number of beds) 0.013 14 job creation 0.013 15 projections of the sectoral health master plan following the snat guidelines 0.012 16 public health policy 0.012 17 equity and territorial balance (great south programs, grey areas, etc.) 0.012 18 international commitments (who, mdgs, sdg3, isr, etc.) 0.002 ijahp article: akhrouf, derghoum/use of analytic hierarchy process model for selection of health infrastructure projects international journal of the analytic hierarchy process 15 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1040 figure 6 overall relative priorities of the sub-criteria (excel software graph) table 6 and figure 6 show the sociodemographic sub-criteria “total population of the territory” and “age structure of the population” are the most important criteria for selecting health infrastructure projects in algeria. this is followed by the epidemiological sub-criteria “communicable diseases” and “noncommunicable diseases” in second place. this order of preference fits perfectly with the practice of selecting health projects in algeria, as they are considered to be important and determining factors in the demand for healthcare for the population. 4.2.2 prioritizing alternatives after establishing the priorities of the criteria, it is now possible to determine how each of the candidate projects is assessed against the selected criteria. in the same way that the prioritization of the criteria was made, the candidate projects are compared in pairs (two to two), taking into account all the established criteria. for our case, three different projects have been identified and must be prioritized. the overall project priorities are illustrated in figure 7. 0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 1 total population of the territory 2 the age structure of the population 3 communicable diseases 4 noncommunicable diseases 5 impacts of the project on the territory 6 planned budget envelope 7 relief of the territory 8 climate of the territory 9 noise pollution on the territory 10 risk of pollution 11 treatment of liquid and solid waste 12 location of the project (construction… 13 hospital capacity (number of beds) 14 job creation 15 projections of the sectoral health master… 16 public health policy 17 equity and territorial balance (great south… 18 international commitments (who,… o v e r a l l r e l a t i v e p r i o r i t i e s o f s u b c r i t e r i a ijahp article: akhrouf, derghoum/use of analytic hierarchy process model for selection of health infrastructure projects international journal of the analytic hierarchy process 16 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1040 figure 7 overall project priorities (data provided by expert choice software) the results show that project 1, “realization of a polyclinic” obtained the highest level of adherence to the objective of selecting projects for the realization of health infrastructure. this project contributes 46.7% (0.467) to the overall target and is followed by project 3, “realization of a hospital with 60 beds” with a contribution of 42.3% (0.423). at the bottom of the ranking is project 2 “realization of a cancer center (cac)” with a contribution of 11% (0.11) compared to the overall objective of selecting projects for the realization of health infrastructure. this result of the final ranking of health infrastructure projects perfectly reflects the public health policy followed by the public authorities in algeria. this observation can be confirmed by the current algerian minister of health who states that “70% of patients admitted to the hospital can be treated at polyclinics. work is underway as part of the sector's plan to direct, at least, half of this rate to local structures to relieve pressure on hospitals, which will allow them to carry out the missions assigned to them” (ministre de la santé, 2021). 4.2.3 consistency of judgments the expert choice software provides an automatic and interactive inconsistency ratio test for each level of hierarchical decomposition. table 8 shows the inconsistency ratios for the criteria and sub-criteria, respectively, given by relevant respondents in this search. the inconsistency ratio for all judgments is 0.0932 (9.32%), lower than the 10% threshold set by saaty (1984). this indicates that inconsistencies in respondents' ijahp article: akhrouf, derghoum/use of analytic hierarchy process model for selection of health infrastructure projects international journal of the analytic hierarchy process 17 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1040 preferences are not significant. there is no need to revise the content of the comparison matrices. table 8 indices of consistency of all judgments (data provided by expert choice software) criteria consistency index level 1: purpose selection of health infrastructure projects 0.1135 level 2: criteria political criterion (l: .037 g: .037) 0.0000 geographic criterion (l: .092 g: .092) 0.0000 sociodemographic criterion (l: .324 g: .324) 0.0000 epidemiological criterion (l: .300 g: .300) 0.0000 technical criterion (l: .025 g: .025) 0.0000 economic and financial criterion (l: .170 g: .170) 0.0000 environmental criterion (l: .052 g: .052) 0.0000 overall inconsistency ratio (global) 0.0932 4.2.4 sensitivity analysis the final priorities of the alternatives will be strongly influenced by the weightings given to the respective criteria. it is useful to perform an analysis of “what would happen if” to see how the final results would be changed if the weights of the criteria had been different. (mu & pereya-rojas, 2018). the sensitivity analysis of the classification of alternatives (health projects) was carried out using the expert choice software. this analysis is helpful for understanding the effect of changing the weights of the main criteria on the ranking of projects. to this end, it was decided to vary the relative weights of certain criteria differently as follows:  using larger variations around the weights of the socioeconomic, epidemiological, economic, and financial criteria since these are more relevant to the overall objective.  using smaller variations around the weight of political and technical criteria. starting from the initial scenario shown in figure 8, the scenario’s sensitivity analysis concerning changes in criteria weights was achieved by changing the weight of a given criterion at a time as shown in table 9. ijahp article: akhrouf, derghoum/use of analytic hierarchy process model for selection of health infrastructure projects international journal of the analytic hierarchy process 18 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1040 figure 8 sensitivity analysis – initial scenario (data provided by expert choice software) we selected five different scenarios:  scenario 1: sensitivity analysis concerning the sociodemographic criterion  scenario 2. sensitivity analysis concerning the epidemiological criterion  scenario 3: sensitivity analysis concerning the economic and financial criterion  scenario 4: sensitivity analysis concerning the technical criterion  scenario 5: sensitivity analysis concerning the political criterion table 9 shows the changes in the relative weights of the main criteria. generally speaking, each scenario (1 to 5) corresponds to a system of relative weights of the project selection criteria that we can obtain in an interactive way using the expert choice software. ijahp article: akhrouf, derghoum/use of analytic hierarchy process model for selection of health infrastructure projects international journal of the analytic hierarchy process 19 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1040 table 9 sensitivity analysis scenarios (data provided by expert choice software) change in weights to consider in it ia l sc e n a ri o s c e n a ri o 1 s c e n a ri o 2 s c e n a ri o 3 s c e n a ri o 4 s c e n a ri o 5 c r it e r ia political 0.037 0.028 0.018 0.023 0.019 0.501 geographical 0.092 0.068 0.044 0.055 0.047 0.048 sociodemographic 0.324 0.501 0.321 0.195 0.166 0.168 epidemiological 0.300 0.221 0.501 0.180 0.153 0.155 technical 0.025 0.019 0.012 0.015 0.501 0.013 economic and financial 0.170 0.125 0.08 0.501 0.087 0.088 environmental 0.052 0.038 0.025 0.031 0.027 0.027 total weights 1 1 1 1 1 1 project weighting results p r o je c t s project 1: realization of a polyclinic 0.467 0.508 0.426 0.471 0.398 0.490 project 2: construction of a 60-bed hospital 0.110 0.104 0.087 0.117 0.139 0.096 project 3: creation of an anti-cancer center (acc) 0.423 0.388 0.487 0.412 0.463 0.414 an examination of table 9 shows that when the weight of one criterion is increased the majority of the weights of the other criteria decrease. it is easy to understand that, for example, if the weight of the sociodemographic criterion increases more than the other criteria, the reduction of the other weights would be proportional because the total sum of the weights is always equal to 1 (saaty, 1984). scenario 1: sensitivity analysis for the sociodemographic criterion by varying the weight of the sociodemographic criterion to reach a value of 50% of the relative importance to the objective, we found that the ranking of the projects remained invariable compared to the initial scenario. project 1 was still ranked first with a priority of 50.8% followed by project 3 with a priority of 38.8% and project 2 with a priority of 10.4% (see figure 9). ijahp article: akhrouf, derghoum/use of analytic hierarchy process model for selection of health infrastructure projects international journal of the analytic hierarchy process 20 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1040 figure 1 sensitivity analysis scenario 1 (data provided by expert choice software) scenario 2. sensitivity analysis for the epidemiological criterion by varying the weight of the epidemiological criterion to reach a value of 50% of the relative importance to the objective, we saw a change in the ranking of projects compared to the initial scenario. now, project 3 was ranked first with a priority of 48.7%, followed by project 1 with a priority of 42.6% and project 2 with a priority of 8.7% (see figure 10). figure 2 sensitivity analysisscenario 2 (data provided by expert choice software) scenario 3: sensitivity analysis for the economic and financial criterion by varying the weight of the economic and financial criterion to reach a value of 50% of the relative importance to the objective, we found that the ranking of projects remained invariable compared to the initial scenario. project 1 was still ranked first with a priority of 47.1%, followed by project 3 with a priority of 41.2% and project 2 with a priority of 11.7% (see figure 11). ijahp article: akhrouf, derghoum/use of analytic hierarchy process model for selection of health infrastructure projects international journal of the analytic hierarchy process 21 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1040 figure 3 sensitivity analysis scenario 3 (data provided by expert choice software) scenario 4: sensitivity analysis for the technical criterion by varying the weight of the technical criterion to reach a value of 50% of the relative importance to the objective, we saw a change in the ranking of projects compared to the initial scenario. project 3 was ranked first with a priority of 46.3%, followed by project 1 with a priority of 39.8% and project 2 with a priority of 13.9% (see figure 12). figure 12 sensitivity analysisscenario 4 (data provided by expert choice software) scenario 5: sensitivity analysis for the political criterion by varying the weight of the political criterion in the range of values [0.0001... 0.94991] from the relative importance to the objective, we found that the ranking of projects remained invariable compared to the initial scenario. project 1 was always ranked first with a priority that varied between 45.75% and 50.99%, followed by project 3 with a ijahp article: akhrouf, derghoum/use of analytic hierarchy process model for selection of health infrastructure projects international journal of the analytic hierarchy process 22 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1040 priority close to project 1 that varied between 44.42% and 40.77% and project 2 with a priority that varied between 9.82% and 8.22% (figure 13). figure 13 sensitivity analysisscenario 5 (data provided by expert choice software) 4.2.5 decision-making once the above steps are completed, it is now possible to make a decision. this is the final step in the ahp analysis. this requires a comparison of the overall priorities achieved and an assessment of whether the gaps are large enough to make a clear choice. it is also necessary to analyze the results of the sensitivity analysis. from this analysis, we can express our final recommendation. if the importance of the sociodemographic criterion is greater than 50% of the overall importance of the criteria in the decision, the best alternative is project 1“realization of a polyclinic”; however, if the importance of the sociodemographic criterion is well below 50%, in other words, the importance of the epidemiological criterion is greater than 50%, in this case, project 3 “realization of a hospital with 60 beds” is the best decision. 4.3 practical implications of the study the methodology developed in this article proposed an approach based on the ahp multi-criteria decision support method for decision-makers and stakeholders to prioritize and select projects in the health sector. the results obtained regarding the importance of the sociodemographic and epidemiological criteria for the selection of health projects as well as the relative importance of polyclinic projects are in line with the current trends that dominate and regulate health projects in algeria. the ahp offers flexibility for the selection of health infrastructure projects and uses several qualitative and quantitative criteria. ijahp article: akhrouf, derghoum/use of analytic hierarchy process model for selection of health infrastructure projects international journal of the analytic hierarchy process 23 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1040 4.4 originality and value of the study this is the first study that implements the ahp in the process of prioritization and selection of projects in the healthcare sector. as a first implementation, the study provides decision-makers with a methodology that takes into account both qualitative criteria such as “contribution to the fight against communicable and non-communicable diseases” or “equity and territorial balance”, and quantitative criteria such as “planned budget envelope” or “number of jobs created” to evaluate alternatives and gives importance to the ranking of experts related to the criteria and performance values of alternatives for criteria. 5. conclusion the study focuses on the problem of selecting public health sector projects while trying to simultaneously achieve a variety of objectives and satisfy selection criteria. a multicriteria decision support model based on the analytical hierarchy process (ahp) was offered to assist in the selection of projects that meet multiple criteria. this approach allows for a sensitivity analysis to select the most satisfactory projects and provides decision-makers with important information. the ahp method takes into account all criteria for evaluating projects, both qualitative and quantitative, and facilitates communication between the decision-makers and experts to increase the amount of information available. the study focuses on the selection of health infrastructure projects in algeria and the decision-making elements are grouped into seven main criteria, including political, geographical, sociodemographic, epidemiological, technical, economic and financial and environmental criteria. the sensitivity analysis provided insights into the solidity of the results and the importance of the different parameters involved. the approach developed can be applied to other problems in selecting social infrastructure projects or economic investment projects. further studies may consider integrated multi-criteria decisionmaking approaches and the integration of artificial intelligence techniques with ahp. the decision support model can also be improved to take into account multiple decisionmakers. ijahp article: akhrouf, derghoum/use of analytic hierarchy process model for selection of health infrastructure projects international journal of the analytic hierarchy process 24 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1040 references ağaç , g., & şimşir , i̇. (2022). optimal site selection of a pandemic hospital using a multi-criteria decision-making approach. international journal of the analytical hierarchy process, 14(1), 1-27. doi: https://doi.org/10.13033/ijahp.v14i1.946 akhrouf, m. (2007). contribution au développement de systèmes d'aide à la décision multicritère (sadm): une application au diagnostic global d'entreprises. revue des réformes économique set intégration en économie mondiale n°2(2007), 73-94. al-shemmeri, t., al-kloub, b. &. pearman, a. (1997). model choice in multicriteria decision aid. european journal of operational research, 97(3), 550-560. doi: https://doi.org/10.1016/s0377-2217(96)00277-9 chatterjee, d. (2013). potential hospital location selection using ahp: a study in rural india. international journal of computer applications, 71(17), 1-7. doi: https://doi.org/10.5120/12447-9144 chatterjee, k., hossain, s., & kar, s. (2018). prioritization of project proposals in portfolio management using fuzzy ahp. opsearch, 55, 478–501. doi: https://doi.org/10.1007/s12597-018-0331-3 ciptomulyono, u. (2000). un modèle d'aide à la sélection des projets : l'intégration de la procédure analyse hiérarchique (ahp) et la programmation mathématique a objectifs multiples. doctoral thesis, universite de droit, d’economie et des sciences, aix marseilles. ishizaka, a., & labib, a. (2011). review of the main developments in the analytic hierarchy process. expert systems with applications 38(11), 14336-14345. ishizaka, a., & nemery, p. (2013). multi-criteria decision analysis: methods and software. west sussex: wiley & sons, ltd. jurı´k, l., horňáková, n., šantavá, e., cagáňová, d., & sablik, j. (2022). application of the ahp method for project selection in the context of sustainable development. wireless networks, 28, 893–902. doi: https://doi.org/10.1007/s11276-020-02322-2 keeney, r., & raiffa, h. (1976). decisions with multiple objectives: preferences and value tradeoffs. new york: cambridge university press. doi: https://doi.org/10.1002/bs.3830390206 khan, a., & ali, y. (2020). analytical hierarchy process (ahp) and analytic network process methods and their applications: a twenty-year review from 2000-2019. international journal of the analytic hierarchy process, 12(3), 369-459. doi: https://doi.org/10.13033/ijahp.v12i3.822 ijahp article: akhrouf, derghoum/use of analytic hierarchy process model for selection of health infrastructure projects international journal of the analytic hierarchy process 25 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1040 kumar, s., & vaidya, o. (2006). analytic hierarchy process: an overview of applications. european journal of operational research 169(1), 1-69. doi: https://doi.org/10.1016/j.ejor.2004.04.028 ministre de la santé. (2021). santé maghreb. mu, e., & pereyra-rojas, m. (2018). practical decision making using super decisions v3, an introduction to the analytic hierarchy process. springer. doi: https://doi.org/10.1007/978-3-319-68369-0_2 oms. (2022, mai 26). tableau de bord de l'oms sur le coronavirus (covid-19). who: https://covid19.who.int/ pomerol, j.-c., & barba-romero, s. (2000). multicriterion decision in management: principles and practice. new york: springer science+business media, llc. doi: https://doi.org/10.1007/978-1-4615-4459-3_11 saaty, t.l. & saaty, r.w. (2022, 5 11). about superdecisions. http://www.superdecisions.com/about/index.php?section=founders saaty, t. (1984). décider face à la compléxité. paris: esf. saaty, t. (1986). axiomatic foundations of the analytic hierarchy process. management science, 32(7), 841-855. doi: https://doi.org/10.1287/mnsc.32.7.841 saaty, t. (1996). decision making with dependence and feedback, the analytic network process. pittsburgh: rws publications. saaty, t., & forman, e. (2022). expert choice. retrieved may 26, 2021 from expertchoice: http://www.expertchoice.com sadi-nezhad, s. (2017). a state-of-art survey on project selection using mcdm techniques. journal of project management 2(1), 1–10. doi: https://doi.org/10.5267/j.jpm.2017.6.001 şahin, t., ocak, s., top, m. (2019). analytic hierarchy process for hospital site selection. health policy and technology, 8(1), 44-45. doi: https://doi.org/10.1016/j.hlpt.2019.02.005 saracoğlu, b. (2015). an ahp application in the investment selection problem of small hydropower plants in turkey. international journal of the analytic hierarchy process, 7(2), 211-239. doi: https://doi.org/10.13033/ijahp.v7i2.198 triantaphyllou, e., & mann, s. (1995). using the analytic hierarchy process for decision making in engineering applications: some challenges. international journal of industrial engineering: applications and practice, 2(1), 35-44. ijahp article: akhrouf, derghoum/use of analytic hierarchy process model for selection of health infrastructure projects international journal of the analytic hierarchy process 26 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1040 vahidnia, m. h., alesheikh, a., alimohammadi, a. (2009). hospital site selection using fuzzy ahp and its derivatives. journal of environmental management, 90(10), 30483056. doi: https://doi.org/10.1016/j.jenvman.2009.04.010 vargas, r. v. (2010). using the analytic hierarchy process (ahp) to select and prioritize projects in a portfolio. paper presented at pmi® global congress 2010-north america, washington, dc. newtown square, pa: project management institute. 2 final_girard_cerreta_de_toro ijahp article: girard, l.f., cerreta, m., de toro, p. / analytic hierarchy process (ahp) and geographical information systems (gis): an integrated spatial assessment for planning strategic choices international journal of the vol. 4 issue 1 2012 analytic hierarchy process 4 issn 1936-6744 analytic hierarchy process (ahp) and geographical information systems (gis): an integrated spatial assessment for planning strategic choices luigi fusco girard department of conservation of architectural and environmental heritage university of naples federico ii, italy e-mail: girard@unina.it maria cerreta∗ department of conservation of architectural and environmental heritage university of naples federico ii, italy e-mail: cerreta@unina.it pasquale de toro department of conservation of architectural and environmental heritage university of naples federico ii, italy e-mail: detoro@unina.it abstract in the contemporary debate regarding environmental assessment and integrated approaches, a spatial decision support system (sdss) designed to help a user or a group of users reach more effective decisions by solving semi-structured spatial problems can be supported by geographical information systems (gis) combined with an analytic hierarchy process (ahp). this paper explores the potential of multi-criteria spatial decision support systems (mcsdss), denominated integrated spatial assessment (isa), for the field of land-use planning. such a system takes into account both technical knowledge regarding the decision problem at hand and the lay knowledge of the local community in the construction of shared planning choices. through the empirical investigation of an operative case study, an integrated evaluative approach implemented by means of mcsdss can go beyond spatial and hierarchical limits taking into account the different components, clarifying weights and recognizing different priorities to define appropriate strategies, considering social participation and dynamic dialogue among different experts in keeping with sustainability principles. keywords: analytic hierarchy process, geographic information system, integrated assessment, multi-criteria spatial decision support system ∗ corresponding author rob typewritten text http://dx.doi.org/10.13033/ijahp.v4i1.102 ijahp article: girard, l.f., cerreta, m., de toro, p. / analytic hierarchy process (ahp) and geographical information systems (gis): an integrated spatial assessment for planning strategic choices international journal of the vol. 4 issue 1 2012 analytic hierarchy process 5 issn 1936-6744 1. introduction assessing the impacts of policies, plans, projects and programs requires particular attention to the values of existing resources and the affected communities. defining what is relevant to an assessment requires a higher level of benchmarking that can enable a broader perspective within the evaluation process in order to balance community values with sustainable development goals. recognizing multiple interdependent values creates the conceptual and empirical foundations for understanding just how these values can be applied to a local development process. this means becoming aware of the “complex social values” of resources (fusco girard, 1987; fusco girard and nijkamp, 1997), taking into account the community’s point of view. today “hard” values the tangible, material and monetary ones are dominant, so that “soft” values the intangible, immaterial and non-monetary ones are often forgotten. recognizing tangible and intangible values is the basis for collective decision-making that includes the development and definition of goals, sharing of knowledge, negotiation and compromise, problem-setting and problem-solving, and needs evaluation along with attention to issues of justice and equity (sinclair, sims, and spaling, 2009). this means supporting stakeholders and communities in clarifying values, becoming more adaptive and proactive, responding to change, setting personal and collective goals, and participating in planning and design decision-making processes. through integrated evaluation approaches, it is possible to reach a balance between preservation and transformation while respecting values and existing forms of capital (human, social, cultural, environmental, economic, etc.). with this perspective, it is essential to consider the different levels of analysis and the many dimensions of evaluation including specific attention to emerging problems, local needs, stakeholder interests, and protection and enhancement of scarce resources (wiek and walter, 2009). an integrated evaluation approach can go beyond spatial and hierarchical limits to consider the different components, clarify weights, recognize different priorities and define appropriate strategies while also considering social participation and a dynamic dialogue among different experts (lee, 2006). the different approaches are described in the literature as forms of “sustainability assessment” evaluated in terms of their potential contributions to the implementation of sustainability. many are actually examples of “integrated assessment” (pope, annandale, and morrison-saunders, 2004) deriving above all from environmental impact assessment (eia) and strategic environmental assessment (sea) (therivel, 2010). indeed, the term “integration” implies that integrated assessment should be more than the sum of separate environmental, social and economic assessments. according to eggenberger and partidário (2000), “integration” means that a new entity is created that establishes new relationships influencing single entities having specific characteristics and dynamics. if this viewpoint is taken into account, it holds that integrated evaluation can become a “key tool” in supporting the decision-making process especially when uncertainty, complexity and values of different social groups are many, differentiated and conflicting (cerreta and de toro, 2010). integrated evaluations not only consider the input of data expressing the impacts of different solutions but they are also “open” to broad public participation in order to offer more information for the evaluation process itself and, in addition, make decision-making processes and their results more acceptable (munda, 2008). participation becomes essential not only in examining and evaluating choices on the social, ethic, political, economic, environmental levels but also in ijahp article: girard, l.f., cerreta, m., de toro, p. / analytic hierarchy process (ahp) and geographical information systems (gis): an integrated spatial assessment for planning strategic choices international journal of the vol. 4 issue 1 2012 analytic hierarchy process 6 issn 1936-6744 legitimating choices and making them acceptable to the community itself. in this view, it is important to combine different approaches within a single framework that integrates different evaluation tools (finnveden, nilsonn, johansonn, personn, moberg, and carlsonn, 2003). in particular, innovative tools that could be useful to consider are those that offer the possibility of combining multi-criteria analysis (mca) and multi-group analysis (mga) with geographical information systems (gis), internet technology, spatial decision support systems, and cellular automata models (rotmans, 2000). moreover, integration of different evaluation models with gis (malczewski, 1999) becomes decidedly important in the construction of a spatial decision support system (geneletti, 2004; vizzari, 2011). in keeping with this perspective, the principal aim of this paper is to advance the contemporary debate on environmental assessment and integrated approaches through an empirical investigation of an operative case study in order to generate, at the same time, a rich understanding of the specificity and complexity of the single case and to provide a solid framework for analyzing the potential and problems of combining ahp and gis in a multi-criteria spatial decision support system. the first part of the article analyzes the characteristics of the sdss with particular reference to mcsdss, highlighting the potential of an integrated approach through the integrated spatial assessment (isa) procedure proposed by the authors. the second part discusses the characteristics of the isa approach through its application to the strategic environment assessment (sea) of the city plan of cava de’ tirreni, a town in southern italy, in order to produce maps of the susceptibility of potential locations for the plan’s strategic visions. 2. multi-criteria spatial decision support systems as tool for integrated assessment a spatial decision support system (sdss) is defined as an interactive, computer-based system designed to support a user or a group of users in reaching more effective decisions by solving semi-structured spatial problems (malczewski, 1999; dye and shaw, 2007) in which hard and soft data interact. spatial decision-making problems can involve the evaluation of many decision alternatives based on multiple qualitative and/or quantitative criteria. a large amount and variety of spatial data can be involved. gis provides useful functions to help face and resolve these spatial decision-making problems. spatial decision-making problems are multi-faceted challenges (eldrandaly, 2010). not only do they often involve numerous technical requirements, but they also concern economic, social, environmental and political dimensions with potentially conflicting values and goals. solutions to these problems involve highly complex processes of spatial data analysis and frequently require advanced means for addressing conditions of physical suitability while considering multiple socio-economic variables. standard gis software is not designed to handle different values, variables and relationships and select and use information in different ways. current gis lack mathematical modeling applications, iterative equation solving, and the simulation capabilities necessary for many spatial decision-making situations. in order to consider these needs, sdss can facilitate such decision conditions through an application that allows users to specify their criteria and preferences interactively through an easy-to-use interface allowing the exploration of possible options, along with analytical functions that can generate feasible solutions based on specified criteria and preferences. the development of sdss is explicitly ijahp article: girard, l.f., cerreta, m., de toro, p. / analytic hierarchy process (ahp) and geographical information systems (gis): an integrated spatial assessment for planning strategic choices international journal of the vol. 4 issue 1 2012 analytic hierarchy process 7 issn 1936-6744 designed to support decision-making processes for complex spatial problems and provides a framework for integrating database management systems with analytic models, graphic display capabilities, the expert knowledge of decision makers and community preferences. an sdss can include the integration of a geographic database management system with analytical modeling capabilities, a visualization component, and a user-friendly decision-making interface to help users reach more effective decisions regarding semi-structured spatial decision problems (dye and shaw, 2007). multi-criteria decision making (mcdm) and a wide range of related analytical techniques offer a variety of decision-making procedures that help represent and integrate choices with available mcdm methods in order to solve “real-world” gis-based planning and management problems. in the decision-making process regarding real-world gis-based problems, related spatial and non-spatial data and acceptable techniques require an interactive system incorporating expert knowledge (karnatak, saran, bhatia, and roy, 2007). a variety of territorial-related information (social, economic and environmental) can be easily combined and related to the characteristics of the different options for territorial use, facilitating the construction of appropriate indicators and improving impact forecasting, in the end leading to a priority classification of various options. integration of multi-criteria analysis, multi-group analysis and gis can be exceptionally fruitful in cases characterized by strong conflict in which the roles of local actors and their relationships and goals are considered structuring elements for the process of information construction in a spatial and dynamic evaluation model (joerin and musy, 2000). in particular, the integration of the analytic hierarchy process (ahp) and geographical information systems (gis) could be of particular importance in landuse management (thirumalaivasan, karmegam, and venugopal, 2003; al-shalabi, bin mansor, bin ahmed, and shiriff, 2006; nekhay, arriaza, and guzmán-álvarez, 2009; şener, şener, nas, and karagüzel, 2010). as compared to traditional forms of gis use, it could become possible to evaluate data covering not only current conditions but also (sumathi, natesan, and sarkar, 2008): 1. spatial characteristics of proposed options; 2. implementation of “what if” data analysis investigating potential scenarios and verifying the importance of the different influencing factors; 3. temporal modification of data following option implementation; 4. expressed preferences of local agents; 5. conflict analysis among the various stakeholders; 6. evaluation of various options in order to obtain a preference priority list; 7. visualization of results through graphic representation. spatial analysis combined with ahp has been used in recent years to support evaluation especially in the field of land-use planning. this paper proposes extending this integration to the context of integrated assessment in order to consider not only the technical aspects of a decision-making problem but also the involvement and participation of the local community in planning choices. with respect to traditional gis use, we propose to take into account not only existing conditions data but also the spatial characteristics of proposed options, data that changes over time, the representation of agents’ preferences, conflict analysis, and impact assessment of the different options. therefore, it could be possible to configure a decision support system which includes “social creativity” as a key component in the decision-making process, with the ijahp article: girard, l.f., cerreta, m., de toro, p. / analytic hierarchy process (ahp) and geographical information systems (gis): an integrated spatial assessment for planning strategic choices international journal of the vol. 4 issue 1 2012 analytic hierarchy process 8 issn 1936-6744 “reflexive community” as a necessary interlocutor. in this way, individual and social creativity can be integrated to face complex problems through innovative approaches. in this light, integrated spatial assessment (isa) (cerreta and de toro, 2010) can be useful in decision-making as a tool that includes technical and political evaluations and refers to complicated and complex value systems in conflicting and changing contexts. the integration of problem structuring methods, public participation gis, multi-criteria and multi-group decision support systems and geographic information systems sustains a decision-making process that allows both the analysis of the complexity of human decisions within a flexible environment in which collective knowledge and learning take on a significant role in the processes, as well as the possibility of exploring a spatial development strategy in keeping with sustainable and complex values. indeed, combining ahp with gis overcomes the limitations of certain techniques through the application of different methods deriving from multiple disciplines to define a more complete and integrated framework for analysis and evaluation. this kind of integration gives rise to a “spatial multicriteria and multigroup analysis.” multicriteria spatial decision-making problems typically involve a set of geographically-defined alternatives from which a choice of one or more options is made with respect to a given set of evaluation criteria. multicriteria spatial analysis differs greatly from conventional multicriteria techniques due to the inclusion of an explicit geographic component. it requires information regarding criterion values and the geographic locations of alternatives in addition to decision makers’ preferences for a set of evaluation criteria. this means that the results of the analysis depend not only on the geographic distribution of attributes but also on the value judgments involved in the decision-making process. therefore, two considerations are of fundamental importance for multicriteria spatial analysis: the gis component (i.e., data acquisition, storage, managing and organizing spatial data, changing and updating the information, etc.) and the multicriteria analysis component (i.e., aggregation of spatial data and decision makers’ preferences into discrete decision alternatives). multicriteria spatial decision support systems (mcsdss) are part of a broader field of spatial decision support systems (sdss); in this field, several specific application frameworks for designing mcsdss have been proposed (carver, 1991; eastman, kyem, toledano, and jin, 1993; jankowski, nyerges, smith, moore, and horvath, 1997; (karnatak, saran, bhatia, and roy, 2007) defining a sharable framework that can resolve real-world spatial decision problem most efficiently. the use of spatial data in a gis context helps in the manipulation, examination and presentation of geographic information (vanderhaegen and muro, 2005). gis allows databases to be connected to spatial features using geographic space as the unifying factor, visualizing and analyzing data in an understandable and communicative way. indeed, gis can be used in all stages of the preparation of environmental assessments, especially because of its capacities for spatial data integration. gis stores, integrates, analyses and displays data and can be employed for data preparation, spatial analysis and the presentation of results. in environmental assessments, spatial data is of use in the screening and scoping phases, in project description, in the establishment of the environmental baseline, the definition of impact mitigation and control, public consultation and participation, and monitoring and auditing. when compared to conventional procedures, applying geo-spatial techniques in environmental studies offers other important advantages such as the identification of the spatial and temporal variability of the impacts (patil, annachhatre, and tripathi, 2002; vanderhaegen and muro, 2005). ijahp article: girard, l.f., cerreta, m., de toro, p. / analytic hierarchy process (ahp) and geographical information systems (gis): an integrated spatial assessment for planning strategic choices international journal of the vol. 4 issue 1 2012 analytic hierarchy process 9 issn 1936-6744 3. integrated spatial assessment (isa) for the cava de’ tirreni1 city plan the integrated spatial assessment (isa) approach was applied to the new city plan of the municipality of cava de’ tirreni in the province of salerno in southern italy. throughout the experiment, the aim was to create a methodology that could help identify the interests involved, create broader cohesion regarding environmental protection and preservation of cultural heritage, stimulate the usability of the territory while respecting existing resources, and finally define territorial impacts deriving from plan strategies and actions. in particular, “location susceptibility” maps were created in order to express the propensity of an area to “receive” a given function taking into account potential impacts through multicriteria assessment (ahp) integrated with gis. using the typical approach of strategic environmental assessment (sea) but translating it into a more complex evaluation process defined as isa, we sought to integrate territorial and environmental aspects with plan strategies and choices while recognizing the important role of environmental effects within the decision-making process as well as the selection of alternative options. in this sense, the use of multicriteria assessment plays a privileged role as a decision-making tool (figure 1). figure 1 the integrated spatial assessment approach in cava de’ tirreni city plan. 1 the working group was organized as follows: planning and scientific coordination, carlo gasparrini with cinzia panneri, paola d’onofrio, mirella fiore, vincenzo rizzi, luigi innamorato, alessia sannolo, anna terracciano, pasquale inglese, daniele cannatella; geomorfology, silvana di giuseppe; agronomy, maurizio murolo; landscape, vito cappiello with anna aragosa; economic and financial feasibility, ettore cinque with andrea mazzella; infrastructure and mobility, giulio valfrè with vincenzo cerreta (d’appolonia spa); strategic environmental assessment, maria cerreta, pasquale de toro, saverio parrella. we thank the technical staff of cava de’ tirreni municipality for their support and collaboration. ijahp article: girard, l.f., cerreta, m., de toro, p. / analytic hierarchy process (ahp) and geographical information systems (gis): an integrated spatial assessment for planning strategic choices international journal of the vol. 4 issue 1 2012 analytic hierarchy process 10 issn 1936-6744 in strategic environmental assessment (sea), isa can be considered a “tool” for creating and identifying territorial impacts deriving from plan strategies and actions. therefore, isa can be considered a learning process seeking to create choices and make decisions in flexible, inclusive and participative terms, revealing explicit and hidden conflicts and interests while enhancing local potential. we created a gis that seeks effective integration of different information emerging during the decision-making process. in particular, for the assessment of plan alternatives, the analytic hierarchy process (ahp) (saaty, 1980) multicriteria method was integrated with gis to predict, in spatial terms, the plan’s impact on different environmental characteristics. public meetings, in-depth interviews, and data and information collection were implemented with the main goal of defining a permanent interaction “platform” supporting dialogue and mutual learning among citizens, experts and municipal administrators. public meetings created direct dialogue with citizens and stakeholders and a common ground for discussion among citizens, professionals and the municipality. the main goal was both to obtain deeper knowledge of cava de’ tirreni, with special focus on the most important issues for future urban, social, economic and cultural transformations, as well as to pinpoint collective needs. thus, “common knowledge” (citizens, associations, civil society, etc.) and “expert knowledge” (technicians and administrations) interacted continuously. three main topics were considered during the meetings regarding the development of cava de’ tirreni: what is the shared vision of the future? what strategies should be used? what actions should be undertaken? for the public consultation process, a questionnaire was formulated in which associations and citizens were asked to express their points of view regarding the city’s present and future. then, in order to further broaden participation, the municipality sent a survey to families to reveal citizen needs and project-ideas for improving and developing the city. during the participative phase, five shared “visions” of the future were generated. they were: “cava as a beautiful and identity-bearing city”, “cava as a regenerated and friendly city”, “cava as a modern and productive city”, “cava as a territorial hub”, “cava as an ecological city”. the visions reflect the community’s perception of complex social values and express its important resources on a number of different levels. the visions were analyzed using the strategic options development and analysis (soda) approach (rosenhead and mingers, 2001), a decision-support system that allows complex problems with non-structured qualitative data to be faced starting from “cognitive maps.” using the decision explorer 3.1.0 software, cognitive maps were created from verbal protocols whose contents were structured according to a formal and methodological approach. the cognitive maps represented the structure of the meeting discussions and were able to maintain a rich amount of data and manage the complexity of the information. consistent with the hierarchical structure of the decision-making process, the visions were broken down into general goals, strategic axes and strategic actions. in detail, the strategic actions were linked to three “core projects” that became the main references for direct implementation in the operative phase. the core projects are the synthesis of the issues that emerged during the participative and consultative process and identify the key transformation and preservation projects within the infrastructural, spatial, functional and symbolic relations systems. ijahp article: girard, l.f., cerreta, m., de toro, p. / analytic hierarchy process (ahp) and geographical information systems (gis): an integrated spatial assessment for planning strategic choices international journal of the vol. 4 issue 1 2012 analytic hierarchy process 11 issn 1936-6744 in order to determine possible locations for the different planning choices, the analytic hierarchy process (ahp) integrated with the gis tools (chen, blong, and jacobson, 2001; malczewski, 2004) allowed decision-making to go beyond the simple overlay of different themes through pairwise comparisons of the criterion established for each hierarchical level. for each of the five visions, a “location susceptibility” map was generated expressing the territory’s propensity to receive a given strategic action considering its potential impacts; the lower the territorial and environmental impacts caused by an action, the greater the susceptibility of the territory to receive it. by considering the typical sea approach and translating it into a more complex evaluation process defined as isa, we sought to integrate social, territorial and environmental aspects in the development of plan strategies and choices while simultaneously recognizing the important role of stakeholder perceptions and environmental effects within a collective decision-making process seeking to identify alternative opportunities. 4. location susceptibility maps for strategic visions the analytic hierarchy process (ahp) organizes the decision-making process hierarchically. the basic ahp process involves the perception, breakdown and synthesis of a decision problem in order to provide a methodology for modeling unstructured problems in the economic, social, and management sciences. the definition of a hierarchy is an abstraction of a system’s structure used in order to study the functional interactions of its components and their impacts on the entire system. this abstraction can take several related forms, all of which are essentially derived from overall goals moving through sub-goals, to the forces that affect these sub–goals, to the people who influence these forces, to the people’s goals and policies, strategies, and finally to the outcomes resulting from these strategies (saaty, 1980). from a procedural point of view, this approach consists of three main phases: 1. construction of a suitable hierarchy; 2. establishing priorities among the elements of the hierarchy by means of pairwise comparisons; 3. verification of the logical consistency of the pairwise comparisons (saaty and peniwati, 2007; saaty and vargas, 2000). in the present case-study, each vision produced during the participative phase was organized according to a tri-level hierarchical structure: 1. environmental themes; 2. criteria; 3. values/characteristics. spatial indicators referring to the nature of the areas linked to a value judgment were associated with the values/characteristics in the third hierarchical level, expressed on a five point scale:1. high location susceptibility ; 2. medium–high location susceptibility; 3. medium location susceptibility; 4. medium–low location susceptibility; 5. low location susceptibility (figure 2). to conduct “spatial assessment,” an extension of the ahp method within an arcgis (marinoni, 2004) environment was used to obtain “location susceptibility” maps. this program was written in visual basic using arcobjects, the arcgis development platform. the program was not developed as a separate executable file but runs as a vba macro in the arcgis environment. the vba module which implements ahp is a useful tool for facilitating land use assessment. indeed, the module accesses an external dynamic ijahp article: girard, l.f., cerreta, m., de toro, p. / analytic hierarchy process (ahp) and geographical information systems (gis): an integrated spatial assessment for planning strategic choices international journal of the vol. 4 issue 1 2012 analytic hierarchy process 12 issn 1936-6744 link library (eigenutl.dll) which performs the eigenvalue and eigenvector calculations of the preference matrix and returns the criteria weights. together with this dynamic link library, the vba macro fills an important gap in arcgis functionality; this made it possible to obtain a pairwise comparison of the criteria referring to each hierarchical level. in order to apply the ahp method to each location susceptibility class, a numerical value (score) and a chromatic scale were associated with the five judgments. to produce the graphic representation of the results, the color given to every pixel is related to every score according to the conventional range from dark green to orange. we selected four main “environmental themes” and identified some important criteria relating to territorial analysis for each. the same weight for all the visions was assigned to each criterion, while for each environmental theme pairwise comparisons were made creating five matrices for each vision (figure 3). regarding the environmental themes on the first level, according to the judgements from the working group experts and the values/characteristics, we obtained pairwise comparisons for the following themes: biosphere, geosphere, landscape and soil. the priority vector for each vision expressing the weight of the environmental theme is illustrated in figure 2. the consistency ratio for each vision ranges from 0,0000 to 0,057; less than 0.10 is considered acceptable. through the ahp method application, it is possible to combine the weights of criteria obtained through pairwise comparisons with scores associated with the different location susceptibility classes, obtaining the related “susceptibility maps” in synergy with the gis environment. it is possible to obtain an overall value for every pixel as a linear combination of the weights of the criteria from the score related to location susceptibility taking into account specific values/characteristics (figure 4). for each vision we obtained the following location susceptibility maps (figures 4-5-6-7-8): 1. classification map of values/characteristics for the biosphere environmental theme (taking as criteria: territorial biopotential index, degree of biodiversity, infrastructure fragmentation index); 2. biosphere location susceptibility map; 3. classification map of values/characteristics for the geosphere environmental theme (taking as criteria: slopes stability, seismic zoning); 4. geosphere location susceptibility map; 5. classification map of values/characteristics for the landscape environmental theme (taking as criteria: landscape units); 6. landscape location susceptibility map; 7. classification map of values/characteristics for the soil environmental theme (taking as criteria: land use, cultivations productivity); 8. soil location susceptibility map; 9. composite location susceptibility map. taking into account the environmental themes and hierarchy criteria and putting the data together from all criteria in the first hierarchical level, we obtained the maps in figures 5 to 9, in which the colours ranging from dark green to orange express the location susceptibility (from high to low) of the strategic actions called for in each vision. ijahp article: girard, l.f., cerreta, m., de toro, p. / analytic hierarchy process (ahp) and geographical information systems (gis): an integrated spatial assessment for planning strategic choices international journal of the vol. 4 issue 1 2012 analytic hierarchy process 13 issn 1936-6744 the same process was applied to all the visions obtaining a location susceptibility map for each. this kind of assessment can truly support planning activity optimizing the propensity of each area and, most of all, locating strategic actions in places in which territorial and environmental impacts can be minimized. the operation’s underlying logic was the “sustainable spatial planning” of territory. however, it must be underlined that a city plan is not an automatic output of the susceptibility maps. the planner takes into account susceptibility maps and designs the plan in conformity with them but, of course, it is possible to find many solutions that are congruent with the susceptibility maps for each function. in this way, it was possible to move from visions to three technical “coreprojects” to guide the city’s transformation (figure 10). 5. discussion the three “core-projects,” representing the conclusion of the mcsdss can synthesize the complexity of the decision-making process in which different forms of knowledge are compared. the collective process of identifying and creating visions and related actions for transforming cava de’ tirreni made it possible to combine the contribution of common knowledge – that could develop visions reflecting the points of view of citizens and different stakeholders (representative of the main age classes that make up the local population, of families, of the main categories of economic and productive activities, of local associations involved in different fields) with the contribution of expert knowledge (experts in the fields of urban planning, geomorfology, agronomy, landscape, economic and financial feasibility, infrastructures and mobility) to identify significant environmental issues and structure an integrated decision-making platform. therefore, while visions and their possible actions were constructed with the involvement of common knowledge, the evaluation of an area’s susceptibility to transformation was carried out with the support of experts thus allowing the selection of criteria and indicators to explain the context’s fundamental characteristics. it was therefore possible to obtain the relative location susceptibility map for each vision; it is clear that evaluation supported the planning phases optimizing the characteristics of each area and, most importantly helped located activities where it is preemptively possible to minimize territorial and environmental impacts, creating an overall strategic planning framework. the plan’s visions and actions designed through the constant interaction of different forms of knowledge are an integral part of a transparent and shared spatial decision-making process that can reduce social, environmental and economic conflicts even during the early decision construction phase. ijahp article: girard, l.f., cerreta, m., de toro, p. / analytic hierarchy process (ahp) and geographical information systems (gis): an integrated spatial assessment for planning strategic choices international journal of the vol. 4 issue 1 2012 analytic hierarchy process 14 issn 1936-6744 figure 2 example of spatial indicators for one vision. ijahp article: girard, l.f., cerreta, m., de toro, p. / analytic hierarchy process (ahp) and geographical information systems (gis): an integrated spatial assessment for planning strategic choices international journal of the vol. 4 issue 1 2012 analytic hierarchy process 15 issn 1936-6744 figure 3 for each vision: pairwise comparison matrix, priorities vectors and consistency ratio. ijahp article: girard, l.f., cerreta, m., de toro, p. / analytic hierarchy process (ahp) and geographical information systems (gis): an integrated spatial assessment for planning strategic choices international journal of the vol. 4 issue 1 2012 analytic hierarchy process 16 issn 1936-6744 figure 4 synergy between ahp and gis. ijahp article: girard, l.f., cerreta, m., de toro, p. / analytic hierarchy process (ahp) and geographical information systems (gis): an integrated spatial assessment for planning strategic choices international journal of the vol. 4 issue 1 2012 analytic hierarchy process 17 issn 1936-6744 figure 5 location susceptibility for vision 1. ijahp article: girard, l.f., cerreta, m., de toro, p. / analytic hierarchy process (ahp) and geographical information systems (gis): an integrated spatial assessment for planning strategic choices international journal of the vol. 4 issue 1 2012 analytic hierarchy process 18 issn 1936-6744 figure 6 location susceptibility for vision 2. ijahp article: girard, l.f., cerreta, m., de toro, p. / analytic hierarchy process (ahp) and geographical information systems (gis): an integrated spatial assessment for planning strategic choices international journal of the vol. 4 issue 1 2012 analytic hierarchy process 19 issn 1936-6744 figure 7 location susceptibility for vision 3. ijahp article: girard, l.f., cerreta, m., de toro, p. / analytic hierarchy process (ahp) and geographical information systems (gis): an integrated spatial assessment for planning strategic choices international journal of the vol. 4 issue 1 2012 analytic hierarchy process 20 issn 1936-6744 figure 8 location susceptibility for vision 4. ijahp article: girard, l.f., cerreta, m., de toro, p. / analytic hierarchy process (ahp) and geographical information systems (gis): an integrated spatial assessment for planning strategic choices international journal of the vol. 4 issue 1 2012 analytic hierarchy process 21 issn 1936-6744 figure 9 location susceptibility for vision 5. ijahp article: girard, l.f., cerreta, m., de toro, p. / analytic hierarchy process (ahp) and geographical information systems (gis): an integrated spatial assessment for planning strategic choices international journal of the vol. 4 issue 1 2012 analytic hierarchy process 22 issn 1936-6744 figure 10 from visions to core-projects. ijahp article: girard, l.f., cerreta, m., de toro, p. / analytic hierarchy process (ahp) and geographical information systems (gis): an integrated spatial assessment for planning strategic choices international journal of the vol. 4 issue 1 2012 analytic hierarchy process 23 issn 1936-6744 6. conclusions the decision-making process carried out in the cava de’ tirreni city plan points out how gis combined with ahp can make full use of gis functions such as spatial analysis, data processing and inquiry where complex data and their mutual influences can be included to describe the position of each factor in space. ahp renders the analysis more flexible. characterized by relative ease in handling multiple criteria, it is simple to understand and effective in managing qualitative and quantitative data. at the same time, gis helps develop a satisfactory man/machine interactive interface. in the cava de’ tirreni city plan, combining ahp with the gis method, the isa approach shows that: 1. it is possible to improve synthetic evaluation by developing the gis capability of spatial analysis and ahp capability of multilayer analysis. the evaluation results and the distribution pattern obtained for each vision represent effective ways to communicate a territory’s complex characteristics; 2. it is useful to create a large and flexible multi-component system in which there are continuous and dynamic exchanges of information among various subsystems/environmental themes, selecting the correct factors for establishing comprehensive spatial indexes and/or indicators; 3. gis can be used in preparing spatial statistics and clustering processes to reveal the most suitable areas for site selection, managing and analyzing large volumes of spatial data from a variety of sources. furthermore, it can handle and simulate the necessary economic, environmental, social, technical, and political constraints; 4. ahp is a powerful tool for solving complex problems with interactions and correlations among multiple objectives; 5. it allows decision-makers to understand the current status of the integrated characteristics of a local context more clearly and to help administrators understand the interactions among visions and core-projects in order to identify and implement shared actions. therefore, the integration of gis and ahp methods provides a mechanism with which complex issues can be thoroughly explored and immediate feedback for decision-makers can be provided. the isa approach used in the case study could be considered the basis for increasing the level of integration of local and expert knowledge in a more extensive participatory gis process geared towards the involvement of different kinds of knowledge in order to improve the completeness of hard and soft data and the consistency of the overall evaluation. recognizing the important role of environmental impacts within a public decision-making process along with the selection of alternative options, we used a typical sea approach, translating it into a more complex evaluation process defined as isa in order to integrate territorial and environmental aspects with the definition of plan strategies and choices. in this kind of decision-making process, local communities can contribute actively to the implementation and updating of gis data thus improving the evaluation of alternative strategies and actions. therefore, in addition to supplying experts with valuable information for increased territorial understanding, they are also made much more aware of the characteristics and values of their own contexts. an integrated evaluation approach can move beyond spatial and hierarchical limits, considering the various components (historical, cultural, environmental, economical, ijahp article: girard, l.f., cerreta, m., de toro, p. / analytic hierarchy process (ahp) and geographical information systems (gis): an integrated spatial assessment for planning strategic choices international journal of the vol. 4 issue 1 2012 analytic hierarchy process 24 issn 1936-6744 social, anthropological, etc.), clarifying weights, recognizing priorities, and defining the correct strategies while taking into consideration social participation, interdisciplinarity and integration. in this perspective, the use of multi-criteria analysis plays a privileged role as a decision-making tool. indeed, through the hierarchical construction of decision goals, it was easy to involve the local community and different experts in obtaining shared visions, strategies and actions. this contributed to the creation of a richer and more complex knowledge framework and to the bottom-up construction of planning ideas. indeed, the different maps obtained through gis use were the expression of multidimensional interaction regarding the meaning and role of the different evaluation criteria together contributing to plan design. they helped improve the technical effectiveness and, at the same time, the transparency of the evaluation process leading to decisions that reflect different needs and expectations. through such evaluation processes, it is possible to help communities and experts become more aware not only of their own views and preferences but also of those of others, in order to define participated and shared solutions. in this sense, assessment has become a fundamental part of planning, and isa can be seen as the preventative verification of environmental and territorial sustainability and, at the same time, a tool for stimulating the identification of alternative solutions within a spatial decision-making process. references al-shalabi, m.a., bin mansor, s., bin ahmed, n., & shiriff, r. (2006), gis based multicriteria approaches to housing site suitability assessment, paper presented to the xxiii fig congress, shaping the change, munich, germany, october 8-13. carver, s.j. (1991), integrating multi-criteria evaluation with geographical information systems, international journal of geographical information systems, 5, 321–339. cerreta, m., & de toro, p. (2010), integrated spatial assessment for a creative decisionmaking process: a combined methodological approach to strategic environmental assessment, international journal of sustainable development, 13(1/2), 17–30. chen, k., blong, r., & jacobson, c. (2001), mce-risk: integrating multicriteria evaluation and gis for risk decision-making in natural hazards, environmental modelling and software, 16, 387–397. dye, a.s., & shaw, s.-l. (2007), a gis-based spatial decision support system for tourists of great smoky mountains national park, journal of retailing and consumer services, 14, 269–278. eggenberger, m., & partidário m. (2000), development of a framework to assist the integration of environmental social and economic issues in spatial planning, impact assessment project appraisal, 18(1), 201–207. eldrandaly, k. (2010), a gep-based spatial decision support system for multisite land use allocation, applied soft computing, 10, 694–702. ijahp article: girard, l.f., cerreta, m., de toro, p. / analytic hierarchy process (ahp) and geographical information systems (gis): an integrated spatial assessment for planning strategic choices international journal of the vol. 4 issue 1 2012 analytic hierarchy process 25 issn 1936-6744 eastman, j.r., kyem, p.a.k, toledano, j. & jin, w. (1993), gis and decision making, united nations institute for training and research (unitar), geneva, switzerland, 127. finnveden, g., nilsonn, m., johansonn, j., personn, å., moberg, å., & carlsonn, t. (2003), strategic environmental assessment methodologies – application within the energy sector, environmental impact assessment review, 23, 91-123. fusco girard, l. (1987), risorse architettoniche e culturali: valutazioni e strategie di conservazione, milano, angeli. fusco girard, l., & nijkamp, p. (1997), le valutazioni per lo sviluppo sostenibile della città e del territorio, milano, angeli. geneletti, d. (2004), a gis-based decision support system to identify nature conservation priorities in a alpine valley, land use policy, 21, 149–60. jankowski, p., nyerges, t.l., smith, a., moore, t.j., & horvath, e. (1997),spatial group choice: a sdss tool for collaborative spatial decision making, international journal of geographical information systems, 11(6), 566–602. joerin, f., & musy, a. (2000), land management with gis and multicriteria analysis, international transactions in operational research, 7, 67–78. karnatak, h.c., saran, s., bhatia, k., & roy, p.s. (2007), multicriteria spatial decision analysis in web gis environment, geoinformatica, 11, 407–429. lee, n. (2006), bridging the gap between theory and practice in integrated assessment, environmental impact assessment review, 26, 57–78. malczewski, j. (1999), gis and multicriteria decision analysis, new york, wiley. malczewski, j. (2004), gis-based land-use suitability analysis: a critical overview, progress in planning, 62, 3–65. marinoni, o. (2004), implementation of analytic hierarchy process with vba in arcgis, computational geosciences, 30, 637–646. nekhay, o., arriaza, m., & guzmán-álvarez, j.r. (2009), spatial analysis of the suitability of olive plantations for wildlife habitat restoration, computers and electronics in agriculture, 65, 49–64. munda, g. (2008), social multi-criteria evaluation for a sustainable economy, new york, springer. patil. a.a, annachhatre, a.p., & tripathi, n.k. (2002), comparison of conventional and geo-spatial eia: a shrimp farming case study, environmental impact assessment review, 22, 361–75. ijahp article: girard, l.f., cerreta, m., de toro, p. / analytic hierarchy process (ahp) and geographical information systems (gis): an integrated spatial assessment for planning strategic choices international journal of the vol. 4 issue 1 2012 analytic hierarchy process 26 issn 1936-6744 pope, j., annandale, d., & morrison-saunders, a. (2004), conceptualising sustainability assessment, environmental impact assessment review, 24, 595–616. rosenhead, j., & mingers, j. (2001), rational analysis for a problematic world revisited: problem structuring methods for complexity, uncertainty and conflict, chichester, j. wiley & sons. rotmans, j. (2000), an integrated planning tool for sustainable cities, environmental impact assessment review, 20, 265–76. saaty, t.l. (1980), the analytical hierarchy process, new york, ny, mcgraw hill. saaty, t.l., & peniwati, k. (2007), group decision making: drawing out and reconciling differences, pittsburgh, pa: rws publications. saaty, t.l., & vargas, l.g. (2000), models, methods, concepts and applications of the analytic hierarchy process, dordrecht, kluwer academic publisher. şener, ş., şener, e., nas, b., & karagüzel, r. (2010), combining ahp with gis for landfill site selection: a case study in the lake beys_ehircatchment area (konya, turkey), waste management, 30, 2037–2046. sinclair, a.j., sims, l., & spaling, h. (2009), community-based approaches to strategic environmental assessment: lessons from costa rica”, environmental impact assessment review, 29, 147–156. sumathi, v.r., natesan, u., & sarkar, c., (2008), gis-based approach for optimized siting of municipal solid waste landfill, waste management, 28, 2146–2160. therivel, r. (2010, 2nd edn), strategic environmental assessment in action, london, earthscan. thirumalaivasan, d., karmegam, m., & venugopal, k. (2003), ahp-drastic: software for specific aquifer vulnerability assessment using drastic model and gis, environmental modelling & software, 18, 645–656. vanderhaegen, m., & muro, e. (2005), contribution of a european spatial data infrastructure to the effectiveness of eia and sea studies, environmental impact assessment review, 25, 123–142. vizzari, m. (2011), spatial modelling of potential landscape quality, applied geography, 31, 108–118. wiek, a., & walter, a.i. (2009), a transdisciplinary approach for formalized integrated planning and decision-making in complex systems, european journal of operational research, 197, 360–370. ijahp article: garuti/new advances of the compatibility index “g” in weighted environments international journal of the analytic hierarchy process 514 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.438 new advances of the compatibility index“g”in weighted environments claudio e. garuti fulcrum ingenieria ltda. santiago, chile claudiogaruti@fulcrum.cl abstract this article addresses the problem of measuring closeness in weighted environments (decision-making environments). this article is relevant because of the importance of having a dependable cardinal measure of distance in weighted environments. a weighted environment is a non-isotropic structure where the different directions (axes) may have different importance (weight) hence, privileged directions exist. in this kind of a structure, it would be very important to have a cardinal reliable index that is able to show how close or compatible the set of measures of one individual is with respect to the group or to any other, or how close one pattern of behavior is to another. a few common examples of the application of this are the interaction between actors in a decision making process (system values interaction), matching profiles, pattern recognition, and any situation where a process of measurement with qualitative variables is involved. keywords: weighted environments; measurement; compatibility index g; order topology 1. introduction this article addresses the problem of measuring closeness in weighted environments (decision-making environments) using the concept of compatibility of priority vectors and value systems. when using the concept of closeness the question immediately comes to mind, what does it mean to be close (when does close really mean close)? thus, when measuring closeness or proximity we should have a point of comparison (a threshold) that makes it possible to compare or make a decision if our positions, system values or priorities are really close. for our purposes, compatibility is defined as the proximity or closeness between vectors within a weighted space (garuti, 2014). we will propose a compatibility index able to measure closeness in a weighted environment. thus, we will be able to assess pattern recognition, medical diagnosis support measuring the degree of closeness between disease-diagnosis profiles, buyer-seller matching profiles, measuring the degree of closeness between homebuyer and seller projects, or degree of matching for employment, measuring the degree of closeness between a person’s profile with the desired position profile, conflict resolution in curricula network design, measuring closeness of two different value systems (the ways of thinking) by identifying and measuring the discrepancies, and in general measuring the degree of compatibility between any priority vectors in cardinal measure bases (order topology) (garuti, 2016, 2014, 2012). mailto:claudiogaruti@fulcrum.cl ijahp article: garuti/new advances of the compatibility index “g” in weighted environments international journal of the analytic hierarchy process 515 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.438 first, the paper presents some theory behind distance (measurement) and closeness concepts in different cases and discusses a nice point of view from statistical and set theory for measuring distance and similarity. then, the concept of scales is presented, including compatibility, compatibility index g and some analogies between g and distance concept. then, a comparison with other compatibility indices present in the literature is shown, highlighting the advantages of gin relation to the others (especially within weighted environments). next, a necessary threshold that allows "when close really means close" in weighted environments is presented. finally, three relatively simple examples are developed, each one presenting a different application for the compatibility index g. one example asks if the order of choice is necessary to say if two rankings are compatible or not, the second is for quality testing (testing saaty’s consistency index through compatibility index g), and the third is for measuring comparability between two different rules of measurement (two different points of view). 2. literature review in metric topology the particular function of distance d(a,b) is used to assess the closeness of two points a, b as a real positive function that keeps 3 basic properties (garuti, 2012): 1.d(a,b) > 0 and d(a,b) = 0 if a=b (definition of zero distance) 2.d(a,b) = d(b,a) (symmetry) 3.d(a,b) + d(b,c) ≥ d(a,c) (triangular inequality) the general function of distance used to calculate the separation between two points is: d(a,b)= lim (i(ai-bi) n ) 1/n (i=1,...,n; n= dimensionofthespace). n k when applying different values of k, different norms of distance appear: for k = 1, then: d(a,b)= iabs(ai-bi). norm1, absolute norm or path norm; this norm measures the distance from a to b within a 1d line, “walking” over the path, in one line-dimension. for k = 2, then: d(a,b)= [(i(ai-bi) 2 )] 1/2 . norm2 or euclidean norm, this norm measures the distance from a to b, within a 2d plane (x-y plane) getting the shortest path (the straight line). for k = +∞, then: d(a,b)= maxi (abs(ai-bi)). norm∞ or norm max; this norm measures the distance from a to b within a ∞d hyperplane, getting the shortest path (the maximum coordinate) from all the possible paths. in the field of statistics, an interesting case of distance calculation is known as distance of mahalanobis which meets the metric properties shown above (mahalanobis, 1936). this distance takes into consideration the parameters of statistics like deviation and covariance which can be assimilated to concepts of weight and dependence in the ahp/anp world. ijahp article: garuti/new advances of the compatibility index “g” in weighted environments international journal of the analytic hierarchy process 516 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.438 its formal presentation is: dm(x,y) =√(x − y) ∑ (x − y)−1 with  the matrix of covariance between x and y. but, for a more simple case (without dependence), this formula can be written as: d2(x1,x2) =√( (𝑥11−𝑥12) 𝜎1 ) 2 + ((𝑥21−𝑥22) 𝜎2 ) 2 or de(x1, x2) =√(𝑋1 − 𝑋2) 𝑇 𝑆 −1 (𝑋1 − 𝑋2), with𝑆 −1the diagonal matrix with the standard deviation of variables x, y. it is interesting to see that the importance of the variable (to calculate distance) is dependent on the deviation value (the bigger the deviation the smaller the importance).this shows that the importance of the variable is not dependent on the variable itself but just on the level of certainty of the variable. however, is this statement always true? in the field of botany, there exists another beautiful formula to measure the concept of similarity among species, this time coming from the set theory domain. this is the “jaccard index of similarity” developed by paul jaccard (1868-1944)and states in a very simple way that the similarity of two sets of objects is given by its ratio of intersection and union, that is: j = ab /(ab), which can be written as: (min(a,b)) /(max(a,b)) considering that the minimum quantity of elements present simultaneously in two sets is given by its intersection and the maximum by its union (jaccard, 1901).an approximate vector expression of the jaccard index (using the dot product expression) can be written as: j = (ab) / (aa + bb – ab) considering that the dot product represents the intersection of two sets (vectors) a and b. if a and b are parallel vectors, then there is a total intersection, and when they are perpendicular vectors there is null intersection. the subtraction in the denominator is to avoid the double counting of elements. thus, the intersection is a way to measure the degree of projection that two vectors may have. however, what happens when the elements in the set have different importance or weight? this could be a relevant issue as will be shown (working within weighted environments). we discuss these approaches since factors such as weight and dependence are at the base of the structure of the ahp and anp (saaty, 2001, 2010). however, instead of having to understand and deal with probabilities and statistics (which by the way are not easy to build and later interpret),the idea here is to apply the natural way of thinking which is based more on priorities than in probabilities. indeed, we can manage the same information in a more comprehensive, complete and easy to explain form by combining ahp/anp with the compatibility index g and working with priorities. this avoids the need to create a big database or have to understand and interpret complex statistical functions. it is important to remember that priorities can include probabilities, but not vice-versa. therefore, the mcdm approach using the ahp/anp method provides a very nice tool for our investigation and treatment of the knowledge and experience that experts possess ijahp article: garuti/new advances of the compatibility index “g” in weighted environments international journal of the analytic hierarchy process 517 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.438 in their different fields, and at the same time stays within the decision making domain (order topology domain) which avoids the need for a huge and costly database where knowledge about the individual behavior can be lost or misinterpreted. 3. hypotheses/objectives in order topology, measurement deals with dominance between preferences (intensity of preference). for instance: d(a,b)=3, means that dominance or intensity of preference of “a” over “b” is equal to 3, or that, a is 3 times more preferred than b. when talking about preferences a relative absolute ratio scale is applied. the term relative is used because priority is a number created as a proportion of a total (percent or relative to the total) and has no need for an origin or predefined zero in the scale. the term absolute is used because it has no dimension since it is a relationship between two numbers of the same scale leaving the final number with no unit. the term ratio is used because it is built in a proportional type of scale (6kg/3kg=2) (garuti, 2012). so, making a general analogy between the two topologies, one might say that: “metric topology is to distance as order topology is to intensity” (garuti, 2012; 2014) an equivalent concept of distance is presented in order to make a parallel between the three properties of distance of metric topology (garuti, 2014; 2012). this is applied in the order topology domain, considering a compatibility function (equation 1) similar to distance function, but over vectors instead of real numbers. consideration: a, b, c are priority vectors of positive coordinates and iai= ibi= ici=1. g(a,b) is the compatibility function expressed as: g(a,b) = ½∑ ((𝒂𝒊 + 𝒃𝒊) 𝑴𝒊𝒏(𝒂𝒊,𝒃𝒊) 𝑴𝒂𝒙(𝒂𝒊,𝒃𝒊) ) (1a) when working in distributive or relative mode, or the priority vectors a, b comes from an unknown model. when the model is known (its means the vector w with the criteria weights is known) and we are working in the ideal or absolute mode of measurement (the rating mode), using rating scales instead of comparing the alternatives, then g takes the form: g(a,b) = ∑ ((𝒘𝒊) 𝑴𝒊𝒏(𝒂𝒊,𝒃𝒊) 𝑴𝒂𝒙(𝒂𝒊,𝒃𝒊) ) (1b) with wi the global weight of criterion i. and (ai,bi) the local values of the profiles a and b. g function presents the following properties: (garuti, 2014, 2012) 1. 0 ≤ g(a,b) ≤ 1 (non negative real number) ijahp article: garuti/new advances of the compatibility index “g” in weighted environments international journal of the analytic hierarchy process 518 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.438 the compatibility function g, returns a non-negative real number that falls in the 0 1 range. with g(a,b)=0, if a and b are perpendicular vectors (a┴b), and represent the definition of total incompatibility between priority vectors a, b. (aºb=0). also, g(a,b)=1, if a and b are parallel vectors, (a=b for normalized vectors), and represent the definition of total compatibility between priority vectors a and b. (aºb=1) 2. g(a,b) = g(b,a)(symmetry) symmetry condition, the compatibility measured from a to b is equal to the compatibility measured from b to a. this is easy to prove, just by interchanging a for b and b for a in the compatibility function g. 3. g(a,b) + g(b,c) ≥ g(a,c) (triangular inequality) 4. if acb and bcc =><=acc (non transitivity of compatibility) if a is compatible with b and b compatible with c, this does not imply that a is necessarily compatible with c. for property 3, it is easy to prove that if a, b and c are compatible priority vectors (i.e. 0.9 ≤ gi ≤ 1.0 for a, b, c), then property 3 is always satisfied. but, this property is also satisfied for the more relaxed (and interesting) condition where only two of the three vectors are compatible. for instance, if a is compatible with b (g(a,b)≥0.9) and a is compatible with c (g(a,c)≥0.9), or some other combination of a, b and c, then condition 3 is also satisfied. this more relaxed condition allows compatible and non-compatible vectors to be combined while property 3 is still satisfied. this situation can be geometrically viewed in figure 1. figure1. maximum circle of compatibility for position a, related to b and c (garuti, 2012) figure 1 shows the compatibility neighborhood for a, in relation with b and c, with its minimum compatibility value of 0.9 represented by the radius of the circle (in the center the compatibility reaches its maximum value of 1.0). thus, g(a,b)=g(a,c)=0.9 represents the minimum compatibility point, or the maximum distance for positions b and c to still be compatible with position a. of course, g(b,c)<0.9 represents a noncompatible position for points b and c. notice that with property 3,g(a,b) + g(b,c) ≥ g(a,c) is still valid, indeed any combination that one can make will keep the inequality satisfied since if c gets closer to a (increasing the right side of the equation), then g(b,c) will also grow. the extreme case is when c is over a, (g(a,c)=1.0) then g(b,a)+g(b,c)=0.9+0.9=1.8>1.0, keeps the inequality satisfied (garuti, 2012). we may also define the incompatibility function as the arithmetic complement of the compatibility where incompatibility=1–compatibility. thus, incompatibility is equivalent to (1–g). the incompatibility concept is closer to the idea of distance since the greater the distance the greater the incompatibility (saaty, 2010; garuti, 2014, 2012). a b 0.9 c 0.9 ijahp article: garuti/new advances of the compatibility index “g” in weighted environments international journal of the analytic hierarchy process 519 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.438 4. generating a threshold for compatibility index g to answer the question, “when does close really mean close?”, it is necessary to first have a reliable index of compatibility. however, that is not sufficient as it is also necessary to have a second condition which is a limit or threshold for the index. it is necessary to have a limiting lower value (minimum threshold) to indicate when two priority vectors are compatible or close to being compatible in order to precisely define when close really means close. we have four different ways to define a minimum threshold for compatibility (garuti, 2012): 1) considering that compatibility ranges between 0-100% (0 1) with 100% being a case of total compatibility (represented by parallel vectors), it is reasonable to define a value of 10% of tolerance (1/10th of 100%) as a maximum threshold of incompatibility to consider two vectors as compatible vectors (which means a minimum of 90% of compatibility to consider two vectors compatible).this explanation is based on the idea of one order of magnitude for an admissible perturbation for measurement. this lower bound is also based on the accepted 10% used in ahp for the consistency index. in the comparison matrix of the ahp, the 10% limit of tolerance inconsistency comes from the consistency ratio (cr) obtained from the consistency index compared to a random index (cr=ci/ri), that in general it has to be less than10%. this says that the farther the ci is from the ri (random index response) the better the cr is. it’s interesting to recall that cr is built as a comparison from the statistical analysis of ri (this idea will be reviewed in the last case analysis). 2) the compatibility index is related to a topological analysis since compatibility is related to the measure of closeness in weighted environments (weighted spaces). in figure 2 below a sequence of 2, 3, 4 and 5 dimension vectors is presented. the first or initial vector is obtained as an isotropic space situation, that is, with equal values (1/n) in each coordinate (no privileged direction in the space); the second one is a vector obtained by perturbing (adding or subtracting) 10% on each coordinate, creating “small crisps” or little privileged directions, then the incompatibility index is calculated with the 5 different formulas(all formulae can be used because it is a near flat space, no singularities, where every formula works relatively well). ijahp article: garuti/new advances of the compatibility index “g” in weighted environments international journal of the analytic hierarchy process 520 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.438 case sensitization for 2-3-4 and 5d homogeneous vectors (perturbing flat space to near flat space) dim coordinates perturbing 10% the initial vector of coordinates and normalizing (saaty’s index) (%) g (%) hilber (%) ivp (%) norm 1 (%) 2d 0,500 0,500 initial 1,010 9,523 8,715 1,01 5,00 0,450 0,550 pertur 3d 0,333 0,333 0,333 initial 0,898 8,192 8,715 0,89 4,30 0,354 0,290 0,354 pertur 4d 0,250 0,250 0,250 0,250 initial 0,910 9,523 8,715 1,01 5,00 0,275 0,225 0,275 0,225 pertur 5d 0,200 0,200 0,200 0,200 0,200 0,969 8,964 8,715 0,97 4,70 0,215 0,176 0,215 0,176 0,215 figure 2. defining a possible threshold of 10% for g function when looking at the outputs for incompatibilities, it is possible to observe a good response for everyone (equal or less than 10%), with g and norm1 circa 10% and 5% as the upper bound in every case. 3) in the figure 3 below, a simple test was run over an excel spreadsheet using the common area example of ahp where the result (the importance of the area of the figures) can be calculated precisely with the typical geometric formulas and then its values normalized to obtain the exact priorities as a function of the size of their areas. it is possible to have a reference point of the element values (the right coordinates for the actual area vector) by doing it this way. the next step is perturbing the actual area values by +/10% producing a new vector of areas. finally, the g function is applied over these two vectors (actual and perturbed) to measure their compatibility, obtaining a value of 91.92% (or 8.08% of incompatibility).this result is very close to the standard error deviation calculated as:  abs(perturbed-actual)/actual=10%. this is showing that 90% might represent a good threshold, considering that the difference between both outputs is related with the significant fact that these numbers are not just numbers but weights. ijahp article: garuti/new advances of the compatibility index “g” in weighted environments international journal of the analytic hierarchy process 521 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.438 figure 3. possible threshold of 10% for g function 4) the last way to analyze the correctness of calculating 90% for a threshold involves working with a random function and filling the area vector with random values and calculating g for every case. the goal is to generate an average g for the case of full random values for the areas (full random means without any previous order among the areas, like figure a is clearly bigger than figure b, and so on), and again producing random values but this time keeping the correct order among the figures (imitating the behavior of a rational dm), then once again generating an average g for this case, and comparing both results against actual values. the average value of g for 15 tries in the first case (keeping no order) was around 50% compatibility and 78% for the second case (keeping the order among the 5 figures).both results show that a limit of 90% might be a good threshold. in the first case, the ratio between threshold and the full random g is almost double 1.8 (0.90 over .50), keeping the 0.90 compatibility threshold far from random responses. in the second case (threshold over sorted figures), the ratio is much closer (as expected) with a value of 1.16 (0.90 over 0.78). this indicates that a previous order may help improve compatibility. this however, is not enough, one needs to consider the weight (not just the preference but the intensity of the preference) which is related to the value of ijahp article: garuti/new advances of the compatibility index “g” in weighted environments international journal of the analytic hierarchy process 522 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.438 the elements that belong to the vector, as well as the angles of both vectors point to point (geometrically viewed as profiles). of course, this test should be carried out for a larger number of experiments in order to have a more reliable response. a second test conducted for 225 experiments (15 people conducting 15 experiments each), showed more or less the same initial results for an average g value in both cases, with and without order (±0.78 & ±0.50). next, figure 4 shows the meaning of ranges of compatibility in terms of index g and its description. degree of compatibility compatibility value range (g%) description very high ≥ 90% very high compatibility compatibility at cardinal level of measurement (totally compatible) high 85 – 89.9 high compatibility (almost totally compatible) moderate 75 – 84.9 moderate compatibility (compatibility at ordinal level) low 65 – 74.9 low level of compatibility very low 60 – 64.9 very low compatibility (almost incompatible) null (random) <60% random compatibility (totally incompatible) figure 4. ranges and meaning of compatibilities another interesting way to illustrate 90% as a good threshold for compatibility is the pattern recognition issue. compatibility is the way to measure if a set of data (vector of priorities or profile of behavior) corresponds to a recognized pattern. for instance, in a medical pattern recognition application, the diagnosis profile (the pattern) is built with the intensity values of signs and symptoms that correctly describe the disease, and is then compared with the signs and symptoms gathered from the patient. when these two profiles match at 90% or more, then the physician can be confident that the patient has the disease. when the profiles match at 85-90%, the physician in general agreed with the diagnoses offered by the software, but when the g value was below 85% (between 7984%), the doctor sometimes had trouble discerning if the new signs and symptoms (the new patient’s profile) were corresponding to the disease initially presented (nonconclusive information).finally, when the matching value (the g index) was below 75%, the physician wasn’t able to clearly recognize in the patient’s profile the disease initially offered. note that the new profiles were built artificially by changing some values of signs and symptoms in an imaginary patient profile. this was done in order to achieve values of 90, 85, 80% and so on with the intention of evaluating when a doctor changed his perception (mostly based on his pattern recognition ability). thus, two vectors may be considered compatible with great certainty or confidence (similar or matching patterns) when g is greater or equal to 90%. also, values between ijahp article: garuti/new advances of the compatibility index “g” in weighted environments international journal of the analytic hierarchy process 523 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.438 85-90% in general have a good chance of being correct (have a good level of certainty or approximation). 5. three simple applications of compatibility index g 5.1 example 1: is the order of choice a must? it used to be said that under the same decision problem two compatible people should make similar decisions. but, what do we mean when we say, “two compatible people should make similar decisions” (garuti, 2014; garuti, 2007)? does it mean that they should make the same choice? consider to the following case: two candidates: a, b are up for an election; three people, p1: choose candidate a; p2 & p3: choose candidate b. p1 and p2 are moderate people, thus their intensity of preference for the candidates are: for p1: 55-45, for candidate a, and p2: 45-55 for candidate b. on the other hand, p3 is an extreme person, thus his intensity of preference is 5-95 for candidate b. is p3 really more compatible with p2 than p1 just because p3 made the same choice of p2? (both have the same order of choice of voting for candidate b). it seems that the order of choice is not the complete or final answer. on the other hand, we know that in order topology, a metric of decision means intensity of choice (dominance of a over b).so, compatibility is not only related to the simple order of choice, but is something more complex and systemic, it is related to the intensity of choice. the next numerical example is shown in figure 5 below. suppose three people have an equal and different order of choice and its related priority vectors. person 1 (p1) person 2 (p2) person 3 (p3) order of choice intensity of choice order of choice intensity of choice order of choice intensity of choice 1º 0.364 3º 0.310 º1 0.501 2º 0.325 2º 0.325 2º 0.325 3º 0.311 1º 0.365 3º 0.174 order totally inverted with p1 order equally with p1 figure 5. comparing intensities and order of choice of 3 people as seen in figure 5, the order of p1 is the same as the order of p3 and the inverse of p2. order(p1) = order(p3) ≠ order(p2) (inverse order actually). considering just the above information, may we say that p1-p3 is closer than p1-p2? calculating g, for both combinations p1-p3 and p1-p2 we found: ijahp article: garuti/new advances of the compatibility index “g” in weighted environments international journal of the analytic hierarchy process 524 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.438 g(p1;p2)= 0.9 (≥90%), which implies that p1 and p2 are compatible choices (very high compatibility) g(p1;p3)= 0.77 (<90%), which implies that p1 and p2 are non-compatible choices (moderate to low). this is a very interesting result, considering that p2 has a totally inverted order of choice compared with p1yet, they are compatible people. on the other hand, p1 and p3, which have the same order of choice, are not compatible people. hence, it is very important to be able to measure the degree of compatibility (alignment) in a reliable way. alice in wonderland of charles lutwidge dodgson (lewis carroll) has a very interesting and pertinent phrase saying: “i tell you, sometimes 1-2-3 might look more like 3-2-1 than 1-2-3” (garuti, 2014; garuti, 2007). 5.2 example 2: mixing consistency and compatibility indices in a metric quality test drive a different and interesting application of g is possible when it is used to check the quality of a metric. when it is possible to compare a metric obtained with some method with the expected or actual metric, then the compatibility index g represents a great tool to test and verify the quality of the created metric. suppose for instance, we want to measure the quality of the following simple example. 5.2.1 presenting the problem (the criticism): we will set a hypothetical problem (a criticism made by some person), about the quality of the consistency index in pair comparison matrices (saaty’s index) (saaty, 2001, 2010).the hypothetical critic says that the consistency index (saaty’s index) is wrong since it may over look some values (comparisons) that are not acceptable by common sense. to illustrate this criticism, the following simple example is presented. suppose there are three bars of equal length like in figure6a. a b c figure 6a. bar length of course, the correct matrix comparison for this situation is the following (consistent) comparison matrix. figure 6b. bar comparisons case 1 the obvious (correct) priority vector “w” is: 1/3, 1/3, 1/3, with 100% of consistency (cr=0). ijahp article: garuti/new advances of the compatibility index “g” in weighted environments international journal of the analytic hierarchy process 525 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.438 suppose now that (due to some visualization mistake), the new appreciation about the bars is shown in figure 6c. figure 6c. bar comparisons case 2 the new (perturbed) priority vector w* is 0.4126, 0.3275, 0,2599, with cr=0,05 (95% of consistency) which according to the theory is the maximum acceptable cr for a 3x3 comparison matrix. the critic claims that the a-c bar comparison has 100% difference (100% of error) which is not an acceptable/tolerable error (easy to see even with the naked eye). also, the global error in the priority vectors is15.85%, calculated with the common formula: e=abs(u-v)/v, for each coordinate and then the average of the coordinates is taken. but, saaty’s consistency index says that (cr=95%) which is tolerable for a 3x3 comparison matrix. hence, the critic claims that saaty’s consistency index is wrong. 5.2.2 the response: the already described problem has at least two big areas of misunderstanding. first, the cr (the saaty’s index of consistency) comes from the eigenvalue-eigenvector problem, so it is a systemic approach, this means that it is not worried about any specific comparison when building the corresponding metric (saaty, 2001, 2010). second, the possible error should be measured by its final result (the resulting metric), not in the prior or middle steps. 5.2.3 the explanation: the first misunderstanding is explained by itself. for the second one, before any calculation, we need to understand what kind of numbers we are dealing with (in what environment we are working), because is not the same to be close to a big priority as to a small one. this is a weighted environment and the measure of the closeness has to consider this situation. we must work in the order topology domain to correctly measure the proximity in this environment. to do this correctly, two aspects of the information must to be considered. first, the intensity(the weight or priority)and then the degree of deviation between the priority vectors (the projection between the vectors).the index that incorporates these two factors simultaneously is the compatibility index g. summarizing, the vectors of correct and perturbed metric are as follows: correct metric (priority vector) : 0.3333 0.3333 0.3333 perturbed or approximated metric (priority vector) : 0.4126 0.3275 0.2599 ijahp article: garuti/new advances of the compatibility index “g” in weighted environments international journal of the analytic hierarchy process 526 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.438 the basic question here is, “how close is the approximated metric to the correct metric?” evaluating g(correct-perturbed), the g value that is obtained is 85.72%, which in numerical terms represents almost compatible metrics (high compatibility).as explained previously, g=90% is a threshold where two priority vectors are considered compatible vectors. also, g=85% is an acceptable lower limit value (high compatibility). hence, the two metrics are relatively close (close enough considering they are not physical measures). of course, better consistency can be achieved. the question is, “do we really obtain a better result when being totally consistent?” the answer is, probably no. in real problems, we never have the “real” answer (the true metric to contrast), and experience shows that pursuing consistent metrics per se, may provide less sustained results. for instance, in the presented problem one could answer that: a-b=2, a-c=2, b-c=1 and he/she would be totally consistent, but consistently wrong. in this case, the new priority vector would be: w**= (0.5, 0.25, 0.25), with cr=0 (totally consistent), and g= 71.5% which means incompatible vectors (low compatibility). thus, a totally consistent metric is incompatible with the correct result. so, at the end it is better to be approximately correct than consistently wrong. the consistency index is just a thermometer not a goal 1 . 5.3 example 3: using compatibility index g to measure the comparability of two different metrics a third example is presented when you have two rules of measurement in the same structure of decision (two different value systems).the problem is then how to combine and or compare the outcome of both. the first option is to try to reach a verbal consensus (the verbal or psychological option).the second is to use the geometric mean (the numerical or statistical option), and the third option is to measure the compatibility (the topological or closeness option). the first two options have long been known and are described in the literature (saaty, 2001, 2010; garuti, 2014, 2012). the third option is based on the compatibility principle and presents the following five advantages: (garuti, 2014) 1. able to find out if the output of one rule is comparable with the other(compatible rules) 2. able to find out the closeness between the 2 rules (how comparable they are?) 3. able to find out the criteria responsible for the possible gap (where to act in the most efficient way?) 4. able to find out the closeness between the initial personal rules with the final group rule. assessing: g(p1,gm) 5. able to use the numerical option (geometric mean) when and where it is necessary (modifying values in the places where most necessary in terms of efficiency). 1 cr=0.05 is the maximum acceptable value for inconsistency thus, i cannot go further with the bar comparison number (that means i cannot put a 3 instead of 2 in the cell (1,3)). if metric b is compatible with metric a, then it is possible to use metric b as a good approximation of a. this is a useful property when metric a is not available (most of the time we don’t know the correct/exact metric). the same exercise was performed from 4x4 to 9x9 matrices that is, putting a value (n-1) in the cell-position: (1,n), (n= matrix dimension), obtaining similar results (sometimes even better). ijahp article: garuti/new advances of the compatibility index “g” in weighted environments international journal of the analytic hierarchy process 527 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.438 for a better explanation, we will use an example to illustrate this idea in detail. suppose a mine company needs to change its shift-work system from the actual 6x1x2x3 (family of shifts, 8 working hours), to the new (desired) shiftwork system 4x4(family of shifts, 12 working hours).6x1x2x3 means a shift of 6 days of work 1 day off, 6 days of work 2 out and 6 days of work 3 out, with 8 hours per day, every week. this nets a total of144 working hours in 24 days of each working cycle. 4x4 means: 4 days of work followed by 4 days off with 12 hours per day every week. this also nets a total of 144 working hours in 24 days of each working cycle. the question is, “are these shifts equivalent? how do we know what shift-work is better (or less risky, since any shift is bad in essence)?”even if the total labor hours are the same (144 working hours), the shiftwork are not equivalent (in terms of quality of life and production). this depends on a variety of interdependent variables (number of working hours per day, entry time, number of free days per year, number of complete weekends per year, number of nights per shift, number of changes day/night/day per cycle, number of sleeping hours, opportunity for sleep, among many others. it also depends on how those variables are settled down and, of course, the weight (the importance that each variable has), which in time depends on which people you ask(workers, managers, stakeholders, owners, family or even the people that live around the mine). suppose we have two evaluation scenarios. in the first evaluation scenario, the decision rule (dr) of measurement is built with the people that work in the 6x1x2x3 shiftwork. in the second evaluation scenario, the dr of measurement is built with the people that work in the 4x4 shift. in each scenario the workers in that particular shiftwork weight the variables involved in the rule of measurement of their shift, since they know their own shiftwork best. as the process concludes, we end up with two different outputs. for the 6x1x2x3 shift (using the first rule of measurement), we have an impact index of 0.33, while for the second shift with the second rule, we have an impact index of: 0.37. of course, one cannot just say that shift 6x1x2x3 is better than 4x4 because it has a lower number for the potential negative impact (0.33<0.37), since they were built with different rules of measurement. in the end, we have two drs for the same problem; of course the question is not what rule is better, but how to make both drs comparable. one option could be to agree to use the same dr for both cases (the consensus, or verbal solution).however, this is not an option since the knowledge is located in different groups of people and is specific for each case; also the workers didn’t feel comfortable making this agreement. another option could be to take the geometric average (gm) of both rules and work with it as the final rule. even if this was a possibility, we really don’t know what we are doing when we combine or mix both rules(for example, we can’t just combine one rule of measurement in meters with one rule in inches).we first need to know if both drs are comparable. figure 7 shows this comparison of two rules of measurement. ijahp article: garuti/new advances of the compatibility index “g” in weighted environments international journal of the analytic hierarchy process 528 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.438 figure 7.comparing two rules of measurement the problem statement can be stated as, “are these two sets of decision criteria (gray and black profiles in figure 7) comparable (compatible)?”recall that they represent two different drs. the black bars show rule one which represents the rule of measurement for 6x1x2x3 shiftwork and is formed by 4 criteria (extracted from the global rule).the gray bars show rule two which represents the rule of measurement for 4x4 shiftwork and is formed by the same 4 criteria, but with different intensities. when we speak of comparable dr, we mean compatible dr (equivalents of measure), that is we can measure the level of risk of the alternatives with any of the two rules described above. to establish if both drs are equivalent it is necessary to calculate g for both profiles: g(profile gray – profile black)= 0.91 (91%) ≥ 90%, which means they are compatible. thus, we can use any of the two drs to measure the effect of the changing shiftwork. moreover, we now may use the geometric mean (gm) of both drs as the final rule, but knowing that we are combining rules that are compatible in fact, we are not mixing two far away points of view. this relevant issue is helpful when working with different groups of decision makers. but, now how are we to make both drs comparable when they are not compatible? or what rule is used to measure the alternatives? the following cases make some light on this. 5.3.1 case to case dr analysis there are four different cases of compatibility for dr where g can be applied: case 1: this has already been described where the drs of each decision making group are compatible. in this case, it is possible to use any of the two rules or (still better) use the geometric mean of both drs. figure 8a shows this graphically 0.1005 0.2116 0.3175 0.3704 0.1043 0.2087 0.2696 0.4174 0.0000 0.1000 0.2000 0.3000 0.4000 0.5000 1 2 3 4 ijahp article: garuti/new advances of the compatibility index “g” in weighted environments international journal of the analytic hierarchy process 529 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.438 figure 8a. compatibility as a concept of distance analytically, g(dr1,gm) and g(dr2,gm) >g(dr1,dr2) > 0.90. thus, the gm rule is clearly (numerically) better than any of the other two. when measuring the alternatives with this final dr, make the results comparable. case 2: the drs of either people (or groups) are not comparable (compatible), but they are compatible with the gm. in this case, take the gm of both rules, and then measure the compatibility of each dr with regards to the gm rule. if both initial drs are compatible with the gm rule, then you may use the gm rule as the final rule. figure 8b shows this graphically. figure 8b. compatibility as a concept of distance analytically, g(dr1,gm) and g(dr2,gm)≥0.9>g(dr1,dr2). when measuring the alternatives with this final dr, make the results to be comparable. case 3: the drs of both people are not compatible, but one (p1) is compatible with the gm. in this case, look in the compatibility profile of the gm with p2,g(gm;p2) for the position with the largest weighted difference and proceed as follows: 1) check if there is any “entry” error in the calculation process of p2 profile (some large inconsistency, or inverse entry in the comparison matrix). 2) check if the comparisons in the matrix associated to that position are what p2 really meant to say. 3) suggest to p2 to test acceptable numbers that produce a bigger g (getting closer to gm rule), until you can fall in case 2.this is shown graphically in figure 8c. p1(dr1 of p1) p2 (dr of p2) gm (dr of p1-p2 combined) g(dr1-dr2) ≥ 0.9 g (p2,gm)>0.9 g (p1,gm)>0.9 p1(dr1 of p1) p2 (dr of p2) gm (dr of p1-p2 combined) g (dr2-gm)>0.9 g (dr1-gm)>0.9 g(dr1-dr2)<0.9 ijahp article: garuti/new advances of the compatibility index “g” in weighted environments international journal of the analytic hierarchy process 530 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.438 figure 8c. compatibility as a concept of distance analytically, g(gm,p1)>0.9>g(p1,p2);g(gm,p2)>g(p1,p2)<0.9. when measuring the alternatives with this final dr, make the results to be comparable. there is also a fourth case, in which the initial drs are not compatible and none of the drs are compatible with gm rule. this is the toughest case, since p1 and p2 have very different points of view. the suggestion for this case is as follows:  try to revise the structure of the model to find some lost criterion or border condition not considered.  the weights of the criteria have to be revisited, and the support information of p1 and p2’s opinions revised. pay special attention to the elements (criteria) where g presents a big difference between p1 and p2. try to negotiate (get closer) on these criteria first. this can be done by changing the weights (plausible changes) of the criteria, the alternatives or both. if there is no changes with the initial position you may (as a last resource) apply the gm as the final rule, but it is probable that both people (or groups) may feel not fully represented by that imposed dr. 5.3.2 example of case to case dr analysis: one, among many, interesting applications of the case discussed previously (case to case dr analysis) is the next extracted example formed by 20 decision makers (dms). the problem is how to cluster them in the best way in terms of similarity (meixner et. al, 2016). in brief, the problem consists of 20 priority vectors which represent 20 dms, of 5 cardinal ranked elements that need to be clustered in the best possible way, in terms of their closeness among each other. to do this task, we first make the compatibility matrix of the 20 dms to measure their specific closeness (dm(i) to dm(j)). in doing this, we discovered (once again) the importance of the concept of “weighted environment” when measuring closeness, since the results obtained with g (in some specific cases) were very different from those obtained in the original study by meixner et. al. (2016) using a variation of norm2, specifically the square euclidean distance. for instance, when searching the minimum distance between the dms using g, the minimum distance was between (dm3-dm18), instead of dm5 with dm15 as in the meixner et. al. (2016) study which used the p1(dr1 of p1) p2 (dr of p2) gm (dr of p1-p2 combined) g (dr2-gm) <0.9 g (dr1-gm)>0.9 g(dr1-dr2)<0.9 ijahp article: garuti/new advances of the compatibility index “g” in weighted environments international journal of the analytic hierarchy process 531 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.438 euclidean distance. in this study, the distance between (dm3-dm18) is 6 times (600%) bigger (far) than (dm5-dm15) which is a very large difference, g shows only 2% (1.02). the maximum distance is found in (dm9-dm10) as the meixner et. al (2016) study found, but the min/max ratio is almost 100 times; g shows less than 2,5 times for the same ratio. this last difference is due to g being based in an absolute ratio scale, not absolute differences as the euclidean norm does (ratio scale is in the core of weighted environment measurement and also in the decision making process). the euclidean distance shows that dm15 is the most aligned (on average) to the rest of the dms, and the worst is dm10. however, g shows that in spite of the fact that dm15 has a very good alignment, dm14 is the best aligned, (slightly better than dm15), and dm9 is the worst aligned, instead of dm10 which, by the way, is also out of line like dm9 (see figure 9). the question here is not just about the ranking order, but about the real intensity or “distance” between the dms preferences measured by their ratio of preference. this measure is quite complicated to interpret if the difference between the ratios of both indices, for some cases, may reach the value of 100 times. by the way, it is acceptable for the absolute numbers of both indices (g and square euclidean distance) to differ even by a large gap. the large difference among its ratios however is not acceptable. this is the main problem of using euclidean distance (or its derivatives) when working within weighted environments. ijahp article: garuti/new advances of the compatibility index “g” in weighted environments international journal of the analytic hierarchy process 532 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.438 ranking by closeness to gm gm – dm(i) alignment qualification 0,8767 14 v.good(almostfull aligned) 0,8642 20 v.good(almost full aligned) 0,8584 15 v.good(almost full aligned) 0,8463 17 good 0,8393 4 good 0,8324 6 good 0,8081 5 good 0,8081 2 good 0,8022 1 good 0,7986 8 good 0,7886 19 regular 0,7816 12 regular 0,7530 18 regular 0,7521 13 regular 0,7475 16 poor 0,7087 11 poor 0,7030 7 poor 0,6132 3 out of line (totally misaligned) 0,6046 10 out of line (totally misaligned) 0,5929 9 out of line (totally misaligned) 0,5929 9 min. value 0,8767 14 max. value figure 9.table with level of alignment between gm and dm(i) the list is obtained by calculating the g(gm,dmi) and comparing the result with the different thresholds. the clusters were obtained by measuring the distance of each dm to the gm. this rapid (and easy) process of clustering was possible due to the fact that the g index has its own thresholds that make it possible to define where the break points are on the list of dms as shown in figure 8. the two largest distances to the gm are dm9 and dm10, which coincide with the largest distance measured directly between them. the min/max ratio is found for dm9 with dm14 which also coincides with the largest ratio among dms in the dms matrix of compatibility, realizing that the clustering process shown in the last table is representative and captures the extreme cases measured for both situations: distance and ratio. with the g index the clustering process (using gm) is much easier and direct. for instance, the first cluster is shown in red (very good and good), the second in yellow (regular) and the third in green (bad).there is also a fourth category in black (out of line) which represents those cases totally out of line in terms of alignment. we can also qualify the entire group. for instance, it seems that this group of 20 people, with respect to the gm, to be a group ijahp article: garuti/new advances of the compatibility index “g” in weighted environments international journal of the analytic hierarchy process 533 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.438 with an important level of agreement. the largest average value is 87.7% which represents, according to g, an agreement of almost cardinal level of compatibility. 6. conclusions a general conclusion of this paper is that measurements of distance in the decision making domain have to be done with g instead of any euclidean based indices if one wishes to be accurate. also, g is a better index for distance calculation in weighted environments especially when analyzed by single elements. another conclusion is that the indices based in euclidean geometry (like square euclidean distance, euclidean distance or j) could work in a statistical way of analysis, that is, when analyzing with a global view a large set of elements, but they may present important troubles when being applied to individual’s behavior (experiments carried out with those indices have shown some troubles, especially when using with weighted profiles of behavior). there are several more specific conclusions that can be drawn from this discussion as well. first, we discovered the great importance of the concept of a weighted environment when measuring closeness. thus, the compatibility index g is a necessary index for distance/alignment measurement in weighted environments (order topology domain) in order to correctly (mathematically correct) measure and declare if two profiles of behaviours are really close. the g index makes the following possible:  a matching analysis process  analysis and testing of the quality of the results  the availability of one more tool for conflict resolution in group decision making to achieve a possible agreement considering that those profiles may represent system values  a pattern recognition process (assessing how close one pattern is to another)  making better benchmarking  membership analysis (closeness analysis to estimate if an element belongs to one set of elements or another) secondly, this analysis shows that the only compatibility index that performs correctly for every case is the g index. this index always keeps the outcome in the 0-100% range, which is an important condition, since any value out of the 0-100% range would be difficult to interpret (and the beginning of a possible divergence). it is also important to note that the g and euclidean outcomes can be close, but the g is much more accurate or sensitive to changes because it is not based on absolute differences (xi) like distance, but on relative absolute ratio scales. we have to remember that we are working on ratio scales (absolute ratio scales to be precise). this is the same behaviour detected in the garuti (2014) and garuti (2012) where the euclidean distance calculation shows no difference in the distance of the parallel trend from case 1 to 6. the euclidean based index cannot detect a bit of difference in the compatibility value among those cases because the absolute difference of the coordinates remains the same. therefore, with a euclidian based index one may reach the wrong conclusion that no difference exists for vector compatibility from cases 1 to 6 (the first case study is as incompatible as 2, 3, 4, 5 ijahp article: garuti/new advances of the compatibility index “g” in weighted environments international journal of the analytic hierarchy process 534 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.438 or 6), which is not an expected result. this unexpected behaviour occurs because the euclidean norm is based on differences, and also because it’s not concerned about the weights of the coordinates and the projection between the priority vectors. it is important to remember that the numbers inside the priority vector represent preferences. hence, in terms of proximity it is better to be close to a big preference (big coordinate) than to the small one and this issue is better resolved in ratio scales 2 . other tests performed in greater spaces (3d to 10d) show the same trend. the bigger the space dimension the greater the likelihood of finding singularity points for the other compatibility indices like hadamard product (saaty’s compatibility index), inverted dot product, weighted dot product, and hilbert’s index. this problematic behavior occurs in both parallel and perpendicular trends 3 . it is interesting to note that function g is not the simple dot product since it depends on two different dimensional factors. on one side, you have the intensity of preference (related with the weight of the element), and on the other side you have the angle of projection between the vectors (the profiles). this means that g is a function based on the intensity of preference (i) and the angle of projection (between the priority vectors, that is: g= f(i,). clearly, the g function is not the simple dot product (as normally defined), but a combination of weight (intensity of preference) and projection (angle between vectors), that is, something more complex and rich with information. it is also important to note that both data (intensity and angle) are normally implicit in the coordinates of the priority vectors and have to be correctly extracted. it also matters if the priorities are presented in relative measurement (rm) or absolute measurement (am) format. there are huge possible applications of this index in different fields. as an example, figure 9 lists the possibilities in the field of social and management sciences. 2 see the complete example in garuti (2014, 2012) 3 details in garuti (2014, 2012) ijahp article: garuti/new advances of the compatibility index “g” in weighted environments international journal of the analytic hierarchy process 535 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.438 on medicine measuring the degree of matching (proximity) between patient and disease diagnose profiles. on buyers-seller profiles measuring the degree of matching between house buyers and sales project. on group decision making (conflict resolution) measuring how close are two (or more) different value systems (where they differ and for how much). on quality tests measuring what mcdm decision method can builds a better metric. on agricultural production& supplier selection measuring the proximity between the cultivate plants against a healthy plant (based on its micro & macro nutrients) and selecting the best nutrient seller. on shiftwork prioritization measuring how close are the different views among the different stakeholders (workers view, company view, community view). on company social responsibility (csr) measuring how close are the different views among the different stakeholders (economic, environmental and social view). figure 9. possible applications of index g all the examples mentioned above are from real cases of application. there are many different applications for index g. this is a summary of all possibilities that g may have: • compatibility of systems value: g is an index able to be used in social and management sciences to measure compatibility of group decision making (dms) intra and inter groups. the expression of g for this case is: g(dm1-dm2), which means level of compatibility(closeness) between dm1 and dm2.with dm1 and dm2 being the decision’s metric of each decision maker. • compatibility for quality test: g can help assess the quality of a built decision metric. as presented in section7.2, g may help evaluate the quality of any new metric based in a ratio scale. the result is achieved by comparing the new metric with some standard or with an already known result. • profiles alignment: g can help establish if two different profiles are aligned. in general, it is not easy to know if two complex profiles are aligned, especially when the profiles are complex with many ijahp article: garuti/new advances of the compatibility index “g” in weighted environments international journal of the analytic hierarchy process 536 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.438 variables, with different importance and different behavior on each one. this is the case when trying to measure the degree of matching between a medical diagnose and a list of diseases, or the degree of matching between a sale project and its possible buyers and many other similar cases. • compatibility for comparability: g can help to establish if two different measures are or are not comparable. one relevant point when comparing numbers from different outcomes is to know if those numbers are comparable or not. for instance, if i know that the impact of strategy a is 0.3 and the impact of strategy b is 0.6, i cannot say that strategy a has twice the impact of strategy b, unless both strategies were measured with exactly the same rule. however, for many reasons, sometimes that is not possible. in that case, i need to know if the rules of measurement are compatible. if so, it is possible to compare both numbers, otherwise it is not possible. • compatibility for sensitive analysis and threshold: g can help establish the degree of membership or the trend for membership (tendency) of an alternative. the idea is equivalent to the classic sensitive analysis when making small changes in the variables. the change resulting in the g value (before and after the sensitive analysis) would show where the alternative is more likely to belong (trend of belonging). ijahp article: garuti/new advances of the compatibility index “g” in weighted environments international journal of the analytic hierarchy process 537 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.438 references aldred, j. (2005). intransitivity and vague preferences. cambridge, uk: emmanuel college. doi:10.1007/s10892-005-7977-9 garuti, c.a. (2007). measuring compatibility in weighted environments: when close really means close? international symposium on ahp, 9, viña del mar, chile. garuti, c. (2012). measuring in weighted environments: moving from metric to order topology. santiago, chile: universidad federico santa maria. doi: 10.5772/63670 garuti, c.a. (2014). compatibility of ahp/anp vectors with known results. presentation of a suggested new index of compatibility in weighted environments. international symposium of the ahp. garuti, c. (2016). consistency and compatibility (two sides of the same coin). international symposium of the ahp. hilbert, d. (1895). ueber die gerade linie als kürzeste verbindung zweier punkte. mathematischeannalen, 46, 91–96.doi:10.1007/bf02096204 jaccard p. (1901). distribution de la flore alpine dans le bassin des dranses et dans quelques regions voisines. bulletin de la sociétévaudoise des sciences naturelles, 37, 241-272. mahalanobis, p.c. (1936). on the generalized distance in statistics. proceedings of the national institute of science of india, 12, 49-55. papadopoulos, a., troyanov, m. (2014). handbook of hilbert geometry. zurich: european mathematical society. doi: 10.4171/147 saaty, t.l. (2001). the analytic network process: decision making with dependence and feedback. pittsburgh, pa: rws publications. doi: 10.4018/978-1-59140-7027.ch018 saaty, t.l. (2010). group decision making: drawing out and reconciling differences. pittsburgh, pa: rws publications. whitaker, r. (2007). validation examples of the analytic hierarchy process and analytic network process. mathematical and computer modeling, 46, 840-859. doi: http://dx.doi.org/10.1016/j.mcm.2007.03.018 https://en.wikipedia.org/wiki/mathematische_annalen https://en.wikipedia.org/wiki/digital_object_identifier https://dx.doi.org/10.1007%2fbf02096204 http://dx.doi.org/10.1016/j.mcm.2007.03.018 ijahp article: rahman, biswas, halder/the application of ahp method for supplier selection of bangladesh’s jute industry: from the perspective of both public and private sectors international journal of the analytic hierarchy process 162 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.508 the application of ahp method for supplier selection of bangladesh’s jute industry: from the perspective of both public and private sectors mustafizur rahman 1 tarunkumer biswas shuvo halder department of industrial and production engineering jessore university of science and technology jessore-7408, bangladesh mostafizkuet09@gmail.com shuvoh38@gmail.com abstract in the competitive global business market, selection of the best supplier plays a vital role for the survival of any manufacturing company. the selection of a good supplier can make it possible for a company to reach the top position in the market. on the other side, the selection of a poor supplier can lead a company to a lower market position or even shutdown. as a developing country, bangladesh has many very small and extremely large manufacturing companies where supplier selection is not seriously considered. these companies are selecting suppliers in the traditional way by giving priority to cost and quality, which acts as an impediment to their development. in this paper, the analytic hierarchy process (ahp) method is applied in order to find the best supplier for the jute industry in bangladesh. the study compares the application process of the method from both the public and private sector perspectives. for this purpose, the data are collected from a government factory named “jessore jute industries ltd” and a private factory named “ahyan jute mills” and their present strategies are compared. this paper recommends to these jute industries that the use of the ahp method in selecting the best supplier can be effective and efficient. keywords: supplier selection; analytic hierarchy process (ahp); multi criteria decision making; jute mills 1. introduction in recent years, increased supply chain profitability is the prime concern for the management of any company. in the case of a manufacturing factory, the success of upstream supply chain management is solely dependent on how good the selected supplier is. this is because supplier selection affects the product quality and is directly or indirectly related to customer satisfaction. day by day the transmuting of customer postures, incrementing varieties of customer demands, advances of acknowledgement: the authors would like to thank the managing director, general manager and all of the staff of the government industry, jessore jute industries ltd, and the private industry ahyan jute mills in bangladesh for providing their wholehearted cooperation to collect data for analysis. mailto:mostafizkuet09@gmail.com mailto:shuvoh38@gmail.com ijahp article: rahman, biswas, halder/the application of ahp method for supplier selection of bangladesh’s jute industry: from the perspective of both public and private sectors international journal of the analytic hierarchy process 163 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.508 information technologies, product proliferation, competition in the international environment, increases in governmental regulations, short product life cycles and environmental consciousness have coerced all types of companies to fixate on supply chain management (digalwar et al., 2014). supply chain is a sequential chain of sundry participants such as sub-suppliers, suppliers, manufacturers and distributors which culminates with customers who authentically integrate value with the products. the raw materials not only flow through the chain and are converted into a final product, but ultimately reach the customers. the main objective of each participant in the chain is to provide the right products to the customers at the right time. moreover, the goal of supply chain management which involves the manager and those participants is to build the most optimal chain (aktepe & ersoz, 2011). the supplier is considered the first stage of this chain. it is always very arduous for the companies to select the right suppliers. recently, supplier selection has become the most consequential quandary because every supplier has both strengths and weaknesses. selection of suppliers requires several criteria to be considered which turns the quandary into a multi-criteria decision-making problem (tahriri et al., 2008). in bangladesh, jute products have a valuable contribution to the economy of the country. therefore, the importance of jute products in bangladesh cannot be ignored (shakil et al., 2013). evaluation and selection of suppliers lead to the prosperity of any jute product factory whether government or private. the quality and cost of jute products are directly cognate to the purchased raw materials (koul & verma, 2011). traditionally, the jute industries in bangladesh select their supplier predicated on the cost and quality and sometimes consider the distribution schedule (tahriri et al., 2008). there are several supplier selection methods available such as different mcdm approaches, linear weighting models etc. (digalwar et al., 2014). bangladeshi jute industries are failing to meet product demand worldwide (akter, 2015). the selection of the right supplier increases productivity. this supplier selection process is composed of four phases: the initial quandary definition, the formulation of criteria, the qualification of potential suppliers, and the final selection among the eligible suppliers (khan et al., 2015). this paper analyzes the current supplier selection system of “jessore jute industries ltd” as a sample of regime industry, and “ahyan jute mills” as a private sector representative. it compares them utilizing the analytic hierarchy process (ahp), a multi-criteria decision-making approach, to select the right supplier. the rest of the paper is structured as follows: section 2 provides a brief review of related previous works. in section 3, the scenario of supplier selection in the jute industry is presented and discussed. the selection methodology using ahp is conducted in section 4. section 5 provides key results of the study with discussion. section 6 concludes the paper. 2. literature review supplier selection for an organization is a process in scm for evaluating the eligible suppliers and selecting the right supplier who can meet the organization’s requirements. the supplier selection process requires considering a number of criteria. dickson (1966) in his seminal work suggested 23 consequential criteria which are utilized for the evaluation and selection of suppliers (dickson, 1966). in his ijahp article: rahman, biswas, halder/the application of ahp method for supplier selection of bangladesh’s jute industry: from the perspective of both public and private sectors international journal of the analytic hierarchy process 164 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.508 work, he suggests, "from the purchasing literature, it is fairly facile to abstract a list of at least 50 distinct factors (characteristics of vendor performance) that are presented by sundry authors as being consequential to consider in a vendor selection decision" (dickson, 1966). a vast amount of research has been done concerning supplier criteria. charles (1991) reviewed 74 publications about supplier selection from 1966 to 1991 and showed that more than 63% of them were in multi-criteria environments (weber, et al., 1991). davidrajuh (2000) reviewed some papers and published a paper which accentuates the paramount criteria and their invariability (davidrajuh, 2000) bross & zhao (2004) concluded on their review that the most valuable supplier selection criteria are cost, quality, accommodation, relationship, and organization. because of these many criteria, supplier selection has become a multi-criteria decision-making quandary which consists of both qualitative and quantitative metrics. since it is the most consequential and indispensable part of a company, an abundance of studies and investigations has been published on supplier selection. alehashem et al. (2013) mentioned 13 paramount criteria for supplier selection in his work. he additionally suggested applying the analytical hierarchy process (ahp) method to identify and evaluate the supplier selection. since supplier selection is a multi-criteria decision-making process, it consists of a number of valuable criteria. many studies have already been done to find these criteria, which usually affect the supplier (vaidya & kumar, 2006). after consulting with jute experts and reviewing some research papers, the authors have identified the important factors which are considered as criteria for selecting a supplier such as price (cost), quality and delivery, responsiveness to customer needs, relationship and business effort, technical support, product appearance, productivity, flexibility, direct cost, trust, responsibility, discipline, financial, warranty, performance history, location, long term relationship, reliability etc. (shakil, ullah & lutfi, 2013; rajesh, & malliga, 2013; dursun & karsak, 2013; verma, 2013; yusuff et al., 2001; khan & islam, 2014; liu & hai, 2005; ghorabaee et al., 2017) mohammad abdolshah, 2013, thiruchelvam & tookey, 2011, ellram, 1990 and stamm and golhar (1993) mentioned 60, 42, 36, 18, and 13 criteria for supplier selection, respectively in their study. weber et al. (1991) reviewed 47 articles on supplier criteria and recommended that the most important criteria are price, quality, delivery, production capacity, and localization. the ahp method was first proposed by saaty in 1980. since then, a large volume of work using the ahp has been carried out that is available in the literature (kazempoor et al., 2015; ramlan & qiang, 2014). among them, the number of works related to the supplier selection problem that use the ahp model is not trivial (tahriri et al., 2008; alehashem et al., 2013; kazempoor et al., 2015). the analytic hierarchy process has been a practical implement in the hands of decision makers and researchers, and it is one of the most widely used multiple criteria decision-making implements (vaidya & kumar, 2006). the authors used the ahp approach to cull and evaluate suppliers for jute industries. by applying the ahp to supplier assessment in a multi-criteria environment, the authors were able to solve the supplier quandary. the ahp method may integrate multiple criteria in the subjective environment of the decision-making process for supplier selection. rather than ahp, several other paradigms have also been used over the last three decades such as fuzzy analytic hierarchy process, data envelopment analysis, ijahp article: rahman, biswas, halder/the application of ahp method for supplier selection of bangladesh’s jute industry: from the perspective of both public and private sectors international journal of the analytic hierarchy process 165 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.508 commixed integer programming, topsis, fuzzy topsis, fuzzy qfd, ahp qfd, analytic network process and expert systems (tahriri et al., 2014; tas, 2012; ayhan, 2013; ahmady et al, 2012; hu et al., 2016; sasi & digalwar, 2015; gurung & phipon, 2016; kilie, 2012; dursun & karsak, 2013; rajesh & malliga, 2013; gupta et al, 2015; sadeghi et al, 2012; sanayei et al, 2010). the selection of suppliers for jute industries requires considering a number of subjective factors. in this situation, the analytic hierarchy process (ahp) provides an effective tool to deal with these subjective factors that may come from multiple sources (yusuff et al., 2001). khan & islam (2014) proposed an incipient model for selecting suppliers for jute industries in a competitive environment. liu & hai (2005) have found widespread applications of the analytic hierarchy process (ahp) in intricate decision-making quandaries involving multiple criteria in systems of many calibers. ghorabaee et al. (2017) reviewed a total of 339 publications and some book chapters from 2001 to 2016 and concluded that the ahp and topsis methods are the most popular approach for multi attribute decision-making (madm) as well as supplier selection. although, a large volume of work exists on the supplier selection problem, there is still no work within the jute industry. in bangladesh, the jute industry has been facing a serious problem with raw materials supply for the past decade. this problem results in higher jute prices, longer lead-times, supply uncertainty and poor relationships between industries etc. for these reasons, this sector has not seen any profit for a long time. selection of the right supplier can mitigate a large portion of this problem. therefore, there is a need for a work that seeks to find a systematic way of supplier selection for the jute industry in bangladesh. the current research is intended to develop a systematic process of supplier selection for the jute industry using the ahp model. 3. present selection procedure of jute suppliers suppliers are an essential part of business for both government and private jute factories. the current supplier selection systems for both types of factories are discussed below. 3.1 private industry private factories like ahyan jute mills conventionally produce various types of jute products. as a jute product-producing factory, they need to have a supplier for collecting raw materials. unfortunately, only 20% of the jute is supplied by the supplier and the remaining 80% of the jute is purchased by their own people. (1) in the 20% jute obtained from suppliers only 5-7% of the suppliers are permanent and others are short-lived. these suppliers are selected mainly based on personal relationships. (2) for purchasing the jute, each of the factories sends their experienced employees to the local markets that are selected by the top management. they buy the jute from the local market and carry it back to the factory in their own trucks. when the jute is purchased, the buyers customarily consider some criteria such as price, quality, distribution etc. however, they do not utilize any method or valuable ijahp article: rahman, biswas, halder/the application of ahp method for supplier selection of bangladesh’s jute industry: from the perspective of both public and private sectors international journal of the analytic hierarchy process 166 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.508 implementation for decision-making. they give priority to feelings rather than facts and logic when buying jute, which is one of the main obstacles for business. 3.2 government industry government factories, like jessore jute industries ltd., customarily use perpetual suppliers. firstly, jute officers customarily contact all the suppliers who are interested in working with these industries. then they ask them about the criteria, which have been previously selected. their considered criteria are price, quality, distribution, reliability etc. after getting the answer from the suppliers, the jute officers arrange an internal meeting with the experts and top management. this group discusses and selects the supplier. they are not utilizing any decision-making method, but still utilizing expert opinion. 4. research methodology the first and most important instruments of this research are interviews with the purchase managers of the jute factories. these interviews were carried out by the authors. next, the interviews were analyzed and the unnecessary data was eliminated. then the data was summarized and interpreted into an easily understandable form. the next step was to input the data into microsoft excel for further analysis and pairwise comparisons were calculated. the final stage involved synthesizing judgments and checking for consistency to make sure that the judgments were accurate. saaty, in 1980, suggested that this calculation would be adequately consistent if the consistency ratio (cr) is less than 0.1(10%). the methodology can be summarized in 4 steps.  step 1: collect data through interviews  step 2: analyze and eliminate unnecessary data  step 3: summarize data into easily understandable form  step 4: calculate data using ahp method  step 5: make a decision the summary of ahp model is that it compares different alternatives (i) against different attributes (j) through an n×n judgment matrix as follows: ijahp article: rahman, biswas, halder/the application of ahp method for supplier selection of bangladesh’s jute industry: from the perspective of both public and private sectors international journal of the analytic hierarchy process 167 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.508 where, i and j are alternatives or attributes to be compared, and aij is a value which represents the comparison between alternatives/attributes i & j. in the matrix    n i ijk ay 1 , where k= 1, 2…n and j = 1, 2 ….n (1) geometric mean is calculated as follows:        n knkkk aaab /1 21  (2) normalized weights are calculated as follows:    n k k k k b b x 1 (3) consistency index (c.i.) is calculated as follows: 1 .. max    n n ic  , where    n k kknn xyxyxyxy 1 2211max  (4) comparision matrix geometric mean normalized weigh j nk 21 i n k     2 1 1 1 1 1 21 21 2221 1112         nknn knkk nk nk aaa aaa aaa aaa sum = nk yyyy  21                           n k b b b b     2 1                           n k x x x x     2 1 ijahp article: rahman, biswas, halder/the application of ahp method for supplier selection of bangladesh’s jute industry: from the perspective of both public and private sectors international journal of the analytic hierarchy process 168 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.508 table 1 consistency index (r.i.) some randomly generated consistency index (r.i) values are, n 1 2 3 4 5 6 7 8 9 10 r.i. 0 0 0.58 0.90 1.12 1.24 1.32 1.41 1.45 1.49 then consistency ratio (c.r.) is found by using the following equation: .. .. . ir ic rc  (5) if c.r. ≤ 10%, the alternatives accepted. otherwise, it is rejected. the authors have identified the most important criteria of suppliers for jute industries which are shown in the table 2, table 2 most important identified criteria by researchers p ri c e q u a li ty d e li v e ry r e li a b il it y c a p a c it y r e la ti o n sh ip p e rf o rm a n c e t im e s e rv ic e o rg a n iz a ti o n dickson(1966)    welber, current and benton (1991)     tullous and munson (1991)       pullman (1998)     zhang, lei, cao, ng (2003)    bross and zhao (2004)      farzadtahriri (2008)     om pal (2013)     dr. devendra singh verma (2013)    from the above table and more analysis of supplier criteria, the authors find that the six most important criteria for jute industries suppliers are price, quality, delivery, performance, reliability and availability. after discussions with management and experts, the authors have considered four alternative suppliers. figure 1 shows the hierarchical structure of supplier selection problem considering six evaluation criteria. ijahp article: rahman, biswas, halder/the application of ahp method for supplier selection of bangladesh’s jute industry: from the perspective of both public and private sectors international journal of the analytic hierarchy process 169 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.508 figure 1 hierarchical structure of the evaluation criteria and suppliers now the pairwise comparison matrix has been prepared using a “scale of relative importance”. this shows the relative importance of one option using grades ranging from 1-9. the scale range is described in table 3. table 3 scale of relative importance intensity of importance definition explanation 1 equal importance two activities contribute equally to the objective 3 moderate importance experience and judgment slightly favor one activity over another 5 strong importance experience and judgment strongly favor one activity over another 7 very strong importance an activity is favored very strongly over another; its dominance demonstrated in practice 9 extreme importance the evidence favoring one activity over another is of the highest possible order of affirmation 2,4,6,8 for compromise between the above values sometimes one needs to interpolate a compromise judgment numerically because there is no good word to describe it. next, the pairwise comparison matrices are analyzed and augmented for the criteria and alternative suppliers in order to obtain the following results. the authors have calculated tables 4-11 using equations 1-5 (shown above). ijahp article: rahman, biswas, halder/the application of ahp method for supplier selection of bangladesh’s jute industry: from the perspective of both public and private sectors international journal of the analytic hierarchy process 170 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.508 evaluation of selected criteria: table 4 comparison and judgment for criteria comparison matrix and judgment for criteria to find the normalized weight criteria price qua lity deliv ery perfo rman ce reliab ility avail abilit y geometric mean normalized weight price 1 1 2 7 6 5 2.7365 0.3429 quality 1 1 3 5 4 7 2.7365 0.3429 delivery 1/2 1/3 1 2 1/2 3 0.8909 0.1116 performanc e 1/7 1/5 ½ 1 1 1/2 0.4389 0.0549 reliability 1/6 1/4 2 1 1 1 0.6609 0.0828 availability 1/5 1/7 1/3 2 1 1 0.5167 0.0647 sum 3.009 2.92 6 8.833 18 13.5 17.5 7.9804 n = 6 λmax = 6.2591 c.i. = 0.05182 c.r. = 4.179% <10% consistency ratio (c.r.) is less than 10%, so accepted. evaluation for alternative suppliers: table 5 evaluation for price comparison matrix and judgment table for price supplier 1 supplier 2 supplier 3 supplier 4 geometric mean normalized weight supplier 1 1 2 1/3 1 0.9036 0.20952 supplier 2 1/2 1 1/2 1 0.7071 0.16395 supplier 3 3 2 1 2 1.8612 0.43156 supplier 4 1 1 1/2 1 0.8408 0.19495 sum 5.5 6 2.333 5 4.3127 n = 4 λmax = 4.1176 c.i. = 0.0389 c.r. =3.92% <10% consistency ratio (c.r.) is less than 10%, so accepted. ijahp article: rahman, biswas, halder/the application of ahp method for supplier selection of bangladesh’s jute industry: from the perspective of both public and private sectors international journal of the analytic hierarchy process 171 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.508 table 6 evaluation for quality comparison matrix and judgment table for quality supplier 1 supplier 2 supplier 3 supplier 4 geometric mean normalized weight supplier 1 1 1/8 4 1/4 0.59460 0.09701 supplier 2 8 1 9 1 2.91295 0.47529 supplier 3 ¼ 1/9 1 1/8 0.24274 0.03960 supplier 4 4 1 8 1 2.37841 0.38807 sum 13.25 2.236 22 2.375 6.12870 n = 4 λmax = 4.1409 c.i. = 0.04699 c.r. = 5.22% <10% consistency ratio (c.r.) is less than 10%, so accepted. table 7 evaluation for delivery comparison matrix and judgment table for delivery supplier 1 supplier 2 supplier 3 supplier 4 geometric mean normalized weight supplier 1 1 3 1 5 1.96798 0.41017 supplier 2 1/3 1 1 6 1.18920 0.24785 supplier 3 1 1 1 3 1.31607 0.27430 supplier 4 1/5 1/6 1/3 1 0.32466 0.06766 sum 2.533 5.167 3.333 15 4.79791 n = 4 λmax = 4.247 c.i. = 0.0824 c.r. = 9.16% <10% consistency ratio (c.r.) is less than 10%, so accepted. table 8 evaluation for performance comparison matrix and judgment table for performance supplier 1 supplier 2 supplier 3 supplier 4 geometric mean normalized weight supplier 1 1 4 9 6 3.83365 0.60615 supplier 2 1/4 1 7 4 1.62657 0.25718 supplier 3 1/9 1/7 1 1/3 0.26970 0.04264 supplier 4 1/6 1/4 3 1 0.59460 0.09401 sum 1.52 5.39 20 11.33 6.32452 n = 4 λmax = 4.225 c.i. = 0.07516 c.r. = 8.35% <10% consistency ratio (c.r.) is less than 10%, so accepted. ijahp article: rahman, biswas, halder/the application of ahp method for supplier selection of bangladesh’s jute industry: from the perspective of both public and private sectors international journal of the analytic hierarchy process 172 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.508 table 9 evaluation for reliability comparison matrix and judgment table for reliability supplier 1 supplier 2 supplier 3 supplier 4 geometric mean normalized weight supplier 1 1 1/4 2 1/3 0.63894 0.13222 supplier 2 4 1 2 1 1.68179 0.34803 supplier 3 1/2 1/2 1 1/6 0.45180 0.09349 supplier 4 3 1 6 1 2.05976 0.42624 sum 8.5 2.75 11 2.5 4.83229 n = 4 λmax = 4.1749 c.i. = 0.05831 c.r. = 6.47% <10% consistency ratio (c.r.) is less than 10%, so accepted. table 10 evaluation for reliability comparison matrix and judgment table for availability, supplier 1 supplier 2 supplier 3 supplier 4 geometric mean normalized weight supplier 1 1 4 1 2 1.68179 0.37537 supplier 2 1/4 1 1/2 1 0.59460 0.13271 supplier 3 1 2 1 3 1.56508 0.34931 supplier 4 1/2 1 1/3 1 0.63894 0.14261 sum 2.75 8 2.833 7 4.48041 n = 4 λmax = 4.0819 c.i. = 0.02731 c.r. = 3.03% <10% consistency ratio (c.r.) is less than 10%, so accepted. table 11 evaluation and ranking of supplier based on the result of above matrices, an overall evaluation is performed using the calculated weights of the alternative suppliers and five measuring criteria, as follows: alternative suppliers criteria and their weights composite weights (∑ kx 𝑜𝑓 𝑐𝑟𝑖𝑡𝑒𝑟𝑖𝑎 × kx of alternative) overall rankin g price quality delivery performance reliability availability 0.3429 0.3429 0.1116 0.0549 0.0828 0.0647 supplier 1 0.20952 0.09701 0.41017 0.60615 0.13222 0.37537 0.21939 4 supplier 2 0.16395 0.47529 0.24785 0.25718 0.34803 0.13271 0.29837 1 supplier 3 0.43156 0.03960 0.27430 0.04264 0.09349 0.34931 0.22485 3 supplier 4 0.19495 0.38807 0.06766 0.09401 0.42624 0.14261 0.25714 2 5. results and discussion in table 10, it is observed that supplier 2 has the highest composite weight and is marked as rank 1. supplier 4 is marked as rank 2 and supplier 1 has the lowest composite weight which is marked as rank 4 thus, the decision is to select the supplier with the highest composite weight which is supplier 2. the application of the ijahp article: rahman, biswas, halder/the application of ahp method for supplier selection of bangladesh’s jute industry: from the perspective of both public and private sectors international journal of the analytic hierarchy process 173 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.508 suggested ahp method has been discussed, and the existing methods of both public and private sectors of jute factories are listed and compared below. government industry vs. private industry system: 1. during supplier selection, the purchase manager of the government jute industry discusses with experts and top management, which is genuinely time consuming. (recently bgmea has become the controller of the jute supply). however, in the private jute industry, factories purchase most of their raw materials (especially jute) by their own hand. 2. private jute industries purchase their raw materials for one year at a time because it requires an astronomically immense amount of area/space to store them. however, government industry uses supplier storage. they can inductively authorize their raw materials when they are required. they do not need any extra area/space. analyzing the above-discussed problems, the authors recommend that both industries use any mcdm approach to select suppliers. as an example, the ahp method was applied to both private and government industry to select the suppliers. application of ahp in private sector:  ahp is a multi-criteria decision-making method. it accumulates all of the paramount criteria which are essential for a supplier. therefore, it is possible to select the optimum supplier by utilizing this method. however, it is not possible in the current system to select the optimum supplier for the private industry.  in the current system, the private industry is responsible for obtaining the raw materials. however, when utilizing ahp, the supplier will be responsible. application of ahp in government sector:  ahp decision-making model is more appropriate than the current “discussion method” which is currently practiced because not all criteria are considered in these discussions. here, supplier selections of two jute sectors have been compared and the application of ahp for both sectors has been shown. both companies can benefit by applying ahp in their supplier selection. 6. conclusion in this paper, the authors have used the ahp approach, as an example of a mcdm approach, to select the best supplier for jute industries in bangladesh, and made a comparison with the present method of both government and private industries. there are some other techniques such as topsis, fuzzy, anp, electre, dematel, promethee etc. that could also be used for this system. the authors have studied the ahp approach because it allows the decision makers of jute industries to rank alternative suppliers predicated on their subjective judgments regarding the attributes that are paramount. while studying both government and private jute industries, the authors have found that neither industry utilizes any modern decision-making method ijahp article: rahman, biswas, halder/the application of ahp method for supplier selection of bangladesh’s jute industry: from the perspective of both public and private sectors international journal of the analytic hierarchy process 174 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.508 to select their supplier. they are failing to select the right suppliers for their industries, which has a negative effect on the success of their business. in this method, the supplier is selected based on some criteria and if the selection criteria can be increased, the result would be more fruitful. this is the limitations of the study. the other mcdm approaches could be applied in this sector to select the right supplier. a comparison analysis could be performed of these methods (topsis, electre, dematel, etc.) with the ahp in a supplier selection decision-making problem. this paper recommends that both private and government jute industries utilize the ahp for supplier selection and evaluation. it would allow the organization to reduce costs, ascertain excellent quality, procure authentic-time distribution, optimum leadtime, mitigate peril, increment and smooth engenderment flow and receive better accommodation of raw materials. ijahp article: rahman, biswas, halder/the application of ahp method for supplier selection of bangladesh’s jute industry: from the perspective of both public and private sectors international journal of the analytic hierarchy process 175 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.508 references abdolshah, m. (2013). a review of quality criteria supporting supplier selection, journal of quality and reliability engineering, 2013, 1-9. doi: http://dx.doi.org/10.1155/2013/621073 ahmady, n., ahmady, e., sadeghi, s. (2012). developing a data envelopment analysis methodology for supplier selection in the presence of fuzzy undesirable factors, international journal of industrial mathematics, 4(3), 247-256. aktepe, a. and ersoz, s. (2011). a fuzzy analytic hierarchy process model for supplier selection and a case study, international journal of research and development, 3(1), 33-37. akter, r. (2015). jute goods exports to australia: an analysis of market potentials for bangladeshi jute products, international journal of management & business studies, 5(2), 21-30. alehashem, m., sheikholeslam, m.n., emamianand, s., moghadam, s.a. (2013). a supplier selection case study by analytical hierarchical process in textile industry, international journal of advanced engineering technology, 2(3), 33-41. doi: http://dx.doi.org/10.12785/aeta/020302 ayhan, m.b. (2013). a fuzzy ahp approach for supplier selection problem: a case study in a gearmotor company, international journal of managing value and supply chains, 4(3), 11-23. doi: 10.5121/ijmvsc.2013.4302 bross, m.e. and zhao, g. (2004). supplier selection process in emerging markets the case study of volvo bus corporation in china, master’s thesis, school of economics and commercial law göteborg university. davidrajuh, r. (2000). automating supplier selection procedures, doctoral thesis, norwegian university of science and technology. dickson, g.w. (1966). an analysis of vendor selection systems and decisions, journal of purchasing, 2(1), 5-17. doi: https://doi.org/10.1111/j.1745493x.1966.tb00818.x digalwar, a.k., borade, a. and metri, b (2014). a fuzzy ahp approach for supplier selection, operations and supply chain management, 7(2), 46 53. doi: 10.31387/oscm0170107 dursun, m., karsak, e. (2013). a qfd-based fuzzy mcdm approach for supplier selection, applied mathematical modelling, 37(8), 5864-5875. doi: https://doi.org/10.1016/j.apm.2012.11.014 ellram, l. (1990). the supplier selection decision in strategic partnerships, journal of purchasing material management, 26(4), 8–14. doi: https://doi.org/10.1111/j.1745493x.1990.tb00515.x ghorabaee, m.k., amiri, m., zavadskas, e.k., antucheviciene, j. (2017). supplier evaluation and selection in fuzzy environments: a review of madm approaches, https://doi.org/10.1111/j.1745-493x.1990.tb00515.x https://doi.org/10.1111/j.1745-493x.1990.tb00515.x ijahp article: rahman, biswas, halder/the application of ahp method for supplier selection of bangladesh’s jute industry: from the perspective of both public and private sectors international journal of the analytic hierarchy process 176 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.508 economic research-ekonomskaistraživanja, 30(1), 1073–1118. doi: https://doi.org/10.1080/1331677x.2017.1314828 gupta, a.k., singh, o., garg, r. (2015). analytic network process (anp): an approach for supplier selection in an automobile organization, european journal of advances in engineering and technology, 2(9), 83-89. gurung, s., phipon, r. (2016). multi-criteria decision making for supplier selection using ahp and topsis method, international journal of engineering inventions, 6, (2), 13-17. hu, h., xiong, h., you, y., yan, w. (2016). a mixed integer programming model for supplier selection and order allocation problem with fuzzy multiobjective, scientific programming, 2016, 1-13. doi: http://dx.doi.org/10.1155/2016/9346781 kazempoor, m., saeedhakaminasab, hemmatian, h. (2015). supplier selection using the ahp in jit production process, research journal of fisheries and hydrobiology, 10(11), 130-142. khan, m., jayant, a., kumar, v. (2015). multi-criteria supplier selection using fuzzy-ahp approach: a case study of manufacturing company, international journal of research in mechanical engineering & technology, 5(1), 73-79. khan, m.a., islam, s.a. (2014). comparison of suppliers and selecting the best supplier in supply chain management-a case study on jute industry, international journal of economic and business review, 2(3). kilic, h.s. (2012). supplier selection application based on a fuzzy multiple criteria decision making methodology, online academic journal of information technology, 3(8), 7-18. doi: 10.5824/1309‐1581.2012.3.001.x koul, s. and verma, r. (2011). dynamic vendor selection based on fuzzy ahp, journal of manufacturing technology management, 22(8), 963-971. doi: https://doi.org/10.1108/17410381111177421 liu, f., hai, h.l. (2005). the voting analytic hierarchy process method for selecting supplier, international journal of production economics, 97(3), 308-317. doi: https://doi.org/10.1016/j.ijpe.2004.09.005 rajesh, g., malliga, p. (2013). supplier selection based on ahp qfd methodology. procedia engineering, 64, 1283-1292, 2013. doi: https://doi.org/10.1016/j.proeng.2013.09.209 ramlan, r. and qiang, l.w. (2014). an analytic hierarchy process approach for supplier selection: a case study, international conference on global optimization and its application. saaty, t.l. (1980). the analytic hierarchy process. mcgraw-hill, new york, usa. sadeghi, m., rashidzadeh, m.a., soukhakian, m.a. (2012). using analytic network process in a group decision-making for supplier selection, informatica, 23(4), 621643. https://doi.org/10.1080/1331677x.2017.1314828 ijahp article: rahman, biswas, halder/the application of ahp method for supplier selection of bangladesh’s jute industry: from the perspective of both public and private sectors international journal of the analytic hierarchy process 177 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.508 sanayei, a., mousavi, s.f., yazdankhah, a. (2010). group decision making process for supplier selection with vikor under fuzzy environment, expert systems with applications, 37(1), 24-30. doi: https://doi.org/10.1016/j.eswa.2009.04.063 sasi, j.c., digalwar, a.k. (2015). application of ahp and topsis method for supplier selection between india & china in textile industry, international research journal of engineering and technology, 2(4), 1730-1738. shakil, m., ullah, m.r., and lutfi, m. (2013). process flow chart and factor analysis in production of a jute mills, journal of industrial and intelligent information, 1(4), 247-254. doi: 10.12720/jiii.1.4.247-254 singh verma, d., pateriya, a. (2013). supplier selection through analytical hierarchy process: a case study in small scale manufacturing organization, international journal of engineering trends and technology, 4(5), 1428-1433. stamm c.l., golhar, d.y. (1993). jit purchasing attribute classification and literature review, production planning & control, 4(3), 273–282. doi: https://doi.org/10.1080/09537289308919445 tahriri, f. osman, m.r., ali, a., yusuff, r.m. and esfandiary, a. (2008). ahp approach for supplier evaluation and selection in a steel manufacturing company, journal of industrial and management, 1(2), 54 – 76. doi: http://dx.doi.org/10.3926/jiem..v1n2.p54-76 tahriri, f.. dabbagh, m., ebrahim, n.a. (2014). supplier assessment and selection using fuzzy analytic hierarchy process in a steel manufacturing company, journal of scientific research and reports, 3(10), 1319-1338. doi: 10.9734/jsrr/2014/8627 tas, a. (2012). a fuzzy ahp approach for selecting a global supplier in pharmaceutical industry, african journal of business management, 6(14), 50735084. tookey, j.e., thiruchelvam, s. (2011). evolving trends of supplier selection criteria and methods, international journal of automotive and mechanical engineering (ijame), 4, 437-454. doi: http://dx.doi.org/10.15282/ijame.4.2011.6.0036 vaidya, o.s., kumar, s. (2006). analytic hierarchy process: an overview of applications, european journal of operational research, 169(1), 1-29. doi: https://doi.org/10.1016/j.ejor.2004.04.028 weber, c.a., current, j.r. and benton, w.c. (1991). vendor selection criteria and methods, european journal of operational research, 50(1), 2-18. doi: https://doi.org/10.1016/0377-2217(91)90033-r yusuff, r.m., yee, k.p., hashmi, m. (2001). a preliminary study on the potential use of the analytical hierarchical process (ahp) to predict advanced manufacturing technology (amt) implementation, robotics and computer-integrated manufacturing, 17(5), 421-427. doi: https://doi.org/10.1016/s0736-5845(01)00016-3 https://doi.org/10.1080/09537289308919445 http://dx.doi.org/10.15282/ijame.4.2011.6.0036 4_kopytov_dec_23_129-147-1 ijahp article: kopytov e., demidovs v., petukhova, n./ the analytic hierarchy process applied to development of train schedule information systems international journal of the analytic hierarchy process 130 vol. 3 issue 2 2011 issn 1936-6744 130 a p p l i c a t i o n o f t h e a n a l y t i c h i e r a r c h y p r o c e s s i n d e v e l o p m e n t o f t r a i n s c h e d u l e i n f o r m a t i o n s y s t e m s eugene kopytov transport and telecommunication institute riga, latvia e-mail: kopitov@tsi.lv vasilijs demidovs, natalia petukhova state join-stock company “latvian railway” & transport and telecommunication institute riga, latvia e-mail: dem@ldz.lv, natalia@ldz.lv abstract this paper considers different choices for the optimal data model of train schedule presentation. the authors have suggested three possible models that differ in building principles, of presenting temporal data of the train schedule in information systems. three popular multiple-criteria decision making methods were examined in order to choose the best model. the study presents the analytic hierarchy process (ahp) as the most suitable one for comparative evaluation of different data presentation models of the train schedule. in the study, thirteen evaluating criteria are developed which are distributed in three groups: hardware, maintenance and performance. the research is carried out for four different classes of is: web-based schedule systems, mobile schedule systems, ticket sales systems and rail traffic management systems. ms excel 2007 was used to display the ahp method; however a visualization tool called conditional formatting has been used to present the most important criteria and the preferred alternatives. keywords: multi-criteria analysis, analytic hierarchy process, pairwise comparison, data model, train schedule, results visualization 1. introduction the principal issues of train traffic schedule storage in databases of the railway is are related to the existence of a multitude of versions conditioned by seasonal cycles of schedule changes and the days of week, systematic and unplanned repair operations, and transfers of working and festive days. the basic problem is the complexity of the data reliability provided, and the immediacy of amending the schedule. frequent alternations make introducing these tasks extremely urgent. building a flexible schedule system taking into account the above limitations and rules, and giving the employees of the railway companies effective mechanisms of controlling the schedule would more successfully meet passenger transportations demands. it is a rob typewritten text http://dx.doi.org/10.13033/ijahp.v3i2.113 ijahp article: kopytov e., demidovs v., petukhova, n./ the analytic hierarchy process applied to development of train schedule information systems international journal of the analytic hierarchy process 131 vol. 3 issue 2 2011 issn 1936-6744 131 challenging scientific-technical task to develop a model of presenting the train traffic schedule data, a method of interaction with the stored data, and a method of searching an actual schedule version for a particular date (kopytov, demidovs, and petukhova, 2008). the efficiency of manipulating the schedule greatly depends on the organization of temporal data in the database. here, various models of presenting temporal data of the train schedule are used. the multi-criteria comparative analysis is used to determine the best model in terms of the given criteria efficiency. therefore, the author’s paper (kopytov, petukhova, and demidovs, 2010) contains the development of a system of quantitative and qualitative criteria; each of the criteria has been comparatively estimated by using two investigated data presentation models of the train schedule. nevertheless, these estimations are insufficient to choose the most suitable model in the case where there is no consideration of criteria significance. it has been assumed that it is enough for an analyst to have the properties of models according to different criteria to choose the optimal one. an absence of strict methodology and criteria ranking in this process of decision making makes it too complicated and its results unreliable. an increase of the number of alternatives further complicates the decision making task. this paper considers several choices for the optimal data presentation model of the schedule. the authors choose among three alternatives and employ the more formalised methods of quantitative and qualitative criteria analysis. there are currently various methods that have been developed and implemented to analyse and choose from a range of alternatives. these methods include multiplecriteria decision making (mcdm), multiple-criteria decision analysis (mcda), and multiple attribute decision making (madm). the existence of this variety of methods makes the issue of choosing the most suitable one rather difficult (triantaphyllou, 2000; lootsma, schuijt, 1995; olson, fliendner, and currie, 1995). the authors have analysed the possibility of employing three of the most popular methods to solve the problem of choosing the best data presentation model of the train schedule. these three methods include the simple additive weighting (saw) method (hwang, yoon 1981), the analytic hierarchy process (ahp) method (saaty, 1980; saaty, 2001) and collections of electre methods (roy, 1996; figueira, mousseau, and roy, 2005). when implementing the electre and saw methods, the authors faced the problem of arranging the alternatives in the criteria table (assigning the weights). the use of a large number of criteria, belonging to different professional knowledge areas resulted in an inadequate estimation of each criterion’s significance. with the help of invited experts, the authors were only able to competently evaluate certain criteria which they know well. the estimations of other criteria have been arrived at by guess-work. since the assigned weights of criteria have a great impact on the alternative choice, the authors have come to the conclusion that this method would result in largely inaccurate results. the ahp seems to be a more attractive choice in this context since it allows structuring the choice procedure as a hierarchy of several levels. it allows the distribution of (cluster) the criteria by several groups, and evaluates the significance of each group’s components. consequently, the different groups of criteria have been evaluated by different qualified experts. for instance, the technical specialists have ijahp article: kopytov e., demidovs v., petukhova, n./ the analytic hierarchy process applied to development of train schedule information systems international journal of the analytic hierarchy process 132 vol. 3 issue 2 2011 issn 1936-6744 132 assessed the technical criteria; the experts from the supporting service have evaluated the operational criteria, while the users have estimated the functional criteria. the opportunity of the pairwise comparison of a smaller number of criteria in every group allows the experts to determine better weighted (more accurate) values according to these criteria. the authors have concluded that the optimal number of criteria in each group should be from 3 to 5. the estimation of the significance of the criteria groups was determined by the experts with greater qualification. the ahp method also allows the possibility of controlling the consistency of the experts’ judgements, making it possible to increase the reliability of estimation. in summary, the multi-criteria analysis determined the ahp as the most suitable method for comparative evaluation of different data presentation models of the train schedule. 2. peculiarities of the trains schedule system generally the train schedule has several versions over a long period of time. in the course of time the various versions rotate (see figure 1). the active schedule of the train’s traffic is determined by a core fundamental (seasonal) timetable and the number of other schedules. the other schedules occur as a result of issues such as a train’s removal, changes in the traffic schedule due to the accidents, pre-planned or unplanned repair operations, public events (the city days, concerts, etc.), and shortrun changes of the route. figure 1 multitude of train schedule versions the schedule of passenger trains is considered is this research. the system of the passenger trains schedule possesses different peculiarities, determining a rather significant level of the complexity of its implementation which is considered below. in practice, the train may simultaneously have several valid schedules for a long period, and one particular train might have different schedules on different days of the week. in this situation, every schedule is fixed taking into account the days of traffic, and different periodicity characteristics, such as weekdays, rest-days, particular rest-days, the first weekday, the last day of the weekend (which sometimes may be monday), or even and odd days ( kopytov, petukhova, and demidovs, 2010). v1 v2 v3 t1 now train schedule versions v1 v2 v3 valid time 21 2. current train schedule 1. old schedule 3.future train schedule 3 ijahp article: kopytov e., demidovs v., petukhova, n./ the analytic hierarchy process applied to development of train schedule information systems international journal of the analytic hierarchy process 133 vol. 3 issue 2 2011 issn 1936-6744 133 let us take as the example of a multi-version train schedule presented in figure 2. as we can see, for the train with number n there are two basic schedules, one for the weekdays (wd), and the second for the rest days (rd). both schedules are actual in the time period [t1, t4], but in different days of the week. then, for repair works we make a change in the schedule of the given train on tuesdays and thursdays for the time period from t2 to t3. thus, in the time period [t2, t3] three actual schedule versions exist that are valid for the days, which correspond to the characteristics wd, rd and tue, thu, with only one schedule version wd or rd, or tue, thu valid for one day. figure 2 variety of the train schedule valid versions: wd, rd and tue, thu we shall illustrate the former statement in figure 3, in which the axis “day of week” shows the days of the week in the following sequence: 1 – monday, 2 – tuesday, etc. the granularity level of this schedule equals one day; therefore, we can state that time in our system has a discrete character and the versions possess the property of periodicity. as we can see in figure 3, on different days one version from the three is the actual train schedule, and the other two are in the “shadow zone”. here, the train schedule for tuesdays and thursdays (characteristic tue, thu) overlaps the weekday schedule with the characteristic wd. figure 3 periodicity in the train schedule on holidays and other special days (i.e. nato summit in riga in november 2006) all the trains run in accordance with the day off schedule. there is an exception for certain trains when the special timetable is assigned for these days. however, in reality only 1% of all trains are assigned with the special schedule. such types of changes in the timetable or special schedule assignments provoke the anomalies (paradoxes). for example, the 19th of november, monday, in 2007 was announced as ijahp article: kopytov e., demidovs v., petukhova, n./ the analytic hierarchy process applied to development of train schedule information systems international journal of the analytic hierarchy process 134 vol. 3 issue 2 2011 issn 1936-6744 134 an additional day off, and all the trains ran on the weekend schedule (like on the 18th of november), this event caused the malfunction of the periodicity as shown in figure 4. it is important to add that several trains with periodicity (sundays) were cancelled on the 18th of november, because the trivial sunday is not only the day off, but also the last day of weekend, and the additional sunday’s trains are considered for the passengers transportation from the traditional locations of the resting facilities on the days off. figure 4 periodicity malfunction in the train schedule one of the serious problems occurring in the considered is of the railway schedule is connected with the overlapping of the objects’ lifespans, when one schedule overlaps another (see figure 5). the variable tn means the moment of reviewing. the railway schedule for the season is fixed in february for the period from the moment of time t1 to t6. later in march, the schedule is changed for the period from the moment t2 to t4, and then in april, one more change is made for the period from t3 to t5. for the inquiry of the train traffic at the moment tn1, we’ll get the schedule fixed in february for [t1, t6], at the moment tn2 we have the schedule fixed in april for [t3, t5], and at the moment tn3 we again have the schedule for [t1, t6]. usually, overlapping schedules occur due to operative updates in a schedule. the peculiarity of the given situation is that when the object is changed it cannot lose the actuality of its state either in the past or in the future; –it is substituted by another version only for a certain period. as a result, more than one actual tuple describing different versions of the same object’s property can exist at one time. the latter contradicts the very idea of temporal database (tansel, clifford, and gadia, 1993; date, darwen, and lorentzos, 2002; jensen, 2000), therefore it is necessary to make special data queries in order to trace and settle such situations (kopytov, demidovs, and petukhova, 2008). ijahp article: kopytov e., demidovs v., petukhova, n./ the analytic hierarchy process applied to development of train schedule information systems international journal of the analytic hierarchy process 135 vol. 3 issue 2 2011 issn 1936-6744 135 figure 5 crossing of object lifespans in db of the train timetable system the schedule changes dynamically; an updated schedule action’s duration (operative changes) may be modified, and schedule versions may be cancelled. due to these and other reasons connected with data integrity support, data storage and display laconism, version control and analysis convenience, it is preferable to arrange a schedule’s temporal data in overlapping versions. there are also peculiarities connected with the schedule data management (schedule registration). first, the system obtains the core fundamental schedule, and then with the time running, the operative changes are made to this schedule. consequently, several versions of the schedule appear. quite often the period of operation of such an altered schedule is so prolonged that it also becomes subjected to certain corrections. the operational time of the corrections can be contracted or prolonged. the data of the train schedule versions include the concise information about the order, in accordance with which the new schedule is introduced, the date of this schedule version comes into operation, the date this schedule version goes out of operation, the type and the number of carriages in the train, the set of stops at the stations with information on the time of arrival or departure, and the type of the transport at the stations (in case of certain repair works the train between the definite stations can be exchanged by the bus). the train schedule system is heavily utilized by the customers. the search for the corresponding train takes place under different conditions and criteria consideration: the station of departure and arrival, the date and the time range of the departure, the possibility of travel in transit, and so on. in practice, this task is complicated by the large volume of the processed data. in the relation database one version of the timetable for one train takes about 1 kb. the takes into account the minimum amount of data without considering the reference information (such as the stations names, trains types, etc.). the integrated volume of all data of the latvian railway passenger trains schedules takes more than 100 mb (about 250 thousand records). 3. models of presenting temporal data of the train schedule in the information system the principal task of the train schedule information system is the storage of the information on train traffic in the past, present and future, as well as provision of ijahp article: kopytov e., demidovs v., petukhova, n./ the analytic hierarchy process applied to development of train schedule information systems international journal of the analytic hierarchy process 136 vol. 3 issue 2 2011 issn 1936-6744 136 access to this information for the system’s users. the efficiency of manipulating the train schedule greatly depends on the organization of temporal data in the database. two possible approaches of temporal data presentation are used: point and interval forms. point or daily form of schedule data presentation is convenient for processing operative and analytical queries, but requires huge memory resources and great labour consumption to maintain the train’s traffic schedule in its actual state. the main problem of this approach lies not in the volume of the stored data, but in the complexity of providing reliable data and the immediacy of amending the schedule. an example of data presentation in point form is shown in table 1. from 1st march until 31st may train number 4387 has a stop at station riga on the working days at 17 o'clock 00 minutes, and on the days off at 17 o'clock 20 minutes. however, from 12th march until 18th may on tuesdays and thursdays it stops at 16 o'clock 59 minutes. in the given example 92 records were entered for 3 months to define the stopping time for one train at one station. table 1 train schedule: data presentation in point form train id station time date … … … … 4387 riga 17:00 03.03.2011 thu 4387 riga 17:00 04.03.2011 fri 4387 riga 17:20 05.03.2011 day-off 4387 riga 17:20 06.03.2011 day-off 4387 riga 17:00 07.03.2011 mon 4387 riga 17:00 08.03.2011 tue … … … … 4387 riga 16:59 14.04.2011 thu 4387 riga 17:00 15.04.2011 fri 4387 riga 17:20 16.04.2011 day-off 4387 riga 17:20 17.04.2011 day-off 4387 riga 17:00 18.04.2011 mon 4387 riga 16:59 19.04.2011 tue … … … … since the schedule is usually made for a long period of time (i.e. an entire season), it is more convenient to use the interval form of data presentation. this form allows substantial reduction of the volume of stored data, and provides flexibility of schedule management. the interval form allows the use of the data of the schedule’s operative temporal database, and accounts for all changes in the schedule as soon as they appear in the system. however, the interval form is inferior to the point form in processing schedule queries and processing analytical queries in particular. for the same example, it is necessary to enter only 3 records shown in table 2, where ],[ es tt is time interval; st and et are the beginning and ending time of the event. ijahp article: kopytov e., demidovs v., petukhova, n./ the analytic hierarchy process applied to development of train schedule information systems international journal of the analytic hierarchy process 137 vol. 3 issue 2 2011 issn 1936-6744 137 in practice a variety of data presentation models exist based on point or interval forms. the authors have suggested three possible models of presenting temporal data of the train schedule: the model of the daily schedule presentation (data presentation in point form), the model based on logical rules, and the model based on temporal elements (data presentation in interval form), these differ in the building principles, the capabilities, and the requirements for the resources etc. in the suggested models two basic sets are introduced for the train schedule description: the set of all stations of the railways },...,,{ 21 lssss = , and the set of the all trains },...,,{ 21 mnnnn = , where l and m are the capacities of the sets s and n correspondingly. let us consider these models in detail. table 2 train schedule: data presentation in interval form train id station time ts te periodicity 4387 riga 17:00 01.03.2011 31.05.2011 work days 4387 riga 17:20 01.03.2011 31.05.2011 days-off 4387 riga 16:59 12.04.2011 18.05.2011 tue, thu the duplicating model is the model of the daily schedule presentation (point form). the given model is based on making a calendar of train traffic for every day it . then i-th schedule version )(niν for the train with number nn∈ will be determined by the tuple: },,,...,,,,,,{ )()()()( 〉〈〉〈〉〈= h n h2 n 21 n 1i n i tstststnν ,,...,2,1 ki = (1) where the pair 〉〈 j n j ts , )( determines the j-th stop of train with number n, the station ss nj ∈ )( exactly, and the train departure time jt ; h is the number of the stops of the train running on the timetable )(niν ; k is the number of days on which the schedule of n-th train is stored in database. it should be noted that the number of stored records (stating on what day, at what time, at what station every train stops) will be approximately equal to the product of multiplication of the number of trains by the average number stops and by the average number of days for which the schedule is stored. for example, using this approach for storing the data of the latvian railway train traffic for a period of one year 2 million records will be needed. . for larger railway companies, the volume of the corresponding database could increase ten times. in order to store schedules for the period of several years, the number of records will increase to the hundreds of millions. as stated previously, the main problem of this approach lies not in the volume of the stored data, but in the complexity of providing reliable data, and the immediacy of amending the schedule. the model lr is the model based on logical rules (interval form). this model is used to calculate a version of the active schedule based on logical rules that take into account all the specificities of the schedule: multi-versioning caused by the operative changes, multi-variance in case of a periodical schedule, and replacements and movements. for the specific schedule version identification for the train with number ijahp article: kopytov e., demidovs v., petukhova, n./ the analytic hierarchy process applied to development of train schedule information systems international journal of the analytic hierarchy process 138 vol. 3 issue 2 2011 issn 1936-6744 138 nn∈ the tuple 〉〈 esf tttcn ,,,, is employed. the periodicity property c presents the logical expression consisting of one or several elementary characteristics of the periodicity mp connected by the logical operations signs ∧∨, and ¬ . then i-th schedule version )(n iν for the train with number nn∈ will be },,,...,,,,,,,,{ )()()()( 〉〈〉〈〉〈〉〈= h n h2 n 21 n 1esf n i tstststttcnν i = 1, 2,..., q , (2) where q is the number of i-th train schedule versions, it is significant that k >> q. the model works with the data of the schedule’s operative temporal database, and accounts for all changes in the schedule as soon as they appear in the system. the central parameter of an access method, relative to which all the algorithms’ logical rules pertain, is the day in which an active train schedule version is needed. however, as mentioned above, the schedule system’s calendar is subject to changes which appear as exception declarations and replacement assignments. the model te is the model based on temporal elements (interval form with reduction to point form) (terenziani, 2003). the peculiarity of this model of determining the active version of the temporal object lies in the preliminary reckoning of the dates of activity for every schedule version, and saving it in the temporal elements for the posterior employment. the timetable version )(niν of the train with number nn∈ can be determined with the temporal element: },,,...,,,,,{ )()()()( 〉〈〉〈〉〈〉〈= h n h2 n 21 n 1i n i tstststenν i = 1, 2,..., q , (3) where the temporal element ite determines all the dates when the version )(n iν is active. the temporal element represents the calculable set of dates, and represents the time determination when the exact version of the train schedule on the extent of its life cycle becomes active. the temporal element comprises all information on the periodicity of the timetable, taking into consideration all operative changes, overlapping versions and special calendars (kopytov, petukhova, and demidovs, 2010). one of the most serious problems of use of temporal elements is the maintenance of the temporal elements in the consistent condition because when the schedule changes the versions temporal elements might become outdated and need to be recalculated. the presence of various models of the data presentation is caused by the existence of a set of different systems using the train schedule, which are characterized by their functionality, resource requirements and traditions of the development. this paper gives consideration to four classes of systems which are described in section 6. 4. hierarchy of the criteria for evaluating the models of presenting temporal data of the train schedule in an information system the search for an optimal model of data organization for a particular is must be performed taking into account the different criteria determining the efficiency of the ijahp article: kopytov e., demidovs v., petukhova, n./ the analytic hierarchy process applied to development of train schedule information systems international journal of the analytic hierarchy process 139 vol. 3 issue 2 2011 issn 1936-6744 139 schedule system on the whole. an initially formulated criteria system (kopytov, petukhova, and demidovs, 2010) has been developed, considering the peculiarities of the choice of data presentation models of the trains schedule in order to make the criteria more universal and understandable. in the process of the criteria system formation the authors have been focused on implementing the models in different railway schedule is ranging from minor systems for mobile facilities (i.e. mobile phones) to global systems of the ticket selling and operating on the most powerful mainframes. this choice has been made taking into consideration the requirements of various categories of is users, including developers, supporting specialists, database administrators, operators, cashiers and passengers. for example, operators are interested in the simplicity of the data input, and developers are interested in supporting the data integrity and necessity of creation of the data preliminary preparation procedure. supporting specialists are concerned about monitoring the timely and correct functioning of this procedure, and the passenger’s primary interest is schedule topicality. this research offers a system of criteria. the criteria are distributed in three groups: hardware, maintenance and performance. the hierarchical structure of the criteria is shown in figure 6. hardware refers to the importance of the effective use of the allocated calculation resources, such as cpu, hdd, and ram. maintenance refers to the cost of the system’s daily exploitation connected with entering a season schedule, random changes, additional data processing, integrity control and the difficulty of adjusting to new rules. performance refers to the requirement to efficiently perform typical tasks of the schedule system, such as searching actual train traffic for a given day, making schedules for short and long time spans, searching activity intervals of a particular version of the schedule, and searching for the nearest schedule change. to perform the calculations of criteria, the authors have used standard algorithms of the ahp method with the commonly used pairwise comparison scale from 1 to 9 (saaty, 2001). this article gives the evaluations of the pair wise comparisons of criteria, and the summary results of the calculations presented in section 7. figure 6 hierarchy of the criteria for evaluating the considered models ijahp article: kopytov e., demidovs v., petukhova, n./ the analytic hierarchy process applied to development of train schedule information systems international journal of the analytic hierarchy process 140 vol. 3 issue 2 2011 issn 1936-6744 140 5. visualization of the evaluations’ results with a large number of criteria and various alternatives, the process of evaluation can be long and tedious. in order to facilitate this process, the authors suggest using a means of visualization (colour and symbolic) which allows for greater comprehension and verification of the results at every step of the evaluation. verification of the numerical data, each of which consists of several numerals, allows the expert to quickly get an accurate picture of the data. this method also helps “to see” which alternative has become the “leader”. in addition to the fact that all calculations have been performed in the ms excel 2007, conditional formatting has been used for clearer presentation. gradient colours, data bars, and icon sets have been used to visually emphasize the most important criteria and to show the preferred alternatives (figure 7). two variants of gradient colour are useful: differentiation in the saturation of one colour, and differentiation in the colour gamma. colour visualization is not always acceptable for different reasons, such as the availability of only black and white print or special feelings the users might associate with different colours. symbolic (sign) indication is more universal than colour visualization, and therefore it has been used in this article (table 4 – table 7). note that with the evaluations change, the indication changes automatically. in section 7, the following icons have been used for the symbolic indication of the criteria influence: , indicating the degree of influence from the maximum value to the minimum value. for the indication of the global priorities alternatives, three icons have been used: for the alternatives having high priority, for the alternatives with middle priority, for the alternatives with low priority (figure 8). figure 7 conditional formatting menu in ms excel 2007 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 5 3 1 ijahp article: kopytov e., demidovs v., petukhova, n./ the analytic hierarchy process applied to development of train schedule information systems international journal of the analytic hierarchy process 141 vol. 3 issue 2 2011 issn 1936-6744 141 figure 8 conditional formatting customization (icon set) 6. systems of the train traffic schedule the train traffic schedule is used in different information systems (is). according to the functional and operational characteristics, we will distinguish four classes of systems: web-based schedule systems, mobile schedule systems, ticket sales systems and rail traffic management systems. web-based schedule systems provide information about the train traffic schedule for passengers using the internet. these systems must have sufficiently broad information to satisfy most of the users’ possible inquiries. these might include information about convenient changes, travel time, or a schedule for a given day, week, month, etc. the user’s internet-browser is only a terminal for the interaction with the schedule system. the storage of data and all the calculation processes take place at the specially designed servers. mobile schedule systems support the users of mobile devices such as mobile phones, and the portable terminal. these systems are characterized by the autonomy of functioning. in contrast to the previous class systems, all calculation processes and data storage are performed in the mobile device, but the periodically updated schedule is downloaded from a remote server. due to the limited technical capabilities of most current mobile devices the efficiency of the available resources is questionable. the systems are characterized by a limited set of available schedule inquiries, which are quite simple (i.e. search of the trains’ departure from one station to another, presentation of the train’s route for the given section with the indication of the stops’ time). ticket sales systems are designed for selling tickets, and the systems are highly specialized. their main task is to search the actual train schedule for a given day. the most essential requirement for these class systems is on-line entry of changes in the schedule. rail traffic management systems are necessary for planning the train traffic. the users of such systems are the railway specialists. the systems are characterized by broad functionality because of the necessity of sorting out various planning tasks and providing results in different forms. in such systems there are issues of coordinating ijahp article: kopytov e., demidovs v., petukhova, n./ the analytic hierarchy process applied to development of train schedule information systems international journal of the analytic hierarchy process 142 vol. 3 issue 2 2011 issn 1936-6744 142 the railway traffic with other means of transport schedules, and issues of allocating time-windows in the train traffic for repair works, the rail traffic management system requires particular accuracy and reliability of data because of the high level of train traffic security that must be provided. these systems also support planning for the locomotive crews, and conductor’s performance, and processing of the locomotivedrivers itinerary. 7. practical application of the ahp method for choosing the model of schedule data presentation as discussed earlier the ahp method has been chosen for evaluating the efficiency of suggested data models for the train traffic schedule presentation in different is. the chosen criteria for the models efficiency are described in section 5. the research carried out for the four different classes of is is discussed in section 6. the summary data of the pairwise comparisons for the criteria of the first hierarchy level for each group of investigated systems, supplemented with symbolic indication, are presented in table 3. the importance of the criteria is evident from the evaluation of the criteria priority vector for different systems. maintenance and performance criteria with identical values 0,467 of priority vectors are more important for the webbased schedule system. hardware criteria with value 0,761 are more important for mobile schedule systems, and performance criteria with value 0,559 are more important for ticket sales systems. maintenance criteria with value 0,709 are more important for rail traffic management systems. we have calculated the matrices of the evaluations of the priority vector for the suggested models for every group of systems based on the evaluation of the criteria priority vector of two levels of the hierarchy. table 4 gives an example of the results of calculating the priorities of the second level criteria “maintenance” for selecting an optimal model of data presentation of the train schedule in a web-based schedule system. the evaluations of the vector of the global alternatives priorities are shown above in table 5. it is important to note that table 4 is supplemented with symbolic indication, but table 5 is supplemented with colour and symbolic indications. similar calculations were made for the mobile schedule, ticket sales and rail traffic management systems. table 6 gives a normalised evaluation of the priority vector of the second hierarchy level for every type of system. the results of the evaluations show substantial differences in the considered criteria priorities in the systems of different classes. ijahp article: kopytov e., demidovs v., petukhova, n./ the analytic hierarchy process applied to development of train schedule information systems international journal of the analytic hierarchy process 143 vol. 3 issue 2 2011 issn 1936-6744 143 table 3 paired comparison matrices first hierarchy level with respect to the goal for different train traffic schedule systems maintenance performance hardware priority vectors web-based schedule system maintenance 1 1 7 0,467 performance 1 1 7 0,467 hardware 1/7 1/7 1 0,066 mobile schedule system maintenance 1 1/2 1/9 0,082 performance 2 1 1/5 0,157 hardware 9 5 1 0,761 ticket sales system maintenance 1 1/2 7 0,371 performance 2 1 6 0,559 hardware 1/7 1/6 1 0,070 rail traffic management system maintenance 1 4 9 0,709 performance 1/4 1 5 0,231 hardware 1/9 1/5 1 0,060 table 4 matrix of evaluations of the vector of the criteria priorities of the “maintenance” group for the web-based schedule system alternatives group priority: 0.467 priorities in group "maintenance" "maintenance" group's criteria entering a season schedule entering on-line changes necessity of additional data processing integrity control adjusting to new rules priority vector 0.334 0.367 0.084 0.166 0.049 model lr 0.444 0.429 0.444 0.615 0.087 0.449 model те 0.444 0.429 0.111 0.319 0.162 0.376 duplicating 0.111 0.143 0.444 0.066 0.751 0.175 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 51 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 5 5 5 5 1 5 5 5 1 3 1 3 1 1 5 1 5 1 ijahp article: kopytov e., demidovs v., petukhova, n./ the analytic hierarchy process applied to development of train schedule information systems international journal of the analytic hierarchy process 144 vol. 3 issue 2 2011 issn 1936-6744 144 table 5 evaluation results for web-based schedule system we calculated the final matrix of the evaluations of the global priority vector for the suggested models of organising the schedule data based on the evaluations of the criteria priority vector of two levels of hierarchy. (table 7), this may be used for choosing a model of data presentation in a particular schedule system. 0,467 0,467 0,067 e n te ri n g a s e a so n s ch e d u le e n te ri n g o n -l in e c h a n g e s n e ce ss ity o f a d d iti o n a l d a ta p ro ce ss in g in te g ri ty c o n tr o l a d ju st in g to n e w r u le s s e a rc h o f a n a ct u a l t ra in s ch e d u le fo r a g iv e n d a y m a ki n g s ch e d u le s fo r b ig s p a n s m a ki n g s ch e d u le s fo r sm a ll sp a n s s e a rc h o f a ct iv ity in te rv a ls o f a p a rt ic u la r ve rs io n s e a rc h o f t h e n e a re st s ch e d u le ch a n g e c p u h d d r a m 0,334 0,367 0,084 0,166 0,049 0,484 0,040 0,198 0,040 0,239 0,540 0,297 0,163 model lr 0,444 0,429 0,444 0,615 0,087 0,449 0,109 0,061 0,126 0,100 0,108 0,110 0,097 0,606 0,333 0,287 0,280 model те 0,444 0,429 0,111 0,319 0,162 0,376 0,345 0,353 0,416 0,800 0,789 0,483 0,570 0,333 0,333 0,461 0,432 duplicating model0,111 0,143 0,444 0,066 0,751 0,175 0,547 0,586 0,458 0,100 0,103 0,407 0,333 0,061 0,333 0,252 0,288 g lo ba l p ri or iti es p ri o ri tie s b y "h a rd w a re " g ro u p criteria of "performance" group p ri o ri tie s b y "p e rf o rm a n ce " g ro u p priorities vector priorities vector criteria of "hardware" group a lte rn at iv es p ri o ri tie s b y "m a in te n a n ce " g ro u p criteria of "maintenance" group priorities vector ijahp article: kopytov e., demidovs v., petukhova, n./ the analytic hierarchy process applied to development of train schedule information systems international journal of the analytic hierarchy process 145 vol. 3 issue 2 2011 issn 1936-6744 145 table 6 a normalised evaluation of the criteria priority vector for different schedule systems table 7 evaluations of the vector of the global criteria priorities for different schedule systems alternatives web-based schedule system mobile schedule system ticket sales system rail traffic management system model lr 0.280 0.353 0.220 0.359 model те 0.432 0.452 0.385 0.363 duplicating model 0.288 0.195 0.395 0.279 the ahp method allows the models to be arranged in the order of efficiency of their use in every system, and shows their difference in the given set of criteria. by performing an evaluation of the alternatives for selecting possible variants to be used by the ahp method reveals their universal character. thus, in the example the universal character of the те model has been shown byit’s advantage in the three systems under consideration. the te model is greater than the nearest models competitors for a web-based schedule system by 14,40%, for a mobile schedule system by 9,90% and for а rail traffic management system by 0,40%. only for the ticket sales system the te model yields the duplicating model by 1,00%. conclusions this article solves the issue of choosing the best data presentation model for the train schedule. this problem was formulated as the multiple-criteria decision making task, and a variety of methods has been developed for its solution. in the authors’ opinion, web-based schedule system mobile schedule system ticket sales system rail traffic management system entering a season schedule 0,334 0,437 0,334 0,070 entering on-line changes 0,367 0,364 0,334 0,238 necessity of additional data processing 0,084 0,074 0,026 0,027 integrity control 0,166 0,077 0,110 0,450 adjusting to new rules 0,049 0,049 0,196 0,215 search of an actual train schedule for a given day 0,484 0,574 0,692 0,556 making schedules for big spans 0,040 0,038 0,077 0,111 making schedules for small spans 0,197 0,183 0,077 0,111 search of activity intervals of a particular version 0,040 0,041 0,077 0,111 search of the nearest schedule change 0,239 0,164 0,077 0,111 cpu 0,540 0,333 0,413 0,260 hdd 0,297 0,333 0,260 0,413 ram 0,163 0,333 0,327 0,327 criteria systems m ai nt en an ce p er fo rm an ce h ar dw ar e 1 3 1 3 5 5 3 5 1 1 5 1 ijahp article: kopytov e., demidovs v., petukhova, n./ the analytic hierarchy process applied to development of train schedule information systems international journal of the analytic hierarchy process 146 vol. 3 issue 2 2011 issn 1936-6744 146 the methods of pairwise comparison are the most suitable for the examined problem solving. the employment of methods of this group has been considered in the process of investigation. three pairwise comparison methods saw, ahp and electre were examined. this study has demonstrated that a major drawback of the electre and saw methods is the complexity of the expert’s evaluation of criteria in different professional knowledge areas. only certain criteria which the experts know well were able to be evaluated competently. the estimations of other criteria have been chosen at random. the ahp seems to be the most attractive method in this context since it allows structuring the choice procedure as a hierarchy of several levels. it allows for the distribution of criteria into several groups and the evaluation of the significance of every group’s component. consequently, the different group’s criteria have been evaluated by different experts with proper qualification. it is necessary to mention other important advantages of the ahp method: • it does not require special software, and can be presented in ms excel; however, alteration of the criteria or the number of alternatives requires certain efforts to implement these changes; • the computable consistency of the judgments allows controlling the accuracy of estimation; • the algorithm of ahp operation and the table form of representation of the principal and intermediate results allows for a visual demonstration of the reason for choosing the certain alternative. with the use of the ahp method this research fulfils the evaluation of the efficiency of application of the three models of data presentation for the train traffic schedule in different information systems with the account of the specifics of their application. to determine an optimal method, a two-level hierarchy system of criteria has been developed with ranging expert evaluations according to the importance and preference for all models in each system under consideration. a consistency ratio has been used to verify the judgements in the criteria evaluation which for different criteria groups, have not exceeded 5,77% for the web-based schedule systems, 8,45% for the mobile schedule systems, 6,92% for ticket sales systems and 6,14% for rail traffic management systems. the ahp method has allowed the models to be arranged in order of efficency of their use for each of the systems, has shown their differences in the given set of criteria, and has allowed discovery of a universal model – model те. the ahp method has been presented in the ms excel 2007. the conditional formatting visualization tool has allowed visual presentation of the most important criteria and of the preferred alternatives. ijahp article: kopytov e., demidovs v., petukhova, n./ the analytic hierarchy process applied to development of train schedule information systems international journal of the analytic hierarchy process 147 vol. 3 issue 2 2011 issn 1936-6744 147 references date, c. j., darwen, h., & lorentzos, n.a. (2002). temporal data and the relational model: a detailed investigation into the application of interval and relation theory to the problem of temporal database management. morgan kaufmann. figueira, j., mousseau, v., & roy, b. (2005). electre methods. in j. figueira, s. greco, m. ehrgott (eds.), multiple criteria decision analysis: state of the art surveys, (pp133-162). springer’s international series, new york: springer science & business media, inc. hwang, c. l., & yoon, k. (1981). multiple attribute decision making: methods and applications. berlin, heidelberg, new-york: springer verlag. jensen, c. (2000). temporal database management. dr. techn. thesis, defended on 14.04.2000, available at: www.cs.auc.dk/~csj/thesis/ kopytov, e., demidovs, v., & petukhova, n. (2008). application of temporal elements in the railway schedule systems. transport and telecommunication, 9 (2), riga, latvia, 14–23. kopytov, e., demidovs, v., & petukhova, n. (2008). modelling of railway schedule in temporal databases. proceedings of the international conference modelling of business, industrial and transport system (mbits’08), riga, latvia, 107–116. kopytov, e., petukhova, n., & demidovs, v. (2010). methods for railway schedule periodicity support in temporal databases. proceedings of the 9th joint conference on knowledge-based software engineering (jckbse’10), kaunas, lithuania, technologija, 178–191. lootsma, f.a., & schuijt, h. (1997). the multiplicative ahp, smart and electre in a common context. journal of multi-criteria decision analysis, 6 (4), 185-196. olson, d.l., fliendner, g., & currie, k. (1995). comparison of the rembrandt system with analytic hierarchy process. european journal of operations research, 82 (3), 522–539. roy, в. (1996). multi-criteria methodology for decision aiding. dordrecht : kluwer academic publishing. saaty, t.l. (1980). the analytic hierarchy process: planning, priority setting, resource allocation. new york: mcgraw-hill. saaty, t.l. (2001). decision making for leaders: the analytic hierarchy process for decisions in a complex world. pittsburgh, pa: rws publications. tansel, a.u., clifford, j., & gadia, s. (1993). temporal databases: theory, design, and implementation. redwood city, ca: the benjamin/cummings publishing company. ijahp article: kopytov e., demidovs v., petukhova, n./ the analytic hierarchy process applied to development of train schedule information systems international journal of the analytic hierarchy process 148 vol. 3 issue 2 2011 issn 1936-6744 148 terenziani, p. (2003). symbolic user-defined periodicity in temporal relational databases. ieee transactions on knowledge and data engineering, 15 (2), 489-509. triantaphyllou, e. (2000). multi-criteria decision making methods: a comparative study. london: kluwer academic publishers. ijahp article: das a, das, s./supplier selection for a pump manufacturing organization by hybrid ahp-topsis technique and its impact on inventory international journal of the analytic hierarchy process 334 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.380 supplier selection for a pump manufacturing organization by hybrid ahp-topsis technique and its impact on inventory ankur das brainware group of institutions, india assistant professor of the department of mechanical engineering dasankur43@gmail.com santanu das kalyani govt. engineering college, india professor and head, dept. of mechanical engineering sdas.me@gmail.com abstract in today’s highly competitive market, organized supply chain management strategy is required by any manufacturing organization to ensure success in both a quantitative and qualitative manner. in order to obtain green manufacturing or a green supply chain management system, the suppliers should play an important role. green manufacturing has an impact on inventory also. this paper describes the supplier selection model using a hybrid ahp-topsis technique related to a pump manufacturing industry that intends to enhance green purchasing of inventory items. different criteria that have a good impact on inventory and the environment are considered for supplier selection. first, the analytic hierarchy process (ahp) is used to determine the weight of each criterion, and then the topsis algorithm is applied to optimally select the supplier. in this work, the ranking of suppliers has effectively chosen the selection of an appropriate supplier maintaining a green supply chain management (scm) system. keywords: green manufacturing; supply chain management; green scm; supplier selection; pump manufacturing industry; green inventory; ahp; topsis; hybrid ahptopsis 1. introduction a proper supply chain management system is required to fulfill the demands of a customer and to ensure profitability. companies in any supply chain should focus on inventory management along with management of production, transportation, etc. to obtain a green supply chain management system, environmentally sustainable inventory is necessary. a green supply chain deals with promotion of green or eco-friendly products in a supply chain. the suppliers who provide eco-friendly products may be termed as green suppliers. the selection of a green supplier may be an objective for purchasing the inventory items if environmental sustainability is of prime concern. besides environmental issues, product quality, procurement cost, lead time of the procured inventory items, etc. have immense importance in supplier selection. selection of wrong suppliers may hamper financial and operational conditions of the supply chain mailto:dasankur43@gmail.com mailto:sdas.me@gmail.com ijahp article: das a, das, s./supplier selection for a pump manufacturing organization by hybrid ahp-topsis technique and its impact on inventory international journal of the analytic hierarchy process 335 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.380 as well as the production. there are many different criteria related to available appropriate suppliers, and it often becomes difficult to select the best supplier to satisfy all the criteria optimally. the supplier selection can be done easily by different multicriteria decision making (mcdm) techniques. the mcdm techniques provide a way to choose the green supplier methodically. this paper aims at selection of a green supplier for a pump manufacturing organization. the selection process leads to purchasing of environmentally-friendly inventory items which confirm less waste generation, recyclability, reusability, low energy consumption criteria, etc. in this context, the ranks of the suppliers are determined by their green performance that deal with the utilization of minimum resources, and protection of the environment from pollution. 2. literature review decision making plays an important role in every perspective of a manufacturing organization. decision making becomes difficult when it deals with multiple criteria. a proper decision making technique is needed to solve problems like this. t.l. saaty introduced a multi-criteria decision making theory known as the analytic hierarchy process (ahp). a decision hierarchy was constructed in the ahp with a proper goal, criteria and alternatives. a pairwise comparison matrix was formed for different criteria. comparison between the alternatives, selection of scale, checking of consistency ratio and judgment of alternatives were done using an illustration. saaty (1977) analyzed the scaling technique for priorities in a hierarchical structure. the consistency of a pairwise comparison matrix was described for different priority levels. the comparison of different scales was discussed and the conversation was extended to the multi criteria decision making technique. finally, this decision making technique was incorporated for a large scale problem. saaty (1990) applied ahp method in a hierarchy structure to solve the multi criteria decision making problem. the priorities of different attributes, ranking and other measurements related to ahp were summarized through this research work. supplier selection requires a multi-criteria decision making approach to adopt in a quantitative as well as a qualitative manner. kahraman et al. (2003) used fuzzy-ahp technique to choose one of the best suppliers based on some criteria of a manufacturing industry. bayazit et al. (2006) introduced an ahp based extended analysis of a supplier selection problem for a turkish construction company. afterwards, a sensitivity analysis was also performed, and the best supplier was selected between two top suppliers. tahriri et al. (2008) described the ahp based multi-criteria decision making model for selecting the best supplier of a steel manufacturing company in malaysia using both qualitative and quantitative criteria. the research work introduced the optimal order quantities and lead time among different suppliers by a systematic execution of the ahp model. lee et al. (2009) used delphi technique to differentiate the traditional supplier and green supplier. ultimately, the selection of the best supplier was done in that work by fuzzy extended analytic hierarchy process (feahp).the sustainability of green production technology was improved by the supplier selection model also. umadevi et al. (2012) applied the ahp technique to select the right vendor for a manufacturing industry. different critical criteria were selected for the vendor selection model, and according to these criteria the best supplier was selected. it was considered to be a great strategic decision by the decision makers. lei et al. (2013) established the importance of corporate social ijahp article: das a, das, s./supplier selection for a pump manufacturing organization by hybrid ahp-topsis technique and its impact on inventory international journal of the analytic hierarchy process 336 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.380 responsibilities (csr) on supply chain. seven criteria and different sub-criteria were chosen to select the best supplier under csr environment. this paper concluded with the preferential judgment of different criteria by ahp technique. aouadni and rebai (2013) developed a mixed integer non-linear programming model for an inventory management decision and random supplier selection to solve supply uncertainty problems. the genetic algorithm (ga) was also used to solve the problem. supplier selection may be done by the group of decision makers to avoid criticality of the task. dragincic and vranesevic (2014) proposed a group decision making technique for supplier selection which was based on the ahp. the ahp based decision making approach was implemented for supplier selection of irrigation equipment. further, this approach was adapted in the field of water planning, management and development also. galankashi et al. (2015) measured the green performance of suppliers by nominal grouping technique (ngt) with respect to the critical criteria of supply chain. finally, weights of criteria were calculated by fuzzy analytical networking process (fanp). a hybrid supplier selection model by combining the ahp and multi-expression programming (mep) was introduced by fallahpour et al. (2015). a supplier selection problem of a textile company was solved by the model proposed. many researchers applied the ahp to solve mcdm problems. for example, sabiruddin et al. (2013) considered a gas metal arc welding process to derive an appropriate set of process parameters. experimental results were analyzed and optimized value of process parameters were selected using the ahp. choudhury et al. (2015) conducted several experiments on surface grinding to get the appropriate grinding condition. they applied the ahp successfully to evaluate the optimal grinding condition to obtain the best grinding performance. topsis method was also employed to find out solutions to a host of problems. wang and xin (2011) established a model of absolute analysis to evaluate the site of a thermal power plant. this model was solved by a multi-criteria decision making technique which is known as topsis (technique for order preference by similarity to ideal solution). different evaluating criteria were introduced to form the normalized weighted matrix. a suitable site was selected from the weighted data. huang (2012) developed an evaluation based model to solve the performance of an electric power supply bureau of chonging city, china. various data such as cost of power supply, volume of power supply, current assets, etc. were collected from eight electric power supply bureaus. the ranking was done by topsis method. zhu et al. (2012) described the quality credit issue on different organizations such as food enterprises, academic institutes, corporate, etc. different evaluation criteria were selected for the air conditioning market. the selection was done by topsis method. organ (2013) applied the topsis method to choose the best private teaching institution for high school students. ertugrul and oztas (2014) presented the fuzzy logic concept to select the most economic mobile line and to achieve the business needs. finally, the suitable mobile lines were selected by topsis method. the major issues in a supply chain like supplier selection problem can be solved by topsis method. eleren and yilmaz (2011) applied the topsis approach to select the best supplier for a textile firm in turkey. initially fuzzy matrix was constructed with preferential judgment of all the criteria against different suppliers. afterwards, topsis method was applied to get the optimum result. shahroudi et al. (2012) described the application of topsis method for supplier selection in the case of an auto supply chain. ijahp article: das a, das, s./supplier selection for a pump manufacturing organization by hybrid ahp-topsis technique and its impact on inventory international journal of the analytic hierarchy process 337 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.380 the supplier selection was done based on some effective criteria and alternatives. sharma and jayaswal (2015) proposed a methodology of supplier selection using topsis approach. different criteria were selected to ensure good coordination between management and suppliers of an automobile industry. weights were given to the criteria by the different experts. finally, ranks of suppliers were obtained or the best supplier was chosen with proper implementation of topsis methodology. considering the effectiveness of the ahp and topsis methods, the combination of these two have been applied by many with success. this method is basically a hybrid method. ghosh (2011) evaluated the faculty performance of an educational institution by multicriteria decision making technique. initially, different attributes of teachers were described. the ahp was applied to find the overall weight of teachers. after that, topsis was applied to rank the best teacher. bhutia and phipon (2012) applied the ahp and topsis method for a supplier selection problem. both the quantitative and qualitative analyses were done to select the best supplier among all the suppliers considered. jingfei et al. (2013) established the status evaluation index system of traffic crowding. the best traffic management system was selected by the ahp-topsis method. mansor et al. (2014) described the requirement of sustainable materials for an automotive parking brake lever component. the four basic criteria were introduced towards the selection of optimal hybrid composite materials for the automotive component. the selection process was done by integrated ahp-topsis technique. the integrated technique was enabled to implement the systematic comparison between the designers of composite materials also. 3. introduction to a case study for selecting green suppliers this paper enumerates the selection process of an appropriate supplier of a pump manufacturing organization which can ensure supply of environmentally-friendly inventory items in the least amount of time. the manufacturing unit is located in a crowded city of west bengal, india. axial flow pumps, mixed flow pumps, centrifugal pumps, submersible pumps, etc. are produced by the organization. these products are highly employed in different water supply projects. here, green supplier selection is necessary for environmental sustainability in the supply chain as well as in the production system. supplier selection criteria are based on green purchasing, i.e. purchasing of environmentally-friendly components, consumables, etc. of a pump manufacturing organization. in the case of the pump manufacturing organization, green purchasing of inventory items include: a) centrifugal pump housing made by cast iron; b) ball bearings implementing advanced lubrication technology; c) impeller made by poly phenylene oxide (ppo) plastic; d) rubber components made by recycled elastomers; e) vegetable oil, grease; f) motor and electrical items that consume less energy; g) pipes made by copper, cast iron, galvanized steel, cross-linked polyethylene tubing; h) wooden case packing, etc. that is environmentally friendly, reusable and recyclable. ijahp article: das a, das, s./supplier selection for a pump manufacturing organization by hybrid ahp-topsis technique and its impact on inventory international journal of the analytic hierarchy process 338 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.380 the slow rate of oxidation of cast iron materials reduces waste generation. the products made by cast iron are also recyclable. so, the centrifugal pump housing made of cast iron is environmentally-friendly and may be preferred. ball bearings lubricated by biodegradable vegetable oil or something similar are preferable and should be included in the green supply chain to promote an environmentally sustainable production system. poly phenylene oxide (ppo) plastic has different properties including corrosion resistance, low ph level, recyclability, reusability, etc. normally, the ppo plastic can be treated as a green component for impeller. different spare parts for horizontal and vertical pumps like the piston ram, pipe end rings, pump seals, relief valves, etc. made by elastomers can be procured as eco-friendly inventory items. the motors that accord higher efficiency are desirable for less energy consumption. rewinding of motors is done in order to avoid higher energy consumption. power consumption in the pump manufacturing industry can be minimized by using capacitors and sleep power recovery systems. the pipes and tubes in different dimensions are used in industrial pumps as well as domestic pumps. pipes and tubes can be made of such metals, plastics and other composite materials that ensure biodegradability, and should have corrosion resistance. moreover, the procured inventory items in a green supply chain need to be cost efficient in addition to having these green properties. table 1 description of criteria criterion criterion name c1 percentage consumption of eco-friendly raw material c2 percentage consumption of finished goods material c3 waste generation c4 energy consumption c5 recyclability of inventory items c6 reusability of inventory items c7 cost c8 lead time high percentage consumption of eco-friendly raw materials and finished goods is essential to assure the purpose of green purchasing. waste generation should be minimized to obtain a pollution free environment. low energy consumption, recyclability and reusability of inventory items consume minimum resources that are required to obtain a cost effective green production system. the major issues like minimum purchasing cost and shortest possible lead time of inventory items are taken into consideration to ensure increased productivity as well as minimum production cost. to implement a green supply chain management system, the suppliers should be assessed based on some attributes or criteria (given in table 1), which are related to environmental issues besides other criteria. these five suppliers are selected by the purchase manager and experts of the organization based on the procedure of tendering. different criteria are introduced to ensure the environmentally friendly and cost effective supply chain. the authors have chosen these criteria based on discussions with some experts of the organization. the description of different suppliers against the attributes is given in table 2. ijahp article: das a, das, s./supplier selection for a pump manufacturing organization by hybrid ahp-topsis technique and its impact on inventory international journal of the analytic hierarchy process 339 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.380 table 2 description of different criteria for different suppliers 4. hybrid ahp-topsis technique for evaluation of appropriate supplier supplier selection is a common issue in the research. the selection may be done by different multi-criteria decision making (mcdm) techniques, such as, analytic hierarchy process (ahp), analytical network process (anp), simple additive weights (saw) method, technique for order preference by similarity to ideal solution (topsis), elimination et choice translating reality (electre), preference ranking organization method (promethee), etc. in this work, a hybrid ahp-topsis model is applied for the selection process. 4.1 application of the ahp for evaluating weights of suppliers to select the green supplier for a pump manufacturing organization, eight different criteria are considered as listed in table 1. the priorities of different criteria are proposed by the authors after discussions with the decision makers of the company, and presented by the nine point scale of ahp. in this hierarchical structure, supplier m, n, o, p and q are chosen as the alternatives. performances of alternatives are analyzed with respect to eight criteria chosen. the hierarchy structure is shown in figure 1. let one consider matrix a. for a matrix, aw= λmaxw (1) where, w = (w1,w2,w3,..) t and λmax ≥ n, when λmax is the largest eigen value of the matrix a. supplier attribute m n o p q c1 high moderate some very low high c2 low high very low some moderate c3 high moderate high some very low c4 low moderate huge high moderate c5 very high recyclable items some recyclable items very less recyclable items some recyclable items almost no recyclable items c6 very high reusable items moderately reusable items low reusable items high reusable items very less reusable items c7 very high cost of inventory items very low raw material cost moderate raw material cost high cost of inventory items very low cost of inventory items c8 shortest possible time period moderate moderate very high very high and not having any exact time period ijahp article: das a, das, s./supplier selection for a pump manufacturing organization by hybrid ahp-topsis technique and its impact on inventory international journal of the analytic hierarchy process 340 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.380 on the other hand, for a consistent matrix, aw = nw. (2) for an inconsistent matrix, the degree of inconsistency is measured by consistency ratio (cr). cr = (λmax n) / (n 1) (3) in the present work, ajk represents the preference of jth criterion over kth criterion. aik represents the preference of ith alternative over kth alternative. figure1. hierarchy structure normalized priorities or priority vectors for a criteria and alternative matrix are determined by equations (4) and (5). wp= rij/ (4) where,rij = ; wherein is the number of criteria. wq = rij / (5) when,rij = ; where n is the number of alternatives. ijahp article: das a, das, s./supplier selection for a pump manufacturing organization by hybrid ahp-topsis technique and its impact on inventory international journal of the analytic hierarchy process 341 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.380 table 3 criteria matrix selection of supplier c1 c2 c3 c4 c5 c6 c7 c8 priority vector c1 1 1/3 1/4 6 4 4 3 5 0.196 c2 3 1 1/3 3 4 4 1/2 2 0.148 c3 4 3 1 4 3 6 6 5 0.265 c4 1/6 1/3 1/4 1 1/3 1/4 1/2 3 0.0484 c5 1/4 1/4 1/3 3 1 4 3 5 0.14 c6 1/4 1/4 1/6 4 ¼ 1 2 4 0.099 c7 1/3 2 1/6 2 1/3 1/2 1 3 0.077 c8 1/5 1/2 1/5 1/3 1/5 1/4 1/3 1 0.025 𝜆max = 9.779, cr = 0.02 the criteria matrix is formed to solve the supplier selection problem. table 3 shows the preferential comparison of different criteria. consumption of raw material (c1) is more preferred than finished goods (c2), as raw material is consumed in larger quantities than finished goods in the case of the pump manufacturing organization. less waste generation is most preferable for an environmentally friendly supply chain and inventory. energy consumption is a less preferable criterion than material consumption. different electrical items used in a pump manufacturing organization consume maximum energy in the case of manufacturing and test run of a pump. recyclability (c5) and reusability (c6) of inventory items are moderately important criteria compared with the eco-friendly material consumption as different parts of a pump are not recyclable and reusable. to obtain the environmentally friendly supply chain, the cost involved should be as low as possible, but it has moderate importance compared to the criteria of material consumption and less waste generation. in this context, lead time (c8) should be the least preferred criterion compared to the others. generally, optimum lead time is required for a supply chain. in the case of a pump manufacturing organization, less waste generation (c3) is highly desirable for improving green performance. hence, it is more preferable than the other criteria mentioned in table 1. with these considerations, priority vectors or priority weights of the criteria matrix are computed by equation 4. the maximum eigen value and consistency ratio of this matrix are calculated by equation 3. the desirable consistency ratio is always less than 10% which indicates acceptable consistency of the matrix. ijahp article: das a, das, s./supplier selection for a pump manufacturing organization by hybrid ahp-topsis technique and its impact on inventory international journal of the analytic hierarchy process 342 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.380 table 4 pair-wise comparison matrix for alternatives for criterion c1 c1 m n 0 p q normalized priorities m 1 3 4 6 2 0.353 n 1/3 1 1/5 2 1/6 0.081 o 1/4 5 1 3 1/2 0.215 p 1/6 ½ 1/3 1 1/4 0.049 q 1/2 6 2 4 1 0.298 𝜆max = 5.381 , cr = 0.085 table 4 shows the pair-wise comparison matrix related to different suppliers with respect to the criterion, percentage consumption of eco-friendly raw material (c1). supplier m supplies a high quantity of environmentally friendly raw material. supplier q is also capable enough to supply a high quantity of eco-friendly raw material, but of a relatively less quantity compared with supplier m. supplier q is less preferable with respect to this criterion. with respect to these judgments, the normalized priorities or weights are assigned to each element of this matrix. table 5 pair-wise comparison matrix for alternatives for criterion c2 c2 m n 0 p q normalized priorities m 1 1/5 2 1/3 1/2 0.0951 n 5 1 3 3 2 0.3302 o 1/2 1/3 1 1/5 1/6 0.0518 p 3 1/3 5 1 1/3 0.2279 q 2 1/2 6 3 1 0.2948 𝜆max = 5.419 , cr = 0.093 table 5 shows pair-wise comparison of different suppliers with respect to criterion c2, i.e. percentage consumption of eco-friendly finished goods material. supplier n has more preference for a supply of environmentally friendly finished goods than other suppliers. the supplier q is also more desirable for high consumption of the finished goods than supplier p, m and o respectively, but less desirable than supplier n. in the same way, a pair-wise comparison matrix has been constructed for criterion, c3 as shown in table 6. the amount of waste generation (c3) is quite less in the case of the items procured from supplier q. the procured items from supplier q are more desirable than that from other suppliers with respect to this criterion. the items from supplier n are desirable for low waste generation, but less desirable than supplier q. supplier m is the most undesirable of the other suppliers as the procured items from this supplier have quite a high waste generation capacity. ijahp article: das a, das, s./supplier selection for a pump manufacturing organization by hybrid ahp-topsis technique and its impact on inventory international journal of the analytic hierarchy process 343 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.380 table 6 pair-wise comparison matrix for alternatives for criterion c3 c3 m n 0 p q normalized priorities m 1 1/5 ½ 1/3 1/6 0.048 n 5 1 4 2 1/2 0.276 o 2 1/4 1 1/3 1/6 0.082 p 3 1/2 3 1 1/4 0.171 q 6 2 6 4 1 0.42 𝜆max = 5.106 , cr = 0.023 table 7 pair-wise comparison matrix for alternatives for criterion c4 c4 m n 0 p q normalized priorities m 1 3 7 2 3 0.361 n 1/3 1 6 2 4 0.3 o 1/7 1/6 1 1/3 1/4 0.042 p ½ ½ 3 1 2 0.157 q 1/3 ¼ 4 ½ 1 0.137 𝜆max = 5.261 , cr = 0.011 table 8 pair-wise comparison matrix for alternatives for criterion c5 c5 m n 0 p q normalized priorities m 1 4 6 4 7 0.403 n 1/4 1 3 3 8 0.279 o 1/6 1/3 1 ¼ 2 0.068 p 1/4 1/3 4 1 6 0.212 q 1/7 1/8 1/2 1/6 1 0.035 𝜆max = 5.39 , cr = 0.013 less energy consumption is a more preferable criterion in the case of green manufacturing as well as green supply chain management. supplier m is more accepted than other suppliers with respect to the criterion c4 as the incurred products from this supplier consume less energy. supplier n is less preferable than supplier m considering this attribute. supplier p and q are moderately preferred, but supplier o is very less preferred with respect to this criterion. the comparison of different suppliers, which is analyzed based on criterion c4, is represented in table 7. table 8 describes the comparison of different suppliers with respect to criterion c5, i.e. recyclability of inventory items. supplier m can deliver highly recyclable items compared to the other suppliers, and hence, assigned more weight. the items that may be ijahp article: das a, das, s./supplier selection for a pump manufacturing organization by hybrid ahp-topsis technique and its impact on inventory international journal of the analytic hierarchy process 344 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.380 supplied by supplier n have high recyclability also. supplier q supplies the items which have low recyclability. table 9 pair-wise comparison matrix for alternatives for criterion c6 c6 m n 0 p q normalized priorities m 1 3 5 2 6 0.329 n 1/3 1 5 3 6 0.297 o 1/5 1/5 1 1/3 4 0.111 p 1/2 1/3 3 1 7 0.229 q 1/6 1/6 1/4 1/7 1 0.032 𝜆max = 5.434 , cr = 0.017 table 10 pair-wise comparison matrix for alternatives for criterion c7 c7 m n 0 p q normalized priorities m 1 1/6 1/4 1/3 1/7 0.035 n 6 1 3 6 4 0.38 o 4 1/3 1 4 1/2 0.187 p 3 1/6 1/4 1 1/6 0.087 q 7 1/4 2 6 1 0.309 𝜆max = 5.397, cr = 0.089 table 11 pair-wise comparison matrix for alternatives for criterion c8 c8 m n 0 p q normalized priorities m 1 3 3 6 7 0.368 n 1/3 1 2 6 6 0.28 o 1/3 1/2 1 5 6 0.234 p 1/6 1/6 1/5 1 3 0.083 q 1/7 1/6 1/6 1/3 1 0.032 𝜆max = 5.327 , c.r = 0.0122 a pair-wise comparison matrix of different suppliers for criterion c6 is established and is shown in table 9. the reusable capacity is high in the case of items procured from supplier m, and is to be mostly favored. the reusable capacity is also high for the items procured from suppliers n and p respectively, but their capacities are less than the procured items from supplier m. therefore, weights assigned to them are lower than that of supplier m. the supplier q can supply the items with low reusability, and hence, is not much favored in this work considering due focus given on environmental friendliness and the use of measures causing less disturbance on the environment. ijahp article: das a, das, s./supplier selection for a pump manufacturing organization by hybrid ahp-topsis technique and its impact on inventory international journal of the analytic hierarchy process 345 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.380 table 10 indicates a pair-wise comparison matrix of different suppliers with respect to the criterion c7, i.e. purchasing cost. the stock items procured from the suppliers n and q are cost effective, i.e. purchasing costs are less than the other suppliers, and they are assigned more weights. moderate costs are needed for procured items from supplier o, and they are given moderate weight. the items procured from supplier m are costlier than others. as they are not cost effective, less weight is assigned to them. minimum lead time is a necessity for purchasing products to achieve reduced manufacturing lead time. in this context, the lead time is minimal for purchasing products from supplier m and n respectively. supplier q supplies eco-friendly products, but the lead time is higher than the other suppliers. so, low weight is assigned to it. table 11 is constructed to compare the suppliers according to their performances over lead time. 4.2 application of topsis for selecting the appropriate suppliers here, the topsis method is used for ranking the suppliers. the best supplier is chosen to obtain a green supply chain management system of a pump manufacturing organization. in this context, n= 5 suppliers should be evaluated and each supplier has m= 8 criteria. an element of the matrix, xij is the j th criterion value of the i th supplier, and the matrix is shown in table 12. first, the decision matrix is normalized. rij = , i= 1, ……., n; j= 1, …….., m. (6) the normalized values of composite matrix and supplier’s priority against criteria are represented by rij and xij respectively. vij = wirij, i= 1, ……., n; j= 1, …….., m. (7) where, the weighted normalized value of j th criterion against i th supplier is represented by vij. positive ideal solution (a + ) = {vj + , j= 1, …….., m} (8) negative ideal solution (a ) = {vj , j= 1, …….., m} (9) vj + = max{vij, i= 1, ….., n} , vj = min{vij, i= 1,……..,n}, where, j is the benefit criteria. the distance from the i th supplier to the positive ideal solutions and the distance from the i th supplier to the negative ideal solutions are represented by si + and si respectively, and they are given by: si + = , i = 1, ….., n (10) ijahp article: das a, das, s./supplier selection for a pump manufacturing organization by hybrid ahp-topsis technique and its impact on inventory international journal of the analytic hierarchy process 346 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.380 si = , i = 1, ….., n (11) next, ci, the quality credit score or ideal solution, is calculated following equation 12. ci= , i = 1, ….., n (12) the normalized priorities for all suppliers with respect to different criteria are determined by equation 6 and presented in table 12. this table represents the decision matrix which is required for initiating the topsis method. table 12 composite vector of distributive mode for topsis weight 0.196 0.148 0.265 0.0484 0.141 0.099 0.077 0.025 supplier a b c d e f g h m 0.353 0.095 0.048 0.361 0.403 0.329 0.035 0.368 n 0.081 0.33 0.276 0.3 0.279 0.297 0.38 0.28 o 0.215 0.051 0.082 0.042 0.068 0.011 0.187 0.234 p 0.049 0.227 0.171 0.157 0.212 0.229 0.087 0.083 q 0.298 0.294 0.42 0.137 0.035 0.032 0.309 0.032 table 13 square root values from the composite matrix weight 0.196 0.148 0.265 0.0484 0.141 0.099 0.077 0.025 supplier a b c d e f g h m 0.125 0.009 0.002 0.13 0.162 0.108 0.001 0.135 n 0.006 0.109 0.076 0.09 0.078 0.088 0.144 0.078 o 0.046 0.002 0.006 .001 0.004 0.012 0.035 0.055 p 0.002 0.051 0.029 .024 0.045 0.052 0.007 0.006 q 0.089 0.086 0.176 .018 0.001 0.001 0.095 0.001 sum 0.27 0.259 0.291 0.266 0.292 0.260 0.283 0.277 square root 0.519 0.509 0.54 0.516 0.54 0.510 0.532 0.526 table 14 normalized decision matrix weight 0.196 0.148 0.265 0.0484 0.141 0.099 0.077 0.025 supplier a b c d e f g h m 0.68 0.186 0.089 0.699 0.746 0.645 0.065 0.699 n 0.157 0.648 0.511 0.582 0.517 0.582 0.713 0.531 o 0.414 0.101 0.153 0.082 0.127 0.217 0.351 0.445 p 0.095 0.447 0.317 0.305 0.392 0.449 0.163 0.158 q 0.574 0.578 0.77 0.265 0.065 0.063 0.58 0.062 ijahp article: das a, das, s./supplier selection for a pump manufacturing organization by hybrid ahp-topsis technique and its impact on inventory international journal of the analytic hierarchy process 347 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.380 table 15 weighted normalized decision matrix weight 0.196 0.148 0.265 0.0484 0.141 0.099 0.077 0.025 supplier a b c d e f g h m 0.1334 0.027 0.024 0.034 0.105 0.063 0.005 0.017 n 0.03 0.096 0.136 0.028 0.072 0.057 0.055 0.013 o 0.081 0.015 0.04 0.003 0.017 0.021 0.027 0.011 p 0.018 0.066 0.084 0.014 0.055 0.044 0.012 0.003 q 0.112 0.086 0.206 0.012 0.009 0.006 0.044 0.001 the summation values of squared normalized priorities with respect to different weights (priority vector) and the square root of the summed values are shown in table 13. calculated normalized values of the decision matrix are presented in table 14. the calculations are done using equation 6. table 16 determination of distance from positive ideal solution supplie r a b c d e f g h si + m 0 0.004 7 0.033 2 0 0 0 0.002 4 0 0.201 n 0.010 2 0 0.004 9 0.0000 3 0.001 0.0000 3 0 0.0000 1 0.127 9 o 0.002 7 0.006 5 0.027 4 0.0008 0.007 6 0.0017 0.000 7 0.0000 3 0.218 5 p 0.013 1 0.000 8 0.014 9 0.0003 0.002 4 0.0003 0.001 7 0.0002 0.184 6 q 0.000 4 0.000 1 0 0.0004 0.009 2 0.0032 0.000 1 0.0002 6 0.117 4 the weighted normalized values of the suppliers are computed by equation 7. the values with respect to eight criteria are shown in table 15. the positive ideal solutions of eight criteria are determined using equation 8. the distance between the positive ideal solutions and each supplier is computed by equation 10 and is presented in table 16. ijahp article: das a, das, s./supplier selection for a pump manufacturing organization by hybrid ahp-topsis technique and its impact on inventory international journal of the analytic hierarchy process 348 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.380 table 17 determination of distance from negative ideal solution suppli er a b c d e f g h si m 0.013 1 0.000 1 0 0.0008 0.0092 0.003 2 0 0.0002 0.170 9 n 0.000 1 0.006 6 0.0119 0.0005 0.004 0.002 6 0.0024 0.0001 0.170 8 o 0.003 9 0 0.0002 8 0 0.0000 7 0.000 2 0.0004 0.0001 0.071 2 p 0 0.002 6 0.0036 0.0001 0.0092 0.001 4 0.0000 5 0.00000 4 0.130 6 q 0.008 8 0.005 0.0332 0.0000 7 0 0 0.0015 0 0.220 6 the negative ideal solutions of eight criteria are obtained by equation 9. the distance between the negative ideal solutions and each supplier is calculated by equation 11 and presented in table 17. table 18 quality credit score for supplier ranking supplier ideal solution rank m 0.4595 3 n 0.5718 2 o 0.2459 5 p 0.4144 4 q 0.6526 1 the ideal solutions or final scores for all suppliers are described in table 18. the ranking of green suppliers are analyzed with these scores which are computed by the equation 12. 5. results and discussion rank of the supplier is determined in this work by the hybrid ahp-topsis technique. the rank is finally obtained by quality credit score or ideal solution which is described in table 18. the preference of suppliers was determined in this chronological order; i.e. supplier q> n > m > p > o. supplier q is the most preferred to obtain a green supply chain management system as well as green manufacturing system in this work. supplier q supplies a very large quantity of environmentally friendly raw material inventory items which are described in table 1. some environmentally friendly materials of finished goods are also consumed from supplier q so that waste generation will be less. the energy consumption by these items is of moderate quantity as shown in table 1. these items are not recyclable, or quite a lesser quantity of them are reusable. the inventory costs of these items are quite low, but the lead time is too high. supplier n supplies moderate quantities of environmentally friendly raw materials and the waste generation is moderate. items supplied by supplier n are moderately recyclable, or reusable, and the ijahp article: das a, das, s./supplier selection for a pump manufacturing organization by hybrid ahp-topsis technique and its impact on inventory international journal of the analytic hierarchy process 349 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.380 lead time is moderate. the raw material cost is quite low for the products delivered by supplier n. supplier o is less preferable as the items delivered by this supplier are not suitable for green manufacturing. from the results, it is shown that high preference is given for low waste generation and low inventory cost. reusability or recyclability has moderate importance on green supplier selection for the pump manufacturing industry considered. it is observed that selection of green supplier leads to green inventory, and minimum waste generation. 6. conclusion from the work presented in this article, the following conclusions may be made. the pump manufacturing organization considered in this work is located in a crowded city of eastern india, and it plans to select an eco-friendly supplier whose aim is sustainability of the environment and cost minimization of inventory items. in this context, the selection of a supplier is made by a hybrid ahp-topsis technique. it enables the organization to find the best suited supplier among those considered. the selected supplier fulfills different criteria which are favorable for the green supply chain management. selection of this green supplier is likely to promote optimal green inventory. thus, it can be stated that the hybrid method adopted using the ahp and topsis is well suited for evaluating the optimal selection of a green supplier considering eco-friendliness of the system. suppliers should be aware about environmental issues, and they need to promote green supply chain and green sustainable manufacturing systems. proper decision making is therefore needed to select green suppliers for this kind of manufacturing unit. this paper does not consider the amount of inventory items, energy consumption, purchasing costs and lead time of inventory items. future works are required to consider all these points to solve the supplier selection problem over a wider perspective. further investigation may also be made with different real life case studies related to a different manufacturing industry using the ahp-topsis hybrid algorithm or other multi criteria decision making techniques. ijahp article: das a, das, s./supplier selection for a pump manufacturing organization by hybrid ahp-topsis technique and its impact on inventory international journal of the analytic hierarchy process 350 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.380 references afshari, a., mojahed, m. & yusuff, r.m. (2010). simple additive weighting approach to personnel selection problem. international journal of innovation, management and technology, 1(5), 511-515. url: ijimt.org/papers/89-m474.pdf aouadni, s. & rebai, a. (2013). inventory management with supplier selection problem: supply uncertainty. proceedings of the international conference on modeling, simulation and applied optimization, 10, 1-5. doi: 10.1109/icmsao.2013.6552577 bayazit, o., karpak, b. & yagc, a. (2006).a purchasing decision: selecting a supplier for a construction company. journal of systems science and systems engineering, 15(3), 217-231. doi: 10.1007/s11518-006-5009-3 bhutia, p.w. & phipon, r. (2012). application of ahp and topsis method for supplier selection problem. iosr journal of engineering, 2(10), 43-50. url: www.iosrjen.org/papers/vol2_issue10%20(part-3)/f021034350.pdf chaudhury, a., mandal, b. & das, s. (2015). selection of appropriate fluid delivery technique for grinding titanium grade-1 using the analytic hierarchy process. international journal of analytic hierarchy process, 7(3), 454-469. doi: http://dx.doi.org/10.13033/ijahp.v7i3.356 dragincic, j. & vranesevic, m. (2014). ahp-based group decision making approach to supplier selection of irrigation equipment. international journal of water resources development: economic and legal aspects, 41(6), 782-791. doi: 10.1134/s0097807814060050 eleren, a. & yilmaz, c. (2011). selection of suppliers by fuzzy topsis model; sample study from turkey. international journal of business and social science, 2(22), 189-200. ertugrul, i. & oztas, t. (2014). business mobile-line selection in turkey by using fuzzy topsis, one of the multi-criteria decision methods. second international conference on information technology and quantitative management, 31, 40-47. doi:10.1016/j.procs.2014.05.243 fontana, m.e. & cavalcante, c.a.v. (2014). use of promethee method to determine the best alternative for warehouse storage location assignment. international journal of advanced manufacturing technology, 70(9), 1615-1624. doi: 10.1007/s00170-0135405-z fallahpour, a., olugu, e.u. & musa, s.n. (2015). a hybrid model for supplier selection: integration of ahp and multi expression programming (mep). international journal of neural computing and application, 10, 1-6. url: link.springer.com/content/pdf/10.1007%2fs00521-015-2078-6.pdf http://ieeexplore.ieee.org/search/searchresult.jsp?searchwithin= http://ieeexplore.ieee.org/search/searchresult.jsp?searchwithin= http://ieeexplore.ieee.org/xpl/mostrecentissue.jsp?punumber=6547264 http://dx.doi.org/10.1109/icmsao.2013.6552577 http://link.springer.com/journal/11518 http://dx.doi.org/10.13033/ijahp.v7i3.356 http://dx.doi.org/10.1016/j.procs.2014.05.243 http://link.springer.com/journal/170 http://link.springer.com/journal/170 ijahp article: das a, das, s./supplier selection for a pump manufacturing organization by hybrid ahp-topsis technique and its impact on inventory international journal of the analytic hierarchy process 351 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.380 galankashi, m.r., chegeni, a., soleimanynanadegany, a., memari, a., anjomshoae, a., helmi, s.a. & dargi, a. (2015). prioritizing green supplier selection criteria using fuzzy analytical network process. procedia of global conference on sustainable manufacturing, 26, 689–694. doi: 10.1016/j.procir.2014.07.044 ghosh, d.n. (2011). analytic hierarchy process & topsis method to evaluate faculty performance in engineering education. universal journal of applied computer science and technology, 1, 63-70. url: www.ijesit.com/volume%203/issue%201/ijesit201401_02.pdf huang, w. & huang, y.y. (2012). research on the performance evaluation of chongqing electric power supply bureaus based on topsis. energy procedia, 14, 899-905. doi: 10.1016/j.egypro.2011.12.1030 jingfei, y., wang, l. & gong, x. (2013). study on the status evaluation of urban road intersections traffic congestion based on ahp-topsis modal. proceedings of 13th cota international conference of transportation professionals, 96, 609616. doi: 10.1016/j.sbspro.2013.08.071 kahraman, c., cebeci, u. &ulukan, z. (2003). multi‐criteria supplier selection using fuzzy ahp. journal of logistics information management, 16(6), 382–394. http://dx.doi.org/10.1108/09576050310503367 lee, a.h. i., kang, h., hsu, c. & hung, h. (2009). a green supplier selection model for high-tech industry. international journal of expert systems with applications, 36, 7917– 7927. doi: 10.1016/j.eswa.2008.11.052 lei, d.x., kumar, d.t., shankar, m., kannan, d. & chen, g. (2013). analyzing criteria and sub-criteria for the corporate social responsibility-based supplier selection process using ahp. international journal of advanced manufacturing technology, 68(1), 907916. doi: 10.1007/s00170-013-4952-7 mansor, m.r., sapuna, s.m., zainudin, e.s., nuraini, a.a. & hambali, a. (2014). application of integral ahp-topsis method in hybrid natural fiber composites materials selection for automotive parking brake lever component. australian journal of basic and applied sciences, 8(5), 431-439. url: www.ajbasweb.com/old/ajbas/2014/special%202/431-439.pdf marinovic, j.s., petkovic, m.d. & stanimirovic, i.p. (2015). application of the electre method to planetary gear train optimization. journal of mechanical science and technology, 29(2), 647-654. doi: 10.1007/s12206-015-0124-z organ, a. (2013). practice over the private teaching institutions selection problem in one of secondary schools with using multiple attribute decision making method of topsis. proceedings of the second cyprus international conference on educational research, 89, 489-495. doi: 10.1016/j.sbspro.2013.08.882 http://dx.doi.org/10.1016/j.procir.2014.07.044 http://dx.doi.org/10.1016/j.procir.2014.07.044 http://dx.doi.org/10.1016/j.egypro.2011.12.1030 http://dx.doi.org/10.1016/j.sbspro.2013.08.071 http://www.emeraldinsight.com/action/dosearch?contribstored=kahraman%2c+c http://www.emeraldinsight.com/action/dosearch?contribstored=cebeci%2c+u http://www.emeraldinsight.com/action/dosearch?contribstored=ulukan%2c+z http://dx.doi.org/10.1108/09576050310503367 http://dx.doi.org/10.1016/j.eswa.2008.11.052 http://link.springer.com/journal/12206 http://link.springer.com/journal/12206 http://dx.doi.org/10.1016/j.sbspro.2013.08.882 ijahp article: das a, das, s./supplier selection for a pump manufacturing organization by hybrid ahp-topsis technique and its impact on inventory international journal of the analytic hierarchy process 352 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.380 saaty, t.l. (1977). a scaling method for priorities in hierarchical structures. journal of mathematical psychology, 15, 234-281. doi: 10.1016/0022-2496(77)90033-5 saaty, t.l. (1990). decision making by the analytic hierarchy process: theory and applications. european journal of operational research, 48, 9-26. url: www.colorado.edu/geography/leyk/geog_5113/.../saaty_2008.pdf saaty, t.l. (2002). decision making with analytic hierarchy process. scientia iranica, 9, 215-229. url: www.colorado.edu/geography/leyk/geog_5113/.../saaty_2008.pdf sabiruddin, k., bhattachariya, s. & das, s. (2013). selection of appropriate process parameters for gas metal arc welding of medium carbon steel specimen. international journal of analytic hierarchy process, 5, 252-267. http://dx.doi.org/10.13033/ijahp.v5i2.184 shahroudi, k. & tonekaboni, s.m.s.(2012). application of topsis method to supplier selection in iran auto supply chain. journal of global strategic management, 12, 123131. doi: 10.20460/jgsm.2012615779 sharma, b. & jayaswal, p. (2015). supplier selection in manufacturing industry using topsis method. international journal of research in engineering and applied sciences, 5(12), 62-66. tahriri, f., osman, m.r., ali, a., yusuff, r. m. & esfandiary, a. (2008). ahp approach for supplier evaluation and selection in a steel manufacturing company. journal of industrial engineering and management, 1(2), 54-76. doi: http://dx.doi.org/10.3926/jiem..v1n2.p54-76 umadevi, k., elango, c. & rajesh, r. (2012). vendor selection using ahp. proceedings of the international conference on modeling optimization and computing, 38, 1946-1949. doi: 10.1016/j.proeng.2012.06.237 wang, t. & xin, b.c. (2011). thermal power plant sitting based on topsis method. procedia of advances in control engineering and information science, 15, 5384–5388. doi: 10.1016/j.proeng.2011.08.998 zhu, x., wang, f., liang, c., lia, j. & sun, x. (2012). quality credit evaluation based on topsis: evidence from air-conditioning market in china. proceedings of theinternational conference on computational science, 9, 1256-1262. doi: 10.1016/j.procs.2012.04.137 http://dx.doi.org/10.1016/0022-2496(77)90033-5 http://www.sciencedirect.com/science/journal/03772217 http://www.colorado.edu/geography/leyk/geog_5113/.../saaty_2008.pdf http://dx.doi.org/10.13033/ijahp.v5i2.184 http://dx.doi.org/10.3926/jiem..v1n2.p54-76 http://dx.doi.org/10.1016/j.proeng.2012.06.237 http://dx.doi.org/10.1016/j.proeng.2011.08.998 http://dx.doi.org/10.1016/j.procs.2012.04.137 http://dx.doi.org/10.1016/j.procs.2012.04.137 ijahp article: guerrero-baena, gómez-limón/ decomposing value creation when assessing investments: a multicriteria approach based on the analytic hierarchy process international journal of the analytic hierarchy process 159 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.236 decomposing value creation when assessing investments: a multicriteria approach based on the analytic hierarchy process m. dolores guerrero-baena faculty of law and business and economic sciences university of cordoba, spain dolores.guerrero@uco.es josé a. gómez-limón faculty of law and business and economic sciences university of cordoba, spain jglimon@uco.es abstract this paper aims to develop a novel capital budgeting method to improve the quality of the appraisal process for productive investments. this will be done by decomposing the total value that is created by the new assets into two components: financial value and nonfinancial capital value, the latter stemming from the intellectual capital of the firm. we propose a methodology based on the analytic hierarchy process (ahp). within the model, four main criteria (financial capital, human capital, structural capital, and relational capital), several subcriteria and the investment alternatives are defined. in order to determine the total value of each alternative, chief executive officer (ceo) preferences are required. a case study on the agrifood sector illustrates the model empirically. this illustrative application evidences the need to consider the impact of productive investments on firms’ intangible assets, as this impact actually affects the choice of optimal investment alternative in the real world. keywords: capital budgeting, multicriteria decision making, nonfinancial capital value, intellectual capital, ahp. 1. introduction the corporate capital budgeting process is one of the most challenging tasks facing firms’ management (baker & english, 2011), as it concerns investment decisions which involve allocating scarce funds over time to achieve a firm’s objectives. in order to support decision-making in investment appraisal processes, traditional financial techniques based on net present value (npv) and the internal rate of return (irr) have been widely employed by corporate decision-makers (brounen, de jong, & koedijk, 2004). however, many authors have identified various problems and shortcomings derived from their application. some of these criticisms question the realism of the firm’s value maximization assumption when analyzing investment alternatives. in this sense, steuer and na (2003) affirm that modern corporations do not pursue the single objective of shareholder wealth maximization assumed by traditional techniques, instead taking into account a full array of objectives concerning the different stakeholders of the firm (shareholders, managers, employees, and customers). more recently, koontz and weihrich (2007) and götze, northcott, and schuster (2008), following the seminal work by freeman (1984), maintain that mailto:dolores.guerrero@uco.es mailto:jglimon@uco.es ijahp article: guerrero-baena, gómez-limón/ decomposing value creation when assessing investments: a multicriteria approach based on the analytic hierarchy process international journal of the analytic hierarchy process 160 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.236 decision-makers in organizations wish to pursue several competing goals rather than a single one as traditional methods assume 1 . another source of criticism is related to the inability of traditional appraisal techniques to recognize the real value generated by an investment, simply because they ignore important qualitative variables, hardly measurable in monetary terms, that also add value to the firm (firouzabadi, henson, & barnes, 2008; kreng, wu, & wang, 2011). this paper focuses on the latter source of criticism of traditional corporate finance theory. thus, assuming that the main objective when appraising investments in for-profit firms is market value creation (dayananda, irons, harrison, herbohn, & rowland, 2002; ross, westerfield, & jordan, 2007), we aim to (partially) solve the inability of conventional appraisal methods to recognize the real value of new productive assets by developing a novel approach to assessing investments. capital budgeting decisions have a major effect on the total or market value of the firm (dayananda et al., 2002), affecting its two components, financial value and nonfinancial capital value. the latter is created by the intellectual capital of the firm, a concept that can be defined as all nonmonetary and nonphysical resources controlled by the firm that contribute to the organization’s value creation (roos, pike, & fernström, 2006). however, the effects of investments on the second component of the firm’s value have, largely, not been considered by traditional financial appraisal techniques for two main reasons. first, because of the difficulty of monetarily quantifying the increase in cash flows from investing in intellectual capital (how much), and second, because of the uncertainty about the point in time where these cash flows will take place (when). therefore, new capital budgeting methodologies capable of decomposing the total value generated by investments into these two components of a firm’s value, financial and nonfinancial capital value, are welcome. this paper intends to develop a novel approach to improve the quality of the investment appraisal process in for-profit firms by decomposing the overall value that is created by new assets into its two components, financial and intellectual value. in order to do so, this paper proposes a methodology based on the analytic hierarchy process (ahp), which permits a more accurate assessment of the value creation of the different project investment alternatives, determining for this purpose the relative importance of each criterion (financial and intellectual value) and subcriterion involved in these decision-making processes. the method proposed is empirically illustrated by a case study of the agrifood sector. the remainder of the paper is structured as follows. section 2 briefly discusses a firm’s total value and its two main components, financial and intellectual value, and describes the ahp technique. furthermore, this section provides an analytical framework to quantify and decompose the real value of an investment project. in section 3, the methodological approach proposed is implemented in a real case study, focusing on investment decisions in the meat industry regarding the food quality control system to be implemented. finally, section 4 presents the conclusions and suggests lines for further research. 1 this circumstance is particularly relevant in nonprofit organizations (i.e., public administrations, ngos, etc.), where other criteria such as employment generation or equity and gender issues can be taken into account. however, for most for-profit firms, the assumption of a firm’s value maximization can be still considered as the most important criterion when making decisions regarding investments (brigham & ehrhardt, 2008). ijahp article: guerrero-baena, gómez-limón/ decomposing value creation when assessing investments: a multicriteria approach based on the analytic hierarchy process international journal of the analytic hierarchy process 161 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.236 2. investment decision-making and value creation 2.1 firm total value: financial and nonfinancial capital value the total economic value of a firm, or simply the market value, is the result of adding up its financial capital (book value) and nonfinancial capital, also called intellectual capital (roos, roos, dragonetti, & edvinsson, 1998; johnson, 1999). hence, a company’s economic value is not merely the sum of the value of its tangible assets, but also the value of its intangible assets (curado, henriques, & bontis, 2011), most of the latter being hidden or invisible for accounting and not reported in any financial statement. the knowledge of high-quality production processes or employees’ talent and knowhow are good examples of intangible assets. in fact, intangible resources controlled by the firm have been identified as major contributors to the generation of persistent profits (villalonga, 2004) and, thus, to increasing market value (cañibano, garcia-ayuso, & sánchez, 2000; sullivan, 2000; edvinsson, 2013). the management of these invisible assets is a key element of business strategy. both financial and nonfinancial indicators should be jointly used to provide a complete measurement of company success and shareholder value (sveiby, 1997). the first component of a firm’s total value (financial capital) has been extensively studied by classical finance theory. both business valuation methods (balance sheetbased methods or income statement-based techniques) and capital budgeting methods such as the discounted cash flow techniques (net present value, internal rate of return, or the discounted payback period) have been proposed for this purpose. the latter of these techniques represents indicators of financial value creation by new investment projects. the net present value (npv) is one of the most frequently used capital budgeting techniques (graham & harvey, 2001). this method evaluates an investment project by discounting its future cash flows to their present values and subtracting the amount of the initial outlay from their sum. if the npv is greater than 0, the project will create value for the firm. two elements must be known to apply this technique: the net cash flow that the investment will generate over its life, that is, cash inflows minus cash outflows, and the discount rate that should reflect the degree of risk inherent in the project under consideration. in contrast, nonfinancial or intellectual capital (ic) is a more recent concept grounded on the resource-based view (rbv) of firm theory (kristandl & bontis, 2007) which has been discussed in the literature over the last two decades. in fact, researchers have yet to fully agree on a definition of this term. some of the most widely accepted definitions focus on the intangible aspect of the assets composing this kind of capital: “the sum of the hidden assets of the company not fully captured on the balance sheet” (roos & roos, 1997) or “the total stocks of all intangible assets and capabilities” (edvinsson & malone, 1997). stewart (1997) provides a complementary view when stating that ic is “the sum of everything everybody in a company knows that gives it a competitive edge”, that is, the full array of knowledge, information, intellectual property and experience useful in generating profits (wealth creation). other definitions of intellectual capital also focus on its ability to provide value and utility for the company (bontis, 1999; sullivan, 2000; roos et al., 2006). there is no doubt that ic is an important source of sustainable competitive advantage for a firm (itami, 1987; roos & roos, 1997) because invisible assets are difficult for competitors to imitate. intangibles are important in the management process, as they have become a crucial resource for the firm, mainly due to their impact on innovation processes (sánchez, chaminade, & olea, 2000). villalonga (2004) found that ijahp article: guerrero-baena, gómez-limón/ decomposing value creation when assessing investments: a multicriteria approach based on the analytic hierarchy process international journal of the analytic hierarchy process 162 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.236 intangibles play an effective role in sustaining a firm’s competitive advantage, measured through the persistence of firm-specific profits. finally, it is worth mentioning that a strong relationship between intellectual capital and business performance has already been found in several empirical papers (chen, zhu, & xie, 2004; phusavat, comepa, sitko-lutek, & ooi, 2011), confirming the hypothesis that intellectual capital is a key element for value creation within the firm. although there is no unique classification of the components of ic, a considerable number of papers in the literature (stewart, 1997; bontis, 1998; roos et al., 1998; youndt, subramaniam, & snell, 2004) have divided it into the following three categories: human capital, structural capital and relational capital (see figure 1). human capital may be defined as the collective capabilities of employees such as (edvinsson & sullivan, 1996): expertise, skills, intelligence and general knowhow of all of the firm´s employees. in the late sixties, likert (1967) postulated that human resources contribute to value creation in the company and, following this theory, edvinsson and sullivan (1996) argued that human capital is a resource because it generates value for the company. this value stems from competence, attitude and the intellectual agility of employees (roos et al., 1998). a firm with more capable employees is likely to earn higher profits than its competitors (cañibano et al., 2000), thus positively affecting the firm’s outcomes (huselid, 1995; hitt, biermant, shimizu, & kochhar, 2001) and also having an impact on its competitive advantages (johnson, 1999; grigoroudis, tsitsiridi, & zopounidis, 2013). the scientific literature provides different attributes that can be measured relative to human capital: knowhow, capability, satisfaction, entrepreneurial spirit, leadership, attitude, creativity, etc. employees’ knowhow, entrepreneurial spirit and employees’ satisfaction are the most highlighted components of human capital (becker, huselid, & ulrich, 2001). structural capital is defined as the organizational ability of the firm to utilize human intellect and innovation to create wealth (johnson, 1999), representing institutionalized knowledge and codified experience stored in databases, routines, manuals, structures and the like (hall, 1992). this type of knowledge “doesn’t go home at night” (stewart, 1997), unlike human capital, and provides coherence and guidance for the whole organization (edvinsson & malone, 1997). the essence of structural capital is the knowledge embedded within the routines of an organization, containing the key elements for productive efficiency, optimization of transaction times, procedural innovativeness and adequate access to information (bontis, 1998). there is also a positive relationship between structural capital and value creation (marr, schiuma, & neely, 2004; díez, ochoa, prieto, & santidrián,2010). chen et al. (2004) break this category of ic down into company culture, organizational structure, organizational learning, operational processes and information systems. the same authors also find that the product quality level is an important structural capital indicator, because it has a direct effect on customer satisfaction and therefore upon customer loyalty. also, numerous authors (chang, chen, & lai, 2008; wu, chang, & chen, 2008) have pointed to organizational routine knowledge as another important indicator of structural capital. lastly, relational capital refers to all resources that are linked to the external relationships of the firm, those connecting it to both other economic agents related to the business (shareholders, customers, suppliers, allies, unions, etc.), and also social or civil agents, such as non-governmental organizations (ngo) or public institutions (martín-de-castro, delgado-verde, lópez-sáez, & navas-lópez, 2011). johnson ijahp article: guerrero-baena, gómez-limón/ decomposing value creation when assessing investments: a multicriteria approach based on the analytic hierarchy process international journal of the analytic hierarchy process 163 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.236 (1999) and bontis (1999) point out that companies’ relational capital has a positive effect on their competitive advantage. customer relations, supplier relations and environmental consciousness are considered the most important components of relational capital (lev, 2004). regarding the first element, a positive association exists between customer satisfaction and market value (anderson, fornell, & mazvancheryl, 2004), because higher customer satisfaction increases the loyalty of existing customers, reduces price elasticity and enhances the firm´s reputation. better supplier-firm interaction may also enhance the reputation of the firm (johnson, 1999). many authors (claver, lópez, molina, & tarí, 2007; lópez-gamero, zaragoza-sáez, claver-cortés, & molina-azorín, 2011) have considered environmental consciousness as an important component of relational capital. furthermore, numerous references in the literature identify a positive relationship between environmental management and firm performance (naffziger, ahmed, & montagno, 2003; montabon, sroufe, & narasimhan, 2007). porter and van der linde (1995) argued that companies which engage in corporate environmental management and green innovation can actively improve their corporate image, charge relatively high prices for green products, sell the knowhow and services of environmental protection, develop new markets and eventually obtain competitive advantages. 2.2 the analytic hierarchy process the ahp method (saaty, 1980) is a structured but flexible technique to support multicriteria decision making, suitable when both qualitative and quantitative aspects need to be considered in the problem. this section provides a brief overview of the ahp technique. for a more detailed explanation of the method, both from a theoretical and a practical point of view, readers can consult saaty (1980) and saaty and vargas (2000). the implementation of ahp involves four phases. in the first step, a complex decision problem is structured as a tree-based hierarchy, with at least three levels: the final ‘target’ at the highest level of the structure, decision ‘criteria’ at an intermediate level and ‘alternatives’ forming the base of the structure. when criteria are abstract or complex, the intermediate level can be split into a series of sequentially organized ‘subcriteria’ levels. the second step is the measurement and data collection, which involves assigning pairwise comparisons (judgments) by the decision maker to all elements (criteria/subcriteria/alternatives) hanging from every node in the hierarchy following saaty’s fundamental scale (see table 1). the comparative judgments start from the target node, comparing all criteria included in the second level of the hierarchy and finish with the (sub)criteria nodes, comparing the alternatives considered in the lowest level. for each node, the hanging elements are pairwise compared according to the decision maker’s opinions on their importance regarding the (sub)criteria considered in the higher level. a questionnaire is designed and used to collect these comparison judgments. ijahp article: guerrero-baena, gómez-limón/ decomposing value creation when assessing investments: a multicriteria approach based on the analytic hierarchy process international journal of the analytic hierarchy process 164 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.236 table 1 the nine-point scale for pairwise comparison in the ahp importance intensity definition 1 equal importance 3 moderate importance of one over another 5 strong importance of one over another 7 very strong importance of one over another 9 extreme importance of one over another 2, 4, 6, 8 intermediate values the judgments provided allow positive matrices to be built for each node with the following structure (saaty’s matrices): 𝐴 = (𝑎𝑖𝑗 )𝑛×𝑛 = [ 𝑎11 𝑎12 … 𝑎1𝑛 𝑎21 𝑎22 … 𝑎2𝑛 … … 𝑎𝑖𝑗 … 𝑎𝑛1 𝑎𝑛2 … 𝑎𝑛𝑛 ] (1) where 𝑎𝑖𝑗 represents the relative importance of the element 𝑖 with respect to the element 𝑗 (both at the same level of the hierarchy). this matrix has two fundamental properties: (i) all the elements in its main diagonal area take a value of one; and (ii) all other elements maintain that pairwise comparisons are reciprocal (if 𝑎𝑖𝑗 = 𝑥, then 𝑎𝑗𝑖 = 1 𝑥⁄ ). given that the second property, one needs to make only 𝑛(𝑛 − 1) 2⁄ of the comparisons to fill in the matrix of judgments. furthermore, if the pairwise comparison matrix 𝐴 satisfies that 𝑎𝑖ℎ × 𝑎ℎ𝑗 = 𝑎𝑖𝑗 for all 𝑖, 𝑗, and ℎ, then 𝐴 is said to be perfectly consistent, meaning that the numerical ratings 𝑎𝑖𝑗 satisfy 𝑎𝑖𝑗 = 𝑤𝑖 𝑤𝑗⁄ , with 𝑤𝑖 and 𝑤𝑗 being the weights of the elements 𝑖 and 𝑗, respectively. in this case, weights for every element can be obtained by normalizing any of the rows or columns of 𝐴. however, decision makers rarely provide perfect consistent judgments in reality (especially for high-order matrices) for a number of reasons (lack of information or an unclear opinion, lack of concentration, etc.). in these cases, the literature proposes different approaches to estimate the weight vector (𝑊 = (𝑤1, … , 𝑤𝑖 , … 𝑤𝑛 )) for each matrix, that is, the vector of the relative priorities of the elements that hang from a common node (local priorities). saaty (1980) proposed two alternatives: the row geometric mean and the main eigenvector. other alternatives also suggested include models based on regression analysis or goal programming (srdjevic, 2005). although there is no agreement in the literature regarding any alternative outperforming another (fichtner, 1986), we have chosen the main eigenvector method as suggested by saaty (2003). following this method the vector of priority weights is obtained by solving the following characteristic equation: 𝐴𝑊 = 𝜆𝑚𝑎𝑥 𝑊 (2) where 𝜆𝑚𝑎𝑥 is the maximum eigenvalue of 𝐴. ahp allows some small inconsistency in decision maker’s judgments, but obtaining reliable weights requires that this inconsistency remains below certain limits; otherwise, the weight vector 𝑊 derived from the eigenvector (or any other) method ijahp article: guerrero-baena, gómez-limón/ decomposing value creation when assessing investments: a multicriteria approach based on the analytic hierarchy process international journal of the analytic hierarchy process 165 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.236 could not be considered trustworthy. in order to check this requirement, a specific measure of inconsistency for each saaty’s matrix 𝐴 needs to be calculated. if the pairwise comparisons provided by the decision maker are completely consistent, the maximum eigenvalue (𝜆𝑚𝑎𝑥 ) of matrix 𝐴 is equal to the number of elements considered (𝑛). then, the amount resulting from the difference 𝜆𝑚𝑎𝑥 − 𝑛 is a measure of the degree of inconsistency within the matrix 𝐴. this is why saaty (1980) defined the consistency index (ci) as: 𝐶𝐼 = (𝜆𝑚𝑎𝑥 − 𝑛)/(𝑛 − 1) (3) denote the consistency index for a randomly generated 𝑛 × 𝑛 matrix as ri. from ci and ri indexes, saaty (1980) defined the consistency ratio (cr) as: 𝐶𝑅 = 𝐶𝐼 𝑅𝐼⁄ (4) if the cr is smaller than 0.1, then the matrix 𝐴 can be considered as having an acceptable consistency, and the resulting weights being deemed as reliable. if the cr is greater than 0.1, the subjective judgments need to be revised. if we wish to compare the relative importance assigned to the different (sub)criteria proposed in the hierarchy, it is necessary to obtain the corresponding global priorities (𝑤𝑖 ∗) for all the (sub)criteria on the same level of the hierarchy, that add up to one. thus, 𝑤𝑖 ∗ is an indicator of the importance of (sub)criterion 𝑖 across the whole set of (sub)criteria considered on this level with respect to the global goal. these global priorities are obtained from the second level down by the hierarchical composition principle, multiplying each local priority by the priority of the parent node in the level immediately above (the second level elements are each multiplied by unity, the weight of the single top level goal). the quantification of local and global priorities and the measurement of the consistency constitute the third stage of the ahp, prioritization. and finally, in the fourth stage, synthesis, the composite priority of each alternative with respect to the decision goal on the top of the hierarchy is generated by the adding of weights to the common nodes at the bottom level 2 . this is why the ahp has been traditionally associated with an additive value function (kamenetzky, 1982) as follows: 𝑉𝑖 = ∑ 𝑣𝑖𝑗 𝑤𝑗 𝑚 𝑗=1 (5) where 𝑉𝑖 (𝑖 = 1, … , 𝑛) are the overall values of decision alternatives; 𝑣𝑖𝑗 (𝑗 = 1, … , 𝑛) are the values of decision alternatives with respect to (sub)criteria 𝑗, and 𝑤𝑗 (𝑗 = 1, … , 𝑚) are the weights of decision (sub)criteria. 2 other aggregation procedures to obtain composite priorities of alternatives have been proposed, including both additive and multiplicative shaped formulations (stam & duarte silva, 2003). ijahp article: guerrero-baena, gómez-limón/ decomposing value creation when assessing investments: a multicriteria approach based on the analytic hierarchy process international journal of the analytic hierarchy process 166 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.236 2.3 proposal for quantifying total value generated by productive investments since ahp is perfectly suited to capital budgeting decision-making, this technique has been widely used in the literature to support investment selection in multicriteria frameworks (vaidya & kumar, 2006), as well as in asset valuation processes (aznar, cervelló, & romero, 2011). however, no research has addressed investment appraisal considering both the financial and intellectual value derived from the capital budgeting decision. furthermore, researchers have paid little attention to the management of intellectual capital with multicriteria techniques. the only exceptions are a few papers focusing on the selection of indicators to measure intellectual capital (han & han, 2004; bozbura, beskese, & kahraman, 2007) and on the relationships linking knowledge assets with company’s performance in a new product development problem (carlucci & schiuma, 2007). being aware that value creation in the firm does not only stem from financial capital, as traditional valuation methods assume, but also from nonfinancial capital, it is necessary to incorporate the latter into capital budgeting appraisal processes. we therefore propose an analytical approach based on the ahp technique that quantifies the total market value created by productive asset investments. the hierarchical structure of the proposed approach has four levels (see figure 1): the final target of the decision problem (market value creation by investments), criteria (the components of financial capital and intellectual capital), subcriteria (indicators of the different components of a firm’s capital) and alternatives (investment alternatives). this hierarchy was developed based on the literature review carried out regarding this field of knowledge. moreover, this structure was found suitable for investment appraisal in for-profit firms in a real setting by a group of academic experts in the fields of corporate finance and management science and by several widely experienced managers. in any case it is worth mentioning that the experts who were consulted agreed that the structure shown in figure 1 must be considered only as a general yet flexible hierarchy, which would need specific fine-tuning before its implementation in case studies. thus, this model needs to be adaptively modified in accordance with each investment appraisal process in order to consider the specific features of the assets to be incorporated and the influence of the investment options on the different components of a firm’s value. this is particularly relevant when defining the concrete set of indicators of the intellectual capital subcriteria, an issue that is directly influenced by the nature of the investment and the features (size, structure, market orientation, etc.) of the firm. finally, it is also worth noting that the experts who were consulted commented that the measurement of the values of investment alternatives with respect to nonfinancial criteria (𝑣𝑖𝑗 in expression (5)) would be the main difficulty for the implementation of this proposal in the real world, taking into account that most of these criteria have an intangible nature (i.e., there is no measurement scale for them). however, all experts agreed that using the ahp to quantify these intangible criteria as proposed by saaty, vargas, and dellmann (2003) is an accurate enough solution in order to make this proposal operative. in our case, this measurement process involves calculating the weights derived from paired comparisons of the investment alternatives considered with respect to their efficiency in attaining each of the nonfinancial criteria. these weights are a measure of the value 𝑣𝑖𝑗 along a ratio scale within a range [0-1]. for homogeneity reasons, the same method of measurement has been used for financial criteria, valuing investment alternatives for these criteria also within a range [0-1]. ijahp article: guerrero-baena, gómez-limón/ decomposing value creation when assessing investments: a multicriteria approach based on the analytic hierarchy process international journal of the analytic hierarchy process 167 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.236 figure 1. decomposition of market value creation in assessing productive investments ijahp article: guerrero-baena, gómez-limón/ decomposing value creation when assessing investments: a multicriteria approach based on the analytic hierarchy process international journal of the analytic hierarchy process 168 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.236 3. an illustrative case study 3.1 model tuning in order to demonstrate the applicability of the proposed model, this section presents a real decision-making problem focused on the selection of an investment project in the food industry. the problem consists of assessing and prioritizing the following three alternative systems for meat-product quality control: (1) establishing a firm´s own traditional laboratory, where samples of products are analyzed to control their quality. (2) acquiring a near infrared spectroscopy (nirs) system, a fast and nondestructive analytical technique based on the absorption of electromagnetic spectrum of the products suitable to control all quality parameters. these features are responsible for the application of the system becoming increasingly widespread in the food industry in general and meat manufacturing in particular (pérez-marín, de pedro sanz, guerrero-ginel, & garrido-varo, 2009). (3) outsourcing analysis and quality control services to an external laboratory. valuation of these three alternatives is a complex task, as food quality control systems have strategic implications for the firm as a whole, impacting both tangible and intangible capital (irudayaraj & reh, 2008). for this reason, it is appropriate to apply the model proposed in the previous section to this case study. based on the value creation model proposed (figure 1), we fine-tuned the hierarchy to be used in this case study. in order to do so, the authors first performed an extensive review of the literature to catalogue indicators of value creation in the food industry regarding each of the subcriteria considered in the general model. later, a group of academic experts (four with expertise in finance and management and two from the food technology field) discussed the catalogue developed, and chose the most relevant subcriteria to be taken into account in order to analyze value creation by the investments in quality control in the meat industry. in this way, eight subcriteria were finally chosen, as displayed in the adapted hierarchy shown in figure 2. lastly, it is worth mentioning that the five firms’ managers (ceo) involved in the empirical application (see next section) also validated the subcriteria chosen and the whole hierarchy proposed, prior to completing the questionnaire developed for the implementation of the methodology. ijahp article: guerrero-baena, gómez-limón/ decomposing value creation when assessing investments: a multicriteria approach based on the analytic hierarchy process international journal of the analytic hierarchy process 169 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.236 figure 2. hierarchical structure used to value an investment in food quality assessment ijahp article: guerrero-baena, gómez-limón/ decomposing value creation when assessing investments: a multicriteria approach based on the analytic hierarchy process international journal of the analytic hierarchy process 170 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.236 3.2 data collection five spanish meat firms with a minimum annual turnover of 1.5 million euros each were selected for the empirical application of the proposed methodology (referred to as a, b, c, d, and e to protect their identity). this size restriction is justified by the technical and financial impossibility of undertaking the investment projects required by alternatives 1 and 2. table 2 shows the main characteristics of the five firms. on the one hand, it is worth noting that a and d are family businesses with annual turnovers of less than €2 million, total assets of 5.3 and 1.9 million, respectively, and that neither has more than ten employees. on the other hand, the rest of the firms are larger corporations displaying higher annual turnovers, ranging from €6.4 million in the case of b to €15.8 million for e, as well as higher total asset figures, amounting to €31.2 million in the case of b. furthermore, for the larger corporations the number of employees ranges from 31 in b to 53 in c. table 2 financial-economic description of the firms firm a b c d e net revenue (m€) 1.9 6.4 14.8 1.6 15.8 total assets (m€) 5.3 31.2 16.6 1.9 14.1 employees 8 31 53 10 46 after selecting the five firms, we contacted their ceos for an interview at their workplaces during which they completed the questionnaire provided in order to obtain the evaluations. thus, each ceo was asked to make pairwise comparisons to obtain the weights of the criteria and subcriteria considered in the analysis. furthermore, these managers also provided the pairwise comparisons required to quantify the contribution of each investment alternative with respect to each subcriterion. 3.3 results first, to determine the consistency of ceo’s in assessing pairwise comparisons, the cr was calculated for each judgment matrix. since cr was smaller than the threshold value of 0.1 in all cases, judgments and the derived weights were considered consistent and valid for the empirical analysis. table 3 summarizes the relative importance derived for financial and non-financial capital in each firm. as can be observed, the family-run firms (a and d) assigned greater relevance to nonfinancial capital value, 75% in both cases; while for large industrial corporations (b, c, and e) financial value is substantially more important, at around 80%. this duality is justified, firstly because family firms prefer to improve their competitiveness and therefore their total value, through long term strategies (d´allura & minichilli, 2012) focused primarily on intangible issues (habbershon, williams, & macmillan, 2003). thus, their value creation strategies are mainly based on increasing their nonfinancial capital value. secondly, the differing importance assigned to financial and nonfinancial capital values by firms can be explained by the agency theory (jensen & meckling, 1976) and, more specifically, by “managerial myopia”. this means that managers of large corporations may pursue their own interests by investing in projects with cash-flows closer in time, but less profitable in ijahp article: guerrero-baena, gómez-limón/ decomposing value creation when assessing investments: a multicriteria approach based on the analytic hierarchy process international journal of the analytic hierarchy process 171 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.236 the long term or by rejecting highly profitable projects because they have smaller cash flows in the short term (byrd, parrino, & pritsch, 1998). however, in family businesses, where ownership and management are not separate, this problem has very little impact; hence, they prefer to sacrifice short-term profitability to generate longterm value, prioritizing attributes associated with nonfinancial capital. by contrast, in larger firms where there is total separation between shareholders and managers, this agency problem does occur, the short-term view taking priority and more relevance being assigned to the attributes associated with financial value (npv). table 3 weights derived for financial and non-financial capital in each firm firm a b c d e financial value 25.0 75.0 83.3 25.0 87.5 financial capital 25.0 75.0 83.3 25.0 87.5 npv 25.0 75.0 83.3 25.0 87.5 nonfinancial capital value 75.0 25.0 16.7 75.0 12.5 human capital 32.1 10.7 11.6 25.0 3.5 skilled labor/knowhow 24.1 1.8 1.5 3.1 0.9 entrepreneurial spirit 8.0 8.9 10.1 21.9 2.6 structural capital 10.7 10.7 2.2 25.0 7.3 product quality 6.8 6.3 1.7 19.2 1.6 manufacturing flexibility 3.3 3.0 0.2 1.9 5.2 lead time 0.7 1.5 0.2 4.0 0.5 relational capital 32.2 3.6 2.9 25.0 1.7 access to distribution channels 28.1 3.0 2.5 21.9 1.5 environmental consciousness 4.0 0.6 0.4 3.1 0.2 table 3 also shows the weights of the criteria and subcriteria in each firm. in reference to nonfinancial capital value, there is no common pattern regarding the weights assigned to the different components. in regards to the two family firms, which assigned a preferential weight to this component of total value, it is worth noting that a gives a strong and similar weight to the criteria related to human capital and relational capital (32% each), while d gives equal importance (25%) to each of the three nonfinancial capital value criteria. in any case, it is worth pointing out that human capital is highly relevant in both firms because it is an essential element of the success of smaller companies (coleman, 2007). also, access to distribution channels, as a subcriterion of relational capital, was perceived as highly significant by the two family firms: 28.1% in a and 21.9% in d. this fact can be explained by the strategic importance of commercial relations in small firms positioned in market segments of high quality and high added value products, which require narrow and specific marketing channels. with reference to larger industrial firms, it should be emphasized that b assigns the most importance (25%) to nonfinancial capital value, primarily to human capital and structural capital, with 10.7% each. in c, much of its nonfinancial capital value is generated by human capital and, in particular, through the entrepreneurial spirit of employees (10.1% out of a total of 16.7% of the nonfinancial capital value). in contrast, in the case of the firm that gives the least weight to this second component ijahp article: guerrero-baena, gómez-limón/ decomposing value creation when assessing investments: a multicriteria approach based on the analytic hierarchy process international journal of the analytic hierarchy process 172 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.236 of value (e), structural capital is the most important criterion, manufacturing flexibility figuring prominently with 5.2%. this reflects the need of this firm to adjust their production portfolio quickly to meet variable customer demands. one common feature of four out of the five firms (all except e) is that product quality is the key subcriterion within structural capital (reaching up to 19.2% in d). this reflects that the focus on quality is the main differentiation and value generation strategy of most of the firms in this food sector. these results corroborate the evidence obtained in youndt et al. (2004), which confirmed the existence of multiple intellectual capital profiles in a wide group of firms. according to these authors, business strategy determines orientation towards different types of intellectual capital. after obtaining the weights of the criteria and subcriteria from the ceos, equation 5 was used to compare the total value generated by each investment alternative in every firm, providing the results shown in table 4. furthermore, in order to validate the proposed model, these results were compared to the alternatives to meat quality control actually chosen by the five firms analyzed. as can be seen, nirs is the alternative that, to a greater extent, creates the most value in four of the five firms (a, b, c, and e). this is considerably ahead of the other two alternatives in larger firms. for example in e, the nirs system records a value of 0.75, compared to 0.17 in the case of traditional lab and 0.08 for outsourcing. however, only two (b and e) of the four firms have already implemented this technology. the ceos at a and c indicated in their interviews that the decision to invest in nirs had actually been taken, but effective implementation of the system would ultimately depend on the restrictions on access to credit arising from the current financial crisis. in fact, these two firms are involved in two research projects for the development and future implementation of nirs in their production processes. regarding firm d, the smallest of the five, outsourcing is the alternative that generates the most value and this is the option the firm actually chose for meat quality control. table 4 market value creation of each alternative firm traditional lab near infrared spectroscopy (nirs) outsourcing quality control system currently implemented a 0.27 0.50 0.23 outsourcing* b 0.26 0.51 0.23 nirs c 0.09 0.73 0.18 outsourcing* d 0.32 0.28 0.40 outsourcing e 0.17 0.75 0.08 nirs * the decision to invest in nirs system has already been taken but it still has not been effective. the practical interest of the theoretical model this paper proposes was thus confirmed when comparing the results obtained by our model and the actual investment decisions taken in each firm. this fact leads to the conclusion that the proposed model is a formalization of the process actually followed by financial decision ijahp article: guerrero-baena, gómez-limón/ decomposing value creation when assessing investments: a multicriteria approach based on the analytic hierarchy process international journal of the analytic hierarchy process 173 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.236 makers to capital budgeting. in this formalized process, all value creation criteria, both financial and nonfinancial, are integrated in a transparent and instrumental way. however, further empirical evidence is required to definitively confirm this proposition. nevertheless, these results clearly evidence that the ceos of the firms surveyed certainly consider nonfinancial capital value creation components in their analysis for capital budgeting decision-making, in addition to traditional financial criteria (financial value measured by npv or irr). 4. conclusions and future research this paper has developed a novel methodology for capital budgeting in for-profit firms, bearing in mind that this is a complex task due to the non-monetary and intangible impacts involved in productive assets investment, which have traditionally been ignored by classical financial techniques. thus, it has been assumed that investment appraisal in for-profit firms should take into account not only financial value creation, that is, the contribution of new assets to the cash-flows of the firm (as npv and irr do), but also the creation of nonfinancial capital value derived from the increase in corporate intellectual capital. given the intangible nature of nonfinancial capital value, as well as its three components (human capital, structural capital and relational capital), an ahp-based model was formulated to quantify and assess both the criteria and subcriteria related to financial value creation and the intangibles related to nonfinancial capital value. in this way, we tried to (partially) avoid the problem of estimating how much cash flows will increase associated with the intellectual capital generation due to productive investments, and the problem of fixing this when it does occur. the proposed investment appraisal model has been applied to a real case study in order to assess several quality control investment alternatives in the meat industry. the empirical application of our model to this case study has demonstrated its feasibility and effectiveness in a real setting and also evidenced the need to consider the impacts of productive investments on a firm’s intangible assets, as these impacts actually affect the selection of optimal investment alternatives in the food industry. furthermore, it is also worth indicating that the model proposed could be applied to any investment decision, albeit following adaptation to each particular case study in order to initially establish the specific intangibles (components or subcriteria related to human capital, structural capital and relational capital) that the investment alternatives under consideration may have an impact on. the results achieved in the empirical application suggest that there is no common pattern explaining the distribution of intellectual capital components. indeed, why firms focus on one type of intellectual capital or another is determined by an organization’s own strategies. therefore, it is necessary for financial decision-makers to link the investment project valuation process to the business strategy followed in order to give more weight to attributes related to it. this diversity of business strategies (disparity in the contribution of financial capital and intangible capital to firms’ total value) can be seen in an integrative way through the methodology proposed here. nevertheless, the conclusions above are still tentative, since they cannot be definitively confirmed based only on a pilot study with an empirical application reduced to five companies. therefore, further empirical investigation is needed to validate these findings. it is suggested that the approach proposed here be implemented with larger samples of firms in different economic sectors and for ijahp article: guerrero-baena, gómez-limón/ decomposing value creation when assessing investments: a multicriteria approach based on the analytic hierarchy process international journal of the analytic hierarchy process 174 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.236 different types of investments. the validity of the proposed methodology and corroboration (or not) of the main conclusions can only be achieved by testing the adaptation of this approach to different contexts in this way. in any case, this paper is expected to provide a contribution to the current literature by providing the development of an investment decision model that can be used by any for profit-firm, entailing the consideration of nonfinancial value creation criteria into the investment valuation process. this novel approach aims to achieve more effective decision-making in order to select the investment alternatives that maximize total value creation or firms’ market value. ijahp article: guerrero-baena, gómez-limón/ decomposing value creation when assessing investments: a multicriteria approach based on the analytic hierarchy process international journal of the analytic hierarchy process 175 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.236 references anderson, e. w., fornell, c., & mazvancheryl, s. k. (2004). customer satisfaction and shareholder value. journal of marketing, 68(4), 172–185. doi:10.1509/jmkg.68.4.172.42723 aznar, j., cervelló, r., & romero, a. (2011). new spanish banking conglomerates. application of the analytic hierarchy process (ahp) to their market value. international research journal of finance and economics, 78, 70–82. baker, h. k., & english, p. (2011). capital budgeting: an overview. in h. k. baker & p. english (eds.), capital budgeting valuation: financial analysis for today´s investment projects (pp. 1–16). new jersey: wiley. becker, b. e., huselid, m. a., & ulrich, d. (2001). the hr scorecard: linking people, strategy, and performance. massachusetts: harvard business press. bontis, n. (1998). intellectual capital: an exploratory study that develops measures and models. management decision, 36(2), 63–76. bontis, n. (1999). managing organizational knowledge by diagnosing intellectual capital: framing and advancing the state of the field. international journal of technology management, 18(5-8), 433–462. bozbura, f. t., beskese, a., & kahraman, c. (2007). prioritization of human capital measurement indicators using fuzzy ahp. expert systems with applications, 32(4), 1100–1112. doi:10.1016/j.eswa.2006.02.006 brounen, d., de jong, a., & koedijk, k. (2004). corporate finance in europe: confronting theory with practice. financial management, 33(4), 71–101. brigham, e. f., & ehrhardt, m. c. (2008). financial management: theory and practice. mason, oh: thomson south-western. byrd, j., parrino, r., & pritsch, g. (1998). stockholder-manager conflicts and firm value. financial analysts journal, 54(3), 14–30. cañibano, l., garcia-ayuso, m., & sánchez, p. (2000). accounting for intangibles: a literature review. journal of accounting literature, 19, 102–130. carlucci, d., & schiuma, g. (2007). knowledge assets value creation map: assessing knowledge assets value drivers using ahp. expert systems with applications, 32(3), 814–821. doi:10.1016/j.eswa.2006.01.046 chang, s. c., chen, s. s., & lai, j. h. (2008). the effect of alliance experience and intellectual capital on the value creation of international strategic alliances. omega, 36(2), 298–316. doi:10.1016/j.omega.2006.06.010 chen, j., zhu, z., & xie, h. y. (2004). measuring intellectual capital: a new model and empirical study. journal of intellectual capital, 5(1), 195–212. ijahp article: guerrero-baena, gómez-limón/ decomposing value creation when assessing investments: a multicriteria approach based on the analytic hierarchy process international journal of the analytic hierarchy process 176 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.236 claver, e., lópez, m. d., molina, j. f., & tarí, j. j. (2007). environmental management and firm performance: a case study. journal of environmental management, 84(4), 606–619. doi:10.1016/j.jenvman.2006.09.012 coleman, s. (2007). the role of human and financial capital in the profitability and growth of women-owned small firms. journal of small business management, 45(3), 303–319. doi:10.1111/j.1540-627x.2007.00214.x curado, c., henriques, l., & bontis, n. (2011). intellectual capital disclosure payback. management decision, 49(7), 1080–1098. d´allura, g. m., & minichilli, a. (2012). family business and competitive strategy research. in g. b. dagnino (ed.), handbook of research on competitive strategy (pp. 346–358). cheltenham: edward elgar. dayananda, d., irons, r., harrison, s., herbohn, j., & rowland, p. (2002). capital budgeting: financial appraisal of investment projects. cambridge: cambridge university press. díez, j. m., ochoa, m. l., prieto, m. b., & santidrián, a. (2010). intellectual capital and value creation in spanish firms. journal of intellectual capital, 11(3), 348–367. edvinsson, l. (2013). ic 21: reflections from 21 years of ic practice and theory. journal of intellectual capital, 14(1), 163–172. edvinsson, l., & malone, m. s. (1997). intellectual capital: realizing your company´s true value by finding its hidden brainpower. new york: harperbusiness. edvinsson, l., & sullivan, p. (1996). developing a model for managing intellectual capital. european management journal, 14(4), 356–364. doi:10.1016/02632373(96)00022-9 fichtner, j. (1986). on deriving priority vectors from matrices of pairwise comparisons. socio-economic planning sciences, 20(6), 341–345. doi:10.1016/00380121(86)90045-5 firouzabadi, s. m., henson, b., & barnes, c. (2008). a multiple stakeholders’ approach to strategic selection decisions. computers & industrial engineering, 54(4), 851–865. doi:10.1016/j.cie.2007.10.015 freeman, r. e. (1984). strategic management: a stakeholder approach. boston: pitman. götze, u., northcott, d., & schuster, p. (2008). investment appraisal: methods and models. berlin: springer. graham, j. r., & harvey, c. r. (2001). the theory and practice of corporate finance: evidence from the field. journal of financial economics, 60(2), 187–243. doi:10.1016/s0304-405x(01)00044-7 grigoroudis, e., tsitsiridi, e., & zopounidis, c. (2013). linking customer satisfaction, employee appraisal, and business performance: an evaluation ijahp article: guerrero-baena, gómez-limón/ decomposing value creation when assessing investments: a multicriteria approach based on the analytic hierarchy process international journal of the analytic hierarchy process 177 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.236 methodology in the banking sector. annals of operations research, 205(1), 5–27. doi:10.1007/s10479-012-1206-2 habbershon, t. g., williams, m., & macmillan, i. c. (2003). a unified systems perspective of family firm performance. journal of business venturing, 18(4), 451– 465. doi:10.1016/s0883-9026(03)00053-3 hall, r. (1992). the strategic analysis of intangible resources. strategic management journal, 13(2), 135–144. doi:10.1002/smj.4250130205 han, d., & han, i. (2004). prioritization and selection of intellectual capital measurement indicators using analytic hierarchy process for the mobile telecommunications industry. expert systems with applications, 26(4), 519–527. doi:10.1016/j.eswa.2003.10.010 hitt, m. a., biermant, l., shimizu, k., & kochhar, r. (2001). direct and moderating effects of human capital on strategy and performance in professional service firms: a resource-based perspective. academy of management journal, 44(1), 13–28. doi:10.2307/3069334 huselid, m. (1995). the impact of human resource management practices on turnover, productivity, and corporate financial performance. academy of management journal, 38(3), 635–672. doi:10.2307/256741 irudayaraj, j., & reh, c. (eds.). (2008). nondestructive testing of food quality. ames, ia: wiley-blackwell. itami, h. (1987). mobilizing invisible assets. boston, ma: harvard university press. jensen, m. c., & meckling, w. h. (1976). theory of the firm: managerial behavior, agency costs and ownership structure. journal of financial economics, 3(4), 305– 360. johnson, w. h. a. (1999). an integrative taxonomy of intellectual capital: measuring the stock and flow of intellectual capital components in the firm. international journal of technology management, 18(5), 562–575. kamenetzky, r. d. (1982). the relationship between the analytic hierarchy process and the additive value function. decision sciences, 13(4), 702–713. koontz, h., & weihrich, h. (2007). essentials of management: an international perspective. new delhi: tata mcgraw-hill. kreng, v. b., wu, c. y., & wang, i. c. (2011). strategic justification of advanced manufacturing technology using an extended ahp model. international journal of advanced manufacturing technology, 52(9-12), 1103–1113. doi:10.1007/s00170010-2805-1 kristandl, g., & bontis, n. (2007). constructing a definition for intangibles using the resource based view of the firm. management decision, 45(9), 1510–1524. lev, b. (2004). sharpening the intangibles edge. harvard business review, 82(6), 109–116. ijahp article: guerrero-baena, gómez-limón/ decomposing value creation when assessing investments: a multicriteria approach based on the analytic hierarchy process international journal of the analytic hierarchy process 178 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.236 likert, r. (1967). the human organization: its management and values. new york: mcgraw hill. lópez-gamero, m. d., zaragoza-sáez, p., claver-cortés, e., & molina-azorín, j. f. (2011). sustainable development and intangibles: building sustainable intellectual capital. business strategy and the environment, 20(1), 18–37. doi:10.1002/bse.666 marr, b., schiuma, g., & neely, a. (2004). the dynamics of value creation: mapping your intellectual performance drivers. journal of intellectual capital, 5(2), 312–325. martín-de-castro, g., delgado-verde, m., lópez-sáez, p., & navas-lópez, j. e. (2011). towards “an intellectual capital-based view of the firm”: origins and nature. journal of business ethics, 98(4), 649–662. doi:10.1007/s10551-010-0644-5 montabon, f., sroufe, r., & narasimhan, r. (2007). an examination of corporate reporting, environmental management practices and firm performance. journal of operations management, 25(5), 998–1014. doi:10.1016/j.jom.2006.10.003 naffziger, d. w., ahmed, n. u., & montagno, r. v. (2003). perceptions of environmental consciousness in us small businesses: an empirical study. sam advanced management journal, 68(2), 23–32. pérez-marín, d., de pedro sanz, e., guerrero-ginel, j. e., & garrido-varo, a. (2009). a feasibility study on the use of near-infrared spectroscopy for prediction of the fatty acid profile in live iberian pigs and carcasses. meat science, 83(4), 627–633. doi:10.1016/j.meatsci.2009.07.012 phusavat, k., comepa, n., sitko-lutek, a., & ooi, k. b. (2011). interrelationships between intellectual capital and performance: empirical examination. industrial management & data systems, 111(6), 810–829. porter, m. e., & van der linde, c. (1995). green and competitive: ending the stalemate. harvard business review, 73(5), 120–134. roos, g., pike, s., & fernström, l. (2006). managing intellectual capital in practice. oxford: butterworth-heinemann. roos, g., & roos, j. (1997). measuring your company’s intellectual performance. long range planning, 30(3), 413–426. roos, j., roos, g., dragonetti, n., & edvinsson, l. (1998). intellectual capital: navigating in the new business landscape. new york: new york university press. ross, s. a., westerfield, r. w., & jordan, b. d. (2007). essentials of corporate finance. boston: mcgraw-hill. saaty, t. l. (1980). the analytic hierarchy process. new york: mcgraw-hill. saaty, t. l. (2003). decision-making with the ahp: why is the principal eigenvector necessary. european journal of operational research, 145(1), 85–91. doi:10.1016/s0377-2217(02)00227-8 ijahp article: guerrero-baena, gómez-limón/ decomposing value creation when assessing investments: a multicriteria approach based on the analytic hierarchy process international journal of the analytic hierarchy process 179 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.236 saaty, t. l., & vargas, l. g. (2000). models, methods, concepts & applications of the analytic hierarchy process. boston: kluwer academic publishers. saaty, t. l., vargas, l. g., & dellmann, k. (2003). the allocation of intangible resources: the analytic hierarchy process and linear programming. socio-economic planning sciences, 37(3), 169–184. doi:10.1016/s0038-0121(02)00039-3 sánchez, p., chaminade, c., & olea, m. (2000). management of intangibles: an attempt to build a theory. journal of intellectual capital, 1(4), 312–327. srdjevic, b. (2005). combining different prioritization methods in the analytic hierarchy process synthesis. computers & operations research, 32(7), 1897–1919. doi:10.1016/j.cor.2003.12.005 steuer, r. e., & na, p. (2003). multiple criteria decision making combined with finance: a categorized bibliographic study. european journal of operational research, 150(3), 496–515. doi: 10.1016/s0377-2217(02)00774-9 stewart, t. a. (1997). intellectual capital: the new wealth of organizations. new york: doubleday/currency. sullivan, p. h. (2000). value driven intellectual capital: how to convert intangible corporate assets into market value. toronto: wiley. sveiby, k. e. (1997). the new organizational wealth: managing and measuring knowledge-based assets. san francisco: berrett-koehler. vaidya, o. s., & kumar, s. (2006). analytic hierarchy process: an overview of applications. european journal of operational research, 169(1), 1–29. doi:10.1016/j.ejor.2004.04.028 villalonga, b. (2004). intangible resources, tobin’s q, and sustainability of performance differences. journal of economic behavior & organization, 54(2), 205– 230. doi:10.1016/j.jebo.2003.07.001 wu, w. y., chang, m. l., & chen, c. w. (2008). promoting innovation through the accumulation of intellectual capital, social capital, and entrepreneurial orientation. r&d management, 38(3), 265–277. doi:10.1111/1467-9914.00120-i1 youndt, m. a., subramaniam, m., & snell, s. a. (2004). intellectual capital profiles: an examination of investments and returns. journal of management studies, 41(2), 335–361. doi:10.1111/j.1467-6486.2004.00435.x microsoft word 5saatyessay_pp61_62_03212009 ijahp essay: saaty/how judgment and measurement are different in science and in decision making international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 61 an essay on how judgment and measurement are different in science and in decision making thomas l. saaty university of pittsburgh pittsburgh, pa e-mail: saaty@katz.pitt.edu abstract in decision making the priority scales are derived objectively after subjective judgments are made, and they reflect the importance of the influences we considered. the process is the opposite of what is done in the physical sciences where the subjectivity of interpreting the final number comes at the end. keywords: ahp decision making, priorities, judgment, objective science, subjective science essay in science, measurements of factors with different ratio scales are combined by means of formulas. the formulas apply within structures and involve variables and their relations. each scale has a zero as an origin and an arbitrary unit applied uniformly in all measurements on that scale but the meaning of the unit remains elusive and only becomes well understood through much practice and use. the meaning and use of the outcome of any measurement on a ratio scale must in the end be interpreted according to the judgment of an expert as to how well it meets our expectations and our experience with it for the situation in which it is being applied or how well it satisfies laws of nature that are always there. the composite numbers derived from formulas that combine the different scales must similarly be understood through experience. science derives results using numbers objectively; that is, everyone gets the same numbers. but their significance is interpreted subjectively; that is, how well the number serves individual or group goals and understanding. in decision making, however, because of the diversity of influences with which it is concerned, and the many decisions that may arise, there are no set laws that characterize commonly encountered structures in such fine detail like there are in science. understanding and familiarity with a situation is needed to structure a problem and the judgment of a human being is always needed to capture importance, preference or likelihood. with the analytic hierarchy process (ahp) these judgments are expressed quantitatively on a common absolute scale that denotes dominance of one element over another so that a best outcome can be derived by combining and trading off different factors or attributes. in the end after applying the ahp rules of composition a multidimensional scaling problem is reduced to a one-dimensional scale of priorities that are relative sets of numbers which belong to an absolute scale. so in the ahp significance is interpreted subjectively at the beginning of the process through judgments, then priority numbers are rob typewritten text http://dx.doi.org/10.13033/ijahp.v1i1.14 rob typewritten text ijahp essay: saaty/how judgment and measurement are different in science and in decision making international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 62 objectively derived from them; that is, everyone would get the same results if they started with the same judgments. priorities are similar to probabilities. the ratios of ahp priorities are meaningful: for example, a priority of .50 is twice a priority of .25. a priority scale is not the same as the ratio scales used in science. ratio scales are like yardsticks in that they have a starting point or a zero and a unit. priority scales do not have a zero nor do they have a fixed and invariable unit. a priority scale is relative in that it is specially derived for a given situation with given factors and alternatives and it is applicable only to that situation. it is not good for all situations and all time like a yardstick is. when the situation changes the priorities may change. in decision making the priority scales are derived objectively after subjective judgments are made, and they reflect the importance of the influences that we take into consideration. the process is the opposite of what is done in the physical sciences when the subjectivity of interpreting what the final number means comes at the end. of course there has to be validation of our decision process through many examples to show it works, gives good answers and corresponds to reality somehow. as in other softer sciences it takes time to validate and establish a reliable methodology. finally, in science measurement is applied uniformly using the same arbitrary unit from the very small to the very large. this cannot be done using judgment in decision making. judgment can only be applied meaningfully to homogeneous groups of elements, otherwise they must be put into different clusters with a common pivot element from one cluster to the next to make it possible to compare the elements in each cluster and then combine measurements across clusters. in the physical sciences there is no way to compare the significance of very small numbers with very large numbers in a systematic and meaningful way except by speaking of orders of magnitude. the meaning of the unit does not change from one order of magnitude to another, thus in the physical sciences the interpretation is left subjective and loose. we look forward to exploring in another article how to bring together science and mathematics based on cartesian axes and decision making with its relative priorities. rob typewritten text rob typewritten text rob typewritten text http://dx.doi.org/10.13033/ijahp.v1i1.14 rob typewritten text ijahp news and events: saaty, mu/t.l. saaty decision making for leaders hackathon international journal of the analytic hierarchy process 1 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.987 t.l. saaty decision making for leaders hackathon rozann saaty creative decisions foundation, president, pittsburgh, usa enrique mu international journal of the analytic hierarchy process, pittsburgh, usa how long does it take to learn and use ahp for managerial decision making? based on a recent experience, it can take a surprisingly short time! students from the university of pittsburgh competed in the first t.l. saaty decision making for leaders hackathon from march 18 to 20, 2022. the hackathon was jointly sponsored by the university of pittsburgh business school and creative decisions foundation (cdf), the nonprofit established by thomas l. and rozann w. saaty to promote the cause of rational decision-making. cdf is a community-based initiative, bringing academia together with businesses, nonprofits, and other organizations. undergraduate and graduate students at the university of pittsburgh were invited to participate in the threeday event and could register individually or in teams of up to three participants. applicants were told they did not need any prior knowledge of ahp as there would be training in ahp and anp methods and related software tools in the morning during the first day of the hackathon. executives from microsoft, the partner company for the hackathon, would present the main concerns of the company and then based on these concerns the teams would select their decision topic. the saaty hackathon hackathons originated as competitive events for programmers that last two or three days, and involve programmers writing code for a challenging problem. these programmers frequently stay up all night to work on the problem. sleeplessness is a point of pride in the programming world, so these programmers adapted well to the speed of the competition. recently, hackathons have moved into the business world with the objective of finding solutions to given industry problems or exploring a particular technology's possibilities. microsoft was chosen as the business partner for the inaugural t.l. saaty hackathon. ijahp news and events: saaty, mu/t.l. saaty decision making for leaders hackathon international journal of the analytic hierarchy process 2 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.987 microsoft executives explain the company’s challenges day one during the first day, microsoft executives dan menicucci, john yokim, and chris jones introduced the aims and strategic objectives of the company and the issues they are facing or have faced in the past. these issues include questions such as should they move more to the cloud and expand their server farms, how can they ameliorate the risks associated with security issues, and how can they attract and retain good employees? also, a current ambition for most companies is to be as green as possible, acknowledge that the environment is deteriorating, and work to limit their impact on it. after the microsoft executive’s presentations, participants were able to have hands-on practice with the decision-making software tools that would be used the next day. at the end of the day, the teams selected their projects and began discussions about how to proceed. certificates in “decision-making for leaders” were awarded to participants who completed the first day's training. dr. elena rokou, chief research scientist of creative decisions foundation, training the teams in ahp/anp methodology and the software tools ijahp news and events: saaty, mu/t.l. saaty decision making for leaders hackathon international journal of the analytic hierarchy process 3 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.987 day two on the second day, the work began as the teams separated into rooms in the university of pittsburgh business school to begin structuring their decision models. dr. elena rokou and dr. lirong wei, both of creative decisions foundation, circulated throughout the building to offer advice and help. the first task for the teams was to clearly define their decision-making problem, and then structure the framework, identify the decision factors, and model the problem, using the software tools they had been introduced to the day before. the teams finalized their case studies in the afternoon of the second day and set up their presentations for the judges to be presented the following day. day three on the third day, the teams presented their decision projects before the preliminary judging panels that were hybrid with some present on zoom and some local judges in the room. to emphasize the global reach of the ahp community, some of the experts that zoomed in from abroad were bill wedley (canada), rafikul islam (malaysia), iwan azis (indonesia), shashi bhattarai (nepal) and mujgan sagir (turkey). luis vargas, marcel minutolo, john saaty, birsen karpak and enrique mu were among the expert judges that participated in person. teams were allotted fifteen minutes to present, followed by a tenminute question and answer (q&a) session. four finalist teams from the first round presented their projects again before the final judging panel. the final judging session was open to the public and non-finalist teams that wished to sit in. metaverse team in action some of the finalists after 2 nd round of judging the members of the winning team were shri krishna kumar (shri) and enkhjargal (taqi) ganbaatar shown with rozann saaty receiving their winning project prize of $3,000. shri is a graduate student in industrial engineering (from india) and taqi is a graduate student in marketing (from japan) at the pitt business school. they did not know each other and neither knew anything about the ahp before signing up for the competition. ijahp news and events: saaty, mu/t.l. saaty decision making for leaders hackathon international journal of the analytic hierarchy process 4 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.987 they were assigned to the same team by the program organizers at the beginning of the first day. shri and taqi receiving their winning prize check from cdf president rozann saaty. the winning decision shri and taqi’s decision project was to figure out, using ahp, how microsoft could better retain employees. microsoft employs approximately 181,000 people worldwide in full-time positions with around 60% residing in the united states. during the current "great resignation", many employees have quit their jobs and, like many other companies, microsoft is struggling with this reality. the winning team’s decision was that microsoft should develop an ahp app to prioritize the attributes of open jobs. job applicants, using the same app, would derive their own personal job priority vectors. an applicant whose personal priority vector was compatible with an open job priority vector, using the ahp compatibility index, would be offered the position. employees hired this way would be less likely to quit. shri krishna kumar (shri) on the right, enkhjargal (taqi) ganbaatar on the left, and dr. elena rokou in the center, after their first-round presentation. ijahp news and events: saaty, mu/t.l. saaty decision making for leaders hackathon international journal of the analytic hierarchy process 5 vol. 14 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i1.987 the epilogue it has been reported by many who teach decision making that their students like the approach very much and find it easy to use. the experience of the winning team seems to confirm this. the way their project unfolded is interesting. shri and taqi did not know anything about the ahp and had not even met each other before the hackathon, but were able to quickly catch on to the ahp by working together. also, their professional experiences were complementary. taqi had worked in japan in human resources (hr), so the hiring issue attracted her. shri, with his background in engineering technology and computer skills, could cope with the theory and software. they realized that matching the priority vectors of employers and employees could be extremely useful in making sure new hires stay with their jobs and were able to develop the winning proposal. interestingly, they had not realized there would be a monetary award until they won the prize. their decision to participate, though not arrived at using ahp, was their first winning decision! . organizers and judges celebrate the successful completion of the first t. l. saaty hackathon ijahp article: brandao, de alencar naas, de oliveira costa neto/sustainable development of gem production: an analysis using the analytic hierarchy process international journal of the analytic hierarchy process 483 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.880 sustainable development of gem production: an analysis using the analytic hierarchy process lilane brandao lilanebrandao@gmail.com irenilza de alencar naas irenilza.naas@docente.unip.br pedro luiz de oliveira costa neto abstract the gems, jewelry, and precious metals sector is essential for developing regions with this activity and there is a high degree of informality in the mining and trading gemstone sectors. the present study analyzed data from the production process of precious stones in the brazilian national market. we identified the importance of the factors contributing to the quality of the gemstone production process concerning mining, cutting, and marketing, under the scope of sustainable chain development. a list of references was used to select thirteen significant factors in the precious stone production process that contributed to the sustainable development of the chain related to mining, cutting, and marketing. we used a multicriteria decision model applying the analytical hierarchy process (ahp) to consider the degree of importance of the production stage process of gemstones and the factors that impact the sustainable development of the production chain. in applying the model to the production stages, the criteria and sub-criteria factors were considered. the results suggest that sustainable development is highly related to gem mining (44.7%, rank 1), moderately related to gem cutting (32.5%, rank 2), and there is little relationship to gem marketing (22 .8%, rank 3), considering the criteria, subcriteria, and alternatives. the requirement related to the technical qualification and skill of the lapidary is the most critical factor in the production process. the second most important factor is commercialization, followed by the preservation of the mining area and the valuable life of the mine. keywords: regional development; semi-precious gemstone; mining; sustainability; supply chain; opal production 1. introduction brazil is recognized for its mineral wealth and variety of gems. the gems, jewelry, and precious metals sector in the country has a high degree of informality in the segments of mining and marketing precious stones. the exploitation of brazilian mineral activity is essential for developing some regions, despite numerous criticisms regarding its predatory nature (ribeiro, 2011). changes in the national and international context have forced organizations to innovate, manage change and promote the sustainable development of their products or services. mailto:lilanebrandao@gmail.com mailto:irenilza.naas@docente.unip.br ijahp article: brandao, de alencar naas, de oliveira costa neto/sustainable development of gem production: an analysis using the analytic hierarchy process international journal of the analytic hierarchy process 484 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.880 the emergence of the concept of sustainability reflects a transformation in global thinking that forces organizations to reconsider how their activities are developed to boost economic development, reducing the negative impacts on the environmental, social, and economic dimensions of the production of goods and services (alves, 2020). this scenario presents challenges for the mineral sector, increasing the complexity of extraction, production, and commercialization. the complexity is due to the interaction between different aspects such as economic factors, environmental and social issues, and government regulations. thus, it is up to the sector to develop strategies to improve quality and services that meet sustainability requirements (arango-aramburo et al., 2017). sustainable development is a continuous process that integrates activities in the economic scope, ensuring economic growth; in the social aspect, taking care of the worker in the work environment and the development of the community; and with the environmental elements, ensuring the protection of natural resources and the environment (dubinski, 2013). however, the current literature did not use the triple bottom line approach to identify a vision of sustainable development in the gem mining sector. thomas saaty developed the ahp in 1990. it offers a system where one can arrange the data of a problem in a hierarchical and judgment-based way and establish priorities for decision making (camanho & moraes, 2007). several academic studies and applications present ahp as a multicriteria decision system that can help production systems (reis et al., 2017; saaty & shang, 2011; reck & schultz, 2016; wang et al., 2019). periaiah et al. (2021) underlined the consequences of bauxite mining on the environment through the impacts of the environment, ecology, and economy using multicriteria analysis. shen et al. (2015) investigated an appropriate implementation approach of green supply chain management practice in the indian mining sector using the ahp. due to the importance and complexity of the gem mining sector in brazil, the present study aimed to evaluate the extent of factors contributing to brazil's sustainable gem production process. in the past decade, there has been increasing discussion about the necessity to shift modern mining to a more sustainable framework, with many mining companies now reporting annually on their sustainability performance alongside their financial results (shen et al., 2015). bui et al. (2017) proposed an indicator-based sustainability assessment framework to assess the sustainability of the mining sector on a national and global scale. sauer & seuring (2017) conducted a systematic literature review on sustainability in mineral supply chains and suggested adopting practices for improving sustainability in the mineral supply chain. the present study focuses on sustainable development issues in the gem mining context and provides suggestions for the decision-making of regional policies. we considered the economic, social, and environmental components (triple bottom line) of this process, in a regional aspect, respecting the drive of the production chain, available human resources, technologies, legislation, and institutions that support the mining activity. we also verified if the selection presents relevant results for decision-making that leads to sustainable development for the gem production process. thus, the study sought to interpret the interdimensional relationships of the gem production chain to guarantee the process results are within the scope of sustainable development applying the analytic hierarchy process (ahp). ijahp article: brandao, de alencar naas, de oliveira costa neto/sustainable development of gem production: an analysis using the analytic hierarchy process international journal of the analytic hierarchy process 485 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.880 2. background 2.1 gems production chain overview the union is responsible for organizing the administration of mineral resources, the mineral production industry, and the distribution, trade, and consumption of mineral products (brasil, 2018). the production of brazilian gems is concentrated in the states of minas gerais, rio grande do sul, bahia, goiás, pará and tocantins (ribeiro, 2011). the performance of the gems, jewelry, and precious metals sector decreased 7.3% in terms of the number of organizations. direct jobs dropped by 5.9% between 2015 to 2017 when observing the wholesale trade jewelry, watches, jewelry including cut precious and semiprecious stones, gem extraction (precious and semi-precious), and gem cutting (ibgm, 2018). according to the brazilian institute of gems and precious metals (ibgm) (ibgm, 2018), commerce is the segment that employs the most people, followed by mining and then the polishing industry. this indicates the low level of transformation of the products present in the supply chain. the production chain includes mining, polishing, production of stone artifacts, the jewelry and veneer industry, jewelry, raw materials, and equipment used in the production process, in addition to activities related to design (ribeiro, 2011). the ministry of mines and energy (mme) considers the production chain of gemstones linked to the jewelry and mineral handicraft sector, represented by the following four aspects: a) the mining and all materials and services used in the extraction of rough stone; b) the stone cutting and artifact industry, encompassing the production of cut stones, handicrafts, and stone artifacts; c) the jewelry and costume jewelry industry, responsible for the production of gold, silver, veneer, and ordinary metal jewelry; and d) the commercialization, both in the domestic and foreign markets. mining and cutting are the links in the chain that present the greatest informality and face significant obstacles to development and are considered declining activities. although the other links have low global participation, they already have greater competitiveness and organization due to the qualitative gains in design and valorization of the national product (ribeiro, 2011). gems can play a symbolic force towards rebuilding the tarnished image of post-conflict areas. conflict regions that previously led to negative marketing of gem brands can, over time, be modified into post-conflict development opportunities and lead to positive brands (makki & ali, 2019). 2.2 mining the mining activity contributes to economic and social development by supplying essential minerals for the development of industries, increasing jobs and services and technological progress, which positively impacts the well-being of the population and the economical functioning of the global economy. however, negative impacts are also found in the mining activity, such as labor diseases related to mining activities, the effects on the landscape, changes in the course of rivers, unemployment, social conflicts, which sometimes occur even after the closure of the mines (alves et al., 2020; arangoaramburo et al., 2017). mining activities face complex problems through the interaction of many different interests of stakeholders including companies, legislators, regulators, the local population, illegal miners, and a global market. these parties are inserted in a context of interdependence characterized by high uncertainty, fluctuating price and ijahp article: brandao, de alencar naas, de oliveira costa neto/sustainable development of gem production: an analysis using the analytic hierarchy process international journal of the analytic hierarchy process 486 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.880 demand, fluctuating rules, and unpredictable environmental and social consequences (arango-aramburo et al., 2017). the environmental impact caused by gemstone mining is considered simple compared to other types of mineral mining since it does not use chemicals and is often restricted to small areas. the most common impacts highlighted in the literature are deforestation, water pollution, siltation of rivers, and landscape alteration caused by ditches, holes, and tunnels (milanez & oliveira, 2013). to mitigate potential impacts from mining activities, organizations must adopt sustainable development strategies related to the environment and society such as conceptualizing sustainability, adopting methods for sustainable development of products and services, supporting companies in their sustainability efforts, and emphasizing the importance of effective communication (alves et al., 2020). public agencies could develop programs that encourage miners to create new products using waste products (mme, 2018). technological advances in the mining sector decrease pollution, production costs, and productivity making the process more efficient. in addition to technological advancement, mining must be concerned with price, usually the main uncertainty linked to the activity (arango-aramburo et al., 2017). the promotion of associative entrepreneurship in mining improves the relationship of the agents involved with the government. it allows miners to accumulate the financial capital necessary to obtain a cleaner and more productive technology (saldarriaga-isaza et al., 2013). with the increase in jobs created, social conflicts tend to decrease, which leads to a rise in attractiveness. other factors can also make the sector more attractive, for instance, advancements in infrastructure, security, legal stability, and political stability in the country (arango-aramburo et al., 2017). in brazil, the activity of mineral extraction is regulated by the mining code. this code governs the regimes for the extraction of mineral resources, shown in table 1 (brasil, 2018). ijahp article: brandao, de alencar naas, de oliveira costa neto/sustainable development of gem production: an analysis using the analytic hierarchy process international journal of the analytic hierarchy process 487 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.880 table 1 rules for the use of mineral resources in brazil rule description rule of concession depends on the ordinance of the minister of state for mines and energy or when granted by the national mining agency (anm) if it has as its object the mineral substances referred to in art. 1 of law no. 6567 1978. rule of authorization depends on license issuance by anm. rule of licensing when dependent on a license issued in compliance with local administrative regulations and registration of the claim with the anm mining permit rule depends on license issuance by anm. monopolization rule as a result of a particular law, it depends on the direct or indirect execution of the federal executive branch. source: brasil (2018) national interest and public utility are fundamental for the development of mining. mineral deposits are characterized by their locational rigidity, which expresses the restriction in selecting areas to implement the activity to generate lesser environmental impacts. they are finite and have economic value. the mining activity includes research, mining, mine development, processing, commercialization of ores, use of tailings and waste, and mine closure (brasil, 2018). the determination of the mine size made by the anm considers the annual mineral production to define its category (table 2). artisanal mining, in turn, constitutes a portion of mining characterized by the use of traditional and inefficient technology and by poorly qualified labor, almost always practiced in independent units, often informal, without titles or licenses, following a business model family and subsistence. most cases involve micro and small mining operations, but there are situations that involve medium-sized or even large-scale mining operating artisanal (mme, 2018). table 2 mining operation category and production category annual production large mine ≥ 1 million t/year mina média 100 thousand to 1 million t/year small mine 10 thousand to 100 thousand t/year micro mine ≤10 thousand t/year source: mme (2018) ijahp article: brandao, de alencar naas, de oliveira costa neto/sustainable development of gem production: an analysis using the analytic hierarchy process international journal of the analytic hierarchy process 488 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.880 although battled by the government, illegal mining often occurs. the formalization of mining activity must be considered a priority for the sector. one way of tackling informality is issuing formal property titles that bind the miner to the land. land title ownership results in avoiding evasion and discouraging the practice of inefficient transient mining and the degradation of the environment (milanez & oliveira, 2013). the informality of artisanal and small-scale mining leads to a reduction in industry contributions to local development, which causes an impact on the economy through underreporting and underestimating production and eliminating formal resources economy through tax evasion. in addition, informality tends to influence inappropriate environmental, social, and labor practices and perpetuate social conflicts and corrupt regimes in certain countries (makki & ali, 2019). 2.3 gem-cutting india, china, and thailand are lapidary centers and dominate the jewelry-making industry. colombia, sri lanka, and asia also benefit from the gems industry to substantiate sustainable livelihoods for economic growth (makki & ali, 2019). in brazil, the main polishing poles are located in são paulo, minas gerais, rio de janeiro, and rio grande do sul. in some cases, the gems exported in a raw state are polished in specialized workshops in other countries and return to brazil with values higher than those for which they were marketed (mme, 2018). the use of rudimentary polishing techniques and the scarcity of skills in processing gems lead to low economic and technological efficiency in the production chain of gems and jewels. the cutting techniques can be divided into three categories as follows: (1) cabochon cutting, generally used in opaque gems, (2) faceted cutting, used in transparent gems, and (3) mixed cutting, which is a combination of the other two techniques (mol et al., 2016; mme, 2018; makki & ali, 2019). the study of cutting and designer models can favor the best use of the gemstone and add value to the production chain. characteristics such as the enhancement and utilization ratio between the weight of the raw and polished material are fundamental properties for gem valorization (mol et al., 2016). in 2019, the brazilian market reimported us$77,706 (commercial value) in precious and semi-precious cut stones (except diamond). the need for importing processed gemstones has been expanding since 2015, with a decrease in polishing and mineral handicraft workshops in the national market (uncomtrade, 2020). actions must be established so that this industry can develop in such a way as to expand the use of brazilian gems in handicrafts, cutting and the manufacture of jewelry, and therefore generate employment and income in the country (mol et al., 2016). in the production chain of gems, a fundamental aspect is the design of the pieces. increasingly, jewelers use the improved design to differentiate themselves, add value to the gemstone, creating a visual identity. it is considered one of the main factors for an organization's success, from product development to marketing through cost optimization, packaging, promotional material, aesthetic standards, visual identity, material adequacy, manufacturing, and ergonomics. in addition, it is an essential factor in the planning, production, and marketing strategy (mme, 2018; milanez & oliveira, 2013; landim, 2010). the designer's responsibilities include being the interpreters of our desires, aspirations, and anxieties in addition to designing the components of the product itself, thereby creating a cultural identity for the product. they must understand the ijahp article: brandao, de alencar naas, de oliveira costa neto/sustainable development of gem production: an analysis using the analytic hierarchy process international journal of the analytic hierarchy process 489 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.880 production techniques and the possibilities and obstacles of each material (landim, 2010). 2.4 commercialization during the 18 th century, mining society paraded adornments of colored gems set in gold and silver as a way of showcasing their fortunes and marking territory. there were no specialized craftsmen in brazil to cut or carve stones, which kept the commercial relationship of jewels between brazil and portugal (oliveira, 2019). brazil is recognized as one of the most important countries in the international market for mineral commodities due to its large mineral reserves and quality of mines and a significant producer and exporter of high-quality minerals, making mining a vital pillar of the brazilian economy (alves, 2020). the country is currently the largest exporter of raw agates and amethysts. almost 90% of extracted gemstones are exported, failing to generate employment and income in the cutting sector (mme, 2018). data on the formal export of precious and semi-precious stones (except diamonds) for the year 2019 show us$39 million for exports of rough gems and us$305 million for exports of cut gems (uncomtrade, 2020). exporting companies have at least one designer to develop actions that contribute to the image of brazil as a country that, in addition to raw gems, also has a jewelry industry (mme, 2018). according to ribeiro (2011), a decisive factor for the evolution of precious stone exports in brazil is the choice and/or permanence of more dynamic consumer markets, with significant growth in incomes. the jewelry industry has endeavored to be transparent concerning the origin and impact on the development of the gem, considering the consumer market's demand for a responsible and ethical supply. the international community has adopted some measures to meet this requirement. however, no enforcement mechanisms systematically promote the necessary ethical practices concerning the gems trade, supply chains, and regulations. modeling the supply chain in adopting ethical standards, responsible purchasing, and sustainability, tracing the gemstones from the mine to the market, is a significant contribution to the improvement in the gems' market value encompassing the service and manufacturing sector, thus contributing to sustainable livelihoods and local development (makki & ali, 2019). 2.5 sustainable development sustainable development is a term widely used in many areas related to human life. the first references to the term appeared in 1972 at the un conference on environment and development. the term that was used was ecodevelopment. in 1983, the un appointed the prime minister of norway, gro harlem brundtland, to chair the environmental area's commission and deepen global proposals in the commission. then, in 1987, the brundtland report, known as our common future, appeared. the document adopted the sustainable expression development with the following definition: "how current generations meet their needs without, however, compromising the ability of future generations to meet their own needs" (brasil, 2012). the broader definition of the term sustainable development is inserted in the 27 principles of sustainable development. a document was signed at the earth summit, held in rio de janeiro, in 1992. the document recommends that new investment actions in the future are indispensable to achieve global sustainable development in the 21 st century and highlights the need for cooperation in creating a sustainable economy in the mineral resource acquisition industry and the technological implementation for its use (dubinski, 2013). ijahp article: brandao, de alencar naas, de oliveira costa neto/sustainable development of gem production: an analysis using the analytic hierarchy process international journal of the analytic hierarchy process 490 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.880 the concept of sustainable development has been discussed (batterham, 2017; dubinski, 2013) since the activity is linked to nonrenewable natural resources. thus, one of the main challenges for the sector's sustainable development consists of applying sustainability in its activities to contribute positively to the environmental, social, and economic aspects (alves et al., 2020). mining is one of the oldest human activities in which humanity reaches natural wealth. a key element of sustainable development in acquiring these resources is rational and economical extraction that is always concerned with the inevitable depletion of natural resources (dubinski, 2013). extractive industries must reduce their environmental impacts through efficient ecological management programs to contribute to sustainable development (shen et al., 2015). in 2014, the brazilian mining institute (ibram), together with the united nations development program (undp), the brazilian government, and supporters promoted the first global dialogue on the extractive activity of mineral resources. afterward, the country joined the proposed un 2030 agenda for sustainable development and sustainable development objectives (sdgs). as a result of further negotiations, a mapping of mining-related impacts on the sdgs started, and the "atlas" (mapping sustainable development objectives in mining) was developed in partnership with the world economic forum, columbia university, and the solutions network for sustainable development, supported by the german international cooperation agency (cni, 2017). the atlas maps the link between mining and the sdgs, based on good practices and sustainable development that might promote actions that eradicate poverty, reduce social inequality, protect the environment, environment and climate, and guarantee a future global society based on sustainability principles (ibram, 2016). the department of sustainable development in mining (ddsm) works in coordination with other sectors of the ministry of mines and energy (mme) to formulate and articulate proposals for public policies, plans, and programs for the sustainable development of mining. the department bases its actions on the external demands of society, government policies, participation in environmental policy forums, the preparation of studies and projects, and training and development actions (mme, 2018). as the scope of the efforts, mineral extension projects for small-scale mining (mpe) were developed (table 3). it is noted that these projects have a primary objective to build a relationship between sustainable development and the activity of mineral extraction, which is a challenge. ijahp article: brandao, de alencar naas, de oliveira costa neto/sustainable development of gem production: an analysis using the analytic hierarchy process international journal of the analytic hierarchy process 491 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.880 table 3 projects on mineral extension, year of approval, and objectives (mme, 2018) project year of approval objective national mineral extension program 2008 to minimize the informality of mining and the negative impacts on the community and the environment, enabling producers for sustainable development small mineral producer support portal (pormin) 2008 to offer remote training for mpe and information relevant to the mineral segment mineral telecenters 2007 to warrant physical means to the pormin and other websites national program for the formalization of mineral production (pronafor) 2005 support the formalization of small-scale mineral production and elaborate on diagnoses on the obstacles to this formalization national seminars on mineral cooperatives 2005 to discuss aspects related to the organization of cooperatives and other forms of associations and public policies aimed at the sustainable development of small and medium-sized mining agenda 21 mineral program 2004 plan, in agreement with public authorities and society, economic alternatives that can guarantee the maintenance of socioeconomic levels of the municipalities in the period after the deactivation of mineral activities source: mme (2018) solving environmental challenges is vital for the activity of mineral extraction and constitutes a pillar for sustainable development. it is necessary to develop technologies to extract mineral resources that eliminate or minimize the negative effects of the activity and restore the soil for later use. another critical factor in the sustainable development of mineral resources extraction is social responsibility, first of all in terms of the health and safety of workers, considering that the activity is characterized by a high level of risk of accidents at work (dubinski, 2013). arango-aramburo et al. (2017) discuss the relationship between technology and socio-environmental performance in the mining process and argue that new technologies can increase efficiency. it also improves the process, reducing production costs and decreasing pollution and depletion of natural resources. another possible contribution is developing preventive solutions to avoid environmental impacts and reduce the cost of corrective actions (milanez & oliveira, 2013). the regional economy is more vulnerable to fluctuations in commodity prices, and adverse effects of expansion and contraction cycles may occur in regions with high economic dependence on mineral extraction. this aspect is considered critical in areas with geographically remote resources, which have less opportunity for economic diversity (mancine & sala, 2018). ijahp article: brandao, de alencar naas, de oliveira costa neto/sustainable development of gem production: an analysis using the analytic hierarchy process international journal of the analytic hierarchy process 492 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.880 3. methodology the ahp is a tool for dealing with decisions under uncertainty, in which subjective judgment is quantified logically and used as a basis for decision making. in the ahp, performance ratings and attribute weights result from a series of peer comparison judgments at the same hierarchy level, assigned a discrete scale from 1 to 9 (taha, 2008, bui et al., 2017). this methodology uses a pairwise comparison system based on rankings (table 4, saaty, 1977). table 4 discrete scale of pairwise comparison weights scale degree of importance 1 equal importance 3 moderate importance 5 high importance 7 very high importance 9 extreme importance 2,4,6,8 intermediate values source: saaty & shang (2011) determination of the relative weights to classify the decision alternatives is the crucial point for applying the ahp. considering that one deals with n criteria in a given hierarchy, the procedure establishes a matrix a= [aij] with i varying from 1 to n and j varying from 1 to n, which quantifies the decision-maker concerning the relative importance of different criteria. the pairwise comparison is made so that the line criterion i (i = 1,2,3,4...., n) is classified concerning the other criteria (taha, 2008). defining element (i,j) from matrix a as aij, the ahp uses the scale in table 4 to measure the degree of importance pairwise. consistency in judgment requires that aij = k imply in aji= 1/k. in addition, all elements aij of the main diagonal of matrix a must be equal to 1, representing the classification of a given criterion to itself. the degree of inconsistency (ʎ) implies whether the judgment was coherent on the part of the decision-maker concerning the pairwise comparisons. if ʎ is smaller or equal to 0.1, then the consistency is acceptable. if ʎ is higher than 0.1, then the inconsistency is high; therefore, another judgment is needed (taha, 2008). in the present study, the multicriterial analysis ahp is used to assess the degree of importance of the factors contributing to the productive process of gems in the sustainable development of the chain. some steps were developed before applying the expert choice® software v. 11 to determine the degree of importance for data processing (figure 1). ijahp article: brandao, de alencar naas, de oliveira costa neto/sustainable development of gem production: an analysis using the analytic hierarchy process international journal of the analytic hierarchy process 493 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.880 figure 1 research process of the analysis source: adapted from periaiah et al. (2021) step 1 – explore the current situation of the gem production process in the national and international scenario, emphasizing artisanal and small production, and acquire better knowledge of the main issues related to the sustainable development of the gem production chain. current references were used to select the factors that influence the sustainable development of the gem production chain, related by the stages of the smallscale production process, with possible effects on the chain. step 2 for the construction of the hierarchy, the chosen factors were considered the subcriteria for applying the ahp, and the stages of the production process, mining, cutting, and marketing criteria were considered (figure 2). figure 2 elements of the multicriteria analysis (ahp) used in the present study ijahp article: brandao, de alencar naas, de oliveira costa neto/sustainable development of gem production: an analysis using the analytic hierarchy process international journal of the analytic hierarchy process 494 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.880 step 3 after selecting the factors, we interpreted the interrelations of the factors and the social, economic, and environmental dimensions, the description of the factors, and the production stages, according to table 5, to build the hierarchy of the factors the ahp process. table 6 indicates the way the interrelations were understood. table 5 description of the factors that influence the sustainable development of the gem production chain, related by the process stages in coherence with the corresponding scope and references ste ps factors description of the factors scope reference m in in g m 1 -w a st e re u se adopt measures to recycle waste, promote the transformation of waste, control the deposition of waste. take steps to mitigate environmental impacts and improve mining processing methods. e n v ir o n m e n t a l /e c o n o m ic -mulinari (2011); -costa & jornada (2015); -matinde (2018); -alves et al. (2020). m 2 h e a lt h a n d s a fe ty o f w o rk e rs a n d th e c o m m u n it y a ro u n d t h e m in e promote awareness for workers about job safety and the environment and risks of mining activity. supervise technical issues of workplace safety and worker health. workplace stability and remuneration, ensuring their families adequate living conditions. s o c ia l/ e n v ir o n m e n ta l -carvalho (201)7; -alves et al. (2020); -bui et al. (2017); -mancini & sala (2018); -dubinski (2013); -gomes et al. (2014). m 3 d e g re e o f fo rm a li ty o f th e p ro c e ss control the official number of gem extractions and exports so that the informal sector does not prosper for the production and sale. monitoring of financial compensation for mineral exploration (cfem) is reflected in benefits for the municipality. issuance of formal property titles that bind the miner to the land controls the miner's evasion. greater visibility of the sector for the elaboration and implementation of public policies. compensation for occupational issues. s o c ia l/ e c o n o m ic /e n v ir o n m e n ta l -barreto & bittar (2010); -mme (2018); -milanez &puppim de oliveira (2013); -bui et al. (2017). m 4 m in e l if e measures must be taken to preserve the mine's useful life and maintain savings after depletion. investments applied for mine rehabilitation and closure. the assessment and management of the uncertainties and risks associated with the development of mineral resources. e n v ir o n m e n ta l/ e c o n o m ic -alves et al. (2020); -bui et al. (2017); -dubinski (2013). m 5 -m in e p ro d u c ti v it y if there is control of the annual quantity of the stone explored in kilograms. e c o n o m ic / e n v ir o n m e n ta l -dubinski, 2013. -cartier (2019) g e m c u tt in g l 1 -d e si g n respect and appreciate local culture, strengthen local resources, and promote the local economy. s o c ia l/ e c o n o m ic -canaan ( 2013); -costa & jornada (2015); -mme (2018); -ribeiro (2011); -milanez & puppim de oliveira( 2013. ijahp article: brandao, de alencar naas, de oliveira costa neto/sustainable development of gem production: an analysis using the analytic hierarchy process international journal of the analytic hierarchy process 495 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.880 l 2 in o v a ti o n innovation in the gem transformation process, valuing the gem's natural characteristics (color, purity, weight) and those artificially developed by cutting and design. e c o n o m ic -endl et al. (2019); -milanez & puppim de oliveira (2013). l 3 t e c h n o lo g ic a l tr a in in g use of technology and training of professionals in polishing skills. gemcutting techniques for better use and shine of the gem. s o c ia l/ e c o n ô m ic a /e n v ir o n m e n ta l -costa & jornada( 2015; -mol et al.( 2016; -ribeiro( 2011. l 4 -p ro d u c ti v it y refers to the control of the faceted amount of the stone (carats). e c o n o m ic / e n v ir o n m e n ta l -cartier (2019) c o m m e rc ia li z a ti o n c 1 -g e m o ri g in refers to if the gem originates from a conflict region or if the gemstone has a certification of origin. s o c ia l/ e n v ir o n m e n ta l/ e c o n o m ic -makki & ali (2019). c 2 a d d e d v a lu e refers to different types of gems and their aesthetic qualities that influence the final price. price is defined by demand/supply. price is determined by the metals added to the gem. e c o n o m ic -costa & jornada (2015); -mancini & sala (2018). c 3 p ro d u c t m a rk e ti n g m o d e c o m m e rc ia li z a ti o n o f th e p ro d u c t packaging, collection catalog, distribution channels. metals added to the gem. participation in fairs and events. s o c ia l/ e c o n o m ic -costa & jornada (2015). c 4 -p ro d u c ti v it y if there is control of the annual quantity of stone sold in raw form, cut and inlaid in gold and silver. e c o n o m ic / s o c ia l kincaid & smith (2021) ijahp article: brandao, de alencar naas, de oliveira costa neto/sustainable development of gem production: an analysis using the analytic hierarchy process international journal of the analytic hierarchy process 496 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.880 table 6 interpretation of the interdimensional relationships of the considered factors scope steps factors social economic environmental m in in g m 1 -w a st e re u se circular economy. impact on the bulky landscape tailings can damage the ecosystem. m 2 h e a lt h a n d s a fe ty o f w o rk e rs a n d t h e c o m m u n it y a ro u n d t h e m in e quality of life in the mine surroundings. risks and work accidents. m 3 d e g re e o f fo rm a li ty o f th e p ro c e ss inadequate labor practices tend to perpetuate social conflicts. underestimation of production. inadequate environmental practices. m 4 m in e li fe economic stability. rational and economical extraction of natural resources. m 5 -m in e p ro d u c ti v it y profitability. land removal. g e m c u tt in g l 1 -d e si g n social welfare. use of the gem. l 2 in n o v a ti o n attractiveness, competitiveness, and market value. l 3 t e c h n o lo g ic a l tr a in in g job opportunities for the local community. processing time and efficiency. use of natural resources. l 4 -p ro d u c ti v it y added value. risks of accidents at work. ijahp article: brandao, de alencar naas, de oliveira costa neto/sustainable development of gem production: an analysis using the analytic hierarchy process international journal of the analytic hierarchy process 497 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.880 step 4 based on the literature, the three consultants evaluated the degree of relevance and compared pairs between the criteria and sub-criteria, following the weight scale. the expert choice® software v.11 was used to achieve the final goal of assessing the degree of importance of the factors contributing to the gems' production process in the sustainable development of the chain. the sub-criteria and criteria weights were given from the selected consultants. one consultant was an academic specialist in quality applied in supply chains. the second was a researcher from the fields of environment and socio-economic development affairs. the third was an academic acting in the artisanal mining of opal in the region of pedro ii municipality, piaui, brazil. we explained the concepts we applied to standardize the knowledge on the topics (tables 5 and 6). we emphasized the idea of evaluating the importance of the factors of the gem production process in sustainable chain development in coherence with the sustainability dimensions. 4. results thirteen factors were selected and considered relevant for the sustainable development of the gemstone production process and divided into three groups according to the process stages. after selection, the factors were considered the sub-criteria and the steps the criteria. the weights of the pairwise comparisons, respecting the saaty scale (table 4), were established by the researchers, based on current literature, and following the analysis method proposed by the ahp. the weights attributed in the pairwise comparisons and the degree of inconsistency (ʎ) can be perceived in the matrices (tables 7 23). the consultants were asked to assess the degree of relative importance between c o m m e rc ia li z a ti o n c 1 -g e m o ri g in social status. gem's value. source certification. c 2 a d d e d v a lu e metals added for jewelry production. c 3 p ro d u c t m a rk e ti n g m o d e c o m m e rc ia li z a ti o n o f th e p ro d u c t high social level target audience. attractiveness. sustainable. c 4 -p ro d u c ti v it y profitability. ijahp article: brandao, de alencar naas, de oliveira costa neto/sustainable development of gem production: an analysis using the analytic hierarchy process international journal of the analytic hierarchy process 498 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.880 the criteria, sub-criteria, and the objective, and based on the respective questions (a, b and c), to form the pairwise comparison matrices: a) how important are the phases of gem production in the sustainable development of the gem production chain? (related to table 7). table 7 reciprocal comparison matrix for criteria (ʎ = 0.08) criteria mining gem cutting comercialization mining 1 2.17 1.74 gem cutting 1 1.87 comercialization 1 b) what is the degree of importance of the sub-criteria in the gem production chain stages also focusing on mining sustainability? (related to tables 8-10). table 8 reciprocal comparison matrix for sub-criteria in the mining stage (ʎ = 0.01) m1 m2 m3 m4 m5 m1 1 1.38 1.35 1.75 1.13 m2 1 1.09 1.09 1.08 m3 1 1.31 1.02 m4 1 1.03 m5 1 table 9 reciprocal comparison matrix for sub-criteria in the gem cutting stage (ʎ = 0.00) l1 l2 l3 l4 l1 1 1.09 1.15 1.17 l2 1 1.22 1 l3 1 1.12 l4 1 table 10 reciprocal comparison matrix for sub-criteria in the commercialization stage (ʎ = 0.00) c1 c2 c3 c4 c1 1 1.08 1.11 1.20 c2 1 1.01 1.01 c3 1 1.22 c4 1 ijahp article: brandao, de alencar naas, de oliveira costa neto/sustainable development of gem production: an analysis using the analytic hierarchy process international journal of the analytic hierarchy process 499 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.880 c) what is the degree of importance of the sub-criteria in the sustainable development of the production chain of gemstones? (related to tables 11-23). table 11 reciprocal comparison matrix for sub-criteria for the alternatives in sub-criteria m1 (ʎ = 0.02) high medium low high 1 1.66 1.91 medium 1 1.75 low 1 table 12 reciprocal comparison matrix for sub-criteria for the alternatives in sub-criteria m2 (ʎ = 0.00) high medium low high 1 1.23 1.68 medium 1 1.31 low 1 table 13 reciprocal comparison matrix for sub-criteria for the alternatives in sub-criteria m3 (ʎ = 0.04) high medium low high 1 1.60 1.75 medium 1 1.62 low 1 table 14 reciprocal comparison matrix for sub-criteria for the alternatives in sub-criteria m4 (ʎ = 0.04) high medium low high 1 1.89 2.01 medium 1 1.90 low 1 table 15 reciprocal comparison matrix for sub-criteria for the alternatives in sub-criteria m5 (ʎ = 0.04) high medium low high 1 1.96 2.03 medium 1 1.95 low 1 ijahp article: brandao, de alencar naas, de oliveira costa neto/sustainable development of gem production: an analysis using the analytic hierarchy process international journal of the analytic hierarchy process 500 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.880 table 16 reciprocal comparison matrix for sub-criteria for the alternatives in sub-criteria l1 (ʎ = 0.00) high medium low high 1 1.24 1.62 medium 1 1.34 low 1 table 17 reciprocal comparison matrix for sub-criteria for the alternatives in sub-criteria l2 (ʎ = 0.03) high medium low high 1 1.92 2.22 medium 1 1.99 low 1 table 18 reciprocal comparison matrix for sub-criteria for the alternatives in sub-criteria l3 (ʎ = 0.03) high medium low high 1 1.73 2.97 medium 1 1.88 low 1 table 19 reciprocal comparison matrix for sub-criteria for the alternatives in sub-criteria l4 (ʎ = 0.00) high medium low high 1 1.23 1.39 medium 1 1.33 low 1 table 20 reciprocal comparison matrix for sub-criteria for the alternatives in sub-criteria c1 (ʎ = 0.02) high medium low high 1 1.8 2.15 medium 1 1.93 low 1 ijahp article: brandao, de alencar naas, de oliveira costa neto/sustainable development of gem production: an analysis using the analytic hierarchy process international journal of the analytic hierarchy process 501 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.880 table 21 reciprocal comparison matrix for sub-criteria for the alternatives in sub-criteria c2 (ʎ = 0.00) high medium low high 1 1.19 1.51 medium 1 1.23 low 1 table 22 reciprocal comparison matrix for sub-criteria for the alternatives in sub-criteria c3 (ʎ = 0.00) high medium low high 1 1.49 1.54 medium 1 1.52 low 1 table 23 reciprocal comparison matrix for sub-criteria for the alternatives in sub-criteria c4 (ʎ = 0.00) high medium low high 1 1.24 1.41 medium 1 1.35 low 1 after pairwise comparisons between the items shown in figure 1, based on the data collected during the survey, a general scenario was obtained and shown in table 24. ijahp article: brandao, de alencar naas, de oliveira costa neto/sustainable development of gem production: an analysis using the analytic hierarchy process international journal of the analytic hierarchy process 502 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.880 table 24 general results relating to the criteria, sub-criteria, and alternatives alternatives criteria sub-criteria high medium low global (%) rank gem mining (0.489) m1 (0.174) 0.038 0.026 0.017 44.7 1 m2 (0.214) 0.047 0.037 0.028 m3 (0.174) 0.038 0.027 0.019 m4 (0.237) 0.052 0.033 0.021 m5 (0.202) 0.044 0.028 0.018 global (%) 46.1 32.1 21.8 gem cutting (0.299) l1 (0.227) 0.030 0.025 0.019 32.5 2 l2 (0.242) 0.032 0.020 0.012 l3 (0.274) 0.037 0.025 0.016 l4 (0.252) 0.034 0.029 0.023 global (%) 44.2 32.8 23.0 gem commercialization (0.212) c1 (0.249) 0.024 0.015 0.009 22.8 3 c2 (0.245) 0.023 0.019 0.016 c3 (0.231) 0.022 0.017 0.012 c4 (0.275) 0.026 0.022 0.017 global (%) 42.4 33.0 24.6 the results suggest that sustainable development is highly related to gem mining (44.7%, rank 1), moderately related to gems cutting (32.5%, rank 2), and there is little relationship to gems commercialization (22.8%, rank 3), considering the criteria, sub-criteria, and alternatives. 4.1 sustainable development of the gemstone production process the criteria objective is to identify the degree of importance of the stages of the gemstone production process in the development of the chain. table 24 shows this degree of significance, indicating what is relevant for the sustainable development of the gem production chain, according to the application of the ahp. it is noteworthy that mining is the most significant stage in the gemstone production chain, with a relevance of 48.9%, followed by gem cutting (29.9%) and, finally, commercialization (21.2%). this result is considered significant due to the low degree of inconsistency (ʎ = 0.03). 4.2 hierarchy to analyze sustainable development in gem mining considering the subcriteria (m, m2, m3, m4, m5). the objective of the subcriteria is to identify the degree of importance of the factors that contribute to sustainable development in the stages of the gem production process. in this category, five relevant factors were selected in the process. the degrees of significance (table 24) indicate its relevance for sustainable development in gem mining, according to the application of the ahp. it is noteworthy that in the mining process, the most ijahp article: brandao, de alencar naas, de oliveira costa neto/sustainable development of gem production: an analysis using the analytic hierarchy process international journal of the analytic hierarchy process 503 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.880 important factors are the preservation of the mine's useful life (23.6%) and worker health and safety (21.4%). this result is significant considering the low degree of inconsistency (ʎ = 0.0107). 4.3 hierarchy to analyze the sustainable development in gem cutting considering the subcriteria (l1. l2, l3, l4). in this category, four relevant factors were selected in the process. data in table 24 shows this degree of significance, showing what is appropriate for sustainable development in gem cutting according to the use of the ahp. in gem cutting, the factor that is the most significantly important is the technical qualification and skill of the stonecutter, with a percentage of 27.9%. 4.4 hierarchy to analyze the sustainable development of gems commercialization considering the sub-criteria (c1. c2, c3, c4). in this category, four relevant factors were selected in the process. the degree of significance (table 24) indicates what is pertinent for sustainable development in the commercialization of gems when using the ahp. with a percentage of 27.5%, productivity in marketing stands out as the most critical factor, which suggests that it is a relevant aspect in the economic viability of the activity. this result is significant considering the low degree of inconsistency (ʎ = 0.0024) presented by the software, which implies that the level of inconsistency is acceptable. 4.5 levels of the sub-criteria importance for the sustainable development of the production process of gems figure 2 shows the degree of importance of all factors considered relevant in the gemstone production process and the result obtained for the degree of importance of these factors, based on the ahp. figure 3 general view of the subcriteria adopted figure 3 shows the degree of importance of sustainable development based on the subcriteria of each stage of the gemstone production process. ijahp article: brandao, de alencar naas, de oliveira costa neto/sustainable development of gem production: an analysis using the analytic hierarchy process international journal of the analytic hierarchy process 504 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.880 5. discussion the mining process found that the essential factors are related to the conservation of the mine's useful life (23.6%) and worker health and safety (21.4%). such a result corroborates the studies by bui et al. (2017) and dubinski (2013). this result suggests that the investment for rehabilitation and closure of a mine and the reduction in the number of fatalities at work are vital for developing any project and the public awareness that mineral resources are nonrenewable assets. among the related factors are the percentage of productivity (20.2%), the reuse of waste, and the degree of formality of the process, with equal importance (17.4%). this result suggests that it is essential to preserve the mine's useful life, minimize waste disposal, and promote the transformation of deposited waste, in addition to issuing formal property titles, linking the miner to the land, and avoiding occupational conflicts. the results agree with shen et al. (2015), who found that mining companies should pay adequate attention to the soft factors for adopting green (sustainable) supply chain management. productivity in marketing stands out as the most critical factor (27.5%), which indicates that it is a relevant factor in the activity's economic viability. according to mancinni and sala (2018), assessing the economic benefits involves meeting the demand for resources, which becomes relevant to subsidize the raw materials policy. the origin of the gem has a percentage of 24.9%. according to makki and ali (2019), the gem's origin is an economic and image-building product factor in post-conflict regions. in addition, the added value presented a percentage of 24.5%, and the form of commercialization represented 23%. the stonecutter's technical qualification and skill in gem cutting appears to be the most significant factor (27.9%). in the study by costa e jornada (2015), this result is well highlighted as an essential factor for adding value to the gem, which goes far beyond the definition of shapes and quality. it is also a source of competitive advantage and performance improvement in the processing industry. the related factors are productivity (25.2%), innovation (24.2%), and design (22.7%). the technological qualification and skills of gem-cutting are critical factors in the jewelry production process. productivity follows in the commercialization stage and the preservation of the mine’s valuable life. the technological capacity and skill of the lapidary are essential for adding value to production and are a crucial issue for the quality of manufacture in each of the stages of the production process. this logic is valid for gem-cutting and the commercialization process, which is seen as productivity. in the mining process, the preservation of the mine's use life stands out, a result that confirms the study by costa e jornada (2015). all factors considered in the present study had a considerable level of relevance since the percentages of importance ranged between a maximum of 27.9% and a minimum of 17.4%. although, in general, our results indicate that the social benefits' importance was related to the worker's health. these findings partially agree with carvalho (2017), who points out that a pivotal aspect of mining success is meeting the community's social goals, related to their approval to mining in their region. the corporate responsibility of mining enterprises has also grown to share the prosperity from mining with regional areas, which it possible through investing in infrastructure. therefore, we believe the future for gem mining depends on adhering to good mining practices to protect the environment ijahp article: brandao, de alencar naas, de oliveira costa neto/sustainable development of gem production: an analysis using the analytic hierarchy process international journal of the analytic hierarchy process 505 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.880 everywhere, taking social responsibility in the region's development, and promoting the community's quality of life. in relation to the social challenges, in the last decade the mining industries have been adopting the diversity and inclusion of the workforce as a strategy to increase efficiency, innovation and align the activity with the sustainable development goals (sdgs). the most significant barrier to such a measure is an executive leadership and the lack of information on diversity and inclusion in international contexts in the commodity industry. while there is some progress, structural barriers still impact the adoption of diversity and inclusion in gem mining. encouraging the establishment of social networks, recruiting employees through business groups, and instituting hiring targets are measures used by industries to increase diversity and inclusion in the workforce. this adoption is seen positively by consumers and employees, increasing revenue, shared value, and participation in the national and international market (kincaid & smith, 2021). artisanal mining almost always takes place in remote areas. far from government regulations, it involves the circulation of money and valuable minerals, which often attract criminal behavior and negative social impacts, adversely affecting the activity. however, positive aspects can be highlighted as they can contribute to regional development and change public policies to reduce conflicts and maximize positive factors (laing & moonsammy, 2021). a limitation of the present study is that it comprises the knowledge of the opal artisanal mining process in a determined region of brazil. we believe that it represents most artisanal gems mining in the country well. we also suggest that future studies expand the social, environmental, and economic dimensions following the 2030 un agenda. 6. conclusions this research aimed to estimate the main factors contributing to the sustainable development of the production chain of gems, evaluating the weights in the steps of mining, cutting, and marketing of gems, using a multicriteria decision model by applying the analytic hierarchy process. these factors were categorized into 13 subcriteria in the three criteria presented. from the results, it was possible to infer that mining activity was the most important criterion in the sustainable development of the chain, directly affecting the community’s development. however, we need to consider the negative impacts that influence the environmental and social environment. the challenge for the sector is balancing the positive socio-economic issues, and the harmful environmental issues to maximize the overall benefit of mining activity. the lapidary technological skills were the most critical subcriteria in sustainable gemstone chain development. they can contribute to employment and income, improve lapidary technical skills, and add value to the gem. the research showed in the alternatives that, in general, sustainable development in the gems production chain is considered high given the related factors. in addition, we believe that assessing the positive aspects and the sector's contribution to sustainable development, considering local and global scales, is challenging. ijahp article: brandao, de alencar naas, de oliveira costa neto/sustainable development of gem production: an analysis using the analytic hierarchy process international journal of the analytic hierarchy process 506 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.880 references alves, w., ferreira, p., & araújo, m. (2020). challenges and pathways for brazilian mining sustainability. resources policy, 74, 101648. doi: https://doi.org/10.1016/j.resourpol.2020.101648 arango-aramburo, s., jaramillo, p., olaya, y., smith, r., restrepo, oj, saldarriagaisaza, a., arias-gaviria, j., parra, jf., larsen, er., gomes-rios, lm., & castelhanos niño, ly. (2017). simulating mining policies in developing countries: the case of colombia. socio-economic planning sciences, 60, 99–113. doi: https://doi.org/10.1016/j.seps.2017.04.002 barreto, s de b., & bittar, smb. (2010). the gem deposits of brazil: location, production and economic impact. boletin de la sociedad geológica mexicana, 62(1), 123-140. issn 1405-3322. batterham, rj. (2017). the mine of the future – even more sustainable. minerals engineering, 107, 2-7. doi: https://doi.org/10.1016/j.mineng.2016.11.001 brazilfederal senate. (2012). federal senate public hearing review. retrieved 4 march 2021, from https://www.senado.gov.br/noticias/jornal/emdiscussao/rio20/temasem-discussao-na-rio20/ecodesenvolvimento-conceito-desenvolvimento-sustentavelrelatorio-brundtland-onu-crescimento-economico-pobreza-consumo-energia-recursosambientais-poluicao.aspx brazil. (2018). mining code, decree 9.406 of june 2018.brasília, d.f. presidency of the republic. retrieved 22 january 2021, from http://www.planalto.gov.br/ . bui, n.t., kawamura, a., kim, k.w., prathumratana, l., kim, t.h., yoon, s.h., jang, m., amaguchi, h., du bui, d., & truong, n.t. 2017. proposal of an indicator-based sustainability assessment framework for the mining sector of apec economies. resources policy, 52, 405–17. doi: https://doi.org/10.1016/j.resourpol.2017.05.005. camanho, r., & moraes, ea. (2007). multicriteria decisions. in costa neto, p.l.o (coord.) quality and competence in decisions. são paulo: editora blucher. canaan, r.p. (2013). design management applied to the value chain of local production arrangements. uemg mg, 119. carvalho, c.a. de. (2017). the role of the apl of the pedro ii opal, piaui, in the structuring of mineral tourism in the municipality. master's in social change and political participation. são paulo: university of são paulo. carvalho, f.p. (2017). mining industry and sustainable development: time for change. food and energy security, 6(2), 61-77. cartier, l.e. (2019). gemstones and sustainable development: perspectives and trends in mining, processing and trade of precious stones. the extractive industries and society, 6, 13–16. doi: https://doi.org/10.1016/j.exis.2019.09.005. https://doi.org/10.1016/j.resourpol.2020.101648 https://doi.org/10.1016/j.seps.2017.04.002 https://doi.org/10.1016/j.mineng.2016.11.001 https://doi.org/10.1016/j.mineng.2016.11.001 https://www.senado.gov.br/noticias/jornal/emdiscussao/rio20/temas-em-discussao-na-rio20/ecodesenvolvimento-conceito-desenvolvimento-sustentavel-relatorio-brundtland-onu-crescimento-economico-pobreza-consumo-energia-recursos-ambientais-poluicao.aspx https://www.senado.gov.br/noticias/jornal/emdiscussao/rio20/temas-em-discussao-na-rio20/ecodesenvolvimento-conceito-desenvolvimento-sustentavel-relatorio-brundtland-onu-crescimento-economico-pobreza-consumo-energia-recursos-ambientais-poluicao.aspx https://www.senado.gov.br/noticias/jornal/emdiscussao/rio20/temas-em-discussao-na-rio20/ecodesenvolvimento-conceito-desenvolvimento-sustentavel-relatorio-brundtland-onu-crescimento-economico-pobreza-consumo-energia-recursos-ambientais-poluicao.aspx https://www.senado.gov.br/noticias/jornal/emdiscussao/rio20/temas-em-discussao-na-rio20/ecodesenvolvimento-conceito-desenvolvimento-sustentavel-relatorio-brundtland-onu-crescimento-economico-pobreza-consumo-energia-recursos-ambientais-poluicao.aspx http://www.planalto.gov.br/ https://doi.org/ https://doi.org/ ijahp article: brandao, de alencar naas, de oliveira costa neto/sustainable development of gem production: an analysis using the analytic hierarchy process international journal of the analytic hierarchy process 507 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.880 cni-national confederation of industries (2017). mining and green economy. retrieved 3 march 2021, from <https://portaldamineracao.com.br> . costa, r.m. da., & jornada, m.i.h. da. (2015, july). productive arrangement of stones, gems and jewels from alto da serra do botucaraí. report ii. porto alegre: fee. retrieved 23 april 2019, from <http://www.fee.rs.gov.br/publicacoes/relatorios/>. dubinski, j. (2013). sustainable development of mining mineral. resources. journal of sustainable mining, 12(1), 1–6. doi: https://doi.org/10.7424/jsm130102. ibgm brazilian institute of gems and precious metals. (2018). the sector in large numbers. retrieved 8 february 2021, from <https://ibgm.com.br/ibgminforma/publicacoes/> ibrambrazilian mining institute. (2016). atlas: mapping the goals of sustainable development in mining. retrieved 3 march 2021, from <https://portaldamineracao.com.br/atlas-mapeia-desenvolvimento-sustentavel-namineracao> kincaid, c., & smith, n.m. (2021). diversity and inclusion in mining: an analysis of indicators used in sustainability reporting. the extractive industries and society, e100981. oi: https://doi.org/10.1016/j.exis.2021.100981. laing, t., & moonsammy, s. (2021). evaluating the impact of small-scale mining on the achievement of the sustainable development goals in guyana. environmental science & policy, 116, 147–59. doi: https://doi.org/10.1016/j.envsci.2020.11.010. landim, p.c. (2010). design, company, society. academic culture, scielo books. são paulo, ed. unesp, retrieved 20 de january 2021, from < http//books.scielo.org>. makki, m., & ali, s.h. (2019). gemstone supply chains and development in pakistan: analyzing the post-taliban emerald economy in the swat valley. geoforum, 100, 166– 175. doi: https://doi.org/10.1016/j.geoforum.2019.01.005 mancini, l., & sala, s. (2018). social impact assessment in the mining sector: review and comparison of indicators frameworks. resources policy, 57, 98–111. doi: https://doi.org/10.1016/j.resourpol.2018.02.002 milanez, b., & oliveira j.a.p. de. (2013). inovação para o desenvolvimento sustentável na mineração artesanal: avanços em um cluster de mineração de opala no brasil. resources policy, 38, 427-434. doi: https://doi.org/10.1016/j.resourpol.2013.07.003 mmeministry of mines and energy. (2018). report 5 – inventory of small-scale mining of gems. retrieved 22 january 202, from http://antigo.mme.gov.br/documents/36144/471889/produto+5.pdf/fdcff1b2-c0b2-59096dff-c3be19614a85 file:///c:/users/lilan/downloads/%3chttps:/portaldamineracao.com.br%3e file:///c:/users/lilan/downloads/%3chttp:/www.fee.rs.gov.br/publicacoes/relatorios/%3e https://doi.org/10.7424/jsm130102 https://ibgm.com.br/ibgm-informa/publicacoes/ https://ibgm.com.br/ibgm-informa/publicacoes/ https://portaldamineracao.com.br/atlas-mapeia-desenvolvimento-sustentavel-na-mineracao https://portaldamineracao.com.br/atlas-mapeia-desenvolvimento-sustentavel-na-mineracao https://doi.org/ file:///c:/users/lilan/downloads/http/books.scielo.org https://doi.org/10.1016/j.geoforum.2019.01.005 https://doi.org/10.1016/j.resourpol.2018.02.002 https://doi.org/10.1016/j.resourpol.2013.07.003 http://antigo.mme.gov.br/documents/36144/471889/produto+5.pdf/fdcff1b2-c0b2-5909-6dff-c3be19614a85 http://antigo.mme.gov.br/documents/36144/471889/produto+5.pdf/fdcff1b2-c0b2-5909-6dff-c3be19614a85 ijahp article: brandao, de alencar naas, de oliveira costa neto/sustainable development of gem production: an analysis using the analytic hierarchy process international journal of the analytic hierarchy process 508 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.880 mol, a., martins-filho, ls, & rocha, r. (2016). application of artificial neural networks in the estimation of parameters in cut gemstone designs. brazilian congress of neural networks. doi: https://doi.org/10.21528/cbrn2005-001 mulinari, m. (2011). analysis of the production process of the extraction of precious stones aiming at profitable and environmentally effective alternatives. master's dissertation, ufsm, rs, 80. oliveira, l.h.o. de. (2019). body adornments in the atlantic world: organic gems in eighteenth-century minas gerais.in rezende, ejc., safar, gh., almeida, m. das g. notebook in time: history in art and designer, 4, 40. belo horizonte, eduemg. retrieved 20 january 2021, from <http://ed.uemg.br/publicacoes/>. periaiah, n., islam, r., & abdullah, m.f. (2021). environmental impact assessment for malaysian bauxite mining industry. international journal of the analytic hierarchy process, 13(1), 1-26. doi: https://doi.org/10.13033/ijahp.v13i1.851 reck, a.b., & schultz, g. (2016). application of the multicriteria decision support methodology in the interorganizational relationship in the beef poultry chain. journal of rural economics and sociology, 54. doi: https://doi.org/10.1590/1234-5678180694790540407 reis, j.g.m., vendrameto, o., costa neto, p.l.de o., & machado, s.t. (2017). evaluation of the economic, social and environmental sustainability of the sugar-energy activity in mato grosso do sul: an analysis with multicriteria. magazine on agribusiness and environment, 10, 673-695. doi: https://doi.org/10.17765/21769168.2017v10n3p673-695 ribeiro, h.m.d. (2011). characterization of the sector of gems, jewelry and precious metals in brazil: perspectives for innovation and sectorial development. sectorial studies series, 11, 48, brasília: senai.dn. saaty, t.l. (1977). a scaling method for priorities in hierarchical structures. journal of mathematics psychology, 15, 234 – 281. doi: https://doi.org/10.1016/00222496(77)90033-5 saaty, t.l., & shang j. (2011). an innovative order of magnitude approach to multicriteria decision making based on ahp: prioritizing divergent intangible humanitarian acts. european journal of operational research, 214, 703-715. doi: https://doi.org/10.1016/j.ejor.2011.05.019 saldarriaga-isaza, a., villegas-palacio, c., & arango, s. (2013). the public good dilemma of a nonrenewable common resource: a look at the facts of artisanal gold mining. resource policy, 38, 224-232. doi: https://doi.org/10.1016/j.resourpol.2013.02.001 sambasivan, m., & fei, n.y. (2008). evaluation of critical success factors of implementation of iso 14001 using analytic hierarchy process (ahp): a case study from https://doi.org/10.21528/cbrn2005-001 http://ed.uemg.br/publicacoes/ https://doi.org/10.13033/ijahp.v13i1.851 https://doi.org/10.1590/1234-56781806-94790540407 https://doi.org/10.1590/1234-56781806-94790540407 https://doi.org/10.17765/2176-9168.2017v10n3p673-695 https://doi.org/10.17765/2176-9168.2017v10n3p673-695 https://doi.org/10.1016/0022-2496(77)90033-5 https://doi.org/10.1016/0022-2496(77)90033-5 https://doi.org/10.1016/j.ejor.2011.05.019 https://doi.org/10.1016/j.resourpol.2013.02.001 ijahp article: brandao, de alencar naas, de oliveira costa neto/sustainable development of gem production: an analysis using the analytic hierarchy process international journal of the analytic hierarchy process 509 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.880 malaysia. journal of cleaner production, 16, 1424-1433. doi: https://doi.org/10.1016/j.jclepro.2007.08.003 sauer, p.c, & seuring, s. 2017. sustainable supply chain management for minerals. journal of cleaner production, 151, 235–49. doi: 10.1016/j.jclepro.2017.03.049. shen, l., muduli, k., & barve, a. (2015). developing a sustainable development framework in the context of mining industries: ahp approach. resources policy, 46, 15– 26. doi: https://doi.org/10.1016/j.resourpol.2013.10.006 taha, h.a. (2018). pesquisa operacional: uma visão geral. 8 ed. são paulo: editora pearson. united nations commodity trade. – un comtrade database general data. retrieved 27 january 2021, from <https://comtrade.un.org/data/> wang, c., yang, s., jiang, cy., wu, gy., & liu, qz. (2019). monte carlo analytic hierarchy process for selection of the longwall mining method in thin coal seams. journal of the southern african institute of mining and metallurgy, 119, 1005-1012. doi: https://dx.doi.org/10.17159/2411-9717/17/375/2020 https://doi.org/10.1016/j.jclepro.2007.08.003 https://comtrade.un.org/data/ https://dx.doi.org/10.17159/2411-9717/17/375/2020 ijahp news and events: mu, e./carlow university award international journal of the analytic hierarchy process 292 vol. 5 issue 2 2013 issn 1936-6744 enrique mu wins carlow university award enrique mu, phd, an associate professor in the school of management at carlow university, and editor-in-chief of the international journal of the ahp, was awarded the dorothy weber cochran '43 award for excellence in faculty scholarship and research at the honors convocation at carlow on thursday, april 18, 2013. the cochran award is the highest level of recognition for faculty scholarship and research at carlow. carlow provost margaret mclaughlin and enrique mu "dr. mu's scholarship in the field of information technology (it) and managerial decision-making using the analytic hierarchy process is an example of the kind of scholarship that this award is meant to recognize," said margaret mclaughlin, phd, provost of carlow university. dr. mu worked with city of pittsburgh officials applying ahp to choose its cloud technology, and this work earned him a proclamation from the mayor of the city, honorable luke ravenstahl, who said dr. mu's contribution was key in making the decision. most remarkable among dr. mu's recent research activities is his work in the use of decision-making methodologies for eyewitness identification, which was funded by carlow's grace ann geibel institute for justice and social responsibility as a signature project. his paper on the subject, co-authored with rachel chung, phd, also on the mba faculty at carlow, won the best paper award at the international symposium of the analytic hierarchy process in 2013 in kuala lumpur, malaysia. mu is an accomplished scholar and educator, who besides being editor-in-chief of the ijahp, is the u.s. director for the latin american society for strategy. prior to joining carlow, he served as program director for the dual-degree mba/mis program in the katz graduate school of business at the university of pittsburgh. before entering academe, he worked for black box corporation, a pittsburgh-based data communications company, as business development manager for latin america and as a customer support vice president for comlasa ltd., a computing solution firm, in the same region. he has also been a project manager for several multimillion-dollar online banking automation projects in peru, colombia, venezuela, and chile. rob typewritten text http://dx.doi.org/10.13033/ijahp.v5i2.175 ijahp news and events: merritt/alaskan salmon plan wins environmental award international journal of the analytic hierarchy process 407 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.512 alaskan salmon plan wins environmental award submitted by margaret merritt, ph.d. resource decision support, fairbanks, alaska the matanuska-susitna (mat-su) salmon research, monitoring and evaluation plan for upper cook inlet, alaska was the winner in the environmental category of the alaska chapter, american planning association (apa) 2015 awards program. the plan, which used the analytic hierarchy process (ahp) to identify, structure and prioritize specific issues related to program goals and objectives, was showcased in november, 2015 during the apa’s annual convention in anchorage, alaska. ms. frankie barker, environmental planner with the mat-su borough, and terry nininger, fish and wildlife commission member, received the award. terry nininger, fish and wildlife commission member (left), and frankie barker, environmental planner with the mat-su borough accepting the 2015 alaska american planning association environmental award ijahp news and events: merritt/alaskan salmon plan wins environmental award international journal of the analytic hierarchy process 408 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.512 initiated in june 2014, the plan was funded with a grant from the state of alaska, department of commerce, community and economic development to develop a strategic research plan that provides partners and governing agencies with information needed to manage, protect and improve upper cook inlet salmon stocks for optimum benefits while maintaining biological productivity and diversity. the plan concerned all five species of salmon (chinook, coho, sockeye, chum and pink). a comprehensive salmon research plan for upper cook inlet had not been previously developed. mat-su borough and upper cook inlet waters addressed by this plan guided by a core planning team, the seven-member mat-su borough fish and wildlife commission established the plan’s scope, goals and objectives. a stakeholder group, consisting of 15 representatives from local, state and federal government agencies, commercial, sport fishing and personal use fishing groups and environmental organizations participated in a two-day facilitated planning workshop to identify and prioritize specific issues related to plan objectives. in addition, the group brainstormed various options that could be implemented to address issues. the public was invited to attend the planning workshop, and review and comment on the draft plan prior to its finalization. a total of 55 research, monitoring and evaluation issues were identified. issue priorities reflected a combination of goal and objective priorities and issue significance discussed by the stakeholder group. individuals in the group often had differing views on the significance of specific issues depending on their interest and expertise. informationsharing and debate during the planning workshop improved agreement within the group. the final priority rankings reflected the combined perspective of the group. with $1.6 million available for funding, in 2015 the commission used the final plan to solicit and rank salmon research, monitoring and evaluation project proposals, where proposal evaluation criteria included the priority of the issue addressed, as well as other criteria (i.e., scientific merit, expertise of the investigators, etc.). a total of 11 projects were selected for funding out of a total of 35 proposals received. these projects will be ongoing through 2018. for more information, the plan can be downloaded from the mat-su borough fish and wildlife commission site: https://www.matsugov.us/boards/fishcommission . https://www.matsugov.us/boards/fishcommission ijahp essay: t. saaty/ our world is moving from fragmentation to integration international journal of the analytic hierarchy process 114 vol. 5 issue 1 2013 issn 1936-6744 our world is moving from fragmentation to integration thomas l. saaty university of pittsburgh katz graduate school of business pittsburgh, pa 15230 saaty@katz.pitt.edu our world today is more and more moving from fragmentation to integration, becoming better unified and interactive in its economics, information sharing, travel, diplomacy, and in medical knowledge and instruments and the importance of health care and even in waging wars. there is more freedom for individuals to express themselves. it is by having a one world view that we will be able to make the best decisions. as more people express themselves, they need a way to make decisions together. conflicts can be resolved rationally and peacefully if concessions can be traded off and with the ahp, which allows for the measurement of intangibles alongside tangibles, better decisions can be made about the tradeoffs. tradeoffs can be balanced so that the parties can use them to offer concessions that will result in what they can consider to be a fair outcome. this method of trading off concessions has been demonstrated successfully in the 1980s conflict in south africa, in the northern ireland conflict, and more recently in the israelipalestinian conflict in an exploratory negotiation that is still in progress. the accuracy of the outcome is assured by making judgments directly and then performing sensitivity analysis to see if changing some judgments leads to a different outcome. seeing the big picture and being able to combine pieces of thinking, including both positive and negative aspects of the problem, is possible because analysis and synthesis are done in a manner that makes sense to our brains. overall change and the acceleration of change that we are experiencing is affecting our human psychology. we, as individuals and as groups, need to be able to cope with the unpredictable changes and growing complexity in our world. stress, uncertainty and frustration increase, minds are overloaded with information and knowledge fragments, and values erode. negative developments are consistently overemphasized, while positive ones are ignored. the resulting climate is one of nihilism, anxiety and despair. while the wisdom gathered in the past has lost much of its validity, we don't have a clear vision of the future either. as a result, we need to look for new things to guide our actions. the ahp way of thinking, is based on an understanding of the relativeness of all mailto:saaty@katz.pitt.edu http://pespmc1.vub.ac.be/tecaccel.html http://pespmc1.vub.ac.be/chinneg.html http://pespmc1.vub.ac.be/chinneg.html ijahp essay: t. saaty/ our world is moving from fragmentation to integration international journal of the analytic hierarchy process 115 vol. 5 issue 1 2013 issn 1936-6744 things, exmined through making comparisons. perhaps an axiom of comparison, that a thing cannot be known until it is compared with something else, should have been included by plato and aristotle along with their three axioms of identity, contradiction and excluded middle; to deal with complexity. starting by making simple comparisons can help us deal with this new complexity. according to the swiss born french philosopher jean-jacques rousseau (1712 –1778), original (“natural”) man had no language, no abstract thought, no moral ideas and no society. he was self-centered, but not cruel, and he felt compassion for his own kind. he was good, but not capable of moral values nor of self-sacrifice. rousseau believed that in society man is “naturally” more evil than virtuous and that his self-interest is in constant opposition to the general interest with social life being characterized by the alienation of men from nature, from each other and from authentic selves. the cure requires the fabrication of a new man and the proper political institutions. it is not enough for men to obey the laws. their minds and wills must also be engaged. the ahp with its group decision making capabilities can help individuals and groups find their best overall priorities. by knowing our priorities perhaps we can take a longer-term perspective and be able to construct a large portfolio of solutions to the world-wide problems of the future facing us. we who work in multicriteria decision making are the best to collect and harness these diverse problems and to try to relate and unify them so that our resources, attention and energy can be appropriately distributed to give us the best yields. we also need to identify the most threatening and serious conflicts in the world and try to get the parties from all sides to identify the issues and the concessions they expect to make and those they expect to receive. finally, we need to teach our children and families how to systematize their decision making so they learn to think carefully about making rational choices. the most urgent task facing us, i believe, is to find honest politicians who can learn to use ahp/anp decision making, when necessary sacrificing their self-interest in favor of those of the broader society, overriding their compulsion to be re-elected. rob typewritten text http://dx.doi.org/10.13033/ijahp.v5i1.183 our world is moving from fragmentation to integration a style guide for papers submitted to the international journal of the analytic hierarchy process ijahp news and events: saaty, rozann / isahp 2013 meeting in kuala lumpur, malaysia international journal of the analytic hierarchy process 177 vol. 3 issue 1 2011 issn 1936-6744 ahp news isahp 2013 meeting in kuala lumpur, malaysia a few of the participants at the isahp 2013 malaysia meeting the 2013 isahp meeting took place in kuala lumpur, malaysia, june 19-23. this international symposium attracted approximately 140 participants, educators, researchers and business people who are experts in decision making with the analytic hierarchy process and the analytic network process. more than 30 countries were represented and more than 95 papers were presented. dr. rafikul islam, of the international islamic university malaysia, served as the 2013 chairman. it was a splendid meeting, unforgettable for the breadth of the scientific contributions, the excellence of the organization and the general good fellowship enjoyed by the participants. the location of the hotel in central kuala lumpur was near the petronas towers and only a short distance from central shopping areas. the tea breaks (more like mini-meal breaks), the lunches and the gala dinner were all a delight, comprised as they were of different cuisines served at every meal to satisfy the local inhabitants: chinese, indians, native malay and muslims. ijahp news and events: saaty, rozann / isahp 2013 meeting in kuala lumpur, malaysia international journal of the analytic hierarchy process 178 vol. 3 issue 1 2011 issn 1936-6744 lunch with its wide variety of tasty food gala dinner “lucky draw” event the gala dinner, with its special performers, gave us a rollicking good time and showed off the beauty and enthusiasm of the native dances of malaysia. chairman rafikul islam conducted a most entertaining “lucky draw” at the gala dinner giving away awards for every achievement imaginable. the best male and female costume awards were deservedly won by the nigerians who appeared wearing their fantastic native costumes.. the spanish table garnered the most awards, their cheering becoming louder and more raucous with each new award they won. one of the most interesting things about the gala dinner was that the main guest speaker, a local politician told about the interesting experience he had with the analytic hierarchy process while running for the office in one of the provinces of malaysia. researchers from the islamic university of malaysia suggested, ten months before the ijahp news and events: saaty, rozann / isahp 2013 meeting in kuala lumpur, malaysia international journal of the analytic hierarchy process 179 vol. 3 issue 1 2011 issn 1936-6744 recent election, that using ahp they could predict how many seats would be won by his party and using their model they successfully predicted the exact number of seats. a paper about this written by students at islamic university won third prize in the student category and may be accessed in the malaysian proceedings on the isahp web site at www.isahp.org. this welcoming speech was one of the most truly knowledgeable and enthusiastic talks we have had the pleasure of hearing. award winning papers prizes were awarded for the three best papers from the senior researchers and from the student attendees. lecturer category. authors title of paper best paper 1 enrique mu rachel chung a new approach to eyewitness police identification best paper 2 yusuf tansel i̇ç mustafa yurdakul development of a two-stage amt option selection model to use in turkish manufacturing companies best paper 3 claudio macuada rubén alarcón astrid oddershede multi-criteria assessment to automate water treatment plants using the analytical hierarchy process student category authors title of paper best paper 1 hsu-shih shih szu-hua chen a conceptual design of a mobile healthcare device – an application of three-stage qfd with anp and triz best paper 2 syahidawati shahwan mustafa omar muhammad rating bba and mmp in their stages of product development based on maqasid framework best paper 3 datin margarita sergeevna peredaryenko dato’ hj rais hussin hj mohamed ariff the analytic hierarchy process: application to the election of the chief minister of perak, malaysia 2013 ijahp news and events: saaty, rozann / isahp 2013 meeting in kuala lumpur, malaysia international journal of the analytic hierarchy process 180 vol. 3 issue 1 2011 issn 1936-6744 sightseeing “as an internationally recognised landmark, the petronas twin towers symbolise the courage, ingenuity, initiative, and determination, energy, confidence, optimism, advancement and zest of a nation…” says tun dr. mahathir mohamad, former prime minister of malaysia (1981-2003). the towers were one of the stops on the tour of the sights around kuala lumpur on the third day of the conference. forthcoming isahp conference will be held in washington, dc, usa, in 2014 plans are proceeding to hold the next isahp conference in washington, dc, in late june 2013. stay tuned for further announcements about this meeting. rozann w. saaty managing editor of ijahp vice president, creative decisions foundation e-mail: rozann@creativedecisions.net rob typewritten text http://dx.doi.org/10.13033/ijahp.v5i1.189 rob typewritten text ahp news isahp 2013 meeting in kuala lumpur, malaysia multi-criteria assessment to automate water treatment plants using the analytical hierarchy process ijahp essay: garuti/artificial intelligence (ai) and ethical artificial intelligence (eai) international journal of the analytic hierarchy process 395 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.896 artificial intelligence (ai) and ethical artificial intelligence (eai) medical decision support system, medical sapiens (ms) claudio garuti fulcrum engineering ltd. medical sapiens spa santiago, chile claudio@medicalsapiens.cl the concept of ai (artificial intelligence) is relatively new and is being used with increasing frequency. the importance of this concept has to do with the increased capacity of what we understand as intelligence. however, it is a delicate concept and easy to misuse and/or misinterpret. a good understanding of what ai is, what it is based on, or should be based on and its forms of practical operation is required. first, i would like to discuss the difference between ai (artificial intelligence) and eai (ethical artificial intelligence). in general, ai is thought of as a program or machine capable of answering difficult questions by generating relationships related to the question within large relational databases. this leads to results where it is not always clear how they were obtained (what was the "reasoning" behind them). in the long term, this can lead to important problems due to misunderstanding and dependency. here, we would ultimately be obeying a machine based on its eventual high predictive capacity. but, in this case, who is really the decision-maker and who is just the “decisionoperator”? this becomes a very delicate question, since the human being, by losing the ability to manage or administer the decision due to his inability to understand why it turned out that way, also loses his freedom of action. some people call it the “freedom to make mistakes” although i prefer to define it as “reaching a dangerous degree of dependency.” even if we started with the hypothesis that the machine is never wrong, this assumption removes a significant degree of freedom in our decision-making process and robs human beings of this crucial process in our development (for the human species our development depends on making good decisions). as a counterpoint, ethical artificial intelligence (eai) is defined as an ai in which the human being is capable, in general terms, of understanding the sequence of steps or the procedure carried out by the ai to reach the result. if this is the case, the ai enhances the capacity of natural intelligence in humans or other species. understanding the sequence of steps or procedures performed by the ai is achieved naturally, for example, in the medical sapiens (ms) 1 system. since the ms system was 1 ms (medical sapiens) is a decision support software for medical diagnoses, emergency triage and doctor specialty derivation. mailto:claudio@medicalsapiens.cl ijahp essay: garuti/artificial intelligence (ai) and ethical artificial intelligence (eai) international journal of the analytic hierarchy process 396 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.896 built using multicriteria models (analytic hierarchy process and analytic network process) based on topological metrics in weighted environments (order topology) instead of statistical metrics (relational databases), it is a real eai system. the ms system is described in more detail later. it is important to note that i’m not detracting from the usefulness and importance of statistical metrics. in fact, there are analyses of this type within ms too (known as data analytics). the idea is not to replace one with the other, but to use each one in its proper capacity in such a way that it is always possible to follow the simulated or real reasoning process. it is also important to understand the basic capabilities of all types of intelligence that are present, or must be present, whether it is natural or artificial. on the capabilities of an ai there are three basic capabilities that all ai systems must have which are listed below in order of importance: 1. pattern recognition capability (recognition) one of the main characteristics of intelligence is the ability to recognize patterns; without this ability, no system can survive. in fact, our own brain begins to work very early on building out its capacity to do this. 2. feedback capability (adaptability) once there is the ability to recognize patterns, it is important to have feedback; that is, be able to receive new information and adapt to this new reality. this adaptability is very important to keep the system updated. however, it should be clarified that this ability should not be thought of as a replacement or substitute for the pattern recognition ability. in fact, feedback must always be used to slightly modify the pattern recognition capability so that its impact occurs progressively. the way in which the initial pattern is modified should be relatively slow as new information is added so that the suggested change can be verified. making slow changes is the way to avoid abrupt (discontinuous) changes that may no longer reflect reality. it should be noted that in decision modeling we accept inconsistency (to a maximum degree) so we can accept new information that may not be consistent with what exists. however, this degree of inconsistency is usually quite limited (in general, it should not exceed 10%) such that it allows change, but it is slow, acceptable and compatible with the existing patterns. 3. optimization capability (efficiency) finally, the system must have the ability to optimize. optimization is closely linked to improving the efficiency of the system. it means doing the same work in a smaller unit of time or space. while this ability is important to recognize, often the advancement of technology allows it to be achieved naturally (almost automatically). ijahp essay: garuti/artificial intelligence (ai) and ethical artificial intelligence (eai) international journal of the analytic hierarchy process 397 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.896 introducing the medical sapiens 2 (ms) system medical sapiens (ms) is a web system designed to help a physician make a presumptive medical diagnosis, triage, or decide which specialist a patient needs. the physician using the system will collect and describe the patient’s signs and symptoms and enter them into the system. the ms system includes patterns of many different diseases built from signs and symptoms, each with its own degree of importance and scale of intensity. then, the most likely disease can be determined by the ms system by comparing the patient’s pattern against all the disease profiles (weighted disease profiles) in its database. ms has a semiological 3 clinical approach; thus, the doctors can easily follow the logic behind the profiles and input the data required by ms. it follows the icd-10 4 classification for diagnoses and the snomed (the international global standards for health terms) so the examining physician can easily describe the disease. the ms system has approximately 5,000 signs and symptoms in its database, and each one has an associated absolute ratio scale, which results in 1,000 different disease profiles. every disease has a profile of signs and symptoms, and their relative dominance (or importance in describing a disease) is given in an absolute ratio scale that has been derived through the ahp/anp (analytic hierarchy/network process) via pairwise comparisons made by specialists (medical doctors) for each part of the body as well as mbe (medicine based on evidence). to initially construct the ms system, the human body was divided topographically into 35 parts, and then different groups of specialists for each part of the body were gathered to build a model (structure and weights) and the absolute ratio scales for each terminal criterion (sign or symptom). finally, all the models were grouped into a large network model (anp model) since the human body is one big interrelated system. for example, in the mouth, the part of the body that belongs to model 25 (the otorhinolaryngological model), there is a terminal criterion (a sign) called “color” which is one of the descriptors of tongue criterion that helps determine the diseases associated with the color of the tongue. its absolute ratio scale is shown in table 1. 2 more information at: www.medicalsapiens.com 3 the study of signs especially (semiotics). webster dictionary 4 icd and snomed are international classification of diseases. icd-10 is the edition number 10. http://www.medicalsapiens.com/ ijahp essay: garuti/artificial intelligence (ai) and ethical artificial intelligence (eai) international journal of the analytic hierarchy process 398 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.896 table 1 intensity scale for color of tongue intensity name (color of tongue) priority black 1.0000 white 0.8802 cyanotic 0.5616 erythematosus 0.2609 yellow-whitish 0.0975 therefore, if the color of the tongue is black, it is very helpful in making a diagnosis. however, if its color is yellow-whitish, it is not very helpful. it is important to note that the ratio scale associated with the criterion describes its capacity to make a diagnosis (not necessarily its degree of mortality or dangerousness). in addition to the modeling process, it is possible to include all personal, family, work and socioeconomic antecedents, which serve as a basis for ruling out some diagnoses. the input process the information given to the ms is structured in a semiological way. this is the same process that physicians use when examining a patient; they first look for signs and symptoms. therefore, physicians easily understand how to enter the ms data since it is the same way they were taught to determine signs and symptoms as they examine a patient. there is a cell for each possible symptom for all the diseases in the database. each cell has intensity options that have associated priorities. the input format used by the physicians is almost 100% via mouse by clicking on the appropriate intensity shown in his/her patient. the physician only selects cells for the symptoms that appear in that particular patient. there is one cell for each possible sign or symptom for every disease. as the physician observes a patient and notices a sign or symptom, they can easily find the symptom in the ms system due to the user-friendly way it is structured. then, the physician rates the symptom’s intensity on its associated scale (for example, color of tongue) by using the mouse to click the correct option as shown in figure 1. ijahp essay: garuti/artificial intelligence (ai) and ethical artificial intelligence (eai) international journal of the analytic hierarchy process 399 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.896 figure 1 example of template screen for otorhinolaryngological model during the physical exam. the words in bold in figure 1 indicate the line to be followed (by moving and clicking the mouse) to insert the doctor’s finding (the tongue sign). of course, the doctor only has to input the signs and symptoms that are displayed in the patient (abnormal signs or symptoms). the priorities of the signs and symptoms that are observed in the patient are then weighted and normalized by the ms system to form a profile of the patient’s possible illness. to determine what the illness is, the profile is compared, one by one, against all known disease profiles that are stored in the ms database. the disease profile that it most closely matches (most similar in terms of compatibility) is most likely the illness the patient has. for a computer, this process takes less than a second. to do this, the ms system uses the compatibility index g (garuti´s index). the g compatibility index is based in set theory and vector algebra and is able to measure the similarity or closeness between two priority vectors in weighted environments in a very precise way. more information about this index can be found in measuring in weighted environments (garuti, 2016). several papers have also been written about this index, the most recent being “set theory justification of g-compatibility index (generalization of jaccard index working within weighted environment)” (garuti, 2021). figure 2 graphically shows the matching process between the patient’s symptoms and different disease profiles. ijahp essay: garuti/artificial intelligence (ai) and ethical artificial intelligence (eai) international journal of the analytic hierarchy process 400 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.896 figure 2 matching process calculation in graphical terms this process is called additive medicine in contrast to the classic differential medicine. with this process, all the possible diseases are always present during the diagnostic process. every time a new symptom is added, ms reorders (re-prioritizes) all the diseases according to its new compatibility. this is an important factor since we know that diseases are intrinsically dynamic, they may change along the timeline, and it should be possible to follow that change without losing any possible diseases. covid-19 is an example of why this process is important. in differential medicine, sometimes a disease may be overlooked because one relevant symptom isn’t present; however, that symptom can appear later. the output process once the pertinent information about a patient has been completed, the ms system displays the information structured in two blocks, anamnesis (patient interrogation) and patient examination. at this point, the doctor can request an ms opinion which is in the form of a prioritized list of diseases expressed in percentages of similarity (compatibility) between the profile of the patient’s disease and the profiles of the diseases included in the ms database. in addition, the output screen has a column where the doctor can find which signs and symptoms could increase the degree of similarity between the patient’s profile and the ijahp essay: garuti/artificial intelligence (ai) and ethical artificial intelligence (eai) international journal of the analytic hierarchy process 401 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.896 disease profiles. this provides a refinement tool for analyzing the signs of the disease and making a different diagnosis. this will help new doctors gain knowledge about the behavior of specific diseases. also, ms can be used as a “learning tool” by students of semiology in the study of medicine. it is important to state that the doctor is ultimately the one who decides the diagnosis of the patient’s disease. however, the ms system helps provide the doctor with an important set of possible diseases in a prioritized form. before using the ms system, the doctor may not have known about the existence of a particular disease or been familiar enough with it to recognize that disease pattern. other features of the medical sapiens (ms) system  ms is compatible with integration in any clinical record system.  ms is an agile system, with an ability to add new diseases, new presentations or evolutions of existing diseases.  ms requires no more than 5 or 6 minutes for the doctor to fill in the necessary data once he/she is trained. on the relationship between the capabilities of the medical sapiens (ms) system and eai regarding the relationship of the medical sapiens system (a medical decision support system) with eai, ms meets the first and most important condition, pattern recognition ability. pattern recognition is one of the main characteristics of the ms medical decision support system. the second capability, feedback, is currently being done manually. however, one of the short-term goals is to automate the process of updating the disease patterns in the system with the new information gathered during the process of comparing the initial diagnosis with the final (discharge) diagnosis and analyzing the closeness or compatibility between any two patterns. it should be noted that constructing new patterns of diseases that have not been known to exist or that were previously not included in the system is possible in the same way. we conclude that the ms system fully complies with the first two capabilities that are most important for any ai system. the last capability of optimization represents a continuous process that must be done in parallel with the natural growth of the system. for example, a new database administrator might accelerate the process or find a new way to input data that makes it easier for the patient or doctor to enter the initial data in the ms system. this new way to input data is currently being worked on; we are programming a graphical way to describe where the pain is located on a human body as specifically as possible using software with zoom capability, which makes it easier for the patient, a nurse or a doctor to pinpoint the location of his/her pain. a simplified version of the three capabilities of the medical decision support system (medical sapiens) is shown in figure 3. ijahp essay: garuti/artificial intelligence (ai) and ethical artificial intelligence (eai) international journal of the analytic hierarchy process 402 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.896 figure 3 process conceptualization of the three aei capabilities conclusion the use of the ahp/anp to construct medical decision models in combination with the g compatibility index (used for pattern recognition in weighted environments) has shown a capacity to greatly assist doctors. it can especially help new doctors and general practitioners learn better diagnostic practices. it shortens the time to reach a final diagnosis, ask for specific exams, and direct the patient to the correct specialist. also, ms can speed up learning about rare diseases, new presentations of old diseases, or the popup of new diseases such as covid-19. furthermore, the way that ms was built allows the clinician to continually maintain control over the system, and to follow and understand why the results at any step were delivered by ms, which is a main requirement of an eai. patterns recognition (pattern of disease) feed-back (disease pattern adaptation) optimization (process efficiency) ijahp essay: garuti/artificial intelligence (ai) and ethical artificial intelligence (eai) international journal of the analytic hierarchy process 403 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.896 references garuti c., (2021). set theory justification of g-compatibility index. generalization of jaccard index working within weighted environment. journal of multicriteria decision analysis, 27, 1-2. doi: 10.1002/mcda.1667 garuti, c. (2016). measuring in weighted environments: moving from metric to order topology (knowing when close really means close). in de felice, f., petrillo, a. & saaty, t. (eds.) applications and theory of analytic hierarchy processdecision making for strategic decisions. intech. doi: 10.5772/63670 garuti, c. (2014). compatibility of ahp/anp vectors with known results. presentation of a suggested new index of compatibility in weighted environments. isahp2014 international symposium of the analytic hierarchy process. garuti, c., & escudey, m. (2005). decision making in complex environments. (1st ed.), santiago, chile: universidad de santiago. jaccard p. (1901). distribution de la flore alpine dans le bassin des dranses et dans quelques régions voisines. bulletin de la société vaudoise des sciences naturelles 37, 241-272. saaty, t.l. (2001). the analytic network process: decision making with dependence and feedback. (1 st ed). pittsburgh, pa: rws publications. saaty, t.l. (2010). the analytic network process: decision making with dependence and feedback. (2 nd ed). pittsburgh, pa: rws publications. http://dx.doi.org/10.1002/mcda.1667 ijahp essay: saaty/pairwise comparisons and their contribution to understanding consciousness international journal of the analytic hierarchy process 199 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.381 pairwise comparisons and their contribution to understanding consciousness thomas l. saaty distinguished university professor university of pittsburgh pittsburgh, pennsylvania, usa e-mail: saaty@katz.pitt.edu making comparisons of elements with respect to criteria is an ability we are born with as are all conscious living things. we derive priorities using informed judgment from comparisons of elements in regard to influence: dominance of the importance, preference or likelihood with respect to a property they share. priorities represent a rank order of the things that are compared. they also represent proportionality of the numbers that represent that order. the priorities are derived from pairwise comparison matrices which lead to the consideration of eigenvalues, in particular the principal eigenvalue, of a positive and occasionally nonnegative matrix. this eigenvalue becomes a useful tool in the measurement of inconsistency of judgments. when we use a hierarchic structure to make a decision we must synthesize these different priorities through multiplication followed by addition in an appropriate manner. with network structures, we use the idea of a supermatrix. in the supermatrix, the entries are block matrices whose columns are eigenvectors derived from paired comparison matrices. the columns of the supermatrix must be stochastic in order for it to converge. this is accomplished by weighting each column of block matrices so the columns of eignenvectors in the blocks sum to one and we can raise the weighted supermatrix to powers to obtain the limiting priorities that represent the answer we seek. with a neural structure, we need eigenfuntions to represent the priorities. the firing of each neuron can be represented by an eigenfunction, an exponentially damped periodic function of period one. there are four fundamental kinds of eigenfunctions that belong to the four division algebras: real, complex, quaternionic and octonionic. each kind is suitable for certain kinds of sensations and thoughts. an article that investigates the interpretation of how these eigenfunctions can be used to describe consciousness in the neural domain will appear in the journal of neural networks. beyond that are other hypercomplex numbers that in addition to non-commutativity as with quaternions and octonions and non-associativity as with octonions are inadequate to characterize them. for example, sedonions which are non-commutative and non-associative have the peculiar property that the product of two non-zero elements can be zero as can happen with multiplying matrices. mailto:saaty@katz.pitt.edu ijahp essay: saaty/pairwise comparisons and their contribution to understanding consciousness international journal of the analytic hierarchy process 200 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.381 a useful contribution of the anp in applications is that it opens a new vista into nonlinear thinking that involves feedback for which language alone is unable to cope, by expressing in words and sentences, the consequences of all the influences that work together to shape the future. logical thinking is linear and language itself is linear and moves from assumptions to conclusions linearly. the decision framework of benefits, opportunities, costs and risks has such complexity in it that logic and language alone cannot cope. kurt gödel’s incompleteness theorem about undecidable proofs may also be questioned in using the comprehensive but incomplete network structures in complex decision making. this is because a built in assumption is that the system they represent is always open to adding new factors that influence the outcome. a style guide for papers presented to the international journal of the analytic hierarchy process ijahp essay: e. mu/ publishing articles in the ijahp international journal of the analytic hierarchy process 163 vol. 4 issue 2 2012 issn 1936-6744 publishing articles in the international journal of the analytic hierarchy process enrique mu ijahp editor-in chief carlow university pittsburgh, pa 15213 email: emu@carlow.edu essay the ijahp receives contributions from all over the world. although the diversity of applications, approaches and world paradigms is one of the strengths of this journal, it can also create problems in terms of authors’ approaches and expectations. the goal of this essay is to present a set of criteria that the editorial board applies to submitted articles, and recommendations for authors on how to get their submissions published. we believe these suggestions will be useful not only for articles submitted to this journal, but for ahp/anp related research papers submitted to other international publication outlets. the ijahp’s mission is to advance the use, development and education of decision making using the analytic hierarchy/network process at all levels worldwide. this mission is broad enough to encompass both articles dealing with theoretical issues as well as those concerned with practical applications. we would like to outline the criteria that the editors use to decide if an article is suitable for the journal and eligible to be sent on for full review. notice that this means we follow a pre-screening process, even prior to forwarding the papers to the journal reviewers. general criteria for potential publications the first general criterion that the editors consider when a paper is submitted for publication is whether or not the theme of the article fits with the mission of the ijahp. as previously mentioned, this journal focuses on the use, development, and education of decision making using ahp/anp. therefore, any papers submitted should be aligned with this mission. we will not accept submissions whose focus is on the use of different multi-criteria decision-making methods and ahp/anp is mentioned merely as an afterthought. furthermore, we will not accept submissions that deal only with methodologies such as data mining and ahp/anp are not mentioned. the second criterion that editors consider is the scholarship of the problem addressed in the paper. if the paper discusses a theoretical ahp/anp problem, the editors would like ijahp essay: e. mu/ publishing articles in the ijahp international journal of the analytic hierarchy process 164 vol. 4 issue 2 2012 issn 1936-6744 to know first, why this problem is important; second, what the problem itself is; and finally, what the paper’s contribution to addressing this problem will be. unfortunately, many authors begin their articles indicating that they are going to compare method x with ahp/anp. they then compare the methods (e.g. by addressing the same problem from both perspectives), and conclude that both methodologies provide similar results (e.g. by using saaty’s comparison index). in these situations the editors would like to know why method x was chosen for the comparison. is there a long standing discussion in the theoretical or practitioner literature about this method and ahp/anp? does method x provide nuances that ahp/anp does not? merely concluding that both methods provide similar results is not enough. the next logical questions should be: why do both methods produce similar results? when may they not produce similar results? if the results are similar, when should method x be used and when should ahp/anp be used? practical applications are subject to similar considerations. how important is the problem at hand? granted, a problem may be very important for a certain group of people, and relatively unimportant for others. since ijahp is read worldwide by a large number of scholars and practitioners, the editors look for the global relevance of the problem at hand. sometimes, the practitioner’s problem may be local or specialized in nature, but the implications of the solution may have far reaching importance. the originality of the problem is important since novel problems have had little prior analysis. for example, it is unlikely that an article about buying a car or selecting a place to go on vacation would be published in this journal, unless, the methodology would offer a new perspective about how to perform this type of ahp/anp analysis. the third criterion considered by the editors is methodology. the most important part of using ahp/anp is the development of the model to address the specific decision. for this reason, knowing how the model was created is important because it allows the reader to assess its validity. while the editors could certainly build an ahp/anp model about how to address our current economic problems, this model would certainly be of little help in comparison to that developed by a group of economic experts. for this reason, the participants involved in building the model, their qualifications, the way their different views were addressed and their judgments combined, and the factors that were hotly debated, etc. constitute critical information to include in the paper so the reader can assess the validity of the model. many authors provide no information about these aspects of model construction, and often do not provide even a cursory literature review. statements like, “upon careful discussion of the problem and relevant literature review, the following hierarchy was obtained…” are not sufficient. many submissions have larger sections dedicated to the discussion of how ahp/anp operates, something that is well-documented everywhere than to the discussion of how the model was created, something that is totally unknown to their readers. the fourth criterion for consideration by the editors is the content that is included in writing about an ahp/anp study and the reporting of the results. although there are not defined standards about ahp/anp reporting (and discussion of this topic alone could ijahp essay: e. mu/ publishing articles in the ijahp international journal of the analytic hierarchy process 165 vol. 4 issue 2 2012 issn 1936-6744 require a full article), some best practices have been established. for example, for ahp, it is important to show the hierarchy and define/explain each of its elements including samples of the various types of pairwise comparison matrices, along with their consistency indices. similarly, sensitivity analysis and its implications should be thoroughly investigated and discussed. potential authors are encouraged to review published articles in the ijahp and similar journals to become familiar with other best practices. in general, authors should review the reporting requirements, based on published articles, of their target journals. the final criterion that the editors consider is the quality of the writing. the article must be pleasant to read and easy – or at least possible – for an educated reader familiar with the ahp to understand. writing in a foreign language is always a challenge, and. the ijahp editors are sensitive to this. we go the extra mile to understand the gist of the article before sending it for full review, and to copyedit accepted articles to make them as clear as possible for our readers. however, it is the author’s responsibility to provide an understandable manuscript to begin the process. we certainly recommend that potential authors for whom english is not their first language use the services of colleagues and free lancers to help copyedit their articles for readability prior to submission. finally, a style guide is provided to demonstrate expected formatting and other requirements. authors that submit papers that do not follow the journal submission guidelines create the impression of sloppiness, which casts an undesirable and perhaps undeserved shadow on the reliability and quality of the submitted article. so please be sure to read and follow the style guide for papers. in summary the goal of this article has been to provide potential authors with the general criteria followed by the ijahp editors when deciding if an article is deemed potentially suitable for the journal and should therefore be sent for full review by peer evaluation. these general criteria are: 1) topic fit with the ijahp aims, 2) scholarship or clarity and relevance of the research problem, 3) sound methodology, 4) content or sensible reporting, and 5) clarity of writing. we believe that these general criteria are useful for publication not only in ijahp but in any other journal dedicated to our field. this article is not intended to be an exhaustive discussion about recommendations for acceptable ijahp submissions; other aspects of this topic will be discussed in future issues. rob typewritten text http://dx.doi.org/10.13033/ijahp.v4i2.159 publishing articles in the international journal of the analytic hierarchy process general criteria for potential publications in summary ijahp article: saaty/five ways to combine tangible with intangibles international journal of the analytic hierarchy process 372 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.395 five ways to combine tangibles with intangibles thomas saaty distinguished university professor university of pittsburgh pittsburgh, pennsylvania, usa e-mail: saaty@katz.pitt.edu abstract this paper presents five different ways to establish weights for the criteria that govern making comparisons. four of these can be done in the context of the ahp, but the fifth and most reliable one is obtained by using the anp. keywords: ahp; anp; decision making; mcda; tangibles; intangibles 1. introduction in decision making there may well be a mix of tangible and intangible criteria in terms of which the alternatives are evaluated. in some cases one may wish to treat the tangible criteria in the same way as the intangible ones and use judgment to determine their relative importance. however, one often uses the data with known existing measurements to rank the alternatives on these criteria the question then is how to use these data alongside the priorities derived by using judgments for the remaining criteria that are intangible. there are five different ways that govern making comparisons to establish weights for the criteria. four of these can be done in the context of the ahp, but the fifth and most reliable one, is obtained by using the anp. 1. the first is the simplest. when there are tangible measurements in the same unit under the criteria, and the criteria are thought to be as important as the measurements of the elements under them then each criterion inherits the proportion of the sum of the weights under it to the total weights under all the criteria that are evaluated using the same scale, such as money. this procedure is applied to all the criteria with a common measurement scale and the alternatives synthesized for each such scale into a single overall criterion such as an economic criterion with several sub-criteria measured in dollars, a technical criterion with sub-criteria measured in meters, and so on. these aggregate criteria are then compared pairwise as to their relative importance with respect to higher level criteria or goals. mailto:saaty@katz.pitt.edu ijahp article: saaty/five ways to combine tangible with intangibles international journal of the analytic hierarchy process 373 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.395 2. the second is when the criteria have importance that is not equal based on experience and determined when they are pairwise compared with respect to higher level criteria or the goal. the resulting top-down priorities for the criteria are multiplied times their inherited proportional weight from the measures of the alternatives under them and the result is normalized to give the final priorities for the criteria. if new alternatives are added or subtracted the proportional priorities of the criteria need to be re-calculated and the multiplication and normalization process repeated. 3. the third way is when there are no measurements; here one simply uses judgments to compare the criteria with respect to higher criteria or the goal with respect to their importance. this is useful when standards are established because of long-term experience and applied to alternatives as they are evaluated. 4. the fourth way to evaluate the criteria is to assume there is inadequate knowledge about them and compare widely ranging alternatives under them and later compare the criteria with respect to higher level criteria or the goal having discovered more about their importance by exploring their occurrence in actual but diverse alternatives. 5. the fifth way is to assume that the criteria depend on the alternatives. there are two ways to include this information. in a straight ahp model, compare the alternatives under each criterion for preference first, and after this education implicitly include the knowledge gained in comparing the criteria with respect to the goal. by turning to the anp, it is possible to compare the alternatives for preference with respect to the criteria, the usual ahp way, but also compare the criteria with respect to how important they are in each of the alternatives. note that if the criteria have more importance than simply that obtained from data about the alternatives, they can be compared with respect to a goal as was done before, then multiply those priorities times the priorities obtained from the alternatives. note that if one were to use a formula from which the measurements of the alternatives are obtained, the criteria would inherit weights involving such a formula. similarly, if one were to apply a formula to assign the criteria weights in terms of higher order criteria, then the same procedure would be used in weighting those higher level criteria. this process can be generalized to all the measurements in a hierarchy or a network. 2. theory/calculation theorem: a necessary and sufficient condition that multiple criteria tangibles measured on the same scale satisfy the normalization condition on the priorities of intangibles is that the measurement of each alternative with respect to a criterion is weighted by the ijahp article: saaty/five ways to combine tangible with intangibles international journal of the analytic hierarchy process 374 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.395 sum of the measurements of all the alternatives with respect to that criterion divided by the sum of the measurements with respect to all the tangible criteria that use measurements on that scale. proof: let ( ) s ij m w , i =1,…, n; j=1,…,m; s=1,…,s be the matrix of measurements of tangible i with respect to criterion j with scale s. for simplicity assume that we have a single scale of measurement such as length in meters and thus we let s=1 and avoid using the superscript s. to combine the measurements of alternative for all the criteria for an alternative i, we form the sum 1 m ij j w   . we then normalize by dividing by each such sum by the total sum of all the measurements of the alternatives with respect to all the criteria obtaining 1 1 1 / . m n m ij ij j i j w w      now assume that if instead of the forgoing we were to normalize the measurements of each alternative with respect to its criterion measurements we have the ratio 1 / n ij ij i w w   and if we add these ratios for all the criteria, we obtain 1 1 / n n ij ij j i w w     . normalizing them for the alternatives we obtain 1 1 1 1 1 1 1 1 1 1 1 1 1 ( ) / ( ( )) ( ) / ( ) / . n ijm n m m m m n m ij ij ij i ij ijn n n n j i j j j j i j ij hj ij hj i h i h w w w w w w w w w w                            for the two results to coincide we must normalize each entry with respect to a criterion by multiplying 1 / m hl hj j w w   by 1 1 1 / m n m hk ij k i j w w      and then summing over l and we have: 1 1 1 1 1 1 1 1 m m hk hlm hl k l m n m n m l hj ij ij j i j i j w w w w w w                                   ijahp article: saaty/five ways to combine tangible with intangibles international journal of the analytic hierarchy process 375 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.395 we now prove the uniqueness of this solution. the supermatrix corresponding to the theorem is given by: 111 21 1 2 1 1 1 212 22 1 2 1 1 1 1 2 1 2 1 1 1 111 12 1 2 1 1 1 221 22 1 2 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 n m m m j j nj j j j n m m m j j nj j j j m m nm m m m j j nj j j j m n n n i i im i i i m n n i i im i i ww w w w w ww w w w w w w w w w w q p ww w w w w ww w w w w                                   1 1 2 1 2 1 1 1 0 0 0 0 0 0 n i n n nm n n n i i im i i i w w w w w w                                                                     the entries of the block matrix q at the top right are the normalized values of the rows of ( ) s ij m w and the entries of the bottom left block p are the normalized values of the columns of ( ) s ij m w . let 1 2 1 1 1 1 1 1 1 1 1 , , , t n n n i i im i i i n m n m n m ij ij ij i j i j i j w w w p w w w                            and 1 2 1 1 1 1 1 1 1 1 1 , , , t m m m j j nj j j j n m n m n m ij ij ij i j i j i j w w w q w w w                            be the vectors of the normalized rows and ijahp article: saaty/five ways to combine tangible with intangibles international journal of the analytic hierarchy process 376 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.395 columns of ( ) s ij m w , respectively. note that pp q and qq p . thus, we have pqq q and qpp p . the vector q is the vector of interest in the theorem. since the matrices pq and qp are stochastic the vectors q and p are their principal right eigenvectors and are thus unique to within a multiplicative constant. the transformation of absolute numbers to relative numbers has little influence over how meaning is assigned to generate priorities on a relative scale whose ratios may not be the same as those of the corresponding absolute numbers. priorities should not be combined with measurements unless they coincide with them, in which case no difficulties arise. however priorities based on information from different scales are a generalization that requires comparison of the criteria with respect to a higher criterion. for emphasis, note that after absolute numbers are converted to priorities, one cannot take the final scale and treat it arithmetically as if it is still the original scale of absolute numbers. 3. examples of the five ways to combine we now illustrate, with examples, how to deal with each of the five ways to combine criteria mentioned above. 3.1 tangibles as intangibles—first case suppose we wish to determine the best value for three vacation sites a, b, c in terms of total travel plus lodging cost as shown in table 1. ijahp article: saaty/five ways to combine tangible with intangibles international journal of the analytic hierarchy process 377 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.395 table 1 choosing the best vacation site alternatives ai criteria travel + lodging cost priorities (ai cost/total cost) c1 travel costs ($) c2 lodging costs ($) a 50 200 250 0.278 b 100 170 270 0.333 c 150 230 380 0.422 total 900 site a would be the preferred site as it has the least travel plus lodging cost. dividing the cost of each alternative by the total cost of $900 yields the fraction of the cost due to the alternative. or, to put it another way, it is the relative cost of the alternative. the total of the travel costs, criterion is 50 + 100+ 250, or $300 and the total of the lodging costs, criterion 2, is 200 + 170 + 230, or $600. thus, travel costs are 300/900 or 1/3 and lodging costs are 600/900 or 2/3 of the measurements in money. these proportions are the priorities of the criteria and sum to 1.0. the same problem can be studied with a hierarchic interpretation where the relative costs or priorities of travel and lodging for each alternative are multiplied by the weights of the criteria (the total costs of travel and lodging) and summed. we obtain the following results which are the same as the second column on the right in table 1: cost a = 300 ´    50 300 + 600 ´    200 600 = $250 cost b = 300 ´    100 300 + 600 ´    170 600 = $270 (1) cost c = 300 ´    150 300 + 600 ´    230 600 = $380 to convert dollar unit measures to priorities divide the cost for each alternative by the sum of the costs for all the alternatives. this converts dollar units to priorities. the sum of a set of priorities should always equal 1. thus we have for the three alternatives: priority of a = $250/$900 = 0.278 (1) priority of b = $270/$900 = 0.300 (2) priority of c = $380/$900 = 0.422 (3) ijahp article: saaty/five ways to combine tangible with intangibles international journal of the analytic hierarchy process 378 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.395 the criteria derived their importance from the alternatives because the units are the same for both criteria, i.e. dollars, so the quantities 300 and 600 can be used to determine the relative importance, or priorities of the criteria c1 and c2. another way to think of it is, how much money does each criterion control contribute? it is natural to conclude that the more money controlled by a criterion, the more important that criterion is. if we compare these criteria for importance with respect to the goal of selecting the best vacation site, we have: table 2 pairwise comparing criteria for importance goal c1 c2 priorities c1 1 1/2 .333 c2 2 1 .667 3.2 tangibles as intangibles—second case to pairwise compare a, b, and c with respect to each criterion, one uses ratios of costs from table 1 to fill out the pairwise comparison matrix. when a is compared with b for relative cost with respect to travel one has 50/100 = 1/2 placed in the (a, b) cell and so on. the pairwise comparison matrix for the alternatives with respect to travel and lodging respectively are given in table 3 and table 4. table 3 relative priorities of the alternatives for travel cost and lodging cost travel cost lodging cost when weighting the priorities of the travel cost and lodging cost of the alternatives by the priorities of criterion 1 (0.333), and criterion 2 (0.667), and adding, we obtain: cost a = .333 x .167 + .667 x .333 = .278 cost b = .333 x .333 + .667 x .283 = .300 cost c = .333 x .500 + .667 x .383 = .422 alternatives a b c priorities a 1 1/2 1/3 0.167 b 2 1 2/3 0.333 c 3 3/2 1 0.500 lodging ( c2) a b c priorities a 1 2/1.7 2/2.3 0.333 b 1.7/2 1 1.7/2.3 0.283 c 2.3/2 2.3/1.73 1 0.383 ijahp article: saaty/five ways to combine tangible with intangibles international journal of the analytic hierarchy process 379 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.395 which is the same as the priorities on the right in table 1 and equation (2). thus, the priorities of the alternatives as obtained by additive hierarchic composition lead to the same solution as an appropriate analysis of the original data using arithmetic. 3.3 numerical judgments as an approximation for elements of the same order of magnitude (i.e., they are homogeneous), the paired comparison judgments in the matrices may be approximated by values from the scale 1-9 based on perception. this is useful when there are no known numerical values to form the ratios. estimating by rounding the ratios to the nearest whole number, the matrices of the above example are shown in table 4. table 4 estimating relative priorities of alternatives for travel cost using the ahp fundamental scale estimating ratios of travel cost c1 for the alternatives alternatives a b c priorities a 1 1/2 1/3 0.163 b 2 1 1/2 0.297 c 3 2 1 0.540 estimating ratios of lodging cost c2 for the alternatives alternatives a b c priorities a 1 1 1 .333 b 1 1 1 .333 c 1 1 1 .333 weighting by the priorities of the criteria, and adding as in (3), we obtain the following results: cost a = .333 x .163 + .667 x .333 = .276 cost b = .333 x .297 + .667 x .333 = .321 cost c = .333 x .540 + .667 x .333 = .402 a is again the preferred alternative, and the numbers are a little different, being based on estimates, but they are fairly close. the approximation using a 1-9 scale could lead to a different choice than the best one, but there would be no need to approximate if exact numbers are known. however, in general, one needs to compare the dollar values according to the importance of their magnitudes and that depends on the individual or different individuals who may then combine or average their judgments by using the geometric mean which has been proven to be the only way to satisfy the reciprocal relation. this example demonstrates that when the criteria weights are described in terms of the unit of measurement of the alternatives, the operations of the ahp can be used to duplicate with ijahp article: saaty/five ways to combine tangible with intangibles international journal of the analytic hierarchy process 380 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.395 relative numbers the answers one gets with arithmetic. however, this is not the purpose of the ahp. the priorities associated with numbers may not vary linearly or monotonically with those numbers. in fact, for each problem the priorities would satisfy the needs of that problem according to the judgments of the individual or group involved. 3.4 inadequate knowledge about the criteria suppose we have three foods and their content measured in milligrams for the two criteria, vitamin x and vitamin y. the importance of the criteria is no longer determined by the total or average milligram content of the alternatives as before, but rather by the needs of the body for that vitamin to remain healthy. it may be harmful to get an excessive amount of one vitamin but healthy to get such an amount from the other. we must establish priorities by comparing the criteria with respect to healthful contribution, and the alternative’s milligram content for their positive contribution to meet body needs. the actual measurements in small quantities and their totals cannot determine the single best food to eat. 3.5 the case of dependence of the criteria on the alternatives all very well, one may say when dealing with more than one criterion that are all tangibles measured on the same existing scale. to make the ahp work correctly it is only necessary to observe that the criteria depend on the alternatives and weight the criteria by the proportion of the total property exhibited by the alternatives under it. as new alternatives are added to the model with varying amounts of the property for each criterion, the criteria weights need to be continually re-scaled as the “total” property belonging to each criterion changes. but what does one do with intangibles? or with properties measured in different scales? before answering that question, let us show that the analytic network process automatically accomplishes the same thing as weighting the criteria appropriately in the ahp. to do this, construct an anp model with the alternatives depending on the criteria and the criteria depending on the alternatives, and enter the known data. construct the supermatrix by normalizing each set of data. raise the supermatrix to powers until it converges to the limit supermatrix and use the priorities for the alternatives and of the criteria from that matrix. they will be the same as the priorities from the ahp with the weighted criteria. it is easy to do as one simply enters the data and it does not require any analysis of whether it is necessary to weight the criteria or not. ijahp article: saaty/five ways to combine tangible with intangibles international journal of the analytic hierarchy process 381 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.395 references saaty, t. l. (2000). fundamentals of decision making and priority theory with the analytic hierarchy process (vol. 6). pittsburgh: rws publications. ijahp: assad/obituary: thomas l. saatycreator of ahp/anp decision making methodology international journal of the analytic hierarchy process 285 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.496 obituary: thomas l. saaty – creator of ahp/anp decision-making methodology arjang assad henry e. haller jr. dean joseph m. katz graduate school of business and college of business administration university of pittsburgh pittsburgh, pennsylvania usa dear colleagues: it is with great sadness that i share the heart-rending news that dr. thomas l. saaty passed away on monday, august 14, 2017. he was 91. tom had been bravely battling a grave illness for some time and will be dearly missed. holding the highest faculty rank of distinguished university professor, tom was an exceptionally gifted researcher and prolific author, and for many years his brilliant mind illuminated the field of business analytics and operations. he is best known as the creator of the decision-making mechanisms analytic hierarchy process (ahp) and analytic network process (anp) for resolving complex problems. these versatile tools had virtually limitless practical applications. tom and his coworkers successfully applied these tools to such issues as arms control and disarmament, geopolitical conflicts, urban design, and countless business applications worldwide. tom was first recruited to the katz school in 1979 by dean emeritus h. j. zoffer. he was previously on the faculty of the wharton school at the university of pennsylvania. prior to his faculty appointments, tom held prominent positions at the arms control and disarmament agency, office of naval research, the u.s. embassy in london, and navy management office, among others. tom’s impeccable academic credentials included a phd in mathematics from yale university, with post-graduate study completed in pure mathematics at the sorbonne, university of paris, and an m.a. in mathematics from yale university, a m.s. in physics from the catholic university of america, and a b.a. from columbia union college. the impressive awards and accolades that tom garnered over the course of his lifetime are too numerous to list, but i will share a few. his cumulative citation count, which is one measure of his impact on the field, exceeds 97,000 citations in google scholar. fittingly, tom was the recipient of the university of pittsburgh’s highest honor for https://doi.org/10.13033/ijahp.v9i2.496 ijahp: assad/obituary: thomas l. saatycreator of ahp/anp decision making methodology international journal of the analytic hierarchy process 286 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.496 research, the chancellor’s distinguished research award, and tom also received our school’s h.j. zoffer medal for meritorious service in recognition of his years of service. tom also received many awards honoring his contributions to his field, including the impact prize from informs and the gold medal from the international society for multi-criteria decision making. on a personal note, i deeply grieve the loss of a good friend whom i admired. tom was very kind to me when i joined the school and frequently welcomed me to his home. it was in these one-on-one conversations that i came to appreciate his amazingly curious and creative mind. of course, tom touched the lives of many during his time with us. as a researcher, as a teacher, and as a friend and colleague, he brought honor and prestige to the institution. international scholars from all corners of the world sought tom out and came to pittsburgh to work with him. we will all miss his prodigious presence at katz. tom is survived by his wife, rozann saaty; his children, linda, michael, emily, john, and daniel; his grandchildren, tom, michael, meghan, amy, kara, monica, emma, nina, max, and fletcher; and his great grandchildren, lydia, fiona, and lilly. tom’s wishes were to not have a funeral or viewing, and the family has asked that in lieu of remembrances such as flowers or donations that people carry on the legacy of his work in their coursework and research. the family may hold a memorial event in the future, and details will be shared as they become available. sincerely, arjang assad henry e. haller jr. dean joseph m. katz graduate school of business and college of business administration https://doi.org/10.13033/ijahp.v9i2.496 ijahp news and events: karpak/ahp/anp is well represented in mcdm 2017, ottawa international journal of the analytic hierarchy process 148 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.479 ahp/anp is well represented in mcdm 2017, ottawa birsen karpak distinguished professor of management http://people.ysu.edu/~bkarpak/ associate editor, international journal of the analytic hierarchy process program committee member, mcdm 2017 mcdm 2017 is progressing well! close to 200 papers have been accepted and about 15% are ahp/anp related. there will be two invited sessions on ahp/anp theory and applications in supply chain management and industrial engineering, each having four papers, in addition to about twenty contributed papers on ahp/anp. it is not possible to identify all the papers involved in the conference that use ahp/anp by itself or combined with other methodologies since some papers are in different application areas and their title does not include ahp/anp. we wanted to highlight some of the studies in order to give you an idea about the papers that will be presented at the conference. this year we have seen paper contributions about ahp/anp even from undergraduate students! one of them, i̇rem karacakaya, has developed a decision support model for warehouse location in humanitarian relief logistics. another paper by berna ünver, proposes an integrated decision support model to calculate the risk scores of workstations in assembly lines. both students are from istanbul technical university. gozde kadıoğlu, a master’s student from istanbul, turkey, has developed a decision support model for the assessment of consumer preferences: a case study on coffee https://doi.org/10.13033/ijahp.v9i1.479 http://people.ysu.edu/~bkarpak/ https://www.ijahp.org/index.php/ijahp https://urldefense.proofpoint.com/v2/url?u=http-3a__sites.telfer.uottawa.ca_mcdm2017_&d=dwmfag&c=0w9vy5nnhl9u_frqx4vrzksnz08jjo3five6wvqrtvo&r=8siruohdnke9xopydtaqfa&m=-wxk_prgfrha3pf8_0zlc-85yelncqhk9wfapturtc0&s=hsidq7ftjk0_z20rsj0_9xf5dqzun9p4mf1rcueqznw&e= http://sites.telfer.uottawa.ca/mcdm2017/ ijahp news and events: karpak/ahp/anp is well represented in mcdm 2017, ottawa international journal of the analytic hierarchy process 149 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.479 house companies (kadıoğlu and topçu). master’s students, crystal thomas, erin whitehouse, matthew yourstowsky and robert woolley from youngstown state university, youngstown, ohio, have analyzed different export markets including germany, india, the united kingdom and brazil for international expansion using the analytic hierarchy process decision-making methodology (thomas, whitehouse, yourstowsky, woolley and karpak). another interesting paper is by claudemir leif tramarico, valerio salomon and fernando marins from sao paulo state university (unesp), brazil titled benefits, opportunities, costs, and risks from advanced planning systems implementation. the authors have assessed advanced planning systems (aps) implementation considering bocr, and the proposed model and the results provide evidences of the aps implementation in a global chemical corporation. luis vargas, from the university of pittsburgh, pennsylvania, presents, how to write a contract with the ahp in one of the invited sessions on ahp/anp theory and applications in supply chain management and industrial engineering. in this paper, the authors propose an approach based on the ahp that could be considered an extension of principled negotiation. as the authors state in the abstract, “principled negotiation looks for fair and equitable solutions to conflicts rather than finding solutions in an environment in which each party considers the other party an adversary. to find fair and equitable solutions one needs to use measurement to determine which options are: (1) best for both parties, and (2) as close as possible to each other in the value provided to the parties.” we thank the telfer school of management at the university of ottawa for hosting one of the leading conferences on multiple criteria decision-making, mcdm 2017. this international gathering of researchers and practitioners is an opportunity to highlight the latest application of mcdm tools to sustainable-management challenges and to explore optimal decision-making around social, environmental, health, safety, and performance objectives. we are all looking forward to the 24th international conference on multiple criteria decision making, on july 10 – 14, 2017 in ottawa: “canada 150: creating a sustainable society,” and also celebrating canada’s 150th birthday! https://doi.org/10.13033/ijahp.v9i1.479 ijahp article: abdou, tkiouat/an ahp application for failure risk-based ranking of electric vehicle projects international journal of the analytic hierarchy process 510 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.884 an ahp application for failure risk-based ranking of electric vehicle projects idriss abdou laboratory of applied mathematical studies and research (lerma), mohammadia school of engineering (emi), mohammed v university, rabat, morocco idriss.abdou92@gmail.com mohamed tkiouat laboratory of applied mathematical studies and research (lerma), mohammadia school of engineering (emi), mohammed v university, rabat, morocco mohamedtkiouat@gmail.com abstract in order to address challenges in the sustainable development of transportation, economy, and environment, governments along with conventional automobile manufacturers and consumers are extremely interested in the development of the electric vehicle (ev) manufacturing industry and market. however, many manufacturers are worried about entering the ev market because of some of the limitations of evs and government economic policies. a framework for failure riskbased ranking of ev projects is proposed that applies the analytic hierarchy process (ahp) as a method of ranking. the hierarchy structure of the ahp is created with the risk categories, risk factors, and ev project candidates at different levels of the decision. by specifying the failure risk categories and failure risk factors, the ranking of ev project failure risks and the ev projects are accomplished via the pairwise comparison in the ahp. the results from the ranking provide useful information for planning and decision making. in fact, the results of the proposed method make it possible to specify the ev projects that are feasible to carry out and to compare the various projects at the technical and economic level. keywords: failure risk-based ranking; ev projects; investment; multi-criteria decision making; analytic hierarchy process (ahp) 1. introduction electric mobility is understood to mean a transport or a moving performance which is intelligent, silent and has zero emission of gaseous pollutants. electric mobility adapts perfectly to the needs and energetic, economic and ecological stakes required by the various treaties, in particular minimizing environmental pollution. its use will enhance the national product in electric vehicle technology. a wide range of models and processes have been submitted by various international firms offering individual, common or goods transport in an efficient manner that partially meets the aforementioned requirements. an assessment of the overall energy consumption of the current traffic in the region encourages us to examine the interest in the integration of electric mobility, but at the same time it requires intelligent adaptation mailto:mohamedtkiouat@gmail.com ijahp article: abdou, tkiouat/an ahp application for failure risk-based ranking of electric vehicle projects international journal of the analytic hierarchy process 511 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.884 to an environment that is difficult to develop (beeton & mayer, 2014; muller & mayer, 2015; hulsmann, 2016). given the current global trend towards environmental sustainability, ev has been recognized as a promising means to reduce dependence on petroleum fuel and carbon emissions in the transportation industry. for example, evs can lower carbon emissions by 30-50% and improve fuel efficiency by 40-60% on the average. scholars and environmentalists regard evs as a generic cure for many environmental issues, such as urban smog and the energy crisis (beeton & mayer, 2014; muller & mayer 2015; nikowitz, 2016; corno et. al., 2017). previously, several ev projects have been proposed (japanese ev ecosystem, ev demonstration programme in china, hawaii ev project, charging up chile, etc.). all of these ev initiatives have faced numerous problems in reality such as countervailing pressures for change in the automotive industry; social dilemma problems; diffusion of innovations; lack of infrastructure, especially charging stations; poor performances of the battery; and the eternal comeback of the fuel cell. consequently, it will be very interesting to evaluate the risk of failure of ev projects. the ev project ranking based on failure risk becomes especially crucial when the investment decision needs to be selective and prudent (beeton & mayer, 2014; nikowitz, 2016; liebl, 2017). this ranking will help administrators identify the appropriate campaign book to make a successful ev project. this paper proposes a framework for ev project ranking based on failure risk. since the failure risk of an ev project is composed of different failure risk categories and failure risk factors which may not be mathematically defined in explicit formulas, the ranking based on the absolute measure of failure risk is not feasible. thus, a ranking method that is based on the notion of relative contribution to failure risk is employed. the applied method is the called the analytic hierarchy process (ahp). the ahp is one of the most widely used multi-criteria decision-making methods (taylan et. al., 2014; li & zou, 2011; hanin et. al., 2021). the method is simple and is able to combine both qualitative and quantitative criteria. the ahp has shown its applicability for ranking problems involving multiple criteria (taylan et. al., 2014). the multi-criteria nature of multiple failure risk categories and failure risk factors makes the application of ahp very suitable. the proposed framework realizes the comparison of ev projects with respect to the failure risks and has significant implications for economic and political decision making (li & zou, 2011). the structure of the paper is as follows. section 2 presents an overview of electric mobility in the world. section 3 describes the proposed framework. in section 4, the framework implementation is shown through an illustrative example. we conclude with some remarks in section 5. 2. overview of electric mobility in the world today, among the many solutions developed to promote the use of energy-efficient and clean modes of transport, electromobility offers interesting alternatives. indeed, in this environmental context favorable to renewable energies, electric mobility which represents an environmental, climatic, technological and societal stake has its role to play. despite some reservations about current electricity production, the expected benefits of this mobility, in terms of improving public health, air quality and reducing greenhouse gases and fine particles in the region and atmosphere, are clearly strong. ijahp article: abdou, tkiouat/an ahp application for failure risk-based ranking of electric vehicle projects international journal of the analytic hierarchy process 512 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.884 the goal of electromobility is to offer a new service, including various uses (car sharing, public transport, merchandise, individual, etc.) that are complementary to the current mobility system, particularly in urban areas. however, the development of electromobility is recent, and many aspects require significant improvements (low battery life of electric vehicles, insufficient and unequal deployment of charging stations in the territory, lack of real standards and regulatory frameworks, low acceptance of this new form of mobility by consumers, etc.). for more details, see leal and kotter (2015) and leal (2015). electric mobility has reached an advanced stage and is benefitting from various developments whose influence can be expected to become more and more important. these developments include high oil prices, carbon constraints, intermodality and the rise of organized car sharing. in fact, the development of vehicle engine technology depends on changes in infrastructure, mobility, the electricity sector and the global car market, evolution of energy prices, and climate policy (dijk et. al., 2013). also, recently several works have discussed the alternatives and barriers for an efficient logistics chain in the manufacturing industry, including the automotive sector, in the context of covid-19 (biswas et. al., 2020; biswas & das, 2020). electric mobility is not intended to replace thermal vehicles in view of the predominance of these vehicles in urban areas. however, it can be a good excuse for a gradual change in behavior. the development of electric mobility and alternative modes of transportation will only be possible with changes in the population's travel habits. for all these reasons, local authorities, through their strategies and actions, are at the forefront of promoting the development of energy efficient and clean modes of transport. through initiatives at the local level, communities can allow a gradual change in the travel habits of citizens and strategies of private actors, through numerous calls for projects. supported by the state, all these actors will be able to act together and create synergy in order to favor this type of mobility in the present and future. in table 1, we present some case studies of ev projects developed in norway, france, germany, china, japan, chile and united states with the problems experienced. ijahp article: abdou, tkiouat/an ahp application for failure risk-based ranking of electric vehicle projects international journal of the analytic hierarchy process 513 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.884 table 1 case studies of ev case study principle and objective of the study challenges and barriers china (beeton & mayer, 2014; tagscherer, 2012; zheng el. al., 2018; bresser el. al., 2018). china is the world's largest car market and showcases its products at a biennial fair in beijing. in 2015, china became the first outlet for electric vehicles; 247,000 "zero emission" passenger cars and utilities were registered this year, which was a 300% increase from 2014 according to the federation of chinese manufacturers. inadequate infrastructure. battery replacement and recycling. low willingness to pay by the consumer. norway (biresselioglu et. al., 2018; leurent & windisch, 2011). last september, 60% of car sales in norway were electric or hybrid cars. this small country of 5 million inhabitants aims to ban the sale of gasoline cars in 2025. norway has 215 clean cars per 10,000 inhabitants, which is the highest in the world. norwegians have even developed a production line for specific cars. the capacity of the electrical system to individually recharge each vehicle with economic conditions that are most adapted to the needs of the user. the adequate size of charging points for users connected. france (bresser et. al., 2018; biresselioglu et. al., 2018; leurent & windisch, 2011) the state has set up an ecological bonus along with other financial incentives to purchase an electric vehicle. these aids are available for both individuals and professionals. they are supplemented by other benefits such as exemption from taxes on company vehicles or the gray card tax. since 2008, the government has introduced an ecological bonus that encourages buyers to move towards new models with low co2 emissions for individuals and professionals. the capacity of the electrical system to individually recharge each vehicle with economic conditions that are most adapted to the needs of the user. optimizing the connection of these installations to the electricity networks. germany (biresselioglu et. al., 2018; leurent & windisch, 2011). in order to develop electromobility, the german government has granted an environmental bonus of up to € 4,000 for the purchase of an electric vehicle and € 3,000 on a rechargeable hybrid retroactive from january 1, 2016. this long-term financial incentive that was asked for by the german manufacturers could finally make sales in germany increase. the development of load control to minimize network needs and optimize the electrical demand. development of a charging infrastructure for electric vehicles. japan (beeton & mayer, 2014; bresser et. al., 2018) the japanese ev market is one of the earliest and strongest ones worldwide in terms of sales and industry entry. since 2005, a combination of factors in japan has led to the second highest levels of ev sales globally. innovative oems (nissan, mitsubishi, toyota), a proactive electric utility (tepco), and leading battery and energy companies (nec, hitachi, mitsubishi, sumitomo) headquartered in japan have entered the ev market. inadequate infrastructure for battery replacement and recycling. low willingness to pay by the consumer. ijahp article: abdou, tkiouat/an ahp application for failure risk-based ranking of electric vehicle projects international journal of the analytic hierarchy process 514 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.884 case study principle and objective of the study challenges and barriers united states (bresser et. al., 2018) in the united states, there are several ev support policies to make evs more attractive and accessible including, purchase or lease purchase tax credits (long-term lease with option to purchase) for evs. discounts on the electricity bill: this offer applies most often for recharging vehicles outside peak hours (off peak-hours). easy access to parking points: parking spaces are free. some even offer free refill possibilities (i.e., tesla superchargers). the parking time may be limited. reduction of registration fees: the registration card must be renewed every year or every two years in the united states. some states offer discounts on these fees. insurance reductions: in california, discounts are 10% on most insurance contracts. the consumer’s acceptance of evs. the proper functioning of the electrical system. development of charging infrastructure for electric vehicles. chile (beeton & mayer, 2014) multiple governmental agencies in chile have recognized the potential of electric vehicles (evs) to contribute to air pollution mitigation and reduction of net energy consumption. to achieve these goals, a target of 70,000 evs by 2020 was established by a nationally appropriate mitigation plan (nama) and e-mobility readiness plan commissioned by the ministerio del medio ambiente (mma) and the ministry of transport and telecommunication (mtt) in 2012. the high cost of electric vehicle (ev) ownership is far out of reach for the typical chilean family. limited policy incentives for electric mobility. table 1 shows that the incentives for electric mobility face several problems to achieve the planned objective. consequently, it will be important to assess the risk of failure of future ev projects and their corresponding ranking. for this purpose, we applied a multi-criteria decision method called the analytical hierarchy process. since it is classified as a developing country, chile is a model for each country that wants to rely on electric mobility. a number of important economic, technological and environmental factors combine to make chile a particularly compelling candidate for the expansion of hybrid and battery (h/b) evs in the latin american market, including the presence of an existing ev charging infrastructure (including latin america’s first ev charging station). however, the weak purchasing power of chilean citizens and lack of stakeholder’s involvement remains a real obstacle. therefore, mobilization of all parties for electric mobility integration is an important key for a successful ev project (beeton & mayer, 2014). 3. description of framework the framework proposed in this paper applies the analytic hierarchy process (ahp) as a method of ranking. the ahp was proposed by saaty (1980) for solving multiijahp article: abdou, tkiouat/an ahp application for failure risk-based ranking of electric vehicle projects international journal of the analytic hierarchy process 515 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.884 criteria decision-making problems. the decision problems are structured in a hierarchical form which includes a goal, criteria, sub-criteria, and alternatives. the comparison is carried out in a pairwise manner for each level: criteria, sub-criteria, and alternatives. saaty and mcfarlan presented the ahp method in detail in saaty (1980) and mcfarlan (1981). the ranking score of the 𝑝-th alternative is obtained from: 𝑆𝑝 = ∑ ∑ 𝑊𝐶𝑟 𝑁𝐶𝑟 𝑞=1 𝐺 𝑟=1 𝑊𝑆𝑟𝑞 𝑊𝐴𝑟𝑞𝑝 ; 𝑝 = 1, … . , 𝑀 where 𝑊𝐶𝑟 is the priority weight for the 𝑟-th criterion, 𝑊𝑆𝑟𝑞 is the priority weight of the q-th sub-criteria under the criterion r, and 𝑊𝐴𝑟𝑞𝑝 is the priority weight for the p-th alternative under the sub-criterion 𝑆𝑟𝑞 . note that under the goal there are g criteria, each of which is denoted by 𝐶𝑟. there are 𝑁𝐶𝑟 sub-criteria under the criteria 𝐶𝑟, each of which is represented by 𝑆𝑟𝑞 . 𝑀 is the total number of alternatives. the alternatives with higher ranking scores have a higher priority or rank (mcfarlan, 1981). the framework for failure risk-based ranking of ev projects has the following steps: 1. create the decision model in a hierarchy form as shown in figure 1. the model is decomposed into the goal, criteria, sub-criteria, and alternatives levels. the goal is ev project ranking based on failure risk. the failure risk of an ev project is attributed to the failure risk categories each of which includes its relevant failure risk factors (mcfarlan, 1981). the criteria level thus contains all failure risk categories whereas the sub-criteria level consists of a number of failure risk factors. the alternative level is at the bottom of the hierarchy structure and has all candidates of ev projects. 2. following the ahp method as described, perform the pair-wise comparisons for the failure risk categories, failure risk factors, and ev project candidates. compute the priority vectors as well as their corresponding cr’s. 3. determine the priority score of each ev project. 4. rank the ev projects according to their priority scores. figure 1 hierarchy structure of ahp ijahp article: abdou, tkiouat/an ahp application for failure risk-based ranking of electric vehicle projects international journal of the analytic hierarchy process 516 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.884 according to the first step in the framework, it is essential to specify the failure risk categories and failure risk factors. there is a series of identified failure risk categories and failure risk factors. however, they can be structured into similar groups. we have determined the failure risk categories and failure risk factors based on the types of problems that have been noted in some previous ev projects all over the world. the resulting failure risk factors were classified into three categories, namely, strategic, process-related and technical. the obtained failure risk categories and factors are shown in table 2 in accordance with the hierarchy structure in figure 1. table 2 failure risk 𝑹𝟏: strategic risk category 𝑹𝟏𝟏: inexperienced consumers 𝑅12: lack of stakeholders’ involvement 𝑅13: support given by government 𝑹𝟐: process-related risk category 𝑅21: unclear scope and requirements 𝑅22: unrealistic schedule 𝑅23: poor budgetary control 𝑹𝟑: technical risk category 𝑅31: poor performance of the battery 𝑅32: inappropriate project development 𝑅33: lack of infrastructures 4. case study in this case, we consider 4 ev projects that need to be ranked according to their respective failure risks (table 3). the details of 4 ev projects are presented in beeton and mayer (2014). the study was carried out under the supervision of three university professors who have specialized in the field of electric mobility (mohamed tkiouat, mohamed maaroufi and mohamed cherkaoui, professors in engineers’ mohammadia school, emi, umv rabat). the results are taken from experts after receiving the pairwise judgments as a group with consensus on each judgment. the failure risk categories and failure risk factors according to chua (2009), presented in table 2, are employed for the purpose of method demonstration. following the ahp, the decision maker evaluates the pairwise comparison matrix of each comparison from which the associated priority vector and consistency ratio cr are computed (chua, 2009; sardi & bona, 2021). in each pairwise comparison, we ask the experts which option is riskier, so that the riskiest factor receives the highest priority. ijahp article: abdou, tkiouat/an ahp application for failure risk-based ranking of electric vehicle projects international journal of the analytic hierarchy process 517 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.884 table 3 presentation of the 4 ev ev project title ev1 free-floating all-electric city cars ev2 plug-in cars as company cars ev3 all-electric car subscription ev4 leasing chain for all-electric cars tables 4-16 present the pairwise comparison with its associated priority vector and consistency ratio cr for the goal, 𝑅1, 𝑅2, 𝑅3,𝑅11, 𝑅12 , 𝑅13, 𝑅21, 𝑅22, 𝑅23, 𝑅31, 𝑅32 and 𝑅33, consecutively. the results from all analyses are summarized in table 7. note that the partial score corresponds to the term 𝑊𝐶𝑟 𝑊𝑆𝑟𝑞 𝑊𝐴𝑟𝑞𝑝 in the equation of ranking score. table 4 matrix of pairwise comparison with its associated priority vector and consistency ratio cr for the goal ev project ranking based on failure risk 𝑹𝟏 𝑹𝟐 𝑹𝟑 priority vector 𝑹𝟏 1 5 2 0.581 𝑹𝟐 1/5 1 1/3 0.109 𝑹𝟑 ½ 3 1 0.309 cr=0 table 5 matrix of pairwise comparison with its associated priority vector and consistency ratio cr for 𝑅1 strategic risk category 𝑹𝟏𝟏 𝑹𝟏𝟐 𝑹𝟏𝟑 priority vector 𝑹𝟏𝟏 1 1/6 1/5 0.081 𝑹𝟏𝟐 6 1 2 0.575 𝑹𝟏𝟑 5 1/2 1 0.343 cr=0.024 ijahp article: abdou, tkiouat/an ahp application for failure risk-based ranking of electric vehicle projects international journal of the analytic hierarchy process 518 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.884 table 6 matrix of pairwise comparison with its associated priority vector and consistency ratio cr for 𝑅2 process-related risk category 𝑹𝟐𝟏 𝑹𝟐𝟐 𝑹𝟐𝟑 priority vector 𝑹𝟐𝟏 1 1/3 1/5 0.106 𝑹𝟐𝟐 3 1 1/3 0.260 𝑹𝟐𝟑 5 3 1 0.633 cr=0.044 table 7 matrix of pairwise comparison with its associated priority vector and consistency ratio cr for 𝑅3 technical risk category 𝑹𝟑𝟏 𝑹𝟑𝟐 𝑹𝟑𝟑 priority vector 𝑹𝟑𝟏 1 3 1/2 0.309 𝑹𝟑𝟐 1/3 1 1/5 0.109 𝑹𝟑𝟑 2 5 1 0.581 cr=0 table 8 matrix of pairwise comparison with its associated priority vector and consistency ratio cr for 𝑅11 inexperienced consumers ev1 ev2 ev3 ev4 priority vector ev1 1 5 3 1/2 0.308 ev2 1/5 1 1/2 1/7 0.064 ev3 1/3 2 1 1/4 0.119 ev4 2 7 4 1 0.507 cr=0.007 ijahp article: abdou, tkiouat/an ahp application for failure risk-based ranking of electric vehicle projects international journal of the analytic hierarchy process 519 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.884 table 9 matrix of pairwise comparison with its associated priority vector and consistency ratio cr for 𝑅12 lack of stakeholders’ involvement ev1 ev2 ev3 ev4 priority vector ev1 1 1/4 2 1/5 0.112 ev2 4 1 5 2 0.472 ev3 1/2 1/5 1 1/4 0.076 ev4 5 1/2 4 1 0.338 cr=0.064 table 10 matrix of pairwise comparison with its associated priority vector and consistency ratio cr for 𝑅13 subsidies given by government ev1 ev2 ev3 ev4 priority vector ev1 1 1/3 1/2 1/4 0.095 ev2 3 1 2 1/2 0.277 ev3 2 1/2 1 1/3 0.161 ev4 4 2 3 1 0.465 cr=0.010 table 11 matrix of pairwise comparison with its associated priority vector and consistency ratio cr for 𝑅21 unclear scope and requirements ev1 ev2 ev3 ev4 priority vector ev1 1 3 5 2 0.470 ev2 1/3 1 3 1/2 0.171 ev3 1/5 1/3 1 1/4 0.073 ev4 1/2 2 4 1 0.284 cr=0.019 ijahp article: abdou, tkiouat/an ahp application for failure risk-based ranking of electric vehicle projects international journal of the analytic hierarchy process 520 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.884 table 12 matrix of pairwise comparison with its associated priority vector and consistency ratio cr for 𝑅22 unrealistic schedule ev1 ev2 ev3 ev4 priority vector ev1 1 2 5 3 0.470 ev2 1/2 1 4 2 0.284 ev3 1/5 1/4 1 1/3 0.073 ev4 1/3 1/2 3 1 0.171 cr=0.019 table 13 matrix of pairwise comparison with its associated priority vector and consistency ratio cr for 𝑅23 poor budgetary control ev1 ev2 ev3 ev4 priority vector ev1 1 3 4 2 0.465 ev2 1/3 1 2 1/2 0.161 ev3 1/4 1/2 1 1/3 0.095 ev4 1/2 2 3 1 0.277 cr=0.010 table 14 matrix of pairwise comparison with its associated priority vector and consistency ratio cr for 𝑅31 poor performance of the battery ev1 ev2 ev3 ev4 priority vector ev1 1 3 5 6 0.546 ev2 1/3 1 3 5 0.267 ev3 1/5 1/3 1 3 0.124 ev4 1/6 1/5 1/3 1 0.061 cr=0.078 ijahp article: abdou, tkiouat/an ahp application for failure risk-based ranking of electric vehicle projects international journal of the analytic hierarchy process 521 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.884 table 15 matrix of pairwise comparison with its associated priority vector and consistency ratio cr for 𝑅32 inappropriate project development ev1 ev2 ev3 ev4 priority vector ev1 1 1/2 1/3 1/5 0.088 ev2 2 1 1/2 1/3 0.190 ev3 3 2 1 1/2 0.271 ev4 5 3 2 1 0.482 cr=0.082 table 16 matrix of pairwise comparison with its associated priority vector and consistency ratio cr for 𝑅33 lack of infrastructure ev1 ev2 ev3 ev4 priority vector ev1 1 3 5 6 0.551 ev2 1/3 1 3 4 0.255 ev3 1/5 1/3 1 3 0.127 ev4 1/6 1/4 1/3 1 0.064 cr=0.069 the ranking score 𝑆𝑝 (described in section 3) for an ev project is the summation of all values of the partial score (according to failure risk factors) for that ev project. applying the equation, the ranking score of ev1 is: s = 0.014 +0.040 +0.020 +0.005 +0.013 +0.032 +0.052 +0.002 +0.100 = 0.280 the values of the ranking score for the other projects are obtained in the same manner. based on the ranking score in table 17, the ev projects in descending order of failure risk are ev2, ev4, ev1, and ev3, respectively. according to 𝑅1 (strategic risk category), ev3 has the lowest failure risk, according to 𝑅2 (process-related risk category), ev3 either has the lowest failure risk and according to 𝑅3 (technical risk category), ev4 is the least risky. ev3 has the lowest failure risk, which means that the investor can place more investment in ev3 than the other ev projects. indeed, the ev3 project has a high chance of success because it does not require a large investment in infrastructure. also, it is beneficial for the consumer to buy an ev in common because he shares the costs with other consumers. we notice that the failure risk of ev1, ev2 and ev4 are convergent which shows the complexity of an ev project. the results also show that the ev1 project is technically difficult to set up and requires a large budget and developed infrastructure (especially the implementation of recharging stations) which is not available in underdeveloped countries. it also requires the determination of consumers to move towards the acquisition of electric vehicles. ijahp article: abdou, tkiouat/an ahp application for failure risk-based ranking of electric vehicle projects international journal of the analytic hierarchy process 522 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.884 ev3 is the least risky project among the four ev projects proposed. as for any ev project, installing the necessary infrastructure is the most important task. when we talk about infrastructure, we are mainly talking about the installation of charging stations for electric vehicles. for this, we have started a technical study of the installation of charging stations for electric vehicles looking at the case of morocco. to do this, we will first establish the different scenarios for the location of charging stations on the motorway section linking tanger med to rabat in order to choose the most optimal scenario that best meets the various requirements. the results of this study will be the subject of another article. table 17 results summarized goal criteria alternatives partial score failure risk category priority vector failure risk factor priority vector ev1 ev2 ev3 ev4 ev1 ev2 ev3 ev4 𝑹𝟏 0.581 𝑅11 0.081 0.308 0.064 0.119 0.507 0.014 0.003 0.005 0.023 𝑅12 0.575 0.112 0.472 0.076 0.338 0.040 0.160 0.025 0.112 𝑅13 0.343 0.095 0.277 0.161 0.465 0.020 0.055 0.032 0.092 summation 0.074 0.218 0.062 0.227 𝑹𝟐 0.109 𝑅21 0.106 0.470 0.171 0.073 0.284 0.005 0.001 0.001 0.003 𝑅22 0.260 0.470 0.284 0.073 0.171 0.013 0.010 0.002 0.004 𝑅123 0.633 0.465 0.161 0.095 0.277 0.032 0.011 0.006 0.020 summation 0.05 0.022 0.009 0.027 𝑹𝟑 0.309 𝑅31 0.309 0.546 0.267 0.124 0.061 0.052 0.025 0.011 0.005 𝑅32 0.109 0.088 0.190 0.271 0.482 0.002 0.006 0.010 0.016 𝑅33 0.581 0.551 0.255 0.127 0.064 0.100 0.045 0.022 0.011 summation 0.154 0.076 0.043 0.032 ranking score 0.280 0.316 0.114 0.290 5. conclusion ev projects have high rates of failure which lead to devastating economic consequences. a framework for ev project ranking based on failure risk is described. the analytic hierarchy process (ahp) is employed as a ranking method. a hierarchy structure which consists of levels including ev project ranking, failure risk categories, failure risk factors, and candidates of ev projects is defined. these levels correspond to the goal, criteria, sub-criteria, and alternatives in the standard ahp structure, respectively. by specifying the failure risk categories and failure risk factors, the ranking of ev project failure risks and thus of ev projects is accomplished via the pairwise comparison in the ahp. the proposed framework realizes the ranking of failure risks even if the risks are characterized in terms of multiple qualitative attributes. the purpose of this study was to select the ev project with a low failure risk taking into account strategic, technical and operational ijahp article: abdou, tkiouat/an ahp application for failure risk-based ranking of electric vehicle projects international journal of the analytic hierarchy process 523 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.884 considerations. this is a reference for all who stakeholders intend to invest in this field. ijahp article: abdou, tkiouat/an ahp application for failure risk-based ranking of electric vehicle projects international journal of the analytic hierarchy process 524 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.884 references beeton, d., & meyer, g. (2014). electric vehicle business models. heidelberg: springer. biresselioglu, m. e., kaplan, m. d., & yilmaz, b. k. (2018). electric mobility in europe: a comprehensive review of motivators and barriers in decision making processes. transportation research part a: policy and practice, 109, 1-13. doi: https://doi.org/10.1016/j.tra.2018.01.017 biswas, t., chatterjee, p., & choudhuri, b. (2020). selection of commercially available alternative passenger vehicle in automotive environment. operational research in engineering sciences: theory and applications, 3(1), 16-27. doi: https://oresta.rabek.org/index.php/oresta/article/view/39/33 biswas, t. k., & das, m. c. (2020). selection of the barriers of supply chain management in indian manufacturing sectors due to covid-19 impacts. operational research in engineering sciences: theory and applications, 3(3), 1-12. doi: https://oresta.rabek.org/index.php/oresta/article/view/62/46 bresser, d., hosoi, k., howell, d., li, h., zeisel, h., amine, k., & passerini, s. (2018). perspectives of automotive battery r&d in china, germany, japan, and the usa. journal of power sources, 382, 176-178. doi: http://dx.doi.org/10.1016/j.jpowsour.2018.02.039 chua, a. y. k. (2009). exhuming it projects from their graves: an analysis of eight failure cases and their risk factors. journal of computer information systems, 49(3), 31-39. corno, f., montanaro, t., migliore, c., & castrogiovanni, p. (2017). smartbike: an iot crowd sensing platform for monitoring city air pollution. international journal of electrical and computer engineering, 7(6), 3602. doi: https://doi.org/10.11591/ijece.v7i6.pp3602-3612 dijk, m., orsato, r. j., & kemp, r. (2013). the emergence of an electric mobility trajectory. energy policy, 52, 135-145. doi: http://dx.doi.org/10.1016/j.enpol.2012.04.024 hanine, y., tkiouat, m., & lahrichi, y. (2021). an alternate framework for socially responsible portfolios optimization applied to the moroccan stock exchange. international journal of the analytic hierarchy process, 13(1), 107-146. doi: https://doi.org/10.13033/ijahp.v13i1.831 hülsmann, m. (2016). markets and policy measures in the evolution of electric mobility. d. fornahl (ed.). springer. doi: http://dx.doi.org/10.1007/978-3-31924229-3 leal filho, w., & kotter, r. (eds.). (2015). e-mobility in europe: trends and good practice. springer. doi: https://doi.org/10.1007/978-3-319-13194-8 leal filho, w., rath, k., mannke, f., vogt, j., kotter, r., borgqvist, m., ... & van deventer, p. (2015). fostering sustainable mobility in europe: the contributions of ijahp article: abdou, tkiouat/an ahp application for failure risk-based ranking of electric vehicle projects international journal of the analytic hierarchy process 525 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.884 the project “e-mobility north sea region”. in e-mobility in europe (pp. 3-25). springer, cham. doi: https://doi.org/10.1007/978-3-319-13194-8 leurent, f., & windisch, e. (2011). triggering the development of electric mobility: a review of public policies. european transport research review, 3(4), 221-235. doi: https://doi.org/10.1007/s12544-011-0064-3 li, j., & zou, p. x. (2011). fuzzy ahp-based risk assessment methodology for ppp projects. journal of construction engineering and management, 137(12), 1205-1209. doi: https://doi.org/10.1061/(asce)co.1943-7862.0000362 liebl, j. (2017). grid integration of electric mobility. springer fachmedien wiesbaden, wiesbaden. doi: https://doi.org/10.1007/978-3-658-15443-1 mcfarlan, f. (1981). portfolio approach to information systems. harvard business review, 59(5), 142-150. müller, b., & meyer, g. (eds.). (2015). electric vehicle systems architecture and standardization needs: reports of the ppp european green vehicles initiative. springer. doi: https://doi.org/10.1007/978-3-319-13656-1 nikowitz, m. (2016). advanced hybrid and electric vehicles. system optimization and vehicle integration. springer. saaty, r. (1996, july). the analytic hierarchy process and utility theory: ratio scales and interval scales. in proceedings of the fourth international symposium on the analytic hierarchy process (pp. 12-15). saaty, r. w. (2003). decision making in complex environment: the analytic hierarchy process (ahp) for decision making and the analytic network process (anp) for decision making with dependence and feedback. pittsburgh: super decisions. saaty, t. l. (1980). multicriteria decision making: the analytic hierarchy process. 2nd ed. new york: mcgraw hill. sárdi, d. l., & bóna, k. (2021, may). ahp-based multicriterial ranking model for the city logistics analysis of urban areas. in liss 2020: proceedings of the 10th international conference on logistics, informatics and service sciences (p. 45). springer nature. doi: 10.1007/978-981-33-4359-7_4 tagscherer, u. (2012). electric mobility in china: a policy review (no. 30). fraunhofer isi discussion papers innovation systems and policy analysis. taylan, o., bafail, a. o., abdulaal, r. m., & kabli, m. r. (2014). construction projects selection and risk assessment by fuzzy ahp and fuzzy topsis methodologies. applied soft computing, 17, 105-116. doi: http://dx.doi.org/10.1016/j.asoc.2014.01.003 zheng, x., lin, h., liu, z., li, d., llopis-albert, c., & zeng, s. (2018). manufacturing decisions and government subsidies for electric vehicles in china: a maximal social welfare perspective. sustainability, 10(3), 672. doi: https://doi.org/10.3390/su10030672 ijahp article: marcarelli, mancini/school and academic performance for ranking high schools: some evidence from italy international journal of the analytic hierarchy process 1 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.948 school and academic performance for ranking high schools: some evidence from italy gabriella marcarelli university of sannio gabriella.marcarelli@gmail.com paola mancini university of sannio paola.mancini@unisannio.it abstract this paper aims to investigate and compare the performance of italian public high schools (hs) to provide a ranking among different typologies of hs. in this paper, seven criteria that refer to students’ school and academic performance were considered. the sample includes 263 high schools in all italian regions, grouped into 6 different types of schools and 3 geographic areas. assuming that all criteria have the same weights, the analytic hierarchy process (ahp) was applied to derive the ranking among the typologies of schools both at a national level and within each geographic area. the main results show that there are significant differences between hs according to criteria related to school and academic performance both within and between geographic areas. the ranking does not vary, but the intensity of preferences may be different according to the area and/or the criterion considered. the application of promethee to the same problem confirms the results obtained by the ahp. keywords: school ranking; school performance; academic performance; ahp; promethee 1. introduction the literature on school ranking is vast. some scholars have mainly focused on school quality and student achievements (eide & showalter, 1998; camanho et al., 2021), others have analyzed the school’s contribution to student academic performance discussing both the so-called “school value-added” (jamelske, 2009; kelly & downey, 2010; schiltz et al., 2018) and the question of “school accountability” affecting the school choice (burgess et al., 2013; hart & figlio, 2015; nunes et al., 2015). the analysis of performance assessments of the upper secondary education system requires the measurement of student achievements at both the school and academic levels. students’ performance may be influenced by many factors including students’ socio-economic status, family background, geographic area of residence, the type of school attended (agasisti & murtinu, 2012; lauer, 2003), school and class size, students’ features, and school management and resources (giambona & porcu, 2018; masci et al., 2018). more recently, aina et al. (2011,2021) demonstrate that differences in university students’ achievements across high schools cannot be limited to the first year at university and have to consider the geographic differences. mailto:gabriella.marcarelli@gmail.com ijahp article: marcarelli, mancini/school and academic performance for ranking high schools: some evidence from italy international journal of the analytic hierarchy process 2 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.948 to evaluate high schools, an adequate data set and appropriate methods are required. eduscopio (giovanni agnelli foundation https://eduscopio.it/) and scuolainchiaro (ministry of education www.miur.gov.it/-/scuola-in-chiaro) data are considered in this paper. the eduscopio project started in 2014 to provide students and their families with a ranking of high schools in the area of residence based on the fga (giovanni agnelli foundation) index 1 . the scuolainchiaro project started in 2011 in response to the need to make all information related to italian schools of all levels available to the community in an organic and structured form. these two portals provide data annually on the students' school career and their academic achievements for each italian school. to evaluate the performance of schools by considering the school and academic careers of students, a multi-criteria approach, which is a useful tool was used. multiple criteria decision analysis (mcda) helps decision-makers face problems characterized by multiple conflicting criteria. a large number of methods have been developed to solve multi-criteria problems. roy (1991) listed different kinds of decision problems such as choice problems, sorting problems, ranking problems and description problems. a comprehensive literature review of common multi-criteria decision making (mcdm) methods is provided by figueira et al. (2005), ishizaka and nemery (2013), and velasquez and hester (2013). the use of combined mcdm methods has led to a new approach to decision analysis (velasquez & hester, 2013). the software available, such as web and smartphone applications, r routines, or ad hoc packages, has made mcdm methods more accessible, increasing their use amongst researchers and the user community (ishizaka & nemery 2013). each mcdm method has both advantages and disadvantages. the selection of the appropriate method depends on the features of the problem, the nature of the criteria, and data availability. when facing a decision problem, it is also important to define the type of output required and the input data. this paper aims to provide the ranking of different types of schools, taking into account the geographic area. the schools are evaluated considering quantitative criteria represented by performance indicators expressed in different units of measurement. many studies have dealt with the application of multi-criteria methods in the field of education (stamenkovic et al., 2016; giannoulis & ishizaka 2010; mancini & marcarelli 2019; goztepe, 2020). by taking into account some performance indicators used by eduscopio and scuolainchiaro and according to the approach proposed by mancini and marcarelli (2019), this study analyzes some criteria related to students’ school and academic achievements. due to the characteristics of the problem (e.g., independence among the elements) and the output required, this work focuses on the analytic hierarchy process (ahp) from among multi-criteria methods proposed in the literature. the available data allows us to avoid some disadvantages of the ahp. there is no inconsistency in the judgment matrices because entries of matrices are ratios between performance indices. after 1 the fga index is an indicator that accounts for the percentage of academic credits at the end of the first year and the average exam score in order to measure the students’ “speed” and “profit” in academic education (https://eduscopio.it/dati-e-metodologia). for further details, see bordignon et al. (2017). ijahp article: marcarelli, mancini/school and academic performance for ranking high schools: some evidence from italy international journal of the analytic hierarchy process 3 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.948 obtaining a complete ranking with scores, a sensitivity analysis is conducted on the weights of criteria. furthermore, in order to compare the results obtained by a full aggregation approach with those derived by an outranking approach, the preference ranking organization method for enrichment of evaluations (promethee) was applied to verify the robustness of the solution. the main contributions of this paper are as follows: application of the ahp to derive the ranking among different types of italian high schools at a national level; highlighting any significant differences between hs within each geographic area; application of the promethee method to verify the robustness of the solution. the remainder of this article is structured as follows. section 2 defines the elements of the problem and makes a descriptive analysis of the data; section 3 introduces the methods applied in our study; section 4 provides the ranking of high schools at a national level and by geographical area and includes the sensitivity analysis; in section 5, a comparative analysis is carried out. in the last section, some conclusions are provided. 2. data data are from the eduscopio and scuolainchiaro portals and refer to the 2019/2020 school year. in addition to the fga index, eduscopio provides information on students' school and academic performance annually. based on the above data, the following performance indicators were considered: hsls = high school leaving score (calculated as the weighted average between the high school leaving score of academic enrolled and not enrolled students), reggrad = proportion of regular graduates, aes = proportion of academic enrolled students, aespfy = percentage of academic enrolled students passing the first year (calculated as the proportion of academic enrolled students), acfy = percentage of academic credits at the end of the first year, eas = exam average score. moreover, the study considers the invalsi test score (computed as the average between individual students' math, reading, and foreign language test scores) provided by scuolainchiaro. the reference universe is made up of 761 public high schools located in the regional capitals. a sample of 263 public hs, stratified by region and type of school, was selected. the schools, ranked by the eduscopio fga index in each capital region, were selected ensuring the representativeness of the various positions in the ranking 2 . following aina et al. (2021), six types of schools were taken into account as follows: 60 scientific lyceums (sl), 40 classic lyceums (cl), 41 linguistic lyceums (ll), 27 human sciences lyceums (hsl), 44 commercial technical high schools (cths), and 51 technological technical high schools (tths). 2 for example, in the case of rome, 13 sl are selected. we consider the 4 best schools, the 5 schools in the middle of the ranking and the 4 worst schools, according to the fga index. ijahp article: marcarelli, mancini/school and academic performance for ranking high schools: some evidence from italy international journal of the analytic hierarchy process 4 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.948 table 1 shows the performance matrix showing the average 3 scores computed for each type of school for the varying criteria. the descriptive analysis shows that, as concerns students’ achievements, cl and sl have the best performance, and technical hs have the worst performance, whereas the performance of ll and hsl are not clearly defined and may vary depending on the criterion. cl are better than sl with regard to most of the criteria related to school career (e.g., hsls, reggrad, and invalsi) and aes; the opposite holds for acfy; they scored similarly as concerns aes and aespfy. data related to 263 hs were processed by hierarchical cluster analysis 4 and organized into groups based on how closely associated they are. this method aims to capture similarities and/or differences among italian high schools and create homogeneous groups according to the criteria taken into account. the main results highlight 6 clusters almost matching the six types of schools (table 2). the first group contains 50.0% of sl and 38.8% of cl; the second group includes about 40% of hsl and 35% of ll; cths and tths schools are located mainly in the fourth, fifth and sixth clusters; the third cluster consists of a residual group of schools. table 1 average scores related to 6 school types and 7 criteria in italy average scores school performance academic performance school type hsls invalsi reggrad aes aespfy acfy eas sl 78.32 4.49 62.62 90.63 90.84 73.24 26.62 cl 80.80 4.67 72.08 91.42 91.30 69.76 27.07 ll 78.41 4.21 59.93 74.20 87.38 58.28 25.32 hsl 76.11 3.98 58.80 69.09 84.09 49.40 24.17 cths 74.37 3.78 45.79 47.88 80.39 46.64 23.55 tths 74.11 4.12 40.98 45.54 76.51 50.17 24.61 3 the average values have been obtained as weighted arithmetic averages according to the number of students in each school. 4 the objective of this method is to find similar groups of subjects, where “similarity” between each pair of subjects means some global measure over the whole set of characteristics. the ward's method was the criterion applied in this hierarchical procedure. ijahp article: marcarelli, mancini/school and academic performance for ranking high schools: some evidence from italy international journal of the analytic hierarchy process 5 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.948 table 2 high schools grouped using cluster analysis school type cluster 1 2 3 4 5 6 sl 50.6 3.1 35.7 0.0 0.0 0.0 cl 39.2 18.8 2.4 0.0 0.0 0.0 ll 10.1 34.4 31.0 12.0 4.5 0.0 hsl 0.0 40.6 14.3 6.0 18.2 0.0 cths 0.0 3.1 4.8 38.0 50.0 38.5 tths 0.0 0.0 11.9 44.0 27.3 61.5 tot (a.v.) 79 32 42 50 22 26 the best performing high schools are in the first cluster (table 3). students’ high school leaving score is almost 80 and the invalsi test score is 4.85, 90% of their graduates enroll in university and, of these, more than 9 out of 10 pass the first year exceeding at least 71% of the academic credits and having an average exam score of 26.9. on the contrary, less performing schools are in the fourth and sixth clusters. for these, the high school leaving score is lower than 74, only 45% of graduates enroll in university; particularly in the fourth group, only 70% of enrolled students pass the first year having obtained about 38% of the academic credits and an average exam score below 24. table 3 average values of 6 clusters according to 7 criteria criteria cluster 1 2 3 4 5 6 hsls 79.75 77.16 75.82 73.95 75.34 73.37 invalsi 4.85 4.16 3.67 3.96 3.91 3.69 reggrad 67.79 66.12 46.46 42.53 56.49 31.37 aes 90.24 76.69 77.12 44.76 52.63 44.15 aespfy 91.54 83.59 85.79 70.45 84.93 81.10 acfy 71.73 49.51 56.42 37.78 55.34 52.49 eas 26.88 24.57 24.84 23.61 24.26 24.20 ijahp article: marcarelli, mancini/school and academic performance for ranking high schools: some evidence from italy international journal of the analytic hierarchy process 6 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.948 3. methodology 3.1. ahp the ahp is a multi-criteria method developed by saaty in the 1970s in order to prioritize different and potentially conflicting objectives and give a ranking of alternatives through pairwise comparisons (saaty & vargas, 1982; aczel & saaty, 1989). the multi-criteria approach has found many fields of application in both individual and collective decision contexts (triantaphyllou & mann, 1995). the ahp makes it possible to analyze a complex problem by combining both qualitative and quantitative aspects into a single framework and generating a set of priorities for alternatives. a detailed explanation of the ahp is found in the study by vargas (1990). this method involves four phases described in table 4. table 4 steps of the ahp step description 1 decomposition of the decision problem into a multi-level hierarchy 2 data collection by means of pairwise comparisons 3 determination of the relative weights, reflecting the relative importance of the elements belonging to each hierarchical level considered with respect to the elements of the immediately higher level 4 aggregation of relative weights to obtain the overall priorities, expressing the importance of alternatives with respect to the overall objective of the evaluation (by applying the principle of hierarchical composition). this priority vector provides the ranking of alternatives saaty uses a pairwise comparison matrix (pcm) to derive the relative weights, which reflect the relative importance of the elements belonging to each hierarchical level with respect to the elements of the level immediately higher. to each pair of elements (xi, xj) of a fixed level, a positive number (aij) is assigned expressing how much xi is preferred to xj as regards to a given criterion; by comparing all n elements of a level, a positive square matrix of the order n is obtained. the value aij > 1 implies that xi is strictly preferred to xj, whereas aij < 1 expresses the opposite preference and aij = 1 means that xi and xj are indifferent. the matrix is obviously reciprocal (aji=1/aij for i, j =1, 2, …, n). before deriving priority vectors, a check of logical consistency in the allocation of judgments is required. the inconsistency may be caused by errors, inaccuracies, or simply by the violation of transitivity or proportionality. to verify the judgments expressed by the dm, several measures of consistency have been proposed in the literature. saaty suggested the consistency index (ci): and the consistency ratio (cr): ijahp article: marcarelli, mancini/school and academic performance for ranking high schools: some evidence from italy international journal of the analytic hierarchy process 7 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.948 where ri (random index) is the average of ci values associated with several pcm (of size n) randomly generated (saaty & vargas, 1982). if cr is less than 0.1, then the matrix may be considered to have acceptable consistency; otherwise, the judgments must to be revised. when a pcm is perfectly consistent, its entries are given by the ratios between pairs of the components of the priority vector (aij = wi/wj): the matrix has unit rank and all its eigenvalues except one are zero; since the sum of the eigenvalues of a matrix is equal to its trace, then n is the only non-zero eigenvalue of a (as a consequence, ci=0 and cr=0). in order to calculate the priority vector associated to each pcm, saaty proposed adopting the eigenvector method (em), that is, the eigenvector associated to the maximum eigenvalue of a. this vector represents the relative weight of the elements of a level with respect to an element of the level immediately higher. finally, by applying the principle of hierarchical composition, relative weights are aggregated to obtain the overall priorities expressing the importance of alternatives with respect to the overall objective of the evaluation. this priority vector provides the ranking of alternatives. saaty (1994) developed a software package called expert choice for use with the ahp that allows priority vectors to be calculated, inconsistencies of matrices to be identified and a sensitivity analysis to be performed. 3.2. promethee promethee is an outranking method introduced by vincke (1992). it uses preference degrees to derive the ranking of alternatives. for each criterion, the decision-maker assigns a preference degree to pairs of alternatives. the preference degree expresses how an alternative is preferred to another one. if the value is 1, then there is a strong preference for an alternative on a given criterion; 0 means that there is no preference; and values between 0 and 1 express that there is some preference. the pairwise preference degrees related to a given criterion are synthesized in the positive, negative, and net flows related to that criterion. the positive flow indicates how an alternative “a” is preferred to all other alternatives, with regard to a criterion, and the negative flow indicates how the other alternatives are preferred to “a”. the positive and negative flows are scores between 0 and 1; net flows, a combination of the above two flows, are valued between -1 and 1. the final ranking of the alternatives is given by the global net flow, obtained as a difference between the global positive and negative flows when considering all criteria and their weights. promethee makes it possible to represent the decision problem in the gaia plane as a two-dimensional plot containing the alternatives, criteria, and information on the preferences of the decision-maker (thresholds and weights). ijahp article: marcarelli, mancini/school and academic performance for ranking high schools: some evidence from italy international journal of the analytic hierarchy process 8 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.948 4. ranking high schools by typology a decision problem consists of a finite set of alternatives (a), a set of criteria with respect to the alternatives are compared, and a weak order relation on a. a multi-criteria problem can be represented by a decision matrix, where each entry represents the value of the i th alternative regarding the j th criterion and each criterion has a weight (triantaphyllou & mann, 1995). given this matrix, several methods exist for aggregating judgments related to each alternative into a synthetic value in order to select the best one. in the first step of our analysis, the ahp is applied to obtain a ranking among the different types of schools taking into account the geographical area. in the second step, the promethee procedure is performed on the same problem in order to compare the results. figure 1 shows the elements of our problem arranged in a hierarchical structure consisting of 3 levels as follows: the bottom level contains the alternatives, that is, the school typologies; the second level contains the criteria, that is, the performance indicators with respect to which the alternatives are evaluated; and the top level is represented by the goal, the ranking among the 6 types of high schools. figure 1 elements of the problem data have been processed using expert choice and r routines. promethee was used from among the different packages containing functions for solving multiple-criteria decision-making problems. using the data in table 1, seven pairwise comparison matrices (pcm) were constructed to evaluate pairs of school typologies with respect to each criterion. all of the pcm are perfectly consistent because their entries are given by the ratios between pairs of performance values (as a consequence, ci and cr are null). therefore, the consistency issue may be disregarded in our study. the weights of the criteria are not derived by pairwise comparison of the criteria with regard to the goal, but they are a priori assigned; in particular, the criteria all have the same weight. next, in order to determine if any geographical differences emerge, the six types of schools were analyzed by area. table 5 contains the performance matrices which show ijahp article: marcarelli, mancini/school and academic performance for ranking high schools: some evidence from italy international journal of the analytic hierarchy process 9 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.948 the average scores computed with regard to the six types of schools for each geographic area (north, center, and south of italy) on the varying criteria. table 5 average scores related to 6 school types and 7 criteria by geographical area area school type school performance academic performance hsls invalsi reggrad aes aespfy acfy eas n o r t h sl 77.44 4.83 60.82 91.87 90.84 73.24 26.62 cl 78.57 5.02 68.05 91.87 91.30 69.76 27.07 ll 78.26 3.97 63.05 75.46 87.38 58.28 25.32 hsl 74.34 3.50 52.01 71.01 84.09 49.40 24.17 cths 73.95 3.64 46.34 47.44 80.39 46.64 23.55 tths 73.78 4.40 41.07 47.10 76.51 50.17 24.61 c e n t e r sl 77.99 4.53 62.44 90.51 91.85 76.99 27.11 cl 81.44 4.31 69.31 91.32 91.98 71.81 27.61 ll 78.14 4.39 56.97 72.76 88.91 60.76 25.39 hsl 76.16 4.23 59.91 71.55 84.55 51.06 24.56 cths 74.32 3.84 47.87 48.58 80.23 45.93 23.57 tths 73.29 3.96 43.05 48.34 79.83 53.68 24.98 s o u t h sl 79.41 4.07 64.45 89.61 90.56 72.81 26.77 cl 81.97 4.80 77.38 91.15 90.87 69.41 27.31 ll 78.81 4.16 60.59 74.74 86.07 56.37 25.35 hsl 76.85 4.06 61.01 66.34 85.33 50.14 24.15 cths 74.79 3.91 44.33 47.98 79.70 48.46 24.16 tths 75.10 3.89 39.58 41.49 73.10 45.79 24.45 the results obtained by applying the ahp (figure 2) show that 3 groups of schools may be identified nationwide. cl and sl have the best performance, ll and hsl occupy the intermediate level, and technical hs have the worst performance. it is interesting to note that the ranking does not vary moving from north to south, but the intensity of preferences may be different according to the area and/or the criterion considered (relative priority vectors are listed in the appendix). for example, cl exceeds sl for ijahp article: marcarelli, mancini/school and academic performance for ranking high schools: some evidence from italy international journal of the analytic hierarchy process 10 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.948 most of criteria related to school performance, particularly in southern italy, whereas sl overcomes cl regarding acfy, especially in the north. figures 3 and 4 show the relative priorities of the alternatives with respect to each criterion. in each geographic area, cl is the best alternative for all criteria except for acfy. the sensitivity analyses showed that the global ranking is a stable solution since varying the weights of the criteria did not produce any significant changes. as figure 5 highlights, minor variations in the ranking of the alternatives occur in particular when the weight of the criterion acfy is greatly increased (for which sl is the best alternative). figure 2 ranking of school typologies for italy by geographical area figure 3 performance sensitivity plot italy ijahp article: marcarelli, mancini/school and academic performance for ranking high schools: some evidence from italy international journal of the analytic hierarchy process 11 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.948 figure 4a performance sensitivity plot north figure 4b performance sensitivity plot – center figure 4c performance sensitivity plot south ijahp article: marcarelli, mancini/school and academic performance for ranking high schools: some evidence from italy international journal of the analytic hierarchy process 12 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.948 figure 5 gradient sensitivity for acfy italy 5. comparative analysis in the second step of the study, the promethee procedure was applied to the performance matrices in tables 1 and 5 in order to compare the results with the solution provided by ahp. the preference and indifference thresholds (table 6) are identified based on a one way analysis of variance test 5 on the differences among 6 means for each criterion. table 6 preference and indifference thresholds scores school performance academic performance hsls invalsi reggrad aes aespfy acfy eas max-min max max max max max max max thresholds indif. (q) 1.74 0.70 1.13 5.11 3.87 3.54 0.71 pref. (p) 2.21 0.89 9.47 16.43 6.75 8.89 1.06 weights wj 0.10 0.15 0.10 0.10 0.15 0.15 0.15 the promethee method provides the final ranking of the alternatives given by the global net flow, which is a result of the difference between the global positive flow and 5 anova is used to test for differences among two or more independent groups, providing a statistical test to verify if two or more means are equal or not, and therefore generalizes the t-test beyond two means. ijahp article: marcarelli, mancini/school and academic performance for ranking high schools: some evidence from italy international journal of the analytic hierarchy process 13 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.948 the negative flow. figure 6 shows that positive and negative flows provide the same ranking at the national level. cl has the best rank because its global positive and negative flows are simultaneously better than the others; on the contrary, cths has the worst rank. there are no incomparable school types because the global positive flow and the negative flow provide the same ranking. table 7 and figure 7 show the rankings of the six types of schools both in italy and in 3 geographic areas: north, center, and south. a comparison of the rankings among areas shows the following: each area confirms the ranking results at the national level: cl ranks at the top, sl has the second rank, ll composes the third group, hsl the fourth, and technical schools (cths and tths) are the last group; with regard to the last positions (tths has the worst rank), there is a rank reversal in the center and the south. the rankings obtained by the ahp and promethee models are similar, so the stability of the solution is verified. table 7 net flows and ranking of school types at a national level and by geographic area alternatives net flows italy ri north rn center rc south rs sl 0.07521521 2 0.06766490 2 0.08951429 2 0.07315623 2 cl 0.10569414 1 0.08274755 1 0.09345714 1 0.11788571 1 ll 0.02295296 3 0.02202566 3 0.01081549 3 0.03469699 3 hsl -0.03566422 4 -0.03282828 4 -0.01501180 4 -0.03198341 4 cths -0.08756714 6 -0.07544853 6 -0.08723914 5 -0.08919477 5 tths -0.08063096 5 -0.06416129 5 -0.09153598 6 -0.10456076 6 figure 6 positive and negative flows for the types of schools in italy ijahp article: marcarelli, mancini/school and academic performance for ranking high schools: some evidence from italy international journal of the analytic hierarchy process 14 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.948 figure 7 positive and negative flows for the types of schools by geographic area the gaia plane summarizes the results of the pca applied to the uni-criterion net flows matrix. the criteria are represented by vectors (figure 8a) and alternatives by points (figure 8b). the length of each vector is a measure of its strength in the alternatives’ differentiation. the horizontal axis distinguishes the alternatives by typology of school. the best schools (lyceums) are on the right, and the worst school types (technical schools) are on the left. in other words, moving from left to right, in the plane, school and academic performance improves. ijahp article: marcarelli, mancini/school and academic performance for ranking high schools: some evidence from italy international journal of the analytic hierarchy process 15 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.948 figure 8a pca graph of variables figure 8b pca graph of individuals 6. conclusions this paper investigates and compares the performance of italian high schools in order to derive a ranking considering the typology and the geographic area. the results show that the ranking among the types of schools does not vary moving from north to south (classic lyceums are the best, technical schools are the worst). the model may assist students and their parents in selecting the type of school to attend; the information makes it possible to make an appropriate choice according to their academic perspectives. high school types are evaluated using criteria that provide a measure of performance under specific (school and academic) features. when applied to small territorial districts, this model may be a useful tool to help public administrations distribute additional financial resources to public schools based on their performance rank. our future works will address the classification of high schools into different categories such as ‘over-performing schools’, ‘average-performing schools’ and ‘weak-performing schools’ by using electre tri (corrente et al., 2016). furthermore, it could be ijahp article: marcarelli, mancini/school and academic performance for ranking high schools: some evidence from italy international journal of the analytic hierarchy process 16 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.948 interesting to verify the ranking of hs when varying the key parameters (e.g., preference and indifference thresholds). ijahp article: marcarelli, mancini/school and academic performance for ranking high schools: some evidence from italy international journal of the analytic hierarchy process 17 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.948 appendix relative priority vectors of school typologies with respect to each criterion alternatives hsls invalsi reggrad aes aespfy acfy eas italy sl 0,169 0,178 0,185 0,216 0,178 0,211 0,176 cl 0,175 0,185 0,212 0,218 0,180 0,201 0,179 ll 0,170 0,167 0,176 0,177 0,171 0,168 0,168 hsl 0,165 0,157 0,173 0,165 0,165 0,142 0,160 cths 0,161 0,150 0,135 0,114 0,157 0,134 0,155 tths 0,160 0,163 0,120 0,109 0,150 0,144 0,163 north sl 0,169 0,191 0,183 0,216 0,178 0,214 0,177 cl 0,172 0,198 0,206 0,216 0,178 0,199 0,180 ll 0,172 0,156 0,190 0,177 0,172 0,169 0,166 hsl 0,163 0,138 0,157 0,168 0,163 0,142 0,160 cths 0,162 0,143 0,140 0,112 0,155 0,128 0,154 tths 0,162 0,174 0,124 0,111 0,154 0,149 0,163 center sl 0,169 0,191 0,183 0,216 0,214 0,211 0,177 cl 0,172 0,198 0,206 0,216 0,199 0,201 0,180 ll 0,172 0,157 0,190 0,177 0,169 0,168 0,166 hsl 0,163 0,138 0,157 0,168 0,142 0,142 0,160 cths 0,162 0,143 0,140 0,112 0,128 0,134 0,154 tths 0,162 0,174 0,124 0,111 0,149 0,144 0,163 south sl 0,170 0,164 0,186 0,218 0,179 0,207 0,175 cl 0,176 0,193 0,223 0,222 0,180 0,203 0,178 ll 0,169 0,167 0,175 0,182 0,173 0,172 0,169 hsl 0,164 0,163 0,175 0,161 0,164 0,142 0,161 cths 0,160 0,157 0,128 0,117 0,160 0,137 0,156 tths 0,161 0,156 0,114 0,101 0,145 0,140 0,162 ijahp article: marcarelli, mancini/school and academic performance for ranking high schools: some evidence from italy international journal of the analytic hierarchy process 18 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.948 references aczel, j., & saaty, t.l. (1989). procedures for synthesizing ratio judgements. journal of mathematical psychology, 27(1), 93–102. doi: https://doi.org/10.1016/00222496(83)90028-7 aina, c., baici, e., & casalone, g. (2011). time to degree: students’ abilities, university characteristics or something else? evidence from italy. education economics, 19(3), 311–325. doi: https://doi.org/10.1080/09645292.2011.585016 aina, c., bratti, m., & lippo, e. (2021). ranking high schools using university student performance in italy. economia politica, 38, 293–321. doi: https://doi.org/10.1007/s40888-020-00208-2 agasisti, t., murtinu, s. (2012). perceived competition and performance in italian secondary school: new evidence from oecd-pisa 2006. british educational research journal, 38(5), 841–858. doi: https://doi.org/10.1080/01411926.2011.588314 bordignon m., carapella p., & turati g. (2017). eduscopio: le determinanti dell'accesso all'informazione, programma education fga working paper, n.56, fondazione giovanni agnelli. burgess, s., wilson, d., & worth, j. (2013). a natural experiment in school accountability: the impact of school performance information on pupil progress, journal of public economics, 106(c), 57-67. doi: https://doi.org/10.1016/j.jpubeco.2013.06.005 camanho, a.s., varriale, l., barbosa, f., & sobral, t. (2021). performance assessment of upper secondary schools in italian regions using a circular pseudo-malmquist index, european journal of operational research, 289(3), 1188-1208. doi: https://doi.org/10.1016/j.ejor.2020.07.050 corrente, s., greco, s. & slowinski, r. (2016). multiple criteria hierarchy process for electre tri methods. european journal of operational research, 252, 191-203. doi: https://doi.org/10.1016/j.ejor.2015.12.053 eide, e., and showalter, m.h. (1998). the effect of school quality on student performance: a quantile regression approach. economics letters, 58, 345–350. doi: https://doi.org/10.1016/s0165-1765(97)00286-3 giambona, f., & porcu, m. (2018). school size and students ’ achievement: empirical evidences from pisa survey data. socio-economic planning sciences, 64(c), 66-77. doi: https://doi.org/10.1016/j.seps.2017.12.007 giannoulis, c., & ishizaka, a. (2010). a web-based decision support system with electre iii for a personalised ranking of british universities. decision support systems, 48(3), 488–497. doi: https://doi.org/10.1016/j.dss.2009.06.008 ijahp article: marcarelli, mancini/school and academic performance for ranking high schools: some evidence from italy international journal of the analytic hierarchy process 19 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.948 goztepe, k. (2020). applying choquet integral approach for ranking high school innovative education. international journal of electrical communication engineering, 6(1), 27-40. hart, c.m.d., & figlio, d.n. (2015). school accountability and school choice: effect on student selection across schools. national tax journal, 68(3s), 875–900. doi: https://doi.org/10.17310/ntj.2015.3s.07 ishizaka, a, & nemery, p. (2013). multi-criteria decision analysis methods and software. new york: wiley. jamelske, e. (2009). measuring the impact of a university first-year experience program on student gpa and retention. higher education, 57(3), 373–391. doi: https://doi.org/10.1007/s10734-008-9161-1 kelly, a., & downey, c. (2010). value-added measures for schools in england: looking inside the ‘black box’ of complex metrics. educational assessment, evaluation and accountability, 22(3), 181–198. doi: https://doi.org/10.1007/s11092-010-9100-4 lauer, c. (2003). family background, cohort and education: a french-german comparison based on a multivariate ordered probit model of educational attainment. labour economics, 10, 231–251. doi: https://doi.org/10.1016/s0927-5371(03)00007-1 mancini, p., & marcarelli, g. (2019). high school choice: how do parents make a choice. international journal of the analytic hierarchy process, 11(1) 91-109. doi: https://doi.org/10.13033/ijahp.v11i1.633 masci, c., witte, k. d., & agasisti, t. (2018). the influence of school size, principal characteristics and school management practices on educational performance: an efficiency analysis of italian students attending middle schools. socio-economic planning sciences, 61, 52–69. doi: https://doi.org/10.1016/j.seps.2016.09.009 nunes, l., reis, a., & seabra, c. (2015). the publication of school rankings: a step toward increased accountability? economics of education review, 49(c), 15-23. doi: https://doi.org/10.1016/j.econedurev.2015.07.008 roy, b. (1991). the outranking approach and the foundations of electre methods. theory and decision, 31, 49-73. doi: https://doi.org/10.1007/bf00134132 saaty, t.l. (1994). how to make a decision: the analytic hierarchy process, interfaces, 24, 41-42. doi: https://doi.org/10.1287/inte.24.6.19 saaty, t.l. & vargas, l.g. (1982). the logic of priorities, applications in business, energy, health, and transportation. boston: kluwer-nijhoff. reprinted in paperback (1991), pittsburgh: rws publications. schiltz, f, sestito, p, agasisit, t., & de vitte, k. (2018). the added value of more accurate predictions for school rankings. economics of education review, 67, 207-215. doi: https://doi.org/10.1016/j.econedurev.2018.10.011 ijahp article: marcarelli, mancini/school and academic performance for ranking high schools: some evidence from italy international journal of the analytic hierarchy process 20 vol. 14 issue 2 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i2.948 stamenkovic, m., anic, i., petrovic, m., & bojkovic, n. (2016). an electre approach for evaluating secondary education profiles: evidence from pisa survey in serbia. annals of operational research, 245, 337–358. doi: https://doi.org/10.1007/s10479015-1823-7 triantaphyllou, e. & mann, s.h. (1995). using the analytic hierarchy process for decision making in engineering applications: some challenges. international journal of industrial engineering: applications and practice, 2(1), 35–44. vargas, l.g. (1990). an overview of the analytic hierarchy process and its applications. european journal of operational research, 48(1), 2–8. doi: https://doi.org/10.1016/0377-2217(90)90056-h velasquez, m. and hester, p. t. (2013). an analysis of multi-criteria decision making methods. international journal of operations research, 10(2), 56-66. vincke, p. (1992). multicriteria decision-aid. chichester: john willey & sons. https://doi.org/10.1016/0377-2217(90)90056-h ijahp: article: saaty, wei/ should the uk have brexited the european union? international journal of the analytic hierarchy process 206 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.415 should the uk have brexited the european union? thomas l. saaty distinguished university professor university of pittsburgh pittsburgh, pennsylvania, usa e-mail: saaty@katz.pitt.edu lirong wei university of science and technology beijing, china e-mail: weileerong@foxmail.com abstract this paper is an analysis of brexit, and asked the question, “should the uk have brexited the european union?” we use the analytic hierarchy process (ahp) to model the decision based on benefit-opportunities-costs and risks (bocr). the ahp structure considers various factors that may be taken into consideration from the perspective of the uk (saaty, 1980). questionnaires were used to obtain pairwise comparison judgments from experts and used to derive priorities for the factors and final decision to brexit. the bocr model results are combined in two ways. both results show that the uk should remain a member of the eu. the referendum used showed the opposite result. the approach followed here can be used to educate people when voting on similar decisions. keywords: brexit; ahp; decision-making, benefits, opportunities, cost, risks, decision analysis, european union, eurozone 1. introduction we believe that short of some kind of disaster and annihilation, the human race and its cultures are gradually drawing together from the agrarian age to villages, towns, cities, megacities and nations. this is despite much conflict and strife and great wars that have killed tens of millions of people. the european union is a great and inspired step towards economic unity and in particular financial, social, and political unity. the world owes much to the english people for this increasing unity not only for the world wide language that draws humanity to communicate together, but also in regards to law and geographic dominance ranging from the united states, mailto:saaty@katz.pitt.edu mailto:weileerong@foxmail.com ijahp: article: saaty, wei/ should the uk have brexited the european union? international journal of the analytic hierarchy process 207 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.415 canada, australia, new zealand, and of course the united kingdom itself to a unified india, despite the break up into pakistan and bangladesh. is it beneficial to the uk and to the world at large in the long run that the uk has broken off from the european union in its brexit with dissent coming from londoners, the scottish and the northern irish people (irwin, 2015; moller & oliver, 2014)? that is the question we wish to comprehensively and logically address in this paper. of course, we need to consider the uk’s economic and political advantages and influence today in the hope that it will be a good decision for the future (dagnis jensen & snaith, 2016). on june 23, 2016, the eligible voters in the uk voted on the eu referendum, "should the united kingdom remain a member of the european union or leave the european union?" the result of the vote was that the uk should leave the european union; the breakdown of the votes by region is shown in table 1 and table 2. table 1 vote results of brexit (23 june 2016) results votes % leave 17,410,742 51.89% remain 16,141,241 48.11% valid votes 33,551,983 99.92% invalid or blank votes 25,359 0.08% total votes 33,577,342 100.00% registered voters/turnout 46,501,241 72.21% *source: wikipedia of “united kingdom european union membership referendum, 2016” ijahp: article: saaty, wei/ should the uk have brexited the european union? international journal of the analytic hierarchy process 208 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.415 table 2 results by voting areas in the united kingdom region turnout remain votes leave votes remain % leave % england (with gibraltar) 73.0% 13,266,996 15,188,406 46.62% 53.38% east midlands 74.2% 1,033,036 1,475,479 41.18% 58.82% east of england 75.7% 1,448,616 1,880,367 43.52% 56.48% london 69.7% 2,263,519 1,513,232 59.93% 40.07% north east england 69.3% 562,595 778,103 41.96% 58.04% north west england 70% 1,699,020 1,966,925 46.35% 53.65% south east england 76.8% 2,391,718 2,567,965 48.22% 51.78% south west england & gibraltar 76.7% 1,503,019 1,669,711 47.37% 52.63% west midlands 72% 1,207,175 1,755,687 40.74% 59.26% yorkshire and the humber 70.7% 1,158,298 1,580,937 42.29% 57.71% northern ireland 62.7% 440,707 349,442 55.78% 44.22% scotland 67.2% 1,661,191 1,018,322 62.00% 38.00% wales 71.7% 772,347 854,572 47.47% 52.53% *source: wikipedia of “united kingdom european union membership referendum, 2016” https://en.wikipedia.org/wiki/gibraltar https://en.wikipedia.org/wiki/east_midlands https://en.wikipedia.org/wiki/east_of_england https://en.wikipedia.org/wiki/greater_london https://en.wikipedia.org/wiki/north_east_england https://en.wikipedia.org/wiki/north_west_england https://en.wikipedia.org/wiki/south_east_england https://en.wikipedia.org/wiki/south_west_england https://en.wikipedia.org/wiki/west_midlands_(region) https://en.wikipedia.org/wiki/yorkshire_and_the_humber https://en.wikipedia.org/wiki/northern_ireland https://en.wikipedia.org/wiki/scotland https://en.wikipedia.org/wiki/wales ijahp: article: saaty, wei/ should the uk have brexited the european union? international journal of the analytic hierarchy process 209 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.415 the overall result of the june 2016 vote was a narrow majority of 51.89% to 48.11% in favor of leaving the eu. we note that only london, northern ireland and scotland are below the majority of 50%. 2. bocr model in a complete analysis of a decision problem we usually consider the benefits (b), opportunities (o), costs (c) and risks (r) involved. for each control criterion of these b, o, c, and r, one derives priorities for the alternatives of a decision with respect to all the significant influences that cause some alternatives to have higher priority than others. one then combines the weights of the alternatives according to the weights of the control criteria of each of the b, o, c and r assessed in terms of strategic criteria (wind & saaty, 1980). strategic criteria are very basic criteria used by individuals and groups to assess whether they should make any of the many decisions they face in their daily operations. strategic criteria do not depend on any particular decision for their priorities but are assessed in terms of the goals and values of the individual or organization. finally, one rates (not compares) the top ranked alternative for each b, o, c and r and uses the resulting weights to combine the values of each alternatives for the four merits and obtain the final answer in the form of priorities whose relative values are important for choosing the best alternative. the synthesized results of the alternatives for each of the four control b, o, c and r merits are combined, along traditional benefit to cost ratio analysis used in economics, to obtain a ratio outcome by taking the quotient of the benefits times the opportunities to the costs times the risks for each alternative (bo/cr), then normalizing the results over all the alternatives to determine the best outcome. this formula is only useful when one is certain that the relative measurements are commensurate, that is of the same order of magnitude. in other words it is meaningless to divide thousands of dollars for benefits, by pennies for costs; this is tantamount to dividing by numbers close to zero. there is another more reliable way to combine the b, o, c, and r that gives the total outcome. the top ranked alternative is rated (not compared) for each of the b, o, c and r with respect to strategic criteria that are needed to determine the merits of any decision. from this rating one then obtains normalized respective weights, b, o, c and r and computes the total outcome bb + ob − cc − rr for each alternative. in evaluating the benefits (opportunities), one responds to the question of dominance: which alternative contributes the most benefits (opportunities), whereas for costs (risks) one responds to the question which alternative costs (is subject to greater risks) more, which is opposite in sense of the benefits and opportunities and must be subtracted from them. it is known that the ranks obtained from ratio and total syntheses need not coincide. ijahp: article: saaty, wei/ should the uk have brexited the european union? international journal of the analytic hierarchy process 210 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.415 3. benefits, opportunities, costs and risks of brexit– the analytic hierarchy process (ahp) four hierarchies (figures 1-4) were developed: one for benefits, opportunities, costs, and risks (bocr) to the uk of exiting from the european union (eu). each hierarchy results in priorities that sum to 1.000 for leave or not leave. the priorities are derived from a group decision process. we administered an online survey to collect responses. the ahp questionnaire we used is shown in the appendix. for each judgment in the questionnaire the respondent selected the dominant factor first, then decided how strongly more dominant the factor was using the 1-9 fundamental scale (saaty, 1977, 1986). the questionnaire was sent to 90 experts who attended the isahp2016 in london and came from the uk. they are all ahp decision making experts; however, not all of them are familiar with the brexit. therefore, in our study we used the judgments of four knowledgeable experts who completed the questionnaire. the consistency of judgments was examined for each expert and found to be adequate. next, we calculated the geometric mean of each judgment from these experts and entered the combined judgment into the super decisions model. the alternatives of the decision are as follows: leave the eu or remain a member of the eu. we constructed the model using the super decisions software and inputting the judgments of diverse experts to compute the priorities. figure 1. benefits hierarchy ijahp: article: saaty, wei/ should the uk have brexited the european union? international journal of the analytic hierarchy process 211 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.415 figure 2. opportunities hierarchy economic 1) result in trade barriers between uk and eu 2) loss of british stocks 3) loss of investments 4) recession in economy 5) sterling fall 6) loss of the biggest trading partner political 1) reduce uk influence on eu 2) harder to keep close foreign-policy links with eu 3) loss of tax revenue from leaving security 1) loss of access to eu assets database and surveillance records 2) increase the difficulty of tracing the international criminal social 1) can no longer travel freely 2) loss of technological collaboration with eu figure 3. costs hierarchy opportunities to uk allow the uk to make its own trade deals allow the uk to design its own regulations leave the eu remain a member of the eu ijahp: article: saaty, wei/ should the uk have brexited the european union? international journal of the analytic hierarchy process 212 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.415 economic 1) decline of financial services industry 2) decline of law firms industry 3) london may no longer be finance center of europe 4) risks of diminishing investment from eu 5) us and uk ties may weaken political 1) diminish uk influence in world affairs 2) harder to keep close foreign-policy links with eu 3) risk of losing the uk's prosperity 4) risk of scotland leaving the uk security 1) cause much possible collateral damage to uk’s security 2) potential risk for conflict with eu 3) weaken border control through eu police cooperation social 1) negative for health service 2) loss of eu food protection 3) unemployment rate may rise figure 4. risks hierarchy the results for the four hierarchies are summarized in table 3. the idealized results in the final column to the right are obtained from each relative priority by dividing each by the largest value. for the benefits we have relative values that sum to 1 of leave (0.82) and remain (0.18). dividing each by 0.82 we obtain the ideal values of leave (1.00) and remain (0.22). ijahp: article: saaty, wei/ should the uk have brexited the european union? international journal of the analytic hierarchy process 213 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.415 table 3 priorities derived from the four hierarchies factors priorities relative (ideal) b e n e fi ts economic 0.250 benefit the fishing industry (able to land more fish) 0.119 leave the eu 0.82 (1) remain a member of the eu 0.18 (0.22) free the uk from eu regulations and bureaucracy 0.555 no longer pay more into eu than receive 0.064 save billions of pounds in eu membership fees 0.262 political 0.655 allow the uk to better control immigration 0.833 relive conflict among british conservatives 0.167 social 0.095 reduce pressure on public services, housing and jobs 0.200 improve self-confidence of the british 0.800 o p p o rt u n it ie s allow the uk to design its own regulations 0.800 leave the eu 0.83 (1) remain a member of the eu 0.17 (0.20) allow the uk to make its own trade deals 0.200 c o st s economic 0.103 result in trade barriers between uk and eu 0.184 leave the eu 0.86 (1) remain a member of the eu 0.14 (0.16) loss of british stocks 0.190 loss of investments 0.154 recession in economy 0.150 sterling falls 0.083 loss of the biggest trading partner 0.239 political 0.231 reduce uk influence on eu 0.117 harder to keep close foreign-policy links with eu 0.614 loss of tax revenue from leaving eu enterprises 0.268 security 0.624 loss of access to eu assets database and surveillance records 0.833 increase the difficulty of tracing the international criminal 0.167 social 0.042 can no longer travel freely 0.250 loss of technological collaboration with eu 0.750 r is k s economic 0.153 decline of financial services industry 0.207 leave the eu 0.87 (1) remain a member of the eu 0.13 (0.15) decline of law firms industry 0.208 london may no longer be finance center of europe 0.288 risks of diminishing investment from eu 0.179 us and uk ties may weaken 0.119 political 0.307 diminish uk influence in world affairs 0.737 risk of losing the uk's prosperity 0.085 ijahp: article: saaty, wei/ should the uk have brexited the european union? international journal of the analytic hierarchy process 214 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.415 risk of scotland leaving the uk 0.177 social 0.219 negative for health service 0.752 loss of eu food protection 0.197 unemployment rate may rise 0.051 security 0.322 cause much possible collateral damage to uk’s security 0.189 potential risk for conflict with eu 0.054 weaken border control through eu police cooperation 0.757 a summary of the idealized priorities shown in table 3 are given in table 4. table 4 idealized priority vectors for leave the eu or remain in the eu benefits (b) opportunities (o) costs (c) risks (r) leave the eu 1.00 1.00 1.00 1.00 remain in the eu 0.23 0.20 0.16 0.15 we structured the hierarchy of strategic criteria shown in figure 5 from the media coverage of voter concerns and prioritized the factors through the usual pairwise comparison process. the priorities of the strategic criteria thus derived are shown in table 5. figure 5. the strategic criteria ijahp: article: saaty, wei/ should the uk have brexited the european union? international journal of the analytic hierarchy process 215 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.415 table 5 overall priorities of strategic criteria economic 0.098 trade/business 0.089 financial market 0.012 political 0.244 immigration 0.225 regulations 0.028 social 0.036 education 0.148 health 0.019 job 0.423 travel 0.054 security 0.622 we then created table 6 for rating the benefits, opportunities, costs and risks of this decision. it has one column for every lowest level of strategic criteria. we defined intensities for the scale of very high (0.42), high (0.26), medium (0.16), low (0.1), and very low (0.06). their priorities were derived by making judgments on pairs of intensities in the usual pairwise comparison matrix, asking how much one intensity was preferred to the other. we evaluated the impact of the highest valued alternative in each of the four hierarchies on the strategic criteria in the ratings table shown in table 6. in every one of the four hierarchies the highest priority alternative was to remain in the eu. we selected the appropriate impact for that alternative each cell in table 5. as an example, for the top leftmost cell (benefits, education), we asked what intensity the beneficial impact of leaving would have on education and concluded it would have a medium impact, so medium was entered in the cell. because it was the highest value alternative in every hierarchy we rated remain in the eu across every row. the overall priorities for benefits (b = 0.291), opportunities (o = 0.224), costs (c = 0.243) and risks (r = 0.242), are shown outlined in red in table 6. table 6. priorities for b, o, c, r, from the ratings table 4. results we obtained the overall priorities shown in table 7 in two ways using the multiplicative formula bo/cr and the additive-negative formula bb+oo-cc-rr. the vectors b, o, c, and r are from table 4 and the b, o, c, r priorities are from table 6. we see that “remain in the eu” is dominant (in bold type) using either formula and is the best alternative. ijahp: article: saaty, wei/ should the uk have brexited the european union? international journal of the analytic hierarchy process 216 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.415 table 7 overall final results results alternatives b b=(0.291) o o=(0.224) c c=(0.243) r r=(0.242) bo/cr bb+oocc-rr leave the eu 1 1 1 1 1 0.029 remain in the eu 0.218 0.2 0.158 0.147 1.892 0.034 *b, o, c and r (the respective columns above)are the priority vectors for the two alternatives; b, o, c and r are the priorities derived by rating against the strategic criteria . 5. conclusion the conclusion of this study is that remain in the eu would have been the best outcome, though it flies in the face of the actual vote which was to leave the eu. in this study we aimed to predict the best outcome. whether it really was the best outcome will need to be determined a few years hence as the dust settles. any decision can be viewed in two ways: what is most likely to occur and what is the best outcome. frequently they are not the same. although this study cannot be considered definitive, the outcome of this exercise leaves one to seriously wonder about citizens voting yes or no. this is a habit that we practice inherited from the past with no way to measure the intangibles involved or determine their importance. there is a talk today about france and austria also thinking of leaving the eu, and among others, the japanese have complained about the large investment they have made in the uk as part of the eu. they say this brexit decision amounts to significant losses to them now in a message issued on the eve of the g20 summit in china in september 2016. the document entitled “japan’s message to the uk and eu” warns of dire consequences for “the interests of the world” if an open europe cannot be maintained. there has not been very strong advocacy for our approach about how to examine the subject carefully before resorting to a yes-no kind of vote on the outcome as was done in the united kingdom (dhingra et al., 2016). expert opinion and the strength of the judgments should play an important role in making such decisions that have global consequences and involve multi-criteria kind of thinking. feelings and intuition may not be adequate to obtain the right result. ijahp: article: saaty, wei/ should the uk have brexited the european union? international journal of the analytic hierarchy process 217 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.415 references dagnis jensen, m., & snaith, h. (2016). when politics prevails: the political economy of a brexit. journal of european public policy, 1-9. doi: http://dx.doi.org/10.1080/13501763.2016.1174531 dhingra, s., ottaviano, g. i., sampson, t., & reenen, j. v. (2016). the consequences of brexit for uk trade and living standards. the centre for economic performance, the london school of economics and political science. irwin, g. (2015). brexit: the impact on the uk and the eu. global council: london. available online at: http://www. globalcounsel. co. uk/system/files/publications/global_counsel_impact_of_brexit_june_2015. pdf. möller, a., & oliver, t. (2014). the united kingdom and the european union: what would a “brexit” mean for the eu and other states around the world? european and global perspectives. dgapanalyse. saaty, t. l., & vargas, l. g. (2012). the possibility of group choice: pairwise comparisons and merging functions. social choice and welfare, 38(3), 481-496. doi: 10.1007/s00355-011-0541-6 saaty, t. l. (1977). a scaling method for priorities in hierarchical structures. journal of mathematical psychology, 15(3), 234-281. doi: http://dx.doi.org/10.1016/00222496(77)90033-5 saaty, t.l. (1980). the analytic hierarchy process. new york: mcgraw-hill. doi: http://dx.doi.org/10.1080/00137918308956077 saaty, t. l. (1986). axiomatic foundation of the analytic hierarchy process. management science, 32(7), 841-855. doi: http://dx.doi.org/10.1287/mnsc.32.7.841 saaty, t. l. (1996). the analytic network process. pittsburgh: rws publications. wind, y., & saaty, t. l. (1980). marketing applications of the analytic hierarchy process. management science, 26(7), 641-658. doi: http://dx.doi.org/10.1287/mnsc.26.7.641 http://dx.doi.org/10.1080/13501763.2016.1174531 http://dx.doi.org/10.1016/0022-2496(77)90033-5 http://dx.doi.org/10.1016/0022-2496(77)90033-5 http://dx.doi.org/10.1080/00137918308956077 http://dx.doi.org/10.1287/mnsc.32.7.841 http://dx.doi.org/10.1287/mnsc.26.7.641 ijahp: article: saaty, wei/ should the uk have brexited the european union? international journal of the analytic hierarchy process 218 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.415 appendix i should the uk have brexited the european union? this is a questionnaire of the ahp benefits-opportunities-costs-risks model to evaluate "should the uk have brexited the eu?" there are five parts to the questions as can be seen below. please do the pairwise comparison of all the criteria using the 1-9 dominance scale of absolute numbers: 9 extremely 8 very strongly to extremely 7 very strongly 6 strongly to very strongly 5 strongly 4 moderately to strongly 3 moderately 2 equally to moderately 1equally click ""next" at bottom of the page to begin. *required ijahp: article: saaty, wei/ should the uk have brexited the european union? international journal of the analytic hierarchy process 219 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.415 ijahp: article: saaty, wei/ should the uk have brexited the european union? international journal of the analytic hierarchy process 220 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.415 ijahp: article: saaty, wei/ should the uk have brexited the european union? international journal of the analytic hierarchy process 221 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.415 ijahp: article: saaty, wei/ should the uk have brexited the european union? international journal of the analytic hierarchy process 222 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.415 ijahp: article: saaty, wei/ should the uk have brexited the european union? international journal of the analytic hierarchy process 223 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.415 (you can get the complete questionnaire at https://goo.gl/forms/dgdba9tprkq757rk2) ijahp news and events: ijahp editor valentina ferretti in the news international journal of the analytic hierarchy process 147 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.474 ijahp editor valentina ferretti in the news valentina ferretti, our ijahp news and events editor, is in the news herself. she has been invited to teach at the guardian’s one-day mba master class. this prestigious publication has organized a one-day professional training focusing on key mba topics that every manager should know and has invited leading world academics to participate. dr. ferretti is currently a fellow in decision science in the department of management at the london school of economics and political science, and has also served as visiting professor at oxford university and the école centrale paris. therefore, being invited to participate in the guardian’s class is a well-deserved accomplishment for dr. ferretti. in a recent communication, she told our journal she plans to cover the following topics in her session:  how do people make choices (system 1 – intuition – versus system 2 – deliberate thinking process)?  what are the key decision traps that we tend to fall into?  how to help customers/others make better decisions in tough situations where stakes are high and there are multiple objectives to be achieved. dr. ferretti stated that her session will thus provide insights on how we make decisions, what are the traps we tend to fall in, and how we can improve our strategic decisionmaking capabilities. as a result, participants will find that learning about how to make better decisions can be leveraged in a number of ways, thus providing participants with a competitive advantage. congratulations to our esteemed colleague for this merited accomplishment and we look forward to her the guardian’s one-day mba session! https://doi.org/10.13033/ijahp.v9i1.474 https://www.theguardian.com/guardian-masterclasses/2016/feb/29/the-one-day-mba-a-day-of-talks-with-leading-business-academics-course https://www.theguardian.com/guardian-masterclasses/2016/feb/29/the-one-day-mba-a-day-of-talks-with-leading-business-academics-course ijahp: mu/ open access journal as global agoras international journal of the analytic hierarchy process 1 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.382 april 2016 open access journals as global agoras dear ijahp reader: the ancient greeks used to congregate in the agora, a central open space or square in their city states, to listen to statements from the council or military bosses or to simply discuss politics as needed. the agora was the center for discussions by the free thinkers in the city. our global knowledge society does not have this physical open space, but the web and in particular open access journals, fill this need for a central conversational cyberspace where scholars, interested in similar topics, can exchange ideas through their essays and research studies. why is this relevant to our journal? when a potential author submits a paper to a journal, s/he is joining a conversation in this modern agora; therefore, it is expected that this author will be familiar with what is being discussed and the different ideas and arguments surrounding the topic. there is nothing more frustrating for a journal editor than to realize that a potential author is not aware of the ongoing conversation about the chosen topic in the current journal. the effect is similar to someone interrupting a conversation to say something that was already said a few minutes ago. there have been several discussions concerning the ethics of requesting potential authors to cite papers from the same journal. at ijahp, we do not condone this approach. however, we do expect potential authors to be familiar with the ongoing ijahp conversation about the topic of choice as this shows their suitability to join our specific agora to discuss the particular selected topic. thanks for your support and enjoy this issue! enrique mu, phd ijahp editor-in-chief ijahp article: vardari, canhasi-kasemi /comparison of two different judgement scales with the ahp gsm operator preference of university students international journal of the analytic hierarchy process 1 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.970 comparison of two different judgment scales with the ahp: gsm operator preference of university students esma canhasi-kasemi faculty of education university “ukshin hoti” prizren prizren, kosova esma.canhas@uni-prizren.com luan vardari 1 faculty of economics university “ukshin hoti” prizren prizren, kosova luan.vardari@uni-prizren.com abstract the analytical hierarchy process (ahp) is a method with elegant mathematical features that is widely used in multi-criteria decision making. one of the main applications of this method, which is frequently preferred by decision makers due to its systematic and understandable structure, includes addressing inadequacies in terms of numerical scales that are generally used in pairwise comparisons. therefore, this study includes two different judgment scales, saaty’s fundamental scale and the balanced scale, which were used in the pairwise comparison stage. after the comparisons were made, the variance related to the consistency ratios and the range of the sensitivity was also observed. in the study, we discuss the use of both judgment scales in a real problem and their effects on priority estimation in the ahp. the study's goal is to evaluate the outcomes of saaty’s fundamental scale and the balanced scale in the ahp technique for the two current operators in kosovo's gsm sector, vala and ipko, and assess the preference of students in kosovo. the required data were obtained through a questionnaire and the importance weights of the decision criteria were calculated separately for each scale and compared. the preference order of the gsm operators was discovered according to each decision criterion and all criteria. the ranking of the weights obtained with both scales resulted in ipko first, followed by vala. the balanced scale made the results lighter in the weight distribution. another important result is that the pairwise comparisons made with the balanced scale yielded results that are more sensitive. in addition, the closeness of the priority vectors obtained with both scales according to saaty’s compatibility index and garuti’s compatibility index was examined. 1 corresponding author ijahp article: vardari, canhasi-kasemi /comparison of two different judgement scales with the ahp gsm operator preference of university students international journal of the analytic hierarchy process 2 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.970 keywords: analytic hierarchy process (ahp); judgment scales; saaty’s compatibility index; gsm operator 1. introduction the mobile phone and gsm sector has some of the fastest technological development. in recent years, there has been a rapid increase in the use of mobile phones in the world. worldwide, the number of mobile phone subscribers has reached 4.6 billion. while the rate of increase in the number of mobile phone subscribers was 154% in 2004 compared to 2000, it reached 2 billion in 2008 with the increase rate of 15.6% compared to 2004. according to the united nations telecommunications agency, the total number of subscribers using mobile phones is expected to reach 5 billion this year. in 2000, vala 900 began operations in kosovo and ipko began operating in december 2007. vala 900 ptk-pt is the first gsm operator in kosovo in this sector. since the year it was founded, it has developed by increasing its capacity and number of users and offering discounts. from 2000 until 2008, the company held 886,000 subscribers and a 42.19% market share. it reached 1 million users this year and covered 87% of kosovo's inhabited area. for mobile phone subscriptions, the market has reached a saturation point. therefore, gsm operators conduct campaigns aimed at convincing subscribers to switch operators, as well as campaigns aimed at convincing people who are not mobile phone subscribers to become subscribers. operators with a low market share try to increase their subscriber numbers with attractive marketing campaigns. the operator who is the market leader tries to prevent its subscribers from switching to other operators with marketing campaigns that highlight the advantages of the company which have resulted in their high number of subscribers. it has been observed that mobile phone subscribers change operators due to the communication campaigns offering cheaper options that emerged as a result of this competition. university students constitute a significant portion of gsm operator subscribers. the aim of this study was to determine the order of preference of the gsm operators using the ahp and evaluate these findings. we also aimed to analyze the results comparatively by applying two different scales recommended for the ahp. saaty's fundamental scale and salo and hämäläinen’s (1997) balanced scale were explained and applied to a real problem, and the results were summarized and interpreted. in practice, the students’ gsm operator selection was based on the existing gsm operators. within the framework of this method, we investigated the gsm operator preference of university students in kosovo. the main purpose of the application was to compare and interpret the gsm operator ranking with the subjective ranking created using the ahp method, which is used in evaluations that include subjective criteria. in the application, survey data obtained in three languages from the relevant gsm users was used as the data set. 2. literature review in the literature, many tools have been used for investment planning in the mobile sector and gsm selection, such as the analytic hierarchy process (ahp), the analytic network ijahp article: vardari, canhasi-kasemi /comparison of two different judgement scales with the ahp gsm operator preference of university students international journal of the analytic hierarchy process 3 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.970 process (anp), fuzzy ahp, etc. some studies used the ahp method to determine the most appropriate gsm operator preference. this section summarizes the literature review focusing on the criteria used in the models. table 1 methods and tools used in investment project evaluation subject/purpose tools/methodology reference  to determine consumers’ preferences in selecting a mobile network operator ahp technique alhazaymeh, k., nasruddin h. (2013)  to determine region evaluation criteria’s priority coefficients (recpc)  to determine country evaluation criteria’s priority coefficients (cecpc)  to determine gsm operator evaluation criteria’s priority coefficients (goecpc).  to determine gsm operator’s scoring (gos), ranking and select the best one ahp technique çavuşoğlu, ö., canolca, m., and bayraktar, d. (2010).  to determine the preferences of university students when buying mobile phone line gsm operators (turkcell, vodafone, avea). ahp technique dündar, s., and ecer, f. (2008).  to determine the best mobile value-added service firm providing the most customer satisfaction fuzzy synthetic evaluation method ahp kuo, y.f., and chen, p.c. (2005)  to analyze the impact of using different judgment scales on the resulting priorities and consistency to default scale as proposed by saaty ahp technique franek, j., & kresta, a. (2014)  to improve the traditional work methods in the selection of fruit ahp technique srđević, z., & srđević, b. (2003). ijahp article: vardari, canhasi-kasemi /comparison of two different judgement scales with the ahp gsm operator preference of university students international journal of the analytic hierarchy process 4 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.970 2.1 analytic hierarchy process – ahp the analytic hierarchy process (ahp) is a multi-criteria decision structured technique for organizing and analyzing complex decisions. it was developed by thomas saaty and has been widely used since 1970. the ahp assists decision makers by enabling the conversion of verbal assessments into numerical values, which allows pairwise comparisons to rank the choices in order of importance. therefore, the ahp is a very helpful way to make a choice when there are several options to weigh in relation to certain aspects (saaty, 1982; saaty & vargas, 2012). the ahp method can be used in both social and physical areas to make measurements (ozdemir, 2018). the ahp method generally consists of the following five main steps: 1. defining and hierarchically displaying the decision problem 2. creating a pairwise comparison matrix 3. determining weight 4. performing a consistency assessment 5. integrating weights and making a final decision  to compare different scale functions and derive a recommendation for the application of scales ahp technique goepel, k. d. (2019).  proportion judgment scale and introduce a new method based on the proportion scale for construction comparison matrix in the analytic hierarchy process (shp) ahp technique decai, h., & liangzhong, s. (2003)  to determine best matching scale according to the mental representation of the verbal scale of each individual decision-maker, verbal scales are first used to compare alternatives with known measures, e.g. surface of figures. ahp technique with pairwise comparisons meesariganda, b. r., & ishizaka, a. (2017).  to determine the best location for a dry port that would support the container terminals within the hinterland of kocaeli ports by applying analytic hierarchy process (ahp) with a case study ahp technique saka, m., & cetin, o. (2020)  ranking of the gsm operators fuzzy anp erginel, n., & sentürk, s. (2011). ijahp article: vardari, canhasi-kasemi /comparison of two different judgement scales with the ahp gsm operator preference of university students international journal of the analytic hierarchy process 5 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.970 in order for the ahp to be applied, the decision problem and goal of the process must be clearly defined. an ahp structure consists of three main levels: goal, factors (criteria), and decision points (alternatives), where the point to be reached is defined as the goal. the decision problem is divided into sections to define the main criteria that must be met in the most general sense to achieve the main goal, and a hierarchy is created. after all the criteria at the lowest level have been worked through, the same process is repeated for the criteria at the higher level. this step is repeated until the main criteria, which are listed below the main objective, are also compared. all of the alternatives are then compared in pairs according to their performance on all criteria at the lowest level of the hierarchy, using pairwise comparisons to obtain the data. the matrix of pairwise comparisons is used to determine the priority of the decision criteria based on the judgment of the decision maker. in the pairwise comparison phase, two different judgment scales, saaty's fundemental scale and the balanced scale, were used. the purpose of using two different scales was to compare them and examine whether there is any valuable variation that could change the ranking of the alternatives. after the comparisons were made, the consistency ratios and the closeness of the priority vectors obtained with both scales between these two vectors according to the saaty compatibility index and the garuti (2017) compatibility index were examined. in recent decades, several modifications of the ahp have been proposed. the pairwise comparison scale is one of the current topics of discussion when considering modifications and improvements (goepel, 2019). the fundamental ahp scale proposed by saaty (1980) consists of integer values from 1 to 9, whose verbal expressions can be seen in table 2. with the real numbers used in this scale, an individual decision maker can accurately express his or her preference for all pairs of alternatives (brunelli, 2015). the other proposed scales also contain a total of 9 numerical values corresponding to the same expressions. essentially, these verbal comparisons are converted into proportional ratings through a one-to-one matching process with a numerical scale (meesariganda & ishizaka, 2017). goepel (2019) categorizes ahp scales into the following three groups: group 1: the maximum entry value in the pairwise comparison matrix is kept at nine as in saaty's fundamental scale. the fundamental scale, the inverse linear scale, the balanced scale, and the generalized balanced scale are grouped in this category. group 2: the maximum range of input values in the pairwise comparison matrix is reduced to values less than nine. the logarithmic scale, square root scale and koczkodaj scale are combined in this category. group 3: the maximum input values and the range of input values in the pairwise comparison matrix are increased to values higher than nine. the power scale, geometric scale, adaptive scale and adaptive-balanced scale are combined in this category. ijahp article: vardari, canhasi-kasemi /comparison of two different judgement scales with the ahp gsm operator preference of university students international journal of the analytic hierarchy process 6 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.970 table 2 verbal judgments equivalent to the saaty’s fundamental scale and the balanced scale verbal expression of the scale saaty’s fundamental scale balanced scale equal importance 1 1.00 weak/slight importance over another 2 1.22 moderate importance over another 3 1.50 moderate plus importance over another 4 1.85 strong/essential importance over another 5 2.33 strong importance over another 6 3 very strong importance over another 7 4 very strong to extreme importance over another 8 5.67 extreme importance over another 9 9.00 source: (saaty, 2008; meesariganda, & ishizaka, 2017) as can be seen in table 2, the verbal expressions are presented on scales from 1 to 9. which scale is better is still an open question, but it can be inferred that saaty’s fundamental ahp scale is not optimal, as all other scales seem to have some supporting evidence (brunelli, 2015). salo and hämäläinen (1997) note that integers from 1 to 9 give uniformly distributed local weights, so there is no sensitivity in comparing items that are close to each other. because of this fact, they proposed a balanced scale (see table 2) in which the local weights are evenly distributed over the weight range (franek & kresta, 2014). the balanced scale was proposed based on empirical experiments with humans and showed that a balanced scale increases the reliability of estimates and the overall consistency of the hierarchy evaluation process as well as reduces the dispersion of calculated weight values. (brunelli, 2015). the scale function of this balanced scale is as follows (goepel, 2019): 𝒄 = 𝟗 + 𝒙 𝟏𝟏 − 𝒙 the real values corresponding to the verbal judgments associated with the balanced scale are shown in table 2. in this article, pairwise comparisons were made by applying the fundamental and balanced scales to the same case study. differences were examined to make a comparison between the two methods and determine if there was a greater variance affecting the decision-making process. agreement with saaty’s and garuti's priority compatibility index was then examined. in addition, if the decision to be made at the end of a study is in a structure that will affect many people, pairwise comparison decision matrices are formed by combining the judgments of different people. in this process of combining judgements, many researchers suggest using the geometric mean method to obtain consistent pairwise comparison matrices. ijahp article: vardari, canhasi-kasemi /comparison of two different judgement scales with the ahp gsm operator preference of university students international journal of the analytic hierarchy process 7 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.970 if the pairwise comparison that reflects the personal judgments of the decision-maker based on the scale 1-9 is represented by a, then aij indicates the importance of feature i according to feature j. the pairwise comparison matrix is obtained as follows when m indicates the number of criteria to be evaluated: when 𝐴 = [ 𝑎11 𝑎12 𝑎21 𝑎22 … 𝑎1𝑛 … 𝑎2𝑛 ⋮ ⋮ 𝑎𝑚1 … ⋱ ⋮ … 𝑎𝑚𝑛 ] = [ 𝑎11 𝑎12 1 𝑎21⁄ 1 𝑎22⁄ … 𝑎1𝑛 … 1 𝑎2𝑛⁄ ⋮ ⋮ 𝑎𝑚1 … ⋱ ⋮ … 𝑎𝑚𝑛 ] , 𝑖, 𝑗 = 1, … , 𝑚, 𝑎𝑖𝑗 > 0, if 𝑎𝑖𝑗 = 1 𝑎𝑗𝑖 and 𝑎𝑖𝑖 = 1, 𝑎𝑖𝑗 = 𝑎𝑖𝑘 𝑎𝑘𝑗 , 𝑖, 𝑗, 𝑘 = 1, … , 𝑚, equalities are provided, the matrix a is fully consistent; otherwise, it is inconsistent (wang & elhag, 2008). having defined the a matrix, its elements need be normalized by dividing the value of each element to the sum of the column. in a normalized matrix, significance values (weight values) are determined by calculating the arithmetic mean of each row. given that 𝑏𝑗 , j indicates the total value of a column, the total value of a column is calculated with the following formula: 𝑏1 = ∑ 𝑎1𝑖 𝑚 𝑖=1 . then, the elements of a matrix are divided by the total value of their column with the following formula: 𝑐𝑖𝑗 = 𝑎𝑖𝑗 /𝑏𝑖 as a result, matrix c with 𝑚 × 𝑚 dimension is found by normalizing the pairwise comparison matrix as follows: 𝐶 = [ 𝑐11 𝑐12 𝑐21 𝑐22 … 𝑐1𝑛 … 𝑐2𝑛 ⋮ ⋮ 𝑐𝑚1 … ⋱ ⋮ … 𝑐𝑚𝑛 ] . c matrix helps specify relative percentage significance values (i.e. their weight values) of the criteria. 𝑤𝑖 = ∑ 𝑐𝑖𝑗 𝑚 𝑗=1 𝑚 w column vector refers to the percentage weight of the criteria calculated with the arithmetic mean of the rows in a c matrix. 𝑊 = [ 𝑤1 𝑤2 ⋮ 𝑤𝑚 ] the validity, and therefore factuality, of the results relies on the consistency of the pairwise comparison matrices. the consistency of results is confirmed with the ijahp article: vardari, canhasi-kasemi /comparison of two different judgement scales with the ahp gsm operator preference of university students international journal of the analytic hierarchy process 8 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.970 consistency ratio (cr). a consistency analysis also helps highlight incorrect judgments and reduce errors. to measure the consistency of the pairwise matrices, the cr is calculated by dividing the consistency index (ci) by the random index (ri). to measure the ci, 𝜆𝑚𝑎𝑥 defines the biggest eigenvalue of a matrix and is calculated with (𝐴 − 𝜆𝑚𝑎𝑥 𝐼)𝑊 = 0 , w = 0. ri is identified with the size of the pairwise matrix. the ri values from 1 to 10 are shown in table 3. 𝐶𝐼 = (𝜆𝑚𝑎𝑥 − 𝑚) 𝑚 − 1⁄ , 𝐶𝑅 = 𝐶𝐼 𝑅𝐼 table 3 ri values n 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 ri̇ 0 0 0.58 0.9 1.12 1.24 1.32 1.41 1.45 1.49 1.51 1.48 1.56 1.57 1.59 if the cr value is less than 0.10, the comparisons of the decision makers are considered to be satisfactorily consistent; if not, they are inconsistent, which indicates that the ahp has not produced significant results (lee, mogi, shin, & kim, 2007). in another study, saaty (1996) proposed 5% and 8% as thresholds for 3x3 and 4x4 matrices, respectively while keeping a 10% threshold for larger matrices. 3. determining the preference order of gsm operators dündar & fatih (2008) determined that mobile phone users consider the following criteria when choosing a gsm operator: call fee, coverage area, line fee and service quality. in another study by yıldız (2019), product features, exterior, customer satisfaction and service were determined to be the most important factors in the selection of gsm operators. the objective of the ahp model in this article is to “determine the gsm operator preference of university students.” the structure of the decision hierarchy is seen in figure 1 below. figure 1 structure of the decision hierarchy ijahp article: vardari, canhasi-kasemi /comparison of two different judgement scales with the ahp gsm operator preference of university students international journal of the analytic hierarchy process 9 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.970 in order, to determine the students’ gsm operator preference, which is the aim of the study, technical competence, service quality, advertising, price, and the use of the same operator in the student's environment were determined as the decision criteria to use in the ahp method. in this study, we constructed the decision hierarchy in order to achieve a reasonable result. the pairwise comparisons were then created in accordance with the analysis of the alternatives. for this purpose, each gsm operator subscriber was asked to weight the criteria according to the degree of importance using saaty’s fundamental scale (1997) and the balanced scale. for these calculations, the evaluations of 100 students who are gsm operator subscribers were selected as samples and considered. table 4 shows the pairwise comparison matrix that was created using saaty’s scale. table 4 pairwise comparison matrix for criteria using saaty’s fundamental scale criteria technical competence service quality advertisement price same gsm operator subscription by others in student's environment technical competence 1 2 2 1/2 1/2 service quality 1/2 1 2 1/2 1/2 advertisement 1/2 1/2 1 1/2 1/3 price 2 2 2 1 1/2 same gsm operator subscription by others in student's environment 2 2 3 2 1 the values of the pairwise comparison matrix adapted to the balanced scale are shown in table 5. ijahp article: vardari, canhasi-kasemi /comparison of two different judgement scales with the ahp gsm operator preference of university students international journal of the analytic hierarchy process 10 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.970 table 5 comparison matrix for criteria using the balanced scale criteria technical competence service quality advertisement price same gsm operator subscription by others in student's environment technical competence 1 1.22 1.22 1/1.22 1/1.22 service quality 1/1.22 1 1.22 1/1.22 1/1.22 advertisement 1/1.22 1/1.22 1 1/1.22 1/1.5 price 1.22 1.22 1.22 1 1/1.22 same gsm operator subscription by others in student's environment 1.22 1.22 1.5 1.22 1 with the help of expert choice, the weights determined as a result of the comparisons made with both scales were calculated and are shown in table 6. the weights of the alternatives were determined according to each criterion by going through similar processes. table 6 weights of criteria computed with saaty’s fundamental scale and balanced scale criteria weights and rank saaty`s scale rank balanced scale rank technical competence 0.184 3 0.198 3 service quality 0.138 4 0.183 4 advertisement 0.095 5 0.162 5 price 0.242 2 0.215 2 same gsm operator subscription by others in student's environment 0.341 1 0.242 1 consistency ratio 0.03 0.0025 as can be seen in table 6, the distribution of weights found by saaty’s scale and the balanced scale varies considerably, but the order of importance of the criteria is the same. the students reported that having the same operator was the most important criterion and ijahp article: vardari, canhasi-kasemi /comparison of two different judgement scales with the ahp gsm operator preference of university students international journal of the analytic hierarchy process 11 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.970 the advertisement criterion was the least important. although the order of the criteria is the same, the fact that the distribution of weights differed depending on the scales is due to the inconsistency accumulated in pairwise comparisons. as can be seen in this application, the pairwise comparison matrix created with the balanced scale has significantly reduced the weight ratios of the criteria. for example, the relative ratio of the price and advertising criteria with saaty’s scale is (0.242/0.095) = 2.547 and for the balanced scale is (0.215/0.162) =1.327. although the absolute differences in the weight ratios are small and the ranks are the same, the only ratio that the ahp method takes into account is the relative ratio; therefore, the above comparison shows one of the situations where significantly different results can occur when different scales are used. to explain the importance of this weight distribution, for example, when the gsm operator in our study wants to allocate its budget according to these criteria, they will be able to make a better distribution of resources knowing these weights. one of the important factors that the ahp considers is cr. if this ratio is less than 0.10, the pairwise comparison is valid. in his study, yıldırım (2019) interpreted the performance of the scales using many scales and several performance measurement methods. measured scale lower cr, which is one of the scale performance measurement methods used, represents the percentage of trials with lower cr values, and these values have been generated by the measured scale rather than saaty’s fundamental scale. the fact that cr values resulting from the balanced scale are lower than those from saaty’s scale does not prove that the balanced scale is a better option. this is an expected result since the numbers in the balanced scale matrix are generally closer together (that’s why it is call balanced), thus they will generally achieve better consistency. if we change the numbers to those that are closer together, we will get an even better consistency. the closer the numbers, the better the cr; however, this is not an argument for choosing the balanced scale. the goal is to not only achieve better consistency but to prove that the judgments are more representative and the final outcome is better (at least in most of the cases). in addition, the closeness of the priority vectors obtained with both scales according to the saaty’s compatibility index was examined. even when vectors are not identical, they can sometimes be considered close to each other. according to saaty (2005) “when two vectors are close, we say they are compatible”. the saaty compatibility index, s, was the first developed measure of compatibility between priority vectors. this index uses the concept of the hadamard product, the element-wise product of two matrices (garuti, 2012). the saaty compatibility index, s, between vectors x and y is obtained with the following equation: 𝑆 = (1 𝑛2⁄ )𝑒𝑇 𝐴⦁𝐵𝑇 𝑒 where n is the number of elements of the vectors, e is a column-matrix with all elements equal to 1, 𝑎𝑖𝑗 = 𝑥𝑖 𝑥𝑗⁄ , 𝑏𝑖𝑗 = 𝑦𝑖 𝑦𝑗⁄ , and ⦁ is the hadamard product operator (saaty, t. l.; peniwati, k., 2013). one desirable property of a consistency index is that it should indicate that a vector is completely compatible with itself. for identical vectors, 𝑆 = 1. if 𝑆 ≤ 1.1 the two vectors are said to be compatible; otherwise, they are not. the garuti compatibility index, g, between vectors x and y is calculated using the equation below. this index is based on a physical interpretation of the inner product of ijahp article: vardari, canhasi-kasemi /comparison of two different judgement scales with the ahp gsm operator preference of university students international journal of the analytic hierarchy process 12 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.970 two vectors, "x, y", given by |x||y| cos α, where α is the angle between vectors x and y. for identical normalized vectors, α = 0 and 'x, y' = 1. for perpendicular (orthogonal) vectors, α = 90° and 'x, y' = 0. for identical normalized vectors, g = 1, which means total compatibility. the smallest possible value is g = 0, which means total incompatibility. 𝑮 = ∑ [ 𝐦𝐢𝐧 (𝒙𝒊, 𝒚𝒊) 𝐦𝐚𝐱 (𝒙𝒊, 𝒚𝒊) (𝒙𝒊 + 𝒚𝒊) 𝟐 ] 𝒏 𝒊=𝟏 if g< 0.9, garuti (2017) proposes that x and y should be considered not compatible. table 6 shows two priority vectors obtained for the criteria using saaty’s scale and the balanced scale. the corresponding elements of vector 1 and 2 appear close to each other based on a cursory examination of the differences between them. so, s = 1.096 for vectors 1 and 2 indicates that they are indeed compatible. g=78.38% indicates that these two vectors are not compatible. the lower threshold for g is 90%); therefore, they are far from compatible. a mathematical simulation should be performed to investigate whether g < 0.9 can be tolerated for vectors with higher n, as has been done for the consistency index (garuti & salomon, 2012). table 7 weights of alternatives computed with saaty’s fundamental scale and balanced scale alternative weighs and ranks saaty`s scale rank balanced scale rank vala 0.378 2 0.469 2 ipko 0.622 1 0.531 1 similarly, table 8 shows that although the weights of the alternatives are different, ipko is the more preferred gsm operator by the students. if we look at the relative ratios, ipko and vala weights ratios were 0.622/0.378 with saaty’s scale, and this ratio was 0.469 for vala and 0.531 for ipko with the balanced scale. since there are only two options here, there is no inconsistency, so the cr values are not compared. table 7 shows two priority vectors determined for the alternatives using saaty’s fundamental scales and the balanced scale. the corresponding elements of vectors 1 and 2 appear to be close to each other based on a cursory examination of the differences between them. therefore, s = 1.035 for vectors 1 and 2 indicates that they are indeed compatible. g=83.3% < 90% indicates that these two vectors are not compatible; therefore, they are far from being compatible. ijahp article: vardari, canhasi-kasemi /comparison of two different judgement scales with the ahp gsm operator preference of university students international journal of the analytic hierarchy process 13 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.970 as a result of this research, the opinions of the subscribers of gsm operators in kosovo about two gsm operators were determined through the analysis including a survey, and a gsm operator ranking was obtained from the perspective of the subscribers. there is full consistency as the consistency ratios of the criteria and options were far less than 10%. 4. conclusions in the use of communication tools, the expectations and preferences of subscribers differ. since university students constitute a significant portion of the population using communication tools, their preference for the actively operating gsm operators (vala, ipko) was determined using the ahp method. in the study, six decision criteria that students consider when choosing a gsm operator were determined. these chosen criteria were technical competence, service quality, advertisement, price, and same gsm operator subscription used by others in the student's environment. students made pairwise comparisons of these decision criteria and gsm operators according to each decision criterion based on the importance level of the ahp method. these pairwise comparisons were evaluated using two different scales: saaty's fundamental scale and the balanced scale. then, the weight scores of the decision criteria of the students, which were obtained from the pairwise comparisons made with both scales, were determined and the comparison was made. during the evaluation, inconsistencies were checked for both scales and it was concluded that the balanced scale had more consistent results in terms of scale performance (pöyhönen, hämäläinen, & salo, 1997). when the results were examined, although the ranks obtained with both scales were the same, differences were observed in the weight distributions. since the weights of the decision-making elements are often more important than their ranking in allocation problems, it is important which scale was used and additional analysis is needed to make a final decision. as can be seen from table 4, the weight distribution of the criteria obtained with the balanced scale in this application has been significantly lightened (this is not necessarily an advantage). one may have many ways to get a lightened distribution of the weights, but that is not a sufficient motive for choosing the balanced scale. this "lightened process" can introduce some bias to the results. the use of the balanced scale instead of the original fundamental scale in the ahp standard process during the determination of the most appropriate gsm operator preference shows that the ahp is robust and reliable as a method to support the decisionmaking process even when the decision scale is changed. as a result, ipko was the first preferred operator and vala was the second preferred gsm operator by the students. also, the first measure of compatibility between priority vectors was developed by saaty and it uses the concept of the hadamard product, which is the elemental product of two matrices. garuti (2007) proposed a different g-index of compatibility based on the inner product's physical interpretation, including the vector projection concept. when two vectors are the same, the threshold s = 1. s ≤ 1.1 is set as an upper bound for congruent vectors. one issue with saaty’s compatibility index is that under certain circumstances, the upper threshold might change to 1.01. tables 6 and 7 show the comparison of two priority vectors obtained using saaty’s fundamental scale and the balanced scale for criteria and alternatives, respectively. the ijahp article: vardari, canhasi-kasemi /comparison of two different judgement scales with the ahp gsm operator preference of university students international journal of the analytic hierarchy process 14 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.970 compatibility of the two priority vectors is calculated using both saaty’s and garuti’s compatibility indices. for the priority vectors of criteria, the result of the garuti index is 78.38% which shows that the two vectors are not compatible with the threshold being 90%. however, the result of the saaty index is 1.096, which is less than the threshold of 1.1, indicating that the two priority vectors are compatible according to the saaty compatibility index. for the priority vector of alternatives, the result of the garuti index is 83.3%, which is below the threshold of 90%, indicating that the two vectors are incompatible. on the other hand, the result of the saaty index is 1.035, which is less than the threshold of 1.1, indicating that the two priority vectors are compatible according to the saaty compatibility index. a mathematical simulation is needed to determine if a g value below 90% can be tolerated for higher n vectors, as is done for the consistency index (garuti & salomon, 2012). subsequent studies should focus on the use of different scales in different decisionmaking problems. in order to determine the best approach, an application in a real-life problem as in this study is recommended or a mathematical programming and simulation in which the results for all judgment scales are compared. ijahp article: vardari, canhasi-kasemi /comparison of two different judgement scales with the ahp gsm operator preference of university students international journal of the analytic hierarchy process 15 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.970 references alhazaymeh, k., nasruddin h. (2013). generalized interval-valued vague soft set. applied mathematical sciences, 7(140), 6983-6988. doi: http://dx.doi.org/10.12988/ams.2013.310575 brunelli, m. (2015). introduction to the analytic hierarchy process. springer. çavuşoğlu, o., canolca m., & bayraktar, d. (2011). gsm operator selection for a call center investment by using ahp. ifors 2011, conference for the international federation of operational research society, melbourne, australia. decai, h., & liangzhong, s. (2003). new method for constructing comparison matrix based on the proportion scales in the ahp. journal of systems engineering and electronics, 14(3), 8-13. doi: https://doi.org/10.1016/j.asoc.2016.12.040 dündar, s., & fatih, e. (2008). determination of university students’ preferences of gsm operator using the analytic hierarchy process method. yönetim ve ekonomi: celal bayar üniversitesi i̇ktisadi ve i̇dari bilimler fakültesi dergisi, 15(1), 195-205. doi: https://doi.org/10.18657/yonveek.335225 erginel, n., sentürk, s., kahraman, c., & kaya, i̇. (2011). evaluating the packing process in food industry using fuzzy and [stilde] control charts. international journal of computational intelligence systems, 4(4), 509-520. doi: https://doi.org/10.1080/18756891.2011.9727809 franek, j., kresta, a. judgment scales and consistency measure in ahp. procedia economics and finance, 12(2014), 164–173. doi: https://doi.org/10.1016/s22125671(14)00332-3 garuti, c., & salomon, v. a. (2012). compatibility indices between priority vectors. international journal of the analytic hierarchy process, 4(2), 152-160. doi: https://doi.org/10.13033/ijahp.v4i2.130 garuti, c. (2017). reflections on scales from measurements, not measurements from scales. international journal of the analytic hierarchy process, 9(3), 349-361. doi: https://doi.org/10.13033/ijahp.v9i3.522 goepel, k. (2019). comparison of judgment scales of the analytical hierarchy process—a new approach. international journal of information technology & decision making, 18(02), 445-463. doi: https://doi.org/10.1142/s0219622019500044 kuo, y., & chen, p. (2006). selection of mobile value-added services for system operators using fuzzy synthetic evaluation. expert systems with applications, 30(4), 612620. doi: https://doi.org/10.1016/j.eswa.2005.07.007 lee, s., mogi, g., shin, s., & kim, j. (2007). an ahp/dea integrated model for measuring the relative efficiency of energy efficiency technologies. ieee international conference on industrial engineering and engineering management, 55-59. doi: https://doi.org/10.1109/ieem.2007.4419150 http://dx.doi.org/10.12988/ams.2013.310575 https://doi.org/10.1016/j.asoc.2016.12.040 https://doi.org/10.18657/yonveek.335225 https://doi.org/10.1080/18756891.2011.9727809 https://doi.org/10.1016/s2212-5671(14)00332-3 https://doi.org/10.1016/s2212-5671(14)00332-3 https://doi.org/10.13033/ijahp.v4i2.130 https://doi.org/10.1142/s0219622019500044 https://doi.org/10.1109/ieem.2007.4419150 ijahp article: vardari, canhasi-kasemi /comparison of two different judgement scales with the ahp gsm operator preference of university students international journal of the analytic hierarchy process 16 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.970 meesariganda, b., & ishizaka, a. (2017). mapping verbal ahp scale to numerical scale for cloud computing strategy selection. applied soft computing, 53, 111-118. doi: https://doi.org/10.1016/j.asoc.2016.12.040 pöyhönen, m., hämäläinen, r., & salo, a. (1997). an experiment on the numerical modeling of verbal ratio statements. journal of multi‐criteria decision analysis, 1(6), 110. doi: https://doi.org/10.1002/(sici)1099-1360(199701)6:1%3c1::aidmcda111%3e3.0.co;2-w saaty, t. (1982). the analytic hierarchy process: planning, priority setting, resource allocation. mcgraw-hill. saaty, t. (1996). decision making with dependence and feedback: the analytic network process (vol. 4922). pittsburgh: rws publications. saaty, t. (2005). theory and applications of the analytic network process: decision making with benefits, opportunities, costs, and risks. pittsburgh: rws publications. saaty, t. & peniwati, k. (2013). group decision-making: drawing out and reconciling differences. pittsburgh: rws publications. saka, m., cetin, o. comparing two judgment scales of ahp with a case study: reaching a decision on a dry port location. wmu journal of maritime affairs, 19, 427–461 (2020). doi: https://doi.org/10.1007/s13437-020-00218-8 salo, a., & hämäläinen, p. (1997). on the measurement of preferences in the analytic hierarchy process. journal of multi‐criteria decision analysis, 6(6), 309-319. doi: https://doi.org/10.1002/(sici)1099-1360(199711)6:6%3c309::aid-mcda163%3e3.0.co;22 srđević, z., & srđević, b. (2003). standard and balanced scale in ahp evaluation of walnut selections and cultivars. letopis naučnih radova poljoprivrednog fakulteta, 27(1), 24-34. wang, y., & elhag, t. (2008). an integrated ahp–dea methodology for bridge risk assessment. computers & industrial engineering, 54(3), 513-525. doi: https://doi.org/10.1016/j.cie.2007.09.002 yıldırım, b. (2019). a new pairwise comparison scale for analytic hierarchy process. [master's thesis, middle east technical university]. yıldız, ö. (2019). determining the factors affecting consumers' preference of the gsm operator; case of agri. [master’s thesis, ağrı i̇brahim çeçen üniversitesi, sosyal bilimler enstitüsü]. doi: https://doi.org/10.31463/aicusbed.981957 https://doi.org/10.1016/j.asoc.2016.12.040 https://doi.org/10.1002/(sici)1099-1360(199701)6:1%3c1::aid-mcda111%3e3.0.co;2-w https://doi.org/10.1002/(sici)1099-1360(199701)6:1%3c1::aid-mcda111%3e3.0.co;2-w doi:%20https://doi.org/10.1007/s13437-020-00218-8 https://doi.org/10.1016/j.cie.2007.09.002 ijahp news and events: ferretti/saaty’s recent visit to china international journal of the analytic hierarchy process 550 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.453 saaty’s recent visit to china valentina ferretti department of management london school of economics and political science the journal is pleased to share some highlights about a recent research visit that tom saaty, the father of the ahp and anp, made to china in october 2016. this was a long-promised trip which became a great triumphal return to the country that held the first international symposium on the ahp in 1988. this first conference was held at tianjin university, in tianjin, the third largest city in china. professor saaty was received with honor, visiting old friends who attended the first conference as students and who have now achieved positions of prominence. this autumn, the host of tom saaty’s visit was professor yong shi of the chinese academy of sciences, a longtime colleague, a friend from the multi-criteria decision making community and a member of the scientific advisory group which advises the president of china, xi jinping, on science and technology. tom saaty, wife rozann and professor yong shi of the chinese academy of sciences ijahp news and events: ferretti/saaty’s recent visit to china international journal of the analytic hierarchy process 551 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.453 during this research visit (from october 19 to november 3), tom saaty gave several lectures, inspiring students as well as colleagues from several chinese universities. we have listed the most relevant ones below:  lecture on creativity at the research center on fictitious economy & data science of the chinese academy of sciences;  lectures and appointment as adjunct professor from october 2016 to september 2018 at the school of economics and management, university of chinese academy of sciences;  appointment as guest professor of college of management and economics of tianjin university (2016-2021) in recognition of this university being the location of the first isahp meeting in 1988;  central university of finance and economics;  north china university of technology;  lecture and appointment of a medal of honor at the beijing jiaotong university;  lecture and appointment as part-time professor for 2016 at the university of science and technology, beijing. top left: tom saaty giving the solution to orienting four cubes so all four colors appear on each side during his creativity lecture at the chinese academy of science. top right: tom saaty lecturing in beijing. bottom: students paying close attention during his lecture students attending chinese universities study ahp as part of the core curriculum in management studies and were enthusiastic and touched by the possibility of attending tom’s lecture. ijahp news and events: ferretti/saaty’s recent visit to china international journal of the analytic hierarchy process 552 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.453 professor jennifer shang of the university of pittsburgh’s graduate school of business offered her thoughts on why the chinese are enthusiastic about the ahp. she said, “the culture in china makes people prefer to take a holistic approach to solve problems. namely, they like a 360-degree view of any problem with everyone offering their opinions, with an aim of synthesizing the different views to arrive at a consensus at the end, very similar to the ahp approach”. tom saaty has hosted many visiting scholars from china at the university of pittsburgh where he is a distinguished university professor, and these scholars gathered in beijing to host a dinner for him. some came from as far off as chengdu province and shanghai for the mongolian-style dinner. we all wish many other successful and inspiring research visits to rozann and tom together! rozann and tom saaty making light with statues at tsinghua university ijahp essay: t. saaty/ getting priorities from a crowd international journal of the analytic hierarchy process 161 vol. 4 issue 2 2012 issn 1936-6744 getting priorities from a crowd: combining judgments from people with differing perspectives thomas l. saaty university of pittsburgh katz graduate school of business pittsburgh, pa 15230 saaty@katz.pitt.edu as people examine and analyze complexity from different points of view, it is usual that increased perspectives eventually lead to confusion and disagreements – a standoff. getting a jury to come to a decision is a good example of what we need to analyze for agreement among the members of a group. the object in choosing a jury is that its members should agree enough to reach a verdict. it is desirable that the different points of view be reconciled enough to arrive at a decision through agreement. however, agreement needs consistency among the jurors. a jury that cannot reach a verdict because of insufficient understanding is a failure because it has wasted time and resources. however, consistency of viewpoint is not the only concern in using a jury. we need validity in their verdict that conforms to the best interpretation of the facts of the case. but the larger the number on a jury or of any group seeking agreement, the greater is the likelihood of a diversity of points of view and of disagreement and therefore of inconsistency. in the ahp we use the geometric mean in a cooperative decision making situation to combine the peoples’ judgments on each pairwise comparison. if the people have different ability and expertise, we can prioritize them and use these priorities as the exponents of their numerical judgment and then take the geometric mean (in that case simply the product) of their judgments. but what if the individuals do not work together cooperatively? what should one do? in practice if it is known that the judges do not wish to work together, each judge should make his or her pairwise comparison judgments throughout the entire structure, and arrive at the final answer in the form of an overall priority vector for the alternatives of the decision. suppose there are n judges. their final priority vectors are combined by multiplying the corresponding elements of the priority vectors together and taking the nth root. in this case each judge has the same expertise or importance and all have the same priority: 1/n, hence the nth root after multiplication. this is the same thing as taking the geometric mean of corresponding priorities. mailto:saaty@katz.pitt.edu ijahp essay: t. saaty/ getting priorities from a crowd international journal of the analytic hierarchy process 162 vol. 4 issue 2 2012 issn 1936-6744 if the judges have their own priorities according to expertise, as we do with their individual judgments mentioned before, we raise the corresponding elements of their respective final priority vectors each to the priority of the judge (their priority represents their power), then multiply them together to get the corresponding elements of the combined final vector. normalize the combined vector if it does not sum to 1. this is also the way to deal with judgments of a group with different backgrounds by appropriately clustering them and combining their final outcomes. in this case the groups are prioritized appropriately according to their knowledge and understanding or even according to the relative number in each group. rob typewritten text http://dx.doi.org/10.13033/ijahp.v4i2.160 getting priorities from a crowd: combining judgments from people with differing perspectives ijahp article: bhandari, nakarmi/a financial performance evaluation of commercial banks in nepal using ahp model international journal of the analytic hierarchy process 318 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.368 a financial performance evaluation of commercial banks in nepal using ahp model ashish bhandari institute of engineering,tribhuvan university,nepal aashishbhandari@gmail.com amrit nakarmi institute of engineering, tribhuvan university,nepal amritnakarmi@ioe.edu.np abstract the inference of this study lies mainly in identifying and prioritizing financial parameters based on which financial institutions are considered efficient. this study attempts to find out the current performance of commercial banks in nepal and develop a multi criteria model to check the health and status of these banks. this research will explore the key performance indicators based on the analytic hierarchy process and identify how these are affecting a bank’s performance in nepal. the study reveals the financial positions of three public and ten private commercial banks. the data are mainly obtained from the central bank of nepal annual audited financial statements of commercial banks (published by the respective banks), and a yearly economic survey. an average of four year ratios from 2008/09 to 2011/12 was evaluated to assess the financial performance of the commercial banks. thirteen commercial banks were selected for the analysis in this study. the financial ratios used to assess bank performance were taken based on the analytic hierarchy process framework in which hierarchical criteria were determined based on cael (capital adequacy, asset quality, efficiency and liquidity. keywords: ahp; bank performance; nepal 1. introduction banking is a major financial institutional system in nepal, which accounts for more than 70% of the total assets of all the financial institutions. the establishment of commercial banks is growing rapidly in the nepalese contemporary situation. this has consequently generated high flows of money in the market, but has also led to massive investments. on the basis of ownership, the commercial banks in nepal can be categorized into two groups, public and private banks. as of mid-july 2012 there were 3 public banks and 29 private sector banks. rastriya banijya bank limited, a public bank, is the largest bank in terms of deposit mobilization in government ownership. the government of nepal owns a 40.49% share in nepal bank limited, another public bank. likewise, the government of nepal now owns 53.5% of the shares of agriculture development bank limited (adbl). private banks in nepal can be further re-grouped into local private banks and foreign joint-venture banks. banks with local private investment are local private banks mailto:aashishbhandari@gmail.com ijahp article: bhandari, nakarmi/a financial performance evaluation of commercial banks in nepal using ahp model international journal of the analytic hierarchy process 319 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.368 while banks that have joint investments with foreign financial institutions and local private investors are the joint-venture banks. as of mid-july 2012, there were seven private joint-venture banks, and 22 locally owned banks. 2. literature review 2.1 performance of banks in nepal the growth of major balance sheet indicators of banks in nepal (figure 1) shows that a major dip in their liquidity position in 2010/11 in nepal was due to poor investment decisions by commercial banks. according to jha & hui (2012) capital risk (car), liquidity risk (cdr), and profitable ratios (roa and roe) have influenced efficiencies, however credit risk (npl) reduces the levels of efficiency of banks in nepal. all the banks, except two public sector banks, namely nepal bank limited (nbl) and rastriya banijya bank limited (rbbl), have met the minimum regulatory capital. similarly, the asset quality of the banking industry also remained well below the red line (5 percent) in the review year. capital funds of banks have significantly improved from 2005 to 2012 (nepal rastra bank, 2012). the liquidity positions of commercial banks have also significantly improved by 59.9% over the previous year 2011. furthermore, nepal rastra bank (2012) states that the financial health of public banks was very poor and thus a reform program was initiated in two banks (except adbl). normally, the financial performance of commercial banks and other financial institutions in the world has been measured using a combination of financial ratios analysis, benchmarking, measuring performance against budget or a mix of these methodologies. the overall performance of the banking industry in the subsequent fy 2011/12 was satisfactory, with total assets of the banks increasing by 23.04 percent, in comparison to the growth of 10.17% in the previous year, the capital adequacy position of the commercial banks also improved significantly (sapkota, 2012). the financial health of joint-venture banks in the camel framework concluded that the health of joint-venture banks is better than that of the other commercial banks (baral, 2005). researchers have tested the structure performance hypotheses in the context of the nepalese banking industry for the period of 2001-2009 under the berger and hannan empirical framework (gajurel & pradhan, 2011). other researchers have developed a performance model using a frontier approach known as data envelopment analysis (dea) for the periods 200708 to 2010-11 to measure the relative efficiency and potential improvement capabilities of nepali banks by scrutinizing intermediation aspects (thagunna & poudel, 2013). ijahp article: bhandari, nakarmi/a financial performance evaluation of commercial banks in nepal using ahp model international journal of the analytic hierarchy process 320 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.368 figure 1. growth of major balance sheet indicators of commercial banks in nepal (nepal rastra bank. (2012). bank supervision report) 2.2 bank performance evaluation the traditional performance rankings of banks is based on simple and consistent factors such as financial returns, returns on asset (roa) and returns on earning (roe) and a mix of multiple linear regression techniques for bank performance have also been used (almazari, 2011; bakar & tahir, 2009; naceur, 2003). nevertheless, performance rankings conducted using roa may not precisely describe institutions that embrace strategies for sustaining top performance. off-site tools like financial ratios from periodic balance sheets and income statements and econometric models (information from financial ratios) have been used (gilbert, meyer, & vaughan, 2002). there have been bank performance evaluations where a function of multiple factors such as quality of assets, liquidity position, capital base, management quality, market sensitivity and earnings have been taken into account (saunders & cornett, 2004; dang, 2011). it is usual to measure the performance of banks using financial ratios such as profits, liquidity, asset quality, attitude towards risk, and management strategies and risk and solvency to assess banks efficiency (wirnkar & tanko, 2008; samad & hassan, 2000). the choice of explanatory variables for bank failures is guided by measures of financial fragility arising from the banking industry classified as liquidity, credit risk, profitability and taxes, size and growth, loan mix, and securities (vilén, 2010). yalçın, ali, & cengiz (2009) identify several criteria such as capital adequacy, assets quality, liquidity, profitability, income expenditure structure, group share and sectoral share. bank performance evaluation encompasses econometric models which gather information from financial ratios from ijahp article: bhandari, nakarmi/a financial performance evaluation of commercial banks in nepal using ahp model international journal of the analytic hierarchy process 321 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.368 periodic balance sheets, and income statements which play an important role in off-site surveillance (gilbert, meyer, & vaughan, 2002). in addition there are internationally adopted practices like advanced risk response operating framework (arrow), and financial inspection rating system (first) which assess risk in banks by looking into banks policy development and implementation, internal control, risk management, accounting policy, account opening and anti-money laundry policies, legal compliance, customer protection management, comprehensive risk management, capital management, credit risk management, asset assessment management, market risk management, liquidity risk management, and operational risk management. it is the financial ratios analysis which can spot better investment options for investors, and this can be dealt with as a multi criteria problem in bank performance evaluation (steuer & paul, 2003). nepal’s central bank, nepal rastra bank (nrb)’s banking supervision department (bsd) uses camels (capital adequacy, asset quality, management quality, earnings, liquidity, and sensitivity to market risk) as the six factors to evaluate banks on a scale from 1 (robust) to 5 (very unstable). 2.3 developing an ahp based model for bank performance evaluation though the central bank in nepal, nepal rastra bank (nrb), is closely monitoring commercial banks through offsite and onsite surveillances, there is a need to use appropriate decision tools to identify and mitigate problems in banks. some researchers convey the imperativeness of the ability of integrated/holistic decision analysis, putting subjective and objective information into a single framework for decision analysis in financial institutions in nepal (bhattarai & shivjee, 2009). the research also states that the application of ahp specifically in the banking and finance sector is less than 3% percent of the total application and that it has become more intense in academic research only after 2000. this research forms the basis of developing an ahp model for bank performance evaluation in nepal. there is a growing need of ahp-based decision support systems in the banking sector. it is also important to note that the financial ratios of banks can be compared without any model. an expert can make comparisons of financial ratios of two or more banks and come up with valuable conclusions. this is because financial ratios are absolute values and can be interpreted by experts. the problem arises when someone wants to compare banks according to more than a few financial ratios. it is easy to conclude which bank is better or best according to one financial ratio, but it is slightly more difficult to determine which bank is better or best in certain business segments or in general. the problem is considerably more complex when someone needs to compare several banks according to their businesses. for such complex problems a model has to be developed in order to measure key performance indicators. the application of ahp in finance emerged mostly after 1990. specific applications of ahp in banking, like credit rating, risk assessments in banking and investment are more apparent after 2000. early ahp approaches for performance evaluation of banks have taken multi criteria like equity capital, capital/assets ratio, profit/income ratio, eva (economic value added), organization efficiency, and value added intellectual capital (vaic) (babić, belak, & tomić-plazibat, 1999). conventional credit rating and bank evaluation tools like camel and basel ii have more recently been combined with ahp, and were found to give more insights on risk ijahp article: bhandari, nakarmi/a financial performance evaluation of commercial banks in nepal using ahp model international journal of the analytic hierarchy process 322 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.368 assessments when such combined applications were made. ahp is one of the most widely used multiple criteria decision making tools. many studies have been done based on ahp, including applications of ahp in different fields such as planning, selecting a best alternative, resource allocations, resolving conflict, optimization, etc. (vaidya & kumar, 2006). in this paper, a model based on the analytic hierarchy process is proposed (saaty, 1980). there are various studies in the literature where the bank performance evaluation has taken ahp into consideration. jabalameli & rasolinezhad (2011) used the combined data envelopment analysis-analytic hierarchy process (dea-ahp) method for performed evaluation and ranking of branches of saderat bank in tehran. lu, wang, & lee’s (2013) method for bank evaluation uses analytic hierarchy process (ahp) to evaluate a bank’s operation risk rating in various stressed scenarios and to prioritize rating items, which would simplify the evaluation process for bank failure prevention. the overall criteria selected were finance, legal compliance, consumer protection and risk management. an integrated ahp/dea model by hunjak & jakovčević (2001) uses financial criteria such as liquidity, efficiency, profitability and asset quality for qualitative rating of banks in croatia. the use of financial ratios divided into four groups including, balance sheet ratios, income statement ratios, profitability ratios and market ratios and several subgroups to analyze banks in croatia forms a basis of this multi criteria decision analysis (čehulić, hunjak, & begičević, 2011). an ahp model to evaluate banks in montenegro has been used by rakocevic & dragasevic (2009), and the financial criteria were liquidity, efficiency, profitability and asset quality. 3. objectives the basic objective of the study is to assess the competency of commercial banks in nepal with respect to their financial performance. the objectives of this study are given below: a. to review ahp based bank assessment literature and contribute to the gap; b. to establish priorities for performance measurement of commercial banks among liquidity, efficiency, profitability, capital adequacy and asset quality indicators; c. to develop and use an ahp based framework to evaluate commercial banks in nepal 4. research design/methodology 4.1 selection of banks for purpose of study the purpose of this study is to evaluate the factors determining the performance of nepalese commercial banks. the data are mainly obtained from the nepal rastra bank bulletin (published by the central bank of nepal), annual audited financial statements of commercial banks (published by the respective banks), and a yearly economic survey. averages of four year ratios from 2008/09 to 2011/12 were evaluated to assess the financial performance of the commercial banks in nepal. thirteen commercial banks were selected for the analysis in this study. the financial ratios used to assess bank performance were taken based on the ahp framework in which hierarchical criteria were ijahp article: bhandari, nakarmi/a financial performance evaluation of commercial banks in nepal using ahp model international journal of the analytic hierarchy process 323 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.368 determined based on cael (capital adequacy, asset quality, efficiency and liquidity). out of the banks selected, 3 were public banks; agriculture development bank limited, nepal bank limited and rastriya banijya bank limited. the following banks were selected from the private list of banks: citizen bank, everest bank ltd, himalayan bank ltd, kist bank, nabil bank ltd, nepal investment bank ltd, nmb bank, prime bank, standard chartered bank ltd and sunrise bank. the selection of major criteria and sub criteria was based on literature findings, nepal rastra bank's key performance indicators for commercial banks, literature reviews, and expert reviews. the ahp model was used to assess the performance of the banks and obtain their normalized rankings. the usual ahp concepts of laying out a hierarchical structure, pairwise comparing to establish priorities of the criteria throughout the structure, then synthesizing to get the priorities of the alternatives are followed. the key financial ratios that were used to develop the model is shown below: goal evaluating performance of commercial banks in nepal liquidity efficiency profitability capital adequacy asset quality l1=liquid as sets to total assets l2=liquid as sets to total deposit l3=to tal c redit/to tal deposit l4=cas h reserve ratio p1=net profit/gross income p2=earnings per share p3=price earnings ratio p4=net profit/loan and advances p5=net profit/to tal assets e1=interest income/loans and advancements e2=staff expenses/total operating expenses e3=interest expenses/total deposit and borrowings e4=total operating expenses/total assets e5=profit per employee c1= core c apital percent c2=supplementary cap ital percent c3=to tal c apital fund a1=non-performing credit/to tal c redit a2=book net-worth a3=networth per share a4=return on equity nbl rbbl citizen kist prime nmb sunrise adbl scbl everest himalay nabil nib criteria subcriteria alternatives figure 2. ahp model ijahp article: bhandari, nakarmi/a financial performance evaluation of commercial banks in nepal using ahp model international journal of the analytic hierarchy process 324 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.368 table 1 criteria and sub criteria for bank performance evaluation criteria sub criteria liquidity l1=liquid assets to total assets l2=liquid assets to total deposit l3=total credit/total deposit l4=cash reserve ratio efficiency e1=interest income/loans and advancements e2=staff expenses/total operating expenses e3=interest expenses/total deposit and borrowings e4=total operating expenses/total assets e5=profit per employee profitability p1=net profit/gross income p2=earnings per share p3=price earnings ratio p4=net profit/loan and advances p5=net profit/total assets capital adequacy c1= core capital percent c2=supplementary capital percent c3=total capital fund asset quality a1=non-performing credit/total credit a2=book net-worth a3=networth per share a4=return on equity 5. data analysis 5.1 financial analysis the mean financial values were calculated from the annual reports of the respective banks and the table below shows the data. ijahp article: bhandari, nakarmi/a financial performance evaluation of commercial banks in nepal using ahp model international journal of the analytic hierarchy process 325 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.368 table 2 calculation of financial parameters l1 l2 l3 l4 e1 e2 e3 e4 e5 p1 p2 p3 p4 p5 c1 c2 c3 a1 a2 a3 a4 sunrise 0.26 0.32 0.80 29.51 9.07 44.54 6.8 1.93 288.07 8.24 7.71 29.18 0.9 0.006 11.48 0.795 12.27 0.038 1747462 107.17 0.08 prime 0.23 0.26 0.83 12.4375 11.78 33.81 7.24 11.105 1146.19 13.69 19.49 12.59 1.8 1.323 11.39 0.918 12.31 0.333 1433153 118.63 0.31 nmb 0.36 0.60 0.80 10.475 9.75 41.01 6.1525 1.29 723.19 11.07 7.19 56.80 1.4 0.820 16.66 0.598 17.52 0.983 1977676 114.57 0.07 kist 0.25 0.30 0.79 7.6675 12.62 37.86 7.505 2.0475 193.88 17.47 4.85 48.36 0.9 0.570 14.45 0.765 15.22 1.723 2117802 105.89 0.05 citizen 0.22 0.26 0.81 12.765 12.83 32.05 7.1275 3.7925 853.96 11.35 14.19 32.80 1.6 1.078 12.39 0.875 13.27 0.873 1737628 109.00 0.11 rbbl 0.31 0.35 0.49 13.07 9.94 42.26 2.4625 4.9725 826.01 26.33 432.68 0.00 4.7 2.163 -19.14 0.000 -19.14 0.109 -8446909 -720.54 1.85 adbl 0.20 0.32 1.07 27.4225 13.96 47.24 5.1425 9.165 615.83 17.65 62.92 1.06 4.4 3.053 13.98 4.273 18.25 9.663 11636570 422.37 0.76 nbl 0.21 0.25 0.75 23.675 13.41 55.44 2.4775 7.395 738.80 8.74 95.19 0.00 1.7 0.748 -10.26 0.000 -10.26 0.016 -1200 -315.47 5.38 nabil 0.21 0.25 0.75 6.3875 11.15 15.48 4.885 5.88 3432.54 25.18 87.87 26.39 3.8 2.538 8.91 1.788 10.70 0.016 4249065 270.75 1.32 nepal investment0.22 0.25 0.80 9.85 11.23 40.35 5.675 1.2 2022.92 20.40 39.15 19.88 2.7 1.875 8.79 2.158 10.95 1.378 4925734 171.00 0.62 scbl 0.37 0.42 0.47 10.855 10.14 23.52 2.1475 3.405 4125.42 33.71 82.44 36.89 6.9 2.653 12.42 1.918 14.34 0.007 3555531 263.25 1.29 himalayan 0.23 0.26 0.75 6.9975 11.56 47.12 4.5 2.20975 1864.29 28.05 44.58 20.83 2.6 1.693 8.99 1.870 10.86 2.998 3843969 219.02 0.70 everest 0.30 0.33 0.75 14.14 10.51 10.43 4.74 5.6975 2344.34 17.51 92.97 16.41 2.9 2.008 8.75 2.143 10.89 0.455 2982825 316.74 1.45 liquidity efficiency profitability capital adequacy asset quality banks it is seen that standard chartered bank has the highest liquid asset to total asset ratio of 0.37, and agriculture development bank has the lowest ratio of 0.20. liquidity was the most important criteria in determining the soundness of banks (31.1%), capital adequacy, asset quality, efficiency and profitability were 21.6%, 18.5%, 14.9% and 13.9% respectively. the ranking of commercial banks was done after prioritization of the financial criteria. it was evident that standard chartered bank was the most efficient and profitable bank with a normalized efficiency and profitability score of 100%. two public sector banks, nepal bank limited and rastriya banijya bank, were ranked in the bottom two in the ranking list. this was particularly due to negative capital adequacy ratios. the average total capital adequacy fund for these banks was –14.87% which is below the minimum capital adequacy requirement (car) of 10% prescribed by nepal rastra bank (nrb). on the contrary adbl bank was performing above average in comparison to the other two public banks. some banks like prime bank and sunrise bank had sound liquidity positions, but these banks have not been maximizing profits which justify their rank. this type of analysis is based on indicators by which central bank nepal rastra bank (nrb) evaluates bank performance where the financial data is compared with the camel benchmark already set. ahp, on the other hand, will first prioritize and rank performance evaluation indicators/criteria because the performance evaluation measures and benchmarks are susceptible to changes in central bank policies. the ahp method will allow experts to first prioritize the indicators and then analyze the already computed financial data to rank banks and also perform a sensitivity analysis. 5.2 performance evaluation of banks by ahp the set of the ahp questionnaire was constructed through expert choice software ver. 11. extensive analysis of the literature and expert views on defining parameters determining performance of banks and bank soundness was determined. the questionnaire prepared was distributed to a group of banking and finance experts, and the corresponding results were synthesized by the ahp method using expert choice software. a total of 15 banking and finance experts from commercial banks, a credit rating agency, a central bank of nepal and chartered accountants were provided the ijahp article: bhandari, nakarmi/a financial performance evaluation of commercial banks in nepal using ahp model international journal of the analytic hierarchy process 326 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.368 questionnaire through email. a total of 13 experts (86.66%) responded. the questionnaire analysis was interpreted through expert choice ver. 11 software. the eigenvalues, consistency indices and consistency ratios obtained from the processing of the information supplied by the participants including the facilitator revealed that the consistency indices (ci) were less than 0.1, indicating consistency in judgmental values of the respondents. to come to the decisions were checked to make sure they were in compliance with the ci. the following tables show the combined pairwise comparison matrices. table 3 pairwise comparison matrix with respect to: liquidity (inconsistency: 0.03): l1 l2 l3 l4 l1 1.0 1.53 0.76 0.80 l2 0.65 1.0 0.90 1.07 l3 1.31 1.12 1.0 1.61 l4 1.25 0.94 0.62 1.0 table 4 pairwise comparison matrix with respect to: efficiency (inconsistency: 0.10) e1 e2 e3 e4 e5 e1 1.0 5.3 2.16 2.5 1.4 e2 0.19 1.0 0.37 0.7 2.0 e3 0.46 2.7 1.0 1.0 3.4 e4 0.40 1.47 1.01 1.0 3.2 e5 0.72 0.51 0.30 0.3 1.0 table 5 pairwise comparison matrix with respect to: profitability (inconsistency: 0.01) p1 p2 p3 p4 p5 p1 1.0 0.8 1.36 1.8 1.12 p2 1.25 1.0 1.84 3.15 1.09 p3 0.74 0.54 1.0 1.11 0.72 p4 0.55 0.32 0.9 1.0 0.70 p5 0.89 0.91 1.39 1.43 1.0 ijahp article: bhandari, nakarmi/a financial performance evaluation of commercial banks in nepal using ahp model international journal of the analytic hierarchy process 327 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.368 table 6 pairwise comparison matrix with respect to: capital adequacy (inconsistency: 0.09) c1 c2 c3 c1 1.0 6.06 0.58 c2 0.16 1.0 0.24 c3 1.71 4.21 1.0 table 7 pairwise comparison matrix with respect to: asset quality (inconsistency: 0.03) a1 a2 a3 a4 a1 1.0 2.9 3.1 3.1 a2 0.35 1.0 0.81 0.5 a3 0.32 1.23 1.0 0.47 a4 0.32 2.02 2.12 1.0 table 8 pairwise comparison matrix with respect to goal ((inconsistency: 0.04) liquidity efficiency profitability car asset quality liquidity 1.0 2.5 1.74 2.06 1.27 efficiency 0.40 1.0 0.91 0.58 1.31 profitability 0.57 1.10 1.0 0.66 0.51 car 0.49 1.72 1.51 1.0 1.54 asset quality 0.79 0.76 1.97 0.65 1.0 based on pairwise comparisons of the financial criteria and sub criteria it was seen that we obtained the priorities of liquidity in banks (w= 0.311), capital adequacy (w=0.216), asset quality (w=0.185), efficiency (w=0.149), and profitability (w=0.139). the overall inconsistency was 0.04 which is acceptable as it is <0.1. further the data grid function in expert choice software was used to assess the relative importance of the financial parameters based on the maximum and minimum value of the financial data. the data grid function determines the importance of the financial parameters modeled. the following table depicts how performance of the commercial banks is determined in terms of parameters classified using increasing and decreasing utility curves to measure their ranking. the data grid assesses the relative importance of the financial parameters based on the maximum and minimum value of the financial data and convert raw data to prioritized information. ijahp article: bhandari, nakarmi/a financial performance evaluation of commercial banks in nepal using ahp model international journal of the analytic hierarchy process 328 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.368 table 9 priorities of financial parameters local weight global weight incr or decr liquidity l: .311 g: .311 l1=liquid assets to total assets l: .245 g: .076 incr l2=liquid assets to total deposit l: .221 g: .069 incr l3=total credit/total deposit l: .305 g: .095 decr l4=crr l: .229 g: .071 incr efficiency l: .149 g: .149 e1=interest income/loans and advancements l: .375 g: .056 incr e2=staff expenses/total operating expenses l: .108 g: .016 decr e3=interest expenses/total deposit and borrowings l: .223 g: .033 decr e4=total operating expenses/total assets l: .192 g: .029 decr e5=profit per employee l: .101 g: .015 incr profitability l: .139 g: .139 p1=net profit/gross income l: .223 g: .031 incr p2=earning per share l: .291 g: .041 incr p3=price earning ratio l: .151 g: .021 incr p4=net profit/loan and advances l: .124 g: .017 incr p5=net profit/total assets l: .211 g: .029 incr capital adequacy l: .216 g: .216 c1= core capital percent l: .401 g: .087 incr c2=supplementary capital percent l: .090 g: .019 incr c3=total capital fund l: .509 g: .110 incr asset quality l: .185 g: .185 a1=non-performing credit/total credit l: .497 g: .092 decr a2=book net-worth l: .130 g: .024 incr a3=net worth per share l: .140 g: .026 incr a4=return on equity l: .233 g: .043 incr ijahp article: bhandari, nakarmi/a financial performance evaluation of commercial banks in nepal using ahp model international journal of the analytic hierarchy process 329 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.368 5.3 ranking of commercial banks the ranking of commercial banks was done after prioritization of the financial criteria. aggregation of the results from individual comparison of the financial criteria and sub criteria and assessing the financial data using data grid functions were done to finally achieve holistic ranking of commercial banks. the results suggest that standard chartered bank is the most efficient bank whereas rastriya banjiya bank and nepal bank limited are the least efficient banks. agriculture development bank on the contrary shows above average efficiency. the normalized ranking of the commercial banks is shown in the figure below. figure 3. normalized ranking of commercial banks 6. sensitivity analysis the sensitivity analysis is where the input data are slightly modified in order to observe the impact on the results. if the ranking does not change the results are said to be robust. the sensitivity analysis is best performed with an interactive graphical interface. the sensitivity analysis for the least efficient banks, rastriya banijya banks and nepal bank limited, has been performed. the relative ranking of rastriya banijya bank can be improved if the capital adequacy factor is improved as the sensitivity analysis. figure 4 shows that the performance of these banks is highly dependent on capital adequacy. these two banks must improve the capital adequacy. standard chartered bank is seen as the most efficient bank and has a strong position in terms of all the financial parameters defined for this study. liquidity holds the maximum priority followed by capital adequacy according to the sensitivity graph shown. ijahp article: bhandari, nakarmi/a financial performance evaluation of commercial banks in nepal using ahp model international journal of the analytic hierarchy process 330 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.368 figure 4. sensitivity graph 7. limitations there are some limitations which narrowed the generalization of this study, e.g., inadequate coverage of the industry, the time period taken, availability of data and other variations. the study is perhaps limited by following factors: 1) this study is limited to a period of only a 4 year trend of the concerned banks, and hence the conclusion drawn is confined only to the above period. 2) this study deals with only 13 commercial banks; other commercial banks have not been considered. 3) this study is based on data derived from the published annual reports of the thirteen banks. 4) this study does not consider management efficiency, market risks and customer satisfaction in evaluating bank performance. 8. conclusions and recommendations the ranking of commercial banks was done after prioritizing the financial criteria. two public sector banks, nepal bank limited and rastriya banijya bank, were ranked at the bottom in the ranking list. this was particularly due to negative capital adequacy ratios. the average total capital adequacy fund for these banks was –14.87% which is below the minimum capital adequacy requirement (car) of 10% prescribed by nepal rastra bank (nrb). on the contrary adbl bank was performing above average in comparison to the other two public banks. some banks such as prime bank and sunrise bank had sound liquidity positions, but these banks have not been maximizing profits to justify their risk. the commercial banks in nepal, and in particular the public sector bank’s lower (negative) capital adequacy values, are associated with lower performance. these banks need to strengthen their capital adequacy ratios above 10% (out of which 6% is core capital percentage). the main reason for the banks not being able to maintain minimum capital is due to the increase in non-performing loans, as such an increase in non-performing loans could decrease the profit. the banks with lower non-performing loans will have to focus on risk management and should act to recover their bad debts. ijahp article: bhandari, nakarmi/a financial performance evaluation of commercial banks in nepal using ahp model international journal of the analytic hierarchy process 331 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.368 deposit organizations such as banks that show a lower than regulatory cash reserve ratio in their annual accounts might lead to depositor mistrust of the bank. it is not mandatory for banks to maintain a minimum percentage of their deposits as investments in government securities in nepal, therefore the banks have to maintain cash reserve ratio as their mandatory liquidity factor. the results of this study show that a multi criteria decision based approach such as the analytic hierarchy process can be used as a supplementary decision support tool to the camels rating system in the bank examination process. the analytic hierarchy process based systems approach explicitly explores the financial characteristics of the banking system and compares the banks with respect to these characteristics, thus, determining differences in the financial structures and positions of the banks. this study also adds to the literature that demonstrates the utility of ahp-based bank evaluations applied to the nepalese banking community, which not only evaluates the performance of banks but also gives insights about where to focus in improving a particular bank in comparison to others. the ability of the dynamic sensitivity analysis feature available with the ahp processing software further helps to overcome the accuracy of the data available from the individual banks, which could be of added value to bank regulators and more comprehensive multi criteria decision analysis methods including sensitivity components as well as qualitative criteria like management efficiency, customer satisfaction and legal compliance which could be incorporated in future research. ijahp article: bhandari, nakarmi/a financial performance evaluation of commercial banks in nepal using ahp model international journal of the analytic hierarchy process 332 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.368 references almazari, ahmed arif (2011). financial performance evaluation of some selected jordanian commercial banks. international research journal of finance and economics 68, 50-63. babić, z, belak, v and tomić-plazibat, n. (1999) ranking of croatian banks according to business efficiency. proceedings of the 5th international symposium on operational research. preddvor, slovenia. bakar, n and tahir, i m. (2009). applying multiple linear regression and neural network to predict bank performance. international journal of business research and management 2 (4), pp. 176-183. doi: http://dx.doi.org/10/5539/ibr.v2n4p176 baral, k. j. (2005). health check-up of commercial banks in the framework of camel: a case study of joint venture banks in nepal. the journal of nepalese business studies 2(1), 41-55. doi: http://dx.doi.org/10.3126/jnbs.v2i1.55 bhattarai, shashi and shivjee, yadav roy (2009). ahp application in banking: unfolding utility in a situation of financial crisis. proceedings of international symposium on analytic hierarchy process, pittsburgh, pa, usa. čehulić, dejan, hunjak, tihomir and begičević, nina. croatia (2011). comparison of a bank's financial ratios using the analytic hierarchy process. proceedings of the 22nd central european conference on information and intelligent systems, 187-193. dang, uyen (2011). the camel rating system in banking supervision a case study. thesis paper for arcada university of applied sciences, international business. gajurel, d. p., & pradhan, r. s. (2011). structure-performance relation in nepalese banking industry. international conference on economics business . doi: http://dx.doi.org/10.2139/ssrn.1665132 gilbert, r, meyer, a, & vaughan, m. (2002). could a camels downgrade model improve off-site surveillance? federal reserve bank of st. louis review, 47-63. hunjak, t., & jakovčević, d. (2001). ahp based model for bank performance. proceedings – 6th isahp, berne, switzerland, 149-158. jabalameli, f., & rasolinezhad, e. (2011). the ranking of chosen branches of saderat bank in tehran city through the ahp and the dea/ahp model. proceedings of the international symposium on the analytic hierarchy process, sorrento, italy. jha, s., & hui, x. (2012). a comparison of financial performance of commercial banks: a case study of nepal. african journal of business management, 6(25) , 7601-7611. doi: 10.5897/ajbm11.3073 http://dx.doi.org/10.3126/jnbs.v2i1.55 http://dx.doi.org/10.2139/ssrn.1665132 ijahp article: bhandari, nakarmi/a financial performance evaluation of commercial banks in nepal using ahp model international journal of the analytic hierarchy process 333 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.368 lu, t.-l., wang, y.-c., & lee, y.-c. (2013). a study on risk based banking supervision. business and information, 164-172. naceur, s. (2003). the determinants of the tunisian banking industry. [online] 2003. http://www.mafhoum.com/press6/174e11. nepal rastra bank. (2012). bank supervision report.: bank supervision department. doi: 10.4324/9781843140115 rakocevic, s., & dragasevic, z. (2009). analysis of the efficiency of banks in montenegro using the ahp. isahp, pittsburgh, pa, usa. saaty, t.l. (1980). the analytic hierarchy process: planning, priority setting, resource allocation, mcgraw-hill, inc. sapkota, m. (2012). trend analysis of nepalese banks from 2005-2010. university of new orleans scholar worls @ uno. samad, a., & hassan, m. k. (2000). the performance of malaysian islamic bank during 1984-1997: an exploratory study. international journal of islamic financial services 1(3) , 1-14. saunders, anthony and cornett, millon marcia. (2004). financial markets and institutions. tata mcgraw hill publishing company. steuer, r. e., & paul, n. a. (2003). multiple criteria decision making combined with finance: a categorized bibliographic study. european journal of operational research 150 (3) , 496-515. doi: http://dx.doi.org/10.1016/s0377-2217(02)00774-9 thagunna, k. s., & poudel, s. (2013). measuring bank performance of nepali banks:a data envelopment analysis (dea) perspective. international journal of economics and financial issues, 3(1) , 54-65. vaidya, o. s., & kumar, s. (2006). analytic hierarchy process: an overview of applications. european journal of operational research, 169, 1-29. http://dx.doi.org/10.1016/j.ejor.2004.04.028 vilén, m. (2010). predicting failures of large u.s. commercial banks. master’s thesis for aalto univeristy school of economics, department of economics. dzeawuni, wirnkar alphonsius and muhammad, tanko, camels and banks performance evaluation: the way forward (june 24, 2008). doi: http://dx.doi.org/10.2139/ssrn.1150968 yalçın, s. n., ali, b., & cengiz, k. (2009). fuzzy performance evaluation in turkish banking sector using analytic hierarchy process and topsis. expert systems with applications, 36 , 11699-11709. doi: http://dx.doi.org/10.1016/j.eswa.2009.03.013 http://dx.doi.org/10.1016/s0377-2217(02)00774-9 http://dx.doi.org/10.1016/j.eswa.2009.03.013 ijahp article: patel, mjema godwin/ linear programming models for estimating weights in analytic hierarchy process and for optimization of human resource allocation international journal of the analytic hierarchy process 299 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.335 linear programming models for estimating weights in analytic hierarchy process and for optimization of human resource allocation gokulananda patel professor of decision science, birla institute of management technology greater noida india gn.patel@bimtech.ac.in godwin mjema institute of finance management dar es salam, tanzania gdmjema@gmail.com k m godwin institute of finance management dar es salam, tanzania casymj30@gmail.com abstract the analytic hierarchy process (ahp) provides a way to rank the alternatives by deriving priorities. in this paper we used linear programming (lp) models to estimate the weights of a pairwise comparison matrix derived within the frame work of the analytic hierarchy process. the priorities obtained for the alternatives served as the coefficients of the objective function of linear programming to optimize a human resource problem at bakhresa food product limited (bfpl). keywords: ahp; linear programming; resource allocation; element dominance; sensitivity analysis 1. introduction the success of any organization lies in its ability to make critical decisions on growth and sustainability. however, decision making is a complex process as it involves multiple stakeholders with different opinions and interests. to avoid making ad-hoc decisions, decision makers are required to evaluate every alternative to the problem. with the analytical hierarchy process, the problem is modeled by the decision maker and is structurally decomposed into a hierarchy consisting of levels of criteria, sub-criteria and alternatives with homogeneous clusters of factors (saaty, 1977, 2007). subsequently, an assessment of the usefulness of elements at each hierarchical level is made. ahp is a very suitable multi-criteria decision making tool proving to be effective in different application areas such as planning, optimization, selection of the best alternative and allocation of resources. it is also a reliable tool in resolving conflicts (saaty& vargas, mailto:gn.patel@bimtech.ac.in mailto:gdmjema@gmail.com mailto:casymj30@gmail.com ijahp article: patel, mjema godwin/ linear programming models for estimating weights in analytic hierarchy process and for optimization of human resource allocation international journal of the analytic hierarchy process 300 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.335 2006). ahp is critical in defining decision making processes taking into consideration decision maker’s input, judgments, views and feelings (vaidya & kumar, 2006). specifying the hierarchy is of crucial importance; the hierarchical structure gives a clear overview of the complex relationships existing in the problem. this is important because it enables the decision maker to take into consideration every aspect in each level of the hierarchy. it also allows a decision maker to take into consideration a set of evaluation criteria, and alternative options from among which the best selection is to be made. the focus of the problem, usually the goal, is the highest level of the hierarchy. there are subsequent levels of criteria further down that include sub criteria, and finally the level of alternatives from which decisions are generated. elements with a global composition may be included along the top levels of the hierarchy. in each hierarchical level division, a pairwise comparison matrix is developed with 𝑛 (𝑛 − 1) 2⁄ number of comparisons, where ‘n’ is the number of criteria or alternatives in each level (garg, rahman, qureshi & kumar, 2012). using a fundamental scale developed by saaty, decision makers are able to assign the corresponding importance of one criterion relative to the other (saaty, 1977). finally, the weights of the elements being compared are estimated. in the end all the pairwise comparison results are synthesized and the decision is made in accordance with the final overall ranking of the alternatives. (saaty,1999). in summary, ahp applications in decision making involve four main procedures. these procedures include: 1) decomposition of the problem, 2) making judgments in the pairwise comparison matrices and checking their inconsistency, 3) improving it to derive the priority weights, and 4) the synthesis step where the final global weights for all the elements in the model are determined. with each comparison matrix, the decision maker commonly uses the eigenvector method (em) or additive normalization (an) or logarithmic least square method (lls) to generate a priority vector. these methods give the estimated relative weights of the elements as a result of the judgments (saaty, 1999). to produce the final weight for the alternatives, weights generated at different levels of the hierarchy are synthesized according to the principle of hierarchic structure (saaty, 1980) in the next section, an, em, lls, and lp methods are explained. the relative weights in a problem have been estimated using all these methods. in the subsequent section, the necessity of using ahp and lp models in human resource selection is covered. 2. estimating weights let 𝐴 = (𝑎𝑖𝑗 )for all 𝑖, 𝑗 = 1,2, … , 𝑛 denote an 𝑛 𝑥 𝑛 pairwise comparison matrix, where 𝑎𝑖𝑗 is the importance of element 𝑖 over the j th element. all the entries in matrix 𝐴 are positive (𝑎𝑖𝑗 > 0) and reciprocal (𝑎𝑖𝑗 = 1/𝑎𝑗𝑖 for all 𝑖, 𝑗 = 1,2, … . 𝑛) . the decision maker wants to compute a vector 𝑤 = (𝑤1,𝑤2, … . 𝑤𝑛 ) of weights associated to pairwise comparison matrix 𝐴 (saaty, 1999). ijahp article: patel, mjema godwin/ linear programming models for estimating weights in analytic hierarchy process and for optimization of human resource allocation international journal of the analytic hierarchy process 301 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.335 the matrix 𝐴 is considered to be consistent when 𝑎𝑖𝑗 = 𝑎𝑖𝑘 𝑎𝑘𝑗 𝑓𝑜𝑟𝑎𝑙𝑙 𝑖, 𝑗, 𝑘 = 1,2, … , 𝑛, which implies that the decision maker is coherent (no error) in his judgments to develop the comparison matrix (saaty, 1999). assuming a contains no error and 𝑤𝑖is the weight of the𝑖 𝑡ℎ element, we then have 𝑎𝑖𝑗 = 𝑤𝑖 𝑤𝑗 , 𝑖, 𝑗 = 1,2, … , 𝑛 (1) summing over all j, we obtain ∑ 𝑎𝑖𝑗 𝑛 𝑗=1 𝑤𝑗 = 𝑛𝑤𝑖 , 𝑖 = 1,2, … , 𝑛 (2) which, in matrix notation, is equivalent to 𝐴𝑤 = 𝑛𝑤. (3) the vector w is the principal eigenvector of the matrix 𝐴 corresponding to the eigenvalue n., alternatively, we can say that the matrix 𝐴 is consistent when 𝐴𝑤 = 𝑛𝑤 (garg, rahman, qureshi & kumar, 2012). 2.1 additive normalization (an) in obtaining the priority vector 𝑤 using the an method, columns are first normalized such that elements of each column of the matrix 𝐴 is divided by the sum of that column; then in each resulting row, normalized elements are summed up and divided by the number of elements in each row which is arithmetic average of the row (srdjevic, 2005). the following equations (4) to (6) describe the above process; 𝑎𝑖𝑗 ′ = 𝑎𝑖𝑗 ∑ 𝑎𝑖𝑗 𝑛 𝑖=1 ⁄ 𝑖, 𝑗 = 1,2, … 𝑛 (4) 𝑤𝑖 = ( 1 𝑛 ) ∑ 𝑎𝑖𝑗 ′ 𝑛 𝑗=1 , 𝑖 = 1,2, … . , 𝑛 (5) it can be observed that ∑ 𝑤𝑖 = 1 𝑛 𝑖=1 (6) if a is consistent, then the columns of the normalized matrix 𝑁 = (𝑎𝑖𝑗 ′ ) of a are identical. if a is not consistent, then we can write 𝐴𝑤 = 𝜆𝑚𝑎𝑥 𝑤, where𝜆𝑚𝑎𝑥 is the principal eigenvalue and given by ijahp article: patel, mjema godwin/ linear programming models for estimating weights in analytic hierarchy process and for optimization of human resource allocation international journal of the analytic hierarchy process 302 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.335 𝜆𝑚𝑎𝑥 = ∑ ∑ 𝑎𝑖𝑗 𝑤𝑗 𝑛 𝑗=1 𝑛 𝑖=1 (7) according to saaty (1980) the consistency of the method can be checked as follows: the consistency index (ci) is given by 𝐶𝐼 = (𝜆𝑚𝑎𝑥 − 𝑛) (𝑛 − 1) ⁄ (8) while the consistency ratio (cr) is given by 𝐶𝑅 = 𝐶𝐼 𝑅𝐼⁄ (9) the random index (ri), which depends on the order of the matrix, is the average ci of a large number of randomly generated matrices. a cr of 0.10 or less is considered acceptable. 2.2 eigenvector method (em) the principal eigenvector 𝜆𝑚𝑎𝑥 of 𝐴 is determined by solving the determinant, det(𝐴 − 𝜆𝑚𝑎𝑥 𝐼) = 0 (10) then using the value of 𝜆𝑚𝑎𝑥, the eigenvector 𝑤 = (𝑤1,𝑤2, … . 𝑤𝑛 ) is find out from (𝐴 − 𝜆𝑚𝑎𝑥 𝐼)𝑤 = 0 (11) the consistency of the matrix is checked using equations (8) and (9), in section 2.1. 2.3 logarithmic least square method (lls) this method has also been developed to estimate the vector of weights (srdjevic, 2005). with lls, the weights 𝑤𝑖,𝑓𝑜𝑟 𝑖 = 1, … , 𝑛, are chosen to minimize the objective ∑ ∑[ln 𝑎𝑖𝑗 − (ln 𝑤𝑖 + ln 𝑤𝑗 )] 2 𝑛 𝑗=1 𝑛 𝑖=1 (12) given that 𝑎𝑖𝑗 = 1 𝑎𝑗𝑖⁄ 𝑓𝑜𝑟𝑎𝑙𝑙𝑖, 𝑗 = 1,2, … , 𝑛, (13) the lls is quite simple: 𝑤𝑖,𝑓𝑜𝑟 𝑖 = 1, … , 𝑛 is given by the geometric mean of the row i (saaty, 1980). 2.4 linear programming approach (lp) there are two desirable properties of a pairwise comparison matrix – element dominance (ed) and row dominance (rd). ijahp article: patel, mjema godwin/ linear programming models for estimating weights in analytic hierarchy process and for optimization of human resource allocation international journal of the analytic hierarchy process 303 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.335 ed is said to be preserved if 𝑎𝑖𝑗 > 1 𝑖𝑚𝑝𝑙𝑖𝑒𝑠 𝑤𝑖 ≥ 𝑤𝑗. if 𝑎𝑖𝑗 is exactly equal to 1, then an argument can be made for either 𝑤𝑖 ≥ 𝑤𝑗 𝑜𝑟𝑤𝑗 ≥ 𝑤𝑖 . rd is said to be preserved if 𝑎𝑖𝑘 ≥ 𝑎𝑗𝑘 𝑓𝑜𝑟 𝑎𝑙𝑙 𝑘 𝑎𝑛𝑑𝑎𝑖𝑘 > 𝑎𝑗𝑘 𝑓𝑜𝑟 𝑠𝑜𝑚𝑒 𝑘 𝑖𝑚𝑝𝑙𝑖𝑒𝑠 𝑤𝑖 ≥ 𝑤𝑗 . if the comparison matrix has cardinal inconsistency, that is, 𝑎𝑖𝑗 ≥ 1, 𝑎𝑗𝑘 ≥ 1, 𝑎𝑛𝑑𝑎𝑘𝑖 ≥ 1, then the only feasible solution is 𝑤𝑖 = 𝑤𝑗 = 𝑤𝑘 . such a comparison matrix would be highly inconsistent. em and lls both warrant rd (but not the ed). in the lp approach we can incorporate ed and rd as constraints, which have additional benefit of detecting cardinal inconsistency by ed constraints. the two stage lp approach (chandran, b., et.al, 2005) is described in sub-sections 2.4.1 to 2.4.3. 2.4.1 first stage: lp to establish the consistency bound in general, any estimate of relative preference 𝑎𝑖𝑗 can be written as 𝑤𝑖 𝑤𝑗 = 𝑎𝑖𝑗 𝜀𝑖𝑗 , 𝑖, 𝑗 = 1,2, … , 𝑛 (14) if the decision maker is consistent then 𝜀𝑖𝑗 is equal to 1. defining three transformed decision variables for the model: 𝑥𝑖 = ln(𝑤𝑖 ) , 𝑦𝑖𝑗 = ln(𝜀𝑖𝑗 ) , and𝑧𝑖𝑗 = |𝑦𝑖𝑗 | the first stage lp can be written as: 𝑀𝑖𝑛𝑖𝑚𝑖𝑧𝑒 ∑ ∑ 𝑧𝑖𝑗 𝑛 𝑗=𝑖+1 (15) 𝑛−1 𝑖=1 subject to 𝑥𝑖 − 𝑥𝑗 − 𝑦𝑖𝑗 = ln 𝑎𝑖𝑗 , 𝑖, 𝑗 = 1,2, … , 𝑛; 𝑖 ≠ 𝑗, (16) 𝑧𝑖𝑗 ≥ 𝑦𝑖𝑗 , 𝑖, 𝑗 = 1,2, … , 𝑛; 𝑖 < 𝑗, (17) 𝑧𝑖𝑗 ≥ 𝑦𝑗𝑖 , 𝑖, 𝑗 = 1,2, … , 𝑛; 𝑖 < 𝑗, (18) 𝑥1 = 0 (19) 𝑥𝑖 − 𝑥𝑗 ≥ 0 𝑖, 𝑗 = 1,2, … , 𝑛; 𝑎𝑖𝑗 > 1 (20) 𝑥𝑖 − 𝑥𝑗 ≥ 0 𝑖, 𝑗 = 1,2, … , 𝑛; 𝑎𝑖𝑗 ≥ 𝑎𝑗𝑘 , 𝑓𝑜𝑟𝑎𝑙𝑙𝑘; 𝑎𝑖𝑘 > 𝑎𝑗𝑘 𝑓𝑜𝑟𝑠𝑜𝑚𝑒𝑘 (21) 𝑧𝑖𝑗 ≥ 0 𝑖, 𝑗 = 1,2, … , 𝑛 (22) 𝑥𝑖 , 𝑦𝑖𝑗 , 𝑢𝑛𝑟𝑒𝑠𝑡𝑟𝑖𝑐𝑡𝑒𝑑𝑖, 𝑗 = 1,2, … , 𝑛 (23) the objective function (15) which is ∑ ∑ 𝑧𝑖𝑗 𝑛 𝑗=𝑖+1 𝑛−1 𝑖=1 , minimizes the sum of logarithms of positive errors in natural log space, whereas the constraint (16) is defining the errors. equations (17) and (18) are the degree of over estimation, (19) sets one of the weight 𝑤1 to zero, (20) preserves element dominance and (21) for row dominance. for a perfectly consistent comparison matrix, 𝑧∗ is equal to zero. ijahp article: patel, mjema godwin/ linear programming models for estimating weights in analytic hierarchy process and for optimization of human resource allocation international journal of the analytic hierarchy process 304 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.335 the objective function provides consistency index 𝐶𝐼(𝐿𝑃) = 2𝑧∗ 𝑛(𝑛 − 1) (24) 2.4.2 second stage: lp to generate a priority vector the first stage lp minimizes the product of all errors 𝜀𝑖𝑗, but multiple optimal solutions may exist. in the second stage lp, the solution that minimizes the maximum errors 𝜀𝑖𝑗 is selected. the second stage lp can be presented as: 𝑀𝑖𝑛𝑖𝑚𝑧𝑒𝑧𝑚𝑎𝑥 (25) subject to ∑ ∑ 𝑧𝑖𝑗 𝑛 𝑗=𝑖+1 = 𝑧∗ (26) 𝑛−1 𝑖=1 𝑧𝑚𝑎𝑥 ≥ 𝑧𝑖𝑗 , 𝑖, 𝑗 = 1,2, … , 𝑛; 𝑖 < 𝑗, (27) and all first stage lp constraints. 𝑧∗is the optimal first stage solution value, 𝑧𝑚𝑎𝑥 is the maximum value of error 𝑧𝑖𝑗 . constraint (26) ensures that only those solutions that are optimal in the first stage lp are feasible in the second stage model. 2.4.3 advantages of the lp approach in an or em or lls, the error from equation (14) is 𝜀𝑖𝑗 = 𝑤𝑖 𝑤𝑗 𝑎𝑗𝑖 , 𝑖, 𝑗 = 1,2, … , 𝑛 (28) and then can be decided which 𝑎𝑖𝑗 can reduce the inconsistency. in the first stage of the lp model, the decision maker may go for a sensitivity analysis on inputs to know which entry in the pair wise comparison matrix should be changed to reduce inconsistency. the question is, how much should the entry should be changed? the values of the dual variables (shadow prices) at optimality provide an indication of the incorrect entries in the pair wise comparison matrix. it is a well-known fact in lp that a shadow price with value k at optimality has the following interpretation: “if we increase the right hand side of the dual variable’s corresponding constraint by one unit, then the value of the objective function increases by k units” ijahp article: patel, mjema godwin/ linear programming models for estimating weights in analytic hierarchy process and for optimization of human resource allocation international journal of the analytic hierarchy process 305 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.335 the dual variable that corresponds to each constraint in (16) changes the value of the objective function for unit change on the right hand side (that is, ln 𝑎𝑖𝑗 ). since it is in natural logarithm space, a dual variable with value k at optimality will increase the objective function value by k units when the corresponding 𝑎𝑖𝑗 increase by a factor of e, the base of natural logarithm, which is a useful information for a decision maker (chandran, b., et.al, 2005). it is now possible to identify which 𝑎𝑖𝑗 to change in order to decrease the inconsistency value by the greatest amount. a dual variable with negative value indicates that 𝑎𝑖𝑗 should increase, while a positive value indicates that 𝑎𝑖𝑗 should be decreased. the following section describes how the ahp was used to analyze the selection process in bfpl in tanzania. 2.5 numerical example in this section, relative weights (principal eigenvector) of a 4x4 comparison matrix (table 1) have been estimated using the an, em, lls and lp approaches. the entries in table 1 are the geometric means of a set of responses from 1/9 to 9. the principal eigenvalue 𝜆𝑚𝑎𝑥, consistency index ci, and consistency ratio cr are calculated using equations (7), (8) and (9) respectively and shown in table 2. relative weights obtained using lp shows that it preserves ed. table 1 4 x 4 pairwise comparison matrix 1 1 0.34 0.53 1 1 1.58 0.60 1/0.34 1/1.58 1 1.24 1/0.53 1/0.60 1/1.24 1 table 2 priority vectors, eigenvalues, consistency index and consistency ratio an em lls lp 𝑤1 0.1567 0.1560 0.1577 0.1583 𝑤2 0.2450 0.2490 0.2388 0.2716 𝑤3 0.3035 0.3017 0.2982 0.2716 𝑤4 0.2948 0.2933 0.3053 0.2985 𝜆𝑚𝑎𝑥 4.25 4.2486 4.1328 4.2667 𝐶𝐼 0.0833 0.0829 0.0443 0.0889 𝐶𝑅 0.0842 0.0837 0.0447 0.0898 the consistency index within the lp framework equation (24) is 0.377. it can be observed that 𝑎23 = 1.58 has been violated in the an, em and lls approaches whereas it has been preserved in the lp approach. any highly overrated cell can be detected using equation (28) for all the four methods. it does not directly give how much the cr has improved. but in the lp approach, the highly overrated cell can be detected with the help ijahp article: patel, mjema godwin/ linear programming models for estimating weights in analytic hierarchy process and for optimization of human resource allocation international journal of the analytic hierarchy process 306 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.335 of a dual variable, and how much the improvement in first stage can be known determined. in this numerical example, cell 𝑎23 = 1.58 has been detected as the most overrated cell and the dual value corresponding equation from (16) has reduced the value of 𝑧∗ from 2.262 to 1.307. 3. human resource allocation at bfpl bakhresa food product limited (bfpl) is a carbonated soft drinks manufacturing company owned and founded by mr. said salim awadh bakhresa in the 1970s. the company, among several others, together make up the azam group of companies, a conglomerate that deals with the production and provision of different products and services. it is among the largest family owned businesses in east and central africa. bfpl generates revenue of a minimum of three million dollars annually. with its headquarters located in the coastal region of dar es salaam, tanzania, this multi-national corporation faces different decision making challenges in its day to day operations and it is in the company’s best interest to manage risky situations that may come from wrong decision making choices. proper decision making choices will ensure the company’s sustainability and prosperity. since a company’s activities help to ensure its longevity, managers and decision makers at bfpl understand the need to evaluate weights and priorities of each activity in relation to its consequent outcome. their aim is to equip the companies with strategic decision making techniques for sustainable growth, and consequently to achieve a competitive upper hand in the market. decision makers must therefore adopt and apply different business practices, methods, and various tools that prove effective in decision making. in any organization, selection of employees to fill different departmental posts is a crucial and sensitive matter (kashi & friedrich, 2013). decision makers are well aware of the costs involved in the improper selection of employees on one hand, and the associated benefits of equipping the company with the right employees on the other. it is therefore vital for companies to not only select and hire employees whose skills and capabilities align well with the goals and objectives of the company, but also to achieve optimality in the return of their investment in human resources (saaty, peniwati & shang, 2007). having limited resources, the need for selection of potential employees must also incorporate the need to provide the company with employees whose values, mission and vision match those of the companies. an effective tool is required, one that will be able to take into consideration both tangible and intangible aspects of criteria to take into consideration when selecting employees. the key is to find proper measurement of the weights and priorities of different criteria such that error and bias in decision making is minimized. with this tool, the right kind of employees will be chosen to man the right kind of tasks in the organization and ultimately help the organization to achieve the optimum value of its investment in human resources (saaty, peniwati, & shang, 2007). ahp proves to be such a useful tool for selection of employees, and the application of linear programming will ensure the company achieves the optimality it requires (saaty, peniwati & shang, 2007; dye & forman, 1992). our purpose here is to illustrate ahp by deriving priorities and applying lp models and to formulate lp models to optimize returns. the company has identified ijahp article: patel, mjema godwin/ linear programming models for estimating weights in analytic hierarchy process and for optimization of human resource allocation international journal of the analytic hierarchy process 307 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.335 the general management and quality assurance departments to fill different positions for bfpl as listed in table 3. with the example given, we seek to obtain the optimal number of posts that need to be filled, and to select only the most qualified potential employees for every post whose contribution will result in the achievement of quality products, increase efficiency, expand capacity, and product development for bfpl (saaty, peniwati & shang, 2007).the selection hierarchy for any human resource allocation problem of bfpl can be represented through figure 1 with a 5-level of hierarchy. the 5 th level of hierarchy of alternatives is not shown, which can be on the bottom. 3.1illustration of human resource requirement and selection problem the problem has been elaborated with hierarchical disintegration, creation of the pairwise comparison matrix, priority weights development and the global weights development. finally, optimal recruitment satisfying constraints imposed by the company is determined (saaty, peniwati & shang, 2007).the company has many departments including human resources, production, general management, quality assurance, and finance and sales. for one of the plants situated near the port of dar es salam, the company wants to employ a technical manager, a driver and assistant(s) within the general management department, as well as lab-technicians and micro-biologists within quality assurance department. the company is looking to invest not more than 11 million tsh to be used as monthly salary packages for all the posts. the particulars of the posts including qualifications, salary packages and other details are given in table 3 (saaty, peniwati & shang, 2007). to begin with, a hierarchical structure with 5 levels is created after analyzing the problem. the highest level (goal) is the selection of potential candidates. the following level is the department level where in this case, there are two departments with vacanies, the general management and quality assurance departments. the third level is the posts to be filled; level four contains criteria for consideration in each of the posts within the departments. for example, the technical manager for the general management department is evaluated based on experience, educational qualification, technical skill and communication skills. this level establishes different intensities for each selection criteria. the lowest level, the alternatives, consists of potential candidates who have shown interest in the post (see figure 1). ijahp article: patel, mjema godwin/ linear programming models for estimating weights in analytic hierarchy process and for optimization of human resource allocation international journal of the analytic hierarchy process 308 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.335 table 3 personnel requirements variable post department number of positions criteria of selection monthly salary in tzs (m) 𝑦1 technical manager general management 1 experience education technical skill communication skill 6.0 𝑦2 driver general management 1 experience education mechanical skill 0.8 𝑦3 assistant general management 1 3 experience education office management communication skill 1.2 𝑦4 lab technician quality assurance 0 1 experience education technical skill communication skill 1.0 𝑦5 micro biologist quality assurance 0 2 experience education technical skill communication skill 0.9 a pair wise comparison matrix is developed for each hierarchical level using the fundamental scale. five senior level personnel from bfpl were selected and a questionnaire was given to them. instead of responding individually, persons responsible for selection helped the authors in providing the ratings in consensus. the priority weights for any level are obtained using the lp approach described in section 2.4. these weights are mentioned in figure1. level 4 demonstrates the criteria considered under each post. these are usually set by the organization to ensure the right candidates are selected for the right kind of job positions. each post has its own specific set of selection criteria. for instance, the technical manager’s post in the general management department has four selection criteria namely; experience, education, technical skills and communication skills. to obtain the weight of each of the criterion a pair wise comparison matrix is developed and the lp approach is further applied for local weights. the global weights of criteria are obtained by synthesizing with the global weight of the root level. for example, the global weight of criteria for technical manager is obtained as (0.473, 0.0118, 0.059, 0.067). 3.2 intensities and scores of criteria to implement the absolute measurement mode in ahp, each selection criterion for every post is further broken down into levels of intensity. these intensities should be located at level 6 (saaty, peniwati & shang, 2007). for example, the evaluation of the technical ijahp article: patel, mjema godwin/ linear programming models for estimating weights in analytic hierarchy process and for optimization of human resource allocation international journal of the analytic hierarchy process 309 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.335 manager consists of the following intensities: (i) experience is divided into three intensities of high (corresponds to 3+ years of experience), medium (1-3 years), and low (less than one year); (ii) education is divided into master, degree and diploma; (iii) technical skills are divided into excellent, good and fair; (iv) communication skills into high, medium and low (saaty, peniwati & shang, 2007). the rating for each position is given in table 4.using the lp approach, priorities of the intensities are obtained from pair wise comparisons and idealized by dividing each by the highest value so that it becomes 1 and the rest follow proportionally (table 5). the evaluation of each candidate is done by a group of experts who conduct multiple interviews relating to required job skills, personality and character, communication skills and ability to work in a group. this evaluation is done according to the criteria required for the successful selection of a potential employee. the experts (not less than 3 in number) generate a pairwise comparison matrix after obtaining the geometric mean of their judgments. the candidate’s synthesized score for each post they applied for is calculated according to the corresponding local weights of the selection criteria shown in table 4.their results are organized into a list in table 6. the total scores for each employee in table 6 have been adjusted multiplying by 3/5 for the general management department and 2/5 for the quality assurance department. ijahp article: patel, mjema godwin/ linear programming models for estimating weights in analytic hierarchy process and for optimization of human resource allocation international journal of the analytic hierarchy process 310 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.335 figure 1.selection hierarchy for bfpl with local (global) weights in table 4, the applicants are rated for quality for each criterion of the position for which they are applying. the standards for quality from which their rating is chosen are given in table 5. ts 0.082 (0.059) ex 0.683(0.081) ex 0.696(0.027) ex 0.696(0.058) ts 0.139 (0.006) om 0.087 (0.003) ex 0.557 (0.023) selection of candidate qa 0.125 gm 0.875 ex 0.659 (0.473) dv 0.135(0.118) ed 0.165 (0.118) lt 0.667(0.083) mb 0.333(0.042) tm0.82 (0.718) as 0.045 (0.039) cms 0.094 (0.067) ed 0.20 (0.024) ms 0.117 (0.014) ed 0.174(0.015) ts 0.087(0.007) cms 0.043 (0.004) cms 0.076 (0.003) ed 0.174(0.007) cms 0.043 (0.002) ed 0.228 (0.009) ijahp article: patel, mjema godwin/ linear programming models for estimating weights in analytic hierarchy process and for optimization of human resource allocation international journal of the analytic hierarchy process 311 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.335 table 4 criteria for positions to be filled with an example of applicant x1’s ratings for the tech. manager position applicants tech. manager experience education tech. skill comm. skill 0.473 0.118 0.059 0.067 𝑥1 high master good high 𝑥2 𝑥3 𝑥4 driver experience education mech.skill 0.081 0.024 0.014 𝑥5 𝑥6 𝑥7 𝑥8 assistant experience education office mgmt comm. skill 0.027 0.007 0.003 0.002 𝑥9 𝑥10 𝑥11 𝑥12 lab-technician experience education tech. skill mech. skill 0.058 0.015 0.007 0.004 𝑥13 𝑥14 𝑥15 𝑥16 micro-biologist experience education tech. skill mech. skill 0.023 0.009 0.006 0.003 𝑥17 𝑥18 𝑥19 𝑥20 ijahp article: patel, mjema godwin/ linear programming models for estimating weights in analytic hierarchy process and for optimization of human resource allocation international journal of the analytic hierarchy process 312 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.335 table 5 ideal priorities of the intensities used for rating alternatives in table 4 criteria by job (global priorities) intensities and idealized priorities technical mangers experience(0.473) education(0.165) technical skills(0.082) communication skills(0.094) high (1) medium (0.55) low (0.302) masters (1) degree (0.363) diploma (0.132) excellent (1) good (0.25) fair (0.125) high (1) medium (0.573) low (0.219) driver experience (0.081) education (0.024) mechanical skills (0.014) high (1) medium (0.55) low (0.302) diploma (0.142) certificate (0.705) secondary (1) high (1) medium (0.210) low (0.088) assistant experience (0.027) education (0.007) office management skills (0.003) communication skills (0.002) high (1) medium (0.55) low (0.302) degree (0.142) diploma (0.705) certificate (1) best (1) good (0.363) fair (0.132) high (1) medium (0.573) low (0.219) lab technician experience (0.058) education (0.015) technical skills (0.007) communication skills (0.004) high (1) medium (0.55) low (0.302) masters (1) degree (0.363) diploma (0.132) excellent (1) good (0.25) fair (0.125) high (1) medium (0.573) low (0.219) micro biologist experience (0.023) education (0.009) technical skills (0.006) communication skills (0.003) high (1) medium (0.55) low (0.302) masters (1) degree (0.363) diploma (0.132) excellent (1) good (0.25) fair (0.125) high (1) medium (0.573) low (0.219) ijahp article: patel, mjema godwin/ linear programming models for estimating weights in analytic hierarchy process and for optimization of human resource allocation international journal of the analytic hierarchy process 313 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.335 table 6 applicant’s priorities obtained from their rating totals number applicants post applied for score 1 𝑥1 technical manager 0.4040 2 𝑥2 technical manager 0.3680 3 𝑥3 technical manager 0.2403 4 𝑥4 technical manager 0.3506 5 𝑥5 driver 0.0295 6 𝑥6 driver 0.0648 7 𝑥7 driver 0.0332 8 𝑥8 driver 0.0453 9 𝑥9 assistant 0.0225 10 𝑥10 assistant 0.0069 11 𝑥11 assistant 0.0145 12 𝑥12 assistant 0.0084 13 𝑥13 lab. technician 0.0169 14 𝑥14 lab. technician 0.0150 15 𝑥15 lab. technician 0.0271 16 𝑥16 lab. technician 0.0085 17 𝑥17 micro. biologist 0.0123 18 𝑥18 micro. biologist 0.0056 19 𝑥19 micro. biologist 0.068 20 𝑥20 micro. biologist 0.0046 3.3 manpower allocation for bfpl two comparable linear models are presented in this section for the best human resource allocation for bfpl. model 1: optimization for individual applicants the objective function coefficients are the scores given in table 6. the decision variables 𝑥1, 𝑥2, … , 𝑥20are binary, subject to salary constraint, upper and lower bound constraints on the number of people given in table 3. 𝑀𝑎𝑥 0.4040𝑥1 + 0.3680𝑥2 + 0.2403𝑥3 + 0.3506𝑥4 + 0.0295𝑥5 + 0.0648𝑥6 + 0.0332𝑥7 + 0.0453𝑥8 + 0.0225𝑥9 + 0.0069𝑥10 + 0.0145𝑥11 + 0.0084𝑥12 + 0.0169𝑥13 + 0.0150𝑥14 + 0.0271𝑥15 + 0.0085𝑥16 + 0.0123𝑥17 + 0.0056𝑥18 + 0.0068𝑥19 + 0.0046𝑥20 ≤ 11 subject to 6𝑥1 + 6𝑥2 + 6𝑥3 + 6𝑥4 + 0.8𝑥5 + 0.8𝑥6 + 0.8𝑥7 + 0.8𝑥8 + 1.2𝑥9 + 1.2𝑥10 + 1.2𝑥11 + 1.2𝑥12 + 1.0𝑥13 + 1.0𝑥14 + 1.0𝑥15 + 1.0𝑥16 + 0.9𝑥17 + 0.9𝑥18 + 0.9𝑥19 + 0.9𝑥20 ≤ 11 (salary constraint) ijahp article: patel, mjema godwin/ linear programming models for estimating weights in analytic hierarchy process and for optimization of human resource allocation international journal of the analytic hierarchy process 314 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.335 𝑥1 + 𝑥2 + 𝑥3 + 𝑥4 = 1 (𝑇𝑒𝑐ℎ𝑛𝑖𝑐𝑎𝑙𝑀𝑎𝑛𝑎𝑔𝑒𝑟) 0 ≤ 𝑥5 + 𝑥6 + 𝑥7 + 𝑥8 ≤ 1 (𝐷𝑟𝑖𝑣𝑒𝑟) 1 ≤ 𝑥9 + 𝑥10 + 𝑥11 + 𝑥12 ≤ 3 (𝐴𝑠𝑠𝑖𝑠𝑡𝑎𝑛𝑡) 0 ≤ 𝑥13 + 𝑥14 + 𝑥15 + 𝑥16 ≤ 1 (𝐿𝑎𝑏𝑇𝑒𝑐ℎ𝑛𝑖𝑐𝑖𝑎𝑛) 0 ≤ 𝑥17 + 𝑥18 + 𝑥19 + 𝑥20 ≤ 3 (𝑀𝑖𝑐𝑟𝑜 𝐵𝑖𝑜𝑙𝑜𝑔𝑖𝑠𝑡) 𝑥𝑗 , j = 1,2, … ,20 are binary model 1 was solved using excel solver to maximize the goal of bfpl and solution is given in table 7. table 7 the optimal solution of lp model 1 selected applicant position salary in million (tsh) 𝑥1 technical manager 6 𝑥6 driver 0.8 𝑥9 assistant 1.2 𝑥15 lab technician 1 𝑥17 micro biologist 0.9 𝑥19 micro biologist 0.9 total salaries 10.8 model 2: optimizing different positions this approach consequently provides the priorities of the five positions found in level four of the hierarchy of figure 1 as coefficients of the objective function. the posts are represented by 𝑦1 𝑡𝑜 𝑦5, such that their values integers denote the vacancy of jobs posts. the previous model made the selection of applicants based on their rating taking the relative importance of the post. with these models we are able to obtain the optimal number of jobs and the selection of the best applicants for those job posts. coefficients of objective function from the fourth level of figure1 are adjusted by multiplying 3/5 with the general management department and 2/5 with the quality assurance department. the model is 𝑀𝑎𝑥 0.492𝑦1 + 0.081𝑦2 + 0.027𝑦3 + 0.2668𝑦4 + 0.1332𝑦5 subject to 𝑦1 + 0.8𝑦2 + 1.2𝑦3 + 𝑦4 + 0.9𝑦5 ≤ 11 𝑠𝑎𝑙𝑎𝑟𝑦 𝑐𝑜𝑛𝑠𝑡𝑟𝑎𝑖𝑛𝑡 𝑦1 = 1 𝑇𝑒𝑐ℎ𝑛𝑖𝑐𝑎𝑙 𝑀𝑎𝑛𝑎𝑔𝑒𝑟 𝑦2 = 1 𝐷𝑟𝑖𝑣𝑒𝑟 𝑦3 ≥ 1 𝐴𝑠𝑠𝑖𝑠𝑡𝑎𝑛𝑡, 𝐿𝑜𝑤𝑒𝑟 𝑏𝑜𝑢𝑛𝑑 𝑦3 ≤ 3 𝐴𝑠𝑠𝑖𝑠𝑡𝑎𝑛𝑡, 𝑈𝑝𝑝𝑒𝑟 𝑏𝑜𝑢𝑛𝑑 𝑦4 ≥ 0 𝐿𝑎𝑏 𝑇𝑒𝑐ℎ𝑛𝑖𝑐𝑖𝑎𝑛, 𝐿𝑜𝑤𝑒𝑟 𝑏𝑜𝑢𝑛𝑑 𝑦4 ≤ 1 𝐿𝑎𝑏 𝑇𝑒𝑐ℎ𝑛𝑖𝑐𝑖𝑎𝑛, 𝑈𝑝𝑝𝑒𝑟 𝑏𝑜𝑢𝑛𝑑 𝑦5 ≥ 0 𝑀𝑖𝑐𝑟𝑜 𝐵𝑖𝑜𝑙𝑜𝑔𝑖𝑠𝑡, 𝐿𝑜𝑤𝑒𝑟 𝑏𝑜𝑢𝑛𝑑 𝑦5 ≤ 2 𝑀𝑖𝑐𝑟𝑜 𝐵𝑖𝑜𝑙𝑜𝑔𝑖𝑠𝑡, 𝑈𝑝𝑝𝑒𝑟 𝑏𝑜𝑢𝑛𝑑 𝑦𝑗 are integers ijahp article: patel, mjema godwin/ linear programming models for estimating weights in analytic hierarchy process and for optimization of human resource allocation international journal of the analytic hierarchy process 315 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.335 table 8 optimal solution second model position selection position salary total salary technical manager 1 6000000 tzs 6000000 tzs driver 1 800000 tzs 800000 tzs assistant 1 1200000 tzs 1200000 tzs lab technician 1 1000000 tzs 1000000 tzs micro biologist 2 900000 tzs 1800000 tzs total salaries 10800000 tzs the optimal solution of model 2 is given in table 8. the solution of model 2 is consistent with the model 1 solution. all the posts can be filled if the company can spend 13.2 million tzs in place of 11 million tzs per month. 4. summary and conclusions ahp can measure intangibles and lp proves to be effective in optimizing the resource allocation problem by also considering tangible measurements. this paper has used both tangible and intangible measures. after converting intangibles by using the ahp technique, priority has also been calculated using lp. element dominance and row dominance have been incorporated as constraints in lp. lp has several advantages over the additive normalization or eigenvectors or lss methods for determining priorities. in section 3, the bfpl case has been presented to fill the vacant post. combined ahp and lp models seem to provide an effective tool. ijahp article: patel, mjema godwin/ linear programming models for estimating weights in analytic hierarchy process and for optimization of human resource allocation international journal of the analytic hierarchy process 316 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.335 references bryson n., (1995). a goal programming method for generating priority vectors. journal of operational research society, 46, 641-648.doi: 10.1057/jors.1995.88 battistoni, e. et al (2013). analytic hierarchy process for new product development. international journal of engineering business management, 5(42), 16.doi: 10.5772/56816 bhatt, r. et al. (2013). best engineering college selection through analytic hierarchy process: a case study. international journal of technology & engineering education,10(2),16. chandran, b. et al. (2005).linear programming models for estimating weights in the analytic hierarchy process. computer& operation research, 32, 22352254.doi:10.1016/j.cor.2004.02.010 cheng, e.,& heng, l. (2001).analytic hierarchy process: an approach to determine measures for business performance. measuring business excellence, 5(3),30-37. http://dx.doi.org/10.1108/eum0000000005864 cogger, c., & yu, p. l. (1985). eigenweight vectors and least distance approximation for revealed preference in pairwise weight ratios. journal of optimization theory and application,46, 483491.doi: 10.1007/bf00939153 crawford, g., & william, c. (1985).a note on the analysis of subjective judgments matrices.journal of mathematical psychology, 25, 387406. doi:10.1016/00222496(85)90002-1 enea, m., & piazza, t. (2004). project selection by constrained fuzzy analytic hierarchy process. fuzzy optimization and decision making, 3, 39-62. doi:10.1023/b:fodm.0000013071.63614.3d fisher, c.d., et al. (1992). human resources management. boston: houghton mifflin company. garg, r., rahman, z., qureshi, m., & kumar, i. (2012). identifying and ranking critical success factors of customer experience in banks. an analytic hierarchy process (ahp) approach. journal of modeling in management, 7(2), 201220.http://dx.doi.org/10.1108/17465661211242813 hassan, n. et al. (2013). selection of mobile network operators using analytic hierarchy process. advances in natural and applied sciences, 7(1), 15. mathis, r.l., & jackson, h.j. (1991).human resources management. minneapolis: west publishing corp. saaty, t. l., (1980). analytic hierarchy process, new york: mcgrawhill.doi:10.1080/00137918308956077 http://dx.doi.org/10.1016/j.cor.2004.02.010 http://dx.doi.org/10.1016/0022-2496(85)90002-1 http://dx.doi.org/10.1016/0022-2496(85)90002-1 http://dx.doi.org/10.1108/17465661211242813 ijahp article: patel, mjema godwin/ linear programming models for estimating weights in analytic hierarchy process and for optimization of human resource allocation international journal of the analytic hierarchy process 317 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.335 saaty, t. l., (1999). basic theory of the analytic hierarchy process: how to make decision. revista de la real academia ciencias exactas fiscicas y naturales, 93(4), 395-453. saaty, t. l. (2003). decision making with the ahp: why the principal eigenvector is important. european journal of operational research, 14, 85-91. doi:10.1016/s03772217(02)00227-8 saaty, t. l., peniwati, k., & shang, s. j. (2007).the analytic hierarchy process and the human resource allocation: half the story. mathematical and computer modeling, 46, 1041-1053.doi:10.1016/j.mcm.2007.03.010 saaty, t. l., (2008). decision making with the analytic hierarchy process. international journal services science, 1(1),83-97.doi: http://dx.doi.org/10.1504/ijssci.2008.017590 saaty, t. l., (2013). on the measurement of intangibles. a principal eigenvector approach to relative measurement derived from paired comparisons, notices of the ams, 60(2), 192-208.doi: http://dx.doi.org/10.1090/noti944 sinuary-stern, z., mehrez, a.,& hadad, y., (2006).an ahp/dea methodology for ranking decision making units. international transactions in operational research, 7, 109-124.doi: 10.1111/j.1475-3995.2000.tb00189.x shuler, r.s., & huber,v. l. (1993). personnel and human resources management. new york: west publishing corp. srdjevic, b., (2005). combining different prioritization methods in the analytic hierarchy process synthesis. computer & operations research, 32, 1897-1919. doi:10.1016/j.cor.2003.12.005 tien-chin, w., & ying-ling, l. (2009).using a multi-criteria group decision making approach to select merged strategies for commercial banks. group decision negotiations, 18, 519-536. doi:10.1007/s10726-008-9112-3 vaidya, s., & kumar, s., (2006).the analytic hierarchy process: an overview of application. european journal of operational research, 169(1), 129.doi:10.1016/j.ejor.2004.04.028 http://dx.doi.org/10.1016/s0377-2217(02)00227-8 http://dx.doi.org/10.1016/s0377-2217(02)00227-8 http://dx.doi.org/10.1016/j.mcm.2007.03.010 http://dx.doi.org/10.1016/j.cor.2003.12.005 http://dx.doi.org/10.1016/j.ejor.2004.04.028 ijahp article: gupta, makrigeorgis/ prioritization of small business enterprise owners concerns about the affordable care act using the analytic hierarchy process international journal of the analytic hierarchy process 178 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.365 prioritization of small business enterprise owner concerns about the affordable care act using the analytic hierarchy process rakesh m. k. gupta board of governors of the federal reserve system, washington, dc, usa grmike_98@yahoo.com christos makrigeorgis college of management and technology, walden university, minneapolis, mn, usa christos.makrigeorgis@waldenu.edu abstract for almost a century, americans lacked a comprehensive health care system. upon the passage of the affordable care act (aca) in 2010, politicians, media, and lobbyists provided conflicting and confusing interpretations of the law. with such varied opinions, small business enterprise (sbe) owners became concerned about the potential adverse financial effects of the aca. in this paper, we researched the problem by applying the analytic hierarchy process (ahp) to create a ranking of the concerns of the sbe owners toward the aca. a sample of 50 sbe owners representing 5 specific industry groups in richmond, virginia, was used to obtain a cross-sectional view of the concerns which allowed us to ascertain their uniformity or variability across these industry groups. the ahp sample data set was obtained via a pairwise comparison questionnaire. the ahp analysis revealed that the topmost sbe owners concerns were insurance premium, quality of care, and tax burden. in addition, these concerns were non-uniformly ranked among the industry groups. however, the highest and lowest concerns in each industry group were the same across the industry groups. keywords: ahp; affordable care act; small business enterprise owners 1. introduction in the 20 th century, several u.s. presidents—president theodore roosevelt, president harry truman, and president william clinton—attempted but were unsuccessful in their efforts to create a comprehensive health care system (oberlander, 2012; parks, 2011) however, in 2010, president barack obama signed the affordable care act (aca) into law (u.s. department of health and human services, n.d.). despite several challenges, the u.s. supreme court upheld the law in 2012 (curfman, abel, & landers, 2012). several studies, research articles, and books alluded to concerns about the adverse financial impact of the law on businesses (geyman, 2012; hellander & bhargavan, mailto:grmike_98@yahoo.com mailto:christos.makrigeorgis@waldenu.edu ijahp article: gupta, makrigeorgis/ prioritization of small business enterprise owners concerns about the affordable care act using the analytic hierarchy process international journal of the analytic hierarchy process 179 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.365 2012). however, the relatively new literature lacked studies that quantify the intensity of specific business owner’s concerns toward the aca. we focused in particular on small business enterprises (sbes) since many of the concerns reviewed in the latest research articles were identifying these concerns as pertinent mostly to such small businesses. more importantly, the lack of quantification of the importance, weighting, or ranking of the sbe owner’s concerns relative to each other provided the motivation for this research to measure the intensity of such concerns. once these concerns were identified, we then developed a ranked hierarchy treating them as criteria in the application of the analytic hierarchy process (ahp) technique (ishizaka & labib, 2011). the first issue facing the researchers was that a standard definition of an sbe was cumbersome to obtain. eyal-cohen (2013) provided definitions based on securities law, health care coverage, labor and employment, patent law, and internal revenue code and described the small employer as an employer hiring up to 100 employees. for the purpose of this research, we considered an sbe as a business in the united states that employs up to 50 full-time equivalent (fte) employees (healthcare, n.d.). the second issue was to identify and apply the proper multi-criteria decision-making (mcdm) technique. for this purpose, several mcdm techniques were considered such as ahp, electre, macbeth, and promethee. the findings of theoretical and experimental validation, ishizaka, balkenborg, and kaplan (2011) revealed no single mcdm method was better than another method. however, we selected ahp for its ability to compare quantitative or qualitative criteria and check the consistency of judgments (aminbakhsh, gunduz, & sonmez, 2013). the ahp method involves a qualitative approach for determining the objective (level 0), criteria (level 1) and subcriteria (level 2 or below), structuring these into a hierarchy, and ranking each within the corresponding hierarchy levels. furthermore, the ahp method requires a quantitative approach for pairwise comparison, consistency checking, and aggregation of judgments. the identification and prioritization of sbe owner’s concerns highlight key perceptions of sbe owners about the impact of the aca on their businesses. from this research, we revealed the crucial concerns that various government agencies and organizations should be focusing on to alleviate or ease the sbe owner’s concerns. the research is significant as it renders a systematic approach to decision-making using the ahp technique. since no such studies were previously conducted, the research provides an avenue to future researchers for a broader sample and a framework for similar complex decision-making problems. 2. literature review 2.1 health care reform plans for health care reform started in 1912 when president theodore roosevelt attempted to provide health insurance to u.s. citizens (parks, 2011). his efforts did not succeed because of opposition from political leaders, businesses, and the insurance industry to socialized medicine (parks, 2011). however, public-private partnership did evolve to provide health insurance to u.s. citizens (parks, 2011). after world war ii, several presidents, including harry truman, richard nixon, james carter, and william clinton, tried to pass a universal health insurance law (oberlander, 2012). again, mistrust ijahp article: gupta, makrigeorgis/ prioritization of small business enterprise owners concerns about the affordable care act using the analytic hierarchy process international journal of the analytic hierarchy process 180 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.365 in u.s. federal government control, political opposition, and opposition from business and industry contributed to the failure of passage of a health insurance law (gable, 2011; oberlander, 2012). the passage of the 1965 medicare and medicaid act allowed u.s. government programs to insure the more vulnerable elderly and low-income population (oberlander, 2012). in 2006, massachusetts implemented health care reform that provided a framework for comprehensive health care. eventually, in 2010, following acrimonious and political wrangling of lawmakers, the aca became a law (oberlander, 2012). according to gardner (2012), the aca encompasses a vision to invest in new infrastructure for improving the quality and reducing the cost of health care. in addition, improving the quality of information, infrastructure, and incentives are all primary objectives of the aca (gardner, 2012). moreover, gardner recommended changing the provider-payer system from a fee-for-service to value-based service for successful implementation of the aca. taking a contrasting view, wilensky (2012) remarked that u.s. health care encountered 3 key problems: millions of uninsured people, high costs of health care, and diminishing quality of care. the aca addressed only the first problem by expanding health care coverage to uninsured people. geyman (2012) argued consumer-driven health care was ineffective in controlling cost and contributed to restricted access, underutilization, and lower quality of health care. consumer-driven health care relies on the premise of the moral hazard theory, which implies patients will over-utilize health care services without cost-sharing arrangements and so will contribute to rising cost of services. geyman identified 3 issues which health care reform should address: (a) management and administration cost, (b) rising cost of health care, and (c) access to and quality of care. 2.2 small business enterprise in attempting to provide a legal definition of a small business, eyal-cohen (2013) asserted that there was a lack of a standard definition in both business and industry. according to eyal-cohen, legal definitions of a small business vary in sections of the law. for a legal definition, a firm’s size was not an important measure (eyal-cohen, 2013). thus eyal-cohen provided a comprehensive definition of a small business based on securities law, health care coverage, labor and employment, patent law, and internal revenue code. eyal-cohen described the small employer as an employer hiring up to 100 employees. however, as stated in healthcare (n.d.), sbes can have only up to 50 fte employees. miller, eibner, and gresenz (2013) conducted a study of the impact of financial regulatory arbitrage of employment-based, self-insured plans on employees. these programs received the exempt status from the provisions of the aca and were not under the purview of state health insurance regulations. miller et al. (2013) evaluated employee concerns about: (a) employer’s financial stability, (b) health benefits and claims adjudication, and (c) appeal process. the findings revealed similarities of health benefits between self-insured and fully insured plans. according to miller et al. (2013), the aca would improve appeal processing for both types of insurance plans. ijahp article: gupta, makrigeorgis/ prioritization of small business enterprise owners concerns about the affordable care act using the analytic hierarchy process international journal of the analytic hierarchy process 181 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.365 kapur, karaca-mandic, gates, and fulton (2012) examined the relationship between the size of a small business and small group health insurance regulations for offering health insurance to its employees while controlling health insurance costs. remarking on these regulations, kapur et al. (2012) noted small group insurance reform regulated insurance policies based on a business size threshold, from 2 or 3 employees to 25 or 50 employees. to ascertain the quality of jobs that included health care coverage, litwin and phan (2013) revealed start-up business owners did not provide health and retirement benefits. the probability of providing such benefits slightly improved following 6 years of business operation (litwin & phan, 2013). 2.3 sbe owner’s concerns speculating how many employers would stop offering health insurance coverage to their employees, buchmueller, carey, and levy (2013) examined theoretical and empirical evidence of health insurance changes since the enactment of the aca. employersponsored health insurance had 3 advantages over the individual market: (a) no income tax on insurance premiums, (b) adverse selection mitigation, and (c) economies of scale. however, the complexity of the law that sbe owners misunderstood rendered small employers confused and indecisive about relevant provisions of the aca (buchmueller, carey, & levy, 2013). addressing the issues of health care and financial planning attributed to the aca, cordell and langdon (2011, 2012) remarked that the rules for tax savings from various pretax accounts would change. small business employers would either offer health coverage or pay additional nondeductible tax for every full-time employee (cordell & langdon, 2012). the individual mandate tax could prevent sbe owners from adding another fte. in a survey of u.s. small businesses, jacobe (2013a) reported 48% of the sbe owners thought the aca was inadequate for their business. a gallup survey conducted in april 2013 with 603 sbe owners also revealed 52% of employers said the aca would reduce quality of care, and 55% thought cost of health care would increase (jacobe, 2013a). in another survey, jacobe (2013b) reported key concerns of sbe owners were health care costs (54%), taxes on businesses (53%), and u.s. government regulations (46%). lepard (2013) presented a legal issue with the contraceptive coverage mandate of the aca and religious rights of corporations. in the absence of a final decision, lepard recommended that the u.s. congress should either include for-profit corporations in the religious employer exemption or require the u.s. government to provide free contraceptive coverage. similar to lepard’s argument, loewentheil (2014) argued for religious accommodations based on the free exercise rights whenever such accommodations impose any burdens on others. loewentheil (2014) used the example of contraceptive coverage mandate of the aca to explain the failure of existing principles supporting religious accommodations. loewentheil (2014) proposed a theoretical framework, balancing the burdens on both religious rights of objectors and supporters. consolidating results from several studies, mcmorrow, blumberg, and buettgens (2011) reported effects of the aca on sbes. they pointed out administrative costs and limited ability to spread risks adversely affected small businesses because of the aca. however, tax credits were helpful to sbe owners to provide health insurance to their employees. ijahp article: gupta, makrigeorgis/ prioritization of small business enterprise owners concerns about the affordable care act using the analytic hierarchy process international journal of the analytic hierarchy process 182 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.365 miller (2011) remarked that the cost of health insurance coverage during a 10-year period ending in 2009 increased by 123% for small to medium-sized businesses. during the same period, insurance coverage dropped from 65% to 59% for small to medium-sized businesses and from 56% to 46% for very small businesses. miller (2011) commented that several factors such as subsidies and premiums could affect the ability of sbe owners to provide health insurance coverage based on the aca. miller (2011) pointed out that an increase in health insurance coverage of small business employees would depend on available subsidies and insurance premiums. in reviewing the impact of the aca on employer-sponsored health insurance plans, tacchino (2013a) explained that rising insurance premiums and play-or-pay tax requirements of the aca became an opportunity for employers to reevaluate their existing plans and offer coverage through health insurance exchanges. specifically, tacchino (2013b) reviewed criteria for identifying employers impacted by the tax and the method of calculating potential employer tax liability. sbe owners either met the requirements or incurred a tax liability for ftes without health insurance coverage or unaffordable minimum essential health insurance coverage. 3. methodology for the purpose of prioritizing the sbe owner’s concerns, we used the ahp technique. in general, researchers can apply the ahp method to solve decision-making problems that involve (a) the selection of an alternative, (b) the prioritization of factors or criteria, or (c) the evaluation of homogeneous criteria (al-hawari, al-bo'ol, & momani, 2011; saaty & shang, 2011; talib, rahman, & qureshi, 2011). more importantly, researchers can use ahp to address decision-making problems that involve a mix of qualitative and quantitative criteria (ishizaka & labib, 2011). in order to be able to model and solve prioritization problems with the ahp technique, researchers rely on the judgment of pairwise comparison criteria using a ratio, verbal, or graphical scale (talib et al., 2011). the ahp method begins with a qualitative approach for determining the objective, criteria, and sub-criteria and structuring these into a hierarchy. furthermore, the ahp method requires a quantitative approach for pairwise comparison, consistency checking, and aggregation of judgments. we used the following 8 steps of the ahp method for conducting the study of the research problem as shown in figure 1. ijahp article: gupta, makrigeorgis/ prioritization of small business enterprise owners concerns about the affordable care act using the analytic hierarchy process international journal of the analytic hierarchy process 183 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.365 figure 1. steps of the analytic hierarchy process (ahp) method in step 1, we identified the following 9 sbe owner’s concerns from the reviewed literature:  administrative cost  complexity of aca  insurance premium  penalty  plan choice  quality of care  religious objection  reporting burden  tax burden in step 2, we created an ahp structure with these 9 sbe owner’s concerns for further research. we used freely-available online software, ahp-os, to form and display the hierarchical structure as shown in figure 2 (goepel, 2013). the software attaches initial values capturing the default local and global priorities of the sbe owners concerns at each level of the hierarchy. level 0 of the hierarchy represents the overall goal of the ahp analysis, level 1 represents the aggregate concerns or criteria, and level 2 represents the specific concerns or sub-criteria of the ahp hierarchy. ijahp article: gupta, makrigeorgis/ prioritization of small business enterprise owners concerns about the affordable care act using the analytic hierarchy process international journal of the analytic hierarchy process 184 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.365 figure 2. ahp structure with initial or default priorities in step 3, we utilized the identified structure from step 2 and figure 2 and created a pairwise comparison survey instrument within the ahp-os software. the scale of measurement was a ratio scale known as saaty’s fundamental scale (saaty & vargas, 2012). the scale is derived from thurstone’s scale designed for pairwise comparisons and uses an importance rating on a scale of 1 to 9. that is, in comparing a to b, the comparison requires that the participant decide 2 things: “is a more important than b or b more important than a?”, and “how much more important is one from the other?”. for example, “a is more important than b” and “7” on the scale from 1 to 9. with m items to be compared within a given level, a total of m(m-1)/2 comparisons are needed. our pairwise comparison survey consisted of 3 parts. part 1 contained 2 items at level 1 thus a total of 2(2-1)/2 = 1 comparison between the choices “health insurance cost” vs “health insurance coverage”. part 2 of the questionnaire contained 4 subcriteria under “health insurance cost” and thus 4(4-1)/2 = 6 pairwise comparison questions. part 3 of the questionnaire contained 5 sub-criteria under the “health insurance coverage” node and thus 5(5-1)/2 = 10 pairwise comparison questions. thus a total of 1+6+10=17 pairwise comparison questions had to be answered by each participant. the entire questionnaire captured with ahp-os software is as shown in figure 3. ijahp article: gupta, makrigeorgis/ prioritization of small business enterprise owners concerns about the affordable care act using the analytic hierarchy process international journal of the analytic hierarchy process 185 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.365 figure 3. ahp-os pairwise questionnaire of sbe concerns on aca 3.1 population and sampling the ahp analysis was based on a sample of 50 sbe owners representing 5 industry groups in richmond, virginia. to begin with, we conducted an online search of small business enterprises within richmond, virginia, and provided subcategories of business industries similar to those provided by u.s. small business administration (n.d.). the classification of industries resulted in 5 industry groups (igs) as shown in table 1. a total of 150 sbe’s were selected to form the sample population. ijahp article: gupta, makrigeorgis/ prioritization of small business enterprise owners concerns about the affordable care act using the analytic hierarchy process international journal of the analytic hierarchy process 186 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.365 table 1 sbe industry group code, name, and classification industry group code name key industries ig1 construction, housing, & real estate construction, housing, commercial and residential real estate ig2 food, beverages, consumer goods, & services food, beverage, restaurants, retail goods and services ig3 health care, pharmaceuticals, & biotechnology dental and medical clinics, pharmaceutical, health care facilities and services ig4 financial, legal, & professional services accounting, financial, educational, legal, and professional services ig5 remaining goods & services agriculture, automotive, energy, technology, transportation and logistics next, a decision on the total sample size and then on the sample size within each industry group had to be made. beginning with the total sample size, a value of 50 was decided based on a number of considerations. first, the strength of the ahp technique is that it requires no minimum sample size and thus researchers can conduct a study using even a single participant (ramanathan, 2001). however, this may introduce the single participant’s bias in research findings. on the other extreme, larger sample sizes (in excess of say 50 participants) can introduce a higher chance of inconsistent comparisons across the participants (aull-hyde, erdogan, & duke. 2006; saaty & vargas, 2012). second, the proximity to the sampled participants within the defined population frame was considered in deciding that a sample size of 50 sbe participants across 5 industry groups of interest was deemed right for this particular ahp application. once the total sample size of 50 was decided, the next task was to spread it roughly evenly across the 5 industry groups so that each can receive an equal chance of representation. once the counts within each ig were decided, the actual sample of sbe’s within each ig was drawn randomly using excel’s rand() function. upon receiving survey results from 10 participants in each industry group, we stopped seeking further participation. the population and sample counts by ig are summarized in table 2 below. table 2 population and participation counts by industry group industry group id sample population sbe owners approached sbe owners participated percentage participation ig1 30 16 10 10/16 = 63% ig2 30 19 10 10/19 = 53% ig3 30 13 10 10/13 = 77% ig4 30 13 10 10/13 = 77% ig5 30 18 10 10/18 = 56% total 150 79 50 50/79 = 63% as seen in table 2, the total sample of 50 was cross-sectional as it roughly represented over 50% of participants across each of the 5 industry groups. ijahp article: gupta, makrigeorgis/ prioritization of small business enterprise owners concerns about the affordable care act using the analytic hierarchy process international journal of the analytic hierarchy process 187 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.365 3.2 data collection and analysis in step 4, we administered the pairwise survey to the random participants of the sbe industry groups and ensured their responses were consistent by checking them in real time. specifically, each participant answered the pairwise comparison question first by choosing the importance of the sbe owners concern, and then by rating its importance on a scale of 1 to 9. the participant finished the survey upon completing each part of the 3part survey questionnaire to rank the group of concerns. the ahp-os software automatically calculated the consistency ratio (cr) of the participant’s responses for each part of the survey (goepel, 2013). if the cr was no more than 10%, that part of the survey was acceptable for further analysis. if, however, the cr was more than 10%, the participant followed step 5 of the ahp method. in step 5, the participant reviewed the final responses for possible remnant inconsistencies. the participant could adjust the responses to make them consistent and acceptable. the participant could make the adjustments either by using suggestions provided by the software logic or by changing the judgments slightly using the ahp-os software (goepel, 2013). the participant could also conduct iterative adjustments ensuring consistency of judgment to survey questions. upon ensuring consistent responses by the participant, that part of the survey was acceptable for further analysis. in step 6 of the data analysis, we ported the data from ahp-os software to specialized ahp excel worksheets for aggregation (goepel, 2013). based on the hierarchical structure of the research problem and upon completion of the step 6 on the collected data, the ahp-os software yielded 150 matrices—50 2×2 matrices, 50 4×4 matrices, and 50 5×5 matrices—providing local priorities of sbe owner’s concerns. in step 7, we aggregated the local priorities by industry groups and the sample by using the specialized ahp excel worksheets (goepel, 2013). finally, in step 8, we synthesized the local priorities into global priorities by using standard excel worksheets. 4. results in order to elaborate on the analysis results, we begin by demonstrating the consistency of judgment from the responses of a specific participant 1 in ig1 labelled ig1-1. to obtain the global priority weights of individual judgments, researchers apply synthesis of individual priorities that requires a multiplication of local priority weights by weights of all parent nodes (bhatt & macwan, 2012). the local priorities at each hierarchy level and the global priority of the concerns resulting from the responses of participant ig1-1 are provided in the appendix. next, the synthesis and aggregation of local priorities of the survey participant’s responses in the entire industry group ig1 resulted in the global priorities or overall weights of group judgments by multiplying local priorities at level 1 and level 2 of the ahp structure. table 3 shows the synthesis of global priorities for ig1. ijahp article: gupta, makrigeorgis/ prioritization of small business enterprise owners concerns about the affordable care act using the analytic hierarchy process international journal of the analytic hierarchy process 188 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.365 table 3 global priority of ig1 group judgments ahp factor local priority at level 1 sbe owners’ concern local priority at level 2 global priority. health insurance cost 0.719202 administrative cost 0.105585 0.075937 insurance premium 0.591494 0.425404 penalty 0.128072 0.092110 tax burden 0.174849 0.125752 health insurance coverage 0.280798 complexity of aca 0.135166 0.037954 plan choice 0.277329 0.077873 quality of care 0.399727 0.112243 religious objection 0.121882 0.034224 reporting burden 0.065896 0.018503 similarly, the synthesis and aggregation of local priorities of survey participant’s responses for ig2 through ig5 are provided in the global priorities as shown in table 4 through table 7. table 4 global priority of ig2 group judgments ahp factor local priority at level 1 sbe owners’ concern local priority at level 2 global priority. health insurance cost 0.646577 administrative cost 0.081469 0.052676 insurance premium 0.656658 0.424580 penalty 0.138345 0.089451 tax burden 0.123527 0.079870 health insurance coverage 0.353423 complexity of aca 0.076807 0.027146 plan choice 0.378897 0.133911 quality of care 0.379973 0.134291 religious objection 0.099916 0.035313 reporting burden 0.064406 0.022763 table 5 global priority of ig3 group judgments ahp factor local priority at level 1 sbe owners’ concern local priority at level 2 global priority. health insurance cost 0.755880 administrative cost 0.097479 0.073682 insurance premium 0.654540 0.494754 penalty 0.107995 0.081631 tax burden 0.139986 0.105812 health insurance coverage 0.244120 complexity of aca 0.159254 0.038877 plan choice 0.284469 0.069445 quality of care 0.399977 0.097643 religious objection 0.081127 0.019805 reporting burden 0.075173 0.018351 ijahp article: gupta, makrigeorgis/ prioritization of small business enterprise owners concerns about the affordable care act using the analytic hierarchy process international journal of the analytic hierarchy process 189 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.365 table 6 global priority of ig4 group judgments ahp factor local priority at level 1 sbe owners’ concern local priority at level 2 global priority. health insurance cost 0.645745 administrative cost 0.094409 0.060964 insurance premium 0.629278 0.406353 penalty 0.107846 0.069641 tax burden 0.168468 0.108787 health insurance coverage 0.354255 complexity of aca 0.087786 0.031099 plan choice 0.267709 0.094837 quality of care 0.518829 0.183798 religious objection 0.057251 0.020282 reporting burden 0.068425 0.024240 table 7 global priority of ig5 group judgments ahp factor local priority at level 1 sbe owners’ concern local priority at level 2 global priority. health insurance cost 0.568874 administrative cost 0.072226 0.041088 insurance premium 0.659648 0.375257 penalty 0.112333 0.063903 tax burden 0.155793 0.088627 health insurance coverage 0.431126 complexity of aca 0.093800 0.040440 plan choice 0.223619 0.096408 quality of care 0.520412 0.224363 religious objection 0.107506 0.046349 reporting burden 0.054663 0.023567 displaying the rankings for the ahp hierarchy by industry groups based on global priorities as shown in figure 4, the insurance premium was found uniformly to be the highest concern of sbe owners, with ranges between 38% and 50% in each industry group. ijahp article: gupta, makrigeorgis/ prioritization of small business enterprise owners concerns about the affordable care act using the analytic hierarchy process international journal of the analytic hierarchy process 190 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.365 figure 4. global priorities of group judgments for industry groups reporting burden was the lowest concern of sbe owners consistently at 2% in each industry group. the ranking order of the remaining concerns varied in each industry group exhibiting the diversity of sbe owner’s opinions. ishizaka et al. (2011) remarked that the reliability of the ahp technique is very high when consistent highest and lowest priority determination is established among participating groups. the results of sbe owner’s opinions confirmed the remarks of ishizaka et al. (2011) by identifying insurance premium and reporting burden as the highest and the lowest priority respectively in each industry group. finally, the synthesis of aggregated local priorities of all participants (industry groups ig1 through ig5) resulted in the global priorities or overall weights of group judgments by multiplying local priorities at level 1 and level 2 of the ahp structure. table 8 shows the global priorities for all participants. table 8 global priority of all participants group judgments ahp factor local priority at level 1 sbe owners’ concern local priority at level 2 global priority health insurance cost 0.670218 administrative cost 0.089940 0.060280 insurance premium 0.639574 0.428654 penalty 0.118023 0.079101 tax burden 0.152463 0.102183 health insurance coverage 0.329351 complexity of aca 0.107826 0.035512 plan choice 0.286489 0.094355 quality of care 0.447566 0.147406 religious objection 0.091796 0.030233 reporting burden 0.066324 0.021844 synthesizing the local priority weights into composite or global priority weights, figure 5 provides the overall picture of all sbe owners’ concerns in richmond, virginia. the insurance premium was the highest concern of all participants at 43% while reporting ijahp article: gupta, makrigeorgis/ prioritization of small business enterprise owners concerns about the affordable care act using the analytic hierarchy process international journal of the analytic hierarchy process 191 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.365 burden was the lowest concern at 2%. the gap between the insurance premium and reporting burden was over 21 times considering all concerns. the sbe owner’s concerns following insurance premium were quality of care, tax burden, and plan choice. figure 5 ranking of sbe owners’ concerns to the aca in fact, the insurance premium was almost 3 times a bigger concern than the next concern, quality of care. moreover, the bottom 3 concerns complexity of aca, religious objection, and reporting burden accounted for less than 10% of overall sbe owner’s concerns. insurance premium (43%) and quality of care (15%) were the only 2 concerns bigger than the average (11%) of all sbe owners’ concerns. 5. conclusions the application of ahp began with the identification of known sbe concerns about aca by utilizing a thorough literature review. from this literature, it was determined that sbe owners exhibited 9 specific concerns. these concerns were encapsulated in a pairwise comparison survey and administered to a sample of 50 sbe owners in richmond, virginia. next, the application of ahp on the collected surveys provided a structure and ranking of these 9 aca concerns to essentially verify their order of importance. specifically, the ahp analysis of the survey results revealed the following key findings:  among health insurance cost concerns, the insurance premium was the topranked concern followed by the tax burden.  among health insurance coverage concerns, quality of care was the topranked concern followed by plan choice.  among all participants, the top ranked concern was insurance premium followed by quality of care, tax burden, and plan choice. in addition, the bottom ranked concerns were the complexity of aca, religious objection, and reporting burden. ijahp article: gupta, makrigeorgis/ prioritization of small business enterprise owners concerns about the affordable care act using the analytic hierarchy process international journal of the analytic hierarchy process 192 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.365  among industry groups, the overall rankings of the concerns were not uniform. however, the insurance premium was the top-ranked and reporting burden, the bottom ranked concern of sbe owners in each industry group. the recommendations to sbe owners, business organizations, u.s. government officials, and researchers are to consider the efficacy of the findings for seeking better ways to reduce sbe owner’s concerns. the prioritization of sbe owner’s concerns to the aca also provides an opportunity for the u.s. health care industry to consider the key concerns for reducing their impact on businesses in the future. u.s. government officials could use these findings to disseminate useful informational materials to the business community for improving the perceptions of sbe owners. because of the changes to health insurance coverage from the aca, sbe owners could also utilize a representative voice to highlight those issues that are crucial to the financial success of a business. sbe owners could leverage this voice to influence the provisions of the aca benefiting the entrepreneurs, employees, and society. sbe owners could seek to redress their concerns from appropriate authorities and find ways to mitigate any financial risks attributable to the concerns. to further the study, researchers could conduct studies with a broader sample drawn from a population from various locations and other industry groups such as oil, manufacturing, and transportation, which were not prevalent in richmond, virginia. researchers could also perform a correlational study involving key sbe owner’s concerns and profitability. the findings from such further research could reveal the impact on profit margins of sbes from key concerns to devise better approaches for mitigating risks. in addition, researchers could apply some simpler rank-generation techniques such as competition, ordinal, or fractional ranking techniques for comparison to ahp. if the techniques yield similar results, then researchers need not apply a complex mcdm technique such as ahp to similar research problems. finally, given that insurance premium was the topranked sbe concern toward the aca, researchers could further consider its sub-factors to understand the impact of the dominant sub-factor on profit margins. we conducted the study to address the problem of prioritizing sbe owner’s concerns and to foster a more shared understanding across industry groups and more importantly within a particular industry group. examining the concerns about the aca with the analytic hierarchy process (ahp) filled a gap in the literature by ranking the concerns. the quantitative research study was conclusive, descriptive, and cross-sectional. the findings revealed the top ranked sbe owner’s concern was insurance premium followed by quality of care and tax burden. the findings from this study offer sbe owners benefit to focus on critical concerns for reducing business costs of health care. moreover, business organizations, researchers, and policymakers could channel sbe owner’s voice for positive social change to address business concerns seeking improvements from the aca. ijahp article: gupta, makrigeorgis/ prioritization of small business enterprise owners concerns about the affordable care act using the analytic hierarchy process international journal of the analytic hierarchy process 193 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.365 references al-hawari, t., al-bo'ol, s., & momani, a. (2011). selection of temperature measuring sensors using the analytic hierarchy process. jordan journal of mechanical & industrial engineering, 5, 451–459. retrieved from http://jjmie.hu.edu.jo/index.htm aminbakhsh, s., gunduz, m., & sonmez, r. (2013). safety risk assessment using analytic hierarchy process (ahp) during planning and budgeting of construction projects. journal of safety research, 46, 99–105. doi:10.1016/j.jsr.2013.05.003 aull-hyde, r., erdogan, s., & duke, j. m. (2006). an experiment on the consistency of aggregated comparison matrices in ahp. european journal of operational research, 171, 290–295. doi:10.1016/j.ejor.2004.06.037 bhatt, r., & macwan, j. e. m. (2012). global weights of parameters for sustainable buildings from consultants’ perspectives in indian context. journal of architectural engineering, 18, 233–241. doi:10.1061/(asce)ae.1943-5568.0000069 buchmueller, t., carey, c., & levy, h. g. (2013). will employers drop health insurance coverage because of the affordable care act? health affairs, 32, 1522–1530. doi:10.1377/hlthaff.2013.0526 cordell, d. m., & langdon, t. p. (2011). health care planning and the new reality. journal of financial planning, 24(1), 38–40. retrieved from http://www.fpanet.org/journal/ cordell, d. m., & langdon, t. p. (2012). financial planning under the affordable care act. journal of financial planning, 25(9), 30–31. retrieved from http://www.fpanet.org/journal/ curfman, g. d., abel, b. s., & landers, r. m. (2012). supreme court review of the health care reform law. new england journal of medicine, 366, 977–979. doi:10.1056/nejmp1114933 eyal-cohen, m. (2013). downsizing the little guy myth in legal definitions. iowa law review, 98, 1041–1099. retrieved from http://www.uiowa.edu/~ilr/ gable, l. (2011). the patient protection and affordable care act, public health, and the elusive target of human rights. the journal of law, medicine & ethics, 39, 340–354. doi:10.1111/j.1748-720x.2011.00604.x gardner, d. b. (2012). the future of the affordable care act: will we abandon health care reform? nursing economics, 30, 40–41, 49. retrieved from http://www.nursingeconomics.net ijahp article: gupta, makrigeorgis/ prioritization of small business enterprise owners concerns about the affordable care act using the analytic hierarchy process international journal of the analytic hierarchy process 194 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.365 geyman, j. p. (2012). cost‐sharing under consumer‐driven health care will not reform us health care. the journal of law, medicine & ethics, 40, 574–581. doi:10.1111/j.1748-720x.2012.00690.x goepel, k. d. (2013). implementing the analytic hierarchy process as a standard method for multi-criteria decision making in corporate enterprises: a new ahp excel template with multiple inputs. proceedings of the international symposium on the analytic hierarchy process 2013, 1–10. retrieved from http://www.isahp.org/proceedings/ healthcare. (n.d.). what is considered a small business? retrieved from https://www.healthcare.gov/what-is-considered-a-small-business/ hellander, i., & bhargavan, r. (2012). report from the united states: the u.s. health crisis deepens amid rising inequality: a review of data, fall 2011. international journal of health services, 42, 161–175. doi:10.2190/hs.42.2.a ishizaka, a., balkenborg, d., & kaplan, t. (2011). does ahp help us make a choice? an experimental evaluation. journal of the operational research society, 62, 1801–1812. doi:10.1057/jors.2010.158 ishizaka, a., & labib, a. (2011). review of the main developments in the analytic hierarchy process. expert systems with applications, 38, 14336–14345. doi:10.1016/j.eswa.2011.04.143 jacobe, d. (2013a). half of u.s. small businesses think health law bad for them. gallup economy. retrieved from http://www.gallup.com/poll/162386/half-small-businessesthink-health-law-bad.aspx jacobe, d. (2013b). health care costs, taxes worry u.s. small businesses most. gallup economy. retrieved from http://www.gallup.com/poll/160100/health care-costs-taxesworry-small-businesses.aspx kapur, k., karaca‐mandic, p., gates, s. m., & fulton, b. (2012). do small‐group health insurance regulations influence small business size? journal of risk and insurance, 79, 231–260. doi:10.1111/j.1539-6975.2011.01421.x lepard, k. (2013). standing their ground: corporations’ fight for religious rights in light of the enactment of the patient protection and affordable care act contraceptive coverage mandate. texas tech law review, 45, 1041–1071. retrieved from http://texastechlawreview.org litwin, s., & phan, p. h. (2013). quality over quantity: reexamining the link between entrepreneurship and job creation. industrial & labor relations review, 66, 833–873. retrieved from http://www.ilr.cornell.edu/ilrreview/ doi: 10.1177/001979391306600405 loewentheil, k. (2014). when free exercise is a burden: protecting "third parties" in religious accommodation law. drake law review, 62, 433–502. retrieved from http://works.bepress.com/kara_loewentheil/1 ijahp article: gupta, makrigeorgis/ prioritization of small business enterprise owners concerns about the affordable care act using the analytic hierarchy process international journal of the analytic hierarchy process 195 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.365 mcmorrow, s., blumberg, l. j., & buettgens, m. (2011). the effects of health reform on small businesses and their workers. the urban institute publications. retrieved from http://www.urban.org/ miller, a. r., eibner, c., & gresenz, c. r. (2013). financing of employer sponsored health insurance plans before and after health reform: what consumers don’t know won’t hurt them? international review of law and economics, 36, 36–47. doi:10.1016/j.irle.2013.04.005 miller, e. (2011). affordability of health insurance to small business: implications of the patient protection and affordable care act. journal of health politics, policy & law, 36, 539–546. doi:10.1215/03616878-1271225 oberlander, j. (2012). unfinished journey: a century of health care reform in the united states. new england journal of medicine, 367, 585–590. doi:10.1056/nejmp1202111 parks, d. (2011). health care reform simplified. doi:10.1007/978-1-4302-3699-3 ramanathan, r. (2001). a note on the use of the analytic hierarchy process for environmental impact assessment. journal of environmental management, 63, 27–35. doi:10.1006/jema.2001.0455 saaty, t. l., & shang, j. s. (2011). an innovative orders-of-magnitude approach to ahp-based multi-criteria decision making: prioritizing divergent intangible humane acts. european journal of operational research, 214, 703–715. doi:10.1016/j.ejor.2011.05.019 saaty, t. l., & vargas, l. g. (2012). the possibility of group choice: pairwise comparisons and merging functions. social choice and welfare, 38, 481–496. doi:10.1007/s00355-011-0541-6 tacchino, a. (2013a). a new way of thinking about employer-sponsored health care coverage strategies. journal of financial planning, 26(7), 48–55. retrieved from http://www.fpanet.org/journal tacchino, a. (2013b). the affordable care act’s ‘play or pay’ tax: determining coverage alternatives. journal of financial planning, 26(2), 38–42. retrieved from http://www.fpanet.org/journal talib, f., rahman, z., & qureshi, m. n. (2011). prioritising the practices of total quality management: an analytic hierarchy process analysis for the service industries. total quality management & business excellence, 22, 1331–1351. doi:10.1080/14783363.2011.625192 u.s. department of health and human services. (n.d.). key features of the affordable care act by year. retrieved from http://www.hhs.gov ijahp article: gupta, makrigeorgis/ prioritization of small business enterprise owners concerns about the affordable care act using the analytic hierarchy process international journal of the analytic hierarchy process 196 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.365 u.s. small business administration. (n.d.). frequently asked questions. retrieved from https://www.sba.gov/advocacy/frequently-asked-questions-aboutsmall-business doi: 10.18356/79c110b0-en wilensky, g. r. (2012). the shortfalls of “obamacare.” new england journal of medicine, 367, 1479–1481. doi:10.1056/nejmp1210763 ijahp article: gupta, makrigeorgis/ prioritization of small business enterprise owners concerns about the affordable care act using the analytic hierarchy process international journal of the analytic hierarchy process 197 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.365 appendix the following tables resulted from the responses of participant 1 in ig1 labelled ig1-1. the responses to pairwise comparison questions by this particular participant for survey part 1 were consistent with cr=0% indicating no inconsistencies at all. next, the ahp analysis of the participant’s responses for part 1 of the survey yielded the 2×2 matrix. the normalized principal eigenvector or local priority shows the local ranking of the survey part 1 at level 1 of the ahp hierarchy. a 2×2 matrix from participant ig1-1 response to survey part 1 ahp criteria health insurance cost health insurance coverage normalized principal eigenvector local priority health insurance cost 1 5 0.833333 83% health insurance coverage 0.2 1 0.166667 17% cr = 0.00 next, the survey part 2 for ig1-1 showed a cr=7% and a 4×4 matrix as follows. the normalized principal eigenvector or local priority shows the local ranking of survey part 2 at level 2 of the ahp hierarchy. a 4×4 matrix from participant ig1-1 response to survey part 2 sbe owners’ concern administrative cost insurance premium penalty tax burden normalized principal eigenvector local priority administrative cost 1 0.2 0.333333 0.333333 0.073637 7% insurance premium 5 1 3 5 0.549502 55% penalty 3 0.333333 1 3 0.247618 25% tax burden 3 0.2 0.333333 1 0.129244 13% cr = .07 the survey part 3 yielded a 5×5 matrix as follows. the normalized principal eigenvector or local priority shows the local ranking of survey part 3 at level 2 of the ahp hierarchy. a 5×5 matrix from participant ig1-1 response to survey part 3 sbe owners’ concern complexity of aca plan choice quality of care religious objection reporting burden normalized principal eigenvector local priority complexity of aca 1 5 4 0.5 5 0.304375 30% plan choice 0.2 1 0.5 0.2 5 0.088876 9% quality of care 0.25 2 1 0.2 5 0.119499 12% religious objection 2 5 5 1 9 0.452127 45% reporting burden 0.2 0.2 0.2 0.111111 1 0.035124 4% cr = 0.08 ijahp article: gupta, makrigeorgis/ prioritization of small business enterprise owners concerns about the affordable care act using the analytic hierarchy process international journal of the analytic hierarchy process 198 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.365 a synthesis of normalized principal eigenvectors or local priority weights following the ahp hierarchical structure yielded the global priority weights of the responses of participant ig1-1 as follows. global priority of participant ig1-1 judgments ahp factor local priority at level 1 sbe owners’ concern local priority at level 2 global priority. health insurance cost 0.833333 administrative cost 0.073637 0.061364 insurance premium 0.549502 0.457919 penalty 0.247618 0.206348 tax burden 0.129244 0.107703 health insurance coverage 0.166667 complexity of aca 0.304375 0.050729 plan choice 0.088876 0.014813 quality of care 0.119499 0.019917 religious objection 0.452127 0.075354 reporting burden 0.035124 0.005854 if the fines content is greater than 3%, the fines offine or all-in aggregate shall be considered non-harmful (e ijahp article: li, wilson, larkin/marginal material evaluation and selection using analytic hierarchy process model international journal of the analytic hierarchy process 64 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13022/ijahp2017.v9.is1.358 marginal material evaluation and selection using analytic hierarchy process model wentao li university of auckland new zealand wli783@aucklandunil.ac.nz doug wilson university of auckland new zealand dj.wilson@auckland.ac.nz tam larkin university of auckland new zealand t.larkin@auckland.ac.nz 1 abstract marginal materials, also called sub-standard materials, have the potential to replace premium materials in local roads. however, the current definition of marginal materials suffers from the limitation of focusing on whether or not each single property meets the corresponding requirement of specifications rather than reflecting the overall performance of the materials. to overcome this limitation and to better understand the concept of ‘marginal material’, this study was conducted using the analytic hierarchy process (ahp) framework to evaluate the overall performance of five aggregates and an assumed boundary aggregate based on multiple factors (various engineering properties and performance). the aggregates were ranked through comparing the overall weight of each material, which was obtained based on the analysis of the relative weights of criteria and sub-criteria along with data processing of engineering properties. the ahp model is a good method to select the best aggregates within a number of given aggregates. it can describe the overall performance of aggregates in a quantitative way, which allows the qualities of the aggregates to be compared to each other so that the proper aggregates can be selected for different road construction purposes. the validation of the ahp model demonstrates that the ahp analyzed qualities of the aggregates match well to their qualities in field road construction, but there is a need 1 acknowledgement: the authors would like to thank the china scholarship council (csc) and new zealand mbie programme for funding mr. wentao li’s phd study at the university of auckland. mailto:wli783@aucklandunil.ac.nz mailto:dj.wilson@auckland.ac.nz mailto:t.larkin@auckland.ac.nz ijahp article: li, wilson, larkin/marginal material evaluation and selection using analytic hierarchy process model international journal of the analytic hierarchy process 65 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13022/ijahp2017.v9.is1.358 to make a combination analysis on the individual properties (specification pass/fail criteria) and overall performance (ahp model) in the process of evaluating the quality of aggregates. keywords: analytical hierarchy process; relative weights; marginal aggregates; road construction; 1. introduction marginal materials, also called sub-standard materials, are “those that do not entirely meet the specifications for normal road materials to be used in a country or region, but which still have the potential to be used successfully in some applications” (brunschwig, 1989). high-quality aggregate materials are being depleted and there are many environmental and other restrictions being placed on the expansion of existing quarries and on the exploitation of new resources. thus, there is now a strong economic and sustainability imperative to preserve premium aggregates for use only where this quality is required, and to better use local marginal materials in appropriate in-service conditions and/or to improve their engineering performance with special treatments. however, local marginal materials are not defined clearly. apart from brunschwig’s definition, another definition by brennan (1984) is that a marginal grade aggregate is an aggregate produced from a more weathered or weather prone rock, or hard rock containing weathered seams or weaker sedimentary rocks, which after processing contain moderate or highly plastic fines, is sensitive to weathering and when compacted will produce a soaked california bearing ratio (cbr) value between 40% and 100%. the two qualitative definitions concentrate on the single properties performance of marginal aggregates (e.g. a property failing to meet a specific specification rather than the overall quality of marginal aggregates). these pass /fail definitions may make some qualified materials appear to be unqualified marginal materials, which results in their failure to be used in roads, causing further waste of materials. for example, if a specific property of a material is slightly lower than the requirement of standards/specifications but the other properties are much higher than the corresponding requirements, the material would be automatically categorized as a ‘marginal’ material. however, this could be a controversial conclusion when considering the repeatability and reproducibility of test methods in standards/specifications which allow a range value for the specific property of the material (astm, 2002; bsi, 2013). furthermore, the recipe specifications result from the classical empirical engineering approach of design on the basis of long-term monitoring of full-scale test roads and index tests (evans & vuong, 2003; rogers, fleming, & frost, 2004). empirical standards/specifications cannot completely assure in-field performance, especially when the conditions change (e.g. loading magnitudes or patterns change, novel materials are proposed, etc.). therefore, the pass/fail requirements of specifications/standards for materials only considering single index properties cannot accurately differentiate marginal materials from premium materials. there is a need to develop a statistical method which can combine the multiple properties and provide an overall evaluation on the quality of the materials. ijahp article: li, wilson, larkin/marginal material evaluation and selection using analytic hierarchy process model international journal of the analytic hierarchy process 66 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13022/ijahp2017.v9.is1.358 the next challenge is how to rank materials based on multiple properties. in other words, the difference in the quality between materials needs to be identified quantitatively so that people can understand which material is better and which is worse when considering performance. general standards/specifications set requirements for materials in single parameter performance of materials (e.g. each property) rather than the overall combined performance of materials. therefore, results are obtained for each property without any effect on other properties if tests are conducted following the current standards/specifications. for example, assume that there are two materials, with a cbr value of 78% and 82% (pass criteria >80), and a sand equivalent (se) value of 42 and 38 (pass criteria >40), respectively. it cannot be concluded which material is better by only comparing their cbr or se test results alone. the better way is to combine the two material properties to obtain the overall performance of each material and then make a more rational decision. the above evidence illustrates that the pass/fail specifications/standards for materials only considering single index properties cannot accurately differentiate marginal materials from premium materials. there is, therefore, a need to develop a mathematical model which can combine the multiple properties and provide an overall evaluation on the quality of the materials so that the materials can be ranked and marginal materials can be identified. specifically, there are tools that achieve the need to integrate the multiple properties of materials. one of these tools is a multicriteria decision-making tool called the analytic hierarchy process (ahp). this research was undertaken to develop a multi-factor mathematical model using an ahp framework to assess the overall performance and ranking of aggregate materials. the objective was to define marginal materials quantitatively, and further to advance the knowledge and understanding of marginal materials using an ahp model. 2. methods and materials 2.1. analytic hierarchy process (ahp) the analytic hierarchy process (ahp) developed by saaty (1980) is a multiple criteria decision-making tool that allows subjective and objective factors to be considered in a decision-making process. it is used to determine the relative weights of selected criteria and sub-criteria in order to obtain an assessment on given alternatives. saaty (1980) established 9 as the upper limit and 1/9 as the lower limit in his scale, which ranges from 1/9 for ‘least important than’, to 1 for ‘equal’, and to 9 for ‘absolutely more important than’ covering the entire spectrum of the comparison. the ahp has been used in a wide range of areas, including engineering, social sciences, and economics (roux iii & makrigeorgis, 2016; saaty & vargas, 2001; strojny & hejman, 2016). a literature overview gives a detailed summary about the application of ahp, which has extended to education, manufacturing, personal and political areas (vaidya & kumar, 2006). some key and basic steps involved in this ahp methodology are (saaty & vargas, 2001) : ijahp article: li, wilson, larkin/marginal material evaluation and selection using analytic hierarchy process model international journal of the analytic hierarchy process 67 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13022/ijahp2017.v9.is1.358 1. identify the overall goal. state the main problem. in this study, the overall goal is to rank all the given materials and further to identify the marginal materials from the given materials. 2. broaden the objectives of the main problem or consider all actors, objectives and its outcome. 3. identify the criteria that must be satisfied in order to fulfill the overall goal. in this study, the goal is to rank all the given materials. according to the new zealand specification, each material in this study is evaluated based on seven engineering properties (stated in section 2.2). therefore, the criteria used to characterize the goal are the seven engineering properties. 4. develop a hierarchy of different levels constituting goal, criteria, sub-criteria and alternatives based on structuring the problem. 5. develop the pairwise comparison matrix (priority matrix) for each level. compare each element in the corresponding level, and then calibrate them on the numerical scale. this requires comparisons, where n is the number of elements with the considerations that diagonal elements of the matrix are equal or ‘1’ and the other elements will simply be the reciprocals of the earlier comparisons. 6. do a consistency test. calculate to find the maximum eigen value λmax, consistency index ci, consistency ratio cr, and normalized values for each criteria/alternative. if the maximum eigen value, ci, and cr are satisfactory then the decision is made based on the normalized values; otherwise the procedure is repeated until these values lie in a desired range. 2.2. introduction to engineering properties (factors) of materials in new zealand, the new zealand transport agency (nzta) m4 specification is the reference or standard specification that sets out requirements for premium basecourse aggregate for use on highways and other heavily trafficked roadways (nzta, 2006). each material property is introduced as follows: crushing resistance: in the aggregate industry the crushing resistance (cr) test provides the relative measure of rock strength. new zealand standards specify that it measures the percentage of fines produced by a specified load which is either 130 kn for aggregate to be used as base course (nzs, 1991), or if aggregates for concrete the number of kilonewtons which produce 10% fines passing a 2.36mm sieve (standard, 1986). california bearing ratio: the california bearing ratio (cbr) provides a measure of resistance of materials to penetration of a standard plunger under controlled density and moisture conditions. in some cases, the soaked cbr test of materials (at least 4 days soaking) is conducted to simulate load-bearing capacity of materials in rainy weather. a soaked cbr of more than 80 is specified in new zealand standard (nzta m4 specification). although it is widely acknowledged as being not wholly satisfactory when used as a performance parameter, cbr has been correlated with pavement performance in many countries over many years and provides a reliable empirical indicator of material behavior (rogers et al., 2004). weathering quality index : the weathering quality index (wqi) provides a method to assess the resistance of an aggregate to the effects of wetting, drying, heating and ijahp article: li, wilson, larkin/marginal material evaluation and selection using analytic hierarchy process model international journal of the analytic hierarchy process 68 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13022/ijahp2017.v9.is1.358 cooling (nzs, 1991). the wqi test involves the rolling force and the expansion/contraction forces of existing clays to physically disaggregate the aggregate thus it is really a measure of the degree of lithification of the material and the nature of its matrix (black, 2009). the wqi consists of the aggregate percentage retained on a 4.75mm sieve fraction and the cleanness value and might be assessed as anyone of aa, ab, ba, ac, ca, bb, bc, cb, and cc. table 1 shows the requirements for weathering quality index specified by nzta m4 specification. ijahp article: li, wilson, larkin/marginal material evaluation and selection using analytic hierarchy process model international journal of the analytic hierarchy process 69 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13022/ijahp2017.v9.is1.358 table 1 requirements for weathering quality index in nzta m4 specification cleanness value percentage retained on 4.75mm sieve 96 to 100 91 to 95 up to 90 91 100 aa ba ca 71 90 ab bb cb up to 70 ac bc cc although the wqi test is designated to measure the original source rock’s degree of weathering, other variables, such as rock matrices and multistage processing (designed to select tougher, high crushing resistance and more durable parts of rocks) also significantly influence the wqi values of rocks, even having a predominant control on the wqi (black, 2009). thus, the wqi, in some cases, is not an appropriate test to determine the level of weathering. sand equivalent: the sand equivalent (se) method covers a field or laboratory test for measuring the relative amounts of silt or clay size particles in fine aggregates or fine fractions of aggregates (nzs, 1991). the se test method is regarded as a rapid method for detecting the presence or absence of detrimental fines or clay-like materials in soils and mineral aggregates; it has been used for over 60 years and is still being used even though there are a number of problems with it (hveem, 1953. black (2009) reported that the crushing regime and the density, size and shape of the sediment particles can cause inaccurate results about percentage of the clay size fraction or the presence or proportion of clay minerals in the material in se test. other researchers also comment on the risk of a poor material being classified as acceptable, and conversely good material being rejected on the basis of a low se value (sameshima, 1977; van barneveld, bartley, & dunlop, 1984). clay index: the clay index (ci) test outlines the method for a methylene blue titration test used to estimate the percentage of expansive clay minerals in natural fines or rock powders (nzs, 1991). the ci test is considered to be a quicker and more cost effective production test than the x-ray diffraction (xrd) or differential thermal analysis (dta) methods (cole & sandy, 1980; stapel & verhoef, 1989). however, all minerals or substances present which have exchangeable cations (ie zeolites) will result in a high ci test results causing a wrong assessment for the material quality (stapel & verhoef, 1989). plasticity index: the plasticity index (pi) method covers the determination of the plasticity of the fine fraction less than 0.425mm of an aggregate (nzs, 1991). this test is derived from a group of tests collectively known as atterberg limits, including shrinkage limit test, liquid limit test, plastic limit test, pi test, and liquidity index test. the plasticity index is the size of the range of water contents where the soil exhibits plastic properties. the pi is the difference between the liquid limit and the plastic limit (pi=ll (liquid limit)-pl (plastic limit)). the determination of the liquid and plastic limits is very subjective and dependent on the experience of the tester (black, 2009). as a consequence, there has been criticism of the plasticity index test (prowell, zhang, & brown, 2005). the clay mineral particle size has a significant impact on its plasticity. for example, highly fine grained illite and kaolinite minerals are strongly plastic while larger grain sizes generally have very low plasticity (black, 2009). ijahp article: li, wilson, larkin/marginal material evaluation and selection using analytic hierarchy process model international journal of the analytic hierarchy process 70 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13022/ijahp2017.v9.is1.358 particle size distribution: the particle size distribution (psd) is a wet or dry sieving test which can help to develop the interlock between particles so that aggregates have enough strength to resist repeated loads. the cumulative weights of material passing the standard set of sieves are recorded and then the cumulative psd curve can be obtained by displaying the results on a grain size versus percentage passing each individual sieve graph. the tested aggregate’s cumulative particle size curve is used to evaluate the psd grade of aggregates through being compared to a defined desirable particle size distribution envelope. psd is very dependent on both the crushing regime and the strength of the material. “crushed volcanic rocks consistently fall within the allowable particle size distribution envelope while many types of greywacke have difficulty doing so” (black, 2009). 2.3. application of ahp method the ahp framework was used to rate the overall performance of the aggregates used in this research based upon the engineering properties and furthermore to identify the aggregates qualities. in the process, a ‘boundary’ aggregate is assumed to be the boundary between premium and marginal aggregates. any aggregate with higher (or lower) overall performance than the ‘boundary’ aggregate would be a premium (or marginal) aggregate. each engineering property of the ‘boundary’ aggregate is assigned a qualified value with the minimum requirement specified by the nzta m4 specification. for example, with the california bearing ratio (cbr) test, nzta m4 specification (2006) specifies that m4 aggregates must have a cbr value of not less than 80. hence, the cbr value of 80 is assigned to the ‘boundary’ aggregate. any aggregate with higher overall performance (i.e. overall weights) than the ‘boundary’ aggregate condition in the ahp framework is regarded as a premium aggregate, and any with lower overall performance than the ‘boundary’ aggregate is regarded as a marginal aggregate. the ‘real’ five aggregates were tested with respect to nzta m4 specification. test data obtained for different properties were transformed into the same unit to make them relatively comparable to each other. the data of ‘boundary’ aggregate conditions were processed with the same method. the data process is shown in section 3 ‘results and discussion’. the following steps were adopted to rank the materials: 1. identify the overall goal. in this study, the overall goal or the main problem is to rank all the given materials, and further to identify the marginal materials from the given materials. 2. identify the criteria that must be satisfied in order to fulfil the overall goal. in this study, the goal is to rank all the given materials. according to the new zealand specification, each material in this study is evaluated based on seven engineering properties (stated in section 2.2). therefore, the criteria used to characterize the goal are the seven engineering properties. 4. develop a hierarchy of different levels constituting goal, criteria, sub-criteria and alternatives based on structuring the problem. the criteria wqi and psd are divided ijahp article: li, wilson, larkin/marginal material evaluation and selection using analytic hierarchy process model international journal of the analytic hierarchy process 71 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13022/ijahp2017.v9.is1.358 into sub-criteria as wqi are directly determined by the result of ‘percentage retained on 4.75mm sieve’ and ‘cleanness value’, and psd by the results of ten sieve apertures. to achieve the goal (finding marginal aggregate materials), six aggregates were provided as alternatives. figure 1 shows the evaluation model on the overall performance of aggregates given multiple factors in the ahp framework. level i is the goal – selecting marginal aggregates based on ranking the six aggregates given multiple properties. levels ii and iii are the criteria and sub-criteria considered for the selection. level iv is the alternatives, six aggregates. level i level ii level iii level iv figure 1. ahp model for evaluation of the overall performance of aggregates given multiple factors ranking of aggregates cr cbr psd pi ci se wqi aggregate 1 boundary aggregate (nzta m4 requirement) aggregate 2 aggregate 3 aggregate 4 aggregate 5 goal criteria alternatives 4.75 0.15 19 2.36 9.5 37.5 1.18 0.6 0.3 0.075 sub-criteria cleanness value percentage retained on 4.75mm sieve ijahp article: li, wilson, larkin/marginal material evaluation and selection using analytic hierarchy process model international journal of the analytic hierarchy process 72 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13022/ijahp2017.v9.is1.358 there are two types of measurements when ranking alternatives involved in the ahp, relative and absolute (saaty, 1986; saaty, 1980). the first ranks a few alternatives by comparing them in pairs and is particularly useful in new and exploratory decisions. the latter rates a number of alternatives by comparing them with a standard in memory developed through experience. it is particularly useful in decisions where there is considerable knowledge to judge the relative importance of the intensities and develop priorities for them (saaty & vargas, 2001). in this case, the aggregates (alternatives) are evaluated using absolute measurement as each property of aggregates (criteria) are tested according to standard testing methods and the corresponding testing values (the ratings of the criteria or subcriteria) are assigned to them. saaty and vargas (2001) explain that absolute measurement, sometimes called scoring, is applied to rank the alternatives (e.g. aggregates) with regard to either the criteria and sub-criteria (e.g. properties of aggregates) or the ratings of the criteria and sub-criteria (e.g. the test results for each property). however, relative measurement is applied to obtain the relative weights of aggregate properties (criteria and sub-criteria) as shown in tables 6, 7 and 8. 2.4. aggregates this paper includes five aggregates from two hard rock greywacke quarries in the auckland – waikato region of new zealand’s north island in terms of their physical and geological properties and four assumed ‘boundary aggregates’. the materials studied were aggregates with a maximum particle size of 40 mm (ap 40) which can be categorized as either premium, or of marginal quality. within the marginal classification there are three aggregates from two quarries which will be considered here, three marginal aggregates and two premium aggregates (the two m4 aggregates). in the following part, these five aggregates are named as m4 aggregate 1 (m4 aggregate from quarry 1), m4 aggregate 2 (m4 aggregate from quarry 2), marginal aggregate 1a and 1b (both from quarry 1) and marginal aggregate 2 (from quarry 2) in accordance with their utilization in field road construction. for example, the two m4 aggregates are applied to basecourse where the premium quality aggregates are needed whilst the three marginal aggregates are applied to low-volume roads as substandard aggregates. engineering property tests were conducted according to nzs (new zealand standard) 4407: 1991 (nzs, 1991). the qualities of the five aggregates were monitored by the quarries and the results were recorded every month from 2010 to 2013. the data of each property during each year were averaged and then listed in tables 2, 3, 4 and 5, of which tables 4 and 5 show the results in 2013 to demonstrate how to process the data in the ahp model. the results of data processing in the other three years will be given in the ‘results and discussion’ section. note that n/a is filled in the tables for the missing data of m4 aggregate 2 in 2010 and 2011. ijahp article: li, wilson, larkin/marginal material evaluation and selection using analytic hierarchy process model international journal of the analytic hierarchy process 73 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13022/ijahp2017.v9.is1.358 table 2 testing results of the aggregates in 2010 to 2012 m4 aggregate1 m4 aggregate2 marginal aggregate 1a marginal aggregate 1b marginal aggregate 2 california bearing ratio (cbr/%) 2010 278 n/a 240 173 125 2011 291 n/a 224 105 165 2012 261 213 194 165 175 crushing resistance (%) 2010 1.1 n/a 2.6 3.6 3.8 2011 1.1 n/a 2.7 2.8 2.7 2012 1.4 2.9 2.0 2.3 3.0 weathering quality index 2010 percentage retained on 4.75mm sieve 98 n/a 93 94 89 cleanness value 97 n/a 89 89 79 2011 percentage retained on 4.75mm sieve 98 n/a 93 92 91 cleanness value 96 n/a 90 95 75 2012 percentage retained on 4.75mm sieve 95 93 93 94 88 cleanness value 91 87 88 98 79 sand equivalent 2010 48 n/a 43 43 38 2011 48 n/a 40 42 41 2012 51 49 39 42 40 clay index 2010 1.0 n/a 2.0 2.6 3.5 2011 1.4 n/a 1.9 2.4 2.9 2012 1.1 1.9 2.0 2.3 3.3 plasticity index 2010 6 n/a 8 11 12 2011 3 n/a 9 9 12 2012 3 7 8 11 11 ijahp article: li, wilson, larkin/marginal material evaluation and selection using analytic hierarchy process model international journal of the analytic hierarchy process 74 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13022/ijahp2017.v9.is1.358 table 3 psd results of aggregates in 2010 to 2012 na 37.5mm 19mm 9.5mm 4.75mm 2.36mm 1.18mm 0.6mm 0.3mm 0.15mm 0.075mm m4 aggregate 1 2010 98.8 73.0 53.1 38.1 25.5 15.9 10.4 6.9 5.2 4.2 2011 99.1 76 52.3 37.6 24.4 15.4 9.9 6.7 5.1 4 2012 99.3 76 53.9 41.2 28.6 18.1 11.4 7.5 5.6 4.5 m4 aggregate 2 2010 n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a 2011 n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a 2012 99.8 73 49.4 36.4 23.5 15.1 10.2 7.4 5.8 4.8 marginal aggregate 1a 2010 99.4 71.2 48.3 33.6 22.2 14.5 10.0 7.1 5.5 4.4 2011 99.5 76.1 52.4 35.9 23.9 15.8 11.1 8.2 6.5 5.4 2012 98.9 72.9 50.0 35.4 23.1 15.4 10.5 7.8 6.1 4.9 marginal aggregate 1b 2010 100 76.7 46.0 25.3 16.7 10.7 8.3 6.3 5.0 4.7 2011 99.3 71.3 44.7 26.0 16.3 11.0 8.0 6.3 5.3 4.3 2012 99.3 71.3 47.0 29.7 18.7 12.7 9.3 7.3 6.0 5.0 marginal aggregate 2 2010 100.0 80.5 51.7 31.7 20.8 14.2 10.8 8.2 6.2 4.5 2011 99.8 77.2 46.8 27.8 18.0 12.5 9.0 7.0 5.8 4.7 2012 99.7 68.3 40.8 24.8 16.0 11.0 8.2 6.5 5.2 4.2 table 4 engineering property data for the five aggregate materials studied na california bearing ratio (cbr/%) crushing resistance (%) weathering quality index sand equivalent clay index plasticity index requirement by nzta m4 specification 80 maximum 10%@130kn percentage retained on 4.75mm sieve cleanness value 40 (minimum) maximum 3.0 maximum 5 or nonplastic (np) m4 aggregate1 265 p 0.6 p 98 p 98 p 58 p 2.0 p 6 n m4 aggregate2 190 p 2.6 p 94 p 92 p 43 n 1.7 p 4 n marginal aggregate 1a 204 p 2.0 p 92 p 92 p 48 p 2.2 p 11 f marginal aggregate 1b 185 p 1.7 p 91 p 95 p 42 n 2.4 p 9 f marginal aggregate 2 165 p 3.6 p 90 p 85 p 38 n 2.9 n 13 f boundary aggregate 80 n 10 n 75 n 91 p 40 n 3.0 n 5 n note 1: the ‘p’ represents pass value. the ‘n’ represents near-failure value. the ‘f’ represents failure value. ijahp article: li, wilson, larkin/marginal material evaluation and selection using analytic hierarchy process model international journal of the analytic hierarchy process 75 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13022/ijahp2017.v9.is1.358 table 5 psd results of the aggregates and nzta m4 specification psd envelope na 37.5mm 19mm 9.5mm 4.75mm 2.36mm 1.18mm 0.6mm 0.3mm 0.15mm 0.075mm m4 aggregate 1 97.8 67 48.3 37 24 15.3 9.8 6.7 5.1 4 m4 aggregate 2 99.8 73.5 50.4 35.9 23.6 15.2 10.4 7.6 5.9 4.6 marginal aggregate 1a 99.0 71.8 48.5 33.8 21.8 14.2 10.0 7.3 5.8 4.8 marginal aggregate 1b 97.0 61.5 33.5 17.5 10.0 6.0 4.0 3.0 2.5 1.5 marginal aggregate 2 100.0 87.5 61.0 37.5 25.0 17.0 13.0 10.0 8.0 7.0 boundary aggregate 100 66-81 43-57 28-43 19-33 12-25 7-19 3-14 0-10 0-7 maximum 100 81 57 43 33 25 19 14 10 7 minimum 100 66 43 28 19 12 7 3 0 0 note 1: 0.075mm, 0.15mm, 0.3mm ..., 37.5mm are the test sieve apertures. note 2: the psd results of the four boundary aggregates are shown in an interval form because they are supposed to sit in the psd envelope specified by nzta m4 specification and any value in the specified envelope is supposed to be equally important. 3. results and discussion 3.1. the weights of engineering properties (criteria) as mentioned earlier, the priorities of criteria and sub-criteria (aggregate properties) were obtained through analyzing pairwise comparison matrices. the pairwise comparison was conducted through interviewing four experts in this field and averaging their scales for individual comparison. pairwise comparisons of homogeneous elements are made in the matrices with a 1-9 scale to represent the degree of importance as shown in tables 6, 7 and 8. table 6 shows the pairwise comparison matrix for material properties and the priority vector (relative weight of each property) in the last column. note in this study the priority vector/importance was worked out by using the programming tool ‘matlab’. clay index with the highest value of 0.259 is the most important criterion influencing the overall performance of aggregates. it is probably because the clay index can more accurately reflect the percentage of expansive clays and better respond to the weathering than the other properties (bartley et al., 2007). the priority vector is used to determine the final relative weight of each aggregate material, which will be discussed in ‘ranking of the materials’. ijahp article: li, wilson, larkin/marginal material evaluation and selection using analytic hierarchy process model international journal of the analytic hierarchy process 76 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13022/ijahp2017.v9.is1.358 table 6 pairwise comparison matrix for material properties na cr cbr wqi se ci pi psd priority vector cr 1 1 2 1/2 1/3 1/2 1/2 0.092 cbr 1 1 1 1 1/3 1/2 1/3 0.085 wqi 1/2 1 1 1 1/4 1/3 1/2 0.076 se 2 1 1 1 1/2 1 1/2 0.120 ci 3 3 4 2 1 2 1 0.259 pi 2 2 3 1 1/2 1 1 0.167 psd 2 3 2 2 1 1 1 0.202 the maximum eigenvalue λmax=7.198, ci (the consistency index) =0.033, and cr (consistency ratio) =0.025. tables 7 and 8 list the pairwise comparison matrices for the sub-criteria of wqi and psd, respectively. the pairwise comparison was also obtained through interviewing the four experts and averaging their scales for each pair comparison. the two subcriteria of wqi with the same value of 0.5 are equally important as shown in table 7. in table 8, test sieve aperture 2.36mm and 0.075mm are supposed to be the two most important sub-criteria of psd, both with a priority value of 0.173. it is probably because the test sieve aperture 2.36mm is considered as the boundary size between coarse and fine particles of aggregates, and 0.075mm is a key sieve aperture to measure the cleanness of aggregates. table 7 pairwise comparison matrix for sub-criteria of weathering quality index (wqi) na percentage retained on 4.75mm sieve cleanness value priority vector percentage retained on 4.75mm sieve 1 1 0.5 cleanness value 1 1 0.5 ci=0 and ri (random index) =0. ijahp article: li, wilson, larkin/marginal material evaluation and selection using analytic hierarchy process model international journal of the analytic hierarchy process 77 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13022/ijahp2017.v9.is1.358 table 8 pairwise comparison matrix for sub-criteria of particle size distribution (psd) mm mm 37.5 19 9.5 4.75 2.36 1.18 0.6 0.3 0.15 0.075 importance 37.5 1 2 1 1/3 1/3 1/2 1/2 1/2 1/2 1/3 0.051 19 1/2 1 1/2 1/5 1/5 1/3 1/3 1/4 1/3 1/5 0.029 9.5 1 2 1 1/3 1/3 1 1/2 1 1/2 1/3 0.059 4.75 3 5 3 1 1 2 2 2 2 1 0.166 2.36 3 5 3 1 1 2 3 2 2 1 0.173 1.18 2 3 1 1/2 1/2 1 1 1 1/2 1/2 0.079 0.6 2 3 2 1/2 1/3 1 1 1 1/2 1/3 0.079 0.3 2 4 1 1/2 1/2 1 1 1 1 1/2 0.087 0.15 2 3 2 1/2 1/2 2 2 1 1 1/2 0.105 0.075 3 5 3 1 1 2 3 2 2 1 0.173 λmax=10.162, ci =0.018, and cr =0.012. 3.2. data process on numeric rating for engineering properties the numeric rating for each property is listed in table 4, which are transformed to table 9 using the tool ‘excel’ according to equations (1) and (2) below (cheng, 1999; strojny & hejman, 2016; torfi, farahani, & rezapour, 2010; yang & hung, 2007; zhang, zhang, wu, shu, & hao, 2005) . the purpose of the transformation is: (i) to make all data/information uniform and further to make the results of the properties comparable (i.e. ci and pi are the ‘smaller-the-better type’ while cbr, cr, sub-criteria of wqi, and se are the ‘larger-the better type’). if the data of the two types are not processed before conducting an ahp process, the relative weights in ahp cannot be obtained reasonably), (ii) to normalize each numeric element into the range of [0, 1] and (iii) to obtain the scoring of each aggregate material under criteria or sub-criteria. (i) the larger the better type: [ min{ }] [max{ } min{ }] ij ij ij ij ij x x r x x    (1) (ii) the smaller the better type: [max{ } ] [max{ } min{ }] ij ij ij ij ij x x r x x    (2) ijahp article: li, wilson, larkin/marginal material evaluation and selection using analytic hierarchy process model international journal of the analytic hierarchy process 78 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13022/ijahp2017.v9.is1.358 where ij x , i =1, 2, …6, j =1, 2, …, 7, is the numeric element in the matrix of table 4 and ij r , i =1, 2, …6, j =1, 2, …, 7, is the normalized numeric element based on equations (1) and (2). table 9 the scoring of each aggregate under each property california bearing ratio (cbr/%) crushing resistance (%) weathering quality index sand equivalent clay index plasticity index percentage retained on 4.75mm sieve cleanness value requirement by nzta m4 specification 0 0 0 0.462 0.100 0 0.889 m4 aggregate 1 1.000 1.000 1.000 1.000 1.000 0.769 0.778 m4 aggregate 2 0.595 0.787 0.826 0.538 0.250 1.000 1.000 marginal aggregate 1a 0.670 0.851 0.739 0.538 0.500 0.615 0.222 marginal aggregate 1b 0.568 0.883 0.696 0.769 0.200 0.462 0.444 marginal aggregate 2 0.459 0.681 0.652 0 0 0.077 0 boundary aggregate 0 0 0 0.462 0.100 0 0.889 (iii) the closer to the specified interval the better type: the passing percentage at each sieve aperture neither belongs to the smaller-thebetter type nor the larger-the better type, but belongs to the closer to the specified intervalthe better type. for example, the passing percentage at 4.75mm is between 28% and 49% with respect to ap 40 (all materials passing 40mm sieve), specified by nzta m4 specification. any number in the interval of 28% 49% is supposed to be the best and equally important. out of the interval, 26% is supposed to be better than 16% as it is closer to the minimum number of the interval (28%). similarly, 46% is supposed to be better than 56% as it is closer to the maximum number of the interval (49%). consequently, table 5 shows the numeric ratings under the sub-criteria of psd (the passing percentage at each sieve) is transformed to table 10 using the tool ‘excel’ according to equation (3) (zhang et al., 2005). 1 1 m , m ij ij x x q q    ij r  1.0, 1 2 [ , ] ij x q q (3) 2 2 , ij ij m x x q m q    ijahp article: li, wilson, larkin/marginal material evaluation and selection using analytic hierarchy process model international journal of the analytic hierarchy process 79 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13022/ijahp2017.v9.is1.358 where ij x , i =1, 2, …6, j =1, 2, …, 10, is the numeric element in table 5; ijr , i =1, 2, …6, j =1, 2, …, 10, is the normalized numeric element based on equation (3); m is the allowable minimum number for ij x and m is the allowable maximum number for ij x ; 1 q is the minimum number of the interval and 2q is the maximum number of the interval. see the previous example, 1 q is 28%, 2q is 49%, m is 0, and m is 100%. table 10 the scoring of each aggregate under each sieve aperture (sub-criteria of psd) na 37.5mm 19mm 9.5mm 4.75mm 2.36mm 1.18mm 0.6mm 0.3mm 0.15mm 0.075m m requirement by nzta m4 specification 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 m4 aggregate 1 0.980 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 m4 aggregate 2 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 marginal aggregate 1a 0.990 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 marginal aggregate 1b 0.970 0.930 0.780 0.630 0.530 0.500 0.570 1.000 1.000 1.000 marginal aggregate 2 1.000 0.660 0.910 1.000 0.789 0.833 1.000 1.000 1.000 1.000 boundary aggregate 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 note: the psd results of the boundary aggregate are assumed to be in the psd envelope specified by nzta m4 specification. 3.3. ranking of the aggregates the data analysis for ranking the aggregate materials is shown in table 11. the overall weight was worked out using ‘excel’. based on mathematical analysis, the aggregates can be ranked as follows: m4 aggregate 1 (0.904) > m4 aggregate 2 (0.833) > marginal aggregate 1a (0.642) > marginal aggregate 1b (0.558) > boundary aggregate (0.346) > marginal aggregate 2 (0.380) where ‘>’ does not mean ‘bigger’, but ‘better’. it reflects a preference for the alternatives (aggregates). the ranking process for the properties of the five aggregates in year 2010, 2011, and 2012 are shown in table 12. a consistent result is obtained, m4 aggregate 1 > m4 aggregate 2 > marginal aggregate 1a > marginal aggregate 1b > boundary aggregate > marginal aggregate 2. ijahp article: li, wilson, larkin/marginal material evaluation and selection using analytic hierarchy process model international journal of the analytic hierarchy process 80 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13022/ijahp2017.v9.is1.358 validation of the ahp model 3.3.1. ranking it can be found that the ahp model can describe the overall performance of aggregates in a quantitative way, which allows the qualities of the aggregates to be compared to each other so that the proper aggregates can be selected for different road construction purposes. as per the ranking for the five aggregates in 2010, 2011, 2012 and 2013, m4 aggregate 1 and 2 are the two best aggregates from among the five aggregates. based on the information provided by the quarries, the two aggregates are used as premium aggregates in field road construction and the other three aggregates are used as sub-standard aggregates in low-volume roads. therefore, the ahp model is a good method to select the best aggregates within a range of aggregates. 3.3.2. selection of marginal aggregates the two m4 aggregates, marginal aggregate 1a and marginal aggregate 1b are better than the boundary aggregate due to the higher overall weight, whilst marginal aggregate 2 with a lower overall weight is worse than the boundary aggregate. as the boundary aggregate is the boundary between premium aggregates and marginal aggregates, the two m4 aggregates, marginal aggregate 1a and marginal aggregate 1b can be defined as ‘premium’ material theoretically based on ahp model analysis whilst the marginal aggregate 2 as ‘marginal’. however, marginal aggregate 1a and 1b are finally utilized in road construction as ‘sub-standard/marginal’ materials rather than premium materials. the reason for the inconsistency is the original individual pass/fail criteria mentioned previously, their pi values obviously exceeding the limit specified by nzta m4 specification and the psd results of marginal aggregate 1b out of the specified limit of the specification. as the real ‘premium aggregates’, m4 aggregate 1 and m4 aggregate 2 are still slightly exceeding the limit of specification in terms of the pi values, i.e. in 2010 and 2013 for m4 aggregate 1, and in 2012 for m4 aggregate 2. but it is not rational to arbitrarily regard them as marginal aggregates because their pi values meet the requirement of nzta m4 specification in the other years. in this scenario, it is probably essential to compare the overall performance of the two m4 aggregates and the boundary aggregate. considering that the overall weights of the two m4 aggregates are much greater than that of the boundary aggregate in table 12, theoretically, the two m4 aggregates can be categorized as premium aggregates, which reconfirm their application as good-quality materials in field road construction. therefore, the validation of the ahp model demonstrates that there is a need to make a combination analysis on the individual properties (specification pass/fail criteria) and overall performance (ahp model) in the process of evaluating the quality of aggregates. ijahp article: li, wilson, larkin/marginal material evaluation and selection using analytic hierarchy process model international journal of the analytic hierarchy process 81 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13022/ijahp2017.v9.is1.358 table 11 data analysis for ranking the materials criteria weights for criteria sub criteria weights m4 aggregate 1 m4 aggregate 2 marginal aggregate 1a marginal aggregate 1b marginal aggregate 2 boundary aggregate wqi 0.076 percentage retained on 4.75mm sieve 0.5 1 0.826 0.739 0.696 0.652 0 cleanness value 0.5 1 0.538 0.538 0.769 0 0.462 ci 0.259 0.769 1 0.615 0.462 0.077 0 se 0.12 1 0.25 0.5 0.2 0 0.1 cbr 0.085 0.778 1 0.222 0.444 0 0.889 cr 0.092 1 0.787 0.851 0.883 0.681 0 pi 0.167 1 0.595 0.67 0.568 0.459 0 psd 0.202 37.5 0.051 0.98 1 0.99 0.97 1 1 19 0.029 1 1 1 0.93 0.66 1 9.5 0.059 1 1 1 0.78 0.91 1 4.75 0.166 1 1 1 0.63 1 1 2.36 0.173 1 1 1 0.53 0.789 1 1.18 0.079 1 1 1 0.5 0.833 1 0.6 0.079 1 1 1 0.57 1 1 0.3 0.087 1 1 1 1 1 1 0.15 0.105 1 1 1 1 1 1 0.075 0.173 1 1 1 1 1 1 overall weights 0.904 0.833 0.642 0.558 0.346 0.38 ijahp article: li, wilson, larkin/marginal material evaluation and selection using analytic hierarchy process model international journal of the analytic hierarchy process 82 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13022/ijahp2017.v9.is1.358 table 12 ranking the materials in 2010, 2011 and 2012 m4 aggregate 1 m4 aggregate 2 marginal aggregate 1a marginal aggregate 1b marginal aggregate 2 boundary aggregate 2010 0.977 n/a 0.709 0.529 0.309 0.470 2011 0.999 n/a 0.625 0.515 0.437 0.360 2012 0.987 0.739 0.609 0.543 0.344 0.397 2013 0.904 0.833 0.642 0.558 0.346 0.380 4. summary and conclusions current definitions about whether a material is classified as a ‘marginal’ material can cause difficulties in fully characterizing and understanding the predicted in-field performance of materials. the understanding of marginal materials suffers from the limitation of not focusing on the overall performance of the materials but on single pass/fail test properties. to overcome this limitation and better understand ‘marginal materials’, this study was conducted using the ahp mathematical model based on multiple factors (various engineering properties and performance). the first step was to determine the relative weights of criteria, including cbr, wqi, cr, se, ci, pi and psd, and the relative weights of sub-criteria, including ‘percentage retained on 4.75mm sieve’ and ‘cleanness value’ of wqi, and sieve apertures of psd. the second step was to process data on the engineering properties of each material using mathematical methods. the third step calculated the overall weight of every material and further ranked the overall performance of the materials. the identification of the ahp analyzed marginal materials was conducted along with the setting of boundary aggregate conditions using the monitored data from 2010 to 2013. the following conclusions can be drawn: clay index (ci) with the highest relative weight is supposed to be the most important property (criterion) influencing the overall performance of aggregates through the weight analysis. the ‘percentage retained on 4.75mm’ and ‘cleanness’ with the same relative weights are of equal importance for weathering quality index (wqi). the sieve 2.36mm and 0.075mm are supposed to be the two of the most important test sieve apertures in affecting the results of particle size distribution (psd) the ahp model provides a good method to select the best aggregates within a range of aggregates. it can describe the overall performance of aggregates in a quantitative way, which allows the qualities of the aggregates to be compared to each other so that the proper aggregates can be selected for different road construction purposes. the setting of a boundary aggregate is very important in the process of analyzing the quality of aggregates using the ahp model. it provides a boundary line of the overall performance between premium and marginal aggregates. the validation of ahp model demonstrates the ahp analyzed qualities of the aggregates match their qualities in field road construction, but there is a need to make a combination analysis on the individual properties (specification pass/fail criteria) ijahp article: li, wilson, larkin/marginal material evaluation and selection using analytic hierarchy process model international journal of the analytic hierarchy process 83 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13022/ijahp2017.v9.is1.358 and overall performance (ahp model) in the process of evaluating the quality of aggregates. the ahp model still has the disadvantage of not completely removing subjectivity from the decision model resulting from deciding pairwise comparison matrices. another disadvantage is that the ahp model may be very time-consuming when obtaining pairwise comparison matrices, which will involve interviews and/or questionnaires. additionally, in many practical cases, the pairwise comparison matrices obtained from interviews and/or questionnaires are uncertain or unable to make precise numerical comparisons, so it is difficult to make a decision with high accuracy. however, it is an improvement over the present practice. the application of the model for ranking materials provides a good quantitative method to transform the values of all material properties into the same unit and r to better compare the overall performance of materials, rather than the limited comparison of single properties. the proposed ahp method enables better decision making when selecting aggregate materials to perform through the lifecycle of the in-field asset and will lead to a better and more economic utilization of local non-renewable mineral resources. ijahp article: li, wilson, larkin/marginal material evaluation and selection using analytic hierarchy process model international journal of the analytic hierarchy process 84 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13022/ijahp2017.v9.is1.358 references astm. (2002). d2419-02 standard test method for sand equivalent value of soils and fine aggregate. doi: 10.1520/d2419-02 bartley, f. g., harvey, c. c., bignall, g., christie, a. b., reyes, a., soong, r., & faure, k. (2007). clay mineralogy of modified marginal aggregates, 318, 109). auckland, new zealand: new zealand transport agency. black, p. m. (2009). geologic inventory of north island aggregate resources: influences on engineering materials properties. auckland, new zealand: the university of auckland. brennan, g. (1984). marginal aggregates for sealed roads: a new zealand experience. paper presented at the australian road research board (arrb) conference, 12th, 1984, hobart, vermont south, victoria, australia. brunschwig, g. (1989). marginal materials: state of the art (pp. 110). paris, france: permanent international association of road congresses (piarc). bsi. (2013). bs en 13242: 2013 aggregates for unbound and hydraulically bound materials for use in civil engineering work and road construction. doi:10.3403/02881113 cheng, c. (1999). evaluating weapon systems using ranking fuzzy numbers. fuzzy sets and systems, 107(1), 25-35. doi: https://doi.org/10.1016/s0165-0114(97)003485 cole, w., & sandy, m. (1980). a proposed secondary mineral rating for basalt road aggregate durability. australian road research, 10(3). evans, g., & vuong, b. (2003). development of performance-based specifications for unbound granular materials: part a: issues and recommendations, 60. sydney, australia. hveem, f. (1953). sand equivalent test for control of materials during construction. paper presented at the highway research board proceedings. nzs. (1991). standards new zealand 4407. new zealand. nzta. (2006). specification for basecourse aggregate tnz m4 , 15. prowell, b. d., zhang, j., & brown, e. r. (2005). aggregate properties and the performance of superpave-designed hot mix asphalt nchrp report 539. doi: doi: https://doi.org/10.17226/13844 http://dx.doi.org/10.3403/02881113 https://doi.org/10.1016/s0165-0114(97)00348-5 https://doi.org/10.1016/s0165-0114(97)00348-5 https://doi.org/10.17226/13844 ijahp article: li, wilson, larkin/marginal material evaluation and selection using analytic hierarchy process model international journal of the analytic hierarchy process 85 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13022/ijahp2017.v9.is1.358 rogers, c. d. f., fleming, p. r., & frost, m. w. (2004). a philosophy for a performance specification for road foundations. proceedings of the institution of civil engineers: transport, 157(3), 143-151. doi: 10.1680/tran.157.3.143.41181 roux iii, i. j., & makrigeorgis, c. (2016). an analytic hierarchy process application to oil sands environmental compliance risk management. international journal of analytic hierarchy process, 8(1), 20. doi: http://dx.doi.org/10.13033/ijahp.v8i1.304 saaty, t. l. (1986). absolute and relative measurement with the ahp. the most livable cities in the united states. socio-economic planning sciences, 20(6), 327-331. doi: https://doi.org/10.1016/0038-0121(86)90043-1 saaty, t. l., & vargas, l. g. (2001). models, methods, concepts & applications of the analytic hierarchy process. springer. sameshima, t. (1977). hydrothermal degradation of basecourse aggregate. satty, t. l. (1980). the analytic hierarchy process. mcgraw-hill, new york. standard, n. z. (1986). 1986 nzs 3111, 58. stapel, e., & verhoef, p. (1989). the use of methylene blue absorption test in assessing the quality of basaltic tuff rock aggregate. engineering geology, 26, 14. doi: https://doi.org/10.1016/0013-7952(89)90011-2 strojny, j., & hejman, w. (2016). ahp based multicriteria comparative analysis of regions of eastern poland. international journal of analytic hierarchy process, 8(1), 24. doi: http://dx.doi.org/10.13033/ijahp.v8i1.373 torfi, f., farahani, r. z., & rezapour, s. (2010). fuzzy ahp to determine the relative weights of evaluation criteria and fuzzy topsis to rank the alternatives. applied soft computing, 10(2), 520-528. doi: https://doi.org/10.1016/j.asoc.2009.08.021 vaidya, o. s., & kumar, s. (2006). analytic hierarchy process: an overview of applications. european journal of operational research, 169(1), 1-29. doi: https://doi.org/10.1016/j.ejor.2004.04.028 van barneveld, j., bartley, f., & dunlop, r. (1984). progress in the study of new zealand aggregates. bulletin of the international association of engineering geology-bulletin de l'association internationale de géologie de l'ingénieur, 30(1), 17-21. doi: 10.1007/bf02594271 yang, t., & hung, c. (2007). multiple-attribute decision making methods for plant layout design problem. robotics and computer-integrated manufacturing, 23(1), 126137. doi: https://doi.org/10.1016/j.rcim.2005.12.002 zhang, j., zhang, d., wu, y., shu, q., & hao, y. (2005). commensuration for the evaluation index value. acta armamentarii, 25(6), 746-751. http://dx.doi.org/10.13033/ijahp.v8i1.304 https://doi.org/10.1016/0038-0121(86)90043-1 https://doi.org/10.1016/0013-7952(89)90011-2 http://dx.doi.org/10.13033/ijahp.v8i1.373 https://doi.org/10.1016/j.asoc.2009.08.021 https://doi.org/10.1016/j.ejor.2004.04.028 https://doi.org/10.1016/j.rcim.2005.12.002 ijahp article: deliktas, torkul, ustun, kiris / an integrated approach for single machine scheduling with sequence-dependent setup times international journal of the analytic hierarchy process 50 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.291 an integrated approach for single machine scheduling with sequence-dependent setup times derya deliktas * department of industrial engineering dumlupınar university kütahya, turkey e-mail: deryadeliktas@hotmail.com orhan torkul department of industrial engineering sakarya university sakarya, turkey e-mail: torkul@sakarya.edu.tr ozden ustun department of industrial engineering dumlupınar university kütahya, turkey e-mail: ozden.ustun@dpu.edu.tr safak kiris department of industrial engineering dumlupınar university kütahya, turkey e-mail: safak.kiris@dpu.edu.tr abstract this study proposes a multi-choice goal programming for the single machine scheduling problem of minimizing the weighted number of tardy jobs, the total weighted completion time and makespan with sequence-dependent setup times. in this problem, there are n candidate jobs for processing in a single machine, each job has a weight, a due date, a processing time, and also sequence-dependent setup times exist between two consecutive jobs. in the first stage of the proposed methodology, the job weights of each job are determined by using the analytic hierarchy process (ahp) method. in the second stage, a 0-1 mixed integer non-linear programming model is built by considering three objective functions and the ideal point is obtained by minimizing the objectives individually. then, the multi-choice goal programming is used to allow the decision makers to set multichoice aspiration levels for each goal. keywords: ahp; single machine scheduling problem; sequence-dependent setup times; multi-choice goal programming; 0-1 mixed integer non-linear programming model ijahp article: deliktas, torkul, ustun, kiris / an integrated approach for single machine scheduling with sequence-dependent setup times international journal of the analytic hierarchy process 51 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.291 1. introduction in the field of scheduling problems sequence-dependent setup times and job weights are very important elements, and non-execution of proper scheduling and sequencing of jobs will cause a significant increase in both makespan and the number of tardy jobs (bahalke et al., 2010). in classical scheduling problems, it is reasonable and necessary to consider scheduling problems with setup times and job weights. setup includes work to prepare the machine, process, or bench for product parts or the cycle. this includes obtaining tools, positioning work-in-process material, return tooling, cleanup, setting the required jigs and fixtures, adjusting tools, and inspecting material. scheduling problems involving setup times can be divided into two classes; the first class is sequence-independent and the second is sequence-dependent setup times. setup is sequence-dependent if its duration depends on both the current and the immediately preceding job, and is sequenceindependent if its duration depends only on the current job to be processed (allahverdi et al., 1999). sequence-dependent setup times are usually found in situations where the facility has a multipurpose machine. some examples of sequence-dependent setups include (i) chemical compounds manufacturing, where the extent of the cleansing depends on both the chemical most recently processed and the chemical about to be processed, and (ii) the printing industry, where the cleaning and setting of the presses for processing the next job depends on its difference from the colour of ink, size of paper and types used in the previous job. the case of sequence-dependent setups can be found in numerous other industrial systems, which include the stamping operation in plastic manufacturing, die changing a metal processing shop, and roll slitting in the paper industry (eren & güner, 2006). in the literature, there are numerous studies that focus on single machine scheduling problems with sequence-dependent setup times. panwalker and iskander (1977) dealt with a scheduling problem which involved both the completion time and the tardiness as criteria and sequence-dependent setup times on single machine case. bianco et al. (1993), fischetti et al. (1993), arcelus and chandra (1983) and miyazaki and ohta (1987) developed exact solution methods for minimizing total completion time problems with sequence-dependent setup. tan and narasimhan (1997) developed a simulated annealing algorithm for the problem of minimizing tardiness, a common measure of due-date performance, in a sequence-dependent setup environment. wang and wang (1997) formed a hybrid algorithm by combining the heuristic with a genetic algorithm for single machine earliness-tardiness scheduling problems with sequence-dependent setup time under a different due date. asano and ohta (1999) developed an optimization algorithm based on the branch-and-bound method to minimize the maximum tardiness. armentano and mazzini (2000) presented a genetic algorithm for a single machine scheduling problem with the objective of minimizing total tardiness. tan et al. (2000) considered the problem of scheduling a single machine for minimizing total tardiness in a sequencedependent setup environment with the comparative performance of branch-and-bound, genetic search, simulated annealing and random-start pairwise interchange. franca et al. (2001) proposed a new memetic algorithm for the total tardiness single machine scheduling problem with due dates and sequence-dependent setup times. gagne et al. (2002) developed ant colony optimization algorithm to minimize total tardiness. mendes et al. (2002) proposed a memetic algorithm and a multiple start approach for the solution of the single machine scheduling problem with sequence-dependent setup times. shin et ijahp article: deliktas, torkul, ustun, kiris / an integrated approach for single machine scheduling with sequence-dependent setup times international journal of the analytic hierarchy process 52 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.291 al. (2002) presented a tabu search algorithm in order to minimize the maximum lateness of the jobs. chang et al. (2004) attempted to solve a single machine scheduling problem in which the objective function was to minimize the total weighted tardiness. lee and asllani (2004) developed an integer programming model and a genetic algorithm for a bicriteria problem (objective of minimizing makespan and number of tardy jobs) with sequencedependent setup times. rabadi et al. (2004) developed a branch-and-bound algorithm to minimize the total amount of earliness and tardiness. eren and güner (2006) proposed an integer programming model for minimization of the weighted sum of total completion time and total tardiness. gupta and smith (2006) proposed two algorithms, a problem space-based local search heuristic and a greedy randomized adaptive search procedure to minimize total tardiness with sequence dependent setup times. kırış and saraç (2009) presented fuzzy goal programming for scheduling the problem of single machine with sequence dependent setup times. bahalke et al. (2010) developed genetic and tabu search algorithms for minimizing makespan with sequence-dependent setup times on a single machine case. sioud et al. (2012) presented a hybrid approach based on the integration between a genetic algorithm and concepts from constraint programming, multi-objective evolutionary algorithms and ant colony optimization for minimizing the total tardiness. there are also studies related to parallel machine scheduling with sequence-dependent setup times. chen (2012) examined the unrelated parallel machine scheduling problem with unequal ready times and sequence-dependent setup times to minimize the weighted number of tardy jobs. chen (2013) also dealt with the unrelated parallel machine scheduling to minimize total weighted completion time with sequencedependent setup times. in this study, we are seeking to minimize the weighted number of tardy jobs, the total weighted completion time and makespan on a single machine under the existence of sequence-dependent setup times. to the best of our knowledge research into this problem using ahp and multi-choice goal programming has not been done. although many researchers focus on the tangible criteria, intangible criteria cannot be neglected in real life problems, and the ahp considers both tangible and intangible criteria. a few studies looking at scheduling problems using the ahp method do exist. wu et al. (2007) used the ahp method to select the best solution for the multi-objective flexible job shop scheduling problem. witkowski et al. (2009) presented an evaluation of a job shop scheduling problem under multiple objectives by using the ahp method for comparing schedules in accordance with multiple objectives. di and ze (2011) proposed a hybrid genetic-tabu search algorithm to solve the flexible job-shop scheduling problem and adopted an ahp application to translate a multi-objective problem into single objective one. fang et al. (2011) combined ahp with grey relational analysis to help make the decision about pre-processing equipment under multi-objective conditions for a multiobjective job shop scheduling problem. lin et al. (2012) used the ahp method to make the parent selection in a genetic algorithm to solve a hybrid flow shop scheduling problem. goal programming is an analytical approach devised to address decision making problems where targets have been assigned to all the attributes and where the decision maker is interested in minimizing the non-achievement of the corresponding goals (romero, 2004). chang (2007) has recently proposed a novel approach namely multi-choice goal programming, which allows decision makers to set multi-choice aspiration levels for each goal (i.e., one goal mapping multiple aspiration levels) to avoid underestimation of decision making. according to multi-choice goal programming, decision makers must not only consider the single aspiration level in the local region, but ijahp article: deliktas, torkul, ustun, kiris / an integrated approach for single machine scheduling with sequence-dependent setup times international journal of the analytic hierarchy process 53 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.291 also develop multiple aspiration levels under given constraints to obtain the global optimal solution in the global region. chang (2008) proposed an alternative method to formulate multi-choice goal programming in which the new approach didn’t involve multiplicative terms of binary variables for solving such problems. an efficient multichoice goal programming formulation based on the conic scalarizing function is proposed by ustun (2012) with three contributions: (1) the alternative formulation allows the decision maker to set multi-choice aspiration levels for each goal to obtain an efficient solution in the global region, (2) the proposed formulation reduces auxiliary constraints and additional variables, and (3) the proposed model guarantees to obtain a properly efficient (in the sense of benson) point. the rapid development of multi-choice goal programming has led to an enormous diversity in models and applications. in practice, the multi-choice goal programming has been applied to the real-world multi-criteria decision making problems, such as supplier selection (liao and kao, 2010; paksoy and chang, 2010), an evaluation of framework for product planning (lee et al., 2010), the plotting a quality management system (ben mahmoud et al., 2010), the aggregate production planning (da silva et al., 2013a, 2013b). according to the standard classification of scheduling problems for three-field notation provided by pinedo (1995), this problem is denoted by 1|𝑠𝑖𝑗 | ∑ 𝑤𝑗 𝑈(𝑛), ∑ 𝑤𝑗 𝐶(𝑛) , 𝐶𝑚𝑎𝑥 . this paper presents a methodology to solve single machine scheduling problems with sequence-dependent setup times in section 2. in the proposed methodology, multiobjective a programming model is combined with the ahp method to determine the job weights. additionally, section 2 deals with an example which integrates the proposed 0-1 mixed integer non-linear programming model formulations and then multi-choice goal programming formulation to find the satisfactory job sequence. conclusions are given in section 3. 2. materials and methodology in this study, a single machine scheduling problem of minimizing the weighted number of tardy jobs, the total weighted completion time and makespan with sequence-dependent setup times was considered. firstly, a 0-1 mixed integer non-linear programming model was provided. there were 6 jobs with 3 customers, and the problem was assumed to be without preemption and breakdown. all jobs were ready at time zero. processing times, due dates and initial setup times are given in table 1. also, the sequence-dependent setup times that exist between the job pairs are given in table 2. table 1 parameters of the jobs jobs 1 2 3 4 5 6 processing time (𝑝𝑗 ) 12 8 3 10 4 18 due date (𝑑𝑗 ) 10 2 72 11 24 60 initial setup time 5 4 6 7 10 3 ijahp article: deliktas, torkul, ustun, kiris / an integrated approach for single machine scheduling with sequence-dependent setup times international journal of the analytic hierarchy process 54 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.291 table 2 setup time of switching from i to job j jobs 1 2 3 4 5 6 1 --7 8 6 14 15 2 5 --18 20 5 8 3 3 11 --19 9 10 4 7 12 1 --6 11 5 8 4 8 3 --16 6 9 2 7 1 2 -- a three-stage solution methodology was proposed to solve the multi-objective scheduling problem. the flow chart of the proposed methodology is given in figure 1. in the first stage, the jobs were evaluated by the ahp to obtain the job weights. these weights were used as parameters for the objectives of the weighted number of tardy jobs and the total weighted completion time in the second stage. also, the ideal point was determined to define the multi-choice goals. the multi-choice goal programming model was constructed and solved to obtain a satisfactory schedule in stage 3. figure 1. flow chart of proposed methodology 2.1 stage 1: job evaluation with ahp method the ahp, developed by saaty (1980), is a technique which considers data or information for a decision in a systematic manner. ahp is mainly concerned with solving decision problems with uncertainties in a multiple criteria characterization. it is based on three principles: (1) constructing the hierarchy, (2) priority setting, and (3) logical consistency (fazlollahtabar et al., 2011). ijahp article: deliktas, torkul, ustun, kiris / an integrated approach for single machine scheduling with sequence-dependent setup times international journal of the analytic hierarchy process 55 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.291 2.1.1 construction of the hierarchy a multi-criteria decision making problem is structured and decomposed into subproblems (sub-objectives, criteria, alternatives, etc.), within the hierarchy. the objective level on the top is the determination of weights of each job, and the second level is the criteria used to determine the weight of each job, including customers and suppliers. the third level is the sub-criteria of each criterion, including reliability, orders frequency / size, work period, net profit, prestige, cost, quality, inventory level and on time delivery, and the bottom level is the six alternatives dealt with in this study. the hierarchy of the proposed problem is given in figure 2. figure 2. hierarchy of proposed problem 2.1.2 priority setting the relative “priority” given to each element in the hierarchy was determined by pairwise comparison of the contributions of elements at a lower level in terms of the criteria (or elements) with a causal relationship (macharis et al., 2004). the decision maker uses a pairwise comparison mechanism, shown in table 3 of saaty (2000) and a 1–9 scale. let 𝐶 = {𝐶𝑗 |𝑗 = 1,2, … , 𝑚} be the set of criteria. the result of the pair-wise comparison on m criteria can be summarized in an m × m evaluation matrix a in which every element 𝑎𝑖𝑗 is the quotient of weights of the criteria, as shown in (1) below: 𝐴 = (𝑎𝑖𝑗 ), 𝑖, 𝑗 = 1, … , 𝑚 (1) the relative priorities are given by the right eigenvector (𝜃) corresponding to the largest eigenvector (𝜆𝑚𝑎𝑥) as: 𝐴𝜃 = 𝜆𝑚𝑎𝑥 𝜃 (2) in case the pair-wise comparisons are completely consistent, the matrix a has rank 1, and 𝜆𝑚𝑎𝑥 = 𝑚. in that case, weights can be obtained by normalizing any of the rows or columns of a. ijahp article: deliktas, torkul, ustun, kiris / an integrated approach for single machine scheduling with sequence-dependent setup times international journal of the analytic hierarchy process 56 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.291 the procedure described above is repeated for all subsystems in the hierarchy. in order to synthesize the various priority vectors, these vectors are weighted with the global priority of the parent criteria and synthesized starting at the top of the hierarchy. as a result, the overall relative priorities to be given to the lowest level elements are obtained. these overall, relative priorities indicate the degree to which the alternatives contribute to the focus. these priorities represent a synthesis of the local priorities, and reflect an evaluation process that permits integration of the perspectives of the various stakeholders involved (fazlollahtabar et al., 2011). 2.1.3 consistency check a measure of consistency of the given pair-wise comparison is needed. the consistency is defined by the relation between the entries of a; that is, we say a is consistent if 𝑎𝑖𝑗 . 𝑎𝑗𝑘 = 𝑎𝑖𝑘, for each i, j, k. if the pair-wise comparisons do not include any inconsistencies, then 𝜆𝑚𝑎𝑥 = 𝑚 . the more consistent the comparisons are, the closer the value of computed 𝜆𝑚𝑎𝑥 is to m. a consistency index (ci), which measures the inconsistencies of pair-wise comparisons, is set to be: 𝐶𝐼 = (𝜆𝑚𝑎𝑥−𝑚) (𝑚−1) (3) the final consistency ratio (cr), on the basis of which one can conclude whether the evaluations are sufficiently consistent, is calculated as the ratio of the ci and the random consistency index (ri), as indicated in equation 4 below: 𝐶𝑅 = 100 ( 𝐶𝐼 𝑅𝐼 ) (4) where m is the number of columns in a and ri is the random index given in table 4, being the average of the ci obtained from a large number of randomly generated matrices. note that ri depends on the order of the matrix, and a cr value of 10% or less is considered acceptable (saaty, 1980). the value 0.1 is the accepted upper limit for cr. if the final consistency ratio exceeds this value, the evaluation procedure needs to be repeated to improve consistency. the measurement of consistency can be used to evaluate the consistency of decision makers as well as the consistency of all the hierarchies. ijahp article: deliktas, torkul, ustun, kiris / an integrated approach for single machine scheduling with sequence-dependent setup times international journal of the analytic hierarchy process 57 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.291 table 3 scale of relative importance intensity scale of importance definition explanation 1 equal importance two activities contribute equally to the objective 2 weak 3 moderate importance experience and judgment slightly favor one activity over another 4 moderate plus 5 strong importance experience and judgment strongly favor one activity over another 6 strong plus 7 very strong or demonstrated importance an activity is favored very strongly over another; its dominance demonstrated in practice 8 very, very strong 9 extreme importance the evidence favoring one activity over another is of the highest possible order of affirmation reciprocals of above if activity i has one of the above nonzero numbers assigned to it when compared with activity j, then j has the reciprocals value when compared with i a reasonable assumption rationals ratios arising from the scale if consistency were to be forced by obtaining n numerical values to span the matrix table 4 the random consistency index r 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝑅𝐼 0.00 0.00 0.58 0.90 1.12 1.24 1.32 1.41 1.45 1.49 1.51 1.48 1.56 1.57 1.59 three experts from the purchasing, production and marketing departments made the pairwise comparison for each criteria and sub-criteria. for example, the pair-wise comparisons matrices for main criteria (c1 and c2), the customers’ sub-criteria (c11, c12, c13, c14, c15) and jobs for customer reliability sub-criterion are given in table 5. the final weights of the jobs are obtained by using ahp as w = (0.2182, 0.2163, 0.2200, 0.1450, 0.1022, 0.0984). ijahp article: deliktas, torkul, ustun, kiris / an integrated approach for single machine scheduling with sequence-dependent setup times international journal of the analytic hierarchy process 58 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.291 table 5 pair-wise comparison matrix and weights from ahp for the evaluation hierarchy criteria and alternative level 2.2 stage 2: model building: 0-1 mixed integer non-linear programming in this stage, a single machine scheduling problem of minimizing the weighted number of tardy jobs, the total weighted completion time and makespan with sequence-dependent setup times was analyzed. the problem was assumed to be without preemption and breakdown, and all jobs were ready at time zero. the problem was defined as a 0-1 mixed integer non-linear programming model with three objectives. the indices, the parameters and the variables used in 0-1 mixed integer non-linear programming model and multichoice goal programming model are given in tables 6, 7 and 8, respectively. parameters of the jobs and setup times are given in tables 1 and 2. table 6 the indices index definition i /j job index used as a unique identifier for each job. 𝐼 / 𝐽 = {1, . . ,6} k job index used to identify the position of a job in a given sequence. 𝐾 = {1, . . ,6} ijahp article: deliktas, torkul, ustun, kiris / an integrated approach for single machine scheduling with sequence-dependent setup times international journal of the analytic hierarchy process 59 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.291 table 7 the parameters parameter definition 𝑝𝑗 processing time for job j 𝑤𝑗 job weight for job j 𝑑𝑗 due time for job j 𝑆0𝑗 setup time of job j in the first sequence position (initial set up time) 𝑆𝑖𝑗 incremental setup time of switching from job i to job j 𝑀 a positive large number n number of jobs (n = 6) table 8 the variables variable definition 𝑥𝑗𝑘 1 if job j is assigned to the kth position in the sequence; 0, otherwise 𝑥𝑖𝑗𝑘 1 if job j is assigned to the kth position in the sequence and preceded by job i; 0, otherwise 𝑈(𝑘) 1 if job in the kth position is tardy; 0, otherwise 𝐶(𝑘) completion time for the job in the kth position in the sequence 𝑆(𝑘) setup time for the job in the kth position in the sequence 𝑃(𝑘) processing time for the job in the kth position in the sequence 𝑑(𝑘) due time for the job in the kth position in the sequence using the above notations, a 0-1 mixed integer non-linear programming model formulation can be given as below: min 𝑍1 = ∑ ∑ 𝑤𝑗 𝑥𝑗𝑘 𝑈(𝑘) 𝑛 𝑘=1 𝑛 𝑗=1 (6) min 𝑍2 = ∑ ∑ 𝑤𝑗 𝑥𝑗𝑘 𝐶(𝑘) 𝑛 𝑘=1 𝑛 𝑗=1 (7) min 𝑍3 = 𝐶(𝑛) (8) subject to ∑ 𝑥𝑗𝑘 = 1, 𝑘 = 1, … ,6 𝑛 𝑗=1 (9) ∑ 𝑥𝑗𝑘 = 1, 𝑗 = 1, … ,6 𝑛 𝑘=1 (10) ∑ ∑ 𝑥𝑖𝑗𝑘 = 1, 𝑖 ≠ 𝑗 𝑘 = 2, … ,6 𝑛 𝑗=1 𝑛 𝑖=1 (11) 𝑥𝑗𝑘 + 𝑥𝑖𝑘−1 − 1 ≤ 𝑥𝑖𝑗𝑘 , 𝑖 = 1, … ,6; 𝑗 = 1, … ,6; 𝑘 = 2, … ,6 𝑎𝑛𝑑 𝑖 ≠ 𝑗 (12) 𝑆(1) = ∑ 𝑆0𝑗 𝑥𝑗1 𝑛 𝑗=1 , (13) 𝑆(𝑘) = ∑ ∑ 𝑆𝑖𝑗 𝑥𝑖𝑗𝑘 , 𝑖 ≠ 𝑗 𝑘 = 2, … ,6 𝑛 𝑖=1 𝑛 𝑗=1 (14) 𝑃(𝑘) = ∑ 𝑥𝑗𝑘 𝑝𝑗 , 𝑛 𝑗=1 𝑘 = 1, … ,6 (15) 𝐶(1) = 𝑆(1) + 𝑃(1) (16) 𝐶(𝑘) = 𝐶(𝑘 − 1) + 𝑆(𝑘) + 𝑃(𝑘), 𝑘 = 2, … ,6 (17) 𝑑(𝑘) = ∑ 𝑥𝑗𝑘 𝑑𝑗 , 𝑘 = 1, … ,6 𝑛 𝑗=1 (18) −𝐶(𝑘) + 𝑑(𝑘) ≤ 𝑀(1 − 𝑈(𝑘)), 𝑘 = 1, … ,6 (19) 𝐶(𝑘) − 𝑑(𝑘) ≤ 𝑀𝑈(𝑘), 𝑘 = 1, … ,6 (20) 𝑈(𝑘), 𝑥𝑗𝑘 , 𝑥𝑖𝑗𝑘 : 0-1 integer, 𝑖 = 1, … ,6; 𝑗 = 1, … ,6 𝑎𝑛𝑑 𝑘 = 1, … ,6 (21) 𝐶(𝑘), 𝑆(𝑘), 𝑃(𝑘), 𝑑(𝑘) ≥ 0, 𝑘 = 1, … ,6 (22) ijahp article: deliktas, torkul, ustun, kiris / an integrated approach for single machine scheduling with sequence-dependent setup times international journal of the analytic hierarchy process 60 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.291 the objective function sets, equations 6-8, are constructed to minimize the weighted number of tardy jobs, and the total weighted completion time and makespan, respectively. equations 9 and 10 guarantee that only one job is assigned to each sequence position and only one sequence location, respectively. equations 11 and 12 guarantee that only one job, job j, is assigned to follow job i. equation 13 determines the setup time of the first sequence position, while equation 14 determines the setup time of the kth (k >1) sequence position. equation 15 determines the processing time, and equation 16 determines the completion time of the first sequence position. equation 17 determines the completion time of the kth (k >1) sequence position, and equation 18 identifies the due date of the job in the kth sequence position. equations 19 and 20 identify the tardy positions of the job sequence, and finally equations 21 and 22 represent the integrality and non-negativity constraints. 2.3 stage 3: job scheduling with multi-choice goal programming multi-choice goal programming can be described in the following cases. the first case: ‘‘the less the better’’ is formulated as: min ∑ [(𝛽 + 𝛼𝑖 )𝑑𝑖 + + (𝛽 − 𝑖 )𝑑𝑖 −]3𝑖=1 , subject to 𝑓𝑖 (𝑥) − 𝑑𝑖 + + 𝑑𝑖 − = 𝑦𝑖 , 𝑖 = 1,2,3, (23) 𝑎𝑖,𝑚𝑖𝑛 ≤ 𝑦𝑖 ≤ 𝑎𝑖,𝑚𝑎𝑥 , 𝑖 = 1,2,3, 𝑑𝑖 +, 𝑑𝑖 − ≥ 0, 𝑖 = 1,2,3, xx (x is a feasible set), where the ith aspiration level yi is the continuous variable restricted between the upper (ai,max) bound and lower (ai,min) bound (ai,min ≤ yi ≤ ai,max); and 𝑑𝑖 + and 𝑑𝑖 − are positive and negative deviations attached to the ith goal |fi(x)-yi| in equation 23; where 𝑑𝑖 + = 𝑚𝑎𝑥 (0, 𝑓𝑖 (𝑥) − 𝑎𝑖 ) and 𝑑𝑖 − = 𝑚𝑎𝑥 (0, 𝑎𝑖 − 𝑓𝑖 (𝑥)) are, respectively, over and under achievements of ith goal; where (+𝛼𝑖) is the positive weight attached to positive deviation 𝑑𝑖 + and ( 𝛼𝑖) is the negative weight attached to negative deviation 𝑑𝑖 − for ith goal. the weight (+𝛼i) is strictly positive and other weight (+𝛼i) is strictly negative because of (,𝛼)  w = {(, 𝛼)r𝑅+ 3 :0 ≤  < min{ 𝛼1, 𝛼2, 𝛼3}}. if over-achievement is considered more desirable than under-achievement then  should be selected close to 𝛼i as soon as possible by considering (,𝛼)  w. in the second case: ‘‘the more the better’’, objective functions which are maximized in the model can be easily transformed to ‘‘the less the better’’ form by multiplying -1 (ustun, 2012). the weights of objectives of minimizing the weighted number of tardy jobs, the total weighted completion time and makespan are used as 0.4, 0.3 and 0.3, respectively. then each objective function is minimized by individually using lingo 11.0 solver, and the ideal solution and the nadir solution are obtained as i = (0.5795, 38.0588, 72) and n = (0.6817, 55.9759, 95), respectively. the decision makers determine the multi-choice goals as follows: 𝑍1 = ∑ ∑ 𝑤𝑗 𝑥𝑗𝑘 𝑈(𝑘) 𝑛 𝑘=1 𝑛 𝑗=1 ≥ 0.5795 and ≤ 0.6000, ijahp article: deliktas, torkul, ustun, kiris / an integrated approach for single machine scheduling with sequence-dependent setup times international journal of the analytic hierarchy process 61 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.291 𝑍2 = ∑ ∑ 𝑤𝑗 𝑥𝑗𝑘 𝐶(𝑘) 𝑛 𝑘=1 𝑛 𝑗=1 ≥ 38.0588 and ≤ 45, 𝑍3 = 𝐶(𝑛) ≥ 72 and ≤ 90, subject to equations 9-22. the objective function values and multi-choice goals are normalized by dividing (ni-ii) values for i = 1, 2, 3. for example, the first objective function z1 is transformed to z1/(n1i1) = ∑ wju(k) n k=1 /0.102. the multi-choice goals are as follows: 𝑓1(𝑥) = ∑ ∑ 𝑤𝑗 𝑥𝑗𝑘 𝑈(𝑘) 𝑛 𝑘=1 𝑛 𝑗=1 /0.102 ≥ 5.670 and ≤ 5.871, 𝑓2(𝑥) = ∑ ∑ 𝑤𝑗 𝑥𝑗𝑘 𝐶(𝑘) 𝑛 𝑘=1 𝑛 𝑗=1 /17.917 ≥ 2.124 and ≤ 2.512, 𝑓3(𝑥) = 𝐶(𝑘) /23 ≥ 3.130 and ≤ 3.913, subject to equations 9-22. therefore, the multi-choice goal programming is obtained by using the objective functions weights and multiple choice goals. min ∑ [(𝛽 + 𝑖 )𝑑𝑖 + + (𝛽 − 𝑖 )𝑑𝑖 −]3𝑖=1 , subject to 𝑓𝑖 (𝑥) − 𝑑𝑖 + + 𝑑𝑖 − = 𝑦𝑖 , 𝑖 = 1,2,3, 𝑎𝑖,𝑚𝑖𝑛 ≤ 𝑦𝑖 ≤ 𝑎𝑖,𝑚𝑎𝑥 , 𝑖 = 1,2,3, 𝑑𝑖 +, 𝑑𝑖 − ≥ 0, 𝑖 = 1,2,3, eqs. (9)-(22). the value of parameter  is taken as 0.29 because this value should be less than the objective function weights and it should be a positive real number. min 0,69𝑑1 + − 0,11𝑑1 − + 0.59𝑑2 + − 0,01𝑑2 − + 0.59𝑑3 + − 0,01𝑑3 −, ∑ ∑ 𝑤𝑗 𝑥𝑗𝑘 𝑈(𝑘) 𝑛 𝑘=1 𝑛 𝑗=1 /0.102 − 𝑑1 + + 𝑑1 − = 𝑦1, 5.670 ≤ 𝑦1 ≤ 5.871, ∑ ∑ 𝑤𝑗 𝑥𝑗𝑘 𝐶(𝑘) 𝑛 𝑘=1 𝑛 𝑗=1 /17.917 − 𝑑2 + + 𝑑2 − = 𝑦2, 2.124 ≤ 𝑦2 ≤ 2.512, 𝐶(𝑘) /23 − 𝑑3 + + 𝑑3 − = 𝑦3, 3.130 ≤ 𝑦3 ≤ 3.913, 𝑑𝑖 +, 𝑑𝑖 − ≥ 0, 𝑖 = 1,2,3, 𝑦1, 𝑦2, 𝑦3: unrestricted variables eqs. (9)-(22). the multi-choice goal programming model is solved by lingo 11.0 solver (schrage, 2008) on an intel (r) core ™ i7-2760qm cpu 2.40 ghz-based computer in a few seconds of computation time for the study. the values of negative and positive deviations are calculated as 𝑑1 + = 𝑑2 − = 𝑑3 − = 0; 𝑑1 − = 0.1896; 𝑑2 + = 0.1346 and 𝑑3 + = 0.0001 , respectively. the efficient objective function values are determined as 0.5795, 47.4196 and 90, respectively. the efficient sequence is obtained as 3-5-2-6-4-1. it means that the efficient point (0.5795, 47.4196, 90) obtained by using multi-choice goal programming falls into the multiple target values for the first and third multi-choice goals because z1 = 0.5795  [0.5796, 0.6] and z3 = 90  [72, 90]. these values are satisfactory for the decision makers. on the other hand, the second multi-choice goal isn’t satisfied by the obtained efficient point, because z2 = 47.4196  [38.0588, 45]. the pessimistic selection of the multiple target values could be the cause of this result. if the decision makers ijahp article: deliktas, torkul, ustun, kiris / an integrated approach for single machine scheduling with sequence-dependent setup times international journal of the analytic hierarchy process 62 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.291 aren’t satisfied with the obtained efficient point, they can change the multiple target intervals and/or the weights of the multi-choice goals. this allows the decision makers a flexible and attractive search over the pareto surface. sensitivity analysis allows the effects of each change on the relative weights of the objective functions to be analyzed. a sensitivity analysis is performed for the different levels of the job weights for each job. as given in table 9, the job weights belonging to the first row are obtained by the ahp method and the values of objective functions can be calculated as 0.5795, 47.4196 and 90, respectively. if the job weights have equal priority as the second row, the values of objective functions can be calculated as 0.5795, 47.4196 and 90, respectively. the different job weights are used to analyze the changes of the objective function values as shown in table 9. it can be seen from table 9 that the results are sensitive to the decision makers’ preferences. table 9 the values of three objective functions according to the different job weights for each job no job weights objective functions w1 w2 w3 w4 w5 w6 z1 z2 z3 1 0.2182 0.2163 0.2200 0.1450 0.1022 0.0984 0.5795 47.4196 90 2 0.1667 0.1667 0.1667 0.1667 0.1667 0.1667 0.5795 47.4196 90 3 0.2107 0.0612 0.0163 0.1860 0.2547 0.2710 0.4579 49.3365 82 4 0.1612 0.1964 0.1073 0.2031 0.2069 0.1252 0.5607 46.5082 86 5 0.1114 0.2056 0.1440 0.3386 0.0998 0.1007 0.6556 44.7115 86 6 0.2331 0.0423 0.0893 0.0474 0.3626 0.2253 0.3228 45.3825 86 7 0.1075 0.1609 0.0224 0.1160 0.2314 0.3619 0.6158 37.0086 72 8 0.1971 0.2611 0.2304 0.1912 0.0694 0.0507 0.6494 54.0412 82 9 0.1251 0.2567 0.1524 0.3187 0.0257 0.1214 0.7005 49.3226 86 10 0.0512 0.2742 0.1697 0.0358 0.1241 0.3451 0.3612 44.6355 82 the efficient points are obtained by using the multi-choice goal programming according to the different levels of the weights of each objective function. trade-offs among the objective function values based on the various weight combinations are given in figure 3. if the weight of the weighted number of tardy jobs is greater than 0.30 and the other weights of the objectives are less than 0.60, then the first objective can achieve the ideal level as seen in figure 3. ijahp article: deliktas, torkul, ustun, kiris / an integrated approach for single machine scheduling with sequence-dependent setup times international journal of the analytic hierarchy process 63 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.291 figure 3. efficient points related to the sensitivity analysis 3. conclusion in this paper, a methodology that consists of ahp, a 0-1 mixed integer non-linear programming model and multi-choice goal programming was proposed to schedule the jobs for a single machine with sequence-dependent setup times. the ahp allows a flexible multi-criteria decision making process by considering tangible and intangible criteria in a production scheduling environment. the scheduling constraints and the multiple objective functions for a single machine scheduling problem with sequencedependent setup times were considered by using multi-objective programming. the multi-choice goal programming allows the decision maker to set multi-choice aspiration levels for each goal to obtain an efficient solution in the global region. additionally, the multi-choice goal programming also guarantees an efficient solution for obtaining the assignment of candidates and reduces auxiliary constraints and additional variables. the proposed approach was implemented with a small-size problem consisting of six jobs. a satisfactory solution was obtained by using the proposed approach. the sensitivity analysis was performed according to the job weights and the objective weights. the proposed methodology supports decision makers in the effective management of the job scheduling process. the proposed methodology can be applied to other fields such as personnel selection, logistics, machine selection, project management, portfolio management, etc. job scheduling problems will become more complex in the near future because of the dynamic environment, and meta-heuristic methods can be required for large size problems. ijahp article: deliktas, torkul, ustun, kiris / an integrated approach for single machine scheduling with sequence-dependent setup times international journal of the analytic hierarchy process 64 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.291 references allahverdi a, gupta j.n.d., & aldowaisan t. (1999). a review of scheduling research involving setup considerations. omega, 27, 219–39. arcelus, f.j., & chandra, r. (1983). on n-1-f setup dependent problems. engineering optimization, 7, 59–67. armentano, v.a., & mazzini, r. (2000). a genetic algorithm for scheduling on a single machine with set-up times and due dates. production planning and control, 11, 713–720. asano, m., & ohta, h. (1999). scheduling with shutdowns and sequence dependent setup times. international journal of production research, 37, 1661–1676. bahalke, u., yolmeh, a.m., & shahanaghi, k. (2010). meta-heuristics to solve singlemachine scheduling problem with sequence-dependent setup time and deteriorating jobs, international journal of advanced manufacturing technology, 50, 749-759. ben mahmoud, h., ketata, r., ben romdhane, t., & ben ahmed, s. (2010). piloting a quality management system for study case using multi-choice goal programming, in: conference proceedings – ieee international conference on systems, man and cybernetics, 2010, article no. 5641929, 2500–2505. bianco, l., mingozzi, a., & ricciardelli, s. (1993). the travelling salesman problem with cumulative costs. networks, 23, 81–91. chang, t.y., chou, f.d. & lee, c.e. (2004). a heuristic algorithm to minimize total weighted tardiness on a single machine with release dates and sequence-dependent setup times. journal of the chinese institute of industrial engineers, 21, 289–300. chang, c.t. (2007). multi-choice goal programming. omega 35, 389–396. chang, c.t. (2008). revised multi-choice goal programming. applied mathematical modeing, 32, 2587–2595. chen, c.l. (2012). iterated hybrid metaheuristic algorithms for unrelated parallel machines problem with unequal ready times and sequence-dependent setup times. international journal of advanced manufacturing technology, 60, 693-705. chen, j.f. (2013). unrelated parallel-machine scheduling to minimize total weighted completion time. journal of intelligent manufacturing, 1-14 (article in press). da silva a.f., marins f.a.s., & montevechi j.a.b. (2013a). multi-choice mixed integer goal programming optimization for real problems in a sugar and ethanol milling company. applied mathematical modelling, 37(9), 6146–6162. ijahp article: deliktas, torkul, ustun, kiris / an integrated approach for single machine scheduling with sequence-dependent setup times international journal of the analytic hierarchy process 65 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.291 da silva a.f., marins f.a.s., & montevechi j.a.b. (2013b). application of mixed binary goal programming in an enterprise in the sugar and energy sector. gestao e producao, 20(2), 321-335. di, l., & ze, t. (2011). a genetic algorithm with tabu search for multi-objective scheduling constrained flexible job shop, proceedings of 2011 cross strait quadregional radio science and wireless technology conference, 1662-1665. eren, t., & güner, e. (2006). a bicriteria scheduling with sequence-dependent setup times. applied mathematics and computation, 179, 378–385. fang, y., wang, f., & wang, h. (2011). research of multi-objective optimization study for job shop scheduling problem based on grey ant colony algorithm. advanced materials research, 308-310, 1033-1036. fazlollahtabar, h., mahdavi, i., ashoori, m.t., kaviani, s., & mahdavi-amiri, n. (2011). a multi-objective decision-making process of supplier selection and order allocation for multi-period scheduling in an electronic market. international journal of advanced manufacturing technology, 52, 1039–1052 fischetti, m., laporte, g., & martello, s. (1993). the delivery man problem and cumulative matroids. operations research, 41, 1055–1064. franca, p.m., mendes, a., & moscato, p. (2001). a memetic algorithm for the total tardiness single machine scheduling problem. european journal of operational research, 132, 224–242. gagne, c., price, w.l., & gravel, m. (2002). comparing an aco algorithm with other heuristics for the single machine scheduling problem with sequence-dependent setup times. journal of the operational research society, 53, 895–906. gupta, s.r., & smith, j.s. (2006). algorithms for single machine total tardiness scheduling with sequence dependent setups. european journal of operational research, 175, 722–739. kırış, ş., & saraç, t. (2009). modeling the fuzzy goal programming for scheduling problem of single machine with sequence dependent set-up times. 1st international fuzzy systems symposium, ankara, turkey, 34-40. lee, s.m., & asllani, a.a. (2004). job scheduling with dual criteria and sequencedependent setups: mathematical versus genetic programming. omega, 32 (2), 145–153. lee, a.h.i., kang, h.-y., yang, c.-y., & lin, c.-y. (2010). an evaluation framework for product planning using fanp, qfd and multi-choice goal programming. international journal of production research, 48(13), 3977–3997. liao, c.n., & kao, h.p. (2010). supplier selection model using taguchi loss function, analytical hierarchy process and multi-choice goal programming. computers and industrial engineering, 58, 571–577. ijahp article: deliktas, torkul, ustun, kiris / an integrated approach for single machine scheduling with sequence-dependent setup times international journal of the analytic hierarchy process 66 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.291 lin, d., lee, c.k.m., & wu, z. (2012). integrating analytical hierarchy process to genetic algorithm for re-entrant flow shop scheduling problem. international journal of production research, 50(7), 1813-1824. macharis, c., springael, j., de brucher, k., & verbeke, a. (2004). promethee and ahp: the design of operational synergies in multicriteria analysis. strengthening promethee with ideas of ahp, european journal of operational research, 153(2), 307–317. mendes, a.s., franca, p.m., & moscato p. (2002). fitness landscapes for the total tardiness single machine scheduling problem. neural network world, 12,165–180. miyazaki, s., & ohta h. (1987). backward scheduling to minimize the actual mean flow time with dependent and independent setup times. proceedings of the. ixth icpr, 2680– 2686. paksoy, t., & chang, c.t. (2010). revised multi-choice goal programming for multiperiod, multi-stage inventory controlled supply chain model with popup stores in guerilla marketing. applied mathematical modelling, 34, 3586–3598. pinedo m. (1995). scheduling: theory, algorithms, and systems. englewood, nj: prentice-hall, inc. panwalker s.s., & iskander, w. (1977). a survey of scheduling rule. operations research, 25, 45–61. rabadi, g., mollaghasemi, m., & anagnostopoulos g.c. (2004). a branch andbound algorithm for the early/tardy machine scheduling problem with a common due-date and sequence dependent setup time. computers and operations research, 31, 1727–1751. romero, c. (2004). a general structure of achievement function for a goal programming model. european journal of operational research, 153, 675–686. saaty, t.l. (1980). the analytic hierarchy process, new york: mcgraw-hill. saaty, t.l. (2000). fundamentals of decision making and priority theory with the analytic hierarchy process. pittsburgh, usa: rws publications. schrage, l. (2008). optimization modeling with lingo, chicago: lindo systems inc. shin, h.j., kim, c.o., & kim, s.s. (2002). a tabu search algorithm for single machine scheduling with release times, due dates, and sequence-dependent set-up times. international journal of advanced manufacturing technology, 19, 859–866. sioud, a., gravel, m. & gagné, c. (2012). a hybrid genetic algorithm for the single machine scheduling problem with sequence-dependent setup times. computers and operations research, 39, 2415-2424. ijahp article: deliktas, torkul, ustun, kiris / an integrated approach for single machine scheduling with sequence-dependent setup times international journal of the analytic hierarchy process 67 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.291 tan, k.c., & narasimhan,r. (1997). minimizing tardiness on a single processor with sequence-dependent setup times: a simulated annealing approach. omega, 25, 619–634. tan, k.c., narasimhan, r., rubin, p.a., & ragatz, g.l. (2000). a comparison of four methods for minimizing total tardiness on a single processor with sequence dependent setup times. omega, 28, 313–326. ustun, o. (2012). multi-choice goal programming formulation based on the conic scalarizing function. applied mathematical modelling, 36, 974–988. wang, l., & wang, m. (1997). hybrid algorithm for earliness–tardiness scheduling problem with sequence dependent setup time, proceedings of the ieee conference on decision and control 2, 1219–1222. witkowski, t., antczak, p., & antczak, a. (2009). multi-objective decision making and search space for the evaluation of production process scheduling, bulletin of the polish academy of sciences: technical sciences, 57(3), 195-208. wu, x., sun, s., yu, j., & cai, z. (2007). a multi -objective scheduling decision making model for the flexible job shop. zhongguo jixie gongcheng/china mechanical engineering, 18(2), 161-165. ijahp article: ali/ an mcdm approach towards m-payment business models evaluation international journal of the analytic hierarchy process 273 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.298 an mcdm approach towards m-payment business models evaluation abid ali school of computer science & software engineering international islamic university islamabad, pakistan abid.msse176@iiu.edu.pk abstract there are many challenges in the adaptation of m-payment technology such as improved service quality, missing standards, lack of content quality, low customer satisfaction, and lack of a business model. the business model plays a critical role in the success of mpayment technology, and there are different m-payment business models, each with their own advantages and disadvantages. project managers have little understanding about the different components of these specific business models. this study surveyed different business model’s evaluation criteria from the literature and industry, and used the analytic hierarchy process to evaluate m-payment business models on the basis of these criteria. the scalability and user centric architecture in the case of service related factors and collaboration & partnership, and response to market trends were the most important factors for sustainability of business models. according to the given criteria, the collaboration model was the most dominant model in the m-commerce domain. a sensitivity analysis was performed in order to find out different views about the final prioritized list under varying conditions. keywords: m-commerce, mpayment business models, analytic hierarchy process. 1. introduction m-commerce (mobile commerce) is a process where mobile devices like mobile phones, pdas, smart phones and other emerging devices can be used to initiate any transaction. the fast adaptation and tremendous growth of mobile and wireless technologies assures the realization of different types of innovative applications. the most important categories among these applications are financial services, location based information, wireless business re-engineering and mobile games etc. new forms of mobile technologies are rapidly transforming the marketplace. today’s business market is extremely dynamic and most organizations are searching for new and innovative ways to optimize their business processes and other parameters for added value. in this regard, mpayment and m-shopping are useful tools for many organizations to achieve their objectives in the current digital world. the use of m-payment has been proposed for online payment services as a way to deal with security and trust problems in electronic transactions (thair, suhuai et al., 2010). there are different stakeholders involved in implementing m-payment systems, and each has a different role and different interests. introduction of mpayment systems is a complex economic game with multiple stakeholders, and has in-depth concerns in different factors. research shows that the immaturity of the market and the consequent unresolved technical, strategic and demand mailto:abid.msse176@iiu.edu.pk ijahp article: ali/ an mcdm approach towards m-payment business models evaluation international journal of the analytic hierarchy process 274 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.298 issues make the adoption of mobile payments highly uncertain (agnieszka,, elaine, & robert 2005). due to many actor’s involvement i.e. operators, banks and independent service providers which play different roles in implementing m-payment services, it is necessary to select a suitable business model to optimize different parameters. the essence of the problem of selecting the best business model is a multi-criterion problem. therefore, the process of creating or selecting a business model has inherited complexity due to the need to balance multiple or even conflicting stakeholder requirements. observing the nature of the problem, we investigated two questions. the first was, “which m-payment business models are adequate for m-payment implementation on the basis of multi and conflicting criteria?”, and the second was, “what is the relative importance of each criterion?” we hypothesize that there is a need for a comprehensive evaluation of existing m-payment business models. this work has two important parts i.e selection of alternatives (business models) and criteria for evaluation of these models. this paper begins by surveying five m-payment business models and ten different evaluation criteria which typify these models. it then delineates the presumptions and procedures to conduct the analytic hierarchy process (ahp). to test our hypothesis, ahp is used to obtain the relative weights among the factors, sub factors and the total values of each m-payment business model based on these weights. this paper is organized as follows: section 2 provides a literature review for prospective analysis while section 3 provides the overview of m-payment business models. section 4 advocates the criteria for evaluation purposes, and section 5 presents a hierarchy of the research method. section 6 touches on the proposed approach while section 7 discusses the sensitivity analysis which is an important part of the proposed approach. lastly, section 8 summarizes the conclusion and future directions. 2. literature review many attempts were made to explore the m-payment process from different perspectives. the literature emphasizes the following two parameters: 2.1business model evaluation different approaches were followed to analyze the m-payment process. the core component of this analysis was business model evaluation. pousttchi, schiessler & wiedemann (2007) proposed a framework for mobile payment business models. this framework facilitates the categorization of m-payment business models. the framework consists of six partial models: market model, value proposition model, implementation model, capital model, distribution and communication model, and threat model. qiang, yan, and tingjie (2008) have named mobile payment as ubiquitous payment and categorized mobile payment business models into four modes: i.e. carrier’s operator independently, mobile network operator centric, financial institutions centric and third party operating. they discussed the disadvantages of these models and recommended some strategies to solve them. research performed by the smart card alliance contactless payment council (smart card alliance, 2008) considered four different business models for mobile payments deployment and discussed their advantages and disadvantages. schierz, schilke, et al., (2010) proposed a conceptual model which focused on factors determining consumer’s acceptance of mobile payment services. the parameters of compatibility, subjective norm and individual mobility are strongly ijahp article: ali/ an mcdm approach towards m-payment business models evaluation international journal of the analytic hierarchy process 275 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.298 supported by empirical results. this study provides useful directions for managers regarding market mobile payment solutions to rectify consumer intention. pousttchi and hufenbach (2012) exercised the extension of mobile payment business model framework with three new variables: mobile marketing service provider, trusted service manager and mobile customer relationship management service provider. cabanillas, leiva, et al. (2013) modified the classical technological acceptance model by including risk as a variable given its relevance in the field. the empirical results showed a particular support for the effects of external influences, of usefulness and, to a lesser extent, of risk. this research showed several directions for companies to focus on consumer intention for using m-payment services. slade, williams et al. (2014) explore the potential of a new model of consumer technology adoption, and its extension from risk and trust perspective in explaining non-users adoption of proximity mobile payment. data analysis shows that the extended model explains more variance in behavioral intention, but performance expectancy remains the strongest predictor across both models. the strong theoretical and practical implications can be derived from findings for strategic development and marketing proximity of m-payment in the uk. 2.2 mcdm applications in m-commerce the wide range applications of mcdm mechanisms have been reported from the existing literature. chou, lee, et al. (2004) evaluated the performance of different payment systems using the analytic hierarchy process which provides the foundation for this study. ondrus (2008) utilized the mcdm method to evaluate the potential of nfc (near field communication) in comparison to other technologies for payment purposes. asghari, amidian, et al. (2010) performed an empirical evaluation of m-payment business models using electre, which is one of the mcdm methods. sharma and gutierrez (2010) provide a framework which characterizes m-commerce business models. 3. m-payment business models a thorough literature review was performed and comprehensive discussions were carried out with industry experts in order to investigate different business models and their advantages and disadvantages. this section discusses some conclusive remarks from the literature review and industry survey. there are five m-payment business models and each model has some advantages and disadvantages. despite several efforts, there is no dominant m-payment business model in the market today. the fundamental components that make business models viable systems are the ability to improve, without ambiguity, the transaction technology in different economic environments. 3.1 operator centric business model in this model, the operator manages the whole business scenario and the decision making. other organizations such as financial institutions are not concerned with the payment process. there are two payment methods used in this model which are prepaid cards and telecommunication phone bills. the main disadvantage of this model is that it cannot support macro payment (smart card alliance, 2008). example: ntt docomo is well known m-payment company which follows the operator centric model for their m-payment services ijahp article: ali/ an mcdm approach towards m-payment business models evaluation international journal of the analytic hierarchy process 276 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.298 3.2 bank centric model in this case, the whole production and management of the m-payment process can be controlled by banks, while operators do not have any concerns (smart card alliance, 2008). operators charge the banks due to the use of sim-based application technology for their m-payment purposes, and operators receive a rental fee from the banks because operators have ownership of the sim toolkit. payments are made through bank accounts, so both micro and macro payments are supported in this model (smart card alliance, 2008). an example of this model is pay box. 3.3 operator centric with bank interface model in this case, operators manage and control the whole business process, but at the same time banks also take part in the payment process. normally, this model solves the issues of the two previous models. it supports both micro and macro payments. for micro payments m-wallet, telecommunication bills or prepaid cards etc. are used while for macro payments bank accounts are used. in comparison with the two previous models, this model provides a unique user interface for communicating with several accounts in different banks (asghari, amidian et al., 2010). 3.4. peer-to-peer model this model adopts a different approach from the previous models. here, a third party provides the m-payment service while using the infrastructure provided by operators and banks. actually, bank accounts and mobile devices are necessary components for the use of this model. this model supports both micro and macro payments (smart card alliance, 2008). an example of this model is paypal. 3.5 collaboration model in this model there are different actors which perform different roles in accomplishing the task of m-payment. the collaboration model is based on collaboration among operators, banks and service managers who manage and control the whole business process and any decision making. it organizes the collaboration among the responsible parties. in this model, banks and operators focus on their main functions; in addition, they have transaction fee income (smart card alliance, 2008). an example of this model is semops. 4. evaluation criteria the most logical part of this research is to find evaluation criteria, which give an abstract and context free evaluation of existing m-payment business models. for this purpose, a through literature review was conducted and collaboration with industry experts was done in order to identify the success factors of m-payment business models which act as evaluation criteria. these success factors were considered to provide a comprehensive analysis of the existing m-payment business models independent of any specific perspective and context. criteria are used to identify the opinions of decision makers for reference in their selection process. sharma and gutierrez (2010) provide a framework, which surveyed the success factors for m-commerce business models evaluation. these factors support sustainability of m-payment business models and categorized them into service and organization related factors. ijahp article: ali/ an mcdm approach towards m-payment business models evaluation international journal of the analytic hierarchy process 277 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.298 4.1 service related factors 4.1.1 interface the interaction layer between a specific business model and its customer can be represented using interface. the ease of use, expediency and accessibility are relative characteristics of any business model which produce a better customer experience and its success. the customer interface covers all customer related aspects, especially the selection of the target customers, the channels through which it contacts them and the kind of relationships the company wants to establish with its customers (pousttchi, schiessler et al., 2009). the business model is more feasible if its interface is more usable. literature strongly claims that in cases where there is good interface design, value propositions are increased several times. a good, easy to use, accessible interface is a deciding factor for the success of a business model (sharma and gutierrez, 2010). 4.1.2 service offering a major determining factor for the success of m-commerce is service affordability—such as low access, subscription, and usage fees (grami and schell, 2004). service offering means a combination of services which establishes end to end connectivity between various functional blocks of a business model’s value chain (sharma and gutierrez, 2010). there are a number of services for the proper functioning of any business model. the set of services which allow any business model to create market and capture value is represented by service offering characteristics. services become a crucial element for the business of many companies (zolnowski,wei et al., 2014). guideline: analyze all the key processes required by the business model to function effectively and determine whether there is an incorporating service component for each of the functions required. 4.1.3 value proposition the sketch of products and services which are offered by a specific business to its valuable customers can be explained by value proposition, and also justify the investment of customers in products or services which are offered by the company. a value proposition is an overall view of an m-payment service provider’s bundle of offers that are of value to the customer (pousttchi, schiessler et al., 2009). the broad business logic and product offerings are represented by it, which provides value to the customers as compared to other competitors. it also tries to explain the questions like:  what are the products and services offered to the customers?  and how do these offerings provide worth to the customers? (sharma and gutierrez, 2010). this factor depends on different services such as macro payment, micro payment, transfer and account statement etc. for example, in the operator centric model, the extensibility has been limited to services which the operator can exhibit. in the collaboration model it has been extended to banks, operators and service manager’s capabilities (asghari, amidian et al., 2010). the value is the core concept in the service domain. it represents inclusive logic for creating value which the business delivers. in order to deliver the value proposition to diverse customers, the firm should have required capabilities to employ available resources and put forward services to market. guideline: a value proposition can be explained by answering the following questions. how much value should the customer expect from the relevant business? ijahp article: ali/ an mcdm approach towards m-payment business models evaluation international journal of the analytic hierarchy process 278 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.298 what is the customer’s intention to select a specific company? how much does the end product or service cost? what is the suitability of the proposed cost? finding answers for these questions and forming the suitable solution with respect to each, facilitates the shaping of the value proposition in a specific business model. the amount of value generation depends upon viability which further depends on better value proposition. more exactly, based on the segments resulting from the application of the moderating effect of gender, companies will be able to define strategies adapted to the influence patterns, producing differentiated value propositions (market segmentation), so that these propositions properly satisfy customers, improving their loyalty, and thus contributing to the achievement of the goals of the companies themselves (cabanillas, fernández et al., 2014) 4.1.4 dynamicity business models change and evolve with time due to changes in external variables; this is called dynamicity. the global business environment is extremely dynamic and ongoing changes in this environment compel companies to regularly review and adapt their business models to carry on their market presence. the literature justifies the idea that static business models are not viable and thus reluctance to change to market needs make them even less viable. guideline: the capability to alter a model in reaction to a dynamic exterior environment (customer expectations, changing business environment, technology innovations, and market needs) and willingness of the organization to alter their business model. 4.1.5 scalability scalability refers to the capability of a business model to extend its services or resources to increase throughput. the scalability of mobile service payment depends on mobile network operators, banks and third party scalability which participated in the payment service (asghari, amidian et al., 2010). it indicates its potential to either handle growing amounts of revenue with ease, or to be readily enlarged. literature shows that in the future friendly or supple value networks will be more desirable and substitute linear and traditional value chains. attention given to mobile payment likewise was mainly focused on specific themes (tan, ooi, k. et al., 2014). guideline: friendly, modular and supple nature of the business model to facilitate the accumulation of resources and services. 4.1.6 user centric architecture the emerging mobile applications are often enthused by new technologies or by new devices. a number of technology architectures/solutions have been proposed to improve cost, functionalities, scalability and security (kim, mirusmonov et al., 2010).we must produce more user-centric business models in order to exploit the synergies of mobile technologies which give prospective customers top priority. there are strong indications from the literature review that a user centric approach towards designing services leads to enhanced user participation and engagement. user centricity is implied by understanding the behavior of expected end-users and using feedback in designing the service. ijahp article: ali/ an mcdm approach towards m-payment business models evaluation international journal of the analytic hierarchy process 279 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.298 guideline: the objectives of businesses are meeting customer necessities, reforming the services and products according to the mindset of the customer, meeting customer hopes and improving the quality of the customer’s experience. 4.2 organization related factors 4.2.1 organizing model an organizing model provides a sketch of how service providers, or the internal departments which offer a single service, organize value chains, business processes, business or organizational strategies, collaborations and partnerships with other value partners to deliver services and products to the end customer. the arrangement of participating entities which aim to effectively deliver different services across the value chain can be described by the organizing model. in any business model different roles and responsibilities are assigned to each participant. in a feasible business model, the organizing model is complete which means that appropriate actors exist who perform tasks specific to their core responsibilities. guidelines: in depth analysis of the roles and responsibilities with respect to each actor in a value chain, and recognition of the proper match between a suitable actor for an exact role or responsibility. 4.2.2 roi (return on investment) roi means good return on investment (roi) to each of the participating partners. the detailed view about the desirable investment in the organization and the related cost models explain the costing structure for variable services and products. both of these components assist in explaining one of the extremely important drivers of a business case. it describes the investments, risks and the revenue streams split across different participating actors in the value chain. there should be proper mechanisms to describe the entire business model and value chain which is generated by it, sufficient roi for all the involved partners that keeps them busy in the value chain. the literature highlights that roi is an important factor for any business and its partners. it is very important to analyze that each participant in the business model is receiving enough roi to stay involved. guideline: it is necessary to investigate whether each participant is receiving sufficient roi to stay engaged in the value chain by developing a complete revenue-cost map for the business model. 4.2.3 collaboration & partnership the lack of cooperation between the key players is a significant barrier to the success of m-payments. all key players have their strengths and weaknesses; the most successful business models could be those based on strong partnership (pousttchi, schiessler et al., 2009). the ability to enable m-commerce success substantially depends on partnerships and external collaborations, which also extends the considerations of the associated roles and responsibilities of various actors and the rate of their participation. it is clear that currently the mobile marketplace is a carrier-dominated one rather than an open market and it requires more severe partnerships with tightly coupled arrangements rather than loose cooperation. a partnership is a voluntarily initiated cooperative agreement between the m-payment service provider and other companies in order to create value (pousttchi, schiessler et al., 2009). in the highly competitive mobile business domain, different ijahp article: ali/ an mcdm approach towards m-payment business models evaluation international journal of the analytic hierarchy process 280 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.298 actors need to collaborate with each other to complement their shortcomings and create valuable propositions for their customers. in fact, for nearly each player, partnerships with a number of other actors are an important part of their business models, as they are required to overcome the complexity of providing a complete end-to-end solution, which requires many complementary competencies (camponovo, 2002). guideline: existence of value based collaborations and partnerships which ultimately bring additional revenue to the entire value chain. 4.2.4 response to market trend for global m-payment services to succeed, a wide range of criteria will have to be met i.e. requirements are not only technology-based or business-based, also economics-based, and requirements that have their origin in the social/cognitive sciences. market needs can be considered the critical success factors for a mobile commerce proposal which have taken first priority from technological factors. an organization’s proper responsiveness towards change in market trends gives it superiority in a highly spirited environment. organizations face new challenges due to dynamic market trends and redirect them towards initiatives uptake. no business model can survive in isolation, and it must consider external market forces. market trends are dynamic in nature and are controlled by many factors such as technology innovations, increases in customer expectations and so forth. these factors further force organizations to change their business models and evolve accordingly. guidelines: analyze the tendencies of a business model and regulate it according to changing market trends. examine how likely or how capable a business model is to change in response to market trends. more responsiveness implies more sustainability. 5. research methodology this research is based on the assumption of interprevitism which posits that reality is socially constructed, multiple interpretations and realities exist and scientific research is time and context dependent (sharma and gutierrez, 2010). alternate value chain models with benefits and drawbacks for each player could be analyzed with economic modeling and design research, and can be backed by interviews and expert panels (dahlberg, mallat et al., 2008). an online survey and interview method were used in order to identify the relative importance of each factor in comparison to other factors. a survey provides a comprehensive system for collecting information to describe, compare or explain knowledge, attitudes and behaviors over large populations. 5.1 sample selection the ahp is a decision making method where users can have less understanding about mpayment business models which may lead to inconsistency of data. in order to avoid inconsistencies during data analysis, we used an expert pool. different and well known m-payment companies, service providers and financial institutions were located using the internet. then proper profiling was done in order to find the most relevant people from industry. cabanillas, fernandez et al. (2014) stated that an individual’s positive experience with a given item in the past will have a decisive impact on current behavior toward that item. individual emailing was done to obtain participant’s consent. ijahp article: ali/ an mcdm approach towards m-payment business models evaluation international journal of the analytic hierarchy process 281 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.298 5.2 questionnaire design there were a total of 21+100 comparisons in the study. in order to achieve the maximum response rate, the transitive property was used to resize the questionnaire. we compared each criterion with their consecutive criterion only one time and the rest of the comparisons were determined using the transitive property. the survey was done in two phases. in the first phase, a relative comparison was done of each criterion, and in the second phase the supporting intensity level of each criterion with respect to each mpayment model was determined. figure 1 shows the format of the question used for ahp. table 1 shows saaty’s scale which measures relative importance of one factor over another. 9 7 5 3 equal 3 5 7 9 figure 1. ahp questionnaire format table 1 saaty’s scale intensity level definition explanation 1 equal preference two factors equally preferred. the objective 3 somewhat more preference one is slightly favored over other on the basis of judgment and experience 5 much more preference one is strongly favored over other 7 very much more preference very strongly preference 9 absolutely more preference extreme preference 2,4,6,8 intermediate values when compromise is done. source: coyle, 2004 5.3 research questions in this study the following research questions were investigated: q.1.what is the relative importance of reported m-payment service and organizational related factors in the selection of an m-payment business model? q.2.which business model is more appropriate on the basis of the factors reported in q1? q.3.which factors are more sensitive in term of relative importance with respect to each m-payment business models? extreme preference extreme preference ijahp article: ali/ an mcdm approach towards m-payment business models evaluation international journal of the analytic hierarchy process 282 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.298 6. m-payment business models evaluation: an mcdm approach there are several stakeholders in the system; a viable and sound business model needs to be developed that will provide a framework for revenue sharing. mcdm is a multicriteria decision making method including several techniques which allow rating a range of criteria, and then ranking them with the opinions of industry experts. the mcdm methods have a high potential to reduce the cost and time and increase the accuracy of decisions and can be an appropriate framework for solving problems. with this characteristic, decision makers have the possibility to easily examine the problem and scale it in accordance with their requirements (asghari, amidian et al., 2010). this section will introduce one of the mcdm methods named ahp, and then use this method and apply it to the expert’s opinions in order to make a comparison between the mentioned business models. 6.1 ahp technique the analytic hierarchy process (ahp) is the most commonly used mcdm (multi criteria decision making) method that was developed by thomas l. saaty. the most innovative aspect of this technique is the modelling of the hierarchy. the nature of the problem may be a highly complex, multi-criteria situation causing conflict. a pair wise comparison of the different elements can be constructed where the values in each cell show the dominancy of each element over another with respect to some given criterion. the largest eigenvalue problem can result from this scaling formulation for each hierarchy. the ahp uses dominance matrices and goes beyond probabilistic measurement. it develops the tradeoff in the course of structuring and analyzing a series of simple reciprocal pair wise comparison matrices. the ahp is based on three major components: 1. decomposition of complex problem into a hierarchy where each level contains some manageable elements and each element is decomposed from another set of elements. the process of decomposition is continued to more specific elements. 2. a measuring methodology is used for prioritizing the elements within each stream of hierarchy. the pair wise approach is used to evaluate each set of elements with respect to other elements of a higher layer. 3. the input to this method is the actual measurement of some parameter or subjective opinion such as preference or satisfaction and the output is a quantified value of each alternative, therefore objective as well as more subjective problems can be quantified. the ahp follows three steps in order to get the final results i.e. qualify decision making framework, pair-wise comparison, and calculating the relative value of each alternative. it provides some space for small inconsistencies in judgment because human beings are not always consistent. the principal eigen vectors give ratio scales and principal eigen value give a consistency index (ci). currently, the ahp is used in many applications such as management sciences and decision making sciences. first proposed by t.l. saaty in the 1970s, the ahp is undoubtedly one of the best decision methods available. ahp mathematically transforms conceptually subjective or fuzzy factors into quantitative variables to evaluate alternatives. we use ahp to quantify the qualitative factors ijahp article: ali/ an mcdm approach towards m-payment business models evaluation international journal of the analytic hierarchy process 283 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.298 considered in this paper and thereby evaluate the performance of the five m-payment business model alternatives. the following procedure can be performed in evaluation using the ahp. first, the construction of a hierarchical structure is carried out, by which the causalities between the factors, sub factors, and alternatives are established. second, the priority weights among the factors are calculated through the pair wise comparison matrix. third, the total value for each payment alternative is calculated based on the priority weights multiplying the data from the expert poll. the following steps are followed using ahp:  pair wise comparison of existing factors which acts as evaluation criteria  finding supporting intensity of each criterion with respect to each alternative  synthesis of the results obtained from the above two steps in order to get final priorities  sensitivity analysis to check sustainability of final results under different conditions figure 2 shows an ahp decision tree which consists of four layers. the top layer shows the objective i.e. evaluation of m-payment models, and the second layer shows the evaluation criteria which consist of service related and organization related factors. similarly, the third layer shows the sub criteria, and fourth layer lists the different alternates which have to be prioritized. 6.2 applying ahp to collected data in this case we have five alternatives and ten evaluation criteria. the evaluation criteria can be categorized as: 1) service related factors 2) organization related factors 6.2.1 pair wise comparison matrix there are ten factors and five alternatives. to calculate the total number of comparisons, we use the formula n (n-1)/2, where n represents the total number of factors. we have six service related factors and four organization related factors, so the total number of comparisons is 6(6-1)/2+4(4-1)/2=15+6=21. similarly, to find the supporting intensity level of each factor with respect to each model, there will be 10*4 =40 comparisons. the all diagonal elements are 1 which indicates that each factor has equal importance to itself. the transitive and reciprocal properties were used e.g. if a12=7 then a21=1/7 and so on. figures 3 & 4 graphically show the relative importance of each factor. in service related factors, scalability and user centric architecture have the top most relative importance. in organization related factors, collaboration & partnerships and response to market trend have top most relative importance. 6.2.2 consistency index (ci) the law of transitivity must be perfectly satisfied in the pair wise comparison matrix. if this is not the case then there will be inconsistencies among the values obtained from expert judgments and the law of transitivity. ijahp article: ali/ an mcdm approach towards m-payment business models evaluation international journal of the analytic hierarchy process 284 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.298 objectives factors alternates figure 2. ahp decision tree if the law of transitivity perfectly holds then max = n. but unfortunately, the estimate of λmax is not equal to n in most cases. therefore, we calculate the ci to determine whether or not the law of transitivity is violated. the formula of the ci is ci= max-1)/ (n-1). when ci = 0, the matrix is entirely consistent, whereas if ci > 0 the matrix is inconsistent. saaty (1980) suggests a range of consistency i.e. if ci > 0.1 then the calculated values are inconsistent and the test will fail. 6.2.3 the priority weights within the hierarchy the priority weights between the factors (and sub factors) are obtained by calculating the eigen vectors in pair wise comparison matrix sets. first, we compared the factors and sub factors to get an overall preference matrix (opm).then, we calculate rvv (relative value weight) by standards methods. the final stage is to construct opm (option performance matrix) and using the equation to get vfm (value for money) i.e. vfm= opm * rvv 6.2.4 data collection we collected data from an expert pool, which included experts from the m-payment domain, for sample data used in ahp. the ahp is primarily a method of decisionmaking in organizations. we therefore conducted an online survey of experts which have considerable experience in the m-payment domain in different multinational organizations. the survey was done in two phases. in first phase, the relative importance evaluation of m-payment business models service related factors organization related factors 1. interface 2. service offering 3. value proposition 4. dynamicity 5. scalability 6. user centric architecture 1. organizing model 2. roi(return on investment) 3. collaboration and partnerships 4. responsiveness to market trend operator centric model bank centric model operator centric with bank interface model peerto -peer model collaboration model ijahp article: ali/ an mcdm approach towards m-payment business models evaluation international journal of the analytic hierarchy process 285 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.298 of each factor was identified on the basis of expert opinions and their personal judgment and experience. in the second phase, the supporting intensity level of each factor with respect to each model was determined. in the survey, we asked the interviewees to measure the degree to which each m-payment business model corresponded to the sub factors on a nine-level ordinal scale. the total value for each payment alternative was then derived by taking the geometric mean for each expert and filling in the corresponding tables. the k-value assignment method was used for collecting data from the experts. 6.2.5 results the eigen vector of the service related factors in table 2 shows that scalability and user centric architecture have more relative importance as compared to other factors. similarly, the eigen value of organization related factors in table 3 shows that collaboration & partnership and response to market trends have more relative importance as compared to other factors table 2 pair wise comparison of service related factors int 1 so 2 vp 3 dy 4 sc 5 uca 6 e.v 7 int 1 2 1 1/7 1/2 ½ 0.092 so 1/2 1 1/2 1/2 1/9 1/9 0.047 vp 1 2 1 1 1/2 ½ 0.118 dy 7 2 1 1 1/2 ½ 0.201 sc 2 9 2 2 1 1 0.271 uca 2 2 2 2 1 1 0.271 1) interface, 2) service offering, 3) value proposition, 4) dynamicity, 5) scalability, 6) user centric architecture, 7) eigen value figure 3. prioritized form of service related factors . table 4 displays the opm (option performance matrix) which shows the supporting intensity level of each factor with respect to each model. table 5 shows the final prioritized form in which the collaboration model is the best model. ijahp article: ali/ an mcdm approach towards m-payment business models evaluation international journal of the analytic hierarchy process 286 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.298 table 3 pair wise comparison of organization related factors om 8 roi 9 c&p 10 rtmt 11 eigen-vector om 1 1/2 1/9 1/9 0.055 roi 2 1 1/2 ½ 0.161 c&p 9 2 1 1 0.392 rtmt 9 2 1 1 0.392 1.000 8) organizing model, 9) return on investment, 10) collaboration and partnership, 11) response to market trend figure 4. prioritized form of organization related factors figure 5 shows the alternatives priorities with respect to service related factors where the cm model has the top priority. figure 6 shows alternative priorities with respect to organization related factors where the opbi model has the top priority. our results are consistent with previous research conducted by (asghari, amidian et al., 2010). table 4 opm (option performance matrix) int so vp dy sc uca om roi c&p rmt bank centric model 0.212 0.038 0.14 0.102 0.12 0.11 0.26 0.19 0.26 0.046 operator centric model 0.111 0.089 0.13 0.102 0.12 0.11 0.18 0.19 0.14 0.17 operator centric using bank interface 0.315 0.25 0.28 0.198 0.23 0.33 0.26 0.36 0.26 0.307 peer to peer model 0.212 0.217 0.13 0.115 0.12 0.11 0.08 0.06 0.08 0.15 collaboration model 0.152 0.40 0.33 0.49 0.40 0.33 0.23 0.19 0.26 0.34 ijahp article: ali/ an mcdm approach towards m-payment business models evaluation international journal of the analytic hierarchy process 287 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.298 figure 5. synthesis with respect to service related factors figure 6. synthesis with respect to organization related factors table 5 final prioritized form priority m-payment business models 1 collaboration model (cm) 2 operator centric using bank interface(opbi) 3 operator centric model (op) 4 bank centric model (bc) 5 peer to peer model (p2p) figure 7. synthesis with respect to objective 7. sensitivity analysis the results of the priorities of different alternatives are very dependent on the relative weights given to the main evaluation criteria. major changes in the final results are possible in case of minor changes in relative values of the main criteria. obviously, the subjectivity factor is involved in human judgment, so it is necessary to test the stability of ijahp article: ali/ an mcdm approach towards m-payment business models evaluation international journal of the analytic hierarchy process 288 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.298 the final results under different conditions. we applied a sensitivity analysis based on those scenarios that replicate alternative future developments or different views on the relative values of the different criteria. through decreasing or increasing the importance of individual criteria, we observed the consequential changes of the priorities and the position of the alternatives. sensitivity analysis is a technique which provides information on the stability of the final ranking. if the alternative ranking is highly sensitive to minute changes in the criteria weights, a careful review of the relative importance is recommended. similarly, extra decision criteria should be incorporated as extremely sensitive levels to a weak bias of the at hand set of criteria. in order to achieve this task the weights of the significant criteria are separately distorted, simulating relative values between 0% and 100%. expert choice 2000 2nd edition software was used to change the local priority values of the chosen subjective factors. figures 8, 9, 10, 11, 12 and table 6 show the sensitivity analysis of different factors and their sensitivity range. in figure 8 we see that the collaboration model at the top position with respect to service related factors, and figure 9 shows that the operator centric model using bank interface is at the top position with respect to organization related factors. figure 10 shows the final synthesis of the results which shows that the collaboration model has top priority. figures 11 and 12 show the % sensitivity (peak relative value) of service and organization related factors which is 30% and 8.3 % beyond which the final prioritized list can be altered. table 6 shows the % sensitivity of all the evaluation factors. figure 8. synthesis with respect to service related factors ijahp article: ali/ an mcdm approach towards m-payment business models evaluation international journal of the analytic hierarchy process 289 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.298 figure 9. synthesis with respect organization related factors figure10. synthesis with respect to objective (final result) ijahp article: ali/ an mcdm approach towards m-payment business models evaluation international journal of the analytic hierarchy process 290 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.298 figure 11. service related factors are changed to 30% figure 12. organization related factors are changes to 8.3 % ijahp article: ali/ an mcdm approach towards m-payment business models evaluation international journal of the analytic hierarchy process 291 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.298 table 6 % change with respect to individual factors s.no factors % change 3 interface 35.4 4 service offering 17.2 5 value proposition 90 6 dynamicity 80 7 scalability 73 8 user centric architecture 73 9 organizing model 55.6 10 roi 83 11 collaboration & partnership 41.5 12 response to market trend 38.2 8. conclusion and future work in this research work, five different mobile payment business models were surveyed. similarly, 10 distinct factors which act as evaluation criteria were surveyed in the field of mobile payment from the relevant literature. a comparison between these five models was made using one of the mcdm techniques. the results show that the collaboration model is the best model on the basis of general criterion taken from the literature, and whose significance was specified using the experts from different multinational organizations through an online survey. a sensitivity analysis was done in order to check the sustainability of the resulting priorities in case of varying conditions. the findings and the proposed generic framework presented in this work provide worth to organizations, customers and other stakeholders which participate in the value chain. the results from this study may also give value to the relevant practitioners, mainly for organizations aspiring to roll out mobile commerce initiatives in the near future including software developers, mobile operators, wireless hardware vendors, and other it industry representatives. this study will also be valuable to organizations looking for innovative and successful business models, whose business is based on m-commerce approaches in order to help them add value. this research work supports managers in the assessment of their business and m-commerce plans by providing substantial understanding about different components of the business models. this framework provides a valuable decision making tool for project managers. in the future, this framework can be validated using different case studies in the m-payment domain. similarly, other mcdm techniques like anp, linear programming etc. can be used in future projects to strengthen these results. this is a generic framework i.e. the evaluation criteria were taken from the literature and their relative importance was measured by experts. in future work, this framework can be tailored to specific contexts. ijahp article: ali/ an mcdm approach towards m-payment business models evaluation international journal of the analytic hierarchy process 292 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.298 references agnieszka,z., elaine,l., & robert,s. (2005). classifying m-payments – a user-centric model. proceedings of the third international conference on mobile business, icmb, 38. asghari, f., amidian,a.a., muhammadi,j & rabiee.h.r. (2010). a fuzzy eletre approach for evaluating mobile payment business models. international conference on management of e-commerce and e-government, ieee. cabanillas, f. j. l., fernandez, j. s., & leiva, f.m. (2014). role of gender on acceptance of mobile payment, journal of industrial management & data systems. 114(2), 220-240. cabanillas, f.l., fernández, j.s.,& leiva, f.m.(2014). antecedents of the adoption of the new mobile payment systems: the moderating effect of age. computers in human behavior, 3, 464–478. cabanillas, f.l., fernández, j.s., leiva,f.m. (2014). the moderating effect of experience in the adoption of mobile-payment tools in virtual social networks: the m-payment acceptance model in virtual social networks (mpam-vsn), international journal of information management, 34, 151–166. cabanillas, f.l., leiva, f.m., & fernandez, j.s. (2013). the impact of risk on the technological acceptance of mobile payment services. international network of business and management, 1, 309–328. camponovo, g. (2002). mobile commerce business models. 2nd m-business conference, austria. chou,y., lee,c,w,. & chung, j. (2004). understanding m-commerce payment systems through the analytic hierarchy process. journal of business research, 57(12), 1423– 1430. coyle, g. (2004). the analytic hierarchy process (ahp). practical strategy: structured tools and techniques. open access material, glasgow: pearson education ltd. isbn 0-273-68220-2, http://www.booksites.net/download/coyle/student files/ahp_technique.pdf. dahlberg, t., mallat,n , ondrus,j., & zmijewska,a. (2008). past, present and future of mobile payments research: a literature review. journal of electronic commerce research and applications, 7, 165–181. grami, a., & schell, b. (2004). future trends in mobile commerce: service offerings, technological advances and security challenges. conference of privacy, security and trust, fredericton, 1-14. josé, r., rodrigues,h.,melro, a., coelho, a., & ferreira, m.c. (2013). design lessons from deploying nfc mobile payments. lncs springer, 86–93. http://www.booksites.net/download/ ijahp article: ali/ an mcdm approach towards m-payment business models evaluation international journal of the analytic hierarchy process 293 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.298 kim, c., mirusmonov, m., lee, i. (2010). an empirical examination of factors influencing the intention to use mobile payment. journal of computers in human behavior, 26, 310–322. ondrus, j. (2008). near field communication: an assessment for future payment systems. journal of information systems and e-business management, 7(3), 347-361. doi: 10.1007/s10257-008-0093-1 pousttchi, k., schiessler, m., & wiedemann, d.g. (2009). proposing a comprehensive framework for analysis and engineering of mobile payment business models. journal of information systems and e-business management, 7(3), 363-393. pousttchi, k., schiessler, m., & wiedemann, d.g. (2007). analyzing the elements of the business model for mobile payment service provision. sixth international conference on the management of mobile business, icmb. pousttchi, k., & hufenbach, y. (2012). mobile payment in the smartphone age extending the mobile payment reference model with non-traditional revenue streams. momm2012, bali, indonesia, 31-38. qiang, yan, & tingjie, l. (2008). a study of mobile payment mode in u-commerce. economics and management school, beijing university of posts and telecommunications, beijing, china. reilly, p.o., duane, a., & andreev, p. (2012). to m-pay or not to m-pay—realizing the potential of smart phones: conceptual modeling and empirical validation. electron markets, 22, 229–241. slade, e., williams, dwivedi,y. & piercy, n. (2014). exploring consumer adoption of proximity mobile payments, journal of strategic marketing, 23(3), 1-15. doi: 10.1080/0965254x.2014.914075 schierz, p. g., schilke, o., & wirtz, b. w. (2010). understanding consumer acceptance of mobile payment services: an empirical analysis, journal of electronic commerce research and applications, 9, 209–216. sharma, s. & gutierrez, j.a. (2010). an evaluation framework for viable business models for m-commerce in the information technology sector, electronic markets, 20(2), 33-52. smart card alliance. (2008). proximity mobile payments business scenarios: research report on stakeholder perspective. a smart card alliance contactless payments council white paper. princeton junction, nj. thair a.d, suhuai, l., peter. s. (2010). consumer acceptance of mobile payments: an empirical study. new trends in information science and service science (niss). 533 – 537. ijahp article: ali/ an mcdm approach towards m-payment business models evaluation international journal of the analytic hierarchy process 294 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.298 tan, g.w., ooi, k.b., chong, s.c., & hew,t.s. (2014). nfc mobile credit card: the next frontier of mobile payment? journal of telematics and informatics, 31, 292–307. zolnowski, a., wei, c., böhmann, t. (2014). representing service business models with the service business model canvas the case of a mobile payment service in the retail industry. 47th hawaii international conference on system science, ieee, 718-727. ijahp news and events: zoffer/pitt creates new ic4cr center to utilize ahp in resolving international conflicts international journal of the analytic hierarchy process 151 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.654 pitt creates new ic4cr center to utilize ahp in resolving international conflicts jerry zoffer professor and dean emeritus katz graduate school of business university of pittsburgh pittsburgh, pa, usa zoffer@katz.pitt.edu the current failure to resolve conflicts worldwide highlights the need for a different approach to conflict resolution. a proposal by university of pittsburgh professors luis vargas and jerry zoffer to create a new international center for conflict resolution (ic4cr) was funded by university of pittsburgh chancellor patrick gallagher and will be housed in the katz graduate school of business. the mission of ic4cr is to provide decision makers with an in-depth understanding of the negotiating positions of all parties and recommend implementation guidelines, based on preferences and priorities, to facilitate resolution of otherwise intractable conflicts. the center will implement this mission by conducting studies of diplomatic and corporate conflicts using the analytic hierarchy process (ahp) developed by the late university of pittsburgh distinguished professor thomas l. saaty. this will require building tradeoff models by eliciting preferences and priorities through the ahp to create a negotiation model which is measurement-based. using the priorities derived from the preferences will allow the decision makers to compute gain/loss ratios of tradeoffs from each party’s perspective. they will be able to identify win-win, non-zero-sum equitable tradeoffs that both parties can claim as a win and permit a road map to facilitate implementing a feasible solution. ic4cr is unique because the use of the ahp helps reduce uncertainty between and among the parties, lessens emotion in the negotiations, and permits a more accurate assessment of the relative value that each side attaches to an issue. ic4cr has already applied this approach to the israeli-palestinian conflict. while the complexities of this conflict will require additional work, much progress has been made in identifying possible tradeoffs which could lead to a solution. another project being developed in the city of pittsburgh is how to reconcile, to the community’s and the police department’s benefit, policing and community interests. another major proposal which may be financed by the federal government is to focus on the seemingly intractable issue of how to improve the recidivism rate and develop programs which will encourage those released from prison to begin more productive lives. ijahp: mu/ijahp and the next generation of ahp/anp scholars international journal of the analytic hierarchy process 204 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.416 ijahp and the next generation of ahp/anp scholars writing a good journal article requires experience. the earlier prospective scholars start writing articles for a journal, the sooner they will learn the art of writing good journal articles. it is for this reason that this journal encourages young scholars and practitioners, whom we consider the next generation of ahp/anp scholars, to submit their papers to this academic outlet. our editorial board makes an extra effort to assist authors in editing their paper to make it not only comprehensible but also engaging for our readers. however, be aware that we do not get involved with the study itself, which must stand on its own in terms of quality and rigor, but rather with the writing of the journal article. in other words, we want to ensure that good studies, developed by this young generation of scholars and practitioners, are not left out due to the steep learning curve involved in learning how to write engaging journal articles. some articles in this issue have been coauthored by seasoned scholars along with next generation of scholars. given the large number of articles in this issue we cannot introduce them all to the reader, however we will highlight a few. our articles section starts with a very timely paper by thomas saaty and lirong wei that asks the provocative question, “should the uk have brexited the european union?” based on the results, we just wish this study would have been done and publicized before the referendum! we would also like to call your attention to the article by ana lucia pegetti and jesse d´assunção rebello de souza jr, titled, “cognitive maps and ahp for supplier selection in a private higher education institution”. this article won an award at isahp2014 in washington, d.c. thomas saaty closes the articles section with the paper, “five ways to combine tangibles and intangibles”. this article is significant because an important characteristic of ahp/anp is precisely the ability to combine tangibles and intangibles. in our essays, reviews and comments section we have a treat for the reader: an essay by william adams and elena rokou, describing the simpleahp initiative to engage children and young decision-makers in the use of ahp, which was successfully presented by the first author at isahp2016, london uk. next, valentina ferretti, our news and events editor informs our readers about the publication of two ahp books this month: first, a book titled applications and theory in analytic hierarchy process decision making for strategic decisions, edited by fabio de felice, thomas l. saaty and antonella petrillo. this book offers a very diverse ijahp: mu/ijahp and the next generation of ahp/anp scholars international journal of the analytic hierarchy process 205 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.416 compendium of ahp applications and some very interesting theoretical discussions. this book will constitute a valuable reference for the ahp scholar and practitioner. it is published by intech and is available both in open access (free) format and as a printed book. the second book is practical decision making: introduction to the analytic hierarchy process using super decisions v2, written by enrique mu and milagros pereyra-rojas and published by springer. this book is aimed at the novice with minimum mathematical background, and the authors indicate that it has been conceived to allow the reader to learn ahp on their own using the freely available software package super decisions v2. next in this section, multichoice, a new anp software package developed by maria mirlova and olga andreichikova at the russian academy of science in moscow, is announced. this news and events section ends with a brief note on the successful isahp2016 meeting in london, uk. enjoy this issue! enrique mu, phd editor-in-chief ijahp news and events: isahp meeting, malaysia, june 23-26, 2013 international journal of the analytic hierarchy process 168 vol. 4 issue 2 2012 issn 1936-6744 rafikul islam chairman, isahp 2013 international islamic university, malaysia email: rislam@iium.edu.my the next biennial meeting of the analytic hierarchy process/analytic network process symposium will be held june 2326, 2013 in hotel istana, located in the heart of kuala lumpur, the capital city of malaysia. this is a truly international event attracting people from more than 40 countries at the last meeting which took place in 2011 in sorrento, italy. it is your chance to share your ideas with people interested in your research area who have the knowledge to give you useful feedback about your work and you can also learn new ideas. the success of this event essentially depends upon your active participation and promotion to your friends. we look forward to seeing you in kuala lumpur in june 2013, if not earlier. http://malaysia2013.isahp.org/index.htm important dates extended abstract submission deadline: 30 december, 2012 notification of acceptance: 31 january, 2013 full paper submission deadline: 28 february, 2013 early registration deadline: 15 march, 2013 normal registration deadline 31 may, 2013 symposium time: 23-26 june, 2013 isahp meeting in malaysia, june 23-26, 2013 mailto:rislam@iium.edu.my http://malaysia2013.isahp.org/index.htm rob typewritten text http://dx.doi.org/10.13033/ijahp.v4i2.161 ijahp news and events: petrillo/italian translation of “creative thinking, problem solving and decision making” international journal of the analytic hierarchy process 548 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.454 italian translation of “creative thinking, problem solving & decision making” by thomas saaty antonella petrillo parthenope university of naples department of mechanical engineering italy a.petrillo@unicas.it this book is the italian translation of “creative thinking, problem solving & decision making” by professor thomas saaty, originally published in 2001 by rws publications. the title of the italian translation published in november 2016 is “pensiero creativo, problem solving e decision making” and was a collaborative effort of fabio de felice, thomas l. saaty and antonella petrillo and published by aracne publisher. we are very grateful to professor thomas l. saaty who gave us the opportunity to work with him on this project. for us it was an inestimable honor, and we are very proud of this recognition. the fundamental premise of this book is that creativity can be taught and learned quite effectively; which might come as a surprise. it may also come as a surprise that intelligence (or the lack of it) does not restrict creativity. the book explores this second premise which has been validated in practice. in fact, creativity only requires enough intelligence to enable one to think a little and to collect one’s ideas in a purposeful manner. one may not be very intelligent, yet be a creative genius. for example, beethoven was an undeniable genius at music but a sad case at arithmetic. this book is intended to be both challenging and entertaining. it is designed to help readers feel more comfortable with the topic, and more confident of their own creative abilities. it aims to inspire readers to try the presented exercises in creativity, and invites them to laugh, scowl and disagree, and most of all, to explore what may, at first seem like unfamiliar territory. each of us possesses immense resources, and this book provides an opening to allow them to come forth. http://www.intechopen.com/books/editor/applications-and-theory-of-analytic-hierarchy-process-decision-making-for-strategic-decisions ijahp news and events: petrillo/italian translation of “creative thinking, problem solving and decision making” international journal of the analytic hierarchy process 549 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.454 link http://www.aracneeditrice.it/aracneweb/index.php/pubblicazione.html?item=9788854 897472 “pensiero creativo, problem solving e decision making” translation by fabio de felice, thomas l. saaty and antonella petrillo, isbn 978-88-548-9747-2, print isbn 978-953-51-2560-0, 364 pages http://www.aracneeditrice.it/aracneweb/index.php/pubblicazione.html?item=9788854897472 http://www.aracneeditrice.it/aracneweb/index.php/pubblicazione.html?item=9788854897472 http://www.intechopen.com/books/editor/applications-and-theory-of-analytic-hierarchy-process-decision-making-for-strategic-decisions http://www.intechopen.com/books/editor/applications-and-theory-of-analytic-hierarchy-process-decision-making-for-strategic-decisions ijahp article: kaftandzieva, cvetkoska/ financial decision making among young adults: an ahp approach international journal of the analytic hierarchy process 458 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.864 financial decision-making among young adults: an ahp approach tamara kaftandzieva ph.d. scholar at department of organizational sciences (management), ss. cyril and methodius university in skopje faculty of economics skopje republic of north macedonia tamara.kaftandzieva@gmail.com violeta cvetkoska associate professor at department of management, ss. cyril and methodius university in skopje faculty of economics skopje republic of north macedonia vcvetkoska@eccf.ukim.edu.mk abstract in an uncertain economic environment, the decision-making process regarding personal finances relies heavily on personal experience and behavior, and is largely influenced by a variety of psychological and socio-demographic factors. the aim of this paper is to analyze the key factors of the decision-making process regarding financial choices of the population of young adults in the republic of north macedonia, and to further explain young people’s motives for the proposed decision and the conditions under which the decision was made. the research was conducted through an ahp-based questionnaire that was distributed to respondents ranging in age from 18 to 35 years. according to the obtained results, the respondents value financial security the most, hence their primary choice is investment in real estate and commodities. young adults are less inclined to invest, especially in the more complex financial instruments. the developed ahp model will help young people make better, fact-based financial choices. keywords: financial decision-making; personal finance; young adult population; ahp 1. introduction personal finance is the financial management which an individual performs over time, and encompasses various activities in order to meet personal financial goals, both shortterm and long-term, by taking into account various risks and future life events. the process of making sound financial decisions is determined by a variety of factors. therefore, spending, saving, and investment dynamics should consider the fulfillment of personal needs while taking into account financial constraints. people make various financial choices in accordance with their desires, motives, attitudes, affinities and goals, but mostly, their individual willingness to take risks. contrary to the common belief that people are logical decision-makers, they in fact behave irrationally and are not able to mailto:vcvetkoska@eccf.ukim.edu.mk ijahp article: kaftandzieva, cvetkoska/ financial decision making among young adults: an ahp approach international journal of the analytic hierarchy process 459 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.864 make fully informed decisions because they are highly susceptible to cognitive biases (kahneman & tversky, 1972). as financial decisions are often made in highly complex and uncertain situations that lack formal rules for decision-making, many conclusions rely on intuition (kahneman et al., 1998). personal finance is a research area that is gaining momentum, especially when taking into account the increased interest in the concept of risk in the business environment. this is due to a greater awareness of risks after a series of corporate scandals, the global financial and economic crisis, as well as the current covid-19 pandemic. currently, this research area is immensely popular due to the intensified interest in incorporating psychology in economic sciences and answering the previously unanswered questions regarding human behavior. the decision-making process itself is considered a cognitive process where investors make a decision based on various alternatives available to them. however, the researchers discovered that psychological and behavioral factors influenced decisionmaking to a large extent. the perception and processing of information about risk-return trade-off and the characteristics of the asset are at the core of the psychological aspects that lead to the process of making financial decisions. however, apart from the underlying information, the presence of psychological individual-specific factors influences investors’ behavior and financial decision-making (charness et al., 2010). moreover, one of the most frequent associations between personal finance and financial education is with financial education; the underlying idea being that without adequate knowledge and skills, people cannot satisfactorily manage their own finances, particularly in a dynamic and complex environment (carlin et al., 2012; lusardi, 2008). financial decision-making is indeed limited by the practical possibilities of exercising financial choices and is immensely influenced by the domestic financial system and economic environment. the current study was conducted to identify whether young adults make personal finance decisions spontaneously or based on a plan and strategy, linking the process with the economic and the demographic developments in the country, and understanding what course of action is necessary to result in an improved outcome. this kind of information is immensely important in the efforts to reduce the intensity of anomalies and systematic deviations from rational judgment in the future. hence, the main objective of our paper is to help the young adult population make better, fact-based financial decisions. the aim of this paper is to analyze the decision-making process regarding financial choices of the young adult population in the republic of north macedonia, and to further explain young people’s motives for the proposed decision and the conditions under which such a decision was made by using the most popular multicriteria decision-making (mcdm) method, the analytic hierarchy process (ahp). the ahp method has been used in the area of finance because it provides a means of structuring and decomposition of otherwise complex decisions, modeling daily real life problems with ease and simplicity, and developing transparent decisions while taking into account various aspects. although decision-making regarding personal finance is inevitably based on quantitative data, the qualitative factors that shape subjective judgments should be considered as well, in order to avoid decision-making based on intuition. therefore, the use of a method that allows the incorporation of qualitative measures into quantitative research is advised. the ahp allows the selection of the best ijahp article: kaftandzieva, cvetkoska/ financial decision making among young adults: an ahp approach international journal of the analytic hierarchy process 460 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.864 alternative out of a range of alternatives, by including both quantitative and qualitative criteria into the decision-making process. the above justifies the methodological correctness of ahp application for the construction of a model that determines the financial decision-making of young adults. on the other hand, the process may be timeconsuming, requiring too much effort and human input. therefore, the decision-maker should use suitable tools to evaluate and solve a particular situation or problem, keeping in mind both the internal and external factors and variety of qualitative and quantitative information integrated therein (saaty, 2012). the developed ahp model may be used by a developing country's young adult population to choose the best financial option. additionally, academics and scholars with an interest in ahp-personal finance may use this model for their own study. also, this research contributes to the ahp-personal finance literature, which is critical for developing nations with low levels of financial literacy. the rest of the paper is structured as follows: section 2 provides a literature review of the mcdm and ahp studies in the field of finance; the ahp methodology used in the analysis is introduced in section 3, while the results are discussed in section 4; and section 5 provides a conclusion and directions for further research. 2. literature review the first bibliographic survey related to the application of mcdm in finance covered 265 references from 1955 to 2001 and found that most are focused on the field of portfolio analysis and general financial planning (steuer & na, 2003). the mcdm approaches complement and enhance the existing normative and descriptive models in financial decision-making, primarily in the field of portfolio selection (zopounidis & doumpos, 2013). the study by zopounidis et al. (2015) covered the period from 2002 to 2014, thus indicating that portfolio management remains the most popular area, apart from the interest that new issues have attracted, with china dominating the output in terms of the greatest number of authors and where they work. almeida-filho et al. (2020) confirmed the findings of earlier literature reviews, indicating that portfolio optimization and the problem of ranking continue to be the areas where studies are most likely to tackle multicriteria approaches. furthermore, they point out that applications of mcdm analysis in finance have increased, which was particularly noticeable after the global economic crisis in 2008, when making more informed and transparent decisions emerged as an imperative, thus growing exponentially and reaching a peak in 2018. however, the fields of management science, economics, accounting, and finance remain underexplored, mainly due to the complex networks of interdependencies and lack of agreement on the priorities that shape the decision-making process, especially in dynamic and turbulent environments (franek & kashi, 2014; goyal et al., 2020). taking into account the theoretical framework, the immense popularity of the mcdm methodologies in the past several years has been evident, with the ahp method growing at a very fast pace. in light of that, mardani et al. (2015) emphasized the importance of the ahp as the most prominent and most used method, as it was applied in 128 of 393 papers mcdm papers that were analyzed. in addition, its original founder, thomas l. saaty, is the most cited author in the field of research (goyal et al., 2020). the ijahp article: kaftandzieva, cvetkoska/ financial decision making among young adults: an ahp approach international journal of the analytic hierarchy process 461 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.864 conclusions of the aforementioned literature reviews were supported by khan and ali (2020), who explored 920 articles in the period from 2000 to 2019, further proving the preference of researchers for the ahp method to resolve complex scenarios based on multiple different criteria and choose the best and most efficient alternative based on the obtained priorities. they found that the distribution of ahp publications by country showed it is predominantly applied in turkey, followed by china and india, whereas north macedonia is among the countries with the lowest ahp publications. the ahp has been applied in numerous areas of finance including real estate, when considering several criteria prior to buying a house or an apartment (obeidat et al., 2018); development of a credit scoring model, when evaluating the creditworthiness of smes (roy & shaw, 2021); sustainable local development, with performance of a swot analysis on the best strategies for implementation of a complementary monetary system (escobar et al., 2020); and efficient allocation of scarce capital resources (zopounidis et al., 2018). however, our particular interest lies in studies on the application of the ahp in financial decision-making regarding investment strategy selection and personal finance. according to anthony and joseph (2017), investors’ decision-making is adversely affected by various psychological/behavioral factors, i.e., investors are most greatly influenced by over-confidence bias and regret aversion. wu et al. (2012), in their study on selection of the best investment alternative of fund investment, bonds investment, stock investment, and real estate investment, adopted the ahp methodology mainly because the wisdom of the group eliminates bias generated by personal preferences, thus improving evaluation accuracy. the study concludes that the best investment strategy is real estate investment, followed by stock strategy and fund strategy, while the worst investment is bonds investment. the findings from gawlik’s paper (2019) focused on the younger generation; the target group was composed of 14 respondents from a sample of 200 university students. the focus on the quality of life determinants indicated that the criteria respondents value the most are safety, stability and certainty. as earlier noted, the ahp method is one of the most popular mcdm methods and its popularity is constantly rising, mainly due to its flexibility and versatility, which has enabled it to be combined with various other techniques instead of as a stand-alone tool (vaidya & kumar, 2006). this is especially important since the complexity of financial decision problems often necessitates the use of an arsenal of analytical approaches. in light of that finding, pradhan et al. (2019) reviewed the literature on the ahp integrated with deahp published between 2000 and 2018, and concluded that due to its high accuracy, the integrated deahp approach is widely applied in the field of performance and efficiency measurement, optimization and ranking of different products or services in a variety of sectors. however, when proposing hybrid and integrated methodologies, special care should be given to keeping the resulting models as simple and user-friendly as possible. sophisticated analytic methodologies are not often used in practice because they are far too time-consuming and complex for financial decision-makers to understand (zopounidis et al., 2015). despite the rising popularity of the topic, we identified a lack of literature dealing with the application of the ahp methodology in personal finance focused predominantly on the young adult population, which motivated us to take a step forward and contribute to ijahp article: kaftandzieva, cvetkoska/ financial decision making among young adults: an ahp approach international journal of the analytic hierarchy process 462 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.864 its enrichment. we searched the scopus database by article title, abstract, and keywords for the terms personal finance, ahp, and decision-making over an all-year period (from the first publication in the scopus database until april, 2021). however, no papers were identified for this purpose. we also performed a search in the web of science core collection database on the topic of personal finance and ahp and decisionmaking between 1900 and (april) 2021. the lack of any papers considering the problem explained in our paper further highlights the originality of our research. furthermore, we decided to conduct the study in the republic of north macedonia since our goal was to examine this issue in a developing country; the republic of north macedonia gained monetary and financial independence in 1992 and their financial system is bank-based. 3. methodology the analytic hierarchy process is a mathematical method for problem-solving that is widely used in multi-criteria decision-making for tackling multi-layered problems in real situations, primarily due to its ability to analyze both qualitative and quantitative criteria (saaty, 1980). the basic model consists of three main parts comprising a hierarchy structure as follows: first, setting a goal that needs to be solved; second, selecting the criteria (with the possibility of further breaking them down to the level of sub-criteria); and third, defining the alternatives that need to be evaluated based on the criteria selected (saaty, 2012). according to saaty (2012), the essence of the ahp method is that both the human judgments and the underlying information can be used in the evaluations and converted into numerical values that can be processed and compared over the entire range of the problem. namely, a numerical weight or priority is derived for each element of the hierarchy through pair-wise comparison, allowing diverse and often incommensurable elements to be compared to one another in a rational and consistent way. the pair-wise comparison judgments are based on the relative importance or preference of the individual, using the fundamental scale developed by saaty (2012). decision-makers are asked to carefully compare the criteria in pairs regarding the goal and assign an intensity of importance from the saaty’s scale of relative importance with 9 degrees (saaty & kearns, 1991, p. 27). the question that should be asked in this comparison is how many times one criterion is more important than the other regarding the goal. the assigned intensity of importance is used to comprise the matrix of relative importance so that the more important criterion will obtain a proper whole number, while the other the reciprocal value. then, the matrix of relative importance is normalized and the weight of each criterion is calculated as an average of values in each row of the normalized matrix. the procedure is repeated for the comparison of the alternatives regarding each criterion asking the question which alternative is preferred more regarding the criterion. for the alternatives, the local priorities are calculated in the same way as the weights of the criteria and are then synthesized into overall priorities. the overall priorities are used to rank the alternatives (the alternative with the highest overall priority will have a rank of 1, etc.). apart from individual decision-making, the ahp is designed for collaborative decisionmaking and problem-solving since a certain number of experts can find a solution close to the real-world situation (saaty, 1986; saaty & peniwati, 2007). when ahp is used for ijahp article: kaftandzieva, cvetkoska/ financial decision making among young adults: an ahp approach international journal of the analytic hierarchy process 463 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.864 group decision-making, the geometric mean should be calculated for each comparison of the elements based on which matrix of relative importance will be filled (aczel & saaty, 2003). the ahp allows the consistency of decision-makers to be checked by calculating the consistency ratio (cr). max ( 1) ci n cr ri ri n      where ci is the consistency index, ri is the random index, λmax is the largest eigenvalue of the n-order matrix, and n is the number of compared elements in pairs, i.e., criteria and alternatives, separately. based on the matrix size, there is a proper random consistency (saaty & kearns, 1991, p. 34). a value of the cr which is less than or equal to 10% is acceptable, and if it is higher, the decision-makers should revise their judgments. the last step when applying the ahp is to conduct a sensitivity analysis which makes it possible to monitor how the change of the inputs (criteria, sub-criteria (if any)) affects the outputs (e.g. rank of the alternatives). 3.1 research design this paper is focused on selecting the relevant criteria that shape the financial choices of young people in the republic of north macedonia as well as defining the alternatives for reaching the goal of making a decision regarding personal finance. this justifies the methodological correctness of the ahp model and its application in decision-making regarding personal finance. therefore, the development of the model was carried out in two independent phases. in the first phase of the research, a questionnaire consisting of 11 questions was prepared (appendix 1). this questionnaire was based on a well-known book in the area of personal finance by ryan and ryan (2015), a survey of financial literacy developed by the oecd (2011) and a personal financial planning questionnaire (2020) normally used when seeking financial advice, with the target group being young adults from 18 to 35 years of age. the first three questions ask about demographic information including gender, age, and level of education, while the rest of the questions focus on gaining information about the respondents’ awareness, knowledge, skills, attitude, and behavior that are necessary to make sound financial decisions. the questionnaire was sent to 75 young adults by email in february 2020; they were given one week to fill it in and send it back to the authors. this paper emphasizes the last three questions of the questionnaire with the aim of choosing the most important criteria when making a financial decision. since the survey was conducted among individuals from 18 to 35 years of age, the alternatives were selected based on their common habits and preferences and the characteristics of the macedonian financial system. since all the respondents in this sample fall into the same category (the youth in the country), no major dispersion is expected in their habits and attitudes. hence, 51 respondents who completed the questionnaire participated in creating the model and selecting the sub-criteria, criteria and the alternatives. in the second phase of the research, an ahp questionnaire was created. this was used to develop a multiple-criteria ahp model where the sub-criteria are grouped into five criteria based on the obtained results from the first stage of the research (questionnaire ijahp article: kaftandzieva, cvetkoska/ financial decision making among young adults: an ahp approach international journal of the analytic hierarchy process 464 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.864 1). once the ahp model was developed and the hierarchy was designed, a subsample of 9 randomly chosen participants was invited to the second stage, where the ranking of each element of the hierarchy was performed. the respondents were asked the hypothetical question, how would they manage their money, supposedly having inherited 10,000 euros. the goal is to determine the financial decision-making among young adults while giving them the same starting point despite their different backgrounds and levels of income, which might influence the results. however, since the ahp has only recently been used among researchers in our country and since the procedure of answering the questions can be time-consuming, we were worried about the respondent’s lack of knowledge regarding the model. therefore, we decided to take a subsample of 9 participants by random sampling. prior to completing the questionnaire, the respondents had been trained and the ahp method was explained to them via a zoom meeting. the design of the questionnaire was specified for the ahp method, and the respondents conducted pair-wise comparisons across all possible combinations of reasons (sato, 2009) using saaty’s scale of relative importance (saaty & kearns, 1991, p. 27). first, they valued the alternatives with regard to each of the sub-criteria in pair-wise comparisons. for that purpose, 16 questions were developed (appendix 2, q1) for each of the sub-criteria respectively. next, the sub-criteria were evaluated regarding the main criteria in 5 different questions for each of the criteria (appendix 2, q2). finally, the criteria were evaluated in pair-wise comparisons with respect to the main goal (appendix 2, q3). each of the respondents filled in a separate ahp questionnaire sent by e-mail and after completion they sent the results back to the authors. although the process of filling out the questionnaire was quite complex and time-consuming, the respondents were able to express their preference for a particular alternative over the others. taking into account that the design of the questionnaire reflects the results to a great extent (sato, 2009), the design of an ahp-based questionnaire must be in that form in order to reflect the relative importance of alternatives to results. keeping in mind the group decision-making, the aggregation of perspectives was obtained by using a geometric mean as in azcel and saaty (1983). each participant’s questionnaire was solved in the super decisions software in order to determine the level of inconsistency. since the software indicates the comparisons that seem most inconsistent, the 9 respondents were asked to review those judgments. therefore, when necessary, the process was repeated, in order to keep the inconsistency below or equal to 10%. the developed model with the respective criteria, sub-criteria and alternatives was shown to the respondents who confirmed it, thus the decision-making was obtained by consensus. 4. results and discussion out of the 75 distributed questionnaires, 51 responses were collected for a response rate of 68%. out of 51 respondents, 29 are female and 22 are male. the majority of the respondents (27) are between 26 and 30 years of age, 16 respondents are between 18 and 25 years old, and 8 respondents are between 31 and 35 years old. regarding their level of education, the largest number of the respondents have acquired higher education (35), 11 have ma degrees and 5 have finished high school. the answers to questions 4 and 5 were used to develop the alternatives in the ahp model. while the majority of the respondents ijahp article: kaftandzieva, cvetkoska/ financial decision making among young adults: an ahp approach international journal of the analytic hierarchy process 465 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.864 answered that they are familiar with each of the financial products (appendix 1, q4), none of them have recently held bonds, shares in pension funds, options or futures (appendix 1, q5), all of which were excluded from the multi-criteria analysis. based on their characteristics, real estate and commodities were merged into one alternative. the aim of question 6 was to determine the respondents’ attitude towards risk and finances and accordingly, their behavior. twenty-three of the respondents are risk-averse (disinclined or reluctant to take risks), 16 are risk-neutral and 12 are risk-tolerant. since information is very important in making a financial decision (decisions about saving, spending and investing), the majority of the respondents (18) rely on recommendations from independent financial advisors or brokers the most, product-specific information picked up in a branch or from sales staff (11), their own previous experience (9), and the advice of friends/relatives (6). thirty-three respondents stated that their personal objective in making financial decisions is to provide a comfortable life, 7 stated that their goal is to provide for their family, 6 answered that their aim is to save regularly, and for the rest, the main goal is to make a major purchase. the remaining questions were developed in order to determine the sub-criteria and group them respectively into criteria for financial decision-making. when making a financial decision, given the investment objectives, the respondents pay the most attention to stability (the average grade of importance is 4.49), followed by information (4.47), return (4.27), liquidity (4.18), volatility (3.98), availability (3.98) and simplicity (3.61). regarding the time frame, the majority of the respondents (33) prefer medium-term investments, 13 prefer short-term investments, while only 5 of the respondents prefer long-term investments. considering the proposed personal characteristics, respondents pay the most attention to risk attitude (4.08), followed by level of income (3.71), financial education (3.69), ability to save money (3.45), financial priorities (3.35) and experience (3.22). based on the judgments of the 9 respondents, we developed a multi-criteria ahp model oriented towards financial decision-making. the hierarchy structure of our ahp model is shown in figure 1. first, the respondents were asked to assess the following five criteria with the respective sub-criteria regarding the goal (how they make financial decisions regarding personal finances): profitability (return, liquidity and volatility); financial security (simplicity, availability, stability and information); time horizon (short-term, medium-term and long-term); personal characteristics (income, ability to save and financial priorities), and readiness (risk aversion, financial education and experience). then, they were asked to evaluate the following alternatives as possible solutions to the proposed goal: saving in the form of a bank deposit, investments on the stock market, buying a life insurance policy, investments in mutual funds, and investments in real estate and/or commodities, with the goal being how you would invest a 10.000 eur inheritance. ijahp article: kaftandzieva, cvetkoska/ financial decision making among young adults: an ahp approach international journal of the analytic hierarchy process 466 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.864 figure 1 hierarchy structure of our ahp model based on the obtained data from the respondents for the pair-wise comparisons for the sub-criteria, criteria and alternatives, we used the geometric mean to obtain the corresponding consensus (aczel & saaty, 1983). first, when evaluating the alternatives with regards to each of the sub-criteria in pair-wise comparisons, when taking the subcriterion return into consideration, for example, the respondents found life insurance moderately more important than bank deposits; hence, the score for life insurance is 3, and for bank deposits is 1 3 . likewise, investment in mutual funds is 4 times more preferable than bank deposits, or bank deposits are 1 4 as preferable as investments in mutual funds. following this procedure, 10 pair-wise comparisons of the alternatives were made for each of the 16 sub-criteria. then, the sub-criteria were evaluated regarding the 5 main criteria. for example, when referring to the criterion of profitability, a total of 3 pair-wise comparisons were made. according to the respondents, return is moderately to strongly more important than liquidity, with a score of 4, hence the score of liquidity is 1 4 . as a result, liquidity is twice as preferred as volatility; therefore, liquidity is 2 and volatility is 1 2 . the criterion return is moderately to strongly more preferred than the criterion volatility, thus noting the score of 4 for return and 1 4 for volatility. table 1 shows the priorities of the criteria and their respective sub-criteria of the normalized matrix. based on the priority vectors, the criterion respondents value the most is financial security (0.46505), with stability (0.55996) being the highest valued sub-criterion. this finding is in line with the findings of gawlik (2019), who also focused on the young adult population. the consistency ratio is below 10% (7.67%), meaning the set of judgments is consistent, reliable and trustworthy. ijahp article: kaftandzieva, cvetkoska/ financial decision making among young adults: an ahp approach international journal of the analytic hierarchy process 467 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.864 table 1 prioritization of main criteria and sub-criteria criteria and sub-criteria priorities financial security 0.46505 availability 0.17666 information 0.17282 simplicity 0.09056 stability 0.55996 personal characteristics 0.07974 ability to save money 0.16342 financial priorities 0.29696 level of income 0.53962 profitability 0.15442 liquidity 0.20813 return 0.66076 volatility 0.13111 readiness 0.25274 experience 0.13650 financial education 0.23849 risk attitude 0.62501 time horizon 0.04806 long-term 0.10473 medium-term 0.63699 short-term 0.25829 table 2 presents the local priorities of each alternative regarding each sub-criterion, as well as their overall priorities, from which the ranking is made. thus, by using this model, the alternatives are ranked in order of the respondents’ preference. ijahp article: kaftandzieva, cvetkoska/ financial decision making among young adults: an ahp approach international journal of the analytic hierarchy process 468 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.864 table 2 summarized results for the alternatives sub-criteria bank deposits life insurance mutual funds real estate and commodities stocks availability 0.36331 0.12011 0.36331 0.09782 0.05546 information 0.11531 0.34887 0.34887 0.13938 0.04758 simplicity 0.39135 0.07098 0.22831 0.26013 0.04924 stability 0.17345 0.19643 0.06535 0.53112 0.03365 ability to save money 0.26868 0.46974 0.06832 0.14536 0.04791 financial priorities 0.15393 0.13340 0.26002 0.38916 0.06349 level of income 0.19005 0.31085 0.31126 0.09253 0.09532 liquidity 0.36957 0.05992 0.12822 0.13559 0.30670 return 0.08883 0.22863 0.30442 0.07906 0.29906 volatility 0.14220 0.10566 0.25502 0.45483 0.04229 experience 0.43444 0.07786 0.19103 0.25529 0.04139 financial education 0.37036 0.05936 0.21534 0.29752 0.05743 risk attitude 0.17382 0.08846 0.15997 0.53621 0.04154 long-term 0.06847 0.36043 0.15406 0.37359 0.04345 medium-term 0.27619 0.05298 0.35456 0.14048 0.17579 short-term 0.36161 0.04859 0.15118 0.08271 0.35591 overall priorities 0.21796 0.16637 0.20365 0.32388 0.08814 ranking 2 4 3 1 5 the results were obtained by using the superdecisions software. the graphical representation of the overall priorities of the alternatives is shown in figure 2. based on the results in table 2 and figure 2, it can be seen that the first priority for the young adult population is investment in real estate and commodities (0.32388), followed by bank deposits (0.21796), and mutual funds (0.20365), although there is a slight difference between the second and the third priority. the respondents are the least interested in life insurance (0.16637) and investments in the stock market (0.08814). figure 2 graphical representation of the overall priorities of the alternatives in superdecisions ijahp article: kaftandzieva, cvetkoska/ financial decision making among young adults: an ahp approach international journal of the analytic hierarchy process 469 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.864 if an investor is willing to save and/or invest before deciding on an asset allocation, they should take into consideration a variety of characteristics such as their investment goals, available resources, the timeline in which they expect a return on investment, and their attitude towards risk. as a result, people can save or invest in a wide range of financial instruments in either the money market or the capital market, each with a different rate of return and thus a different degree of risk, depending on their utility perception, i.e., maximizing wealth. similar to anthony and joseph (2017), we found that consumers’ actions, behavior, attitudes and preferences are what ultimately shape their financial situation and well-being in the short-term and long-term, thus ultimately resulting in undertaking an action. if they prefer to prioritize short-term needs, then they are unlikely to focus on emergency savings or make long-term financial plans. according to the findings of research performed by the oecd (2020) on financial attitudes, only 25% of the population finds it more satisfying to save for the long-term rather than spending. furthermore, 57% of the respondents focus on satisfying short-term financial needs instead of long-term financial goals. however, this is also related to the population’s income level and saving patterns. the obtained results show that the majority of the respondents value financial security the most. therefore, in the proposed hypothetical situation, with those particular alternatives, they would have invested the hypothetical inheritance of 10,000 eur in real estate or commodities at this stage of life because even if they were employed, with their current salaries, they could not save enough to buy a valuable and permanent asset. this finding is in line with wu et al. (2012). the next choice would be saving in bank deposits, which is in line with the traditional propensity of macedonians to use banks for saving despite the low interest rate environment. the banking sector in north macedonia occupies the largest and most dominant part of the financial system, with constant growth in household deposits. however, the relatively low yields offered by deposit products increase the possibility of alternative ways of saving. experience shows that a small percentage of investors make this decision because they consider alternative ways of investing as too risky or complicated. for example, mutual funds are an alternative investment, but despite being one of the fastest growing financial institutions in the last several years, their share of the assets of the financial system is still low; in contrast, their popularity worldwide is rapidly increasing, primarily among investors who are willing to take a bigger risk amid falling interest rates. nonetheless, a small number of the respondents prefer investment in life insurance or on the stock market, partly because of their risk aversion, but also due to lack of information and financial knowledge. although the insurance sector has significant potential for further development in both segments (life and non-life), the young adult population is more oriented towards non-life insurance products at this stage of life, which do not contain a savings/investment component. life insurance is not very popular among young people because of the long period of time between investing and receiving the benefits. the significance of the capital market for the macedonian financial system is small, primarily due to the modest supply of securities and the small volume of stock exchange trading in these instruments, meaning it is still underdeveloped. the lack of innovation and competitiveness of issuers, on the one hand, and the insufficient information and insecurity of potential investors in macedonian companies, on the other hand, also have a certain influence on the underdevelopment of the domestic capital market. in addition to the modest supply of ijahp article: kaftandzieva, cvetkoska/ financial decision making among young adults: an ahp approach international journal of the analytic hierarchy process 470 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.864 attractive financial instruments, insufficient investment is a result of a low standard of living, the risk culture, and the traditional habits of consumption and savings. decision-making regarding personal finance among young adults is a multi-layered, social and economic issue, which is related to the level of youth unemployment and underemployment. despite a slight decrease in the unemployment rate, youth unemployment is a major concern in the macedonian economy. according to research conducted by eurostat (2020), out of the analyzed countries, north macedonia has the highest unemployment rate in the 15-29 age group (30.2%) and only one-fifth of young adults earn enough to make a living on their own. when it comes to decision-making regarding personal finance, another emerging problem is the lack of financial literacy in north macedonia. financial literacy is rapidly being recognized as a core skill, and is a combination of awareness, knowledge, skills, attitude, and behavior necessary to make sound financial decisions and ultimately achieve individual financial well-being. hence, it is essential for consumers operating in an increasingly complex financial landscape. the average financial literacy score for the north macedonian population is 11.8 out of a maximum 21 points. compared to the average financial literacy score in the g20 countries (12.7), the overall financial literacy score in north macedonia is 0.9 points lower, indicating the need to enhance financial literacy initiatives in the country (oecd, 2020). financial literacy and knowledge certainly do not guarantee a secure future yield, but they are necessary in order to be informed and prepared. 4.1 sensitivity analysis the last step of the ahp analysis was performing a sensitivity analysis to show the effect that altering different parameters of the model would have on the choice of the best alternative with regards to personal finances. the sensitivity analysis is primarily conducted because the priorities of the alternatives are highly dependent on the weights assigned to the main criteria. since these weights are usually based on highly subjective judgments, the stability of the ranking under varying criteria weights has to be tested (chang et al., 2007). figures 3-7 display a series of sensitivity analyses that were conducted to investigate the impact of changing the priority of the criteria on the alternatives’ ranking. therefore, two scenarios for each of the criteria were observed, with a total of 10 different scenarios being conducted. first, the criterion of financial security was increased by approximately 25%, then that criterion was decreased by approximately 25%, and the obtained results are presented in figure 3. the results of the sensitivity analysis revealed that a change in the first criterion has no significant influence on the importance of the alternatives, indicating that the overall rank of the final outcome remained unchanged in comparison to the ranking presented in table 2. ijahp article: kaftandzieva, cvetkoska/ financial decision making among young adults: an ahp approach international journal of the analytic hierarchy process 471 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.864 figure 3 performance sensitivity of alternatives when financial security is increased by 25% (left) and decreased by 25% (right) second, the criterion for personal characteristics was increased (left) and then decreased (right) by 25% (figure 4). consistent with previous findings, the 25% increase in this criterion appears to have a slight impact on the outcome with respect to alternatives 2 and 4. the 25% decrease in the criterion weight implies changes in the ranking of alternatives 2, 3 and 4, while the rest remain the same. figure 4 performance sensitivity of alternatives when the personal characteristics criterion is increased by 25% (left) and decreased by 25% (right) third, the criterion for profitability was increased by 25% (left) and decreased by 25% (right), and the model was tested to see if the outcome changed (figure 5). while the decrease in the criterion does not affect the priorities of the selective alternatives, the increase in the criterion changes the rankings for the alternative, except for the alternative bank deposits, which remains second. ijahp article: kaftandzieva, cvetkoska/ financial decision making among young adults: an ahp approach international journal of the analytic hierarchy process 472 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.864 figure 5 performance sensitivity of alternatives when profitability is increased by 25% (left) and decreased by 25% (right) the change in the criterion for readiness has no impact on the rank of the alternatives (figure 6). figure 6 performance sensitivity of alternatives when readiness is increased by 25% (left) and decreased by 25% (right) finally, the aforementioned scenarios were conducted for the criterion time horizon (figure 7) as well. the changes do not affect the priorities of the selected alternatives when the time horizon is decreased by 25%. when the other scenario analysis is conducted, the ranking of the alternatives is changed so that the alternative bank deposits have the first rank, followed by mutual funds, real estate and commodities, stock and life insurance. ijahp article: kaftandzieva, cvetkoska/ financial decision making among young adults: an ahp approach international journal of the analytic hierarchy process 473 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.864 figure 7 performance sensitivity of alternatives when the time horizon is increased by 25% (left) and decreased by 25% (right) based on the performed sensitivity analysis scenarios, we conclude that the 25% change (increase and decrease) in the weight of the criteria for financial security and readiness has not changed the rank order of the five alternatives. additionally, the decrease of 25% of the weight of the criteria for profitability and time horizon has not affected the rank of the alternatives either. in eight of ten scenarios, the alternative real estate commodities remains at the top, except when the profitability criterion is increased by 25%, in which case the mutual fund alternative is on the top; when the same scenario is repeated for the time horizon criterion, the bank deposits alternative is on the top. 5. conclusion in the current dynamic and turbulent environment, decisions are made under uncertainty and risk and people do not have all the necessary information and are not able to make a fully informed decision. research and experience both indicate that investing in risky assets and diversifying the portfolio is a result of behavioral determinants such as risk attitude and emotions, while human factors, which introduce an additional layer of complexity into the risk process, are a very important part of financial decision-making. the theory and practice of finance have evolved rapidly over the past couple of decades, thus creating a number of new opportunities and challenges for designing and implementing analytical tools for supporting financial decisions. ever since their introduction, multi-criteria decision-making methods have been largely used among scholars and are currently experiencing the peak of their popularity, which leads to further development and sophistication. however, there is still room for improvement and development, especially in the area of finance. the decision-making process in the area of finance is considered to be a cognitive process since the investors have to make a decision based on the various available alternatives. investors usually have different risk attitudes, preferences, and perceptions of value as well as information asymmetries, and often behave irrationally and base their decisions on intuition. irrespective of the goals, dealing with financial decisions in a ijahp article: kaftandzieva, cvetkoska/ financial decision making among young adults: an ahp approach international journal of the analytic hierarchy process 474 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.864 multidimensional setting can result in more informed outcomes. therefore, the success in applying the model depends primarily on the researcher's ability to evaluate the alternatives while taking into account a number of criteria. that being the case, the ahp is considered one of the most prominent methods that enables evaluation of both quantitative and qualitative criteria on the same preference scale and has proved to be a relevant and useful approach in financial decision-making. the sample in this research is comprised of the young adult population from the republic of north macedonia, who, because of the aging population and decrease in birth rates, is going to be the conduit of social and economic change, with further implications for the economy and financial system as its integral part. undoubtedly, everyone is aware that a certain degree of risk is an integral component in every segment of life and work, regardless of the type of decision. hence, each individual is risk-tolerant, but to some extent, proactive. in order to examine the variety of factors that affect personal decisionmaking, we developed an ahp model to synthesize the complex decision-making process in the area of personal finance. it consists of 5 criteria with their respective sub-criteria, and 5 alternatives, evaluated with respect to the goal of making a better and fact-based decision regarding personal finances. based on the obtained results, young adults prefer investment in real estate and commodities rather than the other alternatives. young adults today have great confidence in the banking sector. however, the relatively low yields offered by deposit products increase the possibility of alternative ways of saving, hence the third choice of investment in mutual funds. a small number of the respondents prefer investment in life insurance or on the stock market, partly because of their risk aversion, but also due to lack of information and financial knowledge. the results are in line with the macroeconomic trends in the macedonian economy. given unemployment and underemployment rates, young adults are less inclined to save or invest, especially in the more complex financial instruments. that being the case, this research is relevant to the extent that it brings about a better understanding of how individuals make financial decisions, which may allow financial institutions and governments to design programs in order to promote financial inclusion, improve financial literacy, and encourage people to make better decisions, particularly in developing economies. moreover, unemployment and underemployment generate dissatisfaction and support the necessity for more aggressive youth employment policies composed of internship and training programs, qualification, re-training and profiling of the youth into sectors and occupations that require additional workforce. more measures like this are urgently needed because of the rising trend of highly-skilled brain drain that results in highly trained and qualified people emigrating from the country. only by implementing such programs and activities can economic development be accomplished and a democratic and inclusive society be achieved. future investigations are necessary to validate the kinds of conclusions that can be drawn from this study. future research could fruitfully explore this issue by comparing the ideal investment of the young adult population with an ideal investment proposed by financial experts and advisors. therefore, different perspectives and priorities will be obtained. deviations in the decisions of young adults from the best solution proposed by financial analysts can be identified, and accordingly, certain advice will be given in order to improve decision-making regarding personal finances. ijahp article: kaftandzieva, cvetkoska/ financial decision making among young adults: an ahp approach international journal of the analytic hierarchy process 475 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.864 references aczel, j., & saaty, t. l. (1983). procedures for synthesizing ratio judgment. journal of mathematical psychology, 27, 93-102. doi: https://doi.org/10.1016/00222496(83)90028-7 antony, a., & joseph, a. i. (2017). influence of behavioural factors affecting investment decision an ahp analysis. metamorphosis: a journal of management research, 16(2), 107-114. doi: https://doi.org/10.1177/0972622517738833 carlin, b. i., & robinson, d. t. (2012). financial education and timely decision support: lessons from junior achievement american economic review, 102(3), 305-308. doi: https://doi.org/10.1257/aer.102.3.305 chang, c.-w., wu, c.-r., lin, c.-t., & chen, h.-c. (2007). an application of ahp and sensitivity analysis for selecting the best slicing machine. computers & industrial engineering, 52(2), 296-307. doi: https://doi.org/10.1016/j.cie.2006.11.006 charness, g., & gneezy, u. (2010). portfolio choice and risk attitudes: an experiment. economic inquiry, 48(1), 133-146. doi: https://doi.org/10.1111/j.14657295.2009.00219.x de almeida-filho, a. t., de lima silva, d. f., & ferreira, l. (2020). financial modelling with multiple criteria decision making: a systematic literature review. journal of the operational research society, 1-19. doi: https://doi.org/10.1080/01605682.2020.1772021 escobar, a. l., lópez, r. r., guerrero, g. j. e., & cuadrado, s. e. (2020). design of strategies for the implementation and management of a complementary monetary system using the swot-ahp methodology. sustainability, 12(17), 1-23. doi: https://doi.org/10.3390/su12176849 eurostat (2020). youth unemployment rate by sex, age and country of birth. available at: https://appsso.eurostat.ec.europa.eu/nui/show.do?dataset=yth_empl_100&lang=en (accessed on: may 20, 2020). franek, j., & kashi, k. (2014). a review and critique of madm methods and applications in business and management. international journal of the analytic hierarchy process, 6(2), 180-201. doi: https://doi.org/10.13033/ijahp.v6i2.254 gawlik, r. (2019). enhancing the work-life balance through ahp modeling of early career decision-making. international journal of the analytic hierarchy process, 11(2), 181-194. doi: https://doi.org/10.13033/ijahp.v11i2.626 goyal, p., kumar, d., & kumar, v. (2020). application of multi-criteria decision analysis (mcda) in the area of sustainability: a literature review. international journal https://appsso.eurostat.ec.europa.eu/nui/show.do?dataset=yth_empl_100&lang=en ijahp article: kaftandzieva, cvetkoska/ financial decision making among young adults: an ahp approach international journal of the analytic hierarchy process 476 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.864 of the analytic hierarchy process, 12(3), 512-545. doi: https://doi.org/10.13033/ijahp.v12i3.720 kahneman, d., & riepe, m. (1998). aspects of investor psychology. the journal of portfolio management, 24(4), 52-65. doi: https://doi.org/10.3905/jpm.1998.409643 kahneman, d., & tversky, a. (1972). subjective probability: a judgment of representativeness. cognitive psychology, 3(3), 430-454. doi: https://doi.org/10.1016/0010-0285(72)90016-3 khan, a. u., & ali, y. (2020). analytical hierarchy process (ahp) and analytic network process methods and their applications: a twenty-year review from 2000 to 2019. international journal of the analytic hierarchy process, 12(3), 369-459. doi: https://doi.org/10.13033/ijahp.v12i3.822 lusardi, a. (2008). u.s. household savings behavior: the role of financial literacy, information and financial education programs. in foote, c., goette, l., & meier, s. (eds), policymaking insights from behavioral economics. federal reserve bank of boston, 109-149. doi: https://doi.org/10.3386/w13824 mardani, a., jusoh, a., md nor, k., khalifah, z., zakwan, n., & valipour, a. (2015). multiple criteria decision-making techniques and their applications a review of the literature from 2000 to 2014. economic research-ekonomska istraživanja, 28(1), 516571. doi: https://doi.org/10.1080/1331677x.2015.1075139 obeidat, m. s., qasim, t., & khanfar, a. (2018). implementing the ahp multi-criteria decision approach in buying an apartment in jordan. journal of property research, 35(1), 53-71. doi: https://doi.org/10.1080/09599916.2017.1413588 oecd (2020). financial literacy of adults in south east europe. available at: https://www.oecd.org/finance/financial-literacy-of-adults-in-south-east-europe.pdf (accessed on: may 18, 2020). oecd infe (2011). measuring financial literacy: core questionnaire. in measuring financial literacy: questionnaire and guidance notes for conducting an internationally comparable survey of financial literacy. paris: oecd. doi: https://doi.org/10.1787/5k9csfs90fr4-en pradhan, s., olfati, m. & patel, g. (2019). integrations and applications of analytic hierarchy process with data envelopment analysis a literature review. international journal of the analytic hierarchy process, 11(2), 228-268. doi: https://doi.org/10.13033/ijahp.v11i2.632 ryan, j. s. & ryan, c. (2015). managing your personal finances, 7 th edition. cengage learning. http://business.gwu.edu/faculty/cv/annamaria-lusardi.pdf http://business.gwu.edu/faculty/cv/annamaria-lusardi.pdf https://www.oecd.org/finance/financial-literacy-of-adults-in-south-east-europe.pdf ijahp article: kaftandzieva, cvetkoska/ financial decision making among young adults: an ahp approach international journal of the analytic hierarchy process 477 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.864 roy, p. k., & shaw, k. (2021). a credit scoring model for smes using ahp and topsis. international journal of finance & economics, 1-20. doi: https://doi.org/10.1002/ijfe.2425 saaty, t. l. (1980). the analytic hierarchy process: planning, priority setting, resource allocation. new york, ny: mcgraw-hill. saaty, t. l. (1986). axiomatic foundation of the analytic hierarchy process. management science, 32(7), 841-855. doi: https://doi.org/10.1287/mnsc.32.7.841 saaty, t. l. (2012). decision-making for leaders. the analytic hierarchy process for decisions in a complex world. pittsburgh, pa: rws publications. saaty, t.l., & kearns, k.p. (1991). analytical planning: the organization of systems. the analytic hierarchy process series, vol. iv. pittsburgh, pa: rws publications. saaty, t. l., & peniwati, k. (2007). group decision-making: drawing out and reconciling differences. pittsburgh, pa: rws publications. sato, y. (2009). how to measure human perception in survey questionnaires. international journal of analytic hierarchy process, 1(2), 64-82. doi: https://doi.org/10.13033/ijahp.v1i2.31 spectrum financial resources (2021). personal financial planning questionnaire, https://spectrum-cpa.com/upload/547518/documents/ 1925786cd12b8c94.pdf (accessed on: january 20, 2020) steuer, r. e., & na, p. (2003). multiple criteria decision-making combined with finance: a categorized bibliographic study. european journal of operational research, 150(3), 496-515. doi: https://doi.org/10.1016/s0377-2217(02)00774-9 vaidya, o. s., & kumar, s. (2006). analytic hierarchy process: an overview of applications. european journal of operational research, 169(1), 1-29. doi: https://doi.org/10.1016/j.ejor.2004.04.028 wu, w., kou, g., peng, y., & ergu, d. (2012). improved ahp group decision-making for investment strategy selection. technological and economic development of economy, 18(2), 299-316. doi: https://doi.org/10.3846/20294913.2012.680520 zopounidis, c., & doumpos, m. (2013). multicriteria decision systems for financial problems. top, 21(2), 241-261. doi: https://doi.org/10.1007/s11750-013-0279-7 zopounidis, c., doumpos, m., & niklis, d. (2018). financial decision support: an overview of developments and recent trends. euro journal on decision processes, 6(12), 63-76. doi: https://doi.org/10.1007/s40070-018-0078-3 zopounidis, c., galariotis, e., doumpos, m., sarri, s., & andriosopoulos, k. (2015). multiple criteria decision-aiding for finance: an updated bibliographic survey. european ijahp article: kaftandzieva, cvetkoska/ financial decision making among young adults: an ahp approach international journal of the analytic hierarchy process 478 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.864 journal of operational research, 247(2), 339-348. doi: https://doi.org/10.1016/j.ejor.2015.05.032 ijahp article: kaftandzieva, cvetkoska/ financial decision making among young adults: an ahp approach international journal of the analytic hierarchy process 479 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.864 appendix i first-stage, questionnaire 1 1. gender a. male b. female 2. age a. less than 25 years old b. from 26 to 30 years c. from 31 to 35 years 3. education a. high school b. bachelor of science/college/university c. master of science 4. are you familiar with the following financial products (their characteristics and conditions)? a savings account □ yes □ no bonds □ yes □ no stocks and shares □ yes □ no a pension fund □ yes □ no an investment fund □ yes □ no life insurance □ yes □ no real estate □ yes □ no commodities □ yes □ no futures □ yes □ no options □ yes □ no 5. have you recently held any of the following financial products? a savings account □ yes □ no bonds □ yes □ no stocks and shares □ yes □ no a pension fund □ yes □ no an investment fund □ yes □ no life insurance □ yes □ no real estate □ yes □ no commodities □ yes □ no futures □ yes □ no options □ yes □ no 6. please answer the following attitude and behaviour statements (multiple answers are possible) a. before i buy something, i carefully consider whether i can afford it. b. i tend to live for today and let tomorrow take care of itself. c. i find it more satisfying to spend money than to save it for the long term. d. i pay my bills on time. e. i am prepared to risk some of my own money when saving or making an investment. f. i keep a close personal watch on my financial affairs. g. i set long term financial goals and strive to achieve them. h. money is there to be spent. ijahp article: kaftandzieva, cvetkoska/ financial decision making among young adults: an ahp approach international journal of the analytic hierarchy process 480 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.864 7. which sources of information do you feel most influenced your decision about which one to take out? a. product-specific information obtained in a branch or from sales personnel b. the internet c. the financial pages of newspapers d. specialist magazines e. recommendations from an independent financial adviser or broker f. advice of friends/relatives g. television or radio programmes and/or adverts h. my own previous experience 8. please indicate the relative importance of each of the following personal objectives to you: a. saving money regularly b. making a major purchase (e.g., second home, car) c. taking a dream vacation d. develop or revise your investment strategy e. providing a more comfortable life f. making gifts to relatives g. providing for your family 9. please indicate the relative importance of each of the following investment objectives by using a scale of 1 (strongly disagree) to 5 (strongly agree). a. return: dividends or interest to spend and/or reinvest b. liquidity: the ability to quickly convert the investment into cash c. volatility: the level of risk associated with price changes d. simplicity: full understanding of a financial product e. availability: can be purchased or invested in easily f. stability: little or no danger of losing the investment g. information: valuable and on-time information 10. what time-frame do you prefer for your investment? a. short-term investment b. medium-term investment c. long-term investment 11. please indicate the relative importance of each of the following personal characteristics important for financial decision-making by using a scale of 1(strongly disagree) to 5 (strongly agree) a. income b. the ability to save c. financial priorities d. risk aversion e. financial education f. experience ijahp article: kaftandzieva, cvetkoska/ financial decision making among young adults: an ahp approach international journal of the analytic hierarchy process 481 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.864 appendix ii second-stage, ahp-based questionnaire please make the following pair-wise comparisons, by using the fundamental scale, from 1 to 9. 1. pick one of the alternatives (from the left or from the right side) and choose how much more you prefer the selected alternative, regarding the sub-criteria return? bank deposits life insurance 1 2 3 4 5 6 7 8 9 bank deposits mutual funds 1 2 3 4 5 6 7 8 9 bank deposits real estate and commodities 1 2 3 4 5 6 7 8 9 bank deposits stocks 1 2 3 4 5 6 7 8 9 life insurance mutual funds 1 2 3 4 5 6 7 8 9 life insurance real estate and commodities 1 2 3 4 5 6 7 8 9 life insurance stocks 1 2 3 4 5 6 7 8 9 mutual funds real estate and commodities 1 2 3 4 5 6 7 8 9 mutual funds stocks 1 2 3 4 5 6 7 8 9 real estate and commodities stocks 1 2 3 4 5 6 7 8 9 2. regarding profitability as one of the main criteria, pick the sub-criterion (left or right) that is more important to you when making financial decisions and choose how much more. ijahp article: kaftandzieva, cvetkoska/ financial decision making among young adults: an ahp approach international journal of the analytic hierarchy process 482 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.864 return liquidity 1 2 3 4 5 6 7 8 9 return volatility 1 2 3 4 5 6 7 8 9 liquidity volatility 1 2 3 4 5 6 7 8 9 3. having in mind the goal: how to invest/save the inherited 10.000 eur, pick the criteria that you find more important and indicate how much more. financial security personal characteristics 1 2 3 4 5 6 7 8 9 financial security profitability 1 2 3 4 5 6 7 8 9 financial security readiness 1 2 3 4 5 6 7 8 9 financial security time-horizon 1 2 3 4 5 6 7 8 9 personal characteristics profitability 1 2 3 4 5 6 7 8 9 personal characteristics readiness 1 2 3 4 5 6 7 8 9 personal characteristics time-horizon 1 2 3 4 5 6 7 8 9 profitability readiness 1 2 3 4 5 6 7 8 9 profitability time-horizon 1 2 3 4 5 6 7 8 9 readiness time-horizon 1 2 3 4 5 6 7 8 9 ijahp article: roux, makrigeorgis/an analytic hierarchy process application to oil sands environmental compliance risk management international journal of the analytic hierarchy process 92 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.304 an analytic hierarchy process application to oil sands environmental compliance risk management izak j. roux iii roux engineering alberta, canada e-mail:izak@rouxengineer.com christos makrigeorgis college of management and technology walden university minneapolis, mn, usa email: christos.makrigeorgis@waldenu.edu abstract in 2013, oil companies in alberta, canada invested $32 billion in new oil-sands projects. despite the size of this investment, there is a demonstrable deficiency in the uniformity and understanding of environmental legislation requirements that translate into increased project compliance risks. in this paper, we applied the analytic hierarchy process (ahp) to develop a priority list of environmental regulatory compliance risk criteria for oilsands projects. ahp belongs to the family of multicriteria decision-making (mcdm) techniques that utilizes a pairwise comparison matrix solicited from subject matter experts (smes) in the field as input. the overall methodology itself consisted of 4 phases: (1) identification of the initial list of n potential environmental compliance risk criteria and verification of these criteria via a pilot survey; (2) formation of a pairwise comparison survey in the form of an n(n-1)/2 comparison matrix based on the verified criteria; (3) administration of the pairwise comparison matrix to a sample of 16 industryspecific sme’s; and (4) the application of the ahp method using superdecisions as a tool on the collected sample to rank the identified risk criteria. our demonstrated results can potentially inform alberta oil sands industry leaders about the ranking and utility of specific compliance risks as understood by experts and enable a more focused environmental compliance action to help increase legislative and public trust. keywords: ahp; alberta; oil sands; environment; regulatory compliance; risk criteria 1. introduction in the last decade, there has been an increased investment in in-situ oil sands extraction in alberta, canada with more than $32 billion invested in 2013 in new oil-sands projects. recent oil price decreases have exerted pressures on oil sands projects, but the roughly $155 billion capital investments in the last 10 years or so in these projects makes it just too difficult to switch them off. many of these projects have only recently moved into ijahp article: roux, makrigeorgis/an analytic hierarchy process application to oil sands environmental compliance risk management international journal of the analytic hierarchy process 93 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.304 production, thus oil sands companies continue to pump oil to recover these investments by curtailing costs and production in anticipation of competitive pricing beyond 2016. despite current price pressures, canadian oil sands constitute by far the largest region of oil sands in the world. this region covers an area of 140,000 sq km, an area larger than england. in situ oil sands extraction is technically complex. for example, the extraction of oil deposits residing in deep sand deposits requires the application of underground steam in large quantities to increase the viscosity and pump the oil (bitumen) to a processing plant on the surface. the industry employs highly specialized engineers and scientists with expensive equipment resources. in addition and specific to alberta oil sands, project engineers and executives have to achieve required levels of environmental regulatory compliance and sustainability throughout a project’s lifecycle. the successful deployment of new projects depends heavily on past regulatory compliance and risk management successes (rasmussen, 2009). in general, there are hefty public demands for environmental oversight for alberta’s oil sands developments. for example, the environmental compliance risk for oil companies may be higher in alberta than in the gulf of mexico according to a ceres report, a nonprofit environmental organization in the us (nicholls, 2010). other researchers such as briggs, tolliver, and szmerekovsky (2012) reviewed the supply chain risks in a related industry, the upstream crude oil industry. these authors identified environmental and regulatory compliance as one of the six high-risk criteria. during a project’s lifecycle, environmental requirements may change significantly with new regulatory requirements added after project definition which pose a potential risk to the sustainability of a project. the in-situ oil plant may not be able to meet new regulatory compliance within an economic framework. for example, pollard (2009) analyzed environmental risk management and concluded management should pay attention to changes in environmental regulations and possess a sound understanding of environmental risk. resolving environmental problems and reducing risks as well as liability are all positive reasons for implementing a risk management system even before the start of construction (jafari, khorasani, & danehkar, 2010). in summary, the identification and prioritization of risks for in-situ oil sands projects requires a more focused, justified, and rigorous approach. in this paper, we expose the core environmental compliance criteria essential to environmental risk management of insitu oil sands projects. the work is significant from two points of view. first, the use of a systematic approach rather than anecdotal evidence was applied to identify the importance of compliance risk criteria using a cross-sectional sample of smes deeply rooted in the alberta oil sands industry. and second, the exposure of such ranked compliance criteria to the general public increases the level of public trust and the view that the industry is indeed cognizant about such priorities and the need to manage associated risk. ijahp article: roux, makrigeorgis/an analytic hierarchy process application to oil sands environmental compliance risk management international journal of the analytic hierarchy process 94 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.304 2. literature review 2.1 background to the oil sands the business environment has changed in the last decade. shareholders expect better financial performance from companies while adhering to an environmental management plan (flammer, 2012). as global environmental requirements for oil sands increase and as the technology for extracting the oil changes, project engineers and company executives need to understand and support the environmental functions. such an understanding helps ensure that alberta oil sands projects will be compliant with the provincial government regulations. the biggest concentration of oil sand deposits in the world is situated in canada. there is an equivalent of 950 to 1,600 billion barrels of oil in these oil sands of which about 175 billion barrels of oil are recoverable (owen, inderwildi, & king, 2010). great canadian oil sands ltd (later suncor) began production of the oil sands north of fort mcmurray, alberta, canada in 1967 (george, 2012). the oil sands are uncovered in open pit mines; however, the oil sands (80%) in northeastern alberta are at depths of more than 150m, too deep for open pit mining and suitable for in situ extraction (george, 2012). in-situ oil sands extraction is technically complex and involves highly specialized individuals, expensive equipment resources, and registered patents according to the canadian association of petroleum producers (capp, 2010). the in-situ process used by most companies is the steam assisted gravity drainage (sagd) process. butler was the first to introduce steam as a method to move bitumen to a production well (mohebati, maini, & harding, 2010). butler’s steam process developed into the modern sagd process (coskuner, 2009). the main advantages of the sagd process are the use of gravity to drain the oil, lower energy consumption, and relative high recovery efficiency (between 70-75%) (capp, 2010; mohebati et al., 2010). in-situ plants have a small land footprint similar to the footprint of conventional oil well operations, and in-situ plants have no need for tailings ponds. for in-situ projects, the steam-to-oil ratio (sor) is a measure of the energy efficiency of bitumen recovery. according to the ercb report st98-2011, the alberta oil sands industry produced 1.613 million barrels per day (mb/d) of crude bitumen in 2010 (ercb, 2011). the oil sands crude production represents 0.857 mb/d from surface mining and 0.756 mb/d from in situ projects (ercb, 2011). the total crude bitumen production from the alberta oil sands will more than double by 2020, an increase from 1.6 mb/d in 2010 to about 3.3 mb/d (ercb, 2011). the production from in situ projects will exceed production from mining projects within the next few years. doubling production of the oil sands will increase the need for better environmental risk mitigation planning to avoid the potential negative environmental impact. an oil sands project can end in financial disaster if the local jurisdiction deems the project as non-compliant with local regulations. a non-compliant project will not receive a license to start producing. an oil company may spend $500 million on a pilot in-situ project before the start of production and up to $1.5 billion on a commercial project (government of alberta, 2010). coupled with these high capital investments are environmental risks due to the fact that alberta’s oil sands are situated in an environmentally sensitive part of the province. in 2008, ducks (1,600) drowned in the ijahp article: roux, makrigeorgis/an analytic hierarchy process application to oil sands environmental compliance risk management international journal of the analytic hierarchy process 95 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.304 syncrude aurora tailings pond and syncrude, a leading alberta oil sands producing company, paid $3m in penalties, and much more in loss of image since the unfortunate incident gave the company a reputation for producing dirty oil and reminded all oil sands executives that the preservation of the environment is a serious issue (stewart, archer, & trynacity, 2010; stewart et al., 2010). as such, managers of future oil sands projects must consider the environment during project execution. 2.2 environmental compliance regulations the government of alberta measures the environmental impact of the oil sands in terms of several factors or indicators which are as follows (government of alberta, 2009): 1. land use or footprint – the area of boreal forest cleared for operations, all affected land shall be reclaimed and biodiversity shall be maintained (regulated by alberta environment). 2. air pollution – greenhouse gas (ghg) emissions from the oil sands totaled 45 million tons in 2009 (mccarthy, 2010), and these ghg emissions represented 6% of canada’s ghg emissions and 0.1% of global emissions. the alberta provincial government introduced, in 2007, a 12% mandatory reduction in ghg emissions for all large industrial facilities, including oil sands (alberta environment, 2008). 3. water use – the alberta government regulates all fresh water resources in the province. in 2009, irrigation and agriculture represented 44% of the total provincial water usage allocations; the provincial regulator allocated 7% of the water usage to oil sands industry. in-situ projects require about 0.5 barrels of fresh water per barrel of oil, and 80-90% of water used in sagd is recycled (capp, 2010). 4. water quality – the alberta environment regulator regulates the release and the quality of all water. all water must conform to the water quality standards of alberta environment (hurley, sadiq, & mazumder, 2012). 5. people – the oil sands developments have an effect on the communities. consulting with local stakeholders and communities is a regulatory requirement (government of alberta, 2009). project engineers do consider these five environmental indicators because the alberta government constantly monitors these same environmental indicators ensuring proof of compliance by the project owner throughout the project construction and production phases. 2.3 risk management arimura, hibiki, and katayama (2008) explained that an environmental risk management system such as international environmental risk management system (iso 14000), helped companies identify and resolve environmental impact in areas such as natural resources and solid waste generation. edwards and darnall (2010) identified that the implementation of an environmental risk management plan improves the possibility of an organization complying with environmental regulations. therefore, the success of the project and the future sustainability of an oil sands project will depend on the environmental risk management strategy followed by decision makers. ijahp article: roux, makrigeorgis/an analytic hierarchy process application to oil sands environmental compliance risk management international journal of the analytic hierarchy process 96 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.304 2.4 risk management of in situ oil sands projects the bp deepwater horizon oil spill in the gulf of mexico that happened in april 2010 demonstrated how a company with a long-term environmental strategy could still have an environmental incident that was catastrophic to the company’s image (cherry & sneirson, 2011). oil company executives need to ensure an environmental risk management process is in place, and that the company complies with the environmental, health and safety regulations. regulatory compliance risk management is the process of establishing mitigation actions that will help to: (a) protect the company image, (b) protect the company wealth, (c) execute successful oil sands projects, (d) ensure a safe workplace, (e) protect the environment, and (f) maintain close relationships with local communities. in the case of new oil sands projects, engineers will have to work towards regulatory compliance. regulatory compliance is possible when project engineers manage the identified risk criteria. these risk criteria include (a) operational risk, (b) compliance and legal risk, (c) reputation risk, (d) social risk, (e) environment, health, and safety or ehs risk, (f) geologic risk, (g) political risk, (h) financial risk, and (i) economic risk (andersen & mostue, 2012; wagner & armstrong, 2010). briggs et al. (2012) reviewed risk criteria in the oil industry’s supply chain, and they recognized social risk, economic risk, financial risk, technological risk, and environmental and regulatory compliance risk as high-level risk criteria. these risks focus on the business, the public, and the environment. another oil industry study by enyinda, briggs, obuah, and mbah (2012) identified the primary risk criteria as geological and production risks, environmental and regulatory risk, transportation risk, oil availability risk, geopolitical risk, and reputation risk. 2.5 influence of alberta environmental policies on risk management the influence that regulations and government actions have on the risk level of new projects may determine the final compliance of the project. compliance requirements and their influence on projects could have a significant influence on how executives see the future compliance issues and corporate social responsibility as measures of project success (wagner & armstrong, 2010). el-fadel, abi-esber, and ayash (2009) examined the use of regulatory and compliancebased modeling as applied to air quality assessment. el-fadel et al. (2009) stated that environmental regulatory compliance based on science works to a point, and then it becomes a political decision-making situation. the political decision-making is outside the influence sphere of the project engineer or oil company executive. however, it is the regulator and politicians that need to take action based on this scientific evidence. 2.6 the need to apply an mcdm technique the environmental policy uncertainty in the oil and gas industry in alberta influences the decision-making in this industry and the framework most suitable for decision-making. in light of this paper’s focus on identifying and ranking the environmental risk criteria and the fact that such multiple criteria cannot be defined as a monetary value or added together, attention was turned to multi-criteria decision making (mcdm) models. rasmussen (2009) was a key reference who investigated the decision-making process under risk and compliance and proposed a holistic process to decision-making. an ijahp article: roux, makrigeorgis/an analytic hierarchy process application to oil sands environmental compliance risk management international journal of the analytic hierarchy process 97 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.304 mcdm process that considers or manages the human aspect of decision-making is beneficial. an understanding of the openness and endless approach that is qualitative research may also help to get a better understanding of decision-making. a number of mcdm techniques could potentially be applied to rank the environmental risk criteria associated with oil sands extraction projects. the premise was based on a study by huang, keisler, and linkov (2011) who reviewed more than 300 environmental project papers and concluded that there is an increase in the application of mcdm. in addition, huang et al. determined that the methods used the most in environmental analysis included ahp (saaty, 2006), maut (beccacece & borgonovo, 2011), and outranking (jajimoggala & karri 2013). similarly, geng and wardlaw (2013) identified the mcdm methods most used in environmental management included compromise programming, goal programming, as well as ahp. decision-making is a broad subject that influences the project, project executives, project engineers, and the organization (akdere, 2011). it is essential to identify typical decisionmaking models used for multicriteria decisions-making (mcdm particularly for environmental decision-making). a review of the literature regarding the decisionmaking process used in other industries where most of these risks occur found that the ahp is often used as an mcdm. ahp is an mcdm methodology suitable for complex systems that allow for the interpretation of both qualitative and quantitative aspects of decisions (saaty & sagir, 2009). saaty (2006) has used ahp since its introduction in 1980 for a large number of mcdm projects within various industries, and it has been applied in areas such as setting priorities, risk management, quality management, project management, and strategic decision-making. lastly, it is well-known that ahp is preferred over traditional methods such as a multiple-choice survey as it provides superior measurement or discrimination of a participant’s perceptions (estévez, walshe, & burgman, 2013). therefore, ahp is a preferred mcdm framework by many practitioners (aydin & arslan; 2010; podvezko, mitkus & trinkūnienė, 2010). at the end of the process, ahp not only allows the researcher to manage different types of data but also evaluate the consistency of the inputs from decision makers (stoklasa, jandová, & talasová, 2013). following this overall guidance, we also opted to apply ahp. 3. methodology the applied research methodology is in compliance with the following principles: (a) a valid research purpose, (b) an appropriate methodology for data collection and analysis, (c) a manageable research scope and effort, and (d) the researcher’s perception of the real-word problem to be researched (cooper & schindler, 2013). the purpose was to (a) identify the environmental compliance risk criteria, and (b) assess the contributions these risk criteria make to regulatory compliance of in-situ oil sands projects in alberta. the adopted data collection and analysis methodology is based on ahp and consisted of a number of steps beginning with the identification of the initial list of n potential environmental compliance risk criteria using the alberta energy regulator (aer) database, verification of this list of n criteria with five smes, the formation of a pairwise comparison survey in the form of an n(n-1)/2 comparison ijahp article: roux, makrigeorgis/an analytic hierarchy process application to oil sands environmental compliance risk management international journal of the analytic hierarchy process 98 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.304 matrix, the administration of the pairwise comparison matrix to a sample of 16 industryspecific subject matter experts (smes) to solicit their individual pairwise priorities among the identified n risk criteria and the application of the ahp method using superdecisions as a tool on the collected sample to rank each of the n risk criteria. a summary of these steps is shown in figure 1. figure 1. execution steps for specialized survey 3.1 identification of potential risk criteria the study began with the identification of a list of environmental compliance risk criteria. these criteria were identified from past literature and a key database called the aer archival database encompassing all reports filed for regulatory compliance of oil sands ijahp article: roux, makrigeorgis/an analytic hierarchy process application to oil sands environmental compliance risk management international journal of the analytic hierarchy process 99 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.304 projects (ercb, 2008). these reports were self-disclosure reports and were a result of past project engineerand project manager-identified regulatory compliance issues as reported to the aer. 3.2 formation of the pairwise comparison survey the formation of the pairwise comparison survey required two steps. first, a pilot survey was formed utilizing the list of the identified n potential risk criteria. this survey was administered to five smes before adopting the list for forming the final pairwise comparison survey. it is important to note that the five smes who were consulted in the pilot survey step were members of capp, a recognized association of petroleum producers. in addition, the primary author’s personal engineering work in the oil sands field was put to use in encapsulating the relevant regulatory risks and in soliciting the sample through business contacts. the pilot survey administered to the set of 5 smes was analyzed to determine the final list of n risk criteria. once this list of n criteria was finalized, these n criteria were converted to an n(n-1)/2 pairwise comparison matrix or survey and administered at a later point in time to a separate set of 16 sme’s. the sample of smes involved in the research is drawn from the population of risk management smes involved in alberta in-situ oil sands projects. the population was selected because of the high concentration of in-situ oil sands projects in the province. in 2010 there were more than 87 oil sands projects in alberta in various stages of execution (government of alberta, 2010). a targeted or purposeful sample of smes was sought to ensure that these smes took an active involvement in the decision-making process and that they were directly involved in risk management or environment, health, and safety (eh&s) management. eligibility depended on the participant’s organizational function and subject knowledge. the knowledge of individual group members in the ahp decision group is a main concern (saaty, 2006). additionally, individual decision makers with subject expertise and experience can provide valid inputs and be representative of the subject matter (huang, chen, & chang, 2009). therefore, the expert knowledge and experience of the smes recruited contributed to the success of the survey. the sample size of 16 smes was based on firestone (2006) who recommended 15 smes in a decision-making panel, goepel (2011) who recommended using a group of at least five smes for ahp, and the key papers by goepel (2011) as well as saaty (2006) who both concluded that the background and experience of the sme’s were more valuable to the success of ahp than the number of sme’s in the decision-making panel. in this research, the guidelines from firestone and goepel were followed in order to recruit a purposeful sample of 15 smes from the frame. the criteria resulting from execution of the pilot survey are shown by level in figure 2 below. the initial list of criteria used in the pilot survey were close to the final list shown in figure 2 and only included an item called “human compliance” that was deemed not important by the pilot smes. ijahp article: roux, makrigeorgis/an analytic hierarchy process application to oil sands environmental compliance risk management international journal of the analytic hierarchy process 100 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.304 figure 2. risk management hierarchy validated in a pilot study given the resulting set of criteria and their levels, a pairwise comparison survey was defined capturing the risk criteria as a matrix using an excel spreadsheet. a portion of this matrix is shown in table 1 below. ijahp article: roux, makrigeorgis/an analytic hierarchy process application to oil sands environmental compliance risk management international journal of the analytic hierarchy process 101 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.304 table 1 a sample of the pairwise comparison survey matrix 2.1. natural environmental risk factors 2.1.1 air pollution 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 2.1.2 ghg emissions 2.1.1 air pollution 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 2.1.3 land use/foot print 2.1.1 air pollution 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 2.1.4 water use 2.1.1 air pollution 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 2.1.5 water quality 2.1.2 ghg emissions 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 2.1.3 land use/foot print 2.1.2 ghg emissions 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 2.1.4 water use 2.1.2 ghg emissions 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 2.1.5 water quality 2.1.3 land use/foot print 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 2.1.4 water use 2.1.3 land use/foot print 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 2.1.5 water quality 2.1.4 water use 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 2.1.5 water quality next, we included the pairwise comparison survey in an excel spreadsheet for portability and distributability and administered the pairwise comparison survey to 16 smes comprised of senior executives and project engineers of in-situ oil sands companies in alberta, canada. one specialized survey was collected per oil sands company, with the exception of three companies from which two smes participated to check for inter-rater reliability. the purposive sample of project engineers and company executives provided the best possible data since these engineers and executives were directly responsible for the risk management process during sagd project planning and implementation. 3.3 application of the ahp method the pairwise comparison matrix scores from each participant’s excel spreadsheet were then imported in the superdecisions ahp software and ahp was executed to determine the ranking of the risk criteria at each level as detailed in the steps of figure 3 (saaty, 2003). following the ahp run, the ranking of the risk criteria was triangulated with a small group of five oil executives and project engineers. they indicated their acceptance of the final rankings assigned to the risk criteria. ijahp article: roux, makrigeorgis/an analytic hierarchy process application to oil sands environmental compliance risk management international journal of the analytic hierarchy process 102 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.304 figure 3. ahp analysis steps the ahp has the advantage of permitting a hierarchical structure of the criteria and subcriteria, which provides users with a better focus on specific criteria when doing a pairwise comparison. figure 4 shows the hierarchy as it was inserted in superdecisions in order to perform the ahp analyses. the hierarchy contained four levels: the goal (level 0), the risk level (level 1) supporting the goal, level 2 defining or supporting the risk level, and level 3 defining sub-criteria supporting level 2. these four levels were entered in superdecisions. ijahp article: roux, makrigeorgis/an analytic hierarchy process application to oil sands environmental compliance risk management international journal of the analytic hierarchy process 103 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.304 figure 4. risk management hierarchy as shown in superdecisions 4. results following the ahp analysis, the risk rankings were aggregated for n=16 surveys to obtain the final rankings for the compliance risks. for the aggregation of ahp rankings, the ahp practitioner may use either the arithmetic mean or the geometric mean. saaty (2006) proposed the geometric mean rather than the arithmetic mean as the proper way to aggregate judgments by more than one judge. aczel and saaty (as cited by saaty, 2006) proved saaty’s theorem using the geometric mean if all participants have equal importance in the aggregated result. for this study, all smes had equal importance. table 2 shows the ranking of all risks based on the surveys of 16 smes and aggregated using the geometric mean. table 3 shows the overall ranking of all risks. ijahp article: roux, makrigeorgis/an analytic hierarchy process application to oil sands environmental compliance risk management international journal of the analytic hierarchy process 104 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.304 table 2 aggregated risk ranking per level using geometric mean ranking criteria / subcriteria geometric mean weight (%) weight normalized (%) 1 5 regulatory compliance 19.28 22.78 2 4 reputation/post incident 15.15 17.90 3 2 environmental compliance 14.35 16.95 4 1 economics and financial 13.01 15.37 5 7 technological 11.92 14.08 6 6 social compliance 6.66 7.87 7 3 political 4.27 5.04 level 1 project risks total 84.64 100.00 1 12 plant construction cost 28.07 36.10 2 14 project financing 24.47 31.47 3 11 cost of exploration 16.76 21.56 4 13 price of land 8.45 10.87 level 2 economics & financial total 77.75 100.00 1 22 human environmental compliance 48.11 55.64 2 21 natural environmental compliance 38.35 44.36 level 2 environmental compliance total 86.46 100.00 1 31agreements with first nations 55.87 57.75 2 33 involve local municipality 26.84 27.74 3 32 presentations to provincial mlas 14.04 14.51 level 2 political total 96.75 100.00 1 53 comply with the safety codes act 23.07 26.71 2 54 implement a pressure equipment integrity program 20.63 23.89 3 52 comply with the oil sands act 15.93 18.44 4 55 implement a pipeline integrity management program 13.40 15.51 5 51 comply with the pipeline act 13.34 15.45 level 2 regulatory compliance total 86.37 100.00 1 62 employ local people 71.71 74.43 2 61 implement social/community programs 24.64 25.57 level 2 social compliance total 96.35 100.00 ijahp article: roux, makrigeorgis/an analytic hierarchy process application to oil sands environmental compliance risk management international journal of the analytic hierarchy process 105 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.304 ranking criteria / subcriteria geometric mean weight (%) weight normalized (%) 1 215 water quality 27.85 32.37 2 214 water use 23.51 27.32 3 211 air pollution 15.02 17.46 4 212 ghg emissions 10.16 11.81 5 213 land use/footprint 9.50 11.04 level 3 natural environmental compliance total 86.04 100.00 1 221 community disturbance 59.05 65.34 2 222 socioeconomic changes 31.33 34.66 level 3 human environmental compliance total 90.38 100.00 ijahp article: roux, makrigeorgis/an analytic hierarchy process application to oil sands environmental compliance risk management international journal of the analytic hierarchy process 106 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.304 table 3 overall ranking of risks, highest to lowest ranking criteria/subcriteria geometric mean weight (%) weight normalized (%) 1 5 regulatory compliance 9.17 12.37 2 2 environmental risks 7.78 10.50 3 1 economics & financial 7.25 9.78 4 4 reputation/post incident 6.84 9.23 5 7 technological 5.52 7.45 6 22 human environmental compliance 3.45 4.66 7 21 natural environmental compliance 3.23 4.36 8 6 social compliance 3.11 4.20 9 3 political 2.51 3.39 10 53 comply to the safety codes act 2.46 3.32 11 62 employ local people 2.28 3.08 12 12 plant construction cost 2.08 2.81 13 14 project financing 1.86 2.51 14 54 implement a pressure equipment integrity program 1.81 2.44 15 221 community disturbance 1.59 2.15 16 31agreements with first nations 1.43 1.93 17 52 comply to the oil sands act 1.38 1.86 18 222 socioeconomic changes 1.38 1.86 19 51 comply to the pipeline act 1.35 1.82 20 11 cost of exploration 1.32 1.78 21 55 implement a pipeline integrity management program 1.16 1.57 22 215 water quality 1.00 1.35 23 61 implement social/community programs 0.74 1.00 24 33 involve local municipality 0.63 0.86 25 214 water use 0.60 0.81 26 13 price of land 0.58 0.79 27 211 air pollution 0.49 0.67 28 212 ghg emissions 0.38 0.52 29 32 presentations to provincial mla's 0.36 0.49 30 213 land use/footprint 0.32 0.44 74.14 100.00 ijahp article: roux, makrigeorgis/an analytic hierarchy process application to oil sands environmental compliance risk management international journal of the analytic hierarchy process 107 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.304 5. conclusions in summary, the ahp allowed for the definition of a complex problem in terms of criteria and subcriteria, using a top-down breakdown that allows for a bottom-up generation of the relative importance using a pairwise comparison. the ranking of risk criteria applicable to in situ oil sands projects was, in order from highest to lowest, (a) regulatory compliance, (b) company reputations/post incident, (c) environmental compliance, and (e) economics and financials. there was a clear indication of a longterm sustainable approach to in situ oil sands projects with an understanding of the relationship between regulatory compliance, environment, economics, and the public (company reputation/post incident). the identified environmental compliance risks in the oil sands industry in alberta could be applicable to other provinces in canada and the usa. however, applying the environmental compliance risk and mitigation strategy rankings to other provinces and states might need further research that considers unique provincial or usa regulatory requirements. a summary of conclusions of this research is as follows:  risk management of environmental regulatory compliance of oil sands projects should be a primary strategy.  ahp was an ideal mcdm method applied to the ranking of risks in the oil sands industry.  understanding and ranking risks should improve environmental compliance.  regulatory compliance would lead to an improved socioeconomic environment.  a sustainable environment in the oil sands should increase public confidence.  environmental compliance of oil sands projects should be a global business requirement. with future research based on the outcome of this work, the hope is for project engineers and executives to manage their oil sands projects successfully. to be truly successful, oil sands projects should not only be economically successful but also successful in the management of environmental compliance risks. project engineers should implement risk mitigation strategies that would inspire public confidence in the sustainable future of oil sands extraction. the application of ahp shortened the time needed to identify and prioritize risks, and helped participants deeply involved in the oil sands industry avoid confusion in answering survey questions as explained by similar ahp applications (ishizaka, 2012; ishizaka & labib, 2011). it is for these same reasons that researchers and practitioners have applied ahp in such fields as environmental management, forest management, water resource management, energy management, and renewable energy planning. this paper ranked environmental compliance risks in the oil sands industry and showed that ahp is an effective method to gain sme consensus in terms of rankings in a practical setting. ijahp article: roux, makrigeorgis/an analytic hierarchy process application to oil sands environmental compliance risk management international journal of the analytic hierarchy process 108 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.304 references akdere, m. (2011). an analysis of decision-making process in organizations: implications for quality management and systematic practice. total quality management & business excellence, 22, 1317-1330. doi:10.1080/14783363.2011.625180 alberta environment. (2008). specified gas-reporting regulation: alberta environment report on 2007 greenhouse gas emission. retrieved from http://www.environment.alberta.ca andersen, s., & mostue, b. a. (2012). risk analysis and risk management approaches applied to the petroleum industry and their applicability to io concepts. safety science, 50, 2010-2019. doi:10.1016/j.ssci.2011.07.016 arimura, t. h., hibiki, a., & katayama, h. (2008). is a voluntary approach an effective environmental policy instrument? a case for environmental management systems. journal of environmental economics and management, 55, 281-295. doi:10.1016/j.jeem.2007.09.002 aydin, o., & arslan, g. (2010). optimal hospital location with fuzzy ahp. the business review, cambridge, 15, 262-268. retrieved from http://www.jaabc.com/ beccacece, f., & borgonovo, e. (2011). functional anova, ultramodularity and monotonicity: applications in multiattribute utility theory. european journal of operational research, 210, 326-335. doi:10.1016/j.ejor.2010.08.032 briggs, c. a., tolliver, d., & szmerekovsky, j. (2012). managing and mitigating the upstream petroleum industry supply chain risks: leveraging analytic hierarchy process. international journal of business & economics perspectives, 7, 1-20. retrieved from http://www.iabpad.com/ capp, (2010). responsible canadian energy oil sands progress report, for the year ended december 31, 2009. canadian association of petroleum producers. retrieved from http://www.capp.ca cherry, m. a., & sneirson, j. f. (2011). beyond profit: rethinking corporate social responsibility and green washing after the bp oil disaster. tulane law review, 85, 9831038. doi:10.2139/ssrn.1670149 cooper, d. r., & schindler, p.s. (2013). business research methods (12th ed.). newyork, ny: mcgraw-hill irwin. coskuner, g. (2009). a new process combining cyclic steam stimulation and steamassisted gravity drainage: hybrid sagd. journal of canadian petroleum technology, 48, 8-13. retrieved from http://www.onepetro.org edwards, d., & darnall, n. (2010). averting environmental justice claims? the role of environmental management systems. public administration review, 70, 422-433. ijahp article: roux, makrigeorgis/an analytic hierarchy process application to oil sands environmental compliance risk management international journal of the analytic hierarchy process 109 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.304 doi:10.1111/j.1540-6210.2010.02156.x el-fadel, m., abi-esber, l., & ayash, t. (2009). managing emissions from highly industrialized areas: regulatory compliance under uncertainty. journal atmospheric environment, 43, 5015-5026. doi:10.1016/j.atmosenv.2009.06.056 enyinda, c. i., briggs, c., obuah, e., & mbah, c. (2012). petroleum supply chain risk analysis in a multinational oil firm in nigeria. journal of marketing development & competitiveness, 5(7), 37-44. retrieved from business source complete database. (accession no. 71666007) ercb (2008). ercb investigation report; meg energy corp. steam pipeline failure, license no. p 46441, line no. 001, may 5, 2007. retrieved from http://www.ercb.ca/docs/products ercb (2011). ercb report st98-2011alberta’s energy reserves 2011 and supply/demand outlook—economics. retrieved from http://www.ercb.ca/docs/products/sts/ estévez, r. a., walshe, t., & burgman, m. a. (2013). capturing social impacts for decision‐making: a multicriteria decision analysis perspective. diversity and distributions, 19, 608-616. doi:10.1111/ddi.12058 firestone, j. m. (2006). risk intelligence metrics: an adaptive metrics center industry report. executive information systems, inc. flammer, c. (2012). corporate social responsibility and shareholder reaction: the environmental awareness of investors. academy of management journal, 55. doi:10.5465/amj.2011.0744 acad manage j july 20, 2012 amj.2011.0744 geng, g., & wardlaw, r. (2013). application of multi-criterion decision making analysis to integrated water resources management. water resources management, 27, 31913207. doi:10.1007/s11269-013-0343-y george, r. (2012). sun rise, suncor, the oil sands and the future of energy. toronto, canada: harpercollins publishers ltd. goepel, k. d. (2011). ahp-anp practical applications with pros and cons. retrieved from http://www.bpmsg.com/ahp-anp-practical-application-slides/ government of alberta, (2009). energy strategy. retrieved from http://www.alberta.ca government of alberta, (2010). alberta’s oil sands. retrieved from http://www.oilsands.alberta.ca huang, y.-f., chen, c.-j., & chang, h.-h. (2009). a multiple criteria evaluation of creative industries for the cultural creativity center in taiwan. international journal of entrepreneurial behaviour & research, 15, 473-496. doi:10.1108/13552550910983031 ijahp article: roux, makrigeorgis/an analytic hierarchy process application to oil sands environmental compliance risk management international journal of the analytic hierarchy process 110 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.304 huang, i.b., keisler, j., & linkov, i. (2011). multi-criteria decision analysis in environmental sciences: ten years of application and trends. science of the total environment, 409, 3578-3594. doi:10.1016/j.scitotenv.2011.06.022 hurley, t., sadiq, r., & mazumder, a. (2012). adaptation and evaluation of the canadian council of ministers of the environment water quality index (ccme wqi) for use as an effective tool to characterize drinking source water quality. water research, 46, 3544-3552. doi:10.1016/j.watres.2012.03.061 ishizaka, a. (2012). clusters and pivots for evaluating a large number of alternatives in ahp. pesquisa operacional, 32, 87-102. doi:10.1590/s0101-74382012005000002 ishizaka, a., & labib, a. (2011). review of the main developments in the analytic hierarchy process. expert systems with applications, 38, 14336-14345. doi:10.1016/j.eswa.2011.04.143 jafari, j., khorasani, n., & danehkar, a. (2010). using environmental sensitivity index (esi) to assess and manage environmental risks of pipelines in gis environment: a case study of a near coastline and fragile ecosystem located pipeline. world academy of science, engineering and technology, 44, 1101-1111. retrieved from http://waset.org/journals/waset/ jajimoggala, s., & karri, r. r. (2013). decision making model for material selection using a hybrid mcdm technique. international journal of applied decision sciences, 6, 144-159. doi:10.1504/ijads.2013.053273 mccarthy, s. (2010, dec, 21). government’s vow to overhaul environmental monitoring of oil sands development. the globe and mail. retrieved from http://www.theglobeandmail.com mohebati, m. h., maini, b.b., & harding, t.g. (2010, october). optimization of hydrocarbon additives with steam in sagd for three major canadian oil sands deposits. paper presented at unconventional resources and international petroleum conference, society of petroleum engineers calgary, alberta, canada. doi:10.2118/138151-ms nicholls, t. (2010, august). the oil sands' environmental tests. petroleum economist. retrieved from http://www.petroleum-economist.com owen, n. a., inderwildi, o. r., & king, d. a. (2010). the status of conventional world oil reserves—hype or cause for concern? energy policy, 38, 4743-4749. doi:10.1016/j.enpol.2010.02.026 podvezko, v., mitkus, s., & trinkūnienė, e. (2010). complex evaluation of contracts for construction. journal of civil engineering & management, 16, 287-297. doi:10.3846/jcem.2010.33 pollard, s. t. (2009). contributions to resource and environmental risk management. (doctoral dissertation). cranfield university, uk. retrieved from http://dspace.lib.cranfield.ac.uk/handle/1826/4563 ijahp article: roux, makrigeorgis/an analytic hierarchy process application to oil sands environmental compliance risk management international journal of the analytic hierarchy process 111 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.304 rasmussen, m. (2009). foundations of grc: streamlining compliance. corporate integrity, llc. retrieved from http://www.sap.com/sapbusinessobjects/grc saaty, r. w. (2003). decision making in complex environments. the analytical hierarchy process (ahp) for decision making and the analytical network process (anp) for decision making with dependence and feedback. pittsburgh, pa: superdecisions. saaty, t. l. (2006). fundamentals of decision making and priority theory with the analytical hierarchy process, vol. vi of the ahp series. pittsburgh, pa: rws publications. saaty, t. l., & sagir, m. (2009). extending the measurement of tangibles to intangibles. international journal of information technology & decision making, 8, 7-27. doi:10.1142/s0219622009003247 stewart, b., archer, j., & trynacity, k. (2010, june 25). syncrude guilty in alberta duck deaths. cbc news. retrieved from http://www.cbc.ca/news/canada/edmonton/story/2010/06/25/edmonton-syncrude-ducktrial. stoklasa, j., jandová, v., & talasová, j. (2013). weak consistency in saaty's ahp evaluating creative work outcomes of czech art colleges. neural network world, 23, 6177. retrieved from http://www.nnw.cz/ wagner, j., & armstrong, k. (2010). managing environmental and social risks in international oil and gas projects: perspectives on compliance. the journal of world energy law & business, 3, 140-165. doi:10.1093/jwelb/jwq002 http://search.proquest.com.ezp.waldenulibrary.org/indexingvolumeissuelinkhandler/27998/neural+network+world/02013y01y01$232013$3b++vol.+23+$281$29/23/1?accountid=14872 http://search.proquest.com.ezp.waldenulibrary.org/indexingvolumeissuelinkhandler/27998/neural+network+world/02013y01y01$232013$3b++vol.+23+$281$29/23/1?accountid=14872 ijahp news: ferretti/informs 2015 highlights international journal of the analytic hierarchy process 619 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.367 informs 2015 highlights valentina ferretti, department of management, london school of economics and political science the 2015 informs annual meeting took place in vibrant philadelphia. the conference program had a very broad scope and covered cutting-edge topics from data analytics to healthcare applications; from novel optimization methodologies to comprehensive analysis strategies; from powerful modeling and software tools to intriguing industrial applications. the exciting intellectual sphere of the conference offered many opportunities to meet old and new friends, network, and build and enhance careers. the ahp/anp community has again been very active presenting and discussing innovative theories and applications in the field. below are some key pictures from our community at work: figure 1. poster session, “analytic network process: assisting computers to think like humans?” by elena rokou ijahp news: ferretti/informs 2015 highlights international journal of the analytic hierarchy process 620 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.367 figure 2. participants from the session “theory and applications of the analytic network process” chaired by orrin cooper. from left to right: gabriela sava, orrin cooper, rachel chung and enrique mu. figure 3. enrique mu, valentina ferretti and yuji sato, who talked about eyewitness identification using the ahp, the integration between the anp and stakeholders’ analysis and consistency among pairwise comparisons, respectively. ijahp news: ferretti/informs 2015 highlights international journal of the analytic hierarchy process 621 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.367 check the website of the conference (http://meetings2.informs.org/wordpress/philadelphia/) for updates and inspiration about the most recent advances in the ahp/anp community. join us for the 2016 annual meeting which will take place in the music city nashville from november 13 16! http://meetings2.informs.org/wordpress/philadelphia/ prioritizing total quality management implementation criterion for engineering educational: an analytic hierarchy process analysis ijahp article: mehta, verma, seth/prioritizing total quality management implementation criterion for engineering education: an analytic hierarchy process analysis international journal of the analytic hierarchy process issn 1936-6744 35 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.197 prioritizing total quality management implementation criterion for engineering education: an analytic hierarchy process analysis namish mehta 1 department of mechanical engineering truba college of science and technology bhopal, india e-mail: mehtanamish@gmail.com dr. prakash verma department of industrial and production engineering jabalpur engineering college jabalpur, india e-mail: pvjbp@yahoo.com dr. nitin seth department of management indian institute of foreign trade new delhi, india e-mail: nitinseth@aol.in abstract total quality management (tqm), which employs a set of criterions and practices for improving organizational performance was developed for industrial purposes and is now attracting the attention of researchers in the field of education. in this paper, tqm implementation criterions were examined for their relative importance for better implementation in engineering education. a decision support method, the analytic hierarchy process (ahp), has been applied to prioritize and rank 13 tqm implementation criterions and 68 sub criterions by calculating their local and global weights. to effectively implement tqm, engineering education institutions (eeis) should focus on the most important tqm practices such as resource management, student focus and service culture, and address the less important ones at a later stage. by doing so, they would gradually reduce employee resistance to change, especially if positive results are obtained. keywords: total quality management (tqm), engineering education, analytic hierarchy process (ahp), india. 1 corresponding author ijahp article: mehta, verma, seth/prioritizing total quality management implementation criterion for engineering education: an analytic hierarchy process analysis international journal of the analytic hierarchy process issn 1936-6744 36 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.197 1. introduction the propagation of higher education institutions has played a vital role in the economic regeneration of the world. in the market oriented environment, higher education is facing ever increasing expectations and pressure from its stakeholders (sahney, banwet, and karunes, 2004). one of the strategies being adopted globally to meet these expectations is focusing on improving the quality of education (sharma, moon, and bae, 2008). yet, the quality of higher education in many countries falls short of attaining global-level of excellence (senthilkumar and arulraj, 2011), and india is no exception. indian higher education has witnessed mostly unplanned exponential growth in the past few years which is exerting an intense pressure upon the institutions for survival and success (umashankar, and dutta, 2007). engineering education particularly has witnessed a phenomenal expansion both in terms of growth and diversity; as a result there has been a sharp increase in the number of private colleges as well as universities in india. the number of eeis in 1990 was 74 with an intake capacity of 5,200, and this has increased to 2,450 and 1,761,976 respectively in 2012 (www.aicte.org, 2012). due to this explosive growth, the quality issues have been neglected to a great extent. it is imperative to look critically into these issues, so that india will be ready to face challenges in the wake of globalization (sakthivel and raju, 2006a). indian engineering education has followed some kind of quality evaluations to ensure that it offers high standards of teaching and learning as well as research. the regulators for higher education in india are the university grants commission (ugc), the national board of accreditation (nba) and the all india council of technical education (aicte). the nba has a well-defined system of evaluating the quality and has the most stringent norms for quality assessment, but nba accreditation is still optional in india. as a result, many engineering institutes are being run without this accreditation. from a quality management perspective, the quality assurance methods adopted by these regulatory bodies are weak because they rely on inspection and corrective action. this results in inefficiency because inspection activities are tedious and have a high cost. the tqm philosophy emphasizes prevention and continuous improvement, therefore institutions can improve their effectiveness over time with minimum costs by implementing tqm measures (kanji, malek, and tambi, 1999). tqm is a synthesis of a number of discrete principles requiring all employees at every level of an organization to focus his/her efforts to help improve each activity of the organization (mehra, hoffman, and sirias, 2001). there are many issues which need to be addressed and prioritized with regards to implementation of a quality programme in engineering education. this paper tries to identify such issues and address them by prioritizing them in terms of their criticality in improving the quality. talib et al. (2011) have also emphasized the need to determine the implementation priority of tqm criterions so as to achieve maximum benefits and desired goals. therefore, the research is aimed to prioritize the relative importance of tqm criterion and sub criterion for implementation, so that an institute can evaluate their current practices to improve their performance. however, prioritization of the tqm criterion and sub criterion is a complex task as it requires multiple output measurements that match with the multiple objectives of the engineering institution, and also a technique that could provide the correct and required information to the decision makers. ijahp article: mehta, verma, seth/prioritizing total quality management implementation criterion for engineering education: an analytic hierarchy process analysis international journal of the analytic hierarchy process issn 1936-6744 37 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.197 the ahp is one of the most widely used multiple criterion decision making (mcdm) tools which assigns weights to each criterions as per their priority (vidhya and kumar, 2006). therefore the present study uses this approach to determine the relative importance of each criterion in enhancing the quality in engineering education. the major implication of the research will be that eeis will be able to focus on criterions which have a high priority for improving their quality. 2. literature review ahp is a decision-making tool that can help describe the general decision operation by decomposing a complex problem into a multi-level hierarchical structure of objectives, criterion, sub-criterion, and alternatives (saaty, 1977, 1994, 1996, 2000, and 2010). ahp gives weight and rank to all the criterion and sub-criterion within each level of the hierarchy after decomposing a complex, multi-criterion problem into multiple levels of hierarchy (saaty, 1990). the ahp approach is a consensus, inclusive based decision without disregarding any opposing views (yusof and salleh, 2013). ahp is a theory of measurement through pairwise comparisons and relies on the judgments of experts to derive priority scales (saaty, 2010). experts are interviewed and pair-wise comparison judgments are applied to pairs of homogenous criterion, eventually generating the overall priorities for ranking the alternatives (saaty, 1980). ahp also helps to capture both subjective and objective evaluation measures, providing a useful mechanism for checking the consistency of the evaluation measures and alternatives suggested by the team, thus reducing bias in decision making (ho, 2008). the ahp method is extensively used in real life situations, such as maintenance selection problem (bertolini and bevilacqua, 2006), higher education (ho, et al., 2006), optimization of wastewater treatment (zeng, g., jiang, huang, xu, and li, 2007), consumer bank selection decisions (javalgi, armacost, and hosseini,1989), measuring performance (frei and harker, 1999), allocating resources (ramanathan, and ganesh, 1995), new product screening (calantone, benedetto, and schmýdt, 1999), benchmark determination (partovi, 1994), optimizing distribution networks (sharma, moon, and bae, 2008), choosing the best policy after finding a set of alternatives (poh, and ang, 1999), identification of new ventures (jain, and nag, 1996), vocational education (lam, poon, and chin, 2008), software development process (lee, pham, and zhang, 1999), service industries (talib rahman and qureshi, 2011; beshah, and kitaw, 2013). the literature related to ahp found that researchers widely used priority and ranking for a variety of different applications, and therefore this process was applied in the present study. tqm is a management approach for improving organizational performance that encompasses a variety of topics both technical and behavioral (shams-ur, 2004). the subject of tqm has been explored by several researchers and various attributes and measurements for tqm implementation have been suggested. for instance, deming (1986) prescribed tqm in 14 points, which he claimed to be a set of criterions necessary to remain competitive in providing products and services. anderson (1994) studied these criterions, and developed a conceptual framework for tqm using seven concepts which includes visionary leadership, internal and external communication, learning, process management, continuous improvement, employee fulfillment, and customer satisfaction. several studies were conducted to highlight the linkages of tqm with different concepts. talib, rahman & qureshi (2011) identified 17 tqm practices and grouped into three ijahp article: mehta, verma, seth/prioritizing total quality management implementation criterion for engineering education: an analytic hierarchy process analysis international journal of the analytic hierarchy process issn 1936-6744 38 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.197 factor (strategic factors, tactical factors and operational factors) for service industries. sarathy (2013) determined the important factors that influence the tqm practice in the real estate industry using an ahp 1uestionnaire. sagar and tomar (2014) ranked critical success factors of tqm using the ahp. the existing literature shows that although much empirical research has been conducted which deals with tqm practices, studies exploring tqm concepts in the indian context and their effects on overall performance in the engineering education sector are rarely seen. it is also evident from the literature that it is a very challenging and complex task to evaluate tqm criterions in a group. in order to bridge this gap, this paper attempts to provide a framework and a system for the educational policy makers and the performance monitoring committee. this framework is for designing standard multiple performance measurement tools based on tqm criterions for evaluating and comparing the performance of eeis in india. 3. determining the relative weights of tqm implementation criterions and sub criterions for the engineering education sector an ahp-based evaluation model was developed for determining the relative weights of tqm implementation criterions. first, tqm implementation criterion and sub -criterion for the engineering education were identified. then a pair-wise comparison matrix was established by the panel of experts and the eigenvalues of each pair-wise comparison matrix was calculated. finally, after testing the consistency of each comparison matrix, local and global weights of each criterion and sub-criterion were calculated (figure 1). 3.1 identifying the tqm implementation criterion and sub-criterion for engineering education engineering education is essential for the improvement of the technical manpower of a country. in india, many technical institutions have iso9001: 2000 and nba certification, but at times the quality of education in those institutes is not satisfactory (sayeda, rajendran, and lokachari 2010). a widely accepted view on quality is the degree to which stakeholders’ needs and expectations are consistently satisfied. however, formulating a single, comprehensive definition that can integrate the interests of all the customers and stakeholders becomes challenging with the engineering educational system because it has various customers and stakeholders..the “total quality management” philosophy can help build a customer-driven learning organization dedicated to total customer satisfaction through continuous improvement in the effectiveness and efficiency of the organization and its processes within this challenging environment (choppin, 1995). many researchers have formulated frameworks for quality improvements (johnson, 1993; venkatraman, 2007; khan and mahapatra, 2007; mizikaci, 2007; srikanthan and dalrymple, 2003), and these frameworks have been given different names such as continuous quality improvement (cqi), strategic quality management (sqm) or total quality management (tqm). even though there might be some differences among these approaches, the term tqm is considered to be more general and captures the essence of quality improvement (venkatraman, 2007). therefore, implementation of tqm practices in engineering institutions may be integral in helping the students receive a quality education. ijahp article: mehta, verma, seth/prioritizing total quality management implementation criterion for engineering education: an analytic hierarchy process analysis international journal of the analytic hierarchy process issn 1936-6744 39 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.197 prioritizing of tqm implementation criterions for engineering education identify the tqm implementation criterion and sub -criterion for the engineering education identify panel of experts normalize each column of the matrix then calculate the eigenvalue of each pair-wise comparison matrix calculate the local and global weights of each criterion and subcriterion incorporate findings and improve tqm implementation figure 1. schematic representation of the methodology test the consistency of each comparison matrix establish the pair-wise comparison matrix found ok not found ok ijahp article: mehta, verma, seth/prioritizing total quality management implementation criterion for engineering education: an analytic hierarchy process analysis international journal of the analytic hierarchy process issn 1936-6744 40 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.197 tqm is a set of guiding philosophical criterion and sub criterion that provide a foundation for any continuously improving organization. a literature review helped identity the tqm implementation criterion and sub-criterion for the engineering education sector which have been categorised into 13 major criterion and 68 sub criterion with the use of the delphi method (mehta, verma and seth, 2013). 3.2 identifying a panel of experts a panel of experts in the engineering education sector includes those who have expertise by virtue of having a long association with the system like students, lecturers, professors, director/principals, parents, industrialists and support staff. heterogeneous groups with widely varying personalities produce more highly acceptable solutions than homogeneous groups (delbecq, van and gustafson, 1975; rowe, 1994). researchers have different opinions regarding the appropriate size for a panel of experts; however many researchers (maccarthy and atthirawong, 2003; linstone and turoff, 1975; landeta, 1999) have recommended a group size in the range of seven to thirty members. in the present study an expert panel of 26 members was formed which consisted of students (5), lecturers (3), associate professors (3), professors (4), directors/principals (3), parents (3) and industrialists (5). 3.3 establishing the hierarchy structure for tqm implementation criterion when dealing with complex issues, it is most effective to organize them in a hierarchical structure. this hierarchical organization and determine of the inter-relationships is difficult. the first step in the process is to set the goal and then define the criterions to achieve this goal with different alternatives (figure 2). in the present case, the goal is to prioritize tqm implementation criterions for engineering education and rank them by calculating their local and global weights. ijahp article: mehta, verma, seth/prioritizing total quality management implementation criterion for engineering education: an analytic hierarchy process analysis international journal of the analytic hierarchy process issn 1936-6744 41 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.197 3.4 establish the pair-wise comparison matrix in order to determine the importance of specific criterion and sub-criterion, pair-wise comparisons were carried out. the comparison matrix is a square matrix with n x n dimensions. pair-wise comparisons are based on the scale of relative importance that assumes values between 1 and 9 (table 1). table 1 scale of relative importance intensity of relative importance definition explanation 1 equal importance level two elements have equal importance regarding the element in higher level 3 week dominance experience or judgment slightly favors one element 5 strong dominance experience or judgment strongly favors one element 7 demonstrated dominance of one element proved in practice 9 absolute dominance the highest order dominance of one element over another 2,4,6,8 intermediate values compromise is needed this scale can be applied to criterion according to the expert panel judgments. each expert makes a pair-wise comparison of the criterions and assigns them relative scores. since the values on the diagonal represent the same factor, they become 1. if the figure 2. ahp hierarchy prioritizing of tqm implementation criterions for engineering education institutional resource c1 long term strategy and planning c2 excellence human resource management c3 continuous assessment and improvement c4 top management commitment and visionary leadership c5 local weight global weight student focus c6 employee focus c7 alumni focus c8 information management system c9 quality mission & vision statement c10 service culture c11 innovative academic philosophy and method c12 industry institution partnership c13 ijahp article: mehta, verma, seth/prioritizing total quality management implementation criterion for engineering education: an analytic hierarchy process analysis international journal of the analytic hierarchy process issn 1936-6744 42 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.197 preference is used in favor of the factor in the row when the factor in any row is compared with the factor in the column, a fraction (1/importance value) is preferred (yaraliodlu, 2001). table 2 shows the aggregate pair-wise comparison matrix for the thirteen tqm implementation criterions. table 2 establishing a comparison matrix of the tqm implementation criterion tqm implementation criterion c1 c2 c3 c4 c5 c6 c7 c8 c9 c10 c11 c12 c13 c1 institutional resource management 1 2 1 5 6 1/2 3 4 9 7 2 7 8 c2 long term strategy and planning 1/2 1 1/2 4 4 1/3 2 3 9 5 1 6 7 c3 excellence human resource management 1 2 1 5 6 1/2 3 4 9 7 2 7 8 c4 continuous assessment and improvement 1/5 1/4 1/5 1 2 1/5 1/3 1/2 7 2 1/4 3 4 c5 top management commitment and visionary leadership 1/6 1/4 1/6 1/2 1 1/6 1/4 1/3 6 2 1/4 3 4 c6 student focus 2 3 2 5 6 1 4 5 9 6 3 7 8 c7 employee focus 1/3 1/2 1/3 3 4 1/4 1 2 8 5 1/2 6 6 c8 alumni focus 1/4 1/3 1/4 2 3 1/5 1/2 1 8 4 1/3 5 6 c9 information management system 1/9 1/9 1/9 1/7 1/6 1/9 1/8 1/8 1 1/5 1/9 1/5 1/3 c10 quality mission & vision statement 1/7 1/5 1/7 1/2 1/2 1/6 1/5 1/4 5 1 1/5 2 3 c11 service culture 1/2 1 1/2 4 4 1/3 2 3 9 5 1 6 7 c12 innovative academic philosophy and method 1/7 1/6 1/7 1/3 1/3 1/7 1/6 1/5 5 1/2 1/6 1 3 c13 industry institution partnership 1/8 1/7 1/8 1/4 1/4 1/8 1/6 1/6 3 1/3 1/7 1/3 1 similarly, an aggregate pair-wise comparison matrix for the sub-criterion of each thirteen tqm implementation criterion is established. 3.5 normalizing each column of the matrix and calculating the eigenvalue to find a normalized matrix, each element of the comparison matrix should be divided with the sum of its column elements. saaty (1990) suggested that the largest eigenvalue λmax can be calculated by the equation 1. ijahp article: mehta, verma, seth/prioritizing total quality management implementation criterion for engineering education: an analytic hierarchy process analysis international journal of the analytic hierarchy process issn 1936-6744 43 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.197 where aij represents a quantified judgment on a pair of elements and wi and wj are weights (for i, j =1, 2... n.) 3.6 testing the consistency of each comparison matrix to check the consistency of comparison matrix a consistency ratio is defined, which is the ratio between the consistency index (ci) and the random consistency value (ri). the value of the cr should be less than or equal to 0.1 if the order of the matrix is five or more for the comparison matrix values to be consistent (saaty, 1994). whereas on the basis of normalized matrix, ci is calculated using the equation 2. where n is the order of matrix, the random consistency value (ri) for corresponding n is found from the table of random consistency. table 3 shows a normalized matrix with eigenvalues, the consistency index and the consistency ratio of the tqm implementation criterions. (1) λmax = j=1 ∑ n aij wj wi ____ ci = (λmax n) / (n – 1) (2) ijahp article: mehta, verma, seth/prioritizing total quality management implementation criterion for engineering education: an analytic hierarchy process analysis international journal of the analytic hierarchy process issn 1936-6744 44 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.197 table 3 normalized matrix for tqm implementation criterion tqm implementation criterion c1 c2 c3 c4 c5 c6 c7 c8 c9 c10 c11 c12 c13 c1 institutional resource management .15 .18 .15 .16 .16 .12 .18 .17 .10 .16 .18 .13 .12 c2 long term strategy and planning .08 .09 .08 .13 .11 .08 .12 .13 .10 .11 .09 .11 .11 c3 excellence human resource management .15 .18 .15 0.1 6 0.1 6 .12 .18 .17 .10 .16 .18 .13 .12 c4 continuous assessment and improvement .03 .02 .03 .03 .05 .05 .02 .02 .08 .04 .02 .06 .06 c5 top management commitment and visionary leadership .03 .02 .03 .02 .03 .04 .01 .01 .07 .04 .02 .06 .06 c6 student focus .31 .27 .31 .16 0.1 6 .25 .24 .21 .10 .13 .27 .13 .12 c7 employee focus .05 .05 .05 .10 .11 .06 .06 .08 .09 .11 .05 .11 .09 c8 alumni focus .04 .03 .04 .07 .08 .05 .03 .04 .09 .09 .03 .09 .09 c9 information management system .02 .01 .02 .00 .00 .03 .01 .01 .01 .00 .01 .00 .01 c10 quality mission & vision statement .02 .02 .02 .02 .01 .04 .01 .01 .06 .02 .02 .04 .05 c11 service culture .08 .09 .08 .13 .11 .08 .12 .13 .10 .11 .09 .11 .11 c12 innovative academic philosophy and method .02 .02 .02 .01 .01 .04 .01 .01 .06 .01 .02 .02 .05 c13 industry institution partnership .02 .01 .02 .01 .01 .03 .01 .01 .03 .01 .01 .01 .02 λmax = 14.759998, ci = 0.146665, ri = 1.54, cr = 0.095237≤ 0.1 similarly normalized matrix and eigenvalue are calculated for sub-criterions of each criterion. ijahp article: mehta, verma, seth/prioritizing total quality management implementation criterion for engineering education: an analytic hierarchy process analysis international journal of the analytic hierarchy process issn 1936-6744 45 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.197 3.7 calculate the local and global weights of each criterion and sub-criterion local weight is the weight of the each major criterion and sub criterion with respect to the previous hierarchical level. global weight is the weight of the each major criterion and sub criterion with respect to the highest hierarchical level (talib et al., 2011). the global weight of sub criterion is the product of local weight for criterion i to local weight for sub-criterion j with respect to criterion i (drake, 1998). the 13 tqm implementation criterions have the same local and global weights. the local and global weights calculated for each criterion and sub-criterion are shown in table 4. ijahp article: mehta, verma, seth/prioritizing total quality management implementation criterion for engineering education: an analytic hierarchy process analysis international journal of the analytic hierarchy process issn 1936-6744 46 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.197 table 4 weights and rank of thirteen criterions and sixty eight sub-criterions criterion weights of criterion rank of criterion sub-criterion local global weights of subcriterion rank of subcriterion weights of subcriterion rank of sub-criterion (c1) resource management 0.152 ii c11 tangible resource – essential infrastructure 0.5577 1 0.0850 1 c12 tangible resource – support services 0.0417 4 0.0064 40 c13 intangible resource 0.1330 2 0.0203 17 c14 financial resources 0.2676 3 0.0408 6 (c2) long term strategy and planning 0.103 iii c21 social responsibility (sr) 0.3402 1 0.0350 7 c22 professional society activities 0.0449 6 0.0046 46 c23 annual academic calendar 0.2577 0.0265 14 c24 promotional policies/procedure 0.0210 7 0.0022 57 c25 design of course structure based on job requirement 0.1653 3 0.0170 18 c26 opportunities for campus training and placement 0.0718 5 0.0074 34 c27 annual budget utilization 0.0991 4 0.0102 30 (c3) human resource management (hrm) 0.152 ii c31 teamwork 0.1878 2 0.0286 9 c32 training 0.0237 5 0.0036 49 c33 participation and ownership 0.0862 3 0.0131 24 c34 employee empowerment 0.0862 3 0.0131 25 c35 transparency and fairness in recruitment 0.0428 4 0.0065 38 c36 maintain faculty-student ratio 0.3856 1 0.0588 4 c37 communication 0.1878 2 0.0286 10 (c4) continuous assessment and improvement 0.040 vi c41 continuous assessment 0.2964 1 0.0120 27 c42 students’ evaluation 0.0210 5 0.0009 66 c43 improve overall performance of the faculty, staff and students 0.0374 4 0.0015 61 c44 update basic resources 0.0758 3 0.0031 51 c45 comparison of actual with planned performance 0.1364 2 0.0055 43 c46 continuous improvement 0.2964 1 0.0120 28 c47 benchmarking 0.1364 2 0.0055 44 ijahp article: mehta, verma, seth/prioritizing total quality management implementation criterion for engineering education: an analytic hierarchy process analysis international journal of the analytic hierarchy process issn 1936-6744 47 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.197 criterion weights of criterion rank of criterion sub-criterion local global (c5) top management commitment and visionary leadership 0.059 iv c51 management commitment 0.3941 1 0.0234 16 c52 remove barriers to pride of workmanship 0.0957 4 0.0057 42 c53 strategic planning 0.2375 2 0.0141 22 c54 annual budget allocation 0.1539 3 0.0091 31 c55 top management learning 0.0235 4 0.0014 62 c56 top management participation 0.0477 5 0.0028 53 c57 top management encouragement 0.0477 5 0.0028 54 (c6) student focus 0.206 i c61syllabus covered with student satisfaction 0.3748 1 0.0772 2 c62 extra-curricular and co-curricular activities 0.1313 3 0.0271 11 c63 academic development 0.2313 2 0.0476 5 c64 personality development 0.0714 4 0.0147 19 c65 students complaint information 0.0216 6 0.0045 47 c66 students’ feedback 0.0382 5 0.0079 33 c67 motivational activities 0.1313 3 0.0271 12 (c7) employee focus 0.078 iv c71 security of job 0.1867 2 0.0145 20 c72 curriculum development 0.0259 5 0.0020 58 c73 recognition and reward 0.0884 3 0.0069 36 c74 adequate and efficient teaching assistants 0.0884 3 0.0069 37 c75 effective problem solving 0.0453 4 0.0035 50 c76 employee feedback 0.3786 1 0.0295 8 c77 in house r&d activity 0.1867 2 0.0145 21 (c8) alumni focus 0.034 vii c81 alumni feedback 0.7235 1 0.0245 15 c82 information circulation 0.0833 3 0.0028 55 c83 alumni empowerment 0.1932 2 0.0065 39 (c9) information and analysis system 0.010 xii c91 data and information of every process are gathered and analyzed 0.5579 1 0.0055 45 c92 improve and update the information systems 0.2633 2 0.0026 56 c93 display information which reflect institute image 0.0569 4 0.0006 67 c94 display information which attract internal and external stakeholder 0.1219 3 0.0012 64 ijahp article: mehta, verma, seth/prioritizing total quality management implementation criterion for engineering education: an analytic hierarchy process analysis international journal of the analytic hierarchy process issn 1936-6744 48 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.197 criterion weights of criterion rank of criterion sub-criterion local global (c10) quality mission & vision statement 0.026 viii c101 vision statement 0.5059 1 0.0131 26 c102 mission statement 0.3260 2 0.0084 32 c103 students quality goal 0.1137 3 0.0029 52 c104 shared quality policy 0.0543 4 0.0014 63 (c11) service culture 0.103 iii c111 friendly atmosphere in campus 0.6333 1 0.0651 3 c112 internal and external stakeholder pride of workmanship 0.1062 3 0.0109 29 c113 internal and external stakeholder work together 0.2605 2 0.0268 13 (c12) innovative academic philosophy and method 0.022 ix c121 adopt new academic philosophy 0.6434 1 0.0139 23 c122 use new and latest technology 0.2828 2 0.0061 41 c123 right the work first time 0.0738 3 0.0016 60 (c13) industry institution partnership 0.015 x c131 courses/programs partnership 0.0678 4 0.0010 65 c132 expert/guest lecturers 0.2602 2 0.0038 48 c133 industry based projects 0.0348 5 0.0005 68 c134 continuously searching demands of industry 0.1344 3 0.0020 59 c135 placements partnership 0.5028 1 0.0074 35 ijahp article: mehta, verma, seth/prioritizing total quality management implementation criterion for engineering education: an analytic hierarchy process analysis international journal of the analytic hierarchy process issn 1936-6744 49 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.197 3.8 incorporating findings and improving tqm implementation from the calculations carried out in the step 6, eeis can prioritize and rank the tqm criterion and sub-criterion and then allocate the resources accordingly to attain the maximum advantages. the use of the ahp has generic applications because its structure and hierarchy can be easily modified to incorporate specific attributes (banuelas and antony, 2003). ahp can, therefore, be adopted for prioritizing tqm practices in different service sectors and processes according to the specific objectives set by decision-makers. 4. results and discussion the study identified the priority ranking of the 13 tqm implementation criterion and 68 sub criterion for the eeis. by understanding their relative importance, eeis can evaluate their current practices and re-allocate resources and efforts to these criterion and sub criterion to improve their tqm performance. table 4 shows the local weights and global weights of 13 tqm implementation criterion and 68 sub criterion that are normalized based on the ahp analysis and ranked accordingly. implementation criterion “student focus (c6)” with a weight of 0.206 is the most important criterion, followed by “human resource management (hrm) (c3)” with a weight of 0.152, and “service culture (c11)” and “long term strategy and planning (c2)” with same a weight of 0.103.  the top four ranked tqm implementation criterions were “student focus (c6)”, “human resource management (hrm) (c3)”, service culture (c11)” and “long term strategy and planning (c2)”. therefore, the eeis should focus on student satisfaction as the utmost priority by taking care of syllabus coverage, organizing motivational lectures, emphasizing extra-curricular and co-curricular activities, and seeking student feedback for assessment of syllabus coverage and quality of classroom teaching. implementation of these practices will result in long term benefits for the eei and further help in the successful implementation of a tqm programme. without these practices it will be hard for tqm to be implemented effectively and successfully.  human resource management (hrm) (c3) was also one of the top most criterions because it focuses on effective utilization of available human resources to improve and enhance the systems. hrm systems will rate the best if the employees are provided proper training and are involved in the decision making process. along with this, it is also required that a culture of team work is inculcated in the employee’s transparency, and fairness in the recruitment process is maintained. proper faculty-student ratio and communication between faculty and students and between the departments is also important.  eeis should develop a long term strategy (c2) in planning for their course structure so that job based courses can be initiated and training provided. programmes on social responsibility and membership in professional societies will also go a long way in developing a tqm culture in an institute. a proper ijahp article: mehta, verma, seth/prioritizing total quality management implementation criterion for engineering education: an analytic hierarchy process analysis international journal of the analytic hierarchy process issn 1936-6744 50 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.197 service culture (c11) will result in a friendly atmosphere in the institute, helping to provide a synergistic relationship between internal and external stakeholders. discussion regarding important sub criterion of each criterion based on its local weight is as follows:  sub criterion “tangible resource – essential infrastructure (c11)” with a weight of 0.5577 for criterion resource management (c1) was the highest priority. this emphasized the quality and quantity of the entire physical infrastructure such as buildings with adequate laboratory set-ups, a library with adequate and updated books and technical journals, computing facilities and information systems, and assumes a key role in the teaching-learning processes.  sub criterion “social responsibility (sr) (c21)” with a weight of 0.3402 for the criterion long term strategy and planning (c2), plays an important role in grooming the personality of a student and making him a more aware and responsible citizen. the sense of social responsibility can be inculcated in students by organizing welfare activities and general awareness programmes for the surrounding society.  with respect to the criterion human resource management (hrm) (c3), expert’s ranked “maintain faculty-student ratio (c36)” as the highest sub criterion with a weight of 0.3856. this implied that if the institute maintains a proper faculty ratio, faculty can devote quality time to pedagogic activities such as preparing assignments and lecture notes, conducting regular class tests, lab work etc., separate from other developmental activities related to infrastructure, modernization of labs, setting up of new facilities etc.  continuous assessment (c41) plays a significant role in creating and maintaining standards for successful tqm implementation. with a weight of 0.2964, this criterion ranked as the top most sub criterion under continuous assessment and improvement (c4). therefore, the management should have clear objectives and frame policies to review the existing processes so that the ever changing needs and expectations of the stakeholder can be effectively met.  committed support from the top management and their efforts towards neverending quality improvement efforts leads to better customer service and satisfaction. in the present study, experts were also of the same view as evidenced by ranking the sub criterion “management commitment (c51)” highest with a weight of 0.3941 for criterion top management commitment and visionary leadership (c5). this implied that the commitment of the management is a driving force for the successful implementation of tqm programme in an eei also.  sub criterion “syllabus covered with students satisfaction (c61)” with a weight of 0.3748 for criterion student focus (c6), was most important in the expert’s opinion because they believe if management collects regular feedback regarding the syllabus covered and the quality of teaching from the students, it will help in ijahp article: mehta, verma, seth/prioritizing total quality management implementation criterion for engineering education: an analytic hierarchy process analysis international journal of the analytic hierarchy process issn 1936-6744 51 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.197 better analysis of various tqm processes, such as preparing the academic calendar, course curriculum designing, feedback of faculty from students etc.  sub criterion “employee feedback (c76)” with a weight of 0.3786 for criterion employee focus (c7), was important because getting employees suggestions for improvements and also understanding their perceptions of the institute is helpful. .  sub criterion “alumni feedback (c81)” with a weight of 0.7235 for criterion alumni focus (c8), reflected the importance of alumni and obtaining their feedback regarding the course structure, prevailing market trends and teaching learning methods adopted by the institute. alumni can also support the institute by arranging campus placements and giving financial support to students and the institutes if required.  sub criterion “data and information of every process are gathered and analyzed (c91)” with a weight of 0.5579 for criterion information and analysis system (c9), implied the importance of data collection and its analysis regarding various processes in the institute. this may help management improve tqm practices and frame a long term vision for the institute.  sub criterion “vision statement (c101)” with a weight of 0.5059 for criterion quality mission & vision statement (c10), showed the management’s commitment to long term quality. this commitment helps instill confidence in employees towards management’s future plans.  sub criterion “friendly atmosphere in campus (c111)” with a weight of 0.6333 for criterion service culture (c11), signified the importance of a good academic environment in the institute. this not only helps to improve the service culture of the institute, but also creates a good impression on the stakeholders.  sub criterion “adopt new academic philosophy (c121)” with a weight of 0.6434 for criterion innovative academic philosophy and method (c12), focused on the management’s commitment to make necessary changes in teaching learning process as per the needs of the stakeholders.  sub criterion “placements partnership (c135)” with a weight of 0.5028 for criterion industry institution partnership (c13), referred to the interaction of the academia with the corporate bodies, in terms of exchange of ideas, problems and projects for mutual benefit. the management, faculty and students may get to know the demands and requirements of the corporate world which can help them redesign/deliver the course curriculum. this might possibly turn out to be a winwin situation for both because it helps students to discover their interests and capabilities and at the same time helps the corporations find prospective candidates. the three most important tqm implementation sub criterion with respect to global weights are “tangible resource – essential infrastructure (c11)” with a weight of 0.0850 for resource management (c1), “syllabus covered with student satisfaction (c61)” with ijahp article: mehta, verma, seth/prioritizing total quality management implementation criterion for engineering education: an analytic hierarchy process analysis international journal of the analytic hierarchy process issn 1936-6744 52 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.197 a weight of 0.0772 for student focus (c6), and “friendly atmosphere in campus (c111)” with a weight of 0.0651 for service culture (c11). eeis should prioritize these criterions for successful implementation of tqm. these criterions indicate that:  availability of tangible (like infrastructure) resources in eeis can improve the overall quality of initiatives supported by top management to develop effective management for availability of resources whenever needed.  student satisfaction regarding the quality of teaching affects an institute’s performance.  a positive attitude and organized service culture within the institution can help to achieve total quality education. 5. implications from a management perspective, this study offers a number of managerial implications for eei decision-makers.  first, the ahp model developed in this research will be very useful to decisionmakers in framing guidelines for implementing tqm, and in evaluating the effectiveness of their current tqm practices.  second, infrastructural facilities are a pre-condition for an educational institution to carry out its activities smoothly. continuous support from the top management and their effort towards maintaining sufficient infrastructure leads to better customer service and satisfaction. thus, this is a driving force for the successful implementation of tqm programme in eeis.  third, the student focus (c6) is crucial in order to obtain the full advantage of tqm. it is the top management’s responsibility to collect information (data) regarding students’ satisfaction, concerns and issues of their well-being. management should make attempts to hold meetings with the students and involve them in various academic activities such as course curriculum designing, infrastructural development projects, inviting for sharing knowledge, etc. 6. conclusions ahp was used to prioritize tqm implementation criterion and sub criterion for engineering education by identification of their relative importance. a total of 13 tqm implementation criterions and 68 sub criterions were prioritized by calculating their local and global weights. the implementation of the tqm programme requires many changes in institutions, and therefore many people may be reluctant to implement it. the analysis shows that to effectively implement tqm, eeis should focus on the high priority tqm practices such as resource management, student focus and service culture, and address the less important ones at a later stage. doing this would gradually reduce employee resistance to change, especially if positive results are generated. ijahp article: mehta, verma, seth/prioritizing total quality management implementation criterion for engineering education: an analytic hierarchy process analysis international journal of the analytic hierarchy process issn 1936-6744 53 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.197 this research study, although exploratory in nature, provides insights into determining the effectiveness of tqm implementation in eeis; future research could validate the proposed criterion and sub-criterion. further, a self-assessment system could be developed to evaluate an institute’s current performance. this would help to evaluate the strengths and weaknesses of the institute, and provide information to develop appropriate strategies for making improvements. moreover, the applicability of the results of this study are limited in their scope because this study relies on the judgment of only 26 experts from four eeis including five experts from industry and all the 26 experts were from a single country. ijahp article: mehta, verma, seth/prioritizing total quality management implementation criterion for engineering education: an analytic hierarchy process analysis international journal of the analytic hierarchy process issn 1936-6744 54 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.197 references anderson, j. c., rungtusathan, m. and schroeder, r. (1994). a theory of quality management underlying the deming management method. academy of management review, 19, 472–509. banuelas, r., and antony, j. (2003). going from six sigma to design for six sigma: an exploratory study using analytic hierarchy process. the tqm magazine, 15(5), 334–344. bertolini, m. and bevilacqua, m., (2006). a combined goal programming-ahp approach to maintenance selection problem. reliability engineering and system safety, 91(7), 839–848. beshah, b. and kitaw, d. (2013). ahp application in a financial institution. international journal of the analytic hierarchy process, 5(1), 54-71. calantone, r. j., benedetto, c. a. and schmýdt, j.b., (1999). using the analytic hierarchy process in new product screening. the journal of product innovation management, 16, 65-76. choppin j., (1995). total quality management – what isn’t it?. managing service quality, 5(1), 47–49. delbecq a.l., van de ven a.h. & gustafson d.h. (1975). group techniques for program planning: a guide to nominal and delphi processes. glenview, il: scott, foresman and co. deming, e. (1986). out of the crisis. cambridge, ma: mit press. drake, p. (1998). using the analytic hierarchy process in engineering education. international journal of engineering, 14(3), 191–196. frei, f.x. and harker, p.t. (1999). measuring aggregate process performance using ahp. european journal of operational research, 116, 436-442. ho, w., (2008). integrated analytic hierarchy process and its application: a literature review. european journal of operational research, 186(1), 211-228. jain, b.a., & nag, b.n. (1996). a decision support model for investment decision in new ventures. european journal of operational research, 90(3), 473–486. javalgi, r.g., armacost, r.l., and hosseini, j.c. (1989). using the analytic hierarchy process for bank management: analysis of consumer bank selection decisions. journal of business research, 19(1), 33-49. johnson, r.s. (1993). tqm: management processes for quality operations, milwaukee, wi: asqc quality press. ijahp article: mehta, verma, seth/prioritizing total quality management implementation criterion for engineering education: an analytic hierarchy process analysis international journal of the analytic hierarchy process issn 1936-6744 55 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.197 kanji, g. k., malek, a. and tambi, b. a. (1999). total quality management in uk higher education institutions. total quality management & business excellence, 10(1), 129 – 153. khan, m.s. and mahapatra s.s., (2007). a framework for analysing quality in education settings. european journal of engineering education, 32(2), 205-217. lam, m.y., poon, g.k.k., & chin, k.s. (2008). an organizational learning model for vocational education in the context of tqm culture. international journal of quality and reliability management, 25(3), 238–255. landeta, j. (1999), el me´todo delphi, ariel, barcelona. lee, m., pham, h., and zhang, x. (1999). a methodology for priority setting with application to software development process. european journal of operational research, 118, 375-389. linstone, a. and turoff, m. (1975). the delphi method: techniques and applications. reading, ma: addison-wesley. maccarthy, b.l. and atthirawong, w. (2003). factors affecting location decisions in international operations: a delphi study. international journal of operations & production management, 23(7), 794. mehra, s., hoffman, j.m. and sirias, d. (2001). tqm as a management strategy for the next millennia. international journal of operations and production management, 21(5), 855-76. mehta n., verma p. and seth n. (2013). selection of tqm criterions using delphi technique for engineering education in india. international journal of mechanical engineering and research, 3(2), 66-86. mizikaci f. (2006). a systems approach to program evaluation model for quality in higher education. quality assurance in education, 14, 37-53. partovi, f.y. (1994). determining what to benchmark: an analytic hierarchy process approach. international journal of operations and production management, 14(6), 25-39. poh, k.l. and ang, b.w. (1999). transportation fuels and policy for singapore: an ahp planning approach. computers and industrial engineering, 37, 507-525. ramanathan, r. and ganesh, l. s. (1995). using ahp for resource allocation problems. european journal of operational research, 80, 410-417. rowe g. and wright g. (1999). the delphi technique as a forecasting tool: issues and analysis. international journal of forecasting, 15, 353 –375. saaty, t. l. (1980). analytic hierarchy process. new york: mcgraw hill. ijahp article: mehta, verma, seth/prioritizing total quality management implementation criterion for engineering education: an analytic hierarchy process analysis international journal of the analytic hierarchy process issn 1936-6744 56 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.197 saaty, t. l. (1990). multi-criteria decision making: analytic hierarchy process. pittsburgh, pa: rws publications. saaty, t.l. (1977). a scaling method for priorities in hierarchical structures. mathematical psychology, 15, 234-281. saaty, t.l. (1994). fundamentals of decision making and priority theory with the analytic hierarchy process. pittsburgh, pa: rws publications. saaty, t.l. (1996). decision making with dependence and feedback: the analytic network process. pittsburgh, pa: rws publications. saaty, t.l. (2000). decision making for leaders. pittsburgh, pa: rws publications. saaty, t.l. (2010). principia mathematica decernendi: mathematical principles of decision making: generalization of the analytic network process to neural firing and synthesis. pittsburgh, pa: rws publications. sagar m. k. and tomar a. s. (2014). ranking cirtical success factor of total quality management ahp. international journal of mechanical and production engineering, 2, 29-34. sahney, s., banwet, d. k. and karunes, s., (2004). conceptualizing total quality management in higher education. the tqm magazine, 16(2), 145-159. sakthivel, p. b. and raju, r. (2006a). conceptualizing total quality management in engineering education and developing a tqm educational excellence model. total quality management & business excellence, 17(7), 913–934. sarathy p. s. (2014). tqm practice in real-estate industry using ahp. quality & quantity, 47, 2049-2063. sayeda, b., rajendran, c. and lokachari p.s. (2010). an empirical study of total quality management in engineering educational institutions of india. benchmarking: an international journal, 17(5), 728-767. senthilkumar n. and arulraj a. (2011). sqm-hei – determination of service quality measurement of higher education in india. journal of modelling in management, 6(1), 60-78. shams-ur r., (2004). the future of tqm is past. can tqm be resurrected? total quality management & business excellence, 15(4), 411 – 422. sharma, m. j., moon, i., and bae, h. (2008). analytic hierarchy process to assess and optimize distribution network. applied mathematics and computation, vol. 202 no.1, pp. 256–265. http://link.springer.com/journal/11135 http://link.springer.com/journal/11135 ijahp article: mehta, verma, seth/prioritizing total quality management implementation criterion for engineering education: an analytic hierarchy process analysis international journal of the analytic hierarchy process issn 1936-6744 57 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.197 srikanthan, g. and dalrymple, j. (2003). developing alternative perspectives for quality in higher education. international journal of educational management, 17(3), 126-36. talib f. , rahman z. & qureshi m.n., (2011). prioritising the practices of total quality management: an analytic hierarchy process analysis for the service industries. total quality management & business excellence, 22(12), 13311351. umashankar, v. and dutta, k. (2007). balanced scorecards in managing higher education institutions: an indian perspective. international journal of educational management, 21(1), 54-67. venkatraman s., 2007. a framework for implementing tqm in higher education programs. quality assurance in education. 15(1), 92-112. vidhya, o.s. and kumar, s. (2006). analytic hierarchy process: an overview of application. european journal of operational research, 169(2006), 1-29. yaraliodlu, k. (2001). performans degerlendirmede analitik hiyerars i proses. iktisadi ve idari bilimler fakültesi dergisi, 1, 137-145. yusof n. a. m. and salleh s. h. (2013). analytical hierarchy process in multiple decisions making for higher education in malaysia. procedia social and behavioral sciences, 81, 389–394. zeng, g., jiang, r., huang, g., xu, m., and li, j. (2007). optimization of wastewater treatment alternative selection by hierarch grey relational analysis. journal of environmental management, 82, 250-259. www.aicte-india.org http://www.sciencedirect.com/science/journal/18770428 http://www.sciencedirect.com/science/journal/18770428 http://www.sciencedirect.com/science/journal/18770428/81/supp/c ijahp: ferretti/ outstanding reviewer awards international journal of the analytic hierarchy process 622 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.359 outstanding reviewer awards valentina ferretti, london school of economics and political science, department of management the past two years have been extremely active ones for the ijahp. we have had to address the large number of papers submitted simultaneously to the journal due to fast track submissions derived from the isahp 2014 meeting in washington, d.c. we also moved from publishing the journal twice a year to publishing three times a year. all of these circumstances demanded an extraordinary effort, not just from the editorial team, but also from our reviewers. without reviewers, this journal would not have the proper level of quality required by our readers. therefore, we are grateful to all the ijahp collaborators who anonymously reviewed papers submitted to the journal, and we have decided to thank the best reviewers by celebrating them with an award. unfortunately, we cannot give awards to the dozens of colleagues who have contributed to the journal by being a reviewer, but we hope that our honoured reviewers constitute a representation of all the hard working reviewers who make this journal possible. the nomination process for the award was based on the following characteristics identified by the ijahp editors as key features defining an outstanding reviewer:  thoroughness: the ability to provide feedback that shows a comprehensive understanding of the paper under review.  effectiveness: the ability to provide feedback that serves the purpose of the journal (deciding whether the paper is worth publishing or what is needed for this purpose).  timeliness: the ability to respond with feedback by the agreed upon deadline.  quality of the review: this is probably the most important characteristic of a great reviewer and refers to providing feedback that allows the authors to improve their paper, even in the context of rejection. thus, it is a pleasure to announce the winners of the “outstanding reviewer award” for the 2014‐2015 period! ijahp: ferretti/ outstanding reviewer awards international journal of the analytic hierarchy process 623 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.359 stan lipovetsky sr. research director at gfk custom research north america greater minneapolis-st. paul area “it’s my privilege to help with work for ijahp on behalf of all of the ahp community!” werner toth university of natural resources and life sciences, vienna "i'm very thankful and i feel honored to be receiving this award. it is highly motivating in the early stages of my career." claudio garuti general manager fulcrum ingenieria ltda “i am very happy to help the journal in revising the papers. yet, i never thought about receiving any kind of award (i just tried to do my best). so, i was very surprised (and happy) when i received it!” ozden bayazit professor, central washington university “i am honored to receive this award and happy to be able to contribute to the ijahp in a meaningful way.” manuel serafin plasencia professor, universidad politécnica antonio josé de sucre, unexpo ciudad guayana, estado bolivar, venezuela “this award is really important for me, for my university, and for my fellow students because i'm teaching a msc level course about how to develop a research problem, and this award could serve as an example to them” congratulations to our winners and thank you to all ijahp collaborators! https://www.linkedin.com/company/726364?trk=prof-0-ovw-prev_pos https://www.linkedin.com/company/726364?trk=prof-0-ovw-prev_pos https://www.linkedin.com/company/726364?trk=prof-0-ovw-prev_pos ijahp news: ferretti/isahp 2016 international journal of the analytic hierarchy process 201 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.383 we are looking forward to seeing you at the isahp 2016 symposium which will take place august 4 7, 2016, in london, uk ! the international symposium on the analytic hierarchy process (isahp) for decision making brings together researchers, academics, students and other users of ahp/anp to share their research and experiences in decision-making. the isahp is a biennial conference on multi-criteria decision analysis, with a particular focus on the analytic hierarchy process (ahp) and its extension, the analytic network process (anp), (both developed by thomas l. saaty) and the combination/comparison of these with other methods. our last isahp meeting was held in washington, d.c. in 2014. paper proposals presented at isahp cover the major studies of international research in ahp/anp, and provide solutions for current challenges in important areas of decision-making. of particular interest are the topics listed for the conference as well as scholarship developed by students who constitute the new generation of ahp/anp scholars and practitioners. publication opportunities the following journals will publish a selection of papers from the isahp 2016 conference. selected authors will be invited to submit extended versions of their paper by the person responsible for each special issue, and their papers will be fast-tracked for review. for more information click here. • international journal of information technology & decision making (ijitdm) • journal of multi-criteria decision analysis • journal of systems science and systems engineering (jssse) • journal of environmental accounting and management (jeam) • international journal of clinical engineering and health technology assessment (cehta) • international journal of the analytic hierarchy process (ijahp) http://www.isahp.org/ http://www.isahp.org/callforpapers/?publication-opportunities-7 http://www.worldscientific.com/worldscinet/ijitdm http://onlinelibrary.wiley.com/journal/10.1002/(issn)1099-1360 http://www.rccm.tsinghua.edu.cn/jssse.jhtml https://lhscientificpublishing.com/journals/jeam-default.aspx http://www.cehta.org/index.php/ijcehta http://www.ijahp.org/index.php/ijahp ijahp news: ferretti/isahp 2016 international journal of the analytic hierarchy process 202 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.383 young scholars opportunities isahp 2016 will be a great opportunity for students at the masters and phd level to live the experience of an international conference. they will have the opportunity to exchange ideas and get feedback from the top authors in the ahp/anp field and even have one-on-one time with them in our newly introduced “mentoring sessions”. for these sessions you can book time and discuss specific topics with highly acclaimed experts who are members of our community (for more info click here). unattended poster presentations taking into consideration the difficulties of getting funding for conferences, isahp 2016 has also introduced the unattended poster sessions. this is for scholars who want to submit and send their work, even if it is still in progress, but cannot be present during the conference (student-fee registration will be required). social networking event isahp 2016 offers a variety of events for attendees, such as community meetings and receptions, mentoring sessions and a very nice social networking event that will take place on saturday, august 6, at the dickens inn. dickens inn, traditional english pub the dickens inn is an original warehouse building near tower bridge, tower of london and the shard. as an original warehouse building, it's thought to have housed tea or to have been owned by a local brewery. it certainly existed at the turn of the 18th century and may well have been created in the 1700's. this inn was opened by cedric charles dickens, grandson of the famous author charles dickens. we look forward to see you again, at isahp 2016, august 4 – 7, in london, uk! http://www.isahp.org/ http://www.dickensinn.co.uk/ ijahp news: ferretti/isahp 2016 international journal of the analytic hierarchy process 203 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.383 isahp 2014, washington dc, usa ijahp essay: ishizaka/visual management for decision analysis international journal of the analytic hierarchy process 141 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.475 visual management for decision analysis alessio ishizaka centre for operational research and logistics, portsmouth business school, university of portsmouth, portsmouth po1 3de, united kingdom alessio.ishizaka@port.ac.uk abstract visual management has been recently recognized as an excellent way to convey decisions in a clear and convincing way. ahp already used visual techniques for evaluating the pairwise comparisons and performing a sensibility analysis. in this paper, we introduce a new descriptive tool gaia that can be coupled with ahp in order to visualize the entire problem on a unique plane. https://doi.org/10.13033/ijahp.v9i1.475 keywords: visual management; gaia; descriptive tool 1. introduction it is well-known today that the analytic hierarchy process (ahp) is an extremely useful method. several reviews have compiled their success stories (zahedi 1986; golden, wasil et al. 1989; shim 1989; vargas 1990; saaty & forman 1992; forman & gass 2001; kumar &vaidya 2006; omkarprasad & sushil 2006; ho 2008; liberatore & nydick 2008; sipahi & timor 2010; dung, luan et al. 2016). in this paper, we argue that the usefulness of ahp can be enhanced with visual management techniques. visual techniques have long been used in ahp for evaluating the pairwise comparisons ( figure 1) and performing a sensibility analysis ( figure 2). they have been integrated into the main software that supports ahp, and greatly facilitated the decision-making process (ishizaka & labib 2009). however, visual techniques cannot only facilitate the decision-making process but can also be used as a descriptive tool that explains the whole problem (nemery, ishizaka et al. 2012). in the next section, we present gaia, a method that was first coupled with ahp by ishizaka and siraj et al. (2016). https://doi.org/10.13033/ijahp.v9i1.475 mailto:alessio.ishizaka@port.ac.uk https://doi.org/10.13033/ijahp.v9i1.475 ijahp essay: ishizaka/visual management for decision analysis international journal of the analytic hierarchy process 142 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.475 figure 1. graphical scale figure 2. an example of four possible graphical sensitivity analyses in expert choice 2. graphical analysis for interactive aid (gaia) the idea of gaia is to visualise on a plane as much information as possible related to a problem (mareschal & brans 1988). for this purpose, we can use the dimensionality reduction technique of the principal component analysis (pca). the pca is applied on the local priorities of ahp entered in a matrix. data are displayed on a plane with the two axes having the maximal and next-to-maximal dispersions https://doi.org/10.13033/ijahp.v9i1.475 ijahp essay: ishizaka/visual management for decision analysis international journal of the analytic hierarchy process 143 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.475 (collins, ishizaka et al. 2017). these two axes correspond to the first two principal components. figure 3. gaia plane 3. interpreting the gaia plane an illustrative example of a gaia plane with more than two criteria is given in figure 4, where the criteria are represented by four vectors (see blue arrows emanating from centre) and the alternatives are represented by dots. the decision stick (labelled as dmg) represents the performance direction taking into account all criteria. the reading is done by projection on the relevant arrow. for example, alternative 3 is the best performing alternative overall, but on criterion 4, alternative 1 is the best. an angle between two vectors represents the degree of correlation between the two criteria, i.e. the smaller the angle between the two arrows, the more correlated they https://doi.org/10.13033/ijahp.v9i1.475 ijahp essay: ishizaka/visual management for decision analysis international journal of the analytic hierarchy process 144 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.475 are. for example, criterion 1 and 2 are closely correlated, but criterion 3 and 4 have an almost negative correlation. finally, if alternatives are close, they have a similar level of performance on the different criteria (e.g. alternative 2 and 4). figure 4. example of the obtained gaia graph 4. conclusion a multi-criteria decision making method can only be helpful if the decision-maker understands the reasons underpinning the prescribed results. otherwise, she and/or her team will not accept the results, and it will end in unproductive change management resistance. visual management is a tool that has long been helpful in structuring the problem, eliciting the pairwise comparisons and analyzing the sensitivity of the ranking. this paper presented gaia, which allows for the whole problem to be visualized on a unique page. in my consulting experience, a picture has always been the best way to explain results. i strongly believe that ahp will become even more popular if it is used in correlation with gaia. https://doi.org/10.13033/ijahp.v9i1.475 ijahp essay: ishizaka/visual management for decision analysis international journal of the analytic hierarchy process 145 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.475 references collins, a., a. ishizaka, et al. (2017). film production incentives, employment transformation and domestic expenditure in south africa: visualizing subsidy effectiveness. international journal of cultural policy advance online publications., 1-14. doi: doi.org/10.1080/10286632.2016.1255206. dung, t., n. luan, et al. (2016). the analytic approach in green supplier selection: a literature review. arpn journal of engineering and applied sciences, 11(11), 67546762. forman, e. and s. gass (2001). the analytic hierarchy process – an exposition. operations research, 49(4), 469-486. doi: http://dx.doi.org/10.1287/opre.49.4.469.11231 golden, b., e. wasil, et al. (1989). the analytic hierarchy process: applications and studies. heidelberg: springer-verlag. doi: 10.1007/978-3-642-50244-6_3 ho, w. (2008). integrated analytic hierarchy process and its applications a literature review. european journal of operational research, 186(1), 211-228. doi: http://dx.doi.org/10.1016/j.ejor.2007.01.004 ishizaka, a. and a. labib (2009). analytic hierarchy process and expert choice: benefits and limitations. or insight, 22(4), 201–220. doi: 10.1057/ori.2009.10 ishizaka, a., s. siraj, et al. (2016). which energy mix for the uk? an evolutive descriptive mapping with the integrated gaia-ahp visualisation tool. energy 95, 602–611. doi: http://dx.doi.org/10.1016/j.energy.2015.12.009 kumar, s. and o. vaidya (2006). analytic hierarchy process: an overview of applications. european journal of operational research, 169(1), 1-29. doi: http://dx.doi.org/10.1016/j.ejor.2004.04.028 liberatore, m. and r. nydick (2008). the analytic hierarchy process in medical and health care decision making: a literature review. european journal of operational research, 189(1), 194-207. doi: http://dx.doi.org/10.1016/j.ejor.2007.05.001 mareschal, b. and j.-p. brans (1988). geometrical representations for mcda. european journal of operational research, 34(1), 69-77. doi: https://doi.org/10.1016/0377-2217(88)90456-0 nemery, p., a. ishizaka, et al. (2012). enriching descriptive information in ranking and sorting problems with visualizations techniques. journal of modelling in management, 7(2), 130-147. doi: http://dx.doi.org/10.1108/17465661211242778 omkarprasad, v. and k. sushil (2006). analytic hierarchy process: an overview of applications. european journal of operational research, 169(1), 1-29. doi: http://dx.doi.org/10.1016/j.ejor.2004.04.028 saaty, t. and e. forman (1992). the hierarchon: a dictionary of hierarchies. pittsburgh: rws publications. doi: 10.1007/978-1-4612-5443-0_11 https://doi.org/10.13033/ijahp.v9i1.475 http://dx.doi.org/10.1287/opre.49.4.469.11231 http://dx.doi.org/10.1016/j.ejor.2007.01.004 http://dx.doi.org/10.1016/j.energy.2015.12.009 http://dx.doi.org/10.1016/j.ejor.2004.04.028 http://dx.doi.org/10.1016/j.ejor.2007.05.001 https://doi.org/10.1016/0377-2217(88)90456-0 http://dx.doi.org/10.1016/j.ejor.2004.04.028 ijahp essay: ishizaka/visual management for decision analysis international journal of the analytic hierarchy process 146 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.475 shim, j. (1989). bibliography research on the analytic hierarchy process (ahp). socio-economic planning sciences 23(3), 161-167. doi: https://doi.org/10.1016/0038-0121(89)90013-x sipahi, s. and m. timor (2010). the analytic hierarchy process and analytic network process: an overview of applications. management decision, 48(5), 775-808. doi: http://dx.doi.org/10.1108/00251741011043920 siraj, s., l. mikhailov, et al. (2013). priest: an interactive decision support tool to estimate priorities from pairwise comparison judgments. international transactions in operational research, 22(2), 217–235. doi: 10.1111/itor.12054 vargas, l. (1990). an overview of the analytic hierarchy process and its applications. european journal of operational research, 48(1), 2-8. doi: https://doi.org/10.1016/0377-2217(90)90056-h zahedi, f. (1986). the analytic hierarchy process: a survey of the method and its applications. interface, 16(4), 96-108. doi: http://dx.doi.org/10.1287/inte.16.4.96 https://doi.org/10.13033/ijahp.v9i1.475 https://doi.org/10.1016/0038-0121(89)90013-x https://doi.org/10.1016/0377-2217(90)90056-h http://dx.doi.org/10.1287/inte.16.4.96 7 final news herbert simon award final paper ijahp news: ahp paper on the possibility of group welfare functions wins herbert simon award international journal of the analytic hierarchy process 87 vol. 4 issue 1 2012 issn 1936-6744 ahp paper on the possibility of group welfare functions wins herbert simon award warm congratulations to luis vargas (top) and thomas saaty who won the 2011 herbert simon award for outstanding contribution in information technology and decision making for their paper “the possibility of group welfare functions” published in the international journal of information technology & decision making (ijitdm), 2005, vol. 04, issue 02, pages 167176. the paper was ranked the first in overall score. ijitdm formally announced this news in its vol. 11, issue 1 (jan. 2012). this year, the award committee evaluated 30 papers that were published in ijitdm during 2002-2010 for the award according to the following criteria: 1. significance of the paper to the field of information technology decision making; 2. the extent to which the paper is interesting; 3. records of academic citations; 4. theoretical contribution to itdm; 5. use of appropriate methodological rigor; 6. clarity of writing and/or presentation ijitdm has a new isi web of knowledge impact factor of 3.139, ranking of 7th out of 108 in the categories of computer science and artificial intelligence, 5th out of 126 in computer science and information systems, 7th out of 97 in computer science and interdisciplinary application, and 3rd out of 74 in operations research & management science. rob typewritten text http://dx.doi.org/10.13033/ijahp.v4i1.140 rob typewritten text ijahp article: ali, asghar, muhammad/selection of a fighter aircraft to improve the effectiveness of air combat in the war on terror: pakistan air forcea case in point international journal of the analytic hierarchy process 244 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.489 selection of a fighter aircraft to improve the effectiveness of air combat in the war on terror: pakistan air force a case in point yousaf ali, phd assistant professor of operation management, department of management sciences, gik institute of engineering sciences and technology, pakistan yousafkhan@giki.edu.pk ali asghar muzzaffar faculty of mechanical engineering ghulam ishaq khan institute of engineering sciences and technology pakistan aliasgharmuzzaffar@gmail.com noor muhammad, phd assistant professor of entrepreneurship, department of management science, gik institute of engineering sciences and technology, pakistan noormuhammad@giki.edu.pk aneel salman assistant professor department of management science comsats, institute of information technology islamabad, pakistan aneel.salman@comsats.edu.pk abstract the selection of military aircraft, by nature, is a process consisting of conflicting goals and objectives at the conceptual, preliminary, and detailed level. in order to ease the process of making decisions wisely from a varied group of options available, multiple criteria decision making (mcdm) methods are applied effectively. a scenario is put forth pertaining to defense acquisition, when a contemporary air force needs to select and add new and better fighter aircrafts to their pre-existing fleets. this paper studies the pakistan air force (paf) and its goal to improve its aerial defense and precise ground strike capabilities. moreover, this paper aims to help raise the bar of general aerial defense and counter terrorism operations. this research paper also sets an appropriate methodological approach for defense procurement and the fleet up-gradation planning process via the use of the analytic hierarchy process (ahp), an mcdm technique. furthermore, this study specifically focuses on a set of ten technical and economic criteria, applied over six alternative aircraft while, keeping in mind, the counter-insurgency and aerial defense requirements of paf. lastly, a cost benefit analysis (cba) has been applied to ensure that the selected alternative is in line with the economic constraints faced by the limited fiscal budget of pakistan. mailto:yousafkhan@giki.edu.pk mailto:aliasgharmuzzaffar@gmail.com mailto:noormuhammad@giki.edu.pk mailto:aneel.salman@comsats.edu.pk ijahp article: ali, asghar, muhammad/selection of a fighter aircraft to improve the effectiveness of air combat in the war on terror: pakistan air forcea case in point international journal of the analytic hierarchy process 245 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.489 keywords: multi-criteria decision making; analytical hierarchy process; cost benefit analysis; efficiency cost indicator 1. introduction the definition of a "good" overall design for a military weapon system almost always depends on one’s point of view. from a performance standpoint, necessary important phases are required for an advanced multirole strike aircraft. for example, significant ground attack capability exists in the form of delivering a maximum payload to a target at an appropriate range including some air-to-air capability at certain machaltitude combinations. the resulting size and geometry of such an aircraft, however, may result in a poor level of survivability due to increases in their radar and infrared signature. further, passive improvements in these signatures may dramatically drive up the aircraft cost. it is predicted that there will be 5% growth per year in commercial aviation over the next 20-25 years (palut & canziani, 2007). usually, most aircraft are expected to have a service time of 30 years or longer, however, there are various uncertainties that can affect the aircraft’s feasibility and viability during its service period. for example, fuel prices are always fluctuating and this influences the viability of the aircraft. with advancing technology and political difficulties in the world, tensions between nations are also on the rise. not only this, but the contours of war have changed over time too. hence, instead of decisive battles, wars are becoming asymmetric and are being fought under unpredictable circumstances. the enemy is no longer well-defined and the frontlines are not marked. therefore, armies in great numbers are not sufficient to win wars; instead, it is technology, strategy and diplomacy that can lead to winning wars in this 21 st century. because of the existence of weapons capable of mass destruction, nations cannot afford to go for an all-out war. thus, to counter these changes and difficulties, surgical strikes are a rapid option that is available with precise measures and policies. at the center of modern warfare lies a country’s air force. the air force of a nation serves both as its sword and its shield; thereby, giving a nation the capability to strike its enemy a decisive blow, while protecting it from both retaliation and aggression. pakistan is a vulnerable country, easily targeted and prone to attacks. this is evidenced by the fact that just beyond pakistan’s eastern front, lays its neighbor and traditional rival india, with which it has fought four wars in history. moreover, within its borders pakistan is faced by the threat of growing radical extremism. being a nuclear armed nation of 182.1 million 1 , it needs to maintain a strong military force to tackle these continuous security challenges. the most crucial asset for pakistan, in the current era, has been its air force. over the span of time, pakistan has faced global sanctions, which has made it unable to upgrade its fleet of aircraft. at present, the pakistan air force is facing a widespread challenge in the form of radical extremism. this is known to mostly originate from the remote and rugged terrain of the north west frontier that is inaccessible with rapid-response forces at the ground level. to counter this issue, the paf needs efficient state-of-the-art aircraft, equipped and fully competent with the latest precision strike technology for 1 the world bank 2015 global census data (http://data.worldbank.org/indicator/sp.pop.totl) ijahp article: ali, asghar, muhammad/selection of a fighter aircraft to improve the effectiveness of air combat in the war on terror: pakistan air forcea case in point international journal of the analytic hierarchy process 246 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.489 targeting its enemies with pin-point accuracy and minimum collateral damage. these aircraft will also be useful in the fleets responsible for countering international security threats. in general, selection of a combat aircraft is a very complicated matter that is governed by economic, technical and geopolitical constraints. in addition to this, it is necessary for an air force to adhere to its operational and ever-changing defense requirements. therefore, strategic planning is critical in making an efficient and particular choice. in the literature review, the aircraft selection process has been performed in various ways, but specifically focuses on civilian applications in which the issue in question is relatively well-explained. however, a novel approach is required when considering military applications. the implicit purpose of this research paper is to solve the issue regarding selection of aircraft in the pakistan air force. this involves replacing its increasingly aging fleet and matching its requirements to counter terrorist operations while keeping in mind the diplomatic and economic constraints. it is essential not to forget that the problem is the inherent multi-criteria decision making. the analytic hierarchy process (ahp) is used for this purpose to select an appropriate aircraft. ahp has been successfully applied in resource allocation and forecasting. since aircraft selection is a process which is closely connected to these areas, the use of ahp is reasonable in this case. the advantage of this decision support tool is that the final ranking is obtained on the basis of pair-wise assessment between the criteria and the alternatives, both of which are selected as part of this study. additionally, the ahp approach is engaged because its algorithm is rational and easily comprehensible. further, to ensure that the economic constraints are well accounted for a cost benefit analysis was implemented to the periodic operational and maintenance costs involved in addition to the initial investment. this paper is organized in the following order: literature review, methodology, conclusion, discussion of results and future considerations. 2. literature review the analytical hierarchy process was introduced by thomas saaty (saaty, 1990). this method has been applied specifically in various situations, ranging from comprehensive economic studies to very critical defense acquisitions. it has been repeatedly applied on cases pertaining to airline aircraft acquisitions with studies focused on civilian travelling requirements (bhadra, 2003; harasani, 2006; harasani, 2013). it has been used successfully as a measure of decision-making in procurement of defense assets and has been asserted to be a suitable decision making tool for defense acquisitions (tsagdis, 2008). henceforth, the aforementioned reference emphasizes the fact that the complexity of the decisions involved in defense asset procurement are due to social disapproval of budget allocation for the defense sector. this further reflected in the solution to this problem by considering the cause, which is the variety of criteria ranging from technical to socio-economic factors. multi-criteria decision making (mcdm) is a process that allows one to make decisions in the presence of multiple, potentially conflicting criteria (hwang & yoon, 1981; sen and yang, 1998). furthermore, a study that applies the technique for order performance by similarity to ideal solution (topsis) to allow for the selection of an optimal training aircraft in an uncertain environment has been further ijahp article: ali, asghar, muhammad/selection of a fighter aircraft to improve the effectiveness of air combat in the war on terror: pakistan air forcea case in point international journal of the analytic hierarchy process 247 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.489 probed in wang and chang (2007). ahp has also been used in ill-defined environments with a comparison of the results, attained via topsis, to propose a solution to the air combat effectiveness assessment problem; thus framing it as a multi-criteria decision making (mcdm) case (wang et al., 2008a,b). a study based on a certain cost benefit analysis application was considered where the said method had been used to allow for the addition of a new fighter aircraft in the hellenic air force (kyriazis & salavrakos, 2006). the study, closest to the one presented by the authors of this paper, is the evaluation of military training aircrafts through the combination of multi-criteria decision making with ambiguous logic looking at the spanish air force (sánchez-lozano et al., 2015). hence, this research paper aims to associate the weights to the criteria using ahp and further evaluate it using the topsis method. it is essential to know that the data set being addressed has not been studied to date. perhaps, these criteria have been used for the purposes of pakistan air force; however, they have not yet been applied and inspected as a measure of asset procurement by academia. with no more than nine, totally independent alternatives, the usage of ahp may allow for excellent results (salomon & montevechi, 2001). this assertion is strongly supported by another study which lays down a comparison between ahp and anp as options of applicative methods. the study also determines how the complex inter-relation between criterion and alternatives leads to the application of anp in a certain scenario (büyükyazıcı &sucu, 2003). however, the application of anp, though an exception that it is applicable in our present case, may cause complications in data collection through surveying in addition to the error due to incorrect perception of the questionnaire by the respondent. saaty (1980), in his research claimed that if we decide on the available options intuitively we may acquire misleading results since the larger the quantity of options the better the ahp performs. ahp is most useful where teams of people are working on complex problems, especially those with high stakes, involving human perceptions and judgments, whose resolutions have long-term repercussions. it has unique advantages when important elements of the decision are difficult to quantify or compare, or where communication among team members is impeded by their different specializations, terminologies, or perspectives (stewart & belton, 2002). at its simplest, mcdm is a shelter that has all major formal methodologies that require many criteria when making a decision by individuals or even by groups. it originates from operational research and supports a single decision maker making an appropriate decision (martins & mendoza, 2006). hence, the ahp proves to be an important technique in this scenario. while making a decision we need to focus on the purpose of the decision, all the sub criteria for the decision making process and the groups affected by them. ahp is used extensively by leading organizations across the globe (bhushan & rai, 2007). the department of defense in the united states uses ahp frequently and extensively to make major decisions including how to distribute their resources across diversified activity areas (forman & gass, 2001). similarly, the general service department (gsa) in the u.s used the ahp to decide what kind of new technology initiatives they needed in order to meet the increasing demands while at the information technology conference (udo, 2000). apart from governmental organizations, there are many successful examples of ahp use in leading multinationals operating across the globe (saaty, 2008). in 1998, british airways employed this method with their ijahp article: ali, asghar, muhammad/selection of a fighter aircraft to improve the effectiveness of air combat in the war on terror: pakistan air forcea case in point international journal of the analytic hierarchy process 248 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.489 board of directors to determine which entertainment system vendor would be the optimum one for their entire fleet of planes (saaty, 2004). xerox corporation used ahp to allocate their billion dollars of funds in research and development projects (dey, 2002). in 1999, ford motor company used ahp to design a scale for the satisfaction of customers based on the priorities (saaty, 2008). all of these examples support the reliability of the use of this technique in fighter aircraft selection to improve the effectiveness of air combat in the war on terror. thus, ahp provides a convenient solution to be applied on the data set. moreover, it may be asserted that though publications pertaining to defense assets have been previously published in large numbers, among the ones focusing on aerial defense in the context of cost-benefit analysis assessment, ahp is not highly represented. it may be further noticed that works pertaining to the effectiveness of the air force in the war against terrorism remain alarmingly low and thus, it was difficult to review the literature in this particular context. 3. methodology 3.1 analytical hierarchy process the analytical hierarchy process is a powerful and flexible multi-criteria decision making tool (mcdm), introduced and developed by saaty (1990). it involves pairwise comparisons between a set of alternatives for each criterion. comparisons are made using a scale of absolute judgments (1, 3, 5, 7, 9), as well as intermediate values between the two judgments that represents the relative measure of one alternative over another with respect to a given criteria (dožić & kalić, 2015). by reducing complex decisions to a series of simple comparisons and rankings and then synthesizing the results, the ahp not only helps the analysts arrive at the best decision, but also provides a clear rationale for the choices made (wang et al., 2008a). its main steps include: 1. statement of the goal, decision criteria and alternatives. 2. development of a pair wise comparison matrix. 3. development of a standardized/normalized matrix. 4. development of a priority vector. 5. computation of the consistency ratio which should be less than 0.1. 6. development of a priority matrix. after steps 2 through 5 have been performed for each criterion, the results of step 4 are summarized in a priority matrix by listing the decision alternatives vertically and the criteria horizontally. the column entries are the priority vectors for each criterion. 7. development of a criteria pair wise development matrix. 8. development of an overall priority vector. multiplying the criteria priority vector (from step 7) by the priority matrix (from step 6) may then be used to determine the overall ranking of alternatives (step 8). 9. choosing the alternative with the highest rank. in this study, the goal was to select the best combat aircraft for paf that has optimal precision striking capability within a limited budget. wang and chang (2007) proposed a systematic evaluation model to help the air force academy with selection of an optimal training aircraft, mainly from the perspective of pilot drillmasters and trainees (dožić & kalić, 2014). figure 1 explains the flow of the research. ijahp article: ali, asghar, muhammad/selection of a fighter aircraft to improve the effectiveness of air combat in the war on terror: pakistan air forcea case in point international journal of the analytic hierarchy process 249 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.489 figure 1. flow chart of the research there are three levels of hierarchy and at level one the main goal or objective is to select a fighter aircraft to meet the counterinsurgency and air superiority requirements of the pakistan air force. as such, a similar model, based on the literature review and expert opinions (as shown in appendix a) was implemented, which made use of a set of the ten most critical and relevant multiple criteria, namely: service ceiling, maximum take-off weight (mtow), precision target capability, cruising speed, maneuverability, acquisition cost, operation cost, maintainability and availability. these are at level 2 of the hierarchy. of all the current modern combat aircraft, six were shortlisted as decision alternatives, keeping the critical technological requirements of precision target capability and political and financial constraints in view. these are at level three of the hierarchy. the importance of placing political constraints became apparent after a recent occurrence when the u.s. congress rejected partially financing the sale of eight f-16 aircraft through the foreign military financing program (syed, 2016). the alternatives chosen were: dassault rafale, saab jas 39 gripen, mikoyan mig-35, sukhoi su-35, chengdu j-10 and pac jf-17 thunder. the hierarchy structure model of the problem is shown in figure 2. figure 2. hierarchy structural model ijahp article: ali, asghar, muhammad/selection of a fighter aircraft to improve the effectiveness of air combat in the war on terror: pakistan air forcea case in point international journal of the analytic hierarchy process 250 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.489 in order to develop a pair wise comparison matrix for each criteria (table 1), a rigorous literature review using manufacturer’s web sites and research articles was done in order to collect the relevant technical specifications of each alternative. those specifications were used to assign values, on the scale of 1 to 9, in pair wise comparison matrix for each criterion. a microsoft excel template for carrying out computations was used (pyzdek, 2014). through this template, a normalized matrix (table 2), priority vector (table 3) and consistency ratio for each criterion was calculated. ijahp article: ali, asghar, muhammad/selection of a fighter aircraft to improve the effectiveness of air combat in the war on terror: pakistan air forcea case in point international journal of the analytic hierarchy process 251 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.489 table 1 pair wise comparison matrix for service ceiling cr value = 0.004 < 0.1 pair wise comparisons item number 1 2 3 4 5 6 item description dassau lt rafale saab jas 39 gripen mikoya n mig35 sukhoi su-35 chengd u j-10 pac jf17 thunder dassault rafale 1.00 1.00000 0.20000 0.2000 0 0.1666 7 0.33333 saab jas 39 gripen 1.00 1.00 0.20000 0.2000 0 0.1666 7 0.33333 mikoyan mig-35 5.00 5.00 1.00 1.0000 0 0.3333 3 3.00000 sukhoi su-35 5.00 5.00 1.00 1.00 0.3333 3 5.00000 chengdu j-10 6.00 6.00 3.00 3.00 1.00 7.00000 pac jf17 thunder 3.00 3.00 0.33 0.20 0.14 1.00 sum 21.00 21.00 5.73 5.60 2.14 16.67 table 2 standardized matrix for service ceiling standardized matrix dassault rafale saab jas 39 gripen mikoya n mig35 sukh oi su35 chengd u j-10 pac jf17 thund er dassault rafale 0.05 0.05 0.03 0.04 0.08 0.02 saab jas 39 gripen 0.05 0.05 0.03 0.04 0.08 0.02 mikoyan mig-35 0.24 0.24 0.17 0.18 0.16 0.18 sukhoi su-35 0.24 0.24 0.17 0.18 0.16 0.30 chengdu j-10 0.29 0.29 0.52 0.54 0.47 0.42 pac jf17 thunder 0.14 0.14 0.06 0.04 0.07 0.06 table 3 priority vector for service ceiling alternatives priority vector dassault rafale 0.043935647 saab jas 39 gripen 0.043935647 ijahp article: ali, asghar, muhammad/selection of a fighter aircraft to improve the effectiveness of air combat in the war on terror: pakistan air forcea case in point international journal of the analytic hierarchy process 252 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.489 mikoyan mig-35 0.194122677 sukhoi su-35 0.214122677 chengdu j-10 0.41951089 pac jf-17 thunder 0.084372462 similar computations were carried out for each of the ten aforementioned criteria and the priority vectors for each criterion were obtained and combined in the form of a priority matrix as shown in table 4. conspicuously, the table shows the essential factors while carrying out these calculations for the six mentioned alternatives. this way it is easier to figure out which aircraft would be the most suitable amongst all or which particular feature stands out among other characteristics. for example, from the table we can perceive that the sukhoi su-35 has the highest precision target capability rate as compared to the other alternative aircraft, which is estimated to be 0.444 respectively. the data has been compiled in this table which will then shape into a priority matrix. ijahp article: ali, asghar, muhammad/selection of a fighter aircraft to improve the effectiveness of air combat in the war on terror: pakistan air forcea case in point international journal of the analytic hierarchy process 253 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9.i1.489 table 4 priority matrix acquisition cost operation cost cruising speed precision target capability combat radius mtow service ceiling maneuverability availability maintainability dassault rafale 0.024853892 0.042068137 0.043859165 0.257950168 0.040440475 0.165533541 0.043935647 0.106139709 0.055040292 0.117350003 saab jas 39 gripen 0.059670772 0.155277281 0.137430675 0.025935677 0.244689405 0.042697225 0.043935647 0.030107595 0.033615004 0.162488892 mikoyan mig-35 0.152929786 0.080478493 0.48754419 0.082711433 0.038079892 0.253810553 0.194122677 0.353267989 0.345335425 0.064605294 sukhoi su-35 0.072226552 0.025121824 0.146802014 0.444426704 0.405517136 0.434505123 0.214122677 0.358476322 0.345335425 0.051055158 chengdu j10 0.2555856 0.27353918 0.146802014 0.141423021 0.051715962 0.077086391 0.41951089 0.121764709 0.110336926 0.209821978 pac jf17 thunder 0.434733398 0.423515086 0.037561942 0.047552996 0.21955713 0.026367167 0.084372462 0.030243676 0.110336926 0.394678676 ijahp article: ali, asghar, muhammad/selection of a fighter aircraft to improve the effectiveness of air combat in the war on terror: pakistan air forcea case in point international journal of the analytic hierarchy process 254 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.489 table 5 criteria pair wise development matrix service ceiling mtow ptc combat radius cruising speed maneuverability acquisition cost operation cost maintainability availability service ceiling 1 3 3 3 5 2 2 1 3 2 mtow (1/3) 1 2 4 4 2 2 3 2 1 ptc (1/3) (1/2) 1 6 7 6 5 4 4 5 combat radius (1/3) (1/4) (1/6) 1 5 5 (1/5) (1/3) 5 3 cruising speed (1/5) (1/4) (1/7) (1/5) 1 3 2 4 4 3 maneuverability (1/2) (1/2) (1/6) (1/5) (1/3) 1 5 4 5 4 acquisition cost (1/2) (1/2) (1/5) 5 (1/2) (1/5) 1 5 7 4 operation cost 1 (1/3) (1/4) 3 (1/4) (1/4) (1/5) 1 6 4 maintainability (1/3) (1/2) (1/4) (1/5) (1/4) (1/5) (1/7) (1/6) 1 5 availability (1/2) 1 (1/5) (1/3) (1/3) (1//4) (1/4) (1/4) (1/5) 1 ijahp article: ali, asghar, muhammad/selection of a fighter aircraft to improve the effectiveness of air combat in the war on terror: pakistan air forcea case in point international journal of the analytic hierarchy process 255 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.489 in order to acquire data to develop a criteria pair wise development matrix, a questionnaire was carefully designed and dispatched to aviation experts comprised of on-duty, retired paf officers and aeronautical engineers of varying ranks. refer to appendix b for the questionnaire. the challenge was to combine all the questionnaire responses into a single equivalent response so that its data could be used to assign the values, on the scale of 1 to 9, in pair wise comparison matrix between criteria. for each pair wise comparison between one criteria, e.g. between service ceiling and precision target capability, the number of responses for each scale value was recorded and plotted on a histogram as shown in figure 3. figure 3. histogram showing response frequency for service ceiling to precision target capability a weighted arithmetic mean was calculated to define a scale value for that particular pair wise comparison. because a weighted arithmetic mean is based on all the observations, determined for almost every kind of data, it is least affected by fluctuations of sampling and is finite and not indefinite. only the scale values with responses greater than one were considered in the computation of the mean. the mean was chosen as a measure of central tendency to eliminate the error due to incorrect questionnaire perception by the respondent. the expression to evaluate the mean is stated as follows: weighted arithmetic mean = ( ∑(scale value × response frequency) sum of acceptible response frequencies ) for the histogram of pair wise comparison between service ceiling and precision target capability, shown in figure 1, the sample calculation is as follows: weighted arithmetic mean = ( (1 × 3) + (3 × 9) 3 + 9 ) = 3 (to the nearest unit) this procedure was adopted and applied exhaustively for each pair wise comparison as a result of which a criteria pair wise development matrix was developed as shown in tables 5 and 6. 3 9 0 0 1 0 2 4 6 8 10 1 3 5 7 9 response frequency scale value service ceiling to precision target capability ijahp article: ali, asghar, muhammad/selection of a fighter aircraft to improve the effectiveness of air combat in the war on terror: pakistan air forcea case in point international journal of the analytic hierarchy process 256 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.489 moreover, table 5 depicts that the factual data has been computed and blended in a pair mode to form a development matrix of the above mentioned criteria, giving us a clear view of the facts. table 6 also contains a section of sum total of all the descriptive values significantly. the highest value of the sum total is seen to be under the maintainability column which shows a value of 37.20. the table also highlights those figures which are above the range of the staircase of 1.00 figures. additionally, the lowest value of the sum of 5.03 is seen under the service ceiling column. table 6 criteria pair wise development matrix in ms excel template cr value = 0.048 < 0.1 pair wise comparisons item number 1 2 3 4 5 6 7 8 9 10 item description service ceiling mtow ptc combat radius cruising speed maneuverability acquisition cost operation cost maintainability availability service ceiling 1.00 3.000 3.000 3.000 5.000 2.000 2.000 1.000 3.000 2.000 mtow 0.33 1.00 2.000 4.000 4.000 2.000 2.000 3.000 2.000 1.000 ptc 0.33 0.50 1.00 6.000 7.000 6.000 5.000 4.000 4.000 5.000 combat radius 0.33 0.25 0.17 1.00 5.000 5.000 0.200 0.333 5.000 3.000 cruising speed 0.20 0.25 0.14 0.20 1.00 3.000 2.000 4.000 4.000 3.000 maneuverability 0.50 0.50 0.17 0.20 0.33 1.00 5.000 4.000 5.000 4.000 acquisition cost 0.50 0.50 0.20 5.00 0.50 0.20 1.00 5.000 7.000 4.000 operation cost 1.00 0.33 0.25 3.00 0.25 0.25 0.20 1.00 6.000 4.000 maintainability 0.33 0.50 0.25 0.20 0.25 0.20 0.14 0.17 1.00 5.000 availability 0.50 1.00 0.20 0.33 0.33 0.25 0.25 0.25 0.20 1.00 sum 5.03 7.83 7.38 22.93 23.67 19.90 17.79 22.75 37.20 32.00 a microsoft excel template for carrying out computations was used (pyzdek, 2014). through this template, a normalized matrix (table 7), priority vector (table 8) and consistency ratio was calculated. the consistency ratio determined was less than one; hence, the degree of consistency was acceptable. the following table clearly displays how the template was designed to add the figures and information. the table shows that the service ceiling value under the ptc column was up to 0.41, thus making it the highest value in the table, while 0.01 is the most frequent and lowest value shown in the matrix. the table has a number of uniform values which pointedly shows the consistency. ijahp article: ali, asghar, muhammad/selection of a fighter aircraft to improve the effectiveness of air combat in the war on terror: pakistan air forcea case in point international journal of the analytic hierarchy process 257 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.489 table 7 standardized matrix for criteria service ceiling mtow ptc combat radius cruising speed maneuverability acquisition cost operation cost maintainability availability service ceiling 0.20 0.38 0.41 0.13 0.21 0.10 0.11 0.04 0.08 0.06 mtow 0.07 0.13 0.27 0.17 0.17 0.10 0.11 0.13 0.05 0.03 ptc 0.07 0.06 0.14 0.26 0.30 0.30 0.28 0.18 0.11 0.16 combat radius 0.07 0.03 0.02 0.04 0.21 0.25 0.01 0.01 0.13 0.09 cruising speed 0.04 0.03 0.02 0.01 0.04 0.15 0.11 0.18 0.11 0.09 maneuverability 0.10 0.06 0.02 0.01 0.01 0.05 0.28 0.18 0.13 0.13 acquisition cost 0.10 0.06 0.03 0.22 0.02 0.01 0.06 0.22 0.19 0.13 operation cost 0.20 0.04 0.03 0.13 0.01 0.01 0.01 0.04 0.16 0.13 maintainability 0.07 0.06 0.03 0.01 0.01 0.01 0.01 0.01 0.03 0.16 availability 0.10 0.13 0.03 0.01 0.01 0.01 0.01 0.01 0.01 0.03 as the table shows the finalized calculated figures of priority vectors, we can see that the availability and maintainability values are the lowest total values in comparison to other factors. moreover, the precision target capability priority vector values are estimated to be the highest amongst all as shown in the table. table 8 priority vector for criteria criteria priority vector service ceiling 0.173045816 mtow 0.123824884 ptc 0.184514908 combat radius 0.08809149 cruising speed 0.078225126 maneuverability 0.097506387 acquisition cost 0.102863666 operation cost 0.07705485 maintainability 0.039176898 availability 0.035695973 ijahp article: ali, asghar, muhammad/selection of a fighter aircraft to improve the effectiveness of air combat in the war on terror: pakistan air forcea case in point international journal of the analytic hierarchy process 258 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.489 in the end, the overall priority matrix was developed accordingly as shown in table 9. this matrix was necessary in order to deduce a final conclusion pertaining to aircraft alternatives and their rankings. table 9 overall priority vector and ranking of results alternatives overall priority ranking dassault rafale 0.078673059 6 saab jas 39 gripen 0.09445625 5 mikoyan mig-35 0.224425901 1 sukhoi su-35 0.220934651 2 chengdu j-10 0.193553652 3 pac jf-17 thunder 0.187956487 4 the results, as shown in table 9, are that the top three choices should be mig-35, su35 and j-10. furthermore, a cost benefit analysis was applied to counter examine these results. the table depicts the ranking under the grading scale ranging from 1-10 with 1 being the highest value and top rank in the scale to 10 being the lowest and least ranked amongst the choices available. thus, the top ranked alternatives are mig-35, su-35 and j-10 which received the rankings of 1, 2, and 3 respectively. while on the other hand, dassault rafale received the lowest rank with a value of 6, which is least likely to be selected as a suitable alternative. therefore, this table gives a precisely perfect idea of the final results in order to determine the final results, a cost benefit analysis methodology was carefully administrated. 3.2 cost benefit analysis cost benefit analysis (cba) deals with an inspection of a certain process to be performed in order to verify if carrying out such an activity is economically viable. the process generally involves an inspection of various indirect and direct costs incurred on the data. following the determination of costs, any values of tangible and intangible benefits are gathered and assigned numeric values. these values are then gathered and compared with each other to yield a comparison and determine whether the benefits attained at a certain expense yield a project that is economically viable or not. further study on the cba may involve looking at a payback period. this would evaluate the benefits of the cost and see if during that period whether the investment on a project would equal the benefits that are being strived for and would from then onwards, convert the net cash flow on the project to benefits for the investor. that being said, a similar analysis has been performed earlier as a study (kyriazis & salavrakos, 2006). the cba presented in this paper was comparable to the study done for hellenic air force. however, due to the similar nature of the problem being dealt with in this scenario, the strategy of inspection was kept similar to the one proposed in the earlier study. since the study required the execution of an exact cost benefit analysis, it ijahp article: ali, asghar, muhammad/selection of a fighter aircraft to improve the effectiveness of air combat in the war on terror: pakistan air forcea case in point international journal of the analytic hierarchy process 259 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.489 could become intricately complex owing to the indefinite nature of military, strategic, economic and social benefits. therefore, the study was redefined to adopt a factor of efficiency-cost indicator (eci) which was calculated for every fighting aircraft as conducted in the aforementioned study. once again, the purpose of the study is to conduct a verification of the results being obtained by the ahp and as such, a detailed study of the cba, though it may be recommended for future considerations, would not allow for an efficient final result in this study. efficiency cost indicator = ( efficiency score) cost ) the value of the efficiency score was determined on the basis of a selected set of technical characteristics and relative weights associated to each characteristic. the weights were given numeric values based on the responses from the surveys that were distributed to various aeronautics and air defense officials, while the ratings were determined on the basis of a comparative study conducted on the basis of a detailed literature review of the technical specifications and comparative data of each aircraft. the total weighted score or the efficiency score for each aircraft was determined as the sum of weighted scores. these scores were obtained as a product of the weight attributed to each technical criteria and the rating each aircraft attained in the said criteria. efficiency scores for each alternative are shown in table 10. the table illustrates the measure of efficiency of each alternative in a percentage rating. it is evident from the table and figures that the total efficiency score with regards to weight is 25. moreover, the combat radius is only 50% with regards to weighted rating which is the lowest among other factors. the cost was determined as a sum of acquisition cost and operation cost. the operation cost of each aircraft was evaluated on a steady run of the aircraft at standard conditions for a single hour of operation. this data was obtained from the technical manuals of the aircraft manufacturing firms. the evaluation of costs is shown in table 11. this table shows the tallied costs in usd of the alternative aircrafts respectively. as shown in the table, the highest total cost is of the dassault rafale aircraft which is measured as 130.028 million usd. the table also shows that the lowest total price is of the jf-17 thunder, which is astonishing. ijahp article: ali, asghar, muhammad/selection of a fighter aircraft to improve the effectiveness of air combat in the war on terror: pakistan air forcea case in point international journal of the analytic hierarchy process 260 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.489 table 10 efficiency scores for each alternative criteria w e ig h t dassault rafale saab jas 39 gripen mikoyan mig-35 sukhoi su-35 chengdu j-10 pac jf17 thunder r a tin g w e ig h te d sc o re r a tin g w e ig h te d sc o re r a tin g w e ig h te d sc o re r a tin g w e ig h te d sc o re r a tin g w e ig h te d sc o re r a tin g w e ig h te d sc o re service ceiling 2 70% 1.4 70% 1.4 90% 1.8 90% 1.8 100 % 2 60% 1.2 mtow 3 70% 2.1 40% 1.2 80% 2.4 100% 3 50% 1.5 30% 0.9 ptc 5 80% 4 40% 2 70% 3.5 100% 5 50% 2.5 20% 1 combat radius 4 50% 2 90% 3.6 70% 2.8 100% 4 50% 2 90% 3.6 cruising speed 3 70% 2.1 90% 2.7 100% 3 90% 2.7 90% 2.7 70% 2.1 maneuverability 1 80% 0.8 60% 0.6 100% 1 100% 1 80% 0.8 50% 0.5 maintainability 4 80% 3.2 80% 3.2 70% 2.8 70% 2.8 100 % 4 100% 4 availability 3 60% 1.8 50% 1.5 100% 3 100% 3 80% 2.4 80% 2.4 efficiency scores 25 17.4 16.2 20.3 23.3 17.9 15.7 table 11 evaluation of costs alternatives acquisition cost (usd) operation cost (usd/hour) total cost (usd) total cost (million usd) dassault rafale 130,000,000 28,000 130,028,000 130.028 saab jas 39 gripen 45,000,000 28,001 45,028,001 45.008 mikoyan mig35 55,000,000 28,002 55,028,002 55.016 sukhoi su-35 75,000,000 28,003 75,028,003 75.036 chengdu j-10 35,000,000 28,004 35,028,004 35.005 pac jf17 thunder 25,000,000 28,005 25,028,005 25.004 ijahp article: ali, asghar, muhammad/selection of a fighter aircraft to improve the effectiveness of air combat in the war on terror: pakistan air forcea case in point international journal of the analytic hierarchy process 261 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.489 table 12 eci of alternatives alternatives eci ranking dassault rafale 0.133817332 6 saab jas 39 gripen 0.359936011 4 mikoyan mig-35 0.368983568 3 sukhoi su-35 0.310517618 5 chengdu j-10 0.511355521 2 pac jf-17 thunder 0.627899536 1 as shown in table 12, the results show the pac jf-17 thunder as an optimal air combat aircraft. this assessment is based on the technical characteristics and economic constraints, and is followed in rank by the chengdu j-10 and mikoyan mig-35. surprisingly, we see that the jf-thunder has the highest ranking among other aircrafts, while the dassault rafale aircraft has the lowest ranking after all the calculations. thus, a long procedure with various calculations using these methodologies is needed in order to determine the conclusions effectively. this table presents a clear view of one that is the most suitable option for the purposes of this study. 4. discussion of results combat aircraft fleet planning is a process of strategic importance for an air force engaged on multiple fronts with warfare ranging from conventional air superiority to precision driven, counter-insurgency operations. procurement or development of such aircraft requires huge defense budget expenditures, thereby rendering selection of an appropriate aircraft, a key determinant of effectiveness of a modern air force. hence, when selecting an aircraft, the operational requirements of an air force must be carefully evaluated, keeping in view the economic and geopolitical challenges related to defense procurements. the air force is interested in acquiring the best possible aircraft in adequate numbers. the opposition between the requirements and constraints need to be dealt with, ensuring a perfect tradeoff to approach the optimal selection. the combat aircraft selection problem for the pakistan air force was considered in this paper. keeping in view the fiscal defense budget of pakistan, the requirement of precision target capability and geopolitical constraints, six alternatives were shortlisted. by further considering technical and financial characteristics as criteria (service ceiling, mtow, precision target capability, combat radius, cruising speed, maneuverability, acquisition cost, operation cost, maintainability and availability), various aspects of an aircraft purchase were evaluated. the results show that by using the ahp, the mig-35 turns out to be the best possible solution, closely followed by su-35. even though the mig-35 outweighs the technologically superior su-35, in both acquisition and operational cost, the su-35 outweighs the mig-35 in precision targeting capability. the other aspects of both aircraft are somewhat similar. cba was applied to ensure that the constraint of cost was not exceeded, and ensuring that financial constraints were taken into consideration. application of cba to our six alternatives proved that the locally ijahp article: ali, asghar, muhammad/selection of a fighter aircraft to improve the effectiveness of air combat in the war on terror: pakistan air forcea case in point international journal of the analytic hierarchy process 262 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.489 manufactured jf-17 is the most economical choice, closely followed by the j-10. the jf-17 is a technologically inferior aircraft in comparison to the j-10; however, the eci for both aircrafts closely matches. therefore, the j-10 could be considered a better option over the jf-17, fulfilling both the technological and financial constraints. basing a conclusion on the results of the three independent analyses performed, the mikoyan mig-35 offers the best solution to the stated problem since it possesses an optimal trade-off between the technological requirement and the budget limitations. 5. conclusion multi-criteria decision making is a well-known branch of decision making. the ahp is one of the most commonly used methods for decision making in the literature. in our study, we focused on the problem of aircraft selection for the pakistan air force in order to improve the effectiveness of air combat in the war on terror. this however is not only a problem of the pakistan air force, for today’s growing and competitive military air forces, aircraft selection is so important. the results of the three methods of assessment showed different aircraft leading by a small margin. however, it may be noted that the order of preference of the alternatives hints at the reoccurrence of some alternatives. the mig-35 turns out to be the best possible solution, closely followed by su-35 as seen in the ahp application, while the su-35 turns out to be the best option followed by j-10 and mig-35. however, cba concludes that the jf-17 is the most economical choice, closely followed by the j-10 chengdu and the mig-35. owing to its reoccurrence in the preferred order of selection on the basis of the analyses conducted, it was concluded that the mig-35 is the optimal choice in the case of a fleet up gradation scenario for paf. for the future research, the problem can be solved by other mcdm techniques and the solutions can be compared. also, ahp and anp with fuzzy numbers could be used for the aircraft selection processes for military air forces, and intelligent software which calculate solutions automatically can be developed. 6. future considerations a general observation of the results recommends that the pakistan air force should consider the j-10, su-35 and mig-35 as its top choices. the final choice amongst these three aircrafts can only be realized when the economic and geopolitical constraints have been well specified. the su-35 provides the best solution if pakistan can manage to negotiate the price and ensure a guaranteed supply of spares and expert backing. nevertheless, if the pakistan air force could manage to upgrade the j-10 to the tier of its contenders as it did for the mirage 3 while keeping its cost within the desired range, the j-10 could be the best option for pakistan air force. alternately, the mig-35 could serve as the desired option in the case where russia agrees to furnish a continuous supply of spares and technical backing, but disagrees to negotiate the price. as such, it may be affirmed that even though a resolution has been achieved after a well-constructed and much thought out strategy as presented in the determinations of this composition, further written reports relating to a more detailed economic ijahp article: ali, asghar, muhammad/selection of a fighter aircraft to improve the effectiveness of air combat in the war on terror: pakistan air forcea case in point international journal of the analytic hierarchy process 263 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.489 situation (which includes the time based fluctuation of the economy, an assessment of the internal rate of return and net present value) and a thorough political analysis of the case is recommended which may offer a more accurate answer to the scenario. ijahp article: ali, asghar, muhammad/selection of a fighter aircraft to improve the effectiveness of air combat in the war on terror: pakistan air forcea case in point international journal of the analytic hierarchy process 264 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.489 references bhadra, d. choice of aircraft fleets in the us nas: findings from a multinomial logit analysis. 3rd annual technical forum of the atio/aiaa, denver, co, 2003. bhushan, n. & rai, k. (2007). strategic decision making: applying the analytic hierarchy process. london: springer science & business media. doi: 10.1007/b97668 büyükyazıcı, m. & sucu, m. (2003). the analytic hierarchy and analytic network processes. criterion, 1, c1. dey, p. k. (2002). benchmarking project management practices of caribbean organizations using analytic hierarchy process. benchmarking: an international journal, 9, 326-356. doi: https://doi.org/10.1108/14635770210442680 dožić, s. & kalić, m. (2014). an ahp approach to aircraft selection process. transportation research procedia, 3, 165-174. doi: https://doi.org/10.1016/j.trpro.2014.10.102 dožić, s. & kalić, m. (2015_. comparison of two mcdm methodologies in aircraft type selection problem. transportation research procedia, 10, 910-919. doi: https://doi.org/10.1016/j.trpro.2015.09.044 forman, e. h. & gass, s. i. (2001). the analytic hierarchy process—an exposition. operations research, 49, 469-486. doi: https://doi.org/10.1287/opre.49.4.469.11231 harasani, w. i. (2006). evaluation and selection of a fleet of aircraft for a local airline. journal of king. abdulaziz university: engineering sciences, 17(2), 3-16. doi: 10.4197/eng.17-2.1 harasani, w. i. (2013). evaluation and selection of a fleet of aircraft located in madniah, saudi arabia. editorial and advisory board, 4, 40. hwang, c. & yoon, k. (1981). multiple criteria decision making. lecture notes in economics and mathematical systems, 186. kyriazis, n. & salavrakos, i.-d. (2006). defence procurement in greece:: a costbenefit analysis of fighters for the hellenic air force. 10th annual international conference on economics and security, thessaloniki. citeseer, 22-24. doi: https://doi.org/10.1016/j.foreco.2006.03.023 martins, h. & mendoza, g. a. (2006). multi-criteria decision analysis in natural resource management: a critical review of methods and new modeling paradigms. forest ecology and management, 3, 1-22. doi: https://doi.org/10.1016/j.foreco.2006.03.023 palut, m. p. j. & canziani, o. f. (2007). contribution of working group ii to the fourth assessment report of the intergovernmental panel on climate change. cambridge: cambridge university press. https://doi.org/10.1108/14635770210442680 http://www.sciencedirect.com/science/article/pii/s2352146514002658#! http://www.sciencedirect.com/science/article/pii/s2352146514002658#! https://doi.org/10.1016/j.trpro.2014.10.102 http://www.sciencedirect.com/science/article/pii/s2352146514002658#! http://www.sciencedirect.com/science/article/pii/s2352146514002658#! https://doi.org/10.1016/j.trpro.2015.09.044 https://doi.org/%7b0%7d https://doi.org/10.1016/j.foreco.2006.03.023 https://doi.org/10.1016/j.foreco.2006.03.023 ijahp article: ali, asghar, muhammad/selection of a fighter aircraft to improve the effectiveness of air combat in the war on terror: pakistan air forcea case in point international journal of the analytic hierarchy process 265 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.489 pyzdek, t. (2014). ahp spreadsheet [online]. available: http://sixsigmatraining.com/six-sigma-tools/ahp-spreadsheet.html. saaty, t. l. (1980). the analytic hierarchy process. new york: mcgraw-hill. saaty, t. l. (1990). how to make a decision: the analytic hierarchy process. european journal of operational research, 48, 9-26. doi: https://doi.org/10.1016/0377-2217(90)90057-i saaty, t. l. (2004). decision making—the analytic hierarchy and network processes (ahp/anp). journal of systems science and systems engineering, 13, 135. saaty, t. l. (2008). decision making with the analytic hierarchy process. international journal of services sciences, 1, 83-98. doi: http://dx.doi.org/10.1504/ijssci.2008.017590 salomon, v. a. & montevechi, j. a. b. (2001). a compilation of comparisons on the analytic hierarchy process and others multiple criteria decision making methods: some cases developed in brazil. 6th international symposium on the analytic hierarchy process, bern. sánchez-lozano, j., serna, j. & dolón-payán, a. (2015). evaluating military training aircrafts through the combination of multi-criteria decision making processes with fuzzy logic. a case study in the spanish air force academy. aerospace science and technology, 42, 58-65. doi: https://doi.org/10.1016/j.ast.2014.12.028 sen, p. & yang, j.-b. (1998). multiple criteria decision support in engineering design. london: springer. doi: 10.1007/978-1-4471-3020-8 stewart, t. & belton, v. (2002). multiple criteria decision analysis: an integrated approach, london. kluwer academic publisher. syed, b. s. (2016). time running out for pakistan on f-16 decision. dawn news. tsagdis, a. (2008). the use of the analytical hierarchy process as a source selection methodology and its potential application within the hellenic air force. dtic document. udo, g. g. (2000). using analytic hierarchy process to analyze the information technology outsourcing decision. industrial management & data systems, 100(9), 421-429. doi: https://doi.org/10.1108/02635570010358348 wang, j., fan, k., su, y., liang, s. & wang, w. (2008). air combat effectiveness assessment of military aircraft using a fuzzy ahp and topsis methodology. system simulation and scientific computing, 2008. icsc 2008. asia simulation conference-7th international conference on, 2008a. ieee, 655-662. doi: doi: 10.1109/asc-icsc.2008.4675442 wang, r., zhang, a. & shi, z.-w. (2008b). effectiveness evaluation of advanced fighter plane based on power series and fuzzy ahp [j]. fire control and command control, 11, 020. http://sixsigmatraining.com/six-sigma-tools/ahp-spreadsheet.html https://doi.org/10.1016/0377-2217(90)90057-i https://doi.org/10.1016/j.ast.2014.12.028 https://doi.org/10.1108/02635570010358348 https://doi.org/10.1109/asc-icsc.2008.4675442 ijahp article: ali, asghar, muhammad/selection of a fighter aircraft to improve the effectiveness of air combat in the war on terror: pakistan air forcea case in point international journal of the analytic hierarchy process 266 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.489 wang, t.-c. & chang, t.-h. (2007). application of topsis in evaluating initial training aircraft under a fuzzy environment. expert systems with applications, 33, 870-880. doi: https://doi.org/10.1016/j.eswa.2006.07.003 https://doi.org/10.1016/j.eswa.2006.07.003 ijahp article: ali, asghar, muhammad/selection of a fighter aircraft to improve the effectiveness of air combat in the war on terror: pakistan air forcea case in point international journal of the analytic hierarchy process 267 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.489 appendix a a. list of experts and their paf experience rank officers representation years of expertise number of responses air commodore air cdre minimum 10-13 years 12 group captain gp capt minimum 8-10 years 16 wing commander wg cdr minimum 5-10 years 22 squadron leader sqn ldr minimum 5-10 years 11 flight lieutenant flt lt minimum 5 years 28 flying officer flg off minimum 3 years 36 pilot officer plt off minimum 3 years 18 civilian gazetted officers go minimum 5 years 25 ministerial staff ml minimum 5 years 10 technical staff tl minimum 5 years 20 ground combatiers gc minimum 8 years 22 ijahp article: ali, asghar, muhammad/selection of a fighter aircraft to improve the effectiveness of air combat in the war on terror: pakistan air forcea case in point international journal of the analytic hierarchy process 268 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.489 b. questionnaire questionnaire purpose: pakistan's role in the war on terror is a widely discussed topic among policymakers of various countries and political analysts around the world. recent crashes, retirement of old fighter jets and ongoing war on terror has increased the need to purchase new modern aircraft. hence, there is a need for procurement of such modern aircrafts so as to increase our efficiency in war on terror in pakistan. questionnaire methodology: we will use multi-criteria decision making, analytical hierarchy process (ahp) in particular, for our objective. ahp uses a hierarchical structure and pair-wise comparisons. this technique requires data to develop a decision matrix showing pair-wise comparisons between the decision criteria. questionnaire: please mark a cross (x) in the boxes given in the following tables for pair-wise comparisons of the following criteria: 1. service ceiling 2. maximal take-off weight (mtow) 3. precision target capability 4. combat radius 5. maximum cruising speed 6. maneuverability 7. acquisition cost 8. operation cost 9. maintainability 10. availability ijahp article: ali, asghar, muhammad/selection of a fighter aircraft to improve the effectiveness of air combat in the war on terror: pakistan air forcea case in point international journal of the analytic hierarchy process 269 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.489 1. combat radius extremely preferred (9) very strongly preferred (7) strongly preferred (5) moderately preferred (3) equally preferred (1) combat radius service ceiling combat radius mtow combat radius precision target capability combat radius cruising speed combat radius maneuverability combat radius acquisition cost combat radius operation cost combat radius maintainability combat radius availability 2. service ceiling extremely preferred (9) very strongly preferred (7) strongly preferred (5) moderatel y preferred (3) equally preferred (1) service ceiling mtow service ceiling precision target capability service ceiling combat radius service ceiling cruising speed service ceiling maneuverability service ceiling acquisition cost service ceiling operation cost service ceiling maintainability service ceiling availability ijahp article: ali, asghar, muhammad/selection of a fighter aircraft to improve the effectiveness of air combat in the war on terror: pakistan air forcea case in point international journal of the analytic hierarchy process 270 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.489 3. maximum take-off weight (mtow) extremely preferred (9) very strongly preferred (7) strongly preferred (5) moderately preferred (3) equally preferred (1) mtow precision target capability mtow combat radius mtow service ceiling mtow cruising speed mtow maneuverability mtow acquisition cost mtow operation cost mtow maintainability mtow availability 4. precision target capability extremely preferred (9) very strongly preferred (7) stron gly prefer red (5) moderate ly preferred (3) equal ly prefer red (1) precision target capability combat radius precision target capability cruising speed precision target capability maneuverabi lity precision target capability acquisition cost precision target capability operation cost precision target capability maintainabil ity precision target capability availability ijahp article: ali, asghar, muhammad/selection of a fighter aircraft to improve the effectiveness of air combat in the war on terror: pakistan air forcea case in point international journal of the analytic hierarchy process 271 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.489 5. availability extremely preferred (9) very strongly preferred (7) strongly preferred (5) moderately preferred (3) equally preferred (1) availability precision target capability availability combat radius availability service ceiling availability cruising speed availability maneuvera bility availability acquisition cost availability operation cost availability maintainab ility 6. cruising speed extremely preferred (9) very strongly preferred (7) strongly preferred (5) moderately preferred (3) equally preferred (1) cruising speed maneuverability cruising speed acquisition cost cruising speed operation cost cruising speed maintainability cruising speed availability 7. maneuverability extremely preferred (9) very strongly preferred (7) strongly preferred (5) moderately preferred (3) equally preferred (1) maneuverability acquisition cost maneuverability operation cost maneuverability maintainability maneuverability availability ijahp article: ali, asghar, muhammad/selection of a fighter aircraft to improve the effectiveness of air combat in the war on terror: pakistan air forcea case in point international journal of the analytic hierarchy process 272 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.489 8. acquisition cost extremely preferred (9) very strongly preferred (7) strongly preferred (5) moderately preferred (3) equally preferred (1) acquisition cost operation cost acquisition cost maintainability acquisition cost availability 9. operation cost extremely preferred (9) very strongly preferred (7) strongly preferred (5) moderately preferred (3) equally preferred (1) operation cost maintainability operation cost availability 10. maintainability extremely preferred (9) very strongly preferred (7) strongly preferred (5) moderately preferred (3) equally preferred (1) maintainability availability ijahp article: ali, asghar, muhammad/selection of a fighter aircraft to improve the effectiveness of air combat in the war on terror: pakistan air forcea case in point international journal of the analytic hierarchy process 273 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.489 glossary term definition service ceiling the service ceiling is the altitude at which the maximum rate of climb is 100 ft/min. (0.5 m/s) for piston powered aircraft or 500 ft/min (2.5 m/s) for jet powered aircraft. mtow the maximum takeoff weight (mtow) of an aircraft is the maximum weight at which the pilot is allowed to attempt to take off, due to structural or other limits. precision target capability precision targeting capability refers to the attempted aerial execution of a target with some degree of accuracy, with the aim of limiting collateral damage. combat radius combat radius is a related measure based on the maximum distance a warplane can travel from its base of operations, accomplish some objective, and return to its original airfield with minimal reserves. cruising speed cruise is level flight after an aircraft climbs to a set altitude and before it begins to descend. commercial, defense or passenger aircraft are usually designed for optimum performance at their cruise speed. maneuverability maneuverability is the quality in an aircraft which determines the rate at which its attitude and direction of flight can be changed. acquisition cost acquisition cost may include the negotiated and agreed cost of buying the aircraft and additional costs that can be capitalized and include payments for purchase rights or purchase options. these are distinct from manufacturer credits, and include amounts paid to secure the right to buy a certain aircraft at a certain time. operation cost operation cost includes direct and indirect, fixed and variable costs incurred to enable the aircraft to attain usefulness in operation. this may include maintenance cost and cost incurred due to fuel expenditure whilst airborne. maintainability maintainability is defined as the probability of performing a successful repair action within a given time. in other words, maintainability measures the ease and speed with which a system can be restored to operational status after a failure occurs. availability availability is the degree to which a system, subsystem or equipment is in a specified operable and committable state at the start of a mission, when the mission is called for at an unknown, i.e. a random, time. ijahp: mu/beginning our 11 th year! international journal of the analytic hierarchy process 1 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.655 beginning our 11 th year! dear ijahp reader, we are glad to publish our first issue for 2019. it has been 10 years since we published our very first issue, and this is a reason to say thank you for your support. it is the ijahp community as a whole who make this journal possible. we each play a role from authoring papers to reading and recommending the journal to volunteering to be a reviewer. this issue has a great variety of ahp/anp applications, mainly in the public sector, and closes with a thought-provoking discussion of new perspectives to structure complexity. in addition, our special topic articles section includes an article on the use of an intuitionistic fuzzy linguistic ahp method to prioritize investments. dr. cengiz kahraman was guest editor for our special topic section on fuzzy ahp in the previous issue, but due to logistic considerations this article could not be published then. it is published now for your enjoyment, and constitutes our last article on fuzzy ahp as a special topic. enjoy this issue! very sincerely, enrique mu ijahp, editor-in-chief ijahparticle: muniz, nascimento, martins, rangel/ flexibility assessment to mitigate complexity: trucks production analysis international journal of the analytic hierarchy process 48 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.404 flexibility assessment to mitigate complextity: trucks production analysis jorge muniz jr. universidade estadual paulista – unesp brazil jorge86056@gmail.com 1 luis oliveira nascimento universidade estadual paulista – unesp brazil luisdeoliveiranascimento@gmail.com henrique rocha martins universidade estadual do rio de janeiro – uerj brazil prof.henrique_rocha@yahoo.com.br luis alberto duncan rangel universidade federal fluminense campus de volta redonda brazil duncan@metal.eeimvr.uff.br abstract new products are continuously developed in order to support customized demands. flexibility supports customized demands, low costs, and agility, but it remains a challenge with regards to high-volume and high-diversity complexity as observed in trucks production. this research aims to discuss an assessment approach based on ahp application to rank flexibility projects. it is a multicriteria decision method based on the analytic hierarchy process (ahp). decision-makers (88 managers) from different truck plant areas (production, logistics, quality, sales and finance) were interviewed and asked to consider lean thinking, mass customization, and agility to rank flexibility improvement projects that aim to reduce time-to-market and increase company competitiveness increase. https://doi.org/10.13033/ijahp.v9i1.404 1 the authors are grateful to professor valerio antonio pamplona salomon (unesp) for ahp assistance during this research, and to the comments in the review process. in addition, we would like to thank the brazilian national research council (cnpq) which supported this research (proc. 309028/2015-9). https://doi.org/10.13033/ijahp.v9i1.404 mailto:luisdeoliveiranascimento@gmail.com mailto:prof.henrique_rocha@yahoo.com.br https://doi.org/10.13033/ijahp.v9i1.404 ijahparticle: muniz, nascimento, martins, rangel/ flexibility assessment to mitigate complexity: trucks production analysis international journal of the analytic hierarchy process 49 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.404 keywords: flexibility; lean, agility; complexity; trucks 1. introduction this research discusses an assessment approach based on an ahp application to rank flexibility projects in a bus and truck manufacturing plant. the selection and prioritization of such projects was based on manager’s judgment, taking into consideration the company’s need to support flexibility improvement and to mitigate complexity. usually this decision process uses questionnaires with likert scales and/or interviews. the assessment approach and flexibility factors discussed in the present research could interest operations management scholars and practitioners. the incomplete pairwise comparison algorithm – ipc was applied to assess factors that influence flexibility and to mitigate industrial operation complexity (harker, 1987). ipc allows for far fewer judgments from experts in order to consolidate and validate findings. this research was performed in a truck and bus chassis plant, which operates with the concept of modular consortium, where eight different partner companies work simultaneously at the same facility to handle a high-volume/large-variety portfolio, sharing investments, infrastructure, risks, responsibilities, values, and decision processes. the findings are the result of manager’s (experts) judgments from the areas of production, quality, finance and information technology about flexibility, agility, mass customization and lean thinking and their influence on operation complexity. automotive industry models encompass a variety of different platforms, bodies and models produced in assembly lines, and result in component management complexity (naga & kodali, 2016). modrak, marton and bednar (2015) correlated performance decrease with complexity growth. they showed evidence that product variety increases efforts to deliver faster-cheaper-better products as observed at the studied automotive plant. slack (2005) suggests that flexibility in managing an uncertain environment enables high-performance manufacturing with reliability, speed, and low cost. flexibility is an organizational requirement needed to handle consumer’s needs, changes and an uncertain environment (baykasoglu & ozbakir, 2008; boyle, 2006; chang et al., 2007; wahab, wu & lee, 2008). the mantra of lean thinking has contributed to the improvement of industrial competitiveness grounded in waste reduction and continuous improvement (holweg, 2006). lean thinking and flexibility have antagonistic approaches influenced by standardization, operation complexity, supply chain, logistics risks, market dynamics, and consumer behavior changes. based on such a dichotomous situation, the following questions arise: which factors would enable flexibility in a high-volume / high-variety scenario? how can we prioritize or rank flexibility improvement projects based on the conflicts that exist in different areas? these questions can be addressed with a multicriteria technique. ahp-ipc is suitable to rank improvement projects to support a company’s managerial decisions, and consider trade-offs between lean thinking, agility, and mass customization. https://doi.org/10.13033/ijahp.v9i1.404 ijahparticle: muniz, nascimento, martins, rangel/ flexibility assessment to mitigate complexity: trucks production analysis international journal of the analytic hierarchy process 50 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.404 in this context, the research findings would support the decision-making process to improve flexibility and mitigate complexity, and are aligned with the following research themes: (a) organizational agility strategies (hallgren & olhager, 2009), (b) lean thinking maintenance (jeyaraman, kee & teo, 2010); (c) employee engagement in continuous improvement (emiliani & stec, 2004). the article is organized into five sections including the introduction. section 2 examines the basic theoretical underpinnings while discussing lean thinking, agility, flexibility, mass customization and complexity in the industrial context of high-volume and highvariety. the analytical hierarchy process (ahp) with aggregation of individual priorities (aip) is also discussed. the third section addresses the methodological aspects, followed by section 4, which presents and discusses the results; and section 5 sets out the main conclusions and some suggestions for future research. 2. theoretical background baykasoglu and ozbakir (2008), boyle (2006), chang et al. (2007), maccormack, verganti, and iansiti (2001), slack (2005), and wahab, wu, and lee (2008) indicate flexibility as the solution for several scenarios such as demand for quick responses and product variety in a fiercely competitive environment, tight schedules, rapidly changing consumer preferences, and high uncertainty. they suggest this allows for better design and high-performance manufacturing with reliability, speed and low costs. holweg (2006) states that lean thinking changed the relationship between productivity and quality in the automobile industry by connecting waste avoidance with continuous improvement. however, most western manufacturers have not realized that toyota’s production system is an extension of their product development philosophy and not the reverse (baines et al., 2006). hu et al. (2011) highlight the cause-effect relationships between variety, flexibility and complexity. product variety influences production flexibility, which, in turn, results in complexity in the operation. hayes and wheelwright (1979) in their original paper do not indicate options for production systems with high volumes and large variety in traditional assembly line configurations (figure 1). https://doi.org/10.13033/ijahp.v9i1.404 ijahparticle: muniz, nascimento, martins, rangel/ flexibility assessment to mitigate complexity: trucks production analysis international journal of the analytic hierarchy process 51 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.404 figure 1. traditional process–product matrix (hayes and wheelwright, 1979) lean thinking, flexibility, agility, and mass customization influence operations complexity in an uncertain environment as observed in their definitions:  flexibility is the capacity to respond to changes (gupta & buzacott, 1989). kara and kayis (2004) related flexibility to market (demand, product/technology life cycle, variety, customization, and delivery time) and/or operations (machines, materials, and manpower);  agility is the ability to deal with the market uncertainties and deliver goods and services with a high level of service. it is a concept related to flexibility and speed (agarwal; shankar & tiwari, 2006);  mass customization (mc) means the production of higher product variety with cost and volume tradeoffs (boyton, victor & pine ii, 1993). the goal of mass customization is to create customized products, with mass production volumes, costs and competitive efficiencies (smith et al., 2013). with the emergence of production systems with high volumes and large variety (such as dell computers), the traditional process–product matrix was updated to include mass customization (where the truck production fits) and cellular manufacturing concepts (hayes & wheelwright, 1979) see figure 2. https://doi.org/10.13033/ijahp.v9i1.404 ijahparticle: muniz, nascimento, martins, rangel/ flexibility assessment to mitigate complexity: trucks production analysis international journal of the analytic hierarchy process 52 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.404 figure 2. updated process–product matrix (swink et al., 2013) the flexibility to provide complexity mitigation may be influenced by purchase ordering and manufacturing processes through the following factors, as described in table 1:  materials ordering flexibility: order stream and product profitability.  manufacturing flexibility: supplier flexibility cost; frozen time sensivity; and supplier continuous improvement. table 1 flexibility factors to commercial vehicles factors description authors purchase order stream waste on the order input process, aiming to reduce vehicles delivery lead time. leeuw, holweg and williams (2010) cachon and olivares, (2009) product profitability process regarding the lead time that affects high profitability products. wittel, gustafsson and johnson (2013) riesenbacket al. (2005) supplier flexibility cost costs resulting from changes in frozen (re)planning period and volume wang, zhang and ying (2007) chen, paulraj and lado(2007) frozen time sensivity effects produced in supply chain by adoption of different planning materials frozen periods powell (2012) wee and wu, 2009 suppliers continuous improvement lean thinking implementation program in the suppliers’ line up, in order to improve the entire chain capability and reduce costs. kerbacheand van delft (2013) guo and xu (2007) https://doi.org/10.13033/ijahp.v9i1.404 ijahparticle: muniz, nascimento, martins, rangel/ flexibility assessment to mitigate complexity: trucks production analysis international journal of the analytic hierarchy process 53 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.404 3. methods bertrand and fransoo (2002) indicated the need to validate the usability and performance of a model used in quantitative theoretical research to solve real-life operational problems. quantitative model-based empirical research grounded this study. in the present study, the proposed construct is the analytic hierarchy structure– ahs (see figure 4). the quantitative empirical research made it feasible to test the validity of the findings and ahs with respect to real-life operational processes. operations management literature indicated different research methods to approach an operational flexibility decision problem. ahs conceptualization encompasses literature about lean thinking (manufacturing, administration/ office), flexibility, agility, and mass customization, multi-criteria decision methods and analytic hierarchy process (ahp). therefore, ahp was chosen to rank the factors based on expert’s judgment. managers and staff involved in operations management (production & logistics, quality, finance, information technology, and sales, marketing & after-sales) were given an electronic questionnaire. we received 88 responses (55.7% of the population) from 1 industrial vice president, 6 senior managers, 10 managers, 20 supervisors, 2 coordinators, and 49 technical staff/engineers. the group that was interviewed represents a significant sample of a broad spectrum of experience in the automotive production processes. the questionnaire (see appendix) was based on an ahp variation called incomplete pairwise comparison, which decreases the number of questions (harker, 1987). it uses saaty’s (1991) 5-point scale, i.e.: (1) same importance; (3) low importance; (5) middle importance; (7) high importance; and (9) extreme importance. theoretical research results used to rank improvement projects were based on the assumption that the underlying process models are valid and the theoretical solutions are useable and will perform well (bertrand & fransoo, 2002). however, this belief is seldom tested because it increases research time and cost. in the present research, ahs was used in the flexibility projects to assess the better fit between the model and reality. focus groups validated the findings and the research instrument. data were analyzed to rank factors using manager’s judgments (experts). after that, the ahs was used to guide the actions of the flexibility projects and the managers recognized it as a robust decision-making tool. these theoretical and empirical procedures based on focus groups validated the research. the analytic hierarchy process (ahp) is a structured technique of multi-criteria decision analysis for organizing and analyzing complex decisions. developed in the 1970s by thomas l. saaty, the method decomposes decision problems into a hierarchy of subproblems and, after that, compares qualitative or quantitative data to each other, with respect to their impact on an element above them in the hierarchy tree (saaty, 1980). salgado et al. (2012) indicated judgments inserted in the comparisons matrices are often based on the fundamental scale of absolute numbers (saaty, 1980). that is, a linear scale from 1 to 9. value 1, from that scale, is used when it was judged that both objects have the same priority. one implication of the use of the fundamental scale is that the https://doi.org/10.13033/ijahp.v9i1.404 ijahparticle: muniz, nascimento, martins, rangel/ flexibility assessment to mitigate complexity: trucks production analysis international journal of the analytic hierarchy process 54 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.404 comparison matrix will be a positive reciprocal matrix. that is aij=1/aji and aij>0, ∀i ,j=1, 2, …, n. therefore, x , the number of comparisons required to fulfill a comparison matrix can be obtained by equation 1. x= n(n – 1)/2 (1) a limitation in applying the ahp is the time required to complete all possible pairwise comparisons. a large number of comparisons is a concern when using a questionnaire based on ahp. incomplete pairwise comparisons (ipc) is an algorithm developed to reduce the number of comparisons allowing the group to focus on the debate and decrease time used to fill in a comparison matrix (harker, 1987). harker (1987) explains that the two advantages that the ahp has over other multi-criteria methods are the ease of use and the ability to handle inconsistencies in judgments. nevertheless, the author states the capability to handle such inconsistencies, based on the redundancy within the method, is also a drawback because of the amount of work required to make all of the necessary pairwise comparisons. this research restricted pairwise comparisons to the diagonal above the main diagonal of the comparison matrix. figure 3 shows an example of the comparisons needed when ten factors are evaluated. using ipc, the number of comparisons in a ten-factor evaluation is reduced from 45 to 9. figure 3. ahp matrices for complete and incomplete comparisons (f are factors and c are comparisons) now c10 can be obtained by multiplying c1 by c2 and so on. the advantage of this approach is that inconsistency is not possible which is quite useful when dealing with survey studies. the judgments were collected anonymously from executives of automotive companies. the data were aggregated with aggregation of individual priorities – aip, and the findings were discussed with a sample of these executives to assess the adherence with their industrial experience (forman and peniwati, 1998). by reducing the number of comparisons, from x to n – 1, a greater involvement of the respondents was expected. as a matter of fact, the response was 55.7% of the population. a) f1 f2 f3 f4 f5 f6 f7 f8 f9 f10 b) f1 f2 f3 f4 f5 f6 f7 f8 f9 f10 f1 1 c1 c10 c18 c25 c31 c36 c40 c43 c45 f1 1 c1 f2 1 c2 c11 c19 c26 c32 c37 c41 c44 f2 1 c2 f3 1 c3 c12 c20 c27 c33 c38 c42 f3 1 c3 f4 1 c4 c13 c21 c28 c34 c39 f4 1 c4 f5 1 c5 c14 c22 c29 c35 f5 1 c5 f6 1 c6 c15 c23 c30 f6 1 c6 f7 1 c7 c16 c24 f7 1 c7 f8 1 c8 c17 f8 1 c8 f9 1 c9 f9 1 c9 f10 1 f10 1 https://doi.org/10.13033/ijahp.v9i1.404 ijahparticle: muniz, nascimento, martins, rangel/ flexibility assessment to mitigate complexity: trucks production analysis international journal of the analytic hierarchy process 55 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.404 4. data analysis the analytic hierarchy structure (table 2 and figure 4) presents the results calculated from the expert’s/executive’s judgments to increase flexibility based on the criteria of agility, mass customization and lean thinking. the results were aggregated based on forman and peniwati (1998), the aggregation of individual priorities (aip). that is, priorities from each executive were aggregated by arithmetic mean, according to executive area (production, logistics, quality, sales, and finance). aip was the appropriate approach to treat the data collected because of the broad research characteristics including number of areas researched, and size of the population. more importantly, because of the use of direct judgments (first diagonal) in aggregating the priorities (aip), and not the aggregation of estimated values of judgments based on geometric mean (aij), there is a greater significance to the results. the lean office had a lower priority when compared with lean manufacturing, agility, and mass customization, but this is because the major plant focus has, so far, been given to increase the production value-added. table 2 flexibility analytic hierarchy structure objective flexibility increase criteria lean manufacturing lean office agility mass customization 32.6% 14,8% 27,2% 25,3% s u b -c ri te ri a product profitability 21.0% 22.2% 18.5% 19.9% ordering stream 15.2% 22.8% 14.9% 16.0% supplier continuous improvement 16.6% 13.8% 16.9% 16.5% frozen time sensivity 17.1% 15.6% 12.4% 17.5% suppliers flexibility cost 30.0% 25.7% 37.4% 30.1% the lean office had a lower priority when compared with lean manufacturing, agility, and mass customization, but this is because the major plant focus has, so far, been given to increase the production value-added. https://doi.org/10.13033/ijahp.v9i1.404 ijahparticle: muniz, nascimento, martins, rangel/ flexibility assessment to mitigate complexity: trucks production analysis international journal of the analytic hierarchy process 56 vol. 9 issue 1 2017 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v9i1.404 figure 4. analytic hierarchy structure to improve flexibility ijahparticle: muniz, nascimento, martins, rangel/ flexibility assessment to mitigate complexity: trucks production analysis international journal of the analytic hierarchy process 57 vol. 9 issue 1 2017 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v9i1.404 the overall results of the alternatives indicate that the supplier’s additional flexibility presented the greatest importance (31.4%), followed by the focus with profitability products, the analysis of ordering stream (16.5%), suppliers continuous improvement (16.2%) and frozen time sensivity (15.7%). table 3 presents the results consolidated by industrial area. table 3 areas results criteria area production logistics quality sales finance lean manufacturing 37.96% 18.59% 39.59% 28.98% 34.84% lean office 11.71% 10.33% 14.57% 11.95% 12.20% agility 26.30% 37.70% 19.35% 30.63% 27.82% mass customization 24.03% 33.38% 26.48% 28.44% 25.14% respondents 25 19 8 18 18 the alternatives weights support project ranking to improve flexibility. the following lean thinking projects were proposed and submitted for appraisal: (1) make product development faster; (2) reduce ordering lead time; and (3) increase administrative service level such analysis reinforces harker’s statement about ipc, i.e. the time saved enables the group to focus on the debate by simplifying the work involved in making pairwise comparisons. therefore, it gives the individual or group of decision makers more time to debate certain judgments and create different hierarchical structures for the problem, which can then be compared and synthesized (harker, 1987). the decision in regards to the project to be implemented was the lean office, focused on the ordering value stream. the rationale for this choice was as follows: (1) several lean production projects have been developed in the studied company since 2008; therefore, it is a known subject and the results have proved effective. besides that, the principles of agility are also present in the organization by focusing on service tailored to customer needs, the pursuit of satisfaction in specific niches, which results in time-reduction activities, mainly in its supply chain. the mass customization was recently implemented in product development; and (2) the studied company had never tried to analyze/implement lean office techniques. ijahparticle: muniz, nascimento, martins, rangel/ flexibility assessment to mitigate complexity: trucks production analysis international journal of the analytic hierarchy process 58 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.404 implementation was supported by shingijutsu global consulting (sgc), a global company specializing in performance increase efficiency in manufacturing, logistics and processes. sgc consultants attended to three kaizen events focused on logistics, production, and administrative processes, in which previous performance jeopardized the company’s competitiveness. 4.1 managerial implications the results obtained led to improved implementation, discussion with the team and moderation by sgc consultants which created the conditions to reduce 30% of the frozen order placement time, reducing the time-to-market and leading customers to have a higher product value-added perception, which increased the company’s service level and competitiveness. even though the company started lean implementation in 2008, it has, since then focused on manufacturing activities. the initiation of a lean office approach has proven the ability to unveil hidden waste-avoidance opportunities, uplift performance in time-to-market, reduce overall manufacturing throughout time and the planning horizon, all of which increased production flexibility. positive results were recognized by senior management and evidenced in the focus group meetings. 5. conclusions there is a contradiction when the concepts of lean thinking and flexibility are associated. lean thinking is based on waste elimination, standardization, and production levering resulting in cost reduction and capacity optimization; and flexibility requires the adaptation of the entire system based on a new constraint or opportunities which unbalances the lean system. however, lean thinking implementation associated with an agility program enables the enterprise to deal with uncertain scenarios and improves their flexibility. the data analysis showed different perceptions depending on the department, even though they share the same company´s objectives regarding product conformity, parts and production flow, products delivery to customers and profit. in this case, the balance in the research participant’s mix is an important part to unveiling the company global vision. the research results highlighted the importance of lean manufacturing factors, speed, mass customization, and lean office to improve the flexibility. it also established an ahp-ipc-based model for productivity enablement projects rating and prioritization, based on the perceptions of managers and technical staff. future research could monitor results from now on and, upon adhering to an actual trend, and be applied to other products and plants. https://doi.org/10.13033/ijahp.v9i1.404 ijahparticle: muniz, nascimento, martins, rangel/ flexibility assessment to mitigate complexity: trucks production analysis international journal of the analytic hierarchy process 59 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.404 references agarwal, a., shankar, r., &tiwari, m.k. (2006). modeling the metrics of lean, agile and leagile supply chain: an anp-based approach. european journal of operational research, 173, 211-225. doi: http://dx.doi.org/10.1016/j.ejor.2004.12.005 baines, t., lightfoot, h., williams, g.m., & greenough, r. (2006). state-of-the-art in lean design engineering: a literature review on white collar lean. proceedings of the institution of mechanical engineers part b-journal of engineering manufacture, 220(9), 1539-1547. baykasoglu, a., & ozbakir, l. (2008). analysing the effect of flexibility on manufacturing systems performance. journal of manufacturing technology management. 19(2), 172-193. doi: http://dx.doi.org/10.1108/17410380810847909 bertrand, j. w. m. and fransoo, j. c. (2002). operations management research methodologies using quantitative modeling, international journal of operations & production management, 22(2), 241-264 doi: http://dx.doi.org/10.1108/01443570210414338 boyle, t. a. (2006) towards best management practices for implementing manufacturing flexibility. journal of manufacturing technology management. 17(1), 6-21. doi: http://dx.doi.org/10.1108/17410380610639470 boyton, a.c., victor, b., & pine ii, b. j. (1993). new competitive strategies: challenges to organizations and information technology. ibm systems journal, 32(1), 40-64. doi: 10.1147/sj.321.0040 bozókia, s., fülöpa, j., &rónyaib, l. (2007). on optimal completion of incomplete pairwise comparison matrices. mathematical and computer modelling, 52(1-2), 318-333. doi: http://dx.doi.org/10.1016/j.mcm.2010.02.047 cachon, g. olivares, m. (2010). drivers of finished goods inventory in the u.s. automobile industry. management science, 56(1), 202-216. doi: http://dx.doi.org/10.1287/mnsc.1090.1095 chang, s., lin, r., chang, f., chen, r. (2007). achieving manufacturing flexibility through entrepreneurial orientation. industrial management & data systems, 107(7), 997-1017.doi: http://dx.doi.org/10.1108/02635570710816711 chen, i.j., paulraj, a., lado, a.a. (2004). strategic purchasing, supply management and firm performance. journal of operations management, 22, 505-523. doi: http://dx.doi.org/10.1016/j.jom.2004.06.002 emiliani, m.l, stec, d.j. (2004). using value stream mapping to improve leadership. leadership & organization development journal, 25, 622-645. doi: http://dx.doi.org/10.1108/01437730410564979 https://doi.org/10.13033/ijahp.v9i1.404 http://dx.doi.org/10.1016/j.ejor.2004.12.005 https://doi.org/10.1147/sj.321.0040 http://dx.doi.org/10.1016/j.mcm.2010.02.047 http://dx.doi.org/10.1287/mnsc.1090.1095 http://dx.doi.org/10.1016/j.jom.2004.06.002 ijahparticle: muniz, nascimento, martins, rangel/ flexibility assessment to mitigate complexity: trucks production analysis international journal of the analytic hierarchy process 60 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.404 fedrizzia, m. &gioveb, s. (2007). incomplete pairwise comparison and consistency optimization. european journal of operational research, 183(1), 303-313. doi: http://dx.doi.org/10.1016/j.ejor.2006.09.065 forman, e. & peniwati, k. (1998). aggregating individual judgments and priorities with the analytic hierarchy process. european journal of operational research, 108(1), 165-169. doi: 10.1016/s0377-2217(97)00244-0 guo, y.; xu, z. a. (2007). model of lean supplier management based on the lean production. international federation for information processing, 254(1), 717-726. doi: 10.1007/978-0-387-75902-9_81 gupta, d. and buzacott, j. a. (1989). a framework for understanding flexibility of manufacturing systems. journal of manufacturing systems, 8(2), 89-96. doi: https://doi.org/10.1016/0278-6125(89)90028-9 hallgren, m., olhager, j. (2009). lean and agile manufacturing: external and internal drivers and performance outcomes. international journal of operations & production management, 29(10), 976-999. doi: http://dx.doi.org/10.1108/01443570910993456 harker, i.j. (1987). incomplete pairwise comparison in the analytic hierarchy process. mathematical modelling, 9(11), 837-848. doi: https://doi.org/10.1016/02700255(87)90503-3 hayes, r. h., wheelwright, s. c. (1979). link manufacturing process and product life cycles. harvard business review, 133-140. doi: 10.1002/9780470618813.app3 holweg, m. (2006). the genealogy of lean production. journal of operation management, 25(2), 420-437. doi: http://dx.doi.org/10.1016/j.jom.2006.04.001 hu, s. j., ko, j., weyand, l. elmaraghy, h. a. lien, t. k. koren, y., bley, h., chryssolouris, g., nasr, n., shpitalni, m. (2011). assembly system design and operations for product variety. cirp annals manufacturing technology, 60(2), 715– 733. doi: http://dx.doi.org/10.1016/j.cirp.2011.05.004 jeyaraman, k., kee leo, l. (2010). a conceptual framework for critical success factors of lean six sigma. international journal of lean six sigma. 1, 191-215. doi: http://dx.doi.org/10.1108/20401461011075008 kara, s., & kayis, b. (2004). manufacturing flexibility and variability: an overview. journal of manufacturing technology management, 15(6), 466-478. doi: http://dx.doi.org/10.1108/17410380410547870 kerbache, l., van delft, c. (2014). efficient purchaser incentive when dealing with suppliers implementing continuous improvements plans. international transactions in operations research, 21, 673-701. doi: 10.1111/itor.12059 kwiesielewicz, m., van uden, e. (2003). ranking decision variants by subjective paired comparisons in cases with incomplete data. in: v. kumar (ed.), et al., computational https://doi.org/10.13033/ijahp.v9i1.404 http://dx.doi.org/10.1016/j.ejor.2006.09.065 https://doi.org/10.1016/0278-6125(89)90028-9 https://doi.org/10.1016/0270-0255(87)90503-3 https://doi.org/10.1016/0270-0255(87)90503-3 http://dx.doi.org/10.1016/j.jom.2006.04.001 http://dx.doi.org/10.1016/j.cirp.2011.05.004 ijahparticle: muniz, nascimento, martins, rangel/ flexibility assessment to mitigate complexity: trucks production analysis international journal of the analytic hierarchy process 61 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.404 science and its applications–iccsa 2003, lecture notes in computer science, 2669, springer-verlag, berlin/heidelberg, 208–215. doi: 10.1007/3-540-44842-x_22 leeuw, s., holweg, m., williams, g. (2011). the impact of descentralized control on firm-level inventory. international journal of physical distribution & logistics management, 41(5), 435-456. doi: http://dx.doi.org/10.1108/09600031111138817 maccormack, a., verganti, r., &iansiti, m. (2001). developing products on “internet time”: the anatomy of a flexible development process. management science, 47(1), 133152. doi: http://dx.doi.org/10.1287/mnsc.47.1.133.10663 modrak, v., marton, d., &bednar, s. (2015). the influence of mass customization strategy on configuration complexity of assembly systems. conference of intelligent computation on manufacturing engineering (cirp), 33, 538-543. doi: https://doi.org/10.1016/j.procir.2015.06.071 naga, v. & kodali, j.m. (2016). an empirical study for implementation of lean principles in indian manufacturing industry. benchmarking: an international journal, 23(1),183-207.doi: http://dx.doi.org/10.1108/bij -11-2013-0101 powell, d. (2013). erp systems in lean production: new insights from a review of lean and erp literature. international journal of operations & production management, 33(11/13), 1490-1510. doi: http://dx.doi.org/10.1108/ijopm-07-2010-0195 riesenbeck, h., herrmann, a., heitmann, m., algesheimer, r. (2006). an approach to profit-maximizing product design on the basis of the platform concept. international journal of quality & reliability management, 23(7), 788-806. doi: http://dx.doi.org/10.1108/02656710610679815 saaty, t.l. (1980). the analytic hierarchy process. new york, ny: mcgrawhill. doi: http://dx.doi.org/10.1080/00137918308956077 salgado, e. g., salomon, v. a. p., carlos h. p. mello, c. h. p. (2012). analytic hierarchy prioritization of new product development activities for electronics manufacturing. international journal of production research, 50(17), 4860-4866. doi: http://dx.doi.org/10.1080/00207543.2012.657972 slack, n. (2005). the flexibility of manufacturing systems. international journal of operations & production management, 25(12), 1190-1200. smith, s., smith, g.c., jiao, r., & chu, c. (2013). mass customization in the product life cycle. journal of intelligent manufacturing, 24(5), 877-885. doi:10.1007/s10845-0120691-0 swink, m., melnyk, s. a., cooper, m. b., & hartley, j. l (2013). managing operations across the supply chain 2nd. ed., mcgraw hill. https://doi.org/10.13033/ijahp.v9i1.404 http://dx.doi.org/10.1287/mnsc.47.1.133.10663 https://doi.org/10.1016/j.procir.2015.06.071 http://dx.doi.org/10.1080/00137918308956077 http://dx.doi.org/10.1080/00207543.2012.657972 ijahparticle: muniz, nascimento, martins, rangel/ flexibility assessment to mitigate complexity: trucks production analysis international journal of the analytic hierarchy process 62 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.404 wahab, m.i. m., wu, d. & lee, c. (2008). a generic approach to measuring the machine flexibility of manufacturing systems. european journal of operational research, 186, 137–149. doi: http://dx.doi.org/10.1016/j.ejor.2007.01.052 wang, r., zhang, t., ying, s. (2007). analysis of supply chain contract on supplier flexibility. international conference on management science & engineering. doi: 10.1109/icmse.2007.4421915 doi: 10.1109/icmse.2007.4421915 wedley. w.c. (1993). consistency prediction for incomplete ahp matrices. mathematical and computer modelling, 17(4–5), 151-161. doi: https://doi.org/10.1016/0895-7177(93)90183-y wee, h.m., we, s. (2009). lean supply chain and its effect on product cost and quality: a case study on ford motor company. supply chain management: an international journal,14/15, 335-341. doi: http://dx.doi.org/10.1108/13598540910980242 witell, l., gustafsson, a., johnson, m. (2014). the effect of customer information during new product development on profits from goods and services. european journal of marketing, 48(9), 1709-1730. doi: http://dx.doi.org/10.1108/ejm-03-2011-0119 https://doi.org/10.13033/ijahp.v9i1.404 http://dx.doi.org/10.1016/j.ejor.2007.01.052 https://doi.org/10.1109/icmse.2007.4421915 https://doi.org/10.1109/icmse.2007.4421915 https://doi.org/10.1016/0895-7177(93)90183-y ijahparticle: muniz, nascimento, martins, rangel/ flexibility assessment to mitigate complexity: trucks production analysis international journal of the analytic hierarchy process 63 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.404 appendix welcome we are conducting a survey in the truck plant. you have been invited to answer this questionnaire because of your work experience and leadership. the research objective is to evaluate the factors to improve flexibility, that in your opinion, contribute to and mitigate complexity. we need your cooperation to fill it up this questionnaire. certainly, you will respond it within minutes and this effort will be very useful to better know the shop floor and guide improvement opportunities. you can feel free to answer what you think, because the questions not involve confidential aspects of your work. you do not need to identify yourself. scale: importance intensity definition (...from na alternative to other) 1 equal importance 3 little more importance 5 somewhat importance 7 great importance 9 absolute importance to increase production flexibility, indicate the relative importance of the following strategies: the alternatives follow the same structure of questions. interviewees’ profile: how long have you worked in this company (years)? how long have you taken up managerial or supervisory assignment (years)? what is your job title? additional comments: https://doi.org/10.13033/ijahp.v9i1.404 3_islam_et_al_malaysia_vol2_issue2_p79-107 ijahp article: islam, ismail/prioritizing issues in malaysia, a developing country international journal of the 79 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 malaysia on course to become a developed country: prioritizing issues with the analytic hierarchy process rafikul islam1 department of business administration kulliyyah of economics and management sciences international islamic university malaysia p.o. box 10, 50728 kuala lumpur e-mail: rislam@iium.edu.my yusof ismail department of business administration kulliyyah of economics and management sciences international islamic university malaysia p.o. box 10, 50728 kuala lumpur e-mail: yusof@iium.edu.my abstract by the year 2020, malaysia aspires to become a fully developed nation. this lofty vision, known as vision 2020, was unveiled by the former prime minister of malaysia tun dr. mahathir bin mohamad on february 28, 1991. currently, it is generally felt that the nation has achieved 50 to 60 percent of the objectives of vision 2020. what needs to be done to achieve the vision and in what areas? these are the questions that we asked to 759 people living in malaysia (malaysians as well as internationals) in a nationwide survey. as expected, the respondents touched upon a wide variety of issues pertaining to education, economy, technology, quality of life, law and order, r&d, and so on. upon compilation of all the articulated issues, an affinity diagram was developed. subsequently, the analytic hierarchy process has been applied in each component of the affinity diagram. this application identifies the main issues that need to be addressed in order to realize the objectives of vision 2020. the present research findings are expected to provide useful guidelines to the policy makers at the national level in the course of fine tuning vision 2020 strategies. keywords: developed country, vision 2020, affinity diagram, analytic hierarchy process, malaysia 1 corresponding author rob typewritten text http://dx.doi.org/10.13033/ijahp.v2i2.40 rob typewritten text ijahp article: islam, ismail/prioritizing issues in malaysia, a developing country international journal of the 80 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 1. introduction vision is a long-term objective that provides an insight into the future direction in which a nation/organization needs to move, just as an identified destination gives a direction to a journey. a well-defined vision provides focus and direction to the formulation of current programs, which in turn, links current actions to the achievement of future goals. malaysian vision 2020 was unveiled 28 february 1991 by former prime minister of malaysia tun dr. mahathir bin mohamad at the inaugural meeting of the malaysian business council. the gist of this vision is to develop malaysia as a fully developed country2. the vision 2020 statement is (rahman, 1993): “by the year 2020, malaysia is to be a united nation, with a confident malaysian society, infused by strong moral and ethical values, living in a society that is democratic, liberal, caring, economically just and equitable, progressive and prosperous, and in full possession of an economy that is competitive, dynamic, robust and resilient.” in the first open national seminar on vision 2020 (v-2020) in 1991, then-deputy prime minister of malaysia tun abdul ghafar baba stressed malaysia should not only be developed economically, but also in all dimensions including political, social, spiritual, psychological as well as cultural. more importantly, in the efforts towards a developed status, malaysia should also endeavor to create a united, confident, socially just and politically stable society, in which everybody has a place, and takes pride in being a 2 the term developed country is used to categorize countries with developed economies in which the tertiary and quaternary sectors of industry dominate. this level of economic development usually translates into a high per capita income and a high human development index. world bank defines high income countries as countries with per capita gnp of $10,000 or more. traditionally, canada and the united states in north america, japan in asia, australia and new zealand in oceania, and most countries in northern europe and western europe have been considered as developed countries. despite their high per capita gnp, the gcc countries in the middle east are generally not considered developed countries because their economies depend overwhelmingly on oil production and export. generally speaking, a nation is said to be developed if it has well-developed political, economic, physical, educational and infrastructural institutions in place and every citizen, irrespective of his/her class, sex, or birth enjoys the confidence to live his/her life by choice and not by imposition from any sector, provided such liberty is within the accepted moral norms of the society, without fear or inhibition of any kind or from any quarter. developed countries have low level of illiteracy, poverty, infrastructural deficiencies, social imbalances, bureaucratic and political bottlenecks, etc and work in cohesion and unison in one voice and dictum and achieves as a result, the highest in the fields of science, and discoveries, at any given point of time and leads the field with highly developed systems in the area of law, justice and social retribution. ijahp article: islam, ismail/prioritizing issues in malaysia, a developing country international journal of the 81 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 malaysian3. in short, v-2020 reflects the vision of a fully developed and industrialized malaysia in all dimensions by the year 2020. tun mahathir mohamad stated this as follows (mohamad, 1991): malaysia should not be developed only in the economic sense. it must be a nation that is fully developed along all the dimensions: economically, politically, socially, spiritually, psychologically and culturally. we must be fully developed in terms of national unity and social cohesion, in terms of our economy, in terms of social justice, political stability, system of government, quality of life, social and spiritual values, national pride and confidence. specific objectives of v-2020 are: • to have sufficient food and shelter with easy access to health and basic essentials • to eradicate poverty • to remove the identification of race with major economic functions and to have a fair distribution with regard to the control, management and ownership of the modern economy • to maintain annual population growth rate of 2.5% • to double real gdp every ten years between 1990 and 2020 • to have a balanced growth in all sectors, namely: industry, agroforestry, energy, transport, tourism and communications, and banking, that is technologically proficient, fully able to adapt, innovative, with a view to always moving to higher levels of technology it is clear that the focus of v-2020 is not only on the economic aspects of development, but also on the interrelated aspects such as social justice, quality of life, moral and ethical values, work ethics and so on (kassim, 1993). to achieve the various objectives of v2020, the former pm identified nine challenges that, if successfully addressed, will enable malaysia to realize v-2020. these challenges are supported and promulgated by other malaysians, including various ministers. here are those nine challenges: 1. establishing a united malaysian nation 2. creating a psychologically liberated, secure, and developed malaysian society 3. developing a mature democratic community 4. forming a community that has high morale, ethics, and religious strength 5. establishing a mature, liberal and tolerant society 6. establishing a scientific and progressive society 7. establishing a fully caring society 8. ensuring an economically just society 9. establishing a prosperous society. 3 speech delivered by tun abdul ghafar baba at the opening ceremony of national seminar towards a developed and industrialized society: understanding of the concepts, implications and challenges of vision 2020, kuala lumpur, 5-7 december, 1991. ijahp article: islam, ismail/prioritizing issues in malaysia, a developing country international journal of the 82 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 mohamad (1991) said: there can be no fully developed malaysia until we have finally overcome the nine central strategic challenges that have confronted us from the moment of our birth as an independent nation. v-2020 is a buzz word in malaysia, and the government is keen to achieve this lofty vision. however, when we talk to the public about this, they are, in general, of the opinion that many more things need to be done before the nation achieves v-2020. a formal questionnaire was developed to assess the present level of achievement. the details are provided in the following sections. 2. public opinion on realization of v-2020 we contacted 759 people in malaysia to respond to the questionnaires. all the respondents were contacted personally by the researchers and the appointed research assistants (ras). because the survey is on the national vision, the ras visited all the states of malaysia and contacted the respondents personally. although few individuals refused (according to the ras and also the researchers’ own experience) to fill out the questionnaires, overall, a high majority of the respondents cooperated with the ras and filled out the questionnaires. the questionnaires had three sections of which the first one was intended to obtain the respondents’ demographical information (see table 1). in the second section, the respondents were asked to provide their opinion on a number of issues of v-2020. in the third and final section, the respondents were invited to provide suggestions to the malaysian government as well as the common populace. the content of this paper is largely based upon the information obtained in the third section. table 1 respondents’ demographic information variable* frequency percent gender • male • female 417 342 54.9 45.1 race • malay • chinese • indian • others 510 82 61 106 67.2 10.8 8.0 14.0 age group • 20 years or below • 21-30 years • 31-40 years • 41-50 years • above 50 years 23 375 204 99 57 3.0 49.4 26.9 13.0 7.5 highest level of education • o level • a level • diploma 28 17 100 3.7 2.2 13.2 ijahp article: islam, ismail/prioritizing issues in malaysia, a developing country international journal of the 83 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 • professional • bachelors • masters • ph.d. • others 38 353 120 59 41 5.0 46.5 15.8 7.8 5.4 marital status • single • married • divorced 357 395 4 47.0 52.0 0.5 type of employment • public • private • self-employed • others 269 289 61 133 35.4 38.1 8.0 17.5 * missing entries are not considered in the table. malaysia is a multi-religious, multi-racial and multi-lingual country. in addition, the country’s population consists of a sizable percentage of foreigners who belong to mainly three categories: students, professionals, and workers. we were also interested to know their views on the nation’s aspiration to become developed and how far the country has progressed. however, we considered only the professionals from among the foreigners for obtaining the feedback in this questionnaire. the statistics of the respondents across various states of malaysia and the world are provided in table 2. it is to be noted that out of 759 respondents, 106 are foreigners belonging to 28 countries (the list of countries are arranged alphabetically starting and ending with afghanistan and zimbabwe, respectively)4. table 2 origin of the respondents malaysian international state no. of respondents country no. of respondents country no. of respondents johor 44 afghanistan 1 oman 1 kedah 51 albania 1 pakistan 6 kelantan 71 algeria 3 saudi arabia 6 kuala lumpur 69 bangladesh 15 senegal 1 malaka 39 china 2 singapore 1 pahang 18 egypt 2 somalia 1 penang 15 ghana 1 sri lanka 1 perak 70 india 6 sudan 1 perlis 2 indonesia 14 tanzania 1 sabah 8 iran 2 tunisia 1 sarawak 41 kenya 1 turkey 1 selangor 159 malawi 1 uganda 2 negeri simbilan 41 maldives 2 yemen 1 terenganu 20 nigeria 2 zimbabwe 1 4 not all the respondents mentioned their state or nationality in the questionnaire. ijahp article: islam, ismail/prioritizing issues in malaysia, a developing country international journal of the 84 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 the number and percentages of the respondents who provided feedback from various states in malaysia are also shown in a pie chart (see 1). as it is shown, the highest number of respondents are from the state of selangor, (the most developed state in malaysia (new straits times 2009), followed by kelantan, perak and kuala lumpur (a federal territory). figure 1 responses received from the various states of malaysia. after obtaining the demographic information, the respondents were asked seven yes/no questions. the frequency distribution of the responses is provided in table 3. a number of observations can be made from these responses. more than 50% of the respondents said either “no” or “not sure” whether malaysia can achieve the status of developed nation by 2020. however, proportional hypothesis test does not confirm that the minority said “yes” (p = 0.1583). therefore, the responses on the question are evenly distributed between “yes” and “no” and “not sure.” johor, 44, 7% kedah, 51, 8% kelantan, 71, 11% kuala lumpur, 69, 11% melaka, 39, 6% negeri simbilan, 41, 6%pahang, 18, 3% penang, 15, 2% perak, 70, 11% perlis, 2, 0% sabah, 8, 1% sarawak, 41, 6% selangor, 159, 25% terenganu, 20, 3% ijahp article: islam, ismail/prioritizing issues in malaysia, a developing country international journal of the 85 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 table 3 analysis of people’s responses against yes/no type of questions question yes no not sure do you think that malaysia can achieve the status of developed nation by 2020? 366 (48.2) 118 (15.5) 262 (34.5) do you think that the initiatives from the government to achieve the status of developed nation are enough? 190 (25.0) 315 (41.5) 239 (31.5) do you think that the initiatives from the government to achieve the status of developed nation are working properly? 252 (33.2) 238 (31.4) 255 (33.6) is the cooperation of the people with the government to achieve developed nation sufficient? 198 (26.1) 342 (45.1) 205 (27.0) do you think that the present education system in malaysia is able to prepare the nation to be developed by 2020? 219 (28.9) 369 (48.6) 158 (20.8) is the human resource skilled enough and ready to take challenges to achieve the targets of a developed nation by 2020? 153 (20.2) 407 (53.6) 187 (24.6) are the monitoring agencies active enough to monitor the progress pertaining to all the challenges of vision 2020? 135 (17.8) 313 (41.2) 297 (39.1) in the remaining six questions, the majority of the respondents answered in the negative. in particular, the respondents do not think that: • the initiatives taken by the government to achieve the status of a developed nation are enough. • the initiatives taken by the government to achieve the status are working properly. • the cooperation of the people with the government to achieve developed nation is sufficient. • the present education system in malaysia is able to prepare the nation to be developed by 2020. • the human resource is skilled enough and ready to take challenges to achieve the targets of a developed nation by 2020. • the monitoring agencies are active enough to monitor the progress pertaining to all the challenges of vision 2020. the high percentages of responses under the column “not sure” are noteworthy. about 39% (highest percentage under “not sure” category) of the respondents are not aware about the existence of any agencies that monitor the progress of achieving the status. we ijahp article: islam, ismail/prioritizing issues in malaysia, a developing country international journal of the 86 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 also find that 53.6% (overall highest percentage) of the respondents do not think that the country’s human resource is skilled enough and ready to take up the challenges of v2020. 3. suggestions for achieving v-2020 the main purpose of the questionnaire survey was to gather the public suggestions to expedite the process of achieving the developed status. as it was anticipated, the respondents touched upon a variety of issues, and initially, the suggestions were unstructured. people were vocal about government responsibility, equitable distribution of wealth, preventing corruption and, the ubiquitous issue of economy. altogether 423 suggestions (some of these are mere comments) were received of which many are repeated a large number of times. for example, numerous people voiced out “develop human capital,” ”fair and equitable distribution of wealth,” ”eliminate corruption,” etc. all the suggestions provided by the respondents are classified by the researchers into 22 categories (see appendix). the categorization process started with the identification of the suggestions that have “affinity” among them and pertain to a central theme. for example, the common theme of the suggestions ‘ensure that the programs offered in the public and private universities are in the line with market need’ and ‘education should not be only examination oriented’ is “education”. so, these two suggestions were placed under education category. following this approach, the affinity diagram was developed. thirteen of 22 categories possess a number of distinct items. an affinity diagram has been created for these 13 categories (exhibit 1). however, islam (2005) comments that mere construction of an affinity diagram is not sufficient; rather, one needs to prioritize the items within the components of the diagram in order to know which items deserve more attention. the distinct items are singled out by the researchers and used for prioritization by the ahp. for example, the two distinct items under “freedom of expression” are “press freedom” and “public freedom of expression.” similarly, under “human capital development”, the following items can be considered as distinct: • develop more skilled workforce • develop right attitude among citizens • develop a morally upright society • malaysians should seek more knowledge and education • people should possess more positive values such as punctuality, discipline, and commitment • open malaysian mind to accept others’ ideas and thinking in fact, to frame a distinct item, few similar items have been combined. this process does not only reduce the number of items but also makes it amenable to the application of ahp (saaty, 2005). for example, under the category of “social life,” the suggestion, ijahp article: islam, ismail/prioritizing issues in malaysia, a developing country international journal of the 87 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 “inculcate sharing and caring culture in the society” is actually an integrated form of the following suggestions: • culture of caring for each other and respecting others’ opinions. • as a malaysian, (i) think it does not matter if we achieve a developed nation status successfully by 2020 unless we realize that the norm values among malaysians, namely understanding and helping each other are more important in order to maintain and remain prosperous in the country for ever. • more caring, giving and taking. • malaysian should make their heart as nice as good hearted people. • everybody should support whatever has been done by malaysia and think positive for every development. nevertheless, there are items under a specific theme that may appear to overlap one another, e.g., “eradicate corrupt practices” and “stop bribery at all levels of the society.” one may view that stopping bribery is a part eradicating corrupt practices, however, “stop bribery at all levels of the society” has been taken separately because of its standalone nature among all corrupt practices and observed high frequency among all the responses. exhibit 1 the affinity diagram of v-2020 ____________________________________________________________________ co1: remove corrupt leaders co2: eradicate corrupt practices co3: stop bribery at all levels of the society co4: replace ‘know-who’ by ‘know-how’ in awarding government projects ec1: sustainable economic development ec2: control of economy not by one nation but by malaysian nation ec3: elimination of poverty ec4: zero unemployment ec5: high level of per capita income ed1: ensure that the programs offered in the public and private universities are in the line with market need. ed2: equal opportunities of education for all races ed3: more creativity and innovation in education system ed4: enhance quality of education at the primary level ed5: education should not be only examination oriented ed6: develop excellent universities ed7: improve academic system especially in the rural areas corruption economy education ijahp article: islam, ismail/prioritizing issues in malaysia, a developing country international journal of the 88 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 en1: zero pollution en2: development without destroying the environment fe1: press freedom fe2: public freedom of expression go1: strong government with integrity and good governance go2: transparent government go3: close cooperation between private and public sectors go4: avoid bureaucracy go5: appoint a monitoring agency to oversee meeting challenges of vision 2020 go6: elect morally upright leaders in the government go7: change the mindset of politicians go8: full democracy hc1: develop more skilled workforce hc2: develop right attitude among citizens hc3: develop a morally upright society hc4: malaysians should seek more knowledge and education hc5: people should possess more positive values such as punctuality, discipline, and commitment hc6: open malaysian mind to accept others’ ideas and thinking ql1: develop rural areas ql2: improve public transportation including road infrastructure ql3: improve public health-care facilities ql4: improve telecommunication system rd1: r&d in critical sectors in agriculture and industry rd2: invite more scholars (from overseas) to contribute in r&d rd3: recognize and encourage research in universities rd4: provide enough funds for r&d in technological areas and new product development sl1: crime prevention sl2: educate citizens to be more civic-conscious sl3: inculcate sharing and caring culture in the society sl4: more freedom and no racism environment freedom of expression government responsibility human capital development quality of life r&d social life ijahp article: islam, ismail/prioritizing issues in malaysia, a developing country international journal of the 89 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 te1: develop knowledge-based society te2: strong support for scientific development te3: import expertise and technology from abroad te4: achieve technology mastery up1: punishment for wrong doers regardless of their political or racial affiliation up2: involvement of all races in governmental decision making bodies up3: ensure all ethnic groups are equally treated up4: mutual help and cooperation among all malaysians; no discrimination on the basis of race or religion un1: politicians should refrain from making ethnically discriminatory remarks un2: enhance national unity by eliminating ethnic discrimination un3: improve racial understanding and tolerance _______________________________________________________________________________ the ahp prioritization process is applied in each category of the affinity diagram with the help of five experts (two academicians, two civil servants and one businessman). the geometric mean aggregated prioritization matrices are shown in exhibit 2. the numbers are maintained in the range (1/9, 9) by considering the nearest approximation of the respective geometric means. the priorities of the items under each category are shown in table 5. exhibit 2 a sample of completed ahp pairwise comparison matrices __________________________________________________________________ co1 co2 co3 co4 ec1 ec2 ec3 ec4 ec5 co1 1 4 5 7 ec1 1 5 1/4 1 1/2 co2 1 4 3 ec2 1 1/6 1/5 1/3 co3 1 4 ec3 1 3 4 co4 1 ec4 1 3 ec5 1 ed1 ed2 ed3 ed4 ed5 ed6 ed7 en1 en2 ed1 1 3 4 3 5 1 3 en1 1 1/4 ed2 1 1/2 1/3 1 1/4 1/2 en2 1 ed3 1 1/2 5 1/3 1 ed4 1 3 1 1 fe1 fe2 ed5 1 1/5 ¼ fe1 1 3 ed6 1 2 fe2 1 ed7 1 technology unbiased practice unity ijahp article: islam, ismail/prioritizing issues in malaysia, a developing country international journal of the 90 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 gr1 gr2 gr3 gr4 gr5 gr6 gr7 gr8 gr1 1 5 4 5 7 1/3 2 2 gr2 1 1/3 3 1 1/5 1/2 ½ gr3 1 3 3 1/4 3 2 gr4 1 1/3 1/7 1/2 1 gr5 1 1/7 ½ 1 gr6 1 5 5 gr7 1 1/2 gr8 1 hc1 hc2 hc3 hc4 hc5 hc6 hc1 1 1/5 1/5 3 ¼ 2 hc2 1 1 5 1 5 hc3 1 5 2 5 hc4 1 1/4 2 hc5 1 5 hc6 1 ql1 ql2 ql3 ql4 rd1 rd2 rd3 rd4 ql1 1 2 2 2 rd1 1 4 3 3 ql2 1 1/2 3 rd2 1 1/3 1/3 ql3 1 3 rd3 1 3 ql4 1 rd4 1 sl1 sl2 sl3 sl4 te1 te2 te3 te4 sl1 1 7 8 9 te1 1 3 4 1 sl2 1 1/3 2 te2 1 3 1 sl3 1 3 te3 1 1/4 sl4 1 te4 1 up1 up2 up3 up4 un1 un2 un3 up1 1 1/3 1/4 1 un1 1 1 2 up2 1 1/3 ½ un2 1 4 up3 1 5 un3 1 up4 1 ___________________________________________________________________ table 5 priorities of the items in each category of the affinity diagram category items priorities consistency ratio category items priorities consistency ratio corruption co1 co2 co3 co4 0.583 0.238 0.121 0.057 0.14 economy ec1 ec2 ec3 ec4 ec5 0.142 0.044 0.463 0.213 0.138 0.08 ijahp article: islam, ismail/prioritizing issues in malaysia, a developing country international journal of the 91 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 education ed1 ed2 ed3 ed4 ed5 ed6 ed7 0.294 0.057 0.106 0.152 0.042 0.229 0.120 0.04 government responsibility gr1 gr2 gr3 gr4 gr5 gr6 gr7 gr8 0.224 0.052 0.123 0.037 0.052 0.364 0.072 0.076 0.07 environment en1 en2 0.200 0.800 0.00 freedom of expression fe1 fe2 0.750 0.250 0.00 human capital development hc1 hc2 hc3 hc4 hc5 hc6 0.082 0.276 0.313 0.055 0.229 0.044 0.04 quality of life ql1 ql2 ql3 ql4 0.383 0.210 0.295 0.112 0.08 r&d rd1 rd2 rd3 rd4 0.494 0.082 0.270 0.154 0.09 social life sl1 sl2 sl3 sl4 0.713 0.059 0.152 0.076 0.09 technology te1 te2 te3 te4 0.405 0.216 0.079 0.300 0.05 unbiased practice up1 up2 up3 up4 0.109 0.177 0.543 0.171 0.13 unity un1 un2 un3 0.376 0.474 0.149 0.05 from the prioritization exercise, we identify the most important suggestions (selected only the first and second rank holders) under each category and these are shown below: • remove corrupt leaders (co1) • eradicate corrupt practices (co2) • eliminate poverty (ec3) • zero unemployment (ec4) • ensure that the programs offered in the public and private universities are in the line with market need (ed1) • develop excellent universities (ed6) • develop without destroying the environment (en2) • press freedom (fe1) • elect morally upright leaders in the government (gr6) • strong government with integrity and good governance (gr1) • develop morally upright society (hc3) • develop right attitude among citizens (hc2) • develop rural areas (ql1) • improve public health-care facilities (ql3) • r&d in critical sectors in agriculture and industry (rd1) ijahp article: islam, ismail/prioritizing issues in malaysia, a developing country international journal of the 92 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 • recognize and encourage research in universities (rd3) • crime prevention (sl1) • develop knowledge-based society (te1) • achieve technology mastery (te4) • ensure that all ethnic groups are equally treated (up3) • enhance national unity by eliminating discrimination (un2) expounding on the details of all of the above suggestions is beyond the scope of this paper. nevertheless, brief accounts of some of these are provided below. elimination of corruption through empirical investigation, strauss (2001) finds that corruption affects growth and government efficiency in developing countries. mauro (1995) also found corruption as a robust negative determinant of investment, which in turn has negative implication for the rate of growth in the economy. since 1995, transparency international has been publishing an annual corruption perception index (cpi) that ranks the countries of the world according to the degree to which corruption is perceived to exist among public officials and politicians. the organization defines corruption as “the abuse of entrusted power for private gain.” however, what is legally defined to be corruption differs between jurisdictions: a political donation may be legal in some jurisdiction but illegal in another; a matter viewed as acceptable tipping in one country may be construed as bribery in another. the 2008 transparency international poll covered more than 180 countries. a higher cpi means less (perceived) corruption. according to the cpi, malaysia’s rank in 2009 is 56th in the world. this shows malaysia has ample room to improve public perception about corruption. in fact, people have been highly vocal against any kind of corruption in the government machinery. eradication of poverty poverty eradication must be the first step in the larger effort to address disparities in the socio-economic achievements of the country. malaysia’s present poverty eradication strategies include: increasing public health care to spur productivity and reduce absenteeism, providing housing, and cooperating with non-governmental organizations to improve the environment and living conditions. under the present ninth malaysian plan, the government is enhancing amanah ikhtiar malaysia and various capacity building programs to ensure their effectiveness in lifting households out of poverty. the government also wants to reduce rural-urban income ratio from 1:2.11 in 2004 to 1:2.0 by 2010. according to mahathir, no one is against the eradication of absolute poverty regardless of race and irrespective of geographical location. all malaysians, whether they live in the ijahp article: islam, ismail/prioritizing issues in malaysia, a developing country international journal of the 93 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 rural or the urban areas, whether they are in the south, north, east or west, must be moved above the line of absolute poverty. developing human capital it is the human resource that is the most vital for achieving v-2020. a country may be developed materialistically, but if its human resources are kept at bay, then it is unlikely to sustain the economic development. this was communicated to us by numerous people while collecting data collection. human capital has bearing on many aspects, including public delivery or civil service. in fact, civil servants have emphatic role in achieving v2020. to bring smoothness in public administration, bureaucracy and bottlenecks must be removed. one respondent maintained: public delivery system needs to be improved in order to realize v-2020. v-2020 requires complete public readiness to embrace the necessary changes, and to do so, it is imperative for a quantum leap toward human resource development to take place. citing japan, switzerland, and south korea, abdullah (1993) says these countries do not have minerals, and the climate, land structure and soil are not conducive to agriculture. yet they have achieved spectacular economic growth on the strength and ingenuity of their human resources. national unity according to former pm tun dr. mahathir mohamad (mohamad, 1991), the most important among all challenges is ensuring a united malaysian nation: but it would be surprising if the first strategic challenge which i have mentioned the establishment of a united malaysian nation is not likely to be the most fundamental, the most basic. the present malaysian premier, dato’ seri najib tun razak, said (sennyah and sharmini (2005): racial and religious differences, skin color and cultures should not be obstacles to unity, as, in principle, all races had the same aims and desires in life. the remaining categories (other than those included in the affinity diagram) offer many suggestions, but basically revolve around the same issue. therefore, these suggestions are not itemized as categories, and put in the affinity diagram. we have only one item under this type of category. this item has been expressed by different respondents in different ways. in the following we discuss this type of one-item categories. balanced development a country that is developed in only one dimension, e.g., economy, may not be called a developed nation as encapsulated in the statement of v-2020. the country needs to be developed multilaterally. this has been echoed by the following messages from the respondents. as seen below, the messages are essentially the same, only different words are used. ijahp article: islam, ismail/prioritizing issues in malaysia, a developing country international journal of the 94 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 • in my opinion development that we, malaysians, should have, must be balanced in all aspects of life, morally, intellectually, economically, spiritually, socially and politically5. • a more comprehensive meaning of a developed nation so that rather than focusing solely on material/physical development, intellectual, moral and spiritual development will be seen as important as well. • the parameter and definition of development need to be ascertained. development is a holistic concept. infrastructural development alone at the expense of moral, ethical, and social developments may not be a development in the true sense of the term. as such competing to stay at development stage must not ignore the social engineering of the society on moral and spiritual bases. otherwise development for the sake of development without adequate moral and spiritual bases to cushion the shocks of physical development may merely lead to devil-upment (and not development)! • not focused on one area only – must be balanced in all areas. public responsibility the responsibility of realizing v-2020 should not be solely shouldered by the government or a few individuals; rather every citizen of the nation has a stake towards this achievement. this has been articulated by the respondents in manifold ways: • everybody has to play a role in achieving the developed nation status. • malaysians must support all the initiatives taken by the government and new policies promulgated by it. • every citizen should change his or her attitude and mentality to achieve v-2020. • if malaysia does not achieve the status of developed nation by the year 2020, everybody should triple up efforts to achieve the targets. communication as mentioned, achievement of the national vision cannot only be the responsibility of a certain number of individuals; in fact, it is the responsibility of all the people. therefore, a nationwide awareness about v-2020 must be carried out. this matter has been articulated by the people in the following ways: • create awareness in the malaysian society about how important this v-2020 is. • to educate the malaysians more about the v-2020 nine challenges. • i think government should update the public regarding the status of achievement of v-2020. • to educate the malaysians more about the v-2020 nine challenges. ethics regarding ethics, people voice that if the citizens do not behave ethically or morally, then mere materialistic development has no meaning. ethical dimension of the society has been encapsulated by the following suggestions: 5 while articulating the suggestions, all the respondents have not followed the grammatical rules strictly. we have presented the suggestions as articulated by the respondents. ijahp article: islam, ismail/prioritizing issues in malaysia, a developing country international journal of the 95 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 • ethical dimension should be progressively addressed, tackled and solved. unless this aspect of morality is not solved, achievement in other aspects is virtually not possible. • we should establish a fully moral and ethical society before we achieve a developed nation. • first and foremost, malaysians should be aware that in order to develop a nation, they must develop themselves with strong moral and ethical values. • society has to change its way of thinking, more open minded, work honest and fair. family institution many people have re-emphasized the adage, “charity begins at home”. children must be guided properly from the beginning about ethics and morality, and parents have a great role here. the right attitude and integrity in the early stage pave the way for becoming a good citizen in future. this matter has been expressed by the following suggestions: • to strengthen the family leadership. • honor family institution. do not put parents in the condition where they have to work until they leave their children at the hand of foreigners (foreign maids). if family intuition is honored country will also follow suit. • try to develop and nurture the children from the beginning with high ethical and moral values. • teaching moral values to the kids right from early stage. benchmarking benchmarking is a management practice used for achieving radical change in some particular area. in the business context, organizations benchmark others to improve their business processes. there are numerous examples of successful benchmarking exercises (andersen, et al., 2008; fawcett, et al., 2009). similar exercises can be conducted at the national level. for example, malaysia can learn from japan regarding how the country maintains its environment and ecosystem while being engaged in developmental activities. what are the strategies of japan in r&d, transportation, public health, education, etc.? malaysia can learn from all these. a number of respondents have supported this idea: • to follow japan and korea, how they became developed nations. • collaboration with developed countries. spiritual development in general, malaysians are religious minded and they like to see that spiritual development takes place in the minds of all malaysians in tandem with the socioeconomic development of the country. a large number of malaysians’ view is that ijahp article: islam, ismail/prioritizing issues in malaysia, a developing country international journal of the 96 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 materialistic development has no meaning if there is no mental peace among them! the following suggestions are examples of some of their aspiration of dealing with a developed religious life: • no matter how developed our country is, the development of spiritual and mental is very important. • strengthen the spiritual side first. • material development must be complemented with social and spiritual development. more emphasis should be placed to social development and also spiritual development. 4. conclusions achieving the status of a developed country is a vision common to many developing nations. like india, malaysia has even set up a specific timeline (2020) to reach the milestone. the present research is carried out to gauge the progress made to achieve malaysian v-2020. in general, a cross section of the people of the nation believe that many more things need to be done before malaysia can declare herself a developed nation. in addition to the economical aspect, the country also needs to pay serious attention to human resource development – especially prevention of crime and corruption from all levels of the larger society. the survey respondents suggested that the government should play a dual role: on one hand, the government should play its traditional role in strengthening the economy, ensuring quality education to its people and maintaining law and order; on the other hand, it should make the people ready to face the challenges ahead and take the necessary steps to address them successfully. references abdullah, o.y. (1993). human resource development: the key towards a developed and industrialized society. in: hamid, asa, editor. malaysia’s vision 2020: understanding the concept, implications and challenges (p. 315-326), kuala lumpur: pelanduk publications. andersen, b., henriksen, b., & spjelkavik, i. (2008). benchmarking applications in public sector principal-agent relationships. benchmarking: an international journal, 15(6), 723-741. fawcett, s.e., wallin, c., allred, c., magnan, g. (2009). supply chain information sharing: benchmarking a proven path. benchmarking: an international journal, 16(2), 222-246. islam, r. (2005). prioritization of ideas in an affinity diagram by ahp: an example of keconomy. iium journal of economics and management, 13(1), 71-108. ijahp article: islam, ismail/prioritizing issues in malaysia, a developing country international journal of the 97 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 kassim, m.s.m. (1993). vision 2020: its linkages with the sixth malaysian plan and the second outline perspective plan. in: hamid asa, editor. malaysia’s vision 2020: understanding the concept, implications and challenges (p. 67-87), kuala lumpur: pelanduk publications. lim, t. (2009). one malaysia. new straits times, may 31, p. r8. mauro, p. (1995). corruption and growth. quarterly journal of economics, cx(442), 681-712. mohamad , m. (1991). the way forward: vision 2020. www.epu.jpm.my, accessed on april 28, 2008. rahman, o.a. (1993). industrial targets of vision 2020: the science and technology perspective. in: hamid asa, editor. malaysia’s vision 2020: understanding the concept, implications and challenges (p. 271-299), kuala lumpur: pelanduk publications. saaty, t.l. (2005). theory and applications of the analytic network process: decision making with benefits, opportunities, costs, and risks. pittsburgh, pa: rws publications. sennyah, p. & sharmini, p. (2005). go for lifelong education. new straits times, may 17, p. 2. staff reporter (2005). selangor developed state: did you know? new straits times, august 27, p. 30. strauss, t. (2001). growth and government: is there a difference between developed and developing countries? economics of governance, 2, 135-157. appendix: extended affinity diagram • in my opinion, development that we, malaysians should have must be balanced in all aspects of life morally, intellectually, economically, spiritually, socially and politically. so we should not follow the development created by western society • develop intellectually, culturally, religiously and scientifically balanced development ijahp article: islam, ismail/prioritizing issues in malaysia, a developing country international journal of the 98 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 • a more comprehensive meaning of developed nation rather than focusing on material/physical development but intellectual, moral and spiritual development as well • the parameter and definition of development need to be ascertained. development is a holistic concept. infrastructural development alone at the expense of moral, ethical, and social developments may not be a development in the true sense of the term. as such competing to stay at development stage must not ignore the social engineering of the society on moral and spiritual bases. otherwise development for the sake of development without adequate moral and spiritual bases to cushion the shocks of physical development may merely lead to devil-upment (and not development)! • we should balance all aspects of life in order to achieve status of a developed nation • i think a developed nation also needs to have advances and improvements in all aspects – that also includes social and civil institutions. this is an area which malaysia should look into • not focused on one area only – must be balanced in all areas • to follow japan and korea and learn how they became developed nations • collaboration with developed countries • less foreign workers • penetration in foreign markets • emphasis should be on convincing the people and the world that we are serious in that direction • create awareness about vision 2020 in all levels of the nation • i think government should be alert that malaysians still need to be educated with more information about vision and mission 2020 • information regarding vision 2020 should be spread out appropriately • to educate the malaysian more about the 9 challenges of vision 2020 • i believe that the status of a developed nation must be communicated to all malaysians as well as all areas (rural and urban) communication business benchmarking ijahp article: islam, ismail/prioritizing issues in malaysia, a developing country international journal of the 99 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 • malaysia needs to demolish all of the corruption from lower level to higher level of hierarchy of citizens • creation of highly ethical society, free from corrupt practices and encourage healthy competition based on merit • be more civic minded and reduce corruption. corruption will only give malaysia a bad name in the eyes of other nations • malaysia should improve its administration first and stress on ‘zero corruption’ and then malaysia can continue to think about vision 2020 • corrupt leaders must be removed • eradicate corrupt practices • stop bribery among police forces and customs • replace ‘know who’ by ‘know how’ in awarding projects to quality bidder, avoid corruptions and have transparency when awarding projects • no bribery • sustainable economic development • control of economy not by one nation but by the whole nation • malaysia should be free from corruption, poverty and not dependent too much on foreign labor • government should take initiatives to energize investment and business opportunities • eliminate poverty • distribution of wealth equally to all races • fairness in wealth distribution • focus on industrialization rather than agriculture. all the developed countries are industrial, then how can malaysia become a developed country by agriculture? • fair income distribution • eliminate wealth gap (still significant) between chinese and malay • stabilize malaysian economy • zero unemployment • high level per capita income • improve on all areas that can contribute towards improvement of vision 2020, especially the basics of education • i suggest education system to be more effective • educate the people • improve the education system • change the education system education corruption economy ijahp article: islam, ismail/prioritizing issues in malaysia, a developing country international journal of the 100 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 • national education policy should be implemented fully as planned and do not be swayed by political pressure or ethnic pressure groups • ensure that the programs offered in the public and private universities are in the line with market need • revisit educational system • by increasing more programs on education, especially through electronic media • equal in education to all races without looking into their color or religion. be fair in education to all. a form 5 student or form 6 student has the results must be given equal chance to enter university and not on ‘quota’. • entrance to higher education should be by merit not by racial quota • strengthen education system that meet current challenges • education system: more creativity and innovation • free education for everyone from standard 1 to form 5. better training facilities for form 5 school leavers • enhancement in the lower level of education which is primary school • the education system should be reformatted as up-to-date, not only examination oriented • revamp of education system – start school early and inculcate a merit-based system for all levels of study • the education system should be less inclined towards “exam-oriented” culture • education system should prepare one to be creative, innovative, independent and ethical • refine education system, don’t focus on exam, let the students enjoy the learning in more creative and innovative ways • revamp education system to take into consideration of hard and soft skill • stress on education to society • develop excellent universities in malaysia • i would suggest that universities play pivotal role in disseminating good values through its students who become leaders in society after completing their studies • malaysian education system especially on the university level have not reached to the high standard. the government should do something like: bring up more foreigners to local universities • refer education policy to serve its multi-ethnic nations • improve academic system especially in the rural areas • ensure safe environment for the whole nation to live in malaysia • all development projects should be environment friendly and preserve the resources for sustainable development • development without destructing the environment • zero pollution • protection of environment/conservation environment ijahp article: islam, ismail/prioritizing issues in malaysia, a developing country international journal of the 101 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 • ethical dimension should be progressively addressed, tackled and solved. unless this aspect of morality is not addressed, achievement in other aspects virtually not possible • we should establish a fully moral and ethical society before we achieve a developed nation • first and foremost, malaysians should be aware that in order to develop a nation, they must develop themselves with strong moral and ethical values. particularly as muslims, we should adhere to the basics and fundamentals of islam. then only we can speak about developed nation. • society have to change their way of thinking, more open minded, work really honest and fair. • to strengthen the family leadership • honor family institution. do not put parents in the condition where they have to work until they leave their children at the hand of foreign maids. if family intuition is honored country, will also follow suit • try to develop and nurture the children from the beginning with high ethical and moral values (this is the very must step that should be considered) • teaching moral values to the kids right from the early stage • press freedom • fair justice to all malaysians – to voice out their opinion!!! • think tank should be an important tool in achieving developed nation status, people should not be afraid to express innovative/impossible ideas for new creations • more freedom in the expression of speech • full achievement of developed nation status can be attained by 2020 if all the races in malaysia cooperate and they are aware of all activities and issues facing or planning by malaysian government • strong government with integrity and good governance is a must • transparent government • private sector and government should cooperate to achieve malaysian vision 2020. good luck! freedom of expression government responsibility ethics family ijahp article: islam, ismail/prioritizing issues in malaysia, a developing country international journal of the 102 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 • government should show more enthusiasm in realizing the goals of vision 2020 • government and private sector should work together, hand in hand • develop rural area, there are a lot of rural areas need development and good politicians in order to contribute to the economic growth • there are also lack of good educational institutions in the rural areas where a lot of teenagers are neglected/denied from gaining knowledge • avoid bureaucracy • promotion in government sectors for delivery service to public service should be by merit. this is to allow the best persons to serve the public to enhance full development • one bahasa should be english in order to go for globalization • the monitoring agencies of the public sector must be proactive enough to constantly examine and check the objectives of vision 2020 that these are fully achieved as stipulated. an steering committee on vision 2020 should be set up and to be answerable to the pm to check the performance and achievement of all challenges being set. • free and fair election process • hire foreigners to educate malaysians if malaysia does not have someone who is very good at one aspect • to upgrade political life to be more democratic • elect leaders who view positions of leadership as responsibility and not just privilege • development should not be concentrated on urban areas, it should include rural areas as well • government should be more transparent in awarding contracts • the government should be more open in terms of accepting differences of opinions from the public • monitor regularly • close monitoring system must be in place and has to be communicated to the people. the government must be sincere in telling the people the level of achievement and what immediate improvement plans are put forward • government should improve the monitoring the progress towards vision 2020 • punish severely those not conforming and performing • let the best run the job and forgo national pride • depending on foreigners, the government cannot run for long time. also they should look at the moral and ethical sides as well • implementing ict policy by the government with good planning and monitoring system is essential in order to bring benefits for all segments of the society • focus on human capital development • educate people who can produce a civilization. focus more on people investment and culture and mindset are also important • willing to learn new things and equip themselves with technical know how human capital development ijahp article: islam, ismail/prioritizing issues in malaysia, a developing country international journal of the 103 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 • invest on human capital • human resource development should be focused on acquiring skills essential in industrial development • stress on hard and soft skills • invest on skills and education field especially to the university in order to train the young generation towards this goal • we have to be knowledgeable people to compete with a more developed nation. our human capital need to be restructured and realigned with a more positive mindset and think global • to have the right attitude, not necessarily the right skills • first of all, a developed country is created not only in industrial sector but also with society. actually, in this point, malaysian society (people) must be more skilled and in this way more confident • to increase the amount of skilled manpower in the industry • status of developed nation is nothing if the citizens have no values and moral in their lives. as long as many people are not able to develop themselves towards the positive lifestyle (mostly on spiritual), it might be difficult to achieve a developed nation. or may be vision 2020 just a tool for nothing • malaysians should be more independent and seek for more knowledge and education • focus on the attitudes and behaviors of people to make them more ethical and conscientious • the people should possess more positive values such as punctuality, discipline and commitment • should encourage on soft skills rather than physical development • focus on human resource development – people should be encouraged to think freely, creatively challenge the norms while working together hand in hand with the government for the betterment of the society • we need to be mentally developed before we can achieve our vision 2020 • need to change the ‘subsidy’ attitude of malays • emphasize on human capital development and technological development at the same time • human resource development – highly skilled, motivated, possess some vision • be critical, work harder and be far sighted • change the mindset of malaysians • shifts of paradigm, way of thinking among its people and government itself • not just give subsidies and help but focus more towards changing the malaysian way of thinking, especially bumiputra. if government just spoon fed them, may be they cannot stand on their own. and here i would like to highlight that make and improve our country but don’t neglect our moral, and ethical as a ‘malay’ country that rich with “budi bahasa” • mental change of the population • open their minds to accept others’ ideas and thinking ijahp article: islam, ismail/prioritizing issues in malaysia, a developing country international journal of the 104 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 • everybody has to play a role in achieving the developed nation status • all of us should cooperate to develop nation by 2020 and more programs should be implemented through all races and communities • government does so many things and malaysians must support those • every citizen should change his or her attitude and mentality to achieve vision 2020 • if malaysians do not achieve status of developed nation by the year 2020, everybody should triple up efforts to achieve the targets • more stringent actions and monitoring programs should be in place so that the vision 2020 would reach its goal. all malaysians, including the government and private sector should be playing active roles to encourage/stimulate the development. everyone should work together to achieve the vision 2020 • acknowledgement by all society and nations in all aspects can slowly help achieve developed status • maintain all the policies and make sure all malaysians follow the procedures set by the government • everybody should support whatever has been done by malaysia and think positive for every development. now we can see malaysia has klcc, twin tower, kl tower, the big shopping complexes and son on • make malaysia independent in terms of food by putting sufficient importance on agriculture. life gets better when food is cheaper • public transportation not tip top • public health care not up to a developed nation status • to find ways to curb the nation’s traffic congestion • improve the living standard in the rural areas. there are many people in malaysia, help them first • public transportation system must be improved • road infrastructure and telecommunication are two important criteria • r&d in critical sectors – agriculture and industry • inviting more scholars to contribute in the area of research and development • recognize and encourage research in universities • provide enough funds for r&d and all technological areas and new product development quality of life r&d public responsibility ijahp article: islam, ismail/prioritizing issues in malaysia, a developing country international journal of the 105 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 • attitudes of teenagers must be changed. social lives and morality must be closely related. example: 1. mat rempit, 2. internet (x-rated), 3. dadah/ecstacy pill. start from house than school/ipt. give them strong akidah. • focus on moral, ethical development • malaysia’s crime rate is high. it is important to have a civilized society. our county must be clean and healthy place to live in • better rural and urban planning and development to avoid unexpected flood due to irresponsible housing projects and to provide comfortable, reliable and affordable public transportation • more freedom, and less racism • fulfill local needs: it literacy, enhance educational system, solve integrity problems in the society • malaysians should make their hearts as nice as a good hearted people • malaysia should be more courteous and should open their hearts • educate citizens to be more civic-conscious • to educate the citizens to be more civilized • focus on society development – caring, high moral and responsibility • more caring, giving and taking • as a malaysian, think it does not matter if we achieved a developed nation status successfully by 2020 unless we realize that the norm values among malaysians by understanding and helping each other are more important in order to maintain and remain prosperous in the country for ever • need to establish a fully moral and ethical society and a fully caring society • culture of caring each other and respect others’ opinion • increase the moral values in the society • the most important thing is our intention. the rules are already in quran and sunnah. just apply it in our daily life. insha allah, our efforts will be blessed by allah swt • allah had created the rules in quran and sunnah, so why people refuse to implement what our creator said. just implement whatever in quran and sunnah. • no matter how developed our country is, the development of spiritual and mental is very important • strengthen the spiritual side first • more and advanced religious approaches should be properly induced to the people • teach malaysians how to live civilized by instilling and applying islamic concepts. islam is the best way of life • material development must be complemented with social and spiritual development. more emphasis should be given to social development and also spiritual development social life spirituality ijahp article: islam, ismail/prioritizing issues in malaysia, a developing country international journal of the 106 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 • there is a contradiction in the economic growth and social/spiritual growth at the moment. we are improving economically (physically) but the social/spiritual side is suffering. there is no inner peace in the majority of the population. the gap between the rich and poor is widening • islamic principle of life must be infused in everything including the public and private sectors of the nations • islamic basis should be strong • islamic practice, now the country is not fully practicing the islamic laws • back to basics, by adopting every aspect through islamic approach. this will create society that have all the good aspects by devoting themselves to allah • knowledge-based and cultural/religious tolerance • real support for scientific development as well as moral and values • form a body to encourage inventions, innovations competition in science and technology • technology: transfer/import from others and learn not only copy • wider ‘research and development’ field. we should be pioneer, not a good follower • science and technology need to be improved in order to compete with other company and also in order to achieve the vision • introduce a lot more of technological facilities as well as implement the concept of islam hadhari (enlighten islam) • import expertise and technology from abroad • open , fair and transparent government • make it more transparent and responsive to people’s needs. • let there be true democracy where every malaysian is treated equally. • those who commit offences or any wrong doing should be punished regardless of their political party affiliation • government should really check about the programmes that have been planned in order to develop the nation. those programmes should involve all the races in the country so that everyone can contribute to develop the nation • mutual help and cooperation among malaysians, do not differentiate on race, religion and nation • ensure all ethnic groups in malaysia are equally treated technology unbiased practice transparency ijahp article: islam, ismail/prioritizing issues in malaysia, a developing country international journal of the 107 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 • integration of the nation in all aspects should be improved • unite all the malaysians • to achieve a developed nation status, we must improve our society, we need to be more mature, ethical and united with each other • emphasize more on malaysian unity and strength for the malaysian nation • ensure creation of one nation is fully achieved • enhance national unity by reducing ethnic discrimination • improve racial understanding and tolerance • all malaysian societies must be united as malaysian nation. government should play the role to unite malaysians as a whole in one nation, then the other steps can run smoothly to achieve vision 2020 • establish a united malaysia (bangsa malaysia) unity microsoft word 4fiala_pp109-120_vol_i_isue_2_2009-.docx ijahp article: fiala/using an analytic network process model in combinatorial auctions international journal of the 109 vol. 1 issue 1 2009 issn 19366744 analytic hierarchy process using an analytic network process model in combinatorial auctions 1 petr fiala department of econometrics, university of economics praha, 130 67, czech republic e-mail: pfiala@vse.cz abstract auctions are important market mechanisms for the allocation of goods and services. auctions are often preferred to other common processes because they are open, quite fair, and easy to understand by participants, and lead to economically efficient outcomes. design of auctions is a multidisciplinary effort made of contributions from economics, operations research, informatics, and other disciplines. combinatorial auctions are those auctions in which bidders can place bids on combinations of items, called “packages,” rather than just individual items. the advantage of combinatorial auctions is that the bidder can more fully express his preferences. this is particularly important when items are complements. the multiple evaluation criteria can be used. there are dependencies among sellers, buyers, criteria and bundles of items. a variety of feedback processes creates complex system of items. the whole structure seems to be very appropriate to the analytic network process (anp) approach. the anp method makes it possible to deal systematically with all kinds of dependence and feedback in the system of items. by using the anp approach, the preferences of bundles of items can be evaluated. dynamic network process (dnp) as an extension of anp can deal with time-dependent priorities in combinatorial auctions. keywords: combinatorial auctions, preference elicitation, analytic network process, dynamic network process introduction auctions are important market mechanisms for the allocation of goods and services. many modern markets are organized as auctions. combinatorial auctions are those auctions in which bidders can place bids on combinations of items, so called bundles. the advantage of combinatorial auctions is that the bidder can more fully express his preferences. this is particularly important when items are complements. the auction designer also derives value from combinatorial auctions. allowing bidders more fully to express preferences often leads to improved economic efficiency and greater auction overall valuation. however, alongside their advantages, combinatorial auctions raise a host of questions and challenges (cramton et al. 2006, rothkopf et al. 1998). auction theory has caught tremendous interest from both the economic side as well as the internet industry. an auction is a competitive mechanism to allocate resources to buyers based on predefined rules. these rules define the bidding process, how the winner is 1 the paper is supported by the grant agency of czech republic (grant 402/07/0166 “combinatorial auctions – modeling and analysis“). rob typewritten text http://dx.doi.org/10.13033/ijahp.v1i2.47 ijahp article: fiala/using an analytic network process model in combinatorial auctions international journal of the 110 vol. 1 issue 1 2009 issn 19366744 analytic hierarchy process determined, and the final agreement. in electronic commerce transactions, software agents that negotiate on behalf of buyers and sellers conduct auctions. the popularity of auctions and the requirements of e-business have led to growing interest in the development of complex trading models (see bellosta et al. 2004, bichler 2000, oliveira et al. 1999). classification of auctions is based on some specific characteristics as: • the numbers of sellers and buyers, • the number of items, • traded items (indivisible, divisible, pure commodities, structured commodities), • participants’ roles in auctions (one-sided, multilateral auctions), • preferences of the participants, • the form of the private information participants have about preference, • objectives of auctions (optimization, allocation rules, pricing rules), • evaluating criteria, • complexity of bids (simply, related bids), • organization of auctions (single-round, multi-round, sequential, parallel, price schemes). many specific factors of combinatorial auctions can be modeled and analyzed by analytic network process and dynamic network process (anp/dnp) methods. 1. multicriteria combinatorial auctions for our model, we propose to use multidimensional auctions. these auctions can be classified: • multi-unit auction, • multi-item auction, • multi-criteria auction, multi-unit auctions contain multiple units of items and make possible volume discount auction. multi-item auctions can place bids on combinations of items, so called combinatorial auctions. the combinatorial auctions can define multiple criteria as: • revenue maximization the seller should extract the highest possible price, • efficiency the buyers with the highest valuation get the goods, • collusion possibility, auctions with complex bid structures are also called multicriteria auctions because they address multiple attributes of the items (quality, quantity, price) in the negotiation space. multicriteria optimization can be helpful for detailed analysis of combinatorial auctions. buyers can specify aspiration levels that express their desired values on the attributes of the items to be purchased and reservation levels that represent the minimal values required. there are possible combinations of the multidimensional characteristics as multi-criteria, multi-unit, multi-item auctions. multicriteria auctions require several key components to automate the process: ijahp article: fiala/using an analytic network process model in combinatorial auctions international journal of the 111 vol. 1 issue 1 2009 issn 19366744 analytic hierarchy process • a preference model, • a multicriteria optimization model, • a negotiation model. the preference model is used to let the buyer express his preferences. the multicriteria optimization model selects the best offer for the buyer. the negotiation model helps to find a consensus in auctions (fiala, 1997). 2. winner determination problem the problem, called the winner determination problem, has received considerable attention in the literature. the problem is formulated as: given a set of bids in a combinatorial auction, find an allocation of items to bidders to maximize the seller's revenue. it has introduced many important ideas, such as the mathematical programming formulation of the winner determination problem, the connection between the winner determination problem and the set packing problem, as well as the issue of complexity. there are some different approaches to solve winner determination problems. main approaches for solving the problem are exact or approximate algorithms. the algorithms have some pros and cons. we propose to use an iterative approach for solving winner determination problems. the iterative approach is based on a primal-dual algorithm. problem formulation many types of combinatorial auctions can be formulated as mathematical programming problems. from different types of combinatorial auctions we present an auction of indivisible items with one seller and several buyers. let us suppose that one seller offers a set g of m items, j = 1, 2, …, m, to n potential buyers. items are available in single units. a bid made by buyer i, i = 1, 2, …, n, is defined as bi = {s, vi(s)}, where s ⊆ m, is a combination of items, vi(s), is the valuation by buyer i for the combination of items s, the valuation is made by multiple criteria. the objective is to maximize the overall valuation of the seller given the bids made by buyers. constraints establish that no single item is allocated to more than one buyer and that no buyer obtains more than one combination. bivalent variables are introduced for model formulation: xi(s) is a bivalent variable specifying if the combination s is assigned to buyer i (xi(s) = 1). the winner determination problem can be formulated as follows ijahp article: fiala/using an analytic network process model in combinatorial auctions international journal of the 112 vol. 1 issue 1 2009 issn 19366744 analytic hierarchy process ∑ = n i 1 ∑ ⊆ms vi(s) xi(s) → max subject to ∑ ⊆ms xi(s) ≦ 1, ∀ i, i = 1, 2, …, n, ∑ = n i 1 ∑ ⊆ms xi(s) ≦ 1, ∀ j ∊ m, (1) xi(s) ∊ {0, 1}, ∀ s ⊆ m, ∀ i, i = 1, 2, …, n. the objective function expresses the overall valuation. the first constraint ensures that no bidder receives more than one combination of items. the second constraint ensures that overlapping sets of items are never assigned. the winner determination problem, i.e., determining the items that each bidder wins, is not difficult in the case of non-combinatorial auctions. it would take o(nm) time where n is the number of bidders and m is the number of items. but in the case of combinatorial auctions, the winner determination problem is much more complex. the payment determination problem also is an important and non-trivial problem. complexity of the problem complexity is a fundamental question in combinatorial auction design. there are some types of complexity: • computational complexity • valuation complexity • strategic complexity • communication complexity computational complexity covers such questions as: how much computation is expected of the mechanism to compute an outcome given the bid information of the bidders? this is an extremely important question because winner determination problem is an np-complete optimization problem. the winner determination problem turns out to be an instance of a weighted set packing problem. the weighted set packing problem is a problem of finding a disjoint collection of weighted subsets of a larger set with maximal total weight. weighted set packing is a classical np-complete problem. valuation complexity deals with such questions as: how much computation is required to provide preference information within a mechanism? estimating every possible bundle of items requires exponential space and hence exponential time. bidders need to determine valuations for 2m -1 possible bundles. strategic complexity concerns such questions: which of the 2m -1 bundles to bid on? what is the best strategy for bidding? must bidders model behavior of other bidders and ijahp article: fiala/using an analytic network process model in combinatorial auctions international journal of the 113 vol. 1 issue 1 2009 issn 19366744 analytic hierarchy process solve problems to compute an optimal strategy? for instance, in a sealed bid combinatorial procurement scenario, sellers must not only take their valuation of the bundles into consideration but also the bidding behavior of their competitors. this requires sophisticated bidding logic. communication complexity concerns such questions as: how much communication is required between bidders and auctioneer until an equilibrium price is reached the mechanism to compute an outcome? the amount of communication between the bidders and the auctioneer can become quite high. for instance, in an iterative combinatorial auction, where individual valuations are revealed progressively in an iterative manner, the communication costs could be high if the auction were conducted in a distributed manner over space and/or time. the problem of communication complexity can be addressed through the design of careful bidding languages that provide expressive but concise bids. solving the problem the algorithms proposed for solving the winner determination problem fall into two classes: • exact algorithms • approximate algorithms exact approaches to solving the winner determination problem require algorithms that generate both good lower and upper bounds on the maximum objective function value. in general, the upper bound on the optimal solution value is obtained by solving a relaxation of the optimization problem. there are two standard relaxations for winner determination problem: lagrangean relaxation and the linear programming relaxation. by lagrangean relaxation the feasible set is usually required to maintain 0-1 feasibility, but many if not all of the constraints are moved to the objective function with a penalty term. by lp relaxation only the integrality constraints are relaxed, the objective function remains the original function. exact methods come in three varieties: branch and bound, cutting planes and a hybrid called branch and cut. integer programming techniques can be used to handle winner determination in combinatorial auctions with a small enough number of items. there are a few special cases where the winner determination problem can be solved exactly using polynomial time algorithms. for example, paper (de vries and vohra. 2003) looks at polynomially solvable instances of the winner determination problem in terms of the structure of the constraint matrix: totally unimodular matrices, balanced matrices, and perfect matrices. a matrix is said to be totally unimodular if the determinant of every square submatrix is 0, 1, or −1. a 0-1 matrix is called balanced if it has no submatrix of odd order with exactly two 1’s in each row and column. a matrix is said to be perfect if it is the vertex-clique adjacency matrix of a perfect graph. in all these cases, it is shown that the winner determination problem can be solved as a linear program. approximate algorithms have also emerged as a major approach to solving the allocation problem in combinatorial auctions. one way of dealing with hard integer programs is to give up on finding the optimal solution. an approximation algorithm is a polynomial time ijahp article: fiala/using an analytic network process model in combinatorial auctions international journal of the 114 vol. 1 issue 1 2009 issn 19366744 analytic hierarchy process algorithm with a provable performance guarantee. approximate algorithm seeks a feasible solution fast and hopes that it is near optimal. fast approximation algorithms for the winner determination problem can be obtained by translating results for combinatorial problems related to winner determination, mainly the weighted set packing problem and the weighted stable set problem to the context of winner determination (sandholm 2002). approximate algorithms and heuristics include greedy algorithms, simulated annealing, genetic algorithms and others. 3. preference elicitation the key feature that makes combinatorial auctions most appealing is the ability for bidders to express complex preferences over bundles of items, involving complementarity and substitutability. items are complements when a set of items has greater utility than the sum of the utilities for the individual items. items are substitutes when a set of items has less utility than the sum of the utilities for the individual items. two items a and b are complementary, if it holds v({a, b}) > v({a}) + v({b}). two items a and b are substitute, if it holds v({a, b}) < v({a}) + v({b}). different elicitation algorithms may require different means of representing the information obtained by bidders. sandholm and boutilier (2006) describe a general method for representing an incompletely specified valuation functions. a constraint network is a labeled directed graph consisting of one node for each bundle b representing the elicitor's knowledge of the preferences of a bidder. a directed edge (a, b) indicates that bundle a is preferred to bundle b. figure 1 represents an example of a constraint network for bundles of three items (a,b,c). figure 1 constraint network. {a,b,c} {a,b} {a,c} {a} {b,c} {b} {c} ∅ ijahp article: fiala/using an analytic network process model in combinatorial auctions international journal of the 115 vol. 1 issue 1 2009 issn 19366744 analytic hierarchy process the constraint network representation is useful conceptually, and can be represented explicitly for use in various elicitation algorithms. but its explicit representation is generally tractable only for small problems, since it contains 2m nodes. preference elicitation of bundles in a constraint network can be used analytic network process. the analytic hierarchy process (ahp) is a method for setting priorities (saaty, 1996). a priority scale based on reference is the ahp way to standardize non-unique scales in order to combine multiple performance measures. the ahp derives ratio scale priorities by making paired comparisons of elements on a common hierarchy level by using a 1 to 9 scale of absolute numbers. the absolute number from the scale is an approximation to the ratio wj/wk and then is possible to derive values of wj and wk. the ahp method uses the general model for synthesis of the performance measures in the hierarchical structure. j 1 n i jk j u v w = = ∑ . the analytic network process (anp) is the method (saaty, 2001) that makes it possible to deal systematically with all kinds of dependence and feedback in the performance system. the well-known ahp theory is a special case of the analytic network process that can be very useful for incorporating linkages in the system. the structure of the anp model is described by clusters of elements connected by their dependence on one another. a cluster groups elements that share a set of attributes. at least one element in each of these clusters is connected to some element in another cluster. these connections indicate the flow of influence between the elements (see figure 2). sellers criteria buyers bundles figure 2 clusters and connections in multicriteria combinatorial auctions. the clusters in multicriteria combinatorial auctions can be sellers, buyers, bundles of items, and evaluating criteria. paired comparisons are inputs for preference elicitation in combinatorial auctions. a supermatrix is a matrix of all elements by all elements. the weights from the paired comparisons are placed in the appropriate column of the supermatrix. the sum of each column corresponds to the number of comparison sets. the weights in the column corresponding to the cluster are multiplied by the weight of the cluster. each column of the weighted supermatrix sums to one and the matrix is column stochastic. its powers can stabilize after some iteration to the limit supermatrix. the columns of each block of the matrix are identical in many cases, though not always, and we can read off the global priority of the elements. ijahp article: fiala/using an analytic network process model in combinatorial auctions international journal of the 116 vol. 1 issue 1 2009 issn 19366744 analytic hierarchy process recent work has focused on the question of how to limit the amount of valuation information provided by bidders by adaptively limiting the precision of the bids that are specified. 4. iterative combinatorial auctions in the iterative approach, there are multiple rounds of bidding and allocation and the problem is solved in an iterative and incremental way. iterative combinatorial auctions with multiple criteria are proposed in this paper as complex trading models. one way of reducing some of the computational burden in solving the winner determination problem is to set up a fictitious market that will determine an allocation and prices in a decentralized way. in the iterative approach, there are multiple rounds of bidding and allocations and the problem is solved in an iterative and incremental way. iterative combinatorial auctions are attractive to bidders because they learn about their rivals' valuations through the bidding process, which could help them to adjust their own bids. for analysis of iterative combinatorial auctions, this paper proposes to use dynamic version of anp. auctions have emerged as a particularly interesting tool for negotiations. combinatorial auctions provide a mechanism for negotiation between buyers and sellers. various concepts of negotiation models can be used for modeling of combinatorial auctions. there is a connection between efficient auctions for many items and duality theory. the vickrey auction can be taken as an efficient pricing equilibrium, which corresponds to the optimal solution of a particular linear programming problem and its dual. the simplex algorithm can be taken as a static approach to determining the vickrey outcome. alternatively, the primal-dual algorithm can be taken as a decentralized and dynamic method of determining the pricing equilibrium. a primal-dual algorithm usually maintains a feasible dual solution and tries to compute a primal solution that is both feasible and satisfies the complementary slackness conditions. if such a solution is found, the algorithm terminates. otherwise the dual solution is updated towards optimality, and the algorithm continues with the next iteration. the fundamental work (bikhchandani and ostroy 2002) demonstrates a strong interrelationship between the iterative auctions and the primal-dual linear programming algorithms. a primal-dual linear programming algorithm can be interpreted as an auction where the dual variables represent item prices. the algorithm maintains a feasible allocation and a price set, and terminates as the efficient allocation and competitive equilibrium prices are found. for the winner determination problem, we will formulate the lp relaxation and its dual. consider the lp relaxation of the winner determination problem (1): ∑ = n i 1 ∑ ⊆ms vi(s) xi(s) → max subject to ijahp article: fiala/using an analytic network process model in combinatorial auctions international journal of the 117 vol. 1 issue 1 2009 issn 19366744 analytic hierarchy process ∑ ⊆ms xi(s) ≦ 1, ∀ i, i = 1, 2, …, n, ∑ = n i 1 ∑ ⊆ms xi(s) ≦ 1, ∀ j ∊ m, (2) xi(s) ≥ 0, ∀ s ⊆ m, ∀ i, i = 1, 2, …, n. the corresponding dual to problem (2) ∑ = n i 1 p(i) + ∑ ∈sj p(j) → min subject to p(i) + ∑ ∈sj p(j) ≥ vi(s) ∀ i, s, (3) p(i), p(j) ≥ 0, ∀ i, j, the dual variables p(j) can be interpreted as anonymous linear prices of items, the term ∑ ∈sj p(j) is then the price of the bundle s and p(i) = s max [vi(s) −∑ ∈sj p(j)] is the maximal utility for the bidder i at the prices p(j). several auction formats based on the primal-dual approach have been proposed in the literature. although these auctions differ in several aspects, the general scheme can be outlined as follows: 1. choose minimal initial prices. 2. announce current prices and collect bids. bids have to be higher than or equal to the prices. 3. compute the current dual solution by interpreting the prices as dual variables. try to find a feasible allocation, an integer primal solution that satisfies the stopping rule. if such solution is found, stop and use it as the final allocation. otherwise update prices and go back to step 2. concrete auction formats based on this scheme can be implemented in different ways. the most important design choices are the following: • pricing scheme, • price update rule, • way of computing a feasible primal solution in each iteration, • stopping rule, • type of information feedback. the combinatorial auctions can be classified into the auctioneer-side allocation auctions and the bidder-side allocation auctions. the bidder-side allocation auctions were developed for small problems where bidders can cooperate in order to find a better ijahp article: fiala/using an analytic network process model in combinatorial auctions international journal of the 118 vol. 1 issue 1 2009 issn 19366744 analytic hierarchy process allocation by themselves at each step in the iteration. in the auctioneer-side allocation auctions the auctioneer solves the winner determination problem after the bids are collected. the auctioneer then provides some kind of feedback to support the bidders in improving their bids in the next iteration. usually the bidder’s current winning bids and item prices are used as the feedback. the key challenge in the iterative combinatorial auction design is to provide information feedback to the bidders after each step in the iteration. assigning prices to items and/or item bundles was adopted as the most intuitive mechanism of providing feedback. the ahp and anp have been static, but for today’s world, analyzing is very important time dependent decision making. the dhp/dnp (dynamic hierarchy process/ dynamic network process) methods were introduced by saaty (saaty, 2003). there are two ways to study dynamic decisions: structural, by including scenarios, and functional by explicitly involving time in the judgment process. for the functional dynamics, there are analytic or numerical solutions. the basic idea with the numerical approach is to obtain the time dependent principal eigenvector by simulation. the dynamic network process seems to be the appropriate instrument for analyzing dynamic network effects (fiala, 2006). the method is appropriate also for the specific features of multicriteria combinatorial auctions. the method computes time dependent weights for bundles of items of weights of bidders (figure 3). figure 3 time dependent weights. in the multicriteria combinatorial auction model, we take into account auctioneer, bidders, criteria and packages as clusters and different types of connections in the system. there are also some dependencies and feedback among elements and clusters also. the dynamic version of the model is tested. time weights winner loser battle zone ijahp article: fiala/using an analytic network process model in combinatorial auctions international journal of the 119 vol. 1 issue 1 2009 issn 19366744 analytic hierarchy process we used the alpha version of the anp software super decisions developed by creative decisions foundation (cdf) for some experiments testing the possibilities of the expression and evaluation of the multicriteria combinatorial auction models (figure 4). figure 4. multicriteria combinatorial auction model. 5. conclusions for electronic auctions, we propose to use multicriteria iterative combinatorial auctions. combinatorial auction is the important subject of an intensive economic research. iterative process helps the bidders express their preferences. combinatorial auctions promise to increase efficiency and reduce exposure risk in an economic environment where synergy is significant. the winner determination problem is by far the most researched issue in combinatorial auctions. winner determination problem is an npcomplete optimization problem. decentralized way for iterative combinatorial auctions based on primal-dual algorithm seems to be very promising for solving the winner determination problem. there are many variations how to use this approach. multicriteria optimization can be helpful for detailed analysis of combinatorial auctions. the combination of such approaches can give more complex views on auctions. a possible flexible approach is presented. the approach is based on the dynamic network process. ijahp article: fiala/using an analytic network process model in combinatorial auctions international journal of the 120 vol. 1 issue 1 2009 issn 19366744 analytic hierarchy process references bellosta, m., brigui, i., kornman, s., & vanderpooten, d. (2004). a multi-criteria model for electronic auctions. acm symposium on applied computing, 759-765. bichler, m. (2000). an experimental analysis of multi-attribute auctions. decision support systems, 29, 249268. bikhchandani, s., & ostroy, j.m. (2002). the package assignment model. journal of economic theory, 107, 377–406. cdf (creative decisions foundation) www page (2000)www.creativedecisions.net. cramton, p., shoham, y., & steinberg, r. (eds.) (2006). combinatorial auctions. cambridge, mit press. de vries, s., & vohra, r.v. (2003). combinatorial auctions: a survey. informs journal of computing, 15, 284-309, fiala, p. (2006). an anp/dnp analysis of economic elements in today's world network economy. journal of systems science and systems engineering, 15, 131–140. fiala, p. (1997). models of cooperative decision making. gal, t., & fandel, g. (eds.). multiple criteria decision making, heidelberg, springer. oliveira, e., fonsesca, j.m., & steiger-garao, a. (1999). multi-criteria negotiation in multi-agent systems. 1st international workshop of central and eastern europe on multi-agent systems (ceemas'99), st. petersburg. rothkopf, m., pekeč, a., & harstad, r. (1998). computationally manageable combinational auctions. management science, 8, 1131-1147. saaty, t.l. (1996). the analytic hierarchy process. pittsburgh: rws publications. saaty, t.l. (2001). decision making with dependence and feedback: the analytic network process. pittsburgh: rws publications. saaty, t.l. (2003). time dependent decision-making; dynamic priorities in ahp/anp: generalizing from points to functions and from real to complex variables. proceedings of the 7th international conference on the analytic hierarchy process, bali, indonesia, 138. sandholm, t. (2002). algorithm for optimal winner determination in combinatorial auctions. artificial intelligence, 135, 1-54. sandholm, t., & boutilier, c. (2006). preference elicitation in combinatorial auctions. cramton, p., shoham, y., & steinberg, r. (eds.). combinatorial auctions. cambridge, mit press. additional information: ijahp essay: garuti/parallel between space geometry and decision making space international journal of the analytic hierarchy process 364 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.337 parallel between space geometry & decision making space (modelling the growing complexity of measurement) claudio garuti claudiogaruti@fulcrum.cl introduction: the aim of this essay is to show some interesting analogies between space geometry and what we call “decision making (dm) space”, as well as, proposing a possible way of measuring how close (or far) two or more priority vectors are. for this purpose, we suggest applying the compatibility index g as a new way to measure the distance of priority vectors in the dm space. the aim for this essay is that it should be easy to read, (made in a kind of “divertimento” format), without complex formulae or hard math. first of all, we posit that the set of priorities (i.e. p1, p2,…,pn) obtained from an ahp/anp synthesis process can be considered as a vector in an n-dimensional space and each of the priorities (e.g. pn) constitutes a vector component in the specific ndimension. the n-dimensional space in which the priority vector can be plotted is what we define as “decision-making (dm) space” and will constitute the focus of our discussion here. the main idea proposed here is to establish a parallel between the classic cartesian vision of space 1 and its way of measurement through the pythagorean theorem, with the decision making space using the compatibility index g as a sort of “distance measure”. by the way, compatibility is intended as the degree of closeness between 2 decision vectors (like the distance between 2 points in a cartesian system). the essay was written using a physics analogy in three incremental stages of complexity, in terms of measurement, as follows: first, looking at the dm space from the classic cartesian point of reference (homogenous flat space); second, looking at the decisionmaking space from a non-cartesian (homogeneous curved space) 2 to consider a situation in which different dimensions may have different weights; and finally, from the point of view of a flexible bended space framework (see figure 4) to incorporate the decision-making feedback. 1 a cartesian coordinate system is a coordinate system that specifies each point uniquely in a plane by a set of 3 numerical coordinates, which are the signed distances from the point to three fixed perpendicular directed lines, measured in the same unit of length. each reference line is called a coordinate axis or just axis of the system, and the point where they meet is its origin, usually at ordered pair (0, 0,0). the coordinates can also be defined as the positions of the perpendicular projections of the point onto the three axes, expressed as signed distances from the origin (see figure 2). 2 in this approach, coordinate hypersurfaces of different coordinates are also orthogonal, although curved (see figure 3). mailto:claudiogaruti@fulcrum.cl https://en.wikipedia.org/wiki/coordinate_system https://en.wikipedia.org/wiki/point_%28geometry%29 https://en.wikipedia.org/wiki/plane_%28geometry%29 https://en.wikipedia.org/wiki/number https://en.wikipedia.org/wiki/positive_and_negative_numbers https://en.wikipedia.org/wiki/perpendicular https://en.wikipedia.org/wiki/unit_length https://en.wikipedia.org/wiki/origin_%28mathematics%29 https://en.wikipedia.org/wiki/orthogonal_projection https://en.wikipedia.org/wiki/coordinate_hypersurface https://en.wikipedia.org/wiki/orthogonal https://en.wikipedia.org/wiki/curvilinear_coordinates ijahp essay: garuti/parallel between space geometry and decision making space international journal of the analytic hierarchy process 365 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.337 the three stages of complexity: as previously indicated, we can establish three different stages of increasing complexity to explore the analogy between measurement in physical space (metric topology) and measurement in the decision making space (order topology 3 ). before starting with the analogy, we need to establish three basic conditions for the dm space coordinate systems used in this essay: first: every point (x, y, z) of a dm space, can be interpreted as a point with the coordinates of one possible decision vector for a given decision maker (dm), in other words the coordinates (x, y, z) may represent the field of potential priorities for a given dm (similar to the faraday’s interpretation of magnetic fields in physics). second: every point can be represented by a positive normalized vector. the set of coordinates of any point of the space has to add to one (xi+yi+zi=1), no one coordinate may present negative values and all axes range from 0 to 1. third: the decision making structure represents the space decision making which represents the way that the dm makes decisions. in other words, the decision structure synthesizes the rule of measurement in that specific decision making space (an ahp or anp structure for instance). in figure 2, the coordinate system may represent the space of points of possible decisions for a given dm in a (0 1) range for every axe. each component of the decision vector may correspond, for example, to the priority given to each criterion (assuming the decision constitutes prioritization of criteria) and the three axial components (x1, y1 and z1) corresponds to the prioritization given by the first decision maker while (x2, y2 and z2) corresponds to the criteria prioritization given by the second decision maker. 3 order topology is the topology used for measuring the intensity of preferences of the dms. ijahp essay: garuti/parallel between space geometry and decision making space international journal of the analytic hierarchy process 366 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.337 first stage of complexity: the space of non weighted axes (homogenous flat space) figure 2. coordinates system in a homogenous flat space (figure from wikipedia) figure 2, represents the first stage of the analogy between space geometry & geometry of the decision making space; its shows a 3d cartesian coordinate system representing a homogenous or “flat” space (euclidean space). in this stage, the concept of distance or closeness between points is the same that one has learned at school, and applies the classic formula of distance: d(x,y,z)= √ 2 2 2 ) (eq.1). equation 1 represents the situation where every axe is equally important, meaning that one unit of axe x has the same impact (or influence) in the assessment of distance “d” than a unit in axe y or axe z. this stage for the decision making space is equivalent to the situation where every criterion has the same importance or weight (wx=wy=wz=constant). of course, in the decision making domain such a situation is not likely but it is useful as a reference case for a distance calculation between two different points in a homogenous space. the second stage of complexity: weighting the axes the second stage of complexity corresponds to a space where the axes have weights, i.e. each axis may have a different importance from the others, this space can be represented as a bended or curved space (fig.3). in this second stage, the initial weights are fixed, they cannot vary under any circumstance and may be represented as a bended space within a rigid frame. 0 1 1 1 decision making point #1 (x1, y1, z1) decision making point #2 (x2, y2, z2) d http://psychology.wikia.com/wiki/image:coord_planes_color.svg http://psychology.wikia.com/wiki/image:coord_planes_color.svg ijahp essay: garuti/parallel between space geometry and decision making space international journal of the analytic hierarchy process 367 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.337 figure 3. coordinate system for curved space in a rigid frame. (figure from creative common license, wikipedia) the 3 planes (q1, q2, q3) are constant. these bended or curved surfaces cannot be altered by the presence or absence of other objects (alternatives for instance). this decision making geometry corresponds to the standard ahp decision making structure (representing the ahp rule of measure), where each axe that represents each terminal criteria has its own global weight that produces a deflection or a change in the slope of the surface, that weight cannot be changed (unless we change the structure). of course, bigger weights represent bigger slopes. we realize that with weighed axes a variation in one axe ( for instance), may produce different results for distant calculation than a variation in any other for distance calculations. the fact that the importance of a change in x ( ) may differ in importance to a change in y or z ( y or ), even if the size of the change is the same, can represent a big difference for the closeness calculation index. the classic formula of distance calculation (eq.1) will not be adequate anymore. we need to project the coordinates of the straight axes over the bended axes and renormalize the coordinates to correctly represent the new priority points p1 and p2 (as an initial approximation), which means to weight the initial coordinates by the importance of the axes. (most of the time reality in decision making environment is better represented within a heterogeneous space than a homogenous one). continuing with the analogy with the space and structures built on ahp, we may say that in this stage once the structure is built, there is no feedback interaction between criteria or criteria with alternatives; the alternatives must follows the rule of measure established initially in the ahp structure. the space under ahp might be bended only once (at the beginning). however, this frame is still too rigid to represent the full picture due to this artificial restriction to the geometry over the decision making frame. also, this frame shows that this geometry cannot be altered by adding or deleting alternatives. but, we know that this action (adding or deleting alternatives) may change the actual decision making geometry. 0 1 1 1 w1x w2y w3z rigid frame curved space http://en.wikipedia.org/wiki/image:general_curvilinear_coordinates_1.svg ijahp essay: garuti/parallel between space geometry and decision making space international journal of the analytic hierarchy process 368 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.337 observation: when modelling with ahp one has to be careful when adding or deleting elements (even alternatives), since one could be modelling another problem and creating another rule of measure. this is especially true in the relative measurement of ahp, where a change in the set of alternatives (adding or deleting) may produce a change in the priority values of the alternatives including its own rank. the alternatives do not follow all the rules of the rigid frame created in stage 2. thus, if all the alternatives are settled at the same time rank reversal may happen, because the alternatives are connected between them in a relative measurement form. this connection is not fully captured by this rigid frame of decision making. but, if the alternatives are settled one at a time, we break down that connection (for good and for bad) and no rank reversal is produced. that is what we are doing when measuring alternatives with absolute measurement scales specially created for this mode (the absolute measurement mode). the third stage of complexity: getting into a flexible frame (the feedback process) the third stage of this analogy corresponds to an anp model representing a weighted space in a flexible frame (see fig.4): figure 4. coordinate system in a flexible bended space (figure from wikipedia) this decision structure (the anp model), represents a flexible frame of the creating space. in this stage, the alternative can be physically interpreted as a mass of a body that alters (curve) the space (see fig.4). this represents the way the alternative tells the space structure (the anp structure) how much the weights of the criteria have to change due to its presence, changing the curvature of the space where the alternatives belong and so altering its pattern behaviour (the initial line/path is altered/deflected to another position, see fig.4). in this manner, we are creating an interaction network between the frame and the elements that belong to the frame that better recreate the system behaviour (and that’s why we need a flexible frame). we have to keep the anp model interacting with the set of alternatives until reaching a steady point (the final supermatrix’s eigenvector) they conform an interrelated unity that responds for the functionality of the whole system. this is what happens in a relativistic frame between mass and space. this is like the old einstein quote: “the space tells to the mass how to move and the mass tells to the space how to curve”. now we may say, the anp model (the space structure with its own rule of measurement) tells the alternatives how to behave and the alternatives tell the anp ijahp essay: garuti/parallel between space geometry and decision making space international journal of the analytic hierarchy process 369 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.337 model how to re-shape the space structure (re-building its own rule of measurement). in synthesis, the anp represents a new flexible and interdependent rule of measurement, which is very close to what happens in the real decision making world. a final observation of measuring in the growing complexity: the better way to measure closeness (distance) between points, in the second and third stage of complexity (the decision making space in weigthed environment), is using the general compatibility index g. (g= ½ i(a+b)*min(a,b)/max(a,b), with: a, b n-dimensional normalized vectors). other conditions that are important to remember: 1. each axes of this special cartesian system, is bounded between 0 – 1. 2. the curvature of axes also belong to a 0 – 1 scale, where 0 represents no deflection or flat space (0° degree) and 1 represent maximum deflection (90° degree). 3. each node (criterion) will produce a deflection proportional to its own weight. 4. the interaction between nodes and between nodes and alternatives should be seen as the degree of connectivity of the space geometry we are creating. the greater the connectivity, greater the complexity (“rugosity”) of the space. 5. the high degree of connectivity of the system is the main reason that the classic cartesian axes representation is not a good representation anymore. 6. each point in the deflected mesh represents one possible or potential solution for the priority vector before reaching the equilibrium or steady point. 7. this final equilibrium point in the decision making space (using an anp model) may have more than one steady solution. this behaviour has it’s “mirror” in the gravitational strange attractor (a fractal attractor) where a final equilibrium point of a system may have more than one simple attractor. for example, the system earth-sun presents this kind of strange attractor points, called henon attractor in honour of michel henon, an astronomer that first noticed a disturbing behaviour in a simple model of stars orbiting within a galaxy at the observatory of nice (france) in the 1960s. at first glance it was thought that this was some error in calculation, but later, this phenomenon was noticed in almost every orbit (stars and planets). the shape of henon’s orbits is not the classical ellipses that astronomers have used for centuries. if one planet orbits one star in an otherwise empty universe, the linear curves of newton and kepler would accurately model its trajectory. but, in the real universe gravitational attractions from other bodies make the planet´s orbit much less predictable. (henon discovered that this sort of “chaotic behaviour” was an essential part of the dynamics of stellar and planet orbits). in the same analogous situation, if a dm has just one criterion of decision and nothing else (just one body mass), then the multiple possible paths to reach the equilibrium (as graph theory and cesaro´s sum shown) will collapse to just one path and the steady point will be reached immediately (just like a 100% consistent matrix of judgments). in this case, the equilibrium point will be easily represented trough a clean linear function. ijahp essay: garuti/parallel between space geometry and decision making space international journal of the analytic hierarchy process 370 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.337 finally, it is possible to show the whole parallelism in a simple short table as below: figure 5. parallel between space geometry and decision making space it is also interesting to note that when working in a relativistic space the notion of distance change from the classic 3d formula sqrt( 2 2 2 ) to the more complex one 2 x 2 y 2 z 2 i) 4 , makes clear that time is simply another expression of space (the fourth dimension if you like). also, the importance of this fourth axe when relevant (within a relativistic space for instance) is far greater than the other three, distorting the surrounding space (just like a big weight will make it in a dm environment). so, if we consider that distortion as a result of the presence of a big weight (a star for instance), we can say that weights do matter when making measurements, whether we are talking of physical or a decision making environment. notes: it has been mathematically proven that in a strong curved space one would be able to move without applying any external force or changing one’s total mass (contradicting the classic newton’s law of action-reaction). this shows that space is something more complex than just a set of absolute (and independent) points in a cartesian graph. space might be something that one can interact with, moving oneself like laying in a flexible and touchable frame. this is similar to what happens when a decision maker interacts long term with an anp model, seeing how his/her own values are transformed into priorities and these priorities (when seen it in a structured and explicit way) may eventually change his/her own initial values. by the way, the movement in this curved space might be done just like swimming in a pool, where the water of the pool corresponds to this flexible and curved space, the more curved the space the easier it is to move in it by “swimming”. of course, in a nearly flat space, if we try to move ourselves using this swimming process it would take millions of years just to move our body a couple of centimeters 5 . 4 “i”=sqrt(-1), direction perpendicular to the axes x, y, z. c= speed of light. 5 scientific american, august 2009 more idealistic flat space no weights (the dm space is built on an ordinal kind of scale) curved space in a rigid frame ahp model. the dm space is built on a cardinal scale without including feedback, unable to change if any change happens within the frame. curved space in a flexible frame anp model. the dm space is built on a cardinal scale including feedback, enables changes in the frame itself. strange attractor that best represents more complex anp model (more connected) with more than the motion of the bodies one steady point. more realistic ijahp: mu/what constitutes a novel ahp/anp contribution? international journal of the analytic hierarchy process 1 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.483 what constitutes a novel ahp/anp contribution? we often receive inquiries about whether a particular study is novel or original enough to be published in our journal. interestingly enough, whenever a potential author asks this question, the study is likely original enough for publication. many other potential authors simply submit their studies with the assumption of originality, and they are surprised when they are told their study is not original enough to be published. for this reason, the question of what constitutes a novel study deserves some consideration. ahp/anp methodologies were created to address decisions in many different domains. for this reason, and in principle, we would expect a contribution on the methodology (theoretical contribution) or its use (applied contribution). discussions of ahp/anp theoretical aspects are routinely present in our journal, and the paper “achieving the desired level of dependency in anp decision models” by cooper and liu, published in this issue, constitutes a good example of this. however, ijahp also gives opportunities to applied contributions, provided they are original. originality means that either the authors address decisions that have not been dealt with before, as done by bahurmoz and samrgandi’s study in their proposal to solve the current oil economic crisis in saudi arabia, or they address well-known decisions with a certain novel twist, as done by agredo et al. in their development of a model for the selection of a customer relationship management system (a well-known decision) but with a model simplified in such a way that it can be used by small businesses taking into account different stakeholder perspectives. these two characteristics, model simplicity and multiple perspectives constitute the different twist and contribution for the crm selection problem. finally, as in any other journal, we need to make sure that the addressed topic is of interest to a large number of ijahp readers as well as ensuring that the study has been developed following the best scientific practice standards in use in the ahp/anp community. the papers published in this issue are good examples of novel contributions. we look forward to receiving your own original contributions! enjoy our first issue of the year, enrique mu, phd editor-in-chief https://doi.org/10.13033/ijahp.v9i1.483 ijahp news and events: petrillo/digital effect: vision of business and industry 5.0 international journal of the analytic hierarchy process 205 vol. 13 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.866 digital effect visions of business and industry 5.0 dialogues about the difficult relationship between innovation, society and industry antonella petrillo digital effect: visions of business and industry 5.0 is a new book on digitization and innovation and the importance of applying decision making models. figure 1 digital effect: visions of business and industry 5.0 the increase in global competitiveness challenges the manufacturing sector to integrate design and product in order to improve quality and process. this means that in today’s market, even large companies need to be flexible. thus, in recent years, the focus on smart manufacturing systems has pushed companies toward a new variety of highly specific technical solutions. these solutions are characterized by an integrated approach to manufacturing termed “digital manufacturing”. in fact, digital manufacturing systems often incorporate optimization capabilities to reduce time and costs and improve the efficiency of most processes. despite the recognition of the importance of digital manufacturing, most organizations do not have the necessary capabilities. this is a serious issue because the digital revolution is happening now; it is the “present”, not the future! therefore, the priority for every company should be to adapt each digital technology to their business. there are many different tooling processes that digital manufacturing utilizes such as artificial intelligence, automation and robotics, additive technology, human-machine interaction, iot (internet of things), etc. these tools are helping create innovations that will change the nature of manufacturing itself. industry and academic leaders agree that digital manufacturing technologies will transform every “link” in the manufacturing value chain, from research and development, ijahp news and events: petrillo/digital effect: vision of business and industry 5.0 international journal of the analytic hierarchy process 206 vol. 13 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.866 supply chain, and factory operations to marketing, sales, and service. this transformation is known as the fourth industrial revolution, also referred to as industry 4.0. however, the newest trend is called industry 5.0, which is the future, the already penetrating trend of process change that is directed towards closer cooperation between man and machine, and systematic prevention of waste including industrial upcycling. the transformation towards digitalization for all must be harmonized with the threats, opportunities, and dynamics of the industry revolution. in this context, management plays a crucial role, i.e., people, including entrepreneurs, managers, and middle managers in each company who exercise a decision-making function. therefore, managerial decisions can contribute significantly to the management of complex systems such as the implementation of the link between the principles of the economy and a digital approach characterizing smart manufacturing. identifying and outlining the technological, cultural, organizational, social, and managerial changes underlying the digital transformation that are underway is the aim of the new book by fabio de felice and antonella petrillo titled, “digital effect: visions of business and industry 5.0,” published by mcgraw-hill. in seven chapters, the authors address digital transformation through an analysis of enabling technologies (e.g., artificial intelligence, quantum computing, robotics, etc.) as well as the impacts of new forms of work organization, from the need to reform training courses to the dangers of new forms of authoritarianism and social inequality. figure 2 authors of the book these issues are not exclusively the prerogative of scientists or philosophers, but are topics that concern us all. the aim of the book is to combine the technical aspects related to technology with depictions and visualizations of changes connected to digitalization, that affect our daily life and the transformation of the world around us. in summary, this book aims to provide an overview of the new industrial revolution; it is intended to be a useful resource for anyone dealing with this issue. this broad theme not only affects the world of work in the strict sense, but in a global sense, and is a theme that also represents a challenge for the academic, professional and industrial worlds. ijahp news and events: petrillo/digital effect: vision of business and industry 5.0 international journal of the analytic hierarchy process 207 vol. 13 issue 1 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.866 as professor enrique mu, ijahp editor-in-chief, pointed out in a recent interview with the authors of the book, digitization and innovation are two processes that guide today's companies. obviously we cannot ignore the covid pandemic that represents a strong “technological accelerator”. let’s continue to explore the topic further! figure 3 interview with professor mu q#1| digitization and innovation are two processes that guide today's companies. in your opinion, what are the drivers and obstacles to digital transformation? r#1| digital transformation is, basically, a disruptive type of innovation. the presence of an extensive threat, such as the current pandemic, is an important driver to lead us to a complete digital transformation. successful organizations that have optimized processes between customers and suppliers may see these same processes as a possible obstacle to their digital transformation, since it is really difficult for them to change them drastically. q#2| do you think the covid pandemic will be a strong “technological accelerator”? r#2| yes, absolutely. the pandemic has accelerated digital transformation exponentially. for example, the pandemic has literally led 100% of universities and students to embrace distance learning. similarly, many conferences were canceled at the start of the pandemic, but now most of the conferences are offered in a “virtual” format. my opinion is that “hybrid” education and “hybrid” academic conferences will be the norm from now on. ijahp essay: adams, rokou/why simplify ahp? international journal of the analytic hierarchy process 382 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.417 why simplify ahp? william adams wjladams@gmail.com elena rokou creative decisions foundation erokou@icloud.com ahp and anp are very well developed mathematical theories with many practitioners, so why should anyone research methods to simplify these processes? we believe there are many reasons to search for simplifications: 1. if we make the pairwise process simpler, we can get feedback from participants without needing to train them. in other words, a whole world of participants becomes available to us. 2. if we reduce the number of interactions needed to get viable data, we allow participants to give us better data throughout the process. (how many of us have said or heard, “didn’t i already answer this question?”) 3. if we simplify the process of creating viable ahp/anp models, we could open the process to those with far less training in ahp/anp theory. 4. if we simplify ahp, we can present it quickly to untrained individuals and allow them to begin using ahp without needing to understand all of the technical details (until they need those details, or until curiosity overtakes them). these goals, by themselves, show us that researching methods to simplify and streamline ahp could yield great results. however, if we consider an analogy we can see even more reasons. in all likelihood, everyone reading this missive has a cell phone. how many of you readers know exactly how cell signals are transferred from your phone to a cell tower (and from there to another phone)? how many understand the operating system running on your phone? or how your phone takes your text messages and sends them to another person via sms, smtp, or skype? very few people understand the intricacies of such operations, and yet, many people benefit from the simplified user interfaces that others have developed so that we may take advantage of that technology. some of us in the field of ahp should be considering a similar question, “how can we simplify the process of building and participating in an ahp model, so that untrained users can take advantage of this technology, in the same way that i take advantage of my cell phone’s technology?”. for these reasons, and many more, we believe that the difficult research of simplifying and streamlining ahp should be undertaken. for instance, reason #4 stated above is why we developed the simpleahp webapp; so that we could give elementary aged children access to ahp, without needing to explain all of the subtleties involved. mailto:wjladams@gmail.com mailto:erokou@icloud.com ijahp essay: adams, rokou/why simplify ahp? international journal of the analytic hierarchy process 383 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.417 simpleahp is a webapp that is available at http://tiny.cc/simpleahp and whose source code is freely available as well (for interested parties to improve upon and add functionality). simpleahp simplifies the ahp process in 2 ways: first, by restricting to a single pairwise comparison set, and second, by allowing for a simplified pairwise process using a likert scale. the simplified pairwise process for the comparison a versus b offers the following responses: a is better, a is much better, b is better, b is much better, or they are equal. using this simplified scale, we do not need to explain the standard 1-9 scale to participants. instead they are immediately able to understand the questions and submit their answers (children as young as the age of 6 were able to fill out questionnaires with no prior training or explanation). besides simplifying the process itself, we added ways to cluster and analyze the results more efficiently and effectively, giving more insight on the responders and their preferences, i.e. clustering by gender, age group etc. . at the isahp 2016 conference we had 5 youth participants aged 9-14 who used the simpleahp web app to find interesting things out about the preferences of their peers and parents. for instance, will adams aged 12, was able to hand out simplified paper questionnaires to classmates aged 6-12 and use that data and the simpleahp web app to understand their preferences, and the preferences of subgroups of classmates (e.g. what do boys versus girls think). another participant, aged 12, was able to use an online questionnaire and simpleahp to discern ice cream flavor preferences based on age and gender. http://tiny.cc/simpleahp ijahp essay: adams, rokou/why simplify ahp? international journal of the analytic hierarchy process 384 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.417 all of these youth participants had no prior exposure to ahp (aside from my son will adams), and yet they were able to use our simplified questionnaire process and the simpleahp to find answers to real questions they were interested in. there are many places we could look to simplify or streamline ahp to make it more efficient and effective both for new practitioners and those with years of experience. the point is to find ways to make ahp simple and within reach of everybody without any need of specific knowledge. this can be done by converting ahp from a sophisticated mathematical decision making method to an everyday tool to make quick and informed decisions on almost any subject we can think of. the downside of this simplification will be loss of precision and accuracy, however we are not usually more precise when deciding without any method at all. that does not mean that for complex problems we will not use the full extent of the method, but that in everyday and relatively simple models we do not really need to go to the full method. for example, it is a bit like using a fighter jet to cross the street when all you need to do walk (sure it will work, but you will spend the next 6 months in training learning how to fly the thing). on the measurement of intangibles ijahp article: saaty/ the analytic hierarchy process without the theory of oskar perron international journal of the analytic hierarchy process 268 vol. 5 issue 2 2013 issn 1936-6744 the analytic hierarchy process without the theory of oskar perron thomas l. saaty university of pittsburgh pittsburgh, pa 15260 usa saaty@katz.pitt.edu abstract it is known and has been mathematically proven that the principal eigenvector is necessary for deriving priorities from judgments in the analytic hierarchy process (ahp). according to the work of oskar perron, the principal eigenvector can be obtained as the limiting power of a positive matrix. in this paper we show that the principal eigenvector does not need the theory of perron for its existence based on the fact that the principal eigenvalue and corresponding principal eigenvector are transparently obtained for a consistent matrix. by perturbation theory the result is obtained for a near consistent matrix. keywords: theory of perron, ahp eigenvector, pairwise comparison matrix solution 1. introduction we begin by honoring the name of oskar perron who proved a very powerful theorem in mathematics about real positive matrices. i regret not meeting perron (born in 1880) before he died in munich, germany, in 1975. my work on the analytic hierarchy process (ahp) with real positive reciprocal matrices leads to the principal eigenvalue and eigenvector without the need for invoking any of perron’s results as will be shown in this paper. he proved that if ( ) is an ija a n n= × real positive matrix: 0 for 1 , ,ija i j n> ≤ ≤ then there is a positive real number maxλ , called the perron root or the perron–frobenius eigenvalue, such that maxλ is an eigenvalue of a and any other eigenvalue λ (possibly, complex) is strictly smaller than maxλ in absolute value, |λ| < maxλ . there exists an eigenvector w = (w1,…,wn) of a with eigenvalue maxλ such that all components of w are positive: max , 0 for 1 i .iaw w w nλ= > ≤ ≤ there also exists a positive left eigenvector max: , 0. t t iv v a v vλ= > perron also proved that the principal eigenvector w corresponding to maxλ can be obtained by raising the matrix a to infinite powers. mailto:saaty@katz.pitt.edu http://en.wikipedia.org/wiki/complex_number rob typewritten text http://dx.doi.org/10.13033/ijahp.v5i2.191 ijahp article: saaty/ the analytic hierarchy process without the theory of oskar perron international journal of the analytic hierarchy process 269 vol. 5 issue 2 2013 issn 1936-6744 his theorem was later modified by frobenius whose theory assures us of a similar result except that he proved the root may no longer be simple if there are zero entries in the matrix. in the analytic hierarchy process the pairwise comparison judgment matrices are real, positive and reciprocal 1 ( ( ), 0, for all and ) ,ij ji ji ij a a a a i j a = > = and their order is not much larger than 7x7. the entries values lie between 9 and 1/9. interestingly, as we shall show here, the principal eigenvalue and its principal eigenvector can be found for a real reciprocal positive matrix of small order without perron’s theory. the principal eigenvalue and eigenvector can be obtained from the solution of a system of equations without using the powers of the matrix as does perron. we observe that if we know either maxλ or w , we also know the other. if, for example, we know maxλ , we get w by solving, in the familiar way, the homogenous system of linear equations: max 1 , 1,..., . n ij j i j a w w i nλ = = =∑ if we know w then because of the normalization condition 1 1 n i i w = =∑ in our case, after taking the sum on both sides of the equation with respect to i and interchanging the sums on the left we obtain: max max 1 1 1 . n n n j ij i j i i w a wλ λ = = = = =∑ ∑ ∑ in other words we obtain maxλ as the scalar product of the vector w with the vector of column sums of the matrix a. if the matrix has real coefficient that are positive, and if w is real and positive then maxλ is real and positive. but, we have not yet established that it is a simple eigenvalue and that it dominates all other eigenvalues in modulus. we follow two routes to obtain the principal eigenvector :w one is by using the general idea of perturbation of the coefficients of a consistent matrix, which also involves reciprocal values in the transpose position, and the other is by considering the graphtheoretic concept of dominance along paths of different lengths leading to cesaro summability. for us, applied to ahp reciprocal matrices, cesaro summability says that the average value of the normalized vector of the row sums of the powers of a positive reciprocal matrix is equal to the normalized vector of the row sums of the limiting power of that matrix, which, of course, according to perron, is the principal eigenvector of that matrix. the latter again gives the same answer as the theory of perron does without the need for perron’s logic, whether that matrix is consistent (trivial because ak = nk-1a) or inconsistent, by perturbation arguments from the work of j. h. wilkinson that it yields the principal eigenvalue and eigenvector in the limit. ijahp article: saaty/ the analytic hierarchy process without the theory of oskar perron international journal of the analytic hierarchy process 270 vol. 5 issue 2 2013 issn 1936-6744 2. consistent positive reciprocal matrices assume that one is given n stones, 1,..., na a , that we will refer to as alternatives, having known weights ,...,i nw w , respectively. we form a matrix a of pairwise ratios whose ith row gives the ratios of the weights of the ith stone with respect to all the others. 𝐴1 𝐴2 ⋯ 𝐴𝑛 𝐴1 𝑤1/𝑤1 𝑤1/𝑤2 ⋯ 𝑤1/𝑤𝑛 𝐴 = 𝐴2 𝑤2/𝑤1 𝑤2/𝑤2 ⋯ 𝑤2/𝑤𝑛 ⋮ ⋮ ⋮ ⋯ ⋮ 𝐴𝑛 𝑤𝑛/𝑤1 𝑤𝑛/𝑤2 ⋯ 𝑤𝑛/𝑤𝑛 we note that we can recover the vector of weights ,...,i nw w w= by solving the system of equations defined below: aw = 𝑤1/𝑤1 𝑤1/𝑤2 ⋯ 𝑤1/𝑤𝑛 𝑤2/𝑤1 𝑤2/𝑤2 ⋯ 𝑤2/𝑤𝑛 ⋮ ⋮ ⋯ ⋮ 𝑤𝑛/𝑤1 𝑤𝑛/𝑤2 ⋯ 𝑤𝑛/𝑤𝑛 𝑤1 𝑤2 ⋮ 𝑤𝑛 = 𝑛 𝑤1 𝑤2 ⋮ 𝑤𝑛 = 𝑛𝑤 solving this homogeneous system of linear equations aw nw= to find w is a trivial eigenvalue problem, because the existence of a solution depends on whether or not a solution exists if n is an eigenvalue of the characteristic equation of .a but the matrix a has rank one and thus all its eigenvalues but one are equal to zero. the sum of the eigenvalues of a matrix is equal to its trace, the sum of its diagonal elements, which in this case is equal to n. thus n is the largest, or the principal, eigenvalue of a and w is its corresponding principal eigenvector and it is positive and unique to within multiplication by a constant, and thus belongs to a ratio scale. we now know what must be done to recover the weights iw , whether they are known in advance or not. definition: an n by n matrix ( )ija a= is consistent if , , 1,...,ij jk ika a a i j k n= = holds among its entries. a consistent matrix always has the form ( )i j w a w = and we know that 1k ka n a−= . the consistent case has no need for the theorem of perron to prove the existence of a largest real eigenvalue and its corresponding positive eigenvector, nor to prove that this vector is the limit to which powers of the matrix converge. of course, real world ijahp article: saaty/ the analytic hierarchy process without the theory of oskar perron international journal of the analytic hierarchy process 271 vol. 5 issue 2 2013 issn 1936-6744 reciprocal pairwise comparison matrices are very unlikely to be consistent unless they use actual measurement data. now, we give a mathematical discussion to show why when a matrix is inconsistent we still need the principal right eigenvector for our priority vector. it is clear that no matter what method we use to derive the weights ,iw we need to get them back so they are proportional to the expression 1 1,..., n ij j j a w i n = =∑ , that is, we must solve 1 = for 1,..., n ij j i j a w cw i n = =∑ . otherwise, 1 for 1,..., n ij j j a w i n = =∑ would yield another set of different weights and they in turn could be used to form new expressions 1 1,..., n ij j j a w i n = =∑ , and so on ad infinitum. unless we solve the principal eigenvalue problem, our quest for priorities becomes meaningless. we learn from the consistent case that what we get on the right is proportional to the sum on the left that involves the same scale used to weight the judgments that we are looking for. thus we have the proportionality constant c. a better way to see this is to use the derived vector of priorities to weight each row of the matrix and take the sum. this yields a new vector of priorities (relative dominance of each element) represented in the comparisons. this vector can again be used to weight the rows and obtain still another vector of priorities. in the limit (if one exists), the limit vector itself can be used to weight the rows and get the limit vector back perhaps proportionately. our general problem, possibly with inconsistent judgments, takes the form: 𝐴𝑤 = 1 𝑎12 ⋯ 𝑎1𝑛 1 𝑎12 1 ⋯ 𝑎2𝑛 ⋮ ⋮ ⋮ ⋮ 1 𝑎1𝑛 1 𝑎2𝑛 ⋯ 1 𝑤1 𝑤2 ⋮ 𝑤𝑛 = 𝑐𝑤 this homogeneous system of linear equations aw cw= has a solution w if c is the principal eigenvalue of .a that this is the case can be shown using an argument that involves both left and right eigenvectors of .a two vectors 1 1( ,..., ), ( ,..., )n nx x x y y y= = are orthogonal if their scalar product 1 1 ... n nx y x y+ + is equal to zero. it is known that any left eigenvector of a matrix corresponding to an eigenvalue is orthogonal to any right eigenvector corresponding to a different eigenvalue. this property is known as biorthogonality. ijahp article: saaty/ the analytic hierarchy process without the theory of oskar perron international journal of the analytic hierarchy process 272 vol. 5 issue 2 2013 issn 1936-6744 theorem 1: for a given positive matrix ,a the only positive vector w and only positive constant c that satisfy ,aw cw= is a vector w that is a positive multiple of the principal eigenvector of ,a and that the only such c is the principal eigenvalue of .a proof: we know that the right principal eigenvector and the principal eigenvalue satisfy our requirements. we also know that the algebraic multiplicity of the principal eigenvalue is one, and that there is a positive left eigenvector of a (call it z ) corresponding to the principal eigenvalue. suppose there is a positive vector y and a (necessarily positive) scalar d such that .ay dy= if d and c are not equal, then by biorthogonality y is orthogonal to ,z which is impossible since both vectors are positive. if d and c are equal, then y and w are dependent since c has algebraic multiplicity one, and y is a positive multiple of .w thus the proof is complete. let ija be the relative dominance of ia over .ja in order to simplify the notation let the matrix corresponding to the reciprocal pairwise relation be denoted by ( )ija a= . the relative dominance of ia over ja along paths of length k is given by ( ) 1 ( ) 1 1 n k ij j n n k ij i j a a = = = ∑ ∑ ∑ where ( )kija is the (i,j) entry of the k th power of the matrix ( )ija . the total dominance ( )iw a , of alternative i over all other alternatives along paths of all lengths is given by the infinite series ( ) 1 ( )1 1 1 ( ) n k i j j i n n kk i j i j a w a a ∞ = = = = = ∑ ∑ ∑∑ which coincides with the cesaro sum in (equation 1). (equation 1) ( ) 1 ( )1 1 1 1 lim n k i jm j n nx kk i j i j a m a = →∞ = = = ∑ ∑ ∑∑ ijahp article: saaty/ the analytic hierarchy process without the theory of oskar perron international journal of the analytic hierarchy process 273 vol. 5 issue 2 2013 issn 1936-6744 but this limit of weighted averages (the cesaro sum) can be evaluated; we have for an n by n consistent matrix ( )where ( ) ( ) , 1,... ,iij ij j w a a a i j n w = = = that 1k ka n a−= and the limit in equation 1 is simply the eigenvector w normalized. in general, it is also true that the cesaro sum converges to the same limit as does its thk term /k t ka e e a e that yields k step dominance. here we see that the requirement for rank takes on the particular form of the principal eigenvector. we will not assume it for the inconsistent case, but will prove its necessity again for that more general case. we now develop a necessary and sufficient condition for rank preservation. for emphasis, recall from graph theory that an element ( ) of am mija gives the cumulative dominance of the ith element over the jth element along all chains of length m. that is precisely how one measures the consistency relation between that row and each column. in fact, when a is consistent we have from 1m ma n a−= that the entries of ma and those of a differ by a constant thus maintaining consistency. in general we have 𝐴𝑚 = (𝑎𝑖𝑗 (𝑚)) . theorem 2: for a positive reciprocal matrix a lim 𝑚→∞ 𝑎𝑖𝑘 (𝑚) ∑ 𝑎𝑖𝑘 (𝑚)𝑛 𝑖−1 = lim 𝑚→∞ 𝑎𝑖𝑠 (𝑚) ∑ 𝑎𝑖𝑠 (𝑚)𝑛 𝑖−1 , 𝑘, 𝑠 = 1, 2, … , 𝑛. proof. let 𝐵 = 𝑁𝐴𝑁−1 (where n and n-1 are non-singular matrices that we will define later) be the jordan canonical form of a given by: 𝐵 = 𝜆1 𝐵2 ⋱ 𝐵𝑟 , where 𝜆1 ≡ 𝜆𝑚𝑎𝑥 , and 𝐵𝑝 , 𝑝 = 2, 3, … , 𝑟 is the 𝑛𝑝 × 𝑛𝑝 jordan block defined by 𝐵𝑝 = 𝜆𝑝 0 0 ⋯ 0 0 1 𝜆𝑝 0 0 0 0 1 𝜆𝑝 0 0 ⋮ ⋱ ⋮ 0 0 0 ⋯ 1 𝜆𝑝 , ijahp article: saaty/ the analytic hierarchy process without the theory of oskar perron international journal of the analytic hierarchy process 274 vol. 5 issue 2 2013 issn 1936-6744 where 𝜆𝑝 , 𝑝 = 2, … , 𝑟 are distinct eigenvalues with multiplicities 𝑛2, … , 𝑛𝑟 respectively, and ∑ 𝑛𝑝 = 𝑛 − 1𝑟𝑝=2 . we have 𝐴 = 𝑁−1𝐵𝑁 and 𝐴𝑚 = 𝑁−1𝐵𝑚𝐵, where 𝐵𝑚 is given by: 𝐵𝑚 = 𝜆1 𝑚 0 ⋯ 0 0 𝐵2 𝑚 0 ⋮ ⋱ ⋮ 0 0 ⋯ 𝐵𝑟𝑚 , let us denote 𝑁−1 ≡ 𝐷 = �𝑑𝑖𝑗� and 𝑁 = (𝑛𝑖𝑗). we have 𝐴𝑚𝐷𝐵𝑚𝑁 = 𝑛11𝑑11𝜆1 𝑚 + ⋯ , 𝑛12𝑑11𝜆1 𝑚 + ⋯ , … , 𝑛1𝑛𝑑11𝜆1 𝑚 + ⋯ 𝑛11𝑑21𝜆1 𝑚 + ⋯ , 𝑛12𝑑21𝜆1 𝑚 + ⋯ , … , 𝑛1𝑛𝑑21𝜆1 𝑚 + ⋯ ⋮ ⋱ ⋮ 𝑛11𝑑𝑛1𝜆1 𝑚 + ⋯ , 𝑛12𝑑𝑛1𝜆1 𝑚 + ⋯ , … , 𝑛1𝑛𝑑𝑛1𝜆1 𝑚 + ⋯ let 𝑒 = (1, 1, … , 1)𝑇 = 𝑎1𝑤1 + ⋯ + 𝑎𝑟𝑤𝑟 , where 𝑤𝑝 is the principal right eigenvector corresponding to 𝜆𝑝. we have, 𝑒𝑇𝐴𝑚 = 𝑎1𝜆1 𝑚𝑤1𝑇 + ⋯ + 𝑎𝑟𝜆1 𝑚𝑤1𝑇 = �𝑛11 �𝑑11𝜆1 𝑚 + ⋯ , … , 𝑛1𝑛 �𝑑11𝜆1 𝑚 + ⋯ 𝑛 𝑖=1 𝑛 𝑖=1 �. given two columns of 𝐴, 𝑘 and 𝑠 we have, 𝑎𝑖𝑘 (𝑚) ∑ 𝑎𝑖𝑘 (𝑚)𝑛 𝑖=1 = 𝑛1𝑘𝑑𝑖1𝜆1 𝑚+⋯ 𝑛1𝑘 ∑ 𝑑𝑖1𝜆1 𝑚+⋯𝑛𝑖=1 and 𝑎𝑖𝑠 (𝑚) ∑ 𝑎𝑖𝑠 (𝑚)𝑛 𝑖=1 = 𝑛1𝑠𝑑𝑖1𝜆1 𝑚+⋯ 𝑛1𝑠 ∑ 𝑑𝑖1𝜆1 𝑚+⋯𝑛𝑖=1 since both numerators and denominators are polynomials in 𝜆𝑝𝑚, 𝑝 = 1, 2, … , 𝑟, and 𝜆1 = 𝜆𝑚𝑎𝑥 > �𝜆𝑝�, 𝑝 ≠ 1, we have for the ith entries of two arbitrary columns k and s: lim 𝑚→∞ 𝑎𝑖𝑘 (𝑚) ∑ 𝑎𝑖𝑘 (𝑚)𝑛 𝑖=1 = lim 𝑚→∞ 𝑎𝑖𝑠 (𝑚) ∑ 𝑎𝑖𝑠 (𝑚)𝑛 𝑖−1 = 𝑑𝑖1 ∑ 𝑑𝑖1𝑛𝑖−1 . definition: a positive matrix 𝐴 is said to be m-dominant if there exists 𝑚0 such that for 𝑚 ⩾ 𝑚𝑛 either 𝑎𝑖𝑘 (𝑚) ⩾ 𝑎𝑖′𝑘 (𝑚) or 𝑎𝑖𝑘 (𝑚) ⩽ 𝑎𝑖′𝑘 (𝑚)for all k and for any pair i and i'. ijahp article: saaty/ the analytic hierarchy process without the theory of oskar perron international journal of the analytic hierarchy process 275 vol. 5 issue 2 2013 issn 1936-6744 corollary: a positive reciprocal matrix is asymptotically m-dominant. proof. we have from theorem 2 that the normalized columns of ma are the same in the limit. since the elements in each row are identical, the result follows by choosing 0m to be the maximum of its values for each pair of rows. we now show that the rank of an inconsistent matrix a is determined in terms of the powers of .a to do this, we demonstrate that there is a method of estimating w which coincides with the normalized limiting columns of .a this method is precisely the eigenvalue method. theorem 3: lim 𝑚→∞ � 𝑎𝑖𝑘 (𝑚) ∑ 𝑎𝑖𝑘 (𝑚)𝑛 𝑖=1 � = 𝑤𝑖, 𝑖 = 1,2, … , 𝑛. proof: from lim 𝑚→∞ � 𝐴 𝑚𝑒 �|𝐴𝑚|� � = 𝑤, we have 𝑤𝑖 = lim𝑚→∞ � 1 �|𝐴𝑚|� �∑ 𝑎𝑖𝑘 (𝑚)𝑛 𝑘=1 . multiplying and dividing 𝑎𝑖𝑘 (𝑚) by ∑ 𝑎𝑖𝑘 (𝑚)𝑛 𝑘=1 , and then re-grouping the terms, we have, on distributing the limit with respect to the finite sum, 𝑤𝑖 = � lim𝑚→∞ � 𝑎𝑖𝑘 (𝑚) �|𝐴𝑚|� , ∑ 𝑎𝑖𝑘 (𝑚)𝑛 𝑖=1 ∑ 𝑎𝑖𝑘 (𝑚)𝑛 𝑖=1 � 𝑛 𝑘=1 = � � lim 𝑚→∞ 𝑎𝑖𝑘 (𝑚) ∑ 𝑎𝑖𝑘 (𝑚)𝑛 𝑖=1 � 𝑛 𝑘=1 � lim 𝑚→∞ ∑ 𝑎𝑖𝑘 (𝑚)𝑛 𝑖=1 �|𝐴𝑚|� �. by theorem 2 lim 𝑚→∞ 𝑎𝑖𝑘 (𝑚) ∑ 𝑎𝑖𝑘 (𝑚)𝑛 𝑖=1 is the same constant for all 𝑘 hence we have 𝑤𝑖 = lim𝑚→∞ 𝑎𝑖𝑘 (𝑚) ∑ 𝑎𝑖𝑘 (𝑚)𝑛 𝑖=1 � � lim 𝑚→∞ ∑ 𝑎𝑖𝑘 (𝑚)𝑛 𝑖=1 �|𝐴𝑚|� � 𝑛 𝑘=1 . since �|𝐴𝑚|� = ∑ ∑ 𝑎𝑖𝑘 (𝑚),𝑛𝑘=1 𝑛 𝑖=1 the proof is complete. there is a natural way to derive the rank order of a set of alternatives from a pairwise comparison matrix a. the rank order of each alternative is the relative proportion of its dominance over the other alternatives. this is obtained by adding the elements in each row in a and dividing by the total. however, a only captures the dominance of one alternative over each other in one step. but an alternative can dominate a second by first dominating a third alternative, and then the third dominates the second. thus, the first alternative dominates the second in two steps (along a path of length two). it is known that the result for dominance in two steps is obtained by squaring the pairwise comparison matrix. similarly dominance can occur in three steps, four steps and so on, ijahp article: saaty/ the analytic hierarchy process without the theory of oskar perron international journal of the analytic hierarchy process 276 vol. 5 issue 2 2013 issn 1936-6744 the value of each obtained by raising the matrix to the corresponding power. the rank order of an alternative is the average of the relative values for dominance in one step, two steps, and so on. we show below that when we take this infinite series of dominance along paths of length one, two, three, and so on and calculate its limiting value we obtain precisely the principal right eigenvector of the matrix a. this demonstrates that the eigenvector is derived deductively to obtain a relative scale among n alternatives from their matrix of pairwise comparisons. it is the desired solution because it preserves rank order rather than because it is a convenient criterion introduced for minimization purposes. we have theorem 4 below from saaty and vargas (1984). theorem 4: the relative dominance of an alternative is given by the solution of the eigenvalue problem 𝐴𝑤 = 𝜆𝑚𝑎𝑥𝑤. proof: the relative dominance of an alternative along all paths of length 𝑘 ⩽ 𝑚 is given by 1 𝑚 ∑ 𝐴 𝑘𝑒 𝑒𝑇𝐴𝑘𝑒 𝑚 𝑘=1 . let 𝑠𝑘 = 𝐴𝑘𝑒 𝑒𝑇𝐴𝑘𝑒 and 𝑡𝑚 = 1 𝑚 ∑ 𝑠𝑘𝑚𝑘=1 . note that lim mm t→∞ < ∞ . this a consequence of a theorem due to g. h. hardy (1949) which gives the necessary and sufficient conditions for a transformation of a convergent sequence to also be convergent. let 𝑇 be such a transformation mapping (𝑠1, … , 𝑠𝑚) → 𝑡𝑚 = � 𝑐𝑚,𝑘𝑠𝑘 ∞ 𝑘=1 . 𝑇 is regular if 𝑡𝑚 → 𝑠 as 𝑚 → ∞ whenever 𝑠𝑘 → 𝑠 as k → ∞. it is known (hardy, 1949) that 𝑇 is regular if and only if the following conditions hold: (1) ∑ �𝑐𝑚,𝑘� < 𝐻∞𝑘=1 (independent of 𝑚), (2) 𝑐𝑚,𝑘 → 𝛿𝑘 for each 𝑘, when 𝑚 → ∞, (3) ∑ 𝑐𝑚,𝑘 → 𝛿∞𝑘=1 when 𝑚 → ∞, (4) 𝛿𝑘 = 0 for each 𝑘, (5) 𝛿 = 1. here, 𝑐𝑚,𝑘 = � 1 𝑚 𝑓𝑜𝑟 1 ⩽ 𝑘 ⩽ 𝑚 0 𝑓𝑜𝑟 𝑘 > 𝑚. , thus, we have (1) ∑ �𝑐𝑚,𝑘� = ∑ � 1 𝑚 � = 1𝑚𝑘=1 ∞ 𝑘=1 , ijahp article: saaty/ the analytic hierarchy process without the theory of oskar perron international journal of the analytic hierarchy process 277 vol. 5 issue 2 2013 issn 1936-6744 (2) 𝑐𝑚,𝑘 = 1 𝑚 → 0 as 𝑚 → ∞. hence (4) 𝛿𝑘 = 0 for each 𝑘, (3) ∑ 𝑐𝑚,𝑘 = ∑ � 1 𝑚 � = 1 ∞𝑘=1 ∞ 𝑘=1 and hence (5) 𝛿 = 1. it follows that 𝑇 is regular. since 𝑠𝑘 = 𝐴𝑘𝑒 𝑒𝑇𝐴𝑘𝑒 → 𝑤 as 𝑘 → ∞ (saaty, 1980), where 𝑤 is the principal right eigenvector of 𝐴 we have, 𝑡𝑚 = 1 𝑚 ∑ 𝐴 𝑘𝑒 𝑒𝑇𝐴𝑘𝑒 → 𝑤𝑚𝑘=1 as 𝑚 → ∞ in input/output analysis in economics, multipliers are traced by raising the input/output matrix to higher and higher powers and taking their sums to obtain the overall impact of each sector of the economy on every other sector. still, another argument can be constructed from theorem 4 because for large m the normalized columns of am are the same and converge to the principal eigenvector. theorem 5: the cesaro sum, 𝑙𝑖𝑚 𝑚→∞ 1 𝑚 ∑ 𝐴 𝑘𝑒 𝑒𝑇𝐴𝑘𝑒 𝑚 𝑘=1 ,is the principal right eigenvector of a. proof: by theorem 4 of saaty and vargas (1984) we know that, lim 𝑚→∞ 1 𝑚 ∑ 𝐴 𝑘𝑒 𝑒𝑇𝐴𝑘𝑒 𝑚 𝑘=1 = lim𝑚→∞ ∑ 𝐴 𝑘𝑒 𝑒𝑇𝐴𝑘𝑒 𝑚 𝑘=1 multiplying lim 𝑚→∞ 1 𝑚 ∑ 𝐴 𝑘𝑒 𝑒𝑇𝐴𝑘𝑒 𝑚 𝑘=1 by 𝐴 on the left we have, 𝐴� lim 𝑚→∞ 1 𝑚 � 𝐴𝑘𝑒 𝑒𝑇𝐴𝑘𝑒 𝑚 𝑘=1 � = 𝐴� lim 𝑘→∞ 𝐴𝑘𝑒 𝑒𝑇𝐴𝑘𝑒 � = � lim 𝑘→∞ 𝑒𝑇𝐴𝑘+1𝑒 𝑒𝑇𝐴𝑘𝑒 �� lim 𝑘→∞ 𝐴𝑘+1𝑒 𝑒𝑇𝐴𝑘+1𝑒 � = � lim 𝑘→∞ 𝑒𝑇𝐴𝑘+1𝑒 𝑒𝑇𝐴𝑘𝑒 �� lim 𝑚→∞ 1 𝑚 � 𝐴𝑘𝑒 𝑒𝑇𝐴𝑘𝑒 𝑚 𝑘=1 � there is a vector 𝑦 = lim 𝑚→∞ 1 𝑚 ∑ 𝐴 𝑘𝑒 𝑒𝑇𝐴𝑘𝑒 𝑚 𝑘=1 and a constant 𝑑 = lim𝑘→∞ 𝑒 𝑇𝐴𝑘+1𝑒 𝑒𝑇𝐴𝑘𝑒 such that 𝐴𝑦 = 𝑑𝑦. under the assumption that a has r distinct eigenvalues 1,..., rλ λ with multiplicities 1,..., rn n , respectively, by using the jordan canonical form of a we can write 1a n bn−= where n is an invertible matrix and b is shown below: ijahp article: saaty/ the analytic hierarchy process without the theory of oskar perron international journal of the analytic hierarchy process 278 vol. 5 issue 2 2013 issn 1936-6744 𝐵 = � 𝐵1 0 ⋯ 0 0 𝐵2 ⋯ 0 ⋮ ⋮ ⋱ ⋮ 0 0 ⋯ 𝐵𝑟 � and 𝐵𝑝 = ⎣ ⎢ ⎢ ⎢ ⎢ ⎢ ⎡ 𝜆𝑝 0 0 0 ⋯ 0 1 𝜆𝑝 0 0 ⋯ 0 0 1 𝜆𝑝 0 ⋯ 0 0 0 1 𝜆𝑝 ⋱ 0 ⋮ ⋮ ⋮ ⋱ ⋱ ⋮ 0 0 0 ⋯ 1 𝜆𝑝⎦ ⎥ ⎥ ⎥ ⎥ ⎥ ⎤ , 𝑝 = 1, … , 𝑟. we have: 𝐴𝑘 = 𝑁−1𝐵𝑘𝑁, 𝐵𝑘 = ⎣ ⎢ ⎢ ⎡𝐵1 𝑘 0 ⋯ 0 0 𝐵2 𝑘 ⋯ 0 ⋮ ⋮ ⋱ ⋮ 0 0 ⋯ 𝐵𝑟𝑘⎦ ⎥ ⎥ ⎤ the matrix kpb is shown in equation 2 below. letting 𝑁−1 = 𝐷 = �𝑑𝑖𝑗� and 𝑁 = �𝑛𝑖𝑗� we write: 𝐴𝑘 = 𝑁−1𝐵𝑘𝑁 = 𝐷𝐵𝑘𝑁. the first 1n columns of ka are given by 11 k nd b , as shown in equation 3 below, and 𝐷𝑛1+1,𝑛2𝐵1 𝑘 as shown in equation 4 below. ijahp article: saaty/ the analytic hierarchy process without the theory of oskar perron international journal of the analytic hierarchy process 279 vol. 5 issue 2 2013 issn 1936-6744 (equation 2) 𝐵𝑝𝑘 = ⎣ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎡ 𝜆𝑝 𝑘 0 0 0 ⋯ 0 𝑘𝜆𝑝𝑘−1 𝜆𝑝𝑘 0 0 ⋯ 0 𝑘(𝑘−1) 2! 𝜆𝑝𝑘−2 𝑘𝜆𝑝𝑘−1 𝜆𝑝𝑘 0 ⋯ 0 𝑘(𝑘−1)(𝑘−2) 3! 𝜆𝑝𝑘−3 𝑘(𝑘−1) 2! 𝜆𝑝𝑘−2 𝑘𝜆𝑝𝑘−1 𝜆𝑝𝑘 ⋱ 0 ⋮ ⋮ ⋮ ⋱ ⋱ ⋮ 𝑘(𝑘−1)⋯(𝑘−𝑛𝑝+1) (𝑛𝑝−1)! 𝜆𝑝 𝑘−𝑛𝑝+1 𝑘(𝑘−1)⋯(𝑘−𝑛𝑝+2) (𝑛𝑝−2)! 𝜆𝑝 𝑘−𝑛𝑝+2 𝑘(𝑘−1)⋯(𝑘−𝑛𝑝+3) (𝑛𝑝−3)! 𝜆𝑝 𝑘−𝑛𝑝+3 ⋯ 𝑘𝜆𝑝𝑘−1 𝜆𝑝𝑘⎦ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎤ (equation 3) 𝐷𝑛1𝐵1 𝑘 = ⎝ ⎜ ⎜ ⎜ ⎜ ⎛ ⎣ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎡𝑑11𝜆1 𝑘 + 𝑑12 � 𝑘 1 �𝜆1 𝑘−1 + ⋯ + 𝑑1𝑛1 � 𝑘 𝑛1 − 1 �𝜆1 𝑘−𝑛1+1 𝑑21𝜆1 𝑘 + 𝑑22 � 𝑘 1 �𝜆1 𝑘−1 + ⋯ + 𝑑2𝑛1 � 𝑘 𝑛1 − 1 �𝜆1 𝑘−𝑛1+1 ⋮ 𝑑𝑛1𝜆1 𝑘 + 𝑑𝑛2 � 𝑘 1 �𝜆1 𝑘−1 + ⋯ + 𝑑𝑛𝑛1 � 𝑘 𝑛1 − 1 �𝜆1 𝑘−𝑛1+1 ⎦ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎤ ⎣ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎡𝑑12𝜆1 𝑘 + 𝑑13 � 𝑘 1 �𝜆1 𝑘−1 + ⋯ + 𝑑1𝑛1 � 𝑘 𝑛1 − 2 �𝜆1 𝑘−𝑛1+2 𝑑22𝜆1 𝑘 + 𝑑23 � 𝑘 1 �𝜆1 𝑘−1 + ⋯ + 𝑑2𝑛1 � 𝑘 𝑛1 − 2 �𝜆1 𝑘−𝑛1+2 ⋮ 𝑑𝑛2𝜆1 𝑘 + 𝑑𝑛3 � 𝑘 1 �𝜆1 𝑘−1 + ⋯ + 𝑑𝑛𝑛1 � 𝑘 𝑛1 − 2 �𝜆1 𝑘−𝑛1+2 ⎦ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎤ , … , ⎣ ⎢ ⎢ ⎢ ⎡𝑑1𝑛1𝜆1 𝑘 𝑑2𝑛1𝜆1 𝑘 ⋮ 𝑑𝑛𝑛1𝜆1 𝑘⎦ ⎥ ⎥ ⎥ ⎤ ⎠ ⎟ ⎟ ⎟ ⎟ ⎞ ijahp article: saaty/ the analytic hierarchy process without the theory of oskar perron international journal of the analytic hierarchy process 280 vol. 5 issue 2 2013 issn 1936-6744 (equation 4) 𝐷𝑛1+1,𝑛2𝐵1 𝑘 = ⎝ ⎜ ⎜ ⎜ ⎜ ⎛ ⎣ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎡𝑑11𝜆1 𝑘 + 𝑑12 � 𝑘 1 �𝜆1 𝑘−1 + ⋯ + 𝑑1𝑛1 � 𝑘 𝑛1 − 1 �𝜆1 𝑘−𝑛1+1 𝑑21𝜆1 𝑘 + 𝑑22 � 𝑘 1 �𝜆1 𝑘−1 + ⋯ + 𝑑2𝑛1 � 𝑘 𝑛1 − 1 �𝜆1 𝑘−𝑛1+1 ⋮ 𝑑𝑛1𝜆1 𝑘 + 𝑑𝑛2 � 𝑘 1 �𝜆1 𝑘−1 + ⋯ + 𝑑𝑛𝑛1 � 𝑘 𝑛1 − 1 �𝜆1 𝑘−𝑛1+1 ⎦ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎤ ⎣ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎡𝑑12𝜆1 𝑘 + 𝑑13 � 𝑘 1 �𝜆1 𝑘−1 + ⋯ + 𝑑1𝑛1 � 𝑘 𝑛1 − 2 �𝜆1 𝑘−𝑛1+2 𝑑22𝜆1 𝑘 + 𝑑23 � 𝑘 1 �𝜆1 𝑘−1 + ⋯ + 𝑑2𝑛1 � 𝑘 𝑛1 − 2 �𝜆1 𝑘−𝑛1+2 ⋮ 𝑑𝑛2𝜆1 𝑘 + 𝑑𝑛3 � 𝑘 1 �𝜆1 𝑘−1 + ⋯ + 𝑑𝑛𝑛1 � 𝑘 𝑛1 − 2 �𝜆1 𝑘−𝑛1+2 ⎦ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎤ , … , ⎣ ⎢ ⎢ ⎢ ⎡𝑑1𝑛1𝜆1 𝑘 𝑑2𝑛1𝜆1 𝑘 ⋮ 𝑑𝑛𝑛1𝜆1 𝑘⎦ ⎥ ⎥ ⎥ ⎤ ⎠ ⎟ ⎟ ⎟ ⎟ ⎞ thus, finally, we have: 𝐴𝑘 = ⎣ ⎢ ⎢ ⎢ ⎢ ⎢ ⎡�𝑑1𝑗𝑛𝑗1𝜆1 𝑘 + � 𝑑1𝑗𝑛𝑗1𝜆2 𝑘 + ⋯ 𝑛2 𝑗=𝑛1+1 𝑛1 𝑗=1 ⋯ �𝑑1𝑗𝑛𝑗𝑛𝜆1 𝑘 + � 𝑑1𝑗𝑛𝑗𝑛𝜆2 𝑘 + ⋯ 𝑛2 𝑗=𝑛1+1 𝑛1 𝑗=1 ⋮ ⋱ ⋮ �𝑑𝑛𝑗𝑛𝑗1𝜆1 𝑘 + � 𝑑𝑛𝑗𝑛𝑗1𝜆2 𝑘 + ⋯ 𝑛2 𝑗=𝑛1+1 𝑛1 𝑗=1 ⋯ �𝑑𝑛𝑗𝑛𝑗𝑛𝜆1 𝑘 + � 𝑑𝑛𝑗𝑛𝑗𝑛𝜆2 𝑘 + ⋯ 𝑛2 𝑗=𝑛1+1 𝑛1 𝑗=1 ⎦ ⎥ ⎥ ⎥ ⎥ ⎥ ⎤ . ijahp article: saaty/ the analytic hierarchy process without the theory of oskar perron international journal of the analytic hierarchy process 281 vol. 5 issue 2 2013 issn 1936-6744 let us assume 1 2 rλ λ λ≥ ≥ ≥ . then we have the ratio: (equation 5) lim 𝑘→∞ 𝑒𝑇𝐴𝑘+1𝑒 𝑒𝑇𝐴𝑘𝑒 = lim 𝑘→∞ ∑ 𝑑𝑖𝑗𝑛𝑖𝑗𝜆1 𝑘+1 + ∑ 𝑑𝑖𝑗𝑛𝑖𝑗𝜆2 𝑘+1 + ⋯𝑛2𝑖,𝑗=𝑛1+1 𝑛1 𝑖,𝑗=1 ∑ 𝑑𝑖𝑗𝑛𝑖𝑗𝜆1 𝑘 + ∑ 𝑑𝑖𝑗𝑛𝑖𝑗𝜆2 𝑘 + ⋯𝑛2𝑖,𝑗=𝑛1+1 𝑛1 𝑖,𝑗=1 since 1λ is the principal eigenvector of a and 1ay yλ= , then 𝑦 = lim𝑚→∞ 1 𝑚 ∑ 𝐴 𝑘𝑒 𝑒𝑇𝐴𝑘𝑒 𝑚 𝑘=1 is the principal right eigenvector of a. the ratio given in equation 5 shows how to calculate the principal eigenvalue without solving the characteristic equation or using sophisticated mathematical software. we apply it to the following inconsistent pairwise comparison matrix a : 𝐴 = � 1 2 3 4 1/2 1 5 6 1/3 1/5 1 7 1/4 1/6 1/7 1 � 𝐴2 = � 1 4 9 16 1/4 1 25 36 1/9 1/25 1 7 1/16 1/36 1/49 1 � 𝐴10 = � 923637. 1.02324 × 106 2.52682 × 106 7.73336 × 106 850356. 941992. 2.32613 × 106 7.11986 × 106 369456. 409271. 1.01053 × 106 3.09298 × 106 119539. 132434. 327011. 1.00078 × 106 � 𝐴11 = � 4.21087 × 106 4.66478 × 106 1.15187 × 107 3.52551 × 107 3.87669 × 106 4.29457 × 106 1.06043 × 107 3.24561 × 107 1.68418 × 106 1.86579 × 106 4.60711 × 106 1.41002 × 107 544954. 603711. 1.49077 × 106 4.56261 × 106 � computing the ratio below using the 10th and 11th power of the matrix a we obtain a first estimate of the eigenvalue. 𝑒𝑇𝐴11𝑒 𝑒𝑇𝐴10𝑒 = 136340459 29907404 = 4.558753 ijahp article: saaty/ the analytic hierarchy process without the theory of oskar perron international journal of the analytic hierarchy process 282 vol. 5 issue 2 2013 issn 1936-6744 going all the way to the 20th and 21st powers of the matrix and forming the ratio as shown below gives us a second somewhat different estimate of the eigenvalue that is quite close to the value computed by a presumably quite accurate commercial mathematical software package. 𝑒𝑇𝐴21𝑒 𝑒𝑇𝐴20𝑒 = 5.28609 × 1014 1.15953 × 1014 = 4.558805 the four eigenvalues of ,a obtained from the commercial mathematical software, are shown below: 1 2 3 4 4.558805319078529 0.03996796194816203 1.583542975555991 0.03996796194816203 1.583542975555991 0.4788693951822055 i i λ λ λ λ = = − + = − − = − we see that our final ratio has converged to the same value, 4.558805, to six decimal places, and we arrived at it using our method of raising the matrix to a sufficiently high power and computing the eigenvalue as a ratio of two successive powers. 3. conclusion so, in conclusion, we have shown that for ahp pairwise comparison matrices, which are positive and reciprocal, we do not need to use the beautiful and general theorem of perron. ijahp article: saaty/ the analytic hierarchy process without the theory of oskar perron international journal of the analytic hierarchy process 283 vol. 5 issue 2 2013 issn 1936-6744 references hardy, g.h. (1949) divergent series, new york: oxford university press. horn, r.a. & c.r. johnson (2012) matrix analysis, cambridge: cambridge university press. saaty, t.l. & l.g. vargas (1984) inconsistency and rank preservation, journal of mathematical psychology, 28(2), 205-214. saaty, t.l. (1980). the analytic hierarchy process, new york: mcgraw hill international. saaty, t.l. (1985). new light on the theorem of perron, homenaje al professor sixto rios, trabajos de estadistica y de investigacion operativa, 36(3), 253-257. abstract 1. introduction 2. consistent positive reciprocal matrices application of analytic hierarchy process (ahp) to find proper combination of process parameters of gmaw for getting good quality of welding joint for a particular set-up ijahp article: sabiruddin, bhattacharya, das/ selection of appropriate process parameters for gas metal arc welding of medium carbon steel specimen international journal of the analytic hierarchy process 252 vol. 5 issue 2 2013 issn 1936-6744 selection of appropriate process parameters for gas metal arc welding of medium carbon steel specimens kazi sabiruddin mechanical engineering department kalyani government engineering college kalyani741 235, west bengal, india e-mail: kazi4sabir123@gmail.com subhajit bhattacharya mechanical engineering department kalyani government engineering college kalyani741 235, west bengal, india e-mail: sbhattacharya63@gmail.com santanu das mechanical engineering department kalyani government engineering college kalyani741 235, west bengal, india e-mail: sdas.me@gmail.com abstract gas metal arc welding (gmaw) is a semi-automated process used widely for accurate welding in the fabrication industry. the selection of appropriate process parameters of gmaw is essential to obtain the desired weld quality. in the past, much work has been done investigating a variety of workpiece and electrode material combinations. in the present work, an analytical hierarchy process (ahp) based parametric optimization is tried in gas metal arc welding of c45 medium carbon steel specimens using carbon dioxide as the gas shield. the experiments were performed by varying three process parameters, weld speed, weld voltage and weld current. the ahp facilitates the selection of suitable process parameters to obtain a sound weld. in the present experimental domain, optimal conditions are evaluated to be at a weld voltage of 30 v, weld current of 160 a with a weld speed of 475.75 mm/min. keywords: ahp, welding, gmaw, mag, parametric optimization 1. introduction the analytic hierarchy process (ahp) is a simple, widely used decision making tool that can effectively solve a variety of complex multi-criteria problems hierarchically (saaty, 1977; saaty, 1980; vargas et al., 1990). the ahp was and is being employed to solve several managerial, manufacturing and production related decision making problems. this has also been utilized for the optimum selection of process parameters in different mailto:sdas.me@gmail.com rob typewritten text http://dx.doi.org/10.13033/ijahp.v5i2.184 ijahp article: sabiruddin, bhattacharya, das/ selection of appropriate process parameters for gas metal arc welding of medium carbon steel specimen international journal of the analytic hierarchy process 253 vol. 5 issue 2 2013 issn 1936-6744 welding processes. in a recent work, saaty (2009) discussed a method of taking a judgment in decision-making process in contrast to that used in usual science experiments. gas metal arc welding (gmaw) utilizes an arc maintained between the workpiece and an automatically fed wire electrode. argon, helium or a mixture of the two is usually used for welding different metals, and primarily nonferrous metals. when welding steel, some oxygen or carbon dioxide is usually added to improve the arc stability and reduce weld spatter. less costly co2 can be used alone when welding steel, provided that a deoxidizing electrode wire is employed in this process of gmaw known as mag (metal active gas) welding (khanna, 1995; nadkarni, 1996). the gmaw process can be easily mechanized to guarantee high productivity while maintaining good quality. however, to achieve good results, process variables of gmaw need be selected appropriately. nadkarni (1996) reported the relationship of mechanical properties of a welded joint with the degree of compositions of base material, and the effect of main process parameters of welding on the quality of the weld. detailed investigation of the effect of the chemistry of base material on the softening of haz was made by mohandas et al. (1999). hardness and microstructure were compared with the variation of the chemistry of the parent metal and the welding process to gain an understanding of the influence of the alloy chemistry, and the effect of different welding processes on the same low alloy steel. in another work, zumelzu et al. (1999) observed the effect of post-weld heat-treatment and external cooling on the gmaw product. they investigated the quality of the joining of 316l stainless steel specimens under varying conditions through the analysis of microstructure observation. kim and basu (1998) employed mathematical models of the gmaw process to select welding process parameters for obtaining the required weld-bead geometry. all these works reported some success in the respective electrode-workpiece material combinations under variations of the welding processes. in other works, modensi et al. (1999) evaluated the influence of small differences in wire characteristics on operational conditions of co2 gas shielded gmaw. data were evaluated using factorial analysis and graphical techniques to assess the effect of different wire characteristics on the weldment. the results showed that differences in wire diameter produced varying quality of a welded joint. an abductive polynomial network model of the gmaw process was established by simpson and hughes (2006). this network model enabled establishing the relationship between gmaw process parameters such as wire diameter, gas flow rate, welding speed, arc current and welding voltage on the weld bead penetration. the estimated value of weld bead penetration derived from network training was compared with the measured value. jones et al. (1992) observed a relationship between power input to the arc in gmaw, metal transfer process and base plate heating. optimized parameters evolved within the respective experimental domain in other investigations by sabiruddin and das (2005) and jaubari et al. (2007) involving mag welding of different steels under varying conditions. jaubari et al. (2007) recommended a gas mixture of argon, co2 and oxygen for the gmaw process to obtain substantial cost savings with a good control of spatter. in order to discover appropriate process parameters for desired weld quality, a number of works were also done employing the analytical hierarchy process (ahp) for optimal ijahp article: sabiruddin, bhattacharya, das/ selection of appropriate process parameters for gas metal arc welding of medium carbon steel specimen international journal of the analytic hierarchy process 254 vol. 5 issue 2 2013 issn 1936-6744 selection of process variables. ravisankar et al. (2006) and sabiruddin et al. (2009) tried to obtain quality butt joints of aluminium alloys and steels respectively through the selection of a suitable welding process and corresponding process parameters applying the ahp. the selection of appropriate process parameters was also successfully carried out by lai et al. (2009) by applying the ahp for resistance spot welding, and choosing typical edge preparation for obtaining sound welding was tried by liu et al. (2011) using the ahp that could have long fatigue life. in all these works involving the ahp, appropriate process conditions could be achieved to apply in practice. because the optimal process parameters are vital to the quality of the weldment, in this work, a number of experiments have been conducted to determine these parameters. using the ahp, different parameters of co2 gas shielded gmaw process were varied in order to find out an appropriate combination of process parameters. 2. details of experiments in this experimental work, gas metal arc welding of medium carbon steel flats is carried out on an esab india ltd. made gmaw set up with an autok 400 model. an indigenously developed system is used to move the welding torch along a straight path along the gap between the two steel flats to weld with a set speed to have weld deposition under a carbon dioxide gas shield. although there are many factors that influence a weld, we have chosen three main factors that determine heat input to the weld to investigate in this work. these factors are welding current, welding voltage and welding speed. heat input (q) is quite important in welding, and during the gmaw process, heat input is calculated by: q = 0.8 v i / s when v is weld voltage, i is weld current, and s is weld speed. based on the trial tests, a welding current of 140 a, 150 a and 160 a, a welding voltage of 25 v and 30 v, and a welding speed of 370.5 mm/min and 475.75 mm/min are chosen for the present experimental work on joining c45 medium carbon steel specimens as detailed in table 1. twelve experiments are carried out, and the parameters corresponding to each experiment are shown in table 1. without any preheating, specimens (size: 120 mm x 50 mm x 5 mm) are joined by a double-butt joint (in which both sides of the joint are welded) with a root gap of 1.5 mm. the weldment is brought to room temperature by air cooling. the joint is made in a horizontal position with the torch angle of 75° with the horizontal, using a low carbon steel wire electrode of 1.2 mm diameter. the weldments are visually inspected and tested through dye penetration. the presence of a visible crack, a blow hole, and the extent of spatter and uniformity of weld metal deposition is discovered through visual inspection. at some experimental conditions, bubbles of molten metal are scattered around the weld resulting in less penetration and reducing the aesthetic look of the weldment. penetration of weld metal is the depth of penetration of the weld metal going into the gap between two specimens being joined, and is observed through polishing a cut section of the weldment along its cross section. a ijahp article: sabiruddin, bhattacharya, das/ selection of appropriate process parameters for gas metal arc welding of medium carbon steel specimen international journal of the analytic hierarchy process 255 vol. 5 issue 2 2013 issn 1936-6744 bend test is done on a universal testing machine (fine spavy associates & engineers pvt. ltd., miraj, india, modeltun 200: 97/333) that observes the bending strength of the weldment. in this test, the butt welded specimen is placed on two supports, and a downward load is placed onto it at its middle and around the weld region. the bend test is continued up to a bend angle of 45 0 , or when any crack is formed in the weldment, and the corresponding bending load is noted. these observed results are utilized to design the ahp model that will be used to discover the appropriate process parameters. table 1 experimental conditions (alternatives) for welding medium carbon steel flats 3. discussion of experimental results the experimental results that were obtained are given in table 2. observation of weld quality, such as spatter, blow holes, penetration at the joint, uniformity of weld, and presence of surface cracks are shown in tabular form against each experiment. bending load obtained through the bend test, is also included in table 2. at a low travel speed of 370.5 mm/min with an weld voltage of 25 v and weld current of 140 a (experiment 1), large spatter and blow holes are found with less penetration; transverse and longitudinal surface cracks are also observed indicating quite poor weld quality. the bending load for this case is moderate. when weld current is set at 150 a in experiment 2, less spatter and thin welds are noticed. although good penetration is achieved, a few blow holes and apparent toe cracks are found. however, the weld joint appears to be good as it sustains a high bending load of 15.8 kn up to a bend angle of 45 o without fracturing. at a weld current of 160 a (experiment 3), spatter and blow holes are present less, and good penetration is observed. the presence of transverse and longitudinal cracks limits the bending load when the weld gets fractured. when weld voltage is increased to 30 v in experiments 4-6, increase in weld current also causes an increase in heat input from 0.54 kj/mm to 0.62 kj/mm. this results in deep penetration of the weld metal inside the joint. however, this high heat input to the weld sl. no. (alternatives) weld speed (mm/min) weld voltage (v) weld current (a) heat input (kj/mm) a1 370.5 25 140 0.45 a2 370.5 25 150 0.49 a3 370.5 25 160 0.52 a4 370.5 30 140 0.54 a5 370.5 30 150 0.58 a6 370.5 30 160 0.62 a7 475.75 25 140 0.35 a8 475.75 25 150 0.38 a9 475.75 25 160 0.40 a10 475.75 30 140 0.42 a11 475.75 30 150 0.45 a12 475.75 30 160 0.48 ijahp article: sabiruddin, bhattacharya, das/ selection of appropriate process parameters for gas metal arc welding of medium carbon steel specimen international journal of the analytic hierarchy process 256 vol. 5 issue 2 2013 issn 1936-6744 caused non-uniform contraction upon cooling leading to different types of cracks detected in the weld (see table 2). this gives low bending load of the weld. table 2 experimental observation of the weldment in experiments 7-9 with a welding speed of 475.75 mm/min, heat input is less (0.35 kj/mm to 0.4 kj/mm) leading to less penetration and a poor weld joint. this resulted in a considerably less bending load and indicated the presence of different types of cracks. on the other hand, at the weld speed of 475.75 mm/min, an acceptable quality of weld was observed corresponding to a weld voltage of 30 v at all the weld currents selected (experiments 10-12) having good bending strength. no crack, blow hole or spatter was sl. no. weld speed (mm/ min) weld voltage (v) weld current (a) spatter pene tration blow hole uniformity of weld deposition observed crack bend ing load (kn) a1 370.5 25 140 large less large poor weld deposition transverse, longitudinal under-bead crack 8.8 a2 370.5 25 150 less good less thin weld deposition toe crack 15.8 a3 370.5 25 160 less good less thin weld deposition transverse, longitudinal crack 9.2 a4 370.5 30 140 no very good very less good, continuous deposition longitudinal crack 7.8 a5 370.5 30 150 no very good very less good, continuous deposition at haz 9 a6 370.5 30 160 no good no good, continuous deposition transverse crack 10.2 a7 475.75 25 140 some less medium disconti nuous deposition toe crack 7 a8 475.75 25 150 little less medium not a smooth deposition transverse, longitudinal, under-bead crack 7.8 a9 475.75 25 160 no medium no good deposition transverse, longitudinal, root crack 6.5 a10 475.75 30 140 no good no good deposition no crack 13 a11 475.75 30 150 no good no good deposition no crack 13.4 a12 475.75 30 160 no good no good deposition no crack 16 ijahp article: sabiruddin, bhattacharya, das/ selection of appropriate process parameters for gas metal arc welding of medium carbon steel specimen international journal of the analytic hierarchy process 257 vol. 5 issue 2 2013 issn 1936-6744 detected in the weld portion or in the haz (see table 2). in these cases of 475.75 mm/min weld speed and weld voltage of 30 v, increase in heat input from 0.42 to 0.48 kj/mm resulted in an increased of bending load. 4. optimal selection of process parameters the analytic hierarchy process (ahp) was introduced by saaty (1977). the hierarchy structure used in this work is shown in figure 1. the goal or objective of the decisionmaking process is placed at the top level of the hierarchy. the goal or objective of the ahp in this work is the selection of optimum process parameter combination. the criteria and decision alternatives come in the subsequent descending levels. six criteria, as detailed in table 3, are considered in order to determine the best alternative out of a total of 12 alternatives listed in table 1. each alternative corresponds to a typical parametric combination for welding test corresponding to a typical experimental run. in this way, at two levels of weld speed and two levels of weld voltage, weld current is varied at three levels, and hence, twelve (2x2x3 = 12) experimental runs are performed. from these runs, the set of process parameters giving the best quality weld will be selected. the pair wise comparison matrices are formed by comparing an element with the elements of the next higher level. this determines the local priority weights. a typical pair wise comparison matrix (a) is shown in equation (1). here, aij (for i, j = 1,2,3,…….n) is the strength of preference of the alternative ai over aj corresponding to the criterion, c, aji = 1 / aij and aii = 1 for all values of i and j. a = (equation 1) the numerical values of aij are taken from the ratio scale (table 4). when all the elements of the matrix are selected, consistency of the entries of the matrix needs be checked. a comparison matrix is said to be consistent if, aij ajk = aik for all values of i, j and k (equation 2) for a consistent matrix, aij = wi /wj for all values of i and j (equation 3) where, w is the priority weight. c a1 a2 … an a1 a2 a3 . . an a11 a12 … a1n a21 a22 … a2n a31 a32 … a3n . . . . . . an1 an2 … ann ijahp article: sabiruddin, bhattacharya, das/ selection of appropriate process parameters for gas metal arc welding of medium carbon steel specimen international journal of the analytic hierarchy process 258 vol. 5 issue 2 2013 issn 1936-6744 usually, matrix a is rarely consistent, that is, aij ≠ aik akj for some elements of the matrix. then, a priority weight can be evaluated solving equation 4; aw = λm w (equation 4) where, w = (w1,w2,w3,..) t , λm  n, and λm is the largest eigen value of the matrix a. on the other hand, for a consistent matrix, equation 4 becomes; aw = nw (equation 5) for an inconsistent matrix, the degree of inconsistency is measured by consistency index (ci). ci = (λm n) / (n 1) (equation 6) a random index (ri) is computed through evaluating the consistency index of a matrix with the elements randomly generated from the range of ratio scale (1/9, 1/8, 1/7, …1, …, 7, 8, 9). the consistency ratio (cr = ci/ri) is then calculated, and a consistency ratio of up to 10% is considered acceptable. table 3 criteria selected for judging a sound weld local weights, wi of a comparison matrix for a criterion or an alternative are next determined by solving the equation 7. n wi = σ (aij wi )/λm, i=1,2,3,…….,n (equation 7) j=1 if pj (j =1, 2, 3, …, m) are the priority weights of n alternatives with respect to the jth criterion, and if qij are the priority weights of the criteria, then global weights (ri) of alternatives are calculated as m ri = σ (pj qij ), i =1,2,3, ………,n (equation 8) j=1 criterion no. criterion c1 no spatter c2 good penetration c3 no blow hole c4 good weld deposition c5 no surface crack c6 good bending load ijahp article: sabiruddin, bhattacharya, das/ selection of appropriate process parameters for gas metal arc welding of medium carbon steel specimen international journal of the analytic hierarchy process 259 vol. 5 issue 2 2013 issn 1936-6744 the largest global weight thus obtained is the optimum one, and a corresponding alternative is recommended as the optimum solution after saaty (1977), saaty (1980), vargas (1990) and das et al. (2003). -----goal -----criteria - -----alternatives figure 1. hierarchy structure of the ahp used table 4 ratio scale of comparison matrix the pair wise comparison matrix for criteria is constructed to solve the present problem of the selection of optimum parametric combination, and is given in table 5. this table shows preferences for selection of a criterion compared with the other criterion to judge a quality weld. good penetration and good bending load are more highly preferred than spatter, as a high bending load indicates good load sustaining capability of the weld. good weld penetration facilitates this. these weights of preferences have been introduced based on the experiences from different welding tests. good, uniform weld deposition has slightly less preference compared to penetration and bending load, as it has less influence preferential judgment rating extremely preferred 9 very strongly to extremely preferred 8 very strongly preferred 7 strongly to very strongly preferred 6 strongly preferred 5 moderately to strongly preferred 4 moderately preferred 3 equally to moderately preferred 2 equally preferred 1 a11: expt. 11 a2: expt. 2 a1: expt. 1 a12: expt. 12 c5: no crack c4: uniformity in weld deposition c3: no blow hole c2: good penetration c1: no spatter c6: bending load selection of optimum parameter combination in gmaw ijahp article: sabiruddin, bhattacharya, das/ selection of appropriate process parameters for gas metal arc welding of medium carbon steel specimen international journal of the analytic hierarchy process 260 vol. 5 issue 2 2013 issn 1936-6744 than the other factors when determining a good weld. the presence of cracks and blow holes is next in order of preference. even if there is an apparent presence of a crack and blow hole in a weld, if there is high bending strength along with deep penetration, the weld may still be usable. if a crack cannot propagate, and it is arrested summarily, it may not cause any failure in the component. a blow hole presents some discontinuity; however, if bending strength of the weld is good in spite of presence of a blow hole, the weld may be acceptable. with these considerations, priority weights of the criteria matrix are chosen, and the consistency ratio (cr) of the matrix comes out to be less than 10% which signifies consistency of the chosen values. in this work, no commercial software was used for calculations. the local weight is not calculated raising the powers to the pairwise comparison matrix; however, it is calculated using a computer programme written by the authors in c++ language in the following manner: i) first, the elements of each column are normalized by dividing each element of a column by the arithmetic sum of elements of that column. ii) local weight of a row is then calculated by arithmetic mean of the normalized row elements. table 5 the criteria matrix optimum quality weld c1 c2 c3 c4 c5 c6 local weight c1 1 1/7 1/3 1/6 ¼ 1/8 0.0317 c2 7 1 3 1 4 ½ 0.2295 c3 3 1/3 1 1/3 ½ 1/5 0.0716 c4 6 1 3 1 3 ½ 0.2107 c5 4 ¼ 2 1/3 1 1/4 0.0958 c6 8 2 5 2 4 1 0.3606 principal eigen value, λmax = 6.1521, cr = 0.004469 for each criterion (c), preferences of the alternatives (a) are tabulated in table 6 through table 11. table 6 shows the relative priorities within any two alternatives (experiments) considering the occurrence of no spatter (criteria, c1). as a1 alternative (experiment 1) has large spatter, and alternatives a4, a5, a6, a10, a11 and a12 show no spatter, compared to a1 alternative, these six alternatives are assigned a ‘very strong preference’ (a preferential strength of 7). on the other hand, presence of low spatter in a2 and a3 compared to that of a1 alternative, results in assigning the preferential rating of 4 (that is, moderately to strongly preferred). table 7 shows the pair-wise comparison matrix for alternatives with respect to criterion, c2 which is good penetration. compared to less penetration observed in a1 alternative, a7 and a8 alternatives have similar less penetration, and hence, are assigned a value of equal preference (that is 1). similarly, alternative a6 shows good penetration and has astrength of preference of 5 which signifies ‘strongly preferred’ compared with alternative a1. ijahp article: sabiruddin, bhattacharya, das/ selection of appropriate process parameters for gas metal arc welding of medium carbon steel specimen international journal of the analytic hierarchy process 261 vol. 5 issue 2 2013 issn 1936-6744 table 6 pair-wise comparison matrix for alternatives for criterion 1 (no spatter) c1 a1 a2 a3 a4 a5 a6 a7 a8 a9 a10 a11 a12 local weight a1 1 1/4 ¼ 1/7 1/7 1/7 1/5 1/6 1/6 1/7 1/7 1/7 0.0133 a2 4 1 1 1/5 1/5 1/5 ½ 1/3 1/3 1/5 1/5 1/5 0.0266 a3 4 1 1 1/5 1/5 1/5 ½ 1/3 1/3 1/5 1/5 1/5 0.0266 a4 7 5 5 1 1 1 4 2 2 1 1 1 0.1263 a5 7 5 5 1 1 1 4 2 2 1 1 1 0.1263 a6 7 5 5 1 1 1 4 2 2 1 1 1 0.1263 a7 5 2 2 ¼ 1/4 ¼ 1 ½ ½ ¼ 1/4 1/4 0.0387 a8 6 3 3 ½ 1/2 ½ 2 1 1 2 2 2 0.103 a9 6 3 3 ½ 1/2 ½ 2 1 1 2 2 2 0.103 a10 7 5 5 1 1 1 4 ½ ½ 1 1 1 0.103 a11 7 5 5 1 1 1 4 ½ ½ 1 1 1 0.103 a12 7 5 5 1 1 1 4 ½ ½ 1 1 1 0.103 principal eigen value, λmax = 12.689, cr = 0.0039 table 7 pair-wise comparison matrix for alternatives for criterion 2 (good penetration) c2 a1 a2 a3 a4 a5 a6 a7 a8 a9 a10 a11 a12 local weight a1 1 1/3 1/3 1/4 1/4 1/5 1 1 1/3 1/3 1/3 ¼ 0.0263 a2 3 1 1 1/3 1/3 1/4 3 3 1 1 1 1/3 0.0592 a3 3 1 1 1/3 1/3 1/4 3 3 1 1 1 1/3 0.0592 a4 4 3 3 1 1 1/3 4 4 3 3 3 1 0.1329 a5 4 3 3 1 1 1/3 4 4 3 3 3 1 0.1329 a6 5 4 4 3 3 1 5 5 4 4 4 3 0.2354 a7 1 1/3 1/3 1/4 1/4 1/5 1 1 1/3 1/3 1/3 ¼ 0.0263 a8 1 1/3 1/3 1/4 1/4 1/5 1 1 1/3 1/3 1/3 ¼ 0.0263 a9 3 1 1 1/3 1/3 1/4 3 3 1 1 1 ½ 0.0608 a10 3 1 1 1/3 1/3 1/4 3 3 1 1 1 ½ 0.0608 a11 3 1 1 1/3 1/3 ¼ 3 3 1 1 1 ½ 0.0608 a12 4 3 3 1 1 1/3 4 4 2 2 2 1 0.1190 principal eigen value, λmax = 12.3832, cr = 0.0019 table 8 illustrates the pair-wise comparison matrix for alternatives with respect to the criterion c3, that is, lack of a blow hole. compared to presence of large blow holes in experiment 1 (alternative a1), occurrence of less or no blow hole is assigned a priority ratio of moderately to strongly preferred (3 to 5). similarly, table 9, 10 and 11 are constructed for pair-wise comparison matrix of alternatives with respect to criteria c4 (uniformity of weld deposition), c5 (no crack) and c6 (bending load). ijahp article: sabiruddin, bhattacharya, das/ selection of appropriate process parameters for gas metal arc welding of medium carbon steel specimen international journal of the analytic hierarchy process 262 vol. 5 issue 2 2013 issn 1936-6744 table 8 pair-wise comparison matrix for alternatives for criterion 3 (no blow hole) c3 a1 a2 a3 a4 a5 a6 a7 a8 a9 a10 a11 a12 local weight a1 1 1/3 1/3 1/4 1/4 1/5 1 1 1/3 1/3 1/3 ¼ 0.0188 a2 3 1 1 1/3 1/3 1/4 3 3 1 1 1 1/3 0.0188 a3 3 1 1 1/3 1/3 1/4 3 3 1 1 1 1/3 0.0371 a4 4 3 3 1 1 1/3 4 4 3 3 3 1 0.0685 a5 4 3 3 1 1 1/3 4 4 3 3 3 1 0.0685 a6 5 4 4 3 3 1 5 5 4 4 4 3 0.1429 a7 1 1/3 1/3 1/4 1/4 1/5 1 1 1/3 1/3 1/3 ¼ 0.0371 a8 1 1/3 1/3 1/4 1/4 1/5 1 1 1/3 1/3 1/3 ¼ 0.0371 a9 3 1 1 1/3 1/3 1/4 3 3 1 1 1 ½ 0.1429 a10 3 1 1 1/3 1/3 1/4 3 3 1 1 1 ½ 0.1429 a11 3 1 1 1/3 1/3 1/4 3 3 1 1 1 ½ 0.1429 a12 4 3 3 1 1 1/3 4 4 2 2 2 1 0.1429 principal eigen value, λmax = 12.3832, cr = 0.0017 table 9 pair-wise comparison matrix for alternatives for criterion 4 (good weld deposition) c4 a1 a2 a3 a4 a5 a6 a7 a8 a9 a10 a11 a12 local weight a1 1 1 1 1/3 1/3 1/4 1 1 1/2 1/6 1/5 1/5 0.0285 a2 1 1 1 1/3 1/3 1/4 1 1 1/2 1/6 1/5 1/5 0.0285 a3 1 1 1 1/3 1/3 1/4 1 1 1/2 1/6 1/5 1/5 0.0285 a4 3 3 3 1 1 1/3 3 3 2 1/5 1/3 1/3 0.0708 a5 3 3 3 1 1 1/3 3 3 2 1/5 1/3 1/3 0.0708 a6 4 4 4 3 3 1 4 4 3 1/3 1/2 1/2 0.1185 a7 1 1 1 1/3 1/3 1/4 1 1 1/2 1/6 1/5 1/5 0.0285 a8 1 1 1 1/3 1/3 1/4 1 1 1/2 1/6 1/5 1/5 0.0285 a9 2 2 2 1/2 1/2 1/3 2 2 1 1/4 1/3 1/3 0.0506 a10 6 6 6 5 5 3 6 6 4 1 2 2 0.2354 a11 5 5 5 3 3 2 5 5 3 1/2 1 1 0.1557 a12 5 5 5 3 3 2 5 5 3 1/2 1 1 0.1557 principal eigen value, λmax = 12.2903, cr = 0.0015 ijahp article: sabiruddin, bhattacharya, das/ selection of appropriate process parameters for gas metal arc welding of medium carbon steel specimen international journal of the analytic hierarchy process 263 vol. 5 issue 2 2013 issn 1936-6744 table 10 pair-wise comparison matrix for alternatives for criterion 5 (no surface crack) c5 a1 a2 a3 a4 a5 a6 a7 a8 a9 a10 a11 a12 local weight a1 1 1/5 1/3 1/5 1/5 1/5 1/5 1/2 1/2 1/7 1/7 1/7 0.0148 a2 5 1 3 1 1 1 1 4 4 1/3 1/3 1/3 0.0866 a3 3 1/3 1 1/3 1/3 1/3 1/3 2 2 1/4 1/4 1/4 0.0410 a4 5 1 3 1 1 1 1 4 4 1/3 1/3 1/3 0.0866 a5 5 1 3 1 1 1 1 4 4 1/3 1/3 1/3 0.0866 a6 5 1 3 1 1 1 1 4 4 1/3 1/3 1/3 0.0866 a7 5 1 3 1 1 1 1 4 4 1/3 1/3 1/3 0.0866 a8 2 1/4 ½ 1/4 1/4 1/4 ¼ 1 1 1/5 1/5 1/5 0.0250 a9 2 1/4 ½ 1/4 1/4 1/4 ¼ 1 1 1/5 1/5 1/5 0.0250 a10 7 3 4 3 3 3 3 5 5 1 1 1 0.1536 a11 7 3 4 3 3 3 3 5 5 1 1 1 0.1536 a12 7 3 4 3 3 3 3 5 5 1 1 1 0.1536 principal eigen value, λmax = 12.404, cr = 0.0019 table 11 pair-wise comparison matrix for alternatives for criterion 6 (good bending load) c6 a1 a2 a3 a4 a5 a6 a7 a8 a9 a10 a11 a12 local weight a1 1 1/5 1 2 1 1 2 1 2 1/4 1/4 1/5 0.444 a2 5 1 5 6 5 5 6 5 6 2 2 1 0.2074 a3 1 1/5 1 2 1 1 2 1 2 1/4 1/4 1/5 0.4532 a4 ½ 1/6 1/2 1 1 ½ 1 1/2 1 1/5 1/5 1/6 0.2758 a5 1 1/5 1 1 1 ½ 1 1/2 1 1/5 1/5 1/6 0.3167 a6 1 1/5 1 2 2 1 2 1 2 1/4 1/4 1/5 0.4709 a7 ½ 1/6 1/2 1 1 ½ 1 1/2 1 1/5 1/5 1/6 0.2758 a8 1 1/5 1 2 2 1 2 1 2 1/4 1/4 1/5 0.4709 a9 ½ 1/6 1/2 1 1 ½ 1 1/2 1 1/5 1/5 1/6 0.0307 a10 4 1/2 4 5 5 4 5 4 5 1 1 1/2 0.1384 a11 4 1/2 4 5 5 4 5 4 5 1 1 1/2 0.1464 a12 5 1 5 6 6 5 6 5 6 2 2 1 0.2065 principal eigen value, λmax = 12.1907, cr = 0.0010 combining the pair-wise comparison matrix for criteria and that for alternatives, a global matrix is found as shown in table 12. in the present work, a computer programme is developed by the authors using c++ computer programme that takes the input of the element data of all matrices, and computes the consistency ratio and global weight following the same steps as that of manual calculations. ijahp article: sabiruddin, bhattacharya, das/ selection of appropriate process parameters for gas metal arc welding of medium carbon steel specimen international journal of the analytic hierarchy process 264 vol. 5 issue 2 2013 issn 1936-6744 table 12 global weights for alternatives alternatives weld speed (mm/min) weld voltage (v) weld current (a) global weight a1 370.5 25 140 0.0314 a2 370.5 25 150 0.103 a3 370.5 25 160 0.0425 a4 370.5 30 140 0.0715 a5 370.5 30 150 0.073 a6 370.5 30 160 0.1174 a7 475.75 25 140 0.0331 a8 475.75 25 150 0.0372 a9 475.75 25 160 0.0505 a10 475.75 30 140 0.1459 a11 475.75 30 150 0.1292 a12 475.75 30 160 0.1655 5. discussion of ahp results many process parameters influence gmaw or mag performance, and three main parameters are selected for the present investigation. the results, as detailed in table 2, show that the relationship among parameters chosen is not simple enough to draw a clear conclusion. therefore, the ahp is used in this work to discover the appropriate combination of process variables to obtain sound welding. experimental observations made in gmaw show that at a welding voltage of 30 v, 140-160 a welding current and 475.75 mm/min speed condition, a good quality weld is obtained. at a lower weld voltage, weldments begin to exhibit a number of weld defects. the ahp is used to find out the optimized process conditions by choosing suitable weights in the criteria matrix and the alternative matrices, and finally combining these weights to find the global matrix as shown in table 12. the expertise of the authors is utilized to choose the pair-wise comparison ratio, and these are comparable with some other published articles by sabiruddin et al. (2009) and muralidharan et al. (1999). if global weights against each alternative are arranged in descending order, the same appears to be: a12 > a10 > a11 > a6 > a2 > a5 > a4 > a9 > a3 > a8 > a7 > a1. therefore, the ahp indicates that the a12 alternative be chosen for gmaw for joining c45 medium carbon steel specimens. this corresponds to a setting of a weld voltage of 30 v, welding current of 160 a and 475.75 mm/min speed of the welding torch. although, a weld voltage of 30 v with 140-160 a weld current at 475.75 mm/min speed condition (experiments a10 and a11) have been found experimentally to be somewhat good for having a sound weld, the ahp refines the experimental results further to give the optimum welding process parameters within the domain of conditions considered in this work. conditions for experiments 6 and 2 may also be considered since they show global weights slightly less than that obtained from experiments 12, 10 and 11. ijahp article: sabiruddin, bhattacharya, das/ selection of appropriate process parameters for gas metal arc welding of medium carbon steel specimen international journal of the analytic hierarchy process 265 vol. 5 issue 2 2013 issn 1936-6744 6. conclusion the following conclusions may be drawn from the present investigation on joining c45 medium carbon steel specimens using gas metal arc welding employing 100% carbon dioxide as the shielding gas, and to find out the optimal set of process parameters utilizing the ahp. three process parameters, weld speed, weld voltage and weld current were varied to evaluate the best combination of process parameters corresponding to an experimental run within the domain of the present work. as these process parameters have varying influence on weld quality, the ahp was employed to discover the experimental run(s) giving the desired quality of weld. the ahp analysis considered six criteria for joining medium carbon steel specimens optimally, and a weld voltage of 30 v, weld current of 160 a, and welding speed of 475.75 mm/min were chosen for the selected electrode and workpiece. this result corresponded to the maximum global weight of the a12 alternative. this is also agreeable with the experimental results. at this condition, heat input is supposed to be quite favourable to facilitate good weld penetration, high bending strength, and lack of the presence of a crack, spatter and blow hole. therefore, this condition may be recommended for implementation to obtain sound welding. hence, while gas metal arc welding of medium carbon steel workpieces is used, the ahp helps managerial decision-making so that the management may prepare the process sheet specifying the evaluated optimized process parameters to set in order to have a defectfree, good welded joint. ijahp article: sabiruddin, bhattacharya, das/ selection of appropriate process parameters for gas metal arc welding of medium carbon steel specimen international journal of the analytic hierarchy process 266 vol. 5 issue 2 2013 issn 1936-6744 references das, s., islam, r. & chattopadhyay, a.b. (1997). a simple approach for on-line tool wear monitoring using the analytical hierarchy process. proceedings of the imeche, journal of engineering manufacture, 211, 19-27. jatua, r., sarkar, a., nandi, s. & das, s. (2003). on the effects of various parameters for gas metal arc welding of steels. proceedings of conference on advances and recent trends in manufacturing, kalyani, india, 143-150. jaubari, m.k., prasad, n.v., rao, a.t., premchander, v. & nagaraju, j.s. (2007). innovative cost effective concept in gmaw process to improve/ enhance weld aesthetics, quality and fatigue life. indian welding journal, 40(1), 40-45. jones, l.a., eager, t.w. & lang, j.h. (1992). metal transfer control in gas metal arc welding. (unpublished dissertation). massachusetts institute of technology, usa. khanna, o.p. (1995). welding technology. new delhi: khanna publishers. kim, i.s. & basu, a. (1998). a mathematical model of heat transfer and fluid flow in the gas metal arc welding process. journal of materials processing technology, 77, 17-24. lai, x., ji, c., luo, x. & deng, l. (2009). application of ahp method of orthogonal trial to selection of parameters in resistance spot welding. electric welding machine, 49, 7-8. doi: cnki: issn: 1001-2303.0.2006-09-003. liu, x. & gong, s.l. (2011). evaluation on the effect of weld shape on fatigue performance by analytic hierarchy process, advanced materials research, 146/147, 1839-1842. modensi, p.j. & deavelar, r.c. (1999). the influence of small variations of wire characteristics on gas metal arc welding process stability. journal of materials processing technology, 86, 226 – 232. mohandas, t., reddy, g.m. & kumar, b.s. (1999). heat-affected zone softening in high strength low-alloy steels. journal of materials processing technology, 88, 264–294. muralidharan, c., anantharaman, n., balasubramanian, v. & deshmukh, s.g. (1999). selection of a welding process using analytic hierarchy process. journal of the institution of engineers (india), 80, 51–54. nadkarni, s.v. (1996). modern arc welding technology. new dehli: oxford and indian book house. ravisankar, v., balasubramanian, v. & muralidharan, c. (2006). selection of welding process to fabricate butt joints of high strength aluminium alloys using analytic hierarchic process., materials & design, 27, 373-380. http://www.scientific.net/amr.146-147.1839 http://www.scientific.net/amr.146-147.1839 ijahp article: sabiruddin, bhattacharya, das/ selection of appropriate process parameters for gas metal arc welding of medium carbon steel specimen international journal of the analytic hierarchy process 267 vol. 5 issue 2 2013 issn 1936-6744 saaty, t.l. (1977). a scaling method for priorities in hierarchical structures. journal of mathematical psychology,15(3), 234-281. saaty t.l. (1980). analytic hierarchy process. new york: mcgraw-hill. saaty, t.l. (2009). an essay on how judgment and measurement are different in science and in decision making. international journal of the analytic hierarchy process, 1(1), 61-62. sabiruddin, k. & das, s. (2005). characterisation of gas metal arc welding of different steels. proceedings of all india workshop on advances in welding science and technology, shibpur, howrah, india. sabiruddin, k., das, s. & bhattacharya, a. (2009). application of the analytic hierarchic process for optimization of process parameters in gmaw. indian welding journal, 42(1), 38-46. simpson, w. & hughes, p.w. (2006). industrial application of welding signatures for real time quality assurance and process improvement, (unpublished dissertation). school of electrical and information engineering, university of sydney, australia. vargas, l.g. (1990). an overview of the analytic hierarchy process and its applications. european journal of operations research, 48, 72–80. zumelzu, e., sepulvede, j. & ibarra, m. (1999). influence of microstructure on the mechanical behaviour of welded 316l ss. journal of materials processing technology, 94(1), 36–40. ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 355 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 report on an analytic network process (anp) model to estimate the benefits, opportunities, costs, and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms amos n. guiora professor of law s.j. quinney college of law university of utah marcel c. minutolo professor of strategic management school of business robert morris university luis g. vargas professor of business analytics & operations katz graduate school of business university of pittsburgh abstract balancing public good with individual rights is a difficult task; gun policies attempt to do just this. to ensure public safety, local, state, and federal agencies piece together policies that each entity believes will meet the needs of public welfare. when legislating new gun policies, the impact the policies have on gun owners are perceived as a zero-sum game; some groups are perceived to gain while others think they are losing, but the reality is much more nuanced. the reason the impact of these policies on all lawful gun owners has been considered a zero-sum game is largely because to date there has been no research measuring the impact. further, there have been no attempts to quantify the impact that the policies have on lawful gun owners. the sole argument that has been made is about constitutionality. in this paper, we develop an approach based on the analytic hierarchy process (ahp). the approach allows us to develop criteria for evaluating the impact of these policies on lawful gun owners and generate priorities for the criteria from pairwise comparisons. criteria are compared in pairs, thus the term pairwise comparisons. this allows us to score, as with a scorecard model, gun policies for various types of gun owners with respect to the criteria according to the benefits, opportunities, costs, and risks, thereby determining the impact of each policy. we show how relative measurements are derived and illustrate with an example how ratings (absolute measurement) work in the context of an organ donor model. next, we ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 356 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 develop a gun policy model hierarchy to show how the approach might be structured. a hypothetical gun policy  a single shot only policy  is evaluated against the model for various types of lawful gun owners to illustrate the approach. since the model will need to be tested with a large group of stakeholders, we discuss how to develop the priorities from large groups. finally, we discuss how this approach might be implemented. this work may be of interest to policy makers at the federal, state, and local levels since gun control policy happens at every level of governance in the united states. additionally, this work may be of interest to lawful gun owners who may wish to apply the approach to their own context to see how they may be personally impacted by a given gun control policy. finally, researchers may find this research of interest because the approach is applicable in other contexts and presents a novel way to consider the impacts of policy decisions. keywords: gun laws; benefits; opportunities; costs and risks; analytic hierarchy process 1. introduction this paper is not about the gun debate in america, but about how gun control policies impact legal gun owners. in the process of developing a model to measure the impact of gun control policies, we need to introduce a theory based on relative measurement. relative measurement is used when there are no scales to measure certain attributes known as intangibles. an intangible is an attribute that cannot be measured with a scale common to all members of a society, for example, love. there is no common scale to represent how much you love a person. however, everyone has an idea of the intensity of their love for other people. on the other hand, a tangible attribute is one that society measures using a commonly agreed upon tool. for example, temperature, distance, and weight, are measurable using a scale that society has agreed to use, but even when different societies use different scales, the results can be compared depending on the type of scale used. the impacts of gun control policies are not just monetary for some users, but also include intangibles, like not being able to enjoy hunting, or target shooting, or being afraid if they do not have a way to defend themselves. we are not only trying to understand how gun control policies impact legal gun owners, but also which type of gun owner is most affected. there are not many statistics about how existing laws impact behavior, let alone how they impact people’s lives. however, we believe that society as a whole has an idea about how those laws impact and influence what we do from day to day. so, why do we want to measure the impact gun policies have on legal gun owners? perhaps we want to make sure that we allocate resources more efficiently when developing gun control policies. for example, in countries where citizens are not allowed to have any type of gun, there may not be a need for a background check policy. this is not the case in the united states. ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 357 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 studying the impact that a law has on the society in which it is applied helps to better understand the balance that exists between advantages and disadvantages the law creates. (forji, 2010) writes: “it is in the interest of the legal order that it strive for genuine justice in order to dissuade individuals from inducements to contra bonos mores. when from a particular conduct, the benefits or advantages are more than the disadvantages or sufferings to an individual; that individual can certainly be expected to opt for compliance, because the benefits (pleasures) of compliance outweigh the disadvantages (pains) of violation. the reverse conduct would be true for a legal system which is prone to injustice, hardship and sufferings to some or most of its subjects, given that the latter beside their pains are not giving any motivation to abide to the legal order. in this case, the advantages of violation seem just as good if not better as of compliance (p. 86).” this paper is not addressing the issue of what law helps prevent more suicides, mass murders or domestic violence. perhaps, that could be the subject of another paper. here, we are interested in measuring the impact of gun control policies on people who legally own a gun. because the impacts of these gun policies involve intangibles, we need to use relative measurement (see appendix a). the example that follows shows how to use ratings (scorecard) to develop an ordered list of potential recipients of a cadaver liver from a donor. 2. example of how absolute measurement (or ratings) works cook et al. (1990) proposed the use of an ahp model to develop a rating system using absolute measurement for the allocation of cadaver livers for orthotopic transplantation. five major criteria for comparison were established, defined, and rated relative to one another. they were logistic considerations, tissue compatibility, waiting time, financial considerations, and medical status. subcriteria were also established and prioritized in relative terms (see figure 2). patients that met appropriate inclusion screening criteria were rated according to the scales in figure 3. the final weighting can be used to develop an alternative to the rigid computerized multifactorial point system that existed at the time of the publication (see figure 1). in figure 4, the existing list in 1990 showed that patient #6 was the desired choice with a score of 39.76 points. this score was obtained by assigning points to the patient according to the scales in figure 3. the donor points 8  can be seen in the upper righthand corner of figure 6 next to the word logistics. using cook et al.’s system, the list of patients given in figure 4 were also scored (or rated). the patient that was selected for the transplant is now in the 7 th position (see figure 5). clearly, both systems do not provide the same rankings. how does one decide which one is more acceptable? some criteria to consider would be accountability, clarity of weight assignment and capability to explain how the weights were derived. ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 358 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 figure 1 scoring system for liver assignment in 1990 logistics medical urgency distance donor recipient class points (miles) points points 0-50 12 6 1 4 50-500 10 5 2 8 500-1000 8 4 3 12 1000-1500 6 3 4 16 1500-2000 4 2 5 20 2000-2500 2 1 6 24 > 2500 0 0 waiting time points (% to the time waited) longest wait 10 ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 359 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 figure 2 equitable allocation of livers for transplantation figure 3 priorities of criteria, subcriteria and rating scales equitable allocation of livers for transplantation logistics compatibility waiting time financial medical status urgency replant complexity graft preservation abo x-matching size adequate insurance $ but not adequate insurance no dollars no insurance primary anatomy 0-6 months urgency 1, 2, 3 urgency 4, 5, 6 re-transplant 1 multiorgan <10hrs 6 months 1 yr re-transplant 2 "routine" 11-20hrs 1-2 yrs no deterrent deterrent >20hr >2 yrs age cancer hiv hept etoh drug 0.065 ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 360 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 figure 4 example of a liver donor and an ordered list of potential recipients ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 361 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 *numbers in the column total do not add to 1 because of rounding off error. figure 5 final ordered list of patients from figure 4 obtained using the proposed ahp model alternative total logistic|complexity|(l: .206)logistic|graft preservation|(l: .794)compatibility|(l: .502)waiting time|(l: .037)financial|(l: .065)medical status|urgent|(l: .347)medical status|replant|(l: .653) patient 2 0.092 routine <10hrs perfect 0-6 months insurance #4 primary patient 3 0.082 routine <10hrs perfect 0-6 months no $ #3 primary patient 7 0.074 routine 11-20hrs perfect 13-24 months insurance #3 primary patient 8 0.072 routine 11-20hrs perfect 7-12 months insurance #3 primary patient a 0.071 routine 11-20hrs perfect 0-6 months insurance #3 primary patient 4 0.07 difficult 11-20hrs perfect >24 months no $ #3 primary patient 6 0.07 difficult >20hrs perfect >24 months $ #3 primary patient 9 0.07 routine 11-20hrs perfect 0-6 months $ #3 primary patient 1 0.068 routine >20hrs perfect 0-6 months insurance #3 primary patient c 0.053 routine 11-20hrs adequate 13-24 months $ #5 secondary patient 5 0.051 routine 11-20hrs adequate 0-6 months insurance #5 primary patient b 0.049 routine 11-20hrs adequate 7-12 months $ #4 secondary patient 11 0.048 routine 11-20hrs adequate 0-6 months insurance #4 primary patient 10 0.045 routine 11-20hrs adequate >24 months $ #3 primary patient 12 0.043 difficult >20hrs adequate 0-6 months insurance #4 primary patient d 0.042 difficult 11-20hrs adequate 7-12 months no $ #4 primary ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 362 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 3. hierarchy of impacts of gun control policies we propose building a similar system as illustrated in the liver donor model to evaluate the effects of gun control policies on lawful owners of firearms. the difference resides with considering not just benefits and costs, but the uncertainty in benefits, that we call opportunities, and the uncertainty within costs, termed risks. the goal of the model is to measure the impact that gun control policies have on legal gun owners. thus, if we model it as a hierarchy we would have the following: 1. goal: measure the impact of a gun control policy on lawful gun owners. 2. strategic criteria: at the macro-level, gun control policies have an impact on society as a whole. when considering the enactment of gun control policies, policymakers need to consider strategic issues. the political, economic, social, technological environmental, and legal (pestel) framework is commonly used to capture the macro-environment when making strategic decisions. a. political impact: these impacts may include things like the pressure that is being placed on legislators to enact or repeal gun laws; the importance that gun control measures play in an election cycle; the donations being made by lobbying groups; etc. b. economic impact: these impacts may be seen in the growth or contraction around the gun industry related to gun manufacturing. for instance, the implan 1 application has industry code 257, “small arms, ordnance, and accessories manufacturing,” employing 20,768 people with $1.6m in labor income. gun control laws that impact this industry may have an economic impact through direct, induced, and spillover effects. c. social impact: when considering the impact that a gun policy has, social issues need to be considered as well. for instance, there are matters of public good, safety, and health that may be impacted. d. technological impact: research and development, among other technological areas, may be impacted by gun control policies. e. environmental impact: the manufacturing, use, and disposal of guns, accessories, and material may have an environmental impact. hence, consideration of the environment needs to be taken. f. legal impact: the legal framework needs to be considered since gun control policies may be affected by existing laws and statutes. 3. benefits, opportunities, costs, and risks 4. intensity scales for each of the benefits, opportunities, costs, and risks. 1 economic impact analysis for planning  https://www.implan.com ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 363 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 the proposed hierarchy is illustrated in figure 6. it is common sense to assume that not all gun owners think the same about the impact of gun control policies. thus, the strategic criteria need to be prioritized according to different types of gun owners, including 2 : a. recreational i. hunters – there are a variety of types of hunters. for instance, a bird hunter might prefer a shotgun, whereas a deer hunter might prefer a cartridge. there are some hunters that prefer black-powder, whereas others may prefer high-power. some hunters hunt for recreation, whereas others hunt to supplement their food. we do not separate the various types of hunters, but recognize there is a variety. ii. shooting – paper fixed targets, skeet, and pop-up. iii. enthusiasts – these are generally people who feel an affinity to guns. iv. collectors – this category of gun owner tends to possess them for the potential appreciation in value. they may display their guns and could have many in their possession. v. historical reenactment – this category may have weapons for recreating battles such as gettysburg, may participate in demonstrations such as veteran’s day parades, or take part in filming. b. protectionists i. private security guards – these individuals are typically contracted out through a firm to cover banks, buildings, and other locations. they may be required to own their own gun, though their employer may provide one. ii. private citizens – homeowners, personal protection, automobile. iii. law enforcement – these individuals will often be issued a service weapon, but may also have personal weapons that they may decide to carry while working and may also carry while off-duty. iv. retired and active duty military – military members may have basic weapons training, but may also have advanced weapons training. these individuals may have a variety of types of weapons. c. gun shop owners dealers d. survivalists – these individuals may purchase weapons with the idea of needing them should social structures and services fall into disarray. 2 these categories are not mutually exclusive, and a gun owner might fall into multiple categories. it is important to distinguish the categories since a gun policy may affect one type of gun owner more than another. ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 364 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 e. paramilitary – these individuals may possess weapons for the purpose of serving in a militaristic type of organization which is not government backed or supported. the selection of the types of gun owners is based, in part, on a review of the literature for what lawful gun ownership looks like ((moore, 1983). to develop priorities for each group, we needed to compare the strategic criteria according to their perceived importance for each group. for example, for a given group, given two criteria, e.g., political and economic, which one is more important when considering a gun control policy, and how much more important? these pairwise comparisons of the criteria are then used to derive priorities that different groups assign to the different dimensions of a gun control policy. next, we needed to identify the impacts of each specific gun control policy. any gun control policy has several favorable and unfavorable aspects to consider. some of these are certain, others are less so and only likely to materialize. the favorable concerns that are certain are called benefits (b) while the unfavorable ones are called costs (c). the uncertain concerns of a decision that are positive are characterized as opportunities (o) that the decision might create and the negative as risks (r). these concerns can have monetary and non-monetary implications. thus, intangible attributes are at the heart of the measurement of the impacts of a gun control policy on legal owners of guns. the specific impacts of a gun control policy need to be evaluated in terms of the concerns (e.g., benefits), and each of the concerns is measured on its own scale. for example, assume that gun control policies impact the benefit of protection with different intensity that can range from none (i.e., no protection) to very high (i.e., very high protection). the scale for each intangible impact needs to be developed using pairwise comparisons. let us assume we want to construct the intensity scale for the protection benefit. the following matrix shows an example of the pairwise comparisons obtained in response to the following question: “given two intensity levels, e.g., none and low, how much more intense (important) is low than none using the fundamental scale from table 1 in appendix a.” 1 1 1 1 3 5 7 9 1 1 1 3 5 7 1 1 3 5 1 3 1 2 3 4 5 priorities ideal 1. none 1 0.033 0.06 2. low 3 1 0.063 0.12 3. medium 5 3 1 0.129 0.25 4. high 7 5 3 1 0.261 0.51 5. very high 9 7 5 3 1 0.513 1                the values under the word “priorities”, in the table above, are the relative intensities of the benefit protection rating scale. sometimes, it is more convenient to use the ideal form which is obtained by dividing each relative priority by the largest value. in this example, ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 365 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 each priority is divided by 0.513. this will be done for each impact in the concerns identified under benefits, opportunities, costs, and risks. these scales will have to be constructed individually for benefits, opportunities, costs, and risks because the intensity of the scale’s levels depends on the dimension for which they will be used. for example, the pain of a loss or cost is usually perceived to be greater than the pleasure of an equal gain or benefit. thus, the scales will have to reflect these perceptions (kahneman & tversky, 1979). benefits 3 : a. security and peace of mind – the ability of the gun owner to derive a sense of calmness from knowing that she/he owns a weapon that is available for use should the need arise. the sense of security and peace of mind derives from several sub-criteria: 1. protection – under protection, a lawful owner views the gun as an instrument of protection. the protection may be for oneself, one’s family or perhaps property. the scale used to rate potential policies or laws ranges from “provide no protection” to “provides a lot of support for protection or “very high”. 2. respect – under this criterion, the gun owner may possess the weapon out of a sense of esteem derived from the perception that others see the owner as possessing some ‘power’. the scale used to rate the policy or law ranges from “none, it provides no support for a sense of respect” to “very high, the policy supports the owner’s sense of respect.” 3. posturing – this criterion derives from a recognition that there are some gun owners who may possess the weapon in order to present a certain pretense. gun policies and laws may diminish or support the ability of the lawful gun owner to present this pretense. like protection and respect, the scale used to evaluate the policy ranges from none to very high. b. psychological – the security and peace of mind cluster largely constitutes external considerations. this cluster derives from internal positive feelings in this case. the difference between this cluster and the security and peace of mind cluster is subtle, but we estimated that the source of the benefit is derived from differing motivations. it is comprised of the following sub-criteria: 1. self-esteem – this is a measure of how the lawful gun owner feels about him or herself because of the fact that he or she can own a gun. the scale ranges from none to very high. 2. identity – this is a measure of the degree to which gun ownership is integral to how the owner identifies him or herself. the scale ranges from none to very high. 3 here we consider direct, known gains that the lawful gun owner receives. ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 366 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 3. turf – here, we capture the idea that one might own a gun with the intent of defending one’s territory or have the means to defend one’s territory if necessary. this is different from self-defense or protection where the owner is concerned about his or her home. 4. community (belonging) – there are some gun owners who possess the gun because they feel a sense of belonging with others. the act of ownership facilitates the inclusion into a group that is larger than just the individual, e.g., being part of the national rifle association or a local hunting club. c. recreation – this category of lawful gun owners possess guns primarily for recreational purposes or enjoyment. 1. hunting – these are lawful gun owners whose primary reason for gun ownership is for hunting purposes (fowl, deer, etc.). 2. ornamental – gun ownership for the purpose of display. 3. stress release – gun ownership for the purpose of releasing tension. 4. target shooting – these gun owners enjoy shooting at targets (paper, popup, clay, etc.). the target shooters are more competitive than those that shoot for stress release and may participate in competitions. d. constitutional right – this category of individuals tends to possess guns because they consider it a constitutional right. the possession of the weapon is an act of maintaining the right itself. 1. freedom – these types of gun owners possess weapons as an expression of independence. 2. militia – these gun owners possess weapons as an act of upholding the second amendment imperative to maintain “a well-regulated militia.” 3. bear arms these gun owners possess weapons as an act of upholding the second amendment right to “bear arms.” opportunities 4 : a. insurance – this is largely a measure of an individual wanting to keep a gun for safety in case something should happen. b. appreciation – some gun owners hold their guns for an investment. c. identity – these gun owners feel that the gun is a function of their identity and without it may lose some sense of self. costs 5 : a. economic – these include the fiscal costs associated with the implementation of a gun control policy. 1. purchase price – defined as the initial purchase costs associated with the gun or related matter need for the ownership of the gun. 4 potentialities, things that the gun owner may get at some time in the future. 5 direct cost of lawful gun ownership. ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 367 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 2. operational – the ongoing costs of maintaining the ownership of the gun. 3. license – the monetary costs associated with legally owning the gun. 4. insurance – the costs associated with indemnifying the gun owner against potential claims. b. social costs – the costs that society may incur from the implementation of the policy, such as lives lost, injuries, and so on. c. time – the amount of time spent maintaining legal ownership that may result from applying for a license, waiting, etc. d. holding – costs associated with maintaining required storage of the gun associated with policies. e. reporting – costs associated with reporting requirements that may be imposed by the gun policy. risks 6 : a. safety – the risk to the environment (i.e., damage to personal property). b. bodily harm – the injury to oneself or others. c. noncompliance – the ability to maintain legal ownership status as policies change. d. identity – the risk that one may lose some sense of self through the implementation of a particular policy or policies. e. constitutionality – the risk that a policy may be perceived as counter to the intent of the constitution. f. restrictions – defined as the extent to which the lawful gun owner may perceive that the policy places additional or undue restrictions on the owner. g. liability – the perception that the policy places additional liability on the gun owner. h. surrender – the extent to which the gun owner perceives that she or he may be required to surrender personal property. i. intrusion the extent to which the policy is perceived as an imposition in the life of the gun owner. it is important to understand the relative priorities that gun owners place on the criteria to determine the impact that a gun policy has. in a 2019 study, the pew research center 7 found that 67% of gun owners state that protection is the number one reason why they own guns, 38% claim ownership for hunting, 30% for sport shooting, 13% for gun collecting, and 8% for their work. while it may be true that the gun owner’s primary reason for owning a gun is protection, this does not mean that they do not use it for sport 6 the future, potential negative impacts of lawful gun ownership. 7 pew research center (2019), “7 facts about guns in the u.s.” https://www.pewresearch.org/facttank/2019/10/22/facts-about-guns-in-united-states/ https://www.pewresearch.org/fact-tank/2019/10/22/facts-about-guns-in-united-states/ https://www.pewresearch.org/fact-tank/2019/10/22/facts-about-guns-in-united-states/ ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 368 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 shooting or work. hence, by capturing the relative priorities, we are able to get a more nuanced view of gun ownership and establish a more accurate impact score. figure 6 is a graphic representation of the sample hierarchy developed here. the details of the various clusters inside each of the boxes are presented in table 3 along with the intensity scales. table 4 gives the priorities derived for the strategic criteria and the benefits, opportunities, costs, and risks. we derived these priorities to illustrate how the framework can be used to evaluate the impact of specific gun control policies on different groups of legal owners of firearms. to obtain the impact of a gun control policy in terms of, e.g., benefits, we multiplied for each benefit, e.g., protection, its priority (0.0555) by the numeric value of the intensity level with which that benefit is perceived, e.g., none (0.065), and summed across all the benefits. the same was done for opportunities, costs, and risks. the resulting values were combined using the formula: 𝐵𝑒𝑛𝑒𝑓𝑖𝑡𝑠 (𝐵)×𝑂𝑝𝑝𝑜𝑟𝑡𝑢𝑛𝑖𝑡𝑖𝑒𝑠 (𝑂) 𝐶𝑜𝑠𝑡𝑠 (𝐶)×𝑅𝑖𝑠𝑘𝑠(𝑅) . we illustrate its applicability with a fictitious gun control policy we invented for this purpose that we call the “single shot” law. figure 6 hierarchy to measure the impact of gun control policy goal: measure the impact of a gun control policy strategic criteria: political economic social technological environmental legal benefits: a. security & peace of mind a. protection b. respect c. posturing b. psychological a. self-esteem b. identity c. turf d. community (belonging) c. recreation a. hunting (satisfaction) b. ornamental c. stress release d. target shooting d. constitutional right a. freedom b. militia c. bear arms opportunities: a. appreciation (value) b. identity c. ‘insurance’ (safety) costs: a. monetary costs a. purchase price b. maintenance c. license d. insurance b. social costs c. time d. holding (storage) e. reporting risks: a. constitutionality b. identity risks c. liability d. non-compliance e. restrictions f. safety h. bodily harm to self or others i. surrender j. intrusion ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 369 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 4. illustrative example for illustrative purposes, we developed an example case wherein we simulated an ‘expert’ opinion to develop the priorities of the strategic criteria and the benefits, opportunities, costs, and risks (bocr) model. the choice of a fictitious policy was intentional to avoid any prior bias of the policy that reviewers may have. the policy that we chose to evaluate was a “single shot” law. the “single shot” law would make it such that all lawful gun ownership would be limited to firearms that were capable of only a single shot at a time. the single shot policy would make it illegal to own semiautomatic weapons of any kind. a revolver, rifle, or other types of firearms would need a limiting device that would prohibit the use of semiautomatic firing. next, we took the position of the four types of stakeholders, the anti-gun person, the collector, the law officer, and the protectionist and rated the policy with respect to each of the criteria to develop the overall score for the policy. the total scores are reported in tables 5-8. in table 3, we demonstrate how the intensity scales for each element were prioritized. following table 3, we included sample scoring sheets for a ‘blank’ scoring where we intentionally produced a score of “1” to illustrate the method (table 4). following the ‘blank’, in table 5, we included a scoring of a hypothesized individual who is against gun ownership followed by the perspective of a collector (table 6), a police officer (table 7), and someone who owns a gun for protection (table 8). ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 370 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 table 3 intensity scales gun control policy: benefits[1]: a. security & peace of mind a. protection none low medium high very high b. respect none low medium high very high c. posturing none low medium high very high b. psychological a. self-esteem none low medium high very high b. identity none low medium high very high c. turf none low medium high very high d. community (belonging) none low medium high very high c. recreation a. hunting (satisfaction) absent weak moderate strong very strong b. ornamental no value low value moderate strong very strong c. stress release none low medium high very high d. target shooting not allowed controlled strictly controlled limited allowed d. constitutional right a. freedom no support low support moderate strong very strong b. militia no support low support moderate strong very strong c. bear arms no support low support moderate strong very strong opportunities[2]: a. appreciation (value) none low medium high very high b. identity none low medium high very high c.      ‘insurance’ (safety) none low medium high very high costs[3]: a. monetary costs a. purchase price no impact limited moderate high prohibited b. maintenance low moderate high very high extreme c. license low moderate high very high extreme d. insurance low moderate high very high extreme b. social costs no injuries minor injuries substantial injuries (full recovery) substantial injuries (partial recovery) deaths c. time no effort simple paperwork paperwork with waiting period multiple forms substantial time investment d. holding (storage) no requirement simple requirement (e.g. lockbox) complex requirement (e.g. safe) heavy requirement (e.g. weapon and ammunition separation) extreme (everything under lock and separation e. reporting no effort simple paperwork paperwork with waiting period multiple forms substantial time investment risks[4]: a. constitutionality in line with 2nd amendment some limitations moderate limitations high limitations no weapons for civilians b. identity risks none low medium high very high c. liability none some moderate class restrictions & accessories highly liable fully liable d. non-compliance none low medium high full e. restrictions none limited (e.g. class 3) moderate class restrictions & accessories significant restrictions extreme restrictions f. safety none prevent adult from misuse prevent child from misuse prevent domestic abuse mayhem h. bodily harm to self or others no injuries minor injuries substantial injuries (full recovery) substantial injuries (partial recovery) deaths i. surrender none low medium high very high j. intrusion none low medium high very high [1] here we consider direct, known gains that the lawful gun owner receives. [2] potentialities, things that the gun owner may get at some time in the future. [3] direct cost of lawful gun ownership. [4] the future, potential negative impacts of lawful gun ownership. evaluation scales ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 371 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 table 4 priorities of benefits, opportunities, costs, and risks gun control policy: single shot policy totals bo/cr benefits[1]: weights ratings labels ratings numeric 1.0000 a. security & peace of mind 0.0650 a. protection 0.0555 none 0.0650 b. respect 0.0555 none 0.0650 c. posturing 0.0555 none 0.0650 b. psychological a. self-esteem 0.0308 none 0.0650 b. identity 0.0953 none 0.0650 c. turf 0.0150 none 0.0650 d. community (belonging) 0.0473 none 0.0650 c. recreation a. hunting (satisfaction) 0.0214 absent 0.0650 b. ornamental 0.1068 no value 0.0650 c. stress release 0.1068 none 0.0650 d. target shooting 0.1068 not allowed 0.0650 d. constitutional right a. freedom 0.2169 no support 0.0650 b. militia 0.0434 no support 0.0650 c. bear arms 0.0434 no support 0.0650 opportunities[2]: weights 0.0650 a. appreciation (value) 0.4933 none 0.0650 b. identity 0.2568 none 0.0650 c.      ‘insurance’ (safety) 0.2499 none 0.0650 costs[3]: weights 0.0650 a. monetary costs a. purchase price 0.0489 no impact 0.0650 b. maintenance 0.0098 low 0.0650 c. license 0.0219 low 0.0650 d. insurance 0.1093 low 0.0650 b. social costs 0.5134 no injuries 0.0650 c. time 0.1161 no effort 0.0650 d. holding (storage) 0.0881 no requirement 0.0650 e. reporting 0.0926 no effort 0.0650 risks[4]: weights 0.0650 a. constitutionality 0.3924 in line with the 2nd amendment 0.0650 b. identity risks 0.0207 none 0.0650 c. liability 0.0266 none 0.0650 d. non-compliance 0.1329 none 0.0650 e. restrictions 0.0251 none 0.0650 f. safety 0.1492 none 0.0650 h. bodily harm to self or others 0.0717 no injuries 0.0650 i. surrender 0.1254 none 0.0650 j. intrusion 0.0561 none 0.0650 [3] direct cost of lawful gun ownership. [4] the future, potential negative impacts of lawful gun ownership. benefits (b) opportunities (o) costs (c) risks (r) [1] here we consider direct, known gains that the lawful gun owner receives. [2] potentialities, things that the gun owner may get at some time in the future. ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 372 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 table 5 anti-gun perspective gun control policy: single shot policy totals bo/cr benefits[1]: weights ratings labels ratings numeric 0.9494 a. security & peace of mind 0.2934 a. protection 0.0555 very high 1.0000 b. respect 0.0555 very high 1.0000 c. posturing 0.0555 very high 1.0000 b. psychological a. self-esteem 0.0308 low 0.1236 b. identity 0.0953 medium 0.2515 c. turf 0.0150 high 0.5099 d. community (belonging) 0.0473 high 0.5099 c. recreation a. hunting (satisfaction) 0.0214 absent 0.0650 b. ornamental 0.1068 no value 0.0650 c. stress release 0.1068 low 0.1236 d. target shooting 0.1068 not allowed 0.0650 d. constitutional right a. freedom 0.2169 no support 0.0650 b. militia 0.0434 no support 0.0650 c. bear arms 0.0434 strong 0.5099 opportunities[2]: weights 0.0800 a. appreciation (value) 0.4933 none 0.0650 b. identity 0.2568 low 0.1236 c.      ‘insurance’ (safety) 0.2499 none 0.0650 costs[3]: weights 0.0944 a. monetary costs a. purchase price 0.0489 no impact 0.0650 b. maintenance 0.0098 moderate 0.1236 c. license 0.0219 low 0.0650 d. insurance 0.1093 moderate 0.1236 b. social costs 0.5134 no injuries 0.0650 c. time 0.1161 no effort 0.0650 d. holding (storage) 0.0881 simple requirement (e.g., lockbox) 0.1236 e. reporting 0.0926 paperwork with a waiting period 0.2515 risks[4]: weights 0.2621 a. constitutionality 0.3924 moderate limitations 0.2515 b. identity risks 0.0207 high 0.5099 c. liability 0.0266 some 0.1236 d. non-compliance 0.1329 low 0.1236 e. restrictions 0.0251 extreme restrictions 1.0000 f. safety 0.1492 prevent adult misuse 0.1236 h. bodily harm to self or others 0.0717 substantial injuries (full recovery) 0.2515 i. surrender 0.1254 low 0.1236 j. intrusion 0.0561 very high 1.0000 [2] potentialities, things that the gun owner may get at some time in the future. [3] direct cost of lawful gun ownership. [4] the future, potential negative impacts of lawful gun ownership. benefits (b) opportunities (o) costs (c) risks (r) [1] here we consider direct, known gains that the lawful gun owner receives. ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 373 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 table 6 collector’s perspective gun control policy: single shot policy totals bo/cr benefits[1]: weights benefits (b) 4.5307 a. security & peace of mind 0.4700 a. protection 0.05546 low 0.1236 b. respect 0.05546 medium 0.2515 c. posturing 0.05546 low 0.1236 b. psychological a. self-esteem 0.03075 low 0.1236 b. identity 0.09531 low 0.1236 c. turf 0.01502 low 0.1236 d. community (belonging) 0.04730 high 0.5099 c. recreation a. hunting (satisfaction) 0.02135 absent 0.0650 b. ornamental 0.10675 strong 0.5099 c. stress release 0.10675 low 0.1236 d. target shooting 0.10675 allowed 1.0000 d. constitutional right a. freedom 0.21687 very strong 1.0000 b. militia 0.04337 no support 0.0650 c. bear arms 0.04337 low support 0.1236 opportunities[2]: weights 0.1720 a. appreciation (value) 0.49329 medium 0.2515 b. identity 0.25677 low 0.1236 c.      ‘insurance’ (safety) 0.24994 none 0.0650 costs[3]: weights 0.1099 a. monetary costs a. purchase price 0.04888 limited 0.1236 b. maintenance 0.00978 low 0.0650 c. license 0.02186 low 0.0650 d. insurance 0.10930 low 0.0650 b. social costs 0.51340 minor injuries (full recovery) 0.1236 c. time 0.11613 simple paperwork 0.1236 d. holding (storage) 0.08810 simple requirement (e.g., lockbox) 0.1236 e. reporting 0.09256 no effort 0.0650 risks[4]: weights 0.1624 a. constitutionality 0.39245 in line with the 2nd ammendment 0.0650 b. identity risks 0.02069 high 0.5099 c. liability 0.02658 some 0.1236 d. non-compliance 0.13290 low 0.1236 e. restrictions 0.02507 extreme restrictions 1.0000 f. safety 0.14918 prevent child misuse 0.2515 h. bodily harm to self or others 0.07172 minor injuries (full recovery) 0.1236 i. surrender 0.12536 medium 0.2515 j. intrusion 0.05606 none 0.0650 ratings numeric ratings labels [1] here we consider direct, known gains that the lawful gun owner receives. [2] potentialities, things that the gun owner may get at some time in the future. [3] direct cost of lawful gun ownership. [4] the future, potential negative impacts of lawful gun ownership. opportunities (o) costs (c) risks (r) ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 374 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 table 7 officer’s perspective gun control policy: single shot policy total bo/cr benefits[1]: weights 1.7153 a. security & peace of mind 0.4908 a. protection 0.0555 very high 1.0000 b. respect 0.0555 low 0.1236 c. posturing 0.0555 low 0.1236 b. psychological a. self-esteem 0.0308 low 0.1236 b. identity 0.0953 medium 0.2515 c. turf 0.0150 low 0.1236 d. community (belonging) 0.0473 medium 0.2515 c. recreation a. hunting (satisfaction) 0.0214 weak 0.1236 b. ornamental 0.1068 low value 0.1236 c. stress release 0.1068 low 0.1236 d. target shooting 0.1068 allowed 1.0000 d. constitutional right a. freedom 0.2169 very strong 1.0000 b. militia 0.0434 low support 0.1236 c. bear arms 0.0434 strong 0.5099 opportunities[2]: weights 0.1884 a. appreciation (value) 0.4933 low 0.1236 b. identity 0.2568 medium 0.2515 c.      ‘insurance’ (safety) 0.2499 medium 0.2515 costs[3]: weights 0.2250 a. monetary costs a. purchase price 0.0489 moderate 0.2515 b. maintenance 0.0098 moderate 0.1236 c. license 0.0219 high 0.2515 d. insurance 0.1093 moderate 0.1236 b. social costs 0.5134 substantial injuries (full recovery) 0.2515 c. time 0.1161 paperwork with a waiting period 0.2515 d. holding (storage) 0.0881 simple requirement (e.g., lockbox) 0.1236 e. reporting 0.0926 paperwork with a waiting period 0.2515 risks[4]: weights 0.2396 a. constitutionality 0.3924 in line with the 2nd ammendment 0.0650 b. identity risks 0.0207 low 0.1236 c. liability 0.0266 highly liable 0.5099 d. non-compliance 0.1329 high 0.5099 e. restrictions 0.0251 limited class (e.g., class 3) 0.1236 f. safety 0.1492 prevent child misuse 0.2515 h. bodily harm to self or others 0.0717 substantial injuries (full recovery) 0.2515 i. surrender 0.1254 low 0.1236 j. intrusion 0.0561 very high 1.0000 [2] potentialities, things that the gun owner may get at some time in the future. [3] direct cost of lawful gun ownership. [4] the future, potential negative impacts of lawful gun ownership. ratings labels ratings numeric risks (r) costs (c) opportunities (o) benefits (b) [1] here we consider direct, known gains that the lawful gun owner receives. ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 375 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 table 8 protectionist’s perspective table 9 summarizes the total priorities for benefits, opportunities, costs, and risks for the four types of gun owners illustrated in tables 5-8. figure 7 shows that information in a radar display that illustrates the similarities and dissimilarities of the four groups. gun control policy: single shot policy total bo/cr benefits[1]: weights 0.6321 a. security & peace of mind 0.1698 a. protection 0.0555 none 0.0650 b. respect 0.0555 none 0.0650 c. posturing 0.0555 none 0.0650 b. psychological a. self-esteem 0.0308 low 0.1236 b. identity 0.0953 low 0.1236 c. turf 0.0150 none 0.0650 d. community (belonging) 0.0473 low 0.1236 c. recreation a. hunting (satisfaction) 0.0214 very strong 1.0000 b. ornamental 0.1068 no value 0.0650 c. stress release 0.1068 low 0.1236 d. target shooting 0.1068 strictly controlled 0.2515 d. constitutional right a. freedom 0.2169 moderate 0.2515 b. militia 0.0434 no support 0.0650 c. bear arms 0.0434 moderate 0.2515 opportunities[2]: weights 0.1236 a. appreciation (value) 0.4933 low 0.1236 b. identity 0.2568 low 0.1236 c.      ‘insurance’ (safety) 0.2499 low 0.1236 costs[3]: weights 0.1069 a. monetary costs a. purchase price 0.0489 no impact 0.0650 b. maintenance 0.0098 low 0.0650 c. license 0.0219 moderate 0.1236 d. insurance 0.1093 low 0.0650 b. social costs 0.5134 minor injuries (full recovery) 0.1236 c. time 0.1161 no effort 0.0650 d. holding (storage) 0.0881 simple requirement (e.g., lockbox) 0.1236 e. reporting 0.0926 simple paperwork 0.1236 risks[4]: weights 0.3105 a. constitutionality 0.3924 moderate limitations 0.2515 b. identity risks 0.0207 high 0.5099 c. liability 0.0266 some 0.1236 d. non-compliance 0.1329 low 0.1236 e. restrictions 0.0251 extreme restrictions 1.0000 f. safety 0.1492 prevent adult misuse 0.1236 h. bodily harm to self or others 0.0717 substantial injuries (full recovery) 0.2515 i. surrender 0.1254 high 0.5099 j. intrusion 0.0561 very high 1.0000 risks (r) costs (c) opportunities (o) benefits (b) ratings numeric ratings labels [1] here we consider direct, known gains that the lawful gun owner receives. [2] potentialities, things that the gun owner may get at some time in the future. [3] direct cost of lawful gun ownership. [4] the future, potential negative impacts of lawful gun ownership. ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 376 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 table 9 benefits, opportunities, costs, and risks for four groups of gun owners anti-gun collector officer protectionist ideal costly benefits 0.2934 0.4700 0.4908 0.1698 1.0000 0.0650 opportunities 0.0800 0.1720 0.1884 0.1236 1.0000 0.0650 costs 0.0944 0.1099 0.2250 0.1069 0.0650 1.0000 risks 0.2621 0.1624 0.2396 0.3105 0.0650 1.0000 bo/cr 0.9494 4.5307 1.7153 0.6321 236.7 0.0040 to bracket the scores, we developed both the ideal state which includes all of the benefits and opportunities and none of the costs or risks, as well as a costly state where there are no benefits or opportunities, and only costs and risks. in the ideal state, the highest score that a given gun policy could achieve is a 236.7, whereas the lowest score that a policy can receive is a 0.004. this provides a bracket for evaluating the impact that gun control policy can achieve. the distance from the ideal or the costly provides some perspective on the overall gain or loss of a proposed policy. figure 7 graphical depiction of proposed fictitious gun control policy ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 377 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 with respect to the consideration of lawful gun owners, the goal is to maximize the volume in the area in the radar chart between the benefits and the opportunities. note that in figure 7, the full area between benefits and opportunities is taken up in the ideal state, whereas the full area between costs and risk is taken up in the costly state. in figure 8, we remove the brackets and focus solely on the four hypothetical types of lawful gun owners in order to more clearly visualize their cases. in figure 8, we see that the officer’s radar chart (grey) has more area in the benefits and opportunities than the collector (orange), but that the officer also has more volume in the costs and risks offsetting the additional area in the benefits; the two have similar area in the opportunities. the result for the two is that the single-shot policy is better from the collector’s perspective than from the officer’s perspective. figure 8 four perspectives without ideal and costly the bocr approach illuminates many ideas. according to benefits and opportunities, the collector and the officer seem in agreement; according to costs the anti-gun activist, the collector and the protectionist seem to agree; and according to risks the protectionist, the anti-gun activist and the officer are closer to each other. these conclusions can be seen in numerical format in table 9. overall, the protectionist is most impacted, followed by the anti-gun perspective. the one that benefits most is the collector, but the officer sees more benefits and opportunities than costs and risks. ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 378 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 the implementation of this framework is predicated in two steps: 1. developing priorities for the strategic criteria for the group profiles, and 2. evaluating existing or proposed gun control policies by different constituent groups. the information required to accomplish the first step is predicated on the study of preferences of large groups. the next section describes a methodology that could help attain this objective. another possibility to accomplish this goal is to use focus groups. 5. how to develop priorities for large groups developing priorities for large groups is not the same as achieving consensus for a group in a decision-making situation. in the latter, using the ahp we would seek judgments from decision makers and the geometric mean of those judgments could be used as the representative of the group. of course, this would require using some statistical analysis to see if the geometric mean really represents the group. in the former, the situation is like voting, but with intensity of preferences. the issue here is how to combine the different judgments. judgments are from an absolute scale if saaty’s 1-9 scale is used. (vargas, 2016) showed that each vote is equivalent to a pairwise comparison in which the intensity of preference is very large. consider two candidates 1a and 2a . comparing them according to a criterion, we can express how strongly we prefer one candidate to the other. for example, if 1a is preferred to 2a with intensity a the result is a reciprocal matrix of pairwise comparisons given by 1 2 1 1 2 1 1 a a a a a a          the vector of priorities is given by the principal right eigenvector given by 1 1 1 a a a              that converges to the vector (1, 0) t as a   . thus, voting for a candidate is equivalent to pairwise comparing them with an intensity of infinity. when many users are asked for their opinion, the answers (rankings) they provide is called a profile in voting theory. for example, if we ask 100 people how they prefer ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 379 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 candidates 1a and 2a , if 75 1 2 75n   prefer 1a to 2a ( 1 2 75n   ), then 25 should prefer 2 a to 1a ( 2 1 25n   ). we assume that there no ties. then, the profile is written as follows: 1 2 2 1 1 2 1 2 2 1 2 1 ( ) ( ) (75) (25)n n a a a a a a a a                         , and the pairwise voting matrix is given by a a a a n na a a an n                      1 2 1 2 1 2 2 11 1 2 22 1 1 2 751 1 25 25 11 75 . if all the voters that prefer 1a to 2a have an intensity of preference 12a , then we can represent the pairwise voting matrix as follows: 12 12 21 21 21 21 12 12 75 1 1 25 1 25 1 1 75 1 a a a a a a a a                                       so that when 12a   we obtain the pairwise voting matrix given above. the principal right eigenvector of this matrix is given by ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 380 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 12 12 12 21 12 21 21 21 12 21 12 21 75 1 75 25 1 1 25 1 75 25 1 1 a a a a a a a a a a a a                                                     and when 12a   , the voting priorities are given by 75 100 25 100         . if all the voters have different intensity of preference ( )k i j a  , then the pairwise voting matrix is given by 1 2 2 1 2 1 1 2 ( ) ( ) 12 ( ) 1 12 ( ) ( ) 21 ( ) 1 21 ( ) ( ) 21 ( ) 1 21 ( ) ( ) 12 ( ) 1 12 1 1 1 1 1 1 n k k k n k k k n k k k n k k k a a a a a a a a                                                        . finally, if we compare m candidates, the pairwise voting matrix with intensity of preferences would be given by ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 381 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 11 2 1 2 1 2 2 ( )( ) ( )( ) 112 ( )( ) 11 112 ( ) ( ) ( )( ) 121 ( ) ( ) 1 121 1 ( ) ( ) 2 ( ) 1 2 ( ) ( ) 2 ( ) 1 2 11 1 1 1 1 1 1 1 m m m m nn kk m kk kk m n n kk m k k k k m n k m k k m n k m k k m aa aa aa a a a a a a                                                        (3) where ( ) i j n   and ( )k ij a represent the number of voters that prefer i to j and the intensity with which the k th voter prefers i to j, respectively. note that if ( ) , for all and k ij a i j  , the pairwise voting matrix converges to the matrix 12 1 21 2 1 2 1 ( ) ( ) ( ) 1 ( ) ( ) ( ) 1 m m m m w w w w w w                     where ( ) ( ) , ( ) 0 ( ) i j ij j i j i n w n n          , and the voting priorities of the m candidates are given by the principal right eigenvector of the matrix. consider the following profile in which four alternatives are compared by 14 individuals: 1 2 3 4 2 3 4 1 3 4 1 2 4 1 2 3 (4) (3) (5) (2) a a a a a a a a b a a a a a a a a             . if we were to use the counting method, the voting priorities are given by the principal right eigenvector of the matrix: ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 382 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 let us now consider that each decision maker has her own intensity of preference according to table 10. combining the intensity of preferences as in equation 3, we obtain the following matrix: which yields priorities close to the priorities produced by the ranking method. on the other hand, the matrix of geometric means given by yield priorities that are not as close to the priorities obtained by the ranking method. thus, we believe that a way to capture priorities of large groups is by using the ranking method within the context of the eigenvector method of pairwise comparisons. counting a1 a2 a3 a4 priorities a1 1 11/3 6/8 4/10 0.2436 a2 3/11 1 9/5 7/7 0.2027 a3 8/6 5/9 1 12/2 0.3589 a4 10/4 7/7 2/12 1 0.1948 ranking a1 a2 a3 a4 priorities a1 1 3.400928 0.789474 0.557616 0.2634 a2 0.294037 1 1.734146 1.026261 0.2095 a3 1.266667 0.576653 1 5.463415 0.3477 a4 1.793349 0.974411 0.183036 1 0.1794 gm a1 a2 a3 priorities a1 1 2.013483 0.923194 0.6671 0.2624 a2 0.496652 1 1.237874 1.076316 0.2220 a3 1.083195 0.807837 1 2.336644 0.2958 a4 1.499026 0.929095 0.427964 1 0.2198 ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 383 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 table 10 profile b intensity of preferences a1>a2>a3>a4 a2>a3>a4>a1 a3>a4>a1>a2 a4>a1>a2>a3 a1 a2 a3 a4 priorities | a1 a2 a3 a4 priorities | a1 a2 a3 a4 priorities | a1 a2 a3 a4 priorities a1 1 3 5 9 0.5806 | a1 1 0.142857 0.2 0.333333 0.0521 | a1 1 5 0.2 0.333333 0.1364 | a1 1 3 5 0.111111 0.1733 a2 0.333333 1 3 5 0.2554 | a2 7 1 5 5 0.6194 | a2 0.2 1 0.142857 0.2 0.0489 | a2 0.333333 1 3 0.2 0.0967 a3 0.2 0.333333 1 3 0.1141 | a3 5 0.2 1 3 0.2195 | a3 5 7 1 3 0.5558 | a3 0.2 0.333333 1 0.142857 0.0473 a4 0.111111 0.2 0.333333 1 0.0499 | a4 3 0.2 0.333333 1 0.1090 | a4 3 5 0.333333 1 0.2589 | a4 9 5 7 1 0.6828 | | | a1 a2 a3 a4 priorities | a1 a2 a3 a4 priorities | a1 a2 a3 a4 priorities | a1 a2 a3 a4 priorities a1 1 5 7 9 0.6485 | a1 1 0.2 0.333333 0.333333 0.0720 | a1 1 4 0.333333 0.2 0.1272 | a1 1 5 7 0.2 0.2844 a2 0.2 1 3 5 0.2009 | a2 5 1 3 3 0.4925 | a2 0.25 1 0.142857 0.333333 0.0547 | a2 0.2 1 5 0.333333 0.1259 a3 0.142857 0.333333 1 5 0.1100 | a3 3 0.333333 1 5 0.3051 | a3 3 7 1 5 0.5646 | a3 0.142857 0.2 1 0.2 0.0466 a4 0.111111 0.2 0.2 1 0.0407 | a4 3 0.333333 0.2 1 0.1304 | a4 5 3 0.2 1 0.2536 | a4 5 3 5 1 0.5431 | | | a1 a2 a3 a4 priorities | a1 a2 a3 a4 priorities | a1 a2 a3 a4 priorities | a1 1 2 3 5 0.4650 | a1 1 0.166667 0.2 0.333333 0.0503 | a1 1 7 0.5 0.333333 0.2082 | a2 0.5 1 3 5 0.3273 | a2 6 1 7 7 0.6584 | a2 0.142857 1 0.2 0.2 0.0536 | a3 0.333333 0.333333 1 2 0.1342 | a3 5 0.142857 1 5 0.2106 | a3 2 5 1 2 0.4070 | a4 0.2 0.2 0.5 1 0.0736 | a4 3 0.142857 0.2 1 0.0807 | a4 3 5 0.5 1 0.3312 | | | a1 a2 a3 a4 priorities | a1 a2 a3 a4 priorities | a1 1 4 5 7 0.5758 | a1 1 3 0.2 0.2 0.0978 | a2 0.25 1 4 6 0.2613 | a2 0.333333 1 0.2 0.2 0.0576 | a3 0.2 0.25 1 5 0.1187 | a3 5 5 1 5 0.5885 | a4 0.142857 0.166667 0.2 1 0.0442 | a4 5 5 0.2 1 0.2560 | | | a1 a2 a3 a4 priorities | a1 1 5 0.2 0.333333 0.1364 | a2 0.2 1 0.142857 0.2 0.0489 | a3 5 7 1 3 0.5558 | a4 3 5 0.333333 1 0.2589 | ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 384 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 6. implementation of the proposed approach thus far, we have illustrated an approach that may be used to develop a measure of the impact of existing and proposed gun policies for lawful gun owners. assessing the impact of gun policies on lawful gun owners is an important activity since the implementation of any policy may impose unintended hardships on them. while the legislation of any policy is ultimately carried out with the best intentions, without considering the impact on the gun owner the full impact of the policy is unknown. in this effort, the authors assumed the roles of the various stakeholders for sake of illustration. to fully develop the models requires much more involvement and greater input. since gun policies are developed at every level of our government, and gun owners are varied, the final assessment needs to reflect the various levels and types of gun owners. to this end, we propose additional work that needs to be conducted as follows: 1. identification of national-level focus groups with subject matter experts, 2. identification of state-level focus groups with subject matter experts, 3. hold discussions with gun-owner focus groups, 4. develop criteria, 5. develop a distributed platform for priority generation, and 6. estimate policy and law ratings. national and state-level focus groups will provide greater insights into what the criteria for evaluation should be. the results will be a hierarchical structure that more accurately reflects the criteria and priorities at each level. ideally, each state should have a focus group study done such that the criteria that are important to that state and the resultant priorities accurately reflect the state’s interests. the results from the focus groups can be put into a web-based platform for distributed release. the platform will allow users to input their own ratings on existing and proposed legislation to see their resulting scores. the collection of the data along with demographics will facilitate a continuous evaluation of the bo/cr of gun policy in the u.s. that will be updated as priorities and concerns shift and change. ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 385 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 7. how to use the results of the study disagreement between different gun advocate groups is one of the most important issues confronting american society. according to (pierre, 2019), “the gun debate in america is often framed as a stand-off between two immutable positions with little potential to move ahead with meaningful legislative reform. attempts to resolve this impasse have been thwarted by thinking about gun ownership attitudes as based on rational choice economics instead of considering the broader socio-cultural meanings of guns.” in this paper, we present a mechanism to begin a dialogue between different stakeholders in the hope that by identifying the differences in thinking a compromise can be reached. by measuring the impact of a gun policy (law) on different groups one could perhaps modify, in incremental steps, the policy being evaluated, so that the groups can be closer to each other in most of the issues they consider important. it is possible that by using this methodology some groups may discover some misinterpretations or misunderstandings they have about the policy that can help them feel more at ease with it or know how they could lobby for or against it. the focus groups must consist of knowledgeable people from the different constituencies. experience suggests the recommended size of each group should be between five and ten people (cummings et al., 1974). we would need a session to develop priorities for the strategic criteria, the benefits, the opportunities, the costs, and the risks, and a second session to evaluate different gun control policies. it is possible that this evaluation could be done through an electronic questionnaire, in which case it may not be easy to determine who is answering the evaluation. however, given the importance of this step it is preferable to use focus groups. this paper facilitates the development of a proof of concept to show that our methodology captures what stakeholders think about different gun laws and their impact on lawful gun owners. this will enable policy makers to develop policies that are beneficial to society at large. ultimately, the validity of our proof of concept will result in the development of a distributed platform. this platform will enable any stakeholder to rate a proposed/existing policy, and thereafter, policy makers can assess its impact and legislate accordingly. ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 386 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 references cook, d. r., staschak, s., & green, w.t. (1990). equitable allocation of livers for orthotopic transplantation: an application of the analytic hierarchy process. european journal of operational research, 48(1), 49-56. doi: https://doi.org/10.1016/03772217(90)90060-o cummings, l., huber, l., george, p., & arendt, e. (1974). effect of size and spatial arrangements on group decision making. academy of management journal, 17(3), 460475. doi: https://doi.org/10.2307/254650 forji, a. g. (2010). the correlation between law and behaviour as pillars of human society. international journal of punishment and sentencing, 6(3), 84-96. kahneman, d., & tversky, a. (1979). prospect theory: an analysis of decision under risk. econometrica, 47(2), 263-291. doi: https://doi.org/10.2307/1914185 moore, m.h. (1983). the bird in hand: a feasible strategy for gun control. journal of policy analysis and management, 2(2), 185-196. pierre, j. m. (2019) the psychology of guns: risk, fear, and motivated reasoning. palgrave communications, 5, 159. doi: https://doi.org/10.1057/s41599-019-0373-z. saaty, t. l. (1977). a scaling method for priorities in hierarchical structures. journal of mathematical psychology, 15(3), 234-281. doi: https://doi.org/10.1016/00222496(77)90033-5 saaty, t. l. (1980). the analytic hierarchy process, mcgraw-hill publications. saaty, t. l. (1996). the analytic network process. pittsburgh, pa: rws publications. vargas, l. g. (2016). voting with intensity of preferences. international journal of information technology & decision making, 15(4), 839-859. doi: https://doi.org/10.1142/s0219622016400058 https://doi.org/10.1057/s41599-019-0373-z ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 387 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 appendix a: relative measurement relative measurement is the development of a scale to measure an intangible attribute, i.e., an attribute for which no scale to measure exists. thus, each individual has its own scale in mind. however, if there were a scale to measure the attribute, relative measurement would provide an answer in relative terms. for example, consider that we want to measure the weight of three cantaloupes, c1, c2 and c3, but we do not have a weight scale with us. so, we get two cantaloupes, c1 and c2, one in each hand and we try to feel which one is heavier. we do that with each pair of cantaloupes, (c1, c2), (c1, c3) and (c2, c3). we cannot compare all of them at once. each pairwise comparison gives us an estimate of relative heaviness, but how do we decide which one is heaviest? if we decide that between c1 and c2, c2 is heaviest, then we can compare c2 and c3 and we are done. suppose that c2 is the heaviest of the three cantaloupes, so we buy c2. however, our result did not give us an estimate of how much heavier c2 is than the other two cantaloupes. if the store were to give us the actual weights of the cantaloupes, w1, w2 and w3, we could estimate, when we compared c1 and c2 balancing them on our hands, which one is heavier with the ratio w1/w2 or w2/w1. it is easiest to estimate how much heavier the heavier one is, as a function of the lighter one, than vice versa. this is equivalent to what is known in psychometrics as fractionation and multiplication. if we arrange these comparisons in a matrix form, we have: 1 2 3 1 1 1 1 1 2 3 2 2 2 2 1 2 3 3 3 3 3 1 2 3 c c c w w w c w w w w w w c w w w w w w c w w w                   (a.1) note that in equation a.1 we took the ratio of the weights. the result is a number without units. it is an absolute number. on the other hand, had we taken the differences, e.g., w1 w2, if the weight is given in pounds, the difference will be in pounds. thus, taking the ratio to estimate the weight does not need a scale unit. all we need to do is estimate in absolute terms how much heavier one is than the other one. the next step is to obtain from the ratio comparisons i j w w       the relative weights of the objects. relative weights would be 1 2 3 1 2 3 1 2 3 1 2 3 , , w w w w w w w w w w w w            . it turns out that if we multiply the first column by w1, the second by w2, and the third one by w3, and sum across the rows we get: ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 388 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 1 2 3 1 1 1 1 11 2 3 1 2 3 2 2 2 2 1 2 3 2 1 2 3 3 3 3 1 2 33 3 1 2 3 3 3 3 c c c w w w c ww w w w w w w w w c w w w w w w w w w w w w wc w w w w                                    . thus, we recover the weights by dividing each value by the sum of all the values, and this would yield the relative weights of the objects: 1 2 3 1 2 3 1 2 3 1 2 3 , , w w w w w w w w w w w w            in real life, we may not know the weights, but we may be able to guess them by assigning a positive number. for example, we could have the following matrix: 1 2 3 1 2 3 1 1 / 1 1 / 1 / 1 c c c c a b c a c c b c          . (a.2) in matrix notation, entry, e.g., (1,2), represents the row number = 1, and the column number = 2. note that the entries in the positions (1,1), (2,2) and (3,3) are now 1, the result of dividing a positive number by itself. the entries in the transposed positions, i.e., the transposed entry of (1,2) is the entry in the (2,1) position, are the reciprocal of the original entries, e.g., if entry (1,2) is a then the entry (2,1) is 1/a. now the entries of the matrix of pairwise comparisons are educated guesses of the underlying ratios of the weights we seek. to estimate the relative weights, (saaty 1980) proposed a first order approximation by first normalizing to unity each of the columns, and then averaging the resulting rows. for the matrix given in (a.2) we would have: ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 389 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 / (1 ) / ( 1 ) / ( 1) 1 / 1 (1 / ) / (1 ) 1 / ( 1 ) / ( 1) 1 / 1 / 1 (1 / ) / (1 ) (1 / ) / ( 1 ) 1 / ( 1) {[1 / (1 )] [ / ( 1 )] [ / ( 1)]} / 3 {[(1 / ) / (1 ) [ a b c a b c a b c a b c a b a b a a b b c a c a a c b c b c b c a b c a a b b c a                                              1 1 1 1 1 / ( 1 )] [ / ( 1)]} / 3 {[(1 / ) / (1 )] [(1 / ) / ( 1 )] [1 / ( 1)]} / 3 c a b c a c b c b c a b c                      . the mathematical model that provides the solution to our estimation problem is known as a principal eigenvalue problem. we refer to the reader to (saaty 1977) for details of this model. the importance of this model lies with the fact that now we can estimate relative measures for intangible attributes. (saaty 1977) used the absolute scale given in table 1 to estimate the pairwise comparisons. remember that the scale in table 1 is an estimate. if actual measurement were available, we could use those to compute the relative weights. table 1 fundamental scale intensity of importance definition explanation 1 equal importance two activities contribute equally to the objective 3 moderate importance experience and judgment slightly favor one activity over another 5 strong importance experience and judgment strongly favor one activity over another 7 very strong or demonstrated importance an activity is favored very strongly over another; its dominance demonstrated in practice 9 extreme importance the evidence favoring one activity over another is of the highest possible order of affirmation 2, 4, 6, 8 reciprocals of above intermediate values between the two adjacent judgments if activity i has one of the above nonzero numbers assigned to it when compared with activity j, then j has the reciprocal value when compared with i when compromise is needed rationals ratios arising from the scale if consistency were to be forced by obtaining n numerical values to span the matrix ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 390 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 the first experiment performed to try to validate perception with the fundamental scale of table 1 was the optics experiment (saaty 1977). four objects were placed at the following distances measured in yards from a light source: 9, 15, 21, 28. in normalized form these distances are 0.123, 0.205, 0.288, 0.384. next, two independent sets of judges were asked to estimate pairwise comparisons of the brightness of the objects, labeled in increasing order according to their nearness to the source where the judges were located. the result was two pairwise comparison matrices: 1 2 3 4 1 2 3 4 1 1 2 2 3 3 4 4 1st trial 2nd trial 1 5 6 7 1 4 6 7 1 / 5 1 4 6 1 / 4 1 3 4 1 / 6 1 / 4 1 4 1 / 4 1 / 3 1 2 1 / 7 1 / 6 1 / 4 1 1 / 7 1 / 4 1 / 2 1 c c c c c c c c c c c c c c c c                         the relative brightness from the trials was: relative brightness relative brightness (1st trial) (2nd trial) 0.61 0.62 0.24 0.22 0.10 0.10 0.05 0.06 there is a law in optics known as the inverse square law that states that the brightness of an object located at a distance from a source of light is inversely related to the square of the distance of the object to the source of light. table 2 gives the inverse square law of the four objects considered. table 2 inverse square law of optics note that after applying the inverse square law of optics to the distances, the relative values (6 th column of table 2) are very close to the estimates of relative brightness obtained in the two trials. there are many experiments like this that support the idea that normalized distance 1/distance2 distance2 1st trial 2nd trial 9 0.01234568 0.607 0.61 0.62 15 0.00444444 0.218 0.24 0.22 21 0.00226757 0.112 0.10 0.10 28 0.00127551 0.063 0.05 0.06 ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 391 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 judgments could be used to estimate in relative terms the importance of intangibles in decision making. in measurement theory, when a set of objects, or alternatives in general, are compared, they must always be compared with respect to one criterion and only one criterion at a time. no comparison can take place without a criterion in mind. when multiple criteria are involved, we must compare the alternatives with respect to each of the criteria. what makes relative measurement so interesting is that we can now combine the relative scales because they are all measured in the same units. since the criteria may not be equally important one could think about using a weighted average of the different relative scales. there are different opinions as to how to combine the different scales obtained from each criterion, but if one is careful when defining the criteria so that they do not overlap in meaning, the weighted average is usually a good estimate. this is known as hierarchical synthesis because instead of criteria and alternatives, one could think of criteria, subcriteria, sub-criteria of sub-criteria and so on, all the way down the hierarchy to the alternatives that usually occupy the bottom level of the hierarchy. the top level of the hierarchy is the goal. a logical extension of the analytic hierarchy process is the analytic network process (saaty 1996). instead of considering hierarchies as in figure a.1, we consider a network of relations between the criteria, the sub-criteria, the alternatives, and so on as in figure a.2. an important characteristic of networks is that they allow for feedback between clusters. figure a.1. a hierarchy figure a.2. a network for example, when comparing a diagnostic procedure for a disease, alternative procedures need to be prioritized with respect to, for example, accuracy, simplicity, logistics, invasiveness and so on. but, to select a procedure we need to assign priorities to the criteria. however, each procedure could perform better under different criteria. there may be statistics that tell us, for a given procedure, under which criterion it performs better. this would be equivalent to comparing the criteria or attributes of the procedures goal criteria sub-criteria alternatives criteria sub-criteria alternatives ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 392 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 under each alternative. this is a feedback loop from the cluster of alternatives to the cluster of criteria. ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 393 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 appendix b: definitions cluster – a grouping of nodes that share a similar class within the ahp / anp model. collectors – this category of gun owner tends to possess them for the potential appreciation in value. they may display their guns and could have many in their possession. enthusiasts – these are generally people who feel an affinity with guns. gun shop owners dealers historical reenactment – this category may have weapons for recreating battles such as gettysburg, may participate in demonstrations such as veteran’s day parades, or take part in filming. hunters – there are a variety of types of hunters. for instance, a bird hunter might prefer a shotgun, whereas a deer hunter might prefer cartridge. there are some hunters that prefer black-powder, whereas some others may prefer high-power. some hunters hunt for recreation, whereas others hunt to supplement their food. we do not separate the various types of hunters, but recognize there is a variety. law enforcement – these individuals will often be issued a service weapon, but may also have personal weapons that they may decide to carry while working and may also carry while off-duty. node – an individual unit or member of a cluster that shares some characteristic of the other members within the cluster of the ahp / anp model. paramilitary – these individuals may possess weapons for the purpose of serving in a militaristic type of organization which is not government backed or supported. private citizens – homeowners, personal protection, automobile. private security guards – these individuals are typically contracted out through a firm to cover banks, buildings, and other locations. they may be required to own their own gun though their employer may provide one. retired and active duty military – military members may have basic weapons training but may also have advanced weapons training. these individuals may have a variety of types of weapons. shooting – paper fixed targets, skeet, and pop-up. survivalists – these individuals may purchase weapons with the idea of needing them should social structures and services fall into disarray. ijahp news and events: guiora, minutolo, vargas/report on an analytic network process (anp) model to estimate the benefits, opportunities, costs and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms international journal of the analytic hierarchy process 394 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.909 appendix c: acronyms ahp – analytic hierarchy process anp – analytic network process ijahp article: ganguly, kumar /evaluating supply chain resiliency strategies in the indian pharmaceutical sector: a fuzzy analytical hierarchy process (f-ahp) approach international journal of the analytic hierarchy process 153 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.620 evaluating supply chain resiliency strategies in the indian pharmaceutical sector: a fuzzy analytic hierarchy process (f-ahp) approach anirban ganguly professor and assistant dean, director, center for asia pacific business research, economics, & innovation, jindal global business school o. p. jindal global university, haryana, india. aganguly@jgu.edu.in chitresh kumar assistant professor jindal global business school o. p. jindal global university, haryana, india. ckumar@jgu.edu.in abstract over the years, aggressive competition and globalization have resulted in tremendous progress in the development and management of supply chains. there are many highperforming supply chains that have benefitted from globalization. however, globalization of supply chains has also resulted in exposure to increased risks and frequent disturbances at various stages. one way to address these disturbances is to make supply chains resilient in nature. this paper identifies and prioritizes a set of important resiliency strategies for supply chains using the fuzzy analytic hierarchy process (f-ahp). responses from 23 experts from the indian pharmaceutical sector have been analyzed. we found that supply chain agility, visibility, and collaboration were the three most important resiliency strategies that a pharmaceutical organization should follow to address the vulnerability within their supply chain. the findings will serve the supply chain managers and the policymakers in designing resilient supply chains through the better understanding of strategies and formulation of action plans, while also introducing mcdm techniques into resiliency studies. keywords: multi-criteria decision-making (mcdm); supply chain resilience; resiliency strategies; fuzzy analytic hierarchy process (f-ahp); analytic hierarchy process (ahp); indian pharmaceutical sector 1. introduction the rapid shrinkage in the product and technology lifecycle, coupled with ever-changing (and unforeseen) customer preferences has forced organizations to change their operational processes. organizations are no longer considered isolated islands, but rather mailto:aganguly@jgu.edu.in mailto:ckumar@jgu.edu.in ijahp article: ganguly, kumar /evaluating supply chain resiliency strategies in the indian pharmaceutical sector: a fuzzy analytical hierarchy process (f-ahp) approach international journal of the analytic hierarchy process 154 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.620 as inter-connected networks functioning in harmony to achieve common objectives. one way to achieve this is to design and maintain an efficient supply chain that would serve as an important means for achieving a competitive advantage. however, in addition to the shortening life-cycle of products and technology, continuous and uncertain changes in customer demands and preferences have also resulted in extremely complex supply chains. this has led to more vulnerable supply chains. it has been observed that supply chains have been on the receiving end of a number of natural and manmade emergencies, like floods, earthquakes, fire, terrorist attacks and so on. as a weapon to combat these disruptions, organizations have started designing their supply chains in a manner that can respond to these disruptions, and also actually recover to their original state, or even better, which is known as supply chain resiliency (christopher & peck, 2004; ponomarov & holcomb, 2009). resiliency ensures an uninterrupted supply of goods and services to an organization as well as their customers, even after suffering an unforeseen disruption, which can lead to revenue generation and a competitive advantage. in addition, it leads to reduced customer perception of assumed risks fostering risk resilient growth (ambulkar et al., 2015; ganguly et al., 2018a). the concept of resiliency stretches beyond just recovery and into the domain of flexibility, adaptability, maintenance, and recovery, becoming an important and sustainable component of success among organizations (ponomarov & holcomb, 2009; hamel & välikangas, 2003; stoltz, 2004). supply chain resiliency is defined as an organization’s ability to react to negative effects caused by disturbances that occur across supply chains (barroso et al., 2010). shuai et al. (2011) compared resiliency to cell biology and stated that it is just like cell elasticity i.e. the ability of a cell to come back to its original structure after being subjected to certain stress. a resilient supply chain exposed to disturbances can mutate to adapt to the internal and external environment. furthermore, resiliency has become a critical component in present-day supply chains in order to combat external and internal vulnerabilities, along with being restored to a stage of equilibrium, and in the process ensuring higher efficiency and performance levels (longo & oren, 2008). therefore, a resilient supply chain has the ability to thrive and survive in the face of unexpected and turbulent changes. the importance of supply chain resiliency has astronomically elevated in the last two decades, and organizations are trying to design various supply chain resiliency strategies to stay ahead of their competitors. the resiliency strategies have been broadly categorized into two types – proactive and reactive strategies – depending on the nature of the supply chain and the organization in general. tukamuhabwa et al. (2015) stated that most of the supply chain strategies highlighted in the literature are proactive rather than reactive in nature, and therefore are expected to continue ‘operations as normal.’ they further argued that although proactive strategies are desirable to minimize vulnerability in supply chains, managers might be reluctant to justify investments in strategies to mitigate potential disruptive events that may not eventually occur. as a result, a careful trade-off between the selection of proactive vis-à-vis reactive supply chain resiliency strategies should be considered as part of the supply chain resiliency strategy selection for an organization. as observed from the extant literature, substantial academic and practitioner-based research is available on identification and discussion of supply chain resiliency strategies. the strategies discussed are from a proactive and a reactive perspective, and from an organizational and an individual level. however, during the literature review we observed ijahp article: ganguly, kumar /evaluating supply chain resiliency strategies in the indian pharmaceutical sector: a fuzzy analytical hierarchy process (f-ahp) approach international journal of the analytic hierarchy process 155 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.620 that though there were multiple research papers discussing strategies for supply chain resilience, limited research work has identified and discussed the relative importance of a set of critical strategies that might aid an organization in improving the efficiency of its supply chain, especially in the context of the indian pharmaceutical sector. the current research attempts to shed some light on this issue and tries to bridge this gap through initially identifying a set of critical strategies (separating the vital few from the trivial many) associated with supply chain resiliency, and subsequently prioritizes through assessment of their relative criticality. the pharmaceutical supply chain is considered a “systemic network of a set of complex processes, operations and organizational structure which is involved in the discovery, development, and manufacturing of medications” (ganguly et al., 2018a). the indian pharmaceutical sector is one of the largest producers of generic pharmaceutical drugs in the world and a major contributor to the gross domestic product (gdp) in the country. since the operational process of indian pharmaceutical companies is comprised of sales and distribution of generic as well as branded drugs, efficient procurement and inventory management, logistics and distribution have become important ingredients for the pharmaceutical companies for operational efficiency. additionally, the growing importance of reverse logistics, coupled with recycling, disposal, scheduling, and planning has elevated the importance of the supply chain among the pharmaceutical companies, thereby making it important for the pharmaceutical companies to design and develop efficient supply chains which are resilient in nature (narayana et al., 2014). the multi-criteria decision-making (mcdm) model of fuzzy analytic hierarchy process (f-ahp) is used to assess the relative criticality of the identified strategies, thereby adding to the novelty of the current investigation. f-ahp, which stemmed from the basic ahp model, allows the experts to use fuzzy ratios in place of exact ratios (saaty, 1980, 1990, 1994; buckley, 1985; buckley et al., 2001). the basic advantage of f-ahp over its traditional counterpart (ahp) is that it allows the respondents to provide vague or imprecise replies when comparing two alternatives in a hierarchical analysis (buckley, 1985). the ahp’s subjective judgment, selection and preference of decision-makers often cannot reflect the human thinking style. additionally, the fact that the ahp method is mainly used in nearly crisp (non-fuzzy) decision applications might lead to an unbalanced scale of judgment. therefore, avoiding these risks on performance, fuzzy ahp, a fuzzy extension of ahp, was developed to solve hierarchical fuzzy problems. (özdağoğlu & özdağoğlu, 2007). buckley (1990; 1992; 1995) used the application of fuzzy ahp in decision making with great success and buckley et al. (2001) enlisted a three-step process for calculating fuzzy ahp which includes finding the crisp solution, fuzzifying the crisp solution, and using the fuzzified crisp solution to determine the solution to the fuzzy problem. considering the risks associated with a conventional ahp analysis, the authors decided to use fuzzy ahp as the preferred technique in the current study. beginning with a review of concepts of supply chain resiliency and f-ahp, this paper subsequently goes on to identify a set of strategies that are expected to improve supply chain resiliency of pharmaceutical organizations. the identified strategies are then prioritized using f-ahp to assess their relative criticality. the findings of this study are expected to serve as an important roadmap towards understanding the relative importance ijahp article: ganguly, kumar /evaluating supply chain resiliency strategies in the indian pharmaceutical sector: a fuzzy analytical hierarchy process (f-ahp) approach international journal of the analytic hierarchy process 156 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.620 of the strategies that can be subsequently adopted as part of the organization's operational planning process. 2. review of the extant literature 2.1 resilient supply chains the concept of resilience, which stems from the latin word resilīre meaning ‘to spring back or rebound,’ is the ability of a substance or object to spring back into shape (ganguly et al., 2018a). resilience is concerned with the ability of systems to absorb changes and return to an equilibrium state after a temporary disturbance (ponomarov & holcomb, 2009). although the concept of resilience originated from the domain of pure science, it has subsequently filtered down to other domains of engineering, social sciences, and business management (clapham, 1971; holling, 1973). a supply chain is one of the latest inductees into this group. earlier studies in supply chain resiliency were conducted in the uk in the early 21st century, following the (in)famous fuel protests in 2000 that caused considerable disruption in transportation (pettit et al., 2010). the concept of supply chain resiliency was subsequently studied and crystallized by christopher and peck (2004), who were the first to develop a framework for a resilient supply chain. christopher and peck (2004) further suggested that resiliency in a supply chain can be created through a basic set of principles and can serve as one of the most powerful strategies in combating supply chain disruptions. around the same time, sheffi & rice jr. (2005) concluded that a resilient supply chain not only aids an organization in managing risks, but also helps it to be better positioned than the competition and even gain an advantage from disruptions. over the years, the importance of supply chain resiliency has grown exponentially, especially in the face of globalization and persistent uncertainties. recently, there has been a lot of focus on supply chain resiliency, especially in the context of supply chain disruption and sustainability. resiliency has also been a key focus in evaluating the robustness of a supply chain with the idea that there are certain features that if engineered into a supply chain can improve its resilience (yadav et al., 2011; soni et al. 2014). integrating resiliency into supply chains helps them efficiently and effectively recover from disruptions. it should be noted that resiliency does not guarantee elimination of risks, but ensures speedy and efficient recovery following a disruptive event. furthermore, a resilient supply chain does not always translate into a low-cost supply chain, but is definitely more capable of coping with uncertainties (carvalho et al., 2012). therefore, resilience allows an organization to manage disruptions in the supply chain and continue to deliver their products and services to customers (ambulkar et al., 2015). sheffi & rice jr. (2005) argued that resiliency is a major weapon for competitive advantage in the face of unforeseen disruptions, and building redundancy and flexibility into the system can be useful strategies in combating the same. pettit et al. (2010, 2013) proposed two important constructs for resiliency – vulnerabilities and capabilities. they termed the optimal balance between the two as the ‘zone of balanced resilience’. therefore, the goal of a resilient supply chain is to recover to the desired state of the system that has been disturbed within an acceptable time and at an acceptable cost as well as to reduce the impact of a disturbance by changing the impact of a potential threat (haimes, 2006). as a result, a resilient supply chain needs to have the ability to use the lowest possible amount of resources during the recovery process, and the supply chain ijahp article: ganguly, kumar /evaluating supply chain resiliency strategies in the indian pharmaceutical sector: a fuzzy analytical hierarchy process (f-ahp) approach international journal of the analytic hierarchy process 157 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.620 that incurs the lowest resiliency cost will be considered the most resilient supply chain (vugrin et al., 2011). over the years, researchers and industry executives have conducted numerous studies concerning strategies to develop resilient supply chains. the strategies discussed by the researchers range from increasing supply chain visibility (calvalho et al., 2012b; glickman & white, 2006; kraft et al., 2018; pettit et al., 2010, 2013; zhang et al., 2011) and flexibility/agility (christopher & holweg, 2011; ganguly et al., 2018a, 2018b; pettit et al., 2013; rice & caniato, 2003; sheffi & rice jr., 2005; tang 2006; tang & tomlin, 2008) to greater knowledge management (christopher & peck, 2004; rice & caniato, 2003; ponis & koronis, 2012; ponomarov & holcomb, 2009; urciuoli et al., 2014) to appropriate supplier selection (mascaritolo & holcomb, 2008). however, while reviewing the strategies associated with designing a resilient supply chain, we observed that while being very generic in nature, researchers also did not separate the strategies from the ‘trivial many’ to a set of the ‘vital few.’ additionally, prior research has also overlooked prioritizing the identified strategies to assess their relative criticality which, if listed, would aid the policymakers in formulating a better action plan for developing a resilient supply chain. the current study attempts to focus upon this important, but undiscussed issue. 2.2 an overview of the indian pharmaceutical sector the indian pharmaceutical sector, which holds an important position in the global pharmaceutical industry, is one of the key drivers of growth in india. according to a report by pwc india in 2018, the indian pharmaceuticals market is the third-largest in terms of volume and thirteenth-largest in terms of value, along with being the global leader in providing generic drugs to the world. as a result, the importance of the pharmaceutical sector in india has attracted enormous interest from academicians and practitioners alike. the pharmaceutical sector in india has been viewed through multiple research lenses. athreye et al. (2009) studied the dynamic capabilities of four large indian pharmaceutical firms for international collaboration and r&d and innovation. kale & little (2007) studied the process and stages involved in the indian pharmaceutical sector’s transition from an importer to generic exporter to innovator. chittoor & ray (2007) studied the internationalization strategies of 40 indian pharmaceutical firms and their use of conventional market exploitation and exploration strategies towards becoming third world multinational corporations. a study of changing patent laws and the stringent regulatory environment in india and china by grace (2004) found that the enhanced ip protection laws result in revenue losses and increased emphasis on export in profitable regulated markets. similar outcomes were evidenced by greene (2007) during the study of the indian pharmaceutical industry and their effect on the american pharmaceutical market post-2005. saranga (2007) used the dea to analyze how inefficient indian pharmaceuticals firms can manage buyers, vendors and other peer groups post-2005 due to the new intellectual property protection laws. kar & pani (2014) studied supplier selection and evaluation using ahp for 188 indian firms across 12 industries including pharmaceutical industries with attributes applicable to all firms. however, no research work has discussed supply chain resiliency for the indian pharmaceutical sector, and the ijahp article: ganguly, kumar /evaluating supply chain resiliency strategies in the indian pharmaceutical sector: a fuzzy analytical hierarchy process (f-ahp) approach international journal of the analytic hierarchy process 158 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.620 current study is expected to understand the same using fuzzy analytic hierarchy process approach. 2.3 selection of fuzzy ahp for the study fuzzy ahp’s ability to identify and prioritize attributes has been used extensively within various decision-making processes in the different areas of management research. junior et al.’s (2014) comparison and sensitivity analysis of f-ahp and f-topsis found that fahp performs better in terms of time complexity and allowing inclusion of sub-criteria into a hierarchical structure. further, f-ahp performs better in terms of modeling uncertainty with an objective of supplier elimination (junior et al., 2014). triantaphyllou & lin (1996) stated that f-ahp fairs only second to fuzzy revised ahp, and is better than f-topsis and other mcdm methods, and more capable of managing human decision-making ambiguity in a complex environment and with real-life data. further, a comparative analysis of f-ahp and f-topsis for the selection of facility location by ertuğrul & karakaşoğlu (2008) found that both provide similar and consistent ranking of criteria. however, f-ahp has more flexibility in terms of a hierarchical spread of attributes. a study to identify an operating system by balli & korukoğlu (2009) found that f-ahp is a useful technique for the evaluation of complex multi-criteria alternatives with subjectivity, while topsis is more tuned towards ranking alternatives. similar observations have been made by tadic et al. (2009) while evaluating quality goals in an organization and use of f-ahp was found less mathematically cumbersome and more attuned to weight linguistics and qualitative attributes as opposed to topsis. we will build upon these previous studies by substantiating the use of f-ahp in scenarios where qualitative decision-making judgements are to be taken to provide weights for the attributes. 2.4 overview of fuzzy ahp the analytical hierarchy process (ahp) is a multi-criteria decision-making technique for prioritization of alternative attributes when multiple attributes influence the decisionmaking process (saaty, 1980; ganguly & merino, 2015). it (ahp) attempts to achieve consistent pairwise judgments through use of the properties of reciprocal matrices and allows one to rank the actions, criteria, objectives or any attribute relevant to the decision (hughes, 2009). the initialization of the ahp analysis requires the construction of a hierarchy describing the problem statement and then deconstructing it in a hierarchical structure. there are no limitations regarding the number of levels in the hierarchy breaking down the analysis structure in attributes and sub-attributes to substantiate the problem statement and its characteristics (dave et al., 2012). the problem statement or the objective resides at the top of the hierarchical tree, followed by attributes and subattributes. the bottom level consists of alternatives providing the selection set. elaboration of the ahp model is done after the construction of the hierarchy. through a set of pairwise comparison of attributes at the same level, a matrix of relative importance is developed. the pairwise comparisons require the respondents to compare the siblings based on their relative importance to each other. the relative importance is based on a numerical scale with siblings on each side of the scale. this allows the respondent to weigh the pair and provide the relative importance for one of the siblings or select neutrality. ijahp article: ganguly, kumar /evaluating supply chain resiliency strategies in the indian pharmaceutical sector: a fuzzy analytical hierarchy process (f-ahp) approach international journal of the analytic hierarchy process 159 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.620 though the ease of use of the ahp has made is one of the most widely used decisionmaking techniques, the use of a 9-point scale is considered one of the major shortcomings of the method, as it sometimes is not able to cater to the uncertainty in the decisionmaking process (kilincci & onal, 2011). the ahp lacks the ability to allow decisionmaking in uncertain situations when preferences are not objective in nature due to information asymmetry, complexity and other uncertain attributes of the environment (nieto-morote & ruz-vila, 2011). as a result, fuzzy numbers and scales have been developed to consider this issue in decision-making. over the years, fuzzy sets have been subsequently extended to type-2 fuzzy sets, intuitionistic fuzzy sets, hesitant fuzzy sets, pythagorean fuzzy sets, and neutrosophic sets, while new extensions of fuzzy ahp have been proposed by various authors accordingly (kahraman, 2018). introduction of fuzziness within the ahp technique allows this uncertainty to be considered, and the improved technique is called fuzzy-ahp (f-ahp). this insertion of fuzzy theory into the traditional ahp allows one to account for the vagueness in the respondent’s judgment (ayhan, 2013). due to the fuzzy characteristic of f-ahp, determination of weights based on pairwise comparison becomes slightly complex for computation (nieto-morote & ruz-vila, 2011). the refined method uses triangular fuzzy numbers along with linguistic variables for alternative comparison for priority derivation of different selections (kilincci & onal, 2011). using the triangular fuzzy numbers as a replacement for the 9-point scale of the traditional ahp, f-ahp successfully deals with the uncertainty of the decision-making process. the f-ahp uses the extent analysis method for the pairwise analysis and calculates synthetic extent values for the same (kilincci & onal, 2011). after finalization of the weights, the normalized weights are calculated. further, priority weights of the final alternatives are computed through the use of varied weights of criteria and attributes. the alternative with the highest value is chosen as the preferred one, or is the most important attribute, whatever the case may be. the comparison between ahp and f-ahp method has been provided in table 1. table 1 comparison of ahp vis-à-vis f-ahp scales (adapted and modified from ayhan, 2013) ahp scale definition f-ahp scale 1 equally important (1,1,1) 3 somewhat important (2,3,4) 5 fairly important (4,5,6) 7 strongly important (6,7,8) 9 extremely important (9,9,9) 2,4,6,8 the intermittent values (1,2,3), (3,4,5), (5,6,7), (7,8,9) following the above f-ahp scale, if attribute a is strongly preferred over attribute b, then it is denoted by the fuzzy triangular scale as (6, 7, 8) and its inverse as (1/8, 1/7, 1/6) ijahp article: ganguly, kumar /evaluating supply chain resiliency strategies in the indian pharmaceutical sector: a fuzzy analytical hierarchy process (f-ahp) approach international journal of the analytic hierarchy process 160 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.620 and so on. based on this scale, the relative prioritized weights at each level of the hierarchy is determined, which is subsequently used to arrive at the final alternative. 2.5 prior efforts in using ahp and fuzzy ahp in supply chain research ahp and fuzzy (f) – ahp have been used extensively for various kinds of attributebased analysis in supply chain management. a study done by mardani et al. (2015) found 23 (6%) supply chain management papers out of 393 papers reviewed between 2010 and 2014 used ahp, second only to the number of papers in the research area of energy, environment and sustainability. identification of attribute weights for both qualitative and quantitative attributes has been used for prioritizing suppliers in a resilient supply chain (rajesh & ravi, 2015). lópez & ishizaka (2017) have analyzed threats to supply chain resilience arising from offshore outsourcing location decisions using ahp along with fuzzy cognitive maps, and they conclude that offshoring might lead to reduced flexibility, financial strengths and recovery capacity of supply chains. wang et al. (2017) used ahp to analyze the resilient nature of the green construction supply chain. their study found that product quality, commodity cost, legal environment, new technology, service distance and level of informalization etc. had higher weights than time, quantity and service flexibility. ho et al.’s (2015) literature review of supply chain risk management identified the ahp as one of the top five techniques along with mathematical modeling, news vendor model, simulation and game theory. risks in operations areas like machine and equipment failure, design risks and skilled labor scarcity were found to be the most critical ones for green supply chains rather than supplier based, demand forecasting, financial or product recovery for an f-ahp study by mangla et al. (2015). a study of the supply chain for petroleum products from a disruptions perspective using f-ahp found transparency, flexible transportation, secure communication networks, and human resource management as the most weighted attributes from the overall supply chain resiliency perspective (ghasemzadeh et al., 2017). an analysis of 288 peer-reviewed articles published between 2000 and 2017 from the supply chain resilience perspective found the ahp as one of the most used techniques among mcdm after mathematical modeling (kochan & nowicki, 2018). use of ahp for supplier selection based on supply chain resiliency in south-east asian countries like china, vietnam and hong kong found that robustness, reliability, and rerouting are the most important attributes (hosseini & al khaled, 2016). adebanjo et al. (2016) studied lean supply chain management in healthcare services operations using f-ahp and found that operational performance and financial factors are the important attributes describing 96% of the initiatives for lean management. ayhan (2013) analyzed the supplier selection problem for a gear motor company using f-ahp, and the study found quality and origin of raw material as the two most important attributes, others being cost, delivery and after-sales services. chen & yuan (2013) used ahp to select outsourcing destinations in east and south-east asia, and found that based on analysis of costs, human resources, the business environment, and the policy and legal environment, china, philippines, and singapore are the best destinations with cost considerations being the major driving factor (0.44) and the other three attributes scoring nearly equal weights (0.18-0.19). kahraman et al. (2003) used f-ahp to understand the perspective of purchasing managers of white goods in supply chains in turkey for supplier selection, and found that supplier criteria (0.43) and product performance (0.37) outweigh service performance (0.20). among the sub-attributes financial performance ijahp article: ganguly, kumar /evaluating supply chain resiliency strategies in the indian pharmaceutical sector: a fuzzy analytical hierarchy process (f-ahp) approach international journal of the analytic hierarchy process 161 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.620 of supplier criteria (0.70) and professionalism (0.95) in service performance outweighed other sub-attributes. patil & kant (2014) have developed a fuzzy ahp-topsis method to provide ranking for solutions for overcoming knowledge management barriers in supply chains. application of ahp for selection of building material suppliers among 10 usa-based suppliers found that tender price (0.25) and life-cycle cost (0.17) were the most weighed attributes among 10 attributes (plebankiewicz & kubek, 2015). supplier selection among 7 regional contractors for a clothing company in turkey based on the minimum and maximum order quantity, under or over achievement and other mathematical programming constraints found that product quality (0.51) and delivery capability (0.24) have higher values than service capability (0.19) and product discount (0.06). for the selection of a lead-free equipment supplier, the use of f-ahp found that product compatibility, response and maintenance time, and upgrades and expansibility share nearly 50% of the attribute weights among seven attributes (tang & lin, 2011). a study for supplier selection in the indian manufacturing context using f-ahp found that quality (0.41) and cost (0.25) are the most valued attributes as opposed to delivery, service, long-term relationship and flexibility (yadav & sharma, 2015). a study of the garment industries in bangladesh using quality function deployment and ahp to mitigate supply chain vulnerabilities and develop a resilience index found that the most resilient strategies are building backup capacity, relationship with buyers and suppliers and quality control (chowdhury & quaddus, 2015). the ahp has been used to assess the supply chain risk and manage it based on ranking the severity, and its magnitude has been developed by dong & cooper (2016). rathore et al. (2017) used grey-ahp and greytopsis to develop a risk assessment methodology and have applied this to the food product supply chain. however, in spite of mcdm and ahp establishing themselves as important tools in the domain of supply chain research, their utilization in the area of supply chain resiliency has been sparse. the current study is expected to shed some light on this important but sparsely discussed issue. 3. research model and methodology the basic foundation of the current investigation lies on two concatenate research questions which are provided below: rq1: what are the important strategies required to design a resilient supply chain? rq2: how can an organization prioritize the identified strategies to assess their relative criticality, so that they can design a resilient supply chain more effectively? based on the two aforementioned research questions, the objective of the present study was to identify and prioritize a set of strategies that might result in an organization designing a resilient supply chain. the research process started with identifying a set of resiliency strategies and was initially done through a thorough review of the available open literature in the field. approximately 70 research papers were reviewed on resiliency and supply chain resiliency and an initial laundry list of a set of strategies associated with supply chain resiliency was developed. this set was subsequently narrowed down, through interviews with the subject matter experts, to validate their authenticity as well as filter out the important ones. the subject matter experts, who are pivotal to a successful ahp analysis, were identified after careful scrutiny and consisted ijahp article: ganguly, kumar /evaluating supply chain resiliency strategies in the indian pharmaceutical sector: a fuzzy analytical hierarchy process (f-ahp) approach international journal of the analytic hierarchy process 162 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.620 of indian industry professionals having more than 15 years of experience in the domain of supply chain including, being a part of multiple supply chain related projects. in addition, their knowledge regarding supply chain resiliency and disruptions was also taken into consideration as part of the selection process. the discussions and interviews with the smes were conducted formally and informally by the authors through telephone, skype and face to face. salient points arising out of the discussions were duly noted and subsequently used to finalize the set of resiliency strategies. furthermore, the data gathered through the interviews of the experts was subsequently analyzed to remove ambiguity and discrepancies. it was observed, that although the experts were interviewed separately, a distinct pattern emerged regarding the relative importance of the strategies in their opinion. therefore, the top seven strategies, as identified by all the experts were taken as the final set of resiliency strategies and were used for further analysis and prioritization. once the final set of strategies was identified, the next stage was to use the experts to prioritize them in order to assess their relative criticality while not compromising the accuracy of the information. a set of 23 subject matter experts in the supply chain field were identified and given an ahp based questionnaire for prioritization of the identified strategies 1 . the questionnaire was designed with a self-administered survey in mind and the experts were briefed about the format of the ahp questionnaire as it varied from its traditional likert scale counterpart. the data was gathered in the form of a survey questionnaire and was designed for prioritization of the identified risks. the survey was administered to the experts separately and individually in order to eliminate a possible response bias. in this context, it should be mentioned that before the survey was sent out to the targeted experts, a pilot survey was conducted on a small group of four to five industry experts with considerable experience in supply chain management. the purpose of the pilot study was to reveal deficiencies, if any, in the design of the survey before it was deployed within the main study. some of the issues that were revealed by the pilot study were addressed prior to sending out the final survey questionnaire. the subsequent sections of the paper discuss the identified strategies along with using fuzzy ahp to prioritize them. 4. identifying the resiliency strategies in the present-day business scenario, where the competition is often not between organizations, but rather between their supply chains, a major cause of concern for an organization is the failure of its supply chain. disruptions have become fairly frequent in supply chains, especially with rapid globalization and increased complexity of the supply chains. the purpose of the current study, as mentioned previously, was to identify and prioritize a set of strategies that would aid an organization in the design of a resilient supply chain. the initial set of strategies was identified based on a thorough review of the extant literature in the field, coupled with discussions and deliberation with experts associated with the production planning and controls and operations. the initial set of strategies were further refined into a final set through additional discussions with the 1 since ahp is an expert based decision-making technique, the number of survey respondents was low. therefore, the number of respondents in the current study was 23, which can be considered a fairly acceptable number for the ahp. ijahp article: ganguly, kumar /evaluating supply chain resiliency strategies in the indian pharmaceutical sector: a fuzzy analytical hierarchy process (f-ahp) approach international journal of the analytic hierarchy process 163 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.620 experts to arrive at a final, condensed set. the final set of the identified strategies are discussed below. 4.1 increasing supply chain visibility christopher & peck (2004) have defined supply chain visibility as the ability of an organization to clearly see through its supply chain. pettit et al. (2010) conceptualized visibility as the knowledge an organization can have regarding the status of its operating assets and the environment. having clear visibility of a supply chain enables an organization to have a clear idea about the entire supply chain, therefore helping detect signals of future disruptions (tukamuhabwa et al., 2015). in turn, this helps the organization avoid any unnecessary intervention and ambiguous/ineffective decisions in situations of possible disruption, along with quickly responding to disturbances based on real-time assessment (glickman & white, 2006; pettit et al., 2013). greater visibility of a supply chain can also help an organization disclose supply chain resiliency information to its customers, along with having a positive impact on the organization’s social responsibility (kraft et al., 2018). furthermore, greater visibility in the supply chain is required to increase the level of responsiveness of a supply chain, both from the demand and supply context, which is often considered a key component to combatting disruptions (williams et al., 2013). since disruptions in supply chains are considered one of the precursors to resiliency, supply chain visibility was considered one of the key strategies to achieve supply chain resiliency. 4.2 supply chain agility for organizations to thrive and grow in the current market, being agile is one of the fundamental aspects needed for success. agility is the capability of an organization (and its supply chain) to efficiently function in an unpredictable environment marked by continuous (and often uncertain) changes in the business environment by reacting quickly (cho et al., 1996; gunasekaran, 1999). this has become one of the most powerful weapons for organizations and their supply chain. supply chain agility can be considered as the ability to respond quickly to unpredictable events through rapid changes in business processes and systems (christopher & peck, 2004; erol et al., 2010). agility and velocity (which is the speed with which a supply chain can adapt) pertaining to both positive and negative changes in the external environment have become the ability of a supply chain to exploit unexpected emergencies (peck, 2005; ponomarov & holcomb 2009; jüttner & maklan, 2011). therefore, agility is concerned with the continuous search for the most appropriate response to manage uncertainty and unpredictability within the business environment and requires the presence of agile partners both upstream and downstream in the supply chain (ganguly et al., 2018b; christopher & peck, 2004). since agility of a supply chain involves adapting to unforeseen changes (which might even include disruptions) in a cost effective and timely manner, this is often considered as one of the key strategies to achieve resiliency. 4.3 creating supply chain redundancy as a convention, most organizations think of surplus capacity and inventory as a waste, and therefore an undesirable option (christopher & peck, 2004). however, contrary to the common belief, redundancy has often been considered a deliberate strategy to achieve resiliency in supply chains. redundancy can be stated as “the strategic and selective use of spare capacity and inventory that can be used to cope with disruption…” and involves ijahp article: ganguly, kumar /evaluating supply chain resiliency strategies in the indian pharmaceutical sector: a fuzzy analytical hierarchy process (f-ahp) approach international journal of the analytic hierarchy process 164 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.620 use of multiple suppliers, excess inventory and additional facilities, among others (tukamuhabwa et al., 2015). sheffi & rice (2005) argued that resiliency can be enhanced by building redundancy into the supply chain, where significant redundancy can aid in a faster recovery after disruption. strategic deployment of additional inventory or capacity at a certain point (which they termed as ‘pinch-points') can be beneficial to creating a supply chain that is resilient in nature (christopher & peck, 2004). an organization can ensure redundancy through holding extra inventory and having multiple suppliers and locations, all with the objective of dealing with disruptions (christopher & peck, 2004; tomlin, 2006). therefore, redundancy is a capability that can be employed to ensure resilience, regardless of the need for resiliency emerging from demand volatility, or due to supply and/or internal disturbances (purvis et al., 2016). 4.4 supply chain collaboration with increased competition and a turbulent business environment, more and more firms (and their supply chains) have started collaborating to reap greater benefits of operations. the fundamental reason supporting collaboration is that a single company cannot successfully compete by itself and therefore, collaboration involves joint ownership of decisions and collective responsibility for outcomes (min et al., 2005; gray, 1991). a high level of collaboration across supply chains can significantly help reduce disruptions (christopher & peck, 2004). the perspective of supply chain collaboration enables the supply chain partners to extract more relational rents for their own competitive advantage (cao & zhang, 2011). as a result, collaborating supply chain partners will ensure supply chain efficiency for their mutual benefit. through collaboration, supply chain partners can work effectively with other supply chain entities and reduce vulnerabilities (tukamuhabwa et al., 2015). since reducing vulnerabilities is one of the fundamental aspects of a resilient supply chain, supply chain collaboration can be a useful strategy for ensuring a resilient supply chain. supply chain collaboration as a strategy to address disruptions has been repeatedly stressed in the extant literature (leat & revoredo-giha, 2013; pettit et al., 2010, 2013; ponis & koronis, 2012; ponomarov & holocomb, 2009). since resilience is key to addressing and mitigating disruptions in a supply chain, supply chain collaboration was considered to be an important strategy to achieve supply chain resiliency. 4.5 knowledge management across supply chain one of the fundamentals for success of supply chain collaboration is effective sharing of knowledge and information across the different entities of the supply chain (christopher & peck, 2004; jüttner & maklan, 2011). as supply chains have become more and more complex in nature, the difficulty in managing them has increased exponentially, resulting in an increased susceptibility to vulnerability. it has been observed that managers often do not have sufficient knowledge to address various issues within the supply chain. lack of knowledge/information sharing is still a significant roadblock towards efficient functioning of a supply chain (baily & francis, 2008; kong & li, 2008). one of the avenues to address supply chain vulnerabilities is enabling the exchange of information among the members of that supply chain community (christopher & peck, 2004). effective management and sharing of information/knowledge can enable the supply chain to be ready for any disruptive event through increased supply chain visibility, and therefore improvement in event detection and the organization’s reaction (jüttner & maklan 2011; manuj & mentzer 2008). this makes knowledge sharing an important ijahp article: ganguly, kumar /evaluating supply chain resiliency strategies in the indian pharmaceutical sector: a fuzzy analytical hierarchy process (f-ahp) approach international journal of the analytic hierarchy process 165 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.620 strategy for creating a resilient supply chain. furthermore, li et al. (2006) observed that if the information is shared downstream in a timely fashion, then disruptions upstream can be avoided or mitigated. therefore, knowledge and understanding of supply chain structures, coupled with proper knowledge management and sharing can underpin a supply network’s resilience (choi & hong, 2002; scholten et al., 2014). on the contrary, failure to share information/knowledge among the intra and inter supply chain entities might lead to supply chain opacity, leading to compromised resiliency of the supply chain. 4.6 inventory management as mentioned earlier, injecting redundancy into supply chains can be used by organizations as a resiliency strategy. one way to increase supply chain redundancy is through inventory redundancy (barroso et al., 2010). barroso et al. (2010) further argued that building redundant inventory in supply chains can go a long way in maintaining continuous operation of the supply chains, thereby combating any possible disruption. additionally, research has also shown that holding excess inventory is as an important strategy to maintain supply chain resiliency (mckinnon, 2014; ole-hohenstein et al. 2015). boone et al. (2013) mentioned that the strategic alignment of inventory plays a very important role in achieving supply chain resiliency, an argument that supported the findings of carvalho & cruz-machado (2011). therefore, with subsequent discussions with the subject matter experts, it was decided to include this as a strategy for achieving supply chain resiliency. 4.7 appropriate supplier selection since suppliers can be considered one of the major sources of external disruption in modern supply chains, appropriate selection of suppliers requires factoring in resiliency of the supply chains (rajesh & ravi, 2015). supplier selection is a process through which an organization identifies and evaluates its vendors with the objective of signing a possible contract (mital et al., 2018). supplier selection often forms the backbone of successful supply chains and therefore, organizations need to pay careful attention to the supplier selection process. proper selection of suppliers can enable an organization to achieve unprecedented success, while poor selection of suppliers can lead to failure (rahman & haldar, 2018). effective selection of suppliers can aid an organization in building a resilient supply chain, which can lead to a reduction in disruptions and vulnerability (hosseini & khaled, 2016; parkouhi & ghadikolaei, 2017). it can also support a supply chain to successfully combat any disruption and even bounce back from it. additionally, effective supplier selection can aid in financial stability, business continuity, supply chain reliability, and quality, among others (tukamuhabwa et al., 2015). figure 1 exhibits the overall hierarchical structure of the supply chain resiliency strategies. ijahp article: ganguly, kumar /evaluating supply chain resiliency strategies in the indian pharmaceutical sector: a fuzzy analytical hierarchy process (f-ahp) approach international journal of the analytic hierarchy process 166 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.620 figure 1 basic ahp structure for the resiliency strategies 6. using fuzzy ahp to prioritize supply chain resiliency strategies the current study applies the following steps as a part of the f-ahp analysis which are adapted and modified from the study conducted by badri & abdulla (2004) and ayhan (2013). the steps involved are as follows: 1. defining the problem/objective of the study (in the context of the current study, it was ranking the important strategies associated with supply chain resiliency). 2. identifying the attributes for supply chain resiliency and structuring the ahp hierarchy (figure). 3. constructing a pairwise comparison matrix of the attributes with the objective of determining their impact on the overall object of the study. 4. identifying the experts for comparing the identified strategies’ criteria or alternatives in order to obtain pairwise judgments among the identified attributes in step 2; since there was more than one expert, preferences of each experts are averaged. 5. calculating the geometric mean of the fuzzy comparison values of each criterion following the method adapted by buckley (1985). 6. determining the fuzzy weights of each of the criterion and calculating the vector summation of the triangular values. 7. calculating the inverse (-1) power of the summation vector and replacing the fuzzy triangular number, to make it in an increasing order. 8. determining the fuzzy weight of criterion and multiplying each with this reverse vector. 9. based on the survey responses, prioritized weights of the identified attributes are obtained and consistency among the judgments tested. 10. determine the final prioritized set of attributes related to the overall objective of the study and chose the best alternative among a portfolio of alternatives ijahp article: ganguly, kumar /evaluating supply chain resiliency strategies in the indian pharmaceutical sector: a fuzzy analytical hierarchy process (f-ahp) approach international journal of the analytic hierarchy process 167 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.620 the first step of the survey consisted of designing the ahp hierarchy as exhibited in figure 1. once the ahp structure was determined, the next step consisted of experts comparing the identified strategies to determine their relative criticality. the scale used in the questionnaire was the ahp scale provided in table 1. also, although mentioned earlier, it would be worthwhile to reiterate that though the standard ahp scales were used in the survey, the data was subsequently converted to the fuzzy triangular scale (refer table 1) before analysis, in order to facilitate the fuzzy ahp calculations. the identified strategies were compared to the overall objective of the study. table 2 shows the pairwise comparison matrix of the identified supply chain resiliency strategies as provided by one of the respondents 2 . table 2 pairwise comparison of the resiliency strategies visibility agility collaboration redundancy supplier selection inventory km/ks visibility (1.1.1) (1/4,1/3,1/2) (6,7,8) (4,5,6) (6,7,8) (4,5,6) (6,7,8) agility (2,3,4) (1.1.1) (2,3,4) (4,5,6) (6,7,8) (6,7,8) (7,8,9) collaboration (1/8,1/7,1/6) (1/4,1/3,1/2) (1,1,1) (4,5,6) (6,7,8) (4,5,6) (5,6,7) redundancy (1/6,1/5,1/4) (1/6,1/5,1/4) (1/6,1/5,1/4) (1,1,1) (2,3,4) (1,1,1) (2,3,4) supplier selection (1/8,1/7,1/6) (1/8,1/7,1/6) (1/8,1/7,1/6) (1/4,1/3,1/2) (1,1,1) (1/6,1/5,1/4) (3,4,5) inventory (1/6,1/5,1/4) (1/8,1/7,1/6) (1/6,1/5,1/4) (1,1,1) (4,5,6) (1,1,1) (4,5,6) km/ks (1/8,1/7,1/6) (1/9,1/8,1/7) (1/7,1/6,1/5) (1/4,1/3,1/2) (1/5,1/4,1/3) (1/6,1/5,1/4) (1,1,1) after the initial prioritization of the identified strategies, the next step was to determine the geometric mean of the fuzzy comparison value for each of the strategies. the geometric mean is calculated using equation 1: rĩ = [∏ x̃ij n j=1 ] 1 n (1) where, rĩ = geometric mean x̃ij = weights of the attributes and or sub − attributes n = number of attributes / sub-attributes calculating the geometric means was followed by the fuzzy weights of the attributes, which is subsequently followed by de-fuzzying and normalizing the weights. the fuzzy weights were determined by multiplying the geometric mean (rĩ) with the reverse fuzzy triangular numbers (ftns), arranged in ascending order, as illustrated in equation 2. 2 it should be noted here that tables 2 & 3 exhibit the results of one of the respondents out of the 23 respondents surveyed. the primary reason for this is that the ahp response matrix for all the respondents surveyed has the same structure with different values. the final table (table 4) exhibiting the final, prioritized values, is an average of all the survey responses. ijahp article: ganguly, kumar /evaluating supply chain resiliency strategies in the indian pharmaceutical sector: a fuzzy analytical hierarchy process (f-ahp) approach international journal of the analytic hierarchy process 168 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.620 wi = r̃i ⊗ (r̃1 ⊕ r̃2 ⊕ … … . .⊕ r̃n) −1 = (aw, bw, cw) (2) the determined fuzzy weights were then subsequently de-fuzzied and normalized, thereby accounting for the final set of attribute and sub-attribute weights 3 . the findings have been provided in table 3. table 3 geometric mean and normalized weights of the resiliency strategies geometric mean (rĩ) fuzzy weights (wi) mi ni visibility 2.63 3.12 2.63 0.255 0.325 0.331 0.304 0.259 agility 3.28 4.05 4.77 0.318 0.421 0.601 0.447 0.380 collaboration 1.47 1.75 2.08 0.143 0.182 0.262 0.196 0.167 redundancy 0.57 0.69 0.82 0.055 0.071 0.103 0.077 0.065 supplier selection 0.30 0.36 0.43 0.030 0.037 0.055 0.041 0.034 inventory 0.66 0.76 0.87 0.064 0.079 0.110 0.084 0.072 km/ks 0.21 0.24 0.30 0.020 0.025 0.037 0.028 0.023 total 7.95 9.60 10.30 inverse (power of 1) 0.126 0.104 0.097 increasing order 0.097 0.104 0.126 mi = non-fuzzy weights and ni = normalized weights table 3 provides the prioritized weights of the identified strategies to achieve supply chain resiliency. as mentioned earlier, this is the response of one of the 23 experts surveyed as a part of this research project since the ahp response matrix is the same across all the respondents, but with different values. table 4 details the average of all the experts surveyed as a part of the study along with the rankings of the strategies. 3 for a detailed explanation of the fuzzy ahp process and the fuzzy triangular distribution please refer to ayhan (2013) and singh and prasher (2019). ijahp article: ganguly, kumar /evaluating supply chain resiliency strategies in the indian pharmaceutical sector: a fuzzy analytical hierarchy process (f-ahp) approach international journal of the analytic hierarchy process 169 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.620 table 4 ranking of the resiliency strategies along with their normalized weights over the entire set of respondents (n=23) resiliency strategies mean normalized weights rank agility 0.346 1 visibility 0.277 2 collaboration 0.175 3 inventory management 0.078 4 redundancy 0.072 5 supplier selection 0.031 6 knowledge management/sharing 0.021 7 finally, it’s necessary to mention that although the ahp structure exhibited in figure 1 includes the alternatives, as the current study intends to prioritize the resiliency strategies, the last level of the hierarchy (the alternatives – supplier 1, 2 & 3) was considered to be beyond the scope of the current study and therefore was omitted. additionally, it should also be mentioned that following the methodology suggested by leung and cao (2000a, 2000b), a consistency test for the pairwise judgments was performed and consistencies were within the permissible limits. the following section provides a detailed discussion of the findings. 7. results and findings the results provide a glimpse into the supply chain resiliency attitudes in the indian context. it was found that there are similarities as well as contrasts with the global context. while the highest weights for agility and visibility were in line with the global research, a low value for knowledge management was in contrast with the global research. the top three resiliency strategies identified were agility, visibility, and collaboration, contributing towards nearly 0.80 of the attribute weights. the highest weight was for agility (0.37), which is consistent with the understanding that agile supply chains are also agile during external shocks, making them more resilient. this is consistent with the findings of wieland & wallenburg (2013) who discuss agility from the customer care perspective while adding robustness as a measure for agility of supply chains. the second highest attribute weight was for supply chain visibility (0.28). this is consistent with the findings of brandon‐jones et al. (2014) whose confirmatory factor analysis study found factor loading of 0.90 for inventory-based visibility and 0.73 for demand-based visibility covering 264 different organizations. collaboration was the third highest weighted attribute with a weight of 0.18; this is also consistent with the qualitative research work done by scholten & schilder (2015). inventory management (0.08) and redundancy (0.07) have a minor share in attribute weights, which are in line with the global literature. however, the low weights for knowledge management/sharing (0.02) contrast the indian context with global context (li et al., 2006; scholten et al., 2014). overall, the authors found that if a supply chain focuses upon agility, visibility, and collaboration during normal phases, it can be considered a resilient supply chain during disruptive phases and will be able to cope with the negative externalities. ijahp article: ganguly, kumar /evaluating supply chain resiliency strategies in the indian pharmaceutical sector: a fuzzy analytical hierarchy process (f-ahp) approach international journal of the analytic hierarchy process 170 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.620 mcdm and f-ahp have been used extensively in various aspects of supply chain management e.g. facility location, supplier selection, quality control etc. however, research using f-ahp on resiliency strategies are relatively sparse. through this study, the authors intend to provide inroads for f-ahp and other mcdm techniques into the area of supply chain resiliency. this is of critical importance considering the current global economic trade volatility where changing business scenarios guided by cut-throat competition and continuous uncertainties have made decision-making an extremely complex and confusing process. further, the decision-making process in organizations has stretched its boundaries from being just economic in nature to including a plethora of non-economic factors of business operations. as a result, mcdm has started playing a huge role in current day decision making. the current study, using supply chain resiliency strategies as an illustrative example, attests to the importance of mcdm (and f-ahp) in the business decisionmaking process. 8. limitations of the current study while f-ahp is a robust tool for relative importance, the consistency index remains an issue for large groups of data, grouped data sets or an attribute matrix size of more than 4 (saaty, 1990; 1994). however, our study used data points where consistency ratios were less than 0.20, and hence is within the control limits considering the data size of 23 for a seven-attribute comparison matrix. this is also in line with the recommendation of wedley (1993) where 0.2 or less is acceptable for large attribute matrix sizes. a comparison of f-ahp and f-topsis for supplier selection for an automotive company by junior et al. (2014) also uses a ci of less than 0.20 considering it under control limits. further, in other areas of research, ho et al.’s (2005) study of property-specific attributes for office building quality uses a tolerance range of 0.1 to 0.2 for ci when having a large data size. in addition, the consistency is also aligned with other similar studies in the area of supply chain management by adebanjo et al. (2016), büyüközkan et al. (2011), kahraman et al. (2003), kilincci & onal (2011), yadav & sharma (2015), and singh & prasher (2019). this makes the result and outcome of the study more robust, but at the same time it also means that a few of the samples have been left out thereby reducing the number of responses. further, the study focuses on identifying resiliency attributes for a supply chain, however, the resiliency varies based on the criticality and type of disruptions. the current research analyzes the information procured from various industry experts based on their experience in their respective sectors; this can be done at the sectorial level. 9. conclusion and further research the authors have identified the relative importance for various supply chain resiliency attributes, thereby providing managers with specific strategic areas in which to work for improvement. improvements regarding agility, visibility, and collaboration have already been incorporated by most supply chains to improve efficiency and other ijahp article: ganguly, kumar /evaluating supply chain resiliency strategies in the indian pharmaceutical sector: a fuzzy analytical hierarchy process (f-ahp) approach international journal of the analytic hierarchy process 171 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.620 production-based indicators. the study provides a new dimension by further emphasizing these attributes from the resiliency perspective. this will allow firms or supply chains to work better during disruptions and might lead to improved market share. hence, the study should not only be seen as a research work analyzing relative importance, but also as a way of identifying how resilient the existing supply chains are when they have already focused on the above-discussed attributes. this study has considered only the indian pharmaceutical industrial sector and has discussed disruptions without typology or criticality of the disruptions. further research can be done for supply chains of other industrial sectors with industry-based supply chain resiliency measures. also, industries can be studied for resiliency strategies based on the disruption typology and geographic differences while keeping the attributes unchanged. ijahp article: ganguly, kumar /evaluating supply chain resiliency strategies in the indian pharmaceutical sector: a fuzzy analytical hierarchy process (f-ahp) approach international journal of the analytic hierarchy process 172 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.620 references adebanjo, d., laosirihongthong, t., & samaranayake, p. (2016). prioritizing lean supply chain management initiatives in healthcare service operations: a fuzzy ahp approach. production planning & control, 27(12), 953–966. doi: https://doi.org/10.1080/09537287.2016.1164909 ambulkar, s., blackhurst, j., & grawe, s. (2015). firm's resilience to supply chain disruptions: scale development and empirical examination. journal of operations management, 33, 111-122. doi: https://doi.org/10.1016/j.jom.2014.11.002 athreye, s., kale, d., & ramani, s. v. (2009). experimentation with strategy and the evolution of dynamic capability in the indian pharmaceutical sector. industrial and corporate change, 18(4), 729-759. doi: https://doi.org/10.1093/icc/dtp024 ayhan, m. b. (2013). a fuzzy ahp approach for supplier selection problem: a case study in a gear motor company. international journal of managing value and supply chains (ijmvsc), 4(3), 11-23. doi: https://doi.org/10.5121/ijmvsc.2013.4302 badri, m. a., & abdulla, m. h. (2004). awards of excellence in institutions of higher education: an ahp approach. international journal of educational management, 18(4), 224-242. doi: https://doi.org/10.1108/09513540410538813 balli, s., & korukoğlu, s. (2009). operating system selection using fuzzy ahp and topsis methods. mathematical and computational applications, 14(2), 119-130. doi: https://doi.org/10.3390/mca14020119 barroso, a. p., machado, v. h., barros, a. r., & machado, v. c. (2010). toward a resilient supply chain with supply disturbances. paper presented at the ieee international conference on industrial engineering and engineering management (ieem), macao, china. doi: https://doi.org/10.1109/ieem.2010.5674462 boone, c. a., craighead, c. w., hanna, j. b., & nair, a. (2013). implementation of a system approach for enhanced supply chain continuity and resiliency: a longitudinal study. journal of business logistics, 34(3), 222-235. doi: https://doi.org/10.1111/jbl.12021 brandon‐jones, e., squire, b., autry, c. w., & petersen, k. j. (2014). a contingent resource‐based perspective of supply chain resilience and robustness. journal of supply chain management, 50(3), 55-73. doi: https://doi.org/10.1080/09537287.2012.666861 buckley, j. j., (1985). fuzzy hierarchical analysis. fuzzy sets and systems, 17(1), 233– 247. doi: https://doi.org/10.1016/0165-0114(85)90090-9 buckley, j. j. (1990). fuzzy eigenvalues and input-output analysis. fuzzy sets and systems, 34(2), 187-195. doi: https://doi.org/10.1016/0165-0114(90)90158-3 buckley, j. j. (1992). solving fuzzy equations in economics and finance. fuzzy sets and systems, 48(3), 289-296. doi: https://doi.org/10.1016/0165-0114(92)90344-4 buckley, j. j. (1995). joint solution to fuzzy programming problems. fuzzy sets and systems, 72(2), 215-220. doi: https://doi.org/10.1016/0165-0114(94)00353-9 https://doi.org/10.1016/0165-0114(92)90344-4 ijahp article: ganguly, kumar /evaluating supply chain resiliency strategies in the indian pharmaceutical sector: a fuzzy analytical hierarchy process (f-ahp) approach international journal of the analytic hierarchy process 173 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.620 buckley, j. j., feuring, t., & hayashi, y. (2001). fuzzy hierarchical analysis revisited. european journal of operational research, 129(1), 48-64. doi: https://doi.org/10.1016/s0377-2217(99)00405-1 büyüközkan, g., çifçi, g., & güleryüz, s. (2011). strategic analysis of healthcare service quality using fuzzy ahp methodology. expert systems with applications, 38(8), 9047-9424. doi: https://doi.org/10.1016/j.eswa.2011.01.103 cao, m., & zhang, q. (2011). supply chain collaboration: impact on collaborative advantage and firm performance. journal of operations management, 29(3), 163-180. doi: https://doi.org/10.1016/j.jom.2010.12.008 carvalho, h. & cruz-machado, v. (2011). integrating lean, agile, resilience and green paradigms in supply chain management (larg_scm). in pengzhong li (ed.) supply chain management. doi: https://doi.org/10.5772/14592 carvalho, h., cruz-machado, v., & tavares, j. g. (2012). a mapping framework for assessing supply chain resilience. international journal of logistics systems and management, 12(3), 354-373. doi: https://doi.org/10.1504/ijlsm.2012.047606 chen, j. k., & yuan, b. j. (2013). adopting ahp approach on evaluation and selection of outsourcing destination in east and southeast asia. technology management in the itdriven services (picmet), 2013 proceedings of picmet’13, 528–537. chittoor, r., & ray, s. (2007). internationalization paths of indian pharmaceutical firms—a strategic group analysis. journal of international management, 13(3), 338-355. doi: https://doi.org/10.1016/j.intman.2007.05.008 cho, h., jung, m. & kim, m. (1996). enabling technologies of agile manufacturing and its related activities in korea, computers and industrial engineering, 30(3), 323-334. doi: https://doi.org/10.1016/0360-8352(96)00001-0 choi, t. y. & hong, y. (2002). unveiling the structure of supply networks: case studies in honda, acura, and daimler chrysler. journal of operations management, 20(5), 469493. doi: https://doi.org/10.1016/s0272-6963(02)00025-6 chowdhury, m. m. h., & quaddus, m. a. (2015). a multiple objective optimization based qfd approach for efficient resilient strategies to mitigate supply chain vulnerabilities: the case of garment industry of bangladesh, omega, 57, 5–21. doi: https://doi.org/10.1016/j.omega.2015.05.016 christopher, m., & peck, h. (2004). building the resilient supply chain. international journal of logistics management, 15(2), 1-14. doi: https://doi.org/10.1108/09574090410700275 christopher, m., & holweg, m. (2011). supply chain 2.0: managing supply chains in the era of turbulence. international journal of physical distribution & logistics management, 41(1), 63-82. doi: https://doi.org/10.1108/09600031111101439 clapham jr., w. b. (1971). natural ecosystems. macmillan, new york, ny. ijahp article: ganguly, kumar /evaluating supply chain resiliency strategies in the indian pharmaceutical sector: a fuzzy analytical hierarchy process (f-ahp) approach international journal of the analytic hierarchy process 174 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.620 dong, q., & cooper, o. (2016). an orders-of-magnitude ahp supply chain risk assessment framework. international journal of production economics, 182, 144–156. doi: https://doi.org/10.1016/j.ijpe.2016.08.021 erol, o., sauser, b., & mansouri, m. (2010). a framework for investigation into extended enterprise resilience. enterprise information systems, 4(2), 111–136. doi: https://doi.org/10.1080/17517570903474304 ertuğrul, i̇., & karakaşoğlu, n. (2008). comparison of fuzzy ahp and fuzzy topsis methods for facility location selection. the international journal of advanced manufacturing technology, 39(7-8), 783-795. doi: https://doi.org/10.1007/s00170-0071249-8 ganguly, a., & merino, d. n. (2015). an integrated ahp-qfd approach for evaluating competing technological processes. international journal of the analytic hierarchy process, 7(3), 539-559. doi: https://doi.org/10.13033/ijahp.v7i3.315 ganguly, a., chatterjee, d., & rao, h. v. (2017). evaluating the risks associated with supply chain agility of an enterprise. international journal of business analytics (ijban), 4(3), 15-34. doi: https://doi.org/10.4018/ijban.2017070102 ganguly, a., chatterjee, d., & talukdar, a. (2018a). knowledge sharing barriers affecting pharmaceutical supply chain performance. in szmelter a. (ed.) global supply chains in the pharmaceutical industry. igi global (accepted, in press) doi: https://doi.org/10.4018/978-1-5225-5921-4.ch012 ganguly, a., chatterjee, d., & rao, h. (2018b). the role of resiliency in managing supply chains disruptions. in y. khojasteh (ed.), supply chain risk management: advanced tools, models and developments (237-251) singapore: springer. doi: https://doi.org/10.1007/978-981-10-4106-8_14 ghasemzadeh, f., pishdar, m., & antuchevičienė, j. (2017). prioritization of petroleum supply chains’ disruption management strategies using combined framework of bsc approach, fuzzy ahp and fuzzy choquet integral operator. journal of business economics and management, 18(5), 897–919. doi: https://doi.org/10.3846/16111699.2017.1380075 glickman, t. s., & white, s. c. (2006). security, visibility and resilience: the keys to mitigating supply chain vulnerabilities. international journal of logistics systems and management, 2(2), 107-119. doi: https://doi.org/10.1504/ijlsm.2006.009554 grace, c. (2004). the effect of changing intellectual property on pharmaceutical industry prospects in india and china. dfid health systems resource centre, 1-68. gray, b. (1991). collaborating: finding common ground for multiparty problems, 1st edition. san francisco, usa: jossey-bass. gunasekaran, a. (1999). agile manufacturing: a framework for research and development. international journal of production economics, 62(1-2), 87-105. haimes, y. y. (2006). on the definition of vulnerabilities in measuring risks to infrastructures. risk analysis: an international journal, 26(2), 293-296. doi: https://doi.org/10.1111/j.1539-6924.2006.00755.x ijahp article: ganguly, kumar /evaluating supply chain resiliency strategies in the indian pharmaceutical sector: a fuzzy analytical hierarchy process (f-ahp) approach international journal of the analytic hierarchy process 175 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.620 hamel, g., & välikangas, l. (2003). the quest for resilience. harvard business review, 81(9), 52-65. ho, d., newell, g., & walker, a. (2005). the importance of property-specific attributes in assessing cbd office building quality. journal of property investment & finance, 23(5), 424-444. doi: https://doi.org/10.1108/14635780510616025 ho, w., zheng, t., yildiz, h., & talluri, s. (2015). supply chain risk management: a literature review. international journal of production research, 53(16), 5031–5069. doi: https://doi.org/10.1080/00207543.2015.1030467 holling, c. s. (1973). resilience and stability of ecological systems. annual review of ecology and systematics, 4, 1-23. hosseini, s. & khaled, a. a. (2016). a hybrid ensemble and ahp approach for resilient supplier selection. journal of intelligent manufacturing, 30(1), 2017-228. doi: https://doi.org/10.1007/s10845-016-1241-y hughes, w. r. (2009). a statistical framework for strategic decision making with ahp: probability assessment and bayesian revision. omega, 37(2), 463-470. doi: https://doi.org/10.1016/j.omega.2007.07.002 junior, f. r. l., osiro, l., & carpinetti, l. c. r. (2014). a comparison between fuzzy ahp and fuzzy topsis methods to supplier selection. applied soft computing, 21, 194209. doi: https://doi.org/10.1016/j.asoc.2014.03.014 jüttner, u., & maklan, s. (2011). supply chain resilience in the global financial crisis: an empirical study. supply chain management: an international journal, 16(4), 246-259. doi: https://doi.org/10.1108/13598541111139062 kahraman, c., cebeci, u., & ulukan, z. (2003). multi-criteria supplier selection using fuzzy ahp. logistics information management, 16(6), 382–394. kahraman, c. (2018). a brief literature review for fuzzy ahp. international journal of the analytic hierarchy process, 10(2), 293-297. doi: https://doi.org/10.1108/09576050310503367 kale, d., & little, s. (2007). from imitation to innovation: the evolution of r&d capabilities and learning processes in the indian pharmaceutical industry. technology analysis & strategic management, 19(5), 89-609. doi: https://doi.org/10.1080/09537320701521317 kar, a. k., & k. pani, a. (2014). exploring the importance of different supplier selection criteria. management research review, 37(1), 89-105. doi: https://doi.org/10.1108/mrr10-2012-0230 kilincci, o., & onal, s. a. (2011). fuzzy ahp approach for supplier selection in a washing machine company. expert systems with applications, 38(8), 9656-9664. doi: https://doi.org/10.1016/j.eswa.2011.01.159 kochan, c. g., & nowicki, d. r. (2018). supply chain resilience: a systematic literature review and typological framework. international journal of physical distribution & ijahp article: ganguly, kumar /evaluating supply chain resiliency strategies in the indian pharmaceutical sector: a fuzzy analytical hierarchy process (f-ahp) approach international journal of the analytic hierarchy process 176 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.620 logistics management, 48(8), 842-865. doi: https://doi.org/10.1108/ijpdlm-02-20170099 kong, x y., & li, y x. (2008). creating the resilient supply chain: the role of knowledge management resources. paper presented at the 4th international conference on wireless communications, networking and mobile computing (wicom'08), 12 17 october, dalian, china. doi: https://doi.org/10.1109/wicom.2008.1617 kraft, t., valdés, l., & zheng, y. (2018). supply chain visibility and social responsibility: investigating consumers’ behaviors and motives. manufacturing & service operations management, 20(4), 617-636. doi: https://doi.org/10.1287/msom.2017.0685 leat, p., & revoredo-giha, c. (2013). risk and resilience in agri-food supply chains: the case of the asda porklink supply chain in scotland. supply chain management: an international journal, 18(2), 219-231. doi: https://doi.org/10.1108/13598541311318845 leung, l. c., & cao, d. (2000a). on consistency and ranking of alternatives in fuzzy ahp. european journal of operational research, 124(1), 102–113. doi: https://doi.org/10.1016/s0377-2217(99)00118-6 leung, l. c., & cao, d. (2000b). on consistency and ranking of alternatives in fuzzy ahp. european journal of operational research, 124(1), 102-113. longo, f., and oren, t. (2008). supply chain vulnerability and resilience: a state of the art overview. proceedings of european modeling & simulation symposium, edited by campora s. giovanni, september 17–19, italy. lópez, c., & ishizaka, a. (2017). a hybrid fcm-ahp approach to predict impacts of offshore outsourcing location decisions on supply chain resilience. journal of business research, (accepted, in press). doi: 10.1016/j.jbusres.2017.09.050 mangla, s. k., kumar, p., & barua, m. k. (2015). risk analysis in green supply chain using fuzzy ahp approach: a case study. resources, conservation and recycling, 104, 375–390. doi: https://doi.org/10.1016/j.resconrec.2015.01.001 manuj, i., & mentzer, j. (2008). global supply chain risk management. journal of business logistics, 29(1), 133–155. doi: https://doi.org/10.1002/j.21581592.2008.tb00072.x mardani, a., jusoh, a., nor, k., khalifah, z., zakwan, n., & valipour, a. (2015). multiple criteria decision-making techniques and their applications–a review of the literature from 2000 to 2014. economic research-ekonomska istraživanja, 28(1), 516– 571. doi: https://doi.org/10.1080/1331677x.2015.1075139 mascaritolo, j., & holcomb, m. c. (2008). moving towards a resilient supply chain. journal of transportation management, 19(2), 71-83. mckinnon, a. (2014). building supply chain resilience: a review of challenges and strategies report for the joint apec – itf supply chain resilience workshop. christchurch, new zealand. doi: https://doi.org/10.1787/5jrw2z6nnxlq-en https://doi.org/10.1287/msom.2017.0685 ijahp article: ganguly, kumar /evaluating supply chain resiliency strategies in the indian pharmaceutical sector: a fuzzy analytical hierarchy process (f-ahp) approach international journal of the analytic hierarchy process 177 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.620 min, s., roath, a. s., daugherty, p. j., genchev, s. e., chen, h., arndt, a. d., & richey, r. g. (2005). supply chain collaboration: what's happening? international journal of logistics management, 16(2), 237-256. doi: https://doi.org/10.1108/09574090510634539 mital, m., del guidice, m., & papa, a. (2018). comparing supply chain risks for multiple product categories with cognitive mapping and analytic hierarchy process. technological forecasting and social change, 131, 159-170. doi: https://doi.org/10.1016/j.techfore.2017.05.036 narayana, s. a., pati, r. k., & vrat, p. (2014). managerial research on the pharmaceutical supply chain–a critical review and some insights for future directions. journal of purchasing and supply management, 20(1), 18-40. doi: https://doi.org/10.1016/j.pursup.2013.09.001 nieto-morote, a., & ruz-vila, f. (2011). a fuzzy approach to construction project risk assessment. international journal of project management, 29(2), 220-231. doi: https://doi.org/10.1016/j.ijproman.2010.02.002 ole-hohenstein, n., feisel, e., hartmann, e., & giunipero, l. (2015). research on the phenomenon of supply chain resilience: a systematic review and paths for further investigation. international journal of physical distribution & logistics management, 45(1/2), 90-117. doi: https://doi.org/10.1108/ijpdlm-05-2013-0128 özdağoğlu, a., & özdağoğlu, g. (2007). comparison of ahp and fuzzy ahp for the multi-criteria decision making processes with linguistic evaluations. istanbul commerce university journal of science and technology, 6(11), 65-85 parkouhi, s. v., & ghadikolaei, a. s. (2017). a resilience approach for supplier selection: using fuzzy analytic network process and grey vikor techniques. journal of cleaner production, 161, 431-451. doi: https://doi.org/10.1016/j.jclepro.2017.04.175 patil, s. k., & kant, r. (2014). a fuzzy ahp-topsis framework for ranking the solutions of knowledge management adoption in supply chain to overcome its barriers. expert systems with applications, 41(2), 679–693. doi: https://doi.org/10.1016/j.eswa.2013.07.093 peck, h. (2005). drivers of supply chain vulnerability: an integrated framework. international journal of physical distribution & logistics management, 35(4), 210-232. doi: https://doi.org/10.1108/09600030510599904 pettit, t. j., fiksel, j., & croxton, k. l. (2010). ensuring supply chain resilience: development of a conceptual framework. journal of business logistics, 31(1), 1-21. doi: https://doi.org/10.1111/jbl.12009 pettit, t. j., croxton, k. l., & fiksel, j. (2013). ensuring supply chain resilience: development and implementation of an assessment tool. journal of business logistics, 34(1), 46-76. doi: https://doi.org/10.1111/jbl.12009 plebankiewicz, e., & kubek, d. (2015). multicriteria selection of the building material supplier using ahp and fuzzy ahp. journal of construction engineering and ijahp article: ganguly, kumar /evaluating supply chain resiliency strategies in the indian pharmaceutical sector: a fuzzy analytical hierarchy process (f-ahp) approach international journal of the analytic hierarchy process 178 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.620 management, 142(1), 04015057. doi: https://doi.org/10.1061/(asce)co.19437862.0001033 ponis, s. t., & koronis, e. (2012). supply chain resilience: definition of concept and its formative elements. the journal of applied business research, 28(5), 921-930. doi: https://doi.org/10.19030/jabr.v28i5.7234 ponomarov, s. y., & holcomb, m. c. (2009). understanding the concept of supply chain resilience. the international journal of logistics management, 20(1), 124-143. doi: https://doi.org/10.1108/09574090910954873 purvis, l., spall, s., naim, m., & spiegler, v. (2016). developing a resilient supply chain strategy during ‘boom’ and ‘bust'. production planning & control, 27(7-8), 579590. doi: https://doi.org/10.1080/09537287.2016.1165306 rahman, m., & haldar, s. (2018). the application of ahp method for supplier selection of bangladesh’s jute industry: from the perspective of both public and private sectors international journal of the analytic hierarchy process, 10(2), 162-177. doi: https://doi.org/10.13033/ijahp.v10i2.508 rajesh, r., & ravi, v. (2015). supplier selection in resilient supply chains: a grey relational analysis approach. journal of cleaner production, 86, 343-359. doi: https://doi.org/10.1016/j.jclepro.2014.08.054 rathore, r., thakkar, j. j., & jha, j. k. (2017). a quantitative risk assessment methodology and evaluation of food supply chain. the international journal of logistics management, 28(4), 1272–1293. doi: https://doi.org/10.1108/ijlm-08-2016-0198 rice, j. b., & caniato, f. (2003). building a secure and resilient supply network. supply chain management review, 7(5), 22-30. saaty, t. l. (1980). the analytical hierarchy process, planning, priority. resource allocation. pittsburgh, pa, usa. saaty, t. l. (1990). how to make a decision: the analytic hierarchy process. european journal of operational research, 48(1), 9-26. doi: https://doi.org/10.1016/03772217(90)90057-i saaty, t. l. (1994). how to make a decision: the analytic hierarchy process. interfaces, 24(6), 19-43. doi: https://doi.org/10.1287/inte.24.6.19 saranga, h. (2007). multiple objective data envelopment analysis as applied to the indian pharmaceutical industry. journal of the operational research society, 58(11), 14801493. doi: https://doi.org/10.1057/palgrave.jors.2602280 scholten, k., sharkey, p., & fynes, s. b. (2014). mitigation processes – antecedents for building supply chain resilience. supply chain management: an international journal, 19(2), 211-228. scholten, k., & schilder, s. (2015). the role of collaboration in supply chain resilience. supply chain management: an international journal, 20(4), 471-484. doi: https://doi.org/10.1108/scm-06-2013-0191 ijahp article: ganguly, kumar /evaluating supply chain resiliency strategies in the indian pharmaceutical sector: a fuzzy analytical hierarchy process (f-ahp) approach international journal of the analytic hierarchy process 179 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.620 sheffi, y., & rice jr., j. b. (2005). a supply chain view of the resilient enterprise. mit sloan management review, 41(1), 41-48. shuai, y., wang, x., & zhao, l. (2011). research on measuring method of supply chain resilience based on biological cell elasticity theory. in 2011 ieee international conference on industrial engineering and engineering management (ieem), december 6 – 9, singapore, 264-268. doi: https://doi.org/10.1109/ieem.2011.6117919 singh, a., & prasher, a. (2019). measuring healthcare service quality from patients’ perspective: using fuzzy ahp application. total quality management & business excellence, 30(3-4), 284-300. doi: https://doi.org/10.1080/14783363.2017.1302794 soni, u., jain, v., & kumar, s. (2014). measuring supply chain resilience using a deterministic modeling approach. computers & industrial engineering, 74, 11-25. doi: https://doi.org/10.1016/j.cie.2014.04.019 stoltz, p. g. (2004). building resilience for uncertain times. hoboken, nj, usa: wiley. tadic, d., gumus, a. t., arsovski, s., aleksic, a., & stefanovic, m. (2013). an evaluation of quality goals by using fuzzy ahp and fuzzy topsis methodology. journal of intelligent & fuzzy systems, 25(3), 547-556. doi: https://doi.org/10.1007/978-3-31917906-3_22 tang, c. s. (2006). robust strategies for mitigating supply chain disruptions. international journal of logistics: research and applications, 9(1), 33-45. doi: https://doi.org/10.1080/13675560500405584 tang, c., and b. tomlin. (2008). the power of flexibility for mitigating supply chain risks. international journal of production economics, 116(1), 12–27. tang, y. c., & lin, t. w. (2011). application of the fuzzy analytic hierarchy process to the lead-free equipment selection decision. international journal of business and systems research, 5(1), 35. doi: https://doi.org/10.1504/ijbsr.2011.037289 tomlin, b. (2006). on the value of mitigation and contingency strategies for managing supply chain disruption risks. management science 52(5), 639–657. doi: https://doi.org/10.1287/mnsc.1060.0515 triantaphyllou, e., & lin, c. t. (1996). development and evaluation of five fuzzy multiattribute decision-making methods. international journal of approximate reasoning, 14(4), 281-310. doi: https://doi.org/10.1016/0888-613x(95)00119-2 tukamuhabwa, b. r., stevenson, m., busby, j., & zorzini, m. (2015). supply chain resilience: definition, review and theoretical foundations for further study. international journal of production research, 53(18), 5592-5623. doi: https://doi.org/10.1080/00207543.2015.1037934 urciuoli, l., mohanty, s., hintsa, j., & boekesteijn, e. g. (2014). the resilience of energy supply chains: a multiple case study approach on oil and gas supply chains to europe. supply chain management: an international journal, 19(1), 46-63. doi: https://doi.org/10.1108/scm-09-2012-0307 https://doi.org/10.1007/978-3-319-17906-3_22 https://doi.org/10.1007/978-3-319-17906-3_22 ijahp article: ganguly, kumar /evaluating supply chain resiliency strategies in the indian pharmaceutical sector: a fuzzy analytical hierarchy process (f-ahp) approach international journal of the analytic hierarchy process 180 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.620 vugrin, e. d., warren, d. e., & ehlen, m. a. (2011). a resilience assessment framework for infrastructure and economic systems: quantitative and qualitative resilience analysis of petrochemical supply chains to a hurricane. process safety progress, 30(3), 280-290. doi: https://doi.org/10.1002/prs.10437 wang, t.-k., zhang, q., chong, h.-y., & wang, x. (2017). integrated supplier selection framework in a resilient construction supply chain: an approach via analytic hierarchy process (ahp) and grey relational analysis (gra). sustainability, 9(2), 289. doi: https://doi.org/10.3390/su9020289 wedley, w. c. (1993). consistency prediction for incomplete ahp matrices. mathematical and computer modelling, 17(4-5), 151-161. doi: https://doi.org/10.1016/0895-7177(93)90183-y wieland, a., & marcus wallenburg, c. (2013). the influence of relational competencies on supply chain resilience: a relational view. international journal of physical distribution & logistics management, 43(4), 300-320. doi: https://doi.org/10.1108/ijpdlm-08-2012-0243 williams, b. d., roh, j., tokar, t., & swink, m. (2013). leveraging supply chain visibility for responsiveness: the moderating role of internal integration. journal of operations management, 31(7), 543-554. doi: https://doi.org/10.1016/j.jom.2013.09.003 yadav, salik r., nishikant mishra, vikas kumar, and m. k. tiwari. (2011). a framework for designing robust supply chains considering product development issues. international journal of production research, 49(20), 6065-6088. doi: https://doi.org/10.1080/00207543.2010.524258 yadav, v., & sharma, m. k. (2015). multi-criteria decision making for supplier selection using fuzzy ahp approach. benchmarking: an international journal, 22(6), 1158-1174. doi: https://doi.org/10.1108/bij-04-2014-0036 zhang, d., dadkhah, p. & ekwall, d. (2011). how robustness and resilience support security business against antagonistic threats in transport network. journal of transportation security, 4(3), 201–219. doi: https://doi.org/10.1007/s12198-011-0067-2 ijahp article: cebi, ilbahar/warehouse risk assessment using interval valued intuitionistic fuzzy ahp international journal of the analytic hierarchy process 243 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.549 warehouse risk assessment using interval valued intuitionistic fuzzy ahp selcuk cebi ,1 department of industrial engineering, yildiz technical university, besiktas 34349, istanbul, turkey scebi@yildiz.edu.tr esra ilbahar department of industrial engineering, istanbul technical university, macka 34367, istanbul, turkey esrailbahar@gmail.com abstract warehouses are essential components of a supply chain management system. any malfunction in warehouses may have an important effect on the subsequent operations of a firm. for example, an accident that occurs in a warehouse can damage the operations of the company as well as the health of the employees. in this paper, critical operations at a warehouse are considered in terms of occupational health and safety. the originality of the paper is that the proposed approach provides a new perspective on warehouse risk assessment. it is the first time a method involving interval valued intuitionistic fuzzy analytical hierarchy process (ivif-ahp) and a new risk assessment table is utilized to evaluate and categorize the risks in warehouses. as a result of the proposed risk assessment method, hazards in warehouses are categorized as insignificant, marginal, or catastrophic. keywords: warehouse risk management; risk assessment; risk analysis; interval valued intuitionistic fuzzy sets; analytical hierarchy process 1. introduction warehouses are crucial components of a supply chain management system, and efficiently functioning warehouses enable companies to carry out their operations smoothly. since any malfunction in warehouses may influence the subsequent operations of a firm, an accident that occurs in the warehouse can damage the operations of the company as well as the health of the employees. thus, it is important to identify hazards in a warehouse and the risks associated with them in order to be able to prevent potential accidents in warehouses. a risk is defined as a combination of the probability of occurrence of an undesired situation and the severity of that situation. risk assessment consists of an analysis and evaluation process of the risks associated with hazards in operations, and should be done systematically. the main objective in the risk assessment process is to rank the risky 1 corresponding author mailto:scebi@yildiz.edu.tr ijahp article: cebi, ilbahar/warehouse risk assessment using interval valued intuitionistic fuzzy ahp international journal of the analytic hierarchy process 244 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.549 factors in the operations in order to provide risk elimination or reduction by applying protective measures from the most important risk to the least important risk. the first step of a risk assessment is hazard identification. the following steps in the assessment are risk identification, risk analysis, determination of precautions, and risk review, respectively. the determination of precautions is performed based on the risk magnitude obtained from the risk analysis step. in general, risk analysis methods use categorical scales and these scales are inadequate for the evaluation. for instance, let us assume that probability of occurrence and severity are 3 and 4 for case a, 3 and 5 for case b, and 4 and 4 case c, respectively. if we use an l-type risk analysis method, risk magnitudes of these cases are obtained as 12, 15, and 16, respectively. based on the risk evaluation procedure of an l-type matrix method, these results are categorized as high, very high and very high, respectively. sometimes, a mere one point increase on any parameter may change the category of the risk. furthermore, there are some gaps in the risk magnitude scale of the conventional methods since they use categorical data. for instance, in an l-type risk matrix, the scores such as 5, 7, 11, 13, 14, [17, 19], [21, 24] are not obtained by producing any probability of occurrence or severity of occurrence. therefore, in some cases, the sensitivity of the conventional methods to small changes can be pointless. in the scope of this study, a new approach is proposed in order to overcome this deficiency. in this study, after a wide literature review hazards in a warehouse are identified and grouped together to form a hierarchy. then, interval valued intuitionistic fuzzy analytical hierarchy process (ivif-ahp) is utilized to obtain the weights of risks based on probability of occurrence and the severity in the warehouse. then, these weights are used to assess risks in warehouse operations. in this study, since the evaluation process is based on the expert’s judgments and the collected data for the risk severity and probability of occurrence are in linguistic form, fuzzy logic is required for the calculation of risk magnitudes. intuitionistic fuzzy sets are preferred because of their ability to better represent the expert’s opinions by using both membership and non-membership values. the preventive measures that should be taken first, the operations that should be stopped immediately and the processes that should be monitored are determined according to the results of the analysis. the originality of the paper is that the proposed approach presents a new perspective to risk evaluation in terms of occupational health and safety, and it is the first time an ivif-ahp is applied to a risk evaluation procedure. 2. literature review there have been several studies related to hazards in warehouses throughout the literature. forklift accidents were examined by larsson and rechnitzer (1994) and saric et al. (2013). in these studies, it was emphasized that a foot run over by a loaded forklift is a lot more dangerous than expected. it can cause severe trauma and fractures. moreover, since they operate in confined spaces and make maneuvers within narrow aisles, forklifts can be dangerous for the pedestrians nearby. one of the most common forklift accidents involves a forklift striking a pedestrian and possibly leading to severe trauma or death. some other forklift accidents include falling from/by forklift, collisions, overturns, and sudden stops (larsson & rechnitzer, 1994; saric et al., 2013). ren (2012) investigated logistics warehouse fire risk by using the analytic hierarchy process. in the evaluation process, warehouse fire risk was examined under the four categories of warehouse building, goods, management, and environment. tyldesley et ijahp article: cebi, ilbahar/warehouse risk assessment using interval valued intuitionistic fuzzy ahp international journal of the analytic hierarchy process 245 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.549 al. (2004) analyzed the benefits of fire compartmentation in chemical warehouses. moreover, the cost-effectiveness of fire compartmentation with respect to other fire protection measures was emphasized. markert (1998) evaluated the mitigation of the consequences of fires in chemical warehouses. in the study, the performances of four fire testing methods were compared to evaluate the toxicity of fire products. moreover, the impact of the release of unburned pesticides and their pyrolysis products on human health was assessed. qin et al. (2016) examined the feasibility of natural smoke extraction in warehouse buildings whereas miles and cox (1996) utilized a computational fluid dynamics model in order to assess hazards related to warehouse fires. basahel (2015) examined work-related musculoskeletal disorders (msds) in warehouse workers. in the study, lifting and pulling activities in supermarket warehouses was analyzed by using rapid upper limb assessment and a pain selfreport chart to reveal the relation between these activities and lower back, shoulder and lower arm pain. furthermore, the heart rates of workers were continuously recorded to assess the physiological stresses of the activity postures. it was concluded that lifting activity significantly affects low back pain. bouloiz et al. (2013) proposed a system dynamics model to show causal interdependencies among technical, organizational and human related safety factors. a behavioral analysis of safety conditions in a chemical storage unit was conducted by examining different scenarios through vensim software in order to enhance the safety of the system. 3. method in this section, a new approach is proposed to the literature in order to eliminate or reduce the magnitude of any risk in warehouses. the steps of the procedure are as follows: phase 1. planning. in the scope of this phase, the following activities are executed. step 1.1. construction of a risk assessment team. since most of the systems are complex, a risk assessment team whose members have technical expertise or extensive knowledge on the activities at the corresponding working environment has to be constructed. furthermore, the team should consist of members who are familiar with risk analysis methods. step 1.2. definition of data collection procedure. in this step, plans are made about how to gather information and measure information output on a work environment and tasks are allocated to the team members. step 1.3. definition of hazard sources. potential sources of harm or hazardous situations are identified. in this respect, all potential sources of danger including human related, hardware or machine related problems, and systematic errors should be inspected in the hazard identification process. step 1.4. definition of potential risks. one attempts to foresee all risks that are a possibility from any potential source. phase 2. analysis. in this phase, the risk magnitude is obtained based on the probability of occurrence and the severity. step 2.1. determining probability and severity weights. the main purpose of this step is to predict the probability of an undesired event causing the risks identified above. for this, the past statistics or expert views have been widely used in the literature. in particular, expert views are preferred when past statistics are not available. however, expert preferences are not precise and they include uncertainty and vagueness. hence, ijahp article: cebi, ilbahar/warehouse risk assessment using interval valued intuitionistic fuzzy ahp international journal of the analytic hierarchy process 246 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.549 in this study, the probability and severity of the undesired events are obtained by using ivif-ahp. the steps of ivif-ahp are as follows (wu et al., 2013): step 2.1.1. pairwise comparisons of hazards with respect to probability and severity are obtained by using the linguistic terms given in table 1 and these linguistic terms are converted their corresponding interval valued intuitionistic fuzzy numbers. step 2.1.2. from this pairwise comparison, score judgment matrix, �̃�𝑖𝑗 , is obtained by using equation 1. �̃�𝑖𝑗 = �̃�𝑖𝑗 − �̃�𝑖𝑗 = [𝜇𝑖𝑗 𝐿 − 𝜗𝑖𝑗 𝑈 , 𝜇𝑖𝑗 𝑈 − 𝜗𝑖𝑗 𝐿 ], for all i, j=1, 2, . . . , n (1) step 2.1.3. then, interval multiplicative matrix, �̃�, is obtained by using equation 2. �̃� = 10�̃� (2) step 2.1.4. the priority vector of interval multiplicative matrix is calculated as follows: �̃�𝑖 = ∑ �̃�𝑖𝑗 𝑛 𝑗=1 ∑ ∑ �̃�𝑖𝑗 𝑛 𝑗=1 𝑛 𝑖=1 = [ ∑ �̃�𝑖𝑗 − 𝑛𝑗=1 ∑ ∑ �̃�𝑖𝑗 +𝑛 𝑗=1 𝑛 𝑖=1 , ∑ �̃�𝑖𝑗 + 𝑛𝑗=1 ∑ ∑ �̃�𝑖𝑗 −𝑛 𝑗=1 𝑛 𝑖=1 ] (3) step 2.1.5. as �̃�𝑖 (𝑖 ∈ n) are interval numbers, comparison of each �̃�𝑖 with others is made and possibility degree matrix, 𝑃 = (𝑝𝑖𝑗 )𝑛𝑥𝑛, is constructed by using equation 4. let 𝑎 = [𝑎−, 𝑎+] and 𝑏 = [𝑏−, 𝑏+] be interval numbers. the possibility degree of 𝑎 ≥ 𝑏 is as follows: 𝑝(𝑎 ≥ 𝑏) = min {𝐿𝑎+𝐿𝑏,max(𝑎 +−𝑏−,0)} 𝐿𝑎+𝐿𝑏 (4) where 𝐿𝑎 = 𝑎 + − 𝑎− and 𝐿𝑏 = 𝑏 + − 𝑏−. step 2.1.6. prioritization of 𝑃 = (𝑝𝑖𝑗 )𝑛𝑥𝑛 by using equation 5 to obtain weights and rank. 𝑤𝑖 = 1 𝑛 [∑ 𝑝𝑖𝑗 𝑛 𝑗=1 + 𝑛 2 − 1] (5) table 1 linguistic scale for evaluation linguistic terms [µl, µu] [𝒗l, 𝒗u] vh [0.6, 0.8] [0, 0.2] h [0.45, 0.65] [0.15, 0.35] e [0.3, 0.5] [0.3, 0.5] l [0.15, 0.35] [0.45, 0.65] vl [0, 0.2] [0.6, 0.8] ijahp article: cebi, ilbahar/warehouse risk assessment using interval valued intuitionistic fuzzy ahp international journal of the analytic hierarchy process 247 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.549 step 2.2. determination of risk category. to determine the risk category, the chart given in table 2 is utilized based on the values obtained in step 2.1. table 2 risk assessment matrix severity ratio for a risk: wrs likelihood ratio for a risk: wrp wrs≥0.08 0.08>wrs≥0.06 0.06>wrs≥0.04 0.04>wrs≥ 0.02 0.02>wrs ≥0 0.08wrp category i category i category i category ii category iii 0.06wrp<0.08 category i category i category ii category ii category iii 0.04wrp<0.06 category i category ii category ii category iii category iii 0.02wrp<0.04 category ii category ii category iii category iii category iii 0.0wrp<0.02 category iii category iii category iii category iii category iii the interpretations of the categories given in table 2 are as follows: category iii: insignificant. the risk may occur but it is not possible for it to result in any damage or injury. category ii: marginal. the risk has a great potential to constitute a threat to injury or to result in a failure of the system. category i: catastrophic. the risk is quite high and has a great potential to cause serious damage or failure of the system. phase 3. risk evaluation and reduction. in this step, risk reduction, validation analysis, and an audit are conducted. based on the risk magnitude, a set of control measurements are utilized in order to provide risk reduction. for the control measurements, one of the following strategies is preferred: i. elimination of the hazard: the hazard source is omitted from the system in order to remove the hazard from the workplace. in order to achieve this, the process type should be changed. it is difficult to execute this strategy. ii. substitution of the hazard: in this strategy, the substitute of the current process which includes less risk is preferred. iii. hazard control: this strategy includes engineering applications, managerial applications, and personal protective equipment substrategies and these strategies are applied to reduce the current risk. 4. analysis in the literature, some of the potential hazards in warehouse operations are identified as failure to conduct conveyor belt maintenance, vehicle exhaust fumes, not providing personal protective equipment to employees, falls from a height, slip and trips, undetermined forklift road lines, use of forklifts by employees without a driver’s license, potential explosion due to recharging of forklift truck batteries, overloading the forklift, and improper loading / unloading postures and movements of employees (larsson & rechnitzer, 1994; saric et al., 2013; example risk assessment for a warehouse, 2007; basahel, 2015). hazards in a warehouse are classified in table 3, and then, the pairwise comparisons of these hazards are performed. ijahp article: cebi, ilbahar/warehouse risk assessment using interval valued intuitionistic fuzzy ahp international journal of the analytic hierarchy process 248 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.549 table 3 classification of hazards for warehouses h1 physical and chemical hazards h11 lighting h12 air circulation (for vehicle exhaust fumes) h13 loud sounds h14 temperature h15 chemical materials h2 mechanical hazards h21 failure to conduct conveyor belt maintenance h22 failure to conduct forklift maintenance (refueling and periodic maintenance) h23 overloading the forklift h24 recharging of forklift truck batteries – potential explosion by release of hydrogen, spillage of acid h3 workplace hazards h31 falls from a height (working height) h32 wet floors (slips, trips, and falls) h33 undetermined forklift / pedestrian road lines h34 accidents caused by docks h35 improperly stored materials h4 human related hazards h41 not providing personal protective equipment to employees h42 use of forklifts by employees without driver’s license h43 stretching the body by lifting heavy objects h44 not following safety procedures for picking up, putting down and stacking loads h45 not following safety procedures for speed limits of forklift pairwise comparisons, obtained in the linguistic form for the hazards in warehouse, and corresponding consistency ratios are provided in tables 4-13. consistency ratios of the pairwise comparison matrices are calculated as in the classical ahp method by using the crisp numbers corresponding to these linguistic terms. ijahp article: cebi, ilbahar/warehouse risk assessment using interval valued intuitionistic fuzzy ahp international journal of the analytic hierarchy process 249 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.549 table 4 pairwise comparison of main hazards for probability hazards h1 h2 h3 h4 h1 e vh h l h2 vl e l vl h3 l h e l h4 h vh h e cr 0.074 table 5 pairwise comparison of main hazards for severity hazards h1 h2 h3 h4 h1 e vl l vl h2 vh e e vl h3 h e e vl h4 vh vh vh e cr 0.100 table 6 pairwise comparison of sub-hazards of h1 for probability h1 h11 h12 h13 h14 h15 h11 e l l h vl h12 h e h vh l h13 h l e h l h14 l vl l e vl h15 vh h h vh e cr 0.072 table 7 pairwise comparison of sub-hazards of h1 for severity h1 h11 h12 h13 h14 h15 h11 e l l h vl h12 h e h vh l h13 h l e h l h14 l vl l e vl h15 vh h h vh e cr 0.072 table 8 pairwise comparison of sub-hazards of h2 for probability h2 h21 h22 h23 h24 h21 e l vl h h22 h e l h h23 vh h e vh h24 l l vl e cr 0.076 table 9 pairwise comparison of sub-hazards of h2 for severity h2 h21 h22 h23 h24 h21 e l h vl h22 h e h l h23 l l e vl h24 vh h vh e cr 0.076 table 10 pairwise comparison of sub-hazards of h3 for probability h3 h31 h32 h33 h34 h35 h31 e l vl vl l h32 h e l vl h h33 vh h e l h h34 vh vh h e vh h35 h l l vl e cr 0.081 table 11 pairwise comparison of sub-hazards of h3 for severity h3 h31 h32 h33 h34 h35 h31 e vh h vh l h32 vl e l l vl h33 l h e h l h34 vl h l e vl h35 h vh h vh e cr 0.081 ijahp article: cebi, ilbahar/warehouse risk assessment using interval valued intuitionistic fuzzy ahp international journal of the analytic hierarchy process 250 vol. 10 issue 2 2018 issn1936-6744 https://doi.org/10.13033/ijahp.v10i2.549 table 12 pairwise comparison of sub-hazards of h4 for probability h4 h41 h42 h43 h44 h45 h41 e h l vl l h42 l e vl vl l h43 h vh e l h h44 vh vh h e h h45 h h l l e cr 0.072 table 13 pairwise comparison of sub-hazards of h4 for severity h4 h41 h42 h43 h44 h45 h41 e h vh vh h h42 l e h vh h h43 vl l e h l h44 vl vl l e l h45 l l h h e cr 0.072 the probability weights and severity weights of these hazards are given in tables 14 and 15. then, a risk assessment is made by using table 2 and categories of hazards are provided in table 16. table 14 probability weights h1 0.295 h2 0.161 h3 0.230 h4 0.314 h11 0.051 h21 0.036 h31 0.031 h41 0.054 h12 0.070 h22 0.043 h32 0.045 h42 0.042 h13 0.059 h23 0.054 h33 0.054 h43 0.074 h14 0.039 h24 0.028 h34 0.061 h44 0.081 h15 0.076 h35 0.039 h45 0.063 table 15 severity weights h1 0.158 h2 0.270 h3 0.231 h4 0.341 h11 0.027 h21 0.061 h31 0.057 h41 0.088 h12 0.037 h22 0.072 h32 0.031 h42 0.080 h13 0.032 h23 0.047 h33 0.046 h43 0.059 h14 0.021 h24 0.091 h34 0.038 h44 0.046 h15 0.041 h35 0.059 h45 0.068 ijahp article: cebi, ilbahar/warehouse risk assessment using interval valued intuitionistic fuzzy ahp international journal of the analytic hierarchy process 251 vol. 10 issue 2 2018 issn1936-6744 https://doi.org/10.13033/ijahp.v10i2.549 table 16 results of risk assessment in warehouses h1 physical and chemical hazards h11 lighting category iii h12 air circulation category ii h13 loud sounds category iii h14 temperature category iii h15 chemical materials category ii h2 mechanical hazards h21 failure to conduct conveyor belt maintenance category ii h22 failure to conduct forklift maintenance (refueling and periodic maintenance) category ii h23 overloading the forklift category ii h24 recharging of forklift truck batteries – potential explosion by release of hydrogen, spillage of acid category ii h3 workplace hazards h31 falls from a height (working height) category iii h32 wet floors (slips, trips, and falls) category iii h33 undetermined forklift / pedestrian road lines category ii h34 accidents caused by docks category ii h35 improperly stored materials category iii h4 human related hazards h41 not providing personal protective equipment to employees category i h42 use of forklifts by employees without driver’s licence category i h43 stretching the body by lifting heavy objects category ii h44 not following safety procedures for picking up, putting down and stacking loads category i h45 not following safety procedures for speed limits of forklift category i as a result of the proposed assessment, the most critical hazards are identified as h41, not providing personal protective equipment to employees; h42, use of forklifts by employees without driver’s license; h44, not following safety procedures for picking up, putting down and stacking loads; and h45, not following safety procedures for speed limits of forklift. since these hazards are identified as catastrophic, operations or activities involving these hazards must be terminated immediately until these hazards are eliminated. marginal hazards in warehouses are identified as h12, air circulation; h15, chemical materials; h21, failure to conduct conveyor belt maintenance; h22, failure to conduct forklift maintenance; h23, overloading the forklift; h24, recharging of forklift truck batteries – potential explosion by release of hydrogen, spillage of acid; h33, undetermined forklift / pedestrian road lines; h34, accidents caused by docks; and h43, stretching the body by lifting heavy objects. since these hazards have a great potential to ijahp article: cebi, ilbahar/warehouse risk assessment using interval valued intuitionistic fuzzy ahp international journal of the analytic hierarchy process 252 vol. 10 issue 2 2018 issn1936-6744 https://doi.org/10.13033/ijahp.v10i2.549 constitute a threat in a warehouse, even if activities involving these hazards are not terminated, they must be constantly monitored. the rest of the hazards are identified as insignificant, indicating that they can be neglected. 5. conclusions warehouses are significant components of a supply chain management system. efficiently functioning warehouses facilitate the ability of companies to carry out their operations smoothly. an accident occurring in a warehouse can damage both the health of the employees and the operations of the company. therefore, identification of potential hazards in a warehouse and classification of these hazards are required to be able to take necessary actions. in this paper, a new approach in which hazards are categorized based on probability and severity weights calculated through ivif-ahp is proposed. the advantages of the proposed method can be explained as follows: (i) it includes an expert inference system to obtain risk magnitude. as distinct from the matrix type risk assessment method, this method presents robust and sufficient results for a safety specialist. (ii) the risk matrix utilized to obtain the risk magnitudes presents a sensitive and adequate scale. (iii) the results obtained from the analysis are more comprehensive than the conventional methods. in a work environment, some activities might be dependent on each other and the obtained risk magnitude for these activities can be affected by these interdependencies. therefore, in future research, the proposed method can be extended to evaluate the interdependencies among the risks in a work environment. ijahp article: cebi, ilbahar/warehouse risk assessment using interval valued intuitionistic fuzzy ahp international journal of the analytic hierarchy process 253 vol. 10 issue 2 2018 issn1936-6744 https://doi.org/10.13033/ijahp.v10i2.549 references basahel, a. m. (2015). investigation of work-related musculoskeletal disorders (msds) in warehouse workers in saudi arabia. procedia manufacturing, 3, 4643-4649. doi: https://doi.org/10.1016/j.promfg.2015.07.551 bouloiz, h., garbolino, e., tkiouat, m., & guarnieri, f. (2013). a system dynamics model for behavioral analysis of safety conditions in a chemical storage unit. safety science, 58, 32-40. doi: https://doi.org/10.1016/j.ssci.2013.02.013 example risk assessment for a warehouse. (2007). health and safety executive. retrieved from http://www.hse.gov.uk/risk/casestudies/pdf/warehouse.pdf. larsson, t. j., & rechnitzer, g. (1994). forklift trucks—analysis of severe and fatal occupational injuries, critical incidents and priorities for prevention. safety science, 17(4), 275-289. doi: https://doi.org/10.1016/0925-7535(94)90029-9 markert, f. (1998). assessment and mitigation of the consequences of fires in chemical warehouses. safety science, 30(1), 33-44. doi: https://doi.org/10.1016/s09257535(98)00031-9 miles, s. d., & cox, g. (1996). prediction of fire hazards associated with chemical warehouses. fire safety journal, 27(4), 265-287. doi: https://doi.org/10.1016/s03797112(96)00056-2 qin, y., huang, w., xiang, y., zhang, r., lu, p., & tan, x. (2016). feasibility analysis on natural smoke extraction for large space warehouse buildings. procedia engineering, 135, 495-500. doi: https://doi.org/10.1016/j.proeng.2016.01.161 ren, s. (2012). assessment on logistics warehouse fire risk based on analytic hierarchy process. procedia engineering, 45, 59-63. doi: https://doi.org/10.1016/j.proeng.2012.08.121 saric, s., bab-hadiashar, a., hoseinnezhad, r., & hocking, i. (2013). analysis of forklift accident trends within victorian industry (australia). safety science, 60, 176-184. doi: https://doi.org/10.1016/j.ssci.2013.07.017 tyldesley, a., rew, p. j., & houlding, r. c. (2004). benefits of fire compartmentation in chemical warehouses. process safety and environmental protection, 82(5), 331-340. doi: https://doi.org/10.1205/psep.82.5.331.44195 wu, j., huang, h. b., & cao, q. w. (2013). research on ahp with interval-valued intuitionistic fuzzy sets and its application in multi-criteria decision making problems. applied mathematical modelling, 37(24), 9898-9906. doi: https://doi.org/10.1016/j.apm.2013.05.035 http://www.hse.gov.uk/risk/casestudies/pdf/warehouse.pdf ijahp article: von solms/validity of the ahp/anp: comparing apples and oranges international journal of the analytic hierarchy process 2 vol. 3, issue 1, 2011 issn 1936-6744 validity of the ahp/anp: comparing apples and oranges sibs von solms mondi south africa division richards bay mill sibs.vonsolms@mondigroup.co.za abstract determining the validity of the ahp/anp is an important issue. this paper discusses the complex and often controversial field of validation. the debate regarding the validity of the ahp/anp is evaluated, from which the need for a different approach is identified. the ahp/anp is a decision-making methodology that should be located within a qualitative, subjectivist or constructivist paradigm where different measures of validating research become relevant. the goal of this paper is three-fold: first, to locate the ahp/anp within the field of interpretivist qualitative research; second, to argue for the importance of validating the ahp/anp in terms of criteria important to the decision makers rather than some objectively given or normatively defined criteria; third, to stimulate research specifically aimed at testing the validity of the ahp/anp in terms of decision makers’ criteria. keywords: consensus, constructivism, criticisms of the ahp/anp, group decision-making, multi-stakeholder groups, qualitative research, quantitative research, sociological paradigms, subjectivity, validity. 1. introduction at the recent isahp2009, the question was frequently raised as to how presenters were going to prove that their findings or conclusions were valid. the issue of research validity is a complex and often controversial subject [sechrest, 2005] involving both philosophical as well as practical aspects. the paper will take as its starting point the debate regarding the validity of several aspects of the ahp/anp, briefly highlighting both the arguments aimed at showing the ahp/anp to be an invalid method as well as those offered in defense of the ahp/anp including the validation examples discussed by whitaker [2004]. this debate is, however, not the focus of the current paper, but its evaluation leads to the identification of an aspect neglected by the extant literature, i.e. that the ahp/anp is an approach with specific strengths to enhance multi-stakeholder group decision-making. the traditional forms of research validity are tied closely to a quantitative/objectivist research paradigm and it is argued that different and more appropriate criteria are required for research and application within a qualitative/subjectivist paradigm [guba & lincoln, 1994]. applying the ahp/anp in the multi-stakeholder group decision-making field requires the formulation of a new set of validity criteria that are relevant to the problematic of such multi-stakeholder groups and the stimulation of research to test the validity of the ahp/anp as a multi-stakeholder group decision-making (gdm) methodology. traditional approaches emphasize the prediction of outcomes, but what we need are approaches that can assist decision makers to actively make things happen. human development requires action and action requires choice based on rob typewritten text rob typewritten text http://dx.doi.org/10.13033/ijahp.v3i1.60 ijahp article: von solms/validity of the ahp/anp: comparing apples and oranges international journal of the analytic hierarchy process 3 vol. 3, issue 1, 2011 issn 1936-6744 human interest. the validity of the ahp/anp hinges on its ability to support such choice. 2. the ahp validity problem 2.1 proving the invalidity of the ahp/anp a steady stream of papers have been written claiming the invalidity of some technical and mathematical aspects of the ahp/anp methodology. saaty [2008] says that essentially, there have been five main types of criticisms of the ahp all of which have been addressed in the literature (e.g. [saaty et al, 2009]), namely: (1) rank reversal: this is the concern with illegitimate changes in the ranks of the alternatives upon changing the structure of the decision. rank reversals are shown by critics to occur when using comparisons and relative measurement in two ways: first, when new alternatives are added or old ones deleted; and second, when new criteria are added or old ones deleted with the caveat that the priorities of the alternatives would be tied under these criteria and hence argued that the criteria should be irrelevant when ranking the alternatives. rank reversals that followed such structural changes were attributed to the use of relative measurement and normalization. papers relevant to this critique include [watson and freeling, 1982; belton and gear, 1983; dyer, 1990; barzilai and golany, 1994; pérez, 1995; finan and hurley, 2002; pérez et al, 2006; ishizaka and labib, 2009]. (2) consistency & aggregation: the concern is about inconsistent judgments and their effect on aggregating such judgments or on deriving priorities from them. a condition that may not hold with inconsistent judgments is pareto optimality. another condition also inherited from expected utility theory has to do with a relation called condition of order preservation (cop). a paper relevant to this critique is [bana e costa and vansnick, 2008]. (3) geometric mean & multiplicative weighing: the third criticism has to do with attempts to preserve rank from irrelevant alternatives by combining the comparison judgments of a single individual using the geometric mean (logarithmic least squares) to derive priorities and also combining the derived priorities on different criteria by using multiplicative weighting synthesis. papers relevant to this critique include [holder, 1990; van den honert and lootsma, 1996; barzilai, 1999, forthcoming]. (4) fundamental scale: the fourth criticism has to do with people trying to change the fundamental scale. papers relevant to this critique include [ma and zheng, 1991; salo and hämäläinen, 1997; leskinen, 2000]. (5) behavioral aspects of pairwise comparisons: the fifth and final criticism has to do with whether or not the pairwise comparisons axioms are behavioral and spontaneous in nature to provide judgments. 2.2 proving the validity of the ahp/anp proponents demonstrate the validity of the ahp/anp in mainly three ways: (1) counter arguments: in reply to the criticisms listed above, several papers were written to answer and counter the comments made. references relevant to this point include [saaty, 1990, 1999, 2001, 2008, 2010; harker and vargas, 1990; vargas, ijahp article: von solms/validity of the ahp/anp: comparing apples and oranges international journal of the analytic hierarchy process 4 vol. 3, issue 1, 2011 issn 1936-6744 1997; forman and gass, 2001; wijnmalen, 2001; saaty and vargas, 2006; whitaker, 2007; saaty et al, 2009]. (2) applications: the fact that the ahp/anp is one of the most widely used mcdm methodologies is argued as proof of its validity. the point is made that if the ahp/anp was so flawed and invalid, applications would not have proliferated as they have because the results would not have been useful. one specific reference in this regard is [forman and gass, 2001]. (3) validation examples: whitaker [2004] presents thirteen research cases in which the validity of the ahp/anp was demonstrated. this specific approach to testing the validity of the ahp/anp requires a more detailed discussion. 2.3 validity examples whitaker [2004] points out that there are two ways in which the ahp/anp can be validated, i.e. as a predictive tool or as a decision-making tool. the former lends itself to fairly easy validation due to the availability of both a known of real world data and an ahp/anp predicted outcome for comparison. in the latter case, the success of a decision may not be known for some time and is, further, always a matter of interpretation, sometimes influenced by events happening that were – or could – not be known at the time of making the decision [whitaker, 2004]. an additional problem is that in the absence of a known correct outcome, even a decision that can be shown to be a success, may well be sub-optimal because it could be argued that another decision-making tool could have led to an outcome that may have outperformed the ahp/anp decision. whitaker [2004] presents 13 validation examples – 10 for ahp and 3 for anp – all falling in the first (predictive tool) category. this list is clearly only a small sample of many validation exercises done over the years but it can be assumed that it constitutes a fairly representative sample of such validation examples. this assumption is borne out when compared to the examples listed in [saaty, 1997]. whitaker [2004] states that the objectives of collecting theses examples are: (i) to promote the use of the ahp/anp theory as a valid theory; (ii) to show the wide variety of scales against which validation can be accomplished, and (iii) to collect a wide variety of validation examples in one place. this is both a worthy and commendable endeavour and invites an ongoing review of the principles of ahp/anp validation. 2.4 evaluating the proofs when evaluating the validity examples two points are important, points that can be linked to the split between predictive and decision-making tools. first, clearly, the ahp/anp is meant to be more than a tool to predict known outcomes – however, varied these may be. second, it is similarly clear that research on validating the ahp/anp as a decision-making tool – in the thorny world of subjective preferences, values, and perspectives where known outcomes are not available – is surely needed. this two-way split will be used here as a guide to presenting the evaluation of the validity examples. 2.4.1 predicting known outcomes and generalizibility validity is subdivided into discussions on internal and external validity where internal validity refers to whether the research conclusions are adequately supported by the data, and external validity refers to whether the findings of a given research project are generalizable to all similar cases [babbie, 2007]. ijahp article: von solms/validity of the ahp/anp: comparing apples and oranges international journal of the analytic hierarchy process 5 vol. 3, issue 1, 2011 issn 1936-6744 within the gdm research literature (e.g. [laughlin, 1996]) two types of tasks are identified. a task with a demonstrable correct answer is called an intellective task while a judgmental task is one without such a demonstrable correct answer. in many cases tasks are neither purely intellective nor purely judgmental, for example the ‘moon survival’ task which is intermediate between intellective and judgmental because the correct result exists (due to an expert panel rating), but although this correct result is available it cannot be demonstrated within the groups while in session. in [whitaker, 2004] the validation examples all have correct (known) outcomes but are not equivalent to intellectual tasks as the correctness cannot be demonstrated by the decision-makers until afterwards. this is important for the validation of the ahp/anp in that the gdm literature indicates, further, that the decision outcomes differ in terms of the type of task performed [gigone and hastie, 1997; maccoun, 1998]. kirchler and davis [1986] point out that for intellective tasks in gdm outcomes are normally a ‘truth wins’ situation, while in judgmental tasks this changes to ‘majority wins’. in the latter cases, if group member status is very varied, outcomes change further to become ‘power wins’. this has an effect on the generalizibility of the validation examples in two ways. first, the changes in outcome types indicate that generalizing from one task type to a domain including all task types may be suspect (cf [harper et al, 1992]). second, and more important, is the context within which the gdm debate takes place. the validation examples in [whitaker, 2004] all seem to have taken place in, what flood and jackson [1991] call, unitary contexts. the flood & jackson unitary-pluralist-conflictual continuum will be discussed below; now it is important to realize that in unitary contexts no pressure exists to argue for, or maintain allegiance to, specific values, preferences or views and it seems logical to expect ‘majority wins’ outcomes, while in the conflictual contexts pressure clearly exists to strongly side with a particular viewpoint and this could lead to a change to ‘power wins’ outcomes. applying the ahp/anp in multi-stakeholder gdm as envisaged here and elsewhere ([von solms and peniwati, 2001; von solms, 2009]) would seem to require validation over a wider front – including gdm in pluralist and conflictual contexts – over and above the unitary contexts covered in the extant literature (e.g. [saaty, 1997; whitaker, 2004]). 2.4.2 research: rigor vs relevance whereas the previous discussion related mainly to the validity examples, the current point is relevant to all three validity arguments – including the validity examples, the counter examples and the applications argument – and involves the question regarding the practical value of the ahp/anp as opposed to its formal mathematical foundation. forman and gass [2001] argue for the point that the many successful applications of the ahp/anp are proof of its validity. interestingly, many critics acknowledge the successful application of the ahp/anp before documenting their criticisms (e.g. [pérez, 1995; salo and hämäläinen, 1997; barzilai, 1999]). some critics explicitly state that their critique is aimed not at the value of the ahp/anp to decision makers, but specifically at mathematical issues of its underlying theory. for example, barzilai and golani [1994:57] say (italics in the original): “this is an axiomatic (i.e. mathematical) framework, which enables us to gain insight into problematic aspects of the underlying structure and to identify a consistent variant of the ahp. the framework does not address behavioural issues and is therefore independent of the way in which decision makers express their preferences.” in other cases the mathematical focus is implicit. it is clear, for example, that the first four criticisms mentioned in [saaty, 2008] are relevant to mathematical and ijahp article: von solms/validity of the ahp/anp: comparing apples and oranges international journal of the analytic hierarchy process 6 vol. 3, issue 1, 2011 issn 1936-6744 theoretical issues of the ahp/anp. it is, therefore, natural to find that the responses (e.g. [saaty, 1990; harker and vargas, 1990; vargas, 1997]), too, focus on defending these formal aspects. the fifth criticism mentioned in [saaty, 2008] does not, on first blush, seem to relate to a mathematical issue. in the more elaborate discussion in [saaty et al, 2009], however, it becomes clear that the criticism leveled at the ahp/anp is that its process of pairwise priority elicitation is not aligned with the mathematical calculations (e.g. the eigenvector and normalizations) performed thereon. this criticism – presented inter alia in [watson and freeling, 1982; belton and gear, 1983; dyer, 1990] – argues that the decision makers do not understand the pairwise comparison process because what they are (behaviorally) doing is not in line with what the underlying mathematics require. this point will not be elaborated here, suffice to say that this criticism also focuses on a formal aspect of the ahp/anp and as such lies outside of the focus of this paper. what does need consideration is the rigor-relevance debate in the research methodology literature as it clearly has an impact on the current theme. research methodology discussions of research rigor and quality are normally done in terms of validity, reliability and generalizability [mouton and marais, 1990; babbie, 2007]. evaluating the validity of the ahp/anp, hence, brings the question of the rigorrelevance debate to the fore. baldridge et al [2004] indicate that the controversy regarding the practical relevance of academic research is a familiar topic in management articles, books and special issues of academic journals and while this debate has many elements, one of the core questions is whether academic quality (rigor) and practical relevance are mutually exclusive or mutually reinforcing. some argue that academicians and practitioners hold irreconcilable different views of what constitutes academic quality and relevant research. others view the relationship between relevance and academic quality as mutually reinforcing – arguing that if academic theories are not sound or well tested, they have little to offer practice. a similar debate between pure research and action research rages around the question of rigor vs relevance related to the dual goals of action research i.e. the production of action and knowledge [susman and evered, 1978]. eden and huxham [1996] argue that action research is exactly about providing research output that meets the dual requirements of rigor and relevance. rigor and relevance are often – particularly from a positivist perspective seen as being in an inverse relationship and that increased relevance inevitably is achieved at the expense of methodological rigor [susman and evered, 1978]. greenwood and levin [2007] say that the operating assumptions in the conventional social sciences are that greater relevance and engagement automatically involves a loss of scientific validity. the dual objectives of action research lead to the critique that the inclusion of an action component dilutes (contaminates?) the spirit of true research. a strong argument against research aimed at solving practical problems has been a life-long theme of fred n kerlinger. kerlinger [1979:280] says: “it is thought that research in sociology and psychology should be aimed at solving amongst other things problems of prejudice and discrimination, repairing learning deficiencies, improving learning and teaching, ... in general, however, such practical and worthy goals are not the purpose of scientific research. the purpose is theory, and it has no other purpose.” elsewhere, kerlinger [1977:6] states: ijahp article: von solms/validity of the ahp/anp: comparing apples and oranges international journal of the analytic hierarchy process 7 vol. 3, issue 1, 2011 issn 1936-6744 “scientific research never has the purpose of solving human or social problems, making decisions, and taking action.” the critique of action research that the action component is contrary to the true spirit of research is countered by action research proponents by arguing that it is exactly the action component that is one of the reasons for action research to be regarded as superior to traditional research [eden and huxham, 1996; greenwood and levin, 2005, 2007]. particularly, c west churchman and ian mitroff argue that a stance of disinterested objectivity is not only wrong, but the cause of much human tragedy and what is needed is ethical involvement in practice. in mitroff’s [1994:96] own words: “…humankind certainly faces a fundamental problem. (ethics and management are inseparable in the ways that west and i conceive of them.) all the scientific studies, models, and words in the world, … fail to produce significant ethical actions (that is, implementation) to stop a situation that by any standard is morally outrageous. … we can probably generalize safely the following sad conjecture: in most cases, scientific studies, models, words and so forth fail to produce ethical actions to eradicate some important human problem.” in sum: the criticisms of the ahp/anp, the rebuttals, and validity examples almost exclusively address formal, mathematical aspects of ahp/anp theory (cf [millet, 1997]). an important question presents itself at this point: should the ahp/anp be evaluated against the formal mathematical elegance of the underlying theory (i.e. rigor) or against the practical relevance of the method in real world decision-making? the answer to this question must, however, be delayed until after the introduction and brief discussion of traditional views on validity and how the quantitative-qualitative debate has influenced – and changed – these views. 3. validity and the philosophy of science 3.1 traditional validity traditional discussions of validity usually involve a distinction between internal and external validity. internal validity is defined as referring to the fact that a study generates accurate and valid findings of the specific phenomena being studied. therefore, we refer to a project as having produced internally valid results if the constructs were measured in a valid manner, the collected data are accurate and reliable, the analyses are relevant for the type of data, and the final conclusions are adequately supported by the data [mouton and marais, 1990; babbie, 2007]. external validity is defined as referring to a further stage in the research process, whether the findings of a given project are generalizable to all similar cases, and it would therefore be correct to regard external validity and generalizability as synonymous [mouton and marais, 1990; babbie, 2007]. in many studies and research designs, there may be a trade-off between internal and external validity [schram, 2005]. when measures are taken or procedures implemented aiming at increasing the chance for higher degrees of internal validity, these measures may also limit the generalizability of the findings. one way of enhancing internal validity is to perform highly controlled experiments including random sampling and experimental vs control group designs. this situation leads, however, to an artificial situation divorced from the real world and this, in turn, leads many researchers to call for ecologically valid experiments. ijahp article: von solms/validity of the ahp/anp: comparing apples and oranges international journal of the analytic hierarchy process 8 vol. 3, issue 1, 2011 issn 1936-6744 researchers criticize the lack of external validity in many laboratory-based studies with their focus on artificially controlled and constricted environments and the resultant lack in resembling real-world conditions [schram, 2005]. the traditional criteria for validity, reliability and generalizability find their roots in the positivist or foundationalist tradition and the quantitative approaches to research [guba & lincoln, 1994]. mctaggart [1998] points out that the dominant discourse of validity hinges upon the combination of two key quests: the quest for generalization, and the quest for causality (which in other terms is a quest for prediction and control of events). on this view, research is valid if the researcher makes defensible general causal inferences, establishing in the terms of cook and campbell [1979] – ‘molar causal laws’. 3.2 different paradigms methodological monism represents the culmination of the enlightment project and represents the search for a universal, mathematically formulated science as the model of all science and knowledge. this has led to the view that (correctly) all research must be based on the methodology dominant in the natural sciences. auguste comte (1798-1857), the father of positivism, in his famous law of three stages postulated that societies and sciences move inevitably through three consecutive stages. first, a theological stage in which effects are seen to be caused by supernatural agents; second, the transitional metaphysical phase and finally culminating in the third or positive phase in which man seeks to determine the laws which regulate effects. comte believed that the natural sciences were already in the third phase but that the social sciences lagged far behind, and that what was needed was a social physics that would give the person exactly the same role that an atom plays in physics or a planet plays in astronomy, namely, absolute determination without freedom or cavil. john stuart mill (1806-1873) – a one-time disciple of comte urged social scientists to emulate the methods of the natural scientists and promising that, if his advice were followed, this would lead to the rapid maturation of these fields, as well as their emancipation from the philosophical and theological structures that limited them. guba and lincoln [1994] argue that these messages were taken to heart by subsequent social scientists to the extent that scientific maturity is commonly believed to be proportional to the level of quantification or mathematization in a given field. this view of science focuses on verifying or falsifying a priori hypotheses most usefully stated as mathematical (quantitative) propositions. muckler and seven [1992] say that such formulaic precision has enormous utility when the aim is the prediction and control of natural phenomena and that there exists a widespread conviction that only quantitative data are ultimately valid or of high quality – for example in [sechrest, 1992]. recently, strong counter-pressures against the hegemony of methodological monism and quantification have started to emerge. muckler and seven [1992] point out that the acceptance of subjectivity as inevitable pre-dates plato in the views of protagoras; is found in the philosophy of descartes; while kant insisted that all knowledge begins with human experience and cannot be separated from that experience. burrell and morgan [1979], in their classical work, clearly juxtaposes two approaches to social science methodology which they call sociological positivism (an objectivist approach to social science) and german idealism (a subjectivist approach to social science). tsoukas [1994] discusses pepper’s four world hypotheses and shows that two of these, the mechanistic and contextualist systems, are comparable to the burrell and morgan [1979] paradigms of positivism and idealism respectively. flood and jackson [1991] suggest a continuum of contexts ranging from the objectivist ijahp article: von solms/validity of the ahp/anp: comparing apples and oranges international journal of the analytic hierarchy process 9 vol. 3, issue 1, 2011 issn 1936-6744 unitary, through the subjectivist pluralist to the conflictional critical. guba and lincoln [1994] present a very similar topology of social scientific paradigms, listing four, i.e. positivism, post-positivism, critical theory and constructivism. here, the split between positivist and post-positivist is mainly to accommodate the difference between verificationist and popperian falsificationist modes of inquiry. although originally aimed at different issues, the views of burrell and morgan [1979], flood and jackson [1991], tsoukas [1994], and guba and lincoln [1994] have in common the argument that research and knowledge acquisition must be seen as divided at least into two classes, i.e. the quantitative and qualitative research, broadly identifiable with positivism and phenomenology, respectively. qualitative inquiry is distinct and separate from quantitative inquiry. each tradition reflects a commitment to different styles of research, different epistemologies and different representations [guba and lincoln, 1994]. according to bogdan and bilken [1982], qualitative inquiry evolved in response to the inadequacies of quantitative inquiry. quantitative research focuses on causal relationships described in terms of observation statements, verifications and prediction, while qualitative research focuses on exploration of human behavior and the search for understanding through people's actions. guba and lincoln [1994] state that human behavior, unlike that of physical objects, cannot be understood without reference to the meanings and purposes attached by human actors to their activities. therefore, data derived from qualitative inquiry are used to clarify the experience and understanding of a complex phenomenon from the participant's point of view. 3.3 qualitative validity although many researchers posit the distinct attributes of each tradition, there are those who contend that qualitative methods are not clearly differentiated from quantitative methods. the defining attributes of one mode of inquiry can also be found in the other. some qualitative researchers align themselves with quantitative researchers by accepting and redefining their criteria of merit for use in qualitative research and use complicated measures such as computer applications in attempts to ensure objectivity, validity, and objectivity. in contrast, trochim [2006] points out that, depending on their philosophical perspectives, some qualitative researchers reject the framework of validity that is commonly accepted in more quantitative research in the social sciences. they reject the basic realist assumption that there is a reality external to our perception of it. consequently, it doesn't make sense to be concerned with the ‘truth’ or ‘falsity’ of an observation with respect to an external reality (which is a primary concern of validity). these qualitative researchers argue for different standards for judging the quality of research. for instance, guba [1981] proposes four criteria credibility, transferability, dependability, and conformability for judging the soundness of qualitative research; explicitly offers these as an alternative to more traditional quantitatively-oriented criteria internal and external validity, reliability and objectivity and argues that these better reflect the underlying assumptions involved in much of qualitative research. shraderfrechette [1990] says that ever since the failure of carnap's enterprise, epistemologists like sellars and quine have realized that the positivist goal of finding a specific rule or method to guarantee the rationality of science or knowledge is incapable of being achieved (cf [feyerabend, 1978]). this implies that different approaches need to be evaluated in terms of their stated, or implied, epistemological stance rather than against one (superior?) set of epistemic standards. although qualitative and quantitative methods are fundamentally different, there is a propensity to evaluate all approaches against criteria suitable for quantitative research only. according to ryan-nicholls and will [2009], the traditional definition of rigor highlights important attributes of rigor arising in a predominately positivist discourse. ijahp article: von solms/validity of the ahp/anp: comparing apples and oranges international journal of the analytic hierarchy process 10 vol. 3, issue 1, 2011 issn 1936-6744 rigorous research uses logic and accepted systems, and therefore strengthens the arguments researchers make when attempting to demonstrate the significance of findings. rigor is part of the iterative, self-correcting nature of research in which the data collection, systematic analysis, and findings are open to evaluation and replication. a dilemma arises when quantitative researchers attempt to apply this definition and others like it to determine the rigor of qualitative work. when compared with predictive quantitative research designs that are usually focused on measurement and are meticulously organized around notions of equivalence, qualitative inquiry appears to lack the requisite rigor. yet, as ryan-nicholls and will [2009] go on to demonstrate, the problem is not the rigor of qualitative work but the application of inappropriate mechanisms of evaluation. within the qualitative tradition, phenomenological research describes the subjective reality of an event, as perceived by the study population. although focused on the subjective it is very much still an emic approach. participatory action research (par) – an emic form of qualitative investigation – is aimed at allowing the participants to be fully involved. for our current purpose the differences between traditional (positivist) research approaches and par are illuminating. dick and swepson, [1994] point out that action research must be attended by validity measures relevant to the unique circumstances of this form of research. mctaggart [1998] argues that the key commitment in par is to the negotiation of both the research and action among all participants and validity can, therefore, only be achieved if there are appropriate communicative structures in place which allow participants to continue to associate and identify with the work. greenwood and levin [2005] feel that action research should be grounded in pragmatism and say that validity claims are identified as ‘warranted’ assertions resulting from an enquiry process and conclude that co-generated knowledge is deemed valid if it generates warrants for action by the participants in the process. the action researcher should be under no illusion that he will be fully involved, with other participants, in the problem situation and in a social process that will change that situation. he must, therefore, take some responsibility for any practical outcomes that arise and seek to ensure that these represent perceived improvements in the eyes of the participants. susman and evered [1978] argue that in traditional scientific research the emphasis is on predicting outcomes but that in action research the point is to actively make things happen. central to this position are the ideas of the voluntary human agent and that behind every action is individual choice based on human interest. choice is central to taking action and action is central to human development. 3.4 validity and truth ryan-nicholls and will [2009] posit that the search for truth is much more elusive in qualitative vis-à-vis quantitative research. in the former, truth value is subjectoriented rather than defined by the researcher. truth is found in the discovery of human phenomena or experiences as they are lived and perceived by subjects as opposed to the verification of a priori conceptions of such experiences. within the field of epistemology many truth-theories exist, two of the long-standing views are the correspondence and the coherence model of truth [bowden and swartz, 2004]. the former that can be traced back to plato and aristotle is a realist theory that claims the truth of a proposition resides in its correspondence to a fact in the real world. the latter linked to inter alia leibniz, spinoza and hegel – claims that the truth of a proposition arises out of the relationship (coherence) between that proposition and other propositions that have been accepted before [dowden and ijahp article: von solms/validity of the ahp/anp: comparing apples and oranges international journal of the analytic hierarchy process 11 vol. 3, issue 1, 2011 issn 1936-6744 swartz, 2004]. hammond [1996] discusses these two theories in terms of human judgment and decision-making and argues that judgment and decision theories differ in terms of their emphasis on a coherence or correspondence interpretation of human judgment competence. coherence focuses on judgment competence with respect to logical, mathematical, or statistical criteria. what counts as a good judgment or decision from the coherence perspective is whether or not it is congruent with what some well-established and internally consistent set of rules or axioms would have produced. on the other hand, correspondence focuses on judgment competence with respect to empirical accuracy. what counts as a good judgment or decision from the correspondence perspective is how well judgments map onto eventual observable outcomes in the world. hastie [2001] points out that judgment research is focused on the process of how as-yet obscure events, outcomes and consequences in the real world could be inferred. as such, judgment quality is based on accuracy, i.e. correspondence to facts or conditions in the real world. on the other hand, decisionmaking research is focused on preferential choice and action and as such measures decision quality by comparing decisions and behavior against prescriptions of rational, normative models, often taking the form of test for coherence of expectations, values, and preferences [hastie, 2001]. in application to managerial decision-making, it becomes clear that different types of judgments and decisions may evoke a preference for one or the other type of competence. there is strong tendency for prescriptive and normative decision theories to emphasize coherence competence, leading therefore to an interest in biases (i.e. deviations from axiomatic predictions) in decision-making. descriptive theoretical approaches, on the other hand, tend to emphasize correspondence competence, leading to an interest in the factors that influence how accurate (i.e. close to actual outcomes) a decision maker is [hammond, 1996]. gigone and hastie [1997] argue that insufficient research attention has been given to the problem of accuracy in group decision contexts – a finding that highlights the emerging importance of correspondence competence. their work, however, emphasizes the fact that only a small number of judgment tasks can effectively be measured in terms of accuracy. only tasks that have either an objectively demonstrable correct answer, e.g. length of the nile river, or the consensus judgments by a panel of experts, are of this type. the majority of realworld multi-stakeholder decisions would not be amiable to accuracy determination making the use of a correspondence approach to judgment accuracy interesting but of little value. gigone and hastie [1997] argue that coherence approaches will make use of principles of rational judgment and although they cannot address group judgment accuracy they can provide legitimate evaluations of group judgment quality. taket and white [1997] discuss the correspondence theory of truth in respect to community operations research (or). they are adamant that in such circumstances the diverging views of different participants are the critical issues and that these views can only be addressed from a post-structuralist (or post-modern [taket and white, 1994]) perspective. taket and white [1997:99] say: “in relation to representing issues or situations we are forced to reject the idea of a correspondence theory of truth, that reality is ‘out there’, waiting to be discovered and represented accurately. we accept instead the possibilities of specific local, personal and community forms of truth.” they agree with rorty [1989] that a distinction must be made between the world and truth ‘out there’ and while the world exists independently of the human mind, truth cannot so exist. in rorty’s [1989:5] words: ijahp article: von solms/validity of the ahp/anp: comparing apples and oranges international journal of the analytic hierarchy process 12 vol. 3, issue 1, 2011 issn 1936-6744 “the world is out there, but descriptions of the world are not. only descriptions of the world can be true or false. the world on its own – unaided by the describing activities of human beings cannot.” for taket and white [1997], in methods purporting to aid decision structuring in multiple party participative situations all views must be accepted as valid and there would be no problem with conflicting visions because each vision is true in a different world. they distinguish consent-giving from consensus where the former is less restrictive in that it allows cooperation amongst individuals without their necessarily agreeing on all aspects [taket and white, 1994]. an approach opposing the ‘single truth’ view of the enlightenment or modernity is that of constructivism [schwandt, 1994]. the philosophical foundation of constructivism is expressed in four statements; (i) knowledge is not passively accumulated, but rather, is the result of active cognizing by the individual; (ii) cognition is an adaptive process that functions to make an individual's behavior more viable given a particular environment; (iii) cognition organizes and makes sense of experience, and is not a process to render an accurate representation of reality; and (iv) knowing has roots in both biological/neurological construction, and social, cultural, and language based interactions. social constructionism grew out of the more general notion of constructivism and both these perspectives can be located in the domain of post-modernism. whereas the terms of pure (or cognitive) constructivism tend to isolate the thinking process within an individual’s mind, social constructionism calls upon the active participation of others [gergen, 2002]. social constructionism builds on constructivism’s view of knowledge as not objective, universal, or permanent, but it emphasizes knowledge as communal rather than individually created. the notion of social constructivism actively involves social interaction and collaboration in the learning and knowledge-making process and, hence, requires a particular definition of (true) knowledge. this definition is that knowledge is negotiated (through language) until consensus is reached on the group knowledge. within groups the social construction of knowledge relies on consensus between different subjects which is seen as the ultimate criterion to judge knowledge [gergen, 2002]. this form of truth is also called the consensus theory and holds that truth is whatever is agreed upon, or might come to be agreed upon, by some specified group. rescher [1993] argues that a view positing agreement as a prerequisite for truth is evident – in different forms – throughout philosophical history, from aristotle, rousseau, kant and mill to pierce, habermas and rorty. in habermas' [1984] version of the theory, truth is not dependent on actual consensus. rather, it is what would be agreed to by all investigators who follow principles of equal, undistorted, unconstrained communication and who adopt a discursive orientation. thus, habermas [1984] argues that the pragmatic meaning of a truth claim is that it could be verified or made good in a discourse oriented toward rational consensus and not arrived at under conditions of distorted (dominated) communication. 3.5 validity and subjectivity the first major difficulty a view of decision-making validity from within a subjectivist paradigm seems to face, is the question of how a ‘better’ decision is identified – how do we know which decisions are better than others? the philosophical and sociological perspective from which the question is approached will lead to very different answers. within the positivist philosophy and its related sociology of functionalism, objective reality is emphasized. clear goals can be identified and agreement on means to achieve these goals is easy to obtain. ‘better’ is measurable in terms of profit maximization or efficiency increases. the second ijahp article: von solms/validity of the ahp/anp: comparing apples and oranges international journal of the analytic hierarchy process 13 vol. 3, issue 1, 2011 issn 1936-6744 perspective, that of an interpretive sociology and an idealist philosophy emphasizes the subjective nature of reality. different actors will perceive things differently based on each individual’s own weltanschauung and according to the way that he makes sense of reality. accordingly, ‘better’ for one may well be seen as ‘worse’ by another. this perspective excludes the possibility of objectively evaluating different views by positing that nobody can think or evaluate anything other than from within his own worldview [checkland, 1981; flood and jackson, 1991]. ulrich [2002] points out the ethical problem of a ‘we-know-what-is-best-for-everyone’ view and warns against an ‘expert’ minority planning for instead of with the so-called ‘lay’ majority; while anderson et al [1998] show that stakeholders are increasingly demanding an active role in decisions that affect them. a debate that illustrates the issue at hand is that regarding empirical testing of the effectiveness of the dialectic inquiry (di) approach [cosier, 1982; mitroff, 1982]. cosier et al [1978] argue that the empirical evidence for the effectiveness of, what they call, the dialectical inquiry systems (dis) methodology is not as convincing as its theoretical and philosophical support and the limited field studies reported by its proponents have not demonstrated that the dis technique leads to improvement in objective performance or more effective plans. cosier et al [1978] carry out two controlled laboratory and field tests to compare results from dis to results from an alternative planning approach, the devil’s advocate (da), but the results fail to show di as unequivocally superior to da [cosier, 1982]. mitroff and mason [1981] argue that the critique is problematic as it assesses di as an approach to well-structured problems rather than, as it was intended, an approach to ill-structured problems [mitroff, 1982]. according to mitroff and mason [1981] di is a hegelian inquiry system, based on a synthesis of multiple completely antithetical representations that are characterized by intense conflict because of the contrary underlying assumptions. the di method cannot be tested by presenting decision makers with two already formulated worldviews, plans or policies, as true dialectic entails two or more groups actively participating both in formulating and examining of a problem from markedly different points of view [mitroff and mason, 1981; mitroff, 1982; flood and jackson, 1991]. the similarity between the view of mitroff and mason [1981] and the current conceptualization of multi-stakeholder gdm is clear – both involving decisionmaking situations in which the problem needs formulation and debate from radically diverging points of view. the two empirical studies in [cosier et al, 1978] must, from necessity, use objectively known outcomes to be able to test prediction accuracy. the first, based on brunswikian social judgment theory, involves subjective judgment on the part of the participants but the objectively correct outcome is known to the experimenters. similarly the second experiment involves three contrived markets, again unknown to the participants but known to the investigators. the critique of mitroff and mason [1981] and mitroff [1982] is that, although the evaluation involved subjective and uncertain judgments, the problems were not formulated by the participants themselves and this – in their view – is the essence of ill-structured problems; situations where different participants see different problems rather than situations where the participants may differ in their opinion as to the best way to solve a given problem. in their evaluation of this debate flood and jackson [1991] come to the conclusion that approaches like di cannot be subject to empirical verification in the manner attempted by cosier et al [1978] because the philosophy underlying di-like approaches implies that ill-structured problems are, by definition, situations where no objective correct answers exist, but rather that different participants see different realities and the solution cannot be reached by convergence ijahp article: von solms/validity of the ahp/anp: comparing apples and oranges international journal of the analytic hierarchy process 14 vol. 3, issue 1, 2011 issn 1936-6744 on the correct reality but rather through debate, learning and the emergence of a shared – inter-subjective – reality. 4. a new validity problem 4.1 is the ahp/anp a useful decision-making methodology? a criticism leveled at the ahp relates to whether it is useful in gdm situations. aronson et al [1997] claim that their research indicates that there is no advantage in gdm when using ahp instead of unstructured applications and that claims for the success of the ahp is a situation they call ‘much-ado-about-nothing’. this statement belies the many claims made in favor of ahp as a gdm method – for example [dyer and forman, 1992; peniwati, 1999; forman and gass, 2001; saaty and peniwati, 2008], inter alia. in replying to this critique two aspects are relevant. first, the aronson et al [1997] paper does not include any data regarding how the ahp was used exactly; no information is given regarding the hierarchies used, the priorities obtained, or the means of aggregation applied. the exact way in which the ahp was applied cannot be determined and a direct critique of the aronson et al [1997] findings cannot be made. the emphasis is on testing the sharing of common and unique information based on the theoretical underpinnings of the information sampling theory [stasser and titus, 2003] but the fact that the research detail is not available makes it impossible to evaluate whether, on the one hand, ahp really does not enhance information sharing – a fact that would have serious implications for the ahp/anp as a gdm methodology – or on the other hand whether the lack of decision support ascribed to the ahp is caused by the way in which the ahp was applied, instead. second, this research is based on positivist/functionalist premises. in order to establish when and to what extent errors occur, they assume an objectively correct result exists against which individual decisions can be compared. as such the outcomes of this research are of no use in multi-stakeholder gdm as envisaged in this paper. clearly, no objective truth can be assumed under such conditions as it not only belies the pluralistic characteristics of multi-stakeholder groups but also makes the use of such groups redundant – an expert panel would be able to find this true decision without input from stakeholders. in contrast, a pluralist approach to multistakeholder gdm is argued elsewhere [von solms, 1999] and promoted here. banville et al [1998] show that, although stakeholder engagement and mcdm have many points of mutual reinforcing, there is a paucity of research and joint applications of these two fields. the criticism found in [aronson et al, 1997] and comments made in [harper et al, 1992] alert us to the fact that the ahp/anp must be validated from a totally different perspective, i.e. as a gdm tool in a pluralist, subjectivist and qualitative context. to this task we now turn. 4.2 ahp/anp and the paradigms the first issue to be discussed is where the ahp/anp fit in relation to the objectivist-subjectivist paradigms [burrell and morgan, 1979]. saaty [1997] argues that ahp is a new paradigm that some people find hard to accept. science and reason improve the understanding of human existence and the human environment but the facts and understanding obtained through science and reason is fundamentally related to human values, needs, and to the judgments that serve these values and needs [saaty, 2001]. the view that excludes the myriad of intangibles and qualitative human nature from decision-making, by insisting that only tangible and concrete objects, captured through science, can be measured or included in decision-making, is rejected. the world is replete with intangibles, and insistence on measuring only ijahp article: von solms/validity of the ahp/anp: comparing apples and oranges international journal of the analytic hierarchy process 15 vol. 3, issue 1, 2011 issn 1936-6744 tangibles excludes many of the goals and criteria important to political, social and environmental problems [saaty, 1998]. according to saaty [1998] the problem of measuring intangibles is the most important concern facing anyone who wants to grapple successfully with the mathematics of decision-making in all these categories of problems. the measurement of intangibles cannot be done in the traditional scientific manner on an absolute scale with an arbitrary unit of measurement, but must be done in relative terms because an intangible, by definition, does not have a scale with a unit [saaty, 1998, 1999]. he states the problem [saaty, 1998:14] thus: “comparison requires the use of judgment. even informed judgment is subjective. … since relative judgment is more general and includes absolute judgment, it is not difficult to see that what we once though was an objective approach to reality is simply something we agree on, and that all – i mean all – interpretation of reality relates to our own sensory abilities and to our values and goals.” more recently, saaty and his colleagues (e.g. [saaty, 2008, 2010; saaty and peniwati, 2008]) have frequently reiterated the fact that human decision-making is by nature subjective and that this subjectivity must be incorporated into mcdm tools. forman and selly [2001] agree and point out that the fact of multiple criteria – even if the criteria represent tangibles necessitates value judgments as to their relative importance, leading inevitably to subjectivity. saaty [2010] argues that the ahp/anp is similar to the natural sciences (physics specifically) in that both are descriptive rather than normative, but stresses that some significant differences also exist. he specifically mentions that, unlike physics which attempts to find general laws that transcend time and space, the ahp/anp is applied to each problem separately because it is based on subjective judgments that differ from case to case and from one person to another. high quality decisions in complex, ambiguous situations often require multiple perspectives, the expression of contrary viewpoints and the evaluation of multiple alternatives. groups are often reluctant to focus on divergent objectives because of the complexity of processing multiple information streams [clarke et al, 2000]. the ahp is particularly strong in this area as it was developed specifically as a multiple criteria decision-making methodology, allowing multiple perspectives, the expression of contrary viewpoints, and the evaluation of multiple alternatives [saaty, 2001, 2008, 2010]. petkov and mihova-petkova [1997] argue convincingly that ahp performs well as a systems method (cf [saaty and kearns, 1985]), in all three of the problem contexts – unitary, pluralist, and coercive [flood and jackson, 1991] and that it enhances the expression of divergent viewpoints. they also link ahp and ssm [checkland, 1981], a systems methodology applicable to the pluralist context, showing that both approaches allow subjectivity and multiple worldviews [petkov and mihova-petkova, 1996; petkov et al, 1998; petkov et al, 2007]. rosenhead [1996] disagrees and explicitly identifies the ahp as an operations research (or) methodology i.e. a hard systems approach in opposition to problem structuring methods (psm) like ssm. the demarcation between ahp and the psm approaches, according to rosenhead [1996], are the latter’s transparency of method, restricted mathematization and their focus on supporting judgment rather than representing it. petkov and mihova-petkova [1997], on the other hand, argue that it is important in practice to both support and measure human judgment and this strengthens the case for ahp as a systems methodology. ijahp article: von solms/validity of the ahp/anp: comparing apples and oranges international journal of the analytic hierarchy process 16 vol. 3, issue 1, 2011 issn 1936-6744 4.3 ahp/anp and truth theories cooksey [2000] typifies the ahp as a coherence method for he sees it as a prescriptive or normative approach relying on mathematical axioms. contrary, saaty [2008, 2010] and whitaker [2004] argue strongly that the ahp/anp is not a normative theory but is an attempt to develop priorities directly and naturally without insistence on any prescribed notions of rationality. from this it could be deducted that the ahp/anp should be seen as a correspondence method, but this too would be problematic as the reality to which decisions must correspond is a constructed reality based on subjective evaluations, priorities, and values. the dilemma is clarified if we remember that cooksey [2000] – like hammond [1996] and hastie [2001] – work in a paradigm of human judgment and decision-making in which judgments are evaluated in terms of correspondence to objectively known outcomes and decisionmaking in terms of coherence to normatively imposed standards. this paradigm is inadequate to judge the ahp/anp as a decision-making methodology because it does not allow for the possibility of a descriptive approach where the success of a decision is measured against the subjective evaluations by the decision-makers involved [banville et al, 1998]. 4.4 ahp/anp and consensus the inclusion of stakeholders, other than management, in decision-making is a frequent issue in contemporary management literature, particularly in the environmental management and sustainable development fields [anderson et al, 1998; mcewan, 2001]. at the core of stakeholder decision-making is a controversial decision. controversy typically arises, because those with a stake in the decision consequences disagree about the preferred decision alternative. stakeholders can disagree because of differences in values or differences in beliefs about the decision consequences, or both [anderson et al, 1998; slovic, 1999]. multi-stakeholder groups will, by their very nature, represent major difficulties in debate and reaching consensual group choices which could be seen as just and balancing the aspirations of all stakeholders [slovic, 1999; edmunds and wollenberg, 2001; von solms, 2009]. particularly problematic is the use of consensus as choice aggregation in multistakeholder groups. consensus was shown to be less effective and desirable in competitive social contexts [tjosvold and field, 1983]. edmunds and wollenberg [2001] point out that a pure habermasian deliberative approach is inadequate because, although it argues for deliberation within an equitable environment, methods for real-world participative decision-making do not always ensure adequate protection of stakeholders against domination by other, powerful, stakeholders. the deliberative approach must, therefore, be bolstered with some form of protective mechanism. a different choice aggregation method is required that would lead to a fair group outcome even under divergent value systems and inequalities of power while allowing fair trade-offs on salience and relative importance of decision aspects. elsewhere it is argued that the 3-phase variation of the ahp can serve this purpose [von solms and peniwati, 2001; von solms, 2009]. 4.5 ahp/anp and its own validity it seems opportune to locate the use of the ahp in multi-stakeholder groups within a social constructivist-like approach [schwandt, 1994; banville et al, 1998] without committing fully to all the tenets of social constructivism. its emphasis on active learning in a social context with the positive valuing of all participants’ views seems a useful approach. the validity of decision outcomes, within this tradition, would then be measured in terms of the agreement of participants (consensus) as to their own (subjective) observation and evaluation of the outcome rather than against any ijahp article: von solms/validity of the ahp/anp: comparing apples and oranges international journal of the analytic hierarchy process 17 vol. 3, issue 1, 2011 issn 1936-6744 objectively given outcome (correspondence) or normatively imposed standards (coherence). a large number of criteria by which decision-making can be evaluated are found in the literature. we will, here, briefly discuss four criteria that seem appropriate for gdm in a multi-stakeholder situation. these criteria are: (1) decision quality [ettling and jago, 1988; christensen and fjermestad, 1997; brower, 2000]; (2) process fairness [rubin, 1984; peniwati, 1996; lind and tyler, 1988; kim and mauborgne, 1995, 1997]; (3) participant satisfaction [rubin, 1984; ettling and jago, 1988; lind and tyler, 1988; christensen and fjermestad, 1997; brower, 2000]; (4) commitment to implementation [ettling and jago, 1988; brower, 2000; greenwood and levin, 2005]. other criteria, found in the literature, will not be discussed, due to space limitations. process fairness and participant satisfaction are very important in multi-stakeholder situations. the premise of research by peniwati [1996] was that the quality of a decision outcome is determined by the quality of the process (method) used. this assumption holds true if the method is more than a social decision scheme but also provides for adequate social influence [pavitt, 1993]. peniwati [1996] adapts a set of gdm quality criteria from rubin [1984] and includes, inter alia, learning, fairness both in terms of the group members and other stakeholders and rate the ahp as high or very high on these issues. learning represents social influence and, thus, confirms that the ahp is seen as providing adequate change for social interaction (cf [von solms, 2003]). procedural justice theory [lind and tyler, 1988] constitutes an alternative to exchange theory. the latter focuses on decision outcomes while the former is based on the hypothesis that for participants in decision-making processes, the procedures used to arrive at decisions are significant determinants of satisfaction separate from the effect of outcomes [kim and mauborgne, 1995, 1997]. procedural justice theory hypothesizes that the perceived fairness of the process is constituted by two aspects, i.e. voice and dignity. voice enhances procedural fairness when a procedure allows those subjects, concerned with the outcomes, a chance to express their views freely, while dignity refers to procedures that treat group members in a dignified and respectful fashion, i.e. that all member contributions are honored and considered [lind and tyler, 1988]. commitment to implement the decision outcome is an important proof of the participants’ acceptance of the validity of the decisionmaking process. greenwood and levin [2005:54] are adamant: “validity, credibility, and reliability in action research is measured by the willingness of local stakeholders to act on the results of the action research thereby risking their welfare on the ‘validity’ of their ideas and the degree to which the outcomes meet their expectations. thus, cogenerated knowledge is deemed valid if it generates warrants for action.” the criteria, fairness, satisfaction and commitment are generally accepted as subjective and measured using surveys in which the participants express their own subjective views [christensen and fjermestad, 1997; hacker, 1997; brower, 2000]. hacker [1997] points out that self-reported satisfaction measures may involve significant biases. she refers to an observation by muckler and seven [1992] that self-report data are abhorred by objective measurement advocates because misrepresentation and misperception are possible. hacker [1997], however, states that most scholars still include self-report data regarding satisfaction. this is not surprising as bias can only be defined in terms of an objective measure and as ijahp article: von solms/validity of the ahp/anp: comparing apples and oranges international journal of the analytic hierarchy process 18 vol. 3, issue 1, 2011 issn 1936-6744 satisfaction is clearly a subjective perception for which no objective measure exists, satisfaction can only be known if the perceiver reports on it. while the subjective nature of these three criteria is accepted, the situation is more complicated when decision-making quality is considered. in the case of decisionmaking quality, attempts are made in the literature to set up objective means of measuring and comparing quality. watson et al [1988] argue that in group decision support systems (gdss) research the performance of the group is often compared to an objective measure of decision quality but that many organizational gdm occur without prior or post knowledge of the ‘correct’ outcome. only if quality is deemed objectively measurable does the possibility of identifying and eliminating biased results exist. kruglanski and ajzen, [1983] review the literature on bias and error in human judgment and list three ways in which such biases or errors are identified, i.e. normative models, direct measurement and investigator (or expert) judgment. they point out that all three approaches implicitly assume the existence of an objectively verifiable correct result against which actual decisions can be compared to determine the existence of bias or error. the differences are only in the different ways in which this ‘correct’ result is uncovered [kruglanski and ajzen, 1983]. funder [1990] criticizes the kruglanskian lay epistemic theory for focusing on process rather than on content (i.e. accuracy) of judgments. in his view the accuracy of a judgment cannot be evaluated by investigating the process by which judgments are made. funder’s [1990] view is that far too negative a picture of human judgment competence is presented by the error and biases researchers hampering progress in social psychological research. he argues that accuracy research should be located in a realist paradigm of an objectively knowable reality to which judgments can be compared in determining their accuracy. funder and west [1993] clearly state their realist position when they say that inaccuracy can be inferred only if it is assumed that the property that the subjects have been asked to judge ‘exists’ and that one value of the property excludes the possibility of other values of the property. funder [1990] emphasizes accuracy in social judgment to be fashioned on the objectivist brunswikian approach to psychology, i.e. that of correspondence of judgment to objective reality as defined by expert opinion. whereas the work of funder and his colleagues represent the investigator (expert) judgment category of kruglanski and ajzen’s [1983] three-category scheme, the work of kahneman and tversky [1979, 2000], on the other hand, represent the normative models category. this research paradigm is realist too, in that the decisions used are choices between simple monetary gambles with objectively specified probabilities and at most two non-zero outcomes. the use of expert judgment as the benchmark of a true outcome is widely advocated and used [muckler and seven, 1992; christensen and fjermestad, 1997; brower, 2000]. the latter two papers, however, argue that although using expert judgment in this way is widespread, it is, in fact, wrong as real decision-making situations are rarely (if ever) of the type where the true result is known to experts and the groups are only to discover this true result. brower [2000] criticizes the realist judgment research design as being based on simple or contrived situations in which either an objectively correct result exists or an expert panel can provide an outcome that is deemed to be the correct result. muckler and seven’s [1992] review indicates that even within this paradigm, the search for objective measures are not always successful. this is a further indication that an attempt to cling to objectivity and objective measures is not a fruitful paradigm to adopt in gdm research. multistakeholder gdm, particularly on environmental issues, cannot be seen as ‘objective’ and hence no right answer exists against which decision outcomes can be measured. ijahp article: von solms/validity of the ahp/anp: comparing apples and oranges international journal of the analytic hierarchy process 19 vol. 3, issue 1, 2011 issn 1936-6744 thus, decision quality must also be measured in terms of how the participants experience and view the decision outcome. 5. conclusion the ahp/anp is an important method in all types of decision-making situations, particularly in multi-stakeholder gdm, which is becoming more and more important in the world today [petkov et al, 1998; saaty and peniwati, 2008]. an important line of research, discussed here, is the validation of the ahp/anp. this is predominantly done using the judgment paradigm with known outcomes against which to compare the ahp/anp outcomes. we conclude that within this paradigm the validity examples show the ahp/anp to be a valid method. further, the rebuttals of criticisms of the formal aspects of the ahp/anp prove (to our satisfaction) its mathematical foundation to be valid. but these efforts need to be complimented with a new direction in research research to test the validity of the ahp/anp in terms of the world of the multi-stakeholder group, where no known outcomes exist and it is paramount that the subjective preferences and values of the participants be acknowledged and incorporated in a fair and transparent process. we have argued for the need of new validity criteria to show the ahp/anp as capable of assisting real decision-makers in addressing the pressing human problems of inter alia conflict, suffering and environmental degradation. we need to test the validity of the ahp/anp over the whole gambit of decision-making applications. saaty [2010:432] recommends: “…that multicriteria methods put greater emphasis on validation to acquire greater credibility in practice. validation is much more difficult when all judgments depend on feelings alone without memory from the senses and when the criteria are all intangible. but there are other ways to improve the credibility of the outcome that have been discussed in the literature…” the literature referenced by saaty is [whitaker, 2004] in which the ahp/anp is evaluated, albeit from a limited perspective. we have here evaluated and compared the ahp/anp apple and have found it to be redder than other mcdm apples and we have argued that the oranges of multistakeholder gdm have not yet been adequately evaluated. now we need to take up the challenge and move forward to measuring both apples and oranges. references anderson, j, clément, j & crowder, lv (1998). accommodating conflicting interests in forestry concepts emerging from pluralism; unasylva 49/3; 3-10. aronson, je, dennis, ar, hilmer, km & stam, a (1997). analytical hierarchy process in group information exchange and decision making: much ado about nothing; proceedings of the 3rd americas conference on information systems, indianapolis, in; august. babbie, er (2007). the practice of social research; 11th edition; thomson/wadsworth; belmont, ca. ijahp article: von solms/validity of the ahp/anp: comparing apples and oranges international journal of the analytic hierarchy process 20 vol. 3, issue 1, 2011 issn 1936-6744 baldridge, dc, floyd, sw & markoczy, l (2004). are managers from mars and academicians from venus? toward an understanding of the relationship between academic quality and practical relevance; strategic management journal, 25/11; 1063-1074. bana e costa, ca & vansnick, j-c (2008). a critical analysis of the eigenvalue method used to derive priorities in ahp; european journal of operational research 187/3; 1422-1428. banville, c, landry, m, martel, j-m & boulaire, c (1998). a stakeholder approach to mcda; systems research & behavioral science 15/1; 15-32. barzilai, j (1999). on maut, ahp and pfm; proceedings 5th international symposium on the analytic hierarchy process; kobe, japan; august; 57-60. barzilai, j (forthcoming). preference function modeling: the mathematical foundations of decision theory; in figueira, j; greco, s & ehrgott, m (eds); trends in mcda; springer verlag; dordrecht, the netherlands. barzilai, j & golany, b (1994). ahp rank reversal, normalization and aggregation rules; infor 32/2; 57-63. belton, v & gear, t (1983). on a shortcoming of saaty’s method of analytic hierarchies; omega 11/3; 228-230. bogdan, r & biklen, s (1992). qualitative research for education; 2nd edition; allyn and bacon; boston, ma. bowden, b & swartz, n (2004). truth; the internet encyclopedia of philosophy; online at http://www.iep.utm.edu/t/truth.htm brower, hh (2000). group decision making in the boardroom: a model for effectiveness; proceedings 2000 annual conference; midwest academy of management; chicago, il; march. burrell, g & morgan, g (1979). sociological paradigms and organisational analysis; gower; aldershot, uk. checkland, pb (1981). systems thinking, systems practice; john wiley & sons; chichester, uk. christensen, ew & fjermestad, j (1997). challenging group support systems research: the case for strategic decision making; group decision and negotiation 6/1; 351-372. clarke, ma, anand, v & roberson, l (2000). resolving meaning interpretation in diverse decision-making groups; group dynamics: theory, research and practice 4/3; 211-221. cook, td & campbell, dt (1979). quasi-experimentation: design and analysis issues for field settings; rand mcnally college publication company; chicago, il. ijahp article: von solms/validity of the ahp/anp: comparing apples and oranges international journal of the analytic hierarchy process 21 vol. 3, issue 1, 2011 issn 1936-6744 cooksey, rw (2000). managerial judgment and decision making; in dahiya, sb (ed); the current state of business disciplines: volume 5 – management ii; spellbound publications; rohtak, india. cosier, ra (1982). methods for improving the strategic decision: dialectic versus the devil’s advocate; strategic management journal 3/4; 373-374. cosier, ra, ruble, tl & aplin, jc (1978). an evaluation of the effectiveness of dialectical inquiry systems; management science 24/14; 1483-1490. dick, b & swepson, p (1994). appropriate validity and its attainment within action research: an illustration using soft systems methodology; available on-line at http://www.scu.edu.au/schools/gcm/ar/arp/sofsys2.html dyer, js (1990). remarks on the analytic hierarchy process; management science 36/3; 249-258. dyer, rf & forman, eh (1992). group decision support with the analytic hierarchy process; decision support systems 8; 99-124. eden, c & huxham, c (1996). action research for the study of organizations; in clegg, sr; hardy, c & nord, wr (eds); handbook of organizational studies; sage publications; london, uk. edmunds, d & wollenberg, e (2001). a strategic approach to multistakeholder negotiations; development and change 32/2; 231-253. ettling, jt & jago, ag (1988). participation under conditions of conflict: more on the validity of the vroom-yetton model; journal of management studies 25/1; 1-22. feyerabend, pk (1978). against method: outline of an anarchistic theory of knowledge; verso; london, uk. finan, js & hurley, wj (2002). the analytic hierarchy process: can wash criteria be ignored?; computers & operations research 29/8; 1025-1030. flood, rl & jackson, mc (1991). creative problem-solving: total systems intervention; john wiley & sons; chichester, uk. forman, eh, & gass, si (2001). the analytic hierarchy process – an exposition; operations research 49/4; 469-486. forman, eh & selly ma (2001). decision by objectives: how to convince others that you are right; world scientific publications; singapore. funder, dc (1990). process versus content in the study of judgmental accuracy; psychological inquiry 1/3; 207-209. funder, dc & west, sg (1993). consensus, self-other agreement and accuracy of personality judgments: an introduction; journal of personality 61/4; 457-476. gergen, kj (2002). beyond the empiricist/constructionist divide in social psychology; personality and social psychology review 6/3; 188-191. ijahp article: von solms/validity of the ahp/anp: comparing apples and oranges international journal of the analytic hierarchy process 22 vol. 3, issue 1, 2011 issn 1936-6744 gigone, d & hastie, r (1997). proper analysis of the accuracy of group judgments; psychological bulletin 1; 149-167. greenwood, dj & levin, m (2005). reform of the social sciences and universities through action research; in denzin, nk & lincoln, ys (eds); sage handbook of qualitative research; 3rd edition; sage publications; thousand oaks, ca. greenwood, dj & levin, m (2007). introduction to action research: social research for social change; 2nd edition; sage publications; thousand oaks, ca. guba, eg (1981). criteria for assessing the trustworthiness of naturalistic inquiries; educational communication and technology journal 29/2; 75-92. guba, eg & lincoln, ys (1994). competing paradigms in qualitative research; in denzin, nk & lincoln, ys (eds); handbook of qualitative research; sage publications; thousand oaks, ca. habermas, j (1984). the theory of communicative action: reason and rationalisation of society; volume 1; polity press; cambridge, ma. hacker, me (1997). the effect of decision aids on work group performance; unpublished phd thesis; virginia polytechnic institute and state university; blackburg, va. hammond, kr (1996). human judgment and social policy: irreducible uncertainty, inevitable error, unavoidable injustice; oxford university press; new york, ny. harker, pt & vargas, lg (1990). reply to ‘remarks on the analytic hierarchy process’ by js dyer; management science 36/3; 269-275. harper, rm, apostolou, ng & hartman, bp (1992). the analytic hierarchy process: an empirical examination of aggregation and hierarchical structuring; behavioral research in accounting; 4/1; 96-112. hastie, r (2001). problems for judgment and decision making; annual review of psychology 52/1; 653-683. holder, rd (1990). some comments on the analytic hierarchy process; journal of the operational research society 41/11; 1073-1076. ishizaka, a & labib, a (2009). analytic hierarchy process and expert choice: benefits and limitations; or insight 22/4; 201-220. kahneman, d & tversky, a (1979). prospect theory: an analysis of decision under risk; econometrica 47/2; 263-291. kahneman, d & tversky, a (eds) (2000). choice, values and frames; cambridge university press; new york, ny. kerlinger, fn (1977). the influence of research on education practice; educational researcher 6/8; 5-12. ijahp article: von solms/validity of the ahp/anp: comparing apples and oranges international journal of the analytic hierarchy process 23 vol. 3, issue 1, 2011 issn 1936-6744 kerlinger, fn (1979). behavioral research: a conceptual approach; holt, rinehart and winston; new york. ny. kim, wc & mauborgne, ra (1995). a procedural justice model of strategic decision making: strategy content implications in the multinational; organization science 6/1; 44-61. kim, wc & mauborgne, ra (1997). fair process: managing in the knowledge economy; harvard business review 75/4; 65-75. kirchler, e & davis, jh (1986). the influence of member status differences and task type on group consensus and member position change; journal of personality and social psychology 51/1; 83-91. kruglanski, aw & ajzen, i (1983). bias and error in human judgment; european journal of social psychology 13/1; 1-44. laughlin, pr (1996). group decision making and collective induction; in witte, e & davis, jh (eds); group behavior: consensual behavior by small groups; volume 1; lawrence erlbaum associates; mahwah, nj. leskinen, p (2000). measurement scales and scale independence in the analytic hierarchy process; journal of multi-criteria decision analysis 9/4; 163-174. lind, ea & tyler, tr (1988). the social psychology of procedural justice; plenum press, new york, ny. maccoun, rj (1998). biases in the interpretation and use of research results; annual review of psychology 49/1; 259-287. ma, d & zheng, x (1991). 9/9-9/1 scale method of ahp; proceedings 2nd international symposium on ahp; pittsburgh, pa; august. mcewan, t (2001). managing values and beliefs in organisations; financial times/prentiss hall; harlow, uk. mctaggart, r (1998). is validity really an issue for action research?; studies in cultures, organizations and societies 4/2; 211-236. millet, i (1997). the effectiveness of alternative preference elicitation methods in the analytic hierarchy process; journal of multi-criteria decision analysis 6/1; 4151. mitroff, ii (1982). talking past one’s colleagues in matters of policy; strategic management journal 3/4; 374-376. mitroff, ii (1994). the cruel science of world mismanagement: an essay in honor of c. west churchman; interfaces 24/4; 94-98. mittroff, ii & mason, ro (1981). the metaphysics of policy and planning: a reply to cosier; academy of management review 6/4; 649-651. ijahp article: von solms/validity of the ahp/anp: comparing apples and oranges international journal of the analytic hierarchy process 24 vol. 3, issue 1, 2011 issn 1936-6744 mouton, j & marais, hc (1990). basic concepts in the methodology of the social sciences; human sciences research council; pretoria, rsa. muckler, fa & seven, sa (1992). selecting performance measures: 'objective' versus 'subjective' measurement; human factors 34/4; 441-455. pavitt, c (1993). does communication matter in social influence during small group discussion?: five positions; communication studies 44/3-4; 216-227. peniwati, k (1996). the possibility theorem for group decision making: the analytic hierarchy process; unpublished phd thesis; university of pittsburgh; pittsburgh, pa. peniwati, k (1999). a unifying theory for social choice and synergetic group decision making: the analytic hierarchy process; proceedings isahp 1999; kobe, japan; august; 153-158. pérez, j (1995). some comments on saaty’s ahp; management science 41/6; 10911095. pérez, j, jimeno, jl & mokotoff, e (2006). another potential strong shortcoming of ahp; top: journal of the spanish society of statistics and operations research 14/1; 99-111. petkov, d, mcewan, t, von solms, sh & vezjak m (1998). moving towards compliance with standards for environmental protection an example of mixing approaches to 'messy' problems; proceedings 14th european meeting on cybernetics & systems research; vienna, austria; 252-257. petkov, d & mihova-petkova, o (1996). problem structuring in the analytic hierarchy process and soft systems methodology; proceedings 4th international symposium on the analytic hierarchy process; vancouver, canada; july. petkov, d & mihova-petkova, o (1997). the analytic hierarchy process and systems thinking; proceedings 13th international mcdm conference; cape town, rsa. petkov, d, petkova, o, andrew, t & nepal, t (2007). mixing multiple criteria decision making with soft systems thinking techniques for decision support in complex situations; decision support systems 43/4; 1615-1629. rescher, n (1993). pluralism: against the demand for consensus; clarendon press; oxford, uk. rorty, rm (1989). contingency, irony and solidarity; cambridge university press; cambridge, uk. rosenhead, j (1996). what’s the problem? an introduction to problem structuring methods; interfaces 26/6; 117-131. rubin, jz (1984). introduction; in swap, wc & associates (eds); group decision making; sage publications; beverly hills, ca. ijahp article: von solms/validity of the ahp/anp: comparing apples and oranges international journal of the analytic hierarchy process 25 vol. 3, issue 1, 2011 issn 1936-6744 ryan-nicholls, k & will, c (2009). rigor in qualitative research: mechanisms for control; nurse researcher 16/3; 70-85. saaty, tl (1990). an exposition of the ahp in reply to ‘remarks on the analytic hierarchy process’; management science 36/3; 259-268. saaty, tl (1997). that is not the analytic hierarchy process: what the ahp is and what it is not; journal of multi-criteria decision analysis 6/6; 320-339. saaty, tl (1998). reflections and projections on creativity in operations research and management science: a pressing need for a shift in paradigm; operations research 46/1; 9-16. saaty, tl (1999). the seven pillars of the analytic hierarchy process; proceedings 5th international symposium on the analytic hierarchy process; kobe, japan; august; 20-33. saaty, tl (2001). decision-making with dependence and feedback: the analytic network process; 2nd edition; rws publications; pittsburgh, pa. saaty, tl (2008). relative measurement and its generalization in decision making: why pairwise comparisons are central for the measurement of intangible factors – the analytic hierarchy/network process; review of the royal spanish academy of sciences, series a, mathematics 102/2; 251-318. saaty, tl (2010). principia mathematica decernendi: mathematical principles of decision-making; rws publications; pittsburgh, pa. saaty, tl & kearns, kp (1985). analytical planning: the organization of systems; pergamon press; oxford, uk. saaty, tl & peniwati, k (2008). group decision making: drawing out and reconciling differences; rws publications; pittsburgh, pa. saaty, tl & vargas, lg (2006). the analytic hierarchy process: wash criteria should not be ignored; international journal of management & decision making 7/2-3; 180-188. saaty, tl, vargas, lg & whitaker, r (2009). addressing with brevity criticisms of the analytic hierarchy process; international journal of the ahp 1/2; on-line at http://www.ijahp.org/index.php/ijahp/article/view/53/30 salo, aa & hämäläinen, rp (1997). on the measurement of preferences in the analytic hierarchy process; journal of multi-criteria decision analysis 6/6; 309319. schram, a (2005). artificiality: the tension between internal and external validity in economic experiment; journal of economic methodology 12/2; 225-237. schwandt, ta (1994). constructivist, interpretivist approaches to human inquiry; in denzin, nk & lincoln, ys (eds); handbook of qualitative research; sage publications; thousand oaks, ca. ijahp article: von solms/validity of the ahp/anp: comparing apples and oranges international journal of the analytic hierarchy process 26 vol. 3, issue 1, 2011 issn 1936-6744 sechrest, l (1992). roots: back to our first generations; evaluation practice 13/1; 1-7. sechrest, l (2005). validity of measures is no simple matter; health services research 40/5; 1584-1604. shrader-frechette, ks (1990). scientific method, anti-foundationalism and public decisionmaking; risk: issues in health and safety 1/4; 341-363. slovic, p (1999). trust, emotion, sex, politics and science: surveying the risk assessment battlefield; risk analysis 19/4; 689-701. stasser, g & titus, w (2003). hidden profiles: a brief history; psychological enquiry 14/3-4; 304-313. susman, gi & evered, rd (1978). an assessment of the scientific merits of action research; administrative science quarterly 23/4; 582-603. taket, ar & white, la; (1994). doing community operational research with multicultural groups; omega 22/6; 579-588. taket, ar & white, la; (1997). wanted: dead or alive – ways of using problem structuring methods in community or; international transactions in operational research 4/2; 99-108. tjosvold, d & field, rhg; (1983). effects of social context on consensus and majority vote decision making; academy of management journal 26/3; 500-506. trochim, wmk (2006). qualitative validity; in trochim, wmk; the research methods knowledge base; a web-based textbook; this chapter available on-line at http://www.socialresearchmethods.net/kb/qualval.php tsoukas, h (1994). refining common sense: types of knowledge in management studies; journal of management studies 31/6; 761-780. ulrich, w (2002). critical systems heuristics; in daellenbach, hg & flood, rl (eds); the informed student guide to management science; thomsons learning; london, uk. van den honert, rc & lootsma, fa (1996). group preference aggregation in the multiplicative ahp: the model of the group decision process and pareto optimality; european journal of operational research 96/2; 363-370. vargas, lg (1997). comments on barzilai and lootsma – why the multiplicative ahp is invalid: a practical counterexample; journal of multi-criteria decision analysis 6/3; 169-170. von solms, sh (1999). a participative approach to environmental impacts assessment; tappsa journal; november; 34-36. von solms, sh (2003). group polarization, social influence and the analytic hierarchy process; proceedings 7th international symposium on the analytic hierarchy process; bali, indonesia; august; 475-484. ijahp article: von solms/validity of the ahp/anp: comparing apples and oranges international journal of the analytic hierarchy process 27 vol. 3, issue 1, 2011 issn 1936-6744 von solms, sh (2009). homogeneity and choice aggregation in the analytic hierarchy process; proceedings 10th international symposium on the analytic hierarchy process; university of pittsburgh, pittsburgh, pa; august. von solms, sh & peniwati, k (2001). to agree or not to agree, that is the question: choice aggregation in the ahp; proceedings 6th international symposium on the analytic hierarchy process; bern, switzerland; august. watson, rt, desanctis, g & poole, ms (1988). using a gdss to facilitate group consensus: some intended and unintended consequences; mis quarterly 12/3; 463478. watson, sr & freeling, ans (1982). assessing attribute weights; omega 10/6; 582– 583. whitaker, r (2004). validation examples for the analytic hierarchy process and the analytic network process; proceedings 17th international conference on multiple criteria decision analysis; whistler, bc, canada; august. whitaker, r (2007). criticisms of the analytic hierarchy process: why they often make no sense; mathematical and computer modelling 46/7-8; 948-961. wijnmalen, djd (2001). improved structural weight adjustment in top down oriented conventional ahp hierarchies; proceedings 6th international symposium on the analytic hierarchy process; bern, switzerland; august. ijahp article: nisel, özdemir /ahp/anp in sports: a comprehensive literature review international journal of the analytic hierarchy process 405 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3. 448 ahp/anp in sports: a comprehensive literature review seyhan nisel school of business, istanbul university turkey sipahi@istanbul.edu.tr muhlis özdemir school of business, istanbul university turkey muhlisozdemir@istanbul.edu.tr abstract since the invention of ahp/anp, several literature review studies have been presented to summarize theoretical developments and different application areas of its techniques. the purpose of this study is to present a comprehensive literature review of ahp and anp applications in the field of sports. a total of 62 sports related ahp and anp articles were selected, categorized and analyzed in this study. the findings show that ahp and anp techniques have successfully been used for performance evaluation of teams, player selection and ranking, team or club performance ranking and coach evaluation in many sport branches. keywords: ahp; anp; sport applications; literature review 1. introduction the analytic hierarchy process (ahp) technique, developed by thomas l. saaty in the 1970s, is a multi-criteria decision making technique that can incorporate both objective and subjective judgments in the decision making process. it can deal with complex decision problems by decomposing the problem into a hierarchical structure, and it has been widely used in many areas of science, engineering, business and management in the last four decades. the analytic network process (anp) technique, which is a generalization of ahp and developed by saaty in the late 1990’s, can deal with dependencies and feedback in the hierarchy that looks more like a network (saaty, 2001). although its methodological structure is more complex than the ahp algorithm, there are numerous real world applications of anp in many fields and it has been used as an accurate decision making tool in the last 20 years. since its inception, ahp has been studied extensively and has become a prominently used technique in the literature. as a result, several literature review studies have been conducted to summarize theoretical developments and different application areas of ahp and anp techniques. firstly, shim (1989) provided a comprehensive bibliographical survey of studies on ahp. the study used the dialog online information retrieval mailto:sipahi@istanbul.edu.tr mailto:muhlisozdemir@istanbul.edu.tr ijahp article: nisel, özdemir /ahp/anp in sports: a comprehensive literature review international journal of the analytic hierarchy process 406 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3. 448 service and 141 references of methodological and applied papers as well as 21 dissertation studies on ahp for the period from 1979–1988 were categorized. vargas (1990) presented a literature review study of ahp and its applications. this study stated that ahp was a relatively young theory and was finding more advocates in both the theoretical and the applied sciences day by day. it also allocated and classified ahp related papers by area such as economic/management problems, political problems, social problems, technological problems and books of applications. in another study, apostolou and hassell (1993) reviewed ahp papers in accounting literature, and presented a discussion of the ahp and its applications in accounting throughout the years. vaidya and kumar (2006) grouped the references regionally and yearly in order to track the increase of applications of ahp. liberatore and nydick (2008) presented a literature review of the application of the ahp in medical and health care decision making since 1997. in this study, 50 papers were classified by diagnosis, patient participation, therapy/treatment, organ transplantation, project and technology evaluation and selection, human resource planning, and health care evaluation and policy (liberatore & nydick, 2008). ho (2008) gathered related articles that appeared in international journals from 1997 -2006 and reviewed the literature of the applications of the integrated ahps. this paper stated that mathematical programming, quality function deployment, metaheuristics, swot analysis and data envelopment analysis (dea) were commonly combined with the ahp in integrated studies. sipahi and timor (2010) presented a comprehensive literature review of applications of ahp as well as anp for the years 2005‐2009. the paper also covers fuzzy ahp and fuzzy anp applications (sipahi & timor, 2010). ishizaka and labib (2011) theoretically reviewed ahp articles rather than classifying them by application areas. in their study, they discussed problem modelling, pairwise comparisons, judgment scales, derivation methods, consistency indices, incomplete matrix, synthesis of the weights, sensitivity analysis and group decisions issues (ishizaka & labib, 2011). subramanian and ramanathan (2012) reviewed and methodologically analyzed applications of ahp in operations management between the years 1990-2009. they categorized 291 application papers of ahp into operations strategy, process and product design, planning and scheduling resources, project management, managing the supply chain (subramanian & ramanathan, 2012). ahp based approaches for supplier evaluation were studied by bruno et al (2012). in their study, they highlighted the weak and strong points emerging from the implementation of the ahp in a supply chain (bruno et al., 2012). finally, russo and camanho (2015) presented a systematic review of the literature on real cases that applied ahp to evaluate how the criteria were being defined and measured. here, 33 cases used in the literature to build the criteria and calculate the weights in ahp or fuzzy ahp contrary to other techniques which were used to evaluate alternatives were reviewed (russo & camanho, 2015). since the 1980’s, the ahp methodology has been widely used in the field of sports. in the last two decades, anp has also been utilized as a decision making tool in several branches of sports. however, a literature analysis about the use of ahp and anp techniques in the area of sports has not been presented until now. the purpose of this study is to present a comprehensive literature review of ahp and anp applications in several sports branches. the paper is organized as follows. the process of the research methodology is described in the next section. analysis and classifications of the literature ijahp article: nisel, özdemir /ahp/anp in sports: a comprehensive literature review international journal of the analytic hierarchy process 407 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3. 448 are presented in section 3. in this section, articles are categorized and discussed by purpose of study and by sports branch. the last section is devoted to conclusions, remarks and suggestions for future research. 2. research methodology to conduct a comprehensive literature search on sports applications of ahp and anp, peer reviewed journal articles written in the english language from 1980-2016 were carefully retrieved from the isi web of science, sciencedirect, ebscohost (academic source premier and business source complete), emerald, taylor and francis, proquest abi inform, ingenta, ovid, medline and inderscience academic databases. books, book chapters, conference proceedings, and master’s and doctoral theses were excluded from the study. the international journal of the analytic hierarchy process was also included in the study. during the literature search, studies in all areas of sports including physical activity (competitive or non-competitive) were considered. four keywords (“analytic hierarchy process”, “ahp”, “analytic network process”, “anp”) combined with “sport” were used in the first searching phase. the second searching phase was broadened, and the ahp and anp keywords were combined with many sports related descriptors (football (soccer), basketball, baseball, volleyball, olympic, national games, hockey, tennis, racing, swimming, handball, rugby, american football, cycling, riding, cricket, badminton, archery, chess, polo, golf, gymnastic, physical training and stadium in order to obtain a comprehensive list of articles. a total of 62 articles were identified for the literature review, and all articles were meticulously analyzed in terms of sports relation and in terms of technique(s) used in the study. information about 62 articles were coded and recorded in ms excel under the following categories:  publication title  source title (journal name)  author(s)  affiliation of author(s)  publication year  branch(es) of sport  methodology or methodologies used (ahp, anp and other integrated techniques)  problem definition  number of citations in google scholar (as of november 19, 2016) articles were categorized and analyzed according to several questions. section 3 presents the classifications and analysis of the literature. 3. classifications and analysis 3.1 number of publications by ahp and anp as presented in table 1 and figure 1 the ahp method was used significantly in the studies. out of 62 articles, 32 directly used ahp for the decision-making problem, while ijahp article: nisel, özdemir /ahp/anp in sports: a comprehensive literature review international journal of the analytic hierarchy process 408 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3. 448 19 integrated ahp with some other techniques. the technique that was most often integrated with ahp was topsis, followed by dea, delphi method and anp respectively (figure 1). in table 1 and figure 1, some studies were counted more than once for collaborative techniques (for instance, a study which integrated ahp and anp was counted as ahp integrated study and also anp integrated study). table 1 number of publications of ahp and anp figure 1. percentages of integrated methods with ahp in figure 1, other techniques (43%) integrated with ahp were input output analysis, saw, entropy method, regression tree, decision tree, dematel, grey systems, anova, gis and causality diagram. the publications that integrated ahp and these techniques in the field of sports were classified and are presented in the following sections. 3.2 percentage of publications by different period of time figure 2 shows the percentage of publications by the period of time they were published. as can be seen, only 10% of the publications were published before the year 2000. between the years 2000-2009, the number of papers increased with 22% of the papers belonging to that time period. however, between the years 2010-2016, the number of publications increased significantly with 68% of the publications published in the last method number of publications ahp studies 32 ahp integrated studies 19 anp 3 anp integrated studies 2 fuzzy ahp 5 fuzzy ahp integrated studies 3 total 64 ijahp article: nisel, özdemir /ahp/anp in sports: a comprehensive literature review international journal of the analytic hierarchy process 409 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3. 448 seven years. this figure demonstrates a significant increase in the number of publications of ahp in the field of sports in recent years. figure 2. percentages of number of publications by different time period 3.3 classification by country (author affiliation) table 2 presents the frequency distribution of papers by author affiliation. as shown in the table 2, china has the largest number of publications in the field with 17 papers followed by the us (14 papers) and iran (6 papers). korea (5), taiwan (5), croatia (4), and turkey (4) are other significant contributing countries in the field with a number of papers. india, israel and japan are the other countries that have contributions. switzerland, finland, georgia, kenya, and poland are countries that have at least one paper contributed to the field. as can be seen from the table, authors from 15 different countries used ahp or anp techniques in the sports field in their studies. four publications were counted more than once for collaborative studies. table 2 number of publications by country (author affiliations) country name (affiliation of authors) number of publications country name (affiliation of authors) number of publications china 17 israel 2 us 14 japan 2 iran 6 switzerland 1 korea 5 finland 1 taiwan 5 georgia 1 croatia 4 kenya 1 turkey 4 poland 1 india 2 total 66 ijahp article: nisel, özdemir /ahp/anp in sports: a comprehensive literature review international journal of the analytic hierarchy process 410 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3. 448 3.4 classification by journal according to literature search results, six papers out of sixty-two were published in the journal entitled advanced materials research (see table 3). table 3 also indicates that applied mechanics and materials and collegium antropologicum made contributions to the field with four and three papers respectively. table 3 top journals by number of publications journal name number of publications advanced materials research 6 applied mechanics and materials 4 collegium antropologicum 3 interfaces 2 international journal of analytic hierarchy process 2 mathematical and computer modeling 2 other journals 43 total 62 interfaces, international journal of analytic hierarchy process and mathematical and computer modeling are also significant journals with two publications each. forty-three different journals contributed at least one publication to the sports field with ahp or anp models. journals were also classified by the total number of citations of the publications in the literature search. table 4 shows the top fifteen journals with the highest number of citations. the number of citations of the publications was obtained from google scholar on november 19, 2016. when the number of citations is considered, european journal of operational research has the highest number of citations (216) with just one paper followed by interfaces (107) and collegium antropologicum (89) respectively. in table 5, the top ten most highly cited papers have also been presented. ijahp article: nisel, özdemir /ahp/anp in sports: a comprehensive literature review international journal of the analytic hierarchy process 411 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3. 448 table 4 top journals by number of citations journal name number of citations of publications european journal of operational research 216 interfaces 107 collegium antropologicum 89 sport management review 61 journal of the operational research society 35 journal of the operations research society of japan 29 sport, business and management: an international journal 25 computers & operations research 24 management decision 22 mathematical modelling 21 mathematical and computer modelling 13 american journal of operations research 12 expert systems with applications 12 journal of multi‐criteria decision analysis 12 journal of systems science and systems engineering 10 ijahp article: nisel, özdemir /ahp/anp in sports: a comprehensive literature review international journal of the analytic hierarchy process 412 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3. 448 table 5 top 10 most highly cited papers (google scholar, november 19, 2016) publication name methodology used branches of sport number of citations (11/19/2016) partovi, f. y., & corredoira, r. a. (2002). quality function deployment for the good of soccer. european journal of operational research, 137(3), 642-656. ahp & qfd & anp football 216 carlsson, c., & walden, p. (1995). ahp in political group decisions: a study in the art of possibilities. interfaces, 25(4), 14-29. ahp ice hockey 97 trninić, s., & dizdar, d. (2000). system of the performance evaluation criteria weighted per positions in the basketball game. collegium antropologicum, 24(1), 217-234. ahp basketball 79 lee, s., & walsh, p. (2011). swot and ahp hybrid model for sport marketing outsourcing using a case of intercollegiate sport. sport management review, 14(4), 361-369. ahp & swot general 61 sinuany-stern, z. (1988). ranking of sports teams via the ahp. journal of the operational research society, 39(7), 661-667. ahp football 35 nishizawa, k. (1995). a consistency improving method in binary ahp. journal of the operations research society of japan, 38(1), 21-33. ahp baseball, tennis 29 lee, s., & ross, s. d. (2012). sport sponsorship decision making in a global market: an approach of analytic hierarchy process (ahp). sport, business and management: an international journal, 2(2), 156-168. ahp general 25 bodin, l., & epstein, e. (2000). who’s on first–with probability 0.4. computers & operations research, 27(3), 205-215. ahp baseball 24 liao, s. k., & chang, k. l. (2009). select televised sportscasters for olympic games by analytic network process. management decision, 47(1), 14-23. anp olympic games 22 vachnadze, r. g., & markozashvili, n. i. (1987). some applications of the analytic hierarchy process. mathematical modelling, 9(3-5), 185-191. ahp relay race 21 3.5 general classification by branches of sports football (or soccer in us originated studies) was found to be the most popular sport in ahp or anp studies during the literature search. table 6 shows a large amount of papers were related to football (13 publications). basketball is the second most popular sport with nine publications, followed by baseball and the olympic games with seven publications each. athletics, hockey and tennis were also found to be significant sports in the literature. american football, badminton, cricket, and physical education branches can also be considered significant with two publications each. in the literature, ahp was ijahp article: nisel, özdemir /ahp/anp in sports: a comprehensive literature review international journal of the analytic hierarchy process 413 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3. 448 used in many other branches of sports. moreover, several studies that were not directly related to any specific branch of sports, but related to the sports field in general were also found in the literature. these studies were categorized as “other sports or general studies” in table 6, and they are mentioned in more detail in the section 3.10 and also presented in table 11. studies that discussed more than one sports branch were counted more than once. table 6 top 10 highly cited papers (google scholar, november 19, 2016) sport branches number of publications sport branches number of publications sport branches number of publications football 13 athletics 4 badminton 2 basketball 9 hockey 4 cricket 2 baseball 7 tennis 3 physical education 2 olympic and national games 7 american football 2 other sports and general studies 16 total: 71 3.6 football studies as mentioned in the previous section, football has a dominant place among ahp studies in the field of sports. thirteen out of sixty-two articles applied ahp to the football field, and topsis was the technique mainly integrated with ahp in football related studies. as can be seen in the table 7, since the year 1987, the ahp technique has been used for performance evaluation of football teams, player selection, selection of golden ball winners, trainer and coach evaluation, ranking teams and clubs, world cup football tournament evaluation, factor evaluation of rules, measuring football performance and comparing the great records. 3.7 basketball studies basketball was found to be the second most popular sport among ahp studies. as presented in the table 8, a large portion of the studies were carried out after the year 2000, and five studies out of nine were performed in 2014. tang and dong (2014) presented an ahp-dea integrated model for selecting the best coaches in basketball, baseball and hockey. cao (2014) also used ahp for coach evaluation in basketball. in another study, balli and korukoglu (2014) utilized fuzzy ahp and topsis for developing a decision support framework for the selection of player candidates in basketball. also, wan et al. (2014) used ahp and input-output analysis for coach evaluation in basketball, american football and baseball, whereas xiong et al. (2014) used ahp, fuzzy synthetic evaluation and entropy method for coach evaluation in basketball. table 8 shows that the ahp technique was also used in basketball for performance evaluation of players, evaluation of basketball teams, performance evaluation in the game , and for selecting and comparing a number of great records in basketball, baseball, football and hockey (yu et al.; sinuany-stern, 2006; trninić and dizdar, 2000; golden and wasil, 1987). ijahp article: nisel, özdemir /ahp/anp in sports: a comprehensive literature review international journal of the analytic hierarchy process 414 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3. 448 3.8 baseball studies in the literature search, baseball was also found to be a popular sport in ahp studies. table 9 shows seven studies out of sixty-two were related to baseball. table 9 also shows that some other multi-criteria decision-making techniques such as dea, topsis and input output analysis were integrated with ahp in the studies conducted in 2014. coach evaluation, pitcher selection, player and team ranking, and performance evaluation of players were popular subjects in ahp related studies in the baseball field. ijahp article: nisel, özdemir /ahp/anp in sports: a comprehensive literature review international journal of the analytic hierarchy process 415 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3. 448 table 7 football studies publication name authors year method problem definition a mathematical model using ahp priorities for soccer player selection: a case study ozceylan, e 2016 ahp soccer player selection who really won the fifa 2014 golden ball award?: what sports can learn from multi-criteria decision analysis mu, e. 2016 ahp selecting the golden ball winner privatization priorities of iranian football clubs from the perspective of experts shahlaee, j., ghorbanalizadeh ghaziani, f. 2015 ahp, topsis, saw measuring privatization priorities of iranian football clubs data processing and modelling with information technology in choosing college best trainer li, p. l 2014 delphi method, ahp choosing a good college trainer an mcdm reflection on the fifa 2014 world cup mu, e. 2014 ahp selecting the golden ball award winner prioritization of the factors effecting privatization in sport clubs: with ahp & topsis methods-emphasis in football salimi, m. et al. 2012 ahp, topsis prioritization of the factors effecting privatization in sport clubs ranking football teams with ahp and topsis methods kiani mavi, r. et al. 2012 ahp, topsis ranking football teams evaluating the performance of iranian football teams utilizing linear programming hamidi, m. et al. 2011 dea, ahp evaluating the performance of iranian football teams utilizing linear programming farsjum, a fuzzy system for ranking sparse judgment matrices: a case study in soccer tournaments. gholamian, m. r. et al. 2007 ahp world cup soccer tournament evaluation importance of hierarchical structure determining tennis performance for modern defensive baseliner yun, y. k. 2005 ahp prioritizing psychological factors determining football performance quality function deployment for the good of soccer fariborz y. partovi, rafael a. corredoira 2002 ahp,qfd , anp prioritizing and designing rule changes for the game of soccer ranking of sports teams via the ahp sinuany-stern, z. 1988 ahp predicting the ranking of the 16 soccer teams of the israeli national league ranking outstanding sports records golden, b. l., wasil, e. a. 1987 ahp selecting and comparing a number of great records ijahp article: nisel, özdemir /ahp/anp in sports: a comprehensive literature review international journal of the analytic hierarchy process 416 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3. 448 table 8 basketball studies publication name authors year method problem definition assessment of coaches using data envelopment analysis and analytical hierarchy process tang, h. c., dong, y. a. 2014 ahp, dea selecting the best coaches coach evaluation method based on ahp cao, x. 2014 ahp coach evaluation development of a fuzzy decision support framework for complex multi‐attribute decision problems: a case study for the selection of skilful basketball players. ballı, s., korukoğlu, s. 2014 fuzzy ahp, topsis developing a decision support framework for player selection evaluation of college coach capacity in usa wan, f. x., hu, d., tian, j. 2014 ahp, input output analysis coach evaluation evaluation system for college coaching legends xiong, f. et al. 2014 ahp, fuzzy synthetic evaluation, entropy method coach evaluation an exploratory study of long-term performance evaluation for elite basketball players kun-tzu yu, zhong-xin su, rui-chen zhuang 2008 ahp long-term technical performance evaluation method for elite basketball players application of the analytic hierarchy process for the evaluation of basketball teams zilla sinuanystern 2006 ahp evaluation of basketball teams system of the performance evaluation criteria weighted per positions in the basketball game s. trnini, and d. dizdar 2000 ahp performance evaluation criteria weighted per positions in the basketball game ranking outstanding sports records golden, b. l., wasil, e. a. 1987 ahp selecting and comparing a number of great records ijahp article: nisel, özdemir /ahp/anp in sports: a comprehensive literature review international journal of the analytic hierarchy process 417 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3. 448 table 9 baseball studies publication name authors year branch of sport method problem definition assessment of coaches using data envelopment analysis and analytical hierarchy process tang, h. c., dong, y. a. 2014 baseball, hockey, basketball ahp, dea selecting the best coaches professional baseball team starting pitcher selection using ahp and topsis methods. chen, c. c., lee, y. t., tsai, c. m. 2014 baseball ahp & topsis selecting starting pitchers evaluation of college coach capacity in usa wan, f. x., hu, d., tian, j. 2014 basketball, american football, baseball ahp, input output analysis coach evaluation who’s on first–with probability 0.4. bodin, l., epstein, e. 2000 baseball ahp ranking players of a baseball team a consistency improving method in binary ahp nishizawa, k. 1995 baseball, tennis ahp ranking teams in league an analytic hierarchy approach to major league baseball offensive performance ratings lanoue, m. r., & revetta, j. j. 1993 baseball ahp performance evaluation of major league players ranking outstanding sports records golden, b. l., wasil, e. a. 1987 baseball, basketball, football,hockey ahp selecting and comparing a number of great records ijahp article: nisel, özdemir /ahp/anp in sports: a comprehensive literature review international journal of the analytic hierarchy process 418 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3. 448 table 10 studies about olympic or national games publication name authors year branch of sport method problem definition ranking countries more reliably in the summer olympics sagir, m., saaty, t. l 2015 olympic games anp evaluating the medals won and the country scores in the 2012 london olympics ranking countries by medal priorities won in the 2014 sochi winter olympics. saaty, t. l. et al. 2014 olympic games ahp ranking countries rankings in sport by pairwise comparison and league table sitarz, s. 2013 olympic games ahp describing a method of creating rankings in sport (medal tables). on emergency of large scale engineering sports events junying, a. 2012 national games fuzzy ahp emergency assessment who won the winter 2010 olympics? a quest into priorities and rankings. saaty, t. l. 2010 olympic games ahp assigning appropriate weights to each type of medal select televised sportscasters for olympic games by analytic by network process sen-kuei liao, kueilun chang 2009 olympic games anp selecting televised sportscasters for olympic games who won the 2008 olympics? a multicriteria decision of measuring intangibles saaty, t. l. 2008 olympic games ahp country ranking in 2008 olympic games http://www.springerlink.com/content/uw452265k482/?p=c1c2bce945544133a6e4da7857268311&pi=0 ijahp article: nisel, özdemir /ahp/anp in sports: a comprehensive literature review international journal of the analytic hierarchy process 419 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3. 448 3.9 studies about olympic games or national games the literature search revealed that ahp and anp methodologies have also been used in several studies as decision-making tools in the field of olympic games or national games. seven studies have been conducted since 2008, and table 10 shows that thomas l. saaty was the author that contributed the most ahp studies in the field of olympic games. sagir and saaty (2015) presented an anp model for evaluating the medals won and the country scores in the 2012 london olympics. saaty et al. (2014) utilized ahp for ranking countries by medal priorities in the 2014 sochi winter olympics. in another study, sitarz (2013) described a method for obtaining ranking of medal tables in the olympic games using ahp. junying (2012) presented a fuzzy ahp model for emergency assessment in national games. saaty (2010) used ahp again for assigning appropriate weights to each type of medal in the winter 2010 olympics. liao and chang (2009) used the anp technique for selecting televised sportscasters for the olympic games. in the earliest study, saaty (2008) used ahp for obtaining a country ranking in the 2008 olympic games. the studies reviewed and shown in table 10 conclude that ahp and anp techniques can be successfully applied to obtain and develop a medal or country ranking system in the olympic games. 3.10 other sports and general studies besides football, basketball, baseball and the olympic games, ahp and anp techniques have also been used in many other areas of sports in the last 25 years. these sport branches are athletics, hockey, tennis, american football, badminton, cricket, physical training, archery, golf, judo, leisure sports, relay race, running, swimming, volleyball and water polo. table 11 summarizes the studies in these branches of sport. during the literature search, it became apparent that some studies were not directly related to any branch of sports. in these general studies, the ahp technique was used for performance evaluation of sport centers, assessing performance of sports buildings, risk evaluation for stadium construction, and factor evaluation for sport marketing outsourcing decision. ijahp article: nisel, özdemir /ahp/anp in sports: a comprehensive literature review international journal of the analytic hierarchy process 420 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3. 448 table 11 studies about other sports and general studies publication name authors year branch of sport method problem definition study on the influencing factors of applying nanotechnology in sport injury repair based on analytic hierarchy process ding, l. 2016 athletics ahp analyzing main factors influencing the application of nanotechnology in athletes injury repair importance of hierarchical structure determining tennis performance for modern defensive baseliner đurović, n., dizdar, d., zagorac, n. 2015 tennis ahp analyzing offensive and defensive tennis priorities for counterpunchers the relative importance of performance factors in korean archery kim, h. b., kim, s. h., so, w. y 2015 archery delphi method, ahp evaluating factors affecting archery performance ahp-neural network based player price estimation in ipl dey, p. k. et all. 2014 cricket ahp, anp player price estimation choose the best coach based on the rheory of grey system zhao, j. 2014 athletics grey system, ahp selecting the best college coaches coaches ranking: a time-irrelevant data mining solution with flexibility and objectivity chen, y. y. et all. 2014 athletics regression tree, decision tree (cart), ahp ranking of coaches establishment on comprehensive evaluation indication system for leisure sports development level zhang, l., wang, z. l. 2014 leisure sports ahp evaluating the present situation of leisure sports development level evaluation of sports center performance using a fuzzy multicriteria decision-making model wang, c. y 2014 general fuzzy ahp evaluate the performance of sports centers evaluation research on competitive ability of volleyball player based on fuzzy set of evaluation index and weight kai, g. 2014 volleyball ahp evaluating the competitive ability of volleyball player identifying the best coach by an improved ahp model xing, j. et all. 2014 american football ahp evaluation of coaches in college ball game selection and concentration strategy in the sports exchange between north and south korea. yu, k. g., park, s. y. 2014 general delphi method, ahp strategy selection and concentration in inter-korean sports exchange the practice of two-phase recommender system for sporting goods lo, w. t. et all. 2014 badminton ahp developing a recommendation technique to improve the selling of sporting goods specialty stores why do some elite players accomplish their grand slam goals while others fail? djurovic, n. et all. 2014 tennis ahp accelerating the player’s development constructing taipei city sports centre performance evaluation model with fuzzy mcdm approach based on views of managers wang, c. y. et all. 2013 general fahp, fanp, dematel recognizing the influential indicators of sport centre business management in taipei city's sports centre ijahp article: nisel, özdemir /ahp/anp in sports: a comprehensive literature review international journal of the analytic hierarchy process 421 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3. 448 table 11 studies about other sports and general studies (continued) publication name authors year branch of sport method problem definition research on the preventive measures of sports injury based on causality diagram and analytic hierarchy process zhou, z. x 2013 physical education causality diagram, ahp analyzing the factors of injury accident in physical education teaching process the green energy-saving design of stadium chen, x. j. 2013 general ahp assessing environmental performance of green sports building efficiency analysis of provincial departments of physical education in iran soleimani-damaneh j. et all. 2012 physical education dea, ahp analyzing the performance of physical education organizations in iran. forecasting rounds of golf parrott, s. et all. 2012 golf ahp making adjustments to statistical forecasts of golf demand at a course sport sponsorship decision making in a global market: an approach of analytic hierarchy process (ahp). lee, s., ross, s. d. 2012 general ahp identifying the decision making factors of sport sponsorship in the global market context statistical based multi-criteria decision making analysis for performance measurement of batsmen in indian premier league dey, p. k., et all. 2012 cricket topsis, ahp, anova measuring the performance of batsmen fuzzy analytic hierarchy in project risk management-new stadium construction project in weifang city zhang, y., liu, c. j 2011 general fuzzy ahp construction project risk assessment swot and ahp hybrid model for sport marketing outsourcing using a case of intercollegiate sport lee, s., walsh, p. 2011 general ahp, swot examining sport marketing outsourcing decision-making factors using a swot and ahp combined model evaluation of individual and team judo strengths using ahp technique and team competition data hirose, n. et al. 2010 judo ahp evaluating individual and team judo strengths developing computer aided model for selecting talent players in badminton ali agilonu, serkan balli 2009 badminton fuzzy ahp selecting talent players in badminton efficient management design for swimming exercise treatment kim, k. et al. 2009 swimming fuzzy ahp design a swimming-based exercise treatment management system experts opinion about system of the performance evaluation criteria weighted per positons in the water polo game hraste, m., dizdar, d., trninić, v. 2008 water polo ahp water polo players’ actual quality evaluation should a new arena be built in the city of pittsburgh? stacy monarkoa, et all. 2007 hockey anp eecision analysis for building a new arena using geoinformatics to identify suitable middle to long distance athletics' rraining sites in kenya john kiema et al. 2007 running gis, ahp selecting suitable training sites for endurance running ahp in political group decisions: a study in the art of possibilities carlsson, c., walden, p. 1995 ice hockey ahp finding the optimal location for a new ice-hockey arena some applications of the analytic hierarchy process vachnadze, r. g., markozashvili, n. i. 1987 relay race ahp electing team members and their distribution for relay-race stages ijahp article: nisel, özdemir /ahp/anp in sports: a comprehensive literature review international journal of the analytic hierarchy process 422 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3. 448 4. conclusion sporting has become one of the fastest growing areas of industry and entertainment worldwide. in almost all competitive sports branches, it is obligatory to consider all success factors in order to be successful or to win the game. developing tactical strategies and kicking styles, managing time wisely during a game, changing players in the proper period, tactical movements, making decisions quickly, selecting proper players for positions, and building a technical game structure, all require professional decision making abilities. therefore, since the 1970’s, managers, professionals and fans have applied many statistical tools and models in order to analyze sports data systematically in many areas of sports. as a result of the comprehensive literature analysis, it was concluded that ahp and also anp techniques were successfully used in many areas of sport such as football (or soccer), basketball, baseball, the olympic games, athletics, hockey, tennis, american football, badminton, cricket, physical training, archery, golf, judo, leisure sports, relay race, running, swimming, volleyball and water polo. the methods were used mainly for performance measurement, team or player evaluation and ranking, prediction of failure, coach evaluation, rule evaluation in a game, measuring technical efficiency, and for sports marketing. there has also been a significant increase in the number of ahp studies in the sports field especially in the last five years, and it is estimated that this trend will continue as more statistical data will be available for researchers, and the worldwide popularity of many sports branches will continue to grow. we hope that this literature review will guide researchers towards the advancement of ahp and anp applications in many sport fields. ijahp article: nisel, özdemir /ahp/anp in sports: a comprehensive literature review international journal of the analytic hierarchy process 423 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3. 448 references agılönü, a., & ballı, s. (2009). developing computer aided model for selecting players in badminton. journal of human sciences, 6(2), 292-301. apostolou, b., & hassell, j. m. (1993). an overview of the analytic hierarchy process and its use in accounting research. journal of accounting literature, 12, 1. ballı, s., & korukoğlu, s. (2014). development of a fuzzy decision support framework for complex multi‐attribute decision problems: a case study for the selection of skilful basketball players. expert systems, 31(1), 56-69. doi: 10.1111/exsy.12002 bodin, l., & epstein, e. (2000). who’s on first–with probability 0.4. computers & operations research, 27(3), 205-215. doi:10.1016/s0305-0548(99)00002-7 bruno, g., esposito, e., genovese, a., & passaro, r. (2012). ahp-based approaches for supplier evaluation: problems and perspectives. journal of purchasing and supply management, 18(3), 159-172. doi: http://dx.doi.org/10.1016/j.pursup.2012.05.001 cao, x. (2014). coach evaluation method based on ahp. in applied mechanics and materials, 551, 722-726. doi: 10.4028/www.scientific.net/amm.551.722 carlsson, c., & walden, p. (1995). ahp in political group decisions: a study in the art of possibilities. interfaces, 25(4), 14-29. doi: http://dx.doi.org/10.1287/inte.25.4.14 chen, c. c., lee, y. t., & tsai, c. m. (2014). professional baseball team starting pitcher selection using ahp and topsis methods. international journal of performance analysis in sport, 14(2), 545-563. chen, x. j. (2013). the green energy-saving design of stadium. advanced materials research, 724, 1571-1574. doi: 10.4028/www.scientific.net/amr.724-725.1571 chen, y. y., zhang, m. j., shen, z., & chen, j. x. (2014). coaches ranking: a timeirrelevant data mining solution with flexibility and objectivity. advanced materials research, 989, 5010-5013. doi: 10.4028/www.scientific.net/amr.989-994.5010 de fsm russo, r., & camanho, r. (2015). criteria in ahp: a systematic review of literature. procedia computer science, 55, 1123-1132. doi:10.1016/j.procs.2015.07.081 dey, p. k., banerjee, a., ghosh, d. n., & mondal, a. c. (2014). ahp-neural network based player price estimation in ipl. international journal of hybrid information technology, 7(3), 15-24. doi: http://dx.doi.org/10.14257/ijhit.2014.7.3.03 http://dx.doi.org/10.1016/s0305-0548(99)00002-7 http://dx.doi.org/10.1016/j.pursup.2012.05.001 http://dx.doi.org/10.1287/inte.25.4.14 http://dx.doi.org/10.1016/j.procs.2015.07.081 ijahp article: nisel, özdemir /ahp/anp in sports: a comprehensive literature review international journal of the analytic hierarchy process 424 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3. 448 dey, p. k., mondal, a. c., & ghosh, d. n. (2012). statistical based multi-criteria decision making analysis for performance measurement of batsmen in indian premier league. international journal of advanced research in computer science, 3(4), 51-57. ding, l. (2016). study on the influencing factors of applying nanotechnology in sport injury repair based on analytic hierarchy process. journal of computational and theoretical nanoscience, 13(7), 4529-4532. doi: https://doi.org/10.1166/jctn.2016.5316 djurovic, n., stanisic, l., & sbarro, f. why do some elite players accomplish their grand slam goals while others fail?. itf coaching and sport science review, 22(64), 11-13. đurović, n., dizdar, d., & zagorac, n. (2015). importance of hierarchical structure determining tennis performance for modern defensive baseliner. collegium antropologicum, 39(supplement 1), 103-108. gholamian, m. r., fatemi ghomi, s. m. t., & ghazanfari, m. (2007). farsjum, a fuzzy system for ranking sparse judgment matrices: a case study in soccer tournaments. international journal of uncertainty, fuzziness and knowledge-based systems, 15(01), 115-129. doi: http://dx.doi.org/10.1142/s0218488507004406 golden, b. l., & wasil, e. a. (1987). ranking outstanding sports records. interfaces, 17(5), 32-42. doi: http://dx.doi.org/10.1287/inte.17.5.32 hamidi, m., sajadi, h., & soleimani-damaneh, j. (2011). evaluating the performance of iranian football teams utilizing linear programming. american journal of operations research, 1(02), 65. doi:10.4236/ajor.2011.12010 hirose, n., nakamura, m., hirotsu, n., yoshimura, m., suganami, m., & maekawa, n. (2010). evaluation of individual and team judo strengths using ahp technique and tteam competition data. journal of quantitative analysis in sports, 6(4). doi: https://doi.org/10.2202/1559-0410.1153 ho, w. (2008). integrated analytic hierarchy process and its applications–a literature review. european journal of operational research, 186(1), 211-228. doi: http://dx.doi.org/10.1016/j.ejor.2007.01.004 hraste, m., dizdar, d., & trninić, v. (2008). experts opinion about system of the performance evaluation criteria weighted per positons in the water polo game. collegium antropologicum, 32(3), 851-861. ishizaka, a., & labib, a. (2011). review of the main developments in the analytic hierarchy process. expert systems with applications, 38(11), 14336-14345. doi: http://dx.doi.org/10.1016/j.eswa.2011.04.143 https://doi.org/10.1166/jctn.2016.5316 http://dx.doi.org/10.1142/s0218488507004406 http://dx.doi.org/10.1287/inte.17.5.32 https://doi.org/10.2202/1559-0410.1153 http://dx.doi.org/10.1016/j.ejor.2007.01.004 http://dx.doi.org/10.1016/j.eswa.2011.04.143 ijahp article: nisel, özdemir /ahp/anp in sports: a comprehensive literature review international journal of the analytic hierarchy process 425 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3. 448 junying, a. (2012). on emergency of large scale engineering sports events. systems engineering procedia, 4, 416-423. doi:10.1016/j.sepro.2012.01.005 kai, g. (2014). evaluation research on competitive ability of volleyball player based on ruzzy set of evaluation index and weight. journal of applied sciences, 14(20), 2547. doi: 10.3923/jas.2014.2547.2551 kiani mavi, r., kiani mavi, n., & kiani, l. (2012). ranking football teams with ahp and topsis methods. international journal of decision sciences, risk and management, 4(12), 108-126. doi: http://dx.doi.org/10.1504/ijdsrm.2012.046620 kiema, j., kipkemei, a., karanja, f., & musyoka, s. (2007). using geoinformatics to identify suitable middle to long distance athletics' training sites in kenya. international journal of sports science & coaching, 2(4), 473-484. doi:10.1260/174795407783359669 kim, h. b., kim, s. h., & so, w. y. (2015). the relative importance of performance factors in korean archery. the journal of strength & conditioning research, 29(5), 12111219. doi: 10.1519/jsc.0000000000000687 kim, k., kyung, t., kim, w., shin, c., song, y., lee, m. y., ... & cho, y. (2009). efficient management design for swimming exercise treatment. the korean journal of physiology & pharmacology, 13(6), 497-502. doi: https://doi.org/10.4196/kjpp.2009.13.6.497 lanoue, m. r., & revetta, j. j. (1993). an analytic hierarchy approach to major league baseball offensive performance ratings. mathematical and computer modelling, 17(4-5), 195-209. doi:10.1016/0895-7177(93)90188-5 lee, s., & ross, s. d. (2012). sport sponsorship decision making in a global market: an approach of analytic hierarchy process (ahp). sport, business and management: an international journal, 2(2), 156-168. doi: http://dx.doi.org/10.1108/20426781211243999 lee, s., & walsh, p. (2011). swot and ahp hybrid model for sport marketing outsourcing using a case of intercollegiate sport. sport management review, 14(4), 361369. doi: http://dx.doi.org/10.1016/j.smr.2010.12.003 li, p. l. (2014). data processing and modelling with information technology in choosing college best trainer. advanced materials research, 978, 221-225. doi: 10.4028/www.scientific.net/amr.978.221 liao, s. k., & chang, k. l. (2009). select televised sportscasters for olympic games by analytic network process. management decision, 47(1), 14-23. doi: http://dx.doi.org/10.1108/00251740910929678 http://dx.doi.org/10.1016/j.sepro.2012.01.005 https://doi.org/10.4196/kjpp.2009.13.6.497 http://dx.doi.org/10.1016/0895-7177(93)90188-5 http://dx.doi.org/10.1108/20426781211243999 http://dx.doi.org/10.1016/j.smr.2010.12.003 ijahp article: nisel, özdemir /ahp/anp in sports: a comprehensive literature review international journal of the analytic hierarchy process 426 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3. 448 liberatore, m. j., & nydick, r. l. (2008). the analytic hierarchy process in medical and health care decision making: a literature review. european journal of operational research, 189(1), 194-207. doi: http://dx.doi.org/10.1016/j.ejor.2007.05.001 lo, w. t., chang, y. s., sheu, r. k., & jung, j. (2014). the practice of two-phase recommender system for sporting goods. malaysian journal of computer science, 27(2). monarko, s., lambert, k., & sigmund, m. (2007). should a new arena be built in the city of pittsburgh?. mathematical and computer modelling, 46(7), 1160-1182. doi: http://dx.doi.org/10.1016/j.mcm.2007.03.019 mu, e. (2014). an mcdm reflection on the fifa 2014 world cup. international journal of the analytic hierarchy process, 6(1), 124-131. doi: http://dx.doi.org/10.13033/ijahp.v6i1.241 mu, e. (2016). who really won the fifa 2014 golden ball award?: what sports can learn from multi-criteria decision analysis. international journal of sport management and marketing, 16(3-6),239-258. doi: http://dx.doi.org/10.1504/ijsmm.2016.077933 nishizawa, k. (1995). a consistency improving method in binary ahp. journal of the operations research society of japan, 38(1), 21-33. ozceylan, e. (2016). a mathematical model using ahp priorities for soccer player selection: a case study. south african journal of industrial engineering, 27(2), 190-205. doi: http://dx.doi.org/10.7166/27-2-1265 parrott, s., stamey, j., & burcham, t. (2012). forecasting rounds of golf. foresight: the international journal of applied forecasting, 24. partovi, f. y., & corredoira, r. a. (2002). quality function deployment for the good of soccer. european journal of operational research, 137(3), 642-656. doi: http://dx.doi.org/10.1016/s0377-2217(01)00072-8 saaty, t. l. (2001). decision making with dependence and feedback: the analytic network process. pittsburgh. rws publications, 7, 557-570. saaty, t. l. (2008). who won the 2008 olympics? a multicriteria decision of measuring intangibles. journal of systems science and systems engineering, 17(4), 473-486. doi: 10.1007/s11518-008-5092-8 saaty, t. l. (2010). who won the winter 2010 olympics? a quest into priorities and rankings. journal of multi‐criteria decision analysis, 17(1‐2), 25-36. doi: 10.1002/mcda.451 http://dx.doi.org/10.1016/j.ejor.2007.05.001 http://dx.doi.org/10.1016/j.mcm.2007.03.019 http://dx.doi.org/10.13033/ijahp.v6i1.241 http://dx.doi.org/10.7166/27-2-1265 http://dx.doi.org/10.1016/s0377-2217(01)00072-8 ijahp article: nisel, özdemir /ahp/anp in sports: a comprehensive literature review international journal of the analytic hierarchy process 427 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3. 448 saaty, t. l., liu, x., & sanserino, m. (2014). ranking countries by medal priorities won in the 2014 sochi winter olympics. annals of data science, 1(2), 151-172. doi: 10.1007/s40745-014-0012-x sagir, m., & saaty, t. l. (2015). ranking countries more reliably in the summer olympics. international journal of the analytic hierarchy process, 7(3), 589-610. doi: http://dx.doi.org/10.13033/ijahp.v7i3.341 salimi, m., soltanhosseini, m., padash, d., & khalili, e. (2012). prioritization of the factors effecting privatization in sport clubs: with ahp & topsis methods-emphasis in football. international journal of academic research in business and social sciences, 2(2), 102. shahlaee, j., & ghorbanalizadeh ghaziani, f. (2015). privatization priorities of iranian football clubs from the perspective of experts. annals of applied sport science, 3(2), 5768. shim, j. p. (1989). bibliographical research on the analytic hierarchy process (ahp). socio-economic planning sciences, 23(3), 161-167. doi:10.1016/00380121(89)90013-x sinuany-stern, z. (1988). ranking of sports teams via the ahp. journal of the operational research society, 39(7), 661-667. sinuany-stern, z., israeli, y., & bar-eli, m. (2005). application of the analytic hierarchy process for the evaluation of basketball teams. international journal of sport management and marketing, 1(3), 193-207. doi: 10.1057/jors.1988.112 sipahi, s., & timor, m. (2010). the analytic hierarchy process and analytic network process: an overview of applications. management decision, 48(5), 775-808. doi: http://dx.doi.org/10.1108/00251741011043920 sitarz, s. (2013). rankings in sport by pairwise comparison and league table. international journal of modern education and computer science, 5(12), 24. doi: 10.5815/ijmecs soleimani-damaneh, j., soleimani-damaneh, m., & hamidi, m. (2012). efficiency analysis of provincial departments of physical education in iran. international journal of information technology & decision making, 11(05), 983-1008. doi: http://dx.doi.org/10.1142/s0219622012500290 subramanian, n., & ramanathan, r. (2012). a review of applications of analytic hierarchy process in operations management. international journal of production economics, 138(2), 215-241. doi: http://dx.doi.org/10.1016/j.ijpe.2012.03.036 http://dx.doi.org/10.13033/ijahp.v7i3.341 http://dx.doi.org/10.1016/0038-0121(89)90013-x http://dx.doi.org/10.1016/0038-0121(89)90013-x http://dx.doi.org/10.1108/00251741011043920 http://dx.doi.org/10.1142/s0219622012500290 http://dx.doi.org/10.1016/j.ijpe.2012.03.036 ijahp article: nisel, özdemir /ahp/anp in sports: a comprehensive literature review international journal of the analytic hierarchy process 428 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3. 448 tang, h. c., & dong, y. a. (2014). assessment of coaches using data envelopment analysis and analytical hierarchy process. applied mechanics and materials,. 687, 15601563. doi: 10.4028/www.scientific.net/amm.687-691.1560 trninić, s., & dizdar, d. (2000). system of the performance evaluation criteria weighted per positions in the basketball game. collegium antropologicum, 24(1), 217-234. vachnadze, r. g., & markozashvili, n. i. (1987). some applications of the analytic hierarchy process. mathematical modelling, 9(3-5), 185-191. doi:10.1016/02700255(87)90475-1 vaidya, o. s., & kumar, s. (2006). analytic hierarchy process: an overview of applications. european journal of operational research, 169(1), 1-29. doi: http://dx.doi.org/10.1016/j.ejor.2004.04.028 vargas, l. g. (1990). an overview of the analytic hierarchy process and its applications. european journal of operational research, 48(1), 2-8. doi:10.1016/03772217(90)90056-h wan, f. x., hu, d., & tian, j. (2014). evaluation of college coach capacity in usa. advanced materials research, 998, 1693-1696. doi: 10.4028/www.scientific.net/amr.998-999.1693 wang, c. y. (2014). evaluation of sports denter performance using a fuzzy multi-criteria decision-making model. journal of testing and evaluation, 43(6), 1-11. doi: 10.1520/jte20130235 wang, c. y., tsai, p. h., & zheng, h. (2013). constructing taipei city sports centre performance evaluation model with fuzzy mcdm approach based on views of managers. mathematical problems in engineering. 2013, 1-13. doi: http://dx.doi.org/10.1155/2013/138546 xing, j., zhao, c., wang, x. l., & xiang, n. (2014). identifying the best coach by an improved ahp model. abstract and applied analysis, 2014, 1-7. doi: http://dx.doi.org/10.1155/2014/971648 xiong, f., ding, w., & li, j. (2014). evaluation system for college coaching legends. umap journal, 35. yu, k. g., & park, s. y. (2015). selection and concentration strategy in the sports exchange between north and south korea. journal of asian public policy, 8(2), 215-229. doi: http://dx.doi.org/10.1080/17516234.2014.959249 http://dx.doi.org/10.1016/0270-0255(87)90475-1 http://dx.doi.org/10.1016/0270-0255(87)90475-1 http://dx.doi.org/10.1016/j.ejor.2004.04.028 http://dx.doi.org/10.1016/0377-2217(90)90056-h http://dx.doi.org/10.1016/0377-2217(90)90056-h http://dx.doi.org/10.1155/2013/138546 http://dx.doi.org/10.1155/2014/971648 http://dx.doi.org/10.1080/17516234.2014.959249 ijahp article: nisel, özdemir /ahp/anp in sports: a comprehensive literature review international journal of the analytic hierarchy process 429 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3. 448 yu, k. t., su, z. x., & zhuang, r. c. (2008). an exploratory study of long-term performance evaluation for elite basketball players. international journal of sports science and engineering, 2(4), 195-203. yun, y. k. (2005). importance of hierarchical structure of psychological factors determining football performance. international journal of applied sports sciences, 17(2). zhang, l., & wang, z. l. (2014). establishment on comprehensive evaluation indication system for leisure sports development level. applied mechanics and materials , 513, 31473150). doi: 10.4028/www.scientific.net/amm.513-517.3147 zhang, y., & liu, c. j. (2011). fuzzy analytic hierarchy in project risk management-new stadium construction project in weifang city. advanced materials research, 243, 63626368. doi: 10.4028/www.scientific.net/amr.243-249.6362 zhao, j. (2014). choose the best coach based on the theory of grey system. advanced materials research, 998, 1757-1760. doi: 10.4028/www.scientific.net/amr.998-999.1757 zhou, z. x. (2013). research on the preventive measures of sports injury based on causality diagram and analytic hierarchy process. applied mechanics and materials, 380, 1838-1842.doi: 10.4028/www.scientific.net/amm.380-384.1838 ijahp article: kumar, luthra, haleem/critical factors important for effective industry-institute interactions (iii): an indian perspective international journal of the analytic hierarchy process 2 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.319 critical factors important for effective industryinstitute interactions (iii): an indian perspective sanjay kumar international institute of technology and management murthal, india skbhardwaj19711971@gmail.com sunil luthra lecturer, department of mechanical engineering government polytechnic, jhajjar124104, haryana, india sunilluthra1977@gmail.com abid haleem professor, department of mechanical engineering, faculty of engineering and technology, jamia millia islamia, delhi-110016, delhi, india haleem.abid@gmail.com abstract the management of industry-institute interactions (iii) has been found to be an important aspect in today’s market. in this paper an attempt has been made to identify critical factors (cfs) and rank these for effective iii from an indian perspective. the present paper is based on two research stages. initially, an extensive literature review was conducted to identify cfs of iii. forty three cfs for effective iii were identified and categorized into nine dimensions based on input from experts. in the second stage, the analytical hierarchy process (ahp) was utilized to rank these dimensions and cfs for effective iii from an indian perspective. paired comparisons in ahp were based on the opinions of experts (selected from academia and industry). the proposed decision framework may offer some valuable guidelines for policy makers to develop their plan of action in terms of design of short and long term policies and strategies to promote effective iii in india. a sensitivity analysis was also performed to investigate the robustness and priority ranking stability of cfs in the proposed framework. this paper may help india to achieve and manage effective iii leading to potential economic, social, political, cultural and environmental advantages. keywords: critical factors (cfs), indian perspective, industry-institute interactions (iii), analytical hierarchy process (ahp), research and development (r & d), sensitivity analysis 1. introduction a limited number of organizations benefit directly from universities/institutions (as a source of information/knowledge) for their innovative activities and/or the development of innovative products and processes through appropriately managing research and mailto:skbhardwaj19711971@gmail.com mailto:sunilluthra1977@gmail.com mailto:haleem.abid@gmail.com ijahp article: kumar, luthra, haleem/critical factors important for effective industry-institute interactions (iii): an indian perspective international journal of the analytic hierarchy process 3 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.319 development projects (laursen & salter, 2004). in fact, organizations may acquire knowledge and technology from many external sources and these sources may include competing organizations, research organizations, government laboratories, industryinstitute research associations and universities (santoro & chakrabarti, 2002). industrial organizations use a variety of relationships with university/institute research centers to accomplish various knowledge and innovation based targets and goals. large organizations have more intense knowledge transfer capabilities and research support relationships (to strengthen skills and knowledge) for gaining access to university/institute facilities to advance technologies (santoro & chakrabarti, 2002).this may be for the simple reason that they have a large amount of funds available/allocated for this. industry–university/institute alliances may represent an evolving trend for advancing knowledge and new technologies supporting the idea of establishing effective and efficient industry-institute interactions (chakrabarti & lester, 2002). the journey of interactions and cooperation among industry and institutes has taken different forms which include problem solving, curriculum development, study/industrial visits, scholarships, and apprenticeship training and incubation centers. however, industry has not been involved in taking sustainable financial, technical and operational risks in the design, financing and building and operation of educational projects (majumdar, 2008). institute-industry collaborations have been encouraged in many countries by policy-makers, and institutes may play a key role as the economic/technical/knowledge/ innovations facilitator to create an altogether new mechanism for economic development (bozeman, 2000; etzkowitz & leydesdorff, 2000). 1.1 innovation and knowledge management innovation may be visualized as the successful creation, development, implementation and use of knowledge for new or structurally improved products, processes, services and/or organizational forms (greenhalgh et al., 2006). on the other hand, knowledge management has been one of the hottest topics over the past few decades in both the industry and information research worlds (more specifically in universities/institutes), but there is no universal definition of this term, just as there has not been consensus about what constitutes knowledge in the first place (alrawi & alrawi 2011). innovation and knowledge management may play an effective role in motivating, encouraging and channeling industry-institute interactions to gain value for customers, organizations, supply chains, and society (chen & huang, 2007). 1.2 problem areas in industry-institute interactions in india in india, industry-institute interactions are not noticeably observable, and therefore it is necessary to have better (in terms of quality and quantity) interactions among institutions and industry. these interactions and collaborations should have a greater impact and influence on course frame work, curriculum development, student exposure to the industrial atmosphere in the form of industrial visits, research associations, and subsequent placement of young graduates/post graduates/doctorates in industries across the country. with the advent of globalization and the opening up of the indian economy, competition among industries has become significantly stiff. india’s vast network of academic infrastructure churns out over 2 million graduates annually. however, there are growing concerns about parts of the existing available talent pool being unsuitable for employment due to a skill gap between graduate’s skill level and requirements of ijahp article: kumar, luthra, haleem/critical factors important for effective industry-institute interactions (iii): an indian perspective international journal of the analytic hierarchy process 4 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.319 industry (national knowledge commission, 2008). it has become apparent that there is an urgent need to prepare engineering/professional students so that they are skilled and employable in multinational organizations by exposing them to newer technologies and making them aware of various engineering/professional/management methodologies. this may only be achieved well by bridging the gap between industry and the academic institutes (wallin et al., 2014). in india, several problem areas exist in attaining effective industry-institute interactions and collaborations among industries and institutes and experts from academia and industry (kulkarni et al., 2011).these include lack of vision, lack of appropriate policy planning and implementation, a more theoretical approach to business, management, technical/engineering and professional education, outdated syllabi and less frequent syllabi modification, lack of regular interaction and vital contribution of industry experts in the design of syllabi to match recent industry requirements, lack of industrial exposure of faculty, insufficient inclusion of professionals from industry in research programs, lack of invitations to engineers, executives and experts from industry to deliver lectures in institutes (which may go a long way to ensure and establish linkages between academia and industry), limited use of practical methodologies (like role playing, brainstorming, delphi, case studies etc.), copying irrelevant western business practices without consideration of the indian mind set and culture, inhibitions and reservations, a total divide between industry and educational institutes, and insufficient funding. the above problems need to be appropriately addressed, however to do this an open, conducive environment needs to be provided to the many individuals in academic institutions that have bright and innovative ideas that may be useful for industries (vest, 2005). in our study, we have considered universities (government and private); colleges (government and private) offering engineering, business administration and economics, commerce, arts, and other technical and professional courses; schools (government and private); and other research organizations. we have used a single word, “institute”, for all of these throughout the rest of this paper. a brief summary of the organizations involved in research, directly or indirectly, in india has been given in appendix i. 1.3 need and objectives of the research today, there continue to be relevant compelling reasons for industrial organizations and institutes to work together, which include access to highly trained students, facilities and faculty as well as an enhanced image when collaborating with a prominent academic institution (fombrun, 1996). close interaction between the institute and industry/enterprise is seen as the platform for showcasing best practices, latest technological advancements and their implementation, and impact on the industry (majumdar, 2008). the problem areas discussed gave direction to the present research. the objectives of the research are:  to identify critical factors (cfs) important for effective industryinstitute interactions from an indian perspective;  to prioritize the importance of identified cfs for effective industryinstitute interactions to develop a decision framework;  to check the sensitivity of the obtained priority results of the identified cfs. a literature review was performed to identify relevant critical factors for effective industryinstitute interactions. a literature review is an integral part of any research to ijahp article: kumar, luthra, haleem/critical factors important for effective industry-institute interactions (iii): an indian perspective international journal of the analytic hierarchy process 5 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.319 identify the conceptual content of the field, and gives guidance towards theory development (luthra et al., 2014). in the present research, we propose to use an ahp approach for prioritizing the importance of various cfs for effective iii in india. the ahp is a multi-criterion decision making (mcdm) approach which was developed by thomas l. saaty in 1980. it assists in the decomposition, organization and analysis of a complex problem, and its conversion into a multi-level hierarchical structure comprised of an objective function, criteria and sub-criteria (saaty, 1980; luthra et al., 2015). the results may be used to compare and rank the alternatives and, hence, assist the decision maker in making a choice (saaty, 2008; kumar et al., 2009). 2. literature review the university may play an increasingly important role in the innovation of knowledgebased societies as explained in the triple helix theory (etzkowitz & leydesdorff, 2000).this triple-helix model of university-industry-government interactions and its contribution to entrepreneurship, and economic and social development has been outlined in etzkowitz (2011). different possible resolutions of the relations and interactions among these three spheres may help to generate alternative strategies for economic growth, social transformation and sustainable development (etzkowitz and leydesdorff, 2000). the evolution of innovation systems and research and probable solutions for effective university–industry interactions and relations may be reflected in the varying institutional arrangements of university–industry–government relations (wixted & holbrook, 2013). some important contributions for effective industry-institute interactions from research papers published from 2007 to 2014 are presented in table 1. ijahp article: kumar, luthra, haleem/critical factors important for effective industry-institute interactions (iii): an indian perspective international journal of the analytic hierarchy process 6 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.319 table 1 important contributions in the area of effective industry-institute interactions researcher/s and their contributions bercivitz and feldman (2007): this paper examined how innovation strategy influences firm’s level of involvement with university-based research. the results suggested that firms with internal r&d strategies more heavily weighted toward exploratory activities allocate a greater share of their r&d resources to exploratory university research and develop deeper multifaceted relationships with their university research partners. tether and tajar (2008): this paper explored the use of specialist knowledge providers as sources of information in the innovation activities of manufacturing and service firms. perkmann and walsh (2009): this paper analyzed the impact of university–industry relationships on public research. their learning-centered interpretation qualified the notion of entrepreneurial science as a driver of applied university–industry collaboration. jia et al. (2010): this paper suggested cellular automaton model to analyze the diffusion of innovation in industrial clusters. the result of the simulation suggested that industry-universityinstitute cooperation is better especially when enterprise’s r&d capability is very weak or the technology is very difficult. intensifying the external support from local government, industry associations and financial institutions has a positive effect on the diffusion of innovation in industrial clusters. lai (2011): this paper analyzed the willingness to engage in technology transfer (tt) in industry university collaborations (iucs) from three vantage points: technology transferor (university), technology transferee (industry), and the tt intermediary institute. from the vantage point of universities, this study showed that the “transferor's incentive” and “capability of transferor” variables positively influence willingness to participate in tt in an iuc. from the vantage point of industry, the results indicated that “capability of transferee” and “incentive for establishing technological resources” have major influence on willingness to participate in tt in an iuc. from the vantage point of tt intermediary institutes, the results showed that “intermediary's fundamental resources” and “intermediary's transferring process” have a positive impact on willingness to participate in tt. perkmann et al. (2011): this paper investigated how university research quality shapes their engagement with industry. they found that in technology-oriented disciplines, departmental faculty quality is positively related to industry involvement. in the medical and biological sciences they found a positive effect of departmental faculty quality but establish that this does not apply to star scientists. in the social sciences they found some support for a negative relationship between faculty quality and particularly the more applied forms of industry involvement. the implication for science policy makers and university managers was also suggested. vauterin et al. (2012): the paper employed interpretive phenomenological research methods, an investigative case-based approach to study the boundaries, boundary roles and processes involved in university-industry collaborative interaction in the context of finnish international higher education. the findings suggested that conceptualization of the university-industry boundaryspanning processes in international higher education needs to be extended to incorporate elements concerning the power, impact and management of the boundary roles. a better understanding and adequate managing of the boundary roles may help to decrease the perceived market demand uncertainty surrounding international higher education. xias and jin (2012): this paper analyzed the impact of organizational features and context variables on the proneness of university-industry cooperation in the r&d and engineering process. they suggested that the probability of cooperation with university significantly depends on various organizational factors including firm size and enterprise absorptive capacity. they also suggested that the actual pay tax the collection of business-to-market and technical information, and market position are also important driving factors to the engineering research cooperation between enterprises and universities, but their influences vary with the type of product innovation and process innovation. http://www.sciencedirect.com/science/article/pii/s0048733308000784 http://www.sciencedirect.com/science/article/pii/s0048733308000784 ijahp article: kumar, luthra, haleem/critical factors important for effective industry-institute interactions (iii): an indian perspective international journal of the analytic hierarchy process 7 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.319 researcher/s and their contributions şendoğduand diken (2013): industry sector needs to be in collaboration with university in order to reach the information that will contribute its performance. this paper investigated the level at which the university and industry collaboration exists in konya province, to ascertain the frequency of collaboration subjects and the problems encountered during this collaboration. salleh and omar (2013): this paper proposed a successful model for university-industry collaboration focusing on the interaction between university, government, and industry. they emphasized the role of university, government, and industry to work mutually to achieve successful collaboration. guan and zhao (2013): this paper investigated the effects of multiplicative interaction between clustering and reach on members' knowledge creation and patent value based on complex network analysis in nano bio pharmaceuticals field. fiaz (2013): this paper investigated the growing phenomenon of university-industry (u–i) collaboration in high-tech strategic projects in china. he explored the u–i collaboration patterns among chinese universities and high-tech industry on the basis of quantitative analysis using a research instrument. the results showed that u–i collaboration is established and encouraged due to factors such as: r&d tendency, r&d risks and r&d promotion factors such as state incentives. liew et al. (2013): this paper presented the strategic and tactical approaches on university and industry collaboration in the contemporary commercial climate. this paper provided a commercial approach which may be adopted by the university in propagating the collaboration resulting in a win-win situation. jung (2014): this paper examined the national nanotechnology initiative of u.s. government science and technology (s&t) program, affects the nature of university research in nanotechnology. the findings suggested that, at least in the case of the nni, targeted government s&t programs may increase the efficiency of university research, but potentially do so at a price. 3. critical factors (cfs) identified for effective industry-institute interactions (iii) to accomplish the task of identifying critical factors (cfs) for effective iii, a literature survey was conducted by searching various key words like critical factors of iii in india, factors important for successful iii, key factors for iii in india etc. google search and google scholar search engines were used for collecting supporting literature from national and international journals, proceedings of national and international conferences, authentic websites of government and non-government organizations, and reports published by various government/non-government departments and research organizations. forty three cfs for effective iii were identified from the literature review. nine experts (five from academia and four from the manufacturing industry) participated in an idea engineering workshop to validate these critical factors. in the first session of the workshop, the utility of the identified cfs was confirmed, and after a long brainstorming session, the forty three cfs were classified into nine dimensions through expert’s judgments based upon their similarity. these dimensions are as follows: financial interactions, government/nation/regulatory perspectives, technical interactions, markets and customer’s interactions, social interactions, green and environmental interactions, intellectual properties perspectives, human resource interactions and motivation from mutual benefits. in the second session of the workshop, experts were asked to rate the identified dimension and cfs (other details regarding data collection have been provided in section 4).the identified dimensions of cfs and the cfs that are important for effective iii from an indian perspective are shown in table 2. ijahp article: kumar, luthra, haleem/critical factors important for effective industry-institute interactions (iii): an indian perspective international journal of the analytic hierarchy process 8 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.319 table 2 critical factors important for effective iii in an indian perspective cfs s.n. dimensions of cfs identified for effective iii(reference) cfs s.n. cfs identified for effective iii references 1 financial interactions (gidley et al., 2010; fiaz, 2013) 1.1 maintained/improved business margins korres (2012) 1.2 increased market share czarnitzki et al. (2011) 1.3 increase of shareholder returns and goodwill muscio et al. (2013) 1.4 higher profits korres (2012) 1.5 incentives and subsidies salter and martin (2001) 1.6 fund allocation for nation’s r&d mowery (2012); fiaz (2013) 1.7 fund allocation for industry’s/institute’s r&d santoro and chakrabarti (2002) 2 government/nation/regulatory perspectives (bozeman, 2000) 2.1 government policies parthaand david (1994); bruneel et al. (2010); dai et al. (2013) 2.2. improved image of nation bodas freitas et al. (2012); fiaz (2013) 2.3 stability of centre and state government ostrom (2007) 2.4 coordination and cooperation among centre/state governments etzkowitz and leydesdorff (2000) 2.5 visionary and strategically able leadership dai et al. (2013) 2.6 improvement in national education system dai et al. (2013) 3 technical interactions (numprasertchai and igel, 2005) 3.1 improved efficiency albury (2005); faems et al. (2005); peng et al. (2013) 3.2 technological innovations kaufmann and tödtling (2001) 3.3 technology adaptation towards adoption archibugi and pietrobelli (2003) 3.4 technical advancement stage of country santoro and chakrabarti (2002) 3.5 it enablement and communication systems siegel et al., (2003); numprasertchai and igel (2005); fontana et al. (2006) ijahp article: kumar, luthra, haleem/critical factors important for effective industry-institute interactions (iii): an indian perspective international journal of the analytic hierarchy process 9 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.319 cfs s.n. dimensions of cfs identified for effective iii (reference) cfs s.n. cfs identified for effective iii references 4 markets’ and customers’ interactions (jia et al., 2010; wilson, 2012) 4.1 access to new costumer’s and market muscio and nardone (2012) 4.2 improved timing for market introduction barney (1991) 4.3 customer satisfaction siegel et al. (2003) 4.4 managing demand and supply balance polt et al. (2001); xias and jin (2012) 4.5 eagerness to test innovative products sawhney et al. (2005); nambisan (2009) 4.6 income and education level of customers straughan and roberts (1999); diamantopoulos et al. (2003); hartono (2009) 4.7 role of media, advertisement & marketing agencies wells (2007) 5 social interactions (wilson, 2012) 5.1 better utilization of social resources siegel et al. (2003); laursen et al. (2011) 5.2 social justice orecchini et al. (2012) 5.3 better standard of life peng et al. (2013) 5.4 caste system towards specialized workforce yeravdekar and tiwari (2012) 5.5 belief and values bruneel et al. (2010) 6 green and environmental interactions (luthra et al., 2011) 6.1 encouragement to green efforts erkuş-öztürk and eraydın (2010); zhu et al. (2012) 6.2 environment organizations’ efforts kumar et al. (2013); kumar et al. (2014) 6.3 pollution and environment regulations and legislation chien and shih (2007); mudgal et al. (2010); kumar et al. (2013); luthra et al. (2013); 7 intellectual properties perspectives(bruneel et al., 2010) 7.1 better opportunities for interactions bruneel et al. (2010) 7.2 regular workshops, seminar, conferences lagendijkand cornford (2000); cummings and kiesler (2005); cummingsand kiesler (2007) 7.3 encouragement to knowledge and innovation management meyer-krahmer and schmoch (1998); edmondson et al. (2012) http://scholar.google.co.in/citations?user=mpavyfgaaaaj&hl=en&oi=sra http://www.sciencedirect.com/science/article/pii/s0016718599000317 http://www.sciencedirect.com/science/article/pii/s0016718599000317 http://sss.sagepub.com/search?author1=jonathon+n.+cummings&sortspec=date&submit=submit http://sss.sagepub.com/search?author1=sara+kiesler&sortspec=date&submit=submit http://www.sciencedirect.com/science/article/pii/s0048733307001734 ijahp article: kumar, luthra, haleem/critical factors important for effective industry-institute interactions (iii): an indian perspective international journal of the analytic hierarchy process 10 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.319 cfs s.n. dimensions of cfs identified for effective iii (reference) cfs s.n. cfs identified for effective iii references 8 human resource interactions (laine, 2008) 8.1 support and quality of human resource maslach et al. (2001) 8.2 better employment opportunities afonso et al. (2012) 8.3 training programs salleh and omar (2013) 8.4 research based incentive/promotion system com (2007) 9 motivation from mutual benefits(mohan, 2011; vauterin et al., 2012) 9.1 better positioning and brand image khanna et al. (2014) 9.2 long term competitive advantage bruneel et al. (2010);şendoğdu and diken (2013) 9.3 top and middle management commitment and support cameron and quinn (2005); edmondson et al. (2012) ijahp article: kumar, luthra, haleem/critical factors important for effective industry-institute interactions (iii): an indian perspective international journal of the analytic hierarchy process 11 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.319 4. methodology the methodology adopted in this paper utilized two well established approaches:  literature review relevant literature was reviewed to establish the background of the research and identify cfs.  idea engineering workshop and analytical hierarchy process a work shop was conducted in two sessions. the first session was conducted to validate cfs and identify appropriate dimensions of cfs for effective iii. in the second session of the workshop nine experts brainstormed to make pair wise comparisons among the identified dimensions and cfs for effective iii according to saaty’s scale (saaty, 1980). based on the ratings, matrices were formed and the priorities were synthesized appropriately using the step wise procedure of the ahp technique. an ahp framework of cfs for effective iii in the indian perspective was structured as a hierarchy which included three levels: goal of achieving effective iii in india, nine dimensions of cfs, and the cfs under each dimension. the analytic hierarchy process (ahp) was an appropriate technique to rank the dimensions and cfs for effective iii in india because of its methodical and systematic approach. it is a well-established multi attribute decision support tool which uses a multilevel hierarchical structure of objectives, criteria, sub-criteria and alternatives. ahp technique evaluates and ranks the alternatives with respect to various criteria in a natural, pair-wise mode and also compares criteria, or alternatives with respect to a criterion, in a natural, pair-wise mode (saaty, 1980). to do so, it uses a fundamental scale of absolute numbers that has been proven in practice and validated by physical and decision problem experiments. it converts individual preferences into ratio scale weights that can be combined into a linear additive weight for each alternative. the resultant can be used to compare and rank the alternatives and, hence, assist the decision maker in making a choice (saaty, 2000, 2008; kumar et al., 2009; luthra et al., 2015b). 5. data analysis and results 5.1 constructing the hierarchy of dimensions of cfs important for effective iii: 2 nd level in the second level of the hierarchy, nine identified dimensions of cfs for effective iii were identified. table 3 shows the weights that were given by the experts to the dimensions of cfs. ijahp article: kumar, luthra, haleem/critical factors important for effective industry-institute interactions (iii): an indian perspective international journal of the analytic hierarchy process 12 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.319 table 3 ranking of dimensions of cfs important for effective iii from an indian perspective maximum eigen value=9.40178; c.i. = 0.0502219 pair wise comparison matrix of dimensions of cfs 1 2 3 4 5 6 7 8 9 global priority weighting rank 1 1 2 1 1 4 2 2 1 1 0.1568 2nd 2 0.5 1 1 2 1 1 0.3333 0.3333 0.25 0.0720 8th 3 1 1 1 1 2 1 1 0.5 0.333333 0.0928 5th 4 1 0.5 1 1 1 1 0.5 0.5 0.5 0.0780 6th 5 0.25 1 0.5 1 1 1 0.5 0.5 0.5 0.0643 9th 6 0.5 1 1 1 1 1 0.5 0.5 0.5 0.0734 7th 7 0.5 3 1 2 2 2 1 1 1 0.1362 4th 8 1 3 2 2 2 2 1 1 1 0.1545 3rd 9 1 4 3 2 2 2 1 1 1 0.1720 1st from the results shown in table 3, ‘motivation from mutual benefits (0.1720)’ is the most important dimension for effective iii followed by ‘financial interactions(0.1568)’, ‘human resource interactions’(0.1545), ‘intellectual properties perspectives (0.1362)’, ‘technical interactions (0.0928)’,‘market’s and customer’s interactions (0.0780)’,‘green and environmental interactions (0.0734)’,‘government/nation/regulatory perspectives (0.0720) ’and ‘social interactions (0.0643)’. 5.2 constructing the hierarchy of cfs of dimensions important for effective iii: 3 rd level in the third level of the hierarchy, the cfs under each dimension were rated by experts. the maximum eigenvalues, c.i. and pair wise comparison matrix of cfs under various dimensions for effective iii have been shown in appendix ii. consistency ratio (c.r.) values are well within the acceptable range for all cfs of the dimensions matrices, which ensure the decision-makers reliability (see appendix ii). global preference weights of cfs were obtained by multiplying the preference weight values of each dimension with the relative weight of each specific cf. based upon the global preference weights of cfs, the dimensions of cfs that are important for effective iii were ranked. the overall ranking of dimensions and cfs important for effective iii in an indian perspective is summarized in table 4. ijahp article: kumar, luthra, haleem/critical factors important for effective industry-institute interactions (iii): an indian perspective international journal of the analytic hierarchy process 13 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.319 table 4 overall ranking of dimensions and cfs important for effective iii in an indian perspective s. no. dimensions of cfs identified for effective iii preference weight of the dimensions rank cfs s. no. cfs identified for effective iii relative weight of cfs global weight of cfs overall ranking of cfs 1 financial interactions 0.1568 2 nd 1.1 maintained/improved margins 0.0723 0.0113 31 th 1.2 increased market share 0.1045 0.0164 23 rd 1.3 increase of shareholder’ returns 0.0637 0.0100 33 th 1.4 higher profits 0.1452 0.0228 16 th 1.5 incentives and subsidies 0.1891 0.0297 10 th 1.6 fund allocation for nation’s r&d 0.1694 0.0267 15 th 1.7 fund allocation for industry’s/institute’s r&d 0.2558 0.0401 6 th 2 government/nation/ regulatory perspectives 0.0720 8 th 2.1 government policies 0.0726 0.0052 40 th 2.2 improved image of nation 0.0801 0.0058 39 th 2.3 stability of centre and state government 0.2625 0.0189 18 th 2.4 coordination cooperation among centre/state governments 0.2415 0.0174 22 nd 2.5 visionary and strategically able leadership 0.2109 0.0152 25 th 2.6 improvement in national education system 0.1324 0.0096 34 th 3 technical interactions 0.0928 5 th 3.1 improved efficiency 0.1352 0.0125 27 th 3.2 technological innovations 0.1934 0.0179 19 th 3.3 technology adaptation towards adoption 0.1686 0.0156 24 th ijahp article: kumar, luthra, haleem/critical factors important for effective industry-institute interactions (iii): an indian perspective international journal of the analytic hierarchy process 14 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.319 s. no. dimensions of cfs identified for effective iii preference weight of the dimensions rank cfs s. no. cfs identified for effective iii relative weight of cfs global weight of cfs overall ranking of cfs 3.4 technical advancement stage of country 0.1910 0.0177 20 th 3.5 it enablement and communication systems 0.3118 0.0289 12 th 4 markets’ and customers’ interactions 0.0780 6 th 4.1 access to new costumers and market 0.0974 0.0076 37 th 4.2 improved timing for market introduction 0.1218 0.0096 34 th 4.3 customer satisfaction 0.1466 0.0114 30 th 4.4 managing demand and supply balance 0.1098 0.0086 36 th 4.5 eagerness to test innovative products 0.1299 0.0101 32 th 4.6 income and education level of customers 0.1707 0.0133 26 th 4.7 role of media, advertisement and marketing agencies 0.2238 0.0175 21 st 5 social interactions 0.0643 9 th 5.1 better utilization of social resources 0.1469 0.0094 35 th 5.2 social justice 0.1897 0.0122 29 th 5.3 better standard of life 0.4777 0.0307 9 th 5.4 caste system towards specialized workforce 0.0697 0.0045 41 th 5.5 belief and values 0.1160 0.0075 38 th ijahp article: kumar, luthra, haleem/critical factors important for effective industry-institute interactions (iii): an indian perspective international journal of the analytic hierarchy process 15 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.319 s. no. dimensions of cfs identified for effective iii preference weight of the dimensions rank cfs s. no. cfs identified for effective iii relative weight of cfs global weight of cfs overall ranking of cfs 6 green and environmental interactions 0.0734 7 th 6.1 encouragement to green efforts 0.1692 0.0124 28 th 6.2 environment organizations’ efforts 0.3874 0.0284 13 th 6.3 pollution and environment regulations and legislation 0.4434 0.0325 8 th 7 intellectual properties perspectives 0.1362 4 th 7.1 better opportunities for interactions 0.400 0.0545 4 th 7.2 regular workshops, seminar, conferences 0.200 0.0272 14 th 7.3 encouragement to knowledge and innovation management 0.400 0.0545 4 th 8 human resource interactions 0.1545 3 rd 8.1 support and quality of human resource 0.2326 0.0359 7 th 8.2 better employment opportunities 0.1238 0.0191 17 th 8.3 training programs 0.2778 0.0429 5 th 8.4 research based incentive/promotion systems 0.3658 0.0565 3 rd 9 motivation from mutual benefits 0.1720 1 st 9.1 better positioning and brand image 0.1692 0.0291 11 th 9.2 long term competitive advantage 0.3874 0.0666 2 nd 9.3 top and middle management commitment and support 0.4434 0.0763 1 st ijahp article: kumar, luthra, haleem/critical factors important for effective industry-institute interactions (iii): an indian perspective international journal of the analytic hierarchy process 16 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.319 the results have been compiled as follows:  ‘fund allocation for industry’s/institute’s r&d’ was the highest ranked cf and ‘increase of shareholder returns’ was the lowest ranked cf in dimension 1 (financial interactions).  similarly, ‘stability of centre and state government’ was the most important cf and ‘government policies’ was the least important cf in dimension 2 (government/nation/regulatory perspectives).  in dimension 3 (technical interactions), ‘it enablement and communication systems’ obtained the highest rank and ‘improved efficiency’ obtained the lowest rank.  ‘role of media, advertisement and marketing agencies’ was ranked highest and ‘access to new customers and market’ was ranked lowest in dimension 4 (markets and customers).  ‘better standard of life’ was the highest ranked cf and ‘caste system towards specialized workforce’ was the lowest ranked cf in dimension 5 (social interactions).  ‘pollution and environment regulations and legislation’ was the most important cf and ‘encouragement to green efforts’ was the least important cf in dimension 6 (green and environmental interactions).  ‘better opportunity for interaction’ was the highest ranked cf and ‘regular workshops, seminar and conferences’ was the lowest ranked cf in dimension 7 (intellectual properties perspectives).  ‘research based incentive/promotion system’ was the highest ranked cf and ‘better employment opportunities’ was the lowest ranked cf in dimension 8 (human resource interactions).  ‘top and middle management commitment and support’ was the highest ranked cf and ‘better positioning and brand image’ was the lowest ranked cf in dimension 9 (motivation from mutual benefits). the overall ranking of these forty three cfs is as follows:  ‘top and middle management commitment and support (0.0763)’, ‘long term competitive advantage (0.0666)’ and ‘research based incentive/promotion systems (0.0565)’are the top ranked three cfs; and  ‘improved image of nation (0.0058)’, ‘government policies (0.0052’) and ‘caste system towards specialized workforce (0.0045)’are the bottom three cfs; 6. sensitivity analysis in this research, the ‘motivation from mutual benefits’ dimension of cfs for effective iii in an indian perspective had the highest preference weight (table 4), and therefore influences the other dimensions of cfs. for that reason, the ‘motivation from mutual benefits’ dimension has been selected with its value varying from 0.1 to 0.9 with 0.1 as the increment and this change has been reflected in the other dimensions. the changes in other dimensions of cfs for effective iii in an indian perspective have been tabulated in table 5. ijahp article: kumar, luthra, haleem/critical factors important for effective industry-institute interactions (iii): an indian perspective international journal of the analytic hierarchy process 17 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.319 table 5 dimension values when increasing ‘motivation from mutual benefits’ dimension of cfs for effective iii in an indian perspective listed dimensions values of preference weights for listed dimensions of cfs for effective iii 1 0.1568 0.1704 0.1515 0.1280 0.1136 0.0947 0.0755 0.0568 0.0379 0.0189 2 0.0720 0.0783 0.0696 0.0609 0.0522 0.0435 0.0347 0.0261 0.0174 0.0087 3 0.0928 0.1009 0.0897 0.0784 0.0672 0.0560 0.0447 0.0336 0.0224 0.0112 4 0.0780 0.0848 0.0754 0.0659 0.0565 0.0471 0.0376 0.0283 0.0188 0.0094 5 0.0643 0.0699 0.0621 0.0543 0.0466 0.0388 0.0310 0.0233 0.0155 0.0078 6 0.0734 0.0798 0.0709 0.0620 0.0532 0.0443 0.0353 0.0266 0.0177 0.0089 7 0.1362 0.1480 0.1316 0.1151 0.0987 0.0822 0.0656 0.0493 0.0329 0.0164 8 0.1545 0.1679 0.1493 0.1306 0.1120 0.0933 0.0744 0.0560 0.0373 0.0186 9 0.1720 0.1000 0.2000 0.3002 0.4000 0.5000 0.6013 0.7000 0.8000 0.9001 total 1 1 1 1 1 1 1 1 1 1 further, due to variation in dimension weights, specific cf’s weights also change accordingly. at the 0.1 value of ‘motivation from mutual benefits’dimension, the cf ‘research based incentive/promotion system (8.4)’ holds the highest value and the cf ‘caste system towards specialized workforce (5.4)’ holds the lowest value. the cf ‘research based incentive/promotion system (8.4)’ retains the highest weight value until the normal value (0.1720) of ‘motivation from mutual benefits’dimension. when the ‘motivation from mutual benefits’ dimension weights value is varied (from 0.1720 to 0.9), the cf ‘top and middle management commitment and support (9.3)’ holds the highest weight value, and the weight values of other cfs also vary accordingly. the changes in the values of other cfs in order to promote iii in an indian perspective have been tabulated in appendix iii. global preference weights for specific cfs based on the sensitivity analysis are presented in appendix iv. it can be inferred that, global preference weights of the specific cfs vary with respect to the change in weight values of the ‘motivation from mutual benefits’ dimension. therefore, it may be concluded that the ‘motivation from mutual benefits’ dimension is very important for effective iii in an indian perspective. 7. discussion and managerial implications due to globalization, intense global competition, rapid technological changes, shorter product life cycles, and environment and social issues indian industries are under tremendous pressure to incorporate changes into their business practices and are exploring innovative methods to reduce environmental and social impacts, operating costs, lead time, and inventory to sustain their growth in the market. institute research may be beneficial to indian industries by facilitating the advancement of knowledge and new technologies to increase sustainability in today’s business environment. government/regulatory bodies and universities are making efforts to increase academic engagement to solve economic, environmental and other societal issues and to raise revenue for universities (perkmann et al., 2013). this paper provided the identification and ranking of various dimensions of cfs and cfs under each dimension for effective iii in an indian perspective. a comprehensive literature review and idea engineering workshop were carried out to identify and rank these dimensions and the cfs under each dimension. nine dimensions and forty three cfs for effective iii were identified from extensive literature review and expert’s input. ahp methodology was used to rank these ijahp article: kumar, luthra, haleem/critical factors important for effective industry-institute interactions (iii): an indian perspective international journal of the analytic hierarchy process 18 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.319 identified dimensions and cfs for effective iii in an indian perspective. the ‘motivation from mutual benefits’ dimension had the highest global weight, and the ‘social interactions’ dimension was weighted the lowest. these dimensions were ranked 1 st and 9 th respectively. further, ranking of various cfs under each dimension was done. in order to recognize, identify and exploit opportunities from a global perspective, it is critically important to understand what boundaries stand between university-industry partnering initiatives in international higher education and what interaction processes span these boundaries (vauterin et al., 2012). research projects are usually long term and future oriented ‘series of activities’ systematically planned and executed to gain a competitive advantage through:  reduction of cost by involving innovations in raw material types and usage, and processes;  introduction of truly innovative products and services capable of: a) attracting new segments of prospective customers, and b) maintaining existing customers;  sharing of r&d costs and cutting down many consultancy expenditures; and many more. perceived expected better positioning and enhanced brand image may motivate industry players and institutes (organizations involved in research) to collaborate and share the benefits resulting from this agreement to move together towards achieving research goals. india is a democratic country with no condition for limiting the maximum number of political parties leading to the possibility of many uncertainties/instabilities. these uncertainties include:  conflicts of interest may arise from governments composed of two different political parties in states and at the centre.  difficulty in reaching consensus because the chances of governments of different political parties in indian states (28 states) are high.  in the case where no political party has a clear majority, there are two possibilities: a) formation of government by more than one political party; b) reelection. all of these situations/consequences may hamper long-lasting, stable, cordial and fruitful ‘interactions and relations’ among institutes and industries in their efforts to carry out collaborative research, share knowledge and management, and produce innovative management. in fact, stable government funding to universities may complement funding from research contracts and consultations, which may further contribute to a considerable increase in university’s collaborations with industry towards accelerating the knowledge transfer processes (muscio et al., 2013). budget announcements, annual and periodic, reveal fund allocation for various fields, where the team of ministers, planners and experts from finance and economics contribute to understand, perceive, plan, analyze, and refine the recommendations on the basis of information. this information is gathered through collecting primary data (by using techniques like questionnaire based survey etc.), considering secondary data (government reports, organization’s annual report, comparative reports etc.), comparison of cases of ijahp article: kumar, luthra, haleem/critical factors important for effective industry-institute interactions (iii): an indian perspective international journal of the analytic hierarchy process 19 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.319 developed countries and countries passing through the same development phase, and conducting various conferences, workshops and sessions (sanders et al., 2015).in fact, all actors involved must be actively involved in efforts to find appropriate methods for effective iii. our research has implications for international and national bodies, policy makers, industrial organizations and practitioners/managers with an interest in effective industry institute interactions to motivate research and innovation. this research also has implications for scholars who may use the framework and propositions to direct new theoretical and empirical analyses of iii. top managers are concerned with policymaking and the establishment of procedures to facilitate research and innovation activities (zohar, 2010). sufficient fund allocation may help to procure appropriate technology, machinery and research equipment and to hire research experts, scientists and analysts at the national and international level. a variety of skills (technical, professional, administrative, and managerial) are required to carry out the implementation and completion of collaborativeresearch and innovation projects, which may open up new opportunities for employment of trained and skilled professionals. financial objectives and measures for the growth stage will stem from the development and growth of the organization which will lead to increased sales volumes, acquisition of new customers, growth in revenues etc. the economic, social, political, and cultural results of iii are a long process (ayla and i̇şgören, 2010). 8. conclusions effective iii may create new opportunities for academia and business to collaborate for mutual advantage. in this research study, an attempt was made to identify various dimensions and cfs for effective iii in an indian perspective from an extensive literature review. an idea engineering workshop was conducted for pair wise comparison of dimensions and cfs. the ahp methodology was utilized for ranking these identified dimensions and cfs. in conclusion, the robustness of the proposed solution model for this research was monitored through a sensitivity analysis test. the ‘motivation from mutual benefits’ dimension was ranked first. this paper may play an important role in understanding various cfs and ranking them will help to achieve effective iii in india. systematically designed and critically planned and effectively implemented training and skill enhancing programs, in fact, may lead to successful collaboration for achieving certain preset goals towards research and innovation. all pair comparisons in ahp were made on the basis of expert’s opinions (selected from academia and industry). it is natural that the opinions of experts may be biased. it should also be noted that the experts were not selected randomly. based on the literature review and expert opinions, various dimensions and cfs under each dimension for effective iii in an indian perspective were identified and ranked. different multi-criteria decision making models may be applied for the same problem and results can be compared in future studies. further, questionnaire based surveys and case studies may be conducted to validate our research work. also, the interactions among the dimensions and cfs may be examined using ciahp approach proposed by kumar (2014). ijahp article: kumar, luthra, haleem/critical factors important for effective industry-institute interactions (iii): an indian perspective international journal of the analytic hierarchy process 20 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.319 references afonso, a., ramírez, j. j., &díaz-puente, j. m. (2012).university-industry cooperation in the education domain to foster competitiveness and employment. procedia-social and behavioral sciences, 46, 3947-3953. doi:10.1016/j.sbspro.2012.06.177 albury, d. (2005). fostering innovation in public services. public money and management, 25(1), 51-56. all india council of technical education (aicte). online available at: www.aicte-india.org (accessed: december 13, 2013). alrawi, k., &alrawi, w. (2011). knowledge management for marketing management: implementing information technology to distribution channel performance. journal of knowledge & human resource management, 3(4), 37-50. archibugi, d., &pietrobelli, c. (2003). the globalization of technology and its implications for developing countries: windows of opportunity or further burden? technological forecasting and social change, 70(9), 861-883. doi:10.1016/s0040-1625(02)00409-2 association of indian universities (aiu). online available at: www.aiuweb.org (accessed: december 12, 2013). ayla, c. d., &i̇şgören, n. ç. (2010).university-industry cooperation in terms of textileapparel education. procedia-social and behavioral sciences, 2(2), 3437-3441. doi:10.1016/j.sbspro.2010.03.530 barney, j. b. (1991). firm resources and sustainable competitive advantage.journal of management 17(1), 99–120. doi: 10.1177/014920639101700108 bercovitz, j. e., & feldman, m. p. (2007). fishing upstream: firm innovation strategy and university research alliances. research policy, 36(7), 930-948. doi:10.1016/j.respol.2007.03.002 bozeman, b. (2000). technology transfer and public policy: a review of research and theory. research policy, 29(4), 627-655. doi:10.1016/s0048-7333(99)00093-1 bruneel, j., d’este, p., & salter, a. (2010).investigating the factors that diminish the barriers to university–industry collaboration. research policy, 39(7), 858-868. doi:10.1016/j.respol.2010.03.006 cameron, k. s., & quinn, r. e. (2005). diagnosing and changing organizational culture: based on the competing values framework. new york: john wiley & sons. central board of secondary education (cbse), new delhi. online available at: www.cbse.nic.in (accessed: december 25, 2013). central electronics engineering research institute (ceeri). online available at: http:// http://www.ceerichennai.org/ (accessed: december 19, 2013). http://dx.doi.org/10.1016/j.sbspro.2012.06.177 http://www.aicte-india.org/ http://dx.doi.org/10.1016/s0040-1625(02)00409-2 http://www.aiuweb.org/ http://dx.doi.org/10.1016/j.sbspro.2010.03.530 http://dx.doi.org/10.1016/j.respol.2007.03.002 http://dx.doi.org/10.1016/s0048-7333(99)00093-1 http://dx.doi.org/10.1016/j.respol.2010.03.006 http://www.cbse.nic.in/ http://www.ceerichennai.org/ ijahp article: kumar, luthra, haleem/critical factors important for effective industry-institute interactions (iii): an indian perspective international journal of the analytic hierarchy process 21 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.319 central tibetan schools administration (ctsa), delhi. online available at: www.ctsa.nic.in (accessed: december 25, 2013). centre for studies in civilization, project of history of indian science, philosophy and culture (phispc). online available at: www.phispc.nic.in (accessed: december 12, 2013). chakrabarti, a. k., & lester, r. k. (2002). regional economic development: comparative case studies in the us and finland. in engineering management conference, 2002.iemc'02. 2002 ieee international,(2), 635-642. doi:10.1109/iemc.2002.1038510 chen, c. j., & huang, j. w. (2007). how organizational climate and structure affect knowledge management—the social interaction perspective. international journal of information management, 27(2), 104-118. doi:10.1016/j.ijinfomgt.2006.11.001 chien, m. k., & shih, l. h. (2007). an empirical study of the implementation of green supply chain management practices in the electrical and electronic industry and their relation to organizational performances. international journal of environmental science and technology, 4(3), 383-394. com, e. (2007). improving knowledge transfer between research institutions and industry across europe: embracing open innovation. brussels, 4, 2007. council of architecture (coa), india. online available at: www.coa.gov.in (accessed: december 13, 2013). cummings, j. n., &kiesler, s. (2005). collaborative research across disciplinary and organizational boundaries. social studies of science, 35(5), 703-722. doi: 10.1177/0306312705055535 cummings, j. n., &kiesler, s. (2007). coordination costs and project outcomes in multiuniversity collaborations. research policy, 36(10), 1620-1634. doi:10.1016/j.respol.2007.09.001 czarnitzki, d., hanel, p., & rosa, j. m. (2011).evaluating the impact of r&d tax credits on innovation: a micro econometric study on canadian firms.research policy, 40(2), 217-229. doi:10.1016/j.respol.2010.09.017 dai, y., kong, d., & wang, m. (2013). investor reactions to food safety incidents: evidence from the chinese milk industry. food policy, 43, 23-31. doi:10.1016/j.foodpol.2013.08.004 department of technical education (dte), haryana. online available at: http://techeduhry.nic.in (accessed: december 13, 2013). diamantopoulos, a., schlegelmilch, b. b., sinkovics, r. r., & bohlen, g. m. (2003). can socio-demographics still play a role in profiling green consumers? a review of the evidence and an empirical investigation. journal of business research, 56(6), 465-480. doi:10.1016/s0148-2963(01)00241-7 http://www.ctsa.nic.in/ http://www.phispc.nic.in/ http://dx.doi.org/10.1109/iemc.2002.1038510 http://dx.doi.org/10.1016/j.ijinfomgt.2006.11.001 http://www.coa.gov.in/ http://dx.doi.org/10.1016/j.respol.2007.09.001 http://dx.doi.org/10.1016/j.respol.2010.09.017 http://dx.doi.org/10.1016/j.foodpol.2013.08.004 http://techeduhry.nic.in/ http://dx.doi.org/10.1016/s0148-2963(01)00241-7 ijahp article: kumar, luthra, haleem/critical factors important for effective industry-institute interactions (iii): an indian perspective international journal of the analytic hierarchy process 22 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.319 edmondson, g., valigra, l., kenward, m., hudson, r. l., & belfield, h. (2012). making industry-university partnerships work: lessons from successful collaborations. science business innovation board aisbl. erkuş-öztürk, h., &eraydın, a. (2010). environmental governance for sustainable tourism development: collaborative networks and organization building in the antalya tourism region. tourism management, 31(1), 113-124. doi:10.1016/j.tourman.2009.01.002 etzkowitz, h. (2011). the triple helix: science, technology and the entrepreneurial spirit. journal of knowledge-based innovation in china, 3(2), 76-90. doi: http://dx.doi.org/10.1108/17561411111138937 etzkowitz, h., &leydesdorff, l. (2000). the dynamics of innovation: from national systems and “mode 2” to a triple helix of university–industry–government relations. research policy, 29(2), 109-123. doi:10.1016/s0048-7333(99)00055-4 faems, d., van looy, b., &debackere, k. (2005). inter-organizational collaboration and innovation: toward a portfolio approach*. journal of product innovation management, 22(3), 238-250. doi: 10.1111/j.0737-6782.2005.00120.x fiaz, m. (2013). an empirical study of university–industry r&d collaboration in china: implications for technology in society. technology in society, 35(3), 191-202. doi:10.1016/j.techsoc.2013.03.005 fombrun, c. (1996). reputation: realizing value from the corporate image. boston: harvard business school press. doi: 10.1002/npr.4040150212 fontana, r., geuna, a., & matt, m. (2006). factors affecting university–industry r&d projects: the importance of searching, screening and signaling. research policy, 35(2), 309323. doi:10.1016/j.respol.2005.12.001 freitas, i. m. b., geuna, a., & rossi, f. (2013).finding the right partners: institutional and personal modes of governance of university–industry interactions. research policy, 42(1), 50-62. doi:10.1016/j.respol.2012.06.007 gidley, j. m., hampson, g. p., wheeler, l., &bereded-samuel, e. (2010). from access to success: an integrated approach to quality higher education informed by social inclusion theory and practice. higher education policy, 23(1), 123-147. doi:10.1057/hep.2009.24 greenhalgh, t., robert, g., macfarlane, f., bate, p., &kyriakidou, o. (2004). diffusion of innovations in service organizations: systematic review and recommendations. milbank quarterly, 82(4), 581-629. doi: 10.1111/j.0887-378x.2004.00325.x guan, j., & zhao, q. (2013). the impact of university–industry collaboration networks on innovation in nanobiopharmaceuticals. technological forecasting and social change, 80(7), 1271-1286. doi:10.1016/j.techfore.2012.11.013 http://dx.doi.org/10.1016/j.tourman.2009.01.002 http://dx.doi.org/10.1016/s0048-7333(99)00055-4 http://dx.doi.org/10.1016/j.techsoc.2013.03.005 http://dx.doi.org/10.1016/j.respol.2005.12.001 http://dx.doi.org/10.1016/j.respol.2012.06.007 http://dx.doi.org/10.1016/j.techfore.2012.11.013 ijahp article: kumar, luthra, haleem/critical factors important for effective industry-institute interactions (iii): an indian perspective international journal of the analytic hierarchy process 23 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.319 hartono, a. (2008). adopting socio-demographic characteristics in profiling green consumers: a review of hypotheses. jurnal siasat bisnis, 12(1), 55-62. indian council of historical research (ichr), new delhi. online available at: www.ichrindia.org (accessed: december 12, 2013). indian council of philosophical research (icpr), new delhi. online available at: www.icpr.nic.in (accessed: december 12, 2013). indian council of social science research (icssr), new delhi. online available at: www.icssr.org (accessed: december 12, 2013). indian institute of advance study (iias), shimla. online available at: www.iias.org (accessed: december 12, 2013). indian institute of chemical biology (iicb).online available at: http://www.iicb.res.in/iicb1.htm (accessed: december 19, 2013). indian space research organization (isro). online available at: http://www.isro.org (accessed: december 19, 2013). jung, h. j. (2014). the impacts of science and technology policy interventions on university research: evidence from the us national nanotechnology initiative. research policy, 43(1), 74-91.doi:10.1016/j.respol.2013.07.001 kaufmann, a., &tödtling, f. (2001). science–industry interaction in the process of innovation: the importance of boundary-crossing between systems. research policy, 30(5), 791-804. doi:10.1016/s0048-7333(00)00118-9 kendriyavidyalayasangathan, new delhi. online available at: www.kvsangathan.nic.in (accessed: december 25, 2013). khanna, m., jacob, i., & yadav, n. (2014). identifying and analyzing touchpoints for building a higher education brand. journal of marketing for higher education, 24(1), 122143. doi:10.1080/08841241.2014.920460 korres, m. g. m. (2012). technical change and economic growth: inside the knowledge based economy. united kingdom: ashgate publishing, ltd. kulkarni, j. a., pachpande, a., pachpande, s, & kulkarni, j. a. (2011). case studies in management. new delhi, india: pearson education. kumar, s. (2014). contextual interactions analytic hierarchy process (ciahp): introduction and application to analyze interactions among knowledge management implementation capability (kmic) factors. international journal of knowledge and learning, 9(3), 242-263. doi: http://dx.doi.org/10.1504/ijkl.2014.068926 http://www.ichrindia.org/ http://www.icpr.nic.in/ http://www.icssr.org/ http://www.iias.org/ http://dx.doi.org/10.1016/j.respol.2013.07.001 http://dx.doi.org/10.1016/s0048-7333(00)00118-9 http://www.kvsangathan.nic.in/ ijahp article: kumar, luthra, haleem/critical factors important for effective industry-institute interactions (iii): an indian perspective international journal of the analytic hierarchy process 24 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.319 kumar, s., luthra, s., & haleem, a. (2013). customer involvement in greening the supply chain: an interpretive structural modeling methodology. journal of industrial engineering international, 9(1), 1-13. doi:10.1186/2251-712x-9-6 kumar, s., luthra, s., & haleem, a. (2014). critical success factors of customer involvement in greening the supply chain: an empirical study. international journal of logistics systems and management, 19(3), 283-310. doi: http://dx.doi.org/10.1504/ijlsm.2014.065498 kumar, s., parashar, n., & haleem, a. (2009). analytical hierarchy process applied to vendor selection problem: small scale, medium scale and large scale industries. business intelligence journal, 2(2), 355-362. lagendijk, a., & cornford, j. (2000).regional institutions and knowledge–tracking new forms of regional development policy. geoforum, 31(2), 209-218. doi:10.1016/s00167185(99)00031-7 lai, w. h. (2011). willingness-to-engage in technology transfer in industry–university collaborations. journal of business research, 64(11), 1218-1223. doi:10.1016/j.jbusres.2011.06.026 laine, k. (2008). a finnish concept for academic entrepreneurship: the case of satakunta university of applied sciences. industry and higher education, 22(1), 19-28. doi: http://dx.doi.org/10.5367/000000008783877002 laursen, k., & salter, a. (2004). searching high and low: what types of firms use universities as a source of innovation? research policy, 33(8), 1201-1215. laursen, k., reichstein, t., & salter, a. (2011).exploring the effect of geographical proximity and university quality on university–industry collaboration in the united kingdom. regional studies, 45(4), 507-523. doi:10.1016/j.respol.2004.07.004 liew, m. s., shahdan, t. t., & lim, e. s. (2013).enablers in enhancing the relevancy of university-industry collaboration. procedia-social and behavioral sciences, 93, 1889-1896. doi:10.1016/j.sbspro.2013.10.135 luthra, s., kumar, v., kumar, s., & haleem, a. (2011). barriers to implement green supply chain management in automobile industry using interpretive structural modeling technique: an indian perspective. journal of industrial engineering and management, 4(2), 231-257. doi:10.3926/jiem.2011.v4n2.p231-257 luthra, s., garg, d., & haleem, a. (2013). identifying and ranking of strategies to implement green supply chain management in indian manufacturing industry using analytical hierarchy process. journal of industrial engineering and management, 6(4), 930-962. doi: http://dx.doi.org/10.3926/jiem.693 luthra, s., garg, d., & haleem, a. (2014). green supply chain management: implementation and performance–a literature review and some issues. journal of advances in management research, 11(1), 20-46. doi: http://dx.doi.org/10.1108/jamr-07-2012-0027 http://dx.doi.org/10.1016/s0016-7185(99)00031-7 http://dx.doi.org/10.1016/s0016-7185(99)00031-7 http://dx.doi.org/10.1016/j.jbusres.2011.06.026 http://dx.doi.org/10.5367/000000008783877002 http://dx.doi.org/10.1016/j.respol.2004.07.004 http://dx.doi.org/10.1016/j.sbspro.2013.10.135 ijahp article: kumar, luthra, haleem/critical factors important for effective industry-institute interactions (iii): an indian perspective international journal of the analytic hierarchy process 25 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.319 luthra, s., kumar, s., garg, d., & haleem, a. (2015a). barriers to renewable/sustainable energy technologies adoption: indian perspective. renewable and sustainable energy reviews, 41, 762-776. luthra, s., mangla, s. k., &kharb, r. k. (2015b).sustainable assessment in energy planning and management in indian perspective. renewable and sustainable energy reviews, 47, 5873. doi:10.1016/j.rser.2015.03.007 majumdar, s. (2008). industry-institute interaction to public-private partnership: a journey to excellence. in international symposium on public-private partnership in tvet-challenges, opportunities and best practices, manila, 1-11. maslach, c., schaufeli, w. b., & leiter, m. p. (2001).job burnout. annual review of psychology, 52(1), 397-422. doi: 10.1146/annurev.psych.52.1.397 meyer-krahmer, f., &schmoch, u. (1998). science-based technologies: university–industry interactions in four fields. research policy, 27(8), 835-851. doi:10.1016/s00487333(98)00094-8 ministry of human resources development (mhrd). online available at: http://mhrd.gov.in/schemes_technical_eng (accessed: december 11, 2013). mohan, s. r. (2011). understanding interactions between research institutes and industry: indian perspective. journal of technology management for growing economies, 2(2), 113138. mowery, d. c. (2012).defense-related r&d as a model for “grand challenges” technology policies. research policy, 41(10), 1703-1715. doi:10.1016/j.respol.2012.03.027 mudgal, r. k., shankar, r., talib, p., & raj, t. (2010).modelling the barriers of green supply chain practices: an indian perspective. international journal of logistics systems and management, 7(1), 81-107. doi: http://dx.doi.org/10.1504/ijlsm.2010.033891 muscio, a., &nardone, g. (2012).the determinants of university–industry collaboration in food science in italy. food policy, 37(6), 710-718. doi:10.1016/j.foodpol.2012.07.003 muscio, a., quaglione, d., &vallanti, g. (2013). does government funding complement or substitute private research funding to universities?. research policy, 42(1), 63-75. doi:10.1016/j.respol.2012.04.010 nambisan, s. (2009). virtual customer environments: it-enabled customer co-innovation and value co-creation. springer us. doi: 10.1007/978-1-4419-1081-3_6 national council for educational research and training (ncert) new delhi. online available at: www.ncert.nic.in (accessed: december 25, 2013). national council for teachers, india. online available at: www.ncte-india.org (accessed: december 25, 2013). http://dx.doi.org/10.1016/j.rser.2015.03.007 http://dx.doi.org/10.1016/s0048-7333(98)00094-8 http://dx.doi.org/10.1016/s0048-7333(98)00094-8 http://mhrd.gov.in/schemes_technical_eng http://dx.doi.org/10.1016/j.respol.2012.03.027 http://dx.doi.org/10.1016/j.foodpol.2012.07.003 http://dx.doi.org/10.1016/j.respol.2012.04.010 http://www.ncert.nic.in/ http://www.ncte-india.org/ ijahp article: kumar, luthra, haleem/critical factors important for effective industry-institute interactions (iii): an indian perspective international journal of the analytic hierarchy process 26 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.319 national council of rural institutes (ncri), hyderabad. online available at: www.ncri.in (accessed: december 12, 2013). national institute of open schooling, noida, uttar pradesh. online available at: www.nos.org (accessed: december 25, 2013). national knowledge commission.(2008). report of working group on engineering education. working group on engineering education. available at: http://knowledgecomm ission.gov.in downloads/documents/wg_engineer.pdf (accessed: december 13, 2013). national research development corporation (nrdc), india. online available at: http://www.nrdcindia.com (accessed: december 21, 2013). navodayavidyalayasamiti, new delhi. online available at: www.navodaya.nic.in (accessed: december 25, 2013). numprasertchai, s., &igel, b. (2005). managing knowledge through collaboration: multiple case studies of managing research in university laboratories in thailand. technovation, 25(10), 1173-1182. doi:10.1016/j.technovation.2004.03.001 official website of haryana government (2013). online available at: haryana.gov.in(accessed: december 11, 2013). orecchini, f., valitutti, v., &vitali, g. (2012). industry and academia for a transition towards sustainability: advancing sustainability science through university–business collaborations. sustainability science, 7(1), 57-73. doi: 10.1007/s11625-011-0151-3 ostrom, e. (2007). a general framework for analyzing sustainability of. in proc. r. soc. london ser. b, (274), 1931. partha, d., & david, p. a. (1994).toward a new economics of science.research policy, 23(5), 487-521. doi:10.1016/0048-7333(94)01002-1 peng, t., lu, f. j., wang, g. y., san, f. b., & zhao, x. w. (2013). study on the optimization mechanism about behavior selection of healthy pig industry chain. ieri procedia, 5, 161-165. doi:10.1016/j.ieri.2013.11.086 perkmann, m., & walsh, k. (2009). the two faces of collaboration: impacts of universityindustry relations on public research. industrial and corporate change, 18(6), 10331065.doi: 10.1093/icc/dtp015 perkmann, m., king, z., &pavelin, s. (2011). engaging excellence? effects of faculty quality on university engagement with industry. research policy, 40(4), 539-552. doi:10.1016/j.respol.2011.01.007 perkmann, m., tartari, v., mckelvey, m., autio, e., broström, a., d’este, p., &sobrero, m. (2013). academic engagement and commercialisation: a review of the literature on http://www.ncri.in/ http://www.nos.org/ http://www.nrdcindia.com/ http://www.navodaya.nic.in/ http://dx.doi.org/10.1016/j.technovation.2004.03.001 http://dx.doi.org/10.1016/0048-7333(94)01002-1 http://dx.doi.org/10.1016/j.ieri.2013.11.086 http://dx.doi.org/10.1016/j.respol.2011.01.007 ijahp article: kumar, luthra, haleem/critical factors important for effective industry-institute interactions (iii): an indian perspective international journal of the analytic hierarchy process 27 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.319 university–industry relations. research policy, 42(2), 423-442. doi:10.1016/j.respol.2012.09.007 polt, w., gassler, h., schibany, a., rammer, c., &schartinger, d. (2001). benchmarking industry—science relations: the role of framework conditions. science and public policy, 28(4), 247-258. saaty t. l. (2000). fundamentals of decision making and priority theory. pittsburgh, pa: rws publications. saaty, t. l. (1980). the analytic hierarchy process: planning, priority setting, resources allocation. new york: mcgraw hill. saaty, t. l. (2008). decision making with the analytic hierarchy process.international journal of services sciences, 1(1), 83-98. doi: http://dx.doi.org/10.1504/ijssci.2008.017590 salleh, m. s., & omar, m. z. (2013). university-industry collaboration models in malaysia. procedia-social and behavioral sciences, 102, 654-664. doi:10.1016/j.sbspro.2013.10.784 salter, a. j., & martin, b. r. (2001). the economic benefits of publicly funded basic research: a critical review. research policy, 30(3), 509-532. doi:10.1016/s00487333(00)00091-3 sanders, m., clark, r., davidson, b., &jayaraman, s. (2015). gt journey: the importance of accessible rich data sources to enable innovation. in human-computer interaction: users and contexts. springer international publishing. doi: 10.1007/978-3-319-21006-3_9 santoro, m. d., &chakrabarti, a. k. (2002).firm size and technology centrality in industry– university interactions. research policy, 31(7), 1163-1180. doi:10.1016/s00487333(01)00190-1 sawhney, m., verona, g., &prandelli, e. (2005).collaborating to create: the internet as a platform for customer engagement in product innovation. journal of interactive marketing, 19(4), 4-17. doi:10.1002/dir.20046 şendoğdu, a. a., &diken, a. (2013). a research on the problems encountered in the collaboration between university and industry. procedia-social and behavioral sciences, 99, 966-975. doi:10.1016/j.sbspro.2013.10.570 siegel, d. s., waldman, d. a., atwater, l. e., & link, a. n. (2003). commercial knowledge transfers from universities to firms: improving the effectiveness of university–industry collaboration. the journal of high technology management research, 14(1), 111-133. doi:10.1016/s1047-8310(03)00007-5 straughan, r. d., & roberts, j. a. (1999). environmental segmentation alternatives: a look at green consumer behavior in the new millennium. journal of consumer marketing, 16(6), 558-575. doi: http://dx.doi.org/10.1108/07363769910297506 http://dx.doi.org/10.1016/j.respol.2012.09.007 http://dx.doi.org/10.1016/j.sbspro.2013.10.784 http://dx.doi.org/10.1016/s0048-7333(00)00091-3 http://dx.doi.org/10.1016/s0048-7333(00)00091-3 http://dx.doi.org/10.1016/s0048-7333(01)00190-1 http://dx.doi.org/10.1016/s0048-7333(01)00190-1 http://dx.doi.org/10.1002/dir.20046 http://dx.doi.org/10.1016/j.sbspro.2013.10.570 http://dx.doi.org/10.1016/s1047-8310(03)00007-5 http://dx.doi.org/10.1108/07363769910297506 ijahp article: kumar, luthra, haleem/critical factors important for effective industry-institute interactions (iii): an indian perspective international journal of the analytic hierarchy process 28 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.319 tether, b. s., &tajar, a. (2008). beyond industry–university links: sourcing knowledge for innovation from consultants, private research organizations and the public sciencebase. research policy, 37(6), 1079-1095. doi:10.1016/j.respol.2008.04.003 university grant commission of india (2013). online available at: www.ugc.ac.in (accessed: december 11, 2013). vauterin, j. j., linnanen, l., &marttila, e. (2012). value creation in international higher education: the role of boundary spanning in university-industry collaboration. international journal of quality and service sciences, 4(3), 283-298. doi: http://dx.doi.org/10.1108/17566691211269594 vest, c. m. (2005). educating engineers for 2020 and beyond. national academy of engineering, 160-169. wallin, j., isaksson, o., larsson, a., &elfström, b. o. (2014). bridging the gap between university and industry: three mechanisms for innovation efficiency. international journal of innovation and technology management, 11(1), 1-18. doi: 10.1142/s0219877014400057 wei, j. i. a., liu, l. r., & xie, x. m. (2010). diffusion of technical innovation based on industry-university-institute cooperation in industrial clusters. the journal of china universities of posts and telecommunications, 17, 45-50. doi:10.1016/s10058885(09)60583-5 wells, w., spence-stone, r., crawford, r., moriarty, s., & mitchell, n. (2011).advertising: principles and practices. pearson higher education au. wixted, b., & holbrook, j. a. (2013). reconfiguring national science and research systems: the role of charities. centre for policy research on science and technology. online available at: http://www.sfu.ca/cprost/?p=569 xias, f., &jin, w. (2012). on the factors that affect the proneness of university-industry cooperation: a system engineering perspective. systems engineering procedia, 3, 275-281. doi:10.1016/j.sepro.2011.11.030 yeravdekar, v. r., & tiwari, g. (2012).the higher education system in india and its impact on the economy. available at ssrn 2139894. zhu, q., sarkis, j., & lai, k. h. (2012).examining the effects of green supply chain management practices and their mediations on performance improvements. international journal of production research, 50(5), 1377-1394. doi:10.1080/00207543.2011.571937 zohar, d. (2010). thirty years of safety climate research: reflections and future directions. accident analysis & prevention, 42(5), 1517-1522. doi:10.1016/j.aap.2009.12.019 http://dx.doi.org/10.1016/j.respol.2008.04.003 http://www.ugc.ac.in/ http://dx.doi.org/10.1108/17566691211269594 http://dx.doi.org/10.1016/s1005-8885(09)60583-5 http://dx.doi.org/10.1016/s1005-8885(09)60583-5 http://summit.sfu.ca/collection/205 http://www.sfu.ca/cprost/?p=569 http://dx.doi.org/10.1016/j.sepro.2011.11.030 http://dx.doi.org/10.1016/j.aap.2009.12.019 ijahp article: kumar, luthra, haleem/critical factors important for effective industry-institute interactions (iii): an indian perspective international journal of the analytic hierarchy process 29 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.319 appendix i organizations involved in education/research in india authority/ organizations involved in education/research in india hierarchy focus description/role/ functions reference/ website central government the ministry of human resource development (mhrd) is under the overall charge of the human resource development minister. formulation, implementatio n and control of educational and research policies the central government continues to play a leading role in the formulation and implementation of educational policies and programs such as national policy on education (npe), 1986 (modified in 1992). other initiatives include: providing universal access, ensuring retention and improving quality in elementary education, special emphasis on education of girls, establishment of pacesetting schools, inter-disciplinary research, starting more open universities in the states, strengthening of aicte, encouraging sports, physical education, yoga and adoption of an effective evaluation method, etc. http://mhrd.go v.in ministry of human resource development (mhrd) the ministry of human resource development has two departments i.e. department of school education & literacy and the department of higher education. all-round development of indian citizens the essence of hrd is education. the all-round development of indian citizens can be achieved by building strong foundations in education. in pursuance of this mission, the mhrd was created on september 26, 1985, through the 174th amendment to the government of india (allocation of business) rules, 1961. the main objectives of the ministry would be: formulating the national policy on education and ensuring its implementation; planned development, including expanding access and improving quality of educational institutions throughout the country; paying special attention to poor, females and the minorities; provide financial help in the form of scholarships, loan subsidy to deserving students from deprived sections of the society; encouraging international cooperation in the field of education. http://mhrd.go v.in ijahp article: kumar, luthra, haleem/critical factors important for effective industry-institute interactions (iii): an indian perspective international journal of the analytic hierarchy process 30 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.319 authority/ organizations involved in education/research in india hierarchy focus description/role/ functions reference/ website department of school education and literacy headed by a secretary to the government of india. education of equitable quality for all citizens of india the vision of the department of school education & literacy is to ensure education of equitable quality for all in order to fully harness the nation's human potential. it has its eyes set on the “universalization of education” and making better citizens out of our young brigade. for this, various new schemes and initiatives are taken up regularly and recently, those schemes and initiatives have also started paying dividends in the form of growing enrolment in schools. http://mhrd.go v.in department of higher education headed by a secretary to the government of india. world class opportunities of higher education and research to the country the vision of the department of higher education is to realize india's human resource potential to its fullest in the education sector, with equity and excellence. the department of higher education is engaged in bringing world class opportunities of higher education and research to the country so that indian students are not found lacking when facing an international platform. for this, the government has launched joint ventures and signed mous to help the indian student benefit from the world opinion. http://mhrd.go v.in state government hrd minister is assisted by two ministers of state. promote education & research in state state government plays an important role in the formulation, implementation and control of educational and research policies in state in accordance with central government policies haryana.gov.in /‎ university grant commission (ugc) the ugc has a mandate for coordination and determination of standards in higher educational institutions academic and research environment in the university system to improve the academic environment in the university system by promoting collaboration and formal linkages with other universities, national laboratories, institutes of national importance and industrial r & d laboratories in all branches of knowledge through programme of teaching, training and research. www.ugc.ac.in http://www.ugc.ac.in/ ijahp article: kumar, luthra, haleem/critical factors important for effective industry-institute interactions (iii): an indian perspective international journal of the analytic hierarchy process 31 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.319 authority/ organizations involved in education/research in india hierarchy focus description/role/ functions reference/ website other apex bodies in universities and higher education these perform under the department of higher education (dhe), ministry of human resource development various academic and research issues there are seven other apex bodies in universities and higher education: indian council of historical research (ichr), new delhi; indian council of social science research (icssr), new delhi; indian council of philosophical research (icpr), new delhi; national council of rural institutes (ncri), hyderabad; indian institute of advance study(iias), shimla; association of indian universities(aiu); centre for studies in civilization, project of history of indian science, philosophy and culture(phispc) www.ichrindia .org www.icssr.org www.icpr.nic.i n www.ncri.in www.iias.org www.aiuweb.o rg www.phispc.ni c.in all india council for technical education (aicte) and council of architecture apex bodies in technical education promote technical education and research in the country the aicte has also framed various norms and standards to regulate the technical institutions in the country for maintaining quality in these institutions. these norms and standards are uniformly applicable to all the college. similarly council of architecture frames norms and regulations for architecture education imparting colleges. www.aicteindia.org www.coa.gov.i n department of technical education (dte) of state reports to state government to promote technical education and research in the state technical education is the supreme component of hrd with an infinite potential for supplementing greater value to products and services and for improving the quality of life of the people. the dte explicitly provides technically trained manpower in various fields of engineering & technology encompassing diploma, post diploma, degree & post graduate level courses conducted through technical institutions like polytechnics/engineering colleges/ institutions of management/ computer/pharmacy. http://techeduh ry.nic.in central government universities work under mhrd, dhe and ugc academic and research in university forty central universities have been involved in imparting higher education at university level. http://mhrd.go v.in http://www.ichrindia.org/ http://www.ichrindia.org/ http://www.icssr.org/ http://www.icpr.nic.in/ http://www.icpr.nic.in/ http://www.ncri.in/ http://www.iias.org/ http://www.aiuweb.org/ http://www.aiuweb.org/ http://www.phispc.nic.in/ http://www.phispc.nic.in/ http://www.aicte-india.org/ http://www.aicte-india.org/ http://www.coa.gov.in/ http://www.coa.gov.in/ https://www.google.co.in/search?biw=994&bih=626&q=polytechnics/+engineering+colleges/+institutions+of+management/+computer/&spell=1&sa=x&ei=gahfuvpblcimrqebwig4ca&ved=0ccgqbsga https://www.google.co.in/search?biw=994&bih=626&q=polytechnics/+engineering+colleges/+institutions+of+management/+computer/&spell=1&sa=x&ei=gahfuvpblcimrqebwig4ca&ved=0ccgqbsga http://techeduhry.nic.in/ http://techeduhry.nic.in/ ijahp article: kumar, luthra, haleem/critical factors important for effective industry-institute interactions (iii): an indian perspective international journal of the analytic hierarchy process 32 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.319 authority/ organizations involved in education/research in india hierarchy focus description/role/ functions reference/ website state government universities ugc academic and research in university state universities are run by the state government of each of the states and territories of india and are usually established by a local legislative assembly act. as of 30 november 2011, the ugc lists 285 state universities have been involved in imparting higher education at state level. www.ugc.ac.in ; deemed universities private self financing ugc academic and research in university the ugc list from 23 june 2008 lists 130 deemed universities and the ugc list of private universities from 7 june 2012 lists 112 private universities. these are involved in imparting higher education. www.ugc.ac.in government colleges and organizations state government/ universities academic and research in college under university this category includes indian institutes of technology (sixteen), indian institutes of management (thirteen), national institutes of technology (thirty), indian institutes of information technology (four), indian institute of science & indian institute of science education & research (six), national institute of technical teachers' training & research (four), four boards of apprenticeship training and other technical & language institutes. www.aicteindia.org; www.ugc.ac.in private self financing colleges state government/ university academic and research in college under university the higher education system in india includes both private and publicly funded universities. these private self-financing college works under state universities to impart higher education in the state. government research organizations government of india motivate research and innovation in the country the government of india has set up many research organizations like indian space research organization (isro), central electronics engineering research institute (ceeri); indian institute of chemical biology; indian institute of tropical meteorology; indira gandhi centre for atomic research and industrial engineering and operations research. space commission and department of space (dos) in june 1972. http://www.isr o.org http://www.ce erichennai.org/ http://www.iic b.res.in/iicb1.h tm http://www.ugc.ac.in/ http://www.ugc.ac.in/ http://www.ugc.ac.in/ http://www.aicte-india.org/ http://www.aicte-india.org/ http://www.ugc.ac.in/ http://www.isro.org/ http://www.isro.org/ http://www.ceerichennai.org/ http://www.ceerichennai.org/ ijahp article: kumar, luthra, haleem/critical factors important for effective industry-institute interactions (iii): an indian perspective international journal of the analytic hierarchy process 33 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.319 authority/ organizations involved in education/research in india hierarchy focus description/role/ functions reference/we bsite private research organizations national research development corporation under government of india motivate research and innovation in the country the government of india has set up many research organizations like hi-tech bio laboratories, shriram institute for industrial research, dcm limited, varanasi glucose company, madhu chemicals limited. nrdc also acts on behalf of large companies who develop technologies in their r&d laboratories and choose to license them out for commercialization. http://www.nr dcindia.com central board of secondary education and others work under department of school education & literacy promote education there are seven bodies: central board of secondary education, new delhi; national council for educational research and training (ncert) new delhi; national institute of open schooling, noida, uttar pradesh; central tibetan schools administration (ctsa), delhi; navodayavidyalayasamiti, new delhi; kendriyavidyalayasangathan, new delhi; national council for teachers . www.cbse.nic. in www.ncert.nic .in www.nos.org www.ctsa.nic.i n www.navoday a.nic.in www.kvsangat han.nic.in www.ncteindia.org http://www.nrdcindia.com/ http://www.nrdcindia.com/ http://www.cbse.nic.in/ http://www.cbse.nic.in/ http://www.ncert.nic.in/ http://www.ncert.nic.in/ http://www.nos.org/ http://www.ctsa.nic.in/ http://www.ctsa.nic.in/ http://www.navodaya.nic.in/ http://www.navodaya.nic.in/ http://www.kvsangathan.nic.in/ http://www.kvsangathan.nic.in/ http://www.ncte-india.org/ http://www.ncte-india.org/ ijahp article: kumar, luthra, haleem/critical factors important for effective industry-institute interactions (iii): an indian perspective international journal of the analytic hierarchy process 34 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.319 appendix ii ranking of cfs under various dimensions for effective iii ranking of cfs of “financial interactions” dimension maximum eigen value= 7.32176; c.i. = 0.0536266 pair wise comparison matrix of cfs of “financial interactions” dimension 1.1 1.2 1.3 1.4 1.5 1.6 1.7 global priority weighting rank 1.1 1 0.5 1 1 0.25 0.5 0.25 0.0723 6th 1.2 2 1 1 1 0.5 0.5 0.5 0.1045 5th 1.3 1 1 1 0.5 0.25 0.25 0.25 0.0637 7th 1.4 1 1 2 1 1 1 1 0.1452 4th 1.5 4 2 4 0.5 1 1 0.5 0.1891 2nd 1.6 2 2 4 0.5 1 1 0.5 0.1694 3rd 1.7 4 2 4 1 2 2 1 0.2558 1st ranking of cfs of “government/nation/regulatory perspectives” dimension maximum eigen value= 6.25234; c.i. = 0.0504681 pair wise comparison matrix of cfs of “government/nation/regulatory perspectives” dimension 2.1 2.2 2.3 2.4 2.5 2.6 global priority weighting rank 2.1 1 1 0.25 0.25 0.2 1 0.0726 6th 2.2 1 1 0.5 0.333 0.333 0.5 0.0801 5th 2.3 4 2 1 1 2 2 0.2625 1st 2.4 4 3 1 1 1 2 0.2415 2nd 2.5 5 3 0.5 1 1 1 0.2109 3rd 2.6 1 2 0.5 0.5 1 1 0.1324 4th ranking of cfs of “technical interactions” dimension maximum eigen value= 5.23553; c.i. = 0.0588814 pair wise comparison matrix of cfs of “technical interactions” dimension 3.1 3.2 3.3 3.4 3.5 global priority weighting rank 3.1 1 1 0.5 1 0.333 0.1352 5th 3.2 1 1 2 1 0.5 0.1934 2nd 3.3 2 0.5 1 1 0.5 0.1686 4th 3.4 1 1 1 1 1 0.1910 3rd 3.5 3 2 2 1 1 0.3118 1st ranking of cfs of “markets’ and customers’ interactions” dimension maximum eigen value= 7.77297; c.i. = 0.128829 pair wise comparison matrix of cfs of “markets’ and customers’ interactions” dimension 4.1 4.2 4.3 4.4 4.5 4.6 4.7 global priority weighting rank 4.1 1 0.5 0.333 1 1 0.5 1 0.0974 7th 4.2 2 0.5 1 1 0.5 0.5 1 0.1218 5th 4.3 3 1 1 0.5 1 1 1 0.1466 3rd 4.4 1 1 2 1 1 0.333 0.2 0.1098 6th 4.5 1 2 1 1 1 1 0.5 0.1299 4th 4.6 2 2 1 3 1 1 0.5 0.1707 2nd 4.7 1 1 1 5 2 2 1 0.2238 1st ijahp article: kumar, luthra, haleem/critical factors important for effective industry-institute interactions (iii): an indian perspective international journal of the analytic hierarchy process 35 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.319 ranking of cfs of “social interactions” dimension maximum eigen value= 5.50371; c.i. = 0.125927 pair wise comparison matrix of cfs of “social interactions” dimension 5.1 5.2 5.3 5.4 5.5 global priority weighting rank 5.1 1 0.5 0.167 2 3 0.1469 3rd 5.2 2 1 0.25 3 2 0.1897 2nd 5.3 6 4 1 4 2 0.4777 1st 5.4 0.5 0.333 0.25 1 0.5 0.0697 5th 5.5 0.333 0.5 0.5 2 1 0.1160 4th ranking of cfs of “green and environmental interactions” dimension maximum eigen value=3.01829; c.i. =0.00914735 pair wise comparison matrix of cfs of “green and environmental interactions” dimension 6.1 6.2 6.3 global priority weighting rank 6.1 1 0.5 0.333 0.1692 3rd 6.2 2 1 1 0.3874 2nd 6.3 3 1 1 0.4434 1st ranking of cfs of “intellectual properties perspectives” dimension maximum eigen value=3; c.i. = 2.22045e-16 pair wise comparison matrix of cfs of “intellectual properties perspectives” dimension 7.1 7.2 7.3 global priority weighting rank 7.1 1 2 1 0.4 1st 7.2 0.5 1 0.5 0.2 2nd 7.3 1 2 1 0.4 1st ranking of cfs of “human resource interactions” dimension maximum eigen value=4.04582; c.i. = 0.0152731 pair wise comparison matrix of cfs of “human resource interactions” dimension ranking of cfs of “motivation from mutual benefits” dimension maximum eigen value= 3.01829; c.i. =0.00914735 9.1 9.2 9.3 global priority weighting rank 9.1 1 0.5 0.333 0.1692 3rd 9.2 2 1 1 0.3874 2nd 9.3 3 1 1 0.4434 1st 8.1 8.2 8.3 8.4 global priority weighting rank 8.1 1 2 1 0.5 0.2326 3rd 8.2 0.5 1 0.5 0.333 0.1238 4th 8.3 1 2 1 1 0.2778 2nd 8.4 2 3 1 1 0.3658 1st ijahp article: kumar, luthra, haleem/critical factors important for effective industry-institute interactions (iii): an indian perspective international journal of the analytic hierarchy process 36 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.319 appendix iii global preference weights for cfs by sensitivity analysis when ‘increasing “dimension 9 (motivation from mutual benefits)” of cfs changes from 0.1 to 0.9’ cfs for effective iii 9=0.1 9=0.1720 (normal) 9=0.2 9=0.3 9=0.4 9=0.5 9=0.6 9=0.7 9=0.8 9=0.9 1.1 0.0123 0.0113 0.0110 0.0093 0.0082 0.0068 0.0055 0.0041 0.0027 0.0014 1.2 0.0178 0.0164 0.0158 0.0134 0.0119 0.0099 0.0079 0.0059 0.0040 0.0020 1.3 0.0109 0.0100 0.0097 0.0082 0.0072 0.0060 0.0048 0.0036 0.0024 0.0012 1.4 0.0247 0.0228 0.0220 0.0186 0.0165 0.0137 0.0110 0.0082 0.0055 0.0027 1.5 0.0322 0.0297 0.0286 0.0242 0.0215 0.0179 0.0143 0.0107 0.0072 0.0036 1.6 0.0289 0.0266 0.0257 0.0217 0.0192 0.0160 0.0128 0.0096 0.0064 0.0032 1.7 0.0436 0.0401 0.0388 0.0328 0.0291 0.0242 0.0193 0.0145 0.0097 0.0048 2.1 0.0057 0.0052 0.0051 0.0044 0.0038 0.0032 0.0025 0.0019 0.0013 0.0006 2.2 0.0063 0.0058 0.0056 0.0049 0.0042 0.0035 0.0028 0.0021 0.0014 0.0007 2.3 0.0205 0.0189 0.0183 0.0160 0.0137 0.0114 0.0091 0.0068 0.0046 0.0023 2.4 0.0189 0.0174 0.0168 0.0147 0.0126 0.0105 0.0084 0.0063 0.0042 0.0021 2.5 0.0165 0.0152 0.0147 0.0128 0.0110 0.0092 0.0073 0.0055 0.0037 0.0018 2.6 0.0104 0.0095 0.0092 0.0081 0.0069 0.0058 0.0046 0.0035 0.0023 0.0012 3.1 0.0136 0.0125 0.0121 0.0106 0.0091 0.0076 0.0060 0.0045 0.0030 0.0015 3.2 0.0195 0.0179 0.0173 0.0152 0.0130 0.0108 0.0086 0.0065 0.0043 0.0022 3.3 0.0170 0.0156 0.0151 0.0132 0.0113 0.0094 0.0075 0.0057 0.0038 0.0019 3.4 0.0193 0.0177 0.0171 0.0150 0.0128 0.0107 0.0085 0.0064 0.0043 0.0021 3.5 0.0315 0.0289 0.0280 0.0245 0.0210 0.0175 0.0139 0.0105 0.0070 0.0035 4.1 0.0083 0.0076 0.0073 0.0064 0.0055 0.0046 0.0037 0.0028 0.0018 0.0009 4.2 0.0103 0.0095 0.0092 0.0080 0.0069 0.0057 0.0046 0.0034 0.0023 0.0011 4.3 0.0124 0.0114 0.0110 0.0097 0.0083 0.0069 0.0055 0.0041 0.0028 0.0014 4.4 0.0093 0.0086 0.0083 0.0072 0.0062 0.0052 0.0041 0.0031 0.0021 0.0010 4.5 0.0110 0.0101 0.0098 0.0086 0.0073 0.0061 0.0049 0.0037 0.0024 0.0012 4.6 0.0145 0.0133 0.0129 0.0113 0.0096 0.0080 0.0064 0.0048 0.0032 0.0016 4.7 0.0190 0.0175 0.0169 0.0148 0.0126 0.0105 0.0084 0.0063 0.0042 0.0021 5.1 0.0103 0.0094 0.0091 0.0080 0.0068 0.0057 0.0045 0.0034 0.0023 0.0011 5.2 0.0133 0.0122 0.0118 0.0103 0.0088 0.0074 0.0059 0.0044 0.0029 0.0015 5.3 0.0334 0.0307 0.0297 0.0260 0.0223 0.0185 0.0148 0.0111 0.0074 0.0037 5.4 0.0049 0.0045 0.0043 0.0038 0.0032 0.0027 0.0022 0.0016 0.0011 0.0005 5.5 0.0081 0.0075 0.0072 0.0063 0.0054 0.0045 0.0036 0.0027 0.0018 0.0009 6.1 0.0135 0.0124 0.0120 0.0105 0.0090 0.0075 0.0060 0.0045 0.0030 0.0015 6.2 0.0309 0.0284 0.0275 0.0240 0.0206 0.0172 0.0137 0.0103 0.0069 0.0034 6.3 0.0354 0.0325 0.0314 0.0275 0.0236 0.0197 0.0157 0.0118 0.0079 0.0039 7.1 0.0592 0.0545 0.0526 0.0460 0.0395 0.0329 0.0262 0.0197 0.0132 0.0066 7.2 0.0296 0.0272 0.0263 0.0230 0.0197 0.0164 0.0131 0.0099 0.0066 0.0033 7.3 0.0592 0.0545 0.0526 0.0460 0.0395 0.0329 0.0262 0.0197 0.0132 0.0066 ijahp article: kumar, luthra, haleem/critical factors important for effective industry-institute interactions (iii): an indian perspective international journal of the analytic hierarchy process 37 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.319 8.1 0.0391 0.0359 0.0347 0.0304 0.0260 0.0217 0.0173 0.0130 0.0087 0.0043 8.2 0.0208 0.0191 0.0185 0.0162 0.0139 0.0116 0.0092 0.0069 0.0046 0.0023 8.3 0.0467 0.0429 0.0415 0.0363 0.0311 0.0259 0.0207 0.0156 0.0104 0.0052 8.4 0.0614 0.0565 0.0546 0.0478 0.0410 0.0341 0.0272 0.0205 0.0137 0.0068 9.1 0.0169 0.0291 0.0338 0.0508 0.0677 0.0846 0.1017 0.1184 0.1354 0.1523 9.2 0.0387 0.0666 0.0775 0.1163 0.1550 0.1937 0.2330 0.2712 0.3099 0.3487 9.3 0.0443 0.0763 0.0887 0.1331 0.1774 0.2217 0.2666 0.3104 0.3547 0.3991 ijahp article: kumar, luthra, haleem/critical factors important for effective industry-institute interactions (iii): an indian perspective international journal of the analytic hierarchy process 38 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.319 appendix iv sensitivity analysis of cfs for effective iii in an indian perspective 0.0000 0.0500 0.1000 0.1500 0.2000 0.2500 0.3000 0.3500 0.4000 0.4500 1.1 1.3 1.5 1.7 2.2 2.4 2.6 3.2 3.4 4.1 4.3 4.5 4.7 5.2 5.4 6.1 6.3 7.2 8.1 8.3 9.1 9.3 9=0.1 9=normal 9=0.2 9=0.3 9=0.4 9=0.5 9=0.6 9=0.7 9=0.8 9=0.9 instructions to authors for the preparation of manuscripts ijahp article: latora, compagno trapani/a decision support tool for business models analysis international journal of the analytic hierarchy process 219 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.516 a decision support tool for business models analysis antonio giuseppe latora department of electrical, electronics and computer engineering university of catania antonio.latora@dii.unict.it lucio compagno department of electrical, electronics and computer engineering university of catania lucio.compagno@dieei.unict.it natalia trapani department of electrical, electronics and computer engineering university of catania natalia.trapani@dieei.unict.it abstract the business model (bm) concept which has emerged in the scientific literature and business world over the past fifteen years is a new management concept that, although characterised by fuzzy boundaries, can be synthesized as “the rationale of how an organisation creates, delivers and captures value”. many authors see the bm as an excellent tool for the strategic evaluation of business ideas, but the literature review shows a lack of efficient quali-quantitative tools for evaluating potential bm alternatives and selecting the best bm solution from among them. in this paper, we propose a business model decision support tool (bm-dst) based on a qualiquantitative methodology of multiple criteria decision analysis (mcda) called analytic hierarchy process (ahp) and specifically based on a modified ahp procedure called value-analytic hierarchy process (v-ahp). a case study with a numerical example was carried out in the field of industrial plants, specifically in the context of the industrial product-service systems, in order to select the best bm solution from among three potential bm alternatives usually taken into account by an original equipment manufacturer (oem) and engineering-procurement construction (epc) company: transactional project deliveries, project led solutions and life-cycle solutions. keywords: business model canvas; multi criteria decision making; product service system; analytic hierarchy process; value-analytic hierarchy process 1. introduction a business model (bm) is “the rationale of how an organisation creates, delivers, and captures value” and a business model framework is a comprehensive template used to detect the relevant business variables and their relationships with the company’s value proposition (osterwalder & pigneur 2010). however, the business model concept is relatively new and an accurate literature review shows fuzzy mailto:antonio.latora@dii.unict.it mailto:lucio.compagno@dieei.unict.it mailto:natalia.trapani@dieei.unict.it ijahp article: latora, compagno trapani/a decision support tool for business models analysis international journal of the analytic hierarchy process 220 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.516 boundaries. there are multiple business model definitions and business model frameworks which often refer to a specific industrial sector (muegge, 2012; zott et al., 2011; al-debei &avison, 2010). many authors consider the bm an excellent tool for the strategic evaluation of business ideas, but the literature review shows a lack of efficient quali-quantitative tools for selecting the best bm solution within a set of potential bm alternatives. starting from here we propose an appropriate business model decision support tool (bm-dst) based on a quali-quantitative methodology of multiple criteria decision analysis (mcda) called analytic hierarchy process (ahp). in this case, a modified ahp procedure called value-analytic hierarchy process (v-ahp) will be used. the v-ahp combines the traditional ahp procedures for the rating under qualitative criteria and a “lean” procedure for the rating under quantitative criteria (d’urso et al., 2011). a case study with a numerical example was carried out in the context of the industrial product-service systems (meier et al., 2010). in particular, this case aims to select the adequate bm solution to satisfy both customer needs and business competitiveness from the point of view of an original equipment manufacturer (oem) and engineering-procurement-construction (epc) company. oem/epc contractors are transforming themselves from equipment or turnkey plant sellers to service providers, supporting their clients with many additional services (from maintenance to operational performance management). according to peillon et al. (2015) this servitization path is a non-reversible integration of product and service activities rather than a continuous transition from a pure product to a pure service offer. therefore, this transformation requires a rethinking of business models in order to adapt the servitization process and a tool to support a conscious decision about different bm alternatives. 2. literature review the evolution of business model studies shows different authors trying to explore theoretical foundations of value creation in emerging businesses such as e-commerce and e-business (timmers, 1998; amit & zott, 2001). timmers (1998) provided a business model framework for the classification of internet electronic commerce based on four components including architecture, value proposition, business actors and roles, revenue sources. amit and zott (2001) examined the business model of fifty-nine american and european companies, concluding that the potential value creation of e-businesses hinges on four interdependent dimensions, which are efficiency, complementarities, lock-in, and novelty. chesbrough and rosenbloom (2002) explored the role of the business model in capturing value from technology. they state that the ultimate role of the business model for an innovative solution is to ensure that the technological core of the innovation is embedded in an economically viable enterprise. their framework is based on market, value proposition, value chain, cost and profit, value network, and competitive strategy. starting from the numerous business failures related to the internet boom, magretta (2002) clarified that a good business model is essential for each successful organisation. according to him, business models are stories that explain how enterprises work and describe, as a system, how the pieces of a business fit together. he detects only three components of the business model framework which include value proposition, customers, and revenue sources. ijahp article: latora, compagno trapani/a decision support tool for business models analysis international journal of the analytic hierarchy process 221 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.516 hedman and kalling (2003) extended the components of the business model framework to six, e.g. value proposition, customers, resources, network, and architecture, structure. morris et al. (2015) discussed value proposition, customer, internal processes and skills, external positioning, economic model, personal and investor factors. in order to determine whether a firm should modify its business model, johnson et al. (2008) identified the following steps: articulate what makes an existing model successful, watch for signals that the model needs changing, and decide whether reinventing the model is worth the effort. according to this approach, they propose a framework with the following three components: customer value proposition, profit formula, key resources and key processes. zott and amit (2008) examine the fit between a firm’s product market strategy and its business model. they develop a formal model in order to analyze the effects of product market strategy and business model choices on the firm’s performance and arrive at the conclusion that business model and product market strategy are complementary to each other, not substitutes. according to this model two latent variables characterize the design themes of a business model (novelty and efficiency) and the other three latent variables characterize the product market positioning of the firm (differentiation, cost leadership and timing of entry). in 2009, bailetti proposed a tool that enables a product team to design a strong business model at the initial stage of its life cycle. the tool is offer-centric i.e. a business model is linked to an offer, not to a business unit or a product portfolio. according to bailetti, the following six variables affect the strength of a business model: significance, customer value, partner value, profit, leverage, intellectual property. doganova and eyquem-renault (2009) investigate the role played by business models in the innovation process. they adopt a pragmatic approach to business models examining them as market devices, focusing on their materiality, use and dynamics. they show that the business model is a narrative and calculative device that allows entrepreneurs to explore a market and plays a performative role by contributing to the techno-economic network construction of an innovation. key components of business models are product, customer, partners, value, profits and costs. a hierarchical taxonomy of the business model concepts, from which develop a more comprehensive framework appropriate to the business complex nature, was proposed by al-debei and avison (2010). the framework presented by casadesus-masanell and ricart (2010) allows a simple integration of the notions of strategy, business model and tactics. in their formulation, strategy and business model, though related, are different concepts. a business model is the direct result of strategy but is not, itself, strategy. they consider a three component business model framework that includes resources and competencies, internal and external organization, value propositions. according to teece (2010), a business model is significant for its connections with business strategy, innovation management and economic theory, so he developed a business model framework which includes technologies and features of product/service, customer benefit, market segment, revenues stream, mechanism to capture value. johnson (2010) identifies four fundamental building blocks by which a business model works. these building blocks are the customer value proposition that meets a real customer’s needs, the profit formula that lays out how a company makes money delivering the value proposition, the key resources required by value proposition, and the key processes needed to deliver it. these are a subset of the nine components proposed by osterwalder and pigneur (2010) who provide a tool for describing, analyzing, and designing business models (customer segments, value propositions, channels, customer relationships, revenue streams, key resources, key activities, key ijahp article: latora, compagno trapani/a decision support tool for business models analysis international journal of the analytic hierarchy process 222 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.516 partnerships, cost structure). osterwalder and pigneur proposed the business model canvas tool to support designing business models according to their framework. zott et al. (2011) found emerging common themes among scholars of business models. the business models emphasize a system-level and holistic approach to explaining how firms “do business”. firm activities play an important role in the various conceptualizations of the business models proposed. business models seek to explain how value is created, not just how it is captured. in this context, they developed a framework based on value creation, performance, and competitive advantage. george and bock (2011) discuss the nature and implications of dimensional dominance for firm characteristics and behaviour. these findings provide new directions for theory development and empirical studies in entrepreneurship by linking the business model to entrepreneurial cognition, opportunity co-creation and organizational outcomes. more recently, baden-fuller and haefliger (2013) state that business models are fundamentally linked with technological innovation, but the business model construct is essentially separable from technology. they define the business model as a system that solves the problem of identifying who is the customer, engaging with their needs, delivering satisfaction, and monetizing the value. the proposed framework depicts the business model system as a model containing a cause and effect relationship. table 1 describes, in a non-exhaustive way, the evolution of the business model definition and framework, analyzing the most relevant scientific contributions given during the years 1998-2016. table 1 synthesis of studies, definitions and framework components of bm year author(s) title bm definition bm framework components 1998 timmers business models for electronic markets “an architecture of the product, service and information flows, including a description of the various business actors and their roles; a description of the potential benefits for the various business actors; a description of the sources of revenues.”  architecture  value proposition  business actors and roles  revenue sources 2001 amit and zott value creation in ebusiness “the content, structure, and governance of transactions designed so as to create value through the exploitation of business opportunities.”  complementarities  lock-in  efficiency  novelty 2002 chesbrough and rosenbloom the role of the business model in capturing value from innovation: evidence from xerox corporation’s technology spinoff companies “a coherent framework that takes technological characteristics and potentials as inputs, and converts them through customers and markets into economic inputs. the business model is thus conceived as a focusing device that mediates between technology development and economic value creation.”  market  value proposition  value chain  cost and profit  value network  competitive  strategy 2002 magretta why business models matter “stories that explain how enterprises work. a good business model answers peter drucker’s age old questions: who is the customer? and what does the customer value? it also answers the fundamental questions every manager must ask: how do we make money in this business? what is the  value proposition  customers  revenue sources ijahp article: latora, compagno trapani/a decision support tool for business models analysis international journal of the analytic hierarchy process 223 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.516 year author(s) title bm definition bm framework components underlying economic logic that explains how we can deliver value to customers at an appropriate cost?” 2003 hedman and kalling the business model concept: theoretical underpinnings and empirical illustrations “a term often used to describe the key components of a given business. that is customers, competitors, offering, activities and organisation, resources, supply of factors and production inputs as well as longitudinal process components to cover the dynamics of the business model over time”  resources  customers  value proposition  network  architecture  structure 2005 morris et al. the entrepreneur’s business model: toward a unified perspective “concise representation of how an interrelated set of decision in the areas of venture strategy, architecture, and economics are addressed to create sustainable competitive advantage in defined markets”  value proposition  customer  internal processes/skills  external positioning  economic model  personal/investor factors 2008 johnson et al. reinventing your business model “consist of four interlocking elements (customer value proposition, profit formula, key resources, and key processes), that, taken together, create and deliver value”  customer value proposition  profit formula  key resources  key processes 2008 zott and amit the fit between product market strategy and business model: implications for firm performance “a structural template of how a focal firm transacts with customers, partners, and vendors; that is, how it chooses to connect with factor and product markets. it refers to the overall gestalt of these possibly interlinked boundary-spanning transactions”  novelty  efficiency  differentiation  cost leadership  timing of entry 2009 bailetti how open source strengthens business models “the narrative and expected profit and loss statement that define the: importance of getting the job done, solving the problem, or satisfying the need; value delivered to customers, company and other key stakeholders; control over or access to the key resources, processes, and norms required to deliver value”  significance  customer value  partner value  profit  leverage  intellectual property 2009 doganova and eyquemrenault what do business models do? innovation devices in technology entrepreneurship “a narrative and calculative device that allows entrepreneurs to explore a market and plays a performative role by contributing to the construction of the techno-economic network of an innovation”  product  customer  partners  value  profits  costs 2010 al-debei and avison developing a unified framework of the business model concept “an essential conceptual tool of alignment in digital business. it can be depicted as an intermediate layer between business strategy and ictenabled business processes in order to fulfil the missing link created by the complexity of the digitalised environment”  value proposition  value network  value architecture  value finance 2010 casadesusmasanell and ricart from strategy to business models and onto tactics “a reflection of the firm’s realised strategy  resources and competences  internal/external organisation  value propositions 2010 johnson seizing the white “the way in which a company  customer value ijahp article: latora, compagno trapani/a decision support tool for business models analysis international journal of the analytic hierarchy process 224 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.516 year author(s) title bm definition bm framework components space delivers value to a set of customers at a profit” proposition  profit formula  key resources  key processes 2010 osterwalder and pigneur business model generation: a handbook for visionaries, game changers, and challengers “the rationale of how an organisation creates, delivers and captures value”  customer segments  value propositions  channels  customer relationships  revenue streams  key resources  key activities  key partnerships  cost structure 2010 teece business models, business strategy and innovation “the manner by which the enterprise delivers value to customers, entices customers to pay for value, and converts those payments to profit”  technologies and features of product/service  customer benefit  market segment  revenues stream  mechanism to capture value 2010 zott and amit the business model: recent developments and future research “a system of interdependent activities that transcends the focal firm and spans its boundaries”  value creation  performance  competitive advantage 2011 george and bock the business model in practice and its implications for entrepreneurship research “is commonly described and reflects on organisational design, the resource-based view of the firm, narrative and sensemaking, the nature of innovation, the nature of opportunity, and transactive structures”  resource structure  organisational design  transactive structure  value structure 2012 muegge business model discovery by technology entrepreneurs “an explanation of how the business delivers value to a set of customers at attractive profits”  importance  stakeholders value propositions  profit formula  capabilities 2013 baden-fuller and haefliger business models and technological innovation “a system that solves the problem of identifying who is (or are) the customer(s), engaging with their needs, delivering satisfaction, and monetizing the value”  customer  customer engagement  value delivery and linkages  monetization the literature review on bm framework revealed the following:  many definitions of bm have been proposed in literature;  several bm frameworks have been proposed by the authors;  the number of different bm framework components ranges from 3-9;  no qualitative-quantitative business model decision support tool (bmdst), aimed to select the best bm solution within a set of potential bm alternatives, has been detected. the most complete framework, having the highest number of components is the business model canvas proposed by osterwalder and pigneur (2010). the nine business model canvas components are as follows: key resources (kr), key activities (ka), key partnerships (kp), cost structure (c$), value propositions (vp), channels (ch), customer relationships (cr), customer segments (cs), revenue streams (r$). ijahp article: latora, compagno trapani/a decision support tool for business models analysis international journal of the analytic hierarchy process 225 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.516 using the business model canvas as a basis, the number and percentage of frameworks containing each business model canvas component, in a set of twenty bm frameworks analyzed, was detected. results are summarized in table 2. table 2 bm frameworks containing each business model canvas component component of business model canvas framework kr ka kp c$ vp ch cr cs r$ number of bm frameworks containing the component 7 6 7 7 18 2 1 6 10 percentage of bm frameworks containing the component 35% 30% 35% 35% 90% 10% 5% 30% 50% table 2 shows that value propositions (vp) is the most common component detected. the revenue streams (r$) component is present in half of the cases and about one-third of the analyzed frameworks contain key resources (kr), key activities (ka), key partnerships (kp), cost structure (c$) and customer segments (cs) as components, while the components channels (ch) and customer relationships (cr) are uncommon. the framework of osterwalder and pigneur was used in recent studies as a starting point for specific applications and to support the implementation of a productservice systems (bocken et al., 2014). the business model canvas was used to identify and classify the characteristics of the pss business model, to analyze the company business context and allow the choice of the appropriate type of pss and also to support pss business model definition in the capital goods companies (barquet et al., 2011; barquet et al., 2013; adrodegari et al., 2016; azevedo & ribeiro, 2013; peillon et al., 2015). referring to the lack of a business model decision support tool (bm-dst), the literature review concerning the evaluation of potential bm alternatives shows that some authors have addressed, although marginally, the issue of evaluation of potential bm solutions, often just to validate the bm framework proposed by the same authors (osterwalder et al., 2002; osterwalder & pigneur, 2010; casadesusmasanell & ricart, 2010). muegge (2012) provides a tool for business model analysis, which includes an operative process and a worksheet for describing a business model in a concise and explicit manner. hacklin and wallnöfer (2012) explored the implications and limitations of applying the business model as a strategizing device. the bm frameworks oriented to the evaluation of potential bm alternatives are largely qualitative; just two cases out of ten are defined by the authors as quantitative evaluation models, but they really cannot be classified as a business model decision support tool (bailetti 2009; shin and park 2009). boritz and white (2016) provided a synthesis of the research performed on business models, focusing on how business models are defined, what elements are considered in different frameworks, and how they can be presented to stakeholders to enable ijahp article: latora, compagno trapani/a decision support tool for business models analysis international journal of the analytic hierarchy process 226 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.516 and enhance their understanding of an entity’s value creation process. they argue that business models can be used by managers to identify the strengths and weaknesses of their value creation process, by auditors to assist in engagement planning and execution, and by analysts and investors to evaluate the value that an entity creates and will continue to create in the future. in addition, various stakeholders could use business model descriptions for a better understanding of the relationship between an entity’s strategy, resources and outcomes as well as related risks. 3. methodological approach advantages and disadvantages as well as costs and benefits that characterize decisions depend on multiple, often conflicting, points of view or criteria used in the decision-making activity. multiple criteria decision analysis (mcda) is a mathematical discipline which offers a realistic and naturally multidimensional approach to decision theory generating a considerable interest among scientists (bouyssou et al., 2000; figueira et al., 2005; de felice & petrillo, 2013). saaty (1976) developed the analytic hierarchy process, a mcda methodology based on pairwise comparisons among criteria and alternatives, in order to obtain an overall ranking able to represent a “rational decision”. the pairwise comparison, i.e. the definition of a relative importance between entities, according to a criterion, allows the priorities definition for intangible entities, which are free of scales of measurement by definition, but also for tangible entities evaluable on scales with “zero” point and measurement units (aczel & saaty, 1983; de felice & petrillo, 2014). a recent research work simplifies the traditional ahp methodology introducing the v-ahp version, as shown in figure 1 (compagno et al., 2013). the graph shows the flowchart of the v-ahp decision making process which includes: 1. the definition of the evaluation general objective; 2. the selection of evaluation criteria and sub-criteria; 3. the ahp-r rating of evaluation criteria and sub-criteria; 4. the analysis of the qualitative and quantitative performances of the alternatives; 5. the distinction between quantitative and qualitative criteria; 6. the use of the traditional "lean" rating in a relationship scale for the quantitative criteria; 7. the use of the relative ahp rating or of the absolute one (i.e. saaty scale) for the qualitative criteria; 8. merging/composition between "lean" rating and ahp one for the definition of the ranking of alternatives. ijahp article: latora, compagno trapani/a decision support tool for business models analysis international journal of the analytic hierarchy process 227 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.516 2. selecting criteria and sub-criteria of evaluation 3. criteria and/or sub-criteria ahp-r distributive rating 6. "lean" rating of alternatives 4. analysis of qualitative and quantitative alternatives performances 5. quantitative or qualitative criteria? 7. ahp-r or ahp-a rating of alternatives 1. defining the general objective of evaluation quantitative qualitative 8. v-ahp ranking of alternatives figure 1 v-ahp procedure studies on v-ahp arise from the need, highlighted by saaty (1986), to set priorities by pairwise comparisons for both entities: intangible ones, by definition without scales, and tangible ones, evaluable on scales with “zero” point and measurement units. the value-analytic hierarchy process allows the ranking of alternatives. it is the combination of the traditional ahp rating on qualitative criteria and “lean” rating on quantitative criteria. the latter is obtained by the ratio between the value of performance related to the i-th alternative and the sum of performance values related to all the alternatives under consideration. for the analytical discussion of the v-ahp we consider the array l having, as components, n quantitative performance values on ratio scale: 𝐿 = [ 𝑙1 ⋮ 𝑙𝑛 ] (1) the pairwise comparison, operated by the ratio of the n quantitative performance values, gives a matrix b having size n×n, rank 1 and principal eigenvalue λmax = n; columns of the matrix b are the linear combinations of n quantitative performance values. 𝐵 = [ 𝑙1 𝑙1 ⋯ 𝑙1 𝑙𝑛 ⋮ ⋱ ⋮ 𝑙𝑛 𝑙1 ⋯ 𝑙𝑛 𝑙𝑛𝑗] (2) ijahp article: latora, compagno trapani/a decision support tool for business models analysis international journal of the analytic hierarchy process 228 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.516 therefore, the following equation is valid: 𝑩𝑳 = 𝒏𝑳 (3) which can be expressed in a matrix form: [ 𝒍𝟏 𝒍𝟏 ⋯ 𝒍𝟏 𝒍𝒏 ⋮ ⋱ ⋮ 𝒍𝒏 𝒍𝟏 ⋯ 𝒍𝒏 𝒍𝒏𝒋] [ 𝒍𝟏 ⋮ 𝒍𝒏 ] = 𝒏 [ 𝒍𝟏 ⋮ 𝒍𝒏 ] (4) array l, having as components the n quantitative performance values, is then, for the matrix b, the principal eigenvector associated with the principal eigenvalue λmax = n. array w, containing the local weights of the n quantitative performance values, is normalized on unit; it can be obtained by the ratio of quantitative performance values, without the implementation of the traditional saaty ahp procedure. in particular, the distributive rating is realized by the ratio between the value of the performance related to the i-th alternative and the sum of the performance values related to all the alternatives under consideration equation 5. the ideal rating is then obtained by operating the ratio between the quantitative performance value related to the i-th alternative and the maximum quantitative performance value among the n alternatives under consideration in equation 6. 𝑤𝑖 = 𝑙𝑖 ∑ 𝑙𝑖 𝑛 𝑖=1 (5) 𝑤𝑖 𝐼 = 𝑙𝑖 𝑚𝑎𝑥𝑖=1 𝑛 {𝑙𝑖} (6) in this section, a business model decision support tool (bm-dst) that is able to evaluate potential bm alternatives is presented. this bm-dst is based on a modified mcda ahp procedure called value-analytic hierarchy process. the overall objective of the decision-making process, at the first level of the ahp hierarchy, is the selection of the best bm solution in a finite and bounded set of potential bm alternatives identified and assessed by the decision makers. evaluation criteria of potential bm alternatives, at the second level of the ahp hierarchy, are the bm framework components belonging to the framework selected; thus, they are the bm framework criteria used in decision analysis. the object to evaluate, at the third level of the ahp hierarchy, are potential bm alternatives that the decision maker intends to compare to each other, in order to choose the best bm solution in a rational way, taking into account the overall objective and predefined criteria. the process based on the proposed bm-dst is shown in figure 2 and it consists of nine phases that can be overlapped with the v-ahp procedure described in figure 1. ijahp article: latora, compagno trapani/a decision support tool for business models analysis international journal of the analytic hierarchy process 229 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.516 the bm-ldss process consists of: 1. bm framework selection; 2. bm framework criteria identification; 3. ahp rating of bm framework criteria; 4. bm alternatives identification; 5. quantitative/qualitative criteria discerning; 6. rating “lean” of bm alternatives under quantitative criteria, 7. ahp rating of bm alternatives under qualitative criteria; 8. v-ahp rating of bm alternatives; 9. best bm solution selection. 2. id entifying bm framework criteria 3. rating ah p of bm framework criteria 6. rating �lean of bm alternatives 4. id entifying bm alternatives 5. quantitative or qualitative criteria? 7. rating ahp of bm alternatives quantitative qualitative 8. rating v-ahp of bm alternatives 1. selecting a b m framework 9. selecting the best bm soluti on figure 2 business model decision support tool (bm-dst) process phases 1 to 4 can be defined as stages of the bm-dst creation. we select a bm framework by which to build and evaluate bm alternatives, also identifying qualitative and quantitative bm framework components i.e. identifying evaluation criteria of bm alternatives. the ahp based rating of bm framework criteria is calculated in order to identify a finite and bounded number of bm alternatives for the next evaluation step. phases 5 to 7 can be defined as evaluation phases of bm alternatives. a distinction is made between quantitative criteria for which a “lean” rating of alternatives is executed and qualitative criteria for which the traditional ahp rating of the alternative is considered; finally, we calculate v-ahp rating of alternatives. ijahp article: latora, compagno trapani/a decision support tool for business models analysis international journal of the analytic hierarchy process 230 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.516 phase 8 is the selection phase of the best bm solution among the potential bm alternatives previously identified and evaluated. it is logical to select the business solution with a maximum v-ahp rating value, but it is also opportune to remember that a decision support system does not replace the decision maker in decisionprocess but just supports his/her activity. 4. case study a case study and the related numerical example is presented, concerning the proposed business model decision support tool (bm-dsts) in order to compare "pure product” (pp) selling (where the product is a turnkey industrial plant) with "product-service system" according to the definition of "product-oriented system" (pos) and "use-oriented system" (uos) shown in figure 3 (tukker, 2004). value in product product-service system (pss) value in service product (tangible) service (intangible) pure product (pp) product oriented system (pos) use oriented system (uos) result oriented system (ros) pure service (ps) figure 3 classification of pss proposed by tukker (2004) analyzing a similar issue, kujala et al. (2010) described a specific six components bm framework and identified three potential business solutions: (a) transactional project deliveries which are simple systems without additional service components (i.e. pp); (b) project led solutions in which operational services are important parts of the offering but the core is still the project delivery (i.e. pos); (c) life-cycle solutions where project and service components are offered as a single integrated solution (i.e. uos) for the asset performance management during asset lifecycle (kujala et al. 2011). this solution requires a durable partnership between the contractor and customer. starting from the above-mentioned results obtained by kujala et al., a new bm-dst is proposed in order to evaluate the three potential business solutions and to select the best one based on a defined strategy. the following steps refer to each phase shown in figure 2: 1. the judgments were expressed through dedicated brainstorming by a panel of five industrial plant experts assuming the role of decision makers and adopting the perspective of a company (oem provider or epc general ijahp article: latora, compagno trapani/a decision support tool for business models analysis international journal of the analytic hierarchy process 231 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.516 contractor) which is able to extend its value proposition from solution (a) to (c). one of the experts had the role of customer, in order to take into account the customer’s point of view and its relevance in pss business. 2. the bm framework, selected by the decision maker to carry out the numerical example, is the business model canvas proposed by osterwalder and pigneur (2010) which defined a bm as “the rationale of how an organisation creates, delivers and captures value”. 3. the identified bm framework's criteria are the nine business model canvas components:  customer segments (cs);  value propositions (vp);  channels (ch);  customer relationships (cr);  revenue streams (r$);  key resources (kr);  key activities (ka);  key partnerships (kp);  cost structure (c$). osterwalder and pigneur identified two areas in the business model canvas:  the efficiency area, containing the components key resources (kr), key activities (ka), key partnerships (kp), cost structure (c$), and  the value area, containing the components customer segments (cs), value propositions (vp), channels (ch), customer relationships (cr), revenue streams (r$). 4. the analytic hierarchy process (ahp) generates a rating of bm framework criteria. the pairwise comparison matrix of the nine business model canvas components, shown in the first ten columns of table 3, was performed using the saaty fundamental scale. in this specific numerical example, judgments underline a bm based on the offered value with emphasis on value propositions (vp) and revenue streams (r$). table 3 criteria pairwise comparison matrix and results kr ka kp c$ vp ch cr cs r$ wc kr 1 1 1 1 1/9 1/5 1/5 1/5 1/7 4% ka 1 1 1 1 1/9 1/5 1/5 1/5 1/7 4% kp 1 1 1 1 1/9 1/5 1/5 1/5 1/7 4% c$ 1 1 1 1 1/9 1/5 1/5 1/5 1/7 4% vp 9 9 9 9 1 1 5 5 3 33% ch 5 5 5 5 1 1 1 1 1/3 12% cr 5 5 5 5 1/5 1 1 1 1/3 9% cs 5 5 5 5 1/5 1 1 1 1/3 9% r$ 7 7 7 7 1/3 3 3 3 1 21% the value area dominates the efficiency area: i.e. company is strategically focused on actions to increase customer satisfaction and loyalty offering a high value, without sacrificing revenue. the maximum normalized eigenvector w, shown in the last column of table 3, reflects judgments ijahp article: latora, compagno trapani/a decision support tool for business models analysis international journal of the analytic hierarchy process 232 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.516 made by the decision maker. the consistency ratio assumes the value cr = 0.03 (sufficiently lower than the suggested threshold value 0.10). the best business solution within a set of potential business alternatives. 5. bm alternatives are the three above-mentioned potential business solutions: (a) transactional project delivery, (b) project led solution, (c) lifecycle. figure 4 shows a graphic framework of the bm-dst based on business model canvas. the best bm solution key resources key activities key partners bm solution a cost structure value propositions channels customer relationships customer segments revenue streams bm solution b bm solution c figure 4 business model canvas decision support system 6. in this numerical example, only the cost structure (c$) is a quantitative criterion, while the other bm components are qualitative criteria. 7. the cost structure (c$) describes all costs (e.g. only epc costs in case a plus service and operations management costs in case b and c) that the company would sustain for each business solution. financial flows are considerably different from transactional project (pp case would provide a reduced time scale for return on investment for the company) to uos solution for which the cash-flow is extended to the product operational period and payment could be based on the pss operational performances, as defined in the provider-customer agreement. the actual costs for each solution was then calculated obtaining the values reported in table 4. the “lean” rating of bm alternatives under quantitative criteria was carried out as described in equation 5. table 4 shows cost values on monetary scale, inverse of cost values, and “lean” rating of alternatives calculated on inverse cost values, in order to prefer the alternative with minimum cost. ijahp article: latora, compagno trapani/a decision support tool for business models analysis international journal of the analytic hierarchy process 233 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.516 table 4 the “lean” rating of a quantitative bm criterion c$ [m€] [m€ -1 ] wc$ a 12.50 8.00e-02 40% b 15.00 6.67e-02 33% c 18.00 5.56e-02 27% 8. ahp rating of bm alternatives under each qualitative criterion was carried out in the traditional manner. table 5 shows a pairwise comparison matrix, maximum normalized eigenvector and consistency ratio of the bm alternatives under each qualitative criterion. the elements which guided the alternative assessment for each criterion are discussed below. table 5 alternatives pairwise comparison matrix and results kr a b c wkr a 1 1/3 1/3 14% b 3 1 1 43% c 3 1 1 43% cr=0.00 ka a b c wka a 1 1/3 1/7 8% b 3 1 1/5 19% c 7 5 1 73% cr=0.06 kp a b c wkp a 1 1/3 1/5 10% b 3 1 1/3 26% c 5 3 1 64% cr=0.03 vp a b c wvp a 1 1/3 1/5 10% b 3 1 1/3 26% c 5 3 1 64% cr=0.03 cs a b c wcs a 1 1/3 1/3 14% b 3 1 1 43% c 3 1 1 43% cr=0.00 ch a b c wch a 1 1/3 1/3 14% b 3 1 1 43% c 3 1 1 43% cr=0.00 cr a b c wcr a 1 1/3 1/5 10% b 3 1 1/3 26% c 5 3 1 64% cr=0.03 r$ a b c wr$ a 1 1/3 1 20% b 3 1 3 60% c 1 1/3 1 20% cr=0.00 key resources (kr) can be physical, financial, intellectual, or human resources which allow a company to deliver the value proposition to different market segments. they can be owned or leased by the company or acquired from key partners. according to kujala et al. (2010), the oem tends to offer a pos solution to customers with limited skills in maintenance service, and a uos solution to ijahp article: latora, compagno trapani/a decision support tool for business models analysis international journal of the analytic hierarchy process 234 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.516 customers who perceive the technological complexity of the offered product. psss business model requires considerable investments specifically for human resources recruitment, corporate culture and top management commitment (barquet et al., 2011; adrodegari et al., 2016). in order to operate successfully, a business model requires the execution of key activities (ka). the pp transactional solution emphasizes core capability in the physical product, separate service units, and product r&d (galbraith, 2002; helander & möller, 2007; oliva & kallenberg, 2003). the pss providers have to guarantee other key activities before, during and after the unit operations so that service delivery can generate value for the customer (barquet et al. 2011) the pos solution requires localization and centralization of tasks and service capacity utilization (helander & möller 2007; oliva and kallenberg 2003). the life-cycle, user-oriented solution (uos) point out customer-facing units, the strategic role of marketing, business/market competencies and solution repeatability (davies et al., 2006; vargo & lusch, 2004). the key partnerships (kp) component highlights the network of suppliers and partners who cooperate to achieve business success. companies create partnerships for many reasons including optimizing their business models, reducing risk or acquiring (key) resources or (key) activities. this element was evaluated through the nature of relationships (long/short term, price/strategic based) and the position in the value network (adrodegari et al., 2016). in regards to the case study, the pp solution involves the management of a supplier network, services and non-core business (cohen et al. 2006; davies 2004; helander and möller 2007). in the pos solution, the pss provider often uses network service companies that limit customization (davies, 2004; oliva & kallenberg, 2003; windahl et al., 2004). the life-cycle solution emphasizes a large share of value stream, of data and information, the role of external partners and a network of customers (cova & salle, 2008; davies, 2004; windahl et al., 2004). the value propositions component (vp) describes the set of products and services that create value for a specific customer segment, solving a problem or satisfying a need. the pp solution offers a cutting-edge product, warranty and spare-parts availability (with additional costs for customer) to ensure the proper functionality of the system (galbraith 2002; gebauer 2008; markeset & kumar 2004). this solution is preferred by customers who perceive the ownership of the product as a value. in pss solutions it is necessary to define what the customer considers as a source of value; in the pos solution it could be the reduction of the capital costs and known operational costs; in uos solution the co-development, with the pss provider, of a solution that offers best performance and outcome, according to pss provider service portfolio (gebauer, 2008; davies, 2004; davies et al., 2006; vargo & lusch 2004). the customer segments component (cs) defines the different groups of people or organizations an enterprise aims to reach and serve. transactional project deliveries (pp) are usually intended for customers with independent strategies and in-house technological know-how (helander & möller 2007; markeset & kumar 2003). project led solutions (pos) are preferred by customers for which maintenance is a non-core process, so that flexibility can be reached through outsourcing (gebauer , 2008; oliva & kallenberg, 2003; windahl et al., 2004). the uos solution is preferred by customers who rely on pss provider expertise to optimize operations ijahp article: latora, compagno trapani/a decision support tool for business models analysis international journal of the analytic hierarchy process 235 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.516 and who are willing to engage in long-term relationships (davies, 2004; penttinen & palmer, 2007; windahl et al., 2004; adrodegari et al., 2016). the channels component (ch) describes how a company communicates with and reaches its customer segments to deliver a value proposition. referring to the supplier’s marketing approach, kujala et al. (2011) argued that pss providers have more success in delivering a pss solution (pos or uos business model) when the project supplier has a proactive marketing approach. proactive marketing and cocreation of the life-cycle offering is suggested to be especially useful when the solution is perceived to be complex and risky and the pss solution can be offered and priced as more attractive than the transactional solution (crespin-mazet & ghauri, 2007). the customer relationships component (cr) highlights the types of relationships a company establishes with specific customer segments. according to oliva and kallenberg (2003) and kujala et al. (2010), customer relationships increase along a continuum from the transactional project deliveries (pp) to the user-oriented solution (uos). in this scenario, the emphasis of the business model changes from transactional to relationship-based. the revenue streams component (r$) represents the cash-flow a company generates from each customer segment. a business model can involve two different types of revenue streams: transaction revenues resulting from one-off customer payment or recurring revenues resulting from ongoing performance-based payments or from post-purchase customer support deliveries. the pp solution determines transactional revenue (cohen et al., 2006; markeset & kumar, 2005; slywotsky et al., 1998). in the project led solutions (pos), the company assumes a part of the operational risks and can be rewarded with a premium (oliva & kallenberg, 2003; sawhney, 2006). the uos solution determines gain-sharing, performance guarantees, solution profits and pricing on second-best options (davies et al., 2006; sawhney, 2006; slywotzky, et al. 1998). 9. v-ahp rating of bm alternatives was obtained combining the results of tables 3, 4 and 5. in particular, table 3 shows local weights of bm framework criteria; table 6, arising as a combination of tables 4 and 5, depicts local weights of bm alternatives. figure 5 summarizes local weight graphically. table 7 local weight of bm alternatives kr ka kp c$ vp ch cr cs r$ a 14% 8% 10% 40% 10% 14% 10% 14% 25% b 43% 19% 26% 33% 26% 43% 26% 43% 60% c 43% 73% 64% 27% 64% 43% 64% 43% 15% v-ahp rating of bm alternatives is then obtained applying the principle of hierarchical composition in order to calculate bm alternatives global weights by processing the matrix product between the 3×9 matrix of the bm alternatives local weight and the 9×1 array of the bm criteria global weights. ijahp article: latora, compagno trapani/a decision support tool for business models analysis international journal of the analytic hierarchy process 236 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.516 figure 5 local weights of bm alternatives table 7 and figure 6 show the bm alternatives global weights i.e. v-ahp rating of evaluated bm alternatives. table 7 rating v-ahp of bm alternatives bm alternative v-ahp rating a 16% b 37% c 47% figure 6 v-ahp rating of bm alternatives the business alternative to select, based on results of the implemented business model decision support tool (bm-dst), is the business alternative (c) uos 0% 20% 40% 60% 80% kr ka kp c$ vpch cr cs r$ a b c a, 16% b, 37% c, 47% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% a b c ijahp article: latora, compagno trapani/a decision support tool for business models analysis international journal of the analytic hierarchy process 237 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.516 solution which, with a global weight of 47%, emphasizes the pss used-oriented performance. 6. conclusion over the past fifteen years the bm concept has emerged in the scientific literature and business world. it can be seen as a tool for strategic business analysis that is still not very mature and affected by uncertainty in definition and use. in the broad set of definitions characterizing the bm concept, one which we considered for the purposes of this paper defines it as “the rationale of how an organisation creates, delivers and captures value”. many authors consider the bm an excellent tool for the strategic evaluation of business ideas, but probably because of the young age of the concept only a few of them have ventured into the design of a specific tool to evaluate potential business solutions. the literature review shows just two cases out of ten defined by the authors as quantitative evaluation models, but in fact not classifiable as real business model decision support tool (bm-dst). in this paper, we propose a business model decision support tool (bm-dst) to evaluate potential business solutions based on a quali-quantitative methodology of multiple criteria decision analysis (mcda) called analytic hierarchy process (ahp). more precisely our tool is based on a modified ahp procedure called value-analytic hierarchy process (v-ahp) which combines traditional ahp procedures for the rating under qualitative criteria and “lean” procedures for the rating under quantitative criteria. a numerical example of the business model decision support tool (bm-dst) based on the business model canvas proposed by osterwalder and pigneur (2010) was carried out in the field of industrial plants, specifically in the context of projectbased firms, in order to select the best bm solutions from among the following: (a) transactional project deliveries (pp), (b) project led solutions (ros) and (c) lifecycle solutions (uos). a software application was implemented by using a common spreadsheet in order to prove the “lean” characteristic of our business model decision support tool (bmdst) which is easy to implement in any business context. regarding managerial implications, it is worth emphasizing the possibility that the business model decision support tool (bm-dst) offers to decision makers who perform strategic roles within companies. this bm-dst allows the pre-evaluation of all potential business solutions that a company could implement and the choice of the best one on the basis of points of view or criteria considered relevant by the company management. finally, future implications may concern the application of group decision making techniques taking into account several points of view and related weights while adopting both the supplier and customer perspective in order to address the holistic evaluation of business solutions. ijahp article: latora, compagno trapani/a decision support tool for business models analysis international journal of the analytic hierarchy process 238 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.516 references aczel, j., saaty, t.l. 1983. procedures for synthesizing ratio judgments. journal of mathematical psychology 27, 93-102. doi: https://doi.org/10.1016/0022-2496(83)90028-7 adrodegari, f., saccani, n., and kowalkowski, c. 2016. a framework for pss business models: formalization and application, procedia cirp, 47, 519-524. doi: https://doi.org/10.1016/j.procir.2016.03.073 al-debei, m. m., avison, d. 2010. developing a unified framework of the business model concept. european journal of information systems, 19, 359-376. doi: https://doi.org/10.1057/ejis.2010.21 amit, r., zott, c. 2001. value creation in e-business. strategic management journal, 22, 493-520. doi: https://doi.org/10.1002/smj.187 azevedo, a., and ribeiro, h. 2013. new business models elements oriented to productservice machinery industry. in advances in sustainable and competitive manufacturing systems (1277-1289). springer international publishing. doi: https://doi.org/10.1007/978-3319-00557-7_104 bailetti, t. 2009. how open source strengthens business models. open source business resource. baden-fuller, c., and haefliger, s. 2013. business models and technological innovation. long range planning 46(6), 419-426. doi: https://doi.org/10.1016/j.lrp.2013.08.023 barquet, a.p.b., cunha, v.p., de oliveira, m.g., and rozenfeld, h. (2011). business model elements for product-service system. 3rd cirp international conference on industrial product service systems, braunschweig, 332–337. doi: https://doi.org/10.1007/978-3-64219689-8_58 barquet, a.p.b., de oliveira, m.g., román amigo, c., cunha, v.p., and rozenfeld, h. (2013). employing the business model concept to support the adoption of product–service systems pss. industrial marketing management, 42, 693-704. doi: https://doi.org/10.1016/j.indmarman.2013.05.003 bocken, n. m. p., short, s. w., rana, p., and evans, s. (2014). a literature and practice review to develop sustainable business model archetypes. journal of cleaner production, 65, 42-56. doi: https://doi.org/10.1016/j.jclepro.2013.11.039 boritz, j. e., and white, g. 2016. business models and business model descriptions: a research synthesis. doi: http://dx.doi.org/10.2139/ssrn.2715555 bouyssou, d., marchant, t., pirlot, m., perny, p., tsoukiàs a., vincke, ph. (2000). evaluation and decision models: a critical perspective. dordrecht: kluwer academic. doi: https://doi.org/10.1057/palgrave.jors.2601380 casadesus-masanell, r., ricart, j. e. 2010. from strategy to business models and to tactics. long range planning, 43, 195-215. doi: https://doi.org/10.1016/j.lrp.2010.01.004 chesbrough, h. w., rosenbloom, r. s. (2002). the role of the business model in capturing value from innovation: evidence from xerox corporation’s technology spinoff companies. industrial and corporate change, 11, 533-534. doi: https://doi.org/10.1093/icc/11.3.529 cohen, m.a., agrawal, n., agrawal, v. (2006). winning in the aftermarket. harvard business review, 84(5), 129-134. ijahp article: latora, compagno trapani/a decision support tool for business models analysis international journal of the analytic hierarchy process 239 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.516 compagno, l., d’urso, d., latora, a. g., trapani, n. (2013). the value-analytic hierarchy process: a lean multi criteria decision support method. manufacturing modelling, management and control, 7(1), 875-880. doi: https://doi.org/10.3182/20130619-3-ru3018.00573 cova, b., salle, r. (2008). marketing solutions in accordance with the sd logic: co-creating value with customer network actors. industrial marketing management, 37(3), 270-277. doi: https://doi.org/10.1016/j.indmarman.2007.07.005 crespin-mazet, f., pervez, ghauri p. (2007). co-development as a marketing strategy in the construction industry. industrial marketing management, 36(2), 158-172. doi: https://doi.org/10.1016/j.indmarman.2006.02.011 davies, a. (2004). moving base into high-value integrated solutions: a value stream approach. industrial and corporate change, 13(5), 727-756. doi: https://doi.org/10.1093/icc/dth029 davies, a., brady, t., and hobday, m. (2006). charting a path toward integrated solutions. mit sloan management review, 47(3), 39-48. de felice, f., petrillo, a., (2013). multicriteria approach for process modelling in strategic environmental management planning. international journal of simulation and process modelling, 8(1), 6-16. doi: 10.1504/ijspm.2013.055190 de felice, f., petrillo, a., (2014). proposal of a structured methodology for the measure of intangible criteria and for decision making. international journal of simulation and process modelling, 9(3), 157-1660. doi: 10.1504/ijspm. 2014.064392. doganova, l., marie eyquem-renault, m. (2009). what do business models do? innovation devices in technology entrepreneurship. research policy, 38(10), 1559-1570. doi: https://doi.org/10.1016/j.respol.2009.08.002 d’urso, d., latora, a., trapani, n. (2011). the analytical hierarchy process for e-scouting: decision-making for industrial procurement process. proceedings of the international symposium on the analytic hierarchy process isahp, sorrento italy. doi: figueira, j., greco, s., ehrgott, m. (2005). multiple criteria decision analysis. state of the art surveys. boston: springer science. doi: https://doi.org/10.1007/b100605 galbraith, j.r. (2002.) organizing to deliver solutions. organizational dynamics, 31(2), 194207. doi: https://doi.org/10.1016/s0090-2616(02)00101-8 george, g., bock, a. j. (2011). the business model in practice and its implications for entrepreneurship research. entrepreneurship theory and practice, 35, 83-111. doi: https://doi.org/10.1111/j.1540-6520.2010.00424.x gebauer, h. (2008). identifying service strategies in product manufacturing companies by exploring environment–strategy configurations. industrial marketing management, 37(3), 278-291. doi: https://doi.org/10.1016/j.indmarman.2007.05.018 hacklin, f., and wallnöfer, m. (2012). the business model in the practice of strategic decision making: insights from a case study. management decision, 5(2), 166-188. doi: https://doi.org/10.1108/00251741211203515 ijahp article: latora, compagno trapani/a decision support tool for business models analysis international journal of the analytic hierarchy process 240 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.516 hedman, j., kalling, t. (2003). the business model concept: theoretical underpinnings and empirical illustrations. european journal of information systems, 12(1), 49-59. doi: https://doi.org/10.1057/palgrave.ejis.3000446 helander, a., möller, k. (2007). system supplier's customer strategy. industrial marketing management, 36(6), 719-730. doi: https://doi.org/10.1016/j.indmarman.2006.05.007 johnson, m. w., christensen, c. c., kagermann, h. (2008). reinventing your business model. harvard business review, 86(12), 50-59. johnson, m. w. (2010). seizing the white space: business model innovation for growth and renewal. harvard business press. kujala, j., artto, k., parhankangas, a. (2008). factors influencing design and performance of the business model of a project-based firm. project perspectives, 31, 14-17. kujala, s., artto, k., aaltonen, p., turkulainen, v. (2010). business models in project-based firms–towards a typology of solution-specific business models. international journal of project management, 28(2), 96-106. doi: https://doi.org/10.1016/j.ijproman.2009.08.008 kujala, s., kujala, j., turkulainen, v., artto, k., aaltonen, p., wikström, k. (2011). factors influencing the choice of solution-specific business models. international journal of project management, 29(8), 960-970. doi: https://doi.org/10.1016/j.ijproman.2011.01.009 magretta, j. (2002). why business models matter. harvard business review, 80(5), 86-92. markeset, t., kumar, u. (2003). design and development of product support and maintenance concepts for industrial systems. journal of quality in maintenance engineering, 9(4), 376-392. doi: https://doi.org/10.1108/13552510310503231 markeset, t., and kumar, u. (2004). dimensioning of product support: issues, challenges, and opportunities. annual reliability and maintainability symposium: los angeles, california, ieee communications society, 2004, 565-570. doi: 10.1109/rams.2004.1285507 markeset, t., kumar, u. (2005). product support strategy: conventional versus functional products. journal of quality in maintenance engineering, 11(1), 53-67. doi: https://doi.org/10.1108/13552510510589370 morris, m., schindehutte, m., allen, j. (2005). the entrepreneur’s business model: toward a unified perspective. journal of business research, 58, 726-35. doi: https://doi.org/10.1016/j.jbusres.2003.11.001 muegge, s. (2012). business model discovery by technology entrepreneurs. technology innovation management review. 5-16. osterwalder, a., lagha, s.b., pigneur, y. (2002). an ontology for developing e-business models. in value creation for e-business models (65-97). butterworth-heineman. doi: https://doi.org/10.1016/b978-075066140-9/50006-0 osterwalder, a., pigneur, y. (2010). business model generation: a handbook for visionaries, game changers, and challenger. new jersey: john wiley and sons. doi: https://doi.org/10.1111/j.1540-5885.2012.00977_2.x peillon, s., pellegrin, c., and burlat, p. (2015). exploring the servitization path: a conceptual framework and a case study from the capital goods industry. production planning and control, 26(14-15), 1264-1277. doi: https://doi.org/10.1080/09537287.2015.1033492 http://www.google.ca/search?hl=it&tbo=p&tbm=bks&q=inauthor:%22mark+w.+johnson%22&source=gbs_metadata_r&cad=8 http://www.google.ca/search?hl=it&tbo=p&tbm=bks&q=inauthor:%22alexander+osterwalder%22&source=gbs_metadata_r&cad=8 http://www.google.ca/search?hl=it&tbo=p&tbm=bks&q=inauthor:%22yves+pigneur%22&source=gbs_metadata_r&cad=8 ijahp article: latora, compagno trapani/a decision support tool for business models analysis international journal of the analytic hierarchy process 241 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.516 penttinen, e., palmer, j. (2007). improving firm positioning through enhanced offerings and buyer-seller relationships. industrial marketing management, 6(5), 552-564. doi: https://doi.org/10.1016/j.indmarman.2006.02.005 saaty, t.l. (1986). absolute and relative measurement with the ahp. the most livable cities in the united states. socio-economic planning sciences, 20, 327-331. doi: https://doi.org/10.1016/0038-0121(86)90043-1 saaty, t.l. (1990). the ahp: how to make a decision. european journal of operational research, 48, 9-26. doi: https://doi.org/10.1016/0377-2217(90)90057-i saaty, t.l., france j.w., valentine k.r., (1991). modelling the graduate business school admissions process. socio-economic planning sciences, 25, 155-162. doi: https://doi.org/10.1016/0038-0121(91)90013-h saaty, t.l., ozdemir m.s. (2003). why the magic number seven plus or minus two. mathematical and computer modelling, 38(3), 233–244. doi: https://doi.org/10.1016/s08957177(03)90083-5 saaty, t.l. (2006). fundamentals of decision making. pittsburgh: rws publications. saaty, t.l. (2008). relative measurement and its generalization in decision making. racsam, 102(2), 251-318. sawhney, m. (2006). going beyond the product: defining, designing and delivering customer solutions. in the service-dominant logic of marketing: dialog, debate and directions (365380). new york: m.e. sharpe. shin, j., park, y. (2009). on the creation and evaluation of e-business model variants: the case of auction. industrial marketing management, 38(3), 324-337. doi: https://doi.org/10.1016/j.indmarman.2007.06.017 slywotzky, a. j., morrison, d. j., andelman, b. (2007). the profit zone: how strategic business design will lead you to tomorrow's profits. random house digital, inc. stevens, s.s. (1946). on the theory of scales of measurement. science, 25(2684), 677–680. doi: 10.1126/science.103.2684.677 teece, d. j. (2010). business models, business strategy and innovation. long range planning, 43, 172-194. doi: https://doi.org/10.1016/j.lrp.2009.07.003 timmers, p. (1998). business models for electronic markets. electronic markets, 8(2), 3-8. tukker, a. (2004). eight types of product-service system: eight ways to sustainability? experiences from suspronet. business strategy and the environment, 13(4), 246-260. doi: https://doi.org/10.1002/bse.414 vargo, s.l., lusch, r. f. (2004). evolving to a new dominant logic for marketing. journal of marketing, 68(1), 1-17. doi: https://doi.org/10.1509/jmkg.68.1.1.24036 windahl, c., andersson, p., berggren, c., nehler c. (2004). manufacturing firms and integrated solutions: characteristics and implications. european journal of innovation management, 7(3), 218-228. doi: https://doi.org/10.1108/14601060410549900 ijahp article: latora, compagno trapani/a decision support tool for business models analysis international journal of the analytic hierarchy process 242 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.516 zott, c., amit, r. (2008). the fit between product market strategy and business model: implications for firm performance. strategic management journal, 29, 1-26. doi: https://doi.org/10.1002/smj.642 zott, c., amit, r. (2010). designing your future business model: an activity system perspective. long range planning, 43, 216-226. ijahp article: macura, selmic/railway project evaluation with dynamic prioritiesserbia case study international journal of the analytic hierarchy process 15 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.292 railway project evaluation with dynamic priorities –serbia case study dragana macura university of belgrade faculty of transport and traffic engineering vojvode stepe 305, belgrade, serbia d.macura@sf.bg.ac.rs milica selmic university of belgrade faculty of transport and traffic engineering vojvode stepe 305, belgrade, serbia m.selmic@sf.bg.ac.rs 1 abstract strategic decisions, such as transport investments, depend on a number of factors of different relevance that are often changeable over time. evaluation of transport projects is a complex and difficult task, but also crucial for a company’s success in the market. we assume that certain relations of the system elements are functions of time, and thus we apply a dynamic approach – dynamic priorities in multi-criteria decision making. the topic of this paper is time dependent multi-criteria decision making in a transport projects evaluation. the model is tested on real data from the serbian railway network. keywords: transport projects, railway project evaluation, dynamic priorities 1. introduction project evaluation is a very delicate, complex and difficult management task, with an aim to select and rank the considered projects. evaluation of transport projects (including railway projects) has often been an essential step in the transport company’s success. decision making in investment planning is a very complicated process because of many relevant factors, such as: stakeholders (owners, regulators, market, politicians, etc.), system boundaries, transparency, and heterogeneous criteria (jowitt, 2013). cost benefit analysis (cba) is a well-known method in project evaluation. this traditional approach is the most used method in the transport sector. in the rail sector, cba is applied 60% of the time for project evaluations. multi-criteria approach (mca) 1 acknowledgements this research is partially supported by the ministry of science of serbia, grant number 36022. ijahp article: macura, selmic/railway project evaluation with dynamic prioritiesserbia case study international journal of the analytic hierarchy process 16 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.292 is applied in 14% of cases, and other methods are applied 26% of the time. in road projects, cba is used in 56% of cases, and mca is used in 20%. in internal waterway projects, the proportion of the methods used is 32% for cba and 6% for mca, and in sea transport projects evaluation the situation is very similar 2 . there are numerous approaches and methods in the evaluation of transport projects, such as cba (berechman and paaswell, 2005, van wee 2007), maut multi attribute utility theory (tsamboulas, 2007), gp goal programming (ahern and anandarajah, 2007), ahp analytic hierarchy process (lee, 1998; yedla and shrestha, 2003; ferrari, 2003; gercek et al., 2004; tudela et al.,2006; caliskan, 2006), and anp analytic network process (shang et al., 2004; piantanakulchai, 2005; wey and wu, 2007; chang et al., 2009; longo et al., 2009; macura et al., 2011). shiftan et al. (2002) give a review of different methods and approaches to transport project evaluation with an emphasis on the specific features of cba, and quantitative and qualitative multi-criteria analysis in this field. the majority of the mentioned references from the last decade critically analyzed the application of cba in the evaluation of transport projects, and suggested applying the multi-criteria methods to solving the problems being considered. the process of good decision making depends on conditions in the future which normally vary over time; thus, good decision making requires evaluation of likelihood and preference of conditions during different time periods. this is the primary reason for the use of the dynamic approach in the decision making process. there are situations in which changes occur in the structure of a problem, new criteria can be added, or old criteria replaced. sometimes judgments on the criteria change, but the criteria remain the same. there are also cases where judgments on the criteria remain the same; however, judgments on the alternatives change over time. all these combinations are possible in practice (saaty 2007a). basic assumptions of the applied mathematical computation for this method are defined in saaty’s book (2007a). the aim of this paper is to develop a model of an evaluation of railway projects as a support system to decision makers. the model has two parts, a static part and a dynamic part. the authors used the analytic hierarchy process (ahp) approach for the static part of the model, and the dynamic hierarchy process (dhp) approach to dynamic priorities in the dynamic part of the model. the model that was developed was tested on railway infrastructure projects in the serbian railway network. in this model, the main decision maker was pe serbian railways. this paper is organized as follows: it begins with the introduction, then the next section is dedicated to the relevant literature review; next the basic assumptions of the applied approaches, ahp and dhp, are presented. the second part of this section shows the dynamic model of the railway project evaluation, with all system elements and their 2 http://heatco.ier.uni-stuttgart.de/hd1final.pdf http://heatco.ier.uni-stuttgart.de/hd1final.pdf ijahp article: macura, selmic/railway project evaluation with dynamic prioritiesserbia case study international journal of the analytic hierarchy process 17 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.292 mutual relations included. next, results and a discussion are presented followed by some concluding remarks. 2. literature review the literature reviewed falls into three categories: applications of dynamic relations and the ahp approach; applications of the dynamic network process; and hybrid staticdynamic ahp models. this is the first application of dynamic priorities, as well as a dynamic hierarchy process, to this particular transport issue. boateng et al. (2012) analyzed risks in megaprojects, and focused on the dynamic relations in the model after having defined all the relevant risks. researchers and project managers used this model in order to understand various social, economic and environmental systems within a holistic view. tsamboulas et al. (2007) suggested a new multi-criteria analysis approach with the following characteristics: unification of the criteria and differentiation of the project performance over time, as a dynamic variable, and a new approach to transformation of the physical scales to artificial ones. the application of the proposed approach is demonstrated using transportation infrastructure investments. the dynamic hierarchy process was used in gonzáleza et al. (2003) to help the product development team make effective decisions in satisfying customers’ requirements with limited resources, in the case of school furniture design. a dynamic, cross-functional team organization was applied. a simple form of quality function deployment was used to identify the desirable product design, safety, and service features. numerous papers in the relevant literature deal with the dynamic network process. dynamic models try to reflect changes in real or simulated time, taking into account that the network model components are constantly evolving. these models use concepts of state variables, flows, and feedback processes. the network economy was discussed in fiala (2006), referring to the global relationship between economic elements characterized by massive connectivity. the central act of the new era was to connect everything in deep web networks at a number of levels of mutually interdependent relations, where resources and activities were shared, markets enlarged and costs and risks reduced. for such a network, the anp was used, as well as its extension, the dynamic network process, which can deal with time dependent priorities in a networked economy. fiala (2007) also applied the anp approach to designing auctions, which requires a multidisciplinary effort consisting of contributions by economics, operation research, informatics, and other disciplines. the multiple evaluation criteria can be used. there is dependence between sellers, buyers, criteria and bundles of items. a variety of feedback processes create a complex system of items. the preferences of bundles of items can be evaluated by the anp approach. in this paper, the dynamic network process dealt with time dependent priorities in combinatorial auctions. market segmentation, as a critical point of business, was analyzed in nasrabadi et al. (2013). this paper aimed to model a strategy-aligned fuzzy approach to market segment evaluation and selection. a modular decision making support system was developed to select an optimum segment with its appropriate strategies. the dynamic network process was applied to prioritize segment-strategies according to five competitive force factors. ijahp article: macura, selmic/railway project evaluation with dynamic prioritiesserbia case study international journal of the analytic hierarchy process 18 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.292 this model was supported by a case study on strategic priority difference within a short and long-term consideration. feglar and levy (2003) applied dynamic anp in order to improve decision making support in information and communication technology. they focused their attention on the classification of various ict based innovations. there are several papers in the relevant literature that mention the hybrid static dynamic ahp model. wang et al. (2008) used hybrid ahp and the fuzzy mathematics method to deal with dynamic and static, and fixed and non-fixed quantified influence factors. according to the weight matrix and the matrix of the membership degrees, the fuzzy synthetic judge model was established, and the synthetic superior degree of the mining methods based on the influence factors was obtained by the numeration of the fuzzy mathematics methods. in order to make decision-making more objective and accurate, based on the distinction between experts' static weights and experts' dynamic weights, liu et al. (2007) researched a method to decide experts' dynamic weights in interactive decision-making and establish a definition of consensus degree. based on the research of consistency and compatibility, the paper developed a framework of group decision-making for ahp based on experts' dynamic weights. fang and yang (2011) presented the defect of the traditional static ahp nets, and suggested the dynamic ahp net including correlative definitions and illustrations. based on the net definitions, the dynamic ahp net formulas and evaluation node priority algorithm were proposed. a simulation verification example was given and the numeric evaluation result was analyzed. the result was compared with that calculated by the traditional ahp. the result showed that the proposed dynamic ahp net could resolve the uncertainty problem caused by simulation verification. benítez et al. (2012) points out the weakness of static input mode in the ahp approach. they identify this weakness as the need for users to provide all the preference data at the same time, and have the criteria defined from the start. to overcome this weakness, they proposed a framework that allows users to provide partial and/or incomplete preference data at multiple times, i.e. they included the dynamic input mode assumption. an algorithm was developed to determine the new priority vector from the users' new input. 3. model for railway project evaluation (static and dynamic approaches) the model of railway project evaluation makes a rank list of the projects considered and priorities between them in an investment process. in this paper, the authors suggested improvement of the developed model presented for the first time in macura et al. (2011). the model proposed in this paper extends the macura model by including consideration of relevant external projects’ influences and defining their dynamic priorities. the model has a static and dynamic part. the static part focuses on project evaluation based on the criteria which are constant through time, and the dynamic part presents project evaluation related to relevant external, time-changeable projects. 3.1 applied approaches the analytical hierarchy process, developed by thomas saaty, is one of the most popular approaches to multi-criteria decision making (saaty, 2007a). it is used in analyses of decision-making to solve complex problems, the elements of which are objectives, criteria, sub-criteria and alternatives. ahp is a very useful approach because ijahp article: macura, selmic/railway project evaluation with dynamic prioritiesserbia case study international journal of the analytic hierarchy process 19 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.292 of its ability to identify and analyze inconsistency of decision makers in the process of decompression and evaluation of the elements of the hierarchy. ahp mitigates this problem to some extent by measuring the level of inconsistency and gathers information from decision-makers. this is a static approach, which uses the fundamental saaty scale to define priorities. by expanding the ahp approach, it is possible to cope with time dependent priorities. this new approach is called dynamic hierarchy process (dhp) (saaty 2007b). time-dependent decision-making, i.e. dynamic decision-making, is something that is often necessary. however, these alternatives may evolve over time, together with our preference for them, such as, stock market shares, the value of which constantly changes over time. dynamic decision making is reality, not a theoretical concept which can be ignored. it is necessary for technical design problems, in which effects of several project factors change over time and a compromise between them must be made, to allow the system to react differently and continuously during its work time. a typical appearance of the matrix in a dynamic form: (1) 1 0, ( ) ( ) ij ji ij a a t a t    , in a discrete situation, when ( )a t is consistent, we have ( ) ( ) / ( ) ij i j a t w t w t . all relevant equations are developed in saaty (2007b). 3.2 elements of the model for railway project evaluation the developed model has a network structure with four clusters (figure 1). each cluster has a certain number of elements which are explained in detail later in this section. figure 1. proposed model the sections being considered (rail corridor 10 in serbia) are presented in table 1. ijahp article: macura, selmic/railway project evaluation with dynamic prioritiesserbia case study international journal of the analytic hierarchy process 20 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.292 table1 considered alternatives section alternative length of section (km) number of tracks a1 šid-stara pazova 116 double-track a2 subotica-stara pazova 153 single-track a3 resnik-mladenovac-velika plana 70 single-track a4 velika plana-stalać 86 double-track a5 stalać-đunis 17 single-track a6 đunis-trupale 40 double-track a7 niš-preševo 173 single-track a8 niš-dimitrovgrad 104 single-track all the projects, the alternatives of the model, are already a part of the “strategy for the development of railway, road, water, air and intermodal transport in the republic of serbia from 2008 to 2015” 3 . the purpose of the model in this paper is to rank rail investment projects, considering financial and operating aspects (macura et al., 2011). table 2 shows the considered criteria. table2 considered criteria criteria measure type of criteria x1 cost-benefit ratio /  x2 criteria of speed restriction travel time lost [train hours/km]  x3 criteria of rail infrastructure capacity utilization the percent of rail line capacity utilization [%]  x4 criteria of inconsistency with agc & agtc [%]  x5 criteria of traffic volume [train/day]  cost-benefit ratio criterion, x1, represents the ratio between the benefit and the cost of a project. the benefit of a project is considered as a product of the average number of trains on the considered section, the expected increase of transport volume and average revenue per train. the assumption of increase in transport volume is based on the project “general master plan for transport in serbia”. the numerator of the cost/benefit ratio is actually the investment cost. criterion of speed restriction, x2, gives priority to sections where the most time is lost due to reduced speed. total loss of travel time for all trains on a part of line with 3 www.mi.gov.rs ijahp article: macura, selmic/railway project evaluation with dynamic prioritiesserbia case study international journal of the analytic hierarchy process 21 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.292 temporary speed restrictions, which are the result of a poor state of substructure and superstructure, is calculated as follows: 1 ( ) n i i lv vi i lvi pri s s t n v v     [train hours] (2) where lv t is the total lost travel time on a part of rail line with temporary speed restriction, which is the result of poor state of substructure and superstructure [train hours]; i s is the total length of the i -th temporary speed restriction on a particular rail section [km]; i is the number of temporary speed restrictions on a particular section; pri v is the allowed speed according to the designed characteristics of a rail section with the i -th temporary speed restriction [km/h]; lvi v is the speed of the i -th temporary speed restriction [km/h]; vi n is the total annual number of trains on a line section during the temporary speed restriction [trains]. x3, criterion of rail infrastructure capacity utilization, represents the total number of trains that can operate on a rail section during a specified time period. it depends on the number of tracks, technical parameters of a rail line (plan, profile and characteristics of superstructure), existing equipment of a rail line, the characteristics of rolling stock and the actual system of traffic control on the line. rail capacity utilization is the relationship of the relevant number of trains on a rail section and the line capacity. relevant number of trains can be defined as the average daily number of trains increased by the coefficient of irregularity. various approaches can be used for rail capacity calculation. the problem is that different approaches give significantly different values of capacity. in european terms the relevant method is recommended by a rail fiche no. 406 r of the international railway union (uic), which is used in macura et al. (2011). the rail section with less spare capacity will have a priority in this model. the general formula for calculation of rail line capacity utilization is: n n v [%] (3) where  is rail capacity utilization [%]; v n is the relevant number of trains on a rail section [trains/day]; n is rail line capacity [trains/day]. inconsistency with international agreements, x4, is relevant due to the fact that serbia is a signatory to two key european agreements (agc and agtc) and one regional agreement (seecp) related to the parameters of railway infrastructure harmonization of the european network. according to the agc and agtc agreements, the relevant values ijahp article: macura, selmic/railway project evaluation with dynamic prioritiesserbia case study international journal of the analytic hierarchy process 22 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.292 for the considered corridor 10 are given in table 3. the rail sections with the worst parameters will have priority in this model. the next criterion of traffic volume, x5, represents the train traffic intensity on a rail section. based on this criterion, a rail section with larger traffic volume will have priority. this criterion is calculated by the following equation: tpv nnn  [train/day] (4) where: v n is the total average daily number of trains on a particular section [trains/day]; p n is the average daily number of passenger trains [train/day]; t n is the average daily number of freight trains [train/day]. the average numbers of freight and passenger trains are given in table 3. based on the above explanation, all the defined criteria for the considered sections are calculated (macura et al., 2011). the values are given in table 3. by using these data, the pair-wise comparison matrices were developed. the alternative with which a higher effect can be achieved is better ranked. the matrix of a criteria comparison was made by transport expert’s recommendations. table 3 calculated values of the considered criteria for all alternatives alternative c1 [%] c2 [train hours/km] c3 [%] c4 [%] c5 [train/day] a1 0.47 93 17/17 0.014 25 a2 0.45 24 75 0.084 48 a3 0.29 333 61 0.192 52 a4 0.07 31 23/23 0.048 40 a5 0.08 14 45 0.100 44 a6 0.05 14 23/23 0.010 44 a7 1.66 9 34 0.285 23 a8 2.84 12 36 0.267 16 the relevant external projects (macura et al. 2012a, 2012b) can be international or domestic, infrastructure, ecological or social projects, etc. these projects are of great importance in transport project evaluation, since the transport networks are very dependent on their surroundings (including, not only the transport system in the considered country, but also in neighboring countries). it is possible to define the different relevance of these projects on the ranking of the projects. however, choosing the relevant external projects should be done by company management or by the experts. external projects can have different influences depending on their importance in the network. the influence of an external project can be observed in terms of flow of goods ijahp article: macura, selmic/railway project evaluation with dynamic prioritiesserbia case study international journal of the analytic hierarchy process 23 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.292 (increased intensity of flow). this impact can be direct or indirect, by directly or indirectly increasing/decreasing the intensity of flow on the considered rail line. suggested relevant external projects in the model are:  x – vidin-calafat bridge this project can take flow of goods and passengers from corridor 10, and make better service quality on corridor 4.  y – rehabilitation of corridor 4 – with this project, the competitive corridor, corridor 4, becomes stronger in comparison to corridor 10, with the aim of keeping the same freight transport volume on corridor 10, the service quality should be improved.  z – privatization of port “bar” for the considered rail network in serbia, one of the relevant external projects is the forthcoming privatization and improvement of the port of bar. realization of this project would increase the volume of freight transport from montenegro, through serbia, to hungary. the authors suggested using the dynamic approach to define the priorities of relevant external projects, with respect to the effects on the alternatives. the following is an explanation of this proposal. one relevant external project has already been realized, vidin-calafat bridge, project x (opened on june 14 th , 2013). the end of the second project, the rehabilitation of corridor 4, is planned for 2020. project z, privatization of port “bar”, will be realized in the near future, but its effects will become visible in the next few years. we assume that the relevant time horizon for consideration is from 2014 to 2020. in this period, the projects may change over time and all mentioned external projects will have been finished. 4. results and discussion the first part of the model, the static one, was developed using the super decisions software. table 4 presents the alternatives’ weights relative to the criteria. all the calculations were performed using the well-known equations from the ahp approach (bojkovic et al., 2011). table 4 alternatives’ weights relative to criteria cj c1 c2 c3 c4 c5 weights ai 0.256 0.445 0.182 0.041 0.076 a1 0.051 0.211 0.314 0.022 0.037 0.168 a2 0.237 0.113 0.132 0.044 0.097 0.144 a3 0.237 0.045 0.063 0.022 0.151 0.105 a4 0.051 0.075 0.021 0.083 0.220 0.070 a5 0.237 0.045 0.063 0.083 0.151 0.107 a6 0.026 0.028 0.314 0.286 0.037 0.091 a7 0.019 0.045 0.063 0.286 0.020 0.050 a8 0.142 0.437 0.030 0.173 0.287 0.265 sum 1 1 1 1 1 1 ijahp article: macura, selmic/railway project evaluation with dynamic prioritiesserbia case study international journal of the analytic hierarchy process 24 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.292 the authors assumed that relative priorities of relevant external projects were timedependent values. table 5 shows pair-wise comparison matrices for relevant external projects. table 5 pair-wise matrices for relevant external projects 2014 2017 2020 x y z x y z x y z x 1 3 5 1 1 4 1 1 1 y 0.333 1 4 1 1 3 1 1 1 z 0.2 0.25 1 0.25 0.333 1 1 1 1 based on the data from the previous table, the functions of priorities are defined in table 6. table 6 dynamic priorities of relevant external projects x y z x 1 2.857 0.643t 5.333 0.067t y 1 4.167 0.5t z 1 according to the data from the table 6: 12 2.857 0.643a t  ( 2 0.964r  ) (5) 13 5.333 0.067a t  ( 2 0.92r  ) (6) 23 4.167 0.5a t  ( 2 0.964r  ) (7) where: r2 – the r squared values, indicates how well data points fit in a statistical model t – time horizon: 2014 (t=1), 2017 (t=2) and 2020 (t=3) thereafter, using the following equations, the weights for relevant external projects can be obtained (saaty, 2007b). all the computations have been done in matlab. 1 1 3 3 max 13 12 23 12 23 13 ( / ) ( / ) 1a a a a a a    (8) 12 23 13 max ( 1)a a a     (9) 2 12 23 13 max max 23 13 12 max ( 1) ( 1) ( / ) 1 (1 )d a a a a a a           (10) xe w d   (11) max 23 13 12 ( 1) ( / ) ye a a a w d     (12) ijahp article: macura, selmic/railway project evaluation with dynamic prioritiesserbia case study international journal of the analytic hierarchy process 25 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.292 2 max 1 (1 ) ze w d     (13) where: xe t w , ye t w , ze t w weights of the relevant external projects, for , ,x y z , 1, 3t  the following graphs (figures 2a, b, c) show the changes of the relevant external projects weights, xe t w , ye t w and ze t w , over time horizons. this time dependence of the weights was the main reason for using the dynamic approach, i.e. dynamic priorities, in this model. the considered time horizon from 2014 to 2020 is on the x axes. the y axes show the weight of relevant external projects; there are three graphs for each relevant external project, respectively. figure 2a. figure 2b. ijahp article: macura, selmic/railway project evaluation with dynamic prioritiesserbia case study international journal of the analytic hierarchy process 26 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.292 figure 2c. weights xe t w , ye t w and ze t w for relevant external projects x, y and z the weights of the alternatives relative to criteria, j c w , are a constant value (table 4), but the weights of the alternatives relative to external projects, me t w , are time dependent. we assume that the criteria have the weight of 0.7, and the relevant external projects of 0.3 for the whole model. final weights of the alternatives in the model should be calculated by the following equation (table 7): 0.7 0.3 i j ma t c e t w w w  , for 1,8i  , 1, 3t  , 1, 5j  (14) table 7 alternatives weights relative to external projects and final results of the whole model through time horizons 2014 2017 2020 1me w 1ia w 2me w 2ia w 3me w 3ia w a1 0.164 0.168 0.169 0.168 0.200 0.178 a2 0.145 0.144 0.144 0.144 0.133 0.141 a3 0.145 0.117 0.144 0.117 0.133 0.113 a4 0.145 0.092 0.144 0.092 0.133 0.089 a5 0.145 0.118 0.144 0.118 0.133 0.115 a6 0.145 0.107 0.144 0.107 0.133 0.104 a7 0.055 0.052 0.056 0.052 0.067 0.055 a8 0.055 0.202 0.056 0.202 0.067 0.206 the final rank of the alternatives is presented in the figure 3. the evaluation of the process over time can be observed. the authors analyzed the relevant time period for the considered issue (2014-2020). two different time windows are defined due to the following: the first relevant change should be realized about 2017 the end of the external project “privatization of port bar”. ijahp article: macura, selmic/railway project evaluation with dynamic prioritiesserbia case study international journal of the analytic hierarchy process 27 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.292 the second relevant change will occur in 2020 the end of the external project “rehabilitation of corridor 4”. in this case study, the relative importance of the alternatives is not changeable over time. however, during time, some alternatives become more (a1, a7 and a8) or less (a2, a3, a4, a5 and a6) dominant. figure 3. final alternatives weights through time horizons sensitivity analysis the assumption included in equation 14, that the criteria weight is wc=0.7, and for the relevant external projects weight is wrep=0.3, can be modified in order to present a sensitivity analysis. the final results show that the relative project’s rank is the same through the whole time horizon for the first case (wc=0.7, wrep=0.3). the project’s rank is very similar in the second case (wc=0.5, wrep=0.5), the only difference is that the first and the second ranked alternatives, as well as the sixth and the seventh, change places. in the third case (wc=0.3, wrep=0.7), changes are the most evident, especially in the period after 2017. the following figure shows that the projects a8, a4 and a6 are time sensitive, so the project a8 changes even four positions, from the first, second, over the third to the seventh ranked project. the projects a4 and a6 in some cases are in the fifth, sixth or seventh ranked position. ijahp article: macura, selmic/railway project evaluation with dynamic prioritiesserbia case study international journal of the analytic hierarchy process 28 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.292 figure 4. final alternatives rank through time horizons with different criteria and relevant external projects weights 5. conclusions the evaluation of transport projects is a multi-criteria decision making process. this process is of great importance to a company’s success, because very often it defines the company’s share of the market. the model of decision making as a support system should have a flexible structure, suitable for potential modifications. decision makers sometimes need a model which takes into account changes of the system elements, or changes in the system surroundings. this model describes results with all possible modifications and gives suggestions for all of them. the developed model involves changes in the system surroundings, with consideration of the relevant external projects; in addition, it includes changes of the element’s priorities over time horizons, giving the final alternatives weights through time horizons. the analyzed case study is rail corridor 10 in serbia. the relative importance of the alternatives in this model is not significantly changeable over time. however, during time, some alternatives become more or less dominant. with respect to future studies, we recommend analyzing relevant stakeholders and their influence on a decision making process. very often, there are dynamic stakeholders’ preferences which are changeable over time. also, the relevant improvement of the model developed in this paper could consider the fact that some criteria are mutually interconnected, an aspect which can be elaborated by the dynamic network process. ijahp article: macura, selmic/railway project evaluation with dynamic prioritiesserbia case study international journal of the analytic hierarchy process 29 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.292 references ahern, a. and anandarajah, g. (2007). railway projects prioritization for investment: application of goal programming, transport policy, 14, 70-80. benítez, j., delgado-galván, x., izquierdo, j. and pérez-garcía, r. (2012). an approach to ahp decision in a dynamic context. decision support systems, 53(3), 499–506. berechman, j. and paaswell, r. (2005). evaluation, prioritization and selection of transportation investment projects in new york city. transportation, 32, 223249. boateng, p., chen, z., ogunlana, s. and ikediashi, d. (2012). a system dynamics approach to risks description in megaprojects development. organization, technology and management in construction, 4(3), 593-603. bojković, n., macura, d., pejčić-tarle, s. and bojović n. (2011). a comparative assessment of transport-sustainability in central and eastern european countries with a brief reference to the republic of serbia. international journal of sustainable transportation, 5(6), 319-344. caliskan, n. (2006). a decision support approach for the evaluation of transport investment alternatives. european journal of operational research, 175, 16961704. chang, y.h., wey, w.m. and tseng, h.y. (2009). using anp priorities with goal programming for revitalization strategies in historic transport: a case study of the alishan forest railwa'. expert systems with applications, 36(4), 86828690. fang, k. and yang, m. (2011). dynamic ahp net for simulation systems verification. systems engineering and electronics, 3, 707-711. feglar, t. and levy, j.k. (2005). dynamic analytic network process: improving decision support for information and communication technology, paper presented at the isahp, honolulu, hawaii. ferrari p. (2003). a method for choosing from among alternative transportation projects. european journal of operational research, 150, 194-203. fiala p. (2007). using analytic network process model in combinatorial auctions' paper presented at the isahp, chile. ijahp article: macura, selmic/railway project evaluation with dynamic prioritiesserbia case study international journal of the analytic hierarchy process 30 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.292 fiala, p. (2006). an anp/dnp analysis of economic elements in today’s world network economy. journal of systems science and systems engineering, 15(2), 131-140. gercek, h., karpak, b. and kilincaslan t. (2004). a multiple criteria approach for the evaluation of the rail transit networks in istanbul. transport journal, 31, 203228. gonzáleza, m., quesadab, g., and bahillc, t. (2003). improving product design using quality function deployment: the school furniture case in developing countries, quality engineering, 16(1), 45-56. jowitt, p. (2013). decisions, decisions. civil engineering and environmental systems, 30, 285-293. lee, s.m. (1998). analytic hierarchy process for transport project appraisal: an application to korea, phd thesis, the university of leeds institute for transport studies. liu, p., zhang, y., yan, x. and kuang, x. (2007). the method of interactive group decision-making for ahp based on experts' dynamic weights. mathematics in practice and theory,37(13), 85-90. longo, g., padoano, e., rosato, p. and strami s. (2009). considerations on the application of ahp/anp methodologies to decisions concerning a railway infrastructure, paper presented at the isahp, pittsburgh, pa: united states. macura, d., bojović, n., nuhodžić, r., šelmić, m. and bošković, b. (2012a). evaluation of transport projects using multi-criteria decision making method. paper presented at the international conference on traffic and transport engineering icttte, belgrade. macura, d., bošković, b., bojović, n. and milenković, m. (2011). a model for prioritization of rail infrastructure projects using anp. international journal of transport economics, 38(3), 265-289. macura, d., nuhodžić, r., bojović, n. and knežević, n. (2012b). one model for rail infrastructure projects selection. paper presented at the cetra, dubrovnik, 533-538. nasrabadi, a., hosseinpour, mh., and ebrahimnejad, s. (2013). strategy-aligned fuzzy approach for market segment evaluation and selection: a modular decision support system by dynamic network process (dnp). journal of industrial engineering international, 9(1), 1-17. ijahp article: macura, selmic/railway project evaluation with dynamic prioritiesserbia case study international journal of the analytic hierarchy process 31 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.292 piantanakulchai, m. (2005). analytic network process model for highway corridor plannng, paper presented at the isahp, honolulu, hawaii. saaty, t. (2007a). fundamentals of decision making and priority theory with the analytic hierarchy process. pittsburgh, pa: rws publications. saaty, t. (2007b). time dependent decision-making: dynamic priorities in the ahp/anp: generalizing from points to functions and from real to complex variables. mathematical and computer modelling, 46(7-8):860 891. shang, j. s., tjader, y. and ding y. (2004). a unified framework for multicriteria evaluation of transportation projects. ieee transactions on engineering management, 51(3), 300-313. shiftan, y., ben-akiva, m., de jong, g. and hakkert, s. (2002). evaluation of externalities in transport projects. european journal of transport and infrastructure research, 2(3), 285-304. tsamboulas, d. (2007). a tool for prioritizing multinational transport infrastructure investments. transport policy, 14, 11-26. tsamboulas, d., yiotis, g. and mikroudis, g. (2007). a method for multi-criteria analysis in transportation infrastructure investments. international journal of transport economics, 34(1), 113-131. tudela, a., akiki, n. and cisternas, r. (2006). comparing the output of cost benefit and multi-criteria analysis. an application to urban transport investments, transportation research part a, 40, 414-423. van wee, b. (2007). rail infrastructure: challenges for cost-benefit analysis and other ex ante evaluations. transportation planning and technology, 30(1), 31-48. wang, x., zhao, b. and zhang, q. (2008). mining method choice based on ahp and fuzzy mathematics. journal of central south university, science and technology, 39(5), 875−880. wey, w.m. and wu, k.y. (2007). using anp priorities with goal programming in resource allocation in transportation. mathematical and computer modeling, 46, 985-1000. yedla, s. and shrestha, r. (2003). multi-criteria approach for the selection of alternative options for environmentally sustainable transport system in delhi, transportation research part a, 37, 717-729. ijahp: bijan, keramati, salehi/comparison of user satisfaction of ecommerce websites by the analytic network process international journal of the analytic hierarchy process 223 vol. 5 issue 2 2013 issn 1936-6744 comparison of user satisfaction of ecommerce websites by the analytic network process yasaman bijan azad university of qazvin faculty of industrial & mechanical engineering, iran yasi.bijan@gmail.com abbas keramati university of tehran, tehran, iran associate professor of industrial engineering department faculty of engineering keramati@ut.ac.ir mona salehi lulea university of technology industrial marketing, e-commerce and logistics sweden monaasalehi@gmail.com abstract this study proposes a model for comparing the customer satisfaction indices of two or more ecommerce competitors in order to select the most preferred website in a specific context. the importance of customer satisfaction factors from the user’s point of view were calculated in the specific context of ecommerce. this study takes a new step towards integrating satisfaction literature by proposing a model for ranking the american customer satisfaction index (acsi) factors based on users' expectations about different online contexts. hence, the approach provides a new way to compare customer satisfaction among e-business competitors. the suggested model was shaped by merging the analytic network process (anp) approach with the acsi for ecommerce. the model tested two iranian e-recruitment websites through a survey designed and conducted via emails to those who had used both web sites. subsequently, the relative importance of the factors was determined, and finally e-recruitment websites were compared with each other. as a result, the most preferred website with respect to different acsi factors was chosen and the relative importance of each acsi factor considering the influence it had on the use of the erecruitment website and user satisfaction was identified. key word: analytic network process (anp), multi criteria decision making (mcdm), online satisfaction, customer satisfaction index, ebusiness mailto:yasi.bijan@gmail.com mailto:keramati@ut.ac.ir mailto:monaasalehi@gmail.com rob typewritten text http://dx.doi.org/10.13033/ijahp.v5i2.180 ijahp: bijan, keramati, salehi/comparison of user satisfaction of ecommerce websites by the analytic network process international journal of the analytic hierarchy process 224 vol. 5 issue 2 2013 issn 1936-6744 1. introduction the world wide web is one of the most important internet services and it has been largely responsible for the phenomenal growth of the internet in recent years. an increasingly popular and important web-based activity is ecommerce (graja & mcmanis, 2001). ecommerce through the internet has become an important transaction model in international trade (liu, zeng, xu, & koehl, 2008). when designing ecommerce systems it is important to determine the customer believes is required for satisfactory service. studies have shown that the financial value of customer relationship to the company has received growing attention among top executives. managers increasingly tend to see customer satisfaction as a valuable intangible asset and thus an important corporate target (luo, wieseke, & homburg, 2012). customer satisfaction can not only improve the financial situation of enterprises, but also bring about a unique competitive advantage for ecommerce websites and enhance customer loyalty (yang & ding, 2009). customer satisfaction is critical for establishing long-term client relationships, and has considerable impact on customer loyalty (schaupp & bélanger, 2005; chang & wang, 2011) which is an important factor for business growth (kumar, sharma, shah, & rajan, 2013). organizations which are more efficient in providing value for customers leading to higher customer satisfaction are more likely to survive in a competitive situation (kujala & ahola, 2005). online recruitment services are among the most popular applications on the internet. since employers are required to pay for the service, their perceptions of the level of service are typically of the most concern to the recruitment websites, and the recruitment-service quality level for the job seeker is typically ignored. therefore, it is critical to study the e-recruitment customer satisfaction from the job seekers’ perspective for any e-recruitment website to stay profitable in a competitive environment (keramati & salehi, 2013). in addition, in order to avoid ineffectiveness during deployment, it is important to clarify the impact factors of the customer satisfaction index (csi) and develop a suitable assessment method to evaluate its performance. in this research, an analytic network process (anp)–based assessment model was constructed to assess the effects of factors of csi in the context of ecommerce, and to find the preferred erecruitment website with respect to customer satisfaction factors in iran. in order to achieve this goal, we provide a model that considers factors and the customer satisfaction model. in addition, the model has the ability to compare two or more erecruitment websites scientifically through measuring and ranking of relative importance factors. the rest of the paper is organized as follows: section 2 reviews the literature on customer satisfaction models, section 3 provides the research framework, section 4 provides an in-depth description of our research methodology, and section 5 presents http://link.springer.com/search?facet-author=%22hubert+graja%22 http://link.springer.com/search?facet-author=%22jennifer+mcmanis%22 http://www.sciencedirect.com/science/article/pii/s0378475407003576 ijahp: bijan, keramati, salehi/comparison of user satisfaction of ecommerce websites by the analytic network process international journal of the analytic hierarchy process 225 vol. 5 issue 2 2013 issn 1936-6744 the results of our analysis. the last section (section 6) presents the discussion and conclusions of the study. 2. literature review since oliver put forward a cognitive model for characterizing antecedents and consequences of satisfaction in 1980, customer satisfaction and customer satisfaction index (csi) have been widely developed in both theory and applications (liu et al., 2008). while customer satisfaction has been defined in various ways, the high-level conceptualization that appears to have gained the widest acceptance states that satisfaction is a customer’s post-purchase evaluation of a product or service (bayraktar, tatoglu, turkyilmaz, delen, & zaim, 2012). within the existing literature on customer satisfaction research, various customer satisfaction models have been developed based on a cumulative view of satisfaction (bayraktar et al., 2012). in 1989, a swedish researcher (fornell, 1992) built the first model of csi – the swedish customer satisfaction barometer (scsb). the american customer satisfaction index (asci) was set up in 1994 (wood, siegel, feldman, love, rodrigues, malamud, lagana, & crafts, 2008). another well-known csi, the european customer satisfaction index (ecsi) was built by 11 countries of the european union in 1999 (liu et al., 2008). a review of the methods used by the various indices indicates that the acsi methodology meets or exceeds accepted standards for validity and accuracy of online user surveys .additionally, the acsi uses a standard set of core questions across platforms dna its results can be benchmarked to individual and aggregate results. furthermore, the acsi survey methodology allows for the inclusion of custom questions geared to each individual client, and thus permits a blend of both standardized and customized queries. customer satisfaction surveys like the acsi are tools for listening to “what users say” to determine user perceptions of a website’s usefulness and performance. perceptions are inherently subjective, but they do help web managers to understand another facet of user opinion (wood et al., 2008). the american customer satisfaction index (acsi) was first implemented in 1994 as an offline survey measuring customer satisfaction with businesses, and was adapted to the internet in 2002. more than two dozen other federal websites began using the survey in 2002 (wood et al., 2008). the core acsi methodology was developed by professor claes fornell, director of the national quality research center, university of michigan business school, and it is offered as an online service by foresee results, inc. of ann arbor, michigan. the acsi method uses multiple regression analysis to link questions on key elements that drive customer satisfaction with questions on overall customer satisfaction that are in turn linked to questions on future customer behavior. all standardized questions are framed by using a 10-point likert scale (wood et al., 2008). ijahp: bijan, keramati, salehi/comparison of user satisfaction of ecommerce websites by the analytic network process international journal of the analytic hierarchy process 226 vol. 5 issue 2 2013 issn 1936-6744 the foresee results acsi-based questionnaire for websites is composed of a core set of 14 to 20 questions (tullis & albert, 2008). the results for the website are divided into six quality-related elements that drive satisfaction: content, functionality, look and feel, navigation, search, and site performance. the results also assess the overall satisfaction and future behavior of the user (likelihood to return and likelihood to recommend) (wood et al., 2008).  content – accuracy, quality, and freshness of news, information, and content on the website.  functionality – usefulness, convenience, and variety of online features.  look and feel – visual appeal of the site and its consistency.  navigation – organization of the site and ease of navigation.  search – utility and effectiveness of the site’s search tool.  site performance – speed, consistency, and reliability of loading of pages on the website.  likelihood to return – reflects customers’ propensity to return to the site, which provides an opportunity for companies to further engage customers online.  likelihood to recommend – shows how likely online visitors are to engage in positive word-of mouth marketing by recommending the website to a friend, family member, or colleague. the intensity of the online competition has resulted in a simple mouse click to select a new provider (singh, 2002). disruptive internet technologies force ecommerce to focus on the factors that contribute to their success in the competitive environment. some research has been conducted comparing different models’ factors that lead to success in a competitive environment. for example, keramati and salehi (2013) compared website successes in the context of e-recruitment through an analytic network process (anp) approach. different researchers have used different descriptions of measures for customer satisfaction. however, no one has focused on comparing the customer satisfaction indices between two or more e-business competitors in order to select the most preferred website within a context, or on finding the importance of customer satisfaction factors from the users’ points of view in a specific context of ecommerce. therefore, this research tries to fill that gap in the literature. 3. research framework the anp, developed by thomas l. saaty, is a comprehensive decision-making technique that has the capability to include all the relevant criteria which have some bearing on a decision (jharkharia & shankar, 2007). anp is an extension of the analytic hierarchy process (ahp) that models a decision-making framework that assumes a unidirectional hierarchical relationship among the decision levels. ijahp: bijan, keramati, salehi/comparison of user satisfaction of ecommerce websites by the analytic network process international journal of the analytic hierarchy process 227 vol. 5 issue 2 2013 issn 1936-6744 the ahp/anp approaches offer several advantages over other mcdm methods. firstly, they are not proportionately complicated. secondly, they have the supplemental power of being able to mix quantitative and qualitative factors into a decision. thirdly, they can be fit together with other solution approaches such as optimization, and goal programming. fourthly, they may use a hierarchical structuring of the factors involved and finally, their judgment elicitations are completed using a decompositional approach, which has been shown in experimental studies to reduce decision-making errors (taslicali & ercan, 2006). the anp method has advantages over ahp for decision making. one of the advantages is that in the anp rank reversal problem is not an issue, thereby it is more accurate and useful than the ahp as a decision support instrument for intricate situations (taslicali & ercan, 2006). the greatest advantage of the anp model is that it can handle intangible factors based on individual or collective judgment of the situation (saaty & vargas, 2006). therefore, based on these advantages, the anp was used for modeling and comparing in this study. technically, the model consists of clusters and elements where dominance or relative importance of influence is the central concept. judgments are provided based on saaty’s 1–9 scales (saaty, 2005). a score of 1 represents indifference between the two components, and a score of 9 represents overwhelming dominance of the component under consideration over the compared component (keramati & salehi, 2013). the strength of the anp lies in its use of ratio scales to capture all kinds of interactions, formulate accurate predictions, and make better decisions (saaty, 2003). the relationship between different categories of the model complies with the variance or causal model as well as the process model. according to the arrows between the different categories, one category precedes another category because of the process model. further, any increase or decrease in a category would lead to an increase or decrease in the other interrelated category, for instance, content to overall satisfaction according to the process model. however according to the causal or the variance model, a positive (negative) experience in content will lead to increased (decreased) overall satisfaction. certainty to recommend and likelihood to return will occur because of certain overall satisfaction. the decision maker will use a series of pairwise comparisons and compare two components at a time with respect to the source or parent criterion to elicit preferences of various components and attributes. all nodes that are to be compared pairwise are always in the same cluster and are compared with respect to their parent (source) element – the node from which they are connected. this results in local priorities of the nodes with respect to the source node. ijahp: bijan, keramati, salehi/comparison of user satisfaction of ecommerce websites by the analytic network process international journal of the analytic hierarchy process 228 vol. 5 issue 2 2013 issn 1936-6744 in acsi, model clusters have influence on each other. the anp is an approach used to determine the relative importance of variables in the acsi model and to compare the customer satisfaction rate in a similar context of ecommerce. and then to rank alternative websites. acsi is extended through the anp in this research article. the proposed framework is shown in figure 1. ijahp: bijan, keramati, salehi/comparison of user satisfaction of ecommerce websites by the analytic network process international journal of the analytic hierarchy process 229 vol. 5 issue 2 2013 issn 1936-6744 figure 1. acsi is extended through anp in this research article ijahp: bijan, keramati, salehi/comparison of user satisfaction of ecommerce websites by the analytic network process international journal of the analytic hierarchy process 230 vol. 5 issue 2 2013 issn 1936-6744 the parent element directs the arrows in the anp. different categories in the model are compared pairwise with respect to their influence on the parent element. in the acsi model, "customer satisfaction" is influenced by "content,” "functionality,” "look and feel,” "navigation,” "search,” and "site performance". therefore, in the anp framework, customer satisfaction is the parent element. in anp, "content,” "functionality,” "look and feel,” "navigation,” "search,” and "site performance" are compared pairwise with respect to their influence on the parent element. likewise, other categories in the model are compared pairwise with respect to their parent elements. 4. research methodology 4.1. data collection and sampling data for this study were collected from the common online academic union users of the only two iranian e-recruitment websites, www.irantalent.com and www.agahjobs.com. the sampling group consisted of 100,000 university students who had graduated. the group was chosen randomly from http://www.irexpert.ir which consists of graduated iranian university experts. simple random sampling was selected as the most appropriate sampling technique based on probability. the minimum sample size required for the population of 100,000 with a confidence level of 95% and confidence interval of 5 was calculated as 383 (keramati & salehi, 2013). based on this calculation, 383 users were selected randomly and the survey questionnaire (appendix 1) was sent to them by email. at the beginning of the survey, a question asked the users whether or not they had used both websites. then, only those users who gave a positive response to this question were allowed to fill in the rest of the survey. 46 of eligible users returned usable surveys, which were answered completely and consistently. 4.2. measures as mentioned before, saaty suggested a scale of 1 to 9 when comparing two components. a score of 1 indicates that the two options have equal importance or indifference, where a score of 9 indicates the overwhelming dominance of the component under consideration over the compared component in a pairwise comparison matrix (saaty, 1980). 4.3. group decision making in order for more than one person to make a decision, group decision-making techniques are used to integrate the judgments of decision-makers (sadeghi, rashidzadeh & soukhakian, 2012). decision-makers were users of the websites who filled in the survey completely and consistently. in order to achieve consistent judgments, the individual judgments were gathered and individually entered into superdecisions in order to find the consistent responses. this is how we gathered 46 judgments. the next step involved integrating the individual decisions and making a group decision based on complete and consistent judgments. as saaty and vargas http://www.irexpert.ir/ ijahp: bijan, keramati, salehi/comparison of user satisfaction of ecommerce websites by the analytic network process international journal of the analytic hierarchy process 231 vol. 5 issue 2 2013 issn 1936-6744 (2006) state, the reciprocal property plays an important role in combining the judgments of several individuals to obtain a judgment for a group. judgments must be combined so that the reciprocal of the synthesized judgments must be equal to the syntheses of the reciprocals of these judgments. it has been proved that the geometric mean method is the unique way to do that (satty & vargas, 2006). equation 1 in equation 1, wi is the weight given to each decision-makers’ opinion. in this instance it is 1, which means all the decision-makers opinions were given the same weight of importance. m is the number of decision makers, which is 46. aij indicates the elements of the decision-makers’ comparison matrices and indicates the elements of the final comparison matrix which were entered in super decisions software. 4.4. data analysis the anp calculations were implemented through the super decisions software created by saaty (2004) to alleviate the mathematical burden. super decisions provides results including weights of variables with respect to variables that pairwise compared to priorities of alternatives and sensitivity analysis. the results of the questionnaire were analyzed so that the weights of nodes in each cluster with respect to the parent node with which they are compared to were calculated. furthermore, alternatives were ranked with respect to each customer satisfaction variable in the acsi model. according to saaty (2001), the anp is comprised of four main steps: step 1: model construction and problem structuring: our objective was to determine the influence of each element of the model of customer satisfaction of an erecruitment website, and determine whether irantalent.com or agahjobs.com is the best e-recruitment site. the clusters of the constructed model are based on acsi clusters and an alternative cluster and the arrows indicate the direction of impact. the model is shown in figure 1. step 2: formulating the interdependencies and performing paired comparisons between the clusters/elements: this step leads to the creation of a network containing all the decision elements and their inner (within the same cluster) and outer relationships (among elements of different clusters) (bayazit, 2006).  cluster comparisons. the clusters themselves must be compared to establish their relative importance, and then the priorities used to weight the ijahp: bijan, keramati, salehi/comparison of user satisfaction of ecommerce websites by the analytic network process international journal of the analytic hierarchy process 232 vol. 5 issue 2 2013 issn 1936-6744 supermatrix to make it column stochastic (saaty & sodenkamp, 2008). cluster matrix is shown in table 1. table 1 cluster weights matrix  comparisons of elements and comparisons for alternatives: according to saaty and sodenkamp (2008), the unweighted supermatrix is constructed from the priorities derived from the different pairwise comparisons. the nodes, grouped by the clusters that they belong to, are the labels of the rows and columns of the supermatrix. the column of priorities for a node at the top of the supermatrix includes the priorities of the nodes on the left side of the matrix that have been pairwise compared as to their influence with respect to customer satisfaction on that node. the sum of these priorities is equal to one. a reciprocal value is assigned to the inverse comparison (that is, ai j = 1/ai j) where ai j denotes the importance of the ith element compared to the jth element. the unweighted supermatrix of the network of figure 1 is shown in table 2. recommend likelihood to return satisfaction site performance search navigation look & feel functionality content alternatives 0.176 0.196 0.044 1 1 1 1 1 1 0 alternatives 0 0 0.241 0 0 0 0 0 0 0.194 content 0 0 0.114 0 0 0 0 0 0 0.105 functionality 0 0 0.037 0 0 0 0 0 0 0.036 look & feel 0 0 0.104 0 0 0 0 0 0 0.073 navigation 0 0 0.123 0 0 0 0 0 0 0.090 search 0 0 0.337 0 0 0 0 0 0 0.207 site performance 0.824 0.804 0 0 0 0 0 0 0 0.243 satisfaction 0 0 0 0 0 0 0 0 0 0.025 likelihood to return 0 0 0 0 0 0 0 0 0 0.027 recommend ijahp: bijan, keramati, salehi/comparison of user satisfaction of ecommerce websites by the analytic network process international journal of the analytic hierarchy process 233 vol. 5 issue 2 2013 issn 1936-6744 table 2 unweighted supermatrix alternatives content functionality look & feel navigation search site performance satisfaction likelihood to return recommend a g a h jo b s ir a n ta le n t in fo rm a ti o n a c c u ra c y in fo rm a ti o n q u a li ty u se fu ln e ss o f th e in fo rm a ti o n c o n v e n ie n c e o f th e i n fo rm a ti o n e a se o f re a d in g th e s it e c la ri ty o f si te o rg a n iz a ti o n a b il it y t o f in d w a n te d in fo rm a ti o n e a se o f n a v ig a ti o n u se fu ln e ss o f se a rc h r e su lt s p ro v id e s c o m p re h e n si v e se a rc h r e su lt s o rg a n iz a ti o n o f se a rc h r e su lt s s e a rc h f e a tu re s h e lp y o u n a rr o w th e r e su lt s r e li a b il it y o f si te p e rf o rm a n c e o v e ra ll sa ti sf a c ti o n w it h th is s it e m e e t y o u r e x p e c ta ti o n s c o m p a re t o y o u r id e a o f a n i d e a l w e b si te re tu rn t o t h e s it e r e c o m m e n d t h e si te t o s o m e o n e e ls e alternatives agahjobs 0 0 0.294 0.240 0.374 0.293 0.449 0.271 0.267 0.289 0.374 0.290 0.237 0.397 0.280 0.311 0.404 0.372 0.387 0.300 irantalent 0 0 0.706 0.760 0.626 0.707 0.551 0.729 0.733 0.711 0.626 0.710 0.763 0.603 0.720 0.689 0.596 0.628 0.613 0.700 content information accuracy 0.337 0.407 0 0 0 0 0 0 0 0 0 0 0 0 0 0.387 0.257 0.321 0 0 information quality 0.663 0.593 0 0 0 0 0 0 0 0 0 0 0 0 0 0.613 0.743 0.679 0 0 functionality usefulness of the information 0.453 0.542 0 0 0 0 0 0 0 0 0 0 0 0 0 0.684 0.556 0.562 0 0 convenience of the information 0.547 0.458 0 0 0 0 0 0 0 0 0 0 0 0 0 0.316 0.444 0.438 0 0 look & feel ease of reading the site 0.528 0.332 0 0 0 0 0 0 0 0 0 0 0 0 0 0.354 0.378 0.399 0 0 clarity of site organization 0.472 0.668 0 0 0 0 0 0 0 0 0 0 0 0 0 0.646 0.622 0.601 0 0 navigation ability to find wanted information 0.431 0.598 0 0 0 0 0 0 0 0 0 0 0 0 0 0.639 0.542 0.612 0 0 ease of navigation 0.569 0.402 0 0 0 0 0 0 0 0 0 0 0 0 0 0.361 0.458 0.388 0 0 search usefulness of search results 0.190 0.156 0 0 0 0 0 0 0 0 0 0 0 0 0 0.440 0.478 0.260 0 0 provides comprehensive search results 0.263 0.410 0 0 0 0 0 0 0 0 0 0 0 0 0 0.162 0.273 0.242 0 0 organization of search results 0.222 0.205 0 0 0 0 0 0 0 0 0 0 0 0 0 0.171 0.110 0.262 0 0 search features help you narrow the results 0.325 0.229 0 0 0 0 0 0 0 0 0 0 0 0 0 0.227 0.139 0.236 0 0 site performance reliability of site performance 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 satisfaction overall satisfaction with this site 0.366 0.155 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.224 0.379 meet your expectations 0.367 0.391 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.227 0.478 compare to your idea of an ideal website 0.267 0.454 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.549 0.143 likelihood to return return to the site 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 recommend recommend the site to someone else 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 ijahp: bijan, keramati, salehi/comparison of user satisfaction of ecommerce websites by the analytic network process international journal of the analytic hierarchy process 234 vol. 5 issue 2 2013 issn 1936-6744 step 3: constructing the supermatrix: the weights derived from step 2 are used to populate the columns of the supermatrix. either each column of a supermatrix is a normalized eigenvector with possibly some zero entries or all of its block entries are zero. the unweighted supermatrix, which is illustrated in table 2, is then multiplied by the priority weights from the clusters (which were determined in the first bullet point in step 2), yielding the weighted supermatrix (second panel of table 3). this is done because the resulting matrix must be column stochastic, that is, its columns must add to one, for a limit that is not zero in order to exist (saaty & vargas, 2006). ijahp: bijan, keramati, salehi/comparison of user satisfaction of ecommerce websites by the analytic network process international journal of the analytic hierarchy process 235 vol. 5 issue 2 2013 issn 1936-6744 table 3 weighted supermatrix alternatives content functionality look & feel navigation search site performance satisfaction likelihood to return recommend a g a h jo b s ir a n ta le n t in fo rm a ti o n a c c u ra c y in fo rm a ti o n q u a li ty u se fu ln e ss o f th e in fo rm a ti o n c o n v e n ie n c e o f th e i n fo rm a ti o n e a se o f re a d in g th e s it e c la ri ty o f si te o rg a n iz a ti o n a b il it y t o f in d w a n te d in fo rm a ti o n e a se o f n a v ig a ti o n u se fu ln e ss o f se a rc h r e su lt s p ro v id e s c o m p re h e n si v e se a rc h r e su lt s o rg a n iz a ti o n o f se a rc h r e su lt s s e a rc h f e a tu re s h e lp y o u n a rr o w th e r e su lt s r e li a b il it y o f si te p e rf o rm a n c e o v e ra ll sa ti sf a c ti o n w it h th is s it e m e e t y o u r e x p e c ta ti o n s c o m p a re t o y o u r id e a o f a n i d e a l w e b si te re tu rn t o t h e s it e r e c o m m e n d t h e si te t o s o m e o n e e ls e alternatives agahjobs 0 0 0.294 0.240 0.374 0.293 0.449 0.271 0.267 0.289 0.374 0.290 0.237 0.397 0.280 0.014 0.018 0.016 0.076 0.053 irantalent 0 0 0.706 0.760 0.626 0.707 0.551 0.729 0.733 0.711 0.626 0.710 0.763 0.603 0.720 0.030 0.026 0.028 0.120 0.123 content information accuracy 0.065 0.079 0 0 0 0 0 0 0 0 0 0 0 0 0 0.093 0.062 0.077 0 0 information quality 0.129 0.115 0 0 0 0 0 0 0 0 0 0 0 0 0 0.148 0.179 0.164 0 0 functionality usefulness of the information 0.048 0.057 0 0 0 0 0 0 0 0 0 0 0 0 0 0.078 0.063 0.064 0 0 convenience of the information 0.057 0.048 0 0 0 0 0 0 0 0 0 0 0 0 0 0.036 0.051 0.050 0 0 look & feel ease of reading the site 0.019 0.012 0 0 0 0 0 0 0 0 0 0 0 0 0 0.013 0.014 0.015 0 0 clarity of site organization 0.017 0.024 0 0 0 0 0 0 0 0 0 0 0 0 0 0.024 0.023 0.022 0 0 navigation ability to find wanted information 0.031 0.044 0 0 0 0 0 0 0 0 0 0 0 0 0 0.066 0.056 0.064 0 0 ease of navigation 0.042 0.029 0 0 0 0 0 0 0 0 0 0 0 0 0 0.038 0.048 0.040 0 0 search usefulness of search results 0.017 0.014 0 0 0 0 0 0 0 0 0 0 0 0 0 0.054 0.059 0.032 0 0 provides comprehensive search results 0.024 0.037 0 0 0 0 0 0 0 0 0 0 0 0 0 0.020 0.034 0.030 0 0 organization of search results 0.020 0.018 0 0 0 0 0 0 0 0 0 0 0 0 0 0.021 0.013 0.032 0 0 search features help you narrow the results 0.029 0.021 0 0 0 0 0 0 0 0 0 0 0 0 0 0.028 0.017 0.029 0 0 site performance reliability of site performance 0.207 0.207 0 0 0 0 0 0 0 0 0 0 0 0 0 0.337 0.337 0.337 0 0 satisfaction overall satisfaction with this site 0.089 0.038 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.180 0.312 meet your expectations 0.089 0.095 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.183 0.394 compare to your idea of an ideal website 0.065 0.110 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.441 0.118 likelihood to return return to the site 0.025 0.025 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 recommend recommend the site to someone else 0.027 0.027 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 ijahp: bijan, keramati, salehi/comparison of user satisfaction of ecommerce websites by the analytic network process international journal of the analytic hierarchy process 236 vol. 5 issue 2 2013 issn 1936-6744 finally, by increasing a matrix to powers the long-term relative influences of the elements on one another can be obtained. to achieve convergence of the importance weights, the weighted supermatrix is increased to the power of 2k+1, where k is an arbitrarily large number. this new matrix is termed the limit supermatrix. the limit supermatrix possesses the same form as the weighted supermatrix, but all the columns of the limit supermatrix are identical. normalizing each block of the supermatrix can obtain the final priorities of all the elements (table 4). now that the synthesized results from the limit supermatrix have been determined, the limit supermatrix shows the relative e-customer satisfaction of the alternatives irantalent and agahjobs as 0.306 and 0.126, respectively. when normalized they are 0.708 and 0.292, respectively (table 5). ijahp: bijan, keramati, salehi/comparison of user satisfaction of ecommerce websites by the analytic network process international journal of the analytic hierarchy process 237 vol. 5 issue 2 2013 issn 1936-6744 table 4 limit supermatrix alternatives content functionality look & feel navigation search site performance satisfaction likelihood to return recommend a g a h jo b s ir a n ta le n t in fo rm a ti o n a c c u ra c y in fo rm a ti o n q u a li ty u se fu ln e ss o f th e in fo rm a ti o n c o n v e n ie n c e o f th e i n fo rm a ti o n e a se o f re a d in g th e s it e c la ri ty o f si te o rg a n iz a ti o n a b il it y t o f in d w a n te d in fo rm a ti o n e a se o f n a v ig a ti o n u se fu ln e ss o f se a rc h r e su lt s p ro v id e s c o m p re h e n si v e se a rc h r e su lt s o rg a n iz a ti o n o f se a rc h r e su lt s s e a rc h f e a tu re s h e lp y o u n a rr o w th e r e su lt s r e li a b il it y o f si te p e rf o rm a n c e o v e ra ll sa ti sf a c ti o n w it h th is s it e m e e t y o u r e x p e c ta ti o n s c o m p a re t o y o u r id e a o f a n i d e a l w e b si te re tu rn t o t h e s it e r e c o m m e n d t h e si te t o s o m e o n e e ls e alternatives agahjobs 0.126 0.126 0.126 0.126 0.126 0.126 0.126 0.126 0.126 0.126 0.126 0.126 0.126 0.126 0.126 0.126 0.126 0.126 0.126 0.126 irantalent 0.306 0.306 0.306 0.306 0.306 0.306 0.306 0.306 0.306 0.306 0.306 0.306 0.306 0.306 0.306 0.306 0.306 0.306 0.306 0.306 content information accuracy 0.041 0.042 0.042 0.042 0.042 0.042 0.042 0.042 0.042 0.042 0.042 0.042 0.042 0.042 0.042 0.042 0.042 0.042 0.042 0.042 information quality 0.072 0.072 0.072 0.072 0.072 0.072 0.072 0.072 0.072 0.072 0.072 0.072 0.072 0.072 0.072 0.072 0.072 0.072 0.072 0.072 functionality usefulness of the information 0.032 0.032 0.032 0.032 0.032 0.032 0.032 0.032 0.032 0.032 0.032 0.032 0.032 0.032 0.032 0.032 0.032 0.032 0.032 0.032 convenience of the information 0.028 0.028 0.028 0.028 0.028 0.028 0.028 0.028 0.028 0.028 0.028 0.028 0.028 0.028 0.028 0.028 0.028 0.028 0.028 0.028 look & feel ease of reading the site 0.008 0.008 0.008 0.008 0.008 0.008 0.008 0.008 0.008 0.008 0.008 0.008 0.008 0.008 0.008 0.008 0.008 0.008 0.008 0.008 clarity of site organization 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 navigation ability to find wanted information 0.025 0.025 0.025 0.025 0.025 0.025 0.025 0.025 0.025 0.025 0.025 0.025 0.025 0.025 0.025 0.025 0.025 0.025 0.025 0.025 ease of navigation 0.019 0.019 0.019 0.019 0.019 0.019 0.019 0.019 0.019 0.019 0.019 0.019 0.019 0.019 0.019 0.019 0.019 0.019 0.019 0.019 search usefulness of search results 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 providing comprehensive search results 0.018 0.018 0.018 0.018 0.018 0.018 0.018 0.018 0.018 0.018 0.018 0.018 0.018 0.018 0.018 0.018 0.018 0.018 0.018 0.018 organization of search results 0.011 0.011 0.011 0.011 0.011 0.011 0.011 0.011 0.011 0.011 0.011 0.011 0.011 0.011 0.011 0.011 0.011 0.011 0.011 0.011 search features help you narrow the results 0.013 0.013 0.013 0.013 0.013 0.013 0.013 0.013 0.013 0.013 0.013 0.013 0.013 0.013 0.013 0.013 0.013 0.013 0.013 0.013 site performance reliability of site performance 0.131 0.131 0.131 0.131 0.131 0.131 0.131 0.131 0.131 0.131 0.131 0.131 0.131 0.131 0.131 0.131 0.131 0.131 0.131 0.131 satisfaction overall satisfaction with the site 0.028 0.028 0.028 0.028 0.028 0.028 0.028 0.028 0.028 0.028 0.028 0.028 0.028 0.028 0.028 0.028 0.028 0.028 0.028 0.028 meeting your expectations 0.047 0.047 0.047 0.047 0.047 0.047 0.047 0.047 0.047 0.047 0.047 0.047 0.047 0.047 0.047 0.047 0.047 0.047 0.047 0.047 comparing the site with your ideal website 0.048 0.048 0.048 0.048 0.048 0.048 0.048 0.048 0.048 0.048 0.048 0.048 0.048 0.048 0.048 0.048 0.048 0.048 0.048 0.048 likelihood to return return to the site 0.011 0.011 0.011 0.011 0.011 0.011 0.011 0.011 0.011 0.011 0.011 0.011 0.011 0.011 0.011 0.011 0.011 0.011 0.011 0.011 recommend recommend the site to someone else 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 ijahp: bijan, keramati, salehi/comparison of user satisfaction of ecommerce websites by the analytic network process international journal of the analytic hierarchy process 238 vol. 5 issue 2 2013 issn 1936-6744 table 5 the synthesized results of the alternatives alternatives raw normals ideals agahjobs 0.126 0.292 0.413 irantalent 0.306 0.708 1 4.5. sensitivity analysis a sensitivity analysis is conducted in order to determine the stability of the preference ranking among the alternative websites by changing the priority weights of the criteria. if the ranking does not change, the results are said to be robust. in this study, sensitivity is performed by varying the priority of the reliability of site performance criterion, by moving the vertical line and determining the corresponding alternatives priorities. figures 2 and 3 show a graphical representation of sensitivity for the reliability of site performance criterion. in figure 2, priority of the reliability of site performance is 0.72, and the priorities of the alternatives are the original overall synthesized priorities as 0.292 and 0.708 respectively for agahjobs and irantalent which are shown in the table 5. in figure 3, priority of the reliability of site performance is shifted to 0.01, and the ranking between the alternatives has changed. when priority of the reliability of site performance is less than about 0.03, agahjobs.com will receive a higher rank than irantalent.com. when a sensitivity analysis was performed on the other criteria the alternatives’ ranking were not affected. therefore, this analysis shows that irantalent.com is the better e-recruitment website based on customer satisfaction than agahjobs.com. ijahp: bijan, keramati, salehi/comparison of user satisfaction of ecommerce websites by the analytic network process international journal of the analytic hierarchy process 239 vol. 5 issue 2 2013 issn 1936-6744 figure 2. sensitivity graphs for the reliability of site performance when its priority is 0.72 ijahp: bijan, keramati, salehi/comparison of user satisfaction of ecommerce websites by the analytic network process international journal of the analytic hierarchy process 240 vol. 5 issue 2 2013 issn 1936-6744 figure 3. sensitivity graphs for the reliability of site performance when its priority is 0.01 4.6. reliability and validity analysis the reliability analysis of a questionnaire determines its ability to yield the same results on different occasions, and validity refers to the measurement of what the questionnaire is supposed to measure (albadvi, keramati, & razmi, 2007). several precautions were taken in order to assess the reliability of the results. after establishing a questionnaire according to saaty (2004), the questionnaire was reassessed by independent individuals. firstly, we invited a recruitment industry expert to format questions to ensure that they were properly phrased. in addition, anp experts reviewed the questionnaire to make certain that pairwise comparisons were properly established. after gathering the completed questionnaires, the inconsistency ratio for each judgment was checked in order to find any possible errors and eliminate the inconsistent judgments for each group of pairwise comparison for each respondent. the inconsistency measure is useful in identifying possible errors in judgments as well as actual inconsistencies in the judgments themselves (saaty, 2003). for example, if you were to say that a is more important than b, and b is more important than c, and then say that c is more important than a, you are not being consistent. in general, the inconsistency ratio should be less than 0.1 to be considered reasonably consistent (saaty, 2003). 5. results ijahp: bijan, keramati, salehi/comparison of user satisfaction of ecommerce websites by the analytic network process international journal of the analytic hierarchy process 241 vol. 5 issue 2 2013 issn 1936-6744 according to the limit supermatrix table, the importance and influence of each customer satisfaction factor is achieved with respect to their influence on e-recruitment customer satisfaction. with respect to content category, information quality (0.072) in comparison to information accuracy (0.042) has considerable influence on e-recruitment customer satisfaction. usefulness of information (0.032) has more influence on an e-recruitment website than the relative convenience of the information factor (0.028) in the functionality factors. clarity of site organization (0.012) was found to be more beneficial to the customer satisfaction factor in the look and feel category in comparison to ease of reading of the site (0.08). in the navigation category, the ability to find wanted information (0.025) was ranked higher than ease of navigation (0.019) by online users. in the search category, in comparison to other search measures, the factor providing comprehensive search results (0.018) received the most attention from online users, while the factor relative organization of search results (0.011) received the lowest ranking in the limit supermatrix. the highest preference factor in the satisfaction category was comparing the site with your ideal website (0.048), and the factor meeting your expectations with little difference (0.047) in terms of users’ attention. however, the overall satisfaction with the site (0.028) received less attention. synthesized results (table 5) support that the irantalent website was chosen as the priority e-recruitment website with respect to customer satisfaction, and with the highest weighted score of 0.708 when compared to agahjobs website’s score of 0.292. 6. discussion and conclusions this paper has studied the framework for investigating factors that influence customer satisfaction index in ecommerce. for this purpose, the study focused on the relative clusters priorities in the customer satisfaction model and also the priorities of factors in each cluster based on users' opinion and compared and ranked the websites satisfaction based on these priorities. this paper puts forward an approach to find the priorities, and compare the customer satisfaction of e-recruitment websites (agahjobs.com and irantalent.com) based on the priorities. in addition, customer satisfaction is the leading factor that determines online customers’ loyalty. thus, understanding the factors that influence online customer satisfaction is of great importance to ebusinesses (hsu, 2008). since the findings suggest that customer satisfaction is crucial for a better financial performance, providing exceptional customer service by exceeding customers’ expectations is likely to offer opportunities for growth (chi & gursoy, 2009). the anp model helped discover the influence of different factors on e-recruitment customer satisfaction. the mathematical procedure of the anp methodology helps the analyst or the decision maker to include all the factors and elements that users think might influence the alternatives. therefore, users can gradually choose the less important factors, but focus first on the most important ones. the model provides an e-business analyst with a very flexible analysis tool that is easily understood. according to the results, it is recommended that both the websites pay more attention to site performance, a factor that has the most influence on e-recruitment customer satisfaction. some service features increase the overall satisfaction linearly when performance improves (fuller & matzler, 2008). such features have ijahp: bijan, keramati, salehi/comparison of user satisfaction of ecommerce websites by the analytic network process international journal of the analytic hierarchy process 242 vol. 5 issue 2 2013 issn 1936-6744 been labeled as performance factors. hence, it is important to identify which product or service attributes lead to an increase in satisfaction or customer delight when their performances are improved, and which attributes can lead to dissatisfaction through their absence as the customer expects their presence (fuller & matzler, 2008). site performance is all about what the user actually does in interacting with the site. it includes measuring the degree to which users can successfully accomplish a task or a set of tasks. many measures related to the performance of these tasks are also important, including the time it takes to perform each of them, the amount of effort used to perform each (such as number of mouse clicks or amount of cognitive effort), the number of errors committed, and the amount of time it takes to become proficient in performing these tasks (learnability) (hsu, 2012). results suggest that agahjobs.com improve all acsi factors, especially site performance. it scored much weaker than irantalent.com in that aspect, and this factor has a high influence on e-recruitment customer satisfaction. in addition, it is important that agahjobs.com focus on the speed, consistency, and reliability of loading pages on the website to increase users' satisfaction. furthermore, irantalent.com should pay more attention to the factor meeting users' expectations in the satisfaction cluster as there is the least difference between their site and agahjobs.com in this category. the proposed model provides a method with an objective, and it is effective in selecting an optimal independent e-recruitment website based on the acsi, making it highly applicable for academic and commercial purposes. an important limitation of this study lies in the study’s sample size. as mentioned in section 4.1, we needed a sample size of 383 for 95% confidence. however, 46 of eligible users returned usable surveys, which were answered completely and consistently. this number fell well below what was needed. although this analysis still provides quality insight, we cannot claim these 46 respondents actually represent the opinions of the 100,000 graduates. in the study, questionnaires which were found inconsistent in judgments were eliminated. in the future research, the respondents with the most inconsistent answers will be contacted and asked for new judgments. this approach will lead to an improvement in the inconsistency in judgments. ijahp: bijan, keramati, salehi/comparison of user satisfaction of ecommerce websites by the analytic network process international journal of the analytic hierarchy process 243 vol. 5 issue 2 2013 issn 1936-6744 references albadvi, a., keramati, a., & razmi, j. (2007). assessing the impact of information technology on firm performance considering the role of intervening variables: organizational infrastructures and business processes reengineering. international journal of production research, 45(12), 2697–2734. bayazit, o. (2006). use of analytic network process in vendor selection decisions. benchmarking: an international journal, 13(5), 566-579. bayraktar, e., tatoglu, e., turkyilmaz, a., delen, d., & zaim, s. (2012). measuring the efficiency of customer satisfaction and loyalty for mobile phone brands with dea. expert systems with applications, 39(1), 99–106. chang, h. h., & wang, h. w. (2011). the moderating effect of customer perceived value on online shopping behavior. online information review, 35(3), 333 – 359. chi, c. g., & gursoy, d. (2009). employee satisfaction, customer satisfaction, and financial performance: an empirical examination. international journal of hospitality management, 28, 245–253. fornell, c. (1992). a national customer satisfaction barometer: the swedish experience. journal of marketing, 56(1), 6–21. fuller, j., & matzler, k. (2008). customer delight and market segmentation: an application of the threefactor theory of customer satisfaction on life style groups. tourism management, 29, 116-126. graja, h., & mcmanis, j., (2001). modelling user interaction with e-commerce servers. springer berlin heidelberg, 2093, 803-810. hsu, p. f. (2012). selection model based on anp and gra for independent media agencies. quality & quantity, 46(1), 1-17. hsu, s. h. (2008). developing an index for online customer satisfaction: adaptation of american customer satisfaction index. expert systems with applications, 34(4), 3033–3042. jharkharia, s., & shankar, r. (2007). selection of logistics service provider: an analytic network process (anp) approach. omega, 35(3), 274-289. keramati, a., & salehi, m. (2013). website success comparison in the context of e-recruitment: an analytic network process (anp) approach. applied soft computing, 13(1), 173–180. kujala, j., & ahola, t. (2005). the value of customer satisfaction surveys for project-based organizations: symbolic, technical, or none. international journal of project management, 23(5), 404– 409. http://link.springer.com/search?facet-author=%22hubert+graja%22 http://link.springer.com/search?facet-author=%22jennifer+mcmanis%22 ijahp: bijan, keramati, salehi/comparison of user satisfaction of ecommerce websites by the analytic network process international journal of the analytic hierarchy process 244 vol. 5 issue 2 2013 issn 1936-6744 kumar, v., sharma, a., shah, r., & rajan, b. (2013). establishing profitable customer loyalty for multinational companies in the emerging economies: a conceptual framework. journal of international marketing, 21(1), 57-80. liu, x., zeng, x., xu, y., & koehl, l. (2008). a fuzzy model of customer satisfaction index in ecommerce. mathematics and computers in simulation, 77 (5-6), 512–521. luo, x., wieseke, j., & homburg, c. (2012). incentivizing ceos to build customerand employee-firm relations for higher customer satisfaction and firm value. journal of the academy of marketing science, 40, 745–758. saaty, r. w. (2003). decision making in complex environment: the analytic hierarchy process (ahp) for decision making and the analytic network process (anp) for decision making with dependence and feedback. pittsburgh: super decisions. saaty, t. l. (1980). the analytic hierarchy process. new york: mcgraw-hill international. saaty, t. l. (1990). how to make a decision: the analytic hierarchy process. european journal of operational research, 48(1), 9-26. saaty, t. l. (2004). super decisions software. pittsburgh: rws publications. saaty, t. l. (2005). theory and applications of the analytic network process: decision making with benefits, opportunities, costs and risks. pittsburgh: rws publications. saaty, t. l., & vargas, l. g. (2006). decision making with the analytic network process: economic, political, social and technological applications with benefits, opportunities, costs and risks. new york: springer science+ business media, llc. saaty, t. l., & sodenkamp, m. (2008). making decisions in hierarchic and network systems. int. j. applied decision sciences, 1(1), 24-79. sadeghi, m., rashidzadeh, m. a., & soukhakian, m. a. (2012). using analytic network process in a group decision-making for supplier selection. informatica, 23(4), 621–643. schaupp, l. c., & bélanger, f. (2005). a conjoint analysis of online consumer satisfaction. journal of electronic commerce research, 6(2), 95-111. singh, m. (2002). e-services and their role in b2c e-commerce. journal of management service quality, 12(6), 434-446. http://books.google.com.au/books?hl=en&lr=&id=tegygx6jro8c&oi=fnd&pg=pa1&dq=decision+making+with+the+analytic+network+process:+economic,+political&ots=ydlbkwtgdo&sig=oa2lngapovj7gjiwukhsoc2ylaa http://books.google.com.au/books?hl=en&lr=&id=tegygx6jro8c&oi=fnd&pg=pa1&dq=decision+making+with+the+analytic+network+process:+economic,+political&ots=ydlbkwtgdo&sig=oa2lngapovj7gjiwukhsoc2ylaa http://books.google.com.au/books?hl=en&lr=&id=tegygx6jro8c&oi=fnd&pg=pa1&dq=decision+making+with+the+analytic+network+process:+economic,+political&ots=ydlbkwtgdo&sig=oa2lngapovj7gjiwukhsoc2ylaa https://www.google.com.au/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&sqi=2&ved=0ccsqfjaa&url=http%3a%2f%2fwww.informatica.si%2f&ei=iq2cutgzomw6lax_-yhgdq&usg=afqjcng0hx0qs-gqoiam2bfio1wld9qqtg&sig2=sqeofbvdi7fo9ekpvdhgpg ijahp: bijan, keramati, salehi/comparison of user satisfaction of ecommerce websites by the analytic network process international journal of the analytic hierarchy process 245 vol. 5 issue 2 2013 issn 1936-6744 taslicali, a. k., & ercan, s. (2006). the analytic hierarchy & the analytic network processes in multicriteria decision making: a comparative study. journal of aeronautics and space technologies, 2(4), 55-65. tullis, t., & bill, a. (2008). measuring the user experience: collecting, analyzing, and presenting usability metrics. burlington: morgan kaufmann publishers. wood, f. b., siegel, e. r., feldman, s., love, c. b., rodrigues, d., malamud, m., lagana, m., & crafts, j. (2008). web evaluation at the us national institutes of health: use of the american customer satisfaction index online customer survey. journal of medical internet research, 10, e4. yang, j., & ding, y. (2009). b2b e-commerce website customer satisfaction: a formula and scale. international asia symposium on intelligent interaction and affective computing. ieee xplore, wuhan, 191-197. ijahp: bijan, keramati, salehi/comparison of user satisfaction of ecommerce websites by the analytic network process international journal of the analytic hierarchy process 246 vol. 5 issue 2 2013 issn 1936-6744 appendix i this appendix includes the survey questionnaire which was sent to the recruitment websites users by email. at the beginning of the survey, a question asked the users whether they have used both websites. then, only those users who gave a positive answer to this question were allowed to fill in the rest of the survey. a glossary of literature terms is also included in this questionnaire for users’ information. also, it is mentioned that scale of 1–9 is used for two components, with a score of 1 representing indifference between the two components and 9 being overwhelming dominance of the component under consideration. irantalent 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 agahjobs which website do you prefer based on information quality? irantalent 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 agahjobs which website do you prefer based on information accuracy? irantalent 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 agahjobs which website do you prefer based on convenience of the information? irantalent 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 agahjobs which website do you prefer based on usefulness of the information? irantalent 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 agahjobs which website has clearer site organization? irantalent 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 agahjobs which website is easier for you to read? irantalent 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 agahjobs which website is easier for you to find information that you are looking for? irantalent 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 agahjobs which website is easier to navigate? irantalent 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 agahjobs which website has higher rate at the reliability of site performance in your point of view? irantalent 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 agahjobs which website does provide a higher level of your overall satisfaction? irantalent 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 agahjobs which website does meet your expectations more? irantalent 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 agahjobs which website is closer to your idea of an ideal website? irantalent 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 agahjobs which website do you visit more regularly? irantalent 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 agahjobs which website do you prefer to recommend to someone else? ijahp: bijan, keramati, salehi/comparison of user satisfaction of ecommerce websites by the analytic network process international journal of the analytic hierarchy process 247 vol. 5 issue 2 2013 issn 1936-6744 please compare characteristics of each pair for the websites. agahjobs information accuracy 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 information quality usefulness of the information 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 convenience of the information clarity of site organization 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 ease of reading the site ability to find wanted information 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 ease of navigation meet your expectations 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 overall satisfaction with this site meet your expectations 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 compare to your idea of an ideal website overall satisfaction with this site 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 compare to your idea of an ideal website organized search results 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 search features help you narrow the results organized search results 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 comprehensive search results organized search results 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 usefulness of search results search features help you narrow the results 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 comprehensive search results search features help you narrow the results 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 usefulness of search results comprehensive search results 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 usefulness of search results irantalent information accuracy 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 information quality usefulness of the information 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 convenience of the information clarity of site organization 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 ease of reading the site ability to find wanted information 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 ease of navigation meet your expectations 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 overall satisfaction with this site meet your expectations 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 compare to your idea of an ideal website irantalent 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 agahjobs which website has more organized search results? irantalent 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 agahjobs which website has more search features that help you narrow the results? irantalent 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 agahjobs which website does provide more comprehensive search results? irantalent 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 agahjobs which website has more useful search results? ijahp: bijan, keramati, salehi/comparison of user satisfaction of ecommerce websites by the analytic network process international journal of the analytic hierarchy process 248 vol. 5 issue 2 2013 issn 1936-6744 overall satisfaction with this site 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 compare to your idea of an ideal website organized search results 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 search features help you narrow the results organized search results 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 comprehensive search results organized search results 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 usefulness of search results search features help you narrow the results 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 comprehensive search results search features help you narrow the results 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 usefulness of search results comprehensive search results 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 usefulness of search results in your opinion, which of the following characteristics is more encouraging to recommend the websites to someone else? recommend the site to someone else meet your expectations 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 overall satisfaction with this site meet your expectations 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 compare to your idea of an ideal website overall satisfaction with this site 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 compare to your idea of an ideal website in your opinion, which of the following characteristics is more encouraging to return to the websites? return to the site meet your expectations 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 overall satisfaction with this site meet your expectations 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 compare to your idea of an ideal website overall satisfaction with this site 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 compare to your idea of an ideal website which of the following factors help more to meet your expectations? meet your expectations information accuracy 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 information quality usefulness of the information 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 convenience of the information clarity of site organization 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 ease of reading the site ability to find wanted information 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 ease of navigation organized search results 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 search features help you narrow the results organized search results 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 comprehensive search results organized search results 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 usefulness of search results search features help you narrow the results 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 comprehensive search results search features help you narrow the results 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 usefulness of search results comprehensive search results 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 usefulness of search results ijahp: bijan, keramati, salehi/comparison of user satisfaction of ecommerce websites by the analytic network process international journal of the analytic hierarchy process 249 vol. 5 issue 2 2013 issn 1936-6744 which of the following factors help more to reach your overall satisfaction? overall satisfaction with a website information accuracy 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 information quality usefulness of the information 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 convenience of the information clarity of site organization 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 ease of reading the site ability to find wanted information 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 ease of navigation organized search results 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 search features help you narrow the results organized search results 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 comprehensive search results organized search results 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 usefulness of search results search features help you narrow the results 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 comprehensive search results search features help you narrow the results 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 usefulness of search results comprehensive search results 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 usefulness of search results which factor has a better performance compared to your ideal website? compare to your idea of an ideal website information accuracy 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 information quality usefulness of the information 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 convenience of the information clarity of site organization 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 ease of reading the site ability to find wanted information 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 ease of navigation organized search results 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 search features help you narrow the results organized search results 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 comprehensive search results organized search results 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 usefulness of search results search features help you narrow the results 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 comprehensive search results search features help you narrow the results 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 usefulness of search results comprehensive search results 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 usefulness of search results in your opinion, which factor is more important to choose a recruitment website? alternatives content 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 functionality content 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 loyalty to return content 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 look & feel content 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 navigation content 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 others recommendations on the site ijahp: bijan, keramati, salehi/comparison of user satisfaction of ecommerce websites by the analytic network process international journal of the analytic hierarchy process 250 vol. 5 issue 2 2013 issn 1936-6744 content 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 satisfaction content 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 search content 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 site performance functionality 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 loyalty to return functionality 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 look & feel functionality 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 navigation functionality 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 others recommendations on the site functionality 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 satisfaction functionality 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 search functionality 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 site performance loyalty to return 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 look & feel loyalty to return 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 navigation loyalty to return 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 others recommendations on the site loyalty to return 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 satisfaction loyalty to return 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 search loyalty to return 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 site performance look & feel 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 navigation look & feel 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 others recommendations on the site look & feel 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 satisfaction look & feel 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 search look & feel 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 site performance navigation 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 others recommendations on the site navigation 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 satisfaction navigation 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 search navigation 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 site performance others recommendations on the site 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 satisfaction others recommendations on the site 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 search others recommendations on the site 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 site performance satisfaction 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 search satisfaction 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 site performance search 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 site performance what make you more likely to return to your preferred website? likelihood to return compared to the other site 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 satisfaction ijahp: bijan, keramati, salehi/comparison of user satisfaction of ecommerce websites by the analytic network process international journal of the analytic hierarchy process 251 vol. 5 issue 2 2013 issn 1936-6744 what make you more likely to recommend your preferred website? likelihood to recommend compared to the other site 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 satisfaction what would make you more satisfied with your preferred website? satisfaction compared to the other site 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 content compared to the other site 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 functionality compared to the other site 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 look & feel compared to the other site 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 navigation compared to the other site 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 search compared to the other site 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 site performance content 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 functionality content 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 look & feel content 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 navigation content 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 search content 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 site performance functionality 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 look & feel functionality 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 navigation functionality 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 search functionality 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 site performance look & feel 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 navigation look & feel 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 search look & feel 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 site performance navigation 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 search navigation 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 site performance search 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 site performance ijahp article:pacheco, piratelli, silva, belderrain /the design of a performance measurement system for a fruit processing equipment manufacturer international journal of the analytic hierarchy process 331 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.640 the design of a performance measurement system for a fruit processing equipment manufacturer bruna pachecho bruna_scarduelli@yahoo.com.br claudio piratelli clpiratelli@uniara.edu.br ethel silva e-chiari@uol.com.br mischel carmen belderrain carmen@ita.br university of araraquara-uniara brazil abstract several performance measurement systems (pms) are referred to in the literature, but there is little mention of the design stage in the construction of a pms. the purpose of this paper is to design a performance measurement system capable of assisting management and continuously improving the production process of a manufacturer of food production equipment and maintenance service provider headquartered in the state of são paulo. it is based on the method proposed by piratelli (2010) called the performance prism (tpp) framework. it has two phases, and the first is based on the strategic options development and analysis (soda) methodology; the second uses the analytic network process (anp), a multiple criteria decision making (mcdm) method. nine performance indicators were created to meet the needs presented by the stakeholders: the directors, quality coordinator, and production manager, given the faces of the performance prism. the construction of a pms allowed the company being studied to draw up strategy and management plans for the production process. the proposed method was shown to be effective when applied. keywords: the performance prism; soda; anp; intermediate goods industry 1. introduction performance measurement systems (pms) are addressed in the literature as an efficient way to seek continuous improvement and help organizations be competitive in the market. however, there is little in the literature on how to design a pms to meet the particular needs of each company. measuring performance is not a trivial task since it involves considering specific characteristics of each unique company or sector. choong (2013; 2014a) conducted a scientific study using abi/inform proquest, emerald full mailto:bruna_scarduelli@yahoo.com.br mailto:clpiratelli@uniara.edu.br mailto:e-chiari@uol.com.br mailto:carmen@ita.br ijahp article:pacheco, piratelli, silva, belderrain /the design of a performance measurement system for a fruit processing equipment manufacturer international journal of the analytic hierarchy process 332 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.640 text, science@direct and ebsco 1990-2012, and noted a lack of articles about the attributes needed for a pms. choong's (2014b) research recommends the need for a new paradigm in the advancement of pms studies. he recommends a shift in the emphasis on the measurement of a more theoretical approach to the adoption of a more scientific approach to improve the measurement of organizational activities and performance. the multi-methodology used in this work aims to integrate the problems structuring method (psm) and multiple-criteria decision-making (mcdm) which belong to different, soft and hard paradigms of operational research. marttunen, lienert and belton (2017) conducted a study to explore how the psms and mcdm methods are applied together and concluded that they are complementary methods and when applied together result in many synergies and mutual benefits because the combination provides a richer picture of the decision situation and a methodology that can better handle the various phases of decision making. the psm used is the strategic options and development analysis (soda) and the mcdm used is the analytic network process (anp). for eden and simpson (1989), the forerunners of soda, it is an approach to complex organizational problems that uses the constructivist paradigm to help decision-makers structure their ideas, facilitating the construction of a model that brings the objectives of each stakeholder involved in the decision-making process, and integrates the participants into a collective reflection on the problem. according to saaty (1999; 2008), the anp although built on the analytic hierarchy process (ahp), goes beyond it because it includes the dependence between the criteria. the objective of this study is to design a performance measurement system for the production process management at a company that manufacturers fruit processing equipment, based in the state of são paulo, brazil. the specific objectives can be divided into the following steps: 1 identifying the performance criteria for stakeholders through the soda methodology, bearing in mind the faces of tpp framework. 2 faithful modeling of the criteria's dependency relationships and prioritizing them using the anp. 3 applying these frameworks in the production area at the food equipment manufacturing company in the state of são paulo. the second section of this paper provides a brief review of the literature on the main performance measurement frameworks, focusing on presenting tpp framework. the third section presents the operational research methods used in the proposed method, in particular the soda methodology and the anp. the fourth section presents the operational procedures for the proposed method. the fifth section presents the results obtained by applying the method to the study object, and the sixth section presents the final considerations. ijahp article:pacheco, piratelli, silva, belderrain /the design of a performance measurement system for a fruit processing equipment manufacturer international journal of the analytic hierarchy process 333 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.640 2. performance measurement systems "if you can't measure it, you can't manage it" (kaplan & norton, 1997). according to sobreira netto (2007) and neely (1999; 2005), there is a recurring concern about monitoring the performance of organizations. performance measurement is nothing new, and there is a significant number of frameworks available (neely et al., 1997; neely & bourne, 2000). the definition of performance management should be clearly related to the achievement of strategic priorities, as a management tool, and the language should be simple and understandable (atkinson, 2012; bititci et al., 1997; bititci et al., 2000; hanson, melnyk & calantone, 2010). 2.1 frameworks and views of performance measurement systems choong (2013) surveyed the literature on performance measurement systems (published from january 1990 to november 2012) and found 479 articles on abi/inform proquest, emerald full text, science@direct and ebsco. many of the articles made a brief reference to pms and were excluded from the survey, as were articles that used methodologies such as case studies or studies/reviews that addressed the subject too narrowly. after these exclusions, the number of relevant articles in the survey, with more than five citations within two years of each other, fell to 67. the number of articles that address measuring attributes for pms is low showing it is poorly researched. in the survey, kaplan and norton are the most commonly cited authors in reference to the balanced scorecard, followed by neely, who created the performance prism (choong, 2013). this research was updated in 2014 by the same author. 2.2 the performance prism framework according to neely et al. (2002), the performance prism (tpp) attempts to integrate five related perspectives in a three-dimensional framework as follows: a) stakeholder satisfaction who are the company's stakeholders and what do they want and need?; b) strategies what strategies does the company need to implement in order to meet the stakeholders' needs?; c) processes what critical processes are required to carry out these strategies?; d) capabilities what capacities does the company need to operate and improve these processes?; e) stakeholders' contribution what contributions does the company demand of stakeholders to maintain and develop these capacities? for neely et al. (2002), tpp complements the balanced scorecard by analyzing end users, employees, suppliers, regulators, pressure groups or local communities. it reinforces the thesis that these stakeholders can have a significant impact on the organization and the conduction of its activities in addition to building the strategy that is, how the objectives will be achieved. 2.3 pms design neely et al. (2000) proposed a method for the development of a pms in six phases, presented in a workbook about which indicators are needed, the cost-benefits of the process, the purpose of performance measurement for the application, functional testing, ijahp article:pacheco, piratelli, silva, belderrain /the design of a performance measurement system for a fruit processing equipment manufacturer international journal of the analytic hierarchy process 334 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.640 environmental analysis and periodic maintenance of the system. according to rentes, carpinetti and van aken (2002), steps can be taken to design a pms, in other words, to create a system of construction or revision of the organizational performance measurement that allows incorporating improvements in the critical points or failures of the development process and implementation of performance measurement. the steps are as follows: identify the strategic objectives; identify critical factors/areas for success; define high-level metrics related to critical factors/areas for success and goals; implement performance metrics and goals at other levels; plan the operation of the system. note that the above present steps are common to the pms design process, but do not detail or exemplify how to put them into practice. based on the use of the multi-criteria decision aid constructivist (mcda-c) methodology, ensslin et al. (2010) proposed the systematic building of a framework for performance evaluation that is systematically structured into three groups of activities: structuring, evaluation and recommendations. the structuring phase is to identify, organize and ordinally measure the concerns that the decision-maker considers necessary and sufficient for assessment of the context. the second phase, assessment, is an instrument to improve understanding by building cardinal scales and replacement rates to represent their local and global preferences. recommendations comprise the third phase and aim to continue the process of expanding the understanding of the context by trying to understand the consequences of possible decisions in representative criteria for the dimensions considered relevant and in the context as a whole. piratelli (2010) proposed a method to design a pms based on the tpp framework, combining the constructivist soda methodology to identify the performance criteria identified by the stakeholders with the anp method with a rationalist nature, which will be detailed in section 4. 3. operational research methods this section contextualizes the operational research (or) methods used in combination in this study. according to amaral and araujo (1998), the ors essentially assist decision makers in troubleshooting. according to mingers and brocklesby (1997), a very common use of psms is through multi-methodology or combining methods according to each phase of a problem. marttunen, lienert and belton (2017) corroborate this view. mingers and rosenhead (2004) state that each problem is unique, and it is not possible to compare the effectiveness of different methodologies for a specific situation. ackermann et al. (2014) defend the mixture of methods, as this allows the field of research to remain fresh and vibrant, and provides the extensions/adaptations needed to provide the means to manage a wider range of problems. also, the very act of mixing ijahp article:pacheco, piratelli, silva, belderrain /the design of a performance measurement system for a fruit processing equipment manufacturer international journal of the analytic hierarchy process 335 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.640 methods helps broaden awareness in the field of research since many of the combinations incorporate the research of structuring problems in different disciplines. complex decisions must be made in the design phase of a pms, both at the time of structuring the pms (what to measure) and at the time of modeling (how to measure it). the soda methodology will be used to structure the pms and capture how the stakeholders think about organizational performance in order to select the evaluation criteria. the modeling will use the anp, which enables the pms to evaluate the identified criteria while maintaining the same relationships (dependency and/or feedback) surveyed in the phase by those involved in the structuring. 3.1 strategic options development and analysis according to rosenhead (1989), soft or has been developed since 1980 through methodologies such as soft systems methodology (ssm), strategic options development and analysis (soda), and strategic choice approach (sca). for eden and simpson (1989), soda is a methodological approach to complex organizational problems using the constructivist paradigm to assist decision makers in structuring their ideas through a cognitive map. in soda, the objectives for each stakeholder involved in the decision-making process are presented and interrelated in order to get a collective reflection on the problem. hence, software such as banxia's decision explorer® is indispensable (westcombe, 2002). the procedures for preparing and analyzing a cognitive map can be found in eden (1989), ensslin et al. (2001), and piratelli (2010), and are also described in section 4. the constructivist methodology has been applied in several decision situations (ensslin, dutra & ensslin, 2000; ensslin, ensslin & lacerda, 2011; rosa et al., 2012; ensslin et al. 2013). 3.1.1 cognitive map applications in pms ensslin et al. (2010) made use of the cognitive mapping technique to propose a performance evaluation model for outsourced companies in telecommunications with operations in southern brazil. the application identified the criteria deemed important by the project manager and made it possible to build a decision support tool to approve subcontractors. madeira junior, gonçalves and belderrain (2011) presented the structuring of a quality performance evaluation for container terminals using cognitive maps. the authors concluded that the structuring of the evaluation system was consistent with the literature. azevedo et al. (2011) made use of the cognitive mapping technique to evaluate the performance of the budgeting process at a construction company. meetings were held with decision makers and facilitators, lasting 40 and 200 hours, respectively. subsequently, the value hierarchy was developed, identifying fundamental points of view (fpv) and elementary points of view (epv). bortoluzzi et al. (2011) built an evaluation model for economic-financial performance in a case study at seprol computadores e sistemas ltda., considering financial indicators, while using cognitive mapping. the stakeholders were identified and maps were drawn from primary assessment elements (paes) and defined concepts, and the clusters were defined. the hierarchical value structure was then drawn up, identifying the three most prominent dimensions: financial management, human resources and the market. the ijahp article:pacheco, piratelli, silva, belderrain /the design of a performance measurement system for a fruit processing equipment manufacturer international journal of the analytic hierarchy process 336 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.640 financial management performance descriptors identified below-par performance and corrective action was decided on. ensslin et al. (2016) used cognitive maps to build a performance evaluation model for customer service and business performance at a bank, based on the team manager's values and preferences. the objectives were to identify the criteria that demonstrate process performance and identify the status quo with regard to them. the mapping process identified 91 paes and 7 fpvs, allowing the decision maker to determine performance criteria and identify points to be improved. the involvement of the decision maker meant the study was aligned with their perception. however, it limited the research regarding their vision of the context described. furthermore, it does not show whether the suggestions for improvement were successful. 3.2 multicriteria decision making methods multicriteria decision making (mcdm) comprises methods and techniques to assist or support people and organizations in making decisions that are influenced by multiple criteria, bearing in mind the need for prior specification of the objectives intended by the decision maker compared against the alternatives (bana e costa, 1992; rosenhead, 1989; salgado, belderrain & silva, 2009). 3.2.1 the analytic network process the anp method is a generalization of the ahp which allows dependencies between the criteria and influences between the alternatives to be analyzed. the anp does not follow the axiom of independence and if there is dependency between the criteria or influence among the alternatives it judges how dependent a criterion is on another and how much an alternative is influenced by or influences others (hernández, marins & salomon, 2011; kravchenko & seredenko, 2011; mostaf et al. 2015). the possibility of negotiating tangible values with intangible ones is one of the great attractions of the ahp, according to salomon and montevechi (1998). the possibility of negotiating values that do not need to be totally independent is the great attraction of the anp. silva et al. (2009) proposed the application of the anp in 3 steps as follows: 1formulation of the problem, 2-judgments and 3-structuring of the supermatrix and obtaining the results. saaty and vargas (2006) suggest using the super decisions software to perform comparisons and algebraic calculations, referring to the matrix of judgments between related nodes and clusters that present interrelated elements. 3.2.1.1 applications of the anp in pms design carlucci (2010) used the anp to select key performance indicators in a case study at a sofa maker. the model assessed the importance of the existing performance indicators in the manufacturing process and prioritized a set of indicators to provide adequate information with which to evaluate and make decisions. the method proved to be efficient in prioritizing performance criteria, and assessing the influence among them. piratelli and belderrain (2010) addressed the design of a pms based on tpp using the anp method for modeling and rating performance indicators for the production engineering course at an institution of higher education. the model and its results represent the needs of stakeholders (students, organizations, the educational institution and society/government) through 58 performance indicators divided into four groups: ijahp article:pacheco, piratelli, silva, belderrain /the design of a performance measurement system for a fruit processing equipment manufacturer international journal of the analytic hierarchy process 337 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.640 satisfaction, processes, skills and contribution (tpp faces). the model was shown to robustly and accurately reflect the strengths and weaknesses of the course. hu, wang and wang (2012) built a tailor-made homestay pms by reviewing the literature and interviewing experts to make the evaluation framework more comprehensive and more practical. they adopted the anp to obtain the weights and verify the performance of the homestay business through diffuse theory. as far as the fundamentals are concerned, owners and customer groups both weigh the surroundings of the site as well as the characteristics, quality of service, operation and management. as far as the overall performance of the homestay is concerned, customer groups consider it to have reached a satisfactory level. guimarães and salomon (2015) presented an assessment of the priorities of reverse logistics indicators in a small footwear factory in the state of ceará, through the anp. they found that there is a similarity to the study in a sample of brazilian companies for the first four indicators found, which is mainly explained by the priority level of economic drivers 55.7% and the image 24.6%, and the influences that these indicators receive from other indicators. for the other indicators, there were changes in priorities in relation to the sample survey with brazilian companies. they considered the influence of the type of enterprise included in the research and reverse logistics programs. kucukaltan, irani and aktas (2016) built a pms for the logistics sector in turkey, combining the balanced scorecard to identify the performance criteria and since the indicators are not independent of each other, the anp method to analyze the interrelationships. 43 indicators were found, and 15 of them were identified as the most important in the logistics sector. in contrast to common expectations regarding the importance of some specific metrics (eg. timely delivery), research indicates that the educated employee is the most important performance indicator for competitiveness. during the research, a lack of studies that use the anp was noted, with most articles admitting a lack of dependency between the criteria and applying the analytic hierarchy process (ahp). ho and ma (2018) conducted a review of the literature on integrated ahp approaches and applications published between 2007 and 2016 and compared those studies with articles published during the previous decade (1997-2006). performance evaluation, according to this study, is the second most commonly used problem, second only to vendor selection and evaluation. 4. the proposed method – operational procedures the objective of this section is to present the method proposed by piratelli (2010) applied in a higher education institution of the state of são paulo, and to apply it in the company that manufactures equipment for the food industry, as well as provides maintenance services. figure 1 shows the sequence of this method. the first phase of the method is constructivist and qualitative and includes two stages. its goal is to structure the pms using the soda methodology. the second phase is rationalist and quantitative and includes two stages. it models the relationship between ijahp article:pacheco, piratelli, silva, belderrain /the design of a performance measurement system for a fruit processing equipment manufacturer international journal of the analytic hierarchy process 338 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.640 the performance criteria (identified in the structuring stage) to order them for evaluation of the organization's overall performance. hence, it uses the anp method. in phase 1, the first stage is divided into two steps: identifying the stakeholders in the organization, those who are concerned with the problem, and then identifying the decision makers, those who have decision-making authority, the designers of the pms. the second stage has seven steps. the first step is to define the label or briefly describe the problem to be solved, which in this case is a pms for the organization studied. in the second step there will be a brainstorming session with decision makers and stakeholders at the company (primary assessment elements (pae) are generated from the label). in the third step, the individual cognitive maps are drawn. the fourth step is the aggregation of individual maps into a single map. the construction of the aggregate map, the fifth step, is done at a meeting between the facilitator and decision makers to arrive at a consensus to validate the collective learning map. the analysis of the aggregate map by the facilitator is the sixth step, and is essential to the success of the rest of the process. it must identify the concepts in each line of reasoning (tails, means and ends), branches and clusters of the map. the seventh step is the identification of fundamental points of view (fpvs) which are the concepts classified as essential, controllable and measurable. some fpvs need to be broken down into more than one concept to evaluate the same performance, the elementary points of view (epvs). the fpvs and epvs form the family of fundamental points of view (points that decision makers want to measure). the seventh step also classifies aggregated mc concepts in regard to tpp-satisfaction (stakeholders, satisfaction or value delivery processes, the organization's capacities and the contribution from stakeholders). the third stage (phase 2) has four steps and starts with modeling (decision problem structuring) by defining the model's objectives, clusters and network elements (performance criteria and sub-criteria). the criteria and sub-criteria are the fpvs and epvs, respectively which were identified in the seventh step of stage 2 of the first phase. it defines the overall performance of the organization in terms of satisfaction criteria for its various stakeholders. the modeling of the pms follows the four or five faces of the prism (clusters: satisfaction, value delivery processes, capacities and contributions of the stakeholders. strategy is not a measurable face and, therefore, should not be incorporated into the model). the construction of the network starts in the second step, where the clusters' interand intra-relations of dependence are indicated, and those for each network element with the others. dependence relations and feedback between the elements must be in accordance with the concepts identified in the mc. in the third step, the performance criteria and sub-criteria judgments for strategic definition are determined through pairwise comparisons using the saaty scale (1980). the weighting for the performance criteria and sub-criteria determines what is strategic for the organization's performance through an array of priorities. a consistency analysis should be performed in this step. the fourth step presents the model's results, ordering the criteria priorities for the organization's performance. ijahp article:pacheco, piratelli, silva, belderrain /the design of a performance measurement system for a fruit processing equipment manufacturer international journal of the analytic hierarchy process 339 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.640 figure 1 detailed sequence of the method proposed source: piratelli (2010) the fourth stage has five steps. in the first step, indicators are built (a performance criterion must have a descriptor or metric and a value function). the descriptor or metric is a group of impact levels associated with a value function constructed by the decision makers. in the second step, an evaluation of organizational performance is conducted. the organization is evaluated (by the decision makers) in each performance indicator, according to the level at which this indicator best represents its current situation (attributed to its respective value level, according to the value function obtained in the previous step). this evaluation provides the management priorities (third step), calculating the impact of each indicator on the organization's final performance. the impact of each indicator can ijahp article:pacheco, piratelli, silva, belderrain /the design of a performance measurement system for a fruit processing equipment manufacturer international journal of the analytic hierarchy process 340 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.640 be defined as the potential contribution that it would make to the organization's maximum performance. in the fourth step the pms's response sensitivity is analyzed to changes in the weights of the clusters (tpp faces). its goal is to verify the organization's performance and management priorities with regard to the prioritization of each cluster at the expense of the others. finally, in the fifth step, the pms is verified by the group of decision makers. 5. the study the company being studied is classified as an intermediary goods company, and operates in the food equipment manufacturing sector performing custom designing engineering, sales and industrial maintenance projects. it employs 18 people and is iso 9001 certified. one of the recommendations made by iso auditors was the implementation of a pms for the production system since it is its core business and there was not a system in place to analyze its production processes, cost, reworking and so on. the application of the proposed method follows the sequence presented in the operational procedures (section 4). phase 1 constructivist stage 1: (steps 1 and 2) identification of stakeholders and decision makers involved in the pms design. stakeholders identified in the study subject company with regard to the construction of a pms for the production process are the board/quality coordinator and production manager. since the coordinator of quality is a board member management and coordination of quality is considered as a single stakeholder. the identified decision maker is the director/coordinator of quality. stage 2: structuring the pms using soda. step 1: defining the problem label the problem label is "building a pms for the company's production process." step 2: individual brainstorming with decision makers and stakeholders four visits were made from january to july 2014 to discuss the current production process and the difficulties and needs identified by the board. the stakeholders were worked with individually, and an example of an pae was obtained from the board: have indicators (number 1). step 3: drawing up of individual cognitive maps from may to september 2014, two cognitive maps were drawn up and transcribed by january 2015 with the help of banxia's decision explorer® software. step 4: aggregation of individual cognitive maps after the construction of the individual maps, they were aggregated to gather all the concepts and lines of reasoning in a single map. similar concepts were added in one single branch; distinct concepts created new lines of reasoning. in the aggregation of the ijahp article:pacheco, piratelli, silva, belderrain /the design of a performance measurement system for a fruit processing equipment manufacturer international journal of the analytic hierarchy process 341 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.640 maps, several concepts were repeated, showing that the stakeholders have similar views on what is important for the production process and they share the same ideal about what is necessary to the rollout of the pms. in the process of aggregating the mc, it was noted that among the aspects that differentiate the individual mcs was the concern among directors and coordinators about quality regarding iso requirements in order to ensure certification, and the pursuit of continuous improvement in the production process. step 5: construction of the aggregated mc the aggregated mc was conceived of at the meeting with stakeholders where the aggregated cognitive map was presented. the parties involved got to know the whole and learned from the other concepts and views presented. the stakeholders validated 31 concepts in 38 lines of reasoning on the congregate map. figure 2 shows the (congregate) collective maps . figure 2 congregate map step 6: analysis of the congregate mc the congregate map allowed the identification of lines of reasoning and, therefore, the aspects most valued by the company that must be taken into account for composition of the pms. in this study, the clusters found manually were classified according to the four faces of tpp: process satisfaction of stakeholders, contribution by stakeholders and capacities. ijahp article:pacheco, piratelli, silva, belderrain /the design of a performance measurement system for a fruit processing equipment manufacturer international journal of the analytic hierarchy process 342 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.640 step 7: determination of the family of fundamental points of view (fpvs) and classification of concepts of the congregate mc regarding the faces of the performance prism. the concepts of the congregate map's 38 lines of reasoning were classified by the decision makers as essential, controllable and measurable to identify the family of fpvs, whose properties are: essential, manageable, complete, measurable, operational, non-redundant, concise and comprehensible. the isolable property does not apply to the family since dependency was observed between some concepts (potential performance criteria), which justifies the choice of the anp method for the next phase of this research. the decision makers also classified the concepts according to tpp faces. given that the focus of the design is the construction of a pms for the production area, the processes are the face of tpp with the highest density of lines of reasoning on the map. chart 1 shows the fpvs and epvs and their respective performance criteria. measurable epvs give rise to nine performance criteria that can measure them. ijahp article:pacheco, piratelli, silva, belderrain /the design of a performance measurement system for a fruit processing equipment manufacturer international journal of the analytic hierarchy process 343 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.640 chart 1 fpvs and epvs (constructivist phase) and their performance criteria (rationalist phase) concepts (constructive phase) fpv epv tpp face performance criteria (rationalist phase) 4 meet deadlines ... do not meet them x processes deadline 5 comply with production scheduling...delay x processes stock turnover, defect index, stoppage index, scrap index, rework index, percentage of c items. 11 track material ... do not track x processes stock turnover, defect index, stoppage index, scrap index, rework index, percentage of c items. 12 identify / evaluate raw material quality ... do not evaluate x processes / contribution by stakeholders defect / scrap index / supplier qualification index 13 monitor supplier performance ... do not monitor x contribution by stakeholders supplier qualification index 15 identify production bottlenecks ... do not identify x processes stock turnover, defect index, stoppage index, scrap index, rework index, percentage of c items. 16 monitor information on scrap... do not monitor x processes scrap index 17 identify human error in the production process ... keep as is x capacities training 18 monitor costs ... do not monitor x processes stock turnover, defect index, stoppage index, scrap index, rework index, percentage of c items. 19 monitor information on reworking... do not monitor x processes rework index 20 controlling stock ... do not control x processes percentage items c / stock turnover 21 identify / classify items of greater and lesser importance ... do not classify x processes percentage items c 22 identify / classify supplier lead times ... keep as is x contribution by stakeholders supplier qualification index 24 monitor information on the types of product defects ... do not monitor x processes defect index 25 train employees ... do not train x capacities training 27 perform maintenance on equipment... do not perform x processes defect, stoppage, scrap and rework indexes 28 prevent unnecessary stoppages ... stop x processes defect, stoppage, scrap and rework indexes 29 comply with an iso 9001 requirement.. does not comply x processes / stakeholder satisfaction working stock index defects, stop index, index scrap, rework index percentage of items c, delivery time. ijahp article:pacheco, piratelli, silva, belderrain /the design of a performance measurement system for a fruit processing equipment manufacturer international journal of the analytic hierarchy process 344 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.640 phase 2 – rationalist stage 3: multi-criteria modeling of the pms this step includes the modeling of the pms in a network and the organization of the ordering of performance criteria and sub-criteria occur. super decisions software is used as a tool to facilitate the implementation of the anp. step 1: decision problem structuring definition of the model's objectives, clusters, and the network elements (performance criteria and sub-criteria). the first step in the rationalist phase begins with problem structuring. the model's objective is the organization's performance and the performance depends on the performance criteria distributed in the prism faces (clusters). figure 3 modeling with the pms's anp (production process) the performance criteria relations must be faithful to the relations identified in the congregate mc. the modeling must be based on four of the five tpp faces (satisfaction, value delivery processes, capacities and contributions from stakeholders) as strategy, not being measurable, will not be included in the model. strategic direction can only be built after knowing the relative importance of each performance criterion. super decisions software was used to build the model shown in figure 3, which shows relationships of dependence and feedback between the elements of the clusters. step 2: construction of the network: indication of dependency relations and feedback between the elements of the clusters the dependency relationships among the elements of the model in figure 3 are presented in the global reach matrix, table 1, and the local reach matrix, table 2 (both binary). the relationships are extracted and faithful to the congregate map. for example, with regard to the global reach matrix, the processes depend on the objectives clusters, the cluster process itself that influences it and the satisfaction cluster. in the local reach matrix, for example, the defect index depends on training and the supplier qualification index. ijahp article:pacheco, piratelli, silva, belderrain /the design of a performance measurement system for a fruit processing equipment manufacturer international journal of the analytic hierarchy process 345 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.640 table 1 global reach matrix objective processes satisfaction capacity contribution objective 0 1 1 1 1 process 0 1 0 1 1 satisfaction 0 1 0 0 0 capacity 0 0 0 0 0 contribution 0 0 0 0 0 table 2 local reach matrix training supplier qualification index. stock turnover defect index stoppage index scrap index rework index % of c items delivery deadline training 0 0 0 1 0 1 1 0 1 supplier qualification index 0 0 1 1 0 1 1 0 1 stock turnover 0 0 0 0 0 0 0 0 1 defect index 0 0 0 0 0 1 1 0 1 stoppage index 0 0 0 0 0 0 0 0 1 scrap index 0 0 0 0 0 0 0 0 0 rework index 0 0 0 0 0 0 0 0 1 % of c items 0 0 0 0 0 0 0 0 0 delivery deadline 0 0 0 0 0 0 0 0 0 step 3: performance criteria and sub-criteria judgments for strategic definition (and respective consistency analysis). in a meeting with the decision maker, the relative importance of the clusters and performance criteria of the pms were judged through pairwise comparisons. the definition of the weights is strategic to improve the production process at the company, considering the priority of performance criteria. the consistency of judgment matrices was also assessed at this stage, where the facilitator presented the judgments' inconsistencies to make sure that the decision maker was aware of and agreed with them. the inconsistencies found presented a consistency ratio of <0.1 and were validated by the decision maker. in the comparison between the clusters, the weight for the cluster 'process' was greater than for the others, justified by the decision maker by the importance of that cluster to the production process. step 4: getting results from the model. the weights found for the performance criteria were: supplier qualification index (26%), followed by training criteria (20%) and the defect index (18%). these three criteria have a prominent position in the pms, as they make up 64% of the total weight. the delivery ijahp article:pacheco, piratelli, silva, belderrain /the design of a performance measurement system for a fruit processing equipment manufacturer international journal of the analytic hierarchy process 346 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.640 time has a weight of 11%, followed by the rework index, with 7%, and stock turnover and stoppages with 5%. the scrap index and the percentage of c items have the lowest weight with 4%. stage 4: performance evaluation, analysis of the results and validation of the pms model. the pms was subjected to an initial performance evaluation, in order for the decision makers to verify its robustness. step 1: construction of performance indicators (descriptors or metrics for performance criteria and sub-criteria) the decision maker built the performance indicators, which have to have a descriptor, whose objective is to evaluate the performance of possible action in regard to each criterion or sub-criterion (fpv or epv). the value function (vf) is a quantitative instrument to assist the decision maker in ordering the intensity of their preferences between pairs of levels, and the saaty scale was used for this construction. frameworks were constructed for each of the performance criteria listed within the clusters. step 2: performance evaluation (identification of strengths and weaknesses of the performance) and step 3: obtaining management priorities in the evaluation stage of the production process, the decision-maker attributed the value of pv for each criterion corresponding to the level of impact that best represents the performance of the production process, according to built descriptors (table 3). as there is only one decision maker, the overall performance is equivalent to the evaluation of the same for the pms built. the defect index requires the most attention, followed by the supplier qualification index and training index. the prioritization of performance criteria allows the company to evaluate and define its strategy, improving the processes and meeting the iso 9001 requirements that led to this study. table 3 levels indicators assigned by the decider and performance priorities indicator weight level indicator performance priorities % performance indicator defect score 18% 35% 12% 6% supplier qualification index 26% 63% 10% 16% training 20% 63% 7% 13% inventory turnover 5% 9% 5% 1% rework index 7% 35% 5% 3% deadline 11% 63% 4% 7% stop index 5% 33% 3% 2% percentage c items 4% 15% 3% 1% scrap index 4% 33% 2% 1% overall performance 48% step 4: analysis of the pms's response sensitivity to variations in cluster weights given that there is only one decision maker, the analysis of the pms's response sensitivity was performed by varying the cluster weights, in order to check if changing ijahp article:pacheco, piratelli, silva, belderrain /the design of a performance measurement system for a fruit processing equipment manufacturer international journal of the analytic hierarchy process 347 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.640 them varies the pms's performance. changes in the weight values were made systematically by changing the weight of a given cluster to 9 and keeping the others at 1. these disturbances took the pms from 48% in the evaluation of the decision maker to 57% in the highest index found (contribution cluster), which did not result in a significant change to its performance. there were only changes in the weights of indicators and their orders. step 5: validation of the pms in a meeting with the decision maker to validate the pms, the pms was considered representative and valid, and will be used as a continuous improvement tool for production processes. the meetings and the process as a whole were considered to have been opportunities to analyze the production process and its importance to the company. 5.1 comments on the results the certification of iso 9001 presented the need to implement the smd for the company's productive process, and its construction allowed a comprehensive vision for the management of the productive process of this company to be developed. the opportunity to listen to the stakeholders involved in the construction of the cognitive maps was of great value to the organization, considering that the routine running and execution of the activities of the company did not allow those involved to perceive the importance of the performance measures to better guide the strategies and actions of the company. through the use of the anp, it was possible to visualize the performance criteria under levels of importance within the constructed model, to evaluate the current performance and to identify the management priorities that will guide the company with regard to the productive process. the management priorities obtained are highly representative, guiding where corrective actions should be aimed at improving the production process. in addition, with such priorities in place, it is possible to return to the cm and check whether the processes continue to follow the concepts mentioned, or whether they need to be changed. in the case of the defect index, for example, which is in the first position in the ranking of management priorities, as well as having significant weight for the smd, the current level assigned by the decision maker shows the need to return to the mc, reassess the processes, and propose actions for improvement, such as: presenting the indicator to the employees involved, demonstrating the importance of the processes carried out, providing training to improve their qualification, creating an award policy for employees to encourage better results, as well as actions that the company can evaluate and implement. in second place, among the management priorities, is the supplier qualification index, which highlights the need to create a policy for supplier selection, evaluation and monitoring, which will assist both the management process itself and the fulfillment of iso requirements 9001. the training index, which occupies the third place in management priorities and is also a requirement of the standard, demonstrates a special need for the company's human resources and will also assist in the improvement of the index of defects that occupies the first place. the other indicators in the priority list ijahp article:pacheco, piratelli, silva, belderrain /the design of a performance measurement system for a fruit processing equipment manufacturer international journal of the analytic hierarchy process 348 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.640 should also be analyzed and actions should be planned to improve performance and assist the company in the search for continuous improvement. 6. final considerations the proposal to design a performance indicator capable of assisting management and continuously improving the production process at a food equipment manufacturing company was achieved. the performance prism model was adequate to the management of the productive system of the company under study since it was able to identify the needs of the stakeholders involved and contributed to the alignment of the company's strategy to the production area. the processes face was the one that appeared more in the concepts listed and therefore had the greatest prominence in the study. the soda methodology proved to be a great contribution to the work, enabling learning in the construction of the mc and evolving the stakeholders in collective reflections. the construction of mcs by the facilitator proved to be a complex and difficult task, considering the particularities of the stakeholders involved in the elaboration. the use of the decision explorer® software favored the construction of the maps, as well as the aggregation and analysis of them. the application of the anp as a mcdm method to aid decision making was fundamental to modeling smd as a function of tpp. the use of the super decisions® software facilitated the modeling, judgments according to the saaty scale and the inconsistencies with the decision maker. the company that was analyzed felt that the cohesion and the commitment of those involved, even though it has a very small family team, was one of the strengths to the process of construction and implementation of the smd for the productive process. this has achieved satisfactory results with customers, and will certainly improve its strategy in the production process through smd, which can optimize both the process itself and profitability since one of the weaknesses of the company was the absence of performance measures capable of assisting the production management process. it was verified that this process of construction of an smd should not be static. within a period to be defined by the company it should be reassessed in order to verify if the needs of the stakeholders presented in the mcs remain the same, or if changes in the process are necessary for some reason which would result in an update of the smd. 6.1 contributions and search limitations piratelli (2010) conceived a method for the construction of an smd and applied it in a production engineering undergraduate course of a private higher education institution in the interior of the state of são paulo, brazil. the present study used the same method to build an smd for the production process of a company that manufactures equipment for fruit processing. thus, the progress/contribution consists of the replication of the method in another sector, with different needs and numbers involved. the method proved to be able to support the smd construction process as valid for the company under study. as a ijahp article:pacheco, piratelli, silva, belderrain /the design of a performance measurement system for a fruit processing equipment manufacturer international journal of the analytic hierarchy process 349 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.640 result, we highlight the contribution to help the quality management process requested by the company to verify its management priorities, as well as to provide better knowledge of its processes and, consequently, to promote their continuous improvement. using a method of proven effectiveness for one sector in another is to make use of scientific studies to enable improvements, promote interdisciplinary cooperation and contribute to the knowledge of the area as well as, in this case, to solve an industry problem. limitations in the study are related to the fact that the company studied has only one decision maker, and group decisions would certainly broaden the points of view and enrich the study. for future studies, it is suggested the method proposed be applied to other types of companies in other sectors. ijahp article:pacheco, piratelli, silva, belderrain /the design of a performance measurement system for a fruit processing equipment manufacturer international journal of the analytic hierarchy process 350 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.640 references amaral, d. c. & araújo filho, t. (1998). aplicação da metodologia soda no processo de desenvolvimento de produto. in: encontro nacional de engenharia de produção, 18. niterói. anais. niterói:universidade federal fluminense. doi: https://doi.org/10.24873/j.rpemd.2018.05.216 ackermann, f., franco, l. a., rouwette, e., & white, l. (2014). special issue on problem structuring research and practice. euro journal on decision processes, 2 (3/4), 165–172. doi: https://doi.org/10.1007/s40070-014-0037-6 atkinson, m. (2012). developing and using a performance management framework: a case study. measuring business excellence, 16(3), 47-56. doi: https://doi.org/10.1108/13683041211257402 azevedo, r. c., ensslin, l., lacerda, r. d. o., frança, l. a.,, gonzalez, c. j. i., jungles, a. e. & ensslin, s. r. (2011). avaliação de desempenho do processo de orçamento: estudo de caso em uma obra de construção civil. ambiente construido, 11(1). doi: https://doi.org/10.1590/s1678-86212011000100007 bana e costa, c. a. (1992). structuration, construction et exploitation d'un modèle multicritère d'aide à la décision. tese (doutorado em engenharia de sistemas) – instituto superior técnico, universidade técnica de lisboa, lisboa, portugal. doi: https://doi.org/10.3934/krm.2018053 bititci, u. s., carrie, a.s. & mcdevitt, l. (1997). integrated performance measurement systems: a development guide. international journal of operations & production management, 17(5), 522-534. doi: https://doi.org/10.1108/01443579710167230 bititci, u. s., turner, t. & begemann, c. (2000). dynamics of performance measurement systems. international journal of operations & production management, 20(6), 692-704. doi: https://doi.org/10.1108/01443570010321676 bortoluzzi, s. c., ensslin, s. r. & ensslin, l. (2011). avaliação de desempenho multicritério como apoio à gestão de empresas: aplicação em uma empresa de serviços. gestão & produção, 18(3), 633-650. doi: https://doi.org/10.1590/s0104530x2011000300014 carlucci, d. (2010). evaluating and selecting key performance indicators: an anp-based model. measuring business excellence, 14( 2), 66-76. doi: https://doi.org/10.1108/13683041011047876 choong, k. k. (2013). understanding the features of performance measurement system: a literature review. measuring business excellence, 17(4), 102-121. doi: https://doi.org/10.1108/mbe-05-2012-0031 choong, k. k. (2014a). has this large number of performance measurement publications contributed to its better understanding? a systematic review for research and ijahp article:pacheco, piratelli, silva, belderrain /the design of a performance measurement system for a fruit processing equipment manufacturer international journal of the analytic hierarchy process 351 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.640 applications. international journal of production research, 52(14), 4174–4197. doi: https://doi.org/10.1108/ijppm-01-2013-0015 choong, k. k. (2014b). the fundamentals of performance measurement systems: a systematic approach to theory and a research agenda. international journal of productivity & performance management, 63(7), 879-922. doi: https://doi.org/10.1108/ijppm-01-2013-0015 coughlan, p. & coghlan, d. (2002). action research for operations management. international journal of operations & production management, 22(2), 220-240. eden, c. & simpson, p. (1989). using cognitive mapping for strategic options development and analysis. in: rosenhead, j. (ed) rational analysis for a problematic world. london: wiiey. ensslin, l., dutra, a. & ensslin, s. r. (2000). mcda: a constructivist approach to the management of human resources at a governmental agency. international transactions in operational research, 7, 79-100. doi: https://doi.org/10.1111/j.14753995.2000.tb00186.x ensslin, s.r., ensslin, l., back, f. & lacerda, r.t.o. (2013). improved decision aiding in human resource management a case using constructivist multi-criteria decision aiding, international journal of productivity and performance management, 62(7), 735–757. doi: https://doi.org/10.1108/ijppm-04-2012-0039 ensslin, l., montibeller neto, g. & noronha, s. m. (2001). apoio à decisão: metodologias para estruturação de problemas e avaliação multicritério de alternativas. florianópolis: insular. ensslin, l., giffhorn, e., ensslin, s. r., petri, s. m. & vianna, w. b. (2010). avaliação do desempenho de empresas terceirizadas com o uso da metodologia multicritério de apoio à decisão. construtivista. pesquisa operacional, 30(1), 125-152. doi: https://doi.org/10.1590/s0101-74382010000100007 ensslin, s.r., ensslin, l. & lacerda, r.t.o. (2011). a performance measurement framework in portfolio management, management decision, 49(4), 648-668. doi: https://doi.org/10.1108/00251741111126530 ensslin, l., dutra, a., ensslin, s. r., longaray, a.a. & dezem, v. (2016). modelo construtivista de apoio à gestão bancária. espacios, 37(9), 12. doi: https://doi.org/10.14488/enegep2017_tn_sto_243_408_31385 guimarães, j. l. s. & salomon, v. a. p. (2015). anp applied to the evaluation of performance indicators of reverse logistics in footwear industry. procedia computer science, 55, 139–148. doi: https://doi.org/10.1016/j.procs.2015.07.021 hanson, j.d., melnyk, s.a. & calantone, r.a. (2010). defining and measuring alignment in performance management. international journal of operations & http://onlinelibrary.wiley.com/doi/10.1111/j.1475-3995.2000.tb00186.x/full http://onlinelibrary.wiley.com/doi/10.1111/j.1475-3995.2000.tb00186.x/full ijahp article:pacheco, piratelli, silva, belderrain /the design of a performance measurement system for a fruit processing equipment manufacturer international journal of the analytic hierarchy process 352 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.640 production management, 31(10), 1089-1114. doi: https://doi.org/10.1108/01443571111172444 hernández, c.t., marins, f.a.s. & salomon, v.a.p. (2011) análise da importância dos indicadores de desempenho da logística reversa mediante a utilização do analytic network process. xliii simpósio de pesquisa operacional. ubatuba/sp. doi: https://doi.org/10.5151/marine-spolm2015-140144 ho, w. & ma, x. (2018). the state-of-the-art integrations and applications of the analytic hierarchy process. european journal of operational research, 267(2), 399–414. doi: https://doi.org/10.1016/j.ejor.2017.09.007 holmberg, s. (2000). a systems perspective on supply chain measurements. international journal of physical distribution & logistics management, 30(10), 847868. doi: https://doi.org/10.1108/09600030010351246 hu, y. c., wang, j-h. & wang, r-y. (2012). evaluating the performance of taiwan homestay using analytic network process. mathematical problems in engineering, 2012, 1-24. doi: https://doi.org/10.1155/2012/827193 kaplan, r. & norton, d. p. (1997). balanced scorecard: a estratégia em ação. rio de janeiro: campus. kravchenko, t. & seredenko, n. (2011) decision-making with modeling of problem situations using the analytic network hierarchy process. international journal of the analytic hierarchy process, 3(1), 28-45. doi: https://doi.org/10.13033/ijahp.v3i1.81 kucukaltan, b.; irani, z.; aktas, e. (2016). a decision support model for identification and prioritization of key performance indicators in the logistics industry. computers in human behavior, 65, 346–358. doi: https://doi.org/10.1016/j.chb.2016.08.045 madeira junior, a. g., gonçalves, t.j.m. & belderrain, m.c.n. (2011). estruturação do problema de avaliação da qualidade dos terminais de contêineres por meio de mapas cognitivos. sobrapo revista eletrônica pesquisa operacional para desenvolvimento. rio de janeiro, 3(3), 230-241, set. – dez., 2011 doi: https://doi.org/10.14488/16761901.v11i4.643 marins, f.a.s., pereira, m. s., belderrain, m.c.n. & urbina, l. m. s. (2010). métodos de tomada de decisão com múltiplos critérios aplicações na indústria aeroespacial. são paulo: edgard blucher ltda., p.248. marttunen, m., lienert, j. & belton, v. (2017). structuring problems for multi-criteria decision analysis in practice: a literature review of method combinations. european journal of operational research, 263, 1-17. doi: https://doi.org/10.1016/j.ejor.2017.04.041 ijahp article:pacheco, piratelli, silva, belderrain /the design of a performance measurement system for a fruit processing equipment manufacturer international journal of the analytic hierarchy process 353 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.640 mingers, j. & rosenhead, j. (2004). problem structuring methods in action. european journal of operational research, 152, 530-554. doi: https://doi.org/10.1016/s03050483(97)00018-2 mingers, j. & brocklesby, j. (1997). multimethodology: towards a framework for mixing methodologies. the international journal of management science, 25(5), 489-509. doi: https://doi.org/10.1016/s0305-0483(97)00018-2 mostafa, s., abdelhamid, t., chileshe, n. & dumrak, j. (2015). decision support model using anp to align leagile strategies to off-site manufacturing in australia, international journal of the analytic hierarchy process, 7(3), 374-404. doi: https://doi.org/10.13033/ijahp.v7i3.340 neely, a. (1999). the performance measurement revolution: why now and what next. international journal of operations & production management. v. 19. n. 2, p. 205-228. neely, a. (2005). the evolution of performance measurement research: developments in the last decade and a research agenda for the next. international journal of operations & production management, 25(12), 1264-1277. doi: https://doi.org/10.1108/01443579910247437 neely, a., gregory, m. & platts, k. (2005). performance measurement system design: developing a literature review and research agenda. international journal of operations & production management, 25(12), 1228-1263. doi: https://doi.org/10.1108/01443579510083622 neely, a., gregory, m. j. & platts, k. w. (1995). performance measurement system design – a literature review and research agenda. international journal of operations & production management, 15(4), 80-116. doi: https://doi.org/10.1108/01443579510083622 neely, a., richards,h., mills, j., platts, k. & bourne, m. (1997). designing performance measures: a structured approach. international journal of operations & production management, 17(11), 1131-1152. doi: https://doi.org/10.1108/01443579710177888 neely , a., mills, j., platts, k., richards,h., gregory, m., bourne, m. & kennerley, m. (2000). performance measurement system design: developing and testing a process based approach. international journal of operations & production management, 20(10), 11191145. doi: https://doi.org/10.1108/01443570010343708 neely, a., adams, c. & kennerley, m. (2002). the performance prism: the scorecard for measuring and managing business success. london, great britain: prentice hall, pearson education limited. piratelli, c.l. (2010). medidas de avaliação de desempenho de uma instituição de ensino superior: uma abordagem de pesquisa operacional. 300f. tese (doutorado em engenharia aeronáutica e mecânica. área produção) – instituto tecnológico de aeronáutica, são josé dos campos. doi: https://doi.org/10.14393/19834071.2014.26671 ijahp article:pacheco, piratelli, silva, belderrain /the design of a performance measurement system for a fruit processing equipment manufacturer international journal of the analytic hierarchy process 354 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.640 piratelli, c. l. & belderrain, m. c. n. (2010). supporting the design of a performance measurement system with the analytic network process. international journal of the analytic hierarchy process, 2(1), 44-65. doi: https://doi.org/10.13033/ijahp.v2i1.72 rentes, a.f., carpinetti, l.c.r. & van aken, e.m. (2002). measurement system development process: a pilot application and recommendations. in: performance measurement and management conference, world trade centre. boston: performance measurement association. rosa, f. s., ensslin, s. r., ensslin, l. & lunkes, r. j. (2012). environmental disclosure management: a constructivist case. management decision, 50(6), 1117-1136. doi: 10.1108/00251741211238364 rosenhead, j. (1989). rational analysis for a problematic world. london: wiiey. saaty, t. l. (1980). the analytic hierarchy process. new york: mcgraw-hill, 1980. saaty, t. l. (1999). fundamentals of the analytic network process. isahp1999, kobe, japan, august 12-14. saaty, t. & vargas, l. g. (2006). decision making with the analytic network process. berlin: springer science business media. saaty, t. l. (2008). the analytic hierarchy and analytic network measurement process: applications to decisions under risk. european journal of pure and applied mathematics, 1(1), 122–196. salgado, m. c. v., belderrain, m. c. n. & silva, a. c. s. (2009). avaliação do vôo tecnológico xvt02 do veículo lançador de satélites vls-1 por meio de decisão em grupo. journal of aerospace technology management, 1(1), 79-90. doi: https://doi.org/10.5028/jatm.2009.01017990 salomon, v. a.p. & montevecchi, j. (1998). método de análise em redes: sucessor do método de análise hierárquica. produto e produção, 2(3), 107-117. doi: https://doi.org/10.17771/pucrio.acad.29536 sobreira netto, f. (2007). medição de desempenho organizacional: um estudo das vantagens e desvantagens dos principais sistemas sob as óticas teórico-acadêmica e de práticas de mercado. in: xxxi encontro da anpad, rio de janeiro/ rj. doi: https://doi.org/10.5753/ersirj.2018.4656 westcombe, m. (2002). problem structuring: the process of soda modelling. in: workshop on facilitating hypertext-augmented collaborative modeling. college park: university of maryland. http://scholar.google.com.br/citations?view_op=view_citation&hl=pt-br&user=z5kmcr0aaaaj&citation_for_view=z5kmcr0aaaaj:d1gkvwhdpl0c http://scholar.google.com.br/citations?view_op=view_citation&hl=pt-br&user=z5kmcr0aaaaj&citation_for_view=z5kmcr0aaaaj:d1gkvwhdpl0c a style guide for papers presented to the international journal of the analytic hierarchy process ijahp article: jain, rao/ application of ahp tool for decision making of choice of technology for extraction of anti-cancer bioactive compounds of plant origin international journal of the analytic hierarchy process 3 vol.5 issue 1 2013 issn 1936-6744 application of ahp tool for decision making of choice of technology for extraction of anticancer bioactive compounds of plant origin rahi jain centre for technology alternatives for rural areas (ctara) indian institute of technology bombay (iitb) mumbai, maharashtra, india e-mail: 114350002@iitb.ac.in bakul rao centre for technology alternatives for rural areas (ctara) indian institute of technology bombay (iitb) mumbai, maharashtra, india e-mail: bakulrao@iitb.ac.in abstract demand for medicinal plants and their exploitation around the world has prompted international agencies like the world health organization (who) and national health departments like ayush to focus on their sustainable utilization. however, lack of any decision-making methodology makes it difficult for the rural entrepreneur to setup a business. this paper has developed a multi-criteria decision making template, using the analytic hierarchy process (ahp), of ranking the best technology for extraction of nine medicinal plants which grow in western ghats of india. these plants have more than 20% subsidy in cultivation under nmmp and have an anticancer property. the technologies selected for ahp were agitation/centrifugation, cold solvent extraction, reflux/soxhlet extraction, cold percolation, microwave assisted extraction, sonication and hot solvent extraction. the ability of the tool to provide selection of the extraction technology with process flexibility like criteria selection, technology selection and criteria weightages allows its use by rural entrepreneurs, technology facilitators, rural entrepreneur proposal evaluators and policy makers. keywords: extraction technology selection, medicinal plants, analytic hierarchy process, rural entrepreneur 1. introduction the demand for medicinal plants(mp) is increasing (cragg & newman, 2005; world health organization, 2000) due to their usage both as a source of active ingredients for modern medicine (rajasekharan, 2006) as well as for producing herbal products while providing a viable business opportunity to entrepreneurs. business opportunities in processing mp can help small scale collectors and cultivators get on the industrialization ladder and pursue their livelihood as an industrialist. while the large scale nutraceuticals/pharmaceuticals have the resources to carry out their own research towards rob typewritten text http://dx.doi.org/10.13033/ijahp.v5i1.153 ijahp article: jain, rao/ application of ahp tool for decision making of choice of technology for extraction of anti-cancer bioactive compounds of plant origin international journal of the analytic hierarchy process 4 vol.5 issue 1 2013 issn 1936-6744 technical know-how for industrial set-up, the real challenge exists for the small scale or rural entrepreneurs due to their limited resources and capacity. the small-scale or rural entrepreneurs’ must mostly rely on knowledge in the public domain. public domain knowledge on natural products has focused mainly on developing/validating products (choochote et al., 2004; malik et al., 2007; saxena et al., 2007) and developing/optimizing processing technology (bothiraja, joshi, dama, & pawar, 2011; jyothi d khanam s sultana r, 2010). in the case of natural products, “processing technologies” depend upon the mp to be processed, and many times availability of mp for processing depends upon the suitability of the ecosystem /geographical area, current government policies and the market. thus, the real challenge for the small scale/rural entrepreneur is choosing the technology which can process multiple mp so as to make their business viable. this creates a need for a multi-criteria decision making tool to select the best “processing technology” for their business. india, with plans to capitalize on the global mp trade, launched the “national mission on medicinal plants (nmmp)” in 2008.one of the goals was to promote processing by providing financial, technological and market support. however, india’s efforts to enable the rural entrepreneur to process mp is inadequate as nmmp is ‘technology silent’, i.e. it does not provide any information about “processing technologies” that are compatible with different medicinal plants, or the decision-making process for selecting a “processing technology”(national medicinal plants board, 2008).thus, the need for a standardized decision-making process is felt at three different levels: • at the rural entrepreneur level: the task of selecting the appropriate “processing technology” is challenging for the rural entrepreneur with limited resources and technical know-how. • at the nmmp evaluator level: nmmp needs to evaluate the project proposals sent by the different entrepreneurs for financial support. • at the nmmp board level: better trade requires coordination between national and global systems which can be done through nmmp support to rural entrepreneurs in decision making as nmmp can provide much bigger picture to the rural entrepreneur with help of its resources. there are several challenges from the rural entrepreneur’s perspective in using the decision-making process to make their business viable with or without government support. firstly, the amount of information which the entrepreneur needs to process is very high for different parameters like medicinal plants (2400 in indian system of medicine (ism) (ved & goraya, 2008)), extraction technologies (more than 16 (handa, khanuja, longo, & rakesh, 2008; xiang, jianzhong, jing, & yundong, 2011)), bioactive compounds with medicinal property (thousands) and medicinal plant parts. secondly, the time and resources which the entrepreneur may have to invest to acquire all this information could be a limiting factor especially with knowledge and resource constraints. thus, processing this information in order to determine an appropriate “extraction technology/alternative” for each medicinal plant, plant part and bioactive compound can be done by the national level agencies. once this has been done the rural entrepreneur can use the information to identify the best technology based on the ijahp article: jain, rao/ application of ahp tool for decision making of choice of technology for extraction of anti-cancer bioactive compounds of plant origin international journal of the analytic hierarchy process 5 vol.5 issue 1 2013 issn 1936-6744 medicinal plants of his/her choice. this study focuses on developing a simple decisionmaking template which can provide guidance to a rural entrepreneur for making decisions about an “extraction technology/alternative” to enable a viable business model. the analytic hierarchy process (ahp),a simple and easy multi-criteria decision making tool, developed by the mathematician thomas saaty (saaty, 1980), is used for selection of alternatives in the current study. 2. methodology the decision-making template is developed keeping in view the major decisions which the rural entrepreneur has to make while deciding upon the extraction technology/alternative for his/her business. first, the rural entrepreneur must decide upon the medicinal plants whose processing can be done with commercial viability based on the geographical area of the processing unit, source of medicinal plants, markets for the medicinal plant and government policies. second, the entrepreneur must select the extraction technology/alternative for the business, which consists of two sub-tasks namely identification of extraction technologies/alternatives and rating of the extraction technologies/alternatives. these choices are converted into a decision-making template as shown in figure 1. ijahp article: jain, rao/ application of ahp tool for decision making of choice of technology for extraction of anti-cancer bioactive compounds of plant origin international journal of the analytic hierarchy process 6 vol.5 issue 1 2013 issn 1936-6744 figure 1 methodology e xt ra ct io n t e ch n o lo g y/ a lte rn a tiv e s e le ct io n ahp r(j) = ∑p(i,j)w(i) technology selection attribute value for technology attribute weightages technology identification attribute identification extraction technology for target compound s t a g e 1 t e ch n o lo g y id e n tif ic a tio n s t a g e 2 a n a ly tic a l h ie ra rc h ia l p ro ce ss identification of geographical area for processing unit identifyinh the sources of medicinal plants identifying mp with government policies support market of mp product selection of mp extraction technology for selected mp discard mp with no extraction technology m p s e le ct io n ijahp article: jain, rao/ application of ahp tool for decision making of choice of technology for extraction of anti-cancer bioactive compounds of plant origin international journal of the analytic hierarchy process 7 vol.5 issue 1 2013 issn 1936-6744 2.1 selection of medicinal plants in the current study, the method of selecting medicinal plants has been a process of elimination based on geographical area of the processing unit, source of medicinal plants, market for the medicinal plant and government policies to select only those plants which satisfy all the criteria. the geographical area of processing and source of medicinal plants is the western ghats (wgs) of india, which is spread across 5 states (22 districts) (kholkute, 2008) which covers around 6% of india’s total land area, and contains 30% of indian flora, fish, bird and mammal species making it one of the 34 global biodiversity hotspots identified for conservation (critical ecosystem partnership fund (cepf), 2007). the source of medicinal plants and the geographical area of processing are kept the same as the entrepreneurs considered in this study are small scale cultivators who will be using the raw material cultivated on their farms. a survey by the indian council of medical research provided a shortlist of500 medicinal plants in wg with the capability to treat 200 diseases(kholkute, 2008).a screening for government promotion and support to medicinal plants through nmmp (which is giving a subsidy from 20% to 75% on medicinal plant cultivation) resulted in 34 plants as shown in table 1. further screening using the market for medicinal plants is based on the medicinal property possessed by the plants. in the current study, an anti-cancer property is selected, because in the noncommunicable disease category, cancer has been reported as the leading cause of death after cardiovascular disease (world health organization (who), 2008, 2011). ten medicinal plants namely alstonia scholaris r.br., coscinum fenestratum (gertn) colebr., gloriosa superba linn., plumbago zeylanica linn., smilax china linn., terminalia arjuna (roxb.) wt.&arn., terminalia chebula retz., vitex nigundo linn., withania somnifera (linn.) dunaj.andwoodfordia fruticosa kurz. (shown in table1 in bold and italics))were selected for this study. these plants satisfied the criteria of growing naturally in wg, getting more than 20% subsidy in nmmp and being scientifically verified for an anti-cancer property with a specific anti-cancer compound. table 1 medicinal plants mentioned in nmmp list which grow in wgs, with those plants given > 20% subsidy and having an anticancer property (given in bold and italics) wg plants mentioned in nmmp acorus calamus linn. alstonia scholaris r.br. andrographis paniculata (burmi) wall ex. nees aloe vera (linn.) burn. asparagus racemosus wild. coscinum fenestratum (gertn) colebr. aegle marmelos (linn) corr. azadirachta indica a. juss cryptolepis buchanani roem & schult embelia ribes burm. f. boerhaavia diffusa linn. pterocarpus marsupium roxb. garcinia indica choisy centella asiatica (linn.) urban rauwolfia serpentina benth. ex kurz gloriosa superba linn. dioscorea bulbifera linn. saraca asoca (roxb.) de wilde* mesua ferrea linn. gymnema sylvestre r. br. terminalia arjuna (roxb.) wt. & arn. piper longum linn. hemidesmus indicus r.br. terminalia bellirica gaertn.* santalum album linn. ocimum sanctum linn. tylophora asthmatica (linn. f.) wight & am. smilax china linn.** plumbago zeylanica linn. withania somnifera (linn.) dunaj. solanum nigrum linn. terminalia chebula retz. vitex nigundo linn. woodfordia fruticosa kurz. ijahp article: jain, rao/ application of ahp tool for decision making of choice of technology for extraction of anti-cancer bioactive compounds of plant origin international journal of the analytic hierarchy process 8 vol.5 issue 1 2013 issn 1936-6744 (note: medicinal plants in bold and italics gets more than 20% subsidy and have anti-cancer property, *: literature have shown its anticancer property but no specific compound has been reported and hence not considered for this study, **: lack of adequate literature to confirm the extraction technology) 2.2 selection of extraction technology/alternative most of the techniques used in modern herbal processing make use of solvent extraction technology, which is based on the principle of distribution of one or more components between two immiscible or almost immiscible liquids. in some cases, the techniques make use of a single solvent and only compounds soluble in that solvent get extracted (bharathi, philomina, & chakkaravarthi, 2006; kannan et al., 2007). these technologies can be used either alone or in combination. for example, surfactant based extraction has been used along with microwave assisted extraction (mae)(chen, yuchun, & huizhou, 2007), sonication (he et al., 2005; xiang et al., 2011), pressurized liquid extraction (ple)(choi, chan, leung, & huie, 2003; eng, heng, & ong, 2007) and agitation (he et al., 2005). soxhlet extraction has been used along with microwave (garcı́a-ayuso & castro, 1999) as well as sonication (luque-garcı́a & castro, 2004). various extraction technologies under research or commercial use currently include agitation/centrifugation, cold solvent extraction, hot solvent extraction, cold percolation, reflux, ple, super-critical fluid extraction (scfe), sonication, mae, steam distillation, hydro distillation, counter-current extraction (cce), solid-phase extraction (spe), enzymatic extraction, decoction (handa, khanuja, longo, & rakesh, 2008) and surfactant based extraction (xiang, jianzhong, jing, & yundong, 2011). they form the global set of extraction alternatives for this study. the entire extraction alternative selection process has two stages. stage 1 is the selection of the relevant extraction technologies from the global set of extraction alternatives using preliminary criteria as discussed in this section. stage 2 involves final selection of an extraction technology/alternative using ahp. 2.2.1 stage 1: identification of extraction alternatives for medicinal plants in the current study, extraction alternatives with stand-alone use have been considered for analysis from the global set of extraction alternatives. this is based on the criteria that the alternatives used should have been reported for the extraction of an anti-cancer compound from at least one of the medicinal plants selected in section 2.1. vice versa, the medicinal plant needs to have been reported to have been used at least one of the technologies; otherwise as shown in table 1, the mp is dropped from further study. hence, in the current study, only nine mp and seven extraction technologies/alternatives are selected (see table 2). table 2 extraction technology with attributes et raw material characteristics operating characteristics managerial characteristics plant cc pp t tbcp p s t ief aue sa ta m s/t tc agitation (a1) i c.f, g.s iqa, paa plant, seed, stem, tuber low to v h tl, mts, hts, ts, 1 p, np 190 to 12500 man,e lec bc, c, ph 4 to 161 easy easy vb low ijahp article: jain, rao/ application of ahp tool for decision making of choice of technology for extraction of anti-cancer bioactive compounds of plant origin international journal of the analytic hierarchy process 9 vol.5 issue 1 2013 issn 1936-6744 cold percolati on (a2) ii t.a, w.s, c.f triterp ene , sl, iqa bark, root, stem low to normal tl, mts 1 p, np 3400 no bc, c, ph 167 med easy vb med cold solvent (a3) iii p.z, t.a, v.n, w.s, g.s, t.c, a.s, c.f, triterp ene, pa, sl, ia, iqa, paa, lignan, quinon e bark, leaf, root, seed, stem, tuber, plant, low to normal tl, mts 1 p, np 960 to 7200 no bc, c, ph 2 to 54 ve ve vb vl mae (a4) iv w.s sl aerial part, leaf vh hts 1 p, np 2 to 2.5 elec c, ph 17 to 20 med tough med high reflux (a5) v p.z, g.s, v.n, a.s, w.s, c.f quinon e, lignan, sl, ia, iqa, paa bark, leaf, plant, root, stem, tuber high to vh ts, hts 1 p, np 120 to 4300 elec, ther no 1.5 to 200 med med basic med sonicatio n (a6) vi a.s ia bark low to vh tl, mts, ts, hts 1 p, np 540 elec bc, c, ph 32 toug h tough med vh hot solvent (a7) v t.a, w.f, c.f triterp ene, pa, iqa leaf, stem stem high to vh ts, hts 1 p, np 120 elec, ther bc, c, ph 5.1 easy med basic low note: i= (bharathi et al., 2006; chomnawang, trinapakul, & gritsanapan, 2009; kannan et al., 2007; kavina, gopi, & panneerselvam, 2011; rojsanga & gritsanapan, 2005; rojsanga, gritsanapan, & suntornsuk, 2006; tungpradit, sinchaikul, phutrakul, wongkham, & chen, 2011), ii= (malik et al., 2007; rojsanga et al., 2006; upadhyay, pandey, jha, singh, & pandey, 2001), iii= (bothiraja, joshi, dama, & pawar, 2011; chitra, sujatha, polisetti, karri, & reddy, 2011; choochote et al., 2004; dalavayi, kulkarni, itikala, & itikala, 2006; jayaprakasam & nair, 2003; lin et al., 1990; macabeo et al., 2005; malhotra, taneja, & dhar, 1989; malik et al., 2007; r. k. pawar, shivani, singh, & sharma, 2010; r. s. pawar & bhutani, 2005; pinho et al., 1992; saxena et al., 2007; d. v. singh, verma, singh, & gupta, 2002; unnikrishnan, raja, & balachandran, 2008; yibchok-anun, jittaprasatsin, somtir, bunlunara, & adisakwattana, 2009; zhou et al., 2009), iv= (jyothi d khanam s sultana r, 2010; jyothi, khanam, & sultana, 2010; mirzajani, ghassempour, jalaliheravi, & mirjalili, 2010), v= (bharathi et al., 2006; britto & sujin, 2012; deevanhxay et al., 2009; devi, utsumi, takata, & takeda, 2008; jagetia & baliga, 2005; jeyachandran, mahesh, cindrella, sudhakar, & pazhanichamy, 2009; jyothi d khanam s sultana r, 2010; jyothi et al., 2010; rojsanga et al., 2006; tran et al., 2003; venukumar, 2004; zhou et al., 2005), vi= (lee et al., 2012), vii= (kadota et al., 1990; moulisha, kumar, & kanti, 2010; stenhouse, 1867), a.s= alstonia scholaris r.br. aue= ability to use enhancers bc= biochemical c.f= coscinum fenestratum (gertn) colebr. c= chemical cc=compound class elec= electrical et=extraction technology g.s= gloriosa superba linn. hts= hyper thermostable ia= indole alkaloid ief=input energy form ijahp article: jain, rao/ application of ahp tool for decision making of choice of technology for extraction of anti-cancer bioactive compounds of plant origin international journal of the analytic hierarchy process 10 vol.5 issue 1 2013 issn 1936-6744 iqa= isoquinoline alkaloid m= maintenance man= manual mts= mild thermostable np= non polar p.z= plumbago zeylanica linn. p= polar p= pressure applied (atm) pa = phenolic acid paa= phenyl alkylamines pp= plant part s/t= skills/training s= solvents sa= solvent amount (ml/gm of solid sample) sl= steriodal lactone t.a= terminalia arjuna (roxb.) wt. & arn. t.c= terminalia chebula retz. t= temperature in °c t= time in minutes ta= technology availability tbcp=temperature based compound property tc=technology cost ther= thermal tl= thermolabile ts= thermostable v.n= vitex nigundo linn. vb= very basic ve= very easy vh= very high vl= very low w.f= woodfordia fruticosa kurz. w.s= withania somnifera (linn.) dunaj. 2.2.2 stage 2: analytical hierarchy process (ahp) ahp has the following three steps as shown in stage 2 of figure 1: • performing pair-wise comparisons of the criteria and providing weightages forthe criteria for concerned perspective. • prioritization of technologies/alternatives is given for each of the criteria selected. • finally, summation of the product of the criteria and technology/alternative weightages to arrive at best alternative. ahp allows the user to use both the quantitative and qualitative data with consideration for even the subjective aspect of the decision like intuition and personal experiences (montevechi et al., 2010; palcic & lalic, 2009). it has beenused for selection of alternatives for various purposes like: • brand selection (ultrasonic scanning machine (montevechi, guimaraes, oliveira, & friend, 2010), neonatal ventilators (sloane, liberatore, nydick, luo, & chung, 2003)), • site selection and allocation (landfill site (hasan, tetsuo, & islam, 2009; javaheri, nasrabadi, jafarian, rowshan, & khoshnam, 2006; yahaya, ilori, whanda, & edicha, 2010), sustainable coastal tourism (abed, monavari, karbasi, farshchi, & abedi, 2011), railway station (mohajeri & amin, 2010)), • technology selection (waste water treatment for electroplating industry (dabaghian, hashemi, ebadi, & maknoon, 2008), solar thermal collection technology (nixon, dey, & davies, 2010), municipal solid waste management (thampi & rao, 2012)), • policy making (solar energy technologies utilization (elkarmi & mustafa, 1993), energy conservation utilization (kablan, 2004)), • production planning ( integrated production planning considering manufacturing partners (jung, 2011), total productive maintenance justification (kodali & chandra, 2001), push, pull and hybrid push-pull systems classification (razmi, rahnejat, & khan, 2005)), and ijahp article: jain, rao/ application of ahp tool for decision making of choice of technology for extraction of anti-cancer bioactive compounds of plant origin international journal of the analytic hierarchy process 11 vol.5 issue 1 2013 issn 1936-6744 • treatment alternatives (upper limbs treatment for persons with tetraplegia (hummel, snoek, van til, van rossum, & ijzermann, 2005), pharyngitis management in adults (s. singh, dolan, & centor, 2006)). a java-based in-house ahp software developed at iit-bombay which allows the user to provide qualitative and quantitative criteria for analysis has been used for this study (mahajan, ramakrishnan, & date, 2008).the criteria can be benefit or cost based depending on the user’s perspective and accordingly, the benefits criteria value should be maximized and the costs criteria value should be minimized (nijkamp & delft, 1977). the qualitative criteria analysis has been done by performing pair-wise comparisons of alternatives, and giving priority based on higher benefits or minimum costs. this analysis solves the problem of benefit-cost integration. quantitative criteria have been classified as benefits and costs, and the benefit-cost integration problem has been solved by the inhouse ahp software by converting cost into benefit1. 2.2.2.1 identification of criteria for selection the selected extraction alternatives need to be compared using common criteria (good mix of qualitative and quantitative) to predict an appropriate alternative. the commercial application of an extraction technology needs to consider three main decision criterion characteristics: raw material characteristics (rmc), technical characteristics (tc) and managerial/feasibility characteristics (mc). each of these decision criterion characteristics have criteria, a total of 14 have been identified (table 3). the rural entrepreneurs’ need to evaluate alternatives using those 14 criteria requires a strategy which is simple enough for them to understand, however this kind of strategy may be too laborious and time consuming. this leads to the use of two different strategies for this study namely single level hierarchy (figure 2) and two level hierarchy (figure 3). single level hierarchy is a simple strategy, which could be easily understood by the rural entrepreneur with knowledge constraints, as all the 14 criteria are taken simultaneously. two level hierarchy is less laborious and time consuming, and can be used by people with a better understanding of the logic. two level hierarchy uses three criterion characteristics at the first level followed by segregation of the 14 criteria under them. the comparison of the results of these two strategies can help determine the relative change in the ranking of the most appropriate alternative as well as provide the user the option of using a strategy as per his/her competence. 1the java tool normalization scheme developed in (raju, rangaraj, & date, 1995) for conversion of cost to benefit using following normalized equations: benefit criteria: 𝑝𝑖𝑗 = 𝑡𝑖𝑗−𝑡𝑖 𝑚𝑖𝑛 𝑡𝑖 𝑚𝑎𝑥−𝑡𝑖 𝑚𝑖𝑛 (1) cost criteria: 𝑝𝑖𝑗 = 𝑡𝑖 𝑚𝑎𝑥−𝑡𝑖𝑗 𝑡𝑖 𝑚𝑎𝑥−𝑡𝑖 𝑚𝑖𝑛 (2) where, tijis the attribute value of the jth alternative with respect to the ith attribute and 𝑡𝑖 𝑚𝑎𝑥 and 𝑡𝑖 𝑚𝑖𝑛 are the absolute maximum and minimum values among all the alternatives for the ith attribute. these equations have been directly taken from. ijahp article: jain, rao/ application of ahp tool for decision making of choice of technology for extraction of anti-cancer bioactive compounds of plant origin international journal of the analytic hierarchy process 12 vol.5 issue 1 2013 issn 1936-6744 table 3 list of criterion characteristics and criteria selected for ahp with single or two-level hierarchy criterion characteristics criterion name description criterion type cost/ benefit raw material characteristics (rmc) c1 number of plants it is the raw material characteristic of the technology which determines the number of plants among the selected for which the technology has been used. quantitative benefit c2 number of compound classes it is the raw material characteristic of the technology which determines the number of compound classes among the selected classes for which the technology has been used. quantitative benefit c3 number of plant parts it is the raw material characteristic of the technology which determines the number of different types of plant parts of the selected medicinal plants for which the technology has been used. quantitative benefit technical characteristics (tc) c4 temperature range it is the operating characteristic which determines the temperature range and the temperature specific compounds which can be extracted in that range for the extraction technology. qualitative benefit c5 pressure application (atm) it is operating characteristics which determine the pressure which need to be applied for the working of the extraction technology. quantitative cost c6 type of solvents it is the operating characteristics which determines the number of different solvents which can be used by the extraction technology quantitative benefit c7 minimum time (min) it is the operating characteristic which determines the minimum time taken by the extraction technology for the selected plants and compound class. quantitative cost c8 input energy form it is the operating characteristic which determines the different forms of external energy used by the extraction technology. qualitative benefit c9 use of enhancers it is the operating characteristic which determines the number of different types of enhancers which can be used by the extraction technology. quantitative benefit c10 minimum solvent (ml/gm) it is the operating characteristic which determines the minimum solvent taken by the extraction technology for the selected plants and compound class. quantitative cost managerial characteristics (mc) c11 technology availability it is the ease with which the technology or makeshift technology which can be acquired. qualitative benefit c12 maintenance it is the ease of maintenance of extraction technology which includes financial and nonfinancial costs and benefits. qualitative benefit c13 skills/ training it is the skill/training required in order to use the extraction technology. qualitative cost c14 technology cost it is cost of the technology instrument. qualitative cost ijahp article: jain, rao/ application of ahp tool for decision making of choice of technology for extraction of anti-cancer bioactive compounds of plant origin international journal of the analytic hierarchy process 13 vol.5 issue 1 2013 issn 1936-6744 common raw material characteristics mentioned by various researchers are medicinal plant species, target compound class and medicinal plant part as shown in table 2. in this study, these raw material characteristics have been used for designing three criteria: number of plants (c1), number compound classes (c2) and number of medicinal plant parts (c3). some of the criteria important for the extraction of phyto-chemicals include plant material, extraction time, ph, temperature, solvent to solid ratio, extraction procedure and extraction solvent as discussed by tiwari, kumar, kaur, kaur and kaur (2011). many research papers also talk about the use of certain enhancers like chemicals, bio-chemicals and ph with certain extraction technologies to produce better results (table 2). another technical characteristic which can be implicitly identified from research papers is the energy (such as manual, electrical or thermal) used to run the extraction technology. these parameters, which reflect the adaptability of the extraction technology, have been used in the current study and criteria have been developed for technical characteristics. the technical characteristics criteria used are temperature range (c4), pressure application (c5), type of solvents (c6), minimum time (c7), input energy form (c8), use of enhancers (c9) and minimum solvent (c10). managerial characteristics, which influence the technology choice, are technology availability (c11), maintenance (c12), skills/training (c13) and technology cost (c14). quantitative criteria can be beneficial or detrimental depending on the perspective of the rural entrepreneur. an increase in the benefit score increases the score of the technology, while an increase in the cost score decreases the score of the technology. raw material characteristics based criteria (c1-c3) are quantitative criteria providing benefits to the technology user. criteria based on technology operating parameters (c4-c10) are a mix of quantitative and qualitative criteria. the quantitative criteria, pressure (c5), time (c7) and solvent amount (c10) are crucial as they indirectly affect the cost of the technology while solvent type (c6) and enhancers (c9) determine the robustness. temperature range (c4) and input energy form (c8) are qualitative criteria which contribute to the robustness and cost of the technology. management characteristics based criteria (c11c14) are qualitative in nature due to lack of quantitative data. they determine the accessibility of the technology (c11) and cost of the technology as maintenance (c12), labor skills (c13) and initial investment (c14). in the given set of criteria, the aspects of environment and employment are not included, since the existing selection decisions of a rural entrepreneur usually are not focused on the impact on the environment and employment generation. also, certain operating parameters like energy form, pressure applied, operating time and solvent amount can indirectly indicate the relative environmental impact of the technologies. 2.2.2.2 criteria weightages the perspective of the rural entrepreneur determines relative criterion scores. as per the perspective adopted for the ahp process, pair-wise comparison amongst the criteria is performed by giving the relative weightage on the numerical scale of 1 to 9 to obtain the global/local weightages of the criteria for the single level hierarchy (slh) (figure 2)given in table 4 .in the case of two level hierarchy (figure 3), the criteria local weights were determined by performing the pairwise comparison for criteria c1-c3, c4-c10 and c11-c14 (as shown in dark bordered boxes in table 4). further, the criteria local weights ijahp article: jain, rao/ application of ahp tool for decision making of choice of technology for extraction of anti-cancer bioactive compounds of plant origin international journal of the analytic hierarchy process 14 vol.5 issue 1 2013 issn 1936-6744 in table 4 were used to determine the global weights for criteria in two-level hierarchy as shown in table 5. in the present case, criterion characteristics and criteria weightages were determined during a project to identify rural livelihood potential. this activity tried to imitate the rural entrepreneur’s access to information and expertise, and hence has been based on informal interactions. the rural people identified for interactions were four small business holders and farmers with knowledge of medicinal plants and an acumen/potential interest for a medicinal plant based business. the local expertise on running a business and identifying plants for agriculture was used for developing preferences. the weightages to these preferences was given by the researchers based on their personnel experiences; as a result the problem of aggregation of the individual judgments did not arise. the consistency index for single and two level ahp are lower because the weightages were calculated in reference to a single user (rural entrepreneur). in the case of the rural entrepreneur initiative, the commercial interest is assumed to govern the technology selection making managerial characteristics (mc) (c11-c14) the highest priority. these characteristics determine ease of managing a business. cost of technology (c14) is most important as it affects the initial investment. technology availability (c11) is very important as information and supplier accessibility can be a problem in rural areas due to lack of industrialization. skills (c13) are important due to investment of time and money required if skill development/procurement needs to be performed. maintenance (c12) is important as it affects operating cost and breakdown time of the technology. raw material characteristics (rmc) are given higher priority than technological characteristics (tc) since they determine the vagaries for the success of the rural entrepreneur initiative rather than the technological characteristics which have already been scientifically validated. raw material characteristics decide the use of the technology for the desired medicinal plants (c1). they are very important from the rural entrepreneur’s perspective as an opportunity provided by the diverse medicinal plants availability in a region can be used only with technology which can use those plants. similarly, diverse compound class (c2) and plant parts (c3) utilization ability of technology provides the opportunity to produce multiple products and use multiple plant parts from same plant. ijahp article: jain, rao/ application of ahp tool for decision making of choice of technology for extraction of anti-cancer bioactive compounds of plant origin international journal of the analytic hierarchy process 15 vol.5 issue 1 2013 issn 1936-6744 figure 2 single-level hierarchy table 4 pairwise comparison of criteria (c1-c14) for single-level hierarchy (slh)(additional usage of pairwise comparison of criteria in black boxes i.e., c1-c3, c4-c10 and c11c14 used for obtaining local weights for two-level hierarchy (tlh)). perspective=rural entrepreneur criteria c1 c2 c3 c4 c5 c6 c7 c8 c9 c10 c11 c12 c13 c14 slh wt. (local/global tlh wt. (local) c1 1.0 0.9 1.0 2.0 3.0 6.0 1.2 1.5 6.0 1.0 0.8 1.0 0.9 0.7 0.08 0.321 c2 1.2 1.0 1.2 2.3 3.5 7.0 1.4 1.8 7.0 1.2 0.9 1.2 1.0 0.8 0.10 0.366 c3 1.0 0.9 1.0 2.0 3.0 6.0 1.2 1.5 6.0 1.0 0.8 1.0 0.9 0.7 0.08 0.313 c4 0.5 0.4 0.5 1.0 1.5 3.0 0.6 0.8 3.0 0.5 0.4 0.5 0.4 0.3 0.04 0.139 c5 0.3 0.3 0.3 0.7 1.0 2.0 0.4 0.5 2.0 0.3 0.3 0.3 0.3 0.2 0.03 0.091 c6 0.2 0.1 0.2 0.3 0.5 1.0 0.2 0.3 1.0 0.2 0.1 0.2 0.1 0.1 0.01 0.049 c7 0.8 0.7 0.8 1.7 2.5 5.0 1.0 1.3 5.0 0.8 0.6 0.8 0.7 0.6 0.07 0.229 c8 0.7 0.6 0.7 1.3 2.0 4.0 0.8 1.0 4.0 0.7 0.5 0.7 0.6 0.4 0.06 0.179 c9 0.2 0.1 0.2 0.3 0.5 1.0 0.2 0.3 1.0 0.2 0.1 0.2 0.1 0.1 0.01 0.047 c10 1.0 0.9 1.0 2.0 3.0 5.9 1.2 1.5 5.9 1.0 0.8 1.0 0.9 0.7 0.08 0.266 c11 1.3 1.1 1.3 2.7 4.0 8.0 1.6 2.0 8.0 1.3 1.0 1.3 1.1 0.9 0.11 0.264 c12 1.0 0.9 1.0 2.0 3.0 6.0 1.2 1.5 6.0 1.0 0.8 1.0 0.9 0.7 0.08 0.207 c13 1.2 1.0 1.2 2.3 3.5 7.0 1.4 1.8 7.0 1.2 0.9 1.2 1.0 0.8 0.10 0.235 c14 1.5 1.3 1.5 3.0 4.5 9.0 1.8 2.3 9.0 1.5 1.1 1.5 1.3 1.0 0.13 0.294 c.i.=5.081e-6 one tier hierarchy criteria a1 number of plants criteria a2 number of compound classes criteria a3 number of plant part criteria a4 temperature range criteria a5 pressure application criteria a6 type of solvents criteria a7 minimum time criteria a8 input energy form criteria a9 use of enhancers criteria a10 minimum solvent criteria a11 technology availability criteria a12 maintenance criteria a13 skills/training criteria a14 technology cost ijahp article: jain, rao/ application of ahp tool for decision making of choice of technology for extraction of anti-cancer bioactive compounds of plant origin international journal of the analytic hierarchy process 16 vol.5 issue 1 2013 issn 1936-6744 in the case of technology criteria, higher priority is given to the cost related criteria (c4, c5, c7, c8 and c10) as compared to the benefit criteria as these affect the cost of the technology, operating cost and additional investment which need to be made for using that technology. among the cost related criteria, criteria (c7-c10) are given higher priority. solvent amount (c10) plays a critical role as it affects both the operating cost, handling and environmental and water impact of technology on the area. time (c7) will affect the operating cost, net working capital and process cycle as longer time means higher risk. energy form (c8) will affect operating cost and initial investment. figure 3 two-level hierarchy two tier hierarchy criteria a1 number of plants criteria a2 number of compound classes criteria a3 number of plant part criteria a4 temperature range criteria a5 pressure application criteria a6 type of solvents criteria a7 minimum time criteria a8 input energy form criteria a9 use of enhancers criteria a10 minimum solvent criteria a11 technology availability criteria a12 maintenance criteria a13 skills/training criteria a14 technology cost raw material characteristics managerial characteristics technical characteristics ijahp article: jain, rao/ application of ahp tool for decision making of choice of technology for extraction of anti-cancer bioactive compounds of plant origin international journal of the analytic hierarchy process 17 vol.5 issue 1 2013 issn 1936-6744 table 5 pair-wise final normalized criterion characteristics and criteria weightages for two-level hierarchy criterion characteristics local weight ci criteria local weight ci global weight rmc 0.297 8.9e-03 c1 0.321 6.3e-04 0.095 c2 0.366 0.109 c3 0.313 0.093 tc 0.163 c4 0.139 1.4e-03 0.023 c5 0.091 0.015 c6 0.049 0.008 c7 0.229 0.037 c8 0.179 0.029 c9 0.047 0.008 c10 0.266 0.044 mc 0.540 c11 0.264 1.2e-04 0.142 c12 0.207 0.112 c13 0.235 0.127 c14 0.294 0.159 2.2.2.3 criteria scores for technology the quantitative score for the criteria as shown in table 6 has been selected through a detailed literature survey as indicated in table 2. in the case of the benefit related criteria, number of plants (c1), number of compound classes (c2) and number of plant parts (c3) numbers are allocated based on the amount found for each criterion in the literature. for example, reflux technology (a5) was used for six medicinal plants namely plumbago zeylanicalinn.,vitex nigundo linn., withania somnifera (linn.) dunaj.,alstonia scholaris r.br., coscinum fenestratum (gertn) colebr. andgloriosa superba linn. among the eight desired medicinal plants, so the number of plants (c1) criterion score for reflux technology (a5) is six. criterion solvent type (c6) and use of enhancers (c9) scores were calculated based on the number of solvent type (polar and non-polar) and the number of enhancers (chemical, biochemical and ph) respectively which could be used with technology based on its extraction procedure. for example, in the case of reflux (a5), addition of surfactants or enzymes causing a change in solvent ph will not have an effect on extraction as the solvent is first evaporated and during condensation gets mixed with the sample. in the case of microwave assisted extraction very high temperatures can denature enzymes. in the case of cost related criteria, pressure (c5), time (c7) and solvent amount (c10), different researchers have used different operating conditions for the technology of choice in their research. for the current study, to estimate the minimum achievable cost, the minimum score for the technology is taken. for example, for reflux technology (a5) different researchers have reported different scores for time (c7) criterion ranging from 120 to 4320 minutes (bharathi et al., 2006; britto & sujin, 2012; deevanhxay et al., 2009; jeyachandran et al., 2009; jyothi d khanam s sultana r, 2010; jyothi et al., 2010; rojsanga et al., 2006; tran et al., 2003; zhou et al., 2005) for extraction. for this study, to arrive at a minimum achievable cost, the lowest score is selected. the time (c7) criterion for hot solvent extraction (a7) is not available in the literature for the desired ijahp article: jain, rao/ application of ahp tool for decision making of choice of technology for extraction of anti-cancer bioactive compounds of plant origin international journal of the analytic hierarchy process 18 vol.5 issue 1 2013 issn 1936-6744 medicinal plants. reflux (a5) and hot solvent extraction (a7) work on a similar principle of extracting compounds from a sample in the presence of heat through solid phase extraction with the only difference being the reuse of the solvent in case of reflux (a5). as a result of this similarity, the time (c7) score of reflux (a5) is used as a proxy for hot solvent extraction (a7). the qualitative score for the criteria as shown in table 6 is selected based on the extraction procedure and intuition. the temperature range (c4) criteria are based on the operating temperature range of the technology and the temperature based compounds (thermo labile, mild thermostable, thermostable and hyper-thermostable) which can be extracted. a technology with maximum operating range and potential to extract compounds can provide high adaptability and is given the highest score. external energy input (manual, electrical and thermal) can add to the cost of the technology due to unavailability of a consistent source or high energy cost. manual and thermal energy are preferred in rural areas as electricity supply in indian villages is erratic. firewood and solar power can be used as sources of thermal power. the use of more than one source of energy can increase the adaptability of the technology. technology availability (c11) is determined based on the ease of its accessibility or ability to be made in local areas. maintenance (c12) and skills (c13) are determined based on the complexity of technology instruments and the extraction procedure. cost of technology (c14) is determined based both on the possibility of local availability of the resource, complexity of technology and operating procedure. table 6 list of criteria selected for ahp criterion technology a1 a2 a3 a4 a5 a6 a7 c1 2 3 8 1 6 1 3 c2 2 3 8 1 6 1 3 c3 4 3 7 2 6 1 2 c4 low to very high low to normal low to normal very high high to very high low to very high high to very high c5 1 1 1 1 1 1 1 c6 2 2 2 2 2 2 2 c7 190 3393.33 960 2 120 540 120 c8 manual or electrical no no electrical electrical and thermal electrical electrical and thermal c9 3 3 3 2 0 3 3 c10 4 167 2 17 1.5 31.6 5.15 c11 easy medium very easy medium medium tough easy c12 easy easy very easy tough medium tough medium c13 very basic very basic very basic medium basic medium basic c14 low medium very low high medium very high low 2.2.2.4 technology ranking the technology ranking was obtained by the summation of the product of the criteria and technology weightages for the different alternatives as shown in equation 1 below. ijahp article: jain, rao/ application of ahp tool for decision making of choice of technology for extraction of anti-cancer bioactive compounds of plant origin international journal of the analytic hierarchy process 19 vol.5 issue 1 2013 issn 1936-6744 𝑅𝑖 = ∑ 𝑃(𝑖, 𝑗)𝑤(𝑖) 11 𝑖=1 (1) here, rj = ranking of the alternatives, pij = normalized criteria score of the alternatives and wi = criteria weight. an overall ranking of the alternatives based on the total weighted score of the alternatives for the given criteria is created. 2.2.2.5 sensitivity analysis a sensitivity analysis of the tool is performed to determine the robustness of the results and to better understand the reason of the rankings of the alternatives. this allows the estimation of the impact of the individual criteria on the ranking. in this study, this is done by selective elimination of the score of single criterion from the ahp result as it can identify the criteria which played the main role in ranks by understanding the shifts in ranks. the criticality of the criterion depends on both the score distribution of the alternatives for that criterion and criterion weightage. the analysis can help in identifying the distribution of the score alternative which is required in order to maintain or change the ranking along with the maximum shift in ranking which can be expected. in this analysis, ranking of the technology was analyzed by removing a criterion, which is added back while removing any other criterion. for example, when criterion c1 is removed from the criteria set for ahp, it was added back in the criteria set of ahp when criterion c2 was removed. criteria c5 and c6 were not considered for the sensitivity analysis because in the current study scores of criteria c5 and c6 have remained constant for all the technologies. 3. results in the case of the single hierarchy based alternative ranking (table 7), the results show that the cold solvent extraction technology (a3) is the best technology for a rural entrepreneur. this is because the cost associated with this technology is lower as compared to other technologies as a result of the very low investment needed for the skills and technology required. the simplicity of the instruments needed make the technology and the maintenance very easy. this makes the validation of its use for all plants, compound classes and plant parts easy and cheap, which is indicated by the high score in the criteria c1, c2 and c3. the flexibility in the technological parameters, like ability to operate at different temperatures and extract different types of thermodependent compounds, no energy use, ability to use different enhancers with very low solvent requirements, provides a technical advantage to the extraction technology over others, which helps to compensate for the significant amount of time required by this technology. a sensitivity analysis of the technology (table 8) with removal of different criteria from the ahp process one at a time did not affect the technology which indicates its robustness and equitable distribution of the characteristics of the technology over the criteria. ijahp article: jain, rao/ application of ahp tool for decision making of choice of technology for extraction of anti-cancer bioactive compounds of plant origin international journal of the analytic hierarchy process 20 vol.5 issue 1 2013 issn 1936-6744 table 7 final criterion weights and normalized criteria matrix for single level hierarchy criteria agitation cold percolation cold solvent extraction mae reflux sonication hot solvent extraction name weightage a1 a2 a3 a4 a5 a6 a7 c1 0.08 0.140 0.286 1.000 0.000 0.710 0.000 0.286 c2 0.10 0.140 0.286 1.000 0.000 0.710 0.000 0.286 c3 0.08 0.500 0.333 1.000 0.170 0.830 0.000 0.167 c4 0.04 0.230 0.179 0.180 0.030 0.080 0.231 0.077 c5 0.03 1.000 1.000 1.000 1.000 1.000 1.000 1.000 c6 0.01 0.000 0.000 0.000 0.000 0.000 0.000 0.000 c7 0.07 0.940 0.000 0.720 1.000 0.970 0.841 0.965 c8 0.06 0.130 0.300 0.300 0.100 0.030 0.100 0.030 c9 0.01 1.000 1.000 1.000 0.670 0.000 1.000 1.000 c10 0.08 0.980 0.000 1.000 0.910 1.000 0.818 0.978 c11 0.11 0.170 0.122 0.220 0.120 0.120 0.073 0.171 c12 0.08 0.180 0.180 0.230 0.080 0.130 0.077 0.128 c13 0.10 0.180 0.177 0.180 0.100 0.140 0.098 0.137 c14 0.13 0.190 0.135 0.240 0.080 0.130 0.027 0.190 total score 0.353 0.211 0.562 0.245 0.442 0.214 0.332 final rank 3 7 1 5 2 6 4 reflux (a5) is the second best technology. it is a medium cost technology with availability of technology in nearby urban areas, medium level investment in terms of technology cost and maintenance and basic level skills requirement. a5 has fewer managerial characteristics as compared to technologies like agitation (a1), cold percolation (a7) and hot solvent extraction (a4), but the much better raw material characteristics offsets this disadvantage. the two most important technological characteristics c10 and c7 are the strengths of reflux which allows it to compensate for the low score in the other technological criteria. very low weightage of the technological characteristics make this technology robust in the sensitivity analysis (table 8). the two lowest technologies, i.e., sonication (a6) and cold percolation (a2) were at the bottom mainly because of poor raw material characteristics. in the case of sonication, the managerial characteristics are weak because it has high technology and maintenance costs, and it is not easily available. despite the lowest score in the most important characteristics, it was ranked higher than cold percolation because of the very strong major technological characteristics c4, c7, c10 as compared to cold percolation. it slipped to last in the rankings when either c7 or c10 were not considered (table 8). the sensitivity analysis (table 8) indicated changes in the technology ranking with criteria elimination, but only in 3 out of 12 criteria. changes in the technology rankings were observed in all technologies except cold solvent and reflux. criteria c7 and c10 were the main criteria whose elimination showed changes in cold percolation, mae and sonication. secondly, these two attributes showed maximum shift in the rank of the technologies, with cold percolation rank shifting from 7 to 5 and demoting the rank of the microwave and sonication by one level with 4 shifts in rank observed overall. this is ijahp article: jain, rao/ application of ahp tool for decision making of choice of technology for extraction of anti-cancer bioactive compounds of plant origin international journal of the analytic hierarchy process 21 vol.5 issue 1 2013 issn 1936-6744 because cold percolation is very weak in these two criteria. in the case of agitation (a1) and hot solvent extraction (a7), the hot solvent score is only 5.9% less than the agitation score. as a result, there is rank reversal in the absence of the criterion c3.the change in ranking of certain technologies with elimination of certain characteristics indicates that the process can be modified as per the needs of the user. the user can alter the criteria list, criterion weightages, technologies and criterion scores. this tool can be used to select the whole processing strategy by performing the ahp of the different processing steps. table 8 result of sensitivity analysis with selective removal of criteria for single level hierarchy criterion a1 a2 a3 a4 a5 a6 a7 # of technology rank changed # of shifts in rank default score 0.353 0.211 0.562 0.245 0.442 0.214 0.332 0 0 rank 3 7 1 5 2 6 4 no c1 score 0.341 0.187 0.477 0.245 0.382 0.214 0.308 0 0 rank 3 7 1 5 2 6 4 no c2 score 0.339 0.183 0.463 0.245 0.372 0.214 0.304 0 0 rank 3 7 1 5 2 6 4 no c3 score 0.311 0.183 0.477 0.231 0.372 0.214 0.318 2 2 rank 4 7 1 5 2 6 3 no c4 score 0.344 0.203 0.554 0.244 0.439 0.204 0.329 3 4 rank 3 7 1 5 2 6 4 no c7 score 0.287 0.211 0.511 0.175 0.374 0.155 0.264 0 0 rank 3 5 1 6 2 7 4 no c8 score 0.346 0.194 0.545 0.240 0.440 0.208 0.330 0 0 rank 3 7 1 5 2 6 4 no c9 score 0.339 0.197 0.547 0.236 0.442 0.200 0.318 3 4 rank 3 7 1 5 2 6 4 no c10 score 0.270 0.211 0.478 0.169 0.358 0.145 0.250 0 0 rank 3 5 1 6 2 7 4 no c11 score 0.334 0.197 0.537 0.232 0.428 0.206 0.313 0 0 rank 3 7 1 5 2 6 4 no c12 score 0.338 0.196 0.542 0.239 0.431 0.207 0.321 0 0 rank 3 7 1 5 2 6 4 no c13 score 0.336 0.193 0.544 0.236 0.429 0.204 0.318 0 0 rank 3 7 1 5 2 6 4 no c14 score 0.329 0.194 0.531 0.235 0.425 0.210 0.308 0 0 rank 3 7 1 5 2 6 4 the change in hierarchy structure from single hierarchy to two-level hierarchy has not resulted in any major changes in the technology ranking (table 9), with the only change observed in the lowest technologies a2, a6 and a7. this indicates that a change in hierarchy may not result in very strong changes in the current scenario. this means that whether the user tries single hierarchy or multiple hierarchy criteria for ahp, there will not be much impact on the top raking technologies. ijahp article: jain, rao/ application of ahp tool for decision making of choice of technology for extraction of anti-cancer bioactive compounds of plant origin international journal of the analytic hierarchy process 22 vol.5 issue 1 2013 issn 1936-6744 table 9 result of technology decision changes with shift from single-level hierarchy to two-level hierarchy criterion a1 a2 a3 a4 a5 a6 a7 default score 0.353 0.211 0.562 0.245 0.442 0.210 0.332 rank 3 7 1 5 2 6 4 two-level hierarchy score 0.285 0.206 0.521 0.165 0.391 0.132 0.264 rank 3 5 1 6 2 7 4 4. conclusion a multi-criteria decision making tool (using ahp) of ranking the best technology for extraction of nine of the medicinal plants which grow in western ghats of india and have more than 20% subsidy in cultivation under nmmp and have an anticancer property was developed keeping the perspective of a “rural entrepreneur”. the tool can be used by rural entrepreneurs using the developed simple strategy to select technologies as well as by the evaluators of the nmmp to evaluate the proposals from rural entrepreneurs for the successful set up of a medicinal plant processing unit. a java-based software is used to perform ahp for the ranking of the technology for extraction from desired medicinal plants. this process keeps in view the interest and situation of rural entrepreneurs like geographical location, local availability of medicinal plants and other raw materials and product demand in the market. the study successfully selected cold solvent extraction technology which could be most easily and reliably used by the rural entrepreneur in a highly constrained local environment. though, with the addition of more criteria, availability of more research information and technologies, the ranking of technologies could change. the attempt to showcase the utility of such a multi-criteria decision making tool using ahp in making decisions regarding the selection of the extraction technology still applies. the flexibility of the tool lies in the ease of modification as per user needs by selecting the criteria the user wants to consider, allocating weightages to the criteria based on their perspective, inserting alternative criterion scores and modifying the criteria. the strategy of creating the two stage process facilitates the selection of technologies for the user in a simple and easy way. this tool can also be useful for the facilitation agencies to disseminate the appropriate technology for the desired medicinal plants, medicinal plant parts and compound class to the rural entrepreneurs. policy makers and “rural entrepreneur proposal” evaluators can use this tool to provide differentiation in the processing subsidy for medicinal plants. for example, the maximum subsidy can be provided to the process and technology which ranks highest in the results. this will provide a more customized and flexible framework for the rural entrepreneur to work within. rural entrepreneur proposal evaluators can also use this tool to validate the project proposals of rural entrepreneurs and map demand for different types of technologies for the geographical areas based on the results of the ahp. ijahp article: jain, rao/ application of ahp tool for decision making of choice of technology for extraction of anti-cancer bioactive compounds of plant origin international journal of the analytic hierarchy process 23 vol.5 issue 1 2013 issn 1936-6744 references abed, m. h., monavari, m., karbasi, a., farshchi, p., & abedi, z. (2011).site selection using analytical hierarchy process by geographical information system for sustainable coastal tourism.ipcbee, 15, 120–124. bharathi, p., philomina, d., & chakkaravarthi, s. (2006). estimation of colchicine in six different species of gloriosa grown in vivo.indian j. pharm. sci., 68(6), 806–809. bothiraja, c., joshi, p. p., dama, g. y., & pawar, a. p. (2011).rapid method for isolation of plumbagin, an alternative medicine from roots of plumbago zeylanica.european journal of integrative medicine, 3, 39–42. doi:10.1016/j.eujim.2011.02.008 britto, a. j. d. a., & sujin, r. m. (2012).hplc analysis of vitexin and fingerprint of vitex negundo l. int j pharm pharm sci, 4(2), 138–141. chen, s., yuchun, x., & huizhou, l. (2007). microwave-assisted micellar extraction and determination of glycyrrhizic acid and liquiritin in licorice root by hplc. chin. j. chem. eng, 15(4), 474–477. chitra, k., sujatha, k., polisetti, h., karri, s., & reddy, c. u. (2011). standardization of coscinium fenestratum with reference to berberine by high performance thin layer chromatography.rjpbcs, 2(2), 226–229. choi, m. p. k., chan, k. k. c., leung, h. w., & huie, c. w. (2003). pressurized liquid extraction of active ingredients (ginsenosides) from medicinal plants using non-ionic surfactant solutions.journal of chromatography a, 983(1-2), 153–162. chomnawang, m. t., trinapakul, c., & gritsanapan, w. (2009). in vitro antigonococcal activity of coscinium fenestratum stem extract. journal of ethnopharmacology, 122(3), 445–9. choochote, w., rongsriyam, k., pitasawat, b., jitpakdi, a., junkum, a., tuetun, b., & chaiwong, p. (2004). evaluation of the colchicine-like activity of gloriosa superba – extracted fractions for mosquito (diptera: culicidae). journal of medical entomology, 41(4), 672–76. cragg, g. m., & newman, d. j. (2005). biodiversity: a continuing source of novel drug leads. pure appl chem, 77, 7–24. critical ecosystem partnership fund (cepf). (2007). western ghats & sri lanka biodiversity hotspot: western ghats region. arlington, usa. dabaghian, m. r., hashemi, s. h., ebadi, t., & maknoon, r. (2008).the best available technology for small electroplating plants applying analytical hierarchy process.int. j. environ. sci. tech., 5(4), 479–484. ijahp article: jain, rao/ application of ahp tool for decision making of choice of technology for extraction of anti-cancer bioactive compounds of plant origin international journal of the analytic hierarchy process 24 vol.5 issue 1 2013 issn 1936-6744 dalavayi, s., kulkarni, s. m., itikala, r. l., & itikala, s. (2006). determination of withaferin–a in two withania species by rp-hplc method. indian j. pharm. sci., 68(2l), 253–256. deevanhxay, p., suzuki, m., maeshibu, n., li, h., tanaka, k., & hirose, s. (2009). simultaneous characterization of quaternary alkaloids, 8-oxoprotoberberine alkaloids, and a steroid compound in coscinium fenestratum by liquid chromatography hybrid ion trap time-of-flight mass spectrometry.journal of pharmaceutical and biomedical analysis, 50(3), 413–425. devi, p. u., utsumi, h., takata, m., & takeda, s. (2008). enhancement of radiation induced cell death in chicken b lymphocytes by withaferin a. indian j exp biol., 46(6), 437–442. elkarmi, f., & mustafa, i. (1993).increasing the utilization of solar energy technologies (set) in jordan analytic hierarchy process.energy policy, 21(9), 978–984. doi:10.1016/0301-4215(93)90186-j eng, a. t. w., heng, m. y., & ong, e. s. (2007). evaluation of surfactant assisted pressurized liquid extraction for the determination of glycyrrhizin and ephedrine in medicinal plants. analytica chimica acta, 583(2), 289–295. doi:10.1016/j.aca.2006.09.019 garcıá-ayuso, l., & castro, m. d. l. (1999).a multivariate study of the performance of a microwave-assisted soxhlet extractor for olive seeds.analytica chimica acta, 382(3), 309–316. doi:10.1016/s0003-2670(98)00795-8 handa, s. s., khanuja, s. p. s., longo, g., & rakesh, d. d. (eds.).(2008). extraction technologies for medicinal and aromatic plants.trieste, italy: international centre for science and high technology. hasan, m. r., tetsuo, k., & islam, s. a. (2009).landfill demand and allocation for municipal solid waste disposal in dhaka city — an assessment in a gis environment.journal of civil engineering, 37(2), 133–149. he, j., zhao, z., shi, z., zhao, m., li, y., & chang, w. (2005).analysis of isoflavone daidzein in puerariae radix with micelle-mediated extraction and preconcentration.j. agric. food chem, 53(3), 518–523. doi:10.1021/jf048545q hummel, j. m. m., snoek, g. j., van til, j. a., van rossum, w., & ijzermann, m. j. (2005). a multicriteria decision analysis of augmentative treatment of upper limbs in persons with tetraplegia.journal of rehabilitation research and development, 42(5), 635–644. retrieved from http://www.rehab.research.va.gov/jour/05/42/5/hummel.html jagetia, g. c., & baliga, m. s. (2005). the effect of seasonal variation on the antineoplastic activity of alstonia scholaris r. br. in hela cells.journal of ethnopharmacology, 96(1-2), 37–42. doi:10.1016/j.jep.2004.07.024 http://www.rehab.research.va.gov/jour/05/42/5/hummel.html ijahp article: jain, rao/ application of ahp tool for decision making of choice of technology for extraction of anti-cancer bioactive compounds of plant origin international journal of the analytic hierarchy process 25 vol.5 issue 1 2013 issn 1936-6744 javaheri, h., nasrabadi, t., jafarian, m. h., rowshan, g. r., & khoshnam, h. (2006). site selection of municipal solid waste landfills using analytical hierarchy process method in a geographical information technology environment in giroft. iran. j. environ. health. sci. eng., 3(3), 177–184. jayaprakasam, b., & nair, m. g. (2003). cyclooxygenase-2 enzyme inhibitory withanolides from withania somnifera leaves. tetrahedron, 59(6), 841–849. doi:10.1016/s0040-4020(02)01601-0 jeyachandran, r., mahesh, a., cindrella, l., sudhakar, s., & pazhanichamy, k. (2009). antibacterial activity of plumbagin and root extracts of plumbago zeylanica l. acta biologica cracoviensia series botanica, 51(1), 17–22. jung, h. (2011). a fuzzy ahp–gp approach for integrated production-planning considering manufacturing partners.expert systems with applications, 38(5), 5833–5840. doi:10.1016/j.eswa.2010.11.039 jyothi d khanam s sultana r. (2010). optimization of microwave assited solvent extraction of withanolides from leaves of ashwagandha.pharmacie globale (ijcp), 01(04), 1–5. jyothi, d., khanam, s., & sultana, r. (2010). optimization of microwave assisted extraction of withanolides from roots of aswagandha and its comparison with conventional extraction method. int j pharm pharm sci, 2(4), 46–50. kablan, m. (2004). decision support for energy conservation promotion: an analytic hierarchy process approach. energy policy, 32(10), 1151–1158. doi:10.1016/s03014215(03)00078-8 kadota, s., takamori, y., nyein, k. n., kikuchi, t., tanaka, k., & ekimoto, h. (1990).constitutents of the leaves of woodfordia fructicosa kurz. i. isolation, structure, and proton and carbon-13 nuclear magnetic signal assignments of woodfruticosin (woodfordin c), an inhibitor of deoxyribonucleic acid topoisomerase ii. chem. pharm. bull., 38(10), 2687–97. kannan, s., wesley, s. d., ruba, a., rajalakshmi, a. r., & kumaragurubaran, k. (2007).optimization of solvents for effective isolation of colchicines from gloriosa superba l. seeds.natural product research: formerly natural product letters, 21(5), 469–472. doi:10.1080/14786410601129507 kavina, j., gopi, r., & panneerselvam, r. (2011). quantification of colchicine in seed and tuber samples of gloriosa superba by high performance liquid chromatography method. journal of applied pharmaceutical sciences, 01(07), 116–119. kholkute, s. d. (2008). coordinating unit of survey of medicinal plants of western ghats of india. new delhi: indian council of medical research. ijahp article: jain, rao/ application of ahp tool for decision making of choice of technology for extraction of anti-cancer bioactive compounds of plant origin international journal of the analytic hierarchy process 26 vol.5 issue 1 2013 issn 1936-6744 kodali, r., & chandra, s. (2001). analytical hierarchy process for justification of total productive maintenance.production planning & control, 12(7), 695–705. doi:10.1080/0953728001002404 lee, s. j., cho, s. a., an, s. s., na, y. j., park, n. h., kim, h. s., lee, c. w., et al. (2012). alstonia scholaris r. br. significantly inhibits retinoid-induced skin irritation in vitro and in vivo. evidence-based complementary and alternative medicine, 2012(190370), 11. doi:10.1155/2012/190370 lin, t., nonaka, g., nishioka, i., & ho, f. (1990).tannins and related compounds. cii. structures of terchebulin, an ellagitannin having a novel tetraphenylcarboxylic acid (terchebulic acid) moiety, and biogenetically related tannins from terminalia chebula retz. chem. pharm. bull., 38(11), 3004–8. luque-garcı́a, j. l., & castro, m. d. l. (2004). ultrasound-assisted soxhlet extraction: an expeditive approach for solid sample treatment application to the extraction of total fat from oleaginous seeds. journal of chromatography a, 1034(1-2), 237–242. doi:10.1016/j.chroma.2004.02.020 macabeo, a. p. g., krohn, k., gehle, d., read, r. w., brophy, j. j., cordell, g. a., franzblau, s. g., et al. (2005). indole alkaloids from the leaves of philippine alstonia scholaris.phytochemistry, 66(10), 1158–62. doi:10.1016/j.phytochem.2005.02.018 mahajan, s., ramakrishnan, g., & date, a. w. (2008).user friendly package for ranking of technology alternatives using analytical hierarchy process. mumbai, india. malhotra, s., taneja, s. c., & dhar, k. l. (1989).minor alkaloid from coscinium fenestratum.phytochemistry, 28(7), 1998–1999. malik, f., singh, j., khajuria, a., suri, k. a., satti, n. k., singh, s., kaul, m. k., et al. (2007). a standardized root extract of withania somnifera and its major constituent withanolide-a elicit humoral and cell-mediated immune responses by up regulation. life sciences, 80(16), 1525–1538. doi:10.1016/j.lfs.2007.01.029 mirzajani, f., ghassempour, a., jalali-heravi, m., & mirjalili, m.h. (2010). optimisation of a microwave �assisted method for extracting withaferin a from withania somnifera dunal. using central composite design. phytochem. anal., 21(6), 544–549. retrieved from http://onlinelibrary.wiley.com/doi/10.1002/pca.1230/full mohajeri, n., & amin, g. r. (2010).railway station site selection using analytical hierarchy process and data envelopment analysis.computers & industrial engineering, 59(1), 107–114. doi:10.1016/j.cie.2010.03.006 montevechi, j. a. b., guimaraes, i. f., oliveira, m. l. m. d., & friend, j. d. (2010).decision-making with multiple criteria in the selection of ultrasound scavenging system in a private hospital in brazil.international journal of the analytic hierarchy process, 2(1), 14–29. http://onlinelibrary.wiley.com/doi/10.1002/pca.1230/full ijahp article: jain, rao/ application of ahp tool for decision making of choice of technology for extraction of anti-cancer bioactive compounds of plant origin international journal of the analytic hierarchy process 27 vol.5 issue 1 2013 issn 1936-6744 moulisha, b., kumar, g. a., & kanti, h. p. (2010). anti-leishmanial and anti-cancer activities of a pentacyclic triterpenoid isolated from the leaves of terminalia arjunacombretaceae. tropical journal of pharmaceutical research, 9(2), 135–140. retrieved from http://www.ajol.info/index.php/tjpr/article/view/53700 national medicinal plants board. (2008). centrally sponsored scheme of national mission on medicinal plants: operational guidelines (pp. 1–37). new delhi, india: ayush. retrieved from: http://nmpb.nic.in/writereaddata/links/1734335889english1.pdf nijkamp, p., & delft, a. van. (1977). multi-criteria analysis and regional decisionmaking. leiden: martunus nijhoff social sciences division. nixon, j. d., dey, p. k., & davies, p. a. (2010). which is the best solar thermal collection technology for electricity generation in north-west india? evaluation of options using the analytical hierarchy process.energy, 35(12), 5230–5240. doi:10.1016/j.energy.2010.07.042 palcic, i., & lalic, b. (2009).analytical hierarchy process as a tool for selecting and evaluating projects.int j simul model, 8(1), 16–26. doi:10.2507/ijsimm08(1)2.112 pawar, r. k., shivani, s., singh, k. c., & sharma, r. k. (2010). hptlc method for the determination of plumbagin from plumbago zeylanica linn. (root). int. j pharm pharm sci, 2(4), 219–223. pawar, r. s., & bhutani, k. k. (2005). effect of oleanane triterpenoids from terminalia arjuna—a cardioprotective drug on the process of respiratory oxyburst. phytomedicine, 12(5), 391–393. pinho, p. m. m., pinto, m. m. m., kijjoa, a., pharada, k., diaz, j. g., & herz, w. (1992).protoberberine alkaloids from coscinium fenestratum.phytochemistry, 31(4), 1403–1407. rajasekharan, p. e. (2006). potential of plant derived pharmaceuticals. science tech entrepreneur, (april), 1–8. raju, u. s., rangaraj, n., & date, a. w. (1995).the influence of development perspectives on the choice of technology.technological forecasting and social change, 43, 27–43. razmi, j., rahnejat, h., & khan, m. k. (2005).use of analytic hierarchy process approach in classification of push, pull and hybrid push, pull systems for production planning.international journal of operations & production management, 18(11), 1134– 1151. rojsanga, p., & gritsanapan, w. (2005).variation of berberine content in coscinium fenestratum stem in thailand market.mahidol university journal of pharmaceutical http://www.ajol.info/index.php/tjpr/article/view/53700 http://nmpb.nic.in/writereaddata/links/1734335889english1.pdf ijahp article: jain, rao/ application of ahp tool for decision making of choice of technology for extraction of anti-cancer bioactive compounds of plant origin international journal of the analytic hierarchy process 28 vol.5 issue 1 2013 issn 1936-6744 sciences, 32(3-4), 66–70. retrieved from: http://www.pharmacy.mahidol.ac.th/mujournal/_files/2005.66-70.pdf rojsanga, p., gritsanapan, w., & suntornsuk, l. (2006). determination of berberine content in the stem extracts of coscinium fenestratum by tlc densitometry. med. princ.pract., 10400, 373–378. doi:10.1159/000094272 saaty, t. l. (1980). the analytic hierarchy process. new york, usa: mcgraw-hill. saxena, m., faridi, u., mishra, r., gupta, m. m., darokar, m. p., srivastava, s. k., singh, d., et al. (2007). cytotoxic agents from terminalia arjuna. planta medica, 73(14), 1486–1490.doi:10.1055/s-2007-990258 singh, d. v., verma, r. k., singh, s. c., & gupta, m. m. (2002). rp-lc determination of oleane derivatives in terminalia arjuna. j. pharm. biomed. anal., 28(3-4), 447–452. retrieved from http://www.ncbi.nlm.nih.gov/pubmed/12008123 singh, s., dolan, j. g., & centor, r. m. (2006).optimal management of adults with pharyngitis--a multi-criteria decision analysis.bmc medical informatics and decision making, 6, 14.doi:10.1186/1472-6947-6-14 skibniewski, m. j., & chao, l. (1992).evaluation of advanced construction technology with ahp method.journal of construction engineering and management, 118(3), 577– 593. sloane, e. b., liberatore, m. j., nydick, r. l., luo, w., & chung, q. b. (2003).using the analytic hierarchy process as a clinical engineering tool to facilitate an iterative, multidisciplinary, microeconomic health technology assessment. computers & operations research, 30(10), 1447–1465. doi:10.1016/s0305-0548(02)00187-9 stenhouse, j. (1867). xvi.-on the preparation of berberine from coscinium fenestratum. j. chem. soc., 20, 187–188. retrieved from: http://pubs.rsc.org/en/content/articlepdf/1867/js/js8672000187 thampi, a., & rao, b. (2012). development of a decision making tool for choice of technology in municipal solid waste management for local self government bodies a case study of kerala , india. 27th international conference on solid waste technology and management. tran, l. q., tezuka, y., ueda, j., trung, n., maruyama, y., begum, k., kim, h., et al. (2003). in vitro antiplasmodial activity of antimalarial medicinal plants used in vietnamese traditional medicine. journal of ethnopharmacology, 86(2-3), 249–252. doi:10.1016/s0378-8741(03)00045-x tungpradit, r., sinchaikul, s., phutrakul, s., wongkham, w., & chen, s. t. (2011).antiproliferative activity of berberine from coscinium fenestratum on nci-h838 cell line. chiang mai j. sci., 38(1), 85–94. retrieved from: http://www.pharmacy.mahidol.ac.th/mujournal/_files/2005.66-70.pdf http://www.ncbi.nlm.nih.gov/pubmed/12008123 http://pubs.rsc.org/en/content/articlepdf/1867/js/js8672000187 ijahp article: jain, rao/ application of ahp tool for decision making of choice of technology for extraction of anti-cancer bioactive compounds of plant origin international journal of the analytic hierarchy process 29 vol.5 issue 1 2013 issn 1936-6744 http://scholar.google.com/scholar?hl=en&btng=search&q=intitle:antiproliferative+acti vity+of+berberine+from+coscinium+fenestratum+on+nci-h838+cell+line#0 unnikrishnan, k. p., raja, s. s., & balachandran, i. (2008).a reverse phase hplc-uv and hptlc methods for determination of plumbagin in plumbago indica and plumbago zeylanica.indian j. pharm. sci., 70(6), 844–847. retrieved from: http://www.ncbi.nlm.nih.gov/pmc/articles/pmc3040894/ upadhyay, r. k., pandey, m. b., jha, r. n., singh, v. p., & pandey, v. b. (2001). triterpene glycoside from terminalia arjuna. journal of asian natural products research, 3(3), 207–212. retrieved from: http://www.tandfonline.com/doi/abs/10.1080/10286020108041392 ved, d. k., & goraya, g. s. (2008). demand and supply of medicinal plants in india. dehradun: bishen singh mahendra pal singh publishers, dehradun and frlht, bengaluru. venukumar, m. r. (2004). effect of coscinium fenestratum and curculigo orchioides in rat model of hepatotoxicity.mahatma ghandhi university. world health organization.(2000). report of the inter-regional workshop on intellectual property rights in the context of traditional medicine. bangkok, 6-8 dec 2000. world health organization (who). (2008). the global burden of disease 2004 update. geneva, switzerland. world health organization (who).(2011). global status report on noncommunicable diseases 2010. geneva, switzerland. xiang, l., jianzhong, m., jing, x., & yundong, s. (2011). surfactant assisted ethanol extraction of dioscin. journal of medicinal plants research, 5(3), 324–331. retrieved from http://www.academicjournals.org/jmpr/pdf/pdf2011/4feb/xiang et al.pdf yahaya, s., ilori, c., whanda, j. s., & edicha, j. (2010). land fill site selection for municipal solid waste mangement using geographic information system and multicriteria evaluation. american journal of scientific research, 10(10), 34–49. yibchok-anun, s., jittaprasatsin, w., somtir, d., bunlunara, w., & adisakwattana, s. (2009).insulin secreting and α -glucosidase inhibitory activity of coscinium fenestratum and postprandial hyperglycemia in normal and diabetic rats.journal of medicinal plants research, 3(9), 646–651. zhou, h., he, h., luo, x., wang, y., yang, x., & di, y. (2005).three wew indole alkaloids from the leaves of alstonia scholaris.helvetica chimca acta, 88(9), 2508–2512. zhou, y. j., liu, y. e., cao, j. g., zeng, g. y., shen, c., li, y. l., zhou, m. c., et al. (2009). vitexins, nature-derived lignan compounds, induce apoptosis and suppress tumor growth. clin cancer res, 15(16), 5161–5169. doi:10.1158/1078-0432.ccr-09-0661 http://scholar.google.com/scholar?hl=en&btng=search&q=intitle:antiproliferative+activity+of+berberine+from+coscinium+fenestratum+on+nci-h838+cell+line#0 http://scholar.google.com/scholar?hl=en&btng=search&q=intitle:antiproliferative+activity+of+berberine+from+coscinium+fenestratum+on+nci-h838+cell+line#0 http://www.ncbi.nlm.nih.gov/pmc/articles/pmc3040894/ http://www.tandfonline.com/doi/abs/10.1080/10286020108041392 http://www.academicjournals.org/jmpr/pdf/pdf2011/4feb/xiang%20et%20al.pdf application of ahp tool for decision making of choice of technology for extraction of anti-cancer bioactive compounds of plant origin 1. introduction 2. methodology 2.1 selection of medicinal plants 2.2 selection of extraction technology/alternative 2.2.1 stage 1: identification of extraction alternatives for medicinal plants 2.2.2 stage 2: analytical hierarchy process (ahp) 2.2.2.2 criteria weightages 2.2.2.3 criteria scores for technology 2.2.2.4 technology ranking 2.2.2.5 sensitivity analysis 3. results 4. conclusion 1 ijahp article: strojny, hejman/ahp based multicriteria comparative analysis of regions of eastern poland international journal of the analytic hierarchy process 53 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.373 ahp based multicriteria comparative analysis of regions of eastern poland jacek strojny rzeszow university of technology rzeszow, poland e-mail: jstrojny@prz.edu.pl wim hejman wageningen ur, wageningen, nederland e-mail: wim.heijman@wur.nl abstract the modern economic landscape dynamically creates new and often difficult conditions in which all types of organizations must function. in management sciences literature the companies sector is the situation most often analyzed, yet it is also important to improve the management processes in public administration. the global economic crisis that started in 2008 assumed the form of a public finance crisis especially in developed economies. public authorities are forced to meet the rising expectations of democratic societies while maintaining financial discipline. the pressure to optimize public spending constantly increases. decision-making processes must therefore focus on the dimensions most relevant to the development of economic systems data. this research takes into account these realities, and focuses on the competitiveness of regions located in so-called eastern poland. the study uses a multi-criteria comparative analysis based on the ahp method. the procedure involved building models of competitiveness which systemize statistical variables describing the phenomenon of competitiveness. based on these models, measurements were constructed and the level of competitiveness was calculated. the regions selected for the case study are separated according to the polish cohesion policy due to their similar developmental problems. the author is very familiar with the context of development for the selected regions which makes the evaluation of the usefulness of the results obtained by the analysis easier. keywords: public administration; regional competitiveness; strategic management; ahp; comparative analysis 1. introduction the development of regions and other economic systems (national or local) is perhaps the most common topic of research in economic sciences. therefore, it is worth analyzing from the perspective of decision-making processes related to the development of these systems. the research carried out is not only for the understanding of the mechanisms of development, but also for supporting actions connected with their stimulation, maintenance, and monitoring. in such a situation, regional analysis should be treated as a ijahp article: strojny, hejman/ahp based multicriteria comparative analysis of regions of eastern poland international journal of the analytic hierarchy process 54 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.373 strategic management tool, directed at supporting the decision-making processes. preparation of management information and its proper interpretation is the key to properly defining the problems and opportunities of development. hence, the relevance of the formulated strategic objectives depends on the quality of the obtained image of the current situation of the economic system and the directions of changes taking place in it. the process of strategic management development of territorial units in poland faces many barriers of various types. one of them is inadequate, poorly structured strategic analysis. in the majority of local and regional strategies in poland, aggregate measures describing various dimensions of the socioeconomic situation are used in a limited way. the benchmarking method is also not very popular. typically therefore, the strategic analysis is carried out without a broader reference to the situation in other territorial units (benchmarks). this significantly limits the scope of the strategic information obtained and creates the risk of an improper definition of the strategic objectives. however, there are tools that make it possible to gain more complete strategic information. one of these tools is a multi-criteria comparative analysis called the analytical hierarchy process (ahp). the purpose of this article is to describe the use of the ahp method in a multi-criteria comparative analysis using the example of the regions of so-called eastern poland. the structure of the accepted research goals is as follows1: − grq: how are the competitive positions of regions of eastern poland shaped with respect to the other regions of the country? − drq.1: how are the competitive positions of regions measured? − drq.2: what is the competitive position of particular regions of eastern poland? − drq.3: how has the competitive position of the particular regions of eastern poland changed in the past? the study was conducted in order to evaluate the competitiveness of the investigated territorial units. first, a brief review of the literature was performed to find a definition of the concept of regional competitiveness and general challenges for strategic management in public administration. references were also researched concerning the strategic management process and procedures for multi-criteria comparative analysis. then, two models of ahp were created to measure the competitiveness in two dimensions – endogenous potential (seegi model), and attractiveness relative to stakeholders (tcb model). they were used to organize the variables, forming the basis for their selection and construction of aggregate indicators. the local wages of individual elements of the model were obtained on the basis of the judgments of experts dealing with regional development. finally, the article gives a brief summary and conclusions of the audit. 2. managing the regional development three types of subdivisions of territorial administration can be identified in the polish system: 16 regions (voivodeships) at the local level, 314 districts (poviats), 66 towns with district rights, and nearly 2.5 thousand municipalities (cso 2015). the central authorities shape the general conditions for economic development. in poland and other countries of 1 grq – general research question, drq – detailed research question. ijahp article: strojny, hejman/ahp based multicriteria comparative analysis of regions of eastern poland international journal of the analytic hierarchy process 55 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.373 the european zone, regions are the main level of management of territorial development. cohesion policies are created for them and various instruments of financial support are based on them. at the regional level in poland, the major part of the territorial development policy is based on the support of national and especially european funds (potoczak, 2013). poorer regions use these sources of financing to intensify intervention in the development process for creating and sustaining the convergence. since the reforms in 1999, regional and local units of territorial administration have systematically increased their competences, taking over the responsibilities of the central administration (dolnicki, 2012). there has also been growth in their ability to generate revenue through vehicles such as participation in income taxes from residents and businesses (public financial act, 2009; regional and local government's income act, 2003). regional and local authorities have considerable freedom in formulating development goals and initiating actions conducive to the development of the territorial unit (municipality act, 1990; district act, 1998; voivodeship act, 1998). management of the development of territorial units in poland is based on a hierarchical system of plans (strategies) formulated at different levels of territorial administration. they can be distinguished as follows (cm, 2009): − long-term and medium-term strategy of the country and the national functional strategies − trans-regional strategies (e.g. strategy of socioeconomic development of eastern poland, 2020) − regional development strategies and regional functional strategies, − strategies for the development of districts and municipalities the obligation of strategic planning does not include municipalities and districts, but in practice the vast majority of units prepare these documents at a local level. this article focuses not only on managing the development of the regions, but the conditions shaping this process, particularly because the methodology of planning and controlling strategies are universal and also apply to local units. in the management literature the importance of strategic thinking is often stressed, which may be understood as an attribute of the persons responsible for achieving long-term goals (porter, 1996; abraham, 2005; steptoe-warren et. al., 2011). it should also be noted that the primary tool in this process is the strategy – the long-term plan of the activity (mintzberg et. al., 1998). in poland, at the regional and district level, strategic management relies mostly on strategic documents. in some municipalities where the same person is elected as mayor for multiple tenures the bigger role of strategic thinking than formal documents is visible. the concept of strategic management is widely analyzed in the literature. generally speaking, it is a way of modifying the organization by constantly providing care for following the vision for future activity (eden & ackermann, 2013). however, this proactive formula of strategic management requires adaptation to the requirements of public administration. in this perspective, following this vision requires the continuous monitoring of the strategy, modifying accepted assumptions of it, and adapting to changes in the environment, or even anticipating them (krupski, 2007). most often the full process of strategic management requires the following steps: (1) strategic analysis, ijahp article: strojny, hejman/ahp based multicriteria comparative analysis of regions of eastern poland international journal of the analytic hierarchy process 56 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.373 (2) planning the strategic tasks, (3) auditing and final inspection of the strategy realization (dess & miller 1993; gierszewska & romanowska, 1997). other authors such as thompson &. stickland (1993) propose a similar but more extensive description. the relatively low quality of strategic management in polish public administration and awareness of the challenges standing before the administration makes those involved inclined to seek a different systemic solution which would provide an enhancement to the quality of the process (strojny, 2012). this article presents just such an approach, based on a model developed by the author called stram – strategy or administration model which is shown in figure 1. it includes four stages of management: (1) preparation of the strategic information, (2) formulation of the objectives, (3) strategic planning, and (4) implementation of the strategy (including audit and final inspection). in the course mentioned, phases also include the use of four methods supporting the strategic management: (1) ahp (analytic hierarchy process), (2) mbo (management by objectives), (3) benchmarking and (4), pm (project management). figure 1. diagram of strategy for administration model source: own work the first method included in the model is the ahp. it was created by t.l. saaty during the years 1970-1980 (saaty, 1980). it is used to support decision-making processes by allowing the structuring of the problem and the analysis of the preferences (saaty & forman, 1992). it is one of the most popular methods of multi-criteria analysis in the world (prusak et. al., 2014). it is used not only in scientific research, but is also applied to real decision-making processes including those in public institutions. numerous case studies on the usage of the ahp in the decision-making processes in the public sector are listed in literature (saaty, 2008; awasthi, 2012; de luca, 2013; sayyadi & georgiadou et. al., 2013). this shows that, despite a significant advancement in the life cycle of the ahp, this method is still being developed and adapted to changes in organizations and their environment. the presented applications are mostly implemented to solve specific problems, and so at the operational (project) level, implementation into the strategy is quite rare. benchmarking is another method used in the model. it is also very popular in management sciences. it has been applied since the 1970s at xerox (bogan & english, 1994; kuczewska, 2007). it is widely known through the publications, among others, of (4) realization of strategy ahp de ve lo pm en t ga p mbo benchmarking (2) formulation of goals (3) strategic planning pm (1) preparation of the strategic information ijahp article: strojny, hejman/ahp based multicriteria comparative analysis of regions of eastern poland international journal of the analytic hierarchy process 57 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.373 r.j. boxwel (1994). this method helps improve organizations by comparing patterns (balm, 1992; anand & kodali, 2008). due to its efficiency, it has gained great popularity in the practice of management, not only in commercial organizations, but also in public institutions (bowerman et. al., 2000, 2001, 2002; angiola & bianchi, 2015). it is worth noting that performance management is largely based on the concept of mbo which is much older than the methods described above (van dooren et. al., 2014). it was described by p.f. drucker (1954), and was then developed by g.s. odiorne (1965). it can be seen as a comprehensive approach, aimed at orienting the organization toward the objectives (carmen, 2014). its essence is therefore, correcting the formulation and a continuous monitoring of implementation. like the previous methods, the mbo is used to enhance the effectiveness of different types of organizations (kyriakopoulos, 2012) including public administration (moore &staton 1981; pojster & streib, 1995). the orientation toward goals is also a characteristic for the project management (pm) method which in the stram model is used to carry out the entire process of planning and implementation of the strategic tasks. the pm method was established during the manhattan project of world war ii (seymour & hussein, 2014). it became widespread by the 1960s in many business activities, which resulted in the development of a number of approaches, including such well-known ones such as ipma2, pmi3 and prince24 (kwak, 2003). these approaches differ significantly, but they all prepare the organization for efficient project management. pm allows the use of many techniques for planning and controlling projects or strategic programs (ogc, 2009; pmi, 2013; ipma, 2013). therefore, it is worth advocating its use in public administration (abbasi& al-mharmah, 2000; kuipers, et. al., 2014). 3. competitiveness of regions – theory and ahp models as previously discussed, the process of strategic management is one of the primary tools with which the authorities of public administration can shape development processes. this has an effect on the quality of life of residents, investment conditions, and the overall economic situation in the territorial unit. in the modern economic environment, competitiveness is of particular importance and is therefore one of the most important strategic attributes. it is a set of attributes of the entity that allow competition in the relevant market (strategor, 2001). it affects the improvement of occupied competitive position and attractiveness in relation to the other competing entities (feurer & chaharbaghi, 1994; moon & newman, 1995). the concept of competitiveness is mostly analyzed from either a marketing or management perspective in relation to companies (walas-trębacz, 2013; flak & głód 2014; lucato et. al., 2015). however, today it is also the subject of research in the context of the development of territorial units especially the regions. both in the past in the lisbon strategy and in the present in strategy europe 2020, the european commission stresses the need to build a solid foundation for competitiveness. this phenomenon is also subject to continuous monitoring and evaluation in the context of the development of 2 ipma – international project management association. 3 pmi – project management institute. 4 prince – projects in a controlled environment. ijahp article: strojny, hejman/ahp based multicriteria comparative analysis of regions of eastern poland international journal of the analytic hierarchy process 58 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.373 both the entire european union area and particular regions (annoni & dijkstra, 2013; eu, 2014, 2015). in the case of entities, such as territorial units, understanding the competitiveness and analysis of this phenomenon requires the adoption of a market perspective. in order to identify the markets in which territorial units compete, and the parameters of competitiveness and the competitive position, some questions need to be answered. some of these questions are as follows: (1) in which markets do the territorial units compete – who is the client? (2) on what is the rivalry between them based? (3) what instruments do they have shaping the process of competition and building a competitive position? the concept of the market is one of the fundamental concepts in economics and requires no further discussion in this short paper. for the purposes of this research it was assumed that the market is a space for the exchange of certain goods between entities which both provide and consume them. the territorial unit (e.g. region) in this process can be treated as a supply site. the demand site is created by groups of stakeholders that are interested in living, investing and/or visiting in the territory. discussion about this mechanism requires at least a brief appeal to the modern theory of regional development. it consists of many approaches, starting with the theory of comparative costs, exogenous development theory, the new theory of growth, convergence theory, theory of growth poles or new economic geography (martin, 2003). from the perspective of this article, the new growth theory based on the concepts of r.e. lucas (1988, 1990) and p.m. romer (1994, 1998) is particularly interesting. it is assumed that the process of economic growth may accumulate spatially, leading to the creation and deepening of differences in the development of territorial systems. growth is endogenous and depends on endogenous potential as the main factor affecting the level and rate of growth of production. it consists of many elements which include people, knowledge, know-how, technology and financial capital, etc. concentration of the development processes leads to growth centers especially around big metropolises. the process of formulating and developing such centers is described by the growth poles theory and new economic geography (perroux, 1950; krugman, 1995). there is no denying that one of the important factors in the formation of growth centers, in addition to domestic factors, is capital transfers. flows from poorer to richer regions cause a divergence phenomenon that is increasing the differentiation of development. flows in the other direction help poorer territorial units to increase their low endogenous potential and lead to convergence. this refers to the process of equalizing the level of development (linnemann et.al., 1965; wojcik, 2008). both of these processes create the fundamental dilemma in the policy of territorial development in the european union. the concentration of the capital leads to higher competitiveness on the national or european level, but causes the atrophy of development functions in many poorer territories. on the other hand, supporting the process of convergence can be inefficient because of many structural barriers in lower developed regions. therefore, the analysis of the competitiveness of regions is very important as an element of strategic thinking and management at all levels of public authority in europe. it can be assumed that there are markets of mobile capital – people and companies which change their location for living or investing. transactions in this market therefore rely on ijahp article: strojny, hejman/ahp based multicriteria comparative analysis of regions of eastern poland international journal of the analytic hierarchy process 59 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.373 the decision concerning the selection of the territory. what the territory has to offer is generally described by the conditions that determine the possibility of development of the capital in a given location. people are taking into account factors determining quality of life and possibilities for a professional career. enterprises analyze facilities that provide a high efficiency of production processes. additionally, the literature recognizes the role of another market which significantly determines the development of regions. this is the tourist market (ivanov & webster, 2013). the choice of a specific tourist destination depends on many attributes of the region related to natural resources, infrastructure and culture and business environment (crouch, 2010; kaynak & kara, 2012). their conscious formation also affects the ability to generate processes of economic growth and development. the competitiveness of regions has been described assuming the transactional point of view. certainly, it can also be assessed from the point of view of the conditions prevailing in the territory that create the endogenous potential. while this point of view defines what the region has to offer, it may not reflect the actual interest of the customers. therefore, competitiveness as attractiveness relative to certain groups should also be considered. this means that the activity of the customers (their tendency to choose a certain location) is also a measure of competitiveness. based on these considerations two definitions were proposed: − competitiveness of the region is the level of its endogenous potential with respect to the level of endogenous potential in other units. endogenous potential is constructed by the attributes of society, economy, environment, public administration institutions and infrastructure. the evaluation of these features can cause changes in the decisions citizens or companies make about a location. it may also affect the willingness of tourists to visit the region (d.1). − competitiveness of the region is its attractiveness, and therefore the ability to attract mobile capital and tourists. if people choose the territory as a place to live and develop their career, and companies invest there and develop innovations, it means that the territory is competitive in the market of mobile capital. if tourists want to visit the territory it means that it is competitive in the tourism market. if these entities choose the given territory, they build a foundation for socioeconomic development, economic growth and improvement of the standard of living (d.2). on the basis of these definitions, two models of competitiveness were constructed and were used to assess the situation in the regions of eastern poland. construction of these models is an attempt to answer the first detailed research question: how are the competitive positions of regions measured? (drq.1). answering this question provided an operationalization of the definitions of d.1 and d.2 describing the competitiveness of regions by creating two hierarchical ahp models. at the level of criteria the main elements of competitiveness were identified, while at the sub-criteria statistical variables were specified. based on the analysis of literature, two different points of view on this phenomenon appeared. the first point of view stems from the belief that the region's competitiveness should be understood as its endogenous potential, or in other words the set of characteristics that describe it. this approach corresponds to the first definition of competitiveness (d.1). ijahp article: strojny, hejman/ahp based multicriteria comparative analysis of regions of eastern poland international journal of the analytic hierarchy process 60 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.373 the analysis does not diminish to the elements of endogenous potential closely associated with the process of production, which is characteristic to economic theories. in this research other areas that may affect the assessment of the quality of life and involve the wider process of socioeconomic development were taken into account, not just economic growth. in this way, a seegi model consists of five basic dimensions: society (ps), economy (pec), environment (pen), government (pg) and infrastructure (pi). the structure of the model is shown in table 1. table 1 the ahp model of endogenic potential (seegi model) endogenic potential of the region (p) ps society ps1 number of associations, social organizations and foundations to 10 000. inhabitants ps2 share of the working age population in the total population ps3 share of pre-working age population in the total population ps4 natural increase per 1 000 inhabitants ps5 average number of medical and dental consultations per capita pec economy pec1 share of people working in the group of people of working age pec2 share of unemployed people in a group of people of working age pec3 gross salary pec4 gross value of fixed assets of companies on the entity pec5 total industrial production sold per capita pen environment pen1 emission of gas and dust pollutants per km 2 pen2 discharge of industrial wastewater per km 2 pen3 waste generated in the industry per km 2 pen4 share of legally protected area in total area pen5 number of natural monuments on the 100 km 2 pg government pg1 own revenues of local and regional government per capita pg2 revenue from eu funds of local and regional government per capita pg3 investment expenditures of local and regional government per capita pg4 current expenditures on salaries of local and regional government per capita pg5 expenditures for debt service of local and regional government per capita pi infrastructure pi1 length of paved local and regional roads per capita pi2 average share of the population using the media networks in the total population pi3 number of inhabitants per 1 bed in a general hospital pi4 number of beds in tourist accommodations per 1 000 inhabitants pi5 museums including branches per 100 km 2 source: own work. it is also worth considering how the endogenous potential affects the behavior of different types of entities which choose the region as the area of their activity. in this way, a model for evaluating the attractiveness of the region was prepared with respect to selected groups of stakeholders – the tcb model. it takes into account the activity of tourists (at), citizens (ac) and enterprises (ab). it is presented in detail in table 2. ijahp article: strojny, hejman/ahp based multicriteria comparative analysis of regions of eastern poland international journal of the analytic hierarchy process 61 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.373 table 2 the ahp model of attractiveness (tcb model) attractiveness of the region (a) at attractiveness in respect to the tourists at1 number of polish tourists at2 number of foreign tourists at3 average duration of the tourist stay at4 expenditures in gastronomy per capita ac attractiveness in respect to the citizens ac1 net international migration per 100 000 inhabitants ac2 net inter-voivodeships migration per 100 000 inhabitants ac3 number of occupied housing per 1 000 inhabitants ac4 number of live births per 1 000 inhabitants ab attractiveness in respect to the business ab1 investments by private sector per entity ab2 number of commercial companies per 1 000 inhabitants ab3 number of commercial companies with foreign capital per 1 000 inhabitants ab4 total expenditures on r&d per entity in economy source: own work. a full assessment of the competitiveness requires consideration of both of the models described above. comparative assessment of the endogenous potential indicates the possibility of improving individual dimensions of reality in the region. it allows a better definition of specialization and the real possibility of maintaining the existing capital or acquiring a new one. rating the attractiveness shows how existing potential is efficiently utilized and fosters a competitive advantage in the tourism market or the markets of mobile capital. 4. a multi-criteria comparative analysis of the competitiveness of eastern poland a polish territory was selected as the test area and included the following regions: lubelskie, podkarpackie, podlaskie świętokrzyskie and warmińsko-mazurskie. these areas are located on the eastern border of the country, and have a relatively low level of development e.g. by gdp5 per capita. these regions are also drained by large metropolitan areas in the country such as warsaw (capital of the country), krakow (in the south) and the tri-city metropolis (gdansk, gdynia and sopot) in the north. the selected regions therefore have much in common and have to face similar developmental problems. in polish literature and regional policy they are often analyzed under the name of eastern poland. at this level, there are statistical databases available which are useful for measuring competitive position. the analysis of competitiveness is multidimensional and requires the use of quite complex methods. on the other hand, all the procedures presented in the analysis should be able to be implemented in the real strategic management. the assumptions of the research presented in this paper meet both of these requirements. the multi-criteria analysis provided below allows a reduction of the analysis of the compounded reality to one aggregated evaluation (dodgson et. al, 2009). 5 gdp – gross domestic product. ijahp article: strojny, hejman/ahp based multicriteria comparative analysis of regions of eastern poland international journal of the analytic hierarchy process 62 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.373 the first step of the analysis was to assess the importance of the elements of competitiveness using to the seegi and tcb models. the significance of the different dimensions of competitiveness and statistical data describing these dimensions were assessed for the individual models using saaty's scale. the group of experts consisted of 7 persons who deal with issues of regional development and work in the department of economics at rzeszow university of technology. they are familiar with the different theories of regional development and understand the situation in the regions of eastern poland as well. as a result of the judgments, the weights of the local criterion (wc), and sub-criteria (wsc) were calculated. they were used to build the aggregate indexes at both the level of dimensions of competitiveness (i(d)) and the level of general index (i(g)). these analyses were performed for both constructed models (tables 3 and 4). table 3 weights of elements of the endogenic potential (seegi model) indexes ps pec pen pg pi ps1 ps2 ps3 ps4 ps5 pec1 pec2 pec3 pec4 pec5 weights (wi, wij) 0.21 0.27 0.18 0.18 0.16 0.08 0.38 0.23 0.23 0.08 0.29 0.16 0.11 0.14 0.30 indexes pen1 pen2 pen3 pen4 pen5 pg1 pg2 pg3 pg4 pg5 pi1 pi2 pi3 pi4 pi5 weights (wi, wij) 0.29 0.28 0.23 0.12 0.08 0.34 0.20 0.23 0.09 0.14 0.28 0.26 0.25 0.12 0.09 source: own work table 4 weights of elements of the attractiveness (tcb model) indexes at ac ab at1 at2 at3 at4 ac1 ac2 ac3 ac4 ab1 ab2 ab3 ab4 weights (wi, wij) 0.16 0.28 0.56 0.13 0.28 0.30 0.29 0.19 0.16 0.18 0.47 0.10 0.21 0.32 0.37 source: own work the basic problem in the construction of aggregated measures is a variety of statistical data used to describe the dimensions of competitiveness. they have different units, and thus need to be normalized. this requires the use of the formulas of normalization. it means the transformation of the original variables (vi) with different units of measurement, into variables of the same units (ni). in the literature, numerous examples of the testing of the different formulas of normalization can be found. this article presents results based on a formula of zero-unitarisation and therefore the reference point is the extent of the range of variables r=max(vi) – min(vi). equation 1 for stimulants and equation 2 for destimulants is used here: ni= vi − min(vi) max(vi) − min(vi) (1) ni= max(vi) − vi max(vi) − min(vi) (2) the normalization procedure made it possible to bring the values of the variables to a range ni∈〈0;1〉. the indexes for the criteria of competitiveness (equation 3) and the general index (equation 4) were constructed on this basis. i(d)i= � vi ∙ wsc n i=1 (3) ijahp article: strojny, hejman/ahp based multicriteria comparative analysis of regions of eastern poland international journal of the analytic hierarchy process 63 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.373 i(g)i= � i(d)i ∙ wc n i=1 (4) in the analysis of trends the single-based indices (isb) were used. the values of variable xi∈(ni, i(d), i(g)) for any year of the time period (yi) were referenced to the value of variable in the based year yb=2009 6. equation 5 is used here: isb(𝑦𝑖)= 𝑥𝑖(𝑦𝑖) 𝑥𝑖(𝑦𝑏) (5) when single-based indexes value below isb(yi)∈〈0;1), it means a regression in the year yi as compared to the base year yb. the value isb(yi)=1 means stagnation, and values isb>1 indicate a development. based on the constructed models, the database of statistic variables was built on the basis of publicly available (non-commercial) official sources. appropriate, simple operations including normalization of data, aggregation of indexes and calculation of single-based indexes have also been provided. the study included all polish regions. this article focuses on the presentation of the situation in the regions of eastern poland. the results of the study were presented with reference to the research questions posed earlier. an attempt to answer the detailed question, “what is the competitive position of particular regions of eastern poland? (drq.2)”, calculated values of all indicators of the endogenous potential and attractiveness of all investigated regions. the results are shown in table 3 (on the level of aggregated indexes). table 5 indexes of competitiveness of regions of eastern poland in 2013 nr region indexes of the competitiveness indexes of endogenic potential (p, pi) indexes of attractiveness (a, ai) p ps pec pen pg pi a at ac ab 1 lubelskie 0.39 0.41 0.11 0.77 0.36 0.45 0.21 0.08 0.45 0.13 2 podkarpackie 0.49 0.69 0.17 0.84 0.45 0.44 0.33 0.15 0.48 0.31 3 podlaskie 0.44 0.51 0.13 0.85 0.46 0.43 0.19 0.07 0.43 0.11 4 świętokrzyskie 0.32 0.28 0.09 0.53 0.40 0.46 0.13 0.13 0.31 0.04 5 warmińsko-mazurskie 0.49 0.82 0.04 0.90 0.44 0.41 0.19 0.16 0.47 0.05 source: own work based on national statistical data (cso). the endogenous potential of the regions of eastern poland has medium or low evaluations. at the level of the main index (p) podkarpackie and warmińskomazurskie (p=0.49) obtained the highest values. the voivodeship with the highest endogenous potential in the country is mazowieckie (p=0.62). the weakest in the country is one of the regions of eastern poland – świętkorzyskie (p=0.32). the analyzed regions still have a high social potential (ps). characterizing them particularly, it is worth listing warmińsko-mazurskie, which is the best region in the country 6 the year of 2009 is the first year of the time series in presented research. ijahp article: strojny, hejman/ahp based multicriteria comparative analysis of regions of eastern poland international journal of the analytic hierarchy process 64 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.373 (ps=0.82) and also podkarpackie (ps=0.69). other regions of eastern poland experience a strong drainage mainly from the metropolis of warsaw. the weakest in the country in the area of society is świętokrzyskie (ps=0.28). a much bigger difference between regions can be observed in the case of economic potential (pec). here mazowieckie (pec=0.94) showed a significant advantage over the rest of the regions. the best of the regions in eastern poland proved to be podkarpackie (pec=0.17). the weakest in poland is another eastern region – warmińsko-mazurskie (pec=0.04). other eastern regions also have very low levels of economic potential. in others, it is the potential of environment (pen). the examined regions, located in the east of the country retained their rural character. the relatively weak economic development therefore, affects the purity of the environment. the best in the country turns out to be warmińsko-mazurskie (pen=0.90). high marks were also received by other regions of eastern poland. the least is świętokrzyskie (pen=0.53). the worst environmental assessment in the country is achieved by śląskie (pen=0.28). another dimension of the potential refers to the efficiency of institutions of regional and local government (pg) which was assessed from the perspective of the budget (public revenues, expenditures and debt). polish eastern regions fall on a level of average or poor in terms of this indicator. the best situation is in podlaskie (pg=0.46), while the worst is in lubelskie (pg=0.36). the highest potential of regional and local government institutions in the country is dolnośląskie (pg=0.66), and the lowest is opolskie (pg=0.28). the activity of public institutions affects the level of potential of infrastructure. this is quite aligned in eastern poland regions who received assessments in scope pi∈〈0.41;0.46〉. the best in poland is zachodniopomorskie (pi=0.63), while the worst is pomorskie (pi=0.31). the conducted study also allows for a more complex analysis. from the point of view of strategic management it is important to identify the strengths and weaknesses of endogenic potential. this information can be accessed by going down to the most detailed level of analysis which consists of the standardized statistical variables. the study assumed that a feature of the endogenous potential is weak if the variable value of standardized ni≤0.30, and strong, if ni≥0.80. a summary of particular regions is detailed in table 6. ijahp article: strojny, hejman/ahp based multicriteria comparative analysis of regions of eastern poland international journal of the analytic hierarchy process 65 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.373 table 6 the analysis of strengths and weaknesses of regions of eastern poland in 2013 strengths (ni≥0.70): weaknesses (ni≤0.30): lubelskie ⋅ ps5. average number of medical and dental consultations per capita. ⋅ pen1. emission of gas and dust pollutants per km2 ⋅ pen2. discharge of industrial wastewater per km2. ⋅ pen3. waste generated in the industry per km2. ⋅ pg3. investment expenditures of local and regional government per capita. ⋅ pg5. expenditures for debt service of local and regional government per capita. ⋅ pi1. length of paved local and regional roads per capita. ⋅ pi3. number of inhabitants per 1 bed in a general hospital. ⋅ ps2. share of the working age population in the total population. ⋅ pec1. share of people working in the group of people of working age. ⋅ pec3. gross salary. ⋅ pec4. gross value of fixed assets of companies on the entity. ⋅ pec5. total industrial production sold per capita. ⋅ pen5. number of natural monuments on the 100 km2. ⋅ pg1. own revenues of local and regional government per capita. ⋅ pg2. revenue from eu funds of local and regional government per capita. ⋅ pi2. average share of the population using the media networks in the total population. ⋅ pi4. number of beds in tourist accommodations per 1 000 inhabitants. ⋅ pi5. museums including branches per 100 km2. podkarpackie ⋅ ps3. share of pre-working age population in the total population. ⋅ ps4. natural increase per 1 000 inhabitants. ⋅ pen1. emission of gas and dust pollutants per km2. ⋅ pen2. discharge of industrial wastewater per km2. ⋅ pen3. waste generated in the industry per km2. ⋅ pg3. investment expenditures of local and regional government per capita. ⋅ pg5. expenditures for debt service of local and regional government per capita. ⋅ pec1. share of people working in the group of people of working age. ⋅ pec2. share of unemployed people in a group of people of working age. ⋅ pec3. gross salary. ⋅ pec5. total industrial production sold per capita. ⋅ pen5. number of natural monuments on the 100 km2. ⋅ pg1. own revenues of local and regional government per capita. ⋅ pi4. number of beds in tourist accommodations per 1 000 inhabitants. ⋅ pi5. museums including branches per 100 km2. podlaskie ⋅ ps5. average number of medical and dental consultations per capita. ⋅ pen1. emission of gas and dust pollutants per km2. ⋅ pen2. discharge of industrial wastewater per km2. ⋅ pen3. waste generated in the industry per km2. ⋅ pg3. investment expenditures of local and regional government per capita. ⋅ pg5. expenditures for debt service of local and regional government per capita. ⋅ pi1. length of paved local and regional roads per capita. ⋅ pec1. share of people working in the group of people of working age. ⋅ pec3. gross salary. ⋅ pec4. gross value of fixed assets of companies on the entity. ⋅ pec5. total industrial production sold per capita. ⋅ pen5. number of natural monuments on the 100 km2. ⋅ pg1. own revenues of local and regional government per capita. ⋅ pg2. revenue from eu funds of local and regional government per capita. ⋅ pi2. average share of the population using the media networks in the total population. ⋅ pi4. number of beds in tourist accommodations per 1 000 inhabitants. ⋅ pi5. museums including branches per 100 km2. ijahp article: strojny, hejman/ahp based multicriteria comparative analysis of regions of eastern poland international journal of the analytic hierarchy process 66 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.373 świętokrzyskie ⋅ ps5. average number of medical and dental consultations per capita. ⋅ pen1. emission of gas and dust pollutants per km2. ⋅ pen4. share of legally protected area in total area. ⋅ pg3. investment expenditures of local and regional government per capita. ⋅ pg5. expenditures for debt service of local and regional government per capita. ⋅ pi1. length of paved local and regional roads per capita. ⋅ ps2. share of the working age population in the total population. ⋅ ps3. share of pre-working age population in the total population. ⋅ ps4. natural increase per 1 000 inhabitants. ⋅ pec1. share of people working in the group of people of working age. ⋅ pec2. share of unemployed people in a group of people of working age. ⋅ pec3. gross salary. ⋅ pec4. gross value of fixed assets of companies on the entity. ⋅ pec5. total industrial production sold per capita. ⋅ pen5. number of natural monuments on the 100 km2. ⋅ pg1. own revenues of local and regional government per capita. ⋅ pg2. revenue from eu funds of local and regional government per capita. ⋅ pi2. average share of the population using the media networks in the total population. ⋅ pi4. number of beds in tourist accommodations per 1 000 inhabitants. ⋅ pi5. museums including branches per 100 km2. warmińsko-mazurskie ⋅ ps1. number of associations, social organizations and foundations to 10 000 inhabitants. ⋅ ps2. share of the working age population in the total population. ⋅ ps3. share of pre-working age population in the total population. ⋅ pen1. emission of gas and dust pollutants per km2. ⋅ pen2. discharge of industrial wastewater per km2. ⋅ pen3. waste generated in the industry per km2. ⋅ pg3. investment expenditures of local and regional government per capita. ⋅ pec1. share of people working in the group of people of working age. ⋅ pec2. share of unemployed people in a group of people of working age. ⋅ pec3. gross salary. ⋅ pec4. gross value of fixed assets of companies on the entity. ⋅ pec5. total industrial production sold per capita. ⋅ pg1. own revenues of local and regional government per capita. ⋅ pg2. revenue from eu funds of local and regional government per capita. ⋅ pi5. museums including branches per 100 km2. source: own work. the list of strengths and weaknesses precisely indicate areas of improvement and possible foundations for further development of the regions surveyed. undoubtedly, one of the greatest assets of the investigated regions is a clean environment. the development strategies in these regions in the long term must therefore focus on industries that use this element of potential (i.e. organic agriculture or tourism). it should also be noted that the development potential of the industry should not cause hazards for clean air, soil or water. in most cases, regions of eastern poland also have a young population. this is a strength now, but is affected by high foreign or interregional migration which will greatly weaken its social potential in the future. it is worth noting that all of the tested regions have a problem with building advantages based on the identified strengths. the younger part of society creates an attractive labor market. unfortunately, both the level of economic activity and unemployment are among the weaknesses of the whole analyzed area of eastern poland. it is worth noting that a clean environment is also not used in a sufficient manner for the tourist market. the surveyed regions have an underdeveloped tourist infrastructure (accommodation, museums, etc.) in comparison to the rest of the country which weakens its attractiveness for the tourism market. ijahp article: strojny, hejman/ahp based multicriteria comparative analysis of regions of eastern poland international journal of the analytic hierarchy process 67 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.373 the second assessment is based on the tcb model and concerns the analysis of the attractiveness of the regions in selected markets – the tourism market, the market of mobile human capital and the market of mobile investment capital. it is worth emphasizing that there is a high differentiation of attractiveness of regions in the country. at the level of the main index (a) mazowieckie (a=0.94) showed a significant advantage over others. another region, małopolskie, showed significantly lower scores (a=0.59). in this context, the attractiveness of regions of eastern poland should be assessed as low or average. the best is podkarpackie (a=0.33), while the weakest is świętokrzyskie (a=0.19) which is also the least attractive region in the country. the differences in assessing the attractiveness of the tourism market are much smaller. the two leading regions are mazowieckie (at=0.69) and małopolskie (at=0.59). eastern regions are weak here – at∈〈0.07; 0.16〉. mazowieckie (ac=0.97) is the best in poland with regards to attractiveness with respect to citizens while the worst is opolskie (ac=0.13). the assessment of the regions of eastern poland is at an average level – ac∈〈0.43;0.48〉, except świętokrzyskie which scored less (ac=0.31). the last aggregated indicator concerns the evaluation of the attractiveness with respect to companies. here again, by far the best is mazowieckie (ab=1.00). another region, dolnośląskie, received a much lower score (ab=0.39). in this background podkarpackie (ab=0.31) did relatively well – the core place of this region as the fifth in the country. other regions of eastern poland are much less attractive. the weakest were warmińsko-mazurskie (ab=0.05) and świętokrzyskie (ab=0.04). similar to endogenous potential, attractiveness can also be analyzed considering more detailed information. this analysis considers the market situation, and therefore consists of external factors of development. therefore, they seek market opportunities and threats (table 7). ijahp article: strojny, hejman/ahp based multicriteria comparative analysis of regions of eastern poland international journal of the analytic hierarchy process 68 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.373 table 7 the analysis of market opportunities and threats of regions of eastern poland in 2013 opportunities (ni≥0.70): treats (ni≤0,30): lubelskie ⋅ ac1. net international migration per 100 000 inhabitants. ⋅ at1. number of polish tourists. ⋅ at2. number of foreign tourists. ⋅ at3. average duration of the tourist stay. ⋅ at4. expenditures in gastronomy per capita. ⋅ ac2. net inter-voivodeships migration per 100 000 inhabitants. ⋅ ab2. number of commercial companies per 1 000 inhabitants. ⋅ ab3. number of commercial companies with foreign capital per 1 000 inhabitants. ⋅ ab4. total expenditures on r&d per entity in economy. podkarpackie ⋅ ac1. net international migration per 100 000 inhabitants. ⋅ ab1. investments by private sector per entity. ⋅ ⋅ at1. number of polish tourists. ⋅ at2. number of foreign tourists. ⋅ at4. expenditures in gastronomy per capita. ⋅ ac2. net inter-voivodeships migration per 100 000 inhabitants. ⋅ ac3. number of occupied housing per 1 000 inhabitants. ⋅ ab2. number of commercial companies per 1 000 inhabitants. ⋅ ab3. number of commercial companies with foreign capital per 1 000 inhabitants. podlaskie ⋅ ac1. net international migration per 100 000 inhabitants. ⋅ at1. number of polish tourists. ⋅ at2. number of foreign tourists. ⋅ at3. average duration of the tourist stay. ⋅ at4. expenditures in gastronomy per capita. ⋅ ac2. net inter-voivodeships migration per 100 000 inhabitants. ⋅ ac4. number of live births per 1 000 inhabitants. ⋅ ab1. investments by private sector per entity. ⋅ ab2. number of commercial companies per 1 000 inhabitants. ⋅ ab3. number of commercial companies with foreign capital per 1 000 inhabitants. ⋅ ab4. total expenditures on r&d per entity in economy. świętokrzyskie ⋅ ac1. net international migration per 100 000 inhabitants. ⋅ at1. number of polish tourists. ⋅ at2. number of foreign tourists. ⋅ at4. expenditures in gastronomy per capita. ⋅ ac2. net inter-voivodeships migration per 100 000 inhabitants. ⋅ ac4. number of live births per 1 000 inhabitants. ⋅ ab1. investments by private sector per entity. ⋅ ab2. number of commercial companies per 1 000 inhabitants. ⋅ ab3. number of commercial companies with foreign capital per 1 000 inhabitants. ⋅ ab4. total expenditures on r&d per entity in economy. ijahp article: strojny, hejman/ahp based multicriteria comparative analysis of regions of eastern poland international journal of the analytic hierarchy process 69 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.373 warmińsko-mazurskie ⋅ ac1. net international migration per 100 000 inhabitants. ⋅ at1. number of polish tourists. ⋅ at2. number of foreign tourists. ⋅ at3. average duration of the tourist stay. ⋅ at4. expenditures in gastronomy per capita. ⋅ ac2. net inter-voivodeships migration per 100 000 inhabitants. ⋅ ab1. investments by private sector per entity. ⋅ ab2. number of commercial companies per 1 000 inhabitants. ⋅ ab3. number of commercial companies with foreign capital per 1 000 inhabitants. ⋅ ab4. total expenditures on r&d per entity in economy. source: own work in all the surveyed regions, the analysis did not allow for a clear identification of market opportunities which would provide a highly competitive position. instead, all of the investigated areas showed threats to development rather than opportunities. there has been a very weak interest in the studied regions in the tourist market despite the fact that the potential of the natural environment is significant. in the market of mobile human capital there is a particular problem of interregional migration mainly due to the influence of warsaw. this metropolis is also a strong magnet for investment capital. the examined regions are unable to obtain satisfactory levels in attractiveness to both domestic as well as foreign companies. in this area, only podkarpackie is a region with an acceptable level of attractiveness. potentially, a great opportunity for the development of this region is the investment activity of the private sector (ab1=0.78) and expenditure on r&d (ab4=0.59). strategic decision-making in managing the development of the regions also needs to take into account trends during all of the time period from 2009-2013. to describe these trends, single-based indexes were calculated for each year using 2009 as the base year. the choice of date is dictated by both the availability of statistical data (in the case of variables in the seegi model) as well as the onset of the economic crisis. the selected time period may therefore provide interesting information on the impact of the crisis on the regions studied. the results are shown in table 8. table 8 single-based indexes for indexes of competitiveness of regions of eastern poland in 2013 with respect to 2009 nr region single based indexes for indexes of the competitiveness single based indexes for indexes of endogenic potential (p, pi) single based indexes for indexes of attractiveness (a, ai) is(p) is(ps) is(pec) is(pen) is(pg) is(pi) is(a) is(at) is(ac) is(ab) 1 lubelskie 1.13 1.07 1.02 1.00 2.16 1.05 0.86 0.81 0.93 0.78 2 podkarpackie 1.14 1.18 1.07 1.01 1.46 1.11 1.48 0.98 1.08 2.30 3 podlaskie 1.00 1.29 0.90 1.00 0.87 0.91 1.02 0.97 0.93 1.26 4 świętokrzyskie 0.94 0.90 0.97 1.00 0.86 1.00 0.62 1.14 0.79 0.27 5 warmińsko-mazurskie 1.08 1.04 0.82 1.00 1.51 1.10 0.70 0.82 0.78 0.42 source: own work based on national statistical data (cso) ijahp article: strojny, hejman/ahp based multicriteria comparative analysis of regions of eastern poland international journal of the analytic hierarchy process 70 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.373 three of the regions studied improved their endogenous potential (p) in the period since 2009. the fastest increase is observed in the podkarpackie and lubelskie regions. an indicator of the overall potential (p) increased by 14% (the highest growth of potential in the country) and 13% respectively. during the same time, the potential of świętokrzyskie decreased by 6%. the worst situation in the country was identified in the lubuskie region, where the potential had fallen by 8%. the fastest growing social potential (ps) in poland was evident in the improvement in the podlaskie (∆ps=+29%) and podkarpackie (∆ps=+18%) regions. the worst situation is in świętokrzyskie (∆ps=-10%), while the worst in the country is łudzkie (∆ps=39%). when it comes to the economic potential (pec), in the analyzed period, it improved in lubelskie (∆pec=+2%) and podkarpackie (∆pec=+7%). warmińskomazurskie (∆pec=-18%) had the fastest decreasing rate in the country. it is worth noting that the fastest growth rate was recorded in the economic potential of lubuskie (∆pec=+20%). this is a result of significant dynamics of change characterized by the institutional capacity of local government (pg). the index has more than doubled in lubelskie (∆pg=+116%) and it was the best result in the country. podkarpackie (∆pg=+46%) and warmińsko-mazurskie (∆pg=+51%) also significantly improved their situation. in podlaskie and świętokrzyskie the situation worsened. the biggest drop in the country was recorded in opolskie (∆pg=-24%). the last dimension is the potential of infrastructure (pi). its relative assessment in the case of the regions podkarpackie and warmińsko-mazurskie has increased – by 11% and 10% respectively. in other regions of eastern poland it declined. only a few regions were clearly distinguished in the examined years regarding the attractiveness of markets (a). the two most attractive regions– mazowieckie and małopolskie – improved their position by 3% and 4% respectively. the spectacular growth of the overall attractiveness index was also recorded in podkarpackie (∆a=+48%). the eastern region to most lose its attractiveness was świętokrzyskie (∆a=-38%). at the same time, this region improved its attractiveness in the tourist market (at) by 14%. this is the highest growth rate in the country in this market. other regions of the eastern poland that lost the most attractiveness were lubelskie (∆at=19%) and warmińsko-mazurskie (∆at-18%). lubuskie (∆at=-43%) lost the most competitiveness in the tourism market. in the case of the market of mobile human capital (ac), the fastest improvement in competitiveness was mazowieckie (∆ac=+12%). from the regions of eastern poland, the situation has improved for podkarpackie (∆ac=+8%). the attractiveness of the other eastern regions has dropped from 3% in podlaskie to 22% in warmińsko-mazurskie (the worst result in the country). podkarpackie very quickly improved its situation in the market of mobile investment capital (ab). during the analyzed period, the increase rate was ∆ab=+130%. a significant increase (∆ab=+ 26%) was also observed in the region of podlaskie. the competitiveness of other regions in this market fell with the most being in świętokrzyskie (∆ab=-73%). even a simple analysis of the dynamics of change can significantly enrich the management information in relation to statistical analysis. it is worth noting that the examined regions benefit from the opportunities offered by the european union's cohesion policy to maintain endogenous potential and even improve it. for the period 2009-2013 the podkarpackie region showed improvement in all areas of the ijahp article: strojny, hejman/ahp based multicriteria comparative analysis of regions of eastern poland international journal of the analytic hierarchy process 71 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.373 endogenous potential. this region is also unique in terms of improving its attractiveness in particular in relation to businesses. this observation is very useful for the other eastern regions which should look for their path of development on the basis of the benchmark of podkarpackie. the intensive support which regional and local authorities ensure to the innovative sectors (especially connected with the aviation industry) is especially interesting. this direction is based on the well-recognized attributes of the region resulting from the historical traditions of this industry in the analyzed region. 5. conclusions this study focused on two aspects. the first is the cognitive aspect which is concerned with the assessment of the competitive position of regions of eastern poland. the second is the methodological aspect which instead preferred to present the ahp method in the context of the preparation of strategic information in public administration. the relatively low endogenous potential (especially in the economic dimension) and low attractiveness in terms of basic stakeholder groups should be noted in reference to the cognitive aspect. among the analyzed regions, podkarpackie stands out positively and świętokrzyskie negatively. it is worth noting that a detailed swot analysis based on verified models indicates the existence of significant barriers of development in all regions of eastern poland. both the weaknesses in their potential and processes that threaten their competitive position on markets are observed. the possibility of adapting the method of ahp to the procedure of strategic management in public administration should be mentioned in reference to the methodological aspect. the research approach used in this article involves one of many possible forms of application of this method to the practice of strategic management. the example was limited to the activities related to the preparation of strategic information. in this role, the ahp method, linked to the process of normalization of statistical variables allows the performance of a simple multi-criteria comparative analysis. the statistical data used provides a general but comprehensive picture of the situation in a territorial unit, especially using both the seegi and tcb model. attention should also be paid to the potential problems associated with the application of the approach proposed in the article. one of them is the limitation of statistical variables and a procedure for updating the database. the use of the aggregation of variables requires the same time period for all the variables used. the databases are updated with some delay of approximately one year which means that it is possible to prepare mainly an ex-post analysis. in this situation, an acceptable solution might be to forecast trends for 2-3 consecutive years compared to the last year in the available time period. the testing of methods of forecasting and their inclusion in the procedure of analysis presented in this article is the next step in the planned research. ijahp article: strojny, hejman/ahp based multicriteria comparative analysis of regions of eastern poland international journal of the analytic hierarchy process 72 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.373 references abbasi g.y., al-mharmah h. (2000). project management practice by the public sector in a developing country, international journal of project management, 18(2), 105-109. doi:10.1016/s0263-7863(98)00074-x abraham, s. (2005). stretching strategic thinking, strategy & leadership, 33(5), 5-12. doi: http://dx.doi.org/10.1108/10878570510616834 anand, g., kodali, r. (2008). benchmarking the benchmarking model. benchmarking: an international journal, 15(3), 257-291. doi: http://dx.doi.org/10.1108/14635770810876593 angiola, n., bianchi, p. (2015). public managers’ skills development for effective performance management: empirical evidence from italian local governments. public management review, 17(4), 517-542. doi: 10.1080/14719037.2013.798029 annoni, p., dijkstra, l. (2013) eu regional competitiveness index. rci 2013. brussels: economic analysis unit, dg regio. balm, g.j. (1992). benchmarking. a practitioner’s guide for becoming and staying best of the best. schaumburg: qpma press. bogan ch.e., english m.j. (1994). benchmarking for best practices. winning through innovative adaptation. new york: mcgraw-hill. bowerman, m., ball, a., (2000).the modernisation and importance of government and public services: great expectations: benchmarking for best value. public money and management, 20(2), 21-26. doi:10.1111/1467-9302.00207 bowerman, m., ball, a., francis, g. (2001). benchmarking as a tool for the modernization of local government, financial accountability & management, 17(4), 321-329. doi: 10.1111/1468-0408.00136 bowerman, m., francis, g., ball, a., fry, j. (2002). the evolution of benchmarking in uk local authorities, benchmarking: an international journal, 9(5), 429-449. doi: http://dx.doi.org/10.1108/14635770210451455 boxwell, r.j. (1994). benchmarking for competitive advantage. new york: mcgrawhill. carmen, a. (2014). management by objectives. ovidius university annals, economic sciences series, 14(1), 433-436. cm – council of ministry (2009). założenia systemu zarządzania rozwojem. warszawa: rada ministrów. crouch, g. i. (2010). destination competitiveness: an analysis of determinant attributes. journal of travel research, 50(1), 27-45. doi:10.1177/0047287510362776 http://dx.doi.org/10.1016/s0263-7863(98)00074-x ijahp article: strojny, hejman/ahp based multicriteria comparative analysis of regions of eastern poland international journal of the analytic hierarchy process 73 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.373 cso – central statistic office (2015). area and population in the territoral profile in 2015. warsaw: cso, methodology, standards and registers department. de luca, s. (2014). public engagement in strategic transportation planning: an analytic hierarchy process based approach, transport policy, 33, 110-124. doi:10.1016/j.tranpol.2014.03.002 dess, g., miller, a. (1993). strategic management. new york: mcgraw-hill. doi: 10.1177/002218569503700114 district act (1998). warsaw: parliament. dodgson, j.s., spackman, m., pearman, a., phillips, l.d. (2009). multi-criteria analysis: a manual. london: department for communities and local government. dolnicki, b. (2012). samorząd terytorialny. warsaw: wolters kluwer. drucker, p.f. (1954). the practice of management. new york: harper & row. doi: 10.1002/hrm.3930240410 ec – european commission (2014). european competitiveness report 2014. helping firms grow. luxemburg: european union. eden, c., ackermann, f. (2013). making strategy: the journey of strategic management. london, thousand oaks, new delhi: sage publications. feurer, r., chaharbaghi, k. (1994), defining competitiveness: a holistic approach, management decision, 32(2), 49-58. doi: http://dx.doi.org/10.1108/00251749410054819 flak, o., głód, g. (2014). koncepcja i przykład metody badania konkurencyjności przedsiębiorstw. zeszyty naukowe uniwersytetu ekonomicznego w katowicach, 187(14), 87-100. georgiadou, a., kokkinakos, p., panopoulos, d., koussouris, s., askounis, d. (2013). a multicriteria methodology for the selection and prioritisation of public services. ifip advances in information and communication technology, 339, 325-337. doi: 10.1007/978-3-642-37437-1_27 gierszewska, g., romanowska, m., (1997). analiza strategiczna przedsiębiorstwa. warsaw: polskie wydawnictwo ekonomiczne. hill, c., jones, g., schilling, m. (2014). strategic management: theory: an integrated approach. stamford: cengage learning. ipma – international project management association (2013). ipma organizational competence baseline – the standard for moving organizations forward. zurich: ipma. http://dx.doi.org/10.1016/j.tranpol.2014.03.002 http://dx.doi.org/10.1108/00251749410054819 ijahp article: strojny, hejman/ahp based multicriteria comparative analysis of regions of eastern poland international journal of the analytic hierarchy process 74 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.373 ivanov, s., webster, c. (2013). globalisation as a driver of destination competitiveness. annals of tourism research, 43, 628-633. doi:10.1016/j.annals.2013.07.010 kaynak, e., kara, a. (2012). assessing tourism market potential in a dynamic emerging economy: theoretical and empirical insights from cambodia. asia pacific journal of marketing and logistics, 24(2), 199-221. doi: http://dx.doi.org/10.1108/13555851211218020 krugman, p.r. (1995). development geograpy and economic theory. cambridge– london: mit-press. krupski, r. (2007). zarządzanie strategiczne. koncepcje – metody. wrocław: akademia ekonomiczna we wrocławiu. kuczewska j. (2007). europejska procedura benchmarking. programy i działania. warsaw: parp. kuipers, b. s., higgs, m., kickert, w., et al. (2014). the management of change in public organizations: a literature review. public administration, 92(1), 1-20. doi: 10.1111/padm.12040 kyriakopoulos, g. (2012). half a century of management by objectives (mbo): a review. african journal of business management, 6(5), 1772-1786. linnemann, h., pronk, j.p., tinbergen, j. (1965). convergence of economic systems in east and west. rotterdam: netherlands economic institute. lucas, r.e., (1988). on the mechanics of economic development. journal of monetary economics, 22, 3-42. doi:10.1016/0304-3932(88)90168-7 lucas, r.e., (1990). why doesn’t capital flow from rich to poor countries? the american economic review, 80, 92-96. lucato, w. c., júnior, m. v., filho, j. d. (2015). the competitive distance as a tool for competitiveness improvement. international journal of business excellence, 8(2), 181196. doi: http://dx.doi.org/10.1504/ijbex.2015.068207 martin, r. (2003). a study on the factors of regional competitiveness. a final report for the european commission. brusel: directorate-general regional policy. mintzberg, h., quinn, j.b., ghoshal, s. (1998). the strategy process. london: prentice hall. moon, h.ch., newman, s.p. (1995). competitiveness of product, firm, industry, and nation in a global business. international business journal, 5(1), 37-43. doi: http://dx.doi.org/10.1108/eb046319 moore p.d., staton t (1981). management by objectives in american cities. public personnel management journal, 10, 223-232. doi: 10.1177/009102608101000204 http://dx.doi.org/10.1016/j.annals.2013.07.010 http://dx.doi.org/10.1016/0304-3932(88)90168-7 ijahp article: strojny, hejman/ahp based multicriteria comparative analysis of regions of eastern poland international journal of the analytic hierarchy process 75 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.373 municipality act (1990). warsaw: parliament. odiorne g.s. (1965). management by objectives: a system of managerial leadership. new york: pitman publications. ogc – office of government commerce, (2009). managing successful projects with prince2tm. london: tso. perroux, f. (1950). economic space: theory and applications. the quarterly journal of economics, 89-104. doi: 10.2307/1881960 pmi – project management institute (2013). a guide to the project management body of knowledge (pmbok® guide), 5th edition. pennsylvania pmi. poister t.h., streib g. (1995) mbo in municipal government: variations on a traditional management tool. public administration review, 55, 48-56. doi: 10.2307/976827 porter, m.e. (1996). what is strategy? harvard business review, 61-78. potoczek, a. (2013) zarządzanie jednostką terytorialną. wybrane zagadnienia. bydgoszcz: wydawnictwo uczelniane wyższej szkoły gospodarki. prusak a., strojny j., stefanów p. (2014), analityczny proces hierarchiczny (ahp) na skróty – kluczowe pojęcia i literatura. humanities and social sciences, 4, 179-192. doi:10.7862/rz.2014.hss.66 public financial act (2009). warsaw: parliament. regional and local government’s income act (2003). warsaw: parliament. romer, p.m. (1998). increasing returns and long-run growth. the journal of political economy, 94(5), 1002-1037. doi: 10.1086/261420 romer, p.m. (1994). the origins of endogenous growth, journal of economic perspectives, 8(1), 3-22. doi: 10.1257/jep.8.1.3 saaty, t. l. (2008). decision making with the analytic hierarchy process. international journal of services sciences, 1(1), 83-98. doi: http://dx.doi.org/10.1504/ijssci.2008.017590 saaty, t.l., 1980. the analytic hierarchy process. new york: mcgraw-hill. doi: 10.1080/00137918308956077 saaty, t.l., forman, e.h., (1992), the hierarchon: a dictionary of hierarchies. pittsburgh: rws publications. sayyadi, g., awasthi, a. (2012). ahp-based approach for location planning of pedestrian zones: application in montréal, canada. journal of transportation ijahp article: strojny, hejman/ahp based multicriteria comparative analysis of regions of eastern poland international journal of the analytic hierarchy process 76 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.373 engineering, 139(2), 239-246. doi: http://dx.doi.org/10.1061/(asce)te.19435436.0000493 steptoe-warren, g., howat, d. and hume, i. (2011), strategic thinking and decision making. journal of strategy and management, 4(3), 238-250. doi: http://dx.doi.org/10.1108/17554251111152261 strategor, (2001). zarządzanie firmą. strategie, struktury, decyzje, tożsamość. pwe, warszawa. strojny j. (2012). innowacyjne zarządzanie regionem, powiatem i gminą. rzeszow: rzeszow university of technology. thompson, a., strickland, a. (1993). strategic management: concepts and cases. burr ridge: irwin. van dooren, w., bouckaert, g., halligan, j. (2015). performance management in the public sector. london:routledge. doi: 10.1177/0020852308099513 voivodeship act (1998). warsaw: parliament. walas-trębacz, j. (2013). metody i mierniki oceny konkurencyjności przedsiębiorstwa. przegląd organizacji, (4), 33-40. wójcik, p. (2008). dywergencja czy konwergencja: dynamika rozwoju polskich regionów. studia regionalne i lokalne, 32(2), 41-60. http://dx.doi.org/10.1061/(asce)te.1943-5436.0000493 http://dx.doi.org/10.1061/(asce)te.1943-5436.0000493 http://dx.doi.org/10.1108/17554251111152261 ahp based multicriteria comparative analysis of regions of eastern poland 1. introduction 2. managing the regional development 3. competitiveness of regions – theory and ahp models 4. a multi-criteria comparative analysis of the competitiveness of eastern poland 5. conclusions references ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 470 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting in a private small hydropower plants investments selection problem burak omer saracoglu orhantepe mahallesi, tekel caddesi, geziyolu sokak, 34865 dragos, kartal, istanbul, turkey e-mail: burakomersaracoglu@hotmail.com 1 abstract private small hydropower plant investments are more challenging than medium and large private hydropower plant investments when considering engineering analysis. one of the necessary tasks is the selection of the most appropriate private small hydropower investment amongst several alternatives. brokerage, consultancy and private investor activities are a few examples of these kinds of real world activities. there are many multi-criteria decision making (mcdm) or multiple-criteria decision analysis (mcda) methods, and different researchers prefer different methods this research study investigates three methods at once for the same problem: analytic hierarchy process (ahp), elimination and choice translating reality (electre) iii and electre iv. a comparative investigation is conducted on one simple unique selection model for all three methods. this unified model has seven objective factors (catchment area, project runoff, net head, flow rate, firm energy, secondary energy, investment cost). an additional comparison is also made on the criteria weighting amongst equal objective, shannon’s entropy objective and saaty’s subjective criteria weighting. the simplistic unified model is structured in three levels. there are seven alternatives in the predevelopment investment stage. the super decisions software and the electre iii-iv software are implemented in this study. the pairwise comparisons of factors results in an inconsistency value of 0,09511 which is 7 th in the saaty's ahp weighting. the shannon’s entropy objective weighting represents a difference in the rate of the expert decision maker’s evaluations ranging between -67% and 671%. the equal weighting represents a rate change between -64% and 626%. both approaches can't represent human judgments (here only one expert decision maker) well in this model. fortunately, the same 1 acknowledgements: the author would sincerely like to express his deepest thankfulness to mrs. rozann whitaker saaty for her guidance, help and pre-review, to dr. david weiss for his help on the carrot 2 , also to the ren21 secretariat, and to mr. steinar hansen on behalf of the smakraft for the digital graphics and photos. ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 471 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 alternative (alternative 3) ranks first. these findings promise that further studies on this subject can give some clues for the development of an autonomous computer based intelligent decision support system. some observed pros and cons of these methods are also presented in this study. the observations and critical issues are presented during modeling, application, evaluation and analysis to help researchers, consultants, and readers in the small hydropower investments research and practical fields. keywords: ahp; carrot 2 ; electre iii; electre iv; investment; shannon’s entropy; small hydropower plant 1. introduction the importance and effectiveness of renewable energy in solving climate change and economic growth problems have been accepted by governing bodies in many countries in the world today. the renewable energy policy network for the 21st century (ren21), which is an international non-profit association at the united nations environment programme (unep), has recorded that 145 countries in the world have renewable energy support policies and 164 countries have renewable energy targets set by early 2015 (figure 1). accordingly, this research study aims to contribute to scientific studies and developments in this field. figure 1. countries with renewable energy policies and targets (source: ren21, 2015) there are various renewable power generation types that produce clean energy, such as concentrating solar thermal, geothermal, hydro, photovoltaics, and wind (ren21, 2015). today, hydropower is the most widely installed renewable energy type. the history of hydroelectric power plants started in the cragside country house in northumberland, england in 1878. in those days, this technology could only power a single lamp. the first hydropower plant, generating 12,5 kilowatts (kw) in a private and ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 472 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 commercial customer system (equivalent 250 lights), was commissioned on the fox river in wisconsin, usa in 1882 (iea hydropower, 2015; iha, 2015; nha, 2015; schmidt & denny, 2004). this long history has placed hydropower technology in an advantageous position amongst the other renewable energy technologies. technical and technological maturity, very long lifespan and economic competitiveness are some of these advantages (hall, 2003; ihaicld, 2000; lako et al., 2003). thus, hydropower technology should always be on the radar of private investors (power plant portfolio) throughout the world. hydropower plants are usually classified according to their installed capacities. one of the detailed classification approaches splits the installed capacities into the categories large, medium, small, mini, micro and pico. the upper limits and lower limits of these groups differ in several regions. for instance, the upper limit (25.000 kw) and the lower limit (1.000 kw) are common for the small hydropower plants in india. in the united states of america, these limits are presented as 1.000 kw and 30.000 kw (bajaj et al., 2007; berakovic et al., 2009; dragu et al., 2010; erec, 2012; kurien & sinha, 2006; moreire & poole, 1993; saxena, 2007). in this study, the small hydropower plants are accepted as 1 mw < p ≤ 10 mw. this range was chosen based on the installed capacities in the electricity generation licensing procedures in turkey by the electricity market law 6446, the regulation on the unlicensed electricity generation in the electricity market, and the communication concerning the application of regulation on the unlicensed electricity generation on the electricity market of the republic of turkey (official gazette 28603, 2015; official gazette 28783, 2015), and the definition of small hydropower plants supported by the european commission (ec) and the european small hydropower association (esha) (ec, 2014; esha et al., 2008). small hydropower plants are more challenging than large and medium hydropower plants, because they have little or no storage reservoir. their investment analysis and decisions regarding them are more difficult than others. consequently, this study focuses on private small hydropower plant investment decisions. two private small hydropower plants are presented in figure 2. figure 2. two private small hydropower plants in norway: vassvik 3.100 kva (left), ytre alsaker 5.490 kva (right) (source: småkraft as http://www.smaakraft.no/) this research study concentrates on turkey and is conducted there for three main reasons. the first reason is that it is the homeland of the author. the second reason is the http://www.smaakraft.no/ ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 473 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 level of interest and the excitement of private investors (foreign and domestic) about small hydropower plants in turkey demonstrated by the number of license applications. there are more than 200 private small hydropower plant license applications as of september 2013 according to the official website of the emra (republic of turkey energy market regulatory authority) (emra, 2013). the final reason is the characteristics of the private investors' in turkey. simple observations made before this study show that these investors have very different organizations, management approaches, and capabilities. however, similar research studies on this core subject can be organized and performed in different countries because of the existence of small hydropower plants in 148 countries or territories (total 75 gw in 2011/2012: asia 65%, europe 16%, americas 13%, africa 5%, oceania 1%) worldwide (unido and icshp, 2013). in short, this research study isn't particular, specific and unique to turkey, but the comparative case is performed in turkey. another important issue is the knowledge creation, the knowledge acquisition, and the problem statement and scope of this study. there is only one expert decision maker or "intellectual capital" in this application. the declarative, the procedural, the causal, the conditional, the relational, and the pragmatic knowledge types are based on the knowledge created by several resources and the ones acquired as shown in figure 3. the research scope focuses on the second hand tasks such as the brokerage, the consultancy and the private investors' activities. figure 3. data, information, knowledge environment and model in this study (icon: signore_green, 2015; see terms for knowledge: alavi & leidner, 1999) (gis: geographic information system, emra: republic of turkey energy market regulatory authority, dsi: general directorate of state hydraulic works) ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 474 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 finally, this research study was inspired by previous research studies by saracoglu (2015a,b,c) on decision making and small hydropower plant investments. in these studies, electre iii (electre: elimination and choice translating reality/elimination et choix tradusiant la realite), electre iv, shannon’s entropy, ahp (analytic hierarchy process), and dexi (decision expert for education) were implemented in the investment selection of small hydropower plants. moreover, research by saaty and daji (2015), and zanakis et al., (1998) inspired the idea for this research. the main research aim of this study is very well described by a quote: “good decisions come from experience, and experience comes from bad decisions” (possibly by mark twain, american novelist and journalist). according to the author's point of view, these kinds of studies are very important for understanding small hydropower plants, their investments and the decision making methods. researchers and practitioners in this field can learn from this approach and take into account the critical success and failure issues based on the outcomes and findings of this study in the short and long term. in this paper, a comparison is made on the findings of the combination of electre iii, electre iv, and ahp methods with shannon’s entropy, saaty’s ahp, and equal weighting approaches. the comparison is based on a unique model, and therefore no model differences exist to be investigated. moreover, the alternatives are taken from real life applications and are the same for each combination of methods, and therefore no alternative differences exist to be investigated. above all, this comparative study will hopefully contribute to the previous studies in the literature (saracoglu, 2015a,b,c) in order to develop an autonomous computer based intelligent decision support system for the real world cases. 2. previous research in the literature key terms were defined in order to get the best hit rates on online scientific databases and journal websites. moreover, the search activity was performed only on titles, abstracts and keywords at first. this preference eliminates all of the inappropriate documents and will hopefully only show documents where both methods were discussed in the same document. the full text search option has produced widely dispersed articles in previous studies. the selected key terms with operators in this review were very close to the current subject (search queries in table 1). the search terms and queries were defined according to the advanced search tips such as, "choose search terms that are specific or closely related to the topic of interest", "use the singular form of the word", "use connectors" and basic search strategies (sciencedirect®, 2015a,van rijsbergen, 1979). ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 475 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 table 1 search queries (until 25/07/2015) search number(#) search queries #1 "ahp" and "electre iii" and "hydro" #2 "ahp" and "electre iv" and "hydro" #3 "analytic hierarchy process" and "electre iii" and "hydro" #4 "analytic hierarchy process" and "electre iv" and "hydro" #5 "ahp" and "electre iii" #6 "ahp" and "electre iv" #7 "ahp" and "electre" #8 "analytic hierarchy process" and "electre" #9 "analytic hierarchy process" and "elimination and choice translating reality" books, chapters, journals, proceedings, transactions, magazines, and newsletters were all searched during this review until 25/07/2015. the titles and the abstracts were reviewed one by one and the full text of studies, the author viewed as important, were reviewed. some of these documents (according to abstract and full text review) were presented in the following paragraphs in this section. the search results of this literature review were also summarized in table 2 for other researchers and following studies. table 2 summary of literature review (until 25/07/2015) scientific publisher #1 #2 #3 #4 #5 #6 #7 #8 #9 acm digital 0 0 0 0 2 0 5 2 0 asce online 0 0 0 0 0 0 1 1 0 american society 0 0 0 0 0 0 29 16 0 cambridge journals 0 0 0 0 0 0 4 3 1 directory of open 0 0 0 0 6 0 15 3 1 emerald insight 0 0 0 0 0 0 3 1 0 google scholar 95 16 87 11 1.030 201 3.940 3.490 220 hindawi publishing 0 0 0 0 2 0 2 3 0 inderscience 0 0 0 0 0 0 3 0 0 ijahp 0 0 0 0 0 1 1 0 0 science direct 0 0 0 0 4 0 22 9 0 science publishing 0 0 0 0 0 0 0 0 0 springer 12 2 10 1 145 35 619 469 26 taylor & francis 0 0 0 0 2 0 6 5 0 wiley-blackwell 0 0 0 0 0 0 3 3 0 world scientific 0 0 0 0 0 0 2 2 0 there were some papers that presented comparative studies in scientific journals until the date of 25/07/2015; however none of these studies were in real world investments within small hydropower plants. there were only 18 studies in the literature that were close to the current study. lootsma & schuijt (1998), raju & pillai (1999), mahmoud & garcia ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 476 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 (2000), akpinar et al. (2005), gilliams et al. (2005), grau et al. (2010), sawicka et al. (2010), geldermann & schobel (2011), ozcan et.al. (2011), shaverdi et.al. (2011), phogat & singh (2013), ur-rehman & al-ahamri (2013), janiak & zak (2014), massei et al. (2014), samaras et al. (2014), zak et.al. (2014), sabzi & king (2015), and sanchezlozano et al. (2015) applied several mcdm methods. in short, there were 18 comparative studies discussing ahp and electre methods in the literature. these studies were in varying fields such as solar power, transportation, hydropower, production and manufacturing, banking, logistics, painting, and agricultural. the hydropower (water, river, etc.) related studies were by raju and pillai (1999), mahmoud and garcia (2000), and samaras et al. (2014), and none of these studies looked at private investments and small hydropower plants. therefore, to our knowledge this paper is the first study in the private small hydropower plant investment selection problem of this scope (ahp, electre iii, electre iv, and shannon’s entropy). the electre (elimination et choix traduisant la realite: elimination and choice expressing the reality) methods family was born with electre i (electre one) in the 1960s. it was mentioned in many studies that bernard roy was the father of these methods. govindan and jepsen (2014) wrote: "the first electre method was presented by benayoun et al. (1966) who reported on the works of the european consultancy company sema with respect to a specific real world problem. but the first journal article did not appear until 1968, when roy (1968) described the method in detail. later, it was renamed to electre i." the electre methods were grouped under the outranking approach (the european school of multicriteria decision analysis, the european school approach, the french school of multicriteria decision analysis) (huang & chen, 2005; lootsma, 1990). detailed information and studies on the electre methods can be found on the official websites of the lamsade (laboratoire d'analyse et modélisation de systèmes pour l'aide à la decision: http://www.lamsade.dauphine.fr/) in france, the euro working group multicriteria decision aiding (http://www.cs.put.poznan.pl/ewgmcda/), and professor dr. bernard roy (http://www.lamsade.dauphine.fr/~roy/). the electre iii (electre three) was developed for use with pseudo-criteria and the fuzzy binary outranking relations for ranking actions (roy, 1991; lair et al., 2004, damaskos & kalfakakou, 2005). the electre iv (electre four) was recommended for use when working without the relative criteria importance coefficients (roy, 1991; bashiri & hejazi, 2009; figueira et al., 2010). the “decision aiding” term was preferred to the term “decision support”, “decision making”, and “decision analysis” (figueira et al., 2010). for more details about the principles and the equations of the electre iii and the electre iv methods see saracoglu (2015a), ishizaka & nemery (2013) and figueira et al. (2005). several weighting methods such as the rating method, the utility function method, the extreme weight approach and the entropy method have been presented in scientific studies (tzeng et al., 1998). the weighting methods could be grouped by subjective and objective weight assessment methods (lotfi & fallahnejad, 2010). in the literature, the subjective weight assessment was typically based on the decision maker’s preferences; on the other hand the objective weight assessment was often based on hard data. however, there were some occasions when the objective factors were evaluated by the decision maker. moreover, there were also some occasions where the subjective factors http://www.lamsade.dauphine.fr/ http://www.cs.put.poznan.pl/ewgmcda/ http://www.lamsade.dauphine.fr/~roy/ ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 477 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 evaluations by the decision makers were treated with the objective weight assessment approaches (akyene, 2012; lotfi & fallahnejad, 2010; safari et al., 2012; saracoglu, 2015a; also onlinestatbook, 2015 for the statistical basis on this respect). as a result, it was tricky to classify the subjective weight assessment and objective weight assessment methods for our study. in this study, the subjective weight assessment term was used when the decision maker had the discriminative power (e.g. saaty’s ahp subjective criteria weighting) and the objective weight assessment was preferred when hard data had the discriminative power (e.g. shannon’s entropy objective criteria weighting). in other words, when the criteria weights were calculated based on the criteria values of the alternatives, the objective weight assessment term was used. entropy methods are one of the applied objective weight assessment methods (tzeng et al., 1998). some studies support entropy methods, however others do not. (lesne, 2011; logan, 2012; zhou et al., 2013). there were several entropy methods developed in the information and decision analysis research field such as the kullback-leibler divergence (kullback & leibler, 1951), the renyi entropy (bromiley et al., 2004), the de luca and termini, szmidt and kacprzyk (hung & chen, 2009) and the base method as the shannon’s entropy (shannon, 1948; crooks, 2015). there were several applications of the shannon’s entropy in many research fields in the literature (akyene, 2012; fu et al., 2015; safari et al., 2012; saracoglu, 2015a, zou et al., 2006). akyene (2012) presented a solution based on the shannon’s entropy and topsis methods for the mobile phone selection problem where twelve factors (dimensions, weight, screen size, memory, ram, speed, blue tooth, camera, operating system, cpu, battery, price) were taken into account. the weights of these factors were calculated by the shannon’s entropy method, and the selection process was performed by the topsis method. fu et al. (2015) recommended a new publishing index for the scientific institution rankings. the corrected count and number of articles criteria were used for the proposed method, and a comparison of the current and proposed approaches for the asia-pacific ranking of the nature publishing index was given. safari et al. (2012) used the shannon’s entropy and the promethee methods for the supplier selection based on 6 factors (capacity, delivery, quality, shipment accuracy, warranty policies, availability of raw materials). saracoglu (2015a) conducted an experimental research study to understand how private small hydropower plant investments could be selected by two electre methods with different weighting approaches. the weighting approaches were the shannon’s entropy, the equal weighting and the saaty's ahp criteria weight assignment. the electre iii and the electre iv methods were used for the evaluations of the actions based on 17 factors. some of these factors were first evaluated by the expert decision makers, according to the verbal statements (very good to very bad according to the likert 5 type scale. then the ordinal data were used as the interval data for the objective weight assignments. zou et al. (2006) worked on the three gorges reservoir water quality assessment problem where water quality monitoring data for 4 indicators were used. they compared the results with the traditional method. the shannon’s entropy objective weighting method was presented according to the previous studies in the literature as below (abdullah & otheman, 2013; akyene, 2012; zou et al., 2006; shannon, 1948; saracoglu, 2015a): initialized decision matrix: x=(xij)n×m i represents the criteria (1 ≤ i ≤ n) j represents the alternative (1 ≤ j ≤ m) normalized matrix: r=(rij)n×m 0 ≤ rij ≤ 1 (normalized matrix elements rij are between 0 and 1) ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 478 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 where maximization criterion 𝑟𝑖𝑗 = 𝑥𝑖𝑗−min𝑖{𝑥𝑖𝑗} max𝑖{𝑥𝑖𝑗}− min𝑖{𝑥𝑖𝑗} (1) minimization criterion 𝑟𝑖𝑗 = max𝑖{𝑥𝑖𝑗}−𝑥𝑖𝑗 max𝑖{𝑥𝑖𝑗}− min𝑖{𝑥𝑖𝑗} (2) entropy: ei (other representations as h, h(p(x) or h(a))) of the i th criterion 𝑒𝑖 = −𝑘 ∑ 𝑓𝑖𝑗 ln(𝑓𝑖𝑗 ) 𝑚 𝑗=1 , where 𝑓𝑖𝑗 = 𝑟𝑖𝑗 ∑ 𝑟𝑖𝑗 𝑚 𝑗=1 , k=1/ln(m), when fij=0 ⇒ fij ln(fij)=0. (3) weight of entropy of i th criterion 𝑤𝑖 = 1−𝑒𝑖 𝑛−∑ 𝑒𝑖 𝑛 𝑖=1 where 0 ≤ wi ≤ 1, ∑ 𝑤𝑖 = 1 𝑛 𝑖=1 (4) the ahp methodology was developed and first presented in the 1980s by thomas l. saaty (saaty, 1980, 1987, 1990). the description and definition of this method are given as: "in its general form the ahp is a nonlinear framework for carrying out both deductive and inductive thinking without use of the syllogism by taking several factors into consideration simultaneously and allowing for dependence and for feedback, and making numerical tradeoffs to arrive at a synthesis or conclusion. " (saaty, 1987). in this definition, some possible difficult and important terms were "deductive thinking", "inductive thinking", and "syllogism"; for definitions please visit iep (2015) and wikipedia (2015a) for deductive thinking, and inductive thinking, and cambridge university press (2015) and ldoce (2015) for syllogism. the ahp method was grouped under the american school of multicriteria decision approach (pomerol & barba-romero, 2012; lootsma, 1990). this method has been applied in many different research fields such as health, transportation-railway project strategic investment decisions, and shipbuilding-new shipbuilding yards investment location selection in countries master plans (andreichicova & radyshevskaya, 2009; macura and selmic, 2015, saracoglu, 2013). as usual for any scientific research subject some studies supported the ahp method in the literature while others were opposed (barzilai, 1998; saaty, 2004; saaty et.al., 2009). for more details about the principles and the equations of the ahp method see brunelli (2015), saaty (2004) and saaty & sodenkamp, (2010). the scope of this work is to understand, analyze, investigate and compare the findings of one of the american school methods (ahp) and two of the european school methods (electre iii and electre iv) with objective and subjective weighting methods (equal, shannon's entropy, saaty's ahp, and no weighting approaches) on a simplistic unified small hydropower plant investment selection model in the pre-development investment stages. this research study will hopefully contribute to the main aim of building up an intelligent autonomous decision or executive support system (human like computer reasoning and modeling) in the renewable energy field. ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 479 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 3. comparative research model, case, results, and discussion the comparative research model and its case study were built on the pillars of the methods (electre iii, electre iv, ahp, and shannon’s entropy) and the cognitive limitations of humans (long term and short term memory). the simplistic unified and unique research model and its case study have to satisfy all requirements of these methods; hence it must be developed very carefully under the restrictions of these methods. modeling constraints and guidance in this research study (please read the original documents cited in this text to avoid any conceptual, pragmatic, and semantic change or shift). these pillars are found in this study as: 1) apply electre ii, electre iii, or electre iv for ranking problems (figueira et al., 2010, ishizaka & nemery, 2013) 2) use electre iii when expressing the relative importance of the pseudo-criteria are possible (dias et al., 2006) 3) use electre iv under three conditions (not able, does not want, cannot express the relative importance). "using electre iv is only valid if the following two conditions are satisfied: no criterion is either preponderant or negligible when compared to any subset of half of the criteria" (dias et al., 2006) 4) use more than five criteria up to twelve or thirteen in electre methods (figueira et al., 2005) 5) use a maximum of 9 factors in each level in the ahp method (gawlik, 2008; kruger & hattingh, 2006; saaty, 1980) 6) preferable to apply electre methods in problems with large number of alternatives (pohekar & ramachandran, 2004; soncini-sessa et al., 2007) 7) preferable to apply electre methods when at least one criterion can be evaluated by an ordinal scale or a weakly interval scale (figueira et al., 2005) 8) preferable to apply electre methods when criteria have a strong heterogeneity (figueira et al., 2005) 9) build ahp model that is complex enough for capturing the situation and nature of the problem, but small and easy enough for handling changes (saaty, 1987) 10) take into account the principles of sensory information storage (sis), short-term memory (stm) and long-term memory (ltm) in the decision analysis. "sensory information storage holds sensory images for several tenths of a second after they are received by the sensory organs. the functioning of sis may be observed if you close your eyes, then open and close them again as rapidly as possible. as your eyes close, notice how the visual image is maintained for a fraction of a second before fading" (heuer, 1999). "information passes from sis into short-term memory, where again it is held for only a short period of time—a few seconds or minutes. like sis, short-term memory holds information temporarily, pending further processing. this processing includes judgments concerning meaning, relevance, and significance, as well as the mental actions necessary to integrate selected portions of the information into long-term memory" (heuer, 1999). "some information retained in stm is processed into long-term memory. this information on past experiences is filed away in the recesses of the mind and must be retrieved before it can be used. in contrast to the immediate recall of current ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 480 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 experience from stm, retrieval of information from ltm is indirect and sometimes laborious" (heuer, 1999). 11) take into account the principles of working memory by obeying the magical number 7, and the 7±2 rule to guarantee the trustworthiness (miller, 1956; shiffrin & nosofsky, 1994). 12) choose realistic threshold values in the electre methods (rogers & bruen, 1998; saracoglu, 2015a).  the indifference threshold of a criterion (q) has to be the highest value which beyond it the difference is perceptible  the preference threshold of a criterion (p) may be easily selected as at least twice as the indifference threshold (p) where symmetrical about the mean value  the veto threshold of a criterion (v) may be easily defined as at least three times as the preference threshold  for maximization criteria v ≥ p ≥ q, for minimization criteria additive inverse (rogers & bruen, 1998; saracoglu, 2015a) under these rules and guidelines, a simple and unified model is built in this study, because it makes it easier to do the following: 1) understand ahp, electre iii, electre iv, shannon’s entropy weighting, equal weighting, saaty's ahp weighting methods and approaches 2) compare the findings 3) find, collect, and process data and information 4) evaluate factors (e.g. avoid difficulty and too much complexity of evaluating or pairwise comprising of some subjective and objective factors as such ganser, 2001; ganser, 2005; mcdaniel et al., 2015; zukauskas & vveinhardt, 2015) 5) take small steps and open new doors for the development of a human like intelligent autonomous support system based on this, the model criteria were first studied based on three research studies in this specific field (saracoglu, 2015a, b, c). only seven factors were selected and used. during this selection process, the electrical installed capacity (in watts) of a small hydropower plant is intentionally taken into consideration (equation 5) because the plan is to use this equation as an objective function or a constraint in the future combined multi objective optimization and multi criteria decision making research studies. p = ηtr × ηg × ηt × ρw× g × q × hnet (5) (ηtr: efficiency of transformer, ηg: efficiency of generator, ηt: efficiency of turbine, ρw: density of water (kg/m 3 ), g: gravity (m/s 2 ), q: rated discharge (m 3 /s), hnet: net head (m)) (for extraction of this formula/equation see eliasson and ludvigsson, 2000; esha, 2004; esha, 2005) the wisdom, "let first the data speak", is followed in this study, therefore the objective factors are first selected for this simplistic unified comparative model (jorba et al., 2008). these attributes are also investigated using the help of carrot 2 . "carrot 2 is an open source software project that includes several text clustering algorithms which include bisecting k-means, suffix tree clustering and lingo algorithm" (carrot 2 , 2015; zamir & etzioni, 1999; osinski & weiss, 2005). the documents are clustered for each factor on the ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 481 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 carrot 2 desktop and studied one by one on some preferred clusters. the basic private small hydropower plant investment selection factors (criteria/attributes) in this study are as follows: c1: catchment area (drainage basin, catchment, catchment basin, drainage area, river basin, water basin) (objective criteria, km 2 ) (more is better ↑ ↑). the catchment area is defined as, "the land area where precipitation falls off into creeks, streams, rivers, lakes, and reservoirs. it is a land feature that can be identified by tracing a line along the highest elevation between two areas on a map" (usgs, 2015). georges river (2015) defines it as, "an area of land where water collects when it rains, often bounded by hills. as the water flows over the landscape it finds its way into streams and down into the soil, eventually feeding the river. some of this water stays underground and continues to slowly feed the river in times of low rainfall. every inch of land on the earth forms part of a catchment". langbein & iseri (1995) define the drainage area as, "a stream at a specified location is that area, measured in a horizontal plane, which is enclosed by a drainage divide", and drainage basin as, "a part of the surface of the earth that is occupied by a drainage system, which consists of a surface stream or a body of impounded surface water together with all tributary surface streams and bodies of impounded surface water". the european small hydropower association or esha (2004) defines it as, "the whole of the land and water surface area contributing to the discharge at a particular point on a watercourse". when the catchment area of a small hydropower plant is calculated by any means, the location of weirs (diversion dam, diversion weir) is taken into account. if there is more than one weir in the project, the total catchment area as an equivalent catchment area is calculated by the sum of all catchment area values in the evaluation of this study. when two catchment areas of small hydropower plants are compared or evaluated, the large one is preferred to the smaller one in this study. the properties of catchments are not used and considered in this simplistic unified decision aiding (decision making) model. the data of the small hydropower plant investments are directly taken and tabulated in this study. this criterion is a site and design specific factor. the carrot 2 is used for gathering additional information for better perception in an easy and fast way as shown in figure 4. further details of this criterion are discussed in saracoglu (2015a, b, c). figure 4. carrot 2 workbench clustering results for "drainage basin" on 29/07/2015, source: wiki, algorithm: lingo, visualization: foamtree (generated by the carrot 2 http://carrot2.org & the paint.net) http://carrot2.org/ ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 482 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 c2: project runoff (objective criteria, hm 3 ) (more is better ↑ ↑) the project runoff is the mean annual total runoff of the catchment area in this study. this term can also be used as only runoff. langbein & iseri (1995) define it as, "part of the precipitation that appears in surface streams". there are also a few important terms that help to understand the meaning of project runoff for mall hydropower plants. langbein & iseri (1995) define streamflow as, "the discharge that occurs in a natural channel. although the term discharge can be applied to the flow of a canal, the word streamflow uniquely describes the discharge in a surface stream course. the term "streamflow" is more general than runoff, as streamflow may be applied to discharge whether or not it is affected by diversion or regulation". langbein & iseri (1995) define overland flow as "the flow of rainwater or snowmelt over the land surface toward stream channels. after it enters a stream, it becomes runoff". usgs water science school (2015a) discusses surface runoff as, "many people probably have an overly-simplified idea that precipitation falls on the land, flows overland (runoff), and runs into rivers, which then empty into the oceans. that is "overly simplified" because rivers also gain and lose water to the ground. still, it is true that much of the water in rivers comes directly from runoff from the land surface, which is defined as surface runoff. when rain hits saturated or impervious ground, it begins to flow overland downhill. it is easy to see if it flows down your driveway to the curb and into a storm sewer, but it is harder to notice it flowing overland in a natural setting". the esha (2004) defines it as, "the rainfall, which actually does enter the stream as either surface or subsurface flow". when the project runoff of a small hydropower plant is calculated by any means, the location of weirs (diversion dam, diversion weir) is taken into account. if there is more than one weir in the project, the total project runoff as equivalent total project runoff is calculated by the sum of all total project runoff values in the evaluation of this study. moreover, the environmental flow runoff (for ecosystems and human livelihoods) isn't included in this total. when two project runoff values of small hydropower plants are compared or evaluated, the larger one is preferred to the smaller one in this study. the data on the small hydropower plant investments are directly taken and tabulated in this study. this criterion is a site and design specific factor. the carrot 2 is used for gathering additional information for better perception in an easy and fast way as shown in figure 5. further details of this criterion are discussed in saracoglu (2015a, b, c). ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 483 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 figure 5. carrot 2 workbench clustering results for "runoff" on 29/07/2015 source: wiki, algorithm: lingo, visualization: circles (generated by the carrot 2 & the paint.net) c3: net head (objective criteria, m) (more is better ↑ ↑) the net head is defined as, "the gross head minus all hydraulic losses except those chargeable to the turbine" (eia, 2015). the head is sometimes called the water pressure. the esha (2004) defines it as, "the gross head minus the sum of all the losses equals the net head, which is available to drive the turbine". the losses are generally caused by trash racks, pipe friction, bends and valves. the gross head is defined as, "a dam's maximum allowed vertical distance between the upstream's surface water (headwater) forebay elevation and the downstream's surface water (tailwater) elevation at the tail-race for reaction wheel dams or the elevation of the jet at impulse wheel dams during specified operation and water conditions" (eia, 2015). esha (2004) defines the gross head as, "the vertical distance that the water falls through in giving up its potential energy (i.e. between the upper and lower water surface levels)". here, the turbine type is important because some types of turbines discharge the water to the atmosphere. if there are multiple net heads in the project (cascade group projects or other clustered projects), the equivalence of these net heads according to the electrical installed power equation is calculated and used in the evaluation. this equivalent net head is an approximation and assumption of this study. some detailed technical conditions such as total efficiencies of turbines, generators and transformers are assumed to be the same for all alternatives and actions. when two project net head values of small hydropower plants are compared or evaluated, the larger one is preferred to the smaller one in this study. the data on the small hydropower plant investments are directly taken and tabulated in this study. this criterion is a site and design specific factor. the carrot 2 is used for gathering additional information for better perception in an easy and fast way as shown in figure 6. further details of this criterion are discussed in saracoglu (2015a, b, c). ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 484 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 figure 6. carrot 2 workbench clustering results for "net head" on 31/07/2015 source: wiki, algorithm: lingo, visualization: aduna cluster map (generated by the carrot 2 & the paint.net) c4: project design discharge (capacity, discharge, flow, flow rate, rate of flow) (objective criteria, m 3 /s) (more is better ↑ ↑) usgs water science school (2015b) defines discharge as, "the volume of water moving down a stream or river per unit of time, commonly expressed in cubic feet per second or gallons per day. in general, river discharge is computed by multiplying the area of water in a channel cross section by the average velocity of the water in that cross section". here, the maximum project design discharge of the turbined water is taken into account. if there are more than one project design discharges in the project (several weirs, several turbines, cascade group projects or other clustered projects), the equivalence of these discharges according to the electrical installed power equation is calculated and used in the evaluation of this study. this equivalent discharge is an approximation and assumption of this study. some detailed technical conditions such as total efficiencies of turbines, generators and transformers are assumed to be same for all alternatives and actions. when two project discharge values of small hydropower plants are compared or evaluated, the large one is preferred to the smaller one in this study. the data on the small hydropower plant investments are directly taken and tabulated in this study. this criterion is a site and design specific factor. further details of this criterion are discussed in saracoglu (2015a, b, c). c5: firm energy (objective criteria, gwh) (more is better ↑ ↑) esha (2004) defines this as, “the power delivered during a certain period of the day with at least 90 – 95% certainty”. when two project firm energy values of small hydropower plants are compared or evaluated, the larger one is preferred to the smaller one in this study. the data on the small hydropower plant investments are directly taken and tabulated in this study. this criterion is a site and design specific factor. further details of this criterion are discussed in saracoglu (2015a, b, c). c6: secondary energy (objective criteria, gwh) (more is better ↑ ↑) the secondary energy is the remaining part of the total energy generated after the deduction of the firm energy of a small hydropower plant (equation 6). further details of this criterion are discussed in saracoglu (2015a, b, c). ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 485 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 etotal = efirm + esecondary (6) (etotal: total electricity generation (kwh), efirm: firm energy (kwh), esecondary: secondary energy (kwh), kilowatt hour unit is for this study, the unit can be megawatt hour, gigawatt hour, etc.) c7: investment cost (objective criteria, usd) (less is better ↓ ↑) this includes the total estimated investment cost of the private small hydropower plants. the construction cost of the small hydropower plant structures such as the diversion weir, the de-silting tank, the channel, the tunnel, the headpond, or in other words the forebay, the penstock, the powerhouse, the tailrace, and the substation are also taken into account. moreover, all related costs for the electromechanical equipment such the turbines, the generators, the transformers, the control systems, the diesel generators, the cooling systems, and the dewatering systems are all taken into account. transmission lines and their related costs, all of the engineering studies and their related costs, supervision tasks and related costs are all taken into account. moreover, the contingency, the financial costs, the expropriation costs, the insurance costs are all taken into account in this item. in addition to these cost items, the operational costs of the private small hydropower plants are also calculated. the data of small hydropower plant investments are directly taken and tabulated in this study. henceforth, there isn't any design study and cost estimation performed in the current study. it is assumed that the investment costs of private small hydropower plants are the same items for all small hydropower plants. moreover, it is assumed that the investment cost of private small hydropower plants contains all of the cost items of a small hydropower plant. here, the only calculation made is for the currency exchange and time value of money change (investopedia, 2015a). we attempt to find the investment cost calculation date, then the foreignexchange rate data (usd: united states dollar to tl: turkish lira or tl to usd) is taken from the official webpage of the central bank of the republic of turkey and used for the currency exchange (tcmb, 2015) of the investment cost (equation 7). icusd date = ictl date × crtl to usd date (7) (icusd date: investment cost of private small hydropower plant on the investment cost calculation date in usd, ictl date: investment cost of private small hydropower plant on the investment cost calculation date in tl, crtl to usd date: official currency rate of tcmb from tl to usd on the investment cost calculation date) the time value of money change is calculated by the future worth or the future value equation based on the compounded annual interest rate in this study (investopedia, 2015b) (equation 8). the present value and the future value equations, and the detailed information about these terms can be found in gitman et al. (2014), known et al. (2004), panneerselvam (2012), and several other engineering economics, financial management and financial accounting books. icsmall hydropower plant investment today = icsmall hydropower plant investment initial value × (1+i) n (8) (icsmall hydropower plant investment today: investment cost of private small hydropower plant today for this study in usd, icsmall hydropower plant investment initial value: investment cost of private small hydropower plant on the investment cost calculation date in usd, i: interest rate, ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 486 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 approximation as the arithmetic mean of usd annual libor rates in this study, n: years counted from the initial value of year to today as year 2015). the london interbank offered rate (libor) data are taken directly as 1 year/ 12 month rates from moneycafe.com (2015) and globalrates.com (2015). the arithmetic mean of the values from these two different web pages is calculated for each year (equation 9). when only one data point exists, then this data is taken as the libor value in its year. next, the interest rate for a period is calculated by the arithmetic mean of libor values in this period. this interest rate is a good approximation for the calculation of the time value of money change in the investment cost (equation 10). by this approach, the complexity of the inflation in turkey, the redenomination (removal of six zeros in turkish lira) in turkey, and the exchange rates in turkey are eliminated. the libor data and the interest rates are presented in figure 7. liboryear i = (liboryear i moneycafe + liboryear i globalrate) / 2 (9) 𝑖𝑦𝑒𝑎𝑟 𝑗 𝑡𝑜 𝑦𝑒𝑎𝑟 𝑘 = 1 (𝑦𝑒𝑎𝑟 𝑘 −𝑦𝑒𝑎𝑟 𝑗) × ∑ 𝐿𝐼𝐵𝑂𝑅𝑦𝑒𝑎𝑟 𝑖 𝑦𝑒𝑎𝑟 𝑗 𝑦𝑒𝑎𝑟 𝑘 (10) when two investment costs of small hydropower plants are compared or evaluated, the smaller one is preferred to the larger one in this study. this criterion is a site and design specific factor. years i years i years i 2004-2015 2,18% 2005-2015 2,24% 2006-2015 2,15% 2007-2015 1,84% 2008-2015 1,40% 2009-2015 1,00% 2010-2015 0,83% 2011-2015 0,79% 2012-2015 0,80% 2013-2015 0,69% 2014-2015 0,61% 2015-2015 figure 7. libor 1 year or 12 month libor (january) (data: moneycafe.com, 2015; globalrates.com, 2015) (calculation: arithmetic mean) (generated by the apache openoffice 4.1.1 calc http://www.openoffice.org/ & the paint.net) http://www.openoffice.org/ ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 487 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 the current simplistic unified model structure for ahp, electre iii and electre iv has three levels (figure 8). there are seven private small hydropower plant investment alternatives or actions (alternative1 to 7/action1 to 7) (table 4). there is only one expert decision maker who builds the model and presents his thoughts on the factors and alternatives. the objective factors aren't evaluated due to the alternatives because their approximate values are directly taken in several ways (figure 3). the threshold values (indifference, preference, veto) in the electre methods are chosen based on the opinion of the expert decision maker. the criteria weights are calculated by the equal weighting, the shannon's entropy, and the saaty's ahp approaches. while assigning the criteria weights by the pairwise comparisons of the expert decision maker, the linguistic, or verbal evaluation labels are used based on the fundamental scale of the ahp (saaty, 1980; saaty, 2008) and the likert type scale (likert, 1932: extension 1 to 9 of original work 5 point scale) (saracoglu, 2015b). the expert decision maker judgments are made with the help of the super decisions version 2.2 software (http://www.superdecisions.com/) for the saaty's ahp weighting. the inconsistency of the matrix is directly calculated by the super decisions software and recorded. the equal weighting and the shannon's entropy weighting calculations are made directly in a microsoft excel spreadsheet. figure 8. simplistic unified model structure for ahp, electre iii and electre iv http://www.superdecisions.com/ ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 488 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 table 4 private small hydropower plant investment actions/alternatives actions/ alternatives p c1 c2 c3 c4 c5 c6 c7 alternative1 6,56 470,00 302,90 31,54 24,00 0,85 18,45 10.069.000 alternative2 1,91 36,00 26,79 76,36 2,80 0,00 5,12 4.534.000 alternative3 8,55 4.550,00 589,22 19,58 50,00 4,58 24,49 22.054.000 alternative4 5,43 1.243,31 193,71 54,03 12,64 0,00 18,62 17.757.000 alternative5 3,91 352,00 103,20 60,28 7,50 1,25 10,56 3.948.000 alternative6 6,52 1.763,50 274,76 36,00 20,82 0,00 8,83 12.653.000 alternative7 9,23 1.945,00 301,62 59,14 18,00 0,00 30,01 26.654.000 p: installed power (mw: megawatt), c1: catchment area (km 2 ), c2: project runoff (hm 3 ), c3: net head (m), c4: discharge (m 3 /s), c5: firm energy (gwh/year), c6: secondary energy (gwh/year), c7: investment cost (usd) the equal weighting, the shannon's entropy weighting, and the saaty's ahp weighting approaches for voting power or weight of the criteria on this simplistic unified small hydropower plant investment selection model in the pre-development investment stages are evaluated, calculated and compared. the equal weighting is 0,1429 per criteria, that sums to a total of 1,000 for seven factors. the shannon's entropy weighting is calculated according to its equations and the criteria weights are: catchment area 0,1517, project runoff 0,0979, net head 0,0858, discharge 0,1207, firm energy 0,3573, secondary energy 0,1019, investment cost 0,0847. the saaty's ahp weighting, that is the pairwise comparisons of the expert decision maker, starts with an inconsistency of 0,22171. the inconsistency reaches 0,09511 at the 7 th evaluation. at this level, there aren't any additional pairwise evaluations made because the desired inconsistency level in the ahp method is less than 0,10 (saaty, 1990). the inconsistency value per evaluation in the saaty's ahp weighting is presented in figure 9. the recommendations of the super decisions software aren't taken into account when the evaluations are performed again. the number of pairwise comparisons is 21 for 7 factors with respect to the goal. there are 7 evaluations, making the total number of pairwise comparisons 147 in this study. it is observed that the evaluations of the expert decision maker fit a logarithmic trendline with a good r squared (coefficient of determination) value (figure 9). this observation is recorded for future studies. the shannon’s entropy objective weighting representation difference rate (performance) of the evaluations of the expert decision makers ranges between -67% and 671% (figure 10). the equal weighting representation difference rate ranges between -64% and 626% (figure 10). these criteria weight findings show that neither of these objective weighting (equal and shannon's entropy) approaches are helpful in predicting the expert decision maker judgments and behavior on the criteria weights. the criteria weights and their representational capabilities of the saaty's ahp weighting are shown in figure 10. the equal weighting approach doesn't have the capability of discriminating the differences between the factors (figure 10). henceforth, this objective weighting approach can only predict the human judgments on the criteria weights well when the decision maker doesn't know anything about the factors and their details or when the factors being investigated don't have any major differences between themselves. for instance, an ordinary person can't decide or make any judgments on the weights of the factors of the firm energy and the secondary energy because he or she doesn't know what these factors mean and which one is more important and so on (figure ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 489 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 10). although the shannon's information theory (shannon's entropy) strictly depends on the probability of events (probability theory), its representation capability is still questionable and needs very detailed and long term research studies for the criteria weight assignments in the human decision making research field (carter, 2014; shannon, 1948; shannon, 1951). one of the important aspects of the shannon's entropy is that it is based on large quantities of information. further research studies need to investigate this issue with more hard data. the shannon's entropy objective weighting approach predicts fairly well the firm energy criteria weight (figure 10). the performance of the shannon's entropy weighting in predicting the saaty's ahp weighting for other factors is poor in this study. in short, the firm energy, the secondary energy and the investment cost factor weights are all predicted less than the saaty's ahp weighting (figure 10). the discharge, the net head, the project runoff and the catchment area criteria weights are all predicted more than the saaty's ahp weighting (figure 10). it is important to remember what william bruce cameron said, "not everything that can be counted counts, and not everything that counts can be counted". also, edward gibbon was wise when he stated, "the laws of probability, so true in general, so fallacious in particular". these two weighting approaches can't represent the expert decision maker judgments in this study because both the shannon’s entropy and the equal objective weighting approaches are based only on the hard data of the alternatives. however, saaty's ahp subjective weighting assessment is based only on the expert knowledge. this observation is also recorded for future studies. figure 9. inconsistency values of pairwise comparisons in this study ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 490 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 code criterion equal shannon's saaty's ahp c1 catchment area 0,1429 0,1517 0,0197 c2 project runoff 0,1429 0,0979 0,0242 c3 net head 0,1429 0,0858 0,0426 c4 discharge 0,1429 0,1207 0,0401 c5 firm energy 0,1429 0,3573 0,3982 c6 secondary energy 0,1429 0,1019 0,2163 c7 investment cost 0,1429 0,0847 0,2590 sum of weights 1,0000 1,0000 1,0000 figure 10. criteria weights comparison (on top), objective weighting representation rate of saaty's ahp weighting (on bottom) in this study the indifference threshold (q), the preference threshold (p), and the veto threshold (v) of criteria for the electre methods are chosen by the expert decision maker according to item 10 of the modeling constraints and guidance in section 3 (rogers & bruen, 1998; saracoglu, 2015a) (table 5). all of these evaluations are totally subjective judgments. the electre iii-iv software version 3.x is used in this study, so that the parameters and inputs are organized according to the software standards. this software defines the thresholds by two coefficients α and β as such: (𝑞 𝑜𝑟 𝑝 𝑜𝑟 𝑣)𝑗 = 𝛼𝑗 × 𝑔𝑗 + 𝛽𝑗 (11) ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 491 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 (q: indifference threshold, p: preference threshold, v: veto threshold, j: criteria number or criteria, gj: value or evaluation of the action or the alternative on the j th criterion, αj and βj: coefficients of q, p, or v) in this study, the threshold values are decided as the constant values by the expert decision maker, so that the values in table 5 are entered as β values on the electre iii-iv software. this evaluation approach is simpler for the expert decision maker than defining two coefficients α and β. instead of deciding these two coefficients, the expert decision maker directly decides the threshold values. the direct threshold preference is selected on the software. (dias et al., 2006) table 5 threshold values * criterion direction of preferences indifference (qj) preference (pj) veto (vj) catchment area max 500 1.000 3.000 project runoff max 200 400 1.200 net head max 40 80 240 discharge max 10 20 60 firm energy max 0,9 1,8 5,4 secondary energy max 15 30 90 investment cost min 5.000.000 10.000.000 30.000.000 * α values are taken as zero and β values are taken as threshold values the ahp model calculations are performed in the super decisions version 2.2 (see figure 11 for the ahp model screen view) for the equal weighting, the shannon's entropy weighting, and the saaty's ahp weighting approaches. the direct data input table is used for entering the data from table 4 into super decisions. the zero values aren't accepted by the software, hence instead of zero values, values very close to zero are entered (0,00 ≈ 0,00001). moreover, the values on the table are automatically changed to approximate values by the software (e.g. 1.243,31 ≈ 1243.3093; 4.534.000 ≈ 4534000.09397). these values are checked one by one and it is assumed that there will not be any major effect on the results of the calculations in this subject. the matrices for the alternatives with respect to the objective factors aren't studied for their inconsistency because they are the objective factors. in this study, the priorities of the factors and the alternatives are taken by their normalized cluster values in super decisions. the preference orders of the alternatives by the equal weighting approach are alternative 1 (0,1247) (3 rd rank), alternative 2 (0,0509) (7 th rank), alternative 3 (0,3314) (1 st rank), alternative 4 (0,1177) (4 th rank), alternative 5 (0,0918) (6 th rank), alternative 6 (0,1128) (5 th rank), and alternative 7 (0,1708) (2 nd rank). the preference orders of the alternatives by the shannon's entropy weighting approach are alternative 1 (0,1228) (3 rd rank), alternative 2 (0,0324) (7 th rank), alternative 3 (0,4348) (1 st rank), alternative 4 (0,0855) (6 th rank), alternative 5 (0,1118) (4 th rank), alternative 6 (0,0872) (5 th rank), and alternative 7 (0,1255) (2 nd rank). the preference orders of the alternatives by the saaty's ahp weighting approach are alternative 1 (0,1274) (3 rd rank), alternative 2 (0,0325) (7 th rank), alternative 3 (0,4120) (1 st rank), alternative 4 (0,0973) (5 th rank), alternative 5 (0,1159) (4 th rank), alternative 6 (0,0678) (6 th rank) and alternative 7 ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 492 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 (0,1472) (2 nd rank). the findings show that alternative 3 is ranked first for all methods, alternative 7 is ranked second for all methods, and alternative 1 is ranked third for all methods. these findings represent that the first three ranks match for each weighting approach on the ahp method even though their criteria weights aren't the same. the 4 th rank is occupied by alternative 4 in the equal weighting, alternative 5 in the shannon's entropy weighting, and alternative 5 in the saaty's ahp weighting approaches. the 5 th rank is occupied by alternative 6 for equal weighting, alternative 6 in the shannon's entropy weighting, and alternative 4 in the saaty's ahp weighting approaches. the 6 th rank is occupied by alternative 5 in the equal weighting, alternative 4 in the shannon's entropy weighting, and alternative 6 in the saaty's ahp weighting approaches. alternative 2 is ranked seventh for all methods. these findings show that the rankings of alternatives promise to be the same or very similar without any major disturbance with these weighting approaches. hence, it is possible to represent human subjective evaluations by some objective weighting approaches when only considering the final ranks of the ahp method (only considering these factors in private small hydropower plant investments). under these conditions, it is recommended that an investigation be started (review data, feasibility studies, etc.) into alternatives 3, 7, and 1 for private small hydropower plant investment options. figure 11. the ahp model on the super decisions software (open the electronic supplementary material: ijahp2015saatyahpweighting.sdmod) the electre iii and electre iv model calculations are performed on the electre iii-iv version 3.x for the equal weighting, the shannon's entropy weighting, and the saaty's ahp weighting approaches. the direct data input tables are used for entering the data from table 4 into the electre iii-iv. the descending and ascending distillations for both electre iii & iv models are taken from the menu results/distillations on the software. the ranks of the actions in the final pre-order for both electre iii & iv models are taken from the menu results/ranks in final preorder on the software. the complete results for both electre iii & iv models are taken from the menu results/final graph on the software. the descending and ascending distillations (left), the final pre-orders (middle), and the final graph (right) are presented ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 493 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 in figures 12 to 17. other important results (concordance matrix, credibility matrix, ranking matrix) are also taken from the menus and presented in the electronic supplementary material (ijahp2015 electronic supplementary.pdf). it is helpful to be reminded that "descending distillation selects at first the best actions to end the process with the worst ones", oppositely "the ascending distillation selects first the worst actions to end the process with the best ones" (dias et al., 2006). moreover, "an action which is incomparable to a group of others will be ranked at the end of this group in the descending distillation and at the top in the ascending distillation. the actions which are considered equal (equivalence classes) in a distillation are displayed in the same box" (dias et al., 2006). the final pre-orders should be understood as "any action which has no better action will have rank 1 (even if it is incomparable to many others), the actions of ranks 2 are those whose better actions are only of rank 1...." (dias et al., 2006). the final graph is a good representation of the final rankings. in figure 12 (left), alternative 5 is the best action in the electre iii with the saaty's ahp weighting. it is at the first (top of the figure) of the selection process in the descending distillation. alternative 3 follows alternative 5 in the electre iii with the saaty's ahp weighting in the descending distillation. alternative 7 is the worst action in the descending distillation. it is the last (bottom of the figure) of the selection process in the descending distillation (left in figure 12). alternative 3 is the worst action in the electre iii with the saaty's ahp weighting in the ascending distillation (top of the figure). alternative 1 and alternative 5 (in the same box) follow alternative 3 (bottom of the figure). alternative 7 is the best action in the ascending distillation. it is at the last (bottom of the figure) of the selection process in the descending distillation (left in figure 12). alternative 3 and alternative 5 are ranked first in electre iii with the saaty's ahp weighting (middle and right in figure 12). alternative 1 follows these two alternatives, and alternative 7 is the worst preferred action (middle and right in figure 12). under these conditions, it is recommended that an investigation be started (review data, feasibility studies etc.) on alternative 3 and alternative 5 first, and then alternative 1 for private small hydropower plant investment options. figure 12. the electre iii model results on the electre iii-iv version 3 software (open the electronic supplementary material: ijahpsa.elp) by the saaty's ahp weighting, distillations (on left), ranks in final preorder (on middle/center), final graph (on right) ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 494 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 in figure 13 (left), alternative 3 is the best action in the electre iv with the saaty's ahp weighting. it is first (top of the figure) in the selection process in the descending distillation. alternative 6 follows alternative 3 in the descending distillation. alternative 2, alternative 4, and alternative 7 are the worst actions in the descending distillation. they are last (bottom of the figure) in the selection process in the descending distillation. alternative 1 and alternative 3 are the least actions in the ascending distillation (top of the figure). alternative 4 is the best action in the ascending distillation. alternative 3 is ranked first in electre iv with the saaty's ahp weighting (middle and right in figure 13). alternative 1, alternative 5, and alternative 6 follow this alternative, and alternative 4 is the least preferred action. under these conditions, it is recommended that an investigation be started (review data, feasibility studies, etc.) into alternative 3 first, then alternative 1, alternative 5, and alternative 6 for private small hydropower plant investment options. figure 13. the electre iv model results on the electre iii-iv version 3 software (open the electronic supplementary material: ijahpsa.elp) by the saaty's ahp weighting, distillations (on left), ranks in final preorder (on middle), final graph (on right) in figure 14 (left), alternative 3 is the best action in the electre iii with the equal weighting. it is first (top of the figure) in the selection process in the descending distillation. alternative 1, alternative 6, and alternative 7 follow alternative 3 in the descending distillation, and alternative 2, alternative 4, and alternative 5 are the worst actions in the descending distillation. they are last (bottom of the figure) in the selection process in the descending distillation. alternative 3 is the worst action in the ascending distillation (top of the figure). alternative 2 is the best action in the ascending distillation. alternative 3 is ranked first in electre iii with the equal weighting (on middle and right in figure 14). alternative 1, alternative 6, and alternative 7 follow this alternative, and alternative 2 is the least preferred action. under these conditions, it is recommended that an investigation be started (review data, feasibility studies, etc.) of the alternative 3 first, then alternative 1, alternative 6, and alternative 7 for private small hydropower plant investment options. ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 495 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 figure 14. the electre iii model results on the electre iii-iv version 3 software (open the electronic supplementary material: ijahpew.elp) by the equal weighting, distillations (on left), ranks in final preorder (on middle), final graph (on right) in figure 15 (left), alternative 3 is the best action in the electre iv with the equal weighting. alternative 6 follows alternative 3 in the descending distillation. alternative 2, alternative 4, and alternative 7 are the worst actions in the descending distillation. alternative 1 and alternative 3 are the worst actions in the ascending distillation. alternative 4 is the best action in the ascending distillation. alternative 3 is ranked first in electre iv with the equal weighting (middle and right in figure 15). alternative 1, alternative 5, and alternative 6 follow this alternative. alternative 4 is the least preferred action. under these conditions, it is recommended that an investigation be started (review data, feasibility studies, etc.) of alternative 3 first, then alternative 1, alternative 5, and alternative 6 for private small hydropower plant investment options. figure 15. the electre iv model results on the electre iii-iv version 3 software (open the electronic supplementary material: ijahpew.elp) by the equal weighting, distillations (on left), ranks in final preorder (on middle), final graph (on right) ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 496 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 in figure 16 (left), alternative 3 is the best action in the electre iii with the shannon's entropy weighting. alternative 5 follows alternative 3 in the descending distillation. the others are the worst actions in the descending distillation. alternative 3 is the worst action in the ascending distillation. alternative 4 is the best action in the ascending distillation. alternative 3 is ranked first in electre iii with the shannon's entropy weighting (on middle and right in figure 16). alternative 5 follows this alternative. alternative 4 is the least preferred action. under these conditions, it is recommended that an investigation be started (review data, feasibility studies, etc.) for alternative 3 first, then alternative 5 for private small hydropower plant investment options. figure 16. the electre iii model results on the electre iii-iv version 3 software (open the electronic supplementary material: ijahpse.elp) by the shannon's entropy weighting, distillations (on left), ranks in final preorder (on middle), final graph (on right) in figure 17 (left), alternative 3 is the best action in the electre iv with the shannon's entropy weighting. alternative 6 follows alternative 3 in the descending distillation. alternative 2, alternative 4, and alternative 7 are the worst actions in the descending distillation. alternative 1 and alternative 3 are the worst actions in the ascending distillation. alternative 4 is the best action in the ascending distillation. alternative 3 is ranked first in electre iv with the shannon's entropy weighting (on middle and right in figure 16). alternative 1, alternative 5, and alternative 6 follow this alternative. alternative 4 is the least preferred action. under these conditions, it is recommended that an investigation be started (review data, feasibility studies, etc.) for alternative 3 first, then alternative 1, alternative 5, and alternative 6 for private small hydropower plant investment options. ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 497 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 figure 17. the electre iv model results on the electre iii-iv version 3 software (open the electronic supplementary material: ijahpse.elp) by the shannon's entropy weighting, distillations (on left), ranks in final preorder (on middle), final graph (on right) the findings show that alternative 3 is ranked first for all electre iii approaches. these findings represent that the first rank matches for each weighting approach on the electre iii method even though their criteria weights aren't the same. alternative 1 ranks second for the equal weighting and saaty's ahp weighting. alternative 1, alternative 6 and alternative 7 rank second in the equal weighting. this study shows that electre iii can't discriminate much between the alternatives based on the similar findings. even in the saaty's ahp weighting, alternative 3 and alternative 5 rank first. however, these findings show that the rankings of alternatives still promise to be similar with minor disturbance with these weighting approaches in electre iii. hence, there is hope that it is possible to represent human subjective evaluations by some objective weighting approaches when only considering the final ranks on electre iii method (only considering these factors in private small hydropower plant investments) (figure 18). in contrast and more importantly, similar results in the rankings don't necessarily mean that the way of thinking is similar. therefore, future research should focus not only on just the ordinal rankings, but also its degree of dominance (garuti, 2012). the findings show that alternative 3 is ranked first, alternative 1, alternative 5, and alternative 6 are ranked second, alternative 2 and alternative 7 are ranked third, and alternative 4 is ranked fourth for all electre iv approaches. these findings show that all ranks match for each weighting approach on the electre iv method even though their criteria weights aren't the same. alternative 1, alternative 5, and alternative 6 occupy the same rank, while alternative 2 and alternative 7 also occupy the same rank. this study shows that electre iv can't discriminate the alternatives as electre iii and ahp can. even in the saaty's ahp weighting alternative 1, alternative 5 and alternative 6 rank second. however, these findings show that the rankings of alternatives promise to be the similar with no disturbance with these weighting approaches in electre iv. hence, it is possible to represent human subjective evaluations by some objective weighting approaches with only considering the final ranks on electre iv method (only considering these factors in private small hydropower plant investments) (figure 18). in contrast and more importantly, similar results in the rankings don't necessarily mean that the way of thinking is similar. therefore, future research should ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 498 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 focus not only on just the ordinal rankings, but also its degree of dominance (garuti, 2012). it is observed that alternative 3 ranks first for all of the approaches while other alternatives have several ranks. hence, it is believed that it is possible to represent the human subjective evaluations by some objective weighting approaches by only considering the final ranks on some multi criteria decision making methods (only considering these factors in private small hydropower plant investments) (figure 18). if these kinds of findings are found in several studies, some computer based systems which work with very little human efforts can be modeled, designed and developed in the future. ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 499 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 output methods alternative rank priority normal weight 1 equal weight ahp (aew) alternative 1 3 0,1247 alternative 2 7 0,0509 alternative 3 1 0,3314 alternative 4 4 0,1177 alternative 5 6 0,0918 alternative 6 5 0,1128 alternative 7 2 0,1708 2 shannon's weight ahp (ash) alternative 1 3 0,1228 alternative 2 7 0,0324 alternative 3 1 0,4348 alternative 4 6 0,0855 alternative 5 4 0,1118 alternative 6 5 0,0872 alternative 7 2 0,1255 3 saaty's weight ahp (asa) alternative 1 3 0,1274 alternative 2 7 0,0325 alternative 3 1 0,4120 alternative 4 5 0,0973 alternative 5 4 0,1159 alternative 6 6 0,0678 alternative 7 2 0,1472 4 equal weight electre iii (eiiiew) alternative 1 2 alternative 2 4 alternative 3 1 alternative 4 3 alternative 5 3 alternative 6 2 alternative 7 2 5 shannon's weight electre iii (eiiish) alternative 1 3 alternative 2 3 alternative 3 1 alternative 4 5 alternative 5 2 alternative 6 3 alternative 7 4 6 saaty's weight electre iii (eiiisa) alternative 1 2 alternative 2 3 alternative 3 1 alternative 4 4 alternative 5 1 alternative 6 3 alternative 7 5 7 equal weight electre iv (eivew) alternative 1 2 alternative 2 3 alternative 3 1 alternative 4 4 alternative 5 2 alternative 6 2 alternative 7 3 8 shannon's weight electre iv (eivsh) alternative 1 2 alternative 2 3 alternative 3 1 alternative 4 4 alternative 5 2 alternative 6 2 alternative 7 3 9 saaty's weight electre iv alternative 1 2 equal shannon's saaty's equal shannon's saaty's equal shannon's saaty's alternative 1 3 3 3 2 3 2 2 2 2 alternative 2 7 7 7 4 3 3 3 3 3 alternative 3 1 1 1 1 1 1 1 1 1 alternative 4 4 6 5 3 5 4 4 4 4 alternative 5 6 4 4 3 2 1 2 2 2 alternative 6 5 5 6 2 3 3 2 2 2 alternative 7 2 2 2 2 4 5 3 3 3 electre iiiahp electre iv ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 500 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 (eivsa) alternative 2 3 alternative 3 1 alternative 4 4 alternative 5 2 alternative 6 2 alternative 7 3 figure 18. comparative model in this study 4. conclusions, future applications and research in this paper, a comparative study was conducted for three weighting approaches (equal weighting, shannon's entropy, saaty's ahp) with three multi criteria decision aiding methods (ahp, electre iii, electre iv) on a simplistic unified private small hydropower plant investment selection problem in the renewable energy industry. one important observation made during this research study concerns the representation capability of the equal and the shannon's entropy objective weighting methods. unfortunately, these methods can't represent the opinions, and the preferences of the expert decision maker on the factors taken by the pairwise comparisons of the saaty's ahp method. another important observation is related to the mcdm methods. the preferences of the expert decision maker can be considered very easily by the ahp method however, it is not easy to reach the desired level of consistency. often, several attempts must be made to reduce the level of inconsistency, and this process is very time consuming. the electre methods do not have any special way to obtain criteria weights. one of the major disadvantages is perceived while deciding the threshold values which are very subjective for the decision makers. the investigations should be performed well whether or not the thresholds are convincing or unconvincing, however this is very difficult. moreover, the electre iii and the electre iv aren't capable of discriminating the actions when compared to the ahp in this study. one of the main advantages of these methods is their capability to split a big problem into smaller pieces and investigate these pieces in detail to solve the larger problem. they are very helpful in understanding real world problems and their possible solutions. hence, future studies will always recommend to designers and engineers that they gather help and work with these methods in their professional lives. the authors think it is important that lectures on this subject be given at universities in the engineering disciplines. finally, another surprising observation is made concerning the final ranks. even though the weights aren't the same, the same alternative was ranked first. this finding gives support for continuing to work on this subject via an autonomous computer based intelligent decision support system. in the future, we aim to build and investigate data powered decision aiding models. when some statistical methods and approaches are integrated with these weighting methods and multi criteria decision making (ahp, electre iii and electre iv) methods, the solutions of the small hydropower plant investment selection problem will become easier and more collective. in fact, there are already plans to use these methods in several hydropower plant categories and classes. in addition, plans for the application of these methods in solar power plants and the 100% renewable power grid are already being made by the researcher. we hope that this study and following ones will contribute to the practical research of these methods and to the renewable energy. ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 501 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 references abdullah, l. & otheman, a., (2013). a new entropy weight for subcriteria in interval type-2 fuzzy topsis and its application. international journal of intelligent systems and applications, 5(2), 25–33. doi: 10.5815/ijisa.2013.02.03 acm digital library (2015). http://dl.acm.org/ accessed and reviewed until 25/07/2015. akpinar, n., talay, i., gun, s. (2005). priority setting in agricultural land-use types for sustainable development. renewable agriculture and food systems, 20(03), 136–147. doi: http://dx.doi.org/10.1079/raf200483 akyene, t. (2012). cell phone evaluation base on entropy and topsis. interdisciplinary journal of research in business, 1(12), 9–15. alavi, m., & leidner, d.e. (1999). knowledge management and knowledge management systems: conceptual foundations and research issues. 99/34/mkt, insead working paper series, insead, boulevard de constance, france. american society of mechanical engineers (2015). https://www.asme.org/ accessed and reviewed until 25/07/2015. andreichicova, o.n., & radyshevskaya, t.n. (2009). an application of the analytic network process to researching oral health. international journal of the analytic hierarchy process, 1(1), 45–60. doi: http://dx.doi.org/10.13033/ijahp.v1i1.7 asce online research library (2015). http://ascelibrary.org/ accessed and reviewed until 25/07/2015. bajaj, j.l., mukherjee, n., & kulkarni, a.v. (2007). grid-connected small hydropower (shp) development: regulatory issues and challenges. international conference on small hydropower, hydro sri lanka. barzilai, j. (1998). on the decomposition of value functions. operations research letters, 22(4), 159–170. doi:10.1016/s0167-6377(98)00015-7 bashiri, m. & hejazi, t.h. (2009). an extension of multi-response optimization in madm view, journal of applied sciences, 9(9), 1695–1702. doi: 10.3923/jas.2009.1695.1702 benayoun, r., roy, b. & sussman, b. (1966). une méthode pour guider le choix en présence de points de vue multiples. note de travail 49. sema-metra. directionscientifique. berakovic, b., pavlin, z., & stefanac, s. (2009). small hydro–a part of water management. international symposium on water management and hydraulic engineering, ohrid, macedonia. http://dl.acm.org/ http://dx.doi.org/10.1079/raf200483 https://www.asme.org/ http://ascelibrary.org/ http://dx.doi.org/10.1016/s0167-6377(98)00015-7 http://dx.doi.org/10.3923/jas.2009.1695.1702 ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 502 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 bromiley, p.a., thacker, n.a., & bouhova-thacker, e. (2004). shannon entropy, renyi entropy, and information. rapport technique, school of cancer and imaging sciences, university of manchester, 93. http://www.tina-vision.net/docs/memos/2004-004.pdf brunelli, m. (2015). introduction to the analytic hierarchy process. switzerland: springer international publishing. doi: 10.1007/978-3-319-12502-2. cambridge journals online (2015). http://journals.cambridge.org/ accessed and reviewed until 25/07/2015. cambridge university press (2015). syllogism. http://dictionary.cambridge.org/dictionary/british/syllogism accessed in 02/08/2015. carrot2. (2015). http://project.carrot2.org/ accessed in 03/08/2015. carter, t. (2014). an introduction to information theory and entropy. complex systems summer school santa fe. http://astarte.csustan.edu/~ tom/sfi-csss crooks, g.e. (2015). on measures of entropy and information. http://threeplusone.com/info accessed in 01/08/2015. damaskos, x., & kalfakakou. g. (2005). application of electre iii and dea methods in the bpr of a bank branch network, yugoslav journal of operations research, 15(2), 259–276. doi: 10.2298/yjor0502259d dias, j.a., figueira, j.r., & roy, b. (2006). the software electre iii-iv methodology and user manual (version 3.x). university paris-dauphine lamsade. directory of open access journals (doaj) (2015). http://doaj.org/ accessed and reviewed until 25/07/2015. dragu, c., sels, t., & belmans, r. (2010). small hydro power-state of the art and applications. leuven, belgium: esat-elen, energy institute. eliasson, j., & ludvigsson, g. (2000). load factor of hydropower plants and its importance in planning and design. 11 th international seminar on hydro power plants, hydros future in changing markets. vienna, austria: university of technology. emerald insight (2015). http://www.emeraldinsight.com/ accessed and reviewed until 25/07/2015. european commission (ec) (2014). http://ec.europa.eu/research/energy/eu/index_en.cfm?pg=research-hydropower accessed on 02/09/2014. european renewable energy council (erec) (2012). ren21 renewables 2012 global status report. paris, france: european focus. http://www.ren21.net/ http://www.tina-vision.net/docs/memos/2004-004.pdf http://journals.cambridge.org/ http://dictionary.cambridge.org/dictionary/british/syllogism http://project.carrot2.org/ http://astarte.csustan.edu/~%20tom/sfi-csss http://threeplusone.com/info http://doaj.org/ http://www.emeraldinsight.com/ http://ec.europa.eu/research/energy/eu/index_en.cfm?pg=research-hydropower http://www.ren21.net/ ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 503 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 european small hydropower association (esha), swedish renewable energy association (sero) & lithuanian hydropower association (lha), (2008). small hydro energy efficient promotion campaign action (sherpa) project, strategic study for development of small hydropower in the european union brochure. bruxelles, belgium. european small hydropower association (esha) (2004). guide on how to develop a small hydropower plant. bruxelles, belgium. european small hydropower association (esha) (2005). brochure on environmental integration of small hydropower plants. belgium. figueira, j.r., greco, s., roy, b., & slowinski, r. (2010). electre methods: main features and recent developments, laboratoire d'analyse et modélisation de systèmes pour l'aide à la décision cahier du lamsade, 298, 1–34. figueira, j., mousseau, v., & roy, b. (2005). electre methods. in figueira, j., salvatore g., & matthias e. (eds.), multiple criteria decision analysis: state of the art surveys. 133-153. new york: springer science & business media. doi:10.1007/b100605 fu, y., wang, m., & lai, k. k. (2015). a modified nature publishing index via shannon entropy. new york, usa: discrete dynamics in nature and society, hindawi publishing corporation. doi: http://dx.doi.org/10.1155/2015/640924 ganser, d. (2005). terrorism in western europe: an approach to nato's secret staybehind armies. the whitehead journal of diplomacy and international relations, 6, 6995. ganser, d. (2001). fear as a weapon the effects of psychological warfare on domestic and international politics. world affairs, 9(4), 28-44. garuti, c.a. (2012). measuring in weighted environments, moving from metric to order topology. universidad federico santa maria, santiago, chile. pittsburgh, pa: rws publications. isbn: 978-1-888603-21-7. gawlik, r. (2008). preliminary criteria reduction for the application of analytic hierarchy process method. munich personal repec archive mpra, 45348, cracow university of economics. geldermann, j., schobel, a. (2011). on the similarities of some multi‐criteria decision analysis methods. journal of multi-criteria decision analysis, 18, 219–230. doi: 10.1002/mcda.468 georges river (2015). what is a catchment? http://www.georgesriver.org.au/what-is-a-catchment.html accessed in 28/07/2015. gilliams, s., raymaekers, d., muys, b., van orshoven, j., (2005). comparing multiple criteria decision methods to extend a geographical information system on afforestation. computers and electronics in agriculture, 49(1), 142–158. doi: 10.1016/j.compag.2005.02.011 http://dx.doi.org/10.1155/2015/640924 http://www.georgesriver.org.au/what-is-a-catchment.html ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 504 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 gitman, l.j., joehnk, m.d., billingsley, r.s., (2014). personal financial planning. usa: cengage learning, south-western. isbn-13: 978-1-111-97163-2. globalrates.com, (2015). http://www.global-rates.com/interest-rates/libor/americandollar/usd-libor-interest-rate-12-months.aspx accessed in 31/07/2015. google scholar (2015). http://scholar.google.com.tr/ accessed and reviewed until 25/07/2015. govindan, k., jepsen, m.b. (2014) electre–past, present and future, european working group multiple criteria decision aiding, series 3, nº 30 autumn 2014 newsletter bulletin, 15–18 http://www.cs.put.poznan.pl/ewgmcda/newsletter/fall14.pdf grau, j.b., anton, j.m., tarquis, a.m., colombo, f., de los rios, l., cisneros, j.m., (2010). an application of mathematical models to select the optimal alternative for an integral plan to desertification and erosion control (chaco area–salta province– argentina). biogeosciences, 7, 3421–3433. doi:10.5194/bg-7-3421-2010 hall, d.g. (2003). estimation of economic parameters of u.s. hydropower resources. idaho falls, idaho, usa: idaho national engineering and environmental laboratory. heuer, r.j. (1999). psychology of intelligence analysis. washington, d.c.: center for the study of intelligence. isbn 1 929667-00-0 hindawi publishing corporation (2015). http://www.hindawi.com/ accessed and reviewed until 25/07/2015. huang, w.c. & chen, c.h. (2005). using the electre ii method to apply and analyze the differentiation theory, proceedings of the eastern asia society for transportation studies, 5, 2237–2249. hung, c.c. & chen, l.h. (2009). a fuzzy topsis decision making model with entropy weight under intuitionistic fuzzy environment, proceedings of the international multiconference of engineers and computer scientists, 1, hong kong. inderscience publishers (2015). http://www.inderscience.com/ accessed and reviewed until 25/07/2015. internet encyclopedia of philosophy and its authors (iep) (2015). deductive and inductive arguments http://www.iep.utm.edu/ded-ind/ accessed in 02/08/2015. international energy agency implementing agreement for hydropower technologies and programmes (iea hydropower) (2015). what is hydropower's history?. http://www.ieahydro.org/what_is_hydropower's_history.html accessed in 20/07/2015. international hydropower association (iha) (2015). a brief history of hydropower. http://www.hydropower.org/a-brief-history-of-hydropower accessed in 20/07/2015. http://www.global-rates.com/interest-rates/libor/american-dollar/usd-libor-interest-rate-12-months.aspx http://www.global-rates.com/interest-rates/libor/american-dollar/usd-libor-interest-rate-12-months.aspx http://scholar.google.com.tr/ http://www.cs.put.poznan.pl/ewgmcda/newsletter/fall14.pdf. http://www.hindawi.com/ http://www.inderscience.com/ http://www.iep.utm.edu/ded-ind/ http://www.ieahydro.org/what_is_hydropower's_history.html http://www.hydropower.org/a-brief-history-of-hydropower ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 505 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 international hydropower association, international commission on large dams, international energy agency, canadian hydropower association (ihaicld) (2000). hydropower and the world's energy future, the role of hydropower in bringing clean, renewable, energy to the world. paris, france. international journal of the analytic hierarchy process (ijahp) (2015). http://www.ijahp.org/index.php/ijahp accessed and reviewed until 25/07/2015. investopedia (2015a). understanding the time value of money. http://www.investopedia.com/articles/03/082703.asp accessed in 31/07/2015. investopedia (2015b). future value fv. http://www.investopedia.com/terms/f/futurevalue.asp accessed in 31/07/2015. ishizaka, a. & nemery, p. (2013). multi-criteria decision analysis: methods and software 1 st edition. united kingdom: john wiley & sons, ltd. isbn: 978-1-119-974079 janiak, m.k., & zak, j. (2014). multiple criteria evaluation of different redesign variants of the public tram system. transportation research procedia, 3, 690-699. doi:10.1016/j.trpro.2014.10.048 jorba, o., loridan, t., jiménez-guerrero, p., pérez, c., & baldasano, j. m. (2008). linking the advanced research wrf meteorological model with the chimere chemistry-transport model. environmental modelling & software, 23(8), 10921094. doi:10.1016/j.envsoft.2008.02.002 known, a.j., martin, j.d., petty, j.w., & scott, d.f. (2004). foundations of finance: the logic and practice of financial management, 4th edition. beijing, china: pearson education asia limited and tsinghua university press. eisbn: 0-13-047982-9 kruger, h.a., & hattingh, j.m. (2006). a combined ahp-gp model to allocate internal auditing time to projects. orion, 22 (1), 59–76. doi: http://dx.doi.org/10.5784/22-1-33 kullback, s., & leibler, r.a. (1951) on information and sufficiency, the annals of mathematical statistics, 2, 79–86. doi: 10.1214/aoms/1177729694. kurien, j., & sinha, a.k. (2006). small hydro power nabard's involvement and future outlook. national bank for agriculture and rural development mumbai, himalayan small hydropower summit, dehradun. lair, j., rissanen, t. & sarja, a. (2004). life cycle management of concrete infrastructures for improved sustainability methods for optimisation and decision making in lifetime management of structure. european community competitive and sustainable growth programme, 30–51. lako, p., eder, h., de noord, m., reisinger, h., (2003). hydropower development with a focus on asia and western europe. overview in the framework of vleem 2, ecn and verbund plan, the netherlands. http://www.ijahp.org/index.php/ijahp http://www.investopedia.com/articles/03/082703.asp http://www.investopedia.com/terms/f/futurevalue.asp http://dx.doi.org/10.1016/j.trpro.2014.10.048 http://dx.doi.org/10.1016/j.envsoft.2008.02.002 http://dx.doi.org/10.5784/22-1-33 ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 506 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 langbein, w.b. & iseri, k.t. (1995). general introduction and hydrologic definitions, manual of hydrology: part 1. general surface-water techniques, geological survey water-supply paper 1541-a, methods and practices of the geological survey, united states government printing office, washington: 1960, html version 1995, http://water.usgs.gov/wsc/glossary.html accessed in 28/07/2015. lesne, a. (2011). shannon entropy: a rigorous mathematical notion at the crossroads between probability, information theory, dynamical systems and statistical physics. mathematical structures in computer science, 1–40. doi: http://dx.doi.org/10.1017/s0960129512000783 likert, r. (1932). a technique for the measurement of attitudes. archives of psychology, no: 140, new york, usa. liu, h., masera, d. & esser, l., eds. (unido and icshp) (2013). world small hydropower development report 2013. united nations industrial development organization: international center on small hydro power. available from www.smallhydroworld.org. logan, r.k. (2012). what is information?: why is it relativistic and what is its relationship to materiality, meaning and organization. information, 3(1), 68-91. doi:10.3390/info3010068 longman dictionary of contemporary english (ldoce), (2015), syllogism http://www.ldoceonline.com/dictionary/syllogism accessed in 02/08/2015. lootsma, f.a. (1990), the french and the american school in multi-criteria decision analysis. recherche operationnele / operations research, 24(3), 263–285. lootsma, f.a. & schuijt, h., (1998). the multiplicative ahp, smart and electre in a common context. journal of multi-criteria decision analysis, 6, 185–196. doi: 10.1002/(sici)1099-1360(199707)6:4<185::aid-mcda136>3.0.co;2-e lotfi, f.h. & fallahnejad, r. (2010) imprecise shannon’s entropy and multi attribute decision making. entropy, 12, 53–62. doi:10.3390/e12010053 macura, d., & selmic, m. (2015). railway project evaluation with dynamic priorities – serbia case study. international journal of the analytic hierarchy process, 7(1), 19366744. doi: http://dx.doi.org/10.13033/ijahp.v7i1.292 mahmoud, m.r., & garcia, l.a. (2000). comparison of different multicriteria evaluation methods for the red bluff diversion dam. environmental modelling & software, 15(5), 471-478. doi: 10.1016/s1364-8152(00)00025-6 massei, g., rocchi, l., paolotti, l., greco, s., boggia, a. (2014). decision support systems for environmental management: a case study on wastewater from agriculture, journal of environmental management, 146, 491-504. doi:10.1016/j.jenvman.2014.08.012 http://water.usgs.gov/wsc/glossary.html http://dx.doi.org/10.1017/s0960129512000783 file:///c:/users/hidroden/desktop/scientific%20studies/2015%20ijahp%20ejie-2nd%20paper/www.smallhydroworld.org http://www.ldoceonline.com/dictionary/syllogism http://dx.doi.org/10.3390/e12010053 http://dx.doi.org/10.1016/j.jenvman.2014.08.012 ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 507 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 mcdaniel, k.r., ngala, f., leonard, k.m., (2015), does competency matter? competency as a factor in workplace bullying. journal of managerial psychology, 30(5), 597-609. doi: http://dx.doi.org/10.1108/jmp-02-2013-0046 miller, g.a., (1956). the magical number seven, plus or minus two: some limits on our capacity for processing information. the psychological review, 63, 81–97. doi: http://dx.doi.org/10.1037/h0043158 moneycafe, (2015). http://www.moneycafe.com/personal-finance/libor/ accessed in 31/07/2015. moreire, j.r., & poole, a.d. (1993). hydropower and its constraints. in: johansson t.b. et al. (eds), renewable energy: sources for fuels and electricity. washington d.c.: island press 73–120. national hydropower association (nha) (2015). history of hydro. http://www.hydro.org/tech-and-policy/history-of-hydro/ accessed in 20/07/2015. official gazette of the republic of turkey, 30/03/2013 (official gazette 28603), (2015). http://www.resmigazete.gov.tr/eskiler/2013/03/20130330-14.htm accessed in 21/07/2015. official gazette of the republic of turkey, 02/10/2013 (official gazette 28783), (2015). http://www.resmigazete.gov.tr/eskiler/2013/10/20131002-3.htm accessed in 21/07/2015. onlinestatbook, (2015). online statistics education: a multimedia course of study (http://onlinestatbook.com/). project leader: david m. lane, rice university. accessed in 06/11/2015. osinski, s., & weiss, d. (2005). a concept-driven algorithm for clustering search results. intelligent systems, ieee, 20(3), 48–54. doi:10.1109/mis.2005.38 ozcan, t., celebi, n., esnaf, s. (2011). comparative analysis of multi-criteria decision making methodologies and implementation of a warehouse location selection problem. expert systems with applications, 38, 9773–9779. doi: 10.1016/j.eswa.2011.02.022 panneerselvam, r. (2012). engineering economics. new delhi: phi learning pvt. ltd. isbn-978-81-203-1743-7 phogat, m.v.s., & singh, a.p. (2013). selection of equipment for construction of a hilly road using multi criteria approach. procedia social and behavioral sciences, 104, 282291. doi: http://dx.doi.org/10.1016/j.sbspro.2013.11.121 pohekar, s.d., & ramachandran, m. (2004). application of multi-criteria decision making to sustainable energy planning—a review. renewable and sustainable energy reviews, 8(4), 365-381. doi: 10.1016/j.rser.2003.12.007 http://dx.doi.org/10.1108/jmp-02-2013-0046 http://psycnet.apa.org/doi/10.1037/h0043158 http://www.moneycafe.com/personal-finance/libor/ http://www.hydro.org/tech-and-policy/history-of-hydro/ http://www.resmigazete.gov.tr/eskiler/2013/03/20130330-14.htm http://www.resmigazete.gov.tr/eskiler/2013/10/20131002-3.htm http://onlinestatbook.com/ ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 508 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 pomerol, j.c., & barba-romero, s. (2012). multicriterion decision in management: principles and practice (vol. 25). new york, new york: springer science & business media. doi: 10.1007/978-1-4615-4459-3_11 raju, k.s., & pillai, c.r.s. (1999). multicriterion decision making in river basin planning and development. european journal of operational research, 112(2), 249-257. doi: http://dx.doi.org/10.1016/s0377-2217(98)00006-x renewable energy policy network for the 21st century (ren21) (2015). renewables 2015 global status report. paris: ren21 secretariat. isbn 978-3-9815934-6-4 (figure 28. countries with renewable energy policies and targets, early 2015 http://www.ren21.net/wp-content/uploads/2015/06/gsr2015_figure28.jpg) republic of turkey energy market regulatory authority (emra) (2013). (http://www.emra.gov.tr/) or (http://www.epdk.gov.tr). (latest data in september in 2012) rogers, m., & bruen, m., (1998). choosing realistic values of indifference, preference and veto thresholds for use with environmental criteria within electre, european journal of operational research, 107(3), 542–551. doi: 10.1016/s0377-2217(97)001756 roy, b. (1968). classement et choix en presence de points de vue multiples (la methode electre). revue francaise d informatique de recherche operationnelle, 2 (8), 57-75. roy, b. (1991). the outranking approach and the foundations of electre methods, theory and decision, 31, 49–73. doi: 10.1007/bf00134132 saaty, t.l. (1980). the analytic hierarchy process. new york, usa: mcgraw-hill. doii: 10.1080/00137918308956077 saaty, t.l. (1987). the analytic hierarchy process-what it is and how it is used. mathematical modelling, 9(3–5), 161–176. doi:10.1016/0270-0255(87)90473-8 saaty, t.l. (1990). how to make a decision: the analytic hierarchy process. european journal of operational research, 48, 9–26. doi:10.1016/0377-2217(90)90057-i saaty, t.l. (2008). decision making with the analytic hierarchy process. international journal of services sciences, 1(1), 83–98. doi: 10.1504/ijssci.2008.017590 saaty, t.l., & daji, r. (2015). when is a decision making method trustworthy? criteria for evaluating multi-criteria decision making methods. international journal of information technology & decision making, 1–17, doi: 10.1142/s021962201550025x. saaty, t.l., & sodenkamp, m. (2010). the analytic hierarchy and analytic network measurement processes: the measurement of intangibles. in: handbook of multicriteria analysis. germany: springer berlin heidelberg, 91–166. http://dx.doi.org/10.1007/9783-540-92828-7_4 http://dx.doi.org/10.1016/s0377-2217(98)00006-x http://www.ren21.net/wp-content/uploads/2015/06/gsr2015_figure28.jpg http://www.emra.gov.tr/ http://dx.doi.org/10.1016/0270-0255(87)90473-8 http://dx.doi.org/10.1016/0377-2217(90)90057-i http://dx.doi.org/10.1007/978-3-540-92828-7_4 http://dx.doi.org/10.1007/978-3-540-92828-7_4 ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 509 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 saaty, t.l., vargas, l.g., & whitaker, r. (2009). addressing with brevity criticisms of the analytic hierarchy process. international journal of the analytic hierarchy process, 1(2), 121–134. doi: http://dx.doi.org/10.13033/ijahp.v1i2.53 saaty, r.w. (2004). why barzilai’s criticisms of the ahp are incorrect. proceedings of the international mcdm meeting, whistler. sabzi, h.z. & king, j.p. (2015). numerical comparison of multi-criteria decision-making techniques: a simulation of flood management multi-criteria systems. world environmental and water resources congress 2015, 359–373. doi; 10.1061/9780784479162.035 safari, h., fagheyi, m.s., ahangari, s.s., & fathi, m.r. (2012). applying promethee method based on entropy weight for supplier selection. business management and strategy, 3(1), 97-106. doi: http://dx.doi.org/10.5296/bms.v3i1.1656 samaras, g.d., gkanas, n.i., konstantinia, c. (2014). assessing risk in dam projects using ahp and electre i. international journal of construction management, 14(4), 255-266. doi: 10.1080/15623599.2014.971942 sanchez-lozano, j.m., garcía-cascales, m.s., lamata, m.t. (2015). evaluation of suitable locations for the installation of solar thermoelectric power plants. computers & industrial engineering, 87, 343-355. doi:10.1016/j.cie.2015.05.028 saracoglu, b.o. (2013). selecting industrial investment locations in master plans of countries. european journal of industrial engineering, 7(4), 416–441. doi: 10.1504/ejie.2013.055016 saracoglu, b.o. (2015a). an experimental research sudy on the solution of a private small hydropower plant investments selection problem by electre iii/iv, shannon’s entropy, and saaty’s subjective criteria weighting. advances in decision sciences, 2015, 1-20. doi:10.1155/2015/548460 saracoglu, b.o. (2015b). an ahp application in the investment selection problem of small hydropower plants in turkey. international journal of the analytic hierarchy process (ijahp), 7(2), 1–29, doi: 10.13033/ijahp.v7i2.198. saracoglu, b.o. (2015c). an experimental research of small hydropower plant investments selection in turkey by carrot 2 , dexi, dexitree. journal of investment and management, 4(1): 47-60. doi: 10.11648/j.jim.20150401.17 sawicka, h., weglinski, s., witort, p. (2010). application of multiple criteria decision aid methods in logistic systems. logforum, 6(3), 10. saxena, p. (2007). small hydro development in india. international conference on small hydropower, hydro sri lanka, 1–6. http://dx.doi.org/10.5296/bms.v3i1.1656 http://dx.doi.org/10.1016/j.cie.2015.05.028 ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 510 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 schmidt, r. & denny, t. (2004). hydropower, web projects spring 2004, college physics ii, physics 104, instructed by dr. david newman, university of alaskafairbanks, physics department, http://ffden-2.phys.uaf.edu/104_spring2004.web.dir/todd_robyn/page5.htm accessed in 20/07/2015. sciencedirect® (2015a). all advanced search tips http://help.sciencedirect.com/flare/sdhelp_left.htm#cshid=stadv_main_all.htm|startto pic=content%2fstadv_main_all.htm|skinname=svs_sd accessed in 23/07/2015. science direct® (2015b). http://www.sciencedirect.com/ accessed and reviewed until 25/07/2015. science publishing group (2015). http://www.sciencepublishinggroup.com/home/index.aspx accessed and reviewed until 25/07/2015. shannon, c.e. (1948). a mathematical theory of communication. the bell system technical journal, 27, 379–423, 623–656. doi: 10.1002/j.1538-7305.1948.tb01338.x shannon, c.e. (1951). prediction and entropy of printed english. the bell system technical journal, 30(1), 50-64. doi: 10.1002/j.1538-7305.1951.tb01366.x shaverdi, m., akbari, m., tafti, s.f. (2011). combining fuzzy mcdm with bsc approach in performance evaluation of iranian private banking sector. advances in fuzzy systems, 2011, 1-12 pages. doi:10.1155/2011/148712 shiffrin, r.m., nosofsky, r.m., (1994). seven plus or minus two: a commentary on capacity limitations. psychological review, 101(2), 357–361. doi: http://dx.doi.org/10.1037/0033-295x.101.2.357 signore green 1001freedownloads.com (signore_green), (2015). http://www.1001freedownloads.com/free-clipart/architetto-remix-green-blond-man-icon accessed in 20/07/2015. soncini-sessa, r., weber, e., & castelletti, a. (2007). integrated and participatory water resources management-theory ,vol. 1. the netherlands: elsevier. springer (2015). http://www.springer.com/gp/ accessed and reviewed until 25/07/2015. taylor & francis online/journals (2015). http://www.tandfonline.com/ accessed and reviewed until 25/07/2015. tcmb (the central bank of the republic of turkey) (2015). http://www.tcmb.gov.tr/wps/wcm/connect/tcmb+en/tcmb+en & http://www.tcmb.gov.tr/wps/wcm/connect/tcmb+tr/tcmb+tr/main+menu/istatistik ler/doviz+kurlari/gosterge+niteligindeki+merkez+bankasi+kurlarii accessed on 31/07/2015. http://ffden-2.phys.uaf.edu/104_spring2004.web.dir/todd_robyn/page5.htm http://help.sciencedirect.com/flare/sdhelp_left.htm%23cshid=stadv_main_all.htm|starttopic=content%2fstadv_main_all.htm|skinname=svs_sd http://help.sciencedirect.com/flare/sdhelp_left.htm%23cshid=stadv_main_all.htm|starttopic=content%2fstadv_main_all.htm|skinname=svs_sd http://www.sciencedirect.com/ http://www.sciencepublishinggroup.com/home/index.aspx http://psycnet.apa.org/doi/10.1037/0033-295x.101.2.357 http://www.1001freedownloads.com/free-clipart/architetto-remix-green-blond-man-icon http://www.springer.com/gp/ http://www.tandfonline.com/ http://www.tcmb.gov.tr/wps/wcm/connect/tcmb+tr/tcmb+tr/main+menu/istatistikler/doviz+kurlari/gosterge+niteligindeki+merkez+bankasi+kurlarii http://www.tcmb.gov.tr/wps/wcm/connect/tcmb+tr/tcmb+tr/main+menu/istatistikler/doviz+kurlari/gosterge+niteligindeki+merkez+bankasi+kurlarii ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 511 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 tzeng, g.h., chen, t.y., & wang, j.c., (1998) a weight-assessing method with habitual domains, european journal of operational research, 110(2), 342–367. doi:10.1016/s0377-2217(97)00246-4 united nations industrial development organization & international center on small hydro power (unido and icshp), (2013). world small hydropower development report 2013. ur-rehman, a. & al-ahamri, a. (2013). evaluation of alternative industrial robots using ahp and electre: a comparative study. international journal of industrial and systems engineering. 14(1). 58-73. doi: 10.1504/ijise.2013.052921 u.s. department of the interior, u.s. geological survey, usgs water science school (usgs water science school) (2015a). the water cycle usgs water science school, "surface runoff the water cycle" http://water.usgs.gov/edu/watercyclerunoff.html accessed in 28/07/2015. u.s. department of the interior, u.s. geological survey, usgs water science school (usgs water science school) (2015b). "how streamflow is measured, part 2: the discharge measurement" http://water.usgs.gov/edu/streamflow2.html accessed in 28/07/2015. u.s. geological survey (usgs) (2015). usgs current conditions for washington, drainage area http://waterdata.usgs.gov/wa/nwis/current? accessed in 28/07/2015. u.s. energy information administration (eia) (2015). http://www.eia.gov/tools/glossary/ accessed on 28/07/2015. van rijsbergen, c.j. (1979). information retrieval http://www.dcs.gla.ac.uk/keith/preface.html#preface accessed in 23/07/2015. wikipedia the free encyclopedia, wikimedia foundation inc (wikipedia) (2015a). deductive reasoning, https://en.wikipedia.org/wiki/deductive_reasoning accessed in 02/08/2015. wikipedia the free encyclopedia, wikimedia foundation inc (wikipedia) (2015b). inductive reasoning, https://en.wikipedia.org/wiki/inductive_reasoning accessed in 02/08/2015. wiley-blackwell/wiley online library (2015). http://onlinelibrary.wiley.com/ accessed and reviewed until 25/07/2015. world scientific publishing (2015). http://www.worldscientific.com/ accessed and reviewed until 25/07/2015. zak, j., fierek, s., kruszynski, m. (2014). evaluation of different transportation solutions with the application of macro simulation tools and multiple criteria group decision making/aiding methodology. procedia social and behavioral sciences, 111, 340-349. doi:10.1016/j.sbspro.2014.01.067 http://dx.doi.org/10.1016/s0377-2217(97)00246-4 http://water.usgs.gov/edu/watercyclerunoff.html http://water.usgs.gov/edu/streamflow2.html http://waterdata.usgs.gov/wa/nwis/current? http://www.eia.gov/tools/glossary/ http://www.dcs.gla.ac.uk/keith/preface.html%23preface https://en.wikipedia.org/wiki/deductive_reasoning https://en.wikipedia.org/wiki/inductive_reasoning http://onlinelibrary.wiley.com/ http://www.worldscientific.com/ http://dx.doi.org/10.1016/j.sbspro.2014.01.067 ijahp article: saracoglu, mahallesi, caddesi, sokak/a comparative study of ahp, electre iii & electre iv by equal objective & shannon’s entropy objective & saaty’s subjective criteria weighting on the private small hydropower plants investment selection problem in turkey international journal of the analytic hierarchy process 512 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.343 zamir, o., & etzioni, o. (1999) grouper: a dynamic clustering interface to web search results. computer networks, 31(11), 1361–1374. doi: 10.1016/s1389-1286(99)00054-7 zanakis, s.h., solomon, a., wishart, n., dublish, s. (1998). multi-attribute decision making: a simulation comparison of select methods. european journal of operational research, 107(3), 507-529. doi: http://dx.doi.org/10.1016/s0377-2217(97)00147-1 zhou, r., cai, r., & tong, g. (2013). applications of entropy in finance: a review. entropy, 15(11), 4909-4931. doi:10.3390/e15114909 zou, z.h., yun, y., & sun, j.n. (2006). entropy method for determination of weight of evaluating indicators in fuzzy synthetic evaluation for water quality assessment. journal of environmental sciences, 18(5), 1020-1023. doi:10.1016/s1001-0742(06)60032-6 zukauskas, p., & vveinhardt, j. (2015). diagnosis of mobbing as discrimination in employee relations. engineering economics, 64(4), 103-113. http://dx.doi.org/10.1016/s0377-2217(97)00147-1 http://dx.doi.org/10.3390/e15114909 http://dx.doi.org/10.1016/s1001-0742(06)60032-6 ijahp: mu/ahp/anp and world connections international journal of the analytic hierarchy process 1 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.869 ahp/anp and world connections welcome to the first ijahp issue of 2021! needless to say, this past year has been particularly challenging for everyone all over the world due to the pandemic. fortunately, the development of vaccines makes us hopeful that the situation will be controlled in the long run, although a return to the “normal” we knew before is unlikely. this worldwide crisis has made us realize how interconnected we are at global level. a virus that appeared on the other side of the world (for those of us who live in the u.s.) became a worldwide problem. i’ve had the opportunity to experience this level of world connection through our ahp/anp work. while there were more than 30 countries represented at the last international symposium of the analytic hierarchy process this past december, this current ijahp issue is a small sample of what i mean by worldwide connectedness. in this issue, there are 7 countries represented, ranging from morocco and malaysia to italy, brazil and chile, among the different pieces published. this is a good sign because, as the current pandemic has shown, our problems are global and the solutions must have a similar scope. enjoy the issue! enrique mu, phd ijahp editor-in-chief ijahp: saaty/words from the special issue editor international journal of the analytic hierarchy process 310 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.555 words from the special issue editor dear friends and colleagues, it has been very gratifying to me to edit this memorial issue of the ijahp in honor of tom saaty, my husband of 53 years. tom passed away at the age of 91 on august 14 th this year of cancer. he fought it bravely for more than a year. his energy was a force of nature, staying with him almost to the very end. with that he also had an unbelievable memory; a few days before he died, he was teaching a chaplain visiting him a neat new proof of the pythagorean theorem he had learned. in honor of tom and his work in developing the analytic hierarchy process (ahp) and the analytic network process (anp), the ijahp is featuring articles from colleagues who had collaborated with him on research papers, or been visiting scholars at the katz school of business administration at the university of pittsburgh. they have shared stories of the work they did with him and of their personal memories and experiences. this special issue provides a fresh look at the wide variety of areas into which tom ventured in applying the ahp, and is a reminder of the breadth of his interests. he was not your dry single-issue mathematician; he was interested in everything around – the whole human experience, the history of human thinking and how the ahp fit in it. he knew the famous works of philosophers and scientists and mathematicians of old and even religion—he could quote chapter and verse from the bible—and he integrated all this knowledge into his works. some have said that the most amazing thing about him was his synthesizing ability, how he could grapple with complexity and bring it under control. enjoy this special issue in tom’s memory! rozann saaty ijahp, guest editor 1 final_fabio_de_felice_editorial ijahp editorial: de felice, fabio / introduction to special issue on sustainable development international journal of the analytic hierarchy process 2 vol. 4 issue 1 2012 issn 1936-6744 dear colleagues, i am pleased to introduce the first special issue on ahp/anp: multicriteria decision making for sustainable development. the aim of this special issue is to provide a selection of papers from researchers, academics, and practitioners that provide significant insights into the area of decision making for sustainable development. decision making in such a context is complex since it requires multiple evaluations of both tangible and intangible factors. in the field of sustainability, tangible factors relate to a wide spectrum of decision variables ranging from production systems performance to industrial costs. the papers collected in this special issue are intended to provide a representative sample of current research at the interface between ahp/anp and sustainable development. this issue attempts to overcome the lack of “communication” between the two fields. we believe that the ahp/anp approach to modeling environmental activities has enormous potential, and that there is much to be gained from their interaction. this issue of ijahp represents an opportunity for many researchers and scientists around the world to present their own studies and research in this important field. therefore, we consider this publication a chance to encourage the integration of international competencies in the field of sustainable development, supplying important cues for future research. undoubtedly, we can affirm that the interest in sustainable development has definitely increased during the past few years, due to the complexity of the environment in which we live, that compels us to face a number of problems which are very great compared to the resources available. in order to face technical, scientific, social, economic and ijahp editorial: de felice, fabio / introduction to special issue on sustainable development international journal of the analytic hierarchy process 3 vol. 4 issue 1 2012 issn 1936-6744 political problems, we must arrange our priorities and determine which elements may prevail over other ones. however, it often happens that our natural inclination to research and “discovery” yields to the complexity involved in reality. this sometimes exceeds our imagination and reason causes us to stop our ambitions in the quest for truth. a journal like ijahp is entrusted to overcome this condition. a great man, j.b. shaw, affirms that while the reasonable man adapts himself to the world, the unreasonable one persists in trying to adapt the world to himself. we owe the changes and innovations in research to this unreasonable man. the isahp conference and the papers collected in the ijahp are an effort to stimulate that unreasonable man within us, and supply him with a place for discussion and confrontation, in order to freely propose ideas and solutions. the applications, methods, instruments and methodologies proposed in the papers presented in this special issue will contribute to strengthen the current research aimed at improving the quality of our environment. we hope that this issue offers an example of the scope and breadth of the growing amount of literature at the interface between ahp/anp and sustainable development, and that it will help readers get acquainted with these new approaches and stimulate further collaborations between scientists of both disciplines. we would like to express our sincere thanks to ijahp editor-in-chief dr. enrique mu for his kind and welcome offer to devote a special issue of the journal of analytic hierarchy process to this topic. we want to thank all the authors who submitted papers and express our gratitude to the referees for their precious support. special thanks to prof. thomas and rozann saaty, dear friends who have given me the opportunity to organize isahp 2011. prof. fabio de felice isahp 2011 executive organizer university of cassino italy engineering faculty rob typewritten text http://dx.doi.org/10.13033/ijahp.v4i1.143 abstract ijahp article: tsai, lo, lin/an analytical framework with sensitivity analysis to quantify management consultancy selection international journal of the analytic hierarchy process 72 vol.5 issue 1 2013 issn 1936-6744 an analytical framework with sensitivity analysis to quantify management consultancy selection cheng-an tsai department of business administration, shih chein university, taipei taiwan, r.o.c. chengan.tsai@msa.hinet.net sheng-wen lo1 department of information management, national central university, taoyuan taiwan, r.o.c. lo.lauren0717@gmail.com tzu-ming lin department of information management, national central university, taoyuan taiwan, r.o.c. jmlin@mgt.ncu.edu.tw abstract this paper provides a comprehensive review and critique of a firms’ management consultancy selection, and presents a new analytical framework with sensitivity analysis to quantify the priority of criteria and sub-criteria in the management consultancy selection. the advantage of this analytical framework is that it adds the quantitative precision and sensitivity analysis to increase applicability and ease of use for decision making in the consultancy selection. through literature reviews and in-depth interviews and case studies, this paper clarifies the importance of criteria in overcoming the difficulty of ex ante evaluation due to information asymmetries when using the analytic hierarchy process. it also extends the alternatives of consultancy selection from a management consulting company to academic consultants, and 1 corresponding author.. tel.: +0-886-3-422-7151#66500 ; fax: +0-886-3-425-4604 e-mail address: lo.lauren0717@gmail.com mailto:chengan.tsai@msa.hinet.net mailto:lo.lauren0717@gmail.com mailto:lo.lauren0717@gmail.com ijahp article: tsai, lo, lin/an analytical framework with sensitivity analysis to quantify management consultancy selection international journal of the analytic hierarchy process 73 vol.5 issue 1 2013 issn 1936-6744 explores the possibilities of a dynamic selection mechanism depending on the nature of the problem, and the circumstances. the results show that the most important criterion is the type of solution (43.51%), followed by the consultant’s ability (25.79%), then the type of problem (24.17%), and lastly the perceived benefits of the consulting project (6.53%). also, the results suggest that a management consulting company is preferred to academic consultants. further analysis of the limit matrix and sensitivity analysis shows that the type of solution, the consultant’s ability and the perceived benefit of the consulting project have direct and dominant influence in the consultancy selection, and the type of problem has the opposite effect in the consultancy selection. this paper contributes to the theoretical research, and improves the practitioner’s ability to achieve a proper selection in management consultancy. keywords: multiple criteria decision making, management consultancy selection, analytic hierarchy process, priority analysis, sensitivity analysis 1. introduction the purchase of a wide range of external professional consultancy services by enterprises has been a rapidly growing phenomenon in recent decades (altinay, 2004; bennett and smith, 2004; dyer and ross, 2007; freeman et al., 2007; lu et al., 2010; sonmez and moorhouse, 2010; wynstra et al., 2006). the investigation of british small-to-median enterprises (sme) showed that by 2002, 95 percent of smes had already used external consultancy services for over three years (bennett and robson, 2002). a survey revealed that 97 percent of the top 200 companies in the u.k. and u.s. have used professional consultants (clark, 2007). empirical reports also showed that there has been a general shift of management consultancy service from 'west to east' (hanssens et al., 2011). the purchase of external professional consultancy has become the worldwide phenomenon. the main objectives of enterprises using these services include the ability to resolve problems or improve situations when it is not possible to have a quick solution for urgent needs (soriano et al., 2002). enterprises use consultancy services when they have experienced difficulty gaining access to certain modern technologies (kirby and jones-evans, 1997), when they lack certain capabilities or knowledge for growth or competition (kumar et al., 2000; richter and niewiem, 2009), and when they need an rob typewritten text http://dx.doi.org/10.13033/ijahp.v5i1.167 ijahp article: tsai, lo, lin/an analytical framework with sensitivity analysis to quantify management consultancy selection international journal of the analytic hierarchy process 74 vol.5 issue 1 2013 issn 1936-6744 agent of change or standardization of organizational practice around the world (furusten, 2009). in today’s competitive environment, management consultants are important business partners for enterprises in problem solving (balconi and laboranti, 2006; corcoran and mclean, 1998; furusten, 2009; gustafson and di marco, 1973; sáez et al, 2002), knowledge interaction (bell, 1993; knudsen, 2007; natti et al., 2006; richter and niewiem, 2009; sáez et al., 2002; schartinger et al., 2002), new product development (knudsen, 2007; mitchell, 1994), information technology adoption (wei et al., 2005), and product quality control (saremi et al., 2009). enterprises can learn from consultants, and the benefits they produce exceed the cost of their services. as a result, it is critical for enterprises to know how to select an appropriate consultant. previous research has identified many criteria that are involved in consultancy selection. interestingly, some research argued that the existing relations with clients, third-party recommendations, and prior experiences were the most important criteria (bennett and smith, 2004; corcoran and mclean, 1998; richter and niewiem, 2009). others suggested that the nature of the problem was focused on the quality of the consultants, the consultant’s reputation, and product features (dawes et al., 1992; sonmez and moorhouse, 2010; soriano et al., 2002). these contradictory results make it necessary to further investigate the differences from various viewpoints, and to provide a comprehensive overview to help enterprises make a better decision. when encountering business problems, enterprises will seek help from more than one consultant. management consulting companies and academic consultants are the two major sources from which they seek help (furnham and pendleton, 1991; gustafson and di marco, 1973). in taiwan, 38.2 percent of smes employ academic consultants, and 31.6 percent employ management consulting companies (moea, 2008). academics agree on the importance of engaging academic consultants (docherty and smith, 2007; furnham and pendleton, 1991; ormerod, 1996; schartinger et al., 2002), but little attention has been given to academic consultant selection. furnham and pendleton (1991) have argued that there is a large difference between academic consultants and a management consulting company. furthermore, the literature shows inconsistent results when discussing the most important criterion for management consultancy selection. it is debatable which criterions are more important and how they rank in terms of priority. those ambiguities undermine the applicability of current knowledge concerning management consultant selection. this ijahp article: tsai, lo, lin/an analytical framework with sensitivity analysis to quantify management consultancy selection international journal of the analytic hierarchy process 75 vol.5 issue 1 2013 issn 1936-6744 paper plans to reexamine a firms’ consultancy selection through literature review and in-depth interview and case studies to present a new analytical framework of better decision-making in consultancy selection. 2. literature review and research focus management schools should be able to engage in a consultancy-driven environment and conduct research relating to practices performed by a management consulting company (docherty and smith, 2007). ormerod (1996) also recommended that management schools combine management consultancy and research. these two types (sources) of management consultancy seem to have similar features in terms of consultant work. furnham and pendleton (1991), however, reported differences in objectives, methods, and solutions between academic consultants and management consulting companies. an academic consultant’s intention is to create knowledge, to understand the root cause of the problem, or to have a comprehensive understanding from management research. this knowledge may not be useful or applicable. on the other hand, the mission of the management consulting company is to provide quick and applicable solutions for urgent problems. academics “are usually satisfied to know and understand the root cause of the problem, while the consultants use the knowledge” to solve the problem (furnham and pendleton, 1991, p. 16). according to furnham and pendleton (1991), firms may choose between a management consulting company and an academic consultant based on the objective of the consulting project or the solution they are seeking. many researchers suggested that firms should access a more comprehensive set of criteria, including the characteristics of consultancy, consulting ability, service contents, and cost and benefit of the consultancy (burke and bandick, 1997; saremi et al., 2009; wei et al., 2005). this paper argues that firms should employ either an academic consultant or management consulting company based on different criteria or by the different priority of the criteria with the nature of the consultancy in mind. the management consulting company and the academic consultant are empirically identified as the two major sources of consultancy providers, and since they are vastly different it is necessary to refine the criteria used in the selection process and to elaborate the difference of priority between the criteria used. ijahp article: tsai, lo, lin/an analytical framework with sensitivity analysis to quantify management consultancy selection international journal of the analytic hierarchy process 76 vol.5 issue 1 2013 issn 1936-6744 management consultancy is an unregulated market surrounded by information asymmetries that the enterprise must overcome ex ante evaluation as the delivery of the management consultancy is considerably complex and uncertain (bennett and smith, 2004; clark, 1993; fitzsimmons et al., 1998; gallouj, 1997). the enterprise is unable to accurately evaluate the quality of the service or advice purchased because the process of purchasing advice appears to be very complex, and most existing models seem unsuitable (gallouj, 1997; sonmez and moorhouse 2010). sonmez and moorhouse (2010) suggested a two-stage process for selecting consultancy providers. the first stage, named pre-qualification, involves narrowing the choices from a large number of possible service providers. the final stage, differentiators, outlines the firm’s decision to contract with a selected one. the focus of this paper is on the first stage. sonmez and moorhouse (2010) argued that in the first stage a consultant’s reputation, organizational capability, and cost are the main criteria, but the cost has been empirically confirmed as the least important role in the management consultancy selection (dawes et al., 1992; soriano et al., 2002). sonmez and moorhouse’s perspective seems to be inconsistent with empirical results. it is necessary to reinvestigate the criteria used in the first stage. the overall goal of this paper is to reexamine a firms’ consultancy selection process, and to reconstruct the selection criteria in the context of alternative consultancy selections. through a general literature study on consultancy selection, some “more important” criteria are determined. the first is the consultants’ reputation, which is the most important criterion in a clients’ evaluation process (corcoran and mclean, 1998; sonmez and moorhouse, 2010). a consultant's commitment (burke and bandick, 1997; wei et al., 2005), expertise (day and barksdale, 2003; sonmez and moorhouse, 2010), and relevant experience in similar projects (gustafson and di marco, 1973; saremi et al., 2009) are identified as other more important criteria. moreover, a consultant’s relevant experiences in the industry and third-party recommendations are other important criteria to be considered (bennett and smith, 2004). richter and niewiem (2009) highlighted that ‘a consultant’s experience in a client’s industry was an important knowledge-related factor that led clients to decide in favor of an external consultant, and …the lack of relevant industry experience was ranked as the most important reason for clients not to choose a consultant. (p. 276)’. ijahp article: tsai, lo, lin/an analytical framework with sensitivity analysis to quantify management consultancy selection international journal of the analytic hierarchy process 77 vol.5 issue 1 2013 issn 1936-6744 the relationship between clients and service providers are the most influential criteria in the selection process. many managers and employees also showed greater preference for a consulting company with whom they had cooperated before (bennett and smith, 2004; furusten, 2009). firms believe that a good relationship will help a consultant to better understand their needs and facilitate a higher-quality project performed by the consultant. (furusten, 2009; west, 1997).some scholars claimed cost of the work as an important criterion used for selection (sonmez and moorhouse, 2010). others asserted that cost should assume a less important role in the selection of a management consulting company (soriano et al., 2002). consultancy service providers suggested a management consulting company must offer its project at a reasonable price, because quality and cost are the most relevant criteria when engaged in management consulting services. for clients, the cost whether financial or not is not the main concern when the project is beneficial to them (lapiedra et al, 2011; sonmez and moorhouse, 2010; soriano et al., 2002; fontana et al., 2006; schein, 1990). both management consulting companies (corcoran and mclean, 1998) and academic consultants (balconi and laboranti, 2006; sáez et al., 2002) are professionals whose aim is to solve problems. therefore, schein’s (1988, 1990) model reveals that there is no one-size-fits-all solution. a consultant should offer a customized solution based on the type of problem their client has (ormerod, 1996; schein, 1988, 1990). the above literature review focused mainly on the selection of a management consulting company. because of the large difference between the academic consultant and the management consulting company, this study conducted in-depth interview and case studies to explore the firms’ evaluation of the academic consultant and the management consulting company simultaneously. 3. methodology this study first interviewed the enterprises and academics who participated in the academic-industry cooperation program launched by the nsc (national science council) and a group of academic business and management schools in taiwan. during the interview we gathered information related to consultant selection, especially for academic consultants, which is almost absent from literature (docherty and smith, 2007; schartinger et al., 2002). the second step was to identify and rank ijahp article: tsai, lo, lin/an analytical framework with sensitivity analysis to quantify management consultancy selection international journal of the analytic hierarchy process 78 vol.5 issue 1 2013 issn 1936-6744 the criteria used in the multiple sources of selection decision. to do so, the multi-criteria decision making (mcdm) was applied. the case study method is appropriate for exploratory research (yin, 2002), such as this study. the case study is also an empirical investigation into a contemporary phenomenon within its real-life context. a multiple-case study design is used for a more robust replication of results recommended by yin (2002). the unit of analysis for this study is the specific consultancy project, defined as a case. within this context, each of the consultants and managers involved in the specific project could be considered a sample of the case. six cases were selected based on the firms’ industry (two firms in the manufacturing industry; four firms in the service industry) and prior cooperation with two sources of consultancy from the university-industry cooperation project launched by nsc in taiwan. this study performed twenty interviews with responsible managers, including leaders of academic consultancy groups and senior managers. the amount of time spent in in-depth interviews was more than thirty hours (from one to two hours) with the six case studies (case 1, 2 interviews; case 2, 2 interviews; case 3, 4 interviews; case 5, 6 interviews; case 5, 2 interviews; case 6, 4 interviews). the interviews were recorded and transcribed, and the validations were received through feedback and respondents’ validation. the convergence of multiple data sources and key informant reviews was to ensure construction validity. multiple case studies and relevant literature as data sources could help this paper to obtain external validity. the qualitative data was coded and labeled according to qualitative analysis methods (charmaz, 2006). for codification, several stages were applied iteratively throughout the study which resulted in twenty-three indicators, named sub-criteria. these were then categorized into five groups, named criteria, according to relevant perspectives. three criteria that are based on the emphasis of previous literature are consultant’s ability, perceived benefits, and project cost. the last two criteria, based on suggestions from in-depth interviews, are type of problems and type of solutions. these were mentioned very little in previous literature. the five criteria and indicators are given in table 1. as table 1 shows, this study generated five new indicators beyond what the literature discussed, including (1) organizational resources in consultant’s ability, (2) additional ijahp article: tsai, lo, lin/an analytical framework with sensitivity analysis to quantify management consultancy selection international journal of the analytic hierarchy process 79 vol.5 issue 1 2013 issn 1936-6744 resources involved in project cost, (3) cost evaluation of project, (4) performance measurement in perceived benefit, and (5) best practice in the specific type of solution. some interviewees stressed the importance of organizational resources available to clients, such as a market database, designated support teams as backstage support, commercial intellectual property rights, and knowledge bases all over the world. consultants can mobilize these kinds of resources to better serve their clients. ijahp article: tsai, lo, lin/an analytical framework with sensitivity analysis to quantify management consultancy selection international journal of the analytic hierarchy process 80 vol.5 issue 1 2013 issn 1936-6744 table 1 the criteria used for consultancy selection criteria sub-criteira brief description reference consultant’s ability hands-on experiences a consultant’s hand-on experiences in industries. bennett and smith, 2004;gustafson and di marco, 1973; richter and niewiem, 2009 reputation the consultant’s reputation, brand or referral from third party. bennett and smith, 2004;corcoran and mclean, 1998; sonmez and moorhouse, 2010. commitment a consultant’s ability to satisfy a firm’s needs. burke and bandick, 1997; wei et al., 2005. expertise the consultant’s expertise. day and barksdale, 2003; sonmez and moorhouse, 2010; . previous working relationship a consultant’s working relationship with the firm. bennett and smith, 2004;freeman et al., 2007; furusten, 2009; mitchell, 1994; richter and niewiem, 2009; west, 1997 organizational resources the organizational resources available to the project which consultants can mobilize. interviews ijahp article: tsai, lo, lin/an analytical framework with sensitivity analysis to quantify management consultancy selection international journal of the analytic hierarchy process 81 vol.5 issue 1 2013 issn 1936-6744 criteria sub-criteria brief description reference perceived benefit of the project project quality the proposal of the project can meet a firm’s needs. mitchell, 1994; soriano et al., 2002; sonmez and moorhouse, 2010; lapiedra et al, 2011. the ratio of cost/benefit a firm perceives that the financial benefits received from a project are positive related to the project cost. soriano et al., 2002. non-financial benefits a firm perceives that he/she may receive some non-financial benefits from a project, for example, knowledge, the feasibility of organization learning. natti et al., 2006; richter and niewiem, 2009; schein, 1990. financial benefits a firm perceives that he/she may receive financial benefits as soon as he/she carries out the project. fontana et al., 2006. performance measurement a project’s performance can be measured with a group of indicators. interviews project cost consulting fee consulting fees of the project gustafson and di marco, 1973; wei et al., 2005. funding the project has been fund by organization, university-industry cooperation in common. balconi and laboranti, 2006; sáez et al., 2002; additional resources involved a firm has to allocate other resources beyond the contract. interviews cost evaluation the consultancy provider has a well-defined measurement to estimate the total cost of the project. interviews ijahp article: tsai, lo, lin/an analytical framework with sensitivity analysis to quantify management consultancy selection international journal of the analytic hierarchy process 82 vol.5 issue 1 2013 issn 1936-6744 criteria sub-criteria brief description reference the type of problem urgency a firm wishes that his/her problems can solve as soon as possibility. schein, 1990; ormerod, 1996. complexity a firm has to organize a cross-functional team for problem identity and solving. schein, 1990; ormerod, 1996. clearness a firm has known what problem is or has the root of problem before the beginning of project. schein, 1990; ormerod, 1996. the type of solution best practice a firm wishes that he/she may receive best practice from the project. interviews information about problem a firm wishes that he/she may have valuable information relating to a given problem from the project. schein, 1990. feasible solution a firm wishes that he/she may have a feasible solution received from the project. balconi and laboranti,2006;sáez et al.,2002; schein, 1990 problem modeling a firm wishes that he/she may acquire information about the causality of a given problem, and he/she will look for a solution himself/herself. schein, 1990. ijahp article: tsai, lo, lin/an analytical framework with sensitivity analysis to quantify management consultancy selection international journal of the analytic hierarchy process 83 vol.5 issue 1 2013 issn 1936-6744 in dealing with academic consultants, some interviewees argued that firms need additional resources beyond the contract, such as human resources, for project execution. in the case of cooperation with a consulting company, a well-defined cost evaluation and performance measurement is required due to the high expense. some interviewees suggested that providing best practice is a more preferable reason for consultancy selection. with respect to our thesis, firms employ either an academic consultant or a consulting company based on either different criteria or by different priority of criteria. all senior managers agreed that they use similar criteria for the selection of either the academic consultant or the consulting company, but use varied priorities of criteria in the consultancy selection. the results of the case study partially supported our argument. it is necessary to elaborate on the different priority of criteria between the two sources of consultancy. henceforth, this study adopted murry and hammons’ (1995) suggestion to generate a delphi-based questionnaire. we invited nine senior managers to rank the twenty-three indicators from least important to most important (from1 to 9). six of twenty-three indicators were removed due to low mean (< 5); interestingly, four of them are related to project cost. for further evidence, this study asked them to rank the five criteria from 1 to 9. the mean of project cost was still less than five (see table 2). based on the statistical results, this study removed project cost from table 1 and concluded that project cost is the least important criteria in the pre-qualification stage. table 2 the descriptive statistics of criteria criteria mean s.d. the type of solution 8.3 0.5 consultant’s capability 7.7 1.2 the type of problem 7.7 1.5 perceived benefit of consulting project 6.3 0.6 cost 4.3 1.2 the criteria and indicators have been integrated through a multiple decision -making method using the analytic hierarchy process (ahp) to rank the selection of evaluation decision problems (golden et al., 1989; saaty, 1987; saaty, 1994). during the years 2005-2009, ahp has been used exponentially as a managerial decision tool ijahp article: tsai, lo, lin/an analytical framework with sensitivity analysis to quantify management consultancy selection international journal of the analytic hierarchy process 84 vol.5 issue 1 2013 issn 1936-6744 in many industries for strategy evaluation, performance assessment, product and process design, risk evaluation, system selection, cost/benefit analysis, quality evaluation, and measurement of objectives (sipahi and timor, 2010). thus this paper chooses the ahp to formulate the proposed model. in the ahp, a typical decision problem is structured in three levels: the decision goal, criteria used to judge, and the alternatives for the decision goal (sipahi and timor, 2010). the criteria have been structured into a hierarchical decision scheme (see table 1).following ahp, the indicator is called the sub-criteria. the advantages of ahp include its ability to make both qualitative and quantitative decision attributes commensurable, and its flexibility in terms of setting the objective (naesens et al, 2007). this ahp framework has been tested by the case study described below. the company in the case study is synthetic rubber company (src) located in taiwan. the company has more than 1,000 employees, and its revenue was approximately 1.2 billion us dollars in 2010. this company has vast experience cooperating with both academic consultants and management consulting companies. the necessary data (weight and scope per criterion) were provided by six managers of the committee responsible for the consultancy project. 4. analysis and discussion this study follows the ahp procedures recommended by saaty (2008). 1. define the problem and determine the kind of knowledge sought: the purpose of this project for src was to improve its business performance. the project committee expected to obtain consultancy services from either a management consulting company or an academic consultant through the university-industry cooperation plan. therefore, their first decision was to determine the source of the consultancy provider. 2. structure the decision hierarchy from the top with the goal of the decision, then the objectives from a broad perspective, through the intermediate levels to the lowest level: in this ahp model of src, the problem is structured as a hierarchical decision scheme, consisting of four criteria, 18 sub-criteria (see table 1 except project cost), and two alternativesacademic consultant or management consulting company, as depicted in figure 1. ijahp article: tsai, lo, lin/an analytical framework with sensitivity analysis to quantify management consultancy selection international journal of the analytic hierarchy process 85 vol.5 issue 1 2013 issn 1936-6744 figure 1 hierarchical decision structure 3. construct a set of pairwise comparison matrices where each element in an upper level is compared with the elements in the level immediately below: once the objective has been set, the committee should give a priority to each element, criteria, sub-criteria, and alternative. an ahp questionnaire recommended by saaty (1987, 1994) was made for measurement. the ahp measured the intensity of importance of one element over another element using a 1-9 scale through two-by-two comparisons. 4. use the priorities obtained from the comparisons to weigh the priorities in the level immediately below. then for each element in the level below, add its weighed values and obtain its overall or global priority until the final priorities of the alternatives are obtained: for this step, we used the super decisions software (creative decision foundation, 2013), which was developed by saaty, to obtain goal: consultancy perceived qual ity of project the type of problem the type of solution the consulting company the academic consultant consultant’s ability the ratio of cost/benefit project financial performance measurement non-financial benefits hands-on experiences reputation commitmen expertise previous working relationship organizat ional urgency complexity clearness best information about feasible solution problem modeling ijahp article: tsai, lo, lin/an analytical framework with sensitivity analysis to quantify management consultancy selection international journal of the analytic hierarchy process 86 vol.5 issue 1 2013 issn 1936-6744 the result of the elements’ priority. the results of the pair-wise comparisons are given in tables 3 and 4 (all the comparison matrices are presented in appendix 1). both criteria and sub-criteria are ranked according to their own priority in order to evaluate the different elements. it is necessary to check the consistency of the pair-wise comparison. thus, the consistency index (ci) must be used to determine whether it would comply with the principles and considerations of the pairwise comparison. the range of the ci should be below 0.1. moreover, the consistency ratio (cr) is used to measure the coherence of the pair-wise comparison. the acceptable levels of the cr should be less than 0.1 for the matrix of n > 4 (saaty, 2000). the range of ci and cr value of this study is 0.0494 and 0.025, respectively, satisfying the requirements. table 3 the priority of each element criteria priority of criteria sub-criteria priority of sub-criteria global weights (ranks) the type of solution 0.4351 feasible solution 0.5763 0.2507 (1) best practice 0.2418 0.1052 (3) problem modeling 0.1056 0.0459 (9) information about problem 0.0764 0.0332 (10) consultant’s ability 0.2579 reputation 0.3850 0.0993 (4) commitment 0.2470 0.0637 (5) organizational resources 0.1928 0.0497 (8) previous working relationship 0.0639 0.0165 (12) expertise 0.0655 0.0156 (13) hands-on experience 0.0459 0.0118 (15) the type of problem 0.2417 complexity 0.5499 0.1329 (2) urgency 0.2403 0.0507 (6) clearness 0.2099 0.0581 (7) ijahp article: tsai, lo, lin/an analytical framework with sensitivity analysis to quantify management consultancy selection international journal of the analytic hierarchy process 87 vol.5 issue 1 2013 issn 1936-6744 criteria priority of criteria sub-criteria priority of sub-criteria global weights (ranks) perceived benefit of consulting project 0.0653 performance measurement 0.4819 0.0057 (11) the ratio of cost/benefit 0.2137 0.014 (14) project quality 0.1809 0.0118 (16) financial benefits 0.0879 0.0315 (17) non-financial benefits 0.0356 0.0023 (18) the range of independent ci are from 0.000 to 0.0494 cr = 0.025 < 0.1; the range of independent cr are from 0.000 to 0.0441 table 4 priorities of the alternatives criteria the final priority of criteria the management consulting company the academic consultants the type of solution 0.342 0.158 consultant’s ability 0.391 0.109 the type of problem 0.247 0.253 perceived benefit of consulting project 0.392 0.108 priorities of the alternatives (normals) 0.670 0.330 priorities of the alternatives (ideals) 1.000 0.491 rank 1 2 4.1. the priority of criteria the column of priority of criteria in table 3 shows that the most important criterion is the type of solution (0.4351), followed by the consultant’s ability (0.2579), then the type of problem (0.2417), and lastly, the perceived benefit of the consulting project (0.0653). this result implies that in the pre-qualification stage the committee looked for a consultant who had the ability to offer an appropriate solution to the firm’s problem. the empirical results also echo our treatment of the criteria category to ijahp article: tsai, lo, lin/an analytical framework with sensitivity analysis to quantify management consultancy selection international journal of the analytic hierarchy process 88 vol.5 issue 1 2013 issn 1936-6744 separate the type of solution and the type of problem. previous literature only highlighted the importance of providing a customized solution based on the type of a clients’ problems (ormerod, 1996; schein, 1988, 1990). this study distinguishes these two criteria based on the perspective of the interviewees. the type of solution is ranked the highest priority of management consultancy selection in the empirical result. the type of problem is ranked third in the management consultancy selection priority. this result further elaborates on previous scholars’ views. the previous literature mainly emphasized the importance of a consultants’ ability in the management consultancy selection, including hands-on experiences (bennett and smith, 2004; sonmez and moorhouse, 2010 ), reputation (bennett and smith, 2004; corcoran and mclean, 1998; sonmez and moorhouse, 2010), commitment (burke and bandick, 1997; wei et al., 2005), expertise (day and barksdale, 2003; sonmez and moorhouse, 2010), and previous working relationship (bennett and smith, 2004; freeman et al., 2007; furusten, 2009; mitchell, 1994; richter and niewiem, 2009; west, 1997). the empirical results show that the consultants’ ability is ranked second. however, the consultants’ ability, the consultant’s reputation, commitment and organizational resources are still major considerations. the previous working relationship, expertise, and hands-on experiences are the relatively less important sub-criteria. interestingly, the perceived benefit of the consulting project is the least important criteria. some scholars highlighted that the firms expect to receive benefits from consultancy services, e.g. knowledge acquirement and increased sales (fontana et al., 2006; natti et al., 2006; richter and niewiem, 2009; schein, 1988; 1990). other scholars stressed that the whole reason to use the services of the consultants is to benefit the firms in both financial and non-financial aspects (fontana et al., 2006; lapiedra et al, 2011; schein, 1990; sonmez and moorhouse, 2010; soriano et al., 2002). this empirical result shows that the perceived benefit of the consulting project is relatively less important in the first stage of consultancy selection process. the column of priority of sub-criteria in table 3 represents the priority of each sub-criterion within its criteria. for comparison, the global weight of each sub-criterion is multiplied by the priority of its criterion. the column of global weights in table 3 shows the five most important sub-criteria are the feasible solution (0.2507) in the type of solution, complexity (0.1329) in the type of problem, best practice (0.1052) in the type of solution, reputation (0.0993), and commitment (0.0637) in the consultant’s ability. this result implies that the committee looked for both an appropriate solution and the consultant’s reputation. the top five priorities of ijahp article: tsai, lo, lin/an analytical framework with sensitivity analysis to quantify management consultancy selection international journal of the analytic hierarchy process 89 vol.5 issue 1 2013 issn 1936-6744 sub-criteria further elaborate how the committee conducted their consultancy selection. when encountering a complex problem, the committee employed the academic consultant or management consulting company to generate a feasible solution and the best practice through strong commitment and reputation of the consultancy. through in-depth interviews, this study generated five new indicators beyond the literature. only the best practice in the type of solution is ranked among the top five priorities of global weights. the empirical result shows that the best practice is valuable and important for consultancy, which was not often mentioned in the literature. the organizational resources in the consultant’s ability are ranked eighth in the global weights. previous literature emphasized the importance of the consultant’s reputation (bennett and smith, 2004; corcoran and mclean, 1998; sonmen and moorhouse, 2010) and commitment (burke and bandick, 1997, wei et al., 2005) in the consultant’s ability. the empirical result echoes interviewees’ opinions that consultants should mobilize organizational resources such as a market database, designated support teams, commercial intellectual property rights, and knowledge bases all over the world to better serve clients. therefore, the organizational resources should be used to judge the consultant’s ability. the next new indicator is performance measurement in the perceived benefit of the consulting project. this priority is ranked eleventh in global weights. it is still the most important indicator within the perceived benefit of the consulting project, which implies that in the first stage of the consultancy selection process, a well-defined performance measurement should be established for the firm to evaluate how well the project will proceed. although previous literature highlighted the importance of the expected benefit of knowledge acquisition from consultancy services (fontana et al., 2006; richter and niewiem, 2009; schein, 1988; 1990), expected quality of the consultants’ service (schein, 1988; soriano et al., 2002), expected benefit in financial and non-financial aspects(fontana et al., 2006; schein, 1990; sonmez and moorhouse, 2010; soriano et al., 2002), the interviewees argued that a set of solid performance measures are more important than expectations. the empirical result confirms this point of view. the other two new indicators are additional resources involved and cost evaluation in the project cost which have been removed in the process of categorizing the criteria. ijahp article: tsai, lo, lin/an analytical framework with sensitivity analysis to quantify management consultancy selection international journal of the analytic hierarchy process 90 vol.5 issue 1 2013 issn 1936-6744 4.2. selecting the appropriate consultancy provider the most appropriate consultancy provider is presented in table 4, which shows two forms of priority. the row of the priority of alternative (normals) shows the results normalized for each alternative, and the row of the priority of alternative (ideals) present the results obtained by dividing the values in the normalized rows by the largest value in the row (saaty, 2003). table 4 shows the management consulting company (0.670) would be preferred above the academic consultant (0.330) in the priority of alternatives (normals). the interpretation of the row of the priority of alternative (ideals) suggests that the academic consultant is 49.1% as good as the management consulting company. furusten (2009) however argued that it is debatable whether or not the management consulting company is better at focusing on matters that are relevant to organizations than the academic consultant (p 265). the empirical results show that the committee prefers a management consulting company over an academic consultant. 5. improvement analysis it is necessary to know whether each criterion has the dominant or restrained influence on the alternatives of consultancy selection in order to make a better decision. table 4 also illustrates the final priority of criteria obtained from the limit matrix by super decisions software. the limit matrix is obtained when the weighted matrix is multiplied by itself several times to converge to a fixed weight of each element. it will display the intermediate priorities under every node in the model, and the final priority of each element obtained from the limit matrix can represent its dominant or restrained influence on the alternatives (saaty, 2003). for the type of solution, the number of limit priority of criteria for selecting a management consulting company is 0.342, greater than the number for selecting the academic consultant of 0.158. this result complies with the results judged by the priority of alternatives (normals) which implies that the type of solution has a dominant influence on the alternative selection. put another way, a management consulting company would always be preferable over an academic consultant if considering the criteria of the type of solution. the same results happen for the other two criteria: consultants’ ability and the perceived benefit. the result for the type of problem is different. the number of limit priority on criteria for selecting a management consulting company is 0.247, less than that of an academic consultant’s 0.253, which contradicts the results judged by the priority of alternatives (normals). ijahp article: tsai, lo, lin/an analytical framework with sensitivity analysis to quantify management consultancy selection international journal of the analytic hierarchy process 91 vol.5 issue 1 2013 issn 1936-6744 this implies that the type of problem has restrained influence on the alternative selection. put another way, a management consulting company may be a preferred selection only to a certain threshold of the weight of the type of problem. if considering more weight on the type of problem, exceeding the threshold, the academic consultant will become the preferred selection. only the type of problem has this reserved effect, implying that the type of problem has a greater influence on the academic consultant in a certain situation. to find more evidence to support this reversed effect, this study conducted a sensitivity analysis through super decisions software to explore at what threshold the selection would have a reverse effect. the sensitivity analysis shows that if the priority of the type of problem is greater than 0.85, the academic consultant becomes the preferred selection; otherwise, a management consulting company is the best alternative. this paper also conducted the experiment on the other three criteria and found that the outcome is very stable, having no effect on the overall ranking. this paper also conducted a sensitivity analysis to explore the reverse effect on each sub-criterion. the outcome of the sensitivity analysis on each one is very stable and does not change the overall ranks except with three criteria: complexity, problem modeling, and information about problem. the empirical results show that if the priority of complexity, problem modeling and information about the problem are greater than 0.95, 0.91 and 0.91, respectively, the academic consultant becomes the preferred selection; otherwise, the management consulting company is the best. the evidence of the sensitivity analysis shows the type of problem, complexity, problem modeling and information about problem in the type of solution have reversed effect. 6. conclusions this paper has presented the ahp as an analytical framework to quantify the priority of criteria and sub-criteria in management consultancy selection. it adds the quantitative precision and sensitivity analysis to increase the applicability and ease of use for decision making on consultancy selection and responds to the need expressed by ‘most existing models seem unsuitable’ (gallouj, 1997; sonmez and moorhouse 2010). there are several unique features about the proposed framework. first, it incorporates varied perspectives of selecting criteria through literature reviews and in-depth interview and case studies. it clarifies the importance of criteria to overcome the difficulty of ex ante evaluation due to the information asymmetries in management consultancy selection (bennett and smith, 2004; clark, 1993; ijahp article: tsai, lo, lin/an analytical framework with sensitivity analysis to quantify management consultancy selection international journal of the analytic hierarchy process 92 vol.5 issue 1 2013 issn 1936-6744 fitzsimmons et al., 1998; gallouj, 1997). second, previous studies are qualitative in nature and include neither an analytical framework nor corresponding alternatives. this proposed framework does not eliminate the subjectivity, but it adds quantitative precision and fine-tuning to an otherwise qualitative decision-making process. in consonance with the previous evidence, the importance of a consultants’ ability in the management consultancy selection is emphasized (bennett and smith, 2004; burke and bandick, 1997; corcoran and mclean, 1998; day and barksdale, 2003; freeman et al., 2007; furusten, 2009; mitchell, 1994; richter and niewiem, 2009; sonmez and moorhouse, 2010; wei et al., 2005; west, 1997). this paper emphasizes the importance of the type of solution and the type of problem. the results further elaborate the importance of previous scholars’ views (ormerod, 1996; schein, 1988, 1990) and offer the potential to improve the practitioner’s knowledge to achieve a proper selection in management consultancy. third, the proposed framework extends the alternatives of consultancy selection from a management consulting company to an academic consultant. it explores the possibilities of situations where an academic consultant may be preferable by conducting advanced analysis of the limit matrix and a sensitivity analysis of the ahp methodology. this paper finds that the type of solution, consultants’ ability and the perceived benefit of the consulting project have dominant influence on selecting a management consulting company, which implies that a management consulting company would almost always be preferred over an academic consultant. however, the type of problem has restrained influence in selecting a management consulting company, which implies that a management consulting company may be the preferred selection only to a certain threshold of weight of the type of problem. if considering greater weight on the type of problem to exceed the threshold, the academic consultant will be the preferred selection. this reverse effect emphasizes that the type of problem has greater influence on the selection of the academic consultant in certain situations. further exploration of the reverse effect on each sub-criterion, shows that complexity, problem modeling, and information about the problem have the reverse effect. the findings elaborate some scholars’ perspectives (furnham and pendleton, 1991) that academic consultants are usually satisfied to know and understand the root cause of a complex problem and are valuable in helping firms identify the problem or acquire information about the cause of a given problem. finally, the case study provided in this paper demonstrates the applicability and ease of use for decision making in consultancy selection. the findings of this paper provide insights into the original research questions of practical implications. ijahp article: tsai, lo, lin/an analytical framework with sensitivity analysis to quantify management consultancy selection international journal of the analytic hierarchy process 93 vol.5 issue 1 2013 issn 1936-6744 although this paper illustrates the use and benefit of the proposed analytical model, further evaluation and refinement of the model extending to additional field studies, such as small enterprises, may prove that it can have greater validity and applicability. even though the ahp method demonstrates a powerful decision tool for assisting managers in many situations, it does not take into account dependencies and interrelations among factors. however, real world problems usually consist of dependence or feedback between elements. compared to the ahp, the analytic network process (anp) method makes it possible to consider all kinds of dependence and feedback in a decision problem. thus, future study may use the anp to provide a more flexible model to further apply to real world situations. references altinay, l. (2004). implementing international franchising: the role of intrapreneurship. international journal of service industry management, 15(5), 426-443. balconi, m. and laboranti, a. (2006). university–industry interactions in applied research: the case of microelectronics. research policy, 35(10), 1616–1630. bell, e. r. j. (1993). some current issues in technology transfer and academic-industrial relation: a review. technology analysis and strategic management, 5(3), 307-321. bennett, r. j. and robson, p. j. a. (2002). external advice and business link. in a. coshand a. hughes (eds), enterprise challenged: policy and performance in the british sme sector 1999-2002 (57-71). cambridge: university of cambridge, centre for business research. bennett, r. j. and smith, c. (2004). the selection and control of management consultants by small business client. international small business journal, 22(5), 435-462. burke, l. a. and bandick, m. a. (1997). how to select a consultant. training and development, 12, 50-51. charmaz, k. (2006), constructing grounded theory: a practical guide through qualitative analysis, sage publications, london. ijahp article: tsai, lo, lin/an analytical framework with sensitivity analysis to quantify management consultancy selection international journal of the analytic hierarchy process 94 vol.5 issue 1 2013 issn 1936-6744 clark, t. (1993). the market provision of management services, information asymmetries and service quality some market solutions: an empirical example. british journal of management, 4(4), 235-251. clark, t. (2007). the world’s newest profession: management consulting in the twentieth century. administrative science quarterly, 52(1), 142-145. corcoran, j. and mclean, f. (1998). the selection of management consultants: how are governments dealing with this difficult decision? an exploratory study. international journal of public sector management, 11(1), 37-54. creative decisions foundation (2013). super decisions software. received from http://www.superdecisions.com/. dawes, p. l., dowling, g. r. and patterson, p. g. (1992). criteria used to select management consultants. industrial marketing management, 21(3), 187-193. day, e. and barksdale, h. c. (2003). selecting a professional service provider from the short list. journal of business and industrial marketing, 18(6/7), 564 579. docherty, i. and smith, d. (2007). practicing what we preach? academic consultancy in a multi-disciplinary environment. public money and management, 27(4), 273-280. dyer, l.m. and ross, c. a. (2007). advising the small business client. international small business journal, 25(2), 130-151. fitzsimmons, j. a., noh, j. and thies, e. (1998). purchasing business services. journal of business and industrial marketing, 13(4/5), 370-380. fontana, r., geuna, a., and matt, m. (2006). factors affecting university–industry r&d projects: the importance of searching, screening and signaling. research policy, 35(2), 309–323. freeman, s., cray, d. and sandwell, m. (2007). networks and australian professional services in newly emerging markets of asia. international journal of service industry management, 18(2), 152-166. furnham, a. and pendleton, d. (1991). the academic consultant. journal general management, 17(2), 13-19. furusten, s. (2009). management consultants as improvising agents of stability. scandinavian journal of management, 25(3), 264-274. http://www.superdecisions.com/ ijahp article: tsai, lo, lin/an analytical framework with sensitivity analysis to quantify management consultancy selection international journal of the analytic hierarchy process 95 vol.5 issue 1 2013 issn 1936-6744 gallouj, c. (1997). asymmetry of information and the service relationship: selection and evaluation of the service provider. international journal of service industry management, 8(1), 42-64. golden, b. l., wasil, e. a. and levy, d. e. (1989). applications of the analytic hierarchy process: a categorized, annotated bibliography, in b. l. golden e. a. wasil p. t. harker (eds), the analysis hierarchy process: applications and studies (37-58). springer-verlag: berlin, germany. gustafson, d. and di marco, n. (1973). the management consultant selection process. academic of management proceedings, august, 166-71. hanssens, h., derudder, b., taylor, p.j., hoyler, m., ni, p., huang, j., yang, x., and witlox, f. (2011). the changing geography of globalized service provision, 2000-2008. service industries journal, 31(14), 2293-2307. kirby, d. a. and jones-evans, d. (1997). small technology-based professional consultancy services in the united kingdom. international small business journal, 17(1), 155-172. knudsen, m. p. (2007). the relative importance of interfirm relationships and knowledge transfer for new product development success. journal of product innovation management, 24(2), 117–138. kumar, v., simon, a. and kimberly, n. (2000). strategic capabilities which lead to management consulting success in australia. management decision, 38(1), 24-35. lapiedra, r., alegre, j., and chiva, r. (2011). the importance of management innovation and consultant services on erp implementation success. service industries journal, 31(12), 1907-1919. lu, iuan-yuan, su, teng-hu and huang, ing-chung (2010). consulting knowledge and organisation’s absorptive capacity: a communication chain perspective. service industries journal, 30(12), 2007-2022. mitchell, v. m. (1994). problems and risks in the purchasing of consultancy services. the service industrial journal, 14(3), 315-339. moea, (2008), business enterprises survey, ministry of economic affairs, taiwan. murry, j. and hammons, j. (1995). delphi: a versatile methodology for conducting qualitative research. the review of higher education, 18(4), 423-236. ijahp article: tsai, lo, lin/an analytical framework with sensitivity analysis to quantify management consultancy selection international journal of the analytic hierarchy process 96 vol.5 issue 1 2013 issn 1936-6744 naesens, k., gelders, l. and pintelon, l. (2007). a swift response tool for measuring the strategic fit for resource pooling: a case study. management decision, 45(3), 434-449. natti, s., halinen, a. and hanttu, n. (2006). customer knowledge transfer and key account management in professional service organizations. international journal of service industry management, 17(4), 304-319. ormerod, r. (1996). combining management consultancy and research. omega, internation journal of management science, 24(1), 1-12. richter, a. and niewiem, s. (2009). knowledge transfer across permeable boundaries: an empirical study of clients’ decision to involve management consultants. scandinavian journal of management, 25(3), 275-288. saaty, t. l. (1987). the analytic hierarchy process what it is and how it is used. mathematical modeling, 9(3/5), 161-176. saaty, t. l. (1994). how to make a decision: the analytic hierarchy process. interface, 24(6), 19-43. saaty, r. w. (2000). the fundamentals of decision making and priority theory with the analytical hierarchy process. ca: rws publications. pittsburgh. saaty, r. w. (2003). decision making in complex environments: the analytic hierarchy process (ahp) for decision making and the analytic network process for decision making with dependence and feedback. received from http://www.superdecisions.com/demos_tutorials.php3. saaty, t. l. (2008). decision making with the analytic hierarchy process. international journal of services sciences, 1(1), 83-98. áez, c., b., marco, t., g. and arribas, e., h. (2002). collaboration in randd with universities and research centres: an empirical study of spanish firms. r&d management, 32(4), 321-341. saremi, m., mousavi, s. f. and sanayei, a. (2009). tqm consultant selection in smes with topsis under fuzzy environment. expert systems with application, 36(2), 2742-2749. schartinger, d., rammer, c., fischer, m. m. and fröhlich, j. (2002). knowledge interactions between universities and industry in austria: sectoral patterns and determinants. research policy, 31(3), 303–328. http://www.superdecisions.com/demos_tutorials.php3 ijahp article: tsai, lo, lin/an analytical framework with sensitivity analysis to quantify management consultancy selection international journal of the analytic hierarchy process 97 vol.5 issue 1 2013 issn 1936-6744 schein, e. n. (1988). process consultation volume i: its role in organization development. ft press, new jersey. schein, e. n. (1990). a general philosophy of helping – process consultation. sloan management review, 31(3), 57-64. sipahi, s. and timor, m. (2010). the analytic hierarchy process and analytic network process: an overview of applications. management decision, 48(5), 775-808. sonmez, m. and moorhouse, a. (2010). purchasing professional services: which decision criteria?. management decision, 48(2), 189-206. soriano, d. r., roig, s., sanchis, j. r. and torcal, r. (2002). the role of consultants in smes: the use of services by spanish industry. international small business journal, 20(1), 95-103. wei, c. c., chien, c. f. and wang, m. j. (2005). an ahp-based approach to erp system selection. international journal production economics. 96(1), 47-62. west, d. c. (1997). purchasing professional services: the case of advertising agencies. international journal of purchasing and material management, 33(3), 2-9. wynstra, f., axelsson, b. and van der valk, w. (2006). an application-based classification to understand buyer-supplier interaction in business service. international journal of service industry management, 17(5), 474-496. yin, r. k. (2002). case study research: design and methods. sage pubns, london. ijahp article: tsai, lo, lin/an analytical framework with sensitivity analysis to quantify management consultancy selection international journal of the analytic hierarchy process 98 vol.5 issue 1 2013 issn 1936-6744 appendix i comparison matrices comparison matrix for consultancy selection of criteria goal: consultancy selection consultant’s ability perceived quality of the project the type of problem the type of solution consultant’s ability 1 5 1 1/2 perceived quality of the project 1/5 1 1/4 1/5 the type of problem 1 4 1 1/2 the type of solution 2 5 2 1 comparison matrix for consultant’s ability of sub-criteria consultant’s ability hands-on exper iences reputation commitment expertise previous working relationship organizational resources hands-on experiences 1 1/6 1/4 1/2 1/2 1/5 reputation 6 1 2 5 5 3 commitme nt 4 1/2 1 4 4 2 expertise 2 1/5 1/4 1 1 1/4 previous working relationship 2 1/5 1/4 1 1 1/4 organizatio nal resources 5 1/3 1/2 4 4 1 ijahp article: tsai, lo, lin/an analytical framework with sensitivity analysis to quantify management consultancy selection international journal of the analytic hierarchy process 99 vol.5 issue 1 2013 issn 1936-6744 comparison matrix for perceived quality of project of sub-criteria perceived quality of the project project quality the ratio of cost/benefit non-financial benefits financial benefits performance measurement project quality 1 1 6 2 1/3 the ratio of cost/benefit 1 1 7 4 1/4 non-financial benefits 1/6 1/7 1 1/3 1/8 financial benefits 1/2 1/4 3 1 1/4 performance measurement 3 4 8 4 1 comparison matrix for the type of problem of sub-criteria the type of problem urgency complexity clearness urgency 1 1/2 1 complexity 2 1 3 clearness 1 1/3 1 comparison matrix for the type of solution of sub-criteria the type of solution best practice information about problem feasible solution problem modeling best practice 1 3 1/2 2 information about problem 1/3 1 1/8 1 feasible solution 2 8 1 5 problem modeling 1/2 1 1/5 1 ijahp article: tsai, lo, lin/an analytical framework with sensitivity analysis to quantify management consultancy selection international journal of the analytic hierarchy process 100 vol.5 issue 1 2013 issn 1936-6744 comparison matrix for consultant’s ability: hands-on experiences of alternatives consultant’s ability: hands-on experiences the academic consultant the consulting company the academic consultant 1 1/7 the consulting company 7 1 comparison matrix for consultant’s ability: reputation of alternatives consultant’s ability: reputation the academic consultant the consulting company the academic consultant 1 1/6 the consulting company 6 1 comparison matrix for consultant’s ability: commitment of alternatives consultant’s ability: commitment the academic consultant the consulting company the academic consultant 1 1/3 the consulting company 3 1 comparison matrix for consultant’s ability: expertise of alternatives consultant’s ability: expertise the academic consultant the consulting company the academic consultant 1 1/2 the consulting company 2 1 ijahp article: tsai, lo, lin/an analytical framework with sensitivity analysis to quantify management consultancy selection international journal of the analytic hierarchy process 101 vol.5 issue 1 2013 issn 1936-6744 comparison matrix for consultant’s ability: previous working relationship of alternatives consultant’s ability: previous working relationship the academic consultant the consulting company the academic consultant 1 1 the consulting company 1 1 comparison matrix for consultant’s ability: organizational resources of alternatives consultant’s ability: organizational resources the academic consultant the consulting company the academic consultant 1 1/5 the consulting company 5 1 comparison matrix for perceived quality of the project: project quality of alternatives perceived quality of the project: project quality the academic consultant the consulting company the academic consultant 1 1/2 the consulting company 2 1 comparison matrix for perceived quality of the project: the ratio of cost/benefit of alternatives perceived quality of the project: the ratio of cost/benefit the academic consultant the consulting company the academic consultant 1 1/5 the consulting company 5 1 ijahp article: tsai, lo, lin/an analytical framework with sensitivity analysis to quantify management consultancy selection international journal of the analytic hierarchy process 102 vol.5 issue 1 2013 issn 1936-6744 comparison matrix for perceived quality of the project: non-financial benefits of alternatives perceived quality of the project: non-financial benefits the academic consultant the consulting company the academic consultant 1 4 the consulting company 1/4 1 comparison matrix for perceived quality of the project: financial benefits of alternatives perceived quality of the project: financial benefits the academic consultant the consulting company the academic consultant 1 1/7 the consulting company 7 1 comparison matrix for perceived quality of the project: performance measurement of alternatives perceived quality of the project: performance measurement the academic consultant the consulting company the academic consultant 1 1/5 the consulting company 5 1 comparison matrix for the type of problem: urgency of alternatives the type of problem: urgency the academic consultant the consulting company the academic consultant 1 1/6 the consulting company 6 1 ijahp article: tsai, lo, lin/an analytical framework with sensitivity analysis to quantify management consultancy selection international journal of the analytic hierarchy process 103 vol.5 issue 1 2013 issn 1936-6744 comparison matrix for the type of problem: complexity of alternatives the type of problem: complexity the academic consultant the consulting company the academic consultant 1 2 the consulting company 1/2 1 comparison matrix for the type of problem: clearness of alternatives the type of problem: clearness the academic consultant the consulting company the academic consultant 1 1 the consulting company 1 1 comparison matrix for the type of solution: best practice of alternatives the type of solution: best practice the academic consultant the consulting company the academic consultant 1 1/6 the consulting company 6 1 comparison matrix for the type of solution: information about problem of alternatives the type of solution: information about problem the academic consultant the consulting company the academic consultant 1 3 the consulting company 1/3 1 ijahp article: tsai, lo, lin/an analytical framework with sensitivity analysis to quantify management consultancy selection international journal of the analytic hierarchy process 104 vol.5 issue 1 2013 issn 1936-6744 comparison matrix for the type of solution: feasible solution of alternatives the type of solution: feasible solution the academic consultant the consulting company the academic consultant 1 1/3 the consulting company 3 1 comparison matrix for the type of solution: problem modeling of alternatives the type of solution: problem modeling the academic consultant the consulting company the academic consultant 1 3 the consulting company 1/3 1 an analytical framework with sensitivity analysis to quantify management consultancy selection 1. introduction 2. literature review and research focus 3. methodology 4. analysis and discussion 1. 2. 3. 4. 4.1. the priority of criteria 4.2. selecting the appropriate consultancy provider 5. improvement analysis 6. conclusions ijahp news: feretti/mcdm 2015 highlights international journal of the analytic hierarchy process 613 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.360 mcdm 2015 highlights valentina ferretti, department of management, london school of economics and political science the last several months have been particularly interesting and scientifically inspiring for the ahp/anp community with the mcdm 2015 conference in hamburg, the euro conference in glasgow, and the informs annual meeting in philadelphia! we would like to share some of the key moments of the 23 rd international conference on multiple criteria decision making that took place this past summer in the fascinating hanseatic city of hamburg. the conference was held from august 2-7, 2015 at the helmut-schmidt-university, university of the federal armed forces in hamburg (germany) and was a great success with record-breaking attendance! the mcdm 2015 welcomed more participants than any previous mcdm congress with 361 registered participants (of whom 121 were women and 114 were students) hailing from 49 countries. this is exciting evidence that the field is growing, and supports the topic of this year’s conference, i.e. “bridging disciplines” by demonstrating that mcdm is indeed an inter-disciplinarian field. the figure below shows the number of participants by country, with the most participants coming from turkey (62 participants), germany (39), brazil (22), usa (21) and uk (20). the term “others” corresponds to the countries with one or two participants (such as azerbaijan, bahrain, greece, lithuania, australia, latvia, republic of korea, serbia, new zealand, hong kong, ecuador, russia and qatar). figure 1. participants in the mcdm 2015 by country this wide representation of disciplines and countries provided a scientific program that was particularly rich and featured 306 talks! turkey 17% germany 11% brazil 6% usa 6% uk 6%spain 4% italy 4% poland 3% taiwan 3% finland 3% algeria 3% iran 2% netherlands 2% china 2% belgium 2% canada 2% japan 2% colombia 2% india 2% france 1% switzerland 1% portugal 1% vietnam 1% denmark 1% slovenia 1% hungary 1% indonesia 1% czech republic 1% romania 1% others 8% ijahp news: feretti/mcdm 2015 highlights international journal of the analytic hierarchy process 614 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.360 below is a record of some memories from the ahp/anp community at the conference. figure 2. monica garcia melon and birsen karpak at mcdm 2015. monica garcia melon chaired the session titled “ahp / anp for behavioral issues in group decision making”, while birsen karpak presented "big data, supply chain management, ahp / anp state of the art and future directions”. in particular, monica’s session hosted the following talks: “simultaneous interval ahp in a multiple criteria group decision making” by tomoe entani of the university of hyogo, japan, “applying gdm in human resources” by katherina kashi, technical university of ostrava, czech republic, and “influence analysis in gdm. a dematel-sna-ahp comparison in environmental management” by mónica garcía-melón, universitat politecnica de valencia, spain. ijahp news: feretti/mcdm 2015 highlights international journal of the analytic hierarchy process 615 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.360 figure 3. some conference participants from the ahp/anp community. from left to right: ozay ozaydin, özgür kabak, şule önsel ekici, mónica garcíamelón, ilker topcu, gözde kadıoglu, luiz flavio autran monteiro gomes with a student and birsen karpak. ilker topcu chaired a session on integrated mcdm applications, hosting the following talks: “a fuzzy-ahp based approach for a multi-objective multi-level closed-loop supply chain model” by turan paksoy, selçuk university, turkey, “an ahp model for determining appropriate public locations for automated external defibrillator deployment: a case study in the city of valencia” by ilker topcu, istanbul technical university, turkey, “a new multi objective linear programming model for optimizing a closed loop supply chain network using fuzzy ahp” by abdullah yildizbasi, yildirim beyazit university, turkey. gozde kadioglu (an undergraduate student from the technical university of istanbul) presented "the selection of the most effective type of fuel for electricity generation in turkey" in the session “integrated mcdm application -2,” which was chaired by ilker topcu. the following lists, in a nutshell, the other sessions on ahp/anp that were organized at the mcdm 2015 conference. alexander mechitov organized a session on “ahp / anp advances in theory 1” including contributions on “optimizing ahp incomplete comparisons using doptimal design” by ioannis karamitsos, university of sunderland, uk, “the reconstruction of a pairwise comparison matrix from a base” by jacek szybowki, ijahp news: feretti/mcdm 2015 highlights international journal of the analytic hierarchy process 616 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.360 agh university of science and technology, poland, “reaching and checking the weak consistency” by bice cavallo, university of naples, federico ii, italy, and “ordinal inconsistencies in pairwise comparisons: problems and solutions” by helen moshkovich, university of montevallo, usa. petr fiala organized a session on “ahp / anp industrial applications i” including contributions on “multiple criteria methodology for supplier evaluation and purchasing management” by concepción maroto, universidad politécnica de valencia, spain, “in search of intelligent algorithm for securing video streaming over cloud computing infrastructure” by irfan syamsuddin, state polytechnic of ujung pandang, indonesia, and “multicriteria project portfolio management under risk” by petr. fiala, university of economics, prague, czech republic. ririn diar astanti organized a session on “ahp / anp for sustainable management” including contributions on “ranking the barriers to the development of the renewable energy technologies in colombia” by tomás gómez-navarro, universidad politécnica de valencia, spain, “anp and sna applied to the assessment of the sustainability of the cotopaxi natural park management” by tomás gómez-navarro, universidad politécnica de valencia, spain, “methodological framework based on mcdm applicable to develop hydropower decision aid. a case of nepal” by rana pratap singh of the university of natural resources and life sciences, austria (co-author hans peter nachtnebel), and “ethnic entrepreneurship model using analytic network process: a case study among the javanese, chinese and minangkabau in yogyakarta, indonesia” by mahestu n. krishanti, universitas atma jaya yogyakarta, indonesia). fikri dweri organized a session on “anp / ahp industrial applications ii” including contributions on “fuzzy ahp-fuzzy promethee approach in evaluation of e-service quality: case of airline web sites” by pelin celik, karadeniz university, turkey, “an analytical network process approach for measuring influence of stakeholders in a project” by mónica garcía-melón, universitat politecnica de valencia, spain, “determining the factors of reputation management by anp method” by ihsan yüksel, kirikkale university, turkey, and “third party warehouse location selection in a lube oil company: an ahp approach” by fikri dwei, university of sharjah, united arab emirates. salvatore greco organized a session on “ahp / anp advances in theory ii” including contributions on “an optimization approach for the eigenvector method by janos fülóp, institute for computer science and control, hungarian academy of sciences, hungary, “analysis and comparison of ahp by the implementation of fuzzy logic and monte carlo simulation” by laura a. gutierres-bucheli, universidad de los andes, colombia, “heuristic rating estimation, another approach to the ranking procedure in ahp” by konrad kulakowski, agh ust, poland, and “spanning tree pairwise comparison based prioritization for multiple criteria decision aiding” by sajid siraj, consats, institute of information technology, pakistan. ijahp news: feretti/mcdm 2015 highlights international journal of the analytic hierarchy process 617 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.360 metin dagdeviren organized a session on “theory and applications of the ahp / anp” including contributions on “site selection for wind energy plant by using anp” by erdem aksakal, gazi university, turkey, “a compatible study of priorities vectors for reverse logistics indicators: a footwear industry case developed in brazil” by jose l. silveira, regional university of cariri, brasil, “pythia: development of an oracle for automated treatment plan selection in radiation therapy” by sebastian breedveld, erasmus nc, netherlands, and “a new decision making model for bank branch location selection” by metin dagdeviren, gazi university, turkey), pekka korhonen organized a session on “theory and applications of the ahp / anp” including contributions on “efficient weight vectors form pairwise comparisons” by sándor bozóki, institute for computer science and control, hungary, “visualization of inconsistency of a pairwise comparison table in the ahp” by yoichi iida, tokio university of science, japan, “axiomatic properties of inconsistency indexes for pairwise comparisons” by matteo brunelli, systems analysis laboratory, aalto university, finland, and “setting “condition of order preservation requirements for the priority vector estimate in ahp is not justified” by pekka korhonen, aalto university, finland. yannick cornet organized a session on “ahp / anp for sustainability” including contributions on “fair centralized allocation of resource limitations among processes” by sebastian ihrig, tu münchen, germany, “success factors for integration of sustainable practices at high performance building processes through ahp-based mcdm” by francisca molina moreno, universidad politécnica de valencia, spain, and “high speed rail: a mandate from future generations? applying strong sustainability theory to the uks hs2 assessment using multiplicative ahp decision support model” by yannick cornet, technical university of denmark, denmark. other interesting contributions on ahp/anp included in other sessions were: “a socioeconomic problem regrading poplar plantation and a problem solving model by ahp” by mojid azizi, university of tehran, iran, “operational performance evaluation of airlines operating in india using electre and ahp” by omkarprasad vaisya, indian institute of management lucknow, india, “comparison of ahp/anp decision making methodologies in a case study of renewable energy in the dominican republic” by guido guerrero-liquet, universidad politécnica de cartagena, spain, “infrastructure optioneering: an ahp approach” by duanne gilmore, arup, usa, and “assessing and ranking halmas parks in malaysia:an application of importance performance analysis and ahp” by rafikul islam. ijahp news: feretti/mcdm 2015 highlights international journal of the analytic hierarchy process 618 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.360 figure 4. the future organizers of mcdm 2017 and mcdm 2019: ilker topcu and sarah ben amor during mcdm 2015, the executive committee decided on the venues for the next two conferences. the 24th international conference on mcdm will be held in ottawa, canada from july 9-14, 2017 and will be organized by sarah ben amor and her colleagues. the 25th conference will be organized by i̇lker topçu and his colleagues in i̇stanbul in 2019, with the exact dates remaining yet to be decided. this conference provided a wonderful opportunity to view the state of the art approaches, theories and applications in multiple criteria decision making. it was also an excellent opportunity to meet old friends and make new ones and exchange ideas, receive feedback and get inspired. looking forward to meeting you in ottawa and a special thanks to monica garcia melon for her help with this news piece! ijahparticle: vargas/how to write a contract with the ahp international journal of the analytic hierarchy process 274 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.490 how to write a contract with the ahp luis g. vargas professor of business analytics and operations the joseph m. katz graduate school of business university of pittsburgh lgvargas@pitt.edu abstract in this paper, we show how the analytic hierarchy process could be used to develop a legal contract in the process of a negotiation. we illustrate the process with a well-known case used routinely in negotiation courses to illustrate that the ahp is particularly well suited for this type of application where most of the dimensions and criteria are intangibles, and the scales used to measure the gains and costs of parties involved in the negotiation do not always exist. keywords: negotiation; gain and loss ratios; value claim; value creation 1. introduction the dictionary definition of “contract” is “a binding agreement between two or more persons or parties” or “a document describing the terms of a contract.” this implies that a contract has multiple dimensions and the parties must agree on each of the dimensions. for example, in the case of a recruiter trying to hire a candidate for a position in a company, the dimensions could be the signing bonus, salary, job assignment, company car, starting date, number of vacation days, percentage of moving expenses covered, the type of insurance coverage offered, and so on. each dimension has a different impact on each of the parties. there are two types of outcome at work when two parties negotiate: value claim, and value creation (see figure 1). figure 1. value claims, value creation and the pareto frontier ijahparticle: vargas/how to write a contract with the ahp international journal of the analytic hierarchy process 275 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.490 value claim occurs when one party can capture value from the other party during the negotiation process. this is most prevalent among those dimensions of the negotiation that are distributive (i.e., what one party gains, the other party experiences as a comparable loss). however, it can also manifest itself for integrative elements (i.e., when multiple factors are negotiated – some of which are more important to one of the parties, and some of which are more important to the other party). however, for both integrative and compatible dimensions (i.e., factors where the same element is perceived as a gain for both parties), there are also opportunities for exchange that leads to value creation. thus, value creation takes place when both parties are made better off during the negotiation. when value creation occurs, the parties move closer towards the pareto frontier – the point at which neither party can be made better off without the counterparty being made worse off. 2. a simple example we mentioned above that a contract has multiple dimensions and the parties must agree on each of the dimensions for the contract to be accepted by both parties. thus, for a negotiation to arrive at a mutually agreed contract it needs to consider the gains and losses of the parties in each of the dimensions. for example, a recruiter is negotiating with a prospective employee for a position. they need to agree on the conditions of employment. the negotiation involves agreement on several dimensions. each dimension can be considered a benefit or a cost. table 1 shows an example of dimensions of a negotiation and their type. table 1 dimensions and their type in addition, within each type, the dimensions are not equally important. table 2 shows the importance of the dimensions from both, the recruiter’s and the employee’s perspective. dimensions type signing bonus (sb) benefit salary (s) cost job assignment (ja) cost company car (cc) benefit starting date (sd) benefit vacation days (vd) benefit moving expenses reimb (mer) benefit insurance coverage (ic) benefit ijahparticle: vargas/how to write a contract with the ahp international journal of the analytic hierarchy process 276 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.490 table 2 recruiter/employee priorities for benefits and costs in many real-life contract negotiations, neither the dimensions of the contract nor the intensity scales may be known. to make tradeoffs we need to identify the dimensions and the intensity scales. consider the dimensions of the recruiter-candidate example with scales as given in table 3. benefits recruiter employee signing bonus (sb) 0.270 0.270 company car (cc) 0.081 0.081 starting date (sd) 0.108 0.270 vacation days (vd) 0.270 0.108 moving expenses (mer) 0.054 0.216 insurance coverage (ic) 0.216 0.054 costs recruiter employee salary (s) 0.75 0.75 job assignment (ja) 0.25 0.25 priorities priorities ijahparticle: vargas/how to write a contract with the ahp international journal of the analytic hierarchy process 277 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.490 table 3 intensities and the benefits/costs accrued by the recruiter and the candidate these scales are not usually known and need to be constructed using relative measurement. for the moment, consider the intensity scales in table 3 expressed in ideal terms (i.e., the elements are divided by the largest value) in table 4. in this example, the scale values are all equispaced, i.e., they form a linear scale. however, in practice these values would be obtained through prioritization and they do not need to be linear.the negotiation process consists in finding out what value each dimension should take for the recruiter and the candidate so that the total amount they get (benefit/cost ratio) is maximized and satisfies the constraint that neither party gets more than the other, i.e., the contract is fair and equitable (fisher & ury, 1981). intensity recruiter candidate signing bonus (sb) 10% 0 4000 8% 1000 3000 6% 2000 2000 4% 3000 1000 2% 4000 0 salary (s) 60,000.00$ -6000 0 58,000.00$ -4500 -1500 56,000.00$ -3000 -3000 54,000.00$ -1500 -4500 52,000.00$ 0 -6000 job assignment (ja) division a 0 0 division b -600 -600 division c -1200 -1200 division d -1800 -1800 division e -2400 -2400 company car (cc) lux ex2 1200 1200 mod 250 900 900 rand xtr 600 600 de pas 450 300 300 palo lsr 0 0 starting date (sd) 1-jun 1600 0 15-jun 1200 1000 1-jul 800 2000 15-jul 400 3000 1-aug 0 4000 vacation days (vd) 30 days 0 1600 25 days 1000 1200 20 days 2000 800 15 days 3000 400 10 days 4000 0 moving expenses 100% 0 3200 reimbursement (mer) 90% 200 2400 80% 400 1600 70% 600 800 60% 800 0 insurance coverage (ic) allen insurance 0 800 abc insurance 800 600 good health insurance 1600 400 best insurance co. 2400 200 insure alba 3200 0 ijahparticle: vargas/how to write a contract with the ahp international journal of the analytic hierarchy process 278 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.490 table 4 priorities of dimensions and relative scales for the recruiter – candidate case recruiter priorities candidate priorities benefits' priorities costs' priorities benefits' priorities costs' priorities recruiter candidate 0.2702701 0.27027 signing bonus scale ideal scale scale ideal scale 10% 0.01 0.00 4,000 1.00 8% 1,000 0.25 3,000 0.75 6% 2,000 0.50 2,000 0.50 4% 3,000 0.75 1000 0.25 2% 4,000 1.00 0.01 0.00 0.714285918 0.714285918 salary 10000.01 10000.01 $60,000 -6,000 1 0.01 0.00 $58,000 -4,500 0.75 -1,500 0.25 $56,000 -3,000 0.50 -3,000 0.50 $54,000 -1,500 0.25 -4,500 0.75 $52,000 0.01 0.00 -6,000 1.00 0.285714082 0.285714082 job assignment -14999.99 -14999.99 division a 0.01 0.00 0.01 0.00 division b -600 0.25 -600 0.25 division c -1,200 0.50 -1,200 0.50 division d -1,800 0.75 -1,800 0.75 division e -2,400 1.00 -2,400 1.00 0.0810812 0.081081 company car -5999.99 -5999.99 lux ex2 1200 1.00 1200 1.00 mod 250 900 0.75 900 0.75 rand xtr 600 0.50 600 0.50 de pas 450 300 0.25 300 0.25 palo lsr 0.01 0.00 0.01 0.00 0.1081082 0.27027 starting date 3000.01 3000.01 1-jun 1,600 1.00 0.01 0.00 15-jun 1,200 0.75 1,000 0.25 1-jul 800 0.50 2,000 0.50 15-jul 400 0.25 3,000 0.75 1-aug 0.01 0.00 4,000 1.00 0.2702701 0.108108 vacation days 4000.01 10000.01 30 days 0.01 0.00 1,600 1.00 25 days 1,000 0.25 1,200 0.75 20 days 2,000 0.50 800 0.50 15 days 3,000 0.75 400 0.25 10 days 4,000 1.00 0.01 0.00 moving expenses 10000.01 4000.01 0.0540542 0.216216 reimbursement 100% 0.01 0.00 3,200 1.00 90% 200 0.25 2,400 0.75 80% 400 0.50 1,600 0.50 70% 600 0.75 800 0.25 60% 800 1.00 0.01 0.00 0.2162161 0.054054 insurance coverage 2000.01 8000.01 allen insurance 0.01 0.00 800 1.00 abc insurance 800 0.25 600 0.75 good health insurance 1,600 0.50 400 0.50 best insurance co. 2,400 0.75 200 0.25 insure alba 3,200 1.00 0.01 0.00 8000.01 2000.01 ijahparticle: vargas/how to write a contract with the ahp international journal of the analytic hierarchy process 279 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.490 3. the trading model to find the solution of this problem we model it with integer programming. a solution is represented by an 8-by-5 matrix  ijx of 0’s and 1’s. each row corresponds to a dimension and each column corresponds to an intensity of the scale corresponding to that dimension (see table 5). table 5 a solution with the benefit/cost ratios the benefits and costs are obtained from table 4. for example, the benefit for the recruiter of selecting a 6% signing bonus (0.135135) is obtained by multiplying the weight of signing bonus (0.27027) by the scale intensity 3 (0.50). thus, 1 ij x  if the i th dimensions takes the j th intensity value. let ( ) r c ij ij b b and ( ) r c ij ij c c the benefit and cost corresponding to the j th intensity of the i th dimension for the recruiter (candidate). the benefits/costs ratios of the recruiter and the candidate are given by benefits ( ) costs r r i ij ij i j r r r i ij ij i j w x b r x v x c       and benefits ( ) costs c c i ij ij i j c c c i ij ij i j w x b r x v x c       , respectively. the objective is to find a solution *x such that the parties gain as much as possible, ( *) ( *) {min{ ( ), ( )}} s r c a b x x r x r x max r x r x    , where s x is the solution space defined as the set of matrices ( ) ij x that satisfy the conditions 5 1 1 ij j x   , for all i, 0,1ijx  , for all i and j, and the two parties gain the same, i.e., their ratios are equal. a solution intensities 1 2 3 4 5 benefits costs benefits costs sb 0 0 1 0 0 0.135135 0 0.135135 0 s 0 1 0 0 0 0 0.535714 0 0.178571 ja 0 1 0 0 0 0 0.071429 0 0.071429 cc 1 0 0 0 0 0.081081 0 0.081081 0 sd 0 0 1 0 0 0.054054 0 0.135135 0 vd 0 1 0 0 0 0.067568 0 0.081081 0 mer 0 0 1 0 0 0.027027 0 0.108108 0 ic 0 0 1 0 0 0.108108 0 0.027027 0 b/c ratio 0.7790 2.2703 recruiter candidate ijahparticle: vargas/how to write a contract with the ahp international journal of the analytic hierarchy process 280 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.490 a given solution has benefits/costs ratios that are different for the parties. for example, in table 5 we give a solution. in this solution, the recruiter has a lower benefits/costs ratio than the candidate, so the recruiter will try to change to another solution where he will get a greater benefits/costs ratio. table 6 shows the solution in matrix form. table 6 optimal solution translated into the original scale values of the dimensions we have table 7. note that now both the recruiter and the candidate gain the same. table 7 terms of the contract obviously, the scales within each dimension do not have to be linear. for example, if the recruiter and the candidate have relative intensities as given in table 8, the solution (table 9) would not be the same as the one in table 7. the solutions in table 9 are within 3.125% of each other. no other closer solutions exist. 4. general contract model in many contract negotiations, the parties do not always act in good faith or share information with the other party. in this case, one should also consider the perceptions of the parties about the benefits and costs of the tradeoffs. for example, in a merger transaction, the buyer (a) and the seller (b) may not always agree as to the terms of the merger, and hence the transaction may fail. the steps to make tradeoffs in this more general situation are as follows: optimal solution intensities 1 2 3 4 5 benefits costs benefits costs sb 0 0 1 0 0 0.135135 0 0.135135 0 s 0 1 0 0 0 0.357143 0 0.357143 ja 1 0 0 0 0 0 0 0 0 cc 1 0 0 0 0 0.081081 0 0.081081 0 sd 0 0 0 0 1 0 0 0.27027 0 vd 0 0 0 0 1 0.27027 0 0 0 mer 1 0 0 0 0 0 0 0.216216 0 ic 0 0 0 0 1 0.216216 0 0 0 b/c ratio 1.9676 1.9676 recruiter candidate sb s ja cc sd vd mer ic total 6% 56,000.00$ division a lux ex2 1-aug 10 days 100% insure alba points recruiter 2000 -3000 0 1200 0 4000 0 3200 7400 candidate 2000 -3000 0 1200 4000 0 3200 0 7400 ijahparticle: vargas/how to write a contract with the ahp international journal of the analytic hierarchy process 281 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.490 1. identify the dimensions of the problem 2. identify the tradeoffs of each party within the dimensions 3. identify the benefits accrued by a party from the other party’s tradeoffs 4. identify the costs incurred by a party from its own tradeoffs 5. identify the perceived benefits that the other party received from your tradeoffs 6. identify the perceived costs incurred by the other party from their tradeoffs 7. find out what tradeoff each party must make to maximize the total minimum gain they obtain, ensuring that the gains of a party are as close as possible to the other party gains. this is what makes the final contract fair, equitable and balanced. table 8 intensities with non-linear relative scales relative scales dimensions recruiter candidate recruiter candidate signing bonus 10% 0.01 4,000 1e-06 1 8% 1,000 3,000 0.1 0.75 6% 2,000 2,000 0.5 0.5 4% 3,000 1000 0.9 0.1 2% 4,000 0.01 1 1e-06 salary 10000.01 10000.01 $60,000 -6,000 0.01 1 1e-06 $58,000 -4,500 -1,500 0.75 0.1 $56,000 -3,000 -3,000 0.5 0.5 $54,000 -1,500 -4,500 0.1 0.9 $52,000 0.01 -6,000 1e-06 1 job assignment -14999.99 -14999.99 division a 0.01 0.01 1e-06 1e-06 division b -600 -600 0.1 0.1 division c -1,200 -1,200 0.5 0.5 division d -1,800 -1,800 0.9 0.9 division e -2,400 -2,400 1 1 company car -5999.99 -5999.99 lux ex2 1200 1200 1 1 mod 250 900 900 0.75 0.75 rand xtr 600 600 0.5 0.5 de pas 450 300 300 0.1 0.1 palo lsr 0.01 0.01 1e-06 1e-06 starting date 3000.01 3000.01 1-jun 1,600 0.01 1 1e-06 15-jun 1,200 1,000 0.75 0.1 1-jul 800 2,000 0.5 0.5 15-jul 400 3,000 0.1 0.9 1-aug 0.01 4,000 1e-06 1 vacation days 4000.01 10000.01 30 days 0 1,600 0 1 25 days 1,000 1,200 0.1 0.75 20 days 2,000 800 0.5 0.5 15 days 3,000 400 0.9 0.1 10 days 4,000 0.01 1 1e-06 moving expenses 10000 4000.01 reimbursement 100% 0.01 3,200 1e-06 1 90% 200 2,400 0.1 0.75 80% 400 1,600 0.5 0.5 70% 600 800 0.9 0.1 60% 800 0.01 1 1e-06 insurance coverage 2000.01 8000.01 allen insurance 0.01 800 1e-06 1 abc insurance 800 600 0.1 0.75 good health insurance 1,600 400 0.5 0.5 best insurance co. 2,400 200 0.9 0.1 insure alba 3,200 0.01 1 1e-06 8000.01 2000.01 ijahparticle: vargas/how to write a contract with the ahp international journal of the analytic hierarchy process 282 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.490 table 9 terms of the contract for the non-linear intensity case the mathematical model that helps identify the proper contract is given below. let ( ) k x x the scale of the kth dimension. the parties will negotiate on the value of that scale according to their preferences. the realized value of the scale is determined by the benefit, the cost, the perceived benefits and the perceived cost that the value has for each party. let ( ) i k b x be the benefits accrued by party i from the other party tradeoffs in dimension k. let ( ) i k c x be the costs incurred by party i from its own tradeoffs in dimension k. let ( ) i k pb x be the benefits party i perceives the other party receives from its tradeoffs in dimension k, and let ( ) i k pc x be the costs the other party perceives that party i incurs from its tradeoffs in dimension k. thus, for a given dimension k, the gain of party i is given by the benefits it accrues from the tradeoffs of the other party in that dimension times the costs it perceives the other party incurs in that dimension, i.e., ( ) ( ) i k i k b x pc x . similarly, the loss in each dimension k is given by ( ) ( ) i k i k c x pb x . thus, the gain to loss ratio for a party for a given dimension k is given by: ( ) ( ) ( ) ( ) i k i k i k i k b x pc x c x pb x and the total gain-to-loss ratio for a party is given by ( ) ( ) ( ) ( ) i k i k i all k i k i k b x pc x r c x pb x   . this formulation was suggested by saaty (1988) to handle retributive conflicts. let ( ) k x s be a binary variable, where ( ) 1 k x s  if the parties agree on selecting the intensity s of the kth dimension as the best decision for both. the problem now consists in finding values of s for each dimension that maximizes the smallest gain-to-loss ratio of both parties, i.e., [ ( )] [ ( )][ ( )] [ ( )] ( ) , ( ) [ ( )] [ ( )] [ ( )] [ ( )] j k j ki k i k i j s all k i k i k j k j k b x s pc x sb x s pc x s max min r s r s c x s pb x s c x s pb x s                sb s ja cc sd vd mer ic total 10% 56,000.00$ division a lux ex2 1-jul 10 days 90% insure alba points recruiter 0 -3000 0 1200 800 4000 200 3200 6400 candidate 4000 -3000 0 1200 2000 0 2400 0 6600 ijahparticle: vargas/how to write a contract with the ahp international journal of the analytic hierarchy process 283 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.490 subject to ( ) 1 k s x s  and ( ) 1 ( ) i j r s r s   , where  is the tolerance that measures how far the two parties are in terms of their total gain-to-loss ratio. 5. conclusions in this paper, we assume that a multidimensional contract between two parties is an agreement among the parties about the values of the dimensions that gives each party a fair and equitable gain. the values of the dimensions can be estimated through relative measurement when no scales exist. this approach was first used by saaty (1988) to address the conflict in south africa. the main difference between the approach in this paper and that used in the analysis of the palestinian-israeli conflict is that in the later the tradeoffs were analyzed in pairs (saaty & zoffer, 2011; 2013). in the case of a contract, the scales in which the dimensions are measured makes it impossible to analyze all possible pairs of tradeoffs. for example, in the simple case given above, the number of tradeoffs is 8 4 4, 294, 962 7, 296   . here we used a non-linear integer optimization formulation to derive the solutions shown using a genetic algorithm. ijahparticle: vargas/how to write a contract with the ahp international journal of the analytic hierarchy process 284 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.490 references fisher, r. and w. ury (1981). getting to yes: negotiating agreement without giving in. new york: penguin books. doi: 10.2307/40202101 saaty, t. l. (1988). the negotiation and resolution of the confilct in south africa: the ahp. orion, 4(1), 3-25. doi: http://dx.doi.org/10.5784/4-1-488. saaty, t. l. and h. j. zoffer (2011). negotiating the israeli palestinian controversy from a new perspective. international journal of information technology and decision making, 10(1), 5-64. doi: https://doi.org/10.1142/s021962201100421x saaty, t. l. and h. j. zoffer (2013). principles for implementing a potential solution to the middle east conflict. notices of the american mathematical society ,60(10), 13001322. doi: http://dx.doi.org/10.1090/noti1053 https://doi.org/10.1142/s021962201100421x ijahp: mu/making decisions in the new normal international journal of the analytic hierarchy process 175 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.808 making decisions in the new normal dear ijahp reader: welcome to our second ijahp issue in 2020. this issue is being published at a very special moment for our community and the world. the challenges imposed by the covid-19 pandemic highlight the need, more than ever, for making rational decisions in the face of multiple conflicting objectives (e.g., legal, health and economic outcomes). multi-criteria decision-making has never been more important than in the current crisis. the ahp/anp methodology is particularly suitable for the challenge of incorporating health issues within a structured decision-making framework, so we should expect more applications in this area during the following months. we are currently operating in the so-called “new normal.” we will certainly not be back to study or work in the same way as before the pandemic. the essay in this issue titled “what if classroom instruction becomes a thing of the past?” reflects this idea. important changes have also taken place in academic and professional meetings. for the time being, the emphasis will be on virtual gatherings. isahp2020 will take place december 3-6, and will have a virtual format. this will allow many professionals to participate who, due to time or financial constraints would not have been able to participate in a traditional physical gathering. for this reason, the theme for the conference this year is “ahp/anp: the new generation.” as in previous years, ijahp will play a major role in the organization of this event and will offer participants the opportunity for their best work to be reviewed for fast publication. hope to see you there! enrique mu, phd ijahp, editor-in-chief https://www.isahp.org/ ijahp article: chung, yoon/ analysis of elements to improve the walking environment in korean traditional markets using analytic hierarchy process international journal of the analytic hierarchy process 479 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.410 analysis of elements to improve the walking environment in korean traditonal markets using analytic hierarchy process kumho chung chonnam national university south korea kumho@chonnam.ac.kr min-suk yoon chonnam national university south korea msyoon@chonnam.ac.kr abstract in this paper, we evaluate elements to improve pedestrian mobility in order to revitalize traditional markets in south korea. the objective of this study is to analyze the priority of elements for pedestrians in traditional markets using the analytic hierarchy process (ahp). this paper used a two level hierarchy structure to achieve the objective. the first level was composed of four elements and the second level was composed of 28 subelements. the questionnaire used pairwise comparison. the data for analysis was collected through a survey of university students. the ahp process produced the results of this study as follows. the elements in the first level of element priority are interest, convenience, comfort and safety. the sub-elements in the second level of element priority are pitchmen, street vendors, shopping, price negotiation, paving, landscape, street lights, benches, people watching, trash bins, street vegetation, obstacles, information signs, illegal parking, cul-de-sacs, noise, sidewalk width, crowdedness, lack of sidewalk, sidewalk slope, activities, public transportation, traffic, pedestrians, stalls, billboards, vehicle speed, parking lots, speed bumps, and bicycle roads. keywords: ahp; traditional market; revitalization; pedestrian environment 1. introduction 1.1 traditional markets in korea a market, or marketplace, is a location where people regularly gather for the purchase and sale of provisions, livestock, and other goods (oxford, 2016). the government agencies of korea define a market as “a specific facility or place to carry out commodity trading between buyers and sellers”. the same definition is used in “the market law” that was enacted in 1961. the term ‘traditional market’ (tm) is widely used in south korea, but it is difficult to clearly define. ‘the special act to improve tms and shopping districts’ defined it as “a building or place over a certain size that supports both buyers and sellers, is a permanent or regular market that was built before 1980, when modern distributions facilities were opened, and has a need for redevelopment or modernization of aging facilities”. so, the concept of a tm includes the implication that ijahp article: chung, yoon/ analysis of elements to improve the walking environment in korean traditional markets using analytic hierarchy process international journal of the analytic hierarchy process 480 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.410 the market needs redevelopment of aging buildings and modernization to improve slumping retail performance. the act decided on the time criterion, 1980, considering normal re-development time and construction quality of buildings in that era. in other words, the tm is defined by convenience for shoppers, market traditions and the need for re-development and modernization of the buildings and facilities. therefore, the tm concepts are as follows: 1. the tm is formed by a social consensus and is comprised of people, physical elements, time and space. 2. the tm is a place where commodity trading between buyers and sellers is carried out. 3. a tm is a place where old infrastructure needs repairs or maintenance and it is necessary to promote business improvement and modernization of commerce to improve weak distribution function. the functions of the tm in modern society are job creation, direct trading, the formation of local community, a venue for local culture and a center of the local economy (kim & park, 2006). the tm benefits the welfare of citizens, provides a place for social exchange, and serves as the local economic base. since 1960, many regular tms were built due to increasing demand for goods, along with a population explosion in korea. the consumer's demand for minimum movement to make purchases and the supplier's interest in dominating the market led to a boom of tms. however, the tm started declining by the late 1970s due to stores opening outside the tms, changes in consumer purchasing behavior, and internally aging buildings and antiquated management of the market. the tm is the backbone of the city's core business and is located near local city centers. therefore, the main focus of the urban development in korea is improving tms. the korean government made a law to revitalize tms in 1995 and improved it further in 2002, 2004, 2006 and 2010. the government also supported the effort to improve the facilities such as adding arcades, toilets, parking lots and trader training (small and medium business admin. in korea, 2006). these laws and policies focused on facility improvements to revitalize tms. the government has made many laws to focus on improving the market environment to revitalize tms without verifying the effects. there are many studies about acts, management and case studies, but seldom studies of the effect of market environment improvement (chung, 2012). 1.2 walking space at tms walking is the easiest and most basic way for people to move around and is the basic means of transportation at the final stage of travel. there are two types of walking, the objective and the non-objective type. commuting, attending school, shopping and recreation are objective walking activities, while jogging and walking with children are non-objective walking activities. the former seeks the short course, the later uses the interesting course. with this view, walking is not simply transportation or a way to reach a destination, but it can provide vitality and freshness in outdoor spaces. walking is slower than other forms of transportation, and this characteristic creates an opportunity for people to come in contact with the environment and other people to grow their social, ijahp article: chung, yoon/ analysis of elements to improve the walking environment in korean traditional markets using analytic hierarchy process international journal of the analytic hierarchy process 481 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.410 economic and cultural networks. in other words, walking spaces have social significance that supports an eco-friendly environment and city, stimulates the economy, improves connectivity and accessibility of transportation, promotes health of citizens and ensures the safety of the city. walking space is an important outdoor space for people and is a main element of any city. the five primary functions of a city street are place, movement, access, parking and utilities (the department for transport, 2007). in addition, there are many functions of walking spaces as listed in the following table. table 1 functions of walking space (street) function elements of walking space source place, movement, access, parking, utilities department for transport (2007) place, comfort, connectivity and ministry of culture, symbolism, progressive choi, kim (2009) conviviality, convenience, connectivity, conspicuousness, comfort gardener (1996) safety, comfort, convenience, interest chung (2012) most people in korea recognize that walking in tms is uncomfortable, confusing and crowded, and therefore do not want to go there. this makes walkability the critical factor in the revitalization of tms. there are many activities and objects on narrow streets in tms which include walking, sitting, shopping, pitching, price negotiation, cars, cycles and billboards. most tms in korea are located nearby a residential area or cub where there are many pedestrians. however, current policies and legislation for tms in korea don't consider the walking space for consumers, but only concentrate on the seller’s needs. therefore, the effectiveness of tm improvement projects to revitalize tms is minimal. effective revitalization of traditional markets that value local culture requires research of pedestrian space based on fundamental human behavior rather than physical improvement of buildings and structures. the tm is a commercial space created to serve a buyer’s needs, and it is typically located in a place where there is a lot of walking traffic and it is convenient to meet people. characteristics of the markets are such that the walking areas include not only walking and shopping, but also waiting, negotiating, and various other commercial activities. as a result, there are commercial activities and general pedestrian elements (safety, landscape, amenity and convenience) to consider in the walking areas of traditional markets (gardner, 1996). the restoration of the pedestrian areas in traditional markets can most easily revitalize the economy of downtown areas, where it is impractical to change the current organization of urban space. therefore, characteristics of the traditional market and pedestrian areas should be understood in order to revitalize the traditional markets located in old city centers. in summary, the main goal of this study is to improve the pedestrian experience ijahp article: chung, yoon/ analysis of elements to improve the walking environment in korean traditional markets using analytic hierarchy process international journal of the analytic hierarchy process 482 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.410 and thereby revitalize traditional markets. the present paper deals with the priority of elements to improve walking areas in traditional markets using ahp. table 2 hierarchy of walking environment elements in traditional markets top level 1 level 2 improvement elements of walking environment safety sidewalk width, lack of sidewalk, illegal parking, obstacles, sidewalk slope, billboards, bicycle roads, speed bumps comfort street vegetation, street lights, landscape, pedestrians, cul-de-sacs, traffic, vehicle speed, stalls convenience paving, benches, trash bins, information signs, public transportation, parking interest shopping, price negotiation, pitchmen, street vendors, people watching, crowdedness, activities, noise 2. analysis of elements for walking environments in traditional markets 2.1 survey this study utilizes elements for improving the pedestrian experience in traditional markets from the literature review (table 1). a questionnaire was made containing 28 elements: sidewalk width, lack of sidewalk, illegal parking, obstacles, sidewalk slope, billboards, bicycle roads, speed bumps, street vegetation, street lights, landscape, pedestrians, cul-de-sacs, traffic, vehicle speed, stalls, paving, benches, trash bins, information signs, public transportation, parking lots, shopping, price negotiation, pitchmen, street vendors, people watching, crowdedness, activities and noise. these elements were grouped into one of the four main groups: safety, comfort, convenience and interest. the purpose was to identify priority elements for improving the pedestrian experience. the survey was conducted on january 10, 2016 with 20 students at chonnam university in south korea. 2.2 analysis process of ahp the questions using the fundamental scale were made by hierarchy decomposition (saaty, 2000). the four main elements were used from the previous study and the results were produced using pairwise comparison. those four elements were also used for the basis column comparison using the transitivity of preference relation to reduce the number of questions. each respondent’s value was different, and the weight to comparison matrices was made by geometric mean. comparison matrices of elements were calculated by the harker method that can calculate the weights without unknown comparison estimates. the harker’s algorithms can be described as follows (gao, zhang & cao, 2010; dave, desai & raval, 2012): ijahp article: chung, yoon/ analysis of elements to improve the walking environment in korean traditional markets using analytic hierarchy process international journal of the analytic hierarchy process 483 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.410 1. construct a derived reciprocal matrix of comparison matrix. 2. calculate the largest eigenvalue of reciprocal matrix and its associate eigenvector. 3. normalize the eigenvector into a priority weight vector. this means that missing datum in an incomplete comparison matrix change to 0, diagonal components change to numbers of counters of zero plus one. in the harker method, missing components in the comparison matrix change to zero and diagonal components change to one plus numbers of the missing components. that means the maximum eigenvector of the pairwise comparison matrix is to be the weight. after this, the analysis proceeds by normal ahp as follows (saaty, 2008): 1. structure the hierarchy from the top (improvement elements of walking environment), then the objectives from a broad perspective, through the intermediate levels (level 1) to the lowest level (level 2). 2. construct a set of pairwise comparison matrices. each element in an upper level is used to compare the elements in the level immediately below with respect to itself. 3. use the priorities obtained from the comparisons to weigh the priorities in the level immediately below. do this for every element. then for each element in the level below add its weighed values and obtain its overall or global priority. 4. test consistency with the number of elements: n, consistency index (c.i.: -n / n-1), r.i., consistency ratio (c.r.: c.i./r.i.) and maximum eigenvalue ( ). according to saaty and vargas (1982), if the calculated c.r. is lower than 0.1, the judgment matrix is considered consistent (saaty, 1980). 3. results in summary, this study deals with ways to improve the pedestrian experience in order to revitalize traditional markets. the objective of this paper is to find the priority of elements to improve walking areas in traditional markets using ahp. the hierarchy of four main elements and 28 sub-elements were taken from previous research. the survey, pairwise comparison and the ahp process led to the following results. the results of the prioritization of level 1 elements from the ahp process for walking environments are shown in table 3. in descending order of importance, the elements are interest, convenience, comfort and safety. this suggests that korean traditional markets can be best revitalized by upgrading the elements related to interest. ijahp article: chung, yoon/ analysis of elements to improve the walking environment in korean traditional markets using analytic hierarchy process international journal of the analytic hierarchy process 484 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.410 table 3 comparison matrix of walking environment in tms comfort convenience interest safety priority comfort 1.000 0.688 0.377 1.919 0.182 convenience 1.453 1.000 0.304 1.109 0.183 interest 2.652 3.288 1.000 3.800 0.508 safety 0.521 0.902 0.263 1.000 0.127 c.r. 0.027 c.i. 0.030 the walking environment in tms has eight elements of safety. the element with the highest priority of the safety elements is obstacles. table 4 comparison matrix of safety s id e w a lk w id th l a c k o f si d e w a lk il le g a l p a rk in g o b st a c le s s id e w a lk sl o p e b il lb o a rd s b ic y c le ro a d s s p e e d b u m p s p ri o ri ty sidewalk width 7.000 0.000 0.000 0.000 1.643 0.000 0.000 0.000 0.132 lack of sidewalk 0.000 7.000 0.000 0.000 1.460 0.000 0.000 0.000 0.127 illegal parking 0.000 0.000 7.000 0.000 1.767 0.000 0.000 0.000 0.136 obstacles 0.000 0.000 0.000 7.000 2.076 0.000 0.000 0.000 0.145 sidewalk slope 0.609 0.685 0.566 0.482 2.000 2.474 2.765 0.000 0.141 billboards 0.000 0.000 0.000 0.000 0.404 6.000 0.000 2.305 0.127 bicycle roads 0.000 0.000 0.000 0.000 0.362 0.000 7.000 0.000 0.095 speed bumps 0.000 0.000 0.000 0.000 0.000 0.434 0.000 7.000 0.097 c.r. 0.051 c.i. 0.036 the walking environment in tms has eight elements of comfort. the priority of these elements and the comparison matrix is shown in table 5. ijahp article: chung, yoon/ analysis of elements to improve the walking environment in korean traditional markets using analytic hierarchy process international journal of the analytic hierarchy process 485 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.410 table 5 comparison matrix of comfort s tr e e t v e g e ta ti o n s tr e e t li g h ts l a n d sc a p e p e d e st ri a n s c u ld e sa c s t ra ff ic v e h ic le sp e e d s ta ll s p ri o ri ty street vegetation 7.000 0.000 0.000 0.000 1.251 0.000 0.000 0.000 0.121 street lights 0.000 7.000 0.000 0.000 1.986 0.000 0.000 0.000 0.145 landscape 0.000 0.000 7.000 0.000 2.098 0.000 0.000 0.000 0.149 pedestrians 0.000 0.000 0.000 7.000 0.427 0.000 0.000 0.000 0.094 cul-de-sacs 0.799 0.504 0.477 2.342 2.000 2.770 4.148 0.000 0.186 traffic 0.000 0.000 0.000 0.000 0.361 6.000 0.000 2.624 0.127 vehicle speed 0.000 0.000 0.000 0.000 0.241 0.000 7.000 0.000 0.088 stalls 0.000 0.000 0.000 0.000 0.000 0.381 0.000 7.000 0.090 c.r. 0.066 c.i. 0.047 the walking environment in tms has just six elements of convenience. the priority of these elements is shown in table 6. table 6 comparison matrix of convenience p a v in g b e n c h e s t ra sh b in s in fo rm a ti o n si g n s p u b li c tr a n sp o rt a ti o n p a rk in g l o ts p ri o ri ty paving 5.000 0.000 0.000 0.000 5.429 0.000 0.289 benches 0.000 5.000 0.000 0.000 4.074 0.000 0.242 trash bins 0.000 0.000 5.000 0.000 2.691 0.000 0.195 information signs 0.000 0.000 0.000 4.000 2.466 2.947 0.189 public transportation 0.184 0.245 0.372 0.405 2.000 0.000 0.072 parking lots 0.000 0.000 0.000 0.339 0.000 5.000 0.013 c.r. 0.106 c.i. 0.085 the walking environment in tms has eight elements of interest. the priority of these elements and the comparison matrix is shown in table 7. ijahp article: chung, yoon/ analysis of elements to improve the walking environment in korean traditional markets using analytic hierarchy process international journal of the analytic hierarchy process 486 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.410 table 7 comparison matrix of interest s h o p p in g p ri c e n e g o ti a ti o n p it c h m e n s tr e e t v e n d o rs p e o p le c ro w d e d n e ss a c ti v it ie s n o is e p ri o ri ty shopping 7.000 0.000 0.000 0.000 1.944 0.000 0.000 0.000 0.132 price negotiation 0.000 7.000 0.000 0.000 1.539 0.000 0.000 0.000 0.121 pitchmen 0.000 0.000 7.000 0.000 3.538 0.000 0.000 0.000 0.176 street vendors 0.000 0.000 0.000 7.000 2.610 0.000 0.000 0.000 0.150 people 0.515 0.650 0.283 0.383 2.000 2.280 3.681 0.000 0.132 crowdedness 0.000 0.000 0.000 0.000 0.439 6.000 0.000 1.698 0.107 activities 0.000 0.000 0.000 0.000 0.272 0.000 7.000 0.000 0.087 noise 0.000 0.000 0.000 0.000 0.000 0.589 0.000 7.000 0.094 c.r. 0.069 c.i. 0.049 ijahp article: chung, yoon/ analysis of elements to improve the walking environment in korean traditional markets using analytic hierarchy process international journal of the analytic hierarchy process 487 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.410 table 8 priority of walking elements at tm elements on level 1 elements on level 2 priorities overall priorities ranking safety sidewalk width 0.179 0.023 17 lack of sidewalk 0.159 0.020 19 illegal parking 0.193 0.025 14 obstacles 0.226 0.029 12 sidewalk slope 0.109 0.014 20 billboards 0.051 0.006 26 bicycle roads 0.039 0.005 30 speed bumps 0.043 0.005 29 consistency ratio 0.011 consistency index 0.008 comfort street vegetation 0.161 0.029 11 street lights 0.255 0.047 7 landscape 0.270 0.049 6 pedestrians 0.055 0.010 24 cul-de-sacs 0.129 0.023 15 traffic 0.055 0.010 23 vehicle speed 0.031 0.006 27 stalls 0.044 0.008 25 consistency ratio 0.015 consistency index 0.010 convenience paving 0.336 0.061 5 benches 0.252 0.046 8 trash bins 0.167 0.030 10 information signs 0.153 0.028 13 public transportation 0.062 0.011 22 parking lots 0.030 0.006 28 consistency ratio 0.033 consistency index 0.027 interest shopping 0.163 0.083 3 price negotiation 0.129 0.065 4 pitchmen 0.296 0.151 1 street vendors 0.219 0.111 2 people watching 0.084 0.043 9 crowdedness 0.042 0.021 18 activities 0.023 0.012 21 noise 0.045 0.023 16 consistency ratio 0.009 consistency index 0.007 table 8 shows the priorities and rankings of all sub-elements of the walking environments in tms. for all sub-elements, the descending order of importance is ijahp article: chung, yoon/ analysis of elements to improve the walking environment in korean traditional markets using analytic hierarchy process international journal of the analytic hierarchy process 488 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.410 pitchmen, street vendors, shopping, price negotiation, paving, landscape, street lights, benches, people watching, trash trash bins, street vegetation, obstacless, information signs, illegal parking, cul-de-sacs, noise, sidewalk width, crowdedness, lack of sidewalk, sidewalk slope, activities, public transportationation, traffic, pedestrians, stalls, billboards, vehicle speed, parking lots, speed bumps, and bicycle roads. by far the most impactful element in level 1 is interest, but some sub-elements such as people watching, crowdedness, activities and noise are relatively low in the overall order compared with shopping, price negotiation, pitchmen and street vendors. further research is needed with more refined sub-elements in the hierarchy model to improve traditional markets in korea. first, the order of priority for the main elements is interest, convenience, comfort and safety. this means that upgrading elements related to interest will have the greatest impact on the revitalization of korean markets. second, the order of priority for the subelements is pitchmen, street vendors, shopping, price negotiation, paving, landscape, street lights, benches, people, trash trash bins, street vegetation, obstacless, information signs, illegal parking, cul-de-sacs, noise, sidewalk width, crowdedness, lack of sidewalk, sidewalk slope, activities, public transportationation, traffic, pedestrians, stalls, billboards, vehicle speed, parking lots, speed bumps, and bicycle roads. 4. discussion this study showed that ahp was an efficient method to identify priorities of elements of walking environments in tms in korea. these priorities may help designers define and focus on certain elements in improving walking environments. ahp was an effective technique for dealing with the fussy system associated with quantitative and qualitative elements at tms in korea. however, too many reduced pairwise comparisons and the distortion of relative priority between elements should be considered further. it should be taken into account that reduced pairwise comparisons for respondent’s convenience and the following distortion of relative priority between elements remained a problem. it is also necessary to compare studies between tms to reduce the error of generalization. the results of this study should be verified in practicality with case studies as well. 5. conclusion in this study the priority of elements of the walking environment at tms in korea was described. the priority was easily calculated by decomposition hierarchy and pairwise comparison. ahp was an effective technique for dealing with the fussy system associated with quantitative and qualitative elements at traditional markets in korea. ijahp article: chung, yoon/ analysis of elements to improve the walking environment in korean traditional markets using analytic hierarchy process international journal of the analytic hierarchy process 489 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.410 references chung, k. (2012). analysis of user satisfaction for walking environments through structural equation modeling at traditional markets. journal of the regional association of architectural institute of korea, 14(4). dave, h., desai, k. & raval, h. (2012). a decision support system for tool electrode selection for electro discharge machining process using the analytic hierarchy process. international journal of the analytic hierarchy process, 4(2), 89-103. doi: http://dx.doi.org/10.13033/ijahp.v4i2.131 gao, s., zhang, z. & cao, c. (2010). calculating weights methods in complete matrices and incomplete matrices. journal of software, 5(3), 304-311. gardner et al. (1996). a report on the development of a pedestrian strategy for london. etc proceedings, http://www.etcproceedings.org/paper/developing-a-pedestrianstrategy-for-london harker, p.t. (1987). incomplete pairwise comparisons in the analytic hierarchy process. mathematical modelling, 9(11), 837-848.doi:10.1016/0270-0255(87)90503-3 kim, cd. & park, jc. (2006). a study for revitalization of local markets after surrounding-improvement projects. seoul development institute. oxforddictionaries.com. oxford: oxford university press. retrieved 26 may 2016. saaty, r. (2009). participant names and papers. proceedings of the 10th international symposium on the analytic hierarchy/network process. pittsburgh, pennsylvania. saaty, tl. (2008) decision making with the analytic hierarchy process. international journal of services sciences, 1(1), 83-98. doi: 10.1504/ijssci.2008.017590 saaty, tl. & vargas, lg. (1982). the logic of priorities. boston: kluwer-nijhoff pulishing. saaty, tl. (2000). fundamentals of decision making and priority theory with the analytic hierarchy process, vol. vi. pittsburgh: rws publications. saaty, tl. (1980). the analytic hierarchy process. new york: mcgraw-hill. doi: http://dx.doi.org/10.1080/00137918308956077 saaty, tl. (1980). multicriteria decision making: the analytic hierarchy process. pittsburgh, pa: rws publications. small and medium business administration in korea (2006). traditional market activation comprehensive policy. the department for transport (2007). manual for streets. uk. http://dx.doi.org/10.13033/ijahp.v4i2.131 http://dx.doi.org/10.1016/0270-0255(87)90503-3 http://dx.doi.org/10.1504/ijssci.2008.017590 http://dx.doi.org/10.1080/00137918308956077 ijahp article: lipovetsky/an interpretation of the ahp global priority as the eigenvector solution of an anp supermatrix international journal of the analytic hierarchy process 70 vol. 3 issue 1 2011 issn 1936-6744 an interpretation of the ahp global priority as the eigenvector solution of an anp supermatrix stan lipovetsky stan.lipovetsky@gfk.com gfk custom research north america 8401 golden valley rd., minneapolis, mn 55427, usa abstract not only the local priority vectors but also the global synthesized priority vector for the alternatives can be obtained from the eigenproblem solution for the anp supermatrix. this global priority vector retains the main property of any ahp vector – to present the mean dominance of each of the items over the others with respect to the goal of the hierarchy. keywords: ahp local and global vectors, eigenproblem of supermatrix, anp supermatrix. “when making a decision of minor importance, i have always found it advantageous to consider all the pros and cons. in vital matters, however, such as the choice of a mate or a profession, the decision should come from the unconscious, from somewhere within ourselves. in the important decisions of personal life, we should be governed, i think, by the deep inner needs of our nature.” sigmund freud. 1. introduction in the article, an interpretation of the ahp eigenvector solution for the lay person, in the previous issue of this journal (lipovetsky, 2010), we showed how the priority vector for an analytic hierarchy process (ahp) pairwise comparison judgment matrix can be derived through an iterative process and that the vector obtained in this way is an eigenvector of the original judgment matrix. similarly an eigenproblem solution can be obtained for an ahp problem, formulated in the supermatrix of the analytic network process (anp) rather than as a hierarchy; it gives the overall relative priorities of the alternatives with respect to the goal, and also the synthesized relative priorities of the alternatives with respect to all the other nodes in the model as well. 2. the hierarchical synthesis process synthesizing local priority vectors into global priority vectors for an ahp model can be easily performed by multiplying the matrix z comprised of the local eigenvectors of the alternatives with respect to the criteria, times a column vector x of criteria priorities with respect to the goal. this is equivalent to performing additive hierarchical composition. rob typewritten text http://dx.doi.org/10.13033/ijahp.v3i1.90 ijahp article: lipovetsky/an interpretation of the ahp global priority as the eigenvector solution of an anp supermatrix international journal of the analytic hierarchy process 71 vol. 3 issue 1 2011 issn 1936-6744 consider the example shown in table 1 of “choosing the best house” (saaty, 1996, pp. 26-31). this classical ahp example has also been considered in (saaty & kearns, 1985, ch.3; saaty and vargas, 1994, ch.1). this data was also used for testing some new techniques in (lipovetsky, 1996, 2005; lipovetsky & tishler, 1999; lipovetsky & conklin, 2002). in this example three houses are evaluated with respect to 8 criteria. the top row in table 1 contains the weights of the criteria obtained by pairwise comparing the criteria with respect to the goal. this row vector may be labeled 'x (where the prime denotes transposition) so that x is a column vector. the matrix z is the 3 × 8 matrix containing the local priority vectors of the alternatives with respect to the criteria in table 1. the last column contains the product of zx, the global priority vector. as z is 3 × 8 and the x is 8 × 1, zx is a column vector of order 3 × 1. table 1 local and global priority vectors for “choosing the best house” (saaty, 1996) size yard transport facilities neighborhd condition age finance criteria weights .1730 .0540 .1881 .0175 .0310 .0363 .1669 .3332 global vector house a .7536 .6738 .2331 .7466 .7536 .2000 .3333 .0719 .3338 house b .1811 .1007 .0545 .0601 .0653 .4000 .3333 .6491 .3365 house c .0653 .2255 .7124 .1933 .1811 .4000 .3333 .2790 .3296 the anp supermatrix for an ahp model that includes inner-dependent criteria has been considered by saaty (1994, pp. 245-6; 1996, pp. 132-133; 2010, p. 190) and is shown below (with a first row of zeros for the goal): ⎟ ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎜ ⎝ ⎛ = iz yxw 0 0 000 , (1) our house model can be arranged into a supermatrix w with components x, y and z in the form shown in equation (1) from (saaty, 1996, p. 97). the supermatrix w of our house hierarchy consists of a goal with n criteria and m alternatives. the top row of zeros contains the priorities of the goal node with respect to all the other nodes in the model. the goal is a source node that connects only to other nodes and not from them, so it has only zero priorities with respect to the other nodes. the x component is an n × 1 column vector of weights derived by pairwise comparing the criteria with respect to the goal, y is an n × n order matrix of priority vectors derived for inner-dependent criteria, z is an m × n matrix of priority vectors of the alternatives with respect to the criteria, and for a hierarchy, it is necessary to include an identity matrix i of order m × m in the right hand bottom corner of w , thus the matrix w is of the order m + n +1. this formulation leads to a supermatrix w of the 12th order, shown in table 2. if there were subcriteria, the supermatrix would need to be expanded to accommodate them and all the rows would no longer be stochastic. any column can be made stochastic by normalizing it. the vector x is the 8 × 1 vector of criteria priorities with respect to the goal (blue); y is an 8 × 8 matrix of priority vectors, one for each of the criteria in the case ijahp article: lipovetsky/an interpretation of the ahp global priority as the eigenvector solution of an anp supermatrix international journal of the analytic hierarchy process 72 vol. 3 issue 1 2011 issn 1936-6744 where the criteria are inner dependent (shown in yellow), but y contains only zeros as for a hierarchy where the criteria are not inner dependent; z is the 3 × 8 matrix of priority vectors of alternatives with respect to the criteria (in green); and i is the 3 × 3 identity matrix (pink). following saaty’s approach of raising this matrix to powers until it reaches a stable solution (at the second power in this example), we obtain the global priority vector, with respect to the goal, in the first column in table 3. it is the same as the results in table 1 obtained using hierarchic composition. the results for hierarchic composition coincide with the vector product zx. ijahp article: lipovetsky/an interpretation of the ahp global priority as the eigenvector solution of an anp supermatrix international journal of the analytic hierarchy process 73 vol. 3 issue 1 2011 issn 1936-6744 table 2 the supermatrix w containing the local priority vectors for the house hierarchy goal criteria alternatives goal size yard transprt facilities nghbrhd condition age finance house a house b house c goal goal 0 0 0 0 0 0 0 0 0 0 0 0 size 0.173 0 0 0 0 0 0 0 0 0 0 0 yard 0.054 0 0 0 0 0 0 0 0 0 0 0 transprt 0.1881 0 0 0 0 0 0 0 0 0 0 0 criteria facilities 0.0175 0 0 0 0 0 0 0 0 0 0 0 nghbrhd 0.031 0 0 0 0 0 0 0 0 0 0 0 condition 0.0363 0 0 0 0 0 0 0 0 0 0 0 age 0.1669 0 0 0 0 0 0 0 0 0 0 0 finance 0.3332 0 0 0 0 0 0 0 0 0 0 0 house a 0 0.7536 0.6738 0.2331 0.7466 0.7536 0.2000 0.3333 0.0719 1 0 1 alternatives house b 0 0.1811 0.1007 0.0545 0.0601 0.0653 0.4000 0.3333 0.6491 0 1 0 house c 0 0.0653 0.2255 0.7124 0.1933 0.1811 0.4000 0.3333 0.2790 0 0 0 ijahp article: lipovetsky/an interpretation of the ahp global priority as the eigenvector solution of an anp supermatrix international journal of the analytic hierarchy process 74 vol. 3 issue 1 2011 issn 1936-6744 table 3 the limit supermatrix w for the house example; the global priority vector for the alternatives is shown in blue goal criteria alternatives goal size yard transprt facilities nghbrhd condition age finance house a house b house c goal goal 0 0 0 0 0 0 0 0 0 0 0 0 size 0 0 0 0 0 0 0 0 0 0 0 0 yard 0 0 0 0 0 0 0 0 0 0 0 0 transprt 0 0 0 0 0 0 0 0 0 0 0 0 criteria facilities 0 0 0 0 0 0 0 0 0 0 0 0 nghbrhd 0 0 0 0 0 0 0 0 0 0 0 0 condition 0 0 0 0 0 0 0 0 0 0 0 0 age 0 0 0 0 0 0 0 0 0 0 0 0 finance 0 0 0 0 0 0 0 0 0 0 0 0 house a 0.3339 0.7536 0.6738 0.2331 0.7466 0.7536 0.2000 0.3333 0.0719 1 0 0 alternatives house b 0.3365 0.1811 0.1007 0.0545 0.0601 0.0653 0.4000 0.3333 0.6491 0 1 0 house c 0.3296 0.0653 0.2255 0.7124 0.1933 0.1811 0.4000 0.3333 0.2790 0 0 1 ijahp article: lipovetsky/an interpretation of the ahp global priority as the eigenvector solution of an anp supermatrix international journal of the analytic hierarchy process 75 vol. 3 issue 1 2011 issn 1936-6744 we shall now show a second method of obtaining the synthesized global priority vector. instead of raising the supermatrix w of equation (1) to powers, we solve the transpose of this matrix in a straightforward manner for its left eigenvectors. the left eigenvectors of (2) coincide with the right eigenvectors of w but are obtained from the transposed matrix: λγγ =′w (2) for a hierarchy of only three levels, such as the house model, the columns of w are stochastic, that is, they sum to one. the left eigenvectors are needed for the transposed matrix because they correspond correctly to the calculations by its blocks for the needed product zx that gives the synthesized global vector. the left eigenvectors and their eigenvalues of w ′ , the transpose of the matrix in table 2, are given in table 4. three of the columns have eigenvalues equal to 1, and these columns contain the synthesized global (row) vector for the alternative houses in the goal node row: house c = 0.3296, house b = 0.3365, and house a = 0.3339. in this solution the order of the columns is the reverse of the order of the rows. the synthesized result of the alternatives with respect to each criterion can also be read from the rows of table 4. it is useful to note the following to clarify the problem. as is well known, perron proved that every positive matrix has a largest real eigenvalue that is greater than the absolute value of any of the others (including the complex ones). frobenius carried the work on to matrices with zeros in them, like our supermatrix in table 2. he proved there could be more than one largest eigenvalue. and the supermatrix w ′ is a transposed stochastic one whose eigenvalues are all one. so when we transpose it in (2) it becomes a stochastic matrix with the rows equaling one, as in our example which has multiple eigenvalues of 1. table 4 all twelve eigenvectors (columns) and associated eigenvalues (bottom row) for the supermatrix with the synthesized global priorities for the houses in row 1. house c house b house a financ e age con ditio n neig hbour fac ilitie s tran sport yard size goal goal 0.3296 0.3365 0.3339 -1 -1 -1 -1 -1 -1 -1 -1 1 size 0.0653 0.1811 0.7536 0 0 0 0 0 0 0 0 0 yard 0.2255 0.1007 0.6738 0 0 0 0 0 0 0 0 0 transpor t 0.7124 0.0545 0.2331 0 0 0 0 0 0 0 0 0 facilities 0.1933 0.0601 0.7466 0 0 0 0 0 0 0 0 0 neighbo ur 0.1811 0.0653 0.7536 0 0 0 0 0 0 0 0 0 conditio n 0.4 0.4 0.2 0 0 0 0 0 0 0 0 0 age 0.3333 0.3333 0.3333 0 0 0 0 0 0 0 0 0 finance 0.279 0.6491 0.0719 0 0 0 0 0 0 0 0 0 housea 0 0 1 0 0 0 0 0 0 0 0 0 ijahp article: lipovetsky/an interpretation of the ahp global priority as the eigenvector solution of an anp supermatrix international journal of the analytic hierarchy process 76 vol. 3 issue 1 2011 issn 1936-6744 houseb 0 1 0 0 0 0 0 0 0 0 0 0 housec 1 0 0 0 0 0 0 0 0 0 0 0 eigen values 1 1 1 0 0 0 0 0 0 0 0 0 inverting the arrangement of the three main eigenvectors to put the houses back into the proper order with which we started, and extracting the synthesis information we are interested in, we obtain table 5. it shows the global synthesized priorities with respect to the goal in the first row along with the global synthesized priorities of all the other nodes (in rows). the global priority vectors are the same as the original local priority vectors for a simple three level hierarchy. however, for a hierarchy of more than 3 levels, the synthesis is more interesting as the global priorities are not usually the same as the local priorities for the other nodes in the model. table 5 local and global priorities obtained from the main eigenvectors from the supermatrix for the “choosing the best house” example 3. conclusion the eigenvectors in tables 4 and 5 give the same results in the first row for the global synthesis vectors as that obtained using the regular ahp synthesis shown in table 1. thus, the global ahp priority can be found by solving the supermatrix’ eigenproblem for its left eigenvectors, which in a way similar to the local eigenvectors represents the mean dominance of each item over the others. references lipovetsky, s. (1996). the synthetic hierarchy method: an optimizing approach to obtaining priorities in the ahp. european journal of operational research, 93, 550-564. eigenvectors of the supermatrix names of nodes in hierarchy house a house b house c goal .3338 .3365 .3296 size .7536 .1811 .0653 yard .6738 .1007 .2255 transport .2331 .0545 .7124 facilities .7466 .0601 .1933 neighbor .7536 .0653 .1811 condition .2000 .4000 .4000 age .3333 .3333 .3333 finance .0719 .6491 .2790 house a 1 0 0 house b 0 1 0 house c 0 0 1 eigenvalues λ 1 1 1 ijahp article: lipovetsky/an interpretation of the ahp global priority as the eigenvector solution of an anp supermatrix international journal of the analytic hierarchy process 77 vol. 3 issue 1 2011 issn 1936-6744 lipovetsky, s., & tishler, a. (1999). interval estimation of priorities in the ahp. european journal of operational research, 114, 153-164. lipovetsky, s., & conklin, m. (2002). robust estimation of priorities in the ahp. european journal of operational research, 137, 110-122. lipovetsky, s. (2005). analytic hierarchy processing in chapman-kolmogorov equations. international j. of operations and quantitative management, 11, 219-228. lipovetsky, s. (2010). an interpretation of the ahp eigenvector solution for the lay person. ijahp, 2/2, 159-162. saaty, t.l. (1994). fundamentals of decision making and priority theory with the analytic hierarchy process. pittsburgh, pa: rws publications. saaty, t.l. (1996). decision making with dependence and feedback: the analytic network process. pittsburgh, pa: rws publications. saaty, t.l. (2010). principia mathematica decernendi: mathematical principles of decision making: generalization of the analytic network process to neural firing and synthesis. pittsburgh, pa: rws publications. saaty, t.l., & kearns, k.p. (1985). analytical planning. new york: pergamon press. saaty, t.l., & vargas, l.g. (1994). decision making in economic, political, social and technological environment with the analytic hierarchy process. pittsburgh, pa: rws publications. appendix i. additional bibliography bernasconi m., choirat c. and seri r. (2010). the analytic hierarchy process and the theory of measurement. management science, 56, 699-711. lipovetsky, s. (2008). comparison among different patterns of priority vectors estimation methods. international j. of mathematical education in science and technology, 39, 301-311. lipovetsky, s. (2009a). global priority estimation in multiperson decision making. journal of optimization theory and applications, 140, 77-91. lipovetsky, s. (2009b). optimal hierarchy structures for multi-attribute-criteria decisions. journal of systems science and complexity, 22, 228-242. saaty, t.l. (1977). a scaling method for priorities in hierarchical structures. mathematical psychology, 15, 234-281. saaty, t.l. (1980). the analytic hierarchy process. new york: mcgraw-hill. ijahp article: lipovetsky/an interpretation of the ahp global priority as the eigenvector solution of an anp supermatrix international journal of the analytic hierarchy process 78 vol. 3 issue 1 2011 issn 1936-6744 saaty, t.l. (2000). decision making for leaders. pittsburgh, pa: rws publications. saaty, t.l. (2008a). the analytic hierarchy and analytic network measurement processes: applications to decisions under risk. european journal of pure and applied mathematics, 1,122-196. saaty, t.l. (2008b). relative measurement and its generalization in decision making – why pairwise comparisons are central in mathematics for the measurement of intangible factors: the analytic hierarchy/network process, racsam rev. r. acad. cien. ser. a. mat., 102, 251–318. vaidya o.s. and kumar s. (2006). analytic hierarchy process: an overview of applications. european journal of operational research, 169, 1-29. wasil, e.a., & golden, b.l. (2003). celebrating 25 years of ahp-based decision making. computers and operations research, 30, 1419-1420. ijahp news and events: karpak/m. gabriela sava finalist of the 2020 mcdm junior researcher best paper award international journal of the analytic hierarchy process 364 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.778 m. gabriela sava finalist for the 2020 mcdm junior researcher best paper award birsen karpak associate editor, ijahp distinguished professor, youngstown state university in april 2020, m. gabriela sava received the great news that her paper “an analysis of the sensitivity and stability of patients’ preferences can lead to more appropriate medical decisions,” published in annals of operations research was selected as a finalist for the 2020 mcdm junior researcher best paper award. the paper was in collaboration with dr. luis g. vargas and dr. jerrold h. may from university of pittsburgh and dr. james g. dolan from university of rochester medical center. the award honors junior academic researchers and practitioners who work in the area of multi-criteria decision-making for their excellence, creativity and innovation. gabriela was nominated for this prestigious award by the associate editor of the ijahp, dr. birsen karpak from youngstown state university. m. gabriela sava was born in romania, finished her undergraduate studies at bucharest university of economic studies in 2008, and in 2016 graduated with a phd in business analytics and operations from joseph m. katz graduate school of business, university of pittsburgh. since 2016, she has been an assistant professor at the college of business, clemson university and a faculty scholar at clemson school of health research (cushr). methodologically, gabriela’s current research focuses on the use of sensitivity and stability analysis of multi-criteria decision models and on the use of data mining methods for improving resource allocation in preventive healthcare services. the paper nominated for the award is part of one of her research projects that focuses on designing a personalized approach that supports patient-healthcare provider communication in the selection of an appropriate alternative for colorectal cancer screening. to quantify the necessary level of personalization for each medical encounter between the patient and healthcare provider, she developed a new sensitivity and stability analysis for the analytic hierarchy process (ahp)/analytic network process (anp) models. the new methodology can identify how additional medical information can change the patient’s preferences without additional elicitations, and how the healthcare provider can utilize the findings to tailor the discussions during the medical encounter. the work on this project was financially supported by a one-year accelerate grant from clemson university. we wish gabriela the best of luck with the final presentation for this award that will take place during informs annual meeting in washington d.c, november 8-11, 2020. https://link.springer.com/article/10.1007/s10479-018-3109-3 https://link.springer.com/article/10.1007/s10479-018-3109-3 ijahp article: lipovetsky/ahp structuring in best-worst scaling and the secretary problem international journal of the analytic hierarchy process 502 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.332 ahp structuring in best-worst scaling and the secretary problem stan lipovetsky senior research director, gfk north america marketing science, usa stan.lipovetsky@grk.com abstract the analytic hierarchy process (ahp) is the widely known method and methodology of multiple criteria decision making, which enriches many other areas of mathematical and statistical data analysis. this work considers an extension of ahp hierarchical structuring by incorporating it into another method of prioritization known in marketing research as best-worst scaling (bws). bws is used for finding choice probabilities among the compared items, but when there are a large number of items it is rather difficult to apply this approach directly to all the items. the ahp methodology of hierarchical structuring and estimation of local priorities that are then synthesized into global preferences permits one to build bws nested models to facilitate choice evaluations. for instance, the compared items can be divided into several subsets by the criteria of brand, size, packaging, etc. the bws balanced designs and data eliciting procedure can be applied to each of these groups separately, with additional comparisons among the criteria. synthesizing local choice probabilities by the priorities of the criteria yields global probabilities for the items of choice. in this paper we also apply another simple approach, the so-called “secretary problem” from the operations research field, for comparison. numerical results demonstrate that these techniques can be very useful for prioritization problems in marketing research where there are a large number of items. keywords: ahp; bws; hierarchical structuring; secretary problem; choice probability 1. introduction the analytic hierarchy process (ahp) is the widely known method for multiple criteria decision making introduced by thomas saaty (1980), which has been developed and applied in numerous works (saaty 1994, 1996, 2000; golden et al., 1989; saaty and peniwati, 2012; lipovetsky, 1996, 2009, 2010, 2011, 2013). the relation of ahp to other methods of decision theory has been studied in various works. comparing ahp with multi-attribute utility theory (maut) can be found in saaty (1980), belton (1986), vargas (1987), winkler (1990) and gass (2005). practical utility estimations are usually performed in discrete choice modeling (dcm) for finding utility parameters and choice probabilities via multinomial-logit (mnl) models (mcfadden, 1973, 1981; mcfadden and richter, 1990; train, 2003). the comparison between ahp and the dcm conjoint models used in marketing research have been carried out in various studies (mulye, 1998; mailto:stan.lipovetsky@grk.com ijahp article: lipovetsky/ahp structuring in best-worst scaling and the secretary problem international journal of the analytic hierarchy process 503 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.332 scholl et al., 2005; meißner et al., 2008, 2010; scholz et al., 2007, 2010; kallas et al., 2007, 2011; ijzerman et al., 2012). the main conclusion of those works is that although the methods differ the resulting preference structures are pretty similar on the aggregate level, and on the individual level the ahp often demonstrates higher accuracy in choice prediction. the current work notes the possibility of extending the hierarchical structure used in ahp to another method for prioritizing items, namely, the best-worst scaling (bws), also known as the maximum difference (maxdiff) approach. bws is a contemporary technique widely used in marketing research for estimating choice probabilities for many items. it was proposed by jordan louviere (1991, 1993), and advanced in numerous works (for instance, louviere et al., 2000; marley and louviere, 2005; sawtooth 2007; orme, 2010). in bws, each respondent is presented with several subsets of a few items by way of a balanced design. a respondent answers which of the presented items is the best and which is the worst, and individual utilities are estimated in bws using multinomial logit (mnl) modeling. averaging the individual choice probabilities that were found yields the aggregated probabilities. the latter ones can be evaluated by the analytical formulae in the closed-form solution (lipovetsky & conklin, 2014a,b). hierarchically structured bws is similar to the nested logit of the generalized extreme value (gev) models which have a rich elasticity structure in comparison with regular dcm (ben-akiva & lerman, 1985; wen & koppelman, 2001; hensher et al., 2005). the hierarchical approach to the choice based-conjoint (cbc) models widely used in marketing research corresponds to the adaptive choice based-conjoint (acbc) models used for eliciting sequential data by comparing the best choices from the previous subsets of the items (orme, 2006; chrzan & yardley, 2009; netzer & srinivasan, 2011; wirth & wolfrath, 2012; sawtooth software, 2014). in contrast to gev and acbc, the suggested approach using ahp requires neither a complicated actual nesting of mnl models, nor a complex scheme of data eliciting, so it is free of their computational burdens. 2. ahp combined with bws this work considers ahp combined with bws, so let us call this technique ahp-bws. the ahp is used not in its entire methodological approach but only in its hierarchical structuring of the items under comparison, composing the local preferences into the global ones by weighting them by the importance of the criteria. a hierarchical configuration can be outlined by scrutinizing the connections among the alternatives in order to combine them into groups of different criteria. often the compared bws items can be divided into several subsets, for instance, by the criteria of brand, size, packaging, etc. then, the bws balanced design plans and data eliciting procedure can be applied to each of those subsets separately, with an additional bws comparison among the criteria themselves. synthesizing the local subsets of priorities weighted by the criterion preferences yields the global choice probability for all the items. ahp-bws structuring can be especially useful for data with many dozens of items, because dividing them into relatively small subgroups significantly facilitates the difficulties of balanced designs, data elicitation and estimation. possible optimal parameters of the hierarchy can be ijahp article: lipovetsky/ahp structuring in best-worst scaling and the secretary problem international journal of the analytic hierarchy process 504 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.332 evaluated for assembling a structure with the minimum number of needed comparisons among the alternatives and criteria (lipovetsky, 2006). there always could be research problems requiring bws estimation for a very large number of items, say, a hundred items for prioritization evaluated by thousands of responses. for such situations another approach is useful – it is based on what is known in operations research as the “secretary problem”, also called the best choice problem, and related to the hiring and finite horizon algorithms (chow et al., 1964; presman & sonin, 1972; vanderbei, 1980; rose, 1982; freeman, 1983; petruccelli, 1984; samuels, 1985; ferguson, 1989; stein et al., 2003; bearden, 2006). in this paper we will show that results by different techniques are close to each other, and produce similar rankings among the choices. this means that the simple techniques can be successfully implemented on large data sets to serve various practical aims of managerial decisions in marketing research. for an explicit example, let us use data from a real marketing research project with 3,062 respondents who evaluated seventeen products. each respondent received ten tasks of a balanced design of four items shown together, of which they were to indicate which items are the best and the worst ones. using responses only on the choice of the “best” item, the data can be modeled in a dcm approach. as is well known in dcm applications, each task can be presented in several rows, by the number of the items shown in one task, and such an extended matrix of data is completed with the additional column of the binary outcome where the “best” of these items is indicated by the value 1, and other values equal zero (more detail in lipovetsky and conklin, 2014a,b). dcm corresponds to a choice among several outcomes and can be described by a multinomial-logit model with the probability of a choice presented as the following: ,,...,2,1, )exp( )exp( 1 mk xa xa p m j jj kk k    (1) where xj are the variables of binary values indicating it was a j-th item shown to an i-th respondent in a given set or not. the parameters ak are the so-called utilities defining the probability of each k-th choice among all m of them, and the pk are the binary outcome values of the “best” choice. when ak in equation 1 are estimated, the probabilities of different choices are found by this mnl model. repeating the dcm design for the “worst” item chosen, we also obtain four rows for each task, and the outcomes indicated with a value that marks which item is the worst choice. dcm problem of the worst choice item uses the same approach as in equation 1. for a simultaneous estimation of all the best and worst choices in one combined data set the following property is applied: if we change the signs of all predictors then the choice probability defines the absence of a binary event. it means that the design variables in the matrix of the “worst” choices can be taken with a negative sign, keeping the outcome the same as it already is. then in bws, the “best” and “worst” matrices, together with the corresponding outcome variable of the choices, can be stacked by rows into one combined dataset. with such a design, the positive and negative values of the binary predictors will push the outcomes with the values of 1 to the sides of the maximum and ijahp article: lipovetsky/ahp structuring in best-worst scaling and the secretary problem international journal of the analytic hierarchy process 505 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.332 minimum choice probability, respectively. then the hb-mnl model produces individual parameters for each respondent. averaging the individual choice probabilities yields the total choice preferences of the alternatives under consideration. finding utilities and choice probabilities can be performed using specialized software, (for instance, sawtooth software, 2007). as it is shown in lipovetsky and conklin (2014a the estimation of the choice probability on the aggregate level is possible in the closed-form analytical solution (let us denote it as cfans). with counts of the “best” and “worst” choices best j n and worst j n , proportions of the best and worst choices in total j n are j worst j worst jj best j best j nnfnnf /,/  , and the hit rate defined by their difference 2/)1(2/)1( worst j best jjj fffp  . then the choice probability (equation 1) for each item is given by the cfans formula:                          n j x j j x j j m j jj kk k jk f f f f xa xa p 1 1 1 1 / 1 1 )exp( )exp( . (2) where in an i-th row the value is xij=1 if the j-th item is presented and xij=0 if it is absent. consider this problem using the ahp-bws approach. suppose the products belong to three brands, so they can be put into three buckets. as shown in table 1, in this example there are 6, 7, and 4 items in the 1 st , 2 nd , and 3 rd bucket, respectively. instead of using bws applied to all 17 items, we gather bws data for each bucket separately, plus one additional set of comparisons among the buckets themselves. this significantly reduces the number of tasks, that is, the combinations of items, presented to each respondent. indeed, there are 2,380 combinations from 17 by 4, so it is impossible to present all of them to each respondent, and only a small portion of those combinations would be found in an optimum design to be shown to respondents. on the other hand, to implement a hierarchical structure with the items considered within each category, in the first bucket there are 15 combinations from 6 by 4, so it is even possible to present them all, or at least most of them, to each respondent. also, in the second bucket there are 35 combinations from 7 by 4, so it is possible to present at least half of them to each respondent. and in the third bucket there is only 1 combination from 4 by 4, so it can be presented to each respondent. for the higher level bucket of comparisons between the three categories, all three alternatives can be shown to each respondent in only one task. thus, the total number of tasks can be small, but at the same time most of combinations for choices can be shown and checked with each respondent. it makes all the estimation of utilities and choice probabilities more reliable in comparison with a regular non-structured bws approach. ijahp article: lipovetsky/ahp structuring in best-worst scaling and the secretary problem international journal of the analytic hierarchy process 506 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.332 table 1 choice probability within and between buckets, in bucket order frequency in buckets bucket ahp-bws bucket items best item worst item hit rate choice preference item choice probability 1 1 0.224 0.173 0.525 0.255 0.043 1 4 0.425 0.146 0.640 0.255 0.054 1 10 0.257 0.097 0.580 0.255 0.048 1 11 0.042 0.715 0.164 0.255 0.012 1 12 0.365 0.098 0.633 0.255 0.054 1 16 0.099 0.500 0.300 0.255 0.023 2 2 0.050 0.220 0.415 0.411 0.058 2 3 0.762 0.005 0.879 0.411 0.158 2 5 0.316 0.165 0.576 0.411 0.087 2 6 0.035 0.613 0.211 0.411 0.027 2 7 0.281 0.151 0.565 0.411 0.085 2 9 0.252 0.131 0.560 0.411 0.084 2 14 0.052 0.585 0.234 0.411 0.030 3 8 0.027 0.440 0.294 0.334 0.031 3 13 0.807 0.029 0.889 0.334 0.118 3 15 0.083 0.335 0.374 0.334 0.040 3 17 0.082 0.196 0.443 0.334 0.049 table 1 shows the choice proportions of the best and worst items, and the hit rate (described with equation 2), or the absolute choice probability of each of the items, separately defined within each bucket. the next column in table 1 shows the preferences between different buckets, and their product with the hit rate normalized by one is presented in the last column of the synthesized ahp-bws choice probabilities. besides bws, the following estimation for the “secretary problem” (sp), of operations research is possible. briefly, sp can be described as the optimum strategy to maximize the probability of finding the best item when a subset of them is presented to the decision maker. as is known, the best choice in sp is given by the stopping rule of n/e subsamples (n is total number of items, and e is the base of natural logarithm), and picking the next outperforming item. it can also be seen in terms of the gumbel distribution    uxaxxf n  expexp)( , where xn is the maximum x from a sample of size n, a is the scaling parameter reciprocal to the standard deviation, and u is the mode of this distribution, so x-u is the error term. each unobserved component of utility in the discrete choice model in defined by the gumbel distribution, and the difference between two extreme value variables is the logistic distribution (mcfadden, 1973; train, 2003). the point of mode x = u gumbel distribution yields    37.0/10expexp)(  euxf n . in practical terms, it means that for many items we need to consider only about n/e number of them. in the case of many items, we can present to each respondent only one subset of about 0.37*n items and ask which ijahp article: lipovetsky/ahp structuring in best-worst scaling and the secretary problem international journal of the analytic hierarchy process 507 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.332 items are the best and worst ones. then, estimation for the utilities and choice probabilities can be performed using the bws approach. for the example using the data described above, table 2 presents the results of regular bws in comparison with the new approaches. for all the items this table shows the analytical solution for choice probability, then the probability obtained by sawtooth software and by its adjustment advised in bws applications for smoothing (sawtooth software, 2007). in the final two columns, the results of the secretary problem and of ahp-bws are shown. the results vary across the techniques but in general the order of prioritization remains almost the same, especially for the higher probabilities that are the most important items. sawtooth results are skewed towards the best choices, but the adjustment makes them more evenly distributed, and the analytical estimations are between them. the results of the secretary problem (sp) and ahp-bws are close to each other and to the sawtooth adjusted results. table 2 gives the choice probability estimates. table 3 presents the regular pair correlations between the vectors of priorities, that is, the columns of table 2. for instance, the correlation between the analytical and sawthooth columns is 0.989, or the correlation between secretary problem and ahp-bws equals 0.894. so we see that for the suggested techniques of analytical calculations, ahp-bws, and sp produce very similar solutions, although they require much less effort to elicit data and estimate priorities than the regular bws estimations using specialized software. table 2 choice probability estimates item analytical sawtooth sawtooth adjusted secretary problem(sp) ahpbws 1 0.048 0.026 0.051 0.065 0.043 2 0.031 0.013 0.031 0.034 0.058 3 0.256 0.425 0.153 0.166 0.158 4 0.069 0.052 0.108 0.091 0.054 5 0.066 0.067 0.088 0.072 0.087 6 0.016 0.006 0.021 0.019 0.027 7 0.047 0.022 0.083 0.051 0.085 8 0.026 0.002 0.014 0.026 0.031 9 0.061 0.048 0.089 0.081 0.084 10 0.041 0.016 0.047 0.055 0.048 11 0.011 0.001 0.005 0.010 0.012 12 0.068 0.056 0.077 0.085 0.054 13 0.143 0.221 0.135 0.130 0.118 14 0.024 0.013 0.021 0.031 0.030 15 0.028 0.008 0.023 0.026 0.040 16 0.029 0.016 0.022 0.032 0.023 17 0.036 0.008 0.031 0.028 0.049 ijahp article: lipovetsky/ahp structuring in best-worst scaling and the secretary problem international journal of the analytic hierarchy process 508 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.332 table 3 correlations between the columns of choice probability estimates analytical sawtooth sawtooth adjusted secretary problem ahpbws analytical 1 0.989 0.860 0.934 0.907 sawtooth 1 0.801 0.888 0.875 sawtooth adjusted 1 0.959 0.911 secretary problem 1 0.894 ahp-bws 1 3. summary this paper considered an extension of the ahp methodology of hierarchical structuring into the area of the best-worst scaling used in marketing research for choice probability estimation of multiple items. the compared items can be divided into several subsets by the criteria of brand, size, packaging, etc., then the bws balanced design plans and data eliciting are applied to each of the subsets separately, and bws comparison among the criteria themselves is added. synthesizing the local subset priorities by the preferences among the criteria yields the global choice probabilities of the items. another approach is based on the so-called secretary problem known in operations research. the results by the new approaches are close to the standard evaluations, but require much less effort to evaluate. ahp enriches the bws technique, and the proposed methods are especially useful for working with a large number of items. this approach can be extended to other marketing research techniques. ijahp article: lipovetsky/ahp structuring in best-worst scaling and the secretary problem international journal of the analytic hierarchy process 509 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.332 references belton, v. (1986). a comparison of the analytic hierarchy process and a simple multiattribute value function. european journal of operational research, 26, 7-21. doi:10.1016/0377-2217(86)90155-4 gass, s. (2005). model world: the great debate – maut versus ahp. interfaces, 35, 308-312. golden, b.l., wasil, e.a., & harker, p.t., editors (1989). the analytic hierarchy process: applications and studies. berlin: springer. ijzerman, m.j., van til, j.a., & bridges, j.f.p. (2012). a comparison of analytic hierarchy process and conjoint analysis methods in assessing treatment alternatives for stroke rehabilitation. the patient, 5, 45-56. doi:10.2165/11587140-000000000-00000 lipovetsky, s. (1996). the synthetic hierarchy method: an optimizing approach to obtaining priorities in the ahp. european journal of operational research, 93, 550-564. doi:10.1016/0377-2217(95)00085-2 lipovetsky, s. (2006). optimal hierarchy structures for multi-attribute-criteria decisions. journal of systems science and complexity, 22, 228–242. doi:10.1007/s11424-009-91595 lipovetsky, s. (2009). global priority estimation in multiperson decision making. journal of optimization theory and applications, 140, 77-91. doi:10.1007/s10957-008-9447-6 lipovetsky s., (2010). an interpretation of the ahp eigenvector solution for the lay person, international journal of the analytic hierarchy process, 2, 158-162. doi: http://dx.doi.org/10.13033/ijahp.v2i2.42 lipovetsky s. (2011). an interpretation of the ahp global priority as the eigenvector solution of an anp supermatrix. international journal of the analytic hierarchy process, 3(1), 70-78. doi: http://dx.doi.org/10.13033/ijahp.v3i1.90 lipovetsky s. (2011). priority eigenvectors in analytic hierarchy/network processes with outer dependence between alternatives and criteria. international journal of the analytic hierarchy process, 3(2), 172-179. doi: http://dx.doi.org/10.13033/ijahp.v3i2.123 lipovetsky s. (2013). supermatrix eigenproblem and interpretation of priority vectors in analytic network process. international journal of the analytic hierarchy process, 5(1),105-113. doi: http://dx.doi.org/10.13033/ijahp.v5i1.132 kallas, z., gómez-limón, j.a. & barreiro, j. (2007). decomposing the value of agricultural multifunctionality: combining contingent valuation and the analytical hierarchy process. journal of agricultural economics, 58(2), 218 – 241. doi: 10.1111/j.1477-9552.2007.00085.x http://dx.doi.org/10.1016/0377-2217(86)90155-4 http://dx.doi.org/10.1016/0377-2217(95)00085-2 http://dx.doi.org/10.13033/ijahp.v2i2.42 http://dx.doi.org/10.13033/ijahp.v3i1.90 http://dx.doi.org/10.13033/ijahp.v3i2.123 http://dx.doi.org/10.13033/ijahp.v5i1.132 ijahp article: lipovetsky/ahp structuring in best-worst scaling and the secretary problem international journal of the analytic hierarchy process 510 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.332 kallas, z., lambarraa, f., & gil, j.m. (2011). a stated preference analysis comparing the analytical hierarchy process versus choice experiments. food quality and preferences, 22, 181-192. doi: http://dx.doi.org/10.1016/j.foodqual.2010.09.010 meißner, m., scholz, s.w., & decker, r. (2008). ahp versus aca – an empirical comparison. in preisach, c., burkhardt, h., schmidt-thieme, l., & decker, r. (eds.), data analysis, machine learning, and applications (447-454). berlin: springer. doi: 10.1007/978-3-540-78246-9_53 meißner, m., decker, r., & scholz, s.w. (2010). an adaptive algorithm for pairwise comparison‐based preference measurement. journal of multi-criteria decision analysis, 17, 167-177. doi: 10.1002/mcda.461 mulye, r. (1998). an empirical comparison of three variants of the ahp and two variants of conjoint analysis. journal of behavioral decision making, 11, 263-280. doi: 10.1002/(sici)1099-0771(1998120)11:4<263::aid-bdm301>3.0.co;2-t saaty, t.l. (1980). the analytic hierarchy process. new york: mcgraw-hill. doi: http://dx.doi.org/10.1080/00137918308956077 saaty, t.l. (1994). fundamentals of decision making and priority theory with the analytic hierarchy process. pittsburgh: rws publications. saaty, t.l. (1996). decision making with dependence and feedback: the analytic network process. pittsburgh: rws publications. saaty, t.l. (2000). decision making for leaders. pittsburgh: rws publications. doi: 10.1109/tsmc.1985.6313384 saaty, t.l., & peniwati, k. (2012). group decision making. pittsburgh: rws publications. scholl, a., manthey, l., helm, r., & steiner, m. (2005). solving multiattribute design problems with analytic hierarchy process and conjoint analysis: an empirical comparison. european journal of operational research, 164, 760–777. doi: http://dx.doi.org/10.1016/j.ejor.2004.01.026 scholz, s.w., & decker, r. (2007). measuring the impact of wood species on consumer preferences for wooden furniture by means of the analytic hierarchy process. forest products journal, 57(3), 23 28. scholz, s.w, meißner, m., & decker, r. (2010). measuring consumer preferences for complex products: a compositional approach based on paired comparisons. journal of marketing research, 47(4), 685–698. doi: http://dx.doi.org/10.1509/jmkr.47.4.685 vargas, l.g. (1987). priority theory and utility theory. mathematical modelling, 9, 381385. doi:10.1016/0270-0255(87)90496-9 http://dx.doi.org/10.1016/j.foodqual.2010.09.010 http://dx.doi.org/10.1080/00137918308956077 http://dx.doi.org/10.1109/tsmc.1985.6313384 http://dx.doi.org/10.1016/j.ejor.2004.01.026 http://dx.doi.org/10.1509/jmkr.47.4.685 http://dx.doi.org/10.1016/0270-0255(87)90496-9 ijahp article: lipovetsky/ahp structuring in best-worst scaling and the secretary problem international journal of the analytic hierarchy process 511 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.332 winkler, r.l. (1990). decision modeling and rational choice: ahp and utility theory. management science, 36, 247-248. doi: http://dx.doi.org/10.1287/mnsc.36.3.247 additional references on conjoint, discrete choice, and best-worst scaling ben-akiva, m., & lerman, s.r. (1985). discrete choice analysis. cambridge, ma: mit press. doi: 10.1007/s11336-007-9029-9 chrzan, k., & yardley, d. (2009). tournament-augmented choice-based conjoint. proceedings of the sawtooth software conference, 163-169, delray beach, florida. hensher, d.a., rose, j.m., & greene, w.h. (2005). applied choice analysis: a primer. cambridge, england: cambridge university press. lipovetsky, s., & conklin, m. (2014a). best-worst scaling in analytical closed-form solution. journal of choice modelling, 10, 60–68. doi: http://dx.doi.org/10.1016/j.jocm.2014.02.001 lipovetsky, s., & conklin, m. (2014b). finding items cannibalization and synergy by bws data. journal of choice modelling, 12, doi: 10.1016/j.jocm.2014.08.001. lipovetsky, s. & conklin, m. (2015). maxdiff priority estimations with and without hb-mnl. advances in adaptive data analysis, 7(1-2), 1-10. doi: http://dx.doi.org/10.1142/s1793536915500028 lipovetsky, s., liakhovitski, d., & conklin, m. (2015). what's the right sample size for my maxdiff study? sawtooth software conference proceedings, 119141. louviere, j.j. (1991) best-worst scaling: a model for the largest difference judgments, working paper, university of alberta. louviere, j.j. (1993). the best-worst or maximum difference measurement model: applications to behavioral research in marketing. the american marketing association's behavioral research conference. louviere, j.j., hensher, d.a., & swait, j. (2000). stated choice methods: analysis and applications. cambridge, ma: cambridge university press. doi: https://doi.org/10.1017/cbo9780511753831 marley, a., & louviere, j.j. (2005). some probabilistic models of best, worst, and bestworst choices. journal of mathematical psychology, 49, 464–480. doi: http://dx.doi.org/10.1016/j.jmp.2005.05.003 http://dx.doi.org/10.1287/mnsc.36.3.247 http://dx.doi.org/10.1016/j.jocm.2014.02.001 http://dx.doi.org/10.1142/s1793536915500028 http://dx.doi.org/10.1016/j.jmp.2005.05.003 ijahp article: lipovetsky/ahp structuring in best-worst scaling and the secretary problem international journal of the analytic hierarchy process 512 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.332 mcfadden, d. (1973). conditional logit analysis of qualitative choice behavior. in zarembka, p., (ed.), frontiers of econometrics (105-142). new york: academic press. mcfadden, d. (1981). econometric models of probabilistic choice. in manski, c., & mcfadden, d., (eds.), structural analysis of discrete data (198-272). cambridge: mit press. mcfadden, d., & richter, m.k. (1990). stochastic rationality and revealed stochastic preference. in chipman, j., mcfadden, d., & richter, m.k., (eds.), preferences, uncertainty, and optimality: essays in honor of leo hurwicz (151-186). boulder, co: westview press. netzer, o., & srinivasan, v.s. (2011). adaptive self‐explication of multiattribute preferences. journal of marketing research, 48(1), 140–156. doi: http://dx.doi.org/10.1509/jmkr.48.1.140 orme, b.k. (2006). adaptive maximum difference scaling. sawtooth software research paper series. sequim, wa: sawtooth software, inc. orme, b.k. (2010). getting started with conjoint analysis, 2 nd edition. madison, wi: research publishers llc. sawtooth software (2007). technical paper series: the maxdiff/web v6.0, sequim, wa: sawtooth software, inc. sawtooth software (2014). technical paper series: acbc technical paper. sequim, wa: sawtooth software, inc. train, k. (2003). discrete choice methods with simulation. new york: cambridge university press. wen, c.h., & koppelman, f (2001). the generalized nested logit model. transportation research b, 35, 627–641. doi: http://dx.doi.org/10.1016/s0191-2615(00)00045-x wirth r. and wolfrath a. (2012). using maxdiff for evaluating very large sets of items. proceedings of the sawtooth software conference, 59-78. on secretary problem bearden, j.n. (2006). a new secretary problem with rank-based selection and cardinal payoffs. journal of mathematical psychology, 50, 58-59. doi: http://dx.doi.org/10.1016/j.jmp.2005.11.003 chow, y.s., moriguti, s., robbins, h., & samuels, s.m. (1964). optimal selection based on relative rank. israel journal of mathematics, 2, 81–90. doi: 10.1007/bf02759948 http://dx.doi.org/10.1509/jmkr.48.1.140 http://dx.doi.org/10.1016/s0191-2615(00)00045-x http://dx.doi.org/10.1016/j.jmp.2005.11.003 ijahp article: lipovetsky/ahp structuring in best-worst scaling and the secretary problem international journal of the analytic hierarchy process 513 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.332 ferguson, t.s. (1989). who solved the secretary problem? statistical science, 4, 282296. doi:10.1214/ss/1177012498 freeman, p.r. (1983). the secretary problem and its extensions – a review. international statistical review, 51, 189–206. petruccelli, j.d. (1984). best-choice problems involving recall with uncertainty of selection when the number of observations is random. advances in applied probability, 16, 111-130. doi: https://doi.org/10.1017/s0001867800022370 presman, e.l., & sonin, i.m. (1972). the best-choice problem for a random number of objects. theory of probability and applications, 17, 657-668. doi:10.1137/1117078 rose, j.s. (1982). a problem of optimal choice and assignment. operations research, 30, 172–181. doi: http://dx.doi.org/10.1287/opre.30.1.172 samuels, s.m. (1985). a best-choice problem with linear travel cost. journal of the american statistical association, 80, 461-464. stein, w.e., seale, d.a., & rapoport, a. (2003). analysis of heuristic solutions to the best choice problem. european journal of operational research, 151, 140–152. doi: http://dx.doi.org/10.1016/s0377-2217(02)00601-x vanderbei, r.j. (1980). the optimal choice of a subset of a population. mathematics of operations research, 5, 481–486. doi: http://dx.doi.org/10.1287/moor.5.4.481 https://doi.org/10.1017/s0001867800022370 http://dx.doi.org/10.1137/1117078 http://dx.doi.org/10.1287/opre.30.1.172 http://dx.doi.org/10.1016/s0377-2217(02)00601-x http://dx.doi.org/10.1287/moor.5.4.481 ijahp article: köne, büke/ eco-efficiency analysis using analytic hierarchy process approach international journal of the analytic hierarchy process 167 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.477 eco-efficiency analysis using analytic hierarchy process approach aylin çiğdem köne muğla sıtkı koçman university turkey ckone@mu.edu.tr tayfun büke muğla sıtkı koçman university turkey tbuke@mu.edu.tr abstract the environmental efficiencies of 31 member countries of the european environment agency between 1990 and 2011 were evaluated using the gross domestic product using purchasing power parity. categories of imparting pressure on the environment, including global warming, acidification, tropospheric ozone formation and particle formation potentials were considered for each country. these environmental pressure categories were weighted using the analytical hierarchy process methodology. calculations of the results indicated the netherlands, germany, the united kingdom, hungary and ireland ranked 10th, 15th, 17th, 21st and 22nd respectively in 1990, and their ranking changed to 4th, 8th, 3rd, 17th and 11th respectively in 2011. these results point out a marked shift in the trend of economic and environmental efficiency. on the contrary, portugal, malta, croatia and turkey were ranked 7th, 9th, 13th and 14th respectively in 1990, and at the end of the period, they dropped to 14th, 15th, 22nd and 28th respectively. there seems to be a positive linear relationship that exists between calculated eco-efficiency scores and income, but a high value for gross domestic product using the purchasing power parity does not automatically imply good environmental performance or vice versa. keywords: eco-efficiency; air pollutants; environmental pressure; weighting, ahp 1. introduction sustainable development has been accepted as a major development strategy by several countries since 1992 at the united nations conference on environment and development (unced). unced also noted that it is quite clear that the main cause of global pollution is the continued use of natural resources at previous levels. the acceptance of this fact by the international community led to a discussion of special measures to ensure sustainable economic development. in this context, strategies to optimize resources play a particularly important role. eco-efficiency, a tool for sustainability analysis that shows an empirical relationship between economic activity, environmental cost or value and environmental impact, has been proposed as a way to encourage this transformation (figge & hahn, 2004; huppes & ishikawa, 2005; mickwitz et al., 2006). mailto:ckone@mu.edu.tr mailto:tbuke@mu.edu.tr http://www.sciencedirect.com/science/article/pii/s0921800908001407#ref_bib32#ref_bib32 ijahp article: köne, büke/ eco-efficiency analysis using analytic hierarchy process approach international journal of the analytic hierarchy process 168 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.477 eco-efficiency plays an important role in expressing how effective economic activity is in terms of goods and services of nature. by definition, eco-efficiency is the delivery of competitively priced goods and services that meet human needs and bring quality of life while gradually reducing the ecological impact and resource intensity throughout life. eco-efficiency can be defined as the value of a product or service per environmental impact (verfaille & bidwell, 2000; fet, 2003; kuosmanen & kortelainen, 2005; michelsen, et al., 2006). the aim of this study is to evaluate the environmental efficiencies of 31 member countries of the european environment agency between 1990 and 2011 by using the gross domestic product using purchasing power parity (gdpppp) and environmental pressure categories, including global warming (glwp), acidification (acp), tropospheric ozone formation (tfp) and particle formation potentials (pp) for each country. these environmental pressure categories have been weighted by applying the analytical hierarchy process (ahp) methodology (saaty, 1980; saaty & peniwati, 2007). the calculated results are examined regarding changes in environmental performance and its constituents during the working period, identifying the basic factors of environmental performance for each country, and presenting the possibilities and advantages of the used methodology. this study includes the following eea countries with the abbreviations: austria (at), belgium (be), bulgaria (bg), croatia (hr), cyprus (cy), czech republic (cz), denmark (dk), estonia (ee), finland (fi), france (fr), germany (de), hungary (hu), iceland (is), ireland (ie), italy (it), latvia (lv), lithuania (lt), luxembourg (lu), malta (mt), netherlands (nl), norway (no), poland (pl), portugal (pt), romania (ro), slovakia (sk), slovenia (si), spain (es), sweden (se), switzerland (ch), turkey (tr) and united kingdom (gb). due to insufficient data, greece and liechtenstein are excluded. 2. literature review a number of researchers have focused on different measures for eco-efficiency analysis of various alternatives. an eco-efficiency analysis for 24 power plants in a european country was investigated using data envelopment analysis (dea) (korhonen & luptacik, 2004). in the kymenlaakso region of finland, eco-efficiency has been examined at a regional level as an approach to increase the competitiveness of economic activities and to reduce the harmful effects on the environment (seppälä et al., 2005). an empirical study was employed to describe the pattern of a regional industrial system’s eco-efficiency by using real data from 30 provinces in china (zhang et al., 2008). an environmental performance index was constructed by applying frontier efficiency techniques and a malmquist index approach. the proposed model was applied to the dynamic environmental performance analysis of 20 member states of the european union from 1990–2003. the proposed index is used for several air pollutants and the real gross domestic product for each country (kortelainen, 2008). a comparitive eco-efficiency analysis was examined for the usa and six european countries for the period 1960–2002 (holm & englund, 2009). eco-efficiency analysis has been investigated for different european countries using a disaggregated sector-based approach (wursthorn et al., 2010). farming eco-efficiency has been investigated using dea techniques. for example, eco-efficiency scores were calculated at farm and environmental levels for spanish farmers working in the rain-fed agricultural system in campos county (picazo-tadeo et al., 2011). the degree of eco-efficiency convergence in 22 oecd countries was analysed for the http://www.sciencedirect.com/science/article/pii/s0921800908001407#ref_bib39#ref_bib39 http://www.sciencedirect.com/science/article/pii/s0921800908001407#ref_bib39#ref_bib39 ijahp article: köne, büke/ eco-efficiency analysis using analytic hierarchy process approach international journal of the analytic hierarchy process 169 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.477 period 1980–2008. the researchers focused on three air pollutants namely, carbon dioxide, nitrogen oxides and sulphur oxides (camarero et al., 2013). the environment efficiency and resource problem of european countries has been evaluated by specifying a dea model (robaina-alves et al., 2015). masternak-janus, et.al, (2016) examined the concept of eco-efficiency at a regional level as an approach to promote the sustainable transformation of regions, using poland as an example and employing the data envelopment analysis. the industrial eco-efficiency of 30 provinces in china between 2005 to 2013 was measured using the three-stage data envelopment analysis model. according to the results obtained, industrial eco-efficiency is affected by regulation, technical innovation, economic level and industrial structure (zhang et al., 2017). 3. data and methodology 3.1. data individual air pollutants like sulphur dioxide (so2), nitrogen oxides (nox), ammonia (nh3), carbon monoxide (co), non-methane volatile organic compounds (nmvoc), methane (ch4), particulate matter (pm10) and carbon dioxide (co2) released into the atmosphere are not limited to the emission source. these materials only way of transportation over a long distance have a number of drawbacks on the environment and this results in that classification of the air pollutants causing environmental pressure. associated environmental pressure categories of the air pollutants and their conversion factors are presented in table 1 (houghton et al. 1996; de leeuw, 2002; eea, 2011). in this study, the individual air pollutants namely so2, nox, nh3, co, nmvoc, pm10 (eea, 2015a) ch4 and co2 equivalent (eea, 2015b) are obtained from the european environmental agency while gross domestic product using purchasing power parity (gdpppp) and population data are taken from the international energy agency (iea, 2014). after the individual air pollutants information is obtained, inventories were prepared, the environmental pressure categories of glwp, acp, tfp and pp were calculated by means of the conversion factors given in table 1. in this present work, per capita values for gdpppp, glwp, acp, tfp and pp were used in the eco-efficiency calculation in order to make good comparisons between small and high population countries. a summary of the descriptive statistics for the years 1990, 1995, 2000, 2005 and 2010 are given in table 2. ijahp article: köne, büke/ eco-efficiency analysis using analytic hierarchy process approach international journal of the analytic hierarchy process 170 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.477 table 1 individual air pollutants and their conversion factors for environmental pressure categories pollutant (mg) acp (mg of ap eq.) tfp (mg of tfp eq.) pp (mg of pp eq.) so2 0.03125 0.54000 nox 0.02174 1.22000 0.88000 nh3 0.05882 0.6400 co 0.11000 nmvoc 1.00000 ch4 0.014000 pm10 1.00000 ijahp article: köne, büke/ eco-efficiency analysis using analytic hierarchy process approach international journal of the analytic hierarchy process 171 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.477 table 2 descriptive statistics of the variables by years gdpppp (2005 usd per capita) glwp (g co2 eq. per capita) acp (g ap eq. per capita) tfp (g tfp eq. per capita) pp (g pp eq. per capita) 1990 mean 19426 12282476 3411 101865 113061 minimum 7524 3418618 1270 32565 32368 maximum 42691 33772383 7699 315545 1116244 1995 mean 19957 10661394 2594 83272 77021 minimum 6182 3996591 1072 34123 27017 maximum 48280 24823150 5804 226023 488143 2000 mean 23699 10400587 2155 69836 55722 minimum 6838 4211369 928 35263 23147 maximum 60993 22338666 7100 176857 166812 2005 mean 26500 10689907 1864 62659 50350 minimum 9362 4805016 833 33400 20107 maximum 68167 28101283 6953 204177 159597 2010 mean 27289 9954574 1599 50610 42551 minimum 10921 5353920 736 27028 17108 maximum 68537 24113306 9756 138256 200481 3.2. methodology for evaluating eco-efficiency values according to the definition of eco-efficiency (ee) in terms of gdpppp and environmental pressure categories, glwp, acp, tfp and pp can be written as (kuosmanen & kortelainen, 2005; kortelainen, 2008; camarero et al., 2013; camarero et al., 2014): 𝐸𝐸 = (𝐺𝐷𝑃𝑃𝑃𝑃) (𝑊𝐺𝑙𝑊𝑃 )(𝐺𝑙𝑊𝑃) + (𝑊𝐴𝐶𝑃 )(𝐴𝐶𝑃) + (𝑊𝑇𝐹𝑃 )(𝑇𝐹𝑃) + (𝑊𝑃𝑃)(𝑃𝑃) ⁄ (1) where glwp, acp, tfp and pp values are aggregates of the pressures exerted on the environment by the four pressure categories into a single environmental pressure score, whereas wglwp, wacp, wtfp and wpp are the weights assigned to glwp, acp, tfp and pp pressure categories. ijahp article: köne, büke/ eco-efficiency analysis using analytic hierarchy process approach international journal of the analytic hierarchy process 172 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.477 environmental pressure categories have different measurement units in equation 1. for this reason, normalization should be done to make the variables comparable. in this paper, the “distance to reference value normalization” method is applied because it is widely used in environmental problems (seppälä & hämäläinen, 2001; oecd, 2008; saling et al. 2002; plaia, and ruggieri, 2011; plaia, et al., 2013). descriptive statistics of the normalized values of variables are given in table 3. table 3 descriptive statistics of the normalized values of variables by years gdpppp glwp acp tfp pp 1990 mean 0.4550 3.5928 2.6856 3.1280 3.4930 minimum 0.1762 1.0000 1.0000 1.0000 1.0000 maximum 1.0000 9.8790 6.0622 9.6897 34.4860 1995 mean 0.4134 2.6676 2.4200 2.4403 2.8508 minimum 0.1280 1.0000 1.0000 1.0000 1.0000 maximum 1.0000 6.2111 5.4142 6.6238 18.0680 2000 mean 0.3885 2.4696 2.3222 1.9804 2.4073 minimum 0.1121 1.0000 1.0000 1.0000 1.0000 maximum 1.0000 5.3044 7.6509 5.0154 7.2066 2005 mean 0.3887 2.2247 2.2375 1.8760 2.5041 minimum 0.1373 1.0000 1.0000 1.0000 1.0000 maximum 1.0000 5.8483 8.3469 6.1131 7.9374 2010 mean 0.3982 1.8593 2.1726 1.8725 2.4872 minimum 0.1593 1.0000 1.0000 1.0000 1.0000 maximum 1.0000 4.5039 13.2554 5.1153 11.7186 ijahp article: köne, büke/ eco-efficiency analysis using analytic hierarchy process approach international journal of the analytic hierarchy process 173 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.477 after normalization is applied to equation 1, it can be written in the dimensionless form for a country i as follows: (𝐸𝐸)𝑖 = ( 𝐺𝐷𝑃𝑃𝑃𝑃𝑖 𝐺𝐷𝑃𝑃𝑃𝑃𝑚𝑎𝑥 ⁄ ) ((𝑊𝐺𝑙𝑊𝑃 ) ( 𝐺𝑙𝑊𝑃𝑖 𝐺𝑙𝑊𝑃𝑚𝑖𝑛 ⁄ ) + (𝑊𝐴𝐶𝑃 ) ( 𝐴𝐶𝑃𝑖 𝐴𝐶𝑃𝑚𝑖𝑛 ⁄ ) + (𝑊𝑇𝐹𝑃) ( 𝑇𝐹𝑃𝑖 𝑇𝐹𝑃𝑚𝑖𝑛 ⁄ ) + (𝑊𝑃𝐹𝑃 ) ( 𝑃𝑃𝑖 𝑃𝑃𝑚i̇𝑛 ⁄ )) ⁄ (2) where gdppppmax is the maximum reference value while glwpmin, acpmin, tfpmin and ppmin are minimum reference values of 31 countries for each study year. according to equation 2, relative ee of a country i within maximum gdpppp and minimum environmental pressure categories (glwp, acp, tfp and pp) is equal to 1.000. weights of the environmental pressure categories in equation 2 can be determined by some multi-attribute calculation models like the ahp methodology (zhou et al., 2006). in this paper, the weights of each environmental pressure category were determined using the ahp methodology. the ahp is an effective tool for dealing with the complexity inherent in the multicriteria decision-making problem, and can be modelled by dividing the problem at various levels so that it forms a hierarchy. the highest level of the hierarchy is the main objective of the decision problem. the lower levels are tangible and/or nonmaterial criteria and sub-criteria that contribute to the goal. the bottom level is formed by alternatives to be evaluated in terms of criteria. the pairwise comparisons are made with judgments using numerical values taken from the ahp absolute fundamental scale of 1-9 in each hierarchical level. these comparisons lead to dominance matrices in which ratio scales are derived in the form of the main eigenvectors. these matrices are positive and reciprocal ( jiij aa /1 ). the method also calculates the consistency ratio to confirm the consistency of the decisions, and the acceptability of these decisions should be about 0.10 or less (saaty, 1980; saaty & peniwati, 2007). a questionnaire was prepared using paired questions for the comparison analysis to determine the weight of the indicators with the ahp method (aragonés-beltrán et al., 2015). an example of an ahp questionnaire for pairwise comparison of indicators according to the ahp absolute fundamental scale of 1-9 is presented in table 4 (saaty, 1980). twenty-five experts responded to the questionnaires. all of the experts are academicians in different environmental engineering departments of 9 universities in turkey. the personal decisions of the experts were combined to produce only one value for the priorities of the aims; this is evaluated by deducting the geometric mean of those decisions (saaty & peniwati, 2007). the four evaluation criteria are grouped into a cluster; glwp, acp, tfp and pp criterions consider global warming, acidification, tropospheric ozone forming and particulate formation potentials respectively (houghton et al. 1996; de leeuw, 2002; kortelainen, 2008). the ahp decision model for weights of the environmental pressure categories is shown in figure 1. ijahp article: köne, büke/ eco-efficiency analysis using analytic hierarchy process approach international journal of the analytic hierarchy process 174 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.477 table 4 a ahp questionnaire for pairwise comparison of indicators from your point of view, which indicator is more important according to their relative degrees of importance of the air quality for global scale. cr1: glwp cr2: acp which indicator do you consider more important? cr1 ○ cr2 ○ in which degree? 1 3 5 7 9 figure 1. the ahp decision method the super decisions software v.2.2.3 was used to calculate the weights of the environmental pressure categories. the pairwise comparison matrix values obtained from the expert’s judgements were input into this program. there is less than a 0.10 inconsistency ratio between the comparison matrix of pairs. therefore, the pairwise comparison matrix is consistent. the calculated weights of the environmental pressure categories are given figure 2. the highest weight factor is calculated glwp (see figure 2). ijahp article: köne, büke/ eco-efficiency analysis using analytic hierarchy process approach international journal of the analytic hierarchy process 175 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.477 figure 2. weights calculated by using ahp method 4. results and discussion the calculated relative ee scores of the 31 eea member countries for the period 1990-2011 are presented in figure 3. as seen in figure 3, the average eco-efficiency scores increase over the time period 1990-2011. this seems reasonable if one remembers that environmental regulations aimed at reducing pollutant emissions have been in force the longest and in most cases are the most restrictive. the ecoefficiency scores of at, be, fr, de, it, nl, no, se and ch are higher than the average eco-efficiency scores over the time period 1990-2011. conversely, the ecoefficiency scores of bg, cz, ee, fi, hu, is, lv, lt, pl, ro, sk and si are less than the average eco-efficiency scores over the time period 1990-2011. it is also interesting to highlight that the eco-efficiency scores in countries such as hr, is and tr are lower in 2011 than in 1990. in contrast, the largest improvements correspond to cz, lu and sk (see figure 3). figure 3. eco-efficiency scores in eea member countries 0.491 0.145 0.230 0.134 0.000 0.100 0.200 0.300 0.400 0.500 0.600 glwp acp tfp pp weights c ri te ri a 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 0.55 a t b e b g h r c y c z d k e e f i f r d e h u is ie it l v l t l u m t n l n o p l p t r o s k s i e s s e c h t r g b m e a n e c o -e ff ic ie n c y country 1990 2000 2011 ijahp article: köne, büke/ eco-efficiency analysis using analytic hierarchy process approach international journal of the analytic hierarchy process 176 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.477 as seen in figure 4, ch and se are ranked first and second place respectively in both 1990 and 2011, while bg and ee are ranked in the last two places respectively in 1990 and 2011. the economic and environmental conditions of ch and se in the study period continue to change in a positive direction, while the economic and environmental conditions of bg and ee in study period continue to change in a negative direction. this indicates a marked change in the economic and environmental efficiency trend. figure 4. eco-efficiency/gdppp ranking of the countries by years although there seems to be a positive linear relationship that exists between calculated eco-efficiency scores and income, a high value gdpppp per capita does not automatically show a high eco-efficiency score, or vice versa. for example, in 1990 and 2011, lu had the highest gdppp per capita values and was ranked 25th 0 5 10 15 20 25 30 35 a t b e b g h r c y c z d k e e f i f r d e h u is ie it l v l t l u m t n l n o p l p t r o s k s i e s s e c h t r g b r a n k in g country 1990 eco-efficiency gdpppp 0 5 10 15 20 25 30 35 a t b e b g h r c y c z d k e e f i f r d e h u is ie it l v l t l u m t n l n o p l p t r o s k s i e s s e c h t r g b r a n k in g country 2011 eco-efficiency gdpppp ijahp article: köne, büke/ eco-efficiency analysis using analytic hierarchy process approach international journal of the analytic hierarchy process 177 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.477 and 16th in the relative eco-efficiency comparison. on the other hand, gb's relative eco-efficiency score in 2011 ranked 3rd, although it was only the 11th largest value for gdppp per capita (see figure 4). similar studies are found in the literature based on the dea (kortelainen, 2008; camarero et al., 2013; camarero et al., 2014). on the other hand, the dea does not provide any prior information about weights of different environmental pressure categories. this is one of the important weaknesses of the dea. in most cases, optimum dea weights can have unreasonable values; large weights can be assigned to environmental impacts with secondary presence and in general negligible or zero weights can be left for significant adverse effects. as a result, some activities may seem effective, although they only perform well on a single, relatively insignificant criterion. for these reasons, although the dea is not tied to normative evaluation, additional information on the relative importance of different environmental influences can easily be added to this metric by applying soft weight constraints (pedraja-chaparro, et al., 1997; allen & thanassoulis, 2004; kuosmanen & kortelainen, 2005). in the opinion of a decision maker, the question still remains: what is the significance of different environmental impact categories compared to each other? in practice, responding to this question requires data on weights between different environmental impact categories. the environmental pressure categories weights that should be specified in the eco-efficiency calculation are often taken to reflect public preferences for environmental issues and can be obtained by integrating expert investigations into multi-feature assessment models such as ahp (zhou et al., 2006; saaty & peniwati, 2007; köne & büke, 2014). in this study, the environmental pressure categories weights are derived by integrating the expert surveys using the ahp to calculate eco-efficiency scores. this is proposed because the weighting scheme proposed in the dea models do not require any prior information concerning weights of the different environmental pressure categories. 5. policy implications as a result of this paper, several policies can be proposed to increase the ecoefficiency scores of the eea member countries. (1) in order for eco-inefficient countries to approach leading eea countries, they should move towards cleaner production processes, such as increasing environmental awareness and also changing their respective production structures towards less contaminating activities. (2) the country based environmental efficiency performance seems to be a good combination of reducing the density of fossil fuels with improved average productivity of the capital and replacing a significant amount of fossil fuels with energy and renewable energy sources. nuclear and renewable energy sources already contribute to reducing air pollutant emissions. (3) as of february 2017, the number of nuclear power plants operating and under construction in the world are 449 and 60 respectively (iaea, 2017). advanced designs for all types of nuclear reactors will be developed. therefore, the use of nuclear energy will be safer in the future (iaea, 2017a). increasing the proportion of nuclear energy in the total primary energy supply will depend on the social acceptability of nuclear reactors. for this reason, the public should be educated about the radiological risks of nuclear reactors. ijahp article: köne, büke/ eco-efficiency analysis using analytic hierarchy process approach international journal of the analytic hierarchy process 178 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.477 (4) eco-efficiency is related to features like production, the trade sector and the flow of foreign direct investment. in this sense, some countries may decide not to produce goods that pollute the environment in their own right, but measures should be taken to restrict the importability of these goods from other countries. (5) existing taxes should be rearranged to include negative external costs in energy prices. (6) people should be motivated to be aware of environmental issues and rational use of energy to increase environmental efficiency. 6. conclusions assessing eco-efficiency can be considered a new tool to provide sound information to support policy maker’s decisions that will contribute to long-term sustainable development. accordingly, in recent years, research has emerged to evaluate ecoefficiency across firms, industries or economies. working from the definition of ecoefficiency, the eco-efficiency scores of the 31 member countries of the eea were evaluated for four environmental pressure categories for the period 1990-2011. these environmental pressure categories were weighted by applying the ahp methodology. the main policy suggestion of this study is the need to make stricter regulations on air pollution emissions in developed countries, especially in countries where the environmental eco-efficiency level is lower. moreover, although environmental regulations and intergovernmental agreements to date have focused heavily on co2 emissions, future regulations and agreements should take care to reduce nox, so2 and pm10 emissions, which have a significant negative impact on eco-efficiency in all countries. finally, this study has relatively evaluated eco-efficiency comparatively in the eea countries. however, there is a great potential for collecting a large amount of information about this topic of interest and designing better environmental policies for policy makers. in this context, it would be a useful approach and very important to determine the criteria and especially the criteria weights for evaluating the ecoefficiency scores of countries. ijahp article: köne, büke/ eco-efficiency analysis using analytic hierarchy process approach international journal of the analytic hierarchy process 179 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.477 references allen r., thanassoulis, e. (2004). improving envelopment in data envelopment analysis. european journal of operational research, 154, 363–379. doi: https://doi.org/10.1016/s0377-2217(03)00175-9 aragonés-beltrán, p., garcía-melón, m., estruch-guitart, v. (2015). analysis of the participation of stakeholders in environmental management based on anp: application to a spanish natural park, international journal of the analytic hierarchy process, 7(1), 2-14. doi: http://dx.doi.org/10.13033/ijahp.v7i1.276 camarero, m., castillo, j., picazo-tadeo, a. j., tamarit, c. (2013). eco-efficiency and convergence in oecd countries. environmental resource economics, 55, 87– 106. doi: 10.1007/s10640-012-9616-9 camarero, m., castillo-giménez, j., picazo-tadeo, a. j., tamarit, c. (2014). is ecoefficiency in greenhouse gas emissions converging among european union countries? empirical economics, 47, 143–168. doi: 10.1007/s00181-013-0734-1 de leeuw, f.a.a.m. (2002). a set of emission indicators for long-range transboundary air pollution. environmental science & policy, 5, 135–145. doi: https://doi.org/10.1016/s1462-9011(01)00042-9 eea, (2011). european environment agency, individual pollutants and environmental pressure categories. www.eea.europa.eu/data-and-maps/data/eea.../filecached (accessed 16.04.13). eea, (2015a). european environment agency, air pollutant emissions data viewer (lrtap convention) http://www.eea.europa.eu/data-and-maps/data/data-viewers/air-emissions-viewerlrtap (accessed 22.01.15). eea, (2015b). european environment agency, greenhouse gas-data viewer http://www.eea.europa.eu/data-and-maps/data/data-viewers/greenhouse-gases-viewer (accessed 22.01.15). fet, a., m. (2003). eco-efficiency reporting exemplified by case studies. clean technologies and environmental policy, 5, 232–239. doi: 10.1007/978-3-66210270-1_23 figge, f., hahn, t. (2004). sustainable value added-measuring corporate contributions to sustainability beyond eco-efficiency. ecological economics, 48, 173– 187. doi: https://doi.org/10.1016/j.ecolecon.2003.08.005 holm, s., englund, g. (2009). increased eco-efficiency and gross rebound effect: evidence from usa and six european countries 1960–2002. ecological economics, 68, 879–887. doi: https://doi.org/10.1016/j.ecolecon.2008.07.006 houghton, j.t., meiro filho, l.g., callander, b.a., harris, n., kattenburg, a., maskell, k. (1996). climate change 1995: the science of climate change. contribution of working group i to the 2nd assessment report of the https://doi.org/10.1016/s0377-2217(03)00175-9 https://doi.org/10.1016/s1462-9011(01)00042-9 http://www.eea.europa.eu/data-and-maps/data/eea.../filecached http://www.eea.europa.eu/data-and-maps/data/data-viewers/air-emissions-viewer-lrtap http://www.eea.europa.eu/data-and-maps/data/data-viewers/air-emissions-viewer-lrtap http://www.eea.europa.eu/data-and-maps/data/data-viewers/greenhouse-gases-viewer https://doi.org/10.1016/j.ecolecon.2003.08.005 https://doi.org/10.1016/j.ecolecon.2008.07.006 ijahp article: köne, büke/ eco-efficiency analysis using analytic hierarchy process approach international journal of the analytic hierarchy process 180 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.477 intergovernmental panel on climate change. cambridge university press, cambridge. huppes, g. (2007). realistic eco-efficiency analysis why we need better ecoefficiency analysis from technological optimism to realism. technikfolgenabschätzung, 3, 38-45. doi: 10.1007/1-4020-5399-1_1 huppes, g., ishikawa, m. (2005). a framework for quantified eco-efficiency analysis. journal of industrial ecology, 9, 25-41. doi: 10.1162/10881980577524788 iaea, (2017). power reactor information system, https://www.iaea.org/pris/ (accessed 02.02.17). iaea, (2017a). nuclear security: new directions for 21st century https://www.iaea.org/newscenter/focus/nuclear-security-new-directions-21st-century (accessed 02.02.17). iea, (2014). international energy agency, co2 emissions from fuel combustion: annual historical series (1971–2012). oecd/iea, paris. korhonen, p.j., luptacik m. (2004). eco-efficiency analysis of power plants: an extension of data envelopment analysis. european journal of operational research, 154, 437–446. doi: https://doi.org/10.1016/s0377-2217(03)00180-2 kortelainen, m. (2008). dynamic environmental performance analysis: a malmquist index approach. ecological economics, 64, 701-715. doi: https://doi.org/10.1016/j.ecolecon.2007.08.001 köne, a.ç., büke t. (2014). the evaluation of the air pollution index in turkey. ecological indicators, 45 350-354. doi: https://doi.org/10.1016/j.ecolind.2014.04.032 kuosmanen, t., kortelainen m. (2005). measuring eco-efficiency of production with data envelopment analysis. journal of industrial ecology, 9, 59-72. doi: 10.1162/108819805775247846 masternak-janus, a., rybaczewska-błażejowska, m. (2016). comprehensive regional eco-efficiency analysis based on data envelopment analysis: the case of polish regions. journal of industrial ecology, 21, 180-190. doi: 10.1111/jiec.12393 michelsen, o., fet, a.m., dahlsrud, a. (2006). eco-efficiency in extended supply chains: a case study of furniture production. journal of environmental management, 79, 290–297. doi: https://doi.org/10.1016/j.jenvman.2005.07.007 mickwitz, p., melanen, m., rosenström, u, seppälä, j. (2006). regional ecoefficiency indicators: a participatory approach. journal of cleaner production, 14, 1603–1611. doi: https://doi.org/10.1016/j.jclepro.2005.05.025 oecd, (2008). handbook on constructing composite indicators, methodology and user guide. oecd publication, ispra, italy. pedraja-chaparro, f., salinas-jimenez, j., smith, p. (1997). on the role of weight restrictions in data envelopment analysis. journal of productivity analysis, 8, 215– 230. doi: 10.1023/a:1007715912664 https://doi.org/10.1016/s0377-2217(03)00180-2 https://doi.org/10.1016/j.ecolecon.2007.08.001 https://doi.org/10.1016/j.ecolind.2014.04.032 https://doi.org/10.1016/j.jenvman.2005.07.007 https://doi.org/10.1016/j.jclepro.2005.05.025 ijahp article: köne, büke/ eco-efficiency analysis using analytic hierarchy process approach international journal of the analytic hierarchy process 181 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.477 picazo-tadeo, a.j., gómez-limón, j.a., reig-martínez, e. (2011). assessing farming eco-efficiency: a data envelopment analysis approach. journal of environmental management, 92, 1154-1164. doi: https://doi.org/10.1016/j.jenvman.2010.11.025 plaia, a.; di salvo, f.; ruggieri, m.; agró, g. (2013). a multisite-multipollutant air quality index. atmospheric environment,70, 387–391. doi: https://doi.org/10.1016/j.atmosenv.2013.01.028 plaia, a.; ruggieri, m. (2011). air quality indices: a review. reviews in environmental science biotechnology, 10, 165–179. doi: 10.1007/s11157-0109227-2 robaina-alves, m., moutinho, v., macedo, p. (2015). a new frontier approach to model the eco-efficiency in european countries. journal of cleaner production, 103, 562-573. doi: https://doi.org/10.1016/j.jclepro.2015.01.038 saaty t.l. (1980). the analytic hierarchy process. pittsburgh: rws publications. saaty t.l, peniwati k. (2007). group decision making: drawing out and reconciling differences. pittsburgh: rws publications. saling, p., kicherer, a., dittrich-krämer, b., wittlinger, r., zombik, w., schmidt, i., schrott, w., schmidt, s. (2002). eco-efficiency analysis by basf: the method. international journal of lca, 7, 203-218. doi:10.1007/bf02978875 seppälä j., hämäläinen r.p. (2001). on the meaning of the distance-to target weighting method and normalisation in life cycle impact assessment. international journal of lca, 6 211–218. doi: 10.1007/bf02979376 seppälä, j., melanen, m., maenpaa, i., koskela, s., tenhunen, j., hiltunen m.r. (2005). how can the eco-efficiency of a region be measured and monitored? journal of industrial ecology, 9, 117–130. doi: 10.1162/108819805775247972 verfaille, h., a., bidwell, r. (2000). measuring eco-efficiency, a guide to reporting company performance. world business council for sustainable development, geneva. wursthorn, s., poganietz, w.r., schebek, l. (2010). economic–environmental monitoring indicators for european countries: a disaggregated sector-based approach for monitoring eco-efficiency. ecological economics 70, 487–496. doi: https://doi.org/10.1016/j.ecolecon.2010.09.033 zhang b., bi j., fan z., yuan z., ge j. (2008). eco-efficiency analysis of industrial system in china: a data envelopment analysis approach. ecological economics, 68, 306–316. doi: https://doi.org/10.1016/j.ecolecon.2008.03.009 zhang, j., liu, y., chang, y., zhang, l. (2017). industrial eco-efficiency in china: a provincial quantification using three-stage data envelopment analysis. journal of cleaner production, 143, 238-249. doi: https://doi.org/10.1016/j.jclepro.2016.12.123 https://doi.org/10.1016/j.jenvman.2010.11.025 https://doi.org/10.1016/j.atmosenv.2013.01.028 https://doi.org/10.1016/j.jclepro.2015.01.038 http://www.sciencedirect.com/science/article/pii/s0921800908001407#ref_bib39#ref_bib39 https://doi.org/10.1016/j.ecolecon.2010.09.033 https://doi.org/10.1016/j.ecolecon.2008.03.009 http://www.sciencedirect.com/science/article/pii/s0959652616321862 http://www.sciencedirect.com/science/article/pii/s0959652616321862 https://doi.org/10.1016/j.jclepro.2016.12.123 ijahp article: köne, büke/ eco-efficiency analysis using analytic hierarchy process approach international journal of the analytic hierarchy process 182 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.477 zhou, z.; ang, b.w.; poh, k.l. (2006). comparing aggregating methods for constructing the composite environmental index: an objective measure. ecological economics, 59, 305–311. doi: https://doi.org/10.1016/j.ecolecon.2005.10.018 https://doi.org/10.1016/j.ecolecon.2005.10.018 ijahp article: öneren, arar, çelebioglu/ determining the overall stress factors for policemen by ahp method international journal of the analytic hierarchy process 462 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.384 determining the overall stress factors for policemen by ahp method melahat öneren asst. prof., dept. of business administration, kırıkkale university, turkey melahatoneren@mynet.com tayfun arar r.a., dept. of business administration, kırıkkale university, turkey tayfunarar@kku.edu.tr eda sultan çelebioglu master’s student, dept. of business administration, kırıkkale university, turkey celebioglu@mynet.com abstract the country of turkey has seen such mess since “occupy gezi park” in 2013. this mess has included events such as terrorism attacks, suicide bomber explosions in crowded public places, war conditions both inside the country and at the borders, protests against the government and meetings that the authorities deem illegal. in almost all these situations involving disturbances, the main responsibility falls to the policemen. it is important to remember that policemen are human beings and their jobs and responsibilities place huge loads on their shoulders. while the public expects them to do their job, which is to provide public safety, one must also consider the factors that negatively affect them and make their job stressful. this research aims to order all of the stress factors of policemen hierarchically using the analytic hierarchy process (ahp) method, and then suggest solutions to cope with these stressors. according to the results, “danger level” is the greatest stressor, and “age” is the least stressor. keywords: police; stressors; analytic hierarchy process (ahp) 1. introduction throughout the literature, though there are many definitions of what stress is, because the term is multifaceted and used in many fields of science, there is still ambiguity about and little consensus on its definition (karakaya, 2002). however, according to common characteristics of available definitions, stress is defined as a perception of a person that occurs in both the psychological and physical state when the environmental demands exceed what the person is capable of, and results in mostly negative outcomes (michie, 2002; kendall & muenchberger, 2009). the reaction to these kinds of situations by an ijahp article: öneren, arar, çelebioglu/ determining the overall stress factors for policemen by ahp method international journal of the analytic hierarchy process 463 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.384 individual is because of their wish to adapt to the circumstance (tutar, 2004). this reaction is as normal as breathing, and stress may occur anytime and anywhere in life even during routine things at home, or on the street while waiting for the bus or at work. work-related stress happens when the demands from the workplace and the abilities of the employee required to cope with them do not match (blaug et al., 2007; mkumbo, 2014). this work-related stress could be acute or sudden, post traumatic (which is mostly seen in policemen) or chronic (lasts the longest time) (kendall et al., 2000). every profession has situations which create an environment where stress can occur, but professions which require dealing with people like teachers and academicians have more possibilities. in one study involving 157 academicians in sweden, stress factors were researched and time limits were found to be the most affective stressor (lindfors et al., 2009). another study about stressors involving 326 academicians in tanzania showed lack of staff involvement in institutional reform, lack of necessary support systems related to their work and high workload levels were the most important stress factors (mkumbo, 2014). dangerous situations that involve dealing with people, such as firemen and policemen, play a significant and important role in increasing the possibility of stressors. on the one hand, the literature states that the job of a police is not stressful; on the other hand, this occupation has been cited as one the most stressful occupations because of dangers and encounters with violence as mentioned above (berg et al., 2005; violanti & aron, 1995; kuo, 2015; newman and rucker-reed, 2004; storch & panzarella, 1996). these common reasons for the stress of a policemen’s job are magnified by the situation in turkey, and this is an important reason why policing has been chosen as a research area in this study. the security situation of the country cannot be labeled as ‘good’ due to civil war and border safety issues in the south east, terrorism bomb suicide attacks in the large cities and precarious encounters in meetings against government authorities. these situations are a result of the geopolitical situation of turkey which is the bridge between asia, europe and africa, and is also a host for the flow of refugees coming from syria. all of these conditions create an extra workload for the policemen in addition to their routine work such as paperwork, protecting laws and providing security and layout of the civil public. in a conference speech given to policemen in turkey in 2000, the speaker said, “you are all aware of the physical danger of the occupation, but you were not all aware of the psychological danger of this work in the form of stress”; this kind of profession requires psychological bearing as well as physical durability (birkök, 2000). the topic of stress is important because it has both positive and negative effects on people (luthans, 1992). positive stress, called eustress, may result in creativity and productivity, but rarely happens unless there is a controlled limit of stress (kendall & muenchberger, 2009). internationally, an increasing cost trend is seen among industrial countries such as turkey because of work-related stress. this is because when stress is present job performance decreases, and the more stress an employee feels, the more mistakes he/she makes which then results in workplace injuries (kendall et al., 2000). thus, the stress factors or stressors on policemen need to be analyzed and some precautions taken. in the literature, there is research which investigated stressors on policemen. in one study involving 103 police officers in the u.s. for stressors, killing someone in the line of duty ranked as the highest stressor (violanti & aron, 1995). in another study involving 79 police officers in the u.s., administrative matters and relationship with non-police have ijahp article: öneren, arar, çelebioglu/ determining the overall stress factors for policemen by ahp method international journal of the analytic hierarchy process 464 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.384 been found as primary stressors (storch & panzarella, 1996). following this, another study was performed with deputies in the same organization in the u.s., but different factors were found as the highest stressors such as illnesses related with job, injury on duty and closeness to retirement (newman & rucker-reed, 2004). similar studies were done in the european union. the first, took place in norway with 3272 police officers and like the previous research, work injuries was the primary stress factor (berg et al., 2005). in france, a survey with 617 policemen, found surprisingly that not job related but personal characteristics was the greatest stressor (deschamsp et al., 2003). all of those studies mentioned above were performed in developed western countries, which are not like turkey with its war conditions that require more security forces than ever. also, all those studies investigated mostly work-related factors, but in this research overall stressors are examined. the method this study uses is another difference. all other studies involving this topic used questionnaires, but in this research, to be more realistic and rational, and in order to find pioneering solutions based on the results, and in the light of the purpose of this study, the stressors were first grouped as private life related, job related and organizational and then hierarchically ordered by the ahp method. 2. method in this research, the aim is to gather all significant stress factors, not only job related ones, but also organizational and personal factors, through a literature review on policemen. there are three main factors, and they are job related, private life related and organizational and each of them has its own sub-factors. the job related sub-factors are workload, unbalanced work shifts, job characteristics and danger level. within the organizational factors, the sub-factors are poor interpersonal communication, the role in the organization, career development problems, organizational structure and organizational climate and culture. finally, in the private life related factors, the subfactors are age, family problems, personal characteristics, socio-economic conditions and physical & psychological health conditions. in order to accomplish the goal of the study, the factors were first ordered hierarchically by the analytic hierarchy process so as to determine which factor or factors are primarily affecting policemen due to stress and then suggest solutions according to these results. 2.1 job related factors the factors caused by the nature of being a policemen are listed below as gathered from the literature. 2.1.1 workload if the amount of work to do is more than is possible, which can especially happen in extreme situations, this may cause stress. the physical and mental labor and effort that is required in the work may be an overcapacity for the employee (eren, 2014). the feeling of stress at work as a result of an overloading of duties which require more caution because of dealing with criminals, violence and providing the safety of innocent civilians can cause severe results. ijahp article: öneren, arar, çelebioglu/ determining the overall stress factors for policemen by ahp method international journal of the analytic hierarchy process 465 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.384 2.1.2 unbalanced work shifts sometimes rotating night shifts may lead to inadequate sleep for policemen and this decreases their resistance to stress physiologically (violanti & aron, 1995). insufficient sleep may also cause the loss of focus which results in accidents, decreasing the quality and efficiency of the work and worse health conditions all of which causes stress (eren, 2014). there is also the issue of perception of fairness in assigning the rotating shifts that may affect the employee badly, and dwelling on this issue can create a stressful environment. 2.1.3 job characteristics the natural work activities of this occupation include periodic risks, threat, and violence and the ever present reality of using a gun which causes injuries and even death (kuo, 2015). dealing with frequent injuries can make a person have a negative view of life (violanti & aron, 1995). high voltage jobs, monotony, insufficient physical conditions, and death in the line of duty are also job characteristics (tutar, 2004). characteristics of the job of a policemen also include routine paperwork which may cause boredom and make police officers feel useless and be stressful (violanti & aron, 1995). 2.1.4 danger level the danger mentioned here may change from one situation to another. it may occur when stopping a vehicle or entering a building, chasing after a thief with a gun, providing border security, standing against an angry group of people who are in a meeting, or assaulting a suicide bomber’s house (violanti & aron, 1995). 2.2 organizational factors police officers mostly perceive the organization as unsupportive and not responsive to their needs (violanti & aron, 1995). the factors caused by the organization of police headquarters, though showing similar characteristics which may change from one organization to another, are listed below as gathered from the literature. 2.2.1 poor interpersonal communication people understand each other by communication. in organizations, for the work to be done effectively, human interaction is needed. there are times that a person cannot do the job alone and team work may be required. quality relations and effective communications with supervisors and colleagues are two important aspects in the police profession which require teamwork. difficulty working with superiors, inadequate support from supervisors and poorly motivated co-workers may cause stress (berg et al., 2005). while managers who are always critical and non-supportive create stress, a positive social environment and good team work can reduce it (michie, 2002). 2.2.2 role in organization the role of an employee in the organization is important for his or her productivity and efficiency. when there is ambiguity or conflict about the role, this situation may increase the stress level. an employee feels role ambiguity when he or she is not sure about the objectives, responsibilities and expectations of their job (kuo, 2015). role conflicts may create expectations which are hard to satisfy (robbins & judge, 2009). ijahp article: öneren, arar, çelebioglu/ determining the overall stress factors for policemen by ahp method international journal of the analytic hierarchy process 466 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.384 2.2.3 career development problems career development problems include over promotion, under promotion, absence of job security and lack of training (kuo, 2015; michie, 2002). 2.2.4 organizational structure according to the research, bureaucratic structures where a hierarchy exists cause more stress (kuo, 2015). an organization where the manager supports a hierarchy and a chain of order system rather than participative management causes stress in the worker (tutar, 2004). 2.2.5 organizational culture & climate globalization and rapid development of information technology has caused competition within the organization to increase (kendall et al., 2000). this creates a demanding and productive climate in the organization which causes stress on employees. this competition is not as strong in the private sector police as it is in the public sector. an organizational climate in which an atmosphere of friendliness rather than boss-employee separation exists reduces stress levels. an organizational culture which involves all employees no matter their position in the decision making process reduces stress (michie, 2002). on the other hand, the lack of organizational culture, common belief and emotions causes organizational success, decreasing motivation and stress (eren, 2014). 2.3 private life related factors the weight or importance of sub-factors in this section certainly depends on the person. however, we have listed five common sub-factors gathered from the literature. 2.3.1 family problems family members unsupportive about the job, parenting issues, illness of a family member, and long-term strained relations with the spouse are counted as family problems (kuo, 2015; luthans, 1992). though, one who claims he or she is a professional should leave the private life problems at home and vice versa for work problems. 2.3.2 personal characteristics there are some specific personal characteristics such as negative affectivity (pessimism), cognitive distortions, negative thinking patterns and a lack of psychological hardiness that can affect a vulnerable person’s level of perception of things and cause the job to be more stressful than it actually is (kendall et al., 2000). a person who is assertive and good at problem solving and time management is more successful at lowering stress levels (michie, 2002). 2.3.3 socio-economic conditions a rational person would like to work more to maintain his or her family members economic and social life well (eren, 2014). sometimes, for various reasons, no matter how much a person tries, their efforts may not be enough to make a comfortable life for him or her and that can be stressful. ijahp article: öneren, arar, çelebioglu/ determining the overall stress factors for policemen by ahp method international journal of the analytic hierarchy process 467 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.384 2.3.4 age the younger the person is, the less experienced she or he is. in an occupation like police work, the more experience a person has the fewer mistakes which may cause injuries, or even death, they are likely to make. as a person gets older, both in his private and professional life, a common event called middle-age syndrome appears which results in psychological worries and stress. finally, the older a person is the more they think about the future which is another cause of stress. workers in jobs like the police force have a tendency to feel useless after retirement, and begin worrying about this even while still working. 2.3.5 physical & psychological health conditions police work is a hard job that not everyone can cope with. police need to be healthy both physically and psychologically. any vulnerability or tendency toward an illness such as anxiety and depression could distract the focus of the officer from his or her job and result in stress. the overall stress factors for policemen are shown in figure 1 below. ijahp article: öneren, arar, çelebioglu/ determining the overall stress factors for policemen by ahp method international journal of the analytic hierarchy process 468 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.384 figure 1. overall stress factors for policemen 2.4 analytic hierarchy process (ahp) the analytic hierarchy process was developed by thomas l. saaty in 1977 and used for multi criteria decision making with the advantage of being used in both qualitative and quantitative variables at the same time (dong & cooper, 2016). there are four axioms in the method (saaty, 1991). axiom 1 (opposite existence): in the ahp method, when two criterion are compared such as criterion i th and criterion j th , it is shown as 𝑎𝑖𝑗 , and the vice versa demonstration is 𝑎𝑗𝑖 and is calculated as 1 𝑎𝑖𝑗 . in ahp, there is a fundamental scale used that is shown in table 1. job related (c1) workload(c11) unbalanced work shifts (c12) job characteristics (c13) danger level (c14) organizational (c2) poor interpersonal communication (c21) role in organization (c22) career development problems (c23) organizational structure (c24) organizational culture & climate (c25) personal (c3) family problems (c31) personal characteristics (c32) socio-economi conditions (c33) age (c34) physical & psychological health conditions (c35) overall stress factors for policemen ijahp article: öneren, arar, çelebioglu/ determining the overall stress factors for policemen by ahp method international journal of the analytic hierarchy process 469 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.384 table 1 ahp fundamental scale intensity of importance definition explanation 1 equal importance if i th and j th criterion is equally important 3 moderate importance if i th slightly important than j th criterion 5 strong importance if i th strongly important than j th criterion 7 very strong importance if i th much strongly important than j th criterion 9 absolute strong importance if i th absolute strongly important than j th criterion 2,4,6,8 moderate values if the decision maker is irresolute (saaty and vargas, 2012) axiom 2 (homogeneity): in the ahp method, the objects being compared should have similar characteristics within the boundary of the scale developed by saaty from 1 to 9 as seen in table 1. axiom 3 (independence): in the ahp method, there is no relationship and dependence among the criteria as an assumption. axiom 4 (expectancy): in the final ahp table, all criteria and alternatives in the problem are ordered in a hierarchy. there are some steps that must be followed in this method (supçiller & çarpraz, 2011). step 1: build the hierarchical structure from top to bottom; goal, criteria and alternatives respectively. figure 2. hierarchical structure goal alternative 1 alternative 2 alternative 3 criterion 1 criterion 2 ijahp article: öneren, arar, çelebioglu/ determining the overall stress factors for policemen by ahp method international journal of the analytic hierarchy process 470 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.384 step 2: create a pairwise comparison matrix of the main criteria and sub criteria respectively (i= 1, 2,… n; j= 1, 2,.., n) table 2 basic pairwise comparison matrix of criteria (aij). criteria i j i 1 𝑎𝑖𝑗 j 1 𝑎𝑗𝑖 1 step 3: find the priority vector (pv) of each sub and main criterion (crawford &williams, 1985). 𝑤𝑖= ∏ 𝑎 𝑖𝑗 1/𝑛𝑛 𝑗=1 ∑(∏ 𝑎 𝑖𝑗 1/𝑛 )𝑛𝑗=1 (1) step 4: examine the consistency ratio (cr) of priority vectors (dong & cooper, 2016). lmax= ∑ d w𝑖 n i=1 n = (cm-consistency measure) (2) consistency index (𝐶𝐼𝐴) = lmax−n n−1 (3) cra = cia rin∗ (4) random index* table 3 random index numbers n of c 1 2 3 4 5 6 7 8 9 ri 0 0 0,52 0,89 1,11 1,25 1,35 1,40 1,45 (saaty and vargas, 2012) step 5: determine the final global weights of each criterion. in this research, the aim is to determine and hierarchically order overall stress factors for policemen working in turkey. all the criteria are provided by the literature review. to find the weights of each criterion and order them hierarchically, the analytic hierarchy process (ahp) was chosen. the scoring process was performed by an online interview with two chiefs of police and two commissars who carry out work somewhere close to the syrian border of turkey where war conditions are mainly dominant. each participant provided pairwise criterion comparison judgements which were aggregated, using the ijahp article: öneren, arar, çelebioglu/ determining the overall stress factors for policemen by ahp method international journal of the analytic hierarchy process 471 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.384 geometric mean in each case, into a final (aggregate) pairwise comparison matrix shown in tables 4, 5, 6 and 7. 3. results first, pairwise comparison matrices for each comparison of criteria that belong to either job (c1), organization (c2), private life (c3) or main criteria were obtained through the online interview. then, the geometric mean of each participant’s answers for each criteria group was obtained. table 4 pairwise comparison matrix of criteria related with job (c1) overall stress factors for policemen workload (c11) unbalanced work shifts (c12) job characteristics (c13) danger level (c14) pv workload (c11) 1,00 3,00 5,00 0,20 0,22 unbalanced work shifts (c12) 0,33 1,00 5,00 0,18 0,12 job characteristics (c13) 0,20 0,20 1,00 0,14 0,05 danger level (c14) 5,00 5,44 7,00 1,00 0,61 cr=0,1 in table 4, the priority vectors (weights) of sub criteria including workload (c11), unbalanced work shifts (c12), job characteristics (c13) and danger level (c14) of the main criterion job (c1) were found by using equation 1. in this criterion section, the most weighted criterion was danger level with a priority vector of 0,61. consistency indexes for each criterion were obtained using equations 2 and 3. the consistency ratio was found as 0,1 using equation (4). ijahp article: öneren, arar, çelebioglu/ determining the overall stress factors for policemen by ahp method international journal of the analytic hierarchy process 472 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.384 table 5 pairwise comparison matrix of criteria related with organization (c2) overall stress factors for policemen poor interpersonal communicati on (c21 role in organizati on (c22) career developme nt prob. (c23) organization al structure (c24) organization al climate & culture (c25) pv poor interpersonal communicati on (c21) 1,00 0,14 0,14 0,13 0,20 0,03 5 role in organization (c22) 7,00 1,00 1,41 1,00 2,71 0,30 career development prob. (c23) 7,00 0,71 1,00 2,14 2,14 0,29 organizationa l structure (c24) 7,45 1,00 0,47 1,00 3,95 0,26 5 organizationa l climate & culture (c25) 5,00 0,37 0,47 0,19 1,00 0,11 cr=0,05 in table 5, the priority vectors (weights) of sub criteria including poor interpersonal communication (c21), role in organization (c22), career development problems (c23), organizational structure (c24) and organizational climate and culture (c25) of the main criterion organization (c2) were found by using equation 1. in this criterion section, the most weighted criterion was role in organization with a priority vector of 0,30. consistency indexes for each criterion have been obtained by using equations 2 and 3. the consistency ratio was 0,05 by using equation (4). ijahp article: öneren, arar, çelebioglu/ determining the overall stress factors for policemen by ahp method international journal of the analytic hierarchy process 473 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.384 table 6 pairwise comparison matrix of criteria related with private life (c3) overall stress factors for policemen family problems (c31) personal characteristics (c32) socioeconomic cond. (c33) age (c34) physical & psychological health cond. (c35) pv family problems (c31) 1,00 7,94 5,92 9,00 4,21 0,56 personal characteristics (c32) 0,13 1,00 0,14 1,32 0,13 0,04 socio-economic cond. (c33) 0,17 7,00 1,00 5,00 3,00 0,22 age (c34) 0,11 0,76 0,20 1,00 0,33 0,04 physical & psychological health cond. (c35) 0,24 7,94 0,33 3,00 1,00 0,14 cr=0,1 in table 6, the priority vectors (weights) of sub criteria including family problems (c31), personal characteristics (c32), socio-economic conditions (c33), age (c34) and physical and psychological health conditions (c35) of the main criterion private life (c3) were found by using equation 1. in this criterion section, the most weighted criterion was family problems with a priority vector of 0,56. the consistency indexes for each criterion were obtained by using equations 2 and 3. the consistency ratio was 0,1 using equation(4. table 7 pairwise comparison matrix of main criteria overall stress factors for policemen job related factors (c1) organizational factors (c2) personal factors (c3) pv job related factors (c1) 1,00 3,63 9,00 0,69 organizational factors (c2) 0,28 1,00 5,59 0,25 private life factors (c3) 0,11 0,18 1,00 0,06 cr=0,06 ijahp article: öneren, arar, çelebioglu/ determining the overall stress factors for policemen by ahp method international journal of the analytic hierarchy process 474 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.384 in table 7, the priority vectors (weights) of main criteria including job related factors (c1), organizational factors (c2) and private life factors (c3) were found using equation 1. in this criterion section, the most weighted was job related factors with a priority vector of 0,69. the consistency indexes for each criterion were obtained by using equations 2 and 3. the consistency ratio was 0,06 by using equation 4. table 8 final results main factors weights of m.f. sub factors weights of s.f. global weights place in hierarchy c1 0,69 c11 0,22 0,150481 2 c12 0,12 0,085072 3 c13 0,05 0,031437 8 c14 0,61 0,424711 1 c2 0,25 c21 0,035 0,008679 11 c22 0,30 0,075249 4 c23 0,29 0,072760 5 c24 0,265 0,065827 6 c25 0,11 0,027207 9 c3 0,06 c31 0,56 0,032519 7 c32 0,04 0,002275 14 c33 0,22 0,012938 10 c34 0,04 0,002583 13 c35 0,14 0,008261 12 *sum of weights = 1 according to the final results, the most important stress factor is “danger level” by 40,62%. this result is not a surprise when the nature of this occupation and the selected region where this study has been conducted are considered. as mentioned before, the police job is dangerous work compared to many other fields and requires people who are able to be courageous in dangerous situations. the region studied is near the syrian border of turkey which is a very dangerous place because of war conditions within and outside the borders, and also because of the illegal flow of refugee migration in that area. for example, one of the interviewees added a note at the bottom of the questionnaire asking, “have you ever had to stand near a bomb until the bomb-expert comes and hope it will not explode? the answer is certainly the danger level!” although the interviewees do not know the ahp method in detail, and simply scored the criteria, the results show that he was right. the second most important stress factor was found to be the “workload” by 15%. it seems that an overload of work has a notable stressful effect on policemen as proven in the literature. it can be said that job related factors play the most important part in the stress level on policemen since this category included the three most significant stress factors and its own weight was 69%. on the other hand, the least important stress factors were found in the private life related factors category. personal ijahp article: öneren, arar, çelebioglu/ determining the overall stress factors for policemen by ahp method international journal of the analytic hierarchy process 475 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.384 characteristics and age seem to have almost no significance on stress levels. perhaps personal characteristics may play a more important role when it comes to coping with stress. age may be more important in other occupations when related to stress; but in this job, it is obvious that there are more things to consider instead of age when it comes to stress. 4. conclusion and discussion in this study, overall stress factors on policemen were determined and ordered hierarchically by the ahp method. the study was conducted via an online interview with two chiefs of police and two commissars who perform their duties in one of the most dangerous places in turkey. although, the existing literature about stress factors of policemen show some differences in the most important stress factors such as killing someone in the line of duty, administrative matters and relationship with non-police, this study showed that the most important factor in our context is “danger level” (violanti & aron, 1995; storch & panzarella, 1996). “personal characteristics” and “age” were found to be the least important stressors in contrast to the deschamsp et al. (2003) and newman and rucker-reed (2004) studies respectively. the reasons for these differences can be split into two parts which include the macro level and the micro level. from the macro level perspective, turkey is different from the previous research locations in the literature, and from the micro level perspective the region selected is also different. apart from the social and economic development differences, the previous research was performed in western, peaceful and developed countries; this research was conducted in turkey which is closed to the middle east as well as europe. furthermore, the selected region is one of the most dangerous places in turkey near the syrian border where terrorist attacks and war conditions are a daily routine. the reality of the region’s and country’s conditions make it necessary to take precautions and to provide solutions for coping with this increased stress level for the people who are brave enough to sacrifice themselves for the peace and safety of rest. this is important not only for the police’s own psychological health, but also because of the fact that the more stress a person has, the greater the possibility she or he may make mistakes, and their mistakes could be severe enough to result in injuries and deaths. the literature gives considerable suggestions to cope with stress and this should be a person’s own responsibility as much as the responsibility of his or her organization. according to tekiner and tavas (2016), in order to reduce stress an individual should follow strategies such as having positive dreams, exercising, maintaining personal control of behaviors, communication with others, mediation and nutrition control, having hobbies and being extroverted. according to the same study an organization should follow strategies such as creating a supportive organizational climate, job enrichment, making organizational roles clear, planning career development paths and coaching the employees in order to reduce stress. in patterson’s (2003) study, a person should get social support from his or her friends, family, supervisor or coworkers in the form of information, money or emotional support. another study encourages organizational support through time management and problem solving and through the work conditions such as creating a good environment and social support (michie, 2002). in contrast, loriol (2016) suggests that collective coping such as group therapy is more significant than social support in coping with stress. no matter what the occupation is, each job has its own stress factors for the employees. additionally, as a human being, there may be other factors that are involved in making people stressed. ijahp article: öneren, arar, çelebioglu/ determining the overall stress factors for policemen by ahp method international journal of the analytic hierarchy process 476 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.384 some occupations like police work can have a more stressful environment due to the nature of the work, and the stress born from that environment may be multiplied by other factors in people’s private lives. all of those factors may compound on each other, gather together and make life unbearable for an individual. stress is inevitable in life, and up to a point it plays a positive role. in order to keep it beneath this threshold, both individuals and organizations should assume responsibility for its management. this research has some limitations since it is conducted only with policemen, and the results should not be generalized to other occupations. the factors may be different in other jobs. furthermore, this research was conducted in a country where a peaceful environment exists mostly theoretically, and this situation probably had an effect on the results. hence, this research is original because it was conducted in a country whose environment shows disturbance and it also used a multicriteria decision making model called ahp. further research may be conducted in different work areas, in different countries and with other mcdm methods such as the analytic network process (anp) to investigate the relationship among the criteria or other methods such as promethee, electre, vikor or moora. ijahp article: öneren, arar, çelebioglu/ determining the overall stress factors for policemen by ahp method international journal of the analytic hierarchy process 477 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.384 reference berg, a. m., hem, e., lau, b., haseth, k., ekeberg, o. (2005). stress in norwegian police service, occupational medicine, 55(1), 113-120. doi: 10.1093/occmed/kqi023 birkök, c. (2000). poliste çatışma ve stres yönetimi, uluslararası i̇nsan bilimleri dergisi, 1-9. blaug, r., kenyon, a., lekhi, r. (2007). stress at work, www.theworkfoundation.com access date: 17 th april 2016. crawford, g., williams, c. (1985). the analysis of subjective judgment matrices, project air force report prepared for the united states air force, 1-34. deschamps, f., paganon-badinier, i., marchand a. c., merle, c. (2003). sources and assessment of occupational stress in the police, journal of occupational health, 45(1) 358-364. doi: http://doi.org/10.1539/joh.45.358. dong, q., cooper, o. (2016). a peer to peer dynamic adaptive consensus reaching model for the group ahp decision making, european journal of operational research, 250(2), 521-530. doi: http://dx.doi.org/10.1016/j.ejor.2015.09.016 eren, e. (2014). örgütsel davranış ve yönetim psikolojisi, beta, i̇stanbul. karakaya, a. (2002). emniyet mensuplarının demografik özellikleri ile stres faktörleri arasındaki i̇lişki üzerine bir araştırma, teknoloji, 5(2), 71-81. kendall, e., muenchberger, h. (2009). stress at work: using a process model to assist employers to understand the trajectory, occupational concerns and workplace wellhealth, 32(1), 19-25. kendall, e., murphy, p., o’neill, v., bursnall, s. (2000). occupational stress: factors that contribute to its occurrence and effective management, workers’ compensation and rehabilitation commission, http:/wwww.workcover.wa.gov.au access date: 16 th april 2016. kuo, s. (2015). occupational stress, job satisfaction, and affective ccommitment to policing among taiwanese police officers, police quarterly, 18(1), 2754. doi:10.1177/1098611114559039 lindfors, s., eintrei, c., alexanderson, k. (2009). stress factors affecting academic physicians at a university hospital, occupational concerns and workplace well-health, 34(1), 305-313. loriol, m. (2016). collective forms of coping and the social construction of work stress among industrial workers and police officers in france, theory & pschology, 26(1), 112129. doi:10.1177/0959354315616877 http://dx.doi.org/10.1016/j.ejor.2015.09.016 ijahp article: öneren, arar, çelebioglu/ determining the overall stress factors for policemen by ahp method international journal of the analytic hierarchy process 478 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.384 luthans, f. (1992). organizational behavior, istanbul: mcgraw hill. michie, s. (2009). causes and management of stress at work, occupational and. environmental medicine, 59(1), 67-72. doi:10.1136/oem.59.1.67 mkumbo, k. (2014). prevalence of and factors associated with work stress in academia in tanzania, international journal of higher education, 3(1), 1-11. doi: 10.5430/ijhe.v3n1p1 newman, d. w., rucker-reed, m. l. (2004). police stress, state-trait anxiety, and stressors among u.s. marshals, journal of criminal justice, 32(1), 631-641. doi: http://dx.doi.org/10.1016/j.jcrimjus.2004.08.003 patterson, g. t. (2003). examining the effects of coping and social support on work and life stress among police officers, journal of criminal justice, 31(1), 215-226. doi: http://dx.doi.org/10.1016/s0047-2352(03)00003-5 robbins, s. p, judge, t. a. (2009). organizational behavior, new jersey: pearson prentice hall. saaty, t. l. (1991). some mathematical concepts of the analytic hierarchy process, behaviormetrika, 18(29), 1-9. doi: http://doi.org/10/23333/bhmk.18.29_1 saaty, t. l., vargas, l. g. (2012). models, methods, concepts & applications of the analytic hierarchy process, new york: springer. storch, j. e., panzarella, r. (1996). police stress: state-trait anxiety in relation to occupational and personal stressors, journal of criminal justice, 24(2), 99-107. doi:10.1016/0047-2352(95)00058-5 supçiller, a. a., çapraz, o. (2011). ahp-topsis yöntemine dayalı tedarikçi seçimi uygulaması, i̇stanbul üniversitesi i̇ktisat fakültesi ekonometri ve i̇statistik dergisi, 13, 1-22. tekiner, m. a., tavas, b. (2016). employee job satisfaction on the effect of stress management skills gain: sample of police organization, the journal of international social research, 9(42), 1658-1671. tutar, h. (2004). kriz ve stres yönetimi, ankara: seçkin. violanti, j. m., aron, f. (1995)., police stressors: variations in perception among police personnel, journal of criminal justice, 23(3), 287-294. doi:10.1016/00472352(95)00012-f http://dx.doi.org/10.1016/j.jcrimjus.2004.08.003 http://dx.doi.org/10.1016/s0047-2352(03)00003-5 http://dx.doi.org/10.1016/0047-2352(95)00058-5 http://dx.doi.org/10.1016/0047-2352(95)00012-f http://dx.doi.org/10.1016/0047-2352(95)00012-f ijahp article: kamath, naik, prasad/ vendor’s evaluation – using ahp for an indian steel pipe manufacturing company international journal of the analytic hierarchy process 442 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.460 a vendor’s evaluation–using ahp for an indian steel pipe manufacturing company giridhar kamath rakesh naik shiva prasad h c manipal institute of technology department of humanities and management manipal university, manipal india giridhar.bk@yahoo.com, rakeshnr.16@gmail.com hcs.prasad@manipal.edu abstract to improve a firm’s supply chain performance it is essential to have a vendor evaluation process to be able to showcase an organization’s success in the present aggressive market. hence, the process of evaluating the vendor is a crucial task of the purchasing executives in supply chain management. the objective of this research is to propose a methodology to evaluate the vendors for a steel pipe manufacturing firm in gujarat, india. for the purpose of the study, the analytical hierarchy process was used to evaluate the best raw material vendor for this company. multiple qualitative and quantitative criteria are involved in the vendor evaluation process. to solve the complex problem of vendor evaluation, a tradeoff between these multi-criteria is important. the outcomes indicated that the ahp technique makes it simpler to assign weights for the different criteria for evaluating the vendor. research findings showed that quality is the most important criterion followed by delivery, cost and vendor relationship management. keywords: cost, delivery; quality; vendor evaluation; vendor relationship management 1. introduction the field of supply management has been undergoing a transformation from a tactical, transaction oriented role to a strategic capability at many companies. senior executives are discovering that a good, integrated supply management capability is not only necessary, but also required to achieve a competitive advantage. management is realizing that there is potential for procurement to add cash to the bottom line instead of only viewing procurement as a cost center. the procurement function that used to play a lesser role in organizations now has titles like chief procurement officer and corporate vice president of vendor quality and performance management. the senior level executives have to consider many factors such as goals of the organization, financial tolerance, administrative time frame and many more when evaluating a vendor. the raw material costs account for up to nearly 70% of the total cost of the product in many industries (ghodsypour & o'brien, 1998). hence, an overall goal of cost reduction depends upon mailto:giridhar.bk@yahoo.com mailto:rakeshnr.16@gmail.com mailto:hcs.prasad@manipal.edu ijahp article: kamath, naik, prasad/ vendor’s evaluation – using ahp for an indian steel pipe manufacturing company international journal of the analytic hierarchy process 443 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.460 decisions made by the procurement executives. for the success of an organization in a competitive market, selection and evaluation of the vendor plays an important role. for the buying firm, evaluating the best vendor is one of the most challenging tasks. the varied strengths and weaknesses of the vendors make it difficult for the purchasing firm to carefully assess the vendors before ranking them. the evaluation of the vendor would be simple if only one criterion was used in the process of making the decision. however, for the purchasing executives of a steel pipe manufacturing firm, evaluation of the vendor involves a number of criteria and sub-criteria. therefore, if many criteria are involved in the process of selecting the best vendor it is essential to determine whether the criteria are equally weighted or vary according to the type of criteria (yahya & kingsman, 1999). the development of the model for xyz company’s vendor evaluation is essential not only for the benefit of the organization, but also because the vast range of the finished products requires a vast range of raw materials which fluctuate in price, and there are a large number of vendors and projects in process. the vendors are evaluated based on a number of criteria so that the purchasing department can make their vendor selection decision based upon the most essential criteria. evaluation of the vendor is a group multiple criteria decision making (mcdm) problem (ho, xu & dey, 2010). in mcdm the purchasing executives have to analyze the tradeoff among several conflicting criteria in vendor evaluation. the analytical hierarchy process (ahp) is a linear weighing model, and is one of the most used models among the various approaches for the evaluation (john, baby & mangalathu, 2013; sonawane, & rodrigues, 2015). the ahp technique is recommended for the criteria selection in vendor evaluation to ease or eliminate the inaccuracy in this line which is often caused by adjudicating the raw materials or giving attention to only one criterion such as cost or quality. research objectives the main objective of the research is to identify the measure or criteria that impact the evaluation of the vendor at xyz firm. thus, the objectives are: 1. to determine the factors that influences the vendor evaluation at xyz firm. 2. to develop a model that describes the measure for evaluating the vendor. 3. to determine the best raw material supply vendor. research questions 1. what are the criteria for vendor evaluation at xyz firm? 2. which is the best raw material vendor at xyz firm? 2. literature review the process of vendor evaluation becomes a very complicated task as many criteria should be taken into account with more than 20 factors suggested for the evaluation of the vendor that the procurement managers have to consider during the process of selection of the vendor (dickson, 1966; imeri, 2013). the purchasing managers do a lot more than just buying goods. the main job of the managers is to make decisions regarding important criteria along with other people in the organization. other than minimizing the cost, the responsibility of the managers of the procurement department is to select the appropriate vendor to help them accomplish the wide objectives of the firm. ijahp article: kamath, naik, prasad/ vendor’s evaluation – using ahp for an indian steel pipe manufacturing company international journal of the analytic hierarchy process 444 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.460 while meeting the organization’s goal, evaluation of the vendor process helps recognize and differentiate between vendors at an adequate cost. based on the criteria considered for the vendor evaluation, a number of vendors are being selected. in order for an organization to remain sustainable in the competitive market they must provide better quality and services to their customers to satisfy them. therefore, the company should evaluate and select the vendor that is best able to make sure they manufacture a quality product. in order to do this, the company has to spend a significant amount of time evaluating the suitable vendor (alsuwehri, 2011). it is essential to identify the criteria that influence the vendor evaluation process. as suggested by dickson (1966) for vendor selection from a group of criteria, the important criteria like lowest price, quality, and prompt deliveries are considered by many researchers and common metrics used (dickson, 1966; weber, current, & benton,1991). based on twenty-three criteria presented in the studies of weber, current, & benton (1991) and dickson (1966), the most important criteria for evaluation of the vendor are quality, price and delivery and management and organization (alsuwehri, 2011). based on the above literature review, quality, cost, delivery and vendor relationship management were the criteria considered for the organization in the case study. the subcriteria under quality are specification of the raw material or the equipment, warranty, rejection, packing, continuous improvement and top management (yusuff, yee & hashmi, 2001). under the cost criteria net price, ordering and delivery cost, and capital investments are the sub-criteria (yusuff, yee & hashmi, 2001). the measures of delivery for vendor evaluation are late delivery, location and lead time (alsuwehri, 2011). another criterion that is taken into consideration for the study is vendor relationship management (vrm). the process of building and maintaining a sustainable, cordial relation with the supplier with social fabrics is the basis of the relationship apart from the formal business transaction (giunipero & pearcy, 2000). hence, managing the social fabric with the vendor is essential at the nascent stage of business process. due to the commitment to multiple partnerships and since the relationship among the partners is dynamic (not everlasting), vendor relationship management is a crucial issue in the evaluation of the vendor in supply chain management (giunipero & pearcy, 2000). this is because substitution of the partners has become a common practice in the industry in order for companies to improve their performance and meet the multiple market conditions (mowshowitz, 1997). the vendor’s involvement in research and development activities helps the company with continuous improvement (tahriri, et al., 2008). a longterm relationship with the vendor is crucial for a firm to be able to effectively fulfill the demands of the customers (kannan & tan, 2002). the reputation of the vendor should also be taken into account as another sub-criterion in vendor relationship management as it facilitates improvement in both the business and operational perspectives (kannan & tan, 2002). constant vendor evaluation, selection and measurement of performance are essential for the success of every organization. this is especially important when developing a new product. vendor evaluation is a problem that involves multiple criteria which include both tangible and intangible factors. therefore, for a company to acquire higher profits even a small cost reduction in procuring the raw material will make a difference. the process of vendor evaluation has a direct influence on quality, cost, delivery and vendor relationship management. ijahp article: kamath, naik, prasad/ vendor’s evaluation – using ahp for an indian steel pipe manufacturing company international journal of the analytic hierarchy process 445 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.460 3. methodology the specific scheme discussed in the paper is for xyz steel pipe manufacturing company in gujarat, india. the main product of the company is welded mild steel (ms) pipe. the company also manufactures ms plates and coils, bend pipes and offers coating for the pipes. the pipe plants of xyz company are located in india, the u.s. and saudi arabia. the main goal of the research is to evaluate the best raw material vendor using a combination of the ahp technique and the factor rating method. complex, multi-attribute problems can be handled effectively by the ahp technique (yusuff, yee & hashmi, 2001). the ahp technique is found to be useful to help reach a consensus solution to a problem with diverse and conflicting criteria (tam & tummala, 2001). hence, the ahp is found to be very effective in the problem of evaluating the vendor decision to determine the best vendor (yu, & jing, 2004). the ahp approach is used to assign weights to the various criteria, and the factor rating method is used to find the optimal vendor based on the criteria. the criteria and sub criteria for vendor evaluation were identified upon close interaction with the managers and the deputy general manager of the procurement, finance, and quality control departments (see figure 1). the credibility of the research was not affected by this method as managers and the dgm had in-depth knowledge and experience with respect to the raw material vendors. six raw material vendors, each providing american petroleum institute (api) (www.fedsteel.com) and indian standard (is) (www.gipipesindia.com), specification raw material were considered for the study. to ensure the effective development of the model the following five steps were implemented. 3.1 steps step 1: state the criteria for vendor evaluation establishing the criteria is the first step in the process of vendor evaluation. the criteria for vendor evaluation are quality, cost, delivery and vendor relationship management. step 2: define sub-criteria for vendor evaluation the second step is to define the sub-criteria for the above-mentioned criteria. the subcriteria were selected based on the literature review and discussion with the managers and senior level executives of the purchasing, finance and quality control departments. a total of twenty-four sub-criteria were considered for the study; six sub-criteria under each criterion were considered. the sub-criteria that will result in the delivery of raw material by the vendors taking into account the most essential requirements of the organization were selected. step 3: structuring of hierarchy model in this step, weights were assigned to the criteria and sub-criteria. weights were allocated for each criteria and sub-criteria by making pair-wise comparisons with the ahp technique. pair-wise comparison was carried out by obtaining the relative importance for the criteria and sub-criteria. a nine-point scale depicted by saaty (1980) was used for this. the nine-point scale specifies the comparative significance with the levels equal, moderate, strong, very strong, and extreme represented by 1, 3, 5, 7, and 9, respectively. the intermediary importance between two contiguous contrasts is indicated by 2, 4, 6, and 8. experts have accepted the nine-point scale depicted by saaty as it is very scientific for comparisons of two alternate criteria (saaty, 1980). in the ahp technique, the fundamental assumption for the comparison of the criteria is; if criteria x is very strongly preferred to criteria y then it is rated as 7, and if y is strongly preferred to x it is rated as ijahp article: kamath, naik, prasad/ vendor’s evaluation – using ahp for an indian steel pipe manufacturing company international journal of the analytic hierarchy process 446 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.460 1/7 (saaty, 1980). the pair-wise comparison is carried out for all the criteria (see table 1). table 1 pair-wise comparison values preferences ratings equally preferred 1 equally to moderately preferred 2 moderately preferred 3 moderately to strongly preferred 4 strongly preferred 5 strongly to very strongly preferred 6 very strongly preferred 7 very strongly to extremely preferred 8 extremely preferred 9 figure 1. criteria and sub-criteria for vendor evaluation the objective of the study is to evaluate the vendors based on the criteria mentioned for xyz company. to assign the weights for the criteria, the following steps have to be followed (saaty, 1980). ijahp article: kamath, naik, prasad/ vendor’s evaluation – using ahp for an indian steel pipe manufacturing company international journal of the analytic hierarchy process 447 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.460 1. find the sum of values in each column of the pair-wise comparison matrices. 2. divide each value in each column by the parallel column sum. the outcome denotes the normalized matrices. 3. calculate the average of each row of the normalized matrices. the outcome depicts preference vectors. 4. combine the preference vectors for each criterion into one preference matrix that depicts the weights for each criterion. consistency check the comparisons of the criteria made by the executives of the organization have to be validated for consistency to make sure that the model is reliable. the decision made by the executives is consistent, if the consistency ratio (cr) is zero. the cr value for the outcomes of the results is acceptable if the values are less than 0.1 as given by saaty and hu (1998). a cr value above 0.1 is unacceptable, and these results are considered untrustworthy because it is too close to randomness. in this case, the decision may have to be repeated (saaty & hu, 1998). consistency ratio (cr) calculation a cr is calculated by computing λmax which facilitates the calculation of consistency index (ci) taking into account equation 1. ax=λmaxx (1) where a is the preference matrix and x is the eigenvector, so ) x ax (averagemax  (2) ci is computed by the equation 2. 1n nmax ci    (3) the equation for consistency ratio is given by ri ci cr  (4) where ci is the consistency index and ri is the index for the corresponding random matrix. the ri values are depicted from saaty’s table (table 2). table 2 reference value of ri (saaty & tran, 2007) order of randomness 1 2 3 4 5 6 7 8 9 10 random index (ri) 0.00 0.00 0.58 0.90 1.12 1.24 1.32 1.41 1.45 1.49 a pair-wise comparison study for criteria considered for vendor evaluation is shown in table 3. ijahp article: kamath, barkur, naik/vendors’ evaluation – using ahp for an indian steel pipe manufacturing company international journal of the analytic hierarchy process 448 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.353 table 3 pair-wise comparison of criteria criteria q u a l it y c o s t d e l iv e r y v r m quality 1 9 5 9 cost 1/9 1 1/4 3 delivery 1/5 4 1 5 vrm 1/9 1/3 1/5 1 thus, with pair-wise comparison it can be determined that the company extremely prefers quality over cost criterion, and therefore it has been rated nine as defined in saaty’s table (table 1). the preference of cost over quality is therefore rated as 1/9 (0.1111) which is the inverse of the preference of quality over cost criterion. similarly, the company strongly preferred quality over delivery and it is rated five, and quality is extremely preferred over vendor relationship management (vrm) represented with a nine. the company moderately prefers cost over vendor relationship management (rated as three). the company strongly to moderately prefers delivery over cost and strongly prefers over vrm. the criterion compared with itself is rated as one. in order to determine whether the responses of the experts of the procurement department are consistent after the pair-wise comparisons it is necessary to find the consistency ratio (cr). this is computed as given with equation 4. λmax was computed with equation 2 and is shown in table 6. table 4 pair-wise comparison and column sums c ri te ri a q u a l it y c o s t d e l iv e r y v r m quality 1 9 5 9 cost 0.111 1 0.25 3 delivery 0.2 4 1 5 vrm 0.111 0.333 0.2 1 column sums 1.422 14.333 6.45 18 table 5 normalized column sums q u a l it y c o s t d e l iv e r y v r m l o c a l w e ig h t (r o w a v e ra g e s) quality 0.703 0.628 0.775 0.500 0.652 cost 0.078 0.070 0.039 0.167 0.088 delivery 0.141 0.279 0.155 0.278 0.213 vrm 0.078 0.023 0.031 0.056 0.047 ijahp article: kamath, naik, prasad/ vendor’s evaluation – using ahp for an indian steel pipe manufacturing company international journal of the analytic hierarchy process 449 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.460 table 6 λmax calculation a x ax ax 1 9 5 9 * * * * 0.652 = = = = 2.935 = λmax 4.505 0.111 1 0.3 3 0.088 0.355 4.019 0.2 4 1 5 0.213 0.932 4.371 0.111 0.333 0.2 1 0.047 0.191 4.075 λmax= average (4.5047/2.9350, 4.0186/0.3550, 4.3715/0.9317, 4.0746/0.1914) = 4.2424 consistency index= (λmax-n)/ (n-1); i e, (4.2424-4)/ (4-1) = 0.0808 consistency ratio (cr) = ci/ri = 0.0808/0.9 = 0.0898<0.1; therefore, the pair-wise comparisons of criteria for vendor evaluation are consistent. the value of ri for 4 orders of randomness (4 criteria) is 0.9. the row averages from the normalized table determined that quality is the most important criterion with a weight of 65% followed by delivery (21%), cost (9%) and vendor relationship management (4%) for the xyz firm. the pairwise comparisons, normalized table and consistency check for the sub-criteria are analyzed in tables 7 to 18 and the results are analyzed for a consistency check in table 19. table 7 pair-wise comparison between sub-criteria of quality criterion sc-quality ts i s c tc ci ts 1 1 1 1 1 5 i 1 1 1 1 1 6 s 1 1 1 1 1 8 c 1 1 1 1 1 8 tc 1 1 1 1 1 8 ci 0.2 0.167 0.125 0.125 0.125 1 sum 5.2 5.167 5.125 5.125 5.125 36 ts-technical specification, i-inspections, s-api/ is specification, c-certified, tc-test certificate, cicontinuous improvement table 8 normalized table of sub-criteria of quality ts i s c tc ci local weights ts 0.192 0.194 0.195 0.195 0.195 0.139 0.185 i 0.192 0.194 0.195 0.195 0.195 0.167 0.190 s 0.192 0.194 0.195 0.195 0.195 0.222 0.199 c 0.192 0.194 0.195 0.195 0.195 0.222 0.199 tc 0.192 0.194 0.195 0.195 0.195 0.222 0.199 ci 0.038 0.032 0.024 0.024 0.024 0.028 0.029 ijahp article: kamath, naik, prasad/ vendor’s evaluation – using ahp for an indian steel pipe manufacturing company international journal of the analytic hierarchy process 450 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.460 table 9 λmax calculation a x ax ax 1 1 1 1 1 5 * * * * * * 0.185 = = = = = = 1.114 = λmax 6.023 1 1 1 1 1 6 0.190 1.143 6.027 1 1 1 1 1 8 0.199 1.200 6.034 1 1 1 1 1 8 0.199 1.200 6.034 1 1 1 1 1 8 0.199 1.200 6.034 0.200 0.167 0.125 0.125 0.125 1 0.0299 0.172 6.005 table 10 pair-wise comparison between sub-criteria of cost criterion sc-cost l bod drc ce cp cpp l 1 0.143 1 1 0.111 0.111 bod 7 1 1 5 0.143 0.2 drc 1 1 1 1 0.125 0.125 ce 1 0.2 1 1 0.125 0.143 cp 9 7 8 8 1 1 cpp 9 5 8 7 1 1 sum 28 14.343 20 23 2.504 2.579 l-logistics, bod-bulk order discounts, drcdefect replacement cost, ce-cost escalations, cp-credit period, cpp-competitive product price table 11 normalized table of sub-criteria of cost l bod drc ce cp cpp local weights l 0.036 0.010 0.050 0.043 0.044 0.043 0.038 bod 0.250 0.070 0.050 0.217 0.057 0.078 0.120 drc 0.036 0.070 0.050 0.043 0.050 0.048 0.050 ce 0.036 0.014 0.050 0.043 0.050 0.055 0.041 cp 0.321 0.488 0.400 0.348 0.399 0.388 0.391 cpp 0.321 0.349 0.400 0.304 0.399 0.388 0.360 table 12 λmax calculation a x ax ax 1 0.143 1 1 0.111 0.111 * * * * * * 0.038 = = = = = = 0.229 = λmax 6.073 7 1 1 5 0.143 0.200 0.120 0.769 6.394 1 1 1 1 0.125 0.125 0.050 0.343 6.920 1 0.2 1 1 0.125 0.143 0.041 0.253 6.112 9 7 8 8 1 1 0.391 2.661 6.809 9 5 8 7 1 1 0.360 2.379 6.603 ijahp article: kamath, naik, prasad/ vendor’s evaluation – using ahp for an indian steel pipe manufacturing company international journal of the analytic hierarchy process 451 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.460 table 13 pair-wise comparison between sub-criteria of delivery criterion f ti gl dm gc drt f 1 8 1 5 1 1 ti 0.125 1 0.2 0.2 0.125 0.2 gl 1 5 1 7 1 1 dm 0.2 5 0.143 1 0.143 0.2 gc 1 8 1 7 1 5 drt 1 5 1 5 0.2 1 sum 4.325 32 4.343 25.2 3.468 8.4 f-flexibility, ti-tracking information, gl-geographic location, dm-delivery mode, gcgood condition, drt-defect replacement time table 14 normalized column of sub-criteria of delivery f ti gl dm gc drt local weights f 0.231 0.250 0.230 0.198 0.288 0.119 0.220 ti 0.029 0.031 0.046 0.008 0.036 0.024 0.029 gl 0.231 0.156 0.230 0.278 0.288 0.119 0.217 dm 0.046 0.156 0.033 0.040 0.041 0.024 0.057 gc 0.231 0.250 0.230 0.278 0.288 0.595 0.312 drt 0.231 0.156 0.230 0.198 0.058 0.119 0.165 table 15 λmax calculation a x ax ax 1 8 1 5 1 1 * * * * * * 0.220 = = = = = = 1.430 = λmax 6.512 0.125 1 0.2 0.2 0.125 0.2 0.029 0.183 6.322 1 5 1 7 1 1 0.217 1.456 6.705 0.2 5 0.143 1 0.143 0.2 0.057 0.354 6.251 1 8 1 7 1 1 0.312 2.205 7.064 1 5 1 5 0.2 0.2 0.165 1.093 6.605 ijahp article: kamath, naik, prasad/ vendor’s evaluation – using ahp for an indian steel pipe manufacturing company international journal of the analytic hierarchy process 452 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.460 table 17 normalized table of sub-criteria of vendor relationship management ca si br lr rda sui local weights ca 0.071 0.071 0.056 0.074 0.059 0.091 0.070 si 0.071 0.071 0.056 0.074 0.059 0.091 0.070 br 0.357 0.357 0.280 0.369 0.353 0.091 0.301 lr 0.357 0.357 0.280 0.369 0.412 0.545 0.387 rda 0.071 0.071 0.047 0.053 0.059 0.091 0.065 sui 0.071 0.071 0.280 0.062 0.059 0.091 0.106 table 18 λmax calculation a x ax ax 1 8 1 5 1 1 * * * * * * 0.220 = = = = = = 1.430 = λmax 6.512 0.125 1 0.2 0.2 0.125 0.2 0.029 0.183 6.322 1 5 1 7 1 1 0.217 1.456 6.705 0.2 5 0.143 1 0.143 0.2 0.057 0.354 6.251 1 8 1 7 1 1 0.312 2.205 7.064 1 5 1 5 0.2 0.2 0.165 1.093 6.605 from equations 2, 3 and 4 a consistency check for the sub-criteria of quality, cost, delivery and vrm is computed and shown in table 19. table 19 consistency check λmax ci cr decision sub-criteria of quality 6.026 0.005 0.004 consistent sub-criteria of cost 6.576 0.115 0.093 consistent sub-criteria of delivery 6.485 0.097 0.078 consistent sub-criteria of vrm 6.377 0.075 0.061 consistent table 16 pair-wise comparison between sub-criteria of vendor relationship management ca si br lr rda sui ca 1 1 0.20 0.20 1 1 si 1 1 0.20 0.20 1 1 br 5 5 1 1 6 1 lr 5 5 1 1 7 6 rda 1 1 0.17 0.143 1 1 sui 1 1 1 0.167 1 1 sum 14.00 14.00 3.57 2.71 17.00 11.00 ca-conduct audit, si-share information, br-background reputation, lr-long-term relationship, rda-r & d activities, sui-suggestion implementation ijahp article: kamath, naik, prasad/ vendor’s evaluation – using ahp for an indian steel pipe manufacturing company international journal of the analytic hierarchy process 453 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.460 the global weights for each sub-criteria are computed by the product of local weights of sub-criteria and its relevant criteria and shown in table 20. quality is the most important criteria with the highest local weight of 0.652 followed by delivery (0.213), cost (0.088) and vrm (0.047) (table 20). the prioritization of sub-criteria depends on the local weights table 20 assignment of local and global weights c ri te ri a l o c a l w e ig h t sub criteria l o c a l w e ig h ts g lo b a l w e ig h ts c ri te ri a l o c a l w e ig h t sub criteria l o c a l w e ig h ts g lo b a l w e ig h ts q u a li ty 0 .6 5 2 api/ is spec 0.199 0.130 d e li v e ry 0 .2 1 3 good condition 0.312 0.035 certified 0.199 0.130 flexibility 0.220 0.032 test certificate 0.199 0.130 geographic location 0.217 0.011 inspection 0.190 0.124 defect replacement 0.165 0.004 technical specification 0.185 0.121 delivery mode 0.057 0.004 continuous improvement 0.029 0.019 tracking information 0.029 0.003 c o st 0 .0 8 8 credit period 0.391 0.067 v r m 0 .0 4 7 long term relationship 0.387 0.018 competitive product price 0.360 0.047 background reputation 0.301 0.014 bulk order discounts 0.120 0.046 suggestion implementation 0.106 0.005 cost of defect replacement 0.050 0.035 conduct audit 0.070 0.003 cost escalations 0.041 0.012 share info 0.070 0.003 logistics cost 0.038 0.006 r and d activities 0.065 0.003 step 4: sub-criteria order prioritization after the completion of pair-wise comparisons, and calculating the local weights for each criterion, the next step is to arrange the criteria according to the level of importance for evaluating the best vendor. ijahp article: kamath, naik, prasad/ vendor’s evaluation – using ahp for an indian steel pipe manufacturing company international journal of the analytic hierarchy process 454 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.460 table 21 prioritization of global weights sub criteria global weights weigh ts in % sub criteria global weights weights in % api/ is spec 0.130 13.0 long term relationship 0.018 1.8 certified 0.130 13.0 background reputation 0.014 1.4 test certificate 0.130 13.0 delivery mode 0.012 1.2 inspection 0.124 12.4 bulk order discounts 0.011 1.1 technical specification 0.121 12.1 tracking information 0.006 0.6 good condition 0.067 6.7 suggestion implementation 0.005 0.5 flexibility 0.047 4.7 cost of defect replacement 0.004 0.4 geographic location 0.046 4.6 cost escalations 0.004 0.4 defect replacement 0.035 3.5 logistics 0.003 0.3 credit period 0.035 3.5 conduct audit 0.003 0.3 competitive product price 0.032 3.2 share info 0.003 0.3 continuous improvement 0.019 1.9 r & d activities 0.003 0.3 the results show values arranged in decreasing order so that prioritization of the subcriteria can be accomplished (table 21). it is observed that the quality and the delivery criteria occupy the top 10 in the ranking list of sub-criteria. step 5: vendor evaluation the aim of adopting the ahp technique was to assign weights to the different criteria for evaluating the vendor in xyz steel pipe manufacturing company. after assigning the weights for the criteria and sub-criteria and validating the model, a rating scale was given to the senior executives of the procurement department. the respondents were asked to rate the raw material vendors on a scale from 0-10; 0 being the worst and ten being the best (table 22). table 22 ten point likert scale ratings 0 1 2 3 4 5 6 7 8 9 10 preference worst very poor poor significantly below avg. below avg. avg. above avg. significantly above avg. good very good best the ratings for the different vendors for the api and is vendors of raw materials for xyz company are given below in tables 23 and 24 respectively. the sum of global weights of the vendors was computed, and the vendor with the highest total global weights is considered the best vendor based on the criteria and sub-criteria. vendors a and e are the manufacturers of api specification mild steel coils (raw material for pipe manufacturing) and vendors b, c and d are the dealers and distributors of both api and is specification raw materials. vendors p and q are the producers of is specification raw materials. figure 2 graphically shows the weights of the api vendors multiple criteria that were considered. similarly, figure 3 shows the weights of the is vendors criteria that were considered in this study. ijahp article: kamath, naik, prasad/ vendor’s evaluation – using ahp for an indian steel pipe manufacturing company international journal of the analytic hierarchy process 455 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.460 figure 2. weights of api vendors w.r.t criteria figure 3. weights of api vendors w.r.t criteria ijahp article: kamath, naik, prasad/ vendor’s evaluation – using ahp for an indian steel pipe manufacturing company international journal of the analytic hierarchy process 456 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.460 table 23 api vendor’s ratings and weights with respect to different criteria vendor a vendor b vendor c vendor d vendor e s u b c ri te ri a g lo b a l w e ig h ts r a ti n g s w e ig h ts r a ti n g s w e ig h ts r a ti n g s w e ig h ts r a ti n g s w e ig h ts r a ti n g s w e ig h ts s 0.130 9 1.166 9 1.166 9 1.166 9 1.166 9 1.166 c 0.130 9 1.166 9 1.166 9 1.166 9 1.166 9 1.166 tc 0.130 9 1.166 8 1.037 8 1.037 8 1.037 9 1.166 i 0.124 9 1.112 9 1.112 9 1.112 9 1.112 9 1.112 ts 0.121 9 1.085 8 0.964 8 0.964 8 0.964 9 1.085 gc 0.067 9 0.599 9 0.599 9 0.599 9 0.599 9 0.599 f 0.047 9 0.421 8 0.374 7 0.328 7 0.328 8 0.374 gl 0.046 9 0.417 8 0.370 7 0.324 7 0.324 8 0.370 dr 0.035 9 0.317 9 0.317 9 0.317 9 0.317 9 0.317 cp 0.035 8 0.276 9 0.311 9 0.311 9 0.311 8 0.276 cpp 0.032 8 0.255 9 0.286 9 0.286 9 0.286 8 0.255 ci 0.019 9 0.168 7 0.130 7 0.130 7 0.130 9 0.168 lr 0.018 8 0.145 9 0.164 9 0.164 9 0.164 8 0.145 br 0.014 9 0.127 9 0.127 9 0.127 9 0.127 9 0.127 dm 0.012 9 0.109 9 0.109 9 0.109 9 0.109 9 0.109 bod 0.011 8 0.085 9 0.096 9 0.096 9 0.096 8 0.085 ti 0.006 9 0.056 9 0.056 9 0.056 9 0.056 9 0.056 sim 0.005 8 0.040 9 0.045 9 0.045 9 0.045 8 0.040 drc 0.004 8 0.035 8 0.035 8 0.035 8 0.035 8 0.035 ce 0.004 7 0.026 9 0.033 9 0.033 9 0.033 7 0.026 l 0.003 7 0.023 8 0.027 9 0.030 7 0.023 8 0.027 ca 0.003 9 0.030 7 0.023 7 0.023 7 0.023 9 0.030 si 0.003 7 0.023 9 0.030 9 0.030 9 0.030 7 0.023 rda 0.003 8 0.025 5 0.015 5 0.015 5 0.015 8 0.025 sum 8.872 8.593 8.503 8.496 8.782 s-api/is spec, c-certified, tc-test certificate ,i-inspection, ts-technical specification, gc-good condition, f-flexibility, gl-geographic location, drdefect replacement, cp-credit period, cpp-competitive product price, cicontinuous improvement, lr-long-term relationship, br-background reputation, dm-delivery mode, bod-bulk order discounts, ti-tracking information, simsuggestion implementation, drc-defect replacement cost, ce-cost escalations, lr-logistics, ca-conduct audit, si-share info, rda-r & d activities ijahp article: kamath, naik, prasad/ vendor’s evaluation – using ahp for an indian steel pipe manufacturing company international journal of the analytic hierarchy process 457 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.460 table 24 is vendor’s ratings and weights with respect to different criteria vendor p vendor a vendor b vendor c vendor q s u b c ri te ri a g lo b a l w e ig h ts r a ti n g s w e ig h ts r a ti n g s w e ig h ts r a ti n g s w e ig h ts r a ti n g s w e ig h ts r a ti n g s w e ig h ts s 0.130 9 1.166 9 1.166 9 1.166 9 1.166 9 1.166 c 0.130 9 1.166 9 1.166 9 1.166 9 1.166 9 1.166 tc 0.130 9 1.166 9 1.166 8 1.037 8 1.037 9 1.166 i 0.124 9 1.112 9 1.112 9 1.112 9 1.112 9 1.112 ts 0.121 9 1.085 9 1.085 8 0.964 8 0.964 9 1.085 gc 0.067 9 0.599 9 0.599 9 0.599 9 0.599 9 0.599 f 0.047 8 0.374 9 0.421 8 0.374 7 0.328 8 0.374 gl 0.046 8 0.370 9 0.417 8 0.370 7 0.324 8 0.370 dr 0.035 9 0.317 9 0.317 9 0.317 9 0.317 9 0.317 cp 0.035 8 0.276 8 0.276 9 0.311 9 0.311 8 0.276 cpp 0.032 8 0.255 8 0.255 9 0.286 9 0.286 8 0.255 ci 0.019 9 0.168 9 0.168 7 0.130 7 0.130 9 0.168 lr 0.018 8 0.145 8 0.145 9 0.164 9 0.164 8 0.145 br 0.014 9 0.127 9 0.127 9 0.127 9 0.127 9 0.127 dm 0.012 9 0.109 9 0.109 9 0.109 9 0.109 9 0.109 bod 0.011 8 0.085 8 0.085 9 0.096 9 0.096 8 0.085 ti 0.006 9 0.056 9 0.056 9 0.056 9 0.056 9 0.056 sim 0.005 8 0.040 8 0.040 9 0.045 9 0.045 8 0.040 drc 0.004 8 0.035 8 0.035 8 0.035 8 0.035 8 0.035 c 0.004 7 0.026 7 0.026 9 0.033 9 0.033 7 0.026 l 0.003 8 0.027 7 0.023 8 0.027 9 0.030 7 0.023 ca 0.003 9 0.030 9 0.030 7 0.023 7 0.023 9 0.030 si 0.003 7 0.023 7 0.023 9 0.030 9 0.030 7 0.023 rda 0.003 8 0.025 8 0.025 5 0.015 5 0.015 8 0.025 sum 8.782 8.872 8.593 8.503 8.779 ijahp article: kamath, naik, prasad/ vendor’s evaluation – using ahp for an indian steel pipe manufacturing company international journal of the analytic hierarchy process 458 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.460 table 25 api and is vendor’s ranking api is sl.no vendors weights vendors weights 1 vendor a 8.872 vendor a 8.872 2 vendor e 8.782 vendor p 8.782 3 vendor b 8.593 vendor q 8.779 4 vendor c 8.503 vendor b 8.593 5 vendor d 8.496 vendor c 8.503 thus, from the above study it can be shown that vendor a for both api and is specification have the highest weight. 4. results and discussion from the above analysis, it can be determined that the quality criterion is the most important factor in vendor evaluation with the preference score of 0.652. it was also found that delivery is the second most important criterion with the score of 0.213 followed by the cost (0.088) and vendor relationship management (0.047). from the scores computed for the various criteria it can be concluded that the quality of the raw material carries much weight when compared to the other criteria with respect to the firm being studied. the quality followed by price, profile of the vendor and service are the most important contributors for selecting a vendor. this proves earlier studies that state that quality is an important criterion for vendor evaluation (tam & tummala, 2001). tahriri, osman et. al (2008) support the results of this study since they also suggested that quality is an important criterion followed by delivery, cost and responsiveness of the vendors. in order for companies to be sustainable in the competitive market, it is necessary for the firm to provide the best quality product. therefore, quality is an attribute which is a growing indicator for the success of the firm, and the quality of the raw material is essential for the company to provide the best quality product. vendor e should be selected as the best raw material vendor for both api and is specification (8.872) according to the ranking shown in table 25. in the case of api specification vendors, it can be noted that the global weights of vendors a and e quality criterion levels are same (5.864) (figure 2). nevertheless, when it came to the delivery criterion vendor a was ranked highest with the score of 1.918 compared to vendor e at 1.825. therefore, vendor a is given the highest preference. in the case of is specification vendors, it can be noted that the global weights of vendors a, p and q quality criterion levels are same (5.864) (figure 3). at the same time, when it came to the delivery criterion, vendor a was ranked highest with the score of 1.918 compared to vendors p and q at 1.825. vendor p was ranked higher than q when compared to cost criterion. hence, vendor a was ranked the highest compared to all the other vendors evaluated based on the overall weights computed. though the weights of vendor a were low in vrm when compared to the other vendors, it is still ranked highest as the weights of the quality and delivery criteria are more compared to the other criteria. for xyz company, the combined criteria rank vendor a as the best vendor. the company has to focus more on quality even though its objective is to reduce the cost with the maximization of profit in the supply chain to make sure that their customers are satisfied. ijahp article: kamath, naik, prasad/ vendor’s evaluation – using ahp for an indian steel pipe manufacturing company international journal of the analytic hierarchy process 459 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.460 5. conclusion every organization should integrate the evaluation and selection process of vendors with the fundamental long term decisions of the firm. the performance of a firm’s supply chain activities are directly influenced and the vendor evaluation. hence, to ensure the maximization of the supply chain activities the process of evaluating, selecting and managing the vendor is very important. the fundamental objective of the process of vendor evaluation is to accomplish world class quality of the product, minimize the cost, speed up delivery times, and attain the best services from the vendor (tam & tummala, 2001). with this in mind, the ahp technique was used and applied for the selection of criteria for vendor evaluation for a steel pipe manufacturing company to assess the best api and is vendors. the mathematical approach given by saaty (1980) to assign weights for the criteria and sub-criteria was very effective for the evaluation of vendors and selection of the best vendor for the manufacturing company being studied. areas that need improvement can be determined from the vendor evaluation process. vendor evaluation helps the organization increase production by ranking vendor qualities. depending on the various criteria, the alternate vendors can be ranked by the executives of the firm who can make appropriate consideration before ordering the raw materials. strengths and weaknesses of the various vendors can also be determined by this technique within specific criteria. firms with a huge expenditure on raw materials need to evaluate vendors and select those who provide excellent value for purchases made. 6. limitations this section discusses the limitations of the study. first, there is a possibility of response bias because the conclusions of this research were interpreted mainly by deputy general managers and managers of the procurement department of xyz company. therefore, it is suggested that future research overcome these issues by employing various methodologies such as focus groups, in-depth interviews, and brainstorming sessions with experts etc. second, the study was conducted specifically for xyz company, an indian steel pipe manufacturing industry, and may not be appropriate for other industries or other parts of the world. third, only a few variables were considered for this study. other significant factors such as business ethics, government policies, social, political and cultural factors could be included in the future. ijahp article: kamath, naik, prasad/ vendor’s evaluation – using ahp for an indian steel pipe manufacturing company international journal of the analytic hierarchy process 460 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.460 references alsuwehri, y. n. (2011). supplier evaluation and selection by using the analytic hierarchy process approach. engineering management field project, graduate school of science doctoral dissertation, the university of kansas. coulter, e., coakley, j., & sessions, j. (2006). the analytic hierarchy process: a tutorial for use in prioritizing forest road investments to minimize environmental effects. international journal of forest engineering, 17(2), 51–70. dickson, g. w. (1966). an analysis of vendor selection systems and decisions. journal of purchasing, 2(1), 5–17. ghodsypour, s. h. & o’brien, c. (1998). a decision support system for vendor selection using an integrated analytic hierarchy process and linear programing. international journal of production economics 56-57(1),199-212. doi:10.1016/s09255273(97)00009-1.x. giunipero, l. c. & pearcy, d. h. (2000). world class purchasing skills: an empirical investigation. journal of supply chain management, 36(3), 4–13. doi: 10.1111/j.1745493x.2000.tb00081.x ho, w., xu, x., & dey, p. k. (2010). multi-criteria decision making approaches for supplier evaluation and selection: a literature review. european journal of operational research, 202(1), 16-24. doi:10.1016/j.ejor.2009.05.009. x. imeri, s. (2013). key performance criteria for vendor evaluation a literature review. journal of management research and practice, 5(2),63–75. john, k., baby, v. y., & mangalathu, g. s. (2003). vendor evaluation and rating using analytical hierarchy process. international journal of engineering science and innovative technology, 2(3), 447–455. kannan, v. & tan, k. (2002). supplier selection and assessment: their impact on business performance. the journal of supply chain management: a global review of purchasing and supply, 38(4), 11–21. doi: http://dx.doi.org/10.1111/j.1745493x.2002.tb00139.x. mowshowitz, a. (1997). virtual organization. communications of the acm, 40(9), 30– 37. doi: http://dx.doi.org/10.1145/260750.260759. saaty, t. l. & tran, l. t. (2007). on the invalidity of fuzzifying numerical judgments in the analytic hierarchy process. mathematical and computer modelling, 46(7-8), 962– 975. doi: http://dx.doi.org/10.1016/j.mcm.2007.03.022 saaty, t. l. (1980). the analytic hierarchy process. new york: mcgraw-hill. doi: http://dx.doi.org/10.1080/00137918308956077. http://dx.doi.org/10.1016/j.mcm.2007.03.022 http://dx.doi.org/10.1080/00137918308956077 ijahp article: kamath, naik, prasad/ vendor’s evaluation – using ahp for an indian steel pipe manufacturing company international journal of the analytic hierarchy process 461 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.460 saaty, t. l., & hu, g. (1998). ranking by eigenvector versus other methods in the analytic hierarchy process. applied mathematics letters, 11(4), 121–125. sonawane, s., & rodrigues, l. l. (2015). evaluation of suppliers’ performance–quality aspect using ahp & system dynamics techniques. international journal of science and research, 4(6), 2124–2128. tahriri, f., osman, m., ali, a., yusuff, r., & esfandiary, a. (2008). ahp approach for supplier evaluation and selection in a steel manufacturing company. journal of industrial engineering and management, 1(2), 54–76. doi: http://dx.doi.org/10.3926/jiem.v1n2. tam, m. c. y. & tummala, v. m. r. (2001). an application of the ahp in vendor selection of a telecommunications system. omega, 29(2), 171–182. doi: http://dx.doi.org/10.1016/s0305-0483(00)00039-6. weber, c. a., current, j. r., & benton, w. c. (1991). vendor selection criteria and methods. european journal of operational research, 50(1), 2–18. doi: http://dx.doi.org/10.1016/0377-2217(91)90033-r. yahya, s., & kingsman, b. (1999). vendor rating for an entrepreneur development programme: a case study using the analytic hierarchy process method. journal of the operational research society, 50(9), 916-930. doi: http://dx.doi.org/10.2307/3010189. yu, x., & jing, s. (2004). a decision model for supplier selection considering trust. chinese business review, 3(6), 15–20. issn1537-1506. yusuff, r. m., yee, k. p., & hashmi, m. s. j. (2001). a preliminary study in potential use of the analytical hierarchy process (ahp) to predict advanced manufacturing technology (amt) implementation. robotics and computer integrated manufacturing, 17(1),421–427. doi: http://dx.doi.org/10.1016/s0736-5845(01)00016-3 http://dx.doi.org/10.1016/s0305-0483(00)00039-6 http://dx.doi.org/10.1016/s0736-5845(01)00016-3 0 final_preface_mu ijahp editorial: mu, e. / words from the editor-in-chief international journal of the analytic hierarchy process 1 vol. 4 issue 1 2012 issn 1936-6744 june 2012 dear ijahp readers: we are quite happy to enter our fourth year of publication and we start our fourth volume with a special issue on “ahp/anp multicriteria decision making for sustainable development.” i will let our guest editor, dr. fabio defelice of the university of cassino in italy, introduce the special issue himself; however, i would like to indicate that this special issue would not have been possible without the efforts of dr. antonella petrillo, also of the university of cassino, who became the point person for the collection and follow up of the different papers. also, this special issue required an extraordinary effort of a great number of anonymous reviewers who contributed with their time and expertise to review and improve the papers. finally, and as usual, rozann saaty, ijahp managing editor, was key to getting all the papers ready for publication. now i will let fabio defelice introduce this special issue. enrique mu, phd ijahp, editor-in-chief rob typewritten text http://dx.doi.org/10.13033/ijahp.v4i1.146 rob typewritten text ijahp editorial: mu, e. / words from the editor-in-chief international journal of the analytic hierarchy process 88 vol. 4 issue 2 2012 issn 1936-6744 december 2012 dear ijahp readers: the first four articles in this issue focus on manufacturing. it was not our intention to have a special issue on this topic but interestingly enough, there were so many articles in this area that we decided to put them together in this issue. however, there is one important article, compatibility indices between priority vectors, by garuti and salomon, discussing a theoretical topic of great practical importance: how to know how close two prioritizations (e.g. by two different decision makers) are? given the importance of this question we decided to include it in this issue so this discussion could be made available to the ahp community as soon as possible. also, this issue includes an essay about how to get articles published in our journal and offers some valuable tips to would-be authors. this essay discusses the general criteria used by the editorial board to decide whether an article should be submitted to full review (double blind), be modified and resubmitted, or simply is not suitable to enter the ijahp review and publication process. with this issue we close our fourth year of peer-reviewed regular publication. we believe we are ready to start our round of applications to get the journal properly indexed. as of this date we have applied to doj and scopus but we are exploring all the different well-known indexing organizations available to us. this is our homework for this coming year 2013. also, beginning with this issue we have started publishing with an attractive cover designed by antonella petrillo who joined our editorial board for the special issue this year, vol. 4, issue 1, "ahp/anp multicriteria decision making on sustainable development", and who is staying on now on a permanent basis as our assistant managing editor. welcome, antonella! and to our readers, reviewers, and authors, a big thanks! without your participation this flagship journal for our ahp/anp community would not be possible. happy holidays! sincerely, enrique mu, phd ijahp, editor-in-chief rob typewritten text http://dx.doi.org/10.13033/ijahp.v4i2.157 with this issue we close our fourth year of peer-reviewed regular publication. we believe we are ready to start our round of applications to get the journal properly indexed. as of this date we have applied to doj and scopus but we are exploring all t... ijahp news: ferretti/ youngstown state university awards professor birsen karpak title of distinguished professor international journal of the analytic hierarchy process 371 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.344 youngstown state university awards professor birsen karpak title of distinguished professor valentina ferretti it is both a pleasure and an honor to announce and celebrate an important award that professor birsen karpak, one of the members of the editorial board of the international journal of the analytic hierarchy process, recently received. in april 2015, birsen karpak, professor of management at youngstown state university, received a distinguished professorship designation. this is a result of her receiving distinguished professor awards in the areas of teaching, service and scholarship over the years! this formal title denotes an academic honor of the highest distinction, and is awarded to a limited number of the most accomplished professors. there are about 400 professors at youngstown state university, and only 2% are distinguished professors. moreover, birsen is the only professor with this honor in the williamson college of business. this award is thus an important recognition and acknowledges the difficulty in performing outstanding scholarship, research and creative activity, excellent teaching and exceptional service at the same time! besides performing superior service to the youngstown state university and the wider ahp community, and being an outstanding scholar, recognized at the international level, birsen is also an outstanding teacher and a continuous source of inspiration for her students. birsen has also been very active and passionate in motivating and mentoring young researchers like me! many thanks to birsen and warm congratulations from all of us! professor birsen karpak during the award ceremony on april 16, 2015 at the debartolo stadium club true translation from persian text ijahp article: alitaneh /theories on coefficient of variation scales triangle and normalization of different variables: a new model in development of multiple criteria decision analysis international journal of the analytic hierarchy process 283 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.565 theories on coefficient of variation scales triangle and normalization of different variables: a new model in development of multiple criteria decision analysis saeed alitaneh 1 m.sc., birjand university, iran saeed.alitaneh@gmail.com abstract this paper is an attempt to solve various problems by the two factors of mean and standard deviation (sd) of variables, introducing coefficient of variation (cv) of data as the best option for prioritization, scaling, pairwise comparison and normalization of quantitative and qualitative variables. an algorithm was built based on a coefficient of variation scales triangle (cvst) consisting of natural numbers with coefficients of binomial expansion for each line, followed by new and independent grading and scaling. in view of the existing factors, the theory provides higher generalization and maximum reliability rates in comparison to other methods for multiple-criteria decision analysis (mcda). on the other hand, in the normalization process of different variables (i.e. de-scalarization), a precise and infinite model was presented based on coefficient of variation scale triangle (multipurpose triangle), in such a way that decision makers could work with the software in a more convenient and precise manner. therefore, the proposed theories may be considered as a new approach and definition in the valuation and progress of mcda. keywords: theories; coefficient of variation scales triangle (cvst); multiple-criteria decision analysis (mcda) 1. introduction the analytic hierarchy process (ahp) is a multi-criteria decision making (mcdm) method that helps the decision-maker who is facing a complex problem with multiple conflicting and subjective criteria. several papers have compiled ahp success stories in many different fields (vargas, 1990; ho, 2008; golden, wasil, & harker, 1989; harker & vargas, 1990; shim, 1989; saaty, & forman, 1992; forman, & gass, 2001; kumar, & vaidya, 2006; omkarprasad & sushil, 2006; liberatore, & nydick, 2008; zahedi, 1986). the oldest reference found is saaty (1972a). after this, saaty (1977b) precisely described the ahp method. the vast majority of applications still use the ahp as described in this first publication and are unaware of successive developments. the analytical hierarchy process (ahp) technique, developed by thomas l. saaty, is an mcda method that helps decision makers make the best decisions in the face of complex problems consisting of multiple conflicts and internal criteria. the method has already been tested and established in many various fields of work (saaty, 1980c). recently, the application of ahp in animal science for 1 the author is deeply indebted to the late dr. saaty whose contribution to this field will be remembered for years. may his name and memory stay alive throughout eternity. ijahp article: alitaneh /theories on coefficient of variation scales triangle and normalization of different variables: a new model in development of multiple criteria decision analysis international journal of the analytic hierarchy process 284 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.565 the selection of the best dairy cows was applauded by researchers and scholars (alitaneh, naeeimipour, & golsheykhi, 2015). though the ahp covers many of the existing issues in this area, it loses most of its functionality of correlation between various factors, since the assumption is not valid for studying the effects of the interior and exterior environment. this major limitation led the developer of the ahp method to work on and present the analytic network process (anp). this new method takes into account the intertwined relations of decision making elements by replacing the hierarchical structure with a network one. the analytic network process is considered a more expanded version of the ahp (saaty, 1999d). in general, in the ahp method the question that is asked in the pairwise comparisons is which of the 2 elements is more effective. for the same purpose, saaty proposed that the intensity detected for inter-factor comparisons be graded on a scale of 9. thus, based on relative significance the scale of 9 shows that a factor is more significant than another while the scale of 1 shows no difference or equal significance. in ahp, once the relative weight vectors are calculated, saaty suggests a consistency rate (cr) of 0.1 for reliability and acceptance of a judgment on the pairwise comparisons matrix. otherwise, further study of the problem and re-evaluation of the matrices is recommended. briefly, harker, and vargas (1987) evaluated a quadratic and a root square. lootsma (1989) argued that the geometric scale is preferable to the 1–9 linear scale. salo and hamalainen (1997) point out that the integers from 1-9 yield local weights, which are unevenly dispersed, so there is lack of sensitivity when comparing elements, which are preferentially close to each other. based on this observation, they proposed a balanced scale where the local weights are evenly dispersed over the weight range [0.1, 0.9]. earlier ma, and zheng (1991) calculated a scale where the inverse elements x of the scale 1/x are linear instead of the x in the saaty scale (see table 1). ijahp article: alitaneh /theories on coefficient of variation scales triangle and normalization of different variables: a new model in development of multiple criteria decision analysis international journal of the analytic hierarchy process 285 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.565 table 1 different scales for comparing two alternatives scale type values linear (saaty,1977b) 1 2 3 4 5 6 7 8 9 power (harker, and vargas, 1987a) 1 4 9 16 25 36 49 64 81 root square (harker, and vargas,1987b) 1 1.41 1.73 2 2.23 2.45 2.65 2.83 3 geometric (lootsma,1989) 1 2 4 8 16 32 64 128 256 inverse linear (ma, and zheng, 1991) 1 1.13 1.29 1.5 1.8 2.25 3 4.5 9 asymptotical (dodd, and donegan,1995) 0 0.12 0.24 0.36 0.46 0.55 0.63 0.70 0.76 balanced (salo and hamalainen, 1997) 1 1.22 1.5 1.86 2.33 3 4 5.67 9 logarithmic (ishizaka, balkenborg, and kaplan, 2006) 1 1.58 2 2.32 2.58 2.81 3 3.17 3.32 in this research and based on balanced applied theories of grading, some sort of a 9 grade scale was proposed for scalar triangles of cv (cvst), and a type of new approach for de-scalarization of quantitative variables (normalization). the resulting triangular analysis has some special features, including: 1. the ability to grade in higher scales based on the researcher’s decision. 2. reasonable and orderly mathematical structure (binomial expansion). 3. achieving a smaller consistency rate in comparison to saaty’s cr. 4. no limitation in terms of number of variables (n variables) through de scalarization of measurement units based on a range of 1-9 scales. 5. easy registration of analyzed data in software applications based on 1-9 scales. generally, the relative scales triangles are based on cv of natural numbers, and since mean and sd parameters are taken together, the decision maker is able to choose the best criteria and target based on pairwise comparisons. saaty (1977b) showed that the geometric mean is the most suitable mathematical rule for combining ahp judgements. in scale triangle theory however, a scale is used that takes not only the mean of natural numbers but also the variance and sd. thus, it seems cv is a better and more precise method for normalization of numbers, followed by pairwise comparisons with the geometric mean or other methods. on the other hand, the ahp method uses nominal mean and scales (linear de-scalarization) for prioritization of the ijahp article: alitaneh /theories on coefficient of variation scales triangle and normalization of different variables: a new model in development of multiple criteria decision analysis international journal of the analytic hierarchy process 286 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.565 normalization process and pairwise comparisons. in the present paper, normalization and de-scalarization of quantitative variables is based on a scale triangle (multidimensional triangle). 2. data and methodology 2.1 coefficient of variation (cv) when calculating the dispersion of data, one always faces data measured with various scales. thus, to compare the dispersion of data collected from the statistical population measured with various scales, the use of only mean or variance does not provide accurate results as both are dependent on measurement scales. for the same reason, a more reliable scale such as coefficient of variation (cv) is used, which is calculated by dividing the standard deviation (sd) on the mean according to the following equation. 𝐶𝑉 = 𝜎 𝜇 (1) as can be observed, mean (µ) and standard deviation (σ) lack the required level of precision on their own, and may not be considered as the target criteria. the cv is free of dimensions and for the same reason it is suitable for comparison of statistical data with various measurement units. it is therefore quite good for normalization and de-scalarization of different variables and factors (everitt, 1998). this paper attempts to explain the application of the scalar triangle in coordination and comparing various decisions. the scalar triangle helps decision makers conceptualize their decision options in an integrated manner using one scalar structural model, such that the optimal decision includes the ideas of all members and the decision maker. it may then be used as the best tool for making decisions in the shortest possible time. this paper provides an overview of the various stages of decision making and normalization of quantitative and qualitative variables using a scalar triangle in comparison to the hierarchical tree. 2.2 scales triangle the first step includes the creation of a triangular structure of natural numbers (n). in order to have a 9-step grading system the range of numbers was defined as 1-54 and the position of each number in the triangle was called a “cell” (0<n≤54). the purpose of this triangle is to express the studied problem in view of each topic and it needs to have easy software navigation in order to enable quick and optimum decision making. the triangle is graded in 9 lines. however, there is room for expanding the scaled spectrums and the structure of the scalar triangle theory based on n increased number of cells and lines (expansion of the definition of natural numbers in cells, lines and scales). basically, each horizontal axis (x) of the triangle is a set of several numeral cells expressing one line of decision making (see table 2). thus, based on the triangular structure of the table below and the related descriptions, the triangular scales are calculated and analyzed in 9 lines. first, as a can be seen in the table 2, for each line a mathematical function is defined based on the coefficients of binomial expansion, such that the computational potential of each line is an ordinal number specific to the same line. ijahp article: alitaneh /theories on coefficient of variation scales triangle and normalization of different variables: a new model in development of multiple criteria decision analysis international journal of the analytic hierarchy process 287 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.565 table 2 natural numbers triangle numbers equation line(l) 2 1 (𝑋𝑖 + 𝑌𝑗) 𝑙1 1 5 4 3 (𝑋𝑖 + 𝑌𝑗) 𝑙2 2 9 8 7 6 (𝑋𝑖 + 𝑌𝑗) 𝑙3 3 14 13 12 11 10 (𝑋𝑖 + 𝑌𝑗) 𝑙4 4 20 19 18 17 16 15 (𝑋𝑖 + 𝑌𝑗) 𝑙5 5 27 26 25 24 23 22 21 (𝑋𝑖 + 𝑌𝑗) 𝑙6 6 35 34 33 32 31 30 29 28 (𝑋𝑖 + 𝑌𝑗) 𝑙7 7 44 43 42 41 40 39 38 37 36 (𝑋𝑖 + 𝑌𝑗) 𝑙8 8 54 53 52 51 50 49 48 47 46 45 (𝑋𝑖 + 𝑌𝑗) 𝑙9 9 thus, if l and k are normal numbers in the presumed triangle, for (x+y) l we have: l=1,2,…,n k=1,2,…,m (x+y) l =∑ ( 𝐿 𝐾 )𝑋𝐿−𝐾𝑌𝐾𝐿𝐾=1 → ( 𝐿 𝐾 ) = 𝐿! 𝐾!(𝐿−𝐾)! (2) the above equation is a fixed binomial coefficient. hence, the setting of the binomial expansion triangle for each line was based on this. in the next step, the x and y axes must be defined for each cell and the whole line numbers triangle. the basic assumption of the scalar triangle is that each cell is formed by the horizontal x vector (addition of all numbers on x axis) and vertical y vector (multiplication of numbers on the y axis). 𝐴 = {(𝑥, 𝑦)|𝑥, 𝑦 ∈ {1,2, … , 𝑛}, 𝑥 = 𝑦} (3) moreover, to determine the exponent of each line, and if nn is a set of natural numbers with a finite range, for each line we have the following condition: 𝑁𝑛 = {1,2, … , 𝑛} 𝑓: 𝑁𝑛 → 𝐿 𝑓(𝑛) = 𝑙𝑛 (4) besides taking the 9 assumed lines in the triangle, the paper goes on to define the ln condition of a set of natural numbers up to l9. thus, the basics for the exponent of each line equation were defined according to ln condition. now, the final equation for each line of the triangle may be defined as follows: 𝑇𝑖𝑗 = ∑ (𝑋𝑖 + 𝑌𝑗) 𝑙𝑛𝐿 𝑖=1 (5) now, whenever a total of x values and the numeric value of each cell is equal to the last line of the assumed triangle, we have, the x value on the horizontal axis of each cell of the triangle is equal to: 𝑋𝑖 × 𝐶𝑖 = 𝐿9 → 𝑋𝑖 = 𝐿9 × 𝐶𝑖 (6) ijahp article: alitaneh /theories on coefficient of variation scales triangle and normalization of different variables: a new model in development of multiple criteria decision analysis international journal of the analytic hierarchy process 288 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.565 on the other hand, if the result of multiplication of y in the numerical value of each the cell is assumed to be equal to the last line of the triangle, we have, the y value on vertical axis of each triangle cell is equal to: 𝑌𝑗 + 𝐶𝑖 = 𝐿9 → 𝑌𝑗 = 𝐿9 − 𝐶𝑖 (7) to prevent dispersion, and in favor of the normal distribution of data and variables, a base 10 logarithm condition was added to the equation. 𝑇𝑖𝑗 = log10[(𝐿9 × 𝐶𝑖 + 𝐿9 − 𝐶𝑖) 𝑙𝑛] (8) tij: the final number for each cell from numbers triangle equation, where each cell includes a presumed number (tij). cj : the assumed number for each cell in the scales triangle is between 1 and 54 l9 : the assumed number for the last line of the triangle is the natural number 9 ln : the exponent value assumed for each line, which is between natural numbers 1 and 9 in the scales triangle. finally, in order to make a unified and outstanding curve of the balance of the x and y vectors, each natural number existing in each cell (1 to 54) is calculated through the overall ti equation, therefore creating a set of numbers with normal and balanced distribution (see table 3). table 3 balance numbers for the cvst (xi + yj) 2 3.037 3.226 3.380 (xi + yj) 3 5.268 5.439 5.590 5.725 (xi + yj) 4 7.798 7.947 8.085 8.212 8.331 (xi + yj) 5 10.553 10.684 10.807 10.923 11.034 11.139 (xi + yj) 6 13.488 13.603 13.713 13.819 13.921 14.019 14.113 (xi + yj) 7 16.571 16.674 16.773 16.870 16.963 17.053 17.141 17.226 (xi + yj) 8 19.782 19.874 19.964 20.052 20.138 20.221 20.303 20.382 20.460 (xi + yj) 9 23.103 23.187 23.269 23.350 23.428 23.506 23.581 23.656 23.728 23.800 now, in one of the most significant steps and according to equation (8), the total of the normal data for each is calculated for each line using the cv equation and the resulting numbers are introduced as random indices of the scales triangle. by finding a random index equation, one can attain the final goal of this paper, which is prioritization and special scaling/grading by means of a triangular structure of natural numbers (see table 4). it must be noted that a random index is needed for the calculation of an inconsistency rate. nevertheless, in the description and definition of this theory there is no need for the use of a random index in the scales triangle. another advantage of this model is that it can make use of a random index of hierarchical analysis in the calculation of inconsistency rates of numbers triangle. ijahp article: alitaneh /theories on coefficient of variation scales triangle and normalization of different variables: a new model in development of multiple criteria decision analysis international journal of the analytic hierarchy process 289 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.565 table 4 random index (ri) for the cvst in the final step, after the cv is calculated and the values of random indices are set, the final scale (grading) is also calculated in the scales triangle model through the following equation: wij = ximax xij (9) 2.3 for example (line 2) 𝑇𝑖𝑗 = log10[(9 × 3 + 9 − 3) 2] = 3.037 𝑇𝑖𝑗 = log10[(9 × 4 + 9 − 4) 2] = 3.226 𝑇𝑖𝑗 = log10[(9 × 5 + 9 − 5) 2] = 3.380 𝐶𝑉𝑇 = 3.037 , 3.226 , 3.380 = 5.34 (ri) wij = 9 5.34 = 1.6 (equally to moderately) where, wij is the final scale or weight, ximax is the largest relative index, and xij is each of the relative indices. therefore, the final scale was determined according to the analysis of a scales triangle of cv through formulation of natural numbers as described below (table 5). table 5 pairwise comparison scale for cvst preferences one of the most important advantages of using scale triangles is the lower inconsistency rate in comparison to ahp, which is an indication of its higher precision in the analysis of different variables. for instance, pairwise comparison a is related to ahp scaling and matrix b is related to scales triangle scaling (cvst). 10 9 8 7 6 5 4 3 2 1 size of matrix 0.87 1 1.15 1.35 1.62 2 2.61 3.58 5.34 9 ri numerical rating (𝐖𝐢𝐣) verbal judgments of preferences 9 extremely preferred 7.8 very strongly to extremely 6.6 very strongly preferred 5.5 strongly to very strongly 4.5 strongly preferred 3.4 moderately to strongly 2.5 moderately preferred 1.6 equally to moderately 1 equally preferred ijahp article: alitaneh /theories on coefficient of variation scales triangle and normalization of different variables: a new model in development of multiple criteria decision analysis international journal of the analytic hierarchy process 290 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.565 a= [ 1 2 6 5 1/2 1/2 1 4 3 1/3 1/6 1/4 1 1 1/6 1/5 1/3 1 1 1/5 2 3 6 5 1 ] 𝜆𝑚𝑎𝑥 = 5.083 ci=0.0208 cr=0.0186 < 0.1 dividing all the elements of the weighted sum matrices by their respective priority vector element, we obtain: (.2909, .1693, .0547, .0621, .4230) b= [ 1 1.6 5.5 4.5 1 1.6 1 1.6 1 3.4 2.5 1 2.5 1 5.5 1 3.4 1 1 1 5.5 1 4.5 1 2.5 1 1 1 4.5 1.6 2.5 5.5 4.5 1 ] 𝜆𝑚𝑎𝑥 = 5.031 ci=0.0078 cr=0.0070 << 0.1 ok (.2953, .1805, .0615, .0708, .3919) comparing vectors by compatibility: g= 94.02% > 90% thus, priority vectors a and b are compatible vectors (equivalent vectors for measurement purposes). moreover, the inconsistency rates calculated by the saaty method and scales triangle method demonstrated that they had much better results than pairwise comparison a and scale triangle matrix b coefficient of variation. this proved the high precision and efficiency of the proposed method. in this method each cell is a function of an equation, and eventually the total of the equation of each given line is governed by the cv of the same line. table 5 provides good insight into the comparisons and preferences (oral judgment) of a scales triangle containing the significance of various factors. in summary, the basis for a scales triangle may be described in a few stages: 1. creating a triangular structure of natural numbers. 2. creating 9 numeric lines for pairwise comparisons. 3. defining binomial expansion functions for each line of the triangle (x+y) l . 4. calculating or approximating numbers in each cell based on a logarithmic formula (formulation). 5. calculating the cv of a set of numbers produced by the formulation of each line (e.g. in line 1). 6. finding a random index for each line. 7. finding the final rates through the random indices equation. 8. calculating and achieving smaller inconsistency rates. 9. better efficiency and precision in analysis of qualitative variables. 2.4 normalization of variables one of the most important issues in mcdm is the existence of various scales for quantity and quality indices. initially, it was just a lack of a standard for measurement ijahp article: alitaneh /theories on coefficient of variation scales triangle and normalization of different variables: a new model in development of multiple criteria decision analysis international journal of the analytic hierarchy process 291 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.565 of quality indices, for which saaty used a linear de-scalarization technique. instead of his technique, we have applied the scales triangle technique. the variable normalization method is also based on the scales triangle. in general, each variable must be normalized and correlated if it is to be measured and weighed along with other variables so that its measurement and weighting is in relation to all other variables. one the biggest problems of a decision maker is how to enter quantity values into related software (data bigger than 9 and less than 1). the “narrow” range of 1 to 9 should not be a problem when the model is correctly made as can be observed in research works related to medicine, biology and animal science. for instance, there are two dairy cows. cow 1 has excellent body weight, a daily milk production of 110.27 lbs, a somatic cell count of 1459, and milk impurity of 0.83%. cow 2 has medium body weight, a daily milk production of 95.08 lbs, a somatic cell count of 2610, and a milk impurity rate of 0.47%. the decision maker uses a software application to analyze quality variables based on existing scales, but is unable to make suitable analyses in a short time and with high precision in terms of milk production, somatic cells, and impurity rate. also, the ahp/anp software needs considerable time for pairwise comparisons between various groups due to the existence of various units and traits. in order to address such problems, the present study focuses on making precise and actual estimations of comparisons and attempts to develop a suitable model for normalization of quantity variables (de-scalarization). as noted earlier, when using cv for calculation of the mean there is a need for more than one data or variable, and since this is not possible for data available on dairy cattle (each cow has a record of various factors), one must look for a model capable of properly normalizing all records and variables based only on the one reported data. let’s assume a set of rational numbers as a set whose numbers may be written in the following general format: q ={x= a/b | (a,b) z , b < 0} (10) in this method, the range of normalization of a quantity variable is a set of numbers bigger than zero (rational numbers) which must be standardized if the primary variable is to be normalized. thus, the first practical step is to calculate the square root and then the inverse power of the last line of the triangle is used to formulate and finalize the two equations. it must be emphasized again that to analyze the cv of a number there must be more than one data, variable, record, parameter, etc. for the same reason, two equations were extracted from the scales triangle with the maximum continuity and correlation, and their cv was analyzed with minimum error. besides, in order to analyze two desirable and balanced equations tq which is related to the mathematical model of scales triangle, the following steps were followed:  at first, if q is taken as a number bigger than zero and its square root is calculated, we have: x𝑞 = √q  after that, using the equation of the last line of triangle and dividing the power of the first line by that of the last line, the first equation (tq1) was produced, and then the second equation (tq2) included the result of the first equation and the primary variable. at the end of this process, the two ijahp article: alitaneh /theories on coefficient of variation scales triangle and normalization of different variables: a new model in development of multiple criteria decision analysis international journal of the analytic hierarchy process 292 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.565 standard and balanced equations tq1 and tq2 were formulated with maximum correlation and reliability: 𝑇𝑞1 = log10[(𝐿9 × 𝑥𝑞 + 𝐿9 − 𝑥𝑞) 1/𝑙9 ] (11) 𝑇𝑞2 = log10[(𝐿9 × 𝑥𝑞 + 𝐿9 − 𝑇𝑞1) 1/𝑙9 ] (12) in these two equations, due to reduced significant error and maximum reliability of the analysis result, the values of tq1 and tq2 showed better numerical correlation, as confirmed by the correct and relevant results obtained from calculation of their numerical value and cv (2 variables) through equation 13. interestingly, in analysis of numbers bigger than 0 the cv was on a 9-grade scale. it must be noted that this mathematical model is capable of numerical calculations up to n numbers. it must be emphasized that the obtained cv was also normal and lacked statistical units (de-scalarization). 𝐶𝑉𝑇 = ∑ 𝑇𝑞1 𝑛 𝑞>0 𝑇𝑞2 (13) generally, normalization by a scales triangle goes through the following stages: 1. getting the square root of the related data or variable. 2. formulating two equations based on the last line of the triangle. 3. finding more than one data (at least 2) for the cv calculation. 4. calculating the cv for the two data. 5. normalizing several variables with various scales, grading and performing pairwise comparisons. based on this and a normalization procedure on scales triangle, for example the records of dairy cow production rates in table 6, we have: xq = √110.27 = 10.5 𝑇𝑞1 = log10[(9 × 10.5 + 9 − 10.5) 1/9] = 1.0142 𝑇𝑞1 = log10[(9 × 10.5 + 9 − 1.0142) 1/9] = 1.0564 𝐶𝑉𝑇 = 1.0142 , 1.0564 = 2.88 (normalized) table 6 normalizing several variables with various scales milk impurity (%) somatic cell count(number) daily milk (lb) body weight (quality trait) cow 1 0.83 1459 110.27 excellent data 4.53 2.15 2.88 9 normalized milk impurity (%) somatic cell count(number) daily milk (lb) body weight (quality trait) cow 2 0.47 2610 95.08 medium data 4.67 2.03 2.92 4.5 normalized ijahp article: alitaneh /theories on coefficient of variation scales triangle and normalization of different variables: a new model in development of multiple criteria decision analysis international journal of the analytic hierarchy process 293 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.565 as shown above, in this stage all quantity variables are scaled by means of the scales triangle. thus, the pairwise comparisons were performed with high speed and accuracy in comparison to other methods. 3. conclusion one condition for the efficiency of the above process is to prioritize research plans and projects using methods suitable to prioritization. based on the present paper, the theory of scales triangle (cvst) and normalization of quantitative and qualitative variables shows a high potential for extensive use in various fields. this is due to its high capacity in modeling actual problems, high speed of analysis and the ease of learning for users. though mathematical and statistical methods provide optimized results for planning and decision making, such techniques and models often require precise and definite data, which is a serious issue in actual conditions where it is difficult to collect data and takes a lot of time. hence, it may be said with near certainty that this new proposed method may play a major role in grading, normalizing (descalarization) of different variables and making hard decisions. it seems the theories of scales triangle and normalization of numerical variables is an integrated model for analysis for quantitative and qualitative variables. particularly, these theories are designed to show newer, better and faster paths with high usability potential and ease of software navigation. since grading methods of other scholars are based on saaty’s 1-9 scale analysis, it is safe to claim that the new model is independent of any existing grading methods. the scales may be increased or decreased based on increased number of cells and lines in the scales triangle (noting that while solving problems the equations must be calculated according to the last line (ln) of the triangle). the present paper attempted to ease the use of related software by prioritization into a 1-9 scale. efforts were also focused on putting the scaling into a process that is dependent on data and final weights. in the end and given the wide use of ahp/anp, it seems that more future works will focus on the scales triangle method. it is imperative to note that such decision making techniques, like all other methods, only serve to convert data into information for decision makers, and it is the decision maker who has to make the best choice. the method proposed here may lead to better results and changes in pairwise comparisons. the paper demonstrated, on the other hand, that the structure of a numbers triangle is capable of including and analyzing several equations at once. this may lead more researchers into the field in the near future. the author hopes these results play a significant role in the expansion and clarification of comparisons and decisions, and finally creates a unified framework for proper selection and a new applied method for multiple-criteria decision making analysis. finally, the results of this research in relation to a new scale, known as the coefficient of variation scales triangle (cvst), suggest it can be used along with other scales. also, the cvst model can be identified as a suitable and practical method. ijahp article: alitaneh /theories on coefficient of variation scales triangle and normalization of different variables: a new model in development of multiple criteria decision analysis international journal of the analytic hierarchy process 294 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.565 references alitaneh, s., naeeimipour, h., & golsheykhi, m. (2015). a new idea in animal science: the first application of the analytical hierarchy process (ahp) model in selection of the best dairy cow. iranian journal of applied animal science, 5(3), 553–559. dodd, f., donegan, h. (1995). comparison of prioritization techniques using inter hierarchy mappings. journal of the operational research society, 46(4), 492–498. doi: https://doi.org/10.1057/jors.1995.67 everitt, b. (1998). the dictionary of statistics. cambridge: cambridge university press. forman, e., gass, s. (2001). the analytic hierarchy process, an exposition. operations research, 49(4), 469–486. doi: https://doi.org/10.1287/opre.49.4.469.11231 golden, b., wasil, e., & harker, p. (1989).the analytic hierarchy process: applications and studies. heidelberg, germany: springer-verlag. harker, p., vargas, l. (1987). the theory of ratio scale estimation: saaty’s analytic hierarchy process. management science, 33(11), 1383–1403. doi: https://doi.org/10.1287/mnsc.33.11.1383 harker, p., vargas, l. (1990). reply to remarks on the analytic hierarchy process. management science, 36(3), 269–273. doi: https://doi.org/10.1287/mnsc.36.3.269 ho, w. (2008). integrated analytic hierarchy process and its applications, a literature review. european journal of operational research, 186(1), 211–228. doi: https://doi.org/10.1016/j.ejor.2007.01.004 ishizaka, a., balkenborg, d., & kaplan, t. (2006). influence of aggregation and preference scale on ranking a compromise alternative in ahp. proceedings of the multidisciplinary workshop on advances in preference handling, riva del garda, 51–57. doi: https://doi.org/10.2139/ssrn.1527520 kumar, s., vaidya, o. (2006). analytic hierarchy process: an overview of applications. european journal of operational research, 169(1), 1–29. liberatore, m., nydick, r. (2008).the analytic hierarchy process in medical and health care decision making: a literature review. european journal of operational research, 189(1), 194–207. doi: https://doi.org/10.1016/j.ejor.2007.05.001 lootsma, f. (1989). conflict resolution via pairwise comparison of concessions. european journal of operational research, 40(1), 109–116. doi: https://doi.org/10.1016/0377-2217(89)90278-6 ma, d., zheng, x. (1991). 9/9–9/1 scale method of ahp. 2nd international symposium on ahp, pittsburgh, 197–202. omkarprasad, v., sushil, k. (2006). analytic hierarchy process: an overview of ijahp article: alitaneh /theories on coefficient of variation scales triangle and normalization of different variables: a new model in development of multiple criteria decision analysis international journal of the analytic hierarchy process 295 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.565 applications. european journal of operational research, 169(1), 1–29. saaty, t. (1972). an eigenvalue allocation model for prioritization and planning. university of pennsylvania: energy management and policy center. working paper. saaty, t.l. (1980). the analytic hierarchy process. pittsburgh: rws publications. saaty, t.l. (1999). fundamentals of the analytic network process. isahp japan, 12–14. saaty, t.l. (1977). a scaling method for priorities in hierarchical structures. journal of mathematical psychology, 15, 234–281. doi: https://doi.org/10.1016/00222496(77)90033-5 saaty, t., forman, e. (1992). the hierarchy: a dictionary of hierarchies. pittsburgh, pa: rws publications. salo, a., hamalainen, r. (1997). on the measurement of preference in the analytic hierarchy process. journal of multi-criteria decision analysis, 6(6), 309–319. doi: https://doi.org/10.1002/(sici)1099-1360(199711)6:6%3c309::aidmcda163%3e3.3.co;2-u shim, j. (1989). bibliography research on the analytic hierarchy process (ahp). socio-economic planning sciences, 23(3), 161–167. doi: https://doi.org/10.1016/0038-0121(89)90013-x vargas, l. (1990). an overview of the analytic hierarchy process and its applications. european journal of operational research, 48(1), 2–8. zahedi, f. (1986).the analytic hierarchy process: a survey of the method and its applications. interface, 16(4), 96–108. doi: https://doi.org/10.1287/inte.16.4.96 ijahp article: allah pitchay, asmy mohd thas thaker, mohd thas thaker, amin/ determining the relative importance of quality factors for crowdfunding platforms international journal of the analytic hierarchy process 368 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.667 determining the relative importance of quality factors for crowdfunding platforms anwar allah pitchay universiti sains malaysia mohamed asmy mohd thas thaker 1 international islamic university malaysia corresponding author: asmy@iium.edu.my hassanudin mohd thas thaker sunway university, malaysia md. fouad bin amin king saud university, saudi arabia abstract a crowdfunding platform provides a facility for anyone with internet access to transmit an idea to a social network with the goal of gathering funding for their works or projects. a quality crowdfunding platform acts as an indicator of the donors’ ability to provide funds to the projects. in the existing literature on the quality factors of crowdfunding platforms, no systematic approach has been applied to prioritize these quality factors. the task of identifying the quality factors of a crowdfunding platform has been absent in the existing literature. the aim of this study is to use the analytic hierarchy process (ahp) to identify and rank the relative weights of the quality factors of crowdfunding platforms. the results show that ‘functionality’ emerged as the most crucial criterion. the research findings also provide insightful information to crowdfunding platform providers so that they may improve the effectiveness and efficiency of their platforms. keywords: crowdfunding platform; quality factors; prioritization; analytic hierarchy process 1. introduction the revolution of information and communication technology has transformed the world economy from production-based to knowledge-based. people prefer to stay connected and complete tasks efficiently and quickly, and it is in this environment that the internet has emerged. the internet offers new opportunities for soliciting and collecting voluntary contributions, and these opportunities are made available at a lower-cost, and in a wider coverage area and communications network (yan et al., 2006). indeed, the emergence of acknowledgement: the work described in this paper has been financially supported by the international islamic university malaysia (grant id: kenms-rg19-005-0005) ijahp article: allah pitchay, asmy mohd thas thaker, mohd thas thaker, amin/ determining the relative importance of quality factors for crowdfunding platforms international journal of the analytic hierarchy process 369 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.667 the internet has introduced various modes of opportunities to raise funds through online systems, especially crowdfunding platforms. a crowdfunding platform is an internet-based platform that connects fundraisers to funders with the aim of raising money for a particular campaign or project (belleflamme et al., 2015). the campaigns or projects have different features and the funders have diverse preferences concerning these campaigns or projects. the crowdfunding platform provides a facility for anyone with internet access to transmit their idea to a social network with the goal of gathering funding to realize their works or projects. these ideas can range across fields and vary in scope such as musician, academician, product designer, doctor, entrepreneur and educator (babich et al., 2018). the donors can contribute small amounts to specific projects, and in this way the "crowd" collectively contributes to the funding of the project. the process of raising money from the crowdfunding platform is in contrast to traditional fundraising efforts such as securing funds from banks, venture capitalists, and foundations (gerber et al., 2012). the creators or project initiators develop a profile on a crowdfunding platform and explain their monetary goals, planned use of the funds, and timeline for reaching their goals. the crowdfunding platform provides dedicated project pages, analytics and project monitoring, and tutorials prior to and throughout the campaign (gerber & hui, 2003). creators also rely on additional social media platforms like facebook, twitter and youtube to publicize their request for funds from supporters or the “crowd”. various platforms have emerged that offer entrepreneurs and project owners the possibility of raising money from an undefined group of online users (“crowd”). this shows that a crowdfunding platform involves matching the two sides of a market. a crowdfunding platform does not provide a one-to-one arrangement, but a one-to-many arrangement since many projects require more than one funder to reach the funding target and make it successful (mollick, 2014). meanwhile, the incentives provided by fundraisers to funders or donors are different and based on the type of crowdfunding platform. crowdfunding platforms can be divided into four categories, namely donation crowdfunding, reward crowdfunding, lending crowdfunding, and equity crowdfunding (the world bank, 2013). donation-based crowdfunding involves the collection of funds for a social, artistic, philanthropic or other purpose, and is not in exchange for anything of tangible value (massolution, 2015). meanwhile, reward-based crowdfunding is the collection of funds where the investors or donors receive some tangible reward (such as a membership rewards scheme) as a token of appreciation (massolution, 2015). lendingbased crowdfunding is an online platform that matches lenders or investors with borrowers or issuers to provide loans with a certain interest rate that is set by the platform (massolution, 2015). equity-based crowdfunding refers to funds raised online by a business, particularly early-stage funding, by offering equity interests in the business to investors (massolution, 2015). over the last five years, crowdfunding platforms have achieved tremendous growth. in 2014, crowdfunding platforms in europe made up around 60% of all cfps worldwide, while around 20% of all crowdfunding platforms are based in north america (massolution, 2015). in europe in 2014, the u.k. had the most crowdfunding platforms ijahp article: allah pitchay, asmy mohd thas thaker, mohd thas thaker, amin/ determining the relative importance of quality factors for crowdfunding platforms international journal of the analytic hierarchy process 370 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.667 with 70, followed by france with 70 crowdfunding platforms and germany with 50 crowdfunding platforms (massolution, 2015). on the other hand, the majority of campaigns (62%) use donation-based crowdfunding. only 15% of campaigns use reward-based crowdfunding, followed by 22% who use lending-based. equity-based campaigns contribute to less than one percent of the total campaigns (massolution, 2015). meanwhile, in terms of funding volume of crowdfunding platforms, lending-based crowdfunding and donation-based crowdfunding raised usd$1169.7 million and usd$979.3 million, respectively from 2010 to 2012 (iizuka, 2014). malaysia was the first country in southeast asia to give regulatory approval for equity crowdfunding in 2015 and peer-to-peer (p2p) financing in 2016. as of june 2019, there are 21 market-based financing platform operators registered in malaysia. the malaysian securities commission has approved a total of 10 equity-based crowdfunding platforms and 11 lending-based or p2p crowdfunding platforms. furthermore, malaysian equity crowdfunding and p2p financing platforms have collectively raised rm432 million as of june 2019. the funds which have been raised since 2015 have benefitted over 1,200 micro, small, and medium enterprises (msmes), and these financing campaigns attracted more than 12,000 investors, 90% of whom are local and retail investors. many researchers have started to examine different aspects of crowdfunding as the potential and importance these platforms play in raising funds has become clear. for example, research on crowdfunding focuses on the motivations of sponsors and entrepreneurs (barabas, 2012; gerber, hui, & kuo, 2012; ordanini, miceli, & parasuraman, 2011), and factors affecting crowdfunding performance in raising capital (lambert & schwienbacher, 2010; mollick, 2014; zheng et al., 2014). in addition, a few studies have explored the crowdfunding project implementation process to understand sponsors' feelings and experiences (mollick, 2014). some researchers have also pointed out that providing a good online platform is increasingly recognized as a critical factor for its successful implementation (lin, 2012). therefore, determining the quality factors of crowdfunding platforms is crucial for the service providers. it helps the service providers identify and focus on the factor with the highest priority and improve their platform effectiveness. despite the rapid growth of the online community of creators and supporters that are linked with crowdfunding platforms, to the best of the researchers’ knowledge, there are no studies that have empirically investigated the quality factors of crowdfunding platforms, particularly in the malaysian context. the consequence of this dearth of research is that many quality aspects of crowdfunding platforms remain poorly understood. the present study fills this research gap by determining the relative weights of the quality factors of crowdfunding platforms in malaysia. in order to extend the contribution of the study, the quality factors of mobile and online banking and e-commerce as a proxy for crowdfunding platforms are also used. consequently, this study attempts to apply the analytic hierarchy process (ahp) approach to determine the relative weights and priority of the quality factors of crowdfunding platforms. ijahp article: allah pitchay, asmy mohd thas thaker, mohd thas thaker, amin/ determining the relative importance of quality factors for crowdfunding platforms international journal of the analytic hierarchy process 371 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.667 2. literature review 2.1 quality factors in online fundraising platforms the quality factors have a significant role in the successful implementation of any online system or platform. there is a dearth of literature about the effects of the quality factors on crowdfunding platforms. therefore, the present study contributes significant value to the literature because it intends to expand the research to a different concept like crowdfunding platforms in the malaysian context. according to delone and mclean (2003), the success of information systems (is) depends on a few quality factors. for instance, the quality of mobile banking is determined by functionality, content, customer service and interface design. some previous studies that identified the quality factors associated with various mobile and online banking, and e-commerce applications are discussed below. 2.1.1 functionality the term functionality means the quality or state of being functional. it varies widely depending on the subject and the surrounding environment. it is a set of functions related to any device, such as computer software and hardware, which satisfy needs under certain conditions. in the case of mobile banking, for instance, service operators used to offer advanced functions for customer satisfaction such as guaranteed secured transactions, and easy cash receipt and transfer. they invest a huge amount of resources to diversify their functions. the scope and nature of the functionality of any device or system, therefore, remains constant over time. a few characteristics and attributes of functionalities, as mentioned by andreou et al, (2005), are provided here. i. interoperability: this includes three items: a) platform independence (can function in different web or mobile platforms), b) operating system independence (can function in different operating systems), and c) hardware compatibility (can operate in various devices). ii. suitability: this has three different aspects: a) user satisfaction (meeting the user needs), b) service satisfaction (fulfilling the required services), and c) achievability (rendering more services than expected). iii. security: this has mainly two features: a) data encryption (protecting the data), and b) access control (authentication and authorization of the users for accessing the services). iv. compliance: this is comprised of two standards: a) network standards (supported in different networks), and b) communication standards (supporting different network standards). v. accuracy: this is concerned with correctness. the application must generate correct or accurate results with zero margin of error. in addition to the above characteristics of functionalities, hsiu-fen lin (2013) considered four similar attributes that include accessibility, response time, mobility, and security. according to brüntje and gajda (2016), the functionality of a crowdfunding platform has a direct impact on the channeling of investors’ funds. the crowdfunding platform’s functionality consists of the application phase, screening and selection phase, and evaluation phase. all of these phases are essential for the success of the fundraising. the present paper uses the functionality factor to measure its effect on the quality of the crowdfunding platform in the malaysian context. ijahp article: allah pitchay, asmy mohd thas thaker, mohd thas thaker, amin/ determining the relative importance of quality factors for crowdfunding platforms international journal of the analytic hierarchy process 372 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.667 2.1.2 content the meaning of content depends on where the term is being used. for instance, web content is a type of presentation of ideas and messages in the form of texts, images, sounds, videos, and animations, whereas media content is a collection of information or real facts provided to end-users through various media channels i.e., tv, social network, newspapers etc. in the case of mobile content, there are certain key aspects that need to be emphasized such as mobile phone screen, map, gps, graphics, multimedia, sms option, and application for social networks etc. previous studies have been conducted on the importance of content in website design and mobile phone banking. chae et al. (2002) recommend that content is an important factor regardless of the nature of the web site i.e., web-based or wireless. venkatesh and ramesh (2002) identified subcategories of content and found that its relevance is more effective in the wireless network than in the web context. chae et al. (2002) named four categories of information quality of mobile internet services which are connection, content, interaction, and context. only the content category of information quality was found to be significant in affecting users' attitude toward mobile information quality. this finding is similar to vlachos et al. (2003), vlachos and vrechopoulos (2005), and chan and ahern (1999) who suggested that the quality of the content plays a key role in determining the quality of mobile entertainment services. taylor (1986) introduced the concept of quality in his value-added model. it is mostly used for the evaluation of information systems, information and data. since the content is more closely related to building a framework or information system, taylor's recommendations consisted of five characteristics that are required to develop any system or package: a) accuracy is an attribute that adds value to the system by confirming the transfer of error-free data and information, b) comprehensiveness also adds value by providing complete coverage of a definite subject or discipline, c) currency enriches the system by updating the data, d) reliability is an essential element that enhances not only the quality of the system but also the output over a long time, and e) validity which is ensured by the data presented in the system being valid and sound. dresner (2014) mentioned that a crowdfunding platform needs to issue a prospectus with specified content including how the capital that is raised will be used, recent financial performance, and projects. the content includes updates, comments, images and video, and results. yao and zhang (2014) found that the information or content quality disclosed on the online platform has a positive effect and to be influential on crowdfunding project financing. 2.1.3 customer service customer service provides a set of services to the customers before, during and even after the purchase event. it is a process where customers and service providers can communicate through face to face interaction, phone calls, or self-service systems etc. a successful customer service provider is someone "who can adjust themselves to the personality of the guest" (buchanan, 2011). it also means that employees must maintain a good relationship with a client to increase their satisfaction level. a number of studies focus on the customer service quality in marketing. it is mandatory for every discipline to adopt proper policies that enhance the quality of customer service. in the case of mobile banking platforms, the service providers have been successful in providing fast and effective services to their customers via phone, email, live chat and online customer care (ganguli & roy, 2011). ijahp article: allah pitchay, asmy mohd thas thaker, mohd thas thaker, amin/ determining the relative importance of quality factors for crowdfunding platforms international journal of the analytic hierarchy process 373 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.667 online banking services are mostly provided using high-tech communication channels (dean, 2004; rafaeli et al., 2008). according to bitner (1992), “in self-service organizations where very few if any employees are present and customer activity is very high, service support or remote service enhances customer satisfaction and attraction”. the factor “customer service quality” was first developed by parasuraman (1988) and was comprised of ten items including reliability, responsiveness, competence, access, courtesy, communication, credibility, security, understanding the customer and tangibles (kazi, 2011). since then, these dimensions have been reduced to five, namely tangibility, reliability, responsiveness, trust and empathy (che wel, 2003). in addition, brüntje & gajda (2016) stated that having good customer service on a crowdfunding platform can increase investors’ loyalty and attraction. similarly, martin (2012) highlighted that better access to customer services would legitimize a crowdfunding platform and make it successful. furthermore, belleflamme et al. (2010) found that raising money, getting public attention and obtaining feedback on products/services are all relevant factors in determining the success of a crowdfunding platform. the present study adopts some of these customer service quality dimensions and observes their effect on crowdfunding platform quality. as discussed, the quality of the customer service remains the focal point for achieving success in any business, and its proper implementation, particularly in a new platform like crowdfunding, will add value to the existing literature. 2.1.4 interface design an interface refers to a common boundary that connects systems, equipment, concepts and most importantly, human beings (mayrath, nihalani & robinsons, 2011). it is designed to make communication effective between devices, programs and users. interface design is one of the key qualitative factors that determine the quality of products and services. this is to ensure that the end users are ultimately comfortable. for instance, interface design should include various aspects of usability, like easy accessibility and understanding of the whole action because it makes links between interaction and visual design, and information architecture (elalfy, 2005). there are certain guidelines for interface design that enhance the quality of products and services. several different user interface design guidelines were introduced in the 1990’s which are visibility of system status, match between system and the real world, user control and freedom, consistency and standards, error prevention, recognition rather than recall, flexibility and efficiency of use, aesthetic and minimalist design, helping users recognize, diagnose and recover from errors, and help and documentation (wong, 2017). these are essential qualities for any successful interface design. interface design has become a subject of growing interest in online banking and particularly in the mobile banking services where different banks’ clients make various transactions and inquiries such as account balance, transaction history, transfer of funds, pay bills, trade stock and manage asset portfolios, and finding the location of the nearest banks’ atm (matilla, 2004; crossman, 2011). there are few studies that focus on interface design in the context of a crowdfunding platform. shi and guan (2016) found that appealing web-pages, promotional videos and photos of projects become the factors that determine the interface design of a crowdfunding platform. ijahp article: allah pitchay, asmy mohd thas thaker, mohd thas thaker, amin/ determining the relative importance of quality factors for crowdfunding platforms international journal of the analytic hierarchy process 374 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.667 brüntje & gajda (2016) also mentioned that the interface design of a crowdfunding platform includes interaction orientation, user added value, customization and personalization. a good interface design for a crowdfunding platform provides an opportunity for investors or users to re-configure platform websites and products or services according to their specific needs. this factor is becoming increasingly vital for web 2.0 applications and the quality of the crowdfunding platform can be enhanced by adopting a user-friendly interface design like mobile banking (wirtz et al. 2010). 3. research method from the literature, the following research framework was developed for this study. the quality factors of the crowdfunding platform are dependent on four factors: (i) functionality (accessibility, response time, mobility and security); (ii) content (accuracy, currency, relevance, completeness); (iii) customer service (reliability, responsiveness, trust, and empathy), and (iv) interface design (multimedia capability, format, understandability, navigability). figure 1 research framework 3.1 research area and sampling method this research uses a quantitative method in the form of a survey to carry out the analysis. the klang valley area (kuala lumpur and selangor) was selected for this study. this area was chosen to represent an urban area and consists of various ethnic groups or communities. jjudgemental sampling was applied after considering the purpose of the study. unlike other models, the ahp has no general rule about an appropriate sample size, and even works after the first interview. since this method does not require many interviews, the results appeared to be stable after only a few experts were interviewed. peterson, silsbee and schmoldt (1994) and al-harbi (2001) implemented this method with only 5 respondents, armacost, componation, mullens, swart (1994) considered 7 respondents, mawapanga and debertin (1996) chose 18 respondents and huang and yeh (2011) used 25 participants for their studies. the present study interviewed 31 respondents and ranked their opinions regarding the proposed model. the survey questionnaire was distributed manually and self-administered by the respondents. the first part of the questionnaire focuses on the demographic profile of the respondents such as gender, age, marital status, race, education level, income, crowdfunding platform quality factors selection functionality customer services interface design content ijahp article: allah pitchay, asmy mohd thas thaker, mohd thas thaker, amin/ determining the relative importance of quality factors for crowdfunding platforms international journal of the analytic hierarchy process 375 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.667 employment level, occupation, number of family and religion. in order to measure the level of awareness about crowdfunding platforms, questions measuring the individual’s thoughts, feelings and actions were incorporated in the survey. for example, a typical question could be, “have you heard about crowdfunding platforms?” the fundamental objective of this study was to determine the relative importance of the quality factors of crowdfunding platforms. therefore, to meet the research objective, this study applied the fuzzy approach known as the analytical hierarchy process (ahp). the survey questions are strictly based on ahp protocols. table 1 shows the ahp matrix that was designed for this study. this matrix must be completed by the respondents. the scale used with the ahp is different from the normal likert scale, in that a range from ‘1’ for ‘equally important’ to ‘9’ for ‘extremely important” is used, as indicated at table 2. table 1 analytic hierarchy process ahp matrix crowdfunding platform quality factors selection functionality content customer services interface design functionality content customer service interface design ijahp article: allah pitchay, asmy mohd thas thaker, mohd thas thaker, amin/ determining the relative importance of quality factors for crowdfunding platforms international journal of the analytic hierarchy process 376 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.667 table 2 analytical hierarchy process scales scales the verbal judgement of importance description of verbal judgement 1 both are equally important both elements contribute equally 3 moderately important judgement favours one element over another 5 strongly important an element is strongly preferable 7 very strongly important an element is very strongly prevailing 9 extremely important an element is preferred by at least an order of magnitude 2,4,6,8 intermediate values normally used to compromise between two verbal judgements reciprocals when an activity i compared to j is assigned one of the above numbers, then activity j compared to i is assigned its reciprocal 3.2 the analytical hierarchy process (ahp) ahp is widely popular for analyzing complex decisions and is used in various research related to knowledge, experience and individual perceptions (saaty, 2008). ahp is based on the segregation of a decision into specific elements following individual characteristics. ahp uses aggregate individual priorities (aip) to obtain the ranking criteria from the respondents. to calculate the priorities for every element, the judgemental matrix below is used where a1,2 shows the pairwise comparison between element 1 and 2 with regard to the upper level. the entries of a1,2 are controlled by the rule of; a12 >0; a12 =1/a21, a11 =1i c1 c2 cn c1 c2 ……… cn a12 ……… an1 a12 a22 ……… an2 ……… ………. ………. ……….. a12 a22 ann in addition, this paper uses the geometric mean as suggested by forman and peniwati (1998) because in the context of this paper the aggregate individual priorities (aip) structure is used due to the focus on the individual’s priorities, and the most suitable method to deal with individual aggregate priorities is the geometric mean. in this paper, a group of respondents is not considered a new individual, which is another justification for the use of the geometric mean. zahedi (1986) suggested several protocols for implementing the aip prioritization as below: i) break down the decision problem into a hierarchy of interrelated decision elements such as criteria, alternative decision, sub-criteria alternatives. ii) gather the input data postulated by the pairwise matrices of the elements. a= ijahp article: allah pitchay, asmy mohd thas thaker, mohd thas thaker, amin/ determining the relative importance of quality factors for crowdfunding platforms international journal of the analytic hierarchy process 377 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.667 iii) determine the proportion or weight of the decision by using the eigenvalue analysis. iv) finally, take the collective data of all of the respondents of relative proportion or weights to derive the ranking for the decision elements. saaty (1980) also mentioned that there must be some reasonable consistency in deriving the priorities from the paired comparison, in which the number of factors being considered should be less than or equal to 9. the consistency judgement is calculated using the following formula: 𝐶𝑅 = 𝐶𝐼 𝑅𝐼 where the cr is the consistency index and ri is the random index. on the other hand, the consistency index (ci) can be derived based on this formula: 𝐶𝐼 =  max − n 𝑛 − 1 where,  max = largest eigenvalue n = size of the comparison matrix according to saaty, an acceptable consistency index should be less than 0.10. if it is greater than .10, the result is inconsistent, which may be due to the respondents not understanding the questions that they were asked. therefore, the judgement may not be accurate. the analysis of the consistency index can be performed using expert choice software. after an acceptable consistency index is obtained, the next process is to derive the weights, or normalized scores, for the criteria 4. research findings, discussion and conclusion table 3 shows the demographic information of the respondents involved in this research. 25.8% of the respondents were male, and 74.2% were female. most of the respondents were between 31 and 40 years old (61.3%). this percentage seems to be consistent with a previous study conducted by metawa and almossawi (1998), where the perception of people or customers between 20 to 50 years of age had the most impact on policies. thus, this finding may contribute to obtaining the objective of this study. the majority of the respondents were malays (68.1%), followed by chinese, indian and others at 18.1%, 9.4% and 4.4%, respectively. in terms of education level, the majority of the respondents (80.7%) held a bachelor`s degree and postgraduate degree, with 9.7% having a diploma degree, followed by 6.5% with a secondary level and 3.2% with nonformal education. in addition, the employment status of the respondents showed that the majority of respondents were from the public sector (45.2%), 35.5% were from the private sector, and both self-employed and students shared 3.2%. in terms of income level of the respondents, majority (74.3%) of them were having income above rm3000 and 25.8% were having income less than rm3000. ijahp article: allah pitchay, asmy mohd thas thaker, mohd thas thaker, amin/ determining the relative importance of quality factors for crowdfunding platforms international journal of the analytic hierarchy process 378 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.667 table 3 descriptive statistics (%) marital status single 19.4 married 80.6 ethnicity malay 68.1 chinese 18.1 indian 9.4 other 4.4 education level non-formal education 3.2 secondary level 6.5 diploma 9.7 bachelor degree 58.1 postgraduate 22.6 employment status public 45.2 private 35.5 self-employed 3.2 housewife 12.9 student 3.2 number of family members one 9.7 two 19.4 three 29.0 four 22.6 >four 19.4 monthly income rm1001-rm2000 12.9 rm2001-rm3000 12.9 rm3001-rm4000 32.3 rm4001-rm5000 22.6 >rm5000 19.4 n = 31 respondents ijahp article: allah pitchay, asmy mohd thas thaker, mohd thas thaker, amin/ determining the relative importance of quality factors for crowdfunding platforms international journal of the analytic hierarchy process 379 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.667 table 4 malaysian awareness about crowdfunding platforms level of awareness percentage (%) yes no 1. have you ever aware of crowdfunding platform? 54.8 45.2 2. have you ever aware of websites such as kick-starter, pitch in, mystartr and crowdonomic media? 45.2 54.8 3. are you interested to invest your money through a crowdfunding platform to fund a project, a business or personal loan, and other needs? 80.6 19.4 4. what are the factors that influence you to invest your money through a crowdfunding platform? (you may tick more than one). i. functionality 77.4 22.6 ii. content 64.5 35.5 iii. customer service 58.1 41.9 iv. interface design 35.5 64.5 table 4 shows the results of the awareness level of respondents about crowdfunding platforms. there are four questions; the first question was about awareness of crowdfunding platforms and shows that more than half (54.8%) were aware of crowdfunding platforms. since crowdfunding is considered a new platform in malaysia, this level of awareness was a good indicator that the respondents were aware of the existence of crowdfunding platforms. next, the present study also provided several examples of online or website crowdfunding platforms such as kickstarter, pitch-in, mystartr and crowdonomic. the results show that 45.2% of the respondents were aware of the existence of these crowdfunding platforms. this shows that the respondents are either looking for a new avenue to invest their surplus funds or are simply looking to obtain information about the crowdfunding platforms. furthermore, they were willing to contribute and invest in projects that use a crowdfunding platform to raise capital. the results indicated that 80.6%, or a majority of the respondents, are interested in investing in a crowdfunding platform. this is important because it indicates the level of trust the respondents have in online crowdfunding platforms and also shows that their investment behavior has been transformed to a new avenue when compared to the traditional financial intermediaries. this result may be influenced by the movement of the malaysia central bank to support the expansion of crowdfunding platforms in malaysia. this will boost the confidence of investors to invest in crowdfunding platforms. the final question in the survey attempted to identify the factors that influence the experts’ investment in a crowdfunding platform. there are four factors listed on the questionnaire which include functionality, content, customer service and interface design. the results showed more than 50% of the respondents agreed that functionality, content ijahp article: allah pitchay, asmy mohd thas thaker, mohd thas thaker, amin/ determining the relative importance of quality factors for crowdfunding platforms international journal of the analytic hierarchy process 380 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.667 and customer service are important factors that influence their investment in a crowdfunding platform. only 64.5% of the respondents reported being influenced by interface design. table 5 combined priority vectors for all of the criteria of the decision hierarchy crowdfunding platform quality factors selection value ranking functionality 0.2850 1 content 0.2600 2 interface design 0.2340 3 customer service 0.2200 4 consistency index (ci) 0.0100 accepted n = 31 respondents table 5 shows the output generated based on the analytic hierarchy process (ahp). this analysis examined the ranking of the factors and prioritized the factors according to the respondents’ preference. as mentioned previously, we used expert choice (ec) to generate a priority vector as presented in table 6. ec combined the judgments geometrically for each cell of the pairwise comparison matrix and postulated a single pairwise comparison matrix where the system indicated a priority vector. the questionnaires completed by the respondents are shown in appendix 1. we collected demographic information, degree of awareness about crowdfunding and the pairwise comparison judgments. there are four factors, namely (i) functionality, (ii) content, (iii) interface design and (iv) customer service. in order to validate the output of the rankings, a consistency index (ci) was calculated. based on the result, the ci was 0.0100, which is less than the threshold of 0.10. therefore, the ci result was valid and within the acceptable threshold. the highest priority of the criteria for crowdfunding is functionality, followed by content, interface design and customer service, respectively. this means that functionality is the most influential factor for crowd funders. this finding is consistent with andreou et al. (2005) and hsiu-fen lin (2013) who identified that the components of functionality such as interoperability, suitability, security, compliance and accuracy are the most important for the investor who invests in a crowdfunding platform. besides functionality, crowd funders are looking for quality content on the platform, which should deliver the message clearly and be friendly to the investors. chan and alern (1999), venkatesh and ramesh (2002) and vrechopoulos and doukidis (2003) found that the content of an online platform, especially for investment purposes, is important to attract and give confidence to an investor using this platform. therefore, crowdfunding online intermediaries should emphasize the content, which will determine the participation of the investor in their investment online platform. next is interface design, followed by customer service. these two factors are also important because the validity of the results show that all of the factors tested in this ijahp article: allah pitchay, asmy mohd thas thaker, mohd thas thaker, amin/ determining the relative importance of quality factors for crowdfunding platforms international journal of the analytic hierarchy process 381 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.667 study are important. therefore, based on this result, crowdfunding platform providers should emphasize the following ranked priorities based on the study output and in order to attract more investors who are looking for a new avenue of investment, especially in the online platforms. 5. limitations and future research this paper has some limitations. first, this study is limited to respondents in the klang valley of malaysia and cannot be generalized to other states of malaysia. therefore, this study can be enhanced by expanding the coverage area. furthermore, the variables used in this study are limited, and other variables can be incorporated in future research. ijahp article: allah pitchay, asmy mohd thas thaker, mohd thas thaker, amin/ determining the relative importance of quality factors for crowdfunding platforms international journal of the analytic hierarchy process 382 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.667 references andreou, andreas s, panayidou, d, andreou, panayiotis and pitsillides, andreas (2005). preserving quality in the development of mobile commerce services and applications. proceedings of the second iasted international multi-conference on automation, control, and information technology, novosibirsk, russia. (unpublished) al-harbi, k. (2001) application of the ahp in project management. international journal of project management, 19, 19–27. doi: https://doi.org/10.1016/s02637863(99)00038-1 armacost, r.l., componation, p.j., mullens, m.a., swart,w.w (1994). an ahp framework for prioritizing customer requirements in qfd: an industrialized housing application. iie tranactions, 26, 72–79. doi: https://doi.org/10.1080/07408179408966620 babich, v., marinesi, s., & tsoukalas, g. (2018). does crowdfunding benefit entrepreneurs and venture capital investors?. georgetown mcdonough school of business research, 2971685. doi: https://doi.org/10.2139/ssrn.2971685 brüntje, d., & gajda, o. (2016). crowdfunding in europe: state of the art in theory and practice. springer. barabas, r. l. (2012). crowdfunding: trends and developments impacting entertainment entrepreneurs, entertainment. arts and sports law journal, 23(2), 38-40. belleflamme, p., omrani, n., & peitz, m. (2015). the economics of crowdfunding platforms. information economics and policy, 33, 11-28. doi: https://doi.org/10.1016/j.infoecopol.2015.08.003 bitner, m.j. (1992). servicescapes: the impact of physical surroundings on customers and employees. journal of marketing, 56(2), 57-71. doi: https://doi.org/10.2307/1252042 bitner, m.j., brown, s.w. and meuter, m.l. (2000). technology infusion in service encounters. academy of marketing science journal, 28(1), 138-149. doi: https://doi.org/10.1177/0092070300281013 buchanan, leigh (2011). a customer service makeover. inc. magazine. retrieved 29 oct 2012. chae, m., kim, j., kim, h., & ryu, h. (2002). information quality for mobile internet services: a theoretical model with empirical validation. electronic markets, 12(1), 38– 46. doi: https://doi.org/10.1080/101967802753433254 chan, t., & ahern, t. (1999). targeting motivation: adapting flow theory to instructional design. journal of educational computing research, 21(2), 151–163. doi: https://doi.org/10.2190/uj04-t5yb-yfxe-0bg2 http://clok.uclan.ac.uk/18447/ http://clok.uclan.ac.uk/18447/ http://www.inc.com/magazine/20110301/a-customer-service-makeover_pagen_2.html https://en.wikipedia.org/wiki/inc._(magazine) ijahp article: allah pitchay, asmy mohd thas thaker, mohd thas thaker, amin/ determining the relative importance of quality factors for crowdfunding platforms international journal of the analytic hierarchy process 383 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.667 che wel, c.a. & mohd nor, s. (2003). the influences of personal and sociological factors on consumer bank selection decision in malaysia. journal of american academy of business, 3(1/2), 399-404. crossman, p. (2011). banks have yet to create the killer app for mobile banking bank systems & technology, retrieved on 12 february from http://www.banktech.com/channels/banks-have-yet-to-create-the-killer-app/229201117 dean, a.m. (2004). rethinking customer expectations of service quality: are call centers different? journal of services marketing, 18(1), 60-77. doi: https://doi.org/10.1108/08876040410520717 delone & e. r. mclean. (2003). the delone and mclean model of information systems success: a ten-year update. journal of management information systems, 19(4), 9–30. doi: https://doi.org/10.1080/07421222.2003.11045748 dresner, s. (2014). crowdfunding: a guide to raising capital on the internet. john wiley & sons. elalfy, e. (2005). a general look at building application for mobile devices, retrieved on 30 march 2014 from http://csdl2.computer.org/comp/marrs/ds/2005/09/o9005.pdf. ganguli, s. & roy, s.k. (2011). generic technology-based service quality dimensions in banking. the international journal of bank marketing, 29(2), 168-189. doi: https://doi.org/10.1108/02652321111107648 gerber, e. m., hui, j. s., & kuo, p. -y. (2012). crowdfunding: why people are motivated to participate. northwestern university, segal design institute, technical report no.2, 2012. available at: https://www.zotero.org/groups/creative_action_lab/items/ itemkey/nssgirnh hsiu-fen lin (2013). determining the relative importance of mobile banking quality factors. computer standards & interfaces, 35(2013), 195–204. doi: https://doi.org/10.1016/j.csi.2012.07.003 huang, r.y. yeh, c.h. (2011). development of an assessment framework for green highway construction. journal of the chinese institute of. engineering, 31, 573–585. iizuka, m. (2014). le crowdfunding: les rouages du financement participatif 2015. edubanque editions, ed. 2. iso (2010). iso 9241-210:2010. international organization for standardization. retrieved february 2018. kazi omar siddiqi (2011). interrelations between service quality attributes, customer satisfaction and customer loyalty in the retail banking sector in bangladesh. international journal of business and management, 6(3), 12-36. doi: http://dx.doi.org/10.5539/ijbm.v6n3p12 http://www.banktech.com/channels/banks-have-yet-to-create-the-killer-app/229201117 http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=52075 ijahp article: allah pitchay, asmy mohd thas thaker, mohd thas thaker, amin/ determining the relative importance of quality factors for crowdfunding platforms international journal of the analytic hierarchy process 384 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.667 lambert, t. & schwienbacher, a. (2010). an empirical analysis of crowdfunding. louvain-la-neuvwe, louvain school of management, catholic university of louvain. mattila, m. (2004). factors affecting the adoption of mobile banking services. journal of internet banking and commerce, 13(4), 104-121. massolution (2015). 2015cf-crowdfunding industry report, massolution. mayrath, m. c., nihalani, p. k., & robinson, d. h. (2011). varying tutorial modality and interface restriction to maximize transfer in a complex simulation environment. journal of educational psychology, 103(2), 257 268. doi: http://dx.doi.org/10.1037/a0022369 mawapanga, m.n.; debertin, d.l. (1996). choosing between alternative farming systems: an application of the analytic hierarchy process. review of agricultural economics, 18, 385–401. doi: https://doi.org/10.2307/1349623 mollick, e. (2014). the dynamics of crowdfunding: determinants of success and failure. journal of business venturing, 29(1), 1-16. doi:10.2139/ssrn.2088298 ordanini, a., l. miceli, and a. parasuraman (2011). crowd-funding: transforming customers into investors through innovative service platforms. journal of service management, 22(4), 443-470. parasuraman, a., zeithaml, v. l. and berry, l. (1988). servqual: a multiple-item scale for measuring consumer perceptions of service quality. journal of retailing, 64(1), 12-40. peterson, d.l., silsbee, d.g., schmoldt, d.l. (1994). a case study of resources management planning with multiple objectives and projects. environmental management, 1994(18), 729–742. doi: https://doi.org/10.1007/bf02394636 rafaeli, a., ziklik, l. and doucet, l. (2008). the impact of call center employees’ customer orientation behaviors on service quality. journal of service research, 10(3), 239-255. doi: https://doi.org/10.1177/1094670507306685 shi, m., & guan, l. (2016). an empirical study of crowdfunding campaigns: evidence from jing dong crowdfunding platform. 2016 13th international conference on service systems and service management (icsssm). doi: https://doi.org/10.1109/icsssm.2016.7538637 taylor, r. s. (1986). value-added processes in information systems. norwood, nj: ablex publishing. the world bank (2013). crowdfunding’s potential for the developing world. washington, dc: world bank. venkatesh, v. & ramesh, v. (2002). usability of web and wireless sites: extending the applicability of the microsoft usability guidelines instrument. information systems technical reports and working paper. doi:%20http://dx.doi.org/10.1037/a0022369 doi:%20http://dx.doi.org/10.1037/a0022369 ijahp article: allah pitchay, asmy mohd thas thaker, mohd thas thaker, amin/ determining the relative importance of quality factors for crowdfunding platforms international journal of the analytic hierarchy process 385 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.667 vlachos, p. & vrechopoulos, a. (2005). special issue on consumers' behavioral intentions in the mobile internet services market. the hermes newsletter, eltrun, athens university of economics and business. available at http:// www.eltrun.aueb.gr vlachos, p., vrechopoulos, a. & doukidis, g. (2003). exploring consumer attitudes towards mobile music services. international journal on media management, 5(2), 138– 148. doi: https://doi.org/10.1080/14241270309390027 wong, euphemia (2017). user interface design guidelines: 10 rules of thumb. interaction design foundation. wirtz, b. w., schilke, o. & ullrich, s. (2010). strategic development of business models: implications of the web 2.0 for creating value on the internet. long range planning, 43(2-3), 272-290. doi: https://doi.org/10.1016/j.lrp.2010.01.005 yan, c., xin, l. & mackie-mason, j. k. (2006). online fund-raising mechanisms: a field experiment. the berkeley electronic press. yao, h., & zhang, y. (2014). research on influence factors of crowdfunding. international business and management, 9(2), 27-31. zheng, h., li, d., wu, j. & xu, y. (2014). the role of multidimensional social capital in crowdfunding: a comparative study in china and us. information & management, 51(4), 488–496. doi: https://doi.org/10.1016/j.im.2014.03.003 https://www.interaction-design.org/literature/article/user-interface-design-guidelines-10-rules-of-thumb https://www.interaction-design.org/literature/article/user-interface-design-guidelines-10-rules-of-thumb ijahp article: allah pitchay, asmy mohd thas thaker, mohd thas thaker, amin/ determining the relative importance of quality factors for crowdfunding platforms international journal of the analytic hierarchy process 386 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.667 appendix appendix 1 determining the relative importance of crowdfunding platform quality factors this appendix shows the message we sent to potential respondents. part a was for collecting demographic information. part b was used to collect information about the awareness the participants had about crowdfunding. part c were the questionnaires used to collect the pairwise comparison matrix judgments. dear madam/sir, this questionnaire aims to understand the quality factors of crowdfunding platforms. a crowdfunding platform is defined as an ‘online web-based platform that raises a small amounts of money from a large number of individuals or organisations to fund a project, a business or personal loan, and other needs’. your support is the most important factor for the success of this research. the information you provide will be held confidentially and totally anonymously. furthermore, the results will be shown only in an aggregate form, so that it is impossible to trace this information back to you individually. the data collected through this study will only be used for academic research purposes. if you want to get additional information and feedback about the purpose and results of the study please get in touch with us. thank you for your time. dr. mohamed asmy bin mohd thas thaker assistant professor, department of economics faculty of economics & management sciences international islamic university malaysia mobile: 0123904299 email: asmy@iium.edu.my mailto:asmy@iium.edu.my ijahp article: allah pitchay, asmy mohd thas thaker, mohd thas thaker, amin/ determining the relative importance of quality factors for crowdfunding platforms international journal of the analytic hierarchy process 387 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.667 part a: 1. gender [ ] male [ ] female 2. age [ ] 20-30 years [ ] 31-40 years [ ] 41-50 years [ ] > 50 years 3. marital status [ ] single [ ] married [ ] divorced 4. educational level [ ] no education [ ] secondary (spm) [ ] diploma [ ]degree [ ] postgraduate (master/phd) 5. occupation [ ] public sector [ ] private sector [ ] self employed [ ] housewife [ ] retired [ ] student 6. number of family members [ ] one [ ] two [ ] three [ ] four [ ] more than four 7. monthly income [ ] ≤ rm1000 [ ] rm1001 – rm2000 [ ] rm2,001rm3000 [ ] rm3,001rm4000 [ ] rm4,001rm5000 [ ] > rm5000 8. religion [ ] islam [ ] christian [ ]buddhist [ ]hindu [ ]others (_____________) part b. crowdfunding platform awareness crowdfunding platform is defined as an ‘online web-based platform that raises small amounts of money from a large number of individuals or organisations to fund a project, a business or personal loan, and other needs’. 1. have you ever heard of crowdfunding platform? [ ] yes [ ] no 2. have you ever heard of websites such as kick-starter, pitch in, mystartr or crowdonomic media? [ ] yes [ ] no 3. are you interested to donate your money through crowdfunding platform to fund a project, a business or personal loan, and other needs? [ ] yes [ ] no 4. what are the factors that influencing you to donate your money through crowdfunding platform? (you may tick more than one) [ ] functionality [ ] content [ ] customer service [ ] interface design [ ] others (_______) part c. crowdfunding platform quality factors selection note: for each statement below, please compare the relative importance factors with respect to crowdfunding platform quality. choose and circle only one number per row by using the following scale: ijahp article: allah pitchay, asmy mohd thas thaker, mohd thas thaker, amin/ determining the relative importance of quality factors for crowdfunding platforms international journal of the analytic hierarchy process 388 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.667 verbal judgment of preference rating equally preferred 1 equally to moderately preferred 2 moderately preferred 3 moderately to strongly preferred 4 strongly preferred 5 strongly to very strongly preferred 6 very strongly preferred 7 very strongly to extremely preferred 8 extremely preferred 9 crowdfunding platform quality factors selection criteria 1 . functionality 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 content 2 . functionality 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 customer service 3 . functionality 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 interface design 4 . content 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 customer service 5 . content 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 interface design 6 . customer service 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 interface design (i) functionality: accessibility, response time, mobility, security ; (ii) content: accuracy, currency, relevance, completeness ; (iii) customer service: reliability, responsiveness, trust, empathy ; (iv) interface design: multimedia capability, format, understandability, navigability #thanks for your participation# ijahp article: ohta, salomon, silva / selection of industrial maintenance strategy: classical ahp and fuzzy ahp applications international journal of the analytic hierarchy process 254 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.551 selection of industrial maintenance strategy: classical ahp and fuzzy ahp applications robison ohta 1 sao paulo state university (unesp) guaratingueta, sp, brazil robisonohta@hotmail.com valerio a. p. salomon unesp, guaratingueta, sp, brazil messias borges silva unesp, guaratingueta, sp, brazil abstract the selection of a maintenance strategy is a decision often made with uncertainty or subjectivity. this decision involves the prioritization of critical factors since there are several factors to be considered simultaneously. decision-making generally depends on subjective assessments from experts. to deal with multiple factors, analytic hierarchy processes (ahp) is a well-established multiple criteria decision analysis (mcda) method. this article presents an ahp application for the selection of a maintenance strategy by a real industrial plant. four maintenance strategies are considered: corrective maintenance, preventive maintenance, predictive maintenance, and proactive maintenance. decision criteria are cost, quality, safety, value added and viability. then, incorporating the concepts of the fuzzy set theory, fuzzy ahp was applied to the same decision problem. in both applications, corrective maintenance was the strategy with the highest priority, and value added was the highest priority criterion. with the classical ahp application, some comparison matrices produced consistency ratios (cr) greater than 0.10, possibly generated by mistakes or misunderstandings from experts. however, the same result was obtained from fuzzy ahp and validated the result obtained from classical ahp application. the major contribution of the paper is the evidence that fuzzy ahp may be a useful tool to solve the consistency problems in classical ahp applications. keywords: analytic hierarchy process; consistency ratio; fuzzy set theory; maintenance strategy 1 acknowledgements: this project was partially funded by the são paulo research foundation (fapesp) grant no 2017/22963-6. mailto:robisonohta@hotmail.com ijahp article: ohta, salomon, silva / selection of industrial maintenance strategy: classical ahp and fuzzy ahp applications international journal of the analytic hierarchy process 255 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.551 1. introduction maintenance is the term used to address the way organizations try to avoid failures by taking care of their physical facilities (slack, brandon-jones, & johnston, 2016). the goal of maintenance is to ensure that physical items continue to do what their users want them to do (moubray, 2000). then, maintained items are expected to keep their functional capacity of operation. a maintenance strategy ensures the availability of equipment and facilities to allow production with reliability, safety, environmental preservation and adequate costs (pinto & nascif, 2014). there are different maintenance strategy alternatives, but, basically, the four main alternatives are:  corrective maintenance (a1) is when the intervention occurs at the moment, or after, a failure occurs. the failure causes the equipment to be unavailable or have lower performance. corrective maintenance can be planned, or not. often, when unplanned, the intervention is immediate and without service preparation. when planned, the intervention team can be prepared to do the intervention.  preventive maintenance (a2) is when the intervention is carried out as an objective to reduce or avoid the failure of the equipment. the interventions follow a previously prepared plan, and they are periodically carried out.  predictive maintenance (a3) is when the intervention modifies parameters of performance. it prevents failures by means of various parameters, which aim for the continuous operation of the equipment for the maximum possible time.  proactive maintenance (a4) is based on the frequency of occurrence of the failure. a history of these occurrences is made on the equipment and the information is used to determine the root cause of the failure. it generates actions related to the root cause of the failure, searching to increase the life of the equipment. the selection of a maintenance strategy is a multi-criteria decision since it may involve an analysis of benefits, opportunities, costs and risks (karpak, 2017). additionally, this decision may also incorporate subjectivity and uncertainty. analytic hierarchy process (ahp) and fuzzy set theory (fst) may be applied to solve this decision problem. the objective of this paper is to present a model for the selection of a maintenance strategy. two models will be presented and applied in a real case: an ahp model and a fahp (fuzzy ahp) model. the next section presents the results from a bibliometric survey on ahp, fst and maintenance. results are compared in the conclusion section of the paper. therefore, this paper has two major contributions which include: the theoretical or methodological contribution comparing results from two models (an ahp and a fahp), and a practical contribution from the application of both models in a real case. 2. literature review besides the multi-criteria decision nature of maintenance strategy selection, the literature combining ahp, fst and maintenance is very poor or, at least, incipient. a survey on the scopus database with the keywords “ahp”, “fuzzy” and “maintenance” resulted in 167 papers, as presented in figure 1 (elsevier, 2018). with a peek of more than 25 papers published in 2016, the average from 2007–2017 is 16.7 papers/year. ijahp article: ohta, salomon, silva / selection of industrial maintenance strategy: classical ahp and fuzzy ahp applications international journal of the analytic hierarchy process 256 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.551 figure 1 publication numbers per year figure 2 presents the distribution by the countries of first authors for these papers. china is the leading country because of its large population and academic tradition as in almost every field of knowledge. iran is a good surprise, appearing second along with other traditional countries. figure 2 publication numbers by country figure 3 presents paper distribution by research area. engineering, computer science and business management, in this order, are the top areas. ijahp article: ohta, salomon, silva / selection of industrial maintenance strategy: classical ahp and fuzzy ahp applications international journal of the analytic hierarchy process 257 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.551 figure 3 publication percentages by research area from the bibliometric research (figures 1 to 3), there is a growing interest in the subject and an increasing number of citations of keywords ahp, fuzzy ahp and maintenance, in the last years (figure 4). still, the number of papers in this area is low. so, the central question of this research is if the selection of maintenance strategies may be solved with multiple criteria decision analysis (mcda). figure 4 citations in the last 10 years ijahp article: ohta, salomon, silva / selection of industrial maintenance strategy: classical ahp and fuzzy ahp applications international journal of the analytic hierarchy process 258 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.551 3. methodology 3.1. classical analytic hierarchy process the name ahp was given because of its use of hierarchies to make multi-criteria decisions (saaty, 1977; saaty, 1980; saaty, 1991). as in figure 5, the top level has the decision objective, in our case, the selection of a maintenance strategy. in the bottom level, there are the decision alternatives, in our case a1 to a4, which were previously introduced. figure 5 multi-criteria selection of maintenance strategy the criteria c1,…,c5 were identified from the literature of maintenance selection and they were utilized in the case presented in the next section (ge, et al., 2017;w ang, chu, & wu, 2007). that is, this set of criteria was approved by a manager from a real industrial plant. briefly, these are the concepts expressed by each criterion:  safety (c1): the required safety levels have increased lately. for maintenance strategy, it represents the conditions that deter undesirable results, such as accidents, failure, mistakes, and so on. it also refers to the controllability of reducing known threats to an acceptable level, both in terms of personal safety, and plant safety or even in terms of environmental safety.  costs (c2): maintenance costs must not exceed acceptable limits. costs do differentiate a process and have direct influence on the incomes.  quality (c3): proper maintenance management brings a better quality of production and cost reduction, while a poor maintenance management of the equipment leads to the breakdown. this causes the need to invest in repair and perhaps in replacement, thus translating into a high cost, and may also affect other sectors. when it comes to production, poor maintenance management can lead to loss of production and product quality.  value added (c4): in the economic area, value added is the difference between the final and initial price to produce a given product. in the maintenance area, value added means all the benefits and returns of maintenance activities. generally, the higher the value added, the more return will be obtained with a selection of maintenance strategy safety (c1) corrective maintenance (a1) preventive maintenance (a2) predictive maintenance (a3) pro-active maintenance (a4) costs (c2) a1 a2 a3 a4 quality (c3) a1 a2 a3 a4 added value (c4) a1 a2 a3 a4 feasibility (c5) a1 a2 a3 a4 ijahp article: ohta, salomon, silva / selection of industrial maintenance strategy: classical ahp and fuzzy ahp applications international journal of the analytic hierarchy process 259 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.551 greater effectiveness of maintenance and with less entry of services or products. the most relevant factors are inventories of spare parts, loss of production and identification of failures.  feasibility (c5): it is applied to determine whether the maintenance strategy is suitable for the system. according to the different requirements of works and techniques for maintenance strategies, the feasibility criteria can be divided into two relevant evaluation factors. first, labors which includes when managers and maintenance staff prefer the maintenance strategies that are easy to implement and understand. second, the technique reliability, still under development, which is condition-based maintenance and predictive maintenance that may be inapplicable for some facilities. one important ahp assumption is the independency between alternatives and criteria. as a matter of fact, in practice, a manager may decide to implement more than one maintenance strategy, combining them, for different equipment. this is a limitation of this work. we will consider the maintenance strategies as independent of each other, or else, mutually exclusive and collectively exhaustive (lawrence & pasternack, 2013). in ahp, priorities for both sets of criteria and alternatives are obtained with the right eigenvector w, for a comparison matrix a, as in equation 1, where max is a’s maximum eigenvalue. a w = max w (1) for fully consistent matrices, that is, if we have aik=aijajk, max=n (saaty, 1977). otherwise, max>n. as, close as max is to n, the more a may be considered consistent. the consistency index ci=(max–n)/(n–1) is a better measure for the consistency since it also considers the matrix order, n (saaty, 1980). the consistency ratio rc=ci/ri, where ri is a random index, with an upper limit of 0.1 is the most applied consistency parameter. for the comparison matrix presented in table 1, we have max≈3.04, ci≈0.02 and cr≈0.04, which are acceptable. table 1 pairwise comparisons of three criteria criterion 1 criterion 2 criterion 3 priority criterion 1 1 3 5 64% criterion 2 1/3 1 3 26% criterion 3 1/5 1/3 1 10% let us suppose that the decision-maker is quite confident with the comparison between criteria 1 and 2. however, he is not sure about criterion 3. then, he considered different values to compare this criterion with the other ones. however, in the original ahp, he can only provide one value. table 2 presents another comparison matrix with new values, in bold, for the uncertain comparisons in table 1. for table 2, max=3, ci=cr=0, that is, this is a fully consistent matrix. ijahp article: ohta, salomon, silva / selection of industrial maintenance strategy: classical ahp and fuzzy ahp applications international journal of the analytic hierarchy process 260 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.551 table 2 new pairwise comparisons of three criteria criterion 1 criterion 2 criterion 3 priority criterion 1 1 3 3 60% criterion 2 1/3 1 1 20% criterion 3 1/3 1 1 20% comparing tables 1 and 2, the priorities have changed a little. criterion 1 still has the highest priority, a little less than 2/3. however, criteria 2 and 3 are now tied. that is, criterion 3’s new priority is twice the older priority. with fst, both priorities may be considered, or better, both sets of comparisons may be used. 3.2. fuzzy analytic hierarchy process a fuzzy set a in x is characterized by a membership function fa(x) which associates each point in x a real number in the interval [0, 1] (zadeh, 1965). fst literature has a rich history, including type 2 fuzzy sets, interval-valued fuzzy sets, intuitionistic fuzzy sets, fuzzy multisets and more recently, hesitant fuzzy sets (kahraman, oztaysi & onar, 2016). triangular fuzzy sets (tfs) are common membership functions used in engineering (pedrycz, 1994). a tfs is often represented by a vector (l, m, u), being m the modal value, that is fa(m)=1, and l and u being the lower and upper limits, or fa(l)= fa(u)=0. table 3 presents a comparison matrix completed with tfs. note that if aij=(lij, mij, uij), then aji=1/aij=(1/uij, 1/mij, 1/lij) (van laarhoven & pedrycz, 1983). priority vector w was obtained according to equation 2 (chang, 1996). 𝑤𝑖 = (∑ 𝑙𝑖𝑗, 𝑛 𝑗=1 ∑ 𝑚𝑖𝑗, 𝑛 𝑗=1 ∑ 𝑢𝑖𝑗 𝑛 𝑗=1 )(∑ ∑ 1 𝑢𝑖𝑗 ,𝑛𝑗=1 𝑛 𝑖=1 ∑ ∑ 1 𝑚𝑖𝑗 ,𝑛𝑗=1 𝑛 𝑖=1 ∑ ∑ 1 𝑙𝑖𝑗 𝑛 𝑗=1 𝑛 𝑖=1 ) (2) table 3 fuzzy pairwise comparisons of three criteria criterion 1 criterion 2 criterion 3 priority criterion 1 (1, 1, 1) (1, 3, 5) (3, 5, 7) (.22, .60, 1.52) → .78 → 59% criterion 2 (1/5, 1/3, 1) (1, 1, 1) (1, 3, 5) (.10, .29, .82) → .40 → 30% criterion 3 (1/7, 1/5, 1/3) (1/5, 1/3, 1) (1, 1, 1) (.06, .10, .27) → .15 → 11% we can see more similarity between priority vectors from tables 1 and 3 than tables 2 and 3 or tables 1 and 2. in all tables, criterion 1 has the highest priority. in tables 1 and 3, criterion 2 has the second priority and criterion 3 has the lowest. one important observation is that table 3 mixes information from tables 1 and 2. for the decision maker, there is a feeling that no information needs to be discarded. ijahp article: ohta, salomon, silva / selection of industrial maintenance strategy: classical ahp and fuzzy ahp applications international journal of the analytic hierarchy process 261 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.551 4. application 4.1. classical analytic hierarchy process for this study of ahp application, three senior maintenance managers from a global manufacturer of industrial components serving primarily the railroad, vehicular and construction located in southeastern brazil were interviewed. this is a typical industrial plant located in the state of sao paulo. this plant is a tier 2 player in the brazilian rail supply chain, manufacturing structural components for trains, tractors, excavators and heavy machines from multinational groups located mainly in the states of sao paulo and rio de janeiro. table 4 presents the pairwise comparisons on the criteria from the experts, aggregated by geometric mean (saaty & peniwati, 2013). we have max≈5.77, ci≈0.19 and cr≈0.17, which are not acceptable since c r> 0.10. comparison matrices with cr higher than the upper limit represent an alert that comparisons may be not logically connected. after a discussion with experts, the priority vector presented in table 4 was accepted. that is, criteria value added (c4) and quality (c3) have the highest priority, which is reasonable in the highly competitive heavy steel industry market. table 4 pairwise comparisons of criteria to maintenance strategy selection criterion c1 c2 c3 c4 c5 priority c1 1 4.77 0.29 0.18 2.57 15% c2 0.21 1 0.24 0.18 1.67 7% c3 3.45 4.17 1 0.48 5.66 30% c4 5.56 5.56 2.08 1 2.33 41% c5 0.39 0.60 0.18 0.43 1 8% table 5 presents a decision matrix. the priorities in this matrix were obtained from comparison matrices of alternatives regarding each criterion aggregated from individual matrices according to every expert. for c1 and c4, the bolded consistency ratios were greater than the upper limit. this inequality was mainly due to difference in the opinions obtained from expert interviews. table 5 local and overall priorities for maintenance strategies strategy c1 (15%) c2 (7%) c3 (30%) c4 (41%) c5 (8%) overall corrective (a1) 38% 52% 47% 54% 36% 48% preventive (a2) 47% 32% 19% 25% 14% 26% predictive (a3) 6% 7% 21% 12% 30% 15% pro-active (a4) 9% 9% 13% 9% 20% 11% cr 0.17 0.04 0.07 0.20 0.10 0.24 the overall consistency ratio was below the upper limit, so the results are acceptable. however, the judgement values for the two criteria should be reviewed. ijahp article: ohta, salomon, silva / selection of industrial maintenance strategy: classical ahp and fuzzy ahp applications international journal of the analytic hierarchy process 262 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.551 4.2. fuzzy analytic hierarchy process the same experts were invited to fuzzify their comparisons. that is, every previous comparison was assumed as a modal value for a triangular fuzzy set. lower and upper values were added with their review. table 6 presents fuzzy pairwise comparisons on the criteria from the expert interviews, aggregated by geometric mean. we have λlow≈5.07, cilow≈0.02 and crlow≈0.01, λmed≈5.77, cimed≈0.19 and crmed≈0.17 and λhig≈7.43, cihig≈0.61 and crhig≈0.54, where crlow≈0.01>0.10 is acceptable, but crmed and crhig are not acceptable. table 6 fuzzy pairwise comparisons of criteria to maintenance strategy selection criterion c1 c2 c3 c4 c5 c1 (1, 1, 1) (4.25, 4.75, 4.95) (.30, .35, .45) (.15, .25, .25) (3.25, 3.45, 3.75) c2 (.20, .21, .24) (1, 1, 1) (.15, .25, .25) (.15, .35, .40) (1.04, 1.71, 2.31) c3 (3.45, 4, 4,55) (3.70, 4.17, 4.76) (1, 1, 1) (.34, .48, .62) (5.17, 5.74, 6.18) c4 (5.00, 5.56, 6.25) (5.00, 5.26, 6.25) (1.61, 2.08, 2.94) (1, 1, 1) (2.16, 2.33, 2.47) c5 (.33, .39, .49) (.43, .58, .96) (.16, .17, .19) (.40, .43, .46) (1, 1, 1) table 7 presents priority vectors for the criteria with ahp and with fahp. vectors are very close, with the same ordinal vector: [3, 4, 2, 1, 5]. that is, c4 has the highest priority in both applications and it is followed by c3, c1, c2, and c5. table 7 priorities of criteria to maintenance strategy selection criterion ahp fahp c1 15% 22% c2 7% 8% c3 30% 33% c4 41% 32% c5 8% 5% table 8 presents a new decision matrix. priorities were obtained from fuzzy comparison matrices of alternatives regarding each criterion aggregated from individual matrices according to every expert. bolded priorities are higher than the values in the previous decision matrix (table 5). table 8 new local and overall priorities for maintenance strategies strategy c1 (22%) c2 (8%) c3 (33%) c4 (32%) c5 (5%) overall corrective (a1) 45% 52% 27% 48% 39% 40% preventive (a2) 39% 30% 12% 33% 14% 26% predictive (a3) 6% 6% 9% 10% 28% 9% pro-active (a4) 9% 12% 52% 9% 19% 24% the results from both classical ahp and fahp were quite similar. the plant must adopt corrective maintenance as a maintenance strategy. ijahp article: ohta, salomon, silva / selection of industrial maintenance strategy: classical ahp and fuzzy ahp applications international journal of the analytic hierarchy process 263 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.551 5. conclusions mcda is widely used when the decision problem has social, economic, technical and political factors that need to be meticulously evaluated in a globalized world where an organization must produce the maximum, with more quality in less time and minimum cost. classical ahp and fuzzy ahp models were applied and compared in a case study to solve the problem of maintenance strategy selection in an industry. despite the convenience of classical ahp in manipulating the criteria for decision making, imprecision and lack of definition may cause inaccurate judgments with conventional approaches. to fill this gap, fuzzy ahp has a greater advantage in capturing the imprecision of human thought and contributing to a structured resolution. moreover, in the case under study, in both models, corrective maintenance (a1) was the strategy with the highest priority and value added (c3) was the criterion with the highest priority. this goes against the principles of evolution of maintenance to reach world class maintenance, where it seeks to minimize costs and combat the lack of maintenance effectiveness. we can also observe that some pairwise comparison matrices in the classic ahp produced consistency ratios (cr) higher than recommended (cr> 0.10), which was possibly generated by errors or misunderstandings in the judgment of the specialists. however, even with these adverse results, the work showed that the mcda could be used in the selection of a maintenance strategy. ijahp article: ohta, salomon, silva / selection of industrial maintenance strategy: classical ahp and fuzzy ahp applications international journal of the analytic hierarchy process 264 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.551 references chang, d.-y. (1996). applications of the extent analysis method on fuzzy ahp. european journal of operational research, 95(3), 649-655. doi:10.1016/03772217(95)00300-2 elsevier. (2017). scopus preview. accessed: 22 nov. 2017, avaiable at https://www.scopus.com ge, y., xiao, m., yang, z., zhang, l., hu, z., feng, d. (2017). an integrated logarithmic fuzzy preference programming based methodology for optimum maintenance strategies selection. applied soft computing journal, 60, 591-601. doi: 10.1016/j.asoc.2017.07.021 kahraman, c., oztaysi , b., & onar, s. c. (2016). a comprehensive literature review of 50 years of fuzzy set theory. international journal of computational intelligence systems, 9(1), 3-24. doi:10.1080/18756891.2016.1180817 karpak, b. (2017). reflections: mathematical principles of decision making. international journal of the analytic hierarchy process, 9(2), 341-348. doi:10.13033/ijahp.v9i3.521 lawrence , j. a., & pasternack, b. a. (2013). applied management science (second ed.). new york, ny: wiley. moubray, j. (2000). reliability-centered maintenance (second ed.). new york, ny: industrial press. pedrycz, w. (1994). why triangular membership functions? fuzzy sets and systems, 64(1), 21-30. doi:10.1016/0165-0114(94)90003-5 pinto, a. k., & nascif, j. (2014). maintenance: a strategic function (portuguese ed.). r. janeiro, brazil: qualitymark. saaty, t. l. (1977). a scaling method for priorities in hierarchical structures. journal of the mathematical psychology, 15(3), 234-281. doi:10.1016/0022-2496(77)90033-5 saaty, t. l. (1980). the analytic hierarchy process. new york: mcgraw-hill. saaty, t. l. (1991). how to make a decision: the analytic hierarchy process. european journal of operational research, 48(1), 9-26. doi:10.1016/0377-2217(90)90057-i saaty, t. l., & peniwati, k. (2013). group decision making: drawing out and reconciling differences. pittsburgh, pa: rws publications. slack, n., brandon-jones, a., & johnston, r. (2016). operations management (eighth ed.). harlow, uk: pearson. ijahp article: ohta, salomon, silva / selection of industrial maintenance strategy: classical ahp and fuzzy ahp applications international journal of the analytic hierarchy process 265 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.551 van laarhoven, p. j., & pedrycz, w. (1983). a fuzzy extension of saaty’s priority theory. fuzzy sets and systems, 11(1-3), 229-241. doi:10.1016/s0165-0114(83)80082-7 wang, l., chu, j., wu, j. (2007). selection of optimum maintenance strategies based on a fuzzy analytic hierarchy process. international journal of production economics, 107 (1), 151-163. doi: 10.1016/j.ijpe.2006.08.005 zadeh, l. a. (1965). fuzzy sets. information and control, 8(3), 338-353. doi:10.1016/s0019-9958(65)90241-x ijahp article: bahurmoz, samrgandi/an anp model to develop a national transformation plan international journal of the analytic hierarchy process 27 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.481 an anp model to develop a national transformation plan 1 asma bahurmoz professor of operations research, department of business king abdul aziz university saudi arabia bahurmoz@kau.edu.sa nahla samrgandi assistant professor of economics, department of economics king abdul aziz university saudi arabia nsamrgandi@kau.edu.sa abstract in october 2015, the saudi council for economic and development affairs unveiled an austerity plan in response to a drop in the price of oil to around us $26, which threw the country into an unexpected deficit of about $100 billion. later on, an austerity plan evolved into the national transformation plan (ntp) 2020. the ntp focused on the key drivers for economic growth. although only an outline of the ntp was released at the time, we speculate on what will be the best implementation policy. knowing that the government is taking a developmental approach that is inclusive, transparent and accountable to all stakeholders in society, we opted to address this question as a multi criteria decision problem formulated in an analytic network process (anp) model. the anp is a sound methodology for structuring the problem to find which policy to focus on to make the transformation as effective as possible, taking into account social and political factors in addition to economic ones. the anp model showed that to make the transformation effective through blending socio-economic and political factors, policymakers must give priority to management reform over privatization and fiscal reform policies. https://doi.org/10.13033/ijahp.v9i1.481 keywords: analytic network process; economic transformation; strategic planning, saudi vision 2030 1. introduction countries dependent on natural resources are vulnerable to the volatility of the prices of these resources on the international market. a recent fall in the international oil price 1 acknowledgements: we are grateful to hussein alkahily of almoragham investments holding co. for sharing his expertise with us during the course of this research, and to rozann saaty and elena rokou of creative decisions foundation for their assistance in using superdecisions software and comments on an earlier version of the model. https://doi.org/10.13033/ijahp.v9i1.481 mailto:bahurmoz@kau.edu.sa mailto:nsamrgandi@kau.edu.sa https://doi.org/10.13033/ijahp.v9i1.481 ijahp article: bahurmoz, samrgandi/an anp model to develop a national transformation plan international journal of the analytic hierarchy process 28 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.481 prompted the saudi government to attempt to protect the economy from this adversity through the national transformation plan (ntp) 2020. the ntp’s main aim is to accelerate economic growth through diversification and offset the impact of falling oil prices. nevertheless, such a plan is often subject to risks and opportunities in addition to the benefits and costs. therefore, the decision makers face the crucial question of what is the best direction to pursue. our objective is to identify the most effective policy to this end. knowing that the government is taking a developmental approach that is inclusive, transparent and accountable to all stakeholders in society, we implemented the analytic network process (anp). it is a sound methodology for structuring the problem to determine which policy to focus on in order to make the transformation as effective as possible, taking into account social and political factors in addition to economic ones. the problem can be best addressed through multi-criteria decision-making (mcdm). this paper uses the anp with its dependence and feedback approach to prioritization to guide decision-makers in clarifying its complexity. we contribute to the literature on economic transformation policy in a number of ways. our study is pioneering in its assessment of the scrutinizing the probable pros and cons of the proposed ntp in terms of social-economic and political aspects. we consider the most plausible empirical framework in assessing the policy incorporating multi-dimensional parameters. the findings of this study are of paramount importance in terms of the implementation of, and further amendments to the ntp. in what follows, a condensed literature review on the natural resource curse and how other countries have dealt with being dependent on the instable market for natural resources is presented. then a brief introduction to the anp, followed by our proposed framework for the ntp model; finally obtained results and conclusions derived from this research are reported. 2. literature review literature on natural resource wealth and its potential adverse impacts on different economies is reviewed, and examples of countries that have successfully avoided the ‘resource curse’ by adopting reform programs are presented. finally, the disclosed contents of the saudi ntp are discussed with regard to this study. abundant natural resources, particularly oil (known as ‘black gold’), were traditionally viewed as a great boon for exporting states. over the last few decades, however, numerous researchers have focused on the curse effect of abundance of resources instead of them being a blessing for many countries (e.g. gelb, 1988; auty, 1990; sachs & warner, 1995, 1999, 2001). sachs and warner (2001) documented that oil-dependent countries such as those in the gulf cooperation council (gcc), nigeria, mexico, and venezuela during the twentieth century were characterized by sluggish and stagnating economic growth, while resource-poor countries such as singapore, taiwan and hong kong enjoyed sustained economic growth. generally, dependency on the export of key resources in countries prone to the resource curse is associated with massive state intervention in the economy, over-employment in the state sector, and a stunted private https://doi.org/10.13033/ijahp.v9i1.481 ijahp article: bahurmoz, samrgandi/an anp model to develop a national transformation plan international journal of the analytic hierarchy process 29 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.481 sector lacking in diversity. this latter description very much represents the case of saudi arabia. earlier literature argues that there are two channels through which the resource curse can be explained. first, in terms of the negative impact on an economy from anything that gives rise to a sharp inflow of foreign currency, such as the discovery of large oil reserves. the currency influx leads to currency appreciation, making the country’s other products less price-competitive on the export market. this phenomena is called ‘dutch disease’ and it occurs when oil-exporting countries receive a massive influx of foreign currency, leading to an appreciation of the real exchange rate that reduces the international competitiveness of resource-abundant countries and adversely affects growth by shifting resources from high-technology and high-skilled sectors to lowtechnology and low-skilled sectors (al-mamun et al., 2016). second, the resource curse induces rent-seeking motives among private companies lobbying for the management of natural resources, which creates a skewed distribution of the benefits of oil rent. van der ploeg (2011) concludes that whether natural resources can be considered a curse rather being a blessing depends on various factors, including rent seeking, governance and institutional quality, although the latter was deemed to be independent of economic growth by brunnschweiler (2008). prior literature also contends that institutions are not a channel to augment resource-led growth, but rather an outcome of the resource. thus, poor institutional quality is the result of natural resources, as natural resource dependence leads to market and institutional failures that inhibit economic growth (matsuyama, 1992; auty, 1990; sachs & warner, 1995; gylfason, 2001; bhattacharyya & hodler, 2014). such consequences of poor institutional quality would be valid in the case of saudi arabia, while the country is characterized with a moderate score in institutional quality (0.58 according to the international country risk guide (icrg). norway, however, presents an example of substantial economic development despite the major role of petroleum in its national economy. holden (2013) argues that resource attributes comprise a blessing in the case of norway due to a well-functioning policy for, and management of, them. norwegian oil resources are realized as a blessing by virtue of a distributional philosophy, whereby the oil resources are held to belong to the nation; the benefits of revenues from them are therefore to be dispersed among the nation, including future generations. clearly, the fact that this socialist-esque vision was eschewed by the comparable north sea oil power, the uk, after the imf financial crisis in 1976 is evidence for this. the norwegian parliament unanimously adopted ten basic principles in june 1972, focusing on supervision, a sufficient national supply of oil, competition among oil extracting firms, environmental conservation, proper use, rules and regulation, national interest, and on collaboration between national and international oil companies on oil discovery (holden, 2013). a recent investigation by mironov and petronevich (2015) has detected the presence of dutch disease in russia, which is a major international power as well as an oil-exporting country. they argue that a substantial inflow of foreign currency from oil exports precipitated the appreciation of the ruble, which made russian exports less competitive and thus inhibited growth in the manufacturing sector. yet the service sector experienced corresponding growth that offset the decline of manufacturing, and the government launched a new tax policy which was targeted to accrue the major part of oil revenues for the national budget under putin’s reforms in the 2000s. the new tax regime has, https://doi.org/10.13033/ijahp.v9i1.481 ijahp article: bahurmoz, samrgandi/an anp model to develop a national transformation plan international journal of the analytic hierarchy process 30 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.481 however, been inefficient as it did not target gas extraction and metallurgy, and price elasticity remains relatively high (mironov & petronevich, 2015). among countries who underwent economic transformation, malaysia is an instructive case, whose economic transformation plan 2020 was implemented by stimulating a thriving private sector since the 1990s (nair et al., 2014). natural resources function as a blessing in the case of malaysia due to the proper management and distribution of resource revenues. oil revenue coupled with trade liberalization, massive industrial growth, privatization and foreign direct investment fueled the economy. doraisami (2015) documents that economic diversification and a solid development program helped the country to reduce dependence on earnings from natural resources. oil rent accounts for about 4.8 per cent of the gdp and contributes a moderate proportion of government revenue, despite a global position of twenty-ninth in terms of the volume of crude oil exported, and twenty-fourth as a natural-gas producer. malaysia enjoyed about 7 per cent economic growth over the last 25 years. nevertheless, the challenge of diversification faces all economies dependent on oil and gas exports (al bassam, 2015). paz (2014) evaluates the implementation of a resource-based industrialization strategy in the brazilian oil sector, revealing that a substantial augmentation of oil production and exports can generate the effects of currency appreciation (dutch disease), which is counterproductive for economic development. and yet, the implementation of resourcebased development strategies under the continuity of macroeconomic policy is necessary to overcome such phenomena. looney (1990) investigates dutch disease in the context of saudi arabia, finding that an appreciation of the exchange rate for saudi currency has both positive and negative macroeconomic impacts, and identified some specific policies to overcome such problematic outcomes, including more active state involvement in the industrial sector and equalizing wages between the tradable and non-tradable sectors by manipulating exchange rates. saudi economic growth is resource dependent; it is therefore vulnerable to several threats. a more direct impact of resource dependency is economic vulnerability to fluctuations in the price of the key resource; any fluctuation of oil prices in international markets has massive impacts on both exporting and importing states. for instance, although great efforts are underway to diversify the national economy, all socioeconomic development in saudi arabia is currently conditional on the oil sector. there have been initiatives to reduce reliance on oil since the 1970s through various economic development plans, particularly after 1982. in 1985, the non-oil-value addition accounted for about a 77 % share of the gdp. this percentage oscillated between 60 % and 72 % during the period 1986–2010. choudhury and al-sahlawi (2000) documented that a significant increase of gdp from the non-oil sector could be explained as a success of diversification since the fourth development plan (1985–1990) and subsequent plans. an increase in the non-oil sector as a result of the fluctuation in the world’s oil demand reflects swings in world oil prices, and ultimately affects the oil sector (hassan et al., 2011). oil, in saudi arabia, contributes nearly 90 % of export earnings. an abrupt fall in the price of oil towards the end of 2015 caused the country to incur an unexpected deficit of $100 billion, which might grow to 50 % of the gdp if the international oil price remains low, unless the kingdom generates alternative sources of revenue to balance its budget. https://doi.org/10.13033/ijahp.v9i1.481 ijahp article: bahurmoz, samrgandi/an anp model to develop a national transformation plan international journal of the analytic hierarchy process 31 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.481 in response to this drop, the government reacted by developing a long-term plan known as ‘vision 2030’, followed by a proposal for the ntp 2020 (www.vision2030.gov.sa/en). when we initiated this study, not many details were disclosed about the vision. the main motive was, however, to generate income to offset the sudden drop in oil prices via a variety of possible policies. our objective is to identify the best policy to achieve the ntp goal using the anp methodology, which has not been used in previous related studies on ntp plans. 3. the analytic network process to make decisions on complex government issues that encompass all the determinant factors/criteria including tangible and non-tangible factors, the anp developed by saaty (2001, 2008b) easily lends itself to handling such decisions. the anp operates on interaction and feedback both within clusters of elements and between the clusters themselves to capture the effects of complex interaction in societies particularly when alternate decisions involve risk and uncertainty. the uniqueness of the anp is it scales judgments of alternate decisions showing the distribution of influence among different factors and clusters of factors affecting or effected by the alternate decisions. each alternative is evaluated with respect to its bocr on each strategic criterion. “benefits (b) are the good things that would result from taking the decision; opportunities (o), are the potentially good things that can result in the future from taking the decision; costs (c), are the pains and disappointments that would result from taking the decision; and risks (r) are the potential pains and disappointments that can result from taking the decision” (saaty2008b, 134). each of these bocr merits has a system of control criteria to help understand the influence of each separately. for example, the political influence is different from the social influence on a decision and each must be examined in a separate network then combined. so, control criteria and their sub-criteria under each merit are defined, and a subnet and its connection for each control criterion is developed. next, they are pairwise compared to determine the best outcome for each control criterion, the alternatives will be combined in the “ideal form” for all of the control criteria under each of the bocr merits (saaty, 2008b). next, the best alternative under benefits will be selected and used to think of benefits, and the best one under opportunities, which may be different to the one under benefits, will be selected to think of opportunities and so on for costs and risks. finally, these four merits must be rated with respect to the strategic criteria using the ratings mode of the ahp to obtain priority ratings for each merit (saaty, 2008a). then priorities are normalized and used to combine the four vectors of the outcomes for each alternative under bocr to obtain the overall priorities. the overall priorities can be obtained through the ratio bo/cr, alternatively, by using the ratings to weight and subtract the costs and risks from the sum of the weighted benefits and opportunities. the super matrix and its powers are the fundamental tools used to establish all the priority vectors resulting from pairwise comparisons of each anp network. for further details of the anp methodology, readers are referred to saaty (2009; 2008a & b). to sum up the anp analysis is performed in these steps: 1. developing control criteria and sub criteria for each of the bocr, performing pair wise comparisons and then prioritizing them. https://doi.org/10.13033/ijahp.v9i1.481 ijahp article: bahurmoz, samrgandi/an anp model to develop a national transformation plan international journal of the analytic hierarchy process 32 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.481 2. developing decision networks and synthesizing their priorities for each of the control criteria and then also for each of the bocr. 3. rating the bocr merits with respect to the strategic criteria to obtain their priorities and weight the four priority vectors of the alternatives with respect to the bocr and derive the final outcome. the anp has already been successfully used during the last decade and proved very successful in decision making in many industries and fields namely; location selection; predictions societal, political, business and even sports; strategic planning; e-business; resource allocation in transportation; project selection; knowledge management; electrical project portfolio; factors affecting success in enterprises; safety manufacturing systems; to name a few (cheng et al., 2005; saaty and ozdemir, 2005; bahurmoz, 2006; raisinghani et al., 2007; wey and wu, 2007; liang and li, 2008; percin, 2010; smithperera et al., 2010; karpak and topcu, 2010; silvestri et al. 2012 ). 4. structuring the anp model for the saudi ntp the model we used is structured as follows: 1. top-level network: a single network that contains the benefits, opportunities, costs and risks nodes (the bocr nodes), and the strategic criteria that are used to evaluate the importance of the bocr nodes for identifying which policy is best for implementing the proposed ntp. 2. control criteria networks: each of the bocr has a subnet attached to it containing three control criteria, economic, political and social, and sub-control criteria. 3. decision networks: a decision subnet is created for the most important control criteria or sub-control criteria. a cluster of the alternatives is included in every decision subnet in addition to other influencing elements such as stakeholders. restricting decision networks to the most important criteria is intended to eliminate lengthy comparisons. 4.1 the ntp model we developed the ntp model for the saudi case as follows. our goal for this model is which policy the government should follow in order to realize the saudi ntp2020 objectives 4.1.1 the alternatives the ntp was first proposed in october 2015 in reaction to the accelerating drop in oil prices that year. the council of economic and development affairs (ceada) invited experts and senior government officials to attend a workshop in riyadh to discuss proposals and initiatives to establish a landmark plan to bring about national transformation. the plan aims to relieve the national economy from its dependence on oil, initiating a set of economic measures to minimize dependence on oil, attaining maximum efficiency for public sector organizations, combatting corruption, creating more jobs for saudis, and developing health and municipal sectors through privatization. more investments in the untapped mining sector were to be attracted, creating obstacle-free environments to exploit the abundance mineral resources, and minimizing expenditures. https://doi.org/10.13033/ijahp.v9i1.481 ijahp article: bahurmoz, samrgandi/an anp model to develop a national transformation plan international journal of the analytic hierarchy process 33 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.481 no information has been released on how these objectives will be attained. however, one can conclude that the government needs to pursue at least one of the following policies to save its economy from the so-called resource curse, and to continue sustainable growth: 1. fiscal reform: fiscal policy involves generating revenue through altering levels of government spending and tax rates to influence aggregate demand in the economy. for the case of saudi ntp2020, it involves decreasing, and eventually eliminating most subsidies that saudi society has enjoyed for decades, in addition to cutting if not eliminating benefit packages that most civil servants used to have and imposing a package of fees for many other services in addition to income tax on non-citizens. there has been a long debate on whether the fiscal policy will work or not. for instance, husain et al. (2008) and pieschacon (2009) argue that fiscal policy works as a transmission mechanism to manage the oil price slack in a macroeconomic context. on the other hand, sachs and warner (1995) said that government spending on public services such as unemployment benefits, education, health, and subsidies among other things results in the rise of prices of non-tradable goods (e.g. health, education and public services relative to tradable goods). therefore, resources are shifted to a less productive sector, which in turn unfavorably affects growth in the long run. saudi policies are not exceptions to this rule, having so far focused mainly on socio-economic welfare depending on abundant oil income. furthermore, when adopting a fiscal policy of deficit financing, the government borrows a large amount of capital and leaves less for the private sector, which drives up interest rates for private sector (i.e. productive) borrowing. consequently, private investment falls because of higher interest rates and generally high costs (and risks) of investment, which is known as crowding out theory. unfortunately, the saudi economy is currently in the midst of this scenario, which might lead to a recession and social resentment as the government starts implementing this policy. 2. management reform: failure of bureaucratic models in responding to the accelerating demand on governmental organizations to operate in a business fashion models yield new concepts for public management (npm). npm essentially emphasizes “the role of performance measurement, transparency, and accountability in strengthening the link between public administration and the citizens” (mauri et al., 2012). it adopts many quality concepts that focus on customer satisfaction. mauri et al. (2012) state that “transition to a new management model is not without problems at any level of public administration and the cultural systems of public organizations seem to have an important role”. the public sector in saudi arabia is similar to many other bureaucratic systems and suffers from inefficiency and ineffectiveness in spite of having a very inflated budget. according to the bertelsmann transformation index (bti) 2016, saudi arabia ranks 101 among 129 countries, although it ranks 49 in terms of its market economy status. this ranking is based on its performance between february 2013 and january 2015. the vision 2030 called for transparency, accountability, and responsibility: pillars for any successful transformation, and if they were to be applied to management, the result would be effective use of resources. hence, npm or management reform based on output measurement rather than on input measurement will likely greatly reduce the civil service budgets, which will enable the government to direct these saved resources to more productive areas. https://doi.org/10.13033/ijahp.v9i1.481 ijahp article: bahurmoz, samrgandi/an anp model to develop a national transformation plan international journal of the analytic hierarchy process 34 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.481 3. privatization: transfer several public services to the private sector such as education and health. this results in the downsizing of the public sector, generating jobs and participating towards its gdp. a successful example is britain’s experience in privatization. eastern european countries offer another example of adopting privatization as a key micro reform in transforming saudi arabia’s economy. generally, the discipline of a free market will force businesses to be efficient and thereby achieve improvements in economic welfare. and yet, it might only lead to monopoly with the private sector simply assuming the role previously occupied by the public sector. saudi arabia’s initial transformation plan has indicated this direction in its ntp. 4. status quo: do nothing; as a safeguard, we include this alternative in our analysis to give the expert the option to reject all the previous three alternatives. an effective ntp needs to implement all three policies. however, one would ask which of the four policies, should have priority over the others in terms of being attempted first. prioritizing the four alternatives should consider the bocr merits and the interests of all actors in the light of the clearly defined strategic goals of the saudi government. implementing the anp will guide us to prioritize those four alternatives to determine where the government should focus initially in order to achieve its ntp goals over the next five years. 4.1.2 the strategic criteria the sequential saudi development plans (2004–2020) indicate the following long-term strategies that the government is committed to achieve via its planning: 1. international competitiveness. 2. social well-being: human well-being and socio-economic growth. 3. sustainability 4.1.3 merits three control criteria are used to define the influence of each merit. these are economic, political and social factors. we consulted a pool of experts (table 1), to answer 12 basic questions exploring the most expected economic benefits, political benefits, and social benefits, and the opportunities, costs, and risks respectively. the following common points were extracted from these interviews and used to design the bocr part of theanp model. 1-economic benefits 1. decreased tariff barriers on saudi exports: by removing subsidies on petrochemical feedstocks, saudi exports will be more accepted on the international market; tariffs, barriers, and restrictions will be removed. 2. diversification of the economy: there will be an increase in exports facilitated by saudi arabia’s massive infrastructure and manufacturing base. 3. effective public performance and managing resources efficiently: removing subsidies on utilities (water, electricity, and petrol) will increase efficieny and effectivness in using resources. https://doi.org/10.13033/ijahp.v9i1.481 ijahp article: bahurmoz, samrgandi/an anp model to develop a national transformation plan international journal of the analytic hierarchy process 35 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.481 table 1 details of experts consulted in building up the anp model expert # title area of expertise 1 professor, unu-merit. ex minister of education. innovation and policy, governance and institution 1 professor education, social and welfare services/ ngo 1 associate professor. business/ municipality affairs. 2 assistant professors petroleum, energy and resource economics 2 assistant professors finance/ banking/ corporate finance 1 assistant professor decision analysis/ public affairs 1 finance consultant, cpa (texas). ex city bank. saudi finacial market and banking system. 1 head of research centre chamber of commerce. attended the initial workshop held by seada 1 consulting member of national committee for consulting in the council of saudi chambers human resources development. private sector. strategic planning 2-political benefits 1. better relations with countries who put tariffs on saudi products due to high subsidies. as a result of the removal of subsidies, both for the citizens and for industry, saudi arabia will have better relations with importers and close the door on ‘unfair competition practice’ complaints, especially from large importers such as india, china, japan, and the eu, who have already voiced concerns on saudi subsidation of its industry. this will be a major political step, leading to harmony with the international trading world. 2. competitiveness: as a productive country and not relying only on oil. on a local saudi level, there will be more harmony within society when the economy is more productive/competitive and diversified, and not reliant solely on the fluctuating price of oil. 3. stability through sustainable growth: a broadly based and diversified economy gives the country a strong base and foundation for a sustainable economy. 4. national cohesion: a diversified, sustainable economy—based on nationwide agreement on clear goals—gives the country a sense of strength and cohesion, with a common future to strive for. 3-social benefits 1. productive society: empowered individuals will be more productive through substituting dependency for self-reliance. removing subsidies on petrol and utilities will make society move from a dependent model to self-reliance; it will thus become more productive as they will be paying their utilities and petrol on cost. 2. social awareness: an efficient society in its use of water and electricity, and a healthier society as a result of higher taxes on tobacco and soft drinks. with the efficient use of utilities coupled with rising rates due to consumption, society will be equitable; the poor will pay a lower rate, while the rich, who consume more, will pay a higher rate. it is a means of narrowing the income gap. 3. sustainable future for economic growth: as a prelude for expenditures in education, https://doi.org/10.13033/ijahp.v9i1.481 ijahp article: bahurmoz, samrgandi/an anp model to develop a national transformation plan international journal of the analytic hierarchy process 36 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.481 research, and health. with the removal of subsidies, these funds will be directed to investments in education, research, and health; these investments will be future pillars and precursors of economic growth. 4-economic opportunities 1. economic efficiency: the removal of subsidies will lead the private sector to focus on domestic high value added industry and services that will also be augmented by government development investment fund. 2. empower private sector by reducing its dependence on the government. the ntp will only be implemented by the involvement and empowerment of the private sector, thus minimizing the imbalance in the economy that is heavily reliant on government. here, private sector involvement means both local and foreign direct investment (fdi), (yazdan & hosain, 2013) 3. increase government resources: the broadening of the economic sector by increasing private sector involvement will increase the base for employment and government revenues in fees and taxes/zakat. 5-political opportunities 1. compliance with world trade organization (wto): improves wto legitimacy. with the drive to increase involvement of the private sector, both local and fdi, saudi arabia will improve its ease of doing business for foreign investors. this will improve its standing internationally and with the wto. 2. international partnership: improved saudi business standing will enhance the transfer of knowledge/technology and expertise. attracting fdi, together with greater involvement from the local private sector, will improve business. 3. leadership and making a change: due to its highly developed infrastructure and welleducated youth, saudi arabia can easily attain leadership when preciptating change to come out of the bottleneck of solely depending on government handouts that are stagnating the economy. 6-social opportunities 1. transparent society: as the economy moves towards efficiency with greater private sector involvement, it will be forced to become more transparent, and nepotism will therefore be suppressed. 2. skill development will be the focus of education: education will a play stronger role to emphasize this point. even though saudi arabia has a high literacy rate with a highly urbanized society, education will also be directed towards market demand, i.e. an emphasis on skill and technical education rather than pure academia. 3. aligning social segments: economic efficiency and the shift in education towards skills development and a technically adept society will align the conservative forces with modernizing forces based on the empirical evidence of the alternative saudi arabia has to take to survive a changing world. 7-economic costs 1. inflation: the major risk from this transformation is inflation, especially for lower income saudis. to mitigate this effect, rates for utilities will be based on consumption; this will place a smaller burden on lower income saudis. the government is also talking about a social safety net where the lower income saudis will get financial support. it is important that the government gives due consideration https://doi.org/10.13033/ijahp.v9i1.481 ijahp article: bahurmoz, samrgandi/an anp model to develop a national transformation plan international journal of the analytic hierarchy process 37 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.481 to the cost of subsidy removal on lower income saudis. 2. central planning problem: without a clear regulatory framework, involvement of the private sector might not be as efficient as planned. given that the programme is a dramatic one for private sector involvement, efficiency in the shift from a central planning stage (top to bottom) must be carefully considered. private sector intelligentsia involvement could mitigate the efficiency risk in transformation shift. 3. programme cost: the ntp is a dramatic and daring one that will overhaul the economy, but it has not yet addressed the cost of the programme, both directly and indirectly, and how it will be absorbed. 8-political costs 1. resentment if the ntp is not well presented: 2. power sharing: in return for funding the ntp, the private will ask for a share of the power in return; this is a new phenomenon for the arab world, and the government has to adapt to a new concept as such. 3. credibility at stake. by nature humans resist change; more so if the ntp is not well presented to all stakeholders. this is a major risk the government needs to address and have social scientists/demographics experts get involved to address vulnerable segments and implement social safety nets. 9-social costs 1. burden on the consumer: especially those with lower incomes. cost of living will be higher for the lower income strata of society. those who live on the safety net will be greatly affected. the unravelling of society from a patronage to a market-oriented model runs the risk of placing a major burden on it, particularly the vulnerable segment; this has to be addressed with great care. 2. reduces job opportunities for those with low education levels, or those who are not skilled enough to adjust to new job market demands. until the labour market adjusts, there is a major risk of having a segment of the work force that is not prepared to adapt. training and re-training of the labour force will be paramount to alleviate this risk. 10-economic risks 1. environmental changes: the ntp’s main element is to get the economy more connected to world finance and will thus be more dependent and interrelated to its regional and international environment; starting from the well-being of world economy to regional tensions and peace. 2. corruption: a major concern accompanying the implementation of privatization is a lack of transparency during that process, as witnessed in other countries. 3. underestimate the requirements of the ntp: it is expected that as ntp is implemented, new requirements and costs will show up along the way. the government has to be prepared to expect this and must have a plan how to tackle these issues. 11-political risks 1. resentment at the higher cost for saudi companies: higher costs for saudi companies will create resentment from the business class. saudi companies have been accustomed to operating in a relaxed mode that is dependent on government handouts/lavish expenditures. opening up the economy to fdi will put pressure on https://doi.org/10.13033/ijahp.v9i1.481 ijahp article: bahurmoz, samrgandi/an anp model to develop a national transformation plan international journal of the analytic hierarchy process 38 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.481 local companies to adapt. 2. natural resentment to change: local companies will also resent changes as they have always been protected from foreign competition. 3. credibility at stake: addressing these issues will put the credibility of the ntp and the government at stake. the government should have contingency plans for unexpected resentment. 12-social risks 1. marginalizing the middle and lower classes. the ntp represents a major shift in the thinking of society and the way that the economy operates. there is a high possibility that the middle and lower classes will be marginalized. the government should expect this and have plans to address it to mitigate the effect. 2. transition period might take longer. a major shift the ntp is planning to implement in a society that has been on patronage for decades will definitely take a long time to implement. the government should be aware of the potential for a longer transition period, and thus work on programmes to mitigate costs; an efficient social safety net is therefore of paramount importance. figure 1 shows the strategic criteria and bocr. the control criteria, and their subcontrol criteria are embedded in subnets under each of the four merits (bocr). figure 1. top-level network of the ntp model 4.2 the control criteria network for each merit a subnet containing the control criteria but with different control subcriteria is created. figure 2 exhibits the benefits control criteria subnet. the three control criteria are pairwise compared with regard to the benefits (figure 3). then, in the decision subnet under the economic control criterion, sub-criteria of economic benefits: decrease tariff barriers, diversification, and effective public performance are pairwise compared with respect to economic benefits, and so on for the political benefits and the social benefits. these comparisons result in priorities for all the control criteria and subcriteria. (table 2). https://doi.org/10.13033/ijahp.v9i1.481 ijahp article: bahurmoz, samrgandi/an anp model to develop a national transformation plan international journal of the analytic hierarchy process 39 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.481 figure 2. subnet benefits control criteria for the ntp model figure 3. results of prioritization of the control criteria with regard to benefits table 2 detailed priorities of bocr economic opportunities and political benefits are the most important on the positive side (.76 & .60). however, the rates of the social costs and the social risks are also high (.67 https://doi.org/10.13033/ijahp.v9i1.481 ijahp article: bahurmoz, samrgandi/an anp model to develop a national transformation plan international journal of the analytic hierarchy process 40 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.481 & .63). global priorities are computed by multiplying the local priority of the sub-control criteria times the priority of its corresponding control criterion times the weight of its corresponding merit, which has been obtained from the rating system (as will be explained later). for example, the global priority for decreased tariff barriers on saudi exports equals: .23 × .33 × .23 = .017, as shown in table2. 4.3 decision networks there is more than one option when building the decision networks. one can either select the highest sub-control criteria, or the high-priority control criteria, and build a decision subnet for each, or build a decision subnet for each control criterion. we have chosen to exclude all sub-control criteria whose control criteria priority is less than 0.25, and all sub-criteria whose priority is less than 0.03; then we created decision nets for the remaining factors under their corresponding control criteria. thus, we will focus on the most important criteria gathered in six decision subnets. each one contains three clusters; both of the alternatives cluster and a cluster contains the stakeholders who influence decisions in relation with the control criterion. the third cluster contains the sub-control criteria (whose global priority is ≥0.30) and corresponds to each control criterion (whose priority is ≥.25). figure 4 shows the decision network under economic benefits. this decision subnet has two clusters in addition to that containing the alternatives. figure 4. decision network under economic benefits while each alternative affects each stake holder, those stake holders have influence on the decision of the ceada in addition to backward interactions between the alternatives and the economic benefits criteria and the stakeholders. priorities of the alternatives resulting from pairwise comparisons are idealized and weighted by the normalized value of the control sub-criterion of the economic benefits subnet. results are shown in table 3. this procedure is repeated for the political benefits subnet as well. the overall priorities of the alternatives with regard to benefits are shown in table 4. the same procedure is repeated for opportunities, costs, and risks. overall priorities results for alternatives ranking from bocr merits in ideal format are reported in table 5. https://doi.org/10.13033/ijahp.v9i1.481 ijahp article: bahurmoz, samrgandi/an anp model to develop a national transformation plan international journal of the analytic hierarchy process 41 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.481 table 3 alternative rankings from the benefits: economic sub-net alternatives total normal ideal ranking fiscal reform 0.0884 0.1944 0.5132 3 management reform 0.1143 0.2513 0.6634 2 privatization 0.1722 0.3788 1.0000 1 status quo 0.0798 0.1756 0.4635 4 table 4 alternative rankings from the benefits net alternatives total normal ideal ranking fiscal reform 0.4504 0.1763 0.5074 3 management reform 0.8878 0.3474 1.0000 1 privatization 0.8761 0.3428 0.9869 2 status quo 0.3411 0.1335 0.3843 4 table 5 alternatives ranking from bocr merits in ideal format name benefits opport. costs risks fiscal reform 0.5074 1.0000 1.000 1.0000 management reform 1.0000 0.5137 0.2310 0.4378 privatization 0.9869 0.9148 0.6069 0.7221 status quo 0.3843 0.1502 0.5232 0.4997 4.4 rating the bocr with respect to the strategic criteria in a highly complicated decision benefits, opportunities, costs and risks are not equally important; in order to determine their importance in a bocr analysis they are prioritized with respect to the strategic criteria (saaty, 2008a). the strategic criteria which the government has committed itself to achieve in the long term are: international competitiveness, social wellbeing, and sustainability. their priorities are shown in table 6. intensities will then be determined to indicate the degree of fulfillment for benefits and opportunities, or impact for costs and risks. the intensities are categorized into high, medium, and low. then they were pairwise compared: high (1.0), medium (.25), and low (.125). table 6 prioritizing the strategic criteria with respect to goal international competitiveness .0750 social wellbeing .5917 sustainability .3332 these intensities and their priorities are used for each strategic criterion. to select the rating given in table 7, we pose the question, what is the highest-valued alternative for benefits in the synthesized results for the benefit control subnet. from table 5, the highest priority alternative is management reform. keeping that highest alternative in mind, i.e, management reform, perform ratings across the benefits row as to how it https://doi.org/10.13033/ijahp.v9i1.481 ijahp article: bahurmoz, samrgandi/an anp model to develop a national transformation plan international journal of the analytic hierarchy process 42 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.481 contributes to fulfill each of the three strategic criterion. this can then be repeated across the opportunities row for highest value alternative therein, which is, fiscal reform, and so on. note that, the highest alternative will be the worst one for costs (risks), so the question to ask is: how does this worst alternative, for costs (risks) affect the strategic criteria? from table 5, the most costly and risky option is fiscal reform. so we ask, how does each impact the kingdom’s strategic polices? the overall priorities for each of the bocr merits are computed by multiplying the merit rate times the priority of the corresponding strategy and adding across each row. for example, to calculate the benefits priority: .0750*.25+.5917*1.0+.3332*1.0 = 0.9437. to normalize it, we divide .9437 by the summation of the bocr totals. the normalized priorities are given under the priorities column in table 7. table 7 bocr priority ratings idealized intensities: high (1.00), medium (.250), low (.125) merits priorities totals competitive 0.07591727 sustain. 0.591727 wellbeing 0.333216 benefits 0.312413 0.943707 m h h opport. 0.215891 0,652144 m h l costs 0.162389 0.490530 l m h risks 0.309307 0.934325 l h h 5. results the synthesized results for each alternative are obtained in two ways. the first is by directly implementing the multiplicative formula (bo/cr) and normalization (table 5). alternatively, and more importantly for our case, we apply the negative additive formula bb+oo-cc-rr to synthesize the outcomes, where b, o, c and r are the values for the bocr as determined in the ratings (table 7) are used to weight the four vectors under bocr (table 5), which were obtained from the synthesized results. the latter formula might give negative numbers. this means that those alternatives with negative numbers are undesirable. if all are negatives, then we choose the least negative one as the best alternative. the multiplicative formula gives the best alternative for the short term, while the additive (negative) gives the best alternative for the long term. the outcome for each formula is shown in figures 5 and 6 respectively. it is clear that the option for the ntp is to focus on management reform in order to attain its goals with minimum costs and risks. once management reform has taken place, privatization then assumes priority over fiscal reform. figure 5. synthesis of the alternatives in multiplicative formula https://doi.org/10.13033/ijahp.v9i1.481 ijahp article: bahurmoz, samrgandi/an anp model to develop a national transformation plan international journal of the analytic hierarchy process 43 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.481 figure 6. synthesis of the alternatives in additive (negative) formula sensitivity analysis figure 7. sensitivity analysis figure 7 shows that when benefits exceed .35, privatization is best. it appears that management reform succeeded because it is cheap in comparison to the other alternatives (see the costs sensitivity). it is also less risky, but it seems that costs drive the decision towards management reform. when we examine the costs control criteria, it shows that the political cost is the highest (table 2). in summary, management reform and privatization should be addressed first; only after these two policies are well established can one consider fiscal reform if there is a need for it. 6. conclusions given that this study was conducted after the idea of the ntp (known later on as vision 2030) was introduced in a workshop in riyadh, and given that nothing was released apart from very vague ideas and many speculations, the study here was therefore completed before vision 2030 was made public on 25 april 2016 and followed immediately by farreaching internal government alterations. several royal decrees renamed and merged a number of ministries, and other ministries and commissions were reformulated in an effort to improve government responsiveness and efficiency, giving top priority to management reform. the detailed ntp 2020 was published four weeks later. it provides a concrete idea of many policies the government intends to pursue between now and 2020. these policies reflect austerity measures and focus on fiscal reform. changing and merging ministries might not ameliorate the situation unless accompanied by rigorous institutional changes. the philosophy of the npm, which is based on implementing a set of management techniques and practices, mostly associated with private sectors, must be fully adopted. the concepts such as lean thinking and total quality management will yield huge savings and eliminate resource wasting, which is very common in the current bureaucratic system. these sort of changes might be https://doi.org/10.13033/ijahp.v9i1.481 ijahp article: bahurmoz, samrgandi/an anp model to develop a national transformation plan international journal of the analytic hierarchy process 44 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.481 challenging and take time to produce results, but they can be implemented given that the current leadership is serious about real transformation. focusing on fiscal reform in order to generate revenue quickly, even though it might help in the short-term as a ‘quick-fix’ solution, is far more damaging in the long-term as it might result in accumulated dissatisfaction given that the country is suffering from unemployment, low salaried jobs, and the shrinking of the middle class. attempting to impose any further financial duties on the public in general will not help. fiscal reform must be preceded by a radical management reform, and must be accompanied by involving people in the decision and the process. the anp model enables us to include different factors that would influence the saudi ntp performance despite their qualitative and contradictory nature. upon examining the global criteria priorities of all the proposed elements (table 2), “marginalizing the public” is the major political risk that might hinder performance of the proposed ntp. it has the highest global score 15.9%. this indicates the necessity of involving the public and making them part of the planning process. implementing the anp approach made it possible to recognize the magnitude of the risk of such an element. our model findings have proved robust; only a few months after implementing the saudi vision 2030 which focused on financial reform the government has reversed its priorities. royal decrees on 23 april 2017 restored financial allowances and bounces for civil servants and military personnel that had been abolished under austerity measures last september. two members of the cabinet have been removed, one is subject to an impeachment trial. there were many changes that address institutional reforms in response to the people’s voice, and an attempt to keep the economy from sinking into depression. this direction conforms to the anp model developed in this study. anp has an advantage over other decision making methodologies in that it does not require detailed data that is a must for other decision methodologies. for the anp, the expertise of the decision-maker and their deep knowledge of the problem are more important than detailed data. this bring us to the limitation of this study; our model results are based on consulting academics and practitioners who are not fully involved in the ntp due to the difficulty in accessing these people. it is worthwhile to organize a workshop for those who are involved in the ntp and elicit their own judgments. https://doi.org/10.13033/ijahp.v9i1.481 ijahp article: bahurmoz, samrgandi/an anp model to develop a national transformation plan international journal of the analytic hierarchy process 45 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.481 references al bassam, b. a. (2015). economic diversification in saudi arabia: myth or reality? resources policy, 44,112-117.doi: https://doi.org/10.1016/j.resourpol.2015.02.005 al mamun m., sohag k., samargandi n, yasmeen f (2016). does remittance fuel labour productivity in bangladesh? the application of an asymmetric non-linear ardl approach. applied economics. 48(50), 4861-4877. doi: http://dx.doi.org/10.1080/00036846.2016.1167825 auty, richrd m (1990). resource-based industrialization: sowing the oil in eight exporting countries. new york: oxford university press. bahurmoz, a. (2006). a strategic model for safety during the hajj pilgrimage: an anp application. journal of systems science and systems engineering. 15(2), 201–216. doi: 10.1007/s11518-006-5008-4 bhattacharyya s, hodler r. (2014). do natural resource revenues hinder financial development? the role of political institutions. world development, 57, 101-113. doi: https://doi.org/10.1016/j.worlddev.2013.12.003 brunnschweiler, c. n. (2008). cursing the blessings? natural resource abundance, institutions, and economic growth. world development. 36(3), 399-419. doi: https://doi.org/10.1016/j.worlddev.2007.03.004 cheng, eddie w.l.; li, heng; yu, ling (2005). the analytic network process approach to location selection: a shopping mall illustration. construction innovation. 5 (2), 83-98. doi: http://dx.doi.org/10.1108/14714170510815195 choudhury, m. a., & al‐sahlawi, m. a. (2000). oil and non‐oil sectors in the saudi arabian economy. opec review. 24(3), 235-250. doi: 10.1111/1468-0076.00083 doraisami, a. (2015). has malaysia really escaped the resource curse? a closer look at the political economy of oil revenue management and expenditures. resources policy. 45, 98-108. doi: https://doi.org/10.1016/j.resourpol.2015.03.008 gelb ah (1988). oil windfalls: blessing or curse? oxford: oxford university press. doi: 10.1002/jid.3380010307 gylfason t (2001). natural resources, education, and economic development. european economic review, 45(4), 847-859. doi: https://doi.org/10.1016/s0014-2921(01)00127-1 hassan mk, sanchez b, yu js (2011). financial development and economic growth: new evidence from panel data. quarterly review of economics and finance, 51(1), 88104. doi: https://doi.org/10.1016/j.qref.2010.09.001 holden s. (2013). avoiding the resource curse the case norway. energy policy, 63,870876. doi: https://doi.org/10.1016/j.enpol.2013.09.010 https://doi.org/10.13033/ijahp.v9i1.481 https://doi.org/10.1016/j.resourpol.2015.02.005 http://dx.doi.org/10.1080/00036846.2016.1167825 https://doi.org/10.1016/j.worlddev.2013.12.003 https://doi.org/10.1016/j.worlddev.2007.03.004 http://dx.doi.org/10.1108/14714170510815195 https://doi.org/10.1016/j.resourpol.2015.03.008 https://doi.org/10.1016/s0014-2921(01)00127-1 https://doi.org/10.1016/j.qref.2010.09.001 https://doi.org/10.1016/j.enpol.2013.09.010 ijahp article: bahurmoz, samrgandi/an anp model to develop a national transformation plan international journal of the analytic hierarchy process 46 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.481 husain a.m., k. tazhibayeva and a. ter-martirosyan (2008). fiscal policy21 and economic cycles in oil-exporting countries. imf wp/08/253. doi: http://dx.doi.org/10.5089/9781451871111.001 karpak b, topcu i (2010). small medium manufacturing enterprises in turkey: an analytic network process framework for prioritizing factors affecting success. international journal of production economics, 125(1), 60-70. doi: https://doi.org/10.1016/j.ijpe.2010.01.001 liang ch, li q (2008). enterprise information system project selection with regard to bocr. international journal of project management, 26, 810-820. doi: https://doi.org/10.1016/j.ijproman.2007.11.001 looney, r. e. (1990). oil revenues and dutch disease in saudi arabia: differential impacts on sectoral growth. canadian journal of development studies/revue canadienne d'études du développement, 11(1),119-133. doi: http://dx.doi.org/10.1080/02255189.1990.9669386 mauri ag, muccio s (2012). the public management reform: from theory to practice. international journal of advances in management science, 1(3), 47-56. matsuyama k. (1992). agricultural productivity, comparative advantage, and economic growth. journal of economic theory, 58(2), 317-334. doi: https://doi.org/10.1016/00220531(92)90057-o mironov v. v., petronevich a. v. (2015). discovering the signs of dutch disease in russia. resources policy, 46(2), 97-112. doi: https://doi.org/10.1016/j.resourpol.2015.09.007 nair v., chiun l. m., & singh s. (2014). the international tourists’ perspective on malaysia's economic transformation programme (etp). procedia-social and behavioral sciences, 144,433-445. doi: https://doi.org/10.1016/j.sbspro.2014.07.313 paz m. j. (2014). oil and development in brazil: between an extractive and an industrialization strategy. energy policy, 73,501-511. doi: https://doi.org/10.1016/j.enpol.2014.06.021 percin s. (2010). the use of analytic network process in selecting knowledge management strategies. management research review, 33(5), 452 – 471. doi: http://dx.doi.org/10.1108/01409171011041893 pieschacon a. (2009). oil booms and their impact through fiscal policy. manuscript, stanford university. raisinghani hs, meade l, schkade l (2007). strategic e-business decision analysis using analytic network process. ieee transactions engineering management, 54 (4), 673686. doi: 10.1109/tem.2007.906857 https://doi.org/10.13033/ijahp.v9i1.481 http://dx.doi.org/10.5089/9781451871111.001 https://doi.org/10.1016/j.ijpe.2010.01.001 https://doi.org/10.1016/j.ijproman.2007.11.001 http://dx.doi.org/10.1080/02255189.1990.9669386 https://doi.org/10.1016/0022-0531(92)90057-o https://doi.org/10.1016/0022-0531(92)90057-o https://doi.org/10.1016/j.resourpol.2015.09.007 https://doi.org/10.1016/j.sbspro.2014.07.313 https://doi.org/10.1016/j.enpol.2014.06.021 http://dx.doi.org/10.1108/01409171011041893 https://doi.org/10.1109/tem.2007.906857 ijahp article: bahurmoz, samrgandi/an anp model to develop a national transformation plan international journal of the analytic hierarchy process 47 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.481 saaty t. (2001). the analytic network process: decision making with dependence and feedback. pittsburgh (pa): rws publications. saaty t. (2005). theory and applications of the analytic network process. pittsburgh (pa): rws publications. saaty t. (2008a). decision making with the analytic hierarchy process. international. journal of services sciences, 1(1), 83–98. saaty t. (2008b). applications to decisions under risk. european journal of pure and applied mathematics, 1(1), 122-196. saaty t. (2009). applications of analytic network process in entertainment. iranian journal of operations research, 1(2), 41-55. saaty t., ozdemir m. (2005). the encyclicon. pittsburgh(pa): rws publications. sachs jd, warner am (1995). natural resource abundance and economic growth (no. w5398). national bureau of economic research. sachs jd, warner am (1999). the big push, natural resource booms and growth. journal of development economics, 59(1), 43-76. doi: https://doi.org/10.1016/s03043878(99)00005-x sachs jd, warner am (2001). the curse of natural resources. european economic review, 45(4), 827-838. doi: https://doi.org/10.1016/s0014-2921(01)00125-8 van der ploeg, f. (2011). natural resources: curse or blessing? journal of economic literature, 49(2), 366-420. doi: 10.1257/jel.49.2.366 silvestri a, de felice f, petrillo a (2012). multicriteria risk analysis to improve safety in manufacturing systems. international journal of production research, 50(17), 48064821. doi: http://dx.doi.org/10.1080/00207543.2012.657968 smith-perera a, garcía-melón m, poveda-bautista r, pastor-ferrando jp (2010). a project strategic index proposal for portfolio selection in electrical company based on the analytic network process. renewable and sustainable energy reviews, 14, 1569-1579. doi: https://doi.org/10.1016/j.rser.2010.01.022 wey w, wu k (2007). using anp priorities with goal programming in resource allocation in transportation. mathematical and computer modeling, 46(7-8), 985-1000. doi: https://doi.org/10.1016/j.mcm.2007.03.017 yazdan gf, hossein ssm (2013). fdi and ict effects on productivity growth. procediasocial and behavioral sciences, 93, 1710-1715. doi: https://doi.org/10.1016/j.sbspro.2013.10.104 https://doi.org/10.13033/ijahp.v9i1.481 https://doi.org/10.1016/s0304-3878(99)00005-x https://doi.org/10.1016/s0304-3878(99)00005-x https://doi.org/10.1016/s0014-2921(01)00125-8 http://dx.doi.org/10.1080/00207543.2012.657968 https://doi.org/10.1016/j.rser.2010.01.022 https://doi.org/10.1016/j.mcm.2007.03.017 https://doi.org/10.1016/j.sbspro.2013.10.104 ijahp news and events: ferretti, valentina / 22nd international conference on multiple criteria decision making, málaga, spain international journal of the analytic hierarchy process 181 vol.5 issue 1 2013 issn 1936-6744 ahp news ahp/anp was well-represented in the mcdm 2013 meeting in málaga, spain some ahp/anp paper presenters in mcdm 2013, málaga, andalucía, spain. from left to right: ozay ozaydin, fusun ulengin, birsen karpak, valentina ferretti and ilker topcu it will for sure be very difficult to forget such a well-organized, sparkling and outstanding meeting as the mcdm 2013 conference that took place in warm málaga (spain) from the 17th to the 21st of june 2013 (http://www.mcdm2013.decytec.ccee.uma.es/). the general theme of the conference was “mcdm for tomorrow’s world” and called attention to the hard times of economical crisis that we are facing in many countries around the world, thus highlighting the need for decisions that provide us with solutions in the short term, but that alsobuild a solid basis for future development in a more sustainable way. thanks to the active participation of more than 350 attendees, coming from 46 countries all over the world and including theoreticians, method and software http://www.mcdm2013.decytec.ccee.uma.es/ ijahp news and events: ferretti, valentina / 22nd international conference on multiple criteria decision making, málaga, spain international journal of the analytic hierarchy process 182 vol.5 issue 1 2013 issn 1936-6744 developers and people applying mcdm methods in numerous fields ranging from environment and policy, to engineering, healthcare, economics, etc, the conference brought together many research areas, applications, and perspectives into a truly international and interdisciplinary meeting. again the analytic hierarchy process/ analytic network process stream has raised a lot of attention and several sessions were organized throughout the conference. it is thus worthy to briefly recall them so that the interested reader can find an update on the most recent applications and research in his or her field. in a nutshell, the first ahp/anp session, titled “methodological issues (i)” and chaired by professor ludmil mikhailov, took place on monday afternoon and brought together applications in higher open and distance learning systems (by zehra kamisli ozturk), defense planning (by ahmet kandakoglu, y. riza kahraman & ilker topcu), in mcdm with microsoft excel as a tool (by radomir perzina & jaroslav ramik) and amultiobjective evolutionary approach to preference elicitation (by ludmil mikhailov). the second ahp/anp session, titled “ahp/anp for sustainability” and chaired by professor birsen karpak, took place on tuesday morning and included contributions concerning strategic information systems for a sustainable supply chain (by steve taraszewski & birsen karpak), spatial anp for supporting scenario analysis (by valentina ferretti, marta bottero & giulio mondini), mcdm for resource efficient bread supply chains (by aleksander banasik, g.d.h. claassen, argyris kanellopoulos & jack vander vorst), and the prioritization of third party logistics providers (by birsen karpak & ozden bayazit). the third ahp/anp session, titled “ahp/anp & dematel” and chaired by professor gwo-hshiung tzeng, took place on tuesday afternoon and brought together applications concerning optimal resort development investment projects (by chiaochen chang & t. tyrone lin), tourism competitiveness improvement strategy (by gwohshiung tzeng, ming tsang lu, hsin chuan peng & kua hsin peng), adoption of tourist guide mobile applications (by yang chieh chin) and enhancement of smart mobile devices (by gwo hshiung tzeng, chi-yo huang & yu sheng kao). the fourth ahp/anp session, titled “ahp/anp in practice (i)” and chaired by professor michael bruhn barfod, took place on tuesday afternoon and included contributions related to third-party logistic supplier identification (by birsen karpak), assessment of renewable energies (by arayeh afsordegan, núria agell, lázaro v. cremades, monica sanchez & siamak zahedi), and risk analysis (by michael bruhn barfod, kim bangsalling & rob van den honert). ijahp news and events: ferretti, valentina / 22nd international conference on multiple criteria decision making, málaga, spain international journal of the analytic hierarchy process 183 vol.5 issue 1 2013 issn 1936-6744 the fifth ahp/anp session, titled “ahp/anp in practice (ii)” and chaired by professor valerio salomon, took place on wednesday morning and brought together applications concerning hospital site selection (by hafize yilmaz, sait gül & ilker topcu), product ecodesign (by pedro nicolás casas páez & felix antonio cortes aldana), the elephasr project (by fernando pérez rodrìguez, ángeles camino saco, esteban gomez garcìa, alejandro mosquera & alberto rojo-alboreca), and standardization of classroom furniture (by valerio salomon, pedro alonso & fernando marins). the sixth ahp/anp session, titled “ahp/anp applications to development economics” and chaired by professor josé a. gómez-limón, took place on thursday afternoon and included contributions related to the assessment of investments (by marìa doloresguerrero-baena & josé a. gómez-limón), green supplier selection in taiwan (by hsu-shih shih,chiau-ching chen & yi-chun lin), analysis of global competitiveness (by ozay ozaydin, bora cekyay, ozgur kabak, sule onsel ekici & fusun ulengin) and the provision of agricultural public goods (by anastasio villanueva-rodrìguez, manuel arriaza, josé a. gómez-limón & olexandr nekhay). finally, the last ahp/anp session, titled “methodological issues (ii)” and chaired by professor gang kou, took place on friday morning and included research concerning bayesian analysis and group decision making (by gang kou, changsheng lin & yi peng), a structural modeling approach with interrelated components (by jerzy michnik) and, finally, a comparative analysis and evaluation of multi-criteria decision making methods (by gang kou, daji ergu & thomas l. saaty). all these sessions have the merit of having stimulated a cross-fertilization of ideas and having provided many interesting insights for advancing research and practice in the mcdm field, thus creating very positive expectations for the next mcdm meeting that will be held in 2015 in hamburg (germany) and will be chaired by professor martin j. geiger. valentina ferretti department of housing and city (dicas) politecnico di torino italy e-mail: valentina.ferretti@polito.it rob typewritten text http://dx.doi.org/10.13033/ijahp.v5i1.181 ahp news powerpoint presentation 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 fabio de felice, birsen karpak, enrique mu, leandro pecchia, antonella petrillo and luis vargas 12th international symposium on the analytic hierarchy process 2013 panel discussion publishing ahp/anp papers panelists rozann saaty (chairperson) 116 rob typewritten text http://dx.doi.org/10.13033/ijahp.v5i1.187 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 rozann saaty managing editor, ijahp rozann@creativedecisions.net 12th international symposium on the analytic hierarchy process 2013 panel discussion publishing ahp/anp papers introduction 117 mailto:rozann@creativedecisions.net 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 panel discussion publishing ahp/anp papers 118 the purpose of this panel is to offer some helpful suggestions on the art of writing and publishing ahp/anp papers. the members of the panel are all senior ahp practitioners who have published many papers on ahp. each member of the panel will speak for a few minutes on a topic of particular interest to them, then we will open the discussion to the floor for the remaining time. 12th international symposium on the analytic hierarchy process 2013 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 119 rozann saaty chairperson rozann is vice president of the creative decisions foundation. she developed the superdecisions software for the ahp/anp, manages rws publications, which publishes books on the ahp by thomas l. saaty and others, and is the managing editor of the ijahp online journal. she is the wife of thomas l. saaty. enrique mu dr. enrique mu is the co-founder and editor-in-chief of the international journal of the analytic hierarchy process, an online journal which focuses on the promotion and dissemination of ahp/anp knowledge. he will share his experience as editor from the past five years. he is also the winner of the best paper award at this conference, and he will be presenting it again in a special session just before lunch today from 12:30 to1:00. the panelists 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 120 antonella petrillo antonella has a phd degree in mechanical engineering and is now a researcher at the university of cassino (italy) where she conducts research on multicriteria decision analysis and industrial plants. she is a member of the international and local committee of the 2011 isahp meeting in sorrento, italy, and associate editor of the online ijahp journal which focuses on ahp articles. fabio de felice fabio is a professor at the university of cassino (italy). he is a board member of several international organizations and is responsible for scientific research in an industrial plant. co-chair of the 2011 isahp meeting in sorrento, italy, and member of the editorial board of the ijahp, he has more than 100 published papers to his credit. in addition he is the founder of the ahp academy – the international association for the promotion of multicriteria decision making methods. the panelists 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 121 birsen karpak birsen karpak, ph.d. in business administration, professor of management, youngstown state university, associate editor, ijahp, directed an international asi on multiple criteria decision making and risk analysis. dr. karpak received numerous awards in research and service including the distinguished professorship award in scholarship, two times, and another one in teaching. she published articles on ahp/anp applications mainly on business problems. leandro pecchia leandro pecchia, from university “federico ii” of napoli, in italy, is currently on special assignment as a lecturer of biomedical engineering at the university of nottingham, uk. his research interest is in applying ahp in healthcare and medical decision making, especially eliciting user needs in medicine to help in designing more effective healthcare technologies and medical devices. he was appointed treasurer of the health technology assessment (hta) of the prestigious international federation of medical and biomedical engineering in 2012. the panelists 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 122 luis vargas professor at the joseph m. katz graduate school of business. in 1984 he chaired isahp1991, held in pittsburgh, usa. luis has focused his research on decision theory, practical applications of the analytic hierarchy process (ahp), measurement of resource utilization, group decision making, bayesian networks, and forecasting among other interests. he serves on the editorial board of a number of journals, is senior editor of the ijahp online journal, and has edited several special journal issues on ahp. note – dr. luis vargas panel discussion was verbal and without the use of powerpoint slides. for this reason there are no slides corresponding to his panel participation. the panelists 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 end 123 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 enrique mu editor-in-chief, ijahp editor-in-chief@ijahp.org 12th international symposium on the analytic hierarchy process 2013 panel discussion publishing ahp/anp papers what constitutes a publishable ahp/anp application paper? 124 mailto:editor-in-chief@ijahp.org mailto:editor-in-chief@ijahp.org mailto:editor-in-chief@ijahp.org mailto:editor-in-chief@ijahp.org mailto:editor-in-chief@ijahp.org 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 125 http: www.ijahp.org the journal 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 126 • ideally, an application should address a problem that is: – worldwide relevant (even if the particular case is local) – current – totally or partially unsolved • your literature review should contextualize your research problem to demonstrate the above the ahp/anp application 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 127 • methodology does not need to be original but needs to be: – rigorous (e.g. where do criteria come from? literature?) – participant transparent (e.g. who entered the judgments?) – understandable (i.e. follow reporting standards) the methodology 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 128 • the most important contributions of an ahp/anp application paper are the answers to the following questions: • does the model have external validity? (can it be used – even if only as basisby someone else?) • do conclusions are consistent with the analysis and useful? • have we learned anything new about the topic and its analysis? sometimes there is not a final solution • important – ahp/anp analyses have limitations too! results: the model 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 end 129 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 statistical data on ahp papers 12th international symposium on the analytic hierarchy process 2013 panel discussion publishing ahp/anp papers fabio de felice and antonella petrillo university of cassino italy defelice@unicas.it; a.petrillo@unicas.it 130 mailto:defelice@unicas.it mailto:a.petrillo@unicas.it 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 131 statistical data on ahp papers years 2002-2012 http://www.google.com.tr/url?sa=i&rct=j&q=&source=images&cd=&cad=rja&docid=lqtx1ch6gnxmxm&tbnid=whxm6ofpjxobqm:&ved=0cauqjrw&url=http://blog.unmc.edu/library/2011/12/29/web-of-science-replaced-by-scopus/&ei=kxnhufvggomgtab2oygwdq&bvm=bv.48293060,d.zwu&psig=afqjcneoiq5cuu4u3tthx4x3egfkfxdgjq&ust=1372088988783461 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 132 statistical data on ahp papers source title http://www.google.com.tr/url?sa=i&rct=j&q=&source=images&cd=&cad=rja&docid=lqtx1ch6gnxmxm&tbnid=whxm6ofpjxobqm:&ved=0cauqjrw&url=http://blog.unmc.edu/library/2011/12/29/web-of-science-replaced-by-scopus/&ei=kxnhufvggomgtab2oygwdq&bvm=bv.48293060,d.zwu&psig=afqjcneoiq5cuu4u3tthx4x3egfkfxdgjq&ust=1372088988783461 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 133 statistical data on ahp papers source title publishing editor: thomas wohlbier, ttp usa elsevier springer http://www.google.com.tr/url?sa=i&rct=j&q=&source=images&cd=&cad=rja&docid=lqtx1ch6gnxmxm&tbnid=whxm6ofpjxobqm:&ved=0cauqjrw&url=http://blog.unmc.edu/library/2011/12/29/web-of-science-replaced-by-scopus/&ei=kxnhufvggomgtab2oygwdq&bvm=bv.48293060,d.zwu&psig=afqjcneoiq5cuu4u3tthx4x3egfkfxdgjq&ust=1372088988783461 http://www.sciencedirect.com/science/journal/09574174 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 134 statistical data on ahp papers country http://www.google.com.tr/url?sa=i&rct=j&q=&source=images&cd=&cad=rja&docid=lqtx1ch6gnxmxm&tbnid=whxm6ofpjxobqm:&ved=0cauqjrw&url=http://blog.unmc.edu/library/2011/12/29/web-of-science-replaced-by-scopus/&ei=kxnhufvggomgtab2oygwdq&bvm=bv.48293060,d.zwu&psig=afqjcneoiq5cuu4u3tthx4x3egfkfxdgjq&ust=1372088988783461 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 135 statistical data on ahp papers document type http://www.google.com.tr/url?sa=i&rct=j&q=&source=images&cd=&cad=rja&docid=lqtx1ch6gnxmxm&tbnid=whxm6ofpjxobqm:&ved=0cauqjrw&url=http://blog.unmc.edu/library/2011/12/29/web-of-science-replaced-by-scopus/&ei=kxnhufvggomgtab2oygwdq&bvm=bv.48293060,d.zwu&psig=afqjcneoiq5cuu4u3tthx4x3egfkfxdgjq&ust=1372088988783461 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 136 statistical data on ahp papers subject area http://www.google.com.tr/url?sa=i&rct=j&q=&source=images&cd=&cad=rja&docid=lqtx1ch6gnxmxm&tbnid=whxm6ofpjxobqm:&ved=0cauqjrw&url=http://blog.unmc.edu/library/2011/12/29/web-of-science-replaced-by-scopus/&ei=kxnhufvggomgtab2oygwdq&bvm=bv.48293060,d.zwu&psig=afqjcneoiq5cuu4u3tthx4x3egfkfxdgjq&ust=1372088988783461 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 137 ahp academy www.ahpacademy.com 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 end 138 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 journals that publish ahp/anp papers 12th international symposium on the analytic hierarchy process 2013 panel discussion publishing ahp/anp papers birsen karpak youngstown state university, ohio usa bkarpak@ysu.edu 139 mailto:bkarpak@ysu.edu 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 • don’t format the manuscript according to the journal standards for your 1st submission. • don’t add the name of your supervisor to the paper if s/he has not substantively contributed (in some countries it might not be possible though) . • don’t complain about the referee reports, if your paper gets rejected. • don’t throw the paper away if it gets trashed by the referees. there is a suitable journal for most of the papers. 140 general comments 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 • don’t ignore the referee reports if your paper gets rejected. • don’t do everything the referees ask for if your paper gets an r&r. sometimes you might get conflicting comments. • don’t forget to write a detailed ‘letter to the editor’ to explain your revisions and justify your decision not to do what the referees want if you get an r&r. don’t forget to address every point. 141 general comments 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 • be the reviewer for journals that publish ahp/anp papers • be a good one. journals grade the reviewers. • ask the editor if s/he can share with you the final decision and the other reviewers’ comments.(you can judge if you have done a good job and learn from other reviewers) 142 general comments 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 • don’t take anything exactly from existing published article even if it is of your own (you can quote) • never submit the same article for another journal when it has already been published in or accepted by a first journal even if you make some improvements. i know at least one paper that was rejected because a very similar article was already published in another journal. (it is unethical, in the us you may be fired from your university. now it is very easy to catch.) 143 general comments 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 • european journal of operational research (ejor) • international journal of production economics (ijpe) • international journal of production research (ijpr) • decision support systems • expert systems with applications • omega • transportation 144 journals that publish ahp/anp papers 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 • journal of purchasing and supply management • journal of supply chain management • journal of cleaner production • computers in industry • computers& operations research • computers & industrial engineering • transportation research part e: logistic and transportation 145 journals that publish ahp/anp papers 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 • journal of multi-criteria decision analysis (wiley) • mathematical and computer modeling • information sciences (elsevier) • applied mathematical modeling (elsevier) (quite probably it disappeared, just learned from luis vargas) • safety science (elsevier) 146 journals that publish ahp/anp papers 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 • benchmarking: an international journal – indexed in • emerald management reviews • cabell's dictionary of publishing opportunities in management and marketing • inspec • scopus 147 journals that publish ahp/anp papers 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 • journal of manufacturing technology management – is indexed and abstracted in: • abi/inform • cabell's dictionary of publishing opportunities in management and marketing • compendex • emerald management reviews • ergonomic abstracts • inspec • materials business file • metadex • scopus 148 journals that publish ahp/anp papers 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 • journal of systems science and systems engineering (by springer verlag) • international journal of management and decision making (inderscience) • international journal of productivity and quality management(inderscience) • international journal of multicriteria decision making(inderscience) 149 journals that publish ahp/anp papers 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 • international journal of services sciences (inderscience) • international journal of operational research (inderscience) • international journal of services technology and management (inderscience) • international journal of services and operations management (inderscience) 150 journals that publish ahp/anp papers 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 • int. j. logistics systems and management (inderscience) • int. j. of information systems and supply chain management – ijisscm (an official publication of the information resources management association) • journal of applied business research • journal of aeronautics and space technologies 151 journals that publish ahp/anp papers 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 • management decision – is indexed and abstracted in: • abi inform • business periodicals index • business source elite • business source premier • cabell's directory of publishing opportunities in management & marketing • emerald management reviews • scopus • social science citation index (isi) • and others 153 journals that publish ahp/anp papers 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 • international journal of business & management • forest ecology and management • journal of quality in maintenance engineering • south asian journal of management – and quite probably some others 154 journals that publish ahp/anp papers 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 • ahp/anp, being a decision theory, has been applied in many fields, and those fields often have special journals that are receptive to ahp/anp decision making papers. • introduce the methodology to these audience. be very clear. ahp/anp explanation depends on the journal you are publishing 155 journals that publish ahp/anp papers 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 • don’t forget ijahp! • read ijahp, cite it 156 thoughts for ahp/anp applied papers 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 • – you can still publish anp papers with an illustrative validating example. • find a virgin area that anp not yet applied (sustainable supply chains is one of them. there are some applications but not many) • for ahp papers you need real decision makers • do very good literature search in the area you are planning to apply ahp/anp 157 thoughts for ahp/anp applied papers 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 • publishing articles is like playing tennis: – you don’t learn it by reading a book or attending a course. you need to practice • academic journals reject almost all papers. in the leading journals only about 10 percent of the submitted manuscripts get accepted. • editors reject an increasing number of submissions without sending them out for review. 158 general comments 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 • don’t be afraid of rejection letters. • don’t start too low on the journal ladder. • don’t submit the manuscript to more than one journal at a time. • don’t submit unfinished / unpolished manuscripts to get comments from the reviewers. 159 general comments 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 • make the english as good as you can. spell-check and grammar-check the paper with the tools word for windows provides (most ahp/anp papers are written in word, though some journals want latex) • submit the paper as the journal asks you to. sometimes journals want the names of the authors stripped from the paper and the meta-data that is hidden in the paper. make sure it is hidden. sometimes it is very easy to guess the author though s/he has stripped (but not properly) from the paper. 160 general comments 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 • instead of saying “there is no article in this area” say “to the best knowledge of the authors, there is no article in this area • give theoretical base the criteria you are using. just saying that “experts selected this criteria is not sufficient” • we have the advantage of considering the number of factors and interrelationships among these factors that most of the other models cannot explicitly addresses. 161 thoughts for ahp/anp applied papers 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 • give as much detail as possible about the actual problem – who are the decision makers – what are the criteria, sub-criteria – who were the parties creating the structure? • ahp/anp papers should always be specific about who the decision is for. who will implement the outcome? was it implemented or likely to be implemented (if not implemented why? it is much easier to publish an article if the study is implemented we can learn from our mistakes too.) 162 thoughts for ahp/anp applied papers 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 • who gave the judgments, how many were involved, and how was it done. questionnaire? group meeting? • what did you learn from this study? • what are the strengths of the research? • what are possible improvements? • what can be done in future? 163 thoughts for ahp/anp applied papers 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 • the application of the ahp/anp approach should not only be concerned with a ‘final’ solution to the problem, but it also should be applied as a learning tool for decision makers and managers to help understand the various linkages among the various components, clusters, and elements. 164 thoughts for ahp/anp applied papers 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 • provide three names of possible referees for your paper (include their educational institution or affiliation). they should be people familiar with the field in which you applied ahp. if the journal asks so. • be prompt in responding to correspondence from the journal and in getting your paper back with the editors' suggested corrections or improvements. • it is better to give institutional e-mail address. 165 general comments 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 end 166 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 publishing ahp/anp papers articles describing application of ahp in healthcare 12th international symposium on the analytic hierarchy process 2013 panel discussion publishing ahp/anp papers leandro pecchia university of nottingham (uk) leandro.pecchia@nottingham.ac.uk 167 mailto:leandro.pecchia@nottingham.ac.uk 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 168 1. there is a growing number of study focusing on application of ahp in healthcare 2. these applications span from: • medical decision making • health technology assessment • identification of the best treatment for a patient or a specific population of • user need elicitation and human centered design • selection of health technologies • identify the best candidate for a treatment (i.e. candidate for organ donation) 3. this is a very promising and challenging field, but few considerations have to be done before writing a paper 4. this is mainly because healthcare researches present some peculiarities: • the final user of a technology is not a technologist • the main stakeholder is not expert of the field • safety and ethical issues are paramount important 5. thus, it is useful to discuss some issues regarding paper on ahp in healthcare scenario 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 169 introduction this is the section that needs to be better structured • focus only on the specific problem you are trying to solve • support your goal with references (why you did your study?) • present in few words the state of the art (…put all the references here!!!!!) • clarify clearly what your study added/is adding/is going to add/will add • state clearly the goal of this paper (which may be different from all the study) method aim ≠ method = aim ≠ method ≠ aim = method ≠ aim = method = aim results aim ≠ results = aim ≠ results ≠ aim = results ≠ aim = results = aim 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 170 1. describe in details only your relevant contributions: • was ahp used in a peculiar way? • was it a new target of responders? • … 2. spend less words on other methodological bits, remanding the readers to the references for the well established parts 3. i would suggest not to describe the basic method of ahp, unless your paper is the first one describing ahp for the journal your are submitting your work. 4. please minimize the numbers in this section… 5. try to write this section in an understandable way for readers not familiar with ahp (or with other mathematical staff), but do not repeat well known thinks 6. some references in this section are normally due (i.e. saaty fundamental scale) 7. critical data pooling: readers are use to see a mean value and a variace… 8. in healthcare there are some methodological requirements: • minimum sample size estimation (this is essential to start the study) • generality of your results (given with statistical tests and p-value) • mention your ethical approval • opinion study are the base of pyramid of evidence… method 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 171 1. put here all the numbers, tables, diagrams 2. report the results achieved in your study 3. report only the results achieved in your study 4. use graphs and plots to describe the main results (do not repeat all, just the main) 5. no new references here: cite all the studies you need in the introduction… 6. discuss your results against existing literature: • “differently from dr x, we proved that…” • “we confirm the results of dr x with an independent method…” • this may happen. it is the frequent case of a secondary outcome… 7. if there is no previous studies to compare your work… search better and better! results aim ≠ results = aim ≠ results ≠ aim = results ≠ aim = results = aim 1 2 3 results and dicussion 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 172 1. describe in few lines the contribution of your paper 2. describe in few words the learned lesson 3. try to describe the impact of your work 4. highlight possible future directions 5. state clearly the limits of your study conclusion 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 p.s.: please do not blame me, this is only the lesson i learned by writing 28 papers in 2012/13. i would have published much better if somebody had told me these few thinks… 12th international symposium on the analytic hierarchy process 2013 thank you for your time! leandro pecchia university of nottingham (uk) leandro.pecchia@nottingham.ac.uk l.pecchia@warwick.ac.uk (from next september) 173 mailto:leandro.pecchia@nottingham.ac.uk mailto:l.pecchia@warwick.ac.uk 23-26 june 2013, hotel istana, kuala lumpur 23-26 june 2013, hotel istana, kuala lumpur ijahp vol.5 issue 1 2013 issn 1936-6744 174 panel end ijahp article: yom din/do industry and academic experts differ in weighting criteria for agricultural planning and in their loss aversion? international journal of the analytic hierarchy process 161 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.355 do industry and academic experts differ in weighting criteria for agricultural planning and in their loss aversion? gregory yom din 1 department of management and economics the open university of israel raanana, israel e-mail: gregory@openu.ac.il; faculty of exact sciences tel-aviv university tel-aviv, israel abstract agricultural planning models account for changing weather conditions and for high food price volatility. applying the criteria of maximum farmer’s profit in various years enables the use of multi-criteria techniques. this paper presents the application of the analytical hierarchy process (ahp) to estimate the importance of these criteria. the study contributes to the literature by a) addressing expert opinions on the importance of the criteria of profit in normal, dry years, and in years when agricultural prices rise; b) examining differences in weighting criteria between industry and academic experts; c) taking into account the loss averse behavior for these two groups of experts. industry and academic experts were interviewed in the agricultural region of north-eastern israel. changes in profit are approximated by a linear utility function with a positive slope (for both losses and gains) which is steeper for losses (profit in dry years) than for gains (profit in years when prices rise). the ahp method allows for the identification of the importance of the criteria for all respondents as a whole and for academic experts as compared to industrial experts. the share of loss averse respondents and coefficients of loss aversion are identified for both groups of experts. these coefficients are used for explaining differences between industry and academic expert opinions in estimating the criteria importance. keywords: ahp; farmers profit; dry year; prices rise; consistency; loss averse 1. introduction many agricultural planning models, including those for optimal crop selection, production and distribution, and water resource allocation, pertain to various aspects of weather, price and other uncertainties. the models account for marketing and agronomic risks, for changing weather conditions in normal and dry years, and for the volatility of 1 the author gratefully acknowledges dr. zinaida zugman and anna schwartz for their professional advice and insight in the preparation of this article. mailto:gregory@openu.ac.il ijahp article: yom din/do industry and academic experts differ in weighting criteria for agricultural planning and in their loss aversion? international journal of the analytic hierarchy process 162 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.355 food prices that have remained high since the 1970s. these factors greatly affect the profit variability in agriculture. in particular, their management requires selection of crops which allow for a diversified income for farmers and mitigation of the risks (lehmann et al., 2013; rădulescu et al., 2014; tadesse et al., 2014). as for changing weather conditions, although cropland per person in the world in the last 50 years decreased twice from 0.44 to 0.22 hectare/person, the increase in total cultivated land accounted for 12%, and in irrigated land for 117% (fao, 2011). as a result, 70% of the global water supply (part of renewable freshwater resources removed from rivers or aquifers by installing infrastructure) is currently used by agriculture. extreme events, such as heatwaves and droughts, will become more frequent in the coming years, negatively impacting crop yields (fao, 2010). climate models project increased aridity during the current century, in particular, in the middle east (thornton et al., 2014). for this region the maximal climate change impact on agricultural production was estimated for a potential decrease in precipitation. a 20% decrease in precipitation was projected to affect a 38% drop in crop yields (calzadilla et al., 2013). regarding food price volatility, some researchers suggest that liberalizing agricultural trade maintains high prices and mitigates price volatility in the near future (rutten et al., 2013). other researchers have discovered that food prices have become even more volatile in recent years. this phenomenon is attributed to the use of food commodities for production of biofuel, with the biofuels link introducing an additional source of demand variability and ensuing food price volatility (gilbert & mugera, 2014). the output prices combined with the operational and scale efficiency continue to be major factors in explaining farm profitability (tey & brindal, 2015). the analysis of ten global economic models that assess plausible futures for agricultural markets revealed contradictory outcomes. three of the ten reviewed models projected falling real prices, six – increasing, and one – a stable level of prices, while the volatility in general trends remains important (lampe et al., 2014). in agricultural planning models, the best solutions are often selected based on a single economic criterion of maximum farmer profit (dury et al., 2012). the application of the criteria of maximum profit in various years with various weather conditions and price level movements – allows for the use of additional information on weather and price uncertainties and allows for the use of multi-criteria techniques. for this purpose, the weight (importance) attached to various agricultural criteria can be calculated from the collected expert opinions (arriaza & gómez-limón, 2003; bjørndal et al., 2012). the purpose of this study is twofold. firstly, it aims to examine expert opinions on the importance of profit criteria in various years – normal years, dry years, and years of agricultural price rise, and the differences in criteria weighting between industry and academic experts. secondly, the study examines loss averse behavior for these two groups of experts. the study contributes to the literature by a) addressing expert opinions on the importance of the criteria of profit in normal years, dry years, and in years of agricultural price rise; b) examining differences in weighting criteria between industry and academic experts; and c) taking into account the loss averse behavior for these two groups of experts. ijahp article: yom din/do industry and academic experts differ in weighting criteria for agricultural planning and in their loss aversion? international journal of the analytic hierarchy process 163 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.355 in the present study, the expert opinions on the criteria of maximum profit in a normal year, in a dry year, and in a year of agricultural price rise were collected. the data from questionnaires answered by industry and academic experts were processed through the analytic hierarchy process (ahp) method (saaty 1977, 1980). this method is widely used for calculating the weights of the criteria as well as the differences between weights given by experts from various groups, and among others, in agricultural applications (pelizaro, 2011; toledo, 2011). it is assumed in the current study that in agricultural planning models, profit changes can be approximated by a linear utility function with a positive slope (for both losses and gains) which is steeper for losses (in dry years) than for gains (in price rise years, in this study) (loewenstein, 1988). this assumption is based on the predictions of the prospect theory regarding the shape of the value function (tversky & kahneman, 1992). examination of the farmer’s preferences – whether they are loss averse, gain seeking or neutral revealed that the prospect theory was relevant in describing the farmer’s behavior in risk conditions. in many published studies, most of the surveyed farmers were identified as loss averse (coelho et al., 2012). however, further examination of farmer’s loss aversion remains a research topic since this phenomenon can vary across countries and over time. economic crises and chronic losses can further influence a farmer’s loss aversion (rivers & arvai, 2007; ashta & otto, 2011). additionally, it was revealed that the perception of losses and gains can differ for various groups of experts – novice vs. professionals, industry vs. academics, and experts of different ages and levels of education (isin & miran, 2005). 2. materials and methods 2.1. data a sample of 32 experts was surveyed during the study of the agricultural development conducted in north-eastern israel from 2012-14. the surveyed experts were divided into two groups, industry and academic experts. average work experience of the industry experts was 56% longer in duration than that of the academic experts. women comprised 22% of the respondents and the duration of their work experience was shorter in both groups of experts (table 1). table 1 characteristics of experts categories number of respondents and average work experience in agriculture, years men women total number work experience number work experience number work experience industry 13 30 2 16 15 29 academic 12 20 5 15 17 18 total experts 25 25 7 15 32 23 2.2. estimating importance rates of the criteria in-person interviews were conducted at the respondent’s workplaces. the interviews ijahp article: yom din/do industry and academic experts differ in weighting criteria for agricultural planning and in their loss aversion? international journal of the analytic hierarchy process 164 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.355 began with a short discussion of the study’s background – the concept of optimal agricultural planning, water allocation, possible changes in farmer’s profit in dry years and in years of agricultural price rise. in the surveyed agricultural region, all crops are irrigated, but in dry years the water quota for irrigation can be substantially reduced. changes in the share of annual (mainly, vegetables) and perennial (fruit orchards, vineyards, flower greenhouses) crops, temperate (apples) and subtropical (mango, avocado) cultures in the crop structure can result in different profit losses in dry years, and in different gains in years of agricultural price rise. then the experts were asked to compare three pairs of agricultural planning criteria, as given below, and rate each pair as “much less important”, “less important”, “the same importance”, “more important”, and “much more important”. the three pairs were (appendix 1): 1) the importance of “normal year profit” vs. “dry year profit”; 2) the importance of “normal year profit” vs. “profit in an agricultural price rise year”; 3) the importance of “dry year profit” vs. “profit in an agricultural price rise year”; these pairwise comparisons make it possible to complete a saaty’s reciprocal matrix a for every expert:            1/1/1 1/1 1 2313 2312 1312 aa aa aa a (1) where 12a denotes the importance rate of “normal year profit” compared to “dry year profit”, 13a denotes the importance rate of “normal year profit” compared to “profit in an agricultural price rise year”, 23a denotes the importance rate of “profit in a dry year” compared to “profit in an agricultural price rise year”. saaty suggested the scale for importance rates as follows: “much less important” corresponds to 1/9, “less important” to 1/5, “the same importance” to 1, “more important” to 5, and “much more important” to 9 (saaty, 1977). in this study we also used the balanced which was defined as follows: “much less important” 1/2.33, “less important” 1/1.5, “the same importance” 1, “more important” 1.5, and “much more important” 2.33 (scale (salo & hämäläinen, 1997; brunelli, 2014). the consistency index (ci) suggested by saaty for measuring consistency of expert’s judgements is defined as follows:    1/max  nnci  (2) where n is the number of criteria ( 3n in this study), m ax is the principal eigenvalue of the matrix a for a given expert. ijahp article: yom din/do industry and academic experts differ in weighting criteria for agricultural planning and in their loss aversion? international journal of the analytic hierarchy process 165 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.355 saaty suggested comparing the value of ci with the same index averaged over a large number of reciprocal matrices of the same order with random matrix entries. for 3n , this averaged index equals 0.58. he recommended using data of those experts for whom the “consistency ratio” of ci to the value 0.58 was not greater than 10%, and for other experts to attempt to improve the consistency (saaty, 1990). as the alternative test for consistency, the elements of the matrix a (calculated now using the above described balanced scale) were used for calculating the "consistency measure" using the same formula (equation 2). this measure can change from 0 to 1, and its value increases with inconsistencies in the answers of the expert. there are no specific recommendations on the maximum upper limit of this indicator. researchers recommend the use of the consistency measure as an indicator of those experts whose responses can be inconsistent (mustajoki & hamalainen, 2000). in the current study, both consistency ratio and consistency measure were used to test the consistency of the expert answers. the importance rates ija from equation 1 were averaged over all experts and also separately over groups of industry and academic experts to compare the importance, or weights, of all three criteria used in the study. individual judgements were aggregated into a single representative judgement for the entire group or for all experts using their geometric mean. the latter was calculated for every importance rate from the matrix (equation 1). the geometric mean of judgments is the mathematical equivalent of consensus assuming all the experts to be equal in weighting their judgements (saaty, 2008; saaty & vargas, 2007). to estimate the significance of the difference in the calculated importance rates between industry and academic experts, the standard pooled-variance t test for differences in two means was used. it was assumed that the two groups of experts were drawn from the underlying normal populations of industry and academic experts that have the same variance. 2.3. estimating loss aversion the following widely used definition of the loss aversion, which is a concept of the prospect theory, has been employed: )()( xuxu  (3) where u is the utility function for losses (decrease in dry years profit) and gains (increase in profit in price rise years). the inequality equation 3 expresses is one of the basic phenomena of choice under uncertainty that losses loom larger than gains (tversky & kahneman, 1992). the surveyed experts weighted criteria under uncertainty when outcomes were known (one criterion is less important that the other, much less important appendix 1), but not necessarily their probabilities (the experts were not informed about the statistics shown in appendix 2). it was assumed for the questionnaire used in our study that: the additional profit in a normal year equals 0 – the reference point; x is the additional profit (gain) in a price rise year; ijahp article: yom din/do industry and academic experts differ in weighting criteria for agricultural planning and in their loss aversion? international journal of the analytic hierarchy process 166 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.355 -x is the profit received less (loss) in a dry year. it was assumed that %20x and consequently %20 x for the region studied. the assumption was based on the regional crop calculations approved by the israel ministry of agriculture and rural development (received upon request from the kiryat shmona office of the ministry). the following linear utility function u for losses and gains was assumed: 0,)( 1  axapxu for losses, (4) 0,)( 2  bxbpxu for gains. (5) where 1p is the probability of a dry year, 2p is the probability of a price rise year, a and b are the sought coefficients of the utility function. based on the collected 14-year historical data, it was assumed 14/21 p (appendix 2, fig. 1a) and 14/32 p (appendix 2, fig. 2a). for every expert, the importance rate which is assigned to dry year profit is used as the value of the utility function )( xu  ( %20 x ) for losses, and the importance rate assigned to profit in a price rise year is used as the value of the utility function )(xu ( %20x ) for gains. based on the data presented in the section 2.2 matrix a: 12a is interpreted as the importance rate of “profit in a normal year” compared to “profit in a dry year”, or the loss ; 13a as the importance rate of “profit in a normal year” compared to “profit in an agricultural price rise year”, or the gain )(xu . this interpretation, and the above assumptions for 21,, ppx and equations 4 and 5 for the utility function u enable the calculation of the coefficients a and b as follows:   %9.2%2014/2)( 12121121 aaxpaxpxua  ,   %3.4%2014/3)( 13132132 aaxpaxpxub  . measuring loss aversion using the above derived coefficients a and b is important as it can explain a variety of field data, in particular, differences between opinions of agricultural and industrial experts. in this study, the expert for whom equation 3 is valid is classified as loss averse; if the left side of equation 3 equals its right side, the expert is ijahp article: yom din/do industry and academic experts differ in weighting criteria for agricultural planning and in their loss aversion? international journal of the analytic hierarchy process 167 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.355 classified as loss neutral; and if the left side of equation 3 is less than its right side, the expert is classified as gain seeking. the individual loss aversion coefficients are calculated as the ratio )()( xuxu  (6) they demonstrate to what degree risk preferences are exhibited more strongly for losses than for gains (kahneman & tversky, 1979; abdellaoui et al., 2007). 2.4. hypotheses of the study hypothesis 1: the author hypothesizes that the significance of the differences between the average importance rates for various groups of experts (industry affiliated and academic) can be at least partially due to the expert’s different expertise. this is based on the revealed different opinions of industry against academic experts reported in recent empirical studies of multi-criteria decision making methods. the following examples illustrate this point: ahp applied to selecting sires in dairy industry, multiple expert elicitation for studying technological change under uncertainty, and a delphi survey of decision-making in engineering (stokes &tozer, 2002; rai, 2013; shishank & dekkers, 2013). hypothesis 2: the author also speculates that the average importance rate for the criteria pair “normal year profit” and “dry year profit” (losses) will be higher than for the pair “normal year profit” and “profit in an agricultural price rise year” (gains) due to the loss aversion effect. 3. results the consistency ratio was calculated for all 32 experts. for 29 of them this ratio was less than 10% and for the other three experts the ratio was slightly larger, 14-15%. the average consistency ratio was low and equaled 3.3%. low values of the consistency measure (much closer to zero that to 1) also indicated consistency of the expert’s answers. because of the good results of the consistency testing the data of all experts were used for the following analysis. the importance rates (weights) of the three studied maximum profit criteria for different years were calculated. the criterion “normal year profit” had the maximum weight – almost half of the overall importance of the compared criteria which is assumed to be 100% followed by “profit in a price rise year”. the criterion “dry year” had the least importance (table 2). ijahp article: yom din/do industry and academic experts differ in weighting criteria for agricultural planning and in their loss aversion? international journal of the analytic hierarchy process 168 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.355 table 2 importance rates (weights) of the compared criteria compared criteria importance rate (weight) normal year profit 49% dry year profit 22% profit in a price rise year 29% the differences in the importance rates between industry and academic experts were examined. industry experts attached more importance to the criterion “normal year profit” than academic experts, and the difference between these groups of experts was significant at the 5% level. for two other criteria “dry year profit” and “profit in price rise years”, the differences between the groups of the experts were non-significant. this confirms hypothesis 1 that the differences between the importance rates of criteria for different groups of experts, or at least part of them, can be significant (table 3). table 3 importance rates separately for industry and academic experts criterion importance rate t industry experts academic experts difference in % points statistic normal year profit 58% 41% 17% 2.418 dry year profit 19% 25% -6% -0.942 profit in a price rise year 23% 34% -11% -1.603 we now examine hypothesis 2 of the possible loss aversion of the interviewed experts. for both groups, most of the experts were identified as loss averse persons – 87% of industry, 82% of academic experts, and 84% of all experts. but the averages of the coefficients of loss aversion, as calculated by equation 6, were 5% significantly higher for academic experts (table 4). table 4 loss averse / gain seeking experts from different groups of experts number of experts coefficient of loss aversion loss averse gain seeking loss neutral mean std. error industry 13 2 0 1.67 0.13 academic 14 2 1 2.67 0.37 total experts 27 4 1 2.20 0.22 ijahp article: yom din/do industry and academic experts differ in weighting criteria for agricultural planning and in their loss aversion? international journal of the analytic hierarchy process 169 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.355 4. discussion and conclusion the current study identified differences between importance rates of the maximum profit criteria that the surveyed experts selected for use in agricultural planning models. the impact of climate and price factors was taken into account – expert judgements on the criterion of normal year profit were compared to those on dry year profit and profit in agricultural price rise years. using the ahp method, maximum importance was assigned to the criterion normal year profit. the novelty of the suggested approach lies in a) the set of compared criteria that included dry year profit and profit in years of agricultural price rise, and b) the estimate of loss aversion of the interviewed industry and academic experts using their opinions collected and analyzed with the ahp method. the results obtained for industry experts differed from those received for academic experts. industry experts attached more importance than their academic colleagues to the criterion “normal year profit” – 58% and 41%, respectively, and this difference was significant at the 5% level. these results are consistent with those of other studies, where the ahp method gave different results for various groups of experts from the same field. sengar et al. (2014) used the ahp method to rank the major barriers to rural markets in india. the researchers found that industry and academic experts ranked these barriers differently, although both groups of experts viewed organizational barriers as the most important barrier category. stokes and tozer (2002) used the multi-criteria model for the selection of sires in the dairy breeding program. for this purpose, they ranked and weighted several breeding and economic criteria using the ahp method and responses of industry and agricultural experts. some of the criteria were ranked differently by the two groups of experts which led to different results in modelling optimal sire selection. similar results were obtained in assessment of alternative farming activities using the ahp method in the study of chavez et al. (2012). the authors compared criteria such as income, market feasibility and other criteria that received different weights for various groups of experts farmers, researchers, farm advisers, and cooperative and government representatives. the attitude of the experts towards dry year profit (losses) and profit in price rise years (gains) was used in the current study for identifying the experts as loss averse / gain seeking persons and for estimating their loss aversion coefficients. the results are compatible with the findings of other studies where experts in agricultural problems / farmers were identified as loss averse individuals. bocquého et al. (2014) studied a sample of french farmers and found that the farmers were twice as sensitive to losses as to gains. this is compatible with the coefficients of loss aversion in the present study estimated to be 2.20 on average for all experts, and 1.67 and 2.67 for industry and academic experts, respectively (table 4). measuring loss aversion can explain differences in expert opinions and farmer’s decisions. in the current study, the criteria of dry year profit and profit in price rise years are more important for academic experts who are more loss averse than for industrial ijahp article: yom din/do industry and academic experts differ in weighting criteria for agricultural planning and in their loss aversion? international journal of the analytic hierarchy process 170 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.355 experts (consequently, the opposite is true for the criterion profit in normal year profit). a possible explanation for this difference is that academic experts display more knowledge and interest in optimization of crop structure by favoring the crops which allow for diminishing losses in dry years or increasing gains in price rise years. indirect support for this assertion can be found in the study of cotton farmers in china, which concluded that if farmers viewed new varieties of cotton as effective in eliminating pests (as was advertised by scientists), then more loss-averse farmers adopted these varieties sooner (liu, 2013). the present study arrives at the following conclusions: a) the criterion normal year profit is more important than other studied criteria of dry year profit and profit in a price rise year, according to the expert’s opinion. b) the criterion normal year profit is less important for academic than for industrial experts. c) almost the same portion of both groups of experts, 82-87%, was identified as loss averse persons. however, the coefficient of loss aversion was significantly higher for academic experts. ijahp article: yom din/do industry and academic experts differ in weighting criteria for agricultural planning and in their loss aversion? international journal of the analytic hierarchy process 171 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.355 references abdellaoui, m., bleichrodt, h., & paraschiv c. (2007). loss aversion under prospect theory: a parameter-free measurement. management science, 53(10), 1659-1674. doi: http://dx.doi.org/10.1287/mnsc.1070.0711 arriaza, m. & gómez-limón, j.a. (2003). comparative performance of selected mathematical programming models. agricultural systems, 77(2), 155-171. doi:10.1016/s0308-521x(02)00107-5 ashta, a. and otto, p.e. (2011). project valuation in the presence of loss aversion during economic crises. strategic change, 20(5‐6), 171-186. doi: 10.1002/jsc.894 bjørndal, t., herrero, i., newman, a., romero, c., & weintraub, a. (2012). operations research in the natural resource industry. international transactions in operational research, 19(1-2), 39-62. bocquého, g., jacquet, f., reynaud, a. (2014). expected utility or prospect theory maximisers? assessing farmers' risk behaviour from field-experiment data. european review of agricultural economics, 41(1), 135-172. doi: 10.1093/erae/jbt006 brunelli, m. (2015). introduction to the analytic hierarchy process. switzerland: springer international publishing. doi: 10.1007/978-3-319-12502-2. calzadilla, a., rehdanz, k., betts, r., falloon, p., wiltshire, a., & tol, r. s. (2013). climate change impacts on global agriculture. climatic change, 120(1-2), 357-374. doi:10.1007/s10584-013-0822-4 central bureau of statistics, israel. (2015). database of price indices by subject. http://www.cbs.gov.il/reader/?mival=%2fprices_db%2fpricesdb_secondselect_e.html &radio1=1_3 chavez, m.d., berentsen, p.b.m., & oude lansink, a.g.j.m. (2012). assessment of criteria and farming activities for tobacco diversification using the analytical hierarchical process (ahp) technique. agricultural systems, 111, 53-62. doi:10.1016/j.agsy.2012.05.006 coelho, l.a. g., pires, c.m.p., dionísio, a.t., & serrão, a.j.d. c. (2012). the impact of cap policy in farmer's behavior – a modeling approach using the cumulative prospect theory. journal of policy modeling, 34(1), 81-98. doi:10.1016/j.jpolmod.2011.03.009 dury, j., schaller, n., garcia, f., reynaud, a., & bergez j. e. (2012). models to support cropping plan and crop rotation decisions. a review. agronomy for sustainable development, 32(2), 567-580. doi: 10.1007/s13593-011-0037-x fao. (2010). aquastat database. available at www.fao.org/nr/water/aquastat/main/index.stm http://dx.doi.org/10.1287/mnsc.1070.0711 http://dx.doi.org/10.1016/s0308-521x(02)00107-5 http://www.cbs.gov.il/reader/?mival=%2fprices_db%2fpricesdb_secondselect_e.html&radio1=1_3 http://www.cbs.gov.il/reader/?mival=%2fprices_db%2fpricesdb_secondselect_e.html&radio1=1_3 http://dx.doi.org/10.1016/j.agsy.2012.05.006 http://dx.doi.org/10.1016/j.jpolmod.2011.03.009 http://www.fao.org/nr/water/aquastat/main/index.stm ijahp article: yom din/do industry and academic experts differ in weighting criteria for agricultural planning and in their loss aversion? international journal of the analytic hierarchy process 172 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.355 fao. (2011). the state of the world’s land and water resources for food and agriculture (solaw) – managing systems at risk. food and agriculture organization of the united nations, rome and earthscan, london. gilbert, c.l., & mugera, h. k. (2014). food commodity prices volatility: the role of biofuels. natural resources 2014. doi: http://dx.doi.org/10.4236/nr.2014.55019 isin, s. & miran, b. (2005). farmers' attitudes toward crop planning in turkey. journal of applied sciences, 5, 1489-1495. doi: 10.32923/jas.2005.1489.1495 kahneman, d. & tversky a. (1979). prospect theory: an analysis of decision under risk. econometrica: journal of the econometric society, 47(2), 263-291. doi: http://www.jstor.org/stable/1914185 von lampe, m., willenbockel, d., ahammad, h., blanc, e., cai, y., calvin, k., fujimori, s., hasegawa, t., havlik, p., heyhoe, e., kyle, p., lotze-campen, h., mason d'croz, d., nelson, g.c., sands, r.d., schmitz, c., tabeau, a., valin, h., van der mensbrugghe, d., & van meijl h. (2014). why do global long-term scenarios for agriculture differ? an overview of the agmip global economic model intercomparison. agricultural economics, 45, 3–20. doi: 10.1111/agec.12086 lehmann, n., briner, s., & finger r. (2013). the impact of climate and price risks on agricultural land use and crop management decisions. land use policy, 35, 119-130. doi:10.1016/j.landusepol.2013.05.008 liu, e.m. (2013). time to change what to sow: risk preferences and technology adoption decisions of cotton farmers in china. review of economics and statistics, 95(4), 13861403. doi:10.1162/rest_a_00295 loewenstein, g.f. (1988). frames of mind in intertemporal choice. management science, 34(2), 200-214. doi: http://dx.doi.org/10.1287/mnsc.34.2.200 mustajoki, j., & hamalainen, r.p. (2000). web-hipre: global decision support by value tree and ahp analysis. infor j, 38(3), 208-220. pelizaro, c., benke, k., & sposito, v. (2011). a modelling framework for optimisation of commodity production by minimising the impact of climate change. applied spatial analysis and policy, 4, 201-222. doi: 10.1007/s12061-010-9051-7 rădulescu, m., rădulescu, c.z., & zbăganu, g. (2014). a portfolio theory approach to crop planning under environmental constraints. annals of operations research: 1-22. doi:10.1007/s10479-011-0902-7 rai, v. (2013). expert elicitation methods for studying technological change under uncertainty. environmental research letters, 8(4), 041003. doi:10.1088/17489326/8/4/041003 http://dx.doi.org/10.1016/j.landusepol.2013.05.008 http://dx.doi.org/10.1287/mnsc.34.2.200 ijahp article: yom din/do industry and academic experts differ in weighting criteria for agricultural planning and in their loss aversion? international journal of the analytic hierarchy process 173 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.355 rivers, l. & arvai, j. (2007). win some, lose some: the effect of chronic losses on decision making under risk. journal of risk research, 10(8), 1085-1099. doi:10.1080/13669870701615172 rutten, m., shutes, l., & meijerink, g. (2013). sit down at the ball game: how trade barriers make the world less food secure. food policy, 38, 1-10. doi:10.1016/j.foodpol.2012.09.002 saaty, t.l. (1977). a scaling method for priorities in hierarchical structures. journal of mathematical psychology, 15(3), 234-281. doi:10.1016/0022-2496(77)90033-5 saaty, t. (1980). the analytic hierarchy process. new york : mcgraw-hill. doi:10.1080/00137918308956077 saaty, t.l. (1990). how to make a decision: the analytic hierarchy process. european journal of operational research, 48(1), 9-26. doi:10.1016/0377-2217(90)90057-i saaty, t.l. (2008). decision making with the analytic hierarchy process. international journal of services sciences, 1(1), 83-98. doi: http://dx.doi.org/10.1504/ijssci.2008.017590 saaty, t.l., & vargas, l. g. (2007). dispersion of group judgments. mathematical and computer modelling, 46(7), 918-925. doi:10.1016/j.mcm.2007.03.004 salo, a.a., & hämäläinen, r.p. 1997. on the measurement of preferences in the analytic hierarchy process. journal of multi‐criteria decision analysis, 6(6), 309-319. doi: 10.1002/(sici)1099-1360(199711)6:6<309::aid-mcda163>3.0.co;2-2 sengar a., sharma v., agrawal r., & bharti, k. (2014). prioritisation of barriers to rural markets: integrating fuzzy logic and ahp. international journal of business and emerging markets, 6(4), 371-394. doi: http://dx.doi.org/10.1504/ijbem.2014.065584 shishank, s., dekkers, & r. (2013). outsourcing: decision-making methods and criteria during design and engineering. production planning & control, 24(4-5), 318-336. doi:10.1080/09537287.2011.648544 stokes, j.r., & tozer, p.r. (2002). sire selection with multiple objectives. agricultural systems, 73(2), 147-164. doi:10.1016/s0308-521x(01)00079-8 tadesse, g., algieri, b., kalkuhl, m., & von braun, j. (2014). drivers and triggers of international food price spikes and volatility. food policy, 47, 117-128. doi:10.1016/j.foodpol.2013.08.014 tey, y.s., & brindal, m. (2015). factors influencing farm profitability. in sustainable agriculture reviews, pp. 235-255, switzerland: springer international publishing. doi: 10.1007/978-3-319-09132-7_5 thornton, p.k., ericksen, p.j. herrero, m., & challinor, a.j. (2014). climate variability and vulnerability to climate change: a review. global change biology, 20(11), 33133328. doi: 10.1111/gcb.12581 http://dx.doi.org/10.1016/j.foodpol.2012.09.002 http://dx.doi.org/10.1016/0022-2496(77)90033-5 http://dx.doi.org/10.1016/0377-2217(90)90057-i http://dx.doi.org/10.1016/j.mcm.2007.03.004 http://dx.doi.org/10.1016/s0308-521x(01)00079-8 http://dx.doi.org/10.1016/j.foodpol.2013.08.014 ijahp article: yom din/do industry and academic experts differ in weighting criteria for agricultural planning and in their loss aversion? international journal of the analytic hierarchy process 174 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.355 toledo, r., engler, a., & ahumada, v. (2011). evaluation of risk factors in agriculture: an application of the analytical hierarchical process (ahp) methodology. chilean journal of agricultural research, 71(1), 114-121. http://dx.doi.org/10.4067/s071858392011000100014 tversky, a., & kahneman, d. (1992). advances in prospect theory: cumulative representation of uncertainty. journal of risk and uncertainty, 5(4), 297-323. doi: 10.1007/bf00122574 ijahp article: yom din/do industry and academic experts differ in weighting criteria for agricultural planning and in their loss aversion? international journal of the analytic hierarchy process 175 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.355 appendix i the ahp questionnaire all answers were reported on the 9-point scale as “much more important”, “more important”, “the same importance”, “less important”, and “much less important”. please compare the following criteria for agricultural planning: 1. what is the importance of the criterion “profit in a normal year” compared to “profit in a dry year”? 2. what is the importance of the criterion “profit in a normal year” compared to “profit in a year when agricultural commodity prices rise”? 3. what is the importance of the criterion “profit in a dry year” compared to “profit in a year when agricultural commodity prices rise”? please describe yourself: … ijahp article: yom din/do industry and academic experts differ in weighting criteria for agricultural planning and in their loss aversion? international journal of the analytic hierarchy process 176 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.355 appendix ii annual data of water supply and changes in prices of agricultural commodities in israel figure 1a. water supply to the 32 communities in north-eastern israel, % to the 14 years average: calculated by the author based on data collected in mei golan association. the two years of this period, 2001 and 2002, marked with a circle in the figure, were assumed as dry years because the water supply in these years was less than 80% of the average annual supply. 0% 25% 50% 75% 100% 125% 1995 1997 1999 2001 2003 2005 2007 2009 ijahp article: yom din/do industry and academic experts differ in weighting criteria for agricultural planning and in their loss aversion? international journal of the analytic hierarchy process 177 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.355 figure 2a. changes in price indices of fruit and vegetables as compared to a previous year calculated by the author based on data from central bureau of statistics, israel, 2015. the three years of this period, 1997, 1998, and 2007, marked with a circle in the figure, were assumed as years of price rise (more than 4.5% in this figure). -3.0 -1.5 0.0 1.5 3.0 4.5 6.0 1995 1997 1999 2001 2003 2005 2007 2009 ijahp: samoei, gichoya, odero/an ahp variation modelling approach for performance measurement of suppliers of information communication technology products international journal of the analytic hierarchy process 199 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.487 an ahp variation modelling approach for performance measurement of suppliers of information communication technology products daniel samoei moi university, systems administrator,school of information sciences department of information technology kenya dsamoei@mu.ac.ke david gichoya moi university associate dean, school of information sciences associate professor in the department of information technology kenya dgichoya@yahoo.com damaris odero moi university head and senior lecturer, department of library, records &information studies kenya oderodjn@gmail.com abstract selecting suitable suppliers of ict products is always a challenging task for a procuring entity since it requires the consideration of multiple, competing, tangible and intangible criteria in determining optimal suppliers of a given product. this study investigated the evaluation criteria of suppliers of ict products used by moi university with a view to developing a multicriteria decision analysis model for evaluating the performance of suppliers of ict products. hence, the study was guided by a variation of saaty’s analytical hierarchy process theory of measurement and targeted a population of 55 respondents comprised of 33 ict staff and 22 procurement staff. seventeen companies who bid to supply ict products were targeted. purposive sampling was used to select 7 companies who supply ict products as per their prequalification status and there were 16 respondents; 11 from the procurement unit and 5 from the ict directorate respectively. data was collected using questionnaires and documentary reviews. the study findings demonstrated that quality and transport and communication logistics were the most preferred evaluation criteria and sub criteria respectively, and the respondents had different preferential treatment on suppliers of ict products as per the evaluation sub criterions. it is recommended that the model be adopted to assist the procurement unit in evaluating suppliers of ict products and be customized for use in evaluating suppliers of other products at the university and other public institutions in kenya. mailto:dsamoei@mu.ac.ke mailto:dgichoya@yahoo.com mailto:oderodjn@gmail.com ijahp: samoei, gichoya, odero/an ahp variation modelling approach for performance measurement of suppliers of information communication technology products international journal of the analytic hierarchy process 200 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.487 keywords: ahp; ahp variations; ict product; multicriteria decision analysis; performance evaluation; suppliers of ict products 1. introduction the selection of ict suppliers is done in two phases. the first phase is the registration which is normally preceded by appraisal of potential suppliers through analysis of responses to questionnaires for registration in accordance with regulation 8(3) of the public procurement and disposal regulations. the second phase is the pre-qualification of ict suppliers. the purpose of this phase is to enhance short listing of the suppliers for specific procurements as per regulation 23 of the public procurement and disposal regulation 2006 (g.o.k, 2009) dickson (1966) conducted a study to determine, identify and analyze the criteria used in the selection of suppliers and came up with twenty three evaluation criteria that have evolved over time the industry changes. a case in point is the dynamic ict industry where hardware and software produced to the market change periodically. samoei et al. (2016) observed that the suppliers of such products should be appraised continuously so as to determine their performance as per user preference criteria. shalle et al. (2014) noted that buyer-supplier evaluation is the process of evaluating and approving potential suppliers by factual and measurable assessment with the aim of identifying a portfolio of the best class of suppliers. in addition, belton (2006) argued that the evaluation challenge is concerned with a choice between discretely defined alternatives. he also noted that multicriteria decision analysis(mcda) is applicable in situations which necessitate the consideration of different courses of action based on various values. similarly, suppliers of ict products should be selected from a pool of suppliers in the market or from a prequalified list taking into consideration a set of evalution criteria. the supplier selected would be deemed optimal having met the set of evaluation criteria. this is in aggrement with drucker (2002) who posited that, “a decision is a judgement which is a choice between alternatives. it is rarely a choice between right or wrong. it is at best a choice between “almost right” and “probably wrong”-but more often a choice between two courses of action neither of which is provably nearly right than the other” (p. 150). 2. statement of the problem an organization is required to maintain a reliable, efficient and effective supply-chain in order to achieve its core and non-core values and to maintain its competitive advantage in the market. the main categories of factors that make the decision of supplier selection complex are multiple qualitative and quantitative criteria, conflicting objectives of the criteria, involvement of many alternatives because of high competition and internal and external constraints imposed on buying process (mwikali & kavale, 2012). any firm or organization must therefore have effective procedures for selecting suppliers of its goods and services. at moi university, the evaluation and selection of vendors to supply ict products is done by the procurement unit and the ict directorate. selecting suitable suppliers of ict products is always a challenging task. a procuring entity must therefore consider multiple, competing, tangible and intangible criteria ijahp: samoei, gichoya, odero/an ahp variation modelling approach for performance measurement of suppliers of information communication technology products international journal of the analytic hierarchy process 201 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.487 before choosing an optimal supplier for a given product. the supply of ict products needs to take into account the high procurement lead times because the rapid change in computing technology can render some ict requests that are delivered late, obsolete. cases of supply of ict products which do not match the user specifications are prevalent, leading to their rejection during the testing and verification process. in such instances, implementation of ict projects is frequently delayed. such delays lead to non-realization of optimal utilization of the ict investments in the long run. the monetary value arising from such capital intensive ict infrastructure needs to be considered when evaluating the suppliers of ict products, so that only trusted and reliable suppliers are selected. further, it is a popular saying that, “you cannot manage what you cannot measure” and further “if you cannot measure it, you cannot improve it.” samoei et al. (2016) observed that the challenges inherent in selecting and maintaining suppliers of ict products can be minimized by adopting a continous appraisal process. thus, for public institutions like moi university to reap maximum returns on the tangible and intangible ict investments, they ought to adopt a model that measures both quantitative and qualitative criteria being aware of the fact that the lowest bidder is not always the optimum choice. this study assessed the preferences in terms of the criteria and sub criteria used in evaluating suppliers of ict products with a view towatd developing a multi criteria decision model based on the analytical hierarchy process framework (ahp) for evaluating and measuring the performance of the suppliers of ict products at moi university (saaty,1980). consequently, the supplier evaluation and selection issue was addressed based on a hierarchy of criteria and sub criteria upon which the respective suppliers of ict products are evaluated. the ahp based multi-criteria decision making model developed, if adopted by the procurement unit and ict directorate to evaluate the suppliers of ict products, could simplify the complex evaluation process used in determining optimal suppliers based on both quantitative and qualitative criteria. subsequently, this will ensure that the procuring entity upholds the principles of procurement which encompasses professionalism, transparency, accountability, competitiveness, fairness and ethics as embodied in the public procurement and disposal act (g.o.k, 2015). 3. methodology the study adopted the ahp general theory of measurement, albeit with a variation, since it is an ideal decision making method for ranking alternatives when multiple criteria and sub-criteria are presented in the decision making processes (tahriri, 2008). in the case of the supplier of ict products the goal is to evaluate and rank suppliers as per their performance scores on the distinct criteria and sub criteria. tam and tummala (2001) applied ahp in vendor selection of a telecommunication system, which was a complex, multi-person, multi-criteria problem. they established that ahp was very useful since the situation involved several decision makers with different objectives to arrive at a consensus decision. likewise, selection of suppliers of ict products involves users from the ict directorate who provide the product specifications and the procurement unit who source the suppliers. the decision is to ijahp: samoei, gichoya, odero/an ahp variation modelling approach for performance measurement of suppliers of information communication technology products international journal of the analytic hierarchy process 202 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.487 choose an effective supplier based on weighting of the criteria and sub criteria upon which their performance measurement priorities are derived. the following steps, developed by saaty (2008) for applying ahp were followed in the evaluation of criteria and suppliers of ict products: step 1: problem definition and goal determination: the goal was to rate the suppliers of ict products as per their score on various criteria step 2: identification of all the criteria which affect the research problem: this was realized through literature review and interviewing experts. the criteria were separated in accordance with the level of internal relevance and individual independence. step 3: structuring the problem hierarchically: this was from the top (the objectives as per procurement and ict decision makers’ point of view) through the intermediate levels (criteria on which subsequent levels depend) to the lowest level which contains the list of alternatives. step 4: construction of a set of pair-wise comparison matrices (size n x n): this was done for each of the lower levels with one matrix for each element in the level immediately above by using the relative scale measurement shown in table 1. table 1 saaty’s (1-9) scale for comparing criteria, sub-criteria and suppliers of ict products relative to sub-criteria reference level numerical value equally preferred 1 equally to moderately preferred 2 moderately preferred 3 moderately to strongly preferred 4 strongly preferred 5 strongly to very strongly preferred 6 very strongly preferred 7 very strongly to extremely preferred 8 extremely preferred 9 source: (saaty, 1980) the pair-wise comparisons are done in terms of which element dominates the other. if there are “n” criteria in one hierarchy, decision makers will be required to make n ((n -1)/2) judgments in order to develop the set of matrices as per step 4. contrary, reciprocals are automatically assigned in each pair-wise comparison thus: aij =1/aji for i,j =1,2 …n. (1) ijahp: samoei, gichoya, odero/an ahp variation modelling approach for performance measurement of suppliers of information communication technology products international journal of the analytic hierarchy process 203 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.487 the ahp variation used in this study involved using a survey approach to rate the criteria and using the responses to build a perfectly consistent pairwise comparison matrix. this is not classical ahp since we did not question the experts in order to compare the criteria, but rather asked them to rate the criteria, then took an average to build the pairwise comparison matrix. the advantage of this approach is that it avoids the problem of overall inconsistency since the matrix built by calculating the weights as the ratio of the average score of each criteria is perfectly consistent. step 5: hierarchical synthesis: the purpose of this step is to determine weights of the eigenvectors based on the weights of the criteria. the sum is taken over all weighted eigenvector entries corresponding to those in the next lower level of the hierarchy. this is as demonstrated by equations 4 and 5. step 6: consistency test: the purpose of this step is to determine whether the calculation fit the condition for the transitivity in priority. consistency ratio (cr) is used to verify the credibility and reasonability of evaluation and to check whether there is inconsistency principal right in subjective judgments. the cr is acceptable if it does not exceed 0.1(saaty, 1980). the consistency is determined by using the eigenvalue, λ max . consistency index (ci) –which was 0 in our studyand consistency ratio (cr) are calculated thus: 𝐶𝐼 = (λmax – n) n − 1 (2) cr = ( ci rin ) (3) the positive reciprocal matrix generated by the evaluation process yields different ci values at each level referred to as random indexes. λ max is the maximized eigenvector of a pair wise comparison matrix, n is an attribute of the matrix, and rin is a random index as shown in table 2 (saaty, 1980). table 2 saaty’s random index table n 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 r i 0 0 0.5 8 0.9 0 1.1 2 1.2 5 1.3 2 1.4 1 1.4 5 1.4 9 1.5 4 1.4 8 1.5 6 1.5 7 1.5 9 source: (saaty, 1980) step 7: synthesis via normalization: because the pwc was perfectly consistent, the study normalized the weights of the internal level(s) and connected the local weight to acquire the global weights of the criteria in each hierarchy after calculating the weights of all criteria. the application of the aforementioned 7 steps is illustrated in tables 4.1 to 4.1.2 in the determination of respondent’s consistency in judgment on the seven evaluation criteria. 3.1 modelling the suppliers of ict products decision problem figure 1 is the ahp hierarchical structure that guided the evaluation and selection of suppliers of ict products at moi university. the problem hierarchy had four levels, namely; level 0, level 1, level 2 and level 3. level 0 represents the goal of the evaluation ijahp: samoei, gichoya, odero/an ahp variation modelling approach for performance measurement of suppliers of information communication technology products international journal of the analytic hierarchy process 204 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.487 which was to rate the research alternatives based on their performance scores as per criteria. level 1 ranged from i-1 to i+1 and level 2 ranged from (i-1)1 to (i+1)2 representing the criteria and sub criteria respectively that were used in the rating of the alternatives. level 3 represented the alternatives which were the various suppliers of ict products. the lines between levels indicated the relationship amongst the given levels. figure 1. adapted ahp hierarchical structure (samoei, 2016) figure 2 illustrates the conceptualization of the evaluation and selection of suppliers of ict at moi university as an ahp multi-criteria hierarchical decision problem. the study structured the evaluation and selection problem into four levels namely: the goal, the criteria, the sub-criteria and alternatives. the goal (level 0) was to rate the suppliers of ict products according to their score on the given performance sub criterions under study. at level one of figure 2, seven criteria (7) were selected and rated in the study. these were delivery (d), quality (q), supplier status (ss), supplier culture (sc), flexibility (f), financial stability (fs), and commercial interest (ci). the second level of the hierarchy was the sub criteria. a total of twenty seven (27) sub criteria were considered thus: the delivery sub criterions were geographical location (gl) and transport and communication logistics (tcl); the quality sub criterions were quality of product (qp), percentage of on time deliveries (ptd), response to customer requests (rcr) and after sale services (ass); the supplier status sub criterions were its employees (e), reputation in the market (rm),reference clients (rc) and existing relationships (er); the supplier culture sub criterions were trust (t),integrity (ig), professionalism (p), innovation (in) and understanding of organizational goals (uog); the flexibility sub criterions were capacity ( c), technical capability (tc),information sharing capability (is) and availability (a); the financial stability sub criterions were business turnover (bt), cash flow (cf) and tax compliance (txc) and finally the commercial interest sub criterions were competitive pricing (cp), ownership structure/history (osh), intellectual property rights (ipr), non-disclosure of information(ndi) and currency stability (cs). alternative j-1 alternative j alternative j+1 sub criteria (i-1)1 sub criteria (i-1)2 sub criteria i1 sub criteria i2 sub criteria (i+1)1 sub criteria (i+1) criteria i-1 criteria i criteria i+1 goal ijahp: samoei, gichoya, odero/an ahp variation modelling approach for performance measurement of suppliers of information communication technology products international journal of the analytic hierarchy process 205 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.487 finally, the third level of the ahp conceptual model had seven alternatives (suppliers of ict products to be rated based on criteria and sub criteria). these were coded as supplier a, supplier b, supplier c, supplier d, supplier e, supplier f and supplier g respectively (figure 2). table 3 is a description of the seven criteria (level 1) and twenty seven respective performance sub criterions(level 2) and the seven suppliers (level 3) sampled for the study ( tables 8 and 9). ijahp: samoei, gichoya, odero/an ahp variation modelling approach for performance measurement of suppliers of information communication technology products international journal of the analytic hierarchy process 206 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.487 s u b c r i t e r i a goal criteria measurement of the performance of suppliers of ict products 1.1 gl 1.2 tcl 2.1 qp 2.2 ptd 2.3 rcr 2.4 ass 3.1 e 3.2 rm 3.3 rc 3.4 er 4.1 t 4.2 ig 4.3 p 4.4 in 4.5 uog 5.1 c 5.2 tc 5.3 is 5.4 a 6.1 bt 6.2 cf 6.3txc 7.1 cp 7.2 osh 7.3 ipr 7.4 ndi 7.5 cs 1.0. d 2.0. q 3.0.ss 4.0.sc 5.0. f 6.0. fs 7.0 ci supplier a supplier b supplier c supplier d supplier e supplier f supplier g a l t e r n a t i v e s figure 2. an ahp decomposition of the suppliers of ict products multicriteria decision problem adapted with modifications from samoei (2016) ijahp: samoei, gichoya, odero/an ahp variation modelling approach for performance measurement of suppliers of information communication technology products international journal of the analytic hierarchy process 207 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.487 table 3 performance evaluation criteria for suppliers of ict products s/n criteria abbreviation s/n sub-criteria abbreviation description 1.0 delivery d 1.1 geographical location gl location of the supplier relative to the procuring entity 1.2 transport and communication logistics tcl the nature of transport and communication network 2.0 quality q 2.1 quality of product qp quality of the ict product delivered relative to user per specifications 2.2 percentage of on time deliveries ptd the amount of deliveries made on time 2.3 response to customer requests rcr the ability of the supplier to attend to client’s requests/queries 2.4 after sale services ass the support services rendered to the procuring entity by the supplier 3.0 supplier status ss 3.1 employees e the number and quality of supplier’s employees 3.2 reputation in the market rm the supplier’s standing in the ict market 3.3 reference clients rc the nature of clients that the supplier has rendered services 3.4 existing relationships er the nature of ties/bond that exist between the supplier and the university 4.0 supplier culture sc 4.1 trust t trustworthiness of the supplier 4.2 integrity ig commitment to the process of due diligence. 4.3 professionalism p the ability of the supplier to render services professionally 4.4 innovativeness in the ability of the supplier to provide innovative/unique solutions 4.5 understanding of organizational goals uog the supplier’s understanding of the organizations goals/strategic direction ijahp: samoei, gichoya, odero/an ahp variation modelling approach for performance measurement of suppliers of information communication technology products international journal of the analytic hierarchy process 208 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.487 table 3 continued s/n criteria abbreviation s/n sub-criteria abbreviation description 5.0 flexibility f 5.1 capacity c the ability of the supplier varying volumes of requests 5.2 technical capability tc the ict technical infrastructure available 5.3 information sharing capability is the ability and willingness of the supplier to share product information with customer 5.4 availability a a measure of supplier’s willingness to avail services when needed 6.0 financial stability fs 6.1 business turnover bt the amount of sales made in a given period of time 6.2 cash flow cf the supplier’s liquidity 6.3 tax compliance txc the percentage of tax compliance made to kenya revenue authority 7.0 commercial interest ci 7.1 competitive pricing strategies cp the strategies adopted to remain competitive in the ict market 7.2 ownership structure/history osh the organization structure 7.3 implementation of intellectual property rights the ability to implement relevant software and hardware intellectual property rights 7.4 non-disclosure of information ndi the ability to maintain client’s confidential information 7.5 currency stability cs the stability supplier’s trading currency 7.1 competitive pricing strategies cp the strategies adopted to remain competitive in the ict market adapted from samoei (2016) ijahp: samoei, gichoya, odero/an ahp variation modelling approach for performance measurement of suppliers of information communication technology products international journal of the analytic hierarchy process 209 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.487 3.2 data collection and sampling documentary reviews were carried out to determine the criteria and sub criteria used by the ict directorate and procurement unit during the evaluation of suppliers of ict products where seven criteria and their respective27 sub criteria as shown in table 3 were selected for modelling the evaluation and selection problem (figure 2). based on the 7 criteria, 27 sub criteria and the 7 suppliers of ict products sampled for the study (figure 2) , an ahp questionnaire was structured where respondents ticked the desired options from a scale of 1 to 9 (saaty, 1980). fourteen questionnaires were distributed and ten valid questionnaires were returned representing a 71% response rate (table 4) .the respondents who returned the questionnaires was a good representation of the population in relation to the unit of service (table 4). table 4 response rate unit issued returned response rate ict 4 3 75% procurement 10 7 70% total 14 10 71% adapted from samoei (2016) 3.3 hierarchy structure analysis this involved the computations of the priority weights of the proposed ahp measurement model based on the four levels as illustrated in figure 2. this was done through pair wise comparisons of the criteria, sub criteria and supplier of ict products at each level as per ninepoint scale as indicated in table 1(saaty, 1980). hence, tables 6 to 8 illustrate the steps followed in the computation of priority weights of criteria based on the respondents average ratings shown in table 4. the consistency test for the evaluation criteria was 0 indicating a consistency in judgement. table 5 average rating of criteria based on the rating survey criteria d q ss sc f fs ci rating 4 5.25 4.75 3.75 3.88 4.6 5.125 adapted from samoei (2016) ijahp: samoei, gichoya, odero/an ahp variation modelling approach for performance measurement of suppliers of information communication technology products international journal of the analytic hierarchy process 210 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.487 table 6 criteria pair wise comparison matrix d q ss sc f fs ci d d/d d/q d/ss d/sc d/f d/fs d/ci q q/d q/q q/ss q/sc q/f q/fs q/ci ss ss/d ss/q ss/ss ss/sc ss/f ss/fs ss/ci sc sc/d sc/q sc/ss sc/sc sc/f sc/fs sc/ci f f/d f/q f/ss f/sc f/f f/fs f/ci fs fs/d fs/q fs/ss fs/sc fs/f fs/fs f/ci ci ci/d ci/q ci/ss ci/sc ci/f ci/fs ci/ci adapted from samoei (2016) table 7 criteria pair wise comparison matrix adapted from samoei (2016) 3.4 normalized relative weight of criteria the normalized relative weight criteria are obtained by dividing each element in the matrix in table 7 with the column sum. d q ss sc f fs ci d 1 0.7619 0.8421 1.0667 1.0309 0.8696 0.7805 q 1.3125 1 1.1053 1.4 1.3531 1.1413 1.0244 ss 1.1875 0.9048 1 1.2667 1.2242 1.0326 0.9268 sc 0.9375 0.7143 0.7895 1 0.9665 0.8153 0.7317 f 0.9700 0.7390 0.8168 1.0347 1 0.8435 0.7571 fs 1.1500 0.8762 0.9684 1.2267 1.1856 1 0.8976 ci 1.2813 0.9762 1.0789 1.3667 1.3209 1.1141 1 sum 7.8388 5.9724 6.5974 8.3615 8.0812 6.8164 6.1181 ijahp: samoei, gichoya, odero/an ahp variation modelling approach for performance measurement of suppliers of information communication technology products international journal of the analytic hierarchy process 211 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.487 table 8 normalized relative weight of criteria d q ss sc f fs ci sum average d 0.1276 0.1276 0.1276 0.1276 0.1276 0.1276 0.1276 0.8932 0.1276 q 0.1674 0.1674 0.1675 0.1674 0.1674 0.1674 0.1674 1.1719 0.1674 ss 0.1515 0.1515 0.1516 0.1515 0.1515 0.1515 0.1515 1.0606 0.1515 sc 0.1196 0.1196 0.1197 0.1196 0.1196 0.1196 0.1196 0.8373 0.1196 f 0.1237 0.1237 0.1238 0.1238 0.1237 0.1237 0.1237 0.8661 0.1237 fs 0.1467 0.1467 0.1468 0.1467 0.1467 0.1467 0.1467 1.0270 0.1467 ci 0.1635 0.1635 0.1635 0.1635 0.1635 0.1634 0.1634 1.1443 0.1634 sum 1 1 1 1 1 1 1 adapted from samoei (2016) the principal eigenvector is obtained by getting the average across the rows as shown in table 8. 3.5 consistency test determination of principal eigen value (λ max ) 0.1276 λ max = [7.8388 5.9724 6.5974 8.3615 8.0812 6.8164 6.1181] × 0.1674 0.1515 0.1196 0.1237 0.1467 0.1635 = 1.000+0.999+0.999+1.000+0.999+0.999+1.000 = 6.999 ˜ 7.000 determination of consistency index (ci) ci = (λmax n)/(n-1) (refer to equation 1) = (7-7) / (7-1) = 0 ijahp: samoei, gichoya, odero/an ahp variation modelling approach for performance measurement of suppliers of information communication technology products international journal of the analytic hierarchy process 212 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.487 since, cr<0.1, it implies that the respondents evaluation about suppliers of ict products criteria preference is consistent (saaty, 1980). 3.5.1 criteria level weights and ranking the ranking of the seven criteria was based on the normalized principal eigen vector (priority vector) as computed in table 7 and their ranking is as shown in table 9. table 9 criteria level weights and ranking criteria priority weight rank quality(q) 0.16733 1 commercial interest(ci) 0.16335 2 supplier status(ss) 0.15139 3 financial stability(fs) 0.14741 4 delivery(d) 0.12749 5 flexibility(f) 0.12351 6 supplier culture(sc) 0.11952 7 3.5.2 sub criteria level weights and ranking the global weight (priority weight) of the sub criteria was calculated thus: sub criteria global weight = criteria priority weight × sub criteria local weight (4) hence, table 10 shows the global weights of each sub criteria and the ranking within their respective sets of criteria. ijahp: samoei, gichoya, odero/an ahp variation modelling approach for performance measurement of suppliers of information communication technology products international journal of the analytic hierarchy process 213 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.487 table 10 sub criteria level weights and ranking nb: in all the cases the inconsistency ratio (cr= 0) implying that the respondents judgements were perfectly consistent. the results of ahp analysis through expert choice professional version 9.48s25 on the preference of the sub criteria by the respondents as shown in table 10 is summarised thus:  delivery sub criterions: the results indicated that of the two sub criterions considered, transport and communication logistics (tcl) was the most preferred over geographical location (gl) (table 10).  quality sub criterions: the order of preference of the four quality sub criterions in a descending order was: quality of product (qp), percentage of on time deliveries (ptd), and responses to customer requests (rcr) and after sale service (ass) (table 10). criteria priority weight sub-criteria local weight global weights ranking delivery(d) 0.12749 transport and communication logistics 0.444 0.07083 1 geographical location 0.556 0.05666 2 quality(q) 0.16733 quality of product 0.288 0.04816 1 percentage of on time deliveries 0.249 0.04163 2 response to customer requests 0.233 0.03898 3 after sale services 0.230 0.03857 4 supplier status(ss) 0.15139 reputation in the market 0.286 0.04326 1 existing relationships 0.280 0.04235 2 reference clients 0.250 0.03785 3 employees 0.185 0.02794 4 supplier culture(sc) 0.11952 professionalism 0.233 0.02782 1 integrity 0.219 0.02618 2 trust 0.216 0.02586 3 innovativeness 0.172 0.02060 4 understanding of organizational goals 0.159 0.01906 5 flexibility(f) 0.12351 technical capability 0.283 0.03493 1 availability 0.263 0.03244 2 capacity 0.247 0.03056 3 information sharing capability 0.207 0.02557 4 financial stability(fs) 0.14741 tax compliance 0.403 0.05940 1 cash flow 0.306 0.04510 2 business turnover 0.291 0.04290 3 commercial interest(ci) 0.16335 competitive pricing strategies 0.221 0.03604 1 currency stability 0.207 0.03374 2 implementation of intellectual property rights 0.197 0.03221 3 non-disclosure of information 0.192 0.03144 4 ownership structure/history 0.183 0.02991 5 ijahp: samoei, gichoya, odero/an ahp variation modelling approach for performance measurement of suppliers of information communication technology products international journal of the analytic hierarchy process 214 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.487  supplier status sub criterions: the order of preference of the four supplier status sub criterions in a descending order was: reputation in the market (rm), existing relationship(er), reference clients (rc) and employees (e) (table 10).  supplier culture sub criterions: the order of preference of the five supplier culture sub criterions in a descending order was: professionalism (p), integrity (i), trust (t), innovativeness (in) and understanding of organizational goals (uog) (table 10).  flexibility sub criterions: the order of preference of the four flexibility sub criterions in a descending order was technical capability (tc), availability (a), capacity (c) and information sharing capability (is) (table 10).  financial stability sub criterions: the findings indicated that the order of importance of the three financial stability sub criterions in a descending order was: tax compliance (txc), cash flow (cf) and business turnover (bt) (table 10).  commercial interest sub criterions:-the findings showed that the order of preference of the five commercial interest sub criterions in a descending order was: competitive pricing strategies (cp), currency stability (cs), implementation of intellectual property rights (ipr), non-disclosure of information (ndi) and ownership structure/history (osh) (table 10). thus, from the aforementioned analysis, transport and communication logistics (7.083 %), quality of product (4.816%), reputation in the market (4.326%), professionalism (2.782%), technical capability (3.493 %), tax compliance (5.94%) and competitive pricing strategies (3.604%) were the highest ranked sub criteria within their respective sets (table 10). 3.5.3 sub criteria priority ranking table 11 shows the descending order of sub criteria ranking as derived from table 10. ijahp: samoei, gichoya, odero/an ahp variation modelling approach for performance measurement of suppliers of information communication technology products international journal of the analytic hierarchy process 215 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.487 table 11 sub criteria overall ranking sub criteria global weight rank transport and communication logistics 0.07083 1 tax compliance 0.05940 2 geographical location 0.05666 3 quality of product 0.04816 4 cash flow 0.04510 5 reputation in the market 0.04326 6 business turnover 0.04290 7 existing relationships 0.04235 8 percentage of on time deliveries 0.04163 9 response to customer requests 0.03898 10 after sale services 0.03857 11 reference clients 0.03785 12 competitive pricing strategies 0.03604 13 technical capability 0.03493 14 currency stability 0.03374 15 availability 0.03244 16 implementation of intellectual property rights 0.03221 17 non-disclosure of information 0.03144 18 capacity 0.03056 19 ownership structure/history 0.02991 20 employees 0.02794 21 professionalism 0.02782 22 integrity 0.02618 23 trust 0.02586 24 information sharing capability 0.02557 25 innovativeness 0.02060 26 understanding of organizational goals 0.01906 27 3.6 measurement of the performance of the suppliers of ict products the next level was assessing performance of the alternatives (level 3) as per the developed ahp conceptual model (figure 2) who were the 7 suppliers of ict products at moi university who had been randomly selected for the study. they were evaluated based on the sub criterions (level 3).the global weights of each supplier were computed thus: supplier of ict products global weight = supplier′s local weight × sub criteria global weight (5) the findings showed that each of the 7 suppliers performed differently on each of the 27 rating sub criterions (table 7) demonstrating the different preferential treatment by the respondents. the summation of the 27 global weights for each of the 7 individual suppliers gave the performance score of the supplier of ict products at moi university as shown in table 12. ijahp: samoei, gichoya, odero/an ahp variation modelling approach for performance measurement of suppliers of information communication technology products international journal of the analytic hierarchy process 216 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.487 table 12 performance of the suppliers of ict products supplier a supplier b supplier c supplier d supplier e supplier f supplier g criteria subcriteria global weight rank local weight global weight local weight global weight local weight global weight local weight global weight local weight global weight local weight global weight local weight global weight d gl 0.05666 3 0.164 0.009292 0.174 0.009859 0.126 0.007139 0.141 0.007989 0.115 0.006516 0.156 0.008839 0.123 0.0069692 tcl 0.07083 1 0.166 0.011758 0.163 0.011545 0.154 0.010908 0.152 0.010766 0.105 0.007437 0.141 0.009987 0.118 0.0083579 q qp 0.04816 4 0.170 0.008187 0.170 0.008187 0.155 0.007465 0.160 0.007706 0.086 0.004142 0.152 0.007320 0.108 0.0052013 ptd 0.04163 9 0.165 0.006869 0.162 0.006744 0.177 0.007369 0.145 0.006036 0.100 0.004163 0.145 0.006036 0.106 0.0044128 tcr 0.03898 10 0.170 0.006627 0.175 0.006822 0.177 0.006899 0.153 0.005964 0.075 0.002924 0.147 0.005730 0.102 0.003976 ass 0.03857 11 0.198 0.007637 0.167 0.006441 0.149 0.005747 0.152 0.005863 0.090 0.003471 0.121 0.004667 0.121 0.004667 ss e 0.02794 21 0.166 0.004638 0.169 0.004722 0.165 0.004610 0.165 0.004610 0.091 0.002543 0.127 0.003548 0.118 0.0032969 rm 0.04326 6 0.164 0.007095 0.153 0.006619 0.157 0.006792 0.157 0.006792 0.109 0.004715 0.141 0.006100 0.117 0.0050614 rc 0.03785 12 0.167 0.006321 0.146 0.005526 0.152 0.005753 0.171 0.006472 0.106 0.004012 0.147 0.005564 0.111 0.0042014 er 0.04235 8 0.155 0.006564 0.160 0.006776 0.173 0.007327 0.170 0.007200 0.085 0.003600 0.150 0.006353 0.106 0.0044891 sc t 0.02586 24 0.158 0.004086 0.168 0.004344 0.161 0.004163 0.158 0.004086 0.091 0.002353 0.144 0.003724 0.120 0.0031032 i 0.02618 23 0.173 0.004529 0.163 0.004267 0.155 0.004058 0.155 0.004058 0.104 0.002723 0.144 0.003770 0.107 0.0028013 p 0.02782 22 0.168 0.004674 0.154 0.004284 0.148 0.004117 0.159 0.004423 0.120 0.003338 0.139 0.003867 0.112 0.0031158 inn 0.02060 26 0.151 0.003111 0.171 0.003523 0.166 0.003420 0.168 0.003461 0.102 0.002101 0.129 0.002657 0.113 0.0023278 uog 0.01906 27 0.159 0.003031 0.164 0.003126 0.168 0.003202 0.151 0.002878 0.110 0.002097 0.138 0.002630 0.110 0.0020966 f c 0.03056 19 0.165 0.005042 0.160 0.004890 0.166 0.005073 0.163 0.004981 0.099 0.003025 0.141 0.004309 0.108 0.0033005 tc 0.03493 14 0.172 0.006008 0.162 0.005659 0.160 0.005589 0.162 0.005659 0.099 0.003458 0.140 0.004890 0.105 0.0036677 isc 0.02557 25 0.158 0.004040 0.170 0.004347 0.147 0.003759 0.150 0. 0.038360.038360 0.130 0.003324 0.157 0.004014 0.089 0.0022757 a 0.03244 16 0.165 0.005353 0.165 0.005353 0.165 0.005353 0.154 0.004996 0.112 0.003633 0.137 0.004444 0.101 0.0032764 fs bt 0.04290 7 0.177 0.007593 0.159 0.006821 0.160 0.006864 0.160 0.006864 0.105 0.004505 0.138 0.005920 0.132 0.0056628 cf 0.04510 5 0.181 0.008163 0.153 0.006900 0.149 0.006720 0.136 0.006134 0.109 0.004916 0.163 0.007351 0.109 0.0049159 txc 0.05940 2 0.166 0.009860 0.166 0.009860 0.165 0.009801 0.165 0.009801 0.103 0.006118 0.130 0.007722 0.103 0.0061182 ci cps 0.03604 13 0.165 0.005947 0.160 0.005766 0.158 0.005694 0.166 0.005983 0.101 0.003640 0.144 0.005190 0.107 0.0038563 osh 0.02991 20 0.176 0.005264 0.148 0.004427 0.146 0.004367 0.174 0.005204 0.113 0.003380 0.138 0.004128 0.105 0.0031406 iip 0.03221 17 0.176 0.005669 0.147 0.004735 0.144 0.004638 0.155 0.004993 0.132 0.004252 0.135 0.004348 0.111 0.0035753 ndi 0.03144 18 0.159 0.004999 0.167 0.005250 0.164 0.005156 0.134 0.004213 0.130 0.004087 0.117 0.003678 0.130 0.0040872 cs 0.03374 15 0.190 0.006411 0.130 0.004386 0.152 0.005128 0.178 0.006006 0.101 0.003408 0.135 0.004555 0.113 0.0038126 supplier's score 0.16877 0.16118 0.15711 0.15697 0.10388 0.14134 0.111767 supplier's ranking 1 2 3 4 7 5 6 ijahp: samoei, gichoya, odero/an ahp variation modelling approach for performance measurement of suppliers of information communication technology products international journal of the analytic hierarchy process 217 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.487 table12 shows the descending order of performance of the suppliers of ict products being supplier a (0.16877),supplier b (0.16118),supplier c (0.15711),supplier d (0.15697), supplier e (0.14134), supplier f (0.111767) and supplier g (0.10388). 4. conclusion this study investigated and assessed the criteria used to evaluate suppliers of ict products with a view to developing an ahp-variation based multi criteria decision model for measuring the performance of suppliers of ict products. the ahp variation used in this study consisted in using a survey approach to rate the criteria and using the responses to build a perfectly consistent pairwise comparison matrix to calculate the criteria weights. this solves the problem of obtaining inconsistent pairwise comparison matrices when surveying many respondents. the study findings showed that the performance of a supplier of ict products is dependent on an individual score on the varied criteria and respective sub criteria as demonstrated by the ahp modelling approach. based on the evaluation results, a supplier would be deemed optimum if he is extremely preferred on each and every criteria/sub criteria used in the evaluation process, in addition to being rated highly on the highest ranked sub criterions. hence, the ahp-variation multi-criteria decision making model developed can enable a procuring entity to select or reject suppliers of ict products based on the score attained on a given criteria or set of criteria depending on the type of hardware, software or ict service being procured. applied consistently, the model can enable public procuring entities to enhance transparency in the procurement processes. ijahp: samoei, gichoya, odero/an ahp variation modelling approach for performance measurement of suppliers of information communication technology products international journal of the analytic hierarchy process 218 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.487 references belton v. (1990). multi criteria decision analysis-practically the only way to choose. in hendry, l.c. and eglese, r.w. (eds), operational research tutorial papers (53101). birmingham: or society. g.o.k. (2015). public procurement and asset disposal act [online] http://ppoa.go.ke/images/downloads/public%20procurement%20and%20asset%20di sposal%20act%202015.pdf. g.o.k.(2009).public procurement manual for ict [online] www.ppoa.go.ke/images/downloads/manuals/ict_manual.pdf dickson, g.w. (1966). an analysis of vendor selection systems and decisions. journal of purchasing, 2(1), 5-17. drucker ,p. f. (2002). the effective executive: the definitive guide to getting the right things done. united kingdom: harpercollins publishers ltd. mwikali r. and kavale s. (2012). factors affecting the selection of optimal suppliers in procurement management. international journal of humanities and social science, 12(14), 189-193. saaty, t. l. (2008). decision making with the analytic hierarchy process. international journal of services sciences, 1(1), 83-98. doi: 10.1504/ijssci.2008.017590 saaty, t.l. (1980). the analytical hierarchy process. new york: mcgraw-hill. shalle, n.i & guyo, w. & amuhaya, i.m. (2014). effects of buyer/supplier collaboration on eprocurement performance in state corporations in kenya. european journal of management sciences and economics, 1(4), 170-185. samoei, d.k., gichoya, d., & odero, d. (2016). management strategies for consumer supplier relationship in kenya: a case of suppliers of information communication technology products in moi university. international journal of information and communication technology research, 6(12). samoei, d.k. (2016). a modular neural network model for evaluation and selection of suppliers of information communications technology products at moi university. unpublished master’s thesis, moi university , eldoret, kenya. tahriri, f., osman, m. r., ali, a., & yusuff, r. m. (2008). a review of supplier selection methods in manufacturing industries. suranaree journal of science and technology, 15(3), 201-208. team, m. c., & tummala, v. r. (2001). an application of the ahp in vendor selection of a telecommunications system. omega, 29(2), 171-182. doi: https://doi.org/10.1016/s0305-0483(00)00039-6 http://ppoa.go.ke/images/downloads/public%20procurement%20and%20asset%20disposal http://ppoa.go.ke/images/downloads/public%20procurement%20and%20asset%20disposal http://www.ppoa.go.ke/images/downloads/manuals/ict_manual.pdf http://dx.doi.org/10.1504/ijssci.2008.017590 https://doi.org/10.1016/s0305-0483(00)00039-6 ijahp article: saracoglu, mahallesi, caddesi, sokak/an ahp application in the investment selection problem of small hydropower plants in turkey international journal of the analytic hierarchy process 211 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.198 an ahp application in the investment selection problem of small hydropower plants in turkey burak omer saracoglu orhantepe mahallesi tekel caddesi geziyolu sokak 1 34865 dragos, kartal, istanbul, turkey e-mail: burakomersaracoglu@hotmail.com abstract turkey is a country that has had to deal with high electricity consumption due to its rapid growth and development. attempts have been made to address this increase in demand by increasing the electricity supply by means such as boosting resources by privatization. the electricity generation sector has been privatized in turkey; hence investors have been looking for the most appropriate electricity generation projects for a long while. therefore, researchers and practitioners should focus on how the most suitable small hydropower plant project (shpp) investments can be selected. in this study, the analytic hierarchy process (ahp) was applied to make the most appropriate and satisfying decision according to the decision makers, experts or investors for a shpp investment from amongst some alternative shpp investments in turkey. all of these shpps were assumed to be in the pre-development investment stage, which indicated that their prefeasibility and feasibility studies had not been presented or taken into consideration until this study was performed. this main constraint forced the researchers to obtain the data and information from the information forms such as the application document to the general directorate of state hydraulic works during the data and information gathering process. the ahp model of this study was structured in four levels with five main criteria, seventeen basic criteria and five alternatives on the super decisions software. the results indicated that alternative 1 and alternative 5 should be primarily investigated in detail in subsequent shpp investment investigation steps. keywords: analytic hierarchy process; investment; pre-development investment stage; small hydropower plant project; turkey 1 the author would sincerely like to express his deepest thankfulness to mrs. rozann whitaker saaty for her guidance and support on prof. dr. thomas l. saaty's research studies and the super decisions software (http://www.superdecisions.com/). ijahp article: saracoglu, mahallesi, caddesi, sokak/an ahp application in the investment selection problem of small hydropower plants in turkey international journal of the analytic hierarchy process 212 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.198 1. introduction the researchers focused on two important indicators or variables for the electricity demand projections today. one of the important indicators is population growth and the other is income growth (chandran et al., 2010; cook, 2011; dalgaard and strulik, 2011). as a rule of thumb, with today’s technical and technological capabilities, constraints and conditions, it can be said and expressed as a cliché that, when both the population and the income increases the electricity demand will also increase. turkey is one of the growing economies in the caucasus (azerbaijan, georgia, etc.), the mena (middle east and north africa: egypt, lebanon, united arab emirates, morocco, tunisia etc.) and the wana (west asia and north africa) regions. turkey’s population also increases from year to year. the gni (gross national income) per capita, which is one of the basic indicators for the growth and standard of living, and the total population in turkey are presented in figure 1. the average population growth (annual %), arithmetic mean from 1980 to 2014, and the average gni growth (annual %), arithmetic mean from 1988 to 2014, is respectively 1,64 and 4,07 in turkey (world bank, 2015) (for formula see abramowitz and stegun, 1972). under these conditions, the electricity demand can easily be estimated to steadily increase. figure 1. gni per capita and total population in turkey, gni (gross national income), gni per capita: ppp (purchasing power parity), $: united states dollar sign (world bank, 2013). turkish electricity generation sector players are very diversified in their capabilities, capacities, sizes, organization, management styles etc. therefore, the decision making process of each of these players differs. the turkish electricity generation sector players can select either fossil fuels (oil, coal etc.) or renewable energy sources (ress) to increase the electricity supply to compensate the demand. if they select the ress, they 65 66 67 68 69 70 71 72 73 74 75 6.000 7.000 8.000 9.000 10.000 11.000 12.000 13.000 14.000 15.000 16.000 17.000 18.000 19.000 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 m il li o n s g n i p e r c a p it a , p p p ( c u r r e n t in te r n a ti o n a l $ ) years gni per capita, ppp (current international $) population, total t o ta l p o p u la tio n ijahp article: saracoglu, mahallesi, caddesi, sokak/an ahp application in the investment selection problem of small hydropower plants in turkey international journal of the analytic hierarchy process 213 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.198 have some alternative sources such as hydro, wind, geothermal, biomass, biogas, and solar (demirbas, 2002; topcu and ulengin, 2004). hydropower technology has its own advantages such as being technically and technologically mature, having a very long lifespan, and being economically competitive (hall, 2003; iha, 2000; lako et al., 2003). hence, some of the private investors have been interested in the hydropower industry in turkey. the classification of hydropower plants (large, medium, small, mini, micro and pico) by the installed power or capacity (p) shows that there is not any agreed upon classification for the hydropower plants (see table 1) (moreira and poole, 1993; kurien and sinha, 2006; bajaj et al., 2007; saxena, 2007; beraković et al., 2009; dragu et al., 2010; erec, 2012). in this study, shpps were defined as 1 mw < p ≤ 10 mw. the lower limit (1 mw) of this installed capacity was chosen by the author due to the almost total consensus of the research (see table 1). it was also chosen due to the upmost limit of the unlicensed electricity generation activities from the renewable energy sources (without a license or establishing a company or electricity generation license exemptions) in turkey by the electricity market law 6446, the regulation on the unlicensed electricity generation in the electricity market, and the communication concerning the application of regulation on the unlicensed electricity generation on the electricity market of the republic of turkey. the upper limit (10 mw) of this installed capacity was chosen by the author due to the definition of shp (small hydropower plant) supported by the european commission (ec) and the european small hydropower association (esha) (see ec, 2014; esha et al., 2008). table 1 small hydropower plant classification by installed power (kw: kilowatt) india united states china croatia erec <30.000 ≤25.000 dragu et al. 1.000< ≤25.000 1.000< ≤30.000 500< ≤25.000 beraković et al. 50< ≤10.000 saxena 2.000< ≤25.000 bajaj et al. <25.000 <30.000 <25.000 kurien & sinha 1.000< ≤25.000 1.000< ≤30.000 500< ≤25.000 moreire & poole 1.000< ≤25.000 1.000< ≤30.000 500< ≤25.000 (source: moreire and poole, 1993; kurien and sinha, 2006; bajaj et al., 2007; saxena, 2007; beraković et al., 2009; dragu et al., 2010; erec, 2012) the emra’s (republic of turkey energy market regulatory authority) official website) was regularly visited while this study was conducted (emra, 2013. 212 shpps were found under the investigation and evaluation stage of the license application procedure of emra. the total installed electrical power of these 212 shpps was 1.048 mwe (mwe: the electrical installed power in megawatts on the granted license by emra). the shpps' applications to emra have been summarized in figures 2 and 3 to make the actual private sector license applications status in turkey clear. ijahp article: saracoglu, mahallesi, caddesi, sokak/an ahp application in the investment selection problem of small hydropower plants in turkey international journal of the analytic hierarchy process 214 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.198 the relatively small to midsized investors can easily prefer to invest in the shpps, because of very well-known advantages such as having a small capital requirement, being decentralized, and having a great number of potential projects in turkey. the large size investors also prefer to invest in the shpps, and usually own them in several bunches. they have shares of many of them during the same period according to different strategic investment principles. each investor has their own strategy or approach due to their different attributes and qualities. with the shpps availability and electricity generation market conditions in turkey, some private investors have been looking for ways to find projects in which they can be shareholders (or whole owner of the project). also, some of the projects have been waiting for private investors to supply enough capital to continue and finalize the construction and procedural activities to connect to the national grid. figure 2. shpps' applications to emra by region (installed power (electrical) (mwe) (latest 2012 september) (emra, 2012, basemap: wikipedia, 2014). marmara region 25,68 mwe aegean region 22,63 mwe mediterranean region 210,87 mwe central anatolia region 76,78 mwe black sea region 470,10 mwe eastern anatolia region 199,53 mwe southeastern anatolia region 42,35 mwe installed power (electrical power) (mwe) color scheme 0 30 30 60 60 90 180 210 210 240 450 480 colour legend ijahp article: saracoglu, mahallesi, caddesi, sokak/an ahp application in the investment selection problem of small hydropower plants in turkey international journal of the analytic hierarchy process 215 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.198 figure 3. shpps' applications to emra by year (installed power (electrical) (mwe) (latest 2012 september) (emra, 2012). in this paper, a part of a real life case study was presented. in this real life case study, the investors wanted to invest in one or more shpps from the available shpps on the projects’ radar. they asked to find some alternative projects and their related data and information. moreover, an evaluation and recommendation study about the projects was also requested. all requested and related activities were performed, and afterwards the ahp methodology was applied for five shpps on the investor’s radar. due to the concern of the commercially sensitive information involved, the titles or names of these projects were not given in this paper; however all of the necessary and required data and information relevant to these projects is supplied in the following sections. 2. literature review the review of previous research was carried out at two separate times. the first review was carried out until 05/09/2013 in the original manuscript. the second one was until 05/04/2015 in the second round revision of this manuscript. the previous studies were reviewed on scientific online database websites with the help of three key phrases. these phrases were analytic hierarchy process & small hydro (a), ahp & small hydro (b), small hydro (c). the unlimited truncation option was selected on the search preferences of these database websites. henceforth, all types of documents in all types of sources such as books, journals, proceedings, transactions, magazines, and newsletters were searched based on the anywhere on the document rule. as a result, titles, abstracts, main texts, references and all other documents on these database websites were searched in this study. the subjects of these sources were not limited so that all of the subjects such as agricultural sciences, business and management, computer sciences, decision sciences, energy and power were directly reviewed in the current study. the literature review was summarized in table 2. 0 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 160 170 180 190 200 210 220 230 2005 2006 2007 2008 2009 2010 2011 2012 in st a ll e d p o w e r ( e le c tr ic a l p o w e r ) (m w e ) years ijahp article: saracoglu, mahallesi, caddesi, sokak/an ahp application in the investment selection problem of small hydropower plants in turkey international journal of the analytic hierarchy process 216 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.198 table 2 summary of literature review scientific publisher a b c acm digital library 0 0 49 asce online research library 0 0 46 cambridge journals online 0 0 66 doaj 0 0 29 emerald insight 0 0 20 google scholar 130 197 18.900 ijahp 0 0 0 science direct 44 45 2.512 taylor & francis journal 2 2 310 source: (acm digital library, 2015; asce online research library, 2015; cambridge journals online, 2015; doaj, 2015; emerald insight, 2015; google scholar, 2015; international journal of the analytic hierarchy process (ijahp), 2015; science direct, 2015; taylor & francis journal, 2015). there were quite a remarkable number of papers that had considered or focused on small hydro power in a manner (economy, finance, planning etc.) in the scientific journals until 05/04/2015. none of these studies adopted ahp or modeled ahp in the scope and subject of the current study, with only two studies in the literature being close. mladineo et al. (1987) selected some shpp locations using the promethee method (promethee: preference ranking organization method for enrichment evaluations). the most critical criterion in their study was the unit cost (the lower, the more preferable). saracoglu (2015) presented an experimental research study on the selection of private small hydropower plant investments. his main research aim was to build up an intelligent autonomous decision making system for shpps. he used the electre iii (electre: elimination and choice translating reality) and the electre iv methods for the ranking of actions. the voting power of criteria was modeled by three different approaches as the objective weighting by shannon’s entropy, the subjective weighting by saaty’s ahp pairwise comparison, and the equal weighting. the experimental research study was applied based on the group decision making model. in short, there were only a few relevant papers in an enlarged view as given in table 3. table 3 summary of literature review paper subject/topic method mladineo et al., 1987 shpp location selection promethee saracoglu, 2015 private shpp investment selection electre iii/iv, shannon’s entropy objective weighting, saaty’s ahp subjective weighting this extensive literature review showed that this paper would be one of the first studies looking at the selection problem of the shpps’ investments. the ahp methodology that ijahp article: saracoglu, mahallesi, caddesi, sokak/an ahp application in the investment selection problem of small hydropower plants in turkey international journal of the analytic hierarchy process 217 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.198 was developed by thomas l. saaty was preferred for the solution of the problem because of the general advantages of the ahp method (saaty, 1980; saaty, 1990). these advantages include the ability to model daily real life problems with ease and simplicity; ability to reflect the reality of the problems in a true and easy manner; giving experts the ability to express their thoughts in a free, correct and almost perfect manner due to their experiences; giving people with little or no knowledge about decision making methods the opportunity to understand the method; having simple and easy mathematical calculations, which helps the experts concentrate on the problems rather than the difficult mathematical calculations; having the pair-wise comparisons, which helps the experts and decision makers compare one by one each criteria and alternative with respect to the goal and with respect to the alternatives (coyle, 2004; saaty, 2008; saaty, 2010; saaty, 2011; yoon and hwang, 1995). 3. ahp application in the case study there were five shpp investment alternatives (alternative1 to 5: alt1 to 5) in this case study. they were available for sale (like sales in other markets such as agricultural, art, automotive, banking, housing, shipping, and sport). also, some shares (stocks) of them could be bought (exchanged/purchased) in the shpp portfolio market (search for acquisition, asset, joint venture, merger, portfolio, share, and stock terms on the investopedia (investopedia us, 2014)). these shpps were assumed to be in the predevelopment investment stages (see table 4 for the details of these shpps according to the factors). there were seventeen shpps' investment selection factors in this study, which were determined based on the available data and information. the main structures of a shpp must be considered in order to understand the factors well. these main structures are the diversion weir, the de-silting tank, the channel, the tunnel, the headpond/forebay, the penstock, the powerhouse, the tailrace, and the substation. moreover, the electrical installed capacity (in watts) of a shpp should be kept in mind (formula/equation 1). p = ηtr × ηg × ηt × ρw× g × q × hnet (1) (ηtr: efficiency of transformer, ηg: efficiency of generator, ηt: efficiency of turbine, ρw: density of water (kg/m 3 ), g: gravity (m/s 2 ), q: rated discharge (m 3 /s), hnet: net head (m)) (for extraction of this formula/equation see eliasson and ludvigsson, 2000; esha, 2004; esha, 2005) the basic shpps investment selection factors (subjective and objective) in the current study were found, defined, identified, described, determined and selected by the experts from a pool of selection factors of the shpps. these factors were gathered from this study and the other studies of the author due to the selection factors' ability, competence, capability, effectiveness, and help analyzing the alternatives in a very quick, rapid, fast, appropriate, and applicable manner for this so-called the pre-development investment stage. the data and information availability for each factor was also considered. the basic shpps investment selection factors in this study include (see also saracoglu, 2015 for complementary, supportive and supplementary information): ijahp article: saracoglu, mahallesi, caddesi, sokak/an ahp application in the investment selection problem of small hydropower plants in turkey international journal of the analytic hierarchy process 218 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.198 c1: river basin (subjective criteria): this factor was for the evaluation of the main river basin of the project amongst other alternatives’ main river basins with the knowledge and experience of the experts (01 meric-ergene, 02 marmara, 03 susurluk, 04 kuzey ege, 05 gediz, 06 kucuk menderes, 07 buyuk menderes, 08 batı akdeniz, 09 antalya, 10 burdur, 11 akarcay, 12 sakarya, 13 batı karadeniz, 14 yesilırmak, 15 kızılırmak, 16 konya kapalı, 17 dogu akdeniz, 18 seyhan, 19 ası, 20 ceyhan, 21 dicle-fırat, 22 dogu karadeniz, 23 coruh, 24 aras, and 25 van golu) in turkey. it is important to not misunderstand the river basin term, and not to confuse it with the geographical regions (aegean, black sea, central anatolia, eastern anatolia, marmara, mediterranean, and southeastern anatolia) of turkey (for this term visit (eia, 2013)) (eia: united states energy information administration). c2: catchment area (objective criteria, km 2 ) (more is better ↑ ↑): the catchment area was evaluated by the approximate values presented in the information form (dsi, 2013) (dsi: the general directorate of state hydraulic works) of the shpp (for this term visit (eia, 2013)). c3: project runoff (objective criteria, hm 3 ) (more is better ↑ ↑): the project runoff was considered as the mean yearly total runoff and evaluated by the approximate values presented in the information form of the shpp (usgs, 2013) (usgs: united states geological survey). the runoff is normally generated by rainfall and the melting of snow. c4: conveyance structure (subjective criteria): in this factor, the conveyance structure type, the length of conveyance structure and other properties of the conveyance structure were evaluated with knowledge and experience of the experts. for instance, the conveyance structure could be a long tunnel or a short tunnel, or an open channel, or a short tunnel and an open channel, that would make the project complex or simple and difficult to construct or easy to construct and to operate (for this term visit (iwa, 2013)). c5: net head (objective criteria, m) (more is better ↑ ↑): the net heads of the shpps were compared with each other (for this term visit (eia, 2013)). wherever there was more than one net head in the projects, the equivalence of these net heads based on the electrical installed power formula of shpps had been calculated and the equivalence net head was written in the input file. this approximation and assumption could be easily agreed upon by the experts. the technically perfect conditions were assumed such as perfect efficiency. c6: flow rate (objective criteria, m 3 /s) (more is better ↑ ↑): the flow of the shpps was compared with each other (for this term visit (eia, 2013)). wherever there was more than one flow rate in the projects, the equivalence of these flow rates based on the electrical installed power formula of shpps had been calculated and the equivalence flow rate was written in the input file. this approximation and assumption could be easily agreed upon by experts. the technically perfect conditions were assumed such as perfect efficiency. c7: firm energy (objective criteria, gwh) (more is better ↑ ↑): the approximate firm energy of the shpps was compared. esha (2004) defined the firm energy as “the power delivered during a certain period of the day with at least 90 – 95% certainty”. ijahp article: saracoglu, mahallesi, caddesi, sokak/an ahp application in the investment selection problem of small hydropower plants in turkey international journal of the analytic hierarchy process 219 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.198 c8: secondary energy (objective criteria, gwh) (more is better ↑ ↑): the approximate secondary energy of the shpps was compared with each other. the secondary energy is the energy which can be generated by the shpp and this energy is added to the firm energy to reach the total energy. c9: investment cost (objective criteria, usd) (less is better ↓ ↑): the approximate total estimated investment cost of the shpps was compared with each other. the investment cost data in turkish lira currency was gathered from the information forms. the foreign-exchange rate data (1 usd = apprx. 2,06 tl) was taken from the official webpage of the central bank of the republic of turkey (tcmb) visited on september 8,2013 (tcmb, 2013). some other issues such as the escalation, the inflation etc. were not taken into consideration because of the subject of the current study. c10: community attitude (subjective criteria): the public opinion of the local people in the shpps’ locations can be either in positive (giving support and participating) or negative (showing opposition and protesting). if there is some opposition and protests against the shpps by the local people, then there will be a high possibility of the construction work being stopped or blocked, and this will cause some legal cases in the courts. the activities during the construction period and the operation period will be performed in difficult conditions, if there are oppositions and protests. the community attitude was evaluated based on these conditions and principles in this study. c11: transportation (subjective criteria): this factor was for the evaluation of all modes (air, rail, road, water, and cable) of transportation in the view of availability, flexibility, quality, and general conditions of the shpps. during the construction period of the shpps, all kinds and sizes of equipments such as the water or hydropower turbines and generators, materials etc. are transported by means of various transportation modes. the road transportation may be the most important one, because the shpps have to be reached by road by the equipment, materials and people. if the road to shpp site is available and good quality this will be an advantage for the site, otherwise new roads such as the access roads have to be built. in addition, the air transportation is very important because the staff, supervisors, managers and owners will want to reach the shpp site in the short duration, so that the availability of good quality air transportation will be preferable. in the current case study, this factor was evaluated by all means of transportation at once. c12: topography (subjective criteria): this factor was for the evaluation of natural and artificial surface shapes and features of the shpps. the appropriate topography for the construction and operation of the shpp was evaluated during the current study. wherever necessary, the related maps were investigated. c13: geology (subjective criteria): this factor was for the evaluation of general geological conditions and properties of the location of the shpps. the appropriate geology for the construction and operation of the shpp was evaluated during the current study. wherever necessary, the related data, maps and information were investigated. this factor also includes the natural hazards such as earthquakes, landslides etc. ijahp article: saracoglu, mahallesi, caddesi, sokak/an ahp application in the investment selection problem of small hydropower plants in turkey international journal of the analytic hierarchy process 220 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.198 c14: security conditions (subjective criteria): this factor was for the evaluation of the perceived security risks, threats and so forth in the areas of human security, public security, and infrastructure security etc. (see front line, 2005). theft, burglary etc. were considered during the comparisons of the alternatives. the terrorism related issues were not evaluated in this scope under this criteria, they were evaluated under the c15 (terrorism conditions) criteria. c15: terrorism conditions (subjective criteria): this factor was for the evaluation of the perceived terrorism risks, threats, effects and so forth in any type of terrorism (all internationally accepted types such as civil disorder, political, non-political, quasi, ideological, official or state or religious etc.) (visit united nations, 2013) for the definition of terrorism) in the location or nearby locations of the shpp. the experts compared the alternatives based on this factor which was actually a whole group of the types of this factor. (visit unodc, 2013) (unodc: united nations office on drugs and crime) to read and understand this factor). c16: protected areas (subjective criteria): this factor considered all of the protected areas such as natural parks, ecological values (important bird areas, rainforests), cultural values, regional values etc. wherever, there are some protected areas in the shpp site or nearby it, there will be the possibility of some additional procedural work and permissions needed. some of the necessary permissions often can’t be obtained in the protected areas, so that the shpp can’t be built up in those regions. c17: substation conditions (subjective criteria): substation network is very important for the national or international electricity networks. the national or international substation network is a constraint function for the whole electricity system. the generation sources have to connect to the grid by the substations. this factor was for the evaluation of the possibility, capability and conditions of connection to the national grid by the substation. this factor was very critical on the financing, construction, and operation of the utility-scale shpps in turkey. ijahp article: saracoglu, mahallesi, caddesi, sokak/an ahp application in the investment selection problem of small hydropower plants in turkey international journal of the analytic hierarchy process 221 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.198 table 4 shpp investment alternatives (see also saracoglu, 2015 for additional information) alternatives c1 c2 c3 c4 c5 c6 c7 c8 c9 alternative1 08 79 47 channel closed rectangular 3900 369 3 26 15 11.429.000 alternative2 23 61 48 upvc pipe 3300 388 3 6 31 7.783.000 alternative3 21 329 131 channel open rectangular 9600 172 8 16 38 16.551.000 alternative4 22 130 133 tunnel modified horseshoe 8800 135 10 14 29 24.052.000 alternative5 22 553 701 tunnel circular 5900 97 19,5 29 23 27.878.000 in short, there were ten subjective factors c1: river basin, c4: conveyance structure, c10: community attitude, c11: transportation, c12: topography, c13: geology, c14: security conditions, c15: terrorism conditions, c16: protected areas, c17: substation conditions, and seven objective factors c2: catchment area, c3: project runoff, c5: net head, c6: flow rate, c7: firm energy, c8: secondary energy, c9: investment cost. in the ahp analysis, an evaluation of a total of 9 factors has been recommended (gawlik, 2008; kruger and hattingh, 2006; saaty, 1980). this basic principle was based on the magic number 7, and the 7±2 rule, which was built upon research studies on the human psychological limit, human cognition and short term memory capacity. moreover, the magic number 7, and the 7±2 rule was suggested for all of the cognitive studies to ensure the reliability and the trustworthiness of the attendances (miller, 1956; shiffrin and nosofsky, 1994). henceforth, the basic factors in the current study were grouped under the aggregate factors. after strictly obeying the maximum limit of the magic number 7, and the 7±2 rule, the problem was modeled by a multi-layer structure with five main criteria (mc1: essential features of the shpp, mc2: complementary features of the shpp, mc3: electricity generation features of the shpp, mc4: social features of the shpp's site/location, and mc5: essential features of the shpp's site/location) and seventeen sub-criteria as presented in figure 4. ijahp article: saracoglu, mahallesi, caddesi, sokak/an ahp application in the investment selection problem of small hydropower plants in turkey international journal of the analytic hierarchy process 222 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.198 figure 4. the structure of the ahp model in the case study calculations for the ahp model can be performed on some specific ahp software such as the decision lens, expert choice, and super decisions. in addition to these choices of software, spreadsheet software such as microsoft excel and apache openoffice calc can be used to formalize the ahp models. moreover, any mathematical calculation and numerical computation software can be used to code for these kinds of studies such as octave, r, and scilab. in the current study, some of the calculations were performed on the spreadsheet software and some were performed with the help of super decisions (see figure 5 for the screen river basin (subjective criteria) catchment area (objective criteria, km2) project runoff (objective criteria, hm3) conveyance structure (subjective criteria) net head (objective criteria, m) flow rate (objective criteria, m3/s) firm energy (objective criteria, gwh) secondary energy (objective criteria, gwh) investment cost (objective criteria, usd) community attitude (subjective criteria) transportation (subjective criteria) topography (subjective criteria) geology (subjective criteria) security conditions (subjective criteria) terrorism conditions (subjective criteria) protected areas (subjective criteria) substation conditions (subjective criteria) essential features of the shpp complementary features of the shpp electricity generation features of the shpp social features of the shpp's site/location essential features of the shpp's site/location aggregate criteria (main criteria) s m a ll h y d ro p o w e r p la n ts i n v e st m e n t s e le c ti o n i n p re -d e v e lo p m e n t in v e st m e n t s ta g e goal alternatives alternative 1 alternative 2 alternative 3 alternative 4 alternative 5 basic criteria (sub-criteria) level 1 level 2 level 3 level 4 ijahp article: saracoglu, mahallesi, caddesi, sokak/an ahp application in the investment selection problem of small hydropower plants in turkey international journal of the analytic hierarchy process 223 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.198 view of the ahp model of this study). the mathematical calculation and numerical computation software were also used whenever the calculations needed to be checked. ijahp article: saracoglu, mahallesi, caddesi, sokak/an ahp application in the investment selection problem of small hydropower plants in turkey international journal of the analytic hierarchy process 224 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.198 figure 5. screen view of the current ahp model on super decisions software (open the electronic supplementary material: ahp model.sdmod) ijahp article: saracoglu, mahallesi, caddesi, sokak/an ahp application in the investment selection problem of small hydropower plants in turkey international journal of the analytic hierarchy process 225 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.198 in the current case, there were two experts who gave their input on the factors and alternatives. one of the experts has a bachelor’s degree in naval architecture and marine engineering, a master’s degree in industrial engineering, a multi-disciplinary doctorial educational background, and also work experience in the shipbuilding and energy industries. the other expert has a bachelor’s and master’s degree in civil engineering (both in design and construction), and practical work experience in the construction and energy industries. hence, the experts were capable of coping with these kinds of problems and their powerful properties (advantage of the multi-disciplinary expert group) and could support each other. the objective factors were not compared due to the alternatives by the experts because their approximate values were taken directly from the information forms. the alternatives were compared only due to the subjective factors on the cognitive, linguistic, or verbal evaluation labels or statements modeled with the help of the fundamental scale of the ahp and the likert type scale (saaty, 1980; saaty 2008; likert, 1932). the original work by likert included a 5 point scale from “strongly agree” to “strongly disagree", extensions included 1 to 7, 1 to 9, and 0 to 4. this scale was as follows: indifference (1), little moderately more important (2), moderately more important (3), little strongly more important (4), strongly more important (5), quite a lot strongly more important (6), very strongly more important (7), little absolutely more important (8) and absolutely more important (9) (see also boone and boone, 2012; costa and correa, 2010; saaty, 1987; silva et al., 2009). figure 6 details the scales and expressions based on saaty and likert ijahp article: saracoglu, mahallesi, caddesi, sokak/an ahp application in the investment selection problem of small hydropower plants in turkey international journal of the analytic hierarchy process 226 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.198 figure 6. the verbal statements and expressions of this study and the fundamental scale of the ahp (scale & expressions based on saaty's and likert's research studies) intensity of importance definition explanation 1 equal importance two elements contribute equally to the objective 3 moderate importance experience and judgment slightly favor one element over another 5 strong importance experience and judgment strongly favor one element over another 7 very strong importance an activity is favored very strongly over another 9 absolute importance the evidence favoring one activity over another is of the highest possible order of affirmation 2, 4, 6, 8 used to express intermediate values the fundamental scale of the ahp the verbal statements & expressions in this study ijahp article: saracoglu, mahallesi, caddesi, sokak/an ahp application in the investment selection problem of small hydropower plants in turkey international journal of the analytic hierarchy process 227 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.198 a b so lu te ly m o re i m p o rt a n t li tt le a b so lu te ly m o re i m p o rt a n t v e ry s tr o n g ly m o re i m p o rt a n t q u it e a l o t st ro n g ly m o re i m p o rt a n t st ro n g ly m o re i m p o rt a n t li tt le s tr o n g ly m o re i m p o rt a n t m o d e ra te ly m o re i m p o rt a n t li tt le m o d e ra te ly m o re i m p o rt a n t in d if fe re n c e li tt le m o d e ra te ly m o re i m p o rt a n t m o d e ra te ly m o re i m p o rt a n t li tt le s tr o n g ly m o re i m p o rt a n t st ro n g ly m o re i m p o rt a n t q u it e a l o t st ro n g ly m o re i m p o rt a n t v e ry s tr o n g ly m o re i m p o rt a n t li tt le a b so lu te ly m o re i m p o rt a n t a b so lu te ly m o re i m p o rt a n t a b so lu te ly m o re i m p o rt a n t li tt le a b so lu te ly m o re i m p o rt a n t v e ry s tr o n g ly m o re i m p o rt a n t q u it e a l o t st ro n g ly m o re i m p o rt a n t st ro n g ly m o re i m p o rt a n t li tt le s tr o n g ly m o re i m p o rt a n t m o d e ra te ly m o re i m p o rt a n t li tt le m o d e ra te ly m o re i m p o rt a n t in d if fe re n c e li tt le m o d e ra te ly m o re i m p o rt a n t m o d e ra te ly m o re i m p o rt a n t li tt le s tr o n g ly m o re i m p o rt a n t st ro n g ly m o re i m p o rt a n t q u it e a l o t st ro n g ly m o re i m p o rt a n t v e ry s tr o n g ly m o re i m p o rt a n t li tt le a b so lu te ly m o re i m p o rt a n t a b so lu te ly m o re i m p o rt a n t 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 river basin 3 catchment area river basin 3 catchment area river basin 1 project runoff river basin 3 project runoff river basin 5 net head river basin 1 net head river basin 5 flow rate river basin 1 flow rate catchment area 5 project runoff catchment area 5 project runoff catchment area 5 net head catchment area 7 net head catchment area 5 flow rate catchment area 5 flow rate project runoff 7 net head project runoff 5 net head project runoff 7 flow rate project runoff 5 flow rate net head 1 flow rate net head 1 flow rate expert 1: the relative importance of the factors with respect to "essential features of the shpp" expert 2: the relative importance of the factors with respect to "essential features of the shpp" the number of pairwise comparisons is 10 the number of pairwise comparisons is 10 the total number of pairwise comparisons is 20 the experts' opinions or judgments were very carefully recorded with the help of the spreadsheet calculation software microsoft excel and apache openoffice calc (see figure 7). figure 7. data of the relative importance of the factors with respect to "essential features of the shpp" by experts on microsoft excel or apache openoffice calc ijahp article: saracoglu, mahallesi, caddesi, sokak/an ahp application in the investment selection problem of small hydropower plants in turkey international journal of the analytic hierarchy process 228 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.198 in all of the pair-wise comparison matrices that were dependent on human cognition and decision, the consistency ratios (look at consistency index: dave, desai, and raval, 2012; kong and liu, 2005) were calculated by super decisions software. whenever the inconsistency (consistency ratio: cr > 0,1, inconsistency < 0,1) was too high, the suggested judgments by super decisions were examined and the evaluations were reviewed to improve and settle the inconsistency value (see figure 8). ijahp article: saracoglu, mahallesi, caddesi, sokak/an ahp application in the investment selection problem of small hydropower plants in turkey international journal of the analytic hierarchy process 229 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.198 0,04 m c 3 m c 1 m c 5 m c 4 0,02 m c 3 m c 1 m c 5 m c 4 0,01 m c 3 m c 1 m c 5 m c 4 0,06 alt2 alt3 alt4 alt5 0,09 alt2 alt3 alt4 alt5 0,06 alt2 alt3 alt4 alt5 mc2 1 2 5 4 mc2 2 2 4 4 mc2 1,41 2,00 4,47 4,00 alt1 7 1 9 9 alt1 5 2 5 5 alt1 5,92 1,41 6,71 6,71 mc3 2 4 4 mc3 3 3 3 mc3 2,45 3,46 3,46 alt2 7 3 3 alt2 5 4 5 alt2 5,92 3,46 3,87 mc1 3 3 mc1 2 2 mc1 2,45 2,45 alt3 5 5 alt3 7 7 alt3 5,92 5,92 mc5 1 mc5 1 mc5 1,00 alt4 1 alt4 1 alt4 1,00 0,00 c 9 0,00 c 9 0,00 c 9 0,09 alt2 alt3 alt4 alt5 0,10 alt2 alt3 alt4 alt5 0,11 alt2 alt3 alt4 alt5 c4 9 c4 9 c4 9,00 alt1 5 9 3 4 alt1 5 9 4 7 alt1 5,00 9,00 3,46 5,29 alt2 5 5 3 alt2 6 4 3 alt2 5,48 4,47 3,00 0,00 c 8 0,00 c 8 0,00 c 8 alt3 9 9 alt3 9 4 alt3 9,00 6,00 c7 9 c7 5 c7 7,00 alt4 3 alt4 5 alt4 3,87 0,05 c 6 c 5 c 3 c 1 0,08 c 6 c 5 c 3 c 1 0,05 c 6 c 5 c 3 c 1 0,08 alt2 alt3 alt4 alt5 0,09 alt2 alt3 alt4 alt5 0,08 alt2 alt3 alt4 alt5 c2 5 5 5 3 c2 5 7 5 3 c2 5,00 5,92 5,00 3,00 alt1 3 5 5 3 alt1 3 5 3 3 alt1 3,00 5,00 3,87 3,00 c6 1 2 1 c6 1 5 1 c6 1,00 3,16 1,00 alt2 2 3 3 alt2 2 3 3 alt2 2,00 3,00 3,00 c5 2 1 c5 5 1 c5 3,16 1,00 alt3 3 3 alt3 5 5 alt3 3,87 3,87 c3 2 c3 3 c3 2,45 alt4 1 alt4 1 alt4 1,00 0,11 c 1 6 c 1 7 c 1 2 c 1 1 0,11 c 1 6 c 1 7 c 1 2 c 1 1 0,10 c 1 6 c 1 7 c 1 2 c 1 1 0,07 alt2 alt3 alt4 alt5 0,10 alt2 alt3 alt4 alt5 0,08 alt2 alt3 alt4 alt5 c13 3 5 5 4 c13 4 5 7 5 c13 3,46 5,00 5,92 4,47 alt1 3 9 5 5 alt1 3 9 7 5 alt1 3,00 9,00 5,92 5,00 c16 5 2 5 c16 7 3 3 c16 5,92 2,45 3,87 alt2 7 3 1 alt2 5 3 3 alt2 5,92 3,00 1,73 c17 9 9 c17 9 9 c17 9,00 9,00 alt3 7 7 alt3 5 7 alt3 5,92 7,00 c12 3 c12 4 c12 3,46 alt4 1 alt4 3 alt4 1,73 0,11 c 1 4 c 1 5 0,11 c 1 4 c 1 5 0,11 c 1 4 c 1 5 0,10 alt2 alt3 alt4 alt5 0,09 alt2 alt3 alt4 alt5 0,08 alt2 alt3 alt4 alt5 c10 5 9 c10 5 9 c10 5,00 9,00 alt1 5 7 3 3 alt1 3 5 3 3 alt1 3,87 5,92 3,00 3,00 c14 5 c14 5 c14 5,00 alt2 3 7 7 alt2 3 5 5 alt2 3,00 5,92 5,92 alt3 5 3 alt3 3 3 alt3 3,87 3,00 0,38 alt2 alt3 alt4 alt5 0,23 alt2 alt3 alt4 alt5 0,26 alt2 alt3 alt4 alt5 alt4 3 alt4 1 alt4 1,73 alt1 2 2 4 4 alt1 1 2 2 2 alt1 1,41 2,00 2,83 2,83 alt2 2 4 4 alt2 1 4 2 alt2 1,41 4,00 2,83 0,09 alt2 alt3 alt4 alt5 0,09 alt2 alt3 alt4 alt5 0,82 alt2 alt3 alt4 alt5 alt3 2 2 alt3 2 2 alt3 2,00 2,00 alt1 3 5 2 3 alt1 5 7 3 3 alt1 3,87 5,92 2,45 3,00 alt4 4 alt4 4 alt4 4,00 alt2 3 5 3 alt2 3 3 3 alt2 3,00 3,87 3,00 alt3 3 3 alt3 5 3 alt3 3,87 3,00 0,09 alt2 alt3 alt4 alt5 0,09 alt2 alt3 alt4 alt5 0,09 alt2 alt3 alt4 alt5 alt4 3 alt4 3 alt4 3,00 alt1 3 4 7 4 alt1 3 4 9 6 alt1 3,00 4,00 7,94 4,90 alt2 4 7 6 alt2 5 9 7 alt2 4,47 7,94 6,48 alt3 5 3 alt3 7 3 alt3 5,92 3,00 alt4 4 alt4 4 alt4 4,00 0,08 alt2 alt3 alt4 alt5 0,00 alt2 alt3 alt4 alt5 0,02 alt2 alt3 alt4 alt5 alt1 3 3 3 3 alt1 1 1 1 1 alt1 1,73 1,73 1,73 1,73 alt2 3 3 3 alt2 1 1 1 alt2 1,73 1,73 1,73 alt3 3 3 alt3 1 1 alt3 1,73 1,73 alt4 1 alt4 1 alt4 1,00 0,09 alt2 alt3 alt4 alt5 0,09 alt2 alt3 alt4 alt5 0,08 alt2 alt3 alt4 alt5 alt1 4 5 5 5 alt1 4 7 5 5 alt1 4,00 5,92 5,00 5,00 alt2 5 3 3 alt2 7 3 3 alt2 5,92 3,00 3,00 alt3 4 4 alt3 4 5 alt3 4,00 4,47 alt4 1 alt4 3 alt4 1,73 expert 1 expert 2 experts' groupexpert 1 expert 2 experts' group goal: shpp investment selection in pre-development investment stage social features of the shpp's site/location community attitude conveyance structure geology complementary features of the shpp electricity generation features of the shpp essential features of the shpp essential features of the shpp's site/location security conditions terrorism conditions topography transportation protected areas river basin substation conditions figure 8. final super decisions' matrix mode data (pair-wise comparison matrix and inconsistency) (after re-evaluations) & the inconsistency in the first and the second evaluations (n/a: not applicable, green font color: 0,000 ≤ inconsistency ≤ 0,100, blue font color: 0,100 < inconsistency ≤ 0,150, red font color: 0,150 < inconsistency) evaluation re-evaluation evaluation re-evaluation goal 0,04452 n/a 0,02286 n/a complementary features of the shpp 0,00000 n/a 0,00000 n/a essential features of the shpp 0,04936 n/a 0,08001 n/a essential features of the shpp's site/location 0,27329 0,11062 0,19093 0,10626 social features of the shpp's site/location 0,53926 0,11260 0,53926 0,11260 community attitude 0,75121 0,38035 0,46596 0,23121 conveyance structure 0,20973 0,08979 0,17873 0,08829 geology 0,07799 n/a 0,00000 n/a protected areas 0,11993 0,09409 0,15279 0,09437 river basin 0,05743 n/a 0,17080 0,09118 security conditions 0,28657 0,08867 0,24015 0,09768 substation conditions 0,10074 0,07518 0,14943 0,08512 terrorism conditions 0,06827 n/a 0,15242 0,09983 topography 0,20706 0,09595 0,08782 n/a transportation 0,15692 0,08552 0,19634 0,09237 expert 1 expert 2 with respect to (comparisons) ijahp article: saracoglu, mahallesi, caddesi, sokak/an ahp application in the investment selection problem of small hydropower plants in turkey international journal of the analytic hierarchy process 230 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.198 despite the fact that the objective of the current study was to reach an inconsistency lower than 0,10 on the pair-wise comparison matrices, it had to be allowed up to 0,15 in the evaluations of the factors and alternatives because of the impossibility of reflecting the experts true beliefs. although the inconsistency was allowed up to 0,15 in this study, the pair-wise comparison matrix of "the relative importance of the alternatives with respect to community attitude" could not be lower than 0,38035 after almost 50 attempts. the pair-wise comparison matrix of the community attitude due to the alternatives was left at this level (see figure 8). the main cause of this was the scale factor (1 to 9, 1 to 7, or 1 to 5) of the current study. if the scale was selected as 1 to 7 or 1 to 5, the inconsistency of the pair-wise matrices would easily be lower than 0,10 in most of the cases. the matrices about the alternatives with respect to objective factors were not analyzed by the consistency ratio, because of their nature, being objective. the evaluations of the experts could not be directly entered into super decisions 2.2 because there is no experts' group option; only one expert's input screen is available. hence, the group decision outcomes were calculated by the geometric mean formula (see formula/equation 2) with microsoft excel and apache openoffice calc according to saaty's recommendation, and entered into the model on super decisions software. "it has been proved that the geometric mean, not the frequently used arithmetic mean, is the only way to do that. if the individuals are experts, they may not wish to combine their judgments but only their final outcomes obtained by each from their own hierarchy. in that case one takes the geometric mean of the final outcomes." (saaty, 2008). 𝐺 (𝑎1, . . , 𝑎𝑛 ) = (∏ 𝑎𝑖 𝑛 𝑖=1 ) 1/𝑛, where (ai > 0) (2) (for extraction of this formula see abramowitz and stegun, 1972; wolfram research, 2014) in the current study, a total of 270 pair-wise comparisons (20, 2, 2, 20, 20, 6, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20 respectively for goal, mc2, mc3, mc1, mc5, mc4, c10, c4, c13, c16, c1, c14, c17, c15, c12, c11) were performed in the first evaluation. there were approximately 266 pair-wise attempts performed (20, 6, 120, 20, 20, 20, 20, 20, 20 respectively for mc5, mc4, c10, c4, c16, c14, c17, c12, c11) in the second evaluation. hence, the researchers and practitioners should take into account this challenging and very time-consuming effort before starting any study of this scope. in the current study, the priorities of the main factors, the sub-factors and the alternatives were normalized by cluster and limiting values from the priorities button on the computations tab of super decisions. the importance of the main factors were respectively the essential features of the shpp's site/location (0,30434), the social features of the shpp's site/location (0,29716), the electricity generation features of the shpp (0,1799), the essential features of the shpp (0,14142) and the complementary features of the shpp (0,07718). the importance of the factors were respectively found as the terrorism conditions (0,21847), the substation conditions (0,17629), the firm energy (0,15741), the investment cost (0,06946), the geology (0,0661), the security conditions (0,06142), the net head (0,04133), the flow rate (0,04007), the river basin (0,03487), the protected areas (0,03275), the secondary energy (0,02249), the transportation (0,01816), ijahp article: saracoglu, mahallesi, caddesi, sokak/an ahp application in the investment selection problem of small hydropower plants in turkey international journal of the analytic hierarchy process 231 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.198 the project runoff (0,01799), the community attitude (0,01727), the topography (0,01105), the conveyance structure (0,00772), and the catchment area (0,00717). when the experts analyzed these results, they concluded that the results were appropriate and reflected the nature and the solution of the current problem. in this study, the ranks and the preference order of the alternatives was respectively found as alternative 1 (0,23871), alternative 5 (0,21654), alternative 3 (0,18563), alternative 2 (0,18500), and alternative 4 (0,17413). these results were also analyzed by the experts and agreed upon as satisfactory results. the findings of the current study recommended to the experts and investors that they should first start investigating alt1 (0,23871) and alt2 (0,21654) shpp investment options in detail because these alternatives had very close priorities compared to the other alternatives (alt3 (0,18563), alt2 (0,18500), and alt4 (0,17413)). these findings provided the opportunity to move on to new investigation steps, such as reviewing the detailed feasibility studies of these two alternatives and verifying and validating the respective data and information on their studies and reports as much as possible. moreover, the negotiations and discussions on the agreements of these shpp investments could be started and performed with the help of this study such as performing the detailed earnings, the detailed costs investigations, the shareholder agreements and the business models for these two shpp investment options. if these shpp options could not be invested in (no go, no green) for any reason, the other options, alternative 3 (0,18563), alternative 2 (0,18500), and alternative 4 (0,17413) could be taken into account for further detailed investigations because of their close priority values. in this study, the sensitivity analysis was performed and presented with the help of super decisions software. one of the sensitivity analyses was the node sensitivity. the colored lines (red: alt1, green: alt2, blue: alt3, orange: alt4, yellow: alt5) showed the priorities of the alternatives according to the priority of the terrorism, the substation, the firm energy, and the investment cost (see figure 9) (the x-axis of the priority of parameter value: the terrorism conditions, the substation conditions, the firm energy, the investment cost). the black dots on the screen view of the super decisions software indicated the synthesized priorities of the alternatives for a priority of 0,7 0,8 for the terrorism conditions, 0,5 0,6 for the substation conditions, 0,8 0,9 for the firm energy, 0,9 for the investment cost. the alt1 was almost the same as the priority of the terrorism conditions increased from 0 to 1. the alt5 increased its preferability as the priority of the terrorism conditions increased from 0 to 1. however, the preference of the alt1 on the terrorism conditions never changed as the priority of terrorism conditions increased from 0 to 1. similar results were gathered for the other three factors as presented in figure 9. the super decisions model file of this study has been supplied as electronic supplementary material (open ahp model.sdmod). ijahp article: saracoglu, mahallesi, caddesi, sokak/an ahp application in the investment selection problem of small hydropower plants in turkey international journal of the analytic hierarchy process 232 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.198 figure 9. the sensitivity analysis (node sensitivity) for the first four highest priority factors (top left: the terrorism conditions (0,21847), top right: the substation conditions (0,17629), bottom left: the firm energy (0,15741), bottom right: the investment cost (0,06946)) by setting the sensitivity parameter to smart p0 (computations>influence/sensitivity>options> smart p0) and selecting the computations tab and the node sensitivity on the super decisions software. the other sensitivity analysis was the what-if sensitivity analysis for the hierarchies. the colored lines (red: alt1, blue: alt2, black: alt3, green: alt4, yellow: alt5) showed the priorities of the alternatives according to the priority of the complementary features of the shpp, the electricity generation features of the shpp, the essential features of the shpp, the essential features of the shpp's site/location and the social features of the shpp's site/location (see figure 10 respectively top left, top middle, top right, bottom left, and bottom right) (the x-axis of the priority of the parameter value of the 1 st other node). the black dotted line on the screen view of the super decisions software indicated the priority of the 1 st other node and the interceptions with the slanted lines indicated the priorities of the alternatives. the priorities of the alternatives for the original synthesized values were found by moving and dragging the dotted line to the priorities of the alternatives presented in the previous paragraphs. ijahp article: saracoglu, mahallesi, caddesi, sokak/an ahp application in the investment selection problem of small hydropower plants in turkey international journal of the analytic hierarchy process 233 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.198 the alt1 decreased its preferability as the priority of the complementary features of the shpp increased from 0 to 1. the alt5 increased its preferability under the same conditions. although the alt1 was selected on the original synthesized value of 0,07718 for the complementary features of the shpp, the preference of the alt1 decreased and at the same time the preference of the alt5 increased as the priority of the complementary features of the shpp increased from 0 to 1. at about the synthesized value of 0,22 of this main factor, the alt5 had the first rank with 0,227 priority value. similar findings were gathered for the other three alternatives as presented in the figure 10. the super decisions model file of this study has been supplied as electronic supplementary material (open ahp model.sdmod). figure 10. the sensitivity analysis (what-if sensitivity for hierarchies) for the "goal: small hydropower plants investment selection in pre-development investment stage" by selecting the parameter type as "supermatrix", wrt node ("with respect to" node) as "small hydropower plants investment selection in predevelopment investment stage", 1st other node as "complementary features of the shpp" (0,07718) (top left), "electricity generation features of the shpp" (0,1799) (top middle), "essential features of the shpp" (0,14142) (top right), "essential features of the shpp's site/location" (0,30434) (bottom left), "social features of the shpp's ijahp article: saracoglu, mahallesi, caddesi, sokak/an ahp application in the investment selection problem of small hydropower plants in turkey international journal of the analytic hierarchy process 234 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.198 site/location" (0,29716) (bottom right), start as "0,0001", end as "0,99990000000000001", steps as "7" (open the electronic supplementary material to see these inputs and screen views: ahp model.sdmod). 4. conclusions, future applications and research in this paper, the ahp approach and model was applied to determine a solution for an investment selection problem in the energy industry in the turkish shp electricity generation market. the advantages of the ahp method, such as the ease of taking and expressing an expert’s opinion were once more very well observed during the current case study. on the other hand, the disadvantages of the ahp method such as the amount of the pair-wise activities and effort required for the number of alternatives or factors was also an issue in the current case study. the subjective and objective factors were selected by the experts without using any sophisticated method, but mainly due to the data and information availability for each factor in this case study. in general, while selecting the subjective factors, the experts should keep in mind the most challenging issues during the pre-construction, construction and operation phases of the shpps in turkey. the objective factors are related with other scientific areas and methodologies. for instance, the rainfall data should be taken from statistical data tables of the metrological stations. the data and information of the shpp for this aspect cannot be calculated by the experts, but can only be compared with others on the basis of the rough hard data sets. the design of the shpp cannot be revised based on this study, because this scientific area is not in the scope of these kinds of problems. however, the compression of the main characteristics and structures of the shpps can be performed with the help of the subjective and objective factors. the mathematics of the ahp method are not too difficult to understand and adopt. henceforth, the researchers and even the practitioners can easily solve their ahp models. the ahp models can be solved with either the help of computer software or even by hand. moreover, there are several custom written and off-the-shelf software packages for the ahp such as super decisions software 2.2. during this study, one of the crucial observations was about the mental process of the human nature. the experts expressed that the decisions on the factors and the alternatives were often affected by other internal and external factors during each evaluation. henceforth, the evaluations could not be perfect. in the shp sector, the investors' happiness and satisfaction with their shpp investments is very crucial, not only for the attention, and interest on the shpp investments, but also for the health, the economic growth (real, not fictitious and financial markets) and the wealth of the world. henceforth, the increasing number of the research studies and analysis based on the ahp method on the shpp investments will inevitably improve our perception and understanding of the nature of the shpp investments and shpp investors. this study was conducted in a specific setting in the shpp industry in turkey, therefore the results cannot be directly applied to any other situation. ijahp article: saracoglu, mahallesi, caddesi, sokak/an ahp application in the investment selection problem of small hydropower plants in turkey international journal of the analytic hierarchy process 235 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.198 references abramowitz, m. & stegun, i.a. (1972). handbook of mathematical functions with formulas, graphs, and mathematical tables. new york: dover. acm digital library (2015). http://dl.acm.org/ accessed on 04/04/2015. asce online research library (2015). http://ascelibrary.org/ accessed on 04/04/2015. bajaj, j.l., mukherjee, n., & kulkarni, a.v. (2007). grid-connected small hydropower (shp) development: regulatory issues and challenges. international conference on small hydropower, hydro sri lanka, 22-24. beraković, b., pavlin, z., & štefanac, s. (2009). small hydro–a part of water management. international symposium on water management and hydraulic engineering, ohrid, macedonia, 1-5 september 2009. boone, h.n., & boone, d.a. (2012). analyzing likert data. journal of extension, 2tot2, 50(2). cambridge journals online (2015). http://journals.cambridge.org/ accessed on 04/04/2015. chandran, v.g.r., sharma, s., & madhavan, k. (2010). electricity consumption-growth nexus: the case of malaysia. energy policy, 38(1), 606–612. cook, p. (2011). infrastructure, rural electrification and development. energy for sustainable development, 15(3), 304–313. costa, h.g., & correa, p.s. (2010). construction of an ahp-based model to catch criteria weights in post-occupancy evaluation. international journal of the analytic hierarchy process, 2(1), 1936–6744. coyle, g. (2004). the analytic hierarchy process (ahp). usa: pearson education limited. dalgaard, c.j., & strulik, h. (2011). energy distribution and economic growth. resource and energy economics, 33(4), 782–797. dave, h.k., desai, k.p. & raval, h.k. (2012). a decision support system for tool electrode selection for electro discharge machining process using the analytic hierarchy process. international journal of the analytic hierarchy process, 4(2), 89–103. demirbas, a. (2002). turkey’s energy overview beginning in the twenty-first century. energy conversion and management, 43, 1877–1887. directory of open access journals (doaj) (2015). http://doaj.org/ accessed on 04/04/2015. http://dl.acm.org/ http://ascelibrary.org/ http://journals.cambridge.org/ http://doaj.org/ ijahp article: saracoglu, mahallesi, caddesi, sokak/an ahp application in the investment selection problem of small hydropower plants in turkey international journal of the analytic hierarchy process 236 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.198 dragu, c., sels, t., & belmans, r. (2010). small hydro power-state of the art and applications. esat-elen, energy institute, leuven, belgium. dsi (the general directorate of state hydraulic works) (2013). http://www.dsi.gov.tr accessed in 2013. ec (european commission) (2014). http://ec.europa.eu/research/energy/eu/index_en.cfm?pg=research-hydropower accessed on 02/09/2014. eia (u.s. energy information administration) (2013). http://www.eia.gov/tools/glossary/ accessed on 05/09/2013. eliasson, j., & ludvigsson, g. (2000). load factor of hydropower plants and its importance in planning and design. 11 th international seminar on hydro power plants, hydros future in changing markets, 15-17 november 2000, university of technology, vienna, austria. emerald insight (2015). http://www.emeraldinsight.com/ accessed on 04/04/2015. emra (republic of turkey energy market regulatory authority) (2013). http://www.emra.gov.tr/ or http://www.epdk.gov.tr. erec (european renewable energy council) (2012). ren21 renewables 2012 global status report. european focus, paris, france. esha (european small hydropower association) (2004). guide on how to develop a small hydropower plant. belgium. esha (2005). brochure on environmental integration of small hydropower plants. belgium. esha (european small hydropower association), sero (swedish renewable energy association), & lha (lithuanian hydropower association). (2008). small hydro energy efficient promotion campaign action (sherpa) project, strategic study for development of small hydropower in the european union brochure. belgium. front line (2005). protection manual for human rights defenders. ireland: the international foundation for the protection of human rights defenders. gawlik, r. (2008). preliminary criteria reduction for the application of analytic hierarchy process method. munich personal repec archive mpra, 45348. google scholar (2015). http://scholar.google.com.tr/ accessed on 04/04/2015. hall, d.g. (2003). estimation of economic parameters of u.s. hydropower resources. idaho falls, idaho: idaho national engineering and environmental laboratory. http://ec.europa.eu/research/energy/eu/index_en.cfm?pg=research-hydropower http://www.eia.gov/tools/glossary/ http://www.emeraldinsight.com/ http://www.emra.gov.tr/ http://scholar.google.com.tr/ ijahp article: saracoglu, mahallesi, caddesi, sokak/an ahp application in the investment selection problem of small hydropower plants in turkey international journal of the analytic hierarchy process 237 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.198 iha (international hydropower association), international commission on large dams, international energy agency, canadian hydropower association. (2000). hydropower and the world's energy future, the role of hydropower in bringing clean, renewable, energy to the world, paris, france. international journal of the analytic hierarchy process (ijahp) (2015). http://www.ijahp.org/index.php/ijahp accessed on 04/04/2015. investopedia us (2014). http://www.investopedia.com/ accessed on 06/09/2014. iwa (iwa water wiki) (2013). http://www.iwawaterwiki.org/ accessed on 05/09/2013. kong, f., & liu, h. (2005). applying fuzzy analytic hierarchy process to evaluate success factors of e-commerce. international journal of information and systems sciences, 1(3-4), 406–412. kruger, h.a., & hattingh, j.m. (2006). a combined ahp-gp model to allocate internal auditing time to projects. orion, 22 (1), 59–76. kurien, j., & sinha, a.k. (2006). small hydro power nabard's involvement and future outlook. national bank for agriculture and rural development mumbai, himalayan small hydropower summit, october 12-13, 2006, dehradun. lako, p. (2003). hydropower development with a focus on asia and western europe. overview in the framework of vleem 2, ecn and verbund plan, the netherlands. likert, r. (1932). a technique for the measurement of attitudes (archives of psychology, no: 140). new york, the science press. miller, g.a., (1956). the magical number seven, plus or minus two: some limits on our capacity for processing information. the psychological review, 63, 81–97. mladineo, n., margeta, j., brans, j.p., & mareschal, b. (1987). multicriteria ranking of alternative locations for small scale hydro plants. european journal of operational research, 31(2), 215–222. moreira, j.r., & poole, a.d. (1993). hydropower and its constraints. in: johansson t.b., h. kelly, a.k.n. reddy and r.h. williams (eds). renewable energy: sources for fuels and electricity, island press, washington d.c., usa, pp. 73-120. saaty, t.l. (1980). the analytic hierarchy process. new york, usa: mcgraw-hill. saaty, t.l. (1987). the analytic hierarchy process-what it is and how it is used. mathematical modelling, 9(3–5), 161–176. saaty, t.l. (1990). how to make a decision: the analytic hierarchy process. european journal of operational research, 48, 9–26. http://www.ijahp.org/index.php/ijahp http://www.iwawaterwiki.org/ ijahp article: saracoglu, mahallesi, caddesi, sokak/an ahp application in the investment selection problem of small hydropower plants in turkey international journal of the analytic hierarchy process 238 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.198 saaty, t.l. (2008). decision making with the analytic hierarchy process. international journal of services sciences, 1(1), 83–98. saaty, t.l. (2010). the eigenvector in lay language. international journal of the analytic hierarchy process, 2(2), 163–169. saaty, t.l. (2011). the serious omission of comparisons in aristotle’s laws of thought. international journal of the analytic hierarchy process, 3(1), 180–181. saracoglu, b.o. (2015). an experimental research study on the solution of a private small hydropower plant investments selection problem by electre iii/iv, shannon’s entropy, and saaty’s subjective criteria weighting. advances in decision sciences, 2015, article id 548460. doi:10.1155/2015/548460. saxena, p. (2007). small hydro development in india. international conference on small hydropower, hydro sri lanka, 1–6. science direct (2015). http://www.sciencedirect.com/ accessed on 04–05/04/2015. shiffrin, r.m., nosofsky, r.m., (1994). seven plus or minus two: a commentary on capacity limitations. psychological review, 101(2), 357–361. silva, a.c., nascimento, l.p., ribeiro, j.r & belderrain, c.n. (2009). anp and ratings model applied to ssp. proceedings of the international symposium on the analytic hierarchy process, pittsburgh, pa, 10, 1–11. taylor & francis journal (2015). http://www.tandfonline.com/ accessed on 05/04/2015. tcmb (the central bank of the republic of turkey) (2013). http://www.tcmb.gov.tr accessed on 08/09/2013. topcu, y.i., & ulengin, f. (2004). energy for future: an integrated decision aid for the case of turkey. energy, 29, 137–154. united nations (2013). http://www.un.org/en/terrorism/index.shtml accessed on 09/09/2013. unodc (united nations office on drugs and crime) (2013). http://www.unodc.org/ accessed in 2013. usgs (u.s. geological survey) (2013). http://www.usgs.gov/ accessed on 05/09/2013. wikipedia (2014). http://en.wikipedia.org/wiki/geographical_regions_of_turkey accessed on 02/09/2014. wolfram research (2014). http://mathworld.wolfram.com/geometricmean.html accessed on 2/09/2014. http://www.sciencedirect.com/ http://www.tandfonline.com/ http://www.tcmb.gov.tr/ http://www.un.org/en/terrorism/index.shtml http://www.usgs.gov/ ijahp article: saracoglu, mahallesi, caddesi, sokak/an ahp application in the investment selection problem of small hydropower plants in turkey international journal of the analytic hierarchy process 239 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.198 world bank (2013). http://data.worldbank.org accessed on 05/09/2013 world bank (2015). http://data.worldbank.org/indicator/sp.pop.grow/countries?page=1&display=default & http://data.worldbank.org/indicator/ny.gnp.mktp.kd.zg/countries/1w?display=defa ult accessed on 04/03/2015 (added during 2 nd revision of this manuscript) yoon, k.p.; & hwang, c.l. (1995). multiple attribute decision making an introduction. london, uk: sage publications. http://data.worldbank.org/ http://data.worldbank.org/indicator/sp.pop.grow/countries?page=1&display=default%20 http://data.worldbank.org/indicator/ny.gnp.mktp.kd.zg/countries/1w?display=default http://data.worldbank.org/indicator/ny.gnp.mktp.kd.zg/countries/1w?display=default microsoft word 4andreichicovaarticle_pp45_60_03212009 ijahp article: andreichicova, radyshevskaya/applying anp to researching oral health international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 45 an application of the analytic network process to researching oral health o.n. andreichicova russian state university of innovation technologies and business moscow, russia e-mail: alexandrol@mail.ru t.n. radyshevskaya volgograd state medical university volgograd, russia e-mail: olga_andr@hotmail.com abstract the article describes applying the analytic network process to model the influence of various factors on oral health status. the network model, based on the knowledge of experts, enables us to find the factors which have a profound effect on the health status of the organism’s systems under conditions of interdependence. the outcomes obtained are in conformity with observed data. the procedure described here can be used for research and forecasting the health status of the human organism in general. keywords: anp, caries, dentistry, forecasting, oral health 1. introduction the analytic network process developed by thomas l. saaty (saaty t.l., 2001) can be successfully applied to forecasting a functional state of various forms of human physiological activity using the knowledge accumulated by skilled doctors and physiologists. in this paper we consider the problem of modeling oral health, taking into account a number of external and internal factors. the forecasting of the onset and development of dental diseases is an urgent problem in modern dentistry as the incidence of dental caries and periodontium pathologies is very high among the human population and their clinical course is progressive. usually forecasting dental disease is based on clinical presentations of the action of various adverse factors on hard tooth tissues and on the periodontium. as is known, caries and inflammatory diseases of periodontium are caused by both local and general risk factors (leont’ev v.k., shestakov v.t. and voronin v.f., 2003; rybakov a.i. and chelidze l.n., 1990). the well known general risk factors are: content of fluorine in water, the daily diet, stresses, and somatic diseases, which give rise to abnormalities of endocrine regulation, immunity and metabolism. rob typewritten text rob typewritten text http://dx.doi.org/10.13033/ijahp.v1i1.7 rob typewritten text ijahp article: andreichicova, radyshevskaya/applying anp to researching oral health international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 46 in this research we take into consideration the following local factors: the presence of carbohydrate food debris, the presence of microorganisms in dental deposits, and abnormal composition and properties of the oral fluid. besides these factors the resistance of hard tooth tissues (full value of structure, chemical composition, and genetic code) is important to predict the onset of caries. the analytic network process (anp) proposes constructing a qualitative model that describes the influence of external factors upon the system under study and the interdependence of the system’s elements. such a model can be built using the knowledge of highly-skilled experts and it represents the linguistic description of basic components, elements and factors, as well as describing the sense and direction of the flow of influence in the system being researched. the influence of different factors upon the system’s elements (or the mutual influence of internal elements) can be semantically ambiguous, therefore in the initial research stage one should precisely formulate the main goal in terms of which the basic categories of the task will be determined: criteria, components or clusters, elements, judgments. 2. constructing the network model the main goal in this research is to find the factors that make the greatest impact on oral health. the results of scientific research, published scientific works (rybakov a.i. and chelidze l.n., 1990; eriksen h.m. and bjertness e., 1991; hollister m.c. and weintraub j.a., 1993; leont’ev v.k., shestakov v.t. and voronin v.f., 2003; andreichicova o.n. and radyshevskaya t.n., 2003), as well as the experience of applied medicine were used to single out 15 major factors influencing oral health. these factors were grouped into the following 7 clusters. 1) life-support conditions this term means the external influence of environmental factors upon a human organism, in particular upon oral health. the most important factors among them are represented by the following elements of this cluster: 1.1. nutrition compound (foodstuff composition; amount of consumed carbohydrates; softness or hardness of food); 1.2. ecology of life (chemical composition of air and water, including content of fluorine in water; geographic and climatic conditions of existence); 1.3. social conditions (sanitary conditions depending on social environment and personal income). 2) oral hygiene the main external factors determining oral hygiene are: 2.1. skills of proper tooth brushing; 2.2. regularity of dental inspections; 2.3. quality of hygienic means used. training patients to brush teeth properly and select hygienic means for oral cavity care decreases the risk of dental diseases. ijahp article: andreichicova, radyshevskaya/applying anp to researching oral health international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 47 3) status of dental health this cluster characterizes the current state of the oral cavity conditioned by all external and internal influences, as well as by the case history of a patient. there are two main elements here: 3.1. state of teeth; 3.2. state of the periodontium. the state of teeth (presence of carious cavities, poor-quality tooth stoppings and dental deposits) in many respects determines the state of the periodontium (onset and progress of gingivitis and periodontitis). besides, the onset of periodontium diseases can be associated with dental anomalies and deformation, errors during prosthetic alignments, and also with dental traumas. inflammatory diseases of periodontium influence both the general health of the human organism and the state of teeth. a formation of bone recesses at periodontitis finally results in loosening of teeth, while etiological microbe agent promotes caries of hard tooth tissues. 4) general health status this cluster characterizes the influence of basic parameters of human health on the state of oral cavity. among these parameters we consider the following elements: 4.1. hormonal status; 4.2. age; 4.3. level of immune reactivity. we suppose these parameters of general health have an important influence on oral health and no further elaboration is necessary. 5) type of blood circulation this is one of the major internal factors influencing dental and periodontium health (rybakov a.i. and chelidze l.n., 1990). we shall examine the overall influence of this factor without detailing it with a separate subset of elements and we shall presume that various types of blood circulation (eukinetic, hyperkinetic, hypokinetic) influence the state of other systems in different ways. 6) the type of vegetative regulation this includes vascular tone, secretions of salivary glands, and also blood supply to teeth and periodontium. this cluster, as well the previous one (a type of blood circulation), is considered at a generalized level, i.e. it is represented by a single element. the influence of the vegetative regulation type (sympathotonic, normotonic and vagotonic) upon the general health of the human organism as a whole and, in particular, upon oral health is beyond question and it is described in a number of scientific works (rybakov a.i. and chelidze l.n., 1990; eriksen h.m. and bjertness e., 1991; hollister m.c. and weintraub j.a., 1993). 7) salivation system this cluster is detailed the following set of elements whose influence upon oral health is well known (borovskiy e.v. and leont’ev b.k., 1991; leont’ev v.k., shestakov v.t. and voronin v.f., 2003): 7.1. saliva secretion rate; ijahp article: andreichicova, radyshevskaya/applying anp to researching oral health international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 48 7.2. salivation character (qualitative and quantitative saliva composition). such factors as ethnic features, professional insalubrities, harmful habits, etc., were not taken into account in the model because their influence is insignificant in comparison with the factors described above. after identifying basic elements and clusters it is necessary to ascertain the relations between them. the graph of the clusters’ interactions is shown in figure 1, where the arrows specify influence directions. the arrow from one cluster to another means that all or some elements in the first cluster influence all or some elements in the second cluster. feedback loops are internal cycles which correspond to mutual influence between elements in the same cluster. in addition to internal cycles, there can be external cycles formed by connections between different clusters. the examples of external cycles in figure 1 are the closed contours formed by the clusters 3 4 3, 4 6 4, 4 6 5 7 3 4, 4 7 3 – 4, etc. let us consider in detail the meaning and direction of the relations between the clusters in figure 1. 5. type of blood circulation 6. type of vegetative regulation 4. general health status 4.1. hormonal status 1. life-support conditions 1.1. nutrition compound ecology of life social conditions 2. oral hygiene 2.2. regularity of dental inspections 2.3. quality of hygienic means age level of immune reactivity 3. status of dental health 3.1. s of teethtate s of periodontiumtate 7. salivation system 7.1. saliva secretion rate 7.2. salivation character 2.1. skills of proper teeth brushing figure 1 network model for researching oral health. the elements in the first cluster life-support conditions, noticeably influence cluster 3, status of dental health, and cluster 4, general health status. besides that, to a lesser degree, they influence each other, and some of them, social conditions and ecology of ijahp article: andreichicova, radyshevskaya/applying anp to researching oral health international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 49 life, influence the elements of cluster 2, oral hygiene. the most important influences of the first cluster’s elements are the following: social conditions → nutrition compound, social conditions → quality of hygienic means, nutrition compound → state of teeth, ecology of life → level of immune reactivity. it is obvious that the elements of cluster 2, oral hygiene, influence cluster 3, status of dental health. a feedback loop in the second cluster is our assumption that regularity of dental inspections influences other elements of this cluster. in cluster 3, status of dental health, there is the state of teeth element which includes presence of cavities, fillings, and teeth congestion, that in many respects influences the state of periodontium element which, in turn, influences the state of teeth element, i.e., the elements of this cluster influence each other. it is clear that the third and fourth clusters are interdependent, as elements in cluster 3 such as insufficient mastication of food, presence of foci of chronic infections, and inflammations in the oral cavity influence cluster 4, general health status. in turn, the elements of general health status: hormonal status, age and level of immune reactivity; influence elements in cluster 3 status of dental health. general health status besides directly affecting status of dental health, marginally influences status of dental health through cluster 7, salivation system. the type of vegetative regulation depends on the elements in cluster 4, because it can change with age, after sustained diseases and stresses. the cluster type of blood circulation influences general health status, status of dental health (particularly state of periodontium) and salivation system, as the cardiovascular system provides a necessary level of vital activity and is an indicator of the human organism’s adaptability. the cluster type of vegetative regulation influences type of blood circulation, salivation system, status of dental health and general health status. the intensity of these influences may be different, but the fact of their existence is beyond question. the elements of the seventh cluster salivation system mutually influence each other and also status of dental health. the next step in this research is to find out the relative intensity of influence for all elements examined in the system. further, the finished model will be used for calculating limiting priorities of the elements. these priorities characterize the overall contribution of every element in the system being formulated. 3. judging the relative intensity of influence for clusters and elements in the anp intensity of influence is estimated by an expert or several experts using the pairwise comparison process and the fundamental scale (saaty t.l., 2001). the paired comparison technique is the most universal method of measurements, as it can be applied in the absence of any scales and standards, and is particularly useful in determining ijahp article: andreichicova, radyshevskaya/applying anp to researching oral health international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 50 measures for intangible attributes. in comparing two objects with respect to a common attribute (criterion, property), the expert estimates how much one object is preferred over another, choosing a suitable representation from the fundamental scale. if there are several parties participating in making judgments they can come to consensus on a judgment or else the geometric mean of their individual judgments can be used. the comparisons for all pairs of homogeneous objects with respect to a criterion are entered in a reciprocal matrix of pairwise comparisons. there are (n2-n)/2 judgments required to complete an n × n pairwise comparison matrix. the right principal eigenvector is calculated for this matrix, which results in a 1 × n vector of relative priorities. this essentially converts the original (n2-n)/2 judgments to a vector of n numbers from the continuous interval [0, 1] that expresses the relative priority of the elements that were compared. the associated principal eigenvalue also gives a measure of the consistency of the expert’s judgments (saaty t.l., 2001). for our model (see figure 1), it was necessary to enter judgments in 35 pairwise comparisons matrices for elements and 7 matrices for clusters. the latter were used to calculate the clusters’ weight factors. the connections in the network model in figure 1 are represented by the following matrix: clusters 1 2 3 4 5 6 7 1 1 0 0 0 0 0 0 2 1 1 0 0 0 0 0 3 1 1 1 1 1 1 1 b = 4 1 0 1 1 1 1 0 5 0 0 0 0 0 1 0 6 0 0 0 1 0 0 0 7 0 0 0 1 1 1 1 the unit elements in the matrix b signify that the cluster specified at the top (column identifier) influences the cluster specified to the left (row identifier). for each column of this matrix the matrix of pairwise comparisons of corresponding clusters was filled. in filling these cluster comparison matrices, the experts answered a question: “what of two compared clusters influences the given cluster more, and how much more?” or the question “which of the clusters in a pairwise comparison contributes more to the given cluster, and how much more?” in answering these questions, the experts again used the fundamental scale. for example, influence of the first cluster life-support conditions upon other clusters to which it is connected (the non-zero entries in column 1 of the matrix above) is represented by the following matrix: 1 1 2 3 4 w 1 1 1 1/3 1/5 0.100 a1 = 2 1 1 1/3 1/5 0.100 3 3 3 1 1/2 0.283 4 5 5 2 1 0.517 the elements of the first cluster influence each other and also the elements of the second, third and fourth clusters (see the first column in the matrix b). let us consider the explanation of the expert’s results w for the matrix a1 containing the clusters pairwise ijahp article: andreichicova, radyshevskaya/applying anp to researching oral health international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 51 comparisons obtained by answering the question “what of two clusters does the first cluster, life-support conditions, influence more, and how much more?” we can see that the first cluster’s elements have a significant impact on the fourth cluster and somewhat less impact on the third cluster. the second column of the matrix b testifies that the cluster oral hygiene influences itself and the third cluster, thus the pairwise comparisons matrix a2 is: 2 2 3 w a2 = 2 1 1/5 0.167 3 5 1 0.833 we can see that the elements of oral hygiene (cluster 2) influence elements in status of dental health (cluster 3) more than they influence each other. five other pairwise comparison matrices for clusters were similarly filled. the priority vectors thus obtained were used in forming the following matrix of clusters’ weight factors: 1 2 3 4 5 6 7 1 0.1 0 0 0 0 0 0 2 0.1 0.167 0 0 0 0 0 3 0.283 0.833 0.8 0.125 0.143 0.09 0.5 d = 4 0.517 0 0.2 0.5 0.286 0.051 0 5 0 0 0 0 0 0.606 0 6 0 0 0 0.125 0 0 0 7 0 0 0 0.25 0.571 0.253 0.5 the next step is filling out the pairwise comparisons matrices for the elements of the clusters and calculating their priorities. let us consider this procedure using an example from the first cluster, which contains the following elements: 1.1. nutrition compound; 1.2 ecology of life; 1.3 social conditions. these elements influence each other and also the elements of the second, third and fourth clusters. they do not all influence elements in the other clusters. for example, the nutrition compound element in the lifesupport conditions cluster does not influence ecology of life and social conditions, however the last two factors influence other elements in the first cluster. we have the following pairwise comparison matrices for the first cluster’s elements: 1.2 1.1 1.3 w 1.3 1.1 1.2 w c112 = 1.1 1 5 0.833 c113 =1.1 1 5 0.833 1.3 1/5 1 0.167 1.2 1/5 1 0.167 these matrices show that ecology of life influences nutrition compound much more strongly than social conditions, and social conditions influences nutrition compound more strongly than ecology of life. the second cluster’s elements are influenced by all elements of the first cluster except nutrition compound. thus, ecology of life (1.2) influences regularity of dental inspections (2.2) much more strongly than other elements of the second cluster. the ijahp article: andreichicova, radyshevskaya/applying anp to researching oral health international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 52 social conditions (1.3) factor influences quality of hygienic means (2.3) to a greater extent than the others and hardly influences skills of proper tooth brushing (2.1). these influences are represented by the following matrices: 1.2 2.1 2.2 2.3 w 1.3 2.1 2.2 2.3 w c212 = 2.1 1 1/7 1 0.111 c213 = 2.1 1 1/4 1/8 0.077 2.2 7 1 7 0.778 2.2 4 1 1/2 0.308 2.3 1 1/7 1 0.111 2.3 8 2 1 0.615 nutrition compound (1.1) and ecology of life (1.2) influence the elements of the third cluster (status of dental health). the following matrices contain estimations of intensity of their relative influence: 1.1 3.1 3.2 w 1.2 3.1 3.2 w c311 = 3.1 1 5 0.833 c312 = 3.1 1 3 0.75 3.2 1/5 1 0.167 3.2 1/3 1 0.25 we can see from these matrices that the elements of the first cluster influence state of teeth more strongly than state of periodontium. all elements in the first cluster influence the elements of the fourth cluster general health status. these influences are shown in the following matrices: 1.1 4.1 4.2 4.3 w 1.2 4.1 4.2 4.3 w 1.3 4.1 4.2 4.3 w 4.1 1 5 1/2 0.319 4.1 1 3 1/3 0.231 4.1 1 3 1/2 0.3 c411= 4.2 1/5 1 1/9 0.066 c412 =4.2 1/3 1 1/9 0.077 c413 =4.2 1/3 1 1/6 0.1 4.3 2 9 1 0.615 4.3 3 9 1 0.692 4.3 2 6 1 0.6 the matrices above show that nutrition compound (1.1) exerts the most important influence on level of immune reactivity (4.3), lesser on hormonal status (4.1) and nearly does not influence age (4.2). ecology of life (1.2) and social conditions (1.3) influence the elements of the fourth cluster in a similar way. the pairwise comparison matrices for estimating the influence of the elements from other clusters were similarly formed. the priority vectors computed for these matrices were put in the unweighted supermatrix for this network task, which is shown in table 1. the experts’ judgments in all matrices had a good consistency (the index of inconsistency did not exceed 0.02 while the maximum suggested allowable index of inconsistency is 0.10). table 1 unweighted supermatrix. 1 2 3 4 5 6 7 1.1 1.2 1.3 2.1 2.2 2.3 3.1 3.2 4.1 4.2 4.3 5 6 7.1 7.2 1.1 0 0.833 0.833 0 0 0 0 0 0 1.2 0 0 0.167 1.3 0 0.167 0 2.1 0 0.111 0.077 0 0.75 0.25 0 0 0 0 0 2.2 0 0.778 0.308 0.875 0 0.75 2.3 0 0.111 0.615 0.125 0.25 0 ijahp article: andreichicova, radyshevskaya/applying anp to researching oral health international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 53 3.1 0.833 0.75 0 0.75 0.833 0.833 0 1 0.167 0.5 0.25 0.2 0.25 0.75 0.333 3.2 0.167 0.25 0 0.25 0.167 0.167 1 0 0.833 0.5 0.75 0.8 0.75 0.25 0.667 4.1 0.319 0.231 0.3 0 0.2220.236 0 0.667 0.833 0.236 0.25 0 4.2 0.066 0.077 0.1 0.1110.082 0.25 0 0.167 0.082 0.069 4.3 0.615 0.692 0.6 0.6670.682 0.75 0.333 0 0.682 0.681 5 0 0 0 0 0 1 0 6 0 0 0 1 1 1 0 0 0 7.1 0 0 0 0.25 0.75 0.333 0.25 0.75 0 1 7.2 0.75 0.25 0.667 0.75 0.25 1 0 4. calculation of limiting priorities for the factors when elements in a complex system influence each other, their priorities constantly change, therefore one can say that the task being considered becomes a dynamic one. we are interested in a long-term forecast of oral health and in the limiting result of the mutual influences of the factors being considered. the anp enables us to find the steady limit state for a network model with feedback, and also to calculate limiting priorities for all elements and clusters. if the sums of the elements in every column of a supermatrix are equal to unity, then that the matrix is said to be column stochastic. in order to make the original supermatrix in table 1 stochastic, which is necessary for its convergence, one must multiply its blocks that correspond to the clusters by the cluster’s weight factors shown in the matrix d. the result is the weighted supermatrix shown in table 2. raising this matrix to powers until it converges results in the solution, which is a square matrix with identical columns whose elements do not change as the powers increase. in some cases the limiting matrix cycles among 2 or more states in which case the states are combined to give a square matrix with identical columns. this matrix contains the limiting priorities of the elements. we can see them in table 3 and interpret the factors’ contributions to the main goal. by main goal in an anp model we mean the stated objective of the exercise: to find the factors that make the greatest impact on oral health. unlike in an ahp model where the main goal is the node at the top of the structure, the main goal in an anp model does not appear as a node in the model. rather it is the purpose of the model and judgments in the model must be entered with this kept in mind. 5. results discussion the outcomes shown in tables 2 and 3 and in figures 2 and 3 allow us to conclude the following assertions about the mutual influence of external and internal factors: • life-support conditions have the most important influence on state of teeth, level of immune reactivity and hormonal status. • oral hygiene strongly influences state of teeth and less state of periodontium. • the elements of status of dental health noticeably affect each other. to a lesser degree they influence level of immune reactivity. • hormonal status has the most appreciable influence on level of immune reactivity, salivation character and state of periodontium. ijahp article: andreichicova, radyshevskaya/applying anp to researching oral health international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 54 • age noticeably influences hormonal status, saliva secretion rate, level of immune reactivity and vegetative regulation type. • the level of immune reactivity appreciably affects hormonal status and salivation character. • type of blood circulation strongly influences salivation character and has somewhat less influence on level of immune reactivity and state of periodontium. • type of vegetative regulation governs type of blood circulation. besides, it noticeably influence saliva secretion rate. • the elements of the salivation system noticeably affect each other. they also have some influence on status of dental health such that a saliva secretion rate affects state of teeth, and salivation character – state of periodontium. table 2 weighted supermatrix 1 2 3 4 5 6 7 1.1 1.2 1.3 2.1 2.2 2.3 3.1 3.2 4.1 4.2 4.3 5 6 7.1 7.2 1.1 0 0.083 0.117 0 0 0 0 0 0 1.2 0 0 0.023 1.3 0 0.017 0 2.1 0 0.011 0.011 0 0.125 0.042 0 0 0 0 0 2.2 0 0.078 0.043 0.146 0 0.125 2.3 0 0.011 0.085 0.021 0.042 0 3.1 0.295 0.212 0 0.625 0.694 0.694 0 0.8 0.021 0.063 0.032 0.029 0.083 0.375 0.167 3.2 0.039 0.071 0 0.208 0.139 0.139 0.8 0 0.104 0.063 0.094 0.115 0.067 0.125 0.333 4.1 0.206 0.119 0.216 0 0.044 0.047 0 0.333 0.416 0.067 0.013 0 4.2 0.043 0.04 0.073 0.023 0.017 0.125 0 0.083 0.023 0.004 4.3 0.397 0.358 0.432 0.133 0.136 0.375 0.167 0 0.195 0.035 5 0 0 0 0 0 0.606 0 6 0 0 0 0.125 0.125 0.125 0 0 0 7.1 0 0 0 0.063 0.187 0.083 0.143 0.19 0 0.5 7.2 0.187 0.063 0.167 0.428 0.063 0.5 0 table 3 limiting priorities of the elements and clusters 1 2 3 4 5 6 7 0 0 0.56 0.237 0.018 0.03 0.155 1.1 1.2 1.3 2.1 2.2 2.3 3.1 3.2 4.1 4.2 4.3 5 6 7.1 7.2 0 0 0 0 0 0 0.276 0.284 0.087 0.032 0.118 0.018 0.03 0.071 0.084 the priorities shown in tables 1 and 2 can be considered as expert estimations of the influence intensity for the set of the examined factors at the present moment. the limiting priorities (see table 3) can be interpreted as predicting the values of the examined ijahp article: andreichicova, radyshevskaya/applying anp to researching oral health international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 55 factors’ contributions on the overall goal of the system, oral health. the factors having high limiting priorities affect the system to a greater extent. in other words, data from table 3 (and the bars in figure 2) correspond to the concentration of the influence in the elements from all the examined factors. it is easy to note that the elements of dental health status make the greatest impact on the system goal, and furthermore, the limiting priority of state of periodontium is a little bit more than the priority of state of teeth. the limiting priorities of the clusters are shown in figure 3, where the striped bars correspond to the integrated contribution of every cluster on the overall system goal. we can see that the contribution of the third cluster (dental health status) exceeds 50%. the third rank among all the elements (see figure 2) is level of immune reactivity, which is an element of the fourth cluster, general health status, which takes the second place among all the clusters (see figure 3). another element of this cluster – hormonal status – has the fourth rank. the element age is the least significant in the fourth cluster; it can be explained by the negligible influence on it of other factors. the salivation system has the third rank among clusters, and its elements are in fifth and sixth places (see figure 2). the salivation character element is slightly more significant than saliva secretion rate. in fourth place among the clusters is type of vegetative regulation followed figure 2 limiting priorities for the elements of the network model by type of blood circulation. the limiting priorities for the elements from the first and the second clusters are equal to zero. this fact can be explained that their elements are not influenced by other elements; therefore there is no accumulation of influence in them. the node corresponding to the first cluster in figure 1 is a source; and the node, which corresponds to the second cluster, is a flow element with a feedback loop. 0 0.05 0.1 0.15 0.2 0.25 0.3 1.1 nutrition compound 1.2 ecology of life 1.3 social conditions 2.1 skills of proper teeth brushing 2.2 regularity of dental inspections 2.3 quality of hygienic means 3.1 state of teeth 3.2 state of periodontium 4.1 hormonal status 4.2 age 4.3 level of immune reactivity 5 blood circulation type 6 vegetative regulation type 7.1 s aliva secretion rate 7.2 salivation character ijahp article: andreichicova, radyshevskaya/applying anp to researching oral health international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 56 figure 3 limiting priorities for the clusters. the first cluster is a powerful source of influence on the other clusters; only 10% of its influence makes up feedback. the contribution of this cluster’s elements to the overall system goal comes through the third and fourth clusters indirectly (see matrix d above). the second cluster gets about 10% of the first cluster’s influence, and only 16.7% of this value comes back through feedback into it. the contribution of the second cluster’s elements on the main goal of oral health has come through the third cluster (dental health status). in order to find out the elements which have the greatest influence on oral health, we inverted this task by transposing the matrix b. to do that it was necessary to fill out a set of new pairwise comparison matrices, asking the experts a question: “which of two analyzed factors (clusters, elements) influence the given factor (cluster or element) more?” for example, “which of the elements of dental health status (state of teeth or state of periodontium) influences level of immune reactivity more?” the supermatrix of the inverted task is shown in table 4. its columns show the influences of the factors specified in rows (at the left) upon the element that is the heading of the column. raising this matrix to high powers enables us to compute the limiting priorities of influence of the examined factors upon all elements of the model. these priorities called the limiting influence priorities are represented in table 5 and in figures 4 and 5. the outcomes of the inverted task reveal the most influential of the factors examined. the obvious leader among them is type of vegetative regulation with a limiting priority of 0.3207, the second is hormonal status (0.218), and the third is age (0.1776). the distribution of the clusters’ integrated priorities is shown in figure 5, where we can see that the greatest influence upon oral health is general health status (cluster 4) followed by the type of vegetative regulation and the type of blood circulation behind it. the influence of external factors (cluster 1 and 2) is less significant and that coincides with up-to-date concepts in dentistry science (eriksen h.m. and bjertness e., 1991; leont’ev v.k., shestakov v.t. and voronin v.f., 2003). ijahp article: andreichicova, radyshevskaya/applying anp to researching oral health international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 57 table 4 the weighted supermatrix of the inverted task 1 2 3 4 5 6 7 1.1 1.2 1.3 2.1 2.2 2.3 3.1 3.2 4.1 4.2 4.3 5 6 7.1 7.2 1.1 0 0 0 0 0.071 0.071 0.039 0.039 0.019 0 0.027 0 0 0 1.2 0.2 0 0 0 0.214 0.071 0.013 0.013 0.056 0 0.054 1.3 0.8 0 0 0 0.214 0.357 0.006 0.006 0.019 0 0.014 2.1 0 0 0.333 0.167 0.176 0.176 0 0 0 0 2.2 0.75 0 0.333 0.062 0.062 2.3 0.25 0.167 0 0.033 0.033 3.1 0 0 0 0.272 0.026 0 0.013 0 0 0 3.2 0.272 0 0.026 0 0.04 4.1 0 0 0.064 0.122 0 0 0.324 0 0.571 0.097 0.097 4.2 0.02 0.03 0.324 0 0.108 0.286 0.016 0.016 4.3 0.128 0.06 0.108 0 0 0.143 0.048 0.048 5 0 0 0.03 0.03 0.155 0.368 0.155 0 0 0.083 0.083 6 0 0 0.037 0.037 0.266 0.632 0.266 1 0 0.582 0.583 7.1 0 0 0.078 0.029 0 0 0 0 0.174 7.2 0.039 0.087 0.174 0 table 5 demonstrates that external factors in the first and second clusters are not influenced by internal ones. at the same time, the limiting priority of influence of each factor upon all the other factors is the same. the general physiological factors (clusters 4, 5, 6) have the most important influence upon the general goal of oral health and others element in the model. the influence of life-support conditions (cluster 1) is less than the total influence of the above mentioned factors by approximately 10 times. the second cluster (oral hygiene) and the third cluster (dental health status) have small and close limiting priorities of influence, and the seventh cluster (salivation system) is nearly insignificant. ijahp article: andreichicova, radyshevskaya/applying anp to researching oral health international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 58 table 5 limiting priorities for the inverted task 1 2 3 4 5 6 7 0.0458 0.0219 0.0224 0.47 0.116 0.3207 0.0033 1.1 1.2 1.3 2.1 2.2 2.3 3.1 3.2 4.1 4.2 4.3 5 6 7.1 7.2 1.1 0 0 0 0 0 0 0.0083 0.0083 0.0083 0.0083 0.0083 0.0083 0.0083 0.0083 0.0083 1.2 0 0 0 0 0 0 0.0214 0.0214 0.0214 0.0214 0.0214 0.0214 0.0214 0.0214 0.0214 1.3 0 0 0 0 0 0 0.0161 0.0161 0.0161 0.0161 0.0161 0.0161 0.0161 0.0161 0.0161 2.1 0 0 0 0 0 0 0.0081 0.0081 0.0081 0.0081 0.0081 0.0081 0.0081 0.0081 0.0081 2.2 0 0 0 0 0 0 0.0093 0.0093 0.0093 0.0093 0.0093 0.0093 0.0093 0.0093 0.0093 2.3 0 0 0 0 0 0 0.0045 0.0045 0.0045 0.0045 0.0045 0.0045 0.0045 0.0045 0.0045 3.1 0 0 0 0 0 0 0.0104 0.0104 0.0104 0.0104 0.0104 0.0104 0.0104 0.0104 0.0104 3.2 0 0 0 0 0 0 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 0.012 4.1 0 0 0 0 0 0 0.218 0.218 0.218 0.218 0.218 0.218 0.218 0.218 0.218 4.2 0 0 0 0 0 0 0.1776 0.1776 0.1776 0.1776 0.1776 0.1776 0.1776 0.1776 0.1776 4.3 0 0 0 0 0 0 0.0744 0.0744 0.0744 0.0744 0.0744 0.0744 0.0744 0.0744 0.0744 5 0 0 0 0 0 0 0.1159 0.1159 0.1159 0.1159 0.1159 0.1159 0.1159 0.1159 0.1159 6 0 0 0 0 0 0 0.3207 0.3207 0.3207 0.3207 0.3207 0.3207 0.3207 0.3207 0.3207 7.1 0 0 0 0 0 0 0.0015 0.0015 0.0015 0.0015 0.0015 0.0015 0.0015 0.0015 0.0015 7.2 0 0 0 0 0 0 0.0018 0.0018 0.0018 0.0018 0.0018 0.0018 0.0018 0.0018 0.0018 figure 4 limiting priorities of influence for the inverted network task. the results obtained are based on the study of pathological processes in the oral cavity in connection with other organism systems and a human environment. they are well matched with intuitive concepts of the experts, have a reasonable interpretation, and enable us to make a detailed forecast that can be substantiated. the intermediate results are also of interest, as they enable us to analyze the interactions of the elements being ijahp article: andreichicova, radyshevskaya/applying anp to researching oral health international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 59 considered. the application of anp gives an opportunity to estimate the contributions of the external and internal factors that were examined in the oral health system and to predict their influence in view of feedback in the open physiological system. figure 5 limiting influence priorities of the clusters from the inverted network model. the conclusion that the health of oral cavity is governed by the elements of dental health status is well known and validates the expert information used. the new results from this study are the relative priorities describing influence intensity and impact on oral health of such factors as immunity level, hormonal status, vegetative regulation type, blood circulation type, and parameters of salivation system. these results are of a special interest for dentists and physiologists. the influence of the elements of general health status is very strong; it is an order greater than the influence of external conditions. let us note that the small limiting priorities of influence of the external factors should not be interpreted as evidence of their insignificance. the cyclical character of mutual influence between the functional systems in human organism increases the limiting priorities of these factors. for example, the elements of the third cluster (dental health status) influence the elements of the fourth cluster (general health status), which in turn influences dental health status, i.e. the influence in the cycle is gradually increased up to some limiting value. clusters 1 and 2 representing external conditions are not included in the cycles; therefore their impact on dental health status and general health status is distributed throughout the system and appears indirectly. this implies that the conditions of life-support and oral hygienic factors become more important when physiological parameters of the human organism are abnormal. in other words, people with immunodeficiency or with abnormalities of blood circulation, hormonal status and vegetative regulation are influenced by ecology, diet and social conditions to a greater degree. therefore they should care for the oral cavity more attentively and take preventive restorative measures. the application of anp to the problem of oral health forecasting has a number of advantages. in particular we might mention the following: • an opportunity to build a model on the basis of expert information that takes into consideration intangible factors and qualitative parameters; ijahp article: andreichicova, radyshevskaya/applying anp to researching oral health international journal of the vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 60 • considerable time saving and a wider area of application in comparison with statistical models based on empirical data; • an opportunity to check out different hypotheses about a structure involving various sets of factors and the intensity of influences within it. references andreichicova o.n. & radyshevskaya t.n. (2003). application of analytic network process for a forecasting of basic systems’ health in the human organism. information technologies, 7, 45-53 (in russian). borovskiy e.v. & leont’ev b.k. (1991). biology of an oral cavity. moscow: medicine (in russian). eriksen h.m. & bjertness e. (1991). concepts of health and disease and caries prediction: a literature review. scand. j. dent. res., 99 (6), 476-483. hollister m.c. & weintraub j.a. (1993). the association of oral status with systemic health, quality of life and economic productivity. j. dent. educ., 57 (12), 901-912. leont’ev v.k., shestakov v.t. & voronin v.f. (2003). estimation of the basic directions of dentistry development. moscow: medical book (in russian). rybakov a.i. & chelidze l.n. (1990). anatomico-physiological features of an oral cavity and their significance in a pathology, tbilisi: metsnierba (in russian). saaty t.l. (2001). decision making with dependence and feedback; the analytic network process. pittsburgh, pa: rws publications. ijahp article: singh h., singh l. p., singh a., bilga p. s./evaluation and prioritization of factors affecting energy expenditure of workers engaged in manual material handling using analytical hierarchy process international journal of the analytic hierarchy process 440 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.293 evaluation and prioritization of factors affecting energy expenditure of workers engaged in manual material handling using analytical hierarchy process harwinder singh department of mechanical engineering guru nanak dev engineering college ludhiana, punjab-141006, india e-mail: harwin75@rediffmail.com lakhwinder pal singh department of industrial & production engineering dr b. r ambedkar national institute of technology jalandhar, punjab-144011, india e-mail: singhl@nitj.ac.in amandeep singh department of industrial & production engineering dr b. r ambedkar national institute of technology jalandhar, punjab-144011, india e-mail: ip.nitj@gmail.com paramjit singh bilga department of mechanical engineering guru nanak dev engineering college ludhiana, punjab-141006, india e-mail: psbilga@gndec.ac.in abstract the present study aimed to evaluate energy expenditure of workers engaged in a manual material handling task .the various factors/sub factors influencing energy expenditure with physical impact on the human body were prioritized in terms of weight values by using the analytical hierarchy process. the study included a sample of sixty male workers with a mean age ± sd of 40.34 ± 7.65, data with respect to their job activity and physical characteristics were collected using a validated questionnaire. the results showed an average working heart rate ± sd of 124.5±12.24 beats/min and average energy expenditure ± sd of 3370.33 ± 283.86 kcal; these are clear indicators of strenuous activity. the results of the ahp evaluation showed physical workload (pw) as the most important factor followed by physical work capacity (pwc), type of activity (toa), organizational factors (of) and personal factors (pf) with weight values of 0.454139, 0.252781, 0.129274, 0.125318 and 0.038488 respectively. the study concluded with prioritization of various factors contributing to a high rate of energy expenditure which may lead to overexertion and musculoskeletal injuries. the findings indicated an utmost mailto:harwin75@rediffmail.com mailto:singhl@nitj.ac.in mailto:ip.nitj@gmail.com mailto:psbilga@gndec.ac.in ijahp article: singh h., singh l. p., singh a., bilga p. s./evaluation and prioritization of factors affecting energy expenditure of workers engaged in manual material handling using analytical hierarchy process international journal of the analytic hierarchy process 441 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.293 need to redesign job content with the addition of some periods of break time in order for the body to recover from the excessive energy expenditure. keywords: manual material handling (mmh); total daily energy expenditure (tdee); analytical hierarchy process (ahp) 1. introduction manufacturing is the greatest need of a developing country to support economic growth, and involves a number of manual material handling (mmh) tasks performed by blue collar workers (marras, w. s., cutlip, r. g. et al., 2009). these tasks may include activities like lifting, carrying, pulling, pushing or moving a supporting load by workers for particular period of time. if such tasks are performed repetitively with non-ergonomic conditions they may cause either temporary or permanent injury (rossi, d., bertoloni, e. et al., 2013). repetitive tasks with rigorous effort cause overexertion due to a high rate of energy expenditure which ultimately leads to muscular skeletal disorders (msds) in workers (kee, d., & seo, s. r., 2007). moreover, it increases absenteeism and the rate of compensation to workers under such conditions (kee, d., & seo, s. r., 2007).the dynamic mmh tasks demand a high level of energy which may decrease body strength and result in consequent msds (waters, t. r., putz-anderson et al., 1993). the rate of energy expenditure depends upon the type of mmh occupation (light, moderate, heavy, very heavy or extremely heavy) as well as other daily activities performed (indian council of medical research, 2010). the world health organization (who) has adopted a factorial technique to estimate energy requirements depending upon body weight to predict a person’s basal metabolic rate (bmr). at the same time physical activity level (pal) is determined using physical activity ratio (par) values, which are further determined from daily activities to calculate total daily energy expenditure (indian council of medical research, 2010). in developing countries like india, human labor has been engaged as a load transfer device repetitively for loading and unloading activities from conveyor to pallets, carts or directly into trucks/wagons. in such cases, human labor is necessary due to a lack of automation which if used would result in a high cost investment for industries. very few studies have been found that focus on energy expenditure of workers engaged in mmh tasks (puttewar, a. s., & jaiswal, s. b., 2014; ismaila, s., oriolowo, k.et al., 2012; nawi, n. m., yahya, a., et al., 2012; li, k. w., yu, r. et al., 2009; pradhan, c. k., thakur, s., et al., 2007). even fewer studies have reported on indian labor and the influence of energy expenditure on the human body due to mmh activities as part of a worker’s occupation (pradhan, c. k., thakur, s. et al., 2007). in the present study, the repetitive manual material handling activity considered was in the baggage section of a fertilizer firm, where laborers are engaged in loading/unloading 50 kg bags of urea from a running conveyor to trucks/wagons 7.5 meters away (approximately 8 steps). a single break time period of 45 minutes was given to workers during an 8 hour working shift. this mmh activity puts a forceful exertion on the human body that leads to a high rate of energy expenditure that causes over exertion and msds. a number of factors affecting energy expenditure are still unexplored, so there is a need to identify and prioritize these factors. hence, the present study is carried out in order to evaluate and prioritize various factors affecting the rate of energy expenditure using the ijahp article: singh h., singh l. p., singh a., bilga p. s./evaluation and prioritization of factors affecting energy expenditure of workers engaged in manual material handling using analytical hierarchy process international journal of the analytic hierarchy process 442 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.293 analytical hierarchy process (ahp). this will help determine necessary measures for combating the effects of high energy expenditure. 2. methods the methodology used for investigation is described in figure 1 as shown below: figure 1. block diagram of methodology 2.1. selection of workers a sample of sixty male workers at a fertilizer firm was selected using non probability convenience sampling. all of the workers were performing manual lifting and carrying activities without any aid from mechanized machinery/devices. a suitable questionnaire was devised for collecting data pertaining to age, height, weight, body mass index and energy expenditure of the workers. the questionnaire was pre-tested and validated using opinions of experts and chron bach alpha (0.78). 2.2 physical workload the physical workload of the job activity was classified based upon the observed heart rate which was obtained using a polar heart rate monitor. activities were categorized as light, moderate, heavy, very heavy or extremely heavy as mentioned in table 1 (astrand, p. o., 2003). selection of workers identification of parameters data collection analytical hierarchy process results and discussion conclusions ijahp article: singh h., singh l. p., singh a., bilga p. s./evaluation and prioritization of factors affecting energy expenditure of workers engaged in manual material handling using analytical hierarchy process international journal of the analytic hierarchy process 443 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.293 table 1 classification of physical workload physical workload heart rate (beats/min) light work up to 90 moderate work 90-110 heavy work 110-130 very heavy work 130-150 extremely heavy work 150-170 2.3 total daily energy expenditure (tdee) each participant was interviewed using a questionnaire and information was collected about time spent in various physical activities throughout a day. further, total daily energy expenditure (tdee) was calculated from observed data by following a standardized procedure given by the indian council of medical research (2010). total daily energy expenditure is calculated as: tdee (kcal) = predicted bmr× pal where bmr is basal metabolic rate i.e. amount of energy expended daily by humans at rest and calculated as follows: equation for prediction of bmr (kcal/24h): 10.9× body weight (kg) + 833 where pal is physical activity level i.e. a person’s total daily energy expenditure in a 24 hour period divided by basal metabolic rate (bmr), which is calculated as follows: 2.4 analytical hierarchy process the analytical hierarchy process is a decision making tool applied under various complex situations where a number of factors and sub-factors affect the goal simultaneously (singh, h., & kumar, r., 2013; badri, m. a., 2001). the result gives priorities to every factor/sub factor with some weight value by following a systematic methodology (figure 2). a standardized procedure has been given by saaty (1990). ijahp article: singh h., singh l. p., singh a., bilga p. s./evaluation and prioritization of factors affecting energy expenditure of workers engaged in manual material handling using analytical hierarchy process international journal of the analytic hierarchy process 444 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.293 figure 2. systematic methodology of ahp (saaty, t. l., 2008) 2.4.1 goal of the study the goal of the present study is to evaluate various factors/sub factors influencing the total energy expenditure of workers engaged in a manual material handling activity on the basis of weight values. 2.4.2 structure of hierarchy a three-level relative hierarchy model was structured. level 1 refers to the overall objective, level 2 is composed of five main criteria such as physical workload (pw), type of activity (toa), physical work capacity (pwc), organizational factors (of) and personal factors (pf) and level 3 is made up of 23 sub-criteria as shown in figure 3. goal of the study structure decision hierarchy degree of preference construct pair wise comparison matrices normalized matrix consistency checks ijahp article: singh h., singh l. p., singh a., bilga p. s./evaluation and prioritization of factors affecting energy expenditure of workers engaged in manual material handling using analytical hierarchy process international journal of the analytic hierarchy process 445 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.293 figure 3. three-level hierarchy model 2.4.3 degree of preference a 1-9 point scale was used in the pair wise comparison which is a standard procedure used to make decisions in a quantified form. this is shown in table 2. physical work capacity organizational factors personal factors physical workload type of activity energy expenditure extremely heavy light moderate heavy very heavy age of worker body mass index exercise performed psychotropic medication diet intake lifting activity carrying activity pulling activity pushing activity organization environment organization layout break time period scrutiny and restriction training and motivation multiple jobs house-hold activities lack of awareness nature and behavior s ta g e : 1 s ta g e : 2 s ta g e : 3 ijahp article: singh h., singh l. p., singh a., bilga p. s./evaluation and prioritization of factors affecting energy expenditure of workers engaged in manual material handling using analytical hierarchy process international journal of the analytic hierarchy process 446 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.293 table 2 degree of preference (saaty, t. l., 1990) value judgment description 1 equal two alternatives share the same level of importance 3 moderate experience and judgment favors one alternative with respect to the other in little measure 5 strong experience and judgment strongly favor one attribute over another 7 very strong experience and judgment tell that one alternative is much more important than the other 9 extreme the difference of importance is extreme 2,4,6,8 intermediate values used if more precision is needed 2.4.4 pair-wise comparison the importance of i th sub-objective was compared with j th sub-objective. in the current study 23 sub-objectives were considered as shown in figure 3 above. 2.4.5 normalized matrix of different sub-objectives after a pair-wise comparison matrix is obtained, the next step is to divide each entry in a column by the sum of entries in the column to get the value of a normalized matrix. the values of a normalized matrix rij are calculated by using the following formula: the average of the elements in each row gives an estimate of relative weights of subobjectives being compared. thus, the approximate priority weights (w1, w2 . . . wj) for each sub-objective are computed as given in the formula below: 2.4.6 consistency index a consistency check is performed using a consistency index (ci), which is calculated by the following expression: after a ci value, a consistency ratio (cr) is calculated by using the following formula: ijahp article: singh h., singh l. p., singh a., bilga p. s./evaluation and prioritization of factors affecting energy expenditure of workers engaged in manual material handling using analytical hierarchy process international journal of the analytic hierarchy process 447 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.293 table 3 random index values n 1 2 3 4 5 6 7 8 9 10 ri 0.00 0.00 0.58 0.90 1.12 1.24 1.32 1.41 1.45 1.49 where: λmax is the maximum eigen value, n is dimensional matrix. generally, if cr is less than 0.1 then judgments are consistent and acceptable, where random consistency index (ri) is fixed for every dimensional matrix and the same is selected from table 3 as given above. 3. results as per the qualitative data, the results revealed that the mean age±sd of the sample was 40.34 ±7.65 years with the minimum experience of two years in the same occupation. the majority of workers (65%) were illiterate or under middle standard of education. the demographic parameters are exhibited in table 4. in addition, a working heart rate and total daily energy expenditure was recorded as an average of 124.5 ± 12.24 beats/min and 3369.78±284.86 kcal respectively. table 4 physical characteristic of the workers physical characteristic mean and standard deviation 30-40yrs 40-50yrs 50-60yrs height (cm) 168.66 ± 7.45 160.34 ± 4.04 165.25 ± 11.18 weight (kg) 70.8 ± 8.13 69.7 ± 7.81 70.25 ± 5.55 bmi (kg/m 2) 25.29 ± 1.50 26.33 ± 2.28 26.42 ± 2.16 working heart rate (beats/min) 133.5 ± 12.54 124.6 ±13.72 115.4 ± 10.49 tdee (kcal) 3557.3 ± 318.50 3311.30 ± 257.06 3240.74 ±279.03 3.1 analytical hierarchy process a number of factors and sub factors were identified based on the literature and expert advice. subsequently, the ahp was applied to construct a hierarchy for the identification and prioritization of main and sub factors (figure 3). physical work load (pwl) was found to be the most significantly influencing factor followed by physical work capacity (pwc), type of activity (toa), organizational factors (oa) and personal factors (pf) with respect to the objective (table 5-10). ijahp article: singh h., singh l. p., singh a., bilga p. s./evaluation and prioritization of factors affecting energy expenditure of workers engaged in manual material handling using analytical hierarchy process international journal of the analytic hierarchy process 448 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.293 table 5 paired comparison matrix level 1 with respect to objective λmax = 5.38165, ci= 0.0954129, for n=5, cr= 0.0851900 = 8.52% < 10% (acceptable) table 6 paired comparison matrix level 2 with respect to factor ‘pwl’ l ig h t m o d e r a te h e a v y v e r y h e a v y e x tr e m e ly h e a v y w e ig h t light 1 2 1/5 ½ ¼ 0.0861751 moderate 1/5 1 1/5 1/3 ¼ 0.0606928 heavy 5 5 1 3 2 0.426509 very heavy 2 3 1/3 1 ½ 0.154824 extremely heavy 4 4 1/5 2 1 0.271799 λmax = 5.08528, ci= 0.0213193, for n=5, cr= 0.0190350 = 1.90% < 10% (acceptable) table 7 paired comparison matrix level 2 with respect to factor ‘toa’ l if ti n g c a r r y in g p u ll in g p u sh in g w e ig h t lifting 1 1/5 7 3 0.226462 carrying 5 1 9 5 0.629104 pulling 1/7 1/9 1 1/3 0.0423596 pushing 1/3 1/5 3 1 0.102074 λmax = 4.21714, ci= 0.0723807, for n=4, cr= 0.080423 = 8.04% < 10% (acceptable) p w l t o a p w c o f p f w e ig h t pwl 1 2 3 5 9 0.454139 toa 0.5 1 1/3 ½ 5 0.129274 pwc 1/3 3 1 3 5 0.252781 of 1/5 2 1/3 1 3 0.125318 pf 1/9 1/5 1/5 1/3 1 0.0384886 ijahp article: singh h., singh l. p., singh a., bilga p. s./evaluation and prioritization of factors affecting energy expenditure of workers engaged in manual material handling using analytical hierarchy process international journal of the analytic hierarchy process 449 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.293 table 8 paired comparison matrix level 2 with respect to factor pwc λmax = 5.22437, ci= 0.0560931, for n=5, cr= 0.050083 = 5.01% < 10% (acceptable) table 9 paired comparison matrix level 2 with respect to factor ‘of’ o r g a n iz a ti o n a l e n v ir o n m e n t o r g a n iz a ti o n a l l a y o u t b r e a k t im e s c r u ti n y a n d r e st r ic ti o n s t r a in in g / m o ti v a ti o n w e ig h t organizational environment 1 2 1/3 4 0.2 0.136902 organization layout 0.5 1 1/7 2 0.333 0.0695371 break time 3 7 1 7 5 0.532869 scrutiny and restrictions 0.25 0.5 1/7 1 1/9 0.0396523 training/ motivation 2 3 0.2 9 1 0.221039 λmax = 5.27915, ci= 0.0697881, for n=5, cr= 0.06231080 = 6.23% < 10% (acceptable) a g e b m i e x e r c is e p sy c h o tr o p ic m e d ic a ti o n s d ie t w e ig h t age 1 2 5 4 1/5 0.189526 bmi 1/2 1 2 3 1/7 0.107074 exercise 1/5 ½ 1 ½ 1/9 0.0445895 psychotropic medication 1/4 1/3 2 1 1/7 0.0609924 diet 5 7 9 7 1 0.597818 ijahp article: singh h., singh l. p., singh a., bilga p. s./evaluation and prioritization of factors affecting energy expenditure of workers engaged in manual material handling using analytical hierarchy process international journal of the analytic hierarchy process 450 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.293 table 10 paired comparison matrix level 2 with respect to factor ‘pf’ multiple jobs household activitie s lack of awarenes s nature / behavior weight multiple jobs 1 2 7 4 0.523923 house-hold activities ½ 1 4 2 0.270708 lack of awareness 1/7 ¼ 1 ½ 0.0700147 nature/behavior ¼ ½ 2 1 0.135354 λmax = 4.00223, ci= 0.000743219, for n=5, cr= 0.0008222222 = 0.08% < 10% (acceptable) 4. discussion the energy expenditure of workers was found to fall under the heavy workload category as recommended by the indian council of medical research (2010).the heart rate of workers also indicated that their job fell under the heavy workload category as it exceeded 120 beats/min which ultimately puts adverse stress on the human body (maiti, r., 2008). the mean bmi of the majority of workers exceeded the normal range, and this consequently lowers the physical work capacity of these workers (ismaila, s., oriolowo, k., et al., 2012; xu, x., mirka, g. a. et al., 2008). the results from the analytical hierarchy process showed the physical workload as the most significant factor as the workers lift and carry 50 kg loads for 7.5 meters. ultimately, more energy expenditure would be needed to execute their task which directly causes whole body fatigue and muscle injuries (pradhan, c. k., thakur, s. et al., 2007; waters, t. r., putz-anderson, v. et al., 1993). diet has been pointed out as another factor as shown in table 8. improper and lack of sufficient diet intake increases chances of digestive problems, and also decreases retrieval rate of work-related injuries (keusch, g. t., 2003; montain, s. j., & young, a. j., 2003). the present study also highlighted an insufficient break time of 45 minutes as an influencing factor which lowers recovery rate from exertion in the mmh job activity. this is because over exertion and insufficient rest pauses under heavy workload activities increase the chance of muscle injuries (kee, d., & seo, s. r., 2007). the salary of workers was found to be insufficient in light of their requirements therefore making it necessary for them to do multiple jobs which then leads to body fatigue due to restlessness. the current study explored various key factors which were still absent in the literature for developing countries like india, such as heavy workload, inadequate break time period, low income, lack of awareness about health issues, multiple jobs, household activities, nature/behavior and illiteracy rate among manual material handling workers in labor extensive occupations. 5. conclusions manual material handling jobs are chosen by industrial management because of the lack of automation, which if applied, would involve considerable investment from the company. workers can and are being engaged in manual material handling at cheaper wages due to unemployment and lack of awareness about health risks in these occupations. in the present study, the mean energy expenditure of workers revealed these mmh occupations as strenuous activity due to workload. subsequently, the ijahp article: singh h., singh l. p., singh a., bilga p. s./evaluation and prioritization of factors affecting energy expenditure of workers engaged in manual material handling using analytical hierarchy process international journal of the analytic hierarchy process 451 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.293 results of the ahp rated physical workload as the most influencing factor followed by physical work capacity, type of activity, organizational factors and personal factors. in conclusion, the study also explored the fact that the company is disregarding health and safety issues, as reflected in the insufficient diet intake, lack of rest pauses and inadequate salary for the workers which results in them seeking involvement in other occupations and increases the health risks. ijahp article: singh h., singh l. p., singh a., bilga p. s./evaluation and prioritization of factors affecting energy expenditure of workers engaged in manual material handling using analytical hierarchy process international journal of the analytic hierarchy process 452 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.293 references astrand, p. o. (2003). textbook of work physiology: physiological bases of exercise. human kinetics. champaign,il:mcgraw hill. badri, m. a. (2001). a combined ahp–gp model for quality control systems. international journal of production economics, 72(1), 27-40. doi:10.1016/s09255273(00)00077-3 indian council of medical research. expert group. (2010). nutrient requirements and recommended dietary allowances for indians: a report of the expert group of the indian council of medical research. new delhi: indian council of medical research. ismaila, s., oriolowo, k., & akanbi, o. (2012). work capacity assessment of nigerian bricklayers. management science letters, 2(1), 263-272. doi: 10.5267/j.msl.2011.08.014 kee, d., & seo, s. r. (2007). musculoskeletal disorders among nursing personnel in korea. international journal of industrial ergonomics, 37(3), 207-212. doi:10.1016/j.ergon.2006.10.020 keusch, g. t. (2003). the history of nutrition: malnutrition, infection and immunity. the journal of nutrition, 133(1), 336s-340s. li, k. w., yu, r. f., gao, y., maikala, r. v., & tsai, h. h. (2009). physiological and perceptual responses in male chinese workers performing combined manual materials handling tasks. international journal of industrial ergonomics, 39(2), 422-427. doi:10.1016/j.ergon.2008.08.004 maiti, r. (2008). workload assessment in building construction related activities in india. applied ergonomics, 39(6), 754-765. doi:10.1016/j.apergo.2007.11.010 marras, w. s., cutlip, r. g., burt, s. e., & waters, t. r. (2009). national occupational research agenda (nora) future directions in occupational musculoskeletal disorder health research. applied ergonomics, 40(1), 15-22. doi:10.1016/j.apergo.2008.01.018 montain, s. j., & young, a. j. (2003). diet and physical performance, us army research, paper 34. appetite, 40(3), 255-267. doi:10.1016/s0195-6663(03)00011-4 nawi, n. m., yahya, a., chen, g., bockari-gevao, s. m., & maraseni, t. n. (2012). human energy expenditure in lowland rice cultivation in malaysia.journal of agricultural safety and health, 18(1), 45-56. doi: http://dx.doi.org/10.13031/j2012.2013 pradhan, c. k., thakur, s., & chowdhury, a. r. (2007). physiological and subjective assessment of food grain handling workers in west godavari district, india. industrial health, 45(1), 165-169. doi: http://doi.org/10/2486/indhealth.45.165 http://dx.doi.org/10.1016/s0925-5273(00)00077-3 http://dx.doi.org/10.1016/s0925-5273(00)00077-3 http://dx.doi.org/10.1016/j.ergon.2006.10.020 http://dx.doi.org/10.1016/j.ergon.2008.08.004 http://dx.doi.org/10.1016/j.apergo.2007.11.010 http://dx.doi.org/10.1016/j.apergo.2008.01.018 http://dx.doi.org/10.1016/s0195-6663(03)00011-4 ijahp article: singh h., singh l. p., singh a., bilga p. s./evaluation and prioritization of factors affecting energy expenditure of workers engaged in manual material handling using analytical hierarchy process international journal of the analytic hierarchy process 453 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.293 puttewar, a. s., & jaiswal, s. b. (2014). an empirical study of posture related discomfort in rice mill workers. international journal of research in aeronautical and mechanical engineering, 2(5), 50-54. rossi, d., bertoloni, e., fenaroli, m., marciano, f., & alberti, m. (2013). a multicriteria ergonomic and performance methodology for evaluating alternatives in “manuable” material handling. international journal of industrial ergonomics, 43(4), 314-327. doi:10.1016/j.ergon.2013.04.009 saaty, t. l. (2008). decision making with the analytic hierarchy process. international journal of services sciences, 1(1), 83-98. doi: 10.1504/ijssci.2008.017590 saaty, t. l. (1990). how to make a decision: the analytic hierarchy process. european journal of operational research, 48(1), 9-26. doi:10.1016/0377-2217(90)90057-i singh, h., & kumar, r. (2013). hybrid methodology for measuring the utilization of advanced manufacturing technologies using ahp and topsis. benchmarking: an international journal, 20(2), 169-185. doi: http://dx.doi.org/10.1108/14635771311307669 waters, t. r., putz-anderson, v., garg, a., & fine, l. j. (1993). revised niosh equation for the design and evaluation of manual lifting tasks. ergonomics, 36(7), 749776. doi: 10.1080/00140139308967940 xu, x., mirka, g. a., & hsiang, s. m. (2008). the effects of obesity on lifting performance. applied ergonomics, 39(1), 93-98. doi:10.1016/j.apergo.2007.02.001 http://dx.doi.org/10.1016/j.ergon.2013.04.009 http://dx.doi.org/10.1016/0377-2217(90)90057-i http://dx.doi.org/10.1108/14635771311307669 http://dx.doi.org/10.1016/j.apergo.2007.02.001 ijahp news and events: saaty/creative decisions foundation sponsors ahp/anp presentations at 2019mcdm international journal of the analytic hierarchy process 296 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.681 creative decisions foundation sponsors ahp/anp presentations at 2019mcdm rozann saaty president, creative decision foundation the 2019 multi-criteria decision making (mcdm) conference took place at istanbul technical university (itu) in istanbul, turkey from june 17 21. grants that covered the conference registration fee were awarded by the creative decisions foundation (cdf) to selected teachers, researchers with recent phd degrees, and students studying for their phd. applicants were first required to have already been accepted to make a presentation at the 2019 conference on a topic involving ahp or anp, and then they had to submit a paper they had written about their topic along with their application to be considered for a grant. a joint committee formed of the mcdm2019 organizers and personnel from the creative decisions foundation determined the award recipients. cdf grantees displaying their awards at the awards reception an award reception for the cdf grantees was held at the itu pool overlooking the bosphorus strait on monday, june 17. click here to see the pictures of the individual grantees receiving their awards: https://www.dropbox.com/sh/kqv6tlrawf1kkro/aaa5jj_g8dhgtogboor2p16sa?dl=0 https://www.dropbox.com/sh/kqv6tlrawf1kkro/aaa5jj_g8dhgtogboor2p16sa?dl=0 ijahp news and events: saaty/creative decisions foundation sponsors ahp/anp presentations at 2019mcdm international journal of the analytic hierarchy process 297 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.681 twenty-one people from 13 countries received grants. at the reception they were presented with an award and a copy of what some say is the best book ever written on the subject of ahp/anp by thomas l. saaty: principia mathematica decernendi (principles of decision making with the ahp). rozann saaty, president of creative decisions foundation and ilker topcu, chairman of mcdm2019 ilker topcu, chairman of the mcdm2019 conference, received a special award for the way he supported the cdf sponsorship program. he helped the grantees all the way through the process, offering advice on everything from travel issues to the most affordable lodging accommodations. enjoying the pool and live music at the awards reception ijahp news and events: saaty/creative decisions foundation sponsors ahp/anp presentations at 2019mcdm international journal of the analytic hierarchy process 298 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.681 one of the purposes of creative decisions foundation is to help young researchers get started in the publishing game by helping them get their papers that are focused on ahp published. the papers covered a wide range of topics from theory to applications in various fields, and their quality was excellent, exceeding the expectations of the award committee. our ijahp editor-in-chief, dr. enrique mu, was present at the reception to personally invite cdf grantees to consider submitting their work to our official ahp journal. (www.ijahp.org). enrique mu, editor-in-chief of the ijahp journal, milagros pereya and mónica garcía-melón on the dinner cruise along the bosphorus the mcdm conference gave many opportunities to ahp participants to meet each other and one of most celebrated events was the dinner cruise along the bosphorus where breathtaking views of istanbul could be admired. http://www.ijahp.org/ ijahp news and events: saaty/creative decisions foundation sponsors ahp/anp presentations at 2019mcdm international journal of the analytic hierarchy process 299 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.681 creative decisions foundation display of books on ahp conference attendees could browse through the books on ahp that were displayed at the creative decisions foundation exhibit table. most are available for purchase as e-books from amazon.com, google books or apple itunes. links to where to purchase them can be found on creative decision foundation’s book site: www.rwspublications.com. http://www.rwspublications.com/ ijahp: mu/ another tunnel at the end of the tunnel international journal of the analytic hierarchy process 205 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.910 another tunnel at the end of the tunnel dear ijahp reader: the events of the past weeks and days have shown how complex and dynamic today’s environment is. when we thought the pandemic was over, the appearance of a new, more contagious delta variant seriously affected our expectations of getting back to normal (or a new normal) soon. the debates continue about the extent to which, or if, meeting restrictions should be re-imposed, whether vaccination should be mandated, and so forth. if there was only a single factor to consider (i.e., decreasing covid-19 infections) the decisions would be easier, but other, even conflicting, factors, such as economic, moral and mental health, also need to be addressed. a similarly complex situation has occurred with the recent news about the quick collapse of afghanistan upon the american exit. there were clearly many factors at play that were either not considered, not known, or whose impact was underestimated at the time of making or implementing the decision to withdraw. thomas saaty had situations like these in mind when he developed the analytic hierarchy process (ahp) to consider multicriteria decisions in a complex world (saaty, 1980) and later further developed the analytic network process (anp) that can include dependence and feedback in the decision analysis. the feedback occurs not just among the criteria, but also among the alternatives, and even between criteria and alternatives (saaty, 2005). saaty conceived decision-making in a complex world to be either a hierarchy or a network of factors dependent on each other. the mission of the international journal of the analytic hierarchy process (ijahp) is to spread the use of the ahp/anp to solve real, complex, and important problems. as such, this issue includes several important applications; however, we are particularly proud to include a report titled “an analytic network model (anp) to estimate the benefits, opportunities, costs, and risks (bocr) that gun policies and violence prevention interventions have on legal users of firearms” authored by professors amos n. guiora, marcel c. minutolo and luis g. vargas, in this issue. this award-winning study tackles one of today’s most controversial and difficult problems and constitutes a magnificent example of the use of an ahp/anp approach to address complex decisions. enjoy the issue! enrique mu, phd ijahp editor in chief ijahp: mu/ another tunnel at the end of the tunnel international journal of the analytic hierarchy process 206 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.910 references saaty, t. l. (1980). decision making for leaders: the analytic hierarchy process. new york, n.y., mcgraw-hill. saaty, t. l. (2005). theory and applications of the analytic network process. pittsburgh, rws publications. microsoft word 2b_editorial_salomon_vol2_issue1.docx ijahp editorial: salomon / words from the special editor international journal of the analytic hierarchy process vol. 2, issue 1, 2010 issn 1936-6744 words from the special editor  the state of the art of ahp/anp in brazil  as in the world, ahp/anp is the most applied mcdm method in brazil. there are ahp/anp  researchers and users from northeastern to southern brazilian states. however, the brazilian  economy (63%) and population (42%) are concentrated in the southeast region. in this issue,  we have works from three southeast brazilian states: minas gerais, rio de janeiro, and sao  paulo.   in the first article, dimas aguiar and his co‐authors examine the fmea (failure mode and  effect analysis) process. a number of reference tables for the severity, occurrence and  detection of failure have been proposed in the fmea international literature, and they use the  ahp to select the most appropriate reference tables for a brazilian auto parts factory located  in the state of sao paulo. this article is followed by an ahp application in health care  management: the selection of an ultrasonic scanning system for a private hospital in the state  of minas gerais.  professor jose arnaldo montevechi, a famous brazilian consultant of  engineering economics, was the supervisor of this application. the third article in this issue  discusses the use of ahp to determine criteria weights used in post‐occupancy evaluations by  customers of building performance. this civil engineering application of ahp was conducted in  rio de janeiro by prof. helder costa, who authored one of the first ahp books in portuguese.  the last two articles were supervised by prof. carmen belderrain, from the city of sao jose dos  campos, sao paulo. with ph.d. candidate claudio piratelli, prof. belderrain presents the use of  anp in the design of a performance measurement system. in the last article, with leila  nascimento and amanda silva, prof. belderrain presents how the anp was used in a military  decision that gained much attention in the brazilian media in the year of 2009: the purchase of  jet fighters by the brazilian air force.    prof. valério salomon  special issue editor  ijahp  rob typewritten text http://dx.doi.org/10.13033/ijahp.v2i1.68 ijahp essay: saaty/plato and aristotle’s missing axiom of logical thought: creativity and priorities international journal of the analytic hierarchy process 611 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.364 plato and aristotle’s missing axiom of logical thought: creativity and priorities thomas l. saaty i was charged with the task of writing about something that interests me that may also be popular with the public reader. since my childhood a fundamental concern of mine has been how to do things spontaneously and creatively, and i have long been familiar with plato’s three laws of thought. first, that nothing can become greater or lesser, either in number or magnitude, while remaining equal to itself. second, that without addition or subtraction there is no increase or diminution of anything, but only equality. third, that what was not before cannot be afterwards, without becoming and having become (plato, theaetetus, 155). the original three laws of thought comprise the most basic requirements for identification and reasoning, however i now think that there is something missing. these three laws can be expanded to include comparison which must precede the original three laws. the only way we know a thing is by comparing it with other things, or with our memories of other things. how does this affect the way logic is used to analyze problems if the ahp, which is based on comparisons, were to be used instead of the logical laws of thought? the great philosopher schopenhauer acknowledged that truth is the reference of a judgment to something outside it (we can interpret that to mean comparison) as its sufficient reason or ground. i say the idea of relating and comparing things is a fundamental law of nature despite its regrettable absence in the three axioms of logic set down by plato and aristotle nearly two and a half millennia ago. arthur schopenhauer wrote, “every truth is the reference of a judgment to something outside it, and the term intrinsic truth is a contradiction”. an entity can be compared with other entities in relative terms in regards to which of two entities have a property more and how much more it dominates another with respect to that property. creativity has been formally defined in terms of the following four factors: brainstorming, synectics, morphological analysis, and prioritization and order. brainstorming involves reaching out in the imagination for the relevant elements that influence a subject or a problem one has in mind. synectics has to do with relating these elements; it has been defined as the joining together of different and apparently irrelevant elements. morphological analysis is concerned with structuring these elements and their connections in a hierarchy or in a network. finally, prioritization is about ordering the elements according to their importance. while the physical world needs metrics to measure things scientifically, the behavioral world needs order to determine what is very important and what is not so important. this all depends on our imagination and ability to structure a decision having to do with order. in mathematics we have two important types of topology: metric topology and order http://en.wikipedia.org/wiki/plato http://en.wikipedia.org/wiki/theaetetus_(dialogue) ijahp essay: saaty/plato and aristotle’s missing axiom of logical thought: creativity and priorities international journal of the analytic hierarchy process 612 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.364 topology. it has been relatively easier to create devices for measuring things on different scales like temperature, length, mass and time. it has not been so easy to find ways to order things with respect to many criteria. indeed, measurement itself has subjective interpretations and it all comes down to how knowledgeable people interpret things and their measurements. we all have a brain and a mind, and therefore all knowledge about reality is captured by our brain and nervous system. then the mind stores our eventual interpretation and memory and this changes over time as we learn more and more. the question is, “how do we order things with our instincts and elementary nature?” the great french mathematician, henri lebesgue, who was concerned with questions of measure theory and measurement, wrote: "it would seem that the principle of economy would always require that we evaluate ratios directly and not as ratios of measurements. however, in practice, all lengths are measured in meters, all angles in degrees, etc.; that is, we employ auxiliary units and, as it seems, with only the disadvantage of having two measurements to make instead of one. sometimes this is because of experimental difficulties or impossibilities that prevent the direct comparison of lengths or angles, but there is also another reason. in geometrical problems, one needs to compare two lengths, for example, and only those two. it is quite different in practice when one encounters a hundred lengths and may expect to have to compare these lengths two at a time in all possible manners. thus, it is a desirable and economical procedure to measure each new length. one single measurement for each length, made as precisely as possible, gives the ratio of the length in question to each other length. this explains the fact that in practice, comparisons are never, or almost never, made directly through comparisons but with a standard scale." lebesgue did not go far enough in examining why we have to compare. when we deal with intangible factors, which by definition have no scales of measurement, we can only arrange them by comparing them in pairs. making comparisons is a talent we and all living things have. not only can we indicate the preferred object, but we can also discriminate among intensities of preference. through making comparisons using judgments, we are able, by using the ahp/anp, to derive priorities with respect to multiple criteria and combine them to obtain an overall order of things. that is a fulfillment of the fourth requirement of creativity in thinking. ijahp article: mostafa, abdelhamid, chileshe, dumrak/decision support model using anp to align leagile strategies to off-site manufacturing in australia international journal of the analytic hierarchy process 374 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.340 decision support model using anp to align leagile strategies to off-site manufacturing in australia sherif mostafa lecturer, school of natural and built environments university of south australia sherif.mostafa@unisa.edu.au tariq abdelhamid associate professor, school of planning, design and construction, michigan state university, east lansing, mi 48824, usa tariq@msu.edu nicholas chileshe senior lecturer, research education portfolio leader, school of natural and built environments university of south australia, city east campus, adelaide, sa 5001, australia nicholas.chileshe@unisa.edu.au jantanee dumrak lecturer, global project management program, torrens university australia, wakefield campus, adelaide, sa 5000, australia jantanee.dumrak@tua.edu.au abstract the australian housing supply has not been adequate to meet the constantly growing demand. four main factors driving this undersupply in australian housing are: (1) house completion time; (2) cost of finished house; (3) customer preferences and (4) level of skilled labor. offsite manufacturing (osm) could become a key innovation for the future of australian house building as it provides capacity in meeting the growing housing demand, green construction and lesser requirements for a labor force. osm is a modern construction method in which house building components are produced in offsite factories and then transported to the construction site to be assembled. the supply responsiveness of osm can be enhanced by employing lean and agile concepts. in this study, four leagile strategies are suggested to facilitate house builders decision making based on different combinations of housing supply factors. this paper matches these four strategies with the four studied factors in australian house building using the analytical network process (anp). the data employed for the anp model was derived from the actual specifications of 258 houses built in five australian states by five major house builders. the results from the anp model show the suitability in applying each strategy under different degrees influenced by the factors tested. keywords: off-site manufacturing; australian housing supply; leagile strategies; anp mailto:tariq@msu.edu mailto:nicholas.chileshe@unisa.edu.au mailto:jantanee.dumrak@tua.edu.au ijahp article: mostafa, abdelhamid, chileshe, dumrak/decision support model using anp to align leagile strategies to off-site manufacturing in australia international journal of the analytic hierarchy process 375 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.340 1. introduction residential building is one of the leading sectors in the australian economy. it consists of many independent building organizations that construct separate houses, semi-detached houses, townhouses, flats, units and apartments (dowling, 2005). in 2010-2011, the sector reported a significant production value of aud 47 billion. however, the responsiveness of the values of work commenced in residential building is unlikely to keep pace with the growth of other construction activities (abs, 2012). this situation is caused by the imbalance of housing supply and demand. due to the shortage of the housing supply, and neglect of the housing supply challenges, this paper mainly focuses on the australian housing problems from the supply perspective (coag, 2012; nhsc, 2013; liu & london, 2011). this paper addresses four main factors including house construction costs, house completion time, level of skilled labor, and house customer preferences that affect the australian housing supply. it is evident that more research is required to explore the applicability of off-site manufacturing (osm) as a new construction method to improve the house building supply in australia (blismas & wakefield, 2009). in this research, the aim is to enhance the osm uptake in australia by integrating lean and agile concepts. four leagile strategies are introduced to manage the osm house building supply chain. the results of this study will answer the research question: “how can house builders select suitable leagile strategies?” the outcome of the research could lead to the adoption of suitable osm strategies for australian house builders. the cumulative housing supply shortage has been predicted and confirmed by the housing industry alliances such as the national housing supply council (nhsc) and the housing industry association (hia) (coag, 2012; dalton et al., 2011). the housing supply and affordability report produced by nhsc (2013) projects the gaps between the underlying demand and supply in housing from all scenarios between june 2011 and june 2031. in a low build rate situation the difference between demand and supply is forecasted to reach 415,000, 943,000 and 1,558,000 dwellings in low, medium, and high demand growth, respectively. for the high demand growth, the expected shortage of dwellings will be 1,558,000, 1,050,000, and 447,000 dwellings, respectively. it is predicted that the practice of osm will gradually play a major part in australian house building in the coming decades (blismas & wakefield, 2009; hampson & brandon, 2004). the opportunities to adopt osm in australia are centered on detached houses, high-density multi-residential complexes, and public facilities such as hospitals and schools. the development of osm will expand to walling systems, modularized housing and light weight concrete wall panels. despite the potential of osm in australia, the uptake of osm has been limited due to some barriers such as the builders conservatism influenced by the limited success in the past, high fragmentation in the construction industry, a lack of codes and standards, loss of control on-site and into the supply chain, lack of skilled labors, and insufficient industry investment in research and development. employing some concepts such as lean and agile manufacturing could contribute to the success of enhancing osm uptake in the house building industry (manley et al., 2009). four different leagile strategies introduced in this paper are make-to-stock (mts), assemble-to-order (ato), design-to-order (dto) and self-build-house (sbh). using the analytic network process (anp) will enable house builders to select the most appropriate strategy that fits their supply conditions. ijahp article: mostafa, abdelhamid, chileshe, dumrak/decision support model using anp to align leagile strategies to off-site manufacturing in australia international journal of the analytic hierarchy process 376 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.340 this paper is organised into eight consecutive sections. after the introduction, the second section demonstrates the research aim and objectives. the third section reviews the related literature on australian housing supply factors, osm, and lean and agile concepts. section four explains the four leagile strategies for osm and the related case studies. the fifth section summarises the research methodology. the sixth section introduces the proposed anp model which is applied to study the four leagile strategies. the seventh section discloses the results delivered by the anp model based on the data of 258 houses built in five australian states including western australia (wa), south australia (sa), new south wales (nsw), victoria (vic), and queensland (qld). in the final section, the research conclusion provides recommendations for further research. 2. research aim and objectives the aim of this study is to introduce osm house building supply strategies to australian house builders. to achieve this, four research objectives have been developed: 1) highlight the factors contributing to australian housing supply. 2) discuss the potential of osm in australia. 3) present an integrative framework of lean and agile in osm supply chain. 4) develop a decision support model using anp for leagile strategies selection in order to enhance the osm adoption in australia. 3. literature review 3.1 factors influencing australian housing supply the housing supply in australia is comprised of several stages which can influence the housing supply and demand such as strategic planning and development, land release, building approval, construction commencement and completion, strata title registration and availability for occupation (nhsc, 2013). the main focus of this paper is on the house construction process which starts from the commencement of house building and concludes with house completion as the latest statistics of abs (2014) indicated that the number of houses completed is lower than the number of houses commenced. therefore, identifying the factors affecting the house supply that exist in the construction process of the house is a crucial step in achieving the aim of this paper. the main factors discussed are as follows: completion time for new houses, house customer preferences, level of skilled labor and house construction costs. 3.1.1 completion time for new houses the house completion time is the time period between the first and last physical building activity to make a house ready for occupation (dalton et al., 2013). the house completion time is a major factor indicating the quality of housing delivery to house buyers. there is an increase in the average australian house completion time, while the production rate has been found to be relatively stable. the statistics of abs (2014) reported the average completion times of new houses in australian states, territories, and at the national level. using 2003 to 2008 as a base line, the states of new south wales, victoria, south australia, tasmania, and the northern territory experienced an increasing house completion time in 2008 to 2013. the house completion time remained the same in the states of queensland and western australia, and australian capital territory. the completion time progressively increased in all regions in 2003 to 2008. the states of western australia and victoria were recorded as having the longest house completion time at approximately 3.3 ijahp article: mostafa, abdelhamid, chileshe, dumrak/decision support model using anp to align leagile strategies to off-site manufacturing in australia international journal of the analytic hierarchy process 377 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.340 quarters in 2013. the house completion time in the states of nsw, sa and tas is around 2.8 quarters. at the national level, the completion time has been increased from two quarters in 2005 to be three quarters in 2013. 3.1.2 house customer preferences the house type and design are the main factors in a house buyer’s preference (nhsc, 2013). house preferences may vary from person to person based on, but not limited to, household age and income, and family size. house preferences include the size, internal and external design, and location of the house. the average floor area australian dwelling has increased over time. for example, the average floor area of new detached houses increased from 162.4 m 2 to 248.0 m 2 from 1984 to 2009 (abs, 2012). it is evident from an examination of volume builders catalogues such as metricon, one of the largest 20 home builders in australia, that the building of double-story houses and more complex street-facing façades has increased (hia 2013). 3.1.3 level of skilled labor house building is a labor intensive industry with its main product being new dwellings or renovated dwellings. the supply of labor is an important element of the housing supply. according to deewr (2012), shortages reported from 2008 to 2012 in some construction trades were roof tiller, glazier, plumbers and cabinetmakers. house builders are working in a competitive environment in which skilled labor is required. the challenges in house building include new working relationships such as partnering and virtual enterprise as well as changing construction technologies and adopting modern methods of construction (daly, 2009). it can be concluded that skilled labor is an essential component of the house building industry in order to successfully overcome all the mentioned challenges. on the other hand a skills shortage contributes to the undersupply of housing (nhsc, 2013). 3.1.4 house construction costs housing prices are a critical element in determining new housing construction. in australia, house prices have increased in all locations at a similar rate of growth. the abs (2014) reported that the houses price index (hpi) for the weighted average of the eight capital cities increased by 3.4% during the december 2013 quarter. this led to an increase in the average hpi of the eight capital cities by 9.3% during the financial year 2012-2013. the housing supply is a function of the house price. the house price is comprised of the price of land, construction costs, and lagged house stock. the growth of house prices is driven by the increase in the prices of established houses. the study of liu and london (2011) stated that the construction costs are responsible for a higher proportion of the increase in house prices in some regions. this study concluded that the construction costs are a significant component of the poor performance of the australian new housing supply. 3.2 off-site manufacturing in australia in order to respond to the housing shortage, builders are looking for more efficient materials and new methods of construction that can reduce the completion time. one example of a new material is the use of cross-laminated timber (clt) instead of traditional clay bricks. a new method in house building is employing off-site manufacturing (osm) with structural insulated panels (sips) (nhsc, 2013). osm refers to the fabrication of house components in an offsite factory as well as their subsequent activities on a construction site (goulding et al., 2012). it provides several benefits including improving onsite safety by providing a cleaner and tidier construction site as well as enhancing quality of the house components under ijahp article: mostafa, abdelhamid, chileshe, dumrak/decision support model using anp to align leagile strategies to off-site manufacturing in australia international journal of the analytic hierarchy process 378 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.340 controlled factory production. furthermore, osm reduces environmental effects by reducing waste generation, shortening lead time and increasing efficiency and productivity (pan & goodier, 2012). there are four categories of osm based on the degree of offsite work. these categories are: component manufacture and subassembly which are always made in a factory and never considered for onsite construction (e.g. door, trusses, windows); non-volumetric pre-assembly (panels) which are pre-assembled units which do not enclose usable space (e.g. wooden panels and structural insulated panels); volumetric pre-assembly (pods) which are preassembled units which enclose usable space and are typically fully factory finished internally, but do not form the buildings structure (e.g. bathroom and kitchen pods); and modular systems which are pre-assembled volumetric units which also form the actual structure and fabric. previous studies have positively addressed osm in the australian built environment. hampson and brandon (2004) identified osm as a key vision for improving the construction industry by 2020. manley et al. (2009) confirmed that osm has the capability to produce high-volume and high-quality houses based on the efficiencies of the manufacturing principles. likewise, khalfan and maqsood (2014) recommended adopting osm in the australian residential sector for enhancing house affordability, reducing construction time and improving quality. chandler (2014) contended that the underperformance of the australian construction industry could be improved through including the adaption of osm. he also mentioned that there could be an increase in the employment of osm by 2023, which could be achieved through sourcing 15-20% of the total industry turnover offshore. despite the benefits of osm, the uptake in australia is limited due to some challenges which are similar to those in different contexts (blismas & wakefield, 2009). some major challenges are related to concurrent management of two working sites (chang & lee, 2004). these challenges include the potential of insufficient coordination between the offsite and onsite activities, the jumbled on-site processes due to a difference between the production flow at offsite factories and construction flow on-site, and the vague demands from undecided customers. in addition, khalfan and maqsood (2014) highlighted some challenges related to the industry such as the lack of skilled australian supply chain partners and the lack of scale in the residential sector. the non-value added activities (wastes) in the production of house components/modules present further challenges. all of these challenges might lead to a slower response to customer order completion. addressing these challenges has typically followed the implementation of successful concepts from the manufacturing industry, particularly lean and agile concepts (blismas, 2007; mhra, 2003; vidalakis et al., 2013). 3.3 lean and agile concepts in osm 3.3.1 lean concept lean manufacturing was first developed as part of the toyota production system (tps) and later expanded to be known as lean production. lean is an integrated socio-technical system comprised of management practices that focus on eliminating waste from business and production processes so that the time between the customer order and actual product delivery is reduced to the shortest possible time (shahet al., 2008). this is a time to market focused strategy. lean thinking contains five general principles including defining value from the customer perspective, mapping the value stream to achieve the predefined value, creating the flow along the value stream, establishing pull systems and pursuing perfection (womack & jones, 2003). the lean ijahp article: mostafa, abdelhamid, chileshe, dumrak/decision support model using anp to align leagile strategies to off-site manufacturing in australia international journal of the analytic hierarchy process 379 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.340 principles are applied through using a set of tools to identify and eliminate waste in the value stream (mostafa et al., 2013). the lean tools include just-in-time (jit), batch reduction, facility layout, value stream mapping (vsm), visual management system, production levelling, pull production system, total productive maintenance (tpm), quick changeover, standard work, error proofing, kanban, and kaizen (shah & ward, 2003). lean thinking as an application into the construction environment was first discussed by koskela in 1992 (mossman, 2009). a transformation-flow-value concept of production has been developed as a new perspective to improve facility construction performance (koskela, 1992a). according to the concept, the construction production consists of three corresponding processes: a transformation of materials into standing structures, a flow of the materials and information through various production processes and a value creation for customers through the elimination of value loss (bertelsen, 2002; abdelhamid, 2004; pasquire, 2012a). 3.3.2 agile concept the agile manufacturing concept, on the other hand, became popular in 1991. sharifi and zhang (1999) stated that a new competitive environment is a key driver for changes in the manufacturing industry. the competition qualities are continuous improvement, rapid response and quality improvement. the researchers at the iacocca institute in lehigh university (usa) defined the agile concept (yusuf et al., 1999) as: a manufacturing system with extraordinary capabilities (internal capabilities: hard and soft technologies, human resources, educated management, information) to meet the rapidly changing needs of the marketplace (speed, flexibility, customers, competitors, suppliers, infrastructure, responsiveness). a system that shifts quickly (speed and responsiveness) among product models or between product lines (flexibility), ideally in real-time responding to customer demand (customer needs and wants). the agile principles include organizing in order to master change and uncertainty, leveraging the impact of people and information, cooperating to enhance competitiveness and enriching the customer (devor et al., 1997; gunasekaran et al., 2002). the agile concept has three dimensions: drivers, enablers and capabilities as demonstrated in figure 1. the figure is inspired by the work of sharifi and zhang (1999) for the concept of agility drivers interacting with the agility enablers to deliver agility capabilities. however, the projected figure contains drivers and enablers proposed in other research. the agility drivers have been identified in zhang and sharifi (2007) whilst the four categories of agility enablers have been highlighted and discussed in yusuf et al. (1999), gunasekaran et al. (2002) and sharp et al. (1999). ijahp article: mostafa, abdelhamid, chileshe, dumrak/decision support model using anp to align leagile strategies to off-site manufacturing in australia international journal of the analytic hierarchy process 380 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.340 figure 1. agile concept drivers, enablers and capabilities the key drivers of adopting the agile concept are increasing turbulence of the business environment, changes in customer requirement and advancement in technology. the agile capabilities refer to the capabilities that an organization needs to attain to be able to respond to the agility drivers. the capabilities are flexibility, responsiveness, speed, partnership and competency (zhang & sharifi, 2007). agility enablers consist of business practices, methods and tools which enable an organisation to acquire the agile capabilities (sharifi & zhang, 1999). the tools/practices are grouped into four areas: strategies, technologies, systems and human resources. the initiative of agile construction was established in direct response to the latham report published in 1994 (lee, 2003). the report highlighted the uk construction industry requirement to reduce the construction cost by 30% by the year 2000. to achieve this target, the entire industry needed to change. benchmarking has been a method used to stimulate the required change in the construction practices. naim et al. (1999) suggested the employment of agile principles in the construction supply chains to achieve profitable opportunities in dynamic markets. agile construction exemplifies the characteristics of visibility, responsiveness, productivity and profitability (daneshgari, 2010). 3.3.3 leagile concept the integration of lean and agile is one of the better solutions to answer any changes in the world class market competition (agarwal et al., 2006). combining lean and agile within the whole supply chain can be accomplished by using the decoupling point (dp) concept. it is known as leagility. the leagility term was first introduced by naylor et al. (1999). in general, the dp separates the leagile supply chain into lean in the upstream and agile in the downstream (mason-jones & towill, 1999). for market s tr a te g ie s v ir tu a l e n te rp ri se , c o n c u rr e n t e n g in e e ri n g r e -c o n fi g u ra b il it y , st ra te g ic a ll ia n c e s, re e n g in e e ri n g , su p p ly c h a in i n te g ra ti o n , re sp o n si v e l o g is ti c s t e c h n o lo g ie s r a p id h a rd w a re , m o d u la r a ss e m b ly s o ft w a re , in fo rm a ti o n t e c h n o lo g y ( c a d /c a m ) s y st e m s d e si g n s y st e m s, p ro d u c ti o n p la n n in g a n d c o n tr o l sy st e m s, s y st e m s in te g ra ti o n a n d d a ta b a se m a n a g e m e n t h u m a n r e so u r c e s t ra in in g a n d e d u c a ti o n , e m p lo y e e e m p o w e rm e n t, k n o w le d g e w o rk e rs , to p m a n a g e m e n t su p p o rt agility drivers (increasing turbulence of the business environment, advancement in technology and customer dynamic requirements) a g il it y e n a b le r s agility capabilities flexibility, responsiveness, speed, partnership and competency ijahp article: mostafa, abdelhamid, chileshe, dumrak/decision support model using anp to align leagile strategies to off-site manufacturing in australia international journal of the analytic hierarchy process 381 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.340 competition, christopher and towill (2000) emphasised that supply chains must be responsive to market demand changes which can be divided into three critical dimensions; variety, variability (or predictability) and volume. the lean concept is the better alternative where there are high volumes, low variety, and low predictable change environments. conversely, the agile concept is the better option where there are low volumes, high variety, and high predictable change environments. the real demand visibility is limited in most supply chains. the supply chains may be lean prior to dp and agile beyond dp. there are two dps in the leagile supply chains (christopher & towill, 2000). the first dp is the material dp which should ideally lie as far down stream as possible to be close to the final marketplace. the second dp is the information dp which should lie as far upstream as possible in the supply chain. agility beyond the decoupling point is explained by the principle of postponement using a generic or modular inventory to postpone the final commitment while the final assembly or customisation depends on real demand. a leagile supply chain has capabilities to achieve value through different strategies in accordance with the dp positions for the house customer. the leagile house building supply chain mainly focuses on waste removal and responsive mechanisms through applying the excellent practices lean and agile have to offer. the studies of childerhouse et al. (2000) and naim and barlow (2003) focus on using the material dp in the uk house building supply chain. in this paper, the leagile house building supply chain employs the customer order decoupling point (codp) or order penetration point which encompasses both information and materials. the material dp is the stocking point of finished house modules or components. the information dp is the point where the customer demand enters the value chain. 3.3.4 application of lean and agile in osm lean concept is comprised of management practices that focus on eliminating all forms of waste from the value stream (sertyesilisik, 2014). the concept has been widely adopted beyond its origin in automobile manufacturing. kenley (2014) emphasized improving the productivity of the construction industry through production systems intervention. lean production concept is the best known intervention. it has been used by house manufacturers in japan by transferring the knowledge from automobile manufacturing to house manufacturing (barlow & ozaki, 2005). the practice of lean concept in house building requires using factory based production. however, the construction has unique characteristics (i.e., features of output, nature of processes, customer involvement, and supply chain). therefore, lean construction, as extended by koskela (1992b), addresses these specific characteristics. the main challenge of lean construction is related to the interfaces between the offsite factory and the construction site. the production flow at the offsite factory is continuous and different from the construction site which is turbulent. this is due to uncertainties at the construction site such as changes in customer demand or site conditions. this leads to unpredictable delays to achieve the customer order. agile construction was proposed to proactively respond to any onsite uncertainties (daneshgari, 2010). lean construction focuses on creating an efficient physical process of manufacturing (pasquire, 2012b). agile, on the other hand, emphasizes a high level of service through flexibility and customization (naim & barlow, 2003). these factors are important for osm as osm implies standardization of products and processes, and emphasizes flexibility for house customers. ijahp article: mostafa, abdelhamid, chileshe, dumrak/decision support model using anp to align leagile strategies to off-site manufacturing in australia international journal of the analytic hierarchy process 382 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.340 some concerns were found in the existing literature in applying lean or agile as a standalone concept when uncertainties in construction are present (christopher & towill, 2001). many studies suggested a combination of lean and agile concepts in osm (blismas & wakefield, 2009; mhra, 2003). however, these studies were conducted in a different context than the australian house building environment. it was further discovered that no specific lean and agile integration strategy for osm in australia was formulated. combining lean and agile within the whole supply chain can be accomplished by using the decoupling point strategy known as leagile (purvis, gosling, & naim, 2014). in general, the decoupling point separates the supply chain into lean in the factory site and agile in the construction site. 3.3.5 current gap of knowledge on lean and agile concepts in osm the principles of lean and agile are easily extended to different types of organizations (womack & jones, 2003). they can be combined appropriately within designed and operated total supply chains via decoupling points (agarwal et al., 2007). nevertheless, research efforts focused on the shortcomings as well as strengths of such a combination has been inadequate. combining lean and agile practices without a full understanding of their power and limitations may result in major errors. the definition of waste in agile is different from that appropriate to lean therefore, whatever is considered waste in a lean concept may be an essential practice in agile concept (towill & christopher, 2002). capacity requirements are one example of this difference according to mason-jones, naylor, & towill (2000). according to lean concept, unnecessary inventory is waste however; from an agile point of view it is recommended that certain levels of inventory should be sustained to handle unpredictable demands. in the house building sector, synergizing lean and agile concepts may require more examination into their effects on reducing house completion time and construction costs overrun. osm consists of several interrelated sub-areas needed in delivering houses according to customer demands. table 1 presents a summary of the existing literature on the manufacturing application concepts suggested for house building. among the five major countries which are frequently included in house building studies, lean and agile are suggested the most. nevertheless, the integration of these two principles seems insufficient. naim and barlow (2003) suggested leagile supply chain strategies for the uk house building. the strategies are based on using a material decoupling point to separate lean and agile. however, their study has not demonstrated the codp which is regarded as a significant component for the house customers’ preferences and has excluded the practices of lean and agile required for house building supply chain. the literature in the australian context on lean and agile applications of osm reveals a shortage of lean and agile implementation in the house building sector. the development of osm in house building, to some extent, lacks a clear description of the concept and its related parts, including technical, organizational and processrelated issues. in order to achieve an effective building process, osm must be based on a holistic view (supply chain view). however, this can lead to consequences for the structure of the building process in terms of changes of organizational and production related conditions. the general house building process is not designed to handle the whole process as a supply chain. hence, it may require changes to both the process and the management to get the osm system to work effectively with its related parts acting together as a whole and creating maximal value for the customers (lessing,2006). ijahp article: mostafa, abdelhamid, chileshe, dumrak/decision support model using anp to align leagile strategies to off-site manufacturing in australia international journal of the analytic hierarchy process 383 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.340 table 1 literature review of the manufacturing application concepts suggested for house building countries research studies lean agile leagile u.k. re-engineering through pre-assembly (gibb & isack, 2003) * innovative supply chain for customised housing (naim and barlow, 2003) ** ** ** delivering new homes-the offsite way (arif & pannell, 2013) * offsite production a model for building down barriers (nadim & goulding, 2011) * u.s. current use of offsite construction techniques (na lu & liska, 2008) * designers' and general contractors’ perceptions of offsite construction techniques (n. lu, 2009) * whole house and building process redesign (path, 2002) * * technology roadmap for manufactured housing (mhra, 2003) * australia construction 2020 (hampson & brandon, 2004) osm in australia current state and future direction (blismas, 2007) * organizational change in australian building and construction (mcgrath-champ & rosewarne, 2009) * drivers and constraints of osm (blismas and wakefield, 2009) * innovative practices in the australian built environment (manley, mckell, & rose, 2009) * * japan building mass customised housing (barlow and ozaki, 2005) * * choice and delivery in house building (barlow et al., 2003) * * similarities and differences between industrialized housing and car production in japan (gann, 1996) * sweden industrialised house building (lessing, 2006) * * applicability of lean principles and practices (höök & stehn, 2008) ** defects in offsite construction (johnsson & meiling, 2009) * value-driven purchasing of kitchen cabinets in industrialised housing (bildsten, björnfot, & sandberg, 2011) * **: application suggested and applied within the study *: application suggested but not applied within the study ijahp article: mostafa, abdelhamid, chileshe, dumrak/decision support model using anp to align leagile strategies to off-site manufacturing in australia international journal of the analytic hierarchy process 384 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.340 4. leagile strategies for osm supply chain the osm house building supply chain suggested in this paper can be visualized as shown in figure 2. the supply chain is comprised of the house materials suppliers, an offsite factory, designers, construction site, and customers. the osm supply chain must be managed to achieve the customer order. the last planner® system (lps) is used to establish a better coordination among supply chain stakeholders to achieve the house customer demand. lps is used to transfer planning responsibility between construction organization management and the field persons. the lps facilitates the workflow so that labor and material resources can be more productive. the lps encompasses four levels of planning processes with different consecutive spans: master scheduling, phase scheduling, make-ready planning, and weekly work planning (forbes & ahmed, 2011). figure 2. osm supply chain and included leagile strategies the master schedule defines the work to be carried out over the entire duration of a project. it identifies major milestone dates and incorporates critical path method logic to determine overall project duration. phase scheduling generates a detailed schedule covering each project phase such as foundations, structural frame, and finishing. the phase employs reverse phase scheduling and identifies handoffs between the different specialty organizations to find the best way to meet milestones stated in the master schedule. the make-ready (look-ahead) planning indicates the first step of production planning with a time frame ranging from two to six weeks. at this phase, activities are broken down into the level of processes, constraints are identified, responsibilities are assigned, and assignments are made ready (hamzeh et al., 2012). the weekly work planning represents the most detailed plan in the lps showing interdependence between the works of various specialist organizations and guides the production process. at the end of each plan period, assignments are reviewed to measure the reliability of planning and the production system. analyzing reasons for plan failures and acting on these reasons is used as the basis of learning and continuous improvement. the previous research of childerhouse et al. (2000), and naim and barlow (2003) proposed a leagile model to be applied in the uk house building. the model was based on using material dp. in this ijahp article: mostafa, abdelhamid, chileshe, dumrak/decision support model using anp to align leagile strategies to off-site manufacturing in australia international journal of the analytic hierarchy process 385 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.340 paper, the leagile supply chain for australian house building employs the codp which was suggested by olhager (2013). the codp in this study represents information and material dp. the material dp represents the stocking point of finished house modules or components. the information dp denotes the point where the customer order enters the housing supply chain. in this paper, four alternative positions for codp, developing four house building strategies are suggested to be employed in the australian house building supply chain. these strategies are summarized in table 2. ijahp article: mostafa, abdelhamid, chileshe, dumrak/decision support model using anp to align leagile strategies to off-site manufacturing in australia international journal of the analytic hierarchy process 386 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.340 table 2 leagile strategies for osm supply chain strategy description codp location leagile attractiveness to customers benefits to builders make-tostock (mts)  known as speculative house  houses are designed and built based on the builders’ catalogue after the onsite construction activities and finished house building  lean is for cost-related activities before selling  agile is after house construction to reduce the delivery time lower price of a finished house  maximization on house price satisfaction  speed up the return on investment assembleto-order (ato) a variety of houses designs are available to the customers in the catalogues at the offsite factory  lean is employed within the offsite factories  agile is employed in stages of shipments and onsite construction  a degree of flexibility in selecting house components  available of mixed ‘specs’ to match demands  price of house modules and faster completion time more customer satisfaction designto-order (dto) the house design can be delivered to the customers who prefer to have their own house modules house design stage  lean is applied in supplying material and offsite operations  agile is applied for high responsiveness of other activities  more control over house preferences  flexibility to change the predesigned modules more customer satisfaction self-built house (sbh) the house owner is involved in every house building process. the house owners are at their own responsibilities to hire builders to assist them with some onsite construction activities. at house components suppliers  lean is suitable to run the factory to produce house modules,  agile is the best option for quick responses to demands of self-build house suppliers.  various designs  attractive price  high customization  full control over the construction process  opportunities as suppliers  simple designs required  meet different demands  standard components 5. research methodology to achieve the aim of this paper, an exploration of various databases obtained from australian housing bodies, i.e., national housing supply council (nhsc), housing industry association (hia), australian bureau of statistics (abs), australian housing and urban research institute (ahuri), ijahp article: mostafa, abdelhamid, chileshe, dumrak/decision support model using anp to align leagile strategies to off-site manufacturing in australia international journal of the analytic hierarchy process 387 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.340 and council of australian governments (coag) was employed. this database exploration was divided into two stages: constructing a background of osm builders in australia and identifying factors related to the four leagile strategies that could indicate osm uptake in australia. the data sets obtained from these stages were utilized in formulating decision making of osm strategy among the proposed strategies for the australian context. the collected data were examined and allocated reference numbers to facilitate the data analysis process. the process of decision making formulation was conducted using the analytic network process (anp). the exploration process resulted in data from 258 houses built in five australian states by five volume builders to be analyzed further by anp. 6. data analysis 6.1 osm builders in australia the housing 100 report for 2013 presented australia’s most active 100 builders (hia, 2013). their main housing activities contributed around 75% of the housing supply for detached houses and multiunit apartments. in this paper, the top five potential builders supplying houses in the five australian states of western australia (wa), south australia (sa), new south wales (nsw), victoria (vic), and queensland (qld) were analyzed as shown in table 3. the five builders were capable of adopting the four leagile strategies. this adoption capacity was enhanced by their house building work in the five states, market share, decision making and the future trends during the period of 20112014 (hia, 2014). the strategies would allow builders to make decisions to tailor their house building activities. the weightings of the criteria and sub-criteria for each strategy were grounded on the specifications of 258 actual houses. the specifications obtained were classified into groups under each criterion and sub-criterion before pairwise comparisons were conducted to predict the most suitable strategy. table 3 top five australian house builders’ information 2013/2014 builder a builder b builder c builder d builder e hia top 100 2013/2014 1 st 2 nd 3 rd 4 th 5 th states of australia wa wa,vic sa, vic, nsw, qld qld,vic, sa vic, sa, qld house building activity builder builder and developer builder builder and developer builder houses starts during 2013 3443 3199 2837 2432 1692 house market share 13% 12% 10.7% 9.2% 6.4% number of house models 224 102 60 56 36 6.2 leagile strategies’ weighting in each criterion and sub-criteria the imbalance between the housing supply and demand has occurred in all australian states and territories (nhsc, 2013). the five selected potential builders were capable of adopting the four leagile strategies. this adoption capacity was enhanced by their house building work in the five states, market share, decision making and the future trends during the period of 2011-2013 (hia, 2013). selecting a strategy depends on the situation of the house market in australia, given that the demand of house customers’ shapes the housing market. therefore, the builders can respond to house market changes by adopting the suitable strategy. for example, the builders might have to build small floor plan houses, less customized with a medium price for the australian low income groups (to increase housing affordability). this combination could lead to the employment of one or more strategies proposed in this paper. the australian medium income groups might prefer to select house elements and design from the available designs in the builders’ catalogues. in this case, a suitable ijahp article: mostafa, abdelhamid, chileshe, dumrak/decision support model using anp to align leagile strategies to off-site manufacturing in australia international journal of the analytic hierarchy process 388 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.340 strategy for this situation must be carefully determined. the customers have the ability to change the house design to fit their needs. therefore, customers are likely to be involved in designing all house elements. therefore, the four strategies proposed can cover different customers’ demands. the strategies allow house builders to make decisions to tailor their house building activities. the weightings of the criteria and sub-criteria for each strategy were performed through a comparison when a builder employs each strategy. the weighting criteria and sub-criteria for the four strategies are demonstrated in table 4. international journal of the analytic hierarchy process 389 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.340 table 4 leagile strategies’ weighting in each criterion and sub-criteria a1: ato a2: dto a3: mts a4: sbh c1: house price moderate range high range low price range lowmoderate range c1.1: labor cost medium high low low c1.2: construction material cost medium high low low c2: house completion time moderate long short moderateshort c2.1: construction method favorable neutral favorable very favorable c2.2: number of houses under construction moderate many few few c3: house preferences moderate very high low high c3.1: façade options medium very high limited moderate high c3.2: house floor area small floor area larger floor area small floor area suitable for any floor area c3.3: house location flexible location more flexible fixed location highly flexible c4:level of skilled labor medium to high labor intensive medium to high labor intensive medium low c4.1: contractors/sub-contractors medium to high contractors medium to high contractors medium contractors intensive requires less contractors force c4.2: trades medium to high trades medium to high trades medium labor intensive requires less labor force 6.3 anp model anp is a technique in multi-criteria decision analysis (mcda) based on relative assessment of both tangible and intangible criteria (ozdemir, 2005). it is considered an expansion of the ahp for representing and analyzing a network of decision making. anp is an easy technique to apply, and allows for a direct calculation of the combined effects of all the factors, utilizing a markovian process and a more complete set of relationships that are allowed to flow through the network (saaty & shang, 2007). anp can be incorporated with other optimization approaches such as fuzzy and multi objective optimization (saaty & sodenkamp, 2008; bijan, keramati, & salehi, 2014). its pairwise comparisons between the network elements are completed using a decomposition approach that reduces the decision-making errors (ozdemir, 2005). applying anp reduces the rank-reversal problem (sarkis, 2003). according to the advantages stated, anp is employed in this study for facilitating the selection of four leagile house building strategies with respect to the key factors affecting the housing supply in australia. the simple network model for associating the main house undersupply factors and leagile strategies is demonstrated in figure 3. ijahp article: mostafa, abdelhamid, chileshe, dumrak/decision support model using anp to align leagile strategies to off-site manufacturing in australia international journal of the analytic hierarchy process 390 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.340 figure 3. anp model to align leagile strategies with house supply factors in australia the model contains five clusters. the first cluster represents the house completion time factor. under house completion time, there are two nodes namely number of houses under construction (nhuc) and house construction method. the house building costs cluster includes material and labor costs nodes. the third cluster is the level of skilled labor which includes contractors and trade persons. the house customer preferences cluster contains three nodes of house floor area, house location and façade options. the last cluster represents the alternatives which are the four leagile strategies. the model network, comparisons and assessments among the clusters and nodes were created and performed using super decisions software. the graph of the dependencies among the decision model criteria is demonstrated in figure 4. the codes of the criteria used in the figure are demonstrated in table 4. the adjacency matrix can be obtained for the binary relation showed in figure 4. the matrix shows the contextual relationship among the criteria/clusters of the decision model. the matrix elements are wither 1 or 0, respectively, whether a pair of nodes is directly connected or not. ijahp article: mostafa, abdelhamid, chileshe, dumrak/decision support model using anp to align leagile strategies to off-site manufacturing in australia international journal of the analytic hierarchy process 391 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.340 figure 4. the dependencies among the criteria of the decision model 6.4 pairwise comparisons anp can be applied by using super decisions© software to ease mathematical calculations. the software has advantages including a user friendly environment, an evaluation of inconsistency index of assessments, and a sensitivity analysis of results. pairwise comparisons were performed to associate the relationships between all elements at all levels of the anp network. all pairwise numerical comparisons were performed in the super decisions software. saaty’s 1-9 scale allows the comparison of two elements in the hierarchy using verbal or numerical judgments as equally (i.e., has a value of 1), moderately (i.e., has a value of 3), strongly (i.e., has a value of 5), very strong (i.e., has a value of 7), and extremely (i.e., has a value of 9). the intermediate values are used where appropriate as equally to moderately (i.e., has a value of 2), moderately to strongly (i.e., has a value of 4), strongly to very strongly (i.e., has a value of 6), very strongly to extremely (i.e., has a value of 8) (armacost, componation, et al., 1994; saaty, 1980). 6.4.1 cluster comparisons the anp model clusters are compared with respect to each other in order to evaluate the priorities. then the priorities are used to weight the blocks in each column of the supermatrix to make its column stochastic (saaty & sodenkamp, 2008). the cluster matrix is presented in table 5. c1 c2 a c3 c4                  01111 10111 11011 11101 111100 matrixadjacency 4 3 2 1 4321 a c c c c accc c ijahp article: mostafa, abdelhamid, chileshe, dumrak/decision support model using anp to align leagile strategies to off-site manufacturing in australia international journal of the analytic hierarchy process 392 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.340 table 5 cluster matrix weight alternatives c1 c2 c3 c4 alternatives 0.00 0.50 0.50 0.50 0.50 c1 0.18 0.00 0.00 0.00 0.00 c2 0.28 0.00 0.50 0.00 0.00 c3 0.24 0.00 0.00 0.50 0.00 c4 0.3 0.00 0.00 0.00 0.50 6.4.2 comparisons of elements and alternatives the unweighted supermatrix is constructed from the priorities derived from the different pairwise comparisons (saaty & sodenkamp, 2008). the matrix rows and columns contain the cluster nodes labels which are the alternatives and criteria (shown in table 5). the column of priorities for a node at the top of the supermatrix includes the priorities of the nodes on the left side of the matrix that have been compared with respect to leagile strategies on that node. the summation of these priorities is equal to one. the weights derived from the unweighted supermatrix (appendix a) are used to develop the columns in the weighted supermatrix (appendix b). each column is a normalized eigenvector of the unweighted supermatrix with some zero records (bijan et al., 2014). then, the unweighted supermatrix is multiplied by the cluster priority weights (appendix a). finally, the limit supermatrix has been developed using the same process as in the weighted supermatrix. as shown in appendix c, all the columns of the limit supermatrix are identical. 6.5 final priorities after pairwise comparisons, the priorities were synthesized from the goal while the overall priorities were calculated. the overall priorities of the anp model elements are displayed in appendix c. the sbh strategy was considered as the best alternative which received the highest rating of 0.163. the second best strategy was mts which scored 0.092, followed by ato with a score 0.070. the last strategy was dto with a score of 0.069. the priorities of the other factor in the cluster of the housing supply factors were also provided in table 6. the nhuc was considered as the most effective factor among other factors. the nhuc scored 0.124. the shortage of trades and labor costs come as the second and the third most effective factors with a score of 0.118 and 0.071. ijahp article: mostafa, abdelhamid, chileshe, dumrak/decision support model using anp to align leagile strategies to off-site manufacturing in australia international journal of the analytic hierarchy process 393 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.340 table 6 final priorities of the elements of the anp model element name priority vector a4 sbh 0.163 c2.2 nhuc 0.124 c4.2 trades 0.118 a3 mts 0.092 c1.1 labor cost 0.071 a1 ato 0.070 c4.1 contractors/sub-contractors 0.070 a2 dto 0.069 c3.3 house location 0.054 c3.1 façade options 0.050 c3.2 house floor area 0.040 c1.2 material costs 0.038 c2.1 method of construction 0.034 7. results and discussion 7.1 sensitivity analysis a sensitivity analysis is conducted in order to determine the stability of the preference ranking among the alternative websites by changing the priority weights of the criteria. if the ranking does not change, the results are said to be robust. in this study, sensitivity is performed by varying the priority of the reliability of ato strategy by moving the vertical line and determining the corresponding alternatives priorities. figure 5a shows the graphical representation of the sensitivity analysis when the priority of ato strategy is 0.5. the rating of the alternatives is ato, sbh, mts, and dto respectively. when the priority of the ato is reduced to 0.1 (shown in fig. 5b), the rating of the alternatives has changed. the sbh strategy received the higher rank followed by ato and mts, and dto. moreover, the sensitivity analysis was performed between the alternatives rating and other anp elements as presented in figure 6. it was found that changing the priority of any factor has no effect on the alternatives ranking. the ranking was found stable as sbh, mts, ato, and dto respectively. therefore, sbh is considered as the most suitable leagile strategy for all housing supply factors. ijahp article: mostafa, abdelhamid, chileshe, dumrak/decision support model using anp to align leagile strategies to off-site manufacturing in australia international journal of the analytic hierarchy process 394 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.340 a) ato priority=0.5 b) ato priority=0.1 figure 5. sensitivity graphs for the reliability of leagile strategies when ato priority is 0.5 and 0.1 figure 6. sensitivity graph between alternatives and other anp model factors ijahp article: mostafa, abdelhamid, chileshe, dumrak/decision support model using anp to align leagile strategies to off-site manufacturing in australia international journal of the analytic hierarchy process 395 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.340 7.2 implications of the anp model the results from the examination of the anp model showed that sbh strategy performed most effectively among the four factors affecting the housing supply in australia. the sbh strategy could be suggested to australian house builders according to the research results as the strategy that is the most suitable under different combinations of housing supply factors. these results are supported by the hia (2013) report the showed that the largest 100 builders commenced about 36% of all residential dwellings in australia during 2012-2013which indicates that 64% of all residential dwellings have been constructed by small builders or in the form of self-building houses. according to the case study of the state of victoria presented earlier, a group self-build initiative was introduced to support individuals building their own houses (state government of victoria, 2014). a key role of osm in australia is to supply a variety of house modules and components to house-module suppliers so that osm could meet the different types of house needs. the sbh strategy is the best at achieving minimum house cost which enhances the house affordability for lowand medium-income australians. the research results showed that mts strategy ranked as the second among the four alternatives. this strategy could be used for mass house building projects where the builders may have to complete the project within a strict contract timeframe. nevertheless, this research showed that the mts strategy was the least preferred alternative for house customization. noticeably, a major drawback of the mts strategy is the fact that it offers low/no house customization options. however, the strategy may be suitable for the construction of standard house designs. 7.3 limitations and future research this study utilized the actual specifications data from 258 houses built by the top five australian builders. each specification was extracted and placed according to the categories established. the development of the anp model in this study was subject to the data released by the builders and secondary data sources on australian house building. more housing undersupply factors could be further discovered and added to a future study to extend the research boundaries. other factors such as coordination among the stakeholders, land supply, and demographic factors (e.g., economic circumstances of household, number of overseas migrations) may be included. moreover, future research could conduct surveys with the australian house building experts (e.g., house builders, residential developers, architects, and house owners) in order to verify or to refine the anp model displayed in this study. 8. conclusions the australian house building sector has experienced a shortage in housing supply. the house customer preferences, house building costs, completion time and level of skilled labor add more complexities to the design specifications. furthermore, house customer demands are ambiguous and change dynamically. the four strategies proposed in this paper attempted to respond to the factors causing supply shortage and to balance the trade-offs between needs of house builders and customers. this study was carried out using the anp model to facilitate the selection of the leagile osm strategies with respect to the main factors contributing to the shortage of housing supply in australia. the findings from the anp model indicated that sbh was the most suitable strategy among the four strategies proposed for the combination of factors tested. therefore, this strategy should be suggested to the australian house builders. ijahp article: mostafa, abdelhamid, chileshe, dumrak/decision support model using anp to align leagile strategies to off-site manufacturing in australia international journal of the analytic hierarchy process 396 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.340 references abdelhamid, t. (2004). the self-destruction and renewal of lean construction theory: a prediction from boyd’s theory. paper presented at the proceedings of 12th annual conference of the international group for lean construction, copenhagen, denmark. abs. (2012). 2012 year book australia. canberra, act: australian bureau of statistics. abs. (2014). average dwelling completion times. in a. b. o. statistics (ed.), building activity, australia. canberra: abs. agarwal, a., shankar, r., & tiwari, m. k. (2006). modeling the metrics of lean, agile and leagile supply chain: an anp-based approach. european journal of operational research, 173(1), 211-225. doi: 10.1016/j.ejor.2004.12.005 agarwal, a., shankar, r., & tiwari, m. k. (2007). modeling agility of supply chain. industrial marketing management, 36(4), 443-457. doi: 10.1016/j.indmarman.2005.12.004 arif, m., & pannell, i. (2013). delivering new homes – the offsite way. salford uk: buildoffsite. armacost, r. l., componation, p. j., mullens, m. a., & swart, w. w. (1994). an ahp framework for prioritizing customer requirements in qfd: an industrialized housing. iie transactions, 26(4), 7279. doi: 10.1080/07408179408966620 barlow, j., childerhouse, p., gann, d., hong-minh, s., naim, m., & ozaki, r. (2003). choice and delivery in housebuilding: lessons from japan for uk housebuilders. building research & information, 31(2), 134-145. doi: 10.1080/09613210302003 barlow, j., & ozaki, r. (2005). building mass customised housing through innovation in the production system: lessons from japan. environment and planning a, 37(1), 9-20. doi: 10.1068/a3579 bertelsen, s. (2002). bridging the gap–towards a comprehensive understanding of lean construction. paper presented at the proceedings of the 10th annual conference of the iglc, gramado, brazil. bijan, y., keramati, a., & salehi, m. (2014). comparison of user satisfaction of ecommerce websites by the analytic network process. international journal of the analytic hierarchy process, 5(2), 223251. doi: http://dx.doi.org/10.13033/ijahp.v5i2.180 bildsten, l., björnfot, a., & sandberg, e. (2011). value-driven purchasing of kitchen cabinets in industrialised housing. journal of financial management of property and construction, 16(1), 73-83. doi: 10.1108/13664381111116106 blismas, n. (2007). off-site manufacture in australia: current state and future directions. brisbane, qld: cooperative research centre for construction innovation. blismas, n., & wakefield, r. (2009). drivers, constraints and the future of offsite manufacture in australia. construction innovation: information, process, management, 9(1), 72-83. doi: 10.1108/14714170910931552 chandler, d. (2014). a case for an australian construction strategy, commonwealth government productivity and industry discussion paper (discussion paper). retrieved 20 october 2014, from constructionedge http://constructionedge.com.au/a-case-for-an-australian-construction-strategy/ http://dx.doi.org/10.13033/ijahp.v5i2.180 http://dx.doi.org/10.1108/14714170910931552 http://constructionedge.com.au/a-case-for-an-australian-construction-strategy/ ijahp article: mostafa, abdelhamid, chileshe, dumrak/decision support model using anp to align leagile strategies to off-site manufacturing in australia international journal of the analytic hierarchy process 397 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.340 chang, a. s., & lee, k. p. (2004). nature of construction technology. paper presented at the proceedings of the 12th annual conference of the iglc, elsinore , denmark. childerhouse, p., hong-minh, s. m., & naim, m. m. (2000). house building supply chain strategies: selecting the right strategy to meet customer requirements. paper presented at the proceedings of the 8th annual conference of the iglc, brighton, uk. christopher, m., & towill, d. (2001). an integrated model for the design of agile supply chains. international journal of physical distribution & logistics management, 31(4), 235-246. doi: 10.1108/09600030110394914 christopher, m., & towill, d. r. (2000). supply chain migration from lean and functional to agile and customised. supply chain management: an international journal, 5(4), 206-213. coag. (2012). housing supply and affordability reform. canberra act, australia: council of australian governements. dalton, t., hurley, j., gharaie, e., wakefield, r., & horne, r. (2013). australian suburban house building: industry organisation, practices and constraints (a. h. a. u. r. institute, trans.). melbourne, australia: australian housing and urban research institute. dalton, t., wakefield, r., & horne, r. (2011). australian suburban house building: industry organisation, practices and constraints australian housing and urban research institute positioning paper series, 1-56. daly, g. (2009). prefabricated housing australia: skill deficiencies and workplace practice. melbourne: international specialised skills institute. daneshgari, p. (2010). agile construction for the electrical contractor. sudbury, ma: jones & bartlett learning. deewr. (2012). skill shortages in construction trades 2012 (e. a. w. r. department of education, trans.). canberra: department of education, employment and workplace relations. devor, r., graves, r., & mills, j. j. (1997). agile manufacturing research: accomplishments and opportunities. iie transactions, 29(10), 813-823. doi: 10.1080/07408179708966404 dowling, r. (2005). residential building in australia, 1993–2003. urban policy and research, 23(4), 447-464. forbes, l. h., & ahmed, s. m. (2011). modern construction : lean project delivery and integrated practices. boca raton, fl: crc press. gann, d. m. (1996). construction as a manufacturing process? similarities and differences between industrialized housing and car production in japan. construction management and economics, 14(5), 437-450. doi: 10.1080/014461996373304 gibb, a., & isack, f. (2003). re-engineering through pre-assembly: client expectations and drivers. building research & information, 31(2), 146-160. doi: 10.1080/09613210302000 goulding, j., nadim, w., petridis, p., & alshawi, m. (2012). construction industry offsite production: a virtual reality interactive training environment prototype. advanced engineering informatics, 26(1), 103-116. doi: http://dx.doi.org/10.1016/j.aei.2011.09.004 http://dx.doi.org/10.1016/j.aei.2011.09.004 ijahp article: mostafa, abdelhamid, chileshe, dumrak/decision support model using anp to align leagile strategies to off-site manufacturing in australia international journal of the analytic hierarchy process 398 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.340 gunasekaran, a., tirtiroglu, e., & wolstencroft, v. (2002). an investigation into the application of agile manufacturing in an aerospace company. technovation, 22(7), 405-415. doi: http://dx.doi.org/10.1016/s0166-4972(01)00039-6 hampson, k. d., & brandon, p. (2004). construction 2020-a vision for australia's property and construction industry. brisbane, qld: crc construction innovation. hamzeh, f., ballard, g., & tommelein, i. (2012). rethinking lookahead planning to optimize construction workflow. lean construction journal, 15-34. hia. (2014). housing 100 : australia's largest homebuilders and residential developers 2013/2014. in h. e. group (ed.), housing 100. campbell act: housing industry association. höök, m., & stehn, l. (2008). applicability of lean principles and practices in industrialized housing production. construction management and economics, 26(10), 1091-1100. doi: 10.1080/01446190802422179 johnsson, h., & meiling, j. h. (2009). defects in offsite construction: timber module prefabrication. construction management and economics, 27(7), 667-681. doi: 10.1080/01446190903002797 kenley, r. (2014). productivity improvement in the construction process. construction management and economics, 32(6), 489-494. doi: 10.1080/01446193.2014.930500 khalfan, m. m. a., & maqsood, t. (2014). current atate of off-site manufacturing in australian and chinese residential construction. journal of construction engineering, 2014, 1-5. doi: http://dx.doi.org/10.1155/2014/164863 koskela, l. (1992a). application of the new production philosophy to construction cife technical report (stanford university), 72, 81. koskela, l. (1992b). application of the new production philosophy to construction (d. o. c. e. center for integrated facility engineering, trans.). stanford university, ca. lee, s. (2003). agile report: the agile peformance improvement databse: a tool to improve construction performance. bath, uk: school of management, university of bath. lessing, j. (2006). industrialised house-building concept and processes. (licentiate thesis), lund, sweden: lund university. liu, j., & london, k. (2011). analysing the relationship between new housing supply and residential construction costs with regional heterogeneities. australasian journal of construction economics and building, 11(3), 58-67. doi: 10.5130/ajceb.v11i3.2174 lu, n. (2009). the current use of offsite construction techniques in the united states construction industry. proceedings of the construction research congress 2009, 946-955. doi: 10.1061/41020(339)96 lu, n., & liska, r. w. (2008). designers' and general contractors' perceptions of offsite construction techniques in the united state construction industry. international journal of construction education and research, 4(3), 177-188. doi: 10.1080/15578770802494565 manley, k., mckell, s., & rose, t. (2009). innovative practices in the australian built environment sector: an information resource for industry (c. r. c. f. c. innovation, trans.) brisbane, qld: built environment industry innovation council. http://dx.doi.org/10.1016/s0166-4972(01)00039-6 http://dx.doi.org/10.1155/2014/164863 ijahp article: mostafa, abdelhamid, chileshe, dumrak/decision support model using anp to align leagile strategies to off-site manufacturing in australia international journal of the analytic hierarchy process 399 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.340 mason-jones, r., naylor, b., & towill, d. r. (2000). lean, agile or leagile? matching your supply chain to the marketplace. international journal of production research, 38(17), 4061-4070. doi: 10.1080/00207540050204920 mason-jones, r., & towill, d. r. (1999). total cycle time compression and the agile supply chain. international journal of production economics, 62(1), 61-73. doi: http://dx.doi.org/10.1016/s09255273(98)00221-7 mcgrath-champ, s., & rosewarne, s. (2009). organizational change in australian building and construction: rethinking a unilinear ‘leaning’discourse. construction management and economics, 27(11), 1111-1128. mhra. (2003). technology roadmap for manufactured housing (u. s. d. o. h. a. u. development, trans.). new york: partenrship for advanced technology in housing. mossman, a. (2009). why isn’t the uk construction industry going lean with gusto? lean construction journal, 5(1), 24-36. mostafa, s., dumrak, j., & soltan, h. (2013). a framework for lean manufacturing implementation. production & manufacturing research, 1(1), 44-64. doi: 10.1080/21693277.2013.862159 nadim, w., & goulding, j. s. (2011). offsite production: a model for building down barriers: a european construction industry perspective. engineering, construction and architectural management, 18(1), 82-101. doi: 10.1108/09699981111098702 naim, m., & barlow, j. (2003). an innovative supply chain strategy for customized housing. construction management and economics, 21(6), 593-602. doi: 10.1080/0144619032000134129 naim, m., naylor, j., & barlow, j. (1999). developing lean and agile supply chains in the uk housebuilding industry. paper presented at the proceedings of the 7th annual conference of the iglc, berkeley. naylor, b. j., naim, m. m., & berry, d. (1999). leagility: integrating the lean and agile manufacturing paradigms in the total supply chain. international journal of production economics, 62(1–2), 107-118. doi: 10.1016/s0925-5273(98)00223-0 nhsc. (2013). housing supply and affordability issues 2012-13. canberra: national housing supply council. olhager, j. (2013). evolution of operations planning and control: from production to supply chains. international journal of production research, 1-8. doi: 10.1080/00207543.2012.761363 ozdemir, m. s. (2005). the unknown in decision making: the case of a network. paper presented at the the proceedings of the 8th international symposium on the analytic hierarchy process: multicriteria decision making, honolulu, hawaii, usa. pan, w., & goodier, c. (2012). house-building business models and off-site construction take-up. journal of architectural engineering, 18(2), 84-93. doi: doi:10.1061/(asce)ae.1943-5568.0000058 pasquire, c. (2012a). the 8th flow common understanding. paper presented at the proceedings of the 20th annual conference of the international group for lean construction, san diego state university, usa. pasquire, c. (2012b). positioning lean within an exploration of engineering construction. construction management and economics, 30(8), 673-685. doi: 10.1080/01446193.2012.689431 http://dx.doi.org/10.1016/s0925-5273(98)00221-7 http://dx.doi.org/10.1016/s0925-5273(98)00221-7 ijahp article: mostafa, abdelhamid, chileshe, dumrak/decision support model using anp to align leagile strategies to off-site manufacturing in australia international journal of the analytic hierarchy process 400 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.340 path. (2002). technology roadmap: whole house and building process redesign, one year progress report. in prepared for us department of housing and urban development (ed.), partenership for advanced technology in housing. washington dc, usa: office of policy development and research. purvis, l., gosling, j., & naim, m. m. (2014). the development of a lean, agile and leagile supply network taxonomy based on differing types of flexibility. international journal of production economics, 151, 100-111. doi: http://dx.doi.org/10.1016/j.ijpe.2014.02.002 saaty, t. l. (1980). the analytic hierarchy process : planning, priority setting, resource allocation. london: mcgraw-hill international book co. saaty, t. l., & shang, j. s. (2007). group decision-making: head-count versus intensity of preference. socio-economic planning sciences, 41(1), 22-37. doi: http://dx.doi.org/10.1016/j.seps.2005.10.001 saaty, t. l., & sodenkamp, m. (2008). making decisions in hierarchic and network systems. international journal of applied decision sciences, 1(1), 24-79. doi: 10.1504/ijads.2008.017952 sarkis, j. (2003). quantitative models for performance measurement systems—alternate considerations. international journal of production economics, 86(1), 81-90. doi: http://dx.doi.org/10.1016/s0925-5273(03)00055-0 sertyesilisik, b. (2014). lean and agile construction project management: as a way of reducing environmental footprint of the construction industry. in h. xu & x. wang (eds.), optimization and control methods in industrial engineering and construction (pp. 179-196). netherlands:springer. shah, r., chandrasekaran, a., & linderman, k. (2008). in pursuit of implementation patterns: the context of lean and six sigma. international journal of production research, 46(23), 6679-6699. doi: 10.1080/00207540802230504 shah, r., & ward, p. (2003). lean manufacturing: context, practice bundles, and performance. journal of operations management, 21(2), 129-149. doi: 10.1016/s0272-6963(02)00108-0 sharifi, h., & zhang, z. (1999). a methodology for achieving agility in manufacturing organisations: an introduction. international journal of production economics, 62(1), 7-22. sharp, j. m., irani, z., & desai, s. (1999). working towards agile manufacturing in the uk industry. international journal of production economics, 62(1–2), 155-169. doi: http://dx.doi.org/10.1016/s0925-5273(98)00228-x state government of victoria. (2014). group self build, housing and community building. housing & community building: finance & business services. retrieved 29/04/2014, 2014, from http://www.dhs.vic.gov.au/for-individuals/housing-and-accommodation/home-owner-support/groupself-build towill, d., & christopher, m. (2002). the supply chain strategy conundrum: to be lean or sgile or to be lean snd sgile? international journal of logistics research and applications, 5(3), 299-309. doi: 10.1080/1367556021000026736 vidalakis, c., tookey, j. e., & sommerville, j. (2013). demand uncertainty in construction supply chains: a discrete event simulation study. journal of the operational research society, 64(8), 11941204. doi: doi.org/10.1057/jors.2012.156 http://dx.doi.org/10.1016/j.ijpe.2014.02.002 http://dx.doi.org/10.1016/j.seps.2005.10.001 http://dx.doi.org/10.1016/s0925-5273(03)00055-0 http://dx.doi.org/10.1016/s0925-5273(98)00228-x http://www.dhs.vic.gov.au/for-individuals/housing-and-accommodation/home-owner-support/group-self-build http://www.dhs.vic.gov.au/for-individuals/housing-and-accommodation/home-owner-support/group-self-build ijahp article: mostafa, abdelhamid, chileshe, dumrak/decision support model using anp to align leagile strategies to off-site manufacturing in australia international journal of the analytic hierarchy process 401 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.340 womack, j., & jones, d. (2003). lean thinking: banish waste and create wealth in your corporation. london: free press. yusuf, y. y., sarhadi, m., & gunasekaran, a. (1999). agile manufacturing: the drivers, concepts and attributes. international journal of production economics, 62(1–2), 33-43. doi: http://dx.doi.org/10.1016/s0925-5273(98)00219-9 zhang, d., & sharifi, h. (2007). towards theory building in sgile manufacturing dtrategy-a taxonomical spproach. ieee transactions on engineering management, 54(2), 351-370. doi: 10.1109/tem.2007.893989 http://dx.doi.org/10.1016/s0925-5273(98)00219-9 ijahp article: mostafa, abdelhamid, chileshe, dumrak/decision support model using anp to align leagile strategies to off-site manufacturing in australia international journal of the analytic hierarchy process 402 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.340 appendix a unweighted supermatrix alternatives c1 c2 c3 c4 a1 a2 a3 a4 c1.1 c1.2 c2.1 c2.2 c3.1 c3.2 c3.3 c4.1 c4.2 alternatives ato 0.00 0.00 0.00 0.00 0.00 0.13 0.16 0.17 0.19 0.16 0.19 0.17 0.00 dto 0.00 0.00 0.00 0.00 0.00 0.08 0.14 0.08 0.24 0.27 0.21 0.24 0.00 mts 0.00 0.00 0.00 0.00 0.00 0.37 0.27 0.33 0.12 0.14 0.14 0.00 0.00 sbh 0.00 0.00 0.00 0.00 0.00 0.42 0.42 0.42 0.46 0.42 0.45 0.00 0.00 c1 c1.1 0.67 0.67 0.50 0.50 0.00 1.00 0.00 0.00 00.00 0.00 0.00 00.00 0.00 c1.2 0.33 0.33 0.50 0.50 0.00 0.00 0.00 0.00 0.00 00.00 0.00 0.00 00.00 c2 c2.1 0.33 0.25 0.20 0.25 0.00 0.00 0.00 0.00 00.00 00.00 0.00 0.00 00.00 c2.2 0.67 0.75 0.80 0.75 0.00 0.00 1.00 0.00 0.00 00.00 0.00 0.00 00.00 c3 c3.1 0.31 0.41 0.33 0.26 0.00 0.00 0.00 0.00 00.00 0.50 0.00 0.00 00.00 c3.2 0.20 0.33 0.33 0.41 0.00 0.00 0.00 0.00 00.00 0.00 0.00 00.00 0.00 c3.3 0.49 0.26 0.34 0.33 0.00 0.00 0.00 0.00 00.00 0.50 0.00 0.00 00.00 c4 c4.1 0.50 1.00 0.50 0.25 0.00 0.00 0.00 0.00 00.00 0.00 0.00 00.00 0.00 c4.2 0.50 0.00 0.50 0.75 0.00 0.00 0.00 0.00 00.00 0.00 0.00 1.00 0.00 ijahp article: mostafa, abdelhamid, chileshe, dumrak/decision support model using anp to align leagile strategies to off-site manufacturing in australia international journal of the analytic hierarchy process 403 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.340 appendix b weighted supermatrix alternatives c1 c2 c3 c4 a1 a2 a3 a4 c1.1 c1.2 c2.1 c2.2 c3.1 c3.2 c3.3 c4.1 c4.2 alternatives ato 0.00 0.00 0.00 0.00 0.00 0.06 0.08 0.17 0.18 0.08 0.19 0.10 0.00 dto 0.00 0.00 0.00 0.00 0.00 0.04 0.07 0.08 0.24 0.13 0.21 0.19 0.00 mts 0.00 0.00 0.00 0.00 0.00 0.18 0.13 0.33 0.12 0.07 0.14 0.08 0.00 sbh 0.00 0.00 0.00 0.00 0.00 0.21 0.21 0.42 0.46 0.21 0.45 0.12 0.00 c1 c1.1 0.12 0.12 0.09 0.09 0.00 0.50 0.00 0.00 00.00 0.00 0.00 00.00 0.00 c1.2 0.06 0.06 0.09 0.09 0.00 0.00 0.00 0.00 0.00 00.00 0.00 0.00 00.00 c2 c2.1 0.09 0.07 0.06 0.07 0.00 0.00 0.00 0.00 00.00 00.00 0.00 0.00 00.00 c2.2 0.19 0.21 0.22 0.21 0.00 0.00 0.50 0.00 0.00 00.00 0.00 0.00 00.00 c3 c3.1 0.08 0.10 0.08 0.06 0.00 0.00 0.00 0.00 00.00 0.25 0.00 0.00 00.00 c3.2 0.05 0.08 0.08 0.10 0.00 0.00 0.00 0.00 00.00 0.00 0.00 00.00 0.00 c3.3 0.12 0.06 0.08 0.08 0.00 0.00 0.00 0.00 00.00 0.25 0.00 0.00 00.00 c4 c4.1 0.15 0.30 0.15 0.07 0.00 0.00 0.00 0.00 00.00 0.00 0.00 00.00 0.00 c4.2 0.15 0.00 0.15 0.22 0.00 0.00 0.00 0.00 00.00 0.00 0.00 0.50 0.00 ijahp article: mostafa, abdelhamid, chileshe, dumrak/decision support model using anp to align leagile strategies to off-site manufacturing in australia international journal of the analytic hierarchy process 404 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.340 appendix c limit matrix alternatives c1 c2 c3 c4 a1 a2 a3 a4 c1.1 c1.2 c2.1 c2.2 c3.1 c3.2 c3.3 c4.1 c4.2 alternatives ato 0.07 0.07 0.07 0.07 0.00 0.07 0.07 0.07 0.07 0.07 0.07 0.07 0.00 dto 0.07 0.07 0.07 0.07 0.00 0.07 0.07 0.07 0.07 0.07 0.07 0.07 0.00 mts 0.09 0.09 0.09 0.09 0.00 0.09 0.09 0.09 0.09 0.09 0.09 0.09 0.00 sbh 0.16 0.16 0.16 0.16 0.00 0.16 0.16 0.16 0.16 0.16 0.16 0.16 0.00 c1 c1.1 0.07 0.07 0.07 0.07 0.00 0.07 0.07 0.07 0.07 0.07 0.07 0.07 0.00 c1.2 0.04 0.04 0.04 0.04 0.00 0.04 0.04 0.04 0.04 0.04 0.04 0.04 00.00 c2 c2.1 0.03 0.03 0.03 0.03 0.00 0.03 0.03 0.03 0.03 00.00 0.00 0.00 00.00 c2.2 0.12 0.12 0.12 0.12 0.00 0.12 0.12 0.12 0.12 0.12 0.12 0.12 00.00 c3 c3.1 0.05 0.05 0.05 0.05 0.00 0.05 0.05 0.05 0.05 0.05 0.05 0.05 00.00 c3.2 0.04 0.04 0.04 0.04 0.00 0.04 0.04 0.04 0.04 0.04 0.04 0.04 0.00 c3.3 0.05 0.05 0.05 0.05 0.00 0.05 0.05 0.05 0.05 0.05 0.05 0.05 00.00 c4 c4.1 0.07 0.07 0.07 0.07 0.00 0.07 0.07 0.07 0.07 0.07 0.07 0.07 0.00 c4.2 0.12 0.12 0.12 0.12 0.00 0.12 0.12 0.12 0.12 0.12 0.12 0.12 0.00 ijahp essay: saaty/a personal view of the development of the ahp international journal of the analytic hierarchy process 401 vol. 9 issue 3 2017 issn1936-6744 https://doi.org/10.13033/ijahp.v9i3.556 a personal view of the development of the ahp significant milestones during the evolution of the ahp submitted by rozann saaty ijahp managing editor creative decisions foundation rsaaty@ijahp.org this essay is to tell you what i saw as the ahp evolved. i had an inside view as tom and i were married in 1964 before he started developing the ahp. it was an interesting fortyfive years from 1972 until 2017 when tom passed away. tom became interested in how to measure intangibles when he worked for the arms control and disarmament agency in the u.s. state department in the 1960s. this agency conducted the arms control negotiations between the soviets and the americans, and tom said he learned there that most decisions come down to intangibles and emotions, and that there was no way to quantify them. which armaments? how many of theirs equaled how many of ours? these questions were decided by lawyers who came up with the terms, not scientists or mathematicians, or even military people. tom headed a research team that included three nobel prize winners, and even they were not of much help in the negotiations. it took them too long to model a situation and find an answer, and by the time they were ready the negotiations would have moved on. tom left the arms control agency in 1969 because the climate was changing in washington; a republican president, richard nixon, was elected to office after democratic presidents jack kennedy and lyndon b. johnson had held the office for eight years. as the democrats were out of power in the 1970s, the arms talks were sidelined, so tom moved to academia at the wharton school of the university of pennsylvania in the fall of 1969. it was with a sense of awe and amazement that just last month i came across a copy of a report, typewritten and bound with a soft tan cover, that tom had written in the early 1970s while consulting for lmi, the logistics management institute, a think tank for the navy. the report was titled identification of war reserve stock, task 72-04 of contract no. sd-271. i was amazed on two counts; first, that i had found the birth document of the ahp, and second that he still had the report from 45 years ago. the object of the job was to prioritize all the navy war materiel ranging from toilet seats to tanks, and lmi was hired as the consultant. tom laid out some of the core concepts of the ahp for the first time in appendix 3: mathematical method for quantifying essentiality judgments. these concepts included the pairwise comparison matrix formed of ratios from which priority vectors are obtained and simple hierarchic composition. he does not use the word hierarchy, though he illustrates a weight and add process of prioritized properties and prioritized alternatives for each property that is essentially synthesis for a two-level hierarchy. he gives credit to ijahp essay: saaty/a personal view of the development of the ahp international journal of the analytic hierarchy process 402 vol. 9 issue 3 2017 issn1936-6744 https://doi.org/10.13033/ijahp.v9i3.556 previous works: the theorem of perron and frobenius which he used (from gantmacher, the theory of matrices, vol. ii, p. 53-54) and graph and matrix theoretic concepts (from c. berge, the theory of graphs, wiley, 1962, p. 135-138 and d. gale and l. s. shapley, college admissions and the stability of marriage, am. math. monthly, 69, 1962, p. 914). in the report where he is giving credit to others he says that the extension for use to assign weights to properties is due to him. to me, the evolution from the ahp to the anp to complex anp models to neural networks is an interesting story somewhat analogous to how life evolved, starting with a simple cell (the pairwise comparison matrix) and evolving into more complex cells (ahp hierarchies), then to collections of complex cells to form simple organisms (simple anp networks), to complex organisms like animals and plants (complex anp models of many simple networks) and finally to societies of networked organisms formed of individuals with communication links (nnp – neural network process). the reciprocal pairwise comparison matrix was the basic building block of the theory. for example, if aij = 5 the ratio 5/1 is entered in the (i,j) position in the matrix, and 1/5 is entered in the reciprocal (j,i) position. the entries are ratios of absolute numbers, either formed from two measurements of a property using a traditional ratio scale or by using judgments from the fundamental 1-9 scale of the ahp where, 1 – equal, 3 – moderate, 5– strong, 7 – very strong, 9 – extreme. a judgment from the scale is a ratio indicating how many times the dominant element is larger than the dominated one. in each comparison based on judgments the dominated element is the unit in the denominator. but note that in every cell, the unit in the ratio is different, but the totality of the individual ratios is synthesized into an overall relative priority vector. the only role zero plays is when two elements cannot be compared it is entered into the matrix. there is no starting point, no zero, for relative scales. people have often used the scale as if it is an ordinal scale; for example, when one chooses a number from 1 to 5, where 5 is the best, bigger numbers are only known to be better, but not by how much. numbers from an ordinal scale are not the same as the absolute numbers expressed as ratios from the fundamental scale of the ahp. the invention of the fundamental scale of absolute numbers associated with words was a major leap into the unknown by tom. it was the device that allowed him to bring intangibles into the picture along with measurables and tangibles. getting people to use this idea, and being able to understand it required patience, persistence and persuasion. a relative scale is a different way of measuring. when making pairwise comparisons adding or subtracting elements gives completely different priorities. with a traditional ratio scale, such as a “ruler”, when something is measured it stays measured, the measurement does not change regardless of what items are added or removed. tom’s first book on the ahp, the analytic hierarchy process, from mcgraw hill international, appeared in 1980. this new scale of relative measurement led to the rank reversal wars of the 1980s and 1990s, and some people are still revisiting that issue today. valerie belton and t. gear wrote a critical article about rank reversal in the ahp titled, ‘on a short-coming of saaty's method of analytic hierarchies’, omega, 11(3), 228-230, 1983. this became the ijahp essay: saaty/a personal view of the development of the ahp international journal of the analytic hierarchy process 403 vol. 9 issue 3 2017 issn1936-6744 https://doi.org/10.13033/ijahp.v9i3.556 seminal reference in almost every article critical of rank reversal, and it is still cited in articles written today. however, an interesting thing happened at an mcdm meeting in canada, chaired by bill wedley, in 2004. at this meeting, i delivered a paper in a session that was critical of the rank reversal critics of ahp. valerie belton was in the audience and she stood up and said she was so sorry about her paper and that she had not understood, but there was no way she could get it back or undo it as it had taken on a life of its own. the ahp with its single kind of matrix, the pairwise comparison matrix, and a hierarchical model with its top-down weight and add synthesis process to obtain the priorities for the alternatives was followed by the anp (analytic network process) with tom’s first book about it, the analytic network process: decision making with dependence and feedback, rws publications, 1996. this book dealt with single network models, the structure changed from being a hierarchy of elements arranged in levels to a network of clusters of elements with links between the elements. a new data structure was introduced, the supermatrix, a square matrix with all the elements in the network as row and column elements arranged by cluster. the priority vectors are vertical priority vectors stacked in a column derived by pairwise comparing the elements in a cluster that are connected from the column element. “why call it a supermatrix?” people asked. tom defended the name, saying that it was a matrix of matrices in a sense, not his attempt to be grandiose. the supermatrix is the second kind of matrix in the theory, being made up of priority vectors derived from pairwise comparison matrices, quite different from the pairwise comparison matrices composed of judgments. but the eigenvector is again used to derive an overall vector of priorities of all the elements in the network, including the alternatives. his colleague at the wharton school, james bennett–who tom liked and admired him for his brilliance– was very struck by the supermatrix and complimented tom mightily on having come up with it. let me mention that tom was superbly equipped to generalize the hierarchical structures of the ahp to networks. he wrote a book with robert busacker before the ahp titled, finite graphs and networks by mcgraw hill in 1965. this book was heavy with the mathematics for networks and at the same time as he worked out the mathematics for the ahp and anp, shoring up his intuition with theoretical proofs, he was also delving deeper into modeling, writing these four books: mathematical models of arms control and disarmament, (translated to russian), john wiley and sons, 1968; the four-color problem; assaults and conquest, with paul c. kainen, mcgraw-hill international, in 1977; thinking with models with joyce alexander, pergamon press, 1981, and conflict resolution: the analytic hierarchy process with joyce alexander, praeger, 1989. tom was honing another skill along the way. starting in 1964 he collected jokes, putting out a new joke book every two years or so to a total of 23 ending with three volumes published in july 2017. of course, he mostly collected the jokes, he did not invent many of them, but he came up with the titles, practicing capturing the essence of a joke with short succinct phrases. i think this was a very important skill to have as he broke ground in this new field of relative measurement and wrote countless papers about it. he had to ijahp essay: saaty/a personal view of the development of the ahp international journal of the analytic hierarchy process 404 vol. 9 issue 3 2017 issn1936-6744 https://doi.org/10.13033/ijahp.v9i3.556 invent some of the language. the ability to write concisely and with drama about a dense mathematical subject was a great asset, and i think it came partly from his work on jokes. one might argue that the name analytic hierarchy process for the process he invented is no evidence of this skill, but no one has come up with a better name. continuing the development of the anp, a major step tom had to overcome was how to get the supermatrix to converge to a solution when it is raised to powers. the supermatrix converges only if each column of the supermatrix sums to 1.0 (i.e., is stochastic). a column may have several priority vectors stacked on top of each other and as each priority vector sums to 1.0, the entire column sums to something more than one. he had to find a way to get every column to sum to one, therefore he invented the cluster matrix. by pairwise comparing the impact of the clusters linked from a given cluster for their impact on the “parent” cluster, with the usual pairwise comparison matrix of judgments, one gets a priority matrix for the clusters that becomes a column in the cluster matrix. arranging the cluster comparison priority vectors in a square matrix and multiplying the elements in it times the respective components in the supermatrix accomplished this aim and “voila!”, the weighted supermatrix was born. it has the desired property that all its columns sum to 1.0 and it converges when raised to powers to the limit supermatrix which has priorities for all the elements in the network. it turned out that not only does the supermatrix converge after multiplying its elements by the cluster matrix, but humans do have the ability to pairwise compare clusters, judging which one has more impact on the parent cluster. i don’t know how tom thought of this, as he never talked about the insight that led to it. this one is mystical, to me at least. the next evolutionary step was the complex anp model of separate networks benefits, opportunities, costs and risks (bocr) which knits together different individual networks using algebraic equations. following that was tom’s work on the brain, the nnp – neural network process. he believed that the transmission of signals in the brain depended on having some way to synthesize the electrical and chemical inputs to and outputs from a neuron. in essence, the brain is a highly complex network with different kinds of stimuli including visual, auditory, sensory, memories and so on. these stimuli must be synthesized to produce the signals throughout the brain’s network of cells and neurons, and perhaps it is the state of all this at any one moment of time that is what we call consciousness. tom had bookcases full of works on the brain and nervous system, and he had read them all. many of the books have scraps of paper sticking out of them to mark where he found something interesting. to conclude this essay, let me summarize what i think were tom’s contributions and significant advances: 1. developing the reciprocal pairwise comparison matrix of elements where the entries in the cells are ratios of tangibles or human judgments and finding its solution of priorities with the eigenvector (and years went into justifying that the eigenvector is the correct way for inconsistent matrices). the solution is an absolute relative ratio scale vector of priorities. ijahp essay: saaty/a personal view of the development of the ahp international journal of the analytic hierarchy process 405 vol. 9 issue 3 2017 issn1936-6744 https://doi.org/10.13033/ijahp.v9i3.556 2. inventing the fundamental scale of the ahp comprised of absolute numbers that brought human judgment into the pairwise comparison matrix; specifying that this scale should be used for homogeneous groups of elements. 3. using the eigenvalue from the eigenvector solution to create an index for inconsistency in a pairwise comparison matrix of judgments. 4. developing the compatibility index for determining closeness of two priority vectors. 5. creating a hierarchic structure for decision models and a synthesis method to obtain the overall priorities of the alternatives at the bottom of the hierarchy – hierarchies are not the same as trees beyond three levels, a little understood fact. most current ahp software packages use a tree structure. the superdecisions software for anp (www.superdecisions.com) developed by tom and me and is for modeling networks which are a generalization of hierarchies, so it can be used to model true hierarchies. 6. axiomatizing the ahp there are 4 or 5 simple axioms that are fundamental to describing ahp models, and somehow, with his training in classical mathematics, he was able to boil down all the ideas to these few axioms. 7. generalizing from hierarchies to single anp networks with hierarchic levels being replaced by clusters of elements and links between the elements. anp networks in general do not have goals. it is a relative world where priorities are determined by the interactions among the elements. 8. developing the conflict resolution approach from the ahp, a process that he and his colleague dean jerry zoffer used in several trial negotiations between teams from israel and palestine to prioritize things they might trade and arrange them into equitable swaps. tom and jerry considered this to be practice for when the time ever came that the leaders of the two countries truly wanted peace; real negotiations needed to have a process that had been tested for doing tradeoffs. his insight was that there is sometimes a retributive aspect to resolving conflicts. not only do the parties want to receive something of value, they want to inflict costs on the giving party, so every swap is analyzed from the perspective of each party for gain to them and cost to their opponent. 9. inventing the supermatrix that contains the priority vectors from all the pairwise comparison matrices, and the cluster matrix formed by pairwise comparing clusters for their impact on a parent cluster. 10. using the supermatrix to find priorities for all the elements in the network by raising the weighted supermatrix to powers until it converges. finding the solution this way is the same as finding the eigenvector of the supermatrix. surprising fact: the eigenvector is also the solution for the pairwise comparison matrix. 11. developing the complex anp bocr model where formulas are used in the top level to combine priority vectors from lower level networks. 12. developing the hypermatrix for the brain. tom wrote two books about the brain: the brain in 2000 and the neural network process in 2014. http://www.superdecisions.com/ ijahp essay: saaty/a personal view of the development of the ahp international journal of the analytic hierarchy process 406 vol. 9 issue 3 2017 issn1936-6744 https://doi.org/10.13033/ijahp.v9i3.556 it would take many more pages than are allowed in an essay to discuss all the creativity tom had and all the intricacies of mathematics he knew and used. but i would like to close by mentioning that he published three important papers in 2016 and 2017 when he was in his nineties, being an inspiring role model for the rest of us. 1. saaty, t. (2016). seven is the magic number in nature, proceedings of the american philosophical society, 160(4), 335-360. 2. saaty, t. (2017). neurons the decision makers, part i: the firing function of a single neuron, neural networks, 86, 102–114. 3. saaty, t. (2017). part 2—the firings of many neurons and their density; the neural network its connections and field of firings, neural networks, 86, 115– 122. tom liked to be called the “father of ahp” rather than the creator of the method. in this sense, he was right since ahp, anp and now nnp have taken on a life of their own and will transcend, through their worldwide use, the unavoidable finite human life span of tom. it has been said of the greeks that they took no rest themselves and gave none to others. this could also be said of tom. may his soul rest in peace. http://www.sciencedirect.com/science/journal/08936080 http://www.sciencedirect.com/science/journal/08936080/86/supp/c http://www.sciencedirect.com/science/journal/08936080/86/supp/c mergedfile ijahp article: cooper, liu / achieving the desired level of dependency in anp decision models international journal of the analytic hierarchy process 1 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.450 achieving the desired level of dependency in anp decision models orrin cooper university of memphis, fogelman college of business memphis, tn olcooper@memphis.edu guoqing liu university of pittsburgh pittsburgh, pa liugq1990@gmail.com 1 abstract when designing an anp model it is important to acknowledge and properly address whether the elements in the model are dependent on or independent of each other. the decision maker must perform criteria cluster weighting comparisons individually for the criteria clusters in each column of the supermatrix to correctly model when the criteria and alternatives are dependent on one another to accurately capture the dependence. failing to recognize that the criteria in a criteria cluster in one column of the supermatrix is not necessarily equal in weight to the criteria in that same criteria cluster but in another column can lead to misrepresented rankings in the final priorities. in the extreme case, it can remove all dependence from an anp model. two models are used to demonstrate this unintended effect on the final priorities, and also demonstrate a crucial contribution that this effect is independent of the tangibility of the criteria considered. in the third model, the solution is discussed and implemented. a proof is provided in the appendix. this criteria cluster weighting approach further extends the applicability of the anp to additional decisions when a decision maker wishes to represent a fully-dependent anp decision. https://doi.org/10.13033/ijahp.v9i1.450 keywords: analytic network process (anp); dependence; intangible elements; criteria weights 1. introduction 1 acknowledgements: the authors wish to thank bill wedley for sharing his time, references, revisions, and keynote presentation from the isahp 2013 conference. we are also grateful to the reviewers for their thoughtful and detailed feedback. this work was supported in full or in part by a grant from the fogelman college of business & economics at the university of memphis. this research support does not imply endorsement of the research results by either the fogelman college or the university of memphis. https://doi.org/10.13033/ijahp.v9i1.450 mailto:olcooper@memphis.edu mailto:liugq1990@gmail.com https://doi.org/10.13033/ijahp.v9i1.450 ijahp article: cooper, liu / achieving the desired level of dependency in anp decision models international journal of the analytic hierarchy process 2 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.450 the analytic hierarchy process (ahp), and its generalization the analytic network process (anp), take advantage of our natural ability to structure decisions as hierarchies or networks and make relative comparisons or judgments (saaty, 1996). when one decides to create an ahp or anp model certain assumptions must be made about how the decision at hand will be framed. some of the questions one must consider are: 1) what level of dependence (or independence) is there among the criteria and alternatives? 2) what is the meaning of the final priority vector that will be obtained? 3) are there both tangible and intangible criteria in the model? the first two questions about the meaning of the final priority vectors and level of dependence are addressed in further detail below. neither question has a universally correct answer, but different answers to the questions will lead to different model designs with different results. it will also be shown herein that the answers to the first two questions are independent of the third question relating to the tangibility of the elements; and contrary to prior claims that it is the tangibility/intangibility of the elements that is the determinant of how to weight the criteria clusters it is the answer to the first two questions that will determine how the criteria clusters should be weighted. when a decision maker wishes to model a decision where the alternatives are fully-dependent on the defined system or decision and to obtain a final priority vector that is of the form 1 / n i i i a a   in the limit matrix, which provides the relative priority or contribution of an alternative ai with respect to the system of n alternatives being considered, careful attention must be paid to the weighting method that will be used to weight the criteria clusters to convert an unweighted supermatrix which generally contains multiple priority vectors in each column that sum to one and must be weighted to obtain the “weighted” supermatrix that is column stochastic. in this paper, a model that provides final priority vectors of the form 1 / n i i i a a   will be termed fully-dependent anp models. the pairwise comparisons performed to obtain the criteria cluster weights must be performed individually for each column in the supermatrix to obtain a fully-dependent anp model. whereas the current approach is to apply the same criteria cluster weight equally across all columns in the supermatrix to weight the unweighted supermatrix (saaty, 2005, 2008a, 2008b, 2011). the use of a single cluster weight across all columns of the supermatrix within a given cluster fails to recognize that a “one” here, in a given criteria cluster in a certain column, does not necessarily equal a “one” there representing another priority vector in a different column of the supermatrix even though that priority vector is in the same criteria cluster (see figure 1). understanding and applying that statement is at the core of this paper. if the decision maker does not properly address the desired level of dependency sought in the model they will be left with a model that does not necessarily reflect a fully-dependent anp model and can lead to unintended and misleading results in the final, or global, priority vectors in the limit matrix. both the alternative and criteria final priority vectors will be impacted for length and the focus will be on the impact to the final priority vector for alternatives. it has been shown previously that unique criterion weighting for each alternative must be used when tangible elements are considered and/or the model must be validated against https://doi.org/10.13033/ijahp.v9i1.450 ijahp article: cooper, liu / achieving the desired level of dependency in anp decision models international journal of the analytic hierarchy process 3 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.450 actual results (harker & vargas, 1990, saaty, 2011). however, as will be demonstrated in the first example in the model section, this process of weighting the priority vectors in the supermatrix is clearly independent of the tangibility of the elements being considered. this distinction is clearly shown here in this paper for the first time and hence requires decision makers to conscientiously determine the desired level of dependency in their model independent of the tangibility of the elements and then take the necessary steps to achieve it. this paper is not a proposition for a universal level of dependence for all decision models rather that the determination of the level of dependence must be explicitly determined by the decision maker at the outset of the problem and then the proper steps be followed to achieve that desired level of dependence in the model. herein, dependency will be categorized into two categories with meaningful interpretations as: independent (ahp) and fully-dependent (anp). the solution provided in this paper should be used if the decision maker decides they want a model where the alternatives are dependent on the criteria and vice versa. it is also worth noting that a formal acknowledgement of the categories of dependency can also not only separate and distinguish but even provide solutions for many of the criticisms of the ahp/anp. the need for and the potential benefits of the suggested method will be heightened with a review of the relevant literature. subsequently, in the models section, the unintended consequences of not fully capturing the dependence by not providing unique cluster weights in each column of the supermatrix are identified and shown to be independent of the tangibility of the elements. the solution is proposed, demonstrated in an example, and generalized in the proof. obtaining meaningful final priority vectors should be at the core of a decision maker’s objectives. in a fully-dependent anp model performing unique criteria cluster weighting comparisons for each column of the unweighted supermatrix will lead to a more meaningful final priority vector in the limit matrix. 2. literature review 2.1 the ahp and independent criteria weights the first publication using the ahp ranked transportation projects in sudan (saaty, 1977b). the ahp/anp is now the most widely published multi criteria decision making method (wallenius et al., 2008). a more comprehensive review of the theory is found in the following references (saaty, 1977a; saaty, 1986; saaty, 1996, 2005; saaty & cillo, 2008; lipovetsky, 2011, 2013; lipovetsky & conklin, 2015). the process of creating an ahp model is summarized below in nine steps. the criteria weights calculated in an ahp model are independent of the alternatives (dyer, 1990; saaty, 1986; saaty, 1996; schoner, wedley, & choo, 1993). in an ahp model because there is no inner and outer dependence step 5 and step 6 are one in the same and there is no effect on the independence of the criteria from the alternatives. in the ahp literature, there are arguments for and against independent criteria weights (belton & gear, 1983; dyer, 1985; harker, 1987; schoner et al., 1993; wedley & choo, 2011). rather than concluding that independent criteria weights are right or wrong; in general, it is more meaningful to frame the question in the context of what is the desired outcome; that is, if we are dealing with ahp models or fully-dependent anp models. there are many examples where independent criteria weighting can be and should be used. a https://doi.org/10.13033/ijahp.v9i1.450 ijahp article: cooper, liu / achieving the desired level of dependency in anp decision models international journal of the analytic hierarchy process 4 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.450 common use of criteria weighting that is independent of the alternatives is for candidate selection like college program admission formulas. the weighting for each category like grade point average and standardized test scores are set independently from the alternatives or applicants that are evaluated during each cycle. another common application is the use of independent criteria weights in budget/resource allocation (arbel, 1983; zahedi, 1986). the decision maker should ask the question at the outset, “am i intending to evaluate my alternatives against a “standardized” or independent set of criteria?”. if the answer is yes, then the current method for weighting criteria clusters can be used. in other instances, decision makers may not want to use criteria weights that are independent of the alternatives and might say, “i am looking to find the relative value of an alternative with respect to the values that all the alternatives provide to the entire system.” the model that allows decision makers to measure the relative contribution of each alternative with respect to the entire set of alternatives is the anp. step 1: define the problem step 6: weighting the unit sum step 2: organize the criteria and alternatives clusters in each of the columns into a hierarchy/network unweighted supermatrix step 3: perform pairwise comparisons step 7: formation of the weighted supermatrix step 4: consistency ratio estimation step 8: raise the supermatrix to powers step 5: formation of the initial step 9: calculation of global unweighted supermatrix priority vectors 2.2 the analytic network process the anp is the generalized form of the ahp, when designing an anp model a decision maker follows the steps as provided and the decision maker is not restricted to a hierarchal organization but may organize the criteria and alternatives into a network. a network design is a richer model that allows for dependence among the criteria and alternatives (saaty, 2005; saaty & vargas, 2006). in the supermatrix there may be multiple priority vectors in each column that sum to 1. it is then necessary to perform pairwise comparisons among the criteria clusters to aggregate them with respect to their relative contributions and to ensure that the weighted supermatrix is column stochastic (saaty, 2005). the weighted supermatrix is then raised to powers to calculate the global priorities and weights. the process of weighting the criteria clusters will be reviewed in greater detail. 2.3 fully-dependent anp weighting while much has been written to dispute the validity of ahp/anp the purpose of this publication is not to join these arguments for or against the ahp or the anp but rather to focus on the problem definition at the outset of setting up a model (belton & gear, 1983; dyer, 1990; j. w. r. e. dyer, 1985; harker & vargas, 1990; saaty, 1986; saaty, 1990). a specific form of the desired final priority vector in the limit matrix is p = 1 / n i i i a a   , which provides the relative priority or contribution of an alternative ai with respect to the system of n alternatives being considered. in other words, a fully-dependent model is one where the final priority vector depends on the contribution of an alternative with https://doi.org/10.13033/ijahp.v9i1.450 ijahp article: cooper, liu / achieving the desired level of dependency in anp decision models international journal of the analytic hierarchy process 5 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.450 respect the contributions of everything else in the system. in order to obtain a final priority vector of the form 1 / n i i i a a   , the pairwise comparisons to weight the criteria clusters performed as part of step 6 weighting the unit sum clusters in each of the columns of the unweighted supermatrix must be done individually for each criteria cluster in each column regardless of whether the criteria are tangible or intangible. harker and vargas (1990) explained that “if the [criterion importance] vary with the chosen alternative (the reference point) then the supermatrix technique described in harker and vargas (1987) and saaty (1990) must be employed.” their example uses an anp model and obtains a vector of the form 1 / n i i i a a   ; however, their proof only includes a single criterion cluster. in this paper, the supermatrix technique, demonstrated by harker and vargas (1990) for the case of a single criteria cluster, is extended to a fully-dependent anp model; that is, a model where not only the individual criterion weights for each alternative vary but also the criteria cluster weights vary with each alternative. there are three additional properties of anp models that will further clarify this point. first, further investigation into the overall unit of measurement in an anp model will allow criteria clusters to be combined into a general anp model and facilitate comparisons across criteria clusters. (choo, schoner, & wedley, 1999; wedley & choo, 2011). second, according to the current literature this claim would depend on the tangibility of criteria being considered. finally, an important property of the priority vectors that are combined in a supermatrix further underscores the need for the weight for each criteria cluster to be calculated individually for every cluster in each individual column. these properties and their application to a fully-dependent anp model are explained in greater detail below. 2.4 criteria weights and the unit of measurement criteria weights in general are misunderstood and misused and there is no consensus on the meaning or manner of deriving criteria weights (choo, schoner, & wedley, 1999). furthermore, one may not want the criteria weights to be calculated in a way that is independent of how they are used in a decision model. normalization, in and of itself, does not remove the units from the criteria being considered. according to saaty (2004), relative scales do not need a unit of measurement. however, any multiplication by a constant b, b> 0, changes the unit of measure for a ratio vector. the fact that the value for a particular object has changed is evidence of a new unit being used. wedley & choo (2011) explain that ratio scales in the ahp have a unit of measure and the unit of measure is important and useful. the unit of measurement is derived from the topmost node in the total network. the scale that one can obtain from such a unit is transient, depending on the alternatives being considered, but so is the ratio scale itself. focusing on the ratios rather than the rank will improve the efficacy of the ahp. wedley & choo (2011) conclude, “therein lie both the advantage and dilemma of ahp. we do not need explicit knowledge of the underlying unit of measure to derive a ratio scale, yet the derived scale has a unit.” this understanding that the unit of measurement is derived from the topmost node in the hierarchy provides a unit to use as the basis for comparing criteria across clusters. applying this unit concept to anp facilitates comparing clusters together at the individual level of each cluster with respect to their contribution to the https://doi.org/10.13033/ijahp.v9i1.450 ijahp article: cooper, liu / achieving the desired level of dependency in anp decision models international journal of the analytic hierarchy process 6 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.450 other criteria in that column. each column must be treated separately because the weights within one column are only dependent on the topmost node of that column, the column title, in other words everything in that column is dependent on the perspective of the node at the top. the design of the network emphasizes that the common unit of measurement is to be interpreted in the limit matrix. while in a network there is no topmost node of a network the following steps explain how a common unit is obtained: (1) in the unweighted supermatrix there are several units in each column – one unit sum scale for each priority vector of a set of comparisons. (2) in the weighted supermatrix, each of unit sum vector of a column is linked to others so that they are commensurate within the column. (3) when the supermatrix is then made stochastic, each column is in the unit of the totality of influences of each column. (4) in the limit matrix where all columns are the same, each column is in the unit of all influences of the entire supermatrix. after reviewing these important points about criteria weights, the tangibility of the criteria also becomes relevant. 2.5 tangibility the concept of individual criterion weights for each alternative has been used to capture dependency and validate models with “tangible” criteria (harker & vargas, 1990; schoner et al., 1993). while it has been shown that “column normalization” must be used in the case of tangibles, the opposite has been argued for intangibles (harker & vargas, 1990; saaty, 2011; saaty, 2016). the need to individually weight each criteria cluster in each column regardless of the tangibility of the elements in dependent models is demonstrated in model 1 and model 2. in both models, the unintended consequences of not fully representing the dependence are highlighted as the final results will seem counterintuitive because the criteria weights are obtained and applied universally across all columns in the supermatrix and unintended consequences arise. in a decision model the value the elements from one alternative provide towards the alternative’s overall value in most cases will not be equal to what those elements in another alternative provide towards the value of the other alternative. this again is the idea that a one here does not necessarily equal a one there. however, if the criteria clusters are given a single weight for all the columns the individual clusters now provide equal contributions regardless of the differences between the clusters in each column which leads to the unintended effect. this effect is clearly demonstrated in the first examples in the model section where further clarification is provided. it is important to emphasize that this extension is not about using particular ratio scales in the priority vectors whether dealing with tangibles or intangibles (saaty, 2011). rather, this example underscores the importance of using priorities and not just ratio scales in decisions that involve either tangibles or intangibles, or the combination of both. the following example demonstrates the potential difference between ratio scales and priorities. if one had no money but wanted to go to a movie that cost $5.00 and was asked, “which amount of money would you prefer receiving: $5.25 or $4.90?” one should expect the individual to extremely prefer the $5.25 to the $4.90 which would https://doi.org/10.13033/ijahp.v9i1.450 ijahp article: cooper, liu / achieving the desired level of dependency in anp decision models international journal of the analytic hierarchy process 7 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.450 correspond to a rating of 9 in saaty’s 1-9 scale. an additional validation of ratings in the movie example is the use of the consistency index. assume a third value was introduced like $5.50. if the inconsistency of the pairwise comparisons were under the desired value of .1, then the decision maker’s consistency could demonstrate that the choice of such extreme ratings was not just random but reflect the individual’s preferences. “tangibly” speaking $5.50 is not even 2 times better than $4.90; however, in terms of priorities and the goal to see a $5.00 movie there is an extreme difference. this example underscores the focus on using priorities even while using a “tangible” example because the individual’s preferences do not map to a monetary ratio scale. this is the first important clarification about tangibility of the elements. additionally, the models in the next section go one step further and use intangible criteria that do not have a direct conversion or ratio scale to show that the need to recognize and address that a one here is not necessarily equal to a one there is independent of the “tangibility” of the elements. without tangible data or a scale to interpret the ratings and results one might ask, “then how can the results be validated?” the results from the examples are validated by using a similar approach to wedley and choo “the ability of each method to replicate the true composite ratios as a measure of effectiveness. ratio preservation rather than rank preservation is chosen as the measure of effectiveness because ahp produces ratio answers. with perfect accuracy consistency cannot be the reason why ratios are deflected from their true values” (wedley & choo, 2001). this methodology will be followed herein; however, with intangibles it could be more difficult to compare ratios and differences in ratios so an example where the ratios change to such an extent that rank also changes will be used. hence, one is not initially concerned with “rank preservation/reversal” but that the relative ratios have changed to such an extent that it is obvious that ratio preservation has not been maintained due to the effect that can occur with independent criteria cluster comparisons. 2.6 property of supermatrix vectors significant insight is provided by determining the unit of measurement in an anp model and that all the elements are related. there is also a need for clarification about the unit of measure of each of the eigenvectors from the paired comparison across the criteria and alternatives that are entered into the supermatrix. zahir (2007) and wedley (2013) explain that the “unity of normalization does not bear the same level of meaning – neither across the criteria nor across the decision space.” zahir (2007) originally coined the term “a one here does not necessarily equal to a one there.” the application was in the case of an ahp model and did not address the concept of a criteria cluster in one column versus another criteria cluster in another column. this concept can be more easily understood by referring to figure 1. as a supermatrix is an aggregation of multiple individual eigenvectors each summing to one, when they are weighted there is no reason to assume that the contributions of each eigenvector whether in the same column or rows provides an equal contribution to the overall system. this is demonstrated in the illustration in figure 1 that is similar to one displayed by wedley (2013). https://doi.org/10.13033/ijahp.v9i1.450 ijahp article: cooper, liu / achieving the desired level of dependency in anp decision models international journal of the analytic hierarchy process 8 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.450 c1.1 c1.2 c1.3 c2.1 c2.2 c2.3 a1 a2 a3 a4 c1.1 * * * * * * * * * * c1.2 * * * * * * * * * * c1.3 * * * * * * * * * * c2.1 * * * * * * * * * * c2.2 * * * * * * * * * * c2.3 * * * * * * * * * * a1 * * * * * * * * * * a2 * * * * * * * * * * a3 * * * * * * * * * * a4 * * * * * * * * * * * * * 1 a one here does not equal a one here figure 1. sample unweighted supermatrix with unequal eigenvectors figure 1 represents an unweighted supermatrix with four alternatives evaluated with respect to six criteria. the criteria have been organized as three subcriteria within two main criteria clusters. five eigenvectors representing sets of comparisons that sum to unity have been outlined. the three in the upper right hand corner consist of two from the same criteria cluster but from different alternatives and the third from a separate alternative in a separate criteria cluster. with the exception that a1 and a3 were identical alternatives while each eigenvector sums to unity there is no reason that a one here is equal to a one there or in other words that the clusters of dependent criteria provide an equal contribution or the same weight to each alternative. this example further exemplifies the effect of this unintended result because assigning the same criteria weights assumes the units of measurement are the same and can lead to unintended results. in the next section, a numerical example is used to demonstrate the outcomes of this effect. 3. models 3.1 problem definition three anp models are provided below to demonstrate the problem and solution. in each example one must choose which of three individuals to speak at a graduation ceremony. it is important to mention upfront that these examples provide a meaningful context wherein to evaluate the results of the current and proposed methods of weighting criteria clusters and are not meant to be comprehensive or universal models for choosing a graduation speaker. while the weightings are not in any shape or form the “optimal” weights for this type of model, nor are they unique to invoking the problem that is presented, they are reasonable and highlight the proposed concerns. in every example three unique criteria clusters are used to evaluate the potential speakers: prestige, public speaking skills, and availability. it is also worth noting that it is necessary to use more than one criteria cluster because in a model with a single cluster of criteria the criteria cluster is already column stochastic. the plural number of clusters used provides additional insight to the discussion in the literature review in sections 2.3-2.6. https://doi.org/10.13033/ijahp.v9i1.450 ijahp article: cooper, liu / achieving the desired level of dependency in anp decision models international journal of the analytic hierarchy process 9 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.450 the three alternatives in the examples are: 1) the president of a nation, 2) a governor or more local but very prominent leader, and finally 3) a local community leader. in the first example, each criteria cluster will have a single criterion within each cluster. in the subsequent examples, multiple criteria will be included within the criteria clusters. the counterintuitive results in the first two examples of this sample decision model using intangible criteria are used to demonstrate the potential unintended consequences of not capturing the dependence by performing individual cluster weighting comparisons in each column, and also to show that this unintended effect is independent of the tangibility of the elements. 3.2 model 1 figure 2. anp model and ratings for model 1 – unintended independence figure 2 displays the anp model and initial ratings of each of the alternatives with respect to the criteria that will be used in model 1. the criteria weights for the 3 separate criteria clusters here are chosen to represent an emphasis on prestige. from the priority vector for the criteria weights (figures 2 and 3) one could conclude that the value placed on prestige is as important as everything else put together. this vector is obtained by asking the three questions in superdecisions as shown in figure 3 with respect to the entire network. figure 3. model 1 criteria cluster weights while one may assume that the president will be the preferred option because of the emphasis on prestige, the results in table 1 demonstrate that with the use of a single criterion in each of the 3 separate criteria clusters in this anp model and that by performing criteria comparisons as is currently prescribed and demonstrated in figure 3 that this anp model is actually nothing more than an ahp model. the dependency of the alternatives on the criteria is lost. just as with an ahp model both the criteria cluster weights and the individual elements in this anp model are independent of the alternatives and this occurred even though the elements are intangibles. president prominent local criteria weights prestige recognition high medium-high low 0.5 public speaking humor medium medium medium 0.2 availability low medium-low high 0.3 https://doi.org/10.13033/ijahp.v9i1.450 ijahp article: cooper, liu / achieving the desired level of dependency in anp decision models international journal of the analytic hierarchy process 10 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.450 the first matrix in table 1 is the unweighted supermatrix. the entries in the top right hand side are all equal to one because there is only a single criterion in each criteria cluster. the three clusters in the unweighted supermatrix are then weighted to become the weighted anp supermatrix. because according to the current approach a criteria cluster is given the same weight across all the columns in the supermatrix this set of three identical vectors could be replaced by this same vector in an ahp model with a goal node and achieve the same results. for example, the prestige of the local leader is given the same weight as the prestige of the president. likewise, the availability of the president is also given the same weight as the availability of the local leader and hence the criteria cluster weights are independent of the alternatives. in the 2nd example, because there is more than a single criterion in each cluster there will be some level of dependence but ultimately that dependence is compromised just as in this first example. although it is not as clear as when a single criterion is used in each cluster, the effect is similar and hence not limited to the special case of a single criterion. the final results for model 1 in the limit matrix (table 1) show that every alternative is basically equal. not addressing the dependency resulted in the alternatives appearing almost identical. the important findings in the first example are a simple demonstration of the effect of not performing criteria cluster weighting comparisons individually for each alternative and that the undesirable effect applies to both tangible and intangible criteria and therefore is independent of the tangibility of the elements. https://doi.org/10.13033/ijahp.v9i1.450 ijahp article: cooper, liu / achieving the desired level of dependency in anp decision models international journal of the analytic hierarchy process 11 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.450 table 1 model 1 unweighted and weighted supermatrix and limit matrix unweighted supermatrix criteria alternatives recognition humor availability president prominent local criteria prestige recognition 0 0 0 1 1 1 public speaking humor 0 0 0 1 1 1 availability 0 0 0 1 1 1 alternatives president 0.4901 0.3333 0.0738 0 0 0 prominent 0.4507 0.3333 0.1218 0 0 0 local 0.0592 0.3333 0.8044 0 0 0 weighted supermatrix criteria alternatives recognition humor availability president prominent local criteria prestige recognition 0 0 0 0.5000 0.5000 0.5000 public speaking humor 0 0 0 0.2000 0.2000 0.2000 availability 0 0 0 0.3000 0.3000 0.3000 alternatives president 0.4901 0.3333 0.0738 0 0 0 prominent 0.4507 0.3333 0.1218 0 0 0 local 0.0592 0.3333 0.8044 0 0 0 limit matrix criteria alternatives recognition humor availability president prominent local criteria prestige recognition 0 0 0 0.5000 0.5000 0.5000 public speaking humor 0 0 0 0.2000 0.2000 0.2000 availability 0 0 0 0.3000 0.3000 0.3000 alternatives president 0.3338 0.3338 0.3338 0 0 0 prominent 0.3286 0.3286 0.3286 0 0 0 local 0.3376 0.3376 0.3376 0 0 0 this example with a single criterion in each criteria cluster clarifies that the need to perform criteria cluster comparisons for each alternative individually is not affected by the “tangibility” of the elements. regardless of whether the elements are tangible or intangible the criteria weights in this example are independent of the alternatives. there is no longer any dependency in this anp model with intangible criteria even though the needed connections were made in this anp model to achieve dependency. by not making the criteria cluster weighting comparisons separately for each alternative the desired dependency is lost. in the next example, the specific case of a single criterion is extended to a more general case where there are n+1 criteria in m clusters. 3.3 model 2 the second example is similar to the first in that one is choosing from the same three alternatives and using the same three criteria clusters. now with multiple criteria in some clusters, the impact of this effect can be seen in the current application of the anp. once more, the final results will seem counterintuitive because the criteria cluster weightings are not performed individually for each alternative in each column. an additional criterion is added in two of the three criteria clusters. the public speaking skills criteria cluster will now be measured with two individual elements: humor and public relations skills. the prestige criteria cluster will also now be measured with two individual https://doi.org/10.13033/ijahp.v9i1.450 ijahp article: cooper, liu / achieving the desired level of dependency in anp decision models international journal of the analytic hierarchy process 12 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.450 elements: prestige and charisma. the availability cluster will continue with a single element. president prominent local recognition high medium-high low charisma medium medium medium humor medium medium medium public relations medium medium medium availability low medium-low high figure 4. anp model and ratings for models 2-3 for simplification purposes, it is assumed that the candidates have equally perceived values in the amounts of charisma, humor and public relations. while this assumption is not necessary it is only used to provide clarity when the results and the potential unintended consequences of not modeling the dependence are discussed. with that assumption in place, the real differences between candidates is between their availability and prestige. depending on the user defined criteria weightings if the availability criterion is the most important, a less prestigious candidate will be chosen, whereas if the prestige criteria are determined to be more important a more prestigious leader will be chosen. figure 4 lists each alternative (candidate), the criteria, and a verbal rating for each alternative. similar criteria cluster weights are used in this model as those in the previous example; however, in an effort to counter the argument that the solution is to simply put more weight on the prestige cluster that argument is addressed in this example. figure 5 displays the new criteria cluster weights. these weights are used to weight the criteria clusters in the unweighted supermatrix to obtain the weighted supermatrix and the limit matrix in table 2. once again, because of the unintended consequences of failing to fully capture the dependency in the model, the president is not the preferred alternative. figure 5. model 2 criteria cluster weights https://doi.org/10.13033/ijahp.v9i1.450 ijahp article: cooper, liu / achieving the desired level of dependency in anp decision models international journal of the analytic hierarchy process 13 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.450 table 2 model 2 unweighted and weighted supermatrix and limit matrix semi-dependent unweighted supermatrix criteria alternatives recognition charisma humor public relations availability president prominent local criteria recognition 0 0 0 0 0 0.9 0.8 0.1 charisma 0 0 0 0 0 0.1 0.2 0.9 humor 0 0 0 0 0 0.5 0.5 0.5 public relations 0 0 0 0 0 0.5 0.5 0.5 availability 0 0 0 0 0 1 1 1 alternatives president 0.4901 0.3333 0.3333 0.3333 0.0738 0 0 0 prominent 0.4507 0.3333 0.3333 0.3333 0.1218 0 0 0 local 0.0592 0.3333 0.3333 0.3333 0.8044 0 0 0 semi-dependent weighted supermatrix criteria alternatives recognition charisma humor public relations availability president prominent local criteria recognition 0 0 0 0 0 0.5143 0.4571 0.0571 charisma 0 0 0 0 0 0.0571 0.1143 0.5143 humor 0 0 0 0 0 0.0714 0.0714 0.0714 public relations 0 0 0 0 0 0.0714 0.0714 0.0714 availability 0 0 0 0 0 0.2857 0.2857 0.2857 alternatives president 0.4901 0.3333 0.3333 0.3333 0.0738 0 0 0 prominent 0.4507 0.3333 0.3333 0.3333 0.1218 0 0 0 local 0.0592 0.3333 0.3333 0.3333 0.8044 0 0 0 semi-dependent limit matrix criteria alternatives recognition charisma humor public relations availability president prominent local criteria recognition 0 0 0 0 0 0.3231 0.3231 0.3231 charisma 0 0 0 0 0 0.2483 0.2483 0.2483 humor 0 0 0 0 0 0.0714 0.0714 0.0714 public relations 0 0 0 0 0 0.0714 0.0714 0.0714 availability 0 0 0 0 0 0.2857 0.2857 0.2857 alternatives president 0.3098 0.3098 0.3098 0.3098 0.3098 0 0 0 prominent 0.3108 0.3108 0.3108 0.3108 0.3108 0 0 0 local 0.3794 0.3794 0.3794 0.3794 0.3794 0 0 0 the results in the limit matrix in table 2 identify the local leader as the preferred alternative with a priority that is 1.22 times greater than the priority for the president. these results may seem somewhat counterintuitive given the criteria weights used in this example. prestige was considered the most important criteria (.5714); the candidates have identical amounts of public relations (.1429), and availability was half as important as the prestige (.2857). why then is the local leader the preferred alternative? it is in this model, where there are multiple criteria clusters with multiple elements, that a definition for a fully-dependent anp model becomes essential. a fully-dependent anp model is one where not only the weights of the individual criterion elements for each alternative vary from one alternative to the next to reflect the specific contribution each makes to the respective alternative, but also the weighting of the criteria clusters must also reflect the specific contribution made individually to each alternative. because the prestige cluster is weighted the same for each alternative, the criteria cluster weightings are ultimately independent of the alternatives. it is true that the weight given to the prestige cluster is distributed differently among the two criteria elements within that cluster in each alternative. however, because the criteria cluster receives the same weighting for each alternative the contribution of the prestige of the local leader is over https://doi.org/10.13033/ijahp.v9i1.450 ijahp article: cooper, liu / achieving the desired level of dependency in anp decision models international journal of the analytic hierarchy process 14 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.450 weighted and the remainder of the prestige weight for the local leader is then allocated to its charisma (refer to the highlighted cells in the middle of table 2). the prestige and charisma of each alternative are victims of not addressing the model as fully-dependent and the alternatives appear much more similar than they really are. in regards to the public relations cluster, even though the public relations is the same for each alternative, the cluster should probably not provide the same relative contribution to the overall value for each alternative because the relative contribution with respect to each individual alternative will most likely be different. in the availability cluster (also highlighted in the middle of table 2) each alternative has an equal contribution even though the comparisons represented in the left quadrant of the supermatrix seem to imply the availability of each alternative differs significantly. a solution to avoiding these confusing results is to create a fully-dependent anp model by performing individual criteria cluster weighting comparisons for each column in the supermatrix. the proof is provided in the appendix and supported with an example incorporating its results in the next section. the desired eigenvector to be obtained from the synthesis of the fully-dependent supermatrix is defined in the form 1 / n i i i a a   , which provides the relative priority or contribution of an alternative ai with respect to the system of n alternatives being considered. in other words, it is equal to the relative contribution of all the attributes which an individual alternative possesses with respect to the contribution of all the alternatives considered in a decision. the first step in the proof is to show that the max  =1 in a weighted supermatrix which is a column stochastic matrix by the gershgorin circle theorem; the importance of this finding has also been emphasized by lipovetsky who demonstrates that when solving max ax x that max  =1 with multiple lambda equal to 1 (lipovetsky, 2011). next, by invoking the perron–frobenius theorem one can obtain the limiting priority vector. according to the perron–frobenius theorem there is a unique solution to the equation 1ax x and by restricting x to be of a particular form 1 / n i i i a a   therefore the supermatrix a must also be of a particular form wherein the criteria weightings are performed for each alternative individually. for the complete proof refer to the appendix. 4. achieving full-dependency the third example begins with the same initial ratings and unweighted supermatrix which was used in model 2. criteria cluster weighting comparisons will again be used to weight the criteria clusters, but now according to the method shown in the proof. in summary, the criteria cluster weighting comparisons will be performed individually in each column. obtaining unique criteria cluster weights for each column will provide the necessary weights to obtain a column stochastic supermatrix. the cluster weights are obtained for each column using the same pairwise comparison process. the only difference is the questions are asked with respect to an individual alternative and then https://doi.org/10.13033/ijahp.v9i1.450 ijahp article: cooper, liu / achieving the desired level of dependency in anp decision models international journal of the analytic hierarchy process 15 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.450 repeated for each alternative; the criteria cluster weights and inconsistency indices are presented in table 3. 4.1 model 3 it is worth noting the differences in the criteria cluster weights in table 3. the president obtains most of her value from the prestige cluster and little from the availability cluster, while the value of the local leader is more that he is available than that he is prestigious. the criteria cluster weights are unique to each alternative and the contributions of the alternatives are weighted according to the unique contribution of said criteria to the alternative. this is the advantage of recognizing that a “one here does not necessarily equal a one there” and designing a fully-dependent anp model by obtaining criteria cluster weights for each individual cluster in each column. in table 4 the weights from table 3 are used to obtain the weighted supermatrix. this weighted supermatrix is raised to powers and the limit matrix is obtained. table 3 fully-dependent cluster weights and inconsistency index criteria cluster comparisons by column cluster comparisons president cluster comparisons prominent cluster comparisons local prestige public speaking availability eigenvector prestige public speaking availability eigenvector prestige public speaking availability eigenvector prestige 1 8 9 0.8142 prestige 1 6 8 0.7514 prestige 1 0.5 0.25 0.1494 public speaking 0.125 1 2 0.1140 public speaking 0.1667 1 2 0.1782 public speaking 2 1 0.5 0.3764 availability 0.1111 0.5 1 0.0718 availability 0.125 0.5 1 0.0704 availability 4 1 1 0.4742 inconsistency 0.0516 0.0280 0.05156 https://doi.org/10.13033/ijahp.v9i1.450 ijahp article: cooper, liu / achieving the desired level of dependency in anp decision models international journal of the analytic hierarchy process 16 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.450 table 4 model 3 unweighted and weighted supermatrix and limit matrix fully-dependent model fully-dependent unweighted supermatrix criteria alternatives recognition charisma humor public relations availability president prominent local criteria recognition 0 0 0 0 0 0.9 0.8 0.1 charisma 0 0 0 0 0 0.1 0.2 0.9 humor 0 0 0 0 0 0.5 0.5 0.5 public relations 0 0 0 0 0 0.5 0.5 0.5 availability 0 0 0 0 0 1 1 1 alternatives president 0.4901 0.3333 0.3333 0.3333 0.0738 0 0 0 prominent 0.4507 0.3333 0.3333 0.3333 0.1218 0 0 0 local 0.0592 0.3333 0.3333 0.3333 0.8044 0 0 0 fully-dependent weighted supermatrix criteria alternatives recognition charisma humor public relations availability president prominent local criteria recognition 0 0 0 0 0 0.7328 0.6011 0.0149 charisma 0 0 0 0 0 0.0814 0.1503 0.1344 humor 0 0 0 0 0 0.0570 0.0891 0.1882 public relations 0 0 0 0 0 0.0570 0.0891 0.1882 availability 0 0 0 0 0 0.0718 0.0704 0.4742 alternatives president 0.4901 0.3333 0.3333 0.3333 0.0738 0 0 0 prominent 0.4507 0.3333 0.3333 0.3333 0.1218 0 0 0 local 0.0592 0.3333 0.3333 0.3333 0.8044 0 0 0 fully-dependent limit matrix criteria alternatives recognition charisma humor public relations availability president prominent local criteria recognition 0 0 0 0 0 0.4778 0.4778 0.4778 charisma 0 0 0 0 0 0.1209 0.1209 0.1209 humor 0 0 0 0 0 0.1064 0.1064 0.1064 public relations 0 0 0 0 0 0.1064 0.1064 0.1064 availability 0 0 0 0 0 0.1885 0.1885 0.1885 alternatives president 0.3593 0.3593 0.3593 0.3593 0.3593 0 0 0 prominent 0.3495 0.3495 0.3495 0.3495 0.3495 0 0 0 local 0.2912 0.2912 0.2912 0.2912 0.2912 0 0 0 the results from the limit matrix for model 3 show the preferred speaker is the president. the prominent leader is the next most preferred and the local leader is now the least preferred. ratio preservation has been achieved by performing the criteria cluster weighting comparisons for each column (alternative) individually. these differences can be seen in the highlighted cells in the weighted supermatrix in the middle of table 4 and compared with the same entries in the weighted supermatrix in model 2 in table 2. this contrast in the weighted supermatrices in tables 2 and 4 underscores the benefits of calculating the criteria cluster weights separately for each column in an anp model. this final priority vector is fully-dependent on the alternatives and criteria; the final priority vector now reflects the relative contribution of each alternative with respect to the entire network. 4.2 independence of irrelevant alternatives while we agree with wedley and choo (2011) that the argument should be concerned with what is the unit of measurement and rank preservation, we will demonstrate that by https://doi.org/10.13033/ijahp.v9i1.450 ijahp article: cooper, liu / achieving the desired level of dependency in anp decision models international journal of the analytic hierarchy process 17 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.450 implementing step 6, as proposed herein, a fully-dependent anp model is not subject to rank reversal when adding both irrelevant and identical alternatives. rank reversal is still a phenomenon worth resolving (hefnaway & mohammed, 2014). the independence of irrelevant alternatives (iia) hangs on the findings of the gershgorin circle theorem from which we can conclude there is a unique solution to the equation max ax x . (10) in this paper, three alternatives were used and their priorities are equal to the following priority vectors in the limit matrix where represents the total influence of each alternative i: 1 / n i i i a a   = (11) in a fully-dependent anp model because the columns were normalized individually the portion of the supermatrix representing the distribution of the criteria among the alternatives does not need to be changed or modified. rather a new column representing the distribution of the criteria among the new alternative is added. the distributions of the alternatives among each criterion are updated only by comparing the new alternative against the original alternatives. adding copy of will result in the following vector: while the individual priorities of each alternative will be smaller, the relative weight of each alternative with respect to the other alternatives remains unchanged. the iia property is another advantage of fully-dependent anp models. while the advantages of performing cluster comparisons individually for each column has been demonstrated with numerical examples the generalization in general mathematical terms in the proof in the appendix provides an even stronger argument for the advantages of performing the additional cluster comparisons to achieve fullydependent anp models. 5. conclusion the ahp and anp can be categorized as disruptive technology. they are amazing frameworks that are used to model independent criteria and fully-dependent criteria as demonstrated above. because a fully-dependent anp model requires additional pairwise comparisons it is crucial that decision makers determine upfront what they want the final priority vector to represent; that is, whether they want a final priority vector that represents independence or dependence among the criteria and alternatives. the need to address this question upfront in the decision-making process is further underscored by the research discussed in the literature review regarding ahp and anp. fully-dependent anp models as put forth in this paper are not intended to be a universal approach to all anp models; as was stated earlier, there are very good reasons to model the criteria independent of the alternatives. the same can be said for modeling the criteria as fullydependent on the alternatives. future research is needed to determine the meaning and interpretation of final priority vectors if one wishes to use a form of the semi-dependence https://doi.org/10.13033/ijahp.v9i1.450 ijahp article: cooper, liu / achieving the desired level of dependency in anp decision models international journal of the analytic hierarchy process 18 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.450 among the criteria and alternatives where the criteria clusters are weighted equally across columns in an anp model. additionally, it will be useful to quantify the impact of criteria cluster weighting techniques on the coherency of the anp supermatrix. what is most important in this paper is that if one wishes to model a fully-dependent system which will provide a final priority vector of the form in the limit matrix, which is interpreted as the relative priority or contribution of an alternative ai with respect to, or dependent on, the system of n alternatives being considered then the criteria cluster weighting comparisons must be performed individually in each column of the supermatrix. it is also important to recognize that this undesirable effect exists independent of the tangibility of the elements. this point was demonstrated through the discussion in the literature review regarding tangibility and clarified through models contained herein. one potential reason this issue has not been addressed in the literature previously is that prior proofs contained a single criteria cluster and did not contain multiple criteria clusters. the proof contained herein contains multiple criteria clusters and provides the generalization of the proposed method. fully-dependent anp models allow the decision maker to capture a greater level of dependence in the anp than is currently available and will further aid decision makers to make better decisions. https://doi.org/10.13033/ijahp.v9i1.450 ijahp article: cooper, liu / achieving the desired level of dependency in anp decision models international journal of the analytic hierarchy process 19 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.450 references arbel, a. (1983). a university budget problem: a priority-based approach. socioeconomic planning sciences, 17(4), 181-189. doi: https://doi.org/10.1016/00380121(83)90019-8 belton, v., & gear, t. (1983). on a short-coming of saaty's method of analytic hierarchies. omega, 11(3), 228-230. doi: 10.1016/0305-0483(83)90047-6 choo, e. u., schoner, b., & wedley, w. c. (1999). interpretation of criteria weights in multicriteria decision making. computers & industrial engineering, 37(3), 527-541. doi: http://doi.org/10.1016/s0360-8352(00)00019-x dyer, (1990). remarks on the analytic hierarchy process. management science, 36(3), 249-258. doi: http://dx.doi.org/10.1287/mnsc.36.3.249 dyer, j. w. r. e. (1985). a critique of the analytic hierarchy process. technical report, 84/85-4-24, (department of management, the university of texas at austin). harker, p. t. (1987). incomplete pairwise comparisons in the analytic hierarchy process. mathematical modelling, 9(11), 837-848. doi: 10.1016/0270-0255(87)90503-3 harker, p. t., & vargas, l. g. (1990). reply to "remarks on the analytic hierarchy process" by dyer. management science, 36(3), 269-273. doi: http://dx.doi.org/10.1287/mnsc.36.3.269 hefnaway, a. e. & mohammed, a. s. (2014). review of different methods for deriving weights in the analytic hierarchy process. international journal of the analytic hierarchy process, 6(1), 92-123. doi: http://dx.doi.org/10.13033/ijahp.v6i1.226 lipovetsky s., (2011) priority eigenvectors in analytic hierarchy/network processes with outer dependence between alternatives and criteria, international journal of the analytic hierarchy process, 3(2), 172-179. doi: http://dx.doi.org/10.13033/ijahp.v3i2.123 lipovetsky, s. (2011). an interpretation of the ahp global priority as the eigenvector solution of an anp supermatrix. international journal of the analytic hierarchy process, 3(1), 70-78. doi: http://dx.doi.org/10.13033/ijahp.v3i1.90 lipovetsky s., (2013). supermatrix eigenproblem and interpretation of priority vectors in analytic network process., international journal of the analytic hierarchy process, 5 (1), 105-113. doi: http://dx.doi.org/10.13033/ijahp.v5i1.132 lipovetsky, s. & conklin w. (2015). ahp priorities and the markov-chapmankolmogorov steady-states probabilities. international journal of the analytic hierarchy process, 7(2), 349-363. doi: http://dx.doi.org/10.13033/ijahp.v7i2.243 https://doi.org/10.13033/ijahp.v9i1.450 https://doi.org/10.1016/0038-0121(83)90019-8 https://doi.org/10.1016/0038-0121(83)90019-8 http://doi.org/10.1016/s0360-8352(00)00019-x http://dx.doi.org/10.1287/mnsc.36.3.249 http://dx.doi.org/10.1287/mnsc.36.3.269 http://dx.doi.org/10.13033/ijahp.v3i1.90 http://dx.doi.org/10.13033/ijahp.v7i2.243 ijahp article: cooper, liu / achieving the desired level of dependency in anp decision models international journal of the analytic hierarchy process 20 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.450 saaty (1977a). a scaling method for priorities in hierarchical structures. journal of mathematical psychology, 15(3), 234-281. doi: http://doi.org/10.1016/00222496(77)90033-5 saaty (1977b). the sudan transport study. interfaces, 8(1), 37-57. doi: http://dx.doi.org/10.1287/inte.8.1pt2.37 saaty (1986). axiomatic foundation of the analytic hierarchy process. management science, 32(7), 841-855. doi: http://dx.doi.org/10.1287/mnsc.32.7.841 saaty (1990). an exposition on the ahp in reply to the paper "remarks on the analytic hierarchy process". management science, 36(3), 259-268. doi: http://dx.doi.org/10.1287/mnsc.36.3.259 saaty (1996). the analytical hierarchy process (2nd ed. vol. 1). pittsburgh: rws publications. doi: http://doi.org/10.1016/b978-0-08-032599-6.50008-8 saaty (1996) decision making with dependence and feedback: the analytic network process. pittsburgh: rws publications. doi: 10.4018/978-1-59140-702-7.ch018 saaty (2005). theory and applications of the analytic network process: decision making with benefits, opportunities, costs, and risks. pittsburgh, pa: rws publications. doi: 10.1007/978-3-540-92828-7_4 saaty (2008a). the analytic hierarchy and analytic network measurement processes: applications to decisions under risk. european journal of pure and applied mathematics,1,122-196. saaty, (2008b). relative measurement and its generalization in decision making – why pairwise comparisons are central in mathematics for the measurement of intangible factors: the analytic hierarchy/network process, racsam, 102, 251–318. doi: 10.1007/bf03191825 saaty (2011). aligning the measurment of tangibles with intagibles and not the converse. international journal of the analytic hierarchy process, 3(1),79-87. doi: http://dx.doi.org/10.13033/ijahp.v3i1.91 saaty & cillo, b. (2008). the encyclicon: a dictionary of complex decisions using the analytical network process (vol. 2). pittsburgh, pa: rws publications. doi: 10.1007/978-1-4614-7279-7_1 saaty & vargas, (2006). decision making with the analytic network process: economic, political, social and technological applications with benefits, opportunities, costs and risks (vol. 95). springer. saaty (2016). five wasy to combine tangibles with intangibles. international journal of the analytic hierarchy process, 8(2), 372-381. doi: http://dx.doi.org/10.13033/ijahp.v8i2.395 https://doi.org/10.13033/ijahp.v9i1.450 http://doi.org/10.1016/0022-2496(77)90033-5 http://doi.org/10.1016/0022-2496(77)90033-5 http://dx.doi.org/10.1287/inte.8.1pt2.37 http://dx.doi.org/10.1287/mnsc.32.7.841 http://dx.doi.org/10.1287/mnsc.36.3.259 http://doi.org/10.1016/b978-0-08-032599-6.50008-8 http://dx.doi.org/10.13033/ijahp.v3i1.91 http://dx.doi.org/10.13033/ijahp.v8i2.395 ijahp article: cooper, liu / achieving the desired level of dependency in anp decision models international journal of the analytic hierarchy process 21 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.450 schoner, b., wedley, w. c., & choo, e. u. (1993). a unified approach to ahp with linking pins. european journal of operational research, 64(3), 384-392. . doi: 10.1016/0377-2217(93)90128-a. wallenius, j., dyer, fishburn, p. c., steuer, r. e., zionts, s., & deb, k. (2008). multiple criteria decision making, multiattribute utility theory: recent accomplishments and what lies ahead. management science, 54(7), 1336-1349. doi: 10.1287/mnsc.1070.0838 wedley, w. c. (2013). ahp/anp before, present and beyond. paper presented at the isahp 201, kuala lumpar, malaysia. wedley, w. c., & choo, e. u. (2001). a unit interpretation of multi-criteria ratios. paper presented at the proceedings of the sixth international symposium on the analytic hierarchy process, berne, switzerland. wedley, w. c., & choo, e. u. (2011). multi‐criteria ratios: what is the unit? journal of multi‐criteria decision analysis, 18(3-4), 161-171. doi: 10.1002/mcda.463 zahedi, f. (1986). the analytic hierarchy process—a survey of the method and its applications. interfaces, 16(4), 96-108. doi: http://dx.doi.org/10.1287/inte.16.4.96 zahir, s. (2007). a new approach to understanding and finding remedy for rank reversals in the additive analytic hierarchy process. paper presented at the administrative sciences association of canada (asac). https://doi.org/10.13033/ijahp.v9i1.450 http://dx.doi.org/10.1287/inte.16.4.96 ijahp article: cooper, liu / achieving the desired level of dependency in anp decision models international journal of the analytic hierarchy process 23 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13022/ijahp2017.v9.is1.450 ijahp article: cooper, liu / achieving the desired level of dependency in anp decision models international journal of the analytic hierarchy process 24 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13022/ijahp2017.v9.is1.450 ijahp article: cooper, liu / achieving the desired level of dependency in anp decision models international journal of the analytic hierarchy process 25 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13022/ijahp2017.v9.is1.450 ijahp article: cooper, liu / achieving the desired level of dependency in anp decision models international journal of the analytic hierarchy process 26 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13022/ijahp2017.v9.is1.450 ijahp news and events: saaty/best practices in analytic network studies: interview with enrique mu and orrin cooper international journal of the analytic hierarchy process 357 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.779 best practices in analytic network studies: interview with enrique mu and orrin cooper rozann saaty ijahp managing editor enrique mu, gabriela sava and orrin cooper at the anp best practice presentation at informs2017 enrique mu, ijahp editor-in-chief, orrin cooper, ijahp editorial board member and michael peasley, ahp/anp researcher, have recently published the paper “best practices in analytic network studies” (mu, cooper, & peasley, 2020). this study provides a set of guidelines to improve the validity of the anp model and reporting. given the importance of this study, i decided to interview the leading authors about this vital subject for the benefit of ijahp readers. 1. how did you get the idea to write a paper about anp best practices? enrique: i think the first thought about this came to me during the panel discussion “publishing ahp/anp papers” at the 12 th international symposium of the analytic hierarchy process, isahp2013, in kuala lumpur, malaysia. there, i gave a presentation titled, “what makes an ahp/anp paper publishable?” and started to list some of the criteria that, in my editorial and reviewer experience, made a paper publishable such as: who developed the model?, how were the group judgments aggregated?, how was group consistency addressed?, and so forth (defelice et al., ijahp news and events: saaty/best practices in analytic network studies: interview with enrique mu and orrin cooper international journal of the analytic hierarchy process 358 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.779 2013). over the years, i began to compile a checklist of these criteria that i would use as a pre-screening tool for the articles submitted to our ijahp journal. in 2016, when i was discussing a study that would require a literature review of anp papers published the previous year with orrin cooper and michael peasley, the idea came up about taking advantage of this literature review to check how anp studies were reported. the initial set of publishing criteria was corrected and improved in discussions with orrin and michael and as a result of the anp literature review. we called these criteria “checklist of best practices in anp reporting” and started to test them on anp paper submissions in our editorial and reviewers’ activities. orrin: enrique approached me in 2016 while we were working with each other on other projects. i could tell right away that something was bothering him. i could see it weighing on his mind. he talked generally about some of the papers he saw as the editor of ijahp and the inconsistency in what was reported which in some cases led him to doubt the validity of the results in the submitted papers. he suggested the idea of developing a set of best practices for ahp/anp. personal experiences from reviewing journal submissions and working with mba students who were taking dr. saaty’s class about the anp immediately came to my mind. enrique was on to something! i knew this would both help users understand the anp better and improve the quality of anp reports which in the end would further the recognition and future application of the anp. this would be a challenging but influential project to tackle. and so the journey started. 2. i understand you collected a great deal of papers for this study, what were your main findings with respect to what the best practices in anp reporting are? enrique: for just the year 2015, there were more than one hundred studies; however, in the end we focused on 84 relevant studies that were published that year in web of science journals. the preliminary results were presented at informs2017, and we basically found that most papers that were published were not consistent in what they reported. those who did a good job reporting how the anp model had been developed would sometimes not report if and how group inconsistency had been addressed and so forth. these findings further confirmed the need for some tentative standards that would constitute best practices to report anp studies. based on what we found, we further refined our proposed best practices, and a preliminary version of the study and new checklist was presented at isahp2018 in hong kong (mu, cooper, & peasley, 2018). orrin: i will admit i was a little nervous to meet with enrique to review our initial results from the literature review. i even doubted my reviews because of how many papers had not addressed what we determined as crucial information to include. some authors said nothing about consistency in the pairwise comparison matrices – not even something like “we checked the consistency/inconsistency.” few papers presented any sensitivity analysis, and others only provided a part of the final priority vector without any of the supermatrices. many of the weighted supermatrices were not column stochastic and so sparse that there weren’t enough connections to raise the supermatrix to powers. i was very surprised at how inconsistent the reporting was. when our results were the same, it underscored the need to have best practices. our list of best practices went through another revision based on what we had found. ijahp news and events: saaty/best practices in analytic network studies: interview with enrique mu and orrin cooper international journal of the analytic hierarchy process 359 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.779 speaking of revisions, i am grateful for the feedback that others provided during the conference presentations. we reorganized the best practices and checklists and it was interesting at that point to see how the current checklist came together until we finally felt like we got it right, so then we returned to evaluating the papers. 3. what are the main contributions of your study? enrique: the most important contribution is that for the first time, tentative standards for reporting anp studies have been suggested. this allows the overall validity of the anp study to be assessed. these standards are provided in the form of validity checklists for both ahp and anp. orrin: i think it is pretty simple, best practices improve the quality of work. the regression analogy we use in the paper comes to mind where not calculating and reporting a p-value would raise red flags. the standards serve as a guide: first, to make sure the appropriate analysis was done; and, second, that enough information is reported about the study. these two factors increase the validity of the study and exponentially increase the ability to replicate the results and use them in other settings. 4. where can we get these ahp/anp best practices checklists? enrique: the pre-print versions of these checklists are included here as appendices, for the reader’s convenience. the final printed version of the study for which these checklists were developed is available at: https://doi.org/10.1016/j.eswa.2020.113536. we strongly recommend reading the study to understand the validity criteria definitions and their examples. speaking of examples, the anp validation criteria proposed in our study were applied, almost in totality, in an anp study about merging it functions for the city of pittsburgh, which was simultaneously being conducted (mu & stern, 2018). it also shows how to deal with a major concern about reporting with respect to length of the article and using appendices. therefore, we strongly recommend reading this study as a practical example of how to implement the checklist criteria in an actual published paper. 5. do you think these best anp practices will evolve over time? orrin: we definitely hope this remains a living document and is updated as the field continues to mature. we listed a few potential areas that over time may become more advanced and recognized as best practices. best practices must be updated and kept current so that anp researchers do the best research possible. there may even become sub-groupings like ordinal and cardinal consistency, dealing with group decision making, i.e., gaining consensus, dealing with “outliers” in the group, etc. each of these (sub-groupings) is important, but may not apply to a majority of models. this is what we felt was a minimum that applied to anp models in general. it is also important to emphasize that there is definitely more that can be included in reports and analysis. the best practices are not the comprehensive list of everything that can be done; again, i think authors need to recognize it as a minimum bar and a living document. https://doi.org/10.1016/j.eswa.2020.113536 ijahp news and events: saaty/best practices in analytic network studies: interview with enrique mu and orrin cooper international journal of the analytic hierarchy process 360 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.779 6. why is the model development explanation so important? enrique: in any research study, the first step is always ensuring the face validity of the study which in practical terms can be defined as “the degree to which a procedure, especially a psychological test or assessment, appears effective in terms of its stated aims” (lexico.com, 2020). if anp authors want their readers to spend the next hour of their lives reading what should be done about covid-19, they must start by providing face validity for their study. this develops trust between the authors and their readers. during my years as a doctoral student, i had a professor who used to tell me that the face validity phase of research was actually the aha! test of the study. indicating up front who developed the model, why, and how the model (e.g. factors) came to be provides face validity to the overall study and creates the motivation to dig deeper into the details. the greatest contribution of proposing an anp model to address a decision issue is that the model can be adopted, used or adapted by other members of the community with similar decision issues. for this purpose, trust in the validity of the proposed anp model is very important. thank you, enrique and orrin, for an illuminating discussion about best practices for what to include in anp articles and about writing research papers in general. ijahp news and events: saaty/best practices in analytic network studies: interview with enrique mu and orrin cooper international journal of the analytic hierarchy process 361 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.779 references defelice, f., karpak, b., mu, e., pecchia, l., petrillo, a., saaty, r. & vargas, l. (2013). publishing ahp/anp papers. international journal for the analytic hierarchy process, 5(1), 116-174. doi:https://doi.org/10.13033/ijahp.v5i1.187 lexico.com. (ed.) (2020) dictionary.com. uk: lexico.com. mu, e., cooper, o. & peasley, m. (2018). improving analytic network process reporting. paper presented at the international symposium of the analytic hierarchy process, hong kong. mu, e., cooper, o. & peasley, m. (2020). best practices in analytic network process studies. expert systems with applications, 159(30). doi:https://doi.org/10.1016/j.eswa.2020.113536 mu, e. & stern, h. (2018). a contingent/assimilation framework for public is interorganizational decisions. international journal of information technology and decision making, 17(6), 1611-1658. doi: 10.1142/s0219622018420014 https://doi.org/10.13033/ijahp.v5i1.187 https://doi.org/10.1016/j.eswa.2020.113536 ijahp news and events: saaty/best practices in analytic network studies: interview with enrique mu and orrin cooper international journal of the analytic hierarchy process 362 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.779 appendix anp best practices preliminary checklist version: 03-21-2020 source: mu, e., cooper, o., & peasley, m. (2020). best practices in analytic network process studies. expert systems with applications, 159(30). doi:https://doi.org/10.1016/j.eswa.2020.113536 item variable description ideal yours qualitative 1 model model development explanation who developed the model? how was the model developed? decisions models can be developed by the authors, experts and participation of stakeholders. authors should indicate which approach was used, indicating number of participants and their qualifications to participate as appropriate. 2 factor clusters and nodes 2a factor1 clusters and nodes must be clearly identified/labeled. 2b factor2 they must be clearly defined, indicating how they will be measured and used. 2c factor3 the source (e.g. extant literature, expert opinion, stakeholders) for the cluster/node must be specifically cited. 3 excompq example of comparison questions an example for each of the different comparison questions should be provided, including questions related to inner relationship comparisons. if surveys were used, either provide example questions or -even betterinclude the survey as an appendix. 4 gaggreg group decision aggregation if group decision making: how were the group decisions aggregated? was there any additional examination of the aggregation such as group dispersion analysis? 5 gconsist group consistency if group decision making: how was group consistency addressed? (e.g. did you discard respondents that were too inconsistent?) 6 subnet report on each of the sub-networks used in the study for each sub-network (e.g. b, o, c & r) report on the items in the checklist. quantitative 7 influencem influence matrix report the influence matrix showing the node interactions. 8 consist consistency 8a consist1 consistency threshold (e.g. cr <=0.1) explicitly stated. 8b consist2 how inconsistency situations were addressed. 9 clustercomp cluster comparison matrix report the cluster comparison matrix 10 limitmax limit comparison matrix 10a finalcritprior global priority for criteria from limit supermatrix 10b finalterprior global priority for alternatives from limit supermatrix 11 weightsupmax weighted supermatrix report the weighted supermatrix 12 sensitivity sensitivity analysis in complex models, sensitivity analysis is possible at different levels (e.g. individual subnets, strategic criteria). indicate which sensitivity analysis was chosen and why. 13 ratingscale ratings scales if applicable, both the development of the ratings scales and the scales must be reported. 14 subnetaggreg relationship among sub-networks how were sub-networks combined? for example, in bocr analysis there are different ways to combine the subnetworks (e.g. additive, multiplicative). https://doi.org/10.1016/j.eswa.2020.113536 ijahp news and events: saaty/best practices in analytic network studies: interview with enrique mu and orrin cooper international journal of the analytic hierarchy process 363 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.779 ahp best practices preliminary checklist version: 03-21-2020 source: mu, e., cooper, o., & peasley, m. (2020). best practices in analytic network process studies. expert systems with applications, 159(30). doi:https://doi.org/10.1016/j.eswa.2020.113536 item variable description ideal yours qualitative 1 model model development explanation who developed the model? how was the model developed? decisions models can be developed by the authors, experts and participation of stakeholders. authors should indicate which approach was used, indicating number of participants and their qualifications to participate as appropriate. 2 factor clusters and nodes 2 a factor1 clusters and nodes must be clearly identified/labeled 2 b factor2 they must be clearly defined, indicating how they will be measured and used. 2 c factor3 the source (e.g. extant literature, expert opinion, stakeholders) for the cluster/node must be specifically cited. 3 excompq example of comparison questions an example for each of the different comparison questions should be provided, including questions related to inner relationship comparisons. if surveys were used, either provide example questions or -even betterinclude the survey as an appendix. 4 gaggreg group decision aggregation if group decision making: how were the group decisions aggregated? was there any additional examination of the aggregation such as group dispersion analysis? 5 gconsist group consistency if group decision making: how was group consistency addressed? (e.g. did you discard respondents that were too inconsistent?) 6 subnet report on each of the sub-networks used in the study for each sub-network (e.g. b, o, c & r) report on the items in the checklist quantitative 8 consist consistency 8 a consist1 consistency threshold (e.g. cr <=0.1) explicitly stated. 8 b consist2 how inconsistency situations were addressed. 10 limitmax limit comparison matrix 1 0 a finalcritprior global priority for criteria from limit supermatrix 1 0 b finalterprior global priority for alternatives from limit supermatrix 12 sensitivity sensitivity analysis in complex models, sensitivity analysis is possible at different levels (e.g. individual subnets, strategic criteria). indicate which sensitivity analysis was chosen and why. 13 ratingscale ratings scales if applicable, both the development of the ratings scales and the scales must be reported. 14 subnetaggreg relationship among sub-networks how were sub-networks combined? for example, in bocr analysis there are different ways to combine the subnetworks (e.g. additive, multiplicative). https://doi.org/10.1016/j.eswa.2020.113536 ijahp article: yavuz, baycan/application of combined analytic hierarchy process (ahp) and swot for integrated watershed management international journal of the analytic hierarchy process issn 1936-6744 3 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.194 application of combined analytic hierarchy process (ahp) and swot for integrated watershed management fadim yavuz necmettin erbakan university konya, turkey fadimyavuz@konya.edu.tr tüzin baycan istanbul technical university istanbul, turkey tbaycan@itu.edu.tr abstract the most critical issue in watershed management is the active involvement of a range of stakeholder groups in the process. this paper offers an integrated approach to contribute to the integrated watershed management (iwm) process by using the analytic hierarchy process (ahp) and swot (strengths, weaknesses, opportunities, and threats) methods. the paper looks at beyşehir lake basin (blb), the largest freshwater lake and drinking water reservoir in turkey, and focuses on the most critical stage of iwm. this critical stage determines the optimal and agreed upon watershed management strategy from all of the stakeholder’s perspective. this strategy is referred to in this study as the ‘collaborative watershed management (cwm) strategy’. the combined ahp and swot methodology is applied to the real-life problems of: i) how to identify differences among the knowledge, experiences, values and interests of three different stakeholder groups including local communities, local authorities and experts regarding the agreed upon watershed management strategy, and ii) how to determine the cwm strategy that meets the expectations of all stakeholders in blb. the methodology is carried out via stages including describing swot factors, comparing these swot factors pair by pair to determine the relative weights of each, developing strategies based on those factors, evaluating each strategy alternative with respect to each swot factor, and performing final calculations. the study illustrates the feasibility of combining ahp and swot to incorporate stakeholder preferences in the decision making process of iwm. keywords: integrated watershed management, stakeholder-based decision making, analytic hierarchy process (ahp), ahp-swot, beyşehir lake basin 1. introduction integrated watershed management (iwm) has emerged as a new model for watershed planning following the trend towards more holistic and participatory approaches to natural resource management (desteiguer et al., 2003). iwm is the process of managing human activities and natural resources in an area defined by watershed boundaries, and aims to protect and manage natural resources for present and future generations. considering the integrity of the environment, economy and communities and using adaptive environmental management approaches, iwm offers an integrated interdisciplinary approach. iwm recognizes the importance of the human dimension. instead of focusing exclusively on biophysical processes and human impacts, iwm includes stakeholder participation, adaptive management, and experimentation that are compatible with critical ecosystem functions and services. rob typewritten text http://dx.doi.org/10.13033/ijahp.v6i1.194 ijahp article: yavuz, baycan/application of combined analytic hierarchy process (ahp) and swot for integrated watershed management international journal of the analytic hierarchy process issn 1936-6744 4 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.194 stakeholders are the people that directly and/or indirectly take part in watershed planning and management activities in the area and are affected by the actions in the basin. key stakeholders of a watershed may include people who can influence land management decisions, such as individual landowners, farmers, local government officials, representatives from environmental and community groups etc. (bonnell and baird, 2010). iwm is a process-oriented approach that provides a chance for stakeholders to balance diverse goals, and considers how their cumulative actions may affect long-term sustainability of watershed resources (qianxiang et al., 2005). iwm as a decision-making process makes it possible to address multiple issues and objectives, and enables planning in a very complex and uncertain environment. decision making in iwm typically involves several stakeholders with conflicting views. effective participation and conflict resolution are the most important challenges of the iwm approach (sharma et al., 2005). the related literature emphasizes the importance of consensual decision making in collaboration. margerum (1999) states that consensus is important not only for reaching an acceptable decision, but also for building long-term trust and support for outcomes. beierle (2002) suggests that it is the more intensive stakeholder processes that are more likely to result in higher-quality decisions. in order to succeed, iwm must be participatory, integrating all the relevant scientific knowledge/data and user-supplied information regarding the social, economic and environmental processes affecting natural resources at the watershed level. this paper offers an integrated approach to contribute to the iwm process by using the analytic hierarchy process (ahp) and swot (strengths, weaknesses, opportunities, and threats) methods. the paper addresses beyşehir lake basin (blb), the largest freshwater lake and drinking water reservoir in turkey, and focuses on the most critical stage of iwm, where the optimal and agreed upon watershed management strategy is determined by all of the stakeholders. this is referred to in this study as ‘collaborative watershed management (cwm) strategy’. identifying the cwm strategy is an important stage as it represents the culmination of the iwm process and sets the course for the future of the watershed. within this context, the differences among the knowledge, experiences, values and interests of three different stakeholder groups (local communities, local authorities and experts) with regard to the optimal and agreed upon watershed management strategy are assessed with the goal of protecting and restoring aquatic ecosystems, human health and other natural resources in blb. the paper consists of five sections. following a brief review of the stakeholder participation in iwm approach given in the introduction, section 2 describes the methodology of the combined use of ahp-swot. section 3 focuses on the empirical study and describes the case study area and the survey methodology. in this section, the participatory swot analysis for blb and the strategy formulation on the basis of swot analysis are also presented. next, section 4 explains the ahp-swot application steps and discusses the empirical results. the last section evaluates the application of combined ahp and swot as a tool for stakeholder-based decision making in iwm and discusses future research directions. 2. methodology: combined use of ahp and swot as a tool for stakeholderbased decision making in iwm the methodological framework includes the combined use of ahp and swot in developing cwm strategies, tallying swot factors, and prioritizing them with the pairwise comparison technique available with ahp. 2.1 ahp the multitude of watershed planning and management objectives inevitably leads to conflicts among watershed stakeholders or interest groups. it is often impossible to aggregate the objectives into a single criterion or performance measure in the alternative ranking and selection process. thus, multi-criteria (or multi-objective) decision support methods are widely applied in water policy planning and evaluation, strategic watershed planning and management, and ijahp article: yavuz, baycan/application of combined analytic hierarchy process (ahp) and swot for integrated watershed management international journal of the analytic hierarchy process issn 1936-6744 5 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.194 infrastructure development. multiple criteria analysis techniques have been used by water resource practitioners to select or to design alternatives in areas such as river basin planning and development, water resources development, land use management, groundwater/surface water allocation, watershed restoration and water resources quality (mirchi et al., 2010). since ahp is fairly well known for the audience of this journal we will only briefly introduce the methodology. ahp is a mathematical method for analysing complex decisions with multiple criteria. it has been translated into the level of analysis by thomas saaty. the technique has become a widely known and used for solving discrete multiple criteria problems. it has been successfully applied to many complex planning, resource allocation and priority setting problems in business, energy, health, marketing, natural resources and transportation (saaty, 2001). ahp is applied to the decision problem after it is structured hierarchically at different levels, each level consisting of a finite number of elements. fundamentally, ahp works by developing priorities for alternatives and the criteria are used to judge the alternatives. the estimation of the priorities from pairwise comparison matrices is the major component of the ahp. the importance or preferences of the decision elements are compared in a pairwise manner with regard to the element preceding them in the hierarchy. the priority vector can be derived from these pairwise comparison matrices using different techniques. the most commonly used technique is the eigenvector method (mikhailov, 2000). first of all, priorities are derived for the criteria in terms of their importance to achieve the goal, and then priorities are derived for the performance of alternatives on each criterion. these priorities are derived based on pairwise assessments using the judgement or ratios of measurements from a scale if one exists. finally, a weighting and adding process is used to obtain overall priorities for alternatives as to how they contribute to the goal. by additive aggregation ahp finally computes the priorities of the elements at the bottom level of the hierarchy, usually known as the alternatives. their priorities are interpreted with respect to the overall goal at the top of the hierarchy and elements at upper levels such as criteria, sub-criteria etc. are used to mediate comparison process (srdjevic, 2005). with the ahp, a multidimensional scaling problem is thus transformed to a uni-dimensional scaling problem. saaty (2001) suggests ahp as a formal method for rational and explicit decision making. it is a useful tool to analyse decisions in complex social and political problems. ahp is also useful when many interests are involved and a number of people participate in the judgement process. ahp is a straightforward and transparent method that is also able to consider subjective and judgemental information. the technique provides the objective mathematics to process the inescapably subjective and personal preferences of an individual or a group in making a decision. ahp can deal with qualitative as well as quantitative attributes. 2.2 swot swot analysis is a commonly used strategic planning method to evaluate the strengths (s), weaknesses (w), opportunities (o), and threats (t) involved in a project or business venture. generally swot is a list of statements or factors with descriptions of the present and future trends of both the internal and external environment; the expressions of individual factors are general and brief which describe subjective views. however, swot is a convenient and promising way of conducting a situational assessment (wickramasinghe and takano, 2009). 2.3 combined use of ahp-swot the use of ahp in swot analysis supports the strategic planning process quantitatively by providing analytical priorities to the swot factors. the combined use of the ahp and swot analysis has been widely used to support strategic decision-making processes such as institutional situation analysis and strategy selection (arslan, 2010; gürbüz, 2010), economical structure analysis (çelik and murat, 2008), stakeholder analysis in environmental management ijahp article: yavuz, baycan/application of combined analytic hierarchy process (ahp) and swot for integrated watershed management international journal of the analytic hierarchy process issn 1936-6744 6 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.194 (dwivedi and alavalapati, 2009), strategy selection in defense sector (kandakoğlu et al., 2007), developing and selecting strategy in forest management (kurttila et al., 2000; leskinen et al., 2006; masozera et al., 2006; shrestha et al., 2004), developing collaborative strategy in the health sector (osuna and aranda, 2007), selecting strategy in natural resource management (pesonen et al., 2001), project management (stewart et al., 2002), strategy development in industry sector (shinno et al., 2006; taşkın and güneri, 2005), developing and selecting strategy in tourism planning (kajanus et al., 2004; wickramasinghe, 2008), collaborative project evaluation (yılmaz, 2007), selection of the optimal reconstruction solution of a water intake structure within a regional hydro-system (srdjevic et al., 2012), and decision making in information technology (hacımenni, 1998).the technique has been also referred as a’wot in some studies (gürbüz, 2010; kajanus et al., 2004; leskinen et al., 2006; pesonen et al., 2001; taşkın and güneri, 2005; yılmaz, 2007). the first ahp-swot applications (kurttila et al., 2000, shrestha et al., 2004) have only focused on weighting the swot factors. the method has been developed by involving the evaluating processes of the strategy alternatives according to each swot factor and general priority calculations for the strategy alternatives. making pairwise comparisons forces the decision–makers to think over the weights of the swot factors and to analyze the situation more precisely and in more depth than the standard swot does. by integrating ahp with swot, not only the mutual weighting of swot factors, but also the evaluation of alternative strategic decisions can be integrated with ordinary swot analyses. in this way, the most crucial weakness of swot can be avoided (kangas et al., 2001; kangas et al., 2003; saaty and vargas, 2001, cited in dwivedi and alavalapati, 2009; yılmaz, 2007). the ahp-swot method increases and improves the information basis of the strategic planning processes, and not only provides a robust decision support, but also an effective framework for learning in strategic decision support. ahp-swot can be used as a communication and educational tool in the decision making processes if more than one decision maker exists. in addition, separate ahpswot applications for individuals or interest groups can provide a good basis for examining the vision and expectation differences of different stakeholders regarding a particular decisionmaking process (kangas et al., 2001). the ahp-swot combination is carried out in five stages (figure 1) (kangas et al., 2001 and yılmaz, 2007). figure 1. application stages of ahp-swot stage 1−swot analysis: the swot groups (strengths, weaknesses, opportunities and threats) are created. swot factors of each swot group that will be included in the analysis are ranked as neutral as possible. stage 2−pairwise comparisons between swot factors are performed using saaty's (2008) nine point scale (table 1) separately within each swot group. the comparisons are used as input to ijahp article: yavuz, baycan/application of combined analytic hierarchy process (ahp) and swot for integrated watershed management international journal of the analytic hierarchy process issn 1936-6744 7 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.194 the scope, and then the relative priorities of swot factors are calculated using the eigenvector approach of ahp technique. table 1 scale of two-paired comparison at ahp (saaty, 2008) intensity of importance definition explanation 1 equal importance two activities contribute equally to the objective 3 moderate importance experience and judgment slightly favour one activity over another 5 strong importance experience and judgment strongly favour one activity over another 7 very strong importance an activity is favoured very strongly over another; its dominance demonstrated in practice 9 extreme importance the evidence favouring one activity over another is of the highest possible order of affirmation 2, 4, 6, 8 intermediate values intermediate values stage 3− the next stage is the calculation of a list of the relative weights, importance, or value of the s, w, o and t factor groups (technically, this list is called an eigenvector). in this process, if s is absolutely more important than w and is rated at 9, then w must be absolutely less important than s and is valued at 1/9. these pairwise comparisons are carried out for all swot factors to be considered, and the matrix is completed. relative priorities of s, w, o and t factors are based on eigenvector values of the pairwise comparisons. stage 4−in this stage the strategy alternatives for each swot factor are evaluated. here, the relative priority value of each swot group is separately multiplied by the relative priority of each of the swot factors in this group. thus, the overall priority value of each swot factor in the related swot group is derived. this process is repeated for each of the swot groups. finally, the overall priority values of all the swot factors (of which total value is equal to 1) are obtained. at the end of each ahp calculation stage there is a need to calculate a consistency ratio (cr) to measure how consistent the judgments have been relative to large samples of purely random judgments. saaty has proved that the consistent reciprocal matrix, the largest eigen value is equal to the size of the comparison matrix, or λmax–n. the measure of consistency, called the consistency index (ci), is a deviation or degree of consistency using the following formula: ci=(λmax–n) / (n–1) (1) saaty proposes that ci be used by comparing it with the appropriate one. the appropriate ci is called the random consistency index (ri) (table 2). table 2 random consistency index (teknomo, 2006) n 1 2 3 4 5 6 7 8 9 10 ri 0 0 0.58 0.9 1.12 1.24 1.32 1.41 1.45 1.49 then, he proposes what is called cr, which is a comparison between ci and ri: cr= ci / ri (2) ijahp article: yavuz, baycan/application of combined analytic hierarchy process (ahp) and swot for integrated watershed management international journal of the analytic hierarchy process issn 1936-6744 8 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.194 if the value of cr is smaller or equal to 0.1, the inconsistency is acceptable. if the cr is greater than 0.1, the judgments are untrustworthy because they are too close to randomness. the subjective judgment is valueless or must be repeated. saaty suggests that if that ratio exceeds 0.1 the set of judgments may be too inconsistent to be reliable. a cr of 0 means that the judgments are perfectly consistent. stage 5−this stage includes general priority calculations for the strategy alternatives. ahp uses a principle of hierarchic composition to derive composite priorities of alternatives with respect to multiple criteria from their priorities with respect to each criterion. it consists of multiplying each priority of an alternative by the priority of its corresponding criterion and adding over all the criteria to obtain the overall priority of that alternative (saaty, 2003). in this study, weights of strategy alternatives are calculated using the following formula adapted from osuna and aranda (2007): : the global (relative) value of the strategy j (j = 1, 2, ...., n) (3) normalized value of the strategy weights: (4) where : normalized weight of the strategy, m: number of swot factors, n: number of strategies. 3. empirical study: stakeholder-based decision making in blb’s management this study aims to provide a better understanding of i) the critical problems of the blb, ii) the most important advantages of the basin in terms of ‘strengths’ and ‘opportunities’, iii) the problems regarding blb’s management, iv) the most important disadvantages of the basin in terms of ‘weaknesses’ and ‘threats’, v) the possible strategies that would ensure major positive changes towards the basin’s sustainability, vi) knowledge, perceptions and behaviours of the stakeholders (individual and institutional level), and vii) the optimal adaptive watershed management strategy that would be sensitive to the views of all stakeholders in the basin within the context of the field work in blb. household, local government and expert questionnaires are performed to achieve these purposes. 3.1 the case study area: beyşehir lake basin (blb) beyşehir lake, located in the southwest of konya closed basin, is the largest freshwater lake and drinking water reservoir in turkey. the basin, belonging to the konya and isparta province borders (figure 2), is significant both for humans as a source of fresh water, and the environment, due to its wetland ecosystem (babaoğlu, 2007). the lake has international importance according to the ramsar convention criteria. it also holds the statuses of important bird area (iba) and important plant area (ipa). various zones of the lake and its basin are protected under the 1 st , 2 nd and 3 rd degree natural site statuses, and the area has several declared national parks namely, beyşehir lake and kızıldağ. also, archaeological sites exist in the basin, and beyşehir lake has a drinking and potable water conservation area character. ijahp article: yavuz, baycan/application of combined analytic hierarchy process (ahp) and swot for integrated watershed management international journal of the analytic hierarchy process issn 1936-6744 9 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.194 figure 2. location of blb in turkey in recent years, blb has suffered from some environmental and socio-economic problems. inappropriate water policy and non-point source pollution in the lake which have led to variations in water levels have become striking environmental issues at the basin. 3.2. participatory swot analysis the ultimate success of a watershed management largely depends on the accuracy of an effective situational assessment. to assess the blb substantially, first a participatory swot analysis was conducted through expert interviews including a civil engineer, forest engineer, urban planner, hydrologist, geologist and tourism experts. the local authority interviews included the mayor, village headman and an employee, and household interviews were also conducted. next, the judgments of experts, local authorities and local communities regarding swot factors were aggregated. this aggregation helped cope with the difficulty resulting from the original long list of swot factors in ahp technique. the experts’ swot judgments that were close to each other were combined thematically to reduce the number of factors, and in this way blb’s current status was summarized on the basis of a comprehensive and detailed swot analysis. consequently, six strengths, seven weaknesses, six opportunities and eight threats factors were obtained. the swot analysis performed for blb is presented in table 3. table 3 participatory swot analysis for blb w e a k n e s s e s [ w ] s t r e n g t h s [ s ] [w1] inequalities in water use [w2] lack of importance attached to tourism as an instrument in the development of the basin [w3] lack or inadequacy of infrastructure services [w4] scarcity of employment opportunities [w5] problems in the institutional structure and legal system related to problem solving and management in the basin [w6] inadequacy of financial resources for activities to protect the lake [w7] limitations to construction facilities in the basin with national park statuses, inability to efficiently benefit from the lakeshore [s1] geographical position and accessibility [s2] water supply [s3] the environmental importance of the beyşehir lake [s4] supporting means of subsistence such as agriculture, animal husbandry, fishing [s5] historical importance [s6] suitable environment for nature friendly economic activities o p p o r t u n i t i e s [ o ] t h r e a t s [ t ] [o1] positional advantage [o2] construction of new konyaantalya (gembos) motorway [o3] derebucak derivation tunnel [o4] its suitability in terms of tourism development [o5] plans and projects to protect and develop the basin [o6] presence of financial resources such as the european union grant projects, world bank credits etc. [t1] migration of the population to the outside of the basin [t2] climate changes [t3] decline in the amount of lake water [t4] water pollution [t5] overhunting [t6] destruction of the lake ecosystem [t7] high taxes against the rise of the local economy [t8] interventions to basin’s water system from outside the basin ijahp article: yavuz, baycan/application of combined analytic hierarchy process (ahp) and swot for integrated watershed management international journal of the analytic hierarchy process issn 1936-6744 10 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.194 3.3 survey methodology in this study, we structured an analytical hierarchy for the blb’s iwm process based on a swot analysis. we also used ahp to estimate a global value for each of the strategy alternatives. initially, we used tows matrix, developed by weihrich (1982), to describe watershed management options based on the swot factors. tows matrix provides means to develop strategies based on logical combinations of swot factors related to internal strengths (or weaknesses) with factors related to external opportunities (or threats) (wickramasinghe, 2008). tows matrix identifies four conceptually distinct strategic groups to create the strategy alternatives including, i) strength-opportunity (so), ii) strength-threats (st), iii) weaknessesopportunities (wo), and iv) weaknesses-threats (wt). in this context, considering the expert views, we have proposed six strategy alternatives (alt). these alternatives consider the advantages of the strengths and opportunities while also reinforcing the weaknesses in order to develop the best defence strategy to the threats (table 4). table 4 strategy formulation using tows matrix strategy groups strategy alternatives so strategies: maxi-maxi strategies use strengths to maximize opportunities [alt 1] agricultural development [alt 2] environment friendly tourism development: rural tourism wo strategies: mini-maxi strategies reduce internal weaknesses or develop missing strengths are used to minimize external threats [alt 3] collaborative watershed management st strategies: maxi-mini strategies use internal strengths to minimize threats [alt 4] decreasing the water consumption in urban area wt strategies: mini-mini strategies reduce the internal weaknesses to avoid external threats (defensive strategy, worst case scenario) [alt 5] improving water quality control invasive pollutant [alt 6] improving water usage in rural areas and agriculture ahp begins with the development of a decision hierarchy including a main goal, sub-objectives and strategy alternatives. figure 3 and table 3 show the decision hierarchy used in the study. the hierarchy for the described problem was structured in four levels. the top level refers to the main goal, to develop the best watershed management strategy that enables both the environmental and socio-economic sustainability of the blb. the next level consists of decision objectives that take advantage of the strengths (s), to reinforce the weaknesses (w), to use the advantage of opportunities (o) and to develop the best defense to the threats (t). swot factors, described in swot analysis, take part in the third level. finally, the fourth level consists of the strategy alternatives (alt). how important are the internal strengths & weaknesses and the opportunities & threats arising from the external environment, or to what extent should they be ignored to achieve the specified purposes? what are the most important problems of the basin? what is the safest course that would lead to improvement of the lake's environmental conditions and the basin residents’ living conditions? ahp and swot integration has been used to answer these research questions from the perspectives of stakeholders. ijahp article: yavuz, baycan/application of combined analytic hierarchy process (ahp) and swot for integrated watershed management international journal of the analytic hierarchy process issn 1936-6744 11 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.194 figure 3. hierarchical structure to prioritize the swot factors of blb’s sustainability , , : relative importance of each group of factors (s, w, o and t) for the achievement of the strategic objective ( , …, ,): relative importance of the strengths factors (s1, s2, ..., sms) within their group (s) ( , …, ,): relative importance of the weaknesses factors (w1, w2, ..., wmw) within their group (w) ( , …, ,): relative importance of the opportunities factors (o1, o2, ..., omo) within their group (o) ( , …, ,): relative importance of the threats factors (t1, t2, ..., tmt) within their group (t) for any strategy j (j = 1, 2, ...., n); degree of relationship between factor and strategy : : efficiency of strategy j in taking the advantage of the strength factor si (i= 1, 2, ...., ms) : efficiency of strategy j in lessening the effects of the weakness factor wi (i = 1, 2, ...., mw) : efficiency of strategy j in taking the advantage of the opportunity factor oi (i = 1, 2, ...., mo) : efficiency of strategy j in facing the threat factor ti (i = 1, 2,…, mt) the data for the analysis was gathered from a survey conducted in 44 different settlements in blb in march and april of 2010. in order to determine the cwm strategy from the perspective of the stakeholders the following questionnaires were performed: i) 457 household (approximately 1.7 % sample size) questionnaires, ii) 27 local authorities (mayor, village headman and employee) questionnaires, and iii) 22 expert (civil engineer, forest engineer, urban planner, hydrologist, geologist, tourism expert, etc the household and local authority questionnaires were performed face to face by visiting all of the settlements, whereas the expert questionnaires were conducted using different channels such as phone calls and e-mails in addition to face to face interviews. following a pilot study by the authors, a professional survey team was trained and the rest of the survey was completed by this professional team. the survey ijahp article: yavuz, baycan/application of combined analytic hierarchy process (ahp) and swot for integrated watershed management international journal of the analytic hierarchy process issn 1936-6744 12 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.194 sheet was designed in order to be appropriate to the ahp-swot technique, and the decision hierarchy has been developed for blb (see for further information appendix a). for pairwise comparisons, the questionnaire consists of two parts: i) comparison of the two factors in order to determine environmental and socio-cultural sustainability of blb (goal), the most dominant factor (in the case of strength and opportunity) or the least favourable factor (in the case of weakness and threat), and ii) the intensity of importance. in this context, the survey sheet consists of tables comparing each factor in a particular swot category with other factors in the same category. survey participants were asked to compare the stated factor to other factors and evaluate their importance from their perspective. for example, during the pairwise comparisons of s1 and s2 factors, under the strengths heading, the responder first decided which factor was more important, then evaluated their relative importance on a scale of 1-9. the success of any swot factors in determining the best strategy was measured on a scale of 0-9. reliability of the responses to the questionnaire was tested with the “consistency ratio” (cr) formula (formula 1 and formula 2) as prescribed by the ahp technique. the crs of the matrices were below the limit value of 0.1. therefore, the judgments are acceptably consistent. table 5 shows the sampling sites, household size and number of local authority questionnaires, and figure 4 shows the study area and sampling sites. table 5 sampling sites and the sizes of household and local authority questionnaires settlement: h la settlement: h q settlement: q la settlement: h la akburun 6 çiftliközü 4 hüyük 12 2 sağlık 4 1 bademli 4 derbent 13 1 i̇lmen 4 sarıkabalı 4 belceğiz 5 doğanbey 13 i̇mrenler 5 selki 6 beyşehir 131 emen 5 karadiken 4 sevindik 3 budak 6 1 gedikli 4 1 karayaka 3 ş.karaağaç 48 2 burunsuz 5 gencek 5 1 kıreli 9 1 tolca 5 1 çamlıca 6 1 göçeri 5 kızılören 5 2 üstünler 7 1 çarıksaraylar 10 1 gölkaşı 5 1 kurucuova 6 üzümlü 20 çavuş 4 2 gölkonak 5 kuşluca 6 1 yenidoğan 6 1 çiçekpınar 7 2 gölyaka 3 1 mutlu 4 1 y.bademli 12 1 çiflikköy 3 huğlu 13 sadıkhacı 12 1 yeşildağ 10 total: 457 27 * h: household, la: local authory ijahp article: yavuz, baycan/application of combined analytic hierarchy process (ahp) and swot for integrated watershed management international journal of the analytic hierarchy process issn 1936-6744 13 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.194 beyşehi̇r lake akburun belceği̇z tolca sarikabali derbent çariksaray göçeri̇ kireli̇ hüyük çavuş şaki̇karaağaç huğlu yeşi̇ldağ gencek üstünler yeni̇şarbademli̇ kurucuova karadi̇ken beyşehi̇r gölkaşi gölkonak kuşluca budak sağlik kizilören emen selki̇ doğanbey çi̇ftli̇közü mutlu burunsuz çamlica i̇mrenler i̇lmen gedi̇kli̇ çi̇çekpinar gölyaka yeni̇doğan çi̇ftli̇kköy bademli̇ karayaka sadikhaci sevi̇ndi̇k üzümlü diğrak . yeni̇cekale salur ördekçi̇ yakaemi̇r çeltek fakilar göksöğüt örenköy çavındur arak beyköy çalti yukari di̇nek aşaği di̇nek arslandoğmus basdeği̇rmen muratbaşi köprüköy dumanli adaköy armutlu yassibelyeni̇köy kiyakdede akçabelen doğancik üçpinar karaali̇ bayindir göçü kayabaşi samlar ağilönü k.afsar bayat avdancik eğli̇kler i̇saköy değ.alti çukurkent kösk görünmez yeni̇ce suludere pinarbasi gökçekuyu eği̇rler başgözegönen gündoğdu sariköy yunuslar damlapinar karabayat derbenttekke d ği̇şe saraypinar yassiören çukurağil durak tepearasi legend sampling sites settlement river basin boundary province boundary 0 kilometers 4 8 figure 4. study area and sampling sites 4. empirical results and discussion an excel worksheet was used to perform ahp calculations. this section presents the empirical results according to the ahp-swot application steps consecutively. 4.1 priorities of swot factor groups table 6 shows the ahp priorities of the swot factor groups in terms of three stakeholder groups. ‘to develop the best defence to threats’ [t] is the most highly rated swot factor group from the perspective of local communities (40.1%), and also the basic determinant of local authority views (46.0%). contrary to the local community and local authority views, ‘to use the advantage of opportunities’ [o] is dominant (35.0 %) in the holistic perceptions of the experts. while the other two stakeholder groups define [t] category as their primary decision objective, [t] is ranked second in priority by the experts (28.4%). cwm strategy considers the common benefit of all stakeholders and is responsive to their expectations. consequently, we ijahp article: yavuz, baycan/application of combined analytic hierarchy process (ahp) and swot for integrated watershed management international journal of the analytic hierarchy process issn 1936-6744 14 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.194 derived the cwm strategy for blb by congregating stakeholder groups’ assessments that were made separately. [t] is the highest overrated swot factor group of the cwm strategy priorities calculated from the geometric means of three stakeholder group priorities (37.4 %), and ‘to take the advantage of strengths [s]’ is the least rated category (13.6%) (appendix-b, c, d, e). table 6 weights of the decision objectives from the perspectives of stakeholders s t a k e h o l d e r g r o u p s weights local communities local authorities experts overall stakeholders * to take the advantage of strengths 0.102 0.117 0.209 0.136 to reinforce the weaknesses 0.281 0.221 0.157 0.214 to use the advantage of opportunities 0.216 0.202 0.350 0.248 to develop the best defence to threats 0.401 0.460 0.284 0.374 * each value is the geometric mean of the row. 4.2 priorities of the swot factors local weight dispersions regarding swot factors explicitly show the importance of [t4] ‘water pollution’ and [t3] ‘decline in the amount of lake water’ factors from the perspective of local communities. local authorities emphasized the importance of [t] category like local communities, and more highly rated the [t4] ‘water pollution’ and [t6] ‘destruction of the lake ecosystem’ [t] factors. experts emphasized the importance of the [o] category. this group rated [o5] ‘plans and projects to protect and develop the basin’, and [o4] ‘its suitability in terms of tourism development’ the highest. while [t] is accepted as the most important overall swot category with respect to cwm strategy, all of the stakeholders rated [t4] ‘water pollution’, [t3] ‘decline in the amount of lake water’ and [t6] ‘destruction of the lake ecosystem’ the highest factors in this category (appendix-b, c, d, e). figure 5 shows the differences in the swot factor prioritizations of the three stakeholder groups. the most important differences observed in the weight dispersions of swot factors are: i) experts supported the [o] factors with the highest scores and, ii) local authorities supported the [t] factors with the highest scores compared to other stakeholders. figure 5. priorities of the swot factors from the perspectives of stakeholders ijahp article: yavuz, baycan/application of combined analytic hierarchy process (ahp) and swot for integrated watershed management international journal of the analytic hierarchy process issn 1936-6744 15 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.194 4.3 global weight dispersions of strategy alternatives with respect to cwm strategy our findings suggest that amongst six strategy alternatives, the ‘collaborative watershed management (public-corporate-experts cooperation) [alt 3], ‘reduces internal weaknesses or develops missing strengths to minimize external threats’, is perceived as the most important approach (17.4 %) by all stakeholders to solve the basin’s problems (for further information see appendix-f, g). this preference points out that all stakeholders are aware of the necessity of coordination and cooperation to gain effective watershed planning and management activities. stakeholders agree that ‘the suitability of the basin to the development of tourism’ [o4] is an important opportunity. thus, they rated the ‘environment friendly tourism development: rural tourism’ [alt 2] strategy, uses strengths to maximize opportunities, after [alt 3] (16.9 %). this preference points out the importance of providing income sources, sensitive to the basin’s natural resources, for the local people. ‘improving water usage in rural areas and agriculture’ [alt 6] strategy, reduces the internal weaknesses to avoid external threats, is rated third by the stakeholders (16.8 %). while [t4] ‘water pollution’ is perceived as the primary threat to the sustainability of the basin, ‘improving water quality-control invasive pollutant’ [alt 5] strategy, reduces the internal weaknesses to avoid external threats, is ranked fifth in stakeholders’ priorities (16.4 %). ‘decreasing the water consumption in urban areas’ [alt 4] strategy, uses internal strengths to minimize threats, is the lowest rated (% 16.1) strategy. despite the fact that stakeholders rated [t3] ‘decline in the amount of lake water’ (6.6 %) more highly, and [t6] ‘destruction of the lake ecosystem’ (6.0 %) factors, they have not supported [alt 4], ‘developed to improve the amount of water in the basin’ enough. priorities of the stakeholders for the alternatives/strategies developed to restore the water amount show that ‘rural areas’ and the ‘agricultural water consumption’ are perceived as the main reasons for the decrease in water amount. 4.4 comparison of the stakeholders’ alternative preferences table 7 and figure 6 show the results of the sensitivity analysis of each watershed management option. this analysis demonstrates how the strategy alternatives were prioritized relative to other alternatives with respect to each objective as well as the overall objective from the perspective of stakeholders. according to the sensitivity analysis, the experts have the same prioritization with the cwm strategy which represents a shared view of all stakeholders. however, local authorities have the same prioritization with the cwm strategy, only regarding their preferences of alt 3 and alt 2 at the first and second row. table 7 global priorities of the strategic alternatives from the perspectives of stakeholders stakeholders strategic alternatives local communities local authorities experts overall stakeholders* [alt 1] 0.1634 0.1660 0.1640 0.1645 [alt 2] 0.1613 0.1686 0.1784 0.1693 [alt 3] 0.1689 0.1713 0.1817 0.1739 [alt 4] 0.1690 0.1654 0.1487 0.1608 [alt 5] 0.1679 0.1651 0.1576 0.1635 [alt 6] 0.1696 0.1635 0.1697 0.1676 * each value is the geometric mean of the row. ijahp article: yavuz, baycan/application of combined analytic hierarchy process (ahp) and swot for integrated watershed management international journal of the analytic hierarchy process issn 1936-6744 16 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.194 figure 6. sensitivity analysis while comparing the global weight dispersions of alternatives in terms of stakeholder groups, we have determined the differences presented below (figure 6): i) ‘agricultural development’ [alt 1] strategy was supported at the highest level (16.60 %) by the local authorities. ii) ‘environment friendly tourism development: rural tourism’ [alt 2] strategy was supported at the highest level (17.84 %) by the experts. iii) ‘collaborative watershed management’ [alt 3] strategy was supported at the highest level (18.17 %) by the experts. iv) ‘decreasing the water consumption in urban area’ [alt 4] strategy was supported at the highest level (16.90 %) by the local communities. v) ‘improving water qualitycontrol invasive pollutant’ [alt 5] strategy was supported at the highest level (16.79 %) by the local communities. vi) ‘improving water usage in rural areas and agriculture’ [alt 6] strategy was supported at the highest level (16.97 %) by the experts. amongst the alternatives (mini-mini, mini-maxi, maxi-maxi and maxi-mini) aiming to provide sustainability of blb, local authorities mostly preferred ‘maxi-maxi’ and ‘mini-mini’ strategies whereas an aggregate of the stakeholders mostly preferred ‘mini-maxi’ strategies (table 8). however, any significant difference in other stakeholder groups’ preferences was not observed. table 8 rankings by different stakeholders regarding alternatives * each value is the geometric mean of the row. ranking local communities local authorities experts overall stakeholders (cwm strategy)* 1 [0.1696] alt 6: mini-mini [0.1713] alt 3: mini-maxi [0.1817] alt 3: mini-maxi [0.1739] alt 3: mini-maxi 2 [0.1690] alt 4: maxi-mini [0.1686] alt 2: maxi-maxi [0.1784] alt 2: maxi-maxi [0.1693] alt 2: maxi-maxi 3 [0.1689] alt 3: mini-maxi [0.1660] alt 1: maxi-maxi [0.1697] alt 6: mini-mini [0.1676] alt 6: mini-mini 4 [0.1679] alt 5: mini-mini [0.1654] alt 4: maxi-mini [0.1640] alt 1: maxi-maxi [0.1645] alt 1: maxi-maxi 5 [0.1634] alt 1: maxi-maxi [0.1651] alt 5: mini-mini [0.1576] alt 5: mini-mini [0.1635] alt 5: mini-mini 6 [0.1613] alt 2: maxi-maxi [0.1635] alt 6: mini-mini [0.1487] alt 4: maxi-mini [0.1608] alt 4: maxi-mini ijahp article: yavuz, baycan/application of combined analytic hierarchy process (ahp) and swot for integrated watershed management international journal of the analytic hierarchy process issn 1936-6744 17 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.194 ‘agricultural development’ [alt 1] is a strategy that was preferred at a medium and low degree by all stakeholders. ‘environment friendly tourism development: rural tourism’ [alt 2] is a strategy that was more highly rated by experts and local authorities while ignored by local communities. ‘collaborative watershed management’ [alt 3] is a strategy that was highly rated by all stakeholder groups. ‘decreasing the water consumption in urban area’ [alt 4] is a strategy that was more highly rated by the local communities, however preferred by the local authorities and experts at a medium and low degree. ‘improving water qualitycontrol invasive pollutant’ [alt 5] is a strategy that was adopted as a medium and low degree preference by all stakeholders. ‘improving water usage in rural areas and agriculture’ [alt 6] is a strategy the local authorities seriously protested. 4.5 performance of the agreed upon watershed management strategy for blb [alt 3] on swot factors it is a commonly held view among stakeholders that ‘collaborative watershed management’ [alt 3] is the most successful at: i) developing the best defence to threats of [t4] ‘water pollution’, [t6] ‘destruction of the lake ecosystem’, [t3] ‘decline in the amount of lake water’, [t5] ‘overhunting’, and [t8] ‘interventions to basin’s water system from outside the basin’, ii) using the advantage of opportunities of [o4] ‘its suitability in terms of tourism development’, [o3] ‘addition of water to the beyşehir lake through the derebucak derivation tunnel’, [o2] ‘construction of new konyaantalya (gembos) motorway’, and [o5] ‘plans and projects to protect and develop the basin’, and iii) reinforcing the weaknesses of [w5] ‘problems in the institutional structure and legal system related to problem solving and management in the basin’, and [w6] ‘inadequacy of financial resources for activities to protect the lake’. however, the [alt 3] approach is not considered the most successful at taking advantage of strengths (figure 7). * negative values are symbolical and should be ignored. figure 7. performance of the agreed upon watershed management strategy [alt 3] by stakeholders’ shared views on swot factors ijahp article: yavuz, baycan/application of combined analytic hierarchy process (ahp) and swot for integrated watershed management international journal of the analytic hierarchy process issn 1936-6744 18 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.194 5. concluding remarks and recommendations for future research this study evaluated the perceptions of three stakeholder groups—local people, local authorities, and experts—towards the successful watershed management in blb, and explored how professionals/experts and local communities can combine their abilities to resolve the basin’s problems, and how they can work in collaboration to achieve the objectives of joint management. as a methodology, a combination of ahp and swot analysis was used i) to describe the most appropriate watershed management strategies from the perspectives of different stakeholders, and ii) to determine the cwm strategy as an agreed upon strategy that met expectations of all stakeholders and considered their benefits equally. the results of the study show that amongst a set of proposed strategy alternatives ‘collaborative watershed management’ [alt 3] was assumed as the optimal approach to solve the blb’s problems by all stakeholders. their joint strategy preferences show that ‘cooperation between community and public institutions’ is the key to success in watershed management. this study presents a ‘knowledge-based, stakeholder-oriented and comprehensive decision support system’ which provides assistance for water resource planning. the applied ahpswot approach yields a better understanding of participatory planning and more effective decision-making in iwm studies. ahp-swot i) enables the development of guidelines for effective collaboration between stakeholders, thus reduces conflicts, ii) provides a simple, transparent and rapid decision-making process, iii) provides some insights on what can be done to enhance the likelihood of watershed management success, and iv) provides a mechanism to determine an agreed upon watershed management strategy (in this study cwm). such a transparent decision-making process leads to more sustainable watershed planning and management decisions, encourages increasing community capacity to address the important issues in a constructive way, and therefore greatly increases the acceptability of the policy decisions by the public. nevertheless, this study is limited with determining an agreed upon watershed management strategy. due to the independent and hierarchical structure of ahp, watershed management strategies are considered to be independent and the connections among the strategies as well as the strengths, weaknesses, opportunities and threats cannot be evaluated. in order to highlight the interaction and dependence among the strategies, the combined use of anp (analytic network process) and swot can be applied in future studies. acknowledgements the authors are grateful to the blb’s stakeholders for their participation to the questionnaire. the earlier version of this manuscript that is titled ‘stakeholder-based decision making in integrated watershed management’ has been presented at the 51st european congress of the regional science association international: new challenges for european regions and urban areas in a globalised world (30th august 3rd september 2011, barcelona, spain) by the authors. ijahp article: yavuz, baycan/application of combined analytic hierarchy process (ahp) and swot for integrated watershed management international journal of the analytic hierarchy process issn 1936-6744 19 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.194 appendices appendix–a: survey sheet 1. please state the most dominant or the least favorable factor (in the case of strength) in order to perform environmental and socio-cultural sustainability of blb, and compare two factors’ intensity of importance. strengths (s) 1= equal importance; 3= moderate importance; 5= strong importance; 7= very strong importance; 9= extreme importance strengths (s) s1 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 s2 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 s3 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 s4 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 s5 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 s6 2. please state the most dominant or the least favorable factor (in the case of weakness) in order to perform environmental and socio-cultural sustainability of blb, and compare two factors’ intensity of importance. weaknesses (w) 1= equal importance; 3= moderate importance; 5= strong importance; 7= very strong importance; 9= extreme importance weaknesses (w) w1 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 w2 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 w3 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 w4 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 w5 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 w6 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 w7 3. please state the most dominant or the least favorable factor (in the case of opportunity) in order to perform environmental and socio-cultural sustainability of blb, and compare two factors’ intensity of importance. opportunities (o) 1= equal importance; 3= moderate importance; 5= strong importance; 7= very strong importance; 9= extreme importance opportunities (o) o1 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 o2 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 o3 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 o4 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 o5 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 o6 4. please state the most dominant or the least favorable factor (in the case of threats) in order to perform environmental and socio-cultural sustainability of blb, and compare two factors’ intensity of importance. threats (t) 1= equal importance; 3= moderate importance; 5= strong importance; 7= very strong importance; 9= extreme importance threats (t) t1 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 t2 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 t3 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 t4 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 t5 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 t6 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 t7 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 t8 5. please evaluate the importance of strengths, weaknesses, opportunities and threats of the basin to reach the goal of “to develop the best watershed management strategy enables blb’s environmental and socio-economic sustainability together”. ijahp article: yavuz, baycan/application of combined analytic hierarchy process (ahp) and swot for integrated watershed management international journal of the analytic hierarchy process issn 1936-6744 20 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.194 please evaluate the performances of the strategy alternatives (alt 1, alt 2, alt 3, alt 4, alt 5, alt 6) to take the advantage of the strengths, to reinforce the weaknesses, to use the advantage of opportunities or to develop the best defense to the threats via the following scale: [ degree of relationship: 0: no; 1: very weak; 3: weak; 5: medium; 7: strong; 9: very strong ] 6. how much successful is the “agricultural development” strategy (alt 1); 6.1. …to take the advantage of the strengths? 6.2. …to reinforce the weaknesses? 6.3. …to use the advantage of opportunities? 6.4. …to develop the best defense to the threats? alt 1−s1 alt 1−w1 alt 1−o1 alt 1−t1 alt 1−s2 alt 1−w2 alt 1−o2 alt 1−t2 alt 1−s3 alt 1−w3 alt 1−o3 alt 1−t3 alt 1−s4 alt 1−w4 alt 1−o4 alt 1−t4 alt 1−s5 alt 1−w5 alt 1−o5 alt 1−t5 alt 1−s6 alt 1−w6 alt 1−o6 alt 1−t6 alt 1−w7 alt 1−t7 alt 1−t8 7. how much successful is the “environment friendly tourism development” strategy (alt 2); 7.1. …to take the advantage of the strengths? 7.2. …to reinforce the weaknesses? 7.3. …to use the advantage of opportunities? 7.4. …to develop the best defense to the threats? alt 2−s1 alt 2−w1 alt 2−o1 alt 2−t1 alt 2−s2 alt 2−w2 alt 2−o2 alt 2−t2 alt 2−s3 alt 2−w3 alt 2−o3 alt 2−t3 alt 2−s4 alt 2−w4 alt 2−o4 alt 2−t4 alt 2−s5 alt 2−w5 alt 2−o5 alt 2−t5 alt 2−s6 alt 2−w6 alt 2−o6 alt 2−t6 alt 2−w7 alt 2−t7 alt 2−t8 8. how much successful is the “collaborative watershed management” strategy (alt 3); 8.1. …to take the advantage of the strengths? 8.2. …to reinforce the weaknesses? 8.3. …to use the advantage of opportunities? 8.4. …to develop the best defense to the threats? alt 3−s1 alt 3−w1 alt 3−o1 alt 3−t1 alt 3−s2 alt 3−w2 alt 3−o2 alt 3−t2 alt 3−s3 alt 3−w3 alt 3−o3 alt 3−t3 alt 3−s4 alt 3−w4 alt 3−o4 alt 3−t4 alt 3−s5 alt 3−w5 alt 3−o5 alt 3−t5 alt 3−s6 alt 3−w6 alt 3−o6 alt 3−t6 alt 3−w7 alt 3−t7 alt 3−t8 9. how much successful is the “decreasing the water consumption in urban area” strategy (alt 4); 9.1. …to take the advantage of the 9.2. …to reinforce the weaknesses? 9.3. …to use the advantage of opportunities? 9.4. …to develop the best defense to the threats? 1= equal importance; 3= moderate importance; 5= strong importance; 7= very strong importance; 9= extreme importance s to take the advantage of the strengths 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 to reinforce the weaknesses w 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 to use the advantage of opportunities o 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 to develop the best defense to the threats t ijahp article: yavuz, baycan/application of combined analytic hierarchy process (ahp) and swot for integrated watershed management international journal of the analytic hierarchy process issn 1936-6744 21 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.194 strengths? alt 4−s1 alt 4−w1 alt 4−o1 alt 4−t1 alt 4−s2 alt 4−w2 alt 4−o2 alt 4−t2 alt 4−s3 alt 4−w3 alt 4−o3 alt 4−t3 alt 4−s4 alt 4−w4 alt 4−o4 alt 4−t4 alt 4−s5 alt 4−w5 alt 4−o5 alt 4−t5 alt 4−s6 alt 4−w6 alt 4−o6 alt 4−t6 alt 4−w7 alt 4−t7 alt 4−t8 10. how much successful is the “improving water qualitycontrol invasive pollutant” strategy (alt 5); 10.1. …to take the advantage of the strengths? 10.2. …to reinforce the weaknesses? 10.3. …to use the advantage of opportunities? 10.4. …to develop the best defense to the threats? alt 5−s1 alt 5−w1 alt 5−o1 alt 5−t1 alt 5−s2 alt 5−w2 alt 5−o2 alt 5−t2 alt 5−s3 alt 5−w3 alt 5−o3 alt 5−t3 alt 5−s4 alt 5−w4 alt 5−o4 alt 5−t4 alt 5−s5 alt 5−w5 alt 5−o5 alt 5−t5 alt 5−s6 alt 5−w6 alt 5−o6 alt 5−t6 alt 5−w7 alt 5−t7 alt 5−t8 11. how much successful is the “improving water usage in rural areas and agriculture” strategy (alt 6); 11.1. …to take the advantage of the strengths? 11.2. …to reinforce the weaknesses? 11.3…to use the advantage of opportunities? 11.4. …to develop the best defense to the threats? alt 6−s1 alt 6−w1 alt 6−o1 alt 6−t1 alt 6−s2 alt 6−w2 alt 6−o2 alt 6−t2 alt 6−s3 alt 6−w3 alt 6−o3 alt 6−t3 alt 6−s4 alt 6−w4 alt 6−o4 alt 6−t4 alt 6−s5 alt 6−w5 alt 6−o5 alt 6−t5 alt 6−s6 alt 6−w6 alt 6−o6 alt 6−t6 alt 6−w7 alt 6−t7 alt 6−t8 ijahp article: yavuz, baycan/application of combined analytic hierarchy process (ahp) and swot for integrated watershed management international journal of the analytic hierarchy process issn 1936-6744 22 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.194 appendix–b: ahp matrices of local communities swot group weights matrix of local communities strengths weaknesses opportunities threats weights strengths 1.00 0.36 0.47 0.26 0.102 weaknesses 2.74 1.00 1.30 0.70 0.281 opportunities 2.11 0.77 1.00 0.54 0.216 threats 3.91 1.42 1.86 1.00 0.401 cr = 0.0016 consistent (lambda max=3.99565, ri =0.9, ci=0.00145) strengths matrix of local communities s1 s2 s3 s4 s5 s6 weights s1 1.00 0.27 0.30 0.24 0.49 0.36 0.006 s2 3.72 1.00 1.12 0.88 1.84 1.35 0.022 s3 3.33 0.90 1.00 0.78 1.65 1.21 0.020 s4 4.25 1.14 1.28 1.00 2.10 1.54 0.026 s5 2.02 0.54 0.61 0.48 1.00 0.73 0.012 s6 2.75 0.74 0.83 0.65 1.36 1.00 0.017 cr= 0.0008 consistent (lambda max=5.99505, ri =1.24, ci =0.00099) weaknesses matrix of local communities w1 w2 w3 w4 w5 w6 w7 weights w1 1.00 0.45 0.32 0.30 0.33 0.31 0.37 0.015 w2 2.20 1.00 0.70 0.66 0.73 0.69 0.82 0.033 w3 3.14 1.43 1.00 0.95 1.05 0.99 1.18 0.048 w4 3.32 1.51 1.05 1.00 1.10 1.04 1.24 0.050 w5 3.01 1.37 0.96 0.91 1.00 0.95 1.13 0.046 w6 3.18 1.44 1.01 0.96 1.06 1.00 1.19 0.048 w7 2.67 1.21 0.85 0.81 0.89 0.84 1.00 0.041 cr = 0.0012 consistent (lambda max=6.990747, ri=1.32, ci=0.00154) opportunities matrix of local communities o1 o2 o3 o4 o5 o6 weights o1 1.00 0.25 0.19 0.25 0.25 0.30 0.010 o2 4.08 1.00 0.77 1.01 1.00 1.24 0.040 o3 5.29 1.30 1.00 1.31 1.30 1.61 0.052 o4 4.04 0.99 0.76 1.00 0.99 1.23 0.040 o5 4.07 1.00 0.77 1.01 1.00 1.24 0.040 o6 3.28 0.80 0.62 0.81 0.81 1.00 0.033 cr = 0.0022 consistent (lambda max=6.01386, ri=1.24, ci=0.002772) threats matrix of local communities t1 t2 t3 t4 t5 t6 t7 t8 weights t1 1.00 0.25 0.21 0.21 0.32 0.23 0.49 0.25 0.014 t2 4.00 1.00 0.86 0.86 1.27 0.92 1.98 1.00 0.057 t3 4.67 1.17 1.00 1.00 1.48 1.07 2.31 1.17 0.067 t4 4.65 1.16 1.00 1.00 1.47 1.07 2.30 1.16 0.067 t5 3.16 0.79 0.68 0.68 1.00 0.73 1.56 0.79 0.045 t6 4.36 1.09 0.93 0.94 1.38 1.00 2.15 1.09 0.063 t7 2.02 0.51 0.43 0.43 0.64 0.46 1.00 0.50 0.029 t8 4.00 1.00 0.86 0.86 1.27 0.92 1.98 1.00 0.058 cr = 0.0021 consistent (lambda max=7.97885, ri=1.41, ci=0.00302) ijahp article: yavuz, baycan/application of combined analytic hierarchy process (ahp) and swot for integrated watershed management international journal of the analytic hierarchy process issn 1936-6744 23 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.194 performances of the strategy alternatives on swot factors from the view point of local communities strategy evaluation matrix of local communities factors [alt 1] [alt 2] [alt 3] [alt 4] [alt 5] [alt 6] factors [alt1] [alt2] [alt3] [alt4] [alt5] [alt6] s1 6.99 6.91 7.22 7.33 7.21 7.25 s1 0.001 0.001 0.001 0.001 0.001 0.001 s2 6.99 6.91 7.22 7.33 7.21 7.25 s2 0.004 0.004 0.004 0.004 0.004 0.004 s3 6.99 6.91 7.22 7.33 7.21 7.25 s3 0.003 0.003 0.003 0.003 0.003 0.003 s4 6.99 6.91 7.22 7.33 7.21 7.25 s4 0.004 0.004 0.004 0.004 0.004 0.004 s5 6.99 6.91 7.22 7.33 7.21 7.25 s5 0.002 0.002 0.002 0.002 0.002 0.002 s6 6.99 6.91 7.22 7.33 7.21 7.25 s6 0.003 0.003 0.003 0.003 0.003 0.003 w1 6.83 6.73 7.10 7.11 6.92 7.12 w1 0.002 0.002 0.003 0.003 0.003 0.003 w2 6.83 6.73 7.10 7.11 6.92 7.12 w2 0.005 0.005 0.006 0.006 0.006 0.006 w3 6.83 6.73 7.10 7.11 6.92 7.12 w3 0.008 0.008 0.008 0.008 0.008 0.008 w4 6.83 6.73 7.10 7.11 6.92 7.12 w4 0.008 0.008 0.009 0.009 0.008 0.009 w5 6.83 6.73 7.10 7.11 6.92 7.12 w5 0.007 0.007 0.008 0.008 0.008 0.008 w6 6.83 6.73 7.10 7.11 6.92 7.12 w6 0.008 0.008 0.008 0.008 0.008 0.008 w7 6.83 6.73 7.10 7.11 6.92 7.12 w7 0.007 0.007 0.007 0.007 0.007 0.007 o1 7.19 6.99 7.22 7.27 7.40 7.24 o1 0.002 0.002 0.002 0.002 0.002 0.002 o2 7.19 6.99 7.22 7.27 7.40 7.24 o2 0.007 0.007 0.007 0.007 0.007 0.007 o3 7.19 6.99 7.22 7.27 7.40 7.24 o3 0.009 0.008 0.009 0.009 0.009 0.009 o4 7.19 6.99 7.22 7.27 7.40 7.24 o4 0.007 0.006 0.007 0.007 0.007 0.007 o5 7.19 6.99 7.22 7.27 7.40 7.24 o5 0.007 0.007 0.007 0.007 0.007 0.007 o6 7.19 6.99 7.22 7.27 7.40 7.24 o6 0.005 0.005 0.005 0.005 0.006 0.005 t1 6.99 6.97 7.31 7.26 7.24 7.36 t1 0.002 0.002 0.002 0.002 0.002 0.002 t2 6.99 6.97 7.31 7.26 7.24 7.36 t2 0.009 0.009 0.010 0.010 0.010 0.010 t3 6.99 6.97 7.31 7.26 7.24 7.36 t3 0.011 0.011 0.011 0.011 0.011 0.011 t4 6.99 6.97 7.31 7.26 7.24 7.36 t4 0.011 0.011 0.011 0.011 0.011 0.011 t5 6.99 6.97 7.31 7.26 7.24 7.36 t5 0.007 0.007 0.008 0.008 0.008 0.008 t6 6.99 6.97 7.31 7.26 7.24 7.36 t6 0.010 0.010 0.011 0.011 0.011 0.011 t7 6.99 6.97 7.31 7.26 7.24 7.36 t7 0.005 0.005 0.005 0.005 0.005 0.005 t8 6.99 6.97 7.31 7.26 7.24 7.36 t8 0.009 0.009 0.010 0.010 0.010 0.010 total weight 0.1634 0.1613 0.1689 0.1690 0.1679 0.1696 ranking 5 6 3 2 4 1 ijahp article: yavuz, baycan/application of combined analytic hierarchy process (ahp) and swot for integrated watershed management international journal of the analytic hierarchy process issn 1936-6744 24 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.194 appendix–c: ahp matrices of local authorities swot group weights matrix of local authorities strengths weaknesses opportunities threats weights strengths 1.00 0.53 0.58 0.25 0.117 weaknesses 1.89 1.00 1.09 0.48 0.221 opportunities 1.73 0.92 1.00 0.44 0.202 threats 3.93 2.08 2.27 1.00 0.460 cr= 0.0009 consistent (lambda max=3.99756, ri=0.9, ci=0.00081) strengths matrix of local authorities s1 s2 s3 s4 s5 s6 weights s1 1.00 0.50 0.30 0.37 1.09 0.55 0.010 s2 2.00 1.00 0.59 0.74 2.18 1.10 0.020 s3 3.39 1.69 1.00 1.25 3.70 1.86 0.033 s4 2.72 1.36 0.80 1.00 2.97 1.49 0.027 s5 0.92 0.46 0.27 0.34 1.00 0.50 0.009 s6 1.82 0.91 0.54 0.67 1.99 1.00 0.018 cr = 0.0024 consistent (lambda max=6.01504, ri=1.24, ci=0.0030) weaknesses matrix of local authorities w1 w2 w3 w4 w5 w6 w7 weights w1 1.00 0.82 0.59 0.48 0.28 0.33 0.40 0.015 w2 1.23 1.00 0.72 0.59 0.34 0.40 0.49 0.018 w3 1.70 1.39 1.00 0.83 0.47 0.56 0.69 0.025 w4 2.06 1.69 1.21 1.00 0.57 0.68 0.83 0.030 w5 3.61 2.95 2.12 1.75 1.00 1.19 1.46 0.053 w6 3.04 2.48 1.78 1.47 0.84 1.00 1.23 0.044 w7 2.48 2.02 1.45 1.20 0.69 0.82 1.00 0.036 cr= 0.0024 consistent (lambda max=7.01918, ri=1.32, ci=0. 0032) opportunities matrix of local authorities o1 o2 o3 o4 o5 o6 weights o1 1.00 0.43 0.28 0.41 0.25 0.40 0.013 o2 2.32 1.00 0.65 0.96 0.58 0.92 0.030 o3 3.59 1.54 1.00 1.48 0.90 1.43 0.046 o4 2.43 1.05 0.68 1.00 0.61 0.97 0.031 o5 3.98 1.71 1.11 1.64 1.00 1.58 0.051 o6 2.51 1.08 0.70 1.04 0.63 1.00 0.032 cr= 0.0015 consistent (lambda max=6.00941, ri=1.24, ci=0.1518) threats matrix of local authorities t1 t2 t3 t4 t5 t6 t7 t8 weights t1 1.00 0.53 0.24 0.19 0.54 0.22 1.81 0.39 0.021 t2 1.89 1.00 0.46 0.37 1.03 0.41 3.41 0.74 0.040 t3 4.12 2.18 1.00 0.80 2.24 0.90 7.44 1.62 0.087 t4 5.15 2.73 1.25 1.00 2.80 1.12 9.29 2.02 0.109 t5 1.84 0.98 0.45 0.36 1.00 0.40 3.32 0.72 0.039 t6 4.59 2.43 1.11 0.89 2.49 1.00 8.28 1.80 0.097 t7 0.55 0.29 0.13 0.11 0.30 0.12 1.00 0.22 0.012 t8 2.55 1.35 0.62 0.50 1.39 0.56 4.60 1.00 0.054 cr= 0.0019 consistent (lambda max=8.01884, ri=1.41, ci=0.002692) ijahp article: yavuz, baycan/application of combined analytic hierarchy process (ahp) and swot for integrated watershed management international journal of the analytic hierarchy process issn 1936-6744 25 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.194 performances of the strategy alternatives on swot factors from the view point of local authorities strategy evaluation matrix of local authorities swot factors [alt 1] [alt 2] [alt 3] [alt 4] [alt 5] [alt 6] swot factors [alt1] [alt2] [alt3] [alt4] [alt5] [alt6] s1 6.43 6.71 7.29 7.29 7.14 6.96 s1 0.002 0.002 0.002 0.002 0.002 0.002 s2 6.33 6.73 7.00 7.27 7.00 6.73 s2 0.003 0.003 0.003 0.003 0.003 0.003 s3 6.60 8.07 7.40 7.00 7.40 6.53 s3 0.005 0.006 0.006 0.005 0.006 0.005 s4 7.53 6.60 7.00 7.40 7.00 7.00 s4 0.005 0.004 0.004 0.005 0.004 0.004 s5 7.40 7.67 7.53 6.47 6.87 6.60 s5 0.002 0.002 0.002 0.001 0.001 0.001 s6 6.73 7.00 7.40 7.00 7.13 6.73 s6 0.003 0.003 0.003 0.003 0.003 0.003 w1 6.57 6.86 7.50 7.07 7.46 7.32 w1 0.002 0.002 0.003 0.002 0.003 0.002 w2 6.60 7.00 6.73 6.47 6.33 6.47 w2 0.003 0.003 0.003 0.003 0.003 0.003 w3 7.40 7.93 7.27 7.67 7.40 7.33 w3 0.004 0.004 0.004 0.004 0.004 0.004 w4 6.73 6.73 7.27 7.00 7.00 7.53 w4 0.005 0.005 0.005 0.005 0.005 0.005 w5 6.20 6.87 7.00 6.20 6.33 7.00 w5 0.008 0.009 0.009 0.008 0.008 0.009 w6 6.73 7.40 7.53 6.73 6.73 7.40 w6 0.007 0.008 0.008 0.007 0.007 0.008 w7 5.93 6.87 6.60 6.87 7.13 7.20 w7 0.005 0.006 0.006 0.006 0.006 0.006 o1 7.00 7.07 7.43 7.25 7.39 6.93 o1 0.002 0.002 0.002 0.002 0.002 0.002 o2 7.13 7.00 6.87 5.87 7.00 6.07 o2 0.005 0.005 0.005 0.004 0.005 0.005 o3 7.13 7.40 7.53 7.27 7.27 6.87 o3 0.008 0.008 0.008 0.008 0.008 0.007 o4 7.40 7.27 7.53 7.40 7.53 6.73 o4 0.005 0.005 0.005 0.005 0.005 0.005 o5 7.53 7.53 7.40 6.87 7.27 7.53 o5 0.009 0.009 0.009 0.008 0.008 0.009 o6 6.93 6.40 6.20 6.53 6.53 6.27 o6 0.006 0.005 0.005 0.005 0.005 0.005 t1 6.36 6.71 7.07 6.75 7.14 7.04 t1 0.003 0.003 0.004 0.003 0.004 0.004 t2 5.93 5.40 5.80 5.93 6.07 5.80 t2 0.007 0.006 0.007 0.007 0.007 0.007 t3 7.00 6.73 7.27 7.27 6.67 7.13 t3 0.015 0.014 0.015 0.015 0.014 0.015 t4 7.27 7.27 7.53 6.80 7.00 6.73 t4 0.019 0.019 0.019 0.017 0.018 0.017 t5 6.73 7.13 6.73 6.33 6.33 6.40 t5 0.007 0.007 0.007 0.006 0.006 0.006 t6 7.13 6.73 7.00 7.13 6.73 6.00 t6 0.017 0.016 0.017 0.017 0.016 0.014 t7 5.47 6.07 5.93 5.53 5.13 6.07 t7 0.002 0.002 0.002 0.002 0.002 0.002 t8 6.87 7.07 6.87 6.86 6.36 6.86 t8 0.009 0.009 0.009 0.009 0.008 0.009 total weight 0.1660 0.1686 0.1713 0.1654 0.1651 0.1635 ranking 3 2 1 4 5 6 ijahp article: yavuz, baycan/application of combined analytic hierarchy process (ahp) and swot for integrated watershed management international journal of the analytic hierarchy process issn 1936-6744 26 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.194 appendix–d: ahp matrices of experts swot group weights matrix of experts strengths weaknesses opportunities threats weights strengths 1.00 1.33 0.60 0.73 0.209 weaknesses 0.75 1.00 0.45 0.55 0.157 opportunities 1.68 2.23 1.00 1.23 0.350 threats 1.36 1.81 0.81 1.00 0.284 cr = 0.0004 consistent (lambda max=3.99904, ri=0.9, ci=0.00032) strengths matrix of experts s1 s2 s3 s4 s5 s6 weights s1 1.00 0.52 0.72 0.44 1.02 0.51 0.022 s2 1.92 1.00 1.39 0.85 1.95 0.97 0.042 s3 1.38 0.72 1.00 0.61 1.40 0.70 0.030 s4 2.25 1.17 1.63 1.00 2.29 1.14 0.049 s5 0.98 0.51 0.71 0.44 1.00 0.50 0.022 s6 1.97 1.03 1.43 0.88 2.00 1.00 0.043 cr= 0.0014 consistent (lambda max=6.00894, ri=1.24, ci=0.00178) weaknesses matrix of experts w1 w2 w3 w4 w5 w6 w7 weights w1 1.00 0.74 0.37 0.53 0.36 0.57 1.00 0.013 w2 1.35 1.00 0.50 0.71 0.49 0.78 1.35 0.017 w3 2.72 2.01 1.00 1.43 0.99 1.56 2.71 0.034 w4 1.90 1.41 0.70 1.00 0.69 1.09 1.89 0.024 w5 2.74 2.03 1.01 1.44 1.00 1.58 2.73 0.035 w6 1.74 1.29 0.64 0.92 0.63 1.00 1.74 0.022 w7 1.00 0.74 0.37 0.53 0.37 0.58 1.00 0.013 cr= 0.0029 consistent (lambda max=7.02291, ri=1.32, ci=0.003818) opportunities matrix of experts o1 o2 o3 o4 o5 o6 weights o1 1.00 1.08 1.45 0.55 0.47 0.96 0.046 o2 0.93 1.00 1.34 0.51 0.43 0.89 0.043 o3 0.69 0.75 1.00 0.38 0.32 0.66 0.032 o4 1.82 1.97 2.64 1.00 0.85 1.75 0.084 o5 2.13 2.31 3.09 1.17 1.00 2.05 0.098 o6 1.04 1.12 1.51 0.57 0.49 1.00 0.048 cr= 0.0008 consistent (lambda max=6.0051, ri=1.24, ci=0.00102) threats matrix of experts t1 t2 t3 t4 t5 t6 t7 t8 weights t1 1.00 0.49 0.40 0.42 0.39 0.54 0.81 0.83 0.019 t2 2.03 1.00 0.82 0.86 0.80 1.10 1.65 1.68 0.039 t3 2.48 1.22 1.00 1.05 0.98 1.34 2.01 2.05 0.048 t4 2.36 1.16 0.95 1.00 0.93 1.28 1.92 1.95 0.046 t5 2.53 1.25 1.02 1.07 1.00 1.37 2.06 2.10 0.049 t6 1.84 0.91 0.74 0.78 0.73 1.00 1.50 1.52 0.036 t7 1.23 0.61 0.50 0.52 0.49 0.67 1.00 1.02 0.024 t8 1.21 0.60 0.49 0.51 0.48 0.66 0.98 1.00 0.023 cr= 0.0016 consistent (lambda max=7.8461, ri=1.41, ci=0.00219) ijahp article: yavuz, baycan/application of combined analytic hierarchy process (ahp) and swot for integrated watershed management international journal of the analytic hierarchy process issn 1936-6744 27 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.194 performances of the strategy alternatives on swot factors from the view point of experts strategy evaluation matrix of local experts swot factors [alt 1] [alt 2] [alt 3] [alt 4] [alt 5] [alt 6] swot factors [alt 1] [alt 2] [alt 3] [alt 4] [alt 5] [alt 6] s1 4.86 5.82 4.67 3.32 3.45 3.00 s1 0.004 0.005 0.004 0.003 0.003 0.003 s2 5.00 4.00 4.90 5.41 5.14 5.41 s2 0.007 0.006 0.007 0.008 0.007 0.008 s3 5.45 6.73 5.71 4.73 5.73 5.14 s3 0.005 0.006 0.005 0.004 0.005 0.005 s4 6.59 5.00 5.57 4.55 4.36 5.91 s4 0.010 0.008 0.009 0.007 0.007 0.009 s5 3.41 6.59 5.86 4.36 4.36 4.36 s5 0.003 0.005 0.004 0.003 0.003 0.003 s6 4.48 6.64 5.76 4.05 4.73 5.50 s6 0.006 0.009 0.008 0.006 0.007 0.008 w1 5.00 2.77 6.00 4.90 4.50 6.45 w1 0.002 0.001 0.003 0.002 0.002 0.003 w2 4.14 5.86 6.09 3.59 3.77 4.50 w2 0.003 0.004 0.004 0.002 0.002 0.003 w3 5.05 5.36 6.18 5.45 6.18 5.18 w3 0.005 0.005 0.006 0.006 0.006 0.005 w4 5.29 4.59 4.86 3.27 3.27 4.00 w4 0.005 0.004 0.005 0.003 0.003 0.004 w5 3.77 3.67 5.91 3.36 3.45 4.73 w5 0.005 0.005 0.008 0.005 0.005 0.007 w6 4.09 5.36 5.45 4.68 5.05 5.36 w6 0.003 0.004 0.004 0.003 0.004 0.004 w7 2.82 5.64 4.45 4.41 3.45 3.59 w7 0.001 0.003 0.002 0.002 0.002 0.002 o1 4.62 6.09 4.95 3.09 3.64 4.05 o1 0.008 0.011 0.009 0.005 0.006 0.007 o2 4.76 5.86 5.14 2.50 3.50 3.36 o2 0.008 0.010 0.009 0.004 0.006 0.006 o3 5.52 4.18 5.18 4.76 4.82 5.55 o3 0.006 0.004 0.005 0.005 0.005 0.006 o4 4.71 7.09 5.59 3.91 5.91 5.00 o4 0.012 0.018 0.015 0.010 0.015 0.013 o5 5.05 5.45 5.82 5.23 5.64 5.59 o5 0.015 0.016 0.017 0.016 0.017 0.017 o6 5.14 5.24 4.82 4.36 4.77 5.00 o6 0.008 0.009 0.008 0.007 0.008 0.008 t1 4.81 4.77 4.00 2.77 3.09 4.18 t1 0.004 0.004 0.003 0.002 0.003 0.003 t2 2.62 2.27 2.68 2.95 2.77 3.00 t2 0.006 0.005 0.006 0.007 0.007 0.007 t3 6.05 4.36 5.50 5.82 5.00 6.45 t3 0.009 0.006 0.008 0.008 0.007 0.009 t4 4.76 4.77 5.59 5.23 6.45 5.82 t4 0.007 0.007 0.008 0.007 0.009 0.008 t5 4.14 4.55 5.05 3.55 3.45 4.64 t5 0.008 0.009 0.010 0.007 0.007 0.009 t6 4.10 4.64 5.14 5.55 5.00 5.82 t6 0.005 0.005 0.006 0.007 0.006 0.007 t7 2.10 2.43 2.77 2.55 1.59 2.14 t7 0.004 0.004 0.005 0.004 0.003 0.004 t8 6.00 5.40 5.36 5.50 4.64 4.90 t8 0.004 0.004 0.004 0.004 0.003 0.004 total weight 0.1640 0.1784 0.1817 0.1487 0.1576 0.1697 ranking 4 2 1 6 5 3 ijahp article: yavuz, baycan/application of combined analytic hierarchy process (ahp) and swot for integrated watershed management international journal of the analytic hierarchy process issn 1936-6744 28 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.194 appendix–e: swot priorities with respect to different stakeholder groups and cwm strategy local priorities global priorities local communities local authorities experts cwm strategy local communities local authorities experts cwm strategy s1 0.006 0.010 0.022 0.011 0.001 0.001 0.005 0.002 s2 0.022 0.020 0.042 0.026 0.002 0.002 0.009 0.003 s3 0.020 0.033 0.030 0.027 0.002 0.004 0.006 0.004 s4 0.026 0.027 0.049 0.033 0.003 0.003 0.010 0.004 s5 0.012 0.009 0.022 0.013 0.001 0.001 0.005 0.002 s6 0.017 0.018 0.043 0.024 0.002 0.002 0.009 0.003 w1 0.015 0.015 0.013 0.014 0.004 0.003 0.002 0.003 w2 0.033 0.018 0.017 0.022 0.009 0.004 0.003 0.005 w3 0.048 0.025 0.034 0.034 0.013 0.006 0.005 0.007 w4 0.050 0.030 0.024 0.033 0.014 0.007 0.004 0.007 w5 0.046 0.053 0.035 0.044 0.013 0.012 0.005 0.009 w6 0.048 0.044 0.022 0.036 0.014 0.010 0.003 0.007 w7 0.041 0.036 0.013 0.027 0.011 0.008 0.002 0.006 o1 0.010 0.013 0.046 0.018 0.002 0.003 0.016 0.005 o2 0.040 0.030 0.043 0.037 0.009 0.006 0.015 0.009 o3 0.052 0.046 0.032 0.042 0.011 0.009 0.011 0.010 o4 0.040 0.031 0.084 0.047 0.009 0.006 0.029 0.012 o5 0.040 0.051 0.098 0.058 0.009 0.010 0.034 0.015 o6 0.033 0.032 0.048 0.037 0.007 0.006 0.017 0.009 t1 0.014 0.021 0.019 0.018 0.006 0.010 0.006 0.007 t2 0.057 0.040 0.039 0.045 0.023 0.018 0.011 0.017 t3 0.067 0.087 0.048 0.065 0.027 0.040 0.014 0.025 t4 0.067 0.109 0.046 0.070 0.027 0.050 0.013 0.026 t5 0.045 0.039 0.049 0.044 0.018 0.018 0.014 0.017 t6 0.063 0.097 0.036 0.060 0.025 0.045 0.010 0.022 t7 0.029 0.012 0.024 0.020 0.012 0.005 0.007 0.007 t8 0.058 0.054 0.023 0.042 0.023 0.025 0.007 0.016 * the swot priorities for the ‘cwm strategy’ are derived from the geometric means of three stakeholder group priorities. ijahp article: yavuz, baycan/application of combined analytic hierarchy process (ahp) and swot for integrated watershed management international journal of the analytic hierarchy process issn 1936-6744 29 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.194 appendix–f: strategy evaluation matrix for cwm strategy so st [alt 1] [alt 2] [alt 3] [alt 4] [alt 5] [alt 6] s1 6 0,011 0,002 s2 3 0,026 0,003 s3 2 0,027 0,004 s4 1 0,033 0,004 s5 5 0,013 0,002 s6 4 0,024 0,003 w1 7 0,014 0,003 w2 6 0,022 0,005 w3 3 0,034 0,007 w4 4 0,033 0,007 w5 1 0,044 0,009 w6 2 0,036 0,007 w7 5 0,027 0,006 o1 6 0,018 0,005 o2 4 0,037 0,009 o3 3 0,042 0,010 o4 2 0,047 0,012 o5 1 0,058 0,015 o6 5 0,037 0,009 t1 8 0,018 0,007 t2 4 0,045 0,017 t3 2 0,065 0,025 t4 1 0,070 0,026 t5 5 0,044 0,017 t6 3 0,060 0,022 t7 7 0,020 0,007 t8 6 0,042 0,016 0,165 0,169 0,174 0,161 0,164 0,168 16,5 16,9 17,4 16,1 16,4 16,8 4 2 1 6 5 3 s t r a t e g y a l t e r n a t i v e s wo wtswot weight swot factors local weight global weight strengths 0,136 weaknesses 0,214 opportunities 0,248 threats 0,374 t o t a l w e i g h t : n o r m a l i z e d r e l a t i v e i m p o r t a n c e (% ) : r a n k i n g : degree of relationship graphic symbol number no 0 very weak 1 weak 3 medium 5 strong 7 very strong 9 ijahp article: yavuz, baycan/application of combined analytic hierarchy process (ahp) and swot for integrated watershed management international journal of the analytic hierarchy process issn 1936-6744 30 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.194 appendix–g: global weights for cwm strategy swot factors [alt 1] [alt 2] [alt 3] [alt 4] [alt 5] [alt 6] s1 0.002 0.003 0.002 0.002 0.002 0.002 s2 0.005 0.004 0.005 0.005 0.005 0.005 s3 0.004 0.005 0.005 0.004 0.005 0.004 s4 0.006 0.005 0.006 0.005 0.005 0.006 s5 0.002 0.003 0.003 0.002 0.002 0.002 s6 0.004 0.005 0.005 0.004 0.004 0.004 w1 0.002 0.002 0.003 0.002 0.002 0.003 w2 0.004 0.004 0.004 0.004 0.004 0.004 w3 0.006 0.006 0.006 0.006 0.006 0.006 w4 0.006 0.006 0.006 0.006 0.005 0.006 w5 0.007 0.007 0.008 0.007 0.007 0.008 w6 0.006 0.006 0.007 0.006 0.006 0.007 w7 0.004 0.005 0.005 0.005 0.005 0.005 o1 0.004 0.005 0.004 0.003 0.003 0.004 o2 0.007 0.007 0.007 0.005 0.006 0.006 o3 0.007 0.007 0.007 0.007 0.007 0.007 o4 0.008 0.010 0.009 0.007 0.009 0.008 o5 0.010 0.011 0.011 0.010 0.011 0.011 o6 0.007 0.006 0.006 0.006 0.006 0.006 t1 0.003 0.003 0.003 0.003 0.003 0.003 t2 0.007 0.007 0.008 0.008 0.008 0.008 t3 0.011 0.010 0.011 0.012 0.011 0.012 t4 0.012 0.012 0.013 0.012 0.013 0.012 t5 0.007 0.008 0.008 0.007 0.007 0.008 t6 0.011 0.011 0.011 0.011 0.011 0.011 t7 0.003 0.004 0.004 0.004 0.003 0.004 t8 0.008 0.008 0.008 0.008 0.007 0.008 total weight 0.1645 0.1693 0.1739 0.1608 0.1635 0.1676 ranking 4 2 1 6 5 3 ijahp article: yavuz, baycan/application of combined analytic hierarchy process (ahp) and swot for integrated watershed management international journal of the analytic hierarchy process issn 1936-6744 31 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.194 references arslan. e.t. (2010). analitik hiyerarşi süreci yöntemiyle strateji seçimi: süleyman demirel üniversitesi i̇ktisadi ve i̇dari bilimler fakültesinde bir uygulama. süleyman demirel üniversitesi i̇ktisadi ve i̇dari bilimler fakültesi dergisi, 15(2), 455-477. babaoğlu. m. (2007). beyşehir gölü’nün sorunları ve alınması gereken önlemler [online]. http://www.mehmetbabaoglu.gen.tr/beygolresim/beysehirraporu.pdf. accessed: 01.12.2009. beierle. t.c. (2002). the quality of stakeholder-based decisions. risk analysis, 22(4), 739–749. bonnell. j.. & baird. a. (2010). community-based watershed management [online]. http://ohioline.osu.edu/ws-fact/0001.html. accessed: 11.05.2010. çelik. n.. & murat. g. (2008). sayısallaştırılmış swot analizi ile bartın i̇l’inin ekonomik yapısını değerlendirme [online]. 2. ulusal i̇ktisat kongresi. (20-22 subat 2008). dokuz eylül üniversitesi i̇i̇bf i̇ktisat bölümü. i̇zmir. http://www.deu.edu.tr/userweb/iibf_kongre/dosyalar/celik.pdf. accessed: 25.02.2011. desteiguer. j.e.. duberstein. j.. & lopes. v. (2003). the analytic hierarchy process as a means for integrated watershed management [online]. in: renard. k.g.. mcelroy. s.a.. gburek. w.j.. canfield. h.e. and scott. r.l. (eds). proceedings of first interagency conference on research in watersheds. pp.737-740. us department of agriculture. agricultural research service. http://www.tucson.ars.ag.gov/icrw/proceedings/steiguer.pdf. dwivedi. p. & alavalapati. j.r.r. (2009). stakeholders’ perceptions on forest biomassbased bioenergy development in the southern us. energy policy, 37, 1999–2007. gürbüz. f. (2010). a’wot analizi-erciyes üniversitesi endüstri mühendisliği bölümü uygulaması [online]. erciyes üniversitesi fen bilimleri enstitüsü dergisi. 26(4), 369378. hacımenni. e. (1998). analitik hiyerarşi süreci ve bilişim teknolojisi kararlarında uygulanması. i̇zmir. dokuz eylül üniversitesi. sosyal bilimler enstitüsü doktora tezi. kajanus. m. kangas. j. & kurttila. m. (2004). the use of value focused thinking and the a’wot hybrid method in tourism management. tourism management, 25 (4), 499-506. kandakoğlu. a. i. akgün. & topçu. y. i. (2007). strategy development and evaluation in the battlefield using quantified swot analytical method [online]. 9th international symposium on the analytic hierarchy process (august 2-6 2007. viña del mar. chile) online proceedings. http://chile2007.isahp.org/. access: 01.01.2010. kangas. j. kurttila. m. kajanus. m. & kangas. a. (2003). evaluating the management strategies of a forestland estate the s–o–s approach. journal of environmental management, 69, 349-358. http://www.mehmetbabaoglu.gen.tr/beygolresim/beysehirraporu.pdf ijahp article: yavuz, baycan/application of combined analytic hierarchy process (ahp) and swot for integrated watershed management international journal of the analytic hierarchy process issn 1936-6744 32 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.194 kangas. j.. pesonen. m.. kurttila. m.. & kajanus. m. (2001). a’wot: integrating the ahp with swot analysis [online]. sixth international symposium on the analytic hierarchy processisahp 2001 (august 2-4. 2001. bern. switzerland). 189-198. http://www.isahp.org/2001proceedings/papers/037-p.pdf. access: 20.01.2011. kurttila. m. pesonen. m. kangas j. & kajanus. m. (2000). utilizing the analytic hierarchy process (ahp) in swot analysis a hybrid method and its application to a forest-certification case. forest policy and economics, 1(1), 41-52. leskinen. l.a. leskinen. p. kurttila. m. kangas. j. & kajanus. m. (2006). adapting modern strategic decision support tools in the participatory strategy process—a case study of a forest research station. forest policy and economics 8(3), 267-278. margerum. r.d. (1999). integrated environmental management: the foundations for successful practice. environmental management, 24(2), 151–166. masozera. m.k. alavalapati. j.r.r. jacobson. s.k. & shrestha. r.k. (2006). assessing the suitability of community-based management for the nyungwe forest reserve. rwanda. forest policy and economics, 8(2), 206-216. mikhailov. l. (2000). a fuzzy programming method for deriving priorities in the analytic hierarchy process. operational research society, 51(3), 341-349. mirchi. a. watkins. d. jr. & madani. k. (2010). modeling for watershed planning. management. and decision making. in: watersheds: management. restoration and environmental impact. jeremy c. vaughn (ed). environmental science. engineering and technology series. e-book. isbn: 978-1-61209-295-9. http://www.kysq.org/docs/mirchi_watershed.pdf. osuna. e.e. & aranda. a. (2007). combining swot and ahp techniques for strategic planning [online]. 9th international symposium on the analytic hierarchy process (august 26 2007. viña del mar. chile) online proceedings. 1-8. http://chile2007.isahp.org/. accessed: 01.01.2010. pesonen. m.. kurttila. m.. kangas. j.. kajanus. m.. & heinonen. p. (2001). assessing the priorities using a’wot among resource management strategies at the finnish forest and park service. forest science. 47 (4). 534-541. qianxiang. l. igbokwe. k.n. & jiayong. l. (2005). community-based integrated watershed management [online]. www.paper.edu.cn/index.php/.../journal-10042857(2005)01-0060-05. accessed: 03.04.2010. saaty. t.l. (2001). fundamentals of the analytic hierarchy process. in the analytic hierarchy process in natural resource and environmental decision making. schmoldt. d.l.. kangas. j. mendoza. g.a. & pesonen. m. (eds). netherlands: kluwer academic publishers. saaty. t.l. (2003). decision aiding decision-making with the ahp: why is the principal eigenvector necessary. european journal of operational research, 145, 85–91. ijahp article: yavuz, baycan/application of combined analytic hierarchy process (ahp) and swot for integrated watershed management international journal of the analytic hierarchy process issn 1936-6744 33 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.194 saaty. t.l. (2008). decision making with the analytic hierarchy process. international journal of services sciences, 1(1), 83-98. saaty. t.l.. & vargas. l.g. (2001). models. methods. concepts and applications of the analytical hierarchy process. dordrecht. netherlands: kluwer academic publisher. sharma. b.r.. samra. j.s.. scott. c.a.. & wani. s.p. (eds.) (2005). watershed management challenges: improving productivity. resources and livelihoods. colombo. sri lanka: international water management institute, 336. shinno. h. yoshioka. h. marpaung. s. & hachiga. s. (2006). quantitative swot analysis on global competitiveness of machine tool industry. journal of engineering design, 17(3), 251-258. shrestha. r.k. alavalapati. j.r.r. & kalmbacher. r.s. (2004). exploring the potential for silvopasture adoption in south-central florida: an application of swot-ahp method. agricultural systems, 81, 185-199. srdjevic z. bajcetic. r. & srdjevic b. (2012). identifying the criteria set for multicriteria decision making based on swot/pestle analysis: a case study of reconstructing a water intake structure. water resources management, 26(12), 3379-3393. srdjevic. b. (2005). combining different prioritization methods in the analytic hierarchy process synthesis. computers & operations research, 32, 1897-1919. stewart. a.r. mohamed. s. & daet. r. (2002). strategic implementation of it/is projects in construction: a case study. automation in construction, 11, 681-694. taşkın. a. & güneri. a.f. (2005). strateji geliştirmede a’wot hibrit metodu kullanımı ve türk kimya sektöründe bir uygulama çalışması [online]. v. ulusal üretim araştırmaları sempozyumu. (i̇stanbul ticaret üniversitesi. 25-27 kasım 2005). 503-507. http://www.iticu.edu.tr/kutuphane/pdf/uas/m01071.pdf. accessed: 06.04.2011. teknomo. k. (2006). analytic hierarchy process (ahp) tutorial, http://people.revoledu.com/kardi/tutorial/ahp/consistency.htm. accessed: 11.01.2010. weihrich. h. (1982). the tows matrix-a tool for situational analysis. long range planning, 15(2), 54-66. wickramasinghe. v. (2008). analytical tourism disaster management framework for sustainable tourism following a sudden calamity [online]. phd dissertation. division of engineering and policy for cold regional environment. hokkaido university. japan. http://133.87.123.206/e3/alumni/abstract/vasantha.pdf. accessed: 11.01.2010. wickramassinghe. v. & takano. s. (2009). application of combined swot and analytic hierarchy process (ahp) for tourism revival strategic marketing planning: a case of sri lanka tourism. journal of the eastern asia society for transportation studies, 8, 954-969. http://people.revoledu.com/kardi/tutorial/ahp/consistency.htm ijahp article: yavuz, baycan/application of combined analytic hierarchy process (ahp) and swot for integrated watershed management international journal of the analytic hierarchy process issn 1936-6744 34 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.194 yılmaz. e. (2007). a’wot tekniği kullanarak katılımcı yaklaşımla proje değerlendirmesi [online]. doa dergisi (journal of doa). 13. 1-16. doğu akdeniz ormancılık araştırma müdürlüğü. mersin. www.doa.gov.tr/doadergisi/doa13/awot%20.pdf. accessed: 25.02.2011. ijahp article: jayant/selection of reverse logistics service provider (rlsp) using analytic network process (anp): a case study of an automotive company international journal of the analytic hierarchy process 131 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.374 selection of reverse logistics service provider (rlsp) using analytical network process (anp): a case study of an automotive company arvind jayant 1 department of mechanical engineering, sant longowal institute of engineering & technology, longowal, sangrur (punjab)-148106, india e-mail id: arvindjayant@rediffmail.com abstract activities in reverse logistics implementation are practiced extensively by lead acid battery manufacturing industries. one important problem faced by the management in the battery manufacturing industry is the regular supply of spent batteries/ lead from the end users in the supply chain management of battery production. the analytic network process (anp) based decision model presented in this work structures the problem related to options in selection of a reverse logistics service provider for supply of eol lead acid batteries in a hierarchical form and links the determinants, dimensions, and enablers of the reverse logistics service provider with alternatives available to the decision maker. a reverse logistics service provider evaluation problem has been formulated that includes two primary tasks: preliminary screening of the rl service providers by a team of managers and an analytic network process (anp) based model for final selection. the proposed approach, therefore, links the financial and nonfinancial, tangible and intangible, internal and external factors, thus providing a holistic framework for the selection of an alternative for the reverse logistics operations for eol batteries. the results of the present work indicate that acceptable cost between the user and the rl service provider companies is the most important determinant which influences the final selection process. this approach also enables the decision-makers to better understand the complex relationships of the relevant attributes in the decision making which may subsequently improve the reliability of the decision. keywords: reverse logistics; analytic network process; multi-criteria decision making; logistics outsourcing 1. introduction with rapid business growth in globalization, industries with relatively limited resources have to outsource some business functions or operations, and purchase raw materials or components/subcomponents from other small medium enterprises to establish an interrelated supply network. consequently, if they would like to execute green programs 1 the authors express their deep sense of gratitude to management and staff of xyz enterprises for providing an opportunity to carry out the project and extending their cooperation for data collection and providing all kinds of support for the present research work. mailto:arvindjayant@rediffmail.com ijahp article: jayant/selection of reverse logistics service provider (rlsp) using analytic network process (anp): a case study of an automotive company international journal of the analytic hierarchy process 132 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.374 to advance their environmental performance, they not only monitor their own operations, but also coordinate other partners in their supply networks, including reverse logistics activities, material suppliers, manufacturers, distributers, users and so on. now-a-days supply chain managers ensure inclusion of traditional performance criteria as well as environmental criteria and this approach is known as green supply chain management. the outsourcing of reverse logistics activities to third party logistics service providers has now become a common practice. reverse logistics encompasses the logistics involving activities all the way from used products no longer required by the user to products that may be usable again in the market. it is the process of planning, implementing, and controlling the efficient, cost effective flow of raw materials, in-process inventory, finished goods and related information from the point of consumption to the point of origin for the purpose of recapturing value or proper disposal (stock et. al., 1998). the most intuitively related notion with such reverse activities involves the physical transportation of used products from the end user back to the producer. reverse distribution activities involve the removal of defective and environmentally hazardous products from the hands of customers. this also includes products that have reached the end of their usable life. it is a process whereby companies can become more environmentally efficient through reusing and reducing the amount of materials used (bhatnagar et al., 1999). ijahp article: jayant/selection of reverse logistics service provider (rlsp) using analytic network process (anp): a case study of an automotive company international journal of the analytic hierarchy process 133 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.374 figure 1. anp model for selection of rl service provider dimensions enablers determinants acceptable cost (act) competitive services (csr) transportation facility (tfc) service quality (srq) selection of rl service provider overall weighted index (owi) customer perspective (cp) internal business perspective (ibp) financial perspective (fp) 1. customer service (cs) 2. customer satisfaction( csf) 3. customer queries and complaint (cqc) 1. information technology (it) 2. on time delivery (otd) 3. new technologies(nte) 1. market shares (ms) 2. return on investment (roi) 3. recapturing values (rev) provider c provider b provider a csf cs cq c it otd nte ms roi rev ijahp article: jayant/selection of reverse logistics service provider (rlsp) using analytic network process (anp): a case study of an automotive company international journal of the analytic hierarchy process 134 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.374 in recent years many indian manufacturing firms have been forced to move to other states due to higher labor costs and fierce domestic competition. the north indian market has become the most important investment market for indian enterprises. thus, indian enterprises must find cheap resources that are available elsewhere in business environments, and the north indian enterprises are recognized as a popular major investment location. however, the gradual transfer of investment to the northern part of india results in a change in the supply chain which poses severe challenges for the enterprise’s logistics capabilities. due to changes in the business environment, the question of how to select the appropriate third-party logistics (3pl) provider for their own enterprise is becoming an important issue. therefore, we propose a logistics service providers decision model, and focus on the lead acid battery manufacturing industry as the research object. we apply the analytic network process (anp) method to deal with dependent problems and evaluate the interdependence between criteria with a quantitative approach. after comprehensive evaluation, we can assist decision makers of xyz limited in assessing the relative ranking of logistics service providers. the objective of this paper is to introduce a comprehensive decision methodology for the selection of a rl service provider that logistics managers and decision-makers can apply to their organization. the proposed methodology allows for evaluation of alternative providers in two steps: (i) initial screening of the providers by a team of managers, and (ii) anp-based final selection. in this methodology, our focus is to demonstrate the application of anp for the final selection of a logistics service provider. therefore, an anp-based model has been developed and illustrated for a battery manufacturing company. after the introduction, the remainder of the paper is organized as follows. first, we review the literature on logistics outsourcing. the literature review includes the developments in logistics outsourcing, selection criteria for the provider, methods currently being used for the selection of a provider, and finally specific problems related to the selection of a provider. next, we present the methodology for the selection of a provider. an anp based approach for the final selection of a provider is a part of this comprehensive methodology. our focus is on the development of an anp model and its solution. for the purpose of model development, we have identified and named four major criteria as determinants. all these determinants are supported by four sub-criteria, named dimensions. each dimension in this model is separately supported by some enablers as shown in figure 1. finally, the paper concludes with a discussion and managerial implications. 2. literature review the purpose of the literature review is to identify the criteria that need to be considered in logistics outsourcing partner selection. other relevant issues such as the tools/techniques currently being used for the selection of a logistics service provider and specific problems related to the selection of a provider have also been captured. the outcome of the literature review, together with the inputs from industry and academia, have been used to (i) suggest a framework that may be used in short-listing the rl logistics providers, and (ii) develop an anp-based model for the final selection of a rl logistics provider. ijahp article: jayant/selection of reverse logistics service provider (rlsp) using analytic network process (anp): a case study of an automotive company international journal of the analytic hierarchy process 135 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.374 2.1 logistics outsourcing& management according to the langley et al. (2002) 3pl survey, the most common outsourced activities are warehousing, outbound transportation, customs brokerage and inbound transportation. keeping in view the growing trend of logistics outsourcing, many providers are now offering a variety of services. these services mainly involve businessto-business relationships, where the user is not only a critical stakeholder but his customers are also directly affected by the quality of service of the provider. therefore, the user must identify exactly what it needs from the provider. the selection of a proper provider, which suits the needs of the outsourcing company (hereinafter called user), is not an easy task. the complexity of this task increases with an increase in the number of selection criteria. ijahp article: jayant/selection of reverse logistics service provider (rlsp) using analytic network process (anp): a case study of an automotive company international journal of the analytic hierarchy process 136 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.374 table 1 summary of literature on the criteria for the selection of a service provider s.no. selection criteria references 1 acceptable cost act lynch (2002), langley et al. (2003), boyson et al. (1999), stock et al. (1998), tam and tummala (2001) 2 competitive services csr bhatnagar et al. (1999), lynch (2002), langley et al. (2003), boyson et al. (1999) 3 transportation facility tfc gibson et al. (2002), murphy et al. (1992), durvusula et al. (2002) 4 service quality srq razzaque and sheng (1998), langley et al. (2003), stock et al. (1998), thompson (1996) 5 customer perspective cp lynch (2002), stock et al. (1998) 6 internal business perspective ibp lynch (2002), andersson and norrman (2002), boyson et al. (1999), bradley (1994) 7 financial perspective fp andersson and norrman (2002), boyson et al. (1999), gattorna and walters (1996) 8 customer service cs lynch (2002), stock et al. (1998) 9 customer satisfaction csf lynch (2002), stock et al. (1998), thompson (1996) 10 customer queries and complaint cqc razzaque and sheng (1998) 11 information technology it lynch (2002), andersson and norrman (2002), langley et al. (2003), boyson et al. (1999) 12 on time delivery otd stock et al. (1998), gattorna and walters (1996) 13 new technologies nte lynch (2002), andersson and norrman (2002), boyson et al. (1999), thompson (1996), gattorna and walters (1996) 14 market shares ms thompson (1996 15 return on investment roi boyson et al. (1999), bradley (1994), maltz (1995) 16 recapturing values rev lynch (2002), langley et al. (2003), boyson et al. (1999) ijahp article: jayant/selection of reverse logistics service provider (rlsp) using analytic network process (anp): a case study of an automotive company international journal of the analytic hierarchy process 137 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.374 table 2 criteria used in anp model s. no. selection criteria descriptions 1 acceptable cost (act) it refers to the total cost of reverse logistics outsourcing, which should be minimal. 2 competitive services (csr) it refers to the ability of the user and the provider and their support systems to work together in close coordination to achieve success in competitive business environment. it may be classified in terms of the attributes of business process, technology capability, characteristics of other service providers of the user, etc. 3 transportation facility (tfc) transportation capability of the service provider must be very sound and effective to meet the set goals in the reverse logistics business. 4 service quality (srq) service quality of the provider includes many aspects such as on-time collection & delivery of used items, accuracy of order fulfillment, frequency and cost of loss and damage, promptness in attending customers’ complaints etc. 5 customer perspective (cp) customer perspective of the provider may not only provide good service to the user but may also foster a long-term relationship between the user and the provider. 6 internal business perspective (ibp) provision of periodic evaluation of the performance of the provider enables the two parties to identify the gaps in service. on-time shipments, inventory accuracy, shipping errors, reduction in cash-to-cash cycle, logistics cost reduction, and reduction in customer’s complaints may be used as the most important performance measures in reverse logistics outsourcing. 7 financial perspective (fp) a sound financial perspective of the provider ensures continuity of service and regular upgrading of the equipment and services, which are used in reverse logistics operations. 8 customer service (cs) a good performance towards customers of the provider is reflected by measures such as delivery performance, performance-monitoring capability, statistical data reporting to the user, fault diagnosis capability, detailed accounting information, system security, responsiveness, confidentiality of sensitive data, etc. 9 customer satisfaction (csf) level of customer satisfaction of reverse logistics service provider company in the business environment. 10 customer queries and complaint (cqc) it refers to customer queries and complaint handling capacity of the service provider. it can be monitored by observing the past business performance of the company/service provider. 11 information technology (it ) the advanced it capabilities of a provider help in reducing uncertainties and inventory level. 12 on time delivery (otd) two dimensions of otd, namely “speed” and “reliability”, are important for the satisfaction of the user. 13 new technologies (nte) mutual trust-based information sharing between the user and the provider is necessary not only for the continuance of the agreement but also for the continuous improvement of the service. it can be achieved by installing new business technologies like erp, vmi, edi and e-commerce etc. 14 market shares (ms) the market share of the provider reflects its financial performance, customer satisfaction, and reputation in the business market. 15 return on investment (roi) return on investment refers to development of required facilities for reverse logistics operations by case company and service provider and their chances to get sufficient profitability. 16 recapturing values (rev) reverse logistics programs in addition to the various environmental and the cost benefits can proactively minimize the threat of government regulation and can improve the corporate image of the companies by development of rl facilities for the purpose of recapturing value or proper disposal of used /eol products. ijahp article: jayant/selection of reverse logistics service provider (rlsp) using analytic network process (anp): a case study of an automotive company international journal of the analytic hierarchy process 138 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.374 the analytic hierarchy process (gattorna et al., 1996) is one of the widely used approaches to handle multi-criteria decision-making problems like this. however, a significant limitation of the ahp is the assumption of independency among various criteria of decision-making. the analytic network process (anp), on the other hand, captures interdependencies among the decision attributes and allows a more systematic analysis. it also allows inclusion of all the relevant criteria (tangible or intangible, objective or subjective, etc.) that have some bearing on arriving at the best decision. these criteria also have some interdependencies, which cannot be captured by the popular ahp method (lawshe, 1975). therefore, instead of using the commonly used ahp approach, we recommend the use of an anp-based model for the selection of a service provider. fifteen years ago, “logistics” had not yet been explored to a great extent. however today, with the development of information technology and increased customer demand, enterprises have to handle lots of thorny tasks to take care of service problems. therefore, the area of “logistics” is getting considerable attention from enterprises. logistics has evolved from a transportation focus based primarily on agricultural economics to the view that it is a diverse and key component of business strategy, differentiation, and a link to customers (jharkharia & shankar, 2007). with changing market conditions, logistics has become a part of supply chain management that plans, implements, and controls the efficient, effective forward and reverse flow and storage of goods, services and related information between the point of origin and the point of consumption in order to meet customer’s requirements. therefore, according to the definition, logistics includes the flow of goods, services and information related to movements of goods and services from the suppliers to a satisfied customer without waste, namely, the concept of integrated the logistics has also been portrayed in a positive light with its promises to bring higher customer service levels while decreasing distribution costs (langley, 2003). 2.2 third-party logistics third-party logistics (3pl) is also referred to as logistics outsourcing and is an emerging trend in the global market (langley, 2002). according to lieb (1992), third-party logistics (3pl) involves the use of external companies to perform logistics functions that have traditionally been performed within an organization. the functions performed by the third party can encompass the entire logistics process or selected activities within that process. third-party logistics services are mostly focused on transportation and warehousing, etc, and the 3pl service providers must have professional experience in each service. 2.3 3pl provider selection criteria it has become very difficult in a real business environment to select parameters for evaluation of service providers and to develop metrics to evaluate the selection criteria of a logistics service provider. in general, enterprises usually have a variety of different supplier characteristics; however, if the same methods are used to evaluate the different types of suppliers, the result will not represent the real situation. hence, when we select the logistics service provider criteria, we also have to consider that the criteria of selection differ with different types of logistics service providers. in a related strain of research, we have organized some literature on logistics service provider selection criteria. jharkharia and shankar (2007) proposed a methodology that consists of two parts: (1) preliminary screening of the available providers, and (2) an anp-based final ijahp article: jayant/selection of reverse logistics service provider (rlsp) using analytic network process (anp): a case study of an automotive company international journal of the analytic hierarchy process 139 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.374 approach for the final selection of the 3pl service providers. they organized the literature to select the appropriate criteria and form the basis for the development of an anp model. tam and tummala (2001) applied the binary logistics model in high-tech industry for logistics service providers in taiwan. they proposed that the service performance is the strongest criteria in the 3pl service provider selection, followed by service cost and added value. this research summarized some frequently used criteria from the literature which are listed in table 1and brief description about the criteria used in the anp model is also given in the table 2. 3. problem description the company chosen for this study is a battery manufacturing industry located in the northern part of india. the main scope of this study is to evaluate logistics service providers in order to determine who to hire to collect and deliver the end-of-life (eol) lead-acid batteries back to the battery company for the purpose of reclaiming the lead from automotive batteries. in the forward supply chain, the major raw materials such as virgin lead, plastic, and sulphuric acid are procured from different suppliers for new battery production which is used in two wheelers, four wheelers, and for other industrial applications. once the battery is produced in different plants it must be disseminated through distributors, wholesalers, retailers and then customers. after its end of life, the automobile owner leaves the used battery at the automobile service station (initial collection point) where it is replaced by a new one. the used batteries collected at the service stations should be quickly shipped to a centralized return center where returned products are inspected for quality failure and sorted for potential repair or recycling. after inspection, the useless batteries (those not able to be recycled) are disposed of and reusable batteries are transported to disassembly/recycling plants where they are crushed and separated into different components (lead, plastic, acid etc.). the remaining components, except the lead, are sold to the third party for other applications. finally, the recycled lead is transported to the battery manufacturing plants where this secondary lead is used along with the virgin lead for new battery production. a series of interviews and discussion sessions were held in the plant with company management, battery retailers and state pollution control boards officials during this project and the following problem areas were identified for improvement in a closed loop supply chain of lead acid batteries.  uncertainty involved in the supply of spent batteries to the recycling company; company is unable to forecast the quantity of collected eol products.  presence of illegal lead smelting units in the state for unauthorized battery collection & lead recycling operation in business environment.  lack of any well-structured model of reverse logistics practice in the company.  underutilization of existing facilities for the battery closed loop supply chain. to solve these problems and improve business performance, the battery manufacturing company is ready to assign the work of regular supply of end-of-life (eol) batteries to a logistics service provider. the team of managers must be capable of clearly defining the objectives and expectations they have for outsourcing the logistics services so that the provider understands exactly what their goals and objectives are. an accurate description ijahp article: jayant/selection of reverse logistics service provider (rlsp) using analytic network process (anp): a case study of an automotive company international journal of the analytic hierarchy process 140 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.374 of the service requirements would minimize the need for the provider to make assumptions, and ensure fewer surprises down the road. service standards expected from the service providers should include both the present and projected service standards. the focus of this research is building a sound decision support methodology for evaluation and selection of the best reverse logistics service provider in the battery closed loop supply chain in order to minimize the total supply chain cost comprising procurement, production, distribution, inventory, collection, disposal, disassembly and recycling cost by making a responsive supply chain environment. 4. decision support methodology for the selection of a logistics service provider the proposed decision support methodology allows for the assessment of alternative logistics service providers in two steps: (i) initial screening of the providers by a team of concerned managers, and (ii) an anp-based decision support system for the final evaluation of the service providers. often, the initial screening of the service providers is an easy task, but the final selection from the list of short-listed providers is difficult. in this section, we present a methodology for the initial screening of the providers. later, these short-listed providers will be ranked by the anp-based approach. the various steps of the decision support methodology are as follows: 1. form a team of competitive managers and a consultant 2. make a decision regarding type of outsourcing service level required and collection objectives 3. collect and identify functional specifications of the proposed task 4. identify potential reverse logistics service providers 5. evaluate request of rl logistics service providers (rllsp) 6. develop request for proposal offer from service providers 7. evaluate service proposal offer supplied by the logistics service providers 8. perform field visits and inspect facilities of the logistics service providers 9. collect feedback from the existing customers of the service providers 10. make final selection using the anp approach and agreement for service the anp-based decision modeling methodology, which is discussed in the next section, is recommended for the final selection of a rl service provider. for any long term business relationship a business contract between two parties must address scope of work, damage types, individual status, responsibilities, risks and rewards, remedies, extra services, termination, agreement modification, liabilities, limitations, compensation, insurance, rate adjustments, service compensations, performance measurement issues, etc. 5. analytic network process approach many decision problems cannot be built as a hierarchy because of dependencies (inner/outer) and influences between and within clusters (criteria, alternatives). the anp is very useful for solving these kinds of problems. it provides a general framework to ijahp article: jayant/selection of reverse logistics service provider (rlsp) using analytic network process (anp): a case study of an automotive company international journal of the analytic hierarchy process 141 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.374 deal with decisions without making assumptions about the independence of higher-level elements from lower level elements, or about the independence of the elements within a level. in fact, anp uses a network without the need to specify levels as in a hierarchy. ahp was first introduced by saaty (1980), and is based on 1-9 scale. saaty (1996) further developed this issue and suggested the usage of anp to solve the problem of independence on alternatives or criteria and the usage of anp to solve the problem of dependence among alternatives or criteria. the structural difference between ahp (hierarchy) and anp (network) is also shown in figure 2. the figure illustrates that a hierarchy is the simple and special case of a network. figure 2. structural difference between ahp (hierarchy) and anp (network) anp is a combination of two parts: 1) network of criteria and sub criteria that control the interactions 2) the network of influences of elements and clusters a feedback system is represented by a network where nodes correspond to levels or components. the nodes in a cluster (level) may influence some or all the nodes in another cluster. relationships in a network are represented by arcs and the direction of arcs signifies dependency. interdependency between two clusters is shown in figure 1 by two way arcs and inner dependencies are represented by loop arcs. anp is a multi-attribute, decision-making approach based on the reasoning, knowledge, and experience of the experts in the field. it can act as a valuable aid for decision making involving both tangible as well as intangible attributes that are associated with the model being studied. anp relies on the process of eliciting managerial inputs, thus allowing for structured communication among decision makers. thus, it can act as a qualitative tool for strategic decision-making problems. saaty (1980, 1996) proposed the following four main steps of the anp: step 1: model construction and problem structuring: the problem should be clearly stated and decomposed into a rational system, such as a network. the framework can be determined based on the decision maker’s opinion through brainstorming or other appropriate methods. ijahp article: jayant/selection of reverse logistics service provider (rlsp) using analytic network process (anp): a case study of an automotive company international journal of the analytic hierarchy process 142 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.374 step 2: pair-wise comparison matrices and priority vectors: similar to the ahp, pairs of the decision elements at each cluster are compared with respect to their importance towards their control criteria. the clusters themselves are also compared pair-wise with respect to their contribution to the objective. the decision maker is asked to respond to a series of pair-wise comparisons of two elements or two clusters to be evaluated in terms of their contribution to their particular upper level criteria. in addition, interdependencies among the elements of a cluster must also be examined pair-wise where the influence of each element on other elements can be represented by an eigenvector. the relative importance values are determined with saaty’s 1–9 scale, where a score of 1 represents equal importance between the two elements and a score of 9 indicates the extreme importance of one element (row cluster in the matrix) compared to the other one (column cluster in the matrix). a reciprocal value is assigned to the inverse comparison, i.e. aij =1/aji, where aij denotes the importance of ith element over jth one. like with the ahp, pair-wise comparison in the anp is also performed in the framework of a matrix, and a local priority vector can be derived as an estimate of the relative importance associated with the elements (or clusters) being compared by solving the following equation aw=λmaxw (1) where a is the matrix of pair-wise comparison, w is the eigenvector and λ max is the largest eigenvalue of a. if a denotes a consistency matrix, then the eigenvector x can be determined using the following expression: (a−λmaxi)x=0 (2) where i is an identity matrix, the consistency index (ci) and consistency ratio (cr) are used to verify the consistency of the pair wise comparison matrix [10]. the ci and cr values can be defined as below: ci= 𝜆𝑚𝑎𝑥 –𝑛 𝑛−1 (3) cr=ci/ri (4) where n is the number of elements and ri denotes the average consistency index for numerous random entries of same-order reciprocal matrices. if cr≤0.1, then the pair wise comparison matrix is consistent; otherwise, a new comparison matrix is solicited until cr≤0.1. from the pair-wise comparison matrix, using the normalized geometric mean (ngm) method [10], the relative priorities of the elements being compared with respect to their upper level criteria are estimated. priority vectors must be determined for all the comparison matrices. step 3: super-matrix formation: to obtain the global priorities in a system with interdependent influences, the local priority vectors are entered in the appropriate columns of a matrix. as a result, a super-matrix is formed which is actually a partitioned matrix, where each matrix segment represents a relationship between two clusters in a system. let the clusters of a decision system be ck; k=1, 2, 3, . . ., n and each cluster k ijahp article: jayant/selection of reverse logistics service provider (rlsp) using analytic network process (anp): a case study of an automotive company international journal of the analytic hierarchy process 143 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.374 has mk elements, denoted by ek1, ek2, . . ., ekmk. the local priority vectors, obtained in step 2, are grouped and placed in the appropriate positions in a super-matrix based on the flow of influence from one cluster to another, or from a cluster to itself, as in the loop. a standard super-matrix is shown as below [10]. ijahp article: jayant/selection of reverse logistics service provider (rlsp) using analytic network process (anp): a case study of an automotive company international journal of the analytic hierarchy process 144 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.374 𝑪𝟏 𝑪𝟐 ... 𝑪𝑵 𝑒11 𝑒22 ... 𝑒1𝑛 𝑒21 𝑒22 ... 𝑒2𝑛 ... 𝑒𝑁1 𝑒𝑁2 ... 𝑒𝑁 𝑪𝟏 𝑒11 𝑊11 𝑊12 𝑊1𝑁 𝑒22 ... 𝑒1𝑛 ... ... ... ... ... ... 𝑪𝑵 𝑒𝑁1 𝑊𝑁1 𝑊𝑁2 𝑊𝑁𝑁 𝑒𝑁2 ... 𝑒𝑁 figure 3. standard super-matrix for example, the super matrix representation for a hierarchy with three levels, as shown in figure 3 is represented as: wh = 0 0 0 𝑊21 0 0 0 𝑊32 𝐼 (5) in this matrix, w21 is a vector, which represents the impact of the goal on the criteria, w32 is a matrix that represents the impact of the criteria on each of the alternatives, i is an identity matrix and zero entries correspond to those elements having no influence. for the given example, if the criteria are interrelated, the hierarchy is replaced with the network, as shown in figure 1. the interdependency is exhibited by the presence of the matrix element w22 of the super matrix yielding the following matrix: wn = 0 0 0 𝑊21 𝑊22 0 0 𝑊32 𝐼 (6) note that a matrix can replace any zero value in the super-matrix if there is an interrelationship of the elements within a cluster or between two clusters. since there is usually interdependence among clusters in a network, the columns of a super-matrix may ijahp article: jayant/selection of reverse logistics service provider (rlsp) using analytic network process (anp): a case study of an automotive company international journal of the analytic hierarchy process 145 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.374 sum to more than one. however, the super-matrix must be modified so that each column of the matrix sums to unity [10]. step 4: selection of the best alternative: if the super-matrix formed in step 3 covers the whole network, the priority weights of the alternatives can be found in the column of alternatives in the normalized super-matrix. on the other hand, if the super-matrix only comprises clusters that are interrelated, additional calculations must be made to obtain the overall priorities of the alternatives. the alternative with the highest overall priority should be selected as the best choice. in any decision-making process, it is very important to consider the interdependent relationship among criteria that exists in many real life problems. however, the ahp is restricted to solving problems that have a linear unidirectional hierarchical relationship among criteria. the anp does not require this strictly hierarchical structure and therefore, can treat problems having complex interrelationships among criteria (dependences and feedbacks) as is often encountered while making societal, governmental and corporate decisions. it can allow inclusion of criteria, both tangible and intangible, which have some bearing on making the best decision. the looser network structure of the anp makes the representation of any decision problem possible without concern for what order the criteria are in as in a hierarchy. the anp is unique in the sense that it provides synthetic scores, which is an indicator of the relative ranking of different alternatives available to the decision maker. unfortunately, the anp applications have been noticeably limited as compared to the ahp, due to its complexity and high computational time. 5.1 advantages of anp  anp is a comprehensive technique that allows for the inclusion of all the relevant criteria; tangible as well as intangible, which have some bearing on decision-making process (saaty, 1996).  ahp models a decision-making framework that assumes a uni-directional hierarchical relationship among decision levels, whereas anp allows for more a complex relationship among the decision levels and attributes as it does not require a strict hierarchical structure.  in decision-making problems, it is very important to consider the interdependent relationship among criteria because of the characteristics of interdependence that exist in real life problems. the anp methodology allows for the consideration of interdependencies among and between levels of criteria, and thus is an attractive multi-criteria decision-making tool. this feature makes it superior to the ahp which fails to capture interdependencies among different enablers, criteria, and sub-criteria (saaty, 1996).  anp methodology is beneficial in considering both qualitative as well as quantitative characteristics, as well as taking the non-linear interdependent relationship among the attributes into consideration.  anp is unique in the sense that it provides synthetic scores, which is an indicator of the relative ranking of different alternatives available to the decision maker. ijahp article: jayant/selection of reverse logistics service provider (rlsp) using analytic network process (anp): a case study of an automotive company international journal of the analytic hierarchy process 146 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.374 5.2 disadvantages of anp  identifying the relevant attributes of the problem and determining their relative importance in the decision making process requires extensive discussion and brainstorming sessions. also, data acquisition is a very time intensive process for the anp methodology.  anp requires more calculations and formation of additional pair-wise comparison matrices than the ahp process. thus, a careful track of matrices and pair-wise comparisons of attributes is necessary.  the pair-wise comparison of attributes under consideration can only be subjectively performed, and hence the accuracy of the results depends on the user’s expertise knowledge in the area concerned. 6. application of analytical network process (anp) step 1: model development and problem formulation: in this step, the decision problem is structured into its important components. the relevant criteria and alternatives are chosen on the basis of the review of literature and discussion with company management. the relevant criteria and alternatives are structured in the form of a control hierarchy where the criteria at the top level in the model have the highest strategic value. the toplevel criteria in this model are acceptable cost (act), competitive services (csr), transportation facility (tfc) and service quality (srq). these four criteria are called the determinants. in the second level of hierarchy, four sub-criteria called dimensions of the model are placed at the top level of the hierarchy which supports all the four determinants. these dimensions are customer perspective (cp), internal business perspective (ibp), and financial perspective (fp).in the proposed anp model, each of the three dimensions has some enablers which help to achieve that particular dimension. for example, the dimension ibp is supported by the enablers it, otd and nte. these enablers also have some interdependency with one another. for example, in the dimension ibp, enablers it and nte are interdependent as sincere deliveries on time would be necessary for procuring new technologies. the degree of interdependency may vary from case to case and will be captured in later steps. the strength of the anp model is that the feedback and the network structure of the anp makes representation of the decision problem possible without much concern for what comes first and what comes next in a hierarchy. the objective of this hierarchy is to select the best possible alternative that will meet the goals of conducting effective reverse logistics in a battery manufacturing industry. the developed anp model is presented in figure 1. the alternatives that the decision maker wishes to evaluate are shown at the bottom of the model. the opinion of the logistics manager of the company was sought in the comparisons of the relative importance of the criteria and the formation of pair-wise comparison matrices to be used in the anp model. in this paper, mainly for the purpose of brevity, we present and illustrate only the results of the act determinant. the results of all four determinants would be used in the calculation of the logistics overall weighted index (lowi) which indicates the score assigned to a logistics provider. step 2: pair-wise comparison of four determinants: in this step, the decision maker is asked to respond to a series of pair-wise comparisons where two components at a time are compared with respect to an upper level ‘control’ criterion. these comparisons are made so as to establish the relative importance of determinants in achieving the case ijahp article: jayant/selection of reverse logistics service provider (rlsp) using analytic network process (anp): a case study of an automotive company international journal of the analytic hierarchy process 147 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.374 company’s objectives. in such comparisons, a scale of 1–9 is used to compare two options (saaty, 1980). a score of 1 indicates that the two options under comparison have equal importance, while a score of 9 indicates the overwhelming dominance of the component under consideration (row component) over the comparison component (column component) in a pair-wise comparison matrix. when a component has a weaker impact than its comparison component, the range of the scores will be from 1 to 1/9, where 1 indicates indifference and 1/9 represents an overwhelming dominance by a column element over the row element. for the reverse comparison between the components already compared, a reciprocal value is automatically assigned within the matrix, so that in a matrix aij*aji=1. the matrix showing pair-wise comparison of determinants along with the e-vectors of these determinants is shown in table 3. the evectors (also referred to as local priority vector) are the weighted priorities of the determinants and are shown in the last column of the matrix. in this paper, a two-stage algorithm is used for computing the e-vector. for the computation of the e-vector, we first add the values in each column of the matrix, then, divide each entry in each column by the total of that column. the normalized matrix is obtained which permits the meaningful comparison among elements. table 3 pair wise comparison of determinants determinants act csr tfc srq e-vector act 1 5 9 7 0.6322 csr 1/5 1 7 3 0.2268 tfc 1/9 1/7 1 1/3 0.0442 srq 1/7 1/3 3 1 0.0968 c.r 0.0770 table 4 pair wise comparison matrix for dimension under acceptable cost pair wise comparison matrix for dimension under acceptable cost cp ibp fp e-vector cp 1 5 7 0.71 ibp 1/5 1 3 0.21 fp 1/7 1/3 1 0.08 finally, averaging over the rows is performed to obtain the e-vectors. these e-vectors will be used in table 11 for the calculation of logistics overall weighted index (lowi) for alternatives. step 3: pair-wise comparison of dimensions: in this step, a pair-wise comparison matrix is prepared for determining the relative importance of each of the dimensions of logistic providers (cp, ibp and fp) on the determinant of logistics providers. in the model, four such matrices would be formed, one for each of the determinant. the matrix for the ijahp article: jayant/selection of reverse logistics service provider (rlsp) using analytic network process (anp): a case study of an automotive company international journal of the analytic hierarchy process 148 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.374 acceptable cost determinant is shown in table 4. the results of this comparison (evectors) are carried as pja in table 10. step 4: pair-wise comparison matrices between component/enablers levels: in this step, the decision maker is asked to respond to a series of pair-wise comparisons where two components will be compared at a time with respect to an upper level control criterion. the pair-wise comparisons of the elements at each level are conducted with respect to their relative influence towards their control criterion. in the case of interdependencies, components within the same level may be viewed as controlling components for each other, or levels may be interdependent on each other. for a determinant, pair-wise comparison is done between the applicable enablers within a given dimension cluster. all the pair-wise comparison matrices for the dimensions under each determinant are not shown here. for example, in table 5, the relative importance of cqc when compared to cs with respect to cp, in achieving the acceptable cost, is 3. from table 5 it is also observed that for the case company, the enabler cs has the maximum influence (0.58) on cp for the act. similarly, cqc has the minimum influence (0.16) on cp under act. the number of such pair-wise comparison matrices depends on the number of determinants and the dimensions in the anp model. in this model, 12 such pair-wise comparison matrices are formed. the e-vectors obtained from these matrices are imported as a d kja. table 5 pair-wise comparison matrix for customer perspective under the acceptable cost determinant step 5: pair-wise comparison matrices of interdependencies: pair-wise comparisons are performed to consider the interdependencies among the enablers. from table 6, it is observed that nte (0.67) has the maximum impact on the ibp–csr cluster with it as the control enabler over others. it is also observed that the impact of otd on it in ibp– csr cluster is minimal (0.33). therefore, otd is not a problem for the user company and it will have little impact on information technologies in the ibp–csr cluster. for each determinant, there will be 12 such matrices at this level of relationship. the evectors from these matrices are used in the formation of super matrices. as there are four determinants, 48 such matrices will be formed. the e-vectors have been used in the sixth column of the super matrices. step 6: evaluations of alternatives: the final set of pair-wise comparisons is made for the relative impact of each of the alternatives, enablers in influencing the determinants. the number of such pair-wise comparison matrices is dependent on the number of pair-wise comparison matrix for customer perspective under the acceptable cost determinant act/cp cs csf cqc e-vector cs 1 3 3 0.58 csf 1/3 1 2 0.26 cqc 1/3 1/2 1 0.16 c.r 0.042 ijahp article: jayant/selection of reverse logistics service provider (rlsp) using analytic network process (anp): a case study of an automotive company international journal of the analytic hierarchy process 149 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.374 enablers that are included in each of the determinants. in our present case, there are 9 enablers for each of the determinants, which lead to 36 such pair-wise matrices. all the pair-wise comparison matrices are not given here. influence matrices with respect to act determinants are given in the appendix. the e-vectors from this matrix are used in columns 7–9 of desirability indices matrices. the columns 7–9, correspond to providera, provider-b and provider-c, respectively. table 6 pair-wise comparison matrix for enablers step 7: super matrix formations: the super matrix allows for a resolution of the interdependencies that exist among the elements of a system. it is a partitioned matrix where each sub-matrix is composed of a set of relationships between and within the levels as represented by the decision maker’s model. in this model, there are four super matrices for each of the four determinants of the reverse logistics service provider hierarchy network which need to be evaluated. all such super matrices ‘m’, are shown below. the results of the relative importance measures are presented for each of the enablers for the individual’s determinant of the reverse logistics service provider. the values of the elements of the super matrix m have been imported from the pair-wise comparison matrices of interdependencies (for example, table 6). as there are 9 such pair-wise comparison matrices, one for each of the interdependent enablers in the act, there will be 9 non-zero columns in this super matrix. each of the non-zero values in the column is the relative importance weight associated with the interdependent pair-wise comparison matrices. pair-wise comparison matrix for enablers under competitive services, internal business perspective and information technology csr/ibp it otd nte e-vector otd 1 1/2 0.33 nte 2 1 0.67 ijahp article: jayant/selection of reverse logistics service provider (rlsp) using analytic network process (anp): a case study of an automotive company international journal of the analytic hierarchy process 150 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.374 table 7 super matrix ‘m’ for acceptable cost before convergence super matrix ‘m’ for acceptable cost before convergence: cs csf cqc it otd nte ms roi rev cs 0 0.17 0.17 csf 0.91 0 0.83 cqc 0.09 0.83 0 it 0 0.13 0.25 otd 0.75 0 0.75 nte 0.25 0.87 0 ms 0 0.8 0.15 roi 0.33 0 0.85 rev 0.67 0.2 0 in the next stage, the super matrix m is made to converge to obtain a long-term stable set of weights. for convergence to occur, the super matrix needs to be ‘column stochastic’, i.e. the sum total of each of the columns of the super matrix needs to be one. raising the super matrix m to the power 2 k+1 , where k is an arbitrarily large number, allows for the convergence of the interdependent relationships. table 6 represents the super matrix before convergence for determinant acceptable cost, and table 8 represents a converged super matrix for determinant acceptable cost. a similar super matrix will be regenerated for other determents in the analysis. ijahp article: jayant/selection of reverse logistics service provider (rlsp) using analytic network process (anp): a case study of an automotive company international journal of the analytic hierarchy process 151 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.374 table 8 super matrix ‘m’ for acceptable cost after convergence m 8 super matrix ‘m’ for acceptable cost after convergence: m 8 cs csf cqc it otd nte ms roi re v cs 0.1453 0.1453 0.1453 csf 0.4599 0.4599 0.4599 cqc 0.3948 0.3948 0.3948 it 0.1589 0.1589 0.1589 otd 0.4286 0.4286 0.4286 nte 0.4126 0.4126 0.4126 ms 0.3366 0.3366 0.3 366 roi 0.3648 0.3648 0.3 648 rev 0.2985 0.2985 0.2 985 step 8: selection of the best alternative for a determinant: the selection of the best alternative depends on the outcome of the ‘desirability index’. the desirability index, dia, for the alternative i and the determinant a is defined as (jayant, 2012). dia = ∑ ∑ p kja k=1 j j=1 ja a d kja a i kja sikja. (7) in this equation, pja is the relative importance of dimension j in influencing the determinant a. a d kja is the relative importance of an enabler k in influencing the determinant a through dimension j for the dependency (d) relationships. a i kja is the stabilized importance weight of the enabler k in the dimension j and determinant a cluster for interdependency (i) relationships. these values are taken from the converged supermatrix. sikja is the relative impact of alternative i on enabler k of dimension j for determinant a. kja is the index set of enablers for dimension j of determinant a, and j is the index set for dimension j. table 10 presents the desirability indices for the acceptable cost determinant (di act). it is based on the hierarchy using the relative weights obtained from the pair-wise comparison of alternatives, dimensions and weights of enablers from the converged super matrix. these weights are used to calculate a score for the determinants of logistics provider overall weighted index (lowi) for each of the alternative being considered. in table 10, the values of the third column are imported from table 9, which are obtained by comparing the relative impact of the dimensions on the acceptable cost determinant. for example, in improving the acceptable cost, the role of customer perspective is found to be most important (0.71), which is followed by ibp(0.21), fp (0.08).in a similar pattern the desirability indices for all determinants have been generated in the analysis. ijahp article: jayant/selection of reverse logistics service provider (rlsp) using analytic network process (anp): a case study of an automotive company international journal of the analytic hierarchy process 152 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.374 table 9 pair wise comparison matrix for dimension under acceptable cost the values in the fifth column of table 10 are the stable independent weights of enablers obtained through the converged super matrix (table 8). the next three columns are from the pair-wise comparison matrices giving the relative impact of each of the alternatives on the enablers. the final three columns represent the weighted values of the alternatives (pja*a d kja*a i kja*sikja) for each of the enablers. for the purpose of illustration, the value corresponding to provider a for cs is 0.047 (0.71*0.58*0.1453*0.80= 0.047). the summations of these results, for the acceptable cost of each of these alternatives, are presented in the final row of table 10. these results indicate that the provider a with a value of 0.162 has maximum influence on the acceptable cost. it is followed by provider b (0.066) and provider c (0.038). similar analysis is carried out for the other three determinants. in the next step, an index would be calculated to capture the achievement of the overall goal of selecting an alternative. table 10 acceptable cost desirability indices pair wise comparison matrix for dimension under acceptable cost act cp ibp fp e-vector cp 1 5 7 0.71 ibp 1/5 1 3 0.21 fp 1/7 1/3 1 0.08 c.r 0.061 acceptable cost desirability indices: act enabler s pja a d kja a i kja (pja* a d kja* a i kja ) s1kja s2kja s3kja providera providerb providerc 1 2 3 4=1*2*3 5 6 7 8=4*5 9=4*6 10=4*7 cp cs 0.71 0.58 0.1453 0.059 0.80 0.13 0.07 0.047 0.007 0.004 cp csf 0.71 0.26 0.4599 0.084 0.72 0.20 0.08 0.060 0.016 0.006 cp cqc 0.71 0.16 0.3948 0.044 0.10 0.61 0.29 0.004 0.026 0.012 ibp it 0.21 0.67 0.1589 0.022 0.72 0.20 0.08 0.015 0.004 0.001 ibp otd 0.21 0.23 0.4286 0.020 0.62 0.11 0.27 0.012 0.002 0.005 ibp nte 0.21 0.10 0.4126 0.008 0.10 0.73 0.17 0.008 0.005 0.001 fp ms 0.08 0.49 0.3366 0.013 0.69 0.21 0.10 0.008 0.002 0.001 fp roi 0.08 0.20 0.3648 0.005 0.07 0.66 0.27 0.003 0.003 0.001 fp rev 0.08 0.31 0.2985 0.007 0.74 0.16 0.10 0.005 0.001 0.007 d11= 0.162 d12= 0.066 d13= 0.038 ijahp article: jayant/selection of reverse logistics service provider (rlsp) using analytic network process (anp): a case study of an automotive company international journal of the analytic hierarchy process 153 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.374 step 9: calculation of logistic provider overall weighted index (owi): the owi for an alternative i (owii ) is the summation of the products of the normalized desirability indices (dian) and the relative importance weights of the determinants (ca). in the calculation of owi the use of normalized values of dia ensures that the owi values of the alternatives do not change with a large range of absolute values of dia for different determinants. in other words, it may be said that the values of owi using normalized values of dia are unit invariant. the normalized values of desirability indices also ensure that the sum of owi values is equal to 1.00. owi is mathematically represented as owii = dian ca. for example, owi for a is calculated as: owi a =[(0.6322*0.162)+( 0.2268*0.11211)+( 0.0442*0.10412)+( 0.0968*0.21019)] = 0.517499 these results are presented in table 11. it is observed from table 11 that provider a is the most-suited logistics service provider. it is also observed that acceptable cost plays a major role in the selection of a service provider. it is further observed that provider b (0.2878) is found to be more cost effective as compared to c (0.1946). this difference is probably due to the availability of supply chain visibility software availability and other advanced it tools application in the business, which provider a offers to its user in addition to the basic logistics services. if the user chooses provider b or c for the basic logistics needs and separately procures scm solutions and other advanced capability from other vendors, the costs of these services might be higher than what the user would pay to service provider a. however, these results should be viewed in the light of the battery manufacturing company and the inputs provided by the decision-making team in the formation of pair wise comparison matrices. table 11 also demonstrates that acceptable cost plays a major role in the conduct logistics operations as compared to other determinants like competitive services, transportation facility and service quality in the decision analysis. table 11 logistics overall weighted index (lowi) for alternatives logistics overall weighted index (lowi) for alternatives: alternatives act csr tfc srq lowi normalized values for lowi weights 0.6322 0.2268 0.0442 0.0968 provider a 0.162 0.11211 0.10412 0.21019 0.152791 0.517499 provider b 0.066 0.14039 0.08333 0.07993 0.084986 0.287845 provider c 0.038 0.10103 0.052514 0.084856 0.057472 0.194657 ijahp article: jayant/selection of reverse logistics service provider (rlsp) using analytic network process (anp): a case study of an automotive company international journal of the analytic hierarchy process 154 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.374 7. discussion and managerial implications the proposed methodology provides for simplification of a complex multi-criteria decision-making problem. it may also be used to quantify many subjective judgments, which are necessary to evaluate different alternative providers. another advantage of this methodology is that it not only supports group decision-making but also enables us to document the various considerations in the process of decision making. this documentation is useful if the results are to be communicated to various interest groups. in this study, the results indicate that rl service provider a is the first choice of the battery manufacturing company. this may be attributed to its advanced it, customer service, and change management capabilities. the expertise of provider c in the framing of transportation and distribution policy also supports this result. it is pertinent to discuss the priority values of the determinants which influence this decision. from table 3, it is observed that acceptable cost (act = 0.6322) is the most important determinant in the selection of a service provider. it is followed by competitive services (0.2268), service quality (0.0968) and transportation facility (0.0442). although the proposed model ranked only three distinct alternative service providers, the method is capable of comparing more than three providers at the cost of complexity. it needs to be emphasized that despite using a sound algorithm for systematic decision-making, care must be taken in the application of the anp approach. for example, in its application, the user has to compare the prospective providers on a number of pair wise comparison matrices. in these comparisons, the user must verify the capabilities of the providers and should not solely rely on the information given by the prospective providers. experts recommend that the user companies should evaluate the providers by what they have done and not by what they plan to do. although in this case the input to the pair wise comparison matrices is based on the responses to rllsp and visits of groups of managers to the sites of the service provider companies, the bias of the decision maker towards a particular provider cannot be ruled out. group decision-making help avoid this bias. in future research, brainstorming and sharing of ideas and insights can often lead to a better understanding of the issues. scenario building or the delphi method may also be used for the initial screening of alternatives available to the company and pair-wise comparisons. in the case of a delphi process, consensus may be reached by agreeing on the geometric means of individual judgments. in the absence of consensus, voting may also be conducted to arrive at a more acceptable value. compared to low-level enablers, consensus is more desirable for determinants and dimension at the higher level of the anp model. this is because of the higher global weights of determinants and dimensions in the anp model for the calculation of owi. a good number of software is available for anp and decision support systems and may also reduce the complexities in implementing the group decision-making. in light of the results obtained from this study, it may be noted that these results are valid only for the battery manufacturing company in its own decision environment, or for a similar type of industry and should not be generalized to establish the supremacy of one provider over others. further, the application of the proposed methodology may require significant time and resources from managers and decision-makers. yet, when seeking to invest in a long term logistics-outsourcing contract that can potentially reach billions of rupees, a structured analysis, which is provided by this methodology, may help to reduce the risk of poor investment decisions. ijahp article: jayant/selection of reverse logistics service provider (rlsp) using analytic network process (anp): a case study of an automotive company international journal of the analytic hierarchy process 155 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.374 8. summary and conclusions the analytic network process (anp) framework serves as a tool for making a strategic decision as it is related to the selection of the best option out of a finite set of alternatives with the feedback consideration. in this paper, we proposed a decision support methodology to a battery manufacturing company for selection of a logistics service provider. in the current integrated supply chain environment, logistics needs consist of many distinct but interrelated services, each with its own set of requirements and constraints. the authors argued that the selection of logistics partner(s) should not be broken down into a set of stand-alone selections which is the approach traditionally pursued. the selection should be evaluated holistically at the integrated supply chain level, both strategically and tactically, thus requiring an approach that is capable of addressing many interdependent and intangible elements. the paper also provides for a review of the issues which influence the selection of a provider. the anp approach, as a part of this methodology, not only leads to a logical result but also enables the decisionmakers to visualize the impact of various criteria in the final result. the anp approach is capable of taking into consideration both qualitative and quantitative criteria. similar anp-based models may also be developed in other contexts as well. but, as the development and evaluation of these models demand significant time and effort from the decision-makers in the formation of pair wise comparison matrices, these should only be used for long-term strategic decisions where the investments made in the lengthy and cumbersome process of decision making are recoverable. further, though the technique is computationally intensive, the benefits of risk reduction will outweigh the cost and time. using this formulation, decision makers can easily test a number of what-if scenarios. for future research, it would be worthwhile to implement the anp model with the delphi approach for a set of decision makers. such a research endeavor could be used to validate the effectiveness of the anp model. more importantly, managerial implications can be empirically derived regarding the selection of logistics service providers. such research should include a comprehensive sensitivity analysis to examine the significance of individual attributes to the selection decision. it is also worthwhile to compare the proposed anp approach with other evaluation approaches. the anp approach presented in this project has a few limitations as well. for example, the model result efficiency is dependent on the inputs provided by the group of managers in the xyz limited company. the possibility of bias may not be ruled out in the decision-maker towards any particular provider while applying this model. therefore, a team of concerned managers of the company should be constituted for making pair-wise comparison. moreover, the formation of pair wise comparison matrices is a time-consuming and tedious task. inconsistency and human error may be reduced by using software in calculating the pair wise comparison of matrices. ijahp article: jayant/selection of reverse logistics service provider (rlsp) using analytic network process (anp): a case study of an automotive company international journal of the analytic hierarchy process 156 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.374 references andersson d, norrman a. (2002). procurement of logistics services—a minute’s work or a multi-year project. european journal of purchasing and supply management, 8, 3–14. doi:10.1016/s0969-7012(01)00018-1 arvind jayant (2012). evaluation of 3pl service provider in supply chain management: an analytic network process approach, the international journal of business insights and transformation' (ijbit), 6(2), 78-82. banrodt, k. b. and davis, f. w. (1992).the evolution to service response logistics. international journal of physical distribution & logistics management, 22( 9), 3-10. doi: http://dx.doi.org/10.1108/eum0000000000422 bhatnagar r, sohal as, millen r. (1999). third party logistics services: a singapore perspective. international journal of physical distribution and logistics management, 29(9), 569–87. doi: http://dx.doi.org/10.1108/09600039910287529 boyson s, corsi t, and dresner m, rabinovich e. (1999). managing third party logistics relationships: what does it take? journal of business logistics, 20(1), 73–100. bradley p. (1994). what really matters? purchasing, 66–71. durvasula, s., lysonski, s., mehta, s. (2002), understanding the interfaces: how ocean freight shipping lines can maximize satisfaction, industrial marketing management, 31, 491-504. doi:10.1016/s0019-8501(01)00201-2 gattorna jl, walters dw. (1996). managing the supply chain: a strategic perspective. london: macmillan. doi: 10.1007/978-1-349-24841-4_13 gibson, b. j., rutner, s. m., keller, s. b. (2002). shipper-carrier partnership issues, rankings and satisfaction, international journal of physical distribution and logistics management, 32(.8), 669-681. doi: http://dx.doi.org/10.1108/09600030210444917 hertz s, alfredsson m. (2003). strategic development of third party logistics providers. industrial marketing management, 32(2), 139–49. doi:10.1016/s0019-8501(02)00228-6 jayant, p. gupta, s. k. garg (2011). an application of analytic network process to evaluate supply chain logistics strategies. international journal of the analytical hierarchy process, 3(2), 83-98. doi: http://dx.doi.org/10.13033/ijahp.v3i2.76 jayant, p. gupta, s. k. garg (2012). closed loop supply chain for spent batteries: a simulation study. international journal of advanced manufacturing systems, 3(1), 89-99. jayant, v. paul, u. kumar (2014). application of analytic network process (anp) in business environment: a comprehensive literature review. international journal of research in mechanical engineering & technology, 4(3), 29-37. http://dx.doi.org/10.1016/s0969-7012(01)00018-1 http://dx.doi.org/10.1016/s0019-8501(01)00201-2 http://dx.doi.org/10.1108/09600030210444917 http://dx.doi.org/10.1016/s0019-8501(02)00228-6 http://dx.doi.org/10.13033/ijahp.v3i2.76 ijahp article: jayant/selection of reverse logistics service provider (rlsp) using analytic network process (anp): a case study of an automotive company international journal of the analytic hierarchy process 157 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.374 jayaraman, v., r. a. patterson, r.a., rolland e. (2003). the design of reverse distribution networks: models and solution procedures. european journal of operational research, 150, 128–149. doi:10.1016/s0377-2217(02)00497-6 jharkharia, s. and shankar, r. (2007). selection of logistics service provider: an analytic network process (anp) approach, omega, 35(3), 274-289. doi:10.1016/j.omega.2005.06.005 kent, j. l. and flint, d. j. (1997). perspectives on the evolution of logistics thought, journal of business logistics, 18(2), 15-29. langley cj, allen gr, tyndall gr. (2002). third-party logistics study 2002: results and findings of the seventh annual study. illinois, usa: council of logistics management publications. langley cj, allen gr, tyndall gr. (2003). third-party logistics study 2003: results and findings of the eighth annual study. illinois, usa: council of logistics management publications. lieb r. c. (1992) the use of third-party logistics services by large american manufacturers, journal of business logistics, 13(2), 29-47. doi:10.1080/13675560110114270 lynch cf. (2000). logistics outsourcing: a management guide. illinois, usa: council of logistics management publications. doi: 10.1002/j.2158-1592.2001.tb00012.x maltz a b. (1995). why you outsource dictates how. transportation and distribution, 73–80. meade l m, sarkis j. (1998). strategic analysis of logistics and supply chain management systems using the analytic network process. logistics and transportation review, 34(2), 201–15. doi:10.1016/s1366-5545(98)00012-x meade, l. m., & sarkis, j. (1999). analyzing organizational project alternatives for agile manufacturing processes: an analytic network approach. international journal of production research, 37(2), 241–261. doi:10.1080/002075499191751 murphy, p. r., daley, j. m., dalenberg, d. r. (1992). profiling international freight forwarders: a benchmark, international journal of physical distribution and logistics management, 22, 35-41. doi: http://dx.doi.org/10.1108/09600039210010379 razzaque ma, sheng cc. (1998). outsourcing of logistics functions: a literature survey. international journal of physical distribution and logistics management; 28(2), 89– 107. doi:http://dx.doi.org/10.1108/09600039810221667 saaty tl. (1996). decision making with dependence and feedback: the analytic network process. pittsburgh, pa: rws publications. http://dx.doi.org/10.1016/s0377-2217(02)00497-6 http://dx.doi.org/10.1016/j.omega.2005.06.005 http://dx.doi.org/10.1016/s1366-5545(98)00012-x http://dx.doi.org/10.1108/09600039210010379 ijahp article: jayant/selection of reverse logistics service provider (rlsp) using analytic network process (anp): a case study of an automotive company international journal of the analytic hierarchy process 158 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.374 saaty tl. (1980). the analytic hierarchy process. new york, ny: mcgraw-hill. doi:10.1080/00137918308956077 sarkis j. (1998). evaluating environmentally conscious business practices. european journal of operational research, 107, 159–74. doi:10.1016/s0377-2217(97)00160-4 simchi-levi d, kaminsky p, simchi-levi e. (2000). designing and managing the supply chain. singapore: irwin mcgraw-hill. doi: http://dx.doi.org/10.1108/ijqrm.2000.17.7.812.1 stock gn, greis np, kasarda jd. (1998). logistics strategy and structure—a conceptual framework. international journal of operations and production management, 18(1), 37– 52. doi: http://dx.doi.org/10.1108/01443579810192772 tam mcy, tummala vmr. (2001). an application of the ahp in vendor selection of a telecommunications system. omega: international journal of management science, 29(2), 171–82. doi:10.1016/s0305-0483(00)00039-6 thompson tj. (1996). an analysis of third party logistics and implications for usaf logistics. unpublished master’s thesis, air force institute of technology/gtm/lal/96s-15, wright-patterson air force base, ohio. virender, p. jayant, a. (2014). analytical network process (anp) in selection of green supplier: a case study of automotive industry, international scientific journal on science engineering & technology, 17(5), 453-465. http://dx.doi.org/10.1016/s0377-2217(97)00160-4 http://dx.doi.org/10.1108/ijqrm.2000.17.7.812.1 http://dx.doi.org/10.1016/s0305-0483(00)00039-6 ijahp article: jayant/selection of reverse logistics service provider (rlsp) using analytic network process (anp): a case study of an automotive company international journal of the analytic hierarchy process 159 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.374 appendix influence matrices with respect to act determinants table 12 pair-wise comparison matrix for enablers under acceptable cost, customer perspective and customer service act/cp cs csf cqc e-vector csf 1 9 0.91 cqc 1/9 1 0.09 table 13 table 14 table 15 pair-wise comparison matrix for enablers under acceptable cost, customer perspective and customer satisfaction act/cp csf cqc cs e-vector cqc 1 5 0.83 cs 1/5 1 0.17 pair-wise comparison matrix for enablers under acceptable cost, customer perspective and customer queries and complaints act/cp cqc cs csf e-vector cs 1 1/5 0.17 csf 5 1 0.83 pair-wise comparison matrix for enablers under acceptable cost, internal business perspective and information technology act/ibp it otd nte e-vector otd 1 3 0.75 nte 1/3 1 0.25 ijahp article: jayant/selection of reverse logistics service provider (rlsp) using analytic network process (anp): a case study of an automotive company international journal of the analytic hierarchy process 160 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.374 table 16 table 17 pair-wise comparison matrix for enablers under acceptable cost, internal business perspective and on time delivery act/ibp otd it nte e-vector it 1 1/7 0.13 nte 7 1 0.87 pair-wise comparison matrix for enablers under acceptable cost, internal business perspective and new technologies act/ibp nte it otd e-vector it 1 1/3 0.25 otd 3 1 0.75 ijahp article: sapkota, pokharel, pandey /identifying the most suitable sustainable energy system for nepal using analytic hierarchy process international journal of the analytic hierarchy process 353 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.411 identifying the most suitable sustainable energy system for nepal using analytic hierarchy process *prabal sapkota 1 school of engineering, kathmandu university dhulikhel, kavre, nepal prabal@ku.edu.np martina pokharel freelancer suryavinayak, bhaktapur, nepal pokharelmartina@gmail.com madhav p. pandey school of engineering, kathmandu university dhulikhel, kavre, nepal mpp@ku.edu.np abstract the issue of climate change and energy crisis can be resolved by the advancement of sustainable energy systems. the process of energy development in developing countries with a poor economy is complicated. one has to consider numerous factors and subfactors which are important for the system to be acceptable to the multiple stakeholders. involvement of multiple entities makes the process a real case of multi criteria decision making (mcdm). this study deals with identification of various stakeholders, factors, sub-factors and alternatives associated with sustainable energy selection in nepal. the analytic hierarchy process (ahp) has been used as a tool to deal with the mcdm problem in this research. prioritization of alternatives has been obtained with the application of ahp. further, the analysis has also been done based on the perception of multiple stakeholder groups. the result shows that politicians are the most important (61%) among the stakeholders for the development of sustainable energy in nepal. among the alternatives, the majority of the respondents believe that biogas should be given the highest priority. keywords: ahp; mcdm; sustainable; energy systems 1 acknowledgement:this research is based on the research financed under the climate change research grants program implemented by the nepal academy of science and technology. the program is part of the mainstreaming climate change risk management in development project, a component of nepal’s pilot program for climate resilience and is executed by the ministry of population and environment (nepal), financed by the climate investment funds, administered by the asian development bank with technical assistance from icem, metcon and aptec. ijahp article: sapkota, pokharel, pandey /identifying the most suitable sustainable energy system for nepal using analytic hierarchy process international journal of the analytic hierarchy process 354 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.411 1. introduction the increasing demand of energy and the depletion fossil fuel resources have put pressure on researchers to search for alternative energy sources. the current supply of energy is not enough to meet the current demand. the major source of the current supply of energy is from fossil fuel. the use of fossil fuel as a major source of energy comes with two important issues: sustainability and climate change. the first issue is concerned with the long term use of fossil fuels as it is depleting and is non-renewable. for the last few decades, researchers have speculated that if the current trend of fossil fuel use continues it may not last for very long since the deposits of fossil fuels are depleting and will be gone in no time. the other issue is concerned with the changes in environment which results from the increasing trend of emissions that are harmful to the environment. people associated with the development of alternative energy systems are trying to address both of these issues (sapkota & kim, 2009). energy consumption patterns are always changing both in terms of amount and types of resources used. this pattern depends on several factors which include technological innovation which has the major impact backed up by the resource availability (o’connor, 2010). further, the process is gradual requiring efforts and contributions from several sectors. developed countries are able to meet the increasing demand with increasing energy production. whereas, developing countries are lagging behind in their ability to meet the increasing demand with adequate supply. there are several reasons associated with this inability to keep up. the main reasons include lack of development policy, an insufficient economy, lack of resources and political instability. countries like nepal with zero petroleum resources are always in search of alternative sources which could reduce their dependency on fossil fuel. in nepal, the residential sector is the major user of energy, where energy is used for lighting and cooking. the major use of energy is from fossil fuel which is considered responsible for greenhouse gas emissions (ghg), and comes in the form of vehicle fuel, cooking gas and small industries. out of the total ghg, more than two-thirds of the emissions come from the residential sector (moste, 2014). although developing countries seem to be less concerned with climate change, they cannot stay away from adopting necessary measures to mitigate it from their side. even though they are not major contributors to the climate change, they are among the sufferers of the calamity induced by climate change. efforts for climate change mitigation from the developing world could be possible with a paradigm shift with regards to energy use, mainly transition from fossil fuel to clean energy (ipcc, 2011; edenhofer et al., 2012). energy development in nepal has always been slow, and the current energy generation covers only one-third of the total demand. a major portion of the country does not have an access to electricity (nea, 2014). although multiple renewable energy resources are available, due to a low economy all types of energy systems cannot be developed together. further, it also becomes very important to consider several criteria and sub criteria before making any decisions. furthermore, the failure of past energy projects shows evidence of the impact of stakeholders in the development of energy systems. the government seems to be perplexed in choosing the best among the alternatives, as all alternatives seem to be important and feasible. it becomes very important to prioritize ijahp article: sapkota, pokharel, pandey /identifying the most suitable sustainable energy system for nepal using analytic hierarchy process international journal of the analytic hierarchy process 355 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.411 these alternatives based on people’s need, resource availability, technical capability and environment friendliness. furthermore, it is also crucial to identify all the influencing actors that have major impacts on the development of energy systems in nepal. it is a real case of multi criteria decision making (mcdm) and the analytic hierarchy process (ahp) has already been proven as an appropriate tool in such situations (saaty, 2008). this research adopts ahp and identifies and prioritizes major factors, sub-factors, alternatives and actors in sustainable energy development in nepal. 2. rationale and objective of the research for the last decade, nepal has been facing a huge energy crisis. there is an electric power cut of up to eighteen hours per day in the dry seasons. further, the supply of petroleum products including cooking gas (lpg) has worsened in recent times. practically, the country has zero petroleum resources. although nepal is considered very rich in natural resources, there is not much interest in exploring those resources. up to now, the major focus has been on hydropower, and some interest has been given to biogas and solar power. there are several unidentified resources other than hydropower, and several unexplored alternatives. similarly, the geographical and socio-economic conditions of the country have been ignored. identifying the best sustainable source of energy in the nepalese context is the present need of the nation. nepal should focus on developing alternatives to fossil fuel from among the resources available in the country. nepal, being a developing country with a low economy, cannot develop all the energy systems at the same time. it is very important to prioritize based on the importance these energy sources have in people’s lives and their potential to mitigate climate change. this becomes a complex process, as multiple considerations have to be taken into account. one has to consider various criteria (factors), sub-criteria (sub-factors) and potential alternatives before coming to a conclusion of any kind. further, it becomes very crucial to analyze the influence of actors (stakeholders) during the process, and this adds to the complexity in the process. this research has been designed to assist policy makers in developing policies for sustainable renewable energy systems development in nepal. this research provides a framework for renewable energy selection and development. this framework addresses major criteria, sub-criteria and feasible alternatives that need to be considered during the energy selection process, and also highlights the perception of various stakeholders towards energy requirements and environmental consequences. further, it also identifies the major stakeholders that become active during the process. the research further prioritizes possible alternatives, factors, sub-factors and stakeholders who are most responsible for the development of energy sectors. it follows the ahp approach, as this is clearly a multiple criteria decision-making scenario. the major objective of this research is to develop a decision model for sustainable energy selection in nepal. 3. literature review ijahp article: sapkota, pokharel, pandey /identifying the most suitable sustainable energy system for nepal using analytic hierarchy process international journal of the analytic hierarchy process 356 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.411 the increasing demand for energy is being fulfilled by heavy exploitation of fossil fuel deposition. this has created two major concerns: sustainability and climate change. no analysis suggests that there are enough deposits of fossil fuels to even fulfill the current demand. the shortage of energy will have a major impact on developing countries as their economy and development largely depend on the supply of energy. climate change is believed to be caused by an increased level of greenhouse gases (ghg) and the use of fossil fuel produces, co2, one of the main ghg agents. development of renewable and sustainable energy sources can address both the issue of sustainability and climate change. at present, the research on renewable and alternative energy sources is at a peak. the proposed source should be sustainable, affordable and adoptable and also produce environmentally benign byproducts (sapkota & kim, 2009; sapkota & kim, 2010). 3.1 factors (criteria) associated with energy selection there has been a series of innovations in the process of energy development and use in the last few centuries. this transition of the energy system has been a gradual process with the involvement of numerous factors. it is very important to identify all the factors which could play a significant role in the development of any energy systems. an ipcc report on renewable energy sources and climate change mitigation emphasized scientific, technological, environmental, economic and social aspects of the contribution of renewable energy sources to the mitigation of climate (ipcc, 2011). selection criteria for the renewable energy systems are found to differ from one research to the other. similarly, factors, sub-factors, actors and the energy system choices are also different according to the geographical location in which the research has been done. a study carried out in pakistan by amer & diam (2011) used multiple factors: technical, economic, social, environmental and political and wind energy, solar photovoltaic, solar thermal and biomass energy options as the alternatives in the decision model. further, wang et al. (2009) summarized the criteria of energy supply systems from technical, economic, environmental and social aspects. similarly, abbasi et al. (2010) discussed the environmental impacts on biomass energy and akella et al. (2008) discussed renewable energy system’s impact on social, economic and environmental, and other factors. 3.2 renewable energy alternatives renewable energy technologies (rets) as well as biofuels have been accelerating rapidly during the past decades, both in technology performance and cost competitiveness (arent et al., 2011). globally, biofuel contributes about 12% of the primary energy supply (ipcc, 2001). biogas systems can support sustainable communities by reducing methane emissions, improving water quality, producing a local source of renewable heat, electricity and fuel, and strengthening the local economy by reducing energy costs and generating revenue. countries like nepal, where biogas production and utilization has been practiced for the last few decades can accelerate the process of biogas production from domestic waste, manure and agriculture waste to reduce the greenhouse gas that causes climate change and produce cheap sustainable energy for the local people. biogas systems have the potential to capture methane that would escape into the atmosphere and utilize it to create energy. other byproducts of biogas systems include non-energy products such as nutrient rich soil amendments, pelletized and pumpable fertilizers, and even feedstock for plastics and chemicals (usda, 2014). gas-to-liquid (gtl) technology enables conversion of ijahp article: sapkota, pokharel, pandey /identifying the most suitable sustainable energy system for nepal using analytic hierarchy process international journal of the analytic hierarchy process 357 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.411 biogas into other energy carriers with higher energy density, facilitating fuel distribution (moghaddama et al., 2015). research works carried out in nepal found several renewable energy technologies in practice consisting of biogas, improved water mill (iwm), stand-alone micro-hydro plants, mini-grid micro-hydro plants, solar pv home systems, mud-improved cooking stoves and metal-improved cooking stoves ( aepc, 2013; sapkota ,2014). another renewable energy source that researchers are considering these days for the replacement of gasoline is biodiesel. for a country like nepal with zero petroleum resources, development of biodiesel as a substitute to gasoline not only solves the energy crisis but also reduces the greenhouse gas emissions. converting biomass feedstock to biodiesel or bio-ethanol is an environmentally-friendly process; so is using biofuel for transportation. because of their compatibility with the natural carbon cycle, biodiesel offers the most beneficial alternative for reducing greenhouse gases from the transportation sector (nrel, 1999). various renewable energies such as solar, hydropower, biogas, wind and geothermal energies are already in practice, and some are in the phase of commercialization (sapkota & kim, 2009; sapkota & kim, 2010). a special report on renewable energy sources and climate change mitigation presented six renewable energy sources to the mitigation of climate change: bioenergy, direct solar energy, geothermal energy, hydropower, ocean energy and wind energy (edenhofer et al, 2012). yadoo & cruickshank (2012) found that renewable energy mini-grids powered by biomass gasifiers or micro-hydro plants appear to be the favored option due to their lower levelized costs. yuksel (2008) highlighted the development of renewable energy sources like hydropower, biomass, wind and geothermal to mitigate climate change and for the sustainable development of turkey. similarly, cristóbal (2011) used wind power, hydroelectric, solar thermal, solar thermoelectric, photovoltaic, biomass, biogas and biofuels as alternatives in spain. bhattarai & fujiwara (1995) worked on evaluation of appropriate scale of hydropower development for nepal using ahp with the inclusion of actors (hurdles) that may exist during the process. 3.3 barriers to renewable energy development for the development of renewable and sustainable energy technologies, several constraints have to be faced especially in developing countries like nepal. luthra et al. (2015) have reported barriers in indian contexts and categorized these into seven dimensions which include economical & financial; market; awareness & information; technical; ecological and geographical; cultural & behavioral; and political & government issues. several reviews have been done on potential technological, economic, social or public barriers to renewable energy investment (richards et al., 2012). similarly, wang et al. (2010) highlighted three barriers of using clean development mechanism (cdm) to promote renewable energy in china as the dilemma of additionality, lower proportional cers (certified emission reduction credits) revenues on the investment and lack of incentives for renewable technology transfer. ijahp article: sapkota, pokharel, pandey /identifying the most suitable sustainable energy system for nepal using analytic hierarchy process international journal of the analytic hierarchy process 358 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.411 wee et al. (2012), found conversion cost, location selection, distribution network and other factors like capital investment, operation and maintenance costs, and capacity factor costs to be the barriers. whereas, mirza et al.(2009) broadly classified barriers to renewable energy development as policy and regulatory barriers, institutional barriers, fiscal and financial barriers, market-related barriers, technological barriers and information and social barriers. 3.4 multi-criteria decision making in renewable energy selection even though nepal is rich in natural resources, it is facing an energy crisis. the current energy scenario of nepal shows that it should immediately produce more energy to meet the growing energy demands of the country. choosing the best energy system for a country like nepal has always been problematic. in order to make a successful choice, selection should consider several perspectives including barriers (hurdles). there is always a need of multi-dimensional analysis. existence of multiple stakeholders and numerous parameters and too many alternatives creates a multi criteria decision-making (mcdm) problem, and should be commonly formulated by stakeholders in a complex decision-making process. the multi-criteria decision method (mcdm) has become increasingly popular in decision-making for sustainable energy because of the multi-dimensionality of the sustainability goal and the complexity of socio-economic and biophysical systems (theodorou et al. 2010; wang 2009). mcdm techniques are gaining popularity in sustainable energy management. the techniques provide solutions to the problems involving conflicting and multiple objectives. several methods based on weighted averages, priority setting, outranking, fuzzy principles and their combinations are employed for energy planning decisions (pohekar et al. 2004). it is observed that the most popular tool for mcdm that has been used globally by researchers is the analytical hierarchy process (ahp) (pohekar et al. 2004; ahmad et al. 2014; amer & diam, 2011). ahp uses a pair-wise comparison approach between the two conflicting scenarios, and there is a provision to verify the consistency. ahp is capable of breaking complex problems into smaller parts that can be easily handled by human intelligence (saaty, 2008). ahp has been used in selection, evaluation and performance comparison between the different power production options (kabir et al.2003; polatidis et al. 2006; mateo, 2012, terrados et al., 2010; daniel et al., 2010). bhattarai & fujiwara (1995) worked on an evaluation of the appropriate scale of hydropower development for nepal using ahp, and nachtnebel & singh (2015) used ahp in prioritizing nepalese hydropower development and identified inexperience, inadequate funding and political stability as development setbacks. apart from this there has not been much work done in nepal regarding energy selection and planning. all the past works mentioned above were primarily based on the energy selection focusing on technological advancement and availablility of resources. actors (hurdles) that may exist during the process have not been considered except in the study done by bhattarai & fujiwara (1995). this work tries to evaluate all the possible energy sources based on those actors who could influence the energy selection process. ijahp article: sapkota, pokharel, pandey /identifying the most suitable sustainable energy system for nepal using analytic hierarchy process international journal of the analytic hierarchy process 359 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.411 3.5 probable energy systems for nepal several sustainable energy options are available to meet the increasing nepalese energy demand. some of the systems are already in practice whereas others could be feasible in nepal. some of the energy options for nepal are listed below. hydropower: hydropower is the most developed energy system in nepal. although nepal has tremendous potential of hydroelectricity, less than one percent of the total capacity has been generated until now (nachtnebel & singh, 2015). solar: in recent times, the use of solar energy either in the form of a solar home system or water heating system has increased. abundantly available sunlight can be tracked to generate this energy. nepal has already proven to have good potential in energy generation from sun. wind: in nepal, the popularity of wind energy is growing and the government has shown good interest in harnessing energy from the wind. studies show that nepal has a good prospect of electricity generation from wind. biogas: energy production from cattle dung is already in practice in nepal. this has already proven an effective energy source in rural populations. biogas can be a good substitute to liquid petroleum gas (lpg). further, it can be generated from municipal organic waste. biodiesel: this can be a good alternative to fossil petroleum products. nepal has not developed any such product yet, but slowly non edible oil sources like jatropa which are easily available in nepal have become a concern of research for energy specialists. this technology could be the most important energy system for countries like nepal with zero petroleum resources. fuel cell: in developed countries, the research and development of fuel cells is at a peak. because of its wide range of applicability, fuel cell could be the future energy source of the world. fuel cell cars are already in practice in developed countries. countries like nepal with huge water potential can convert water to useful hydrogen energy for fuel cell. nuclear: this has already been the major source of energy generation in developed countries. if nepal wishes to meet the ever increasing energy demand of the country, it should move towards nuclear energy at some point. although nuclear energy has some consequences, this can be one of the best solutions to the acute power shortage of the country. the current energy scenario of nepal shows that nepal should immediately develop more energy production to meet the growing energy demand of the country. nepal should not just focus on the development of electricity but also address the production of liquid petroleum products for vehicles and means that can provide alternatives to the cooking gas (lpg). ijahp article: sapkota, pokharel, pandey /identifying the most suitable sustainable energy system for nepal using analytic hierarchy process international journal of the analytic hierarchy process 360 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.411 4. methodology 4.1 research design this study has adopted four tools in addition to literature review which include a questionnaire, delphi method, one on one interviews and model development using ahp. the steps followed during the research work are shown in the research framework in figure 1. figure 1. research framework a group of experts from four different areas has been formed. the group consists of energy experts (2); an energy project manager (1); with social science/political background (1); and an electrical engineer (1). all the group members have master’s degrees and two years of work experience in the field of their expertise. the initial questionnaire to identify actors, factors, sub-factor and alternatives has been developed based on the opinion of these experts and the literature. the developed questionnaire was administered among educated nepalese people. thirty-five valid responses were collected during this process. the responses were analyzed and further literature consulted. based on the result of the preliminary survey and literature review, factors, sub-factors, alternatives and actors were identified and a hierarchy model based on ahp theory was developed. the hierarchy model developed is shown in figure 2. the weight that has been given to the actors by the respondents has been further used during sensitivity analysis of the actors. a pair-wise comparison questionnaire was developed based on the hierarchy model. this questionnaire was administered in the form of an interview among groups of all the identified actors. 4.2 model development a hierarchy model based on ahp theory was developed during the course of the research as shown in figure 2. here, the goal “sustainable energy system for nepal” is at the top at level 1. the second level (level 2) has five actors; these are the people (or agencies) that have a direct or indirect relationship to the energy system development in nepal. the selection of these actors was done based on the expert’s opinion, literature review and the view of people whose responses were collected during the first stage of data collection. the actors identified for the achievement of the goal are donor agencies, politician/political parties, end users (general people), energy experts and business people ijahp article: sapkota, pokharel, pandey /identifying the most suitable sustainable energy system for nepal using analytic hierarchy process international journal of the analytic hierarchy process 361 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.411 (people associated with the energy business). all of these actors are found to have influence in the past during the time of energy project development and implementation in nepal. the third level of the hierarchy has five factors that are important in classifying an energy system as a sustainable system. these are environmental factors, technological factors, socio-economic factors, cultural factors and future prospects. all the factors have subfactors which are kept at level 4. table 1 shows the list of factors and sub-factors that have been identified during the course of the research. the identification of these factors and sub-factors has also been done based on expert opinion, literature review and responses received from the respondents during the first stage of the questionnaire. ghe: greenhouse gas emission, oi: other impacts, fr: financial requirements, jb: job opportunity, tm: technology maturity, tnl: technology known to local actors , eco: economy, sus: sustainability, fr: financial requirements figure 2. hierarchy model alternatives identification was done based on the expert opinions and literature review. the majority of the identified alternatives are in practice in nepal. a few of the alternatives like solar and biogas are at a high level of development and use whereas, other sources like wind and biodiesel are in the developing stages. there has been never been any nuclear energy system in nepal. hydropower has not been included as one of the alternatives. there is still a dilemma regarding the consideration of larger hydropower operations as renewable and sustainable because of their impact on ecology, displacement of large number of human inhabitants, and in light of global warming, the potential of hydropower may be unpredictable due to change in rainfall patterns and melting of himalayan glaciers. micro hydro can be a potential alternative, but there is still a problem regarding grid integration and connection. ijahp article: sapkota, pokharel, pandey /identifying the most suitable sustainable energy system for nepal using analytic hierarchy process international journal of the analytic hierarchy process 362 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.411 table1 identified factors and sub-factors factors sub-factors explanation of factors/ sub-factors environmental ghe greenhouse gases emission: carbon dioxide, methane etc emission. oi other impacts: noise, health hazards, negative ecological impact etc. technological tnla technology known to the local actors: how easy the system and technology is to operate at local level, whether the technology is well known by the locals. tm technology maturity: how mature is the technology, what about the scaling up, economies of scale etc? safe safety in operation: how safe is the system to operate, what sorts of risks are possible? socio-economic fr financial requirements: how much is the initial investment and operation & maintenance cost. job job prospect: what about the employment opportunity with the development of the desired system? can it provide job to local people? future prospect eco economic: what about the economic benefit to the people, how much will be their saving and other benefits? sus sustainability: how sustainable is the system? how long can the system be operated economically? what about the resources needed? culture is the system compatible with local custom, culture, tradition and even policies? 4.3 tools and technique the analysis of this research was done based on the theory of the analytical hierarchy process (ahp). all the calculations were done using excel 2007. a geometric mean of the responses obtained from the pair-wise comparison questionnaire was calculated. this mean value was further used to calculate consistency ratio, priority vector, row average and finally the prioritization of actors, factors, sub-factors and alternatives. an inconsistency ratio has been calculated for all the data obtained. the data with an inconsistency ratio less than 10% was used for further calculations. 4.4 data collection the total number of samples collected for the first stage of data collection was 35. the number of valid samples obtained during the second stage of data collection was 25, five from each group of actors. ijahp article: sapkota, pokharel, pandey /identifying the most suitable sustainable energy system for nepal using analytic hierarchy process international journal of the analytic hierarchy process 363 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.411 5. calculations and results the calculations were done for two different conditions separately. the first one was with inclusion of actors and the second one was without actors. 5.1 inclusion of actors the geometric mean of the group data was calculated, individually among the group of actors and collectively. the mean value obtained was arranged in a matrix form and further calculations were done. a sample of the matrix developed during the process is shown in the table 2. table 2 sample matrix developed the consistency of the developed model was checked and the overall inconsistency calculated is 4%. a calculation to identity the most important actors involved in keeping the goal of sustainable energy system development for nepal was done. a matrix developed for this work is shown in table 2. here, there are four actors so the size of the matrix is 4x4. further calculations were done using the value of the matrix and the obtained result is shown in the figure 3. figure 3. result of prioritization of actors ijahp article: sapkota, pokharel, pandey /identifying the most suitable sustainable energy system for nepal using analytic hierarchy process international journal of the analytic hierarchy process 364 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.411 the majority of the respondents believe that politicians (61%) are mainly responsible for the sustainable energy development in nepal followed by donors (16.7%), end users (11.1%) and the other two actors, business people and energy experts are believed to be equally responsible at 5.6% each. similar calculations were done for level 3 factors, level 4 sub-factors and level 5 alternatives. in the case of alternatives, the matrix becomes 5x5. the result obtained for the prioritization of alternatives is shown in figure 4. figure 4. result of prioritization of alternatives the results show that the majority of the respondents believe that biogas should be developed as a sustainable energy system for nepal (37.3%) followed by solar/ pv (25.1%), biodiesel (19.1%), wind (10.4) and finally nuclear (8.1%). 5.2 results with exclusion of actors a second case with exclusion of actors was developed and analyzed. figure 5 shows a model without actors. figure 5. hierarchy with exclusion of actors ijahp article: sapkota, pokharel, pandey /identifying the most suitable sustainable energy system for nepal using analytic hierarchy process international journal of the analytic hierarchy process 365 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.411 the result with exclusion of actors shows that the most important factors to be considered while selecting sustainable energy for nepal are socio-economic ( 36.24%) followed by future prospect (30.63%), environmental (18.02%), technological (11.61%) and culture (3.5%). furthermore, respondents believe that job creation (24.02%) should be the prime focus followed by economic wellbeing (23.78). the detailed results of the case are shown in figure 6. tm: technology maturity, tnl: technology known to local, financial req: financial requirement figure 6. factors and sub-factors identification excluding actors there was only a slight change among alternatives even after the exclusion of actors. the preference of biogas (30%) is still at the highest priority followed by solar (22.9%) and biodiesel (22.8%). the results for alternatives preferences are shown in figure 7. figure 7. alternatives preference with exclusion of actors ijahp article: sapkota, pokharel, pandey /identifying the most suitable sustainable energy system for nepal using analytic hierarchy process international journal of the analytic hierarchy process 366 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.411 5.3 discussion and sensitivity analysis the results that were obtained were unexpected for most of the respondents. they were expecting solar/pv to be the first priority among all the alternatives instead of biogas. further analysis showed that the results diverted towards the view of politicians as they have been given the highest priority among the actors (61%). in the view of politicians, biogas should be the sustainable energy option for nepal. because of this, the decision will be focused predominantly on the view of the politicians. further, analysis showed that politicians put socio-economic factors as the top priority, in this case 48.5 %. among socio-economic factors, the highest priority was found to be given to job opportunity. politicians believe that development of biogas can give an energy solution to the country in a sustainable manner. the analysis from the perspective of political people is shown in figure 8. figure 8. factor, sub-factor and alternative preference from the actor: politician the effects of other actors and factors were analyzed. apart from energy experts, there was no significant change in the outcome when other actors were changed. when energy expert were given a major priority of 81%, biodiesel becomes the first choice followed by biogas. this scenario is shown in the figure 9. ijahp article: sapkota, pokharel, pandey /identifying the most suitable sustainable energy system for nepal using analytic hierarchy process international journal of the analytic hierarchy process 367 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.411 figure 9. preference of alternative from the view of actor: energy expert furthermore, when environmental factors were given the highest priority (80%), solar energy becomes the first priority followed by biogas and biodiesel. this situation is shown in figure 10. figure 10. analysis with 80% priority on environmental factor the result represents the need of nepalese people from all the sectors. the current need of the people is energy for cooking and lightning. furthermore, people reckon that the socio-economic factor should be of highest priority among factors while selecting sustainable energy systems for nepal. among sub-factors, job creation was rated as the ijahp article: sapkota, pokharel, pandey /identifying the most suitable sustainable energy system for nepal using analytic hierarchy process international journal of the analytic hierarchy process 368 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.411 most important. people have not given much concern to environmental factors as environment consequences are of less concern in developing countries like nepal. 6. conclusions the main objective of this research is to develop a framework for sustainable energy selection for nepal that could be useful for policy makers and researchers in designing and implementing sustainable energy systems. energy policy development is a difficult process, as multiple considerations have to be made. the process becomes more complex when there are constraints in resources and multiple alternatives exist. furthermore, in the case of developing countries like nepal the complexity of the selection process becomes more rigorous with the involvement of multiple actors who become active at various stages of project development. lessons learnt from the failure of past projects have already proven the strength of hurdles these actors may create in the projects. among the five most important actors identified, the majority of the respondents believe that politicians are most responsible for the development of energy systems in nepal. similarly, among the five important factors identified for the sustainability of energy systems, the respondents have rated socio-economic as the most important. furthermore, the majority of the respondents believe that biogas should be given priority as a sustainable energy system for nepal followed by biodiesel and solar, wind and nuclear. although, the finding of this research is based on the subjective judgment of a selected few people, the developed model will be useful and applicable in most cases. furthermore, this model will be very useful in conducting stakeholders and risk analysis during project development and implementation. ijahp article: sapkota, pokharel, pandey /identifying the most suitable sustainable energy system for nepal using analytic hierarchy process international journal of the analytic hierarchy process 369 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.411 references abbasi t. et al. (2010). biomass energy and the environmental impacts associated with its production and utilization. renewable and sustainable energy reviews, 14(3), 919– 937. doi: http://dx.doi.org/10.1016/j.rser.2009.11.006 aepc. (2013). study on role of renewable energy technologies in climate change mitigation and adaptation options in nepal. alternative energy promotion center, nepal. ahmad s. et al. (2014). selection of renewable energy sources for sustainable development of electricity generation system using analytic hierarchy process: a case of malaysia. renewable energy, 63, 458. doi: http://dx.doi.org/10.1016/j.renene.2013.10.001 akella a. k. et al. (2008). social, economical and environmental impacts of renewable energy systems. renewable energy, 34, 390–396. doi: http://dx.doi.org/10.1016/j.renene.2008.05.002 amer, m. & diam t. (2011). selection of renewable energy technologies for a developing county: a case of pakistan. energy for sustainable development, 15, 420. doi: http://dx.doi.org/10.1016/j.esd.2011.09.001 arent d.j. et al., (2011). the status and prospects of renewable energy for combating global warming. energy economics, 33, 584. doi: http://dx.doi.org/10.1016/j.eneco.2010.11.003 bhattarai, s. & fujiwara, o. (1995). evaluation of appropriate scale of hydropower development for nepal: analytical hierarchy process approach. infrastructure planning and management program, asian institute of technology. cristóbal, s.j.r. (2011). multi-criteria decision-making in the selection of a renewable energy project in spain: the vikor method. renewable energy, 36, 498. doi: http://dx.doi.org/10.1016/j.renene.2010.07.031 daniel j. et al. (2010). evaluation of the significant renewable energy resources in india using analytical hierarchy process. springer. doi: 10.1007/978-3-642-04045-0_2 edenhofer. o. et al. (2012). ipcc special report renewable energy sources and climate change mitigation. cambridge, united kingdom new york, usa: cambridge university press. ipcc. (2001). climate change, impacts, adaptation and vulnerability, report of the working group ii, uk: cambridge university press, 967. doi: http://dx.doi.org/10.1016/s0168-1923(03)00039-x ipcc. (2011). summary for policymakers. in: ipcc special report on renewable energy sources and climate change mitigation. cambridge, united kingdom and new york, ny, usa: cambridge university press. http://dx.doi.org/10.1016/j.rser.2009.11.006 http://dx.doi.org/10.1016/j.renene.2013.10.001 http://dx.doi.org/10.1016/j.renene.2008.05.002 http://dx.doi.org/10.1016/j.esd.2011.09.001 http://dx.doi.org/10.1016/j.eneco.2010.11.003 http://dx.doi.org/10.1016/j.renene.2010.07.031 http://dx.doi.org/10.1016/s0168-1923(03)00039-x ijahp article: sapkota, pokharel, pandey /identifying the most suitable sustainable energy system for nepal using analytic hierarchy process international journal of the analytic hierarchy process 370 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.411 kabir, a.b.m.z. et al. (2003). selection of renewable energy sources using analytic hierarchy process. isahp 2003, bali, indonesia. luthra, s. et al. (2015). barriers to renewable/sustainable energy technologies adoption: indian perspective. renewable and sustainable energy reviews, 41,762. doi: http://dx.doi.org/10.1016/j.rser.2014.08.077 mateo, j.r.s.c. (2012), multi-criteria analysis in the renewable energy industry. london: springer-verlag limited. doi: 10.1007/978-1-4471-2346-0 mirza u.k. et al. (2009). identifying and addressing barriers to renewable energy development in pakistan. renewable and sustainable energy reviews, 13, 927. doi: http://dx.doi.org/10.1016/j.rser.2007.11.006 moghaddama e.a. et al. (2015). energy balance and global warming potential of biogasbased fuels from a life cycle perspective. fuel processing technology, 132, 74–82. doi: http://dx.doi.org/10.1016/j.fuproc.2014.12.014 moste (2014), second national communication to united nations framework convention on climate change. ministry of science and technology, government of nepal. nachtnebel, h.p. & singh, r.p.(2015). prioritizing hydropower development using analytical hierarchy process (ahp) a case study of nepal. international journal of the analytic hierarchy process, 7(2), 313-336. doi: http://dx.doi.org/10.13033/ijahp.v7i2.253 nea (2014). annual report. nepal electricity authority. nrel (1999), biofuels: a solution for climate change, u.s. department of energy. o’connor p.a. (2009). energy transitions. boston university. pohekar s.d.et al. (2004). application of multi-criteria decision making to sustainable energy planning—a review. renewable and sustainable energy reviews, 8, 365. doi: http://dx.doi.org/10.1016/j.rser.2003.12.007 polatidis, h. h. et al. (2006). selecting an appropriate multi-criteria decision analysis technique for renewable energy planning. energy sources, 1,181–193. doi: http://dx.doi.org/10.1080/009083190881607 richards, g.et al. (2012). barriers to renewable energy development: a case study of large-scale wind energy in saskatchewan, canada. energy policy, 42, 691. doi: http://dx.doi.org/10.1016/j.enpol.2011.12.049 saaty, t. l. (2008). decision making with the analytic hierarchy process. international journal of services sciences, 1(1), 83-98. doi: http://dx.doi.org/10.1504/ijssci.2008.017590 http://dx.doi.org/10.1016/j.rser.2014.08.077 http://dx.doi.org/10.1016/j.rser.2007.11.006 http://dx.doi.org/10.1016/j.fuproc.2014.12.014 http://dx.doi.org/10.1016/j.rser.2003.12.007 http://dx.doi.org/10.1080/009083190881607 http://dx.doi.org/10.1016/j.enpol.2011.12.049 ijahp article: sapkota, pokharel, pandey /identifying the most suitable sustainable energy system for nepal using analytic hierarchy process international journal of the analytic hierarchy process 371 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.411 sapkota a. et al (2014). role of renewable energy technologies in rural communities’ adaptation to climate change in nepal. renewable energy, 68, 79. doi: http://dx.doi.org/10.1016/j.jiec.2009.01.002 sapkota,p. & kim, h.(2009). zinc–air fuel cell, a potential candidate for alternative energy. journal of industrial and engineering chemistry, 15, 445. doi: http://dx.doi.org/10.1016/j.jiec.2010.01.024 sapkota, p. & kim, h.(2010). an experimental study on the performance of a zinc air fuel cell with inexpensive metal oxide catalysts and porous organic polymer separators, journal of industrial and engineering chemistry, 16, 39. doi: http://dx.doi.org/10.1016/j.jiec.2010.01.024 theodorou s. et al. (2010). the use of multiple criteria decision making methodologies for the promotion of res through funding schemes in cyprus: a review. energy policy, 38, 7783. doi: http://dx.doi.org/10.1016/j.enpol.2010.08.038 terrados, j. et al. (2010). energy planning: a sustainable approach. spain: idea research group, university of jaén. usda. ( 2014). biogas opportunities roadmap. u.s. department of agriculture, u.s. environmental protection agency, u.s. department of energy. wang, j.j. et al. (2009). review on multi-criteria decision analysis aid in sustainable energy decision-making. renewable and sustainable energy reviews, 13, 2263. doi: http://dx.doi.org/10.1016/j.rser.2009.06.021 wang, q. et al. (2010). barriers and opportunities of using the clean development mechanism to advance renewable energy development in china. renewable and sustainable energy reviews, 14, 1989. doi: http://dx.doi.org/10.1016/j.rser.2010.03.023 wee h. m. et al. (2012). renewable energy supply chains, performance, application barriers, and strategies for further development. renewable and sustainable energy reviews, 16, 5451. doi: http://dx.doi.org/10.1016/j.rser.2012.06.006 yadoo, a. & cruickshank, h. (2012). the role for low carbon electrification technologies in poverty reduction and climate change strategies: a focus on renewable energy minigrids with case studies in nepal, peru and kenya. energy policy, 42, 591. doi: http://dx.doi.org/10.1016/j.enpol.2011.12.029 yuksel, i. (2008). global warming and renewable energy sources for sustainable development in turkey. renewable energy, 33, 802. doi: http://dx.doi.org/10.1016/j.renene.2007.05.040 http://dx.doi.org/10.1016/j.jiec.2009.01.002 http://dx.doi.org/10.1016/j.jiec.2010.01.024 http://dx.doi.org/10.1016/j.jiec.2010.01.024 http://dx.doi.org/10.1016/j.enpol.2010.08.038 http://dx.doi.org/10.1016/j.rser.2009.06.021 http://dx.doi.org/10.1016/j.rser.2010.03.023 http://dx.doi.org/10.1016/j.rser.2012.06.006 http://dx.doi.org/10.1016/j.enpol.2011.12.029 http://dx.doi.org/10.1016/j.renene.2007.05.040 ijahp article: minhas/gis based geo hazard assessment of pakistan for future urban development using ahp international journal of the analytic hierarchy process 405 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.334 gis based geo hazard assessment of pakistan for future urban development using ahp amer imran minhas institute of geographical information system (igis) islamabad, pakistan ms12amer@gmail.com abstract pakistan has been subject to frequent earthquakes, which are often severe (especially in the north and west), and severe flooding along the indus river after heavy monsoon rains (july to august every year). loss to life and property as a result of these natural disasters has been very high in the recent past. in order to mitigate these losses, an integrated decision support which could help planners make complex decisions accurately and quickly is required. this first ever study aims to provide a multi criteria decision analysis framework resulting in the regionalization of the territories of pakistan according to the level of vulnerability to these natural disasters. site suitability for urban development in pakistan was assessed by the application of gis and the analytical hierarchy process (ahp). gis can effectively store, retrieve, manipulate, analyse and display the spatial in site selection problems. ahp can be used to calculate weights of criteria while the decision maker remains consistent in judging and allocating comparative preferences to criteria. weighted scores were aggregated in two clusters namely environmental (elevation, slope, aspect, distance from rivers, land surface temperature and precipitation) and hazard (flood extents, earthquake density and intensity). the results of these two clusters were then synthesized using an innovative scheme to obtain a suitability index map. indices in the map were classified into four categories representing extremely suitable, suitable, less suitable and worst regions for urban development. this study shows how an effective multi criteria decision support method can be developed to select suitable sites for urban development in order to reduce exposure to natural disasters. urban development should be planned in extremely suitable areas. keywords: gis; analytical hierarchy process (ahp); urban development; pakistan; hazard 1. introduction urban development in non-hazardous zones remains an uphill task in pakistan. the pace, scale and spatial reach of anthropological actions make the society increasingly dependent on environmental and urban planning solutions in order to reduce its exposure to natural hazards. there is a necessity for documentation that briefly presents the type and scale of these events as an aid to urban administration for decision making. seismic ijahp article: minhas/gis based geo hazard assessment of pakistan for future urban development using ahp international journal of the analytic hierarchy process 406 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.334 hazards, landslides, rock falls, floods, torrential floods, excessive erosion, droughts, coastal cyclones and forest fires are some of the significant natural hazards within the territory of pakistan; these natural processes can directly and indirectly endanger the environment, populace and property. recent earthquakes in pakistan demonstrated that the region is highly seismic (naseer, khan et al., 2010). the himalayan mountains in the north, mid-oceanic ridges in the south and earthquake belts surrounding the indian plate all render pakistan a risk prone area. the kashmir earthquake of october 8, 2005 had widespread destructive effects with in excess of 86,000 people killed and over 80,000 severely injured (mulvey, awan et al., 2008). the indus monsoon flood in 2010 was one of the greatest river disasters in recent history which affected more than 14 million people in pakistan (gaurav, sinha et al., 2011). though there have been isolated efforts to map these vulnerabilities at various organizations, departments and even at the country level, an integrated data / map containing information on the spatial occurrence of major calamities is not publicly available. consequently, the objective of this research is to provide a multi criteria decision analysis framework that results in the regionalization of the territory of pakistan according to the level of vulnerability to different natural hazards and environmental conditions. this analysis is especially important as an attempt to categorize areas within the country according to their levels of the risk from these events (peduzzi, concato et al., 1996). knowledge of the susceptibility of a given area to these environmental risks is important for spatial development. by understanding the nature and the spatial distribution of natural events in pakistan, actions can be undertaken to reduce the risks. the aim of this research is to determine the geographical distribution of the major types of hazardous occurrences in pakistan. based on this analysis, the ability to create an integral map of the natural hazards identifying the areas prone to certain natural threats within the territory of pakistan will be achieved. the integration of gis and multi criteria decision analysis (mcda) has attracted significant interest over the last 15 years or so (malczewski, 2006). at the most rudimentary level, gis-mcda can be thought of as a process that transforms and combines geographical data and value judgments (the decision-maker’s preferences) to obtain information for decision making (malczewski, 1999). using the integral map of different hazards, mcda will be conducted over areas of low intensity values for all major hazards. the objective of this study is to develop a geographic information system (gis)-based land suitability analysis model for locating optimal sites for urban development against environmental threats. for this purpose, criteria of topography, precipitation, temperature, distance from rivers, potential risk areas from flood and earthquakes will be used. the outcomes of this study will be a land suitability model for urban developments in pakistan. ijahp article: minhas/gis based geo hazard assessment of pakistan for future urban development using ahp international journal of the analytic hierarchy process 407 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.334 2. study area located in south asia, pakistan borders iran to the southwest, afghanistan to the west and north, china to the northeast, and india to the east (figure 1). pakistan is among the most vulnerable areas in the age of climate change (watson, iwamura et al., 2013). urban sprawl is on the rise with population figures totalling 180 million people, making it the sixth most populous country. population density has risen to 270.77 / sq. km. the area is 796,095 sq. km. and pakistan’s river system consists of more than 60 small and large rivers. all of pakistan’s major rivers originate in the northern highlands of the himalaya, karakoram and hindukush mountain ranges and flow south. these rivers have always provided ideal conditions for human settlement and the growth of politics, arts and culture. the frequency and intensity of the occurrence of floods remains very high due to unusually heavy monsoon rains during the normal season that runs from july to september resulting in heavy losses of life and property. pakistan is situated in a highly seismically active region which has experienced many disastrous earthquakes during historical times posing a constant threat to lives and property of people. figure 1. map of pakistan and its neighbours ijahp article: minhas/gis based geo hazard assessment of pakistan for future urban development using ahp international journal of the analytic hierarchy process 408 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.334 3. materials and methods 3.1 integration of gis and mcda geographical information systems (gis) are computer systems or software that can collect, manage, analyse and display spatially referenced data. gis have always been considered good decision support tools because of their map displaying capabilities. multicriteria decision analysis (mcda) is a set of mathematical tools and methods that help a decision maker solve several kinds of problems such as choice, ranking, sorting, and classification. the integration of gis and mcda has attracted significant interest among urban planners since the 1990s (omitaomu, blevins et al., 2012). this integration allows multiple criteria to be taken into account when dealing with spatial decision problems. the principle of the method is to divide the decision problems into smaller understandable parts, analyse each part separately and then integrate the parts in a logical manner (malczewski, 1996). therefore, creation of a decision tree is a main underlying step of mcda. 3.2 analytical hierarchy process (ahp) ahp is a powerful tool in applying mcda and was introduced and developed by saaty in 1980 (bagheri & azmin, 2010; saaty, 1980). in the ahp method, obtaining the weights or priority vector of the alternatives or the criteria is required. for this purpose, saaty (1980) has used and developed the pairwise comparison method (pcm), which is explained in detail in the next part of the work (kordi, 2008). the ahp can handle inconsistency in judgment of the analyst while allocating importance to each criterion by checking a consistency ratio which is not implemented in other techniques like simple additive weighting (saw) etc. 3.3 site selection site suitability assessment is similar to choosing an appropriate location except that the goal is not to isolate the best alternatives but to map a suitability index for the entire study area (al-shalabi, mansor et al., 2006). combining gis and mcda for site planning involves many tasks including data gathering and structuring, and computation of criteria using spatial analysis. most government departments don’t have adequate data available to mitigate hazards. in such a situation, site suitability maps could help planners (dueker & barton, 1990; geertman & toppen, 1990; wang, 1994). these maps would be useful for several years and many decisions. following a similar approach, eastman et al. (1993) produced a land suitability map for an industry near kathmandu using idrisi (a raster gis) and ahp (saaty, 1990). 3.4 selection of criteria various factors influence the choice of urban settlement sites which include social, economic, political, environmental, hazards and availability of services and others. after a detailed literature review and consultation of experts, two clusters of criteria were considered i.e. environmental and natural hazard groups. while selecting the environmental criteria only those factors were considered which are not changeable or affected by others over time. these criteria include elevation, slope, aspect, temperature, precipitation and distance to rivers, whereas land use, land cover, road, rail and trade routes, socio economy and political influences etc. were neglected. while considering the ijahp article: minhas/gis based geo hazard assessment of pakistan for future urban development using ahp international journal of the analytic hierarchy process 409 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.334 hazards, only majors i.e. floods and earthquakes were used, as their spatial spread covers the whole of pakistan. 3.5 generating criteria maps data were obtained from various sources as described in table 1. then they were processed using arcgis 10.1 to obtain criteria maps of the same spatial resolution and projection system i.e. 100 meters and universal transverse mercator (utm) respectively (ersi, 2012). in order to process the data together, it was necessary to transform it to a common spatial resolution and projection system. a detailed flow diagram of this methodology followed for this study is shown in figure 2. table 1 data description type of data source spatial resolution meters duration elevation, slope, aspect shuttle radar topography mission (srtm) 90 11 day mission in feb 2000 land surface temperature mod11c3 product of moderate resolution imaging spectroradiometer (modis) 5600 2000-2013 precipitation 3b42 product of tropical rainfall measuring mission (trmm) 1998-2013 vector data survey of pakistan earthquake events usgs earthquake hazard program 2000-2013 flood extents un habitat 2010-2013 ijahp article: minhas/gis based geo hazard assessment of pakistan for future urban development using ahp international journal of the analytic hierarchy process 410 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.334 figure 2. methodology ijahp article: minhas/gis based geo hazard assessment of pakistan for future urban development using ahp international journal of the analytic hierarchy process 411 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.334 3.6 calculation of criteria scores ahp was used to calculate the importance of each layer towards achieving the ultimate goal. there is a need to transform the different attribute values to a common scale in order for ahp to be used to obtain the contribution value of each layer. therefore, after consulting experts in the teaching faculty of the school of civil and environmental engineering (scee) in the national university of science and technology (nust) in pakistan and consulting similar studies conducted by researchers (dai, lee et al., 2001; al-shalabi, mansor et al., 2006; özgen, 2010), new score values were assigned at a common scale of 0 to 10 and resultant layer maps are shown in figure 3. ahp consists of three steps. in the first step, the problem was defined and broken down into simple understandable parts, known as structural hierarchy (shown in figure 4). the goal is to develop a hierarchy with the top level as the goal (suitable site selection for urban development) and ladders down from general to specific levels ending with nine attributes. each level in the network must be linked with the next one (şener, süzen et al., 2006). ijahp article: minhas/gis based geo hazard assessment of pakistan for future urban development using ahp international journal of the analytic hierarchy process 412 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.334 elevation slope aspect temperature precipitation distance from rivers flood extents earthquake intensity earthquake density 1 score values 10 figure 3. transformed criteria map layers ijahp article: minhas/gis based geo hazard assessment of pakistan for future urban development using ahp international journal of the analytic hierarchy process 413 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.334 figure 4. structural hierarchy of the problem criteria are compared with each other to determine relative importance of each in accomplishing the objective in the second step of ahp. this was achieved through a pairwise comparison matrix which was built by assigning numerical values to each pair of constraints using guidelines given by saaty (table 2). separate matrices were built for environmental and hazard groups and are shown in tables 3 and 4 respectively. then the weights of layers were calculated by normalizing values in each column of the matrix and calculating the row mean. consistency of judgment in assigning the priority values was checked by a consistency ratio (cr) which was 4%, thus being well within the specified limit of 10% discussed by saaty. after multiplying the weights with score values, weighted layers were obtained. standardized scores and weighted layer values are shown in table 5. the weighted layers in each group were added to obtain the environmental and hazard maps (figure 5). in the third and final step, results were synthesized. in order to combine the two resultant maps in order to obtain the final suitability index map, each was classified into four classes. an environmental suitability map was classified using values of 1 to 4, and a hazard suitability map was assigned with values of 10, 20, 30 and 40. the higher the score the more suitable the site is for urban development in pakistan. this classification scheme aimed at retaining the original contributing value of both clusters. these two layers were aggregated to calculate the final suitability score map (figure 6). the suitability index shows values of 11, 12, 13, 14, 21, 22, 23, 24, 31, 32, 33, 34, 41, 42, 43 and 44. in raster gis these indices represent alternatives from which better ones can be chosen by town planners for locating settlements. ijahp article: minhas/gis based geo hazard assessment of pakistan for future urban development using ahp international journal of the analytic hierarchy process 414 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.334 table 2 saaty’s pairwise comparison prioritization table intensity definition 1 3 5 7 9 equal importance moderate importance strong importance very strong importance extreme importance # 2,4,6 & 8 can be used to express intermediate values $ reciprocals can be used for inverse judgments table 3 pairwise comparison matrix for environmental group e le v a ti o n s lo p e d is ta n c e to r iv e r a v e r a g e p r e c ip it a ti o n a v e r a g e te m p e r a tu r e a sp e c t weight of layer (%) elevation 1 2 3 5 6 8 39 slope 1/2 1 3 4 5 8 29 distance to river 1/3 1/3 1 2 4 7 15 average precipitation 1/5 1/4 1/2 1 2 4 8 average temperature 1/6 1/5 1/4 1/2 1 3 6 aspect 1/8 1/8 1/7 1/4 1/3 1 3 consistency ratio = 3% ijahp article: minhas/gis based geo hazard assessment of pakistan for future urban development using ahp international journal of the analytic hierarchy process 415 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.334 table 4 pairwise comparison matrix for hazard group f lo o d e a r th q u a k e z o n e s e a r th q u a k e d e n si ty weight of layer (%) flood 1 3 5 64 earthquake zones 1/3 1 3 26 earthquake density 1/5 1/3 1 10 consistency ratio = 4% ijahp article: minhas/gis based geo hazard assessment of pakistan for future urban development using ahp international journal of the analytic hierarchy process 416 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.334 table 5 standardized scores and weighted layer values criteria weight attribute value rank score environmental group elevation 38.76 -140 feet 0-1000 1000-1500 1500-2000 2000-2500 2500-3000 3000-3500 3500-4000 4000-8569 0 10 9 7 5 3 2 1 0 0 387.60 348.84 271.32 193.80 116.28 77.52 38.76 0 slope 28.77 0-3 percent 3-5 5-7 7-10 10-15 15-20 20-< 8 10 7 5 3 2 1 230.16 287.70 201.39 143.85 86.31 57.54 28.77 aspect 2.80 flat north north east east south east south south west west north west 5 1 3 7 10 8 6 4 2 14.00 2.80 8.40 19.60 28.00 22.40 16.80 11.20 5.60 distance to rivers 15.56 0-500 meters 500 -1000 1000-2000 2000-3000 3000-< 1 3 5 8 10 15.56 46.68 77.80 124.48 155.60 temperature 5.51 -18-0 degree celsius 0-10 10-15 15-25 25-33 33-40 40-46 1 3 8 9 10 7 5 5.51 16.53 44.08 49.59 55.10 38.57 27.55 precipitation 8.60 0-50 mm/sq. km 50-150 150-200 200-300 300-400 1 2 4 5 7 8.60 17.20 34.40 43.00 60.20 ijahp article: minhas/gis based geo hazard assessment of pakistan for future urban development using ahp international journal of the analytic hierarchy process 417 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.334 400-500 500-689 8 10 68.80 86.00 hazard group earthquake density 10.47 iv-less richter scale iv-vi vi-vii vii-viii viii-< 1 4 6 8 10 10.47 41.88 62.82 83.76 104.7 earthquake intensity 25.83 0-5 events / 100 sq. km 50-100 100-200 200-300 300-400 400-500 500-635 10 8 6 5 3 2 1 258.3 206.64 154.98 129.15 77.49 51.66 25.83 flood extents 63.70 0 1 1 10 63.70 637 environmental group map hazard group map figure 5. environmental and hazard group maps ijahp article: minhas/gis based geo hazard assessment of pakistan for future urban development using ahp international journal of the analytic hierarchy process 418 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.334 figure 6. final suitability index map 4. findings and discussion the study was performed in a manner that the ancient settlement priorities of the masses could be traced back through their roots in commonly known environmental conditions. the same priorities are represented in the form of suitability zones for future planning of urban development. secondly, two major natural hazards i.e. floods and earthquakes were mapped and aggregated to mitigate future losses. the processing of data resulted in three kinds of suitability maps i.e. environmental, hazard and aggregated final score map. the scores of the environmental and hazard group maps represent higher suitability for urban development in areas with higher index value and vice versa in their respective criteria domain; whereas the scores of the final suitability map need to be decoded keeping in view the score values in the two contributing group maps. a combination of higher values in both represents suitability while the union of lower values shows lesser suitability. 4.1 environmental group map the environmental group map encompassed the contribution of six factors namely elevation, slope, aspect, distance to rivers, mean temperatures and precipitation since the year 2000 onwards. the percentage of emphasis each criterion had on this group map is described in table 5 above. after a detailed consultative and cyclic process of evaluation the analyst found elevation to be the biggest contributor, or in other words it had the highest priority, of the people who settled in pakistan throughout history with the weight of this layer being 39%. on the other hand, aspect remained the lowest, having the least priority of the people who settled in pakistan despite being a mountainous country all along its reach; the weight of this layer was just 3%. this shows that the people in pakistan almost completely disregarded the importance of the availability of sunlight for their agricultural crops and settlement purposes as compared to other important factors. ijahp article: minhas/gis based geo hazard assessment of pakistan for future urban development using ahp international journal of the analytic hierarchy process 419 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.334 aggregated results of all six weighted layers showed scores from 5 to 1001. higher score values showed higher suitability for settlement and vice versa. this means that the combined effect of all six layers when analysed collectively showed distinct results for every 100 sq. m. which is the spatial resolution of the analysis. the area of pakistan was divided into four zones. the first zone contained scores from 5 to 333 which represented the least priority area for settlement and are shown in red in figure 5 above. the second zone contained score values from 333 to 598 which represented less suitable areas for settlement and are shown in yellow in figure 5 above. the third zone contained score values from 598 to 781 which represented suitable areas for settlement and are shown in light green in figure 5 above. the fourth zone contained score values from 781 to 1001 which represented the most suitable areas for settlement and are shown in dark green in figure 5 above. while comparing the map of the environmental group and population shown above in figures 5 and 7 respectively, it is evident that the areas of the most suitable zone in the first map shown in dark green generally conform to the areas of high density population in darker shades of brown in the second map. this is because the masses choose environmentally better places due to their awareness of the conditions. figure 7. population density map population ijahp article: minhas/gis based geo hazard assessment of pakistan for future urban development using ahp international journal of the analytic hierarchy process 420 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.334 4.2 hazard group map the hazard group map encompassed the contribution of three factors namely flood occurrence of major events, earthquake density and earthquake intensity from the year 2000 onwards. the percentage of emphasis each criterion had on this group map is described in table 5 above. after a detailed consultative and cyclic process of evaluation, the analyst found flood areas as the highest contributor with the weight of this layer being 64%. on the other hand, earthquake density remained the lowest, being the least priority, as it counted all the events above richter scale iv and above irrespective of its intensity. the weight of this layer came out to be just 10%. this shows that the importance of a greater number of events is less important for settlement purposes as compared to the greater intensity of similar events i.e. earthquakes. aggregated results of all three weighted layers showed scores from 14 to 1000. as before, higher score values showed higher suitability for settlement and vice versa. this means that the combined effect of all three layers when analysed collectively showed distinct results for every 100 sq. m. which is spatial resolution of the analysis. the area of pakistan was divided into four zones. the first zone contained scores from 5 to 685 and represented the least priority areas for settlement and are shown in red in figure 5 above. the second zone contained score values from 685 to 774 and represented less suitable areas for settlement; these areas are shown in yellow in figure 5. the third zone contained score values from 774 to 860 and represented suitable areas for settlement; these areas are shown in light green in figure 5. the fourth zone contained score values from 860 to 1000 representing the most suitable areas for settlement and are shown in dark green in figure 5. while comparing the map of the hazard group and population shown above in figures 5 and 7 respectively, it is evident that the areas of the most suitable zone in the first map shown in dark green generally do not conform to the areas of high density population in darker shades of brown in second map. this is because the masses were not aware of the hazardous conditions surrounding them. 4.3 synthesis of environmental and hazard group layers in order to combine the two group layers and maintain the sovereignty of individual groups; first each one was classified into four classes and allocated separate series numbers. series 1, 2, 3 and 4 was allotted to the environmental group, and series 10, 20, 30 and 40 was allotted to hazard group. after reclassifying and simply adding the two groups a final score map was obtained which showed suitability score values of 11, 12, 13, 14, 21, 22, 23, 24, 31, 32, 33, 34, 41, 42, 43 and 44 (shown in figure 6 above). generally, the higher scores represented higher suitability and vice versa. but a critical review of the score values showed that the combination of higher score values in both the contributing maps i.e. environmental and hazard groups actually represent more suitability and not a numerically higher score value. therefore, when traced back to their original routes it was determined that the indices can be grouped into four clusters. the first cluster having a combination of lower values in both the group layers can be grouped i.e. first two values from each which are 11, 12, 21 and 22. the second cluster having a combination of lower middle values in both the group layers can be grouped i.e. first two values from hazard group and third / fourth value of environmental group which are 13, ijahp article: minhas/gis based geo hazard assessment of pakistan for future urban development using ahp international journal of the analytic hierarchy process 421 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.334 14, 23 and 24. the third cluster having a combination of lower values environmental group and higher values in hazard group which are 31, 32, 41 and 42. the fourth cluster having a combination of higher values from environmental group and higher values in hazard group which are 33, 34, 43 and 44. 4.4 final map the final map was complex to read and assimilate the individual score values. therefore, based on the discussion in section 3.3 above, the map was reclassified into four zones and shown in figure 6. the first zone having values of 11, 12, 21 and 22 were assigned the colour red representing areas which were worst for living or future urban development. the second zone having values of 13, 14, 23 and 24 were assigned the colour yellow and represented areas which were less suitable for living. the third zone having values of 31, 32, 41 and 42 were assigned the colour light green and represented areas which were suitable for living. the fourth zone having values of 33, 34, 43 and 44 were assigned the colour dark green colour and represented areas which were most suitable for living. a visual comparison of the final classified map (figure 6) and population density map (figure 7) showed that most of the settlement areas in pakistan are within the highest suitability zone i.e. zone 1 in green, however, a few settlement areas in the northern, middle west and southern parts of the country lay in lower suitability areas i.e. zone 2 in yellow. 4.5 statistical analysis 4.5.1 statistical analysis of area and population conformity statistical analysis of area and population conformity to suitability zones show that 72% of the population lives in 54% of the total area which is extremely suitable for living, thus conforming to the desired standards. however, the suitable zones are thinly populated with an increased population in less suitable zones as compared to suitable zones. the worst zone is thinly populated. the details are shown in figure 8. 4.5.2 statistical analysis of roads and rails statistical analysis of linear features also shows a similar trend as that of the population. the most suitable zone comprises the bulk i.e. 66% metaled roads, 60% tracks and 78% rail roads. the details are shown in figure 9. 4.5.3 district risk index analysis a district risk index analysis resulted in two kinds of outputs. one was the high population density districts, and the second was high risk districts where the less suitable and worst zones overlapped. when high profile districts were compared in both sheets it was found that some major high population districts are also in high risk zones. these districts are shown in table 6 and include islamabad, sialkot, narowal, muzaffarabad and quetta. ijahp article: minhas/gis based geo hazard assessment of pakistan for future urban development using ahp international journal of the analytic hierarchy process 422 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.334 figure 8. area and population conformity to suitability zones figure 1. roads, tracks and rail vs. suitability zones 3 25 1 72 12 20 14 54 worst less suitable suitable extremely suitable population (%) area (%) 1194 3% 13752 29% 995 2% 31550 66% 523 2% 8338 36% 403 2% 13623 60% 46 0.9% 1725 21% 8 0.1% 6297 78% worst less suitable suitable extremely suitable roads tracks railways ijahp article: minhas/gis based geo hazard assessment of pakistan for future urban development using ahp international journal of the analytic hierarchy process 423 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.334 table 6 high risk districts district area distribution in suitability zones (%) population distribution (%) as per density (pers / sq km) worst less suitable suitable extremely suitable scarce (5) low (200) high (3k) very high (60k) abbottabad 46 54 0 0 6 58 33 3 islamabad 0 63 0 37 4 60 35 1 muzaffarabad 70 30 0 0 16 40 42 2 narowal 0 97 0 3 11 60 29 0 nowshera 0 66 5 28 7 60 33 0 peshawar 0 60 0 40 61 36 3 0 quetta 39 61 0 0 2 6 88 4 ijahp article: minhas/gis based geo hazard assessment of pakistan for future urban development using ahp international journal of the analytic hierarchy process 424 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.334 5. conclusion suitable site selection for urban development is a multifaceted problem which considerably affects the cost and time in building and maintaining urban facilities. the research aimed to achieve the objective through the integration of the multicriteria decision analysis (mcda) framework using the analytical hierarchy process (ahp) and gis. this involved four steps including selection of criteria, preparation and standardization of criteria maps, pairwise comparison through ahp to calculate contributing weights of each criteria layer and synthesis of results. the methodology resulted in the regionalization of the territory of pakistan according to the level of suitability for future urban development, thus achieving its goal. the study was conducted at the regional level using environmental and hazard factors. the results can either be used by all concerned with mitigating hazards and locating environmentally suited localities, or as a base template for overlying by additional factors for optimized local level studies. ijahp article: minhas/gis based geo hazard assessment of pakistan for future urban development using ahp international journal of the analytic hierarchy process 425 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.334 references al-shalabi, m. a., s. b. mansor, et al. (2006). gis based multicriteria approaches to housing site suitability assessment. xxiii fig congress, shaping the change, 8-13. bagheri, m. and w. n. azmin (2010). application of gis and ahp technique for landuse suitability analysis on coastal area in terengganu. world automation congress, 1-6. dai, f., c. lee, et al. (2001). gis-based geo-environmental evaluation for urban land-use planning: a case study. engineering geology, 61(4), 257-271. doi: 10.1016/s00137952(01)00028-x esri 2012. arcgis desktop: release 10.1 redlands, ca: environmental systems research institute. gaurav, k., r. sinha, et al. (2011). the indus flood of 2010 in pakistan: a perspective analysis using remote sensing data. natural hazards 59(3), 1815-1826. doi: 10.1007/s11069-011-9869-6 kordi, m. (2008). comparison of fuzzy and crisp analytic hierarchy process (ahp) methods for spatial multicriteria decision analysis in gis. geocarto international, 27(2), 193. malczewski, j. (1996). a gis-based approach to multiple criteria group decisionmaking. international journal of geographical information systems, 10(8), 955971. doi: 10.1080/10106049.2011.643128 malczewski, j. (1999). gis and multicriteria decision analysis, new york: john wiley & sons. malczewski, j. (2006). gis‐based multicriteria decision analysis: a survey of the literature. international journal of geographical information science, 20(7), 703-726. doi: 10.1080/13658810600661508 mulvey, j., s. awan, et al. (2008). profile of injuries arising from the 2005 kashmir earthquake: the first 72h. injury, 39(5), 554-560. doi: 10.1016/j.injury.2007.07.025 naseer, a., a. n. khan, et al. (2010). observed seismic behavior of buildings in northern pakistan during the 2005 kashmir earthquake. earthquake spectra, 26(2), 425-449. doi: 10.1193/1.3383119 omitaomu, o. a., b. r. blevins, et al. (2012). adapting a gis-based multicriteria decision analysis approach for evaluating new power generating sites. applied energy, 96, 292-301. http://dx.doi.org/10.1016/j.apenergy.2011.11.087 ijahp article: minhas/gis based geo hazard assessment of pakistan for future urban development using ahp international journal of the analytic hierarchy process 426 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.334 özgen, c. (2010). evaluation of settlement sites beyond the scope of natural conditions and hazards by means of gis based mcda: yeşilirmak catchment, thesis submitted to middle east technical university. peduzzi, p., j. concato, et al. (1996). a simulation study of the number of events per variable in logistic regression analysis. journal of clinical epidemiology, 49(12), 1373-1379. doi: http://dx.doi.org/10.1016/s0895-4356(96)00236-3 saaty, t. l. (1980). the analytic hierarchy process: planning, priority setting, resources allocation. new york: mcgraw hill. saaty, t. l. (1990). how to make a decision: the analytic hierarchy process. european journal of operational research, 48(1), 9-26. doi:10.1016/0377-2217(90)90057-i şener, b., m. l. süzen, et al. (2006). landfill site selection by using geographic information systems. environmental geology, 49(3), 376-388. watson, j. e., t. iwamura, et al. (2013). mapping vulnerability and conservation adaptation strategies under climate change. nature climate change, 3, 989-994. doi:10.1038/nclimate2007 http://dx.doi.org/10.1016/s0895-4356(96)00236-3 http://www.jclinepi.com/article/s0895-4356(96)00236-3/abstract http://www.jclinepi.com/article/s0895-4356(96)00236-3/abstract http://dx.doi.org/10.1016/0377-2217(90)90057-i ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 51 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry majid azizi professor, college of agriculture and natural resources, university of tehran mazizi@ut.ac.ir hamzehali mansouri ms. graduate student, university of tehran abstract this study was carried out because of the lack of research on strategic planning in iran's home furniture market. accordingly, in this study, we developed a decision-making model to select the best solution for strategic planning in the industry. after determining the strategic criteria or factors affecting the developed model, the control criteria of four merits or subsets of competencies were identified. to determine all the effective factors in the model, the sub-criteria of four control criteria and their connections were identified. the following four possibilities were considered as potential solutions: entrance to a foreign market (s1), increase portfolio (s2), emphasis on scientific management of mixing elements of marketing (s3), and generate research and development units (s4). the analytic network process (anp) and the super decisions software were used to synthesize and analyze the model. it was found that all calculated decisions were influenced by the strategic criteria. a value-weighted competency model was calculated in the first stage with the influence of strategic criteria on the competency model. hierarchical design decisions were made for each of the competencies and their subsets (144 sub-criteria and 22 middle indices). paired comparison matrices associated with the degree of the importance of each of the competencies were achieved in the second stage. in the final stage, subsets of the competency weights and their sub-options were identified with the combination of the competencies and the best solution was obtained. keywords: analytic network process; strategic criteria; control criteria; benefits; opportunities; costs; risks; home furniture; marketing; sensitivity analysis 1. introduction marketing is the process of planning and executing the conception, pricing, promotion, and distribution of ideas, goods, and services. marketing strategy is a long-term, forwardlooking approach and part of the overall plan of any organization or business with the fundamental goal of achieving sustainability by understanding the needs and wants of mailto:mazizi@ut.ac.ir ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 52 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 customers (baker 2008). marketing strategy highlights the role of marketing as a link between the organization and its customers. strategic marketing planning demands that the organization recognize the challenges they face and their effect on its potential for future success. an organization must learn to focus on customers and their needs at all times and explore every avenue that may provide a differential advantage over its competitors (mcdonald, 2006). one of the most vital factors in running a successful business is the strength of the strategic marketing plan. conducting market research, distilling the findings , and acting on recommendations in the marketing plan helps a business define what it will sell, to whom it will sell, how it will communicate to the buyers, and when to communicate (ahmed, 2019). the process of creating a plan facilitates a common understanding among all stakeholders in an organization. the plan informs management decisions, the behavior of the employees towards institutional goals, and also the response among current and potential clients. the plan is also subjected to changes over time to meet changing demands. overall, the strategic marketing planning process connects the production engine to consumption (lake, 2019). effective marketing starts with a considered, well-informed marketing strategy. a good marketing strategy helps the industry define the vision, mission, and business goals, and outlines the steps it needs to take to achieve these goals. the marketing strategy of the industry affects the way managers run their entire business, and therefore, should be planned and developed in consultation with their team. it is a wide-reaching and comprehensive strategic planning tool that describes the business and its products and services; explains the position and role of the products and services in the market; profiles customers and competition; identifies the marketing tactics that the industry will use; and allows managers to build a marketing plan and measure its effectiveness (www.business.qld.gov.au). strategic marketing is a method through which an organization differentiates itself from its competition by focusing on its strengths to provide better service and value to its customers. in summary, the goal of strategic marketing is to make the most of an organization’s positive differentiation over its competition through the consumers’ perspective (mehta, 2019). in the current business landscape, marketing strategies are very important for each company. the business world is changing constantly, and to decrease environmental pressures, successful companies have to pay careful attention to three important subjects including the expansion of globalization, the development of modern technologies, and economic turmoil (petersen, 2015). a review of the major prescriptive literature about marketing strategic planning (anderson, 1982; aaker, 2010; cravens, 2006; perreault, 2011) shows that the following marketing factors influence marketing strategic planning either implicitly or explicitly: identifying and selecting the target market; marketing mix; and marketing plan. a target market is a particular group of consumers at which a product or service is aimed (levy, 2012). the marketing mix is a combination of factors that can be controlled by a company to influence consumers to purchase its products ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 53 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 (huang, 2014). a marketing plan is a business's operational document for outreach and advertising to generate leads and reach its target market (westwood, 2013). segmentation, targeting, and positioning help in strategic planning (patil, 2017) and are also an antecedent of the marketing mix and the marketing plan (londhe, 2014). good marketing strategic planning should be based on the marketing plan. during the last two decades, the furniture industry has grown rapidly and surpassed the clothing and footwear industries, ranking first in the world of light industries as a fastgrowing, high-value-added, and employment-generating industry. the furniture industry is more important than other sectors of the wood industry because the furniture industry accounts for 80% of its total production centers, 4% of its labor force, and 72% of the value-added of the entire wood industry in iran (zolghadr, 2009). the furniture industry has about a 2% ($ 350 billion) share of the global market and is growing every year. meanwhile, iran's share of the global furniture trade is only 0.02 percent ($6 million). (home and office furniture exporters union 2009; ghofrani et al. 2014). a bigger market share would bring in more customers and thus more profit. a wider range of choices would help satisfy more desires and needs from the customers’ point of view. future sales turnover will be affected by pricing policies, competitor actions, and advertising campaigns as well as by the level of customer satisfaction achieved (petukhov, 2015). furniture is a global product in national and international trade with abundant export and import capacities. these effects are evident in the well-being of individuals and communities, as well as in revenue and outcomes for consumers, manufacturers, retailers, and marketers. the analytical network process (anp) is a flexible method that can demonstrate the mutual interactions between all factors in decision-making procedures. this method is an independent decision support model, belonging to the family of multi-criteria analyses, which was developed by saaty (2005) and saaty and vargas (2006a). 2. literature review kao et al. (2015) developed the anp with the following four criteria: control device, product design, product reliability, and market environment. each criterion consisted of multiple sub-criteria, and the result showed that portable control devices, user-friendly interface, aesthetic product design, personal information protection, and product compatibility affect consumer adoption of smart home applications were the most important. this research contributes to the literature by understanding consumer adoption in the area of communication activities and by extending the anp to marketing research. the anp was also used to determine the best alternative for investment and development of construction panels. the results of this study showed that wood cement product has the highest priority. furthermore, concerning the influence of the overall factors on the merits of benefits, costs, opportunities, and risks, opportunities are the most important factor in decision-making. also, costs and risks were more sensitive than benefits and opportunities in this study (azizi & modarres, 2011). ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 54 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 azizi and ray (2015) presented a decision-making model developed to select the best solution for raw material supply to the wood cellulosic industry. the anp and ahp were used to synthesize and analyze the model. this research indicated that alternative c, long-term contracts with suppliers, was the most appropriate solution to provide raw material to the cellulosic industries of iran under the assumptions and value judgments of the experts surveyed in the study. alternative c minimized costs and risks, and therefore created less costly and less risky conditions for the existing cellulosic product manufacturers and potential investors due to a more reliable, higher quality raw material supply. azizi and mehdikhanloo (2018) applied the anp to determine the best solution for the role of design in the furniture industry. the result of the study showed that using a combination of fashion and engineering designs in furniture production is a proper solution to apply to the role of design in the furniture industry. research results show that the most effective indicators for optimization of wood consumption in the construction of classical furniture were ergonomic, style, skill training and inlays in the classical furniture industry, and the use of specialized labor was a priority solution (ghofrani et al., 2017). taliscali and ercan (2006) and alves et al. (2008) demonstrated the “user-friendly” nature and the application of qualitative and quantitative factors in the evaluation as fundamental advantages of the ahp/anp, in comparison with other mcdm methods. the basic difference is that anp has a network structure that allows the analysis of dependence among elements of the model. this makes the anp more powerful in situations with complex levels of uncertainty, and allows the analysis to more closely model reality. azizi and ray (2016) developed a model for privatization of the newsprint paper industry by applying the anp and bocr structure. the study showed the option of encouragement for investment as an appropriate solution to privatize the iranian newsprint paper industry. the highest priority was encouragement for investment returns in each of the benefits and opportunities criteria. encouragement for investment would create increased domestic and foreign investment in the iranian newsprint paper industry. this option provides plenty of opportunities for privatization and economic activities. after implementation, the resulting investment in private ventures opens doors to the opportunity of reducing administrative costs through methods such as limiting rules and heavy taxes, as regulators are presented with tangible proof of increased profits and state revenue, made possible by the increased efficiencies of private sector management. this research asks the following questions: what are the strategic criteria or factors affecting this study? what are the control criteria in this study? which of the identified solutions is the most important solution for marketing strategic planning in the home furniture industry? in the iranian home furniture industry, up-to-date information is not available to determine a marketing strategy; therefore, manufacturers cannot determine the best way ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 55 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 to sell their products more successfully. this research gap suggests the need for a strategic plan to solve the problem of producers. this study was conducted to help solve the problem. this study aimed to develop a decision network model to find the best solution for marketing strategic planning in iran's home furniture production. in the network model, the merits are divided into four sub-sections as benefits, opportunities, costs, and risks (bocr). these merits are influenced by the strategic criteria (political, legal, development, and technological, cultural and social, and economic factors) to obtain the weighted values for each factor. this process follows the anp. 3. methodology 3.1 the analytic network process (anp) we will review the anp in this section because it is the most suitable technique for our study. the anp can be considered a general form of the ahp and is more concerned with the network structure. one of its advantages is that it allows for dependence and includes independence (saaty, 2006b). the anp can prioritize groups or clusters of elements; it handles interdependencies better than the ahp and can support complex, network decision-making with various intangible criteria (tsai, leu, liu, lin & shaw, 2010). the advantage of this theory is its ability to be used with cases involving dependence and feedback and the generalization of the super-matrix approach. it allows interactions and feedback within clusters (inner dependence) and between clusters (outer dependence). the anp couples two parts that consist of a control hierarchy or network of criteria and sub-criteria that control the interactions in the system being studied, and a network of influences among the elements and clusters. the network varies from criterion to criterion and a super-matrix of limiting influence is computed for each control criterion. finally, each of these super-matrices is weighted by the priority of its control criterion and the results are synthesized through addition for all the control criteria. to determine the effective criteria and sub-criteria as well as solutions for the role of design, the following steps were performed. after reviewing the internal and external resources and interviewing the producers and academicians, a comprehensive list of effective criteria was developed to enable us to understand all the important criteria for decision-making concerning the role of design. to do this, the views of 22 wood and furniture industry experts were used. to identify the indicators, we coordinated with experts in the home furniture market and industry. the content of the networks and the factors involved were previously discussed with the respondents. the number of questions was based on the number of comparisons of criteria and alternatives individually. after gathering these responses from the experts, it was possible to synthesize and analyze the model. three types of questionnaires were prepared. the first questionnaire was used to determine the weight value of the indicators and sub-indicators (8 people, including 5 experts and members of the home furniture union and 3 university professors). the second questionnaire was used to compare a pair of subnets and their nodes to the solutions and feedback within the networks (14 people, including 7 home furniture manufacturers, 4 economics and marketing experts, 3 university professors in the wood industry and ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 56 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 management). the third questionnaire was used to determine the weight value of the strategic criteria that affect marketing (4 macro experts in the field of economics, market and production of iranian home furniture). the questionnaires were prepared using super decisions software. a sample of the questionnaire was printed and distributed among the experts. for example, the questionnaire included a comparison to determine which criterion is more important, economic and marketing or social, cultural or political? another comparison concerning export under the subnetwork economic and market control criteria was performed to determine which one of the solutions is more important, entrance to a foreign market (s1) or increase portfolio (s2)? the research pattern is as follows (figure 1): figure 1 research pattern there are 144 sub-criteria in 22 middle criteria and four main groups (economic and market, technical and human, supply and production, and social, cultural, political) (see ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 57 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 tables 1-4). weighting values of criteria and sub-criteria are obtained by pairwise comparisons and feedback between the criteria, sub-criteria, and the four solutions. table 1 control criteria and sub-criteria of benefits main criteria of benefits sub-criteria economic and market/brand emergence and development of regional national brands possibility of receiving certification (strengthening the brand) economic and market/advertisements helping effective and targeted product advertisements the possibility of using the appropriate headline in advertisements economic and market/export tax exemption in exchange for exports benefiting from export awards currency resulting from non-oil exports economic and market/investment production of joint products under license economic and market/marketing attracting, retaining, and increasing # of customers selecting profitable markets design and implementation of internal marketing economic and market/competitiveness increasing the share of foreign markets stabilization of domestic markets economic and market/sales and after-sales services increasing profitability increasing turnover customer satisfaction economic and market/infrastructure utilizing the scientific and technological infrastructures of universities utilizing the capacity of furniture clusters supply and production/quality control increasing the quality and accuracy of production possibility of product grading supply and production/packaging the need for beautiful and economical packaging ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 58 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 supply and production/ r&d supply and production/production process moving within the boundaries of marketing knowledge improving product quality increasing production speed reducing production costs supply and production/raw material using local raw materials technical and human/technology using the maximum power of machines using raw materials with high efficiency technical and human/educational (industryuniversity relationship) consulting services of university professors to industry providing technical manpower for industries technical and human/human resources activating the market of related academic disciplines increasing creativity and innovation technical and human/managerial proper management of human resources increasing customer satisfaction increasing employee satisfaction technical and human/design and ergonomics observing standards technical and human/creativity and innovation promoting the position of innovation in the company's marketing strategy social cultural-political/cultural reviving the reputation of iranian furniture social cultural-political/political increasing political influence increasing political relations social cultural-political/social responding to customer needs ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 59 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 table 2 control criteria and sub-criteria of opportunities main criteria of opportunities sub-criteria economic and market/advertisements possibility of receiving a certificate (advertising development) influencing the attitude and behavior of consumers finding a customer for products and services economic and market/sales and after-sales services knowing the future markets creating demand and increasing market share product warranty economic and market/export weakening foreign brands active in the market moving in the direction of resistance economy participating in domestic and foreign exhibitions economic and market/competitiveness creating a competitive environment in the industry economic and market/marketing possibility of segmenting the target market helping the product pricing strategy helping to market products recognizing marketing opportunities economic and market/investment the joint production of export product the possibility of creating a common brand in the domestic market economic and market/infrastructure using the capacity of graduates using the capacity of research centers economic and market/brand branding of iranian furniture correcting the weaknesses of previous brands supply and production/raw material shortening the material supply chain using new raw materials supply and production/quality control receiving quality certificates supply and production/packaging help reduce the volume of packaging ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 60 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 supply and production/production process development of mass production approach timely delivery of the product supply and production/ r&d the possibility of creating and developing research and development units in companies process optimization new product design transfer and absorption of new technologies technical and human/technology use of new machines the possibility of using new technical knowledge technical and human/educational (industryuniversity relationship) the possibility of establishing marketing companies by graduates technical and human/human resources increasing the level of employment increasing the specialization and skills of the labor force technical and human/managerial possibility of using outsourcing opportunities improving the thoughts, beliefs, and motivations of customers technical and human/creativity and innovation the possibility of innovation in the process, product, and service the possibility of innovation in marketing the possibility of encouraging people to innovate technical and human/design and ergonomics design and ergonomics differentiate the product from the products of competitors creating innovation and creativity in products social, cultural and political/cultural production with the culture and needs of the society changing and developing consumption patterns social, cultural and political/political strengthening political and economic diplomacy ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 61 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 social, cultural and political/social improving the quality of life growth of furniture consumption increasing trust in the national brand table 3 control criteria and sub-criteria of costs main criteria of costs sub-criteria economic and market/advertisements the cost of various advertisements economic and market/sales and after-sales services the cost of creating a store in the target countries the possibility of increasing the cost of the product economic and market/export the cost of analyzing export target markets lack of sufficient knowledge of export target markets economic and market/marketing the cost of participating in the exhibition the cost of marketing a new product lack of awareness of market needs economic and market/competitiveness reduce profit margins of products economic and market/brand brand registration fee supply and production/raw material increasing the cost of raw material supply supply and production/quality control purchasing machines for the quality control unit supply and production/packaging the cost of designing a new package to fit the product supply and production/ production process creating separate production lines supply and production/ r&d the cost of creating and developing research and development units research costs ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 62 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 technical and human/creativity and innovation the cost of employees' failures and mistakes in the field of innovation technical and human/educational (industryuniversity relationship) the cost of training marketing forces technical and human/ technology the cost of purchasing new technology and machinery technical and human/human resources the cost of providing specialized and skilled manpower human resources the cost of holding training courses technical and human/design and ergonomics spending time and money to design a new product the cost of testing furniture social, cultural and political/cultural the cost of analyzing the type of culture, ideas and attitudes of the target market social, cultural and political/political the cost of analyzing and examining political constraints social, cultural and political/social the cost of studying and classifying social groups ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 63 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 table 4 control criteria and sub-criteria of risks main criteria of risks sub-criteria economic and market/advertisements limited financial resources economic and market/marketing failure of marketing goals lack of sufficient information about target markets economic and market/competitiveness the possibility of intensifying competition between domestic producers the weakness of rules and regulations regarding competition economic and market/infrastructures existence of redundant laws and regulations economic and market/investment risk of guaranteeing the return and security of foreign investment economic and market/sales and after-sales services the possibility of losing the domestic market the possibility of losing the popularity of the main brand decrease in sales due to recession economic and market/export the possibility of increasing the final price the possibility of customs tariff fluctuations in target markets currency price fluctuations economic and market/brand the possibility of stimulating competitors to increase the brand supply and production/ r & d weak knowledge and research and development skills supply and production/raw materials probability of declining quality of raw material supply the need to use imported raw materials technical and human/educational (industryuniversity relationship) lack of training and experience of graduates in the field of professional marketing and sales non-compliance of industry needs with university education ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 64 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 technical and human/human workforce instability of skilled workforces technical and human/managerial lack of flexibility of managers technical and human/technology machinery is not up to date technical and human/design and ergonomics lack of familiarity with furniture design and testing software social, cultural and political/social possibility of changing nuclear values (beliefs) changes in customs and habits social, cultural and political/cultural possibility of promoting the consumption of furniture with a foreign brand social, cultural and political/political restrictive economic sanctions political and regional tensions ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 65 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 3.2 data/model analysis the overall structure of the decision making is shown in figure 2. figure 2 overall structure of decision making in the current research, the anp model is described by the above elements. the subnetworks are shown in figures a1-a5 in the appendix. 3.3 alternatives 1) entrance to a foreign market (s1) this alternative includes the following four solutions: (1) identifying competitive target markets; (2) identifying competitors' capacity and plan for their strengths and weaknesses; (3) identifying future market opportunities; and (4) identifying the position and size of target markets. 2) increase portfolio (s2) brand portfolio refers to all brands offered by a company. by introducing various products and services, a company closes off the pathway for other brands to enter into the customers’ lives. today, many companies market their products under several brands which increases the variety of their products and brands so that they are associated with different groups of customers. therefore, it becomes possible to increase brand diversity ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 66 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 by using information technology for professional branding and diversity in production by creating attractive designs. 3) emphasis on the scientific management of mixing elements (4p) of marketing (s3) integrated marketing elements are a set of controllable marketing tools that the company combines to attempt to meet the target market. in other words, the marketing mix includes all the activities that the company can do to affect the demand for its products. the four ps of marketing are the key factors involved in the marketing of goods and services. they are the product, price, place, and promotion of a good or service. 4) generate research and development units (s4) this involves establishing research and development units (r&d) in the field of marketing and sales of home furniture. the alternative include the following solutions: supporting creativity and innovation, product quality development, designing new products based on high technology, increasing productivity and orientation of consumers' interests, motivations, and behavior. the anp and super decisions software were used to synthesize and analyze the model. 3.4 strategic criteria in this research, the merits of benefits, costs, opportunities, and risks are weighted by five general factors that fit into one of the following broad categories: development and technological (0.193), cultural and social (0.105), economic (0.299), political (0.246), and legal (0.154). an example of these comparative questions is shown in appendix figure a0. ratings of general factors are done by pairwise comparison of the lower level factors and summation of the main factors at the top level. the final weight of the strategic criteria shows that the economic criteria with the weight of 0.299 have the highest priority. the economic criteria have the highest impact on decision-making for marketing strategic planning in the furniture industry. 3.5 prioritizing bocr the process of decision making requires us to analyze a decision according to benefits (b), the good things that would result from making the decision; opportunities (o), the potentially good things that could result in the future from making the decision; costs (c), the pains and disappointments that could result from making the decision; and risks (r), the potential pains and disappointments that could result in the future from making the decision. there are four vectors, one of the merits is obtained by forming the ratio bioi/ciri for alternatives i from each of the four vectors. the synthesized ideals for all the control criteria under each merit may result in an ideal whose priority is less than one for that merit. only an alternative that is ideal for all the control criteria under a merit receives the value one after synthesis for that merit. the alternative with the largest ratio is chosen for some decisions. companies and individuals with limited resources often prefer this type of synthesis (saaty and vargas, 2011). to categorize the criteria which have been used in the current research, we divided them into favorable and unfavorable categories. the decision-maker considers the favorable criteria as benefits and the unfavorable criteria as costs. the possible events are also divided into opportunities and risks depending on whether they are considered to be ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 67 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 positive or negative (saaty, 2001a). since bocr are not equally important, it is necessary to prioritize them by using five possible ratings ranging from “very high” to “very low”. the results of the influence of the strategic criteria on the merits of benefits, costs, opportunities, risks, and the priority of the above-mentioned merits are reported in table 5. table 5 rating of the model to obtain bocr weighing values very high (1), high (0.51), medium (0.256), low (0.163), very low (0.057) benefits costs opportunities risks economics (0.299) very high high very high high development (0.193) high medium medium medium political (0.246) very high high very high high cultural (0.105) medium low medium medium legal (0.154) high high high high overall normalized priorities 0.325 0.183 0.304 0.188 3.6 control criteria network the network of control criteria under benefits, costs, opportunities, and risks are as follows: economics and marketing; technical and human; supply and production; social, cultural, and political. the structure of the framework for benefits, merits, and the control criteria and sub-criteria are extracted from super decisions software as illustrated in appendix figures a1-a5. the framework for costs, opportunities, and risks can be developed similarly. to apply the anp, the super decisions software was used. in the bocr structure the following formula is used in calculations (saaty, 2001a): (benefits)*(opportunities)/ (costs)*(risks) (1) 4. results and discussion after the hierarchy is drawn up for the criteria and sub-criteria that influence the selection of the appropriate alternative, the questionnaire was prepared to gather the opinions of the experts. the criteria and sub-criteria were evaluated based on pairwise comparisons. an example of these comparison questions is shown in appendix figure a6. then, the priority rate of each criterion and sub-criteria were compared. first, the geometric mean was calculated for each one of the matrix cells by the following formula (saaty, 2000). the bocr was ranked with the same method concerning overall factors in the previous section (table 1). group judgments aij = (aij1 ×aij2 ×…×aijn) 1/n i, j = 1, 2,…n (2) ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 68 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 where aij represents the comparison ratios between the element ith row and element jth column in the pairwise comparison matrices and n is the number of decision-makers. after the geometric means of all matrix cells were calculated, the results were normalized and the weights of the criterion and sub-criterion were obtained through the integration of the weight of the low-level elements into the weight of the related upper-level elements. the results for the geometric mean matrices and weighted supermatrix for benefits, costs, opportunities, and risks criteria and sub-criteria were extracted from super decisions software and the standard formula is the multiplicative formula. the results of the control criterion, economic and market, had the highest priority concerning benefits (0.325), costs (0.183), opportunities (0.304), and risks (0.188) for marketing strategic planning in the country's furniture industry. from the perspective of professors and activists in the home furniture industry, the factor of economic attractiveness motivates people to economic activity. 4.1. results of economic and market and control criteria of benefits the results show economic and market under benefits had a higher priority (0.311) in comparison to other control criteria (figure 3). in the sub-network of economic and market under benefits, exports (0.182) had the highest priority. the weighted values of the other criteria are as follows: marketing (0.176), branding (0.135), and infrastructure (0.122) (figure 4). if we export, we will obtain financial and political benefits in the form of currency in the country, and this is possible with marketing support. figure 3 results of control criteria under benefits 0.311 0.192 0.274 0.222 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 economic and market social cultural political supply and production technical and human weighting value c ri te ri a ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 69 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 figure 4 results of subnetwork under the economic and market of benefits 3.2. result of economic and market and control criteria of opportunities the results show economic and market under opportunities had a higher priority in comparison to other control criteria (figure 5). in the sub-network of the economy and market, exports (0.166) had the highest priority. marketing (0.154), infrastructure (0.124), branding (0.123), advertisements (0.109) and investment (0.096) are other leading factors (figure 6). by focusing on exports, we can enter the target markets, and with the help of marketing, we can achieve the highest sales. branding gives rise to a product that can differentiate itself from competitors with the help of advertising, providing after-sales service, and a product warranty. figure 5 results of control criteria under opportunities 0.135 0.176 0.094 0.086 0.122 0.112 0.182 0.091 0 0.05 0.1 0.15 0.2 branding marketing advertisement competitiveness infrastructure investment export sale & after sale weighting value c ri te ri a 0.428 0.102 0.289 0.181 0 0.1 0.2 0.3 0.4 0.5 economic ans market social cultural and political supply and production technical and human weighting value c ri te ri a ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 70 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 figure 6 results of subnetwork under economic and market of opportunities 4.3. results of economic and market and control criteria of costs the results indicate economic and market under costs had a higher priority in comparison with other control criteria (figure 7). in the economic and market sub-network, marketing (0.231) had the highest priority when compared with other criteria (figure 8). the reason that marketing had priority over other criteria is the role of its sub-factors (costs of participating in the exhibition, marketing of a new product, and lack of knowledge of market needs) because marketing operations and product introduction are costly. therefore, from the experts’ perspective, this is logically justified and causes this criterion to be selected as the most effective criterion when compared to other criteria. figure 7 results of control criteria under costs 0.154 0.123 0.109 0.095 0.124 0.096 0.166 0.132 0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 marketing branding advertisments competitiveness infrastructure investment export sale and services after sale weighting value c ri te ri a 0.316 0.184 0.275 0.225 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 economic ans market social cultural and political supply and production technical and human weighting value c ri te ri a ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 71 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 figure 8 results of subnetwork under economic and market of costs 4.4. results of economic and market and control criteria of risks economic and market under risks (0.39) had the highest priority when compared with other control criteria (figure 9). in the economics and market sub-network, export (0.152) had the highest priority (figure 10) followed by investment (0.132) and infrastructure (0.129). export criteria risks are higher than other criteria because there is a high risk that the cost of the product will increase and investors will not invest in the home furniture market due to unclear competition rules and regulations. on the other hand, the danger of having an information privilege in the home furniture market has made it impossible for an investor to encourage profitable production and export of his product. from the experts’ perspective, the export criterion had a higher priority in decision-making for investment and competition in iran and is therefore considered the most effective criterion when compared to other economic and market sub-criteria under the risks network. 0.231 0.169 0.176 0.201 0.223 0 0.05 0.1 0.15 0.2 0.25 marketing branding advertisments export sale and services after sale weighting value c ri te ri a ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 72 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 figure 9 results of control criteria under risks figure 10 results of subnetwork under the economic and market of risks 4.5. solutions 4.5.1 results of solutions concerning export under sub-network of economic and market control criteria of benefits the results indicate that emphasis on scientific management of mixing elements (4p) of marketing (s3) (0.307) had the highest priority concerning export under benefits (figure 11). generate research and development units (s4) (0.257), entrance into a foreign 0.39 0.134 0.277 0.199 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 economic and market social cultural and political supply and production technical and human weighting value c ri te ri a 0.115 0.114 0.111 0.126 0.129 0.132 0.152 0.119 0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 marketing branding advertisement competitiveness infrastructure investment export sale and services after weighing value c ri te ri a ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 73 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 market (s1) (0.226), and increase portfolio (s2) (0.209) are other leading solutions. if the company delivers the right product to the right customer at the right place at the right time at the right price (4p), it will make it easier for the customer to choose the product or service provided. as a result, these factors lead to customer satisfaction with the product or service provided, which in the long term increases the number of customers, while exporting the product leads to profitability. figure 11 results of solutions concerning export under sub-network of economic and market control criteria of benefits 4.5.2. results of solutions concerning export under sub-network of economic and market control criteria of opportunities increase portfolio (s2) (0.332) had the highest priority for export under opportunities (figure 12). entrance to a foreign market (s1) (0.257), emphasis on scientific management of mixing elements (4p) of marketing (s3) (0.231), and generate research and development units (s4) (0.179) are other leading solutions. by increasing the variety of goods and brands, it is possible to communicate with more groups of customers, and by attracting customers from abroad, the conditions for exporting the product in the future are provided. 0.226 0.209 0.307 0.257 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 s1 s2 s3 s4 weighting value a lt e rv a ti ve ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 74 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 figure 12 results of solutions concerning export under sub-network of economic and market control criteria of opportunities 3.5.3. results of solutions concerning marketing under sub-network of economic and market control criteria of costs concerning marketing under costs, increase portfolio (s2) (0.273) had the highest priority followed by emphasis on scientific management of mixing elements (4p) of marketing (s3) (0.259), entrance to a foreign market (s1) (0.241) and generate research and development units (s4) (0.226) (figure 13). a growth portfolio can take place over both the short and long term, but substantial growth in the short term generally carries a higher cost. figure 13 results of solutions concerning marketing under sub-network of economic and market control criteria of costs 0.257 0.332 0.231 0.179 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 s1 s2 s3 s4 weighting value a lt e rn a ti ve 0.241 0.273 0.259 0.226 0 0.05 0.1 0.15 0.2 0.25 0.3 s1 s2 s3 s4 weighting value a lt e rn a ti ve ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 75 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 4.5.4. results of solutions concerning export under sub-network of economic and market control criteria of risks the results show that entrance to a foreign market (s1) (0.315) had the highest priority concerning export under risks (figure 14). increase portfolio (s2) (0.24), emphasis on scientific management of mixing elements (4p) of marketing (s3) (0.233), and generate research and development units (s4) (0.211) are other leading solutions. entrance into foreign markets (s1) requires an additional commitment on the investor’s part. this twofold investment style, i.e., money only vs. money plus varying amounts of dedication, makes up the magnitude of the risk involved. money plus commitment entails traditional modes of foreign market entry which includes high risks for investors. figure 14 results of solutions concerning export under sub-network of economic and market control criteria of risks 4.6. final outcome the final result of the strategic criteria shows that economic factors are the most important with a weight of 0.299. for managers and employers of the country's home furniture industry, these factors are more effective than other factors in influencing people's decision-making. also, bocr indicates that benefits have higher significance than other competencies regarding the marketing strategy planning of the home furniture industry. in terms of selection, the third solution (s3) or emphasis on scientific management of mixing elements (4p) of marketing is considered the best solution. by integrating the weights of the merits of benefits, costs, opportunities, and risks and the weights of choices against the above-mentioned merits, the final scores are reported in table 6. 0.315 0.24 0.233 0.211 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 s1 s2 s3 s4 weighting value a lt e rn a ti ve ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 76 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 table 6 final outcome for priorities of the alternatives merits alternatives benefits (0.325) opportunities (0.304) costs (0.183) risks (0.188) final outcome additive ranking s1 0.261 0.255 0.242 0.258 0.265 2 s2 0.234 0.247 0.261 0.252 0.219 3 s3 0.284 0.281 0.261 0.250 0.304 1 s4 0.219 0.215 0.235 0.238 0.210 4 as table 6 shows, emphasis on scientific management of mixing elements (4p) of marketing (s3) had the highest priority and is the most suitable solution for marketing strategic planning in the country's furniture industry followed by entrance to a foreign market (s1), increase portfolio (s2), and generate research and development units (s4). accordingly, we can list the following concerning the control criteria: 4.6.1. economic and market index in the field of economics and market, s3 the capacity of graduates, can be used to design and implement internal marketing in the field of home furniture. on the other hand, effective and targeted advertising helps influence consumer's attitudes and behavior. these conditions create demand and increase market share and stabilize domestic markets. the result is attracting, retaining, and increasing customers and increasing sales and company's profitability. thus, by planning and applying this solution, it is possible to identify marketing opportunities and help market products. finally, with a proper pricing strategy and product warranty, customer satisfaction is achieved and sales and profitability of companies is increased in the long term. 4.6.2. supply and production index in the field of supply and production, by applying the s3 solution, the supply chain for materials is shortened and more local raw materials are used. increasing the quality and accuracy of production and process optimization can lead to a reduction in production costs and an increase in production speed. naturally, it results in timely delivery of the product to customers which creates customer satisfaction, and can help increase profits in the long run. by emphasizing the third solution, it is possible to improve the quality of the product by moving within the boundaries of marketing knowledge and receiving quality certificates, which builds trust with customers. 4.6.3. technical and human index in the technical and human field, by emphasizing the s3 solution, it is possible to produce beautiful and distinctive products compared to those offered by competitors, which satisfies customers through proper management of human resources and ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 77 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 innovation and creativity in the production process. by applying s3, it is possible to promote the position of innovation in the marketing strategy of companies by using new technical knowledge and innovation in marketing. establishing marketing companies by graduates activates the market for related academic disciplines. 4.6.4. social, cultural, and political index in the social, cultural, and political spheres, the use of the s3 solution causes the change and development of consumption patterns, and production of home furniture in line with the culture and needs of society. in this case, the needs of the target customers will be met while increasing trust in the national brand, which can lead to growth in consumption of home furniture in the country. 4.7. sensitivity analysis since there may be different judgments about the comparison of priority rates of benefits, opportunities, costs, and risks or their sub-criteria, saaty (2001b) suggests that a sensitivity analysis of the results be performed. to perform a sensitivity analysis, we applied software developed by saaty (2001b). the results are illustrated in table 7. from table 6, the solution emphasis on scientific management of mixing elements (4p) of marketing (s3) had the highest priority and is the most suitable solution for the marketing strategy planning of the furniture industry followed by entrance to a foreign market (s1), increase portfolio (s2), and generate research and development units (s4). after changing the weights of one criterion, the priorities also change as shown in table 7. table 7 results of the sensitivity analysis s3>s1>s2>s4 merits basic weight new weight number of changes new priorities benefits 0.325 costs 0.183 0.268 0.339 0.463 0.631 4 s3>s1>s4>s2 s1>s3>s4>s2 s1>s4>s3>s2 s4>s1>s3>s2 opportunities 0.304 0.103 1 s3>s1>s4>s2 risks 0.188 0.328 0.415 0.565 0.631 4 s3>s1>s4>s2 s3>s4>s1>s2 s3>s4>s2>s1 s4>s3>s2>s1 costs and risks are more sensitive than benefits and opportunities because the sensitivity analysis of costs and risks shows (table 7) that there are four changes in the priority of ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 78 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 the alternatives, while the sensitivity analysis of the opportunities shows that there is only one change in the priority, and there are not any changes in the priorities regarding benefits. figures a7 and a8 in the appendix show a sensitivity analysis of benefits, costs, opportunities, and risks which has been done in super decisions software. a sensitivity analysis is based on the change in the weight of the indicators relative to their base weight, provided that the priority of the option changes in the new weight of the index. like the cost index, the priorities of the solutions have changed four times due to weight changes. the choice of sensitive indicators depends on the number of changes in the prioritization of options so that more sensitive indicators cause the most change in the prioritization of solutions. 5. conclusion to present the best approach for the marketing strategy planning of the home furniture industry, the four following options were analyzed through a combination of multicriteria decision techniques: entrance to a foreign market (s1); increase portfolio (s2; emphasis on scientific management of mixing elements (4p) of marketing (s3); generate research and development units (s4). in the first stage, the strategic criteria were identified and based on these criteria, the weighted values of bocr (benefits, opportunities, costs, and risks) were determined. the weighted values of the merits were derived from the influence of the strategic criteria on benefits, costs, opportunities, and risks in the second stage. in the third stage, the weighted values of the alternatives derived in the prior stages were synthesized using the anp and super decisions software. we concluded that the anp with a specific emphasis on scientific management of mixing elements (4p) of marketing (s3) is the proper solution to apply to the marketing strategy planning of the furniture industry. emphasis on scientific management of mixing elements (4p) of marketing had the highest priority in parts of benefits, opportunities, and costs. there were four main control criteria in the research. concerning the economics and market index, s3 the capacity of graduates, can be used to design and implement internal marketing in the field of home furniture. regarding the supply and production index, by applying s3, the supply chain of materials is shortened and more local raw materials are used. in terms of the technical and human index, by applying s3, it is possible to produce beautiful and distinctive products compared to competitors that satisfy customers through proper management of human resources and innovation and creativity in the production process. from the social, cultural, and political point of view, the use of the s3 solution results in a change and development of consumption patterns and production of home furniture in line with the culture and needs of society. the results of the sensitivity analysis revealed that the costs and risks are more sensitive than opportunities and risks. the results can be used since there may be different judgments about the comparison of priority rates. based on the investigation, a suitable solution was obtained to sell the products of the home furniture industry, which can effectively help find a market for the sale of these products. at the same time, this strategy is very dynamic because it determines how it is possible to change the prioritization of solutions according to changing conditions in the social, economic, and political environments of the decision-makers. this issue is also ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 79 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 very important for decision-makers and managers who are familiar with the different decision-making methods because the anp can provide modeling, and this method is very flexible for this type of program. a limitation in this study was some home furniture experts’ unfamiliarity with the criteria defined in this model. therefore, more time was required to justify the criteria and help the experts become familiar with this model which was necessary in order to obtain their opinions. meanwhile, a new strategy was developed for the manufacturers; therefore, more information about the characteristics of the method should be given to this group. ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 80 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 references aaker, d.a. (2010). strategic market management: global perspective. john wiley & sons. alves, s.n.l., simões, s.a.c., neyra, b.m.c. (2008). literature review about anp method application in problem of supplier selection. brazilian symposium of operational research. september 2–5, joão pessoa, brazil. anderson, p.f. (1982). marketing strategic planning and the theory of the firm. the journal of marketing, 46(2), 15-26. doi: https://doi.org/10.1177/002224298204600203 ahmed a. (2019). the importance of a strategic marketing plan. leaf group ltd. https://bizfluent.com/about-7217955-importance-strategic-marketing-plan.html azizi m. and mehdikhanloo g. (2018). applying anp to analyze the role of design in the furniture industry, international journal of the analytic hierarchy process, 10(2), 178-218. doi: https://doi.org/10.13033/ijahp.v10i2.588 azizi, m., and modarres m. (2011). a decision-making model for investment and development of construction panels, journal of forestry research, 22(2), 301-310. doi: 10.1007/s11676-011-0166-8. azizi, m., ray, c., (2016). proposing a decision model for privatization of newsprint paper industry by applying anp, cogent business & management, 3(1), 1-19. doi: http://dx.doi.org/10.1080/23311975.2016.1197088. azizi m., ray c. (2015). introducing a status of access to raw material resources for manufacturers of cellulosic industries, forest policy and economics, 51, 17-28. doi: 10.1016/j.forpol.2014.11.008. baker m. (2008). the strategic marketing plan audit. cambridge strategy publications. cravens, d.w. (2006). strategic marketing, vol. 7. new york, ny: mcgrawhill. ghofrani m., zare a., azizi m. (2017). identifying and prioritizing indicators and effective solutions to optimize the wood consumption in the construction of classical furniture by using ahp (case study: qom province). iranian journal of wood and paper industries, 7(4), 573 -589. ghofrani, m., farshchi, v., and azizi, m. (2014). investigate and identify the indicators of customer loyalty to the brand in the furniture industry by ahp. iranian journal of wood and paper science research, 29(2), 310-323. home and office furniture exporters union (2009). exporters union newsletter, 18th hofex international exhibition. https://bizfluent.com/about-7217955-importance-strategic-marketing-plan.html https://doi.org/10.13033/ijahp.v10i2.588 http://dx.doi.org/10.1080/23311975.2016.1197088 ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 81 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 huang, r. (2014). how brand awareness relates to market outcome, brand equity, and the marketing mix. in fashion branding and consumer behaviors. springer (113-132). doi: https://doi.org/10.1007/978-1-4939-0277-4-8 kao, l., chiu, c., lu, c. & kao, h. (2015). applying analytic network process to explore consumer adoption of radical innovation: a case study of smart home technologies. 23 rd international conference on multi-criteria decision making (mcdm 2015). hamburg, germany. lake l. (2019). strategic marketing plan, components, and benefits. https://www.thebalancesmb.com/what-is-strategic-marketing-plan-4043393 levy, m. w. (2012). retailing management (vol. 6). new york: mcgraw-hill/irwin. londhe, b. r. (2014). marketing mix for next-generation marketing. procedia economics and finance, 335-340. doi: https:// doi.org/10.1016/s2212-5671(14)00201-9 mcdonald m. (2006). the marketing review. westburn publishers ltd. mehta v. (2019). the role of strategic marketing in an organization. berlin school of business and innovation (bsbi). www.berlinsbi.com/en/blog/career-advice/the-role-ofstrategic-marketing-in-an-organisation. patil, d. d. (2017). marketing-mix for strategy building. journal of multidisciplinary engineering science and technology (jmest), 4(4), 7146-7149. perreault, w.d. (2011). basic marketing: a marketing strategy planning approach. new york, ny: mcgrawhill. petersen, j. a. (2015). marketing communication strategies and consumer financial decision making: the role of national culture. journal of marketing, 44-63. doi: http://doi.org/10.1509/jm.13.0479 petukhov a. (2015). a development strategy for a furniture company during crises period, a thesis. centria university of applied sciences. saaty t. (2001a). decision making with dependence and feedback: the analytic network process. pittsburgh, pa: rws publications. saaty t. (2001b). decision on national missile defense program, 6th international symposium on the ahp. bern, switzerland. saaty, t. (2005). theory and applications of the analytic network process. pittsburgh, pa: rws publications. saaty, t., vargas l., (2011). the encyclicon (volume 3): a dictionary of complex decisions using the analytic network process. pittsburgh, pa: rws publications. http://doi.org/10.1509/jm.13.0479 ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 82 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 saaty, t.l., vargas, l.g. (2006a). decision making with the analytic network process. new york, new york: springer science. saaty, t. (2006b). rank from comparisons and from ratings in the analytic hierarchy/network processes. european journal of operational research, 168, 557-570. doi: https://doi.org/10.1016/j.ejor.2004.04.032 tsai, w., leu, j., liu, j., lin, s., & shaw, m. (2010). a mcdm approach for sourcing strategy mix decision in it projects. expert systems with applications, 37, 3870-3886. doi: https://doi.org/10.1016/j.eswa.2009.11.031 taliscali, a.k., ercan, s. (2006). the analytic hierarchy & the analytic network processes in multicriteria decision making: a comparative study, journal of aeronautical and space technology, 2(4), 55–65. westwood, j. (2013). how to write a marketing plan. kogan page publishers. zolghadr, m. (2009). export orientation of iranian wood products and furniture, the 2 nd book of the bank of iran wood and paper and furniture magazine. tehran, iran. ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 83 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 appendix figure a0 sample questionnaire of comparison of strategic criteria under goal ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 84 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 figure a1 control criteria network under benefits ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 85 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 figure a2 sub network under benefits/ economics and market ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 86 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 figure a3 subnetwork under benefits/supply and production ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 87 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 figure a4 subnetwork under benefits/technical and human ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 88 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 figure a5 subnetwork under benefits/social, cultural, and political ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 89 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 figure a6 sample of the questionnaire ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 90 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 figure a7 sensitivity analysis concerning benefits and costs ijahp article: azizi, mansouri/analysis of marketing strategic planning in the home furniture industry by applying anp: a case study of the iranian home furniture industry international journal of the analytic hierarchy process 91 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.813 figure a8 sensitivity analysis concerning opportunities and risks microsoft word 1b_words_from_the_editor_vol2_issue1.docx ijahp preface: mu/words from the editor-in-chief of the ijahp international journal of the analytic hierarchy process vol. 2, issue 1, 2010 issn 1936-6744       words from the editor‐in‐chief  i am pleased to introduce our first special issue about the status of ahp/anp applications in brazil. this  issue has a distinguished guest editor, prof. valerio salomon, currently an associate professor at sao  paulo state university, who has had vast experience in the field of ahp/anp. valerio salomon obtained  his ph.d. in industrial engineering from the university of sao paulo (usp), brazil, in 2004. he teaches  industrial engineering in graduate and undergraduate courses at the engineering school of  guaratingueta. prof. salomon was a visiting scholar at the university of pittsburgh from august to  september 2007. his areas of interest include the ahp and fuzzy sets. because of all this, i feel confident  this special issue is in very good hands. enjoy!    enrique mu, phd  editor‐in‐chief  ijahp      rob typewritten text http://dx.doi.org/10.13033/ijahp.v2i1.66 rob typewritten text rob typewritten text rob typewritten text rob typewritten text ijahp: mu/ on the nature of essays international journal of the analytic hierarchy process 147 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.345 august 2015 on the nature of essays dear ijahp reader: this is our second issue in our new publication frequency of three issues a year. it includes, as usual, several articles on the theory and practice of ahp/anp. furthermore, it also includes a thought-provoking essay by claudio garuti titled “parallel between space geometry & decision making space.” a comment on our policy for essays may be in order. aldous huxley, a famous essayist, used to say that an essay is a literary device to say almost everything about almost anything. we agree with this definition and for this reason you will notice a great variety of topics have been published in the essay section of this journal. an essay does not follow the standard double-blind review process, but is accepted/rejected by the editorial team based on its relevance, originality and thought-provoking potential. in other words, it is accepted based on trust in the author. in any case, we prefer to err on the side of creativity rather than on the side of clarity or, god forbid, even accuracy. our main goal for an essay is to get the reader to think in a different way. if you think you have anything to say about something that fulfills the above criteria, you are welcome to submit it to our journal. thanks for your support and enjoy this issue! enrique mu, phd ijahp editor-in-chief ijahp: mu/ beyond the information age and into the new decision age international journal of the analytic hierarchy process 152 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.690 beyond the information age and into the new decision age beyond the information age was the theme of the mcdm 2019 meeting in istanbul this past june. what does the theme mean? a quick consultation with dr. ilker topcu, conference chairman, revealed that the theme came from a discussion among the organizers of the conference about what is next after the information age. in effect, we live in an age characterized by a shift from industrial production to information processing. still, any age in human history is expected to come to an end and be replaced by another one. so, the question: “what comes next?” is perfectly valid. while the mcdm organizers did not reach an agreement on what is next, the fact that they chose beyond the information age as a theme for the most important conference on decision-making is quite telling. i think the message is that now that we have large amounts of information and easy communication available, the so-called information and communication technology (ict), the next step is to be able to use this capability for effective decision-making. we are still far from this as shown by the passionate, but many times low-level public discourse about problems such as climate change, migrations, pandemics and others. based on this, i propose that the next age should be called the decision age, which could be defined as an age in which our current ict capabilities are used to make effective decisions at all levels. this is the implicit message of the mcdm choice of theme this year. we are far from this kind of routinely effective decision-making. in our information age, too much information may lead to excessive analysis, information overload, lack of strategic focus and illusory knowledge (quickly accessed information that may mislead us into thinking we now know about the topic when in reality we have just scratched the surface). finally, decision-making analysis is still practiced mainly by specialists who many times lack the power or ability to convince the actual decision-makers to take the best course of action. therefore, we celebrate the choice of the theme beyond the information age for mcdm2019 and ijahp welcomes the opportunity to contribute to the upcoming decision age with the publication of our second issue this year. beginning in our next issue, selected peer-reviewed ahp/anp articles submitted from mcdm2019 will be published. very sincerely, enrique mu, phd ijahp editor-in-chief microsoft word mu award ijahp news and events: editor-in-chief enrique mu earns best paper award at isahp 2013 international journal of the analytic hierarchy process 175 vol. 3 issue 1 2011 issn 1936-6744 ahp news ijahp editor-in-chief enrique mu earns best paper award at the 2013 isahp symposium in malaysia for his paper “a new approach to eyewitness identification,” dr. enrique mu, editor-in-chief of the international journal on the analytic hierarchy process, received the best paper award at the 12th isahp symposium in kuala lumpur, malaysia, june 23-26, 2013. the paper was written with his colleague, dr. rachel chung, both professors at carlow university, pittsburgh, pennsylvania. their research applies the analytic hierarchy process (ahp), a prioritization and selection methodology used in operations research, to the problem of eyewitness identification in the field of law enforcement. this year’s isahp symposium attracted about 150 participants, ahp multicriteria decision-making experts, representing more than 30 countries worldwide, who delivered more than 95 papers. in the award process, as explained by dr. rafikul islam, isahp 2013 program chair, a short list of seven papers was identified on the basis of the referees’ feedback from the first round review of papers. the shortlisted papers were then sent to three referees, who considered a paper’s originality, contribution to the discipline, coherence (including ijahp article: mu, saaty/a style guide for papers submitted to the international journal of the analytic hierarchy process 176 methodology), literature review, and writing, to select the best paper in the second and final decision round. treating eyewitness identification as an ahp decision-making problem promises to increase the ratio of successful identification by eyewitnesses by approximately 50%. more importantly, mu and chung’s study suggests that false identifications could decrease in a similar proportion. this is an important contribution to the law enforcement field, given that more than 70% of innocent people recently released from jail, after being exonerated from any crime based on dna evidence, were found guilty originally because of wrong eyewitness identification. although more experiments are still needed to make these preliminary findings conclusive, it is a very promising line of research. dr. mu is an associate professor in carlow university’s mba program as well as director of the summer executive training program at the university of pittsburgh where he is also an affiliated faculty member. he is an accomplished scholar and educator, who is editor-in-chief of the international journal of the analytic hierarchy process (ijahp) dedicated to the diffusion of effective decision making. rob typewritten text http://dx.doi.org/10.13033/ijahp.v5i1.182 rob typewritten text ijahp news and events: ferretti, valentina / informs 2012 meeting, phoenix, arizona international journal of the analytic hierarchy process 166 vol. 4 issue 2 2011 issn 1936-6744 valentina ferretti at the grand canyon, arizona informs 2012 highlights the informs annual meeting took place last month in sunny and fascinating phoenix, arizona, usa, october 14-17, 2012. the general theme of the conference was informatics rising and if you were not lucky enough to attend it, you can update yourself on all of its interesting sessions by following this link: http://meetings2.informs.org/phoenix2012/. with about 4000 participants coming from all over the world, the conference covered the broad landscape of operation research and practice and brought together different research areas, applications and perspectives into a truly international meeting. taking into account that the mission of the ijahp is to advance the use, education and development of the analytic hierarchy/network process (ahp/anp) for decision-making, it is worth highlighting that this year there have been three ahp/anp sessions during the informs meeting. in a nutshell, the first ahp/anp session, titled “ahp/anp applications” and chaired by professor birsen karpak, took place on sunday the 14th and brought together contributions ranging from spatial anp in the context of undesirable facilities location problems (ferretti v.), to the evaluation of the performance of the brazilian financial managers through the ahp (autran l.f. and andrade r.), to the evaluation of emergency distribution centers (ozaydin o. and topcu i.), to cloud computing vendor selection for the city of pittsburgh (mu e. and stern h.). the second ahp/anp session, titled “applications of the analytical network process” and chaired by professor orrin cooper, took place on monday the 15th and included contributions concerning moral and ethical decision-making processes (adolphson d. and cooper o.), the ability to measure intensity of preference (sava g.m. and vargas l.g.), fuzzy ahp and multi-objective problems (meng c., son y.j. and xu d.), and criteria weights dependencies from the alternatives (cooper o. and liu g.). the third ahp/anp session, titled “ahp/anp for sustainability” and chaired by professor birsen karpak, took place on wednesday the 17th and brought together research and applications in the information system field (taraszewski s. and karpak b.), for third party logistic companies evaluation (bayazit o. and karpak b.), in the sustainable supply chains domain (karpak b. and lamb c.) and in the women owned small businesses field (mcmahon a. and karpak b.). http://meetings2.informs.org/phoenix2012/ ijahp news and events: ferretti, valentina / informs 2012 meeting, phoenix, arizona international journal of the analytic hierarchy process 167 vol. 4 issue 2 2011 issn 1936-6744 all these sessions have the merit of having provided very interesting applications in many different fields, thus stimulating the debate on further research concerning methods to make better decisions and supplying important cues for future developments. valentina ferretti department of housing and city (dicas) politecnico di torino italy e-mail: valentina.ferretti@polito.it rob typewritten text http://dx.doi.org/10.13033/ijahp.v4i2.158 ijahp editorial: mu, e./words from the editor-in-chief international journal of the analytic hierarchy process 1 vol.5 issue 1 2013 issn 1936-6744 july 2013 dear ijahp readers: welcome to the first issue of 2013! this year also marks our fifth year of regular publication. it has been quite a hectic first half of the year. the mcdm conference in málaga, spain and the 12th isahp 2013 in kuala lumpur kept ahp/anp researchers and practitioners quite busy meeting deadlines and presenting their work. we have included notes about the highlights from these two conferences in this issue to keep you informed. in particular, this editor was surprised and humbled about being awarded the best paper at isahp 2013. this award is a strong incentive for all of us to keep working to spread the ahp/anp as an important decision-making tool in all areas of human endeavor. speaking of spreading the word about decision-making, i cannot thank the ijahp editorial team enough. these dedicated people have worked particularly hard on this issue and have been key to the success of the publication as a whole: rozann saaty, antonella petrillo, and birsen karpak. besides peer-reviewed articles, our journal includes an essay and a news and events section. we encourage our readers to submit contributions to all the sections of the journal. we are also working on many other initiatives to improve the standing of the journal. some of these initiatives include the inclusion of digital object identifiers (dois) for articles, an application to scopus, and invitations to several outstanding members of our ahp/anp community to join the editorial board. the updated list of board members is included on the ijahp home page. the members of the editorial board will serve for a three-year period, and will assist in the editorial process on an as-needed basis. we cannot thank them enough for their willingness to join our team. this issue is comprised of an eclectic collection of ahp/anp articles. first, jain and rao perform a very comprehensive review of the literature as well as present their research of experts’ opinions to develop a very thorough model for the prioritization of different methods for rural extraction of anti-cancer bioactive compounds from plants. next, we have a discussion by oyatoye and daniel of critical success factors (csfs) for poverty ijahp editorial: mu, e./words from the editor-in-chief international journal of the analytic hierarchy process 2 vol.5 issue 1 2013 issn 1936-6744 reduction programs in africa. using a survey approach with 950 respondents, they conclude that past poverty reduction programs have not considered several csfs and alternatives. in our third paper, beshah and kitaw propose a model for allowing banks to make financing decisions from among multiple competing projects. in our fourth paper, lo, tsain and lin provide a very impressive literature review to develop a decision model for selecting suitable management consultants. finally, stan lipovetsky completes his discussion of the supermatrix eigenvector problem and its interpretation of priorities, this time for the case of the analytic network process. to conclude this issue, dr. thomas saaty discusses, in our essay section, the worldwide challenges of moving from fragmentation to integration. this is further discussion on the topic that he addressed in his opening speech at our 12th isahp 2013 meeting in kuala lumpur. i feel confident you will enjoy this journal issue. many thanks! enrique mu, phd editor-in-chief, ijahp rob typewritten text http://dx.doi.org/10.13033/ijahp.v5i1.188 ijahp preface: mu/do we need more ahp/anp studies? international journal of the analytic hierarchy process 403 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.451 do we need more ahp/anp studies? recently, a paper submitted to a major journal was rejected by the editor with the indication that the ahp method was very well known so “another ahp study” was not needed for the time being. being a scholar who also publishes in areas beyond ahp/anp, i was quite surprised by this argument. i have never seen a paper rejected with the argument that “structural equation modeling (sem) is a well-known method so we do not need “another sem study.” the ahp/anp are tools that have been around for several decades, and have proven to be very suitable as multi-criteria decision-making methods to address all varieties of managerial and social problems and to integrate multiple opinions from a great variety of stakeholders. in a world where problems become more complex over time and where opinions are becoming polarized, our opinion would be that the number of studies addressing problematic decisions should increase proportionally. therefore, we consider that on the contrary, we need more ahp/anp studies. the issue is not if the method is well-known, but rather if the target problem has been solved. for that reason, the international journal of the analytic hierarchy process does not determine whether to accept or reject an article based on how sophisticated or exotic the methodology is but instead on whether it does a good job addressing the target problem. the day when there are less decisions to make instead of more will be the day when we can say, “we need less ahp/anp studies”. however, that day seems still very far from now. consistent with the above, this december issue brings eight new ahp/anp studies. the issue begins with a review of the literature for ahp/anp sports applications by nisel and ozdemir from istanbul university, and is followed by an interesting simulation of a tsunami evacuation by kohara and sugiyama from the chiba institute of technology in japan. next, there are four very practical applications: a vendor’s selection for an indian steel pipe (kamath, barkur et al); determination of overall stress of policemen (oneren, arar, et al); a study to improve pedestrian mobility to revitalize traditional markets in south korea (ching and yoon), and a study of how to prioritize performance measures using ahp by r. vachnazde from the free university of tbilisi in georgia. our last two articles to close this issue are more theoretical in nature: stan lipovestky extends ahp by incorporating the best-worst scaling marketing research technique and demonstrates ijahp preface: mu/do we need more ahp/anp studies? international journal of the analytic hierarchy process 404 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.451 its use by tackling the classic secretary problem. finally, claudio garuti reports on the new advances of using the compatibility index g in weighted environments. our ijahp team believes there is enough variety in this issue to keep our readers engaged in reading ahp/anp studies over the holiday break. happy holidays! enrique mu, phd ijahp, editor-in-chief ijahp article: ekmekci, aragonés-beltrán, rodríguez de sanabria gil, topcu /a decision model for determining public locations suitable for aed deployment: a case study in the city of valencia international journal of the analytic hierarchy process 151 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.492 a decision model for determining public locations suitable for automated external defibrillator (aed) deployment: a case study in the city of valencia esin ekmekci istanbul technical university, department of industrial engineering istanbul, turkey esinekmekci@gmail.com pablo aragonés-beltrán universitat politécnica de valència, department of engineering projects valencia, spain aragones@dpi.upv.es rafael rodríguez de sanabria gil universitat politécnica de valència, department of business administration and management valencia, spain rarodrig@upvnet.upv.es y. ilker topcu istanbul technical university, department of industrial engineering istanbul, turkey ilker.topcu@itu.edu.tr abstract automated external defibrillators (aed) offered a new perspective on resuscitation by enabling first-responders to deliver life-saving defibrillation within the critical first minutes after sudden cardiac arrest (sca). this raised the question about where to place the aeds. this study aims to provide a novel approach to the problem and to serve as a guideline for health policy decision makers in future projects. we used the analytic hierarchy process (ahp) to form a decision model with four main criteria, and six subcriteria. “response time” was the most important criterion with an importance of 65.07%. locations with the highest scores were a sports center, two stadiums, the central market, and the central bus station. a simple ranking of the alternatives would not be sufficient for the purpose of this study because the aim is to offer a guide for selecting locations for deploying aeds rather than ranking the alternatives according to their suitability. therefore, we formed priority groups. sensitivity analysis showed that especially the alternatives in the first-priority group are not highly sensitive to changes, emphasizing their importance. keywords: automated external defibrillator (aed); public access defibrillation (pad); sudden cardiac arrest (sca) mailto:esinekmekci@gmail.com mailto:rarodrig@upvnet.upv.es mailto:ilker.topcu@itu.edu.tr ijahp article: ekmekci, aragonés-beltrán, rodríguez de sanabria gil, topcu /a decision model for determining public locations suitable for aed deployment: a case study in the city of valencia international journal of the analytic hierarchy process 152 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.492 1. introduction early treatment is the key success factor for resuscitation of sudden cardiac arrest (sca) patients since survival chances decrease by about 10% with every minute without defibrillation (nolan et al., 2010). automated external defibrillators (aed) have proven to be safe and effective for use by not only healthcare professionals but also laypeople. public-access defibrillation (pad) is critical for improving survival rates of out-ofhospital sca (folke et al., 2009). a recent study shows that even the lay aed response without any cpr may increase survival rates (capucci et al., 2016). the appropriate placement of aeds is the key to improving the effectiveness of pad programmes (nielsen et al., 2013). locating aeds has been an important topic in emergency medicine. however, most studies are retrospective and the topic has not been studied thoroughly in the multicriteria decision making (mcdm) domain. in this study, we aim to provide a guideline by using the analytic hierarchy process (ahp), a well-known and widely used mcdm technique (saaty, 1977). section 2 provides an overview of previous studies, and section 3 discusses the methodology used. section 4 elaborates on the case study and the decision model while section 5 presents our findings. section 6 draws some conclusions and provides suggestions for future research. 2. background 2.1 locating aeds studies about aeds mainly focus on determining the cost-effectiveness of the units or equitable allocations of the aeds (myers & mohite, 2009; cram et al., 2003; rauner & bajmoczy, 2003). these mostly analyze the cost per qaly (quality-adjusted life-year) gained (folke et al., 2009; nichol et al., 2009). the literature reviewed suggests that this issue has not been studied thoroughly in the mcdm domain. we did not find any study using ahp for determining suitable aed locations. most studies focus on historical data of sca incidents and propose these high-risk locations for aed deployment (folke et al., 2009; nielsen et al., 2013; chan et al., 2013; gilchrist et al., 2010; muraoka et al., 2006; page et al., 2013). there are also studies focusing on specific locations such as university campuses, basketball arenas, and public gardens (myers & mohite, 2009; motyka et al., 2005; imai et al., 2008). 2.2 criteria sca risk is the most common criterion used for determining aed locations in the literature as it is intuitional to link high sca risk with the need of having an aed in that location. many researchers link this criterion to historical data of sca incidents and advise a location type for aed installation if there is a past incidence of sca in that location (folke et al, 2009; nielsen et al, 2013; muraoka, 2006; brooks et al., 2013; norton & norton, 2008). tsai et al. (2012) determine not only the location, but also the time of sca incidents throughout the city to match high-risk locations with the shops of a market chain with the aim of finding which shops are suitable for aed deployment. motyka et al. (2005) focus merely on the possible response time of a cardiac arrest ijahp article: ekmekci, aragonés-beltrán, rodríguez de sanabria gil, topcu /a decision model for determining public locations suitable for aed deployment: a case study in the city of valencia international journal of the analytic hierarchy process 153 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.492 incident for determining suitable aed locations in a basketball arena. studies focusing on university campuses suggest using coverage capability, response time, or a combination of both (myers & mohite, 2009; timmons & crosbie, 2014; whitneycashio et al., 2012). siddiq et al. (2013) introduce the term “effective range”, defined as “the maximum straight-line distance between the aed and the cardiac arrest such as that there is a reasonable probability of retrieval and use prior to ems arrival”, and they run a mathematical model to achieve the highest coverage based on retrospective incident data and pre-defined aed effective range values. some studies focus on first-aid responders (i.e. potential users of the aed in case of an emergency) and the importance of a trained user being available (lubin et al., 2004; myerburg et al., 2005; schneider et al., 2004). some studies use a combination of criteria such as “high population density or high rate of previous sca incidents”, “population density, risk level of people and response time”, “high population density, emergency medical services (ems) response time being longer than 5 minutes and the expectance of at least one sca incident in 5 years”, or “high population, sca risk and availability of user” (becker et al., 1999; balady et al., 2002; gilchrist et al., 2010; colquhoun et al., 2008). 2.3 alternatives aeds in airports, airplanes and casinos have proven to be effective in prior studies, and major train stations, public squares and pedestrianized areas are places of frequent incidents of sca (folke et al., 2009). in a research study, covering a 28-month period, sports facilities, transportation facilities and other public areas were the locations with most sca incidents (nielsen et al., 2013). brooks et al. (2013) state that retail stores, offices, shopping centers, industrial areas, race tracks/casinos, jails, hotels, hostels/shelters, convention centers, railway stations, campuses, sports arenas, swimming pools, and golf courses have high sca risk. muraoka et al. (2006) recommend railway stations, hospitals, elderly homes, playgrounds and golf courses for aed deployment. a study covering six major cities in the netherlands shows that shops, catering facilities, hotels, museums, municipality buildings, cinemas, companies, office complexes, and educational institutions were locations most frequently chosen for aed deployment (huig et al., 2014). according to a retrospective study, railway stations, nursing homes, medical facilities, and fitness facilities are the places where aed use is most frequent (sasaki et al., 2011). another study states that it would be reasonable to deploy aeds in schools, fitness centers, sports arenas, stadiums, and locations of temporary sports events (myerburg et al., 2005). shopping malls, work places, exhibition halls, airports, harbor terminals, railway stations, sports arenas, fitness centers, golf courses, community centers, elderly homes, and jails are locations of relatively frequent sca incidents, and would benefit from aed deployment (becker et al., 1999). it was reported that aeds were installed in public locations, airports, sports arenas and golf courses in los angeles (eckstein, 2012). a study reviewing the early results of the national defibrillator program in uk and wales states that busy public places such as airports and major railway stations, sports facilities, workplaces, shopping centers, exhibition halls and major sporting venues are equipped with aeds (colquhoun et al., 2008). there are also studies focusing on a particular location type and assessing risks and benefits of aed deployment in these locations, such as fitness facilities and sports ijahp article: ekmekci, aragonés-beltrán, rodríguez de sanabria gil, topcu /a decision model for determining public locations suitable for aed deployment: a case study in the city of valencia international journal of the analytic hierarchy process 154 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.492 venues, schools, subway stations, and university campuses (page et al., 2013; muraoka et al., 2006; norton & norton, 2008; balady et al., 2002; berger, 2009; sweoret al., 2013; watson et al., 2013; gianotto-oliveira et al., 2015; myers & mohite, 2009; whitneycashio et al., 2012; aragonés-beltrán et al., 2014). 3. methodology determining appropriate locations for aeds is a problem of a complex nature with multiple criteria and alternatives, requiring a multi-criteria decision making (mcdm) approach to solve it. the analytic hierarchy process (ahp) is an mcdm method developed by saaty (saaty, 1977). it structures the decision problem as a hierarchy consisting of multiple levels: the goal, criteria (together with sub-criteria if necessary) and alternatives. saaty and vargas (2012) explain the rationale of ahp as the hierarchical decomposition of complex systems being a basic device which is already used by the human mind to cope with diversity. for this reason, the ahp was chosen as the method to be used in this study. ahp is considered an easy-to-use tool by many researchers, and is used in various domains such as marketing, finance, education, public policy, economics, medicine, and sports (wu et al., 2007). ahp is preferred mainly because it allows decision makers to analyze complex problems with a systematic approach that breaks down the problem into levels which make it simpler and more affordable (aragonés-beltrán et al., 2014). the main strong point of ahp is that it enables handling not only tangible but also intangible criteria. on the other hand, ahp has also been subject to criticism, mainly because of the time it requires for a decision maker to make pairwise comparisons in complex models containing many levels (wu et al., 2007). ahp may also be used for rating alternatives according to their compliance to an “ideal”. instead of pairwise comparisons, each alternative is associated with a performance category such as “excellent”, “average”, or “poor”. the importance of each alternative is obtained by multiplying the importance of the criterion and the compliance degree of the alternative. the ratings technique is recommended when there are a large number of alternatives (saaty, 2006). 4. a real-life case study 4.1 selected city and decision maker public access defibrillation has been a topic of interest in valencia, and valencia polytechnic university (upv) already has its own aed program. for years, the authorities responsible for the health policies of the city of valencia and upv have been sensitive about training non-medical staff in the use of aeds, a process which is organized and standardized through the valencian school of health studies. trainings are regulated by a decree published in the official journal of the generalitat valenciana (decree 220/2007 dated 02.11.2007). upv has trained 25% of its staff and 6 aeds have been installed throughout the campus according to population density and distance criteria. the plan in the future is to equip two mobile security guards with aeds based on a study that shows that their response ijahp article: ekmekci, aragonés-beltrán, rodríguez de sanabria gil, topcu /a decision model for determining public locations suitable for aed deployment: a case study in the city of valencia international journal of the analytic hierarchy process 155 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.492 time is less than three minutes. because of the success of the upv’s aed program the decision was made to extend the mcdm-based model to determine suitable aed locations in the city of valencia. the decision maker (dm) was the coordinator of the aed program in upv, a cardiologist and professor in healthcare technology and biomedical engineering programs in the school of business administration and management of the upv. a senior lecturer of project management in the department of engineering projects of the upv supervised and facilitated the decision making process. 4.2 constituents of the decision model a specific decision making process was designed for this case study (see figure 1), based on the decision making process suggested by topcu (2000). figure 1. the decision making process. ijahp article: ekmekci, aragonés-beltrán, rodríguez de sanabria gil, topcu /a decision model for determining public locations suitable for aed deployment: a case study in the city of valencia international journal of the analytic hierarchy process 156 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.492 4.2.1 evaluation criteria after reviewing previous studies (as explained in detail in section 2.2) four main criteria were agreed on: 1. coverage capacity: number of people and the size of the area covered by the aed. it has two sub-criteria: ’population coverage’ (i.e. the number of people the aed will serve) and ‘physical coverage’ (i.e. the physical area the aed will serve). 2. sca risk: likelihood of witnessing sca in the premises. it consists of two subcriteria: ‘risk level of people’ and ‘risk level of activities’. 3. response time: time needed to arrive at the victim’s location. it has two subcriteria: ‘time to place’ (i.e. the time it takes for an ambulance to arrive at the place from the moment it is called) and ‘time to furthest point’ (i.e. the time it takes for the medical professionals to reach from the parking spot of the premises to the furthest point in the building, which reflects the “vertical response time” mentioned in previous studies). 4. availability of an assigned user: likelihood of a trained person to be near and able to recognize the need for aed. literature review on criteria affecting selection of locations for aed deployment shows that previous research agrees on the effect of criteria on the goal:  an alternative with higher population coverage is a more suitable location for aed deployment (balady et al., becker, 1999; gilchrist et al., 20012; 2002, whitney-cashio et al., 2012). “population coverage” is a benefit attribute.  an alternative with higher physical coverage is a more suitable location for aed deployment (motyka et al., 2005; myers & mohite, 2008). “physical coverage” is a benefit attribute.  people with higher sca risk make an alternative more preferred for aed deployment (american heart association, 2001; balady et al., 2002; becker et al., 1999; brooks et al., 2013; folke et al., 2009; muraoka et al., 2006; nielsen et al., 2013; tsai et al., 2012). “risk level of people” is a benefit attribute.  locations where high-risk activities are performed are more preferred alternatives for aed deployment (american heart association, 2001; norton & norton, 2008; page et al., 2013). “risk level of activities” is a benefit attribute.  longer response time of emergency medical services is linked to higher preference as an alternative for aed deployment (american heart association, 2001; balady et al., 2002; gilchrist et al., 2012). “time to place” is a benefit attribute.  longer response time within the premises is linked to higher preference as an alternative for aed deployment (american heart association, 2001; whitneycashio et al., 2012). “time to furthest point” is a benefit attribute. 4.2.2 potential locations after reviewing previous studies and making meetings with the experts, 80 locations were selected as alternatives for the model. these locations were marked on a map using google maps and were sent to the decision maker to ensure that he had detailed knowledge on each. the map is accessible at https://www.google.com/maps/d/edit?mid=zcy0ey5rkume.kdqolnsmhk8m. https://www.google.com/maps/d/edit?mid=zcy0ey5rkume.kdqolnsmhk8m ijahp article: ekmekci, aragonés-beltrán, rodríguez de sanabria gil, topcu /a decision model for determining public locations suitable for aed deployment: a case study in the city of valencia international journal of the analytic hierarchy process 157 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.492 4.2.3 decision hierarchy a hierarchy of the decision model was constructed as seen in figure 2. figure 2. hierarchy of the decision model a 1-to-5 scale from very low to very high was used for rating the alternatives according to criteria population coverage, physical coverage, risk level of people, and risk level of activities; while a 1-to-5 scale from very short to very long was used for time to place and time to furthest point, and a 1-to-3 scale (not available, possible, available) was used for availability of assigned user. 4.2.4 pairwise comparisons a questionnaire was prepared and sent to the decision maker. the questionnaire consists of seven sections as indicated below:  introduction: brief explanation of the model and the survey.  explanation of criteria: the definitions of the criteria and sub-criteria.  explanation of the survey: detailed explanation of how to answer the questions.  explanation of the scale in spanish: the spanish translation of saaty’s fundamental scale to assist the decision maker.  part 1: questions for pairwise comparison of the criteria.  part 2: questions for determining rating intensities of the criteria.  part 3: questions for determining attributes of alternatives. in part 1 of the questionnaire, the decision maker is asked to make pairwise comparisons of criteria according to the goal and also those of the sub-criteria according to the criteria. in this part, saaty’s fundamental scale is used. in part 2, the decision maker is asked to assign points to each performance level (i.e. rating intensity) from 0 to 100, according to ijahp article: ekmekci, aragonés-beltrán, rodríguez de sanabria gil, topcu /a decision model for determining public locations suitable for aed deployment: a case study in the city of valencia international journal of the analytic hierarchy process 158 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.492 its compliance degree. for this study, using the rating intensities that were assigned by the decision maker is preferred rather than assigning equal intervals to the performance levels. in other words, instead of automatically assigning 0, 25, 50, 75, and 100 points for the five levels of the criterion population coverage (c1.1), the assigned points 10, 20, 50, 80, and 100 are used. in part 3, the decision maker is asked to select performance levels of the criteria for each of 80 alternatives. before responding to the questions, ahp and the nature of the pairwise comparisons were explained in detail to the dm by the ahp expert, and the dm answered the questions under his surveillance. 5. results and sensitivity analysis according to the results of the study, response time is by far the leading criterion with an importance of 65.07%, followed by sca risk with 23.13%, coverage capacity with 7.65%, and availability of assigned user with 4.14%. the sub-criteria of response time and sca risk have equal importance with respect to the main criterion. the importance of physical coverage with respect to coverage capacity is 83.33% while population coverage has an importance of 16.67%. figure 3. importance of criteria according to the goal researchers agreed that a simple ranking of the alternatives would not be sufficient for the purpose of this study because the aim is to offer a guide for selecting locations for deploying aeds rather than ranking the alternatives according to their suitability. therefore, we formed priority groups. the global scores of all alternatives were plotted on a graph in descending order to determine possible separation points (see figure 4). observing these separation points, we divided alternatives into four priority groups. ijahp article: ekmekci, aragonés-beltrán, rodríguez de sanabria gil, topcu /a decision model for determining public locations suitable for aed deployment: a case study in the city of valencia international journal of the analytic hierarchy process 159 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.492 figure 4. grouping alternatives according to their global scores the first-priority group consists of 20 alternatives with the highest global scores. this group represents the locations that should be considered as first priority or may be the subject of a pilot study. other groups of descending priorities have 15, 21, and 24 alternatives respectively. for a better and more sound understanding of the model, we performed a sensitivity analysis of the results to changes in the weights of the main criteria. eight scenarios were tested and compared with the original results, each scenario representing the situation where one of the main criteria has either 100% or 0% importance. for a better comparison in terms of priorities of the groups, the number of alternatives belonging to each priority group was not kept the same for all scenarios. instead, the global scores of the alternatives belonging to each scenario were analyzed, similar to what had been done for the original results, and the alternatives were grouped accordingly. this caused differences in the number of alternatives in each group for different scenarios. the number of alternatives in each priority group of all scenarios is summarized in table 1. ijahp article: ekmekci, aragonés-beltrán, rodríguez de sanabria gil, topcu /a decision model for determining public locations suitable for aed deployment: a case study in the city of valencia international journal of the analytic hierarchy process 160 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.492 table 1 number of alternatives in priority groups for each scenario orig. scn 1 (wc3 =1) scn 2 (wc3 =0) scn 3 (wc2 =1) scn 4 (wc2 =0) scn 5 (wc1 =1) scn 6 (wc1 =0) scn 7 (wc4 =1) scn 8 (wc4 =0) first-priority 20 67 20 16 22 21 19 58 23 second-priority 15 5 20 21 19 9 13 22 12 third-priority 21 6 19 26 18 31 29 18 fourth-priority 24 2 21 17 21 19 19 27 we believe that a closer look at the sensitivity of the first-priority group to the scenarios would be useful. the rankings and groups of the first 20 alternatives are shown in table 2 with colors showing their priority groups: red for first priority group, orange for second, and yellow for third. these alternatives are mostly in the first-priority group for all scenarios. the first 10 alternatives especially show strong insensitivity to changes, illustrating their robustness. ijahp article: ekmekci, aragonés-beltrán, rodríguez de sanabria gil, topcu /a decision model for determining public locations suitable for aed deployment: a case study in the city of valencia international journal of the analytic hierarchy process 161 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.492 table 2 ranking of first-priority group for each scenario alt. # name rank scn 1 scn 2 scn 3 scn 4 scn 5 scn 6 scn 7 scn 8 a27 nazaret sports center 1 1 14 4 1 31 1 1 1 a30 valencia city stadium 2 2 1 1 2 1 2 1 2 a01 central market 3 2 3 4 2 1 4 1 3 a07 valencia central bus station 3 2 3 4 2 1 4 1 3 a29 stadium on av. de les corts vlc. 5 2 7 1 5 8 2 1 5 a08 city hall (ayuntamiento) 6 2 8 4 7 12 4 1 6 a15 malvarrosa beach 6 2 8 4 7 12 4 1 6 a18 upv vera campus 6 2 8 4 7 12 4 1 6 a16 uv blasco ibáñez campus 9 2 12 4 19 12 10 59 6 a17 uv tarongers campus 9 2 12 4 19 12 10 59 6 a28 uv sports center av. pr. reig 11 2 14 4 23 31 4 1 11 a21 cabanyal sports center 12 2 16 17 7 12 13 1 13 a25 school of san josé 13 2 17 17 12 22 13 1 15 a20 sports center in garden turia 14 2 19 4 53 31 10 59 11 a52 cabanyal train station 15 2 20 28 7 12 20 1 16 a24 alboraya industrial zone 16 2 21 17 19 12 28 59 13 a26 nazaret football pitch 17 2 22 17 23 31 13 1 17 a69 patacona sports complex 17 2 22 17 23 31 13 1 17 a76 mercavalència wholesale center 17 2 22 17 23 31 13 1 17 a77 barri de la llum sports center 17 2 22 17 23 31 13 1 17 ijahp article: ekmekci, aragonés-beltrán, rodríguez de sanabria gil, topcu /a decision model for determining public locations suitable for aed deployment: a case study in the city of valencia international journal of the analytic hierarchy process 162 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.492 6. conclusions and further suggestions the aim of this study was to provide a novel approach to the problem of selecting suitable locations for aeds by using the analytical hierarchy process (ahp), and to utilize the decision model on a case study, aiming to serve as a guideline for health policy decision makers in future projects. after reviewing previous studies on this topic, the criteria and alternatives were determined. the decision problem consisted of four main criteria, six sub-criteria, and 80 alternatives. ratings were done by a decision maker who is an expert on aed programs under the supervision of a professor who is an expert on ahp. response time was the most important criterion with an importance of 65.07%, followed by sca risk (23.13%), coverage capacity (7.65%), and availability of assigned user (4.14%). however, it was interesting to observe that remote locations such as public gardens did not have high preferences. this shows that although the criterion response time dominates the selection, other criteria play an important role since most locations do not differ much in terms of response time. we believe that a simple ranking of the alternatives is not sufficient for the purpose of this study. we formed priority groups instead. global scores of all alternatives were analyzed and four groups with descending priorities were formed. sensitivity analysis showed that especially the alternatives in the first-priority group are not highly sensitive to changes, illustrating their robustness. this is particularly true for the first ten locations as shown in table 2. although providing a novel approach, this study may be further improved. the first step would be expanding the geographical area of the survey. using multiple decision makers might also provide valuable results. additionally, different decision making models might be used to compare results. finally, yet importantly, it is obvious that having one aed would not be enough for some locations, considering that the size and physical structure of locations such as a stadium or a shopping mall make it impossible for a responder to bring the aed to the patient in less than 10 minutes, which is the rule of thumb in the literature. a valuable contribution to this study would be determining the number of aeds needed in each location. ijahp article: ekmekci, aragonés-beltrán, rodríguez de sanabria gil, topcu /a decision model for determining public locations suitable for aed deployment: a case study in the city of valencia international journal of the analytic hierarchy process 163 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.492 references aragonés-beltrán p., chaparro-gonzález p., pastor-ferrando j.p., pla-rubio a. (2014). an ahp (analytic hierarchy process)/anp (analytic network process)-based multicriteria decision approach for the selection of solar-thermal power plant investment projects. energy, 66, 222-38. doi: https://doi.org/10.1016/j.energy.2013.12.016 balady g.j., chaitman b., foster c., froelicher e., gordon n., van camp s. (2002). automated external defibrillators in health/fitness facilities: supplement to the aha/acsm recommendations for cardiovascular screening, staffing, and emergency policies at health/fitness facilities. circulation, 105, 1147-50. doi: https://doi.org/10.1161/hc0902.105998 becker l., eisenberg m., fahrenbruch c., cobb l. (1999). public locations of cardiac arrest: implications for public access defibrillation. circulation, 97, 2106-9. doi: https://doi.org/10.1161/01.cir.97.21.2106 berger s. (2009). automated external defibrillators utility, rational, and controversies. pace, 32, 75-59. doi: 10.1111/j.1540-8159.2009.02390.x brooks s.c., hsu j.h., tang s.k., jeyakumar r., chan t.c.y. (2013). determining risk for out-of-hospital cardiac arrest by location type in a canadian urban setting to guide future public access defibrillator placement. annals of emergency medicinen 61, 530-8. doi: http://dx.doi.org/10.1016/j.annemergmed.2012.10.037 capucci a., aschieri d., guerra f., pelizzoni v., nani s., villani g.q., et al. (2016). community-based automated external defibrillator only resuscitation for out-of-hospital cardiac arrest patients. american heart journal, 172, 192-200. doi: http://dx.doi.org/10.1016/j.ahj.2015.10.018 chan t.c.y., li h., lebovic g., tang s.k., chan j.y.t., cheng h.c.k., et al. (2013). identifying locations for public access defibrillators using mathematical optimization. circulation, 127, 1801-9. doi: https://doi.org/10.1161/circulationaha.113.001953 cho y., je s., lim t., kang h. (2012). safe campus using wireless managed automated external defibrillator (aed). resuscitation, 83, e183-4. doi: http://dx.doi.org/10.1016/j.resuscitation.2012.03.038 colquhoun m.c., chamberlain d.a., newcombe r.g., harris r., harris s., peel k., et al. (2008). a national scheme for public access defibrillation in england and wales: early results. resuscitation, 78, 275-80. doi:http://dx.doi.org/10.1016/j.resuscitation.2008.03.226 cram p., vijan s., fendrick a.m. (2003). cost-effectiveness of automated external defibrillator deployment in selected public locations. journal of general internal medicine, 18, 745-54. doi: https://doi.org/10.1046/j.1525-1497.2003.21139.x https://doi.org/10.1016/j.energy.2013.12.016 https://doi.org/10.1161/01.cir.97.21.2106 http://dx.doi.org/10.1016/j.annemergmed.2012.10.037 http://dx.doi.org/10.1016/j.ahj.2015.10.018 http://dx.doi.org/10.1016/j.resuscitation.2012.03.038 http://dx.doi.org/10.1016/j.resuscitation.2008.03.226 ijahp article: ekmekci, aragonés-beltrán, rodríguez de sanabria gil, topcu /a decision model for determining public locations suitable for aed deployment: a case study in the city of valencia international journal of the analytic hierarchy process 164 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.492 eckstein m. (2012). the los angeles public access defibrillation (pad) program: ten years after. resuscitation, 83, 1411-2. doi: http://dx.doi.org/10.1016/j.resuscitation.2012.03.029 folke f., lippert f.k., nielsen s.l., gislason g.h., hansen m.l., schramm t.k., et al. (2009). location of cardiac arrest in a city center: strategic placement of automated external defibrillators in public locations. circulation, 120, 510-29. doi: https://doi.org/10.1161/circulationaha.108.843755 gianotto-oliveira r., gonzalez m.m., vianna c.b., alves m.m., timerman s., filho r.k., et al. (2015). survival after ventricular fibrillation cardiac arrest in the sao paulo metropolitan subway system: first successful targeted automated external defibrillator (aed) program in latin america. journal of the american heart association, 4, 185-93. doi: https://doi.org/10.1161/jaha.115.002185 gilchrist s., schieb l., mukhtar q., valderrama s., yoon p., sasson c., et al. (2010). a summary of public access defibrillation laws, united states, 2010. preventing chronic disease, 9, 110196. doi: http://dx.doi.org/10.5888/pcd9.110196 huig i.c., boonstra l., gerritsen p.c., hoeks s.e. (2014). the availability, condition and employability of automated external defibrillators in large city centers in the netherlands. resuscitation, 85, 1324-9. doi: http://dx.doi.org/10.1016/j.resuscitation.2014.05.024 imai k., fujii a., nabeshima k. (2008). aed location in public spaces: a case study in ueno park using voronoi diagrams with obstacles. journal of asian architecture and buidling engineering, 7, 271-8. doi: http://doi.org/10.3130/jaabe.7.271 lubin j., chung s.s., williams k. (2004). an assessment of public attitudes toward automated external defibrillators. resuscitation, 62, 43-7. doi: http://dx.doi.org/10.1016/j.resuscitation.2004.02.006 motyka t.m., winslow j.e., newton k., brice j.h. (2005). method for determining automatic external defibrillator need at mass gatherings. resuscitation, 65, 309-14. doi: http://dx.doi.org/10.1016/j.resuscitation.2004.09.016 muraoka h., ohishi y., hazui h., negoro n., murai m., kawakami m., et al. (2006). location of out-of-hospital cardiac arrests in takatsuki city – where should automated external defibrillator be placed? circulation, 70, 827-31. doi: http://doi.org/10.1253/circj.70.827 myerburg r.j., estes iii n.a.m., fontaine j.m., link m.s., zipes d.p. (2005) task force 10: automated external defibrillators. journal of american college of cardiology, 45, 1369-71. doi: https://doi.org/10.1016/j.jacc.2005.02.017 myers d.c., mohite m. (2009). locating automated external defibrillators in a university community. journal of the operational research society, 60, 869-72. doi: https://doi.org/10.1057/palgrave.jors.2602615 http://dx.doi.org/10.1016/j.resuscitation.2012.03.029 http://dx.doi.org/10.5888/pcd9.110196 http://dx.doi.org/10.1016/j.resuscitation.2014.05.024 http://dx.doi.org/10.1016/j.resuscitation.2004.02.006 http://dx.doi.org/10.1016/j.resuscitation.2004.09.016 https://doi.org/10.1016/j.jacc.2005.02.017 ijahp article: ekmekci, aragonés-beltrán, rodríguez de sanabria gil, topcu /a decision model for determining public locations suitable for aed deployment: a case study in the city of valencia international journal of the analytic hierarchy process 165 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.492 nichol g., huszti e., birnbaum a., mahoney b., weisfeldt m., travers a., et al. (2009). cost-effectiveness of lay responder defibrillation for out-of-hospital cardiac arrest. annals of emergency medicine, 54, 226-35. doi: http://dx.doi.org/10.1016/j.annemergmed.2009.01.021 nielsen a.m., folke f., lippert f.k., rasmussen l.s. (2013). use and benefits of public access defibrillation in a nation-wide network. resuscitation, 84, 430-4. doi: http://dx.doi.org/10.1016/j.resuscitation.2012.11.008 nolan j.p., soar j., zideman d.a., biarent d., bossaert l.l., deakin c., et al. (2010). european resuscitation council guidelines for resuscitation 2010 section 1: executive summary. resuscitation, 81, 1219-76. doi: http://dx.doi.org/10.1016/j.resuscitation.2010.08.021 norton k.i., norton l.h. (2008). automated external defibrillators in the australian fitness industry. journal of science and medicine in sport, 11, 86-9. doi: http://dx.doi.org/10.1016/j.jsams.2007.12.008 page r.l., husain s., white l.y., rea t.d., fahrenbruch c., yin l., et al. (2013). cardiac arrest at exercise facilities – implications for placement of automated external defibrillators. journal of american college of cardiology, 62, 2102-9. doi: https://doi.org/10.1016/j.jacc.2013.06.048 rauner m., bajmoczy n. (2003). how many aeds in which region? an economic decision model for the austrian red cross. european journal of operational research, 150, 3-18. doi: https://doi.org/10.1016/s0377-2217(02)00777-4 saaty t.l. (1977). a scaling method for priorities in hierarchical structures. journal of mathematical psychology, 15, 234-81. doi: https://doi.org/10.1016/00222496(77)90033-5 saaty t.l. (2006). rank from comparisons and from ratings in the analytic hierarchy/network processes. european journal of operational research, 168, 557-70. doi: https://doi.org/10.1016/j.ejor.2004.04.032 saaty t.l., vargas l.g. (2012). models, methods, concepts & applications of the analytic hierarchy process. new york: springer science+business media. sasaki m., iwami t., kitamura t., nomoto s., nishiyama c., sakai t., et al. (2011). incidence and outcome of out-of-hospital cardiac arrest with public access defibrillation, circulation, 75, 2821-26. doi: http://doi.org/10.1253/circj.cj-11-0316 schneider l., sterz f., haugk m., eisenburger p., scheinecker w., kliegel a., et al. (2004). cpr courses and semi-automatic defibrillators – life saving in cardiac arrest?. resuscitation, 63, 295-303. doi: http://dx.doi.org/10.1016/j.resuscitation.2004.06.005 http://dx.doi.org/10.1016/j.annemergmed.2009.01.021 http://dx.doi.org/10.1016/j.resuscitation.2012.11.008 http://dx.doi.org/10.1016/j.resuscitation.2010.08.021 http://dx.doi.org/10.1016/j.jsams.2007.12.008 https://doi.org/10.1016/j.jacc.2013.06.048 https://doi.org/10.1016/s0377-2217(02)00777-4 https://doi.org/10.1016/0022-2496(77)90033-5 https://doi.org/10.1016/0022-2496(77)90033-5 https://doi.org/10.1016/j.ejor.2004.04.032 http://dx.doi.org/10.1016/j.resuscitation.2004.06.005 ijahp article: ekmekci, aragonés-beltrán, rodríguez de sanabria gil, topcu /a decision model for determining public locations suitable for aed deployment: a case study in the city of valencia international journal of the analytic hierarchy process 166 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.492 siddiq a.a., brooks s.c., chan t.c.y. (2013). modeling the impact of public access defibrillator range on public location cardiac arrest coverage. resuscitation, 84, 904-9. doi: http://dx.doi.org/10.1016/j.resuscitation.2012.11.019 swor r., grace h., mcgovern h., weiner m., walton e. (2013). cardiac arrest in schools: assessing use of automated external defibrillators (aed) on school campuses. resuscitation, 84, 426-9. doi: http://dx.doi.org/10.1016/j.resuscitation.2012.09.014 timmons s., crosbie b. (2014). why do organisations implement automated external defibrillators?. health, risk & society, 16, 355-69. doi: http://dx.doi.org/10.1080/13698575.2014.926314 topcu, y.i. (2000). integrated decision aid model for multiattribute problem solving, unpublished ph.d. thesis, istanbul technical university, institute of science and technology, istanbul. tsai y., ko p.c., huang c., wen t. (2012). optimizing locations for the installation of automated external defibrillators (aeds) in urban public streets through the use of spatial and temporal weighting schemes. applied geography, 35, 394-404. doi: https://doi.org/10.1016/j.apgeog.2012.09.002 watson a.m., kannankeril p.j., meredith m. (2013). emergency response planning and sudden cardiac arrests in high schools after automated external defibrillator legislation. journal of pediatrics, 163, 1624-7. doi: http://dx.doi.org/10.1016/j.jpeds.2013.07.024 whitney-cashio p., sartin m., brady w.j., williamson k., alibertis k., somers g., et al. (2012). the introduction of public access defibrillation to a university community: the university of virginia public access defibrillation program. american journal of emergency medicine, 30, e1-8. doi: http://dx.doi.org/10.1016/j.ajem.2011.07.005 wu s., lee a., tah j.h.m., aouad g. (2007). the use of a multi-attribute tool for evaluating accessibility in buildings: the ahp approach. facilities, 25, 375-89. doi: https://doi.org/10.1108/02632770710772478 http://dx.doi.org/10.1016/j.resuscitation.2012.11.019 http://dx.doi.org/10.1016/j.resuscitation.2012.09.014 http://dx.doi.org/10.1080/13698575.2014.926314 https://doi.org/10.1016/j.apgeog.2012.09.002 http://dx.doi.org/10.1016/j.jpeds.2013.07.024 http://dx.doi.org/10.1016/j.ajem.2011.07.005 ijahp article: özdemir /seven international journal of the analytic hierarchy process 395 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.552 seven in memoriam: thomas l. saaty thomas l. saaty will be in our hearts, not only as a great scientist, but also a peacemaker who worked hard for world peace and conflict resolution. mujgan sagir özdemir 1 osmangazi university engineering and architecture faculty industrial engineering department eskişehir, turkey mujgan.sagir@gmail.com abstract the limitations on human performance, especially on processing information, have continuously received attention since miller’s (1956) seminal article. thomas l. saaty added to the body of knowledge with a paper titled, ‘why the magic number seven plus or minus two’, and this memoir essay summarizes that paper and its main contribution to the literature and science (saaty & özdemir, 2003). the paper concludes by supporting the idea that to serve both consistency and redundancy, it is best to keep the number of elements to seven or less, and therefore miller’s seven plus or minus two is indeed a limit on our ability to process information. thomas l. saaty was born on the 7th month of the year and passed away on august 7*2 = 14, 2017. keywords: analytic hierarchy process; mental limits; consistency 1. introduction when tom saaty and i decided to write this paper which was my first joint paper with him, i had already been expecting how serious, impatient, quick and productive tom would be (saaty & özdemir, 2003). he was a perfect organizer, a hard worker and completed tasks immediately. there was never any time, location, or business related excuses to generate an idea and not work on it. this work ethic and characteristics explain how this great man was so productive and able to write hundreds of papers, many books and have a great influence on thousands of people. 1 acknowledgment: i have carried everything i learned from tom with me to my country and to my students. hundreds of class projects have been completed in my creativity class, and dozens of papers using ahp and anp have been published by my graduate students. i am grateful to him for these and for giving me a lens through which to see the world clearly and truly. mailto:mujgan.sagir@gmail.co ijahp article: özdemir /seven international journal of the analytic hierarchy process 396 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.552 our motivation in the paper was to discuss validity and redundancy issues by considering miller’s (1956) famous article about the “magic number seven” where he mentioned variance and covariance or correlation to explain his findings. it was such an interesting topic, however; somehow, it was difficult to make the connections. the following section will briefly explain some of the main findings of the paper by making connections with miller’s channel capacity. in 1956, george miller conjectured that there is an upper limit on our capacity to process information on simultaneously interacting elements with reliable accuracy and with validity. he stated that the "amount of information" is exactly the same concept that we have talked about for years under the name of "variance." the equations are different, but if we hold tight to the idea that anything that increases the variance also increases the amount of information we cannot go far astray. the similarity of variance and amount of information might be explained this way. when we have a large variance, we are very ignorant about what is going to happen. if we are very ignorant, then when we make the observation it gives us a lot of information. on the other hand, if the variance is very small, we know in advance how our observation will come out, so we get little information from making the observation (miller, 1956). connecting and extending this idea to decision making was an interesting topic, especially when consistency is the key issue in ahp and anp. our motivation in the main paper was to show that in making preference judgments on pairs of elements in a group as we do in the analytic hierarchy process (ahp), the number of elements in the group should be no more than seven. this is because of the consistency of information derived from relations among the elements. the next section briefly explains the main paper (saaty & özdemir, 2003). 2. main findings the ahp relies on relative judgment about the degree or intensity of dominance of one stimulus of a pair over the other with respect to a given property present to the observer. such a comparison is made by first identifying the smaller or lesser stimulus as the unit and then estimating how many times the greater stimulus is a multiple of that unit. when all the comparisons are made, a scale of priorities is derived from them that represents the relative dominance of the stimuli. we learn from this approach that “not only must the sensations be homogeneous or close in order for the comparisons to be meaningful, but also there must be a limit to the number we can process at one time.” (saaty, 2003, p.235) on the other hand, maintaining consistency in our judgments is also required. we describe a method of deriving, from the observer’s quantified judgments, a set of weights, wi to be associated with individual stimuli. what this approach achieves is to put the information into usable form without deleting information residing in the qualitative judgments. as defined in detail in the main paper, let a1,…,an, be the set of stimuli. the quantified judgments on pairs of stimuli are represented by an n-by-n matrix a. the problem is to assign to these n stimuli a set of numerical weights, w1,…,wn , that would reflect the recorded judgments. in the simplest terms, a priority vector w can be used to weight the columns of its matrix and sum the elements in each row to obtain a new priority vector and repeat the process, thus obtaining an infinite set of priority vectors. the question is which is the real priority vector? such ambiguity is eliminated if we ijahp article: özdemir /seven international journal of the analytic hierarchy process 397 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.552 require that a priority vector satisfy the condition, , 0aw cw c  . in other words, ratios of priorities in the new vector coincide with the same ratios in the old vector. (saaty & özdemir, 2003). in light of this, for the validity of the vector of priorities to describe the response, greater redundancy is needed and therefore a large number of comparisons. on the other hand, for consistency a small number of comparisons is needed. to find the optimum number, the psychological idea of the consistency of judgments and its measurement is discussed in the paper together with the central concept in matrix theory, and the size of our channel capacity to process information (saaty & özdemir, 2003). it is the principal eigenvalue of a matrix of paired comparisons. it is shown that every 2-by-2 positive reciprocal matrix is consistent, though not every 3-by-3 positive reciprocal matrix is consistent, but in this case we are fortunate to have explicit formulas for the principal eigenvalue and eigenvector (saaty, 1996). as we know, the consistency ratio (c.r.) of a pairwise comparison matrix is the ratio of its consistency index to the corresponding random index value in table 1. figure 1 below is a plot of the first two rows of table 1. table 1 random index the notion of order of magnitude is essential in any mathematical consideration of changes in measurement. when one has a numerical value, say between 1 and 10 for some measurement, and wishes to determine whether change in this value is significant or not, one reasons as follows: a change of a whole integer value is critical because it changes the magnitude and identity of the original number significantly. if the change or perturbation in value is of the order of a percent or less, it would be so small (by two orders of magnitude) and would be considered negligible. however, if this perturbation is a decimal (one order of magnitude smaller) we are likely to pay attention to modify the original value by this decimal without losing the significance and identity of the original number as we first understood it to be. order 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 r.i. 0 0 0.52 0.89 1.11 1.25 1.35 1.40 1.45 1.49 1.52 1.54 1.56 1.58 1.59 first order differences 0 0.52 0.37 0.22 0.14 0.10 0.05 0.05 0.04 0.03 0.02 0.02 0.02 0.01 ijahp article: özdemir /seven international journal of the analytic hierarchy process 398 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.552 0 0,2 0,4 0,6 0,8 1 1,2 1,4 1,6 1,8 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 r a n d o m i n c o n si st e n c y number of elements compared figure 1. plot of random inconsistency thus, in synthesizing near consistent judgment values, changes that are too large can cause dramatic change in our understanding, and values that are too small cause no change in our understanding. we are left with only values of one order of magnitude smaller that we can deal with incrementally to change our understanding. it follows that our allowable consistency ratio should be not more than about .10. in the light of the above explanations, the quality of response to stimuli is determined by three factors. these factors are accuracy or validity, consistency, and efficiency or amount of information generated. our judgment is much more sensitive and responsive to large perturbations. when we speak of perturbation, we have in mind numerical change from consistent ratios obtained from priorities. the larger the inconsistency and hence the larger the perturbations in priorities, the greater our sensitivity to make changes in the numerical values assigned. conversely, the smaller the inconsistency, the more difficult it is for us to know where changes should be made to produce not only better consistency but also better validity of the outcome. similarly, the third row of table 1 gives the differences between successive numbers in the second row. figure 2 is a plot of these differences and shows the importance of the number seven as a cutoff point beyond which the differences are less than 0.10 where we are not sufficiently sensitive to make accurate changes in judgment on several elements simultaneously. stability of the principal eigenvector also imposes a limit on channel capacity and highlights the importance of homogeneity. to a first order approximation, perturbation w1 in the principal eigenvector w1 due to a perturbation a in the matrix a where a is consistent as given by wilkinson (1965): ijahp article: özdemir /seven international journal of the analytic hierarchy process 399 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.552 w)wv )-/(w a v( = w jj t jj11 t j n 2j= 1   here, t indicates transposition. the eigenvector w1 is insensitive to perturbation in a; if the number of terms n is small, if the principal eigenvalue 1 is separated from the other eigenvalues j, and, if none of the products vj t wj of left and right eigenvectors is small but if one of them is small, they are all small. however, v1 t w1, the product of the normalized left and right principal eigenvectors of a consistent matrix is equal to n that as an integer is never very small. if n is relatively small and the elements being compared are homogeneous, none of the components of w1 is arbitrarily small and correspondingly, none of the components of v1 t is arbitrarily small. their product cannot be arbitrarily small, and thus w is insensitive to small perturbations of the consistent matrix a. the conclusion is that n must be small, and one must compare homogeneous elements. 0.00 0.10 0.20 0.30 0.40 0.50 0.60 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 number of elements compared r a n d o m i n c o n si st e n c y d if fe re n c e s figure 2. plot of first differences in random inconsistency 3. conclusions saaty’s paper on “magic number seven” concludes that, “the consistency of judgments is necessary for us to cope effectively with experience but it is not sufficient. we need redundancy of informed judgments to improve validity. however, redundancy gives rise to inconsistency. therefore, we need to make a tradeoff between consistency and redundancy that implies validity.” (saaty, 2003, p.243) our measure of random inconsistency reveals that as the number of elements being compared is increased the measure of inconsistency decreases so slowly that there is insufficient room for improving the judgments and therefore also consistency. we conclude that to serve both consistency and redundancy, it is best to keep the number of elements seven or less. ijahp article: özdemir /seven international journal of the analytic hierarchy process 400 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.552 references miller, g.a. (1956). the magical number seven plus or minus two: some limits on our capacity for processing information, psychological review, 63(2), 81-97. doi: http://dx.doi.org/10.1037/h0043158 saaty, t.l. (1996). multicriteria decision making, the analytic hierarchy process. pittsburgh, pa: rws publications. saaty, t.l, & özdemir m.s. (2003). why the magic number seven plus or minus two, mathematical and computer modelling, 38, 233-244. doi: https://doi.org/10.1016/s0895-7177(03)90083-5 wilkinson, j.h. (1965). the algebraic eigenvalue problem. oxford:clarendon press. doi: https://doi.org/10.1017/s0013091500012104 http://psycnet.apa.org/doi/10.1037/h0043158 https://doi.org/10.1016/s0895-7177(03)90083-5 https://doi.org/10.1017/s0013091500012104 https://doi.org/10.1017/s0013091500012104 5_jayant_dec23_p_148-170 ijahp article: jayant, gupta, garg/ an application of the analytic network process to evaluate supply chain logistics strategies international journal of the vol. 3 issue 2 2011 analytic hierarchy process 149 issn 1936-6744 an application of the analytic network process to evaluate supply chain logistics strategies arvind jayant* department of mechanical engineering sant longowal institute of engineering & technology, longowal sangrur-148106 (india) arvindjayant@rediffmail.com p. gupta department of mechanical engineering sant longowal institute of engineering & technology, longowal sangrur-148106 (india) pardeepmech@yahoo.com s.k. garg department of mechanical engineering delhi technological university delhi-110042(india) skgarg63@yahoo.com1 abstract the purpose of this work is to describe an application of the analytic network process (anp) method to model the influence of various factors on supply chain logistics strategic decision making in competitive business environment. logistics plays an increasingly important strategic role for organizations that strive to keep pace with market changes and supply chain integration. logistics and supply chain management are currently evolving due to external factors such as strategic alliances, technological changes, cycle time compression and an increasingly competitive environment. the present model is flexible enough to structure this complexity by evaluating logistics strategic strategies by utilizing a systemic multi-attribute analytical technique. this paper explores and illustrates an analytical framework in a real life environment to assess an organization’s logistics strategy and challenges with varying levels of success. keywords: multiple criteria decision making, analytical network process, logistics strategy, supply chain management, e-procurement 1 the authors express their deep sense of gratitude to management and staff of xyz limited for providing an opportunity to carry out the project and extending their cooperation for data collection. rob typewritten text http://dx.doi.org/10.13033/ijahp.v3i2.76 rob typewritten text rob typewritten text rob typewritten text ijahp article: jayant, gupta, garg/ an application of the analytic network process to evaluate supply chain logistics strategies international journal of the vol. 3 issue 2 2011 analytic hierarchy process 150 issn 1936-6744 1. introduction scm involves coordinating and managing all the activities from raw materials procurement to the delivery of the final product to customers by the efficient use of it/is. the aim of scm is to globally optimize material and information flows in sc by horizontal integration between companies within sc and vertical integration of existing business processes in each company (nakagawa and sekitani, 2004). supply chain logistics plays an increasingly important strategic role for organizations that strive to keep pace with market changes and supply chain integration. traditionally, supply management and logistics have been delegated to operational level personnel in purchasing and distribution departments (mead and sarkis, 2001). logistics and supply management are currently evolving due to external factors such as strategic alliances, technological changes, cycle time compression and an increasingly competitive environment. a framework for logistics research includes strategy, structure, and performance. strategy can be defined as plans to meet relatively long-term organizational objectives that have broad corporate functional implications. (begicevic et al.,2006), these developments explain how a successful logistics strategy has moved from an internal focus emphasizing integration with other enterprise functions, such as production and marketing, with a linkage to the overall corporate strategy, to an external focus of integrating supply chains and cycle time compression (mead and sarkis, 2001). the complexity of logistics strategic decisions and choices has increased with the number of dimensions that need to be considered. in order to attain the aim of scm, managers within sc need to make strategic decisions for supplier selection, buying strategies, capital equipment purchasing, supplier performance evaluation, long-term partnerships between buyers and suppliers, effective purchasing and distribution etc. these are usually ambiguous and unstructured problems that include both tangible and intangible factors involving complicated criteria with interdependent relationships (korpelaa et al., 2001). a desirable methodology for such managerial issues is to allow for the synthesis of these factors and to help managers to structure the decision making problem. analytic hierarchy process (ahp) and its extension, analytic network process (anp) are systematic approaches that can deal with both quantitative and qualitative factors under multiple criteria (saaty, 1994). it is widely known that ahp and anp are practical tools of multiple criteria decision analysis. many applications and case-studies using ahp and anp are reported in various fields of business and industry (wind and saaty, 1980 and zahedi, 1986). ahp in scm has been a popular approach for supplier selection (barbarosoglu et al., 1997 and çebi et al., 2003), the design of supply chain networks (cakravastia et al., 2002 and min et al., 1999), and supplier performance evaluation (fung et al., 2001). anp is also applied to the same type of problems on scm as ahp (sarkis and sundar raj, 2002) because anp allows for the network structure modeling including all ahp models. the network modeling capability adds anp with new applications to scm such as the strategic decision analysis of a long-term partnership within sc (sarkis, 1997). ahp, the origin of anp, sometimes provides an irrational ranking of the details and this is called rank reversal phenomenon. by exploiting the network modeling effectively, anp may mitigate the possibility of rank reversal phenomenon (schenkerman, 1994). ijahp article: jayant, gupta, garg/ an application of the analytic network process to evaluate supply chain logistics strategies international journal of the vol. 3 issue 2 2011 analytic hierarchy process 151 issn 1936-6744 the anp model has been used in the exercise to structure this complexity by evaluating logistics strategic strategies by utilizing a systemic multi-attribute analytical technique. part of the difficulty in analytically modeling strategic decisions is their basis in quantitative and qualitative information with multiple dimensions. a quantitative model that can be used to integrate qualitative information and quantitative values and analysis is the analytic hierarchy process (ahp), but a primary limitation is that its basic relationships do not allow for an integrated dynamic modeling of the environment. ahp assumes the system elements are uncorrelated and are singly directionally influenced by a hierarchical relationship. a more general evaluation approach defined as the analytic network process (anp) or systemwithfeedback approach may be used to assess a dynamic multi-directional relationship among decision attributes. the anp approach is a non-linear, network relationship among various factors. (saaty, 2001). the present work is part of a consultancy project done by the authors for a telecom company of north india. in this project we provided support in the area of inbound logistics of the supply chain to the company. the company has a good number of overseas suppliers, and the lead time is very long. therefore, a huge inventory is necessary to maintain product flow in the supply chain. we recommended to the company to utilize full e-procurement for essential raw materials. for anp analysis a team was constituted including the following experts: vice president (marketing), vice president (ppc), vice president (scm), and the authors. all the pairwise comparisons were done by this team. for decision modeling anp is preferred due to interdependence of factors in the model. the chance for fraud is very high in-online procurement, so, the company wanted to make this strategic decision by considering all the aspects. now, the company has developed a group of online purchasing suppliers. they have also sorted out the problem of high lead time, and developed long term, trusting relationships with suppliers and are doing business successfully. the main domain covered by this paper is inbound logistics of supply chain. 2. a logistics framework for supply chain coordination and integration according to strategic alignment models, the success of organizational strategy is dependent upon the strategy and adopted technology, practices or systems. understanding and measuring the dynamic nature of logistics will make an organization more competitive. as products advance through the cycle of integration, rapid growth, maturity, and decline, a different logistic configuration may be more economical. the framework for improving inbound logistics and good manufacturer supplier relationship is shown in figure 1. the overall objective of the model is to determine the optimum strategic logistics system or alternative for xyz ltd. based on logistics strategy, and make the relationships flexible enough to meets the changing customer needs. in the end the various systems being considered will be weighted, and the one that ranks the highest will be suggested. the logistics system is evaluated on three different levels (or clusters); the supply chain/organizational relationship involved, the principles of logistics required, and the attributes of principles of logistics. ijahp article: jayant, gupta, garg/ an application of the analytic network process to evaluate supply chain logistics strategies international journal of the vol. 3 issue 2 2011 analytic hierarchy process 152 issn 1936-6744 figure 1 framework for logistics strategic analysis (jayant, 2002) figure 2 shows the framework in a representation conducive to anp. as can be seen the current procurement system (c.p.s) is compared to two other alternatives i.e., partial eprocurement (p.e.p) system and full e-procurement (f.e.p), for maintaining competitive logistics strategy and good relations. the end result of the model indicates the system which best meets the needs of the decision maker based on interaction between the three different levels. figure 3 represents a super matrix relationship for the anp network model. inbound logistic strategy supply chain relationship principal of logistics attributes logistics system ijahp article: jayant, gupta, garg/ an application of the analytic network process to evaluate supply chain logistics strategies international journal of the vol. 3 issue 2 2011 analytic hierarchy process 153 issn 1936-6744 figure 2.anp frame work for representations of relationships for the logistics strategic analysis selection of logistics strategy for improvement of supplier-manufacturer relationship commodity strategic alliance virtual relationship selective risk information substitution transaction simplification inventory velocity kaco imsf src aod lsi fc uif sai lsi jts fmo fdo uif sai lsi kaco imsf src aod lsi fc jts fd o fm o current procurement system partial e-procurement full e-procurement ijahp article: jayant, gupta, garg/ an application of the analytic network process to evaluate supply chain logistics strategies international journal of the vol. 3 issue 2 2011 analytic hierarchy process 154 issn 1936-6744 figure 3 network model for super matrix relationship 3. reputation systems (differential alternatives) 3.1 current procurement system in the current procurement system transactions are analyzed as repeated games. when business entities engage in repeated transactions in a market, they need to be concerned about their reputation. the reputation ideally should serve as an effective enforcement measure for honest behavior. the enforcement comes from the idea that dishonest behavior against one agent causes sanctions or retaliations by other agents in the same market. at present, the company is using a traditional method of purchasing the materials from the different suppliers. they keep a list of multiple vendors for the same items because they do not want to risk that materials will be unavailable. in the current system the vendors are not connected through the internet, rather information is shared on the phone or by courier services. the company simply places the order to the vendors by couriers and receives confirmation on the phone. limitations of the current system include lack of collaborative planning, lack of flexibility, and the chance of fraud and poor service. 3.2 partial e-procurement system partial e-procurement means the supplier and manufacturer are not connected through the internet directly, but they can mail each other and access limited information about each other through opening the company’s website. neither party is connected through the cyber-mediary. the manufacturer cannot check the status of the pending order with the supplier, and the supplier cannot access the information related to the company’s product demand, product type or the new product launch in advance so that he can plan production and schedule as needed. in this type of relationship the chance of fraud is greater because neither party is connected through any community or e-procurement portal. organizational relationship logistics principles ijahp article: jayant, gupta, garg/ an application of the analytic network process to evaluate supply chain logistics strategies international journal of the vol. 3 issue 2 2011 analytic hierarchy process 155 issn 1936-6744 3.3 full e-procurement system web-based procurement is already becoming a must-do for companies with progressive approaches to maintenance, repair and operations (mro) purchasing. the full eprocurement involves applying software technologies that have some of the highest return-on-investment. full e-procurement systems identify supplier rationalization opportunity by analyzing supplier spending for a commodity. to purchase a product or service, an end user browses a web-based catalog. the user selects items, and these items along with company specific pricing are passed back to the e-procurement system “shopping cart”. when the cart is checked out, any needed approvals are processed using web-based workflows, e-mails and extranet access. the approvals are based on user or cost-center limits, at the order level or in aggregate. upon approval, the shopping cart is turned into a purchase requisition or po in the back-office purchasing system. vendors that are paid via terms usually receive an electronic funds transfer at pre-specified times (gilbert, 2000).the most advanced mro procurement solutions available today are internet-based, and offer improved requisition-to-payment efficiencies, reduced maverick buying with electronic catalogs, sell-side system integration, internet-based procurement holds the potential to dramatically reengineer and improve purchase-to-order processes for indirect goods and services. 4. supply chain relationship/organizational relationships the organization needs to determine what type of supply chain relationship strategy will help it achieve its greatest competitive advantage. supply chain strategy level is a dynamic environment where various choices exist. supply chain strategy includes a continuum extending from commodity providers to virtual enterprise membership. commodity, strategic alliances and virtual enterprises from a spectrum of relationships that may exist among enterprises, with the relations of the inter-enterprise business processes become more unified and integrated along this spectrum. the strategic alignment of an organization’s logistic network needs to be synchronized with the demands of the competitive environment. an enterprise that fails to respond to the environmental demands is placed at a disadvantage relative to competitive firms. we have chosen three types of relationships to discuss. these relationships are commodity relationships, strategic alliances and virtual relationships (mead and sarkis, 2001). commodity relationships among enterprises focus on customers choosing suppliers based on price, quality, and reliability. the relevant business processes will be sparsely linked compared with the linkages to be found at the virtual relationship end of the spectrum, and financial/legal relationships will be less strongly coupled. the goal of a strategic alliance is to provide benefits to all sides of the relationship. the tangible benefits include cost and time reductions, whereas intangible benefits of partnering include flexibility and customer satisfaction. 5. the principles of logistics and principles of attributes the principles of logistics provide a foundation for consistent evaluation of logistics activities and strategies. logistics principles are defined with attributes for management of these principles. these attributes, which form the third level within the analysis ijahp article: jayant, gupta, garg/ an application of the analytic network process to evaluate supply chain logistics strategies international journal of the vol. 3 issue 2 2011 analytic hierarchy process 156 issn 1936-6744 framework, are based on expert opinion and literature. table 1 is a summary of the four principles of logistics along with the supporting attributes for their effective management. 5.1 principle of selective risk with the growth and implementation of information systems, information is abundant. however, the logistics manager needs to determine how the logistics systems are designed, implemented and managed in order to provide the appropriate information at the appropriate time. the principle of selective risk’s objective is to design logistics systems so that performance is directly related to the importance of the product or customer of the enterprise. logistics strategies are based on the level of service desired for a specific customer. detailed knowledge about the customer is necessary in order to implement this principle. attributes of processes and systems for aiding in the management of selective risk, may include knowledge about customers (kaco) (e.g. sales, length of time as customer), knowledge about competition (kaco), and range of service capabilities of logistics system (src). table1 principles of logistics and attributes of systems for management of logistics principles principles of logistics (mead and sarkis, 2001) attributes for management of logistics principles (mead and sarkis, 2001) selective risk (s.r) knowledge about competition (kaco) service range capabilities (src) inventory management system flexibility (imsf) information substitution (i.s.) accuracy of data (aod) level of system integration (lsi) forecasting capabilities (fc) transaction simplification (t.s.) user-interface friendliness (uif) level of system integration (lsi) supplier access to information (sai) inventory velocity (i.v.) just-in-time support (jts) flexible manufacturing operation (fmo) flexible distribution options (fdo) 5.2 principle of transactional simplification procurement logistics has been occupied with numerous operational transactions such as order entry, order fulfillment, and order delivery. the objective of the transaction simplification principle is to improve the efficiency and effectiveness of the transactional processes of the parties. this principle can be executed from both a technical and managerial perspective. three attributes for effective management of the principles of transactional simplification have been selected. the first attribute is user-interface friendliness (uif). this is created by a long term relationship, which helps to create mutual understanding hence simplification of the procedure. the second, attribute is the level of system integration (lsi), and the third is supplier access to operational information (sai). ijahp article: jayant, gupta, garg/ an application of the analytic network process to evaluate supply chain logistics strategies international journal of the vol. 3 issue 2 2011 analytic hierarchy process 157 issn 1936-6744 5.3 principle of information substitution the principle of information substitution is based on the fact that the cost of information is less than other resources. some attributes that will aid in effectively managing information substitution include the amount of coverage of information linkages, accuracy of data (aod), level of systems integration and forecasting capabilities (fc). 5.4 principle of inventory velocity the logistics role of facilitating the flow of inventory from raw materials to end-user serves as the basis of the principle of inventory velocity. inventory velocity is not new to logistics. quick response (qr), a business strategy developed in1989 to shorten the time for production and distribution, and reduce inventory throughout the supply chain directly impacts the inventory velocity principle. attributes of the processes and systems, which aid in managing the inventory velocity principal, include just-in –time (jts) support mechanisms, flexible manufacturing operation (fmo), and flexible distribution option (fdo) (cross-docking capabilities). 6. anp analysis and solution methodology the framework is presented through a network of decision model relationships. the levels of the network framework include the supply chain relationships, the principles of logistic level, the attributes level, and alternative selection level. these levels impact the overall goal of maintaining a competitive logistics strategy. the components of the organizational relationship are commodity, strategic alliances and virtual relationship. the principles level contains the four principles of logistics discussed earlier. the attributes level is composed of the components, which help to monitor, deploy and manage the principle (jayant, 2002). three logistics systems (alternatives) are considered. they are the current procurement system (c.p.s.) i.e. current system, partial e-procurement (p.e.p) and full e-procurement (f.e.p). the goal of this model is to select the most appropriate logistics system for a given enterprise operating to improve inbound logistics, good relationships, reduce fraud and enhance competitive logistics strategy. step 1: model construction and problem structuring: the first step is to construct a model to be evaluated. we will use the model that was summarized in figure 2. the relevant criteria and alternatives are structured in the form of a hierarchy where the higher the level, the more ‘strategic’ the decision. the top-most elements are decomposed into subcomponents and attributes. the model development requires the development of attributes at each level and a definition of their relationships. steps 2: pairwise comparisons matrices between component and levels: the pairwise comparisons of the elements in each level are conducted with respect to their relative importance towards their control criterion. a scale of 1 to 9 have been used when comparing two components, with a score of 1 representing indifference between the two components, and 9 being overwhelming dominance of the component under consideration (raw component) over the comparison component (column component). if a component has some level of weaker impact the range of scores will be from 1 to 1/9, where 1 represents indifference, and 1/9 being an overwhelming dominance by a column element over the row element. when scoring is conducted for a pair, a reciprocal value is automatically assigned to the reverse comparison within the matrix. that is, if aij is a ijahp article: jayant, gupta, garg/ an application of the analytic network process to evaluate supply chain logistics strategies international journal of the vol. 3 issue 2 2011 analytic hierarchy process 158 issn 1936-6744 matrix value assigned to the relationship of component i to components j then aij are equal to 1/aij. a pairwise comparison matrix is required for each of the organizational relationships for calculation of the impact of each of the logistics principles. after ircompleting the pairwise comparisons, the local priority vector w is calculated. here a two-stage algorithm that involves forming a new ncn matrix by dividing each element in a column by the sum of the column elements, and then summing the elements in each row of the resultant matrix and dividing by the n elements in the row. this is referred to as the process of averaging over normalized columns. this is represented as: (1) where: wi = the weighted priority for component i j= index number of columns (components) i = index number of rows (components) the first step includes the calculation of relative importance weights between principles of logistics for each relationship. we have to make one matrix each for the three relationships. the relative weights are expressed as pij (that is for i th relationship and jth principle of logistic). these have been calculated using equation 1. this has been shown in the table 2 where comparison has been done for commodity. the selective risk logistic gets 1 when compared with the same, and it has been allotted 3 when compared with transactional simplification because the selective risk knowledge information is more important i.e. if we go for a company with low transactional cost but a reputation that is not good in the market then the ultimate output will be that buyer will be cheated. so s.r. has been weighted higher than t.s. the comparisons have been done for all the three relationships in the same manner. (ii) for each proposed relationship, considering that relationship and one principle of logistics, the pairwise comparison has been done in all the attributes of that logistic. thus for each relationship we have done four such comparisons as we have chosen four logistics, and over all we have done twelve such comparisons. after comparison, weights are calculated using equation 1 for commodity, strategic alliance and virtual relationship. the relative importance weights are shown as qijk in desirability index matrix i.e. weight for i th relationship, jth logistics and kth attribute. steps 3: pairwise comparison matrices of interdependencies: here pairwise comparison for interdependencies between the attributes have been done considering the impact of one attribute over another for each particular relationship picking up one principle of logistics at a time. thus for each relationship, three comparison matrices have to be calculated for each logistics principle. therefore for each relationship we get twelve such matrices and over all thirty-six such matrices for the complete system. the weights are calculated using equation 1. a sample matrix is shown in the table number 2-6 for commodity; similar matrices are developed for strategic alliance and virtual relationship. step 4: super matrix formulation and analysis: table 19 shows a super matrix for commodity before convergence detailing the results of relative importance measures for each of the attribute enabler for commodity. since there are twelve pairwise comparisons matrices, there will be twelve non-zero sub columns in the super matrix before convergence. each of the non-zero values in the column in super matrix four is the wi = ∑ i i=1{aij / ∑ j j=1 aij} / j ijahp article: jayant, gupta, garg/ an application of the analytic network process to evaluate supply chain logistics strategies international journal of the vol. 3 issue 2 2011 analytic hierarchy process 159 issn 1936-6744 relative importance weight associated with interdependency pairwise comparison matrix. similar super matrices are developed for strategic alliance and virtual relationship. thus, we get three such matrices before convergence for each relationship. the elements of anp system may interact along many paths. for the measurement of priorities to be meaningful, uniformity is necessary when considering all paths of the network. the super matrix conversion helps to evaluate this framework and get stable values in the matrix to be used in calculation of the desirability index. the super matrix after convergence allows a resolution of the effects of interdependence that exists between the elements of the system. the super matrix before convergence is a portioned matrix, where each sub matrix is composed of a set of relationships between two levels. therefore, super matrices before convergence are converged for getting long-term stable term of weight. for this, the power of super matrix before convergence is raised to an arbitrary large number. here convergence has reached at 35th power. table 20 shows the values after convergence for commodity. similar analyses are done for strategic alliance and virtual relationship. these values are shown in the desirability index table as rijk i.e. value of relative importance for ith relationship jth logistics and kth attribute. for commodity table 2 pairwise comparison matrix between principles of logistics for commodity commodity s.r i.s t.s i.v weight s.r 1 5 3 4 0.45 i.s 0.2 1 2 4 0.20 t.s 0.33 0.5 1 2 0.13 i.v 0.25 0.25 0.5 1 0.22 table 3 pairwise comparison matrix between attributes for commodity and s.r commodity s.r kaco imsf src weight kaco 1 0.25 0.25 0.12 imsf 4 1 4 0.63 src 4 0.25 1 0.25 table 4 pairwise comparison matrix between attributes for commodity and i.s. commodity i.s aod lsi fc weight aod 1 4 2 0.54 lsi 0.25 1 0.25 0.12 fc 0.5 4 1 0.34 ijahp article: jayant, gupta, garg/ an application of the analytic network process to evaluate supply chain logistics strategies international journal of the vol. 3 issue 2 2011 analytic hierarchy process 160 issn 1936-6744 table 5 pairwise comparison matrix between attributes for commodity and t.s. commodity t.s uif sai lsi weight uif 1 2 4 0.55 sai 0.5 1 3 0.39 lsi 0.25 0.33 1 0.16 table 6 pairwise comparison matrix between attributes for commodity and i.v. commodity i.v jts fmo fdo weight jts 1 0.33 0.25 0.12 fmo 3 1 5 0.25 fdo 4 0.2 1 0.63 attributes pairwise comparison table 7 pairwise comparison matrix for interdependencies between the attributes for commodity and s.r. commodity and s.r kaco imsf src weight imsf 1 4 0.8 src 0.25 1 0.2 table.8 pairwise comparison matrix for interdependencies between the attributes for commodity and s.r. commodity and s.r imsf kaco src weight kaco 1 4 0.75 src 0.25 1 0.25 ijahp article: jayant, gupta, garg/ an application of the analytic network process to evaluate supply chain logistics strategies international journal of the vol. 3 issue 2 2011 analytic hierarchy process 161 issn 1936-6744 table 9 pairwise comparison matrix for interdependencies between the attributes for commodity and s.r. commodity and s.r src imsf kaco weight imsf 1 0.25 0.2 kaco 4 1 0.8 table 10 pairwise comparison matrix for interdependencies between the attributes for commodity and i.s. commodity and i.s aod lsi fc weight lsi 1 0.25 0.2 fc 4 1 0.8 table 11 pairwise comparison matrix for interdependencies between the attributes for commodity and i.s. commodity and i.s lsi aod fc weight aod 1 7 0.85 fc 0.14 1 0.15 table 12 pairwise comparison matrix for interdependencies between the attributes for commodity and i.s. commodity and i.s fc lsi aod weight lsi 1 0.5 0.67 aod 0.5 1 0.33 table 13 pairwise comparison matrix for interdependencies between the attributes for commodity and t.s. commodity and t.s uif sai lsi weight ijahp article: jayant, gupta, garg/ an application of the analytic network process to evaluate supply chain logistics strategies international journal of the vol. 3 issue 2 2011 analytic hierarchy process 162 issn 1936-6744 sai 1 0.25 0.2 lsi 4 1 0.8 table 14 pairwise comparison matrix for interdependencies between the attributes for commodity and t.s commodity and t.s sai lsi uif weight lsi 1 3 0.75 uif 0.33 1 0.25 table 15 pairwise comparison matrix for interdependencies between the attributes for commodity and t.s commodity and t.s lsi uif sai weight uif 1 5 0.84 sai 0.2 1 0.16 table 16 pairwise comparison matrix for interdependencies between the attributes for commodity and i.v commodity and i.v jts fmo fdo weight fmo 1 0.2 0.16 fdo 5 1 0.84 table 17 pairwise comparison matrix for interdependencies between the attributes for commodity and i.v commodity and i.v fmo jts fdo weight jts 1 4 0.8 fdo 0.25 1 0.2 table 18 pairwise comparison matrix for interdependencies between the attributes for commodity and i.v. commodity and i.v fdo fmo jts weight ijahp article: jayant, gupta, garg/ an application of the analytic network process to evaluate supply chain logistics strategies international journal of the vol. 3 issue 2 2011 analytic hierarchy process 163 issn 1936-6744 fmo 1 0.33 0.25 jts 3 1 0.75 table 19 super matrix for commodity before conversion commodity kaco imsf src aod lsi fc uif sai lsi jts fmo fdo kaco 0 0.75 0.8 0 0 0 0 0 0 0 0 0 imsf 0.8 0 0.2 0 0 0 0 0 0 0 0 0 src 0.2 0.25 0 0 0 0 0 0 0 0 0 0 aod 0 0 0 0 0.85 0.33 0 0 0 0 0 0 lsi 0 0 0 0.2 0 0.67 0 0 0 0 0 0 fc 0 0 0 0.8 0.15 0 0 0 0 0 0 0 uif 0 0 0 0 0 0 0 0.25 0.84 0 0 0 sai 0 0 0 0 0 0 0.2 0 0.16 0 0 0 lsi 0 0 0 0 0 0 0.8 0.75 0 0 0 0 jts 0 0 0 0 0 0 0 0 0 0 0.8 0.75 fmo 0 0 0 0 0 0 0 0 0 0.16 0 0.25 fdo 0 0 0 0 0 0 0 0 0 0.84 0.2 0 table 20 super matrix convergences to ‘long term’ weights at m 35 for commodity commodity kaco imsf src aod lsi fc uif sai lsi jts fmo fdo kaco 0.43 0.43 0.43 0 0 0 0 0 0 0 0 0 imsf 0.38 0.38 0.38 0 0 0 0 0 0 0 0 0 src 0.18 0.18 0.18 0 0 0 0 0 0 0 0 0 aod 0 0 0 0.36 0.36 0.36 0 0 0 0 0 0 lsi 0 0 0 0.29 0.29 0.29 0 0 0 0 0 0 fc 0 0 0 0.33 0.33 0.33 0 0 0 0 0 0 uif 0 0 0 0 0 0 0.4 0.4 0.4 0 0 0 sai 0 0 0 0 0 0 0.15 0.15 0.15 0 0 0 lsi 0 0 0 0 0 0 0.44 0.44 0.44 0 0 0 jts 0 0 0 0 0 0 0 0 0 0.43 0.43 0.43 fmo 0 0 0 0 0 0 0 0 0 0.16 0.16 0.16 fdo 0 0 0 0 0 0 0 0 0.39 0.39 0.39 step 5: pairwise comparison for different alternatives: in this step the pairwise comparison between different alternatives is done for each relationship considering each logistics principle at a time. in this we get four such matrices for each relationship and overall twelve matrices. weights calculated are shown in a sample matrix in table 21. the values being shown in the desirability index matrix as sij1 i.e. relative importance weight for ith relationship jth logistics and 1th alternative. the two-way arrow in the ijahp article: jayant, gupta, garg/ an application of the analytic network process to evaluate supply chain logistics strategies international journal of the vol. 3 issue 2 2011 analytic hierarchy process 164 issn 1936-6744 figure 3 shows an interdependence of organizational relationship and principles of logistics. table 21 the pairwise comparison between different alternatives for commodity and s.r commodity and s.r c.p.s p.e.p f.e.p weight c.p.s 1 0.33 0.166 0.09 p.e.p 3 1 0.5 0.25 f.e.p 6 2 1 0.66 cr=0.675 step 6: selection of best alternative: the selection of the best alternative depends on the calculation of the ‘desirability index’ for an alternative kth. the equation for this is defined by: (2) where, di1 = desirability index for i th relationship considering 1st alternative pij = relative importance weight for j th principle of logistic considering ith relationship qijk = relative importance weight between attributes for k th attribute considering ith. rijk = relative importance weight for k th attribute considering ith relationship jth logistics & kth attribute (for interdependencies) sij1 = relative importance weight for 1 st alternative considering ith relationship & jth logistics and 1th alternative di1 = ∑j=1 3 ∑k=1 3 pij qijk rijk sij1 ijahp article: jayant, gupta, garg/ an application of the analytic network process to evaluate supply chain logistics strategies international journal of the vol. 3 issue 2 2011 analytic hierarchy process 165 issn 1936-6744 table 22 desirability index for different alternatives for commodity step 7: relative importance weights for the relationship: the pairwise comparison matrix between relationships is calculated by the pairwise comparison between different relationships and the value calculated is shown in table 23. the results show that commodity is most important relationship for supplier-manufacturer relationship with a value of 0.54. table 23 the pairwise comparison matrix between different supply chain relationships commodity strategic alliance virtual relationship weight commodity 1 3 3 0.54 strategic alliance 0.33 1 6 0.35 virtual relationship 0.33 0.166 1 0.11 cr=0.0373 step 8: calculation of trust performance weighted index: table 27 shows the values of trust performance weighted index. the values of the desirability index com modity pij attri butes qijk rijk siji siji siji c.p.s p.e.p f.e.p 1 2 3 4= 1x2x3 5 6 7 8=4x5 9=4x6 10=4x7 s.r 0.5 kaco 0.12 0.43 0.026 0.09 0.25 0.66 0.002 0.00645 0.017 s.r 0.5 imsf 0.63 0.38 0.120 0.09 0.25 0.66 0.011 0.02993 0.079 s.r 0.5 src 0.25 0.18 0.023 0.09 0.25 0.66 0.002 0.00563 0.015 i.s 0.2 aod 0.54 0.36 0.039 0.11 0.31 0.56 0.004 0.01205 0.022 i.s 0.2 lsi 0.12 0.29 0.007 0.11 0.31 0.56 0.001 0.00216 0.004 i.s 0.2 fc 0.34 0.33 0.022 0.11 0.31 0.56 0.002 0.00696 0.013 t.s 0.14 uif 0.55 0.4 0.031 0.1 0.3 0.6 0.003 0.00924 0.018 t.s 0.14 sai 0.39 0.15 0.008 0.1 0.3 0.6 0.001 0.00246 0.005 t.s 0.14 lsi 0.16 0.44 0.010 0.1 0.3 0.6 0.001 0.00296 0.006 i.v 0.16 jts 0.12 0.43 0.008 0.16 0.24 0.6 0.001 0.00198 0.005 i.v 0.16 fmo 0.25 0.16 0.006 0.16 0.24 0.6 0.001 0.00154 0.004 i.v 0.16 fdo 0.63 0.39 0.039 0.16 0.24 0.6 0.006 0.00943 0.024 d11= d12= d13= sum 0.036 0.09077 0.211 ijahp article: jayant, gupta, garg/ an application of the analytic network process to evaluate supply chain logistics strategies international journal of the vol. 3 issue 2 2011 analytic hierarchy process 166 issn 1936-6744 calculated for different alternatives are given in the appendix for different relationships (tables 24, 25, 26). they are multiplied by respective weights for relationship, and total values are summed up for each alternative. the values are then normalized in table 27. the value calculated is a performance-weighted index. table 27 desirability index calculation for logistics systems alternatives 7. results the used analytic network process (anp) framework serves as a tool for making a strategic decision related to the selection of the best option out of finite set of alternatives with the feedback consideration. the analytic network process calculations for online procurement suggest shifting to full e-procurement from the current procurement system. the final values of trust performance weighted index for alternatives are 0.08 for cps, 0.18 for pep and 0.74 for fep. since the alternative with the maximum score is to be chosen the analysis suggests shifting to a full e-procurement system. the comparison scale may vary slightly from manager to manager and company to company, but the holistic view considered in the approach includes the feedback to minimize irregularities. if the analysis is applied with constant involvement of experienced managers then the present approach provides a sound strategy to choose the best alternative. 8. conclusions and discussions the availed anp model is capable of taking into consideration both qualitative, quantitative and multiple dimensions of information into the analysis, which is a powerful and necessary characteristic for any strategic evolution. the process becomes cumbersome if the number of logistic and system alternatives are increased to a large extent, but with the use of software and group decision support systems the barriers in calculation and implementation of this technique may be lessened. there is a strong move towards community as a social structure in the online world with applications such as ebay and i-village. the implication for transaction is to take advantage of buy side solutions, and use social structure solutions to support interpersonal exchange. thus the use of a full e-procurement system at full horizon will help to eliminate cheating, establish online trust, and provide flexibility to the company. there are many vendors who provide buy-side solutions like skyva, index system and others. they provide all types of support and infrastructure to the company for erelationship commodity strategic alliance virtual relationship performance normalized index index weight 0.54 0.35 0.11 c.p.s 0.036 0.018 0.095 0.036 0.08 p.e.p 0.091 0.052 0.083 0.076 0.18 f.e.p 0.211 0.102 1.54 0.319 0.74 ijahp article: jayant, gupta, garg/ an application of the analytic network process to evaluate supply chain logistics strategies international journal of the vol. 3 issue 2 2011 analytic hierarchy process 167 issn 1936-6744 procurement. this analysis framework can be used for selection or justification of various logistics strategies and systems for trust building e-markets. this approach is advantageous because it helps the management have more confidence in the buyer (manufacturer)-supplier relationship for online procurement. it also provides a structure for an organization to develop and enhance inbound logistics strategy. for future studies, we suggest developing a more detailed model for suppliermanufacturer relationships by considering different factors. a comparative study may be performed on the basis of results obtained by anp modeling in different business environments. further recommendations include the application of others concepts such as group decision making to expand the results to a sector or to different companies. appendix table 24 desirability index for different alternative for commodity commodity pij attrib utes qijk rijk siji siji siji c.p.s p.e.p f.e.p 1 2 3 4= 1x2x3 5 6 7 8= 4x5 9 =4x6 10= 4x7 s.r 0.5 kaco 0.12 0.43 0.026 0.09 0.25 0.66 0.002 0.00645 0.017 s.r 0.5 imsf 0.63 0.38 0.120 0.09 0.25 0.66 0.011 0.02993 0.079 s.r 0.5 src 0.25 0.18 0.023 0.09 0.25 0.66 0.002 0.00563 0.015 i.s 0.2 aod 0.54 0.36 0.039 0.11 0.31 0.56 0.004 0.01205 0.022 i.s 0.2 lsi 0.12 0.29 0.007 0.11 0.31 0.56 0.001 0.00216 0.004 i.s 0.2 fc 0.34 0.33 0.022 0.11 0.31 0.56 0.002 0.00696 0.013 t.s 0.14 uif 0.55 0.4 0.031 0.1 0.3 0.6 0.003 0.00924 0.018 t.s 0.14 sai 0.39 0.15 0.008 0.1 0.3 0.6 0.001 0.00246 0.005 t.s 0.14 lsi 0.16 0.44 0.010 0.1 0.3 0.6 0.001 0.00296 0.006 i.v 0.16 jts 0.12 0.43 0.008 0.16 0.24 0.6 0.001 0.00198 0.005 i.v 0.16 fmo 0.25 0.16 0.006 0.16 0.24 0.6 0.001 0.00154 0.004 i.v 0.16 fdo 0.63 0.39 0.039 0.16 0.24 0.6 0.006 0.00943 0.024 d11= d12= d13= sum 0.036 0.09077 0.211 ijahp article: jayant, gupta, garg/ an application of the analytic network process to evaluate supply chain logistics strategies international journal of the vol. 3 issue 2 2011 analytic hierarchy process 168 issn 1936-6744 table 25 desirability index for different alternatives for strategic alliance strat egic alliance pij attributes qijk rijk siji siji siji c.p.s p.e.p f.e.p 1 2 3 4= 1x2x3 5 6 7 8= 4x5 9= 4x6 10= 4x7 s.r 0.28 kaco 0.65 0.3 0.055 0.09 0.32 0.59 0.005 0.017 0.032 s.r 0.28 imsf 0.2 0.18 0.010 0.09 0.32 0.59 0.001 0.003 0.006 s.r 0.28 src 0.15 0.39 0.016 0.09 0.32 0.59 0.001 0.005 0.010 i.s 0.47 aod 0.56 0 0.000 0.13 0.27 0.6 0.000 0.000 0.000 i.s 0.47 lsi 0.24 0 0.000 0.13 0.27 0.6 0.000 0.000 0.000 i.s 0.47 fc 0.2 0 0.000 0.13 0.27 0.6 0.000 0.000 0.000 t.s 0.13 uif 0.21 0.17 0.005 0.1 0.31 0.59 0.000 0.001 0.003 t.s 0.13 sai 0.27 0.45 0.016 0.1 0.31 0.59 0.002 0.005 0.009 t.s 0.13 lsi 0.61 0.37 0.029 0.1 0.31 0.59 0.003 0.009 0.017 i.v 0.12 jts 0.37 0.43 0.019 0.13 0.27 0.6 0.002 0.005 0.011 i.v 0.12 fmo 0.32 0.41 0.016 0.13 0.27 0.6 0.002 0.004 0.009 i.v 0.12 fdo 0.31 0.16 0.006 0.13 0.27 0.6 0.001 0.002 0.004 d21= d22= d23= sum 0.018 0.052 0.102 ijahp article: jayant, gupta, garg/ an application of the analytic network process to evaluate supply chain logistics strategies international journal of the vol. 3 issue 2 2011 analytic hierarchy process 169 issn 1936-6744 table 26 desirability index for different alternatives for virtual relationship virtual relationship pij attributes qijk rijk siji siji siji c.p.s p.e.p f.e.p 1 2 3 4= 1x2x3 5 6 7 8= 4x5 9= 4x6 10= 4x7 s.r 0.46 kaco 0.21 0.16 0.015 0.13 0.28 0.59 0.002 0.004 0.077 s.r 0.46 imsf 0.69 0.43 0.136 0.13 0.28 0.59 0.018 0.038 0.077 s.r 0.46 src 0.12 0.39 0.022 0.13 0.28 0.59 0.003 0.006 0.077 i.s 0.13 aod 0.14 0.43 0.008 0.2 0.12 0.68 0.002 0.001 0.136 i.s 0.13 lsi 0.63 0.37 0.030 0.2 0.12 0.68 0.006 0.004 0.136 i.s 0.13 fc 0.23 0.18 0.005 0.2 0.12 0.68 0.001 0.001 0.136 t.s 0.11 uif 0.21 0.32 0.007 0.2 0.12 0.68 0.001 0.001 0.136 t.s 0.11 sai 0.69 0.19 0.014 0.2 0.12 0.68 0.003 0.002 0.136 t.s 0.11 lsi 0.1 0.21 0.002 0.2 0.12 0.68 0.000 0.000 0.136 i.v 0.3 jts 0.54 0.89 0.144 0.27 0.12 0.61 0.039 0.017 0.165 i.v 0.3 fmo 0.16 0.71 0.034 0.27 0.12 0.61 0.009 0.004 0.165 i.v 0.3 fdo 0.3 0.43 0.039 0.27 0.12 0.61 0.010 0.005 0.165 d31= d32= d33= sum 0.095 0.083 1.540 ijahp article: jayant, gupta, garg/ an application of the analytic network process to evaluate supply chain logistics strategies international journal of the vol. 3 issue 2 2011 analytic hierarchy process 170 issn 1936-6744 references begicevic, n., divjak, b., hunjak, t. (2006). development of ahp based model for decision making on e-learning implementation. 17th international conference on information and intelligent systems iis 2006, varaždin, faculty of organization and informatics, university of zagreb, 375-383. barbarosoglu, g., t., yazgaç (1997). an application of the analytic hierarchy process to the supplier selection problem. production and inventory management journal, 38, 1421. cakravastia, a., toha, i.s. nakamura, n. (2002). a two-stage model for the design of supply chain networks. international journal of production economics, 80, 231-248. çebi, f., bayraktar d. (2003). an integrated approach for supplier selection. logistics information management, 16, 395-400. liles, d.h., sarkis, j. (1997). justifying strategic alliances and partnering: a prerequisite for virtual enterprising. omega, 25, 29-42. fung, r.y.k., wong, c.f.y.(2001). “supplier environmental performance evaluation using ahp,” proceedings of the 6th international symposium on the analytic hierarchy process, isahp 2001, berne, switzerland, august 2-4, pp. 111-118, 2001. ghodsypour, s.h., brien, c.o. (1998). a decision support system for supplier selection using an integrated analytic hierarchy process and linear programming. international journal of production economics, 56-57, 199-212. hadfield, r., walton s.v., sroufec r., melnykd, a.s. (2002). applying environmental criteria to supplier assessment: a study in the application of the analytical hierarchy process. european journal of operational research, 141, 70-87. jayant, a. (2002) development of model for supplier-manufacturer relationship in supply chain management: a case study. master degree dissertation (industrial engineering), iit, delhi. korpelaa, j., lehmusvaara, a. tuominen, m. (2001). an analytic approach to supply chain development. international journal of production economics, 71, 145-155. meade, l., sarkis, j. (2001), strategic analysis of logistics and supply chain management systems using the anp. logistics and transportation research review, 34(3), 201215. min, h., e. melachrinoudis. (1999). the relocation of a hybrid manufacturing/ distribution facility from supply chain perspectives: a case study. omega, 27, 75-85. ijahp article: jayant, gupta, garg/ an application of the analytic network process to evaluate supply chain logistics strategies international journal of the vol. 3 issue 2 2011 analytic hierarchy process 171 issn 1936-6744 mohanty, r.p., deshmukh, s.g. (1993). use of analytic hierarchic process for evaluating sources of supply. international journal of physical distribution & logistics management, 23, 22-28. schenkerman, s. (1994). avoiding rank reversal in ahp decision-support models. european journal of operational research, 74, 407-419. saaty, t.l. (2001), the analytic network process: decision making with dependence and feedback, pittsburgh, pa: rws publications. nakagawa, t., sekitani, k. (2004). a use of analytic network process for supply chain management. asia pacific management review, 9(5), 783-800. wind, y., saaty, t.l. (1980). marketing applications of the analytic hierarchy process. management science, 26, 641-656. zahedi, f. (1986). the analytic hierarchy process: a survey of the method and its applications. interfaces, 16, 96-108. ijahp article: ali, arif, munir, azeem, khan/can india-pakistan sports promote peace? a mcdm approach international journal of the analytic hierarchy process 60 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.642 can india-pakistan sports promote peace? a mcdm approach yousaf ali ph.d assistant professor department of management science ghulam ishaq khan institute of engineering sciences & technology topi, swabi, kpk, pakistan; yousafkhan@giki.edu.pk shayan arif ghulam ishaq khan institute of engineering sciences and technology u2015419@giki.edu.pk fahad munir ghulam ishaq khan institute of engineering sciences and technology u2015104@giki.edu.pk muhammad bilal azeem ghulam ishaq khan institute of engineering sciences and technology u2015353@giki.edu.pk amin ullah khan graduate research assistant department of management science ghulam ishaq khan institute of engineering sciences & technology gem1847@giki.edu.pk abstract sports is an effective vehicle that is being adopted by almost every country to improve ties with other countries. india and pakistan share a long border, but do not share a very pleasant history. over the years, these two countries have shared a fierce rivalry and no measures that have been taken have had a long-term positive impact on their relationship. none of the attempts at reconciliation have proved to be completely successful, and the relationship remains unbalanced. the factors that can improve the ties between the two countries are numerous, and sports is one of them. the objective of this study is to determine how much of an influence cricket, being the most popular sport in both countries, has on the overall relationship between them. for this purpose, data was gathered from people from both countries through a questionnaire-based survey and then analyzed by mcdm approaches like ahp (analytic hierarchy process) and topsis (technique for order performance by similarity to ideal solution). after performing the quantitative analysis, the study hopes to come to a solid conclusion about what people from both countries think about which alternative has the greatest influence on the ties between the two mailto:yousafkhan@giki.edu.pk mailto:2015419@giki.edu.pk mailto:u2015104@giki.edu.pk mailto:u2015353@giki.edu.pk mailto:gem1847@giki.edu.pk ijahp article: ali, arif, munir, azeem, khan/can india-pakistan sports promote peace? a mcdm approach international journal of the analytic hierarchy process 61 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.642 countries, and how the selected factors are influenced by the type of cricket matches played between india and pakistan. the positive results obtained from this research could be of great importance for the governments as well as for the people of both countries. this can divert the minds of people and politicians from all the controversies and wars in the past, and help move towards more healthy competition on the sports field. keywords: sports; cricket; peace; rivalry; india; pakistan; mcdm 1. introduction the relationship between the two most important and powerful countries in asia (india and pakistan) has been questionable since their independence. not long after their independence, india and pakistan tried to develop positive political relations, but these efforts have been haunted by unpleasant events. since their independence, the two countries have suffered three critical wars, one undeclared war and multiple military standoffs. the battle over kashmir is the key central motivation behind these conflicts as well as the war between pakistan and india in 1971, and the war which led to the liberation of bangladesh, formerly known as east pakistan (rose & sisson, 1991). many different attempts have been made to improve relations such as the summits in shimla, lahore, and agra. ever since 1985, the relationship between the two countries has been strained mainly due to events like the conflict in siachen, the fortification of the revolt in 1989 at kashmir, the kargil war in 1999 and the nuclear tests performed by both countries. in an effort to calm things down, measures have been taken like the 2003 cease-fire and the lahore bus, which proved to be very successful for the intended purpose. these attempts at peace did not prove to be enough because of the continuous assaults at border crossings and genocide in kashmir. furthermore, the attack on the indian parliament in 2001 nearly pushed the two countries to the edge of a nuclear war. another critical point in the relationship of the two countries was the samjhauta express bombings in 2007, which led to the killing of 68 people, the majority of whom were muslims. another serious hit to the peace talks came in 2008 after the mumbai attack on the taj hotel (stelter & cohen, 2008). most recently, on february 26, 2019, indian jets bombed inside pakistani territory and claimed to have hit a camp belonging to an armed group. this act infuriated pakistan and their army warned that they would respond to india’s aerial bombing. in response, the pakistan air force (paf) shot down an indian fighter jet and captured its pilot who was released later as part of a "peace gesture" by pakistan (aljazeera, 2019). the list of disputes is very long, but many effective and non-effective measures have also been taken to tackle these disputes which cause problems for not only these two countries, but also for other countries. sports is one of the effective measures that has been implemented to ensure stability in the region and to improve peace between the two neighboring countries. many sports activities and tournaments have taken place between the two countries, but the most important and frequently played sport is cricket. both countries have one of the best cricket teams and players in the world, and they often play cricket together all around the globe. different types of cricket tournaments have been played between both countries over the years. there are friendly series, tournaments, and cup matches, but the games played most frequently are in leagues like the psl (pakistan super league) and the ipl (indian premier league). in these leagues, players from all around each of the ijahp article: ali, arif, munir, azeem, khan/can india-pakistan sports promote peace? a mcdm approach international journal of the analytic hierarchy process 62 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.642 countries come together and play on the same teams. however, in recent years, the indian players have not been allowed to participate in the pakistan super league, and the same is true for the pakistani players not being allowed to play in the indian premier league. this is a result of the unpleasant political situation between both countries and the egos and rivalry that has been passed down through many generations of political parties. the series matches between india and pakistan are always considered some of the most crucial matches around the globe, and people love to see these teams competing in the stadium. the same is true for tournament and world cup matches, and no matter which country or city the match is being held in, the stands are always packed and people travel around the globe to see india and pakistan play against each other. furthermore, people would also love to see the pakistani players playing in the indian premier league and indian players participating in the pakistan super league, despite the political differences. therefore, whether cricket has a positive or negative effect on the ties between pakistan and india and their people, and how strong this effect is, is a big question that needs to be answered. the purpose of this study is to find a logical answer to this important question. for this purpose, a questionnaire-based survey was drafted that was completed by the public in both pakistan and india. in this questionnaire the people shared their thoughts about the impact that different criteria like match frequency, the location of matches, the role of the media, political agendas, public relations and type of matches, have on the relationship between these two countries. after collecting data from these questionnaires, the necessary mathematical approaches were applied to determine the weights of these factors. finally, the impact of these factors on cricket and as a result the impact of cricket on the relationship between pakistan and india was calculated and analyzed. the first part of this study, the questionnaire, provided the qualitative data that was necessary for the second part of the research, which was the analysis to discover a quantitative result. much work has already been conducted on this very important issue of the relationship between india and pakistan, but very few have discussed it from the perspective of sports. most of the data that was used was qualitative and came from sources such as reports, newspaper articles, published journals, blogs, and research papers. many researchers have discussed the issue of india and pakistan, but there are very few who have researched the impact of cricket on the ties of these countries in a quantitative manner. the study aims to determine a productive outcome and communicate an essential message to individuals of both nations. this outcome will discuss how cricket matches between india and pakistan can influence the tenuous situation between the two nations. in addition, if cricket can fulfill this vital purpose, then expanding the number of games like, football, squash, badminton, kabaddi, and hockey could have a huge impact on the ties between these countries. cricket is considered an efficient method to find a proper solution for the peace of the region and especially to ease the conflict between the two countries. for this purpose, this study proposes five criteria on which the five alternatives will be analyzed. those alternatives include the different formats/types of tournaments played by various countries, and the aim is to determine an exact alternative that can prove to be the most effective in bringing india and pakistan together. for this purpose, the study will incorporate the analytic hierarchy process (ahp) and the technique for order of preference by similarity to ideal solution (topsis). the ahp was developed by thomas l. saaty in 1970 for complex decision-making ijahp article: ali, arif, munir, azeem, khan/can india-pakistan sports promote peace? a mcdm approach international journal of the analytic hierarchy process 63 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.642 situations that exist in any industry or field (golden, wasil, & harker, 1989). in this study, the ahp will be incorporated to assign weights to the factors that will be considered after a vigorous literature review. furthermore, to rank the alternatives based on the weighted criteria, topsis is used for the quantitative analysis. topsis was introduced by hwang & yoon (1981) to rank alternatives based on their distance from ideal solutions. the basics of this method are that an alternative is chosen as best if its distance from the ideal solution is the shortest and worst when the alternative's distance from the ideal solution is the longest. the research has been arranged in the following format. the next two sections contain a literature review and methodology, after which the results are calculated and discussed, and finally a conclusion is drawn. 2. literature review sporting events are considered very effective in bringing the people of different countries together. people travel to various places to cheer on and support their home team and to be a part of an outstanding crowd. such events are necessary and useful to alleviate unfriendly and awkward relationships between different countries and to break the ice by bringing people from different countries together on a single platform (murray, 2012). since 1947, pakistan and india have not been able to alleviate their unfriendly relationship and be united, and we believe that the long-term cure for this situation is through sports only. according to showkat (2018), the relationship between pakistan and india is a major issue and has been a hot topic among researchers in the past few years. after 1947, both countries created their cricket teams and there were some players who played for both sides like abdul hafeez kadar, amir elahi, and gul mohammad. the first test series between india and pakistan was played in october of 1952 in india, and india won 2-1. this was the first time that the pakistan cricket team played an international game and acquired test status. both countries have a very strategic geographical location in the region of south asia and have been hostile towards each other for a long time. the many issues between the two include the dispute of kashmir, the issues of a water treaty, and cross-border terrorism. as a consequence of these issues, both have faced four significant wars in 1947, 1965, 1971 and 1999. the indo-pak ties have resulted in much diplomacy, and cricket has had a neutralizing effect in every conflict. citizens of both countries have used cricket as a way to bond and have enjoyed their time watching the matches and forgetting all of the harsh times. cricket as a trust-building option was presented by pakistani president general zia in 1987 when he visited india out of the blue to watch a cricket match. rizvi (2015) also pointed out that even though the people were divided by their political views, their love of cricket gave these two nations a common ground, a place where they both learned to appreciate each other even when cold, hostile conditions existed between the two nations. it has always been the love of cricket that brought these two opposed states together like the kargil war in 1999 where the ties were completely dissolved and later resumed in 2003-2004. rizvi (2015) also pointed out that this process was rather slow and would not readily solve major issues like the never-ending fight for kashmir, but it would certainly give a chance for both countries to come and sit at a table. this willingness to talk will help develop proper communication of the issues and could lead to a sensible solution that would be advantageous to both parties. ijahp article: ali, arif, munir, azeem, khan/can india-pakistan sports promote peace? a mcdm approach international journal of the analytic hierarchy process 64 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.642 furthermore, another article by iftikhar (2014) conveyed that the proper time for a cricket match between the two nations is whenever there is any violence, tussle or challenge that they blame each other for. for example, similar to pakistan, the indian cricket team has also visited pakistan a few times, with the first visit being in 1954. this was done to mask the real problems at that time, such as the conflict for kashmir or the water distribution disagreements. according to the study by beutler (2008), the international community has realized the importance and strength of sports in peacekeeping. in 2001, the first advisor was appointed for sports development for peace; this is evidence that the un has utilized sports as a cost-efficient option to achieve development goals for the millennium and to spread peace. it has also been shown that sports can contribute to universal education, public health, gender equality, prevention of hiv and aids, poverty reduction and environmental stability which promotes peace and reduces conflict. the members of the un have recognized the role of sports in dealing with the numerous domestic and foreign policies ever since the 2005 international year of sports and physical education. to find a solution to resolve such issues requires proper decision-making to determine the necessary steps that need to be taken to make it a success. for this purpose, many studies have decided to use multi-criteria decision-making techniques, also known as mcdm. there are various techniques within mcdm such as dematel, vikor, analytic network process (anp), ahp and topsis (ali, shah & khan, 2018). some mcdm techniques are carried out within a fuzzy environment. a fuzzy environment is formed when there is vagueness and ambiguity in the data. fuzzy data is then integrated with mcdm techniques like vikor or topsis to convert the data into numerical standards (ali, awan, bilal, khan, petrillo, & khan, 2019). one such study that takes into account the situation of the fuzzy environment and combines the fuzzy data with the mcdm technique is awan & ali (2019). the study incorporated vikor and grey relational analysis in a fuzzy environment, depicting the role and reliability of mcdm techniques in this environment. this study incorporates ahp and topsis methods which are being used by various researchers and have given reliable results. one study by metin dag˘deviren (2008) observed the process of weapons selection as a strategic problem that had an effect on the defence system. selecting the correct weapon from among several options is a mcdm problem. the research focused on an assessment model based on ahp and topsis to help the required personnel select the best weapon in a fuzzy environment where a verbal variable was used to assign a set of triangular based numbers to ambiguity and biases that were present in the decision. in this problem, ahp was used to find a proper structure for the problem so that the definite values could be attached to the required standards, and fuzzy topsis was adopted to assign ranks to the proposed alternatives. furthermore, the study showed the effectiveness of hybrid techniques. the mcdm technique called analytic hierarchy process (ahp) was the brainchild of thomas l. saaty (saaty, 1980). this method has proven to be an effective tool for making the best decision in a situation that requires complex decision making or priority setting. it takes into account the subjective and objective parts of the decision and diminishes the biases in the process of decision making. the ahp presents a very strong and malleable technique that includes final results calculated by matrix comparison based on the criteria and options, both of which are selected by the user. ijahp article: ali, arif, munir, azeem, khan/can india-pakistan sports promote peace? a mcdm approach international journal of the analytic hierarchy process 65 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.642 the calculations performed are controlled by the knowledge of the decision-makers and are transformed to multi-criteria rankings. the research conducted by byun (2001) for the selection of an automobile model shows how the technique was successful in providing a proper path in a decision where a few select choices were available. in the case of shopping, it was necessary to take into account the components that delivered the features that made the decision for the purchaser much easier and therefore, lead to the acquisition of the correct item. it provided attributes that make consumer decision-making easy and comfortable and therefore, led to the purchase of the right car. dožić and kalić (2014) used the ahp to solve a type of aircraft selection problem for a known route network and forecasted air travel demand. ali, asghar, muhammad, and salman (2017) and ali, iftikhar and edwin (2017) also used the ahp for the selection of a fighter aircraft and the assessment of career selection problems, respectively. the technique for order performance by similarity to the ideal solution also known as topsis is used to solve mcdm problems. it helps organize the decision-making problems, and perform the analysis and comparisons needed to rank and select several externally determined alternatives based on distance measures from the ideal solution. deng, yeh and willis (2000) used topsis to solve the problem of multiple financial ratios that were required to adequately evaluate and rank the relative performance of competing companies. the topsis approach identified the relevance of financial ratios to the evaluated result and indicated the performance differences between the companies based on each financial ratio. another study by balcerzak and pietrzak (2016) examined the progress of european countries in implementing the concept of sustainable development. topsis was used because the sustainable development concept is considered a multidimensional phenomenon. topsis evaluated the objects in terms of multidimensional economic phenomena based on the set of detailed economic attributes (variables). the dynamic synthetic index describing the relative level of sustainable development of the countries was created, which enabled the researchers to propose a rating of the countries and group them into homogenous subsets. based on the literature review, the reliability of the techniques is definite and can fulfill the scope and objective of this study. the novelty of this study is that it quantitatively analyzes the role of cricket in the improvement of ties between pakistan and india. much research has been done on this issue of whether cricket can improve ties between pakistan and india, but none of the research has approached it in a quantitative manner as this study intends to do. the study also aims to recommend necessary steps to governments of both countries to ensure a healthy relationship for the sake of their people . ijahp article: ali, arif, munir, azeem, khan/can india-pakistan sports promote peace? a mcdm approach international journal of the analytic hierarchy process 66 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.642 3. research methodology the criteria and alternatives for this study were collected and combined after a rigorous literature review. the factors that had the potential to be most effective in the analysis and fulfillment of the purpose of this study were adopted. the research model was developed using the literature and knowledge of cricket, and the factors or criteria that can have a greater impact on friendly ties between the two countries based on different formats of cricket. the literature included studies that were being carried out for the revival of relations between countries based on sports. in these studies the criteria included the frequency of matches, media coverage, public relations, match locations and political agendas. the alternatives such as exhibition, tournament and league matches will be analyzed based on the above-mentioned distinct criteria. the research goal is to determine which alternative of sports format is the most vital and efficient in the recovery and development of friendly ties between india and pakistan. the best alternatives determined from the analysis can be recommended to both governments so that the relationship can be strengthened. similarly, in this research the analysis is being carried out using the ahp and topsis. first, pairwise comparisons of the factors were completed and then certain weights were assigned using the ahp method. next, by using the obtained weights the topsis method was applied to the ratings to generate rankings of suitable alternatives for the research objective. as mentioned above, the study has three alternatives that were analyzed based on five distinct criteria. the hierarchical structure for the study is depicted in figure 1. figure 1 hierarchical structure a1.exhibition matches c1. frequency of matches c2. media coverage c3. public reaction c4. match locations c5. political agendas a2.league matches c1. frequency of matches c2. media coverage c3. public reaction c4. match locations c5. political agendas a3.tournament matches c1. frequency of matches c2. media coverage c3. public reaction c4. match locations c5. political agendas ijahp article: ali, arif, munir, azeem, khan/can india-pakistan sports promote peace? a mcdm approach international journal of the analytic hierarchy process 67 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.642 the criteria and the alternatives are as follows; criteria: 1. the frequency of matches (c1) 2. media coverage (c2) 3. public reaction (c3) 4. match locations (c4) 5. political agendas (c5) alternatives: a. exhibition matches (a1) b. league matches (a2) c. tournament matches (a3) 3.1 data collection the selected topic was based on a social issue; therefore, the data was collected from laymen i.e. the public living on both sides of the border instead of using an expert opinion. therefore, the constructed questionnaire utilized terms that were relatively easier for the public to understand. the mode of data collection was a questionnairebased survey which was completed by the people according to their understanding. the distribution of the questionnaire took place in different pakistani and indian forums so that a reliable number of responses from both sides could be generated to improve the accuracy of the overall collected data. our target audience was not limited by age, race, religion, and nationality but by their compassion for and their love of sports. the designed questionnaire used a nine-value scale and a seven-value scale for ahp and topsis, respectively. the questionnaire is shown in appendix b. through the suggested data collection strategy, 65 people responded. their profession and it’s percentage of the whole sample are shown in table 1. table 1 breakdown of the participants based on their profession profession number of participants percentage % student 15 23 government employee 12 18.5 private sector employee 9 14 health and medical 12 18.5 education 17 26 table 1 shows that the majority of people who responded were from the education sector such as teacher/lecturer/professor, followed by students. this makes sense since the younger generation is more inclined towards sports. the analytic hierarchy process (ahp) is one multi-criteria decision-making strategy that was initially created by thomas l. saaty (saaty, 1980). it is an mcdm tool that can be used to consider several criteria and sub-criteria on various levels to rate the alternatives. it has numerous applications in the modern world including conflict resolution, budget allocation, project selection, health care, manufacturing, transportation and many more. ijahp article: ali, arif, munir, azeem, khan/can india-pakistan sports promote peace? a mcdm approach international journal of the analytic hierarchy process 68 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.642 the influence of these five factors on each other will be determined through the questionnaire that was filled out by the public of both india and pakistan. the influence of these factors on each other is divided into nine different choices. the ratings are assigned to these choices and shown in table 2 (saaty, 1980). table 2 ahp linguistic scale one of the factors mentioned above will be compared to every other factor, and the influence of the first factor on the second will be any one of the ratings in the table. the influence of the second factor on the first will be its reciprocal. using these rating values, a pairwise comparison matrix for each criterion will be formed. the ahp pairwise comparison values are depicted in table 3. that pairwise matrix will be normalized and used to determine the priorities and decision matrix. the priorities will reveal the weight of every factor in our research. the decision matrix will let us know the comparative weights of every factor based on the eigenvector of the decision matrix. in the end, a consistency ratio of our results will be calculated. steps involved in ahp: 1. compare the criteria matrix in pairs using values from 1 to 9 as stated in table 2. when both criteria have the same importance ‘1’ will be used. a square matrix named a will be formed when criteria 𝑖 and criteria 𝑗 will be compared in pairs, 𝑎𝑖𝑗 will represent how criteria 𝑖 is more important than criteria 𝑗. when 𝑖 = 𝑗, 𝑎𝑖𝑗 = 1 and so 𝑎𝑗𝑖 = 1 𝑎𝑖𝑗 choice rating extremely unimportant -9 very strongly unimportant -7 strongly unimportant -5 moderately unimportant -3 equally important 1 moderately important 3 strongly important 5 very strongly important 7 extremely important 9 ijahp article: ali, arif, munir, azeem, khan/can india-pakistan sports promote peace? a mcdm approach international journal of the analytic hierarchy process 69 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.642 table 3 ahp pairwise comparison criteria 1 criteria 2 criteria 3 criteria 4 criteria 1 1 3 4 7 criteria 2 1/3 1 6 8 criteria 3 ¼ 1/6 1 2 criteria 4 1/7 1/8 1/2 1 11 12 13 14 21 22 23 24 31 32 33 34 41 42 43 44 a a a a a a a a a a a a a a a a a             (1) 2. normalize the geometric mean of rows in the matrix of comparisons and construct the relative normalized weight (𝑊𝑗). 3. matrix a3 will be constructed by making a3 = a1* a2 4. matrix a4 will be constructed by making a4 = a3 /a2 5. the average of matrix a4 will give the maximum eigenvalue. 6. find the consistency index (c.i.) using the formula below: 𝐶𝐼 = 𝐿𝑚𝑎𝑥−𝑛 𝑛−1 (2) consistency ratio (cr) = 𝐶𝐼 𝑅𝐼 (2) cr = ci/ri, where, ri is random index if the consistency ratio is below 10%, the results are consistent and can be considered acceptable, but if the consistency ratio is above 10% judgments must be reexamined. the topsis method is used to solve ranking problems in the real world. although it is a very popular and a simple method, topsis has some limitations. these limitations include its inability to adequately handle the inherent uncertainty related to the mapping of the decision maker's perception of crisp values (ceballos, lamata, & pelta, 2016). in the traditional formulation of topsis, personal judgments are signified with crisp values. however, in several sensible cases, the human preference model is unsure, and decision-makers may be reluctant or unable to assign crisp values to the comparison judgments (chan & kumar, 2007). the need to use crisp values is one of the problematic points within the crisp analysis method. this is ijahp article: ali, arif, munir, azeem, khan/can india-pakistan sports promote peace? a mcdm approach international journal of the analytic hierarchy process 70 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.642 because decision-makers typically feel more assured when they use interval judgments instead of using single numeric values to express their judgements. some criteria are difficult to measure using crisp values, and they are typically neglected throughout the analysis. the rationale for employing a triangular fuzzy range is that it is intuitively simple for the decision-makers to use and calculate. additionally, modeling exploitation of triangular fuzzy numbers has been verified to be a good approach for formulating all issues wherever the data is subjective and inexact. triangular applications are employed most frequently for representing fuzzy numbers. this technique gives preference to order by comparison with the ideal solution also known as the positive ideal solution. in this, the cost characteristic will be minimized and benefit characteristics will be maximized. the negative ideal solution is the opposite of the positive ideal solution (also known as an ideal solution) in the fuzzy topsis technique. in this technique, the cost characteristic will be maximized, and benefit characteristics will be minimized (gupta & kumar, 2018). let us say we have the score of every option (m) against every criterion (n), so the score xij will be the option i with respect to criterion j. we have a matrix x= (xij) m x n matrix. now j is the set of benefit attributes, the higher the number of attributes the better it is. now j’ is the set of negative attributes, and lesser the number of attributes the better it is. table 4 topsis linguistic variable very poor (vp) (0; 0; 1) poor (p) (0; 1; 3) medium poor (mp) (1; 3; 5) fair (f) (3; 5; 7) medium good (mg) (5; 7; 9) good (g) (7; 9; 10) very good (vg) (9; 10; 10) steps: 1. to find the rankings construct of a decision matrix, convert the obtained data using the linguistic variable from table 4. the matrix structure will be as follows while every ‘f’ will have 3 components f= (a, b, c). the alternates for j will be denoted by aj, j = 1, 2, 3, 4 ..., j; the i th attribute will be represented by fi, i = 1, 2, 3, 4 ..., n, related to ith alternative; the crisp values will be fij and will indicate the ratings of performance of each alternate ai with respect fj criteria. 2. construct a normalized decision matrix, after gaining the initial decision matrix. to transform the several criteria scales to a comparable scale in a fuzzy environment, the method of linear scale transformation is used. * * * , , ij ij ij ij j j j a b c r c c c          , ij i * j cmaxc  (3) ijahp article: ali, arif, munir, azeem, khan/can india-pakistan sports promote peace? a mcdm approach international journal of the analytic hierarchy process 71 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.642 where  , ,ij ij ij ijx a b c are the elements of the initial decision matrix. 3. the normalized weighted decision matrix will be constructed, where ij ν j ij w r  , j = 1,2,3,4…m , i = 1,2,3,4…n (4) 4. determine the fuzzy ideal and fuzzy negative-ideal solutions. 1 2 m a {ν , ν ,...ν }      (5) 1 2 m a {ν , ν ,...ν }      (6) where jν  = (1,1,1) and jν  = (0,0,0), j=1, 2, 3, 4 ..., m. 5. calculate the separation measure: – ideal separation m i ij j j 1 s s(ν , ν )      i=1, 2, 3, 4 ..., n (7) – negative-ideal separation m i ij j j 1 s s(v , v )      i = 1, 2, 3, 4 ..., n (9) where ij js(ν , ν )  and ij js(ν , ν )  are distance measurements. 6. calculate the relative closeness to the ideal solution. )s(s s c ii i* i     , 0< * i c <1, i = 1, 2, 3, 4 ..., n (8) * i c =1 if ai=a + * i c =0 if ai=a 7. rank the preference order – a set of alternatives can now be preference ranked according to the descending order of * i c . 4. results after the collection of responses from the appropriate resources, the collected data was transferred to a spreadsheet for the application of the proper calculations to generate results. this study also incorporated a combined mcdm where the analytic hierarchy process was utilized to generate weights for the selected criteria, and then topsis was applied to generate the ranking of the alternatives. this was used to determine which alternatives were the best for the purpose of this study. using the linguistic variable provided in table 2, we converted the data for the ahp into a numerical format to obtain a qualitative response after the conversion arithmetic mean was calculated to obtain a pairwise comparison as shown in table 5. ijahp article: ali, arif, munir, azeem, khan/can india-pakistan sports promote peace? a mcdm approach international journal of the analytic hierarchy process 72 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.642 table 5 pairwise comparison matrix c1 c2 c3 c4 c5 c1 1 2 2 1 1 c2 0.5 1 1 1 1 c3 0.5 1 1 1 1 c4 1 1 1 1 1 c5 1 1 1 1 1 pair wise comparasion matrix by following the steps for ahp discussed in the methodology section, the values obtained in table 6 show that the factors that have a major effect on the study are c1 (having the highest weight), with c4 and c5 coming in second, and c2 and c3 coming in last. the consistency ratio obtained from the pairwise comparison matrix is 0.017<0.1. since the consistency ratio was well within the required range, the calculated weights are reliable and can be used in further calculations. table 6 priority vector, weights calculation after the implementation of the ahp, topsis’s triangular fuzzy numbers were used to assign numerical values to linguistic variables from the data that was obtained from the questionnaire. appendix a shows different stages of the results calculated from the data obtained. table 7 computation of close coefficient along with the rankings cc ranking a1 0.966666667 1st a2 0.138080495 3rd a3 0.324364281 2nd computation of close coeffecient after performing the final calculation, the obtained ranks for the alternatives were as follows: a1 ranked in first place, a3 in second place and a2 came in last (table 7.). in table 7, the a1 alternative represents ‘exhibition matches’, a2 represents ‘league matches’ and lastly, a3 represents ‘tournament matches’. priority vector 2 0 c1 0.26333 λmax 5.077452 c2 0.171667 ci 0.019631 c3 0.171667 ri 1.12 c4 0.19667 cr 1.70% c5 0.19667 ijahp article: ali, arif, munir, azeem, khan/can india-pakistan sports promote peace? a mcdm approach international journal of the analytic hierarchy process 73 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.642 5. discussion based on the results, exhibition matches ranked first among the alternatives or in other words, was given the highest priority by the respondents. exhibition matches are single event matches that are not a part of any competition, but rather are tested series, one day series and t20s matches or a combination of all three (press, 2000). there are three types of international cricket matches (howtheyplay.com, 2018) 1. tests this is the highest standard of cricket, and the players who set records in these are the greatest players (allsopp & clarke, 2004). currently, a standard test match lasts 5 days, with approximately 90 overs bowled per day. there are two innings per team without any restriction on the number of overs faced by the batting team. there are usually two reasons for the end of the inning, when the batting team gets out or when the captain declares the innings (happens when the captain is satisfied with runs scored or if a team is in a relatively safe position, giving the opposing team the chance to bat). 2. one-day internationals odi's are limited-overs matches in which 50 overs are bowled per inning. they only have 2 innings, 1 inning per team. both teams have colored clothing; white balls are used and there are some fielding and bowling restrictions. 3. twenty 20 internationals twenty 20 matches, also knowns as t20s, are the newest form of international cricket. there are only 20 overs per innings. this is now the most athletic and passionate format of international cricket. fans enjoy it more than other formats because it is a short game and mostly revolves around hard-hitting the ball. it is suggested that these formats are followed in exhibition matches between pakistan and india in order to gain better results and involvement from every type of supporter for these games. there are many advantages of exhibition cricket. in countries like india and pakistan, it is a tool to unite the divided nations because regardless of religion or caste a cricket match between these two countries can help put all differences aside and unite the people under the flag of their respective country. for the pakistanis and indians living in foreign countries, people from both nations interact with each other either by discussing the scores or commenting on the match. international matches also provide vital marketing opportunities to companies located in pakistan and india. a series of matches provides a base for the future development of sports in these nations, and provides a chance to meet and understand the people living on the other side. the demand for sporting equipment like replica shirts, sneakers, and other cricket items will increase. media coverage of the matches will not only cover the cricket match, but also provide a glimpse into the culture of the country and places which will attract tourists from all over the world. the least ranked alternative was league matches. these results also conform to the weights that each criterion carried as calculated by the ahp decision-making model because currently pakistan and india have two separate leagues. not since the first season of the indian premier league have the pakistani players been allowed to take part in the league, and the same is true for indians being allowed to participate in the pakistan super league. in addition to this, one might argue from the economic ijahp article: ali, arif, munir, azeem, khan/can india-pakistan sports promote peace? a mcdm approach international journal of the analytic hierarchy process 74 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.642 increase that each leagues season could contribute to each country. most of the league matches for pakistan are held in dubai due to unstable conditions, and therefore have not been of much value to pakistan yet. similarly, the leagues include players from different countries so the particular unity is not as easy to observe during those matches when compared to the games held on a national level. the league matches, being the worst alternative, can be justified by the fact that many people do not watch league matches or just watch a game to see their favorite players like shahid khan afridi, chris gayle and ab de villiers. therefore, these games cannot contribute much towards the betterment of ties between the two countries. it is recommended that players from both countries be allowed to take part in each country’s leagues because this might help melt the ice between these two countries. if there is a match between pakistan and india, people from both sides of the border will find time in their very busy lives to watch the match because it is tradition. these games possess an intense rivalry and competitiveness, and this competitiveness is felt by everyone on both sides of the border. this competitiveness would not be seen in league matches because both countries would not be alone in the league but would be a small part of something bigger than themselves. this notion is further supported by the number of viewers for a match between pakistan and india, which has been 47.45 million (laghate, 2017). this is in contrast to the viewership for the other 60 matches in ipl 2017 which was 411 million. this makes 6.85 million views per match, and the total viewership for the psl 2018 was only 8.2 million for a total of 15 matches (akbar, 2018). these figures support the idea that exhibition matches should be more supported and relied upon than league matches. 5.1 limitations the study tried to cover all of the relevant aspects of the required objective, but there are a few limitations. one of the limitations is that the study was conducted only in pakistan and india. the issues between these countries were the main focus, and this was the reason that the survey was only conducted in these two countries. another limitation in this study is that the data collection was more focused on the general population as there were no other options available that could prove to be more feasible. 5.2 future improvements future studies can highlight conflicts between other countries and also take into account relevant data collection from politicians, government officials, sports players, commentators, and cricket experts. similarly, the study can also be extended to other sports to resolve disputes between different countries. 6. conclusion the results extracted from the research and analysis suggest that exhibition matches, and bilateral series conducted between pakistan and india are the most followed cricketing events as compared to the league matches that each country holds every year. the results allow us to confidently conclude that governments of both sides should focus on increasing the frequency of bilateral series. this will help people compete on the sports grounds rather than arguing on social media and talk shows. some of the drawbacks this study faced are the number of criteria selected, and the fact that an expert opinion was not easily available. furthermore, the data gathered focused on pakistan and india which is why this model cannot be easily implemented elsewhere. more accurate results can be obtained using the developed model if there ijahp article: ali, arif, munir, azeem, khan/can india-pakistan sports promote peace? a mcdm approach international journal of the analytic hierarchy process 75 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.642 is access to expert opinions and a greater number of criteria are involved. involving experts would help generate more accurate weights for the ahp decision model, which in turn will give a more accurate alternative ranking. ijahp article: ali, arif, munir, azeem, khan/can india-pakistan sports promote peace? a mcdm approach international journal of the analytic hierarchy process 76 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.642 references akbar, f. (2018). cricingif. retrieved nov 6, 2018, from https://www.cricingif.com/stories/11434/psl-viewership-figures-are-something-wecan-cherish ali, y., asghar, a., muhammad, n., & salman, a. (2017). selection of a fighter aircraft to improve the effectiveness of air combat in the war on terror: pakistan air forcea case in point. international journal of the analytic hierarchy process,, 9(2), 244-273 doi: https://doi.org/10.13033/ijahp.v9i2.489 ali, y., awan, m. a., bilal, m., khan, j., petrillo, a., & khan, a. a. (2019). risk assessment of china-pakistan fiber optic project (cpfop) in the light of multicriteria decision making (mcdm). advanced engineering informatics, 40, 36-45. doi: https://doi.org/10.1016/j.aei.2019.03.005 ali, y., iftikhar, n., & edwin, c. a. (2017). assessment of career selection problems in developing countries: a mcdm approach. . international journal of the analytic hierarchy process, 9(2), 219-243. doi: https://doi.org/10.13033/ijahp.v9i2.488 ali, y., shah, z. a., & khan, a. u. (2018). post-terrorism image recovery of tourist destination: a qualitative approach using fuzzy-vikor. journal of tourism analysis:doi: https://doi.org/10.1108/jta-05-2018-0016 aljazeera (2019). india-pakistan tensions: latest updates. https://www.aljazeera.com/news/2019/02/india-pakistan-tensions-latest-updates190227063414443.html allsopp, p., & clarke, s. (2004). rating teams and analysing outcomes in one-day and test cricket. journal of the royal statistical society series a, 164(4), 657-667. doi: https://doi.org/10.1111/j.1467-985x.2004.00505.x awan, m. a., & ali, y. (2019). sustainable modeling in reverse logistics strategies using fuzzy mcdm. management of environmental quality, 30(5), 1132-1151. doi: https://doi.org/10.1108/meq-01-2019-0024 balcerzak, a., & pietrzak, m. b. (2016). application of topsis method for analysis of sustainable development in european union countries. the 10th international days of statistics and economics. conference proceedings.prague: institute of economic research. beutler, i. (2008). sport serving development and peace: achieving the goals of the united nations through sport. sport in society , 11(4), 359-369. doi: https://doi.org/10.1080/17430430802019227 byun, d.-h. (2001). the ahp approach for selecting an automobile purchase model. information and management, 38(5), 289-297. doi: https://doi.org/10.1016/s03787206(00)00071-9 ceballos, b., lamata, m., & pelta, d. (2016). a comparitive analysis of mcdm methods. progress in artificial intelligence, 5(4), 315-322. https://www.aljazeera.com/news/2019/02/india-pakistan-tensions-latest-updates-190227063414443.html https://www.aljazeera.com/news/2019/02/india-pakistan-tensions-latest-updates-190227063414443.html ijahp article: ali, arif, munir, azeem, khan/can india-pakistan sports promote peace? a mcdm approach international journal of the analytic hierarchy process 77 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.642 chan, f., & kumar, n. (2007). global supplier development considering risk factors using fuzzy extended ahp-based approach. omega, 35(4), 417-431. doi: https://doi.org/10.1016/j.omega.2005.08.004 deng, h., yeh, c.-h., & willis, r. (2000). inter-company comparison using modified topsis with objective weights. computers & operations research, 27(10), 963973. doi: https://doi.org/10.1016/s0305-0548(99)00069-6 dožić, s., & kalić, m. (2014). an ahp approach to aircraft selection process. 17th meeting of the euro working group on transportation, ewgt2014. sevilla: transportation research procedia. doi: https://doi.org/10.1016/j.trpro.2014.10.102 golden, b. l., wasil, e. a., & harker, p. t. (1989). the analytic hierarchy process. berlin, heidelberg: springer. gupta, h., & kumar, m. (2018). a frame work to overcome barriers to green innovation in smes using bwm and fuzzy topsis. science of the total environment, 633, 122-139. doi: https://doi.org/10.1016/j.scitotenv.2018.03.173 howtheyplay.com. (2018). how they play. retrieved nov 1, 2018, from https://howtheyplay.com: https://howtheyplay.com/team-sports/types-of-cricketmatches hwang, c.-l., & yoon, k. (1981). methods for multiple attribute decision making. berlin, heidelberg: springer. iftikhar, m. (2014). sportsanddev.org. retrieved oct 5, 2018, from https://www.sportanddev.org/en/article/news/role-cricket-india-pakistan-conflict laghate, g. (2017). the econimic times. retrieved nov 15, 2018, from https://economictimes.indiatimes.com/industry/media/entertainment/media/indiapakistan-odi-creates-tv-viewership-history/articleshow/59168451.cms metin dag˘deviren, s. y. (2008). weapon selection using the ahp and topsis methods under fuzzy environment. expert systems with applications, 36(4), 81438151. doi: https://doi.org/10.1016/j.eswa.2008.10.016 murray, s. (2012). the two halves of sports-diplomacy. diplomacy & statecraft, 23(3), 576-592. doi: https://doi.org/10.1080/09592296.2012.706544 press, c. u. (2002). cambridge advanced learner's dictionary & thesaurus. retrieved nov 8, 2018, from https://dictionary.cambridge.org: https://dictionary.cambridge.org/dictionary/english/exhibition-match rizvi, u. h. (2015). let cricket diplomacy strengthen pakistan-india relations. retrieved oct 8, 2018, from https://blogs.tribune.com.pk/story/27617/let-cricketdiplomacy-strengthen-pakistan-india-relations/ ijahp article: ali, arif, munir, azeem, khan/can india-pakistan sports promote peace? a mcdm approach international journal of the analytic hierarchy process 78 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.642 rose, l., & sisson, r. (1991). war and secession: pakistan, india, and the creation of bangladesh (2 nd ed.). california, berkeley: university of california press. doi: https://doi.org/10.1086/ahr/96.4.1265 saaty, t. l. (1980). the analytical hierarchy process (2 ed.). pittsburgh: mcgraw hill. showkat, n. (2018). cricket diplomacy between india and pakistan: a case study of leading national dailies of both the countries (the hindu & dawn). journal of mass communication & journalism, 3(1) 1-10. doi: https://doi.org/10.4172/21657912.1000142 stelter, b., & cohen, n. (2008, november 30). citizen journalists provided glimpses of mumbai attacks. retrieved july 10, 2019, from https://www.nytimes.com/2008/11/30/world/asia/30twitter.html ijahp article: ali, arif, munir, azeem, khan/can india-pakistan sports promote peace? a mcdm approach international journal of the analytic hierarchy process 79 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.642 appendix a table b alternatives c1 c2 c3 c4 c5 a1 (0;6.4;10) (0;7.10769;10)(0;7.06153;10)(0;7.18461;10)(0;6.15384;10) a2 (0;6.24615;10)(0;6.81538;10)(0;6.81538;10)(0;6.50769;10)(0;5.78461;10) a3 (0;6.36923;10)(0;6.76923;10)(0;6.67692;10)(0;6.63076;10)(0;6.16923;10) factors table b1 alternatives c1 c2 c3 c4 c5 a1 (0;0.64;1) (0;0.710769;1)(0;0.706153;1)(0;0.718461;1)(0;0.615384;1) a2 (0;0.624615;1)(0;0.681538;1)(0;0.681538;1)(0;0.650769;1)(0;0.578461;1) a3 (0;0.636923;1)(0;0.676923;1)(0;0.667692;1)(0;0.663076;1)(0;0.616923;1) normalization of matrix factors table b2 alternatives c1 c2 c3 c4 c5 a1 (0;0.1685;0.2634) (0;0.1219;0.1716) (0;0.1211;0.1716)(0;0.1413;0.1967)(0;0.1210;0.1967) a2 (0;0.1476;0.2634) (0;0.1169;0.1716) (0;0.1169;0.1716)(0;0.1280;0.1967)(0;0.1137;0.1967) a3 (0;0.1505;0.2634) (0;0.1161;0.1716) (0;0.1145;0.1716)(0;0.1304;0.1967)(0;0.1213;0.1967) a* (0;0.1685;0.2634) (0;0.1219;0.1716) (0;0.1211;0.1716)(0;0.1413;0.1967)(0;0.1213;0.1967) a(0;0.1476;0.2634) (0;0.1161;0.1716) (0;0.1145;0.1716)(0;0.1280;0.1967)(0;0.1137;0.1967) weighted decision matrix factors table b3 c1 c2 c3 c4 c5 d* a1 0 0 0 0 0.01 0.01 a2 0.0834 0.0408 0.0374 0.0665 0.0503 0.2784 a3 0.0774 0.0439 0.0469 0.0603 0 0.2285 computation of fpis table b4 c1 c2 c3 c4 c5 da1 0.0834 0.0439 0.0469 0.0665 0.0493 0.29 a2 0 0.0163 0.0283 0 0 0.0446 a3 0.0311 0 0 0.0283 0.0503 0.1097 computation of fnis ijahp article: ali, arif, munir, azeem, khan/can india-pakistan sports promote peace? a mcdm approach international journal of the analytic hierarchy process 80 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.642 appendix b ijahp article: ali, arif, munir, azeem, khan/can india-pakistan sports promote peace? a mcdm approach 81 ijahp news and events: international symposium on analytic hierarch processisahp 2018 international journal of the analytic hierarchy process 300 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.498 the upcoming isahp conference, isahp 2018, will take place in hong kong, july 1315, 2018 in the grand hyatt hong kong and will be chaired by dr luis vargas and dr jennifer shang. the theme of this conference will be: ahp/anp in technology, entrepreneurship and corporate social responsibility and focuses on three areas that have changed the world in recent years. however, rather than considering these as independent areas, we need to think of them as interlocked because their future depends on each other. thus, it is befitting that the ahp/anp focuses on them at this point in time, for the anp helps us study interdependent systems. without technology, the implementation of the ahp/anp would not have been possible. ahp/anp helps understand entrepreneurship and cultivate entrepreneurial talent to meet social and economic needs, reduce stress from joblessness, and promote social harmony. without the entrepreneurial capabilities and tenacity of the initiators of the theory, the spread of the theory all over the world would not have been possible. finally, corporate social responsibility (csr) comprises different stakeholders, who are concerned with sustainability, quality, and corporate brand image. csr is multi-faceted and has longterm impacts on economic, social, and environmental well-being, especially in health, safety, equity, humanity, child-care, and education. without the ability to measure intangibles, it would not have been possible to understand the contribution that corporations are making to our society in the social dimension. in a sense, they represent the values of the creator of the theory, thomas l. saaty, who pushed developers beyond their comfort zones and against all odds, to create technology and spread ideas all over the world to show that intangibles can be measured in relative terms. these three focus areas should be explored in all disciplines. researchers and practitioners who would like to attend this symposium are encouraged to explore ideas in new and established areas without fear of being wrong. if it takes hundreds of times to solve a problem, we will have learned more than just to solve the problem, but hundreds of ways not to solve that problem. http://www.isahp.org/location/ http://www.isahp.org/ http://www.isahp.org/ ijahp news and events: international symposium on analytic hierarch processisahp 2018 international journal of the analytic hierarchy process 301 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.498 intuition, past experience and knowledge are the primary requisites of good decision making. however, without measurement all this is futile. measurement is needed to understand the strength of relationships in multidimensional systems. these are the ingredients of the focus areas of this symposium. the venue : grand hyatt hong kong this venue is not only nice, but is strategically located next to shopping areas and public transportation. we expect a large participation of academics from the asian region, in addition to our customary attendees. hong kong is not only a beautiful and vibrant destination but, in spite of the common perception, it is relatively easy to travel to and quite affordable. hong kong does not require a visa for an impressively large number of countries and it is also a hub for many airlines. mark your calendar and stay tuned for our isahp 2018 call for papers to be released soon by our program co-chairs dr. luis vargas and dr. jennifer shang. important dates: open submission system to authors 1 november 2017 individual paper presentations submission deadline 6 march 2018 poster presentations submission deadline 15 april 2018 http://www.isahp.org/location/ ijahp news and events: saaty/isahp 2018 conference in hong kong, july 12-15 international journal of the analytic hierarchy process 298 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.594 isahp2018 conference in hong kong, july 12-15 rozann saaty this year the international symposium on the analytic hierarchy process (isahp2018) was dedicated to the memory of thomas saaty, the father of the analytic hierarchy process, and my husband of 52 years, who passed away on august 17, 2017, at the age of 91. the conference chair was luis vargas, tom’s longtime colleague at the university of pittsburgh's joseph m. katz graduate school of business where he held the chair of distinguished university professor. their names are linked on many ahp articles and books that they co-authored during the past 40 years. the conference was an outstanding gathering of friends and colleagues from tom’s long lifetime. among the attendees were people who served as chairmen from seven previous conferences: luis vargas (pittsburgh, 1991 and hong kong, 2018), kirti peniwati (bali, 2005) rozann saaty (pittsburgh 2009), fabio defelice and antonella petrillo (sorrento, 2011), rafikul islam (kuala lumpur, 2013), and enrique mu (washington, d.c., 2014). for those who weren’t able to attend, and for those who were there and would like to remember the conference, i am going to summarize the main events in this symposium day by day. day 1 –thursday, july 12, pre-conference workshops the first day of the conference was a workshop day that began with a 3-hour seminar by dr. elena rokou on the superdecisions software for ahp and anp. elena rokou is the executive research director of creative decisions foundation and has designed the new look and updated the superdecisions software which the foundation makes available to students and researchers around the world. intangibles can be measured using human judgment and included in multicriteria decision making with the analytic hierarchical process (ahp) and the analytic network process (anp). they are the most powerful synthesis methodologies for combining judgments and data to effectively rank options and predict outcomes (https://www.superdecisions.com). https://www.superdecisions.com/ ijahp news and events: saaty/isahp 2018 conference in hong kong, july 12-15 international journal of the analytic hierarchy process 299 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.594 luis vargas, elena rokou, and her daughter francesca. the youngest conference attendee luis vargas, professor at the university of pittsburgh, presented the second workshop on conducting negotiations, an extension of the "getting to yes" approach where parties do not see each other as adversaries but as collaborative teams, looking for a fair and equitable solution to the conflict. the ahp is a perfect approach to negotiation and conflict resolution because with it negotiation teams can measure intangibles and valuerelated issues that are usually at the heart of the process and arrive at equitable tradeoffs. in the workshop, professor vargas demonstrated a unique way to handle negotiations using ahp. he has facilitated conflict resolution sessions for the past 10 years involving the israelis and palestinians. the results from this work were valuable to the participants from both sides as they found feasible tradeoffs among the many issues that will have to be resolved once the two parties decide they are actually willing to make peace. when they decide to make peace, even though it has not been in the cards so far, this work will prove to be invaluable in working out the details of the settlement. ijahp news and events: saaty/isahp 2018 conference in hong kong, july 12-15 international journal of the analytic hierarchy process 300 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.594 enrique mu presenting a workshop on publishing ahp/anp papers the third workshop was presented by dr. enrique mu, professor at carlow university and summer program director at the university of pittsburgh. it was an extremely valuable workshop about how to improve your chances of getting your ahp/anp paper published. enrique is the editor-in-chief of the international journal of the ahp (ijahp), now indexed in scopus, and has served as a reviewer for many publications as well as publishing many ahp/anp papers of his own. during the workshop he gave each participant a comprehensive but succinct list of what to include in a paper and equally important what not to do. he emphasized the need to be specific about who had made the judgments and why they were qualified to do so. the last workshop, presented by me, was “the art of structuring ahp and anp models”. i focused on the basics of making a judgment in answering a pairwise comparison question. a judgment on a pair of elements should always hold the dominated element in mind as the unit, estimate how many times larger the dominant one is, and enter that number in the pairwise comparison matrix. too often users treat the fundamental 1 to 9 scale of the ahp as an ordinal scale, rather than a scale of absolute numbers. the solution vector for the pairwise comparison matrix is a relative absolute scale with ratio properties. it does not have a zero. i also covered how to prevent “rank reversal”. a long-time criticism leveled at the ahp is that the ranks of existing alternatives can change when new ones are added or removed. this can be avoided by re-weighting the criteria rozann saaty presenting a workshop ijahp news and events: saaty/isahp 2018 conference in hong kong, july 12-15 international journal of the analytic hierarchy process 301 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.594 to properly include the effect of the structural dependence among elements. there are examples where rank reversal occurs and is understandable and desirable, so it is more a question of “which is correct for this situation?” there are completely understandable reasons for either allowing rank reversal or not allowing it, and it is up to the decision maker to decide in advance which is appropriate. the welcome reception on the opening workshop day provided a chance to relax and enjoy a beverage at the beautiful pool house of the hyatt hotel after a full day of attending workshops. day 2 –friday, july 13, first regular conference day luis vargas opening the conference with a welcome address friday morning luis vargas, chairman of isahp2018, and professor at the katz graduate school, university of pittsburgh, where tom saaty worked for 30 years, opened the meeting with a welcome address. luis worked with tom from the time he arrived at the wharton school of the university of pennsylvania as a young graduate in mathematics on the prestigious juan marche fellowship from spain. tom and luis moved to the katz graduate school of the university of pittsburgh in 1979 and continued their collaboration through the years, writing many seminal papers and books about the ahp and anp and applying the ahp in a significant conflict resolution exercise. they organized negotiation meetings along with jerry zoffer, dean emeritus of the katz graduate school of business, between representatives of palestine and israel for 10 years, the most recent being held in the fall of 2017 after tom passed away. ijahp news and events: saaty/isahp 2018 conference in hong kong, july 12-15 international journal of the analytic hierarchy process 302 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.594 john saaty giving the memorial address for his father the memorial address for tom was given by his son john following luis vargas’ opening welcome. john and his brother daniel founded decision lens, inc. in virginia in 2002, a company that produces software for organizations based on ahp. they have taken the ahp from being a scholarly exercise into the public realm. in his talk, john gave a brief overview of the customers that now use the software, both government agencies and businesses, to align their priorities and resources to do more with less. dr. yong shi, chinese academy of sciences, beijing, china, main keynote address ijahp news and events: saaty/isahp 2018 conference in hong kong, july 12-15 international journal of the analytic hierarchy process 303 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.594 yong shi of key lab of big data mining and knowledge management, chinese academy of sciences, beijing, china, gave the keynote address on this opening day. he talked of tom saaty’s influence on the chinese academic community through ahp/anp and also through his long interest in and lecturing on creative thinking. yong reviewed how ahp/anp has been widely used in china for decision-making problems since the 1980's. tom’s book, creativity, problem solving and decision making, was translated by yong shi and his colleagues into chinese. they have distributed it widely in china so that it has become a bestseller at airport and train stations where many books are sold. yong reminisced about tom’s energy on his last trip to china in october 2016 when he was a guest of the chinese academy of sciences and lectured at seven famous chinese universities around beijing. on the same trip he also visited tianjin university in tianjin, china, where there were still professors at that school who as students in 1988 had attended the first isahp conference in tianjin organized by professor shubo xu. luis vargas and monica garcia-melon, universitat polytecnica of valencia, valencia, spain, enjoying the great food ijahp news and events: saaty/isahp 2018 conference in hong kong, july 12-15 international journal of the analytic hierarchy process 304 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.594 day 2 –friday evening, july 13, awards reception on friday evening there was an awards reception. the best papers from this year were awarded for innovation. one of the papers awarded was that of enrique mu and his students julie e. forbes and abigail m. hebb. enrique mu and students receiving an innovation award for their paper the following papers received an award:  bayesian identification of homogeneous subgroups of actors in a local ahp-multicriteria decision making context (poster presentation), josé maría moreno-jiménez, university of zaragoza; alfredo altuzarra, facultad de economia y empresa universidad de zaragoza (spain); pilar gargallo, universidad de zaragoza; manuel salvador, universidad de zaragoza.  implementation of an online software tool for the analytic hierarchy process (ahp-os), klaus d goepel, business performance management singapore ijahp news and events: saaty/isahp 2018 conference in hong kong, july 12-15 international journal of the analytic hierarchy process 305 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.594  judgment scales of the analytical hierarchy process – the balanced scale, klaus d goepel, business performance management singapore  an anp-based simulation of vulnerability of two-way coupled socio-ecological systems, luis antonio bojórquez-tapia, lancis unam, mexico; hallie eakin, school of sustainability, arizona state university; marco a janssen, school of sustainability, arizona state university; andrés baeza, school of sustainability, arizona state university  project evaluation in local governments to realize women active promotion society in japan, ryo koizumi, suwa university of science; yoichi iida, suwa university of science  a multi-criteria model for selecting the most suitable classifier for supporting assistive technology adoption in people with dementia, miguel angel ortiz barrios, universidad de la costa, colombia; antanas verikas, halmstad university; chris nugent, ulster university; mark donnelly, ulster university; leo galway, ulster university; macarena espinilla, university of jaen; ian cleland, ulster university  risk perception of uncertainties in supply chain, yuji sato, graduate school of management, chukyo university, japan  using the ahp to establish inclusive housing development priorities for industry, ali lakhani, griffith university; heidi zeeman, griffith university; rafikul islam, international islamic university malaysia; david watling, griffith university; courtney j. wright, griffith university; dianne smith, curtin university  an anp model to improve pedestrian accessibility in the city center of cartagena de indias (colombia), monica garciamelon, universitat politecnica de valencia, spain; hannia karime gonzálezurango, universitat politecnica de valencia; michela le pira, university of catania; giuseppe inturri, university of catania  assessment of health-related quality of life (hrql) in low-back pain patients before and after vax-d disc decompression treatment. analysis using the computer assisted analytic hierarchy process (ahp), ivan naumov, cyberspace solutions, inc; albert mancini, fisioterapia center; kenneth vinton, pain relief & wellness strategies center; james kittelberger, physical therapy center; thomas l. saaty, university of pittsburgh, u.s.  ethical decision making in action: evaluating hospital care attendance approaches, julie e. forbes, carlow university upmc presbyterian-shadyside; abigail m. hebb, carlow university upmc ijahp news and events: saaty/isahp 2018 conference in hong kong, july 12-15 international journal of the analytic hierarchy process 306 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.594 presbyterian-shadyside; enrique mu, carlow university college of leadership and social change  program accreditation: a network model for criteria dependencies and prioritization, mujgan sagir ozdemir, esogu, turkey day 3 –saturday, july 14, regular conference day iwan azis, one of indonesia’s most famous economists and ahp scholar professor iwan j. azis presented the first keynote address on saturday titled, “ahp/anp relevance in a more complex global economy”. he has taught at cornell university since the early 1990s. during 2005-2010, he was director of graduate studies of the regional science program there. since 2015 he has also been teaching as faculty of economics and business, university of indonesia. iwan’s talk was especially relevant today because of the tariff wars started by the current president of the united states, donald trump. his vantage point as an eminent economist familiar with both the american position from his long tenure at the university of cornell and from his position as a famous economist very familiar with the economic positions of countries in asia, gave us an inside look at what the prospects are for the world economy in the future. tom and iwan azis were friends from the 1990’s when tom spent weeks at a time in indonesia working with iwan on economic issues specific to indonesia. their close relationship continued with visits back and forth to the u.s., the last one being a trip by iwan to pittsburgh only a few weeks before tom passed away. ijahp news and events: saaty/isahp 2018 conference in hong kong, july 12-15 international journal of the analytic hierarchy process 307 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.594 shashi bhattarai of nepal giving keynote address shashi bhattarai is the chairman of knowledge holding international pvt. ltd., and nepal consultant, the world bank group and united nations industrial development organization (unido). the title of shashi’s address was “in memory of professor thomas saaty: demand driven cases of the analytic hierarchy process application”. he gave a very interesting talk about recent ahp applications in nepal and bhutan with stories derived from demand responsive, practical and internalized utility of ahp by the users, hosts and enthusiasts from that part of the world. shashi shared a story with me about when he first met tom at a conference in ottawa, canada around 1990. he was a young scholar when he attended the conference and tom was being attacked vigorously by critics of the ahp. shashi was impressed by tom’s ability to withstand and rebut the fire. he became an enthusiastic supporter of ahp, applying it many times in nepal and spreading it among other researchers there. ijahp news and events: saaty/isahp 2018 conference in hong kong, july 12-15 international journal of the analytic hierarchy process 308 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.594 hao gang, city university of hong kong, giving keynote address dr. hao gang is a professor in the department of management science at the city university of hong kong. it was our good fortune that she is based in hong kong and agreed to give a talk at the isahp2018 conference. professor hao described an interesting project she is currently involved with that uses ahp. many small and medium sized companies in china today are considering outsourcing their manufacturing to the smaller emerging market asian countries near china. gang and her colleagues are developing an ahp decision template that managers can use to decide which country is best-fitted for their purposes. gang was one of the first phd students from mainland china allowed to come to the u. s. to study. she arrived at the university of pittsburgh shortly after the first isahp meeting in tianjin, china, in 1988, and she received her phd at the katz graduate school of business working with thomas saaty and luis vargas. her talk was particularly impressive because of how thoroughly she understood, remembered and was able to apply ahp. ijahp news and events: saaty/isahp 2018 conference in hong kong, july 12-15 international journal of the analytic hierarchy process 309 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.594 day 4 –sunday, july 15, final conference day dr. kirti peniwati, jakarta, indonesia, giving the final memorial address dr. kirti peniwati of jakarta, indonesia, gave the final keynote address. kirti obtained her phd in decision science from the katz graduate school of business, university of pittsburgh, working with tom and luis vargas and went on to have a distinguished career at the ppm management institute in jakarta, one of the premiere graduate schools in indonesia. she has been an active decision facilitator since her retirement from ppm. kirti wrote an amazing thesis when she was a phd student that said kenneth arrow’s “impossibility of group choice” theorem is not impossible if you apply ahp. she also co-authored the book “group decision making” with tom. in her talk, kirti told how working with tom had affected her life and among other memories recalled that many of their most significant discussions occurred walking to the university together on some mornings. many people in the audience had similar memories and it brought some of them to the point of tears. as his wife and collaborator of 52 years, it has been my pleasure to write this news article and i want to thank the many friends and colleagues of thomas saaty who loved him and came to hong kong, from far places in many cases, to share in this special gathering. thank you! ijahp news and events: saaty/isahp 2018 conference in hong kong, july 12-15 international journal of the analytic hierarchy process 310 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.594 group photo taken just before the conference broke up on the last day identification of outliers and robust estimations in the ahp ijahp article: lipovetsky/priority vector estimation: consistency, compatibility, precision international journal of the analytic hierarchy process 577 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.801 priority vector estimation: consistency, compatibility, precision stan lipovetsky 1 stan.lipovetsky@gmail.com abstract various methods of priority vector estimation are known in the analytic hierarchy process (ahp). they include the classical eigenproblem method given by thomas saaty, developments in least squares and the multiplicative approach, robust estimation based on transformation of the pairwise ratios to the shares of preferences, and other approaches. in this paper, the priority vectors are completed with validation of data consistency, comparisons of vectors’ compatibility, and estimation of precision for matrix approximation by vectors. the numerical results for different data sizes and consistency show that the considered methods reveal useful features, are simple and convenient, and capable of facilitating practical applications of the ahp in solving various multiplecriteria decision making problems. keywords: ahp priority vector estimations; consistency measures; s-index and g-index of compatibility; precision of fitting 1. introduction the analytic hierarchy process (ahp) is a widely used methodology and a set of methods for solving various problems of prioritization. developed by thomas saaty and expanded in numerous works by many authors, it is currently one of the main approaches for managers and practitioners who need to apply multi-criteria decision making to reach their goals. in this work, the term ahp is used not in its whole rich entirety but in a narrower sense as a method of finding local priority vectors by a pairwise comparison matrix. estimations of priority vectors in the ahp include the classical eigenproblem method (em) proposed by saaty (1977, 1980, 1994, 1996, 2005), the least squares (ls) solution and the multiplicative or logarithmic (ln) least squares described in saaty and vargas (1984, 1994) and lootsma (1993, 1999), and numerous other modifications (for instance, lipovetsky, 1996, 2009, 2013; lipovetsky and tishler, 1999). particularly, priority vector robust estimation (re) which is not prone to possible inconsistencies in pairwise judgements can be based on the ratio transformation to the shares of preferences and obtained by markov chain modeling for steady-state probabilities (lipovetsky and conklin, 2002, 2015). the current work presents the results of comparisons between the em, ln, ls, and re using several characteristics of closeness for the obtained solutions, including pair 1 acknowledgement: i am grateful to three reviewers whose comments improved the work. ijahp article: lipovetsky/priority vector estimation: consistency, compatibility, precision international journal of the analytic hierarchy process 578 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.801 correlations, the so-called saaty compatibility index (s-compatibility) (saaty, 2005; saaty and peniwati, 2007), and the garuti compatibility index or g-compatibility (garuti, 2007; garuti and salomon, 2011). for different sizes and consistency of the matrices of judgement used in the classical ahp literature, the priority vectors are calculated, their compatibility indices estimated, and characteristics of the matrix fit by the vectors are described. in general, the explored methods are simple and convenient and can significantly facilitate practical applications of the ahp for optimum solutions in various problems. the paper is organized as follows: section 2 describes the methods of priority estimation, section 3 defines the measures of compatibility and quality of fit, section 4 discusses several numerical examples, and section 5 concludes on the obtained results. 2. priority vector estimations let us briefly describe several main methods of priority vector estimations. the general form of the ahp pairwise priority ratios matrix can be written as follows: 𝐴 = ( 1 𝑎12 . . . 𝑎1𝑛 𝑎21 1 . . . 𝑎2𝑛 . . . . . . . . . . . . . . . . . . . . . . . . . . . 𝑎𝑛1 𝑎𝑛2 . . . 1 ). (1) this is a saaty matrix of pairwise judgements among n items, elicited from an expert. each element aij shows a quotient of preference of the i-th item over the j-th item in their comparison, so we have the transposed-reciprocal elements 𝑎𝑖𝑗 = 𝑎𝑗𝑖 −1. a theoretical saaty matrix of pair comparisons defines each ij-th element as a ratio of the unknown priorities wi and wj: 𝑊 = ( 𝑤1/𝑤1 𝑤1/𝑤2 ... 𝑤1/𝑤𝑛 − − − − − − − − − − − − − − − − 𝑤𝑛 /𝑤1 𝑤𝑛 /𝑤2 ... 𝑤𝑛 /𝑤𝑛 ) = 𝑤 ∗ ( 1 𝑤 )′. (2) the vector-column w consists of the elements w1, w2,…, wn, the vector-row (1/w)’ contains the reciprocal values 1/w1, 1/w2,…, 1/wn, and the right-hand side of the relationship (equation 2) shows the outer product of these two vectors (where the prime denotes transposition). from equation (2), it is easy to find the identical relation 𝑊𝑤 = 𝑛𝑤 for the theoretical matrix and vector. for the obtained matrix (equation 1), a similar relationship can be presented as the eigenproblem: 𝐴𝛼 = 𝜆𝛼 , (3) where the first eigenvector alpha for the maximum eigenvalue 𝜆 defines the vector of priorities. this is the eigenvector method em of the classical ahp. another known way is the least squares estimation for the priority vector which can be expressed via the following eigenproblem: ijahp article: lipovetsky/priority vector estimation: consistency, compatibility, precision international journal of the analytic hierarchy process 579 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.801 (𝐴𝐴′)𝛼 = 𝜆2𝛼 , (4) the main vector alpha yields the priority vector in the ls approach. the third popular approach to priority estimation is called the multiplicative or logarithmic technique. it can be reduced to calculating the elements of the priority vector as the geometric means of the elements in each row of the matrix (equation 1): 𝛼𝑖 = √∏ 𝑎𝑖𝑗 𝑛 𝑗=1 𝑛 . (5) the obtained ahp priority vectors are also standardized by the total of the elements, so a solution is divided by the total of all of the elements and the sum of the normalized components equals one: 𝛼𝑖 𝑛𝑜𝑟𝑚𝑎𝑙𝑖𝑧𝑒𝑑 = 𝛼𝑖 /𝑠𝑢𝑚(𝛼𝑖 ) . (6) this is the priority vector estimation in the ln approach. the solution with robust estimation (re) is less prone to possible inconsistencies in the pairwise judgements; let us introduce a theoretical matrix of shares as follows: 𝑈 = ( 𝑤1/(𝑤1 + 𝑤1) 𝑤1/(𝑤1 + 𝑤2) ... 𝑤1/(𝑤1 + 𝑤𝑛 ) − − − − − − − − − − − − − − − − 𝑤𝑛 /(𝑤𝑛 + 𝑤1) 𝑤𝑛 /(𝑤𝑛 + 𝑤2) ... 𝑤𝑛 /(𝑤𝑛 + 𝑤𝑛 ) ), (7) each element uij in equation (7) is defined as i-th priority in the sum of i-th and j-th priorities: 𝑢𝑖𝑗 = 𝑤𝑖 𝑤𝑖+𝑤𝑗 = 𝑤𝑖/𝑤𝑗 1+𝑤𝑖/𝑤𝑗 . (8) to estimate the priority vector using the matrix (equation 7), we write identical equalities: { 𝑤1 𝑤1+𝑤1 (𝑤1 + 𝑤1) + 𝑤1 𝑤1+𝑤2 (𝑤1 + 𝑤2)+. . . + 𝑤1 𝑤1+𝑤𝑛 (𝑤1 + 𝑤𝑛 ) = 𝑛𝑤1 − − − − − − − − − − − − − − 𝑤𝑛 𝑤𝑛+𝑤1 (𝑤𝑛 + 𝑤1) + 𝑤𝑛 𝑤𝑛+𝑤2 (𝑤𝑛 + 𝑤2)+. . . + 𝑤𝑛 𝑤𝑛+𝑤𝑛 (𝑤𝑛 + 𝑤𝑛 ) = 𝑛𝑤𝑛 . (9) then, with notation (equation 8) we present the system (equation 9) as: { (𝑢11 + ∑ 𝑢1𝑗 𝑛 𝑗=1 )𝑤1 + 𝑢12𝑤2+. . . +𝑢1𝑛𝑤𝑛 = 𝑛𝑤1 − − − − − − − − − − 𝑢𝑛1𝑤1 + 𝑢𝑛2𝑤2+. . . +(𝑢𝑛𝑛 + ∑ 𝑢𝑛𝑗 𝑛 𝑗=1 )𝑤𝑛 = 𝑛𝑤𝑛   . (10) ijahp article: lipovetsky/priority vector estimation: consistency, compatibility, precision international journal of the analytic hierarchy process 580 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.801 in the matrix form the system (equation 10) can be written as: (𝑈 + 𝑑𝑖𝑎𝑔(𝑈𝑒))𝑤 = 𝑛𝑤, (11) where u is the matrix (equation 7), e denotes a uniform vector of n-th order, and diag(ue) is a diagonal matrix of totals in each row of matrix u. in the classical ahp, the pair ratios wi/wj (equation 2) are estimated by the elicited values aij (equation 1). using aij in equation (8), we obtain the empirical estimates bij of the pairs’ shares: 𝑏𝑖𝑗 = 𝑎𝑖𝑗 1+𝑎𝑖𝑗 . (12) this transformation of the elements of a matrix a (equation 1) yields a pairwise share matrix b with the elements (equation 12). these elements (equation 12) are positive, less than one, and have a property 𝑏𝑖𝑗 + 𝑏𝑗𝑖 = 1. this means that the transposed elements bij and bji are skew-symmetrical off the diagonal bii=0.5, so 𝑏𝑖𝑗 − 𝑏𝑖𝑖 = −(𝑏𝑗𝑖 − 𝑏𝑖𝑖 ). for the empirical saaty matrix a (equation 1), we have the eigenproblem (equation 3) in place of the theoretical relationship (equation 2). similarly, using the empirical skewsymmetric matrix b (equation 12) in place of theoretical matrix u, we represent the system (equation 11) as the eigenproblem: (𝐵 + 𝑑𝑖𝑎𝑔(𝐵𝑒))𝛼 = 𝜆𝛼, (13) where α as the main eigenvector. this is the re vector of priority, and its properties have been studied in the works of lipovetsky and conklin (2002, 2015). 3. measures of consistency, compatibility, and precision due to the general methodology of the ahp, the so-called consistency index (ci) equals 𝐶𝐼 = 𝜆−𝑛 𝑛−1 (14) where 𝜆 is the maximum eigenvalue of the matrix in the problem (equation 3), and n is the matrix order. the so-called random consistency index (ri) is a constant tabulated in the ahp for various n, and the consistency ratio (cr) equals the following value: 𝐶𝑅 = 𝐶𝐼 𝑅𝐼 . (15) a value of cr up to 10% is considered to indicate a small inconsistency in the matrix of the pairwise comparisons (equation 1), and therefore an acceptable matrix; however, if the cr > 10%, a review of the elicited judgements could be required. ijahp article: lipovetsky/priority vector estimation: consistency, compatibility, precision international journal of the analytic hierarchy process 581 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.801 for comparisons between the obtained solutions, several characteristics can be applied. among those are the pairwise correlation between the elements of two vectors, which can be reduced to the expression: 𝑟(𝑥, 𝑦) = ∑ (𝑥𝑖−�̅�)(𝑦𝑖−�̅�) 𝑛 𝑖=1 √∑ (𝑥𝑖−�̅�) 2𝑛 𝑖=1 ∑ (𝑦𝑖−�̅�) 2𝑛 𝑖=1 = ∑ 𝑥𝑖𝑦𝑖− 1 𝑛 𝑛 𝑖=1 √∑ 𝑥𝑖 2− 1 𝑛 𝑛 𝑖=1 √∑ 𝑦𝑖 2− 1 𝑛 𝑛 𝑖=1 , (16) where a bar above the variables denotes the mean values and those equal 1/n for the vectors normalized by equation (6). without the items 1/n for centering (and this value is small for a bigger n), the measure (equation 16) coincides with the cosine as a normalized projection of one vector onto another one. the closer a correlation or cosine is to 1, the higher the similarity of the two solutions. the cosine values repeat the correlations but are slightly bigger, so for a more conservative measure the correlation is preferred. another good measure of closeness between two vectors is the so-called saaty compatibility index (s-compatibility) (saaty, 2005; saaty and peniwati, 2007; garuti and salomon, 2011). this index can be built as follows. for two vectors x and y of an n-th order, build a matrix x with its elements defined as quotients xij=xi/xj of the components of the vector x, and a matrix y with its elements defined as quotients yij=yi/yj of the components of the vector y. take the transposed matrix y’ with the elements y’ij=yj/yi and find the hadamard element-wise product of these two matrices x*y’, then the sindex is defined as the normalized total of the elements of this matrix: 𝑆 = 1 𝑛2 ∑ 𝑋𝑖𝑗 𝑌𝑖𝑗 ′𝑛 𝑖,𝑗=1 = 1 𝑛2 ∑ 𝑥𝑖 𝑥𝑗 𝑛 𝑖,𝑗=1 𝑦𝑗 𝑦𝑖 . (17) if two vectors coincide, this index equals 1. within 10% of discrepancy, when 𝑆 ≤ 1.1, the vectors are considered compatible; otherwise, when 𝑆 > 1.1, they are incompatible (saaty and peniwati, 2007). a further development of a compatibility measure in the so-called compatibility index g was proposed in garuti (2007) and garuti and salomon (2011) where it was defined as: 𝐺 = ∑ 𝑚𝑖𝑛(𝑥𝑖,𝑦𝑖) 𝑚𝑎𝑥(𝑥𝑖,𝑦𝑖) 𝑛 𝑖=1 𝑥𝑖+𝑦𝑖 2 . (18) due to recommendation in garuti (2007), the values g < 0.9 correspond to incompatible vectors, otherwise the vectors are compatible. to check a precision of fit for the pairwise judgements by the priority vector estimate, we can use a definition of the elements ajk as quotients of preference between each pair of jth and k-th items. with a vector-column of priority estimate alpha, we find its elementreciprocal vector-row (1/alpha)’ and build their outer product by the same pattern as used in equation (2). with this outer product we find the quality of its fit for the matrix a equation (1). the standard error (ste) is a measure of the mean distance between the observed and estimated pairwise ratios: 𝑆𝑇𝐸 = √ 1 𝑛2 ∑ (𝑎𝑗𝑘 − 𝛼𝑗 𝛼𝑘 ) 2 𝑛 𝑗,𝑘=1 . (19) ijahp article: lipovetsky/priority vector estimation: consistency, compatibility, precision international journal of the analytic hierarchy process 582 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.801 another convenient measure of the precision for a matrix approximation by the vectors outer product is the mean absolute error (mae): 𝑀𝐴𝐸 = 1 𝑛2 ∑ |𝑎𝑗𝑘 − 𝛼𝑗 𝛼𝑘 |𝑛𝑗,𝑘=1 . (20) the smaller the values of fit (equations 19-20), the better the quality of the vector estimate. the measures of ste and mae can be obtained by using equations (19-20) only for the off-diagonal pairwise ratios equal or above 1 because they correspond to the elicited quotients of preference, and the reciprocal values below 1 are simply added at completion of the matrix (equation 1) of pairwise judgements. besides the characteristics of the residual mean values assessed via standard deviation (equation 19) or absolute deviation (equation 20), the quality of approximation of the pairwise judgements by the obtained priority vectors can be checked by a measure reminding the coefficient of multiple determination r 2 that is widely used in regression analysis. as shown in lipovetsky (2009), this coefficient can be defined via the observed and estimated paired ratios of the priorities: 𝑅2 = 1 − 𝑅𝑆𝑆 𝐸𝑆𝑆 = 1 − ∑ (𝑎𝑗𝑘− 𝛼𝑗 𝛼𝑘 ) 2 𝑛 𝑗,𝑘=1 ∑ (𝑎𝑗𝑘−1) 2𝑛 𝑗,𝑘=1 . (21) in the numerator (equation 21) the residual sum of squares (rss) of the estimated priority deviations from the elicited values is used, and the denominator is presented by the equivalent sum of squares (ess) which assumes all the same preferences αj /αk ≡ 1. the coefficient (equation 21) shows how much the found priorities outperform the case of absence of preferences among the alternatives. the better the approximation of the paired judgements by the estimated priorities is the closer the rss is to zero, so the coefficient of determination r 2 is bigger and closer to one. in the absence of preferences αj/αk = 1, the numerator equals the denominator, and r 2 = 0. for the exact fit ajk = αj/αk for all judgements, rss = 0, and r 2 = 1. the value r 2 commonly belongs to the interval from 0 to 1, which makes it a very convenient measure for comparison of the priority vectors obtained by different techniques. only really poor estimates can produce a residual total rss above the value of the equivalent residuals ess, and it would be indicated by a negative r 2 . the characteristic (equation 21) corresponds to the ste measure (equation 19) of squared deviations, but it is possible to build the other estimates, for example, using the mae residuals (equation 20) as well. 4. numerical comparisons for priority estimations let us consider numerical examples of the priority estimations for three classical ahp problems. example 1: the problem of “choosing the best home” as described in saaty and kearns (1985), saaty and vargas (1994) and saaty (1996). this matrix is also used for checking ijahp article: lipovetsky/priority vector estimation: consistency, compatibility, precision international journal of the analytic hierarchy process 583 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.801 some new approaches (lipovetsky, 1996; lipovetsky and tishler, 1999; lipovetsky and conklin, 2002, 2015). the criteria of comparison are: 1 – size of house, 2 – location to bus, 3 – neighborhood, 4 – age of house, 5 yard space, 6 – modern facilities, 7 – general condition, 8 – financing. the matrix of pairwise comparisons a (equation 1) for this problem is presented in table 1a. in this example with n=8, the maximum eigenvalue (equation 3) of the matrix in table 1a equals 𝜆 = 9.669. with the random consistency for this case ri=1.41, the consistency index and consistency ratio (equations 14-15) are: 𝐶𝐼 = 9.669−8 7 = 0.238, 𝐶𝑅 = 0.238 1.41 = 0.169. a value of cr up to 10% is considered to indicate some inconsistency, so the obtained result of 17% is acceptable with a reservation, when the data could require a review of the elicited judgements, and in lipovetsky and conklin (2002) it was shown how to identify and to adjust the data in this case. table 1a example 1: choosing the best home problem. pairwise comparison matrix. item 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 1/5 1/3 1/7 1/6 1/6 3 4 5 1 3 1/5 1/3 1/3 5 7 3 1/3 1 1/6 1/3 1/4 1/6 5 7 5 6 1 3 4 7 8 6 3 3 1/3 1 2 5 6 6 3 4 1/4 1/2 1 5 6 1/3 1/5 6 1/7 1/5 1/5 1 2 1/4 1/7 1/5 1/8 1/6 1/6 1/2 1 several methods of priority estimation for this data are presented in table 1b. in its upper part, there are four estimates of the priority vector as follows: the classical em solution (equation 3), the ls estimation (equation 4), the ln technique (equation 5), and the robust estimation re (equation 13). all the vectors are normalized by the total of their elements equaling one (equation 6). judging by eye, all of the solutions are very similar by weight and the smallest and the biggest by importance are the age and financing of the house, items 4 and 8, respectively. for comparison between the obtained four priority vectors we applied the measures (equations 16-18) of correlations, s-compatibility, and gcompatibility, presented after the vectors in the three matrices in table 1b. judging by the correlations, all of the vectors are close enough by their structure and the ls is a bit further from the other three. the measure of s-compatibility proves that the em, ln, and re vectors are similar, within less than the required threshold of 10% of the s-index deviation from one. the more sensitive g-compatibility demonstrates that the pair of em and ln vectors are close with g=0.927, and the two vectors ln and re are close with g=0.912, which are ijahp article: lipovetsky/priority vector estimation: consistency, compatibility, precision international journal of the analytic hierarchy process 584 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.801 values above the threshold 0.9 needed for viewing the corresponding vectors as compatible. table 1b example 1: choosing the best home problem. priority vector estimations item em ls ln re 1. size of house 0.173 0.199 0.175 0.150 2. location to bus 0.054 0.100 0.063 0.054 3. neighborhood 0.188 0.148 0.149 0.141 4. age of house 0.018 0.017 0.019 0.022 5. yard space 0.031 0.045 0.036 0.037 6. modern facilities 0.036 0.065 0.042 0.041 7. general condition 0.167 0.184 0.167 0.163 8. financing 0.333 0.242 0.350 0.392 correlations em 1 0.935 0.988 0.972 ls 0.935 1 0.933 0.881 ln 0.988 0.933 1 0.991 re 0.972 0.881 0.991 1 s-compatibility em 1 1.113 1.015 1.028 ls 1.113 1 1.071 1.122 ln 1.015 1.071 1 1.010 re 1.028 1.122 1.010 1 g-compatibility em 1 0.774 0.927 0.865 ls 0.774 1 0.809 0.742 ln 0.927 0.809 1 0.912 re 0.865 0.742 0.912 1 precision ste 2.071 1.849 1.813 1.831 mae 1.079 1.083 0.958 0.934 r 2 0.423 0.540 0.558 0.549 the last segment at the bottom of table 1b displays the precision by equations (19-21) for each vector solution. by the minimum standard error ste, the best model is the ln, and by the mean absolute error mae, the best model is the re. the values of the mae also suggest that an average deviation from the observed pair judgements evaluated by the obtained quotients from a priority vector is not more than one unit. the coefficient of multiple determination r 2 in the last row of table 1b shows by its maximum values that the ln and re models outperform the other two models, though all r 2 values are not ijahp article: lipovetsky/priority vector estimation: consistency, compatibility, precision international journal of the analytic hierarchy process 585 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.801 high, which indicates a difficulty in approximation of inconsistent judgements by a priority vector in any estimation. example 2: the problem of “distance from philadelphia” is one of the first ahp problems described by saaty (1977). the distance of six cities from philadelphia was estimated by the criterion: for each pair of cities, how many times farther is the more distant city located from philadelphia than the nearer one? the elicited data is presented in table 2a. table 2a example 2: distance from philadelphia problem. pairwise comparison matrix airport cai tyo ord sfo lgw ymx cairo.cai 1 0.333 8 3 3 7 tokyo.tyo 3 1 9 3 3 9 chicago.ord 0.125 0.111 1 0.167 0.2 2 sanfrancisco.sfo 0.333 0.333 6 1 0.333 6 london.lgw 0.333 0.333 5 3 1 6 montreal.ymx 0.143 0.111 0.5 0.167 0.167 1 the maximum eigenvalue (equation 3) in this example equals 𝜆 = 6.454. the random consistency for n=6 is ri=1.24, then the consistency index and consistency ratio equations (14-15) are: 𝐶𝐼 = 6.454−6 5 = 0.091, 𝐶𝑅 = 0.091 1.24 = 0.073 . the value of cr=7.3% is less than the 10% permitted, which allows one to conclude that the data on pair judgements is sufficiently consistent. table 2b presents the results of priority estimations for this example and is organized as the previous table 1b, but with one additional column of the actual shares of distances known in this case. ijahp article: lipovetsky/priority vector estimation: consistency, compatibility, precision international journal of the analytic hierarchy process 586 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.801 table 2b example 2: distance from philadelphia problem. priority vector estimations city em ls ln re actual 1. cairo 0.262 0.254 0.260 0.239 0.278 2. tokyo 0.397 0.305 0.399 0.447 0.361 3. chicago 0.033 0.047 0.035 0.034 0.032 4. san francisco 0.116 0.186 0.116 0.104 0.132 5. london 0.164 0.184 0.163 0.147 0.177 6. montreal 0.027 0.024 0.027 0.029 0.019 correlations em 1 0.943 1.000 0.990 0.991 ls 0.943 1 0.941 0.898 0.973 ln 1.000 0.941 1 0.991 0.990 re 0.990 0.898 0.991 1 0.962 actual 0.991 0.973 0.990 0.962 1 s-compatibility em 1 1.064 1.000 1.009 1.024 ls 1.064 1 1.064 1.106 1.045 ln 1.000 1.064 1 1.008 1.027 re 1.009 1.106 1.008 1 1.060 actual 1.024 1.045 1.027 1.060 1 g-compatibility em 1 0.821 0.993 0.900 0.914 ls 0.821 1 0.820 0.753 0.854 ln 0.993 0.820 1 0.905 0.908 re 0.900 0.753 0.905 1 0.823 actual 0.914 0.854 0.908 0.823 1 precision ste 1.390 1.340 1.333 1.523 2.295 mae 0.696 0.862 0.686 0.790 1.012 r 2 0.794 0.809 0.810 0.753 0.438 we see that in general the vectors are similar and each one makes sense as proportionally scaled distances from philadelphia to other cities in the usa, as well as to other countries and continents. the pair correlations also show that the vectors are closely related to the actual distances, and the same is supported by the s-compatibility index. g-compatibility indicates that the em and ln vectors are compatible with the actual shares of distances. the precision of the reproduction of the judgement matrix is high, especially with the ls and ln methods. the precision measured by ste, mae, and r 2 of the actual distances is the worst one within the other values in the last rows of table 2b. this means that the pair judgements on distances correspond to the priority vectors rather than to the actual distance shares. therefore, in this data we do not need to use the actual data to consider compatibility among the vectors. ijahp article: lipovetsky/priority vector estimation: consistency, compatibility, precision international journal of the analytic hierarchy process 587 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.801 example 3. the data for this problem is given in whitaker (2007) where the area of five geometric figures was compared – see the matrix of pair judgements in table 3a. table 3a example 3: geometric figures’ area problem. pairwise comparison matrix figure circle triangle square diamond rectangle circle 1 9 2.5 3 6 triangle 0.111 1 0.2 0.286 0.667 square 0.4 5 1 1.7 3 diamond 0.333 3.5 0.588 1 1.5 rectangle 0.167 1.5 0.333 0.667 1 the maximum eigenvalue of this matrix is 𝜆 = 5.026. the random consistency for n=5 is ri=1.12, so the consistency index and consistency ratio (equations 14-15) equal the following values: 𝐶𝐼 = 5.026−5 4 = 0.006, 𝐶𝑅 = 0.006 1.12 = 0.006 . cr=0.6% which proves a very high level of consistency of this data. this can be explained by the pairwise ratios that were used where not only the integer numbers but also the rational numbers (like 2.5 or 3.5) were permitted in the preference evaluation. table 3b presents the priority estimation results for this example, and is organized as table 2b, with the additional column of the actual shares of the areas measured for these figures. ijahp article: lipovetsky/priority vector estimation: consistency, compatibility, precision international journal of the analytic hierarchy process 588 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.801 table 3b example 3: geometric figures’ area problem. priority vector estimations em ls ln re actual 1. circle 0.488 0.464 0.487 0.496 0.470 2. triangle 0.049 0.050 0.049 0.049 0.050 3. square 0.233 0.248 0.233 0.225 0.240 4. diamond 0.148 0.159 0.148 0.147 0.140 5. rectangle 0.082 0.078 0.082 0.083 0.090 correlations em 1 0.998 0.999 0.999 0.999 ls 0.998 1 0.998 0.995 0.998 ln 0.999 0.998 1 0.999 0.999 re 0.999 0.995 0.999 1 0.998 actual 0.999 0.998 0.999 0.998 1 s-compatibility em 1 1.003 1.000 1.000 1.003 ls 1.003 1 1.003 1.004 1.007 ln 1.000 1.003 1 1.000 1.003 re 1.000 1.004 1.000 1 1.003 actual 1.003 1.007 1.003 1.003 1 g-compatibility em 1 0.948 0.999 0.982 0.955 ls 0.948 1 0.948 0.931 0.953 ln 0.999 0.948 1 0.982 0.956 re 0.982 0.931 0.982 1 0.940 actual 0.955 0.953 0.956 0.940 1 precision ste 0.253 0.195 0.253 0.289 0.279 mae 0.145 0.110 0.145 0.162 0.165 r 2 0.987 0.992 0.987 0.983 0.985 all the vector estimates in this data practically coincide; the pair correlations are very high, and both the sand gindices prove compatibility among the estimates and with the actual observations. the precision measured by the ste, mae, and r 2 characteristics demonstrates a high quality of fit of the data by any of the estimated vectors of priority and by the actual values as well. 5. conclusions the paper considered several methods of priority vector evaluation in the ahp. they include the classical eigenproblem method, least squares, multiplicative or logarithmic approach, and a robust estimation based on transformation of the pairwise ratios to the shares of preferences. together with estimation of the vectors, validation of data ijahp article: lipovetsky/priority vector estimation: consistency, compatibility, precision international journal of the analytic hierarchy process 589 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.801 consistency and comparison of vectors by correlations, the sand gcompatibility indices were also completed. the numerical results for different data sizes and consistency indices demonstrate that all of the methods produce compatible results for the consistent data, otherwise a discrepancy between the different methods of the priority estimation would be observed. therefore, the data consistency should always be proved before the vector evaluation. another important conclusion concerns the precision assessment for the data matrix approximation by the obtained priority vectors. any regular statistical modeling requires a verification of the produced results by some quality characteristics. for example, in regression analysis, measures like the residual standard error ste, mean absolute error mae, and coefficient of multiple determination r 2 are commonly employed. applying them in the ahp environment can enrich the evaluation and interpretation of the results on priority modeling and is demonstrated on the numerical estimations performed in the paper. for instance, in the data for example 1 with a low consistency, the r 2 values are also not high which indicates a difficulty of approximation of inconsistent judgements by a priority vector in any estimation, and by mae values a mean deviation of the quotients of a priority vector’s elements from the observed pair judgements could be as big as one unit. in example 2 with a good consistency, the precision of the reproduction of the judgement matrix by the found priority vectors is high enough, although at the same time the actual distances occurred to yield the worst vector for approximation of the elicited pairwise priority matrix therefore, in this data we should not use the actual data on distances to check the compatibility with the obtained estimates of the vectors. example 3 with a perfect consistency yields all vectors of high compatibility and of a great quality of the elicited judgements reconstruction by each vector’s quotients of preference. the considered methods of priority vector estimation and characteristics of their quality are convenient and helpful in practical applications of the ahp for solving various multicriteria decision making problems. ijahp article: lipovetsky/priority vector estimation: consistency, compatibility, precision international journal of the analytic hierarchy process 590 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.801 references garuti a. c. (2007). measuring compatibility (closeness) in weighted environments., proceedings of the international symposium on the ahp, vina del mar, chile, august 2– 6. garuti a. c. and salomon v.a.p. (2011). compatibility indices between priority vectors. international journal of the analytic hierarchy process, 4(2), 152-160. doi: https://doi.org/10.13033/ijahp.v4i2.130 lipovetsky s. (1996). the synthetic hierarchy method: an optimizing approach to obtaining priorities in the ahp. european journal of operational research, 93, 550-564. doi: https://doi.org/10.1016/0377-2217(95)00085-2 lipovetsky s. (2009) global priority estimation in multiperson decision making. journal of optimization theory and applications, 140, 77-91. doi: https://doi.org/10.1007/s10957-0089447-6 lipovetsky s. (2013). supermatrix’ eigenproblem and interpretation of priority vectors in analytic network process. international journal of the analytic hierarchy process, 5(1), 105-113. doi: https://doi.org/10.13033/ijahp.v5i1.132 lipovetsky s. and conklin w.m. (2002). robust estimation of priorities in the ahp. european journal of operational research, 137, 110-122. doi: https://doi.org/10.1016/s0377-2217(01)00071-6 lipovetsky s. and conklin m. (2015). ahp priorities and markov-chapmankolmogorov steady-states probabilities. international journal of the analytic hierarchy process, 7(2), 2015, 349-363. doi: https://doi.org/10.13033/ijahp.v7i2.243 lipovetsky s. and tishler a. (1999). interval estimation of priorities in the ahp. european journal of operational research, 114, 153-164. doi: https://doi.org/10.1016/s0377-2217(98)00012-5 lootsma f. (1993) scale sensitivity in the multiplicative ahp and smart. journal of multi-criteria decision analysis, 2, 87-110. doi: https://doi.org/10.1002/mcda.4020020205 lootsma f. (1999). multi-criteria decision analysis via ratio and difference judgement. dordrecht, boston, london: kluwer academic publishers. doi: https://doi.org/10.1007/b102374 saaty t.l. (1977). a scaling method for priorities in hierarchical structures. journal of mathematical psychology, 15, 234–281. doi: https://doi.org/10.1016/00222496(77)90033-5 saaty t.l. (1980). the analytic hierarchy process. new york: mcgraw-hill. ijahp article: lipovetsky/priority vector estimation: consistency, compatibility, precision international journal of the analytic hierarchy process 591 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.801 saaty t.l. (1994) fundamentals of decision making and priority theory with the analytic hierarchy process. pittsburgh, pa: rws publications. saaty t.l. (1996) decision making with dependence and feedback: the analytic network process. pittsburgh, pa: rws publications. saaty t. l. (2005) theory and applications of the analytic network process: decision making with benefits, opportunities, costs, and risks. pittsburgh, pa: rws publications. saaty t.l. and kearns k.p. (1985). analytical planning. new york: pergamon press, new york. saaty t.l. and peniwati k. (2007). group decision-making: drawing out and reconciling differences. pittsburgh, pa: rws publications. saaty t.l. and vargas l.g. (1984). comparison of eigenvalue, logarithmic least squares and least squares methods in estimating ratios. mathematical modelling, 5, 309-324. doi: https://doi.org/10.1016/0270-0255(84)90008-3 saaty t.l. and vargas l.g. (1994). decision making in economic, political, social and technological environment with the analytic hierarchy process. pittsburgh, pa: rws publications, pittsburgh, pa. whitaker r. (2007). validation examples of the analytic hierarchy process and analytic network process. mathematical and computer modelling, 46, 840–859. doi: https://doi.org/10.1016/j.mcm.2007.03.018 ijahp: mu/about this special issue international journal of the analytic hierarchy process 309 vol. 9 issue 3 2017 issn1936-6744 https://doi.org/10.13033/ijahp.v9i3.554 about this special issue december 2017 dear ijahp readers: we publish this issue still besieged by the sadness of the passing away of our founder and creator of the ahp methodology, dr. thomas saaty –simply “tom” for all of us who were lucky to work with him and to enjoy his collegiality and friendship. for this reason, this issue is dedicated to his memory and who better person to guest edit this special issue than rozann saaty, his close collaborator and lifelong companion. while tom created ahp, rozann was decisive in its diffusion worldwide through the creative decisions foundation and her research on ahp validation. rozann has worked long hours to edit this special issue in which several scholars, who worked closely with tom, have selected their most memorable ahp papers. they will re-visit these articles and add their own insights and comments. rozann, as guest editor, will provide the proper introduction to this special issue honoring tom’s memory. very sincerely, enrique mu, phd ijahp editor-in-chief ijahp article: wedley/ saaty 1977: the building blocks international journal of the analytic hierarchy process 311 vol. 9 issue 3 2017 issn1936-6744 https://doi.org/10.13033/ijahp.v9i3.532 saaty 1977: the building blocks william c. wedley beedie school of business simon fraser university 8888 university drive burnaby, b. c. canada, v5a 1s6 wedley@sfu.ca abstract saaty’s 1977 article is his first comprehensive publication of the ideas behind ahp. he reveals his creativity in a new method for ratio measurement that includes pairwise ratio matrices, derived ratio scales from those matrices, and checks on the consistency of data. his ingenuity in using ratio measures is revealed by the use of hierarchical structures to display priorities and then a rescaling of them in a manner that allows synthesis for a composite ratio result. face validity is provided by many supporting examples and mathematical validity is provided by the solution to many theorems. keywords: ahp origins; ratio measurement; creativity, ingenuity 1. introduction in the early 1970’s, thomas saaty was developing and testing new concepts for decisionmaking. he already had an exemplary reputation in queuing theory, operations research and mathematics, and was turning his attention to a comprehensive methodology that could solve many types of problems, including ones central to him such as peace and conflict resolution. what emerged was the analytic hierarchy process (ahp) that gained extreme popularity, widespread use, and strong scrutiny. saaty’s subsequent work centered around this framework but always in an evolving manner. he was a creative individual who sought improvements, new insights and novel applications for his methodology. what started as ahp for complex hierarchies grew into the analytic network process (anp) for even more complex structures that had interdependencies. in 1977, saaty published ‘a scaling method for priorities in hierarchical structures’ (saaty, 1977). i consider that article to be the genesis of ahp and the kernel for anp. in that article, saaty presented all the theory and validations for the work he had been doing in the previous years. the words “process” and “hierarchy” are used extensively throughout the article, yet nowhere is there reference to the analytic hierarchy process. that title came a few years later. noteworthy about the 1977 article is saaty’s choice for the first comprehensive release of his ideas, the journal of mathematical psychology. it is the prime outlet frequented by measurement theorists. this signifies that the foundations of ahp/anp are in measurement theory. yet saaty states in the article that his core ideas were improvised, ijahp article: wedley/ saaty 1977: the building blocks international journal of the analytic hierarchy process 312 vol. 9 issue 3 2017 issn1936-6744 https://doi.org/10.13033/ijahp.v9i3.532 grew completely out of applications, and then had to be integrated into the main stream of literature. this means that his methodology was not an advancement of a particular thread of research; rather, it was an integration of many concepts more akin to the software of microsoft or microcomputers of apple that were disruptive technologies of that era. ahp was disruptive as well. like a bill gates or steve jobs, saaty took preexisting concepts along with creativity, ingenuity and determination to develop a dominant product. the depth and breadth of saaty’s knowledge and his ability to synthesize concepts is displayed in the article. it contains all the building blocks of ahp and the foundations for the evolution to anp. there are sections on: 1. ratio scales from reciprocal matrices, supported by nine rigorous theorems proving facts such as: λmax.=n for a consistent matrix proof of existence of a ratio solution for aw=λmaxw uniqueness and existence of the eigenvector solution proof that the limit of a is the normalized eigenvector times a constant proof that an order type wi/wj involves all the information in a and its powers 2. justification for the scale via validation with real life examples (distance from philadelphia, inverse law of light, wealth of nations, weight estimation) 3. hierarchies that can be used to represent both the structural and functional relations of a system. examples are given for sudan planning and backward and forward planning hierarchies 4. formal properties of hierarchies for getting composite answers from local priorities. support comes from rigorous definitions, 3 additional theorems and five examples (school selection, psychotherapy, choosing a job, vacation selection and conflict resolution) 5. methods for decomposition, aggregation and clustering in a hierarchy, including the number of comparisons required to create vectors of the hierarchy 6. relations to other research and work it is not the purpose here to go into the details of the article. saaty elaborates on those concepts in subsequent publications and books. what is covered here is the historical context of saaty’s invention, advice from saaty that bears re-emphasis, and some explanations regarding the uniqueness of saaty’s contribution. regarding the contribution in a historical context, it should be noted that ratio measurement, pairwise comparison matrices, eigenvectors and hierarchies were well established in 1977. saaty was novel in how he used and combined the concepts. the following comments are divided into three sections: personal reaction to the article, creativity illustrated in the micro aspects and ingenuity in the macro aspects. ijahp article: wedley/ saaty 1977: the building blocks international journal of the analytic hierarchy process 313 vol. 9 issue 3 2017 issn1936-6744 https://doi.org/10.13033/ijahp.v9i3.532 2. personal reaction to the article to study his work, to meet him, and to communicate and discuss with thomas saaty is a very personal experience. those experiences rate “nine” on my ahp preference scale and explain why i am writing in the first person. i first came across saaty’s 1977 article in the spring of 1981 in melbourne, australia while visiting monash university during sabbatical leave. being aware of both the benefits and deficiencies of quantitative techniques, i was investigating how to add qualitative factors to quantitative models. a colleague from canada, ernie love, passed through melbourne and paid a visit. knowing my interests, he gave me a short article by charles whaley, entitled simply, ‘fuzzy decision making’ (whaley, 1979). that article was not about fuzzy set theory that was popular at the time, but it did have two components that were to completely guide my future avenue of research. the first was a very simple computer program written in basic, called fuzzy. the second was the reference to saaty (1977). the software program was not pure ahp, because it did not use pairwise comparison matrices to get priorities for alternatives. nonetheless, it did use saaty’s eigenvector routine to develop priorities for criteria. i programmed whaley’s software on monash’s mainframe and then invited many volunteers to test the procedure. i connected them to the mainframe via the communication mode of that time, an acoustic coupler attached to a telephone, facsimile machines and paper printouts. today, the computer is on the telephone, accessing the cloud, and instantaneously available. since a computer is essential for ahp calculations, widespread microcomputer advances and early ahp programs such as expert choice were essential for the spread of ahp. later, super decisions played the same function for the dissemination of anp. my 1981 tests with many people were successful. uncannily, the paired comparison procedure captured people’s preferences. everyone seemed to agree that the eigenvector routine returned priorities that represented their values. one humorous test by my 13-year old daughter both shocked and sold me on the technique. as with all of my volunteers, i explained to my daughter that she should use the technique for a multi-criteria problem – one that had a number of alternatives that were evaluated according to different criteria. she immediately accepted the assignment, saying, “i will use it to choose a boyfriend!” she dutifully entered the names of dean and steven who frequently came around for poolside swims. then she entered john, emelio, david and …. knowing that the number of comparisons can increase with a larger n, i politely suggested she limit the alternative list after she reached seven. next, she proceeded to enter criteria: “personality”, “looks”, and behold: “kissability!” shocked by the “kissability” criterion from my 13-year old, i probably should have terminated the session right there (users must be experienced and able to recall the phenomena). nonetheless, we continued. that was a good choice, because i now realize that “preference for kissability” is not the same as actual “kissability” – just as the preference for the weight of a suitcase (lighter is better) is not the same as the actual weight of the suitcase. in my daughter’s test, dean (the athlete) was ranked first, and steven (the brain) was close behind. seven years later, my daughter married steven. https://expertchoice.com/ https://superdecisions.com/ https://superdecisions.com/ ijahp article: wedley/ saaty 1977: the building blocks international journal of the analytic hierarchy process 314 vol. 9 issue 3 2017 issn1936-6744 https://doi.org/10.13033/ijahp.v9i3.532 stories like that and other experiences by users made me become one of the populist supporters of this new technique. of course, my curiosity led me to the saaty (1977) reference at the end of the whaley article. i was able to find and photocopy it at monash’s library. over the years, i have read the article many times, each time learning something new. so why did saaty’s ahp become such a phenomenal success? some explanation is provided by the creativity and ingenuity displayed in the article. herein, i use the word “creativity” to imply a novel invention and “ingenuity” to imply cleverness in combining concepts. 3. the micro aspects – creativity to understand the creativity of thomas saaty, it is important to consider the micro aspects of the article. this includes how he used the pairwise comparison matrix, how he derived a ratio scale from it, and how he used matrix information to measure consistency. 3.1 the pairwise ratio matrix saaty characterizes his pairwise comparison matrix as a dominance matrix as opposed to proximity, profile or conjoint matrices that were then in common use. thurstone (1928) and subsequent researchers used comparison matrices with ordinal or probabilistic statements about one stimulus being more important than another. saaty’s questioning procedure started out in a similar manner, but went one step farther. first, he began with an ordinal question that identified the dominant of two objects, and then a ratio question regarding how many times more dominant. this second question establishing the intensity of dominance means that each comparison within the matrix creates an estimated ratio scale between the two objects: [aij, ajj=1] t . with many of these comparisons in the matrix and with unity along the diagonal, each column vector is a different estimate of the ratio relations. those multiple vectors as estimates indicate that redundancy is built into the comparisons of the matrix. ratio estimates is what distinguishes saaty’s comparison matrix from other matrix types. to denote this difference, it should be called the pairwise ratio matrix (prm). there are many techniques for deriving a summary ratio scale from a prm. backed by mathematical justification, saaty selected and staunchly defended the principal right eigenvector as the best technique. 3.2 magnitude estimation vs eigenvector estimation by choosing ratios for psychometric measurement, saaty entered an arena that had a long history of controversy. in 1932, the british association for the advancement of science appointed a committee from two divisions (mathematical/physical sciences and psychology) to debate whether or not qualitative factors could be measured (stevens, 1946). eight years later, after numerous meetings, they were still at a stalemate. stevens, from the united states, was the main proponent for psychophysical measurement. by 1977, he had developed several methods involving ratios, the most popular of which was magnitude estimation (stevens, 1971). from the alternative objects, this technique required the evaluator to specify a standard stimulus (a modulus) with a specified magnitude value such as 10. the evaluator then considers other objects and assigns ijahp article: wedley/ saaty 1977: the building blocks international journal of the analytic hierarchy process 315 vol. 9 issue 3 2017 issn1936-6744 https://doi.org/10.13033/ijahp.v9i3.532 numerals to them that are relative in intensity to the standard stimulus. in ratio form, such numbers can be renormalized to any desired unit. as narens (1996) observed, any object can be the standard stimulus for magnitude estimation and that object can be given the number 1 as the subjective norm. other objects take their intensities from that unit. using this unity modulus approach, magnitude estimation could be used to establish each column of the prm. in general, however, magnitude estimation is used to establish a single ratio vector under assumed error-free conditions. saaty’s technique was different and went much farther. saaty knew and accepted the fact that comparison estimates are likely to have imprecision. he also recognized that the comparison ratio to the dominated object implies the reverse ratio of the dominated to the dominant. by adopting the reciprocal property for the prm (aji =1/aij,), redundancy in estimation is achieved with half the number of possible comparisons and no necessity of generating every column according to magnitude estimation procedures. since each cell in saaty’s prm is an estimated conversion factor between two objects, transitive impacts can be calculated across multiple rows and columns (e.g. aij*ajk=aik). such indirect estimation of cell values is instrumental in both the eigenvector calculation and the summary statistic of consistency. it is from the redundant comparisons that saaty’s procedure provides more information – not only an elegant ratio average, but also a check on the consistency of the underlying data. magnitude estimation and other techniques did not do that. 3.3 accuracy vs. consistency in developing his theory, saaty started with the assumption that a true ratio scale actually exists for the phenomena being measured. we do not know that true scale – our problem is to derive it for n objects based upon the estimates in the prm. as saaty points out, if all estimates are perfectly accurate, the prm will be perfectly consistent and the true scale can be derived from any row, column or spanning tree of the matrix. prms, however, are estimates from humans. they are seldom perfectly consistent. this means that the derived priorities will not be a perfect replication of the underlying true ratio scale. in his article saaty (1977) was very cognizant of this distinction. for the first three pages of the section on ratio scales, he used aw=nw to refer to the true matrix of ratios and a’w’=λmaxw’ for the estimated matrix. thereafter, he dropped the primes and used aw=λmaxw. this may have been unfortunate, because some users think consistency reduction and λmax closer to n is the ultimate goal. saaty, however, was quite clear. he noted that, “improving consistency does not mean getting an answer closer to the “real” life solution. it only means that the ratio estimates in the matrix, as a sample collection, are closer to being logically related than to being randomly chosen ….” (saaty, 1977) probably guided by this statement, saaty later proposed the consistency ratio (cr) as the ratio of a prm’s consistency index (ci) to the average consistency index of randomly generated prms (i.e., cr=ci/rci). he suggested a rule of thumb that being 90% away from random (i.e. cr<.10) was acceptable and cr<.20 as tolerable. ijahp article: wedley/ saaty 1977: the building blocks international journal of the analytic hierarchy process 316 vol. 9 issue 3 2017 issn1936-6744 https://doi.org/10.13033/ijahp.v9i3.532 interestingly, he never pursued a statistical test for the consistency index that he mentions in the paper. saaty invited and looked upon a certain amount of inconsistency as being desirable. he felt that it was the inconsistent part of worldly phenomena that sparked and motivated new discoveries and human progress. if there was perfect consistency and homeostasis there would be no dynamic to promote change. in my personal contact with saaty, i found him to be about 80-90% consistent, 10-20% inconsistent, and usually right. our discussions were about the 10-20% part. 3.4 variability of intangibles based upon pairwise comparison estimates, saaty’s procedure produces a derived scale from the data. such a procedure could be used to create a stable scale for some tangible phenomenon that never changes. saaty’s procedure, however, had much more flexibility. as noted, saaty’s scale is a derived scale. it determines the measures rather than using some predefined scale to measure. in the case of intangible and qualitative factors, preferences can change. for example, the preference for “kissability” can change with the first kiss. saaty’s procedure has the flexibility to capture such changes. substitute updated comparisons, calculate a new derived scale, and then ask the decision maker whether those priorities represent the new situation. most prior techniques failed to recognize the transient nature of preferences. 4. the macro aspect – ingenuity saaty’s prm procedure can be used to derive ratio scales for many different types of objects: alternatives, criteria, strategies, scenarios, etc. what he needed was some way to integrate the derived scales from those different aspects into a unifying structure. from there, he could aggregate to get a composite answer. 4.1 hierarchies to achieve structure for problems, he chose the concept of hierarchies. in the 1977 article, saaty comments favorably on how herbert simon enriched the concept of a hierarchy (simon, 1962; simon and ando, 1961). referring to many different types of systems in nature, economics and society, simon had developed an important theory of how complex systems could be structured and analyzed as a hierarchy. along with that, simon provided the mathematical analog for aggregating variables within the hierarchy. he observed that subsystem variables at a lower level might be an aggregate of variables at a still lower level. saaty used this hierarchical framework to organize and display the elements of a decision. hierarchies not only provide a cognitive map of relationships but they also act as placeholders for the values of each element. saaty realized that the eigenvector values from his prm procedure could become the values for each cluster. from the outset, he adopted the convention of calling them priorities and normalizing them to sum to unity. since each cluster summed to unity, the implication is that all cluster priorities represent within-cluster effects. saaty needed some method to determine hierarchical effects. ijahp article: wedley/ saaty 1977: the building blocks international journal of the analytic hierarchy process 317 vol. 9 issue 3 2017 issn1936-6744 https://doi.org/10.13033/ijahp.v9i3.532 4.2 rescaling in ratios, saaty had the answer. the only permissible transformation of a ratio scale without destroying ratio relationships is multiplication by a positive constant. since such multiplication maintains ratios but not values, the new values signify that the ratio measurement is now in a different unit of measure. starting at the top of the hierarchy with a value of unity for the whole hierarchy, saaty used the scaling concept to make lower levels aggregate to the value in a higher level. by multiplying lower level priorities by the value of the higher connecting node, each cluster which formerly summed to unity is rescaled to a new sum which is the value of its parent node. thus, saaty used simon’s concept that the value at one level can be the aggregate of the values immediately below. note that saaty’s rescaling changes priority values within the hierarchy, but not the underlying relative ratios. ultimately, each element of the hierarchy receives its portion of the unit value at the top of the hierarchy. put differently, rescaling puts all values into a single unit representing elements and influences of the entire hierarchy. rescaled to a common hierarchical unit, it is possible to sum values of the same object that are in different parts of the hierarchy. in this manner, saaty aggregates to get a comprehensive solution. unlike other methods, that composite answer possesses ratio properties. it should be noted that if an alternative is removed from lower levels and remaining alternatives keep their same values, then hierarchical composition produces the same ratio results. this is because there has been no further rescaling (change of unit) in the hierarchy. however, if the remaining alternatives are renormalized once again to unity, then: (1) the unit of measurement for those remaining cluster alternatives has changed (values are different) (2) hierarchic rescaling still maintains the original ratio relationships between those remaining alternatives (values within a cluster change but ratios do not). (3) hierarchical synthesis to get a composite result can lead to changes in the ratios of composite results (different values are composed) (4) if the change in composite ratios is sufficient enough, the remaining alternatives can display a reversal in rank in the early days of ahp, too much time and effort were expended on the rank reversal debate. it seems incomprehensible that such a debate would be carried out on ordinal terms when ahp is a ratio method. resorting to a lower order of measurement also lowered the possibilities for analysis (stevens, 1946). i believe that the debate would have been resolved had it been conducted on a ratio basis. perhaps the debate was deflected by how easy it is to convert ratios to ordinals or intervals. many do! that, however, is not the intent of ahp. to get its full worth, one should use its measurement in ratios. 4.3 networks saaty makes reference to concepts that would eventually become anp. he notes that, “any system can be represented by a large interaction matrix whose rows and columns are components of the system.” he comments that, “… when component i and component j interact strongly, the i, jth entry is near ±1” and “… when they do not interact, the entry is near zero.” he also commented that, “… through the concept of a reachability matrix ijahp article: wedley/ saaty 1977: the building blocks international journal of the analytic hierarchy process 318 vol. 9 issue 3 2017 issn1936-6744 https://doi.org/10.13033/ijahp.v9i3.532 and its powers, a distinct hierarchic structure is often discerned.” with entries positive or zero, those concepts are the attributes of an anp matrix and the anp process (saaty, 1977). 5. summary a web of science search reveals that saaty’s 1977 article has been cited over 2400 times. that is remarkable for a technical publication. ahp and anp have been very successful. but why? was it ratio paired comparisons, derived scales, consistency measurement, hierarchies, rescaling, composition or structure? of course, ahp is an amalgam of all of those. i claim that the introduction of ratio comparisons to pairwise comparison matrices, the derivation of a ratio vector from those matrices, and the provision of a consistency check on those matrices was the creative and novel aspect of thomas saaty’s invention. the ability to foresee that those derived ratios could be analyzed in hierarchical structures illustrates his ingenuity. he was both a master of minutiae (the micro) and innovator of integration (the macro). what he wrote in the 1977 article provided the building blocks for ahp and anp. ijahp article: wedley/ saaty 1977: the building blocks international journal of the analytic hierarchy process 319 vol. 9 issue 3 2017 issn1936-6744 https://doi.org/10.13033/ijahp.v9i3.532 references narens, l. (1966). a theory of ratio magnitude estimation, journal of mathematical psychology, 40, 109-129. doi: https://doi.org/10.1006/jmps.1996.0011 saaty, t. l. (1977). a scaling method for priorities in hierarchical structures, journal of mathematical psychology, 15, 3, 1977, 234-281. doi: https://doi.org/10.1016/00222496(84)90003-8 simon, h. a. (1962). the architecture of complexity, proceedings of the american philosophical society, 106, 6 467-482. simon, h. a and ando, a. (1961). aggregation of variables in dynamic systems, econometrica, 29, 2, 111-138. doi: 10.2307/1909285 stevens, s. s. (1946). on the theory of scales of measurement, science, 103, 677-680. doi: 10.1126/science.103.2684.677 stevens, s. s. (1951). mathematics, measurement and psychophysics, s. s. stevens (ed.) handbook of experimental psychology, john wiley & sons, inc., 1-49. stevens, s. s. (1971). issues in psychophysical measurement, psychological review, 78(5), 426-450. thurstone, l. l. (1928). attitudes can be measured, the american journal of sociology, xxxiii, 529-554. doi: https://doi.org/10.1086/214483 whaley, c. p. (1979). fuzzy decision making, interface age, 4, 87–91. https://doi.org/10.1006/jmps.1996.0011 https://doi.org/10.1016/0022-2496(84)90003-8 https://doi.org/10.1016/0022-2496(84)90003-8 https://doi.org/10.1086/214483 ijahp article: kashi/ahp in personnel management: can the key competencies change with company’s strategy? international journal of the analytic hierarchy process 39 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.297 ahp in personnel management: can the key competencies change with company’s strategy? katerina kashi vsb technical university of ostrava department of management ph.d. student katerina.kashi@vsb.cz abstract many companies today use competency models in human resources management. welldesigned competency models can be used by the company in many ways. they can aid human resource management in the following ways: recruitment, performance management, training and development, recognition and awards, compensation and succession planning. the main aim of this article is to determine key competencies for a top manager in a middle size automotive company. firstly, the paper begins with a description of competency models and their development and utilization. next, the competencies are decomposed so that ahp can be used for their evaluation, and weights are computed according to ahp methodology for individual competencies. then, individual competencies are ranked in order of importance. further, the author compared the current key competencies rankings with results from last year to see if there have been any changes in the rankings as a result of changes in the company’s strategy. keywords: ahp; analytic hierarchy process; competency models; key competencies 1. introduction in today’s competitive environment companies must think about their business strategy, especially when it comes to the kind of competencies a business needs to have in order to compete in a specific environment. competency models, which should be designed for all key positions in the company, show what competencies are necessary for an individual position. the question is what the key competencies in each position are and can they change according to a company’s strategy? the objective of this article is to present a competency model where a group of competencies and individual competencies are ranked according to importance in order to determine key competencies for a top manager position. the evaluation by ahp will be executed in a middle sized automotive company and performed by five experts in the company who will decide in consensus on the competencies preferences. the results will be compared with last year’s results in order to see if the company’s strategy or manufacturing scope can change the ranking of key competencies. the hypothesis is that the key competencies will change slightly with the company’s strategy/manufacturing scope. ijahp article: kashi/ahp in personnel management: can the key competencies change with company’s strategy? international journal of the analytic hierarchy process 40 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.297 2. competency models a competency model is a framework which lists the competencies required for effective performance in a specific job or group of jobs. a competency is a human capability which is required for effective performance. it can be comprised of personal characteristics, knowledge, skills and abilities. personal characteristics can include: work habits, cooperation with others, manners, mental agility, proactivity etc. knowledge is acquired through learning and experience and can be described as awareness, information or understanding about rules, principles, theories, concepts etc. skills represent the capacity to actually perform mental or physical tasks with a specific outcome such as managing a six sigma project or leading the lean team. ability is often a composition of several capacities which enable us to learn and perform. these are usually very difficult to develop since they have a strong component of innate capacity, e.g. the ability to think analytically is more natural for some individuals than others (marelli, tondora & hoge, 2005). competencies have to lead to effective performance, which means that the performance of a person with a competency must be significantly better than that of a person without it. competencies are components of a job which are reflected in behavior that is observable in a workplace (sanghi, 2007). a well-designed competency model can be used by the company’s human resources in various ways. it can serve as a base for:  recruitment – measures are developed for evaluating resumes, interview guides, written or performance tests are prepared based on specific (required) competencies,  performance management – guides for managers are developed to help them conduct discussions with their employees about their performance, creation of ratings to help managers in the assessment of each competency, develop such performance appraisal process and forms which incorporate the competencies,  succession planning – develop instruments to assess the employees’ competencies who have the potential for advancement, design tools which will help the managers to assess the critical competency gaps in the pool of succession candidates,  recognition and rewards – design recognition and rewards programs which are based on employee’s demonstration of highly valued competencies,  compensation – design compensation program where the salary (pay) of employees is increased based on the evidence of their proficiency in selected competencies,  training and development – utilize the competencies to design needed training and other learning activities, create feedback instruments to evaluate employee needs for specific competency development, develop planning guides which provide employees with specific suggestions on how to acquire or strengthen each competency, (marelli et al., 2005). the analytic hierarchy process (ahp) can be used to be able to determine the key competencies. the ahp method is a technique which enables one to organize and analyze complex decisions. it helps the decision makers find a solution that best suits the goal and their understanding of the problem. the method provides a rational framework to structure a decision problem, to represent and quantify its elements which relate to those elements of the overall goal, and to evaluate alternative solutions. this method is suitable for competency modeling because of the possibility to decompose the goal/objective to several criteria and sub criteria (saaty,1994; saaty & peniwatti, 2007). hsiao et. al. (2011) used the analytic hierarchy process to analyze selection criteria for ijahp article: kashi/ahp in personnel management: can the key competencies change with company’s strategy? international journal of the analytic hierarchy process 41 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.297 recruitment of five different roles in the area of information systems. whereas zolfani et. al. (2012) perceived selection of new employees or groups of employees as a fundamental problem in the human resources area. in this research, the ahp method was used to identify which competencies are the most important, i.e. what are the core competencies for the top manager position of a middle size manufacturing company producing automotive parts. 3. ahp method analytic hierarchy process is a framework of logic and problem solving that spans the spectrum from instant awareness to fully integrated consciousness by organizing perception, judgments and feelings into a hierarchy of forces which influence decision results. the method is based on innate human ability to utilize information and experience to estimate relative magnitudes through paired comparison. the hierarchy represents a complex problem in a multilevel structure with the first level being the goal followed by levels of factors, criteria and sub-criteria. it can decompose a complex problem in search of cause-effect explanations into steps which form a linear chain. users of ahp firstly decompose their decision problem into a hierarchy of more easily understood sub-problems, each of which can be analyzed independently. the elements of the hierarchy can be related to any aspect of the decision problem, can be tangible or intangible, carefully measured or just roughly estimated. once the hierarchy is drawn, the decision makers systematically evaluate its elements by comparing them one to another two at a time, with respect to their impact on the element above in the hierarchy. the ahp then converts these evaluations into numerical values which can be processed and compared over the entire range of the initial problem. a numerical weight or priority, which is derived for each element of the hierarchy, allows often incommensurable elements to be compared to one another in a rational and consistent way. an illustration of a three level decomposition is shown in figure 1(saaty, 1994). figure 1. a three level hierarchy based on zmeskal (2003; 2012), the ahp methodology has three main phases: structuring the hierarchy, executing paired comparison between the elements and decision alternatives and synthesizing the results. these priorities are unique within a positive multiplicative constant and therefore belong to an underlying ratio scale. they become unique through normalization. the resulting derived vector reflects the proportion of the decomposition of unity which each element receives. the results of paired comparisons for n attributes is organized into reciprocal n x n matrix  ijss : ijahp article: kashi/ahp in personnel management: can the key competencies change with company’s strategy? international journal of the analytic hierarchy process 42 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.297 12 1 12 2 1 2 1 ... 1 / 1 ... ... ... ... ... 1 / 1 / ... 1 n n n n s s s s s s             s (1) this matrix s contains the qualified information from an expert user about the relations between all the pairs of attributes. the element sij can be described as the importance ratio of i-th and j-th attributes. so there are two basic properties for the elements of saaty’s matrix s: 1 , 1,..., 1 / ii ij ji s i j n s s    (2) the importance of i-th attribute can be measured using the weight vi. the sum of weights of all n attributes must be equal 1. 1 1 1,..., n i i v i n    (3) in an ideal situation the saaty’s matrix s is consistent and each element sij can be defined as a ratio of the corresponding weights vi and vj: , 1,...,i ij j v s i j n v   (4) using the laws of linear algebra it can be shown that the vector of weights is the eigenvector of saaty’s matrix s and belongs to its largest eigenvalue λmax. to quantify the relation between i-th and j-th attributes the saaty’s method uses the integer scale 1 to 9 for the elements of the matrix s, where: sij = 1 is used for indifference, 3 – weak preference, 5 – strong preference, 7 – very strong preference, and 9 – absolute preference. an even number of points (2, 4, 6, and 8) can be used for more precise differentiation. the impact of positive preference is expressed in the interval sij ∊ (1; 9, the scale of inverse preference belongs to an interval sij ∊ 1/9; 1), see saaty (1994). in real situations the saaty’s matrix s is not fully consistent. if inconsistency is not too large, the vector of weights v can be approximated by the vector w using nonlinear logarithmic – quadratic optimization to minimize the expression: 2 1 1 log log , 1,..., n n i ij i j j w s i j n w            (5) ijahp article: kashi/ahp in personnel management: can the key competencies change with company’s strategy? international journal of the analytic hierarchy process 43 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.297 it can be shown that vector w minimizing the expression (5) can be calculated using the weighted geometric mean of the elements of saaty’s matrix s rows: 1 1 1 n n ij j i nn n ij i j s w s        (6) the inconsistency of saaty’s matrix s can be evaluated by the consistency index ci defined as: max 1 n ci n     (7) the more accurate indicator of inconsistency is consistency ratio cr defined by saaty (1994) as: ci cr ri  (8) where ri is the random index (aguarón et al., 2003; alonso & lamata, 2006; franek & kresta, 2014) which values are derived from empiric research to make the values of consistency ratio cr independent of saaty’s matrix s rank (for n=1,2 the ri is equal to 0), see table 1. table 1 random index (ri) n 3 4 5 6 7 8 9 10 ri (saaty, 1980) 0.58 0.9 1.12 1.24 1.32 1.41 1.45 1.49 ri (aguarón et al., 2003) 0.525 0.882 1.115 1.252 1.341 1.404 1.452 1.484 ri (franek & kresta, 2014) 0.525 0.882 1.110 1.250 1.341 1.404 1.451 1.486 the inconsistency of saaty’s matrix measured by consistency ratio cr is considered as cr ≤ 0,1 to be able to interpret the elements of w vector as the good estimation (approximation) of attributes weights. the ri used by e.g. expert choice software uses simple calculation of the riexp (daniel & oyatoye, 2011): exp 1.99( 2)n ri n   (9) a measure of consistency for rgmm method of weights approximation was proposed by aguarón & moreno-jimenéz (2003) calling it the gci. from a practical point of view, the interpretation of the gci is analogous to that proposed by saaty for the consistency ijahp article: kashi/ahp in personnel management: can the key competencies change with company’s strategy? international journal of the analytic hierarchy process 44 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.297 ratio used with the eigenvector method in conventional ahp. the gci can be calculated as follows:               ij j i ij w w s nn gci  2 loglog )2)(1( 2 , (10) where n is the number of criteria, wi/wj is the ratio of approximated weights. the approximated thresholds were estimated by aguarón & moreno-jimenéz (2003) where: gcimax for n=1,2 is 0.31; for n=3 is 0.3147; for n=4 is 0.3526 and for n>4 is 0.37. using the geometric mean formula (6) the local weights w are calculated for each matrix s in ahp structure (each group of criteria / sub criteria). the global weights represent a relative distribution of weights across the whole group of sub criteria. to calculate global weights the local weights for i-th particular sub criteria group wi has to be multiplied by relevant j-th criteria local weights wj as follows: ij i j w w w  (11) where wij are the particular global weights. the sum of global weights in ahp structure has to be equal to 1: , 1 for all possible combinations of , ij i j w i j (12) 4. utilizing ahp within competency models evidently there is not a single unified competency model that would work for everybody, since every company requires different competencies from its employees. the preferences in competencies could differ based on the company’s business, size, people, culture etc. pulakos (2009) has used a general competency model which does not divide individual competencies into any groups, i.e. strategic thinking, planning work, technical proficiency, critical thinking, learning and development, communicating with others, representing the organization and collaboration with others. however, we believe that a model with “general” competencies can be established. based on the results of the author’s previous research executed in 2011 where employees of one manufacturing company located in the czech republic were asked about their satisfaction with the performance appraisal system and the choice of evaluated competencies, the following competencies were mentioned the most often: work with information, problem solving, leadership, change management, effective communication, active listening, negotiating, team cooperation, motivating others, relevant professional knowledge, business knowledge, strategic thinking, analytical thinking, proactivity, creativity, mental agility and emotional resilience (kashi, 2012). table 2 shows the division of each group of competencies and their description. ijahp article: kashi/ahp in personnel management: can the key competencies change with company’s strategy? international journal of the analytic hierarchy process 45 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.297 table 2 competencies explanation groups competencies explanation man work with information manager’s ability to work well with new information, its organization and also the ability to convey the message to his/her subordinates problem solving the ability to weight pros and cons in uncertain or ambiguous situations calling for a high level of judgment or perhaps intuition leadership the skill or rather gift to lead people delegating the skill to hand over any work which can be done by others. this can be used as highly motivating tool for empowering employees change management the manager’s ability to deal with change, implement and execute any changes in company, i.e. within company’s restructuring or product change int effective communication good communication skills negotiating how well can the manager negotiate i.e. with his/her suppliers, employees, co-workers etc. active listening the ability to listen to his/her subordinates’ needs, ideas without immediate refusal team cooperation how well does the manager cooperate with other team members, is he/she supportive and helpful motivating others very important skill of a top manager, the ability to motivate his/her subordinates for their best possible performance tech relevant professional knowledge top manager’s specific knowledge i.e. hr manager’s knowledge of the legislation and labor code, quality manager – the knowledge of iso norms etc. business knowledge thorough understanding of general business functions and specific areas of knowledge strategic thinking ability to come up with alternative viable strategies or business models that deliver customer value. it is a management competency required to carry-out strategy. analytical thinking the ability to use logical and optimization techniques to make decisions per proactivity manager’s ability to respond quickly but within the company long term goals. this competence also includes abilities such as dedication, commitment and responsibility creativity ability to come up with original solutions or ideas mental agility ability to grasp problems quickly, to understand whole situation in a timely manner emotional resilience manager’s ability to deal with stressful situations, i.e. working under stress, meeting deadlines etc. source: sanghi (2007), modified by authors for our model we began with the competencies from table 2 from which we selected the ones that were chosen most often by the employees as the most important competencies. ijahp article: kashi/ahp in personnel management: can the key competencies change with company’s strategy? international journal of the analytic hierarchy process 46 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.297 then, based on consultation with experts (company’s hr managers) these competencies were divided into the four following groups (criteria): managerial competencies, interpersonal competencies, technical competencies and personal qualities. each group was then divided into several (4 or 5) sub-criteria, which belong into the particular group. we used this division because the individual competencies, in our opinion, belong into these groups. proposal of the decomposition of competencies for the utilization of ahp is shown in figure 2. figure 2. the decomposition of competencies for the utilization of ahp to be able to identify key competencies, one position (the top manager) was chosen for this illustration. the evaluation was executed via interview and the company’s experts i.e. company’s hr manager, financial manager, production manager, quality manager and ceo decided the preferences for each pair. the experts assessed the criteria based on the following information. the assessors (the five experts) assigned the following values if the row was preferred before a column: a – the elements are indifferent, b – there is a slight preference, c – there is a strong preference, d – there is a very strong preference, e – there is an absolute preference. if the column was preferred before a row the reciprocal values were assigned, e.g. 1/a, 1/b, 1/c, 1/d, 1/e. the letter evaluation was intentionally proposed by the authors to eliminate the problem of assigning an average number to all comparisons. an example is shown for the first level of the decomposition in the following table 3. ijahp article: kashi/ahp in personnel management: can the key competencies change with company’s strategy? international journal of the analytic hierarchy process 47 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.297 table 3 pair-wise comparison table example for competencies groups for year 2013 objective managerial interpersonal technical personal managerial a b b c interpersonal 1/b a 1/c b technical a c personal a 5. results the evaluation of the criteria and sub-criteria was done by five experts in the manufacturing company in order to find out if the competencies had changed with the change of a strategy/manufacturing program. the company’s strategy is very general and concentrated on developing customized products that serve their diverse customers and add shareholder value. the matrices were completed based on the expert’s decisions. the experts had to decide in consensus on all pair-wise comparisons. the results for individual matrices for each competency are described and evaluated below. firstly, experts evaluated groups of competencies based on pair-wise comparison. then, the weights for individual groups of competencies were calculated based on the formulas (see equation 6). tables 4 and 5 show the pairwise comparisons that were made for the competency groups in 2013 and 2014. these illustrate the changes in thinking of the decision makers in the company, and show how they have changed their preferences resulting in changes in the weights/priorities (see table 6 for comparison). table 4 pair-wise comparisons, weights and consistency for groups of competencies for 2013 competencies managerial interpersonal technical personal weights (wi) managerial 1 3 1/3 5 0,263 interpersonal 1/3 1 1/5 3 0,118 technical 3 5 1 7 0,564 personal 1/5 1/3 1/7 1 0,055 λmax 4,135 ci 0,135 ri 0,882 cr 0,153 n 4 cr (riexp) 0,135 riexp 0,995 gci 0,154 ijahp article: kashi/ahp in personnel management: can the key competencies change with company’s strategy? international journal of the analytic hierarchy process 48 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.297 table 5 pair-wise comparisons, weights and consistency for groups of competencies for 2014 competencies managerial interpersonal technical personal weights (wi) managerial 1 5 2 7 0,498 interpersonal 1/5 1 1/5 3 0,101 technical 1/2 5 1 7 0,352 personal 1/7 1/3 1/7 1 0,049 λmax 4,142 ci 0,142 ri 0,882 cr 0,161 n 4 cr (riexp) 0,142 riexp 0,995 gci 0,177 next the consistency ratio was calculated (see equation 8) using ri estimated by franek & kresta (2014), and riexp calculated from equation 9. the gci measure was calculated using equation 10. gci and the consistency ratio were within limits in all comparison matrices. the resulting priorities of competency groups for the company for 2013 and 2014 are shown in table 6. the ranking of individual competencies for both years are shown in table 7. table 6 global weights for groups of competencies for 2013 and 2014 competencies groups 2013 2014 managerial 26,34% 49,75% interpersonal 11,78% 10,12% technical 56,38% 35,18% personal 5,50% 4,94% from the aforementioned table, it is evident that the ranking for the groups of competencies has changed. in 2013 the company’s experts preferred technical competencies the most, whereas in 2014 they preferred managerial competencies. in 2013 managerial competencies were the second most preferred while in 2014 the technical ones were the second most preferred. also in 2013, personal competencies were more important than interpersonal ones; however in 2014 these ranked in the opposite order. ijahp article: kashi/ahp in personnel management: can the key competencies change with company’s strategy? international journal of the analytic hierarchy process 49 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.297 table 7 ranking of competencies determined for top manager position global weights of competencies for top manager ranking 2013 2014 1 strategic thinking 25,11% strategic thinking 15,67% 2 business knowledge 17,54% leadership  15,17% 3 professional knowledge 8,91% change management  14,18% 4 change management 8,12% business knowledge  10,94% 5 leadership 6,09% problem solving  9,14% 6 delegating 6,09% delegating 9,14% 7 problem solving 4,89% professional knowledge  5,56% 8 analytic thinking 4,82% negotiating  3,43% 9 negotiating 3,99% analytic thinking  3,01% 10 effective communication 3,21% effective communication 2,75% 11 proactivity 2,23% work with information  2,13% 12 active listening 2,07% proactivity  2,01% 13 mental agility 1,97% active listening  1,78% 14 motivating others 1,66% mental agility  1,77% 15 work with info 1,14% motivating others  1,43% 16 stress resilience 0,86% stress resilience 0,77% 17 team cooperation 0,86% team cooperation 0,74% 18 creativity 0,44% creativity 0,39% ijahp article: kashi/ahp in personnel management: can the key competencies change with company’s strategy? international journal of the analytic hierarchy process 50 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.297 from table 7 it is evident that the core competencies (top ten) for a top manager have changed significantly. in 2013 the top ten competencies included strategic thinking with 25.11%, followed by business knowledge with 17.54%, professional knowledge with 8.91%, change management with 8.12%, followed by leadership and delegating with 6.09%, problem solving with 4.89%, analytic thinking with 4.82%, negotiating with 3.99% and effective communication with 3.21%. in 2014, the top ten competencies included strategic thinking with 15.67%, followed by leadership with 15.17%, change management with 14.18%, business knowledge with 10.94%, followed by problem solving and delegating with 9.14%, professional knowledge with 5.56%, negotiating with 3.43%, analytic thinking with 3.01% and effective communication with 2.75%. all changes in the ranking are shown by arrows. these major differences can be attributed to a change of the top manager’s position in the company. in 2013, before all the changes, the company was working in a routine and basically under the parent company’s management. this meant that the top managers concentrated mostly on strategic thinking and business and professional knowledge were the most important for them. however, then the sbu of the company in scotland has gone through a major change and all the manufacturing lines were moved to the sbu in the czech republic. when several new manufacturing lines came in and about 50 new operators were hired, the core (and mostly preferred and required) competencies changed. although strategic thinking remains in first place, it has much less weight; leadership has moved from the fifth place to the second place with a much higher weight. the same has happened with the change management which is now in the third place with a weight of 14.18%, followed by business knowledge where the weight has decreased, and problem solving where the ranking has changed from seventh place to fifth place. now, the top managers must not only fulfill the parent company’s strategy and plan, but they must also lead their subordinates to achieve the new goals and objectives. 6. conclusions the key competencies for a top manager in an automotive company have changed in the period of one year. although the company’s vision and mission has not changed at all, the manufacturing program has changed significantly. the company’s key top manager’s competencies at the beginning of 2013 were different than at the beginning of 2014. the reason is that the company’s strategy has changed. in 2013 the company was manufacturing the usual products, however at the end of 2013 many new lines were moved to the plant from another one. therefore, the managerial competencies in 2014 are different because the managers now have to concentrate much more on leadership and change management instead of on just operational management. the conclusion of this research is that the core competencies should be adjusted if the company’s manufacturing program or the strategy significantly changes. although this study was limited to a particular company, the authors believe that it can bring some innovation into any company’s management. however, the author’s intent is not to make broad conclusions for the whole domain of competency models but to suggest a way n which hr analysts can apply the ahp approach to their work process. further research will be focused on different types of firms and positions. ijahp article: kashi/ahp in personnel management: can the key competencies change with company’s strategy? international journal of the analytic hierarchy process 51 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.297 references aguarón, j., et al. (2003). consistency stability intervals for a judgement in ahp decision support systems. european journal of operational research, 145(2), 382-394. doi:10.1016/s0377-2217(02)00544-1 aguarón, j., & moreno-jiménez, j. m. (2003). the geometric consistency index: approximated thresholds. european journal of operational research, 147(1), 137-146. doi:10.1016/s0377-2217(02)00255-2 alonso, j. a. & lamata, t. (2006). consistency in the analytic hierarchy process: a new approach. international journal of uncertainty, fuzziness and knowledge-based systems, 14(4), 445−459. doi: 10.1142/s0218488506004114 franek, j. & kresta, a. (2014). judgment scales and consistency measure in ahp. procedia economics and finance, 12(2014), 164–173. doi:10.1016/s22125671(14)00332-3 daniel, a. c. & oyatoye, e. o. (2011). strategies for strengthening the capacity of grassroots women in nigeria to overcome poverty. proceedings of the international symposium on the analytic hierarchy process 2011, 1-8. hsiao, wh et al. (2011). selection criteria of recruitment for information systems employees: using the analytic hierarchy process (ahp) method. african journal of business management, 5(15), 6201-6209. kashi, k. (2012). analýza systému vzdělávání a rozvoje zaměstnanců. [analysis of the employees’ training and development system]. diploma thesis. vsb – technical university of ostrava, faculty of economics. marelli, a. f., j. tondora & m. a. hoge. (2005). strategies for developing competency models. administration and policy in mental health and mental health services research, 32(5-6), 533-561. doi: 10.1007/s10488-005-3264-0 pulakos, e.d. (2009). performance management: a new approach for driving business results. singapore: wiley-blackwell. doi: 10.1111/j.1744-6570.2011.01242_4.x saaty, t.l., (1980). the analytic hierarchy process. new york: mcgraw-hill. doi: 10.1080/00137918308956077 saaty, t. l. (1994) fundamentals of decision making and priority theory with the analytic hierarchy process. pittsburgh: rws publications. saaty, t. l., k. peniwatti & j. s. shang (2007). the analytic hierarchy process and human resource allocation: half the story. mathematical and computer modelling, 46 (78), 1041-1053. http://dx.doi.org/10.1016/s0377-2217(02)00544-1 http://dx.doi.org/10.1016/s0377-2217(02)00255-2 http://dx.doi.org/10.1016/s2212-5671(14)00332-3 http://dx.doi.org/10.1016/s2212-5671(14)00332-3 ijahp article: kashi/ahp in personnel management: can the key competencies change with company’s strategy? international journal of the analytic hierarchy process 52 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.297 sanghi, s. (2007). the handbook of competency mapping. london: sage. doi: http://dx.doi.org/10.4135/9788132108481.n6 zmeskal, z. (2012). aplikace dekompozicnich vicekriterialnich metod ahp a anp ve financnim rozhodovani. (application of decomposition multicriteria methods ahp and anp in financial decision making.) in: 6th international scientific conference managing and modelling of financial risks. ostrava: všb technická univerzita ostrava, 689699. zmeskal, z. (2003). soft approach to company financial level multiple attribute evaluation. in: proceedings of the 21st international conference mathematical methods in economics. praha: ceska zemedelska univerzita v praze, 273-280. zolfani, s. h. & antucheviciene, j. (2012). team member selecting based on ahp and topsis grey. inzinerine ekonomika-engineering economics, 23(4), 425-434. doi: http://dx.doi.org/10.5755/j01.ee.23.4.2725 http://dx.doi.org/10.4135/9788132108481.n6 http://dx.doi.org/10.5755/j01.ee.23.4.2725 ijahp news and events: karpak/ mcdm 2017 international conference held for second time in canada international journal of the analytic hierarchy process 302 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.502 multiple criteria decision making (mcdm 2017) international conference held for a second time in canada birsen karpak distinguished professor of management associate editor, international journal of the analytic hierarchy process the 24th international conference on multiple criteria decision making (mcdm 2017) was held at the telfer school of management, university of ottawa, on july 10-14, 2017. this was the second time this conference was held in canada; the first time being in whistler in 2004, organized by bill wedley. we had a wonderful welcome by the school of management; and the scientific program as well as the social events were outstanding. there is a very good essay about the conference by sarah ben amor (general chair) at http://www.mcdmsociety.org/newsletters/92017-september-2017 my reviews will be limited to the sessions i attended which were mostly those related to ahp/anp theory and applications. there were 200 participants, including more than 60 students. the conference organizers were able to attract 12 participants from industry. most of the participants were from canada (36) followed by turkey (24), the united states (17), brazil (14) and poland (14). there were 4 plenaries that included:  blair feltmate of the intact centre on climate adaptation at the university of waterloo: un-natural alliances: financial and ecological expertise must align to address the contagion of climate change  jyrki wallenius from the aalto university school of business: a voting advice model and its application to parliamentary elections in finland  tuure tuunanen from the university of jyväskylä: design science research: theory ingrained artifact and deriving theories from artifact  jack kitts, president and chief executive office of the ottawa hospital: is it possible to create a sustainable healthcare system in canada? when i was listening to the presentation by jack kitts, i was thinking about how much opportunity exists for those of us who implement ahp/anp by itself or in combination of other mcdm methodologies. he impressed me, as a ceo, with his understanding of multiple criteria methodologies and his appreciation of a data driven scientific approach of decision making. i thought his presentation should be disseminated widely; we might be able to secure it and publish it as an essay for our audience in the next issue of ijahp. there were 2 invited sessions on ahp/anp theory and applications in supply chain management and industrial engineering organized by b. karpak, g. buyukozkan and s. https://urldefense.proofpoint.com/v2/url?u=http-3a__www.mcdmsociety.org_newsletters_92017-2dseptember-2d2017&d=dwmgaq&c=0w9vy5nnhl9u_frqx4vrzksnz08jjo3five6wvqrtvo&r=8siruohdnke9xopydtaqfa&m=au9irzyxy_ctovpcthlpbc0jffd2hyuwq6pz3i1j1zq&s=u30fvn9ctou5cu1edtoxsfvegqaexhht3b-ggoyo4vy&e= ijahp news and events: karpak/ mcdm 2017 international conference held for second time in canada international journal of the analytic hierarchy process 303 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.502 guleryuz. one session was chaired by seyhan nisel, and she also presented an integrated approach for developing a user satisfaction model for enterprise resource planning (b.karpak, s. nisel, r. nisel) (figure 1). figure 1. seyhan nisel, an integrated approach for developing a user satisfaction model for enterprise resource planning in the same session luis vargas presented, how to write a contract with the ahp (l. vargas) (figure 2), which can be read as an article in this current issue. gozde kadioglu, a master’s student from istanbul technical university, turkey, identified cafe nero as the most preferable coffee house followed by starbucks in her ahp based, a decision support model for the assessment of consumer preferences: a case study on coffee house companies (g. kadioglu, i.topcu). another interesting presentation in this session was prioritization of factors affecting dimensions of sustainable supply chain management (g. kucukyazici, s.b. amor, i. topcu) by gunes kucukyazici. these authors prioritized social, environmental, and economic dimensions of supply chain management. ijahp news and events: karpak/ mcdm 2017 international conference held for second time in canada international journal of the analytic hierarchy process 304 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.502 figure 2. luis vargas, how to write a contract with the ahp the second session was chaired by birsen karpak. karpak’s mba students crystal volinchak matthew yourstowsky, erin whitehouse and robert woolley, from youngstown state university utilized the analytic hierarchy process (ahp) decision-making model to select the optimal market for international expansion for abc corporation located in ohio (figure 3). they identified canada as the best export market for the company among china, mexico, canada, germany and the united kingdom (birsen karpak, faculty advisor). konrad kulakowski, agh university of science and technology, poland, presented inconsistency in the ordinal pairwise comparisons method with and without ties. sezin guleryuz of bartin university, turkey proposed a fuzzy framework to evaluate service quality in turkish public hospitals. she contended that the framework proposed by her integrating dematel, anp and topsis in the healthcare sector has not been offered in the literature. she shared with us the application of the methodology into a public hospital in bartin, turkey. the session ended with another dematel, anp integration to evaluate transportation modes presented by karpak (buyukozkan, karadag co-authors). ijahp news and events: karpak/ mcdm 2017 international conference held for second time in canada international journal of the analytic hierarchy process 305 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.502 figure 3. some students attending the conference with their advisors. first row, birsen karpak; second row from left to right, crystal volinchak, erin whitehouse, matthew yourstowsky, and robert woolley; third row, from left to right irem karacakaya, hafize yilmaz, berna unver, ilker topcu, sait gul, ozgur kabak; last row, ozay ozaydin there were presentations using ahp/anp as a methodology by itself or in combination with other methodologies in additional applied sessions as well. i attended two of them. in innovation for sustainable development (organized by schillo, lopez, chaired by sandra lopez) yannick cornet, from technical university denmark, denmark, recommended hiring more transport planners with sustainability experience into government planning agencies and used multiplicative variant of ahp with other mcdm approaches (figure 4). berna unver (figure 3) undergraduate student from istanbul technical university, turkey developed an integrated decision support model to calculate risk scores of workstations in assembly lines (m. isik, o. kabak, i. topcu co-authors). the authors used anp to prioritize factors causing failures on the workstations of the assembly lines in the automotive industry. ijahp news and events: karpak/ mcdm 2017 international conference held for second time in canada international journal of the analytic hierarchy process 306 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.502 figure 4. yannick cornet, giving future generations voice: constructing a sustainability viewpoint in transport appraisal (y. cornet, m.b. barfod, m.j. barradale and r. hickman) there were interesting papers in the decision support for humanitarian relief (organized by aktas, cevik, topcu and chaired by hafize yilmaz) (figure 3). i was able to listen to only two of them. irem karacakaya (figure 3), another undergraduate student from istanbul technical university, turkey developed a decision support model based on ahp for warehouse location in humanitarian relief logistics (i. topcu co-author) (figure 3). in the second paper, the authors (h. yilmaz and o kabak, both in figure 3) proposed a hesitant fuzzy mathematical programming model to contribute to the planning of mitigation efforts for an earthquake disaster expected in 30 years in istanbul. there were quite a few special issues announced in the essay by sarah ben amor (general chair) at http://www.mcdmsociety.org/newsletters/92017-september-2017 one of which was from our own antonella petrillo and her colleagues: theory and applications of fuzzy systems and multicriteria decision analysis, in advances in fuzzy systems, edited by antonella petrillo, fabio de felice, alessio ishizaka. a half-day outing, an afternoon cruise, was organized along the ottawa river and was a great opportunity for informal interaction and relaxing (figure 5). my husband, cengiz karpak is always with me as great supporter. newlywed couples sezin güleryüz and coşkun ergül and makbule and ahmet kandakoglu, who just moved to canada, shared our table. the conference banquet was held at the historical chateau laurier hotel, in one of its most elegant rooms. https://urldefense.proofpoint.com/v2/url?u=http-3a__www.mcdmsociety.org_newsletters_92017-2dseptember-2d2017&d=dwmgaq&c=0w9vy5nnhl9u_frqx4vrzksnz08jjo3five6wvqrtvo&r=8siruohdnke9xopydtaqfa&m=au9irzyxy_ctovpcthlpbc0jffd2hyuwq6pz3i1j1zq&s=u30fvn9ctou5cu1edtoxsfvegqaexhht3b-ggoyo4vy&e= ijahp news and events: karpak/ mcdm 2017 international conference held for second time in canada international journal of the analytic hierarchy process 307 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.502 figure 5. half-day outing, an afternoon cruise along the ottawa river. on the left makbule kandakoglu, ahmet, sezin güleryüz starting from the far right coşkun ergül, birsen karpak, ahmet kandakoglu, cengiz karpak it was great to be with my longtime friends jyrki and hannela wallenius, and murat koksalan our mcdm society president, banu lokman, secretary for the society. we shared memories with the current and future conference chairs (figure 6). figure 6. from left to right: banu lokman, jyrki wallenius, sarah ben amor, hannela wallenius, birsen karpak, i̇lker topçu, and murat köksalan ijahp news and events: karpak/ mcdm 2017 international conference held for second time in canada international journal of the analytic hierarchy process 308 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.502 the 25th international conference on mcdm will be held in istanbul, june 16-21, 2019, and will be organized by i̇lker topçu and his colleagues. ilker received the flag from a very successful conference general chair, sarah ben amor. i am sure he and his colleagues will show us the famous turkish hospitality. i am expecting a huge ahp/anp community there since there are so many mcdm researchers in turkey, and quite few of them are using ahp/anp and new variants, and istanbul is very close to our other community members. ijahp: mac-ginty, carrasco, oddershede, vargas/strategic foresight using an analytic hierarchy process: environmental impact assessment of the electric grid in 2025 international journal of the analytic hierarchy process 186 vol. 5 issue 2 2013 issn 1936-6744 strategic foresight using an analytic hierarchy process: environmental impact assessment of the electric grid in 2025 ronald mac-ginty* department of industrial engineering university of santiago of chile e-mail: ronald.macginty@usach.cl raul carrasco department of industrial engineering university of santiago of chile raul.carrasco.a@usach.cl astrid oddershede department of industrial engineering university of santiago of chile astrid.oddershede@usach.cl manuel vargas department of industrial engineering university of santiago of chile manuel.vargas@usach.cl abstract this paper presents a strategic foresight study of the electrical grid throughout the south american region until 2025. the study considered the climate change phenomenon and many different energy sources, proposing a new methodology through the analytic hierarchy process (ahp) and the monte carlo simulation. the study also considered the earthquake in japan and nuclear plant accident in fukushima, and the technological convergence that will occur over the next 15 years in the electric grid sources. the research involved political, economic, social and technological (pest) factors. through pest analysis and the involvement of an expert panel, it was possible to select the most influential variable for each pest factor. in order to prioritize these factors and evaluate the different technological alternatives, an ahp model was developed. then a monte carlo simulation was run 1000 times for electric generator source clusters. four prospective scenarios of the electrical grid structure until 2025 in the south american mailto:ronald.macginty@usach.cl mailto:raul.carrasco.a@usach.cl mailto:astrid.oddershede@usach.cl mailto:manuel.vargas@usach.cl ijahp: mac-ginty, carrasco, oddershede, vargas/strategic foresight using an analytic hierarchy process: environmental impact assessment of the electric grid in 2025 international journal of the analytic hierarchy process 187 vol. 5 issue 2 2013 issn 1936-6744 region were defined. the study highlighted the contribution of renewable energy adding nuclear power as the main mix group as a source of energy by 2025. this indicates that it is possible to anticipate an electric grid until 2025 in the south american region with low impact on climate change. keywords: climate change, energy sources, ahp, pest analysis, monte carlo method 1. introduction the sustainability of the electricity industry is a matter of study and debate in all societies. decisions for the construction of the electric grid through a set of energy sources should consider a host of factors associated with the environmental and economic impact of the installations. in addition, the accident of the nuclear plant in fukushima has had an effect on the decisions made regarding the capacity and composition of the electric grid. the technological convergence is allowing the development of new sources for generating energy with the contribution from different fields such as biotechnology, nanotechnology, information technology and so on. the consideration of these new technologies and their impact over the next 15 years was performed by the use of suitable methodologies that include the analytic hierarchy process. all of the above compelled us to consider the following two research questions. first, considering the climate change phenomena as a global environmental problem, what will the structure of the electric grid be in the south american region by the year 2025? second, how do you evaluate the impact of technological convergence? the paper is organized as follows: section 2 provides a literature review for prospective analysis while section 3 provides the proposed methodology, its application and results analysis. the conclusions are given in section 4. 2. literature review for strategic foresight and planning scenarios to establish the conditions and actions necessary to build the best possible future, foresight should be able to identify the probable scenarios according to their probability of occurrence (godet, 2000). those who have the responsibility of making decisions should consider all the variables that delineate the most likely scenario through the use of validated and recognized methodologies by the scientific community in order to define future scenarios of a community, organization or a geographic region(godet, i buisán, & posiello, 1995; godet & monti, 2000). in our review of the literature we found that the delphi method, the odds of bayes method, the cross-impact matrix method, and exploration of the environment and morphological analysis method are used the most (glenn & gordon, 2009). the ahp, developed by professor saaty (saaty, 2001, 2007), has shown great capacity of prioritizing both, quantitative and qualitative criteria. this method in conjunction with the monte carlo simulation allows the reduction of the uncertainty of the actions for decision-making (emblemsvåg & tonning, 2003; liu, rasul, amanullah, & khan, rob typewritten text http://dx.doi.org/10.13033/ijahp.v5i2.195 ijahp: mac-ginty, carrasco, oddershede, vargas/strategic foresight using an analytic hierarchy process: environmental impact assessment of the electric grid in 2025 international journal of the analytic hierarchy process 188 vol. 5 issue 2 2013 issn 1936-6744 2010). all of the above encourages us to study the phenomenon of the growth of energy consumption, its impact, and decision-making of prospective scenarios using these tools. according to vecchiato (2012), strategic management literature defines the micro environment and the macro environment by distinguishing sectors with which the firm has direct contact and from sectors that affect the firm indirectly respectively. the macro environment is made up of the political, economic, societal and technological landscapes (pest) which surround the business micro environment. 2.1 emerging trends in electricity generation in the last decade, 70% of the energy consumed by the global economy was fossil fuels (canton, 2006). hubbert model, according to canton, designates the maximum level of oil production during this decade, increasing the average price trend projection. it is crucial to replace fossil fuels with cleaner sources in order to reduce the carbon footprint. this is the starting point for all the new technologies like hydrogen, magnetism and superconductivity. trends and the newly discovered state of the art technologies that can be applied to emerging power generation technologies are shown in table 1. these technologies were divided into the following categories: superconductor materials, quantum computers, photovoltaic paints, nanoscience and biotechnology. each of these categories raises potential scenarios that have different impacts on quality of life. ijahp: mac-ginty, carrasco, oddershede, vargas/strategic foresight using an analytic hierarchy process: environmental impact assessment of the electric grid in 2025 international journal of the analytic hierarchy process 189 vol. 5 issue 2 2013 issn 1936-6744 table 1 trends of emerging power generation technologies that underpin the major changes category area of knowledge applications potential scenario materials physics of materials • superconductivity, in which some metals and alloys lead electricity without meeting resistance. roads and railway with superconductors at room temperature may cause super-magnetic effect, capable of making vehicles and trains to move through magnetic levitation. information technologies quantum computers • the physicist david deutsch, envisioned how officer a tidy realizing that calculations that needed a time virtually infinite in a traditional computer could be done quickly in a quantum computer. nuclear fusion will be available. nanotechnologies applications nanotechnology and new materials • processing, storage, distribution technologies, the new advanced photovoltaic catalysts (organic solar cells). • nano-electronics. • energy generation, as through the paintable solar technologies applied to buildings and outdoor solar structures. • built environments capable of adapting to future climates. environmental nano photonics: nano science for energy efficiency and environmental sustainability • painted "cooling" coatings that reduces the demand of energy. • switchable 'smart' surfaces, properties change according to conditions or needs. • nano-structured windows for more energy efficient buildings. reduction of the urban "heat island" effect through "colder suburbs” created through the application of roofs of all the buildings nanostructured coatings to reflect heat. • reducing the "maximum load" energy demand during the summer. biotechnology and the environment biotechnology, energy and industrial production • petroleum diesel and substitute fuels • conversion of lignocellulose to ethanol, biodiesel from algae technologies. • bio-industry focused on crops genetically modified high-value industrial engineering. • the transport of liquid fuel safety is achieved without affecting food production / security. • bioeconomy mitigates future oil shocks / price rises. biotechnology and food production • genetic modification (gm) to change the performance characteristics. • access to good price essential nutrients, for the rapid expansion of the world's population "clean technology" new solutions for energy, water and transportation • next generation of energy systems. • the adaptation of technologies for water purification. • biological sequestration technologies. • functional "green wall / ceiling systems". • new energy technologies well adapted in poor countries • charging power base renewable (for example, using biomass as a source of energy undervalued) • affordable and low-cost energy source: adapted from (alford, keenihan, & mcgrail, 2012; kaku, 2011; kurzweil, 2005; national nanotechnology initiative, 2012). ijahp: mac-ginty, carrasco, oddershede, vargas/strategic foresight using an analytic hierarchy process: environmental impact assessment of the electric grid in 2025 international journal of the analytic hierarchy process 190 vol. 5 issue 2 2013 issn 1936-6744 2.2 prospective studies for the electrical industry prospective studies related to electric energy sources are divided into two phases until 2025. the literature review indicates the global installation of traditional green technologies. wind and solar power lead investments on all continents. this first phase will be accompanied by substantial improvements in the production of uranium enriched by laser (weinberger, 2012). this method will drastically reduce the production cost and increase the ability to obtain the product for nuclear fission plants. finally, electric vehicles with lithium or hydrogen batteries generated a double effect on the economy of big cities. more efficient, less polluting vehicles with storage capacities of energy equivalent to several nuclear power plants will allow the duality in the flow of energy, with vehicles possibly buying and selling power to the electrical grid system (kaku, 2011). the second phase involves the convergence of emerging trends that will allow more capacity to generate electricity where the emerging technologies will be connected to large regional or national distribution networks and technologies. more efficient solar cells, photovoltaic batteries including nanotechnology and smart distribution will begin sometime in the next decade. 2.3 greenhouse gas emission the various energy sources used in power generation create emissions of greenhouse gases when considering the full life cycle of each project. these emissions can be seen from the construction of the project, the power production, to the energy expenditure in operation and maintenance, and decommissioning and closure of the facility. in table 2, the emission values for each technology can be explained by factors specific to each installation as climatic variables, energy demand and the method of power production. the largest variation corresponds to the photovoltaic technology. in the last few years this technology has improved ostensibly regarding energy efficiency per unit of surface, and in its manufacturing process (edenhofer, pichs-madruga, & sokona, 2011; laguna, 2002; meier, 2002; monroy, 2002; weinberger, 2012). the first units had a broadcast life cycle comparable to fossil fuel-based computers, a decade later photovoltaic units maintain cycles similar to renewable energy. ijahp: mac-ginty, carrasco, oddershede, vargas/strategic foresight using an analytic hierarchy process: environmental impact assessment of the electric grid in 2025 international journal of the analytic hierarchy process 191 vol. 5 issue 2 2013 issn 1936-6744 table 2 different energy source’s emissions during the life cycle in tons of co2 per gwh technology average minimum value maximum value brown coal 1.054 790 1372 coal 888 756 1310 oil 733 547 935 natural gas 499 362 891 solar photovoltaic 85 13 731 biomass 45 10 101 nuclear 29 2 130 hydropower 26 2 237 wind 26 6 124 source: adapted from report comparison of lifecycle greenhouse gas emissions of various electricity generation sources (world nuclear association report, 2010). through statistical analysis of hierarchical grouping (ward with euclidean distance measurement methodology), we obtained two hierarchical groups: sources with high level of gas emission greenhouses (lignite, coal, oil and natural gas), and sources of lowemission (photovoltaic, biomass, nuclear, hydropower and wind). 2.4 fukushima effect the effect in the international community of the fukushima accident created pressure for the deactivation of nuclear power units in many countries. however, the difficulty in the short term of replacing this energy source with others with less impact on climate change, taking into consideration the fear in the population regarding possible nuclear accidents, compels us to explore two aspects of nuclear units. these are the equipment installed in seismic geographical areas, and units planned or installed in safe areas. 3. methodology and analysis the prospective analysis of the impact of energy sources to the electricity grid considered the full lifecycle of energy sources. the proposed methodology can be summarized in six steps as indicated in section 3.1, and represented in six steps as depicted in figure 1. ijahp: mac-ginty, carrasco, oddershede, vargas/strategic foresight using an analytic hierarchy process: environmental impact assessment of the electric grid in 2025 international journal of the analytic hierarchy process 192 vol. 5 issue 2 2013 issn 1936-6744 pest factors assessment dendrogram cluster grouping building scenaries literature review pest grouping ranking alternative technologies using ahp and sensitive analysis figure 1. proposed methodology 3.1 proposed methodology 1) literature review: a comprehensive literature review is required to recognize the main factors involved in electricity generation, the development of new technologies and the control of environmental impacts from different sources (see table 1). 2) pest grouping: main factors are grouped to each of the pest factors according to the judgments of a previously selected expert team (see table 3 and table 4). 3) pest factors assessment: an ahp model is designed for pest factors prioritization (figure 2). 4) building scenarios: four scenarios are created in graphic xy using the method described by godet (2000). mean objective is to mitigate impact of “climate change”, so our scenario target is in quadrant i (figure 3). 5) dendrogram cluster grouping: dendrogram technique is proposed to group technologies in family clusters. this allows for identification of a technology with a similar impact to the environmental impact (see table 3). 6) ranking alternative technologies using ahp and sensitive analysis: in this step the overall outcomes are obtained, furthermore a montecarlo simulation is run with a vba excel 1000 times to improve the results (momani & ahmed, 2011). ijahp: mac-ginty, carrasco, oddershede, vargas/strategic foresight using an analytic hierarchy process: environmental impact assessment of the electric grid in 2025 international journal of the analytic hierarchy process 193 vol. 5 issue 2 2013 issn 1936-6744 3.2 application according to step 1 of the methodology, a list of associated variables of energy generation technologies is arranged in table 3 based on chatzimouratidis & pilavachi, (2008, 2009) and the world nuclear association report (2010). table 3 associated variables list of energy generation technologies item technology polluting emissions(3) land occupied(2) cost(1) new jobs(1) social rejection(1) energy efficiency(2) technological availability(2) units tco2/gwh ha cents/kwh 1-300 % 1-100 1 brown coal 1.054 2,5 5,4 2500 100 39 85 2 coal 888 2,5 5,4 2500 100 39 85 3 oil 733 2,5 5,0 2500 100 37 92 4 natural gas 499 2,5 4 2460 33 39 91 5 solar photovoltaic. 85 35 75 5370 20 9 20 6 biomass 45 5000 14 36055 33 14 80 7 nuclear 29 2,5 4 2500 300 33 96 8 hydropower 26 750 8 2500 33 80 50 9 wind 26 100 7 5635 20 35 38 source: adapted from (1) (chatzimouratidis & pilavachi, 2008),(2) (chatzimouratidis & pilavachi, 2009) and from (3) (world nuclear association report, 2010). the involved variables from table 3 are studied and grouped into the four pest factors according to the judgments of the expert team (environmental engineers, chemical engineers, sociologist, energy specialists, economists), as shown in table 4. table 4 variables associated to pest factors pest factors variable political polluting emissions land occupied economic cost societal new jobs social rejection technological energy efficiency technological availability according to step 3, the pest factors will be assessed through ahp. an ahp hierarchy model is structured and validated by the expert team. then a pair wise comparison through the hierarchy structure to derive the priority matrix for each level of the structure is carried out. figure 2 shows the initial basic ahp model. ijahp: mac-ginty, carrasco, oddershede, vargas/strategic foresight using an analytic hierarchy process: environmental impact assessment of the electric grid in 2025 international journal of the analytic hierarchy process 194 vol. 5 issue 2 2013 issn 1936-6744 figure 2. the basic ahp decision model to assess factors the resultant relative weights for pest factors are indicated in table 5. table 5 pest factors relative weights factors relative weights (%) political 38.06 economic: 30.60 social: 8.59 technological: 22.75 3.3 building scenarios building scenarios is the subsequent recommended step. variables are related the variables by pairs, such as political-social and economic-technological which facilitates the creation of four scenarios and are introduced into the x-y diagram using the procedure specified by godet (2000). see figure 3. the first scenario named “conquering the future” supports positive thinking to mitigate the impact of “climate change”. this might be possible if the selected technologies produce low levels of carbon footprint and if they are less expensive (axis y positive). ijahp: mac-ginty, carrasco, oddershede, vargas/strategic foresight using an analytic hierarchy process: environmental impact assessment of the electric grid in 2025 international journal of the analytic hierarchy process 195 vol. 5 issue 2 2013 issn 1936-6744 on the other hand, axis x, positive socio-political presents the expectation of achieving government support related to environmental politics and social approval issues. this scenario defines the objective of this study, which is to find the most adequate technology for power generation source for the scenario “conquering the future” to 2025. figure 3. scenarios of the electric grid in 2025 on the other hand, all restrictions on global energy supply policies will establish any of the other three remaining scenarios where restrictions will reduce the capabilities and benefits of clean and low cost energy for human life. 3.4 dendrogram cluster grouping a dendrogram cluster is used to group technologies with similar environmental impact according to table 3. an initial classification of technologies, revealing three major groups is shown in figure 4. the first focuses on all the energy sources from fossil fuels, (1, 2, 3, and 4), the second focuses on nuclear energy (7), and the third focuses on all renewable sources (5, 6, 8, and 9). figure 4. cluster dendrogram for evaluating similarity between energy sources (software r, 2012) wasted opportunities conquering the future ecological crisis waiting for a future negative socio-political positive socio-political positive economic-technological negative economic-technological ijahp: mac-ginty, carrasco, oddershede, vargas/strategic foresight using an analytic hierarchy process: environmental impact assessment of the electric grid in 2025 international journal of the analytic hierarchy process 196 vol. 5 issue 2 2013 issn 1936-6744 considering the initial dendrogram classification and the energy generation capacity of these groups (i, ii, iii) from figure 3, the expert team proposed another arrangement involving four clusters to be evaluated through ahp. the resultant clusters are indicated in table 6. table 6 resultant clusters energy generation sources cluster i) nuclear energy ii) renewable energy and nuclear iii) fossil energy iv) fossil energy and nuclear the ranking results and sensitivity analysis obtained from montecarlo simulation between the four selected technologies clusters are presented in figure 5. the simulation features predominance in the use of renewable technologies supported by nuclear sources with 78% of the preference. figure 5 ranking results and sensitivity analysis with the highest score of the selected choice, the renewable nuclear mix source meets a positive pest evaluation as is therefore located in quadrant i as seen in figure 6. ijahp: mac-ginty, carrasco, oddershede, vargas/strategic foresight using an analytic hierarchy process: environmental impact assessment of the electric grid in 2025 international journal of the analytic hierarchy process 197 vol. 5 issue 2 2013 issn 1936-6744 figure 6. selected electric generation source in the positive scenario according to the pest criteria. 4. conclusions the proposed methodology used the ahp method based on pest criteria to evaluate different energy source groups. a sensitivity analysis using montecarlo simulation, to take into account the uncertainty in human judgment supported the cluster ranking. the renewable-nuclear group was the best choice for the south american region that considered the impact of the “climatic change” phenomena until 2025. grouping the alternatives according to similarity or choosing a representative variable between many alternatives contributes to the understanding of the problem and can save resources. a matrix of scenarios allows for defining the exploratory stages and components that identify and determine the future of the industry. in our case, pest criteria define the expected scenario. the renewable-nuclear cluster reached 78% of the preference, which means that with this kind of technology the “climate change” impact in the south american region until 2025 will be minimized. finally, it is necessary to say that at any time new energy technology sources might emerge and may change the projections, so it is advisable to use this methodology on a periodic basis and to analyze the trends. acknowledgements the authors would like to thank the industrial engineering department of university of santiago of chile for their support. positive economic-technological positive political-social i.renewable nuclear iv iii ii ijahp: mac-ginty, carrasco, oddershede, vargas/strategic foresight using an analytic hierarchy process: environmental impact assessment of the electric grid in 2025 international journal of the analytic hierarchy process 198 vol. 5 issue 2 2013 issn 1936-6744 references alford, k., keenihan, s., & mcgrail, s. (2012). the complex futures of emerging technologies: challenges and opportunities for science foresight and governance in australia. journal of futures studies, 16(4), 67–86. canton, j. (2006). the extreme future: the top trends that will reshape the world for the next 5, 10, and 20 years. new york: dutton. chatzimouratidis, a. i., & pilavachi, p. a. (2008). multicriteria evaluation of power plants impact on the living standard using the analytic hierarchy process. energy policy, 36(3), 1074–1089. chatzimouratidis, a. i., & pilavachi, p. a. (2009). technological, economic and sustainability evaluation of power plants using the analytic hierarchy process. energy policy, 37(3), 778–787. edenhofer, o., pichs-madruga, r., & sokona, y. (2011). renewable energy sources and climate change mitigation: special report of the intergovernmental panel on climate change. cambridge, uk: cambridge university press. emblemsvåg, j., & tonning, l. (2003). decision support in selecting maintenance organization. journal of quality in maintenance engineering, 9(1), 11–24. doi:10.1108/13552510310466765 glenn, j. c., & gordon, t. j. (2009). futures research methodology: the millennium project: version 3.0. washington, d.c.: milennium project. godet, m. (2000). the art of scenarios and strategic planning: tools and pitfalls. technological forecasting and social change, 65(1), 3–22. godet, m., i buisán, e. p., & posiello, j. g. (1995). de la anticipación a la acción: manual de prospectiva y estrategia. alfaomega méxico. godet, m., & monti, r. r. (2000). la caja de herramientas de la prospectiva estratégica. gerpa. kaku, m. (2011). physics of the future: how science will shape human destiny and our daily lives by the year 2100. knopf doubleday publishing group. kurzweil, r. (2005). the singularity is near: when humans transcend biology. new york: viking penguin. ijahp: mac-ginty, carrasco, oddershede, vargas/strategic foresight using an analytic hierarchy process: environmental impact assessment of the electric grid in 2025 international journal of the analytic hierarchy process 199 vol. 5 issue 2 2013 issn 1936-6744 laguna, i. (2002). generación de energía eléctrica y medio ambiente. gaceta ecológica, (65), 53–62. liu, g., rasul, m. g., amanullah, m. t. o., & khan, m. m. k. (2010). ahp and fuzzy assessment based sustainability indicator for hybrid renewable energy system. in universities power engineering conference (aupec), 2010 20th australasian (pp. 1–6). meier, p. j. (2002). life-cycle assessment of electricity generation systems and applications for climate change policy analysis. (unpublished doctoral dissertation). university of wisconsin, madison. momani, a. m., & ahmed, a. a. (2011). material handling equipment selection using hybrid monte carlo simulation and analytic hierarchy process, world academy of science, engineering and technology, 59, 953–958. monroy, i. l. (2002). la generación de energía eléctrica y el ambiente. gaceta ecológica, (065), 53–62. national nanotechnology initiative. (2012). saaty, t. l. (2001). decision making with dependence and feedback: the analytic network process (2 ed.), pittsburgh: rws publications. saaty, t. l. (2007). the analytic hierarchy and analytic network measurement processes: applications to decisions under risk. european journal of pure and applied mathematics, 1(1), 122–196. vecchiato, r. (2012). environmental uncertainty, foresight and strategic decision making: an integrated study. technological forecasting and social change, 79(3), 436– 447. doi:10.1016/j.techfore.2011.07.010 weinberger, s. (2012). laser plant offers cheap way to make nuclear fuel. nature, 487(7405), 16–17. world nuclear association report. (2010). comparison of lifecycle greenhouse gas emissions of various electricity generation sources. ijahp article: mallick, sarkar, das/a unified decision framework for inventory classification through graph theory international journal of the analytic hierarchy process 183 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.482 a unified decision framework for inventory classification through graph theory bivash mallick assistant professor, department of industrial engineering & management maulana abul kalam azad university of technology, west bengal, india bivash.mallick@gmail.com bijan sarkar professor, department of production engineering, jadavpur university, kolkata, india bsarkar@production.jdvu.ac.in santanu das professor and head, department of mechanical engineering kalyani govt. engineering college, kalyani, india sdas.me@gmail.com abstract conventionally, a traditional abc analysis based on a single criterion of annual consumption cost is employed in industry to facilitate classification of inventory items. however, other criteria may be important in inventory classification such as lead time, item criticality, storage cost, etc. hence, for situations like this many multiple criteria decision-making methods are available and the analytic hierarchy process (ahp) is a popular one. the present article demonstrates a new approach by integrating graph theory (gt) and the analytic hierarchy process (ahp) as a decision analysis tool for multi-criteria inventory classification. in this paper, 47 disposable items used in a respiratory therapy unit of a hospital were considered for a case study. output of this hybrid method shows more precise results than that of either traditional abc or the ahp classification methods. as the proposed decision analysis tool is a simple, logical, systematic and consistent method, it may be recommended for application in diverse industries handling multi-criteria inventory classification systems. keywords: abc classification; graph theory; analytic hierarchy process; inventory classification; graph theory-ahp integration; hybrid system 1. introduction inventories are resources of any kind that have an economic value. appropriate inventory control is necessary because efficiency and cost of its operation are generally largely mailto:bivash.mallick@gmail.com mailto:bsarkar@production.jdvu.ac.in mailto:sdas.me@gmail.com ijahp article: mallick, sarkar, das/a unified decision framework for inventory classification through graph theory international journal of the analytic hierarchy process 184 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.482 affected by both its excess and shortage. inventory control is thus essential to determine the item(s) to indent (i.e., to order) along with its quantity, time to indent and the optimum stock to maintain so that purchasing and storage costs are reduced to a minimum (mallick, dutta, & das, 2012).hence, the planning and control of inventory attract substantial attention from the management in an organization. in a large organization, the inventory that must be maintained consists of a great variety of items. statistics reveal that just a handful of items account for the bulk of the annual expenditure on materials. these few items, called ‘a’ items, therefore hold the key to the business. the other items, known as ‘b’ and ‘c’ items, are large in number, however their contribution is less. traditional abc analysis is performed based on the consumption values of inventory items. consumption values are arranged in descending order. cumulative consumption values are then converted corresponding to cumulative percentages. a, b and c classifications are then made based on the cumulative percentage figures. to classify the inventories, the break point percentages can be chosen by the management depending on the number of items that can be effectively managed under each category (flores, olson, & dorai, 1992). a number of researchers have questioned the focus on the consumption value as a single criterion. there are many instances when other criteria, other than the annual consumption value, may be significant in deciding the importance of an inventory item (cohen & ernst, 1988). in these cases, multiple criteria decision-making methods are helpful. a few investigators have worked on multi-criteria inventory classification. flores and whybark introduced a multi-criteria inventory classification in 1986 and 1987 (flores & whybark, 1986, 1987). though it included several criteria, e.g. obsolescence, lead times, substitutability, reparability, criticality and commonality, this approach became increasingly complicated if more than two criteria were considered to classify inventory items. flores et al. (1992) applied the analytic hierarchy process (ahp) for multiple criteria inventory classification. guvenir & erel (1998) applied genetic algorithm (ga) to solve the problem of multiple criteria inventory classification. their proposed method called genetic algorithm for multi-criteria inventory classification (gamic) is within the framework of the ahp to deal with multi-criteria abc analysis. on the other hand, braglia et al. (2004) used the ahp models to solve various multi-attribute decision problems at different levels of the decision tree. they identified the best control strategy for the spare stocks by defining the inventory policy matrix linking different classes of spare parts with the possible inventory management policies. ramanathan (2006) proposed a weighted linear optimisation model for multiple criteria abc (mcabc) inventory classification. the performance score of each item was obtained using a data envelopment analysis (dea). the defect of his model was that an item with a high value in an unimportant criterion was inappropriately classified under class a. this drawback was rectified by zhou & fan (2007) who incorporated balancing features for multi-criteria inventory classification (mcic) by using most favourable and least favourable scores for each item. in another work, bhattacharya et al. (2007) ijahp article: mallick, sarkar, das/a unified decision framework for inventory classification through graph theory international journal of the analytic hierarchy process 185 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.482 presented a distance-based multi-criteria consensus framework utilizing the concepts of the topsis (technique for order preference by similarity to ideal solution) model. the technique took into account various conflicting criteria having different units of measurement, which were then ranked in categories a, b and c using topsis. cakir & canbolat (2008) presented an inventory classification system based on fahp, integrating the fuzzy concept with real inventory data and designed a decision support system. torabi et al. (2012) applied a modified version of a common weight dea-like model to abc inventory classification in those cases where both quantitative and qualitative criteria existed, while kabir & hasin (2013) developed a multi-criteria inventory classification model through integration of fuzzy analytic hierarchy process (fahp) and artificial neural network approach. soylu & akyol (2014) incorporated the preferences of the decision maker into the multi-criteria inventory classification decisionmaking process in terms of reference items into each class. in a recent work, a new method of evaluation based on distance from average solution (edas) was introduced for multi-criteria inventory classification (mcic) problems (ghorabaee, 2015). a comparative analysis (involving seven sets of criteria weights and spearman’s correlation coefficient) was also put forward to show the validity and stability of the proposed method in problems related to mcdm. liu et al. (2016) mentioned the necessity of considering the noncompensation in the multiple criteria abc analysis. the authors proposed a new classification approach based on the outranking model to cope with such a problem. mallick et al. (2016) presented a multicriteria inventory classification system by moora (multi-objective optimization on the basis of ratio analysis) method for hospital inventory management. none of the multiple criteria decision-making methods mentioned above is foolproof and a single multi-criteria decision-making method doesn’t exist that can be applied to all decision problems with equal efficiency. therefore, there is a need for a simple, systematic, logical and consistent method or tool to guide decision makers in making an optimal selection. hence, a methodology was developed by integrating graph theory (gt) and the analytic hierarchy process (ahp) for multi-criteria inventory classification to solve a typical hospital inventory management problem. the outcome of the present application of this hybrid method was also compared with certain previous researchers dealing with same problem. 2. the proposed methodology graph theory is a logical and systematic approach that originated from combinatorial mathematics. the graph theory consists of the digraph representation, the matrix representation and the permanent function representation. the digraph is the visual representation of the variables and their interdependencies. the matrix converts the digraph into mathematical form and the permanent function is a mathematical representation that helps to determine the numerical index (faisal, banwet, & shankar, 2007).various researchers have addressed the application of the graph theory approach to engineering systems in a significant number of papers. ijahp article: mallick, sarkar, das/a unified decision framework for inventory classification through graph theory international journal of the analytic hierarchy process 186 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.482 rao (2007) in his book has demonstrated how this methodology could be effectively used for decision making in various situations in the manufacturing environment. the graph theory and matrix approach (gtma) method has been employed to total quality management evaluation of an industry (grover, agrawal, & khan, 2004), contractor ranking (darvish, yasaei, & saeedi, 2009), non-traditional machining processes selection (chakladar, das, & chakraborty, 2009), equipment selection (paramasivam, senthil, & rajam ramasamy, 2011), and assessing the vulnerability of supply chains (wagner & neshat, 2010). the analytic hierarchy process (ahp) developed by saaty (1980) has been successfully applied to multi-criteria inventory classification by flores et al. (1992).the ahp has been used in a variety of business decision-making settings. a significant feature of the ahp is the consistency check of relative importance of the attributes. in view of this advantage, for multi-criteria inventory classification, a methodology combining graph theory and the ahp method is proposed. the ahp is applied for determination of the relative weights of the selected attributes and graph theory is applied for evaluation and ranking of the alternatives. this hybrid approach is similar to the one reported by rao and parnichkun (2009), singh and rao (2011) and lanjewar et al. (2016). the proposed hybrid method is a simple, fast, logical, systematic and consistent method, that may be recommended for use as a decision making method for multi-criteria inventory classification. however, it can be used for any type of decision making situation. it enables more critical analysis than the common multi-attribute decision making methods since any number of quantitative and qualitative attributes can be considered. in the permanent procedure, even a small variation in the attribute leads to a significant difference in the overall performance value making it convenient to rank the alternatives in a descending order with a clear difference in the overall performance value. moreover, the proposed procedure using the graphical representation not only enables the analysis of the alternatives, but also facilitates visualization of various criteria along with their interrelations. the steps for the proposed methodology are presented below after rao (2007), chakladar et al. (2009), and darvish et al. (2009). step1identify the inventory attributes or criterion for decision matrix. step 2generate the decision matrix using raw inventory data. a typical multi-criteria decision making problem can be precisely presented with a decision matrix showing the performance of different alternatives with respect to various attributes or criteria as given in equation1. d = [xij]i= 1,… ,m, j= 1,….,n (1) where xij is the performance measure of i th alternative on jth criteria, m is the number of alternatives; n is the number of criteria. information stored in a decision matrix is generally incommensurable, wherein performance ratings w.r.t. different criteria are ijahp article: mallick, sarkar, das/a unified decision framework for inventory classification through graph theory international journal of the analytic hierarchy process 187 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.482 expressed after considering different units of measure. therefore, it is desirable to transform data into comparable values, using the normalization procedure of step 3. step 3normalize the matrix calculation. the normalized values of a criterion assigned to the alternatives are calculated using the following formula: 𝐹𝑖 − 𝐹𝑚𝑖𝑛 𝐹𝑚𝑎𝑥 − 𝐹𝑚𝑖𝑛 where, fi is the i th value of the factor, fmax is the maximum value of the factor, and fmin is the minimum value of the factor under transformation (flores et al.,1992). there are also some other methods of normalizing which incorporate negative values into the ahp system (millet & schoner, 2005). step 4construct the relative importance matrix a relative importance matrix (equation 2) is constructed using a pair-wise comparison scale of absolute numbers from 1 to 9 as proposed by saaty (1980). an element when self-compared is assigned a value of 1. assuming that there are n number of criteria in a decision-making problem, the pair-wise comparison of i th criterion with respect to the j th one yields a square matrix, where aij = 1 when i = j and aji = 1/aij (aij is the comparative importance of i th criterion with respect to j th one). m = [aij] = [ a11 a12 … a1n a21... a22 … a2n... ai1 ai2 … ann ] (2) step 5digraph model of the inventory attributes showing their interdependencies. a digraph consists of a set of nodes n= {ni} (i=1, 2, m) and a set of directed edges e= {eij}. a node ni represents i th selection criterion/attribute and edges represent the relative importance among the attributes. the total of nodes, m, is equal to the number of selection criteria considered. if a node i has relative importance over another node j in the selection process, a directed edge or arrow is drawn from node i to node j (eij). if j is having relative importance over i, then a directed edge or arrow is drawn from node j to node i(eji). the digraph depicts the graphical representation of the interdependence between various decision attributes taken two at a time and their relative importance for quick visual perception. step 6develop other square matrix based on the digraph as the number of nodes and their interrelations increase, the digraph becomes complex. to address this problem, the digraph is usually represented in a matrix form. matrix representation of a digraph gives one-to-one representation, taking all the attributes (ai) and their relative importance (aij) into account. the matrix b for a digraph can be represented as equation 3: ijahp article: mallick, sarkar, das/a unified decision framework for inventory classification through graph theory international journal of the analytic hierarchy process 188 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.482 b = [ 𝐴𝑖 𝑎12 𝑎13 … … 𝑎21 𝐴2 𝑎23 … … 𝑎31 𝑎32 𝐴3 … … ⋮ ⋮ ⋮ ⋮ 𝑎1𝑀 𝑎2𝑀 𝑎3𝑀 ⋮ ⋮ 𝑎𝑀1 𝑎𝑀2 𝑎𝑀3 … … 𝐴𝑀 ] … (3) where ai is the value of the i th attribute represented by node ni and aij is the relative importance of the i th attribute over the j th one represented by the edge eij. step 7calculate the overall performance value using matrix method. the overall performance value of each inventory item can be found by calculating the permanent function value of matrix b, i.e. per (b) (rao, 2007). the permanent of a matrix was introduced by cauchy in 1812. at that time, while developing the theory of determinants, he also defined a certain subclass of symmetric function that later was named permanent by muir. in the expression for the permanent of the matrix, as no negative sign appears, no information is lost. this permanent function is the determinant of a matrix, considering all the terms as positive. step 8rank the inventory items based on overall performance value. in this step, all inventory items are ranked according to their overall performance value arranged in descending order. 3. illustrative example the hybrid system, i.e. integrated graph theory and the analytic hierarchy process, is applied in this paper for classifying inventory items using the data referred to in flores et al. (1992) corresponding to hospital inventory management. flores applied the multiple criteria abc analysis method based on the analytic hierarchy process (ahp) using data obtained from a traditional abc inventory classification presented by reid (1987) now, in order to demonstrate and validate the proposed procedure, the multi-criteria inventory classification integrating graph theory (gt) and the analytic hierarchy process and the various steps of the methodology, given in section 2, are described below: step 1four attributes are identified for the multiple criteria abc analysis: (1) average unit cost (auc); (2) annual dollar usage (adu); (3) critical factor (cf): 1, 0.50, or 0.01 being assigned to each of the 47 disposable items, where values imply very critical, moderately critical, and non-critical respectively; (4) lead time (lt) (weeks) being the time that it takes to receive a replenishment order after it is placed, ranging from 1 to 7 weeks. ijahp article: mallick, sarkar, das/a unified decision framework for inventory classification through graph theory international journal of the analytic hierarchy process 189 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.482 step 2generation of decision matrix using raw inventory data based on the 47 disposable items referred to as s1 through s47 are taken from flores et al.(1992) (table 1). table 1 decision and normalised matrix item decision matrix normalised matrix aud ($) adu ($) cf lt aud ($) adu ($) cf lt s1 49.92 5840.64 1 2 0.21866 1.00000 1.00000 0.16667 s2 210 5670.00 1 5 1.00000 0.97066 1.00000 0.66667 s3 23.76 5037.12 1 4 0.09098 0.86183 1.00000 0.50000 s4 27.73 4769.56 0.01 1 0.11036 0.81582 0.00000 0.00000 s5 57.98 3478.80 0.5 3 0.25800 0.59385 0.49495 0.33333 s6 31.24 2936.67 0.5 3 0.12749 0.50063 0.49495 0.33333 s7 28.2 2820.00 0.5 3 0.11265 0.48057 0.49495 0.33333 s8 55 2640.00 0.01 4 0.24346 0.44961 0.00000 0.50000 s9 73.44 2423.52 1 6 0.33346 0.41239 1.00000 0.83333 s10 160.5 2407.50 0.5 4 0.75840 0.40963 0.49495 0.50000 s11 5.12 1075.20 1 2 0.00000 0.18053 1.00000 0.16667 s12 20.87 1043.50 0.5 5 0.07687 0.17508 0.49495 0.66667 s13 86.5 1038.00 1 7 0.39721 0.17413 1.00000 1.00000 s14 110.4 883.20 0.5 5 0.51386 0.14751 0.49495 0.66667 s15 71.2 854.40 1 3 0.32253 0.14256 1.00000 0.33333 s16 45 810.00 0.5 3 0.19465 0.13492 0.49495 0.33333 s17 14.66 703.68 0.5 4 0.04656 0.11664 0.49495 0.50000 s18 49.5 594.00 0.5 6 0.21661 0.09778 0.49495 0.83333 s19 47.5 570.00 0.5 5 0.20685 0.09365 0.49495 0.66667 s20 58.45 467.60 0.5 4 0.26030 0.07604 0.49495 0.50000 s21 24.4 463.60 1 4 0.09410 0.07536 1.00000 0.50000 s22 65 455.00 0.5 4 0.29227 0.07388 0.49495 0.50000 s23 86.5 432.50 1 4 0.39721 0.07001 1.00000 0.50000 s24 33.2 398.40 1 3 0.13706 0.06415 1.00000 0.33333 s25 37.05 370.50 0.01 1 0.15585 0.05935 0.00000 0.00000 s26 33.84 338.40 0.01 3 0.14018 0.05383 0.00000 0.33333 s27 84.03 336.12 0.01 1 0.38515 0.05344 0.00000 0.00000 s28 78.4 313.60 0.01 6 0.35767 0.04956 0.00000 0.83333 s29 134.34 268.68 0.01 7 0.63071 0.04184 0.00000 1.00000 s30 56 224.00 0.01 1 0.24834 0.03415 0.00000 0.00000 ijahp article: mallick, sarkar, das/a unified decision framework for inventory classification through graph theory international journal of the analytic hierarchy process 190 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.482 s31 72 216.00 0.5 5 0.32643 0.03278 0.49495 0.66667 s32 53.02 212.08 1 2 0.23380 0.03211 1.00000 0.16667 s33 49.48 197.92 0.01 5 0.21652 0.02967 0.00000 0.66667 s34 7.07 190.89 0.01 7 0.00952 0.02846 0.00000 1.00000 s35 60.6 181.80 0.01 3 0.27079 0.02690 0.00000 0.33333 s36 40.82 163.28 1 3 0.17425 0.02371 1.00000 0.33333 s37 30 150.00 0.01 5 0.12144 0.02143 0.00000 0.66667 s38 67.4 134.80 0.5 3 0.30398 0.01882 0.49495 0.33333 s39 59.6 119.20 0.01 5 0.26591 0.01613 0.00000 0.66667 s40 51.68 103.36 0.01 6 0.22725 0.01341 0.00000 0.83333 s41 19.8 79.20 0.01 2 0.07165 0.00925 0.00000 0.16667 s42 37.7 75.40 0.01 2 0.15902 0.00860 0.00000 0.16667 s43 29.89 59.78 0.01 5 0.12090 0.00592 0.00000 0.66667 s44 48.3 48.30 0.01 3 0.21076 0.00394 0.00000 0.33333 s45 34.4 34.40 0.01 7 0.14291 0.00155 0.00000 1.00000 s46 28.8 28.80 0.01 3 0.11558 0.00059 0.00000 0.33333 s47 8.46 25.38 0.01 5 0.01630 0.00000 0.00000 0.66667 step 3the quantitative values of the inventory attributes, which are given in table 1, are normalized. it is to be noted that all four criteria considered in this study are positively related to the importance level of inventory items (ramanathan, 2006). step 4relative importance relation matrix (a1) as shown in table 2 and weight (wi) of each attribute for the above-mentioned criteria is kept the same as in flores et al. (1992). table2 relative importance relation matrix the weight (wi) of each attribute is taken as: wauc: 0.07872; wadu: 0.09161; wcf: 0.41969; and wlt: 0.40999 after flores et al. (1992). step 5digraph model of the inventory attributes showing their interdependencies is shown in figure 1. as explained, the nodes in the digraph represent criteria. the interactions among criteria are represented by edges. relative importance relation matrix (a1) auc adu cf lt auc 1 1 1/8 ¼ adu 1 1 1/3 1/6 cf 8 3 1 1 lt 4 6 1 1 ijahp article: mallick, sarkar, das/a unified decision framework for inventory classification through graph theory international journal of the analytic hierarchy process 191 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.482 figure1. digraph model of the inventory attributes step 6the attributes matrix for the above digraph is prepared as equation 4. attribute auc adu cf lt b = auc adu cf lt [ a1 1 8 4 1 a2 3 6 1 8⁄ 1 3⁄ a3 1 1 4⁄ 1 6⁄ 1 a4 ] (4) step 7the characteristic permanent function for overall performance value can be written as equation 5 (rao, 2007): per(b) = ∏ ai + ∑ ∑ ∑ ∑ (aijaji)akal 4 l=k+1 3 k=1 4 j=i+1 3 i=1 4 i=1 + ∑ ∑ ∑ ∑(aijajkaki+ aikakjaji)al 4 l=1 4 k=j+1 3 j=i+1 2 i=1 + [∑ ∑ ∑ ∑ (aijaji)(aklalk) 4 l=j+2 3 k=i+1 4 j=i+1 1 i=1 + ∑ ∑ ∑ ∑ (aijajkaklali + ailalkakjaji 4 l=j+1 4 k=i+1 3 j=i+1 1 i=1 )] (5) overall performance value is calculated using the values of ai’s and aij’s for each inventory item. ai’s are obtained from the normalized decision matrix and aij’s are obtained from relative importance matrix (table 2). step 8according to graph theory and matrix method, classification of inventory items based on the overall performance value arranged in descending order are shown in table 3. ijahp article: mallick, sarkar, das/a unified decision framework for inventory classification through graph theory international journal of the analytic hierarchy process 192 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.482 in order to obtain comparison, the same distribution of 10 class a, 14 class b and 23 class c items was maintained. inventory items with an overall performance value of 17.20748099 and above were classified as class a items, those with scores of 15.50751142 and below were classified as class c items, while those with scores between 17.20748099 and 15.50751142 were classified as class b items. the abc classification using proposed method as shown in table 3 is compared with the result of traditional abc and the ahp classification methods as reported in flores et al. (2002). table 3 a comparison of abc classification a comparison of abc classification as determined by: item reid (1987) using only a single criterion. flores et al. (1992) using ahp for multiple criteria inventory classification. this paper using the hybrid system i.e. integrated graph theory and ahp for multiple criteria inventory classification. annual dollar usage rank abc ahp (weighted score) rank abc overall performance value rank abc s1 5840.64 1 a 0.59684 7 a 18.98048509 5 a s2 5670 2 a 0.86066 2 a 26.10663828 1 a s3 5037.12 3 a 0.71080 4 a 19.5330094 4 a s4 4769.56 4 a 0.08342 42 c 13.82213779 37 c s5 3478.8 5 a 0.41910 24 b 16.68158993 11 b s6 2936.67 6 a 0.40029 26 c 15.83281893 16 b s7 2820 7 a 0.39728 27 c 15.70537438 19 b s8 2640 8 a 0.26535 37 c 15.1261434 25 c s9 2423.52 9 a 0.82538 3 a 20.43548102 2 a s10 2407.5 10 a 0.50995 13 b 18.72709566 6 a s11 1075.2 11 b 0.50456 17 b 14.91889907 27 c s12 1043.5 12 b 0.50314 18 b 15.78723561 17 b s13 1038 13 b 0.87690 1 a 20.33433735 3 a s14 883.2 14 b 0.53502 12 b 17.36928478 8 a s15 854.4 15 b 0.59480 8 a 16.77468681 10 a s16 810 16 b 0.37207 29 c 14.79540034 29 c s17 703.68 17 b 0.42707 22 b 14.8344986 28 c s18 594 18 b 0.57539 9 a 16.66968694 12 b s19 570 19 b 0.50592 16 b 15.97287062 15 b ijahp article: mallick, sarkar, das/a unified decision framework for inventory classification through graph theory international journal of the analytic hierarchy process 193 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.482 s20 467.6 20 b 0.44018 21 b 15.45752996 22 b s21 463.6 21 b 0.63900 6 a 16.29263253 13 b s22 455 22 b 0.44250 20 b 15.56395933 21 b s23 432.5 23 b 0.66237 5 a 17.51474999 7 a s24 398.4 24 b 0.57302 10 a 15.71356327 18 b s25 370.5 25 c 0.01771 47 c 11.96390314 47 c s26 338.4 26 c 0.15263 40 c 12.98776323 41 c s27 336.12 27 c 0.03521 45 c 12.53370856 43 c s28 313.6 28 c 0.37435 28 c 15.32649641 23 b s29 268.68 29 c 0.46347 19 b 16.83859629 9 a s30 224 30 c 0.02268 46 c 12.13135833 46 c s31 216 31 c 0.50975 14 b 16.17631119 14 b s32 212.08 32 c 0.50937 15 b 15.24071987 24 b s33 197.92 33 c 0.29309 33 c 14.23044817 34 c s34 190.89 34 c 0.41335 25 c 14.59149575 32 c s35 181.8 35 c 0.16044 38 c 13.28125951 39 c s36 163.28 36 c 0.57224 11 b 15.69489064 20 b s37 150 37 c 0.28485 34 c 13.89944772 35 c s38 134.8 38 c 0.37004 30 c 14.7489373 31 c s39 119.2 39 c 0.29574 32 c 14.34183744 33 c s40 103.36 40 c 0.36078 31 c 14.75658732 30 c s41 79.2 41 c 0.07482 44 c 12.14000067 45 c s42 75.4 42 c 0.08164 43 c 12.37195808 44 c s43 59.78 43 c 0.28339 35 c 13.84673594 36 c s44 48.3 44 c 0.15362 39 c 13.0396684 40 c s45 34.4 45 c 0.42138 23 b 14.96412365 26 c s46 28.8 46 c 0.14582 41 c 12.75973946 42 c s47 25.38 47 c 0.27461 36 c 13.49604159 38 c ijahp article: mallick, sarkar, das/a unified decision framework for inventory classification through graph theory international journal of the analytic hierarchy process 194 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.482 table 4 a comparison of annual dollar usage percentage of class a, b and c type of items for integrated graph theory-ahp,traditional abc and the ahp class of items % of items each category annual dollar usage percentage integrated graph theory-ahp (proposed method for multiple criteria inventory classification.) traditional abc (reid (1987) using only a single criterion.) flores et al. (1992) using ahp for multiple criteria inventory classification a 21 54 74 43 b 30 23 19 30 c 49 23 7 27 comparative analysis of annual dollar usage percentage of a, b and c type of items of three types of abc analyses is shown in table 4. table 4 illustrates that 54% of the annual dollar usage for 21% of item (a type) was obtained by the proposed method. in accordance with the traditional abc by reid (1987),where only one criteria has been considered, 30% of item (b type) accounts for 19% of annual dollar usage. in consonance with flores et al. (1992), 49% of items (type c) are responsible for the 27% annual dollar usage. therefore, it can be stated that for any organization, both inventory cost-control as well as multi-criteria decision making can be done more effectively by applying the proposed method from a materials management point of view. the suggested hybrid method envisages the proper controlling of materials (47 items) in comparison to the other two methods. 4. conclusions in the present investigation, the combination of graph theory and the analytic hierarchy process was applied for multi-criteria inventory classification. it may be noted that, as per the author’s purview, this kind of hybrid system has not been used earlier to classify inventory items. a case study on a hospital inventory management system for 47 disposable items used in a respiratory therapy unit was considered. the results obtained using the proposed method were found to be more precise than that of solely the ahp classification method. hence, this hybrid method being a simple, fast, logical, systematic and consistent method, may be recommended for use as a decision making method for multi-criteria inventory classification. it can also be used for any type of decision-making situation. it enables more critical analysis than the common multi-attribute decision making methods since any number of quantitative and qualitative attributes can be considered. in the ijahp article: mallick, sarkar, das/a unified decision framework for inventory classification through graph theory international journal of the analytic hierarchy process 195 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.482 permanent procedure, even a small variation in the attribute leads to a significant difference in the overall performance value making it convenient to rank the alternatives in a descending order with clear difference in the overall performance value. moreover, the proposed procedure using the graphical representation not only enables the analysis of the alternatives, but also facilitates visualization of various criteria along with their interrelations. the suggested hybrid method envisages the proper controlling of materials in comparison with traditional abc and the ahp methods. the present paper considers the decision taken under certainty, which is otherwise often highly uncertain and risky for the decision-makers. therefore, this hybrid method may elevate applicability considering uncertainty and vagueness in attribute values. ijahp article: mallick, sarkar, das/a unified decision framework for inventory classification through graph theory international journal of the analytic hierarchy process 196 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.482 references bhattacharya, a., sarkar, b., & mukherjee, s. k. (2007). distance-based consensus method for abc analysis. international journal of production research, 45(15), 3405– 3420. doi: https://doi.org/10.1080/00207540600847145 braglia, m., grassi, a., & montanari, r. (2004). multi-attribute classification method for spare parts inventory management. journal of quality in maintenance engineering, 10(1), 55–65. doi: https://doi.org/10.1108/13552510410526875 cakir, o., & canbolat, m. s. (2008). a web-based decision support system for multicriteria inventory classification using fuzzy ahp methodology. expert systems with applications, 35(3), 1367–1378. doi: https://doi.org/10.1016/j.eswa.2007.08.041 chakladar, n. das, das, r., & chakraborty, s. (2009). a digraph-based expert system for non-traditional machining processes selection. international journal of advanced manufacturing technology, 43(3–4), 226–237. doi: https://doi.org/10.1007/s00170-0081713-0 cohen, m. a., & ernst, r. (1988). multi-item classification and generic inventory stock control policies. production and inventory management journal, 29(3), 6–8. doi: https://doi.org/10.1017/cbo9781107415324.004 darvish, m., yasaei, m., & saeedi, a. (2009). application of the graph theory and matrix methods to contractor ranking. international journal of project management, 27(6), 610–619. doi: https://doi.org/10.1016/j.ijproman.2008.10.004 faisal, m. n., banwet, d. k., & shankar, r. (2007). quantification of risk mitigation environment of supply chains using graph theory and matrix methods. european journal of industrial engineering, 1(1), 22–39. doi: https://doi.org/10.1504/ejie.2007.012652 flores, b. e., olson, d. l., & dorai, v. k. (1992). management of multicriteria inventory classification. mathematical and computer modelling, 16(12), 71–82. doi: https://doi.org/10.1016/0895-7177 (92)90021-c flores, b. e., & whybark, d. c. (1986). multiple criteria abc analysis. international journal of operations & production management, 6(3), 38–46. doi: http://dx.doi.org/10.1108/eb054765 flores, b. e., & whybark, d. c. (1987). implementing multiple criteria abc analysis. journal of operations management, 7(1–2), 79–85. https://doi.org/10.1016/02726963(87)90008-8. doi: 10.1016/0272-6963(87)90008-8 ghorabaee, m. k. (2015). multi-criteria inventory classification using a new method of evaluation based on distance from average solution (edas). informatica (netherlands), 26(3), 435–451. doi: https://doi.org/10.15388/informatica.2015.57 grover, s., agrawal, v. p., & khan, i. (2004). a digraph approach to tqm evaluation of ijahp article: mallick, sarkar, das/a unified decision framework for inventory classification through graph theory international journal of the analytic hierarchy process 197 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.482 an industry. international journal of production research, 42(19), 4031–4053. doi: https://doi.org/10.1080/00207540410001704032 guvenir, h., & erel, e. (1998). multicriteria inventory classification using a genetic algorithm. european journal of operational research, 105(1), 29–37. doi: https://doi.org/10.1016/s0377-2217(97)00039-8 kabir, g., & hasin, m.a.a. (2013). multi-criteria inventory classification through integration of fuzzy analytic hierarchy process and artificial neural network. international journal and systems engineering, 14(1), 74–103. doi: https://doi.org/10.1504/ijise.2013.052922 lanjewar p. b., rao r. v., kale a. v., taler j. and ocłoń p(2016). evaluation and selection of energy technologies using an integrated graph theory and analytic hierarchy process methods. decision science letters, 5, 327–348 liu, j., liao, x., zhao, w., & yang, n. (2016). a classification approach based on the outranking model for multiple criteria abc analysis. omega, 61, 19–34. doi: https://doi.org/10.1016/j.omega.2015.07.004 mallick, b., dutta, o. n., & das, s. (2012). a case study on inventory management using selective control techniques. journal of the association of engineers, india, 82(1&2), 10–24. doi:10.22485/jaei/2012/v82/i1-2/119950 mallick, b., sarkar, b., & das, s. (2016). application of the moora method for multicriteria inventory classification. in proceeding of the international conference on frontiers in optimization: theory and applications, kolkata (p. 40). millet, i. & schoner, b., (2005). incorporating negative values into the analytic hierarchy process, computers & operations research, 32(12), 3163–3173. paramasivam, v., senthil, v., & rajam ramasamy, n. (2011). decision making in equipment selection: an integrated approach with digraph and matrix approach, ahp and anp. international journal of advanced manufacturing technology, 54(9–12), 1233– 1244. doi: https://doi.org/10.1007/s00170-010-2997-4 rao r. v. (2007). decision making in the manufacturing environment: using graph theory and fuzzy multiple attribute decision making methods. (pp. 7–25). springer science & business media. doi: 10.1007/978-1-4471-4375-8 ramanathan, r. (2006). abc inventory classification with multiple-criteria using weighted linear optimization. computers and operations research, 33(3), 695–700. doi: https://doi.org/10.1016/j.cor.2004.07.014 rao, r. v., & parnichkun, m. (2009). flexible manufacturing system selection using a combinatorial mathematics-based decision-making method. international journal of production research, 47(24),6981-6998. ijahp article: mallick, sarkar, das/a unified decision framework for inventory classification through graph theory international journal of the analytic hierarchy process 198 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.482 reid, r. a. (1987). the abc method in hospital inventory management: a practical approach. production and inventory management journal, 28(4), 67–70. saaty, t. l. (1980). the analytic hierarchy process. mcgraw-hill inc. doi: https://doi.org/0070543712 singh, d., & rao, r. (2011). a hybrid multiple attribute decision making method for solving problems of industrial environment. international journal of industrial engineering computations, 2(3), 631-644. soylu, b., & akyol, b. (2014). multi-criteria inventory classification with reference items.computers & industrial engineering, 69, 12–20. doi: https://doi.org/10.1016/j.cie.2013.12.011 torabi, s. a., hatefi, s. m., & pay, b. s. (2012). abc inventory classification in the presence of both quantitative and qualitative criteria. computers and industrial engineering, 63(2), 530–537. doi: https://doi.org/10.1016/j.cie.2012.04.011 wagner, s. m., & neshat, n. (2010). assessing the vulnerability of supply chains using graph theory. international journal of production economics, 126(1), 121–129. doi: https://doi.org/10.1016/j.ijpe.2009.10.007 zhou, p., & fan, l. (2007). short communication a note on multi-criteria abc inventory classification using weighted linear optimization. european journal of operational research, 182(3), 1488–1491. doi: https://doi.org/10.1016/j.ejor.2006.08.052 ijahp article: oyatoye, adebiyi, amole/evaluation subscribers preference for service attributes of mobile telecommunications in nigeria using analytic hierarchy process (ahp) international journal of the analytic hierarchy process 171 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.299 evaluating subscribers preference for service attributes of mobile telecommunications in nigeria using analytic hierarchy process (ahp) emmanuel olateju oyatoye department of business administration university of lagos, nigeria e-mail: eoyatoye@unilag.edu.ng sulaimon olanrewaju adebiyi department of business administration federal university of agriculture abeokuta (funaab) e-mail: lanre18april@gmail.com bilqis bolanle amole distance learning institute department of business administration university of lagos, nigeria e-mail:amolebb@yahoo.com abstract policy and strategies in the growing nigerian telecommunications industry can only have significant impact if they are substantially driven by research on what subscribers prefer and why they prefer it? thus, there is a need for an operations research model (ahp) to evaluate customer preferences for their mobile telecommunications attributes in order to direct policy and strategies toward what is important to subscribers. this paper built a hierarchical model for choice/determinant of subscriber’s preferences for mobile telecommunications attributes in nigeria using seven main attributes as the criteria for evaluation. the four main players in the global system for mobile communication (gsm) market (mtn, airtel, glo and etisalat) are the alternatives. an ahp based questionnaire was administered among students of tertiary institutions in lagos. out of the four hundred questionnaires distributed, three hundred and eighty six were completed, returned and found suitable for the analysis. the data was analysed, considering the set of evaluation criteria (service attributes), and a set of alternative (network providers) scenarios from which the best decision was to be made. we generated a weight for each evaluation criterion and scenario according to the information provided by the decision makers (stakeholders). ahp was used to combine the objective and scenario evaluations to determine the ranking for scenarios. the results revealed that an average student preferred network providers with low rates (affordable), followed by quality of connections and a reliable data plan for internet service. a priority was done for the criteria to direct strategic decisions in the telecommunication industry towards meeting subscriber’s needs. coverage was not given significant priority as the respondents assumed that all providers have similar coverage. the results of this study will help improve the quality of decision making by stakeholders. keywords: ahp; subscribers; preference; service attributes’; gsm; telecommunication mailto:lanre18april@gmail.com mailto:amolebb@yahoo.com ijahp article: oyatoye, adebiyi, amole/evaluation subscribers preference for service attributes of mobile telecommunications in nigeria using analytic hierarchy process (ahp) international journal of the analytic hierarchy process 172 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.299 1. introduction global system for mobile communication (gsm) in nigeria was introduced thirteen years ago and unlike when it was first introduced there is now an increasing need for mobile phone services by the average nigerian. besides, the subscriber identification module (sims) now goes almost for free to subscribers/customers as there is no disparity in the price it is sold to high, medium or low profile citizen as it was done when it was launched in 2001. subscribers who have an increasing need for the use of a variety of mobile telecommunication attributes are faced with a great deal of complexity in their decision making. they must not only decide which of the network providers to choose, but also decide which of the network providers is best suited to meet their economic, social and psychological needs. the name analytic hierarchy process explains its application logic (silva, alver, and marins, 2009). analytic means that it assists in the measurement and synthesizing of a series of factors involved in complex decisions. hierarchy focuses on that fact that it is the adaptable way of finite intelligence to assume a complex situation. finally, process is a series of actions, alterations or functions that leads to an end or result. the continuous marketing efforts of network providers in nigeria as well as the perceived ineffectiveness of the regulatory authority, national communication commission(ncc), in the telecommunication sector has made the evaluation of subscriber preferences for service attributes necessary. using the ahp will produce results that are beneficial to all the stakeholders, as the subscribers will be provided with a rational way for decision making that requires comparisons, priority and tradeoffs. this powerful multi-criteria tool will allow the operator to understand what is important to subscribers among the criteria and in what order, so as to direct marketing effort/activities as well as strategies to enhance customer satisfaction. also, the regulator (ncc) will benefit by understanding the needs and complaints of subscribers so as to balance the interest in their policies and also to assess how subscribers perceive the regulators. this study is very important for the policy makers, subscribers, telecom operators and the growth of the nigerian telecommunication industry since organizations in the 21 st century can only survive the turbulent business environment when the needs and wants of the consumers/subscribers are identified and satisfied. thus, the study helps evaluate the preferences of subscribers of the mobile services attribute in nigeria, especially in relation to their day-to-day experiences using gsm. 2. literature review 2.1 application of ahp ahp is one of the most popular and powerful methods for decision-making and was proposed by saaty (1977, 1980). since its inception, the ahp has been used by many individuals and groups in various fields because of its user-friendly interface for multi-criteria decision-making (vargas, 1990). in the ahp process, data on decision maker’s judgments, called pairwise comparisons, are aggregated and the degree of importance of each alternative is quantified for each decision-maker. this procedure ijahp article: oyatoye, adebiyi, amole/evaluation subscribers preference for service attributes of mobile telecommunications in nigeria using analytic hierarchy process (ahp) international journal of the analytic hierarchy process 173 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.299 identifies not only the most important alternative, but also the preference for all alternatives for each decision-maker (crawford and williams, 1985). ahp philosophy is based on the intention to provide a comprehensive and rational framework for structuring a decision problem, for representing and quantifying its elements, for relating those elements to overall goals, and for evaluating alternative solutions. the method is based on mathematics and psychology. also, as a ‘normative’ model of decision making, as opposed to a ‘descriptive’ model which allows describing the way a decision maker (dm) actually makes decision, a normative model enables a dm to defend his choice over competing alternatives. presenting and clarifying the essential concepts of ahp for the decision making process, saaty (2008) submits that the measurement of concepts changes from one situation to another, and that they depend on a value system which varies from one person to another. thus, their importance cannot be measured once and for all, but must be determined in terms of individual values. the most appropriate way to measure such attributes is by comparing their relative importance with respect to a higher goal. from the comparisons, one derives a scale of priorities which are in relative values. judgments come first, priorities second. the ahp method refers to ‘alternatives’, ‘attributes’ and ‘goals’. the term ‘alternative’ is synonymous with ‘option’, ‘policy’, ‘method’ etc. the term ‘attributes’ may be referred to as ‘objectives’ or ‘factors’ or ‘criteria’. in any problem scenario multiple ‘attributes’ exist, and in the problem-solving process a dm generates relevant attributes for each setting. keeney and raffia (1976) suggested the use of a literature survey and /or a panel of experts to identify the attributes in the problem area. yoon and hwang (1995) stressed the necessity that attributes represent the desired mission and suggested one way to derive the attributes hierarchically from a super goal; goal hierarchy formulation starts with the listing of overall performance objectives serving a super goal. they noted that this hierarchy should consist of at least three levels: focus or overall goal at the top, multiple criteria that define alternatives in the middle, and competing alternatives at the bottom. therefore, by applying the ahp to survey research questionnaires, respondent’s perceptions can be clarified more precisely than by traditional methods (sato, 2001). thus, it would be more appropriate to not limit the study of subscribers/customer preferences for attributes to conjoint analysis (see, oyatoye, adebiyi and amole, 2013a; oyatoye, adebiyi and amole, 2013b), but for the customers/subscribers, who are the essence of telecommunication industry, to express their perceptions of the concept of mobile network attributes and their choice of a particular network provider through a pairwise comparison. 2.2 conceptual model for the study in this study, the mobile network attributes were chosen based on literature and a preliminary interview with some subscribers. the interview aimed at identifying those attributes that subscribers of mobile telephones valued most. this allowed for proper grouping of the attributes identified both from the literature and from the preliminary subscriber survey. attributes identified include quality of calls, tariff, coverage, promotion, message delivery and complaint management. an average subscriber of a mobile phone needs to be satisfied with all of these attributes to a ijahp article: oyatoye, adebiyi, amole/evaluation subscribers preference for service attributes of mobile telecommunications in nigeria using analytic hierarchy process (ahp) international journal of the analytic hierarchy process 174 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.299 reasonable extent in order for him/her to continue to patronize the network providers. this is important since the mnp allows him/her to switch providers and still maintain the same number. besides, when the service providers know the value that subscribers place on each attribute comparatively then they can implement effective marketing strategies to keep the profitable customers and persuade potential subscribers to join their network. the ahp methodology involves decomposition of the decision problem into a set of variables that are organized into a hierarchy, and enables decision makers to compare and make choices among a number of alternatives. it allows for criteria priorities and making a series of trade-offs among the criteria in relation to goals as well as the alternatives. the identification of the objective and the criteria were required to implement ahp for this study. the ahp model adopts the use of a diagram in the form of a hierarchy to model real-life situations (saaty, 1980). the four levels of the model for this study were as follows: first level: the objective which is the evaluation of subscribers’ preferences for service attributes of mobile telecommunications in nigeria. second level: service attributes of mobile telecommunications in nigeria which include quality of calls, tariff, coverage, data plan, promotion, message delivery and complaint management. third level: the components of each service attribute of the mobile network provider in nigeria. the sub-criteria to quality of calls include call clarity, call drop and connectivity. the sub-criteria of tariff include call/cost, sms/cost and internet services/cost. the sub-criteria to coverage are wide, cities concentrated, and services while travelling. the sub-criteria to internet data plan include sufficient data with high costs, insufficient data plan with low cost, and sufficient data plan but not effective. the sub-criteria to promotional activities of the mobile phone services providers in nigeria include free data services, free sms and/or free calls. under the message delivery criteria, the sub-criteria are prompt delivery, slow delivery, and poor delivery. the complaint management sub-criteria are response on time, respond late but effective and network providers’ agent do not respond. fourth level: the alternatives considered for this study were the following mobile telecommunication network providers in nigeria: mtn, airtel, glo and etisalat. these were in the market as the gsm carriers at the time of this study. the ahp model for this study is shown in figure 1. ijahp article: oyatoye, adebiyi, amole/evaluation subscribers preference for service attributes of mobile telecommunications in nigeria using analytic hierarchy process (ahp) international journal of the analytic hierarchy process 175 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.299 figure 1. the ahp model mtn = mobile telecommunication network glo = global communication network etisalat = emirates telecommunications corporation preference for mobile network attributes in nigeria message delivery promotion coverage internet data plan complaint management tariff quality of calls call clarity call/per cost insufficient data with low cost free calls prompt delivery respond on time wide call drop sms/ cost slow delivery cities internet services /cost sufficient data with high cost bonus sms respond late but effective service out while traveling connectivity free call poor delivery sufficient data but not effective don’t respond etisalat mtn airtel glo ijahp article: oyatoye, adebiyi, amole/evaluation subscribers preference for service attributes of mobile telecommunications in nigeria using analytic hierarchy process (ahp) international journal of the analytic hierarchy process 176 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.299 3. research methodology this study was quantitative research which employed the application of ahp for evaluating subscriber’s preference for mobile network attributes in nigeria. the analytic hierarchy process (ahp) is a mathematically based, multi-objective decision making tool which was introduced by saaty (1990). it uses the pair wise comparison method to rank order alternatives of a problem that are formulated and solved in a hierarchical structure. table 1 questionnaire distribution gsm operator sample using the network percentage (%) mtn 253 27.38 airtel 246 26.62 glo 219 23.70 etisalat 206 22.30 total 924 100 in this paper, a convenient sampling method was used to select four hundred subscribers of mobile network providers to complete the ahp based questionnaires. this sampling method was adopted since not all students and teaching staff could be reached at the time of questionnaire distribution. three hundred and eighty six of the four hundred questionnaires distributed were appropriately completed and returned, thus, evaluation was based on the correctly completed questionnaires. from table 1, it is evident that many of the respondents were patronizing multiple network providers, as the total subscriber identification module (sim) of the 386 respondents increased to 924. the sample was drawn to include subscribers of the four active mobile telecommunication services providers in nigeria. these are mtn, aitel, glo and etisalat subscribers who completed the ahp based questionnaire about their experiences concerning the attributes listed with network service providers for comparisons in relation to the goal and alternatives. the questionnaires were distributed to people on the campuses of the three higher institutions of learning [university of lagos, akoka, lagos, nigeria (unilag); yaba college of technology, yaba lagos (yabatech); federal college of education technical, akoka, lagos, nigeria (fcetech)]. this group was comprised of students, staff (teaching and non-teaching), and visitors to the campuses, as well as the business people who could read and understand the questions. in order to get a prompt response, surveys were distributed in places where people could take the time to complete the questionnaire, such as a lecture room for students after lectures, offices of lecturers and non-teaching staff, the canteen/cafeteria, and relaxation centres to mention a few. 27.38 % of respondents reported patronizing mtn as their network operator. airtel followed with 26.62% of respondents as subscribers, and glo mobile and etisalat followed with 23.70% and 22.30%, respectively. the respondents combined the network sims in various proportions as the researchers carefully sorted them out to enable the computation of the percentages shown in table 1. the data retrieved were analysed using the ibm spss version 2.0 and expert choice 2000 software. ijahp article: oyatoye, adebiyi, amole/evaluation subscribers preference for service attributes of mobile telecommunications in nigeria using analytic hierarchy process (ahp) international journal of the analytic hierarchy process 177 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.299 4. data presentation, analysis and interpretations table 2 frequency distribution of respondents by demographic status variables frequency percentage (%) gender male 202 52.3 female 184 47.7 total 386 100 age below 21yrs 172 44.5 21-30yrs 128 33.2 31-40yrs 49 12.7 41-50yrs 32 8.3 51 and above 5 1.3 total 386 100 marital status single 283 73.3 married 98 25.4 divorced 0 0 widowed 2 1.2 total 386 100 educational status secondary 203 52.6 nce/nd 81 20.9 b. sc 61 15.8 mba/m. sc 14 3.7 ph. d 5 1.3 others 22 5.7 total 386 100 occupational status teaching 36 9.3 non-teaching 52 13.5 student 271 70.2 others 27 6.9 total 386 100 income per month below 89 999 271 70.2 90 000-199 999 78 20.2 200 000-399 999 20 5.2 400 000-599 999 11 2.8 600 000-above 6 1.6 total 386 100 nature of sim prepaid (pay as you go) 303 78.5 postpaid 21 5.4 both 62 16.1 total 386 100 source: field survey, 2014 ijahp article: oyatoye, adebiyi, amole/evaluation subscribers preference for service attributes of mobile telecommunications in nigeria using analytic hierarchy process (ahp) international journal of the analytic hierarchy process 178 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.299 the results of the study show that 202 respondents were male while 184 were female respondents, representing 52.3% and 47.7%, respectively. age distribution of respondents reveals that 172 (44.5%) were below 21 years, 128 (33.2%) were between the ages of 21 to 30 years, 49 respondents representing 12.7% were in the age grade 3140 , 32 (8.3%) were between the ages of 41-50 years, and 5 (1.3%) were above the golden age (50 years and above). concerning the marital status of the respondents, 283 (73.3%) of the total respondents were single, and 98 (25.4%) were married at the time of this study. only 2 (1.2%) were widowed, and none reported being divorced. the highest qualification held by a majority of our respondents at the time of this study is a secondary school certificate (wassce/gce) (52.6%), followed by people with a national diploma (nd or nce) (20.9%). most of the respondents were currently acquiring higher education in the three institutions of higher learning used as case study. twenty two (22) respondents, representing 5.7%, held professional qualifications, while the remaining 21.3% included both first and higher degree holders. thus, with the level of education of our respondents, they were appropriate to fill the ahp based questionnaires. many of the respondents reported being students, with 271 (70.2%) choosing student as their occupation. this category consisted of college of education students, national and higher national diploma students, undergraduates as well as postgraduate students. the remainder was made up of 36 (9.3%) teaching staff, 52 (13.5%) non-teaching staff, as well as 27 (6.9%) others. the majority of respondents claimed that they had an income less than 90, 000 naira per month with 70.2% (271) falling into this category. 29.8% of the respondents earned above 90, 000 naira per month. many of the respondents claimed to be using prepaid sim, while some used post-paid, and others combined both. 4.1 consistency index (ci) the consistency ratios of most of the pairwise comparison matrices were less than 0.1; hence, judgments of the respondents were assumed to be consistent, and therefore acceptable. those that were inconsistent were revised through revised judgment procedures, which allows for the removal of values that are at the extreme and their replacement with appropriate values as suggested by the system from one analysis to another. 4.2 composite priorities the analytical hierarchy process model used for this study has four levels: the goal, the criteria, the sub-criteria and the alternatives. the priorities for the customer’s preference of the network attributes that determines their selection in sim of network providers/operators in the nigerian telecommunications industry are presented in the tables that follow. ijahp article: oyatoye, adebiyi, amole/evaluation subscribers preference for service attributes of mobile telecommunications in nigeria using analytic hierarchy process (ahp) international journal of the analytic hierarchy process 179 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.299 table 2 composite priorities of the criteria with regard to subscriber’s preference for mobile network attributes in the selection of a network provider goal: evaluating preference for service attributes quality of calls (qoc) tariff (t) coverage (c) data plan (dp) promotion (p) message delivery (md) complaint management (cm) pooled average composite priority 0.192 0.291 0.078 0.160 0.116 0.085 0.078 relative preference ranking 2 1 6.5 3 4 5 6.5 the criteria with respect to the goal of evaluating subscribers preference for mobile network attributes (which ultimately determines which network subscribers decide to patronize) revealed that an average customer preferred a good/cheap tariff system before considering the quality of calls. this follows economic theory which states that price will determine demand for goods and services. it is the willingness to buy, and the ability to pay that makes an effective demand. subscribers also preferred a good data plan followed by frequent promotions/bonuses from their service providers, and good message delivery in that order. both coverage and complaint management had equal priority. thus, more subscribers will patronize network providers that have a considerate tariff in terms of calls, sms and data plan. any network operator that wants to attract more subscribers should focus on having a considerate tariff, closely followed by quality calls, a good data plan, ensure effective promotional mix, and good message delivery. coverage and complaint management is of little importance to susbcribers compared to the five previously mentioned attributes. this can serve as a policy guide to service providers, as it will assist in performing the first basic function of marketing which is customer need and want identification. it will also help to redirect the marketing efforts of a service provider towards meeting the needs and want of subscribers. the priority will also form the basis for the regulatory agency to effectively monitor the attributes that are of greatest importance to subscribers, as well as to determine the order in which to address the attributes. table 3 composite priorities of the sub-criteria with regards to subscriber’s preferences for quality of calls (qos) of mobile network provider sub-criteria under quality of calls calls clarity (cc) call drop (cp) ease of connectivity (con) pooled average composite priority 0.591 0.097 0.312 relative preference ranking 1 3 2 ijahp article: oyatoye, adebiyi, amole/evaluation subscribers preference for service attributes of mobile telecommunications in nigeria using analytic hierarchy process (ahp) international journal of the analytic hierarchy process 180 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.299 from table 3, the pooled average composite priority of sub-criteria under quality of calls (qos) revealed that subscribers preferred call clarity over ease of connectivity and call drop. thus, mobile service operators should focus on clarity of calls first, closely followed by ease of connectivity, while minimising call drop. table 4 composite priorities of the sub-criteria with regards to subscriber’s preference for cheapness of tariff (t) of mobile network provider sub-criteria under tariff cost per call (cpc) sms per cost (spc) internet services per cost (ipc) pooled average composite priority 0.486 0.172 0.342 relative preference ranking 1 3 2 table 4 reveals that subscribers preferred cheap call rate to cheap internet facility as a sub-criteria of tariff. cost per call was mostly preferred, followed by internet services per cost and later sms per cost. service providers should ensure that the cost of calls is the cheapest option, as most subscribers in the low income group only use gsm to make calls. this is followed by the cost of internet facility and sms, respectively. this suggests that subscribers preferred a cheap internet facility (an environment that is mostly academic based) to cheap sms, but preferred cheap calls the most. table 5 composite priorities of the sub-criteria with regards to subscriber’s preference for coverage (c) of mobile network provider sub-criteria under coverage wide (w) cities concentrated (cicon) services out while on travelling (sowt) pooled average composite priority 0.6348 0.3141 0.0511 relative preference ranking 1 2 3 table 5 shows that subscribers believe that wide coverage is good for their choice network. this is followed by cities concentrated network, and the least preferred which is a network provider that does not have coverage while the subscriber is traveling. thus, wide coverage is important to customers in their choice of network. table 6 composite priorities of the sub-criteria with regards to subscriber’s preference for data plan (dp) of mobile network provider sub-criteria under data plan sufficient data with high costs (sdwc) insufficient data plan with low cost (idpwlc) sufficient data plan but not effective (sdpbne) pooled average composite priority 0.4315 0.5421 0.0264 relative preference ranking 2 1 3 ijahp article: oyatoye, adebiyi, amole/evaluation subscribers preference for service attributes of mobile telecommunications in nigeria using analytic hierarchy process (ahp) international journal of the analytic hierarchy process 181 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.299 table 6 displays the results of the pooled average composite priority and relative preference ranking of sub-criteria with regards to subscriber’s preference for data plan as criteria for evaluating network attributes for determining which service providers to patronize. the results revealed that subscribers believe that an insufficient data plan with low cost was the most preferred among the sub-criteria of data plan, as sufficiency of the data plan ranked below this. subscribers preferred a data plan that was insufficient with low cost, followed by a sufficient data plan with high cost while they least preferred a sufficient data plan that was not effective or productive (as this will consume time and effort without achieving what the users want to achieve). table 7 composite priorities of the sub-criteria with regards to subscriber’s preference for promotion (p) of mobile network provider sub-criteria under promotion free data services (fds) free sms (fsms) free calls (fc) pooled average composite priority 0.4913 0.0515 0.4572 relative preference ranking 1 3 2 the above table shows subscribers preference for sub-criteria under promotion. the results revealed that subscribers preferred free data services more than free calls and free sms. subscribers preferred network providers providing more data services the most, followed by free calls, while the least preferred is free sms. thus, promotional services that give a data plan were preferred to free calls and sms. this may be due to the fact that the majority of the respondents involved in the study were students who use internet services for many things such as voice/video calls, whatsapp, bbm, chat on, google plus, email, and so on. table 8 composite priorities of the sub-criteria with regards to subscriber’s preference for message delivery (md) of mobile network provider prompt delivery (pd) slow delivery (sd) poor delivery (pd) pooled average composite priority 0.8966 0.0856 0.0178 relative preference ranking 1 2 3 table 8 shows the results of subscriber’s preference for the sub-criteria of message delivery in selecting a network of choice. it is evident from the results that subscribers preferred prompt delivery of short service message most, followed by slow delivery while they least preferred the poor delivery. therefore, subscribers will always patronize a service provider that gets their messages across promptly over those that deliver late or have poor delivery. ijahp article: oyatoye, adebiyi, amole/evaluation subscribers preference for service attributes of mobile telecommunications in nigeria using analytic hierarchy process (ahp) international journal of the analytic hierarchy process 182 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.299 table 9 composite priorities of the sub-criteria with regards to subscribers preference for complaint management (cm) of mobile network provider response on time (rot) respond late but effective (rlbe) network providers’ agent do not respond (dnr) pooled average composite priority 0.4821 0.4821 0.0358 relative preference ranking 1.5 1.5 3 observations from table 9 show that with regards to complaint management by a network provider, subscribers preferred timely responses and late but effective equally while they least preferred the network provider or agent not responding to their calls. this indicates that the time it takes to answer a call is irrelevant to subscribers as long as the response is effective. that is to say, subscribers prefer to achieve their purpose by getting a good response to a complaint or clarification of a question. table 10 composite priorities of the alternatives with regard to subscriber’s preference for quality of calls mtn airtel glo etisalat pooled average composite priority 0.2716 0.2566 0.2566 0.2152 relative preference ranking 1 2.5 2.5 4 the pooled composite priority and relative preference ranking of alternatives to quality of calls revealed that from the experiences of services provided to subscribers as at the time of this study, mtn is mostly preferred , followed by airtel and glo who are preferred equally, while etisalat is the least preferred with respect to quality of calls. table 11 composite priorities of the alternatives with regard to subscriber’s preference for cheap tariff mtn airtel glo etisalat pooled average composite priority 0.1568 0.4704 0.1299 0.2429 relative preference ranking 3 1 4 2 ijahp article: oyatoye, adebiyi, amole/evaluation subscribers preference for service attributes of mobile telecommunications in nigeria using analytic hierarchy process (ahp) international journal of the analytic hierarchy process 183 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.299 table 11 shows the pooled composite priority and relative preference ranking of alternatives to cheapness of tariff. it reveals that from experiences of services provided to subscribers at the time of the study, airtel is the cheapest and most preferred, followed by etisalat, mtn and glo. table 12 composite priorities of the alternatives with regard to subscriber’s preference for network coverage mtn airtel glo etisalat pooled average composite priority 0.3931 0.2484 0.2761 0.0824 relative preference ranking 1 3 2 4 table 12 shows the relative preference ranking of subscribers among the players (operators) in the gsm market with respect to coverage. it reveals that subscribers believe that if coverage is their basis for decision making, their utmost preference is mtn with the widest coverage. glo is the second most preferred followed by airtel and then etisalat. table 13 composite priorities of the alternatives with regard to subscriber’s preference for internet data plan mtn airtel glo etisalat pooled average composite priority 0.2457 0.2908 0.2310 0.2325 relative preference ranking 2 1 4 3 table 13 contains the relative preference ranking of subscribers, among the players (operators) in gsm market, with respect to data plan (internet facilities). it reveals that subscribers believe that if internet services are the basis for decision making, their utmost preference is for airtel followed by mtn. next etisalat was preferred, and glo was the least preferred with respect to data plan. table 14 composite priorities of the alternatives with regard to subscribers’ preference for frequent promotion mtn airtel glo etisalat pooled average composite priority 0.2177 0.2345 0.3133 0.2345 relative preference ranking 4 2.5 1 2.5 ijahp article: oyatoye, adebiyi, amole/evaluation subscribers preference for service attributes of mobile telecommunications in nigeria using analytic hierarchy process (ahp) international journal of the analytic hierarchy process 184 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.299 observations from table 14 show that with regards to frequency of promotional activities of the network provider, subscribers believe that glo is the most preferred, followed by airtel and etisalat who were equally preferred, while mtn is the least preferred. table 15 composite priorities of the alternatives with regard to subscriber’s preference for message delivery mtn airtel glo etisalat pooled average composite priority 0.2428 0.2571 0.2344 0.2657 relative preference ranking 3 2 4 1 for preference of alternatives with respect to message delivery, subscribers preferred etisalat as the most, followed by airtel and mtn, while glo was least preferred. thus, if a decision on a network provider was based on message delivery, subscribers would prefer etisalat, before airtel, mtn and glo. table 16 composite priorities of the alternatives with regard to subscriber’s preference for subscriber complaint management mtn airtel glo etisalat pooled average composite priority 0.2116 0.2133 0.2669 0.3082 relative preference ranking 4 3 2 1 among the four network providers currently in the gsm market in nigeria, subscribers believed that etisalat was the most preferred in the resolution of complaints or issues. this is followed by glo, airtel and mtn in that order. this is an indication that the least preferred network providers with respect to complaint management need to have more effective customer care centres. 5. conclusion and recommendations this paper illustrated the application of the ahp methodology in the evaluation of subscriber’s preferences for mobile telecommunications network attributes in nigeria, as well as the levels of services provided under each criteria and sub-criteria in a pairwise comparison on the basis of saaty’s 9 point scale. from the data collected and analysed, mobile telecommunications attributes identified were: quality of calls, tariff, coverage, data plan, promotion, message delivery and complaint management. ijahp article: oyatoye, adebiyi, amole/evaluation subscribers preference for service attributes of mobile telecommunications in nigeria using analytic hierarchy process (ahp) international journal of the analytic hierarchy process 185 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.299 most of the subscribers preferred a quality network with low tariffs, a sufficient data plan that are more effective than the network provider having wider coverage. thus ahp methodology helps to evaluate customer preferences for network attributes as well as preferences that subscribers have for the service providers with respect to each of the criteria. these criteria will ultimately inform their selection of a network provider. the results are a guide to stakeholders so as to ensure sustainable development in the nigerian telecommunications industry. the study provides a basis for directing marketing efforts of the service providers towards meeting the telecommunications needs and wants of subscribers so as to reap better profits and retain profitable subscribers. this study further corroborates saaty’s position (1994) that the analytical hierarchy process (ahp) is the most recognized method of preference elicitation and the selfexplicated approach (sattler and hensel-börner, 2000). it is the easiest method to apply since it directly reverts to utility values and importance ratios and thereby increases the cognitive load of dealing with the different forms of uncertainty in more abstract terms (hoeffler, 2003). based on the results of this study, the following recommendations are made: service providers should improve service delivery on attribute(s) where they are least preferred by subscribers. marketing efforts should be geared towards the criteria in order to achieve the goal, operators should focus on criteria in the following order: tariff, then quality of calls, then data plan, then promotion, followed by message delivery while complaint management and coverage could be vice versa. the regulatory agency should evaluate quality of service delivery based on needs indicated by subscribers, thus helping service providers to give what subscribers’ want and not what they can give. ijahp article: oyatoye, adebiyi, amole/evaluation subscribers preference for service attributes of mobile telecommunications in nigeria using analytic hierarchy process (ahp) international journal of the analytic hierarchy process 186 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.299 references crawford, g., williams, c., (1985). a note on the analysis of subjective judgment matrices. journal of mathematical psychology, 29, 387–405. hoeffler, s. (2003). measuring preferences for really new products. journal of marketing research, 40, 4, 406-420. keeny, r.l. and raiffa, h (1976). decisions with multiple objectives. new york: wiley. oyatoye e.o, adebiyi s.o and b. b, amole (2013a). an empirical study on consumers preference for mobile telecommunication attributes in nigeria, british journal of economics, management & trade, 3(4), 419-428. oyatoye e.o, adebiyi s.o and b. b, amole (2013b). an application of conjoint analysis to consumer preference for beverage products in nigeria, acta universitatis danubius, oeconomica, 9(6), 43-56. saaty, t.l., (1977). a scaling method for priorities in hierarchical structures. journal of mathematical psychology, 15, 234–281. saaty, t.l. (1980). analytical hierarchy process planning, priority setting, resource allocation. new york: mcgraw-hill. saaty, t. l. (1990). an exposition of the ahp in reply to the paper remarks on the analytic hierarchy process. management science, 36(3), 259-268. saaty, t. l. (1994). how to make a decision: the analytic hierarchy process. interfaces, 24(6), 19-43. saaty, t.l. (2008). relative measurement and its generalization in decision making why pairwise comparisons are central in mathematics for the measurement of intangible factors; the analytic hierarchy/network process, racsam, 102(2), 251318. sattler, h. and hensel-börner, s. a (2000). comparison of conjoint measurement with self-explicated approaches. in gustafsson, a., herrmann, a., huber, f. (eds.), conjoint measurement: methods and applications. berlin: springer, 121133. sato, y., (2001). the impact on scaling on the pair-wise comparison of the analytic hierarchy process. the proceeding on the sixth international symposium on the analytic hierarchy process, 421–430. silva, h.a., alver, l.h.d. and marins, f.a.s. (2009). using ahp to evaluate the performance of the quality, environment, occupational health and safety management systems, proceedings of the 10th international symposium on the analytic hierarchy/network process multi-criteria decision making, july 29 august 1 2009, university of pittsburgh, pittsburgh, pennsylvania, usa. ijahp article: oyatoye, adebiyi, amole/evaluation subscribers preference for service attributes of mobile telecommunications in nigeria using analytic hierarchy process (ahp) international journal of the analytic hierarchy process 187 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.299 vargas, l.g., (1990). an overview of the analytic hierarchy process and its applications. european journal of operational research, 48, 2–8. yoon, k.p. and hwang c.l. (1995). multiple attribute decision making – an introduction. california: sage publication. topsis-ahp based approach for selection of reverse logistics service provider: a case study of mobile phone industry ijahp article: ali, iftikhar, edwin/ assessment of career selection problems in developing countries: a mcdm approach international journal of the analytic hierarchy process 219 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.488 assessment of career selection problems in developing countries: a mcdm approach yousaf ali, phd assistant professor of operation management department of management sciences gik institute of engineering sciences and technology pakistan yousafkhan@giki.edu.pk naveed iftikhar faculty of mechanical engineering ghulam ishaq khan institute of engineering sciences and technology pakistan u2013287@giki.edu.pk cedric aimal edwin, phd department of management sciences gik institute of engineering sciences and technology, kpk pakistan aimal@giki.edu.pk abstract career selection is one of the most vital and challenging decisions in the life of an individual. in many cases, the individual is not solely responsible for their career choice. in order to address this problem and provide remedial measures, a sample of engineering students was taken from different institutions in pakistan. based on the student’s current and previous academic performance and interest in their projects, a weighted index was constructed with empirical evidence. this index reflects the overall comprehensiveness of the career decision. the study revealed that 57.4% of the students are enrolled because of self-interest, while the remaining have other reasons. furthermore, it was found that a significant number of students experience poor outcomes because of bad decisions in relation to how they choose their career. data analysis suggests that the majority of the students who choose their career due to the influence of others show poor performance on the index rating. the reason for the career choice of the students indicates a highly significant ((p-value = 0.003) association. a gamma test is used to determine the direction of the mailto:yousafkhan@giki.edu.pk mailto:u2013287@giki.edu.pk mailto:aimal@giki.edu.pk ijahp article: ali, iftikhar, edwin/ assessment of career selection problems in developing countries: a mcdm approach international journal of the analytic hierarchy process 220 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.488 association. the results also indicate a negative trend when there is a shift of career choice from one due to self-interest to one based on the outlook of the career. this research takes into account all of the important factors in career choice, and is particularly useful for guidance counselors of high schools and colleges as it provides a framework for career choice decisions for students. finally, the researchers propose a decision making criteria and a numerical model for guiding counselors and students. keywords: p-value; ahp; goodness of the career decision; guidance counselor; gamma test 1. introduction choice of career is not only central to one’s lifestyle, but is also a vital aspect of the physical and emotional well-being of individuals and their families. choosing a career is a crucial problem for any student. most students face questions such as, what am i really going to do?, can i really get there?, am i sure of the path?, what are the risks?, and if i don’t make it, do i have any other options? traditional factors that influence a student’s career choice decision include the following: mark obtained in the annual examination, choice made by parents, and advice of teachers, friends and others. in-depth observations were made in this study to reveal that these determinants of choice are not at all the objectives in nature. some ambitious parents dictate their own unfulfilled career aspirations to their children and compel them to choose a course which perhaps the child does not like or does not have aptitude for. on the other hand, some students have parents who are aware of psychometric testing and career counseling sessions and are able to help their children select a proper career with these tools. similarly, friends and colleagues also persuade students to adopt certain professional careers. with advancements in technology and the emergence of new fields of study, more complexities in career selection for students have been created. many students are attracted by the outlook a certain career offers such as job opportunities, salary and career advancement (oh, 2007). career selection based on an individual’s own choice or even due to an external agent can lead to drastically adverse outcomes. thus, this study looks at the significant effect of all the above mentioned factors on the career choice of an individual and which factors are more important than the rest. ijahp article: ali, iftikhar, edwin/ assessment of career selection problems in developing countries: a mcdm approach international journal of the analytic hierarchy process 221 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.488 in order to eliminate any chances of regret, students should begin the process of career decision making in a rational manner. guidance from trained career counselors at a proper time can significantly reduce disappointment and waste of time and energy. in pakistan, career selection choice is an important issue because the proportion of illiteracy is higher than in other countries. students who are enrolled in school are receiving inadequate guidance and not receiving counselling which is essential for them to excel in their field of choice and perform to their utmost potential. there is no proper arrangement of guidance counselors or a decision making criteria for them, which leaves them with vague, incomplete and unfocused advice. therefore, we have constructed a decision making criteria for guidance counselors. using this decision-making criterion, a numerical method for estimation of a good decision is constructed in order to help students select a better alternative in career selection. 2. literature review the researchers deploy the ahp method to evaluate a good career selection criterion (ozgar c. , 1999). many students in their final year of college remain unclear and indecisive about choosing their career. there are many factors which affect their selection of choosing a career, for example, fizer (2013) determined those factors which affect agriculture student’s choice of career path. college career centers are stocked with instruments (both computerized and pencil/paper) which test student’s interests and abilities. these instruments are designed to help students determine, understand and comprehend which careers/majors they are most suited for. earlier researchers found that student’s interests as determined by the campbell interest and skill survey (ciss) correlated well with student’s majors (hensen, joalda and neuman, 1999). professors were found to be most influential in helping students choose a major (kaynama, shohreh, & louis, 1996). kark & james (1990) determined that students select majors based on the quality of the program and career opportunities. whereas, tame (1996) explored the factors affecting students who become math/science majors, specifically looking at self-efficacy and vocational interests. the study found that extrovert students and those with artistic skills were less likely to become math/science majors. with the large amount of choices that are available, one must make compromises about outcomes (liberatore & miller, 1995). certain majors can be of maximal interest to a student, however not offer enough potential economic payback. a student might sense pressure from parents to ijahp article: ali, iftikhar, edwin/ assessment of career selection problems in developing countries: a mcdm approach international journal of the analytic hierarchy process 222 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.488 select one choice, but is certain that he will make another choice. the decision-making process is complex and judgment and instinct only will not always be enough to respond to the several compromises that must be chosen from among the results. saaty created a decision-making approach called the analytic hierarchy process (ahp), that he claims is, “…natural to our intuition and general thinking” (saaty, 1994). by breaking the problem down into numerous minor, simpler decisions, and then requesting that respondents rank them, the ahp provides decision makers with a structural tool which can help them solve bigger problems. the aim of this research is to improve a choice support model using the ahp in order to help students select a career. a comprehensive survey was conducted to develop a model to guide and evaluate career choice selection among students. the ahp model, which comprises the criteria, is shown in the following section. in order to create the model, a survey was organized by distributing questionnaires to dual sections of the senior capstone strategic management class, and dual sections of the sophomore business statistics class. sixty students from the strategic management class and 57 students from the business statistics class responded to the survey. the survey asked the students to list the influences which played a central role in their career choice selection. they arranged and examined the replies and made a list of the top 15 influences of their career choice. once this list was constructed, they gathered similar or connected influences into 3 groups. the 3 groups were called interest in subject, influence of others and career. after grouping of the influences, hierarchical relations inside each group were considered. the interest in subject and influence of others clusters were only one level of hierarchy, however the career cluster consisted of three sub-criteria including compensation, job availability and growth and job requirements. the job requirements criterion included two sub-criteria, computer usage and interpersonal skills. (swann & henderson, 1998). finally, a computerized data entry database was created in order to ask the students a sequence of inquiries. the major set of questions will be used to create the pairwise comparison matrices. the first criteria pairwise comparison matrix will be used to create the comparative importance of the 3 main criteria. (saaty, 2008). the second pairwise comparison matrix will be used to create the comparative importance of the 3 sub-criteria of the career criterion (compensation, job availability and growth and job requirements). the third and final criteria pairwise comparison matrix will be used to compare the two sub-criteria of the job requirements (computer usage and interpersonal skills) (gibney & ijahp article: ali, iftikhar, edwin/ assessment of career selection problems in developing countries: a mcdm approach international journal of the analytic hierarchy process 223 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.488 shang, 2007). for example, in developing the first criteria pairwise comparison matrix, the computerized data entry program will ask the students their relative judgment of the importance of interest in subject in comparison to influence of others. after that, weight is assigned to each criteria ranging from 1 to 1/9. the outcomes of the ahp model that is created will explore some different areas of information. first, the constancy of student verdicts will be investigated by creating several correlated pairwise comparison matrices. however, if the level of variation is great (consistency ratio is greater than 10%), the outcomes may be unfair and the pair wise comparisons will be reviewed. ahp not only recognizes wherever the major inconsistencies are, but also proposes reviews in the pairwise comparisons to increase the consistency. after achieving a suitable level of consistency, researchers may continue with the rest of the analysis. our objective will be to prioritize the criteria in terms of importance. establishing the relative importance of the factors will hopefully lead to a higher level of understanding of the criteria and the interaction between them. then, we will utilize this knowledge and understand how to better assist the students in selecting their majors. in addition, the results may assist college admissions and recruiting departments to target their efforts based on student characteristics and profile. targeting and preparation of college promotional materials may also be affected. for example, if a student possesses certain characteristics and interests, the college admissions/recruiting department may decide to use certain promotional materials related to majors the student may identify with and/or has high likelihood of success (jones & ewell. t. (1987). 3. methodology 3.1 data collection data is collected through a survey questionnaire that was developed and distributed among engineering students 1 . the data is based on 94 responses from various universities across pakistan such as ghulam ishaq khan institute of engineering sciences and technology (giki), national university of sciences and technology (nust), university of engineering and technology peshawar (uetp) and various faculties such as mechanical engineering, material engineering, computer engineering, electrical engineering etc. 1 for more details see appendix d ijahp article: ali, iftikhar, edwin/ assessment of career selection problems in developing countries: a mcdm approach international journal of the analytic hierarchy process 224 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.488 3.2 data analysis the data obtained from the questionnaire was analyzed using spss and matlab software 2 . career selection analysis reflects various factors which will lead to a determination of the goodness of their choice. each of these factors is linked to questions in the questionnaire. these various factors include student’s satisfaction level, grade point average, project interest level, reason for doing a master’s of science (ms) and not doing a ms, subjects in which they performed better in the past versus current engineering discipline chosen, field related extracurricular activities in the past and if the student thinks he/she would perform better if he/she choose another career. the multiple choices in the questionnaire (corresponding to each of the factors) were assigned relative weights. one example of relative weights of the satisfaction level is given below. table 1 relative weights for satisfaction level factor: satisfaction level multiple choice options weights a) extremely dissatisfied 1 b) dissatisfied 2 c) satisfied 3 d) extremely satisfied 4 the responses were obtained and weights were given to each factor. all these factors were combined to quantify a variable termed ‘goodness of the decision’ denoted by d to obtain a value for the decision of an individual. this decision variable d is given by the following formula: d = 3 sl + 5 gpa + 5 pi + rfdm + 2 rfndm + wac + 2 weca + 5 scsvf (1) where,  sl stands for satisfaction level (weight obtained from question 3 in the questionnaire)  gpa stands for grade point average (weight obtained from question 4 in the questionnaire) 2 matlab code is available in appendix a ijahp article: ali, iftikhar, edwin/ assessment of career selection problems in developing countries: a mcdm approach international journal of the analytic hierarchy process 225 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.488  pi stands for project interest (weight obtained from question 6 in the questionnaire)  rfdm stands for reason for doing ms  rfndm stands for reason for not doing ms,  wac stands for weight for another career (student’s opinion about his performance if he choose another career),  weca variable weights are obtained from questions 7, 8, 9 and 12 in the questionnaire. the relative weights of each of the variables are given according to the relative importance each factor has with respect to the other, which we get from logic and intuition. for example, satisfaction level is a relatively less important factor to consider when evaluating a good or bad decision made by a student. consider that a student can be extremely satisfied (weight = 4), but his gpa is poor and he has no interest in field related projects, so his overall decision score should be negative even though his relative weight for satisfaction level is high. the overall weight of satisfaction level is 3 which is less than the overall weight for gpa and project interest which is 5 as they are relatively more important factors as compared to satisfaction level. after importing our online data to spss, the decision score for each student was evaluated based on the multiple choices the participant selected in the questionnaire. the overall value obtained is then compared to a standard value for decision variable d. this value is obtained by putting the weights of all the variables beyond which a decision is considered good and below which a decision is considered bad. for example, to obtain a standard value we take the weight of the sl variable which stands for satisfaction level =3 (satisfied). this means if a person is dissatisfied the sl value should contribute to a bad decision in the overall decision variable d. similarly other values are taken such as gpa=4 (between 2.5 to 3), pi=2 (interested), rfdm=3, rfndm=4, wac=2, weca=3 and scsvf=4. therefore, the standard value for decision variable d comes out to be 78. this value 78 is the cut off value, which means any score above 78 is taken as a good decision. for the scsvf variable we linked a student’s previous performance in a particular subject (questions 10 and 11) to their current faculty (question 1) via the matlab program (appendix a). each particular choice in faculty is given a value from 1 to 3, and similarly a value is given for each particular choice for the subjects that an individual was better at in school and at the college level. in case of a mismatch between the comparison in the numbers, we have assigned a lower weight i.e. 3, for example if a student was good at chemistry ijahp article: ali, iftikhar, edwin/ assessment of career selection problems in developing countries: a mcdm approach international journal of the analytic hierarchy process 226 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.488 (value assigned is 2) in college and school, and he has chosen computer science (value assigned is 3) as his faculty, he has made a bad decision and should have selected chemical or material engineering (value assigned is also 2). in the case the student has selected a subject other than anything engineering related, we have assigned an even lower weight of 1. after calculating the d variable, a chi square and gamma test were used to evaluate the data and obtain the p value between decision variable d and the career choice selection factor (obtained from question 2). this enabled us to observe the level of significance between the career choice selection factor (self-interest, influence of others, career outlook) and a good career choice decision (d>78). moreover, it also shows whether the individual whose choice was engineering due to self-interest has made a good career choice or not. similarly, it provides insight on how many individuals chose engineering due to the influence of others, whether they made a good career choice and how many individuals chose engineering due to its career outlook (job opportunities, career advancement, salary etc.) and whether they made a good career choice. a gamma test is used to show the direction of association. after assessment/evaluation of the problem we have to solve the problem by constructing a decision criteria for better career choice selection by analytical hierarchy process (ahp) for various cases using the proposed indexation formula and taking into account the significant factors such as the grades of the students in their respective subjects, their field of interest, parental influence, peer pressure, career advancement, salaries and job opportunities. we have provided weights for each of the cases under consideration. these weights were calculated from the questionnaire (questions 13, 14, 15, 16, 17, 18, and 19) where the students were asked about their preference for each criterion with respect to the other. the weights obtained from the ahp are then used to derive a numerical formula for career choice decision makers by taking into account all the factors. using a preference scale for each factor, similar to the indexation formula used in ahp, any subject can put a relative preference value in the numerical model and evaluate a suitable career choice. 4. findings and discussion from the survey results given in the appendix we found that about 57.9% people chose their career due to self-interest, 20.8% of people due to influence of others and an almost equal amount due to the ijahp article: ali, iftikhar, edwin/ assessment of career selection problems in developing countries: a mcdm approach international journal of the analytic hierarchy process 227 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.488 outlook of a career (job opportunities, salary etc.). the career choice of the students indicates a highly significant (p-value = 0.003) association. it is evident from the cross table that the percentages of scores greater than 78 are remarkably higher (79.4%) than those that got index scores less than 78 for self-interest, which means that if 100% people have made a good decision only 79.4% have selected a career due to self-interest. on the other hand, those choosing a career due to the influence of the others possess the least percentage possibility (5.9%) of making a good decision. while out of those choosing career on the basis of job opportunities, 14.7% have made a good decision. the direction of the association is assessed by making use of the gamma test, which indicates that as we move from career due to self-interest to career on the basis of job opportunities the percentages of 78 scores decline significantly (p-value) . table 2 index score versus carrier choice selection factor index score carrier choice selection factor total self interest influence of others career (job opportunities, salary) less than 78 27 18 15 60 45.0% 30.0% 25.0% 100.0% 78 to 100 27 2 5 34 79.4% 5.9% 14.7% 100.0% total 54 20 20 94 57.4% 21.3% 21.3% 100.0% table 3 chi-square tests value df sig. pearson chi-square 11.487 2 .003 n of valid cases 94 ijahp article: ali, iftikhar, edwin/ assessment of career selection problems in developing countries: a mcdm approach international journal of the analytic hierarchy process 228 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.488 table 4 symmetric measures value std. error approx. t sig. ordinal by ordinal gamma -.535 .164 -3.029 .002 n of valid cases 94 the findings of the report also show that about 63% of the people have chosen a bad career. figure 1 shows the comparison between the number of individuals who have chosen a good career divided by the total number of people in that respective career choice factor and with respect to their career choice factor. figure 1. percentage of students with good career decision with respect to career selection reason figure 1 shows that the percentage of people who have selected a viable and stable career option due to career outlook (job opportunities, salary, career advancement etc.) are significantly higher than those due to influence of others. students who choose their career due to self-interest have the largest percentage of good decisions (50%). figure 1 provides the evidence of a gamma test and a chi-square test as it shows that the direction of association is negative i.e. percentage of students with good career decision decreases significantly as career choice selection factors change from self-interest to influence of others. in order to solve the problem, we construct a career selection criteria based upon the analytical hierarchy process (ahp).we use several different criteria i.e. self-interest, career outlook, and influence of [value] % [value] % [value] % 0 10 20 30 40 50 60 self intrest career outlook influence of others% o f st u d e n ts w it h g o o d c a re e r d e ci si o n career choice selection factor ijahp article: ali, iftikhar, edwin/ assessment of career selection problems in developing countries: a mcdm approach international journal of the analytic hierarchy process 229 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.488 others to evaluate a good career decision. these major criteria are then divided into sub criteria i.e. influence due to parents or peers, career outlook, career advancement and salary and job opportunities etc. the guidance counselor or a student who wishes to evaluate a good career decision needs to know which of the above mentioned criteria is more important than the other. for this purpose, through an extensive survey, we tried to understand which category people prefer, i.e. how much weight they assign to self-interest, career outlook, and influence of others with respect to each other. those weights are then assigned a place in the ahp matrix in order to rank the criteria and calculate the preference weight for each criteria. using these preference weights we have formed a career decision formula in which the guidance counselors can put in the perspective of a student in the respective category and get the right career decision. figure 2. ahp model the priority table in appendix b illustrates the relative rating for importance of criteria. based on the survey results given in the appendix, the following results are obtained.  self-interest is extremely to very strongly more important as compared to influence of others ijahp article: ali, iftikhar, edwin/ assessment of career selection problems in developing countries: a mcdm approach international journal of the analytic hierarchy process 230 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.488  self-interest is strongly more important as compared to career outlook  influence of others is moderately less important as compared to career outlook  job opportunities is moderately to strongly less important as compared to salary  job opportunities is extremely to very strongly less important as compared to salary  salary is moderately to strongly less important as compared to career advancement  parental pressure is moderately less important as compared to peer pressure table 4 shows the priority matrix for criteria such as self-interest, influence of others and career outlook. the preference weights of these criteria, their ranks and their consistency ratio can be observed in table 5. we found that the most important factor to consider is self-interest with a weight of 74.18%. table 5 ahp matrix for evaluating criteria, self-interest, influence of others and career outlook table 6 preference weights, ranks and consistency ratio for self-interest, influence of others and career outlook criterion weights rank self interest 74.2 % 1 influence of others 7.5 % 3 career 18.3 % 2 consistency ratio (cr) = 0.046 (4.6%) self interest influence of others career outlook self interest 1 8 5 influence of others 2 1 1/3 career outlook 1/5 3 1 ijahp article: ali, iftikhar, edwin/ assessment of career selection problems in developing countries: a mcdm approach international journal of the analytic hierarchy process 231 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.488 table 6 shows the priority matrix for sub criteria in career outlook such as job opportunities, salary and career advancement. the preference weights of these criteria, their ranks and their consistency ratio (cr) are mentioned in table 7. we found that the most important factor to consider is career advancement with a weighting of 70.71%. table 7 ahp matrix for evaluating criteria, job opportunities, salary and career advancement table 8 preference weights, ranks and consistency ratio for job opportunities, salary and career advancement criterion weights rank job opportunities 7.0 % 3 salary 22.3 % 2 career advancement 70.7 % 1 consistency ratio (cr) = 0.056 (5.6%) table 8 shows the priority matrix for sub criteria in influence of others such as parental pressure and peer pressure. the preference weights of these criteria, their ranks (rk) and their consistency ratio (cr) can be observed in table 9. we find that the most important factor to consider is parental pressure with a weight of 74.97%. job opportunities salary career advancement job opportunities 1 1/4 1/8 salary 4 1 ¼ career advancement 8 4 1 ijahp article: ali, iftikhar, edwin/ assessment of career selection problems in developing countries: a mcdm approach international journal of the analytic hierarchy process 232 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.488 table 9 ahp matrix for evaluating sub-criteria, parental pressure and peer pressure table 10 preference weights, ranks and consistency ratio for parental pressure and peer pressure criterion weights rank parental pressure 75 % 1 peer pressure 25 % 2 consistency ratio (cr) = 0.001 (0.1%) the consistency ratio is less than 0.1 for all the above decision matrices, therefore, the degree of consistency is acceptable and the ahp results are meaningful. based upon these weights, we calculated the following career selection formula for guidance counselors. cdv(a) = 0.74 sia + 0.183(0.707 caa + 0.223 sa + 0.07 joa) + 0.075 (0.75 ppa + 0.25 prpa) (2) where  a represents alternative  cdv (a) stands for career decision value for alternative a  sia stands for subject interest in that particular priority  caa stands for career advancement in that particular priority  sa stands for salary in that particular priority  joa stands for job opportunities in that particular priority  ppa stands for parental pressure in that particular priority  prpa stands for peer pressure in that particular priority the counselor or career choice decision maker has to weight each criteria with regard to a preference scale given in the appendix c. for example, a student may prefer engineering extremely more as compared to business with respect to interest and prefer business more strongly as compared to engineering with respect to influence of parental pressure peer pressure parental pressure 1 ¼ peer pressure 4 1 ijahp article: ali, iftikhar, edwin/ assessment of career selection problems in developing countries: a mcdm approach international journal of the analytic hierarchy process 233 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.488 others such as his parents. while he/she is not limited by other preferences, for example he/she may not be influenced to a career due to its outlook. so we put these values in equation 1 to calculate a career decision value for both engineering preference and also business preference. in this case as the decision is not based upon career outlook so: caa = sa= joa= prpa= 0 (no preference), let p = 1 for engineering, then we have si1 = 9 (extremely more preferred), pp1 = 1/5 (moderately less preferred), so cdv (1) = 6.67 now let p = 2 for business, then we have si2 = 1/9 (extremely less preferred), pp2 = 5 (moderately more preferred), cdv (2) = 0.363 so as cdv (1) > cdv (2) which implies preference 1 is better, this means that engineering is a better decision than business in this case and the guidance counselor has to suggest engineering as a suitable career for the student. similarly, if there are three alternatives we first use this formula for two alternatives using the same procedure as mentioned above and then select the career choice value for the alternative that is larger. now this selected alternative is compared with the third alternative and again the same method is applied to select an optimal career choice. 5. conclusion the potential objective of this research was to investigate the effects of career choice selection on students based on survey data collection. it was found that a majority of the students selected their career choice due to self-interest. the ratio of students enrolled due to influence of others and students that enrolled due to career outlook (job opportunities, salary, career advancement etc.) was almost equal. empirical data was used to obtain an overall career decision variable which shows how many students have made a poor decision and selected a bad career path. career choice selection due to the influence of others quite often leads to a bad career choice for an individual. this is evident from the scope of study as p value hints that there is a significant difference between a good career selection and the reason for selecting a career choice. the results also suggest an alarming element that most of the students are not able to realize their full ijahp article: ali, iftikhar, edwin/ assessment of career selection problems in developing countries: a mcdm approach international journal of the analytic hierarchy process 234 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.488 potential, as 63% of the students surveyed have made a bad career choice. furthermore, 50% of the people that have chosen their career due to self-interest have also selected the wrong career option. therefore, this research indicates an urgent need for career guidance counselors in schools and colleges in order to help students select a suitable career for themselves. we present a robust ‘guidancecounselor-criteria’ upon which suitable career choice selections could be effectively based. we have selected various career choice selection criteria and using ahp (analytical hierarchy process) we assign weight to each criterion which shows which criteria is more important for selection of a good career. these weights are used to construct a formula to work on the basis of decision preferences of a respective student in various criteria. the overall score from this formula is used to compare various career preferences of a student. the preference with a higher score is selected as the best possible career choice for that particular student. ijahp article: ali, iftikhar, edwin/ assessment of career selection problems in developing countries: a mcdm approach international journal of the analytic hierarchy process 235 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.488 references fizer, d. (2013). factors affecting career choices of college students enrolled in agriculture. a research paper presented for the master of science in agriculture and natural science degree at the university of tennessee, martin. hensen, jo alda & neuman. (1999). evidence of concurrent predication interest and skill survey ciss. journal of career assesment, 7(3), 239247. doi: https://doi.org/10.1177/106907279900700304 jones, d. p., & ewell, p. t. (1987). accountability in higher education: meaning and methods. boulder, co: national center for higher education management systems. kark, & james, j. (1990). selection of graduate major: implications for student recruitment. college and university, 66(1), 35-40. kaynama, shohreh, & louis, s. (1996). using consumer behavior and decision model to aid a student in college major. journal of marketing of higher education, 7(2), 57-73. doi: http://dx.doi.org/10.1300/j050v07n02_05 liberatore, & miller, m. j. (1995). a decision support approach for transport carrier and mode selection. journal of business logistics, 16(2), 85-99. oh, h. (2007). business school students career perceptions and choice decisions. university of florida: david f. miller center for retailing education and research. ozgar, c., strasser, s., schroeder, d. (1999). using the analytical hierarchy process to select a college major. indiana: valparasio university. swann, c, and henderson, s. (1998). handbook for the college admissions profession. westport, ct: greenwood publishing group. saaty, t.l. (1994). how to make a decision: the analytic hierarchy process, interfaces, 24(6), 19–43. doi: https://doi.org/10.1287/inte.24.6.19 saaty, t.l., 2008. decision making with the analytic hierarchy process. international journal of services sciences, 1(1), 83-98. doi: http://dx.doi.org/10.1504/ijssci.2008.017590 https://doi.org/10.1177/106907279900700304 https://doi.org/%7b0%7d ijahp article: ali, iftikhar, edwin/ assessment of career selection problems in developing countries: a mcdm approach international journal of the analytic hierarchy process 236 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.488 tame, l. r. (1996). efficacy expectations and vocational interests as mediators between sex and choice of math/science college majors: a longitudinal study. journal of vacational behavior, 49(3) 77-91. doi: https://doi.org/10.1006/jvbe.1996.0044 https://doi.org/10.1006/jvbe.1996.0044 ijahp article: ali, iftikhar, edwin/ assessment of career selection problems in developing countries: a mcdm approach international journal of the analytic hierarchy process 237 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.488 appendix a matlab program for scsvf variable for n=1:71 if c(n)~=4 && s(n)~=4 if c(n)~=0 && s(n)~=0 if b(n)==c(n)||b(n)==s(n) a(n,:)= 4 end if b(n)~=c(n)||b(n)~=s(n) a(n,:)= 2 end end end if c(n)==4 && s(n)==4 a(n,:)=1 end if c(n)==0 && s(n)==0 a(n,:)=2 end end ijahp article: ali, iftikhar, edwin/ assessment of career selection problems in developing countries: a mcdm approach international journal of the analytic hierarchy process 238 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.488 appendix b priority table comparison scale for the importance of criteria in ahp judgment rating extremely more important 9 8 very strongly more important 7 6 strongly more important 5 4 moderately more important 3 2 equally important 1 1/2 moderately less important 1/3 1/4 strongly less important 1/5 1/6 very strongly less important 1/7 1/8 extremely less important 1/9 ijahp article: ali, iftikhar, edwin/ assessment of career selection problems in developing countries: a mcdm approach international journal of the analytic hierarchy process 239 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.488 appendix c comparison scale for importance of alternatives using equation judgment rating extremely more preferred 9 8 very strongly more preferred 7 6 strongly more preferred 5 4 moderately more preferred 3 2 equally preferred 1 1/2 moderately less preferred 1/3 1/4 strongly less preferred 1/5 1/6 very strongly less preferred 1/7 1/8 extremely less preferred 1/9 no preference 0 ijahp article: ali, iftikhar, edwin/ assessment of career selection problems in developing countries: a mcdm approach international journal of the analytic hierarchy process 240 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.488 appendix d questionnaire for effects of career selection choice on students 1. faculty : mechanical, electrical, electronics computer science, computer engineering material, chemical 2. how did you come about your career ? self interest influence of others (parental pressure, peer pressure) career (job opportunities, salary) ? 3. what is your satisfaction level regarding your career? extremely dissatisfied dissatisfied satisfied extremely satisfied 4. what is your gpa ? below 2 between 2 and 2.2 between 2.2 and 2.5 between 2.5 and 3 above 3 5. how much interest do you take in your course/final year engineering projects ? not interested interested very interested 6. do you want to apply for ms after graduation ? yes no 7. if yes, what is the reason ? i can’t get a job without ms i don’t want to go to an industry, i prefer teaching i like to study engineering further other: 8. if no, then what do you want to do after graduation ? apply for a job(due to interest) apply for a job(due to financial issues) ijahp article: ali, iftikhar, edwin/ assessment of career selection problems in developing countries: a mcdm approach international journal of the analytic hierarchy process 241 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.488 i want to study something other than engineering(mba, css etc) start business or start my own company(related to engineering) start business or start my own company(not related to engineering) other: 9. if you were to choose another career would your performance be better? yes no maybe 10. in which subject/course did you perform best during entire school period? maths or physics chemistry computer science other: 11. which subject/course did you perform best during entire fsc/a-level? maths or physics chemistry computer science other 12. which type extracurricular activities you participate in before entering university? related to engineering or science not related to engineering or science not involved in any extracurricular activities for below questions, please give comparatively rating between given decision criteria i.e which factor is more important in resulting to a bad the career choice decision for a student in your opinion 13. self-interest is __________ factor as compared to influence of others(parental pressure, peer pressure) extremely more important strongly more important moderately more important equally important moderately less important ijahp article: ali, iftikhar, edwin/ assessment of career selection problems in developing countries: a mcdm approach international journal of the analytic hierarchy process 242 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.488 strongly less important extremely less important 14. career (job opportunities, salary) is __________ factor as compared to influence of others(parental pressure, peer pressure) extremely more important strongly more important moderately more important equally important moderately less important strongly less important extremely less important 15. self-interest is __________ factor as compared to career (job opportunities, salary) extremely more important strongly more important moderately more important equally important moderately less important strongly less important extremely less important 16. job opportunities is __________ factor as compared to salary extremely more important strongly more important moderately more important equally important moderately less important strongly less important extremely less important 17. job opportunities is __________ factor as compared to career advancement extremely more important strongly more important moderately more important equally important moderately less important strongly less important ijahp article: ali, iftikhar, edwin/ assessment of career selection problems in developing countries: a mcdm approach international journal of the analytic hierarchy process 243 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.488 extremely less important 18. salary is __________ factor as compared to career advancement extremely more important strongly more important moderately more important equally important moderately less important strongly less important extremely less important 19. parental pressure is __________ factor as compared to peer pressure extremely more important strongly more important moderately more important equally important moderately less important strongly less important extremely less important ijtm/ijcee page templatev2 ijahp article: chaudhury, mandal, das/ selection of appropriate fluid delivery technique for grinding titanium grade-1 using the analytic hierarchy process international journal of the analytic hierarchy process 454 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.356 selection of appropriate fluid delivery technique for grinding titanium grade-1 using the analytic hierarchy process ayan chaudhury department of mechanical engineering kalyani government engineering college kalyani741 235, west bengal, india email: chaudhury.ayan17@gmail.com bijoy mandal department of mechanical engineering kalyani government engineering college kalyani741 235, west bengal, india email: bijoymandal@gmail.com santanu das department of mechanical engineering kalyani government engineering college kalyani741 235, west bengal, india email: sdas.me@gmail.com abstract grinding is commonly used in industry for the finishing or semi-finishing of different mechanical components. in this process, a wheel is rotated at a high speed. the wheel is made of abrasive particles known as grits. during grinding, high grinding zone temperature is experienced leading to several grinding defects. to control these thermal defects grinding fluid is usually employed mainly to cool and lubricate the grinding region. however, most of the applied grinding fluid cannot reach the grinding zone as it is deflected by the stiff air layer formed around the wheel periphery. several attempts have been made in the past to overcome this problem in order to guarantee better fluid delivery. in this paper, two newly developed methods, a pneumatic barrier and a compound nozzle are considered to serve this purpose. grinding experiments are conducted on titanium grade-1 specimens under four environmental conditions, which include dry, flood cooling, flood cooling with pneumatic barrier set up and cooling using a compound nozzle. under each environment, 10 grinding passes are undertaken using 10, 20 and 30 m infeed. data obtained are used to optimize the grinding performance by employing the analytic hierarchy process (ahp). the ahp results show compound nozzle fluid delivery at 20 m infeed to be the appropriate condition for grinding titanium grade-1 within this experimental domain. this condition is supposed to deliver grinding fluid deep into the grinding zone thereby controlling grinding temperature effectively and may be recommended to the industry. keywords: grinding; grinding fluid; fluid delivery technique; pneumatic barrier; surface roughness; grinding forces; flood cooling nozzle; compound nozzle; analytic hierarchy process; ahp; grinding titanium grade-1; optimization ijahp article: chaudhury, mandal, das/ selection of appropriate fluid delivery technique for grinding titanium grade-1 using the analytic hierarchy process international journal of the analytic hierarchy process 455 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.356 1. introduction surface grinding is done using an abrasive wheel rotating at a high speed. in grinding, a high temperature is generated at the grinding zone, and this high temperature is the cause of several grinding defects. to control for these thermal defects grinding fluid is usually employed. engineer et al. (1992) observed that only 4 to 30% of grinding fluid passes through the grinding zone in flood cooling systems, and hence, a large quantity of grinding fluid is wasted. formation of a stiff air layer around the periphery of a rotating grinding wheel is the prime reason for wastage of the grinding fluid. researchers (inasaki, 1998; rowe, 2009; wu et al., 2007) have found that the generation of an air layer around a rotary grinding wheel is due to viscous friction between the wheel surface and the air in its vicinity, and because of the centrifugal force developed due to high rotational speed of the wheel. morgan et al. (2008) and parthasarathy & malkin (2009) investigated the effect of different grinding fluid delivery systems on grinding performance, and finally recommended certain conditions of fluid delivery to give desired grindability. several attempts were made to control this air layer and to increase the grinding fluid penetration into the grinding zone (brinksmeier et al. 1999; ebbrell et al., 2000; irani et al., 2005; morgan et al., 2008; palhade et al., 2009; parthasarathy & malkin, 2009). different grinding fluid delivery techniques were developed for this. mandal et al. (2011a) reported that if rexine is pasted to both side faces of the grinding wheel, then less air pressure is developed near the wheel. this is likely due to the suppression of the axial suction of the air through the grinding wheel pores so that the centrifugal throw of air gets substantially reduced. a newly developed compound nozzle fluid delivery system and pneumatic barrier setup were reported to be successful grinding fluid delivery techniques (mandal et al., 2011b; 2012; 2014). the performance of applying a grinding fluid for different workpiece materials varies widely, particularly when grinding exotic, difficult-to-grind materials. titanium alloys are an example of this kind of material due to their typical adhesion characteristics and mechanical properties. titanium alloys are extensively used as bio transplants and in the areas of aeronautics, cryogenic vessels, etc. hence, although it is a challenge to the machining person it is absolutely necessary to find the appropriate condition for good grinding performance. turley (1985), xu et al. (2003), teicher et al. (2006) and palhade et al. (2009) carried out detailed investigations on some titanium alloys under different grinding environments using different grinding wheel materials. they compared the grinding performance at each of the conditions used and put forward some recommendations. biswas et al. (2012) and mandal et al. (2013) carried out experimental investigations on grinding titanium grade 1 using silicon carbide and alumina wheels respectively. biswas et al. (2012) could perform grinding operations in wet condition with some success using a typically designed compound nozzle, while mandal et al. (2013) reported grinding a similar workpiece in wet conditions using a pneumatic barrier system. the grindability of any material is generally dependent on the properties of the workpiece material, type of grinding wheel and parametric condition of grinding and its environment. hence, it becomes difficult to make a decision about the appropriate grinding condition when variations of parameters and environmental conditions increase. different methods using fuzzy set theory, genetic algorithm, neural networks, etc. were applied by different groups of researchers such as morgan et al. (2008), sun et al. (2001) and others to solve decision making problems and find the best alternative or condition among many choices. the analytic hierarchy process (ahp) is widely used in various fields of managerial decision-making and similar cases to solve multi-objective decision making problems (saaty, 1977; 1980; vargas, 1990; wu et al., 2007; sabiruddin et al., 2013). it is a simple but powerful and flexible decision making tool to solve various complex multi-criteria decision making ijahp article: chaudhury, mandal, das/ selection of appropriate fluid delivery technique for grinding titanium grade-1 using the analytic hierarchy process international journal of the analytic hierarchy process 456 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.356 (mcdm) problems hierarchically. sun et al. (2001) reported a two-grade fuzzy synthetic decision-making system with the use of the analytic hierarchy process (ahp) for evaluating the performance of grinding fluid. some researchers also used the ahp to determine grinding performance (huang et al., 2005; wang et al., 2006; shi et al., 2008). in the present experimental work, grinding experiments are performed on a titanium grade 1 workpiece with varying infeed under different environmental conditions, such as dry, conventional flood cooling, flood cooling with pneumatic barrier and cooling with compound nozzle. a comparison is made among all these conditions with respect to grinding forces, surface quality, grinding chip, wheel wear, etc. the observations made in the grinding experiments are then used to determine the optimized condition to obtain good grindability using the analytic hierarchy process (ahp). 2. the analytical hierarchy process applied in the present work, a simple analytical hierarchy process (ahp) is considered which is similar to that used by sabiruddin et al. (2013). the hierarchy structure chosen is shown in figure 1. at the top of the hierarchy structure, there is the goal or the objective of this study which is the selection of a grinding condition with good grindability. figure 1. the hierarchy structure chosen the grindability, or the ease of grinding of a workpiece-grinding wheel combine, is judged by (engineer et al., 1992; rowe, 2009): i) grinding force, grinding energy or specific grinding force (grinding force per unit volume of material removed) requirement, or f ratio (ratio of tangential force component and normal force component) ii) grinding temperature iii) wheel grit wear, wheel material loss, or g ratio (material removal rate, mrr / wheel material removal, wmr) iv) ground surface quality including surface finish and integrity, that is, absence of open and sub-surface cracks or tensile residual stresses v) favourable chip formation for good grindability, grinding force or energy requirement should be less, grinding temperature is to be as low as possible, wheel grit wear or wheel material removal is ijahp article: chaudhury, mandal, das/ selection of appropriate fluid delivery technique for grinding titanium grade-1 using the analytic hierarchy process international journal of the analytic hierarchy process 457 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.356 a = to be low, and surface quality is to be good. chips produced should be predominantly of the slice type with much less spherical type chips. in the present work, three grindability judgement characteristics are considered such as grinding force, wheel wear and surface integrity. therefore, these three criteria are in the hierarchy structure (figure 1). twelve alternatives corresponding to the experimental runs conducted at three infeed and four environmental conditions are chosen in this work. the pair wise comparison matrices are constructed by comparing an element with the elements of the next higher level. this helps to determine the local priority weights. a typical pair wise comparison matrix is shown in equation 1. (1) in equation 1, each matrix element, aij represents the strength of preference of the alternative ei over ej with respect to the criterion (c), aji = 1/aij and aii = 1 for values of i and j from 1 to n. values of aij are selected from the ratio scale enlisted in table 1. next, consistency of the matrix is checked through calculation of consistency ratio (cr) that is given by cr = (ci/ri). table 1 ratio scale of comparison matrix preferential judgment rating extremely preferred 9 very strongly to extremely preferred 8 very strongly preferred 7 strongly to very strongly preferred 6 strongly preferred 5 moderately to strongly preferred 4 moderately preferred 3 equally to moderately preferred 2 equally preferred 1 the consistency index (ci) = (m-n)/(n-1) and the random index (ri) is the consistency index of a matrix with random numbers from (1/9, 1/8, 1/7, ……..1.……..7, 8, 9) scale. m is the largest eigen value of the matrix a with n being the size of the matrix. a consistency ratio of less than or equal to 10% is acceptable. local weights, wi are evaluated through equation 2. wi = , i = 1, 2, 3, ….. n (2) when pj (j = 1, 2, 3, ……. m) are the priority weights of n alternatives for the j th criterion, and qij are the priority weights of the criteria, the global weights (ri) of the alternatives are determined (saaty, 1977; saaty, 1980; sabiruddin et al., 2013) from equation (3). ri = , i = 1, 2, 3, …… n (3) c e1 e2 e3 ………. ………. en e1 a11 a12 a13 ………. ………. a1n e2 a21 a22 a23 ………. ………. a2n e3 a31 a32 a33 ………. ………. a3n … … … … ………. ………. … en an1 an2 an3 ………. ………. ann ijahp article: chaudhury, mandal, das/ selection of appropriate fluid delivery technique for grinding titanium grade-1 using the analytic hierarchy process international journal of the analytic hierarchy process 458 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.356 the global weight of the largest value is considered to be the optimum value indicating the decision. 3. experimental details experimental details are shown in table 2. the grinding performance is observed under different conditions during surface grinding of a titanium grade 1 specimen. a wheel velocity of 30 m/s and a table feed of 7 m/min are maintained throughout the experiment. three infeed of 10m, 20 m and 30 m are chosen. four environmental conditions including dry, flood cooling, flood cooling with pneumatic barrier and cooling with compound nozzle are considered to observe their effects. up grinding mode is followed for all the experiments. ten grinding passes are undertaken at each condition. in flood cooling, the grinding fluid is allowed to pass through a commonly used nozzle (outer diameter 6 mm) placed 10 mm above the workpiece. its discharge is 1000 ml/min. however, only 350 ml/min of the fluid was found to go through the wheel-workpiece contact zone (mandal et al., 2014). correspondingly, some cooling and lubricating effects could be observed. the grinding fluid is expected to have a better effect when the pneumatic barrier setup is used along with flood cooling, as the high air velocity coming out of the pneumatic nozzle may disturb the stiff air layer formed around the wheel resulting in better entry of the grinding fluid into the grinding zone. a pneumatic gauge pressure of 400 mm of water column, or 3.90 kpa, is employed. in this work, a pneumatic nozzle at a radial distance of 10 mm from the wheel periphery is positioned at a swivel angle (α) of 30 o and polar angle (θ) of 45 o as has been recommended in other works (mandal et al., 2011b; 2012). this configuration is shown schematically in figure 2, and its photograph is depicted in figure 3. discharge of grinding fluid is kept at 1000 ml/min. for the compound nozzle fluid delivery system, a specially designed and fabricated nozzle is used. the fluid delivery through this nozzle is set at 475 ml/min, as this discharge (that is 52.5% less than that used in the flood cooling experiments) has already been reported by mandal et al. (2014) to break the stiff air layer without using a pneumatic barrier. a schematic diagram indicating the compound nozzle is given in figure 4. ijahp article: chaudhury, mandal, das/ selection of appropriate fluid delivery technique for grinding titanium grade-1 using the analytic hierarchy process international journal of the analytic hierarchy process 459 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.356 table 2 experimental details grinding machine surface grinding machine make: maneklal & sons, india main motor power: 1.5 kw grinding wheel used specification: aa 46/54 k5 v8 make: carborandum universal limited, india size : 200 mm x 13 mm x 31.75 mm dressing detail dressing tool: single point 0.5 carat diamond dresser dressing depth: 20 µm speed of dressing: 0.36 m/min grinding condition mode of grinding: up grinding grinding wheel velocity (vc): 30 m/s table feed: 7 m/min infeed: 10, 20 and 30 µm grinding environment dry wet with water soluble oil (1:20)  flood cooling with a flow rate of 1000 ml/min  flood cooling using pneumatic barrier with a flow rate of 1000 ml/min  fluid delivery using compound nozzle with a flow rate of 475 ml/min pneumatic barrier setting polar angle (θ): 45 o swivel angle (α): 30 o pneumatic barrier pressure: 400 mm of water column (3.90 kpa) workpiece detail titanium grade1 composition: 99.85% ti, 0.01% n, 0.12% fe and 0.02% o hardness: 220 hb, size: 120 mm x 65 mm x 6 mm a sushma industries, bengaluru, india made 3 channel strain gauge grinding dynamometer (model: sa116) is used for measuring grinding force components (tangential, ft and normal, fn). the surface roughness of the ground surface is measured after ten passes using surtronic 3+ talysurf (make: taylor hobson, india). the ground surface is observed under a mitutoyo, japan made tool makers microscope. with it, the grinding chips collected during the 9 th pass are examined in order to determine their forms. the grinding wheel wear is measured using a mitutoyo, japan made dial indicator. wheel loading, that is, the phenomenon in which chip particles get attached within intergrit spaces, is noted visually after each experiment. all the experiments are replicated once, and the averaged data are used for the analysis. ijahp article: chaudhury, mandal, das/ selection of appropriate fluid delivery technique for grinding titanium grade-1 using the analytic hierarchy process international journal of the analytic hierarchy process 460 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.356 figure 2. schematic representation of the cooling arrangement figure 3. photograph of the pneumatic barrier setup figure 4. schematic diagram of the compound nozzle 4. experimental results and discussions results of the grinding experiments on titanium grade 1 under different grinding conditions are represented in table 3. these data are used to determine the appropriate condition to obtain the best grindability within this experimental domain utilizing the ahp. both the tangential (ft) and normal (fn) grinding forces in all experimental conditions considered are presented in table 3. the grinding forces at the 5 th and 10 th pass and their maximum values at each experiment are shown in table 3. it is observed that both the grinding forces are lower at grinding with the compound nozzle fluid delivery system than that at the other environmental conditions. when the flood cooling with pneumatic barrier system is used, it requires slightly higher forces than with the compound nozzle system, but significantly less than the dry and ijahp article: chaudhury, mandal, das/ selection of appropriate fluid delivery technique for grinding titanium grade-1 using the analytic hierarchy process international journal of the analytic hierarchy process 461 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.356 flood cooling systems. this shows that there is better penetration of grinding fluid into the grinding zone with the pneumatic barrier than the common flood cooling system. the flood cooling also shows only small improvement over the dry condition as most of the grinding fluid applied cannot reach the grinding zone. corresponding to this reason, expectedly, less chip re-deposition/ grit indentation and less presence of wheel wear are observed while using the compound nozzle. this indicates the favoured grinding situation with the compound nozzle fluid delivery system that may have caused better temperature control through supplying a larger quantity of the grinding fluid into the grinding zone by penetrating the stiff air layer covering the wheel. better lubrication properties and longer retention of the grit sharpness may have resulted in less force requirement. table 3 comparison of grinding environment for titanium grade1 response grinding environment dry flood cooling flood cooling with pneumatic barrier cooling with compound nozzle infeed in µm infeed in µm infeed in µm infeed in µm 10 20 30 10 20 30 10 20 30 10 20 30 f t (n ) 5 th pass 11.8 16.7 20.6 10.8 14.2 19.6 10.3 13.7 15.7 9.8 12.3 15.7 10 th pass 12.8 15.7 20.6 10.3 14.7 14.7 8.3 12.3 12.8 8.3 10.8 10.8 max 13.2 17.7 25.5 11.3 15.7 19.6 10.3 13.7 16.7 10.3 12.3 15.7 f n ( n ) 5 th pass 47.6 64.7 78.1 42.2 53.5 68.7 41.2 49.1 57.4 34.3 42.2 57.9 10 th pass 43.7 58.9 84.9 41.2 49.5 57.4 40.7 41.7 54.0 39.2 40.2 51.0 max 49.1 64.7 91.2 45.1 59.4 68.7 44.1 50.0 58.9 39.2 47.1 57.9 average surface roughness 2.88 3.27 3.3 2.4 2.71 2.51 2.02 1.77 2.39 1.73 1.41 1.81 surface burn few few severe no few severe no no few no no few chip redeposition few few large few few large no no few no no few surface cracks no yes yes no no yes no no yes no no yes chip forms observed s, sl s, sl s, sl s, sl s, sl s, sl s, sl, l s, sl s, sl s, sl, l s, sl s, sl note: l: long curl chips, s: short segmented chips, sl: slice type chips from table 3, it is observed that the average roughness values under different infeed conditions are lower under the compound nozzle fluid delivery condition than the dry, the flood cooling and the flood cooling with pneumatic barrier conditions. observing the surface burn, surface crack, chip formation and chip redeposition, it may be stated that there is no remarkable difference between the flood cooling with pneumatic barrier system and the fluid supply through compound nozzle system. within these four grinding environments, it is observed that the compound nozzle system apparently performs better than the other three conditions. the ahp, a decision making tool, is used in this work to select the best fluid delivery technique within the domain of this experimental investigation. ijahp article: chaudhury, mandal, das/ selection of appropriate fluid delivery technique for grinding titanium grade-1 using the analytic hierarchy process international journal of the analytic hierarchy process 462 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.356 5. optimisation using the ahp there are twelve alternatives corresponding to twelve sets of grinding experiments. there are three infeed which are 10 µm, 20 µm and 30µm, and four environmental conditions. ten grinding passes are conducted at each experimental run. values of different response parameters that are evaluated for obtaining the optimum grinding performance in this work are shown in tables 4 through 7. table 4 lists the average tangential force (ft) and normal force (fn) components of each experimental run or alternative. values of force components obtained from the 6 th through the 10 th pass are noted, and their mean values are the averaged ft or fn values. readings of the first five passes were not taken because after the initial four or five passes the grinding process gets somewhat stabilized and starts grinding truly with the set infeed value. it may be noted that the need for a high value of grinding force indicates poor grindability. the depth of the wheel groove occurring at the periphery of the grinding wheel through grit wearing and subsequent dislodgement of abrasive grits is measured after ten grinding passes and is given in table 5. the removal of wheel grits to a large extent is not desired for good grinding; however, a small quantity of wheel material removal is needed to remove chip loaded wheel grits. table 4 average fn and ft force values for different alternatives alternative average fn (n) average ft (n) a1 46.8 12.6 a2 59.4 16.4 a3 84.8 21.8 a4 41.8 10.6 a5 53.2 14.8 a6 59.2 15.6 a7 41 9.4 a8 42.6 13 a9 52 13.6 a10 38 8.6 a11 39.4 11.2 a12 44.8 10.6 the surface characteristic is determined from observations of surface roughness, surface burn, presence of grit indentation/redeposition of chips and presence of surface crack. table 6 shows the weight given to those parameters and also the ratings given to different alternatives accordingly. a surface roughness rating is decided based on a 9 point scale as there are a wide range of roughness values possible during grinding, while for the others a 5 point scale rating is used. the experimental runs that give higher surface roughness, higher surface burn, large grit indentation or chip redeposition and surface cracks are assigned a lower rating, so that a large value surface characteristic (sc) corresponds to desired grindability. for example, experimental run a1 and a2 are assigned a surface roughness rating ‘1’ as high average surface roughness values are obtained at these conditions. on the other hand, a rating of ‘9’ is given to experiment run a11 which experiences the lowest surface roughness or high surface finish. similarly, severe surface burn occurred in experiment runs a3 and a6 and a rating of ‘1’ is attributed to these. a rating of ‘5’ is given to those conditions showing no surface burn. in the same way, ratings of ‘1’ or ‘5’ are assigned to the case of large scale grit indentation and chip redeposition, or no such occurrence of indentation or redeposition respectively. a rating of ‘5’ indicates an absence of any surface crack following the same consideration. the weight of ijahp article: chaudhury, mandal, das/ selection of appropriate fluid delivery technique for grinding titanium grade-1 using the analytic hierarchy process international journal of the analytic hierarchy process 463 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.356 each of these four surface parameters is assigned following a 9 point scale. as the presence of a surface crack in a ground workpiece makes it unusable or not acceptable, a large weight of ‘9’ is attributed to it. occurrence of surface burn and grit indentation/ chip redeposition on the ground surface are the next two important aspects to judge grindability and these are assigned a rating of ‘7’. surface roughness is less important considering grindability than the other three surface parameters, and hence, a weight of ‘5’ is assigned to it. these weights and ratings have been selected by the authors led by s. das based on their experience in grinding research during the last several years. a summation of (weightage*rating) is made for each of the alternatives and given in table 7. this value is divided by the summation of weights to get the relative weights of surface characteristic for each alternative. the table below shows the reading of relative weights for each alternative. the summation of weights = (5+7+7+9) = 28. it is evident that is the higher the value of weighted averages, the better the surface characteristic of ground surface showing good grindability. table 5 wheel wear obtained after each experiment alternatives wheel wear (µm) a1 60 a2 300 a3 460 a4 30 a5 80 a6 200 a7 40 a8 100 a9 150 a10 50 a11 80 a12 100 ijahp article: chaudhury, mandal, das/ selection of appropriate fluid delivery technique for grinding titanium grade-1 using the analytic hierarchy process international journal of the analytic hierarchy process 464 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.356 table 6 weight and rating of surface parameters for different alternatives to determine surface characteristic alternatives surface roughness surface burn grit indentation/ chip redeposition surface crack wt. rating wt. rating wt. rating wt. rating a1 5 2 7 3 7 3 9 5 a2 1 2 3 2 a3 1 1 1 2 a4 5 5 3 5 a5 3 3 3 5 a6 4 1 1 2 a7 6 5 5 5 a8 8 5 5 5 a9 5 2 3 2 a10 8 5 5 5 a11 9 5 5 5 a12 7 2 3 2 as was already mentioned, the objective or goal of this ahp problem is the selection of an optimum grinding condition to obtain the best grindability for a titanium grade 1 workpiece within the domain of the experiments performed. the criteria are grinding force (ft and fn), wheel wear (w) and surface characteristic (sc). there are twelve alternatives to correspond with the twelve sets of grinding conditions. the hierarchy structure of this ahp problem is shown in figure1. table 7 evaluation of weighted average of surface characteristic (sc) for each alternative alternatives ∑(weight*ratings) weighted averages = ∑(weights*ratings)/28 a1 97 3.46 a2 58 2.07 a3 37 1.32 a4 126 4.5 a5 102 3.64 a6 52 1.86 a7 145 5.18 a8 155 5.54 a9 78 2.79 a10 155 5.54 a11 160 5.71 a12 88 3.14 ijahp article: chaudhury, mandal, das/ selection of appropriate fluid delivery technique for grinding titanium grade-1 using the analytic hierarchy process international journal of the analytic hierarchy process 465 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.356 table 8 pair-wise comparison matrix for criteria goal grinding force (ft & fn) wheel wear (w) surface characteristic (sc) geometric mean (gm) criteria weight ft & fn 1 5 1/3 1.18563 0.27178 w 1/5 1 1/8 0.2924 0.06703 sc 3 8 1 2.8845 0.6612 λm= 3.0455, cr= 0.007075. table 9 pair-wise comparison matrix for alternatives for criterion 1 (grinding force, ft & fn) ft & fn a1 a2 a3 a4 a5 a6 a7 a8 a9 a10 a11 a12 gm local wt. a1 1 4 6 1/3 3 4 1/3 1/3 3 1/5 1/4 1/2 0.98 0.05 a2 1/4 1 3 1/5 1/2 1 1/5 1/5 1/2 1/6 1/6 1/4 0.38 0.02 a3 1/6 1/3 1 1/8 1/4 1/3 1/8 1/7 1/4 1/9 1/9 1/7 0.20 0.01 a4 3 5 8 1 4 5 1/2 2 4 1/4 1/3 3 1.91 0.11 a5 1/3 2 4 1/4 1 2 1/4 1/4 1/2 1/6 1/5 1/3 0.53 0.03 a6 1/4 1 3 1/5 1/2 1 1/5 1/5 1/3 1/6 1/6 1/4 0.37 0.02 a7 3 5 8 2 4 5 1 2 4 1/4 1/3 3 2.15 0.12 a8 3 5 7 1/2 4 5 1/2 1 4 1/4 1/3 2 1.63 0.09 a9 1/3 2 4 1/4 2 2 1/4 1/4 1 1/6 1/5 1/3 0.59 0.03 a10 5 6 9 4 6 6 4 4 6 1 2 5 4.27 0.24 a11 4 6 9 3 5 6 3 3 5 1/2 1 4 3.31 0.19 a12 2 4 7 1/5 3 4 1/3 1/2 3 1/5 1/4 1 1.15 0.06 the pair-wise comparison matrix for the three criteria with respect to the goal is constructed as per equation 1 and is shown in table 8. local weights are obtained by normalising the geometric means of strength of preferences of each criterion over the other with respect to the goal. it is understood that grinding force is more important than wheel wear, but surface characteristic had a larger influence than the other criteria considering grindability. the pair-wise comparison matrices for 12 alternatives for each of the 3 criteria are constructed, and local weights are calculated. weights of the alternative matrices are chosen based on the experimental observations as detailed in tables 3 through 7. maximum eigen value, λm and consistency ratio, cr are computed for each pair-wise comparison matrix, and cr is found to be well below 10% for these matrices indicating consistency of the matrices (tables 8 through 11). the global weights (ri) of the alternatives are determined following equation 3 and are given in table 12 arranged in a decreasing order. it can be seen that the global weight of alternative a11 is the maximum that corresponds to an infeed of 20µm with the compound nozzle cooling arrangement. so, it can be concluded that this corresponds to the optimum grindability condition for a titanium grade 1 workpiece when the alumina wheel is used. therefore, the condition of 20µm infeed with compound nozzle cooling arrangement can be recommended for surface grinding operations in related industry that match the obtained surface characteristic. the next best condition that can also be adopted corresponds to an infeed of 10µm with the compound nozzle cooling arrangement. ijahp article: chaudhury, mandal, das/ selection of appropriate fluid delivery technique for grinding titanium grade-1 using the analytic hierarchy process international journal of the analytic hierarchy process 466 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.356 table 10 pair-wise comparison matrix for alternatives for criterion 2 (wheel wear) wheel wear a1 a2 a3 a4 a5 a6 a7 a8 a9 a10 a11 a12 gm local wt. a1 1 6 7 1/4 2 5 1/3 3 4 1/2 2 3 1.81 0.10 a2 1/6 1 2 1/8 1/5 ½ 1/7 1/4 1/3 1/7 1/5 1/4 0.29 0.02 a3 1/7 1/2 1 1/9 1/6 1/3 1/8 1/5 1/3 1/8 1/6 1/5 0.22 0.02 a4 4 8 9 1 4 7 2 5 6 2 4 5 4.05 0.23 a5 1/2 5 6 1/4 1 4 1/3 1/2 3 1/3 1 2 1.28 0.07 a6 1/5 2 3 1/7 1/4 1 1/6 1/3 1/2 1/6 1/4 1/3 0.4 0.02 a7 3 7 8 1/2 3 6 1 4 5 2 3 4 3.08 0.18 a8 1/3 4 5 1/5 1/2 3 ¼ 1 2 1/4 1/2 1 0.84 0.05 a9 1/4 3 4 1/6 1/3 2 1/5 1/2 1 1/5 1/3 1/2 0.57 0.03 a10 2 7 8 1/2 3 6 ½ 4 5 1 3 4 2.65 0.15 a11 1/2 5 6 1/4 1 4 1/3 1/2 3 1/3 1 2 1.28 0.07 a12 1/3 4 5 1/5 1/2 3 ¼ 1 2 1/4 1/2 1 0.84 0.05 table 11 pair-wise comparison matrix for alternatives for criterion 3 (surface characteristic) sc a1 a2 a3 a4 a5 a6 a7 a8 a9 a10 a11 a12 gm local wt. a1 1 2 3 1/4 1/2 3 1/5 1/6 1 1/6 1/7 1/3 0.54 0.03 a2 ½ 1 2 1/5 1/3 1 1/6 1/7 1/2 1/7 1/8 1/4 0.35 0.02 a3 1/3 1/2 1 1/6 1/4 1/2 1/7 1/8 1/3 1/8 1/9 1/5 0.25 0.01 a4 4 5 6 1 3 5 1/2 1/3 4 1/3 1/4 2 1.56 0.09 a5 2 3 4 1/3 1 3 1/4 1/5 2 1/5 1/6 1/2 0.76 0.04 a6 1/3 1 2 1/5 1/3 1 1/6 1/7 1/2 1/7 1/8 1/4 0.34 0.02 a7 5 6 7 2 4 6 1 1/2 5 1/2 1/3 3 2.2 0.12 a8 6 7 8 3 5 7 2 1 6 1 1/2 4 3.12 0.17 a9 1 2 2 1/4 1/2 2 1/5 1/6 1 1/6 1/7 1/3 0.52 0.03 a10 6 7 7 3 5 7 2 1 6 1 1/2 4 3.12 0.17 a11 7 8 8 4 6 8 3 2 7 2 1 5 4.31 0.24 a12 3 4 4 1/2 2 4 1/3 1/4 3 1/4 1/5 1 1.1 0.06 table 12 global weights for alternatives alternatives global weights a11 0.21328 a10 0.19029 a8 0.14215 a7 0.12528 a4 0.10204 a12 0.06110 a1 0.04188 a5 0.04102 a9 0.03036 a2 0.01989 a6 0.01969 a3 0.01306 ijahp article: chaudhury, mandal, das/ selection of appropriate fluid delivery technique for grinding titanium grade-1 using the analytic hierarchy process international journal of the analytic hierarchy process 467 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.356 6. conclusions the analytical hierarchy process (ahp) is employed in this work to determine the optimum conditions for obtaining the desired grindability for the grinding of titanium grade-1 with an alumina grinding wheel. the three criteria that were considered are grinding force, wheel wear and surface characteristic. the alternative a11 was determined to be the optimum condition and employed the compound nozzle with 20 m infeed. this result is also agreeable with the experimental findings. the compound nozzle system of grinding fluid delivery may have penetrated the stiff air layer around the grinding wheel thereby suppressing a steep rise in grinding zone temperature, and hence, reducing thermal related problems in grinding. however, at 30 m infeed, large material removal takes place and force and temperature are naturally higher than at a lower infeed. therefore, the optimal condition of 20 m infeed is justified. it can be said that the ahp can be efficiently used for solving multiple objective decision making problems such as finding out the best grinding condition to have desire grindability. ijahp article: chaudhury, mandal, das/ selection of appropriate fluid delivery technique for grinding titanium grade-1 using the analytic hierarchy process international journal of the analytic hierarchy process 468 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.356 references biswas, d., sarkar, a., mandal, b. & das, s. (2012). exploring grindability of titanium grade 1 using silicon carbide wheel. reasona technical magazine, 11, 3946. brinksmeier, e., heinzel, c. & wittmann, m. (1999). friction, cooling and lubrication in grinding. annals of the cirp, 48(2), 581-598. doi:10.1016/s00078506(07)63236-3 ebbrell, s., woolley, n.h., tridimas, y.d., allanson, d.r. & rowe, w.b. (2000). the effect of cutting fluid application methods on the grinding process. international journal of machine tools and manufacture, 40, 209–223. doi:10.1016/s08906955(99)00060-7 engineer, f., guo, c. & malkin, s. (1992). experimental measurement of fluid flow through the grinding zone. transactions of the asme, journal of engineering for industry, 114, 61-66. doi:10.1115/1.2899759 huang, h.z., li, y.h. & xue, l.h. (2005). a comprehensive evaluation model for assessments of grinding machining quality. key engineering materials, 291-292, 157-162. doi: 10.4028/www.scientific.net/kem.291-292.157 inasaki, i. (1998). fluid film in the grinding arc of contact. meeting of the cirp, paris, 27, 31-35. irani, r.a., bauer, r.j. & warkentin, a. (2005). a review of cutting fluid application in the grinding process. international journal of machine tools and manufacture, 45, 1696-1705. doi: 10.1016/j.ijmachtools.2005.03.006 mandal, b., das, g.c., das, s. & banerjee, s. (2014). improving grinding fluid delivery using pneumatic barrier and compound nozzle. production engineering research and development, 8, 187–193. mandal, b., majumder, s., das, s. & banerjee, s. (2011a). formation of a significantly less stiff air-layer around a grinding wheel pasted with rexine leather. international journal of precision technology, 2(1), 12-20. mandal, b., singh, r., das, s. & banerjee, s. (2011b). improving grinding performance by controlling air flow around a grinding wheel. international journal of machine tools and manufacture, 51, 670-676. mandal, b., singh, r., das, s. & banerjee, s. (2012). development of a grinding fluid delivery technique and its performance evaluation. materials and manufacturing processes, 27, 4, 436-442. mandal, b., biswas, d., sarkar, a., das, s. & banerjee, s. (2013). improving grindability of titanium gr.1 using a pneumatic barrier. reasona technical journal, 12, 37-45. morgan, m.n., jackson, a.r., wu, h., baines-jones, v., batako, a. & rowe, w.b. (2008). optimisation of fluid application in grinding. cirp annals– manufacturing technology, 57, 363-366. doi: 10.1016/j.cirp.2008.03.090 http://dx.doi.org/10.1016/s0007-8506(07)63236-3 http://dx.doi.org/10.1016/s0007-8506(07)63236-3 http://dx.doi.org/10.1016/s0890-6955(99)00060-7 http://dx.doi.org/10.1016/s0890-6955(99)00060-7 ijahp article: chaudhury, mandal, das/ selection of appropriate fluid delivery technique for grinding titanium grade-1 using the analytic hierarchy process international journal of the analytic hierarchy process 469 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.356 palhade, r.d., tungikar, v.b. & dhole, g.m. (2009). application of different environments in grinding of titanium alloys (ti-6al-4v): investigations on precision brazed type monolayered cubic boron nitride (cbn) grinding wheel. journal of the institution of engineers (india)– production engineering division, 90, 9-13. parthasarathy, a. & malkin, s. (2010). effect of fluid application conditions on grinding behaviour. proceedings of the institute of mechanical engineering, part b: journal of engineering manufacture, 224, 225-235. doi: 10.1243/09544054jem1526 rowe, w.b. (2009). principles of modern grinding technology. 1 st ed., u.k.: elsevier. sabiruddin, k., bhattacharya, s. & das, s. (2013). selection of appropriate process parameters for gas metal arc welding of medium. international journal of analytic hierarchy process, 5(2), 252-267. doi: 10.13033/ijahp.v5i2.184 saaty, t.l. (1977). a scaling method for priorities in hierarchical structures. journal of mathematical psychology, 15(3), 234-281. doi: 10.1016/0022-2496(77)90033-5 saaty, t.l. (1980). analytic hierarchy process. new york: mcgraw-hill. shi, x., liu, g., wang, q. & kang, r. (2008). study on a grinding quality assessing method based on fuzzy decision combined with analytic hierarchy process (ahp). key engineering materials, 359-360, 543-547. sun, j., ge, p. & liu, z. (2001). two-grade fuzzy synthetic decision-making system with use of an analytic hierarchy process for performance evaluation of grinding fluids. tribology international, 34(10), 683–688. doi: 10.1016/s0301679x(00)00152-3 teicher, u., ghosh, a., chattopadhyay, a.b. & kunanz, k. (2006). on the grindability of titanium alloy by brazed type monolayered superabrasive grinding wheels. international journal of machine tools and manufacture, 46, 620-622. doi: 10.1016/j.ijmachtools.2005.07.012 turley, d.m. (1985). factor affecting surface finish when grinding titanium and a titanium alloy (ti-6al-4v). wear, 104, 323–335. doi:10.1016/0043-1648(85)900407 vargas, l.g. (1990). an overview of the analytic hierarchy process & its applications. european journal of operations research, 48, 72–80. wang, m., yu, a.b. & zang, y. (2006). grindability evaluation of advanced ceramics with ahp approach. key engineering materials, 304-305, 222-226. wu, h., lin, b., cai, r. & morgan, m.n. (2007). measurement of the air boundary layer on the periphery of a rotating grinding wheel using lda. journal of physics: conference series, 76(1), paper no. 012059. doi: 10.1088/1742-6596/76/1/012059 xu, x., yu, y. & huang, h. (2003). mechanisms of abrasive wear in the grinding of titanium (tc4) and nickel (k417) alloys. wear, 255, 1421-1426. doi:10.1016/s00431648(03)00163-7 http://dx.doi.org/10.1016/0043-1648(85)90040-7 http://dx.doi.org/10.1016/0043-1648(85)90040-7 http://dx.doi.org/10.1016/s0043-1648(03)00163-7 http://dx.doi.org/10.1016/s0043-1648(03)00163-7 ijahp article: garg, ganesh/an analytical hierarchy process approach for a covid-19 risk assessment study amid the latest re-open and unlock phase in india international journal of the analytic hierarchy process 565 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.814 an analytical hierarchy process approach for a covid-19 risk assessment study amid the latest re-open and unlock phase in india arpan garg research scholar; dept. of mathematics & scientific computing national institute of technology hamirpur, india arpan1996garg@gmail.com talari ganesh assistant professor; dept. of mathematics & scientific computing national institute of technology hamirpur, india ganimsc2007@gmail.com abstract the long duration of the covid-19 pandemic has now compelled people to come out of their cells and begin engaging in various activities using different modes and in different places, which involves a high risk of being infected by covid-19. this work aims to analyze the degree of the risk involved in such activities with the implementation of the latest re-open and unlock phase in india, the second most covid-19 affected country. seven important activities and the mode and place of conducting these activities are considered as the main criteria and sub-criteria of this study. the responses are formed with a pairwise comparison matrix, and an analytical hierarchy process (ahp) approach is used to calculate the criteria/sub-criteria weights. an integrated ahp analysis output representation consisting of the degree of risk, grades, ranking, and major reasons of risk for each criterion and sub-criteria of this work is presented. this risk assessment study may help identify risky activities so that people can choose other available alternatives. keywords: covid-19; ahp; risk assessment; mcdm; decision-making 1. introduction the novel corona virus (covid-19), or syndrome corona virus-2 (sars-cov-2), originated in wuhan city of hubei province in china and has become a worldwide pandemic (wang et al., 2020). the common symptoms of covid-19 include dry cough, fever, malaise, and headache (wang et al., 2020) which are normal symptoms of sickness as the seasons change which makes it tedious to verify unless a check-up for covid-19 is done. the who has found that an infected person can transmit the virus to others who are within 6 feet. however, recent studies have found that possible virus transmission can occur over a distance of 2 meters (setti et al., 2020). as of 10:10 gmt, october 10, 2020, worldometer (2020) had reported over 37,155,642 confirmed cases including 1,073,338 deaths worldwide. according to the registered data, the united states of america, brazil and india are most covid-19 affected nations. to prevent a possibly mailto:arpan1996garg@gmail.com mailto:ganimsc2007@gmail.com ijahp article: garg, ganesh/an analytical hierarchy process approach for a covid-19 risk assessment study amid the latest re-open and unlock phase in india international journal of the analytic hierarchy process 566 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.814 more critical situation, the government of india ordered a nationwide lockdown (2020) on march 24, 2020 which initially came into force for 21 days starting on march 25, 2020 (lockdown implementation order, 2020). due to the lockdown implementation, all important but non-essential activities such as traveling, gathering, use of delivery services, entertainment activities, and many other important activities from a financial point of view were stopped. in order to maintain social distancing, a series of unlock phases was soon started due to the financial and management requirements. phase 4 re-open and unlock was announced through the ministry of home affairs website on august 29, 2020, which allows various activities to take place with some restrictions. the exponential increase in the number of cases and casualties has become a major concern for all the health bodies including the who and the indian council for medical research (icmr) because for the last few days, india has continuously recorded more than 75,000 cases per day (worldometer, 2020). the easing of restrictions for conducting various essential activities that was issued in the latest phase of unlock guidelines are applicable to more than 1.38 billion citizens of india (population of india, 2020) which potentially increases the covid-19 risk. this situation has motivated the authors to conduct a risk assessment study for many important daily life activities. this study aims to calculate the risk involved in many activities that are likely to take place in this phase of unlock. in this work, the authors tried to list various activities, and the mode and place of conducting these activities with their relative risk of covid-19 infection. the authors have applied the analytical hierarchy process (ahp) with the row geometric mean method (crawford and williams, 1985) to calculate relative risk weights of the main criteria and sub-criteria. the activities and mode or place of conducting an activity are considered as criteria and sub-criteria, respectively in this study. the risk weight of the criteria/sub-criteria helps identify extremely risky activities. 2. focus of the work sars-cov-2 has become one of the worst healthcare emergencies that does not yet have a promising treatment or vaccination available, which makes a relative risk assessment of various activities and the places of conducting these activities very important to minimize the fatal growth rate of infections. this work highlights seven main activities that are necessary for normal living and likely to take place in response to the latest re-open and unlock phase. the seven activities include job, shopping, gathering, entertainment, traveling, residency, and utilizing delivery services and have been listed as our main criteria. the various modes and places of conducting these activities based on their expected strength and possibility of contact with different people are considered as the sub-criteria of this study. the main criteria and sub-criteria with their notations are listed in table 1. in this work, the standard layer hierarchy and the proper hierarchical structure representing all of the main criteria and sub-criteria of this study are respectively shown in figures1 and 2. the analyzed risk of conducting the main activities are listed in table 2 and the cumulative risk weight, ranking, and description as well as the major factors and reasons are discussed in table 3. this work helps people understand the risk involved in the mode of conducting an activity at some particular place so that available ijahp article: garg, ganesh/an analytical hierarchy process approach for a covid-19 risk assessment study amid the latest re-open and unlock phase in india international journal of the analytic hierarchy process 567 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.814 alternatives can be considered or the activity can be avoided. it is important to remain safe and clear during this pandemic period. table 1 main criteria and their sub-criteria criteria sub-criteria job 1 a hospital 11 a , government office 12 a , private office 13 a shop 2a grocery store 21a , vendor 22a , shopping mall 23a gathering 3 a marriage ceremony 31 a , cremation ceremony 32 a , conference and seminar 33 a , worship place 34 a entertainment 4a stadium 41a , movie hall 42a , zoo and water park 43a traveling 5 a personal vehicle 51 a , bus, train and taxi 52 a , airline and ship 53 a residency 6 a joint family 61 a , nuclear family 62 a , hosteller 63 a utilizing delivery services 7 a food delivery 71 a , e-commerce 72 a 3. ahp solution the analytical hierarchy process (ahp), developed by saaty (1977), is a highly practical procedure that helps solve complex multicriteria decision making (mcdm) problems. the ahp hierarchy structure is a three layer structure (chaiyaphan and ransikarbum, 2020) as shown in figure 1, where the top, middle and final layer of the hierarchy structure are i) goal or target ii) criteria, sub-criteria (if any), and iii) possible choice or solution to the problem, respectively. figure 1 hierarchy structure of ahp ijahp article: garg, ganesh/an analytical hierarchy process approach for a covid-19 risk assessment study amid the latest re-open and unlock phase in india international journal of the analytic hierarchy process 568 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.814 the ahp has widely been used in healthcare, risk assessment, construction, supply chain, automotive, and many other fields by researchers and statisticians. simon et al. (2019) applied the ahp model for priority analysis of various strategies for malaria control. improta et al. (2019) applied the ahp for health technology assessment (hta) of optoelectronic biosensors for oncology. rajak and shaw (2019) performed an ahp analysis for evaluation and selection on mobile health (mhealth) applications. also, the ahp has been an impressive support system for researchers in risk assessment studies. lyu et al. (2020) used fuzzy ahp in the risk assessment of mega city infrastructure, and rivera et al. (2020) applied the ahp to study the risk factors for the spread of covid-19 in peru. the application of the ahp requires the sequential fulfillment of several major steps (rosenbloom, 1997) listed below: 1. break down the problem into a hierarchy of a finite number of decision elements. 2. compute the pairwise comparison values for all possible pairs of criteria at every level of the hierarchy structure and construct a pairwise comparison matrix with tolerable inconsistency. according to saaty (1977), the degree of inconsistency can be measured using the expressions given below: 1 max    n ci n ri ci cr  where, ci is the consistency index; max  is the largest eigen value; n is the number of criteria; cr is the consistency ratio, and a ri value 0.58, 0.9 and 1.32 for 4,3n and 7 respectively, is the suggested random index value (saaty, 1987). it is suggested that the cr value must be less than 0.1. 3. compute the preference weight using the suitable ahp analysis. for a pairwise comparison matrix   ij aa  , th i row geometric mean (crawford and williams, 1985) value is: nn j iji aw 1 1            4. rank the decision criteria after aggregating the relative weights. ijahp article: garg, ganesh/an analytical hierarchy process approach for a covid-19 risk assessment study amid the latest re-open and unlock phase in india international journal of the analytic hierarchy process 569 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.814 table 2 pairwise comparison of main criteria, normalized weight and grade criteria 1a 2a 3a 4a 5a 6a 7a criteria weights grades consistency check job 1 a 1 3 1 2 2 6 5 0.2677 a shopping 2 a 0.33 1 0.5 1 0.33 3 3 0.1043 d gathering 3 a 1 2 1 4 2 5 4 0.2632 b 0.0327ci entertainment 4 a 0.5 1 0.25 1 0.5 2 2 0.0947 e 0.0242 =cr travelling 5 a 0.5 3 0.5 2 1 4 3 0.1745 c 1.32 =ri residency 6 a 0.1667 0.33 0.2 0.5 0.25 1 0.5 0.0408 g utilizing delivery services 7 a 0.2 0.33 0.25 0.5 0.33 2 1 0.0548 f ijahp article: garg, ganesh/an analytical hierarchy process approach for a covid-19 risk assessment study amid the latest re-open and unlock phase in india international journal of the analytic hierarchy process 570 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.814 figure 2 hierarchy of activities and their respective places 4. results and discussion the authors have identified seven important activities as the main criteria of this risk assessment study. these activities were specifically chosen by considering the latest gathering restrictions in the phase 4 re-open and unlock (2020) guidelines issued by the ministry of home affairs. also, the mode and place of conducting these selected activities have been identified as their sub-criteria and are shown in table 1. since the ahp is able to provide very reliable output for complex multicriteria decision making (mcdm) problems with little mathematical computation, the authors were able to perform all of the analysis manually using the row geometric mean method with precision (crawford and williams, 1985). ijahp article: garg, ganesh/an analytical hierarchy process approach for a covid-19 risk assessment study amid the latest re-open and unlock phase in india international journal of the analytic hierarchy process 571 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.814 the authors observed that the opinions of different health experts regarding how safe/unsafe a location is are subject to the presence or absence of some pre-determined factors. generally, medical experts/doctors would be the most likely people to provide responses for a healthcare study such as this; however, since they are required to perform their duties to every possible extent during this pandemic the authors have identified some major factors that are favorable to covid-19 virus transmission as follows: i) a high population density or a large number of people present/involved in an activity, ii) maintaining less than 6 feet of social-distancing, and iii) physically close and frequent contact among people. next, the pairwise comparison matrix was shaped by an expert data analyst and statistician based on their experience and understanding of the correlation of identified factors with the criteria and sub-criteria of this study. a consistency analysis was used to ensure that the pcm was tolerably inconsistent. the degree of risk involved in a criterion was calculated as criteria weight by applying the row geometric mean method. all of the criteria were assigned a grade from a to g based on their high to low degree of risk; this is shown in table 2. moreover, the pairwise comparison matrices, consistency-checks, and relative weights for all of the sub-criteria within the same level of the hierarchy are shown in table 4. this ahp analysis for a covid-19 risk assessment presents an integrated outcome representation in table 3. this representation includes all criteria and sub-criteria with their degree of covid-19 risk in the form of criteria and sub-criteria weight, the cumulative sub-criteria weight derived by multiplication of local sub-criteria weight with their respective main criteria weight which leads to form a global sub-criteria weight and ranking. the authors have also expressed various expected reasons of risk involvement in this representation. table 3 integrated outcome representation with expected major reasons of risk s.no criteria /sub-criteria criteria weight subcriteria weight cumulative normalized subcriteria weight grade and ranking expected major reasons of risk 1 a job 0.267682 a frequent contact with external objects makes it the most serious activity of the study 11 a hospital 0.745006 0.199425 1 being in direct contact with virus infected patients, health care workers like doctors, nurses and others are at extreme risk of covid-19 12 a government office 0.098552 0.026380 14 need for continuous operational state for proper implementation of guidelines 13 a private office 0.156441 0.041876 9 the limited space available and lack of sanitization facilities in many private offices 2 a shopping 0.104275 d the basic requirement of goods and other materials has resulted in shopping as a main criterion 21 a grocery store 0.196246 0.020463 17 the reach to a limited number of customers from a specific territory makes shopping at grocery stores less risky than shopping at vendors or the mall 22 a vendors 0.49223 0.051327 7 covering a very large area of the city increases the risk of being in contact with the virus 23 a shopping mall 0.311522 0.032484 11 finite, but sufficiently large customer capacity 3 a gathering 0.263216 b a gathering is the second most risky activity because meeting with a large group of people ijahp article: garg, ganesh/an analytical hierarchy process approach for a covid-19 risk assessment study amid the latest re-open and unlock phase in india international journal of the analytic hierarchy process 572 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.814 proportionally increases the probability of covid19 infection 31 a marriage ceremony 0.222189 0.058483 4 permission for 50 people gathering at a marriage ceremony 32 a cremation ceremony 0.119379 0.031422 12 permission for at most 20 people gathering at a cremation ceremony 33 a conference and seminar 0.200771 0.052846 6 large participation in a conference and seminar 34 a worship place 0.457659 0.120463 2 high visiting frequency and the non registering nature of a worship place 4 a entertainment 0.094716 e need for a healthy mood and good physical health are very essential at this time which demands that entertainment be one of the main criterion of this study 41 a stadium 0.196976 0.018656 18 the risk of coming in contact with sports equipment that may carry the virus, yet stadiums have sufficient space for maintaining distance from others 42 a movie hall 0.311239 0.029479 13 the density of the audience in a movie hall is very high and increases the risk 43 a zoo and water park 0.491783 0.046580 8 possible infection through animals and contaminated water 5 a travelling 0.174485 c a list of requirements demands that one travel by various modes of transportation, which makes it an essential criterion of this study 51 a personal vehicle 0.122542 0.021381 16 traveling by personal vehicle eliminates many possibilities of infection, yet their maintenance requires visits to many public places 52 a bus, train and taxi 0.319130 0.0556835 5 use of the same seats by many people and high capacity of commuters, categorizes use of a bus, train, and taxi as extremely risky 53 a airline and ship 0.558326 0.097419 3 overseas origin of passengers from various countries and a very large capacity of these modes of transportations are the main reasons for the extreme risk. 6 a residency 0.040795 g the residency area of an individual with a number of people is a major concern of this study; however, the ahp analysis finds that it has optimal risk involvement 61 a joint family 0.537895 0.021943 15 a large number of family members makes it slightly risky 62 a nuclear family 0.297788 0.012148 20 small number of family members is the main reason for a nuclear family to have the second to last position in most risky sub-criteria 63 a hosteller 0.164315 0.006703 21 being a hosteller is the last ranked sub-criteria with the minimum risk involved 7 a utilizing delivery services 0.054828 f utilization of various resources without going out is the main reason for inclusion of various delivery and online purchase facilities under the category of ‘utilizing delivery services’ in this study 71 a food delivery 0.666666 0.036552 10 involvement of many hands from cooking to delivery 72 a e-commerce 0.333334 0.018276 19 careful and standard measures, but the product visits many stations ijahp article: garg, ganesh/an analytical hierarchy process approach for a covid-19 risk assessment study amid the latest re-open and unlock phase in india international journal of the analytic hierarchy process 573 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.814 table 4 pair wise comparison of sub-criteria with consistency check and weight criteria subcriteria subcriteria weight consistency check 1 a 11 a 12 a 13 a 11 a 1 6 6 0.745006 0.0268=ci 12 a 0.1667 1 0.5 0.098552 0.046=cr 13 a 0.1667 2 1 0.156441 2 a 21 a 22 a 23 a 21 a 1 0.5 0.5 0.196246 0.0269=ci 22 a 2 1 2 0.492230 0.0464=cr 23 a 2 0.5 1 0.311522 3 a 31 a 32 a 33 a 34 a 31 a 1 2 1 0.5 0.222189 32 a 0.5 1 0.5 0.33 0.119379 0.0136 =ci 33 a 1 2 1 0.33 0.200771 0.0151=cr 34 a 2 3 3 1 0.457659 4 a 41 a 42 a 43 a 41 a 1 0.5 0.5 0.196976 0.0268=ci 42 a 2 1 0.5 0.311239 0.0462 =cr 43 a 2 2 1 0.491783 5 a 51 a 52 a 53 a 51 a 1 0.33 0.25 0.122542 0.0077=ci 52 a 3 1 0.5 0.319130 0.0133=cr 53 a 4 2 1 0.558326 6 a 61 a 62 a 63 a 61 a 1 2 3 0.537895 0.0027ci 62 a 0.5 1 2 0.297788 0.0047cr 63 a 0.33 0.5 1 0.164315 7 a 71 a 72 a 71 a 1 2 0.666666 0ci 72 a 0.5 1 0.333334 ijahp article: garg, ganesh/an analytical hierarchy process approach for a covid-19 risk assessment study amid the latest re-open and unlock phase in india international journal of the analytic hierarchy process 574 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.814 5. conclusions covid-19 is spreading exponentially and for a developing country like india with a population of 1.38 billion (population of india, 2020) it has become an extremely tedious health care management problem. the government of india has promptly responded to the pandemic and imposed a nationwide lockdown (2020). many restrictions have been eased in a sequence of re-open and unlock phases. in the latest phase 4 re-open and unlock (2020) guidelines, the authors have observed that large gatherings, interstate transportation, utilization of various delivery services and many other activities are no longer under strict control which might be responsible for a sudden increase in covid19 cases. to analyze the degree of covid-19 risk involved in many important activities through different modes and at different places, the authors conducted a covid-19 risk assessment using a very popular mcdm technique, the analytic hierarchy process (ahp). a simple and structured application of the ahp has derived very significant outcomes. the risk assessment reflects that doing a job has the highest risk of covid-19 infection among all seven listed activities because it requires frequent contact with external objects that may carry covid-19. moreover, gathering is an occasional event, but has the potential possibility of covid-19 transmission to a large number of people coming from different places which makes it the second most risky activity of the study, followed by traveling, shopping, entertainment, utilizing delivery services and residency. the authors found that cumulative sub-criteria weights and global sub-criteria ranking were very helpful in identifying the most risky places to conduct the corresponding activity. based on the rankings, the hospital is the most risky work station and healthcare workers such as doctors, nurses and other medical staff are at extreme risk of covid-19 infection because they are in direct contact with covid-19 patients. the absence of proper identification and record placement policies and the frequency of visiting are the expected reasons for a place of worship to be the second most risky sub-criteria of this study. all of the other sub-criteria and their global rankings and major reasons for risk are explained in table 3. the authors also conclude that being a hosteller and living in a nuclear family are two of the least risky sub-criteria when compared to others. in this covid-19 risk assessment, the authors tried to express the real time suitability of the ahp approach. the authors suggest that many other healthcare management problems can be analyzed with a large number of different activities in future. this study can potentially help select a suitable alternative activity or mode and place of conducting an activity to avoid various risk factors involved with the parent activity. ijahp article: garg, ganesh/an analytical hierarchy process approach for a covid-19 risk assessment study amid the latest re-open and unlock phase in india international journal of the analytic hierarchy process 575 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.814 references badillorivera, e., fow-esteves, a., alata-lopez, f., viru-vasquez, p., &acuña, m. (2020). environmental and social analysis as risk factors for the spread of the novel coronavirus (sars-cov-2) using remote sensing, gis and analytical hierarchy process (ahp): case of peru. medrxiv, preprint. doi: https://doi.org/10.1101/2020.05.31.20118653 chaiyaphan, c., &ransikarbum, k. (2020). criteria analysis of food safety using the analytic hierarchy process (ahp)a case study of thailand’s fresh markets. e3s web of conferences, 141, 02001. doi: https://doi.org/10.1051/e3sconf/202014102001 crawford, g., & williams, c. (1985). a note on the analysis of subjective judgment matrices. journal of mathematical psychology, 29(4), 387-405. doi: https://doi.org/10.1016/0022-2496(85)90002-1 lockdown implementationorder (2020). available :https://www.mha.gov.in/sites/default/files/pr_nationallockdown_26032020_0.pdf improta, g., perrone, a., russo, m. a., & triassi, m. (2019).health technology assessment (hta) of optoelectronic biosensors for oncology by analytic hierarchy process (ahp) and likert scale. bmc medical research methodology, 19(1), 140. doi: https://doi.org/10.1186/s12874-019-0775-z lyu, h. m., shen, s. l., zhou, a., & yang, j. (2020). risk assessment of mega-city infrastructures related to land subsidence using improved trapezoidal fahp. science of the total environment, 717, 135310. doi: https://doi.org/10.1016/j.scitotenv.2019.135310 phase 4 re-open and unlock (2020).availablehttps://www.mha.gov.in/sites/default/files/mhaorder_unlock4_290820 20.pdf population of india (2020).available: https://www.worldometers.info/worldpopulation/india-population/. rajak, m., & shaw, k. (2019).evaluation and selection of mobile health (mhealth) applications using ahp and fuzzy topsis. technology in society, 59, 101186. doi: https://doi.org/10.1016/j.techsoc.2019.101186 rosenbloom, e. s. (1997). a probabilistic interpretation of the final rankings in ahp. european journal of operational research, 96(2), 371-378. doi: https://doi.org/10.1016/s0377-2217(96)00049-5 saaty, r. w. (1987). the analytic hierarchy process—what it is and how it is used. mathematical modelling, 9(3-5), 161-176. doi: https://doi.org/10.1016/02700255(87)90473-8 https://www.mha.gov.in/sites/default/files/pr_nationallockdown_26032020_0.pdf https://www.mha.gov.in/sites/default/files/mhaorder_unlock4_29082020.pdf https://www.mha.gov.in/sites/default/files/mhaorder_unlock4_29082020.pdf https://www.worldometers.info/worldpopulation/india-population/ ijahp article: garg, ganesh/an analytical hierarchy process approach for a covid-19 risk assessment study amid the latest re-open and unlock phase in india international journal of the analytic hierarchy process 576 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.814 saaty, t. l. (1977). a scaling method for priorities in hierarchical structures. journal of mathematical psychology, 15(3), 234-281. doi: https://doi.org/10.1016/00222496(77)90033-5 saaty, t. l. (1990). how to make a decision: the analytic hierarchy process. european journal of operational research, 48(1), 9-26. doi: https://doi.org/10.1016/03772217(90)90057-i setti, l., passarini, f., de gennaro, g., barbieri, p., perrone, m. g., borelli, m., ...&miani, a. (2020). airborne transmission route of covid-19: why 2 meters/6 feet of inter-personal distance could not be enough. international journal of environmental research and public health.17(8): 2932. doi: 10.3390/ijerph17082932 simon, j., adamu, a., abdulkadir, a., & henry, a. s. (2019). analytical hierarchy process (ahp) model for prioritizing alternative strategies for malaria control. asian journal of probability and statistics, 1-8. doi: https://doi.org/10.9734/ajpas/2019/v5i130124 wang, c., horby, p. w., hayden, f. g., &gao, g. f. (2020). a novel coronavirus outbreak of global health concern. the lancet, 395(10223), 470-473. doi: https://doi.org/10.1016/s0140-6736(20)30185-9 worldometers.covid-19 coronavirus pandemic. available: https://www.worldometers.info/coronavirus/country./india/. https://www.ncbi.nlm.nih.gov/pmc/articles/pmc7215485/ https://www.ncbi.nlm.nih.gov/pmc/articles/pmc7215485/ https://dx.doi.org/10.3390%2fijerph17082932 https://www.worldometers.info/coronavirus/country./india/ wsc' 16 preparing manuscripts ijahp article: dehghanimohammadabadi, kabadayi/a two-stage ahp multi-objective simulation optimization approach in healthcare international journal of the analytic hierarchy process 117 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.701 a two-stage ahp multi-objective simulation optimization approach in healthcare mohammad dehghanimohammadabadi mechanical and industrial engineering department northeastern university boston, ma, usa m.dehghani@northeastern.edu nihan kabadayi department of production management istanbul university school of business avcilar, istanbul, turkey nihank@istanbul.edu.tr. abstract quality of care is crucial for patients' satisfaction and safety in healthcare centers. the majority of hospitals attempt to implement facility-wide improvements to ensure highquality care delivery. this study proposes a combined simulation-optimization (so) and multi-criteria decision making (mcdm) approach to accurately assess the impact of quality improvement initiatives on different facets of the healthcare system. in this framework, first, the importance (weights) of the different healthcare criteria is determined by health providers using the analytic hierarchy process (ahp) approach. then, the weights provided by the ahp are applied in a simulation-optimization environment to determine the most efficient action that leads to the most desirable quality of care. simulation provides a platform to examine the effectiveness of different improvement efforts and calculate their impact on the system performance measures. the proposed model is generic enough to be applied to similar problems in different domains. keywords: analytic hierarchy process; multi-criteria decision making; simulationoptimization; healthcare operations; simheuristic; simio 1. introduction simulation is a promising technique to study complex, stochastic, and non-linear systems. within simulation, the discrete-event simulation (des) is a popular approach with the ability to mimic the dynamics of real systems (moon & phatak, 2005). des provides a well-established mechanism for many types of modeling processes (alt & lieberman, 2010) and is an effective decision-making tool for tactical and operational level decisions (dehghanimohammadabadi, 2016). using des, a decision-maker can compare different solutions (scenarios) and evaluate their impact on the system’s performance. however, determining the most efficient and practical solution requires a great deal of effort with careful analysis to ensure reliable results. this becomes even more challenging in problems with multiple responses (criteria), in which the system needs to be optimized with respect to multiple objectives. mailto:m.dehghani@northeastern.edu mailto:nihank@istanbul.edu.tr ijahp article: dehghanimohammadabadi, kabadayi/a two-stage ahp multi-objective simulation optimization approach in healthcare international journal of the analytic hierarchy process 118 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.701 the existing des software packages are usually equipped with a built-in optimizer tool, such as optquest. this feature enables users to optimize the expected performance of the simulated model based on pre-determined objectives or criteria. in the case of multiobjective problems, these optimizers can use a weighted sum method to transform all of the objectives into a single-objective by varying weights in [0, 1]. therefore, determining the weights of objectives (criteria) is a critical and somewhat challenging step in deploying multi-objective simulation-optimization (mso). to address this challenge, this study proposes a general conceptual framework for integrating one of the most extensively used mcdm methods, the analytic hierarchy process (ahp), with a des module. saaty’s ahp (saaty, 1980) is a pairwise comparison method designed to capture relative judgments in a manner that ensures consistency (chen, 2006). this algorithm provides an effective procedure to deal with complex decision-making and can assist in identifying and weighing criteria (pun et al., 2017). as depicted in figure 1, the development of this model has two stages. the first stage takes advantage of ahp to include several decision-makers’ preferences to determine the weight of the objectives. in the second stage, the so model takes the weights calculated from the ahp to form a weighted sum of objective functions and combines them into a scalar fitness function. using the new single-objective model, so explores the solution space in order to obtain the best configuration for the simulation model. therefore, the main contributions of this paper are listed below:  developing a hybrid mcdm-so model considering multiple objectives,  providing implementation aspects of the proposed model by integrating three modules, and  promoting the advantage of the new proposed model in a healthcare setting. the rest of this paper is organized as follows. section 2 reviews some related literature and discusses the novelty of this work. in section 3, an effort is made to explain the implementation details of the proposed model and interactions between its components. to illustrate the applicability of this hybrid model, a healthcare case study is discussed in section 4 followed by a sensitivity analysis. this paper is concluded in section 5 by giving some remarks and future speculations. figure 1 general structure of the integrated simulation-optimization and mcdm framework ijahp article: dehghanimohammadabadi, kabadayi/a two-stage ahp multi-objective simulation optimization approach in healthcare international journal of the analytic hierarchy process 119 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.701 2. literature review the combination of simulation and mcdm methods has been used in different areas of research. hsu and pan (2009) proposed an integrated model to rank dental quality attributes. in this model, monte carlo simulation and the ahp method are combined to determine the service quality dimensions of dental services. their application results show that the quality perceptions of patients and service providers are different. eskandari et al. (2011) offered an integrated model to reduce waiting times of patients in an emergency department (ed) at a governmental hospital in tehran, iran. in the proposed model, the simulation was used to create different scenarios of patient flow processes, while the ahp and the technique for order preference by similarity to ideal solution (topsis) methods were applied to evaluate the performance of alternative scenarios. baccouche et al. (2011) developed a two-stage model to solve a supply chain design problem. in the first stage of the model, they used the crowding clustering genetic algorithm (ccga) to measure the performance of alternative designs using simulation. after they obtained enough alternatives with acceptable performance in the first step, they aimed to determine a collective design that could satisfy the expectations of all of the members of the decision-making team in the second step of the problem. a multiplicative variant of the popular ahp method was applied for the second stage of the problem. in a study conducted by meng (2015), an integrated des and ahp method to develop a model to improve the process of the design of a grafting operation was proposed. gul et al. (2016) combined a computer simulation method with mcdm methods for interval type-2 fuzzy ahp and electre to evaluate the performance of an ed in a university hospital. they suggested that the integrated method was a suitable method to assess the performance of the ed. in addition, the method helps to determine the optimal number of nurses and doctors for three shifts by trying different scenarios. baležentis & streimikiene (2017) conducted a study to develop a multi-criteria ranking model to determine the most suitable energy planning for the european union (eu). their model was a combination of three methods, weighted aggregated sum/product assessment (wasps), the additive ratio assessment (aras) method, and topsis, to evaluate alternative energy planning scenarios. in the proposed model, monte carlo simulation was used to generate the egalitarian weights to supplement mcdm methods. in another study, bamakan and dehghanimohammadabadi (2015) introduced a new quantitative risk analysis and assessment methodology by integrating ahp and monte carlo simulation. in this article, the ahp is applied to create favorable weights for security characteristic criteria. then, a monte carlo simulation is utilized to handle the stochastic nature of risk assessment. none of the existing models address the applicability of mcdm approaches to perform a multi-objective simulation-optimization. this study aims to encourage the advancement of mcdm models such as ahp, within the simulation optimization environment. the proposed framework takes advantage of both worlds for the first time to create a decision support system (dss) based on a simulation model. 3. the two-stage ahp-multi-objective so framework structure the proposed model deploys simulation-optimization to solve stochastic problems with multiple objectives. this hybrid model consists of two main modules (i) an mcdm ijahp article: dehghanimohammadabadi, kabadayi/a two-stage ahp multi-objective simulation optimization approach in healthcare international journal of the analytic hierarchy process 120 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.701 module that deploys ahp, and (ii) a simulation-optimization module that optimizes the performance of the simulated system. this framework starts with a decision-making module (ahp), where experts’ opinions are taken into account to attain the relative importance of performance criteria and their weights. by summing the objective functions multiplied by weighting coefficients, the multi-objective model is transformed into a single-objective function. then, in an iterative manner, so leverages this single-objective function to find an optimal or closeto-optimal configuration of the simulated system. in the following sections, an effort is made to explain the importance of each of these components and their detailed procedure, followed by a discussion about their interaction. 3.1 decision making module: analytic hierarchy process (ahp) the ahp is one of the popular mcdm methods, which was developed by saaty (1980). since then, it has been used to solve different kinds of decision-making problems (supplier selection, facility location analysis, forecasting, choice of technology, risk modeling, performance evaluation, etc.) in the literature. it is an appropriate method for analyzing complex real-life problems as it allows experts to incorporate their knowledge and experience to generate a solution. one of the main advantages of this method is its relative ease in handling multiple criteria. in addition, the ahp allows both qualitative and quantitative data to be evaluated effectively. therefore, it is a suitable method to solve healthcare performance evaluation problems, which include both tangible and intangible criteria. the ahp application process can be summarized in six steps as listed below: step 1: define the problem and construct the hierarchy of the problem; step 2: compute weights of the criteria with the help of pairwise comparisons of experts’ judgment. construct a pairwise comparison matrix (size 𝑛 × 𝑛) which is composed of the values that describes the relative importance between two alternatives. in this step, pairwise comparisons are performed using saaty’s (1980) fundamental 9-point scale; step 3: normalize pairwise comparison matrix; step 4: calculate the weights of alternatives using the pairwise comparison matrix. the weight vector (𝑤) is calculated by computing the average of each row of the decision matrix developed in the first step; step 5: measure the consistency within the pairwise comparison matrix. the consistency index (𝐶𝐼) is computed using equation 1, where 𝜆𝑚𝑎𝑥 is the unique largest eigenvalue and 𝑛 is the matrix size. 𝐶𝐼 = 𝜆𝑚𝑎𝑥−𝑛 𝑛 − 1 (1) step 6: finally, the acceptance of the consistency of each judgment matrix is tested. the consistency ratio (𝐶𝑅) is defined using equation 2, where an average random consistency index (𝑅𝐼) is used according to the size of each comparison matrix. the decision is accepted when 𝐶𝑅 < 0.1. ijahp article: dehghanimohammadabadi, kabadayi/a two-stage ahp multi-objective simulation optimization approach in healthcare international journal of the analytic hierarchy process 121 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.701 𝐶𝑅 = 𝐶𝐼 𝑅𝐼 (2) it needs to be noted that, in any ahp analysis, the number of experts depends on many factors, namely their availability, the level of their heterogeneity, experience, and domain knowledge (karczmarek et al., 2017). obviously, having a large number of experts’ opinions helps the aggregation process to be more effective and makes the differences between the preferences distinct. as a result, the weights provided by the ahp would be more reliable to form a robust single-objective function to be used by the optimization module. the following sections describe how the ahp results are utilized in an so model to solve the problem. 3.2 optimization module: genetic algorithm (ga) many studies could successfully employ metaheuristics to develop simulationoptimization models and solve large-scale complex stochastic problems with reasonable computer resource consumption. these models are called simheuristics, where a metaheuristic algorithm is used in conjunction with a simulation model to find the optimal or near-to-optimal configuration of the simulation model (dehghanimohammadabadi et al., 2017; juan et al., 2015). genetic algorithm (ga) is one of the popular metaheuristic algorithms, which mimics biological evolution (dehghanimohammadabadi & keyser, 2017). ga is based on the assumption that the potential solution of any problem can be represented by a set of parameters that are referred to as genes of a chromosome. the degree of goodness of the chromosome for the problem is reflected by a positive fitness value related to the objective value of the problem (man, tang & kwong, 1999). a random population of 𝑁 individuals, which is composed of potential solutions to the problem, is created at the beginning of the ga search. then, these individuals are evaluated for their so-called fitness, i.e. in this case, the weighted-sum of the model’s criteria. then, individuals with the higher fitness scores are selected to create a mating pool of size 𝑁 . this created population evolves in successive generations steps until a predetermined termination condition is satisfied (marseguerra, zio & podofillini, 2002). genetic operators, such as crossover and mutation are applied in a probabilistic manner to some individuals from the mating pool to produce offspring for the next generation (chambers, 2019). these operators aim to create a new generation that contains better offspring (solution) to the problem the crossover operator is based on the exchange of subtrees while the mutation is based on the random change in the tree (kokol et al., 2012). ga performance could be affected by the setting values for various parameters, such as crossover rate, population size, and mutation rate. therefore, parameter optimization is one of the critical steps of ga. this algorithm is designed to keep a delicate balance between the exploration of the feasible domain and the exploitation of good solutions (carson & maria, 1997). due to its wide applicability and efficiency, ga has been successfully used in several so studies such as facility layout optimization (azadivar & wang, 2000), risk management (yin, win & hsu, 2017), scheduling (dahghanimohammadabadi, 2016; al-dhaheri, jebali & diabat, 2016), and supply chain management (göçken, 2017, 2015). as a result, ga is selected to complete this hybrid model. ijahp article: dehghanimohammadabadi, kabadayi/a two-stage ahp multi-objective simulation optimization approach in healthcare international journal of the analytic hierarchy process 122 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.701 the ga process can be summarized in four steps as listed below: step 1: selection of a pair of individuals as parents that are going to transmit their genetic material to the next generation; step 2: crossover of the parents, with generation of two children; step 3: genetic mutation is applied to maintain the diversity of the population; step 4: replacement in the population, so as to maintain the population number n constant. since this model leverages a combination of simulation and optimization, each generated solution in ga needs to be tested in a simulated environment. details of the simulation model approach and its interaction with the optimization module are described in the following sections. 3.3 simulation module: discrete event simulation (des) discrete event simulation is a great tool to model complex systems and the interactions between individuals and their environments (alshaebi et al., 2017). this approach helps decision-makers take advantage of running the model under different settings and configurations and evaluate their impact to obtain the most desirable settings. the following section describes the implementation aspects of the proposed framework and how the simulation module is integrated with the optimization module to perform so through specialized software. 3.4 framework implementation to implement the proposed framework, three software packages including ms excel, matlab, and simio are linked together to optimize a model’s performance with multiple criteria. the holistic view of this framework structure and its components are shown in figure 2. in the first stage of the model, ahp analysis is performed to calculate the criteria weights. this calculation can be performed relatively quickly using microsoft excel. then, the attained weights are transferred to the second stage to execute the so operation. this stage includes optimization (ga) and simulation (des) modules which are iteratively linked together to optimize the model. due to its computational power and wide set of functions, matlab is used as the main platform to deploy ga. matlab has a large community of committed users who are figure 2 holistic view of the proposed two-stage ahp-multi-objective simulation optimization framework ijahp article: dehghanimohammadabadi, kabadayi/a two-stage ahp multi-objective simulation optimization approach in healthcare international journal of the analytic hierarchy process 123 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.701 developing and sharing algorithms (belevich et al., 2016; ozgur et al., 2017), which also increases the chance of this generic model being used by other users. more importantly, the interaction between matlab and the applied simulation model is seamless due to the advancement conducted by dehghanimohammadabadi and keyser (dehghanimohammadabadi & keyser, 2017). finally, the des package that is being used in this study is simio, which is developed in c# (vieira et al., 2016) and enables a user to perform customized operations. the integration of simio and matlab proposed by dehghanimohammadabdi and keyser (2015) makes the development of the decision support system feasible and easy-toimplement. the pseudocode presented in figure 3, clearly shows the important steps of the stages and their interactions. after performing ahp analysis in the first stage, the so model is initiated. in this stage, in every iteration of ga, a number of solutions are generated through cross-over and mutation operators. each solution provides a new configuration of the simulation model, which basically determines a specific value for all of the simulation model controls. by generating each solution in ga, the simulation model is triggered to obtain the expected performance of the given solution and its fitness. in other words, the simulation model acts as a cost function (evaluation function) of the ga, in which solutions will be tested. after stopping criteria are met, the algorithm ends its run and releases the best-obtained solution up to that point. figure 3 pseudocode of the proposed two-stage ahp-multi-objective simulation optimization framework ijahp article: dehghanimohammadabadi, kabadayi/a two-stage ahp multi-objective simulation optimization approach in healthcare international journal of the analytic hierarchy process 124 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.701 4. case study: using the ahp-so model in a healthcare setting a healthcare case study is discussed in this section to demonstrate the applicability of the proposed model. it needs to be noted that the main goal of this section is to discuss implementation aspects of the work, and its general capacity to be replicated by researchers to solve simulation-optimization problems with multiple objectives. therefore, a typical example from the simio library is used to validate the model’s efficiency and illustrate its great potential and prospective applications. the selected simulation model is “hospitalemergencydepartment.spfx” which represents a small emergency department. this simulation model is available to all simio users in the “\documents\simio\examples” directory and is accompanied by a very detailed documentation. interested readers are referred to the model details from the provided documentation by simio in the same directory “hospitalemergencydepartment.pdf”. this ed model includes a waiting area, a registration desk, a triage room, a radiology station, a billing area, 6 beds and 6 rooms that are used for patients that are admitted into the hospital. 4.1 ahp analysis in this model, the ahp algorithm is used to determine the weights (importance) of the objectives that are applied in the multi-objective simulation model. the input of the ahp algorithm is the pairwise comparison results provided by the members of the decisionmaking team including the financial director of the hospital, physicians and nurses who work in the ed. ms excel is used to deploy ahp because of its popularity, ubiquity, and more importantly, its compatibility with matlab. after compiling judgments, the obtained weights for all of the criteria are transferred to matlab to execute the simulation-optimization process. the aim of this model is to determine the best combination of controls, number of nurses, number of doctors, and number of registration desk staff to improve the ed performance measures. multiple criteria are considered to evaluate the performance of the system. a hierarchical structure is created to provide decision makers with a visual presentation to assist with creating the pairwise comparisons. as illustrated in figure 4, the hierarchy figure 4 ahp structure and criteria of ed ijahp article: dehghanimohammadabadi, kabadayi/a two-stage ahp multi-objective simulation optimization approach in healthcare international journal of the analytic hierarchy process 125 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.701 of this model consists of three levels: the first level comprises the main goal, the second level comprises the performance measures, and the last level includes three sub-criteria for each of these criterion. the main goal of the problem is defined as the best configuration of the emergency department and performance measurement criteria are determined as patient satisfaction, resource utilization, and system performance. each of these criteria consists of three sub-criteria, which are described as follows.  average length of stay: the average time between a patients admission and discharge  average wait time for a bed: the average amount of time that a patient waits for a bed to become available  average wait time for a room\physician: the average time a patient waits for a room to become available and be visited by the physician  physician utilization: the percentage of time the physician spends with a patient  nurse utilization: the percentage of time a nurse spends with a patient and provides care  room/bed occupancy rate: the percentage of a time a room/bed is occupied  cost of staff: the total cost of healthcare providers based on usage cost rate and idle cost rate  number of diversions: the number of times that a new patient fails to enter the ed due to the lack of an available room  total leave without being seen (lwbs): the number of patients who arrive in the main entrance but leave because of either a long waiting time or a full waiting area to understand the importance of these factors and their influence on the model, experts’ judgments are collected using the ahp to obtain the weights of each criterion and subcriteria. the responses of five experts are used to perform the pairwise comparison and get local weights of the main criteria and the sub-criteria. the calculated cr value for all of the analysis is less than 0.1, which guarantees the quality of the results. interested readers can find the experts’ responses and the details of the ahp analysis at this link. table 1 lists the ahp results and global weights of the er sub-criteria. these weights are then used to form the weighted sum multiple-objective function in the so section. https://www.dropbox.com/s/tfa41lh551vnwmv/ahpfinalresults.xlsx?dl=0 ijahp article: dehghanimohammadabadi, kabadayi/a two-stage ahp multi-objective simulation optimization approach in healthcare international journal of the analytic hierarchy process 126 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.701 table 1 ahp results for global weights of the er criteria and sub-criteria main criteria local weights sub-criteria local weights global weights patient satisfaction 0.274 avg. length of stay 0.111 0.031 avg. wait time for a bed 0.475 0.130 avg. wait time to see doctor 0.414 0.114 resource utilization 0.155 physician utilization 0.582 0.091 nurse utilization 0.253 0.039 room/bed occupation rate 0.165 0.026 system preference 0.570 cost of staff 0.305 0.174 number of diversions 0.108 0.062 lwbs 0.587 0.335 sum 1.000 1.000 as can be seen from table 1, the most important criterion related to the quality of the hospital ed is determined as lwbs with 33.5 %. the rate of patients who leave without being seen (lwbs) by a physician in eds has critical importance in terms of the safety of human life and the quality of care in hospitals. therefore, it is usually a major concern for healthcare providers and used as an ed performance evaluation metric in hospitals. according to the results, cost of staff is also another important criterion that determines the performance of ed. besides, average waiting time for a bed and average waiting time to see a doctor are also significantly important metrics for ed performance due to their negative impacts on the patients’ satisfaction and health. 4.2 simulation-optimization model the calculated weights from ahp are applied to formulate the multi-objective simulation-optimization problem. as shown in equation (3), a weighted sum formula transforms all of the objectives (𝑓𝑖 ) into a single-objective function (𝑧) . all of the objective functions are weighted using global weights obtained from the ahp analysis (𝑤𝑖 ). since different objectives have different units and magnitudes, all of the objective functions are divided by their max value (𝑓𝑖 𝑚𝑎𝑥 ) to be scaled between [0,1]. 𝑧 = ∑ 𝑤𝑖 𝑓𝑖 𝑚𝑎𝑥 𝑓𝑖 𝑚 𝑖=1 (3) the simheuristic model used the weighted sum objective function to perform the best combination of controls. the defined range for the model inputs (number of nurses, number of doctors, and number of registration desk staff) is defined between min=2 and max=10. therefore, in each iteration, ga provides a new combination of inputs, and after a certain number of iterations, returns the best-ever-found results. for this case study, a max-iteration of ga is set to 20, with a population size of 10, and a simio replication size of 100. ijahp article: dehghanimohammadabadi, kabadayi/a two-stage ahp multi-objective simulation optimization approach in healthcare international journal of the analytic hierarchy process 127 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.701 4.3 results the proposed hybrid model is developed using matlab r2018a, and simio 11. after 20 iterations and generating 230 solutions, ga determined the best model configuration. the solution results of the so model suggest five (5) nurses, (4) doctors, and (2) registration desk staff for the ed model (table 2). the simulation helps to run the model with multiple replications and estimate the expected performance level for each of the criteria. respective descriptive statistics of 100 simulation replications are tabulated in table 3 for all of the criteria. these results show physician utilization has the highest variability (half-width = 0.595) while cost of staff experienced less change through the replications (half-width = 0.002). the ahp analysis determined lwbs, cost of staff, and wait time to see doctor as the top three measures for the ed model. therefore, to analyze and better understand these factors, a graphical representation of results is provided in figure 5. as shown in figure 5-(a), there are only 12 cases where the system observes lwbs (12%), and on the positive cases, the values are very minimal (lwbs < 3). this aligned with the ahp results where lwbs attained the highest weight which emphasizes its tendency to reduce the number of unvisited patients. the other two box-plots in figure 5-(b, c), provide insights regarding cost of staff, and wait time to see the doctor. these results indicate that, under the optimal configuration of the model, it is most likely that a patient is visited by a doctor within 0.65 hours of his/her arrival. table 2 final model configuration (best ga solution) solution values number of nurses, 5 number of doctors 4 number of registration desk staff 2 table 3 performance of the ed model based on the best ga solution main criteria sub-criteria (responses) mean min max halfwidth patient satisfaction avg. length of stay 42.276 33.325 58.360 1.171 avg. wait time for a bed 14.711 12.787 17.559 0.176 avg. wait time to see doctor 0.580 0.534 0.697 0.005 resource utilization physician utilization 57.975 50.527 65.445 0.595 nurse utilization 47.987 41.864 54.171 0.489 room/bed occupation rate 33.148 26.506 39.779 0.522 system preference cost of staff 44.166 44.138 44.196 0.002 number of diversions 0.250 0.000 3.000 0.119 lwbs 0.163 0.000 2.823 0.104 ijahp article: dehghanimohammadabadi, kabadayi/a two-stage ahp multi-objective simulation optimization approach in healthcare international journal of the analytic hierarchy process 128 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.701 4.4 sensitivity analysis to evaluate the stability of the priority ranking, a sensitivity analysis is performed by varying the main criteria weights of ahp. this shows how altering the ranking factors could affect the final results and the selection process. in this section, three scenarios are studied. as indicated in equation 4, the weight (𝑊𝑖) of each of the main criteria (𝐹𝑖) is thought to change (one at a time), while the summation of new weights (𝑊𝑖 ′) adds up to 1 (equation 5). three control parameters namely 𝛼 , 𝛽 , and 𝛾 are defined to quantify changes to patient satisfaction, resource utilization, and system preference weights, respectively. 𝑧 = (1 + 𝛼)𝑊1𝐹1 + (1 + 𝛽)𝑊2𝐹2 + (1 + 𝛾)𝑊3𝐹3 (4) ∑ 𝑊𝑖 ′ = 1 3 𝑖 (5) as tabulated in table 4, in each scenario, one of the main criteria is changed in three levels (100%, 200%, and 300%) and its impact is evaluated on the ed model configuration. therefore, in each setting, a new set of weights is obtained for the main criteria and the sub-criteria to consequently form a new objective function. then, the so model is deployed based on the new objective function to find the best configuration of the simulated model. this analysis helps to figure out which of the main criteria is the most critical with the highest impact on the ed system. figure 6 illustrates a heatmap of new weights in each scenario and provides an overview perspective of how the importance of criteria changes from one scenario to another. details of scenarios and their corresponding results are discussed as follows: figure 5 simulation results of top three criteria based on the best ga solution ijahp article: dehghanimohammadabadi, kabadayi/a two-stage ahp multi-objective simulation optimization approach in healthcare international journal of the analytic hierarchy process 129 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.701 figure 6 heatmap of new weights in each scenario * weights with higher values are highlighted with a darker color α=100% α=200% α=300% β=100% β=200% β=300% γ=100% γ=200% γ=300% avg. length of stay 0.030 0.048 0.059 0.067 0.026 0.023 0.021 0.019 0.014 0.011 avg. wait time for a bed 0.130 0.204 0.252 0.286 0.113 0.099 0.089 0.083 0.061 0.048 avg. wait time to see doctor 0.113 0.178 0.220 0.249 0.098 0.087 0.077 0.072 0.053 0.042 physician utilization 0.090 0.071 0.058 0.050 0.156 0.207 0.246 0.057 0.042 0.033 nurse utilization 0.039 0.031 0.025 0.022 0.068 0.090 0.107 0.025 0.018 0.014 room/bed occupation rate 0.026 0.020 0.017 0.014 0.044 0.059 0.070 0.016 0.012 0.009 cost of staff 0.174 0.137 0.113 0.096 0.151 0.133 0.119 0.222 0.244 0.257 number of diversions 0.062 0.048 0.040 0.034 0.053 0.047 0.042 0.079 0.086 0.091 lwbs 0.335 0.263 0.217 0.184 0.290 0.256 0.229 0.427 0.469 0.494 sum 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 sce nario 1 sce nario 2 sce nario 3 basesub-crite ra table 4 experimental setting for the ahp sensitivity analysis main criteria scenarios 𝜶 𝜷 𝜸 patient satisfaction 1 100%, 200%, 300% 0% 0% resource utilization 2 0% 100%, 200%, 300% 0% system preference 3 0% 0% 100%, 200%, 300% 4.4.1. scenario 1: patient satisfaction sensitivity analysis in this scenario, the first main decision factor, patient satisfaction, is subject to change. the importance of patient satisfaction is increased in three steps with three levels, 𝛼=100%, 200%, and 300%. as is evident in table 5, by increasing change levels, the weight of patient satisfaction is increased and this increment is compromised by the decreasing weight of the other two criteria. table 5 scenario 1 setting patient satisfaction sensitivity analysis main criteria base model 𝛼 = 0% level 1 𝛼 = 100% level 2 𝛼 = 200% level 3 𝛼 = 300% patient satisfaction 27.40% 29.34% 31.17% 32.91% resource utilization 15.50% 15.09% 14.69% 14.32% system preference 57.10% 55.58% 54.13% 52.76% sum 100.00% 100.00% 100.00% 100.00% under each setting, the so model is executed to understand how the model setting changes and how sensitive it is to the patient satisfaction measures. as depicted in figure 7, by putting more weight on patient satisfaction and its sub-criteria (average length of stay, average wait time for a bed, and average wait time to see doctor), number of nurses and number of doctors in the model remain unchanged. the only decision variables that are liable to change is number of registration desk staff which changes from 2 to 5 in order to improve the quality of care considering more weight for the patient satisfaction. ijahp article: dehghanimohammadabadi, kabadayi/a two-stage ahp multi-objective simulation optimization approach in healthcare international journal of the analytic hierarchy process 130 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.701 this seems rational since the cost of registration desk staff is relatively lower than doctors and nurses, and could increase the efficiency of the ed process. insight 1: increasing the importance of the patient satisfaction criteria increases the number of registration desk staff while the other two controls remain the same. 4.4.2. scenario 2: resource utilization sensitivity analysis the second factor that is changed to analyze its effect is resource utilization. this main criteria includes three sub-criteria, namely physician utilization, nurse utilization, and room/bed occupation rate. similar to the first scenario, this factor is changed in three levels (𝛽=100%, 200%, and 300%). table 6 shows the corresponding weights for these levels and the results are shown in figure 8. since this scenario aims to increase resource utilization, the number of doctors is reduced in all levels by 1 (from 4 to 3) to make doctors busier and make their utilization larger. interestingly, in contrast to number of doctors, the number of registration desk staff is increased from 2 to 6. this huge change is mainly due to the fact that utilization of these staff is not taken into account. the only utilization measures included in this study is for nurses and doctors, and therefore, the reduction in doctors staffing in the model is compensated for by increasing the registration desk staff. insight 2: physician utilization is one of the sub-criteria of the resource utilization factor. therefore, by increasing 𝛽, the number of doctors is reduced (from 4 to 3) to improve all physicians utilization. table 6 scenario 2 setting resource utilization sensitivity analysis main criteria base model 𝛽 = 0% level 1 𝛽 = 100% level 2 𝛽 = 200% level 3 𝛽 = 300% patient satisfaction 27.40% 26.98% 26.58% 26.18% resource utilization 15.50% 16.79% 18.04% 19.25% system preference 57.10% 56.23% 55.38% 54.56% sum 100.00% 100.00% 100.00% 100.00% figure 7 scenario 1 sensitivity analysis results ijahp article: dehghanimohammadabadi, kabadayi/a two-stage ahp multi-objective simulation optimization approach in healthcare international journal of the analytic hierarchy process 131 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.701 4.4.3. scenario 3: system preference sensitivity analysis similar to the first two scenarios, the third main factor of the ahp analysis is studied under three levels (table 7) and the results are depicted in figure 9. these results suggest keeping the number of doctors and nurses untouched in all three levels of 𝛾. the only observed change is for number of registration desk staff, which slightly increases as 𝛾 increases. this implies that, even though increasing number of staff implies more cost to the system, it facilitates lowering the number of diversions, and lwbs. insight 3: the only input variable that is subject to 𝛾 changes is the number of registration desk staff. table 7 scenario 3 setting system preference sensitivity analysis main criteria base model 𝛾 = 0% level 1 𝛾 = 100% level 2 𝛾 = 200% level 3 𝛾 = 300% patient satisfaction 27.40% 25.92% 24.59% 23.39% resource utilization 15.50% 14.66% 13.91% 13.23% system preference 57.10% 59.42% 61.50% 63.37% sum 100.00% 100.00% 100.00% 100.00% figure 8 scenario 2 sensitivity analysis results ijahp article: dehghanimohammadabadi, kabadayi/a two-stage ahp multi-objective simulation optimization approach in healthcare international journal of the analytic hierarchy process 132 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.701 5. conclusion and future works this work proposes an integrated ahp-so based model to solve multi-objective stochastic problems. by integrating ahp with a so model, this framework provides a realistic weight for the objective functions and makes the so model structure more reliable. this framework proceeds in two stages. in the first stage, ahp is used to prioritize the given system performance measures and evaluate their weights. then, in the second stage, the provided weights from the ahp are used to transform all of the objective functions into a single-objective function in the so model. at this stage, the so model runs the simulated model to determine the best configuration of the system using the provided single-objective function. to show the applicability of the proposed framework, this model is applied to configure an emergency department staff setting. in this case study, the so model determines the best staffing level for nurses, doctors, and registration desk staff by considering multiple objective functions including patient satisfaction, resource utilization, and system preference. finally, a sensitivity analysis is applied to investigate the influences of performance criteria on the ed model configuration. this paper encourages the advancement of mcdm models such as ahp, within the simulation optimization environment. therefore, researchers and practitioners in both fields can benefit from this model and deploy it to other task domains. it is a simple structure that makes it easy to implement and generic enough to be applied in many disciplines. however, this promising approach is in its initial stage and can be extended in many directions. from the mcdm point of view, there are plenty of mcdm approaches that can be used to facilitate the so model construction. a study can extensively combine a variety of mcdm methods with a so model and evaluate their performance under different circumstances. considering a fuzzy mcdm approach with so is also desirable since both approaches address uncertainty in the model. figure 9 scenario 3 sensitivity analysis results ijahp article: dehghanimohammadabadi, kabadayi/a two-stage ahp multi-objective simulation optimization approach in healthcare international journal of the analytic hierarchy process 133 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.701 references al-dhaheri, n., jebali, a. and diabat, a. (2016). a simulation-based genetic algorithm approach for the quay crane scheduling under uncertainty. simulation modelling practice and theory, 66, 122–138. doi: https://doi.org/10.1016/j.simpat.2016.01.009 alshaebi, a., dauod, h., weiss, j. and yoon, s.w. (2017). evaluation of different forklift battery systems using statistical analysis and discrete event simulation. iie annual conference. proceedings, 1637–1642. alt, j.k. and lieberman, s. (2010). representing dynamic social networks in discrete event social simulation. proceedings of the 2010 winter simulation conference, 1478–1489. doi: https://doi.org/10.1109/wsc.2010.5679046 azadivar, f. and wang, j. (2000). facility layout optimization using simulation and genetic algorithms. international journal of production research, 38(7), 4369–4383.doi: https://doi.org/10.1080/00207540050205154 baccouche, a. ,goren, s., huyet, a.l. and pierreval, h. (2011). an approach based on simulation optimization and ahp to support collaborative design: with an application to supply chains. 2011 ieee workshop on computational intelligence in production and logistics systems (cipls), 1– 7. doi: https://doi.org/10.1109/cipls.2011.5953360 baležentis, t. and streimikiene, s. (2017). multi-criteria ranking of energy generation scenarios with monte carlo simulation. applied energy, 185, 862–871. doi: https://doi.org/10.1109/cipls.2011.5953360 bamakan, s.m.h. and dehghanimohammadabadi, m. (2015). a weighted monte carlo simulation approach to risk assessment of information security management system. international journal of enterprise information systems (ijeis), 11(4), 63–78. doi: https://doi.org/10.4018/ijeis.2015100103 belevich, i., joensuu, m., kumar, d.,vihinen, h. and jokitalo, e. (2016). microscopy image browser: a platform for segmentation and analysis of multidimensional datasets. plos biology, 14(1), e1002340. doi: https://doi.org/10.1371/journal.pbio.1002340 carson y. and maria, a. (1997). simulation optimization: methods and applications. proceedings of the 29th conference on winter simulation, 118–126. chambers, l.d. (2019). the practical handbook of genetic algorithms: new frontiers, volume ii. chen, c.f. applying the analytical hierarchy process (ahp) approach to convention site selection. journal of travel research, 45(2), 167–174. doi: https://doi.org/10.1177/0047287506291593 dehghanimohammadabadi, m. (2016). iterative optimization-based simulation (ios) with predictable and unpredictable trigger events in simulated time, western new england university. doi: https://doi.org/10.1109/wsc.2015.7408423 dehghanimohammadabadi, m., rezaeiahari, m. and keyser, t.k. (2017). simheuristic of patient scheduling using a table-experiment approach simio and matlab integration application. proceedings of the 2017 winter simulation conference. ijahp article: dehghanimohammadabadi, kabadayi/a two-stage ahp multi-objective simulation optimization approach in healthcare international journal of the analytic hierarchy process 134 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.701 dehghanimohammadabadi, m. and keyser, t.k. (2017). intelligent simulation: integration of simio and matlab to deploy decision support systems to simulation environment. simulation modelling practice and theory, 71, 45–60. doi: https://doi.org/10.1109/wsc.2017.8248015 dehghanimohammadabadi, m. and keyser, t.k. (2015). smart simulation: integration of simio and matlab | simio. proceedings of the 2015 winter simulation conference. doi: https://doi.org/10.1109/wsc.2017.8248015 eskandari, h., riyahifard, m., khosravi, s. and geiger, c.d. (2011). improving the emergency department performance using simulation and mcdm methods. proceedings of the 2011 winter simulation conference (wsc), 1211–1222. doi: https://doi.org/10.1109/wsc.2011.6147843 göçken, m., dosdoğru, a.t., boru, a. and geyik, f. (2017). characterizing continuous (s, s) policy with supplier selection using simulation optimization. simultion, 0037549716687044. doi: https://doi.org/10.1177/0037549716687044 göçken, m., dosdoğru, a.t., boru, a. and geyik, f. (2015). (r,s,s) inventory control policy and supplier selection in a two-echelon supply chain: an optimization via simulation approach. proceedings of the winter simulation conference, california, usa. doi: https://doi.org/10.1109/wsc.2015.7408320 gul, m., celik, e., gumus, a.t. and guneri, a.f. (2016). emergency department performance evaluation by an integrated simulation and interval type-2 fuzzy mcdm-based scenario analysis. european journal of industrial engineering, 10(2), 196–223. doi: https://doi.org/10.1504/ejie.2016.075846 hsu, t.h. and pan, f.f. (2009). application of monte carlo ahp in ranking dental quality attributes. expert systems with applications, 36(2), 2310–2316. doi: https://doi.org/10.1016/j.eswa.2007.12.023 juan, a.a., faulin, j., grasman, s.e., rabe, m. and figueira, g. (2015). a review of simheuristics: extending metaheuristics to deal with stochastic combinatorial optimization problems. operations research perspectives, 2, 62–72. doi: https://doi.org/10.1016/j.orp.2015.03.001 karczmarek, p., pedrycz, w., kiersztyn, a., and rutka, p. (2017). a study in facial features saliency in face recognition: an analytic hierarchy process approach. soft computing, 21(24), 7503–7517. doi: https://doi.org/10.1007/s00500-016-2305-9 kokol, p., pohorec, s., štiglic, g. and podgorelec, v. (2012). evolutionary design of decision trees for medical application. wiley interdisciplinary reviews: data mining and knowledge discovery, 2(3), 237–254. doi: https://doi.org/10.1002/widm.1056 man, k.f. tang, k.s. and kwong, s. (1999). genetic algorithms: concept and design. london: springer verlag. marseguerra, m., zio, e., and podofillini, l. (2002). condition-based maintenance optimization by means of genetic algorithms and monte carlo simulation. reliability engineering & system safety, 77(2), 151–165. doi: https://doi.org/10.1016/s0951-8320(02)00043-1 ijahp article: dehghanimohammadabadi, kabadayi/a two-stage ahp multi-objective simulation optimization approach in healthcare international journal of the analytic hierarchy process 135 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.701 meng, c. (2015). simulation-based decision support for agricultural supply chain performance improvement. the university of arizona. moon, y.b. and phatak, d. (2005). enhancing erp system’s functionality with discrete event simulation. industrial management & data systems, 105(9), 1206–1224. doi: https://doi.org/10.1108/02635570510633266 ozgur, c., colliau, t., rogers, g., hughes, z. and myer-tyson, b. (2017). matlab vs. python vs. r. journal of data science, 15(3), 355–372. pun, k.p., tsang, y.p., choy, k.l., tang, v. and lam, h.y. (2017). a fuzzy-ahp-based decision support system for maintenance strategy selection in facility management. 2017 portland international conference on management of engineering and technology (picmet), 1–7. doi: https://doi.org/10.23919/picmet.2017.8125300 saaty, t.l. (1980) the analytic hierarchy processs. new york: mcgraw-hill. talbi, e.g. (2009). metaheuristics: from design to implementation. john wiley & sons. vieira, a.a.c., dias, l.s., pereira, g., oliveira, j.a., carvalho, m.s. and martins (2016). automatic simulation models generation of warehouses with milk runs and pickers. 28th european modeling and simulation symposium, emss 2016, 231–241. yin, p.y., wu, t.h. and hsu, p.y. (2017). risk management of wind farm micro-siting using an enhanced genetic algorithm with simulation optimization.” renewable energy, 107, 508–521. doi: https://doi.org/10.1016/j.renene.2017.02.036 author biographies mohammad dehghanimohammadabadi is assistant teaching professor of the mechanical and industrial engineering department at northeastern university (boston, usa). he received his ph.d. in engineering management from western new england university (usa) in 2016. his research is mainly focused on developing and generalizing reinfrocement learning (rl) and simulation-optimization frameworks in different disciplines such as healthcare, supply chain, and manufacturing. his website address is http://www.mie.neu.edu/people/dehghani-mohammad and his e-mail address is m.dehghani@northeastern.edu. nihan kabadayi is assistant professor in the school of business at istanbul university (istanbul, turkey) where she has been a faculty member since 2005. she received her ph.d. in supply chain management from istanbul university in 2013. her research interest includes supply chain management, healthcare operations management, quality management, and multi-criteria decision making. her e-mail address is nihank@istanbul.edu.tr. http://www.mie.neu.edu/people/dehghani-mohammad mailto:m.dehghani@northeastern.edu mailto:nihank@istanbul.edu.tr ijahp essay: salomon/ absolute measurement and ideal synthesis on ahp international journal of the analytic hierarchy process 538 vol. 8 issue 3 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i3.452 absolute measurement and ideal synthesis on ahp valerio a. p. salomon sao paulo state university (unesp) guaratingueta, sp, brazil salomon@feg.unesp.br as in most of the multi-criteria decision analysis (mcda) methods, the application of the analytic hierarchy process (ahp) runs through three major steps: first, structuring; second, measuring; and third, synthesizing. the ways to conduct these three steps makes mcda different from the other methods. originally, ahp application consisted of hierarchical structuring, relative measurement and distributive synthesis (saaty t. l., 1977; saaty t. l., 1980). more than any other mcda method, ahp theory and practice evolved, with different ways to perform the three steps. network structuring, for instance, implies a violation of the axiom of independence (vargas, 1990). that is, instead of hierarchical structuring, criteria and alternatives may depend on or influence each other. as a matter of fact, this generalization of ahp is another mcda method, the analytic network process (anp) (saaty t. l., 1997; ishizaka & nemery, 2013). absolute measurement, also known as “ratings”, and ideal synthesis are different ways to apply ahp (saaty, 1986; millet & saaty, 2000). the combination of ratings with ideal synthesis may bring many advantages for ahp application. two advantages that this combination yields are the ability to increase the set of alternatives to more than nine and ranking (saaty, vargas, & whitaker, 2009). in spite of the advantages of the absolute measurement/ideal synthesis approach, the original relative measurement/normalized synthesis is the preferred way to apply ahp. in the last two years, in volume 7 and volume 8, the international journal of the ahp published 48 papers. only five of these papers addressed theoretical issues, that is, with no application. from thoe 43 practical papers with application, 31 presented applications of ahp alone, seven papers presented applications of anp alone, and five papers bring combinations of ahp with other theories, like fuzzy sets or linear programing. only three papers presented applications of ahp with ratings and ideal synthesis (tramarico, marins, urbina, & salomon, 2015; saaty & wei, 2016; bhandari & nakarmi, 2016). then, in the overwhelming majority of applications, measurement was relative and synthesis were distributive. expert choice and super decisions were the most commonly used software. however, both brands of software enable ratings and ideal synthesis. the purpose of this essay is not to investigate why ahp has not often been applied with absolute measurement and ideal synthesis. my purpose is to call attention to different ways to apply ahp, highlighting three advantages: allowing one set alternatives greater than nine, avoiding rank reversal and providing overall priorities based on ideal priorities. absolute measurement implies in alternatives compared with standard levels, instead of pairwise compared to each other, as in ahp’s original relative measurement. the first advantage of absolute measurement is that there is no boundary for the set of alternatives. in relative measurement, the set of alternatives must be less or equal than nine, or else, “seven, plus or minus two” (saaty & ozdemir, 2003). another advantage to using ratings mailto:salomon@feg.unesp.br ijahp essay / absolute measurement and ideal synthesis on ahp international journal of the analytic hierarchy process 539 vol. 3 issue 1 2016 issn 1936-6744 is the opportunity to avoid biases. in relative measurement, the pairwise comparisons of alternatives can keep some historical trends. absolute measurement seems to provide a less partial or unbiased measurement, comparing alternatives with a standard (salomon, tramarico, & marins, 2016). with ideal synthesis, priorities are not normally distributed. that is, the sum of the priority vectors components will not be equal to one hundred percent. in this way of synthesis, the highest priority regarding each criterion will be equal to one. normalizing priorities creates a dependency among priorities. however, when deleting an old alternative, or inserting a new one, normalized priorities can lead to illegitimate changes in the rank of alternatives, known as rank reversal (rr). rr was firstly associated with ahp (belton & gear, 1983). nevertheless, the application of other mcda methods, such as electre, maut, promethee and topsis, can also lead to rr (triantaphyllou, 2000). combining absolute measurement with ideal synthesis will always preserve ranks (saaty, vargas, & whitaker, 2009). firstly, the discussion shall be on the legitimacy of rr. that is, rr does happen in real world decision problems (saaty & sagir, 2009). in the case rr is not a major concern, then, relative measurement and normal synthesis may be adopted in the ahp application. on the other hand, if the decision-maker is looking for rank preservation, then absolute measurement and ideal synthesis are proper ways to apply ahp. let us consider a decision of project selection by a company. the decision criteria are benefits, opportunities and risks; the alternatives are projects x, y and z. table 1 presents the pairwise comparisons matrix and the priorities of the criteria. table 1 priorities of benefits, opportunities and risks criterion b o r eigenvector priority benefits (b) 1 4 9 3.3 0.74 opportunities (o) 1/4 1 2 0.79 0.18 risks (r) 1/9 1/2 1 0.38 0.09 tables 2, 3 and 4 present the pairwise comparisons matrices and the priorities of the alternatives regarding the criteria. all comparisons matrices are consistent. table 2 priorities of projects regarding benefits project x y z eigenvector priority x 1 5/3 5 2.0 0.56 y 3/5 1 3 1.2 0.33 z 1/5 1/3 1 0.41 0.11 ijahp essay / absolute measurement and ideal synthesis on ahp international journal of the analytic hierarchy process 540 vol. 3 issue 1 2016 issn 1936-6744 table 3 priorities of projects regarding opportunities project x y z eigenvector priority x 1 1/9 1/3 0.33 0.08 y 9 1 3 3 0.69 z 3 1/3 1 1 0.23 table 4 priorities of projects regarding risks project x y z eigenvector priority x 1 1/5 4/5 0.54 0.14 y 5 1 4 2.7 0.69 z 5/4 1/4 1 0.68 0.17 table 5 presents the decision matrix (with local priorities regarding each criterion) and the decision vector (with overall priorities) 1 . due to its highest overall priority, project x will be selected. table 5 local and overall priorities of projects x, y and z project b (0.74 ) o (0.18) r (0.09) overall x 0.56 0.08 0.14 0.44 y 0.33 0.69 0.69 0.43 z 0.11 0.23 0.17 0.14 now, let us suppose that for some reason (for example, problems with raw material imported from distant countries), project z became unfeasible. if the decision were not announced, tables 6, 7,8 and 9 present updating for tables 2,3,4 and 5, just deleting project z. table 6 new priorities of projects regarding benefits project x y eigenvector priority x 1 5/3 1.29 0.63 y 3/5 1 0.77 0.38 1 in the examples in this paper, the lower the r priority, the lower the risks of the project; therefore, the overall priorities can be calculated as a weighted sum. ijahp essay / absolute measurement and ideal synthesis on ahp international journal of the analytic hierarchy process 541 vol. 3 issue 1 2016 issn 1936-6744 table 7 new priorities of projects regarding opportunities project x y eigenvector priority x 1 1/9 0.33 0.10 y 9 1 3 0.90 table 8 new priorities of projects regarding risks project x y eigenvector priority x 1 1/5 0.45 0.17 y 5 1 2.2 0.83 table 9 new local and overall priorities of projects x and y project b (0.74) o (0.18) r (0.09) overall x 0.63 0.10 0.17 0.49 y 0.38 0.90 0.83 0.51 as we can see, in this example a rr occurs. considering project z, project x has a higher priority than project y. however, project z has the lowest priority among the three projects. and, surprisingly, after deleting project z from the decision, project y became the highest priority vector. now, let us apply the ideal synthesis with the same data, that is, with the same comparisons. tables 10,11 and 12 present the local priorities with ideal synthesis. the comparison matrices and the right eigenvectors are the same from tables 2,3 and 4. table 10 ideal priorities of projects regarding benefits project x y z eigenvector priority x 1 5/3 5 2.0 1 y 3/5 1 3 1.2 0.6 z 1/5 1/3 1 0.41 0.2 table 11 ideal priorities of projects regarding opportunities project x y z eigenvector priority x 1 1/9 1/3 0.33 0.11 y 9 1 3 3 1 z 3 1/3 1 1 0.33 ijahp essay / absolute measurement and ideal synthesis on ahp international journal of the analytic hierarchy process 542 vol. 3 issue 1 2016 issn 1936-6744 table 12 ideal priorities of projects regarding risks project x y z eigenvector priority x 1 1/5 4/5 0.54 0.2 y 5 1 4 2.7 1 z 5/4 1/4 1 0.68 0.25 table 13 presents the decision matrix (with local priorities regarding each criterion) and the decision vector (with overall priorities). due to its highest overall priority, project x will be selected, as in table 5. table 13 local and overall priorities with ideal synthesis of projects x, y and z project b (0.74) o (0.18) r (0.09) overall x 1 0.11 0.2 0.77 y 0.6 1 1 0.71 z 0.2 0.33 0.25 0.23 tables 14, 15 and 16 present the new local priorities with ideal synthesis. the comparison matrices are the same from tables 6, 7 and 8 which are the same of table 2, 3 and 4, just deleting project z. table 14 new ideal priorities of projects regarding benefits project x y eigenvector priority x 1 5/3 1.29 1 y 3/5 1 0.77 0.6 table 15 new ideal priorities of projects regarding opportunities project x y eigenvector priority x 1 1/9 0.33 .11 y 9 1 3 1 table 16 new ideal priorities of projects regarding risks project x y eigenvector priority x 1 1/5 0.45 0.2 y 5 1 2.2 1 ijahp essay / absolute measurement and ideal synthesis on ahp international journal of the analytic hierarchy process 543 vol. 3 issue 1 2016 issn 1936-6744 table 17 new local and overall priorities with ideal synthesis of projects x and y project b (0.74) o (0.18) r (0.09) overall x 1 .11 0.2 0.77 y 0.6 1 1 0.71 this numeric example illustrates that, with ideal synthesis, rr can be avoided. that will be important for the decision maker because there are some situations in the real world when rr is unjustifiable, undesired and perhaps unfair. however, another great advantage from ideal synthesis is the value in the priority. that is the “0.77” for project x in tables 13 and 17 represent a concept similar to “utility” (ishizaka & nemery, 2013). this is the degree of satisfaction expected by the decision maker with the selection of project y. for some decision problems it can makes more sense than the “44%” or “49%” from tables 5 and 9. i expect to have made the case in this essay for the convenience of using the absolute measurement/ideal synthesis when applying ahp, in particular by new researchers and users of this mcda method. currently, as we can see in the ijahp papers, ideal synthesis has not been applied, as it could or should be. however, the way one applies ahp is still a question of opinion. this essay does not prove, and i do not intended to prove, that the absolute measurement/ideal synthesis approach is better than the original, with just one example. exhaustive experiments may be able to prove or even clearly identify situations in which one way may be better than another way. this is a great subject for a future research. ijahp essay / absolute measurement and ideal synthesis on ahp international journal of the analytic hierarchy process 544 vol. 3 issue 1 2016 issn 1936-6744 references belton, v., & gear, t. (1983). on a short-coming of saaty’s method of analytic hierarchies. omega, 11(3), 228–230. doi: http://dx.doi.org/10.1016/03050483(83)90047-6 bhandari, a., & nakarmi, a. m. (2016). a financial performance evaluation of commercial banks in nepal using ahp model. international journal of the analytic hierarchy process, 8(2), 318–333. doi: http://dx.doi.org/10.13033/ijahp.v8i2.368 ishizaka, a., & nemery, p. (2013). multi-criteria decision analysis. chichester: wiley. doi: 10.1002/9781118644898 millet, i., & saaty, t. l. (2000). on the relativity of relative measures – accommodating both rank preservation and rank reversals in the ahp. european journal of operational research, 121 , 205–212. doi: http://dx.doi.org/10.1016/s0377-2217(99)00040-5 saaty, t. l. (1977). a scaling method for priorities in hierarchical structures. journal of mathematical psychology, 15(3), 234–281. doi: http://dx.doi.org/10.1016/0022-2496(77)90033-5 saaty, t. l. (1980). the analytic hierarchy process. new york: mcgraw-hill. doi:http://dx.doi.org/10.1080/00137918308956077 saaty, t. l. (1986). absolute and relative measurement with the ahp. the most livable cities in the united states. socio-economic planning sciences, 20(6), 327–331. doi: http://dx.doi.org/10.1016/0038-0121(86)90043-1 saaty, t. l. (1997). the analytic network process. pittsburgh: rws. doi: 10.1007/0-387-33987-6_1 saaty, t. l., & ozdemir, m. s. (2003). why the magic number seven plus or minus two. mathematical and computer modelling, 38(3–4), 233–244. doi:10.1016/s0895-7177(03)90083-5 saaty, t. l., & sagir, m. (2009). an essay on rank preservation and reversal. mathematical and computer modelling, 49(5–6), 1230–1243. doi: http://dx.doi.org/10.1016/j.mcm.2008.08.001 saaty, t. l., & wei, l. (2016). should the uk have brexited the european union. international journal of analytic hierarchy process, 8(2), 206–223. doi: http://dx.doi.org/10.13033/ijahp.v8i2.415 http://dx.doi.org/10.1016/0305-0483(83)90047-6 http://dx.doi.org/10.1016/0305-0483(83)90047-6 http://dx.doi.org/10.13033/ijahp.v8i2.368 http://dx.doi.org/10.1016/s0377-2217(99)00040-5 http://dx.doi.org/10.1016/0022-2496(77)90033-5 http://dx.doi.org/10.1080/00137918308956077 http://dx.doi.org/10.1016/0038-0121(86)90043-1 http://dx.doi.org/10.1016/j.mcm.2008.08.001 http://dx.doi.org/10.13033/ijahp.v8i2.415 ijahp essay / absolute measurement and ideal synthesis on ahp international journal of the analytic hierarchy process 545 vol. 3 issue 1 2016 issn 1936-6744 saaty, t. l., vargas, l. g., & whitaker, r. (2009). addressing with brevity criticism of the analytic hierarchy process. international journal of the analytic hierarchy process, 1(1), 121-134. doi:http://dx.doi.org/10.13033/ijahp.v1i2.53 salomon, v. a., tramarico, c. l., & marins, f. a. (2016). analytic hierarchy process applied to supply chain management. in f. de felice, t. l. saaty, & a. petrillo (eds.), applications and theory of analytic hierarchy process – decision making for strategic decisions (pp. 1–16). rijeka: intech doi: 10.5772/64022. tramarico, c. l., marins, f. a., urbina, l. m., & salomon, v. a. (2015). benefits assessment of training on supply chain management. international journal of the analytic hierarchy process, 7(2), 240–255. oi: http://dx.doi.org/10.13033/ijahp.v7i2.272 triantaphyllou, e. (2000). multi-criteria decision making methods. dordrecht, the netherlands: kluwer. doi: 10.1007/978-1-4757-3157-6_2 vargas, l. g. (1990). an overview of the analytic hierarchy process and its applications. european journal of operational research, 48(1), 2–8. doi: http://dx.doi.org/10.1016/0377-2217(90)90056-h http://dx.doi.org/10.13033/ijahp.v7i2.272 http://dx.doi.org/10.1016/0377-2217(90)90056-h ijahp article: mattioli, ricciardo lamonica/an evaluation of methods to make transitive the multiple comparisons matrices international journal of the analytic hierarchy process 56 vol. 3 issue 1 2011 issn 1936-6744 an evaluation of methods to make transitive the multiple comparisons matrices elvio mattioli polytechnic university of marche italy e-mail: e.mattioli@univpm.it giuseppe ricciardo lamonica* polytechnic university of marche italy e-mail: g.ricciardo@univpm.it abstract in this paper we analyse the response of several methods to construct indices for consistent multiple comparison. we consider also the close formal connection between the comparison of preference judgements and the comparison of economic aggregates. to evaluate the various scaling methods, we have used official data furnished by eurostat. consequently our analysis is based on real-life data and not on simulations as is usually the case in study in this kind. the most important results that we have achieved are the close concordance of the weights obtained with the various methods and the robustness of the evaluations performed. keywords: ratio-scale matrices, multiple comparisons, index numbers. 1. introduction the objective of this paper is to examine some procedures of economic entity comparisons. as is well known, comparisons may be bilateral (or binary) when individual pairs of entities are compared, or they may be multiple (or multilateral) when the intention is instead to grade the entities compared. the binary comparisons are simpler to set up and easier to implement. however, the information obtained in this way and arranged in appropriate ratio-scale matrices is normally inconsistent when used for multilateral comparisons. we shall illustrate this aspect in the second section. in the third section we discuss methods which enable consistent matrices to be constructed even independently of matrices for binary relationships. this is a line of inquiry which has been pursued and developed especially by economic statisticians. the results obtained from application of the methods discussed in the second and third sections, results which enable the entities compared to be ranked consistently, are analyzed in the fourth section. our procedure is to consider official data furnished by eurostat. the analysis is consequently based on real-life data, and not on simulations as is usually the case in ___________ * corresponding author rob typewritten text http://dx.doi.org/10.13033/ijahp.v3i1.77 ijahp article: mattioli, ricciardo lamonica/an evaluation of methods to make transitive the multiple comparisons matrices international journal of the analytic hierarchy process 57 vol. 3 issue 1 2011 issn 1936-6744 studies of this kind. although on the one hand this might be considered a limitation, in that only a limited number of cases are examined, on the other it has the advantage that situations of little or no practical importance are eliminated, thereby yielding, we believe, a clearer and more concrete picture. the fifth section makes some concluding remarks. 2. methods to give consistency to reciprocal positive matrices constructed by binary comparisons binary comparisons between alternatives in order to rank preferences for each of them or, in other words, to determine an ordered set of weights to be associated with them, have for some time been the subject of a broad strand of studies (saaty, 1980; saaty and vargas, 1984; crowford and williams, 1985) falling under the general heading of ahp (analytic hierarchy process) and which is still developing in various directions. of particular interest among these various lines of development are those involving the more systematic use of statistical tools (haines, 1998; carriere and finster, 1992; basak, 2002) and those that combine ahp with various methods of multicriterion analysis (lootsma, 1997; lootsma, ramanathan and schuijt, 1998; guitouni and martel, 1998). to restrict the discussion to the matters treated empirically in this study, we consider the set a = {a1, …, an} of the n alternatives to compare, and we use c to denote the n order square matrix whose generic element cij expresses the extent to which alternative ai is more important than aj, assuming that this extent is expressible on a ratio scale measurement. at most n(n-1)/2 comparisons are required because a minimum consistency is imposed whereby: cji = 1/cij i,j = 1, … , n (1) so that if, for example cij = 3 that is, if ai is deemed to be three times more important than aj, the importance of aj is one third that of ai hence cji = 1/3. it straightforwardly follows from (1) that cii =1 for i = 1, …, n. matrices with positive elements possessing property (1) are known as “positive reciprocal matrices”. it should be pointed out, however, that constraint (1) does not ensure complete consistency when multiple comparisons are made. in fact, this consistency only comes about if the elements of the positive reciprocal matrix c satisfy the relations: cij =cik•ckj ∀ i, j, k (2) if the weights (importance) wi to associate with the individual alternatives ai were known, and if matrix c were consistent, one would have: cij = wi/wj i, j = 1, … ,n (3) ijahp article: mattioli, ricciardo lamonica/an evaluation of methods to make transitive the multiple comparisons matrices international journal of the analytic hierarchy process 58 vol. 3 issue 1 2011 issn 1936-6744 note that the weights wi, which we shall consider in what follows as components of the vector w, are determined up to a multiplication by a constant, which is arbitrarily determinable. as is evident from (3), in the case of perfect consistency, the vector w of the weights would be immediately deducible from any row or column of matrix c. in practice, however, only rarely does this matrix display the complete consistency defined by (2) when it is actually constructed. this feature has prompted the development of various methods to determine the weights by minimizing the divergence, opportunely defined, between the values cij and the theoretical ones wi/wj. various studies (golany and kress, 1993; dodd, donegan and mcmaster, 1995) have used stochastic simulation techniques to assess the performance of the various adjustment methods. in our empirical analysis, we applied the best known and most widely used of these methods, which are now briefly surveyed. before beginning the survey, however, we would point out that the multiplier constant up to which the weights are defined can be chosen in various ways: by giving unitary value to one component of vector w or imposing the unit-sum constraint or the unitary product constraint on the components of vector w. obviously, this is an arbitrary choice made purely for reasons of computational convenience. 2.1 dominant eigenvalue method (henceforth de) this is the method originally proposed by saaty (1980) and based on the consideration that if the relations in (3) hold, then matrix c has one single eigenvalue different from zero whose value corresponds to the order of the matrix. consequently the vector of the weights w coincides with the corresponding eigenvector. cw = λw (4) in the practice, the eingenvector w associated with the dominant eigenvalue λ of the c is considered to be a reasonable evaluation of the vector of the weights. frobenius’s theorem ensures the positivity of the components of w and therefore the latter’s acceptability in the context considered. moreover, it is possible to measure the degree of approximation associated with this evaluation by means of the index: i(c) = (λ n)/(n-1) (5) known as the “consistency ratio”. 2.2 modified dominant eigenvalue method (henceforth mde) introduced by cogger and yu (1985), this method is a variant of the previous one. bearing in mind that matrix c is reciprocal, this technique considers only the upper triangle. using t to denote the matrix such that: ijahp article: mattioli, ricciardo lamonica/an evaluation of methods to make transitive the multiple comparisons matrices international journal of the analytic hierarchy process 59 vol. 3 issue 1 2011 issn 1936-6744 ⎩ ⎨ ⎧ ≥ = otherwise0 ji if c t ijij (6) and g to denote the diagonal matrix with the elements: gii = n-i+1 i=1,…,n (7) the weights correspond to the solution of the system: (g-1t u)w = 0 (8) where u is a unitary matrix. the solution is obtained recursively by means of the relations: ∑ += −=−= n 1ij jiji 1n,...,1i)1n/(wcw (9) 2.3 direct least squares method (henceforth dls) this is the classic least squares method by which the weights are determined so as to minimize the objective functions: ϕ1(w) = ∑∑ = = − n 1i n 1j 2 j i ij )w w c( (10) in this case the solution w does not have an analytical definable expression but must be calculated with iterative methods. 2.4 weighted least squares method (henceforth wls) this is a method similar to the previous one but differing from it in the weighting given to deviations from the square. in this case the objective function to minimize takes the following form: ϕ2(w) = =−∑∑ = = 2 j n 1i n 1j 2 j i ij w)w w c( ∑∑ = = − n 1i n 1j 2 ijij )wwc( (11) it has been shown (blankmeyer, 1987) that unlike the previous dls method, which may have multiple solutions, (11) admits to only one and strictly positive solution. this can be straightforwardly obtained by solving the following system of linear equations: ∑∑ ≠≠ =λ++−−+ n kj jkjjk n ki k 2 ik 0w)cc(w)1nc( k=1,…,n (12) generated by the first-order conditions for the lagrange auxiliary function constructed by taking (11) as the objective function and constraining the weights to unitary sum. ijahp article: mattioli, ricciardo lamonica/an evaluation of methods to make transitive the multiple comparisons matrices international journal of the analytic hierarchy process 60 vol. 3 issue 1 2011 issn 1936-6744 2.5 logarithmic least squares method (henceforth lls) given the multiplicative nature of the positive reciprocal matrix c, it is natural to measure the divergences between observed values cij and the theoretical values wi/wj considering the ratio between them. that is, by calculating the relative error given by the deviation of this ratio from unity. considering, for obvious reasons of simplicity, the logarithm of these ratios and the least squares method, the objective function becomes: ϕ3(w) = =∑∑ = = n 1i n 1j 2 ji ij ) w/w c (ln ∑∑ = = −− n 1i n 1j 2 jiij )wlnwlnc(ln (13) selecting the arbitrary multiplying constant of the weights so that their product is unitary, the components of vector w that minimize (13) correspond to the geometric mean of the elements in the corresponding row of matrix c: ∏ = = n 1j n 1 iji )c(w i=1, …, n (14) 3. methods to construct consistent matrices even without binary comparisons the methodology for performing accurate comparisons among economic phenomena observed on different occasions, namely index number theory, is a key area of research for economic statisticians and to which they have made a number of crucial contributions. with index numbers, comparison is made among the relative differences of a phenomenon by examining their corresponding intensities on different occasions. for example, if we consider the unit price of a generic good on two occasions a and b, which may be two regions, two periods, and so on, the ratios: b a p p and a b p p (15) measure the relative difference in the price on occasion a with respect to occasion b considered as the base situation, and vice versa. simple index numbers obviously satisfy the consistency conditions mentioned in the second section. however, when the phenomenon is analysed across a number of entities, or, to refer to the example again, when the unit prices of n goods are considered, the statistical relationships that simultaneously and succinctly measure the relative diversity of prices in the situations compared are known as “complex index numbers” or simply as “index numbers”. there are various methods with which to define complex index numbers. however, many of the complex indexes proposed and widely used do not permit consistent comparisons. ijahp article: mattioli, ricciardo lamonica/an evaluation of methods to make transitive the multiple comparisons matrices international journal of the analytic hierarchy process 61 vol. 3 issue 1 2011 issn 1936-6744 this is a problem with particular practical implications for the calculation of purchasing power parities. the theoretical and practical importance of comparison among complex index numbers is certainly not restricted to price index numbers, for there is an immediate logical symmetry between the latter and the index numbers of quantities. nevertheless, in what follows, we shall restrict our treatment to the former, both because it was on these that we conducted our empirical analysis and because they suffice to illustrate the issue analysed here. in the following sections we briefly survey the best known methods for the construction of complex and consistent index numbers. we assume that n situations are to be compared, each of them characterized by m goods described by the following vectors of prices and quantities: pi = [pi1, pi2, …, pim]; qi = [qi1, qi2, …, qim] ; i=1,…,n (16) 3.1 the gini-elteto-köves-szulc method (henceforth geks) this method was originally proposed by gini (1924) and then taken up by elteto and köves (1964) and szulc (1964). it is based on the matrix c of binary comparisons whose elements are fisher price index numbers: 2 1 m 1k m 1k m 1k m 1k ij jkik jkjk ikik jkik qp qp . qp qp c ⎟ ⎟ ⎟ ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎜ ⎜ ⎜ ⎝ ⎛ = ∑ ∑ ∑ ∑ = = = = i, j = 1, …, n (17) independently of the authors cited above, the logarithmic least squares method mentioned earlier has been proposed in order to give consistency to the comparisons expressed by the c matrix which is evidently positive reciprocal. 3.2 the theil method (henceforth t) in this case too, consideration is made of a c matrix of binary comparisons, which are then made consistent by means of the logarithmic least squares method. unlike in the previous case, however, the elements that make up the matrix are törnquist bilateral indices: k jk ik m 1k ij p p c α = ∏ ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛ = i, j=1, …,n (18) where: ⎟ ⎟ ⎟ ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎜ ⎜ ⎜ ⎝ ⎛ +=α ∑∑ == m 1k m 1k jkjk jkjk ikik ikik k qp qp qp qp 2 1 (19) ijahp article: mattioli, ricciardo lamonica/an evaluation of methods to make transitive the multiple comparisons matrices international journal of the analytic hierarchy process 62 vol. 3 issue 1 2011 issn 1936-6744 3.3 the economic commission for latin america method (henceforth ecla) this method too can be traced back to the work of gini, and it has been used by the economic commission for latin america. its distinctive feature is that it directly and consistently constructs the c matrix of comparisons, whose elements are defined in the following manner: ∑ ∑ = == m 1k m 1k ij kjk kik qp qp c i, j=1, …, n (20) where ∑ = = n 1i ikk q n 1 q represents the mean of the quantities of the k-th good treated in the set of statistical units considered. 3.4 the geary-khamis method (henceforth gk) nor does this method require the construction of a matrix of binary comparisons to be then rendered consistent. rather, it is constructed directly, by means of (3), from the weights wi, which in the context treated by gear and khamis act as conversion factors. this technique proposed by geary (1958) and khamis (1969), is of iterative type and is divided into two phases. in the first, the conversion factors (weights) wi are used to determine the mean price (πk) of each good in the basket of the statistical collective: ; q qpw n 1i n 1i ik ikiki k ∑ ∑ = ==π k=1, …,m (21) in the second phase, (22) below is used to determine the conversion factors: ; qp q w m 1k m 1k ikik ikk i ∑ ∑ = = π = i=1, …, n (22) each iteration involves two phases: in the first, after giving wi an arbitrary initial value, (21) is used to determine the mean prices; in the second, the latter is inserted in (22) to determine the conversion factors (weights). the procedure is reiterated until the solutions of two successive iterations are judged equal to each other. 3.5 the gerardi method (henceforth g) this technique does not substantially differ from the gk method. as regards the evaluation procedure it is entirely analogous to it, the only difference being that in this case the mean prices of the individual goods are calculated using a geometric mean of the prices observed on the n occasions: ijahp article: mattioli, ricciardo lamonica/an evaluation of methods to make transitive the multiple comparisons matrices international journal of the analytic hierarchy process 63 vol. 3 issue 1 2011 issn 1936-6744 n 1 n 1i ikk p ⎟⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛ =π ∏ = k=1, …,m (23) 4. empirical analysis of the methods to construct indices for multiple consistent comparisons in this section we describe a series of experiments conducted on empirical data from official sources in order to examine the responses of the various methods surveyed above. the analysis was carried out using information on prices and real spending volumes of goods and services purchased in the countries belonging to the eurostatoecd purchasing power parity programme. the programme was started in the 1980s to compare the price and volume levels of gross domestic product (gdp) of the member state of the european union and the member countries of the oecd. see for details eurostat (1985) and eurostat (2000). the gdp of each country is divided in 36 comparable items of goods and services. for each of them the prices and the real spending volumes are known. the data available were used to construct two distinct binary comparison matrices: one with fisher price index numbers, and one with törnqvist price index numbers. application of the lls method to give them consistency led, respectively, to the geks and the t method. as regards the other methods (de, mde, dls and wls) which can be used to give consistency to binary comparison matrixes, we applied these only to the matrix of fisher index numbers, because we had ascertained that, in the various situations examined, this matrix differs to a negligible extent from the corresponding matrix of törnqvist index numbers. taking account of the multiplicative nature and the asymmetry of these matrices, comparison was made using the following similarity index (mean logarithmic variation): ; to f ln )1n(n 1 mvl n 1i n 1j ij ij∑∑ = = − = (24) where fij and toij are respectively the fisher and törnqvist index numbers calculated for the purpose of comparison between the i-th and the j-th country. the values assumed by index (24) in the four sets of data on which we made our subsequent calculations are given in table 1, which shows that the divergence between the two types of matrix never exceeded the threshold of 0.38%. it may also be of interest to point out that the behaviour of the index matched our a priori expectations regarding the heterogeneity of the four situations analysed. ijahp article: mattioli, ricciardo lamonica/an evaluation of methods to make transitive the multiple comparisons matrices international journal of the analytic hierarchy process 64 vol. 3 issue 1 2011 issn 1936-6744 table 1 mean of the logarithmic variations between the fisher and törnqvist indices. data set mvl europe 15, eurostat comparable data year 1998 0.0014 europe 12+3, eurostat comparable data year 1985 0.0022 all country with eurostat comparable data, year 1998 0.0022 all country with eurostat comparable data, year 1985 0.0038 the first set of calculations was performed on the 1998 price indexes of the 15 countries of the european union. the results are given in the following table 2. table 2 normalized weights (*) – countries of the european union – 1998. de mev dls wls lls geks gk g t ecla mean euro15 100 100 100 100 100 100 100 100 100 100 austria 6.81 6.79 6.8 6.79 6.81 6.85 6.83 6.81 6.78 6.81 belgium 2.49 2.49 2.49 2.49 2.49 2.5 2.5 2.49 2.49 2.49 denmark 10.83 10.85 10.85 10.84 10.83 10.93 10.91 10.83 4.48 10.15 finland 15.11 15.09 15.14 15.08 15.11 15.24 15.2 15.1 14.9 15.11 france 13.95 13.95 13.85 13.94 13.95 14.06 14.07 13.96 13.88 13.96 germany 46.36 46.36 46.41 46.38 46.36 46.6 46.68 46.34 46.06 46.39 greece 0.38 0.38 0.38 0.38 0.38 0.39 0.39 0.38 0.38 0.38 england 143.1 143.4 143.0 143.1 143.1 143.3 143.2 143.2 143.0 143.1 ireland 130.1 130.1 130.2 130.1 130.1 131.9 131.6 130.1 128.6 130.3 italy(**) 57.95 57.92 57.97 57.99 57.95 58.62 58.61 57.92 57.46 58.04 luxemburg 2.19 2.19 2.21 2.21 2.19 2.22 2.23 2.18 2.16 2.2 holland 46.69 46.65 46.52 46.65 46.69 46.96 46.98 46.71 46.51 46.71 portugal 0.71 0.71 0.71 0.71 0.71 0.73 0.72 0.71 0.7 0.71 spain 0.71 0.71 0.71 0.71 0.71 0.71 0.71 0.71 0.71 0.71 sweden 9.4 9.45 9.35 9.38 9.4 9.48 9.48 9.4 9.28 9.4 msd 0.18 0.19 0.18 0.18 0.18 0.48 0.41 0.18 1.49 (*) these weights are interpretable as purchasing power parities. (**) expressed as thousands of lire. all value are multiplied by 100 the most striking aspect to emerge from table 2 is the close concordance indeed, often the perfect coincidence of the results obtained when applying the various methods. as a synthesis measure we considered only the following consistency index (mean squared deviation) of the weights with respect to those calculated as a mean of the nine methods applied: 2 1 n 1i 2 h )ww(n 1 msd iih ⎥ ⎦ ⎤ ⎢ ⎣ ⎡ −= ∑ = h=1, …, 9 (25) where: ijahp article: mattioli, ricciardo lamonica/an evaluation of methods to make transitive the multiple comparisons matrices international journal of the analytic hierarchy process 65 vol. 3 issue 1 2011 issn 1936-6744 ∑ = = 9 1h ihi w 9 1 w i=1, …, n (26) the values of this index highlight not only the negligible amount of deviation, directly deducible from the close concordance already mentioned, but also a slightly different behaviour from the others, in order, of the g, gk and ecla methods, namely, precisely those methods which do not presuppose a matrix of binary comparisons to be adjusted. in order to verify the reliability of these results as regards both their stability in time and their robustness, we conducted three further tests. the first consisted in replication of the analysis using comparable data from further back in the past, specifically those relative to 1985, and including in the analysis the three countries (austria, finland, and sweden) which did not yet belong to the union at the time. the results are given in table 3, and they are a perfect match with the previous ones, displaying considerable stability in the degree of the concordance over time. it must be pointed out that stability is referred to concordance in the methods’ responses and not to the weight calculated in 1988 and 1995, which depict different situation. as regards the consistency index in the three methods differing most from the others, we find, as well as gk and g, also the dls method, but the difference with the ecla method (0.0040 as opposed to 0.0039) is so slight that it can be taken to be entirely insignificant. table 3 normalized weights(*) – countries of europe of 12 plus the three countries (austria, finland, and sweden) that joined later – 1985. de mev dls wls lls geks gk g t ecla mean euro12+3 100 100 100 100 100 100 100 100 100 100 austria 7.17 7.12 7.16 7.18 7.17 7.21 7.23 7.17 7.14 7.17 belgium 2.65 2.65 2.64 2.65 2.65 2.68 2.67 2.66 2.65 2.66 denmark 12.13 12.09 12.06 12.12 12.13 12.27 12.16 12.12 12.07 12.13 finland 19.68 19.71 19.49 19.66 19.68 20.18 20.05 19.68 19.31 19.72 france 16.38 16.38 16.33 16.38 16.38 16.44 16.44 16.39 16.36 16.39 germany 47.84 47.84 47.89 47.84 47.84 48.16 48.41 47.87 47.46 47.91 greece 1.44 1.45 1.44 1.45 1.44 1.49 1.5 1.44 1.37 1.45 england 209.6 209.2 208.5 209.6 209.6 212.0 211.3 209.8 209.1 209.9 ireland 167.0 167.9 166.1 167.0 167.0 166.4 166.7 166.8 166.3 166.8 italy (**) 91.67 91.78 91.7 91.59 91.67 92.39 92.15 91.69 90.61 91.7 luxemburg 2.73 2.74 2.74 2.75 2.73 2.78 2.79 2.72 2.68 2.74 holland 46.99 46.96 46.99 47.01 46.99 47.25 47.21 46.94 46.91 47.03 portugal 1.66 1.67 1.63 1.64 1.66 1.81 1.79 1.66 1.52 1.67 spain 1.23 1.23 1.22 1.23 1.23 1.28 1.26 1.23 1.19 1.23 sweden 14.3 14.15 14.11 14.27 14.3 14.84 14.66 14.25 14.04 14.32 msd 0.07 0.33 0.4 0.08 0.07 0.6 0.41 0.03 0.39 (*) these weights are interpretable as purchasing power parities. (**) expressed as thousands of lire. all value are multiplied by 100 ijahp article: mattioli, ricciardo lamonica/an evaluation of methods to make transitive the multiple comparisons matrices international journal of the analytic hierarchy process 66 vol. 3 issue 1 2011 issn 1936-6744 in order to conduct thorough verification of not only the stability in the concordance of the responses but also the robustness of the responses obtained with the various methods, we again replicated the calculations, at the two times already considered, including in the two occasions the countries not belonging to the european union in its present form but for which comparable data, again furnished by eurostat, were available. the results are set out in tables 4 and 5. once again they confirm the close concordance of the responses and the slight divergence of the gk, g and ecla methods. to these results should now be added the considerable robustness of the evaluations of the weights inferable from comparison of table 2 with table 4, and table 3 with table 5. leaving further comparisons to the reader, here we merely point out that, as regards the weight of italy, the mean value in 1985 changes only from 0.5504 to 0.5806 because of the inclusion of cyprus, iceland, norway, poland and switzerland. likewise with the 1985 data, because of the inclusion of australia, canada, japan, new zealand, norway, united states and turkey, the mean value shifts from 0.9170 to 0.9171, and therefore to only a very slight extent bearing in mind the significance and the order of magnitude of those weights. table 4 normalized weights(*) – countries for which standardized eurostat data are available – 1998. de mev dls els lls geks gk g t ecla mean euro15 100 100 100 100 100 100 100 100 100 100 austria 6.81 6.78 6.81 6.78 6.81 6.84 6.81 6.8 6.78 6.8 belgium 2.49 2.49 2.51 2.49 2.49 2.5 2.5 2.49 2.49 2.5 denmark 10.84 10.86 10.87 10.86 10.84 10.93 10.91 10.84 10.74 10.85 finland 15.11 15.08 15.16 15.14 15.1 15.24 15.15 15.1 14.9 15.11 france 13.93 13.93 13.87 13.93 13.93 14.06 14.08 13.95 13.87 13.95 germany 46.38 46.38 46.42 46.57 46.38 46.63 46.71 46.36 46.02 46.43 greece 0.38 0.38 0.38 0.38 0.38 0.39 0.39 0.38 0.38 0.38 england 143.2 143.5 143.0 143.2 143.2 143.3 143.4 143.4 143.0 143.3 ireland 129.9 129.9 130.3 129.8 129.9 131.9 131.4 130.0 128.5 130.2 italy (**) 57.94 57.92 57.94 57.94 57.94 58.54 58.51 57.9 57.5 58.01 luxemburg 2.19 2.2 2.23 2.22 2.19 2.23 2.23 2.19 2.15 2.2 holland 46.67 46.64 46.5 46.65 46.67 46.96 46.97 46.69 46.51 46.7 portugal 0.71 0.71 0.72 0.71 0.71 0.73 0.72 0.71 0.7 0.71 spain 0.71 0.71 0.71 0.71 0.71 0.71 0.71 0.71 0.71 0.71 sweden 9.38 9.35 9.39 9.39 9.38 9.48 9.46 9.38 9.28 9.39 cyprus 220.9 224.2 222.9 220.7 220.9 225.5 225.2 221.0 215.2 221.8 iceland 1.12 1.12 1.11 1.1 1.12 1.13 1.12 1.12 1.11 1.11 norway 9.74 9.69 9.69 9.63 9.74 9.97 9.87 9.74 9.6 9.74 poland 53.36 53 52.88 53.09 53.35 58.22 57.56 52.9 49.28 53.74 switzerland 47.28 47.33 47.44 47.46 47.28 47.8 47.79 47.29 46.43 47.34 msd 0.23 0.54 0.31 0.3 0.23 1.33 1.16 0.26 1.8 ijahp article: mattioli, ricciardo lamonica/an evaluation of methods to make transitive the multiple comparisons matrices international journal of the analytic hierarchy process 67 vol. 3 issue 1 2011 issn 1936-6744 (*) these weights are interpretable as purchasing power parities. (**) expressed as thousands of lire. all value are multiplied by 100 table 5 normalized weights(*) – countries for which standardized eurostat data are available – 1985. de mev dls els lls geks gk g t ecla mean euro12 100 100 100 100 100 100 100 100 100 100 austria 7.17 7.16 7.17 7.17 7.17 7.16 7.21 7.18 7.12 7.17 belgium 2.65 2.65 2.65 2.65 2.65 2.68 2.67 2.65 2.65 2.66 denmark 12.15 12.14 12.13 12.15 12.15 12.28 12.17 12.15 12.13 12.16 finland 19.7 19.74 19.73 19.69 19.7 20.05 19.98 19.71 19.49 19.75 france 16.39 16.39 16.4 16.39 16.39 16.46 16.45 16.4 16.34 16.4 germany 47.92 47.92 48.15 47.88 47.92 47.79 48.36 47.92 47.73 47.96 greece 1.44 1.44 1.44 1.44 1.44 1.51 1.49 1.44 1.36 1.44 england 209.5 209.2 208.3 209.5 209.5 212.8 211.3 209.7 208.5 209.8 ireland 167.4 168.1 169.4 167.3 167.4 166.7 167.4 167.5 166.2 167.5 italy (**) 91.63 91.68 91.91 91.57 91.63 92.4 92.28 91.69 90.58 91.71 luxemburg 2.73 2.73 2.73 2.73 2.73 2.72 2.78 2.72 2.69 2.73 holland 47.02 47.01 47.04 47.04 47.02 47.54 47.25 46.98 46.78 47.08 portugal 1.66 1.66 1.65 1.65 1.66 1.84 1.8 1.66 1.51 1.68 spain 1.23 1.23 1.23 1.23 1.23 1.29 1.27 1.23 1.19 1.24 sweden 14.3 14.27 14.12 14.33 14.3 14.79 14.6 14.27 14.25 14.36 australia 97.36 97.93 97.35 97.3 97.36 96.89 97.28 97.21 97.64 97.37 canada 97.96 97.97 97.3 97.92 97.96 98.07 100 97.52 98.48 98.13 japan 0.51 0.51 0.51 0.51 0.51 0.54 0.52 0.51 0.51 0.51 n. zealand 86.57 85.97 86.89 86.49 86.56 88.41 88.4 86.59 83.31 86.58 norway 13.34 13.34 13.37 13.34 13.34 13.74 13.72 13.34 13.12 13.4 united st. 121.1 121.8 119.6 121.2 121.1 121.0 124.4 120.5 121.1 121.3 turkey 0.7 0.7 0.7 0.7 0.7 0.76 0.76 0.71 0.64 0.71 msd 0.09 0.27 0.65 0.1 0.09 0.79 0.91 0.21 0.82 (*) these weights are interpretable as purchasing power parities. (**) expressed as thousands of lire. all value are multiplied by 100 5. conclusions there is a large body of literature on the methodology with which to perform consistent comparisons. these methods have been analysed in a relatively independent manner by scholars of decision-making processes in business and by economic statisticians with regard to the comparison of economic aggregates. in this paper we have first emphasised the close connection between the two approaches: the one that uses subjective preference judgements, and the one based on objective data regarding prices and quantities. using data of the latter type, which are more incontrovertible, we have conducted a series of calculations using both cross-section data and time series data. the results that we believe to be most significant are the following: ijahp article: mattioli, ricciardo lamonica/an evaluation of methods to make transitive the multiple comparisons matrices international journal of the analytic hierarchy process 68 vol. 3 issue 1 2011 issn 1936-6744 − the close concordance, sometimes almost the coincidence, of the weights obtained with the various methods; and − the notable robustness of the evaluations performed. the most immediate conclusion to be drawn from our analysis is that these methods should be used more widely. which method in particular should be selected is of little importance from the practical point of view, given the extremely close concordance obtained in the responses. more specifically as regards index numbers, we have provided clear evidence that those methods which prove incoherent in multiple comparisons should be discarded. these were methods used in the past because of a computational simplicity which is now wholly irrelevant, and they are still too widely used. references basak, i. (2002). on the use of information criteria in analytic hierarchy process. european journal of operational research 141, 200-216. blankmeyer, e. (1987). approaches to consistency adjustments. journal of optimization theory and applications, 54, 479-488. carriere, j., finster, m. (1992). statistical theory for ratio model of paired comparisons. journal of mathematical psychology, 36, 450-460. cogger, k.o., yu, p.l. (1985). eigenweight vectors and least distance approximation for revealed preference in pairwise weight ratios. journal of optimization theory and applications, 46, 483-491. crowford, g., williams c. (1985). a note on the analysis of subjectives judgements matrices. journal of the mathematica psychology 29, 387-405. diewert, w. e., nakamura, a. o. (1993). essays in index number theory. vol. 1 north holland, amsterdam. dodd, f.j., donegan, h.a., mcmaster, t.b.m. (1995). reassessment of consistency criteria in judgment matrices. the statistician 44, 1, 31-41. eltetö, o., köves, t. (1964). one index computation problem of international coparision. statisztikai szemle. eurostat (1985). purchasing power parities and related economic indicator. result for 1985. eurostat (2000). purchasing power parities and related economic indicator. result for 1998. geary, r. g. (1958). a note on the comparison of excange rate and purchasing power between countries. journal of the royal statistical society, vol. 121, part i. gerardi, d. (1978). alcuni aspetti metodologici riguardanti il calcolo delle parità di potere di acquisto tra i paesi della comunità europea. atti della 29^ riunione scientifica della società italiana di statistica. ijahp article: mattioli, ricciardo lamonica/an evaluation of methods to make transitive the multiple comparisons matrices international journal of the analytic hierarchy process 69 vol. 3 issue 1 2011 issn 1936-6744 gini, c. (1924). quelques consideration on sujet de la construction des nombres indices des prix et des question analogues. metron vol. 4 n. 1. golany, b., kress, m. (1993). a multicriteria evaluation of methods for obtainig weights from ratio-scale matrices. european journal of operational research 69, 210-220. guitouni, a., martel, j.m. (1998). tentative guidelines to help choosing an appropriate mcda method. european journal of operational research 109, 501-521. haines, l.m., (1998). interval judgements in the analytic hierarchy process: a statistical perspective..in t.j.stewart e r.c. honert (van den) (eds), trends in multicriteria decision making. lecure notes in economics and mathematical systems, n. 465, 87-95, springer-verlag, berlin. khamis, s. h. (1969). a new system of index number for national purposes. atti della 97^ sessione dell’istituto internazionale di statistica. lootsma, f.a., ramanathan, r., schuijt, h. (1998). fairness and equity via concepts of multi-criteria decision analysis. in t.j.stewart e r.c. honert (van den) (eds), trends in multicriteria decision making. lecure notes in economics and mathematical systems, n. 465, 215-226, springer-verlag, berlin. lootsma, f. a. (1997). multicriteria decision analysis in a decision tree. european journal of operational research 101, 442-451. saaty, t.l. (1980). the analytic hierarchy process. mcgraw hill, new york. saaty, t.l., vargas, l.g. (1984). comparison of eigenvalues, logaritmic least squares and squared methods in estimating ratios. mathematical modelling 5, 309324. stuvel, g. (1989). the index-number problem and its solution. the macmillan press ltd, london. szulc, b. (1964). index number of multilateral regional comparison. przeglad statystieny n. 3. leery ijahp news and events: topcu/countdown for mcdm 2019 international journal of the analytic hierarchy process 148 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.647 countdown to mcdm 2019 there is less than two months to go! you are all welcome to the 25th international conference on multiple criteria decision making (mcdm 2019) that will take place at istanbul technical university, istanbul, turkey from june 16 th to june 21 st , 2019. you can find all the details at the official web site of the conference: https://www.mcdm2019.org. the organizers invite you to bookmark this page, and check it regularly for news, updates, and recent developments. you may also follow the meeting on twitter (twitter.com/mcdm2019) and on instagram (www.instagram.com/mcdm2019) the submission and review processes are finished and abstract status notifications have been sent out to the authors. there were 290 submissions. the ahp/anp community will be well represented at the conference with:  one plenary speech “voting with intensity of preferences” by luis vargas (the university of pittsburgh)  a tutorial by orrin cooper (the university of memphis)  several presentations for ahp/anp theory and applications there is a big ahp/anp community in turkey. ahp/anp researchers as well as practitioners will have a great opportunity to discuss a variety of topics whether on theory or applications, including extensions and applications of ahp/anp. besides the speech by dr. vargas, there will be three more plenary speeches from distinguished speakers:  “quantitative models for decision-support in healthcare applications” by pinar keskinocak (georgia tech)  “multi-objective combinatorial optimization – beyond the bi-objective case” by kathrin klamroth (university of wuppertal)  “new results for solving imperfect-information games” by tuomas sandholm (carnegie mellon university) http://www.mcdm2019.org/ ijahp news and events: topcu/countdown for mcdm 2019 international journal of the analytic hierarchy process 149 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.647 sabanci muzesi the “outing on wednesday” will be at sakip sabanci museum and emirgan park, followed by a dinner cruise on the bosphorus.  sabancı university's sakıp sabancı museum, a private fine arts museum, is located at one of istanbul's oldest settlements on the bosphorus.  emirgan park, the tulip’s homeland, is one of the biggest parks in istanbul. it is said to be the most beautiful park in istanbul.  a boat ride on the bosphorus gives a splendid panorama, and provides a unique experience to see many beautiful sites including palaces, monuments, and mansions accompanied by trees and flowers along the european and asian sides of the bosphorus. dolmabahce palace emirgan korusu http://www.sakipsabancimuzesi.org/en ijahp news and events: topcu/countdown for mcdm 2019 international journal of the analytic hierarchy process 150 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.647 the social program also includes a welcome reception (sunday evening) and a conference banquet (thursday night) you are all most welcome to mcdm2019, the 25 th organization of mcdm conferences. i̇stanbul’da görüşmek üzere! (see you in istanbul!) sincerely, y. ilker topcu professor, istanbul technical university mcdm 2019 conference chair ijahp article: saaty, sagir/ranking countries more reliably in the summer olympics international journal of the analytic hierarchy process 589 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.341 ranking countries more reliably in the summer olympics thomas l. saaty katz graduate school of business 322 mervis hall, pittsburgh, pa, usa e-mail: saaty@katz.edu mujgan sagir* eskisehir osmangazi university eskisehir, turkey e-mail: mujgan.sagir@gmail.com abstract in this paper we consider the many intangible criteria that influence the outcome of the summer olympics by using the analytic network process, and apply the ideas to evaluate the medals won and the country scores in the 2012 london olympics. both the categories of games and the events in each game are considered in this weighting process. different events of the same category game could have different properties. this work shows that the current way of counting the total number of medals is not a bad way of ranking countries. with minor modifications, this systematic approach for ranking countries can be used for any summer olympics. keywords: or in sports; country ranking; olympic games; analytic network process; rating 1. introduction the modern olympic games are a major international event featuring summer and winter sports in which thousands of athletes participate in a variety of competitions. they are considered to be the world's foremost sports competitors and represent nearly 200 nations who may participate. the games are currently held biennially, with summer and winter olympic games alternating, thus each occurring every four years. their creation was inspired by the ancient olympic games which were held in olympia, greece for more than 1000 years from the 8th century bc to the 4th century ad. there have been a number of studies conductedwhich have focused on the olympic games and other olympic movements. for example, andrew (2000)’s study researched why countries show different performance in olympics. this study had three objectives with the key objective being, to examine the influences of factors affecting the olympic performance.two specific objectives were (i) to produce a mathematical model facilitating the prediction of the olympic tally, and (ii) to identify the degree of factors that have influence on the olympic performance. bernard (2000) studied different variables in olympic games success investigation. on the other hand, wade (2006) http://en.wikipedia.org/wiki/multi-sport_event http://en.wikipedia.org/wiki/summer_olympic_games http://en.wikipedia.org/wiki/winter_olympic_games http://en.wikipedia.org/wiki/winter_olympic_games http://en.wikipedia.org/wiki/ancient_olympic_games http://en.wikipedia.org/wiki/olympia,_greece ijahp article: saaty, sagir/ranking countries more reliably in the summer olympics international journal of the analytic hierarchy process 590 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.341 studied the prediction of medal winners. our motivation in this paper comes from the need for a scientific methodology to interpret the number of medals the countries have and a way to rank them. qualification rules for each of the olympic sports are set by the international sports federations (ifs) that governs that sport's international competition. for individual sports, athletes typically qualify by attaining a certain rank in a major international event and thus gain recognition on the if's ranking list. national olympic committees (noc) may enter a limited number of qualified athletes in each event. in the ancient olympics, no medals were awarded. first-place winners were given an olive branch to wear on their head, and second and third place winners did not receive anything. in the first modern games held in 1896 silver and bronze medals were awarded to first and second place winners. in 1900, most winners received cups or trophies instead of medals. in the 1904 games in st. louis, gold replaced silver as the medal awarded for first place, followed by silver and bronze medals awarded to second and third place winners. nowadays, the media decides which country has won the olympics by adding all the medals won by athletes from that country. however, this kind of practice seems selfdefeating because it assumes that all gold, silver and bronze medals should be counted as equal in merit and all games and events are assumed equally important. nevertheless, it is only an approximate way and as it turns out, not an entirely faulty way of deciding which country is the overall winner of medals. no methodically scientific way to deal with multicriteria ranking involving intangibles has been used to assign appropriate priority weights to each type of sport and medal won in that sport for the olympic games. here we propose using the analytic network process (anp) for the measurement of intangibles, along with their dependence and feedback, to weight the criteria which we think play an important role in assigning priorities to games and events. our methodology consists of the following steps: 1) an anp model is developed to assign weights to different criteria used to prioritize different games and events in each game. 2) expert knowledge is used to define the criteria and evaluate the games and events. 3) we prioritize the significance of the three types of medals (gold, silver and bronze), weight them by the overall priorities of the events to obtain the overall priority of a medal won and add these priorities to obtain the priority rank of a country. 4) our results do not violate intuition about the number and value of the medals and in fact take greater consideration by including the merits of the events in which they are won. thus, we also show that the ranking of countries produces results that are reasonably close to the current results of adding all medals won but with some important exceptions. http://en.wikipedia.org/wiki/sport_governing_body http://en.wikipedia.org/wiki/sport_governing_body ijahp article: saaty, sagir/ranking countries more reliably in the summer olympics international journal of the analytic hierarchy process 591 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.341 2. criteria to weight the summer olympic games the priorities of different kinds of games depend on several factors. table 1 lists the relevant basic criteria groupings or clusters and the elements in each cluster. figure 1 represents the top level network of the analytic network process model together with the criteria. table 1 criteria and subcriteria cluster elements in cluster game requirements required physical characteristics, required training time game other factors risk level involved, energy spent, duration of the act people involved number of competitors, strength of competitors, audience reaction, popularity living environment effects of daily life on the game, effects on daily life, financial resources needed natural environment season or climate suitability, topographic pattern, absence of pollution other factors sport commercialization, technology, political factors according to table 1 and figure 1 the main cluster of criteria is “game requirements” which consists of “required physical characteristics” and “required training time”. for some sports, one may need physical characteristics that deal with strength. for example, in gymnastics one needs to practice for years starting at a young age. the second cluster of criteria is “other factors related to the game” which includes “risk level involved”, “energy spent” and “duration of the act”. the “risk level” involved is a criterion in our analysis because it affects people’s attitudes; some people find it more challenging and encouraging to take risks. we have also included an “energy spent” criterion because in certain games more energy is needed. for example, wrestling requires a high amount of energy spent, and certain sports also involve higher risk as in some gymnastics events. the duration of an event is another concern because some games last for a relatively long time, and in certain cases as in the marathon, a medal for this game deserves a greater value. a third cluster of criteria is “people involved” which consists of the “number of competitors”, the “strength of competitors” and also “audience reaction, popularity”. here, we think that there are some games that have greater popularity and this makes these sports more attractive to attend (e.g. football, tennis). on the other hand, to be successful in a game that has many competitors is more difficult. the fourth cluster is “living environment”, with criteria “effects of daily life on the game”, “effects on daily life” and “financial resources needed”. some sports can be influenced by the daily life of the competitors; for example football players are usually careful about being involved in too large amounts of social activities and entertainment ijahp article: saaty, sagir/ranking countries more reliably in the summer olympics international journal of the analytic hierarchy process 592 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.341 that may affect their physical strength during the game due to lack of sleep. this is equally true of organizing daily life in a way to support success in the sport, such as being careful about ones diet. financial resources are also important in some sports both to facilitate a player’s needs and to make it possible to be choosy in meeting basic needs as desired. tennis, skiing or ice-skating need specific professional environments and specific equipment for training which can be costly. the fifth cluster is “natural environment” whose criteria are “season or climate suitability”, “topographic pattern” and “absence of pollution”. climate and topographic patterns have important effects on pollution which is an undesirable factor particularly for events like canoeing and marathon. the sixth cluster is “other factors” related to political, economic and social issues. political factors can determine whether the games would be attended by some countries. technology affects performance and in the long run new technology can change performance in a sport very significantly. figure 1 gives a screen view from the superdecisions software with the clusters and their criteria from table 1 along with their interconnections. figure 1. anp top level model as an illustration, table 2 gives a view of paired comparisons related to the effect of “risk level involved” and “duration of the act” on the criterion “energy spent”. “duration of the act” has two times greater influence on the “energy spent” criterion than “risk level” does. table 3 presents the criteria weights. these judgements have been obtained by interviewing different experts who have been a judge or competitor in different olympic games, and geometric mean is used. ijahp article: saaty, sagir/ranking countries more reliably in the summer olympics international journal of the analytic hierarchy process 593 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.341 table 2 comparisons between “risk level involved” and “duration of the act” on “energy spent” energy spent risk level involved duration of the act priorities risk level involved 1 1/2 0.3333 duration of the act 2 1 0.6667 table 3 criteria priorities criterion priority criterion priority required physical characteristics 0,0606 effects of the daily life on the game 0,0935 required training time 0,0735 effects on the daily life 0,0170 risk level involved 0,0107 financial resources needed 0,0136 energy spent 0,0692 season or climate suitability 0,0805 duration of the act 0,0740 topographic pattern 0,0921 number of competitors 0,0539 absence of pollution 0,0673 strength of competitors 0,0735 sports commercialization 0,0463 audience reaction, popularity 0,0760 political factors 0,0296 technology 0,0679 3. how to evaluate different games and different events involved in each game based on the previous discussion, it appears that declaring a winning country by adding all medals may not reflect the quality of the games that are won by the athletes from that country. the difference in the quality of the sports themselves is an important factor. by using the fundamental scale of absolute numbers of the ahp given in table 4, one can compare the importance of different games and the importance of the events involved in each game (saaty, 2004). ijahp article: saaty, sagir/ranking countries more reliably in the summer olympics international journal of the analytic hierarchy process 594 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.341 table 4 fundamental scale of absolute numbers intensity of importance definition explanation 1 equal importance two activities contribute equally to the objective 2 weak or slight 3 moderate importance experience and judgment slightly favor the dominance of one activity over another 4 moderate plus 5 strong importance experience and judgment strongly favor the dominance of one activity over another 6 strong plus 7 very strong or demonstrated importance an activity is favored very strongly in dominating over another activity; its dominance may even be demonstrated in practice 8 very, very strong 9 extreme an activity extremely domiantes another activity the experts who provided the judgments were a group of four people who have taken part in sports games as a referee, a coach and/or an athlete for over 12 years. we interviewed them for several days with regard to the criteria to be considered in the evaluation of the games and their events. we prepared a questionnaire to obtain the weights for each game and event, and then used the geometric mean to aggregate their judgments into a representative judgment for the group. when an inconcistency was discovered, we discussed the possibility of changing a judgment with the relevant person in order to reduce the inconsistency to an acceptable level and be closer to consensus on that set of judgments. we also did a literature review to obtain detailed data and information related to the criteria and the games themselves. finally, we applied our approach to rank the winning countries for the 2012 london summer olympics. table 5 lists all the games for this particular olympics. table 5 2012 london summer olympic games archery cycling gymnastics shooting triathlon athletics diving handball swimming volleyball badminton equestrian judo synchronized swimming water polo basketball fencing modern pentathlon table tennis weightlifting boxing field hockey rowing taekwondo wrestling canoe football sailing tennis ijahp article: saaty, sagir/ranking countries more reliably in the summer olympics international journal of the analytic hierarchy process 595 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.341 we categorized olympic games in terms of the “risk levels involved”, “energy spent” and “duration of the act”. tables 6, 7 and 8 present these categories, noticing that games could fall into different categories for different criteria. table 6 summer olympic games according to the “risk level involved” high medium low modern pentathlon football boxing fencing equestrian diving basketball canoeing taekwondo volleyball weightlifting water polo cycling badminton field hockey gymnastics sailing tennis table tennis judo athletics rowing archery wrestling triatlon shooting badminton swimming synchronized swimming handball table 6 implies that sports like diving and weightlifting have higher risks. they are usually considered dangerous sports that may cause harmful injuries and even death. on the other hand, according to the classification of “energy spent” in table 7, we obtain a different grouping of sports, i.e. fencing is a game that needs less energy when compared with other games like football. table 7 summer olympic games according to the “energy spent” high medium low weightlifting modern pentathlon sailing badminton archery athletics basketball volleyball table tennis fencing swimming football equestrian diving shooting wrestling water polo canoeing synchronized swimming field hockey tennis rowing handball cycling badminton gymnastics triatlon boxing judo taekwondo similarly, table 8 groups the games according to “duration of the act”. football and volleyball have long durations while others, like taekwondo and wrestling, generally take shorter times. ijahp article: saaty, sagir/ranking countries more reliably in the summer olympics international journal of the analytic hierarchy process 596 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.341 table 8 summer olympic games according to the “duration of the act” high medium low athletics triatlon canoeing field hockey archery fencing basketball voleyball gymnastic rowing boxing shooting cycling swimming sailing badminton taekwondo football handball diving water polo modern pentathlon table tennis equestrian weightlifting tennis synchronized swimming judo wrestling there were 29 games in the 2012 london olympics. in order to prioritize them we used the anp ratings module by evaluating them one at a time. the categories “low, medium, high” or “low, medium, high, very high” were chosen for the 17 criteria with an appropriate adjustment for the number of categories in each group (3 or 4). tables 9(a) and 9(b) show a screen view of the rating module from the super decisions software. five different experts from different professions were consulted. table 9 (a) screen view of the rating module to weight the games (for the criteria 1-7) ijahp article: saaty, sagir/ranking countries more reliably in the summer olympics international journal of the analytic hierarchy process 597 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.341 table 9(b) screen view of rating module to weight the games (for the criteria 8-17) table 10 summarizes all the game priorities. table 10 2012 london summer olympic games weights olympic game weight olympic game weight olympic game weight olympic game weight archery 0.019 equestrian 0.024 rowing 0.042 triathlon 0.045 athletics 0.053 fencing 0.024 sailing 0.038 volleyball 0.038 badminton 0.022 field hockey 0.027 shooting 0.023 water polo 0.029 basketball 0.045 football 0.054 swimming 0.046 weightlifting 0.032 boxing 0.023 gymnastics 0.048 synchronized swimming 0.030 wrestling 0.029 canoe 0.043 handball 0.021 table tennis 0.021 cycling 0.049 judo 0.024 taekwondo 0.024 diving 0.021 modern pentathlon 0.037 tennis 0.061 ijahp article: saaty, sagir/ranking countries more reliably in the summer olympics international journal of the analytic hierarchy process 598 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.341 tables 11 and 12 below present examples of category comparisons from the rating scale for the criteria “energy spent” and “physical characteristics”, respectively. table 11 category comparisons on the criteria “energy spent” energy spent high medium low priorities high 1 2 3 0.5396 medium 1/2 1 2 0.2970 low 1/3 1/2 1 0.1634 table 12 category comparisons on the criteria “physical characteristics” energy spent very high high medium low priorities very high 1 2 4 6 0.4990 high 1/2 1 3 5 0.3129 medium 1/4 1/3 1 2 0.1202 low 1/6 1/5 1/2 1 0.0679 table 13 presents the priorities of the events. table 13 2012 london summer olympic events priorities games events event priorities games events event priorities archery archery modern pentathlon modern pentathlon 0,011862 athletics decathlon 0.014525 rowing double sculls 0,012572 heptathlon 0.013967 eight 0.012273 jump 0.012536 four 0.01159 marathon 0.016925 lightweight four 011405 pole vault 0.012636 pair 0,012273 0.012273 relay 0.009546 quadruple sculls 0,012273 shot put 0.012382 single sculls 0,013401 0,012572 throw 0.0091 track 0.011834 walk 0.010107 badminton 0.007193 sailing 470 0.008177 ijahp article: saaty, sagir/ranking countries more reliably in the summer olympics international journal of the analytic hierarchy process 599 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.341 49 er 0.009312 elliot 0.009312 finn 0.011755 laser 0.012341 rs-x 0.009312 star 0.009312 basketball basketball 0.014477 shooting pistol 0.007328 rifle 0.004107 rifle 25m 0.004107 rifle 50m 0.007328 skeet 0.007328 trap 0.007328 boxing bantam 0.003306 swimming backstroke 0.008692 fly 0.003919 breaststroke 0.009397 heavy 0.004534 butterfly 0.010527 light 0.003428 freestyle 0.009397 light fly 0.003638 marathon 0.014701 light heavy 0.00362 medley 0.009397 light welter 0.00362 middle 0.004044 super heavy 0.007356 welter 0.006076 canoeing slalom 0.013815 synchronize d swimming synchronized swimming 0,00976 sprint 0.011023 cycling bmx 0.013819 table tennis table tennis 0.006955 mountain 0.01568 road 0.010855 track 0.01568 diving platform 0.0069 taekwondo 49 0.004095 springboard 0.00481 57 0.004095 58 0.004095 67 0.006577 68 0.007224 80 0.007657 equestrian dressage 0.004108 tennis tennis 0,01949 eventing 0.007657 jumping 0.005944 fencing epee 0.007842 foil 0.006532 sabre 0.004283 triathlon triathlon 0,014624 0.014624 ijahp article: saaty, sagir/ranking countries more reliably in the summer olympics international journal of the analytic hierarchy process 600 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.341 field hockey field hockey 0,00848 volleyball beach 0.012032 indoor 0.010881 football football 0,017058 water polo water polo 0,009229 gymnastic artistic 0.015236 weightliftin g 48 0.00652 rhythmic 0.013007 53 0.00652 trampoline 0.01403 56 0.00652 58 0.00652 62 0.00652 63 0.00652 69 0.00652 75 0.00652 +75 0.006704 77 0.006704 85 0.006704 94 0.009347 105 0.010491 105+ 0,010491 handball handball 0,00675 wrestling freestyle 0.009463 greco-roman 0.005714 greco-roman 120 0.005714 greco-roman 60 0.005994 grecoroman84 0.008799 judo j1 48 0.005035 j2 52 0.005035 j3 57 0.005035 j4 60 0.005546 j5 63 0.005546 j6 66 0.006 j7 70 0.006287 j8 73 0.006287 j9 78 0.006287 j10 81 0.005777 j11 90 0.007659 j12 100 0.007659 j13 100+ 0.007659 ijahp article: saaty, sagir/ranking countries more reliably in the summer olympics international journal of the analytic hierarchy process 601 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.341 4. how to obtain medal weights and their priorities for each event under different games the relative values of gold, silver and bronze medals were studied in an earlier work as follows (saaty, 2008). thirteen sets of comparisons and their actual outcomes are shown in tables 14-19. from the pairwise comparison judgments between different types of medals, one derives the priorities of different types of medals under 13 possible situations and then averages them to obtain the priorities of gold, silver and bronze medals. in table 14, a gold medal is very slightly favored over a silver medal and is not dependent on whether it is moderately or extremely favored over a bronze medal. table 14 gold slightly over silver gold silver bronze relative values gold silver bronze relative values gold 1 2 3 0.55 gold 1 2 9 0.61 silver 1/2 1 3/2 0.27 silver 1/2 1 5 0.32 bronze 1/3 2/3 1 0.18 bronze 1/9 1/5 1 0.07 table 15 shows that the gold medal is moderately favored over the silver medal and from very strongly to extremely over the bronze medal, and is noy dependent on whether a silver medal is moderately or strongly favored over a bronze medal. table 15 gold moderately over silver gold silver bronze relative values gold silver bronze relative values gold 1 3 7 0.64 gold 1 3 7 0.65 silver 1/3 1 3 0.26 silver 1/3 1 5 0.28 bronze 1/5 1/3 1 0.10 bronze 1/7 1/5 1 0.07 gold silver bronze relative values gold 1 3 9 0.67 silver 1/3 1 3 0.27 bronze 1/9 1/3 1 0.06 in table 16, the strength of a gold medal over a silver medal increases even more to between moderately and strongly and a gold medal is favored nearly very strongly to extremely over a bronze medal, while a silver medal is only moderately favored over a bronze medal in both cases. ijahp article: saaty, sagir/ranking countries more reliably in the summer olympics international journal of the analytic hierarchy process 602 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.341 table 16 gold between moderately and strongly over silver gold silver bronze relative values gold silver bronze relative values gold 1 4 6 0.69 gold 1 4 9 0.73 silver 1/4 1 3 0.22 silver 1/4 1 3 0.20 bronze 1/6 1/3 1 0.09 bronze 1/9 1/3 1 0.07 in table 17, a gold medal is strongly favored over a silver medal and very strongly to extremely favored over a bronze medal. table 17 gold strongly over silver gold silver bronze relative values gold silver bronze relative values gold 1 5 7 0.72 gold 1 5 9 0.74 silver 1/5 1 4 0.21 silver 1/5 1 4 0.19 bronze 1/7 1/4 1 0.07 bronze 1/9 1/4 1 0.07 in table 18, a gold medal is considered strongly more important than a silver medal and extremely more than a bronze medal while a silver medal is first moderately and then between moderately and strongly more important over a bronze medal. table 18 gold very strongly over silver gold silver bronze relative values gold silver bronze relative values gold 1 7 9 0.79 gold 1 7 9 0.78 silver 1/7 1 3 0.15 silver 1/7 1 4 0.16 bronze 1/9 1/3 1 0.06 bronze 1/9 1/4 1 0.06 in table 19, a gold medal is extremely important over a silver medal and a bronze medal while a silver medal is first strongly important and then extremely important than a bronze medal. table 19 gold extremely over silver gold silver bronze relative values gold silver bronze relative values gold 1 9 9 0.80 gold 1 9 9 0.78 silver 1/9 1 5 0.15 silver 1/9 1 9 0.18 bronze 1/9 1/5 1 0.05 bronze 1/9 1/9 1 0.04 the tables given above give the priorities of different types of medals under 13 different situations shown in tables 14-19, and then the priorities of different types of medals are ijahp article: saaty, sagir/ranking countries more reliably in the summer olympics international journal of the analytic hierarchy process 603 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.341 obtained by taking the geometric mean of the priorities derived from the judgment matrices above as seen in table 20. table 20 the 13 vectors of priorities from tables 10-15 and their average total geo. mean g* 0.55 0.61 0.64 0.65 0.67 0.69 0.73 0.72 0.74 0.79 0.78 0.80 0.78 9.15 0.6900 s 0.27 0.32 0.26 0.28 0.27 0.22 0.20 0.21 0.19 0.15 0.16 0.15 0.18 2.86 0.2000 b 0.18 0.07 0.10 0.07 0.06 0.09 0.07 0.07 0.07 0.06 0.06 0.05 0.04 0.99 0.0060 *g: gold, s: silver, b: bronz now we re-rank the countries that won medals in the 2012 london summer olympics according to a our method which considers not only the total medals won but also the weighted priority of each game and event under each game. the rank of the countries when simply counting medals won is shown in the sixth column of table 21. following the traditional way of counting the total number of medals won, the usa is the top ranked country followed by china and russia. however, the ranking of the countries that won medals in the 2012 london summer olympics (shown in the last column of table 21) is different when based on our finer approach. for example, the ukraine won 20 medals in boxing, canoeing, fencing, gymnastic, rowing, shooting, weighlifting and wrestling. this includes 6 gold medals 5 silver medals, and 9 bronze medals. when counting the total number of medals won, the ukraine is ranked 10 th place (if we use the total number of gold medals as a second criterion for the countries when the total number of medals are the same, then ukraine ranked as 12th place in the current methodology, south korea is 9 th , italy is 10 th , netherland is 11 th and ukraine is 12 th ), but when considering the priorities of the events and the games by proposed methodology the ukraine’s rank moves to 15 th place. this is because gymnastics is one of the important games considered in this research, and the ukraine won a bronze medal in gymnastics. on the other hand, they won 5 medals in boxing and 2 medals in fencing events which have relatively lower priorities. another example is latvia which won only two medals, one in cycling (gold) and the other in beach volleyball (bronze). when considering the number of medals won, latvia is one of the lowest ranked countries by current ranking system. as we explained above in the ukraine example, the countries that have an equal number of medals are ranked as the same. latvia, bulgaria, indonesia, dominican republic and so on are ranked 25 th by the current ranking. however, although latvia is ranked 25 th , it is actually 57 th if we use the number of gold medals as the second criterion (when we just count the countries above latvia in the current order, latvia is 57 th not 25 th ). when we look at it like this, we can more easily interpret the difference from the proposed rank and say that latvia deserves a better rank even though it has just two medals because the priorities of those games are high. our methodology ranked latvia 42 th because the priorities of the events in which latvia won medals are relatively high. a similar example is tunisia which won only three medals. these medals were won in the marathon, track and swimming, all events with higher priorities as compared to other games. ijahp article: saaty, sagir/ranking countries more reliably in the summer olympics international journal of the analytic hierarchy process 604 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.341 table 21 2012 london olympics medals current and proposed ranking comparisons current methodology proposed methodology country gold silver bronze total medal count current ranking total priority score proposed ranking usa 46 29 29 104 1 0.4348 1 china 38 27 23 88 2 0.2515 3 russian 24 25 32 81 3 0.2237 4 great britain 29 17 19 65 4 0.2868 2 germany 11 19 14 44 5 0.1288 5 japan 7 14 17 38 6 0.0756 8 australia 7 16 12 35 7 0.0899 7 france 11 11 12 34 8 0.1248 6 south korea 13 8 7 28 9 0.0710 9 italy 8 9 11 28 9 0.0686 11 netherlands 6 6 8 20 10 0.0502 13 ukraine 6 5 9 20 10 0.0404 15 hungary 8 4 6 18 11 0.0687 10 canada 1 5 12 18 11 0.0356 20 spain 3 10 4 17 12 0.0402 16 brazil 3 5 9 17 12 0.0256 27 cuba 5 3 7 15 13 0.0258 26 kazakhstan 7 1 5 13 14 0.0396 17 new zeland 6 2 5 13 14 0.0536 12 belarus 2 5 5 12 15 0.0384 18 iran 4 5 3 12 15 0.0300 23 jamaica 4 4 4 12 15 0.0450 14 kenya 2 4 5 11 16 0.0345 21 czech republic 4 3 3 10 17 0.0278 24 azerbaijan 2 2 6 10 17 0.0202 32 poland 2 2 6 10 17 0.0209 31 romania 2 5 2 9 18 0.0238 29 denmark 2 4 3 9 18 0.0358 19 ijahp article: saaty, sagir/ranking countries more reliably in the summer olympics international journal of the analytic hierarchy process 605 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.341 colombia 1 3 4 8 19 0.0190 34 sweden 1 4 3 8 19 0.0190 34 ethiopia 3 1 3 7 20 0.0343 22 mexico 1 3 3 7 20 0.0177 37 georgia 1 3 3 7 20 0.0074 50 north korea 4 0 2 6 21 0.0195 33 croatia 3 1 2 6 21 0.0215 30 south africa 3 2 1 6 21 0.0275 25 india 0 2 4 6 21 0.0146 40 mongolia 0 2 3 5 22 0.0043 56 turkey 2 2 1 5 22 0.0163 38 lithuania 2 1 2 5 22 0.0094 45 ireland 1 1 3 5 22 0.0040 57 trinidad and tobago 1 0 3 4 23 0.0095 44 switzerland 2 2 0 4 23 0.0189 35 norway 2 1 1 4 23 0.0186 36 slovenia 1 1 2 4 23 0.0048 59 argentina 1 1 2 4 23 0.0098 43 serbia 1 1 2 4 23 0.0063 53 malaysia 0 1 3 4 23 0.0021 64 finland 0 1 2 3 24 0.0037 58 tunisia 1 1 1 3 24 0.0254 28 uzbekistan 1 0 2 3 24 0.0085 47 armenia 0 1 2 3 24 0.0160 39 belgium 0 1 2 3 24 0.0087 47 thailand 0 2 1 3 24 0.0024 63 indonesia 0 1 1 2 25 0.0020 65 dominic republic 1 1 0 2 25 0.0083 48 latvia 1 0 1 2 25 0.0109 42 chinese tapei 0 1 1 2 25 0.0004 73 estonia 0 1 1 2 25 0.0027 61 puerto rico 0 1 1 2 25 0.0031 60 bulgaria 0 1 1 2 25 0.0068 51 ijahp article: saaty, sagir/ranking countries more reliably in the summer olympics international journal of the analytic hierarchy process 606 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.341 egypt 0 2 0 2 25 0.0025 62 moldova 0 0 2 2 25 0.0016 67 qatar 0 0 2 2 25 0.0020 64 greece 0 0 2 2 25 0.0019 66 singapore 0 0 2 2 25 0.0014 69 algeria 1 0 0 1 26 0.0083 47 bahamas 1 0 0 1 26 0.00668 52 botswana 0 1 0 1 26 0.0024 63 guatemala 0 1 0 1 26 0.0020 65 grenada 1 0 0 1 26 0.0083 49 uganda 1 0 0 1 26 0.0118 41 slovakia 0 1 0 1 26 0.0050 55 montenegro 0 1 0 1 26 0.0014 69 venezuela 1 0 0 1 26 0.0055 54 bahrain 0 0 1 1 26 0.0090 46 gabon 0 1 0 1 26 0.0015 68 morocco 0 0 1 1 26 0.0012 70 portugal 0 1 0 1 26 0.0022 64 tajikistan 0 0 1 1 26 0.0003 74 cyprus 0 1 0 1 26 0.0025 62 afghanistan 0 0 1 1 26 0.0007 71 hong kong 0 0 1 1 26 0.0016 67 saudi arabia 0 0 1 1 26 0.0006 72 kuwait 0 0 1 1 26 0.0007 71 * the rows in bold show that the rank of a country obtained by the proposed method is the same as its current rank 5. conclusion training for competition in the olympics requires time and resources with different types of events having different characteristics. individual games require more concentration while team games require more cooperation. to become successful in gymnastics, one usually has to start training at a very young age (five or six). the duration of a volleyball game is usually about one to two hours, and the marathon takes about two hours and requires more energy when compared to other events. on the other hand, archery takes only a few seconds. thus, a medal should be given a different value depending on which game it is won for. we propose that ranking countries in the olympics should not only be decided by counting the total medals won, but also by the type of game in which the medal was won. in this study we prioritized different games and the events under each ijahp article: saaty, sagir/ranking countries more reliably in the summer olympics international journal of the analytic hierarchy process 607 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.341 game. our elaborate approach to the olympics shows that counting the total numbers of medals won is not a bad way of ranking countries. finally, while it is known that multicriteria decision making is very important in optimal allocation of limited resources, it may not always produce radically better results than much simpler existing ways of ranking. for the last winter olympics (2014) in sochi, russia, there were more noticeable differences in the two rankings methods so that norway which ranked third according to the number of medals won, actually ranked first according to prioritization of the different kinds of games. this is a significant finding because ranking first would have been a very distinguished and celebrated outcome for norway. we also performed compatibility index calculations to compare actual ranking and the estimated one as shown in appendix. since the ranks are slightly different, the index is obtained was 1,22 which could be acceptable and reasonable. ijahp article: saaty, sagir/ranking countries more reliably in the summer olympics international journal of the analytic hierarchy process 608 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.341 references andrew, b. b., meghan, r. b., (2000). who wins the olympic games: economic development and the medal total, national bureau of economic development, nber working paper series, no. 7998, jel no. o10, l83. bernard, a. b., meghan r. b., (2000). who wins the olympic games: economic development and medal total, nber working paper, 7998. wade, d. p., (2006) predicting the medal wins by country at the 2006 winter olympic games: an econometrics approach, national graduate institute for policy studies, tokyo, japan. forsyth, s., china 2008 olympic medal tally by population, and china 2008 olympic medal tally by gross domestic product (gdp), http://simon.forsyth.net/olympics.htm. saaty t.l., (2004). decision makingthe analytic hierarchy and network processes (ahp/anp), journal of systems science and systems engineering, 13(1), 1-35. doi:10.1007/s11518-006-0151-5 saaty, t. l., (2008). who won the 2008 olympics? journal of systems science and systems engineering, 17, 4, 473-486. doi: 10.1007/s11518-008-5092-8 the world factbook 2013-14. washington, dc: central intelligence agency, 2013. https://www.cia.gov/library/publications/the-world-factbook/index.html http://www.andrew.bernard.org/ http://www.kellogg.northwestern.edu/faculty/directory/busse_meghan.aspx http://simon.forsyth.net/olympics.html http://simon.forsyth.net/olympics.htm https://www.cia.gov/library/publications/the-world-factbook/index.html ijahp article: saaty, sagir/ranking countries more reliably in the summer olympics international journal of the analytic hierarchy process 609 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.341 appendix compatibility index analysis pairwise comparison matrix from actual data a1 a2 a3 a4 a5 a6 a7 a8 a1 1 1,14433 1,32934 1,59712 2,00000 2,67470 3,96429 7,92857 a2 0,8738739 1,00000 1,16168 1,39568 1,74775 2,33735 3,46429 6,92857 a3 0,7522523 0,86082 1,00000 1,20144 1,504504505 2,01205 2,98214 5,96429 a4 0,6261261 0,71649 0,83234 1 1,25225 1,67470 2,48214 4,96429 a5 0,5 0,57216 0,66467 0,79856 1 1,33735 1,98214 3,96429 a6 0,3738739 0,42784 0,49701 0,59712 0,747747748 1 1,48214 2,96429 a7 0,2522523 0,28866 0,33533 0,40288 0,504504505 0,674699 1 2,00000 a8 0,1261261 0,14433 0,16766 0,20144 0,252252252 0,337349 0,5 1 pairwise comparison matrix from estimated data a1 a2 a3 a4 a5 a6 a7 a8 a1 1 1,743141 1,959768 1,528591 3,403727 5,798942 4,876529 3,512821 a2 0,573677 1,00000 1,12427 0,87692 1,95264 3,32672 2,79755 2,01522 a3 0,510265 0,88946 1,00000 0,77999 1,736801 2,95899 2,48832 1,79247 a4 0,654197 1,14036 1,28207 1 2,22671 3,79365 3,19021 2,29808 a5 0,293796 0,51213 0,57577 0,44909 1 1,70370 1,43270 1,03205 a6 0,172445 0,30060 0,33795 0,26360 0,265569 1 0,84093 0,60577 a7 0,205064 0,35746 0,40188 0,31346 0,697981 1,189153 1 0,72035 a8 0,284672 0,49622 0,55789 0,43515 0,968944 1,650794 1,388209 1 transpose of comparison matrix from estimated data a1 a2 a3 a4 a5 a6 a7 a8 a1 1 0,573677 0,510264599 0,654197 0,29379562 0,172445 0,20506387 0,284672 a2 1,7431412 1,00000 0,88946 1,14036 0,51213 0,30060 0,35746 0,49622 a3 1,9597675 1,12427 1,00000 1,28207 0,57577 0,33795 0,40188 0,55789 a4 1,5285914 0,87692 0,77999 1 0,44909 0,26360 0,31346 0,43515 a5 3,4037267 1,95264 1,736801242 2,22671 1 0,265569 0,69798137 0,968944 a6 5,7989418 3,32672 2,95899 3,79365 1,70370 1 1,18915344 1,650794 a7 4,8765295 2,79755 2,48832 3,19021 1,43270 0,84093 1 1,388209 a8 3,5128205 2,01522 1,79247 2,29808 1,03205 0,60577 0,72035 1 result of hadamard (cell-wise) multiplication of previous two matrices a1 a2 a3 a4 a5 a6 a7 a8 a1 1 0,6564758 0,678315814 1,044833 0,587591241 0,461239 0,81293176 2,257039 a2 1,52329 1 1,03327 1,59158 0,89507 0,70260 1,23833 3,43811 a3 1,47424 0,96780 1 1,540334 0,86625 0,67998 1,19846 3,32742 a4 0,95709 0,62831 0,649209948 1 0,56238 0,44145 0,77805 2,16019 a5 1,70186 1,11723 1,154400826 1,778163 1 0,35516 1,38350 3,84117 a6 2,16807 1,42329 1,47064 2,26527 1,27394 1 1,76250 4,89342 a7 1,23012 0,80754 0,83441 1,28527 0,72281 0,56738 1 2,77642 a8 0,44306 0,29086 0,30053 0,46292 0,26034 0,20436 0,36018 1 ijahp article: saaty, sagir/ranking countries more reliably in the summer olympics international journal of the analytic hierarchy process 610 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.341 total 10,49773 6,89150 7,12077 10,96837 6,16837 4,41215 8,53394 23,69377 cell sum of previous matrix = 78,286 number of alternatives (n) = 8 saaty compatibility index = sum/n 2 1,22 microsoft word 5saatyvargaswhitaker_pp121-134_vol_1_issue_2_2009.docx ijahp essay: saaty, vargas, whitaker/addressing criticisms of the ahp international journal of the 121 vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process addressing with brevity criticisms of the analytic hierarchy process thomas l. saaty university of pittsburgh saaty@katz.pitt.edu luis g. vargas university of pittsburgh lgvargas@pitt.edu rozann whitaker creative decisions foundation rozann@creativedecisions.net a new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it. -max planck abstract the paper provides an overview that covers the main criticisms of the ahp and the authors’ replies to them. because there have been many papers that reply to criticisms, the thrust here is to classify them and reply to them briefly in each category without giving lengthy repetitions of what is already known in the literature. keywords: rank reversal, inconsistent judgments, preserving rank, fundamental scale, pairwise comparisons 1. introduction in this paper we address five types of criticisms of the ahp. one is the concern with illegitimate changes in the ranks of the alternatives, called rank reversal, upon changing the structure of the decision. it was believed that rank reversal is legitimate only when criteria or priorities of criteria or changes in judgments are made. rank reversals were shown by critics to occur when using comparisons and relative measurement that is essential in prioritizing criteria and also alternatives on intangible criteria in two ways: first, when new alternatives are added or old ones deleted; and second, when new criteria are added or old ones deleted with the caveat that the priorities of the alternatives would be tied under these criteria and hence argued that the criteria should be irrelevant when ranking the alternatives. rank reversals that followed such structural changes were attributed to the use of relative measurement and normalization. rating alternatives one at a time with respect to the criteria using the ideal mode, always preserves rank. also, the ideal mode is used with paired comparisons to preserve rank. but rank can and should reverse under more general conditions than had previously been recognized as in introducing copies or near copies of alternatives and criteria turn out not to always be so rob typewritten text http://dx.doi.org/10.13033/ijahp.v1i2.53 ijahp essay: saaty, vargas, whitaker/addressing criticisms of the ahp international journal of the 122 vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process strictly independent among themselves and from the alternatives. the second concern is about inconsistent judgments and their effect on aggregating such judgments or on deriving priorities from them. a modicum of intransitivity and numerical inconsistency, usually not considered or thought to be permissible in other theories, is permissible in the ahp so that decisions can be treated realistically rather than axiomatically truncated. a condition that may not hold with inconsistent judgments is pareto optimality. pareto optimality is an ordinal relation which demands of a method used to aggregate judgments of individuals in a group to a representative collective judgment for that group that when all individuals in the group prefer a to b then the group judgment must prefer a to b. because in the ahp judgments are not ordinal, it is possible to aggregate the individual judgments into a representative group judgment with or without pareto optimality. another condition also inherited from expected utility theory has to do with a relation called condition of order preservation (cop): for all alternatives 1 2 3 4, , ,x x x x ,such that 1x dominates 2x and 3x dominates 4x , if the evaluator's judgments indicate the extent to which 1x dominates 2x is greater than the extent to which 3x dominates 4x , then the vector of priorities w should be such that, not only 1 2( ) ( )w x w x> and 3 4( ) ( )w x w x> (preservation of order of preference) but also that 31 2 4 ( )( ) ( ) ( ) w xw x w x w x > (preservation of order of intensity of preference). this condition holds when judgments are consistent but may or may not hold when they are inconsistent. it is axiomatically imposed, sacrificing the original intent of the ahp process to derive priorities that match the reality represented by the judgments without forcing consistency. the third criticism has to do with attempts to preserve rank from irrelevant alternatives by combining the comparison judgments of a single individual using the geometric mean (logarithmic least squares) to derive priorities and also combining the derived priorities on different criteria by using multiplicative weighting synthesis. the fourth criticism has to do with people trying to change the fundamental scale despite the fact that it is theoretically derived and tested by comparing it with numerous other scales on a multiplicity of examples for which the answer was known. the fifth and final criticism has to do with whether or not the pairwise comparisons axioms are behavioral and spontaneous in nature to provide judgments. interestingly, the ahp/anp provides a way to make complex decisions in the most general structures encountered in real life. it makes it possible to derive priorities for all the factors in such structures and synthesize them for an overall outcome, as no other method can because one can build scales for tangibles and intangibles, yet we know little about criticisms of framing and validating problems within such a wide perspective that includes structures, not only for dependence and feedback, but also for benefits, opportunities, costs and risks analyzed separately and then synthesized for the final outcome or in conflict resolution with or without a moderating negotiator. we give an overview that covers the main criticisms and our replies to them. because we and others have written numerous papers in reply to criticisms, we have opted to classify them briefly in each category without giving lengthy repetitions of what is already known in the literature. ijahp essay: saaty, vargas, whitaker/addressing criticisms of the ahp international journal of the 123 vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 2. rank reversal 2.1 change in structure by adding/deleting alternatives in relative measurement, unlike measurement on a scale with an arbitrary unit where alternatives are assigned a value independently of other alternatives, when alternatives are compared on several criteria and their weight aggregated, their ranks can change when alternatives are added or deleted (watson and freeling 1982; belton and gear 1983; dyer and ravinder 1983; dyer 1990). the ahp with its ideal mode preserves rank in rating alternatives (millet and saaty 2000). this is equivalent to measuring alternatives one at a time. adding or deleting alternatives can have no effect on the value and rank of any other alternative. all known software programs that people use implement the ideal mode. in addition when paired comparisons are used, again the ideal mode is often used to preserve rank by idealizing only the first set of alternatives but not after. thereafter, any new alternative is only compared with the ideal and its priority value is allowed to exceed one before weighting and adding and normalizing. this way the rank of the existing alternatives is always preserved. it is interesting to point out that the distributive mode of the ahp (uniqueness is important), the ideal mode of the ahp (uniqueness is not important), and utility functions (use of interval scales for the ideal), yield the same ranking of alternatives with surprisingly high frequency, except for the case of copies or near copies of an alternative in which the distributive mode always reverses rank, which is legitimate when the uniqueness of the most preferred alternative is important (saaty and vargas 1993). here is an illustration of rank reversal due to corbin and marley (corbin and marley 1974). the first example concerns a lady in a small town, who wishes to buy a hat. she enters the only store in town, and finds two hats, a and b, that she likes equally well although she leans toward a. however, suppose that the sales clerk discovers a third hat, 1a identically to a. then the lady may well choose hat b for sure (rather than risk the possibility of seeing someone wearing a hat just like hers), a result that contradicts regularity. luce and raiffa, wrote in their book games and decisions, published in 1957 four variations on the axiom about whether rank should or should not be preserved with counterexamples in each case and without concluding that it always should and why. they write: "adding new acts to a decision problem under uncertainty, each of which is weakly dominated by or is equivalent to some old act, has no effect on the optimality or nonoptimality of an old act. and elaborate it with if an act is non optimal for a decision problem under uncertainty, it cannot be made optimal by adding new acts to the problem. ijahp essay: saaty, vargas, whitaker/addressing criticisms of the ahp international journal of the 124 vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process and press it further to the addition of new acts does not transform an old, originally non-optimal act into an optimal one, and it can change an old, originally optimal act into a non-optimal one only if at least one of the new acts is optimal. and even go to the extreme with: the addition of new acts to a decision problem under uncertainty never changes old, originally non-optimal acts into optimal ones. and finally conclude with: the all-or-none feature of the last form may seem a bit too stringent ... a severe criticism is that it yields unreasonable results." the question is not whether rank should be preserved, because it is widely believed that it cannot and should not always be preserved (tversky et al. 1990), but it is whether or not the assumption of independence applies, an assumption used by most multi-criteria methods. adding copies or near copies of an alternative until the universe is full of them can depreciate the value and also the rank of that alternative and, as a counter example, negates the possibility of proving a theorem that the rank of independent alternatives must always be preserved when the judgments remain the same and no criteria are added or deleted or their weights changed. a criterion such as “manyness” that represents the number of alternatives cannot be used in the ranking because it forces the dependence of the ranking of each alternative on the existence of every other alternative thus contradicting the assumption of independence. it is illogical (or we might say also wrong) for all multicriteria methods which all use the rating of alternatives one at a time not to take this into account. utilitarian philosophers of the 18th century believed that people ought to desire those things that will maximize their utility. however, this utilitarian viewpoint was abandoned because it was deemed that utility was impossible to measure. instead, structural accounts of rationality and formal definitions of utility such as rational choice theory were favored. in rational choice theory, the criteria are assumed utility independent and the condition empirically tested. but because the criteria cannot be separated from the alternatives, the resulting weights are not really importance weights but scaling constants. consequently, according to strong advocates of this theory, independence of the criteria among themselves must be assumed (keeney and raiffa 1976; kamenetzky 1982). contrary to this assumption, in the ahp/anp everything can depend on everything else including itself!!! in the ahp/anp, rank is always allowed to change. it is preserved only when the criteria are conditions imposed on the alternatives and possibly attributes that have had long standing and acquired an importance of their own apart from any particular alternative (saaty 1991a). for example, we all have the habit of ascribing human kind of rationality to how the universe operates, and assign ijahp essay: saaty, vargas, whitaker/addressing criticisms of the ahp international journal of the 125 vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process rationality a high priority. it is not the way some dervishes and ascetics and certainly not the way plants and animals feel about it. 2.2 change in structure by adding/ deleting criteria in general, it is known in decision making that if one alters criteria or criteria weights then the outcome of a decision will change possibly leading to rank reversal. this is precisely what some authors use to criticize the ahp. there are two situations. the first is called “wash criteria” which involves the deletion of criteria that are assumed irrelevant because the alternatives have equal or nearly equal priorities under them (finan and hurley 2002). the second is called “indifferent criteria” which involves the addition of criteria again assumed irrelevant for the same reason as “wash criteria” (perez et al. 2006). in the first case the authors made the error of renormalizing the weights of the remaining criteria that then gave rise to rank reversal because the weights of the criteria were changed (saaty and vargas 2006). in the second case the addition of a new criterion that was irrelevant also led to rank reversal for exactly the same reason of changing the weights of the criteria. it is surprising that anyone would want to add irrelevant criteria and use it to make an important decision. this approach treats the weights of the criteria not as representative of their importance but as scaling constants like in multi-attribute utility theory (keeney and raiffa 1976). the correct approach to deal with wash and indifferent criteria is not to delete them or add them but simply to, in the former case, assign zero priorities to the alternatives and keep that criterion, and in the latter case not to add them or if added to, consider this a new decision respecting the influence of added criteria on the final outcome which, as we said above, could lead to different priorities and ranks. 3. consistency, pareto optimality and order preservation pareto optimality in ordinal preference settings is a condition imposed on preferences which says that if every member of a group prefers a to b, then the group must also prefer a to b. this condition is also known as unanimity. underlying this condition is the hidden assumption of the transitivity of preferences. in the ahp with its reciprocal condition on the judgments, the geometric mean has been shown to be the unique way to derive a group judgment from the individual judgments under fairly general conditions. note that pareto optimality as used in economic and social practice applies to a final ordering of each individual of all the alternatives and not to judgments that obtain that order. in the ahp because preference order is indicated by priorities rather than by an ordinal statement of preference, pareto optimality always holds when the stated condition is satisfied, and there is no problem with pareto optimality. when pareto optimality is applied to judgments, there two possibilities: the first is when all judgments in a pairwise comparison matrix ( )ija a= are consistent (i.e., , , ,ij jk ika a a i j k= ∀ and ija have the form ij i ja w w= where the iw ’s are the priorities), in which case one has transitivity and also pareto optimality. the second is when the judgments are inconsistent. in this case pareto optimality holds under restrictive conditions like row dominance for each individual, i.e., there is an ordering of the rows and corresponding judgments in each row. ijahp essay: saaty, vargas, whitaker/addressing criticisms of the ahp international journal of the 126 vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process one may ask: why should pareto optimality be imposed on a method that uses cardinal preferences when it already has a process for aggregating individual judgments, along with the importance of the individuals involved, into a group judgment? if the members of the group are agreeable to using the geometric mean to combine their judgments, even if pareto optimality is not satisfied, why should their combined judgment be any less valid than any other procedure that satisfies pareto optimality? finally, pareto optimality is not universally regarded as a desired condition in all decisions. a common criticism of a state of pareto efficiency is that it does not necessarily result in a socially desirable distribution of resources, as it may lead to unjust and inefficient inequities (sen 1993; barr 2004). a condition that mirrors preferences expressed with interval scale value functions is the condition of order preservation (cop) (bana e costa and vansnick 2008). in interval scale value theory, a value function v must satisfy the condition that if a consequence i is preferred to a consequence j more than a consequence h is preferred to a consequence k then v(i)-v(j)>v(h)-v(k). note that preferences are ordinal and hence no intensity of preference or judgment is involved. on the other hand, an individual imposing cop assigns judgments to the preferences. thus, if ij hka a> then i h j k w w w w > . this condition is always satisfied if the judgments are consistent because all logical methods of deriving priorities yield the same priorities. when the judgments are inconsistent, only the eigenvector obtains priorities that capture the transitivity of dominance reflected in the judgments. a major property of consistent judgments arranged in a matrix ( )ija a= is that it satisfies the condition 1k ka n a−= , where n is the order of a, so all powers of a are essentially equal to a. now dominance of an inconsistent matrix no longer satisfies this condition and one must consider priorities derived from direct dominance as in the matrix itself, second order dominance obtained from the square of the matrix and so on. the total dominance of each element is obtained as the normalized sum of its rows. the result is an infinite number of priority vectors each representing a different order of dominance. the cesaro sum of these vectors is equal to the priority vector obtained from the limiting powers of the matrix. thus, only the eigenvector gives the correct ordering and priority values. cop imposes a condition on the priorities based solely on the original preferences without regard to dominance of higher order, and it thus likely to lead to the wrong priorities and order. in fact, we know of the existence of examples to support this statement (salomon 2008). cop was devised for use in a method known as macbeth (bana e costa et al. 2003). however, the value functions derived are interval scales, so cop is expressed as ratios of differences. finding the value function that satisfies cop in macbeth involves an optimization technique that yields a non-unique solution!! to summarize, cop forces the condition that 1ija > should imply i jw w> , which is not always true when the judgments are inconsistent; violates the integrity of the eigenvector as the way to derive priorities capturing higher order interactions among the judgments; artificially forces adjustment of the judgments without asking the decision maker if the ijahp essay: saaty, vargas, whitaker/addressing criticisms of the ahp international journal of the 127 vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process altered value is acceptable in his framework of understanding; and yields invalid results for single matrices with known measurements. 4. priority derivation and synthesis with the geometric mean a number of people, in their concern with always preserving rank, look for schemes to synthesize inconsistent judgments and also priorities (holder 1990). the only other method that has been proposed and pursued in the literature has been the geometric mean for a single matrix (barzilai 1997), in which the elements in each row of the matrix are multiplied, the nth root taken and the resulting vector normalized. this process does not capture the effect of transitivity of dominance in the case of inconsistent judgments and hence, it can lead to wrong priorities and order (saaty 1991b). synthesizing priorities derived in any manner by raising them to the power of the priority of the corresponding criterion and then multiplying the outcome (lootsma 1993; barzilai and lootsma 1997) has the shortcoming that 0 1 and 0 p qx y p q x y< < < < < ⇒ > for some p and q. this means that an alternative that has a smaller value under a less important criterion is considered to be more important than an alternative that has a larger value under a more important criterion, which is absurd. one can also show the absurdity of this process of synthesis because it yields wrong known results. by considering alternatives with known measurements under two or more criteria which then inherit their importance from the measurements under them, normalizing these measurements, raising them to the power of the priority of their corresponding criterion and multiplying, one obtains a different outcome than simply adding the measurements and then normalizing them (vargas 1997). 5. altering the fundamental scale a number of authors have proposed changes in the fundamental 1-9 scale of the ahp more as a passing suggesting without either a proof of the resulting improvement if any or validation examples to test their assertions (ma and zheng 1991; salo and hamalainen 1997). 6. are the axioms about comparisons behaviorally meaningful? people who subscribe to expected utility theory claim (see for example, dyer 1990, p.251) “…each of these axioms has a clear and obvious meaning as a description of choice behavior. therefore, each axiom can be debated on the basis of its appeal as a normative descriptor of rationality, and each axiom can also be subjected to empirical testing.” this statement is the basis for the criticism of the fundamental scale in the ahp. are paired comparisons behaviorally based or are they an invention of ours? the harvard psychologist arthur blumenthal (blumenthal 1977) believes that there are two types of judgment: “comparative judgment which is the identification of some relation between two stimuli both present to the observer, and absolute judgment which involves the relation between a single stimulus and some information held in short term memory about some former comparison stimuli or about some previously experienced ijahp essay: saaty, vargas, whitaker/addressing criticisms of the ahp international journal of the 128 vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process measurement scale using which the observer rates the single stimulus.” in the analytic hierarchy process (ahp) we call the first relative measurement and the second absolute measurement. in relative measurement we compare each alternative with many other alternatives and in absolute measurement we compare each alternative with one ideal alternative we know of or can imagine, a process we call rating alternatives. the first is descriptive and is conditioned by our observational ability and experience and the second is normative, conditioned by what we know is best, which of course is relative. comparisons must precede ratings because ideals can only be created through experience using comparisons to arrive at what seems best. it is interesting that in order to rate alternatives with respect to an ideal as if they are independent can only be done after having made comparisons that involve dependence to create the ideal or standard in the first place. making comparisons is fundamental and intrinsic in us. they are not an intellectual invention nor are they something that can be ignored. the need for quantifying the intensity of preferences is all around us. donald j. boudreaux writes: “my third reason for not voting is that voting registers only each voter’s order of preferences and not that voter’s intensity of preferences. unlike in private markets where i can refuse to buy a good or service if i judge its price to be too high—and then decide to buy that same product if its price falls—in elections each voter merely gets to say which candidate he prefers above all who are on the ballot. if i vote for smith rather than jones, this means only that i prefer smith to jones. my vote for smith reveals nothing about how much i prefer smith to jones.” (boudreaux 2008) paired comparisons consist of two steps. first, as in utility theory, there is a binary comparison, for example, alternative a is preferred to alternative b. second, we must decide with how much more intensity we prefer a to b. because in expected utility theory preferences are built on lotteries, it is already assumed that intensity of preference is accounted for, even though utilities do not always represent intensity of preference (sarin 1982). without the probability function one is left with ordinal utilities which yield only ranking. probabilities play the role of the fundamental scale in the ahp. on the other hand, the ahp articulates the intensity of pairwise comparison preferences using an instinctively built-in absolute scale. the mathematician and cognitive neuropsychologist stanislas dehaene writes: “introspection suggests that we can mentally represent the meaning of numbers 1 through 9 with actual acuity. indeed, these symbols seem equivalent to us. they all seem equally easy to work with, and we feel that we can add or compare any two digits in a small and fixed amount of time like a computer.” (dehaene 1997) pareto (1848-1923) rejected altogether the idea that quantities of utility mattered. he observed that if we map preferences onto edgeworth’s indifference curves, we know everything necessary for economic analysis. to map these preferences, we make pairwise comparisons between possible consumption bundles. the agent will either be indifferent between each bundle, or else will prefer one to the other. by obtaining comparisons between all bundles, we can draw a complete map of an individual’s utility. these comparisons were ordinal in nature and did not go far enough to represent intensity of preference. ijahp essay: saaty, vargas, whitaker/addressing criticisms of the ahp international journal of the 129 vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process 7. general observations we gave above arguments about the major issues. the references include many papers we know about, our published responses to some and also references to papers we wrote mostly on the subject of rank preservation and reversal. the first paper questioning some aspect of the ahp was that by watson and freeling (watson and freeling 1982). the authors questioned the validity of the questioning process by means of which judgments are elicited (saaty et al. 1983). belton and gear (belton and gear 1983) built an example of a simple hierarchy with three criteria and 3 alternatives, and showed that adding a copy of an alternative made rank reversal was possible (saaty and vargas 1984). the same problem was reported (dyer and ravinder 1983). later dyer (dyer 1990) used the same arguments to challenge the validity of the axioms and the principle of hierarchic composition, and provided his own solution which he considered to be consistent with expected utility theory!! (harker and vargas 1990; saaty 1990) holder (holder 1990) criticized the eigenvector method by questioning the validity of the optics experiment and the principle of hierarchic composition, for the same reason which was rank reversal (saaty 1991b). the same criticisms were voiced in (lootsma 1993; salo and hamalainen 1997; finan and hurley 2002; hurley 2002; perez et al. 2006). all these authors criticize the principle of hierarchy composition. salo and hamalainen (salo and hamalainen 1997) also criticize the composition principle in the analytic network process. other authors have criticized the ahp on the grounds that the 1-9 scale is not appropriate (ma and zheng 1991; lootsma 1993; salo and hamalainen 1997). in group decision making the geometric mean has been criticized because it violates pareto optimality (lootsma 1993). there have been people who expect to put their own default numbers in an ahp structure without input about the particular decision and get rational numerical outcomes. one such person has published notes against the ahp and other decision methods with strongly made arguments mostly published in unrefereed journals is jonathan barzilai (barzilai 1998). he has been promising for many years to provide the scientific community his own decision theory. the third author has shown (whitaker 2004; whitaker 2007a) in detail where his thinking is in error. one of his fundamental assumptions is that in order for paired comparisons to be valid, the underlying scale must be a ratio scale. he totally ignores the fact that paired comparison judgments are represented by numbers from an absolute scale and that the derived priority scales are relative scales of absolute numbers with no zero and no unit. in addition, he assumes that criteria are equally weighted when in fact he has assigned actual measurements to the alternatives on a known scale. it can be easily shown that when several criteria have alternatives measured on the same scale the weights of the criteria are given by the sum of the measurements of the alternatives under each divided by the sum under all the criteria. in that case, weighting the relative measurements of the alternatives under each criterion by the weight of that criterion and adding over the criteria yields the same relative measurement as one obtains by first simply summing the measurements for each ijahp essay: saaty, vargas, whitaker/addressing criticisms of the ahp international journal of the 130 vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process alternative and then normalizing the final sums for all the alternatives to put their total measurements in relative form. this is a common error that people make. indeed it is the same error made by s. schenkerman (schenkerman 1997) in his paper in which he starts out by assuming some measurements for the alternatives with respect to each criterion so that he can perform arithmetic operations on them to get an answer. he then sets up an ahp problem to see if it gives the same answer as he got, but in relative terms. he assigns the criteria equal weights, but he has already given the alternatives measurements under each criterion and a fortiori he is no longer free to assign the criteria arbitrary weights of his own choice. here we note that there are many ways to combine measurements on objects using formulas to get an overall answer. it is naïve to assume that the straightforward process of weighting and adding in a hierarchic structure should always yield results that coincide with the formula even if one can, by carefully modeling the problem as a decision, use the hierarchic weighting approach to obtain the same answer. for attributes/properties for which a scale has not yet been developed, barzilai assumes that there cannot be information about them that can be measured and hence paired comparisons with respect to criteria are invalid. he announces by fiat and without proof that hierarchic composition is linear and that it generates nonequivalent value functions from equivalent decompositions. in fact both theory and many examples show that hierarchic composition is nonlinear, and the value functions generated are valid when it is done correctly. replies to the issues in such papers have been properly addressed in the literature and will not be repeated here. 8. conclusions – our concern with validation in decision making in general it is considered scientifically justifiable to require some sort of objective validation of numbers derived as answers in decision making. people in the field of decision making, particularly the normative kind, seem to be oblivious to the issue of validation as if it is a requirement they do not have to heed. it is true that judgments and priorities are subjective, but this does not mean that what a decision maker obtains by following the number crunching dictates of some theory will be justifiable to use in practice. it may be that results from their theory appear reasonable to the creators of it who are conditioned by a few techniques they know well, but they may have no real credibility in practice. nor is the consent of the decision maker proof of anything because he may not be sophisticated in demanding justification according to more rigid standards of knowledge and practice. nor is it proof that the technique is right if the decision outcome worked out successfully one time or even a few times. the ahp is a psychophysical theory that finds some of its validations in measurement itself. here are two examples and there are many others that would fill a book. some are with single matrices, some with hierarchies and some even with networks (whitaker 2007b). for brevity and to give the reader an idea of how it is done, we illustrate with two simple examples here. ijahp essay: saaty, vargas, whitaker/addressing criticisms of the ahp international journal of the 131 vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process an audience of about 30 people, using the ahp 1-9 scale with reciprocal values and coming to consensus on each judgment (instead of the geometric mean which is the proven way to use to combine judgments in the ahp), provided judgments from their general knowledge and experience about what people drink to estimate the dominance of the consumption of drinks in the united states (which drink listed on the left of table 1 is consumed more in the us over a drink listed at the top of table 1, and how much more than that drink?). the derived vector of relative consumption and the actual vector, obtained by normalizing the consumption given in official statistical data sources, are at the bottom of table 1. table 1 which drink is consumed more in the u.s.? those who did the example could not possibly have known the answers in advance but the results confirmed the accuracy of their judgments. recently the second author applied judgments to estimate the relative size of the populations of seven cities in spain. the judgments and the close outcome when the priorities are compared to the relative actual values are shown in table 2. table 2 which city has the larger population? madrid barcelona valencia sevilla zaragoza malaga bilbao priorities actual in millions relative actual madrid 1 2 5 5 6 6 9 0.429 3.400.000 0.434 barcelona 1/2 1 2 2 3 3 4 0.197 1.500.000 0.192 valencia 1/5 1/2 1 1 1 1.5 2 0.091 740.000 0.095 sevilla 1/5 1/2 1 1 1 1 2 0.086 700.000 0.089 zaragoza 1/6 1/3 1 1 1 1 2 0.079 600.000 0.077 malaga 1/6 1/3 1/1.5 1 1 1 1 0.068 528.000 0.067 bilbao 1/9 1/4 1/2 1/2 1/2 1 1 0.048 358.000 0.046 coffee wine tea beer sodas milk water drink consumption in the u.s. coffee wine tea beer sodas milk water 1 1/9 1/5 1/2 1 1 2 9 1 2 9 9 9 9 5 1/3 1 3 4 3 9 2 1/9 1/3 1 2 1 3 1 1/9 1/4 1/2 1 1/2 2 1 1/9 1/3 1 2 1 3 1/2 1/9 1/9 1/3 1/2 1/3 1 the derived scale based on the judgments in the matrix is: coffee wine tea beer sodas milk water .177 .019 .042 .116 .190 .129 .327 with a consistency ratio of .022. the actual consumption (from statistical sources) is: .180 .010 .040 .120 .180 .140 .330 ijahp essay: saaty, vargas, whitaker/addressing criticisms of the ahp international journal of the 132 vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process we recommend that multicriteria methods put greater emphasis on validation to acquire greater credibility in practice. validation is much more difficult when all judgments depend on feelings alone without memory from the senses, and when the criteria are all intangible. but there are other ways to improve the credibility of the outcome that have been discussed in the literature (whitaker 2007b). references bana e costa, c.a., j.-m. d. corte and j.-c. vansnick (2003). macbeth. working paper 03.56. london, london school of economics. bana e costa, c.a. and j.-c. vansnick (2008). a critical analysis of the eigenvalue method used to derive priorities in the ahp. european journal of operational research 187(3) 1422-1428. barr, n. (2004). economics of the welfare state. new york, oxford university press (usa). barzilai, j. (1997). deriving weights from pairwise comparison matrices. journal of the operational research society 48(12) 1226-1232. barzilai, j. (1998). on the decomposition of value functions. operations research letters 22 159-170. barzilai, j. and f. a. lootsma (1997). power relations and group aggregation in the multiplicative ahp and smart. journal of multi-criteria decision analysis 6 155-165. belton, v. & a.e. gear (1983). on a short-coming of saaty's method of analytic hierarchies. omega 11(3) 228-230. blumenthal, a. (1977). the process of cognition. englewood cliffs, new jersey, prentice-hall, inc. boudreaux, d. j. (2008). the freeman: ideas on liberty foundation for economic education 58(3). corbin, r. & a.a.j. marley (1974). random utility models with equality: an apparent, but not actual, generalization of random utility models. journal of mathematical psychology 11 274-293. dehaene, s. (1997). the number sense, oxford university press. dyer, j.s. (1990). remarks on the analytic hierarchy process. management science 36(3) 249-258. dyer, j.s. & h.v. ravinder (1983). irrelevant alternatives and the analytic hierarchy process. working paper, the university of texas at austin. ijahp essay: saaty, vargas, whitaker/addressing criticisms of the ahp international journal of the 133 vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process finan a& hurley (2002). the analytic hierarchy process: can wash criteria be ignored? computers and operations research 29(8) 1025-1030. harker, p.t. & l.g. vargas (1990). reply to "remarks on the analytic hierarchy process" by j.s. dyer. management science 36(3) 269-273. holder, r.d. (1990). some comments on the analytic hierarchy process. journal of the operational research society 41(11) 1073-1076. hurley, w.j. (2002). letters to the editor: strategic risk assessment. canadian military journal summer 3-4. kamenetzky, r.d. (1982). the relationship between the analytic hierarchy process and the additive value function. decision sciences 13(4) 702-713. keeney, r.l. & h. raiffa (1976). decisions with multiple objectives: preferences and value tradeoffs. new york: john wiley & sons. lootsma, f.a. (1993). scale sensitivity in the multiplicative ahp and smart. journal of multi-criteria decision analysis 2, 87-110. ma, d. &x. zheng (1991). 9/9-9/1 scale method of ahp. proceedings of the 2nd int.'l symposium on the ahp, pittsburgh, pa, university of pittsburgh. 1, 197-202. millet, i. & t.l. saaty (2000). on the relativity of relative measures – accommodating both rank preservation and rank reversals in the ahp. european journal of operational research 121, 205-212. perez, j., j.l. jimeno & e. mokotoff (2006). another potential shortcoming of ahp. top 14(1) 99-111. saaty, t.l. (1990). an exposition of the ahp in reply to the paper: remarks on the analytic hierarchy process. management science 36(3) 259-268. saaty, t.l. (1991a). rank and the controversy about the axioms of utility theory ─ a comparison of ahp and maut. the 2nd international symposium on the analytic hierarchy process, pittsburgh, pa. saaty, t.l. (1991b). response to holder's comments on the analytic hierarchy process. the journal of the operational research society 42(10) 909-914. saaty, t. l. & l.g. vargas (1984). the legitimacy of rank reversal. omega 12(5) 513516. saaty, t.l. & l.g. vargas (1993). experiments on rank preservation and reversal in relative measurement. mathematical and computer modeling 17(4/5) 13-18. saaty, t.l. & l.g. vargas (2006). the analytic hierarchy process: wash criteria should not be ignored. int'l j'l of management and decision making 7(2/3) 180-188. ijahp essay: saaty, vargas, whitaker/addressing criticisms of the ahp international journal of the 134 vol. 1 issue 1 2009 issn 1936-6744 analytic hierarchy process saaty, t.l., l.g. vargas & r. e. wendell (1983). assessing attribute weights by ratios. omega 11(1) 9-13. salo, a.a. & r.p. hamalainen (1997). on the measurement of preferences in the analytic hierarchy process. journal of multi-criteria decision analysis 6(6) 309-319. salomon, v.a o. (2008). an example of the unreliability of macbeth applications. 4th international conference on production research, june, sao paulo, brazil. sarin, r.k. (1982). strength of preference and risky choice. operations research 30(5) 982-997. sen, a. (1993). markets and freedom: achievements and limitations of the market mechanism in promoting individual freedoms. oxford economic papers 45(4) 519-541. schenkerman, s. (1997). inducement of nonexistent order by the analytic hierarchy process. decision sciences 28 (2) 475-482. tversky, a., p. slovic & d. kahneman (1990). the causes of preference reversal. the american economic review 80(1) 204-215. vargas, l.g. (1997). why the multiplicative ahp is invalid: a practical counterexample. journal of multi-criteria decision analysis 6, 169-170. watson, s.r. & a.n.s. freeling (1982). assessing attribute weights. omega 10(6) 582583. whitaker, r. (2004). why barzilai’s criticisms of the ahp are incorrect. int'l meeting of the multi-criteria decision making society, whistler, canada. whitaker, r. (2007a). criticisms of the analytic hierarchy process: why they often make no sense. mathematical and computer modeling 46(7/8) 948-961. whitaker, r. (2007b). validation examples of the analytic hierarchy process and analytic network process. mathematical and computer modelling 46(7/8) 840-859. ijahp article: agredo, mu, vella/ evaluation of customer relationship management (crm) for small service businesses international journal of the analytic hierarchy process 109 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.473 evaluation of customer relationship management (crm) systems for small service businesses shannon agredo carlow university sagredo@live.carlow.edu enrique mu carlow university emu@carlow.edu catherine vella carlow university cavella@live.carlow.edu abstract although several customer relationship management (crm) evaluation frameworks have been developed in the past years, they are usually complex, oriented toward large organizations and from the firm’s perspective only. this study proposes a crm evaluation model which is simple for a small business to use and takes into account three different perspectives: the business, the staff and the customer. the need for this model arose when one of the co-authors had to evaluate and select a crm system suitable for a small business. also, although the evaluation will be done within the context of a specific business, this model and the multiple perspective approach can be used with minimum modifications as an evaluation and reference tool for other crm decision-makers in small service businesses. https://doi.org/10.13033/ijahp.v9i1.473 keywords: customer relationship management; crm evaluation; ahp; crm perspectives 1. introduction there are many small businesses that rely heavily on a customer relationship management (crm) system to perform successfully. while working for a small business, one of the authors questioned whether or not this specific organization should keep their current crm system or invest in a new one and if so, which one. while there are many definitions for a crm system, for the purpose of this study a crm system is defined as a technological tool that allows “a strategic approach that is concerned with creating improved shareholder value through the development of appropriate relationships with key customers and customer segments” (lee et al., 2014; friedrich, et al., 2012; baran and galk, 2017). because small businesses do not have information specialists, an intuitive and simple evaluation approach was needed. for this reason, it became clear that https://doi.org/10.13033/ijahp.v9i1.473 mailto:sagredo@live.carlow.edu mailto:emu@carlow.edu mailto:cavella@live.carlow.edu https://doi.org/10.13033/ijahp.v9i1.473 ijahp article: agredo, mu, vella/ evaluation of customer relationship management (crm) for small service businesses international journal of the analytic hierarchy process 110 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.473 using the analytic hierarchy process (ahp) approach would be the most efficient way to help the organization make this decision. the reason for this is that ahp is very intuitive for practitioners and has been used in multiple selection tasks such as the selection of the best extraction technology for medicinal plants and the selection of a suitable business model for mobile commerce (jain & bakul, 2013; ali, 2015). while doing research looking for a suitable crm evaluation model, we found that there were many articles previously published using ahp to evaluate crm systems including open source crm systems. however, we realized that most, if not all, of these articles were focused only on complex crm systems for large organizations. for this reason, this study proposes an original selection model that allows small businesses to benefit from its use. the objective of this study was to identify the key factors to take into consideration when performing a crm evaluation for a small business. we decided that while doing the analysis, we would come up with a model that can be useful as a quick evaluation and reference tool for other small service business crm decision-makers. 2. literature review we first performed a systematic search on the web of a science database. keywords such as “crm systems”, “crm small business”, “crm family business”, “crm familyowned company” and “crm selection and crm evaluation” were used and produced 451 results. we reviewed those results and initially came up with a total of 14 studies relevant to this study as shown in table 1. 1 the relevant studies were divided into four different categories and are discussed below. dated literature colombo and francalanci (2004) proposed selecting crm packages based on architectural, functional and cost requirements using an ahp model for that purpose. similarly, min (1992) proposed an ahp model to select crm packages. however, we decided that literature that was more than ten years old was too dated for our study since crm has evolved very fast during the last ten years. therefore, these articles were reviewed, but not directly used in our study. non-peer-reviewed literature we also decided that our model should be based on academic findings rather than unverified claims in the discipline. for this purpose, we decided to read practitioner’s material for general knowledge purposes, but did not use it directly in the construction of our model. jones (2013), goldenberg (2008) and kostojohn (2011) fell into this category of practitioner-oriented reference books; these were read but not directly used in our model due to their lack of peer-review academic validation. similarly, ghavani and 1 the number of relevant studies increased to 17 once we added, based on a reviewer’s suggestion, studies published from jan 2016-april 2017 as shown in table 1. https://doi.org/10.13033/ijahp.v9i1.473 ijahp article: agredo, mu, vella/ evaluation of customer relationship management (crm) for small service businesses international journal of the analytic hierarchy process 111 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.473 alireza (2006) was not used for our model, given that this was a master’s thesis. one particular mention corresponds to friedrich and breitner (2016) which corresponded to a chapter in a book written mainly in german and describes a process model not suitable for a small firm. similarly, baran and galka (2017) describe crm marketing functions that although useful for practitioners do not directly help small businesses make crm selection decisions. non-selection models there was also literature that discussed crm systems from the point of view of process models; that is models that describe crm implementation as a process consisting of several stages in which crm selection is only the first stage in the process. chang et al. (2015), pollard et al. (2006) and friedrich et al. (2012) fell into this category. therefore, their usefulness was only marginal. one exception is rigby et al. (2003) which provided a good introduction of why choosing the right crm had strategic importance for winning customer loyalty. this was used for the motivation part and to provide context to our crm evaluation. https://doi.org/10.13033/ijahp.v9i1.473 ijahp article: agredo, mu, vella/ evaluation of customer relationship management (crm) for small service businesses international journal of the analytic hierarchy process 112 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.473 table 1 list of relevant references for the study item# reference main findings 1 min, h. (1992) it describes how to decide the appropriate crm software for your organization. this article was too dated for our use as it was written in 1992. 2 rigby, et al. (2003) this article describes how to decide what crm software should be used. good reference about the importance of crm systems. 3 colombo and francalanci (2004) this article describes the development of an ahp model to select a crm system among multiple crm packages. this article is too old for our purposes; it is more than 10 years old and crms have evolved ever since. 4 ghavami and olyaei (2006). discusses crm impact on customer retention. this result was not incorporated for our research purposes as it was not peer-reviewed. 5 pollard, et al. (2006) it proposes a process model in which crm implementation moves from stage to stage. not suitable for selection but for implementation. 6 jones, p. (2013) a guide to selecting the best crm solution and it is useful as a reference but it is not peer-reviewed and therefore, not used in our study. 7 goldenberg (2008) this is another good article identifying crms and keeping customer relationships. however, it is a practitioner piece, not peer-reviewed. 8 jadhav and sonar (2009) describes how to decide what crm software should be used using ahp process. good reference for methodology. 9 friedrich, et al. (2011) this article provides a thorough description of how experts would use ahp to choose the best crm system for large companies. while too complex for our purposes, we were able to use the major parts to develop our model. 10 kostojohn (2011) generic discussion of crm systems. not peer-reviewed and therefore, not used in our study. 11 friedrich, et al. (2012) this article was extremely complex and process oriented and very specialized for large companies in the automotive industry. 12 lee, et al. (2014) this article uses ahp to select an open access crm package. it also provides information for organizations to choose a crm system for its functionality aspect as well as from the perspective of the organization. this article was also key for the current study. 13 chang, et al. (2015) this article proposes a model to aid in the selection of crm products and the evaluation of crm vendors. however, the focus is more on the implementation than on the selection. 14 friedrich and breitner. (2016) this is a chapter in a book which is mostly in german. also, a process model is not suitable for a small business firm. 15 keramati and shapari (2016). describes the criteria for choosing a crm system, however the use of three combined methodologies is too complex and not suitable for a small business. 16 pedron, et al. (2016). this article does describe possible criteria businesses may take into consideration when choosing a crm system, however the main focus was on how to implement the crm system. 17 baran and galka (2017). this book provides a detailed description of a crm system as it’s need in any business, how to use it for marketing, as well as the challenges it can cause. this provided the study with a foundation of the qualities of crm systems and their functions. https://doi.org/10.13033/ijahp.v9i1.473 ijahp article: agredo, mu, vella/ evaluation of customer relationship management (crm) for small service businesses international journal of the analytic hierarchy process 113 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.473 selection models finally, the articles that were more directly relevant to the development of our model were jadhav et al. (2009), who performed an exhaustive review of the crm evaluation literature including ahp techniques, and friedrich et al. (2011) and lee et al. (2014), who provided very comprehensive and large scale models but which were structured enough to allow for fragmentation and simplification for our proposed model for small businesses. friedrich et al. (2011), in particular, provided a thorough description of how experts would use ahp to choose the best crm system for large companies. we were able to review different types of criteria and sub-criteria used for crm selection. they propose three key criteria: quality, cost and functionality. quality involves several subcriteria such as data integration, modifiability/maintainability, performance, popularity, portability, reliability, resources, security, timeliness, training/support and usability. cost is composed of six dimensions or sub-criteria: maintenance, preparation/installation, resources, systems costs, training/support and upgrade. functionality consists of account management, call center, campaign management, contact management, customer service, field service, internet, lead/opportunity management, relationship management, reporting and sales management. the second key article was lee et al. (2014). this article provided information for organizations to choose a crm system for its functionality aspect as well as from the organization’s perspective. we were able to reference the complex hierarchy shown in figure 1 to help further our research. figure 1. the hierarchical structure for open source crm software selection source: lee et al. (2014) https://doi.org/10.13033/ijahp.v9i1.473 ijahp article: agredo, mu, vella/ evaluation of customer relationship management (crm) for small service businesses international journal of the analytic hierarchy process 114 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.473 3. methodology first, decision factors for crm evaluation were identified based on information from extant literature (table1), and then requirements from organizational experts were gathered. after we reviewed the literature and gathered the requirements from the client organization, we came to the conclusion that ahp stakeholder analysis needed to be done from three different perspectives: the business, the user (staff), and the client. from each of these perspectives, we identified the most important criteria, developed ratings decision models for each, and finally integrated the different perspectives by multiplying the priorities of the alternatives. this last step is common practice in the integration of multiple perspectives in which each decision model is different but the alternatives are common (saaty & mu, 1997; mu & pereyra-rojas, 2016). 4. developing the decision models an ahp ratings model analysis was done for each of the perspectives prior to their integrations as will be shown next. 4.1 business perspective the first decision hierarchy was created from the business perspective as this is a very important factor when evaluating a new crm system for a small business. for this purpose, a business professor with entrepreneurship, practical, and academic experience assisted with the business model definitions and the weighting of elements. 4.1.1 business perspective definitions friedrich et al. (2011) helped identify the three most important criteria that define the business perspective. these three criteria were quality, functionality, and cost. the business perspective model elements are shown in table 2. https://doi.org/10.13033/ijahp.v9i1.473 ijahp article: agredo, mu, vella/ evaluation of customer relationship management (crm) for small service businesses international journal of the analytic hierarchy process 115 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.473 table 2 business perspective criteria source: friedrich et al. (2011) criteria key elements quality usabilityhow ‘user friendly’ is the application? business people need to be able to extract reports with ease while ensuring their staff and customers can use it with a minimum number of mistakes. data integrationease of sharing data between technical and business processes. even small businesses have multiple systems (e.g. accounting) that the crm will need to share and integrate data with. functionality contact managementtracking all information and communication activities linked to contacts. includes ability to keep track and contact customers through multiple media channels such as linkedin, facebook, etc. relationship managementcontinuous level of engagement between customers and clients that allows initiatives such as birthday gifts and loyalty initiatives cost maintenance costs – refers to the costs to keep the system running. includes preventive maintenance, repairs and associated costs to keep the system running. system costs – this is the actual cost of the application itself and may consist of monthly payments for the base system and additional costs for upgrades. preparation and installation costs – the business may need to incur in infrastructure upgrades (e.g. new computers) to satisfy the application technical requirements. quality criterion the quality criterion involves a strict and consistent commitment to certain standards that achieve uniformity of a product in order to satisfy specific customer or user requirements. it consists of two sub-criteria: usability and data integration. friedrich et al. (2011) defines usability as a way to improve customer satisfaction by ease of use. according to the expert’s opinion, if the system is too difficult to manage, then it raises the chance of mistakes occurring, thus causing customer satisfaction to decrease. data integration is the ease of sharing data between technical and business processes. this is further defined by the expert as being able to collect data (usually client information) and being able to view and use this data for daily business procedures. functionality criterion the functionality criterion is the range of operations the application can perform to help with the overall business performance and consists of two sub-criteria: contact and relationship management. contact management is the ability of the application to track information and communication activities between the small business and clients. relationship management allows the application to uphold a continuous level of engagement between businesses and clients. https://doi.org/10.13033/ijahp.v9i1.473 ijahp article: agredo, mu, vella/ evaluation of customer relationship management (crm) for small service businesses international journal of the analytic hierarchy process 116 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.473 cost criterion the cost criterion is the cost to the small business to purchase, install and maintain the application throughout its use. three sub-criteria have been identified as important for our study: maintenance, system and preparation/installation costs. maintenance cost is the cost for the small business to preserve the full functionality of the application. systems cost is the cost of the application itself. according to the expert’s opinion, many crm systems allow businesses to pay monthly, or pay one large sum for the system and then pay extra for any upgrades that might become available later on. preparation and installation costs include the cost to the small business to implement the actual application. the expert states that some small businesses will also have to purchase software such as computers, etc. to be able to host the new application in addition to the system itself. there also might be the purchase of payment transaction equipment to consider as well. 4.1.2 crm evaluation model from business perspective the hierarchical decision model for the business perspective is shown in figure 2. the decision model has three levels since each criterion has its own sub-criteria. figure 2. business perspective decision model https://doi.org/10.13033/ijahp.v9i1.473 ijahp article: agredo, mu, vella/ evaluation of customer relationship management (crm) for small service businesses international journal of the analytic hierarchy process 117 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.473 4.2 user (staff) perspective unlike a corporation, a small business does not have the funds to hire a specialized crm staff to manage a complex crm system; therefore, we decided that the user or staff perspective should also be considered when choosing a crm system. for this purpose, a staff member currently using a crm system participated in development of the business model using this perspective and the weighting of its elements. this staff member handles daily issues with the current crm system, including a lack of system organization, which makes it difficult to use particularly in accessing customer information. this overall lack of performance creates scheduling difficulties that waste time resulting in staff member frustrations and a potential loss in revenue. for these reasons, the participation of this staffer participation was key to address this model perspective. 4.2.1 user (staff) perspective definitions again, based on the extant literature and a staff expert opinion we identified the three most important criteria for the user (staff) perspective hierarchy (friedrich et al. 2011). these three criteria are: usability, performance, and accessibility. their definitions (key elements) are shown in table 3. table 3 user (staff) perspective criteria with key elements and weight source: friedrich et al. (2011) and staff crm expert opinion (2016) criteria key elements usability (friedrich et al., 2011; lee, 2014) how ‘user friendly’ is the application? staff must be able to use the system with ease and minimum of errors. performance (friedrich et al., 2011) schedulingability to easily schedule appointments time savingreducing the amount of time needed to complete a task accessibility (staff expert opinion, 2016) the quality of requested content being available when needed and using multiple platforms (e.g. ipad) usability criterion as previously defined, the usability criterion is the ease with which one can learn and successfully use the application. as it is too costly for a small business to hire specialized crm staff, their own staff must be able to utilize the system from day to day. performance criterion performance criterion concerns how well a system can process the functions required of it. according to friedrich et al. (2011), if the application does not perform its required functions external consulting may be needed causing costs that are higher than budgeted. two performance functions (sub-criteria) are important: scheduling and time saving. scheduling is how well the application can schedule appointments. this is specifically for small businesses that rely on appointments for their clientele to be successful. according to the expert’s opinion, when a client experiences scheduling problems, it https://doi.org/10.13033/ijahp.v9i1.473 ijahp article: agredo, mu, vella/ evaluation of customer relationship management (crm) for small service businesses international journal of the analytic hierarchy process 118 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.473 causes customer satisfaction to decrease, which potentially causes loss of revenue. time saving is how well the application can reduce the time needed to complete a task. similar to the scheduling sub-criteria, if too much time is wasted in completing a task, it can also cause clients to become unsatisfied. also, the more time focused on a single activity limits how many clients can be managed at a time. this is another potential loss in revenue. accessibility criterion this criterion concerns how well the application can make requested content available when needed. according to expert opinion, it is crucial for client information to be not only easily accessible, but that this information is pertinent to the user’s position. this, again, allows for overall customer satisfaction. 4.2.2 evaluation model from user (staff) perspective figure 3 depicts the hierarchical decision model developed for the user (staff) perspective. only the performance criterion has two sub-criteria: scheduling and time saving. figure 3. user (staff) perspective decision model 4.3 customer perspective because a small business’s success rate depends heavily on the satisfaction of their customers, the effect of a crm system on these customers is very important. for this perspective, we realized that we did not really need a customer from the specific target business but rather a customer that used the same services provided by the client. based on this we selected as representative of the customer stakeholders, a person who was a heavy user of these services to assist in refining the model definitions and weighting. 4.3.1 consumer perspective definitions once more, a ratings model was used to identify two criteria for the customer perspective. these criteria, customer scheduling and customer engagement, are described in table 4. https://doi.org/10.13033/ijahp.v9i1.473 ijahp article: agredo, mu, vella/ evaluation of customer relationship management (crm) for small service businesses international journal of the analytic hierarchy process 119 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.473 table 4 customer perspective with key elements and weight source: friedrich et al. (2011) and customer representative opinion (2016) criteria key elements customer scheduling online schedulingallowing clients to schedule their appointments online customer engagement messagingability to efficiently and effectively inform clients of vital information customer scheduling criterion this criterion measures whether or not the application allows clients to schedule their own appointments. according to the customer representative’s opinion, clients are too busy during their daily lives and need the option of scheduling their own appointments when it is convenient for them. the key functionality is called the online scheduling subcriterion. this sub-criterion describes whether the system allows clients to schedule their appointments online or not. as mentioned before, clients want the opportunity to plan their appointments themselves with the convenience of online scheduling. customer engagement criterion the customer engagement criterion describes how well the application provides opportunities for the small business to communicate with its clients. this means allowing the business to inform the client of new products, sales, and even new services. in this case, the identified key functionality is the messaging sub-criterion, which is the ability for the system to efficiently and effectively inform clients of vital information. this information consists of anything that might be of importance to the client like a new product launch or that the business will be unexpectedly closed. there are certain situations where the client needs to be informed immediately of something that affects them and their appointment, and it is crucial to the success of the business that these clients can be informed as promptly as possible. 4.3.2 customer perspective evaluation model the hierarchical decision model for the customer perspective is shown in figure 4. both the scheduling criterion and the engagement criterion have multiple sub-criteria, based on the extant literature and discussion with the client representative. https://doi.org/10.13033/ijahp.v9i1.473 ijahp article: agredo, mu, vella/ evaluation of customer relationship management (crm) for small service businesses international journal of the analytic hierarchy process 120 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.473 figure 4. customer perspective decision model 4. 4 alternatives to perform the analysis on these three factors, we evaluated three possible crm systems: millennium, mikal and salon iris. the mikal system was chosen first because it is the current crm system the expert staff member is using, albeit an older version. millennium and salon iris were chosen next because each of them encompasses areas that mikal had not yet developed when this study was performed. millennium’s focus is on new technology for more developed features, while salon iris focuses on the latest accessibility trends. each of these are described in more detail below, however with these three crm systems, we have created a model that potentially embraces each crm type. 4.4.1 millennium this crm system was created in 1987 and is used in small businesses in over 36 countries. one advantage is that it can be cloud based or privately hosted. it can also be accessible on any internet device. millennium’s website describes how their focus on the latest technology provides it with an advantage for the business, the staff members who will use it, and the customers because it allows the system to be accessible at all times (milleniumit, 2017). the ease of access minimizes the difficulty in making any lastminute scheduling changes as well providing the greatest customer service. 4.4.2 salon iris salon iris is a crm system developed in 1999 in wixom, michigan (salon iris, 2017). like millennium, it allows users to access it on any internet device. however, a significant advantage is that it has its own personalized app for android or iphone allowing easier access for any of its users. with this ability to be accessible on an app comes the advantage of being extremely user-friendly. the layout of the salon iris app is simple allowing users to take advantage of its features without typical layout frustrations. staff members can use basic data to access any customer information particularly their appointment schedule. this can raise customer satisfaction as it allows immediate communication with ease. https://doi.org/10.13033/ijahp.v9i1.473 ijahp article: agredo, mu, vella/ evaluation of customer relationship management (crm) for small service businesses international journal of the analytic hierarchy process 121 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.473 4.4.3 mikal founded in 1982, mikal is a crm system that is slowly developing with innovations using new technology (mikal, 2017). its latest feature is a system that will look over the appointment schedule, collect data on any openings, and then reach out to specific clients to let them know of these openings. however, staff experience with the system provided the knowledge that mikal is behind with their own technology causing their system to be extremely hard to use. it lacks an organized layout and ease of access, which causes many mistakes to occur. like a domino effect, these mistakes thus cause customers to be unhappy with the small business as their appointments have a higher chance of being scheduled incorrectly. 5. analysis and results using the ahp methodology and the newly released super decisions software v3 to perform the calculations, each stakeholder representative compared pairwise the criteria of the corresponding perspective (ensuring inconsistency was less than 0.1 in all cases) to obtain the weights corresponding to the criteria/sub-criteria corresponding to each of the perspective hierarchies (super decisions, 2017). next, an ahp ratings model was used for each of the perspectives where a single scale ranging from 1 to 5 (1-deficient, 2 below average, 3 – average, 4 – above average and 5-excellent) was used to evaluate the alternatives with respect to each and all criteria. each scale item had an associated value (0.2, 0.4, 0.6, 0.8 and 1.0 respectively) which was used for the quantitative assessment of the alternatives as will be shown in tables 5, 6 and 7. again, the representative of the business perspective performed the evaluation of alternatives from this perspective, the staff representative did the same for the staff (user) perspective and the customer representative 5.1. business perspective the hierarchy and results for the business perspective are shown in table 5. the crm system that had the highest preference from the business perspective was salon iris with 0.401, followed by millennium with 0.385, and trailing far behind was mikal with 0.215. these results indicate that salon iris is, although slightly, the best choice due to its userfriendly layout, low cost of installation and management, and its ability to provide a steady connection between the business and its clients. more clearly, the results also indicate that mikal is by far the worst choice as it is a higher cost for a poorly organized system. this can be fully appreciated in figure 5 where the priorities are shown in relation to the maximum (ideal) priority, in this case iris. millenium follows at 95.9% of the ideal alternative and mikal accrues only 53.5% of the ideal priority. mikal is the clear loser from a business perspective. figure 5. ideal business priorities for the crm alternatives https://doi.org/10.13033/ijahp.v9i1.473 ijahp article: agredo, mu, vella/ evaluation of customer relationship management (crm) for small service businesses international journal of the analytic hierarchy process 122 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.473 sensitivity analysis for the business perspective appendix a shows a graphic and dynamic sensitivity analysis performed using super decisions v3 capabilities. as can be seen in figures (a1) and (a2) in this appendix, as the importance of the quality criterion increases, millenium starts catching up with iris. more specifically, if quality gets more than 65% of the overall importance then millenium becomes a better crm choice although not by much. this is due to the fact that millenium is a better choice with respect to each of the quality sub-criteria (1.1 usability, 1.2. data integration and 1.3 performance) as shown in table 5. on the other hand, with respect to cost, iris is always the best alternative as seen in figures (a3) and (a4). finally, if functionality gets above 24.5% of the overall criterion importance, then iris becomes the best option as shown in figures (a5) and (a6). in summary, our sensitivity analysis confirms that salon iris is indeed the best option in most of the scenarios, and when it loses the lead with respect to millenium, it is not by much. furthermore, this sensitivity analysis shows that mikal is the worst alternative in all cases. 5.2. staff (user) perspective the hierarchy and results for the business perspective are shown in table 6. the highest preference from the staff (user) perspective was millennium with 0.41 followed closely by salon iris with 0.39. table 6 results show that millennium is the best choice as it provides the staff with a crm system that is easy to use allowing scheduling to be simple and the ease of access to customer information to be superior. these results also indicate that mikal is the least optimal choice as it greatly lacks in all areas of the user (staff) criterion. https://doi.org/10.13033/ijahp.v9i1.473 ijahp article: agredo, vella, mu / evaluation of customer relationship management (crm) systems for small service businesses international journal of the analytic hierarchy process 123 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13022/ijahp2017.v9.is1.473 ijahp article: agredo, mu, vella/ evaluation of customer relationship management (crm) for small service businesses international journal of the analytic hierarchy process 124 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.473 while the business perspective showed that iris had a slight advantage respect to millenium, our results from the user (staff) perspective suggest the opposite; that is, millenium now has a slight advantage, approximately 5%, over iris as shown in figure 6. still, mikal is no competition for the two leading crm systems. figure 6. ideal user (staff) priorities for the crm alternatives sensitivity analysis for the user (staff) perspective appendix b shows the graphic and dynamic sensitivity analysis for this perspective. figures b1-b2 show that as the importance of performance increases, millenium becomes an even more preferable alternative. similarly, figures b5-b6 show that increasing the importance of accessibility means that millenium also increases its preference. the only situation in which salon iris may tie with millenium is if the performance criterion gets 100% of all the criterion importance, which constitutes a very unlikely scenario, as shown in figures b3-b4. in summary, the results of the sensitivity analysis show that millenium is a robust decision from the user (staff) perspective. so far, we have found that salon iris is a robust decision from a business perspective while millenium is a robust decision from a user (staff) perspective although the difference is small (about 5%) in each case. our next step is to evaluate the alternatives from the customer’s perspective. 5.3 customer (client) perspective the hierarchy and results for the business perspective are shown in table 7. the crm system that had the highest preference from the client perspective was millennium with 0.40 followed by salon iris with 0.33. millennium is the winner over salon iris because it allows clients to easily schedule online, but also because it provides a way to constantly engage clients with important communication from the business. once again, mikal is shown to be the worst choice for a small business as it lacks in client engagement. while the two previous perspectives showed no more than 5% difference between salon iris (winner in the business perspective) and millenium (winner in the staff perspective); this time, millenium is a better choice, from a customer perspective, with a 15% lead over salon iris as shown in figure 7. https://doi.org/10.13033/ijahp.v9i1.473 ijahp article: agredo, mu, vella/ evaluation of customer relationship management (crm) for small service businesses international journal of the analytic hierarchy process 125 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.473 figure 7. ideal customer (client) priorities for the crm alternatives sensitivity analysis for the customer (client) perspective appendix c shows the graphic and dynamic sensitivity analysis for this perspective. figures c1-c2 show no difference in the preferences of the alternative for the case of variation of the importance for the scheduling criterion. similarly figures c3-c4 show no difference in the alternative preferences for the case of changing the priorities of the engagement criterion. a look at table 6 suggests that millenium and salon iris differ in terms of the scheduling sub-criteria. millenium is better for scheduling reminders while iris is better for online scheduling. if scheduling reminders was given even more weight, millenium would be an even better option. in summary, the sensitivity results indicate that millenium is clearly the best crm alternative from the point of view of the customer. now that we have completed the evaluation of the crm systems from the different perspectives we need to proceed to integrate them. 5.4 strategic integration of the perspectives standard practices of perspective integration of alternatives consist in a multiplicative aggregation of the priorities of the alternatives in each perspective (mu & pereyra-rojas, 2017). however, for this study, we decided to consider the situation where not all the perspectives were given the same importance 2 . for this purpose, we aggregated the local priorities of the alternatives (“local” with respect to each perspective) using an ahp model to strategically address the crm systems in terms of the different weights that could be assigned to each perspective as shown in figure 5. we call this a strategic integration of the perspectives to follow on the strategic criteria proposed by saaty and ozdemir (2005). the proposed model for the integration of the three perspectives is shown in figure 8. rather than performing judgment comparison, the local priorities of the alternatives with respect to each perspective (tables 4, 5 and 6) were entered into the model. this initial analysis assumed that all three perspectives (business, staff and customer) were equally important as shown in the priority results in figure 9. also, the results were summarized in table 8 for the reader’s convenience. 2 this makes sense since in a small business, the business (owner) perspective usually has a very high importance with respect to the interests of the customers and staff. https://doi.org/10.13033/ijahp.v9i1.473 ijahp article: agredo, mu, vella/ evaluation of customer relationship management (crm) for small service businesses international journal of the analytic hierarchy process 126 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.473 figure 8. model for the strategic integration of perspectives figure 9. results for the initial scenario of the strategic integration of perspectives table 8 perspective integration assuming equal importance business (0.333) user/staff (0.333) customer (0.333) priorities millennium 0.385 0.417 0.400 0.401 mikal 0.215 0.185 0.260 0.220 salon iris 0.401 0.398 0.339 0.379 5.5 sensitivity analysis of the perspective integration results the results from table 8 assume that all the perspectives are equally important. however, an argument can easily be made that these perspectives cannot be equally weighted. in particular, in the case of small service businesses, the business perspective is usually also the owner’s perspective and as such carries a lot of weight. the team members representing the business, staff and customer views got together and compared the perspectives pairwise and reached an agreement on the judgments to be entered. in other words, there was no need to aggregate the judgments using the geometric mean as recommended by saaty and peniwati (2007) and the consistency issues were also addressed during the meeting. the results are shown in table 9. https://doi.org/10.13033/ijahp.v9i1.473 ijahp article: agredo, mu, vella/ evaluation of customer relationship management (crm) for small service businesses international journal of the analytic hierarchy process 127 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.473 table 9 perspective integration assuming different importance business (0.635) user/staff (0.078) customer (0.287) priorities millennium 0.385 0.417 0.400 0.392 mikal 0.215 0.185 0.260 0.226 salon iris 0.401 0.398 0.339 0.383 the comparison of the results of tables 8 and 9 show that in the case of perspectives with equal importance, millenium is a slightly better option (approx. 5%), but when making the business view more important, millenium’s advantage decreases to approximately 2% and we could consider them tied for practical purposes. the team discussions suggested that a situation in which both a business and customer perspective were equally important was worth exploring given that financial (business) success is highly correlated with customer satisfaction. therefore, we decided to explore this situation and obtained the results shown in table 10. table 10 perspective integration: same business & customer weights business (0.45) user/staff (0.10) customer (0.45) priorities millennium 0.385 0.417 0.400 0.395 mikal 0.215 0.185 0.260 0.232 salon iris 0.401 0.398 0.339 0.373 a more systematic exploration of the sensitivity analysis can be done using the graphic sensitivity capabilities of super decisions v3 as shown in figure 10. https://doi.org/10.13033/ijahp.v9i1.473 ijahp article: agredo, mu, vella/ evaluation of customer relationship management (crm) for small service businesses international journal of the analytic hierarchy process 128 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.473 (a) business (b) staff (c) customer figure 10. sensitivity analyses with respect to each of the perspectives from figures 10b and 10c, we can conclude that millenium is a robust decision with respect to the staff and customer perspective, having a strong advantage the more the customer’s perspective becomes important. with respect to the business perspective, figure 10a, millenium loses its advantage only when the business perspective gets above 70% of the overall importance (while possible, this is an unlikely scenario given that crm customer satisfaction is also important for business success). in other words, millenium constitutes the most robust decision taking into account all these sensitivity scenarios. 6. limitations given that this evaluation was done with one single evaluator for each perspective, there might be some criticism that one stakeholder expert representative for each perspective is not enough. however, a small business usually has only one expert in each stakeholder category and does not have the ability to perform large scale survey studies, which would be the ideal situation from an academic point of view. furthermore, the model as such holds for a small service business and it is expected that the weights will actually change from business to business. for example, a struggling business may give a larger weight to the business perspective than to the others. also, the software evaluation is totally subjective and based on the evaluator’s perception of the software based on the documentation and demos; however, this should not lead the reader to conclude that one software package is better than the other. the reader should perform his or her own assessment when using the proposed model. 7. conclusions this study has provided a basic model that all small businesses can use when deciding which crm system is most optimal for their business. while previous studies have provided information to choose a crm system, they have proven to be too difficult for https://doi.org/10.13033/ijahp.v9i1.473 ijahp article: agredo, mu, vella/ evaluation of customer relationship management (crm) for small service businesses international journal of the analytic hierarchy process 129 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.473 small business owners who lack crm system knowledge. not only is this study tailored to the specific needs of small businesses, it was completed in a way that provides clear and basic understanding for any reader without an extensive ahp or crm system education. unfortunately, with any decision, there comes the threat of bias. when the current crm staff member provided their small business owner with the study results stating that millennium was a better choice over mikal, the owner may be subjected to cognitive biases and decide against making a change. as shown above, a crm is important to the success of any small business because it makes engaging customers and managing marketing opportunities possible which in turn benefit the business. therefore, selecting a suitable crm is extremely important. this study can now provide a simple, easier way to make a detailed and justified decision about which crm system is best based on the views of the business, the user, and the clients. as a potential avenue for future research we think that surveying a greater number of stakeholders for each perspective could make streamlining the importance given to the different crm function criteria possible. in summary, we have learned the key factors in adopting a crm system; second, we have developed a crm evaluation framework for small service businesses that can serve as a reference and best practice reference for decision-maker practitioners interested in considering multiple crm perspectives. finally, to our knowledge, this is the first time an ahp study has been conducted using the new super decisions v3 and for that reason we felt compelled to provide some notes, based on our experience in this study, in appendix d. https://doi.org/10.13033/ijahp.v9i1.473 ijahp article: agredo, mu, vella/ evaluation of customer relationship management (crm) for small service businesses international journal of the analytic hierarchy process 130 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.473 references ali, a. (2015). an mcdm approach towards m-payment business models evaluation. international journal of the analytical hierarchy process, 7(2), 273-294. doi: http://dx.doi.org/10.13033/ijahp.v7i2.298 baran, r. j., & galkla, r. j. (2017). customer relationship management: the foundation of contemporary marketing strategy. new york: routledge. colombo, e., & francalanci, c. (2004). selecting crm packages based on architectural, functional, and cost requirements: empirical validation of a hierarchical ranking model. requirements engineering, 9(3, 186-203. doi:10.1007/s00766-003-0184-y chang, j., yen, d. c., young, d., & yuan-ku, c. (2015). critical issues in crm adoption and implementation. inderscience online. international journal of services technology and management. 3(3), 311-324. doi: http://dx.doi.org/10.1504/ijstm.2002.001633 friedrich, i., breitner, m. h., & sprenger, j. (2011). discussion and validation of a crm system selection approach with experts. amcis proceedings. friedrich, i., kosch, l., breitner, mh. (2012). a practical test of a process model for customer relationship management system selection with an automotive supplier. ecis 2012 proceedings. paper 21.
 ghavami, a., & olyaei, a. (2006). the impact of crm on customer retention. lulea university master thesis. goldenberg, b. j. (2008). crm in real time: empowering customer relationships. new jersy: information today inc. jadhav, a. s., & sonar, r. m. (2009). evaluating and selecting software packages: a review. information and software technology, 51(3), 555-563. doi:10.1016/j.infsof.2008.09.003 jain, j., & bakul, r., (2013). application of ahp tool for decision making of choice of technology for extraction of anti-cancer bioactive compounds of plant origin. international journal of the analytic hierarchy process, 5(1), 3-29. doi: http://dx.doi.org/10.13033/ijahp.v5i1.153 jones, p. (2013). a guide to selecting the best crm solution for your small business. erp cloud news. kermati, a., & shapouri, f. (2016). multidimensional appraisal of customer relationship management: integrating balanced scorecard and multi criteria decision making approaches. information systems and e-business management, 14(2), 217-251. doi:10.1007/s10257-015-0281-8 kostojohn, s. (2011). evaluating software and consultants. crm fundamentals, 65-92. https://doi.org/10.13033/ijahp.v9i1.473 http://dx.doi.org/10.13033/ijahp.v7i2.298 http://dx.doi.org/10.13033/ijahp.v5i1.153 ijahp article: agredo, mu, vella/ evaluation of customer relationship management (crm) for small service businesses international journal of the analytic hierarchy process 131 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.473 doi:10.1007/978-1-4302-3591-0_4 lee, y., tang, n., & sugumaran, v. (2014). open source crm software selection using the analytical hierarchy process. information systems management, 31(1), 2-20. doi: http://dx.doi.org/10.1080/10580530.2013.854020 milleniumit (2017). retrieved from http://www.millenniumit.com/what-we-do/systemsintegration/our-business-areas/customer-relationship-management mikal (2017). retrieved from http://www.mikal-salon-software.com min, h. (1992). selection of software: the analytic hierarchy process. international journal of physical distribution and logistics management, 22(1), 42-52 mu, e. and pereyra-rojas, m. (2017). practical decision making: an introduction to the analytic hierarchy process using super decisions. springer: n.y. pedron, c. d., picoto, w. n., dhillon, g., & caldeira, m. (2016). value-focused objectives for crm system adoption. industrial management & data systems,116(3), 526-545. doi:10.1108/imds-01-2015-0018 pollard, et al. (2006). simplified framework of crm. journal of information technology case and application research, 8(2), 24-38. http://dx.doi.org/10.1080/15228053.2006.10856087 rigby, d. k., reichheld, f., & dawson, c. (2003). winning customer loyalty is the key to winning crm strategy. ivey business journal, 519, 2. saaty, t. l. and mu, e. (1997). the peruvian hostage crisis of 1996-1997: what should the government do? socio-economic planning sciences: the international journal of public sector decision-making, 31 (3), 165-172. doi: https://doi.org/10.1016/s00380121(97)00015-3 saaty, t. l. and ozdemir, m. s. (2005). the encyclicon: a dictionary of decisions with dependence and feedback based on the analytic network process. pittsburgh, pa rws publications. saaty, t.l., & peniwati, k. (2007). group decision-making: drawing out and reconciling differences. pittsburgh, pa: rws publications. salon iris (2017). retrieved from https://www.saloniris.com super decisions (2017). retrieved from http://www.superdecisions.com https://doi.org/10.13033/ijahp.v9i1.473 http://dx.doi.org/10.1080/10580530.2013.854020 http://dx.doi.org/10.1080/15228053.2006.10856087 https://doi.org/10.1016/s0038-0121(97)00015-3 https://doi.org/10.1016/s0038-0121(97)00015-3 ijahp article: agredo, mu, vella/ evaluation of customer relationship management (crm) for small service businesses international journal of the analytic hierarchy process 132 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.473 appendices appendix a. crm sensitivity analysis: business perspective (a1) graphic sensitivity: quality (a2) dynamic sensitivity respect to quality (a3) graphic sensitivity: cost (a4) dynamic sensitivity respect to cost (a5) graphic sensitivity: functionality (a6) dynamic sensitivity respect functionality https://doi.org/10.13033/ijahp.v9i1.473 ijahp article: agredo, mu, vella/ evaluation of customer relationship management (crm) for small service businesses international journal of the analytic hierarchy process 133 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.473 appendix b. crm sensitivity analysis: user (staff) perspective (b1) – graphic sensitivity: performance (b2) – dynamic sensitivity: performance (b3) – graphic sensitivity: usability (b4) – dynamic sensitivity: usability (b5) – graphic sensitivity: accessability (b6) – dynamic sensitivity: accessability https://doi.org/10.13033/ijahp.v9i1.473 ijahp article: agredo, mu, vella/ evaluation of customer relationship management (crm) for small service businesses international journal of the analytic hierarchy process 134 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.473 appendix c. crm sensitivity analysis: customer (client) perspective (c1) – graphic sensitivity: scheduling (c2) – dynamic sensitivity: scheduling (c3) – graphic sensitivity: engagement (c4) – dynamic sensitivity: engagement https://doi.org/10.13033/ijahp.v9i1.473 ijahp article: agredo, mu, vella/ evaluation of customer relationship management (crm) for small service businesses international journal of the analytic hierarchy process 135 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.473 appendix d. notes on the use of super decisions v3. these notes do not pretend to be a comprehensive review of super decisions v3 but to provide the readers with a quick idea of what to expect when using it. first of all, when designing a model for the first time what is striking is how easy it is to use. our only instruction was the video tutorial available in super decisions (2017). however, sd v3 is easy to use because most of the options are visible (the interface does not rely on drop down menus but rather on a “what you see is what you can do” approach). for aestheticians, you will miss the artistic possibilities of drawing models in the previous version (“color is missed” said one of the authors). also, the ability to endlessly size and re-size clusters and nodes is missing, and you have to use a scroll down bar to see all the nodes within a cluster. the top bar menu which has the options “network”, “judgments” and “ratings” follow the logical progression of developing a ratings model. this is the greatest advantage of this software version. those who have developed ratings models using the current super decisions v 2.8 will certainly welcome the simplicity found in v 3.0. indeed, this was the main reason to adopt this software version for our study. figure d-1 shows the screen for the staff perspective model used in this study. figure d1. ratings model screen what makes sd v3 so convenient for ratings models is that all the instructions are visible and in a sequential order (steps 1, 2 and 3) and in a single screen, as shown in fig d-1. the authors were able to develop their ratings models with minimum instruction, just by reading the instructions on the screen. if you are going to be working with ratings models, sd v3 this is the version to use. overall, in our experience, sd v3 is “much easier” to use and it is particularly useful for ratings models (“in ratings, it is really convenient”) while it still has a “vintage flavor” as indicated by one of the co-authors of this paper. https://doi.org/10.13033/ijahp.v9i1.473 ijahp: mu/ ahp/anp community mourns the loss of thomas l. saaty international journal of the analytic hierarchy process 150 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.499 the ahp/anp community mourns the loss of thomas l. saaty sadly, we start this issue by announcing the death of the creator of the ahp/anp methodology, dr. thomas l. saaty. he was also the founder of this journal aimed at spreading ahp/anp among decision-makers. for those who met him and who were lucky to be taken under his wing, he will always remain in our hearts and minds as tom, a teacher, a mentor, a beloved friend and probably the most unique person that crossed our paths. we will miss him deeply! enrique mu ijahp, editor-in-chief ijahp article: islam/work on resolving israeli-palestinian conflict international journal of the analytic hierarchy process 326 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.523 work on resolving israeli-palestinian conflict rafikul islam department of business administration kulliyyah of economics and management sciences international islamic university malaysia jalan gombak, 53100 kuala lumpur, malaysia e-mail: rislam@iium.edu.my abstract professor thomas l. saaty is a world-renowned operations research (or) scientist. his death has created a vacuum which can hardly be filled. his absence in scholastic writing in the field of mcdm will be widely felt for a long time. the world of or will remember him not only as a mathematician, originator of ahp and anp, but also as a good human being. he has numerous contributions in the field of or, particularly mcdm. in the present article, i have highlighted his wonderful contribution in resolving the decadeslong israeli-palestinian conflict. keywords: israeli-palestinian conflict; analytic hierarchy process; thomas l. saaty 1. introduction thomas l. saaty was a born-genius, and is one of the very few individuals i admire the most. i have seen his love towards humanity. he is unique. it is amazing how a mathematician can be so well-versed in so many other areas of knowledge. i have read many of his articles, most of which are theoretical in nature as they deal with ahp methodology itself. however, he has also written numerous application papers that cover diverse areas including management, education, sociology, transportation, energy planning, human resource management, conflict resolution and the list goes on. in this article, i will briefly highlight his work on conflict resolution, especially referring to the israeli-palestinian conflict (saaty & zoffer, 2011). in 1977, professor saaty developed the analytic hierarchy process (ahp) and the paper was published in the journal of mathematical psychology (saaty, 1977). this is not a very well-known journal, at least within the management community. at the time he introduced it, he might not have anticipated how powerful the method would be in solving so many problems in such diverse areas. this is where tom’s success lies * . the quantum and magnitude of ahp applications are simply marvellous. * i prefer to refer professor thomas l. saaty as tom. in fact, to maintain decorum, i used to address him as professor saaty, but one day he asked me, “rafikul, when are we going to be friends? just call me tom!” ijahp article: islam/work on resolving israeli-palestinian conflict international journal of the analytic hierarchy process 327 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.523 tom once fondly advised me to promote ahp to politicians; he said, “it is they who need more applications of ahp”. he always wanted to see a world where everybody was living peacefully and symbiotically. we chose, “better world through better decision making” as the theme of the 12 th international symposium on the ahp held in kuala lumpur in 2013. tom could not attend the meeting due to health reasons, but as a chairman i requested that he write a paper on the theme which he promptly did. the paper is available at http://www.mim.org.my/wpcontent/uploads/2016/03/mmrsamplearticle.pdf. in our personal meetings, tom often used to say how worried he was about the ongoing conflicts in many parts of the world. once, i said to him that the root cause was the israeli-palestinian conflict, and he agreed. 2. israeli-palestinian conflict one of the most complicated and devastating conflicts that the human race has witnessed throughout history is the israeli-palestinian conflict which has raged on both sides for the last six decades. numerous people, including high profile politicians of powerful nations in the world, have conducted countless meetings/sessions to talk through the muchneeded solutions that would be acceptable to both sides. however, in practicality, these solutions were not fully acceptable to either party as many important intangible factors had not been included in the solution. no scientific method had been applied to make a rational trade-off between costs and benefits for both sides. the ahp fills the muchneeded gap in this area. the israeli–palestinian conflict is a prolonged and interminable struggle between parties committed to unyielding positions related to identity, religion and territory. further, the conflict has been compounded by great power rivalries, weapon sales, and economic and social disparities. some of the world’s best negotiators, diplomats and leaders have grappled with the resolution of this conflict. however, despite their best efforts, the current condition continues to torment all the parties (israel and palestine). where is the end? let’s have a fresh look. it is possible to look at resolutions from a purely scientific angle. the ahp approach, however, does not guarantee the resolution of the crisis; this requires proper implementation by fully committed parties. to address any kind of human conflict, intangible factors must be dealt with. the usual face-to-face discussions over the negotiation table are often marred by emotionally charged participants, and emotions often prevail over human reasoning. why have countless numbers of meeting deliberations failed to see a lasting solution? it is partly because people have not generated enough kinds of concessions, and have not traded them off properly. this is where ahp can play an important role. the ahp solution process cannot replace the formal face-to-face negotiation process; rather ahp findings can supplement the formal negotiation process. saaty & zoffer (2011) write: this initiative only sought to test the ahp methodology on a problem that previously evaded resolution… however, it was agreed that the work is exploratory in nature and intended to demonstrate how the method can be used http://www.mim.org.my/wp-content/uploads/2016/03/mmrsamplearticle.pdf http://www.mim.org.my/wp-content/uploads/2016/03/mmrsamplearticle.pdf ijahp article: islam/work on resolving israeli-palestinian conflict international journal of the analytic hierarchy process 328 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.523 over a short period of time to arrive at a process that moves the negotiation process forward (p. 13). a successful solution derived by a scientific process depends upon the implementation of the recommendation. the implementation team must recognise the value of ahp findings, especially the priorities of the concessions made by both the parties. briefly, ahp requires carrying out the following three tasks: 1. decompose the complex decision making problem into specific manageable items. one of the items should be defining the goal of the problem. in the present case, the goal is, “achieving a consensus peace accord between israelis and palestinians.” place similar items into groups, then a hierarchy should be constructed with all similar items placed in a particular level. regarding the decomposition, saaty & zoffer write (2011): logical thinking is linear and deals with issues one at a time starting with assumptions and drawing conclusions. it involves many such repetitions of cause and effect thinking but then has no way to combine the many conclusions into a single overall outcome, except by making new assumptions not included in the original considerations. decomposing the segments takes advantage of human linear thinking to focus our attention on pairwise comparisons, a much simpler way of accurately assessing interaction. even if individual judgment may be less than accurate, such errors are compensated for by the quantity of judgments that are made. in a sense, an appropriate analogy would be trying to explain exactly how the pieces of a machine work together as a system, without understanding what the relationship of each part of that machine is to another. (p. 62) referring to the israeli-palestinian conflict saaty & zoffer write (2011): in laying out the framework for conflict resolution, one needs to sort the elements into groupings or clusters that have similar influences or effects. one must also ijahp article: islam/work on resolving israeli-palestinian conflict international journal of the analytic hierarchy process 329 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.523 arrange them in some rational order to trace the outcome of these influences. one then constructs a hierarchy of several levels. (p. 8) 2. perform pairwise comparisons for the elements belonging to one specific level with respect to a common element occupying the immediate higher level in the hierarchy. the pairwise comparisons for every possible pair ensure the exactness and validity of the priorities that are generated. 3. synthesize all the “local” priorities to obtain the overall priorities of the elements belonging to the last level of the hierarchy. it is to be noted that saaty and zoffer applied both the relative and absolute measurement process of ahp for the israeli-palestinian conflict. harvard psychologist arthur blumenthal, cited in saaty & zoffer (2011), wrote: comparative judgment, which is the identification of some relation between two stimuli both present to the observer, and absolute judgment, which involves the relation between a single stimulus and some information held in short term memory about some former comparison stimuli or about some previously experienced measurement scale with which the observer rates the single stimulus. (p. 23) blumenthal’s observation is confirmed by the way people do rankings. saaty & zoffer (2011) wrote: they (people) either compare things with each other, or rate them one at a time with respect to a standard they have in mind. when things are intangible, as the issues are in the middle east conflict, they must be compared. on the other hand, the concessions or the alternatives of any decision can be rated one at a time because they can be numerous, which makes it time consuming to compare them in pairs. (p. 23) ijahp article: islam/work on resolving israeli-palestinian conflict international journal of the analytic hierarchy process 330 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.523 3. the process resolving the israeli-palestinian conflict was close to the heart of tom. he and jerry zoffer managed to secure some grants from the david berg foundation of new york and from their own university of pittsburgh to invite participants from israel as well as palestine to meetings. two meetings took place, one in june 2009 and the other in march 2010. each meeting spanned over 3 days with comprehensive discussions, deliberations and negotiations. tom decided to change the participants of half of the group in the second meeting to see further fresh ideas on the conflict. these meetings took place before the saaty & zoffer (2011) paper was written. they went on to hold four more meetings after that, with the last being held in september 2017 after tom passed away. the detailed work that came from these meetings that defined the possible trade-offs were so valuable that their lists and marked-up maps were requested by the u.s. state department. it would be remiss not to mention the valuable work of professor luis vargas of the university of pittsburgh who facilitated all the meetings, built the ahp models, built the trade-off model used to lead the participants toward a compromised solution that they considered a win-winequitable solution. he was a co-author with saaty and zoffer on subsequent papers. saaty & zoffer (2011) stated the objectives of this special face-to-face meeting between the israelis and palestinians as follows: to identify the issues, major and minor, and to examine the relative significance or priority of the issues currently inhibiting solution of the israeli-palestinian conflict to share knowledge and insights about the current israeli-palestinian situation from different points of view to construct a comprehensive model of the situation to explore the benefits and costs of alternative courses of action all the participants were well-versed about the conflict, but they were briefed about the ahp method. initially, the group generated many issues regarding the conflict. later, criteria were obtained from these issues. saaty & zoffer (2011) write: we consider each party’s list of issues, which if addressed by the other party by making concessions, would provide sufficient benefit to that side toward meeting their goal. they, in turn, would be willing to make concessions to the other side to balance those concessions with an equivalent trade-off. we refer to these issues as criteria. (p. 9) the goal was to find the lasting solution to end the decades-long conflict. the crux of the deliberation process was the generation of concessions from the israeli as well as the palestinian point of view. without concessions by both parties, it is unlikely that a conflict such as this can be resolved. if palestinians make some concessions to the israelis, it will benefit the israelis but at the cost of palestinians. the same applies the other way around. understandably, some of the benefits and costs are intangibles. these ijahp article: islam/work on resolving israeli-palestinian conflict international journal of the analytic hierarchy process 331 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.523 must be quantified to reach a meaningful conclusion. ahp can elegantly do the job using its inherent 1-9 fundamental scale. saaty & zoffer write (2011): it is to be noted that when we speak of measurement, we are not only referring to tangibles like length, mass, time, or money, but also to intangibles like political and social influence and of the diversity of emotions like love and hate, religious ideology, participant’s attitude, and quality-of -performance, to name just a few that are within the boundaries of current thinking. ahp solves an age-old problem of decision makers only being able to measure accurately tangible factors with mere mention of intangibles where no measurement instruments were available. this process can be used not only to measure but also to integrate all factors, tangible and intangible, and provide a comparable scale that permits combining and trading them off. (p. 10) bringing people from both sides to a discussion table is not new. in fact, concessions have already been generated in a number of u.s.-brokered peace deals. but what is new in the ahp discussion session was the evaluation of the concessions based on economic, social, geographic, humanitarian and historical ground. saaty and zoffer leveraged the strengths of ahp and the fast-hand knowledge of the participants in the conflict to arrive at a scientific solution. the israeli-palestinian conflict is unique in length as well as breadth. there are countless concerns that surround the conflict. within a 3-day meeting, it is nearly impossible to pin everything down, so the groups decided to limit their attention to certain concerns. interestingly, there was unanimous agreement on the nature of the conflict. debates surfaced, however, on the concerns. these concerns differed according to which constituent group was putting them forward. saaty & zoffer write (2011): for example, among the palestinian key constituents are palestinian refugees, hamas followers, fatah followers, palestinians who still live in israel, and diaspora palestinians. among the israeli constituents are the ultra-right orthodox community, israelis living in settlements in the west bank, those associated with the likud movement, those associated with the labor party, and those more ijahp article: islam/work on resolving israeli-palestinian conflict international journal of the analytic hierarchy process 332 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.523 actively seeking peace as a primary objective, without dwelling on the details of it. (p. 12) the model they laid out cut across the visible as well as the invisible boundaries between the two parties. the application has clearly shown the versatility of ahp, especially in proposing solutions to end the world’s most deliberated conflict. but it must be admitted that ahp propositions are not the end-in-itself. saaty & zoffer (2011) write, “we need to begin by emphasizing that the outcome of our effort is a beginning of an elaborate undertaking to produce a viable solution to the israeli–palestinian conflict.” (p. 6) the present application of ahp requires assessments of the benefits, costs, perceived benefits and perceived costs of the proposed trade-offs. this can be done by answering a series of questions that include: which benefits are more important than others?, which consequences weight more heavily than others?, and what scenarios are likely to take place? as mentioned before, the hierarchy consisted of several levels: the overall goal, and a set of criteria that captures the values represented by the goal. the concessions (trade-offs) occupying level 3 are evaluated with respect to the criteria to determine their priorities in serving the goal. a representative hierarchy is shown in figure 1. figure 1: a partial hierarchy of the israeli-palestinian conflict legend: c1: integrity and unity of israeli society post agreement c2: security c3: strengthening the alliance with the united states c4: make israel more attractive to jewish diaspora and israeli citizens c5: end of claims and end of conflict c6: legitimization of the state of israel c7: stop being occupiers c8: peace, economy and stability in region c9: maintain the jewish majority of israel alongside the arab minority c10: weakening the radical forces in the middle east headed by iran israeli cost from own concessions c1 c2 c3 c4 c5 c6 c7 c8 c9 c10 ic 1 ic 2 ic 17 ijahp article: islam/work on resolving israeli-palestinian conflict international journal of the analytic hierarchy process 333 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.523 israeli’s concessions description ic 1 abandon the idea of a jewish state ic 2 accept palestinian full control of the borders of the palestinian state and its outlets ic 3 accept the historical responsibility for the palestinian refugee problem ic 4 accept the palestinian refugees’ right to return ic 5 accept to abide by the status quo in the holy places in jerusalem ic 6 accept to abolish the law of return ic 7 accept to respect the integrity of the west bank and gaza by allowing free and safe passage between the two areas ic 8 accept east jerusalem as the capital of the palestinian state ic 9 accept a two-state solution on the borders of june 4, 1967 ic 10 allow all parties to have equal access to and control of religious sites and holy places ic 11 allow the sharing of all natural resources between palestinians and israelis ic 12 comply with all applicable un resolutions ic 13 evacuate settlers of jewish settlements on land claimed by the palestinians with or without compensation ic 14 release all political prisoners, including those who are israeli citizens ic 15 share jerusalem as two capitals of two states ic 16 solve the palestinian refugee problem in a just and agreed-upon manner ic 17 stop incitement by the religious and national education and religious leaders in israel against muslims and arabs and guarantee the rights of israeli minorities to deal with the israeli-palestinian conflict, saaty and zoffer took a retributive conflict resolution approach that takes into consideration the benefits to a from concessions by b and the costs to a of the return concessions a makes as well as a’s perception of the benefits b gets from the concessions a makes, and of the costs to b of the concessions b makes. similar considerations are made from b’s perspective. the concept of retributive function is used in synthesizing the outcomes (profits and costs) of the concessions made by both parties. let us consider two parties named a and b. in the negotiation process, a considers a concession from b not only with respect to the benefit to a but also the costs to b in providing the concession. a not only wants to gain from concessions made by b, but wants to ensure, because of the retributive nature of the long-standing conflict, that they cost b, or inflict some pain on b, which might be non-monetary, as well. therefore, a’s gain from a concession provided by b is described by taking the product of a’s benefits and b’s costs. ijahp article: islam/work on resolving israeli-palestinian conflict international journal of the analytic hierarchy process 334 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.523 hence, we have: concessionsa'fromgainsb'ofperceptionsa' concessionsb'fromatogain ratiosa'  btoloss atogain b       concessionsa'fromcostssa'benefitsperceivedsb' concessions'fromcostssb'benefitssa' in a similar way, b’s ratio can be defined. finally, measure of equality between the parties in the trade of concessions may be calculated as ato(loss)gaineretributiv ' '  ratiosb ratiosa ideally, we seek a solution by investigating how closely matched are gain-to-loss ratios of concessions and how to trade them off in such a way that neither side’s ratio is much less than that of the other. tom and zoffer (2011) commented: in fact, there is usually more than one solution to complex negotiations. what is needed is a degree of moderation on both sides that makes it possible for the gain-to-loss ratios to be sufficiently close for trade-offs against each other. (p. 46) table 1 shows the gain/loss ratios derived for the selected concessions given by israelis and palestinians. ijahp article: islam/work on resolving israeli-palestinian conflict international journal of the analytic hierarchy process 335 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.523 table 1 matching concessions with corresponding gain/loss ratios in second meeting the gain/loss ratios were estimated through a series of ratings exercises. table 2 provides a sample of the israeli concession ratings relative to the criteria set. israeli’s concessions gain/loss palestinian’s concessions gain/loss adhere to international law accept two-state solution (israeli control of jerusalem) right to education 1083 adhere to international law 1084 comply with un resolutions accept a demilitarized palestinian state (not including jerusalem) right to a free economy, economic development & trade 461 end of incitement of anti-israeli sentiment in school 457 accept un declaration of human rights free access, safety, and movement of israelis in palestinian territory removal of wall & other barriers 1008 renounce & reign-in violence 1016 respect the territorial integrity of gaza and west bank acceptance of non-contiguous palestinian state shared administration of resources 2.46 allow jewish settlers to stay in judea and samaria with israeli citizenship and under israeli law 2.47 palestinian control of own natural resources accept sharing of natural resources shared jerusalem drop opposition to trade & normal relations with israel turnover settlement with or without compensation 4.26 work cooperatively with israel 4.23 total 2558.72 2563.7 ijahp article: islam/work on resolving israeli-palestinian conflict international journal of the analytic hierarchy process 336 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.523 table 2 israeli concession ratings relative to criteria (israeli benefits from palestinian concessions) concessions control jerusalem & holy places (0.245) refugee compensation & settlement (0.251) increasing security (0.193) permanent borders (0.000) controlling & rationing of water (0.162) human rights (0.022) settlements in palestinian territory (0.127) total ideals accept two-state solution very low excellent excellent negligible medium 0.680 0.736 acceptance of non-contiguous state negligible excellent excellent negligible negligible 0.605 0.655 acknowledge israel’s existence as a jewish state excellent excellent excellent very high medium 0.924 1.000 acknowledge israel’s existence as an independent state medium high high very high medium 0.762 0.824 agree to compromise to demand of right of no return high excellent excellent excellent very high 0.917 0.992 declare against iranian nuclear development 0.000 0.000 drop opposition to trade & normal relations with israel negligible very low very low negligible high 0.402 0.435 incitement of anti-israeli sentiment in school excellent excellent excellent negligible very high 0.853 0.923 lobby arab states to allow israelis right to return 0.000 0.000 make compromise on the status of jerusalem excellent low excellent medium 0.653 0.707 denounce & reign-in violence excellent very low excellent high excellent 0.795 0.860 seek assistance for a legitimate settlement of refugees negligible excellent excellent negligible 0.556 0.602 sharing of natural resources negligible very low medium negligible high 0.459 0.497 work cooperatively with israel negligible very high very low very high negligible 0.677 0.732 ijahp article: islam/work on resolving israeli-palestinian conflict international journal of the analytic hierarchy process 337 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.523 the present application of ahp in resolving the israeli-palestinian conflict has many matrices of priorities. as a sample, table 3 shows various priorities for israeli concessions. table 3 priorities derived for israeli’s concessions concessions israeli’s costs israeli’s perception of palestinian’s benefits israeli’s total loss palestinian’s benefits palestinian’s perception of israelis’ costs palestinian’s total gain (1) (2) (1)*(2)*1 000 (3) (4) (3)*(4)*100 0 1 1 1 1000.00 0.8830 0.9683 855.03 2 0.6445 0.7637 492.18 0.9894 0.9717 961.31 3 0.9051 0.2705 244.88 0.9574 0.7835 750.15 4 0.9470 0.8253 781.53 0.8830 0.9515 840.11 5 0.1961 0.5405 106.01 0.7979 0.7583 605.05 6 0.8824 0.4280 377.70 0.5426 0.7410 402.01 7 0.1984 0.5149 102.15 0.9787 0.9054 886.12 8 0.8299 0.8068 669.54 1 0.9692 969.17 9 0.0545 0.8205 44.75 0.9787 0.9080 888.67 10 0.1006 0.5323 53.55 0.8085 0.5459 441.41 11 0.1120 0.2853 31.96 0.6702 0.5260 352.52 12 0.8596 0.9571 822.76 0.9787 0.9075 888.20 13 0.3593 0.8915 320.31 0.9255 1 925.53 14 0.5178 0.4781 247.56 0.7553 0.6508 491.59 15 0.1633 0.6027 98.42 0.8511 0.7334 624.17 16 0.1806 0.7329 132.34 0.8830 0.9174 810.08 17 0.0741 0.1110 8.23 0.4149 0.4991 207.05 the session concluded with the following resolutions known as “the pittsburgh principles”. 1. a two-state solution on the borders of the 4th of june 1967, with mutually agreed upon land swaps. 2. israel must respect the integrity of the west bank and gaza by allowing free and safe passage between the two areas, and the palestinian state must guarantee that any agreement reached with israel will be accepted and supported by the majority of the palestinian people both in gaza and the west bank. 3. east jerusalem will be the capital of the palestinian state. the parties will maintain the status quo of the holy places in jerusalem. 4. acknowledge israel’s existence as a jewish state without jeopardizing the rights of its minority israeli citizens. 5. evacuation of israeli settlers from the palestinian territories that are not included in the land swaps. 6. palestinian full control of the borders of the palestinian state and its outlets, and deployment of a temporary agreed upon multinational military monitoring system in the jordan valley. ijahp article: islam/work on resolving israeli-palestinian conflict international journal of the analytic hierarchy process 338 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.523 7. solve the palestinian refugee problem in a just and agreed upon manner. 8. a demilitarized palestinian state. 9. agreed upon international monitoring mechanism and agreed upon binding international arbitration mechanisms. 10. the full implementation of these principles will conclude the conflict and the address the claims of the two parties. 4. conclusion numerous negotiations and peace meetings have been conducted to resolve the decadeslong, extremely complicated conflict between the israelis and palestinians. the world has, however, seen no tangible positive output coming from those prolonged sessions. there could be several reasons for this, one of which includes an inflexible mind set for both parties when the time to implement the proposed solution comes. both sides have some concessions that are non-negotiable, e.g., the israelis do not want to allow the palestinian refugees to return to their homeland; on the other hand, the palestinians refusal (especially hamas) to acknowledge the existence of a full-fledged jewish state. further, to fully implement the proposed solution, one needs to ensure the participation of all the stakeholders when formulating the policy. excluding one sizeable party, which has a tremendous stake in the conflict, will only cause lasting peace to elude the people in the region. nevertheless, a highly commendable effort has been undertaken by saaty and zoffer to rekindle the hope of a peace in the region. as mentioned before, ahp will not replace the formal peace negotiation process; rather it supplements and supports the process. ahp findings must be used by politicians to push the peace process forward. let me make some concluding remarks about the ahp application to the conflict. (1) a major outcome of the process is the identification of 106 issues and numerous concessions pertaining to the conflict. note that this happened because of the spontaneous participation of both parties in a friendly atmosphere. (2) all the concessions were prioritized strictly based upon their desirability using a scientific method, the ahp, to solicit judgments from the parties. (3) ahp has shown how human judgments, emotions, feelings, aspirations can all be combined in a holistic manner resulting in deriving priorities for the concessions. (4) the ahp exercise made it possible for the participants to consider a wide variety of potential trade-offs. the beauty of the process is that it favoured neither side. the main advantages of applying the ahp in this conflict are summarized below (saaty & zoffer, 2011): ahp facilitates conflict resolution by organizing perceptions, feelings, judgements and memories into a framework that exhibits the forces that influence judgement and their priority outcomes. (p.8) the traditional approach involving diplomacy and face to face negotiations has led to an inconclusive outcome, partially attributable to attitudes colored by strong emotions on both sides. our ijahp article: islam/work on resolving israeli-palestinian conflict international journal of the analytic hierarchy process 339 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.523 approach attempts to address the impact of negative attitudes by focusing the participants on making judgments that measure the intensity of their perceptions about the influences that each of the issues brings to bear upon the final outcome. (p.9) no other approach provides an opportunity for priorities to be identified which organize the issues and concessions in a way that cannot occur when only face to face negotiations are being conducted. the priorities provide a perspective that allows negotiators to grasp the role or relative importance which each issue or concession plays in the total conflict. the chaos which has characterized much of the past 60 years of face to face and of shuttle negotiations is minimized as a result of the organized approach which is intrinsic to the ahp process. (p.12) may the ahp’s resolutions be instrumental in resolving the israeli-palestinian conflict permanently! that would be a fitting tribute to tom for his work. ijahp article: islam/work on resolving israeli-palestinian conflict international journal of the analytic hierarchy process 340 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.523 references saaty, t. l. (1977). a scaling method for priorities in hierarchical structures. journal of mathematical psychology, 15(3), 234-281. doi: https://doi.org/10.1016/00222496(77)90033-5 saaty, t.l., & zoffer, h.j. (2011). negotiating the israeli-palestinian controversy from a new perspective. international journal of information technology and decision making, 10(1), 5-64. doi: https://doi.org/10.1142/s021962201100421x https://doi.org/10.1016/0022-2496(77)90033-5 https://doi.org/10.1016/0022-2496(77)90033-5 https://doi.org/10.1142/s021962201100421x microsoft word inglesproceso general de determinaciã³n de umbral teã³rico2 070620.docx ijahp article: garuti/how to obtain local and global thresholds in ahp/anp international journal of the analytic hierarchy process 183 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.802 how to obtain local and global thresholds in ahp/anp claudio garuti fulcrum ingenieria claudiogaruti@fulcrum.cl abstract this paper has two main objectives. the first objective is to provide a mathematically grounded technique to construct local and global thresholds using the well-known rate of change method. the next objective, which is secondary, is to show the relevance and possibilities of applying the ahp/anp in absolute measurement (am) compared to the relative measurement (rm) mode, which is currently widely used in the ahp/anp community. the ability to construct a global threshold would help increase the use of ahp/anp in the am mode (rating mode) in the ahp/anp community. therefore, if the first specific objective is achieved, it would facilitate reaching the second, more general objective. for this purpose, a real-life example based on the construction of a multi-criteria index and threshold will be described. the index measures the degree of lag of a neighborhood through the urban and social deterioration index (usdi) based on an ahp risks model. the global threshold represents the tolerable lag value for the specific neighborhood. the difference or gap between the neighborhood’s current status (actual usdi value) and this threshold represents the level of neighborhood deterioration that must be addressed to close the gap from a social and urban standpoint. the global threshold value is a composition of 45 terminal criteria with their own local threshold that must be evaluated for the specific neighborhood. this example is the most recent in a large list of ahp applications in am mode in vastly different decision making fields, such as risk disaster assessment, environmental assessment, the problem of medical diagnoses, social responsibility problems, bocr analysis for the evolution of nuclear energy in chile in the next 20 years and many others. (see list of projects in appendix). keywords: threshold; scales; absolute measurement; ahp/anp; rate of change; semaphore 1. introduction the first objective of this paper is to provide a mathematically grounded technique to determine the tolerable risk level for each criterion (local risk thresholds) to obtain the global threshold (gt) of a risks model in order to analyze the set of alternatives based on their profile behavior and classify them as acceptable/unacceptable (or equivalently as red/green, red/yellow/green, or more gradual levels between red and green). a multicriteria measurement index that assesses and prioritizes solution alternatives for a mailto:claudiogaruti@fulcrum.cl ijahp article: garuti/how to obtain local and global thresholds in ahp/anp international journal of the analytic hierarchy process 184 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.802 given problem might not be adequate in a situation where time and resource constraints exist. the main benefit of having a gt associated with this measurement index is the ability to assess if the alternatives are good enough for the proposed goal and to discard the alternatives that do not fulfill the threshold condition. eventually, not even the best alternatives, according to the index, will satisfy the threshold. in other words, the gt defines the conditions for the green/yellow/red scores to classify the alternatives. this is valuable information for the decision maker to possess before the cost information is considered; with a focus on effectiveness, the gt allows ineffective alternatives to be discarded even if it costs nothing. in the same manner, only the green or yellow alternatives could be considered, while the red ones are discarded, regardless of their cost. a “threshold” is a reference value within an ordered scale that defines a level of acceptability. for example, in the chilean school score scale (ranging from 1 to 7), the acceptance reference level is 4.0. this means that all students with a final score below this threshold at the end of the year are not allowed to move to the next class level. the ahp/anp provides a ranking value for the alternatives. in the case of am models, the ranking values are scores that may be interpreted with values between 0 and 100%. in a benefits model, the ideal alternative that scores 100% has been assessed at the highest scale level for each criterion. in a risk model, the ideal is 0, which also represents the best condition of no risk. in the ahp/anp context, the gt is a value between 0 – 1 (or 0 – 100%) that indicates the reference of acceptability for the ranking scores of the alternatives. therefore, all alternatives whose final scores are above (benefits model) or below (risks model) the gt are considered to satisfactorily comply with the reference condition. nevertheless, we are not only concerned with the classification aspect, which has already been addressed with several methods and software (topsis 1 for example), but we are also concerned with how to determine the critical equilibrium points that precisely define the classification zones within the measurement scale for either local and global thresholds (splitting between red and yellow, between yellow and green, etc.). the same process applies for tangible or intangible criteria as with continuous or discrete measurement scales, as defined in the ahp/anp model. it should be noted that this procedure may only be carried out when the ahp/anp rating mode (am mode) meets the following condition: the calculation of the theoretical breakpoint (tolerance threshold) for each measurement scale is based on the rate of change method. 2. measurements in ahp/anp and rate of change: literature review the relative measurement (rm) and the absolute measurement (am) are the two types of measurements used in ahp/anp models. the difference exists only at the alternative level. in the rm mode, the alternatives for each terminal criterion present in the ahp/anp model are pair compared, and it can be determined how many times one element fits into the other 1 technique for order of preference by similarity to ideal solution was developed by yoon, hwang, chin-lai (1981). ijahp article: garuti/how to obtain local and global thresholds in ahp/anp international journal of the analytic hierarchy process 185 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.802 using saaty’s fundamental scale. this approach (comparing one alternative against the others) generates a connection or dependency between the alternatives, and when the set of alternatives is modified by the addition or removal of an alternative, this modification may produce a change in the ranking of the alternatives. this effect is also known as rank reversal in the literature and is widely used to alter preferences in the marketing environment. this relative measurement is appropriate if the user wishes to maintain the dependency relationships that exist among the alternatives. on the other hand, if the user does not want the order of the alternatives to be modified, or if there is a large set of alternatives to be evaluated (over 10 alternatives), or the user is interested in measuring the performance of the alternative regarding a reference or threshold value (for example, a maximum risk value or a minimum benefit value), then the am mode should be selected since it provides the opportunity to easily determine if the alternative’s performance is good or bad (green or red). the am mode, also known as “ratings”, is a different method to rank alternatives with the ahp (saaty, 1986; millet & saaty, 2000) that provides two additional advantages over the relative mode of measurement. the first advantage is the ability to increase the set of alternatives above nine, and the other is the ability to preserve the ranking among the alternatives (saaty, vargas, & whitaker, 2009). nevertheless, it seems that the application of ahp in the am mode is not as widespread in the ahp community as its benefits would suggest. valerio salomon (2016) wrote an interesting essay discussing how little the am is used compared with rm by referencing the numbers of papers that were published in the previous two years in the international journal of the analytic hierarchy process. in my personal experience, the few times that i have seen the am mode applied in ahp/anp models, there have been problems such as poor description of the ordinal scale levels, attempts to directly transform the ordinal scale by using a graphic software option, or the use of a common scale applicable over all terminal criteria. likely, in this last case the final rating value would be very easy to interpret according to the unique scale in use. these problems significantly reduce the accuracy in the alternative’s assessment, and therefore, jeopardize the model’s results. a key step in obtaining the gt is the calculation of an individual scales threshold, which is achieved by applying the rate of change method. the definition of rate of change refers to the extent to which one variable is modified in relation to another. “it is then the magnitude that compares two variables from their units of change. in the case where the variables are not related, their rate of change will be null. by the way, the instantaneous rate of change corresponds to the well-known concept of derivative of a function” (orton, 1994). according to orton a. (1984), “the rate of change method is a manifestation of proportionality, and we know that ratio and proportion present many difficulties for learners”, and “rate of change is perhaps a more general concept than gradient”. when calculating the rate of change between two related variables, it is possible to determine the breakpoint in any cardinal scale regardless of whether it has a qualitative or quantitative description or a continuous or discrete function. it is only necessary that the associated measurement scales are monotonically increasing or decreasing. ijahp article: garuti/how to obtain local and global thresholds in ahp/anp international journal of the analytic hierarchy process 186 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.802 3. absolute measurement assessment mode the am mode is equivalent to using a stick to measure an object. no matter how many objects there are in the set or whether objects are added or removed, the initial measurement will not be altered. this is because the am mode uses absolute measurement scales which provide absolute values instead of comparing the alternatives among themselves as objects. thus, the evaluation of an alternative does not depend on the behavior of any other alternative or any property that the alternatives may possess (for example, the property of being unique within the set). this advantage is accompanied by the challenge of needing a measurement scale for each terminal criterion in the model that also complies with being an absolute ratio scale. most of the time when we measure something in absolute terms an interpretation of that measurement is required before an action occurs. if we measure the quality of an alternative within a benefits model or if we measure the risks of an action within a risk model, it would be very useful to have a reference value for comparison and to help determine if the alternative is good or bad (green or red in the semaphore). for example, in the case of a quality measurement model, what is the minimum quality level or minimum acceptable value under which an alternative is no longer eligible? similarly, with a risks model, what is the maximum risk or stress level that the decision maker would be willing to accept for an alternative to be a viable option? in both cases, many criteria are required to build a comprehensive index to measure the alternative’s global performance (the behavior profile). setting gts allows alternatives to be assessed and helps create strategic long-term scenarios that are acceptable for stakeholders participating in the decision-making process (community, government, strategic partners etc.). in a selection problem (only one alternative will be selected), the expectation is that the ahp/anp ranking will clearly provide the best alternative. in this case, the threshold is the assessment of whether the selected alternative is “good enough” given the parameters of the problem. in problems where the ahp/anp model is a first step that introduces a multicriteria score (generally including intangible factors normally not handled by many operational methods), the gt allows the set of viable options to be reduced ensuring that a minimal benefit level is satisfied (benefit model) or a maximum risk level is ensured (risks model). in benchmarking problems, which are quite common with the am because of the larger sets of alternatives, the red/yellow/green classification provided by the gt is required. in order to understand global and individual behavior, groups are defined and thresholds are applied, even with implicit conditions. the ahp/anp is basically a compensation method. however, the local thresholds (lts) allow a non-compensation process, (at least, avoiding unacceptable trades-off) which is another reason for the am rating mode to be used in ahp/anp. as discussed previously, the four following conditions are needed to build a gt: 1. the existence of an ahp/anp model in am/rating mode 2. measurement scales for each terminal criterion. these scales must reflect the intensity ratios between their levels and should be strictly increasing or ijahp article: garuti/how to obtain local and global thresholds in ahp/anp international journal of the analytic hierarchy process 187 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.802 decreasing levels of intensity, ranging from 0 to 1. 3. a transformation function that respects the value limits between the adjacent levels of the scale and is capable of measuring the rate of change between them (in order to find the equilibrium point between both levels). 4. the transformation function must be a strictly increasing or decreasing function 2 . 4. transforming the measurement scale into a cardinal scale on a daily basis, many descriptive or qualitative scales are used in very different contexts (ex. very high, high, moderate, low); quantitative scales in terms of percentages are used (ex. over 66%, between 33-66%, less than 33%), or scales are used directly by levels (ex. 1, 2, 3, 4, 5). the vast majority of these numerical scales are ordinal scales, that is, they only order the levels of the scale in an increasing or decreasing mode, but there is no meaningful interpretation of the numbers themselves (same order is provided by 1,2 3; 10, 11, 35 or 20, 400, 480). clearly, the ordinal scale does not work if the goal is to build a gt in a multicriteria model. therefore, the first step is to transform these qualitative and ordinal scales into cardinal scales or absolute ratio scales where the ratio of the numbers reflects knowledge about the measurement scale. it should be noted that the absolute ratio scale is a particular example of a ratio scale where the invariant is the identity function (y = x). it is called absolute because it is dimensionless (does not have a unit). some examples of these kinds of numbers are  (the proportion between the perimeter and diameter of a circumference), the number of persons in a room, or the set of real numbers. thus, it is possible to use these numbers for any arithmetic operation needed (saaty, 2000, garuti, 2017). 4.1 explaining the process by developing a real (simplified) example the process of obtaining the gt of a risk multi-criteria model will be developed through a simplified real example. the following hierarchical model is the model built for (and with) the chilean ministry of social development and family (msdf) and the chilean ministry of housing and urbanism (minvu). starting at the top, the model builds the global risk index usdi with two strategic criteria (urban deterioration and social vulnerability), then the following six sub-strategic criteria: environmental, physical and functional deterioration under urban deterioration; and population, community and housing vulnerability fall under social vulnerability. the two hierarchical levels below the goal with the eight criteria represent the strategic part of the model that is shared with the neighbors to determine their interests through the paircomparison process for three matrices (1 of 2x2 plus 2 of 3x3, totaling 7 questions). under the population vulnerability sub-strategic criteria, there are two indicators as follows: low education level and lack of school attendance, each with its own measurement scale 3 . part 2 although this requirement is the general case for transformation functions from ordinal to cardinal scales, it is still possible to operate in the special case when the function changes the sign of the slope. for example, with the comfort transformation function that is based on temperature, comfortable levels are in the middle and uncomfortable levels are at the extremes. in such cases, the function must be separated into two parts, the increasing and the decreasing parts, and the local threshold calculated for each one. 3 the complete model has 45 terminal criteria or indicators. ijahp article: garuti/how to obtain local and global thresholds in ahp/anp international journal of the analytic hierarchy process 188 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.802 of the model can be seen in figure 1. the scale of the low education level criterion is represented by the box surrounded by a red line. figure 1 usdi model source: usdi scale manua. mdsf the measurement scales are generated through pairwise comparison matrices that compare the different scale levels, which normally belong to an ordinal/qualitative scale (the comparison process should be done even if the scale belongs to a quantitative type of scale). through the pairwise comparisons matrix, the ordinal scale is transformed into an absolute ratio scale by the eigenvector operator, which provides the priority vector. the pairwise comparison matrix must be carried out by an expert who is familiar with the field that the indicator belongs to. the associated measurement scale for the low education level indicator is described below. table 1 ordinal scale for low education level source: usdi scale manual mdsf level 1: high over 25% of the population between the ages of 18 and 29 has not completed secondary education (the last year of high school). level 2: moderate fewer than 25% and more than 5% of the population between the ages of 18 and 29 has not completed secondary education (the last year of high school) level 3: low fewer than 5% of the population between the ages of 18 and 29 has not completed secondary education (the last year of high school) level 4: very low 100% of the population between the ages of 18 and 29 has completed secondary education (the last year of high school has been completed) note: levels and their definitions, including the range values, were provided by the mdsf team leader since the scale is currently in use for this indicator by mdsf in chile. as previously explained, it is possible to transform this ordinal descriptive scale into a cardinal scale of ratios through a pair comparison matrix and the eigenvector operator to usdi index urban deterioration environmental deterioration physical deterioration functional deterioration social vulnerability population vulnerability low education level lack of school attendance community vulnerability housing vulnerability ijahp article: garuti/how to obtain local and global thresholds in ahp/anp international journal of the analytic hierarchy process 189 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.802 determine the intensity value of each level (the priority vector). the results are shown in table 2 with values provided by mdsf and minvu experts within the project team. the ratio of inconsistency (ri) represents the ratio between the matrix inconsistency index and a random index of inconsistency, and the smaller the value the better (for 4x4 matrices the threshold value is 8.5%), as defined in ahp methodology. saaty (1988) provides references on how the priority vector and ri values are obtained. table 2 pair comparison matrix of the scale low education level source: usdi scale manual low education level high moderate low very low priority vector high 1 2 7 9 1 moderate ½ 1 4 6 0.5524 low 1/7 1/4 1 3 0.1736 very low 1/9 1/6 1/3 1 0.0847 ri = 0.0309 (3.09%) the column on the right is the normalized eigenvector (ideal mode of normalization) which corresponds to the priority vector and reflects the ratio of preferences between the different intensity levels of the scale. this scale is now an absolute ratio scale, that is, the high level is exactly 1.801 times more intense than the moderate level (1/0.5524) and the moderate level is 3.198 times more intense than the low level (0.5524/0.1736), and all the values are dimensionless. the priority vector can also be seen as the rate of change between the levels. for instance, the moderate level is (approximately) half of the high level. therefore, two moderates are needed to make one high (2:1). this priority vector is also called the transformation function because it transforms an ordinal scale into a cardinal scale. once the cardinal scale has been built, it is possible to calculate the threshold for the low education level indicator. in a risks model, this threshold should represent the point of maximum stress that the expert is not willing to exceed. the process of transforming the user-defined scale into a cardinal scale does not consider “its numeric values” (in this case, 25% of the age ranges of 18 to 29); therefore, the process is absolutely applicable for totally descriptive scale levels or even continuous functions. since the threshold is defined by the priority vector that is already built, there are also no restrictions for building a threshold for a qualitative scale. 5. calculating the local threshold by topological analysis the scales’ lts are obtained from the transformation functions as described above. the transformation function represents, in cardinal terms, the difference in intensity between the levels described in the ordinal scale and transforms it into an absolute ratio scale. there are several ways to calculate the indicator’s associated threshold, normally called the lt as follows: ijahp article: garuti/how to obtain local and global thresholds in ahp/anp international journal of the analytic hierarchy process 190 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.802 transformation function for terminal criterion: low education level 1,0 0 0,9 0 0,8 0 0,7 0 0,6 0 high moderate low very low  order topology analysis (applying the rate of change technique on the transformation function). • geometric analysis of the transformation function (gravity center analysis). • statistical analysis of the transformation function. in this study, we will mainly refer to the order topology analysis because it is the most accurate and relatively simple to perform. figure 2 below shows the transformation function of the low education level indicator. figure 2 example of transformation function source: usdi scale manual mdsf 5.1 rate of change technique the first and most appropriate method to calculate an indicator’s lt is order topology analysis through the rate of change technique. in this scheme, the aim is to determine the expert's rate of change while moving from one level to an adjacent level of the scale. to accomplish this, the rate of change between two adjacent levels is analyzed; in this case, the moderate and low (m/l) levels are used. it is assumed that the local risk threshold occurs within this range. the following equation based on the idea of balance or equilibrium (application of the gravity center idea) is a rate of change function proposed as a measure of the rate of change between the adjacent moderate (m) and low (l) levels in order to obtain the equilibrium point (ep) which in turn represents the scale´s lt: ep = (𝑀+(𝑀/𝐿)∗𝐿) (1+(𝑀/𝐿) (1) source: fulcrum engineering to determine if equation (1) is suitable as a measure of rate of change, the limits between the adjacent levels must be analyzed; in this case, where l approaches m (lm or m/l1) and m moves away from l (m >> l o m/l ∞). if both limits (upper and level high moderate low very low value 1,0 0,5524 0,1736 0,0847 ijahp article: garuti/how to obtain local and global thresholds in ahp/anp international journal of the analytic hierarchy process 191 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.802 lower) belong to the range [l, m], then this is a good starting point for the ep. 5.2 analysis of cases at the limit a) case analysis when l tends towards m (lm or m/l1): ep = lim 𝑀 𝑡𝑜 𝐿 𝑀 + ( 𝑀 𝐿 ) ∗ 𝐿 1 + ( 𝑀 𝐿 ) = (m + 1 ∗ l) / (1 + 1) = l = m the equation above corresponds to a case where the expert considers the moderate and low levels similar and is indifferent to which of the two values (approximately) is present, and in this case, the best value for the lt is precisely its arithmetic average (see table 2). simply put, for the expert this means that the change from a low level of deterioration to a moderate level is of little importance, and the expert is not very concerned with this degree of alteration (they are open to a change from l to m or m to l without considering it a problem). b) case analysis when m becomes much bigger than l (m >> l or m/l  ∞): the other possibility with the ep equation is when m becomes much bigger than l. in this case, the move from the low to moderate level is a considerable change. the expert is concerned when levels change from low to moderate or vice versa. simply put, the expert is concerned when levels change from l to m and the degree of acceptance of this change is low or nil. the more concerned the expert is about the rate of change, the closer the equilibrium point (ep) will be to the value of the low level (minimum risk or minimum stress level) which represents the expert’s comfort zone. at the limit when (m/l)  ∞, and when the l´hopital’s property is applied for this kind of limit, the new ep becomes: ep = lim𝑀 𝐿 𝑡𝑜 ∞ 𝑀+( 𝑀 𝐿 )∗𝐿 1+( 𝑀 𝐿 ) = (0 + l) / (0 + 1) = l thus, the rate equation is as follows: l ≤ 𝑀+( 𝑀 𝐿 )∗𝐿 1+( 𝑀 𝐿 ) ≤ m. this means that the function values belong to the range [l, m]. in this way, the function that is chosen to represent the rate of change adequately covers the range. furthermore, since these two levels are adjacent, we may say that the proposed equation adequately approximates the rate of change of the transformation function in this range. also, the reader may note that the proposed equation resembles the center of gravity equation for the geometric equilibrium point between two adjacent levels (m and l). equation (1) represents how many units of l are needed to “balance” one unit of m for the critical ep. we will return to this point later from a geometric point of view. ijahp article: garuti/how to obtain local and global thresholds in ahp/anp international journal of the analytic hierarchy process 192 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.802 selection of the adjacent levels selecting the rate of change’s best function depends on the number of adjacent levels in the scale. the following table links the number of levels of a rating scale with a possible rate of change function. this function should be considered when calculating the equilibrium point (ep). table 3 adjacent values of the rate of change according to number of levels source: fulcrum engineering # levels general description of the scale ep (maximum stress point in a risk model) 2 dichotomic scale (0 – 1) 0 3 high – moderate – low (𝑀 + (𝑀/𝐿) ∗ 𝐿) / (1 + 𝑀/𝐿) 4 high – moderate – low – very low (𝑀 + (𝑀/𝐿) ∗ 𝐿) / (1 + 𝑀 /𝐿) 4 very high – high – moderate – low (𝐻 + (𝐻/𝑀) ∗ 𝑀) / (1 + 𝐻 /𝑀) 5 very high – high – moderate – low – very low m optional: (h+(h/l)*l) / (1+h/l) the procedure seeks the best ep (when the rate of change between m and l begins to be less attractive). at this point, the process stops and selects that value for m/l. this should be the expert’s point of maximum tolerance for making a change in the rating scale and therefore reaching the best ep (the critical ep). when the formula for various integer values of m/l is tabulated, it is possible to see the behavior of the theoretical lt in this specific variable and choose the best value. it has to be noted that the scales in table 3 may have one more level (the zero level), which does not appear in the scales. this is because in an absolute ratio scale the existence of the zero is not necessary. (never put level zero in the pair comparison matrix, when it is needed, level zero can be added to the scale that is already created). table 4 values of the rate of change for the low educational level indicator rate of change m/l =1 m/l = 2 m/l = 3 m/l = 4 m/l = 5 m/l = 6 expression & value (m+1l) /2 = 0.3630 (m+2l) /3 = 0.2999 (m+3l) /4 = 0.2683 (m+4l) /5 = 0.2494 (m+5l) /6 = 0.2367 (m+6l) /7 = 0.2277 differential % 0,063 17.4% 0,032 10.5% 0,019 7.2% 0,013 4.9% 0,009 3.8% ijahp article: garuti/how to obtain local and global thresholds in ahp/anp international journal of the analytic hierarchy process 193 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.802 marginal rate of change curve 0.4 0.35 0.3 0.25 0.2 0.15 0.1 0,05 0 m/l = 1 m/l = 2 m/l = 3 m/l = 4 m/l = 5 m/l = 6 figure 3 marginal rate of change source: fulcrum engineering having set out the differentials for the m/l ratio, it is easy to see how the rate of change becomes less interesting as the fraction m/l grows. in terms of decreasing marginal risk, going from m/l=4 to m/l=5 is unattractive (the risk decreases very little, 0.013 or 4.9% from m/l=4). the change going to m/l= 6 is still worse since the differential of risk tends towards zero (0.009). thus, this is a very inefficient point for the ep (the associated costs will probably increase highly). as the preferred differential or sensitivity to change decreases, the rate of change value for (m/l) becomes less attractive, and although moving from one rate of change to a higher rate of change decreases the expert’s stress associated with the risk (greater comfort zone), it also indicates that we are not in the level of ep of maximum exchange or the maximum stress level for the expert in a risks model, which may create a sub-efficient ep. 5.4 optimal value calculation for (m/l) when looking at the limits of m/l and the slope of the curve in figure 4, it is obvious that the optimum point is between the rates of change m/l=2 and m/l=4 (range [2, 4]). before point m/l=2, the slope is too high and the value is reaching a very high risk level. on the other hand, after m/l=4, the slope is too low and the value is reaching a low risk level (the comfort zone). in the range [2 – 4], however, the rate of change is still interesting with an acceptable level of relative risk. considering this, the numerical value of the local threshold for this terminal criterion should be between 0.2490 and 0.2999 (table 3). as a first approximation, the rate of change could be taken as 3 or at most 4 but not higher because for rates of change greater than 4 the spreads are very low, which makes the rate of change unattractive. the rate of change m/l=3 corresponds to the closest integer to 3.182, which is the exact value for m/l given by the priority vector for this rating scale (see figure 2). that is, it corresponds to the direct value of moderate over low delivered by the priority vector of this criterion. it is important to note that the value of m/l=3.182 is obtained from the pairwise comparison matrix provided by the expert. this is important because it shows the expert’s final rate of change for m/l and may represent the optimal value for this criterion. the tabulation of the different options for m/l are shown in table 5. ijahp article: garuti/how to obtain local and global thresholds in ahp/anp international journal of the analytic hierarchy process 194 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.802 table 5 values for the rate of change with the range [2,4] rate of change m/l =2 m/l =3 m/l =3.182 m/l =4 (𝑀 + (𝑀/𝐿) ∗ 𝐿) / (1 + 𝑀/𝐿 0.2999 0.2683 0.2642 0.2494 when applying the m/l=3 value in this rating scale, the output for the lt is 0.2683. however, nothing prevents the exact ratio (3.182) from being used instead of m/l=3. in this case, the lt value is 0.2642. it should be noted that in the initial pairwise comparison matrix, the expert gave the value of 4 for the comparison between the moderate and low levels (4 represents moderate to strong on saaty’s fundamental scale). however, when the rest of the comparisons are considered, it is clear that in the matrix equilibrium point the expert is willing to go from m/l=4 (the initial comparison value) to an integer value closer to m/l=3 (3.182) (moderate in saaty´s scale). this implies that the expert is open to accepting a higher risk, therefore, allowing the level to change from m/l=4 to m/l= 3 (or 3.183 to be accurate). this means that 3.183 might be considered a good value when reaching the maximum stress value. in this case, the lt that corresponds to the low education level criterion is 0.2642 or 26.4%. the ep value for the low education level criterion is much closer to the low level (0.1736) than to the moderate level (0.5524) on the priority rating scale. although this was expected, it is very important to calculate the value of the lt as precisely as possible in order to correctly obtain the gt of the risks model, which is our main objective. 5.5 calculating the global threshold (gt) once the lt of each indicator has been calculated, it is possible to calculate the gt for the entire risks model using equation (2): gt = ∑ lti ∗ gwi 𝑖 i = 1, to number of terminal criteria (indicators) (2) when: lti = local threshold of criterion “i” gwi = global weight of criterion “i” source: fulcrum engineering it is remarkable that this simple equation provides such valuable information. this is because the equation is based on the application of the same measure concept for the evaluation of alternatives in the ahp/anp model with the am mode. that is, the calculation of the gt is obtained by knowing the lt value criterion to criterion, multiplied by the criterion’s global weight. also, having the lt profile's values point to point allows the gt profile to be built by weighting each lt value and plotting it. gt is a virtual alternative that can be defined and its performance behavior may be interpreted as the model's gt profile. a simplified example of the previous expression applied to the case study is given that assumes that the example in figure 1 contains four terminal criteria. the real model contains 45 terminal criteria, but only the first four are used to demonstrate the use of the above expression for gt calculation. ijahp article: garuti/how to obtain local and global thresholds in ahp/anp international journal of the analytic hierarchy process 195 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.802 transformation ratio function terminal criterion: low education level 1.00 0.80 0.60 0.40 0.20 0.00 h m l v l table 6 gt calculation example terminal criterion m/l lt(i) gw(i) lt(i) * gw(i) exposure to emission sources 1.777 (2) 0.2999 0.5288 0.1586 exposure to pollutants primary standard (dichotomic indicator) 0 0 0.1454 0 exposure to noise 1.777 (2) 0.2999 0.1604 0.0481 exposure to micro landfills 3.633 (4) 0.2494 0.1654 0.0413 sum 1.0 0.2480 according to table 6 and using equation (2), the gt of the model is 0.2480 or 24.8%. thus, if an alternative exceeds 0.248, then the alternative’s global risk is excessive and should not be accepted as a feasible alternative (unless the alternative is modified according the criteria where the local threshold is being surpassed). 6. calculating the local threshold by gravity center as previously discussed, there are other methods available to calculate the different lts of a model. the geometric calculation method (based on the gravity center calculation) is interesting because of its similarity to equation (1) used to calculate the rate of change. in this case, the lt value is obtained through the geometric calculation of the gravity center over the transformation function. figure 2 shows a curve that is a product of a relatively simple geometric calculation and can be approximated by a straight line to the moderate value and then another straight line with less slope to the low value as shown in figure 4. level high moderate low very low value 1,0 0,5524 0,1736 0,0847 figure 4 example of transformation function source: fulcrum engineering the ordinate (height) of the gravity center (gc) of a triangular figure is located at one-third of the height, thus: ijahp article: garuti/how to obtain local and global thresholds in ahp/anp international journal of the analytic hierarchy process 196 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.802 h1 = (1 – 0.1736) /3 = 0.2755 (height). a1 = ½ (2x0.8264) = 0.8264 (area) h2 = (0.1736 – 0.0847) /3 = 0.0296 (height). a2 = ½ (2*0.0889) = 0.0889 (area) applying the known equation of gc: gc = i (hi*ai) / ai = (0.2755 x 0.8264 + 0.0296 x 0.0889) / (0.8264 + 0.0889) = 0.2516. it is interesting to note how close the value calculated by rate of change of equation (1) (0.2642) is to the value calculated by gc (0.2516); the difference is less than 5%. the similarity of these two values is not a coincidence; it is because equation (1) for the rate of change resembles the gc formula. therefore, it corresponds to how much more moderate weighs in relation to low in relative terms, where the ep is located, or the gc of the moderate and low levels. in this case, moderate weighs 3 times more than low (exactly: 3.182), which in terms of gc, generates the formula: ul = (1xm + 3xl) / (1 + 3) or in its exact version: ul = (1xm + 3.182xl) / (1 + 3.182) in graphic terms, this is visualized as: figure 5 rate of change vs. gravity center source: fulcrum engineering it is important to recall that the geometric calculation is problematic. when the function is transformation ratio function terminal criterion: low education level 1,0 0 0,9 0 0,8 0 0,7 0 0,6 0 0,5 0 lt calculated by rate of change 0,2 0 lt calculated by center of gravity 0,1 0 0,0 h m l v l ijahp article: garuti/how to obtain local and global thresholds in ahp/anp international journal of the analytic hierarchy process 197 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.802 complex (which is generally the case), the possibility of making errors when calculating the gc is significant. in this particular case, approximating the calculation was simple using straight lines (without significant error). however, most of the functions are more complex than the case shown in this example. for this reason, the use of this method is not advised. in addition, the method is highly dependent on the geometric shape of the figure (at a high level of detail) which makes it difficult to program. 7. limitations it is well known that all methods have limitations. the main limitation in this case is the definition and selection of the rate of change functions. the ep functions presented in table 2 are functions that are empirically built using the gc concept as a guide. although these functions have been tested in many projects over the years, they have only been presented as a good starting point for the ep or functions that respect the border conditions (the up and down limit analysis conditions). currently, these ep functions have given particularly good results; however, they need to be further analyzed and tested. this could be the topic for a future study. 8. other applications for the threshold concept it should be noted that the benefits of the gt are worth the process of its construction using the lts. lts are useful tools by themselves and particularly useful in identifying where to improve alternatives and ensuring certain thresholds are satisfied by all alternatives, avoiding compensation. the local and global threshold profiles act as reference points for the highest acceptable risk scenario. these profiles can be used to determine where the greatest gaps (the weighted difference between the current local usdi and lt) are located and generate specific actions/projects to reduce those gaps for the associated criterion. depending on the budget, these actions/projects would reduce the gap for the main lts (or all of them) until the neighborhood projected usdi index complies with the gt. this scheme has the advantage of precisely determining the requirements that the improvement project’s portfolio should fulfill and be able to design effective and efficient neighborhood development plans. as indicated at the end of section 3, the lt application allows the ahp/anp to act as a non-compensatory method by forcing alternatives to follow the lt for every terminal criterion, making the alternative risk acceptable for each criterion. therefore, from all the alternatives that are risk-acceptable under a selection scenario, the alternative with the lowest global risk value should be selected. in a case where no alternative can accomplish all the lts, the following analysis would be helpful. the weighted lts (plotted together) represent the weighted theoretical threshold's profile. this theoretical profile can be compared with the alternative’s profile in terms of compatibility, using the compatibility index g. if both profiles are compatible, the alternative should be acceptable even if one or more lts are not satisfied by the alternative. ijahp article: garuti/how to obtain local and global thresholds in ahp/anp international journal of the analytic hierarchy process 198 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.802 the question may arise, why is this “compensation” through the compatibility index g) acceptable? it is because the topological distance between compatible profiles is close enough to make them topologically similar profiles. finally, although the next description corresponds to a lateral effect (and is a bit complicated), it could be interesting to note that at times (and often unintentionally) some strategic players use the lt value to define a desired future scenario. for example, defining that lt of a disaster risk model must be at the low level of the scale in year 2030. using the definition (using the lt value as the desired scenario) allows the user to reach the highest value for the rate of change where m is much larger than l (m/l  ∞), which corresponds to the maximum comfort (or minimum risk) scenario as shown by the limit of equation (1) in section 5. in this way, the player is moving away from the critical ep that is, moving away from the maximum acceptable stress value against risk and selecting a sub-efficient ep. this example clearly shows the conceptual (and numerical) difference that exists when the lt is used as a desired scenario (which is possible, but not recommended), compared to when the lt is used as a maximum acceptance of change (critical ep) in a risk model, which is how the lt should be used. 9. conclusions and comments this paper has shown how a precise and easy technique can be applied to determine the local and global threshold values of a risks model. this technique could be applied from two-level scales (dichotomous scales) to five-level scales, which is the maximum number of levels normally used in these models. having defined local and global thresholds in the ahp/anp provides a reference point to compare the alternative’s behavior, either a minimum benefit level for a benefits model, or a maximum risk level for a risks model. it is truly important to identify a feasible set of alternatives that will create strategic long-term scenarios. therefore, a decision maker will be able to select from the alternatives that fulfill the gt alternatives. for example, the technique set out in this paper makes it possible to extract the best set of alternatives that meet a minimum profit and a maximum risk level, excluding those that do not provide minimum quality or exceed a maximum risk. the importance of determining these values (lt and gt) is because of the ability to make a decision regarding the behavior of a large set of alternatives by having a reference point to compare the behavior of the alternatives (either a minimum benefit for a benefits model or a maximum risk for a risks model). as established in the introduction, it is very important to know what the feasible set of alternatives is, since it will help create a strategic long-term scenario that is acceptable to any stakeholder (community, government, strategic partners who has participated in the decision-making process. thus, the decision maker will be able to make a better selection of one or more of the alternatives that are minimally acceptable. for instance, to extract the set of best alternatives that stay within a minimum profit and a maximum risk, or in other words, to exclude projects or actions that do not overcome a minimum of quality or exceed a maximum of risk. this process has been used by fulcrum engineering for more than 25 years in its multicriteria consulting processes where it has calculated ep, lt and gt (easily programmed) in multiple studies and projects for the construction of global performance indicators. in particular, it has been used in projects such as the prioritization of investment portfolios for ijahp article: garuti/how to obtain local and global thresholds in ahp/anp international journal of the analytic hierarchy process 199 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.802 sustainable development, disaster risk management, corporate social responsibility, healthy shiftwork systems, environmental impact assessment and in general with respect to the selection of feasible alternatives for both benefits and risk models (see list of projects in appendix). whenever it was possible to compare the results that were obtained with real results, they were very similar. it is important to note that the values obtained for the lts, when possible, should be contrasted with practical examples to determine how closely they align with reality, and the calculations should be adjusted as necessary. this is known as the calibration process and is performed using available cases where the model can be applied. as generally observed, the adjustments required are minor and the initial value (theoretical value) is very close to the final one. there are other alternative mechanisms available to define lts for the terminal criteria of a model as noted earlier, and some are as follows:  experts’ agreement on the possible values for each criterion, usually called the empirical threshold or technical agreement.  application of the median value (widely used in ordinal scales).  more basic mechanisms, such as using the lowest non-zero value on the rating scale of the relevant criterion (minimum threshold or minimum stress). experts’ agreement of values for each criterion is used quite frequently when the correct experts are involved, and when this method is used, it is possible to measure the compatibility of the theoretical and technical threshold profiles. on the one hand, the theoretical threshold profile is calculated with the rate of change method, and on the other, the technical or empirical threshold profile is obtained as an ad-hoc value estimated by the expert for each terminal criterion. a good quality test is to measure the compatibility of the two profiles (theoretical vs. empirical). compatibility of these two profiles may be obtained through the compatibility index g for weighted environments (garuti, 2012). if these two profiles are g-compatible, g>0.85 (better if g≥0.90), then the certainty that the gt value is adequate will be solid. finally, several authors have criticized the ahp/anp and pair comparison approaches based on their “lack of objectivity” and “compensation process.” while this is not the place to discuss this criticism, this paper provides alternative ways to determine the gt that can be applied to derive the best value at the user’s convenience. one must remember that thresholds are not values to be “discovered”, but built and agreed upon by the experts within the team. the best way to confirm that the calculated value is approved by all (which is the basic concept of “objectivity”) is through the calibration process, over a nonhomogeneous set of possible conditions. the value of the threshold is subject to the same assumptions as the whole model; therefore, if the assumptions do not apply at a certain moment and the model needs to be updated, the threshold value should be revisited. the problem with the compensation process has been already explained and is clear how the use of lt, gt and (eventually) the compatibility index may solve this issue. ijahp article: garuti/how to obtain local and global thresholds in ahp/anp international journal of the analytic hierarchy process 200 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.802 references garuti c. (2017). reflections on scale from measurement, not measurement from scales. international journal of the analytic hierarchy process, 9(3), 349-361. garuti, c. (2016). new advances of the compatibility index g in weighted environments. international journal of the analytic hierarchy process, 8(3), 514-537. doi: garuti c. (2017). reflections on scale from measurement, not measurement from scales. international journal of the analytic hierarchy process, 9(3), 349-361. garuti, c. (2016). new advances of the compatibility index g in weighted environments. international journal of the analytic hierarchy process, 8(3), 514-537. garuti, c., solomon v.a. (2011). compatibility indices between priority vectors. international journal of the analytic hierarchy process, 4(2), 152-160. doi: https://doi.org/10.13033/ijahp.v4i2.130 garuti, c. (2016). measuring in weighted environments, from metric to order topology (when close really means close). in: de felice, f., petrillo, a. & saaty, t. (eds.) applications and theory of analytic hierarchy process decision making for strategic decisions, intech open. doi: https://doi.org/10.5772/63670 garuti, c., solomon v.a. (2011). compatibility indices between priority vectors. international journal of the analytic hierarchy process, 4(2), 152-160. doi: garuti, c., solomon v.a. (2011). compatibility indices between priority vectors. international journal of the analytic hierarchy process, 4(2), 152-160. doi: https://doi.org/10.13033/ijahp.v4i2.130 garuti, c. (2016). measuring in weighted environments, from metric to order topology (when close really means close). in: de felice, f., petrillo, a. & saaty, t. (eds.) applications and theory of analytic hierarchy process decision making for strategic decisions, intech open. doi: https://doi.org/10.5772/63670 garuti, c. (2012). measuring in weighted environments, from metric to order topology (when close really means close). in defelice, petrillo, saaty (eds) applications and theory of analytic hierarchy process decision making for strategic decisions. intech open. doi: 10.5772/63670 millet, i. & saaty, t.l. (2000). on the relativity of relative measures – accommodating both rank preservation and rank reversals in the ahp. european journal of operational research, 121, 205–212. doi: http://dx.doi.org/10.1016/s0377-2217(99)00040-5 s orton, a. (1984). understanding ratio of change “mathematics at school”, 13(5), 23-26. saaty, t. l. (1986). absolute and relative measurement with the ahp. the most livable cities in the united states. socio-economic planning sciences, 20(6), 327–331. doi: http://dx.doi.org/10.1016/0038-0121(86)90043-1 saaty t.l. (1988). decision making for leaders. pittsburgh, pa: rws publications. https://ideas.repec.org/b/ito/pbooks/4173.html https://ideas.repec.org/b/ito/pbooks/4173.html https://doi.org/10.5772/63670 https://ideas.repec.org/b/ito/pbooks/4173.html https://ideas.repec.org/b/ito/pbooks/4173.html http://dx.doi.org/10.1016/0038-0121(86)90043-1 ijahp article: garuti/how to obtain local and global thresholds in ahp/anp international journal of the analytic hierarchy process 201 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.802 saaty, t. l., vargas, l. g., & whitaker, r. (2009). addressing with brevity criticism of the analytic hierarchy process. international journal of the analytic hierarchy process, 1(1), 121-134. doi: http://dx.doi.org/10.13033/ijahp.v1i2.53. saaty, t. l., & sagir, m. (2009). an essay on rank preservation and reversal. mathematical and computer modelling, 49(5–6), 1230–1243. doi: http://dx.doi.org/10.1016/j.mcm.2008.08.001. saaty, t. & peniwati, k., (2010). group decision making: drawing out and reconciling differences. pittsburgh, pa, usa: rws publications. salomon v. a. (2016). absolute measurement and ideal synthesis on ahp. international journal of the analytic hierarchy process, 8(3), 538-545. doi:10.13033/ijahp.v8i3.452 yoon, hwang and chin-lai. (1981). multiple attribute decision making: methods and application a state of the art survey. berlin heidelberg: springer verlag. doi: 10.1007/9783-642-48318-9 garuti, c. (2016). measuring in weighted environments, from metric to order topology (when close really means close). in: de felice, f., petrillo, a. & saaty, t. (eds.) applications and theory of analytic hierarchy process decision making for strategic decisions, intech open. garuti, c. (2012). measuring in weighted environments, from metric to order topology (when close really means close). in defelice, petrillo, saaty (eds) applications and theory of analytic hierarchy process decision making for strategic decisions. intech open. doi: 10.5772/63670 millet, i. & saaty, t.l. (2000). on the relativity of relative measures – accommodating both rank preservation and rank reversals in the ahp. european journal of operational research, 121, 205–212. doi: http://dx.doi.org/10.1016/s0377-2217(99)00040-5 s orton, a. (1984). understanding ratio of change “mathematics at school”, 13(5), 23-26. saaty, t. l. (1986). absolute and relative measurement with the ahp. the most livable cities in the united states. socio-economic planning sciences, 20(6), 327–331. doi: http://dx.doi.org/10.1016/0038-0121(86)90043-1 saaty t.l. (1988). decision making for leaders. pittsburgh, pa: rws publications. saaty, t. l., vargas, l. g., & whitaker, r. (2009). addressing with brevity criticism of the analytic hierarchy process. international journal of the analytic hierarchy process, 1(1), 121-134. doi: http://dx.doi.org/10.13033/ijahp.v1i2.53. saaty, t. l., & sagir, m. (2009). an essay on rank preservation and reversal. mathematical and computer modelling, 49(5–6), 1230–1243. doi: http://dx.doi.org/10.1016/j.mcm.2008.08.001. http://dx.doi.org/10.1016/j.mcm.2008.08.001 http://dx.doi.org/10.13033/ijahp.v8i3.452 https://ideas.repec.org/b/ito/pbooks/4173.html https://ideas.repec.org/b/ito/pbooks/4173.html http://dx.doi.org/10.1016/0038-0121(86)90043-1 http://dx.doi.org/10.1016/j.mcm.2008.08.001 ijahp article: garuti/how to obtain local and global thresholds in ahp/anp international journal of the analytic hierarchy process 202 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.802 saaty, t. & peniwati, k., (2010). group decision making: drawing out and reconciling differences. pittsburgh, pa, usa: rws publications. salomon v. a. (2016). absolute measurement and ideal synthesis on ahp. international journal of the analytic hierarchy process, 8(3), 538-545. doi:10.13033/ijahp.v8i3.452 yoon, hwang and chin-lai. (1981). multiple attribute decision making: methods and application a state of the art survey. berlin heidelberg: springer verlag. doi: 10.1007/9783-642-48318-9 http://dx.doi.org/10.13033/ijahp.v8i3.452 ijahp article: garuti/how to obtain local and global thresholds in ahp/anp international journal of the analytic hierarchy process 203 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.802 appendix projects (since 2015) chilean copper corporation (2013-2015). construction of a multicriteria measurement indicator for prioritization of different working days for different mining divisions of the chilean copper corporation (codelco). chile: fulcrum engineering. chilean copper corporation (codelco) (2017). multicriteria model (ahp) to measure the risk level of the new andina mine project. chile: daes environmental consulting co., fulcrum engineering. chilean ergonomics commission (cen) (2015-2018). creation and application of a multicriteria workload evaluation model in chilean companies. super-intendency of pensions. chile: fulcrum engineering. chilean institute of construction (2013). construction of a multicriteria measure index for environmentally sustainable buildings. chile: fulcrum engineering. chilean nuclear energy commission (cchen) (2018). benefits, costs, opportunities and risks analysis of alternatives (bcor) for the evolution of nuclear energy in chile in the next 20 years and its relationship with the different sectors of the country, particularly with the health sector. chile: fulcrum engineering. ministry of housing and urbanism (minvu) (2020-2021). construction of the index of evaluation of urban and housing projects by quality attributes. plus, a software for evaluation of urban and housing projects index (euhp). a multicriteria index based on quality attributes (2020in progress). chile: ministry of housing and urbanism (minvu). chile: fulcrum engineering. ministry of social development and family (msdf) (2019-2020). development of a multicriteria methodology and its application in case studies. chile: fulcrum engineering. organization for german cooperation (giz), public works ministry of mexico, public works ministry of costa rica, public works ministry of chile (2019-2020). creation and application of a multicriteria road criticality index of risk for road projects in mexico, costa rica and chile. chile: fulcrum engineering. ministry of social development of family (msdf) (2018-2019). methodological guide for the construction and facilitation of multicriteria models for risk disaster assessment (with practical workshops on application and computer model). united nation (cepal) chile: fulcrum engineering. ministry of social development and family (2016-2017). creation and application of multicriteria models for project configuration, formulation and evaluation of portfolios of investment initiatives in the context of territorial strategic plans. chile: fulcrum engineering. national emergency office (2018-2019). creation and application of the methodological guide of a multi-criteria assessment model for disaster risk management in 345 municipalities in chile. chile: fulcrum engineering. ijahp article: garuti/how to obtain local and global thresholds in ahp/anp international journal of the analytic hierarchy process 204 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.802 presidential program, “i love my neighborhood”. creation and application of urban and social deterioration index (usdi) for investment portfolio prioritization. (2019-2020). chile: ministry of social development and family (mdsf). plus, the scales manual for urban and social deterioration index (usdi) (2019-2020). chile: ministry of social development and family (mdsf). regional government of araucanía (gore-araucanía) (2015-2016). computer system based on a multi-criteria tool (ahp) for the evaluation of investment portfolios for the regional government of araucanía (chile), through the creation of an evaluation website (system miip2.0). chile: fairware engineering, fulcrum engineering. ijahp article: moons, pintelon, jorissen, de ridder, everaerts/identification of multistakeholder value in medical decision-making international journal of the analytic hierarchy process 82 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 identification of multi-stakeholder value in medical decision-making karen moons ku leuven belgium karen.moons@kuleuven.be liliane pintelon liliane.pintelon@kuleuven.be pieterjan jorissen pieterjan.jorissen@student.kuleuven.be dirk de ridder dirk.deridder@uzleuven.be wouter everaerts wouter.everaerts@uzleuven.be abstract the recent trend is towards value-based healthcare, which is characterized by the relationship between quality of care and the associated costs. however, value in healthcare is a largely unmeasured and misunderstood concept. in particular, a definition of multi-stakeholder value in healthcare decision-making is missing. in this case study, we review the radical prostatectomy procedure because previous studies have shown discrepancies between what the patient’s value most and what the experts think the patient’s value most. the objective of this research is to address this gap by identifying a multi-stakeholder definition of value which will improve healthcare decision-making. multi-criteria decision-making (mcdm) techniques, more specifically the analytic hierarchy process (ahp), are applied to prioritize the quality of care indicators according to the following six pillars of quality: safety, effectiveness, efficiency, timeliness, patientcenteredness and equity. in addition, the average cost of a radical prostatectomy is estimated. a distinction is made between three treatment stages in order to calculate the overall quality score by assigning weights to criteria in each treatment stage. according to the ahp weights, the pillars of effective, patient-centered and equitable care contribute the most to value creation. finally, some of the challenges of mcdm studies are addressed, such as conflicting preferences between stakeholder groups. in conclusion, this case study stresses the need to adapt operational research methods and knowledge to be applied in value-based healthcare. the ahp is a suitable approach to address the needs of healthcare decision-makers, to set priorities, and to identify value improvement opportunities while considering all of the stakeholders involved in the full care cycle. keywords: value-based healthcare; analytic hierarchy process; multi-stakeholder decision-making mailto:karen.moons@kuleuven.be mailto:liliane.pintelon@kuleuven.be mailto:pieterjan.jorissen@student.kuleuven.be mailto:dirk.deridder@uzleuven.be mailto:wouter.everaerts@uzleuven.be ijahp article: moons, pintelon, jorissen, de ridder, everaerts/identification of multistakeholder value in medical decision-making international journal of the analytic hierarchy process 83 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 1. introduction since the 1990s, the healthcare sector has changed rapidly due to the need to deliver efficient and effective healthcare services because of increased competition and the growing influence of patients. increasing pressure to decrease costs and improve the quality of patient care is driving healthcare organizations towards adopting more efficient healthcare delivery processes. however, measuring the performance of these processes is complicated due to the complex nature of healthcare organizations and the involvement of multiple stakeholders. performance management in both the industrial and service sectors provides a competitive advantage as it enables organizations to control their supply chain strategy, implement continuous improvement programs and improve decision-making by focusing on achieving strategic, tactical or operational goals in the most efficient way (maestrini et al., 2017). in a healthcare context, however, slow progress in performance improvement is a result of the existence of many performance definitions among the various stakeholders. each stakeholder has their own, possibly conflicting goal for efficient performance management, including high quality, cost containment, safety, access to services, patient-centeredness or satisfaction. according to porter (2010), creating high value for patients should be the main goal of any healthcare organization. value improvement reflects the shared goal that unites the interests and activities of patients, payers, care providers and suppliers. value is defined as health outcomes achieved per dollar spent, and therefore value improvement also addresses cost containment in hospitals (mukherjee, 2008). value-based healthcare with the goal of achieving high-quality patient care at the lowest possible cost is a hot topic in today’s research. however, until now care decisions have often been based only on the hospital’s perspective, and therefore ignore the patient’s preferences (forbes, hebb & mu 2018). this work considers the medical services that are provided to patients who are diagnosed with prostate cancer. prostate cancer is the most common form of cancer observed among men (richman et al., 2005; roth, weinberger & nelson, 2008). this study focused on one treatment option, namely surgery that is also referred to as (robot-assisted) radical prostatectomy. a multi-stakeholder definition of value is missing for this procedure. value measurement should encompass all of the provided services that aim to meet the desired patient outcomes. in the case of a radical prostatectomy, this involves three stages of the treatment including the pre-operative, per-operative and post-operative stage. value creation in the post-operative stage mainly refers to longer-term costs and patient outcomes, such as the need for ongoing interventions or sustainable recovery (institute of medicine (iom), 2006). current measures in healthcare organizations cover outcomes for single departments or a whole hospital rather than patient outcomes for the full care cycle. value measurement over the full care cycle is needed in order to realign reimbursement practices with value so that payments cover the full care cycle as a reward for value creation. in belgium, a reform of the hospital financing system aims to allocate resources to hospitals according to the perceived quality of care (stephani et al., 2018). however, the definition of quality of care is unclear and every hospital claims that they have the best quality of care for specific medical disciplines. this work stresses the need to rigorously measure performance using operational research tools in value-based prostate cancer care by clearly defining quality of care and determining the associated costs, while considering all of the stakeholders involved in the full radical prostatectomy care cycle. ijahp article: moons, pintelon, jorissen, de ridder, everaerts/identification of multistakeholder value in medical decision-making international journal of the analytic hierarchy process 84 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 high quality of care is defined collectively by multiple outcomes. the institute of medicine (iom) (2001) defines six dimensions of quality of care, namely safe, effective, patient-centered, timely, efficient and equitable care. however, there may be multiple competing patient outcomes (e.g. near-term safety versus long-term functionality). relevant quality indicators are identified to measure value creation by relating quality of care to the associated costs. the objective of this work is to map the trade-offs between the patient outcomes, set priorities and construct a ranking of the quality criteria according to their relative importance, depending on each stakeholder’s perspective. this is a problem including multiple criteria which necessitates the use of mcdm techniques. in this study, different mcdm methods are described and their healthcare applications are discussed. the ahp is selected as the most suitable for solving this ranking or prioritization problem. this research mainly contributes to the literature by incorporating multiple stakeholders’ views to determine value in healthcare, whereas other studies only focus on the views of surgeons or patients (de bekker-grob et al., 2013; maclennan et al., 2015; marsh et al., 2014a; martin et al., 2015). the stakeholders included in this study are patients, urologists, nursing staff, hospital management, general practitioners and payers. in addition, the authors propose solutions to address typical challenges in mcdm studies, such as inconsistencies, knowledge gaps, conflicts between stakeholders or uncertainty. finally, the ranking of the criteria is combined with the associated costs of radical prostatectomy resulting in a value ratio that provides a guideline to identify opportunities for value improvement. 2. literature review this research investigates the application of mcdm techniques to identify a multistakeholder value definition by addressing the needs of healthcare decision-makers. value-based healthcare is a hot topic in the literature and will be discussed below. a brief overview is presented of different prostate cancer treatments. in addition, the literature is reviewed to determine the most suitable mcdm method and typical mcdm applications in healthcare are given. 2.1 value-based healthcare a shift in focus from volume to value is needed in the healthcare sector and is described in porter and lee (2016) as the movement from ‘fee-for-service’ to ‘value-based’ payment models. in the former model, hospitals and physicians are paid based on the number of delivered services, while payments in the latter model are based on patient health outcomes. all of the actors involved in healthcare delivery benefit from a valuebased model because of lower costs, higher patient satisfaction, reduced risks, reduced spending, etc. the transformation to value-based healthcare requires restructuring how healthcare delivery is organized, measured and reimbursed (porter & lee, 2013). porter (2010) advocates adopting value as the most important objective in healthcare performance management in order to obtain efficient healthcare systems while maximizing patient care quality. value in value-based healthcare is defined as “the health outcomes achieved that matter to patients relative to the cost of achieving those outcomes” (porter & lee, 2013). however, value in healthcare is hard to measure and interpret which makes it challenging to define value in a healthcare setting. therefore, the definition of value requires clarification. the first component in this definition refers to health outcomes. health outcomes are condition-specific and multi-dimensional, such that no single outcome captures the results of care (porter, 2010). furthermore, outcomes ijahp article: moons, pintelon, jorissen, de ridder, everaerts/identification of multistakeholder value in medical decision-making international journal of the analytic hierarchy process 85 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 should include both patients’ near-term and longer-term health circumstances in order to encompass the ultimate results of care and sufficient measurement of risk factors. porter (2010) proposes a hierarchy for ordering patient outcomes as follows: level 1 = health status achieved or retained, level 2 = recovery process, and level 3 = sustainability of health. costs make up the second component in the value definition, and refer to the total costs of the full care cycle rather than individual services (porter, 2010). porter (2010) introduces ‘integrated practice units’ which cover the full care cycle for patients with similar symptoms or medical conditions in order to avoid healthcare providers tendency to focus on individual services for treatment. a value-based healthcare delivery system helps hospital managers measure both outcomes and costs. although monitoring costs allows for improving efficiency in healthcare, focusing only on cost reduction can possibly lead to low-quality effective care. therefore, outcomes must be considered relative to costs. lee et al. (2016) present a value-driven approach and show that achieving better quality at a lower cost is possible. first, treatment costs are identified based on actual patient use or time spent in each treatment of the full care cycle. next, an opportunity index is calculated by multiplying the coefficient of cost variation with the total direct cost. highly variable and high cost factors offer the greatest opportunities for value improvement. in addition, thresholds are determined for quality or outcome indicators to compute a perfect care index. the objective is to strive for a perfect care index of 100%, which means that the threshold is met for every aspect of the treatment. the final step combines the perfect care indicators with cost variability into a quality over cost ratio to improve care by reducing costs and/or improving quality. lee et al. (2016) showed an 11% reduction in costs with improved health outcomes for total joint replacement. from a cost perspective, a justified estimation of the average cost of the treatment (radical prostatectomy in this case study) is required to determine value. this is challenging due to a lack of accurate cost information or information on how costs relate to the outcomes achieved (porter & lee, 2013). time-driven activity-based costing is a suitable technique for understanding different types of costs, and allows for substantial cost reductions without negatively affecting outcomes (kaplan & porter, 2011). the literature discusses direct and indirect costs that should be taken into account when determining a base-case average cost. typical cost drivers include operating room cost, medical supply cost, surgeon professional fees, hospital room cost, pharmaceuticals and medication cost, investment costs for medical equipment, laboratory cost, etc. (hayes et al., 2013; hyams et al., 2012; lotan, cadeddu & gettman, 2004; swartenbroekx et al., 2012). 2.2 prostate cancer treatments prostate cancer is the most common form of cancer observed among elderly men in belgium, though it has a high survival rate of 97% (belgian cancer registry, 2017). typically, prostate cancer occurs in three phases, and the aggressiveness of the tumor behavior increases with each phase from a localized cancer to metastatic cancer. in this section, three treatment options are briefly presented for localized prostate cancer as follows: expectant management, surgery and radiation therapy (litwin & tan, 2017; prostate cancer foundation, 2014): ijahp article: moons, pintelon, jorissen, de ridder, everaerts/identification of multistakeholder value in medical decision-making international journal of the analytic hierarchy process 86 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682  expectant management is a treatment for patients with low-risk prostate cancer (e.g. prostate-specific antigen (psa) is less than 10ng/ml) in order to monitor tumor behavior while not undergoing definitive therapy. watchful waiting is suggested for palliative patients, whereas active surveillance intends to cure patients who develop significant disease through psa testing or prostate biopsies. the major benefit is that patients do not perceive side effects with regard to their physical functioning.  surgery or radical prostatectomy involves removing the prostate and seminal vesicles. open radical prostatectomy is the traditional procedure characterized by an incision in the lower abdomen. this technology has been widely replaced with robotic radical prostatectomy, which requires smaller incisions for inserting the surgical robot’s arms. although recovery time is observed to be lower for the latter type of surgery, the choice depends on the preference of the urologist. typical side effects of surgery are urinary incontinence or erection problems.  radiation therapy is used for killing or damaging the dna of the localized or locally advanced prostate cancer cells with ionizing radiation or photons. technological advancements introduce non-uniform radiation beams to reduce impact on the surrounding tissues in order to minimize side effects, such as bowel or hormonal problems. the patient chooses the preferred treatment with assistance from experts or evidencebased decision support tools. research shows that patients often prefer surgery, even though experts recommend alternative prostate cancer treatments (roth et al., 2008). in the remainder of this work, we focus on radical prostatectomy treatment. patient preferences and value for prostate cancer treatments are studied in the literature (loeb, 2016), but discrepancies remain between what patients value most and what experts think that patients value most. the increasingly central role of patients suggests an emerging role for shared medical decision-making in treatment selection (litwin & tan, 2017). 2.3 mcdm in healthcare the healthcare sector is characterized by complexity, uncertainty and conflicting interests between multiple stakeholders, which makes mcdm a suitable approach to evaluate different quality criteria while taking into account multiple stakeholders’ perspectives (adunlin, diaby & xiao, 2015; forbes et al., 2018; lee, donaldson & cook, 2003). mcdm is defined as “any method that establishes criteria, weights them in terms of importance, and scores each alternative on each criterion to create an overall assessment of value” (marsh et al., 2014a). this approach is especially useful for making consistent, rational and transparent decisions for complex problems with possibly conflicting perspectives (thokala et al., 2016). mcdm techniques can be divided into three categories including value measurement, outranking and goal programming. value measurement methods evaluate alternatives based on a single overall value, such that alternative a is preferred to alternative b whenever the value for a is higher. this value can be estimated by multiplying the performance score of the different alternatives with the weights assigned to each criterion (marsh et al., 2014b). a second method uses an outranking approach for comparing alternatives. it relies on the dominance principle and does not require any numerical value to select the best alternative. an alternative is dominant when it is preferred for ijahp article: moons, pintelon, jorissen, de ridder, everaerts/identification of multistakeholder value in medical decision-making international journal of the analytic hierarchy process 87 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 important criteria, whereas dominated alternatives are eliminated (thokala et al., 2016). goal programming is the third approach used to measure alternatives by computing distance from the ideal and most negative results. however, not all mcdm methods are recommended for solving any decision problem. in this research, the goal is to prioritize the quality criteria. therefore, the nature of the decision problem does not allow the application of goal programming methods since we are interested in identifying the most ideal result instead of using it as input. precise values for criteria weights are essential and because of this the value measurement approach is preferred to the outranking method. value measurement methods are the most applied mcdm methods in healthcare, and some typical examples are the ahp, analytic network process (anp), paprika and multi-attribute utility (value) theory (mau(v)t) (marsh et al., 2014a). typically, there are six main components in a value measurement approach that include the definition of the problem statement, criteria identification, performance measurement, alternatives scoring, criteria weighting and calculation of aggregate scores. please see zardari et al. (2015) and marsh et al. (2017) for more information on the theoretical foundations and different weighting methods that support mcdm. the ahp is the most suitable method for this problem because it allows simplification of the complexity inherent in a hospital system by integrating qualitative and quantitative criteria, and it is able to deal with multiple conflicting stakeholder objectives. moreover, ahp is a relative approach, using pairwise comparisons, and ensures transparent and systematic decision-making. the ahp simplifies the complex decision problem by forming a hierarchy, which consists of a goal, objectives and criteria (saaty, 1990a, 2008). stakeholders are asked to pairwise compare criteria on a 1-9 ratio scale. based on the stakeholder judgments, local priority vectors are extracted to calculate criteria weights. pairwise comparisons are perceived to be easier than comparing all of the criteria at once, and increase the involvement of the decision-maker when prioritizing quality indicators. in addition, a consistency ratio is included to ensure the quality of the judgments. furthermore, ahp is a powerful tool for decision-making because of its flexibility and ability to capture both quantitative and qualitative criteria (hariharan et al., 2004). for these reasons, ahp is used in the majority of cases to generate weights for different criteria in healthcare applications (marsh et al., 2017). however, decisionmakers should also pay attention to the limitations and challenges when performing an mcdm study. these challenges include rank reversal, interdependency, conflicting preferences, uncertainty, etc. mcdm has a few applications in healthcare. schmidt et al. (2015) systematically reviewed ahp applications in five domains of healthcare research, namely the development of clinical guidelines, healthcare management, government policy, shared decision-making and biomedical innovation. marsh et al. (2014a) also distinguished six types of mcdm applications in healthcare, ranging from hospital selection to prioritization of patients for access to health services. more specifically, in prostate cancer treatments, richman et al. (2005) applied ahp as a decision-making tool to produce individual, rational and clinically appropriate decisions without physician bias. 3. methodology the movement towards value-based healthcare pushes hospitals to restructure organization, measurement and reimbursement of care delivery processes. according to ijahp article: moons, pintelon, jorissen, de ridder, everaerts/identification of multistakeholder value in medical decision-making international journal of the analytic hierarchy process 88 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 porter (2010), the overarching goal for any stakeholder must be to maximize value for patients by achieving the best outcomes at the lowest cost. value improvement provides a competitive advantage to any healthcare organization and its stakeholders. however, stakeholders often have conflicting goals for measuring the quality of care. moreover, today’s decision-making is typically based on the hospital’s perspective, and therefore ignores intangible variables such as the patient’s perceived value (forbes et al., 2018). this research was conducted at the university hospital uz leuven (belgium) during the first half of 2018. the hospital specializes in several medical disciplines, but we focus on the urology department, and more specifically (robot-assisted) radical prostatectomy treatment. the decision problem was to identify a definition for value in this treatment, depending on multiple, possibly conflicting perspectives of the stakeholders. value is defined as the ratio of quality over cost. first, relevant quality indicators were identified and prioritized using the ahp. the ahp is a transparent tool that supports decisionmakers by visualizing quality criteria in a hierarchy structure and assigning weights to these criteria (saaty & vargas, 2006). the pairwise comparison process was conducted with all of the stakeholders that are involved in the treatment. in total, 33 patients, 5 urologists, 3 members of the nursing staff, 2 representatives of hospital management, 2 general practitioners and 1 representative of the health insurance sector participated in this case study. based on the multiple stakeholder judgments, different criteria were assessed to determine what constitutes quality of care in this setting. in the second step, the key quality criteria were compared with the associated cost factors to determine the best opportunities for improving value. 3.1 ahp in medical decision-making in the first step of the ahp methodology (saaty, 1990a), the decision problem was decomposed into a hierarchical structure. the overall goal is shown at the top of the hierarchy, namely identifying the multi-stakeholder definition of value in prostate cancer treatment. at the second level, quality and costs were determined to be the objectives to measure value. for each objective, several criteria were identified from the literature review and discussion with a number of the stakeholders. figure 1 shows the hierarchy structure to determine value in prostate cancer treatment. an overview of the selected criteria, abbreviations and their definitions can be found in appendix a. ijahp article: moons, pintelon, jorissen, de ridder, everaerts/identification of multistakeholder value in medical decision-making international journal of the analytic hierarchy process 89 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 figure 1 ahp hierarchy structure the second step of the ahp uses pairwise comparisons to extract each stakeholder’s preferences in order to prioritize the quality criteria. a maximum of 13 quality-related criteria were included in the hierarchy in order to satisfy the time constraint for judging pairwise comparisons. value measurement should encompass all of the provided services that aim to meet the desired patient outcomes. in the case of radical prostatectomy, this involves the three stages of the treatment that include pre-operative, per-operative and post-operative. the quality criteria are divided into three groups depending on the treatment stage. the pre-operative stage includes criteria that are relevant for measuring services before the surgery, such as psa-level, gleason score and clinical stage. in the second stage, the per-operative criteria measure the quality of the surgical procedure, and whether or not it is assisted by a robot. these criteria involve total time spent in the operating room, surgical margins and the necessity of blood transfusions. finally, the post-operative stage refers to longer-term patient outcomes that are a result of the treatment, such as urinary incontinence, erectile dysfunction and time to return to normal functioning. this categorization of the criteria allows them to be compared within the same treatment stage, but ignores interdependencies between different treatment stages. according to experts, it is difficult to compare criteria at different stages because of the different nature of the criteria (e.g. operative factors versus operation outcomes). the ahp is a suitable method because it ignores these interdependencies. the weights for the quality criteria were determined using pairwise comparisons. each stakeholder expressed his preference by comparing two criteria on a ratio scale from 1 to 9, with 1 representing equal importance and 9 extreme preference for one criterion. an example of a pairwise comparison between erectile dysfunction and urinary incontinence is shown in figure 2. the obtained ratios were inserted into a pairwise comparison matrix for all of the criteria. from this matrix, the weights for the respective criteria were calculated via the principal eigenvector method (saaty, 1990a). the resulting weights represent a measure for the relative importance of the quality criteria. ijahp article: moons, pintelon, jorissen, de ridder, everaerts/identification of multistakeholder value in medical decision-making international journal of the analytic hierarchy process 90 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 figure 2 pairwise comparison between erectile dysfunction and urinary incontinence based on the weights assigned to each criterion in the separate treatment stages, the overall quality score can be calculated. the iom (2001) defines six objectives of quality of care, namely safe, effective, patient-centered, timely, efficient and equitable care. the criteria were reallocated to these pillars of quality in order to derive the final weights for the quality aspects. the quality objectives as defined by the iom (2001) are discussed in table 1. in addition to the quality indicators, information on cost types was required to identify the definition of value for a radical prostatectomy. in this case study, the average cost of the different aspects of the treatment was estimated based on the approach suggested by the belgian health care knowledge center (kce) (swartenbroekx et al., 2012). the cost components included the operating room, investment, surgeon wage, hospital stay, medication and overhead costs. figure 1 shows the hierarchy with the quality criteria and cost components. lee et al. (2016) proposed a value-driven method to identify value improvement opportunities. table 1 overview of six dimensions of quality of care according to iom quality objective description criteria safe avoiding injuries to patients from the care that is intended to help them complications blood transfusions effective providing services based on scientific knowledge to all who could benefit and refraining from providing services to those not likely to benefit (avoid overuse and underuse, respectively) surgical margins surgeon experience additional therapy patientcentered providing care that is respectful of and responsive to individual patient preferences, needs and values, and ensuring that patient values guide all clinical decisions urinary incontinence erectile dysfunction time to return to normal functioning timely reducing waiting time and sometimes harmful delays for both those who receive and those who provide care waiting time is not included in this study efficient avoiding waste, including waste of equipment, supplies, ideas and energy total time in operating room length of stay equitable providing care that does not vary in quality because of personal characteristics such as gender, ethnicity, geographic location, and socioeconomic status psa level gleason score clinical stage ijahp article: moons, pintelon, jorissen, de ridder, everaerts/identification of multistakeholder value in medical decision-making international journal of the analytic hierarchy process 91 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 4. results this section describes the results of applying ahp to the value measurement prioritization problem in medical decision-making. value is defined as the ratio of quality over cost. for every stakeholder group, ahp-based priority vectors were computed, which contained weights for the quality criteria in three treatment stages. typical problems such as inconsistencies in the pairwise comparisons, conflicts between actors or knowledge gaps are addressed. the criteria weights were reassigned to the iom-based hierarchy, and represented the six dimensions of quality of care. in addition to quality, cost components were analyzed in order to estimate the average cost of radical prostatectomy treatment. finally, a multi-stakeholder definition of value was determined by combining the results from the quality and cost criteria into a value ratio, and opportunities for value improvement are identified. 4.1 prioritization of quality of care criteria value measurement requires a clear definition of quality of care. the ahp is used to determine the weights that represent the relative importance of several quality criteria while considering the possibly conflicting interests of different stakeholder groups. in the ahp, pairwise comparisons are used to elicit preferences for the included criteria. due to the incomparable nature of the three treatment stages in radical prostatectomy, namely pre-operative, per-operative and post-operative, the criteria were split up and compared accordingly (see figure 1). as an example, we discuss the prioritization of the quality criteria according to 11 pre-operative patients. the same procedure was applied to other stakeholder groups. a priority vector, containing the weights for each criterion, for each pre-operative patient was obtained based on the pairwise comparison matrix. equation 1 presents the original pairwise comparison matrix pwo and priority vector wo for the criteria involved in the pre-operative treatment stage. more information on deriving weights according to the principal eigenvector method can be found in saaty (1990). 𝑃𝑊𝑂 = ( 𝑃𝑆𝐴 𝐶𝑆 𝐺𝑆 ) ( 1 9 1 7 1 9 7 1 5 1 5 1)  𝑤𝑂 = ( 0.2344 0.0606 0.7050 ) (1) 𝐶𝑅𝑂 = 0.7276 the consistency of this matrix was checked by calculating the consistency ratio (cr). this cr should be lower than 10% to ensure qualitative judgments. in this example, the cr exceeded the threshold value and therefore further computations were required. the weighted consistency method by jarek (2016) was applied to reduce the inconsistency. in this approach, the pairwise comparison matrix pwo is adapted by multiplying the priority vector wo with the reciprocal of its transpose (𝑤𝑂 ∗( 1 𝑤𝑂 ) 𝑇 ). this new matrix takes into account the original stakeholder judgments by taking the geometric mean of the original and adapted matrix. this process continues until the cr constraint is satisfied (i.e., cr is lower than 10%). equation 2 contains the new pairwise comparison matrix pwn and the corresponding priority vector wn. ijahp article: moons, pintelon, jorissen, de ridder, everaerts/identification of multistakeholder value in medical decision-making international journal of the analytic hierarchy process 92 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 𝑃𝑊𝑁 = ( 𝑃𝑆𝐴 𝐶𝑆 𝐺𝑆 ) ( 1 5 1 4 1 5 4 1 8 1 8 1)  𝑤𝑁 = ( 0.2370 0.0643 0.6986 ) (2) 𝐶𝑅𝑁 = 0.0904 the resulting ahp-based priority vector, which contains weights for every criterion in the different treatment stages, was calculated for each stakeholder in each stakeholder group. whenever the cr exceeded its limit, the weighted consistency method was applied. the next step was to determine one ranking for the stakeholder group by integrating the priority vectors of the 11 pre-operative patients. however, different experiences during the treatment or personal characteristics could cause conflicts within the same stakeholder group. an average would not satisfy any stakeholder. song and hu (2009) proposed using the cluster similarity approach to combine stakeholders’ conflicting views within one stakeholder group. the degree of similarity is calculated for all of the vector combinations by determining the distance between two vectors dij. this distance value ranges between 0 and 1, indicating the degree of similarity between the vectors. for example, pre-operative patient 3 and 5 have a similarity degree as seen in equation 3. 𝑑35 = 𝑤3𝑤5 ‖𝑤3‖‖𝑤5‖ = ∑𝑤3𝑤5 √∑𝑤3 2 √∑𝑤5 2 = 0.7638 (3) the distance values for all of the vector combinations were combined into a similarity score matrix, which was used for deciding on the cluster groups. a threshold value of dij equal to or greater than 0.75 was used to decide if the two vectors belonged to the same cluster. this step was iterated for every row in the matrix and resulted in a final cluster grouping. next, the cluster weights were determined based on the cluster size. a bigger cluster received a higher weight since it represented more stakeholders and would therefore have a higher impact on the result. the cluster weights are calculated as follows in equation 4. 𝑤(𝐶1) = 𝑆1 𝑆1 2 +𝑆2 2 = 8 82 +32 = 0.1096 (4) 𝑤(𝐶2) = 0.0411 # this assumes that the pre-operative patients are divided into two clusters c1 and c2, with respective cluster sizes s1 = 8 and s2 = 3. cluster c1 contains the most stakeholders and therefore has the greatest weight. finally, one ranking of quality criteria was defined, which reflected the preferences of the stakeholder group. the weighted arithmetic mean (wam) was used to multiply the cluster weights w(ci) and the priority vectors wi in the respective cluster ci, and sum for all of the clusters. the final ranking for pre-operative patients is shown in table 2. ijahp article: moons, pintelon, jorissen, de ridder, everaerts/identification of multistakeholder value in medical decision-making international journal of the analytic hierarchy process 93 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 table 2 ranking of quality criteria according to 11 pre-operative patients psa cs gs tor sm se bt 0.0943 0.0606 0.1251 0.0409 0.1267 0.2491 0.0799 los comp ui ed ttrf at 0.0185 0.0650 0.0612 0.0275 0.0274 0.0237 this priority vector was calculated based on the hierarchy shown in figure 1, which divides the criteria into the pre-, perand post-operative treatment stages. however, the main goal was to determine a definition for quality of care. therefore, the criteria weights were reassigned to the corresponding pillars of quality according to the iom (2001). table 3 presents the resulting quality weights according to the pre-operative patients. table 3 weights for six quality dimensions: pre-operative patient group vs. all stakeholders quality of care weight (pre-operative patients) weight (all stakeholders) equitable 0.28 0.214 timely not available not available effective 0.3995 0.3968 efficient 0.0595 0.0497 safe 0.145 0.1337 patient-centered 0.116 0.2144 for each stakeholder group, the quality criteria were prioritized as described above. the final goal was to combine all of the stakeholders’ viewpoints into one priority vector that contained the quality criteria weights. however, the obtained quality weights show conflicts between stakeholder groups. in addition, not every stakeholder group is equally important to the decision-making process. therefore, we needed a weighting method to determine the weights of each stakeholder group in order to define the overall ranking of the quality criteria that represent all of the stakeholder groups. a similar approach was used as before when tackling conflicts within the stakeholder groups, namely the wam based on cluster similarity that was used to compute the overall priority vector. the seven stakeholder groups were assigned to clusters based on their degree of similarity, as explained in equation 3. three clusters remained, and the cluster weights were computed according to equation 4, and wam was applied to derive the final weights. a reallocation of the weights was required to determine the quality according to the iom definition of quality of care (see table 3). from the table, we find that the pillars effectiveness, patient-centeredness and equity mostly contribute to high-quality care. effective care is represented by the high weights attributed to surgical margins and surgeon experience, whereas equity and patient-centeredness are achieved by focusing on gleason score and limiting post-operative consequences such as urinary incontinence or additional therapy, respectively. 4.2 cost estimation in radical prostatectomy in this work, the average costs for the different aspects of radical prostatectomy treatments at the hospital under study were estimated based on the approach suggested by ijahp article: moons, pintelon, jorissen, de ridder, everaerts/identification of multistakeholder value in medical decision-making international journal of the analytic hierarchy process 94 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 the kce (swartenbroekx et al., 2012). the kce manual provides a generic methodology that is applicable to all belgian hospitals. at uz leuven, 170 radical prostatectomy procedures are performed each year. at our facility, the average length of stay (los) is 3.05 days and the average duration of the surgical procedure (tor) is 2 3 hours, which is similar to the estimated 184 minutes suggested by niklas et al. (2016). the direct costs are related to different aspects of the treatment, including the cost of the hospital stay, physician salaries, investment costs for the robot, medication cost and the operating room cost. in addition, overhead costs are included for heating, facility, maintenance, cleaning and other indirect costs. the total cost for a radical prostatectomy is estimated to be €6684 (7316 usd). 4.3 multi-stakeholder value in radical prostatectomy the final step in identifying a multi-stakeholder definition of value involves combining the obtained results for the quality criteria and cost components to find the best opportunities for improving value. lee et al. (2016) suggest a value-driven approach to address the needs of healthcare decision-makers. the methodology starts with identifying costs related to different aspects of the treatment and determining the average cost of a radical prostatectomy, as mentioned in section 4.2. next, the costs were allocated to the corresponding quality aspects of the treatment. in the second step, an opportunity index was used to identify potential areas for cost reduction. the opportunity index was calculated based on the variability of the costs. cost variability is derived by dividing the standard deviation by the mean of the different types of costs. this is essential to identify aspects of the treatment that have a high cost or high variability. due to the limited availability of data, however, the cost variability cannot be derived and will be considered in future work. the third step as proposed by lee et al. (2016) focused on the key quality indicators needed to achieve perfect care. the thresholds for the perfect care indicators were determined with binary variables. for example, the threshold value for surgeon experience is 100 cases. the binary variable equals 1 if the surgeon has performed more than 100 cases and 0 if otherwise. reaching 100% for all of the indicators would result in perfect care, and hence care with a higher value. in the final step, quality and cost information were combined in order to select the best opportunities to improve value. the most important quality criteria, as identified in table 3, were linked to the respective cost types. for example, the length of stay (los) is influenced by the cost of the hospital stay and medication costs, while urinary incontinence (ui) is influenced by investment costs in medical equipment, operating room costs and physician salary. by analyzing the relationships, the opportunities for value improvement were identified for the criteria with high weights and a high opportunity index. due to the absence of appropriate cost data, the opportunity index cannot be calculated. therefore, another approach was used to calculate the ratio of the quality criteria and the estimated costs related to this aspect of the treatment. this ratio indicates the relative importance of a quality criterion per euro. for example, the value ratio of surgical margins equals 0.42, which represents a high relative importance per euro, and therefore a good opportunity for value improvement. table 4 gives an overview of the value ratios for each of the key quality indicators. ijahp article: moons, pintelon, jorissen, de ridder, everaerts/identification of multistakeholder value in medical decision-making international journal of the analytic hierarchy process 95 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 table 4 value ratio based on quality weight and cost criteria quality weight cost (€) value ratio sm 0.1586 3782 0.42 se 0.1458 3782 0.39 ui 0.0971 3782 0.26 at 0.0925 3782 0.21 comp 0.0873 4084 0.26 ttrf 0.0605 4084 0.15 ed 0.0568 3782 0.15 los 0.0178 676 0.24 5. discussion value-based healthcare is characterized by the relationship between quality of care and associated costs. the objective of this research was to identify a definition for value in (robot-assisted) radical prostatectomy that was dependent on multiple, possibly conflicting, stakeholder preferences. the stakeholders included in this study were patients, urologists, nursing staff, hospital management, general practitioners and payers. the ahp is the most suitable method for solving a prioritization problem to address the needs of healthcare decision-makers. however, a few challenges must be overcome to gain the full potential of the ahp methodology. 5.1 challenges of ahp inconsistencies can occur due to limitations of knowledge or the complex nature of the decision problem. saaty (2008) allows for 10% of inconsistent judgments in order to achieve qualitative results. however, when this threshold is exceeded, a method is needed to reduce the inconsistency. in this work, the weighted consistency method by jarek (2016) was applied to reduce the inconsistency without changing the stakeholder’s judgments. on the other hand, knowledge limitation can lead to incomplete pairwise comparisons. these gaps can be filled by taking the geometric mean of other stakeholder judgments for the respective criterion (hua, gong & xu, 2008). the goal was to find a multi-stakeholder definition of value for improving medical decision-making. however, stakeholders often have conflicting views, which may lead to discrepancies in the value ranking. reaching a consensus is often believed to be the best approach to balance conflicting views (dyer & forman, 1992). in this work, we applied a cluster similarity approach, as proposed in song and hu (2009), to deal with conflicts between and within stakeholder groups. the results show differences between preoperative patients and post-operative patients. the former group values effective and equitable care (e.g. psa, gs, se) more than the latter group, who focus mainly on the quality of outcomes (e.g. ttrf, ui, at) and therefore assigns higher weights to the postoperative, patient-centered criteria. this difference can be explained by taking into account the situation of the patient. one has already been through all of the treatment stages, and the other is just starting the treatment. for example, the psa-level received a high priority from the pre-operative patients, whereas the other stakeholders gave it lower weight. although the psa has few medical implications for further treatment outcomes, this criterion is often misunderstood by pre-operative patients, and therefore more ijahp article: moons, pintelon, jorissen, de ridder, everaerts/identification of multistakeholder value in medical decision-making international journal of the analytic hierarchy process 96 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 information about the implications of the psa level are needed. as shown in figure 3, equitable care is very important to the payers, nursing staff and pre-operative patients as they assigned higher weights to the pre-operative criteria. another difference was observed in the post-operative phase, where the urologists and hospital management attached more importance to patient-centeredness, such as complications (comp) and additional therapy (at) since these indicate how they performed during the per-operative stage. from the results, we also observed some similarities among the stakeholders, such as the low importance of the operation time (tor) and length of stay (los), as these criteria reflect cost factors or efficiency, which are relatively unimportant when compared with criteria that are related to the quality of care. effective care, on the other hand, received high weights from all of the stakeholders involved. surgical margins (sm) are related to the quality of patient outcomes and can predict recurrence (fontenot & mansour, 2013). in addition, the research shows that surgeon experience (se) impacts the patient outcome as well as the probability of complications (fossati et al., 2017; di pierro et al., 2014). in addition, preferences can differ within stakeholder groups due to personal characteristics or previous experiences with medical treatments. a further classification of the preand post-operative patient groups according to their age and level of education would give more insight into the different preferences. the preliminary results showed that age does not necessarily influence preferences since older patients’ assigned similar weights to erectile dysfunction and complications when compared to younger patients. furthermore, the level of education had an influence on the pre-operative criteria, as patients with a lower level of education attached more importance to the psa level than patients with a higher level of education. as described above, a better understanding of this medical term is required. however, further research is required to obtain statistically significant results for a larger sample size. ijahp article: moons, pintelon, jorissen, de ridder, everaerts/identification of multistakeholder value in medical decision-making international journal of the analytic hierarchy process 97 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 figure 3 radar plot showing different stakeholder preferences for quality of care indicators 5.2 limitations this research also has some limitations. due to the small sample size in this case study, the results cannot be validated for all of the stakeholder groups. a larger sample size would lead to more reliable results and allow for stratification to identify preference differences within stakeholder groups. furthermore, the ahp approach does not allow for interdependency between criteria to simplify the problem. however, the relative importance of the treatment stages is not necessarily independent, since poor preor per-operative conditions may cause undesired patient outcomes. moreover, uncertainty perceived by stakeholders during pairwise comparisons is not considered in this ahp approach. this can be resolved by incorporating fuzzy logic into the ahp. finally, the cost estimations in this work were not very accurate due to the limited availability of data. as a consequence, the value-driven approach by lee et al. (2016) could not be implemented. an alternative approach was used to calculate the value ratios in table 4, which are not very accurate due to limited information on cost variability and how the different cost components relate to the quality aspects. the value ratios are useful to identify relevant criteria to improve the value of the treatment by limiting cost variability. the high value ratio for the length of stay is remarkable. due to low costs and the variability of hospital stays, limiting the variability could improve value. in addition, value can be improved by offering training opportunities to surgeons and increasing costawareness. ijahp article: moons, pintelon, jorissen, de ridder, everaerts/identification of multistakeholder value in medical decision-making international journal of the analytic hierarchy process 98 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 5.3 future perspectives further research should focus on determining the cost drivers and cost variability to identify significant value improvement opportunities. furthermore, an alternative method can be used for weighting the stakeholder groups to obtain the overall ranking of the quality indicators. instead of cluster similarity, weights for stakeholder groups can be calculated based on the size of the groups or the knowledge of the stakeholder about each respective criterion. in the latter method, urologists will give higher weights for the gleason score, whereas patients would give higher weights for post-operative consequences. the resulting weights can be varied with a sensitivity analysis to investigate the impact on the overall ranking (forbes et al., 2018). finally, alternatives can be added to the hierarchy. in this case study, the alternatives represent the different treatment options (i.e., surgery, radiation therapy or waiting). the identified criteria weights were used to assign an overall score for each alternative. as a result, the ahp helps decision makers choose the treatment that provides the most value based on a ranking of the alternatives. if the model is extended with additional alternatives, this allows it to be validated in terms of functionality and consistency. 6. conclusion this work stresses the need to rigorously measure value using mcdm techniques in value-based prostate cancer care by clearly defining quality of care and determining the associated costs, while considering all of the stakeholders involved in the radical prostatectomy care cycle. healthcare providers that increase value will be the most competitive. the ahp is used as an innovative and structured approach to address the needs of healthcare decision-makers. the major challenge lies in mapping all of the stakeholder preferences into one value definition in order to identify opportunities for quality improvement and cost containment. in this research, an overall ranking of the criteria was constructed according to the six dimensions of quality of care. the results show that the pillars effective, patient-centered and equitable care contribute most to value creation. although the information on cost aspects was not accurate, a value ratio was computed to identify opportunities for value improvement. this case study shows the possibilities and challenges of applying ahp as a guideline for medical decision-making. ijahp article: moons, pintelon, jorissen, de ridder, everaerts/identification of multistakeholder value in medical decision-making international journal of the analytic hierarchy process 99 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 references adunlin, georges, vakaramoko diaby and hong xiao (2015). application of multicriteria decision analysis in health care: a systematic review and bibliometric analysis. health expectations 18(6), 1894–1905. doi: https://doi.org/10.1111/hex.12287 de bekker-grob, e. w., bliemer, m.c.j., donkers, b. et al. (2013). patients’ and urologists’ preferences for prostate cancer treatment: a discrete choice experiment. british journal of cancer, 109, 633-640. doi: https://doi.org/10.1038/bjc.2013.370 belgian cancer registry, bcr. 2017. cancer fact sheet prostate cancer. dyer, r.f., and forman, e.h. (1992). group decision support with the analytic hierarchy process. decision support systems, 8(2), 99-124. doi: https://doi.org/10.1016/0167-9236(92)90003-8 fontenot, p.a. & mansour, a.m. (2013). reporting positive surgical margins after radical prostatectomy: time for standardization. bju international, e290-299. doi: https://doi.org/10.1111/j.1464-410x.2012.11640.x forbes, j., hebb, a. and mu, e. (2018). ethical decision making in action: evaluating hospital care attendance approaches. international journal of the analytic hierarchy process, 10(3), 313-347. doi: . https://doi.org/10.13033/ijahp.v10i3.592 fossati, n., di trapani, e., gandaglia, g.et al. (2017). assessing the impact of surgeon experience on urinary continence recovery after robot-assisted radical prostatectomy: results of four high-volume surgeons. journal of endourology, 31(9). doi: https://doi.org/10.1089/end.2017.0085 hariharan, seetharaman, prasanta k. dey, harley s. l. moseley, areti y. kumar, and jagathi gora (2004). a new tool for measurement of process‐based performance of multispecialty tertiary care hospitals. international journal of health care quality assurance, 17(6), 302–12. doi: https://doi.org/10.1108/09526860410557552 hayes, j.h., ollendorf, d.a., pearson, s.d. et al. (2013). observation versus initial treatment for men with localized, low-risk prostate cancer: a cost-effectiveness analysis. annals of internal medicine. doi: https://doi.org/10.7326/0003-4819-158-12-20130618000002 hua, z., gong, b. and xu, x. (2008). a ds-ahp approach for multi-attribute decision making problem with incomplete information. expert systems with applications, 34(3), 2221-2227. doi: https://doi.org/10.1016/j.eswa.2007.02.021 hyams, e.s., partin, a.w. pierorazio, p.m. et al. (2012). impact of robotic technique and surgical volume on the cost of radical prostatectomy. journal of endourology, 27(3). doi: https://doi.org/10.1089/end.2012.0147 institute of medicine (iom). 2001. crossing the quality chasm: a new health system for the 21th century. ijahp article: moons, pintelon, jorissen, de ridder, everaerts/identification of multistakeholder value in medical decision-making international journal of the analytic hierarchy process 100 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 institute of medicine (iom). 2006. performance measurement: accelerating improvement. washington, dc. jarek, s. (2016). removing invonsistency in pairwise comparison matrix in the ahp. multiple criteria decision making, 11, 63–76. doi: https://doi.org/10.22367/mcdm.2016.11.05 kaplan, r.s. and porter, m.e. (2011). how to solve the cost crisis in health care. harvard business review. lee, r.c., donaldson, c. and cook, l.s. (2003). the need for evolution in healthcare decision modeling. medical care, 41(9),1024–33. doi: https://doi.org/10.1097/01.mlr.0000083746.54410.cf lee, v.s., kawamoto, k., hess, r. et al. (2016). implementation of a value-driven outcomes program to identify high variability in clinical costs and outcomes and association with reduced cost and improved quality. jama journal of the american medical association, 316(10), 1061-1072. doi:10.1001/jama.2016.12226 litwin, m.s. and tan, h.j. (2017). the diagnosis and treatment of prostate cancer: a review. jama journal of the american medical association, 317(24), 2532-2542. doi: 10.1001/jama.2017.7248. loeb, s. (2016). evidence-based versus personalized prostate cancer screening: using baseline prostate-specific antigen measurements to individualize screening. journal of clinical oncology, 34(23), 2684-2686. doi: https://doi.org/10.1200/jco.2016.68.2138 lotan, y., cadeddu, j.a. and gettman, m.t. (2004). the new economics of radical prostatectomy: cost comparison of open, laparoscopic and robot assisted techniques. journal of urology, 172(4), 1431-1435. doi: https://doi.org/10.1097/01.ju.0000139714.09832.47 maclennan, s., bekema, h.j., williamson, p.r. et al. (2015). a core outcome set for localised prostate cancer effectiveness trials: protocol for a systematic review of the literature and stakeholder involvement through interviews and a delphi survey. trials, 16(1), 76. doi: https://doi.org/10.1186/s13063-015-0598-0 maestrini, v., luzzini, d., maccarrone, p. and caniato, f. (2017). supply chain performance measurement systems: a systematic review and research agenda. international journal of production economics, 183(a), 299-315. doi: https://doi.org/10.1016/j.ijpe.2016.11.005 marsh, k. goetghebeur, m., thokala, p. and baltussen, r. (2017). multi-criteria decision analysis to support healthcare decisions. springer international publishing. doi: https://doi.org/10.1007/978-3-319-47540-0 marsh, k., lanitis, t., neasham, d. et al. (2014a). assessing the value of healthcare interventions using multi-criteria decision analysis: a review of the literature. ijahp article: moons, pintelon, jorissen, de ridder, everaerts/identification of multistakeholder value in medical decision-making international journal of the analytic hierarchy process 101 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 pharmacoeconomics, 32(4), 345–65. marsh, k., lanitis, t., neasham, d. et al. (2014b). assessing the value of healthcare interventions using multi-criteria decision analysis: a review of the literature. pharmacoeconomics. doi: https://doi.org/10.1007/s40273-014-0135-0 martin, n.e., massey, l., stowell, c. et al. (2015). defining a standard set of patientcentered outcomes for men with localized prostate cancer. european urology, 67(3), 460-467. doi: https://doi.org/10.1016/j.eururo.2014.08.075 mukherjee, a. (2008). redefining health care: creating value-based competition on results. international journal of pharmaceutical and healthcare marketing. niklas, c., saar, m., berg, b. et al. (2016). da vinci and open radical prostatectomy: comparison of cinical outcomes and analysis of insurance costs. urologia internationalis, 96(3). doi: https://doi.org/10.1159/000431104 di pierro, g.b., wirth, j.g., ferrari, m. et al. (2014). impact of a single-surgeon learning curve on complications, positioning injuries, and renal function in patients undergoing robot-assisted radical prostatectomy and extended pelvic lymph node dissection. urology, 84(5), 1106-1111. doi: https://doi.org/10.1016/j.urology.2014.06.047 porter, michael e (2010). what is value in health care? the new england journal of medicine, 363, 2477-2481. doi: 10.1056/nejmp1011024 porter, m.e. and lee, t.h. (2013). the strategy that will fix health care. harvard business review. porter, m.e. and lee, t.h. (2016). from volume to value in health care: the work begins. jama journal of the american medical association, 316(10), 1047-1048. prostate cancer foundation, pcf (2014). understanding surgery for prostate cancer. richman, m.b., forman, e.h., bayazit, y. et al. (2005). a novel computer based expert decision making model for prostate cancer disease management. journal of urology, 174(6), 2310-2318. doi: https://doi.org/10.1097/01.ju.0000181829.07078.22 roth, a.j., weinberger, m.i. and nelson, c.j. (2008). prostate cancer: quality of life, psychosocial implications and treatment choices. future oncology, 4(4), 561-568. doi: https://dx.doi.org/10.2217%2f14796694.4.4.561 saaty, t.l. (1990a). how to make a decision: the analytic hierarchy process. european journal of operational research, 48(1), 9–26. saaty, t.l. (1990b). how to make a decision: the analytic hierarchy process. european journal of operational research, 48(1), 9-26. doi: https://doi.org/10.1016/03772217(90)90057-i saaty, t.l. (2008). decision making with the analytic hierarchy process. international ijahp article: moons, pintelon, jorissen, de ridder, everaerts/identification of multistakeholder value in medical decision-making international journal of the analytic hierarchy process 102 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 journal of services sciences, 1(1), 83. saaty, t.l. and vargas, l.g. (2006). decision making with the analytic network process. economic, political, social and technological applications with benefits, opportunities, costs and risks. international series in operations research management science. springer. schmidt, k., aumann, i., hollander, i. et al. (2015). applying the analytic hierarchy process in healthcare research: a systematic literature review and evaluation of reporting. bmc medical informatics and decision making, 15(1), 112. doi: https://doi.org/10.1186/s12911-015-0234-7 song, y. and hu, y. (2009). group decision-making method in the field of coal mine safety management based on ahp with clustering. 6th international conference on information systems for crisis response and management (iscram 2009). stephani, v., crommelynck, a., durant, g. et al. (2018). financiering van ziekenhuisverblijven met een grote variabiliteit in het zorgproces. swartenbroekx, n., obyn, c., guillaume, p. et al. (2012). handleiding voor op-kostengebaseerde prijsbepaling van ziekenhuisinterventies. brussel. thokala, p., devlin, n., marsh, k. et al. (2016). multiple criteria decision analysis for health care decision making an introduction: report 1 of the ispor mcda emerging good practices task force. value in health, 19(1), 1–13. doi: https://doi.org/10.1016/j.jval.2015.12.003 zardari, n.h., ahmed, k., moniruzzaman shirazi, s. and yusop, z.b. (2015). weighting methods and their effects on multi-criteria decision making model outcomes in water resources management. international journal of operations research. springer. ijahp article: moons, pintelon, jorissen, de ridder, everaerts/identification of multistakeholder value in medical decision-making international journal of the analytic hierarchy process 103 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.682 appendix a overview of quality criteria, definitions and abbreviations criterion definition abbreviation psa level prostate specific antigen level prior to the surgery psa clinical stage cancer stage before the treatment starts cs gleason score prostate cancer score assigned based on microscopic appearance of tumor before surgery gs time in operating room total time patient is in the operating room tor surgical margins thickness of normal tissue around cancer cells, examined from pathologic report. positive surgical margins mean that the cancer cells touch the edge, whereas negative surgical margins mean there is a ring of normal tissue around cancer cells sm surgeon experience number of patients that the surgeon has treated during the past year se blood transfusion necessity of one or more blood transfusions during surgery bt length of stay number of days the patient stays in the hospital after surgery los complications occurrence and severity of undesired consequences after surgery comp urinary incontinence occurrence of unwanted urine leakage from bladder and/or pain during urination ui erectile dysfunction ability to have an erection and quality of the erection ed time to return to normal functioning number of days after surgery until patient can function normally as he could before surgery ttrf additional therapy necessity of ongoing interventions after surgery at ijahp article: boltürk, onar, öztayşi, göztepe, kahraman/ multi-attribute warehouse location selection in humanitarian logistics using hesitant fuzzy ahp international journal of the analytic hierarchy process 271 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.387 multi-attribute warehouse location selection in humanitarian logistics using hesitant fuzzy ahp eda boltürk sezi çevik onar başar öztayşi cengiz kahraman industrial engineering department, istanbul technical university, 34367, istanbul, turkey kahramanc@itu.edu.tr, cevikse@itu.edu.tr, oztaysib@itu.edu.tr, bolturk@itu.edu.tr, kerim göztepe institute of natural sciences, industrial engineering sakarya university, turkey kerimgoztepe@gmail.com abstract deploying warehouses at strategic locations becomes an important issue for humanitarian relief organizations in order to improve their relief aid capability and rescue plan. the delivery of sufficient technical equipment and provision of shelter and reinforcement to victims is a significant event during relief operations. warehouse location selection in humanitarian logistics (hl) is a challenging process because choosing a non-optimal location may cause additional problems during rescue activities. the conventional decision making tools used for a warehouse location selection problem tend to be less effective in dealing with the imprecise or vague nature of the linguistic assessment. in many situations, the values of the qualitative attributes are often incompletely determined by the decision-makers. the fuzzy set theory can capture this type of uncertainty. in this paper, a recent extension of ordinary fuzzy sets, namely hesitant fuzzy sets, is used for considering the decision makers hesitancy in the evaluation. to solve the hl warehouse location selection problem, we propose a new hesitant fuzzy analytic hierarchy process (ahp) method. we also present a hl warehouse location selection case study for a turkish humanitarian relief organization by using hesitant fuzzy preference information. keywords: warehouse location selection; multi-attribute decision-making (madm); fuzzy logic; humanitarian logistics; hesitant fuzzy sets 1. introduction for the purpose of alleviating the suffering of vulnerable people, the activities of planning, implementing and controlling the efficient, cost-effective flow of and storage of goods and materials from production to consumption are called humanitarian logistics (thomas & kopczak, 2005). another definition for humanitarian logistics is given in this way: the processes and systems involved in mobilizing people, resources, skills and knowledge to help vulnerable people affected by disaster (van wassenhove, 2006). https://scholar.google.com.tr/citations?view_op=view_org&hl=tr&org=11256784414184154164 ijahp article: boltürk, onar, öztayşi, göztepe, kahraman/ multi-attribute warehouse location selection in humanitarian logistics using hesitant fuzzy ahp international journal of the analytic hierarchy process 272 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.387 natural disasters, both rapid-onset events (such as earthquakes and floods) and slowonset events (such as hunger, poverty and drought) or man-made crises (such as war and civil unrest), increase vulnerability of nations or regions and seriously affect local and national economies (roh et al., 2015). in this paper, we focus on rapid-onset natural disasters which appear suddenly with no warning. hence, disaster management, a vital issue to deal with natural and man-made disasters, needs a systematic approach (ahmadi et al., 2015). disaster management has four main phases which are mitigation, preparation, response and recovery (ivgin, 2013). longterm efforts should be made to keep the occurrence of disasters in the mitigation phase (chou et al., 2015). the preparation phase is also another important part of disaster management. government or social organizations should make a plan which includes the preparation phase before the occurrence of a disaster. the warehouse location in humanitarian logistics has high importance since it determines the success of the disaster response after an event. there are few studies on the warehouse location problem regarding humanitarian relief logistics in the literature. warehouse location selection in humanitarian logistics has also drawn a lot of attention from humanitarian relief organizations in recent years. the world’s biggest humanitarian relief organizations, such as world food programme (wfp), international federation of red cross and red crescent societies (ifrc), and action against hunger (aah), have begun to deploy strategic pre-positioned warehouses around the world. different researchers have studied the importance of the preparedness phase and the need for pre-positioned warehouses in humanitarian relief operations, but only a small number of papers are related to the warehouse location selection problem in humanitarian logistics (dekle et al., 2005; balcik and beamon, 2008; huang et al., 2015; rath & gutjahr, 2014; florez et al., 2015). the evaluation process for the warehouse location decision usually includes different and possibly conflicting tangible and intangible attributes, which requires an evaluation to be performed with vague and incomplete information (onut & soner, 2007; demirel & kahraman, 2010). this reality generally forces decision makers to model the problem by applying a fuzzy multi-attribute decision-making (madm) approach. the fuzzy set theory has a history starting with the introduction of ordinary fuzzy sets by l. a. zadeh (1965) and continuing with the extensions of these sets as illustrated in figure 1 (kahraman et al., 2016). ijahp article: boltürk, onar, öztayşi, göztepe, kahraman/ multi-attribute warehouse location selection in humanitarian logistics using hesitant fuzzy ahp international journal of the analytic hierarchy process 273 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.387 figure 1. history of the fuzzy set theory ordinary fuzzy sets and their new extensions have been extensively used in the solution of industrial problems: outsourcing (bottani & rizzi, 2006; kahraman et al., 2010), transportation (chana & kuchta, 1996; kaya et al., 2012), energy (heo et al., 2010; cevik onar et al., 2015), urban transformation (olazabal & pascual, 2016; oztaysi et al., 2016), engineering economics (shahriari, 2011; kahraman et al., 2015). in the literature, the most used madm method is the analytic hierarchy process (ahp) developed by saaty (1980). it has been used in the solution of various madm problems: outsourcing (atkinson et al., 2015), capability development (dangol et al, 2015) and personnel management (kashi, 2016), etc. in our proposed madm method, we utilize hesitant fuzzy sets (hfss), which are the latest extension of ordinary fuzzy sets. hfss are a novel extension of fuzzy sets aiming at modeling the uncertainty caused by the hesitation arising in the assignment of membership degrees of the elements to a fuzzy set. a hfs is defined in terms of a function that returns a set of membership values for each element in the domain (rodriguez et al., 2014). in this paper, we used hesitant fuzzy linguistic term sets (hflts) which was introduced by rodríguez et al. (2012). through the usage of hflts, different linguistic evaluations of different actors can be aggregated without loss of information. hflts enable expressing the hesitance existing in linguistic evaluation as clearly as possible. a multi-attribute warehouse location selection problem in humanitarian logistics employs linguistic evaluations which mostly include hesitance of multiple experts. the proposed hesitant fuzzy madm method evaluates warehouse location alternatives in humanitarian logistics using a number of tangible and intangible ordinary fuzzy sets, zadeh (1965) type-2 fuzzy sets, zadeh (1975 interval-valued fuzzy sets, zadeh (1975), grattanguiness (1975), jahn (1975), sambuc (1975) intuitionis tic fuzzy sets, atanassov (1986) fuzzy multisets, yager (1986) nonstationa ry fuzzy sets, garibaldi and ozen (2007) hesitant fuzzy sets, torra (2010) ijahp article: boltürk, onar, öztayşi, göztepe, kahraman/ multi-attribute warehouse location selection in humanitarian logistics using hesitant fuzzy ahp international journal of the analytic hierarchy process 274 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.387 attributes. the proposed madm method is based on a new hesitant fuzzy ahp method for the evaluation of alternative warehouse locations. the novelty of the developed fuzzy hflts based ahp method is its capability to overcome the hesitancy involved in multi criteria warehouse selection problem with linguistic assessments of multiple experts. our paper is structured as follows.: in section 2, a literature review on humanitarian logistics (hl) and related definitions is presented. warehouse location selection attributes in hl are given in section 3. section 4 focuses on the methodological background of the hesitant fuzzy sets and hesitant fuzzy ahp. a case study and a sensitivity analysis are given in section 5. finally, conclusions and future directions are presented in section 6. 2. humanitarian logistics (hl) the creation of an effective disaster supply chain to deliver necessary goods to disaster relief organizations is an essential function of disaster management. this function is also called humanitarian logistics. humanitarian logistics is a broad term that covers the operations concerning supply chain strategies, processes, and technologies that will maintain the flow of goods and materials needed for the humanitarian effort. the management of the supply chain in disaster relief operations is considered an essential element in the resolution of a crisis since the tsunami in south east asia (december, 26th 2004) and hurricane katrina (august, 2005). security is a very important requirement in humanitarian logistics. in the aftermath of a disaster, many goods (e.g., medicine, foods), which are usually available in normal conditions, became extremely valuable and a potential target of thieves. supply chain management for business applications had a long evolution and many companies have well established supply chains around the world, but the strategic goal of commercial supply chains and disaster supply chains is different. commercial supply chains are focused on quality and profitability whereas humanitarian supply chains must be focused on minimizing loss of life and suffering. the works on hl can be classified into conceptual and strategic works and operation research (or) based works. some recent papers on conceptual and strategic hl are as follows. vaillancourt (2016) developed a theoretical framework to better understand incentives and obstacles to consolidation of materials in humanitarian logistics. l'hermitte et al. (2016) explored the underlying strategic mechanisms of agility in a humanitarian logistics context. based on the research conducted in business disciplines, the paper empirically examines a set of four strategic dimensions (being purposeful, being action-focused, being collaborative, and being learningoriented) and identifies an emergent relationship between these capabilities and agile humanitarian logistics operations. vega and roussat (2015) investigated the role of logistics service providers in humanitarian relief. leiras et al. (2014) presented a literature review of hl that aims to identify trends and suggest some directions for future research. some recent papers on operation research (or) based hl are as follows. tofighi et al. (2016) addressed a two-echelon humanitarian logistics network design problem involving multiple central warehouses and local distribution centers and developed a novel two-stage scenario-based possibilistic-stochastic programming approach. ransikarbum and mason (2016) presented a multiple objective, integrated network optimization model for making strategic decisions in the supply distribution and https://www.scopus.com/authid/detail.uri?authorid=57188550272&amp;eid=2-s2.0-84961621937 https://www.scopus.com/authid/detail.uri?authorid=56416407700&amp;eid=2-s2.0-84954372836 https://www.scopus.com/authid/detail.uri?authorid=7202152323&amp;eid=2-s2.0-84954372836 ijahp article: boltürk, onar, öztayşi, göztepe, kahraman/ multi-attribute warehouse location selection in humanitarian logistics using hesitant fuzzy ahp international journal of the analytic hierarchy process 275 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.387 network restoration phases of humanitarian logistics operations. their model provides an equity-based solution for constrained capacity, budget and resource problems in post-disaster logistics management. gralla et al. (2015) provided a basis for the design and improvement of simulated emergency training exercises, which are common in the humanitarian practice community. özdamar and ertem (2015) presented a survey that focused on the response and recovery planning phases of the disaster lifecycle. the related mathematical models developed in this research area were classified in terms of vehicle/network representation structures and their functionality. díaz-delgado and gaytán iniestra (2014) dealt with the relationship between a flood risk assessment and the humanitarian logistics process design related to emergency events caused by flooding. the magnitude and timing of the flooding is estimated using a forecasting model that requires a hydrologic component to convert rainfall into runoff as well as a hydraulic component to route the flow through the stream network predicting time and severity of the flood wave. figure 2 illustrates the frequencies of publications on hl with respect to publication years. this figure was obtained by entering “humanitarian logistics” as an “article title, abstract or keywords” to the scopus database considering the publications up to august 2016. as seen in figure 2, researchers have been focused on hl especially after the year 2005 since disasters such as the tsunami in south east asia in 2004 and hurricane katrina in 2005 which caused the deaths of many people. figure 2. frequencies of publications on hl with respect to publication years figure 3 presents the journals publishing hl papers. the journal of humanitarian logistics and supply chain management is the leading journal publishing hl papers. the journals that publish the second most hl papers are procedia engineering and socio economic planning sciences. 0 10 20 30 40 50 60 70 80 90 2004 2006 2008 2010 2012 2014 2016 https://www.scopus.com/authid/detail.uri?authorid=6506956166&amp;eid=2-s2.0-84919779039 https://www.scopus.com/authid/detail.uri?authorid=56460374900&amp;eid=2-s2.0-84919779039 ijahp article: boltürk, onar, öztayşi, göztepe, kahraman/ multi-attribute warehouse location selection in humanitarian logistics using hesitant fuzzy ahp international journal of the analytic hierarchy process 276 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.387 figure 3. journals publishing hl works figure 4 shows the document types of the publications on hl. articles and conference papers are by far the first two most popular ways of publication media for hl works. figure 4. document types of the publications on hl figure 5 illustrates the subject areas of the publications on hl. business, management and accounting, decision sciences, engineering, computer science, and social sciences are the leading subject areas used in hl. 0 10 20 30 40 50 60 70 0 50 100 150 200 250 ijahp article: boltürk, onar, öztayşi, göztepe, kahraman/ multi-attribute warehouse location selection in humanitarian logistics using hesitant fuzzy ahp international journal of the analytic hierarchy process 277 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.387 figure 5. subject areas of the publications on hl 3. warehouse location selection in hl warehousing is essentially an act of storing goods between the time they are assembled and the time they are handed to the customer. warehousing commonly depends on human resources and required facilities and equipment costs (stock & lambert, 2001). besides, warehouse performance directly affects the whole supply chain performance of a firm or organization (tuzkaya & önüt, 2009). ineffective warehouse location selection, design or management will threaten the achievement of a humanitarian relief organization and result in unnecessarily high costs (pazour & carlo, 2015). logistic researchers commonly agree that warehouse location is a problem of strategic level network design (powers, 1989; özcan et al., 2011; ashrafzadeh et al., 2012). this kind of decision is long-term and the influence of the warehouse location selection affects the profitability of the company. a warehouse system not only reduces cost and simplifies operations, but also allows companies to focus on their main targets (choi et al., 2001). we know that items are transformed into final products by being processed sequentially at multiple locations in supply chain networks. hence, a warehouse is a vital part of a typical supply chain management. in recent years, the demand for strategic stock-holding for humanitarian purposes has become increasingly vital. human relief organizations or governments notice that a large number of high-impact natural and man-made disasters influence the stability of states (guha-sapir et al., 2013; roh et al., 2015). for instance, the 2004 earthquake and resulting tsunami in south asia caused approximately 230,000 deaths and displaced 1.7 million people in many urban areas. more than 40 countries and 700 nongovernmental organizations (ngos) provided humanitarian materials (russell, 2005; cozzolino, 2012). 0 20 40 60 80 100 120 140 160 180 200 ijahp article: boltürk, onar, öztayşi, göztepe, kahraman/ multi-attribute warehouse location selection in humanitarian logistics using hesitant fuzzy ahp international journal of the analytic hierarchy process 278 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.387 warehouse location selection is a multi-attribute decision making problem including both tangible and intangible attributes. table 1 summarizes the attributes used in warehouse location selection problems in the literature. table l literature on warehouse location selection attributes researchers research areas main evaluation attributes for warehouse (or logistics center) selection korpela, j., tuominen, m., (1996). a decision aid in warehouse site selection. reliability compliance; flexibility special, strategic compatibility alberto, p., (2000). the logistics of industrial location decisions. environmental aspects, cost, quality of living, local incentives, time reliability provided to customers, response flexibility to customer's demand, integration with customers sarkis, j., sundarraj, r.p., (2002). hub location selection for digital equipment corporation cost, accessibility, time, regulatory, risk, labour, strategy demirel, t., demirel, n. ç., & kahraman, c. (2010). warehouse location selection using choquet integral. costs, labor characteristics, infrastructure, markets, macro environment kayikci, y., (2010) logistics center location using conceptual model. economical scale, national stability, intermodal operation and management, international market location, and environmental effect li, y., liu, x., chen, y. (2011). selection of logistics centre weather and landform condition, water supply, power supply, solid cast-off disposal, communication, traffic, candidate land area, candidate land shape, candidate land circumjacent main line, candidate land land-value, freight transport, and fundamental construction investment. özcan, t., çelebi, n., & esnaf, ş. (2011). warehouse location selection problem using multi-attributes decision making methodologies. unit price ($/m2), stock holding capacity (unit), average distance to shops (kilometer), average distance to main suppliers (kilometer), movement, flexibility, roh, s. y., jang, h. m., & han, c. h. (2013). warehouse location decision factors in humanitarian relief logistics. location, logistics, national stability, cost, cooperation. roh, s., pettit, s., harris, i., & beresford, a. (2015). the pre-positioning of warehouses at regional and local levels. location, national stability, cost, cooperation, logistics based on the above literature review, warehouse location selection attributes have been determined as follows:  geographical location (roh, 2015): geographic location refers to a position on the earth. absolute geographic location of a point is defined by two coordinates, longitude and latitude. this attribute is crucial in order to provide a cost-effective flow and storage of goods. it has four sub-attributes determined as “proximity to disaster areas”, “logistics experts” availability”, “warehouse security”, “proximity to urban facilities”, and “closeness to other warehouses”.  transport connectivity (vitoriano et al., 2011; barbarosoglu and arda, 2004): transportation is a critical issue in humanitarian relief operations to deliver ijahp article: boltürk, onar, öztayşi, göztepe, kahraman/ multi-attribute warehouse location selection in humanitarian logistics using hesitant fuzzy ahp international journal of the analytic hierarchy process 279 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.387 aid at the right time and to the right place. we know that the deliveries must be fast, fair and safe. decision makers must consider the actual fuel availability, available vehicles, climate, road conditions and airports and port’s capacity after the disaster during planning process of transportation operations (wassenhove & martinez, 2012). this attribute includes the subattributes “availability of seaport and airport”, “near to (potential) beneficiaries”, “adequate warehouse facilities”, “adequate warehouse infrastructure”, and “warehouse accessibility”.  cost (roh et al., 2013; kahraman et al., 2007; sari et al., 2013): cost effective flow and storage of goods and materials is one of the main issues in humanitarian logistics together with the planning, implementing and controlling processes (tomasini &wassenhove, 2009). when developing a warehouse location selection plan for a new organization, decision makers must assign cost estimates in order to assess whether an organization’s budget will cover costs. costs are usually underestimated; it should be analyzed during the execution of humanitarian relief operations. although there are different types of cost in management (blocher et al., 2008), we have classified the sub-attributes as “storage cost”, “cost relate to logistics”, “land cost”, “labor price”, and “replenishment cost”.  stable government (roh et al., 2015; seaman, 1999): a stable political situation is important for the operation of the pre-positioned warehouse. if the political, economic, and social state of a country is very fragile and unstable, it will be difficult for a humanitarian organization to operate their supply chain in a risky and dangerous environment. the stable government attribute has four sub-attributes “cooperation with logistics agents”, “political and economic stability”, “existence of other agents (ngos)”, and “it/communication”.  labor availability (roh et al., 2013; demirel et al., 2010): today’s warehouses include large amounts of materials, machines, and people and have complex infrastructures. thus, the availability of qualified labor is another issue for our proposed model. this is one of the main requirements of warehouse management in order to perform a better humanitarian relief operation. we assume that the qualified labor does not have the same standards at each location. the sub-attributes of availability of labor are “trained and qualified personnel”, “flexible customs regulations”, “population density”, and “climate”. based on the determined main-attributes and sub-attributes, the hierarchy of the proposed model is given in figure 6. there are 5 main attributes and 4 sub-attributes under each main attribute, and 5 hl warehouse location alternatives (hlw). ijahp article: boltürk, onar, öztayşi, göztepe, kahraman/ multi-attribute warehouse location selection in humanitarian logistics using hesitant fuzzy ahp international journal of the analytic hierarchy process 280 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.387 hlw1 hlw2 hlw3 hlw4 hlw5 alternatives figure 6. hierarchy of the problem 4. preliminaries: hesitant fuzzy sets torra (2010) introduced hesitant fuzzy sets (hfss) since determining the membership degree of an element to a fuzzy set is not an easy work. the difficulty comes from possible values that cause hesitation about which one would be the right one. definition 1: let x be a fixed set, a hesitant fuzzy set (hfs) on x is in terms of a function that when applied to x returns a subset of [0, 1] (torra, 2010). mathematical expression for hfs is as follows: 𝐸 = {< 𝑥,ℎ𝐸(𝑥) > | 𝑥𝜖 𝑋 }, (1) where ℎ𝐸(𝑥) is a set of some values in [0, 1], denoting the possible membership degrees of the element 𝑥𝜖 𝑋 to the set e. xia and xu (2011) give some basic definition about h as follows. definition 2: the upper and lower bound is defined as in eq. 2 and eq. 3. ℎ−(𝑥) = minℎ(𝑥); (2) ℎ+(𝑥) = maxℎ(𝑥); (3) definition 3: the compliment of h is given in eq. 4. ℎ𝑐 = ∪𝛾∈ℎ {1 − 𝛾}; (4) warehouse location selection in humanitarian logistics geographical location proximity to urban facilities proximity to disaster areas closeness to other warehouses warehouse security cost storage cost investment cost labor price replenishment cost transport connectivity availability of seaport and airport near to (potential) beneficiaries adequate warehouse facilities adequate warehouse infrastructure labor availability trained and qualified personnel flexible customs regulations population density climate stable government cooperation with logistics agents political and economical stability existence of other agents (ngos) it/communication ijahp article: boltürk, onar, öztayşi, göztepe, kahraman/ multi-attribute warehouse location selection in humanitarian logistics using hesitant fuzzy ahp international journal of the analytic hierarchy process 281 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.387 definition 4: the envelope of h, aenv(h), is an intuitionistic fuzzy set which is defined as 𝐴𝑒𝑛𝑣(ℎ) = {𝑥,𝜇(𝑥),𝑣(𝑥)} (5) where 𝜇(𝑥) = ℎ−(𝑥) (6) 𝑣(𝑥) = 1 − ℎ+(𝑥) (7) the basic operations on hfss can be found in zhang and wei (2013). rodríguez et al. (2012) have introduced hesitant fuzzy linguistic term sets (hflts) to improve the elicitation of linguistic information in decision making when experts hesitate among several linguistic terms to express their assessments. these sets provide greater flexibility to elicit comparative linguistic expressions by using context-free grammar that formalizes the generation of flexible linguistic expressions. hence, we prefer the use hflts in this paper. definition 5. the envelope of an hesitant fuzzy linguistic term sets (hflts), env(hs), is a linguistic interval whose limits are obtained by means of its upper bound and lower bound: 𝑒𝑛𝑣(𝐻𝑆) = [𝐻𝑆−,𝐻𝑆+], 𝐻𝑆− ≤ 𝐻𝑆+, (8) where the upper bound and lower bound are defined as 𝐻𝑆+ = 𝑚𝑎𝑥{𝑆𝑖} = 𝑆𝑗,𝑆𝑖 ≤ 𝑆𝑗 𝑎𝑛𝑑 𝑆𝑖 ∈ 𝐻𝑆,∀𝑖, (9) 𝐻𝑆− = 𝑚𝑖𝑛{𝑆𝑖} = 𝑆𝑗,𝑆𝑖 ≥ 𝑆𝑗 𝑎𝑛𝑑 𝑆𝑖 ∈ 𝐻𝑆,∀𝑖, (10) definition 6. an owa operator of dimension n is a mapping owa: 𝑅𝑛 → 𝑅, so that 𝑂𝑊𝐴(𝑎1,𝑎2,…,𝑎𝑛) = ∑ 𝑤𝑗𝑏𝑗 𝑛 𝑗=1 (11) where 𝑏𝑗 is the jth largest of the aggregated arguments 𝑎1,𝑎2,…,𝑎𝑛, and 𝑊 = (𝑤1,𝑤2,…,𝑤𝑛) 𝑇 is the associated weighting vector satisfying wi ∈ [0,1], i = 1,2,…,n and ∑ wi = 1 n i=1 . definition 7. a triangular fuzzy membership function �̃�=(a, b, c) is used as the representation of the comparative linguistic expressions based on hflts, the definition domain of �̃� should be the same as the linguistic terms {si, . . . , sj} ∈ hs. the min and the max operators are used to compute a and c. 𝑎 = 𝑚𝑖𝑛{𝑎𝐿 𝑖 ,𝑎𝑀 𝑖 ,𝑎𝑀 𝑖+1,…,𝑎𝑀 𝑗 ,𝑎𝑅 𝑗 } = 𝑎𝐿 𝑖 (12) 𝑐 = 𝑚𝑎𝑥{𝑎𝐿 𝑖 ,𝑎𝑀 𝑖 ,𝑎𝑀 𝑖+1,…,𝑎𝑀 𝑗 ,𝑎𝑅 𝑗 } = 𝑎𝑅 𝑖 (13) the remaining elements 𝑎𝑀 𝑖 ,𝑎𝑀 𝑖+1,…,𝑎𝑀 𝑗 ∈ 𝑇 should contribute to the computation of the parameter b. the aggregation operator owa will be used to aggregate them: 𝑏 = 𝑂𝑊𝐴𝑊𝑆(𝑎𝑀 𝑖 ,𝑎𝑀 𝑖+1,…,𝑎𝑀 𝑗 ) (14) ijahp article: boltürk, onar, öztayşi, göztepe, kahraman/ multi-attribute warehouse location selection in humanitarian logistics using hesitant fuzzy ahp international journal of the analytic hierarchy process 282 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.387 5. hesitant fuzzy ahp model in the proposed hesitant fuzzy ahp method, we first determine the main and subattributes and the hierarchy for the warehouse location selection problem, then make a multi-attributes evaluation of the warehouse location alternatives to illustrate how the proposed model is used to solve it. the steps of the hesitant fuzzy ahp extended based on buckley (1985)’s ahp method are given: step 1. pairwise comparison matrices for attributes, sub-attributes and alternatives are constructed and expert’s evaluations using linguistic terms are collected. step 2. using the scale given in table 1, the linguistic terms are transformed into triangular fuzzy numbers (tan et al., 2014) and trapezoidal fuzzy numbers. table 2 linguistic scale for hesitant fuzzy ahp linguistic term abb. triangular fuzzy number trapezoidal fuzzy number absolutely high importance (ahi) (7,9,9) (7,9,9,9) very high importance (vhi) (5,7,9) (5,7,7,9) essentially high importance (eshi) (3,5,7) (3,5,5,7) weakly high importance (whi) (1,3,5) (1,3,3,5) equally high importance (ehi) (1,1,3) (1,1,1,3) exactly equal (ee) (1,1,1) (1,1,1,1) equally low importance (eli) (0.33,1,1) (0.33,1,1,1) weakly low importance (wli) (0.2,0.33,1) (0.2,0.33,0.33,1) essentially low importance (esli) (0.14,0.2,0.33) (0.14,0.2,0.2, 0.33) very low importance (vli) (0.11,0.14,0.2) (0.11,0.14,0.14,0.2) absolutely low importance (ali) (0.11,0.11,0.14) (0.11,0.11,0.11,0.14) each element (�̃�𝑖𝑗 𝑘 ) of the pairwise comparison matrix �̃�𝑘 is a fuzzy number corresponding to its linguistic term. the pairwise comparison matrix is given by; �̃�𝑘 = | 1 �̃�21 𝑘 ⋮ �̃�𝑛1 𝑘 �̃�12 𝑘 1 ⋮ �̃�𝑛2 𝑘 … … ⋮⋮⋮ … �̃�1𝑛 𝑘 �̃�2𝑛 𝑘 ⋮ 1 | (15) where (�̃�𝑖𝑗 𝑘 ) represents the kth expert’s evaluation on comparison of ith element to jth element. step 3. the consistency of each fuzzy pairwise comparison matrix is examined. in order to check the consistency of the fuzzy pairwise comparison matrices, pairwise comparison values are defuzzified by the graded mean integration approach (hsieh & chen, 1999). assume  ijaa ~ ~  is a fuzzy positive reciprocal matrix and  ijaa  is its defuzzified positive reciprocal matrix. if the result of the comparisons of  ijaa  is consistent, then it can imply that the result of the comparisons of  ijaa ~ ~  is also consistent (buckley, 1985). according to the graded mean integration approach, a triangular fuzzy number ),,( ~ umla  can be transformed into a crisp number by employing the below equation: ijahp article: boltürk, onar, öztayşi, göztepe, kahraman/ multi-attribute warehouse location selection in humanitarian logistics using hesitant fuzzy ahp international journal of the analytic hierarchy process 283 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.387 𝐴 = 𝑙+4𝑚+𝑢 6 (16) if the pairwise comparisons are not consistent, experts must reevaluate the pairwise comparisons. step 4: identification of conflicts and renewing the evaluations. the evaluations of the experts are checked for their closeness to each other. if the evaluations are not close then experts are informed of the need to discuss the situation and renew their evaluations. step 5: fuzzy envelope approach, proposed by liu and rodriguez (2014), is used to combine expert evaluations. the scale given in table 1 is sorted from the lowest (𝑠0) to the highest (𝑠𝑔). assume the expert evaluations vary between two terms i.e. 𝑠𝑖 and 𝑠𝑗. then 𝑠0 ≤ 𝑠𝑖 < 𝑠𝑗 ≤ 𝑠0. the parameters a and d of the trapezoidal fuzzy membership function �̃� = (a, b, c, d) are computed as 𝑎 = min{𝑎𝐿 𝑖 ,𝑎𝑀 𝑖 ,𝑎𝑀 𝑖+1,…. .𝑎𝑀 𝑗 ,𝑎𝑅 𝑗 } = 𝑎𝐿 𝑖 (17) 𝑑 = min{𝑎𝐿 𝑖 ,𝑎𝑀 𝑖 ,𝑎𝑀 𝑖+1,… . .𝑎𝑀 𝑗 ,𝑎𝑅 𝑗 } = 𝑎𝑅 𝑗 (18) 𝑏 = { 𝑎𝑚 𝑖 , 𝑖𝑓 𝑖 + 1 = 𝑗 𝑂𝑊𝐴𝑤2 (𝑎𝑚 𝑖 ,…. .𝑎𝑚 𝑖+𝑗 2 ), 𝑖𝑓 𝑖 + 𝑗 𝑖𝑠 𝑒𝑣𝑒𝑛 𝑂𝑊𝐴𝑤2 (𝑎𝑚 𝑖 ,…. .𝑎𝑚 𝑖+𝑗−1 2 ), 𝑖𝑓 𝑖 + 𝑗 𝑖𝑠 𝑜𝑑𝑑 (19) 𝑐 = { 𝑎𝑚 𝑖+1 , 𝑖𝑓 𝑖 + 1 = 𝑗 𝑂𝑊𝐴𝑤1 (𝑎𝑚 𝑗 ,𝑎𝑚 𝑗−1 …..𝑎𝑚 𝑖+𝑗 2 ), 𝑖𝑓 𝑖 + 𝑗 𝑖𝑠 𝑒𝑣𝑒𝑛 𝑐 = 𝑂𝑊𝐴𝑤1 (𝑎𝑚 𝑗 ,𝑎𝑚 𝑗−1 ,…. .𝑎𝑚 𝑖+𝑗+1 2 ), 𝑖𝑓 𝑖 + 𝑗 𝑖𝑠 𝑜𝑑𝑑 (20) owa operation given in definition 6 requires a weight vector. filev and yager (1998) define the first and second type of weights using α parameter which belong to the unit interval [0,1]. the first kind of weights 𝑊1 = (𝑤1 1,𝑤2 1 …𝑤𝑛 1) is defined as: 𝑤1 1 = 𝛼2,𝑤2 1 = 𝛼2(1 − 𝛼2),…….𝑤𝑛 1 = 𝛼2(1 − 𝛼2) 𝑛−2 the second kind of weights 𝑊2 = (𝑤1 2,𝑤2 2 …𝑤𝑛 2) is defined as: 𝑤1 2 = 𝛼1 𝑛−1,𝑤2 2 = (1 − 𝛼1)𝛼1 𝑛−2,…….𝑤𝑛 2 = 1 − 𝛼1, where 𝛼1 = 𝑔−(𝑗−𝑖) 𝑔−1 and 𝛼2 = (𝑗−𝑖)−1 𝑔−1 where g is the number of terms in the evaluation scale, j is the rank of highest evaluation and i is the rank of lowest evaluation value of the given interval. step 6: collaborative pairwise comparison matrix (�̃�) is formed. ijahp article: boltürk, onar, öztayşi, göztepe, kahraman/ multi-attribute warehouse location selection in humanitarian logistics using hesitant fuzzy ahp international journal of the analytic hierarchy process 284 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.387 �̃� = | 1 �̃�21 ⋮ �̃�𝑛1 �̃�12 1 ⋮ �̃�𝑛2 … … ⋮⋮⋮ … �̃�1𝑛 �̃�2𝑛 ⋮ 1 | (21) where �̃�𝑖𝑗 = (𝑐𝑖𝑗𝑙 ,𝑐𝑖𝑗𝑚1 ,𝑐𝑖𝑗𝑚2 ,𝑐𝑖𝑗𝑢 ) since the fuzzy envelopes, obtained in the previous step are trapezoidal fuzzy numbers, reciprocal values are calculated as follows: �̃�𝑗𝑖 = ( 1 𝑐𝑖𝑗𝑢 , 1 𝑐𝑖𝑗𝑚2 , 1 𝑐𝑖𝑗𝑚1 , 1 𝑐𝑖𝑗𝑙 ) (22) step 7: fuzzy geometric mean for each row (�̃�𝑖) of the collaborative pairwise matrix is computed using eq. 23. �̃�𝑖 = (�̃�𝑖1 ⨂�̃�𝑖2 …⨂�̃�𝑖𝑛) 1/𝑛 (23) step 8: the fuzzy weight (�̃�𝑖) of each attribute (or alternative) is calculated using (�̃�𝑖) values as follows: �̃�𝑖 = �̃�𝑖 ⨂(�̃�1 ⊕ �̃�2 …⊕ �̃�𝑛) −1 (24) in this study, r̃1 ⊕ r̃2 …⊕ r̃n value is accepted as the maximum parameter of the absolutely high importance in table 1 in order to decrease the deviation in the weights. step 9: the next step is to calculate the fuzzy performance scores of each alternative. to this end, steps 1 – 7 are repeated for each pairwise comparison matrix formed according to the predetermined decision model. the final fuzzy score of each alternative is calculated by eq 25. �̃�𝑖 = ∑ �̃�𝑗�̃�𝑗,∀𝑖. 𝑛 𝑗=1 (25) where �̃�𝑖 is the fuzzy performance score of alternative i; �̃�𝑗 is the weight of the attribute j, and �̃�𝑗 is the performance score of alternative i with respect to attribute j. step 10: trapezoidal fuzzy numbers are defuzzified in order to determine the importance ranking of the alternatives. defuzzification of the trapezoidal fuzzy numbers is made using eq. 26 (sahoo et al., 2016). 𝐷 = 𝑐𝑙+ 2𝑐𝑚1 + 2𝑐𝑚2+ 𝑐𝑢 6 (26) step 11: the alternatives are ranked according to the defuzzified values and the alternative with the best score is selected. 6. a case study the presented multi-attribute hl warehouse location selection model was applied for an earthquake prone area in the northwest of turkey. this area contains eight districts and has 23 million inhabitants. approximately 67,000 km² and 8.5% of turkey's revenue comes from this region. it is turkey's main industrial region and includes the city i̇stanbul, the center of turkish economy. however, in this region there is a giant ijahp article: boltürk, onar, öztayşi, göztepe, kahraman/ multi-attribute warehouse location selection in humanitarian logistics using hesitant fuzzy ahp international journal of the analytic hierarchy process 285 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.387 earthquake risk, and the expected damage of an earthquake here is very large anticipating a high number of deaths and physical destruction. the level of damage occurred by the izmit earthquake in 1999 was so high that it killed around 17,000 people and left almost half a million people homeless. the prioritization of five feasible hl warehouse location alternatives is required in marmara region. the possible alternatives are lüleburgaz, çorlu, kocaeli, bursa, and gönen. the main and sub-attributes given in fig. 6 are used in the multi-attribute evaluation. the five alternative locations for a warehouse are shown in fig. 7. figure 7. alternative locations for warehouse selection a team of three experts discussed the importance of the main attributes and came to a compromise. table 3 presents the compromised pairwise comparisons of main attributes using hflts. table 3 pairwise comparison of main attributes using hflts comparison of main attributes w.r.t. goal geographical location cost transport connectivity labor availability stable government geographical location ee between wli and ee between eli and ehi between ehi and eshi between wli and eli cost ee between ehi and eshi between ehi and eshi between eli and ehi transport connectivity ee between ehi and whi between esli and eli labor availability ee between ehi and whi stable government ee using the owa operations defined in eqs. 17-20, hfltss are aggregated into trapezoidal fuzzy sets as in table 4. ijahp article: boltürk, onar, öztayşi, göztepe, kahraman/ multi-attribute warehouse location selection in humanitarian logistics using hesitant fuzzy ahp international journal of the analytic hierarchy process 286 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.387 table 4 aggregated hflts scores comparison of main attributes w.r.t. goal geographical location cost transport connectivity labor availability stable government geographical location (1,1,1,1) (0.2,0.926,1,1) (0.333,1,1,3) (1,2.778,3.222,7) (0.2,0.333,1,1) cost (1,0.931,1.08,5) (1,1,1,1) (1,2.778,3.222,7) (1,2.778,3.222,7) (0.333,1,1,3) transport connectivity (0.333,1,1,3) (0.143,0.31,0.36, 1) (1,1,1,1) (1,1,3,5) (0.143,0.319,0.348,1) labor availability (0.143,0.31,0.36,1) (0.143,0.31,0.36, 1) (0.2,0.333,1,1) (1,1,1,1) (1,1,3,5) stable government (1,1,3,5) (0.333,1,1,3) (1,2.872,3.14,7) (0.2,0.333,1,1) (1,1,1,1) table 5 presents the defuzzified weights of the main attributes. geometric means are calculated by using eq. 23. normalized weights are obtained based on eq. 24. defuzzified weights are calculated by using eq. 26. table 5 calculation of defuzzified weights of the main attributes geometric means normalized weights defuzzified weights geographical location (0.508,1.266,1.364,2.141) (0.041,0.193,0.263,0.782) 0.213 cost (1,1.637,1.83,3.956) (0.082,0.249,0.361,1) 0.280 transport connectivity (0.467,0.746,1.019,1.968) (0.038,0.114,0.201,0.719) 0.169 labor availability (0.253,0.423,0.6,1) (0.021,0.064,0.119,0.365) 0.091 stable government (0.508,0.989,1.752,3.201) (0.041,0.151,0.346,1) 0.247 tables 6-10 present the pairwise comparison matrices of the sub-attributes with respect to the main attributes geographical location, cost, transport connectivity, labor availability, and stable government respectively. table 6 pairwise comparison of sub-attributes using hflts w.r.t. geographical location w.r.t. geographical location proximity to urban facilities proximity to disaster areas closeness to other warehouses warehouse security proximity to urban facilities ee betweeen ali and vli betweeen ee and ehi betweeen esli and wli proximity to disaster areas ee betweeen vshi and ahi betweeen whi and eshi closeness to other warehouses ee betweeen esli and eli warehouse security ee from table 6, the trapezoidal fuzzy weights of the sub attributes with respect to geographical location, are obtained as (0.039,0.056,0.086,0.253), (0.246,0.502,0.823,1), (0.03,0.063,0.087,0.192), and (0.068,0.165,0.278,0.761), respectively. ijahp article: boltürk, onar, öztayşi, göztepe, kahraman/ multi-attribute warehouse location selection in humanitarian logistics using hesitant fuzzy ahp international journal of the analytic hierarchy process 287 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.387 table 7 pairwise comparison of sub-attributes using hflts w.r.t. cost w.r.t. cost storage cost investment cost labor price replenishment cost storage cost ee betweeen wli and eli betweeen ehi and eshi whi investment cost ee betweeen eshi and vshi betweeen whi and eshi labor price ee betweeen wli and eli replenishment cost ee from table 7, the trapezoidal fuzzy weights of the sub attributes with respect to cost, are obtained as (0.075,0.207,0.428,0.924), (0.148,0.315,0.778,1), (0.027,0.056,0.126,0.289), and (0.046,0.081,0.185,0.568), respectively. table 8 pairwise comparison of sub-attributes using hflts w.r.t. transport connectivity w.r.t. transport connectivity availability of seaport and airport near to (potential) beneficiaries adequate warehouse facilities adequate warehouse infrastructure availability of seaport and airport ee ehi betweeen ehi and whi betweeen ehi and eshi near to (potential) beneficiaries ee betweeen ee and ehi betweeen ehi and whi adequate warehouse facilities ee betweeen ee and whi adequate warehouse infrastructure ee from table 8, the trapezoidal fuzzy weights of the sub attributes with respect to transport connectivity, are obtained as (0.13,0.266,0.487,1),(0.099,0.206,0.364,0.774), (0.066,0.157,0.276,0.588), and (0.036,0.117,0.214,0.393), respectively. table 9 pairwise comparison of sub-attributes using hflts w.r.t. labor availability w.r.t. labor availability trained and qualified personnel flexible customs regulations population density climate trained and qualified personnel ee betweeen whi and vshi betweeen ee and ehi betweeen eshi and vshi flexible customs regulations ee betweeen eli and ee betweeen ehi and whi population density ee betweeen whi and eshi climate ee from table 9, the trapezoidal fuzzy weights of the sub attributes with respect to labor availability, are obtained as (0.158,0.413,0.546,1), (0.053,0.124,0.198,0.543), (0.091,0.246,0.332,0.778), and (0.028,0.058,0.113,0.276), respectively. ijahp article: boltürk, onar, öztayşi, göztepe, kahraman/ multi-attribute warehouse location selection in humanitarian logistics using hesitant fuzzy ahp international journal of the analytic hierarchy process 288 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.387 table 10 pairwise comparison of sub-attributes using hflts w.r.t. stable government w.r.t. stable government cooperation with logistics agents political and economical stability existence of other agents (ngos) it/ communication cooperation with logistics agents ee betweeen wli and eli betweeen eli and ehi eshi political and economical stability ee betweeen whi and eshi eshi existence of other agents (ngos) ee betweeen ehi and whi it/communication ee from table 10, the trapezoidal fuzzy weights of the sub attributes with respect to stable government, are obtained as (0.077,0.193,0.364,0.791), (0.152,0.334,0.716,1), (0.054,0.114,0.243,0.728), and (0.029,0.058,0.109,0.213), respectively. the next step is to obtain the pairwise comparison matrices of alternatives with respect to each sub-attribute. in our case, there are 20 matrices of such comparisons. due to the space constraints we only present one of them. table 11 gives the pairwise comparison of alternatives using hflts with respect to proximity to urban facilities. table 11 pairwise comparison of alternatives using hflts w.r.t. proximity to urban facilities w.r.t. proximity to urban facilities hlw1 hlw2 hlw3 hlw4 hlw5 hlw1 ee betweeen vli and wli betweeen whi and eshi betweeen esli and wli betweeen whi and vshi hlw2 ee betweeen eshi and vshi betweeen wli and eli betweeen whi and eshi hlw3 ee betweeen vli and esli betweeen esli and wli hlw4 ee eshi hlw5 ee table 12 shows the trapezoidal fuzzy weights of the main and sub-attributes all together. ijahp article: boltürk, onar, öztayşi, göztepe, kahraman/ multi-attribute warehouse location selection in humanitarian logistics using hesitant fuzzy ahp international journal of the analytic hierarchy process 289 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.387 table 12 trapezoidal fuzzy weights of main and sub-attributes main attribute weight sub-attribute weight geographical location (0.041,0.193,0.269,0.782) proximity to urban facilities (0.002,0.011,0.023,0.198) proximity to disaster areas (0.02,0.097,0.221,0.782) closeness to other warehouses (0.001,0.012,0.023,0.15) warehouse security (0.001,0.032,0.075,0.595) cost (0.082,0.249,0.361,1) storage cost (0.006,0.052,0.155,0.924) investment cost (0.012,0.078,0.281,1) labor price (0.002,0.014,0.045,0.289) replenishment cost (0.004,0.02,0.067,0.568) transport connectivity (0.038,0.114,0.201,0.719) availability of seaport and airport (0.005,0.03,0.098,0.719) near to (potential) beneficiaries (0.004,0.023,0.073,0.557) adequate warehouse facilities (0.003,0.018,0.055,0.423) adequate warehouse infrastructure (0.001,0.013,0.043,0.283) labor availability (0.021,0.064,0.119,0.365) trained and qualified personnel (0.003,0.026,0.065,0.365) flexible customs regulations (0.001,0.008,0.024,0.198) population density (0.002,0.016,0.04,0.284) climate (0.001,0.004,0.013,0.101) stable government (0.041,0.151,0.346,1) cooperation with logistics agents (0.003,0.029,0.126,0.791) political and economical stability (0.006,0.05,0.248,1) existence of other agents (ngos) (0.002,0.017,0.084,0.728) it/communication (0.001,0.009,0.038,0.213) table 13 shows the final defuzzified weights of the alternatives with respect to subattributes. according to these results, the prioritization of the alternatives is bursa (0.261) > kocaeli (0.232) > corlu (0.191) > luleburgaz (0.171) > gonen (0.146). turkish humanitarian relief organization should establish the first two warehouses in bursa and then kocaeli. ijahp article: boltürk, onar, öztayşi, göztepe, kahraman/ multi-attribute warehouse location selection in humanitarian logistics using hesitant fuzzy ahp international journal of the analytic hierarchy process 290 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.387 table 13 weights of the alternatives alternative scores sub-attribute bursa corlu gonen kocaeli luleburgaz proximity to urban facilities 0.02 0.038 0.007 0.039 0.015 proximity to disaster areas 0.085 0.093 0.131 0.177 0.037 closeness to other warehouses 0.027 0.016 0.008 0.031 0.005 warehouse security 0.116 0.035 0.016 0.081 0.045 storage cost 0.055 0.068 0.094 0.025 0.189 investment cost 0.134 0.22 0.093 0.07 0.225 labor price 0.028 0.036 0.026 0.022 0.06 replenishment cost 0.108 0.043 0.062 0.092 0.032 availability of seaport and airport 0.13 0.05 0.089 0.144 0.06 near to (potential) beneficiaries 0.102 0.063 0.041 0.106 0.02 adequate warehouse facilities 0.073 0.05 0.034 0.082 0.014 adequate warehouse infrastructure 0.059 0.015 0.025 0.036 0.011 trained and qualified personnel 0.08 0.043 0.028 0.061 0.014 flexible customs regulations 0.013 0.021 0.033 0.006 0.04 population density 0.049 0.032 0.015 0.059 0.008 climate 0.004 0.011 0.019 0.007 0.018 cooperation with logistics agents 0.158 0.093 0.06 0.136 0.034 political and economical stability 0.226 0.171 0.076 0.118 0.174 existence of other agents (ngos) 0.136 0.091 0.052 0.136 0.052 it/communication 0.045 0.021 0.011 0.037 0.023 normalized total score 0.261 0.191 0.146 0.232 0.171 7. conclusion a large number of high-impact natural and man-made disasters, such as floods, earthquakes, storms, and civil disturbance or war have occurred in dissimilar parts of the world in recent years. this condition triggered the need for well-organized stockholding for humanitarian purposes. recent studies in mcdm show that hesitant fuzzy set (hfs) exposes a new viewpoint on fuzzy decision making. contrary to ordinary fuzzy sets, hfss characterize fuzziness by setting out all the possible values while assigning the membership degree of the elements of a set. based on these new arguments, we proposed a hesitant fuzzy multi-attribute method to solve hl warehouse location selection problems. our model based on hesitant fuzzy ahp successfully evaluated alternative hl warehouse location alternatives. we extended buckley’s ordinary fuzzy ahp method to its hesitant fuzzy version since the other fuzzy ahp have been seriously criticized in the literature methods (chang, 1996; van laarhoven and pedrycz 1983; etc.). hesitant linguistic term sets provided the flexibility to elicit comparative linguistic expressions by using context-free grammar. the fuzzy linguistic scale is based on the ahp’s classical 1-9 scale. our hierarchy included 5 main attributes, 16 ijahp article: boltürk, onar, öztayşi, göztepe, kahraman/ multi-attribute warehouse location selection in humanitarian logistics using hesitant fuzzy ahp international journal of the analytic hierarchy process 291 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.387 sub-attributes and 5 hl warehouse location alternatives. this research yielded several useful managerial insights. first, the proposed method allows one to deal with incomplete information due to the vagueness of the criteria such as “cooperation with logistics agents” and “political and economic stability”. second, it provides a systematic approach for solving complex problems that involve many stakeholders. this systematic approach can be used to solve other decision making problems with the same characteristics. a limitation of the proposed method is the cumbersome calculations that are involved. however, this difficulty can be overcome by developing software for this purpose. for further research, other extensions of fuzzy sets such as intuitionistic fuzzy sets, type-2 fuzzy sets, multi fuzzy sets, etc. can be used in the proposed model above. the obtained results can be compared with our results. ijahp article: boltürk, onar, öztayşi, göztepe, kahraman/ multi-attribute warehouse location selection in humanitarian logistics using hesitant fuzzy ahp international journal of the analytic hierarchy process 292 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.387 references ahmadi, m., seifi, a., tootooni, b. (2015). a humanitarian logistics model for disaster relief operation considering network failure and standard relief time: a case study on san francisco district. transportation research part e: logistics and transportation review, 75, 145-163. doi: http://dx.doi.org/10.1016/j.tre.2015.01.008 alberto, p. (2000). the logistics of industrial location decisions: an application of the analytic hierarchy process. international journal of logistics research and applications, 3(3), 273–289. ashrafzadeh, m., rafiei, f. m., isfahani, n. m., zare, z. (2012). application of fuzzy topsis method for the selection of warehouse location: a case study. interdisciplinary journal of contemporary research in business, 3(9), 655-671. atkinson, m.a., bayazit, o., karpak, b. (2015). a case study using the analytic hierarchy process for it outsourcing decision making. international journal of information systems and supply chain management, 8(1), 60-84. doi: 10.4018/ijisscm.2015010104 balcik, b., & beamon, b. m. (2008). facility location in humanitarian relief. international journal of logistics, 11(2), 101-121. barbarosoglu, g., arda, y., (2004). a two-stage stochastic programming framework for transportation planning in disaster response. journal of the operational research society, 55(1), 43-53. doi:10.1057/palgrave.jors.2601652 blocher, e., chen, k. h., lin, t. w. (2008). cost management: a strategic emphasis. mcgraw-hill/irwin. bottani, e., rizzi, a. (2006). a fuzzy topsis methodology to support outsourcing of logistics services. supply chain management, 11(4), 294-308. doi: http://dx.doi.org/10.1108/13598540610671743 buckley, j.j. (1985). fuzzy hierarchical analysis. fuzzy sets and systems, 17, 233– 247. doi:10.1016/0165-0114(85)90090-9 campbell, a. m., jones, p. c. (2011). prepositioning supplies in preparation for disasters. european journal of operational research, 209(2), 156-165. doi: http://dx.doi.org/10.1016/j.ejor.2010.08.029 cevik onar s., oztaysi b., otay i̇., kahraman c. (2015). multi-expert wind energy technology selection using interval-valued intuitionistic fuzzy sets. energy, 90, 274285. doi: http://dx.doi.org/10.1016/j.energy.2015.06.086 chakravarty, a. k. (2014). humanitarian relief chain: rapid response under uncertainty. international journal of production economics, 151, 146-157. doi: http://dx.doi.org/10.1016/j.ijpe.2013.10.007 chanas, s., kuchta, d. (1996). a concept of the optimal solution of the transportation problem with fuzzy cost coefficients. fuzzy sets and systems, 82 (3), 299-305. doi:10.1016/0165-0114(95)00278-2 http://dx.doi.org/10.1016/j.tre.2015.01.008 http://www.google.com.tr/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0ahukewie2qxssrboahumosakheyvclcqfggdmaa&url=http%3a%2f%2fendnote.com%2fdownloads%2fstyle%2finternational-journal-logistics-research-and-applications&usg=afqjcnhir17janup8tt-xu6trapcfvnfqg&bvm=bv.129389765,d.d2s http://www.google.com.tr/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0ahukewie2qxssrboahumosakheyvclcqfggdmaa&url=http%3a%2f%2fendnote.com%2fdownloads%2fstyle%2finternational-journal-logistics-research-and-applications&usg=afqjcnhir17janup8tt-xu6trapcfvnfqg&bvm=bv.129389765,d.d2s http://dx.doi.org/10.1108/13598540610671743 http://dx.doi.org/10.1016/0165-0114(85)90090-9 http://dx.doi.org/10.1016/j.ejor.2010.08.029 http://dx.doi.org/10.1016/j.energy.2015.06.086 http://dx.doi.org/10.1016/j.ijpe.2013.10.007 http://dx.doi.org/10.1016/0165-0114(95)00278-2 ijahp article: boltürk, onar, öztayşi, göztepe, kahraman/ multi-attribute warehouse location selection in humanitarian logistics using hesitant fuzzy ahp international journal of the analytic hierarchy process 293 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.387 chang, d.y. (1996). applications of the extent analysis method on fuzzy ahp. european journal of operational research, 95, 649–655. doi:10.1016/03772217(95)00300-2 choi, t. y., dooley, k. j., rungtusanatham, m. (2001). supply networks and complex adaptive systems: control versus emergence. journal of operations management, 19(3), 351-366. doi: http://dx.doi.org/10.1016/s0272-6963(00)00068-1 chou, j. s., yang, k. h., ren, t. c. (2015). ex-post evaluation of preparedness education in disaster prevention, mitigation and response. international journal of disaster risk reduction, 12, 188-201. doi: http://dx.doi.org/10.1016/j.ijdrr.2015.01.002 cozzolino, a. (2012). humanitarian logistics and supply chain management. in humanitarian logistics (pp. 5-16). springer berlin heidelberg. doi: 10.1007/978-3642-30186-5_2 dangol, r., bahl, m., karpak, b. (2015). timing cooperative relationships with sequential capability development process to reduce capability development tradeoffs. international journal of production economics, 169, 179-189. doi: http://dx.doi.org/10.1016/j.ijpe.2015.07.014 dekle, j., lavieri, m.s., martin, e., emir-farinas, h., francis, r.l. (2005). a florida country locates disaster recovery centres. interfaces 35, 133–139. demirel, t., demirel, n. ç., kahraman, c. (2010). multi-attributes warehouse location selection using choquet integral. expert systems with applications, 37(5), 3943-3952. doi: http://dx.doi.org/10.1016/j.eswa.2009.11.022 díaz-delgado, c., gaytán iniestra, j. (2014). flood risk assessment in humanitarian logistics process design. journal of applied research and technology, 12(5), 976984. doi: http://dx.doi.org/10.1016/s1665-6423(14)70604-2 filev, d. and yager, r.r. (1998). on the issue of obtaining owa operator weights, fuzzy sets and systems, 94(2), 157-169. doi:10.1016/s0165-0114(96)00254-0 florez, j. v., lauras, m., okongwu, u., dupont, l. (2015). a decision support system for robust humanitarian facility location. engineering applications of artificial intelligence, , 46, part b, 326-335. doi: http://dx.doi.org/10.1016/j.engappai.2015.06.020 gabel, m. j. (1998). the endurance of supranational governance: a consociational interpretation of the european union. comparative politics, 463-475. doi 10.2307/422334 gralla, e., goentzel, j., chomilier, b. (2015). case study of a humanitarian logistics simulation exercise and insights for training design, journal of humanitarian logistics and supply chain management, 5(1), 113-138. doi: http://dx.doi.org/10.1108/jhlscm-01-2014-0001 guha-sapir, d., hoyois, p., and below, r., (2013). annual disaster statistical review 2013, the numbers and trends, 1-41. http://dx.doi.org/10.1016/0377-2217(95)00300-2 http://dx.doi.org/10.1016/0377-2217(95)00300-2 http://dx.doi.org/10.1016/s0272-6963(00)00068-1 http://dx.doi.org/10.1016/j.ijdrr.2015.01.002 http://dx.doi.org/10.1016/j.ijpe.2015.07.014 http://dx.doi.org/10.1016/j.eswa.2009.11.022 https://www.scopus.com/authid/detail.uri?authorid=6506956166&amp;eid=2-s2.0-84919779039 https://www.scopus.com/authid/detail.uri?authorid=56460374900&amp;eid=2-s2.0-84919779039 https://www.scopus.com/source/sourceinfo.uri?sourceid=21100201949&origin=recordpage http://dx.doi.org/10.1016/s1665-6423(14)70604-2 http://dx.doi.org/10.1016/s0165-0114(96)00254-0 http://dx.doi.org/10.1016/j.engappai.2015.06.020 https://www.scopus.com/authid/detail.uri?authorid=56003369000&amp;eid=2-s2.0-84958763797 https://www.scopus.com/authid/detail.uri?authorid=6504699848&amp;eid=2-s2.0-84958763797 https://www.scopus.com/authid/detail.uri?authorid=57130996400&amp;eid=2-s2.0-84958763797 https://www.scopus.com/record/display.uri?eid=2-s2.0-84958763797&origin=resultslist&sort=plf-f&src=s&st1=humanitarian+logistics&nlo=&nlr=&nls=&sid=d276c8a523ef00fb07591423c52c0f7f.wlw7nkkc52nnqnxjqaqrla%3a10&sot=b&sdt=b&sl=29&s=title%28humanitarian+logistics%29&relpos=20&citecnt=1&searchterm=#corrauthorfooter https://www.scopus.com/source/sourceinfo.uri?sourceid=21100446510&origin=recordpage https://www.scopus.com/source/sourceinfo.uri?sourceid=21100446510&origin=recordpage http://dx.doi.org/10.1108/jhlscm-01-2014-0001 ijahp article: boltürk, onar, öztayşi, göztepe, kahraman/ multi-attribute warehouse location selection in humanitarian logistics using hesitant fuzzy ahp international journal of the analytic hierarchy process 294 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.387 heo, e., kim, j., boo, k.-j. (2010). analysis of the assessment factors for renewable energy dissemination program evaluation using fuzzy ahp. renewable and sustainable energy reviews, 14 (8), 2214-2220. doi: http://dx.doi.org/10.1016/j.rser.2010.01.020 hsieh, c.h. and chen, s.h. (1999). a model and algorithm of fuzzy product positioning. information sciences, 121, 61–82. doi: http://dx.doi.org/10.1016/s00200255(99)00050-x huang, s., wang, q., batta, r., nagi, r. (2015). an integrated model for site selection and space determination of warehouses. computers & operations research, 62, 169-176. doi: http://dx.doi.org/10.1016/j.cor.2014.10.015 ivgin, m. (2013). the decision-making models for relief asset management and interaction with disaster mitigation. international journal of disaster risk reduction, 5, 107-116. doi: http://dx.doi.org/10.1016/j.ijdrr.2013.08.005 kahraman c, beskese a., kaya i. (2010). selection among erp outsourcing alternatives using a fuzzy multi-criteria decision making methodology. international journal of production research 48(2), 547-566. doi: 10.3233/ifs-151722 kahraman c., çevik onar s., öztayşi b. (2015). engineering economic analyses using intuitionistic and hesitant fuzzy sets, journal of intelligent & fuzzy systems, 29(3), 1151-1168. kahraman, c., ates, n. y., çevik, s., gülbay, m. (2007). fuzzy multi‐attribute cost– benefit analysis of e‐services. international journal of intelligent systems, 22(5), 547-565. doi: 10.1002/int.20213 kahraman, c., öztayşi, b., çevik onar, s. (2016). a comprehensive literature review of 50 years of fuzzy set theory. international journal of computational intelligence systems 9(1), 3-24. doi: http://dx.doi.org/10.1080/18756891.2016.1180817 kashi, k., franek, j. (2016). ahp in personnel management: can the key competencies change with company’s strategy?. international journal of analytic hierarchy process, 8(1), 39-52. doi: http://dx.doi.org/10.13033/ijahp.v8i1.297 kaya i., öztayşi, b., kahraman c. (2012). a two-phased fuzzy multicriteria selection among public transportation investments for policy-making and risk governance. international journal of uncertainty, fuzziness and knowledge-based systems, 20(31), 31-48. doi: http://dx.doi.org/10.1142/s021848851240003x leiras, a., de brito, i., jr., queiroz peres, e., rejane bertazzo, t., tsugunobu yoshida yoshizaki, h. (2014). literature review of humanitarian logistics research: trends and challenges. journal of humanitarian logistics and supply chain management, 4(1), 95-130. doi: http://dx.doi.org/10.1108/jhlscm-04-2012-0008 l'hermitte, c., tatham, p., bowles, m., brooks, b. (2016). developing organisational capabilities to support agility in humanitarian logistics: an exploratory study. journal of humanitarian logistics and supply chain management , 6(1), 7299. doi : http://dx.doi.org/10.1108/jhlscm-02-2015-0006 http://dx.doi.org/10.1016/j.rser.2010.01.020 http://dx.doi.org/10.1016/s0020-0255(99)00050-x http://dx.doi.org/10.1016/s0020-0255(99)00050-x http://dx.doi.org/10.1016/j.cor.2014.10.015 http://dx.doi.org/10.1016/j.ijdrr.2013.08.005 http://www.worldscientific.com/author/kahraman%2c+cengiz http://www.worldscientific.com/author/oztaysi%2c+basar http://www.worldscientific.com/author/cevik+onar%2c+sezi http://dx.doi.org/10.1080/18756891.2016.1180817 http://dx.doi.org/10.13033/ijahp.v8i1.297 http://dx.doi.org/10.1142/s021848851240003x https://www.scopus.com/authid/detail.uri?authorid=21934110800&amp;eid=2-s2.0-84929164235 https://www.scopus.com/authid/detail.uri?authorid=6507093265&amp;eid=2-s2.0-84929164235 https://www.scopus.com/authid/detail.uri?authorid=57130955100&amp;eid=2-s2.0-84929164235 https://www.scopus.com/authid/detail.uri?authorid=57131055400&amp;eid=2-s2.0-84929164235 https://www.scopus.com/authid/detail.uri?authorid=57131081300&amp;eid=2-s2.0-84929164235 https://www.scopus.com/authid/detail.uri?authorid=57131081300&amp;eid=2-s2.0-84929164235 https://www.scopus.com/source/sourceinfo.uri?sourceid=21100446510&origin=recordpage https://www.scopus.com/source/sourceinfo.uri?sourceid=21100446510&origin=recordpage https://www.scopus.com/authid/detail.uri?authorid=57188550272&amp;eid=2-s2.0-84961621937 https://www.scopus.com/authid/detail.uri?authorid=36142313900&amp;eid=2-s2.0-84961621937 https://www.scopus.com/authid/detail.uri?authorid=55575292900&amp;eid=2-s2.0-84961621937 https://www.scopus.com/authid/detail.uri?authorid=7201573030&amp;eid=2-s2.0-84961621937 https://www.scopus.com/source/sourceinfo.uri?sourceid=21100446510&origin=recordpage ijahp article: boltürk, onar, öztayşi, göztepe, kahraman/ multi-attribute warehouse location selection in humanitarian logistics using hesitant fuzzy ahp international journal of the analytic hierarchy process 295 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.387 li, y., liu, x., chen, y. (2011). selection of logistics centre location using axiomatic fuzzy set and topsis methodology in logistics management. expert systems with applications, 38, 7901–7908. doi: http://dx.doi.org/10.1016/j.eswa.2010.12.161 liu, h., rodriguez, r.m. (2014). a fuzzy envelope for hesitant fuzzy linguistic term set and its application to multicriteria decision making. information sciences, 258, 220–238. doi: http://dx.doi.org/10.1016/j.ins.2013.07.027 olazabal, m., pascual, u. (2016). use of fuzzy cognitive maps to study urban resilience and transformation. environmental innovation and societal transitions, 18, 18-40. doi: http://dx.doi.org/10.1016/j.eist.2015.06.006 onut, s., soner, s. (2007). transhipment site election using the ahp and topsis approaches under fuzzy environment. waste management, 28 (9), 1552–1559. doi: http://dx.doi.org/10.1016/j.wasman.2007.05.019 özcan, t., çelebi, n., esnaf, ş. (2011). comparative analysis of multi-attributes decision making methodologies and implementation of a warehouse location selection problem. expert systems with applications, 38(8), 9773-9779. doi: http://dx.doi.org/10.1016/j.eswa.2011.02.022 özdamar, l., ertem, m.a. (2015). models, solutions and enabling technologies in humanitarian logistics. european journal of operational research, 244(1), 55-65. doi: http://dx.doi.org/10.1016/j.ejor.2014.11.030 oztaysi b., cevik onar s., bolturk e., kahraman c. (2015). hesitant fuzzy analytic hierarchy process. 2015 ieee international conference on fuzzy systems (fuzzieee), istanbul, 1-7. doi: 10.1109/fuzz-ieee.2015.7337948 oztaysi b., cevik onar s., kahraman c. (2016). fuzzy multicriteria prioritization of urban transformation projects for istanbul. journal of intelligent & fuzzy systems, 30 (4), 2459-2474. doi: 10.3233/ifs-152016 pazour, j. a., carlo, h. j. (2015). warehouse reshuffling: insights and optimization. transportation research part e: logistics and transportation review, 73, 207-226. powers, r. (1989). optimization models for logistics decisions. journal of business logistics 10 (1), 106–121. doi: http://dx.doi.org/10.1016/j.tre.2014.11.002 ransikarbum, k., mason, s.j. (2016). multiple-objective analysis of integrated relief supply and network restoration in humanitarian logistics operations. international journal of production research, 54(1), 49-68. doi: http://dx.doi.org/10.1080/00207543.2014.977458 rath, s., gutjahr, w. j. (2014). a math-heuristic for the warehouse location–routing problem in disaster relief. computers & operations research, 42, 25-39. doi: http://dx.doi.org/10.1016/j.cor.2011.07.016 rawls, c.g., turnquist, m.a. (2010). pre-positioning of emergency supplies for disaster response. transportation research. part b, 44, 521–534. doi: http://dx.doi.org/10.1016/j.trb.2009.08.003 http://dx.doi.org/10.1016/j.eswa.2010.12.161 http://dx.doi.org/10.1016/j.ins.2013.07.027 http://dx.doi.org/10.1016/j.eist.2015.06.006 http://dx.doi.org/10.1016/j.wasman.2007.05.019 http://dx.doi.org/10.1016/j.eswa.2011.02.022 https://www.scopus.com/authid/detail.uri?authorid=7004162696&amp;eid=2-s2.0-84930479121 https://www.scopus.com/authid/detail.uri?authorid=36023330600&amp;eid=2-s2.0-84930479121 https://www.scopus.com/source/sourceinfo.uri?sourceid=22489&origin=recordpage http://dx.doi.org/10.1016/j.ejor.2014.11.030 http://dx.doi.org/10.1109/fuzz-ieee.2015.7337948 http://dx.doi.org/10.1016/j.tre.2014.11.002 https://www.scopus.com/authid/detail.uri?authorid=56416407700&amp;eid=2-s2.0-84954372836 https://www.scopus.com/authid/detail.uri?authorid=7202152323&amp;eid=2-s2.0-84954372836 https://www.scopus.com/source/sourceinfo.uri?sourceid=27656&origin=recordpage https://www.scopus.com/source/sourceinfo.uri?sourceid=27656&origin=recordpage http://dx.doi.org/10.1080/00207543.2014.977458 http://dx.doi.org/10.1016/j.cor.2011.07.016 http://dx.doi.org/10.1016/j.trb.2009.08.003 ijahp article: boltürk, onar, öztayşi, göztepe, kahraman/ multi-attribute warehouse location selection in humanitarian logistics using hesitant fuzzy ahp international journal of the analytic hierarchy process 296 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.387 rodriguez, r. m., martinez, l., torra, v., xu, z. s., herrera, f. (2014). hesitant fuzzy sets: state of the art and future directions. international journal of intelligent systems, 29, 495–524. doi: 10.1002/int.21654 rodríguez, r.m., martínez, l. and herrera f. (2012). hesitant fuzzy linguistic term sets for decision making. ieee transactions on fuzzy systems, 20, 109–119. doi: 10.1109/tfuzz.2011.2170076 roh, s. y., jang, h. m., han, c. h. (2013). warehouse location decision factors in humanitarian relief logistics. the asian journal of shipping and logistics, 29(1), 103-120. doi: http://dx.doi.org/10.1016/j.ajsl.2013.05.006 roh, s., pettit, s., harris, i., beresford, a. (2015). the pre-positioning of warehouses at regional and local levels for a humanitarian relief organisation. international journal of production economics, 170, part b, 616-628. doi: http://dx.doi.org/10.1016/j.ijpe.2015.01.015 russell, t. e. (2005). the humanitarian relief supply chain: analysis of the 2004 south east asia earthquake and tsunami. doctoral dissertation, massachusetts institute of technology. saaty t.l. (1980). the analytic hierarchy process. new york: mcgraw-hill. doi: http://dx.doi.org/10.1080/00137918308956077 sahoo, n.k., mohanty, b.s., tripathy, p.k. (2016). fuzzy inventory model with exponential demand and time-varying deterioration. global journal of pure and applied mathematics, 12(3), 2573–2589. sarkis, j., sundarraj, r.p. (2002). hub location at digital equipment corporation: a comprehensive analysis of qualitative and quantitative factors. european journal of operational research, 137, 336–347. doi: http://dx.doi.org/10.1016/s03772217(01)00138-2 seaman, j. (1999). malnutrition in emergencies: how can we do better and where do the responsibilities lie? disasters, 23(4), 306–315. doi: 10.1111/1467-7717.00120 shahriari, m. (2011). mapping fuzzy approach in engineering economics. international research journal of finance and economics, 81, 6-12. shqair, m., altarazi, s., al-shihabi, s. (2014). a statistical study employing agentbased modeling to estimate the effects of different warehouse parameters on the distance traveled in warehouses. simulation modelling practice and theory, 49, 122135. doi: http://dx.doi.org/10.1016/j.simpat.2014.08.002 stock, j. r., lambert, d. m. (2001). strategic logistics management (vol. 4). boston, ma: mcgraw-hill/irwin. tan, r.r., aviso, k.b., huelgas, a.p., promentilla, m.a.b. (2014). fuzzy ahp approach to selection problems inprocess engineering involving quantitative andqualitative aspects. process safety and environmental protection, 92, 467–475. doi: http://dx.doi.org/10.1016/j.psep.2013.11.005 http://dx.doi.org/10.1109/tfuzz.2011.2170076 http://dx.doi.org/10.1016/j.ajsl.2013.05.006 http://dx.doi.org/10.1016/j.ijpe.2015.01.015 http://dx.doi.org/10.1080/00137918308956077 http://dx.doi.org/10.1016/s0377-2217(01)00138-2 http://dx.doi.org/10.1016/s0377-2217(01)00138-2 http://dx.doi.org/10.1016/j.simpat.2014.08.002 http://dx.doi.org/10.1016/j.psep.2013.11.005 ijahp article: boltürk, onar, öztayşi, göztepe, kahraman/ multi-attribute warehouse location selection in humanitarian logistics using hesitant fuzzy ahp international journal of the analytic hierarchy process 297 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.387 thomas, a., kopczak, l. (2005). from logistics to supply chain management: the path forward in the humanitarian sector. white paper, san francisco, ca: fritz institute. tofighi, s., torabi, s.a., mansouri, s.a. (2016). humanitarian logistics network design under mixed uncertainty. european journal of operational research, 250(1), 239-250. doi: http://dx.doi.org/10.1016/j.ejor.2015.08.059 tomasini, r. m., van wassenhove, l. n. (2009). from preparedness to partnerships: case study research on humanitarian logistics. international transactions in operational research, 16(5), 549-559. doi: 10.1111/j.1475-3995.2009.00697.x torra, v. (2010). hesitant fuzy sets. international journal of inteligent systems, 25, 529-539. tuzkaya, u. r., önüt, s. (2009). a holonic approach based integration methodology for transportation and warehousing functions of the supply network. computers & industrial engineering, 56(2), 708-723. doi: http://dx.doi.org/10.1016/j.cie.2007.09.003 ucal sari, i., oztaysi, b., kahraman, c. (2013). fuzzy analytic hierarchy process using type-2 fuzzy sets: an application to warehouse location selection, in doumpos and grigoroudis (ed.), multiattribute decision aid and artificial intelligence, (pp. 285-308). john wiley & sons. doi: 10.1002/9781118522516.ch12 vaillancourt, a. (2016). a theoretical framework for consolidation in humanitarian logistics. journal of humanitarian logistics and supply chain management, 6(1), 223. doi: http://dx.doi.org/10.1108/jhlscm-01-2015-0001 van laarhoven, p.j.m., pedrycz, w. (1983). a fuzzy extension of saaty’s priority theory. fuzzy sets and systems, 11, 199–227. doi:10.1016/s0165-0114(83)80082-7 van wassenhove, l. n. (2006). blackett memorial lecture. humanitarian aid logistics: supply chain management in high gear. journal of the operational research society, 57(5), 475–489. van wassenhove, l. n., pedraza martinez, a. j. (2012). using or to adapt supply chain management best practices to humanitarian logistics. international transactions in operational research, 19(1-2), 307-322. vega, d., roussat, c. (2015). humanitarian logistics: the role of logistics service providers. international journal of physical distribution and logistics management, 45(4),352-375.doi: http://dx.doi.org/10.1108/ijpdlm-12-2014-0309 vitoriano, b., ortuño, m. t., tirado, g., montero, j. (2011). a multi-attributes optimization model for humanitarian aid distribution. journal of global optimization, 51(2), 189-208. doi: 10.1007/s10898-010-9603-z xia, m.m., xu, z.s.(2011) hesitant fuzzy information aggregation in decision making. international journal of approximate reasoning, 52, 395–407. doi:10.1016/j.ijar.2010.09.002 zadeh, l.a. (1965). fuzzy sets. information and control, 8, 338-353. doi:10.1016/s0019-9958(65)90241-x https://www.scopus.com/authid/detail.uri?authorid=35318773400&amp;eid=2-s2.0-84949934887 https://www.scopus.com/authid/detail.uri?authorid=8625505200&amp;eid=2-s2.0-84949934887 https://www.scopus.com/authid/detail.uri?authorid=19639023300&amp;eid=2-s2.0-84949934887 https://www.scopus.com/source/sourceinfo.uri?sourceid=22489&origin=recordpage http://dx.doi.org/10.1016/j.ejor.2015.08.059 http://dx.doi.org/10.1016/j.cie.2007.09.003 https://www.scopus.com/source/sourceinfo.uri?sourceid=21100446510&origin=recordpage http://dx.doi.org/10.1108/jhlscm-01-2015-0001 http://dx.doi.org/10.1016/s0165-0114(83)80082-7 https://www.scopus.com/authid/detail.uri?authorid=56200317100&amp;eid=2-s2.0-84929146916 https://www.scopus.com/authid/detail.uri?authorid=25925426200&amp;eid=2-s2.0-84929146916 https://www.scopus.com/source/sourceinfo.uri?sourceid=144922&origin=recordpage http://dx.doi.org/10.1108/ijpdlm-12-2014-0309 http://dx.doi.org/10.1016/j.ijar.2010.09.002 http://dx.doi.org/10.1016/s0019-9958(65)90241-x ijahp article: boltürk, onar, öztayşi, göztepe, kahraman/ multi-attribute warehouse location selection in humanitarian logistics using hesitant fuzzy ahp international journal of the analytic hierarchy process 298 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.387 zhang, n., wei, g. (2013). extension of vikor method for decision making problem based on hesitant fuzzy set. applied mathematical modelling, 37, 4938– 4947. doi: http://dx.doi.org/10.1016/j.apm.2012.10.002 http://dx.doi.org/10.1016/j.apm.2012.10.002 microsoft word 3adamus_pp_83-108_vol i_issue_2_2009.docx ijahp article: adamus/when should poland enter the euro zone? international journal of the 83 vol. 1 issue 2 2009 issn 1936-6744 analytic hierarchy process when should poland enter the euro zone? decision making with the anp wiktor adamus jagiellonian university institute of economics and management gronostajowa 3, 30 – 387 krakow, poland adamus@uj.edu.pl homo supermatricus – the humans of the future will make momentous decisions this way with the supermatrix -t.l. saaty abstract the paper presents a solution to a real, current and significant problem for poland. poland, as a new eu country should join the economic and monetary union – the euro zone – after fulfilling the convergence criteria set by the maastricht treaty, as soon as possible. entering the zone, it should gain in many economic, social and political fields. we can therefore look for new opportunities. however, entering the euro zone also results in costs and risks. the paper aims to find such an alternative, which would maximize benefits and opportunities, and minimize costs and risks. the alternatives were developed – early, medium-term and late entry of poland’s into the euro zone. the problem was solved by use of analytic network process. whether using the multiplicative or the additive/negative mathematical formulae, the conclusions show, without any doubt, that poland should enter the euro zone late, that is, after 2011. keywords: euro zone, convergence indicators, analytic network process (anp) 1. introduction january 1, 2002 brought the european union into life – 300 million eu inhabitants (austria, belgium, finland, france, germany, greece, holland, ireland, italy, luxembourg, portugal and spain) received a new euro currency. on may 1, 2004, 10 subsequent countries (cyprus, the czech republic, estonia, hungary, latvia, lithuania, malta, poland, the slovak republic and slovenia); and on january 1, 2007, two more countries, bulgaria and romania, committed themselves to enter the monetary union as soon as possible after they fulfilled convergence criteria. according to the maastricht treaty of february 2, 1992, all eu member countries shall, as the end result, enter the economic and currency union. entering countries had or have to accept the treaty without any conditions. only great britain and denmark had not entered the euro zone, but they have an “opt-out”. sweden did not manage to get social acceptance on giving up their “crown” and accepting euro in 2003 by referendum, without defining the date of its entry into the currency union. in the euro line, bulgaria and romania have been members since january 1, 2007. poland’s entering the euro zone is the subject of many academic conferences (a. nowak, a. stępniak, 2003; strategies 2002, positive and negative sides 2006), analyses in the form of articles and books, newspapers articles and journals, especially rzeczpospolita, gazeta wyborcza and wprost. one internet browser search yielded 16,600 items for “poland’s entry into the euro zone”. rob typewritten text http://dx.doi.org/10.13033/ijahp.v1i2.55 ijahp article: adamus/when should poland enter the euro zone? international journal of the 84 vol. 1 issue 2 2009 issn 1936-6744 analytic hierarchy process poland is divided on the issue. for example, monetary policy council and its president prof. leszek balcerowicz favor a quick entry, claiming it will accelerate its economy (positive sides,. 2006). according to balcerowicz, poland’s entry into the monetary union will have a positive impact on: inflation control, low, long term interest rates, better price comparability in euros, transaction prices costs reduction in the foreign exchange transactions, lower margins on the money borrowed on international market l. balcerowicz claims poland loses 0.2% of its economic growth by postponing its entry into the euro zone. similar opinion is shared by a.s. bratkowski, pekao bank main economist (poland in the euro zone …, 2006). however, he does not mention costs and risks related to this entry. also, there are no detailed calculations related to the loss. prof. jakub mazur offers a contrary opinion. mazur identifies poland’s entry into the euro zone with the loss of national identity (mazur 2006). in his article “arguments for and against the economic and monetary union”, mazur (2002) comments on 14 points presented by the propagators of poland’s entry into the economic and currency union and 14 points given by the opponents of the euro zone entry. as a conclusion he divides polish economists into groups of supporters – those who are for the very fast membership in the euro zone (for example a. bratkowski, k. jakubiszyn) and those who suggest a slow or medium-term entering process (e.g. e. pietrzak, d. rosati). mazur supports the idea to keep polish zloty without entering the euro zone. similarly gary s. becker – noble prize winner in the field of economy in 1992 r. discourages poland from entering euro zone (bureaucrats' currency, 2006). also skeptical towards euro currency were paul de grauwe – one of the euro fathers, and the late milton friedman – nobel prize winner in the field of economy (1976), the most widely known 20th century economist (piński & piński 2006). bogdan borusewicz, poland’s senate president (positive sides ....2006) who seems to hold the most balanced opinion on that issue, claiming that it would be politically irresponsible on the part of the government if the planned entrance were to take place before poland reaches a direct level of economic convergence. this decision could result in enormous social costs and the threat of its fast exclusion from the euro zone. he cites italy, where the costs of remaining in the euro zone are higher than exit costs. the business centre club supports fast entry into the euro zone, its opinion motivated by the foreign investments growth and accelerating the economic growth ratio. poland president lech kaczyński, together with minister of finance zyta gilowska, support the referendum on the date (undetermined) of poland’s entry into the euro zone. because of a significant economic distance between poland and the former 15 “old” eu members (amounting to 20 to 30 years in relation to the most developed countries), a decision on entering the euro zone will take into account the following four merits: benefits poland shall obtain after entering the monetary union costs we shall bear in relation to preparation for and presence in the euro zone extra benefits, that is opportunities to be drawn from our presence in the monetary union extra costs, that is risks related to our entry into the euro zone ijahp article: adamus/when should poland enter the euro zone? international journal of the 85 vol. 1 issue 2 2009 issn 1936-6744 analytic hierarchy process this paper aims to estimate those parameters. sooner or later, poland has to enter the euro zone. the open question is the timing. therefore, a decision on certain alternatives shall be made. however, each decision on the issue has elements of discovery, irrational incidentally, economic, social, political, organizational, and managerial and other results. the practice to undertake decisions concentrates on weighing alternatives to fulfill a set of desired objectives. a decision means a choice of one of them. in each decisional problem there is at least one optimal decision where it can be objectively stated that there is no other, better decision, keeping the decision process neutral. the problem is to choose that alternative that fulfils a comprehensive objectives’ set. poland’s entry into the euro zone is a multicriteria problem, which requires the participation of many players. the behavior and opinions of various participants follow differences in reality perception and processes involved in this reality. they also follow the fact that each person represents a separate world of values, and the participants’ viewpoints are based on various, often conflicting, value systems, related points of view assumed at assessment, and a different reality perception. that leads to making multicriteria decisions on poland’s entry into the euro zone. as far as solving multicriteria, the literature points to various methods (figueria, greco & ehrgott, eds. 2005), but the best ones are ahp, anp (adamus & gręda 2005) and they were used in this paper to make a decision on the optimal timing of poland’s entry into the economic and monetary union. following prof. a. stępniak (zone, 2003) the article will present three different variants of poland’s entry into the euro zone, providing other entry dates and countries that can enter the monetary union together with poland. • early entry into the euro zone (in 2008/2009), together with the following countries: cyprus, malta, the slovak republic; • medium-term entry into the euro zone (in 2010/2011), with the following countries: lithuania, latvia and estonia; • late entry into the euro zone (after 2011), with the following countries: czech republic, hungary and perhaps bulgaria and romania. those alternatives shall be analyzed in the context of benefits, costs, opportunities and risks. poland’s entry into the euro zone shall be given a holistic approach accounting for all the parameters. poland shall enter the euro zone when the relation between benefits and opportunities exceeds costs and risks. the problem thus defined shall be solved with the use of multicriteria decision support method anp – analytic network process (saaty, 2001), (figueria, greco & ehrgott, eds. 2005). poland and eu members (with exception of great britain and denmark) must be ready to enter the monetary union after fulfilling the following conditions: • conditions for economy stabilization convergence • institutional adjustments, • european union acceptance. ijahp article: adamus/when should poland enter the euro zone? international journal of the 86 vol. 1 issue 2 2009 issn 1936-6744 analytic hierarchy process 2. conditions for euro zone membership: convergence criteria defined by maastricht treaty the european union treaty signed on february 7, 1992 in maastricht set to determine macroeconomic criteria provided below, which shall be used to evaluate eu member states aspiring to the monetary union. the convergence (approximation) criteria have to be fulfilled by all states in the euro zone not only nominally but also in real sense, that is to say they must have effective economy in a relatively long period of time. • annual budget deficit (of public finance) – percentage of planned or real gdp deficit measured in market prices shall not exceed 3% annually. • public debt – the share of debt in gdp in the year preceding the analysis, measured in market prices shall not exceed 60% of gdp. exceeding those two most important criteria in 2006 by hungary (budget deficit amounted to 4% of gdp, and the economic growth has not exceeded 4% of gdp) moved the country’s entry into the euro zone to the years 2010 to 2014. • inflation rate – should be lowered than 1.5 % over its average rate in the three eu countries with lowest inflation rate (2.8% in 2006). • long term interest rate –should not be higher than the average in three countries with the lowest inflation level increased by 2% (approximately 6% in 2006). interest rates shall be measured based on the long term state t-bonds or other comparable instruments, taking into account differences in national deficits. • stable currency exchange rates – for two years before the euro zone entry, the national currency must remain in erm ii (european currency exchange rates) where differences cannot be larger than +/15 % from the central parity. poland, like other euro zone candidates, cannot count for any preferences in this respect. interpretation of convergence criteria for candidate countries is stricter than the one used for the current euro zone members. an\ example is lithuania, which was not accepted to the monetary union. it has exceeded the inflation rate by only 0.03%. the eu committee for economic and monetary has rejected its attempts to join the euro zone. institutional adjustments the moment they joins the euro zone, poland and other countries will be forced to adapt their monetary policy instruments to the requirements of european central bank. the most important problem is overflow of the polish banking sector which generates steps taken by the polish national ban, which is opposite to those of the european central bank, which puts the money into the banks instead of taking the overflow from them. also, the open market operations must be adapted to the system obligatory within ecb. 3. poland’s economic position relative to other euro zone countries and those aspiring to join the european union. at the moment, the euro zone includes 13 states: austria, belgium, finland, france, germany, greece, holland, ireland, italy, luxembourg, portugal, spain, and since january 1, 2007, table 1 selected macroeconomic indicators for the years 1995-2000-2005 -2006 of states in the euro zone and those attempting to join country gdp dynamics (%) budgetary balance (%) public debt (% gdp) inflation (%) 1995 2000 2005 2006 1995 2000 2005 2006 1995 2000 2005 2006 1995 2000 2005 2006 germany 1.0 2.9 0.9 2.8 -3.3 -1.2 -3.3 -1.7 58.3 60.3 67.7 71.4 1.7 2.1 2.1 1.7 france 2.0 3.8 1.8 2.0 -5.5 -1.4 -1.5 -2.6 52.8 57.4 66.8 75.0 1.8 1.8 2.0 1.7 italy 3.0 3.1 4.0 1.9 -7.6 -0.5 -1.8 -4.5 125.3 110.6 106.4 119.9 5.2 2.6 2.3 2.1 holland 2.2 3.5 0.7 2.9 -4.2 2.2 -0.3 0.5 79.0 56.0 52.9 59.7 1.5 2.3 1.6 1.2 belgium 2.0 3.6 1.5 3.2 -4.3 0.1 0.1 0.1 132.2 109.3 93.3 90.7 1.1 2.7 2.6 1.8 luxembourg 3.8 9.1 3.5 6.2 2.7 5.8 -1.9 0.1 5.8 5.6 6.2 9.8 1.0 3.2 3.9 2.7 spain 3.0 4.2 3.4 3.9 -7.0 -0.3 1.1 1.8 67.2 60.4 43.2 47.1 2.2 3.4 3.7 3.5 portugal 3.0 3.7 0.8 1.3 -4.6 -1.5 -6.0 -3.9 65.9 53.4 63.9 73.2 2.2 2.8 3.1 ireland 12.0 10.1 4.7 6.0 -2.2 4.5 1.0 2.9 78.9 39.0 27.6 30.4 2.5 5.3 2.5 3.9 austria 2.0 3.5 1.8 3.1 -5.2 -1.6 -1.5 -1.2 69.4 63.6 62.9 68.9 1.5 2.0 1.9 1.4 finland 5.0 5.1 2.2 5.5 -3.7 6.9 2.6 3.8 58.1 44.0 41.1 46.2 0.3 3.0 1.1 1.6 greece 2.0 4.4 3.7 4.3 -0.2 -0.8 -2.4 -2.3 110.1 102.8 107.5 92.5 2.2 2.9 3.4 3.2 great britain 3.0 3.8 1.7 2.8 -5.4 4.4 -3.6 -2.9 53.9 45.7 42.8 46.6 2.3 2.1 2.1 2.3 sweden 4.0 4.4 2.6 4.2 -7.9 4.0 2.9 2.1 77.6 65.7 50.3 53.9 2.7 1.0 1.3 1.4 denmark 3.0 2.6 2.8 3.2 -2.3 2.5 4.9 4.2 73.3 57.1 35.8 34.7 2.5 3.0 2.0 1.9 euro zone 4.7 2.7 -5.0 0.2 1.0 -1.6 73.4 70.1 70.8 76.1 2.7 2.4 2.3 2.2 bulgaria 5.5 -11.3 29.9 5.0 the czech republic 4.8 6.4 -2.6 -2.9 30.5 35.6 1.6 2.6 estonia 9.8 -10.5 4.5 4.1 lithuania 7.6 -7.2 18.7 2.7 latvia 10.2 12.7 12.1 6.7 poland 7.0 4.0 3.5 5.8 2.9 3.0 -2.5 -3.9 51.6 42.4 50.2 49.8 27.9 10.8 1.3 1.3 romania 4.1 -8.6 15.2 9.0 slovenia 4.0 -1.4 29.1 2.5 the slovak republic 5.5 8.3 -2.9 -3.4 34.5 37.0 2.8 4.5 hungary 3.9 3.9 -6.1 -9.2 61.5 73.2 3.5 3.9 maastricht criterion 03.0% =<60 .0 =<2.8 source: calculations based on: eurostat cia factbook. 2006, unicredit new europe research network ijahp article: adamus/when should poland enter the euro zone? international journal of the 88 vol. 1 issue 2 2009 issn 1936-6744 analytic hierarchy process slovenia. new eu members and sweden, which has not determined by referendum the date of its entry into the monetary union, are in line to join the union. table 1 presents selected macroeconomic indicators for the years 1995, 2000, 2005 and 2006 of countries already present in the euro zone and of those attempting to join. indicators can be divided into three groups: • gross domestic product (gdp) dynamics, • fiscal criteria (public finance deficit and public debt), • inflation. analyzing poland’s gdp dynamics in the years 1995, 2000, 2005 and 2006 in relation to countries in the euro zone, we may say that gdp tendency is similar to countries such as finland or sweden. most probably, had it not been for the cooling of poland in 1995 (balcerowicz), the dynamics of its gdp would be one of the highest in europe. in 2006 gdp in poland increased by 6% in comparison to the year 2005, and the analysts predict its growth in 2007 up to 7%. unfortunately, this indicator does not take into account many features, such as knowledge, education, health, beauty, cultural and natural resources, sensitivity and courage; in other words all that makes life sensible. fiscal criteria are not favorable for poland. although the relation of public debt to gdp does not exceed 60% set by the eu (49.8% in 2006), yet budget deficit in 2006 exceeds the admissible limit of 3%. it should be underlined that poland uses the right to include resources gathered in pension funds in the public finance sector (up to the year 2008). if part of pension system reform costs were included in the budget deficit, then one would expect the budget to exceed by even higher value. therefore, public finance regulation will be the largest challenge before poland’s entering the euro zone. among the countries attempting to join the euro zone, it was only slovenia, the slovak republic and the czech republic fulfilled budget deficit criterion in 2005. among all eu countries, in 2005, the budgetary surplus was present in denmark (4.9%), sweden (2.9%), finland (2.6%), spain (1.1%), ireland (1.0%) and belgium (0.1%). the largest deficit was experienced by the countries aspiring to the euro zone were: latvia (12.7%), bulgaria (-11.3%) and estonia (-10.5%). among candidate countries to enter the monetary union in 2005, the lowest deficit was present in slovenia (-1.7%). the debt of euro zone countries still exceeds the agreed limit of 60% set by the eu (70.8% average in 2005, 76.1 % in 2006); including the largest countries: germany, france and italy. in the maastricht treaty, high penalties were levied on the countries not following convergence criteria. unfortunately, it is impossible to enforce them. reference value for inflation criteria in 2005 was 2.8 percentage point. poland has fulfilled this criterion since 2003, a year before it entered the european union until now (may 2007). among the countries aspiring to enter the euro zone in 2005, the criterion was fulfilled by the czech republic, lithuania, poland and the slovak republic as well as countries outside the euro zone: denmark, sweden and great britain. poland also fulfils a long term interest rate criterion. due to the lack of all macroeconomic criteria listed in table 1 for cyprus and malta within the eu structure and candidates to the euro structure, this paper does not analyze their results. cyprus and malta already participate in the european exchange rate mechanism (erm ii), and plan to have common euro currency already in 2008. also lithuania, latvia, estonia and denmark have introduced their currencies to the european rate mechanism ii (2006). ijahp article: adamus/when should poland enter the euro zone? international journal of the 89 vol. 1 issue 2 2009 issn 1936-6744 analytic hierarchy process other countries do not hurry to change their domestic currency into euro. prof. jan winiecki of the polish economists society observed that (glapiak 2007) while some all the time can not cope with the necessity to fulfill conditions defined in the maastricht treaty, others move the date of euro zone entry for political reasons. the date poland enters the euro zone depends on the date all maastricht treaty convergence criteria are fulfilled, including the most important one: regulating public finance. poland’s entry into the euro zone not only means enormous benefits, as some think, but also great responsibility for euro currency stability and the eu economy. the key important element for poland as part of the eu structures is reasonable and responsible use of synergy effect, through integration with eu countries to further the increase its civilization level. poland’s entering the monetary union is connected to multiple benefits but also costs, as well as opportunities and risks for the society and the polish economy. those values were verbally defined in hundreds of articles and discussions. however, none of those estimate the values for benefits, costs, opportunities and risks. therefore, this paper aims to estimate their levels. to solve the problem, the author uses a multicriteria decision support method anp – analytic network process (saaty, 2001). 4. the essence of anp analytic network process – anp is a new theory supporting decision process, as a more developed version of analytic hierarchy process (saaty, 2000), one of the most widely known multicriteria decision making method. anp can be used to solve the most complex decision problems. in the anp method we use a systematic approach to multicriteria problems. it was initiated as a reaction to increasing specialization and the lack of communication between different fields. one of the founders of the systemic approach, l. von bertalanffy (biologist), in the 1950s, claimed that live organisms and also organizations are complex systems. in his view, the whole is more than a sum of elements. in order to able to say something about the whole, and also connections among them, the parts have to be examined in context. the basic assumptions of the systemic approach are as follows • everything is related to everything else • nothing stems from anything in a simple way – we can never predict all consequences of any phenomenon, process or activity. network thinking is an interesting tool, inspiring the decision maker’s intellect in solving multicriteria problems. it is based on the paradigm of a holistic and systematic approach to the problem. to use it, however, one needs to use a precise and adequate language. to make terms and notions precise requires multilateral discussions of decision makers, experts and other decision making process participants. that, in turn, makes it possible to consider variants and deepen self-criticism of all process participants. using the network thinking process take more time, but solving difficult problems can possibly spare many unpleasant surprises. using systemic reasoning, we can perceive connections, feedbacks, loops between events, processes, persons and network elements. in making multicriteria decisions, a decision maker has to compare one with different goals, criteria, subcriteria, decisional variants etc. those comparisons are made based on his knowledge, experience and intuition. ijahp article: adamus/when should poland enter the euro zone? international journal of the 90 vol. 1 issue 2 2009 issn 1936-6744 analytic hierarchy process everyone has the ability to subjectively experience the world with self-awareness, selfconsciousness, the freedom of choice, the ability to make conscious or unconscious decisions, and the ability to differentiate between the good and the evil. that all amounts to validating actions – both his own and those of others. for the anp model, it is the network that is a very important notion. it is a structure of clusters, nods and connections between the nods. a nod, as a network element, has a very wide meaning. it can designate people, institutions, machines, technology, workplaces, information carriers etc. in other words, nods are subjects resulting from relations with others. all connections between them that make sense can be treated as relations between the nods. for the network elements defined in a general sense should be defined in more detail. in such networks, nods can signify virtual entities or people participating in the network, with some engagement as decision makers, managers and implementations (schael 1997). usually, networks have feedbacks (lines going both directions between clusters) and can have internal dependencies (a loop in the cluster points to nods in which clusters have connections with each other). a hierarchy is a simple type of network. a cluster is logical grouping of factors or elements within a given decision to be considered. factors or elements are nods. in the network, the parts as groups of elements (responding to levels in the hierarchy) are not present in any given order. connections of components are made by determining whether and to what extent elements of a given group influence elements of another group, and vice versa. it assumes the form of arrows, which in that case can go both ways (feedback). components with network elements also have loops, if elements within them are internally dependent (internal dependence). in relation to groups of alternatives in the network, it can (although does not have to) include feedback to other components. networks can occur (can be generated) from the hierarchy through gradual increase of a number of hierarchical connections. saaty (2001) introduces a specific terminology for this type of hierarchy and their modifications to the feedback system. it follows the fact that each decision problem can be presented in the network form. presenting problem structure in the network form results from the fact that many decision problems cannot be presented in the form of hierarchy, as they require taking into account interdependencies and influences of elements positioned on higher levels of hierarchy upon elements on lower levels. anp introduces a free form of ordering elements instead of a tight importance chain (as in the hierarchy) but also the importance of alternatives themselves, which determine the importance of criteria. it is not only the importance of criteria that determines the importance of alternatives (as in the case of hierarchy) the author of ahp/anp methods is and american mathematician professor thomas saaty from the university of pittsburgh. he started working on the ahp method development in the early 1970s, and in 1975, started developing the anp model. as late as in 2001 his book on anp model was published, entitled decision making with dependence and feedback, the analytic network process. t.l. saaty is an author and co-author of many books and more that 300 articles on ahp/anp methods. feedback makes it possible to take into account future factors (in the present perspective) that will determine what we expect and what we aim to achieve in the future. a practical tool, serving the implementation of the anp and allowing for undertaking more complex decisions is software developed by rozann saaty with william adams super decision©. ahp method is supported by the expert choice software. as for software, it should be noted that both ahp and anp had to wait until fast computers came into being to be used more widely. anp supermatrix requires the computer support to a much greater extent than hierarchically structured ahp. that is why, three ijahp article: adamus/when should poland enter the euro zone? international journal of the 91 vol. 1 issue 2 2009 issn 1936-6744 analytic hierarchy process years after saaty’s book on anp method and developing and spreading super decision problems, it became possible to account for dependencies between elements and their feedback. it brought wide interest and use of ahp/anp models. 5. steps in choosing the best alternative for poland’s entry into the euro zone in the framework of the analytic network process (anp). 1. presenting a decisional problem in details in the context of anp method, that is, decision’s goals, criteria, subcriteria, actors decision process participants; their objectives, points of view on the problem to be considered, and possible alternatives of poland’s entry to the economic and monetary union in the european union. providing detailed factors that will influence the final decision. 2. describing control criteria and subcriteria within four control hierarchies for merits (personal merits): benefits (b), opportunities (o), costs (c) and risks (r) of the above defined decision. 3. in control criteria subsets for bocr, we build a structure in the form of a hierarchy tree including key elements (criteria and subcriteria). each subnet is constructed in the form of a cluster with objectives’ nod. clusters of the main bocr merits (benefits, opportunities, costs and risks) subcriteria clusters are related to one of the main criteria. in order to calculate priorities we connect goals with criteria and each criterion with their subcriteria. in order to calculate priorities for subcriteria, we make pairwise comparisons for all network elements. each subcriterion is a potential “control criterion” in the further network analysis. 4. we make criteria pairwise comparisons separately for each bocr, and next subcriteria to criteria in the four control hierarchies (on t.l. saaty’s fundamental scale: from 1 to 9). in the case of benefits and opportunities, we ask a question while comparing criteria and subcriteria what ensures the largest benefits in the case of poland’s entry into the euro zone and what constitutes the largest hidden benefits – opportunities in relation to the control criterion. in the case of costs and risks a question should be asked: what is the largest cost in the preparation phase? at the moment poland enters the monetary union what will lead to the largest hidden costs and risks? the worst alternative gets the highest priority for risks and costs. we calculate global priorities by multiplying priorities for a criterion times a priority for a subciterion for the four designated bocr merits. 5. for further implementation of decision process, those subcriteria which for the four bocr control criteria account for about 70% to 80% of the total sum of priorities for all subcriteria. these are usually those sub criteria, which achieved the merits for the global priorities over 3%. the values of global priorities for subcriteria were calculated by dividing each local priority for a subcriterion by four (four bocr control criteria). author experience shows that it is usually 30% to 40% of the most important subcriteria from the entire network (with priorities over 3%) that account for circa 70% to 80% of the total sum of priorities for all subcriteria. it significantly simplifies our decision process to follow, because we only analyze the most important criteria determining poland’s entry into the euro zone, following the vilfredo pareto “20/80” rule. this rule, formulated by the italian economist, means that 80% of results achieved by each manager follows implementation of 20% of his tasks whereas 80% of activities are responsible only for 20% of effects. ijahp article: adamus/when should poland enter the euro zone? international journal of the 92 vol. 1 issue 2 2009 issn 1936-6744 analytic hierarchy process 6. constructing a general clusters’ (or components’) network and their elements, concerning all bocr control criteria. subnets in relation to control criteria are the lowest network level in the model of poland’s entry into the euro zone. 7. building decision subnets for each selected control criterion. during that step, we introduce a cluster of alternatives (defined previously as a) fast poland’s entry into the euro zone, b) midterm poland’s entry into the euro zone c) late poland’s entry into the euro zone. a cluster of the above defined alternatives must be present in each decision subnet with other clusters. 8. for each control criterion (or subcriterion) those clusters are defined (with their elements) which influence other clusters (or elements) in relation to a given criterion, or they themselves can be under influence of other clusters or elements. importance (being under influence or exerting influence) must be applied to all criteria of all control hierarchies for a total response. in a graphic presentation of the model, clusters are connected with arrows following their external and internal dependencies and influences. there is an arrow from each cluster connecting it to another cluster that influences that cluster or its elements. 9. we make appropriate connection between nods and make pairwise comparisons of clusters with each other, in relation to a given criterion. in benefits (b) decision subnet, we make pairwise comparisons of clusters (on t. saaty’s fundamental scale), by asking questions which cluster (or its elements) are the most beneficial in the context of analyzed subcriterion. similar questions are asked with opportunities (o) subnet. the best alternative receives the highest priorities for benefits and opportunities. in risks (r) and costs (c) decision subnets during cluster pairwise comparisons, we ask which are more expensive or risky. the worst alternative receives the highest priorities for risks and costs. 10. for each control criterion, we construct a supermatrix. appropriate categories should include priorities following cluster comparisons in the net. 11. each pair of elements within the clusters are compared in accordance to their influence on each element in subsequent clusters with which they are connected (external dependency) or elements of the same clusters (internal interdependency). while making comparisons, we should always take into account the criterion that is a context for our comparisons. elements are compared in accordance with influence of a given element influences another element to a greater extent, and the greater importance within t. saaty’s scale than another element it is compared to. those comparisons must take into account control criterion or control hierarchy criterion. 12. clusters are also compared, pairwise, with respect to their influence on the control criterion. calculated weights are used to weigh elements of respective blocks of supermatrix columns. zero is assigned to those comparisons where there is no influence between compared clusters. this way we obtain a supermatrix stochastically weighted. 13. estimate the importance of each subsystem, that is benefits (b), opportunities (o), costs (c) and risks (r) of a problem in question. in taking any decision its significance differs in relation to (b), opportunities (o), costs (c) and risks (r). hence we have to prioritize them by estimating strategic criteria. those criteria constitute our system of merits, with respect to them we estimate the importance of best alternatives (b, o, c, r) of the problem we are solving – for example very high, high, medium, low, very low. ijahp article: adamus/when should poland enter the euro zone? international journal of the 93 vol. 1 issue 2 2009 issn 1936-6744 analytic hierarchy process 14. synthesizing priorities limited by weighting each idealized criteria. we choose the best alternative using the multiplicative formula (bo/cr) dividing a multiplication of variant priorities for benefits and opportunities by the multiplication of costs and risks the other formula for choosing the best variant is additive – negative bb + oo – cc – rr. in the latter formula the importance of each subsystem i.e. benefits (b), opportunities (o), costs (c) and risks (r) for a given problem must be estimated. the optimal variant has a higher result. 15. finally, a sensitivity analysis of the final result is performed. the analysis concerns “what – if” questions. it make possible to determine whether the final answer is stable and to what extent it will allow to change the input data concerning evaluations or priorities. it is particularly interesting to see whether those changes can be measured with a compatibility index. 6. prioritization of anp model control criteria for poland's entry into the euro zone table 2 presents prioritization of control criteria for the merits: benefits, costs, opportunities and risks. prioritization is based on pairwise comparison on t.l. saaty's fundamental scale (1 9). criteria were divided into economic and social, and compared among each other with respect to benefits, opportunities, costs, and risks. with respect to all merits, economic criteria overweigh the social ones. next, local priorities were calculated for 27 subcriteria for all merits. among benefits, it was the economic criterion that achieved the highest priority stable economic growth (0.4340), in the category of opportunities, the most important economic subcriterion is the growth of export (0.4990) and social tax lowering (0.7500). among costs the most important subcriteria proved the following: dependence upon the eu economic trends (0.4030) and the adjustment of monetary policy instruments to the requirements of the european central bank (0.3680). a relatively high priority was assigned to social costs giving up own monetary policy (0.4070). the highest risks were assigned to the lack of competitiveness of products (0.5140) and the increase in energy prices (0.2330). global priorities vector was normalized to the value of 1. subcriteria, for which the priorities are equal or exceed (0.03) (3%), were included in the analysis (marked bold in the table). 7. decision subnet analysis for each selected control criterion bocr at this stage, a cluster of three alternatives was introduced into the model. these alternatives had been defined earlier as the following scenarios: (a) early entrance of poland into the euro zone, (b) medium-term entrance of poland into the euro zone, (c) late entrance of poland into the euro zone. figure 1 presents the 13 sub-criteria, selected earlier from amongst 27 sub-criteria, analyzed using the super decisions software. ijahp article: adamus/when should poland enter the euro zone? international journal of the 94 vol. 1 issue 2 2009 issn 1936-6744 analytic hierarchy process table 2 prioritization of control criteria and elements of anp model for benefits, opportunities, costs and risks of poland’s entry into the euro zone merits criteria subcriteria local priorities global priorities benefits economic (0.7500) eliminating exchange risks 0.1080 0.0200 lowering costs of obtaining capital 0.0510 0.0090 stable economic growth 0.4340 0.0820 public finance improvement 0.3070 0.0580 increase in debt support for the economy 0.1000 0.0190 social (0.2500) people trust to money 0.4790 0.0300 closer tights with eu 0.2810 0.0180 unprofitable companies restructuring 0.2400 0.0150 opportunities economic (0.8000) export increase 0.4990 0.1000 investment capital inflow 0.1340 0.0270 manufacturing unit costs decrease 0.0830 0.0170 financial system stability 0.2840 0.0570 social (0.2000) tax lowering 0.7500 0.0380 free market strengthening 0.2500 0.0120 costs economic (0.6700) adaptation of monetary policy instruments to ecb requirements 0.3680 0.0620 adaptation of open market operations to ecb requirements 0.2290 0.0380 dependence on eu economic trends 0.4030 0.0680 social (0.3300) increase of bankrupt companies 0.2540 0.0210 flexibility decrease in economic growth 0.3390 0.0280 giving up own monetary policy 0.4070 0.0340 risks economic (0.8000) euro rate fall + strengthening of currencies outside eea 0.1770 0.0360 lack of competitiveness outside new products and quality increase 0.5140 0.1030 energy prices increase outside eu 0.2330 0.0470 speculating capital inflow 0.0760 0.0150 social (0.2000) unemployment increase 0.3590 0.0180 weakening of national identity 0.4230 0.0210 social unrest 0.2180 0.0110 ijahp article: adamus/when should poland enter the euro zone? international journal of the 95 vol. 1 issue 2 2009 issn 1936-6744 analytic hierarchy process figure 2 illustrates the anp model of “benefits” of poland’s entry into the euro zone. the subcriteria with global priority equal to or bigger than 0.03 were analyzed under the subset of “benefits”. figure 3 shows the anp model of “costs” of poland’s entry into the euro zone. the sub-criteria with global priority equal to or bigger than 0.03 were analyzed under the subset of “costs”. figure 4 presents the anp model of “opportunities” of poland’s entry into the euro zone. the sub-criteria with global priority equal to or bigger than 0.03 were analyzed under the subset of “opportunities”. figure 5 illustrates the anp model of “risks” of poland’s entry into the euro zone. the subcriteria with global priority equal to or bigger than 0.03 were analyzed under the subset of “risks”. figure 6 demonstrates a network of connections and mutual influences between particular elements for the permanent economic growth sub-criterion. it has been assumed that its indicator (gross domestic product – gdp) is dependent on decisions of the following institutions: government, parliament, monetary policy council, ministry of finance, european parliament, national bank of poland, and on individual decisions of consumers and producers (enterprises). by comparing these elements pairwise, their priorities were estimated, which illustrate their relations in the context of the permanent economic growth. in addition, the three alternatives were compared with each other within a net of their influences and feedbacks with the three elements presented by figure 7, and their priorities were calculated in the context of the permanent economic growth. similar procedures were performed for the remaining 12 sub-criteria in relation to all merits, namely: benefits, costs, opportunities and risks. the overall results for all comparisons are shown in table 3. according to the outcomes presented in table 3, the optimal date of joining poland to the euro zone is the latest possible date, after 2011 to be more specific, together with the countries such as czech republic, hungary and perhaps bulgaria and romania. for the 3 sub-criteria analyzed as benefits, only one (improved condition of public finances) indicates late entrance to the euro zone as optimal, while the two remaining (permanent economic growth and people’s trust in money) indicate medium-term period of the entrance. considering opportunities of poland’s entrance into the euro zone, it can be observed that from amongst three components analyzed, only one (increased export) points to the medium-term scenario, while the remaining two components (financial system stability and tax rate decrease) suggest the latest possible entry. table 3 shows the analysis of the four components related to costs and three components related to risks. their priorities decisively point to poland’s late entry to the euro zone. ijahp article: adamus/when should poland enter the euro zone? international journal of the 96 vol. 1 issue 2 2009 issn 1936-6744 analytic hierarchy process table 3. prioritization of decision alternatives for control criteria and subcriteria in the models: benefits, costs, opportunities and risks. merits benefits costs criteria economic (0.7500) social (0.2500) economic (0.6700) social (0.3300) alternatives stable economic growth (0.4340) public fin. improve ment (0.3070) people’s trust in money (0.4790) adjustment of monet. policy to ecb (0.3680) open market adjustment (0.2290) economic tendency improvement (0.4030) own monetary policy resignation. (0.4070) late entry into euro zone 0.8759 1.0000 0.6122 0.0867 0.0867 0.1014 0.1314 middle time entry into euro zone 1.0000 0.1895 1.0000 0.2873 0.2873 0.4110 0.3624 early entry into euro zone 0.3607 0.0838 0.3255 1.0000 1.0000 1.0000 1.0000 merits opportunities risks criteria economic (0.8000) social (0.2000) economic (0.8000) alternatives stable financial. system (0.2840) export growth (0.4990) tax lowering (0.7500) lack of competition (0.5140) euro rate fall (0.1770) energy price increase (0.2330) late entry into euro zone 1.0000 0.4971 1.0000 0.1314 0.1771 0.1619 middle time entry into euro zone 0.3462 1.0000 0.3815 0.3624 0.9410 0.4162 early entry into euro zone 0.1240 0.5092 0.4369 1.0000 1.0000 1.0000 ijahp article: adamus/when should poland enter the euro zone? international journal of the 97 vol. 1 issue 2 2009 issn 1936-6744 analytic hierarchy process the overall results are shown in table 4. table 4 decision alternatives’ prioritization for benefits, opportunities, costs and risks alternatives benefits opportunities costs risks late entry into euro zone 0.8714 0.6795 0.0991 0.1070 medium time frame entry into euro zone 0.7237 0.7629 0.3414 0.3819 early entry into euro zone 0.2600 0.3695 1.0000 0.7478 in selecting the optimal alternative date of poland’s entry into the euro zone, it is essential to link the values of the priorities’ alternatives with the control hierarchy of benefits (b), costs (c), opportunities (o) and risks (r). it can be performed in two ways: (1) using mathematical formula (bo/cr), in which the values of variants’ priorities for benefits multiplied by opportunities (b*o) are divided by the values for costs multiplied by risks (c*r); (2) using mathematical formula (bb+oo-cc-rr), which requires defining levels of importance for sub-systems of benefits (b), costs (c), opportunities (o) and risks (r). in the latter case, so called strategic criteria are defined, for which their importance is estimated. this stage allows us to look at the problem from more general perspective, that is, from the viewpoint of the criteria such as: increased welfare of polish society, financial integration of the national banking system with the european central bank, social and political integration with the eu countries. each of the main listed criteria have their subcriteria. a model of the strategic criteria is illustrated in figure 8. the strategic criteria enable the connection of results for benefits and opportunities with those for costs and risks, and presenting the final result. the scale (personal criteria), based on which the importance of benefits, costs, opportunities and risks have been estimated is presented in table 5. ijahp article: adamus/when should poland enter the euro zone? international journal of the 98 vol. 1 issue 2 2009 issn 1936-6744 analytic hierarchy process table 5 determining ratings priorities personal criteria very high high medium low very low pi very high 1 2 3 4 5 0.42 high 1/2 1 2 3 4 0.26 medium 1/3 1/2 1 2 3 0.16 low 1/4 1/3 1/2 1 2 0.10 very low 1/5 1/4 1/3 1/2 1 0.06 total 1.00 table 6 clearly demonstrates that the most important criterion to be considered in analyzing the country’s entrance to the euro zone is increased welfare of polish citizens (0.43), followed by financial integration (0.28). the values derived from prioritization of the strategic criteria for benefits (0.3280) and opportunities (0.1840) are higher than the respective values derived for costs (0.2720) and risks (0.2160). at this stage, it can be asserted that benefits and opportunities from poland’s entry to the euro zone will outweigh the potential costs and risks. yet, the main research objective of this study is to recommend when poland should join the euro zone. as explained in the previous sections, this is the only decision-making option we can consider. in order to provide an explicit answer, third step of analysis was performed, which included the above estimated priorities for benefits, costs, opportunities and risks. paircomparisons of three decision-making variants based on importance of the bocr values led to the selection of the best possible alternative (table 7). ijahp article: adamus/when should poland enter the euro zone? international journal of the 99 vol. 1 issue 2 2009 issn 1936-6744 analytic hierarchy process table 6 priorities estimation for merits: benefits, costs, opportunities and risks ratings priorities: very high (0.42), high (0.26), average (0.16), low (0.10), very low (0.06) criteria subcriteria merits benefits (b) costs (c) opportunities (o) risks (r) improvement of poles’ living standards (0.43) per capita gdp level reaching the eu level (0.14) high (0.26) very high (0.42) low (0.10) high (0.26) economic growth stabilization (0.15) very high (0.42) high (0.26) average (0.16) average (0.16) export dynamic growth (0.14) high (0.26) average (0.16) low (0.10) average (0.16) direct investment capital (0.14) high (0.26) high (0.26) low (0.10) average (0.16) private spending increase (0.14) high (0.26) low (0.10) average (0.16) low (0.10) unemployment rate fall (0.14) very high (0.42) high (0.26) average (0.16) average (0.16) economic development flexibility (0.15) low (0.10) high (0.26) low (0.10) average (0.16) financial integration (0.28) monetary policy transferred to ecb (0.16) low (0.10) average (0.16) low (0.10) average (0.16) exchange risk elimination (0.48) high (0.26) very low (0.06) average (0.16) low (0.10) foreign eu currency reserves elimination (0.29) average (0.16) low (0.10) average (0.16) low (0.10) speculation capital intensive inflow (0.07) very low (0.06) very high (0.42) very low (0.06) very high (0.42) social integration (0.18) labor mobility increase (0.37) high (0.26) high (0.26) average (0.16) low (0.10) easier trading exchange between poland and eu countries ue (0.38) high (0.26) low (0.10) average (0.16) low (0.10) limits on national sovereignty with respect to own monetary policy (0.25) very low (0.06) average (0.16) very low (0.06) average (0.16) political integration (0.11) european identity acquiring (0.15) high (0.26) average (0.16) average (0.16) low (0.10) closer ties with eu (0.24) high (0.26) high (0.26) average (0.16) average (0.16) foregoing national identity (0.15) very low (0.06) average (0.16) low (0.10) average (0.16) decreasing possibility of competition between governments in fields of investment and manufacturing (0.46) very low (0.06) high (0.26) low (0.10) high (0.26) priorities 0.3280 0.2720 0.1840 0.2160 ijahp article: adamus/when should poland enter the euro zone? international journal of the 100 vol. 1 issue 2 2009 issn 1936-6744 analytic hierarchy process table 7 synthesis of final priorities of the three alternatives alternatives benefits (0.3280) opportunities (0.1840) costs (0.2720) risks (0.2160) multiplicative formula bo/cr additive – negative formula bb+oo-cc-rr late entry into euro zone 0.8714 0.6795 0.0991 0.1070 40.4241 0.3520 medium time frame entry into euro zone 0.7237 0.7629 0.3414 0.3819 3.3219 0.1797 early entry into euro zone 0.2600 0.3695 1.0000 0.7478 0.0961 -0.3347 the optimal alternative for poland is late entrance into the euro zone. such outcome has been achieved by both formulas: multiplicative and additive-negative. the problem discussed in this paper represents a “hot topic” in poland, and was analyzed based on the anp method. this method allows solving multi-criteria problems in reality. it requires the advanced knowledge and experience of decision makers, as well as a great amount of work to grasp all significant factors and their mutual interactions. it has been suggested that nearly all complex problems can be solved using this method. ijahp article: adamus/when should poland enter the euro zone? international journal of the 101 vol. 1 issue 2 2009 issn 1936-6744 analytic hierarchy process figure 1 control hierarchy of dependencies and feedbacks in the model of poland’s entry into the euro zone. ijahp article: adamus/when should poland enter the euro zone? international journal of the 102 vol. 1 issue 2 2009 issn 1936-6744 analytic hierarchy process figure 2 subnet under benefits following poland’s entry into the euro zone figure 3 subnet under costs following poland’s entry into the euro zone economic main goal: poland’s entry into the euro zone control criteria for costs social economic control subcriteria adaptation of monetary policy instruments to ecb requirements [subnet] adaptation of open market operations to ecb requirements [subnet] dependence on eu economic trends [subnet] social control subcriteria increase of bankrupt companies flexibility decrease in economic growth giving up own monetary policy [subnet] economic main goal: poland’s entry into the euro zone control criteria for benefits social economic control subcriteria eliminating exchange risks lowering costs of obtaining capital stable economic growth [subnet] public finance improvement [subnet] increase in debt support for the economy social control subcriteria people’s trust in money [subnet] closer ties with eu unprofitable companies restructuring ijahp article: adamus/when should poland enter the euro zone? international journal of the 103 vol. 1 issue 2 2009 issn 1936-6744 analytic hierarchy process figure 4: subnet under opportunities following poland’s entry into the euro zone figure 5: subnet under risks following poland’s entry into the euro zone economic control criteria for risks social economic control subcriteria euro rate fall + strengthening of currencies outside eea [subnet] lost of the exchange-rate related competitive advantage in export [subnet] energy prices increase outside eu [subnet] speculation capital inflow social control subcriteria unemployment increase weakening of national identity social unrest main goal: poland’s entry into the euro zone economic main goal: poland’s entry into the euro zone control criteria social economic control subcriteria export increase [subnet] investment capital inflow manufacturing unit costs decrease financial system stability [subnet] social control subcriteria tax lowering [subnet] free market strengthening ijahp article: adamus/when should poland enter the euro zone? international journal of the 104 vol. 1 issue 2 2009 issn 1936-6744 analytic hierarchy process figure 6: example of an anp subnet model benefits of “poland’s entry into euro zone” ijahp article: adamus/when should poland enter the euro zone? international journal of the 105 vol. 1 issue 2 2009 issn 1936-6744 analytic hierarchy process figure 7 anp subnet for “stable economic growth” under economic benefits lower house parliament senate government government monetary policy council experts minister of finance ministry of finance national bank of poland national bank of poland late entry into euro zone alternatives middle time entry into euro zone early entry into euro zone products and services polish economy euro commission eu parliament ijahp w. adamus / when should poland enter the euro zone? decision making with the anp international journal of the 106 vol. 1 issue 2 2009 issn 1936-6744 analytic hierarchy process figure 8: strategic criteria model for poland’s entry into “euro zone” ijahp w. adamus / when should poland enter the euro zone? decision making with the anp international journal of the 107 vol. 1 issue 2 2009 issn 1936-6744 analytic hierarchy process 8. final conclusions the issue of poland’s entry into the euro zone is not only very important for poland but also very complex, strongly related to the main decision makers in poland , including the government, ministry of finance, poland’s economy, european central bank, other banks in poland etc. those relations are internally and externally connected with feedbacks and loops. poland, preparing to enter the euro zone, should strengthen its economic growth, and then, during a longer period of time, fulfill all the criteria of the maastricht treaty. only such sequence can improve the welfare of the poland’s citizens, which will benefit not only poland, but also the european union. poland’s entry into the euro zone shall take place as soon as possible, but only after the benefits and opportunities overweigh costs and risks. the date of the entry into the euro zone shall be determined in the group of experts (academics, politicians, practitioners) with the use of anp method. the problem of entry date shall not be solved by means of a referendum. finding appropriate solution requires significant knowledge and experience, an accounting of all possible criteria and their mutual dependence both within and outsider the system. as a result of a detailed network analysis of the problem with use of analytic network process and the solution achieved with super decisions, the following conclusions can be drawn: 1. the anp model accounts for 27 bocr criteria, including economic and social factors determining poland’s entry into the euro zone. 2. as a result of prioritization, both mathematical formulae give the same results for poland’s late entry in the euro zone as the best alternative. 3. poland’s entry into the euro zone shall only take place when joint benefits and opportunities exceed costs and risks 4. sensitivity analysis may slightly change the priorities of discussed analyses, but that would require extreme conditions for bocr prioritization and their control criteria. 5. from the analytic network process perspective, the problem to be solved reveals its significant complexity, various dependencies with main decision makers in poland and the economies and cultures of poland, european union and the rest of the world. ijahp article: adamus/when should poland enter the euro zone? international journal of the 108 vol. 1 issue 2 2009 issn 1936-6744 analytic hierarchy process references adamus w., gręda a., (2005). wspomaganie decyzji wielokryterialnych w rozwiązywaniu wybranych problemów organizacyjnego i menedżerskich ,(multiple criteria decision suport in organizational and management chosen problems solving), badania operacyjne i decyzje, nr 2 blaski i cienie wejścia polski do strefy euro (light and shade of poland’s entrance into the euro zone). conference organized by the senate committee for the european union, 15.11.2006, www.senat.gov.pl.k6/agendaseminar/blaski/061115.htm. figueria j., greco s., & ehrgott m. (edit.) (2005). multiple criteria decision analysis: state of the art surveys, springer science + business media, inc. glapik e. (2007). blaknący czar wspólnej waluty (fading light of common currency). rzeczpospolita 12.03.2007 mazur j. (2002), argumenty za i przeciw unii gospodarczej i walutowej (arguments for and against the european economic and monetary union). głos 31.08.2002 mazur j. (2006). likwidacja polskiego złotego, likwidacja państwa (elimination of the polish zloty, elimination of the state), http://web.1asphost.com (tera 2) likwidacja państwa polskiego.htm piński a. & piński j., euro 2012. wprost, 19 (14 may 2006 r), s. 44 – 48. polska w strefie euro: szanse i wyzwania (poland in the euro zone: opportunities and challenges). conference organized by the institute for international affairs and the polish sejm 13.10.2006, rzeczpospolita: head-arok 22.gif saaty r.w. (2002). decision making. the analytic network process (anp) for dependence and feedback, a manual for the anp software super decisions, creative decisions foundation, pittsburgh, pa saaty t.l. (1980). the analytic hierarchy process. new york: mcgraw-hill. saaty t.l. (2001). the analytic network process. pittsburgh, pa: rws. publications. saaty t.l. (2005). theory and applications of the analytic network process. pittsburgh, pa: rws publications. schael t. (1997). theorie et practique du workflow. paris: springer strategia integracyjna z unią europejską na okres poakcesyjny 2004 – 2015 (european integration strategy for post-accession period 2004 – 2015), scientific conference 25.06.2002, www.prezydent.pl nowak, a.z. & stępniak, a. (eds.) (2003) strefa euro – wyzwanie dla polski (the euro zone – a challenge for poland). wydawnictwo naukowe wydziału zarządzania uniwersytetu warszawskiego, warszawa 2003 waluta biurokratów (the currency of bureaucrats). interview with g.s. becker by l.m. bednarz, wprost nr 19 (14 may 2006) ijahp article: kumar, r, singh h., singh a./a framework for evaluation of vendors in the automotive industry international journal of the analytic hierarchy process 488 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.696 a framework for evaluation of vendors in the automotive industry raman kumar associate professor, department of mechanical engineering, chandigarh university, mohali-140413, india ramankakkar@gmail.com harwinder singh professor, department of mechanical engineering, guru nanak dev engineering college, ludhiana, punjab141006, india harwin75@rediffmail.com amrinder singh assistant professor, north west institute of engineering and technology, punjab 142053, india amrinderbirdi@gmail.com abstract vendor selection is the first step in the product realization process that starts with the purchasing of materials and ends with delivering the products. the objective of the present research is to select the best vendor in a leading automobile organization. the multi-criteria decision making techniques of fuzzy quality function deployment (qfd) and the analytical network process (anp) were applied to achieve reliable results. a case study of a manufacturing unit in northern india was used to validate the proposed framework. the output of the qfd showed the pre-qualified vendors as v3, v2 and v7 with relative user requirement (rur) values of 0.188, 0.145 and 0.134, respectively. the final ranking of the vendors is presented using the anp model. keywords: vendor selection; ranking; multi-criteria decision making; qfd; anp 1. introduction the process of vendor ranking is essential to effectively purchase items such as raw materials, spare parts, etc. (john et al, 2005). in a vendor selection problem, the following two factors are critical; the performance of the materials and the performance of the vendors. the vendor or supplier selection problem is considered a typical problem due to involvement of multiple criteria and their respective sub-criteria (kumar et al., 2012). in manufacturing industries, approximately 50% of quality rejection is due to the poor quality of the material purchased from various vendors (talluri and narasimhan, 2003). undoubtedly, many of the world’s successful organizations have a competitive advantage because of their direct and indirect networks in the vendor chain. therefore, it is vital to complete a thorough investigation on the assessment of vendor selection mailto:ramankakkar@gmail.com mailto:harwin75@rediffmail.com mailto:amrinderbirdi@gmail.com ijahp article: kumar, r, singh h., singh a./a framework for evaluation of vendors in the automotive industry international journal of the analytic hierarchy process 489 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.696 because it can expand consumer loyalty by enhanced quality and focused capacity (onut et al, 2009). the selection of a vendor is the responsibility of the purchasing department and needs to consider both qualitative and quantitative factors. the vendor selection problem is vital for both the private and public sectors. however, the private sector also concentrates on this issue to survive in today’s turbulent market scenario. the previous research discovered four explicit criteria such as quality, service, delivery and price for vendor selection in both the public and private sectors. in addition, reputation and location are also important, but their relative significance is subject to discussion. a proficient vendor selection process should be established and is of vital significance for effective supply chain management (sonmez, 2006). 1.1. needs in vendor selection  assess and monitor supplier performance in order to reward suppliers who meet a company’s expectations.  provide benchmark data, which will allow vendors to establish where they are placed in relation to the best performers in their industry so they can improve their overall competitiveness in the market.  provide feedback so that specific actions can be taken to correct identified performance weaknesses. 1.2 quality function deployment (qfd) quality function deployment is an important tool in multi-criteria decision making developed in the late 1960s in japan by akao (1990). the aim of qfd is to improve the level of customer satisfaction and organization profitability. the steps involved in qfd are presented below: step1: identify the required attributes for the product the first step in qfd provides the required attributes for the product to fulfill the requirements of the manufacturer, for example, percentage rejection, on time/every time delivery, lead time, product durability, etc. step 2: identify the required enablers to rate the performance of the vendors through benchmarking, literature review and opinions from the organization at all levels, identify the required enablers to rate the performance of the vendors, for example, product cost, annual turnover, and geographical location. step 3: transform linguistic expressions into quantitative values rao (2013) presented the systematic conversion of the qualitative value into a crisp number using the fuzzy concept. the triangular fuzzy function is shown in figure 1. the conversion of the linguistic term into crisp scores is shown in table 1. ijahp article: kumar, r, singh h., singh a./a framework for evaluation of vendors in the automotive industry international journal of the analytic hierarchy process 490 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.696 figure 1 triangular fuzzy function table 1 conversion of linguistic terms into crisp scores (5-point scale) linguistic term fuzzy number crisp score low m1 0.115 below average m2 0.295 average m3 0.495 above average m4 0.695 high m5 0.895 step 4: determine the relationship between the criteria and the enabler identify the relationship between the criteria and the enabler through the team of managers completing the questionnaire. the relative weights of the enablers can be calculated by computing the technical significance rating (tsr) and the relative technical significance rating (rtsr). step 5: determine relationship between the enablers and the vendors identify the relationship between the enablers and the vendors through objective data and ijahp article: kumar, r, singh h., singh a./a framework for evaluation of vendors in the automotive industry international journal of the analytic hierarchy process 491 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.696 the team of managers completing the questionnaires. to measure the functioning of each alternative supplier with respect to enablers, the user requirement (ur) and the relative user requirement (rur) is computed using the technical significance rating. step 6: mathematical model formulation the mathematical model is divided into two sub-problems because of two contradictory objectives: i. maximizing the user requirements ii. minimizing lead time. first, the sub-problem is solved by considering total consumer satisfaction (tcs) as the objective function of integer programming and the predetermined value of the maximum threshold level of lead time. step 7: identify potential vendors using the tora software ((palanisamy and zubar, 2012) potential vendors are identified through the taha operational research algorithm (tora) software and a vendor pool is formulated. 1.3 analytic network process methodology (anp) the anp converts a decision problem into a network and performs pairwise comparisons to measure the weights of the network elements and rank the alternatives. only unidirectional hierarchical relationships are represented with the ahp. the anp allows for multifaceted interrelationships among the decision levels and attributes. the steps involved in the analytical network process (anp) are presented below: step 1: identify the criterion and sub-criterion for vendor ranking identify the criteria and sub-criteria for ranking the pre-qualified vendors through literature review, brainstorming and soliciting the opinions of employees at all levels in the organization. step 2: construct the anp model the model is a framework that in some ways represents something in the real world. the model starts with an idea of what the decision is about, what the alternatives are and what criteria should be used in the network. then, the model is built using the superdecisions software that produces results including the factors and alternatives of the problem and their structure and how they are connected. a benefits/opportunities/costs/risks (b/o/c/r) model was suggested for evaluation and selection of venders and suppliers. step 3: degree of preference the intensity of importance on a scale of 1-9 is used to represent compromises among the preferences (saaty, 1996). step 4: perform pairwise comparisons to determine the priorities of the criteria and subcriteria perform the pairwise comparisons and determine the priorities of each criteria and subcriteria by inputting the data collected from the questionnaire into the superdecisions ijahp article: kumar, r, singh h., singh a./a framework for evaluation of vendors in the automotive industry international journal of the analytic hierarchy process 492 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.696 software ((palanisamy and zubar, 2012). the questionnaire is completed by the expert team of managers. step 5: perform pairwise comparisons to determine the priorities of the alternatives perform pairwise comparisons of the alternatives with respect to the criteria to determine the priorities of the alternatives from the data obtained from the questionnaire. step 6: check the inconsistency ratio after the pairwise comparisons, it is necessary to verify the consistency of the judgments. if the judgments are not consistent, a mistake may have occurred in the judgments or in the formulation of the problem, making it necessary to correct the pairwise comparisons or the formulation of the problem. inconsistency is calculated automatically while inputting data from the questionnaire into the superdecisions software and the inconsistency value should be less than 0.1. however, if the judgments are consistent, the next step should be executed (saaty, 1996) step 7: construct unweighted, weighted and limit matrices an unweighted matrix indicates pairwise comparisons whose direct or indirect relationships among all of the elements are performed in the network. a weighted supermatrix is the form of an unweighted matrix which is stochastic, in other words, the column totals are equal to 1. a limit matrix is obtained by taking the power of the weighted matrix until its rows become fixed. a limit matrix signifies the suitable alternative. step 8: rank the vendor alternatives according to synthesized priorities rank the vendors according to the overall synthesized priorities of the alternatives of the whole model. the ahp/anp are the most commonly used techniques for the vendor selection problem. the complexity of the vendor selection process depends on the business type, size of the organization and budget of the purchasing department [6]. however, due to its complexity, researchers have focused on implementing hybrid mcdm tools to achieve the most reliable results. in the present work, the fuzzy concept is used to minimize subjective error while experts score the vendors. the organization of the paper is as follows. section 2 presents a review of the relevant literature. section 3 describes the present work. the last section concludes the paper as well as presents guidelines for further research. 2. literature review zhang et al. (2004) proposed an application of the analytical network process (anp) for vendor selection in an electronic company. an anp model was formulated and applied to the problem of evaluating eleven vendors based on the following criteria: quality, price, delivery, reciprocal arrangements and service capacity. the cluster weights and priorities of all of the sub-criteria were combined to determine the overall priority weights of the vendor systems. the results showed that quality had the priority weight of 0.6280 and was the most important criterion in the evaluation of the vendors. kirytopoulos et al. (2008) presented a systematic methodology for the ranking of suppliers in the ijahp article: kumar, r, singh h., singh a./a framework for evaluation of vendors in the automotive industry international journal of the analytic hierarchy process 493 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.696 pharmaceutical industry. the anp was implemented for the evaluation of the best supplier offer. the results indicated that the supplier brand co. ranked first for the service provider in the pharmaceutical industry. it was shown that the proposed model was accurate for priority changes and the result was unaffected when a sensitivity analysis was applied. ho et al. (2009) presented a hybrid approach for the evaluation of the best strategic 3pl. the integrated qfd and ahp approaches comprised a series of three houses of quality. a case example of a hard disk components supplier was examined and the qfd approach was used for the analysis of the criteria that affected the supplier selection. the integrated approach including qfd and ahp provided a benchmark, and the results were reliable. li et al. (2011) used fuzzy analytical network process (fanp) for evaluation of 3pls.the fanp was implemented to overcome the limitations of the anp. the proposed methodology has the advantage that it adequately deals with the judgments derived from the information and the problem of interdependence and feedback among the elements of the system. a case example of an optical company was examined with the help of the proposed method. the presented approach is capable of capturing the vagueness and fuzziness during value judgment elicitation. palanisamy and zubar (2012) proposed a hybrid multi-criteria decision making model for ranking vendors in an automobile organization. the vendor ranking was based on benefits, opportunities, costs and risks. the proposed methodology consisted of two techniques as follows: quality function deployment implemented for pre-qualification of vendors and an analytical network process-based final ranking of vendors. v2 ranked first followed by v5, v7 and v16. andronikidis (2014) presented a hybrid multi-criteria decision making model of qfd and anp to design high quality services in the banking sector. the qfd integrated quantitative techniques and the anp was used to determine the priority of customers’ bank selection criteria. the proposed model was implemented with a case problem in the banking sector and the priorities concluded that better service offerings to meet or exceed customers’ needs lead to improved sales and higher customer satisfaction. the rest of the relevant literature review is presented in table 2. ijahp article: kumar, r, singh h., singh a./a framework for evaluation of vendors in the automotive industry international journal of the analytic hierarchy process 494 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.696 table 2 literature review author/’s years description kirytopoulos, leopoulos and voulgaridou 2008 presented a model for supplier selection in pharmaceutical organizations. the anp was implemented for the selection of the best supplier offer. the supplier brand company ranked first in supplier selection. qian 2009 made an attempt to introduce the concept of an artificial neural network algorithm in the domain of vendor selection. the proposed framework had the ability to perform analyses according to changes in the business environment. koul and verma 2011 provided a new direction by solving the problem of vendor selection with a time axis. the mathematical system was developed which had the capability to capture the effect of uncertainty in vendor selection. hui and yang 2013 developed a two-step service method (i.e., field index library, match description of service patterns, service composition description) as a solution for the vendors. this methodology can have a significant impact on solving the practical needs of vendors. palanisamy, and zubar 2013 implemented mcdm techniques to formulate a hybrid process with fuzzy qfd and anp to rank the vendors in terms of their overall performances. when compared to the individual approaches, the proposed hybrid model effectively assisted the vendor ranking process. shih et al. 2014 analyzed the environmental issues in the selection of a vendor. an ahp-bocr frame model was presented to obtain reliable results. kamath et al. 2016 developed a framework for selection of a vendor using the ahp in an indian steel pile manufacturing organization. the managerial implications were presented to achieve the most reliable results. kant and dalvi, 2017 presented a systematic questionnaire to measure the importance of supplier selection criteria. the validity of the questionnaire was demonstrated by collecting responses from a total of 34 automobile industries in india. mathiyazhagan, 2017 provided a framework for the evaluation of a supplier ijahp article: kumar, r, singh h., singh a./a framework for evaluation of vendors in the automotive industry international journal of the analytic hierarchy process 495 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.696 sudhakar and bhalotia with respect to environmental criteria. a case study was demonstrated using the ahp technique to validate the proposed framework. aggarwal et al. 2018 made an attempt to solve a multi-objective optimization problem of vendor selection and order allocation. significant managerial implications were provided and thoroughly discussed. suraraksa and shin 2019 presented an integrated model including both a quantitative and qualitative approach. the ahp was applied to evaluate the selection of vendor criteria. mohammed et al. 2019 developed a hybrid mcdm algorithm for the selection of a vendor. a framework consisting of traditional and resilience criteria was proposed to select an appropriate vendor using the electre and topsis methods. it has been proven that resilience criteria have a significant role in the selection of a vendor 3. present work in this research work, a hybrid multi-criteria decision making methodology consisting of fuzzy qfd-anp was used to evaluate the vendors. fuzzy qfd was used to create a pool of pre-qualified vendors based on certain criteria and sub-criteria and the anp was implemented to achieve the final ranking of the pre-qualified vendors. 3.1 introduction to the case organization a leading manufacturer in the indian automotive components industry began its journey in 1938 in ludhiana. this automotive manufacturing company is a proud supplier of components to various indian original equipment manufacturers (oems) and has established itself as a reliable supplier for many years. the annual turnover of this company is 150 crore (approx. 2.03 crore dollars). the list of oem customers includes telco, volvo india limited, swaraj mazda limited, mahindra & mahindra, maruti udyog limited, ashok leyland, and punjab tractors limited, etc. the company employs approximately 1,000 employees and has an infrastructure that includes modern testing facilities equipment and workshops, a casting shop, a machine shop, wire drawing, electroplating, heat treatment, a welding shop, a paint shop, a tool room, packaging and dispatch. this leading manufacturing industry faces problems with rating vendors of sae-8620 material in the purchasing department. spring pins and king pins of all types are made from this material and its monthly consumption is very high at approximately 35 tons per month. 3.2 implementation of fuzzy quality function deployment (qfd) fuzzy qfd was applied to reduce the number of potential vendors by screening them with certain basic criteria and sub-criteria. the mathematical model was solved using ijahp article: kumar, r, singh h., singh a./a framework for evaluation of vendors in the automotive industry international journal of the analytic hierarchy process 496 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.696 integer programming and tora, and provided the decision makers with the optimal number of vendors (palanisamy and zubar, 2012). step1: identify the required attributes for the product the first step in qfd is to provide the required attributes for the product to fulfill the requirements of the manufacturer. in this research, three criteria and six sub-criteria were included in the qfd to create a pool of pre-qualified vendors as shown in table 3. table 3 criteria and sub-criteria selected criteria sub-criteria definition quality percentage rejection number of rejections per total produced. product durability life of the product. delivery order lead time duration of time between setting up an order and receipt of the order. delivery on time/every time consistency of meeting delivery deadlines. flexibility volume flexibility ability to adjust product volume demanded. customization ability to customize the product demanded by the buyer. step 2: identify the required enablers to rate the performance of the vendors in this research, product cost (pc), annual turnover (at), geographical location in kms (gl), experience (exp), technical capability (tc), attitude (att) and accuracy of order fulfillment (aof) enablers were identified to rate the performance of vendors through benchmarking, literature review and solicited opinions from the organization at all levels. step 3: determine the geometric mean value the geometric mean of the award score given by the experts was calculated by the formula given below and the values are shown in table 4. ijahp article: kumar, r, singh h., singh a./a framework for evaluation of vendors in the automotive industry international journal of the analytic hierarchy process 497 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.696 table 4 values after geometric mean of data collected from the case company product cost accuracy of order fulfillment annual turnover geographical location in kms technical capability experience attitude pr 0.115 0.625 0.115 0.115 0.789 0.68 0.184 pd 0.539 0.115 0.473 0.115 0.587 0.515 0.146 old 0.374 0.68 0.239 0.539 0.382 0.16 0.587 dotet 0.184 0.789 0.21 0.539 0.435 0.205 0.638 vf 0.789 0.233 0.336 0.115 0.639 0.115 0.382 cus 0.741 0.146 0.184 0.115 0.295 0.3 0.473 step 4: pairwise comparison in the qfd, pairwise comparisons of the quality, flexibility and delivery criteria were performed with the superdecisions software (saaty, 2006). the priorities of quality, flexibility and delivery are expressed in figure 2. figure 2 comparison of criteria step5: relative technical significance rating the priority rating pi of 60 assigned to quality, tsr and rtsr was calculated for each enabler as shown in table 5. for example: product cost enabler, tsr was calculated: tsr= 60 (0.115+0.539) + 26 (0.374+0.184) + 14 (0.789+0.740) tsr= 75.154 rtsr was calculated by rtsr= 75.154/528.426= 0.142 ijahp article: kumar, r, singh h., singh a./a framework for evaluation of vendors in the automotive industry international journal of the analytic hierarchy process 498 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.696 table 5 relationship between the criteria and the enabler and between the enablers and the vendors criteria sub-criteria product cost accuracy of order fulfillment annual turnover geographical location in kms technical capability experience attitude quality (60) percent rejections 0.115 0.625 0.115 0.115 0.789 0.680 0.184 product durability 0.539 0.115 0.473 0.115 0.587 0.515 0.146 delivery (26) order lead time 0.374 0.680 0.239 0.539 0.382 0.16 0.587 delivery on time every time 0.184 0.789 0.210 0.539 0.435 0.205 0.638 flexibility (14) volume flexibility 0.789 0.233 0.336 0.115 0.639 0.115 0.382 customization 0.741 0.146 0.184 0.115 0.295 0.3 0.473 tsr 75.154 87.9 54.24 45.048 116.864 85.6 63.62 rtsr 0.142 0.166 0.103 0.85 0.221 0.162 0.120 step 6: relative user requirements for calculation of user requirements (ur) and relative user requirements (rur), each vendor was rated against each enabler. the ur and rur were calculated in table 6. step 7: mathematical model formulation the qualitative data namely, quality, delivery and flexibility were transformed into quantitative data using fuzzy qfd. this data was combined with lead time to formulate the mathematical model. the lead times of the vendors are mentioned in table 7. the team decided that lead time must not exceed 45 days as shown in figure 3. since rur has to be maximized, the first sub-problem is palanisamy and zubar, 2012: ijahp article: kumar, r, singh h., singh a./a framework for evaluation of vendors in the automotive industry international journal of the analytic hierarchy process 499 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.696 table 6 user requirements and relative user requirement table 7 lead time of vendors (data provided by organization) s. no vendors lead time in days 1 vendor 1 45 2 vendor 2 40 3 vendor 3 45 4 vendor 4 50 5 vendor 5 47 6 vendor 6 45 7 vendor 7 42 8 vendor 8 50 s.no v1 v2 v3 v4 v5 v6 v7 v8 product cost(0.142) 0.104 0.104 0.104 0.146 0.115 0.125 0.146 0.156 accuracy of order fulfilment (0.166) 0.147 0.147 0.147 0.118 0.118 0.103 0.132 0.088 geographical location (0.85) 0.172 0.171 0.214 0.168 0.172 0.0009 0.150 0.144 technical capability (0.221) 0.168 0.168 0.168 0.071 0.062 0.109 0.168 0.853 annual turnover(.103) 0.239 0.287 0.837 0.002 0.075 0.120 0.170 0.022 experience(0.16 2) 0.125 0.174 0.156 0.081 0.067 0.096 0.220 0.078 attitude(0.120) 0.158 0.158 0.147 0.103 0.077 0.112 0.147 0.087 ur 0.286 0.298 0.386 0.224 0.223 0.1 0.276 0.262 rur 0.139 0.145 0.188 0.109 0.109 0.049 0.134 0.127 ijahp article: kumar, r, singh h., singh a./a framework for evaluation of vendors in the automotive industry international journal of the analytic hierarchy process 500 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.696 figure 3 lead time constraint in tora software formulate minimizing the lead time problem in tora software the outcome of maximizing the tcs problem was 0.188 for the given threshold value of lead time. to obtain alternative optimal solutions, the minimum value of tcs was relaxed to 0.100 in the minimum lead time problem and the problem becomes: (palanisamy and zubar, 2012) step 8. output of qfd qfd was implemented on eight vendors of sae-8620 steel and the pre-qualified vendors are shown in table 8. the expert team decided that a pool of three vendors was satisfactory for making the final selection. table 8 qfd result alternate solutions rur lead time vendors 0.188 43 v3 0.145 40 v2 0.134 42 v7 3.3 implementation of the analytic network process (anp) step 1: construct the anp model the process of decision making for vendor ranking requires an evaluation of the decision according to the benefits (b), opportunities (o), cost (c), risk (r) model. in this research, many sub-criteria under benefits, opportunities, cost, and risk were identified for ranking vendors in the superdecisions software as shown in figure 4. ijahp article: kumar, r, singh h., singh a./a framework for evaluation of vendors in the automotive industry international journal of the analytic hierarchy process 501 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.696 step 2. design of the anp model in superdecisions software the network model was constructed using the design module of the superdecisions software (saaty, 2003). the anp model was constructed with control criteria and subcriteria classified by four merits namely, benefits, opportunities, costs and risks as shown in figure 5. for each control criterion of the b, o, c, r, the priorities for the alternatives of the decision are derived from all of the significant influences that cause some of the alternatives to have higher priorities. step 3. pairwise comparison of different control criteria with respect to vendor selection the pairwise comparison matrix was developed using group decision making with four experts who work at different levels in the organization. the pairwise comparison of the control criterion with respect to the vendor selection cluster was done with the software and the priorities of the control criteria were obtained as shown in figure 6. figure 4 anp-bocr model ijahp article: kumar, r, singh h., singh a./a framework for evaluation of vendors in the automotive industry international journal of the analytic hierarchy process 502 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.696 figure 5 design of anp–bocr model in superdecisions software figure 6 node comparison with respect to vendor selection step 4. verification of the consistency of the judgments after the pairwise comparisons are made, the consistency of the judgments must be confirmed. if the judgments are not consistent, there may have been a mistake in the judgments or in the formulation of the problem, and it is necessary to correct the pairwise comparisons or the formulation of the problem. four experts at different levels in the organization were selected to complete the questionnaire. the inconsistency was automatically calculated while the data from the questionnaire was input into the software, and the inconsistency value must be less than (0.1) as shown in figure 7. figure 7 representation of inconsistency ijahp article: kumar, r, singh h., singh a./a framework for evaluation of vendors in the automotive industry international journal of the analytic hierarchy process 503 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.696 step 5. priorities determined with the analytical network process (anp) model the limiting values of the bocr model were obtained from the supermatrix and the priorities of the bocr model were obtained by normalizing the respective cluster. step 6: ranking of vendors based on benefits, opportunities, costs and risks the output was obtained from the anp-bocr network model, and vendor v2 is the best vendor with respect to the benefits merit, followed by v3 and v7. with respect to opportunities, v2 is the best vendor, followed by v3 and v7. with respect to costs, v7 is the best supplier, followed by v3 and v2. with respect to risks, v2 is the best supplier, followed by v3 and v7 and is shown in figure 8. all of the results were obtained based on normal values. figure 8 ranking of vendors with respect to benefits, opportunities, costs and risks step 7. overall synthesized priorities of vendors in the anp–bocr model the overall synthesized priorities of the vendors in the anp-bocr were obtained based on normal values and the overall ranking of the vendors is v2, v3, andv7. therefore, vendor v2 is the best vendor from among the three vendors as shown in figure 9. figure 9 overall synthesized priorities of vendors in anp-bocr model ijahp article: kumar, r, singh h., singh a./a framework for evaluation of vendors in the automotive industry international journal of the analytic hierarchy process 504 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.696 4. results and discussion fuzzy (qfd) was used to form a pre-qualified vendor pool and the house of quality (hoq) was used to convert the experts’ responses from linguistic expressions to quantitative data. the relative importance rating indicates the priority for any engineering characteristic and becomes the basis for the decision making about what actions should be taken to improve the particular engineering characteristics. the qfd and anp ranking results were compared with the organizational rating of vendors. the first hoq gives the technical significance rating (tsr) and the relative technical significance rating (rtsr) as shown in figure 10. the second hoq gives the relative user requirements (rur) as shown in figure 11. figure 10 relative technical significance rating after the formation of the vendor pool, the pairwise comparisons were input into the superdecisions software and based on the ranking of vendors that was obtained, a final selection was made. the following results were obtained which illustrate the ranking of the vendors under the four merits of benefits, opportunities, costs, risks and the total ranking. vendor v2 is the best vendor with respect to the benefits merit, followed by v3 and v7. the evaluation of the vendors was done based on normal values as shown in figure 12. figure 11 relative user requirements ijahp article: kumar, r, singh h., singh a./a framework for evaluation of vendors in the automotive industry international journal of the analytic hierarchy process 505 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.696 figure 12 ranking of vendors with respect to benefits  vendor v2 is the best vendor with respect to the opportunities merit, followed by v3 and v7. the evaluation of the vendors was done based on normal values as shown in figure 13.  vendor v7 is the best vendor with respect to the costs merit, followed by v3 and v2. the evaluation of the vendors was done based on normal values as shown in figure 14. figure 13 ranking of vendors with respect to opportunities ijahp article: kumar, r, singh h., singh a./a framework for evaluation of vendors in the automotive industry international journal of the analytic hierarchy process 506 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.696 figure 14 ranking of vendors with respect to costs  vendor v7 is the best vendor with respect to the risks merit, followed by v3 and v2. the evaluation of the vendors was done based on normal values as shown in figure 15. figure 15 ranking of vendors with respect to risk  vendor v7 is the best vendor with respect to the total ranking of the anpbocr, followed by v3 and v2. the evaluation of the vendors is done based on normal values as shown in figure 16. figure 16 ranking of vendors with respect to the total ranking of the anp-bocr model ijahp article: kumar, r, singh h., singh a./a framework for evaluation of vendors in the automotive industry international journal of the analytic hierarchy process 507 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.696  overall, the synthesized priorities of the vendors in the anp-bocr were obtained based on normal values and the overall ranking of the vendors is v2, v3, andv7. therefore, vendor v2 is the best vendor as shown in table 9. table 9 ranking of qualified vendors of sae-8620 steel based on the anp-bocr model alternatives raw values ideal values normal values ranking vendor v2 0.705668 1 0.462572 1 vendor v3 0.567337 0.803970 0.371894 2 vendor v7 0.25258 0.357857 0.165534 3 table 10 shows the comparison of the vendor ranking done by the organization and the hybrid multi-criteria decision making approach of the qfd and anp. the organization had not adopted any multi-criteria decision making framework for the selection of a vendor; in fact, they provided the ranking based only on their expertise. according to the organization, the vendor v2 ranked second, v3 ranked third, and v7 ranked fourth. however, in the ranking from the hybrid multi-criteria decision making approach of qfd-anp, vendor v2 ranked first, followed by v3 and v7. the results of the vendor ranking showed that when it is done based only on the quality, delivery and quality of the system criteria it is not satisfactory for evaluation of the best vendors. the criteria included in this research are also very crucial for ranking and evaluating the best vendors. table 10 comparison of the case organization’s vendor ranking with the qfd-anp method vendors organization vendor ranking qfd-anp vendor ranking v2 2 1 v3 3 2 v7 4 3 5. conclusions and future work in the present work, a model was implemented for the problem of vendor selection in an automotive industry. the combined qfd and anp approach was implemented to obtain reliable results. tora and super decisions software were used to minimize the computation time and chance for error. the result of the qfd show that vendor v2, v3 and v7 are good suppliers of the sae-8620 material. the final ranking of the vendors was achieved using the anp approach. the results showed that the technical capability is the best enabler based on the subjective weights in the selection of vendors for the selected organization. the outcome of the proposed work was that v2 is the best vendor for the selected case company. this work could be extended by using a sensitivity ijahp article: kumar, r, singh h., singh a./a framework for evaluation of vendors in the automotive industry international journal of the analytic hierarchy process 508 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.696 analysis. other decision making approaches such as topsis and vikor could be implemented to compare the results. a limitation of this work is that the proposed anp model is only applicable to the case company. ijahp article: kumar, r, singh h., singh a./a framework for evaluation of vendors in the automotive industry international journal of the analytic hierarchy process 509 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.696 references aggarwal, r. singh, s.p., & kapur, p.k. (2018). integrated dynamic vendor selection and order allocation problem for the time dependent and stochastic data. benchmarking: an international journal, 25(3), 777-779. doi: https://doi.org/10.1108/bij-05-2017-0085 akoa, y. (1990). history of quality function deployment in japan. international academy for quality, 3(1), 183-196. andronikidis, a., & andreas, c. (2014), the application of quality function deployment in service quality management. the tqm journal, 21(2), 319-333. doi: https://doi.org/10.1108/17542730910965047 gencer, c., & gurpinar, d. (2007). analytic network process in supplier selection: a case study in an electronic firm. applied mathematical modeling, 31(11), 2475-2486. doi: https://doi.org/10.1016/j.apm.2006.10.002 ho, w., bennett, d.j., mak, k.l., chuah, k.b., lee, c.k.m. & hall, m.j. (2009). strategic logistics outsourcing: an integrated qfd and ahp approach. ieee international conference on industrial engineering and engineering management, 24(2), 1434-1438. doi: https://doi.org/10.1109/ieem.2009.5373059 hui, c. & yang, l. (2013). hierarchical management mode applied in 3pl. international conference on environmental science and information application technology (esiat), 3(2), 199-202. john, k., sun, y., & du, y. (2005). vendor evaluation and rating using analytical hierarchy process. international journal of engineering science and innovative technology, 2(3), 447-455. kant, r. & dalvi, m.v. (2017). development of questionnaire to assess the supplier evaluation criteria and supplier selection benefits. benchmarking: an international journal, 24(2), 359-383. doi: https://doi.org/10.1108/bij-12-2015-0124 kirytopoulos, k., leopoulos, v. & voulgaridou, d. (2008). supplier selection in pharmaceutical industry: an analytic network process approach. benchmarking: an international journal, 15(4), 494-516, 2008. doi: https://doi.org/10.1108/14635770810887267 kirytopoulos, k., leopoulos, v., & voulgaridou, d. (2008). supplier selection in pharmaceutical industry: an analytic network process approach. benchmarking: an international journal, 15(4), 494-516. doi: https://doi.org/10.1108/14635770810887267 kumar, r., singh, h., & dureja, j. s. (2012). an approach to analyze logistic outsourcing problem in medium-scale organization by cfpr and vikor. management, 23(7), 885898. doi: https://doi.org/10.1108/17410381211267718 ijahp article: kumar, r, singh h., singh a./a framework for evaluation of vendors in the automotive industry international journal of the analytic hierarchy process 510 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.696 koul, s. & verma, r. (2011). dynamic vendor selection based on fuzzy ahp. journal of manufacturing technology management, 22(8), 963-971. doi: https://doi.org/10.1108/17410381111177421 li, c., sun, y. & du, y. (2011). selection of 3pl service suppliers using a fuzzy analytic network process. control and decision conference, 12(2), 2174-2179. doi: https://doi.org/10.1109/ccdc.2008.4597709 mathiyazhagan, k., sudhakar, s., & bhalotia, a. (2017). modeling the criteria for selection of suppliers towards green aspect: a case in indian automobile industry. opsearch, 1-20. doi: https://doi.org/10.1007/s12597-017-0315-8 mohammed, a., harris, i. and dukyil, a. (2019). a trasilient decision making tool for vendor selection: a hybrid-mcdm algorithm, management decision, 57(2), 372395. doi: https://doi.org/10.1108/md-04-2018-0478 onut, s., kara, s.s. & işik, e. (2009). long term supplier selection using a combined fuzzy mcdm approach: a case study for a telecommunication company. expert systems with applications, 36(5), 3887-3895. doi: https://doi.org/10.1016/j.eswa.2008.02.045 palanisamy, p. & zubar, a. (2012). hybrid mcdm approach for vendor ranking. journal of manufacturing technology management, 24(2), 905-928. doi: https://doi.org/10.1108/jmtm-02-2012-0015 qian, l. (2009). an ann pruning algorithm based approach to vendor selection. kybernetes, 38(3/4), 314-320. doi: https://doi.org/10.1108/03684920910943994 rao, r.v. (2007). decision making in manufacturing environment using graph theory and fuzzy multiple attribute decision making methods presents the concepts and details of applications of madm methods. springer series in advanced manufacturing. doi: https://doi.org/10.1007/978-1-4471-4375-8_1 saaty, t.l. (1996). decision making with dependence and feedback: analytic network process. pittsburgh: rws publications. saaty, t.l. (2004). decision making-the analytic hierarchy and network processes (ahp/anp). journal of systems science and systems engineering, 13(1), 1-35. doi: https://doi.org/10.1007/s11518-006-0151-5 shih, h. s., cheng, c. b., chen, c. c., & lin, y. c. (2014). environmental impact on the vendor selection problem in electronics firms-a systematic analytic network process with bocr. international journal of the analytic hierarchy process, 6 (2), 202-227. doi: https://doi.org/10.13033/ijahp.v6i2.256 sonmez, m. (2006). a review and critique of supplier selection process and practices. occasional papers series, loughborough university, uk. https://doi.org/10.1109/ccdc.2008.4597709 https://www.emerald.com/insight/search?q=ahmed%20mohammed https://www.emerald.com/insight/search?q=irina%20harris https://www.emerald.com/insight/search?q=abdulsalam%20dukyil https://www.emerald.com/insight/publication/issn/0025-1747 ijahp article: kumar, r, singh h., singh a./a framework for evaluation of vendors in the automotive industry international journal of the analytic hierarchy process 511 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.696 suraraksa, j. and shin, k. s. comparative analysis of factors for supplier selection and monitoring: the case of the automotive industry in thailand. sustainability, 11(4), 981. doi: https://doi.org/10.3390/su11040981 talluri s., & narasimhan, r. (2003). vendor evaluation with performance variability: a max-min approach. european journal of operational research, 146(4), 543-552. doi: https://doi.org/10.1016/s0377-2217(02)00230-8 zhang, h., li, x., liu, w., li, b. & zhang, z. (2004). an application of the anp in vendor selection of a 4pl system. proceedings of the ieee international conference on systems, man and cybernetics, 2(7), 1255-1260. doi: https://doi.org/10.1109/icsmc.2004.1399797 ijahp article: chaudhari, wasu, sarode/ ranking different enablers/drivers of sustainable supply chain management by using ahp in indian manufacturing industries international journal of the analytic hierarchy process 272 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.711 ranking different enablers/drivers of sustainable supply chain management by using ahp in indian manufacturing industries jayant suresh chaudhari chaudharijay2000@gmail.com department of mechanical engineering lokmanya tilak college of engineering koparkhairne. navi mumba, university of mumbai dr. renu wasu renuwasu@gmail.com department of applied chemistry lokmanya tilak college of engineering koparkhairne. navi mumbai, university of mumbai dr. avinash sarode avinashsarode@gmail.com department of mechanical engineering lokmanya tilak college of engineering koparkhairne. navi mumbai, university of mumbai abstract in a global economy characterized by environmental, social and economic factors, environmental sustainability is currently one of the main concerns of industry and the economic sector. a large number of periodicals and special volumes related to the sustainable supply chain have been published. this paper intends to identify the drivers for sustainable supply chain management (sscm) implementation. twenty-eight enablers were identified and categorized using an extensive literature survey to improve the effectiveness of sscm implementation. the authors attempted to identify the drivers/enablers and constructed a framework, which analyzed the sscm using the ahp. keywords: sustainable; supply chain; drivers/enabler; ahp 1. introduction for any country, economic development is supported by the growth of its manufacturing industries. currently, manufacturing industries are experiencing tough competition. each industry must strive to improve productivity in all of its spheres of activity in order to survive (sarode & khodke, 2011). because of the environmental movement, the term sustainable development has evolved over the past 30 years. sustainable development is defined as development that meets the needs of the present without compromising the ability of future generations to meet their own needs (brundtland commission, 1987). sustainability is considered an innovative approach, including changes in previous existing processes, new technology, improved methods of management, and new mailto:chaudharijay2000@gmail.com mailto:renuwasu@gmail.com mailto:avinashsarode@gmail.com ijahp article: chaudhari, wasu, sarode/ ranking different enablers/drivers of sustainable supply chain management by using ahp in indian manufacturing industries international journal of the analytic hierarchy process 273 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.711 production systems, which may bring changes in supply chain management (scm). these possible areas of change include old policies, production activities, inventory of goods and product management, and dispatching (jayaratne et al., 2011). currently, sustainability has become a global concern, and therefore, organizations are motivated to revisit their supply chain operations and consider their environmental and social impact (capaldi, 2005). this has given rise to sustainability and scm, green supply chain management (gscm), as well as sustainable supply chain management (sscm) (ashby et al., 2012). the integration of sustainability into scm began with a focus on merging "green" considerations with scm practices. therefore, sscm is an extension of the gscm concept. according to carter and rogers (2007), sscm is the strategic, transparent integration and achievement of the social, environmental and economic objectives of an organization by the systematic coordination of key interorganizational business processes to enhance each company's long-term economic performance and supply chain. sustainability in supply chains needs to reduce the environmental, social and economic impact. basically, enablers/drivers are defined in layman’s terms as an entity that makes something possible or easy. therefore, enablers for sustainable supply chains are processes that can drive a supply chain to be sustainable. in this paper, we attempt to identify and rank the drivers/enablers for sustainable supply chains. in order to identify the enablers in sscm, it is necessary to prepare a method that is capable of collecting the appropriate information. these enablers/drivers will be further incorporated into sscm to facilitate decision-making. therefore, the authors have identified 28 enablers to solve the above problem. the data was obtained from various manufacturing industries in india. twenty enablers were categorized within the seven main criteria, which include regulation, society, market, environment, economic, corporate, organization. this paper ranks the enablers in the context of indian manufacturing using the ahp with the goal of enhancing the supply chains. the structure of this paper is as follows: the literature review is presented in section 2, while section 3 presents the solution methodology with the ahp framework. the ranking of the enablers/drivers are discussed in the results and discussion in section 4. finally, section 5 summarizes the conclusion and future scope of research. 2. literature review this section reviews the literature on sscm and identifies the enablers that are important to the execution of sustainable practices in indian organizations. the literature review was used to identify gaps in the research. svensson (2007) presented an empirical study in order to illustrate the aspects of scm through the expansion of existing theories, and introduced several new terms such as first, second, and n-order supply chains in order to enhance corporate efforts in sscm. faisal (2010) presented an approach to adapt sustainable practices in a supply chain by analyzing the dynamics between various enablers that help transform a supply chain into a truly sustainable entity. the ism approach was used to present a hierarchy-based model. wittstruck and teuteberg (2010) contributed to the sscm research by providing a model that explains which factors impact sscm success and how sscm should be ijahp article: chaudhari, wasu, sarode/ ranking different enablers/drivers of sustainable supply chain management by using ahp in indian manufacturing industries international journal of the analytic hierarchy process 274 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.711 established to enable successful sscm networks in electronic industries that provides benefits and successful results. carter and easton (2011) demonstrated that the environment is, of course, a key component of the triple bottom line and has been at the forefront of sscm research on climate change. the sscm field has evolved from independent social and environmental research to corporate social responsibility to the beginnings of a convergence of sustainability perspectives as the triple bottom line and the emergence of sscm as a theoretical framework. wolf (2011) concurred that the supply chain is a set of business entities that are directly involved in the upstream or downstream flow of products, services, and information from a source to a customer. this definition places the consumer at the end of the supply chain and reflects a linear production paradigm that assumes a constant input of natural resources. diabat and kannan (2011) developed a model of the drivers that affect the implementation of green supply chain management (gscm) practices in organizations using an ism methodology. walker and jones (2012) pointed out that there is a wide gap between what practitioners say and what they actually do about the sustainability of supply chains; often they only provide lip service to sustainable supply chain management. chikanikova and mont (2012) found that food retail sustainability in the supply chain could largely be explained as an approach to corporate risk management, and therefore, maintain a competitive position, i.e., compliance strategy and taxonomy development of drivers and barriers drawing on hoffman's framework and analyzing their relative importance for the initiation of upstream, in-store and downstream sustainability initiatives. zailani (2012) investigated the extent of implementation of sustainable supply chain management practices (environmental purchasing and sustainable packaging) and found that environmental purchasing has a positive effect on three categories of outcomes (economic, social and operational), whereas sustainable packaging has a positive effect on environmental, economic and social outcomes. grzybowska (2012) identified the enablers of sustainability in supply chains and explored their mutual relationships. sixteen enablers were identified, and top management and adequate adoption of reverse logistic practices (environmental performance) had the highest driving and dependence power. gopalkrishnan et al. (2012) demonstrated that social and environmental initiatives can increase financial gains, thereby encouraging supply chains to take a positive approach to sustainability. through a detailed literature review, mathiyazhagan et al. (2013) identified pressures for gscm implementation. sixty-five pressures were identified and categorized into six major groups. then, the most common acceptable pressures were identified and prioritized using the analytical hierarchy process through a questionnaire survey from different industrial sectors. beske and seuring (2014) identified five key categories that are highly important to sscm as follows: orientation towards scm and sustainability, continuity, collaboration, risk management, and proactivity. they also described distinctive practices that allow an organization to follow the goals formulated in the five key categories. marshall et al. (2014) developed a multidimensional concept and measure of social and environmental sscm practices based on a multi-stage procedure involving a literature review, expert qsort and pre-test process, pilot test and survey. ali diabat et al. (2014) found influential enablers for sscm using interpretive structural modelling (ism) from thirteen recommended enablers in five indian textile units located in southern india. these revealed that five enablers dominate the industry's practices including adoption of safety standards, adoption of green practices, community economic welfare, health and safety issues, and employment stability. grimm et al. (2014) focused on the food industry and ijahp article: chaudhari, wasu, sarode/ ranking different enablers/drivers of sustainable supply chain management by using ahp in indian manufacturing industries international journal of the analytic hierarchy process 275 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.711 helped overcome the complexities and unique challenges of sub-supplier management and identified 14 csf’s which were classified into focal firm, relationship, supply chain partner, and context-related csfs. through a literature review, tay et al. (2014) identified the barriers and drivers of sscm implementation and found that there are factors that have been documented to influence an organization in making a decision to implement sscm. luthra et al. (2014) analyzed six critical success factors to the implementation of gscm to achieve sustainability, and four expected performance measures were extracted using factor analysis. dubey and gunasekaran (2015) attempted to develop a responsive sustainable supply chain network that can respond to a degree of uncertainty due to uncontrollable forces and developed a multi-objective milp model to handle high uncertainties related to demand and supply. stiller and gold (2015) studied the neglected issue of how to include the social dimension of sustainability into scm and developed some of the following categories through an analytical framework: reconceptualizing supply chain design, supply base continuity, decommodization, traditional supplier development, novel supplier development, transparency and traceability, and reward and incentive systems. luthra and haleem (2015) identified various hurdles in the implementation of sscm in the indian automotive industry. the ism methodology was utilized to understand the contextual relationship among these identified hurdles, their interdependence, and the hierarchy levels to implement sscm practices in the indian automobile sector. gopal and thakkar (2016) analyzed twentyfive critical success factors (csfs) based on organizational theory and modeled them to execute successful implementation of sustainable supply chain practices in the indian automobile industry. sarode and kole (2016) found that environmental policy for gscm, green design, initiation of top management support, involvement of suppliers and vendors in green practices, green manufacturing practices, reverse logistics, and recycling programs are the major subcritical factors according to the literature. dubey et al. (2017) identified drivers for the adoption of sscm, and proposed the use of tism and a cross impact matrix-multiplication applied to classification (micmac) analysis to test a framework that extrapolates sscm drivers and their relationships. raut et al. (2017) tried to identify the numerous csfs that are needed to implement sscm practices, and attempted to explore the interdependence between them, which presented considerable challenges due to the complex nature of green practices, customers, suppliers, cost pressures and uncertainty of regulations. mathiyazhagan et al. (2017) analyzed and prioritized the most important drivers for the implementation of gscm in the indian construction industry and 27 drivers were identified within seven categories. this paper tried to present a benchmarking framework for ranking the drivers for implementation of the gscm. from the above literature review, it is evident that the past research studies on the implementation of sustainable practices have been conducted in different countries and industries. not many studies have covered the importance of sscm implementation practices/issues in indian manufacturing industries. very few research studies have focused on the manufacturing industry, and fewer still have dealt with sustainable implementation practices. this shows that there is a research gap in the implementation of sustainable practices in the manufacturing sector. table 1 lists the enablers/drivers derived from the literature. ijahp article: chaudhari, wasu, sarode/ ranking different enablers/drivers of sustainable supply chain management by using ahp in indian manufacturing industries international journal of the analytic hierarchy process 276 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.711 table 1 enablers/drivers from literature drivers authors g o v e r n m e n t r e g u la ti o n is o 1 4 0 0 1 n g o h e a lt h & s a fe ty c o m p e ti ti v e n e ss s u p p li e r m a n a g e m e n t a d o p ti o n o f e n v ir o n m e n ta l s td . a d o p ti o n o f g r e e n p r a c ti c e s g r e e n d e si g n g r e e n m a r k e ti n g g r e e n p a c k a g in g g r e e n p u r c h a si n g a d o p ti o n o f s a fe ty s td . in it ia ti o n o f t o p m a n a g e m e n t s tr a te g ic p la n n in g m u tu a l t r a n sp a r e n c y c o ll a b o r a ti v e w it h p a r tn e r s t e c h n o lo g y m a n a g e m e n t q u a li ty m a n a g e m e n t c o st p e r fo r m a n c e r e w a r d s & i n c e n ti v e s o r g a n iz a ti o n m a n a g e m e n t o r g a n iz a ti o n c a p a b il it y e ff o r t e m p lo y e e i n v o lv e m e n t & t r a in in g c o r p o r a te s o c ia l r e sp o n si b il it y r e v e r se l o g is ti c s it e n a b le m e n t l o g is ti c s & t r a n sp o r ta ti o n faisal (2010)         wolf (2011)      gopalkrishna n et al. (2012)          walker, jones (2012)           wittstruck, teuteberg (2012b)       buykozkan, cifci (2013)            beske, seuring (2014)           chein, sinh (2007)       diabat et al. (2014), mathiyazhga n, kannan devika         grim et al. (2014)     stiller & gold (2014)      chkanikova & mont (2015)      kuo-chung shang et al. (2015)            dubey (2015)       ferreira (2015)         jabbour (2015b)       luthra et al. (2015)               ijahp article: chaudhari, wasu, sarode/ ranking different enablers/drivers of sustainable supply chain management by using ahp in indian manufacturing industries international journal of the analytic hierarchy process 277 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.711 arefhervani, joseph sarkis (2005)        kole (2016)            zailani, jeyaraman (2012)    diabat, govindan (2010)        svensson (2007)        tay, rahman, aziz, sidek (2014)             gopal, thakkar (2016)          pan jehfouladgar an (2014)            marshall, mccarthy, mcgrath (2014)    dubey, gunasekaran , childe, wamba (2016)          raut, narkhede (2016)                    3. methodology the goal of this work is to investigate how sscm practices influence the different dimensions of sscm performance and the competitiveness of an organization based in india. 3.1 overview of ahp the analytical hierarchy process (ahp) is a theory of general measurement used to derive relative scales from discrete and consistently matched comparisons. these comparisons are made using actual measurements or a baseline scale reflecting the relative strength of preferences and feelings. the ahp is particularly concerned about the inconsistency of discrepancy, its measurement, and the dependence of its structure within and among the elementary groups. it has found wider applications in multi-criteria decision-making, planning, resource allocation and conflict resolution. the ahp, in its general form, is a nonlinear framework for deductive and inductive reflection without the use of syllogism, and simultaneously accounts for several factors of synthesis or inference. t.l. saaty developed the ahp at the wharton school (pennsylvania university, philadelphia, pennsylvania) between 1971-1975. ijahp article: chaudhari, wasu, sarode/ ranking different enablers/drivers of sustainable supply chain management by using ahp in indian manufacturing industries international journal of the analytic hierarchy process 278 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.711 the ahp is a multi-criteria decision-making approach introduced by saaty (1980) that consists of three main operations as follows: hierarchy construction, priority analysis, and consistency verification. the decision variables hierarchy is the subject of a pairwise ahp comparison. the pairwise comparisons are based on a nine-point scale that converts human preferences as equally, moderately, strongly, very strongly or extremely preferred. 3.1.1 steps of the ahp methodology 1) to identify the enablers and structure the prioritization hierarchy model. 2) to prepare a questionnaire and gather data for the pairwise comparisons. 3) to determine the standardized weights in each category for each enabler and specific enablers. 4) to check the consistency of the judgments by calculating the consistency ratio (cr), and eventually revising the comparative matrices by asking experts if the consistency is too low in the judgments. if the cr is less than 0.1, the judgments will be consistent. 5) to synthesize and analyze the data using the ahp technique. the acceptable cr range varies depending on the size of the matrix. the following guidelines are provided when making decisions based on the cr.  when the cr value is equal to or less than the recommended value for a specific matrix size, the matrix evaluation is acceptable or has a good level of consistency in the comparative judgments represented in that matrix. this helps ensure the reliability of the decision-makers in determining the priorities of the criteria.  if the cr is greater than the acceptable value, the judgments in that matrix are inconsistent and the evaluation process should be reviewed, reconsidered and improved. 3.2 identification of the main and sub-drivers in sscm sscm implementation has been attempted using several drivers that can enable a sustainable supply chain. twenty-eight enablers/drivers were identified from the literature review and categorized into seven main categories as seen in table 1 as follows: 1. regulatory these drivers are exercised in the form of standards, laws, procedures, and incentives of national or supranational (regional or international) regulatory institutions to promote sustainability practices. 2. societal these pressures help raise public awareness of various sustainability issues. 3. market market drivers are responsible for the shape of the market which organizations consider a major concern. 4. corporate integrating the principle of sustainability at a strategic level is the prerequisite for successfully achieving the sustainability goals of the organizations. 5. environmental this definition contains language that is related to the environmental dimension of sustainability, for example, product recycling and reuse, natural resource exploitation, water use, disposal of chemical wastes, product life-cycle impact, etc. ijahp article: chaudhari, wasu, sarode/ ranking different enablers/drivers of sustainable supply chain management by using ahp in indian manufacturing industries international journal of the analytic hierarchy process 279 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.711 6. economic this definition includes language related to the economic dimension of sustainability. it may also include monetary savings in terms of reducing transportation costs, inventory management, logistics and freight, energy consumption, etc. 7. organization – it has access to adequate resources and direct influence on the organization’s motivation for sustainability. figure 1 is based on table 1 and shows the number of studies that are focused on the enablers. this shows that the maximum number of studies considered the following enablers: initiation of top management, organization management, supplier management, strategic planning, and mutual transparency. figure 1 enablers vs. number of studies 3.3 framework for sscm drivers the ahp was used to prioritize the enablers for a sustainable supply chain. an ahpbased framework with four levels of hierarchy as shown in figure 2 was developed. level 1 of the hierarchy indicates the research objective, i.e., to analyze drivers/enablers for sustainability enhancement. level 2 comprises a main driver/enabler category. in level 3, the enablers/drivers are subcategorized. the last level of the hierarchy prioritizes the drivers/enablers. ijahp article: chaudhari, wasu, sarode/ ranking different enablers/drivers of sustainable supply chain management by using ahp in indian manufacturing industries international journal of the analytic hierarchy process 280 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.711 table 2 abbreviation used in framework of sscm drivers abbreviation drivers abbreviation 1.regulation [reg] 5. corporate [cor] government regulation [reg 1] initiation of top management [cor 1] iso 14000 [reg 2] strategic planning [cor 2] 2. society [soc] mutual transparency [cor 3] non-government organization [soc 1] collaborative with partners [cor 4] health & safety [soc 2] technology management [cor 5] 3. market [mar] quality management [cor 6] competitiveness [mar 1] 6. economic [eco] supplier management [mar 2] cost performance [eco 1] 4. environment [env] rewards & incentives [eco 2] adoption of environment standard [env1] 7.organization [org] adoption of green practices [env 2] organization management [org 1] green design [env 3] organization capability effort [org 2] green marketing [env 4] employee training [org 3] green packaging [env 5] corporate social responsibility [org 4] green purchasing [env 6] reverse logistics [org 5] adoption of safety standard [env 7] it enablement [org 6] logistics & transportation [org 7] ijahp article: chaudhari, wasu, sarode/ ranking different enablers/drivers of sustainable supply chain management by using ahp in indian manufacturing industries international journal of the analytic hierarchy process 281 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.711 figure 2 ahp framework for ranking the drivers in sscm 3.4 development of survey instrument based on the literature review, the list of drivers used in the sscm was developed. in the pre-testing phase of the questionnaire, industry representatives were consulted about their views on drivers and sub drivers, which were selected for further study. each sub driver in the questionnaire was most important to the main driver/enabler and was based on a five-point likert scale. we performed two surveys; the first survey included an overview of all of the sub drivers, and the second survey consisted of pairwise comparisons among the main drivers with the ahp. both of the questionnaires were divided into two sections; the first section collected organizational information, and the second section, which was the body of the survey was arranged in tabular format with multiple choice grid variables ranging from not important to very important, which represented the likert scale and was used because it was easy for the respondent to understand. in the second survey of main drivers for pairwise comparison, saaty’s nine-point scale was used. this scale is used to assign relative weights to the pairwise comparisons between the main drivers. 3.5 data collection the data collection involved meeting manufacturing industries in india and sending them the questionnaire. academicians and industry people with relevant subject matter expertise reviewed the questionnaire. we developed the pilot study using the first 15 responses, and subsequent follow-ups were done. their responses were analyzed and incorporated into the questionnaire before it was executed. the data was collected using convenience sampling of 166 respondents who are top and middle level management identification of essential drivers/enablers of sscm in indian manufacturing industry. level 1 level 2 level 3 level 4 ijahp article: chaudhari, wasu, sarode/ ranking different enablers/drivers of sustainable supply chain management by using ahp in indian manufacturing industries international journal of the analytic hierarchy process 282 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.711 executives in industrial engineering, operations, and sustainable supply chain management. the questionnaires were designed to facilitate the data collection for the ahp and pairwise comparisons. the questionnaires were sent to relevant experts in 166 companies that were selected with the help of an indian industry directory and through the internet. of the total 166 questionnaires that were mailed out, 35 were returned by the end of four months, representing a response rate of 21.08%. to increase the response rate, a reminder was sent to each of the companies after two weeks and in some cases personal calls were also made. we received 12 additional responses after these reminders. due to time constraints, we began our analysis with these 47 responses, which provided a response rate of 28.31% which was acceptable. a response rate of more than 20% is acceptable for data analysis (malhotra & grover, 1998). 3.6 response from survey we performed this survey based on saaty’s scale throughout manufacturing industries. we sent the questionnaire to the manufacturing industries and received responses from some of the targeted area. the pie chart in figure 3 shows the number of years of experience that the respondents possessed. thirty-two percent of the respondents had from 11-15 years of experience, 24% had from 16-20 years of experience, 23% had 6-10 years of experience, and 21% had less than 5 years. figure 3 aggregation of survey 4. results and discussion the collected data was processed using the ahp as described earlier. this section gives a stepwise processing of the data and discussion of the results. 4.1 relative weights for sscm drivers the responses were collected by sending the questionnaire via google survey and e-mail. the average values of the 47 responses were used to construct a matrix (7 x 7) for the pairwise comparisons as shown in table 3. ijahp article: chaudhari, wasu, sarode/ ranking different enablers/drivers of sustainable supply chain management by using ahp in indian manufacturing industries international journal of the analytic hierarchy process 283 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.711 table 3 pair wise comparison matrix for drivers reg sc mar env cor eco org reg 1 1.53 2.21 3.67 2.78 2.92 3.86 soc 0.65 1 1.96 2.65 3.10 2.87 2.89 mar 0.45 0.51 1 1.97 2.04 2.62 2.78 env 0.27 0.37 0.5 1 2.04 2.88 2.15 cor 0.35 0.32 0.49 0.49 1 2.10 2.12 eco 0.34 0.34 0.38 0.34 0.47 1 1.88 org 0.25 0.34 0.35 0.46 0.471 0.53 1 column total 3.31 4.41 6.89 10.58 11.901 14.92 16.68 the relative importance of the row element with respect to the corresponding column element is indicated by each cell value in table 4. if the row element dominates the column element, then the cell value is a decimal or otherwise. likewise, the matrix's diagonal is unity, as a value compared to itself is 1. the values obtained in table 4 were standardized by dividing each cell value by the total column in order to facilitate data handling. the standardized matrix is presented in table 5. table 4 normalized matrix for pairwise comparison of drivers reg soc mar env cor eco org eigen/priorit y vector reg 0.302 0.346 0.32 0.346 0.233 0.195 0.231 0.281 soc 0.196 0.226 0.44 0.250 0.26 0.192 0.173 0.248 mar 0.135 0.115 0.145 0.186 0.171 0.175 0.166 0.156 env 0.081 0.083 0.072 0.094 0.171 0.193 0.128 0.117 cor 0.105 0.072 0.071 0.046 0.084 0.140 0.127 0.092 eco 0.102 0.077 0.055 0.032 0.039 0.067 0.112 0.069 org 0.075 0.077 0.05 0.043 0.039 0.035 0.059 0.054 column total 1 1 1 1 1 1 1 ijahp article: chaudhari, wasu, sarode/ ranking different enablers/drivers of sustainable supply chain management by using ahp in indian manufacturing industries international journal of the analytic hierarchy process 284 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.711 table 5 calculation of λmax reg soc mar env cor eco org weighted sum calculation of λmax 0.281 0.248 0.156 0.117 0.092 0.069 0.054 reg 0.281 0.379 0.344 0.429 0.255 0.201 0.208 2.097 7.306 soc 0.183 0.248 0.305 0.310 0.285 0.198 0.156 1.73 6.975 mar 0.127 0.126 0.156 0.230 0.187 0.180 0.150 1.156 7.410 env 0.076 0.091 0.078 0.117 0.187 0.198 0.116 0.863 7.376 cor 0.099 0.079 0.076 0.057 0.092 0.144 0.114 0.661 7.184 eco 0.096 0.084 0.059 0.039 0.043 0.069 0.101 0.491 7.115 org 0.07 0.084 0.054 0.053 0.043 0.036 0.054 0.394 7.296 λmax 7.237 ci = 0.0395 consistency ratio(cr) = 0.029 the ahp's results are consolidated in table 6, which shows the prioritization of the enablers in the sustainable supply chain. manufacturing industries can focus on meaningful enablers to be more efficient in sustaining the market. the complete results of table 6 with the global rank and local rank of the enablers/drivers are in appendix 1. the top 15 results of these enablers are discussed below. ijahp article: chaudhari, wasu, sarode/ ranking different enablers/drivers of sustainable supply chain management by using ahp in indian manufacturing industries international journal of the analytic hierarchy process 285 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.711 table 6 ranked drivers for sscm in indian manufacturing industries enablers priority weights for enablers global priority weights for enablers global rank government regulation (reg 1) 0.826 0.232 1 non-government organization (soc 1) 0.741 0.183 2 competitiveness (mar 1) 0.848 0.132 3 health & safety (soc 2) 0.258 0.0639 4 iso 14001 (reg 2) 0.173 0.0486 5 adoption of environment standard (env 1) 0.355 0.0415 6 initiation of top management (cor 1) 0.348 0.0301 7 supplier management (mar 2) 0.151 0.0235 8 organization management (org 1) 0.409 0.022 9 strategic planning (cor 2) 0.187 0.0172 10 collaboration with partners (cor 4) 0.183 0.0168 11 adoption of green practices (env 2) 0.143 0.0167 12 mutual transparency (cor 3) 0.169 0.0164 13 green design (env 3) 0.140 0..0163 14 green marketing (env 4) 0.102 0.0119 15 4.2 discussion this section discusses the obtained results and presents some managerial implications of the research. regulation: regulation drivers have a major impact on the sustainability approaches of organizations and may have the ability to dictate that organizations adopt certain sustainability practices. there are two enablers listed in this category. the regulation category (reg) received the highest priority weight (0.281). government regulation (reg 1) ranked first in overall drivers (0.232) and iso 14001 (reg 2) ranked fifth overall with 0.0486. these certifications are very important for companies as they enhance the company's image and show that their work or company is certified by the government. society: the society category (soc) ranked second with a relative weight of 0.248. the society enablers help raise public awareness about various sustainability issues such as resource scarcity, environmental damage, human rights, social well-being, etc. in this category, there are two drivers listed. non-governmental organizations (soc 1) ranked ijahp article: chaudhari, wasu, sarode/ ranking different enablers/drivers of sustainable supply chain management by using ahp in indian manufacturing industries international journal of the analytic hierarchy process 286 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.711 second overall (0.183), and health and safety (soc 2) ranked fourth overall with a relative weight of (0.0639). market: the market category (mar) ranked third overall with a priority weight of 0.156. the market category is responsible for the market shape, which is considered one of the main concerns of an organization. competitiveness and supplier management are categorized within the market driver. competitiveness (mar 1) is ranked third (0.132) and supplier management (mar 2) is ranked eighth (0.0235). suppliers can provide valuable sustainable ideas and suggestions, but cooperation and integration in supply chains can more effectively support sustainability. environment: environment was ranked fourth overall with a priority weight of 0.117. in the environment category, adoption of environment standards (env 1) obtained the highest rank and adoption of safety standards (env 7) received the lowest rank. adoption of environment standards reduces the environmental impact of a company and improves operational efficiency and efficiency aspects (0.0415). green practice adoption (env 2) had a relative weight of (0.0167), and green design (env 3) ranked 14 th with a relative weight of 0.0163. ecological design saves material and costs, reduces emissions, accidents, consumption of energy and waste. green marketing (env 4) ranked 15 th overall (0.0119). the use of plastic, as we all know, is harmful to the environment, which increases the need for green packaging (0.0094). green purchasing (env 6) is a process that involves material reduction, reuse and recycling, with a relative weight of 0.0081. corporate: integrating the principle of sustainability at a strategic level is the prerequisite for successfully achieving industrial sustainability goals. the relative weight of the corporate category (cor) is 0.092. within this category, the initiation of top management (cor 1) is the most important and the least important is technology management (cor 6). top management's commitment includes management's effort and financial support for sustainability implementation (0.0328) and ranked seventh. strategic planning (cor 2) is an integral part of any organization and an important step in successfully implementing supply chain management (0.0172). in the supply chain, mutual transparency (cor 3) shapes the sourcing, procurement, logistics, partnerships and customer practices of industries or companies every day (0.0164). in the overall ranking, collaboration with partners (cor 4) ranked 11 th (0.0168). management of technology (cor 5) has a relative weight of 0.0053. in the context of supply chains, quality management (cor 6) is defined as a performance enhancement approach that is based on systems that leverage opportunities created by upstream and downstream connections with suppliers and customers (0.0057). economic: in the economic category, the cost performance driver (eco 1) is more important than rewards and incentives (eco 2). sustainable practices include material reduction, reuse, and recycling, which in turn reduce the cost of purchasing materials, component manufacturing, production time, energy consumption, waste treatment, waste discharge and logistics (0.0044). incentives, rewards, tax rebates, or soft loans will encourage companies to implement practices that are sustainable (0.0024). organization: in the organization category, there were seven sub categories of drivers. organization management obtained the highest rank, and transportation ranked the lowest. there is a great need for organizational commitment (org 1) from top managers ijahp article: chaudhari, wasu, sarode/ ranking different enablers/drivers of sustainable supply chain management by using ahp in indian manufacturing industries international journal of the analytic hierarchy process 287 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.711 and support from mid-level managers and other staff (0.022). organizational capacity assessment (org 2) is necessary in order to implement green practices and evaluate the maintenance of sustainability in the organization (0.0108). in employee involvement and training (org 3), the awareness of senior members of management about the benefits of sustainability will help them make environmentally friendly decisions (0.0042). csr (org 4) is an integral part of the process of wealth creation, although it does not provide the company with an immediate financial benefit. if properly managed, it will enhance business competitiveness and maximize returns (0.0054). reverse logistics (org 5) addresses the reuse of the products related operations (0.0055) and includes refurbishing and remanufacturing activities. process management enabled by it (org 6) will be useful in strategic planning by giving access to information in real time (0.0036). logistics and transportation (org 7) aims to organize forward distribution of transportation, warehousing, packaging and inventory management from the manufacturer to the consumer. environmental considerations opened up recycling and disposal markets and led to an entirely new reverse logistics subsector (0.002). table 6 shows the drivers as they are ranked by the ahp analysis. due to strict government and environmental regulations and the demands of environmental accountability, environmental issues have become an intrinsic part of strategic planning in organizations (walton et al., 1998). a sustainable supply chain may help organizations gain a competitive advantage and secure the loyalty of all of the stakeholders in the coming years, including shareholders and investors (gladwin, 1992). the top management of a firm and the decision-makers must know the importance of the various sustainable csfs and the tools and techniques needed to implement them. the csfs for sustainability have considerable challenges because of the complex nature of green practices, customer, supplier, cost pressures and regulation uncertainty. in fact, implementing sustainability practices is considered a thankless task that increases the overall cost of a product (hsu et al., 2008). in general, developing countries implement sustainability practices that are enforced by legislation, and in developed countries, sustainability is used as a tool to reach out to socially and environmentally conscious customers and build a positive brand image (de brito et al., 2008). luthra (2015) and mathiyazhagan (2017) ranked a few enablers/drivers, while in this paper, we ranked the top fifteen enablers/drivers that will help indian manufacturing industries/companies select the appropriate driver to improve their supply chain in the context of sustainable supply chain management. 5. conclusion a comprehensive literature review was conducted to identify various enablers/drivers that help implement sscm practices. based on the literature review, 28 drivers were identified and divided into seven categories these categories included regulation, society, market, environment, corporate, economic and organization. government law/regulation, ngo, green design, green marketing, etc. were also identified as drivers. this study found that not all enablers/drivers have the same influence on the adoption of sscm, and focused on enablers/drivers for sscm from an indian perspective. the results of this research successfully rank the ahp-based enablers/driver’s priorities. this has provided a comprehensive industry solution for enabler identification and given a benchmark for the implementation of industrial sscm. all of the pairwise comparisons made in the ijahp article: chaudhari, wasu, sarode/ ranking different enablers/drivers of sustainable supply chain management by using ahp in indian manufacturing industries international journal of the analytic hierarchy process 288 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.711 ahp were based on the experts’ opinions. the categorization of the drivers is not the final verdict on the subject, as many other relevant drivers could be identified and categorized depending on the goals and perspectives of future studies. in addition, this study included only 28 enablers; hence, more enablers need to be considered in future studies using statistical methods for validation. the analytic network process (anp) and interpretive ranking process (irp) may be considered for further studies to determine the ranking of the main drivers and sub-drivers. in future studies, the fuzzy ahp, which is able to give experts the freedom to express their judgments through natural language, may be considered. ijahp article: chaudhari, wasu, sarode/ ranking different enablers/drivers of sustainable supply chain management by using ahp in indian manufacturing industries international journal of the analytic hierarchy process 289 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.711 references ahi p. & searcy, c. (2013). a comparative literature analysis of definitions for green and sustainable supply chain management. journal of cleaner production, 52, 329-341. doi: https://doi.org/10.1016/j.jclepro.2013.02.018 ahi, p. & searcy, c. (2015). an analysis of metrics used to measure performance in green and sustainable supply chains. journal of cleaner production, 86(1), 360-371. doi: 10.1016/j.jclepro.2014.08.005 ahmed n. (2003). environmental concerns, efforts and impact: an empirical study. midamerican journal of business, 18(1), 61-69. ashby, a., leat, m. & smith m.h., (2012). making connections: a review of supply chain management and sustainability literature. supply chain management: an international journal, 17(5), 497-516. doi: https://doi.org/10.1108/13598541211258573 beske, p., land, a. & seuring, s. (2014). sustainable supply chain management practices and dynamic capabilities in the food industry: a critical analysis of the literature. international journal of production economics, 152, 131-143. doi: https://doi.org/10.1016/j.ijpe.2013.12.026 beske, p. & seuring, s. (2014). putting sustainability into supply chain management. supply chain management international journal, 19(3), 322-331. doi: https://doi.org/10.1108/scm-12-2013-0432 brandenburg, m., govindan, k., sarkis, j. & seuring, s. (2014). quantitative models for sustainable supply chain management: development and directions. european journal of operation research, 233, 299-312. doi: https://doi.org/10.1016/j.ejor.2013.09.032 brundtland commission, our common future, oxford university press: oxford, 1987. business for social responsibility (2007). perspectives on information management in sustainable supply chains. http://www.bsr.org/reports/bsr_info-management-supplychains.pdf capaldi, n. (2005). corporate social responsibility and the bottom line. international journal of social economics, 32(5), 408-423. doi: https://doi.org/10.1108/03068290510591263 carter, c. r. & easton, p. l. (2011). sustainable supply chain management: evolution and future directions. international journal of physical distribution & logistics management, 41(1), 46-62. doi: https://doi.org/10.1108/09600030810882816 carter, c. & rogers, d. (2007). a framework of sustainable supply chain management: moving toward new theory. international journal of physical distribution and logistics management, 38(5), 360 – 387. doi: https://doi.org/10.1108/09600030810882816 https://doi.org/10.1016/j.jclepro.2013.02.018 https://www.researchgate.net/deref/http%3a%2f%2fdx.doi.org%2f10.1016%2fj.jclepro.2014.08.005 http://www.bsr.org/reports/bsr_info-management-supply-chains.pdf http://www.bsr.org/reports/bsr_info-management-supply-chains.pdf ijahp article: chaudhari, wasu, sarode/ ranking different enablers/drivers of sustainable supply chain management by using ahp in indian manufacturing industries international journal of the analytic hierarchy process 290 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.711 chaabane, a., ramudhin, a. & paquet, m. (2012). design of sustainable supply chains under the emission trading scheme. international journal of production economics, 135(1), 37-49. doi: https://doi.org/10.1016/j.ijpe.2010.10.025 chaudhari, j.s. & sarode, a. d. (2018). identification of drivers in sustainable supply chain management: a review. international journal of research in science & engineering, 4(2), issn: 2394-8299. chkanikova, o. & mont, o. (2015). corporate supply chain responsibility: drivers and barriers for sustainable food retailing. corporate social responsibility environment management, 22(2), 65-82. doi: https://doi.org/10.1002/csr.1316 clift, r. (2003). metrics for supply chain sustainability. springer-verlag. corbett, c. j. & kleindorfer, p.r. (2003). environmental management and operations management: introduction to the third special issue. production and operations management, 12(3), 287–289. doi: https://doi.org/10.1111/j.1937-5956.2003.tb00203.x daly, h. & cobb, j. (1989). for the common good: redirecting the economy toward community, the environment and a sustainable future. boston: beacon press. de brito, m.p., carbone, v. & blanquart, c.m. (2008). towards a sustainable fashion retail supply chain in europe: organisation and performance. international journal of production & economics, 114(2), 534–53. doi: https://doi.org/10.1016/j.ijpe.2007.06.012 dehghanian, f. & mansour, s. (2009). designing sustainable recovery network of endof-life products using genetic algorithm. resources, conservation and recycling, 53(10), 559-570. doi: https://doi.org/10.1016/j.resconrec.2009.04.007 diabat, a. & govindan, k. (2011). an analysis of the drivers affecting the implementation of green supply chain management. resources, conservation and recycling, 55, 659-667. doi: https://doi.org/10.1016/j.resconrec.2010.12.002 diabat, a., kannan, d. & mathiyazhagan, k. (2014). analysis of enablers for implementation of sustainable supply chain management a textile case. journal of cleaner production, 83, 391-403. doi: https://doi.org/10.1016/j.jclepro.2014.06.081 dubey, r., gunasekaran, a. & childe, s.j. (2015). the design of a responsive sustainable supply chain network under uncertainty. international journal of advance manufacturing technology, 80(1), 427-445. doi: https://doi.org/10.1007/s00170-015-6967-8 dubey, r., gunasekaran, a., papadopolus, t., childe, s., shibin, k. t. & wamba, s. (2017). sustainable supply chain management: framework and further directions. journal of cleaner production, 142, 1119-1130. doi: https://doi.org/10.1016/j.jclepro.2016.03.117 elkington, j. (1997). cannibals with forks: the triple bottom line of 21st century business. environmental quality management, 2, 37-51. doi: https://doi.org/10.1002/tqem.3310080106 https://doi.org/10.1016/j.ijpe.2010.10.025 https://doi.org/10.1002/tqem.3310080106 ijahp article: chaudhari, wasu, sarode/ ranking different enablers/drivers of sustainable supply chain management by using ahp in indian manufacturing industries international journal of the analytic hierarchy process 291 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.711 faisal, m.n. (2010). sustainable supply chains: a study of interaction among the enablers. business process management journal, 16(3), 508-529. doi: https://doi.org/10.1108/14637151011049476 ferreira, m.a., jabbour, c.j.c. & de sousa jabbour, a.b.l. (2015). maturity levels of material cycles and waste management in a context of green supply chain management: an innovative framework and its application to brazilian cases. journal of material cycles waste management, 19, 516-525. doi: https://doi.org/10.1007/s10163-015-0416-5 gladwin, t. (1992). the meaning of greening: a plea for organizational theory. in fischer k, schott j. (eds), environmental strategies for industry. washington, dc: island press. gopal, p.r.c. & thakkar, j. (2016). sustainable supply chain practices: an empirical investigation on indian automobile industry. production planning control management operation, 27(1), 49-64. gopalakrishnan, k., yusuf, y.y., musa, a., abubakar, t. & ambursa, h.m. (2012). sustainable supply chain management: a case study of british aerospace (bae) systems. international journal of production economics, 140(1), 193-203. doi: https://doi.org/10.1080/09537287.2015.1060368 grimm, j.h., hofstetter, j.s. & sarkis, j. (2014). critical factors for sub-supplier management: a sustainable food supply chains perspective. international journal of production economics, 152, 159-173. doi: https://doi.org/10.1016/j.ijpe.2013.12.011 grover r., grover, r., rao, b. & kejriwal, k. (2016). supplier selection using sustainable criteria in sustainable supply chain management. international journal of economics and management engineering, 10(5), 1775-1779. grzybowska, k. (2012). environment issues in supply chain management. springerverlag. doi: 10.1007/978-3-642-23562-7_2 hamidzera p. & bahiroie, n. (2014). role of critical success factors in sustainable supply chain management. international journal of applied research on industrial engineering, 1(5/6), 320-328. hertwich, e. g. & peters, g.p. (2009). carbon footprint of nations: a global, trade-linked analysis. environment science technology, 43(16), 6414–6420. doi: https://doi.org/10.1021/es803496a hervani a. & helms m. (2005). performance measurement for green supply chain management. benchmarking: an international journal, 12(4), 330-353. doi: https://doi.org/10.1108/14635770510609015 hsu, c.w. & hu, a.h. (2008). green supply chain management in the electronic industry. international journal of environment, science and technology, 5(2), 205–16. ijahp article: chaudhari, wasu, sarode/ ranking different enablers/drivers of sustainable supply chain management by using ahp in indian manufacturing industries international journal of the analytic hierarchy process 292 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.711 jayaratne, p., styger, l. & perera, n. (2011). sustainable supply chain management: using the sri lankan tea industry as a pilot study. 25th annual australia new zealand academy of management conference, 1, 1-22. jinturkar, a., deshmukh, s., sarode, a., sunapwar, v. & khodke, p. (2014). fuzzy-ahp approach to improve effectiveness of supply chain. in kahraman, öztayşi (eds), supply chain management under fuzziness (35-59). springer. doi: https://doi.org/10.1007/978-3-642-53939-8_3 kothari, c.r. (2004). research methodology: methods and techniques. delhi: new age international publication. laroche m., bergeronand, j., & forleo, g.b. (2001). targeting consumers who are willing to pay more for environmentally friendly products. journal of consumer marketing, 18(6), 503-520. doi: https://doi.org/10.1108/eum0000000006155 linton, j., klassen, r. & jayaram, v. (2007). sustainable supply chains: an introduction. journal of operations management, 25(6), 1075-1082. doi: https://doi.org/10.1016/j.jom.2007.01.012 luthra, s. & haleem a. (2015). hurdles in implementing sustainable supply chain management: an analysis of indian automobile sector. procedia – social and behavioral sciences, 189, 175-183. doi: https://doi.org/10.1016/j.sbspro.2015.03.212 luthra, s., garg, d. & haleem, a. (2014). critical success factors of green supply chain management for achieving sustainability in indian automobile industry. production planning control, 26(5), 339 -362. malhotra, m.k. & grover, v. (1998). an assessment of survey research in pom: from constructs to theory, journal of operations management, 16(4), 407–425. doi: https://doi.org/10.1016/s0272-6963(98)00021-7 markley, j. m & davis, l. (2007). exploring future competitive advantage through sustainable supply chains. international journal of physical distribution and logistics management, 37(9), 763-774. doi: https://doi.org/10.1108/09600030710840859 marshall, d., mccarthy, l., heavey, c. & mcgrath, p. (2015). environmental and social supply chain management sustainability practices: construct development and measurement. production planning control, 26(8), 673-690. doi: https://doi.org/10.1080/09537287.2014.963726 mathiyazhagan, k., dattta, u., bhadauria, r., singla, a. & krishnamoorthi, s. (2017). identification and prioritization of motivational factors for the green supply chain management adoption: case from indian construction industries. operation research society of india, 55(1), 202-219. doi: https://doi.org/10.1007/s12597-017-0316-7 mathiyazhagan, k., govindan, k. & haq, a. (2014). pressure analysis for green supply chain management implementation in indian industries using analytic hierarchy process. javascript:void(0) javascript:void(0) https://doi.org/10.1007/s12597-017-0316-7 ijahp article: chaudhari, wasu, sarode/ ranking different enablers/drivers of sustainable supply chain management by using ahp in indian manufacturing industries international journal of the analytic hierarchy process 293 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.711 international journal of production research, 52(1), 188-202. doi: https://doi.org/10.1080/00207543.2013.831190 muduli, k., & barve, a. (2011). role of green issues of mining supply chain on sustainable development. international journal of innovation, management and technology, 2(6), 484. rakesh, r., narkhede, b. & gardas, b. (2016). to identify the critical success factor of sustainable supply chain management practices in the context of oil and gas industries: ism approach. renewable and sustainable energy reviews, 68, 33-47. doi: https://doi.org/10.1016/j.rser.2016.09.067 saaty, r. decision making in complex environments, the analytic network process (anp) for dependence and feedback. superdecisions. seuring, s. & müller, m. (2008). from a literature review to a conceptual framework for sustainable supply chain management. journal of cleaner production, 16(15), 1699-710. saaty t. l. & ozdemir, m.s. (2003). why the magic number seven plus or minus two. mathematical and computer modelling, 38, 233-244. doi: https://doi.org/10.1016/j.jclepro.2008.04.020 saaty t. l. (1980). the analytic hierarchy process. new york: mcgraw-hill. sarode a. d. & khodke p. m. (2011). a framework for performance measurement of supply chain management. international journal of advanced engineering technology, 2(4), 182-190. sarode a.d., sunnapwar, v. k. & khodke, p. m., (2008). a literature review for identification of performance measures for establishing a framework for performance measurement in supply chains. international journal of applied management and technology, 6(3), 241-273. sarode, a.d., adarsh, t.g. & khodke, p.m. (2010). development and validation of performance measures for vendor selection in indian manufacturing industries. supply chain management journal, 2(26), 31-34. sarode, a.d. & kole, s. (2016). a literature overview on green supply chain management and critical factor. international journal of advanced engineering and innovative technology, 3(1), 1-5. sarode, a.d., sunnapwar, v.k. & khodke, p.m., (2010). improving effectiveness of supply chain by selecting an appropriate supplier: an analytic hierarchy process approach. international journal advance manufacturing systems, 9(2), 129-144. doi: https://doi.org/10.1142/s0219686710001855 shen, l., muduli, k. & barve, a. (2015). developing a sustainable development framework in the context of mining industries: ahp approach. resource policy, 46, 1526. doi: https://doi.org/10.1016/j.resourpol.2013.10.006 https://doi.org/10.1080/00207543.2013.831190 http://scholarworks.waldenu.edu/ijamt/vol6/iss3/12/ http://scholarworks.waldenu.edu/ijamt/vol6/iss3/12/ http://scholarworks.waldenu.edu/ijamt/vol6/iss3/12/ https://doi.org/10.1016/j.resourpol.2013.10.006 ijahp article: chaudhari, wasu, sarode/ ranking different enablers/drivers of sustainable supply chain management by using ahp in indian manufacturing industries international journal of the analytic hierarchy process 294 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.711 sodhi, m. & tang, c. (2017). corporate social sustainability in supply chains: a thematic analysis of the literature. international journal production research. doi: 10.1080/00207543.2017.1388934 srivastava (1995). the role of corporations in achieving ecological sustainability. academy of management review, 20(4), 936-60. stiller, s. & gold, s. (2014). socially sustainable supply chain management practices in the indian seed sector: a case study. supply chain forum international journal, 15(1), 52-67. doi: https://doi.org/10.1080/16258312.2014.11517333 svensson, g. (2007). aspects of sustainable supply chain management (sscm): conceptual framework and empirical example. supply chain management international journal, 12(4), 262-266. doi: https://doi.org/10.1108/13598540710759781 tay, m.y., rahman, a.a., aziz, y.a. & sidek, s. (2015). a review on drivers and barriers towards sustainable supply chain practices. international journal of social science and humanity, 5(10), 892-897. doi: https://doi.org/10.7763/ijssh.2015.v5.575 teuteberg, f. & wittstruck, d. (2010). a systematic review of sustainable supply chain management research: what is there and what is missing? mkwi 2010 – corporate environment and sustainable management, 1001-1015. walker, h. & jones, n. (2012). sustainable supply chain management across the uk private sector. supply chain management international journal, 17(1), 15-28. doi: https://doi.org/10.1108/13598541211212177 walton, s.v., handfield, r.b. & melnyk, s.t. (1998). the green supply chain: integrating suppliers into environmental management process. international journal of purchasing and materials management, 34(1), 2-11. doi: https://doi.org/10.1111/j.1745493x.1998.tb00042.x wittstruck, d. & teuteberg, f. (2012). understanding the success factors of sustainable supply chain management: empirical evidence from the electrics and electronics industry. corporate social responsibility environment management, 19(3), 141-158. doi: https://doi.org/10.1002/csr.261 wolf, j. (2011). sustainable supply chain management integration: a qualitative analysis of the german manufacturing industry. journal of business ethics, 102(2), 221-235. doi: https://doi.org/10.1007/s10551-011-0806-0 zahir, s. (1999). clusters in group: decision making in the vector space formulation of the analytic hierarchy process. european journal of operational research, 112, 620– 634. doi: https://doi.org/10.1016/s0377-2217(98)00021-6 zailani, s., jeyaraman, k., vengadasan, g. & premkumar, r. (2012). sustainable supply chain management (sscm) in malaysia: a survey. international journal of production economics, 140(1), 330-340. https://doi.org/10.1080/16258312.2014.11517333 https://doi.org/10.1108/13598541211212177 https://doi.org/10.1111/j.1745-493x.1998.tb00042.x https://doi.org/10.1111/j.1745-493x.1998.tb00042.x https://doi.org/10.1016/s0377-2217(98)00021-6 ijahp article: chaudhari, wasu, sarode/ ranking different enablers/drivers of sustainable supply chain management by using ahp in indian manufacturing industries international journal of the analytic hierarchy process 295 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.711 appendix 1 ranked drivers of sscm in indian manufacturing industries enabler category priority weights for enabler category enablers priority weights for enablers global priority weights for enablers local rank global rank regulation (reg) 0.281 government regulation (reg 1) 0.826 0.232 1 1 iso 14001 (reg 2) 0.173 0.0486 2 5 society (soc) 0.248 non-government organization (soc 1) 0.741 0.183 1 2 health & safety (soc 2) 0.258 0.0639 2 4 market (mar) 0.156 competitiveness (mar 1) 0.848 0.132 1 3 supplier management (mar 2) 0.151 0.0235 2 8 environment (env) 0.117 adoption of environment standard (env 1) 0.355 0.0415 1 6 adoption of green practices (env 2) 0.143 0.0167 2 12 green design (env 3) 0.140 0..0163 3 14 green marketing (env 4) 0.102 0.0119 4 15 green packaging (env 5) 0.081 0.0094 5 17 green purchasing (env 6) 0.070 0.0081 6 18 adoption of safety standard (env 7) 0.041 0.0047 7 23 corporate (cor) 0.092 initiation of top management (cor 1) 0.348 0.0301 1 7 strategic planning (cor 2) 0.187 0.0172 2 10 mutual transparency (cor 3) 0.169 0.0164 4 13 collaboration with partners (cor 4) 0.183 0.0168 3 11 technology management (cor 5) 0.058 0.0053 6 22 quality management (cor 6) 0.062 0.0057 5 19 economic (eco) 0.069 cost performance (eco 1) 0.064 0.0044 1 24 rewards and incentives (eco 2) 0.035 0.0024 2 27 organization (org) 0.054 organization management (org 1) 0.409 0.022 1 9 organization capabilities effort (org 2) 0.200 0.0108 2 16 employee training (org 3) 0.079 0.0042 5 25 corporate social responsibility (org 4) 0.100 0.0054 4 21 reverse logistics (org 5) 0.103 0.0055 3 20 it enablement (org 6) 0.067 0.0036 6 26 logistics and transportation (org 7) 0.038 0.002 7 28 ijahp article: chaudhari, wasu, sarode/ ranking different enablers/drivers of sustainable supply chain management by using ahp in indian manufacturing industries international journal of the analytic hierarchy process 296 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.711 details of the calculations in table 6 are given in appendix 1. from the 47 responses, we calculated the geometric mean and eigen values for each of these drivers. because the consistency indices were in the acceptable range, they did not need to be revised. ijahp article: volinchak, whitehouse, yourstowsky, wooley, karpak/ analyzing corporate expansion to international markets: the case of germany, united kingdom, canada, mexico and china international journal of the analytic hierarchy process 98 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.574 analyzing corporate expansion to international markets: the case of germany, united kingdom, canada, mexico and china 1 crystal volinchak 2 erin whitehouse matthew yourstowsky robert wooley williamson college of business youngstown state university youngstown, oh cmvolinchak@ysu.edu emwhitehouse@student.ysu.edu mryourstowsky@student.ysu.edu rgwoolley@student.ysu.edu birsen karpak distinguished professor department of management youngstown state university bkarpak@ysu.edu abstract in this study, the authors utilized the analytic hierarchy process (ahp) decision-making model to select the optimal market for international expansion for abc corporation 1 this paper received “best student achievement in international business award for graduate students”, youngstown state university, williamson college of business, april 18, 2018. 2 acknowledgement: this project allowed our group to become better researchers, taught us how to use ahp methodology in real-life decision making and allowed us to network with colleagues around the world. this was a fantastic experience for all of us and it will not be forgotten. being able to represent youngstown state university at the mcdm, 2017 conference was an honor. we learned and did things that students cannot learn in the classroom. working alongside dr. karpak allowed us to have a hands-on experience with the project and she was there when questions needed addressed. we feel that our research benefitted abc and allowed them to gain a better understanding of what market they should export to. we are beyond grateful for this experience and glad that we were selected to go to ottawa, canada and to now be submitting our research to the ijahp journal. the authors also thank the export expert mr. mousa kassis, cgbp, director, ohio small business development center (sbdc) export assistance network, williamson college of business administration of youngstown state university, for identifying abc company and giving his expert judgments on criteria evaluations. mailto:cmvolinchak@ysu.edu mailto:emwhitehouse@student.ysu.edu mailto:mryourstowsky@student.ysu.edu mailto:rgwoolley@student.ysu.edu mailto:bkarpak@ysu.edu ijahp article: volinchak, whitehouse, yourstowsky, wooley, karpak/ analyzing corporate expansion to international markets: the case of germany, united kingdom, canada, mexico and china international journal of the analytic hierarchy process 99 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.574 located in ohio 3 . the benefits of exporting to nine different countries: germany, the united kingdom, canada, mexico, netherlands, china, united arab emirates, australia and brazil were analyzed. for the sake of more precise and in-depth research, preliminary studies performed on these nine countries were used to determine the top five markets: china, mexico, canada, germany and the united kingdom. preliminary research included multiple factors about these nine countries. market size, market growth rate, market consumption capacity, market intensity, market receptivity, commercial infrastructure, trade barriers, contribution margins, country risk and the growth rate of construction were the qualitative and quantitative criteria specifically considered. the importance of each criteria and sub-criteria were determined with export market experts and company decision makers. the ahp analysis enabled the authors to determine the best possible export market for the company by evaluating the data from china, mexico, canada, germany and the united kingdom. the robustness of the results was tested using sensitivity analysis. sensitivity analysis results were then discussed with the decision makers. the best market was selected and alternative markets were presented with degrees of preference. managerial implications of the study and future research directions will be discussed. key words: ahp implementation; prioritizing export markets; case study; multi-criteria analysis 1. introduction this study utilizes the analytic hierarchy process (ahp) decision-making methodology and decision-making software in order to make a managerial decision about exporting abc corporation's products to a new market (saaty, 2010; karpak, 2017; expert choice 11.5, expertchoice©). abc corporation is an ohio based iso certified leader in rubber and silicone extrusions manufacturing that was formed in 1991. abc's capabilities include producing a wide range of profile designs in both dense and sponge materials. commonly used rubber extrusion materials include silicone, epdm, polyisoprene, neoprene, nitrile, butyl, sbr and natural rubber. abc currently serves various industries such as automotive, mass-transit/railway, architectural/construction, water control, container, pipe, and appliance gaskets and others. abc also provides in-house die tooling, on-site silicone mixing and silicone color matching, as well as printing, cutting, splicing, and taping. in the natural course of business, expansion limited within the borders of the united states itself limits any future growth potential. to remedy this issue, since the times of the spice trade, businesses have been expanding their reach far past the borders of the united states. on average, companies who export their services or products see faster growth in sales, greater job creation, and their employees earn more than in nonexporting firms (international trade administration, n.d.). abc's current exports account for x 4 % of their total sales, and they want to grow their export sales. one of the objectives of this research was to contribute impactful knowledge to a local company to 3 company name has been disguised for confidentiality reasons. 4 again % of exports sales has been disguised for confidentiality reasons. ijahp article: volinchak, whitehouse, yourstowsky, wooley, karpak/ analyzing corporate expansion to international markets: the case of germany, united kingdom, canada, mexico and china international journal of the analytic hierarchy process 100 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.574 aid in the decision making of its export endeavors. after meeting with abc, the researchers, company decision makers and export experts determined that this research would focus on the construction industry for the most immediate growth opportunities. a literature review is discussed in section two. this section illustrates the gap in the literature and emphasizes the contribution of this paper. in section 3, the top five countries out of the nine considered in the study are introduced. from the perspective of this study, these five countries are china, uk, germany, canada and mexico. section 4 explains the methodology, the data-base in addition to data elicited from the company, and the two phases of application of ahp. these two phases are prioritizing nine potential countries (phase 1) and a more detailed analysis of the top five countries (phase 2). section 5 gives the results obtained and discusses how the authors handled inconsistency. managerial implications are articulated in section six. conclusions and future research directions are given in the final section of the paper. 2. literature review a literature review suggests that research on international market selection remains fragmented; most of the existing studies are on market entry mode selection. the literature review shows that integrated frameworks and comprehensive studies of market selection process have been rare (sakarya et al., 2007). the authors contend that most studies in the market/market entry mode selection literature focus on quantitative aspects of the process and neglect qualitative aspects. we found very few articles on ahp in international market expansion. one of the articles was on identifying the critical success factors for the information service industry in a developing international market (chen &wang, 2010). criteria used in this study were different than the criteria we use in our study since the authors wanted to identify international market entry modes. in addition, since the product researched was a software service it was quite different than a physical product. as the authors assert, “generally, software product has a characteristic which differs a great deal from those of a physical product.” the authors discovered from reviewing the earlier literature that there are few research reports focused on identifying the critical factors for investors to successfully enter the international market. the two most relevant studies to our research were papers presented at the international symposium of the analytic hierarchy process in 2014. one of them compared brazil, india and china as potential export markets for the u.s. (huston et al., 2014). the second one identified the most attractive emerging market for a business to enter among the countries of turkey, russia, india, and china (tizio, et al., 2014). both studies used ahp as a methodology. the general approaches to foreign market selection are composed of stages like preliminary screening, identification/in-depth screening and final selection (sakarya et al., 2007). preliminary assessment or “screening” identifies potential markets as candidates for subsequent in-depth analysis. macro-level indicators are used to eliminate countries that do not meet the firm’s objectives. the research of both huston et al. (2014) ijahp article: volinchak, whitehouse, yourstowsky, wooley, karpak/ analyzing corporate expansion to international markets: the case of germany, united kingdom, canada, mexico and china international journal of the analytic hierarchy process 101 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.574 as well as tizio et al. (2014) did an initial screening to identify potential markets for any company, but subsequent in-depth analysis and final selection were missing. we attempt to fill this gap in the literature. this study focuses on a small medium enterprise (sme) which to the best of our knowledge there is no study on smes market expansion using the ahp. in addition, the “one size fits all” approach is not a good idea. we used specific criteria and the specific weights that are quite particular to the company size and industry. 3. country introductions from the perspectives of this research initially, we considered nine different countries which included germany, the united kingdom, canada, mexico, netherlands, china, united arab emirates, australia and brazil. preliminary studies performed on these nine countries were used to determine the top five markets which were determined to be china, mexico, canada, germany and the united kingdom. due to the space limitation of the journal we will only give country profiles of the top five countries researched. 3.1 germany germany is a strong exporting market. the market size of germany is relatively large. the urban population of germany is 62,341,809, which makes up 77% of the total population (countries in the world, 2017). that population ranks as the 18 th largest in the world. the population uses the sixth most electricity in the world, roughly 540,100 million kwh in 2012 (world bymap, 2012). as for market intensity, the personal consumption percentage of gdp continues to decrease every year for germany. the last known personal consumption percentage of gdp for germany was 53.9% in 2015. it has had a steady decrease since 2005 (germany, n.d.). this shows that families are not willing to spend money to have an increase in gdp. unlike market intensity, the market growth rate for germany has exceeded expectations. “the german economy advanced by 1.9 percent in 2016 after growing by 1.7 percent in 2015, beating market expectations of a 1.8 percent expansion” (trading economics, n.d.). germany has seen growth over the last several years and it is predicted to increase in years to come. market consumption capacity is made up of the income share of the middle class and the annual disposable income of the middle class. the middle class is shrinking in germany as fast, if not faster than, the middle class in the united states; between 1991 and 2013 the middle class shrank by 5%. germany has the best infrastructure in the world (germany, n.d.). paved roads are plentiful, airlines and airports are located across the country, and internet access, and cellular service is vastly available. germany ranks 16 th in the world in trade as a percentage of gdp. germany is a big exporter for companies in the united states, importing roughly $62 billion from the u.s. as for economic freedom, germany ranks 17 th in the world, and is considered "mostly free". they compare to countries like iceland, japan and the united states. this freedom contributes to making it a great exporting country. however, germany does have some risk politically. the risk average in western europe is 80 and germany is slightly above with a score of 83. this needs to ijahp article: volinchak, whitehouse, yourstowsky, wooley, karpak/ analyzing corporate expansion to international markets: the case of germany, united kingdom, canada, mexico and china international journal of the analytic hierarchy process 102 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.574 be taken into consideration by a company when picking where country to export their products. 3.2 united kingdom with a population of 64.5 million, the united kingdom is not necessarily one of the larger countries to export to. however, it makes up for its smaller size with a high urban population of just over 82% and a steady gdp growth rate of 2.3%. the united kingdom also boasts a high income per capita and a gdp per capita of over $41,000. a couple notable companies based in the uk are bp (energy) and fiat (automotive) which are both top competitors in their respective industries (globaledge, united kingdom, n.d.; globaledge, 2016). the united kingdom recently left the eu which led to a slight increase in their country risk rating, which is now a3. however, the business climate rating is a1 which is the lowest rating achievable. regarding infrastructure, the uk has 462 airports, over 81 million cell phones, nearly 400,000km of highways and over 16,000km of railroads. these factors point to a very well-developed country with an already established and stable infrastructure (globaledge, united kingdom, n.d.; globaledge, 2016). the united kingdom currently imports just under 10% (over $58 trillion) of its goods from the united states which is one of its top three trade partners including germany and china (globaledge, united kingdom, n.d.; globaledge, 2016). we believe that given the factors stated above, the uk will be a strong partner in trade and will yield a high return on investment for abc company. 3.3 canada canada, being the top-pick for abc, naturally had the best overall qualities in regard to export. there is no tariff tax rate when marketing to canada, so this immediately increases the potential for a larger profit. this is also the closest nation in proximity to the united states therefore reducing shipping costs. canada's business risk rating is a1, meaning that there is virtually no real outstanding risk when doing business based on the country’s laws/politics (globaledge, canada, n.d.) the compound annual growth rate as a percentage of gdp in canada is the greatest (2.3%) of all of the countries in the study; this means that canada's growth is also reflected directly in the gdp. this along with the income share of the middle class (52.9%) is also the highest of all countries in the study. one of the biggest factors was the government spending on construction growth rate in which canada scored highest by a large margin (canada scored 18.7, while the next highest was only 13.28) (wisertrade, 2017). one final point for canada was their ease of doing business score (22 nd ). this being one of abc's first ventures into international export, a country that is considered "easy" to do business with is a big plus. 3.4 mexico mexico was one of the larger markets of the top five countries with an urban population of over 100 million people. the urban population is also growing at 1.64% which shows growth in mexico relative to its population. the income share of the middle class is also 47% which states that mexico has a large middle class in their economy. ijahp article: volinchak, whitehouse, yourstowsky, wooley, karpak/ analyzing corporate expansion to international markets: the case of germany, united kingdom, canada, mexico and china international journal of the analytic hierarchy process 103 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.574 mexico is known as a lesser developed country that is struggling with its infrastructure. mexico spends roughly $55 million of private spending on construction (globaledge, 2016). this shows that mexico is spending money on construction which is a possible sign of growth. they have a paved road density of 322,859 km and 250 airports scattered across the country. however, the government spending on construction is declining at 6.47% annually (globaledge, 2016). they have 42 ports for exporting which allows for easy shipping access. with a close location to the united states, mexico is also cheaper to ship to from the u.s. in comparison to germany, china and the united kingdom. mexico imports $55 billion per capita from the united states. however, its economic freedom index rating is 64 which is quite low compared to the other four countries. its political freedom index is also relatively low at 6.82 (globaledge, 2016). mexico has a business risk rating of a b which is one of the lowest scores out of our original nine countries. one thing that mexico excels in is that it has a 0% tariff tax rate because of nafta. this country could make a strong export partner for abc. 3.5 china china also ranked very high. the massive urban population of over 700 million paired with china's urban population growth percentage (which was the highest in the study) made china a very strong candidate for abc (center for strategic and international studies, 2017) a large population that yielded a very high consumer expenditure figure, which was over 4.3 million, the highest in the study. some of the drawbacks that impaired china were the ease of doing business score and the presence of non-tariff barriers such as the protection and enforcement of trade secrets. china also has a business risk rating of b, and a political freedom index of 4.81. these factors added an element of ambiguity which would not be ideal for the abc. the final statistic that should be mentioned is the private spending on construction, which was over 28 billion; this greatly increased the potential for profitability of exporting to this nation (wisertrade, 2017). 4. methodology wiser trade is a database resource provided by the ohio export assistance network. it provides detailed trade statistics for over 150 countries worldwide (wisertrade, 2017). we began by analyzing nine countries based on the wiser trade harmonized code database reports for three of abc's top products: gasket, washers & other seals, of vulcanized rubber (hs code 401693), silicones, in primary forms (hs code 391000) and ethylene-propylene-nonconjugated diene rubber (hs code 400270) (wiser trade, 2017). the report supplied the top 10 countries to which the united states exports these products. we deduced that the nine recurring and growing countries that should be analyzed were germany, australia, united arab emirates, brazil, united kingdom, mexico, canada, netherlands and china. at this time, we developed a 2-phase research approach. phase one uses preliminary research and criteria to rank all nine markets and phase 2 takes the top markets from phase one and adds more specific criteria that relates to the abc corporation. ijahp article: volinchak, whitehouse, yourstowsky, wooley, karpak/ analyzing corporate expansion to international markets: the case of germany, united kingdom, canada, mexico and china international journal of the analytic hierarchy process 104 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.574 4.1 phase 1 in order to conduct analysis on the potential markets, the researchers developed multiple market identification criteria. these criteria were designed to reflect the qualities of markets that abc might thrive in. the criteria were derived from the 2016 global edge market potential index study which was developed by international marketing experts, abc decision makers and export experts (çavuşgil, 1997; çavuşgil et al., 2004). the specific criteria used for phase 1 were market size, market intensity, market growth rate, market consumption capacity, market receptivity, commercial infrastructure, economic freedom, and country risk (globaledge, 2016; çavuşgil, 1997; çavuşgil et al., 2004). each of the areas of comparison includes multiple quantitative and qualitative subcriteria. a few of the more important quantitative sub-criteria are cost, insurance and freight (cif) costs, tariff tax rates and government and private spending on construction. an example of qualitative data in this study is non-tariff trade barriers. each of the criteria also includes multiple quantitative sub-criteria. some examples of important subcriteria for the preliminary research were urban population and private consumption as a percentage of gdp. each of the initial criteria was not only sourced from but also weighted based on global edge's 2017 market potential index ratings (globaledge, 2017). the global edge study ranks the market potential to provide guidance to companies that plan to expand their markets internationally. this indexing study is conducted by the michigan state university international business center to help companies compare prospect markets on several dimensions. eight dimensions are chosen to represent the market potential of a country on a scale of 1 to 100. the dimensions are measured using various indicators and are weighted to determine their contribution to the overall market potential index. global edge weights market size and market intensity as the most significant criteria (globaledge, 2017). figure 1 shows the weights that were used for phase 1 of our research. figure 1 criteria weights for phase 1 ijahp article: volinchak, whitehouse, yourstowsky, wooley, karpak/ analyzing corporate expansion to international markets: the case of germany, united kingdom, canada, mexico and china international journal of the analytic hierarchy process 105 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.574 after all the initial data was collected for the nine markets and the initial criteria were weighted, we then performed the minimum number of pair-wise comparisons of the alternatives (countries) with respect to the above criteria to derive priorities of the countries in terms of export market. after the pair-wise judgements were made, the results were tested for robustness using sensitivity analysis. sensitivity analysis is a test that determines how sensitive the priorities of each alternative are to the change of criteria importance. to analyze sensitivity, we changed the weights of each criteria to see how much variation of the criteria there could be before the outcome (top export markets) changed. we were able to change the priorities of the criteria drastically before the outcome changed. this tells us that our results are robust. results were then discussed with the export expert who was in agreement with our findings. at this time, we were confidently able to identify the best markets for abc's international expansion. figure 2 shows that china, uk, germany, canada and mexico are the top five markets. these are the markets that phase 2 focuses on. figure 2 most promising export markets 4.2 phase 2 during phase 2 we added criteria that are more specific to our analysis. the specific criteria were designed to reflect the construction industry and abc’s specific needs. the research team, the faculty advisor, abc decision makers and the export expert decided on these more specific criteria. private and government spending on construction and the logistics performance index (lpi) score were sub-criteria added to the commercial infrastructure criteria. trade barriers, which consisted of tariff barriers, non-tariff barriers and ease of doing business scores, as well as contribution margins which included shipping, insurance and freight costs were also added. it is interesting to note that unlike other studies, we were able to utilize qualitative data. we also decided that for abc market intensity was not an important factor because market intensity is based more on the individual consumer purchasing capabilities. we do not believe that this would be a driving force of the construction and infrastructure growth within each country and so it was deleted from phase 2. as a team, we decided that although the global edge weights provided a strong starting point, it was now time to weight the final criteria with the benefit of abc's expansion into the construction market specifically. to arrive at our final criteria weights (figure 3), ijahp article: volinchak, whitehouse, yourstowsky, wooley, karpak/ analyzing corporate expansion to international markets: the case of germany, united kingdom, canada, mexico and china international journal of the analytic hierarchy process 106 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.574 we worked with abc decision makers and our export expert to perform pair-wise analysis for each criteria and sub-criteria. after the export expert performed pair-wise analysis for every criterion, the top criteria were contribution margin, infrastructure and schedule b. figure 3 final criteria weights our export expert then performed pair-wise comparison on the new criteria and finished the incomplete comparisons for every criterion for the new markets. based on these comparisons of the new criteria, we found that a few of the more important sub-criteria were cost, insurance, and freight (cif) costs, tariff tax rates, non-tariff barriers, government, and private spending on construction. entire final criteria and sub-criteria are given in the appendix. 5. results after all the comparisons were made, the final results were determined. the ideal mode results show that if abc were to export to one new market it should be canada at 26.8% (figure 4). ideal mode was selected in order to identify one country as an export partner for abc. while china was previously the number one country, canada is now the best market for abc to explore exporting its construction products to. this occurred because canada excelled in the newly added criteria which were weighted quite high. ijahp article: volinchak, whitehouse, yourstowsky, wooley, karpak/ analyzing corporate expansion to international markets: the case of germany, united kingdom, canada, mexico and china international journal of the analytic hierarchy process 107 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.574 figure 4 ideal mode ranking of export markets distributive mode results show that if abc wishes to export to multiple locations, they should export about 27% of their products to canada, 22% to china, 18% to the uk, 17% to germany and 15% to mexico (figure 5). in this analysis, canada is still the best market for abc to explore. figure 5 distributive mode ranking of export markets canada was identified as the top market for abc. figure 6 shows that canada has outperformed the other four markets in the market consumption, economic and political freedom, trade barriers and contribution margin criteria. these criteria were all weighted heavily, especially contribution margin. in addition, canada was simply never the worst in any criteria. ijahp article: volinchak, whitehouse, yourstowsky, wooley, karpak/ analyzing corporate expansion to international markets: the case of germany, united kingdom, canada, mexico and china international journal of the analytic hierarchy process 108 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.574 figure 6 sensitivity graph 5.1 inconsistency in the ahp, an inconsistency measure is useful for identifying possible judgment errors and measures the logical inconsistency of the judgments. inconsistency occurs when misjudgments are made during pair-wise comparisons and is measured using a consistency ratio index commonly referred to simply as inconsistency (saaty, 2010; karpak, 2017). it is important for inconsistency ratings to be below or equal to 10%. our final inconsistency rating was 5%. though our main focus was not to achieve consistency but to elicit the importance rating of the criteria and preference of the alternatives in the mind of the decision maker, the consistency ratio was less than 0.1 in all the pairwise comparison matrices in our model, indicating that judgments made were consistent. we did not consciously monitor the inconsistency rating, but instead relied on the true and pure judgements of our experts. 6. managerial implications canada is the top export market for 35 out of 50 states in the u.s. and is a strong choice for a new-to-export strategy. canada is a highly receptive, open and transparent market for u.s. products and services with canadians spend more than 60 percent of their disposable income on goods and services from the u.s. close geographic proximity and initiatives between our governments make cross-border business increasingly seamless. since the implementation of the north american free trade agreement (nafta) in 1994, trade between our two counties has more than doubled (export.gov, 2017). figure 7 shows that canada outperforms the other countries in regard to trade barriers and contribution margin. ijahp article: volinchak, whitehouse, yourstowsky, wooley, karpak/ analyzing corporate expansion to international markets: the case of germany, united kingdom, canada, mexico and china international journal of the analytic hierarchy process 109 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.574 figure 7 sensitivity analysis of trade barriers and contribution margin we wanted to dive even deeper into the robustness tests that expert choice offers. because of sensitivity analyses, we know that china will only outperform canada with respect to contribution margin if the weight of importance of the contribution margin criteria falls below 15% (figure 8). figure 8 contribution margin sensitivity analysis canada will continue to outperform china in infrastructure unless the weight of the infrastructure criteria is increased to over 35% (figure 9). both of the findings show that the results are robust and a considerable change in weights would have to occur before these results vary. ijahp article: volinchak, whitehouse, yourstowsky, wooley, karpak/ analyzing corporate expansion to international markets: the case of germany, united kingdom, canada, mexico and china international journal of the analytic hierarchy process 110 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.574 figure 9 sensitivity analysis for infrastructure 7. conclusions and future research directions when an organization is confronted with choosing the best market the decision can often be intricate. export market prioritization problems are multi-criteria problems which might have many qualitative and quantitative factors that should be considered. we propose ahp as a multi-criteria business analytics approach in export market expansion problems. we developed a comprehensive ahp model to prioritize export markets for a rubber and silicone extrusions manufacturing company. as sakarya et al. (2007) assert most studies in the market/market entry mode selection literature focus on quantitative aspects of the process and neglect qualitative aspects. with the ahp model, we were able to consider qualitative as well as quantitative factors when assessing the different export markets. canada was determined as the most promising export market. chen and wang (2010) found out from reviewing the earlier literature that there are few research reports focused on identifying the critical factors for investors to successfully enter the international market. our study fills this gap in the literature. we identified critical success factors and prioritized them for abc company. sensitivity analysis was performed to determine the robustness of the final export market ranking. this illustrated to the decision makers that even if the importance of certain criteria change, the overall ranking of the potential export markets does not change though the degree of preference is increased or decreased. the company managers and the export expert were very certain with their decisions. the sensitivity analysis illustrated to the decision makers its importance in practical decision making and also illustrated the robustness of the best solution. in real life it is difficult, if not impossible, for a decision maker or a group of decision makers to adequately consider all of the factors of a complex decision. they need some ijahp article: volinchak, whitehouse, yourstowsky, wooley, karpak/ analyzing corporate expansion to international markets: the case of germany, united kingdom, canada, mexico and china international journal of the analytic hierarchy process 111 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.574 kind of decision support system, and ahp with its software is one of them. without these decision support systems, managers might only be able to consider a subset of important criteria, ignoring some others which might lead to incorrect decisions. in general, decision-makers are not aware of the relative accuracy of elicitation methods. in this study, among the non-dominated elicitation methods, the decision-makers were presented verbal pairwise comparisons mode to transmit their judgments. this was because it was known from our earlier research that verbal comparison mode performs best in terms of ease of use and accuracy criteria. this research can be replicated for all of abc's industries including but not limited to the automotive and appliance gasket sectors. the important criteria will change with each varying industry. this is the recommended method for abc to explore new markets and is a tool the export expert will be utilizing with abc, and others, in the future. the next steps for abc include the export expert's assistance with execution. the expert will need to use this research to explore suppliers in canada. they will be able to utilize a grant through the state of ohio to conduct export exploration travel. when ahp is implemented for a decision analysis, criteria and the alternatives are considered to be independent. if there is dependence among the criteria and/or alternatives the analytic network process (anp) is more suitable. the anp, however, requires far more comparisons which may be daunting in a practical decision environment. in real life, managers might prefer simplicity over precision. also, determining the interdependence links and their intensity are not trivial tasks. still, this is a new area of research to explore. the analytic hierarchy process methodology was easily understood by the managers. with the expert choice or decision lens software ahp empowers sensitivity analysis of the results. sensitivity analysis is important in practical decision making to illustrate robustness of the results. this study showed the decision makers that the ahp is an easy to use methodology for complex export market selection problems especially if some criteria are qualitative and others are quantitative. ijahp article: volinchak, whitehouse, yourstowsky, wooley, karpak/ analyzing corporate expansion to international markets: the case of germany, united kingdom, canada, mexico and china international journal of the analytic hierarchy process 112 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.574 appendix final criteria and sub-criteria goal: best export market market size electricity consumption urban population market growth rate compound annual growth rate of gdp compound annual growth rate of primary energy use urban population growth infrastructure government spending-construction paved roads lpi number of airports shipping ports private construction spending market receptivity per capita imports from us trade as % of gdp market consumption capacity consumer expenditure income share of middle class economic freedom business risk rating economic freedom index political freedom index trade barriers tariff tax rate non-tariff barriers ease of doing business score schedule b steel cement design and draft engineering contribution margin ijahp article: volinchak, whitehouse, yourstowsky, wooley, karpak/ analyzing corporate expansion to international markets: the case of germany, united kingdom, canada, mexico and china international journal of the analytic hierarchy process 113 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.574 criteria definitions market size: market size carries a rough estimate of the market potential using the country’s urban population and electricity consumption measured in billions of kilowatts in this study. (the world bank group, 2014; the world bank, 2016) high electricity consumption implies a country is improving and has faster economic growth. growing electricity consumption is a positive identifier of enhanced standards of living and economic development. (tizio, et al., 2014) market intensity: market intensity utilizes two variables: gross national income (gni) per capita using purchasing power parity (ppp) in us dollars and private consumption as a percentage of gdp. the first variable, gni per capita in ppp, is the gross national income divided by the mid-year population, converted into ppp. this is based on purchasing power parity, which measures the purchasing power of other countries’ currencies for the same goods. this allows for a more accurate comparison of standards of living. private consumption as a percentage of gdp observes the value of household and non-profit institution purchases divided by the total population. this is a variable to measure private purchases (cavusgil, 1997). market growth rate: market growth rate observes the compound annual growth rate (cagr) of primary energy use and the compound annual growth rate (cagr) of gdp (constant 2005 us$). increase in gdp accounts for all final sales in the market value of goods and services. additionally, market growth rate observes how the world is urbanizing and the differences in urban population growth rate/population percentage in the countries we studied (the world bank group, 2016). each region of the world is urbanizing at different paces and is creating different degrees of economic growth as a result of urbanization. (globaledge, 2017). infrastructure: commercial infrastructure looks at the most variables: government spending on construction, private construction spending, paved road density as a percentage, and logistics performance index (lpi), number of airports and shipping ports. commercial infrastructure measurements define the overall accessibility of distribution and communication abilities of the country. because the costs associated with overcoming issues from infrastructure and connectivity can quickly become overwhelming to an expanding business, commercial infrastructure is an important dimension to pay attention to when looking at a potential market to enter. market receptivity: market receptivity utilizes the variables of per capita imports from the us and trade as a percentage of gdp in this study. these two variables measure the openness of the country to trade with the united states of america (us) and other foreign countries as well. a country’s imports alone are not a clear indicator of its receptivity to other markets. therefore, it is important to measure its total trade as a percentage of gdp, as well as how open to trade they are with the us. (tizio, et al., 2014) market consumption capacity: market consumption capacity is measured by two variables, percentage share of middle-class in consumption/income and consumer ijahp article: volinchak, whitehouse, yourstowsky, wooley, karpak/ analyzing corporate expansion to international markets: the case of germany, united kingdom, canada, mexico and china international journal of the analytic hierarchy process 114 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.574 expenditure. the middle class has a good amount of purchasing capacity, and makes up between 20 to 80 percent of the country’s income (cavusgil, 1997). businesses looking to enter into a country’s market need to consider purchasing power of the middle class and goods and services popular to them. raising household income of the low to middle class is the force driving much of the economic growth (tizio, et al., 2014). economic freedom: economic freedom uses the business risk rating, economic freedom index and the political freedom index. the economic freedom index ranks countries based on the number of and intensity of government regulations on wealth creating activities. political rights rating score is derived from measures such as election process, functioning of government, freedom of expression and belief, and organizational rights. the freedom in the world global report assesses real-world rights and civil liberties experienced by individuals not government reports (freedom house, 2018). countries are given a rating of free, partly free, or not free and a numerical rating of 1 to 7 with 1 being very free with a range of civil liberties and 7 being few to no civil liberties (tizio, et al., 2014). business risk rating: "probability of loss due to economic and/or political instability in the buyer's country, resulting in an inability to pay for imports. (online business dictionary, n.d.) the business risk rating (brr) is determined by the associated risk of investing in the foreign country. these ratings were found using standard & poor’s banking industry country risk assessment (bicra) rating. these scores reflect the country’s’ economic strength and creditworthiness of its financial institutions. bicra scores examine rated and unrated financial institutions and their relationship to the country’s banking industry as a whole and are scored on a scale of 1 to 10, 1 being the lowest risk banking and 10 being the highest-risk banking (tizio, et al., 2014). trade barriers: "a government imposed restriction on the free international exchange of goods or services." (online business dictionary, n.d.) in this study trade barriers consisted of tariff barriers, non-tariff barriers and ease of doing business scores. schedule b: "the export classification system of the united states, and is administered by the foreign trade division of the census bureau, which is part of the u.s. department of commerce." (nemer, n.d.) contribution margin: contribution margin (cm), defined as selling price minus variable cost, is a measure of the ability of a company to cover variable costs with revenue. in this study shipping, insurance and freight costs were used as a proxy for cm since the company did not want to share contribution margins in different countries. ijahp article: volinchak, whitehouse, yourstowsky, wooley, karpak/ analyzing corporate expansion to international markets: the case of germany, united kingdom, canada, mexico and china international journal of the analytic hierarchy process 115 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.574 references center for strategic and international studies (2017). how well-off is china's middle class? retrieved from http://chinapower.csis.org/china-middle-class/ chen, m.k., wang, s.c. (2010). the critical factors of success for information service industry in developing international market: using analytic hierarchy process (ahp) approach, expert systems with applications, 37(1), 694-704. doi: https://doi.org/10.1016/j.eswa.2009.06.012 countries in the world by population, 2017. retrieved from http://www.worldometers.info/world-population/population-by-country/ çavuşgil, s.t., (1997). measuring the potential of emerging markets: an indexing approach, business horizons, 40(1), 87-91. doi: https://doi.org/10.1016/s00076813(97)90030-6 çavuşgil, s.t., kıyak, t., yeniyurt, s. (2004). complementary approaches to preliminary foreign market opportunity assessment: country clustering and country ranking, industrial marketing management, 33(7), 607-617. doi: https://doi.org/10.1016/j.indmarman.2003.10.005 decision lens. https://www.decisionlens.com expertchoice©. www. expertchoice.com export.gov (2017). canada, market overview-canada. retrieved from https://www.export.gov/article?series=a0pt0000000pattaaw&type=country_commer cial__kav freedom house. (2018). retrieved from https://freedomhouse.org/report/freedom-world2018-table-country-scores germany. retrieved from http://data.worldbank.org/country/germany globaledge. canada: statistics. retrieved from https://globaledge.msu.edu/countries/canada/statistics globaledge. (2016). market potential index (mpi) 2016. retrieved from https://globaledge.msu.edu/mpi globaledge. (2017). market potential index (mpi) 2017. retrieved from https://globaledge.msu.edu/mpi/data/2017 globaledge. united kingdom: statistics. retrieved from https://globaledge.msu.edu/countries/united-kingdom http://chinapower.csis.org/china-middle-class/ http://www.worldometers.info/world-population/population-by-country/ https://www.decisionlens.com/ https://www.export.gov/article?series=a0pt0000000pattaaw&type=country_commercial__kav https://www.export.gov/article?series=a0pt0000000pattaaw&type=country_commercial__kav https://freedomhouse.org/report/freedom-world-2018-table-country-scores https://freedomhouse.org/report/freedom-world-2018-table-country-scores http://data.worldbank.org/country/germany https://globaledge.msu.edu/countries/canada/statistics https://globaledge.msu.edu/mpi https://globaledge.msu.edu/mpi/data/2017 https://globaledge.msu.edu/countries/united-kingdom ijahp article: volinchak, whitehouse, yourstowsky, wooley, karpak/ analyzing corporate expansion to international markets: the case of germany, united kingdom, canada, mexico and china international journal of the analytic hierarchy process 116 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.574 international trade administration. exporting is good for your bottom line. retrieved from http://www.trade.gov/cs/factsheet.asp huston, m., roberts, j. and ferguson, j. (2014). comparing the emerging markets of brazil, india and china: who's the best?, international symposium of the analytic hierarchy process, washington, d. c. karpak, b. (2017). reflections: mathematical principles of decision making, international journal of the analytic hierarchy process, 9(3), 341-348. doi: https://doi.org/10.13033/ijahp.v9i3.521 nemer, l. understanding the schedule b. retrieved from www.shippingsolutions.com/blog/understanding-the-schedule-b. online business dictionary –businessdictionary.com, www.businessdictionary.com/definition saaty, t. l. (2010). mathematical principles of decisions making. pittsburgh: rws publications. sakarya s. et al. (2007). market selection for international expansion. assessing opportunities in emerging markets, international marketing review, 24(2), 208-238. doi: 10.1108/02651330710741820 tizio, p.n., lucas, d., schrecengost, l.r., and kuhns, m.a. (2014). selection of the most attractive emerging market for a business to enter among the countries turkey, russia, india, and china: an analytical hierarchy approach, international symposium of the analytic hierarchy process, washington, d. c. trading economics. germany economic indicators [data file]. retrieved from http://www.tradingeconomics.com/germany/indicators the world bank group. (2014). world development indicators: urbanization. retrieved from http://wdi.worldbank.org/table/3.12 the world bank group. (2016). urban population (% of total). retrieved from http://data.worldbank.org/indicator/sp.urb.totl.in.zs?locations=in the world bank. (2016). electricity consumption per capita. retrieved from http://databank.worldbank.org/data/reports.aspx?source=world-development-indicators wisertrade. (2017). world institute for strategic economic research. retrieved from www.wisertrade.org/home/portal/index.jsp. world bymap. (2012). list of countries electricity consumption [data file]. retrieved from world.bymap.org/electricityconsumption.html. http://www.trade.gov/cs/factsheet.asp http://www.shippingsolutions.com/blog/understanding-the-schedule-b http://www.businessdictionary.com/definition http://wdi.worldbank.org/table/3.12 http://data.worldbank.org/indicator/sp.urb.totl.in.zs?locations=in http://databank.worldbank.org/data/reports.aspx?source=world-development-indicators http://www.wisertrade.org/home/portal/index.jsp ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 178 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 applying anp to analyze the role of design in the furniture industry majid azizi professor, department of wood and paper science and technology faculty of natural resources college of agriculture and natural resources, university of tehran gholamreza mehdikhanloo m.s. graduate, department of wood and paper science and technology faculty of natural resources college of agriculture and natural resources, university of tehran abstract this study was carried out due to the severe dearth of research on the role of design in furniture production and the furniture market in iran. accordingly, this research proposes a decision making model to select the best solution for the role of design in this industry. the four possibilities that can be considered as potential solutions include using fashion design in furniture production (s1), using engineering design in furniture production (s2), using a combination of fashion and engineering designs in furniture production (s3), and applying the leading country’s design capability with an outsourcing approach in furniture production (s4). the analytic network process and super decision software were used to synthesize and analyze the model. it was found that all of the calculated decisions were influenced by the strategic criteria. a value-weighted competency model was calculated in the first stage with the influence of strategic criteria on the model. hierarchical design decisions were made for each of the competencies and their subsets (298 sub-criteria and 31 middle indices). paired comparison matrices associated with the degree of importance of each of the competencies were achieved in the second stage. in the final stage, subsets of the competency’s weights and their sub-options were identified with the combination of the competencies and the best solution was obtained. keywords: analytic network process; strategic criteria; benefits; opportunities; costs; risks; furniture; design 1. introduction lack of innovation and creativity in design and copying domestic and foreign works are among the most prominent problems in iran’s furniture industry. nowadays, competition among sellers and manufacturers is greater because of an increased number of wood industry manufacturing units and an increase in the amount of supply. in this environment, the one who takes advantage of a more distinctive design while having the same raw material and machineries as others is the winner. presenting diverse and new designs for different tastes, along with observing qualitative and ergonomic standards, ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 179 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 can play an effective role in developing iran’s furniture industry and gaining shares in the international furniture markets. gazo (2005) regards a furniture manufacturer’s capability to achieve design ability and the execution of this valuable tactic as the prerequisite for success in malaysia’s furniture industry. the design of new furniture is usually done with three things in mind which include structure design i.e. the physical shape and appearance of furniture, the raw materials used, and the manufacturing technology. new design of products and technological aspects are factors for development in the furniture industry in southeast asia according to ratnasingam and ioras (2003). merrell et al. (2018) presented an interactive furniture layout system that assists users by suggesting furniture arrangements based on interior design guidelines. their results prove that the suggestion generation functionality measurably increases the quality of furniture arrangements produced by participants with no earlier training in interior design. interior designers consider furniture layout according to color and material, lighting design, arrangement of art and accessories, carpeting, and detailed traffic patterns through the space. liu and li (2006) presented an appealing furniture design. they indicated that in order to adapt to the needs of diverse aesthetic values, the appearance of modern furniture should pay great attention to the use of space and should emphasize the concept of appeal design. like humanization design, appeal design embodies more human concerns and creates a personalized design on a deeper level. appeal design is also the important content that people pursue in the information age. they also explored the new trends of furniture design by researching the role of the concept of appeal in furniture design and discussed anglicizing appeal design's content and characteristics, and combining humanization design concepts with the information age. apostolou, sakkas, and mentzas (2004) studied big corporations that are active in the furniture industry internationally and came to the conclusion that most of them tend to use new technologies to develop a new measure. these companies transferred a part of their shares to their customers through the internet, and have also displayed information about their products on the internet for customers. this has resulted in a high level of customer satisfaction and a commitment to the company itself. the use of design is very widespread such as design for performance, requesting modification in terms of aesthetics, ease of production, sustainability, and reliability, quality, and business processes. identifying the best solutions for the use of the role of design in furniture production and its market improvement is the objective of this research. a study by swann & birke (2005) showed that creativity and design influence r&d, and creativity and design play an important role in the innovation and performance of a business. research conducted for this study also shows that companies with a higher intensity of design are more likely to produce an innovative product. design costs have a positive correlation with the company's productivity growth (bessant, whyte & neely, 2005), a study by gemser and leenders (2001) on dutch companies shows that design integration in the development of new products has a significant positive impact on the company’s performance (profit, turnover, sales, and exports). a model for using the anp in supplier selection is developed and implemented in an electronic company (gencer & gurpinar, 2007). anp models are also used for locating ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 180 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 facilities strategically, selection of the appropriate energy policy for turkey, and for product mix planning in a semiconductor fabricator (partovi, 2006; ulutas, 2005; chang, et al. 2005). anp was also used to determine the best alternative for a raw material mix in iranian facial tissue plants. the results of this study showed that using virgin pulp mixed with rejected paper makes the production more efficient. furthermore, the long fibers of the pulp prevent environmental harm, and benefits and costs are more sensitive than opportunities and risks in this study (azizi & modarres, 2010). these facts encouraged us to carry out this study as a steppingstone toward further environmental oil terminals site selection studies, and also encouraged us to focus on their criteria selection. the analytical network process (anp) is a flexible method that is able to demonstrate the mutual interactions between all factors in decision-making procedures. this method is an independent decision support model, belonging to the family of multi-criteria analyses, which has been developed by saaty (2005) and saaty and vargas (2006). ocampo et al. (2018) developed a fuzzy-bocr-ahp framework. they indicated that the identification of drivers is performed by individually taking into account the benefits, opportunities, costs and risks (bocr) associated with these drivers. due to the hierarchical structure of the proposed framework and the complexity of the decision, a fuzzy-bocr-ahp approach was adopted in order to address the uncertainty in judgment elicitation in ahp. the results show that market demand is the most significant driver while the most significant sustainable practice is lean practice. the prioritization of drivers based on the fuzzy-bocr-ahp decision making framework and their prioritization targeting the triple bottom line (tbl) perspectives suggest useful and interesting results. this study proposes the use of the analytic network process to develop a decision making tool for the best use of paraguay’s hydroelectric surpluses within the framework of a sustainable policy while considering quantitative and qualitative aspects that are difficult to identify through usual evaluation approaches. this tool has a highly scientific and avant-garde component that enables essential decisions to be made that would produce the greatest benefits for the integral development of the country (amarilla et al., 2018). the analytic network process is a tool used to evaluate reverse logistics features because it considers interdependence relations. it uses a model that applies anp and seeks to achieve a sustainable business performance by analyzing triple bottom line (tbl) using reverse logistics practices which are measured and evaluated (guimaraes & salomon, 2015). due to the interdependencies between given selection criteria, site selection for wind power plants is made by using the analytic network process. an application of site selection is made in turkey for five alternatives sites by using the proposed decision model to demonstrate that it is suitable for the wind energy plant site selection (aksakal et al., 2015). kao et al. (2015) developed anp with the following four criteria: control device, product design, product reliability and market environment. each criterion consists of multiple sub-criteria. the result shows that portable control device, user friendly interface, aesthetic product design, personal information protection and product compatibility affect consumer adoption of smart home applications the most. this ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 181 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 research contributes to the literature by understanding consumer adoption in the area of communication activities and by extending anp to marketing research. 2. methodology 2.1 the analytic network process (anp) since the most suitable technique for our study seems to be the analytic network process (anp), we will review it briefly in this section. the analytical network process (anp), a generalization of the analytic hierarchy process (ahp) method for multi-criteria decision making, provides a broader framework for decision making in complicated environments. the advantage of this theory over the ahp is its ability to extend its use to the cases of dependence and feedback and generalization of the super-matrix approach. it allows interactions and feedback within clusters (inner dependence) and between clusters (outer dependence). the anp is a coupling of two parts. the first part consists of a control hierarchy or network of criteria and sub-criteria that control the interactions in the system being studied. the second is a network of influences among the elements and clusters. the network varies from criterion to criterion and a super-matrix of limiting influence is computed for each control criterion. finally, each of these super-matrices is weighted by the priority of its control criterion and the results are synthesized through addition for all the control criteria. in order to determine the effective criteria and sub-criteria as well as solutions for the role of design, the following steps were performed. after reviewing the internal and external resources and interviewing the producers and academicians, a comprehensive list of effective criteria was developed to enable us to understand all the important criteria for decision making in relation to the role of design. to do this, the views of 40 wood and furniture industry experts were used. the aim of this study is to develop a decision network model to find the best solution for the role of design in the country's furniture production and market. in the network model, merits are divided into four sub-sections as benefits, opportunities, costs and risks (bocr). these merits are influenced by strategic criteria (political, legal, development and technological, cultural and social and economic factors) to obtain weighting values for each. this process follows the principle of the analytic network process (anp). there are 296 sub-criteria in 31 middle criteria and five main groups (marketing and economic, technical and workforce, supply and production, social and cultural, and environmental) (see tables 1-4). weighting values of criteria and sub-criteria are obtained by pairwise comparisons and feedbacks between criteria, sub-criteria and the four solutions. ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 182 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 table 1 control criteria and sub-criteria of benefits main criteria of benefits sub-criteria economic/ competitiveness access and mastery of knowledge economic/ competitiveness share capital and technology economic/ competitiveness the expansion of cooperation between businesses economic/ competitiveness the possibility of creating static competition economic / export cancel export tax economic / export stimulation of demand by foreign buyers economic / export removing foreign competitors who are active in the country's furniture market economic / investment access to technology economic / investment access to management techniques economic / investment lower prices and upgrade competitiveness enterprises economic / infrastructure taking advantage of scientific and technological infrastructure of universities economic / infrastructure taking advantage of the capacity of clusters of furniture marketing and sale / distribution control over brand identity marketing and sale / distribution decreasing waste in direct transportation marketing and sale / profit decreasing fixed price marketing and sale / profit increase in profit marketing and sale / profit increase in turnover marketing and sale / services after sale customer satisfaction marketing and sale / branding possibility of upgrade the brand marketing and sale / branding possibility of description of firm marketing and sale / propaganda according to tastes of the clients ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 183 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 marketing and sale / propaganda taking advantage of high quality catalogs marketing and sale / marketing attract, retain and gain customers marketing and sale / marketing choosing a profitable target markets marketing and sale / marketing design and implementation of internal marketing technical/ design creation of wealth and value added technical/ design optimization of wood consumption technical/ design savings resulting from transportation with design of assembly capability products technical/ design remove the method of trial and error in the production of a new product technical/ design correct advice in the field of furniture layout technical/ design increase in apparent charm combined with suitable colors technical/ design build products tailored to consumer space technical/ ergonomics observance of the standards technical/ ergonomics increase the performance of the furniture technical/ ergonomics calculate the values of the incoming loads technical/ ergonomics increase the strength of the structural furniture technical/ ergonomics subtracting the weight of the furniture (mass of used materials) technical/ ergonomics upgrade the health level technical/ technology maximum use of capacity of the machines human/ training consulting member of academics to the industry human / training possibility making prototypes product for the industry human / training supply of technical manpower for the industry human / training possibility of training internships students for the industry ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 184 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 human / training art and architecture students encouraged to fashion design in the field of furniture human / training engineering design students are encouraged to design engineering in the field of wood furniture human / management proper management of human resources human / management increase employee satisfaction human / management enhance enterprise communications outsourcing human / management correct management of expectations human / management reducing complaints human / work force activating market of academic disciplines human / workforce rest of the labor force (easier to understand how to cut and assemble structures) human / workforce no need to research and use of force skilled in the design (in the case of outsourcing the design section) human / innovation increasing creativity and innovation human / innovation possibility of correct modeled (the lack of a working copy) supply/ raw material use of indigenous raw materials production/ productivity improve the performance production/ productivity reducing time to build a product production/ productivity high speed assembly production/ productivity reducing duplication production / quality control increased accuracy and quality in work production / quality control possibility of product rating production / quality control becoming mechanized quality control production / quality control ease of control and inspection during production production / quality control connect the control unit with the purchase process, warehouse, production production / efficiency reducing waste production / efficiency create a good working environment production / efficiency increase the efficiency of the production production / efficiency enhance the ability of individual skills ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 185 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 production / packaging possibility of economic packaging design production / r&d move the boundaries of design knowledge production / r&d create welfare for human beings production / r&d increasing belief next to simplify and expedite matters production / r&d increasing competitive power production / r&d improve the quality of the product production/ production process increased accuracy in the work machine production/ production process ease of manufacture production/ production process reducing production cost production/ production process speed of the production of new products on the market (reverse engineering) production/ production process keep track of production strategy eto production/ production process keep track of production strategy mto production/ production process keep track of production strategy mts social, cultural & politic/ social ability to meet the needs of the customer social, cultural & politic/ cultural restoring the credibility of the iranian furniture environmental/ environmental reducing energy consumption in the use of new technology environmental/ environmental principle of operation of forests environmental/ environmental possibility of green supply chain management (gscm ) environmental/ environmental reducing use of chemicals and contaminants environmental/ environmental reducing the production of carbon (using less resources, recycle more and reduce waste production) ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 186 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 table 2 control criteria and sub-criteria of opportunities main criteria of opportunities sub-criteria economic/ export facilitation and export promotion economic/ export non-oil exports of technology exchange economic/ export move is in line with an economy measure economic/ export development of a mandatory standard for export economic / export offering bonuses to new designs economic / export contribute to the inbreeding and extraversion economy to export economic / investment reducing risk of investment economic / investment steer outbound investments with high added value economic / investment possibility of production based on cheap labor economic / investment improvement of performance of the domestic industry economic / investment the possibility of contracting marriage, ttc and khtc economic / infrastructure development of intellectual property law economic / infrastructure increase government incentives economic / infrastructure using capacity of the graduates economic / infrastructure awareness of the rules of international trade economic / competitiveness increasing share of foreign markets economic / competitiveness possibility of creation of dynamic competition economic / competitiveness stabilization of domestic markets economic / competitiveness creation a competitive atmosphere in the industry economic / competitiveness proving competitive advantage economic / competitiveness proving relative advantage economic / competitiveness possibility of adoption of competitive policies marketing and sale / profit high profit margins for designed products marketing and sale / warranty product warranty marketing and sale / after sales service customer relationship management marketing and sale / after sales service increasing after sales service ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 187 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 marketing and sale / propaganda contribution of the effective and targeted advertising of the product marketing and sale / propaganda taking advantage of the appropriate headline in advertising marketing and sale / propaganda possibility of receiving awards and certifications (development of advertising) marketing and sale / branding branding iranian furniture marketing and sale / branding contribute to the development of the brand marketing and sale / branding customer loyalty to the brand marketing and sale / branding possibility of receiving awards and certifications marketing and sale / branding development of cultural branding marketing and sale / marketing contribution of products marketing marketing and sale / marketing possibility of segmenting market marketing and sale / marketing contribution of product pricing strategy marketing and sale / marketing possibility of the creation, exchange and providing value to customers marketing and sale / marketing opportunity of design and compilation of performance measurement program technical/ design revision of beauty and elegance of the product technical/ design development of multi-purpose modular furniture technical/ design differentiation of the product with other competitors technical/ design possibility of customization of the product technical/ design possibility of using handmade eco-chic pieces technical/ design possibility of design in imitation of nature-biomimetic technical/ design management of amount of textiles and furniture accessories technical/ ergonomics applying more frequently technical/ ergonomics possibility of research on raw materials technical/ ergonomics possibility of research on fashion and furniture style technical/ ergonomics increasing safety of furniture technical/ ergonomics feasibility analysis of human body system technical/ ergonomics possibility of assessing the physical dimensions (anthropometry) technical/ ergonomics taking advantage of bio-mechanics in furniture design ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 188 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 technical/ technology conversion of knowledge and technology to new product technical/ technology opportunity to use the new machines technical/ technology possibility of the use of new technical knowledge technical/ technology possibility of export of technical and engineering services to other countries human/ training possibility of research contracts with universities human / training possibility of consultations and the implementation of industry research human / training access of industry to universities’ labs human / training possibility of establishment of technological firms by alumni human / training enhancement of role of universities to create new high-tech industries human / management taking advantage of outsourcing opportunities human / management development of applying physical capital human / management improvement of employee’s motivation, beliefs and thoughts human / management ease of management of human resources human / management promote strategic management practices human / work force increasing level of employment human / workforce increasing expertise and skills of the workforce human / innovation design of appropriate structure for supporting creativity human / innovation possibility of innovation in process, product and services human / innovation possibility of innovation in marketing human / innovation increasing culture of teamwork human / innovation promotion of innovation in firm strategy supply/ raw material use of new raw materials ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 189 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 supply/ raw material correct choice of materials for manufacturing supply/ lateral use the proper fabric and accessories supply/ lateral development of auxiliary industries production/ productivity complete using production technologies production/ productivity growth of productivity production / quality control feasibility analysis of various experiments production / quality control possibility of record test results production / quality control documentation and records management quality tests production / quality control taking advantage of previous experience in testing production / quality control possibility of support a variety of measuring methods production / quality control possibility of definition of quality indicators production / efficiency ease employees performance evaluation production / efficiency improvement of production per capita production / efficiency possibility of timing produce production / efficiency possibility of capacity assessment production / packaging contributing low volume packaging production / packaging ease of assembly by the consumer production / packaging ease of visualizing and understanding contents of the package production / packaging identity of building in the packaging design production / packaging help to maintain healthy goods production / packaging the possibility of using different packaging shapes production / r&d possibility of establishment of the r&d unit in companies production / r&d feasibility of aesthetic furniture research production / r&d optimization of processes production / r&d new product design production / r&d transfer and uptake of technology production / r&d defensive strategy formulation production/ production process development of mass production approach production/ production process on time delivery production/ production process flexibility of prices respect to customers production/ production process flexibility of the machines, production lines and workforce production/ production process track production strategy oem production/ production process track production strategy odm ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 190 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 production/ production process track production strategy obm social, cultural & politic/ social improving quality of life social, cultural & politic/ social upgrading level of industrial unions social, cultural & politic/ social growth of consumption of household furniture social, cultural & politic/ social increasing sense of national confidence social, cultural & politic/ cultural presenting culture of iranian by design social, cultural & politic/ cultural production respect to culture and need of the society social, cultural & politic/ cultural change and development of consumption pattern social, cultural & politic/ politic increasing political influence (with the design of a new product) social, cultural & politic/ politic increasing political relations social, cultural & politic/ politic taking advantage of the strategic advantage environmental/ environmental possibility of recycling furniture (biological degradation) environmental/ environmental lack of using plastic packaging environmental/ environmental effective using available energy and resources environmental/ environmental possibility of receiving environmental certificates for export to europe environmental/ environmental optimization of exploitation of raw materials environmental/ environmental possibility of return of the products to the company for recycling ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 191 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 table 3 control criteria and sub-criteria of costs main criteria of costs sub-criteria economic/ export transference of foreign markets to leading competitors economic/ export cost analysis of the export target markets economic/ export decreasing global furniture exports economic/ export negative competition between domestic exporters economic / export lack of awareness of the export target markets economic / import sharp rise of imports economic / import exit of currency to the abroad marketing and sale / marketing cost of detailed study of the market marketing and sale / marketing lack of awareness of needs of the market marketing and sale / marketing cost of marketing a new product marketing and sale / marketing cost of participation in the exhibitions marketing and sale / sale cost of creating continuous shops technical/ design lack of appropriate idea for new products technical/ design spending time and cost for new product design technical/ design cost of salaries, taxes and social security of design branch technical/ ergonomics cost of testing product in laboratories technical/ ergonomics cost of materials research technical/ ergonomics lack of national anthropometry technical/ ergonomics weakness of science in kinesiology joints technical/ ergonomics cost and time for the nordic questionnaire analysis technical/ ergonomics possibility of assessing the physical dimensions (anthropometry) technical/ ergonomics cost of test centers creation and testing furniture technical/ technology lack of specialized data collection and processing ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 192 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 human / management problem of attracting creative employees in furniture design human / management cost of management remuneration human / workforce cost of skilled and professional manpower supply human / workforce unemployment of graduates human / workforce cost of holding periodic training human / innovation weakness of creativity and innovation power human / innovation cost of the dignity befitting the bonus human / innovation cost of the failures and mistakes of the staff in the field of innovation supply/ raw material increasing cost of raw materials supply/ raw material sudden cessation of supply of raw materials supply/ raw material dependency to the suppliers of raw materials supply/ raw material drop in the quality of supply of raw materials supply/ lateral secondary industry downturn related to furniture production / r&d financial, machines and equipment problems production / r&d cost of aggressive strategy (in field of development of a new product and technological change) social, cultural & politic/ cultural lack of pay attention to iranian culture in the production environmental/ environmental taking indiscriminate sources of wood (the destruction of the forests) environmental/ environmental increasing greenhouse gas emissions environmental/ environmental accumulation of waste resulting lack of flexibility of recycling environmental/ environmental ban on exports to the european union due to disagree with the green supply chain ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 193 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 table 4 control criteria and sub-criteria of risks main criteria of risks sub-criteria economic/ competitiveness fierce competition among domestic manufacturers economic/ competitiveness monopoly of information in the market economic/ competitiveness anti-competitive practices such as import economic/ competitiveness anti -competitive cartel action economic/ competitiveness absence of codified rules and regulations regarding competition economic / export risk of increasing fixed price of the products economic / export lack of liquidity and working capital for export products economic / export lack of understanding tools and machines in producing furniture economic / investment powerful presence of foreign competitors economic / investment risk of net use of the investment market economic / investment pure using capital for resources instead of productivity economic / investment possibility of increasing competition level marketing and sale / warranty loss of trust to internal product marketing and sale / branding rising foreign brands of furniture technical/ design lack of familiarity with modern software design technical/ design risk of copying designed products by domestic and foreign competitors technical/ technology low level of technical knowledge technical/ technology dependency to leading countries in knowledge of design human / training lack of training and experience of graduates human / training lack of compatibility between industry needs and university training human / management lack of ability of protection of investment in sector of design and creativity human / management lack of flexibility of managers human / management lack of agility of firm ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 194 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 human / workforce lack of stability of skillful workforce supply/ raw material risk of security of raw materials supply supply/ raw material risk of reduction of materials quality supply/ raw material using non-native raw materials production / r&d lack of copyright law production / r&d lack of trust to the degree of usefulness production / r&d lack of logical and scientific programs social, cultural & politic/ cultural promoting consumption culture with foreign brand social, cultural & politic/ politic risk of economic sanctions social, cultural & politic/ politic risk of changing economic policies of government social, cultural & politic/ politic risk of foreign investment security 2.2 data/model analysis the overall structure of decision making is as follow (figure 1): figure 1 overall structure of decision making in the current research, the anp model is described by the above elements. the subnetworks are shown in figures a1-a9 in the appendix. ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 195 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 2.3 alternatives there are four potential alternatives for the role of design: 1) using fashion design in furniture production (s1). cooperation among academic members is the topic of this approach. furniture production considers interior architecture, arts, crafts, etc. 2) using engineering design in furniture production (s2). the steps of the engineering design process are to a) define the problem; b) do background research; c) specify requirements; d) brainstorm solutions; e) choose the best solution; f) do development work; g) build a prototype; and h) test and redesign. 3) using a combination of fashion and engineering design in furniture production (s3). innovative design and engineering design are topics of this approach. 4) applying the leading country’s design capability with an outsourcing approach in furniture production (s4). s4 means working in partnership with the leading countries in interior design as well as a reverse engineering procedure (working copy) production of furniture. the analytic network process and super decision software were used to synthesize and analyze the model. 2.4 strategic criteria in this research, the merits of benefits, costs, opportunities, and risks are weighted by five general factors that fit into one of the following broad categories: development and technological (0.193), cultural and social (0.105), economic (0.299), political (0.24), and legal (0.154). ratings of general factors are done by pairwise comparison of the lower level factors and summation of the main factors at the top level. the final weight of the strategic criteria shows that the economic criteria with the weight of 0.299 have the highest priority. the economic criteria have the highest impact on decision making for the role of design in the furniture industry. 2.5 prioritizing bocr to categorize the criteria which have been used in the current research, we divided them into favorable and unfavorable categories. the decision maker considers the favorable criteria as benefits and the unfavorable criteria as costs. the possible events are also divided into opportunities and risks depending on whether they are considered to be positive or negative (saaty, 2001a). since bocr are not equally important, it is necessary to prioritize them. five possible ratings ranging from “very high” to “very low” are used. the results of the influence of the strategic criteria on the merits of benefits, costs, opportunities, risks, and the priority of the above mentioned merits are reported in table 5. ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 196 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 table 5 rating of the model to obtain bocr weighing values very high (1), high (0.51), medium (0.252), low (0.124), very low (0.065) 2.6 control criteria network the network of control criteria under benefits, costs, opportunities and risks are as follows: economic and marketing, the workforce and technical, supply and production, social, cultural and political, and environmental. the structure of the framework for benefits, merits and its control criteria and sub-criteria are extracted from super decisions software, as illustrated in appendix figures a1-a9. the framework for costs, opportunities and risks can be developed similarly. to apply the anp, the super decisions software was used. in the bocr structure the following formula is used in calculations (saaty, 2001a): (benefits)*(opportunities)/ (costs)*(risks) (1) 3. results after the hierarchy is drawn up for criteria and sub-criteria which influence the selection of the appropriate alternative, the questionnaire was prepared to gather the opinions of the experts. the criteria and sub-criteria were evaluated based on pairwise comparisons. an example of these comparison questions is shown in figure a10. then, the priority rate of each criterion and sub-criteria were compared. first, the geometric mean was calculated for each one of the matrix cells by the following formula (saaty, 2000). bocr was ranked with the same method with respect to overall factors in the previous section (table 1). group judgments aij = (aij1 ×aij2 ×…×aijn) 1/n i, j = 1, 2,…n (2) where aij represents the comparison ratios between element ith row and element jth column in the pairwise comparison matrices and n is the number of decision makers. benefits costs opportunities risks economic (0.299) very high medium very high high political (0.246) high high very high high legal (0.154) very high high high high cultural &social (0.105) high low high medium development and technological (0.193) high medium high high overall normalized priorities 0.313 0.146 0.332 0.207 ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 197 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 after the geometric means of all matrix cells are calculated, the results are normalized and the criterion and sub-criterion weighting values are obtained through the integration of the weight of the low-level elements into the weight of the related upper-level elements. the results of geometric mean matrices and weighted supermatrix for benefits, costs, opportunities and risks criteria and sub-criteria are extracted from super decisions software and the standard formula is the multiplicative formula. 3.1 result of marketing and economic control criteria of benefits figure 2 result of control criteria under benefits figure 3 result of subnetwork under economic of benefits 0.581 0.418 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 economic marketing and sale weighing value 0.341 0.235 0.234 0.188 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 infrastructure competitiveness investment export weighting value ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 198 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 3.2 result of marketing and economics control criteria of opportunities figure 4 result of control criteria under opportunities figure 5 result of subnetwork under marketing and sale of opportunities 0.395 0.604 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 economic marketing and sale weighing value 0.183 0.35 0.231 0.077 0.103 0.053 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 marketing branding propaganda services after sale profitability warranty ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 199 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 3.3 result of marketing and economics control criteria of costs figure 6 result of control criteria under costs figure 7 result of subnetwork under economic of costs 0.604 0.395 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 economic marketing and sale weighing value 0.75 0.25 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 export import weighting value ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 200 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 3.4 result of marketing and economics control criteria of risks figure 8 result of control criteria under risks figure 9 result of subnetwork under economic of risks 0.671 0.328 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 economic marketing and sale weighing value 0.555 0.275 0.168 0 0.1 0.2 0.3 0.4 0.5 0.6 export investment competitiveness weighting value a x is t it le ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 201 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 3.5 results of solutions with respect to infrastructure under subnetwork of economics control criteria of benefits figure 10 results of solutions with respect to infrastructure under subnetwork of economics control criteria of benefits 3.6 results of solutions with respect to branding under subnetwork of marketing and sale control criteria of opportunities figure 11 results of solutions with respect to branding under subnetwork of marketing and sale control criteria of opportunities 0.199 0.288 0.418 0.094 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 s1 s2 s3 s4 weighting value s o lu ti o n 0.205 0.285 0.448 0.06 0 0.1 0.2 0.3 0.4 0.5 s1 s2 s3 s4 weighting value s o lu ti o n ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 202 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 3.7 results of solutions with respect to export under subnetwork of economic control criteria of costs figure 12 results of solutions with respect to export under subnetwork of economic control criteria of costs 3.8 results of solutions with respect to export under subnetwork of economic control criteria of risks figure 13 results of solutions with respect to export under subnetwork of economic control criteria of risks 0.289 0.138 0.092 0.479 0 0.1 0.2 0.3 0.4 0.5 0.6 s1 s2 s3 s4 weighting value s o lu ti o n 0.124 0.285 0.428 0.161 0 0.1 0.2 0.3 0.4 0.5 s1 s2 s3 s4 weighting value s o lu ti o n ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 203 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 by integrating the weights of the merits of benefits, costs, opportunities and risks and the weights of choices against the above mentioned merits, the final scores are reported in table 6. table 6 final outcome for priorities of the alternatives as table 6 shows, using a combination of fashion and engineering designs in furniture production (s3), has the highest priority and is the most suitable solution for the role of design in furniture production and marketing. the second, third and fourth highest priorities respectively are using fashion design in furniture production (s1), using engineering design in furniture production (s2), and applying the leading country’s design capability with an outsourcing approach in furniture production (s4). 4. discussion the results of the control criteria, marketing and economic, have the highest priority with respect to benefits (0.328), costs (0.409), opportunities (0.365) and risks (0.406) for the role of design in the furniture industry. this result is due to the motivation of economic activity, the success in decision making and the profitability of economic activity according to the view point of furniture experts. the results show the economic criteria under benefits (0.581) have a higher priority in comparison to marketing and sale (0.418) (figure 2). also in the subnetwork of economics under benefits, infrastructure (0.341) has the highest priority. the weighted values of the other criteria are as follows: competitiveness 0.235, investment 0.234, and export 0.188 (figure 3). it is not possible for the furniture industry to succeed in the fields of export, investment and competitiveness without creating a science and technology infrastructure, a government support infrastructure and a furniture clusters capacity. the results show marketing and sale under opportunities (0.604) have a higher priority in comparison to economics (0.395) (figure 4). in the subnetwork of marketing and sale, branding (0.35) has the highest priority. marketing (0.183), propaganda (0.231), services benefits (0.313) opportunities (0.332) costs (0.146) risks (0.207) final outcome additive ranking s1 0.218 0.177 0.217 0.188 0.225 2 s2 0.228 0.211 0.233 0.227 0.217 3 s3 0.301 0.362 0.222 0.296 0.396 1 s4 0.252 0.249 0.327 0.287 0.161 4 alter. mer. ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 204 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 after sale (0.077), profitability (0.103) and warranty 0.053 respectively are other leading factors (figure 5). by focusing on branding, the product can be distinguished from other products that rivals offer, and with the help of advertising, providing after-sales service and a product warranty, the customer's perspective can lead to creating marketing goals and profitability for the company. the results indicate economics under costs (0.604) has a higher priority in comparison with marketing and sale (0.395) (figure 6). in the economics sub-network, export (0.75) has a higher priority in comparison with import (0.25) (figure 7). lack of knowledge about the conditions of the export target markets reduces the motivation of domestic producers to export furniture. the result will be the loss of beneficial export markets for the benefit of foreign competitors, the tendency towards weak domestic markets and negative and unhealthy competition. economic under risks (0.671) has a higher priority in comparison with marketing and sale (0.328) (figure 8). in the economics subnetwork, export (0.555) has the highest priority (figure 9). investments (0.275) and competitiveness (0.168) follow respectively in priority. rising prices of the product is a risk and the rules and regulations of competition are not clear, also the risk of a monopoly of information on the iranian furniture market makes it impossible to export investment. results indicate that using a combination of fashion and engineering designs in furniture production (s3) (0.418) has the highest priority with respect to infrastructure under benefits (figure 10). using engineering design in furniture production (s2) (0.288), using fashion design in furniture production (s1) (0.199) and applying the leading country’s design capability with an outsourcing approach in furniture production (s4) (0.0937) respectively, are other leading solutions. utilizing the combination of fashion and engineering designs (s3) in furniture production can lead to the greatest use of the capacity of the scientific and technological infrastructure of the universities. with respect to the results, using a combination of fashion and engineering designs in furniture production (s3) (0.448) has the highest priority with respect to branding under opportunities (figure 11). using engineering design in furniture production (s2) (0.285), using fashion design in furniture production (s1) (0.205) and applying the leading country’s design capability with an outsourcing approach in furniture production (s4) (0.06) respectively, are other leading solutions. using a combination of fashion and engineering designs in furniture production (s3) can restore the potential of branding iranian furniture at the international level. with respect to export under costs, applying the leading country’s design capability with an outsourcing approach in furniture production (s4) (0.479) has the highest priority and the second, third and fourth are using fashion design in furniture production (s1) (0.289), using engineering design in furniture production (s2) (0.138) and using a combination of fashion and engineering designs in furniture production (s3) (0.092) (figure 12). the solution s4 will lead to the loss of export markets for the benefit of leading foreign competitors. this is because in the field of furniture design dependence on leading countries is a major weakness and the market will tend towards the leading countries. results show using a combination of fashion and engineering designs in furniture production (s3) (0.428) has the highest priority with respect to export under risks (figure ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 205 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 13). using engineering design in furniture production (s2) (0.285), applying the leading country’s design capability with an outsourcing approach in furniture production (s4) (0.161) and using fashion design in furniture production (s1) (0.124) respectively are other leading solutions. with respect to the high risk of the export criteria, s3 will lead to increased costs of skilled designer employment, the risk of job security for skilled labor, and the risk of not using machinery related to the design of a product engineered and compatible with customer's requirements. the final result of the strategic criteria shows that economic factors with a weight of 0.299 are the most important ones. for managers and employers of the country's furniture industry, these factors are more effective than other factors in influencing people's decision making role. also, bocr indicates that opportunities have higher significance than other competencies regarding the role of design in the production and furniture market. in terms of selection, the third solution (s3) or the use of the combination of fashion design and engineering design in the marketplace and furniture manufacturing is considered the best solution. accordingly, we can list the following with respect to the control criteria: economic index: if this solution (s3) is planned and implemented by companies and furniture manufacturing workshops in the country, the maximum available capacity in the scientific and technological infrastructure of universities can be provided. therefore, it would be possible to be considered a serious competitor among the foreign competitors and be able to protect the share of the country's furniture market in favor of domestic power. this solution creates a new competitive advantage which utilizes the strengths of the country in order to promote self-reliance in furniture design and production. by adopting this solution in the field of foreign investment, the country's furniture industry tends towards high value-added products. on the other hand, investment contracts will transfer the technical knowledge and method of doing business and will use the capabilities of graduates who have learned the design knowledge. marketing and sale index: by using the solution (s3) in the market and furniture industry of the country, it is possible to restore the potential of the country to brand iranian furniture. this also provides the opportunity to create the basis for the emergence and prosperity of iranian brands in the international market. most of the major exporting countries have been able to use fashion and engineering design simultaneously and create strong and reputable brands in their global markets which in turn creates gains in market share. through the design of new and diverse products in the market which are not only beautiful and colorful but vary in price and have the right structural and ergonomic power, customer’s attraction will be enhanced and market development increased. supply index: (s3) can be used to design and manage the use of indigenous and even non-indigenous materials. furniture products such as fabric, foam, paint, metal, glass and fittings can be used. products must work reliably and be readily available. if there were problems, suitable alternatives could be found by trial and error. production index: the use of solution (s3) could lead to the creation or development of a r&d unit in companies that would be beneficial. a precise understanding of an execution plan can reduce the trial and error of production to a minimum as well as ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 206 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 replication due to human error. these errors are the greatest contributors to reduced efficiency in production, and by reducing their affect the accuracy and quality of work increases, but the production time decreases. identifying the production time of the product can provide a fair judgment of labor productivity. also, with the planning of production capacity and per capita production for each individual, the productivity of the workforce can be enhanced by some incentives. reducing duplication and increasing production efficiency will lead to savings in raw material consumption, and on the other hand increasing labor productivity and reducing human error can reduce production time and increase production. furthermore, the positive outcome of these events in production leads to lower production costs and increased margins which, in addition to raising the competitiveness of the product, leads to the timely scheduling and delivery of production orders. on the other hand, the proper design of the product, allows assessment of the packaging of the product in various forms, so that in the case of export, the shipping costs could be reduced significantly. technical index: by using the third solution (s3) a design change can always be made to a product that is not in line with competitor’s products, and is distinct and different from other products. if the use of new machines is not mandatory, a market share can only be achieved by differentiating the design. the development of modular furniture is an example of this distinction in design. in addition, it is possible to issue technical and engineering services in the field of product design and set up of production units in other countries and provide an appropriate valuation for the country. it also provides for the development of design and ergonomics and can design and manufacture products in accordance with national and international anthropometric standards that, in addition to the beauty and elegance in terms of structural strength and bearing loads, can match the standards of the day. workforce index: the third solution (s3) offers the possibility of job creation for college graduates in the fields of architecture, art, industrial design, and wood industry. they can operate in the field of designing and producing new products by establishing new knowledge-based companies. universities can also produce industry-specific portfolio designs for the furniture industry, tailor-made with up-to-date designs. social, cultural, and political index: by using the third solution (s3) a sense of selfconfidence is created in the domestic producer. the industry can be restored and offered to world markets by designing in accordance with the culture of the community, in the form of national furniture and creating unique charm, while respecting the customer's sensitivity. if we can continuously apply beauty, quality, precision, elasticity and elegance in the field of furniture production, over time, a national cultural identity will be formed internationally, indicating quality and trust in the furniture of the country. environmental indicator: by using the combination of fashion and engineering design (s3) we can use the optimal amount of raw materials available to reduce the harvest and utilization of forest resources. with principled design, it is possible to recycle most of the consumables in the furniture and achieve success in reducing the production of waste from the use of furniture. even in the design of furniture, green raw materials can be used that can be degraded in the event of their release into nature. green supply chain management (green supply, green design, green production, green transportation, green ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 207 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 packaging) can be planned and implemented in the furniture industry. in this case, it will be possible to obtain environmental certifications for the furniture manufacturers. 5. sensitivity analysis since there may be different judgments about the comparison of priority rates of benefits, opportunities, costs, and risks or their sub-criteria, saaty (2001b) suggests a sensitivity analysis of the results. to perform a sensitivity analysis, we apply the software developed by saaty (2001b). the results are illustrated in table 7. from table 6, the solution using a combination of fashion and engineering designs in furniture production (s3) has the highest priority and is the most suitable solution for using a design role in furniture production and marketing. the second, third and fourth priorities are using fashion design in furniture production (s1), using engineering design in furniture production (s2), and applying the leading country’s design capability with an outsourcing approach in furniture production (s4). after changing the weights of one criterion, the priorities also change as shown in table 7. table 7 the results of sensitivity analysis s3>s2>s4>s1 new priorities new weight number of changes basic weight merits s3>s2>s1>s4 0.134 1 0.313 benefits s3>s4> s2 > s1 0.255 1 0.146 costs s3>s2>s1>s4 s3>s1>s2>s4 0.06 0.42 2 0.332 opportunities s3> s4 >s2> s1 0.294 1 0.207 risks opportunities is more sensitive than benefits, costs and risks because the sensitivity analysis of opportunities shows (table 7) that there are two times the changes in priority of alternatives while the sensitivity analysis of the other merits shows that there is only one change in the priority. figures a11 and a12 in the appendix show a sensitivity analysis of benefits, costs, opportunities and risks which has been done in super decisions software. ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 208 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 6. conclusion to present the best approach on the role of design in the country's furniture production and market, the four following options were analyzed through a combination of multicriteria decision techniques: using fashion design in furniture production (s1); using engineering design in furniture production (s2); using a combination of fashion and engineering designs in furniture production (s3); applying the leading country’s design capability with an outsourcing approach in furniture production (s4) in the first stage, the overall strategic factors were identified, and based on those factors, the weighting values of bocr (benefits, opportunities, costs, and risks) were determined. weighting values of the merits were derived by the influence of the strategic criteria on benefits, costs, opportunities and risks in the second stage. in the third stage, the weighted values of the alternatives derived in the prior stages were synthesized using the analytic network process (anp) and super decisions software. our conclusion is that the analytical network process shows using a combination of fashion and engineering designs in furniture production (s3) is a proper solution to apply to the role of design in the furniture industry. using a combination of fashion and engineering designs has the highest priority in parts of benefits, opportunities, and risks. in the research, there are eight main control criteria. with respect to the economic index, s3 would create the greatest use of available capacities in the scientific and technological infrastructure of universities. according to the marketing and sale index, s3 restores the potential of the country to brand iranian furniture. regarding the supply index, s3 can be used to design and manage the use of indigenous and non-indigenous materials in design. with respect to the production index, creation of an r&d unit in companies that can provide significant benefits for companies is possible by applying s3. according to the technical index, by applying s3 a design change can lead to a product that is not in line with a competitor’s products and distinguished from other products. in terms of the manpower index, s3 offers the possibility of job creation for college graduates. from the social, cultural, and political point of view, using s3 provides a sense of self-confidence in the domestic producer, and makes it possible to respond to customer needs. from the environmental point of view, applying s3 leads to using the optimal amount of raw materials available to reduce the harvest and utilization of forest resources. results of the sensitivity analysis revealed that the opportunities are more sensitive than benefits, costs and risks. the result can be used since there may be different judgments about the comparison of priority rates. ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 209 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 references aksakal, e., aktas a., kabak, m. & dagdeviren, m. (2015). site selection for wind energy plant by using analytic network process. 23 rd international conference on multi criteria decision making (mcdm 2015). hamburg, germany. amarilla, r., osorio, a., blanco, g., liamosas, c., fernandez, f. (2018). integrated multi criteria planning model of the use of hydroelectricity surplus of paraguay based on analytic network process. international symposium on the analytic hierarchy process (isahp2018). hong kong. apostolou, d., sakkas, n., & mentzas, g. (2004). enabling knowledge sharing in supply chain management: a case study in the wood/furniture sector. retrieved from imu.iccs.ntua.gr azizi, m., and modarres m. (2010). a decision model to select facial tissue raw material a case from iran, or insight, 23(4), 207-232. doi: https://doi.org/10.1057/ori.2010.12 bessant, j., whyte, j. and neely, a. (2005). dti think piece: management of creativity and design within the firm, advanced institute for management (aim) and imperial college. chang, s.h., a.h.i.lee, and pearn, w.l. (2005). analytic network process approach for product mix planning in semiconductor fabricator. international journal of production economics. 96(1), 15-36. doi: https://doi.org/10.1016/j.ijpe.2004.02.006 gazo, r. (2005). a review of competitive strategies of furniture manufacturers. forest product journal, 55(10), 66-73. gemser, g. and leenders, m.a.a.m. (2001). how integrating industrial design in the product development process impacts on company performance, the journal of product innovation management, 18, 28-38. doi: https://doi.org/10.1016/s07376782(00)00069-2 gencer, c., and gurpinar d. (2007). analytic network process in supplier selection: a case study in an electronic firm. applied mathematical modeling, 31, 2475-2486. doi: https://doi.org/10.1016/j.apm.2006.10.002 guimaraes, j., salomon, v. (2015). a compatibility study of priorities vectors for reverse logistics indicators: a footwear industry case developed in brazil. 23 rd international conference on multi criteria decision making (mcdm 2015). hamburg, germany. kao, l., chiu, c., lu, c. & kao, h. (2015). applying analytic network process to explore consumer adoption of radical innovation: a case study of smart home technologies. 23 rd international conference on multi criteria decision making (mcdm 2015). hamburg, germany. ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 210 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 liu, y. & li, j. (2006). the furniture design and research based on the concept of appeal. 7th international conference on computer-aided industrial design and conceptual design. hangzhou, china. merrell, p., schkufza, e., li, z., agrawala, m. & koltun, v. (2018). interactive furniture layout using interior design guidelines. berkley: stanford university & university of retrieved from vis.berkeley.edu/papers/furniturelayout ocampo, l., vallecera, j., nunez, d. & galagar, k. (2018). identifying drivers of sustainable manufacturing at firm level using a fuzzybocr0ahp framework. international symposium on the analytic hierarchy process (isahp2018). hong kong. partovi, y.f. (2006). an analytic model for locating facilities strategically, omega, 34(1), 41-55. doi: https://doi.org/10.1016/j.omega.2004.07.018 ratnasingam j., ioras f. (2003). the sustainability of the asian wooden furniture industry, european journal of wood and wood products, 61(3), 233-237. doi: https://doi.org/10.1007/s00107-003-0382-9 saaty, t. (2000). fundamentals of decision making with analytic hierarchy process. pittsburgh, pa: rws publications. saaty t. (2001a). decision making with dependence and feedback: the analytic network process. pittsburgh, pa: rws publications. saaty t. (2001b). decision on national missile defense program, 6 th international symposium on the ahp. bern, switzerland. saaty, t. (2005). theory and applications of the analytic network process. pittsburgh, pa: rws publications. saaty, t.l., vargas, l.g. (2006). decision making with the analytic network process. new york, new york: springer science. swann, p. and birke, d. (2005). dti think piece: how do creativity and design enhance business performance? a framework for interpreting the evidence?, university of nottingham business school. ulutas, b.h. (2005). determination of appropriate energy policy for turkey. energy 30(2005), 1146-1161. doi: https://doi.org/10.1016/j.energy.2004.08.009 https://ieeexplore.ieee.org/xpl/mostrecentissue.jsp?punumber=4127003 https://ieeexplore.ieee.org/xpl/mostrecentissue.jsp?punumber=4127003 https://www.springerprofessional.de/en/european-journal-of-wood-and-wood-products/4941462 https://www.springerprofessional.de/en/european-journal-of-wood-and-wood-products-3-2003/5056872 ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 211 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 appendix figure a1 control criteria network under benefits figure a2 subnetwork under benefits/ economics ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 212 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 figure a3 subnetwork under benefits/marketing figure a4 subnetwork under benefits/technical ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 213 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 figure a5 subnetwork under benefits/workforce figure a6 subnetwork under benefits/supply ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 214 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 figure a7 subnetwork under benefits/production figure a8 subnetwork under benefits/social cultural & politic ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 215 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 figure a9 subnetwork under benefits/environmental ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 216 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 figure a10 sample of questionnaire ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 217 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 figure a11 sensitivity analysis with respect to benefits and costs ijahp article: azizi, mehdikhanloo/applying anp to analyze the role of design in the furniture industry international journal of the analytic hierarchy process 218 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.588 figure a12 sensitivity analysis with respect to opportunities and risks ijahp article: kaw, manaf/integration of delphi technique and analytical hierarchy process in selecting the type of dam: a case study of bungoh catchment in sarawak, malaysia international journal of the analytic hierarchy process 20 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.493 integration of delphi technique and analytical hierarchy process in selecting the type of dam: a case study of bungoh catchment in sarawak, malaysia karmun kaw department of environmental sciences faculty of environmental studies universiti putra malaysia, 43400 upm serdang, selangor, malaysia karmunkaw18@gmail.com latifah abd manaf department of environmental sciences faculty of environmental studies universiti putra malaysia, 43400 upm serdang, selangor, malaysia latifahmanaf@upm.edu.my abstract dams are built ancient structures, which serve to retain, collect, and store water. over the years, the function of dams has been diversified to flood prevention, water level regulation, and even recreational purposes. however, selecting a suitable type of dam based on the characteristics of a potential dam site is consistently a concern in the preliminary construction stage. therefore, this study attempts to integrate the delphi technique and analytical hierarchy process (d-ahp) to develop a set of influential attributes, which assists in the selection of a suitable type of dam for the potential dam site. these influential attributes are determined based on comprehensive literature reviews and corroborated by twelve experts from relevant fields through three rounds of interviews. using the delphi technique, 9 important criteria and 25 sub-criteria are finalized. expert’s judgments are measured through pairwise comparisons to derive eigenvectors. based on prioritisation of ahp, the gravity dam scores the highest total weight, and is selected as the optimal type of dam for bungoh catchment. the selection of the gravity dam is quantified based on the developed influential attributes, which include environmental criteria, social criteria, and engineering criteria. essentially, the selection takes the characteristics of the potential dam site into account during the pairwise comparison process. in this context, the developed set of influential attributes could objectively assist related organizations in their decision making process. these attributes are also applicable in the preliminary stage of any dam development project. 1 keywords: analytical hierarchy process (ahp); delphi technique; multi-criteria decision making (mcdm); type of dam this work is supported by exploratory research grant scheme (ergs/1/11/stwn/upm/02/8-5527027) ijahp article: kaw, manaf/integration of delphi technique and analytical hierarchy process in selecting the type of dam: a case study of bungoh catchment in sarawak, malaysia international journal of the analytic hierarchy process 21 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.493 1. introduction dams are structures that impound water or underground streams. they serve the purpose of retaining, collecting and storing water, which is then evenly diverted between locations (beck et al., 2013). dams have been constructed for centuries globally to provide water for agricultural and municipal uses (azami et al., 2012). however, the suitability of a constructed dam type in a potential dam site is consistently questioned. moreover, the decision making process over the suitability of a dam type typically involves a trade-off from among many intangibles (saaty, 2008). therefore, this study attempts to integrate both the delphi technique and analytical hierarchy process (d-ahp) to develop a set of influential attributes in selecting a suitable type of dam for the potential dam site. based on expert’s judgments over a scale of absolute judgments, ahp assists the measurement of each attribute through pairwise comparisons to derive priority scales (saaty, 2008). the priority scales are synthesized by multiplying them with the priority of their parent nodes and adding all such nodes (saaty, 2008). in recent years, the integration of the delphi technique and ahp method (d-ahp) has been employed to further refine the attributes used in the decision making process (arof, 2015). in fact, the integration of d-ahp itself is academically recognized (arof, 2015). in developing a decision making model, the delphi technique is typically utilized in the preliminary research stage to identify prominent variables for selection, which is subsequently followed by the ahp method to determine the weight of selected variables (chung & her, 2013; da cruz et al., 2013; lee et al., 2014; moradi et al., 2014; sayareh & alimini, 2014). the delphi technique is a widely used method to achieve convergence of opinion in relation to the real-world knowledge, solicited from experts within specific areas (hsu & sandford, 2007). during the 1950s, the delphi technique was developed by the rand corporation in the united states to support military strategic-oriented surveys (chaves et al., 2012). it was later applied externally by dalkey & helmer (1963), where the delphi technique was specifically defined by means of obtaining the most reliable collective opinion from a group of specialists, subjected to a combination of questionnaires and/or interviews and controlled feedback in a series of cycles (chaves et al., 2012). theoretically, the delphi technique could be continuously iterated until a consensus is achieved (hsu & sandford, 2007). nevertheless, three iterations are often sufficient to collect the required information before achieving a consensus in most cases (cyphert & gant, 1971; brooks, 1979; ludwig, 1997; custer et al., 1999). meanwhile, ahp is a multi-criteria decision making (mcdm) method, which principally consists of three levels, namely the main goal, criteria, and alternatives (adamcsek, 2008; gawlik, 2008; yau, 2009). ahp is a decision support tool to solve complex problems, in which the main goal level could be extended into forces and actors influencing the decision making process; the criteria level could be divided into both criteria and sub-criteria; and the alternatives could be treated as action scenarios and prognostics of their effects (gawlik, 2008). the number of criteria to be taken into account could be rather extensive in a strategic decision making process particularly in a disorderly environment of actual economic reality. theoretically, more determinants at the preparatory stage are associated with improved decision making. however, multiplying the number of factors obfuscates the main goal and potentially affects the final decision (saaty, 1990; gawlik, 2008). thus, a certain number of criteria are necessary to mitigate the problem. according to saaty (2003), the number of analyzed criteria and alternatives in a decision making process should ijahp article: kaw, manaf/integration of delphi technique and analytical hierarchy process in selecting the type of dam: a case study of bungoh catchment in sarawak, malaysia international journal of the analytic hierarchy process 22 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.493 not exceed 7 (+/2), which gives a maximum of 9 criteria/alternatives to ensure appropriate precision in the obtained results. 2. methodology the ahp method involves six essential steps, which are in the following order: (1) define the main goal or objective, (2) structure the ahp hierarchy, (3) conduct pairwise comparisons, (4) calculate the eigenvectors, (5) evaluate the consistency, and (6) obtain the total weights and overall ranking of the alternatives (safari et al., 2010; yasser et al., 2013). for this study, the main goal is to select a suitable type of dam for a potential dam site. subsequently, the delphi technique is integrated into the ahp to obtain the necessary data from participants within their domain of expertise and gather their opinion specifically on the influential criteria in selecting the optimal type of dam alternative (hsu & sandford, 2007). the data collection stage proceeds as follows: first round: a structured questionnaire is developed based on extensive reviews of related literature, which serves as the cornerstone of soliciting important criteria in determining the optimal type of dam (custer et al., 1999). according to hsu & sandford (2007), utilizing a structured questionnaire in the preliminary round is acceptable and considered a common modification in the delphi process format. after obtaining responses from all twelve experts, the gathered information is then converted into a revised questionnaire, which is utilized in the second round of data collection. second round: the revised questionnaire is distributed to each selected expert and these experts are required to rank the criteria as well as the sub-criteria to establish preliminary priorities, in which areas of agreement and disagreement are identified (ludwig, 1994). in some cases, delphi panellists are required to state the rationale concerning their ranking priorities among the provided items (jacobs, 1996). in this round, initial consensus is obtained and the actual outcomes are presented based on the expert’s responses. third round: the revised questionnaire is redistributed among experts with a summary from the previous round. in this round, experts are required to refine their judgments regarding the relative importance of these items. consequently, a finalized questionnaire containing influential attributes (criteria and sub-criteria) is developed to structure the ahp hierarchy in selecting the optimal type of dam. nevertheless, a slight increase in the degree of consensus between the final two rounds is typically expected (weaver, 1971; dalkey & rourke, 1972; anglin, 1991; jacobs, 1996). with the developed hierarchy structure, pairwise comparisons are conducted at each level, namely goal, criteria, sub-criteria, and alternatives. experts are required to determine the relative preference for the elements in each level using an underlying semantic 9-point scale, representing different intensities of importance, which was developed by thomas saaty (table 1). ijahp article: kaw, manaf/integration of delphi technique and analytical hierarchy process in selecting the type of dam: a case study of bungoh catchment in sarawak, malaysia international journal of the analytic hierarchy process 23 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.493 table 1 saaty's scale of preferences in the pairwise comparison process (saaty, 1990) intensity of importance definition explanation 1 equal importance two activities contribute equally to the objective 3 moderate importance experience and judgment slightly favour one activity over another 5 strong importance experience and judgment strongly favour one activity over another 7 very strong or demonstrated importance an activity is favoured very strongly over another; its dominance is demonstrated in practice 9 extreme importance the evidence favouring one activity over another is of the highest possible order of affirmation. 2,4,6,8 intermediate values between adjacent scale values when compromise is needed all the attributes in the hierarchy are compared in the pairwise comparison matrix as shown in equation 1, where a is the pairwise comparison matrix, w1 represents the weight of element 1, w2 represents the weight of element 2, and wn represents the weight of element n (saaty, 1990; yasser et al., 2013): 𝐴 = [ 𝑤1 𝑤1 𝑤2 𝑤1 ⋮ 𝑤𝑛 𝑤1 𝑤1 𝑤2 𝑤2 𝑤2 ⋯ ⋯ 𝑤1 𝑤𝑛 𝑤2 𝑤𝑛 ⋮ ⋱ ⋮ 𝑤𝑛 𝑤2 ⋯ 𝑤𝑛 𝑤𝑛] (1) as pairwise comparisons are conducted by experts, eigenvectors or relative weights of the attributes are calculated using equation 2. following that, the eigenvalues of matrix a are obtained using simultaneous solutions of equation 3 and equation 5, where λmax is the principal eigenvalue of matrix a, w (equation 4) is the eigenvectors, and i is the unit matrix (yasser et al., 2013): 𝐴 𝑥 𝑊 = 𝜆 𝑥 𝑊 (2) (𝐴 − 𝜆𝑚𝑎𝑥𝐼) 𝑥 𝑊 = 0 (3) 𝑊 = (𝑤1, 𝑤2, 𝑤3, … , 𝑤𝑛) (4) ∑ 𝑤𝑖 = 1 𝑛 𝑖=1 (5) once the eigenvectors are obtained, they are subjected to consistency checking. this is an important step to evaluate the degree of reasonability of expert’s judgments (saaty, 2008). therefore, a consistency index (ci) (equation 6) is used, where n is the dimension of the pairwise comparison matrix (yasser et al., 2013). consistency index = 𝜆𝑚𝑎𝑥−𝑛 𝑛−1 (6) ijahp article: kaw, manaf/integration of delphi technique and analytical hierarchy process in selecting the type of dam: a case study of bungoh catchment in sarawak, malaysia international journal of the analytic hierarchy process 24 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.493 the value of the ci is calculated with random numbers using a random consistency index (rci). table 2 shows the values of rci for (1–10) dimensional matrices. for each matrix, a consistency ratio (cr) (equation 7) is calculated by dividing ci with rci. cr is an appropriate index for consistency judgment (yasser et al., 2013). when cr exceeds 0.1, the judgments are considered untrustworthy because they are too close for comfort to randomness and the process is valueless or must be repeated (teknomo, 2006). table 2 random consistency index (rci) (saaty, 1980) n 1 2 3 4 5 6 7 8 9 10 r.c.i 0 0 0.58 0.9 1.12 1.24 1.32 1.41 1.45 1.49 consistency ratio = consistency index random consistency index (7) the final stage of ahp is to obtain the total weights and overall ranking of the alternatives for the type of dam. the eigenvector (relative weight) of each criteria and sub-criteria (attribute) as well as the eigenvectors of alternatives are combined based on equation 8, where wi is the total weight of alternative i, 𝑤𝑖𝑗 𝑎 is the eigenvector of alternative i with respect to attribute j, 𝑤𝑖𝑗 𝑐 is the eigenvector of criterion j, m is the number of criteria, and n is the number of alternatives (yasser et al., 2013): 𝑊𝑖 = ∑ (𝑤𝑖𝑗 𝑎 ) (𝑤𝑗 𝑐) (𝑖 = 1, 2, … , 𝑛) 𝑚𝑗=1 (8) 3. results and discussion 3.1 developing the d-ahp hierarchical structure for dam type selection the integration of d-ahp selects the most influential attributes in determining the suitable type of dam. twelve experts are selected from the fields of ecological studies, socio-economic, biodiversity conservation, water quality, biology conservation, environmental impact assessment, environmental management, forest hydrology, wildlife ecology, and engineering to participate in this study. the years of experience among these selected experts in their respective field range from 3 to 10 years. table 3 shows 9 important criteria and 25 sub-criteria, which are selected by these experts after three iterations of a questionnaire survey. these attributes are used during the decision making process. the influential attributes are represented with abbreviations of c1 to c9 and sc1 to sc25 respectively, while the four alternatives are assigned with 'a', 'b', 'c' and 'd'. these attributes are then used to form the hierarchy structure as shown in figure 1. ijahp article: kaw, manaf/integration of delphi technique and analytical hierarchy process in selecting the type of dam: a case study of bungoh catchment in sarawak, malaysia international journal of the analytic hierarchy process 25 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.493 table 3 the most influential attributes in selecting the type of dam criteria definition sub-criteria definition c1 topography sc1 valley shape sc2 accessibility sc3 valley cross section sc4 river plan c2 hydrology sc5 river catchment sc6 average runoff sc7 daily flow sc8 groundwater sc9 flood regime c3 water quality c4 geology sc10 foundation type sc11 bearing capacity sc12 foundation thickness sc13 foundation dip sc14 foundation jointing sc15 foundation permeability sc16 seismicity sc17 safety c5 climate sc18 climate type sc19 rainfall sc20 temperature sc21 evaporation sc22 humidity c6 flora & fauna c7 land use c8 economical condition sc23 material supply sc24 new technology sc25 skilled contractor c9 local community the hierarchy structure consists of four levels. the first level is the objective (goal) of d-ahp, which is to select the optimal type of dam. the second level is the criteria, which consists of 9 criteria: (1) topography, (2) hydrology, (3) water quality, (4) geology, (5) climate, (6) flora & fauna, (7) land use, (8) economical condition and (9) local community. meanwhile, the third level consists of 25 sub-criteria, namely (1) valley shape, (2) dam site accessibility, (3) changes in valley cross section, (4) condition of the river in plan, (5) river catchment, (6) average runoff, (7) daily flow, (8) groundwater, (9) flood regime, (10) foundation type, (11) foundation bearing capacity, (12) foundation thickness, (13) foundation dip, (14) foundation joint, (15) foundation permeability, (16) seismicity, (17) safety, (18) climate type, (19) rainfall, (20) temperature, (21) evaporation rate, (22) humidity, (23) supply of the manufactured materials, (24) application of new technology and (25) skilled contractors. the fourth level is the type of dam, specifically embankment dam, gravity dam, arch dam, and buttress dam. ijahp article: kaw, manaf/integration of delphi technique and analytical hierarchy process in selecting the type of dam: a case study of bungoh catchment in sarawak, malaysia international journal of the analytic hierarchy process 26 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.493 figure 1 d-ahp hierarchy structure in selecting the type of dam to select optimal dam type hydrology climate local community land use water quality flora & fauna topography geology economical condition embankment dam gravity dam arch dam buttress dam type rainfall temperature evaporation humidity valley shape access cross section river plan material supply new technology skilled contractors type bearing capacity thickness dip jointing permeability seismicity safety catchment runoff flow groundwater flood regime ijahp article: kaw, manaf/integration of delphi technique and analytical hierarchy process in selecting the type of dam: a case study of bungoh catchment in sarawak, malaysia international journal of the analytic hierarchy process 27 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.493 brief explanations about the selected attributes are provided below: topography: in general, topography dictates the first choice in selecting the type of dam. the topographic considerations include surface configuration (such as valley shape, condition of the river in plan, changes in valley cross section) and accessibility to the potential dam construction site (walters, 1962; sherard et al., 1963; thomas, 1976; emiroglu, 2008). hydrology: this attribute includes the existence of river catchment and groundwater, the level of average runoff, daily flow, and flood regime at the potential dam site. the proposed type of dam depends on the purpose the dam serves. water quality: the effects of aggressive water on a dam may influence the type of dam; stored water in a reservoir might contain dissolved chemicals (such as acids) that are harmful for concrete (mason, 1990; emiroglu, 2008). geology: the type, bearing capacity, thickness, dip, jointing, and permeability of the geological foundations at the potential dam site are a set of decisive factors in selecting the type of dam (deere, 1976; soderberg, 1979; bureau of reclamation, 1987; bell, 1993; becue et al., 2002). however, the foundation limits the type of dam to a certain extent, which could be modified by considering the height of a proposed dam (emiroglu, 2008). climate: this attribute comprises climate types, rainfall, temperature, evaporation rate, and humidity, where the design of a dam could be considerably affected as the weather plays an essential role during the dam construction period (sherard et al., 1963; armstrong, 1977). flora & fauna: the existence of exotic flora and fauna at a dam site has an influence on the selection of dam type. the principal influence of environmental laws and regulations in selecting the type of dam is necessary to consider optimum environmental protection, in which the type of dam, size of dam, locations, and other aspects could be influenced (emiroglu, 2008). land use: it is recognized that a dam is a large artificial structure in regards to its height and type. thus, the location and natural environment should be taken into account when a potential dam site is selected (arai et al., 2009). if the constructed dam affects land use of the selected dam site, mitigation plans or other potential dam sites should be considered to preserve the existing settlements, arable land, and pastures in that site. economic condition of a country: a country's economic condition plays an important role in influencing the type of dam as the dam construction depends heavily on the supply of manufactured material such as cementitious materials, steel, asphalt, and others (sherard et al., 1963). the application of new technology for dam construction may be deployed if a country is economically strong, which enables an intricate dam framework with the assistance of skilled personnel (emiroglu, 2008). local community: local community refers to the villagers near the potential dam site. this attribute might seem to have no direct influence to the selection of dam type. however, with this large artificial structure at the potential site, the displacement of neighbouring settlements should be taken into consideration as they could potentially lose their source of income. forest or arable land nearby is typically ijahp article: kaw, manaf/integration of delphi technique and analytical hierarchy process in selecting the type of dam: a case study of bungoh catchment in sarawak, malaysia international journal of the analytic hierarchy process 28 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.493 used to grow crops, which is the local community’s source of income. while the construction of a dam is meant to bring comfort and contribute to human development, the natural environment is also a source of income for the local community; a balance between a manmade environment and a natural environment should exist for environmental preservation. 3.2 pairwise comparison and calculations the hierarchy structure (figure 1) is utilized to conduct pairwise comparison to obtain the final ranking of the type of dam for the case study of bungoh catchment in sarawak, malaysia. the bungoh catchment is located between 1.184° and 1.296° latitude and between 110.106° and 110.242° longitude (latifah et al., 2014). pairwise comparison is an important component, where expert’s judgments are recoded based on saaty's scale of preference, as previously shown in table 1. referring to the developed hierarchy structure (figure 1), experts work together as a unit to aggregate their judgements for the comparison matrices according to their knowledge, experience, and expertise (forman & peniwati, 1998). a group interview with twelve experts was conducted to discuss and come up with an agreement regarding the intensity of importance for each comparison matrix. the pairwise comparison of d-ahp is conducted by comparing each level [1] criteria towards goal, [2] sub-criteria towards criteria, and [3] alternatives towards criteria and subcriteria. table 4 shows the comparison matrix of criteria towards the goal. using equations 2-5, the eigenvectors (relative weights) of the attributes and alternatives are obtained and provided in the final column, as shown in table 4. the sum of the relative weights equals1, as these relative weights are normalized. relative weights indicate the priority of each criterion in this pairwise comparison. c1 has the highest priority with the highest relative weight (0.271), followed by c4 (0.202), c8 (0.142), c9 (0.122), c6 (0.113), c7 (0.076), c2 (0.028), c3 (0.024), and c5 (0.022). referring to table 2, the principle eigenvalue (λmax) for this pairwise comparison matrix is 9.66 while rci is 1.45 for 9 criteria. ci (0.083) and cr (0.057) are calculated using equations 6 and 7 respectively. based on table 4, cr is less than 0.1, which indicates an acceptable and consistent decision. this process is repeated with all pairwise comparisons to obtain respective eigenvectors. ijahp article: kaw, manaf/integration of delphi technique and analytical hierarchy process in selecting the type of dam: a case study of bungoh catchment in sarawak, malaysia international journal of the analytic hierarchy process 29 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.493 table 4 pairwise comparison of criteria towards goal c1 c2 c3 c4 c5 c6 c7 c8 c9 weight c1 1 9 8 2 5 2 3 3 5 0.271 c2 1/9 1 1 1/7 2 1/4 1/3 1/4 1/8 0.028 c3 1/8 1 1 1/7 2 1/6 1/5 1/6 1/9 0.024 c4 1/2 7 7 1 8 2 3 2 2 0.202 c5 1/5 1/2 1/2 1/8 1 1/2 1/5 1/8 1/7 0.022 c6 1/2 4 6 1/2 2 1 2 1 1 0.113 c7 1/3 3 5 1/3 5 1/2 1 1/3 1/2 0.076 c8 1/3 4 6 1/2 8 1 3 1 2 0.142 c9 1/5 8 9 1/2 7 1 2 1/2 1 0.122 principle eigen value (λmax) consistency index (ci) random consistency index (rci) consistency ratio (cr) 9.66 0.083 1.45 0.057 3.3 priority analysis and overall ranking priority analysis lists the calculated eigenvectors (relative weights) for three levels of pairwise comparison matrices. the first column in table 5 reveals the eigenvectors of criteria with respect to the specified objective (goal). the second column reveals the eigenvectors of sub-criteria towards criteria, and the final four columns reveal the eigenvectors of alternatives towards criteria and sub-criteria. the total weight of each alternative is calculated using equation 8, as tabulated in the final row of table 5. ijahp article: kaw, manaf/integration of delphi technique and analytical hierarchy process in selecting the type of dam: a case study of bungoh catchment in sarawak, malaysia international journal of the analytic hierarchy process 30 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.493 table 5 the total weight of each type of dam criteria subcriteria alternatives a b c d c1 (0.271) sc1 (0.711) 0.047 0.413 0.238 0.302 sc2 (0.154) 0.593 0.208 0.081 0.118 sc3 (0.068) 0.445 0.360 0.127 0.069 sc4 (0.068) 0.505 0.308 0.091 0.096 c2 (0.028) sc5 (0.522) 0.422 0.327 0.106 0.124 sc6 (0.071) 0.579 0.202 0.104 0.115 sc7 (0.071) 0.662 0.116 0.090 0.132 sc8 (0.062) 0.076 0.318 0.080 0.526 sc9 (0.274) 0.087 0.525 0.218 0.171 c3 (0.024) 0.066 0.277 0.315 0.342 c4 (0.202) sc10 (0.272) 0.074 0.072 0.481 0.372 sc11 (0.199) 0.052 0.223 0.419 0.306 sc12 (0.044) 0.472 0.315 0.122 0.091 sc13 (0.106) 0.064 0.255 0.450 0.231 sc14 (0.113) 0.628 0.180 0.091 0.101 sc15 (0.025) 0.627 0.213 0.077 0.083 sc16 (0.052) 0.077 0.376 0.404 0.143 sc17 (0.189) 0.062 0.210 0.506 0.222 c5 (0.022) sc18 (0.063) 0.498 0.197 0.139 0.166 sc19 (0.408) 0.498 0.197 0.139 0.166 sc20 (0.094) 0.498 0.197 0.139 0.166 sc21 (0.372) 0.498 0.197 0.139 0.166 sc22 (0.062) 0.498 0.197 0.139 0.166 c6 (0.113) 0.176 0.231 0.430 0.163 c7 (0.076) 0.389 0.079 0.170 0.362 c8 (0.142) sc23 (0.548) 0.058 0.380 0.305 0.257 sc24 (0.241) 0.129 0.125 0.340 0.405 sc25 (0.210) 0.095 0.089 0.265 0.551 c9 (0.122) 0.103 0.395 0.135 0.367 total weight 0.176 0.277 0.272 0.275 the total weight for alternatives a, b, c, and d is 0.176, 0.277, 0.272, and 0.275 respectively. as a result, alternative b, which is the gravity dam, has the highest total weight; hence, it is considered the optimal dam type for the selected site, which is the bungoh catchment. the selection of a dam type is quantified based on the developed influential attributes, which include environmental criteria, social criteria, and engineering criteria. essentially, the selection takes the characteristics of a potential dam site into account during the pairwise comparison process. according to singh & sharma (1976), the type of dam is typically dictated by the foundation (geology) and valley shape (topography). the foundation and valley shape at the bungoh catchment is a rock foundation and a narrow u-shaped valley respectively. the selection of a gravity dam for bungoh catchment satisfied both of the fundamental attributes, as a gravity dam is to be constructed on a sound rock foundation with a narrow shaped valley to ensure it is seismically safe. although the total weights of both alternative c and alternative d are slightly lower than alternative b, experts suggested that alternative b is a more suitable type of dam for bungoh catchment. ijahp article: kaw, manaf/integration of delphi technique and analytical hierarchy process in selecting the type of dam: a case study of bungoh catchment in sarawak, malaysia international journal of the analytic hierarchy process 31 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.493 4. conclusion the selection of a dam type is influenced by various factors, which may not be welldocumented. this complicates the decision making process and the decision made may be rather subjective since a set of systematic processes does not exist. therefore, a type of multi-criteria decision making (mcdm) method is utilized to address this problem. in this study, the integration of delphi-analytical hierarchy process (dahp) develops a set of influential attributes (9 criteria and 25 sub-criteria), which assists in selecting the type of dam for a potential dam site. through the developed hierarchy structure, a complicated decision making process is simplified and could be performed objectively, as it takes three fundamental elements into account, namely environmental criteria, social criteria, and engineering criteria, rather than focusing on one specific criterion. the outcome of this study is proven effective as it is in line with the selected type of dam for bungoh catchment, which is the gravity dam. in addition, the developed ahp hierarchy structure, not only could be employed for bungoh catchment, but it is also applicable in the preliminary stage of any dam development project. ijahp article: kaw, manaf/integration of delphi technique and analytical hierarchy process in selecting the type of dam: a case study of bungoh catchment in sarawak, malaysia international journal of the analytic hierarchy process 32 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.493 appendix example of pairwise comparison matrix calculation comparison of alternatives with respect to valley shape sub-criterion the pairwise comparison matrix in table 6 shows that gravity dam and arch dam are of "very strong importance" to embankment dam, with the intensity of importance of 7. both of the judgement values are in reciprocal values (1/7). this is due to both the pairwise comparison between embankment dam and gravity dam and the pairwise comparison between embankment dam and arch dam favouring the gravity dam and arch dam over embankment dam. meanwhile, buttress dam is of "strong importance" compared to an embankment dam, with the intensity of importance of 6. the selection of intensity importance of 6, which is the intermediate value between the intensity of importance of 5 and 7, signifies the need of compromising, as revealed in table 1. the judgement value is in reciprocal values (1/6) as the pairwise comparison between embankment dam and buttress dam favours a buttress dam over an embankment dam. the pairwise comparison value equals 1 when an embankment dam is compared to itself. on the other hand, a gravity dam is of "moderate importance" compared to an arch dam, with the intensity of importance of 3. this signifies that the judgement slightly favours a gravity dam over an arch dam. gravity dam is of "equal importance" compared to buttress dam, with the intensity of importance of 1. this represents that the comparison between a gravity dam and a buttress dam contributes equally to the objective. arch dam has "equal importance" compared to buttress dam, with the intensity of importance of 1. this means that the comparison between arch dam and buttress dam contributes equally to the objective as well. the reciprocal matrix value is used to complete the comparison. for example, the pairwise comparison value for embankment dam with respect to gravity dam equals to 1/7; thus, the pairwise comparison value for gravity dam with respect to embankment dam equals to 7. the subsequent step after pairwise comparison is the calculation of eigenvectors. there are several methods to calculate the eigenvectors. one of the methods, which gives a very good approximation, is to multiply the entries in each row of the matrix and then take the n th root of that product (coyle, 2004). the n th root is summed and that value is used to normalize the eigenvectors. the summed eigenvectors is equivalent to 1.00. in the following matrix, the 4 th root for the first row is 0.242 and this value is divided by 5.184, which equals 0.047, as the first element in the eigenvector. the eigenvector for gravity dam, arch dam and buttress dam is 0.413, 0.238 and 0.302 respectively. ijahp article: kaw, manaf/integration of delphi technique and analytical hierarchy process in selecting the type of dam: a case study of bungoh catchment in sarawak, malaysia international journal of the analytic hierarchy process 33 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.493 table 6 pairwise comparison and calculation of eigenvectors embankment dam gravity dam arch dam buttress dam n th root of product of values eigen vectors embankment dam 1 1/7 1/7 1/6 0.242 0.047 gravity dam 7 1 3 1 2.141 0.413 arch dam 7 1/3 1 1 1.236 0.238 buttress dam 6 1 1 1 1.565 0.302 total 5.184 1.000 calculation of principle eigenvalue the following step is the calculation of the principle eigenvalue (λmax). the matrix of judgments is multiplied by the calculated eigenvectors to obtain a new vector, as shown in the following calculation: embankment dam = 1*0.047 + 1/7*0.413 + 1/7*0.238 + 1/6*0.302 = 0.190 gravity dam = 7*0.047 + 1*0.413 + 3*0.238 + 1*0.302 = 1.758 arch dam = 7*0.047 + 1/3*0.413 + 1*0.238 + 1*0.302 = 1.007 buttress dam = 6*0.047 + 1*0.413 + 1*0.238 + 1*0.302 = 1.235 these vectors of four elements (0.190, 1.758, 1.007, and 1.235) are the product of a x w (equation 2) and the theory of ahp stated in equation 3 and equation 5, where λmax is determined by dividing each component with the corresponding eigenvector: embankment dam = 0.190/0.047 = 4.043 gravity dam = 1.758/0.413 = 4.257 arch dam = 1.007/0.238 = 4.231 buttress dam = 1.235/0.302 = 4.089 the mean of these values equals 4.155 and this value is λmax of the matrix. if any of the estimated λmax is less than n or in this case, n = 4, this denotes that there is an error in the calculation. this serves as a useful sanity check (coyle, 2004). testing of the consistency ratio next, the ci is determined through equation 6 and since n = 4 for this matrix, the ci is: ci = (4.155 4) / (4 1) = 0.052 the final step is the calculation of cr. for this set of judgments, referring to table 2 for rci value, using the obtained ci as the corresponding value, cr is calculated (saaty, 1980; coyle, 2004). therefore, cr (equation 7) for this matrix is: cr = 0.052 / 0.90 = 0.058 calculation of total weight using equation 8, the total weight of each dam type alternative is calculated. the following is an example of the calculation to obtain total weight for the embankment dam, which is by integrating the eigenvectors of decision elements: wa = (0.271*0.711*0.047) + (0.271*0.154*0.593) + (0.271*0.068*0.445) + (0.271*0.068*0.505) + (0.113*0.176) + (0.076*0.389) + (0.022*0.063*0.498) + (0.022*0.408*0.498) + (0.022*0.094*0.498) + ijahp article: kaw, manaf/integration of delphi technique and analytical hierarchy process in selecting the type of dam: a case study of bungoh catchment in sarawak, malaysia international journal of the analytic hierarchy process 34 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.493 (0.022*0.372*0.498) + (0.022*0.062*0.498) + (0.028*0.522*0.442) + (0.028*0.071*0.579) + (0.028*0.071*0.662) + (0.028*0.062*0.076) + (0.028*0.274*0.087) + (0.202*0.272*0.074) + (0.202*0.199*0.052) + (0.202*0.044*0.472) + (0.202*0.106*0.064) + (0.202*0.113*0.628) + (0.202*0.025*0.627) + (0.202*0.052*0.077) + (0.202*0.189*0.062) + (0.024*0.066) + (0.122*0.103) + (0.142*0.548*0.058) + (0.142*0.241*0.129) + (0.142*0.210*0.095) = 0.176 ijahp article: kaw, manaf/integration of delphi technique and analytical hierarchy process in selecting the type of dam: a case study of bungoh catchment in sarawak, malaysia international journal of the analytic hierarchy process 35 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.493 references adamcsek, e. (2008). the analytical hierarchy process and its generalizations. eötvös loránd university. anglin, g. l. (1991). instructional technology past, present and future. englewood, co: libraries unlimited inc. arai, o., baba, k., & hirose, t. (2009). selection of type of dams and reservoirs. in y. takahasi (ed.), water storage, transport, and distribution (pp. 103–113). tokyo, japan. armstrong, e. l. (1977). selection of the type of dam. in a. r. golze (ed.), handbook of dam engineering (pp. 267–289). new york: van nostrand reinhold. arof, a. (2015). the application of a combined delphi-ahp method in maritime transport research a review. asian social science, 11(23), 73–82. doi: http://dx.doi.org/10.5539/ass.v11n23p73 azami, k., takemoto, m., otsuka, y., yamagishi, s., & nakazawa, s. (2012). meteorology and species composition of plant communities, birds and fishes before and after initial impoundment of miharu dam reservoir, japan. landscape and ecological engineering, 8, 81–105. doi: https://doi.org/10.1007/s11355-011-01703 beck, m. w., claassen, a. h., & hundt, p. j. (2013). environmental and livelihood impacts of dams: common lessons across development gradients that challenge sustainability. international journal of river basin management, 10(1), 73–92. doi: https://doi.org/10.1080/15715124.2012.656133 becue, j.-p., degoutte, g., & lautrin, d. (2002). choice of site and type of dam. in small dams guidelines for design, construction and monitoring (pp. 17–22). france: cemagref editions and engref. bell, f. g. (1993). engineering geology. london: blackwell scientific. brooks, k. w. (1979). delphi technique: expanding applications. north central association quarterly, 54(3), 377–385. bureau of reclamation, b. (1987). design of small dams. denver, colorado: u.s. government printing office. chaves, s., mazzon, j. a., & de souza, c. a. (2012). proposition of a method to measure rankings using the delphi technique. future studies research journal, 4(2), 52–75. doi: http://dx.doi.org/10.24023/futurejournal/2175-5825/2012.v4i2.118 chung, c. c., & her, m. t. (2013). port state control perception of the safe management of bulk carrier. in proceedings of the international forum on shipping, ports & airports (ifspa) (pp. 435–444). hong kong: hong kong polytechnic university. coyle, g. (2004). the analytic hierarchy process (ahp). principal strategy. open access material, (1980), 1–11. http://dx.doi.org/10.5539/ass.v11n23p73 https://doi.org/10.1080/15715124.2012.656133 http://dx.doi.org/10.24023/futurejournal/2175-5825/2012.v4i2.118 ijahp article: kaw, manaf/integration of delphi technique and analytical hierarchy process in selecting the type of dam: a case study of bungoh catchment in sarawak, malaysia international journal of the analytic hierarchy process 36 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.493 custer, r. l., scarcella, j. a., & stewart, b. r. (1999). the modified delphi technique: a rotational modification. journal of vocational and technical education, 15(2), 1–10. doi: http://dx.doi.org/10.21061/jcte.v15i2.702 cyphert, f. r., & gant, w. l. (1971). the delphi technique: a case study. phi delta kappan, 52, 272–273. da cruz, m. r. p., ferreira, j. j., & azevedo, s. g. (2013). key factors of seaport competitiveness based on stakeholder perspective: an analytic hierarchy process (ahp) model. maritime economics & logistics, 15(4), 416–443. doi: https://doi.org/10.1057/mel.2013.14 dalkey, n.c. and helmer, o. (1963). an experimental application of the delphi method to the use of experts. management science, 9(3), 458–467. doi: https://doi.org/10.1287/mnsc.9.3.458 dalkey, n. c., & rourke, d. l. (1972). experimental assessment of delphi procedures with group value judgments. in n. c. dalkey, d. l. rourke, r. lewis, & d. snyder (eds.), studies in the quality of life: delphi and decision-making (pp. 55– 83). lexington, ma: lexington books. deere, d. u. (1976). dams on rock foundation, some design questions. in rock engineering for foundations and slopes: proceedings speciality conference (pp. 55–85). co: asce, boulder. emiroglu, m. e. (2008). influences on selection of the type of dam. international journal of science & technology, 3(2), 173–189. forman, e., & peniwati, k. (1998). aggregating individual judgments and priorities with the analytic hierarchy process. european journal of operational research, 108, 165–169. doi: https://doi.org/10.1016/s0377-2217(97)00244-0 gawlik, r. (2008). preliminary criteria reduction for the application of analytical hierarchy process method (no. 45348). retrieved from https://mpra.ub.unimuenchen.de/45348/ hsu, c., & sandford, b. a. (2007). the delphi technique: making sense of consensus. practical assessment, research & evaluation, 12(10), 1–8. jacobs, j. m. (1996). essential assessment criteria for physical education teacher education programs: a delphi study. west virginia university, morgantown. lamaakchaoui, c., azmani, a., & jarroudi, m. el. (2015). an ahp-based model for selecting complementary products. international journal of computer applications, 120(22), 975–980. retrieved from http://research.ijcaonline.org/volume120/number22/pxc3904425.pdf latifah, a. m., gabriel, t. w., & les, m. (2014). identification of the potential dam site of bungoh catchment by assessing ecosystem loss score and ecosystem fragmentation score using the rarity and viability value. international journal of development research, 4(5), 1113–1121. http://dx.doi.org/10.21061/jcte.v15i2.702 https://doi.org/10.1287/mnsc.9.3.458 https://doi.org/10.1016/s0377-2217(97)00244-0 https://mpra.ub.uni-muenchen.de/45348/ https://mpra.ub.uni-muenchen.de/45348/ http://research.ijcaonline.org/volume120/number22/pxc3904425.pdf ijahp article: kaw, manaf/integration of delphi technique and analytical hierarchy process in selecting the type of dam: a case study of bungoh catchment in sarawak, malaysia international journal of the analytic hierarchy process 37 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.493 lee, c., wan, j., shi, w., & li, k. (2014). a cross-country study of competitiveness of the shipping industry. transport policy, 35, 366–376. doi: https://doi.org/10.1016/j.tranpol.2014.04.010 ludwig, b. (1997). predicting the future: have you considered using the delphi methodology? journal of extension, 35(5), 1–4. ludwig, b. g. (1994). internationalizing extension: an exploration of the characteristics evident in a state university extension system that achieves internationalization. the ohio state of university, columbus. mason, p. j. (1990). the effects of aggressive water on dam concrete. international water power & dam construction, 39–42. doi: https://doi.org/10.1016/09500618(90)90026-w moradi, a., etebarian, a., shirvani, a., & soltani, i. (2014). development of a fuzzy model for iranian marine casualties management. journal of fuzzy set valued analysis, 1–17. doi: 10.5899/2014/jfsva-00186 saaty, t. l. (1980). the analytical hierarchy process. new york: mcgraw hill inc. saaty, t. l. (1990). how to make a decision: the analytical hierarchy process. european journal of operational research, 48, 9–26. doi: https://doi.org/10.1016/0377-2217(90)90057-i saaty, t. l. (2003). rank, normalization and idealization in the analytic hierarchy process. in 7th international symposium on the analytic hierarchy process (isahp) (pp. 57–64). bali, indonesia. saaty, t. l. (2008). decision making with the analytic hierarchy process. international journal of services sciences, 1(1), 83–98. doi: 10.1504/ijssci.2008.017590 safari, m., ataei, m., khalokakaie, r., & karamozian, m. (2010). mineral processing plant location using the analytical hierarchy process a case study: the sangan iron ore mine (phase 1). international journal of mining science and technology, 20, 691–695. doi: https://doi.org/10.1016/s1674-5264(09)60264-7 sayareh, j., & alimini, h. r. (2014). a hybrid decision-making model for selecting container seaport in the persian gulf. the asian journal of shipping and logistics, 30(1), 75–95. doi: https://doi.org/10.1016/j.ajsl.2014.04.004 sherard, j. l., woodward, r. j., gizienski, s. f., & clevenger, w. a. (1963). earth and earth-rock dams. engineering problems of design and construction. new york: wiley. singh, b., & sharma, h. d. (1976). earth and rockfill dams. meerut, india: prabhat press. soderberg, a. d. (1979). expect the unexpected: foundation for dams in karst. bulletin of the association of engineering geologists, xvi(3), 409–425. doi: https://doi.org/10.2113/gseegeosci.xvi.3.409 https://doi.org/10.1016/j.tranpol.2014.04.010 https://doi.org/10.1016/0950-0618(90)90026-w https://doi.org/10.1016/0950-0618(90)90026-w https://doi.org/10.1016/0377-2217(90)90057-i http://dx.doi.org/10.1504/ijssci.2008.017590 https://doi.org/10.1016/s1674-5264(09)60264-7 https://doi.org/10.1016/j.ajsl.2014.04.004 https://doi.org/10.2113/gseegeosci.xvi.3.409 ijahp article: kaw, manaf/integration of delphi technique and analytical hierarchy process in selecting the type of dam: a case study of bungoh catchment in sarawak, malaysia international journal of the analytic hierarchy process 38 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.493 teknomo, k. (2006). analytic hierarchy process (ahp) tutorial. retrieved from http://people.revoledu.com/kardi/tutorial/ahp/ thomas, h. h. (1976). the engineering of large dams. london, uk: wiley. triantaphyllou, e., & mann, s. h. (1995). using the analytic hierarchy process for decision making in engineering applications: some challenges. international journal of industrial engineering: theory, applications and practice, 2(1), 35–44. walters, r. s. (1962). dam geology. london: butterworths. weaver, w. t. (1971). the delphi forecasting method. phi delta kappan, 52(5), 267–273. yasser, m., jahangir, k., & mohmmad, a. (2013). earth dam site selection using the analytic hierarchy process (ahp): a case study in the west of iran. arabian journal of geosciences, 6, 3417–3426. doi: https://doi.org/10.1007/s12517-012-0602-x yau, y. (2009). multi-criteria decision making for urban built heritage conservation: application of the analytic hierarchy process. journal of building appraisal, 4(3), 191–205. doi: https://doi.org/10.1057/jba.2008.34 http://people.revoledu.com/kardi/tutorial/ahp/ ijahp news: ferretti/ isahp meeting in washington d.c. international journal of the analytic hierarchy process 290 vol. 5 issue 2 2013 issn 1936-6744 isahp meeting in washington dc, june 29 – july 2, 2014 valentina ferretti the next international symposium on the analytic hierarchy process/analytic network process (isahp) will take place between june 29 and july 2 in washington dc, at the very heart of its elegant and fascinating downtown (http://www.isahp.org/). this is the first time that a consecutive annual conference will take place, and this will ensure that from now on the international conference on multicriteria decision making and the international symposium of the analytic hierarchy process will not occur in the same year. thus all interested attendees will be able to participate in both of these important events. the isahp is a truly international event that brings together researchers, teachers, students and users of ahp/anp to share their research and experiences in decision-making. the last meeting, which took place in kuala lumpur, malaysia, in june 2013 attracted approximately 140 participants from more than 30 countries around the world, and is remembered as an excellent meeting thanks to the constructive learning environment and the exceptional breadth of the scientific contributions! there are several highlights to note about the isahp 2014 meeting. first, there will be a student track dedicated to undergraduate/master students to incentivize participation from younger researchers. also, there will be a focus on cooperative research, and therefore the emphasis will be on organized sessions rather than on individual isolated submissions. for conference evaluation purposes only a paper http://www.isahp.org/ ijahp news: ferretti/ isahp meeting in washington d.c. international journal of the analytic hierarchy process 291 vol. 5 issue 2 2013 issn 1936-6744 proposal will be needed 1 . the conference chairman this year will be professor enrique mu, recent winner of the best paper award at the isahp 2013! last but not least, you may be happy to know that participating in the isahp 2014 meeting will give you the opportunity to not only attend a very interesting and stimulating event, but also to participate in the 4 th of july celebrations in the heart of the capital! so, please submit your proposals, and contribute to the success of this important event! for details visit the symposium website at http://www.isahp.org key dates paper and session proposal submission deadline: march 31, 2014 rolling notification of acceptance with final acceptance no later than: april 15, 2014 full paper submission deadline: april 30, 2014 early registration deadline (for presenters and non-presenters): may 1, 2014 normal registration deadline (available to non-presenters only): may 2, 2014 – june 16, 2014 symposium dates: june 29, 2014 – july 2, 2014 1 full papers are optional for those who would like their work to appear in the conference proceedings or to explore the potential for publication in any of the sponsoring journals and may be submitted after initial proposal submission and acceptance for the symposium. http://www.isahp.org/ rob typewritten text http://dx.doi.org/10.13033/ijahp.v5i2.211 ijahp: mu/the critical role of journal reviewers international journal of the analytic hierarchy process 372 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.357 the critical role of journal reviewers any journal relies on voluntary peer reviewers to ensure the quality of research articles. since reviewers are usually as busy as any of us, their voluntary contribution is especially appreciated by our ijahp editorial team. reviewing a paper requires a set of skills, which can be summarized as thoroughness, effectiveness, timeliness and quality. thoroughness is the ability to review the paper in its entirety and, most importantly, get an understanding of the topic and author’s discussion of it. authors highlight the importance of this ability when they complain that the reviewer “didn’t get it.” authors may be at fault for lack of clarity, and it is the role of the reviewer to point this out. other common situations occur when a reviewer runs out of steam halfway through the paper, or when the reviewer focuses mainly on formatting and minor issues and avoiding –perhaps for lack of time – getting deep into a discussion of the paper. effectiveness refers to the feedback provided by the reviewer. the feedback must allow the editors to make up their minds regarding inclusion of the paper in the journal, and also allow the authors to improve their paper. judgments such as “paper is not suitable for this journal,” or “literature review is poorly written” are not very helpful. on the other hand, comments such as “should the authors [do this], their paper would be more suitable”, or “authors should use a table to summarize [be specific]” provide pointers to the authors about what could be done to improve the paper or in a worst-case scenario, the problems that led to its rejection. similarly, the review must be useful for the editors who will have to make a final decision on the paper. if the reviewer avoids taking a position on the paper, the editors are left clueless about what to do with it. timeliness is a very important issue for the editors since it allows them to plan ahead. also, timeliness helps to avoid the problem of waiting for a reviewer’s report without knowing when it will arrive or if it will arrive at all. finally, and like in any other activity, the job must be done with quality. if the review is done with high standards of quality, the comments will be on target, clear, well-structured and will allow the authors to improve their paper as well as help the editors make up their minds about whether or not to accept the paper. quality is harder to define but, like the old joke, we will recognize it as soon as we see it. since we have moved from publishing two issues a year to three, the reviewer’s role has become even more critical for ijahp success, in particular for this first year of increased production. for this reason, we have decided to honor our reviewers through the selection of four reviewers who exhibit the qualities highlighted above. please read the “reviewer awards” piece in our news and events section to find out who our honored reviewers are. we did not use a scientific approach for the selection (ahp was definitely not involved), but requested that each editor choose a single exemplary reviewer; therefore, we are certain that several valuable collaborators were not included. unfortunately, we ca ijahp: mu/the critical role of journal reviewers international journal of the analytic hierarchy process 373 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.357 not give awards to the dozens of colleagues who have contributed such exemplary reviews for our journal. we hope that our honored reviewers constitute rather a representation of all the hard-working reviewers who make this journal possible. to all our reviewers, thank you very much! sincerely, enrique mu, phd editor-in-chief ijahp news and events: karpak and slowinski/bernard roy award 2022 international journal of the analytic hierarchy process 1 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.1069 dr. banu lokman: 2022 recipient of prestigious bernard roy award birsen karpak youngstown state university, usa roman słowiński poznań university of technology, poland it is great pleasure for us to share with the mcdm/a community that dr. banu lokman was the 2022 recipient of prestigious bernard roy (br) award. dr. banu lokman is a reader in operational research at the school of organisations, systems and people, university of portsmouth, uk. we have known dr. lokman as an excellent member of the mcdm community for over 10 years and have always thought that she deserves an award. we are very pleased that she was recognized by the euro working group on multiple criteria decision aiding (ewg mcda). the award was given during the ewg mcda business meeting in agios nicolaos, crete last september by yannis siskos, the 2022 jury chair. congratulations dr. lokman, we are proud of you! https://www.linkedin.com/in/acoaaa5jlpob_bhls8-9ktyaeyqmzurvv79vuvm https://www.linkedin.com/in/acoaaa5jlpob_bhls8-9ktyaeyqmzurvv79vuvm ijahp news and events: karpak and slowinski/bernard roy award 2022 international journal of the analytic hierarchy process 2 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.1069 dr. lokman was nominated by salvatore corrente, university of study of catania, catania, italy, matteo brunelli, university of trento, trento, italy and miłosz kadziński, poznan university of technology poznań, poland. their nomination letter to professor yannis siskos, chair of the jury for the 2022 bernard roy award of ewg-mcda reads as follows: “as members of the ewg on mcda, we would like to nominate banu lokman for the bernard roy award 2022 of ewg-mcda. we believe that she fully deserves the award for the following reasons: ● she has made an excellent contribution to the development of theory and methodologies in multiple criteria decision aiding, as evidenced by her publications in prestigious scientific journals. her research activity has mainly focused on multiobjective integer and mixed integer programming problems, which are utilized to model real-life complex decision problems. she developed efficient exact and heuristic algorithms to solve these computationally hard problems and studied their applications in energy and healthcare. her recent works aim to deliver innovative solutions that address the challenges businesses face today, integrating machine learning and clustering tools into the mcda. ● she has collaborated with pioneers and many qualified researchers in the field of mcda. she held positions in finland, turkey, and the uk and carried out numerous academic visits. she is also in collaboration with healthcare professionals, working on designing decision support tools. ijahp news and events: karpak and slowinski/bernard roy award 2022 international journal of the analytic hierarchy process 3 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.1069 ● she has established a strong reputation in the field of mcda through her globally recognized research and services in professional bodies and international journals. she serves as an associate editor at omega; she was a board member of the informs section on mcdm (2018-2020) and secretary of the international society on mcdm (2015-2019). she served as a jury member for the mcdm doctoral dissertation award (2015, 2017, 2019) and the informs mcdm junior researcher best paper award (2018-2020). ● she has been very active in the organizational side of mcda initiatives. she organized the 26 th international conference on mcdm at the university of portsmouth in june 2022. she organized the mcdm stream of the uk’s or63 conference in september 2021. she will lead and deliver a natcor course, a uk-based ph.d. course on mcdm, in september 2022. ● she has received many acknowledgments for her research, course management, and teaching activities. we want to mention just two facts: the funding she obtained from the scientific and technological research institution of turkey (tubitak) and the promotion to reader in operational research in june 2022 at the university of portsmouth. in conclusion, we are pleased to emphasize, in addition to her skills, the enthusiasm that has always characterized banu lokman’s activities, an aspect that has undoubtedly favored the research collaborations she has established. we are convinced that offering the bernard roy award 2022 to banu lokman would be appreciated by the entire mcda community and would be a valuable stimulus for the community researchers.” sincerely, salvatore corrente, matteo brunelli, miłosz kadziński ijahp: mu/ijahp is ten years old international journal of the analytic hierarchy process 1 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.576 ijahp is ten years old april 20, 2018 this issue marks the start of our tenth year in production. our ijahp team is proud to reach this important milestone, and we can assure you this year will bring important improvements for our publication. as a start, we have recently migrated to the newest ojs platform which has allowed us to provide a more vibrant look for the journal as well as improve the publication workflow process. also, this is a special year because for the first time we will need to work without the inspiration and enthusiasm of tom saaty, ahp/anp creator and founder of this journal. while he was the spiritual force behind ahp/anp diffusion, we remain fully committed to continue his legacy. for this year, we plan to publish a special issue on fuzzy ahp and to sponsor the international symposium of the analytic hierarchy process isahp2018 that will take place in hong kong, july 12-15. we expect to publish the best papers presented at the isahp2018 meeting, and will also be present in a panel of editors to share our collective experience with the new generation of ahp/anp scholars and practitioners. in addition, we will offer a workshop on how to improve anp reporting to help improve the overall quality of anp publications. looking forward to meeting you at isahp2018 in hong kong! enrique mu, phd ijahp editor-in-chief ijahp article: peniwati/group decision making: drawing out and reconciling differences international journal of the analytic hierarchy process 385 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.533 group decision making: drawing out and reconciling differences kirti peniwati ppm graduate school of management jakarta, indonesia kirti@indo.net.id abstract why is group decision making so important today? in our increasingly complex environment, decision making becomes more and more challenging for leaders and practitioners. working in groups appears to be the norm because the alignment of visions and actions are critical for an organization. a leader or a group facilitator needs a supporting system to make collective thinking effective. the book, group decision making: drawing out and reconciling differences, written by thomas saaty and myself shows that the ahp is the scientific approach for supporting group processes in the current and future complex environment (saaty & peniwati, 2008). keyword: ahp; group decision making; facilitating decisions 1. introduction globalization breaks down geographical and cultural human-made boundaries which creates opportunities as well as risks for those affected. global challenges for humanity expresses these challenged when it asks the question, “how can decision making be enhanced by integrating improved global foresight during unprecedented accelerating change?” under these conditions of global complexity and interdependence, it is almost impossible for decision makers to gather and understand the information required to make and implement coherent policy. at the same time, the consequences of incoherent policies are so serious. dealing with the uncertainty, unpredictability, ambiguity and surprise in the environment, together with the need to synthesize collective knowledge and ideas, it is quite challenging even for competent leaders or group facilitators. they need a simple support system that works. 2. general remarks complex problem solving remains the most important of the top 10 skills for the future as reported by the montreal gazette, which also lists critical thinking, creativity, coordinating with others, and judgment and decision making among the top 7(montreal gazette, 2016). creativity, which implies learning competence, emerges as a new item in the top 10 list of future skills. edward de bono, the world’s leading authority in conceptual thinking, maintains that other than the judgment type of thinking to answer mailto:kirti@indo.net.id ijahp article: peniwati/group decision making: drawing out and reconciling differences international journal of the analytic hierarchy process 386 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.533 the usual question of ‘what is’, we also need the design type of ‘what is possible’ questions (de bono, 1990). design thinking with a positive mindset appears to be increasingly needed in this new millennium. it is the creative decision-making process that focuses more on an organization’s strengths to capture opportunity offered by the environment. in business, it is the attempt to match people's needs with what is technologically feasible, and how a viable business strategy can convert into customer value and market opportunity. ahp shows such a capability, for example, in its unique forward-backward planning approach. it enables us to use our knowledge and technology to design better ways forward. ahp/anp is arguably the only scientifically based decision making approach that has been widely applied by organizations and companies around the world. for example, ibm rochester used the ahp for benchmarking with other best-of-breed companies, leading to its winning the malcolm baldridge award (the silverlake project, 1992; eyrich, 1991). martin marietta air traffic systems used the ahp to evaluate alternative architectures for a communication system for the federal aviation administration (faa), involving 30 experts representing many divisions and regions of the organization (the silverlake project, 1992; eyrich, 1991). the cooperative fish and wildlife research unit of the alaska department of fish and game applied the ahp to the management of the recreational fishery, being complicated by the conflicting objectives of multiple stakeholders (bristol bay, 2005). the role of the ahp can be increasingly significant in the future. the ahp’s new fundamental scale removes the general impossibility of rational ordinal aggregation of arrow in his nobel winning theory (arrow, 1951). the new scale also makes ahp applicable to problems with a high level of complexity. ahp structures represent either a tacit understanding of a problem, or are the outcome of design thinking such as in strategy formulation or architectural design. the analytical planning of the ahp offers a unique approach to integrate looking forward (projecting the future) and looking backward (designing strategy). in a choice problem, the creative identification of new alternatives can make the difference between a mediocre and an excellent decision. the body of ahp knowledge includes requirements and suggestions for eliciting and synthesizing diverse collective knowledge and ideas. it requires mastery in group facilitating. for example, it requires knowledge of the selection of group members and knowing whether or not smaller groups need to be formed and assigned different parts of the ahp structure. it is important to make a distinction between high-level decision makers that will judge the relative importance of criteria and experts that will evaluate and judge relative preference for alternatives with respect to the criteria. a group judgment can be meaningful only if it is an aggregation of homogeneous judgments, hence significant differences would need to be discussed by the group and brought into alignment first. here, ahp’s compatibility measure that can be used to determine how far apart two priority vectors are would be useful. the challenge in applying ahp in group decision making is striving for group coherence, making a group think like an individual and dealing with complacency as well. fortunately, ahp does not need total perfection to produce a reasonably valid outcome; it simply asks for an acceptable level of accuracy and consistency. a group, in working to align the thinking of its members ijahp article: peniwati/group decision making: drawing out and reconciling differences international journal of the analytic hierarchy process 387 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.533 with ahp, could improve everyone’s cognitive flexibility, which is one of the top ten future skills needed (montreal gazette, 2016). facilitating a group to get them to think together is an important skill to have because organizations need to mobilize their members to carry out the coordinated actions that will be necessary to make their common larger vision a reality. the ahp/anp offers an approach that can help a group facilitator design and systematically apply a set of techniques to stimulate an effective group process. the ahp/anp is a paradigm shifting theory because of its approach to measuring priority. its use by people who master the ahp/anp and have group facilitating skills would enable them to strive for highly contextual global policies. academics could design prototype ahp studies to inspire, promote, and support leaders and practitioners to implement them in their own real situations. 3. summary the book, group decision making: drawing out and reconciling differences, offers an overall reference to use for ahp/anp and for group decision making in general. writing it grew my interest in growing my competence in designing and facilitating collective thinking processes for a variety of contexts and purposes. it inspired me to broaden and enrich my personal knowledge and mastery. i see ahp as the trigger and the means for my personal growth. key contents of the book: 1. it explains the need for ahp, with its structured approach, to be able to deal with complexities (both external and internal) as well as interdependence of factors which is often underestimated in decision making. it suggests how to introduce ahp to an organization, and how to ensure a valid and useful outcome. 2. the ahp concepts are covered in depth including its fundamental breakthrough idea of a new kind of scale, an absolute relative ratio scale, it’s use in making tacit comparisons (with validation examples), and its consistency measure for determining the coherence of the individual decision makers. explanations of key issues such as rank preservation and reversal as well as dispersion of individual judgments in a group are covered. the generalization from ahp to anp and from discrete to continuous functions are explained. there is an explanation about what the consistency index indicates, showing that seven elements is the best cut off for the number of elements to be compared to ensure that the answer will be sensitive enough to ferret out which are the inconsistent judgments. it is assumed that the readers are familiar with the axioms of the ahp and their implications when applying it. 3. there are examples of a wide range of ahp applications from simple to complex problems, including its sophisticated approach for conflict resolution and the use of the benefits-opportunities-costs-risks (bocr) framework. one ahp application that deserves a book of its own to explain is the unique approach of analytical planning (forward-backward planning).the silverlake project, a sophisticated application by ibm, shows how a series of interdependent ahp models was constructed then used to allocate resources for the thousands of items needed for its ijahp article: peniwati/group decision making: drawing out and reconciling differences international journal of the analytic hierarchy process 388 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.533 new as400 computer, which went on to become one of the most acclaimed computers of its day. 4. there are comprehensive ideas and recommendations for applying ahp in group decision making, not only about how to prevent number crunching by aggregating judgments the right way, but also about how to organize individuals in a group to ensure a satisfactory group outcome. specifically, the formal approach for aggregating individual judgments into a representative group judgment is explained in detail along with tips on how to use the inconsistency measures as the indicator to find and reconcile differences among the members of the group. ijahp article: peniwati/group decision making: drawing out and reconciling differences international journal of the analytic hierarchy process 389 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.533 references arrow, k. j. (1951). social choice and individual values. new york: john wiley & sons. doi: https://doi.org/10.1007/978-3-531-90400-9_6 bristol bay chignik planning workgroup, (2005). strategic plan for the subsistence fisheries resource monitoring program, southwest region, bristol bay and chignik areas, office of subsistence management, fisheries information services division, anchorage, alaska. de bono, e. (1990). new thinking for the new millennium. uk: penguin books, limited. doi: https://doi.org/10.1016/s0024-6301(99)00110-7 eyrich, h.g., (1994). benchmarking to become the best of breed, manufacturing systems, 9(4), 40-47. montreal gazette, (2016, jan. 20). t h e t o p 1 0 j o b s k i l l s f o r t h e f u t u r e , 2 0 2 0 t o b e e x a c t . m o n t r e a l g a z e t t e r e t r i e v e d f r o m http://montrealgazette.com/storyline/the-top-10-job-skills-for-the-future-2020-to-be-exact saaty, t. l., & peniwati, k. (2008). group decision making: drawing out and reconciling differences. pittsburgh, pa: rws publications. the silverlake project transformation at ibm, (1992). new york, ny: oxford university press. book available as an ebook from: https://www.amazon.com/s/ref=nb_sb_noss/131-6172906-3394917?url=searchalias%3daps&fieldkeywords=group+decision+making%3a+drawing+out+and+reconcilinig+differences https://doi.org/10.1016/s0024-6301(99)00110-7 http://montrealgazette.com/storyline/the-top-10-job-skills-for-the-future-2020-to-be-exact https://www.amazon.com/s/ref=nb_sb_noss/131-6172906-3394917?url=search-alias%3daps&field-keywords=group+decision+making%3a+drawing+out+and+reconcilinig+differences https://www.amazon.com/s/ref=nb_sb_noss/131-6172906-3394917?url=search-alias%3daps&field-keywords=group+decision+making%3a+drawing+out+and+reconcilinig+differences https://www.amazon.com/s/ref=nb_sb_noss/131-6172906-3394917?url=search-alias%3daps&field-keywords=group+decision+making%3a+drawing+out+and+reconcilinig+differences ijahp essay: mu/what if classroom instruction becomes a thing of the past? international journal of the analytic hierarchy process 355 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.805 what if classroom instruction becomes a thing of the past? enrique mu abstract until recently, there was no doubt about what constituted a university education and how it was carried out. suddenly, the covid-19 pandemic occurred, and in a few weeks, not only education, but the entire world changed. in the new normal, post-pandemic world, it is possible that teaching face-to-face courses will be the exception, not the rule, in the u.s. and the latin american and caribbean regions. furthermore, this virtual instruction will possibly be at massive levels with tens or hundreds of thousands of students at a time, modeled after massive open online courses (moocs). virtual instruction has been present in some form in most institutions of higher education in recent years and its growth has been systematically increasing. in the u.s., the growth in the number of higher education students enrolled exclusively in online courses has been increasing (4% annually for undergraduate students and 7.4% for postgraduate students from 2017-2018). this growth has been even greater in many parts of latin america and the caribbean. an article in el espectador entitled “an increasingly less physical education” from january 29, 2018, reported that the growth rate of the virtual modality was on the order of almost 99% in colombia. in argentina, distance education has a long history, and currently, institutions such as the universidad argentina de la empresa (uade) have the virtual modality as an important part of their educational offerings. before the pandemic, higher education institutions had already implemented some forms of distance learning for certain courses. however, despite constant growth in this modality, as previously explained, the vast majority of these institutions made only a small fraction of their courses available in a virtual format with the exception of certain institutions that specialized in distance education. covid-19 ended all resistance to distance learning in just a few weeks. teachers had no choice but to convert their courses to an online format because, unless a course was converted to a virtual format, it could not otherwise be taught. while there will still be discussions about the effectiveness of virtual delivery after the quarantines are over, the reality is that almost all courses are now available and can be taught online. in the u.s., educational institutions have had to ask themselves whether it is ethical to force students to physically rejoin classes due to the risk of contagion (in addition to considering the possibility of being prosecuted in the event of death by contagion of some student). the answer seems to be that universities are going to offer courses in both face-to-face and virtual modalities and let the students (and teachers) choose at their own risk. although the sudden change to virtual instruction surprised many institutions in the u.s. that were not prepared to take up the challenge, latin america and the caribbean have had more difficulty adapting to this new format. the biggest change in education could ijahp essay: mu/what if classroom instruction becomes a thing of the past? international journal of the analytic hierarchy process 356 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.805 occur due to the sudden acceptance and respectability of massive and open online courses (moocs). in 2011, stanford university offered an artificial intelligence course online for free which enrolled 160,000 students from around the world. this greatly impacted one of the teachers, sebastien thrun, and led him to found udacity, the first company to offer this type of course. there are currently many companies such as coursera, udemy, and consortia of higher education institutions such as edx that offer mooc courses. by providing instruction to hundreds of thousands of students around the world, these companies achieve impressive economies of scale and can deliver modern, personalized courses and programs in a short period of time. until recently, the willingness to give credit for the coursera mooc course, "human-computer interaction" offered by the university of helsinki in 2012 was considered a finnish rarity. however, many wellrespected universities have also started producing their own for-credit moocs in the u.s., latin america and the caribbean region. in the u.s., mit created the non-profit mitx in 2012. harvard, as well as other institutions such as the university of california, berkeley, the university of texas system, boston university, and others, joined the group which has been renamed edx. this shows that large institutions that have a lot of credibility can join forces to produce moocs. in latin america, the university of sao paulo in brazil (2013) in association with coursera, and the autonomous university of mexico (2019) in association with veduca, have also started offering moocs. why is this important? a mooc approach offers the ideal financial model for higher education institutions (the cost per student is very low) and for students (the cost of tuition for the course is very low). a sign of the acceptance of moocs is that several of these mooc providers, for example edx, have even started offering degrees at the graduate level for very affordable tuition prices. the emergence of moocs could change the role of teachers as well. if a comparison were made between the theater (where the play is the final product) and the academy, it could be said that in the current educational model, teachers are the producer, screenwriter, stage director and main actor, all in one. this model is no longer sustainable. students have become clients, and similar to their demand for attractive and engaging theater works, they demand educational courses that meet their specific individual interests, in an entertaining way, and at very low tuition costs. an optimistic view sees the role of faculty as that of expert generalists who, similar to a conductor, ensure the harmonious performance of task specialists. a pessimistic view sees the primary role of teachers as that of screenwriters or, if they do not work with the original script, as script adapters or the curators of academic material. hopefully, some teachers will still be able to teach their own content, or play the role of the lead actor in the educational theater. however, producer organizations are likely to decide who is a "marketable" instructor, or perhaps create, through artificial intelligence techniques, appropriate digital versions of teachers to provide perfect instruction. great teachers are still needed to impact students in a way that no algorithm can, and therefore, face-to-face instruction will be necessary, but it will be the final measured detail of education, not the workhorse. that is, the time when all higher education students received face-to-face instruction, most of the time, in most courses will be a memory of the past. ijahp article: gul, topcu / a multi-attribute decision support model for the selection of touristic activities international journal of the analytic hierarchy process 560 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.354 a multi-attribute decision support model for the selection of touristic activities sait gul halic university, turkey saitgul@halic.edu.tr ilker topcu istanbul technical university ilker.topcu@itu.edu.tr abstract people who wish to travel or participate in a touristic activity often do not have certain information about available travel destinations, group tours, and touristic events. furthermore, they have their own personal expectations and preferences, especially regarding time and budget limitations. therefore, they do not want to spend their limited time collecting information about travelling instead of actually travelling. besides, the individualistic dimensions of tourism planning and marketing studies have a significant importance on national economies all over the world, particularly for nations whose tourism income is becoming a bigger share of their total national income. this study aims to develop a touristic suggestion model for tourist candidates with regards to their personal expectations and preferences about tourism. the analytic hierarchy process (ahp) and technique for order preference by similarity to ideal solution topsis multi-attribute decision-making methods are used in this study to analyze the problem. the proposed model was built in three main phases: structuring, modeling and analyzing. the ahp method was used for prioritizing the related criteria obtained from the tourist candidates, and then topsis was used for assessing global preference of alternatives. finally, a recommendation to the decision maker is made with the most appropriate alternative. keywords: touristic preferences; touristic decision factors; touristic activity alternatives; ahp; topsis 1. introduction: tourism and tourism planning the notion of tourism can be defined in two different ways. the individual meaning includes travelling with the intention of resting, being amused, sightseeing, recognition, and so forth. the social meaning includes economic, cultural and technical endeavors in a region or country to attract tourists (turkish language institution, 2012). a tourist is a person who fits the individual definition of the word described above. because a tourist is a person who performs the touristic activity de facto, we use the notion of “tourist candidate” in this paper. our decision maker is not the tourist; he/she is a candidate who wishes to travel places that he/she is not familiar with. mailto:saitgul@halic.edu.tr mailto:ilker.topcu@itu.edu.tr ijahp article: gul, topcu / a multi-attribute decision support model for the selection of touristic activities international journal of the analytic hierarchy process 561 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.354 tourism activity is the tourist’s visit to an area or participation in the activities which are organized in regions where the tourist’s knowledge is limited or does not exist. another name for touristic activities in the literature is tourist attraction. lew (1987) states that without tourist attractions there would be no tourism and without tourism there would be no tourist attractions. in essence, tourist attractions consist of all elements of a non-home place that draw discretionary travelers away from their homes. they usually include landscapes to observe, activities to participate in and experiences to remember. tourist candidates usually have limited information regarding alternative touristic activities from which to choose. in addition to this scarcity of information, each tourist candidate has distinct preferences, expectations, and time and budget limitations. also, they do not want to spend their restricted time collecting information about touristic attractions. it is necessary for a touristic attraction that is being suggested to be appropriate to the tourist candidate’s expectations and preferences because of the nature of tourism (regarding its features of recreation, entertainment and leisure time). this means that tourism planning is a personalized or individualized activity and should be approached individualistically. tourism planning also has social and economic meaning. the service industry is considerably important for many countries, and tourism is a very important part of it. tourism income can be crucial particularly for underdeveloped and developing countries that possess limited raw materials and production resources, or that suffer from economic, political or technological difficulties. nevertheless, tourism can also be a very important component of a national economy which already has manufacturing competences and proficiencies. the tourism industry has a sensibility that needs to be paid attention to in terms of both national economies and industrial constituents because tourism companies (agencies, hotels, suppliers, etc.) are important parts of national economies in terms of foreign exchange entering the country. the authors of this paper, acting as decision analysts, developed a recommended model for the selection of touristic activities considering tourist candidate’s expectations, preferences, time, and budget limitations. the aim of this model is to reduce or eliminate the required time spent collecting information about touristic activity alternatives, and to find the best fit between tourist candidate’s expectation and suggestions of touristic activities (or selection) that are provided. 2. literature review forty-four papers were examined and classified in this study, and, a taxonomy was built that divided the research into several classes in terms of their concepts, purposes, methods and technological resources used and inclusion of cognitive processes (table 1). the first part of the taxonomy included papers introducing mathematical models for tour planning and routing; these studies rarely have common traits. the methods utilized are linear programming (kurata, 2009, 2010), mixed integer linear programming (zhu et al., 2010), partial solved constraint satisfaction (seifert, 2007b), constraint network (seifert et al., 2010), orienteering and team orienteering problem (vansteenwegen et al., 2011), arc orienteering problem (souffriau et al., 2011) and a special mathematical method aiming to determine a fitness function for genetic algorithm processing (maruyama et al., 2004; brilhante et al., 2015; lee et al., 2009; garcia et al, 2011). ijahp article: gul, topcu / a multi-attribute decision support model for the selection of touristic activities international journal of the analytic hierarchy process 562 vol. 7 issue 3 20115 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.3544 table 1 taxonomy of literature on tourism research m a t h e m a t i̇c a l m o d e l in g goal method and technology used c la ss if ic a ti o n t o u ri st ic p ri o ri ti z a ti o n r o u ti n g d e v e lo p in g in d iv id u a li st ic t o u rs d e v e lo p in g g ro u p t o u rs m c d m v e c to r s im il a ri ty a rt if ic ia l in te ll ig e n c e g e n e ti c a lg o ri th m s h e u ri st ic m e th o d s u se r in v o lv e m e n t c o m p u te r t e c h n o lo g ie s m a p s s ta ti st ic s a p p l ic a t io n c o g n it iv e p r o c e s s e s zhu vet al. (2010) x x x x x x kurata (2009) x x x x x x kurata (2010) x x x x x x x x alptekin and büyüközkan (2010) x x x x x seifert et al. (2007) x x x x x seifert (2007a) x x x x x x seifert (2007b) x x x x x x x x tenbrink and seifert (2010) x x x x x huang and bian (2007) x x x x x x x niaraki and kim (2009) x x x x x x x yim and diğ. (2004) x x x x x x souffriau and vansteenwegen (2010) x x vansteenwegen et al. (2011) x x x x x x x souffriau et al. (2011) x x x x x x tomai et al. (2005) x x x x x x x ricci and werthner (2002) x x x lew (1987) x x richter et al. (2004) x x x schiefelbusch et al. (2007) x x x lew and mckercher (2006) x x maruyama et al. (2004) x x x x x x x garcia et al. (2010) x x x x x x ijahp article: gul, topcu / a multi-attribute decision support model for the selection of touristic activities international journal of the analytic hierarchy process 563 vol. 7 issue 3 20115 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.3544 table 1 (continued) taxonomy of tourism research m a t h e m a t i̇c a l m o d e l in g goal method and technology used a p p l ic a t io n c o g n it iv e p r o c e s s e s c la ss if ic a ti o n t o u ri st ic p ri o ri ti z a ti o n r o u ti n g d e v e lo p in g in d iv id u a li st ic t o u rs d e v e lo p in g g ro u p t o u rs m c d m v e c to r s im il a ri ty a rt if ic ia l in te ll ig e n c e g e n e ti c a lg o ri th m s h e u ri st ic m e th o d s u se r in v o lv e m e n t c o m p u te r t e c h n o lo g ie s m a p s s ta ti st ic s kang et al. (2006) x x x x x x srisuwan and srivihok (2007) x x x x x x nagata et al. (2006) x x x x x x x x schiaffino and amandi (2007) x x x x x x su and chang (2010) x x x x x shoval and raveh (2004) x x x x um and crompton (1990) x x x x lorenzi et al. (2011) x x x x x koceski and petrevska (2012) x x x x x x x x batet et al. (2012) x x x x x x x lucas et al. (2013) x x x x x x kurata and hara (2013) x x x x x lee et al. (2009) x x x x x x x x garcia et al. (2011) x x x x x x x moreno et al. (2013) x x x x x x x mutinda and mayaka (2012) x x x cheng et al. (2013) x x x x bifulco and leone (2014) x x x x x x umanets et al. (2014) x x x x x x x büyüközkan and ergün (2011) x x x x x x brilhante et al. (2015) x x x x x x yeh and cheng (2015) x x x x x x x ijahp article: gul, topcu / a multi-attribute decision support model for the selection of touristic activities international journal of the analytic hierarchy process 564 vol. 7 issue 3 20115 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.3544 the second part of taxonomy classified the papers in terms of their goals. this classification built five different groups of goals. • classification: cognitive layers in decision making (tenbrink & seifert, 2010), operations research techniques in tour planning (shoval & raveh, 2004; souffriau and vansteenwegen, 2010; vansteenwegen et al., 2011), classification of touristic attractions (lew, 1987; mutinda & mayaka, 2012), divergence distance from accommodation region (lew & mckercher, 2006), sustainable tourism planning methods (schiefelbusch et al., 2007; cheng et al., 2013), and tourist attitudes (um & crompton, 1990). • touristic prioritization: tour or touristic activity suggestions considering tourists’ preferences (tomai et al., 2005; kang et al., 2006; nagata et al., 2006; huang & bian, 2007; schiaffino & amandi, 2007; kurata, 2009, 2010; niaraki &kim, 2009; alptekin & büyüközkan, 2010; garcia et al., 2010; büyüközkan & ergün, 2011; garcia et al., 2011; lorenzi et al., 2011; batet et al., 2012; koceski et al., 2012; kurata & hara, 2013; lucas et al., 2013; moreno et al., 2013; umanets et al., 2014; brilhante et al., 2015; yeh & cheng, 2015). • tour planning in terms of human cognitive decision making (richter et al., 2004; lew & mckercher, 2006; seifert, 2007b; seifert et al., 2007) or in terms of intercity transportation modes and so on (yim et al., 2004; nagata et al., 2006; niaraki & kim, 2009; lee et al., 2009; su & chang, 2010; souffriau et al., 2011; vansteenwegen et al., 2011; kurata and hara, 2013; bifulco and leone, 2014). • individualistic tour planning aims to determine a touristic tour plan just for an individual tourist candidate (ricci & werthner, 2002; maruyama et al., 2004; tomai et al., 2005; kang et al., 2006; huang and bian, 2007; schiaffino & amandi, 2007; seifert, 2007a, 2007b; seifert et al., 2007; sirisuwan and sirivihok, 2007; kurata, 2009, 2010; alptekin & büyüközkan, 2010; garcia et al., 2010; tenbrink & seifert, 2010; zhu et al., 2010; souffriau et al., 2011; büyüközkan and ergün, 2011; garcia et al., 2011; lorenzi et al., 2011; batet et al., 2012; koceski et al., 2012; kurata & hara, 2013; lucas et al., 2013; moreno et al., 2013; umanets et al., 2014; brilhante et al., 2015; yeh and cheng, 2015). • group tour planning aims to determine a touristic tour plan for groups of tourist candidates (nagata et al., 2006; koceski et al., 2012). the third part of the taxonomy was created according to the methods and technological resources the papers used. the methods varied from multi-criteria decision-making techniques and statistics to the intelligent methods like artificial intelligence and genetic algorithms. the sub-components of the technological resources are the usage of computers (computer programming software included) and maps. with respect to their concept, aim and scope, only five papers did not use the methods or technologies mentioned above (lew, 1987; lew & mckercher, 2006; schiefelbusch et al., 2007; mutinda & mayaka, 2012; cheng et al., 2013). the studies utilized from two to three methods in several harmonized ways instead of just one method while accepting its advantages and disadvantages. in this way, a method’s inadequate and failing feature(s) can be removed by another adapted method(s). meanwhile, it should be noted that no ijahp article: gul, topcu / a multi-attribute decision support model for the selection of touristic activities international journal of the analytic hierarchy process 565 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.354 paper which proposed a methodology for touristic attraction using merely multi-criteria decision-making method(s) could be found in the tourism research literature. the fourth part of the taxonomy is the study’s inclusion of application regarding the proposed models or methods. the majority of the literature introduced applications showing the model’s functionality and practicality. for example, while moreno et al. (2015) designed a recommendation system (sigtur/e-destination) applied to spain’s most attractive tourist destinations, yeh and cheng (2015) applied their model in taiwan. similar touristic attraction recommendation models are applied by tomai et al. (2005), büyüközkan and ergün (2011), umanets et al. (2014), bifulco and leone (2014) etc. the last part of the taxonomy included the papers concerning research about tourist’s cognitive processes, attitudes and behaviors. for example, seifert et al. (2007), seifert (2007a; 2007b) and tenbrink & seifert (2010) examine routing problems of tourists having partial information about touristic destinations and they took behavioral decision making principles and tourist’s cognitive processes into account while recommending alternative routes. 3. decision making: ahp & topsis decision making can be described as a selection process of a possible alternative from among whole alternatives, while the selected alternative should be the most convenient to the decision maker’s goal or goals (evren and ulengin, 1992). topcu (2000) described decision making as problem solving with selecting, ranking, or sorting available decision options (alternatives). in general, decision making can be accepted as a conscious process that intends to select, apply, and control the best course of action in order to achieve a goal. a problem needs two different features to be qualified as a decision-making problem (ozden, 1989): “the presence of multiple courses of action (alternatives)” and “differing results of each course of actions from the others.” keeney (1982) demonstrated 12 factors contributing to the complexity of decision problems which are as follows: multiple objectives, difficulty of identifying good alternatives, intangibles, long-time horizons, many impacted groups, risk and uncertainty, risks to life, interdisciplinary substance, several decision makers, value tradeoffs, risk attitude of decision makers and sequential nature of decisions. these difficulties and the complexity encountered in decision making make the decision process a vital managerial problem that needs to be solved attentively. decision making is a complex process that can be affected from different and discrete factors, or it can have multiple different goals. the decision process usually takes into account different courses of action or alternatives. saaty (2004) stated that “decision making involves criteria and alternatives to choose from. the criteria usually have different importance and the alternatives in turn differ in our preference for them on each criterion. to make such tradeoffs and choices we need a way to measure. measuring needs a good understanding of methods of measurement and different scales of measurement.” thus, we can exploit multi-criteria decision making (mcdm) methods to achieve this measurement. mcdm can be defined as the evaluation of alternatives for the purpose of selection or ranking, using a number of qualitative and/or quantitative criteria that have different measurement units (ozcan, celebi, & esnaf, 2011). mcdm methods ijahp article: gul, topcu / a multi-attribute decision support model for the selection of touristic activities international journal of the analytic hierarchy process 566 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.354 can be used to keep the decision process under control and acquire decision results quickly and easily when there are several alternatives and criteria (heriscakar, 1999). the most important issue in mcdm is that the measurements being used include the decision maker’s personal preferences and subjectivity levels of these preferences; that is, the preferences and expectations differ from one decision maker to another. thus, the decision result or output will be defined according to the decision maker’s goal and opinion because the importance of each decision factor is determined by the decision maker’s preferences and expectations. in this paper we aim to develop a touristic activity suggestion model depending on the tourist candidate’s expectations, preferences, and time and budget boundaries. therefore, this model has 33 preference factors that should be considered in touristic decision problems. the ahp is used for prioritizing the related criteria obtained from the tourist candidates, and then topsis is used for assessing the global preference of alternatives. the matrix possesses two main information constituents which include the importance of the factors presenting the decision maker’s opinions, expectations and limitations and the points of each alternative which will be collected separately from tourism experts and agencies depending on each decision factor. 3.1 the analytic hierarchy process (ahp) plenty of decision-making methods have been developed in the literature. the analytic hierarchy process (ahp), developed by saaty in the 1970s, is one of the most preferred of these methods, and has a wide area of usage in decision-making problems. ahp relies on the prioritization of decision alternatives by considering and analyzing the factors that can possibly affect the decision. the ahp is a theory of relative measurement on absolute scales of both tangible and intangible criteria based on paired comparison judgment of knowledgeable experts (ozdemir & saaty, 2006). it includes the decision maker’s personal experience, information, preferences and expectations in the decision process. thus, it may be said that the ahp is a powerful decision-making methodology to determine priorities among different criteria (isiklar & buyukozkan, 2007). besides the individualization characteristic, ahp is a suitable method for group decision making analyses according to the features of problem. however, the method needs some modifications to make it convenient for group decision making in order to provide consistency between integrated decision points; for example, calculating the weighted means of comparison points of multiple decision makers (saaty, 1999). the ahp includes five basic steps summarized as follows: i. building the hierarchy of the decision model: first of all, the problem should be described and exhibited clearly. ahp methodology needs the decision model to be decomposed into a hierarchy including a goal (touristic activity selection in this study) at the top level, decision criteria (33 different factors, e.g. cost, type of activity, or type of accommodation) and sub-criteria (e.g. thematic activities, sporting activities, or resting) at the following level and the decision alternatives (10 selected touristic activity, e.g. cappadocia, blue cruise, or cultural treasures of central european) at the bottom level of the hierarchy. a common hierarchy is given in figure 1. ijahp article: gul, topcu / a multi-attribute decision support model for the selection of touristic activities international journal of the analytic hierarchy process 567 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.354 figure 1. general framework of decision hierarchy ii. building the comparison matrices: the ahp helps the decision maker to reflect on his/her personal thoughts, emotions, and preferences in the decision process. the method makes this possible by building the factor comparison matrices or the alternative comparison matrices, according to the factors, separately. the decision maker can use the fundamental 1–9 scale as shown in table 2 which is was designed to assess the priority scores: 1 indicates equal importance, 3 moderately more, 5 strongly more, 7 very strongly, and 9 indicates extreme importance (saaty, 1980). these points show the importance assessments between two compared objects and the cells in the matrix will be completed using these points. we should build an “n × n” matrix sorting the row and column elements with the same array while “n” factors or alternatives were being compared. then, the following rules are applied. table 2 basic prioritization scale numerical assessment linguistic meaning 1 equal important 3 moderately more important 5 strongly more important 7 very strongly important 9 extremely more important 2, 4, 6, 8 intermediate values of importance reverse to the multiplication if element i is more important than element j, this is the importance points of element j to the element i. aij is the relative importance point of element i with respect to element j. in this instance, equation 1 represents a as the comparison matrix. a = (aij) aij > 0 i,j=1, 2, …, n (1) aji is the relative importance point of element j with respect to element i. this gains its importance from the “reverse to the multiplication” value of aij (saaty, 1980). aji = 1 / aij aij > 0 i,j=1, 2, …, n (2) ijahp article: gul, topcu / a multi-attribute decision support model for the selection of touristic activities international journal of the analytic hierarchy process 568 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.354 hence, if the decision-making problem consists of n criteria and m alternatives, the decision matrix takes the form as given in equation 3. iii. determining the factor’s importance and alternative’s preference points: after the comparison matrices have been built, the matrix should be converted to an eigenvector in order to determine the percentage of the importance grade for all elements. in an eigenvector, the sum of the importance for all elements is 100% (saaty, 1980). a = ( 1 a12 … a1n 1 a12 1 … a2n … … … … 1 a1n 1 a2n … 1 ) (3) iv. calculating the consistencies: ahp also calculates a consistency ratio to reflect the consistency of the decision maker’s judgments during the evaluation phase. in this way, ahp methodology aims to control the decision process. it needs this control operation because the decision maker’s opinion and preferences cannot be controlled easily. if the equation “aij x ajk = aik” is valid for all i, j, and k elements, we can accept that the “a” matrix is consistent (taha, 2007). basically, a decision process cannot be totally consistent. for this reason, the existence of an acceptable inconsistency should be determined. ahp calculates a consistency ratio (cr) and requires that the ratio should not exceed the 10% threshold. the closer the consistency ratio is to zero, the greater the consistency. if a convenient ratio cannot be obtained, the decision maker can go back to the second step and redo the calculations of comparison. v. determining the alternative’s preference points and making a decision: the decision maker decides to choose an alternative in this step. the matrices which were found consistent will be synthesized as follows, in order to help the decision maker and to conclude numerical preference values for each alternative. a. each column matrix showing the preference of alternatives is inserted into a new matrix’s column. sorting of this column insertion should be at the same sequence of factors. b. an integrated matrix consisting of the preferences of all alternatives and the column matrix consisting of the importance of factors are multiplied. each alternative’s global preference point is determined as a result of this multiplication. c. when the preferences are sorted in descending order, an order of preference regarding the alternatives is formed. the alternative that has the biggest weight as a result of ahp indicates the most appropriate one according to the decision maker’s preferences. 3.2 technique for order preference by similarity to ideal solution (topsis) the technique for order preference by similarity to ideal solution (topsis) method was developed by hwang & yoon (1981) with the aim of supporting the decision maker and easing their decision processes. the method is based on the interpretation of the ijahp article: gul, topcu / a multi-attribute decision support model for the selection of touristic activities international journal of the analytic hierarchy process 569 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.354 decision problem, which is illustrated on a geometric plane and includes n evaluation criteria and m decision alternatives. the logic behind the method is that m points should be represented in n-dimensional space (yoon & hwang, 1995). the topsis method is based on choosing the best alternative having the shortest distance to the ideal solution and the farthest distance from the negative-ideal solution. the ideal solution (a*) is the solution that maximizes the benefit and also minimizes the total cost. on the contrary, the negative-ideal solution (a-) is the solution that minimizes the benefit and also maximizes the total cost (işıklar & büyüközkan, 2007). equations 4 and 5 denote the mathematical context of the ideal and negative-ideal solutions. a * = (x1 * , x2 * , … , xn * ) xj * : the value of the best alternative for the j th factor (4) a = (x1 , x2 , … , xn ) xj : the value of the worst alternative for the j th factor (5) topsis method has six steps as follows (yoon & hwang, 1995): i. calculation of the normalized values: topsis begins with normalizing the factors in the decision matrix in order to make the factor convenient for the comparison. equation 6 gives the normalized value (rij) of the decision element aij. 𝑟𝑖𝑗 = 𝑎𝑖𝑗 √∑ 𝑎𝑖𝑗 2𝑚 𝑖=1 2 (6) ii. building the weighted normalized decision matrix: each evaluation factor (decision criterion) has different importance that shows the factor’s degree of influence. a performance value of an alternative with respect to a factor which can be seen in the row representing that alternative and in the column representing the factor is multiplied with the factor’s weight (wj). thus, after all the multiplication operations are done, the weighted normalized decision matrix has been built. equation 7 presents the calculation of the weighted normalized values. the sum of the importance of these (wj) should be 1. 𝑣𝑖𝑗 = 𝑤𝑗 𝑥 𝑟𝑖𝑗 (7) iii. determining the ideal and negative-ideal solutions: the ideal solutions are determined by considering the aforementioned information. this process is like that presented in equation 8 for the ideal solution and equation 9 for the negative-ideal solution. cluster j1 specifies the benefit criteria and cluster j2 specifies the cost criteria. a * = {v1 * , v2 * , … , vn * } ={(max𝑖 𝑣𝑖𝑗 | 𝑗 ∈ 𝐽1), (min𝑖 𝑣𝑖𝑗 | 𝑗 ∈ 𝐽2), 𝑖 = 1,…,𝑚} (8) a = {v1 , v2 , … , vn } = {(min𝑖 𝑣𝑖𝑗 | 𝑗 ∈ 𝐽1), (max𝑖 𝑣𝑖𝑗 | 𝑗 ∈ 𝐽2), 𝑖 = 1,…,𝑚} (9) iv. calculating the separation measures: the separation measures are calculated one by one using the n-dimensional euclidean distance which are given in equation 10 for the separation from the ideal (si * ) and equation 11 for the separation from the negative-ideal solutions (si ). the number of these values is equal to the number of alternatives. ijahp article: gul, topcu / a multi-attribute decision support model for the selection of touristic activities international journal of the analytic hierarchy process 570 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.354 𝑆𝑖 ∗ = √∑ (𝑣𝑖𝑗 − 𝑣𝑖𝑗 ∗ )2𝑛𝑗=1 (10) 𝑆𝑖 − = √∑ (𝑣𝑖𝑗 − 𝑣𝑖𝑗 −)2𝑛𝑗=1 (11) v. calculation of the relative closeness to the ideal solution: the closeness measure which is given in equation 12 is calculated using (si * ) and (si ) values. it is evident that 0 ≤ ci * ≤ 1 is provided and ci * = 0 occurs if ai = a and ci * = 1 occurs if ai = a * . 𝐶𝑖 ∗ = 𝑆𝑖 − 𝑆𝑖 −+𝑆𝑖 ∗ (12) vi. ranking the preference order: the decision alternatives are ranked in descending or ascending order based on the relative closeness measures. the winning alternative has the maximum ci * in topsis. the chosen alternative is then recommended to the decision maker. the details of the proposed model using ahp and topsis are given in the next section. in this model, ahp provides the importance of decision factors and topsis analyzes the decision matrix which includes the importance of factors and the points of the alternatives with respect to the decision factors determined by the tourism experts. topsis is more appropriate to the touristic decision problems because it allows one to take the real assessment points from the well-informed experts while ahp does not. the ahp method can only be operated by the decision maker and needs only the decision maker’s opinion which can be uninformed in this situation. topsis combines the information which is taken separately from the decision maker and the tourism experts in a matrix. ijahp article: gul, topcu / a multi-attribute decision support model for the selection of touristic activities international journal of the analytic hierarchy process 571 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.354 4. touristic activity suggestion model based on combined ahp – topsis methods 4.1 the importance of the problem and the aim of the paper the service industries, e.g. tourism, banking and insurance, have gained a density in recent years for the economies of developed countries because they have moved their manufacturing lines to underdeveloped or developing countries. this choice is based on several motives like low labor cost or tax allowance. the organization for economic cooperation and development (oecd) supports this indication and points out that service industries of affiliated countries have a greater portion in their annual internal revenues, approximately 70%. the revenue portion of tourism, one of the most important service sub-industries, has a large influence on a national economy although its share differs. countries try to attend or arrange international organizations where they have a chance to promote themselves in order to announce their touristic and cultural beauties and raise awareness of their tourist attractions. quality, customer satisfaction, and continuous improvement are vital notions for service companies as with manufacturing companies. thus, planning activities is important within the tourism industry. both countries and tourism firms have to work on increasing their tourism attractions because of high competition in the global tourism industry. in spite of political or economic disagreements between countries and the interdiction of internal or external tourism in some totalitarian countries, people’s “travelling, resting, and discovering” requirements have endless and permanent features. these tourist candidates need a comprehensive tourism guidance because they can be totally uninformed or have inadequate information regarding possible touristic activities. the touristic selection problem has many alternatives and factors which could affect the tourist candidate’s decision. hence, the tourist candidates need help deciding where to go or which activity to attend. huang & bian (2009) argue that a travel plan consists of a number of stages, such as choosing destinations, selecting tourist attractions, choosing accommodations, deciding routes, and so forth. the problem has several criteria to be considered simultaneously which makes it ideal for the use of multiple criteria decision-making methods. 4.2 building the model and the running principles the model was built with three steps: structuring the problem (s), modeling the problem (m), and analyzing the problem (a). figure 2 shows the framework for the model. the abbreviations si, mi and ai show the sub-steps of the problem while the intermittent lines separate the main steps. (s) structuring the problem while a decision problem needs to be structured, the decision maker should clarify all components regarding the problem in detail. the elements of the problem and the relationships among them should be described fairly and clearly for this purpose. the ahp’s hierarchy-based methodology allows the exhibition these relationships within the content of the decision problem as required. ijahp article: gul, topcu / a multi-attribute decision support model for the selection of touristic activities international journal of the analytic hierarchy process 572 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.354 (s1-a) determining the preference factors: the model needs to include the determination factors which the tourist candidates take into account when making a decision because it aims to suggest the most appropriate touristic alternative. the literature and the websites serving the tourism consultancy introduce the factors affecting touristic preferences that can be classified into six types which include type of activity, transportation mode, type of accommodation, season, security level, and cost. (s1-b)determining the alternative restrictive factors: by considering a restriction procedure revealing the tourist candidate’s (decision maker’s) preferences regarding some features of the alternatives we aim to reduce the number of alternatives. this will accelerate the use of the model and shortens the time needed by the system. this is important especially when we consider that the model is operated on the computer because the decision matrix includes and analyzes only the alternatives which the tourist candidate is actually interested in. the available websites provide just this operation to help the tourist candidates. the alternative restrictive factors are as follows: date range, cost range, type of tourist candidate (single, married couple, married with children), and features of alternative (the existence of wi-fi, airport shuttle service, outdoor or indoor swimming pool, guarded parking lots, etc.). in addition, preference factors can be used for this operation, for example the convenience between the tourist candidate’s opinion about the type of activity or transportation mode and what the activity provides. for example, if a tourist candidate wants an activity to involve an outdoor swimming pool, the model won’t contain the alternatives that do not have one. ijahp article: gul, topcu / a multi-attribute decision support model for the selection of touristic activities international journal of the analytic hierarchy process 573 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.354 figure 2. the frame of the proposed touristic suggestion model (s2-a) building the hierarchy with the preference factors: it is vital that determined factors should be combined in a decision matrix in order to be analyzed using ahp with the purpose of calculating the factor’s importance and showing the tourist candidate’s preferences. ahp methodology uses the hierarchic logic to exhibit the main factors with their sub-constituents together in a tree shape, including a goal on the top, the factors and sub-factors on the first and following levels, and the alternatives on the bottom level of the tree. figure 3 presents the hierarchy of the touristic activity selection problem. (s2-b) building the decision matrix: the decision matrix is a structured and operational matrix that allows suggestion of a touristic activity to the individual tourist candidate by combining the importance of touristic preferences and the points showing the expert’s assessments for touristic activity alternatives. the matrix’s rows include the alternatives and their points obtained from the tourism experts with respect to factors in the corresponding columns. the importance of the factors resides in the second row after the row including the factor’s names. the touristic activity suggestion model’s decision matrix is given in table 3. ijahp article: gul, topcu / a multi-attribute decision support model for the selection of touristic activities international journal of the analytic hierarchy process 574 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.354 figure 3. the decision hierarchy of the touristic activity suggestion model (s3) determining the touristic activity alternatives: a touristic activity can be defined as an activity that an individual or a tourist group may join, be attracted to or appreciate; for example tour packages, camping, thematic activities. the activity alternatives should be appropriate to the hierarchy’s type of activity branch. these alternatives could be selected from several extents, for example a region, country, geographic location, or around the world. the scope of the extent of the activity is determined by the tourist candidates, tourism experts, or tourism consultants considering their capacity. for the example application, five domestic and five outbound activities were selected. they can be found in the decision matrix’s first column (table 3). these alternatives have different focuses, for example a skiing focused alternative, a blue tour focused alternative, or a cultural tour in east europe are included. (m) modeling the problem after the determination of the decision problem’s components, a decision model must be built. this phase has two main operations: 1) obtaining the data from the tourist candidate regarding their touristic preferences and the data from the tourism experts regarding their assessment points; 2) completing the decision matrix by putting the data in the proper cells of the matrix. between these two operations, the problem must be analyzed using the ahp. (m1-a) obtaining the tourist candidate’s preference points by the comparison matrices: in this phase, the tourist candidate completes the survey in order to obtain the date which will be used in ahp to calculate the importance of the tourist candidate’s preferences, opinions, and expectations. the survey allows comparison of the decision factors with each other. then these comparison points are entered into the comparison matrices. the consistency of the matrices will be obtained providing that the tourist ijahp article: gul, topcu / a multi-attribute decision support model for the selection of touristic activities international journal of the analytic hierarchy process 575 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.354 candidate compares all the factors and does not void any question in the survey. the results of this phase are comparison matrices which will be analyzed by the ahp in “analyzing the problem (a)”. (m1-b) obtaining the tourism expert’s points regarding the alternatives: all of the factors, except the sixth one which is cost, have a non-quantized character and need assessment by the experts. this will provide each factor with a point that can be used in the numerical analysis. the cost has its real numerical value and can be used directly in the matrix. table 3 by topsis is used to obtain the expert points of the alternatives which will be used in analyzing the decision matrix. all points relating to all alternatives are collected and saved in a database in order to allow them to be used in the following steps of the model. the scoring of the main factors and the sub-factors located on the branches will be done using the measures below. • type of activity: points are determined by selecting 0 or 1, for example if an activity contains a cultural event, it is assigned a 1; otherwise it is assigned a 0 for the cultural activity branch of the type of activity. • transportation mode: points are determined by selecting 0 or 1, for example if an activity can be arrived at by air, it is assigned a 1; otherwise, it is assigned a 0 for the airway branch of the transportation mode. • type of accommodation: points are selected between 0 and 100 showing the service quality of the accommodations, for example a full pension hotel’s service quality may be is assigned an 80, however if the tourist candidate wants to swim in sea and the hotel has some disadvantages concerning its distance to the sea, its point can decline to 70 or 60. • season: points are determined by selecting 0 or 1, for example if an activity is appropriate for the winter season, it is assigned a 1; otherwise, it is assigned a 0 for the winter branch of the season. • security: points are selected between 0 and 100 showing the possibilities of internet connection, and so on or availability and proximity of hospitals, police stations, and so forth. ijahp article: gul, topcu / a multi-attribute decision support model for the selection of touristic activities international journal of the analytic hierarchy process 576 vol. 7 issue 3 20115 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.3544 table 3 the decision matrix type of activity (0-1) transportation mode (0-1) c u lt u r a l a c ti v it y h e a th -r e la te d a c ti v it y f e st iv a ls r e li g io u s o r g a n iz a ti o n g a st r o n o m ic a c ti v it y a q u a ti c s f is h e r y c li m b in g s k ii n g h ik in g h o te ls r e la x a ti o n p e r so n a l c a r b u s s e a w a y a ir w a y t o u r s importance 0,125 0,021 0,204 0,013 0,058 0,006 0,002 0,004 0,001 0,019 0,077 0,01 0,017 0,033 0,095 0,006 0,045 cultural treasures of east europe 1 0 1 0 1 0 0 0 0 0 1 1 0 0 0 1 1 greek islands 1 0 1 0 1 1 1 0 0 1 1 1 0 0 1 0 1 south korea & japan 1 1 1 1 1 0 0 1 0 1 1 1 0 0 0 1 1 heaven on earth zanzibar 1 0 0 0 1 1 0 0 0 1 1 1 0 0 1 1 1 salzburg austria skiing tour 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 1 1 urgup & goreme & cappadocia 1 0 0 1 1 0 0 0 0 0 1 0 1 1 0 0 1 from a to z black sea 1 1 1 0 1 1 1 0 0 1 1 0 1 1 1 1 1 blue cruise 1 0 0 0 1 1 1 0 0 1 1 1 0 0 1 0 1 assos & troy & dardanelle 1 0 0 0 1 0 0 0 0 1 1 0 1 1 1 0 1 iznik sansarak canyon hiking 0 1 0 0 0 0 0 1 0 1 0 0 1 1 1 0 1 ijahp article: gul, topcu / a multi-attribute decision support model for the selection of touristic activities international journal of the analytic hierarchy process 577 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.354 table 3 (continued) the decision matrix type of accommodation (0-100) season (0-1) security (0-100) c o s t ( t l ) m o te l h a lf p e n si o n f u ll p e n si o n a ll i n c lu si v e u lt r a g u e st h o u se s a u tu m n w in te r s p r in g s u m m e r h e a lt h c a r e c a r e o v e r se le e v e g a s s ta ti o n s l o c a l t r a n sp o r ta ti o n c o m m u n ic a ti o n p o ss ib il it ie s importance 0,006 0,006 0,021 0,041 0,03 0,004 0,007 0,002 0,013 0,03 0,028 0,002 0,001 0,008 0,012 0,053 cultural treasures of east europe 0 40 40 90 90 0 0 0 1 1 100 100 100 80 100 1500 greek islands 0 0 0 100 90 0 1 0 1 1 70 40 0 0 100 2200 south korea & japan 0 90 90 100 100 0 0 0 1 1 100 100 100 100 100 6700 heaven on earth zanzibar 0 50 90 90 0 0 1 0 1 1 90 60 60 50 80 4500 salzburg austria skiing tour 0 0 80 80 0 0 0 1 0 0 100 90 60 20 60 1800 urgup & goreme & cappadocia 50 90 90 100 100 60 1 1 1 1 60 80 60 40 60 290 from a to z black sea 50 80 90 100 100 70 1 0 1 1 50 70 40 30 50 780 blue cruise 0 0 80 100 60 0 1 0 1 1 40 40 0 0 80 1750 assos & troy & dardanelle 70 70 80 70 0 30 1 0 1 1 80 90 20 60 90 200 iznik sansarak canyon hiking 0 0 0 0 0 0 1 0 1 1 0 60 50 0 30 75 ijahp article: gul, topcu / a multi-attribute decision support model for the selection of touristic activities international journal of the analytic hierarchy process 578 vol. 7 issue 3 20115 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.3544 • cost: the concerned activity’s real cost will be entered to the matrix. (m1-c) obtaining the tourist candidate’s preferences regarding the alternative restrictive factors: the tourist candidate will be asked to specify his/her preferences about the alternative restrictive factors which are given in the (s1-b) phase. when the tourist candidate selects the range between $750 and $1000 for the cost of the activity, the activities which are priced below $750 and over $1000 will be eliminated and not be included in the decision matrix. thus, the number of alternatives will be reduced and the operation of the model is accelerated. after the m1 step is done, the first phase (a1-a – calculating the importance of the factors using ahp) of the “analyzing the problem (a)” step will be operated and its outputs will be used in the m2-a phase. otherwise, the result of the m1-b and a1-b (determining the appropriate alternatives to the preference of the tourist candidate) phases will be used in m2-b phase. (m2-a) entering the importance to the decision matrix: in the a1-a phase, the importance of the 33 factors is determined by the ahp methodology so that all the factors will be weighted as 1; that is, the sum of the factor importance is 1. in this phase, these numerical values are entered into the “importance” row of the decision matrix (table 3), under the related names of the factors. (m2-b) taking the chosen alternative’s expertise points from the database: the model has determined the alternatives that will be analyzed in the decision matrix after the a1-b alternative elimination phase. in this phase, the model retrieves the expert points of chosen alternatives from the database and enters them into the decision matrix. the output of the “modeling the problem (m)” step is the decision matrix which will be analyzed in the a2 phase (determining the preference order of the alternatives by using topsis). (a) analyzing the problem this step has three phases and includes the ahp and topsis. the result of this step is the decision (touristic activity) suggestion for the decision maker (tourist candidate). (a1-a) calculating the importance of factors using ahp: after gathering the data in the m1-a phase, these data are analyzed by the ahp to calculate importance which shows the degree of the tourist candidate’s preference according to each one of the decision factors. after analyzing the comparison matrices, the importance result should be combined with the logic of ahp. here we give a brief example. the main factor “security” and it sub-factor’s importance are 5.1% for the main factor; 54.4% for the healthcare center; 3.2% for the over-sleeve forces; 3.1% for the gas station; 15.3% for the local transportation and 24% for communication possibilities. now, the importance of the main factor and the sub-factor’s importance should be multiplied one by one in order to allocate the main factor’s weight to the sub-factors. healthcare center: 5.1% x 54.4% = 2.8% over-sleeve forces: 5.1% x 3.2% = 0.2% gas station: 5.1% x 3.2% = 0.1% ijahp article: gul, topcu / a multi-attribute decision support model for the selection of touristic activities international journal of the analytic hierarchy process 579 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.354 local transportation: 5.1% x 15.3% = 0.8% communication possibilities: 5.1% x 24% = 1.2% we have determined the importance of all 33 factors and we are ready to enter them into the decision matrix through the m2-a phase. (a1-b) determining the appropriate alternatives to the preference of the tourist candidate: in the m1-c phase, the model obtained the preferences regarding the restrictive factors. in this phase, the model determines the proper alternatives and sends this information to the m2-b phase in order to retrieve data about the chosen alternatives from the database. (a2) determining the preference order of the alternatives using topsis: the result of the m2 phase is the decision matrix. the model analyzes the matrix in this phase using topsis to order the alternatives according to their fitness to the tourist candidate’s preferences represented with the importance determined by ahp. topsis needs three types of information: the factor’s importance, chosen alternatives, and the alternative’s factor points assigned by the experts and saved in the database. the result of topsis is the sequencing of the alternatives so that the most appropriate alternative is ordered first, and the least appropriate one is ordered last. for this purpose, the model uses ci * (relative closeness to the ideal solution) values to sequence them in descending order. (a3) suggesting the most appropriate alternative to the tourist candidate: the model suggests the biggest c* value having alternative to the tourist candidate as the most appropriate one to their expectations, preferences, opinions, and senses. thus, the model allows the suggestion of a touristic activity for the individualistic tourist candidates. the organization or the website using this model online has a chance to suggest several alternatives as they are ordered, and the ultimate decision may be left to the tourist candidate allowing for the greatest touristic satisfaction. 5. a case study as described in the previous section, the ahp-topsis combined decision suggestion model aims to help lead the tourist candidate, who can be uninformed or have limited information about available touristic activity alternatives, in choosing an activity so that their desired pleasure is maximized. the model needs some information which is obtained from the tourist candidate. these data and expert points were obtained from the person of interest. we have employed the model for an individual tourist candidate as an example. the data obtained from the tourist candidate will be entered into the comparison matrices and the matrices will be analyzed with ahp. this analysis gives the importance of all the decision factors (ahp hierarchy in figure 3). the main factor’s comparison matrix and the importance are given in table 4. first, the determination of the main factor’s importance, then the sub-factors showed as the branch of each main factor, except the “cost”, should be calculated. the ultimate importance was calculated by the multiplication operation of the main factors and their branches, as mentioned previously, and it means that the main factor’s importance weight is distributed to sub-factors one by one. the most important factor was found to be “type of activity” for this tourist candidate. the ultimate importance of the total 33 decision factors are presented in table ijahp article: gul, topcu / a multi-attribute decision support model for the selection of touristic activities international journal of the analytic hierarchy process 580 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.354 5. the importance of these is the numerical representation of the tourist candidate’s preferences and they will be entered into the relevant row of the decision matrix (table 3). table 4 the comparison matrix and the importance of the main factors type of activity transportation mode type of accommodation season security cost importance type of activity 1 7 9 7 9 4 54,10% transportation mode 0,14 1 3 4 7 3 19,61% type of accommodation 0,11 0,33 1 3 3 3 10,79% season 0,14 0,25 0,33 1 1 1 5,16% security 0,11 0,14 0,33 1 1 2 5,06% cost 0,25 0,33 0,33 1 0,5 1 5,29% according to the topsis methodology, the decision matrix was first normalized, then weighted and produced the weighted normalized decision matrix. this matrix analyzed as the method required and si * (the separation from the ideal solution) and si (the separation from the negative-ideal solution) values were calculated. the ci * topsis sequencing point was calculated using these two values, and the results and the preference order of the alternatives are given in table 6. the alternative with the highest value was “from a to z black sea”, and it should be suggested to the tourist candidate as the alternative which is the most appropriate for his/her preferences. these results indicate that this tourist candidate wants accommodations in an allinclusive hotel in the summer season, to attend festivals and cultural activities or rest in the hotel, to be transported by a boat, have a healthcare center near the activity region and does not give great importance to the cost. the black sea alternative will be suggested for this tourist candidate having these preferences. ijahp article: gul, topcu / a multi-attribute decision support model for the selection of touristic activities international journal of the analytic hierarchy process 581 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.354 table 5 the ultimate importance of the decision factors type of activity 54.1% type of accommodation 10.8% thematic activity 77.90% 42.1% motel 5.90% 0.6% cultural 29.80% 12.6% half pension 5.10% 0.6% health 4.90% 2.1% full pension 19.80% 2.1% festivals 48.60% 20.5% all inclusive 37.30% 4.0% religious org. 3.00% 1.3% ultra all inclusive 28.00% 3.0% gastronomic 13.70% 5.8% guest houses 3.90% 0.4% sporting activity 6,00% 3.2% 10.8% aquatics 20.00% 0.6% season 5.2% fishery 5.40% 0.2% autumn 13.20% 0.69% climbing 11.30% 0.4% winter 3.40% 0.18% skiing 4.10% 0.1% spring 25.00% 1.30% hiking 59.20% 1.9% summer 58.40% 3.04% resting 16,10% 8.7% 5.20% hotel 88.90% 7.7% security 5.1% relaxation 11.10% 1.0% healthcare center 54.40 % 2.8% 54.1% oversleeve forces 3.20% 0.2% transportation mode 19.6% gas station 3.10% 0.2% personal car 8.70% 1.7% local transportation 15.30 % 0.8% bus 16.90% 3.3% communication possibilities 24.00 % 1.2% seaway 48.10% 9.4% 5.1% airway 3.20% 0.6% cost 5.3% 5.3% tours 23.10% 4.5% 19.6% table 6 separation measures and topsis points si * si ci * order from a to z black sea 0,0136 0,1326 0,9071 1 greek islands 0,0299 0,1285 0,8112 2 cultural treasures of east europe 0,0477 0,1235 0,7212 3 south korea & japan 0,0587 0,1213 0,6737 4 assos & troy & dardanelle 0,1044 0,0839 0,4456 5 blue cruise 0,106 0,0786 0,4257 6 urgup & goreme & cappadocia 0,1104 0,0761 0,4081 7 heaven on earth – zanzibar 0,1091 0,0741 0,4044 8 iznik sansarak canyon hiking 0,1193 0,0627 0,3444 9 salzburg austria skiing tour 0,1244 0,0459 0,2695 10 ijahp article: gul, topcu / a multi-attribute decision support model for the selection of touristic activities international journal of the analytic hierarchy process 582 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.354 6. conclusions and future research we have chosen to apply the multiple attribute decision-making (madm) methods (ahp & topsis) because the madm methods can be used to quantify the individual (tourist candidate for the problem analyzed) sense, thought and preferences. the madm methods need to determine the decision factors affecting the tourist candidate’s preferences. the literature has furnished 33 factors within six main factors (type of activity, transportation mode, season, cost, security and the type of accommodation). the proposed suggestion model utilizes ahp and topsis; ahp is used for the weighting of the tourist candidate’s preferences and topsis is used to analyze the decision problem represented as a matrix form. the tourist candidate provides the data model required by completing a survey with the aim of comparing the factors and sub-factors compatibly with the ahp methodology. the decision matrix includes two inputs: the importance calculated by the ahp and the assessment points of the alternatives determined by the tourism experts. the topsis method analyzes the decision matrix evaluating the points and importance and combines the tourist candidate’s and tourism expert’s opinions together with regard to planning the touristic activity. this assessment results in an alternative suggestion of a touristic activity which is the most compatible with the tourist candidate’s preferences. furthermore, this model can also be used by an individual tourist candidate or a group of tourists who want the model to suggest an activity which satisfies their combined and complex preferences. the model requires some modifications about data collecting and processing. the literature indicates that the papers that have developed a touristic decision support model include the madm methods in relation with some others, for example genetic programming, fuzzy logic, artificial intelligence, but there is no paper using only madm methods to suggest an activity to the tourist candidates. with this paper, we aimed to fill this gap by exhibiting a decision support model utilizing only a combination of the ahp and topsis methods. after operating the model for the sufficient number of tourist candidates, we have reached three important conclusions. • the model suggests the alternative which has been given the most appropriate points from the biggest importance having factors. thus, the model is consistent and logical because the aim of the model is the determination of the best alternative according to the tourist candidate’s expectations and preferences. • the “cost” decision factor is the most important one for the majority of tourist candidates who used the model. • the importance of the first four factors having the highest value ranges from 45% to 64%, and the arithmetic mean of these values is 53%. according to this result, we can state that the first four factors having the highest importance constitute half of the importance forming the decision. therefore, we can determine that the number of decision factors may be reduced. a broader application of the survey can be useful for the final decision to lessen the number of factors. ijahp article: gul, topcu / a multi-attribute decision support model for the selection of touristic activities international journal of the analytic hierarchy process 583 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.354 the fast development of information and communication technologies and the global spread of the internet and electronic business have changed the structures of industries around the world. the accelerating and synergistic interaction between technology and tourism has brought fundamental changes and created new opportunities for tourism companies in marketing, tourism destination planning and advertising processes. therefore, future research could look at the digitalization of the proposed model which would allow it to be used on the internet by the touristic service providers. the electronic presentations of tourism like this can be named as “e-tourism”. the survey which gathers the tourist candidate’s preferences can be completed through the tourism agent’s website, the mathematical operations of the model can be done in a relational database and the chosen activity can be suggested to the tourist candidate through the internet. with this simplistic model, the tourist candidate has spent very little of his/her limited time using the website while he/she was at home or in the office. additionally, this system could be the first one in its area because the current websites that support the tourist candidate’s decisions restrict the number of alternatives and therefore address only a branch of the proposed model. our model approaches the problem in a holistic way. the second suggestion is the study of the usage of other methods in order to determine their relevance for this decision problem, for example elimination et choix traduisant la realité (electre) or preference ranking organization method for enrichment evaluation (promethee). the application results of the combined methods could be compared and discussed, then the most useful and appropriate one could be implemented in the operation according to the fit between the activity and the tourist’s pleasure obtained from the proposed activity. the third suggestion is the elimination of the individualistic focus of the model. this would allow it to be used for evaluating and planning activity selections and areas and organizing these in a more general way. first, we could implement the survey with a sufficient number of tourists who had participated in the activity in the region, and then we could calculate the geometric mean of these assessment points in the modeling phase of the model in order to be informed about a region or activity. a method like this could be useful for gathering tourist’s opinions and allow the authorities of the region to change the planning processes or orient the investments to align with the tourist’s expectations about the region. the final suggestion is to research the elimination of the individualistic focus of the model and direct it to operate for a group of tourist candidates (a couple, a family, a team, a group of friends, etc.). the change within the data collecting and its processing could be the same as in the previous proposition. each group member could complete the survey and the geometric mean could be useful to determine a collective decision. for the last two suggestions, some modifications would be required to the model. for the third suggestion, the alternative restriction function should be discarded because the model will be used for all the activity selections in the inspected region; that is, we do not need it, or we can add new factors relating to the region’s requirements. for the fourth suggestion, other methods could be used, for example group decision-making techniques or fuzzy logic. these modifications allow the model to be practical for the discussed future goals. ijahp article: gul, topcu / a multi-attribute decision support model for the selection of touristic activities international journal of the analytic hierarchy process 584 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.354 references alptekin, g. i., & büyüközkan, g. (2011). an integrated case-based reasoning and mcdm system for web based tourism destination planning. expert systems with application, 38, 2125-2132. doi:10.1016/j.eswa.2010.07.153 batet, m., moreno, a., sánchez, d., isern, d., & valls, a. (2012). turist@: agent-based personalised recommendation of touristic activities. expert systems with applications, 39, 7319–7329. doi:10.1016/j.eswa.2012.01.086 brilhante, i. r., macedo, j. a., nardini, f. m., perego, r., & renso, c. (2015). on planning sightseeing tours with tripbuilder. information processing and management, 5(2), 1-15. doi:10.1016/j.ipm.2014.10.003 bufilco, g. n., & leone, s. (2014). exploiting the accessibility concept for touristic mobility. procedia – social and behavioral sciences, 111, 432-439. doi:10.1016/j.sbspro.2014.01.076 büyüközkan, g., & ergün, b. (2011). intelligent system applications in electronic tourism. expert systems with application, 38(6), 6586-6598. doi:10.1016/j.eswa.2010.11.080 cheng, q., su, b., & tan, j. (2013). developing an evaluation index system for lowcarbon tourist attractions in china – a case study examining the xixi wetland. tourism management, 36, 314–320. doi:10.1016/j.tourman.2012.10.019 cho, k. t. (2003). multicriteria decision methods: an attempt to evaluate and unify. mathematical and computer modeling, 37(9-10), 1099-1119. doi:10.1016/s08957177(03)00122-5 evren, r., & ülengin, f. (1992). yönetimde çok amaçlı karar verme. teknik üniversite matbaası, gümüşsuyu, i̇stanbul. garcia, a., arbelaitz, o. linaza, m. t., vansteentwegen, p., & souffriau, w. (2010). personalized tourist route generation, icwe-10 proceedings of the 10 th international conferences on current trends in web engineering, springer-verlag, heidelberg, 486497. doi: 10.1007/978-3-642-16985-4_47 garcia, i., sebastià, l., & onaindia, e. (2011). on the design of individual and group recommender systems for tourism. expert systems with applications, 38, 7683–7692. doi:10.1016/j.eswa.2010.12.143 heriscakar, e. (1999). gemi ana makina seçiminde çok kriterli karar verme yöntemleri ahp ve smart uygulaması, gemi i̇nşaatı ve deniz teknolojisi teknik kongresi bildiri kitabı, 240-256, i̇stanbul. huang, y., & bian, l. (2007). a bayesian network and analytic hierarchy process based personalized recommendations for tourist attractions over the internet. expert systems with applications, 36(1), 933-943. doi:10.1016/j.eswa.2007.10.019 http://dx.doi.org/10.1016/j.eswa.2010.07.153 http://dx.doi.org/10.1016/j.eswa.2012.01.086 http://dx.doi.org/10.1016/j.ipm.2014.10.003 http://dx.doi.org/10.1016/j.sbspro.2014.01.076 http://dx.doi.org/10.1016/j.eswa.2010.11.080 http://dx.doi.org/10.1016/j.tourman.2012.10.019 http://dx.doi.org/10.1016/s0895-7177(03)00122-5 http://dx.doi.org/10.1016/s0895-7177(03)00122-5 http://dx.doi.org/10.1016/j.eswa.2010.12.143 http://dx.doi.org/10.1016/j.eswa.2007.10.019 ijahp article: gul, topcu / a multi-attribute decision support model for the selection of touristic activities international journal of the analytic hierarchy process 585 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.354 hwang, c. l., & yoon, k. (1981). multiple attribute decision making: methods and applications. springer-verlag, berlin/heidelberg/new york. doi: 10.1002/zamm.19800600932 işıklar, g., & büyüközkan, g. (2007). using a multi-criteria decision making approach to evaluate mobile phone alternatives. computer standards & interfaces, 29(2), 265-274. doi:10.1016/j.csi.2006.05.002 kang, e. y., kim, h., & cho, j. (2006). personalization method for tourist point of interest (poi) recommendation. kes 2006, part i, lnai 4251, 392-400. doi: 10.1007/11892960_48 keeney, r. l. (1982). decision analysis: an overview. operations research. 30(5), 803-838.doi: http://dx.doi.org/10.1287/opre.30.5.803 koceski, s., & petrevska, b. (2012). empirical evidence of contribution to e-tourism by application of personalized tourism recommendation system. annals of the alexandru ioan cuza university – economics, 59(1), 363–374. doi: 10.2478/v10316-012-0025-6 kurata, y. (2009). challenges in user-adaptive tour planning systems. agile 2009 workshop on adaptation in spatial communication, sfb/tr 8 spatial cognition report series, 19-26. kurata, y. (2010). interactive assistance for tour planning. spatial cognition vii, lnai 6222. 289-302. doi: 10.1007/978-3-642-14749-4_25 kurata, y., & hara, t. (2013). ct-planner4: toward a more user-friendly interactive day-tour planner. in information and communication technologies in tourism 2014, 73– 86. springer international publishing. doi: 10.1007/978-3-319-03973-2_6 lee, c. s., chang, y. c., & wang, m. h. (2009). ontological recommendation multiagent for tainan city travel. expert systems with applications, 36, 6740–6753. doi:10.1016/j.eswa.2008.08.016 lew, a. a. (1987). a framework of tourist attraction research. annals of tourism research, 14, 553-575. doi:10.1016/0160-7383(87)90071-5 lew, a., & mckercher, b. (2006). modeling tourist movements – a local destination analysis. annals of tourism research, 33(2), 403-423. doi:10.1016/j.annals.2005.12.002 lorenzi, f., loh, s., & abel, m. (2011). personaltour: a recommender system for travel packages. in proceedings of the 2011 ieee/wic/acm international conference on intelligent agent technology, iat 2011, lyon, france. doi:10.1109/wi-iat.2011.69 lucas, j., luz, n., moreno, m., anacleto, r., figueiredo, a., & martins, c. (2013). a hybrid recommendation approach for a tourism system. expert systems with applications, 40(9), 3532–3550. doi:10.1016/j.eswa.2012.12.061 http://dx.doi.org/10.1016/j.csi.2006.05.002 http://dx.doi.org/10.1287/opre.30.5.803 http://dx.doi.org/10.2478/v10316-012-0025-6 http://dx.doi.org/10.1016/j.eswa.2008.08.016 http://dx.doi.org/10.1016/0160-7383(87)90071-5 http://dx.doi.org/10.1016/j.annals.2005.12.002 http://dx.doi.org/10.1109/wi-iat.2011.69 http://dx.doi.org/10.1016/j.eswa.2012.12.061 ijahp article: gul, topcu / a multi-attribute decision support model for the selection of touristic activities international journal of the analytic hierarchy process 586 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.354 martel, j. m. (1999). multicriteria decision aid: methods and application. cors – scro 1999 annual conference. windsor, ontario, june 7-9. maruyama, a., shibata, n., murata, y., yasumoto, k. & ito, m. (2004). p-tour: a personal navigation system for tourism. proceedings of the 11 th world congress on its. doi: 10.1109/aina.2004.1283747 moreno, a., valls, a., isern, d., marin, l., & borràs, j. (2013). sigtur/e-destination: ontology-based personalized recommendation of tourism and leisure activities. engineering applications of artificial intelligence, 26, 633–651. doi: doi:10.1016/j.engappai.2012.02.014 mutinda, r., & mayaka, m. (2012). application of destination choice model: factors influencing domestic tourists destination choice among residents of nairobi, kenya. tourism management, 33, 1593-1597. doi:10.1016/j.tourman.2011.12.008 nagata, m., murata, y., shibata, n., yasumoto, k., & ito, m. (2006). a method to plan group tours with joining and forking. seal 2006, lncs 4247. 881-888. doi: 10.1007/11903697_111 niaraki, a. s., & kim, k. (2009). ontology based personalized route planning system using a multi-criteria decision making approach. expert systems with applications, 36, 2250-2259. doi:10.1016/j.eswa.2007.12.053 ozcan, t., celebi, n., & esnaf, s. (2011). comparative analysis of multi-criteria decision making methodologies and implementation of a warehouse location selection problem. expert systems with applications, 38, 9773-9779. doi:10.1016/j.eswa.2011.02.022 ozdemir, m., & saaty, t.l. (2006). the unknown in decision making – what to do about it. european journal of operational research, 174(1), 349-359. doi:10.1016/j.ejor.2004.12.017 ozden, k. (1989). yöneylem araştırması. hava harp okulu yayınları, i̇stanbul. ricci, f., & werthner, h. (2002). case base querying for travel planning recommendation. information technology & tourism, 4, 215-226. doi: http://dx.doi.org/10.3727/109830501108751001 richter, k., f., klippel, a., & freksa, c. (2004). shortest, fastest, -but what next? – a different approach to route directions. beitrage zu den münsteraner gi-tagen, 205-217. saaty, t. l. (1990). how to make a decision: the analytic hierarchy process. european journal of operational research, 48, 9-26. doi:10.1016/0377-2217(90)90057-i saaty, t. l. (1999). decision making for leaders: the analytic hierarchy for decisions in a complex world. 3rd edition, rws publications, pittsburgh. http://dx.doi.org/10.1109/aina.2004.1283747 http://dx.doi.org/10.1016/j.engappai.2012.02.014 http://dx.doi.org/10.1016/j.tourman.2011.12.008 http://dx.doi.org/10.1016/j.eswa.2007.12.053 http://dx.doi.org/10.1016/j.eswa.2011.02.022 http://dx.doi.org/10.1016/j.ejor.2004.12.017 http://dx.doi.org/10.3727/109830501108751001 http://dx.doi.org/10.1016/0377-2217(90)90057-i ijahp article: gul, topcu / a multi-attribute decision support model for the selection of touristic activities international journal of the analytic hierarchy process 587 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.354 saaty, t. l. (2004). decision making – the analytic hierarchy and network processes (ahp/anp). journal of systems science and systems engineering, 13(1), 1-35. doi: 10.1007/s11518-006-0151-5 schiaffino, s., & amandi, a. (2009). building an expert travel agent as a software agent. expert systems with applications, (36), 1291-1299. doi:10.1016/j.eswa.2007.11.032 schiefelbusch, m., jain, a., schafer, t., & müller, d. (2007). transport and tourism: roadmap to integrated planning developing and assessing integrated travel chains. journal of transport geography, 15, 94-103. doi:10.1016/j.jtrangeo.2006.12.009 seifert, i. (2007a). collaborative assistance with spatio-temporal planning problems. spatial cognition v, lnai 4387, 90-106. ddoi: 10.1007/978-3-540-75666-8_6 seifert, i. (2007b). region-based model of tour planning applied to interactive tour generation. human-computer interaction, part iii, hcii 2007, lncs 4552, 499-507. doi: 10.1007/978-3-540-73110-8_54 seifert, i., barkowsky, t. & freksa, c. (2007). region-based representation for assistance with spatio-temporal planning in unfamiliar rnvironments. proceedings of location based services and telecartography, 179-191.doi: 10.1007/978-3-540-367284_14 shoval, n., & raveh, a. (2004). categorization of tourist attractions and the modeling of tourist cities: based on the co-plot method of multivariate analysis. tourism management, 25, 741-750. doi:10.1016/j.tourman.2003.09.005 souffriau, w., & vansteenwegen, p. (2010). tourist trip planning functionalities: stateof-the-art and future. icwe 2010 workshops, lncs 6385, 474-485. doi: 10.1007/978-3642-16985-4_46 souffriau, w., vansteenwegen, p., berghe, g. v., & oudheusden, d. v. (2011). the planning of cycle trips in the province of east flanders. omega, 39, 209-213. doi:10.1016/j.omega.2010.05.001 srisuwan, p., & srivihok, a. (2007). personalized trip information for e-tourism recommendation system based on bayes theorem. ifip international federation of information processing, 255, 1271-1275. doi: 10.1007/978-0-387-76312-5_53 su, j. m., & chang, c. h. (2010). the multimodal trip planning system of intercity transportation in taiwan. expert systems with applications, 37, 6850-6861. doi:10.1016/j.eswa.2010.03.037 taha, h. a. (2007). yöneylem araştırması. literatür yayıncılık (6. basımdan çeviren ş. alp baray ve şakir esnaf), i̇stanbul. tenbrink, t., & seifert, i. (2010). conceptual layers and strategies in tour planning. cognition process, 12(1),109-125. doi:10.1007/s10339-010-0373-9 http://dx.doi.org/10.1016/j.eswa.2007.11.032 http://dx.doi.org/10.1016/j.jtrangeo.2006.12.009 http://dx.doi.org/10.1016/j.tourman.2003.09.005 http://dx.doi.org/10.1016/j.omega.2010.05.001 http://dx.doi.org/10.1016/j.eswa.2010.03.037 ijahp article: gul, topcu / a multi-attribute decision support model for the selection of touristic activities international journal of the analytic hierarchy process 588 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.354 tomai, e., spanaki, m., prastacos, p. & kavouras, m. (2005). ontology assisted decision making – a case study in trip planning for tourism. iotm workshops, lncs 3762, 11371146. doi: 10.1007/11575863_136 topçu, y. i̇. (2000). çok ölçütlü sorun çözümüne yönelik bir bütünleşik karar destek modeli, yayımlanmamış doktora tezi, i̇stanbul teknik üniversitesi fen bilimler enstitüsü, i̇stanbul. um, s., & crompton, j. l. (1990). attitude determinants in tourism destination choice. annals of tourism research, 17(3), 432-448. doi:10.1016/0160-7383(90)90008-f umanets, a., ferreira, a., & leite, n. (2014). guideme – a tourist guide with a recommender system and social interaction. procedia technology, 17, 407-414. doi:10.1016/j.protcy.2014.10.248 vansteenwegen, p., souffriau, w., & oudheusden, d. v. (2011). the orienteering problem: a survey. european journal of operational research, 209(1), 1-10. doi:10.1016/j.ejor.2010.03.045 yeh, d. y., & cheng, c. h. (2015). recommendation system for popular tourist attractions in taiwan using delphi panel and repertory grid techniques. tourism management, 46, 164–176. doi:10.1016/j.tourman.2014.07.002 yim, h. s., ahn, h. j., kim, j. w., & park, s. j. (2004). agent-based adaptive travel planning system in peak seasons. expert systems with application, 27(2), 211-222. doi:10.1016/j.eswa.2004.01.004 yoon, k. p., & hwang, c. l. (1995). multiple attribute decision making – an introduction. sage publications, california. zhu, c., hu, j. q., wang, f., xu, y., & cao, e. (2010). on the tour planning problem. annals of operations research. springer science+business media, llc 2010. http://dx.doi.org/10.1016/0160-7383(90)90008-f http://dx.doi.org/10.1016/j.protcy.2014.10.248 http://dx.doi.org/10.1016/j.ejor.2010.03.045 http://dx.doi.org/10.1016/j.tourman.2014.07.002 http://dx.doi.org/10.1016/j.eswa.2004.01.004 ijahp essay: vargas, zoffer/applying ahp in conflict resolution international journal of the analytic hierarchy process 143 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.649 applying ahp in conflict resolution luis g. vargas university of pittsburgh the joseph m. katz graduate school of business lgvargas@pitt.edu h.j. zoffer university of pittsburgh the joseph m. katz graduate school of business zoffer@katz.pitt.edu the current failure to resolve conflicts worldwide highlights the need for a different approach to conflict resolution. a new international center for conflict resolution (ic4cr), to be housed in the joseph m. katz graduate school of business, was funded by the university of pittsburgh chancellor patrick gallagher in response to a proposal by professors luis vargas and jerry zoffer. the mission of ic4cr is to provide decision makers with an in-depth understanding of the negotiating positions of all parties and recommend implementation guidelines, based on preferences and priorities, to facilitate resolution of otherwise intractable conflicts. we propose to implement this mission by conducting studies of diplomatic and corporate conflicts using the analytic hierarchy process (ahp), building tradeoff models with the following strategies and conducting simulations on the models developed. o eliciting preferences and priorities from the parties through the ahp to create a negotiation model which is measurement based; o using the priorities from the preferences to compute gain/loss ratios of tradeoffs from each party’s perspective; o identifying win-win, non-zero-sum equitable tradeoffs that both parties can claim as a win; o developing a road map to facilitate implementing a feasible solution; jerry zoffer thomas saaty and luis vargas mailto:lgvargas@pitt.edu mailto:zoffer@katz.pitt.edu ijahp essay: vargas, zoffer/applying ahp in conflict resolution international journal of the analytic hierarchy process 144 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.649 an example of what ic4cr can accomplish is the pittsburgh initiative that received over $600,000 of funding over a period of eight years. these funds were raised from private foundations by dean emeritus and professor jerry zoffer. for over seven years, small groups of distinguished israeli and palestinian experts met under the auspices of conflict resolution researchers from the university of pittsburgh. at the core of this privatelyfunded project is the application of an advanced “trade-off” model based on the ahp developed by the late university of pittsburgh distinguished professor thomas l. saaty. the result was a preference-based-priority road map for resolving the israeli-palestinian conflict that was made available to national leaders and decision makers of both parties. a literature review shows the uniqueness of our approach in contrast to the traditional conflict resolution centers. below are some examples of the vision and mission of existing centers [names withheld]: o “[xyz]’s mission is to bring people together to find lasting and effective solutions to conflict;” o “[xyz] is an innovative center committed to developing knowledge and practice to promote constructive conflict resolution, effective cooperation, and social justice. we partner with individuals, groups, organizations, and communities to create tools and environments through which conflicts can be resolved constructively and just and peaceful relationships can thrive;” o “[xyz] … is guided by a fundamental commitment to human rights and the alleviation of human suffering. it seeks to prevent and resolve conflicts, enhance freedom and democracy, and improve health.” ic4cr is different from these in that it has a definite approach with a process to be followed: o ic4cr is based on the application of the ahp to the study and analysis of conflicts; o the chief purpose of ahp is to provide decision makers with objective, numerical parameters regarding specific core issues. from such a valuation model, decision makers have access to a rational based model/tool for addressing and resolving specific, complex issues; o the primary benefit of the ahp, whether used internally or together between the parties, is to reduce uncertainties—between and among the parties—on the relative value of core issues as negotiators address the “trade-off/exchange” component of negotiations. the information produced by this tool enhances rational-based decision-making, helps reduce emotion in negotiations, and assesses more accurately the relative value that each group attaches to an issue. ahp has been implemented in other international conflicts such as south africa and northern ireland; ijahp essay: vargas, zoffer/applying ahp in conflict resolution international journal of the analytic hierarchy process 145 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.649 o the trade-off model is predicated upon the development and application of a process that reflects both in-depth understanding of the issues and the values attached to them by the respective parties (or sub-parties), and the importance of an issue in relation to other issues, of lesser, similar, or greater value. of equal importance is determining the value the other side attaches to that issue and the value both sides attach within the context of a trade-off or trade-offs; o the process requires assigning numerical values that measure the respective importance of each issue for the parties involved; it is that assessment/assignment that enables rational based decision-making in the context of potential trade-offs. ahp focuses on articulation and application of self-interest in a paradigm emphasizing trade-offs whereby both sides seek to “expand the pie” —and avoid zero-sum calculations that emphasize maximization of benefits for one side, to the detriment of the other side. the foundations of ic4cr are that the center’s approach to conflict resolution is based on the analytic hierarchy process. the tradeoff model developed from the ahp priorities is based on the following seven ideas: 1. each party identifies a set of concessions (trade-offs); 2. each trade-off that a party gives away yields for that party a set of costs (not necessarily monetary) and a perceived set of benefits for the party receiving it; 3. each trade-off that a party receives generates a set of benefits and a perceived set of losses for the party giving it away; 4. the benefits, costs, perceived benefits and perceived costs are prioritized using the ahp; 5. the trade-offs are evaluated according to the benefits, costs, perceived benefits, and perceived costs (see figure 1); 6. the trade-offs of the parties are paired to decide which pairs are acceptable. acceptable means both parties benefit from the trade-off and that they receive more than they lose from the trade-off they give away. acceptability of a pair of trade-offs is implemented using the gain-loss ratio. gain-loss ratios are not symmetric for the parties. this is not a zero-sum game; 7. acceptable pairs of trade-offs are identified with the additional condition that the gain-loss ratio of a pair of concessions is as close as possible to each other for the parties (i.e., within a small percentage of each other) yielding the desired-for balanced agreement. ijahp essay: vargas, zoffer/applying ahp in conflict resolution international journal of the analytic hierarchy process 146 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.649 figure 1 gain to loss ratio ic4cr has already applied this approach to the israeli-palestinian conflict. while the complexities of this conflict will require additional work, much progress has been made in identifying possible tradeoffs which could lead to a solution. another project being developed is how to improve, to the community’s and the police department’s benefit, reconciliation between policing and community interests. a major proposal to be financed by the federal government will focus on the seemingly intractable issue of how to improve the recidivism rate and develop programs which will encourage those released from prison to begin more productive lives. the idea of this center is not new; tom saaty worked in conflict resolution for many years. he wrote a white paper with laurence r. klein describing peace-war scenarios in the middle east. the first author used those scenarios to analyze the conflict, and the result was a paper in the european journal of operational research. no sooner had the paper appeared, at least this is how it seemed, than tom had the idea of his international center for conflict resolution (iccr). he wrote an essay describing how the center could be formed, and called it “creating an opportunity and making it possiblea far-out crazy idea worth considering for the long run survival of a civilized world.” the first two paragraphs of his essay give credence to the creation of our modest center which albeit has the same title, but we refer to it by the acronym ic4cr. he wrote: “everyone agrees that several international conflicts and terrorism today carry the seed of prolonged and deadly military confrontations and wars and also the overt or covert spread and use of weapons of mass destruction and continuing conflict worldwide. the potential use of dirty bombs with material taken from unprotected sites will keep western nations nervous, alert and worried for a very long time into the future. there are nearly a dozen countries with nuclear ijahp essay: vargas, zoffer/applying ahp in conflict resolution international journal of the analytic hierarchy process 147 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.649 capability and their number is growing. minority groups in pursuit of their goals resort to acts of terrorism. killing or incarcerating terrorists appears to be the only mode of reply. but there can be another more positive way to deal with terrorists that is in harmony with the preservation of law and order and the function of international institutions. history teaches us not to trust short lulls of peace. the world needs an effective approach to conflict resolution more than ever before to ensure a lasting peace and better relations and understanding among disadvantaged peoples and among feuding nations, such as india and pakistan, china and taiwan, north korea and the united states, israel and dissatisfied palestinians and other groups that support their resistance. the major concern of international conflict resolution today, however, is how to avoid war rather than how to attain peace. the continuation of a conflict through terrorism may be the new style of unsettling the established order in the world. to change this attitude, we need a new way to seek out the parties to understand their grievances and try to address them in a workable way.“ thomas l. saaty 1926-2017 ijahp news and events: our ijahp editorial team remembers tom international journal of the analytic hierarchy process 287 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.500 our ijahp editorial team remembers tom we asked each of our editorial team members to contribute memories, either pictures or simple words about tom, and your will find the different contributions below. for more pictures and memories of tom, or to add your own contributions, visit his remembrance website at: https://www.forevermissed.com/thomas-saaty/#about rozann saaty remembers tom tom and rozann saaty (rozann is the managing editor of the ijahp online journal) visited china in october 2016. tom gave lectures on decision making with ahp and creativity at 7 major universities in beijing and was honored at tianjin university in tianjin, china, where the first international symposium on ahp took place in 1988. many of the young scholars who attended the conference in 1988 are still at the university and have become senior administrators at tianjin university (see picture below). https://www.forevermissed.com/thomas-saaty/#about ijahp news and events: our ijahp editorial team remembers tom international journal of the analytic hierarchy process 288 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.500 rozann and tom saaty and professor wei gu of the university of science and technology in beijing visiting peking university tom sampling an ice at the beijing olympic park ijahp news and events: our ijahp editorial team remembers tom international journal of the analytic hierarchy process 289 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.500 tom saaty lecturing at central finance and economic university in beijing tom chilling out during his lecture tour in china ijahp news and events: our ijahp editorial team remembers tom international journal of the analytic hierarchy process 290 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.500 tom saaty in beijing with some of the academics who were visiting scholars at the university of pittsburgh in recent years birsen remembers tom my dear friend thomas saaty i have known tom since the 1990’s. we visited each other quite a few times a year, sometimes at tom and rozann’s home or at my home in canfield, where we would visit mill creek park or the canfield bicycle trail. i attended several of tom’s classes and sometimes my husband cengiz did too; cengiz was a guest speaker for tom’s creativity class. cengiz and tom enjoyed each other’s company and both always eagerly anticipated the next visit. we were there when he received his gold medal in 2004 along with his whole family. he had about 6000 citations at that time, and it was a big achievement. i attended most of the isahp symposiums, starting in 1996 in vancouver. in vancouver we celebrated tom’s 70 th birthday and, as usual, all the family was there. at the isahp symposium, in switzerland in 2001, a swiss performer played a big alpine horn, and tom followed up this performance with one of his own, playing a tiny violin (less than 10 inches). he played it very well, and everyone was pleasantly surprised, and enjoyed some good laughter. we also attended some mcdm (multiple criteria decision making) meetings together. in ankara in 2000, tom received the mcdm gold medal. this is the highest honor that the international society on multiple criteria decision making bestows upon a scholar ijahp news and events: our ijahp editorial team remembers tom international journal of the analytic hierarchy process 291 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.500 who, over a distinguished career, has markedly contributed to the theory, methodology, practice and professional development of mcdm. we liked chatting whether it was in pittsburgh, pa or in canfield, oh, enjoyed browsing in parks, or wandering around on bicycle trails. i learned how to cook for him without salt, adding different ingredients instead. he liked my spinach pie. a constant topic of discussion was handel’s ice-cream or desserts, in which we debated how much to allow him and how to prevent him eating a lot! i enjoyed our thanksgivings together, celebrating nina’s (john and shawna’s daughter, tom and rozann’s granddaughter) birthday and our ping pong competition. we watched over the years as the children became so competitive, and it was becoming harder to beat them. these thanksgivings always included guests, and students, mostly international students working with him. at thanksgiving last year when i saw tom talking to nina, i was sad thinking that quite possibly this might be the last thanksgiving with tom. we enjoyed having so many dinners and lunches in tom and rozann’s dining room where rozann always cooked very delicious meals for us. it was a great treat for me, and i adored the ambience of the room and wonderful conversations. we will continue sharing a table and friendship with rozann, and when we come together we will exchange the good memories of you. as you said we will look forward, we will miss you a lot, but cherish you with your good jokes, with the people we met because of you, and with your colorful personality. tom received the gold medal, 2004, pittsburgh ijahp news and events: our ijahp editorial team remembers tom international journal of the analytic hierarchy process 292 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.500 birsen and tom chatting in pittsburgh, 2006 at tom & rozann’s home birsen, tom and rozann at mill creek park, youngstown, 2007 ijahp news and events: our ijahp editorial team remembers tom international journal of the analytic hierarchy process 293 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.500 rozann, birsen, and tom, canfield bicycle trail, 2009 tom’s last thanksgiving, 11/24/2016. elena, nina, and tom; chinese students in second row ijahp news and events: our ijahp editorial team remembers tom international journal of the analytic hierarchy process 294 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.500 birsen and tom, at tom and rozann’s home, february 4, 2017, pittsburgh elena rokou remembers tom in memoriam… i met tom back in 2009, and since then i was lucky enough to have him as a mentor and later on i was blessed to be called friend. he was the most truly genius person that i have ever met. he always had time to teach all of us, not only about ahp, but about life as well. he was most happy when he had students around him and was always a loving father to all of them. he would ask for your opinion and listen carefully no matter how old or how experienced you were. i will never meet someone as brilliant and loving as thomas l. saaty. i hope that some of his questions about the universe have been answered now and i wish he could share them with us! tom and researchers from china and italy in 2016 ijahp news and events: our ijahp editorial team remembers tom international journal of the analytic hierarchy process 295 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.500 tom and rozann saaty with emily harker and elena rokou in sorrento during isahp2011 antonella petrillo remembers tom my dearest thoughts the first time i personally met tom was in 2005, with my twin sister laura, during our vacation in pittsburgh. for me, tom was one of the most important “events” in my life because he “marked” both my personal and academic life. in 2011, i had the opportunity to spend a period of time in pittsburgh with tom and rozann and their family, and that was one of the most beautiful and unforgettable times in my life. i will always and forever be grateful to tom, for the opportunities he gave me. tom gave me the greatest gift anyone could give another person he believed in me. one thing i’ll remember forever, tom used to say to me…"antonella ... you do not always have to run. it’s important to have time to think… to think of effective solutions to problems, to do better and better.” i am so sorry for his loss. he will be missed, forever. laura and antonella with tom pittsburgh 2005 ijahp news and events: our ijahp editorial team remembers tom international journal of the analytic hierarchy process 296 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.500 enrique mu remembers tom i invited tom to talk to my decision-making class in spring 2015. he was so fascinating that one of my students told me that his talk had been akin to having an out-ofbody experience. i attended one of tom’s last ahp classes at katz. he was already in chemotherapy treatment and had lost hair so he would wear a red beret that gave him a very distinctive look. i was surprised that he still maintained such a high level of energy to teach his class. ijahp news and events: our ijahp editorial team remembers tom international journal of the analytic hierarchy process 297 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.500 tom motivated a whole generation of scholars and practitioners to dedicate their professional life to the improvement of decision-making through the use of ahp/anp. this is one of my best memories of him, when he addressed the attendees of the isahp2014 in washington, dc. valentina ferretti remembers tom i still remember the unbelievable feeling of surprise and happiness when in 2013 i received an email from tom saying that he had heard positive things about my paper on the anp and spatial analysis that received the wiley practice prize in 2011, and that he was inviting me to spend a month in pittsburgh to discuss research ideas together and collaborate in his anp course. it was for me a true honor to meet tom in person as i had been studying many of his books and papers throughout my phd and post doc positions! but meeting him and rozann for real was not only an honor, it was also an inspiring and touching experience from both the professional and personal point of view. i spent a beautiful beginning of autumn in pittsburgh, enjoying challenging discussions about the ahp/anp and decision making in general together with tom in their house courtyard. i was impressed by tom’s passion for his work and for science and will always remember him as an inspiring scientist and role model. it was so beautiful to see and perceive how much he loved studying, researching and developing new ideas, and this experience showed me that if you love your job you are going to enjoy it for your entire life. tom was the most creative and dynamic mind i had the pleasure to interact with, and i will always remember the lovely time spent together while i was in pittsburgh. one picture that i kept throughout the years was able to catch the beauty of the very positive environment he created around him and the big ahp family is a testament of this. ijahp news and events: our ijahp editorial team remembers tom international journal of the analytic hierarchy process 298 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.500 tom, rozann, valentina and birsen enjoying lunch at birsen’s place in october 2013 ijahp article: cooper/the magic of the analytic hierarchy process international journal of the nalytic hierarchy process 372 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.519 the magic of the analytic hierarchy process (ahp) orrin cooper university of memphis, 332 fogelman college business admin bldg., memphis, tennessee 38152, usa olcooper@memphis.edu abstract dr. thomas saaty developed the analytic hierarchy process (ahp) with the underlying goal of making it simple and accessible to the lay user. in saaty’s own words, the ahp is based on how “ordinary people process information” and “express the strength of their judgments” (saaty, 1994, p. 37). because he was successful in developing the ahp in accordance with these goals, when decision makers use the ahp their experience can feel magical as they find pairwise comparisons natural and can relate to the final priorities. careful investigation of the axioms, theorems, and proofs shows that the ahp is more than just magic and provides scientific justification of the highest order. five important components of the ahp and some background into the history of its development are summarized and highlighted from saaty’s article, “how to make a decision: the analytic hierarchy process” (saaty, 1994). keywords: analytic hierarchy process (ahp); thomas saaty 1. introduction there is something about magic that fascinates us. when something that is considered nearly impossible is performed, we marvel and ask, “how was it done?” i remember my first time using the analytic hierarchy process (ahp) for a class assignment to model a decision about where to locate a semiconductor production facility. the process felt intuitive and natural, and the final answer made sense. as we performed the sensitivity analysis and i was able to test patterns, i was hooked. it almost seemed magical how stepby-step the process of making and aggregating the pairwise comparisons felt so natural, comfortable, and intuitive, yet somehow when all the parts were aggregated they represented an answer for a complex decision. after class, i went home and built and tested a personal decision model that worked and made sense. i knew then that it had to be more than just magic; the ahp is a complex interdisciplinary multi-criteria decisionmaking method composed of many axioms, theorems, and proofs. in saaty’s own words, “it must be simple and accessible to the lay user, and must have scientific justification of the highest order” (saaty, 1994, p. 40). in his article, “how to make a decision: the analytic hierarchy process”, dr. thomas saaty expounds on the scientific justification for the analytic hierarchy process. rozann and thomas saaty constantly welcomed many, many people from all over the world into their home as they shared the ahp with them. dr. saaty would only ask of his mailto:olcooper@memphis.edu ijahp article: cooper/the magic of the analytic hierarchy process international journal of the nalytic hierarchy process 373 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.519 students or guests to do two things in return. first, to go and use what they had learned, and second, to share it with others, because that is the best way to get the ahp into the hands of other people. i had the opportunity on multiple occasions to hear saaty share firsthand the journey he went through to develop this decision-making approach. one of the most meaningful sections of his article is where he explains some of the history and background that drove him to develop the analytic hierarchy process (ahp) and the analytic network process (anp). saaty explains that his motivation to develop a “decision theory that would have lasting value” was driven by his own personal experience (saaty, 1994, p. 37). the ahp did not come about as an interesting mathematical model developed in the abstract, but in response to saaty’s efforts to deal with real problems he faced in his job. after graduating with a phd in mathematics from yale university and before entering academia, saaty spent many years working in government agencies including the u.s. state department in the arms control and disarmament agency. in this position, he worked with some of the world’s greatest minds, but the group was unsuccessful in developing ways to deal with negotiations and trade-offs in diplomatic and political decisions. saaty was determined to develop a method based on how humans make decisions naturally. he stepped back and asked an important question for which the answer is one of the reasons for the popularity of the ahp. he asked, “how do ordinary people process information in their minds in attempting to make a decision, and how do they express the strength of their judgments?” (saaty, 1994, p. 37) he goes on to explain how these questions led him to consider “hierarchies and networks, paired comparisons, ratio scales, homogeneity and consistency, priorities, ranking” which are all important components of the ahp (saaty, 1994, p. 27). some additional details are provided by ferretti and saaty (2014) from an interview with thomas saaty. in the interview, he shared how he would wake up at night and not be able to sleep again until he would write down the thoughts that would come into his mind; it was a process that took time and many struggles. it was a real treat to hear the details behind the development of the ahp and hear the passion in his voice decades later. because saaty was able to answer that fundamental question, the ahp has become a popular decision-making method over the decades. in 2008, wallenius et al. (2008) showed that the ahp/anp was the most widely published multicriteria decision-making method. with that important context in front of us, we can now proceed to review the other meaningful contributions in the foundational interfaces article (saaty, 1994). in this article, saaty begins by providing other evidence and motivation for using the ahp to solve problems. two examples are given as follows: 1) “the hospice problem”, where options for taking care of terminally ill patients were considered; and 2) “a firm evaluates its employees for raises” where a ratings model was used to rank employees for raises. these examples show not only how one could use the ahp, but also provide the context to present many of the key principles and theorems of the ahp in more practical terms that are accessible to a non-technical audience. citing a book that he published in 1982, saaty explains that a decision-making approach should have the following characteristics (saaty, 1982, p.20):  be simple in construct  be adaptable to both groups and individuals ijahp article: cooper/the magic of the analytic hierarchy process international journal of the nalytic hierarchy process 374 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.519  be natural to our intuition and general thinking  encourage compromise and consensus building  not require inordinate specialization to master and communicate the ahp possesses each of those characteristics. in line with saaty’s rule of “seven plus or minus two”, there are five important contributions contained in the interfaces article which will be discussed in greater detail below. these five principles are it really works, the process, from relative comparisons to deriving relative priorities, the mathematical elegance of the ahp, and the benefits from using the ahp (saaty, 1994). 2. five important contributions 2.1 it really works! in addition to the two complex models presented in the article, there are two other simple examples that should be discussed. because of their simplicity, one can quickly experience the ahp and see that it really works! figure 1, which was printed as figure 4 in the interfaces article, contains five shapes that can be compared in terms of their areas using pairwise comparisons. the result will be a very good approximation to the relative areas of the shapes that one could obtain by using a ruler and the formulas for calculating the areas of the shapes. the figures were used in an experiment with mba students to validate the use of the 1-9 scale (whitaker, 2007). saaty would also have his students do this exercise without knowing the relative weights beforehand as a part of his first lecture each semester. as the mba students do the pairwise comparisons and see that it really works, they begin to engage and get excited about the ahp. figure 1. five shapes to calculate relative areas the second example is when students are asked to approximate the relative size of a cherry tomato to a watermelon. in the article, saaty compares a blueberry to a watermelon to explain the importance of homogeneity and clustering (saaty, 1994, p. 35). when a participant sees this example for the first time, she initially feels overwhelmed and only able to make a reasonable guess without either buying enough cherry tomatoes to try it out or potentially developing a set of measuring tools (rulers) and formulas (volume). when she is presented with the idea of comparing the cherry tomato (or blueberry) to other fruits including a plum, the pairwise comparisons suddenly feel manageable. but, it is not until the next step of comparing the plum to other objects including a grapefruit that the lightbulb comes on. after the subsequent comparisons culminating with the cluster that includes the watermelon, one feels empowered that anything can be compared through relative comparisons if the problem is structured properly. in experiencing these two examples participants informally apply many of the ijahp article: cooper/the magic of the analytic hierarchy process international journal of the nalytic hierarchy process 375 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.519 principles of the ahp that were formalized and proven by saaty and see for themselves that it really works. 2.2 the process itself saaty explains, “i believe that making decisions in real life situations depends on the depth and sophistication of the structures decision makers use to represent a decision or prediction problem” (saaty, 1994, p. 39). to help guide decision makers through the process of structuring a problem saaty provides ten suggestions (saaty, 1994, p. 22): (1) identify the overall goal. what are you trying to accomplish? what is the main question? (2) identify the sub-goals of the overall goal. if relevant, identify time horizons that affect the decision. (3) identify criteria that must be satisfied to fulfill the sub-goals of the overall goal (4) identify sub-criteria under each criterion. note that criteria or sub-criteria may be specified in terms of ranges of values of parameters or in terms of verbal intensities such as high, medium, low. (5) identify the actors involved. (6) identify the actor’s goals. (7) identify the actor’s policies. (8) identify options or outcomes. (9) for yes-no decisions, take the most preferred outcome and compare the benefits and costs of making the decision with those of not making it. (10) do a benefit/ cost analysis using marginal values. these suggestions follow a logical process that decision makers can use to approach and think about decisions in general. it is also no coincidence that this process naturally flows into a hierarchy. this “ten step cheat sheet” can be used as a guide to structure personal decisions or when guiding a group through the decision-making process. these steps should not be taken for granted or glossed over; as saaty said, “perhaps the most creative part of decision making that has a significant effect on the outcome is modeling the problem” (saaty, 1994, p. 22). this is another advantage of using the ahp to make decisions, the process that you go through helps you thoughtfully consider/discover what matters and what impacts the decision as you design the hierarchy. through the pairwise comparison process there is a way to quantify the relative importance and impact of each of the elements that were identified. this process of following the structured ten steps and creating a hierarchy and making pairwise comparisons also leads to meaningful discussions. further discussions and insight occur when exploring the robustness of the decision using sensitivity analysis. in one sense, the process itself is as important as the method. 2.3 from relative comparisons to priorities making relative comparisons is an innate ability that humans have and is at the core of the ahp. saaty often used the example of comparing apples with respect to their size without using rulers and/or scales, but by making relative comparisons to demonstrate this point. here is where saaty brought psychology and math together to take advantage of an innate human ability to obtain priority vectors. saaty defined priorities as numerical ranks measured on a relative absolute scale with a ratio property. “a ratio scale is a set of positive numbers whose ratios remain the same if all the numbers are multiplied by an ijahp article: cooper/the magic of the analytic hierarchy process international journal of the nalytic hierarchy process 376 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.519 arbitrary positive number”, he said (saaty, 1994, p. 28). furthermore, he states, “ratio scales are the fundamental tool of the mind that people use to understand magnitudes” (saaty, 1994, p. 40). the process of obtaining priorities is broken down into smaller steps. saaty explains, “the ahp is based on the innate human ability to make sound judgments about small problems” (saaty, 1994, p. 21); he defines a judgment or comparison as “the numerical representation of a relationship between two elements that share a common parent” (saaty, 1994, p. 25). in other words, the decision can be decomposed into smaller subsections with as many levels as the decision maker needs until the elements are “small” enough that sound judgements can be made as the elements are compared. the numerical relationships are elicited through relative comparisons with respect to importance, preference, or likelihood, and this is where the psychology comes in. “cognitive psychologists have recognized for some time that people are able to make two kinds of comparisons, absolute and relative. in absolute comparisons, people compare alternatives with a standard in their memory that they have developed through experience. in relative comparisons, they compare alternatives in pairs according to a common attribute” (saaty, 1994, p. 33). relative comparisons also allow us to measure “intangibles” and even combine tangible and intangible criteria within the same model. in the apple example, the weight or size are “intangible” if one does not have rulers or scales, but that does not limit the ability of one who is familiar with size and weight to derive meaning and make relative comparisons. the priority vector from the relative comparisons will be very close to that obtained from first weighing the apples on a scale and then expressing the weights relative to the other apples. intangible properties have a unit of measurement or else even relative comparisons could not be made. another way of looking at intangibles in the ahp is to think of them as properties where using a formalized measurement tool is prohibitive because of reasons like cost, current technological limitations, or that it is socially inhibitive to use a formalized measuring tool. by breaking the intangibles down into smaller problems and making relative comparisons between two elements meaningful priorities can be obtained. this leads us to the beauty of the math of the ahp and how to get final priorities. 2.4 the mathematical elegance saaty touched on the axioms, theorems, and proofs of the ahp at a very high and practitioner-oriented level throughout the article. the real meat of the mathematics can be found in various other publications, but key principles like redundancy and the consistency ratio were presented. redundancy among the pairwise comparisons and the consistency ratio are both important in that they help increase the decision maker’s confidence in the decision. another important mathematical property of the ahp that was discussed was the three ways that can be used to get a priority vector from a perfectly consistent pairwise comparison matrix. in the hospice example, saaty explains that one can add the values in each row of the pairwise comparison matrix and divide that summation by the sum of all the entries in the pairwise comparison matrix. there you have it! this is the easiest way to get the priority vector; however, it is no coincidence that there are other ways to get the same result. two other methods include solving the equation ; or raising the matrix to large powers and normalizing the sums of the rows when it converges. these ways, which work for a consistent matrix, also work if there is some inconsistency and they take into account all the intransitivities between the elements (saaty, 2013). each solution could be considered the most elegant ijahp article: cooper/the magic of the analytic hierarchy process international journal of the nalytic hierarchy process 377 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.519 for different reasons, but what is most important is the idea that there is more than one way to get the same result. in some ways, this is like the pythagorean theorem that has been proven in many different ways. there is value in knowing the solution is not a special case. 2.5 five benefits even though human behavior does not always follow theoretical and normative considerations, saaty argues, based on his experience, that an analytical approach can still be very useful. saaty concludes by sharing five benefits of using the “descriptive analytical approach” to decision-making (saaty, 1994, p. 40):  first, the morphological way of thoroughly modeling the decision, induces people to make explicit their tacit knowledge.  second, particularly in the framework of hierarchies and feedback systems, the process permits decision makers to use judgments and observations to surmise relations and strengths of relations in the flow of interacting forces moving from the general to the particular, and to make predictions of most likely outcomes.  third, people are able to incorporate and trade off values and influences with greater accuracy of understanding than they can by using language alone.  fourth, people are able to include judgments that result from intuition and emotion as well as those that result from logic.  finally, a formal approach allows people to make gradual and more thorough revisions and to combine the conclusions of different people studying the same problem in different places (saaty & alexander 1989). the benefits from the morphological way of modeling and the framework were touched upon in the prior sections of this paper where the value of the structure and process in the ahp were discussed. the third and fourth benefits are related, and deal with the ability to make pairwise comparisons. the integration of language and mathematics provides a richer model than if either were used alone. recognizing and incorporating the decision maker’s intuition and emotion is critical. first, because both intuition and emotion are required in making a decision, a quantitatively optimal decision may no longer be optimal if it makes one feel miserable. additionally, by being able to incorporate intuition and emotion into the decision, the feeling of trust through validation and buy-in from the decision maker can increase. additional buy-in is due in part to acknowledging and incorporating important mental components of human decision making into the decision. the ability to synthesize input from different people or get them to come to consensus is self-evident in how the ahp has been so widely applied in group decision-making. 3. conclusion these fundamentals of the ahp reviewed herein, from the ability to make pairwise comparisons to the psychological and structural benefits of using the ahp, should not be taken for granted. nor can we forget our initial experience with the ahp, the moment where we saw that it really works, and let that excitement diminish. in spite of the ease provided by modern software, we should not overlook the elegant math behind the ahp. these concepts are at the very core of the ahp and can almost be taken for granted the more familiar one becomes with the process, but if they are taken for granted, glossed ijahp article: cooper/the magic of the analytic hierarchy process international journal of the nalytic hierarchy process 378 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.519 over, and not given the proper attention, then the ahp would become just another multicriteria decision-making method. as saaty said, “i believe that making decisions in real life situations depends on the depth and sophistication of the structures decision makers use to represent a decision or prediction problem” (saaty, 1994, p. 39). these fundamental principles of the ahp facilitate meaningful decision-making. saaty changed the way we measure things; the greatest way to recognize his contributions is to use them to tackle the complex decisions in our day and spread awareness of the “magic” of the method. ijahp article: cooper/the magic of the analytic hierarchy process international journal of the nalytic hierarchy process 379 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.519 references saaty, t. l., (1982). decision making for leaders. pittsburgh, pa: rws publications. doi: https://doi.org/10.1080/00137918308967693 saaty, t. l. & alexander, j. (1989). conflict resolution: the analytic hierarchy process. new york: preager. saaty, t. l. (1994). how to make a decision: the analytic hierarchy process. interfaces, 24(6), 19-43. doi: https://doi.org/10.1287/inte.24.6.19 saaty, t. l. (2013). the analytic hierarchy process without the theory of oskar perron. international journal of the analytic hierarchy process, 5(2), 268-293. doi: http://dx.doi.org/10.13033/ijahp.v5i2.191 ferretti, v., & saaty, t. (2014). questions and answers: valentina ferretti interviews tom saaty. international journal of the analytic hierarchy process, 6(2), 132-143. doi: http://dx.doi.org/10.13033/ijahp.v6i1.235 wallenius, j., dyer, fishburn, p. c., steuer, r. e., zionts, s., & deb, k. (2008). multiple criteria decision making, multiattribute utility theory: recent accomplishments and what lies ahead. management science, 54(7), 1336-1349. doi: 10.1287/mnsc.1070.0838 whitaker, r. (2007). validation examples of the analytic hierarchy process and analytic network process. mathematical and computer modelling, 46(7), 840-859. doi: https://doi.org/10.1016/j.mcm.2007.03.018 https://doi.org/10.1080/00137918308967693 https://doi.org/10.1287/inte.24.6.19 https://doi.org/10.1016/j.mcm.2007.03.018 ijahp news: ferretti/ x optima & vi red-m conference highlights international journal of the analytic hierarchy process 288 vol. 5 issue 2 2013 issn 1936-6744 ahp news: x optima &vi red-m conference highlights valentina ferretti the x optima & vi red-m 2013 participants last october, two important events took place in concepción, chile. from the 27 th to the 30 th of october the university of concepción hosted the 10 th biennial conference of the chilean institute of operations research (optima), and the 6 th meeting of the ibero-american network on multi-criteria decision (red-m). this year the events attracted about 120 participants, 30% of which were students, coming from belgium, italy, spain, usa, canada, mexico, colombia, brazil and chile. within this truly international environment, four sessions were dedicated to multi-criteria decision making methods, two of which were themed on the analytic hierarchy process (ahp). the first one, chaired by professor claudio garuti, from chile, concerned multi-criteria decision making methods for developing countries; the second one, chaired by professor josé marìa moreno jiménez, from spain, concerned cognitive approximation in multicriteria decision making with two real applications in business and decisionmaking. moreover, other parallel sessions discussed the application of ahp in the logistic, operation management and environmental contexts. some key contributions in a nutshell are following: professor valerio salomon from ijahp news: ferretti/ x optima & vi red-m conference highlights international journal of the analytic hierarchy process 289 vol. 5 issue 2 2013 issn 1936-6744 brazil discussed the usage of the compatibility index to group decisionmaking; professor isabel spencer from chile presented an application for the construction of a sustainability index for the assessment of buildings’ performances; dr. astrid oddershede from chile presented an application in the context of the reuse of organic waste; dr. juan carlos osorio from colombia illustrated an application for integrated solid waste management in supply chain; dr.claudio macuada from chile proposed an application of the ahp to evaluate technologies for water treatments, while marco fuentes from chile presented an application of the ahp dealing with hospital waste management. the general good fellowship enjoyed by the participants, together with the active learning environment which was able to provide guidelines to the youngest participants as well as new insights for everyone, ensured the success of the meeting. don’t miss the next conference which will likely be in monterrey, mexico, in 2014! finally, a special thanks to dr. javier pereira for organizing this conference for red-m 2013, and to claudio garuti, from fulcrum engineering consulting (chile), and valerio salomon, from sao paolo state university (brazil), for bringing our attention to this interesting event and providing all the relevant information. for details, please visit the conference website at www.optima2013.cl. http://www.optima2013.cl/ rob typewritten text http://dx.doi.org/10.13033/ijahp.v5i2.212 ijahp article: tarique, islam, mohamed, razak, hamdan/constructing a maqasid (objective) based performance measurement index for islamic banks international journal of the analytic hierarchy process 328 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.770 constructing a maqasid (objective) based performance measurement index for islamic banks kazi md. tarique university of liberal arts, dhaka, bangladesh kmtarique79@gmail.com rafikul islam 1 international islamic university malaysia jalan gombak, 53100 kuala lumpur, malaysia rafik500@yahoo.com mustafa omar mohamed department of economics, kulliyyah of economics and management sciences, international islamic university malaysia, mustafa@iium.edu.my dzuljastri abdul razak department of finance, kulliyyah of economics and management sciences, international islamic university malaysia, dzuljastri@iium.edu.my hamdino bin hamdan department of finance, kulliyyah of economics and management sciences, international islamic university malaysia hamdino@iium.edu.my abstract the current study aims to develop a maqasid al-shari’ah (objective of islamic law) based new performance evaluation index for islamic banks. in order to achieve the objective, the current study conducted twelve semi-structured interviews of academics and practitioners who specialize in islamic banking and finance (ibf) to select the variables for the index. in order to validate the selected variables, a focus group discussion was arranged that consisted of five experts in ibf. then, in order to prioritize the selected maqasid variables, items and sub-items, the analytic hierarchy process (ahp) was employed. the thirty respondents included industry experts, shari’ah scholars and academics. lastly, an index was developed and tested on some selected islamic banks in malaysia and bangladesh to rank their maqasid based performance. the findings revealed that rhb islamic bank ranked first among the sampled islamic banks. acknowledgement: the authors are grateful to the ministry of education, malaysia for the fundamental research grant, no. frgs15-227-0468, which has been used to carry out this research. mailto:kmtarique79@gmail.com mailto:mustafa@iium.edu.my mailto:dzuljastri@iium.edu.my ijahp article: tarique, islam, mohamed, razak, hamdan/constructing a maqasid (objective) based performance measurement index for islamic banks international journal of the analytic hierarchy process 329 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.770 overall, malaysian islamic banks performed better than bangladeshi islamic banks. the novelty of this study lies in its contribution to the theory of maqasid al-shari’ah with respect to banking performance evaluation. the index is expected to assist policy makers, practitioners and other stakeholders to determine whether the performance of islamic banks is in line with maqasid al-shari’ah and therefore adopt necessary policy changes accordingly. keywords: islamic banks; performance evaluation; maqasid al-shari’ah based index; analytic hierarchy process 1. introduction in the present era, banks rank highest in importance among all financial institutions. in a modern economy, no individual firm or government can afford to avoid them. the central bank of every country supervises and regulates the health of their economy through commercial banks. banks facilitate the activities of firms and individuals by providing savings and investment services. due to changes in technology, government regulations and globalization, banks are now offering various innovative services. some of the prominent services include automatic teller machines, debit cards, credit cards, electronic-banking, etc. (mohammed, 2011). however, there is a void in the industry when it comes to catering to the banking needs of muslims. the muslim community abides by the law of the almighty (shari’ah) and must refrain from giving and receiving interest (riba). since conventional banks are purely based on interest, god-fearing muslims are not able to participate in the fruits of banking facilities. even though banks have existed for 400 years, it was not until 1975 that an islamic bank emerged. after the establishment of dubai islamic bank in uae, the world has seen rapid growth of islamic banks and other islamic financial service institutions in the muslim world. different western countries also began introducing islamic banks after being impressed by the success of islamic banks and the growing demand for them. today, the conventional bank hsbc has opened an islamic banking facility “hsbc amanah” as its subsidiary. in summary, the financial service industry has now become more competitive and mature with the introduction of islamic banks. even though the market size of islamic banking assets in comparison with global banking assets is only 1.5%, the sector has achieved phenomenal growth. between 2006 and 2011, the assets of islamic banking institutions doubled and stood at $900 billion (financial times, 2011). in 2019, the shari’ah compliant assets rose 3% and were $2.5 trillion in comparison to $2.4 trillion in 2018. the experts forecast that by 2024, shari’ah compliant assets will rise to $3.5 trillion (ifdr, 2019). the islamic banks will make up 70% of the forecasted $3.5 trillion assets which is $1.76 trillion (ifdr, 2019). the major chunk of global islamic assets is shared by iran, saudi arabia, and malaysia (ifdr, 2019). in 2019, 62% of global islamic banking assets were shared among these countries (ifdr, 2019). a bullish economy and expanding customer base prompted this impressive growth and made these countries a lucrative market for islamic banks. among these countries, malaysia is aspiring to become a regional hub for islamic banking and finance (ifdr, 2019). ijahp article: tarique, islam, mohamed, razak, hamdan/constructing a maqasid (objective) based performance measurement index for islamic banks international journal of the analytic hierarchy process 330 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.770 even though the proponents of islamic banks consider it a recent innovation when compared to 400 years of conventional banking history, the critics are already questioning their objectives and success. this is due to the lack of evidence in the finance sector to support the huge expectations of change that were supposed to be a result of islamic banking. the claims of growth and popularity made by the islamic banking industry have not fulfilled the industry expectations (mohammed, 2011; usmani, 2008; choudhury, 2006; obaidullah & latiff, 2008). 1.1 statement of problem the biggest complaint against islamic banks is with regard to shari’ah compliance. many scholars and researchers have questioned the shari’ah validity of the asset and liability products that are offered by the banks. usmani (2008) asserts that islamic banking products seem to be shari’ah compliant on the surface, but are not when examined separately within the gambit of social effects. many islamic banking products are too complex in nature (obaidullah & latiff, 2008). by combining two or three shari’ah compliant mechanisms into one financing product, they are simply mimicking the conventional products when considering the economic effect (obaidullah & latiff, 2008). this complexity also increases the transaction cost (obaidullah & latiff, 2008). however, this persistence and innovation in developing new islamic banking products has kept the industry moving forward (akhter et al., 2009). there are also misgivings expressed by the scholars regarding the fulfilment of maqasid al-shari’ah through the activities of islamic banks. as mentioned, many scholars have suggested that maqasid al-shari’ah should be the objective of islamic banks. however, siddiqi (2006) expressed his concern regarding the failure to realize maqasid al-shari’ah by islamic banks in a changing environment. even though the islamic banks (ibs) are growing fast, this rapid growth results in a failure to fulfill maqasid (dusuki & bouheraoua, 2011). mohammed and shahwan (2013) found that the objectives of islamic banks are mostly profit oriented rather than a social based entity. ahmed (2011) states that by fulfilling both the substance and form of islamic law at the time of agreement between customer and bank, the ibs can promote the objective of islamic economics, which will help fulfill maqasid al-shari’ah in a holistic manner. there are very few studies on the performance of islamic banks anchored in maqasid alshari’ah. in their annual reports, islamic banks focus mainly on financial performance rather than maqasid based performance. the form of the products and services offered by islamic banks may differ from conventional banks, but the substance in many cases is similar. as a result, the performance measurement system of ibs requires profound study (mohammed, 2011). further, the concept of social well-being, equity and equality is somewhat absent in ib’s activities. therefore, it is essential to conceptualize the maqasid related issues with respect to the performance of ibs. also, an index needs to be developed to assess the maqasid based performance of islamic banks. 1.2 objectives of the study the main objectives of the present study are the following:  to identify an appropriate maqasid al-shari’ah based theoretical framework to develop a maqasid (objective) based performance evaluation index (mpei); ijahp article: tarique, islam, mohamed, razak, hamdan/constructing a maqasid (objective) based performance measurement index for islamic banks international journal of the analytic hierarchy process 331 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.770  to identify the dimensions, elements and measures that are necessary to operationalize the components of maqasid al-shari’ah;  to test the acceptability and validity of mpei as a viable model;  to establish the priority of the components and their variables representing the index; and  to test the index by measuring the performance of selected islamic banks in malaysia and bangladesh. 2. literature review the pioneering work on the functions and objectives of islamic banks was initiated by chapra (2000). according to chapra (2000), ibs should play a significant role in achieving the socio-economic goals of islam along with providing quality financial services to the muslim community. algaoud and lewis (2007) concluded that the primary objectives of islamic banking and finance include: (1) abolition of riba (interest) from all financial transactions, (2) equitable distribution of wealth and income, and (3) economic development. on the other hand, rulindo, mardhatillah and hidayat (2008) asserted that as a shari’ah compliant financial system, the objectives and direction of the creation of islamic banks must achieve the objectives of shari’ah. however, aliyu et al. (2017) stressed that in order to do so, ib’s should strike a balance between environmental, societal, and institutional sustainability. 2.1 theory of maqasid al-shari’ah the term ‘maqasid’ (plural) originated from the arabic term ‘maqsid’ (singular) which actually implies objective, goal, purpose, etc. therefore, the term maqasid al-shari’ah can be expressed as the purposes, goals or objectives of islamic law. many scholars related the term maqasid to the term ‘ maslih’ that represents people’s interest (ashur, 2006). as an early contributor to al-maqasid theory, abdul-malik al-juwaini also used the term public interests (al-masalih, al-ammah) and al-maqasid interchangeably. the term ‘unrestricted interests’ (al-masalih al-mursalah) was coined by abu hamid alghazali and explains the objectives of shari’ah. therefore, it can be claimed that the maqsid, objective, purpose, goal, principle or end in the islamic law lays in the ‘interest of humanity’ (auda, 2008). according to ibn ‘ashur (al-raysuni, 1992), the muslim scholars of maqasid al-shari’ah such as alghazali, al-juwaini, al-shatibi agree that the broad objectives of shari’ah (islamic law) are to attain benefits for humanity (maslahah) and to avert all types of harm (mafsadah). 2.2 al-ghazali's theory of al-maqasid the theory of maqasid al-shari’ah by al-ghazali has lasted until the present time. numerous muslim scholars have made extensive use of al-ghazali's theory in their works. in al-mustasfa (n.d.), al-ghazali clearly spelled out the premises of maslahah. he divided them into the following three categories: daruriyat (necessities), hajiyaat (complements) and tahsiniyaat (embellishments). he claimed that the shari’ah was prescribed to mankind to fulfill these three objectives. in terms of their priority and importance, they should be ranked in this sequence. that is, a human being’s basic ijahp article: tarique, islam, mohamed, razak, hamdan/constructing a maqasid (objective) based performance measurement index for islamic banks international journal of the analytic hierarchy process 332 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.770 necessities should be fulfilled before fulfilling complements, and complements should be fulfilled before fulfilling embellishments (auda, 2008). imam al-ghazali states that necessities are elements that in their absence, the system of a nation can run into chaos. the elements that facilitate human lives are termed as complements. on the other hand, embellishments are factors that relate to ethical and moral conduct. al-ghazali further classified necessities into five protections (aldharuriyat al-khams), namely religion (aldin), life (al-nafs), intellect (al-aql), progeny (al-nasl) and wealth (al-mal) (ashur, 2006). al-ghazali’s theoretical framework is presented in figure 1. figure 1 al-ghazali’s maqasid framework as seen in figure 1, imam al-ghazali considers maslahah as maqasid which can be achieved by fulfilling three elements, namely necessity, complement and embellishment. necessity can be fulfilled with the preservation of religion, life, intellect, progeny, and wealth. 2.3 abu zaharah's theory of al-maqasid as discussed above, al-ghazali's theory of al-maqasid is considered a breakthrough and it has been adopted and adapted by many muslim scholars in the past and present. however, al-ghazali's theory has some limitations in application and dimension. ibn taymiyyah (d. 1327) criticized the approach for confining maslahah into three levels and the necessities into five elements as something that is individualistic (al-raysuni, 1992). he argued that maslahah is much wider than the classification adopted by the scholars of jurisprudence. he said that maslahah includes other dimensions such as justice, care for the family and neighbors and faithfulness to the covenants. ibn taymiyya's view is also shared by modern scholars such as atiyah (2008) who is of the opinion that maslahah is very wide. it encompasses justice, education, healthcare, sustainable development and care for the environment to mention a few. this view is very pertinent, especially today when human capital is an important and a core ingredient for development. the only case al-maslahah necessity protection of religion protection of life protection of intellect protection of progeny protection of wealth complemnents embellishments ijahp article: tarique, islam, mohamed, razak, hamdan/constructing a maqasid (objective) based performance measurement index for islamic banks international journal of the analytic hierarchy process 333 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.770 where a considerable attempt was made to broaden the view was in the work of abuzaharah (1997) who added two more dimensions to al-ghazali’s maslahah. figure 2 shows abu zaharah’s theoretical framework. figure 2 abu zaharah’s framework of maqasid 2.4 conceptual framework the framework presented in figure 3 combines al-ghazali and abu zahara’s frameworks to measure the performance of islamic banks. figure 3 maqasid model to measure the performance of islamic banks maqasid al shariah education justice maslahah maqasid al shari’ah justice financial ratios education financial ratios maslahah embellishments financial ratios complements financial ratios necessity protection of religion financial ratios protection of life financial ratios protection of mind financial ratios protection of progeny financial ratios protection of wealth financial ratios ijahp article: tarique, islam, mohamed, razak, hamdan/constructing a maqasid (objective) based performance measurement index for islamic banks international journal of the analytic hierarchy process 334 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.770 2.5 banks’ performance measurement based on maqasid al-shari’ah the literature review revealed that there remains a dearth of study on performance measurement of islamic banks that focuses on maqasid al-shari’ah. among the few studies, bedoui and mansour (2013) constructed a pentagon shaped performance measurement scale based on maqasid al-shari’ah. the scale was developed based on five facets under necessity namely, religion, life, intellect, progeny and wealth. the purpose of the measurement scale was to look at the contribution of islamic banks with respect to human welfare, economic and social stability, and prevention of corruption against the wealth maximization agenda of conventional banks. however, the authors admitted that the study was theoretical in nature and has limited scope for practical implementation. mohammed (2011) operationalized abu zaharah’s theory of maqasid al-shari’ah and constructed a model of performance measurement based on maqasid alshari’ah (pmms). he used financial ratios and measured the performance of some selected conventional banks and islamic banks. the results revealed the superior performance of islamic banks over conventional banks. the studies conducted by antonio, sanrego and toufiq (2012) and reni, muklis and cholisini (2014) adopted the maqasid framework of abu zaharah and the pmms model of mohammed (2011) in terms of operationalization of maqasid variables. antonio, sanrego and toufiq (2012) used maqasid al-shari’ah to measure the performance of islamic banks in indonesia and jordan. in terms of indexing, they used the simple additive weighting (saw) method. the findings showed that the indonesian islamic banking industry performed better than their jordanian counterpart. some recent studies took a different approach while measuring maqasid based performance. laela et al. (2018) sought to find the link between the co-alignment of management accounting strategy and the maqasid al-shari’ah based performance of indonesian islamic banks. the study found that maqasid al shari’ah based performance of islamic banks in indonesia had a positive relationship with the co-alignment between strategic management accounting practices, low cost oriented strategies, and mechanistic management control systems. julia and kassim (2019) used the maqasid al shari’ah framework to evaluate the green banking performance of some selected bangladeshi banks. the study compared five islamic banks with five conventional banks. the findings of the study revealed that islamic banks are ahead of conventional banks with respect to preservation of wealth, faith and intellect in relation to green banking activities. puspitasari et al. (2017) used data envelopment analysis (dea) to measure the efficiency of islamic banks in indonesia and malaysia. the input variables used were individual education, justice and welfare. however, none of the banks were found to be totally efficient in all of the aspects. 3. methodology the present research adopted a mixed method approach that was comprised of an extensive literature review, sekaran’s method of operationalization, interviews, focus group discussion, prioritization and secondary data analysis. the first phase of the research focused on an extensive literature review and sekaran’s method of operationalizing a concept (c) by first breaking it down into dimensions (d). next, the dimensions were further broken down into measurable items, namely elements (e). then, ijahp article: tarique, islam, mohamed, razak, hamdan/constructing a maqasid (objective) based performance measurement index for islamic banks international journal of the analytic hierarchy process 335 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.770 expert interviews were conducted involving twelve experts to finalize the primary variables. this was followed by a focus group discussion (fgd) of eight experts that was used to validate the primary variables. the experts for both the interviews and the fgd were selected by adopting judgmental and snowball sampling techniques. in the second phase, weights were assigned for each maqasid concept, dimension and sub-dimension. in most of the previous studies, the authors used equal weights for all of the maqasid variables. that is, it was assumed that all of the variables have equal importance. however, while measuring performance, some variables may carry higher weight than others. the second phase of this study used the analytic hierarchy process (ahp) to prioritize and determine the weights for the maqasid concepts, dimensions, and sub-dimensions. the ahp is considered a highly scientific and internationally recognized method of prioritizing choices and assigning weights. recently, the ahp was applied to prioritize maqasid elements in the context of liveability determination of malaysian cities (dali et al., 2019). in the last stage, an index was created to determine the score of each bank under study with respect to their achievement of maqasid al-shari’ah. these banks are ranked according to the maqasid score in descending order. 3.1 phase 1selection of variables based on the model identified in figure 3, this section attempts to operationalize the concepts of education, justice and maslahah into dimensions, elements, and measures. by means of a thorough literature review, content analysis, and logical inductive reasoning, the key terms for each dimension were identified corresponding to their concepts. the literature corresponding to their dimensions is presented below. educating individual (c1) and justice (c2) the literature review identified that mohammed et al. (2008) and mohammed and taib (2009, 2015) used sakaran’s concept to operationalize maqasid variables. later, antonio, sanrego and toufiq (2012) and reni, muklis and cholisini (2014) also adopted the model with little change and empirically tested it in different contexts. ibrahim and momin (2015) also operationalized the maqasid concepts using the same methodology. however, these studies only operationalized the concepts of abu zahrah’s framework, namely educating individuals, establishing justice, and public interest. this research adopts the dimensions, elements, and measures of the abovementioned studies with respect to educating individuals and establishing justice. protection of religion (al-din) (c3.1.1) the arabic word al-din is derived from its root word “d-a-n-a” which literally means judgement, accountability, submission, obedience and peace (amin et al., 2015). in summary, it means “way of life.” according to ibn‘ashur (1998), the concept of ‘protection of religion’ refers to safeguarding the faith of every individual from any form of distortion or confusion. atiyah (2008) suggested that religion should be protected on three levels, namely individual, family and ummah. he opined that, in the individual realm, strengthening the belief in allah, performing obligatory worship prescribed in the quran, embracing good character and remaining obedient is necessary to protect or safeguard religion. he ijahp article: tarique, islam, mohamed, razak, hamdan/constructing a maqasid (objective) based performance measurement index for islamic banks international journal of the analytic hierarchy process 336 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.770 also suggested that, at the community level, religion will be protected through praying in the congregation and performing socio-religious activities. however, contemporary scholars like auda (2008), quoting from ibn ashur, gave a re-interpreted meaning of ‘protection of religion’ as ‘freedom of faith’ or ‘freedom of belief’. these two terms mean that muslims should have the freedom to practice their religious activities, both at the individual and community level. a bank has no religion, but the objective behind the establishment of islamic banks is to protect the religious beliefs of muslims through establishing a riba free banking system. all muslims should have the opportunity to conduct business transactions, free of riba, gharar, maysir, etc. (ashur, 2006). in relation to islamic banking performance measurement, zaheer and rasool (2017) suggested three dimensions of protection of religion. these are shari’ah compliance, funding structure, and interest free funding. protection of life (al-nafs) (c3.1.2) protection of life (al-nafs) can be defined in a number of ways. according to atiyah (2008), protection of life means preventing life from being destroyed in the form of death, as well as protecting certain parts of the body from b e i n g harmed or damaged, so that the life can benefit by using those parts of the body. ashur (2006) reinterpreted the term protection of life as protection of human dignity, honor, and human rights. atiyah (2008) supports ibn ashur by arguing that human beings have psychological needs like dignity and honor which differentiates them from other creatures. in an attempt to operationalize the protection of life, different studies have emphasized issues like healthcare, poverty reduction, human rights, education, and good governance among others (esen & esen, 2015; bedoui, 2014; hapsari & herianingrum, 2014; mili, 2014; seman & dzolkarnaini, 2019). in the case of banks, zaheer and rasool (2017) used qard hasan loan, charitable activities, and equity financing while operationalizing the protection of life. protection of intellect (al-aql) (c3.1.3) atiyah (2008) considered the protection of intellect (aql) a three-step process as follows: to develop, preserve, and utilize the mind. al-ghazali considered intellect as the foundation and starting point of knowledge (chapra, 2008). protecting intellect also means safeguarding it from harm with things like drugs, alcohol, etc. ibn ashur reinterpreted the concept as “propagation of scientific thinking” or “avoiding brain drain” (chapra, 2008). the studies attempting to operationalize ‘protection of intellect’ emphasize proper education, training, research facilities, and freedom of expression (esen & esen, 2015; bedoui, 2014; haspari & herianingrum, 2015; mili, 2014; seman & dzolkarnaini, 2019). however, all of these dimensions were already captured under the concept of educating individuals. protection of progeny (al-nasl) (c3.1.4) the protection of progeny (al-nasl) can be defined as safeguarding everything that would ensure the progress and survival of the family and develop the future generations (amin et al., 2015). islam encourages an effective, productive, and healthy society. as a result, it legitimizes marriage, prohibits adultery and emasculation to preserve progeny. ijahp article: tarique, islam, mohamed, razak, hamdan/constructing a maqasid (objective) based performance measurement index for islamic banks international journal of the analytic hierarchy process 337 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.770 the prime objective of marriage is to ensure the continuity of the human race and to preserve posterity (ashur, 2006). married couples also enjoy sexual, emotional, and materialistic benefits (al sari, 2010). however, the twentieth century scholars of maqasid have reinterpreted the concept of “preservation of progeny” to a more familyoriented concept. ibn ashur, for example, referred to the term as “care for family” (ashur, 2006). currently, in malaysia or bangladesh (in these two countries the islamic banking system has a strong foothold), all islamic banks are registered under the security and exchange commission and operate as a public limited company. as a result, for islamic banks the term ‘family” can be replaced by “stakeholder” which consists of customers, employees, shareholders, and government. accordingly, caring for the family would mean caring for the interests of the stakeholders (mohammed et al., 2015). protection of wealth (al-mal) (c3.1.5) property or wealth (al-mal) is considered an important issue in the shari’ah. when referring to ‘protection of wealth’, ashur (2006) defined it as safeguarding the wealth of the ummah from being destroyed or a shift of ownership to others without any benefit to the ummah. al-shatibi in al-raysuni (2005) defined it as prevention from injustice, denial of orphans from their property, corruption, wastefulness, envy, and fraud. the concept of ‘protection of wealth’ is more in line with the functions and operations of islamic banks, which can be measured more objectively when compared with other maqasid concepts. an islamic bank accepts the deposits of its customers as amanah or qard in the current account, and as investment in a fixed or savings account. as a result, the bank has to play the role of trustee and mudarib (manager) (auda, 2008). in both cases, the bank needs to protect the wealth from all sorts of harm while at the same time increasing wealth by effectively utilizing its assets. the bank management must also increase the wealth of its shareholders. operationalizing complement (c3.2) and embellishment (c3.3) according to al-shatibi in al-raysuni (2005), complements are related to interests or objectives when fulfilled, and contribute to relieving hardship and difficulty. embellishment on the other hand, refers to the interests that are less important than necessities and complements but function to enhance and complete their fulfillment. when defining complement, attiyain al-raysuni (2005) gave the example of a hungry person who will suffer hardship, but will not die, if he does not find food. a person’s craving for sweets, chocolates, mutton and silk garments is an example of embellishment. al-shatibi in al-raysuni (2005) also mentioned commendable customs and habits, observing rules of etiquette and a high moral standard as examples of embellishment. operationalizing complements and embellishment with respect to islamic banking performance measurement is tricky, as no studies have operationalized these two concepts. as a result, the authors have operationalized the concepts on their own. next, the selected variables were presented to the experts for their opinions. twelve experts who were carefully selected from academics, a pool of shari’ah scholars and practitioners by adopting judgmental and snowball sampling techniques were interviewed. the opinions of the interviewees were recorded and then transcribed. the ijahp article: tarique, islam, mohamed, razak, hamdan/constructing a maqasid (objective) based performance measurement index for islamic banks international journal of the analytic hierarchy process 338 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.770 transcribed data were then analyzed for common themes. the interviewees suggested quite a number of changes in the dimensions and measures. the results of the analysis are synthesized and presented in table 1. table 1 synthesis of the modified dimensions, elements and measures concepts dimensions elements measures c1-educating individuals d1-advancement of knowledge scholarship education grant/total expense research research grant/total expense d2-instilling new skills training training expense/total expense d3-creating awareness of islamic banking publicity /marketing publicity or marketing expense / total expense product disclosure sheet (pds) available? yes/no c2-establishing justice d4-fair dealings with customers fair return roe -rod risk sharing vs. risk transferring mudarabah & musharakah investment/ total investment d5-fair dealing with community and environment fair compensation to community and environment csr/total expenses d6-fair dealing with employees protection against mistreatment & unfair dismissal employee association (yes/no) c3.1.1-protection of religion d7-shariah compliance shari’ah compliance report disclosure number of incidence per year. shariah non-compliance income (snci) / net income d8: shariah audit committee adequacy of shariah audit committee member number of members in shariah audit team d9: supporting religious activity amount spent on religious activity donation for mosques, madrasahs & other religious activity / total expense c3.1.2 -protection of life d10-fulfilling basic needs zakat zakat + sadaqah distribution /net asset d-11 fulfilling social needs donation in medicare and hospital amount spent on medicare and hospital /total expenses investment in microfinance investment in microfinance/ total investment reporting on esg in annual report yes/no c3.1.4 protection d-12 technological facility investment in investment in technology/ ijahp article: tarique, islam, mohamed, razak, hamdan/constructing a maqasid (objective) based performance measurement index for islamic banks international journal of the analytic hierarchy process 339 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.770 concepts dimensions elements measures of intellect (aql) technology total expense d-13 avoid brain drain employee attrition employee attrition ratio d-14 research and development investment in product innovation and r&d investment in r&d/ total expense d-15 succession plan succession program (yes/no) c3.1.4 protection of progeny (nasl) d16-protecting shareholders return on capital net income/total capital d-17 protecting customers marriage and housing marriage assistance (yes/no) home financing/ total investment d-18 protecting employees employee benefit allowance & bonus expenses/total expenses d19 -supporting government payment of tax tax paid/ profit before tax 3.1.5 protection of wealth (mal) d20-financial performance wealth maximization market value / book value asset utilization net income /total asset d-21 reducing wealth gap investment in real sector investment in the real economic sector/total investment investment in sme/ total investment investment in agriculture/ total investment d-22 minimizing risk credit risk non-performing investment/ total investment liquidity risk cash & purchased fund/ total deposit solvency risk total equity /total asset snci risk snci/ total investment c3.2 – complements d-23 facilitate withdrawal, deposit and transfer of fund transfer mechanisms number of atms & cdm / total number of branches d-24 products that are not considered as core banking product investment in salam, khafala, arrahnuetc contract investment in salam, khafala, ar-rahnu/ total investment c3.3 embellishment d-25 facilitate transfer of fund transfer mechanisms i) online banking ii)mobile banking yes/no ijahp article: tarique, islam, mohamed, razak, hamdan/constructing a maqasid (objective) based performance measurement index for islamic banks international journal of the analytic hierarchy process 340 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.770 concepts dimensions elements measures d-26 beautification/renovation renovation expenditure renovation expenditure/ total expenses d-27 special services deposit linked with takaful yes/no mobile top-up service yes/no priority banking yes/no 3.2 phase -2: prioritization and assigning weights to maqasid dimensions after finalizing the dimensions of maqasid and its measures, in phase two, a weight for each maqasid concept and dimension was assigned. in a summated scale, all of the independent variables do not carry equal weight when measuring a dependent variable. therefore, prioritization is necessary. this research uses thomas l. saaty’s decisionmaking method, popularly known as the analytic hierarchy process (ahp) to carry out the prioritization task (saaty, 2010). step 1: identification of the goal, criteria, and the sub-criteria: in this case, the goal is to prioritize the concepts, dimensions and elements of the maqasid model to evaluate banking performance. the criteria, which are directly aligned to the goal, are the three major concepts maslahah, justice and educating individuals. the sub-criteria under maslahah are necessities, complements and embellishments. under necessity, there are five sub-criteria as follows: protection of life, religion, intellect, wealth and progeny. the hierarchy of the dimensions and sub-dimensions are presented in figure 4. step 2: construction of the pair-wise comparison matrices (pcms) for all three criteria and five sub-criteria. ahp-based questionnaires were used to collect data from 30 respondents comprising academics, practitioners, shari’ah experts and shari’ah committee members. normally, the sample size for ahp data collection is small and the present sample size is considered adequate (takala, suwansaranyu, & phusavat, 2006). after this, the weights from the pcms are computed using superdecisions software, version 3.0. step 3: synthesis of the weights: the weights derived from the superdecisions software have been synthesized. the synthesis process provides the overall priorities of the sub-criteria. in order to appropriately rank the dimensions and sub-dimensions, respondents were carefully selected from different groups. while selecting the respondents, it was ensured that they had appropriate knowledge of maqasid al-shari’ah as well as islamic banking. accordingly, the research identified four groups of respondents, namely academics, shari’ah experts, shari’ah committee members and practitioners. the academics are involved in teaching and research on islamic banking and finance, but have limited shari’ah background. the shari’ah experts had sound knowledge of and a background in shari’ah and also have knowledge about islamic banking. the shari’ah committee members are members of a bank’s shari’ah supervisory committee. lastly, the practitioners work at the decision making level of an islamic bank. altogether, thirty respondents were interviewed. a sample pairwise comparison matrix is shown in table 2. ijahp article: tarique, islam, mohamed, razak, hamdan/constructing a maqasid (objective) based performance measurement index for islamic banks international journal of the analytic hierarchy process 341 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.770 this pairwise comparison matrix was developed by aggregating the individual judgments using the geometric means of the individual judgments. here, the five protections of necessity namely, protection of religion (por), protection of life (pol), protection of intellect (poi), protection of progeny (pop) and protection of wealth (pow) are compared with each other. finally, all of the local weights need to be synthesized to obtain a global or overall weight. the results of the analysis with priority values and rank are presented in table 3. table 2 a sample average pairwise comparison matrix concept por pol poi pop pow por 1 1.99 3.72 4.74 4.49 pol 1 4.55 4.36 2.89 poi 1 0.633 0.883 pop 1 0.699 pow 1 table 3 synthesis of ahp ranking and priority values level: criteria and subcriteria items rank (priority values) 1. level 2: criteria/dimensions maslahah 2 (0.343) level 3: sub-criteria/sub dimensions necessities 1 (0.707) level 4: sub criteria protection of religion 1 (0.430) level 5: sub criteria shariah compliance 1 (0.581) shariah audit committee 2 (0.262) supporting religious activities 3 (0.157) level 4: sub criteria protection of life 2 (0.303) level 5: sub criteria fulfilling basic needs 1 (0.774) fulfilling social needs 2 (0.226) level 4: sub criteria protection of intellect 5 (0.079) level 5: sub criteria technological facilities 4 (0.158) avoid brain drain 2 (0.251) research & development 1 (0.363) succession plan 3 (0.228) level 4: sub criteria protection of progeny 4 (0.087) level 5: sub criteria protecting shareholders 3 (0.212) protecting customers 1 (0.433) protecting employees 2 (0.279) supporting government 4 (0.076) level 4: sub criteria protection of wealth 3 (0.101) ijahp article: tarique, islam, mohamed, razak, hamdan/constructing a maqasid (objective) based performance measurement index for islamic banks international journal of the analytic hierarchy process 342 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.770 level: criteria and subcriteria items rank (priority values) level 5: sub criteria financial performance 3 (0.311) reducing wealth gap 1 (0.377) minimizing risk 2 (0.312) level 3: sub-criteria/sub dimensions complements 2 (0.208) level 4: sub criteria facilitate withdrawal, deposit and transfer of fund 1 (0.766) supplementary products 2 (0.234) level 3: sub-criteria/sub dimensions embellishments 3 (0.085) level 4: sub criteria facilitate online transfer of fund 1 (0.604) beautification/renovation 3 (0.151) special services 2 (0.245) 2. level 2: criteria/dimensions educating individuals 3 (0.229) level 3: sub criteria advancement of knowledge 3 (0.237) instilling new skills 2 (0.297) creating awareness of islamic banking 1 (0.466) 3. level 2: sub-criteria/sub dimensions justice 1 (0.428) level 3: sub criteria fair dealings with customers 1 (0.414) fair dealings with employees 3 (0.265) fair dealings with community and environment 2 (0.321) ijahp article: tarique, islam, mohamed, razak, hamdan/constructing a maqasid (objective) based performance measurement index for islamic banks international journal of the analytic hierarchy process 343 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.770 maslahah (0.343) justice (0.428) educating individuals (0.229) embellishment (0.074) complements (0.203) necessity (0.723) special services (0.245) facilitate wdt of fund (0.766) supplementary products (0.234) creating awareness of ib (0.466) instilling new skill (0.297) advancement of knowledge (0.237) fair dealings with community & env (0.265) fair dealings with employees (0.321) fair dealings with customers (0.414) facilitate online transfer of fund (0.604) protection of religion (0.430) beautification/ renovation(0.151) protection of wealth (0.101) protection of life (0.303) protection of progeny (0.087) protection of intellect (0.079) shariah compliance (0.581) shariah audit committee (0.262) 62 protection of religion protection of religion supporting religious activity (0.157) fulfilling social need (0.226) fulfilling basic need (0.774) avoid brian drain (0.251) technological facility (0.158) research & development (0.0363) succession plan (0.228) supporting govt (0.076) protecting employees (0.279) protecting customers (0.433) protecting shareholder (0.212) reducing wealth gap (0.377) financial performance (0.312) figure 4 maqasid hierarchy model minimizing risk (0.312) prioritization maqasid al shariah dimensions ijahp article: tarique, islam, mohamed, razak, hamdan/constructing a maqasid (objective) based performance measurement index for islamic banks international journal of the analytic hierarchy process 344 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.770 3.3 stage 3: index development and empirical test after the prioritization of the variables and assignment of weights, the maqasid index or mi was developed. after developing the mi, data from the annual reports of some selected banks were collected to empirically test the model. the mi for banks was developed in two stages. in the first stage, maqasid scores (ms) for the sampled banks were calculated. next, the mi was developed by following the methodology of the wellknown indices such as human poverty index (hpi), gender related development index (gdi), and the human development index (hdi). calculation of maqasid score (ms) for individual banks the calculation of the ms of the individual bank requires the application of a three-step process. according to the model, the final step is the weighted summation of the three concepts (c): educating individual (c1), justice (c2) and maslahah (c3). the mathematical formula is: 𝑀𝑆𝑖 = ∑ 𝑊𝑖 𝐶𝑖 … … … … … … … … … … … … … … … … . (1) 𝑛 𝑖=1 where, wi = weight of the i th element and n = number of elements in order to calculate the ms, first, the weights of c1, c2, c3 need to be determined. the justice and educating individuals are directly measured through financial ratios (e1.i…1.n) and (e2.i…2.n). the mathematical formula is the following: 𝐸𝑑𝑢𝑐𝑎𝑡𝑖𝑛𝑔 𝐼𝑛𝑑𝑖𝑣𝑖𝑑𝑢𝑎𝑙 (𝐶1) = ∑ 𝑊𝑖 𝐸1.𝑖 … … . … (2) 𝑛 𝑖=1 and 𝐽𝑢𝑠𝑡𝑖𝑐𝑒 (𝐶2) = ∑ 𝑊𝑖 𝐸2.𝑖 … … … … … … … … … . … (3) 𝑛 𝑖=1 the maslahah (c3) has three components as follows: necessity (c3.1), complements (c3.2) and embellishment (c3.3). maslahah is determined in the following manner: 𝑀𝑎𝑠𝑙𝑎ℎ𝑎ℎ (𝐶3) = ∑ 𝑊𝑖 𝐶3.𝑖 … … … … … … … … … . (4) 𝑛 𝑖=1 complements (c3.2) and embellishments (c3.3) are directly calculated using financial ratios. the mathematical expressions are: 𝐶𝑜𝑚𝑝𝑙𝑒𝑚𝑒𝑛𝑡𝑠 (𝐶3.2) = ∑ 𝑊𝑖 𝐶3.2.𝑖 … … … … … … … (5) 𝑛 𝑖=1 and 𝐸𝑚𝑏𝑒𝑙𝑙𝑖𝑠ℎ𝑚𝑒𝑛𝑡𝑠 (𝐶3.3) = ∑ 𝑊𝑖 𝐶3.3.𝑖 … … … … … … . (6) 𝑛 𝑖=1 necessity (c3.1), according to imam al-ghazali, has five components. protection of religion (c3.1.1), protection of life (c3.1.2), protection of intellect (c3.1.3), protection of ijahp article: tarique, islam, mohamed, razak, hamdan/constructing a maqasid (objective) based performance measurement index for islamic banks international journal of the analytic hierarchy process 345 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.770 progeny (c3.1.4) and protection of wealth (c3.1.5). as a result, the derivation of the necessity is calculated as: 𝑁𝑒𝑐𝑒𝑠𝑠𝑖𝑡𝑦(𝐶3.1) = ∑ 𝑊𝑖 𝐶3.1.𝑖 … … … … … … … … … … (7) 𝑛 𝑖=1 the five elements under necessity (c3.1) are calculated using the financial ratios, as follows: protection of religion (𝐶3.1.1) = ∑ 𝑊𝑖 𝐶3.1.1.𝑖 … … . … … … … (8) 𝑛 𝑖=1 protection of life (𝐶3.1.2) = ∑ 𝑊𝑖 𝐶3.1.2.𝑖 … … … … … . … … … . (9) 𝑛 𝑖=1 protection of intellect (𝐶3.1.3) = ∑ 𝑊𝑖 𝐶3.1.3.𝑖 … … … … … … … . (10) 𝑛 𝑖=1 protection of progeny (𝐶3.1.4) = ∑ 𝑊𝑖 𝐶3.1.4.𝑖 … … … … … … … . (11) 𝑛 𝑖=1 protection of wealth (𝐶3.1.5) = ∑ 𝑊𝑖 𝐶3.1.5.𝑖 … … … … … … … … . (12) 𝑛 𝑖=1 3.4 calculation of the maqasid index (mi) as mentioned above, the computation of the mi adopted the methodology used to develop similar oecd indices (jrc-ec, 2008). first, the linear scale technique (lst) was used for the ms of each bank, as follows: 𝑀𝐼𝑖 = 𝑀𝑆 − 𝑚𝑖 𝑀𝑖 − 𝑚𝑖 … … … … … … … … … … … … … … … … … . . (13) where, ms = actual score of an individual bank mi = minimum score mi = maximum score the score of each bank will remain in the interval between 0 and 1, where 0 means the bank did not fulfill the maqasid al-shari’ah at all, and 1 means the bank fulfilled the maqasid al-shari’ah to the fullest. a bottom up approach was used to derive the maqasid index. first, the maqasid score for the lowest level of dimensions was calculated; these are the five protections under necessity. then, the summated weighted value of the five protections gave the score for the concept necessity. next, the scores for complements and embellishment were calculated. then, the weighted summated score of necessity, complement and embellishment provided the score for maslahah. next, the scores for justice and educating individuals were calculated. finally, the summated weighted score of maslahah, justice and educating individuals provided the index value for a particular bank. ijahp article: tarique, islam, mohamed, razak, hamdan/constructing a maqasid (objective) based performance measurement index for islamic banks international journal of the analytic hierarchy process 346 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.770 the developed index was used to measure the maqasid performance of some selected banks in malaysia and bangladesh. the top five islamic banks from malaysia and six banks from bangladesh were selected based on their asset size. banks from only malaysia and bangladesh were chosen to maintain a standardized data set because it would be difficult to harmonize the annual report data of different banks from different countries due to difference in accounting and reporting practices. moreover, shari’ah permissibility of banking products differs from one country to another. the profiles of the sampled banks are provided in table 4. table 4 profile of the banks bank name code total assets in million local currency country bank rakyat br 92,285,928 malaysia bank islam malaysia barhad bi 49,767,067 malaysia rhb islamic bank rhb 44,076,741 malaysia may bank islamic mi 156,352,454 malaysia cimb islamic cimb 54,559,181 malaysia first security islamic bank fsi 256,604 bangladesh exim bank exim 264,653 bangladesh social islamic bank si 180,112 bangladesh islamic bank bangladesh limited ibbl 725,768 bangladesh shahjalal islamic bank sib 141,262 bangladesh al-arafa islamic bank aai 229,106 bangladesh table 5 maqasid score and ranking of sample banks bank maqasid score rank bank rakyat 0.4892 3 bank islam malaysia barhad 0.4434 4 rhb islamic bank 0.5152 1 may bank islamic 0.3667 9 cimb islamic 0.4911 2 first security islamic bank 0.3225 10 exim bank 0.4234 6 social islamic bank 0.3738 8 islamic bank bangladesh limited 0.4219 7 shahjalal islamic bank 0.3007 11 al-arafa islamic bank 0.4412 5 ijahp article: tarique, islam, mohamed, razak, hamdan/constructing a maqasid (objective) based performance measurement index for islamic banks international journal of the analytic hierarchy process 347 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.770 table 5, which provides the overall maqasid scores of the 11 banks, reveals that rhb bank topped the maqasid index with the highest maqasid score of 0.5152. the second and third positions were held by cimb islamic and bank rakyat, respectively. the index also shows that malaysian islamic banks performed better than the bangladeshi islamic banks. 4. analysis of results this research stands on the maqasid al-shari’ah framework which is multi-dimensional in nature. according to the conceptual framework, maqasid al-shari’ah can be achieved by the fulfillment of three shari’ah objectives, namely maslahah, educating individuals and justice. fulfilment of maslahah again depends on fulfilling necessity, complements and embellishments of the society. last, but not least, in order to fulfill necessity, the protection of the five essentials needs to be ensured. all of these dimensions and subdimensions have significant impact on the overall value of the maqasid index. table 6 provides the pertinent descriptive statistics on all the concepts discussed in the preceding section. table 6 descriptive statistics of maqasid index and its components dimensions statistics min max mean standard deviation skewness kurtosis maqasid index (mi) .3007 .5152 .4171 .0696 -.321 -.865 educating individuals .0000 .9999 .7808 .3809 -1.906 2.006 justice .0003 .9998 .4409 .2623 .757 1.440 maslahah .4607 .8209 .6387 .1297 -.020 -1.506 embellishments .0000 1.000 .7708 .3171 -1.845 2.970 complements .0000 1.083 .4687 .4128 .187 -1.699 necessity .3011 .8615 .6728 .1912 -1.023 .123 protection of religion (por) .0000 1.000 .6452 .3273 -.730 -.469 protection of life (pol) .0002 .9998 .8798 .2923 -3.292 10.885 protection of intellect (poi) .0000 .9999 .2071 .3012 2.101 4.833 protection of progeny (pop) .0007 1.000 .5161 .3923 .055 -2.090 protection of wealth (pow) .3054 1.7153 .6691 .4310 1.718 2.815 the mean score of the index among the sampled banks is 0.42. a low mean score suggests that overall the banks that were evaluated are not fully adhering to the maqasid al-shari’ah. this is evidence for the claim of some scholars who maintain that islamic banking activities are not fully in line with maqasid al-shari’ah (siddiqi, 2006; ijahp article: tarique, islam, mohamed, razak, hamdan/constructing a maqasid (objective) based performance measurement index for islamic banks international journal of the analytic hierarchy process 348 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.770 mohammed & shahwan, 2013; dusuki & bouheraoua, 2011). decomposition of the index revealed that the mean score for educating individuals and maslahah are quite high, 0.78 and 0.64, respectively. however, the mean score for justice of the sampled banks is 0.44. this may be the cause for the overall low means scored by the sampled banks. the low score of justice by the banks is evidence for the claim of the critics that ibs are not operating on the basis of justice. however, mohammed (2011) claimed that the concept of justice, equity and equality are ambiguous in the activities of islamic banks. from the descriptive statistics in table 6, it is clear that the variations in index value are very high in all five protections. in most of the cases, the minimum value is 0.00, whereas the maximum value is close to 1. in the case of por, bank rakyat scored the highest with a value 1.00 against the mean value of 0.6452 and a standard deviation (sd) of 0.3273. in the case of pol, poi and pop, first security islamic bank was first in all three with a value of 0.9998, 0.9999 and 1.00 against the mean value of 0.8798, 0.2071 and 0.5161, respectively. lastly, the descriptive statistics on pow reveal that cimb islamic bank scored the highest with a value of 1.7153 against the mean value of 0.6691 and a sd of 0.4310. next, the discussion highlights the scores related to necessity, complements and embellishments. first, the scores on necessity indicate the superiority of first security islamic bank with a value of 0.8615. cimb islamic and al-arafa bank ranked second and third with values of 0.8587 and 0.8541, respectively. the mean value and sd in this case are 0.6728 and 0.1912. second, in the case of complements, exim bank ranked first with a score of 1.083 against a mean of 0.4687 and sd of 0.4128. finally, the analysis of embellishments showed the superiority of exim bank with a value of 0.9999. cimb islamic and first security bank also scored very close (0.9963 and 0.9962, respectively). the mean value and sd in this category are 0.7708 and 0.3171. figure 5 presents the graphical view of the comparisons across the 11 banks. figure 5 maqasid score of necessity, complements and embellishments for the selected banks (n=11) 0.0000 0.5000 1.0000 1.5000 2.0000 2.5000 3.0000 necessity complements embellishments ijahp article: tarique, islam, mohamed, razak, hamdan/constructing a maqasid (objective) based performance measurement index for islamic banks international journal of the analytic hierarchy process 349 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.770 the analysis of the scores on maslahah reveals the superiority of rhb islamic with a value of 0.8209. the mean and sd in this case are 0.6387 and 0.1297. a low sd value in this case indicates that the maslahah scores for all of the sampled banks are close. in the case of educating individuals, exim bank scored the highest with a score of 0.9999 against a mean value of 0.7808 and sd of 0.3809. lastly, the descriptive statistics of the maqasid scores of justice revealed the superiority of rhb islamic bank with a score of 0.9998 against a mean value of 0.4409 and sd of 0.2623. the graphical presentation of these scores for the sampled banks can be found in figure 6. figure 6 maqasid score of educating individual, justice and maslahah for the selected banks (n=11) 5. specific contribution of the work performance evaluation of banks is an ongoing process. there are continued efforts to develop new and improved methods to capture the true performance of banks. as discussed earlier, with reference to objectives as well as operations, islamic banks are different from conventional banks. performance evaluation of islamic banks demands scrutinizing not only the financial performance, but also gauging how completely the banks have fulfilled the maqasid al-shari’ah. there are very few studies that have used maqasid al-shari’ah to evaluate islamic banks. in doing so, those studies either used al ghazali’s theory or abu zahrah’s theory. this study has combined both the theories to develop a new model. the previous studies that used maqasid al-shari’ah to measure the performance of islamic banks did not use any rigorous technique to assign weights to different maqasid al-shari’ah variables. most of the studies assigned equal weights to the variables. the studies that used unequal weights used expert opinions regarding the weights of maqasid -0.5 0 0.5 1 1.5 2 2.5 3 educating individual justice maslahah ijahp article: tarique, islam, mohamed, razak, hamdan/constructing a maqasid (objective) based performance measurement index for islamic banks international journal of the analytic hierarchy process 350 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.770 al-shari’ah variables and later averaged them. this study used the analytic hierarchy process (ahp) to prioritize the maqasid variables and items that are used to operationalize maqasid. the use of a scientific method such as ahp to evaluate islamic banking performance is the first of its kind. finally, the implementation of the maqasid index will not only help banks reach their social objective, but will also be fruitful commercially. fulfilling maqasid will create a positive brand image for islamic banks which will attract more muslims and non-muslim customers. therefore, this will create a larger scope for earning more revenue. 6. conclusion the most important job in the process of developing an index is the selection of appropriate, meaningful and measurable variables. in the present research, the use of an extensive literature review and sekaran’s method of operationalization helped select the primary variables. expert interviews and focus group discussion were arranged to finalize and validate the variables. the experts’ opinions greatly improved the measurability of the selected variables with respect to performance measurement of islamic banks based on maqasid al-shari’ah. next, appropriate weights were assigned for each maqasid concept, dimension and sub-dimension through another level of expert interviews. the data were analyzed using the analytic hierarchy process (ahp), a multi-criteria decision making tool. the results of the ahp analysis showed that the banks should give higher priority to justice followed by maslaha and educating individuals in order to achieve maqasid al-shari’ah. finally, an index was developed in accordance with the internationally acclaimed methodology by jrc-ec (2008). the developed index was then tested on some selected banks from malaysia and bangladesh. the results revealed that rhb islamic bank ranked top among the sampled banks. overall, the islamic banks of malaysia performed better than the islamic banks of bangladesh. the maqasid scores for the bangladeshi islamic banks were higher than the malaysian islamic banks with respect to fulfilling necessity; however, the low scores in complements and embellishments resulted in an overall low score for the bangladeshi banks. the index value was scaled within 0 to 1. the highest score earned by rhb islamic bank was 0.5152. the mean score of the sampled banks is 0.4171, which is evidence for the critics’ claim that islamic banks fall short in fulfilling the maqasid al shari’ah. necessary steps should be taken by islamic banks to improve the score on justice, necessity and complements to forge ahead in fulfilling maqasid al-shari’ah. ijahp article: tarique, islam, mohamed, razak, hamdan/constructing a maqasid (objective) based performance measurement index for islamic banks international journal of the analytic hierarchy process 351 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.770 references abu-zaharah, m. (1997). usul al-fiq. cairo: dar al-fikr al-‘arabi. ahmed, h. (2011). maqasid al-shari’ah and islamic financial products: a framework for assessment. isra international journal for islamic finance, 3(1), 140-160. akhter, w., akhtar, n., & jaffri, s. k. a. (2009). islamic micro-finance and poverty alleviation: a case of pakistan. proceeding of the 2 nd cbrc, lahore, 1-8. al sari, a.m. (2010). basic human needs: the islamic theory. unpublished thesis. algaoud, l. m., & lewis, m. k. (2007). islamic critique of conventional financing. handbook of islamic banking. london, uk: edward elgar. doi: https://doi.org/10.4337/9781847205414.00011 aliyu, s., hassan, m. k., mohd yusof, r., & naiimi, n. (2017). islamic banking sustainability: a review of literature and directions for future research. emerging markets finance and trade, 53(2), 440-470. doi: https://doi.org/10.1080/1540496x.2016.1262761 al-raysuni, a. (1992). nazariyyat al-maqàsidinda l-imam al-satibi. al-mu'assasa alyami'iyya li-l-dirasat wa-l-nasr wa-l-tawzi'. doi: https://doi.org/10.1163/18778054_cmr_com_25560 al-raysuni, a. (2005). imam al-shatibi's theory of the higher objectives and intents of islamic law. herndon, usa: international institute of islamic thought. doi: https://doi.org/10.2307/j.ctvkjb1w9 amin, r. m., yusof, s. a., haneef, m. a., muhammad, a., & oziev, g. (2015). the integrated development index (i-dex): a new comprehensive approach to measuring human development. islamic economic: theory, policy and social justice, 2(13), 159171. antonio, m. s., sanrego, y. d., & taufiq, m. (2012). an analysis of islamic banking performance: maqasid index implementation in indonesia and jordania. journal of islamic finance, 176(813), 1-18. ashur, m. a. t. i. (2006). ibn ashur: treatise on maqasid al-shariah. herndon, usa: the international institute of islamic thought. doi: https://doi.org/10.2307/j.ctvkc673b.21 atiyah, g. e. (2008). nahw taf’eel maqasid al-shariah [towards revitalizing almaqasid]. herndon, usa: the international institute of islamic thought. auda, j. (2008). maqasid al-shariah as philosophy of islamic law: a systems approach. herndon, usa: international institute of islamic thought (iiit). doi: https://doi.org/10.2307/j.ctvkc67tg https://doi.org/10.2307/j.ctvkjb1w9 ijahp article: tarique, islam, mohamed, razak, hamdan/constructing a maqasid (objective) based performance measurement index for islamic banks international journal of the analytic hierarchy process 352 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.770 bedoui, h. e. (2014). socio-economic index based on maqasid shariah. proceeding of developing a framework for maqasid-al-shariah based index of socio-economic development, 1, 33-76. doi: https://doi.org/10.1007/s11948-014-9561-9 bedoui, h. e., & mansour, w. (2014). performance and maqasid al-shari’ah’s pentagon-shaped ethical measurement. science and engineering ethics, 21(3), 555-576. chapra, m. u. (2000). the future of economics: an islamic perspective. leicester, u.k.: the islamic foundation. doi: https://doi.org/10.1007/s11948-014-9561-9 chapra, m. u. (2008, october). the global financial crisis: can islamic finance help minimize the severity and frequency of such a crisis in the future?. in a paper presented at the forum on the global financial crisis at the islamic development bank on (vol. 25). doi: https://doi.org/10.1057/9780230361133_5 choudhury, m. a. (2006). what is the scope of islamic economics and finance? journal of islamic economics and finance, 2(1), 1-42. collins, m. g., steiner, f. r., & rushman, m. j. (2001). land-use suitability analysis in the united states: historical development and promising technological achievements. environmental management, 28(5), 611-621. doi: https://doi.org/10.1007/s002670010247 dali, n. m., abdullah, a., & islam, r. (2018). prioritization of the indicators and subindicators of maqasid al-shariah in measuring liveability of cities. international journal of the analytic hierarchy process, 10(3), 348-371. doi: https://doi.org/10.13033/ijahp.v10i3.597 dusuki, a.w. & bouheraoua, s. (2011). the framework of maqasid al-shari’ah (objectives of the shari’ah) and its implications for islamic finance. isra research publication, 22, 316-336. ernst and young. (2014-15). world islamic banking competitiveness report. u.k. esen, m. f. a n d esen, a. (2014). identification and classification of maqasid alshariah terms for socio-economic development index. proceeding of developing a framework, for maqasid-al-shariah based index of socio-economic development, 1, 171-182. doi: https://doi.org/10.2139/ssrn.2469132 financial times. (2011). islamic finance (supplement, may12). hapsari, m. i. & herianigrum, s. (2014). maqashid index a success indicator of the social economic development. in the paper of international seminar “developing a framework for maqashid al-sharia-based index of socio economic development,” yogyakarta june 26th-27th. ibn‘ashur, m. a. t. (1998). maqāṣid al-sharīʿah al-islamiyyah (ed.) muhammad altahir al-misawi. kuala lumpur: al-basa’ir. doi: https://doi.org/10.2307/j.ctvkc673b.39 ijahp article: tarique, islam, mohamed, razak, hamdan/constructing a maqasid (objective) based performance measurement index for islamic banks international journal of the analytic hierarchy process 353 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.770 ibrahim, s. h. m. & momin, e. (2015). towards achieving a maqasid shariah oriented islamic banking. 10 th international conference on islamic economics and finance, 23-24 march, jeddah. ifdr. (2019). icd-refinitiv. islamic finance development report. joint research centre-european commission (jrc-ec). (2008). handbook on constructing composite indicators: methodology and user guide. paris: oecd publishing. doi: https://doi.org/10.1787/9789264043466-en julia, t., & kassim, s. (2019). exploring green banking performance of islamic banks vs conventional banks in bangladesh based on maqasid shariah framework. journal of islamic marketing, 11(3), 729-744. doi: https://doi.org/10.1108/jima-10-2017-0105 laela, s. f., rossieta, h., wijanto, s. h., & ismal, r. (2018). management accountingstrategy coalignment in islamic banking. international journal of islamic and middle eastern finance and management, 11(4), 667-694. doi: https://doi.org/10.1108/imefm04-2017-0088 mili, m. (2014). a structural model for human development: does maqāṣid al-sharīʿah matter! islamic economic studies, 130(1562), 1-19. doi: https://doi.org/10.12816/0008095 mohammed, m. o. (2011). maqasid based performance measurement system. unpublished doctoral dissertation. university sains malaysia, penang, malaysia. mohammad, m. o., & shahwan, s. (2013). the objective of islamic economic and islamic banking in light of maqasid al-shariah: a critical review. middle-east journal of scientific research, 13(13), 75-84. mohammed, m. o., & taib, f. m. (2009). testing the performance measured based on maqashid al-shariah (pmms) model on 24 selected islamic and conventional banks. in 2 nd langkawi insaniah-irti international conference (life), langkawi. malasyia. mohammed, m. o., & taib, f. m. (2015). developing islamic banking performance measures based on maqasid al-shari’ah framework: cases of 24 selected banks. journal of islamic monetary economics and finance, 1(1), 55-77. doi: https://doi.org/10.21098/jimf.v1i1.483 mohammed, m. o., tarique, k. m., & islam, r. (2015). measuring the performance of islamic banks using maqāṣid-based model. intellectual discourse, 23, 401-424. mohammed, m. o., razak, d. a., & taib, f. (2008). the performance measures of islamic banking based on the maqasid framework, paper presented in iium international conference (intac iv), 24-26 june, kuala lumpur. obaidullah, m., & latiff, h. s. h. a. (2008). islamic finance for micro and medium enterprises. islamic research & training institute islamic development bank, centre ijahp article: tarique, islam, mohamed, razak, hamdan/constructing a maqasid (objective) based performance measurement index for islamic banks international journal of the analytic hierarchy process 354 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.770 for islamic banking, finance and management universiti brunei darussalam. doi: https://doi.org/10.32350/ibfr.2017.04.03 puspitasari, n., rukmana, d. h., & sukarno, h. (2017). analysis of islamic banking efficiency using maqashid shariah approach (study on islamic banks in indonesia and malaysia). journal of finance and banking review, 2(4), 1-7. reni, d. m., & cholisini, a. (2014). impact of the perception of islamic bank management on islamic bank objective to the social and economic performance using maqāṣid al-sharī ‘ah approach. proceeding of developing a framework for maqāṣid al-sharī ‘ah based index of socio-economic development. yogyakarta: java, indonesia. doi: https://doi.org/10.35632/ajiss.v24i1.415 rulindo, r., mardhatillah, a., & hidayat, s. e. (2008). analysis of risk attitudes, spirituality, and shari’ah consciousness of current and prospective investors (an empirical survey in malaysia). islamic capital markets, 307-318. saaty, t. l. (2010). principia mathematica decernendi: mathematical principles of decision making: generalization of the analytic network process to neural firing and synthesis. pittsburgh: rws publications. seman, j. a., & dzolkarnaini, n. (2019). construction of maqāṣid al-sharīʿah-based index of socio-economic development: principles and issues. in towards a maqāṣid alsharīʿah index of socio-economic development (pp. 159-188). london: palgrave macmillan. doi: https://doi.org/10.1007/978-3-030-12793-0_5 siddiqi, m. n. (2006). islamic banking and finance in theory and practice: a survey of state of the art. islamic economic studies, 13(2), 1-48. takala, j., suwansaranyu, u., & phusavat, k. (2006). a proposed white‐collar workforce performance measurement framework. industrial management & data systems, 106(5), 644-662. doi: https://doi.org/10.1108/02635570610666421 usmani, t. (2008). islamic finance: musharakah & mudarbah. journal of islamic banking and finance, 25(3), 41-53. zaheer, s., & rasool, w. (2017). maqasid al shariah index for islamic banking institutions. journal of islamic business and management, 7(1), 28-38. doi: https://doi.org/10.26501/jibm/2017.0701-003 ijahp article: cooper/ the analytic network process: astronomy, icebergs, and big box stores international journal of the analytic hierarchy process 135 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.566 the analytic network process: astronomy, icebergs, and big box stores orrin cooper university of memphis, 332 fogelman college business admin bldg. memphis, tn 38152 olcooper@memphis.edu phone: 1 901 678 4546 fax: 1 901 678 4015 abstract the analytic network process (anp) is a disruptive technology that has had significant impacts in the field of decision-making. by drawing on an analogy to the field of astronomy we can see that even with all that has been developed we must avoid the illusion of thinking that the field is mature and fully discovered. the anp has many parallels to icebergs from what portion is visible to the value of providing relevant warning products. one of the most important contributions going forward will be the discovery of the more complex and hidden relationships and tests that anp decision makers can use to test their models. these discoveries will improve both the reputation of the anp and decision maker’s confidence in their models. without these discoveries, the anp runs the risk of becoming like a big box retail store. keywords: analytic network process (anp); dependence; data quality; judgmental data 1. introduction in 1888 newcomb said, "we are probably nearing the limit of all we can know about astronomy" (wolff, 1999). later in 1903 newcomb said, “what lies before us is an illimitable field, the existence of which was scarcely suspected ten years ago, the exploration of which may well absorb the activities of our physical laboratories, and of the great mass of our astronomical observers and investigators for as many generations as were required to bring electrical science to its present state” (newcomb, 1903). “about 7/8 ths of an iceberg is below the water line” (us coast guard, 2018). “big box retail stores are losing relevance, while e-commerce and specialty stores grow in appeal” (yohn, 2016). with all that has been published regarding the analytic network process (anp), one may be tempted to say the same as newcomb said about astronomy that we are reaching the limit of what can be known about the anp. while applications of the anp will obviously continue, there is also much yet to discover in terms of the theory of the anp. this theory must be discovered and unraveled, otherwise the anp may become much like big box stores. while big box stores will not entirely disappear in the near future, they are losing considerable market share. in 2008, the anp had the largest “market share” in terms of published multi criteria decision making methods (wallenius et al., 2008). the ijahp article: cooper/ the analytic network process: astronomy, icebergs, and big box stores international journal of the analytic hierarchy process 136 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.566 direction of the anp research in the next few years will be a major determining influence on how the anp will be positioned going forward. “a general network representing the appropriate connections among its elements” on page 258 of saaty’s 1977 publication regarding the ahp might be considered the first reference to the anp. two possible publications that qualify for the formal debut of the anp are saaty (1996) and saaty & takizawa (1986). the validity of the method has been demonstrated through case studies (whitaker, 2007). consistency, the most commonly used method to test for the quality of the judgement data, has received a great deal of attention (brunelli, canal, & fedrizzi, 2013; grzybowski, 2016). with all that has been done regarding the anp, there are reasons to feel that the anp is mature and identify with what newcomb said in 1888 about astronomy. looking back, it is easy to see that there was, and still is, so much more to discover about astronomy. the same can and will be said of the anp. recognizing the potential for further development of the theory of the anp is an important first step. searching for the new discoveries will uncover additional theory that once discovered and visible, will eventually become common knowledge that can increase the quality and scope of anp applications. 2. icebergs using an analogy of an iceberg when talking about the anp is insightful. when one comes upon an iceberg, the great beauty and grandeur of what is above the water can be recognized. the visible portion provides a trove of information that can be studied and modeled. however, somewhere around 7/8 ths of the iceberg is unseen because it is below the water (us coast guard, 2018). to capture and more accurately model iceberg behavior, the unseen 7/8 ths must also be discovered, studied, and addressed, as well as continuing to study other properties about the visible portion. scientists are still learning about and incorporating in the analysis these visible properties like the age of the ice or chemical make-up. there are also external factors like weather conditions that can be added to the analysis. in 1913, nations recognized the value of studying and tracking icebergs and formed the international ice patrol (iip). part of the iip mission is to “provide relevant iceberg warning products” (us coast guard, 2017). there are similarities between icebergs and the anp. the part of the anp that “can be seen” is amazing and has been used to solve complex problems. however, just as with the submerged portions of icebergs present unseen/undetected dangers that when uncovered have and will significantly improve iceberg models, the same applies to the “submerged portions” of the anp. we should view the anp as an iceberg and search for more than what is already visible. there may also be more to learn about what is already developed, as in the case of the part of the iceberg above the water. careful analysis of external factors can lead to other important opportunities for integration to improve upon the anp at present. finally, other statistical and decision-making methods have relevant and sometimes very comprehensive warning products that can be assimilated with the iip for icebergs. for example, in structural equation modeling (sem) there are different tests to check for many different types of validity and reliability. the most accepted “quality” tests for the anp are the consistency index and case studies of comparing model results to actual values. the prestige and reputation of the anp will increase as more “validity” and “reliability” tests are developed and integrated or even regulated as best practices as a type of iip for the anp. ijahp article: cooper/ the analytic network process: astronomy, icebergs, and big box stores international journal of the analytic hierarchy process 137 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.566 critics have tried to attack the validity of anp, in some cases due to a lack of understanding, but possibly because they came across an issue that was still below the water. reviewing those critiques is not the purpose of this article. the important point is to recognize there is more we can learn about the anp theory. i had this experience during my doctoral studies while trying to build and validate anp models to better understand each part of the supermatrix. some of the “simple” decisions i tried to model had other issues, and the final priority vectors seemed counterintuitive. i became frustrated and almost abandoned the method. thankfully, shortly thereafter, i had the impression to try two more models; for one the final priority vector worked and for the other it did not. comparing the two models led to the conclusion and mathematical proof that when converting the unweighted supermatrix to a weighted supermatrix to fully capture the dependency in anp models, one should perform the cluster weighting comparisons individually for each alternative or criterion column and not just normalize all of the columns in a cluster equally. by making the additional cluster weighting comparisons for each column, the dependency of each individual criterion or alternative is more precise (cooper & liu, 2017). in terms of capturing the full potential of the other “7/8 ths” of the anp, it has not been done yet simply because we do not even know what it is at present – it is still submerged. by not pushing and testing the anp theory, we have not yet discovered its full potential and current limits. the ability of the anp to deal with intangibles can provide what on the surface appears to be a real challenge to validation; but intangibles do not have to be the red herring towards validation. it is also important to develop and prove the mathematical theory behind these new discoveries as they are more robust than using case studies alone. the integration of stakeholders into anp models is one example of an anp iceberg where many applications have been published and are now visible, but the theory is still submerged. for example, putting all the stakeholders in a criteria cluster and making connections to the related criteria will lead to significantly different outcomes than if the stakeholders are represented with different networks. the modeling of stakeholders varies widely even for similar decisions. there should be room for customization, but without a theoretical justification it is also fertile ground for hidden or unintended results. the theoretical implications of how to include stakeholders is an example of one important contribution yet to be uncovered. another area of the anp iceberg that is “under the surface” which will provide some of the most significant contributions to the theory of the anp is in data quality tests/checks. both while peer reviewing an article and working with students with “good” models, i have seen that some models have worked very well and others did not. what made the difference? what test could i have told them to run to identify the difference? consistency tests are necessary, but not sufficient. it will become more important going forward to have best practices, more testing/screening tools and the ability to identify more exceptions/problems for anp models. from the beginning, saaty recognized that such tests were worth pursuing, “as yet there is no statistical theory (to the best of our knowledge) which would assist us in deciding how well judgmental data corresponds to reality. … it is clear that this is an area of research that is worth pursuing” (saaty, 1977, emphasis added). there are relationships and tests that are more complex than what have been discovered that will help test the ijahp article: cooper/ the analytic network process: astronomy, icebergs, and big box stores international journal of the analytic hierarchy process 138 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.566 models, designs, inputs, and relationships and in turn provide additional unrealized benefits. coherency is one example of an innovative and important data quality check to test a sort of consistency at the level of the entire supermatrix that is very crucial since without this check the decisions cannot be trusted (cooper & yavuz, 2016; yavuz & cooper, 2017). this however is just the beginning, both in terms of additional tests to improve/test data quality, and in terms of the idea of coherency testing. bigger and brighter minds will take coherency and other data checks to a completely new level. these additional tests will serve at least two important purposes. first, they will serve as a shield or filter to protect the integrity of the anp and increase its reputation. the tests will help address the idea of garbage in leads to garbage out because then if a model does not pass the quality tests one can argue that the anp is not being done. the filters and shields will also reduce misunderstandings about the anp. second, more ways to test or prove something is a good thing. anp models, in particular, take a significant amount of work to design and complete. intentional decision makers should be asking, ‘what can we do to increase our confidence in the meaning of the final answers?’ if they could show that the model passes this test, and this test, and this test, etc. then it could increase their confidence and motivation to engage in the work required to get “real” results. 3. big box stores the anp works and is very useful in its current form and has been used in incredible applications. however, if we fail to continue to innovate, to differentiate and adapt to the market then the anp may become like a big box store retailer in multi criteria decision making. some may argue it is like differentiation that is universal, and will stand the test of time. however, it is also possible that newer methods that are able to use other means to achieve many of the same advantages of the anp with cheaper costs or customized applications to specific types of problems will be similar to specialty stores that are challenging big box stores. big data and biometrics have the potential to be the ecommerce that disrupts much of the need to use intangibles/anp because metrics will be attainable/generated through these other tools. they also provide a tremendous opportunity for integration with the anp. unless the anp theory continues to be developed, the magic of the anp will likely never disappear, but other methods will provide similar benefits and potentially have other benefits or solve challenges that exist in the anp. recognizing that the limits of the anp theory have not been reached and thinking of the anp as an iceberg, breaks down an important barrier and opens the door to making new discoveries. it is much like breaking the four-minute mile barrier, and opening the door for new and more rapid and frequent advancements. the theory of the anp can be advanced in many ways. some of the most valuable contributions will be in terms of the data quality tests that need to be developed. these discoveries will both improve the reputation of the anp, and increase decision maker’s confidence in their models. without these advancements, the anp may become synonymous with big box stores. ijahp article: cooper/ the analytic network process: astronomy, icebergs, and big box stores international journal of the analytic hierarchy process 139 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.566 references brunelli, m., canal, l., & fedrizzi, m. (2013). inconsistency indices for pairwise comparison matrices: a numerical study. annals of operations research, 211(1), 493509. doi: https://doi.org/10.1007/s10479-013-1329-0 cooper, o., & liu, g. (2017). achieving the desired level of dependency in anp decision models. international journal of the analytic hierarchy process, 9(1), 2-26. doi: https://doi.org/10.13033/ijahp.v9i1.450 cooper, o. & yavuz, i. (2016). linking validation: a search for coherency within the supermatrix. european journal of operational research, 252(1), 232-245. doi: https://doi.org/10.1016/j.ejor.2015.12.045 grzybowski, a. z. (2016). new results on inconsistency indices and their relationship with the quality of priority vector estimation. expert systems with applications, 43, 197212. doi: https://doi.org/10.1016/j.eswa.2015.08.049 newcomb, s. (1903). the universe as an organism. science, 17(421), 121-129. doi: 10.1126/science.17.421.121 saaty, t. l. (1977). a scaling method for priorities in hierarchical structures. journal of mathematical psychology, 15(3), 234-281. doi: https://doi.org/10.1016/00222496(77)90033-5 saaty, t. l. (1996). decision making with dependence and feedback: the analytic network process. pittsburgh, pa: rws publication. saaty, t. l., & takizawa, m. (1986). dependence and independence: from linear hierarchies to nonlinear networks. european journal of operational research, 26(2), 229-237. doi: https://doi.org/10.1016/0377-2217(86)90184-0 us coast guard, (2017). about international ice patrol (iip). 1/31/2018, from https://www.navcen.uscg.gov/?pagename=iiphome us coast guard, (2018). how much of an icebery is below the water. 2018, from https://www.navcen.uscg.gov/?pagename=iiphowmuchofanicebergisbelowthewater wallenius, j., dyer, j. s., fishburn, p. c., steuer, r. e., zionts, s., & deb, k. (2008). multiple criteria decision making, multiattribute utility theory: recent accomplishments and what lies ahead. management science, 54(7), 1336-1349. doi: 10.1287/mnsc.1070.0838 whitaker, r. (2007). validation examples of the analytic hierarchy process and analytic network process. mathematical and computer modelling, 46(7-8), 840-859. wolff, s. (1999). the aas: its next 100 years. paper presented at the bulletin of the american astronomical society. https://www.navcen.uscg.gov/?pagename=iiphome https://www.navcen.uscg.gov/?pagename=iiphowmuchofanicebergisbelowthewater ijahp article: cooper/ the analytic network process: astronomy, icebergs, and big box stores international journal of the analytic hierarchy process 140 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.566 yavuz, i., & cooper, o. (2017). a dynamic clustering method to improve the coherency of an anp supermatrix. annals of operations research, 254(1-2), 507-531. doi: https://doi.org/10.1007/s10479-017-2403-9 yohn, d. (2016). big-box retailers have two options if they want to survive. harvard business review. https://hbr.org/2016/06/big-box-retailers-have-two-options-if-theywant-to-survive a style guide for papers presented to the international journal of the analytic hierarchy process ijahp article: mm allama, ms parvez, sm ali, a azeem/ using the analytical hierarchical process to prioritise safety management elements in manufacturing in bangladesh international journal of the analytic hierarchy process 104 vol. 4 issue 2 2012 issn 1936-6744 an analytical hierarchical process for prioritising the safety management elements for a manufacturing organization in bangladesh m.m. allama department of industrial and production engineering (ipe) bangladesh university of engineering and technology (buet) dhaka-1000, bangladesh m.s. parvez department of industrial and production engineering (ipe) bangladesh university of engineering and technology (buet) dhaka-1000, bangladesh s.m. ali department of industrial and production engineering (ipe) bangladesh university of engineering and technology (buet) dhaka-1000, bangladesh email: mithun@ipe.buet.ac.bd dr. a. azeem department of industrial and production engineering (ipe) bangladesh university of engineering and technology (buet) dhaka-1000, bangladesh abstract this paper identifies typical hazard and risk elements in a manufacturing organization, studies management elements currently used in the organization, and presents an analytic hierarchy process (ahp) decision model for assessing the priority of safety management elements. specifically, the paper addresses a hierarchy decision model for assessing the priority of safety management elements of a battery manufacturing company in bangladesh. safety management elements and decision criteria are identified by using osha and niosh standards. empirical data are collected through personal interviews with safety personnel, experts and professionals in the battery manufacturing company and through spot surveying. using the analytical hierarchy process, a list of six decision criteria and ten safety management elements, which constitute the ahp alternatives, are identified and their relative importance is evaluated. using ahp methodology, the top three safety elements that have been identified and are needed to implement a safety management system (sms) include a personal protection program, emergency preparedness and safety organization. the identification of core decision criteria and safety management elements found in this research may be useful for effective implementation of sms in manufacturing organizations. mailto:mithun@ipe.buet.ac.bd ijahp article: mm allama, ms parvez, sm ali, a azeem/ using the analytical hierarchical process to prioritise safety management elements in manufacturing in bangladesh international journal of the analytic hierarchy process 105 vol. 4 issue 2 2012 issn 1936-6744 keywords: analytical hierarchy process (ahp), safety management elements (sms), health and safety, decision making, manufacturing organization. 1. introduction a safety management system (sms) is an explicit element of the corporate management’s responsibility which explains a company’s safety policy and defines how it intends to manage safety as an integral part of its overall business. current trends show that organizations are beginning to invest attention into the organizational and management impact of safety performance, particularly the function of health and safety management. the interest in health and safety management is a result of major disasters that highlighted the failure of management to protect the health and safety of their workers and comply with the occupational health and safety act which calls them to fulfill their responsibilities as an employer to ensure that workers have a safe work-place (hale, heming, carthey & kirwan, 1997). in general, legislation is inadequate to address problems that many organizations experience in managing health and safety in the workplace. this is largely due to the “people” element where people tend to engage in safe or unsafe behavior according to their own interpretation of the rules, and that unsafe behavior then leads to accidents. safety experts fleming and lardner (1999) discovered that human factors contributed to 80 – 90% of all industrial accidents as people neglected the correct procedure for performing their job. for this reason, effective health and safety management and its relation to productivity have been considered an important element when managing the interaction between systems and people. currently, safety and health protection has become a major positive factor in favor of economic growth and productivity (abdul raouf, 2004). industry concerns have focused on identification and assessment of potential risks and elimination of unaccepted risks. a safety management system (sms) contains a number of elements including safety policy, job hazard analysis and safety and health awareness etc. this provides guidance for enterprises to manage risks and improve their safety and health performance. a sms facilitates occupational safety and health management by providing systematic approaches for continual identification, evaluation and control of hazards and risks (rondinelli and berry, 2000). ahasan (1993, 1994, 2000b), khan (1994, 1997, 2000) and raihan (1997) have investigated many industries in bangladesh and found that the majority of them do not have a long-term stake in environmental and climatic considerations. some others (fariduddin et al., 1975; rahman, 1993; ahmad et al., 1997; sadeque et al, 1998) surveyed different workplaces in bangladesh and analyzed local workers’ energy expenditure, and health, safety and ergonomic issues. rahman et al. (2000) expressed concern about using injury information for injury surveillance at the local level in bangladesh. from these studies, it has been proven that workers usually worked long hours in unsafe conditions without using any personal protective devices (ppds), for instance. therefore, most of the workers' health, safety and well-being were deteriorating. it is also well known that the present status of occupational health and safety/ ergonomics is still at the rudimentary level in bangladesh because the factory owners (fo) and employers’ associations (ea) usually consider these elements a costly luxury. it is also true that they usually lack money, resources and other elements to provide improved health and safety facilities to all the workers. in bangladesh, field surveys and rob typewritten text http://dx.doi.org/10.13033/ijahp.v4i2.78 ijahp article: mm allama, ms parvez, sm ali, a azeem/ using the analytical hierarchical process to prioritise safety management elements in manufacturing in bangladesh international journal of the analytic hierarchy process 106 vol. 4 issue 2 2012 issn 1936-6744 workplace inspections are conducted, however, only a few such studies and research have been conducted based on in-depth case studies. workers’ physical workload, heat stress and thermoregulatory related studies were conducted (ahasan et al. 1997), but those studies do not contain all types of empirical data and ergonomic information. some anthropometric data of female garment workers (khan 1997) and other work-related information can be found in a few studies (sadeque et al., 1997; ahasan et al., 1997d,e; ahasan et al., 1998b). other studies have highlighted female workers’ economic, social and health aspects (rahaman et al., 1991; majumder & chowdhury, 1992; zohir & majumder, 1996). these studies may contribute some data and information, but they do not emphasize the practical importance of ohs and ergonomic applications. there is a pressing need to investigate the current industry situation in safety and health management and identify decision criteria for determining the implementation priority of safety management elements for manufacturing industries of bangladesh. this research aims to present a hierarchy decision model for assessing the priority of safety management elements. using the analytical hierarchy process (ahp) methodology, the model addresses the identification of relative importance of decision criteria and safety management elements with respect to the effective sms implementation in manufacturing enterprises. 2. objectives the objective of the present work is to study how the tools and techniques of the analytical hierarchy process (ahp) can be applied to implement a safety management system in a manufacturing company in bangladesh. to conduct the study, industrial hazards and several safety elements will be identified. then, depending on the importance given by the industrial professional safety management elements, the elements will be prioritized using the analytical hierarchy process (ahp). 3. methodology in order to investigate the relative priorities of the safety management elements for a manufacturing company, empirical data will be collected. from this information, an analytical hierarchy model will be developed to prioritize safety management elements. the methodology will be as follows: (1)identification of safety management elements and decision criteria most important to an effective safety management system using osha (occupational safety and health act) and niosh (national institution for occupational safety and health) standards; (2)acquisition of empirical data from personal interviews with evaluators such as safety personnel, experts and professionals in the industry who can evaluate the importance of decision criteria and safety management elements with respect to the corporate goals, resources and constraints in their organizations; (3)formulation of an analytical hierarchy model based on the evaluation to prioritize safety management decision criteria and relative importance of safety management elements. the process is summarized in the flowchart below. ijahp article: mm allama, ms parvez, sm ali, a azeem/ using the analytical hierarchical process to prioritise safety management elements in manufacturing in bangladesh international journal of the analytic hierarchy process 107 vol. 4 issue 2 2012 issn 1936-6744 4. conduct of an ahp study a list of attributes is generated and consolidated once ahp is confirmed as an appropriate solution method. ahp offers a two-stage process to cope with the complexity of the problem. the first of these two stages involves hierarchy construction. the attributes are organized in a hierarchy-type structure that reflects their mutual relationships. the primary goal of the research occupies the highest level of the structure, followed by “sets of attributes,” which are organized in several more hierarchy levels. a typical second-level attribute set includes all of the secondary goals that together contribute to achieving the primary goal. the purpose of this study was to determine the priority of decision criteria and elements for sms implementation in manufacturing enterprises. to investigate the relative priorities of the safety management elements for manufacturing industries, the authors initiated a study in bangladesh using the ahp methodology. the study involves the decomposition of a complex problem into a multilevel hierarchical structure of characteristics and criteria with the last hierarchical level constituting the decision alternatives. ahp can accommodate both objective and subjective judgments of the evaluators involved in order to make trade-offs and determine priorities among them. the goal of sms implementation was decomposed into six criteria identified from interviews with industrial personnel. these criteria are: (1) client requirement, (2) employee requirement, (3) insurance company requirement, (4) cost effectiveness, (5) effect on production rate, and (6) expertise requirement. a group of five industry experts were interviewed to evaluate these criteria and elements. the identify safety management elements and decision criteria acquire empirical data develop weights for decision criteria prioritize decision criteria and safety management elements using ahp develop rating for each safety management for each decision criterion calculate weighted average rating for each decision alternative choose the one with the highest score ijahp article: mm allama, ms parvez, sm ali, a azeem/ using the analytical hierarchical process to prioritise safety management elements in manufacturing in bangladesh international journal of the analytic hierarchy process 108 vol. 4 issue 2 2012 issn 1936-6744 evaluators included two safety professionals, two manufacturing specialists, and one top level manager. after the evaluation of these criteria ten safety management elements were evaluated. these were collected from the safety manuals of different manufacturing industries of bangladesh. these elements are: (1) safety organization, (2) safety policy, (3) safety training, (4) in-house safety rules, (5) personal protection program, (6) accident/incident investigation, (7) emergency preparedness, (8) safety committee, (9) job hazard analysis, and (10) occupational health assurance program. these are shown in table 1. table 1 ten safety management elements safety elements definition safety management elements normalized weight safety organization a hierarchy that ensures effective implementation of safety and health at work place. safety policy a legal requirement for an employer to prepare, and keep a written, up-to-date statement of their policy regarding the health and safety of their employees. safety training training to make the employees competent in health and safety. in-house safety rules rules for achieving safety management objectives. personal protection program a program to learn about standard precautions to prevent skin and mucous membrane exposure when in contact with blood and body fluids. accident/incident investigation a process to determine the cause or causes of an accident or series of accidents so as to prevent further incidents of a similar kind. emergency preparedness the capability that enables an organization or community to respond to an emergency in a coordinated, timely, and effective manner to prevent the loss of life and minimize injury and property damage. safety committee committee composed of medical, dental, nursing, engineering, administrative, and other staff members whose purpose is to oversee safety practice. job hazard analysis analysis that can be used to define and control the hazards associated with a certain process, job or procedure. occupational health assurance program development and maintenance of health awareness in a work place. weights of individual elements are compared and their respective priorities are obtained. inconsistency ratios are calculated to verify the consistency of the comparison process. ijahp article: mm allama, ms parvez, sm ali, a azeem/ using the analytical hierarchical process to prioritise safety management elements in manufacturing in bangladesh international journal of the analytic hierarchy process 109 vol. 4 issue 2 2012 issn 1936-6744 5. analysis and results 5.1 evaluation of decision criteria the industrial personnel have made pair wise comparison of decision criteria. from their evaluation a single pair-wise comparison matrix is developed which is shown in table 2. table 2 empirical data collection for decision criteria decision criteria of sms implementation client requirement (a1) employee requirement (a2) insurance requirement (a3) cost effectiveness (a4) effect on production rate (a5) expertise requirement (a6) client requirement(b1) 1 1/3 3 1/3 1/5 1 employee requirement(b2) 3 1 7 1 1/3 7 insurance requirement(b3) 1/3 1/7 1 1/9 1/9 1/3 cost effectiveness(b4) 3 1 9 1 1/3 5 effect on production rate(b5) 5 3 9 3 1 7 expertise requirement(b6) 1 1/7 3 1/5 1/7 1 then multiplying the values in each row together and calculating the nth root of said product normalizing the aforementioned nth root of products to get the appropriate weights and calculating and checking the consistency ratio (cr) which is shown below. geometric mean for client requirement g1 = (a1b1*a2b1*a3b1*a4b1*a5b1*a6b1) ^ (1/n) = (1*1/3*3*1/3*1/5*1) ^ (1/6) = 0.64197 normalized weight for client requirement x1 = 𝑔1∑ 𝑔𝑖6𝑖=1 = 0.64197 8.92 = 0.07197 consistency index, c.i = 𝜆𝑚𝑎𝑥−6 6−1 = 0.05332 where 𝜆𝑚𝑎𝑥= y1x1 + y2x2 + y3x3 + y4x4 + y5x5 + y6x6 = 6.26661 from saaty’s table, the randomly generated consistency index r.i. (for n=6) = 1.24 col. sums y1=13.33 y2=5.62 y3=32 y4=5.64 y5=2.12 y6=21.33 ijahp article: mm allama, ms parvez, sm ali, a azeem/ using the analytical hierarchical process to prioritise safety management elements in manufacturing in bangladesh international journal of the analytic hierarchy process 110 vol. 4 issue 2 2012 issn 1936-6744 the consistency ratio, c.r = 𝐶.𝐼 𝑅.𝐼 = 0.05332 1.24 = .043 or, 4.3% < 10%; so, the decision criteria in terms of c.r are acceptable. from the above calculation, the final normalized weights and priorities of the decision criteria for sms implementation are shown in table 3. table 3 weights and priority of decision criteria for sms implementation decision criteria of sms implementation normalized weight priority client requirement 0.07197 4 employee requirement 0.21445 2 insurance requirement 0.02711 6 effect on production rate 0.42176 1 cost effectiveness 0.21142 3 expertise requirement 0.05382 5 consistency ratio = 0.043 the top three criteria are identified as effect on production rate, employee requirement and cost effectiveness. 5.2 evaluation of safety management elements a pair-wise comparison matrix for each criterion, with each matrix containing the pairwise comparisons of the performance of safety management elements on each criterion is developed. this is done by multiplying the values in each row together and calculating the nth root of said product, normalizing the aforementioned nth root of product values to get the corresponding ratings. these pair-wise comparison matrices are shown in tables 4-9. ijahp article: mm allama, ms parvez, sm ali, a azeem/ using the analytical hierarchical process to prioritise safety management elements in manufacturing in bangladesh international journal of the analytic hierarchy process 111 vol. 4 issue 2 2012 issn 1936-6744 table 4 weights evaluation of safety management elements with respect to client requirement safety management elements normalized weight safety organization 0.32 safety policy 0.15 safety training 0.07 in-house safety rules 0.04 personal protection program 0.04 accident/incident investigation 0.02 emergency preparedness 0.04 safety committee 0.14 job hazard analysis 0.12 occupational health assurance program 0.07 table 5 weights evaluation of safety management elements with respect to employee requirement safety management elements normalized weight safety organization 0.04 safety policy 0.02 safety training 0.06 in-house safety rules 0.06 personal protection program 0.27 accident/incident investigation 0.10 emergency preparedness 0.17 safety committee 0.02 job hazard analysis 0.10 occupational health assurance program 0.17 table 6 ijahp article: mm allama, ms parvez, sm ali, a azeem/ using the analytical hierarchical process to prioritise safety management elements in manufacturing in bangladesh international journal of the analytic hierarchy process 112 vol. 4 issue 2 2012 issn 1936-6744 weights evaluation of safety management elements with respect to insurance requirement safety management elements normalized weight safety organization 0.07 safety policy 0.21 safety training 0.06 in-house safety rules 0.17 personal protection program 0.12 accident/incident investigation 0.03 emergency preparedness 0.05 safety committee 0.18 job hazard analysis 0.03 occupational health assurance program 0.08 table 7 weights evaluation of safety management elements with respect to effect on production rate safety management elements normalized weight safety organization 0.18 safety policy 0.10 safety training 0.03 in-house safety rules 0.08 personal protection program 0.17 accident/incident investigation 0.04 emergency preparedness 0.18 safety committee 0.03 job hazard analysis 0.10 occupational health assurance program 0.10 table 8 ijahp article: mm allama, ms parvez, sm ali, a azeem/ using the analytical hierarchical process to prioritise safety management elements in manufacturing in bangladesh international journal of the analytic hierarchy process 113 vol. 4 issue 2 2012 issn 1936-6744 weights evaluation of safety management elements with respect to cost effectiveness safety management elements normalized weight safety organization 0.13 safety policy 0.18 safety training 0.05 in-house safety rules 0.18 personal protection program 0.06 accident/incident investigation 0.06 emergency preparedness 0.12 safety committee 0.03 job hazard analysis 0.08 occupational health assurance program 0.12 table 9 weights evaluation of safety management elements with respect to expertise requirement safety management elements normalized weight safety organization 0.04 safety policy 0.20 safety training 0.13 in-house safety rules 0.09 personal protection program 0.03 accident/incident investigation 0.09 emergency preparedness 0.05 safety committee 0.17 job hazard analysis 0.14 occupational health assurance program 0.06 the composite weight for each safety management element is calculated by summation of the products of each decision criterion and safety management element. the overall implementation priorities of individual elements are determined by combining the element evaluation results against criteria from the final evaluation of safety management elements which is shown in table 10. ijahp article: mm allama, ms parvez, sm ali, a azeem/ using the analytical hierarchical process to prioritise safety management elements in manufacturing in bangladesh international journal of the analytic hierarchy process 114 vol. 4 issue 2 2012 issn 1936-6744 for an example, the composite weight for safety organization is computed as follows: 0.07197*0.32+0.21445*0.04+0.02711*0.07+0.21142*0.13+0.42176*0.18 +0.05*0.04 = 0.13906 table 10 overall weights evaluation of safety management elements safety management elements client require -ment (0.072) employee requirement (0.214) insurance require -ment (0.027) cost effectiveness (0.211) effect on production rate (0.422) expertise require -ment (0.05) composite weight overall ranking safety organization 0.32 0.04 0.07 0.13 0.18 0.04 0.139 3 safety policy 0.15 0.02 0.21 0.18 0.10 0.20 0.112 5 safety training 0.07 0.06 0.06 0.05 0.03 0.13 0.050 9 in-house safety rules 0.04 0.06 0.17 0.18 0.08 0.09 0.097 7 personal protection program 0.04 0.27 0.12 0.06 0.17 0.03 0.150 1 accident/ incident investigation 0.02 0.10 0.03 0.06 0.04 0.09 0.058 8 emergency preparedness 0.04 0.17 0.05 0.12 0.18 0.05 0.145 2 safety committee 0.14 0.02 0.18 0.03 0.03 0.17 0.047 10 job hazard analysis 0.12 0.10 0.03 0.08 0.10 0.14 0.098 6 occupational health assurance program 0.07 0.17 0.08 0.12 0.10 0.06 0.114 4 the top three safety management elements for battery industries are identified as (1) personal protection program, (2) emergency preparedness, and (3) safety organization. ijahp article: mm allama, ms parvez, sm ali, a azeem/ using the analytical hierarchical process to prioritise safety management elements in manufacturing in bangladesh international journal of the analytic hierarchy process 115 vol. 4 issue 2 2012 issn 1936-6744 6. conclusion safety management systems (sms) is a term used to refer to comprehensive systems designed to manage the safety, health, environmental and general risk aspects of an industry. an sms is the specific application of quality management to safety. safety management systems (sms) help companies identify safety risks before they become bigger problems. the safety management system approach moves beyond the traditional reactionary system to one which tries to predict areas of exposure. this is done through assessment of any residual risk areas in airworthiness and operations, and then supplementing these with operational knowledge and professional judgment. therefore, sms implementation provides guidance for manufacturing enterprises in managing and improving safety and health performance. the ultimate goal is to implement, as far as possible, the applicable safety management elements in order to maximize the effectiveness of improving safety and health performance. however, to implement all or most elements concurrently is impractical especially for small and medium enterprises. therefore, the priority of implementation can be determined for manufacturing enterprises so that they can focus their resources on fewer elements at a time. industrial personnel can evaluate the relative importance of safety management elements with respect to the corporate goals, resources and constraints of their respective organization using the analytical hierarchy process. the top three elements for battery industries were identified as (1) personal protection program, (2) emergency preparedness, and (3) safety organization. interestingly, though safety policy is a vital element for sms implementation it ranked fifth place. the evaluators were mainly concerned about individual safety rather than hazard source identification and removal. in this paper a hierarchy decision model was established to prioritize safety management elements for battery manufacturing industries. it will help industries to determine these elements in the design, establishment and implementation of their sms. an effective sms will provide a means of achieving enhanced safety performance which meets or exceeds basic compliance with the regulatory requirements associated with safety and quality. it is a well-established fact that enhanced safety performance is founded upon a proactive safety culture inherent in all the organization’s safety related activities. ijahp article: mm allama, ms parvez, sm ali, a azeem/ using the analytical hierarchical process to prioritise safety management elements in manufacturing in bangladesh international journal of the analytic hierarchy process 116 vol. 4 issue 2 2012 issn 1936-6744 references abdul raouf, s.i. (2004). productivity enhancement using safety and maintenance integration an overview, kybernetes, 33(7), 1116-26. ahasan, mr (1993). an ergonomic study on the manual pump for lifting surface water. in: nielsen r and jorgensen k (eds), advances of industrial ergonomics and safety–v (659-665). london: taylor & francis. ahasan, mr (1994). small and cottage industries in bangladesh–ergonomic perspective, proposals and prospects for the development of working conditions and environment. licentiate thesis, university of oulu, finland. ahmad, sa, ahasan, mr, salim-ullah, mh, hadi ska & khan aw (1997). respiratory symptoms among textile mill workers in bangladesh. in: khalid hm (ed), human factors vision–care for the future (111-116), proceedings of the 5th south east asian ergonomics society conference (kuala lumpur, malaysia), international ergonomics association press. fariduddin, km, rahman, mm & ahsanullah, ab (1975). study of energy expenditure and food intake of some working class people of bangladesh. bangladesh medical research council bulletin, 1(1), 24–30. fleming, m. & lardner, r. (1999). safety culture – the way forward. the chemical engineer, 685, 16-18. hale, a.r., heming, b.h.j., carthey, j. & kirwan, b. (1997). modeling of safety management systems, safety science, 26(1/2), 121 – 140. . khan, m. (ed). (1988). labour administration: profile on bangladesh. geneva: asia— pacific labour administration of the international labour office, 118. khan, mr (1994). sickness, disease, treatment and medical costs by socio-economic variables in bangladesh. bangladesh institute of development studies, research monograph, 15, 279. khan, z (1997). musculoskeletal problems of female garments worker in bangladesh: some anthropometrics. unpublished master’s thesis, department of human work science, luleå university of technology, luleå: sweden. majumder, pp & chowdhury, s (1992). poshak shilpa niojito narisramik samajik nirapattahinotar gati prokiti (in bengali). magazine mashik chinta, 14, 18–21. rahman, ri (1993). occupational mobility among women in manufacturing industries of dhaka city: findings from an enterprise level survey. delhi, india: ilo/artep. ijahp article: mm allama, ms parvez, sm ali, a azeem/ using the analytical hierarchical process to prioritise safety management elements in manufacturing in bangladesh international journal of the analytic hierarchy process 117 vol. 4 issue 2 2012 issn 1936-6744 rahman, f, anderssen & syanstrom l (2000). potential of using existing injury information for injury surveillance at the local level in developing countries: experiences from bangladesh. public health. 114 (2), 133–136. raihan, a (1997). ergonomics approaches in industrialization of developing countries: case study in bangladesh. from experience to innovation. iea’97(148-150). finnish institute of occupational health. rondinelli, d.a. and berry, m.a. (2000). environmental citizenship in multinational corporations: social responsibility and sustainable development. european management journal, 18(1), 70-84. sadeque, s, ahasan, mr & uddin, z (1998). health, nutrition and population program in rural bangladesh (in: abstracts). paper presented in the international conference of open care, oulu deaconess institute, finland, 26. law, w.k., chan, a.h.s., & pun, k.f. (2006). prioritizing the safety management elements: a hierarchical analysis for manufacturing enterprises. industrial management & data systems, 106(6), 778-792. zohir, sc & majumder, pp (1996b). garment workers in bangladesh: economic, social and health condition. research monograph no 18, dhaka: bangladesh institute of development studies. an analytical hierarchical process for prioritising the safety management elements for a manufacturing organization in bangladesh 1. introduction 2. objectives 3. methodology microsoft word 3b_aguiar_fmea_vol 2_issue1.docx ijahp article: aguiar, de souza, salomon / evaluate scoring criteria for failure mode and effect analysis international journal of the analytic hierarchy process 3 vol. 2, issue 1, 2010 issn 1936-6744 an ahp application to evaluate scoring criteria for failure mode and effect analysis (fmea) dimas campos aguiar iochpe maxion wheels and chassis division cruzeiro, sp, brazil e-mail: dimastjs@gmail.com helder jose celani de souza fleury diagnostics sao paulo, sp, brazil e-mail: hcelani@uol.com.br valerio a. p. salomon sao paulo state university guaratingueta, sp, brazil e-mail: salomon@feg.unesp.br abstract the failure mode and effect analysis (fmea) has been developed by several researchers who have proposed distinct reference tables to score the severity, occurrence and detection of a failure. this paper aims to evaluate, using ahp, several proposals for the application of process fmea, as well as to obtain and offer recommendations for its application. five reference tables for severity, six tables for occurrence, and six tables for detection are presented. these reference tables are evaluated for their application in the brazilian automotive industry by using critical analysis and proper judgments. the scientific contribution of this paper is to provide a way to select a reference table, available in the fmea literature, for the application of the fmea process. keywords: ahp, process fmea, quality management. 1. introduction the failure mode effect and analysis (fmea) was first applied in 1949 by the u.s. army. in the 1960s, it was effectively used by national aeronautics and space administration (nasa) in the apollo project. as reported by fernandes and rebelato (2006) and sharma et al. (2007), fmea was developed to identify potential failures in processes by defining its causes and effects. in the 1980’s, fmea became a reference for processes developments, initially, in the aerospace industry. in 1988, the ford motor company published an instructions manual for design fmea and process fmea, which had been used in product development and manufacturing processes (society of automotive engineers 2001). the use of the fmea is considered a key element in quality process planning, according to stamatis (1995), palady (1995), reid (2005) and teng et al. (2006): the organizations save resources and they have high levels of customer satisfaction when they perform a full application of fmea. thus, fmea is a very powerful method when applied correctly, but, otherwise, does not show its benefits (devadasan et al. 2003). assuming that the technique for using fmea is public and known, details on how to fill the fmea  corresponding author rob typewritten text rob typewritten text rob typewritten text http://dx.doi.org/10.13033/ijahp.v2i1.69 rob typewritten text rob typewritten text rob typewritten text rob typewritten text rob typewritten text ijahp article: aguiar, de souza, salomon / evaluate scoring criteria for failure mode and effect analysis international journal of the analytic hierarchy process 4 vol. 2, issue 1, 2010 issn 1936-6744 form and to make the relationship between its columns are not covered in this paper. they can be found in many publications, such as kmenta and ishii (2000) and terninko (2003). the main objective of this paper is to compare different reference tables to score detection, occurrence and severity, identifying them in relevant literatures, and to propose the best combination for process fmea application. for this purpose, the analytic hierarchy process (ahp) is applied. in addition, the literature research, theoretical basis, judgments and results analysis, and the conclusions report were also used. 2. theoretical basis this section presents a selection of several publications about techniques for fmea application with focus on reference tables and their scores. ahsen (2008) affirms that the fmea helps managers to perform resource allocations more efficiently when oriented by risks prioritization. thus, relations among the severity of the failure mode, the frequency at the failure may occur and the probability of detection failure, the process fmea aims to define, demonstrate and improve the engineering solutions in response to costs, maintainability, productivity, quality, and reliability. this section will follow the sequence of a typical fmea form, that is, severity, occurrence, and detection scorings. severity. each failure mode has to be classified according to the impact of its effects. severity defines this classification with a score ranging from 1 to 10: score 1 is the least serious and score 10 is the most serious (tozzi 2004). ben-daya and raouf (1996), chang et al. (2001), and yang et al. (2006) adopted a criterion in which two aspects of high subjectivity can be noted. the first aspect is the use of customer dissatisfaction as a comparison standard. each failure mode considered in fmea must have a potential effect that causes customer dissatisfaction. the severity score will indicate the degree of such dissatisfaction. another aspect is the parameterization of more than one value for the same situation, which can contribute to a less accurate scoring. this proposal (s1) is showed in the table 1. table 1 (s1) severity scoring based on customer satisfaction (adapted from ben-daya and raouf (1996), chang et al. (2001), and yang et al. (2006)) severity score the customer will probably not notice 1 slightly annoyance 2 or 3 customer dissatisfaction 4 to 6 high degree of customer dissatisfaction 7 or 8 safety: regulatory consequence 9 or 10 other references have different criteria. terninko (2003) adopts a common idea for the severity score that ranges from 1 to 10, classified from no one severity to catastrophic severity. this criterion (s2) is presented in the table 2. ijahp article: aguiar, de souza, salomon / evaluate scoring criteria for failure mode and effect analysis international journal of the analytic hierarchy process 5 vol. 2, issue 1, 2010 issn 1936-6744 table 2 (s2) another severity scoring (adapted from terninko (2003)) severity score no one 1 very slight 2 slight 3 minor 4 moderate 5 significant 6 major 7 extreme 8 serious 9 catastrophic 10 in the fmea applications conducted by tozzi (2004), the reference to score severity is composed of five intervals correlated to 10 scores, considering customer dissatisfaction as a comparison standard and the use of more than one score for the same situation, as shown in the table 3 (s3). table 3 (s3) severity scoring based on the effect of the failure (adapted from tozzi (2004)) effect score minimum (failure affects the system performance at minimum level and the major customers may not perceive this kind of failure) 1 or 2 low (when causes light customer dissatisfaction due a system light loss of performance or degradation) 3 or 4 moderate (when causes any customer dissatisfaction due systems malfunctioning or performance reduction) 5 or 6 high (when causes customer dissatisfaction) 7 or 8 very high (when impacts operation safety or involves government regulatory deviations) 9 or 10 the definition of severity scores from 1 to 10 is the most observed in practice. but, it is not a mandatory rule. matos (2004) adopted scores ranging from 1 to 5. this range considers the customer’s perception of the failure with its respective effect. this criterion facilitates the assignments, restricts the number of possible combinations for the risk priority number, and allows different risks grouping for the same situations. this proposal (s4) by matos (2004) is presented in the table 4. table 4 (s4) severity scoring from 1 to 5 (adapted from matos (2004)) severity score it is reasonable to expect the customer does not perceive the failure 1 the customer perceives the failure, but does not show dissatisfaction 2 the customer will perceive failures and be dissatisfied 3 the customer will be dissatisfied, but its safety is not affected 4 the customer will be dissatisfied and its safety is affected 5 ijahp article: aguiar, de souza, salomon / evaluate scoring criteria for failure mode and effect analysis international journal of the analytic hierarchy process 6 vol. 2, issue 1, 2010 issn 1936-6744 a criterion for severity scoring based on the importance of necessity is adopted by fernandes and rebelato (2006) and presented in the table 5. this proposal (s5) does not include intermediate situations scores with the values two, three, five, seven and nine. table 5 (s5) severity scoring based on the necessity importance (adapted from fernandes and rebelato (2006)). importance criteria score very low necessity related to the secondary functions, not relevant to the customer 1 low necessity related to the secondary functions, relevant to the customer 4 moderate necessity related to the primary functions, little relevant to the customer 6 high necessity related to the primary functions of the product or service 8 very high necessity related to the user safety 10 occurrence. to define an occurrence score, a potential cause must be interpreted as how the failure could occur, which is described like something that can be corrected or controlled. the idea is to identify the origin of each failure mode, based on historical data. each of these cases must be classified by an occurrence score, conventionally estimated in a scale from 1 to 10. table 6 presents occurrence scoring (o1) proposed by braglia (2000), which considers historical data for the mean time between failures (mtbf). table 6 (o1) occurrence scoring based on the mean time between failures (adapted from braglia (2000)) occurrence mtbf score remote more than 10 years 1 low 2 to 10 years 2 or 3 moderate 6 months to 2 years 4 or 6 high 3 to 6 months 7 or 8 very high less than 3 months 9 or 10 ben-daya and raouf (1996), chang et al. (2001) and yang et al. (2006) define other way to occurrence scoring (o2): from the probability of failures, without considering the occurrence of potential causes and with a score ranging from 1 to 10, as shown in the table 7. table 7 (o2) occurrence scoring based on the probability of failure (adapted from ben-daya and raouf (1996), chang et al. (2001), and yang et al. (2006)) occurrence probability score remote chance of failure less than 1/20,000 1 low failure rate around 1/10,000 2 or 3 moderate failure rate around 1/1,000 4 to 6 high failure rate around 1/100 7 or 8 very high failure rate around 1/10 or higher 9 or 10 terninko (2003) adopted a similar reference to score occurrence, as shown in the table 8 (o3). the term “ratio” was preferred instead of “probability”. ijahp article: aguiar, de souza, salomon / evaluate scoring criteria for failure mode and effect analysis international journal of the analytic hierarchy process 7 vol. 2, issue 1, 2010 issn 1936-6744 table 8 (o3) occurrence scoring (adapted from terninko (2003)) occurrence ratio score almost never 3/106 1 remote 100/106 2 very slight 1,000/106 3 slight 10,000/106 4 low 150,000/106 5 medium 300,000/106 6 moderate high 400,000/106 7 high 500,000/106 8 very high 666,667/106 9 almost certain 900,000/106 10 according to tozzi (2004), the failure rate should be estimated by applying statistical procedures with historical data collected in similar cases. otherwise, some subjective analysis would be required. process capability index (cpk) based scores for occurrence is shown in the table 9 (o4). table 9 (o4) occurrence scoring based on the process capability index (adapted from tozzi (2004)) occurrence probability cpk score minimum (very unlikely occurrence failure) 2.0 or higher 1 1.6 to 2.0 2 low (rare occurrence failures) 1.2 to 1.6 3 1.0 to 1.2 4 moderate (occasional occurrence failures) 0.9 to 1.0 5 0.7 to 0.9 6 high (frequent occurrence failures) 0.6 to 0.7 7 0.4 to 0.6 8 very high (almost inevitable occurrence failures) 0.3 to 0.4 9 0.3 or smaller 10 table 10 presents the criterion (o5) to score occurrence in fmea applications according to matos (2004). table 10 (o5) occurrence scoring from 1 to 5 (adapted from matos (2004)) occurrence of failure score very remote probability to occur 1 low number of occurrences 2 moderate number of occurrences 3 high number of occurrences 4 alarming failures 5 fernandes and rebelato (2006) proposed occurrence scores based on failure rates. historical occurrence of the potential causes of each failure should be considered, as shown in the table 11 (o6). ijahp article: aguiar, de souza, salomon / evaluate scoring criteria for failure mode and effect analysis international journal of the analytic hierarchy process 8 vol. 2, issue 1, 2010 issn 1936-6744 table 11 (o6) occurrence scoring based on failure rates (adapted from fernandes and rebelato (2006)) occurrence failure ratios score almost impossible 1/1,500,000 1 remote 1/150,000 2 low 1/15,000 3 relative low 1/2,000 4 moderate high 1/400 5 moderate high 1/80 6 high 1/20 7 repeated failures 1/8 8 very high 1/3 9 extreme high 1/2 or less 10 detection. with the identification of failures consequences the prevention and detection must be addressed. table 12 presents the scoring criteria (d1) adopted by braglia (2000), which assesses different combinations of controls to detect a failure. table 12 (d1) detection scores from the chance of non-detection (adapted from braglia (2000)) naked eye visible switchboard or indirectly controllable visible after an inspection periodic inspection score yes partial no direct indirect no yes no yes no x 1 x x x x x 2 x x x x 3 x x x x 4 x x x x 5 x x x x x x x x 6 x x x x x x x x 7 x x x x 8 x x x x 9 x x x x 10 the prevention actions impact on system features, either in terms of product or process, in order to reduce the risk of each failure by the definition of prevention controls. puente et al. (2002) argue that these controls allow the direct action on potential causes of a particular failure mode, while the detection controls are possible to detect the failure mode before reaching the next operation stage. in addition, it has a control plan and a detection system that act preventively in the process and product, respectively. the detection scoring (d2) adopted by ben-daya and raouf (1996), chang et al. (2001) and yang et al. (2006) appears more consistent for the failure detection evaluation purpose, and considers the defect probability to reach the customer, as shown in table 13. ijahp article: aguiar, de souza, salomon / evaluate scoring criteria for failure mode and effect analysis international journal of the analytic hierarchy process 9 vol. 2, issue 1, 2010 issn 1936-6744 table 13 (d2) detection scoring based on the chance of a defect reaching the customer (adapted from ben-daya and raouf (1996), chang et al. (2001), and yang et al. (2006)) detection probability of individual defect reaching the customer score very high 0% to 5% 1 high 6% to 15% 2 16% to 25% 3 moderate 26% to 35% 4 36% to 45% 5 46% to 55% 6 low 56% to 65% 7 66% to 75% 8 remote 76% to 85% 9 86% to 100% 10 terninko (2003) proposed detection frequency based scores (d3), as shown in the table 14. table 14 (d3) detection scoring based on the detection frequency (adapted from terninko (2003)) detection frequency detection frequency control performance score almost certain 900,000/106 detected before problem 1 very high 666,667/106 2 high 500,000/106 3 moderate high 400,000/106 4 medium 300,000/106 corrective action possible 5 low 100,000/106 6 slight 10,000/106 7 very slight 1,000/106 8 remote 100/106 9 almost impossible 3/106 catastrophe occurred 10 table 15 presents the detection scoring criterion (d4) in fmea applications according to matos (2004). table 15 (d4) detection scoring from 1 to 5 (adapted from matos (2004)) detection score very high probability to have the failure detected 1 high probability to have the failure detected 2 medium probability to have the failure detected 3 low probability to have the failure detected 4 very low probability to have the failure detected 5 table 16 presents the criterion (d5) proposed by leal et al. (2005), which is based on the probability of detecting the failure cause related to the preventive control. table 16 (d5) detection scoring (adapted from leal et al. (2005)) ijahp article: aguiar, de souza, salomon / evaluate scoring criteria for failure mode and effect analysis international journal of the analytic hierarchy process 10 vol. 2, issue 1, 2010 issn 1936-6744 detection preventive control score almost certain the maintenance almost certainly will detect the failure cause 1 very high very high chance to detect the failure cause 2 high high chance to detect the failure cause 3 moderate high moderate high chance to detect the failure cause 4 moderate moderate chance to detect the failure cause 5 low low chance to detect the failure cause 6 very low very low chance to detect the failure cause 7 remote remote chance to detect the failure cause 8 very remote very remote chance to detect the failure cause 9 absolutely uncertain the maintenance does not detect the potential failure cause or the maintenance does not exist 10 another criterion for detection scoring (d6) was proposed by andrade and turrioni (2000), which focused on environmental analysis, as shown in the table 17. table 17 (d6) detection scoring (adapted from andrade and turrioni (2000)) detection degree score the current controls will certainly detect, almost immediately, and the reaction can be instantaneous. 1 or 2 there is a high probability that the aspect is detected soon after its occurrence, being possible a rapid reaction. 3 or 4 there is a moderate probability that the aspect is detected in a reasonable period of time before an action can be taken, but the effects may be seen. 5 or 6 it is unlikely that the failure is detected or it will take more time before an action can be taken, and the effects will be seen. 7 or 8 the failure will not be detected in any time or there is no reaction possible (under normal operating conditions). 9 or 10 3. ahp application in section 2 there are 17 reference tables to facilitate the scoring of detection, occurrence, and severity for the fmea application in a company. the decision problem from the scoring criteria presented in those reference tables is to choose only one criterion for detection, one for occurrence, and another one for severity. to evaluate the referenced tables presented in section 2, ahp methodology will be applied. this decisionmaking method was selected because it allows choosing the best alternative considering multiple criteria expressed through qualitative or quantitative values (saaty, 2001). montevechi and pamplona (1996) argued that the use of ahp method is recommended when it involves human opinion. with the ahp, it is possible to build comparison matrices, and then, set the priorities between the alternatives, in our case, the reference tables. one important feature of the ahp method is the consistency checking of the judgments that make the comparison matrix. this is done solving a linear algebra problem (equation 1), where a is the comparison matrix, w is its right eigenvector and max is its principal eigenvalue: a w = max w (1) ijahp article: aguiar, de souza, salomon / evaluate scoring criteria for failure mode and effect analysis international journal of the analytic hierarchy process 11 vol. 2, issue 1, 2010 issn 1936-6744 the consistency checking of the comparison matrices is made comparing max with n, the matrix order, usually with equation 2, where  is the consistency index. to a comparison matrix be considered consistent,  must be equal or less than 0.1.  = (max – n)/(n – 1) (2) for the analysis at hand, three hierarchies will be used, one for each of the fmea dimensions: severity, occurrence, and detection. the alternatives in each hierarchy will be the different scoring criteria discussed previously. therefore, the severity hierarchy has five possible alternatives (s1-s5) corresponding to the five different scoring criteria (tables 1-5) discussed in the previous section. similarly, the occurrence hierarchy has six alternatives (o1-o6) corresponding to each of the possible scoring criteria (tables 6-11). finally, the detection hierarchy also has six possible alternatives (d1-d6) corresponding to the scoring criteria (tables 12-17) previously discussed. tables 18 to 20 present the comparison matrix for each of the three hierarchies. all the judgments were done by a couple of quality engineers who are experienced in fmea applications in the automotive industry. table 18 severity scoring criteria judgments s1 s2 s3 s4 s5 priority s1 (ben-daya and raouf (1996), chang et al. (2001), and yang et al. (2006)) 1 1/3 1/2 2 1 0.14 s2 (terninko 2003) 1 2 4 3 0.40 s3 (tozzi 2004) 1 3 2 0.24 s4 (matos 2004) 1 1/2 0.08 s5 (fernandes and rebelato 2006) 1 0.14 table 19 occurrence scoring criteria judgments o1 o2 o3 o4 o5 o6 priority o1 (braglia 2000) 1 5 4 3 8 4 0.42 o2 (ben-daya and raouf (1996), chang et al. (2001), and yang et al. (2006)) 1 1/2 1/3 4 2 0.10 o3 (terninko 2003) 1 1/2 5 1 0.12 o4 (tozzi 2004) 1 6 2 0.20 o5 (matos 2004) 1 1/5 0.03 o6 (fernandes and rebelato 2006) 1 0.12 ijahp article: aguiar, de souza, salomon / evaluate scoring criteria for failure mode and effect analysis international journal of the analytic hierarchy process 12 vol. 2, issue 1, 2010 issn 1936-6744 table 20 detection scoring criteria judgments d1 d2 d3 d4 d5 d6 priority d1 (braglia 2000) 1 4 6 3 7 3 0.42 d2 (adapted from ben-daya and raouf (1996), chang et al. (2001), and yang et al. (2006)) 1 3 1/2 4 1/2 0.12 d3 (terninko (2003) 1 1/4 2 1/4 0.06 d4 (matos 2004) 1 5 1 0.19 d5 (leal et al. 2005) 1 1/5 0.04 d6 (andrade and turrioni 2000) 1 0.19 for tables 18 to 20, the values found for  are, respectively, 0.009, 0.046, and 0,047. as all these values are less than 0.1, the judgments provided by the engineers can be considered as consistent. the main result of the ahp application is the indication of the use of terninko (2003) to score the severity, and the use of braglia (2000) to score both occurrence and detection. it is important to state that this result is limited to a company with similar features of the company where the data were collected, that is, an industrial company from brazil at the end of 2009. 4. conclusions this paper has first presented different scoring criteria for detection, occurrence, and severity in the context of fmea application in different situations identified in publications about this topic. the use of ahp has provided a simple way to choose only one criterion for detection, one for occurrence, and another one for severity. it was done by comparing different proposals and prioritizing them within three hierarchies. for future studies, it is suggested to develop an action research in the manufacturing environment through the combination of reference tables for the process fmea scores recommended in this work. another recommendation is the application of others concepts, such as group decision making, to expand the results to a sector or to different companies. references andrade, m.r.s. & turrioni, j.b. (2000). a method for environmental impacts analysis with fmea utilisation. proceedings of 20th national meeting of production engineering. sao paulo, brazil. (in portuguese). ahsen, a. (2008). cost-oriented failure mode and effects analysis. international journal of quality & reliability management, 25(5), 466 476. ben-daya, m. & raouf, a. (1996). a revised failure mode and effects analysis model. international journal of quality & reliability management, 13(1), 43 47. braglia, m. (2000). mafma: multi-attribute failure mode analysis. international journal of quality & reliability management, 17( 9), 101 -1033. chang, c.l., liu, p.h. & wei, c.c. (2001). failure mode and effects analysis using grey theory. integrated manufacturing systems, 12 (3), 211-216. ijahp article: aguiar, de souza, salomon / evaluate scoring criteria for failure mode and effect analysis international journal of the analytic hierarchy process 13 vol. 2, issue 1, 2010 issn 1936-6744 devadasan, s.r., muthu, s., samson, r.n. & sankaran, r.a. (2003). design of total failure mode and effects analysis program. international journal of quality & reliability management, 20(5), 551 568. fernandes, j.m.r. & rebelato, m.g. (2006). proposal of a method to integrate qfd and fmea. gestao producao,13(2), 245-259. (in portuguese). kmenta, s. & ishii, k. (2000). scenario-based fmea: a life cycle cost perspective. baltimore, md: asme publications. leal, f., pinho, a.f. & almeida, d.a. (2005). failure analysis with fmea and grey theory. proceedings of 25th national meeting of production engineering. porto alegre, brazil. (in portuguese). matos, r.b. (2004) performance indicators for sawed wooden improvement in small enterprises: a case study. master degree dissertation (forest resources). piracicaba, brazil: usp. (in portuguese). montevechi, j.a.b. & pamplona, e.o. (1996) hierarchy analysis of investments. proceedings of 16th national meeting of production engineering. piracicaba, brazil. (in portuguese). palady, p. (1995) failure modes and effects analysis. portland, or: pt publications. puente, j., pino, r., priore, p. & fouente, d.l. (2002). a decision support system for applying failure mode and effects analysis. international journal of quality & reliability management, 19(2), 137-151. reid, r.d. (2005). fmea: something old, something new. quality progress, 38, 90-93. saaty, t.l. (2001). analytic hierarchy process, decision making for leaders. pittsburgh, pa: rws publications. sharma, r.k., kumar, d., kumar, p. (2007). modeling and analysing system failure behaviour using rca, fmea and nhppp models. international journal of quality & reliability management, 24(5), 525-546. society of automotive engineers (2001). arp 5580: recommended failure modes and effects analysis (fmea) practices for non-automobile. usa:g-11 sae reliability committee publications. stamatis, d.h. (1995). failure mode and effect analysis, fmea from theory to execution. milwaukee, wi: asq quality press. teng, s.g., ho, s.m., shumar, d. & liu, p.c. (2006) implementing fmea in a collaborative supply chain environment. international journal of quality & reliability management, 23(2), 179-196. terninko, j. (2003). reliability / mistake-proofing using failure mode and effect analysis (fmea). proceedings of 57th annual quality congress, kansas city, mi. tozzi, a.r. (2004). development of program to check the process of cables releasing with fmea aid. mba thesis. porto alegre, brazil: ufrgs. (in portuguese) yang, c.c.; lin, w.t.; lin, m.y.; huang, t. (2006). a study on applying fmea to improving erp introduction an example of semiconductor related industries in taiwan. international journal of quality & reliability management, 23(2), 298-322. isahp 2009 symposium, july 29-august 1, 2009, pittsburgh, pennsylvania, usa ijahp article: m azizi/ a decision making model regarding wood panel selection international journal of the analytic hierarchy process 137 vol. 4 issue 2 2012 issn 1936-6744 a decision making model regarding wood panel selection majid azizi faculty of natural resources university of tehran karaj, tehran, iran email: mazizi@ut.ac.ir abstract identification of preferable wood panels according to market share, competition and quality is necessary until the investors and manufacturers help to develop the industry and preserve the market share. the quality and quantity of iranian wood panels is growing successfully. the preservation of market share will need to identify preferable wood panels taking into consideration several criteria and their intensities. wood panel’s criteria include moisture percentage, density, thickness swelling percentage, water absorption percentage, and bending strength. each one of the criteria has three levels of intensity. in this paper, the criteria and their intensities have been evaluated by applying ahp. then, the wood panels have been ranked according to the ahp evaluation. the results indicate that density of the product and its high intensity has the highest priority. the ghazvin panel has the highest priority, and moisture percentage criterion is very sensitive in comparison with other criteria. keywords: ahp, wood panel, intensity, attribute, priority 1. introduction wood panel is one of the major products of iranian wood industries. at present, the panel’s quality and quantity are growing. in the past decade, the growth in production of the panel has been fast especially in constructional panel consumption. iranian wood panels production increased by 87.8% from 1997 (382 322 m3) to 2007 (718003 m3), and again increased by 18% from 2003 to 2004. it decreased slightly in 2005 due to national economic conditions (azizi et al , 2009). also, in the same period, per capita consumption of particleboard in iran increased by 60% and population increased by 17.7%. as a matter of fact, the increase of consumption of particleboard panels in the past decade has led to the consideration of the quality of specifications of the panel product. there are 17 particleboard panels factories in the country, including 10 factories in the north, 3 factories in the northwest, 2 factories in the center and 2 factories in the south. in the current research we performed three steps to identify preferable wood panels according to market share, competition and quality. first, we determined the major criteria which affect specification of the particleboard product. we selected five criteria based on expertise from the institute of standards and industrial research of iran. the attributes which were considered the most relevant by the experts were (1) moisture percentage, (2) mailto:mazizi@ut.ac.ir rob typewritten text http://dx.doi.org/10.13033/ijahp.v4i2.75 rob typewritten text ijahp article: m azizi/ a decision making model regarding wood panel selection international journal of the analytic hierarchy process 138 vol. 4 issue 2 2012 issn 1936-6744 density, (3) thickness swelling percentage (4) water absorption percentage, and (5) bending strength. to be more precise, the institute of standards and industrial research of iran use number 2496 as the national standard for16 millimeter thickness particleboard (cellulose and packaging research group, 2002). the numbers 814, 813, 2489, 2488 and 2332(national standard numbers) are related to moisture percentage; density; thickness swelling percentage; water absorption percentage and bending strength, respectively. in the second step, three levels of intensities have been evaluated for each of the criteria: high (h), medium (m) and low (l) (figure 1). in the third step, we extracted five of the factories which included gorgan a , ghaemshahr, gorgan b, ghazvin and neka units, and then obtained the specifications of the panels according to the institute of standards and industrial research of iran. these factories were selected because their distribution and location were suitable, and most were located in the north of iran. specifications of the particleboard panels are shown in table1. table 1 specification of particleboard panel (institute of standards and industrial research of iran) bending strength(kg/cm2) water absorption (%) thickness swelling (%) density (g/cm3) humidity (%) criteria 160 50 12 0.6-0.8 6-8 standard amounts 160-210 25-75 6.65-23 0.6-0.8 6-8 overall amounts to select the best panel, the analytic hierarchy process method was used. this method was first invented by thomas l. saaty in 1970s, and is used in decision making processes which have qualitative and quantitative criteria (saaty, 2000). for example, the ahp was used to select the best facial tissue according to the customer’s perspective by azizi and noori (2007). azizi (2008) used the ahp method to determine effective criteria to locate selection of wood composite units in the khuzestan province, and also obtained the highest priority city. alkaner and das (2008) used the ahp in their research which indicated a framework to select the optimal technological alternative within the context of generic ship dismantling facility development. feglar (2008) developed the ahp model to allow comparison of a public based project management with two private based project management systems. bruno et al (2009) suggested using multi-criteria models and methods in reference to supplier selection problems (ssp). an overview of the current proposals based on ahp and its variants to cope with the ssp is provided in his paper. in azizi and taheri (2009), a hierarchy was used to prioritize benefits, costs, opportunities and risks (bocr) regarding the proper selection of cooperatives management in iran’s northern forests using the analytic hierarchy process ratings approach. the final synthesis of the system showed collective management of the having the highest priority. the analytic hierarchy process and group decision making have also been used to calculate non-development criteria values for the particleboard industry. in ijahp article: m azizi/ a decision making model regarding wood panel selection international journal of the analytic hierarchy process 139 vol. 4 issue 2 2012 issn 1936-6744 this study, the results indicated that lack of raw material, non-expert elements, lower production quality, changing of management, long decision making process and ancient technology are the preferable criteria, respectively (azizi, 2009). the ahp was applied to identify and prioritize the cleaner production implementation of a paper making mill. (ghorbannezhad et al, 2009). finally, azizi (2005) applied ahp to determine effective criteria to select the best choice of raw material procurement in paper making factories in iran. the results showed no harm on environment was the highest priority in terms of benefits. 2. methodology 2.1. preparing questionnaire for the first and second stage in order to analyze the candidate products and identify the preferred ones, the initial step was to identify the criteria. a comprehensive list of factors was prepared, and a hierarchy of these factors was constructed to establish their weight using the analytic hierarchy process. then, a questionnaire was designed to evaluate each criterion’s contribution to the decision process. this questionnaire was distributed among six experts in iranian wood panel factories. the individual judgments were checked for consistency, and the aggregated opinion was derived using teamec 2000. in the second stage, a questionnaire was prepared with regard to the intensities of the criteria and specifications of the test data to select the best alternative. then, the questionnaire was distributed to two experts in the institute of standards and industrial research of iran, one academic, and two experts in iranian wood panel factories. the questionnaires were gathered and synthesized by expert choice 2000. 2.2. description of the criteria 1. moisture percentage: this is with respect to the national standard number 814. the precision of measurement to determine moisture percentage of the product is 0.1% and the range of the moisture is 6-8%. in other words, moisture content has been calculated via moisture content's arithmetical means of all of the related test samples (institute of standards and industrial research of iran, 2002). the moisture test has been done on samples with every form and size whose surface is a minimum 50*50 mm2. every sample was weighted with a precision of 0.01 gram during sampling. every test sample is dried at 103±2°c to reach a constant weight. the test sample is fanned in dry air and then weighted as soon as possible so that increasing moisture percentage is not more than 0.1 %. the moisture percentage of each test sample is obtained with the following formula with a precision of 0.1 %, and the minimum number of the samples is four. moisture percentage is calculated as following (equation 1): h= (mh – mo): mo *100 (1) mh: test sample weight before drying (gram) mo: test sample weight after drying (gram.) h: moisture percentage of sample test (%) ijahp article: m azizi/ a decision making model regarding wood panel selection international journal of the analytic hierarchy process 140 vol. 4 issue 2 2012 issn 1936-6744 2. density: this is with respect to the national standard number 813.the range of the product is 0.6-0.8 g/cm3. precision of the density measurement is 0.01 g/cm3, and the size of the test sample is 50*50 mm2. the minimum number of the samples is six. 3. thickness swelling percentage: this is according to the national standard number 2489, and is based on floatation of the sample in water 20±2 °c with the dimension of the sample being 100*200 mm2.the variation of the thickness is 12% after 2 hours of floating. the minimum number of samples is eight. 4. water absorption percent: this is according to the national standard number 2488, and based on floating of the sample in distilled water 20±2 temperature for 2 hours and then measuring the weight of the water absorption percentage in relation to dry position. the standard water absorption percentage is 50%. the size of the test sample is 152*152 mm2, and the minimum number of the samples is four. 5. bending strength: this is according to the national standard number 2332. the bending strength is measured and limitation of the strength 160 kg/ cm2 is defined. however, the size of the test sample is calculated as following: l= 24t+50 l: length of test sample (millimeter) t: thickness of test sample (millimeter) width of the test sample: if the nominal thickness is more than 6 millimeters, the width of the sample test will be 76 millimeters. if the nominal thickness is equal or less than 6 millimeters, the width of the sample test will be 50 millimeters. the minimum number of the samples is four. 2.3. selection of the product with greatest overall manufacturer’s preference (saaty, 2000) step 1: determine the manufacturer's preferred attributes by developing a matrix that compares attributes in pairs considering product desirability (table 2). step 2: determine manufacturer's preferred intensity of the attributes by developing five matrices that compare intensity levels in pairs with respect to each attribute (tables 3-7). next, we want to synthesize these evaluations to obtain the set of overall priorities that will indicate the preferable product to manufacture. step 3: group the priorities of the intensities (h, m, and l) for each of the five attributes in columns and enter the priorities of the attributes. then multiply each column by the priority of the corresponding attribute to obtain the weighted vectors of priority for the intensities (table 9). step 4: select from each column the element with the highest priority to obtain the vector of desired attribute intensities. then, add this row and divide each entry by the total to get the normalized vector of desired attribute intensities. ijahp article: m azizi/ a decision making model regarding wood panel selection international journal of the analytic hierarchy process 141 vol. 4 issue 2 2012 issn 1936-6744 step 5: determine the perceived product standings by developing matrices that compare the five panels (gorgan a , ghaemshahr, gorgan b, ghazvin and neka) in pairs with respect to the most desired attribute intensities (tables 10-14). step 6: group the priorities of the panels according to each of the desired attributes intensity in columns and enter the normalized priorities above the columns. then, multiply each column by the normalized priority of the corresponding attribute intensity to obtain the weighed vectors of priority for the desired attribute intensities for each panel (tables 37). step 7: add each of the five rows to obtain the overall priorities of the five panels (tables 15-16). step 8: perform a sensitivity analysis (table 17). 2.4. analytic hierarchy process ahp is a decision-making method which enables decisions to be made which are dependent on several criteria or multi-criteria decisions. according to the ahp method, first the given structure and then the criteria relevant to decision making are compared with each other. then the priority rate of each criterion is determined specifically. the numbers 1-9 are the standard scale which is used in two-by-two comparisons (saaty, 2000). the analytic hierarchy process has several advantages. these include: 1) ability to break criteria into manageable components, 2) allows a group to make a specific decision for consensus or tradeoff, 3) provides an opportunity to examine disagreements and stimulate discussion and opinions, 4) offers opportunities to change criteria and modify judgments, 5) forces one to face the entire problem at once, 6) offers an actual measurement system by enabling one to estimate relative magnitudes and derive ratio scale priorities accurately, 7) organizes, prioritizes and synthesizes complexity within a rational framework, 8) interprets experiences in a relevant way without reliance on a black box technique like a utility function, and 9) makes it possible to deal with conflicts in perception and judgment (saaty, 2000). figure 1 shows the hierarchy of effective criteria for the panels. ijahp article: m azizi/ a decision making model regarding wood panel selection international journal of the analytic hierarchy process 142 vol. 4 issue 2 2012 issn 1936-6744 figure 1 hierarchy of effective criteria for the panels (h: high intensity, m: medium intensity, l: low intensity) 3. results the group of experts, with the aid of expert choice software 2000, were able to participate in group decision making and produce a comparison matrix of the first level, comparison matrices of their intensities, weighted values of the effective criteria and their intensities, comparison matrices of the alternatives with respect to criteria and their intensities, and a sensitivity analysis (tables 2-17). note that in all the following tables the judgments on the diagonal aii = 1, so those cells are left blank. also aji = 1/ aij so as the cells below the diagonal are determined by the values above the diagonal, the cells below are left blank as well. in these tables, when the judgment is a number greater than 1, the row element is dominant. for example, (moisture percentage, water absorption) =1.097, and thus moisture percentage is more important. when the judgment is less than one the number is written as an inverse in these tables, so inverses always mean the column element is dominant. for example, the judgment for (moisture percentage, density) is 1/1.978 and thus density is more important than moisture percentage. panel selection moisture percent h m l density h m l thickness swelling percent h m l water absorption percent h m l bending strength h m l ijahp article: m azizi/ a decision making model regarding wood panel selection international journal of the analytic hierarchy process 143 vol. 4 issue 2 2012 issn 1936-6744 table 2 comparison matrix of the first level with respect to goal bending strength water absorption thickness swelling density moisture percentage goal 1.0141/ 1.097 1.1421/ 1.9781/ moisture percentage 1.233 2.334 2.116 density 1.170 2.466 thickness swelling 1.9121/ water absorption bending strength inconsistency = 0.02 table 3 comparison matrix of the intensities with respect to moisture percentage low medium high 1.057 1.508 high 1.0 medium low inconsistency = 0.01 table 4 comparison matrix of the intensities with respect to density low medium high 3.797 1.944 high 1.0 medium low inconsistency = 0.03 table 5 comparison matrix of the intensities with respect to thickness swelling low medium high 1.9851/ 1.4421/ high 1.6471/ medium low inconsistency = 0.00 ijahp article: m azizi/ a decision making model regarding wood panel selection international journal of the analytic hierarchy process 144 vol. 4 issue 2 2012 issn 1936-6744 table 6 comparison matrix of the intensities with respect to water absorption low medium high 1/2.129 1/1.817 high 1/1.829 medium low inconsistency = 0.00 table 7 comparison matrix of the intensities with respect to bending strength low medium high 5.514 3.532 high 3.301 medium low inconsistency = 0.06 table 8 results of comparing the importance of the criteria with respect to the goal bending strength water absorption thickness swelling density moisture percentage criteria 0.200 0.115 0.209 0.313 0.163 weighting value table 9 results of comparison matrices for criterion intensities low medium high intensities criteria 0.325 0.289 0.386 moisture percentage 0.12 0.335 0.545 density 0.472 0.304 0.225 thickness swelling 0.49 0.311 0.199 water absorption 0.094 0.241 0.665 bending strength ijahp article: m azizi/ a decision making model regarding wood panel selection international journal of the analytic hierarchy process 145 vol. 4 issue 2 2012 issn 1936-6744 table 10 comparison matrix of the alternatives with respect to moisture percentage (high intensity) gorgana ghaemshahr gorganb ghazvin neka gorgana 2.091 1.319 1.148 1.148 ghaemshahr 1/1.903 1/2.338 1/1.820 gorganb 1.148 1.148 ghazvin 1.430 neka inconsistency = 0.01 table 11 comparison matrix of the alternatives with respect to density (high intensity) gorgana ghaemshahr gorganb ghazvin neka gorgana 1.148 1.084 1.0 1.0 ghaemshahr 1.148 1.107 1/1.037 gorganb 1/1.148 1/1.059 ghazvin 1.084 neka inconsistency = 0.00 table 12 comparison matrix of the alternatives with respect to thickness swelling (low intensity) gorgana ghaemshahr gorganb ghazvin neka gorgana 1.319 3.021 1/1.974 2.177 ghaemshahr 1.184 2.954 1/1.966 gorganb 1/2.954 1/1.469 ghazvin 2.630 neka inconsistency = 0.02 table 13 comparison matrix of the alternatives with respect to water absorption (low intensity) gorgana ghaemshahr gorganb ghazvin neka gorgana 1/2.064 2.220 1/2.277 3.227 ghaemshahr 3.898 1.0 1/3.936 gorganb 1/3.816 1/1.643 ghazvin 4.565 neka inconsistency = 0.02 ijahp article: m azizi/ a decision making model regarding wood panel selection international journal of the analytic hierarchy process 146 vol. 4 issue 2 2012 issn 1936-6744 table 14 comparison matrix of the alternatives with respect to bending strength (high intensity) gorgana ghaemshahr gorganb ghazvin neka gorgana 1/1.245 1.0 1/2.352 2.70 ghaemshahr 1.319 1.319 1/4.292 gorganb 1/2.550 3.446 ghazvin 4.643 neka inconsistency = 0.02 table 15 comparison matrices results of the alternatives with respect to moisture percentage (high intensity), density (high intensity), thickness swelling (low intensity), water absorption (low intensity) and bending strength (high intensity) neka ghazvin gorganb ghaemshahr gorgana alternatives criteria 0.203 0.232 0.219 0.108 0.237 moisture percentage (h) 0.200 0.205 0.184 0.203 0.209 density(h) 0.118 0.380 0.103 0.161 0.237 thickness swelling(l) 0.82 0.34 0.074 0.324 0.18 water absorption(l) 0.061 0.356 0.173 0.240 0.169 bending strength(h) table 16 final outcome, overall inconsistency = 0.02 alternatives gorgana ghaemshahr gorganb ghazvin neka weighing value 0.204 0.206 0.158 0.294 0.137 ranking 3 2 4 1 5 ijahp article: m azizi/ a decision making model regarding wood panel selection international journal of the analytic hierarchy process 147 vol. 4 issue 2 2012 issn 1936-6744 table 17 sensitivity analysis results description: basic priority: ghazvin (gn) ghaemshahr (gr) gorgana (ga) gorganb (gb) neka (n) changes times new priority new weight basic weight criteria 4 gn ga gr gb n 0.208 0.163 moisture percent gn ga gb gr n 0.622 gn ga gb n gr 0.723 ga gn gb n gr 0.993 3 gn ga gr gb n 0.678 0.313 density gn ga gr n gb 0.812 ga gn gr n gb 0.985 2 gn ga gr gb n 0.265 0.209 thickness swelling gn ga gr n gb 0.805 2 gn ga gr gb n 0.082 0.115 water absorption gn gr ga n gb 0.877 2 gn ga gr gb n 0.157 0.2 bending strength gn gr gb ga n 0.958 table 18 panel specifications of gorgana, ghaemshahr, gorgan b, ghazvin and neka units criteria gorgana ghaemsha hr gorganb ghazvin neka humidity (%) 6.55 8 6.4 6 6.7 density(g/cm3) 0.71 0.72 0.74 0.8 0.797 thickness swilling (%) 10.5 10.16 12 6.65 23 water absorption (%) 42 25 55 25 75 bending strength(kg/cm2) 180 195 178 210 162 ijahp article: m azizi/ a decision making model regarding wood panel selection international journal of the analytic hierarchy process 148 vol. 4 issue 2 2012 issn 1936-6744 4. discussion 4.1 criteria analysis density has a weighted value of 0.313 which is the highest priority for the panel, and the overall consistency ratio of the current research is 0.02 (table 8). density of the product is one of the major criteria which has an influence on water absorption, dimension swelling, bending resistance and internal adhesive. manufacturers tend to produce particle board that is high intensity density in comparison with other intensities (table 9). with respect to density intensities, the density with a high intensity (0.545) has the highest priority (table 9). furthermore, the difference between high and low intensities of density is large, and this influences the quality and specification of particleboard in a distinctive range. this is confirmed by the standard organization. with respect to production condition, the density intensity influences the other specifications of the board. thickness swelling has the second priority with a weighted value of 0.209 (table 8). the value of thickness swelling of the board is specified after 2 hours of floating. however, in the markets, panels which have high thickness swelling percentage are inappropriate boards and indicate an undesirable production situation with regard to raw material density and press conditions. in fact, the panels with low intensity of the thickness swelling have the higher priority. the low intensity of thickness swelling is considered (0.472) (table 9). the difference between high and low intensities of thickness swelling is high too. according to the results of the standard organization, high and low intensities of thickness swelling are 23% and 6.65% respectively (table1). this confirms the difference of low and high intensities in the current research. also, the market tends to supply panels with low intensity of thickness swelling. bending strength of the panel has third priority (0.200), otherwise, the boards which have a desirable production situation in relation to raw material and press condition, will have high bending strength. in the panel bending, strength has high sensitivity. considering the results, bending strength with high intensity is preferable. the high intensity of bending strength has a weighted value of 0.665 with highest priority, and low intensity has a weighted value of 0.094 (table 9). the difference between high and low intensities for bending strength (0.571) is very high which indicates that this criterion is very sensitive and important in panel products. otherwise, in the market, panels with low intensity of bending strength (160 kg/cm2 ) in comparison with high intensity of bending strength (210 kg/cm2 ) (table1) do not have any advantage, according to the manufacturers. desirability of the boards increases with high intensity of bending strength, however other criteria of the panels should not decrease due to it. humidity or moisture percentage has the fourth priority (0.163). this criterion depends on press temperature, dryer conditions and environment humidity. humidity of the panel must be checked after press and dryer because the humidity of the board influences density, bending strength, color etc. in addition, the panels with high intensity of moisture percentage have higher priority. the range of humidity percentage is 6-8% (table1). the results show that humidity percentage with high intensity (0.386) and low intensity (0.325) are not very different from each other ijahp article: m azizi/ a decision making model regarding wood panel selection international journal of the analytic hierarchy process 149 vol. 4 issue 2 2012 issn 1936-6744 (table 9). in fact, high intensity humidity is a favorable factor in the panel and increases bending strength and density of the panel. water absorption percentage has the lowest priority (0.115) in comparison with the other criteria. with respect to the standard measurements, the boards are floating during the test. there is not however a similar situation with regards to furniture and construction panels as well as water absorption impressed by density, press condition, glue and additive material. according to the results, the panels with low intensity of water absorption are preferable. the low intensity of water absorption with a weighted value of 0.49 is considerably different in comparison with the high intensity with a weighted value of 0.199(table 9). according to the standard organization results, high and low intensities of water absorption are 75% and 25 % respectively (table1). the large difference between high and low intensities indicates the significance of the intensities difference in the panels. water absorption criterion has lowest priority in comparison with other criteria; however, the range of intensities variation is very effective in panel specification. 4.2 alternative analysis we evaluated five wood panels in iran in terms of manufacturer' aspects under titles of gorgana, ghaemshahr, gorganb, ghazvin and neka in order to improve the panels quality and procurement of market requirements. the research was based on criteria intensities. according to the final results, panels ghazvin, ghaemshahr, gorgana, gorganb and neka have 0.294, 0.206, 0.204, 0.158 and 0.137 weighted values, respectively (table 16). accordingly, the ghazvin panel had the highest priority in comparison with the other panels. according to the criteria, intensities of thickness swelling/l, water absorption/l and bending strength/h, ghazvin panel had the highest priority. this is consistent with particleboard panel’s data from the institute of standards and industrial research of iran which shows the ghazvin panel having the best conditions with regard to thickness swelling (6.65%), water absorption (25%) and bending strength (210 kg/cm2). the results indicate that the derived scales (weighted values of the criteria) based on the judgments of comparison matrices in ahp and the actual relative weights are compatible (table 18). the ghazvin panel, which is the first alternative, has a higher priority concerning all of five criteria in comparison with the ghaemshahr panel as the second alternative. the differences are as follows: thickness swelling (0.219), moisture percentage (0.124), bending strength (0.116), water absorption (0.016) and density (0.002). this influences the final priority as shown in table 15. the ghazvin panel has a higher preference than the ghaemshahr panel in regards to the results of the three criteria, thickness swelling, moisture percentage and bending strength. of course, the range of changes in the weighted values of the panels with respect to the three above mentioned criteria is high (table 15). otherwise, the differences between the maximum and minimum of alternatives’ weighted values as to thickness swelling (second criteria), bending strength (third criteria) and moisture percentage (forth criteria) are 0.277, 0.295 and 0.129 respectively. therefore, these differences influence the panel’s priorities. moreover, thickness swelling is related to raw material density and press conditions, moisture percentage influences density, bending strength and color. bending strength is very important in marketing panel products. the range of changes in weighted values of the panels in relation to density ijahp article: m azizi/ a decision making model regarding wood panel selection international journal of the analytic hierarchy process 150 vol. 4 issue 2 2012 issn 1936-6744 criterion is low (0.025) (table 15). accordingly, in spite of the fact that the density criterion has the highest priority among other criteria the importance of comparison of the panels regarding high density intensity indicated the alternatives’ importance is very close together. the compatibility of this result can be shown in table 18 when considering minimum (0.71) and maximum (0.8) density. since there may be different judgments in the comparison of priority rates of the criteria or their sub-criteria, a sensitivity analysis is applied to achieve stability and compatibility of the analysis (saaty, 2001). within our criteria hierarchy, we find that the ratios of the alternatives could change by increasing or decreasing one of the criteria. given these results (table 17), all of the criteria are sensitive. moisture percentage and density are more sensitive than the other criteria. finally, changes in alternatives priorities are four and three times in relation to changes of moisture percentage and density’ weighing values, respectively. 5. conclusion in this research panel products of five major plants were evaluated in order to develop the best wood panel and increase its market share and competition capability in iran. after building a decision tree and obtaining weighted values of intensities, criteria and alternatives, we obtained the highest priority product which was the ghazvin panel. references alkaner, s.& das, p.k. (2008). selection of technology alternative for safe ship dismantling a case for analysis multiple – expert view on criteria weighting. or 50 conference, university of york, united kingdom. azizi, m. (2009). determination of major non-development criteria for iranian particleboard industry by applying analytic hierarchy process. the journal of the institute of wood science, 19: 95-103. azizi, m. ghorbannezhad p. and hatefnia h. (2009). estimation of wood demand for wood panels in iran by the year of 2012. journal of forestry research, 20 (2): 179182 azizi, m. & noori, p. (2007). selecting the best facial tissue with respect to customer’s perspectives by using ahp. isahp 2007, vina del mar, valparaiso, chile. azizi, m. (2008). location selection of wood composite units in khuzestan province. or 50 conference, university of york, united kingdom azizi, m. (2005). decision making for raw material procurement in paper making factory. proceeding of 8th international symposium of the ahp, university of hawaii, usa. azizi m. & taheri f. (2009). a decision making model for compatible management of the forester local cooperatives in the north of iran. proceeding of the 10 th ijahp article: m azizi/ a decision making model regarding wood panel selection international journal of the analytic hierarchy process 151 vol. 4 issue 2 2012 issn 1936-6744 international symposium on the analytic hierarchy / network process, university of pittsburgh, pittsburgh, pennsylvania, usa. bruno g., esposito e., passaro r. and g. andrea (2009). the analytical hierarchy process in the supplier selection problem, proceeding of the 10 th international symposium on the analytic hierarchy / network process, university of pittsburgh, pittsburgh, pennsylvania, usa. cellulose and packaging research group (2002). “properties and test methods, no. 2496”, institute of standards and industrial research of iran. feglar, t. (2008). the most preferable management system for small and medium enterprises. or 50 conference, university of york, united kingdom. ghorbannezhad p., azizi m. & layeghi m. (2009). a strategic model for cleaner production implementation in a paper making mill, proceeding of the 10 th international symposium on the analytic hierarchy / network process, university of pittsburgh, pittsburgh, pennsylvania, usa saaty, t.l. (2000). decision making for leaders, pittsburgh, pa: rws publications. ijahp article: unver, isik, kabak, topcu/an ahp based prioritization model for risk evaluation factors in the automotive industry international journal of the analytic hierarchy process 83 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.563 an ahp based prioritization model for risk evaluation factors in the automotive industry berna unver istanbul technical university department of industrial engineering istanbul, turkey bernaunver@sabanciuniv.edu mine isik bogazici university department of industrial engineering istanbul, turkey mineisikk@gmail.com ozgur kabak istanbul technical university department of industrial engineering istanbul, turkey kabak@itu.edu.tr y. ilker topcu 1 istanbul technical university department of industrial engineering istanbul, turkey ilker.topcu@itu.edu.tr abstract due to product variety and modeling structure, the automotive manufacturing process requires state-of-the art production methods that cause a high complexity level in operations which assembly operators work in a mixed-assembly environment. to maintain a competitive advantage, companies should take a different approach that considers the methodologies which ensure excellence in operations. this study aims to identify and prioritize potential risk factors that cause errors and failures by applying the analytic hierarchy process to improve the production quality in a manufacturing process of mixed model assembly lines in the automobile industry. thus, numerous risk factors under three main categories including human-focused, design and process-driven are discussed in this work. the most important contribution of this study is the application of acknowledgment: the authors wish to thank to managers and specialists of the automotive company for their invaluable cooperation during structuring and modeling phases of the proposed model. special recognition is due to anonymous reviewers for their helpful comments and suggestions. mailto:bernaunver@sabanciuniv.edu mailto:mineisikk@gmail.com mailto:kabak@itu.edu.tr mailto:ilker.topcu@itu.edu.tr ijahp article: unver, isik, kabak, topcu/an ahp based prioritization model for risk evaluation factors in the automotive industry international journal of the analytic hierarchy process 84 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.563 this methodology to find and rank the risk factors based on their importance in a worldleading automotive company in turkey. keywords: automotive; assembly line; workstation; process complexity; ahp 1. introduction the automotive sector has become an area that offers a wide range of options to its customers. automotive companies should adapt customer-oriented production systems to manufacture varieties of options that the customers demand. companies need to meet short product life cycles, increased production volumes, innovative designs and total quality under cost constraints. as a result, the companies have to deal with a high level of complexity to sustain their competitive advantage (personne et al., 2014). innovative, customer-focused companies that have an ability to make continuous improvement and process development will protect their position in the future and have a competitive advantage. for this purpose, companies should focus on the following factors:  quality and reliability of the products,  increase in product variety,  appropriate production time and quantity regarding customer demand,  high customer satisfaction level. it is an inevitable fact that these factors make the company operations more complex which can result in various errors in assembly lines. automotive assembly lines have mixed-model characteristics in general. mixed-model assembly lines are considered an effective method to process different products and obtain product diversity on the same line. today, assembly lines often include both manual and automatic assembly operations. efficiency and effectiveness of the assembly lines dramatically increase with improvement of the lines with automation which result in a reduction in error rates. mixed-model assembly lines include at least one of the characteristics below (urrutia et al., 2014).  technically complex operations,  excess amount of parts, components and sub-assembly,  high level of diversity,  a large number of workstations within the line,  demand with high change values in the market. the number of operations and different types of products in mixed-model assembly lines affect the process performance, line efficiency and quality aspects (zhu et al., 2008). a higher number of operations, components, and sub-assembly have a positive impact on the complexity level of assembly lines. on the other hand, technically complex operations increase the likelihood of human error if the operation is performed manually. complex operations lead to an increase in uncertainty in the process. diversity is a factor which is driven by both market needs and stable demands. therefore, a high level of ijahp article: unver, isik, kabak, topcu/an ahp based prioritization model for risk evaluation factors in the automotive industry international journal of the analytic hierarchy process 85 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.563 diversity makes assembly lines more complex (zeltzer et al.,2013). businesses, facing demand with high changes in the market, can prefer differentiation of sub-assembly processes, workstations or operations. for this reason, it also has an effect on increasing the complexity level. in order to deal with complexity in an assembly line, it is important to find out the risk factors that cause errors and failures. the human factor is one of the main sources of error in assembly lines (mattsson et al., 2012). operators make mistakes for various reasons such as forgetting, choosing the wrong material or equipment, not following instructions, and misinterpreting. for example, the major reason for choosing the wrong material or tool has a high connection with human attention. this attention requires more energy, so the body tries to remove energy-consuming behavior. this is called passive attention mode. operators, especially those who work in mixed-model assembly lines, should have an active attention mode in order to perform tasks as expected. supportive systems are necessary for operators to make right choices or operation as a guidance during passive attention mode (mattsson et al.,2012). the information presented to the operators should include instructions about both product and process. operators should make some decisions if there is more than one option. in this case, the operator needs to know information about which part to assemble. other instruction is important when unique variants are involved in order to point out how to assemble instead of what to assemble. overall, the assembly instructions should also involve a decision and support process. to analyze human errors thoroughly, workstations should be examined from the perspective of risk factors that cause complexity and become a pending problem in the assembly lines. this study aims to identify and to prioritize risk factors on workstations in an assembly line of an automotive company. within the scope of the study, a real life case study is conducted in a leading turkish automotive company. this case study utilizing an analytic hierarchy process (ahp) application on assembly lines is expected to be the main contribution of this paper as, to our knowledge, there is no application of ahp on assembly lines in the automotive sector. the paper is organized as follows. section 2 describes the proposed decision framework, identifying risk factors, clustering them, and prioritizing the factors. section 3 shows the findings, the priorities of the risk factors. conclusions and further suggestions are given in section 4. 2. proposed decision framework this study aims to identify the risk factors that cause errors and failures in a mixed assembly line of an automotive company. the decision framework that is proposed to achieve this aim is based on three main stages:  identification of the factors which cause errors on the workstations  development of a hierarchical model by clustering factors  prioritization of the factors ijahp article: unver, isik, kabak, topcu/an ahp based prioritization model for risk evaluation factors in the automotive industry international journal of the analytic hierarchy process 86 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.563 2.1. identification of factors first, a literature review was conducted in order to identify risk factors causing errors within the scope of the study. based on an intensive literature review, we found a preliminary list of factors that may cause errors on the workstations. the literature that provided the most benefit is antani k.r (2014), zeltzer l. (2013), personne r. (2014). secondly, the factor list was revised at a meeting held at the automotive company of the case study with eight managers and ten specialists working at quality control and assembly departments of the firm. managers from quality control department are:  production quality manager  production quality engineering administrator  quality method and planning administrator  assembly/suspension process quality administrator managers from assembly department are:  car assembly production administrator  minivan assembly production administrator  car assembly engineering administrator  minivan assembly engineering administrator specialists from quality control department are:  quality method and planning specialist  two quality audit specialists specialists from assembly department are:  assembly final approval leader  lean team leader  two car assembly team leaders  two minivan assembly team leaders  process improvement specialist the final list of factors includes fourteen factors as given in table 1. table 1 final factor list alert and error-proof system assembly design cognitive load component accessibility ergonomics feature design material characteristics sequence and number of task tooling and fixture training and experience utilization and saturation variation in the process work environment workers’ reliability ijahp article: unver, isik, kabak, topcu/an ahp based prioritization model for risk evaluation factors in the automotive industry international journal of the analytic hierarchy process 87 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.563 2.2. development of a hierarchical model by clustering factors in the second stage, we interacted with managers and specialists of the company to classify the factors based on their similarities. although there may be some dependencies among factors, i.e., relation between work environment and ergonomics, as the participants of the meeting argued that the model seems to have few dependencies, the dependencies are ignored. the resulting factors and the hierarchy constructed are as shown in table 2.the explanations of the main factors and factors are given in the following subsections. table 2 factor groups and factors a. design-driven factors a.1 feature design a.2 assembly design a.3 component accessibility a.4 material characteristics b. process driven factors b.1 tooling and fixture b.2 sequence and number of task b.3 variation in the process b.4 utilization and saturation b.5 alert and error-proof system c. human-focused factors c.1 ergonomics c.2 training and experience c.3 cognitive load c.4 work environment c.5 workers’ reliability design-driven factors design has a great impact on the complexity of the production systems. the dfm (design for manufacturing) approach is a widely-used method in the design process. the dfm approach, which aims to make the production easier and more economical, is based on the principle of simplification of design. therefore, it has great importance for the design process. design-driven complexity is classified into four main factors:  feature design  assembly design  component accessibility  material characteristics features are characteristics that define the geometric and functional specifications of a component. geometric features represent certain geometric accuracy and surface quality. on the other hand, functional features define specified functional requirements and expected life. ijahp article: unver, isik, kabak, topcu/an ahp based prioritization model for risk evaluation factors in the automotive industry international journal of the analytic hierarchy process 88 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.563 assembly design determines how to assemble different components. the assembly process generally consists of two distinct operations: handling followed by insertion. both operations could be done manually or automatically. for accurate assembly operations, it should be required to focus on number of parts at the assembly station and manual process time of the components and tools. component accessibility shows lack of direct access or difficulty in access to the assembly area or components and has impact on complexity level. the difficulty level of accessing both area and components causes both increases in errors and complexity level while operating. material characteristics reflect certain properties that may be required due to the function of a part and can introduce complexity in the assembly process due to other inherent properties. material characteristics contain mechanical requirements such as strength, resistance to breakage, hardness, and ability to withstand vibrations; physical requirements such as weight, electrical conductivity, and appearance; and service requirements that show an ability to process under extreme temperature or corrosion resistance. process driven factors process driven factors are essential parameters to design, develop and implement a process that produces components that could consistently meet design specifications depending on a specified cost. process driven factors include:  tooling and fixture  the number of tasks and assembly sequence  variation in the process  utilization and saturation  alert and error-proof systems tooling and fixtures are used to hold the workpiece securely and present the workpiece to the machining tool or to the operator in order to enable efficient processing while meeting dimensional specification and cycle time * . a variety of tooling and fixtures are used in operations. their reliability is taken into consideration when performing an operation. tooling and fixtures should be designed within the allowable ranges. both the number of tasks in the station and assembly sequence have an impact on the process. the number of individual tasks assigned to the workstation reflects the total number of operations within the station. an increase in the number of tasks requires more effort for an operator while performing many operations in turn. the assembly sequence may be defined or left up to the operator for the consecutive operations. if the sequence is left up to the operator, there is certainly a decision-making mechanism for the operator. however, there are supportive tools to prevent errors and to ensure the operator’s performance if the sequence is predefined and specified. * cycle time is used in the meaning of the time it takes to produce successive units on an assembly line. ijahp article: unver, isik, kabak, topcu/an ahp based prioritization model for risk evaluation factors in the automotive industry international journal of the analytic hierarchy process 89 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.563 variability reflects lack of repeatability when completing a given task. repeating the same operation several times enables operators to get experience and to decrease completion time for a given task. on the other hand, variation also represents product mix, which refers to the difference between expected and real frequency among models, versions and options. the utilization is defined as the ratio of the total time of the tasks assigned to the station in the cycle time. utilization allows professionals to make a performance measurement on the basis of work stations. additionally, saturation means total preliminary times for the operator’s tasks. the systems applied to prevent and remove errors can be grouped under the heading of error and fool-proofing tools. alert and error-proof systems are especially important for critical operations on the operational base. usage of these tools provide prevention of errors for operators with audio-visual systems at certain areas of the work stations. human-focused factors human-focused factors include the development of tools that facilitate improved performance, safety, and user satisfaction. these factors have a direct effect on human performance. human-focused factors are classified as:  ergonomics  training and experience  cognitive load  work environment  workers’ reliability ergonomics analyzes the interactions between the worker and the working environment. if there is a mismatch between operator and environment, the operator’s ability to perform a task decreases and leads to the possiblity that it will become seriously dangerous. this situation may lead to fatigue and illness in time. from this point of view, ergonomic conditions and their impacts are extremely important for operator’s health and ability to work confidently. training and experience level have a major role in matching operations with workers. the number of training hours in the station is a certain factor regarding learning and performing the task in a right way. cognitive loading can be evaluated as a total function of control, material choice, equipment choice and judgment. operators should make the right choice from a number of alternatives in the station. the excess amount of control, choice, and judgment for the operator directly affects the error risk. in other words, as the level of cognitive load for the operator increases, the risk of errors increases. work environment includes environmental conditions and illusion, and has direct effects on operators in the workstation and surrounding areas. the level of lighting, noise, ijahp article: unver, isik, kabak, topcu/an ahp based prioritization model for risk evaluation factors in the automotive industry international journal of the analytic hierarchy process 90 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.563 motion, thermal stress, and air quality constitutes work environment conditions. on the other hand, illusion can be defined as an instance of a wrong or misinterpreted perception of objects by operators. worker reliability reflects the power of the worker to cope with the complexity. worker reliability ratings are formed depending on mental ability analysis, competence, and physical properties. the power of determination of worker reliability is crucial for the operation to assign it to the appropriate operator. 2.3. prioritization of the factors after the determination of the factors, we developed a hierarchical model at the second stage. the ahp, proposed by saaty (1980), is an approach best suited to treat hierarchical decision-making problems and therefore, for these kind of problems, it is the most widely used method in practice and the most cited method in the literature. besides, in this study, as the aim is just prioritizing identified criteria (factors) and there is no evaluation of alternatives, a prioritization technique such as ahp is necessary and used in the third stage of this study. first, we utilized super decisions software to represent the hierarchical model as can be seen in figure 1 (super decision, 2018). then, a pairwise comparison questionnaire survey was prepared in accordance with ahp. accordingly, a pairwise comparison questionnaire, a part of which can be seen in figure 2, was sent to the managers and specialists to assess their judgments representing the relative importance of factor groups with respect to risk evaluation of assembly lines and the relative importance of factors with respect to each factor group. in the following step, we computed geometric means of all paired comparison judgments for each question in order to reveal the aggregated group judgments. utilizing the assess/compare module of the super decisions software, we arranged these group judgments in pairwise comparison matrices. at the final step of the decision-making process, we computed the priorities of the factor groups and factors as well as inconsistencies, utilizing the computations module of the same software which did all the matrix algebra. the revealed priorities are given in the following section. the inconsistency ratio for the pairwise comparison matrix with respect to the goal is 0.05%. the inconsistency ratios for the pairwise comparison matrices with respect to design driven, process driven, and human focused factors groups are 1.11%, 1.51%, and 0.65%, respectively. ijahp article: unver, isik, kabak, topcu/an ahp based prioritization model for risk evaluation factors in the automotive industry international journal of the analytic hierarchy process 91 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.563 figure 1 hierarchical model figure 2 part of pairwise comparison questionnaire 3. findings according to the results of the survey, the priorities of the main factors are shown in figure figure 3. the most important main factor found is “process driven factors”. it’s importance is 46.41% (nearly half of the total priorities of main factors). therefore, we can state that “process driven factors” are significantly more important than both humanfocused and design-driven factors. the other two main factors, namely “human-focused factors” and “design-driven factors” have a very similar importance level around 27%. the importance level of the factors in design-driven factors are found as given in figure 4. the results indicated that “assembly design” is the most important factor among design-driven factors with a priority of 40.94%. the importance of other designdriven factors is around 20% each. ijahp article: unver, isik, kabak, topcu/an ahp based prioritization model for risk evaluation factors in the automotive industry international journal of the analytic hierarchy process 92 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.563 figure 3 priorities of factor groups figure 4 priorities of design-driven factors the results related to the process-driven factors are presented in figure 5. the participants assessed the importance of “variations in the process” as 36.15%. other process driven factors except the “tooling and fixture” factor have an importance around 19%. the tooling and fixture factor is the least important with 7.2%. figure 5 priorities of process driven factors ijahp article: unver, isik, kabak, topcu/an ahp based prioritization model for risk evaluation factors in the automotive industry international journal of the analytic hierarchy process 93 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.563 the importance of the human-focused factors is presented in figure 6. according to judgments of managers and specialists, the most important human-focused factor is found to be “training and experience” (33.60%), followed by “reliability of workers” (21.34%). “work environment” is not as important as other human-focused factors. figure 6 priorities of human-focused factors when the priorities are aggregated in accordance with ahp, global priorities of the factors are seen in figure 7, and the descending order of priorities can be seen in table 3. based on global priority values, the most important factor is found as “variation in the process” (16.78%). a group of factors follows this factor with their global importance between 8%-11%, namely, “assembly design”, “training and experience”, “alert and error-proof system”, “utilization and saturation”, and “sequence and number of task”. on the other hand, the factors “tooling and fixture” and “work environment” are the least important factors for the participating managers and specialists. ijahp article: unver, isik, kabak, topcu/an ahp based prioritization model for risk evaluation factors in the automotive industry international journal of the analytic hierarchy process 94 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.563 figure 7 global priorities of the factors table 3 descending order of global priorities of the factors factor priority b.3 variation in the process 16.78% a.2 assembly design 10.94% c.2 training and experience 9.03% b.5 alert and error-proof system 8.96% b.4 utilization and saturation 8.74% b.2 sequence and number of task 8.60% a.3 component accessibility 5.75% c.5 workers’ reliability 5.74% a.1 feature design 5.08% a.4 material characteristics 4.95% c.3 cognitive load 4.84% c.1 ergonomics 4.72% b.1 tooling and fixture 3.34% c.4 work environment 2.56% ijahp article: unver, isik, kabak, topcu/an ahp based prioritization model for risk evaluation factors in the automotive industry international journal of the analytic hierarchy process 95 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.563 4. conclusions and further suggestions this study aims to identify and prioritize risk factors in an assembly line of an automotive company. for this purpose, a three-stage methodology was used. initially, the risk factors were identified based on literature review. subsequently, a hierarchical model was developed by clustering the factors. finally, the factors were prioritized using the ahp method. the most important contribution of this study is the application of this methodology to find and rank the risk factors based on their importance in a world-leading automotive company in turkey. the ahp application utilized is expected to be the first application on assembly lines in the automotive sector. we integrate literature, expert opinions, field experience and real-life data to shape the risk factors in workstations of an automotive company. according to the results, we determined 14 risk factors under three factor groups. the priorities of these factors are quite interesting. we found variations in the process as the most important factor. in the mixed-model assembly lines, the variations are inevitable because of its nature. actually, these lines are designed for handling variations. therefore, the complexity of the line that causes errors and failures are due to the inherent property of the mixed-model assembly line. however, the situation is not as desperate as it seems since the variation in a line or a single station can be controlled to decrease the complexity. for instance, by planning a uniform production plan that will incur an appropriate scheduling of the models, versions, or options; the management can reduce the assembly time variations in the stations. assembly design is the second important factor based on global priorities. design has enormous impacts on assembly lines because good assembly design provides cost minimization as well as process improvements. there are possible improvements in the context of the assembly design factor. for instance, changes in specific task’s designs via new assembly technologies can result in better assembly outputs, when considered from the perspective of long-term consequences. the only drawback is the high investment required in the initial stage to make assembly improvements. despite the high initial investment, it is an obvious fact that companies should give priority to assembly design to be able to compete in the market with the help of technological advances. the training and experience factor, which is the third important factor, can be considered as the option with the least cost to achieve the goal of risk reduction. due to changing technology, employees become under-qualified which results in a high probability of failure. this factor aims at removing potential causes of failure due to lack of knowledge. it necessitates understanding the potential risks related to humans in order to shape a specific training which allows employees not only to protect themselves but also make timely and accurate actions when necessary. when we shared our suggestions with the managers and specialists of the company, they found them very interesting and useful. we decided to start a new study to develop a multi-criteria decision support system for the prediction and quantification of the risk of errors on the workstations in the company. in this future research, we will take the risk of errors into consideration as alternatives. then, we will identify attributes, as sub-factors ijahp article: unver, isik, kabak, topcu/an ahp based prioritization model for risk evaluation factors in the automotive industry international journal of the analytic hierarchy process 96 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.563 of the decision model on hand, to evaluate these alternatives. before prioritizing the attributes, the dependencies among attributes will be discussed with the managers and specialists. if dependencies and feedbacks exist, the model will be a network and the analytic network process (anp) approach will be utilized. the model developed is a general model that could be applied on any mixed model assembly line in an automobile industry, not just the particular company studied. however, the priorities may be company specific. it would be interesting to implement the model in another mixed model assembly line with different managers and specialists and compare the priorities of the factors. ijahp article: unver, isik, kabak, topcu/an ahp based prioritization model for risk evaluation factors in the automotive industry international journal of the analytic hierarchy process 97 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.563 references antani, k.r. 2014. a study of the effects of manufacturing complexity on product quality in mixed-model automotive assembly, doctoral dissertation, clemson university, https://tigerprints.clemson.edu/cgi/viewcontent.cgi?article=2541&context=all_dissertatio ns mattsson, s. & gullander, p. & harlin, u., & bäckstrand, g. & fasth, a. & davidsson, a., 2012. testing complexity index – a method for measuring perceived production complexity, 45th cirp conference on manufacturing systems, 3, 394-399. doi: https://doi.org/10.1016/j.procir.2012.07.068 personne, r. & matinlassi, v., 2014. part assurance in a mixed-model assemble line, master of science thesis. saaty, t.l. (1980) multicriteria decision-making: the analytic hierarchy process. pittsburgh, pa: rws publications. super decisions, 2018, https://www.superdecisions.com/ urrutia, u.a. & webb, p. & summers, m., 2014. analysis of design for x methodologies for vomplex assembly processes : a literature review, asme 2014 international design engineering technical conferences and computers and information in engineering conference, 1-11. doi: doi:10.1115/detc2014-34955 zeltzer, l. & limeré, v. & landeghem, h.v. & aghezzaf, e. & stahre, j., 2013. measuring complexity in mixed-model assembly workstations, international journal of production research, 51(15), 4630-4643. doi: https://doi.org/10.1080/00207543.2013.783246 zhu, x. & hu, j.s. & koren, y. & marin, p.s., 2008. modeling of manufacturing complexity in mixed-model assembly lines, journal of manufacturing science and engineering, 130, 1-10. doi: 10.1115/1.2953076 ijahp article: al-rahbi, abushammala, qazi/application of the analytic hierarchy process for management of soil erosion in oman international journal of the analytic hierarchy process 104 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.683 application of the analytic hierarchy process for management of soil erosion in oman alia khalfan hamdoon al-rahbi department of civil engineering, middle east college muscat, oman alrahbialia@hotmail.com mohammad f.m. abushammala* department of civil engineering, middle east college muscat, oman eng_abushammala@yahoo.com wajeeha a. qazi department of civil engineering, middle east college muscat, oman wajiha23@hotmail.com abstract the level of soil degradation worldwide is alarming due to its potential for causing serious problems such as threatening food security. the sultanate of oman faces the problem of soil erosion which also disturbs waterways, infrastructure and agriculture. the problem of soil erosion has become increasingly apparent in oman and is due to inappropriate land management. this occurs particularly with dams where sedimentation is a common issue and in the aflaj water systems. currently, there are no appropriate methods being practiced that would overcome the problem of soil erosion in oman, and no specific studies are available that discuss suitable methods for soil protection in oman. therefore, this study is an initiative to overcome the problem of soil erosion in oman by proposing the best soil erosion protection method using the analytic hierarchy process (ahp). in order to achieve this goal, this study reviews different methods of preventing soil erosion and the factors governing their selection. the important and controlling factors were considered in an ahp model to rank the soil erosion protection methods. based on the ahp model, pairwise comparisons were conducted with the help of 15 experts from different sectors including authorities responsible for soil conservation in oman, decision-making governmental departments and research institutes. the results of the ahp analysis indicated that the most suitable soil erosion protection method, based on the considered factors, is erosion control fences, followed by protection of the gully head and revetment. as no previous studies on suitable methods for soil conservation in oman exist, this study bridges the gap by providing valuable information on the best soil protection methods that could help soil conservation authorities in oman as they decide about possible future strategies.the study also indicated that the ahp model is a suitable guiding framework for decision-making because it involves a group of experts for the selection of soil protection methods based on the situation in oman. mailto:alrahbialia@hotmail.com mailto:eng_abushammala@yahoo.com mailto:wajiha23@hotmail.com ijahp article: al-rahbi, abushammala, qazi/application of the analytic hierarchy process for management of soil erosion in oman international journal of the analytic hierarchy process 105 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.683 keywords: soil erosion; soil protection; analytic hierarchy process; multi-criteria decision analysis 1. introduction soil is the earth’s fragile skin that anchors all of life to earth. it comprises countless species that create a dynamic and complex ecosystem and is one of the most precious resources to humans. the loss of half of the topsoil on the planet in the last 150 years can be attributed to the phenomenon of soil erosion (pimentel, 2006). the loss of soil from land surfaces by erosion is a global issue, which has adverse effects on the productivity of all natural ecosystems including agriculture, forests and rangeland ecosystems. soil erosion is considered a major environmental problem and is a cause of increased concern around the world (avni, 2008; pimentel, 2006). over 35% of new south wales in australia is affected by some sort of soil erosion, which impacts the productivity of agricultural land and the quality of river systems and stormwater catchments due to sediment transport (department of environment and climate change nsw, 2007). it has been estimated that water erosion affects 41 million hectares of total area in the near east and north africa (nena) region. approximately 60% (135 million hectares) of soil degradation resulted from wind erosion, which makes it the most common environmental problem faced by the region. the cropped areas in the nena region are severely affected by the accumulation of eroded material in agriculture fields, irrigation canals and water harvesting areas. the gezira irrigation scheme in sudan is an example of the problem. here, wind erosion has resulted in sand encroaching which affects the efficiency of the irrigation system. moreover, as a result of water and wind erosion, the area with fertile cultivated soil in the el-witia area (libya) decreased by 31% from 1986 to 1996, whereas the area with low fertile soils increased by 38% (food and agriculture organization of the united nations (fao), 2015). in arid and semi-arid regions, soil erosion has become a critical issue because it significantly affects the agricultural potential in those regions. the process of gully erosion within the drainage basins is very active in the rocky and hilly areas of the middle east, such as most of jordan, syria and northern egypt, and this severely decreases biodiversity, natural biomass, agricultural potential, and irrigation efficiency (avni, 2008). similar negative impacts of soil erosion have become increasingly apparent in oman, especially in the case of dams and the aflaj water systems. the omani government has built a number of recharge dams across wadis to detain runoff and recharge the aquifers, where groundwater reserves store approximately 99% of the demand for irrigation water (prathapar & bawain, 2014). the gully erosion within the drainage basins results in sediment transport along with water. these sediments settle upstream from the dam during detention and significantly reduce the rate of infiltration and recharge of the aquifers in oman (prathapar & bawain, 2014). moreover, 40% of the agricultural land in villages depends on the aflaj water system for irrigation. sand encroachment into these aflaj water systems from soil erosion decreases irrigation efficiency and, therefore agricultural potential (prathapar & bawain, 2014). the only possible way to deal with these issues is to adopt a suitable method for the prevention of soil erosion, but currently no information is available on proper practices of soil protection in oman due to the lack of specific studies based on the situation. ijahp article: al-rahbi, abushammala, qazi/application of the analytic hierarchy process for management of soil erosion in oman international journal of the analytic hierarchy process 106 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.683 the decision to select an appropriate soil protection method is complex and involves many factors, such as types and characteristics of soil, climate and cost of the soil protection methods. as the number and complexity of soil protection methods for conservation of soil grows, so do the strategic judgements required for the effective assessment of these soil erosion protection methods. this has led to the popularity of multi-criteria decision analysis (mcda) methods being used in environmental decisionmaking (loken, 2007). this research opted to use a mcda technique, specifically the analytic hierarchy process (ahp), to address oman’s growing soil erosion problem by selecting the best soil protection method while considering the factors influencing their performance. this research will hopefully bridge the gap between insufficient knowledge on suitable methods for preventing soil erosion that can be adopted in oman and the controlling criteria, and will assist the omani authorities who are responsible for soil protection in decision-making. 2. methodology the ahp method, which was developed by thomas saaty in 1970, allows decision makers to arrange the complex problem into a multilevel hierarchy structure (saaty, 1980). the top level of the hierarchy structure represents the goal of the study, the middle level represents the criteria and the lowest level consists of the alternatives (alexander, 2012). it allows decision makers to conduct pairwise comparisons to evaluate the importance of the variables in each level of the hierarchy, and the alternatives present in the lowest level to make the best decision from among the several alternatives. moreover, this method performs a consistency test to screen out inconsistent judgements. the effectiveness of the ahp method has led to its popularity in research. furthermore, the international scientific community that deals with complex decision problems has declared the ahp method a robust and flexible method (ehrgott, 2010). the forthcoming steps of the ahp method that were developed by saaty have been followed in this study (nixon, dey, ghosh, & davis, 2013; saaty, 1980; saaty & sagir, 2009; qazi, abushammala & azam, 2018). 2.1 define objective this stage includes defining the objective statement, which is to rank the soil erosion protection methods for soil conservation in oman and the factors influencing the goal. 2.2 construct hierarchy structure the next phase is the construction of the problem as a hierarchy. this involves splitting the goals, criteria and alternatives of the complex problem into multiple levels. 2.3 make a pairwise comparison to generate a matrix once the hierarchy structure is developed, a pairwise comparison is performed to form judgmental matrices. first, the pairwise comparison of the selected criteria is performed with respect to the objective of study, and then the comparison between the alternatives with respect to the individual criterion is performed. saaty’s nine-level standardized comparison scale which is illustrated in table 1 was used to assign the judgements in the pairwise comparisons (ehrgott, 2010). ijahp article: al-rahbi, abushammala, qazi/application of the analytic hierarchy process for management of soil erosion in oman international journal of the analytic hierarchy process 107 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.683 table 1 saaty’s nine-point scale for pairwise comparison numerical rating verbal judgments of preferences between alternatives i and alternatives j 1 i is equally important to j 3 i is slightly more important than j 5 i is strongly more important than j 7 i is very strongly more important than j 9 i is extremely more important than j 2,4,6,8 intermediate values 2.4 determine the priorities of the alternatives in order to determine the priorities of alternatives, the ahp methodology was used with the assistance of the super decisions software to solve the judgmental matrices. the results obtained from the pairwise comparisons were inserted into the super decisions software, which uses the principles of the ahp methodology. the working principle of the ahp method involves the computation of local priority vectors (pve) by normalizing the vectors in each column of the matrix, followed by the calculation of the average of the resulting matrix. the global priorities of each alternative are then computed by synthesizing the local priorities over the hierarchy. 3. results and discussion 3.1 ahp model development the main objective of this research was to select the best soil erosion protection method to conserve soil in oman and control the problem of sediment which affects water quality. in order to apply the ahp method, a consultation with a group of experts was carried out to identify/modify the soil erosion protection methods and the criteria influencing their selection. after a thorough literature review, a consultation process was conducted with 15 experts from different sectors including authorities responsible for soil conservation in oman, decision-making governmental departments and research institutes. the expert advisory group was formed based on their related knowledge background and experience in the field of study and involved specialists from different organizations. the group comprises managers whose duties include performance monitoring and measurement, and experts from several operational levels such as engineers and technicians, academic staff, and consultants. finally, based on the ahp method, the modified criteria and alternatives were distributed into a multi-level hierarchy structure as illustrated in figure 1. ijahp article: al-rahbi, abushammala, qazi/application of the analytic hierarchy process for management of soil erosion in oman international journal of the analytic hierarchy process 108 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.683 figure 1 hierarchy structure the hierarchy is arranged into three levels (figure 1). the top level consists of the goals of the current study. the middle level is comprised of the main criteria including the types and characteristics of soil in oman (rocky and sandy), the climate of oman (arid/semi-arid climate) and the cost of the soil protection methods, which are discussed in table 2. meanwhile, the bottom level represents the alternatives which includes stone gabions, land husbandry, windbreaks, hollows or pits, soil conditioning, field cropping practices, erosion control fences, protection of the gully head, reshaping gully systems, revetments, and brush mattress as shown in table 3. ijahp article: al-rahbi, abushammala, qazi/application of the analytic hierarchy process for management of soil erosion in oman international journal of the analytic hierarchy process 109 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.683 table 2 finalized criteria with description criteria description type and characteristics of soil considering the type and characteristics of the soil is important for selecting the appropriate soil protection method, as soil erodibility is an estimate of the soil’s ability to resist erosion, depending on the physical characteristics of each type of soil. the texture of soil is a factor that affects erodibility, however permeability, organic matter and structure are also important. for instance, soils with high infiltration rates, organic matter and better soil structure show better resistance to erosion; whereas, sandy loam, sand and loam textured soils are less susceptible to erosion than very fine sand, silt and certain clay textured soils. this factor helps determine how vulnerable the soil is to erosion, and the level and type of soil protection measures that should be taken (el-swaify, 1997; ritter, 2012). climate soil erosion is the process of the topsoil being worn away by natural physical forces like wind and water, or through tillage. therefore, climate has a great influence on the selection of a soil erosion prevention method because it helps identify the sort of erosion that prevails in the particular region, i.e., wind erosion or water erosion. depending on the reason for the land degradation, a better and effective soil erosion prevention method can be adopted that targets the specific type of erosion (el-swaify, 1997; ritter, 2012). cost of soil protection method the cost of soil protection methods varies and hence, this factor should be considered in order to make an economically sound decision. not only should the adopted method help reduce the soil erosion, but it should also be economically feasible. therefore, the cost involved in implementing the strategy and its operational and maintenance cost should be taken into account (el-swaify, 1997). ijahp article: al-rahbi, abushammala, qazi/application of the analytic hierarchy process for management of soil erosion in oman international journal of the analytic hierarchy process 110 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.683 table 3 alternatives selection and description alternatives description stone gabion gabions are baskets composed of hexagonal woven wire mesh. when filled with rocks, these baskets form flexible and permeable structures like a retaining wall to control erosion. the wires are either heavily zinc coated, pvc coated or are made of stainless steel. stone gabions are available in various sizes, but the most popular size is 2 m long x 1 m wide x 1 m tall (south african national roads agency, n.d.). land husbandry this method involves the active management of vegetation, rainwater, soils and slopes so that there is an increase or no loss of stability, productivity and usefulness for the chosen purpose. this management is performed in a wide range of environments from fields to landscapes, with native vegetation of every kind, and in pastures and plantations (overseas development institute, 1997). windbreaks windbreaks are trees planted in rows to prevent wind from causing soil erosion. the number of rows in a windbreak depends on the space available and the reason for planting them. however, the windbreaks that are intended to reduce soil erosion usually consist of a single row that is planted parallel to the cropping patterns (wilson & josiah, 2004). hollow or pit pits are planted to harvest precipitation which in turn prevents water runoff, reduces erosion and increases infiltration. pits are holes that are 5-15 cm deep that are dug 50-100 cm apart from each other. this method not only reduces soil erosion, but also helps stabilize production of crops by increasing soil moisture or allowing irrigation in a dry period (bot & benites, 2005; waelti & spuhler, 2019). soil conditioning the use of soil conditioners or composted mulch is another way to reduce the direct impact of wind and rain on the soil surface, improve soil structure, water holding capacity, and water infiltration, increase plant growth and reduce runoff. it causes an increase in plant growth which is crucial for long-term erosion control (department of environment and climate change nsw, 2007). field cropping practice in this method, a series of different types of crops are grown on the given land area in sequential seasons. growing only one type of crop for a long time weakens the soil and leaves it uncovered for periods of time, making it susceptible to erosion. this method promotes soil health by restoring nutrients and mitigating the accumulation of pathogens and pests which results from growing ijahp article: al-rahbi, abushammala, qazi/application of the analytic hierarchy process for management of soil erosion in oman international journal of the analytic hierarchy process 111 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.683 only one plant species (hillel & hatfield, 2005). erosion control fences these fences are made up of synthetic geotextile fabric that is woven to provide small openings to let water pass through and retain sediments. the low permeability rates of these fences allow sediments to settle and water to pass through slowly. they have been used in the construction industry for decades to keep soil on construction sites. erosion control fences work best when they are installed on uniform slopes with few obstructions (robichaud & brown, 2002; u.s. environmental protection agency, 2012). protection of the gully head in this method, diversions are constructed above the gully area to divert runoff away from the gully heads to reduce scouring, and to let sediments accumulate and allow vegetation to grow in the gullies. since surface water runoff erodes soil along drainage lines, it is important to protect gully heads and prevent headward erosion (department of economic development, 2017; geyik, 1986). revetments revetments are onshore structures which protect the shoreline from erosion, infiltration by water or scouring by waves to ensure and improve the stability of bank slopes. these structures are usually made of a layer of concrete, stone, asphalt or masonry to shield the natural sloping shorelines (new york state department of environmental conservation, n.d.; oyo international corporation, 2007). brush mattress this method involves laying a brush mattress on a slope and fastening it with wire and stakes. these mattresses provide dense woody vegetation on slopes allowing them to protect the soil surface from erosive forces. they are used mainly on streambanks where the velocity is less than 6 feet per second and erosive conditions are formed due to excessive runoff (new york state department of environmental conservation, 2005). reshaping gulley systems this method should be adopted in cases where the gulley systems are very severely eroded with crumbling, collapsing and dried out vertical sidewalls. in such cases, it is more practical to reshape the gulley system entirely. when gulley heads and banks are reshaped, they should be shaped into a gentle slope (1:1 slope) (conservation management services, 2015; geyik, 1986). 3.2 application of the ahp a total of four matrices were arranged for pairwise comparison based on the ahp methodology. the first matrix presented a pairwise comparison of the criteria with respect to the goal of the study, and the other three matrices involved in the pairwise comparison of soil protection methods with respect to each criterion (oman soil, oman climate and cost). the matrices were formed by the judgements of 15 experts, who ijahp article: al-rahbi, abushammala, qazi/application of the analytic hierarchy process for management of soil erosion in oman international journal of the analytic hierarchy process 112 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.683 allocated a numerical rating for the comparison of each element in the questionnaire from saaty’s nine-point scale (table 1). only two questionnaires failed the consistency test and were removed from the study, while the remaining 13 questionnaires proved to be valid and usable because most of the judgements that were given by the experts were consistent. next, the hierarchy structure and judgements that were obtained from all of the valid questionnaires were inserted in the super decisions software, which processes data based on the ahp methodology. the results of the ranking of the soil protection methods computed by the software are displayed in terms of ideal priorities, normalized priorities and raw priorities (figure 2). figure 2 overall priorities of all of the alternatives obtained from the super decisions software the idealized priorities were obtained by dividing the limiting column by the largest values in the column. consequently, the results showed that the ‘protection of the gully head’ method has 74.7% of the appeal of the ‘erosion control fences’ method, ‘revetment’ has about 63.7%, ‘soil conditioning’ has 62%, ‘wind breaks’ has 56.2%, and ‘using stone gabions’ is about 44.8% as appealing as the ‘erosion control fences’ method. ijahp article: al-rahbi, abushammala, qazi/application of the analytic hierarchy process for management of soil erosion in oman international journal of the analytic hierarchy process 113 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.683 the analysis in this study showed that the best soil erosion protection method based on the conditions in oman is the ‘erosion control fences’ method, followed by ‘protection of the gully head’, ‘revetment’, ‘soil conditioning’, ‘wind breaks’ and ‘using stone gabions’. 4. conclusion this study assessed the different soil erosion protection methods for soil conservation in oman using the ahp technique. tackling soil erosion is a pressing need because of the problems that erosion causes. the study demonstrates that the ahp model is a suitable guiding framework for decision-making because it involves a group of experts for the selection of soil protection methods based on the situation in oman. based on the criteria preference for soil conservation in oman, the results from the ahp analysis show that the ‘erosion control fences’ method is the best performing option followed by ‘protection of the gully head’ and ‘revetment’. the erosion control fences method to prevent soil erosion has huge future potential in oman because the country has mostly rocky and hilly areas which are prone to soil erosion during heavy rains. this method is very beneficial and suitable for oman because it works best when installed on uniform slopes. when used in this manner, these fences could help keep the eroded soil on the slopes and prevent its intrusion onto the highways and roads, and therefore keep it out of wadis and waterways. moreover, the process of gully erosion within the wadis is also very active in oman, and by adopting either the gully head protection method or revetments, as proposed by this study, the shorelines could be protected from erosion and scouring. the gully head protection method would promote sediment accumulation and vegetation growth in wadis as well. overall, the proposed methods to conserve soil in oman have huge future potential. additionally, the adoption of these strategies to address the problem at the point of origin would help protect the waterways from sediment, the quality of the water, and the agricultural area, and would also help maintain the efficiency of the irrigation systems in oman. ijahp article: al-rahbi, abushammala, qazi/application of the analytic hierarchy process for management of soil erosion in oman international journal of the analytic hierarchy process 114 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.683 references avni, y. (2008). glacial–interglacial cycles, soil erosion and natural desertification in the middle east. sediment dynamics in changing environments (proceedings of a symposium held in christchurch, 325, 383–390. alexander, m. (2012). decision-making using the analytic hierarchy process (ahp) and sas/iml ® . baltimore, md: nesug. bot, a., & benites josé. (2005). the importance of soil organic matter: key to droughtresistant soil and sustained food production. rome: food and agriculture organization of the united nations. conservation management services. (2015). soil erosion control and veld rehabilitation for the gouritz cluster biosphere reserve. the gouritz cluster biosphere reserve and conservation management services. retrieved from https://gouritz.com/wpcontent/uploads/2019/02/soil-erosion_soil-erosion-control-and-veld-rehabilitationguidelines_coetzee-stroebel_2015-compressed.pdf doi: https://doi.org/10.4135/9781526465412 department of environment and climate change nsw. (2007). reducing soil erosion with compost. retrieved from https://www.epa.nsw.gov.au/yourenvironment/waste/waste-facilities/organics-processing-facilities//media/epa/corporate%20site/resources/warrlocal/070413-reducingerosion.ashx?la=en&hash=23443ee8260232d77e5518995e3ecb01248dc2ba department of economic development. (2017). gully erosion. retrieved from http://agriculture.vic.gov.au/agriculture/farm-management/soil-and-water/erosion/gullyerosion el-swaify, s. (1997). factors affecting soil erosion hazards and conservation needs for tropical steeplands. soil technology, 11(1), 3–16. doi: https://doi.org/10.1016/s09333630(96)00111-0 ehrgott, m. (2010). multiple criteria decision making for sustainable energy and transportation systems. berlin: springer. francis, c. a. (2005). crop rotation. in d. hillel (eds.), encyclopedia of soils in the environment. oxford: elsevier/academic. food and agriculture organization of the united nations (fao). (2015). status of the worlds soil resources: main report. rome. geyik, m. p. (1986). fao watershed management field manual: gully control. rome: fao. https://gouritz.com/wp-content/uploads/2019/02/soil-erosion_soil-erosion-control-and-veld-rehabilitation-guidelines_coetzee-stroebel_2015-compressed.pdf https://gouritz.com/wp-content/uploads/2019/02/soil-erosion_soil-erosion-control-and-veld-rehabilitation-guidelines_coetzee-stroebel_2015-compressed.pdf https://gouritz.com/wp-content/uploads/2019/02/soil-erosion_soil-erosion-control-and-veld-rehabilitation-guidelines_coetzee-stroebel_2015-compressed.pdf ijahp article: al-rahbi, abushammala, qazi/application of the analytic hierarchy process for management of soil erosion in oman international journal of the analytic hierarchy process 115 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.683 loken, e. (2007). use of multicriteria decision analysis methods for energy planning problems. renewable and sustainable energy reviews, 11(7), 1584–1595. doi: https://doi.org/10.1016/j.rser.2005.11.005 new york state department of environmental conservation. protection against wave-based erosion. retrieved from https://www.dec.ny.gov/docs/water_pdf/waverosionrevetment.pdf new york state department of environmental conservation. (2005). biotechnical measures for erosion and sediment control. retrieved from http://www.dec.ny.gov/docs/water_pdf/sec4.pdf nixon, j.d., dey, p.k., ghosh, s.k., & davis, p.a. (2013). evaluation of options for energy recovery from municipal solid waste in india using the hierarchical analytical network process. energy, 59, 215–223. doi: https://doi.org/10.1016/j.energy.2013.06.052 overseas development institute. (1997). better land husbandry: re-thinking approaches to land improvement and the conservation of water and soil. retrieved from https://www.odi.org/sites/odi.org.uk/files/odi-assets/publications-opinion-files/2953.pdf doi: https://doi.org/10.1163/1872-9037_afco_asc_1285 oyo international corporation. (2007). countermeasures against river erosion. retrieved from http://open_jicareport.jica.go.jp/pdf/11856424_04.pdf pimentel, d. (2006). soil erosion: a food and environmental threat. environment, development and sustainability, 8, 119–137. doi: https://doi.org/10.1007/s10668-0051262-8 prathapar, s.a., & bawain, a.a. (2014). impact of sedimentation on groundwater recharge at sahalanowt dam, salalah, oman. water international, 39(3), 381–393. doi: https://doi.org/10.1080/02508060.2014.895889 qazi, w.a., abushammala, m.f.m, & azam, m. (2018). multi-criteria decision analysis of waste-to-energy technologies for municipal solid waste management in sultanate of oman. waste management and research, 36(7), 594–605. doi: https://doi.org/10.1177/0734242x18777800 robichaud, p. r., & brown, p. r. (2002). silt fences: an economical technique for measuring hillslope soil erosion. fort collins, co: u.s. dept. of agriculture, forest service, rocky mountain research station. doi: https://doi.org/10.2737/rmrs-gtr-94 ritter, j. soil erosion – causes and effects (2012). ministry of agriculture, food and rural affairs. south african national roads agency. gabions, erosion protection. retrieved from https://www.nra.co.za/content/chapter_15.pdf saaty, t.l. (1980). the analytic hierarchy process. new york: mcgrawhill. http://www.dec.ny.gov/docs/water_pdf/sec4.pdf https://www.odi.org/sites/odi.org.uk/files/odi-assets/publications-opinion-files/2953.pdf ijahp article: al-rahbi, abushammala, qazi/application of the analytic hierarchy process for management of soil erosion in oman international journal of the analytic hierarchy process 116 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.683 saaty, t.l., & sagir, m. (2009). an essay on rank preservation and reversal. mathematical and computer modelling, 49(5–6), 1230–1243. doi: https://doi.org/10.1016/j.mcm.2008.08.001 u.s. environmental protection agency. (2012). stormwater best management practice: silt fences. retrieved from https://www3.epa.gov/npdes/pubs/siltfences.pdf wilson, j., & josiah, s. (2004). windbreak design. university of nebraska-lincoln extension, institute of agriculture and natural sciences. retrieved from https://nfs.unl.edu/documents/windbreakdesign.pdf waelti, c., & spuhler, d. (2019, june 13). planting pits. retrieved from https://sswm.info/sswm-solutions-bop-markets/affordable-wash-services-andproducts/affordable-technologies-and/planting-pits ijahp news and events: mu/virtual isahp2020 will take place on december 3-6, 2020 international journal of the analytic hierarchy process 365 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.807 virtual isahp2020 will take place on december 3-6, 2020 it is a pleasure to announce that the international symposium of the analytic hierarchy process will have its first virtual meeting, isahp2020, on december 3 – 6, 2020. the theme for this symposium is “ahp/anp: the next generation”, and the focus will be to socialize the new generation of ahp/anp researchers/practitioners within our community. completed research, research in progress and research proposals are all invited for the symposium. keep the following dates in mind: events important dates call for papers september 1 start of proposal submissions september 14 submission deadline october 23 submission results announcement november 6 isahp program committee conference chairman: enrique mu program co-chairs: antonella petrillo elena rokou head of scientific committee: birsen karpak conference manager: lirong wei creative decisions foundation (conference sponsor) president: rozann saaty details will be provided on the isahp2020 conference website and the call for papers announcement will be coming soon. https://www.isahp.org/ ijahp news and events: rokou/there is still time to submit papers to isahp 2018! international journal of the analytic hierarchy process 141 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.579 there is still time to submit papers to isahp 2018! you are still on time to submit papers to the international symposium of the analytic hierarchy process (isahp 2018) that will take place in hong kong july 12-15, 2018. the required isahp papers should be only 3-5 pages long. prof. luis vargas and prof. jennifer shang from the university of pittsburgh have agreed to co-chair this symposium. the themes of this symposium “ahp/anp in technology, entrepreneurship and corporate social responsibility“ focuses on three areas that have changed the world in recent years. however, rather than consider them as being independent, we need to think of them as interlocked because their future depends on each other. thus, it is befitting that the ahp/anp focuses on them now, for the anp helps us to study interdependent systems. we also expect a large participation of academics from the asian region in addition to our customary attendees. in particular, we have invited very interesting keynote speakers to join us in this meeting celebrating the life and legacy of dr. thomas l. saaty, creator of ahp/anp. professor yong shi from the key lab of big data mining and knowledge management, chinese academy of sciences, beijing, china will be one of our esteemed keynote speakers and he will be talking about professor saaty and his long influence on the chinese academic community through his ahp/anp and the theory of creative thinking. he will first review how ahp/anp has been widely used in china for various decision-making problems since the 1980's. then he will discuss the importance of professor saaty's landmarked book "creative thinking, problem solving and decision making" in the current "innovative and entrepreneurial movement" in china. professor gang hao from the city university of hong kong, a great supporter of the organization of the conference in hong kong will be another keynote speaker. the pre-conference day will be full of interesting workshops. in the morning we will have a super decisions seminar by dr. elena rokou and the attendees will receive a certificate to testify their knowledge of super decisions. in the afternoon, we will have four workshops that include “how to conduct a negotiation using ahp” by dr. luis vargas, “how to improve your chance of getting your ahp/anp paper published” by dr. enrique mu, “group decision making using anp the calculations” by dr. william adams and “the art of structuring ahp and anp models” by rozann whitaker saaty. we have organized a panel titled publishing your work: panel of journal editors where editors in international scientific journals, namely: rafikul islam, international journal of business and systems research, annals of management science; enrique mu, international journal of ahp ; valerio salomon, annals of management science; and luis vargas, journal of multi-criteria decision analysis will discuss the important factors that make an ahp/anp paper publishable and share their experiences with the audience. this is an opportunity for participants to ask questions pertinent to their own interests and concerns when publishing their work. http://www.isahp.org/schedule/?workshops-8 ijahp news and events: rokou/there is still time to submit papers to isahp 2018! international journal of the analytic hierarchy process 142 vol. 10 issue 1 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i1.579 furthermore, after the success at isahp2016 in london, we will again have mentoring sessions in hong kong. this will give our younger conference attendees an opportunit to exchange ideas with and get feedback from the top authors in the ahp/anp field and have one-on-one time with them. participants will be able to book time and discuss specific topics with highly acclaimed experts and members of our community. hong kong is not only a beautiful and vibrant destination, but it is also a cradle of business innovation. you still have time to join us in this memorable event. you can also participate even without presenting a paper and have the opportunity to meet colleagues and world experts in the ahp/anp field and attend workshops on state of the art ahp/anp techniques. ijahp article: budimčević, marcikić-horvat, mimović /measuring performance of middle east airlinesahp approach international journal of the analytic hierarchy process 391 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.595 measuring performance of middle east airlines – ahp approach kristina budimčević etihad airwais kristinab@etihad.ae aleksandra marcikić-horvat, phd assistant professor, university of novi sad faculty of economics subotica amarcikic@ef.uns.ac.rs predrag mimović full professor, university of kragujevac faculty of economics mimovicp@kg.ac.rs abstract over the past decade, the airline industry has undergone a number of changes thanks to accelerated technological progress, changes in consumer preferences and turbulent political events. bearing in mind the dynamic development of the airline industry, the concept of performance evaluation has been given increasing importance. therefore, there is a need to further explore the defined terms and relations in order to properly respond to the changes that have occurred. using the models of multi-criteria analysis, it is possible to perform a comprehensive assessment of an airline’s performance. accordingly, the study used the analytic hierarchy process (ahp) in order to evaluate the performance of three middle east airlines, emirates, etihad and qatar airways, with the goal of improving the service quality and customer satisfaction. for the purpose of conducting a comprehensive analysis, besides the ahp model, statistical non-parametric tests were used as well, in order to examine the preference of tangible and intangible components of service quality. key words: multi-criteria analysis and decision making; performance evaluation of the airlines; quality of the service; customer satisfaction; analytic hierarchy process 1. introduction in a modern business environment, it is crucial for airlines to constantly adjust and improve their performance in order to stay competitive and provide a desirable flight experience to their passengers. due to liberalization and increased global competition, meeting customer expectations is no longer sufficient to keep passengers loyal to an airline. an adequate system of evaluation and performance measurement should be in place in a dynamic business environment like aviation. thus, it is crucial for airlines to adopt an effective performance measurement system. during the past decade, a large number of multi-criteria methods and models have been developed in order to analyze an airline’s performance and provide a high level of service quality. in accordance with the above, the subject of the study is performance evaluation of the three middle east airlines emirates (abu dhabi, united arab emirates), etihad ijahp article: budimčević, marcikić-horvat, mimović /measuring performance of middle east airlinesahp approach international journal of the analytic hierarchy process 392 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.595 (abu dhabi, united arab emirates) and qatar airways (doha, qatar) using the ahp method, in order to improve the quality of their service and customer satisfaction. it is a multi-criteria decision-making problem with the goal of determining which components of service quality passengers consider most important when choosing an airline, as well as which of the three airlines has the highest level of service quality. besides that, the study aims to determine whether passengers consider tangible (physical) or intangible (human factor) dimensions of service quality more important. accordingly, a multi-criteria decision-making model was created in expert choice software in order to perform the evaluation of the model elements, criteria and alternatives. considering the structure of the study, the introduction summarizes a multi-criteria decision-making problem in relation to the performance evaluation of the airlines, followed by a brief overview of the literature and description of the methodology used. in the study, the emphasis is on implementing the ahp method, a well-known and validated approach for solving complex, multi-criteria decisionmaking problems. besides the ahp model, statistical non-parametric tests were also used, in order to evaluate the preference between tangible and intangible components of service quality. it is crucial to structure the decision-making problem in the correct way and to select adequate criteria, in order to be able to choose the optimal alternative. hierarchical decomposition of the defined problem greatly enhances transparency and contributes to more efficient problem solving, thereby increasing the consistency of decision-making and reducing the errors. 2. literature review performance evaluation and measurement has recently become a significant research interest since it is crucial in a dynamic business environment that each company has an effective performance management system. this requires a measurement of different indicators, i.e. performance criteria that are in accordance with the defined strategy of the organization. armstrong and murlis (2007) emphasized that it is a natural and basic management process that should be led by the management and form an integral part of their daily work practice. in this context, it is important to mention the study of tangen (2005), who divided all the methods for evaluation of organizational performance into three categories depending on the system requirements for performance measurement, as well as on the applied approach i.e. what should be measured and how it should be measured. an effective performance measurement system allows a company to measure its performance in accordance with the defined strategy. numerous authors used the analytic hierarchy process (ahp) in their research, and a good overview of ahp applications is given by vaidya and kumar (2006), sipahi and timor (2010), ishizaka and labib (2011) and subramanian and ramanathan (2012). however, none of them used ahp for the performance evaluation of middle east airlines. when it comes to the quality of service in the airline industry, many authors have discussed this topic, especially since the relationship between customer satisfaction and airline profitability has been established (namukasa, 2013; kamakura et al, 2002; bukhari, ghoneim, dennis & jamjoom, 2013, etc). as the airline industry belongs to the service sector where most of the airlines offer competitive prices and loyalty programs, the main form of differentiation is superior service quality. park, robertson & wu (2004) emphasized that the service quality is a key factor for an airline’s survival and sustainability. on the other hand, amongst the studies related to airline service quality and passenger satisfaction, the following authors can be distinguished: campbell & vigar-ellis (2012), massarat & jha (2014), yayla-kullu & tansitpong (2013), de jager & van zyl (2013). when it comes to performance evaluation of the middle east airlines, ijahp article: budimčević, marcikić-horvat, mimović /measuring performance of middle east airlinesahp approach international journal of the analytic hierarchy process 393 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.595 research carried out by massarat & jha (2014) and surovitskikh (2007) can be distinguished. both authors applied the servqual model in their studies in order to assess the service quality of middle east airlines. the passengers were asked to evaluate the five dimensions of the model in order to determine what is important when choosing a carrier. when it comes to the evaluation of service quality of three middle east airlines, so far no similar research has been done using the same methodology; hence, this study represents a theoretical and practical contribution within the given area of research. 3. methodology the analytic hierarchy process (ahp) is a theory of measurement through pairwise comparisons that relies on the judgements of experts to derive priority scales (table 1, saaty & kearns, 1985). it is an intuitive method for formulating and analyzing decisions which can be successfully used to measure the relative impact of multiple factors on possible outcomes, as well as to forecast their relative probabilities. the structures of the ahp must include all the factors used to determine the best outcome; thus, the outcome is a result of the factors included and the judgments used. the analytic hierarchy process breaks down the decision-making problem into its key elements according to their common characteristics, in the form of a hierarchical model. the problem is usually formulated as a three level hierarchy with the explicitly defined goal at the highest level, criteria on the next lower level and alternatives on the lowest level. hierarchical structuring is an effective process that helps each decision-maker cope with the complexity of the problem. once a hierarchical structure is formed, pair-wise comparisons are conducted in order to determine the relative strength of the elements in the hierarchy. at this stage, the decision-maker’s preferences play a key role in the problem-solving process. the comparisons are made using a scale of absolute judgements that represents how much more one element dominates another with respect to a given attribute (saaty, 2008). when comparing a pair of criteria, the question to be asked is what is more important or what has a greater impact, based on which their relative significance is determined. however, when comparing two alternatives with respect to a particular criterion, it should be determined what is more preferable. the overall preference of an alternative is calculated as a weighted sum of the criteria and alternative results for that particular criterion. ijahp article: budimčević, marcikić-horvat, mimović /measuring performance of middle east airlinesahp approach international journal of the analytic hierarchy process 394 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.595 table 1 scale of comparison 1–9 intensity of importance definition explanation 1 equal importance two activities contribute equally to the objective 3 moderate importance experience and judgment slightly favor one activity over another 5 strong importance experience and judgment strongly favor one activity over another 7 very strong or demonstrated importance an activity is favored very strongly over another; its dominance demonstrated in practice 9 extreme importance the evidence favoring one activity over another is of the highest possible order of affirmation 2, 4, 6, 8 mean value of two estimates a compromise is needed reciprocals of the above if activity i has one of the above nonzero numbers assigned to it when compared with activity j, then j has the reciprocal value when compared with i source: saaty, t. l. & kearns p. k. (1985) once the estimations are done for each part of the model, the information is synthesized in order to show the general overall preference. the synthesis provides a complete ranking of the alternatives with respect to the main goal, where the rank shows how each alternative is evaluated against each criterion. however, the judgements may be inconsistent, thus it might be required to change the judgements in order to obtain better consistency. an important indicator of consistency is the consistency index, where the value should be below 10% in order to be acceptable, otherwise decision-makers should reconsider their estimates and seek additional information. 4. structure of the research problem previously conducted studies considered various dimensions of service quality whilst evaluating airline performance; therefore, they have been more or less complex and comprehensive compared to the conducted study. the authors, bowen, headley & luedtke (1992), have considered nineteen different factors of service quality when evaluating airlines. these factors include fleet size and age, fares, baggage handling, passenger complaints, staff kindness, etc. on the other hand, tsaur, chang and yen (2002) identified the following criteria as the most important ones when it came to taiwanese airlines: staff courtesy and responsiveness, safety, cabin cleanliness, comfort, in-flight entertainment and good flight connections. generally, there are different dimensions of the airline service quality that can be grouped into two categories: tangible, i.e. the physical quality that can be measured (quality of meals, seat comfort, cabin cleanliness, ife) and intangible (staff kindness, physical appearance, language proficiency, handling passenger complaints) and their importance largely depends on the subjective assessment of the service users. the ijahp article: budimčević, marcikić-horvat, mimović /measuring performance of middle east airlinesahp approach international journal of the analytic hierarchy process 395 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.595 distinction between these two groups is often not clear, so some of the above dimensions, such as language proficiency, can belong to both categories. however, for the purpose of this analysis, we will consider it as an intangible dimension. in order to perform the analysis, evaluation data was collected from passengers and experts in the aviation industry. for the purpose of the analysis, two types of questionnaires were used with precisely defined questions related to the service quality and customer satisfaction. the sample included 200 respondents, i.e. the passengers of the three airlines, emirates, etihad and qatar airways. the distribution of the questionnaires was done electronically and in person. in the first questionnaire, a likert scale from 1 to 5 was used and the passengers had the opportunity to evaluate the importance (preference) of tangible and intangible dimensions of service quality (see appendix). in the second questionnaire, saaty's scale from 1 to 9 was used in order to rate the importance, i.e. preference of 8 selected service quality components, as well as to evaluate the service quality of each of the three airlines. 4.1 survey 1 – evaluation of preferences by likert scale in the first survey, passengers were given the opportunity to express whether they considered tangible or intangible components of service quality to be more important. they expressed their preference by using a likert scale from 1 to 5, with 1 being the lowest and 5 the highest preference (see survey i in appendix). in order to carry out the mentioned analysis, two non-parametric tests were used, chi-square test and kolmogorov-smirnov test. table 2 tangible and non-tangible criteria tangible criteria quality of meals seat comfort cabin cleanliness ife (movies, etc) nontangible criteria kindness of staff physical appearance language proficiency handling passenger’s complaints тable 3 evaluating the importance of tangible criteria value observed n expected n residual 3,00 14 66.3 -52.3 4,00 66 66.3 -.3 5,00 119 66.3 52.7 total 199 source: spss sоftware ijahp article: budimčević, marcikić-horvat, mimović /measuring performance of middle east airlinesahp approach international journal of the analytic hierarchy process 396 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.595 тable 4 evaluating the importance of non-tangible criteria value observed n expected n residual 3,00 14 66.7 -52.7 4,00 65 66.7 -1.7 5,00 121 66.7 54.3 total 200 source: spss sоftware таble 5 chi-square test for evaluating the importance of tangible & intangible components of service quality тangible non-tangible chi-square 83.106 a 85.930 b df 2 2 asymp. sig. .000 .000 source: spss sоftware таble 6 kolmogorov-smirnov test for evaluating the importance of tangible & intangible components of service quality n 199 200 normal parameters a,b mean 4.5276 4.5350 std. deviation .62603 .62508 most extreme differences absolute .373 .377 positive .225 .228 negative -.373 -.377 kolmogorov-smirnov z 5.258 5.325 asymp. sig. (2-tailed) .000 .000 source: spss software based on the test results, we concluded that there is a statistically significant difference in the importance of tangible and intangible criteria if the risk of error is below 1%. although the difference is almost negligible, the results of both tests (tables 3 & 4) show that passengers find the intangible criteria (human factor) more important in relation to the tangible (physical component). this conclusion is based on the obtained chi-square test value which is 83,106 for tangible criteria and 85,930 for intangible (table 5). by observing the results of the kolmogorov-smirnov test, a similar conclusion can be reached, as the test value of the tangible criteria was 5.258 and of the intangible 5.325 (table 6). even though the difference is extremely small, it can be concluded that passengers of the me airlines slightly favor the intangible dimension of the service quality over the tangible one. this indicates the growing importance of the human factor (the intangible component), as it can significantly ijahp article: budimčević, marcikić-horvat, mimović /measuring performance of middle east airlinesahp approach international journal of the analytic hierarchy process 397 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.595 affect the perception of the service quality, while the physical component can be easily copied thanks to modern technological solutions. 4.2 survey 2 – prioritization and ranking of airlines: ahp model as stated above, the empirical part of the research is focused on the performance evaluation of three middle east airlines emirates, etihad and qatar airways, in order to improve the quality of their service and customer satisfaction. the study seeks to determine which components of the service quality passengers consider most important when choosing an airline, as well as which of the three airlines has the highest level of service quality and customer satisfaction. with reference to the previously conducted studies and available literature, the most common tangible and intangible components of the airline service quality were identified. accordingly, the experts from the aviation industry selected the eight most important components, which were then evaluated by the passengers of the three mentioned airlines using saaty's scale from 1 to 9. following the pairwise comparisons of the criteria, the alternatives were pairwise compared with respect to each criterion. the illustration below shows the hierarchical structure of the defined multi-criteria decision-making problem in the ahp software (figure 1). goal me airlines evaluation & ranking criteria quality of the meals seat comfort cabin cleanli -ness ife (movies etc) kindness of staff physical appearance language proficiency handling passenger’s complaints alternatives emirates etihad qatar figure 1 ahp model of airlines evaluation and ranking bearing in mind the defined hierarchical structure, the main goal is to evaluate and rank the three middle east airlines. the selected criteria, four tangible (quality of meals, seat comfort, cabin cleanliness, in-flight entertainment) and four intangible (staff kindness, physical appearance, language skills and handling complaints) are on the level below. at the lowest level are the alternatives, etihad, emirates and qatar airways which are evaluated against the elements on the higher hierarchical levels, i.e. criteria and goal. the main condition for the implementation of the ahp model in this study was that all the interviewed respondents (passengers and staff) had used the services of the three middle east airlines at least once. 5. ahp model results after defining the elements of a decision-making problem, the pairwise comparisons followed (see survey ii in appendix). firstly, all the criteria were compared in pairs with respect to the main goal in order to determine their relative importance, i.e. contribution to the main goal. the same process was repeated, for 200 respondents, after which the geometric mean of the obtained criteria weights of all individual evaluations was calculated. these were ultimately calculated for 120 respondents because the others were eliminated due to inconsistent responses and misunderstanding of the ahp methodology. when there are multiple decision makers, in order to find the final weights, i.e. priorities of the criteria and the ranks of ijahp article: budimčević, marcikić-horvat, mimović /measuring performance of middle east airlinesahp approach international journal of the analytic hierarchy process 398 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.595 the alternatives, the following formula is used to calculate the geometric mean (saaty & peniwati, 2008). k ik kk ki ww    1   i (1) where wi represents the final weight of the i criterion, аnd ikw the relative weight of the i element calculated using k evaluator. in accordance with the above, the geometric mean of all 120 criteria evaluations from all the respondents, is the following: 𝑤𝑖 = √𝜋𝑘=1 𝑘=120𝑤𝑖𝑘 120  i (2) where for criterion k1 the final weight of the estimates of all 120 respondents is equal to: w𝐾1 = √𝑤𝐾11 × 𝑤𝐾12,…,× 𝑤𝐾1120 120 = 0,04989 (3) following the same logic, the priorities of the criteria were calculated using the geometric mean of the weighted coefficients of all individual ahp evaluations within the model (table 7). таble 7 obtained criteria weights within ahp method criteria priority airlines ranking 0.00000 1. quality of meals 0.04989 2. seat comfort 0.30786 3. cabin cleanliness 0.12102 4. ife 0.02654 5. staff kindness 0.26115 6. physical appearance 0.03609 7. language proficiency 0.06476 8. handling complaints 0.13270 consistency index: 0.09 (9%) ijahp article: budimčević, marcikić-horvat, mimović /measuring performance of middle east airlinesahp approach international journal of the analytic hierarchy process 399 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.595 figure 2 obtained criteria weights (priorities) by observing the obtained criteria weights, it can be noticed that the greatest importance for the passengers is seat comfort (0.307), followed by staff kindness (0.261), with the lowest priority being in-flight entertainment (0.026). this indicates that passengers rated this criterion as the least important when choosing an airline. for intangible criteria, the lowest ranked criterion is physical appearance of the staff (0.036). the advantage of the ahp method is the ability to measure the error in reasoning by calculating a consistency index for the resulting matrix of comparisons. in the table above, the consistency index (ci) was approximately 0.09 which is within the acceptable limits as it is lower than 0.10 (10%). if ci was greater than 10%, the reasons behind the inconsistency should be determined and the pairwise comparisons should be repeated. if the repetition of the procedure does not lead to an acceptable limit of 0.10, all the results should be discarded and the entire process should be repeated from the beginning. in our model, the estimations were repeated several times whenever the inconsistency was greater than 0.10, and the respondents were willing to make it consistent. subsequently, the overall consistency index, calculated as the geometric mean of all individual consistency indices, was approximately 0.09. following the comparisons of the criteria, the next step was to pairwise compare all the alternatives (airlines) against each criterion, in order to calculate their local priorities and the final ranks by finding the geometric mean of all respondent’s evaluations. when two alternatives are compared with respect to a particular criterion, the general preference of an alternative will be calculated as the weighted sum of criteria weight and the alternative result for that respective criterion (saaty & kearns, 1985). firstly, the pairs of alternatives are compared with respect to each criterion to obtain their local priorities, while their global priorities are obtained by the synthesis of the results in relation to all criteria simultaneously. the final results indicate the ranking of the alternative on each criteria as shown in table 8. 4% 31% 14% 2% 27% 3% 6% 14% аhp criteria weights 1. quality of meals 2. seat comfort 3. cabin cleanliness 4. ife 5. staff kindness 6. physical appearance ijahp article: budimčević, marcikić-horvat, mimović /measuring performance of middle east airlinesahp approach international journal of the analytic hierarchy process 400 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.595 таble 8 ahp local priorities of alternatives with respect to each criterion criteria/alternatives emirates etihad qatar 1. quality of meals 0.287 0.600 0.111 2. seat comfort 0.222 0.624 0.131 3. cabin cleanliness 0.426 0.310 0.182 4. ife 0.234 0.650 0.112 5. staff kindness 0.188 0.595 0.183 6. physical appearance 0.215 0.523 0.199 7.language proficiency 0.293 0.526 0.160 8. handling complaints 0.157 0.471 0.337 the results (table 8) show that etihad airways has the most favorable results except on the cleanliness of the cabin criterion. it can also be noted that qatar airways is ranked lowest for all criteria, except for handling passenger’s complaints. таble 9 final priorities of the alternatives within ahp model name ideals normals raw ranking emirates 0.437062 0.239803 0.119901 2 etihad 1.000000 0.548670 0.274335 1 qatar 0.385526 0.211527 0.105763 3 table 9 shows the final priorities of the alternatives calculated as the geometric mean of the final priorities of all individual assessments within the ahp model. based on the obtained results, we concluded that etihad airways is the optimal choice for the passengers with a synthesized global priority of 0.548. emirates airlines is the second best alternative with a global priority of 0.239, while qatar airways is the lowest ranked alternative with a global priority of 0.211. figure 3 final (global) priorities of the alternatives the final outcome is not surprising given the fact that etihad airways had better ratings in seven out of eight selected criteria in comparison with the other two airlines. finally, a sensitivity analysis was carried out in order to identify the impact of the change in criteria weights on the alternatives ranking. emirates 24% etihad 55% qatar 21% ahp final priorities of the alternatives emirates etihad qatar ijahp article: budimčević, marcikić-horvat, mimović /measuring performance of middle east airlinesahp approach international journal of the analytic hierarchy process 401 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.595 table 10 the sensitivity of alternative priorities, as a consequence of changing the relative importance of the criteria cabin cleanliness (calculated using superdecision software) cabin cleanliness emirates etihad qatar 0.000 0.184 0.584 0.232 0.071 0.192 0.579 0.229 0.143 0.201 0.573 0.226 0.214 0.209 0.568 0.223 0.286 0.218 0.562 0.219 0.357 0.227 0.557 0.216 0.429 0.235 0.551 0.213 0.500 0.244 0.546 0.209 0.571 0.253 0.540 0.207 0.643 0.261 0.535 0.204 0.714 0.269 0.529 0.200 0.786 0.279 0.524 0.197 0.857 0.287 0.519 0.194 0.929 0.296 0.513 0.190 1.000 0.304 0.508 0.188 table 10 shows the priorities of the three airlines as cabin cleanliness relative weight, i.e. importance, is increased. the results show that the increase in importance of cabin cleanliness from 0.000 to 1.000 does not significantly affect the final ranking of the alternatives. however, it will lead to a somewhat lower priority for etihad airways and higher priority for emirates. the priority of emirates increased from 0.184 to 0.304, while the priority of etihad and qatar airways decreased from 0.584 to 0.508 and from 0.232 to 0.188 respectively. in any case, etihad will still represent the optimal choice for travelers since the sensitivity analysis clearly showed that the system is robust and that small or even bigger changes in the criteria weights will not significantly affect the overall ranking in terms of the best alternative. this confirms that a certain inconsistency in the pairwise comparison that caused a slight difference in criteria weights will not affect the final ranking of the alternatives. accordingly, it can be concluded that etihad airways is the optimal choice for the travelers and it can be chosen with full confidence. similar results are obtained with respect to other criteria as well, except for seat comfort. it was observed that when this criteria weight increases from 0.001 to 0.999, the final priority of emirates also increases from 0.17 to 0.48, while the priorities of etihad and qatar airways decrease from 0.61 to 0.35 and 0.22 to 0.17 respectively. the change in criteria weight will result in rank inversion, so emirates will become the highest ranked alternative if the seat comfort weight increases from 0.5 to 0.667 (figure 4). ijahp article: budimčević, marcikić-horvat, mimović /measuring performance of middle east airlinesahp approach international journal of the analytic hierarchy process 402 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.595 figure 4 dynamic sensitivity diagram after changing the weight of the "seat comfort" criteria (superdecision software) 6. conclusion in accordance with the conceptual framework of the research, a further theoretical and practical contribution to the development of this research area is expected. it can be concluded that the conducted research has provided a theoretical and practical contribution to the further development of this area of research. the results can serve as important guidelines for the management of the middle east airlines, as well as for the other airlines around the globe. the results of the study showed that the ahp method of multi-criteria analysis gives a comprehensive performance assessment of the airlines that will contribute to greater service quality and customer satisfaction. in order to carry out a detailed analysis, it is important to take into account all the components of the service quality that are important for the passengers when choosing an airline. thanks to the implementation of the ahp method, the most important criteria were identified and evaluated which can give information to the management of the airlines about their service quality and how the passengers perceive it. the results showed that the passengers of the three airlines found seat comfort and staff kindness as the most important factors, while physical appearance (grooming) and in-flight entertainment are the least important. on the other hand, when it comes to the ranking of the alternatives, etihad airways is their preferred choice bearing in mind the selected criteria. emirates airlines is the second best choice, while qatar airways is the lowest ranked option, though both emirates airways and qatar airways are far less preferred than etihad airways. the sensitivity analysis confirmed that the system is robust and that small or even bigger changes in the relative weights of the criteria will not significantly impact the overall ranking when it comes to choosing the best alternative. this is useful information for travelers, as well as for future researchers within the airline industry. the conducted analysis shows the growing importance of multi-criteria methods in solving complex decision-making problems. on the other hand, when it comes to the statistical analysis that was used for evaluating the importance of tangible and intangible components of the service quality, the results have pointed out the growing importance of the intangible component, i.e. human factor. however, as the obtained difference between tangible and intangible criteria is negligible, it can be concluded that the service quality is a multi-dimensional concept where all of its elements, as a whole, have a synergetic impact on the passenger’s satisfaction. it is very important 0 0.2 0.4 0.6 0.8 1 1.2 1 2 3 4 5 6 7 seat comfort 1 emirates 2 etihad 3 qatar ijahp article: budimčević, marcikić-horvat, mimović /measuring performance of middle east airlinesahp approach international journal of the analytic hierarchy process 403 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.595 that the airline management is aware of this fact, so that its focus is on all the aspects of the service quality. based on the results of the conducted research, a positive impact on the further development of the economic theory and practice can be expected. the theoretical contribution is the enrichment of the literature, along with a precise definition of the factors in the research. on the other hand, the practical contribution is that passengers have more information to inform their decision making about airlines as well as helping managers design an effective business strategy that will enhance their airline’s competitiveness and sustainability. the limitations and shortcomings of the conducted research are mainly related to data collection, sample structuring and the number of service components that were taken into consideration. data was collected primarily from passengers in the abu dhabi airport, as well as from the employees of the three airlines. however, since the emirates and etihad hubs are located within a relatively small geographical distance in the uae, the majority of the respondents were their passengers, while qatar airways passengers represented a smaller portion of the sample. this limitation has perhaps been reflected in the final results; therefore, it is important to highlight it. another disadvantage is that only eight components of service quality were taken into account. bearing that in mind, the empirical research would be much more comprehensive with a larger number of service components which has useful implications for future researchers. none of the previous studies used the ahp method in the evaluation of the service quality of these three me airlines. the problem could be further analyzed, in turn delivering more detailed and comprehensive results if sub-criteria were identified and new criteria added, and if direct and reverse interactions were considered within and among the groups of different sub-criteria. this would render the problem even more complex, while on the other hand it would show the actual situation in a more realistic manner. such problems could be resolved by applying the analytical network process, as an extension of the ahp model, that includes complex networks of influence between the model elements regardless of the level at which they are located. ijahp article: budimčević, marcikić-horvat, mimović /measuring performance of middle east airlinesahp approach international journal of the analytic hierarchy process 404 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.595 references armstrong, m., murlis, h. (2007). reward management: a handbook of remuneration strategy and practice. london: kogan page. bowen, b., headley, d., luedtke, j. (1992). a quantitative methodology for measuring airline quality. journal of aviation/aerospace education & research, 2(2), 27-33. doi: https://doi.org/10.15394/jaaer.1992.1063 budimčević, k., mimović, p. (2013). application of ahp method in the selection process of the optimum destination of etihad airways. economic themes, 51(3), 499–514. bukhari, s.m.f, ghoneim, a., dennis, c. & jamjoom, b. (2013). the antecedents of travellers’ e-satisfaction and intention to buy airline tickets online. journal of enterprise information management, 26(6), 624-641. doi: https://doi.org/10.1108/jeim-07-2013-0040 campbell, b. & vigar-ellis, d. (2012). the importance of choice attributes and the positions of the airlines within the south african domestic passenger airline industry as perceived by passengers at durban international airport. southern african business review, 16(2), 97-115. cronin, j.j., taylor, s.a., (1992). measuring service quality: a reexamination and extension. journal of marketing, 56(3), 55-68. doi: https://doi.org/10.2307/1252296 de jager, j.w., van zyl, d. (2013). airline service quality in south africa and malaysia – an international customer expectations approach. journal of economics and behavioral studies, 5(11), 752-761. ishizaka, a., labib, a. (2011). review of the main developments in the nalytic hierarchy process. expert systems with applications, 38(11), 14336–14345. doi: https://doi.org/10.1016/j.eswa.2011.04.143 kamakura, w.a., mittal, v., de rosa, f., mazzon, j.a., (2002). assessing the service-profit chain. marketing science, 21 (3), 294-317. doi: https://doi.org/10.1287/mksc.21.3.294.140 massarat, g., jha, s. (2014). assessing customer perception of service quality: comparative study of airlines in uae. world review of business research, 4(2), 291-303. namukasa, j. (2013). the influence of airline service quality on passenger satisfaction and loyalty: the case of uganda airline industry. the tqm journal, 25 (5), 520-532. doi: https://doi.org/10.1108/tqm-11-2012-0092 park, j., w., robertson, r., wu, c., l. (2004). the effect of airline service quality on passengers’ behavioral intentions: a korean case study. journal of air transport management, 10, 435-439. doi: https://doi.org/10.1016/j.jairtraman.2004.06.001 javascript:__dolinkpostback('','mdb~~bsh%7c%7cjdb~~bshjnh%7c%7css~~jn%20%22economic%20themes%22%7c%7csl~~jh',''); https://doi.org/10.2307/1252296 https://doi.org/10.1016/j.eswa.2011.04.143 https://doi.org/10.1287/mksc.21.3.294.140 https://doi.org/10.1108/tqm-11-2012-0092 https://doi.org/10.1016/j.jairtraman.2004.06.001 ijahp article: budimčević, marcikić-horvat, mimović /measuring performance of middle east airlinesahp approach international journal of the analytic hierarchy process 405 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.595 saaty, t.l. (2008). decision making with the analytic hierarchy process. international journal of services sciences, 1(1), 83–98, doi: https://doi.org/10.1504/ijssci.2008.01759 saaty, t.l., kearns, k. p. (1985). analytical planning: the organization of systems, the analytic hierarchy process series, vol. iv. pittsburgh, pa: rws publications. saaty, t.l., peniwati, k. (2008). group decision making: drawing out and reconciling differences. pittsburgh, pa: rws publications. sipahi, s., timor, m. (2010). the analytic hierarchy process and analytic network process: an overview of applications. management decision, 48(5). doi: 775-808, https://doi.org/10.1108/00251741011043920 subramanian, n., ramanathan, r. (2012). a review of applications of analytic hierarchy process in operations management. international journal of production economics, 138, 215-241. doi: https://doi.org/10.1016/j.ijpe.2012.03.036 surovitskikh, s., lubbe. b. (2007). positioning of selected middle eastern airlines in the south african business and leisure travel environment. journal of air transport management, 14(2), 75-81. doi: https://doi.org/10.1016/j.jairtraman.2007.12.001 tangen, s. (2005). analyzing the requirements of performance measurement systems. measuring business excellence, 9(4), 46-54. doi: https://doi.org/10.1108/13683040510634835 tsaur, s., chang, t., yen, c. (2002). the evaluation of airline service quality by fuzzy mcdm. tourism management, 23(2), 107-115. doi: https://doi.org/10.1016/s0261-5177(01)00050-4 yayla-kullu, h.m., tansitpong, p. (2013). a critical evaluation of u.s. airlines’ service quality performance: lower costs vs. satisfied customers. journal of management and strategy, 4(4), 1-11. doi: http://dx.doi.org/10.5430/jms.v4n4p1. vaidya, o.s., kumar, s. (2006). analytic hierarchy process: an overview of applications. european journal of operational research, 169 (1), 1-29. doi: https://doi.org/10.1016/j.ejor.2004.04.028 https://doi.org/10.1504/ijssci.2008.01759 https://doi.org/10.1108/00251741011043920 https://doi.org/10.1016/j.ijpe.2012.03.036 https://doi.org/10.1108/13683040510634835 https://doi.org/10.1016/s0261-5177(01)00050-4 http://dx.doi.org/10.5430/jms.v4n4p1 https://doi.org/10.1016/j.ejor.2004.04.028 ijahp article: budimčević, marcikić-horvat, mimović /measuring performance of middle east airlinesahp approach international journal of the analytic hierarchy process 406 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.595 appendix survey i a) please evaluate 3 me airlines based on the 4 tangible & 4 non-tangible criteria below: for that purpose use the following likert scale with the range 1-5, with the following meanings: 1 = low quality/satisfaction 5 = high quality/satisfaction tangible criteria/ alternative quality of meals seat comfort cabin cleanliness ife (movies, etc) emirates etihad qatar non-tangible criteria/ alternative kindness of staff physical appearance language proficiency handling passengers’ complaints emirates etihad qatar 1 please evaluate your overall satisfaction with the service quality of each me airline: emirates etihad qatar 1 = completely unsatisfied 5 = completely satisfied 2 please evaluate the importance of tangible & non-tangible criteria using the likert scale: tangible criteria non-tangible criteria 1 = not very important 5 = very important ijahp article: budimčević, marcikić-horvat, mimović /measuring performance of middle east airlinesahp approach international journal of the analytic hierarchy process 407 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.595 survey ii a) assessing criteria importance according to the main goal please assess the importance of the following criteria when evaluating the airline: which of the 2 criteria below is more important to you, and by how much, having in mind the main goal of ahp model: evaluation and ranking of 3 me airlines? *for that purpose use the following saaty scale with the range 1-9, with the following meanings: thomas saaty scale for comparison 1-9 intensity of importance definition 1 equal importance 3 somewhat more important 5 much more important 7 very much more important 9 absolutely more important 2, 4, 6, 8 intermediate values criteria evaluation matrix quality of meals seat comfort cabin cleanly -ness ife (movies, etc) kindness of staff physical appear ance language proficiency handling passenger’s complaints quality of meals 1 seat comfort 1 cabin cleanli ness 1 ife (movies,et c) 1 kindness of staff 1 physical appearance 1 language proficiency 1 handling passenger’ s complaints 1 ijahp article: budimčević, marcikić-horvat, mimović /measuring performance of middle east airlinesahp approach international journal of the analytic hierarchy process 408 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.595 b) evaluating alternatives (airlines) preference according to 8 criteria below when comparing 2 airlines below, which one you prefer, having in mind the chosen criteria below?(for every criteria individual) alternative emirates etihad qatar emirates 1 etihad 1 qatar 1 *for evaluation purpose, please use the same saaty scoring scale with the range 1-9. ijahp article: goepel/ implementation of an online software tool for the analytic hierarchy process (ahp-os) international journal of the analytic hierarchy process 469 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.590 implementation of an online software tool for the analytic hierarchy process (ahp-os) klaus goepel bpmsg (business performance management singapore) drklaus@bpmsg.com abstract the analytic hierarchy process (ahp) remains a popular multi-criteria decision method. the author has implemented a free, web-based ahp online system with noteworthy features, allowing for the detailed analysis of decision problems. besides standard functions, like flexible decision hierarchies, support to improve inconsistent judgments, and alternative evaluation and sensitivity analysis, the software can handle group input, calculate group consensus based on shannon α and β-entropy and estimate weight uncertainties based on randomized small variations of input judgments. in addition, different ahp judgment scales can be applied a posteriori and alternative evaluation can be done using the weighted sum (wsm) or weighted product model (wpm). this flexibility opens up opportunities to study decision projects under various parameters. the author’s intention was to provide a complete and free software tool for educational and research purposes where calculations and algorithms are well documented and all input data and results can be exported in an open format for further processing or presentation. the article describes the basic concept and structure of the software and the underlying mathematical algorithms and methods. challenges and practical experiences during the implementation, validation and productive phase of the software are highlighted. keywords: multi-criteria decision-making; analytic hierarchy process; ahp software; ahp online system; ahp group decision making 1. introduction the analytic hierarchy process (ahp) was developed by t. l. saaty in 1980 and remains a widespread multi-criteria decision method (mcdm) (saaty, 1980). based on pairwise comparison inputs, weights are calculated by finding the dominant right eigenvector (ev) of a positive reciprocal decision matrix. calculations for simple decision problems can be done with a spreadsheet program, but for more complex decision problems a software tool is needed (goepel, 2013). there are several ahp software packages available on the market (see annex), and some of them have been described or compared in the literature (ossadnik & kaspar 2013; ishizaka & labib 2009; siraj et al. 2015). most of these packages target companies and pursue a business objective. their focus lies on the application, and less on the methods and algorithms, which are often not fully transparent to users. the author’s intention was to provide a complete and free software tool for educational and research purposes where ijahp article: goepel/ implementation of an online software tool for the analytic hierarchy process (ahp-os) international journal of the analytic hierarchy process 470 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.590 methods and algorithms are well documented and validated. as a result, a web-based ahp online system (ahp-os) was developed and is available in its full functionality to any user for non-commercial purposes. the software is implemented using php, an open source general-purpose, object oriented scripting language that is especially suited to web development. database function calls use structured query language (sql), the most widely used database language with available open source database engines like sqlite or mariadb. the whole package is developed from scratch, using only a few other available open source packages for general supporting functions, like sending e-mails or user login and registration (annex 1). functions and features of ahp-os include the following:  flexible definition of decision hierarchies as text input, following a simple syntax with multi-language support using unicode character coding,  weight calculation (hierarchy mode) and alternative evaluation (alternative mode) using the ahp eigenvector method,  pairwise comparison input, highlighting the top-3 most inconsistent judgments,  partial judgments,  a posteriori application of different ahp judgment scales,  group decision making using weighted geometric mean aggregation of individual judgments (wgm-aij),  group consensus calculation based on shannon α and β-entropy,  weight uncertainty estimation using monte carlo simulation,  sensitivity analysis,  weighted sum model (wsm) and weighted product model (wpm) for the aggregation of alternatives,  export of input and result data as comma separated value (csv) files for further processing or presentation in a spreadsheet program. the software has a modular structure and is implemented using object oriented programming (oop). it consists of five major objects (classes): 1. ahp class for all basic ahp calculations, 2. ahphierarchy class for decision hierarchy related functions, 3. ahpdb class for database handling, 4. ahpgroup class for result calculations and 5. ahpadmin class for all administrative functions like user and project administration. these classes contain all ahp methods. in the following, we will describe the major functions in more detail. 2. decision hierarchies and eigenvector calculations 2.1 hierarchies decision hierarchies in the program are defined using plain text input consisting of a sequence of statements following a simple syntax. this provides different advantages: ijahp article: goepel/ implementation of an online software tool for the analytic hierarchy process (ahp-os) international journal of the analytic hierarchy process 471 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.590  hierarchy definitions in plain text form can be modified or duplicated by simple copy and paste functions.  they can easily be archived as text files.  unicode character coding allows for multi-language support; languages like chinese, korean, thai etc. are supported.  local weights can be predefined within the hierarchy text. this is useful when weights are already known, e.g. from a previous project or when they are derived from other mcdm methods, and ahp-os is only used for alternative evaluation. the syntax is defined as follows: <hierarchy> → <branch>; [{<branch>;}] <branch> → <node>: <leafs>, <leafs> [,<leafs>] <leafs> → {<leaf> [ = <weight>]} for all <leafs> in a <branch> the sum of <weight> has to be one. a <node> of the second and any further <branch> has to be one of the <leaf> in <leafs>. each <node> represents a decision matrix, and the corresponding priority vector’s dimension is the number of <leafs>. figure 1 shows a simple example of a two level hierarchy. in the first level (main criteria), weights for the two criteria are predefined as 30% and 70%. for the sub-criteria, weights are automatically set to the default value 1/nleaf. figure 1 example of a decision hierarchy with 2 levels, 3 nodes and 4 leafs this hierarchy in figure 1 is defined by the following hierarchy text: ahp-project: criterion-1=0.3, criterion-2=0.7; criterion-1: sub-criterion a, sub-criterion b; criterion-2: sub-criterion c, sub-criterion d; all methods related to the decision hierarchy are coded in the ahphierarchy class of the software. the class includes a parser to translate the hierarchy text into a multidimensional array. the parser checks for syntax errors and cleans the input text from redundant and impermissible characters. supporting functions to extract nodes, branches or leafs from the hierarchy are included in this class. ijahp article: goepel/ implementation of an online software tool for the analytic hierarchy process (ahp-os) international journal of the analytic hierarchy process 472 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.590 2.2 ahp eigenvector method once a hierarchy is defined and stored in the database, weights can be calculated through the following steps. each step is coded as a method in the ahp class of ahp-os: 1. get pairwise comparisons (pwc): function get_pair_comp() 2. apply selected ahp judgment scale: function calcscale() 3. fill decision matrix from pairwise comparisons: function getmatrixfrompwc() 4. find the eigenvector using the power method: function set_evm_evec() 5. calculate the dominant eigenvalue from the eigenvector: function setevmeval() 6. calculate the consistency ratio cr: function setalonsocr() 7. calculate the inconsistency matrix: function get_incon_matrix() 8. identify and highlight the top three inconsistencies: function get_inconsistency() 9. go back to step 1 until the user submits his judgments. figure 2 shows an example of the display of pairwise comparisons to the user. figure 2 pairwise comparison input all pairwise comparisons are internally stored in the format 𝑝𝑤𝑐 = (𝑎1,𝑎2, … 𝑎𝑛𝑝𝑐 ), (𝑥1,𝑥2, … 𝑥𝑛𝑝𝑐 ) (1) with integers 𝑎𝑖 ∈ [ 0,1], 𝑥𝑖 ∈ [1, 𝑀], m = 9 and 𝑖 = 1 … 𝑛𝑝𝑐 , with 𝑛𝑝𝑐 is the number of pairwise comparisons. 𝑛𝑝𝑐 = 𝑛2−𝑛 2 . (2) for n criteria the n x n decision matrix is then filled from pwc. for 𝑎𝑖 = 0 we take 𝑥𝑖 , for 𝑎𝑖 = 1 we have to take the reciprocal of 𝑥𝑖 . for example, for three criteria with 𝑝𝑤𝑐 = (0,0,1), (3, 5, 7) the decision matrix is m = ( 1 3 5 1 3 1 1 7 1 5 7 1 ) (3) the selected format minimizes redundancy and uses less memory than storing the complete positive reciprocal matrix. the dominant eigenvalue λ of m is calculated using the power method (larsen, 2013). the number of iterations is limited to 20, this is sufficient for an accepted approximation error of 1.e-7. as a result we get the local priority vector with weights wi for criterion i = 1 to n. ijahp article: goepel/ implementation of an online software tool for the analytic hierarchy process (ahp-os) international journal of the analytic hierarchy process 473 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.590 2.3 consistency ratio instead of saaty’s original consistency ratio calculation (saaty, 2008), based on the average random consistency index rin 𝐶𝑅 = λ−𝑛 (𝑛−1)𝑅𝐼𝑛 (4) we use the linear fit proposed by alonso and lamata (2006) to calculate the consistency ratio cr 𝐶𝑅 = 𝜆−𝑛 2.7699 ∙ 𝑛−4.3513−𝑛 (5) it can be used for matrices larger than 10 x 10. in the case of a decision hierarchy with more than one node, cr of each hierarchy node is calculated, and for the global weights the program shows the maximum of all crs. 2.4 calculation of the top-3 inconsistencies if the consistency ratio cr exceeds 10%, the software calculates the top-3 inconsistent judgments. for all elements of the decision matrix dij we construct the inconsistency matrix based on the calculated weights wi and wj as 𝑒𝑖𝑗 = 𝑑𝑖𝑗 ∙ 𝑤𝑗 𝑤𝑖 (6) and identify the three largest 𝑒𝑖𝑗 . the corresponding pairwise comparisons are then highlighted for the decision makers, allowing them to make some adjustments (saaty, 2003). ijahp article: goepel/ implementation of an online software tool for the analytic hierarchy process (ahp-os) international journal of the analytic hierarchy process 474 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.590 table 1 ahp judgment scales implemented in the software no name scale function comment 0 ahp scale 𝑐 = x saaty (1980) 1 logarithmic scale 𝑐 = log𝑎 (𝑥 + a − 1) ishizaka et. al (2010) a = 2 2 root square scale 𝑐 = √x 𝑎 harker, vargas (1987) we use a = 2 3 inverse linear scale 𝑐 = 9 10 − 𝑥 ma-zheng (1991) 4 balanced scale 𝑐 = 9 + 𝑥 11 − 𝑥 salo, hämäläinen (1997) for [0.1,0.9] 5 generalized balanced scale 𝑐 = 9 + (𝑛 − 1)𝑥 9 + 𝑛 − 𝑥 n number of criteria goepel (2019) 6 adaptive-balanced scale 𝑐 = (9𝑛 − 10)(𝑥 − 1) + 80 (9𝑛 − 10)𝑥 − 89𝑛 + 90 (𝑛 − 1) goepel (2019) 7 adaptive scale 𝑐 = 𝑥 1+ ln (𝑛−1) ln 9 goepel (2019) 8 power scale 𝑐 = x𝑎 harker, vargas (1987) a = 2 9 geometric scale 𝑐 = 𝑎𝑥−1 lootsma (1994), a = 2 2.5 ahp judgment scales over the last several decades, a variety of judgment scales, different from saaty’s fundamental 1 to 9 scale, have been proposed. in ahp-os, pairwise comparisons are stored with their original judgment values (equation 1); therefore, we can apply different scales a posteriori and study the effect of different scales (table 1) on the resulting weights. x is the judgment value (x = 1 … 9); c resp. 1/c the ratio used as entry into the decision matrix. goepel (2019) gives a comparison of these scales, based on weight boundaries and weight ratio, weight uncertainty and weight dispersion. 3. ahp-os database and data structure the software uses the php data object (pdo) interface to sql databases. the current implementation can handle either two sql servers, open source mariadb, or the public domain sqlite database engine. the implemented database structure is comprised of four database tables: 1. users: table for user registration and login. registered users can be project authors and can store and manage their own ahp projects. 2. projects: table of ahp projects with unique session code, project name, project description and hierarchy definition. its foreign key is the user name from the table users. ijahp article: goepel/ implementation of an online software tool for the analytic hierarchy process (ahp-os) international journal of the analytic hierarchy process 475 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.590 3. judgments (pwc): table containing all pairwise comparisons with the decision maker’s name and nodes of the decision hierarchy. foreign key is the session code from the table projects. 4. alternatives: table with alternative names. foreign key is the session code from the table projects. this structure minimizes redundancies and keeps the database slim. in order to identify projects, each project gets a unique 6-character session code. as described before, decision hierarchies are defined with plain text input, and pairwise comparisons are stored in a simple integer format. for group decision making, participant’s judgments are stored in the pwc table. participants do not need to register as users; they just need to follow a link with the unique session code provided by the ahp project author. they then can start to input their judgments by opening the link and providing their name. (field participant in table pwc). the ahpdb software class handles the interfacing with the sql database server. the following sql statement is an example and will result in a table of all projects for a specified user name, showing the unique project session code, the project name, number of alternatives (if any), number of participants (decision makers) with pairwise comparison inputs, a short project description and the creation date/time of the project. the database does not store results; all results are calculated on the fly using methods in the software class ahpgroup. select session_code, project_name, count(alt) as alt, count(pwpart) as part, description, date(date_time) from projects left join (select distinct pwc.session_code as pwsc, pwc.participant as pwpart from pwc) on projects.session_code = pwsc left join (select alternatives.session_code as altsc, projects.session_code as alt from projects, alternatives where projects.session_code = alternatives.session_code group by alternatives.session_code ) on projects.session_code = altsc where projects.user_name = :user_name group by session_code order by projects.date_time desc; 3.1 basic data structure each node in the decision hierarchy represents a decision matrix with the number of branches as number of criteria. for the example given in figure 1, ahp project, criterion-1 and criterion-2 are the nodes of the hierarchy with two branches each, and sub-criterion a to d build the leafs of the decision hierarchy. in the software, we need to calculate the weights of each branch or leaf of all nodes and for all decision makers (participants). the weights (local priorities) are stored in a 3-dimensional array with array keys [participant], [node] and [leaf]. the array key [participant = 0] holds the consolidated group results; the array key [node = “ptot”] holds the global priorities. this data structure applies to both, hierarchy and alternative evaluation, where for the latter ijahp article: goepel/ implementation of an online software tool for the analytic hierarchy process (ahp-os) international journal of the analytic hierarchy process 476 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.590 the hierarchy’s leafs become nodes with alternatives as branches. for any calculation, the software has to loop through all nodes. once the weight calculations are done, the results (local priorities) are inserted into the hierarchy definition text, for example criterion-2 becomes criterion-2=0.7. the modified hierarchy text can then be used to define alternatives and be stored as a new project in the database for alternative evaluation. 3.2 partial judgments as pairwise comparisons for each node are stored in the pairwise comparisons database table, it is possible for participants to do a partial evaluation, i.e. to give their input only for a part of the hierarchy or the alternatives. this can be useful, if a project requires inputs from different experts with specific expertise in selected nodes of the decision hierarchy. 4. group decision making 4.1 aggregation of individual judgments one of the major advantages of a web based ahp software is that it allows global access. once a decision hierarchy is defined and stored as a project on the server, decision makers can give their inputs from any location, as long as they have internet access. in ahp-os, each project is identified by a unique session code and participants only need a link with this session code to start their pairwise comparisons and submit their judgments. the number of participants is practically unlimited. in order to consolidate all participant’s judgments, several group ahp aggregation techniques are available, but not all group ahp methods are equally convenient, and the selection of the method might depend on the specific application (grošelj. p et al., 2015; wen-hsiang wu et al., 2008). in the current version of the software, we use the weighted geometric mean aggregation of individual judgments (wgm-aij) as it is the only method that meets several required axiomatic conditions such as the reciprocal property. we calculate the geometric mean and standard deviation of all k participant’s individual judgments 𝑝𝑤𝑐𝑘 to form the consolidated decision matrix 𝑎𝑖𝑗 cons. sum over k participants 𝑝𝑤𝑐𝑥 = ∑ ln(𝑝𝑤𝑐𝑘 ) 𝐾 𝑘=1 (7) square sum over k participants 𝑝𝑤𝑐𝑥2 = ∑ [ln(𝑝𝑤𝑐𝑘 )] 2𝐾 𝑘=1 (8) geometric mean 𝑝𝑤𝑐cons = exp ( 𝑝𝑤𝑐𝑥 𝐾 ) (9) standard deviation 𝑝𝑤𝑐sd = exp ( √ 𝑝𝑤𝑐𝑥2 − 1 𝐾 𝑝𝑤𝑐𝑥∙𝑝𝑤𝑐𝑥 𝐾−1 ) (10) using the pairwise comparison data from equation 9 to fill the consolidated decision matrix we get 𝑎𝑖𝑗 cons = (∏ 𝑎𝑖𝑗 𝐾 𝑘=1 ) 1 𝐾 (11) ijahp article: goepel/ implementation of an online software tool for the analytic hierarchy process (ahp-os) international journal of the analytic hierarchy process 477 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.590 the standard deviation 𝑝𝑤𝑐sd of individual judgments (equation 10) is used to estimate weight variations based on judgment variations. 4.2 group consensus although it is always possible to calculate a group result using wgm-aij, it does not makes sense in all cases. for example, if we have two opposite judgments for two criteria (x and 1/x), an aggregation will result in equal weights for both criteria. in fact, there is no consensus, and equal weights may result in a deadlocked situation to solve a decision problem. therefore, it is necessary to measure the consensus for the aggregated group result. we use shannon entropy and its partitioning in two independent components (alpha and beta diversity) to derive the ahp consensus indicator. we analyze the priority distribution of criteria among different decision makers. we use the following relations (goepel, 2013): shannon γ-entropy 𝐻𝛾 = ∑ (−𝑤avgln 𝑤avg) 𝑛 𝑗=1 (11) with 𝑤avg = 1 𝑘 ∑ 𝑤𝑗𝑘 𝑘 𝑗=1 shannon α-entropy 𝐻𝛼 = 1 𝑘 ∑ ∑ (−𝑤𝑗ln 𝑤𝑗 ) 𝑛 𝑗=1 𝑘 𝑖=1 (12) shannon β-entropy 𝐻𝛽 = 𝐻𝛾 − 𝐻𝛼 (13) the similarity measure s (equation 14) depends on the number of criteria, and we used a linear transformation to map it to a range from 0 to 1 (equation 15). 𝑆 = exp(− 𝐻𝛽 ) = 1 𝐷β (14) 𝑆 ∗ = s−𝐷α min 𝐷γ max⁄ 1−𝐷α min 𝐷γ max⁄ (15) in general dα min = 1 and dγ max = n. in the analytic hierarchy process (ahp) minimum alpha entropy 𝐻α min = ln(𝐷α min) is a function of the maximum scale value m (m = 9 for the fundamental ahp scale) and the number of criteria n. 𝐻𝛼 min = − ( 𝑀 𝑛+𝑀−1 ) 𝑙𝑛 [ 𝑀 𝑛+𝑀−1 ] − ( 𝑛−1 𝑛+𝑀−1 ) 𝑙𝑛 [( 1 𝑛+𝑀−1 )] (16) maximum gamma entropy 𝐻γ max = ln (𝐷γ max) is 𝐻𝛾 max = ln (𝑛) (17) the correction factor cor is 𝑐𝑜𝑟 = 𝑒𝑥𝑝[𝐻𝛾 max − 𝐻𝛼 min] −1 = 𝐷α min 𝐷γ max⁄ = exp (𝐻𝛼 min) 𝑛 (18) ijahp article: goepel/ implementation of an online software tool for the analytic hierarchy process (ahp-os) international journal of the analytic hierarchy process 478 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.590 the ahp consensus indicator is based on the relative homogeneity s 𝑆 = 1 𝐷𝛽 − 1 𝐷𝛽 min (19) ahp group consensus indicator 𝑆 ∗ = ( 1 exp (𝐻𝛽) − 1 exp (𝐻𝛾 max−𝐻𝛽 min) ) / (1 − 1 exp (𝐻𝛾 max−𝐻𝛽 min) ) (20) 𝑆 ∗ = 1 d𝛽 ⁄ − 𝑐𝑜𝑟 1− 𝑐𝑜𝑟 (21) the consensus of the complete hierarchy is calculated as weighted arithmetic mean of the consensus of all hierarchy nodes (categories); for alternatives, we take the consensus indicator of the consolidated alternative evaluation matrix weighted with the global priorities. the ahp consensus indicator (equation 21) ranges from 0% (no consensus) to 100% (full consensus). the whole range is categorized into five categories according table 2. table 2 interpretation of ahp consensus indicator s * s* consensus ≤ 50% very low 50% 65% low 65% 75% moderate 75% 85% high ≥85% very high 5. weight uncertainties as shown by goepel (2019), weight uncertainties due to rounding of the judgment to integers can exceed 10% and could affect the results of a decision. in order to get an estimation of uncertainties, we generate nvar randomized variations of the original judgments with 𝑝𝑤𝑐𝑟𝑗 = (𝑎1, … 𝑎𝑖 , … 𝑎𝑛𝑝𝑐 ), (𝑥1 + ∆𝑥1, … 𝑥𝑖 + ∆𝑥𝑖 , … , 𝑥𝑛𝑝𝑐 + ∆𝑥𝑛𝑝𝑐) (22) with 𝑎1 = 0 for 𝑥𝑖 + ∆𝑥𝑖 ≥ 1, and 𝑎1 = 1 for 𝑥𝑖 + ∆𝑥𝑖 < 1, where ∆𝑋 are randomly uniform distributed values (-0.5, 0., +0.5). ∆𝑋~ 0.5 ∙ 𝑈(−1 , +1)/√𝐾 (23) ijahp article: goepel/ implementation of an online software tool for the analytic hierarchy process (ahp-os) international journal of the analytic hierarchy process 479 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.590 and k is the number of participants. for each variation, 𝑝𝑤𝑐𝑟𝑗 the eigenvector solution 𝑤𝑗 and consistency ratio cr is calculated. we then capture the maximum and minimum of the weights for all cr < 0.25. ∆𝑤(+) = (𝑤𝑚𝑎𝑥 − 𝑤) = (max[𝑤𝑗 = 𝐸𝑉(𝑝𝑤𝑐𝑟𝑗 )]𝑗=1 𝑁𝑉𝐴𝑅 − 𝑤) (24) ∆𝑤(−) = (𝑤 − 𝑤min) = (𝑤 − min[𝑤𝑗 = 𝐸𝑉(𝑝𝑤𝑐𝑟𝑗 )]𝑗=1 𝑁𝑉𝐴𝑅 ) (25) figure 3 shows a typical example of weight uncertainties of a project with four criteria using the standard ahp 1 to 9 scale. figure 3 calculated weights for a project with four criteria showing weight uncertainties due to rounding alternatively, users can also display uncertainties using the standard deviation of judgments 𝑝𝑤𝑐sd (equation 10) if there is more than one participant. it gives, in addition to the ahp consensus, an idea about the weight variations based on participant’s judgment variations. 𝑝𝑤𝑐𝑟𝑗 = (𝑎1, … 𝑎𝑖 , … 𝑎𝑛𝑝𝑐 ), (𝑥1∆𝑥1, … 𝑥𝑖 ∆𝑥𝑖 , … , 𝑥𝑛𝑝𝑐∆𝑥𝑛𝑝𝑐 ) (26) ∆𝑋~𝑁(𝜇, 𝜎2) (27) ∆𝑥𝑖 = exp( 1 2 log 𝑝𝑤𝑐sd ∙ ∆𝑋) (28) 5.1 overlap the program provides a list of criteria or alternatives overlapping within the estimated weight uncertainties. overlapping criteria or alternatives should get the same ranking, as they cannot be differentiated within the estimated uncertainties. ijahp article: goepel/ implementation of an online software tool for the analytic hierarchy process (ahp-os) international journal of the analytic hierarchy process 480 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.590 6. alternative evaluation 6.1 weighted sum and weighted product model in ahp, the preference pi of alternative ai is calculated using the weighted sum model (wsm): 𝑃𝑖 = ∑ 𝑎𝑖𝑗 𝑤𝑗 𝑛 𝑗=1 (29) with wj the weight of criterion cj, and aij the performance measure of alternative ai with respect to criterion cj. performance values are normalized. ∑ 𝑎𝑖𝑗 𝑛 𝑗=1 = 1 (30) users can also select the weighted product model (wpm), where alternatives are aggregated using the product instead of the sum 𝑃𝑖 = ∏ 𝑎𝑖𝑗 𝑤𝑗𝑛 𝑗=1 (31) some of the first references to this method are due to bridgman (1922) and miller and starr (1969). 7. sensitivity analysis sensitivity analysis is a fundamental concept in the effective use and implementation of quantitative decision models, whose purpose is to assess the stability of an optimal solution under changes in the parameters. sensitivity analysis will answer two questions (triantaphyllou, 1997):  which is the most critical criterion?, and  which is the most critical performance measure, changing the ranking between two alternatives? the most critical criterion is defined as the criterion ck, with the smallest absolute (𝛿𝑘,𝑖,𝑗 abs ) or relative (𝛿𝑘,𝑖,𝑗 rel ) change of the current weight wk by the amount of δkij changing the ranking between the alternatives ai and aj. the most critical measure of performance is defined as the minimum absolute or relative change of the current value of aij such that the current ranking between alternative ai and aj will change. the calculation is different for wsm and wpm. 7.1 weighted sum model (wsm) to find the most critical criterion we calculate for each pair of alternatives ai, aj, with i = 1 to n and i < j 𝛿𝑖,𝑗,𝑘 abs (𝐴𝑖 , 𝐴𝑗 , 𝑊𝑘 ) = 𝑃𝑗−𝑃𝑖 𝑎𝑗𝑘−𝑎𝑖𝑘 (32) with |𝛿𝑖,𝑗,𝑘 abs (𝐴𝑖 , 𝐴𝑗 , 𝑊𝑘 )| < 𝑊𝑘 and 𝛿𝑖,𝑗,𝑘 rel (𝐴𝑖 , 𝐴𝑗 , 𝑊𝑘 ) = 𝛿𝑘,𝑖,𝑗 abs /𝑊𝑘 ijahp article: goepel/ implementation of an online software tool for the analytic hierarchy process (ahp-os) international journal of the analytic hierarchy process 481 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.590 to find the most critical measure of performance we calculate for all alternatives ai and ak with i ≠ k and each criterion 𝛿𝑖,𝑗,𝑘 abs (𝑊𝑗 , 𝑎𝑘𝑗 , 𝑎𝑖𝑗 ) = 𝑃𝑖−𝑃𝑘 𝑃𝑖−𝑃𝑘+𝑊𝑗(𝑎𝑘𝑗−𝑎𝑖𝑗+1) (33) with |𝛿𝑖,𝑗,𝑘 abs (𝑊𝑗 , 𝑎𝑘𝑗 , 𝑎𝑖𝑗 )| < 𝑊𝑗 and 𝛿𝑖,𝑗,𝑘 rel (𝑊𝑗 , 𝑎𝑘𝑗 , 𝑎𝑖𝑗 ) = 𝛿𝑘,𝑖,𝑗 abs /𝑎𝑖𝑗 (34) 7.2 weighted product model (wpm) to find the most critical criterion for each pair of alternatives ai, ak, with i = 1 to n and i < k we calculate log (∏ (𝑎𝑖𝑦 𝑎𝑘𝑦⁄ ) 𝑊𝑦𝑛 𝑦=1 ) (35) and 𝛿(𝐴𝑖 , 𝐴𝑘 , 𝑊𝑗 ) = log(∏ (𝑎𝑖𝑦 𝑎𝑘𝑦⁄ ) 𝑊𝑦𝑛 𝑦=1 ) log(𝑎𝑖𝑗 𝑎𝑘𝑗⁄ ) (36) with 𝛿abs(𝐴𝑖 , 𝐴𝑘 , 𝑊𝑗 ) < 𝑊𝑗 then we look at the smallest 𝛿rel(𝐴𝑖 , 𝐴𝑘 , 𝑊𝑗 ) = 𝛿abs 𝑊𝑗 (37) to find the most critical measure of performance for the wpm we calculate for all alternatives ai and ak with i ≠ k and each criterion 𝛿rel(𝐴𝑖 , 𝐴𝑘 , 𝑊𝑗 ) = (1 − √ 𝐴𝑘 𝐴𝑖 𝑊𝑗 ) (38) and look for the smallest 𝛿rel(𝐴𝑖 , 𝐴𝑘 , 𝑊𝑗 ) < 1 (39) and 𝛿abs = 𝑎𝑖𝑗 𝛿 rel (40) 7.3 program output for sensitivity analysis depending on the results, the program will then display the following text output: a) the solution for the top alternative ax is robust. b) the percent-top critical criterion is cy, a change from y% by absolute dy% will change the ranking between alternatives ai and ak. c) the percent-any critical criterion is the same as above. ijahp article: goepel/ implementation of an online software tool for the analytic hierarchy process (ahp-os) international journal of the analytic hierarchy process 482 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.590 d) the percent-any critical criterion is cz, a change from z% by absolute dz% will change the ranking between alternatives ai and aj. e) the percent-any critical performance measure is for alternative al under criterion cz. a change from z% by absolute dz% will change the ranking between ai and aj. when exporting the results, the complete sensitivity tables (equation 32 and 33 or equations 36 and 40) will be exported. 8. data export ahp-os is intended for educational and research purposes, therefore it is important to provide an extensive possibility of data export for further analysis or presentation. the most common and universal format is the comma separated value (csv) text format. it can be easily imported into most spreadsheet programs, independent from the operation system. the software provides download functions for input data (hierarchy, decision matrices), local and global weights with estimated uncertainties, alternative tables and sensitivity analysis. 9. software validation coding and implementation of the complete ahp-os software package resulted in approximately ten thousand lines of code; therefore, testing and validation became essential. with each new release, the author goes through a pre-defined checklist for unit, integration and system testing. in addition, in order to ensure that the user can rely on the accuracy of the program results, test cases were designed for black box testing. we compared program results with manually calculated results and results published in the literature. a simple test case with known results is the case where we judge one criterion x-times more important than all others. the weight for this criterion has to be (goepel, 2019) 𝑤ahp = 𝑥 𝑥+𝑛−1 (41) for example, if all criteria are equal (x = 1), it follows that 𝑤ahp = 1/𝑛, and for four criteria, with one criterion nine times more important than all others, it follows 𝑤ahp = 9 12 𝑜𝑟 75%. in the next step we can apply different scales functions according table 1 by replacing x with the scaled value c. group decision making can be validated by verifying that the elements of the consolidated group matrix show the geometric mean of the elements of the individual matrices. testing the ahp consensus indicator is done by simulating k participants for a hierarchy with k leafs, and each participant judges one different leaf extreme more important than all others. the resulting consensus should show 0% for all scales. ijahp article: goepel/ implementation of an online software tool for the analytic hierarchy process (ahp-os) international journal of the analytic hierarchy process 483 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.590 as a test case for weight uncertainties we use the derivate of equation 41 d𝑤 d𝑥 = 𝑛−1 (𝑥+𝑛−1)2 (42) with n = 2, dx = 0.5 and x’ = x ± dx/2, x > 1 we get ∆𝑤(+) = 0.5 (𝑥+1.25)2 (43) ∆𝑤(−) = 0.5 (𝑥−0.75)2 (44) for example, for two criteria a and b with a three times more important than b, we get w = 75% with ∆𝑤(+) = 2.8% and ∆𝑤(−) = 3.6%. alternative evaluation using the weighted sum or weighted product model can be verified exporting the results and using a spreadsheet program and its sumproduct function. as a test case for sensitivity analysis we use a spreadsheet program with exported data and subtract 𝛿abs from the original weight 𝑊𝑗 w′𝑘 = 𝑊𝑘 − 𝛿 abs (45) using wsm equation 29 or wpm equation 31 for alternative aggregation the performance measure of alternative ai should now equal that of alternative aj. in a similar way we can calculate 𝑎′𝑖𝑗 = 𝑎𝑖𝑗 − 𝛿 abs (46) and alternative ai should have the same performance measure as alternative ak. for wsm we have to renormalize 𝑎𝑖𝑗 according equation 30. 10. experience and outlook ahp-os was originally released in 2014, and features described in this paper were developed gradually over the last three years. with the currently implemented functionality, the software tool has reached a state where it covers most of the possible options for the classical analytic hierarchy process. it allows for a wide range of parameter variations to analyze and study a specific project under different aspects. this and the possibility of transparent data export makes it an ideal tool for study, education and further research. up to the date of writing this paper, more than 7000 users have registered for the software, and on average, there are at least 500 active users over a three-month period. ahp projects handled with ahp-os cover a wide range of applications like healthcare, climate, risk assessment, supplier selection, hiring, it, marketing, environment, transport, project management, manufacturing or quality assurance. numerous projects are used to acquire group inputs; the number of participants goes up to 320 in a single project. multilanguage support is utilized for languages like chinese, korean, russian, hebrew, greek, ijahp article: goepel/ implementation of an online software tool for the analytic hierarchy process (ahp-os) international journal of the analytic hierarchy process 484 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.590 thai, vietnamese or arabic. lecturers at universities make use of ahp-os as a tool for their students when teaching multi-criteria decision-making methods. overall, the number of software bugs has been kept reasonably low, more often user’s feedback and questions relate to the understanding of the ahp and have straightforward solutions. the main challenges in maintaining the software include updates of the underlying open source tools due to security reasons, or smaller updates requested by users. these updates always require extensive testing and validation because of the complexity of the software. a few extensions of ahp-os are under consideration for the future. for example, a selection of different group aggregation techniques in addition to wgm-aij. as a major extension, a cluster analysis of group consensus is planned. the concept of shannon entropy and its partitioning in alphaand beta-entropy opens the possibility to identify clusters of sub-groups with high consensus within a group of decision makers. this could help to gain a deeper insight into group decision-making processes. ijahp article: goepel/ implementation of an online software tool for the analytic hierarchy process (ahp-os) international journal of the analytic hierarchy process 485 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.590 references alonso, j. a., lamata, t. (2006). consistency in the analytic hierarchy process: a new approach. international journal of uncertainty, fuzziness and knowledge-based systems, 14(4), 445−459. doi: https://doi.org/10.1142/s0218488506004114 bridgman, p.w. (1922). dimensional analysis. (revised edition). new haven, ct, u.s.a.: yale university press. goepel, k. d. (2013). implementing the analytic hierarchy process as a standard method for multi-criteria decision making in corporate enterprises – a new ahp excel template with multiple inputs. proceedings of the international symposium on the analytic hierarchy process for multicriteria decision making, kuala lumpur, malaysia. goepel, k. d. (2019). comparison of judgment scales of the analytical hierarchy process a new approach. international journal of information technology and decision making vol.17(2019). doi: 10.1142/s0219622019500044 grošelj, p. stirn, l. z., ayrilmis, n., kuzman, m. k. (2015). comparison of some aggregation techniques using group analytic hierarchy process. expert systems with applications, 42, 2198–2204. doi: https://doi.org/10.1016/j.eswa.2014.09.060 harker, p., vargas, l. (1987). the theory of ratio scale estimation: saaty's analytic hierarchy process. management science, 33(11), 1383–1403. doi: https://doi.org/10.1287/mnsc.33.11.1383 ishizaka,a., labib, a. (2009). analytic hierarchy process and expert choice: benefits and limitations. or insight, 22(4), 201–220. doi: https://doi.org/10.1057/ori.2009.10 ishizaka, a., balkenborg, d., kaplan, t. (2010). influence of aggregation and measurement scale on ranking a compromise alternative in ahp. journal of the operational research society 62, 700–710. doi: https://dx.doi.org/10.2139/ssrn.1527520 larsen, r. (2013). elementary linear algebra. boston, ma: cengage learning. lootsma, f. (2008). conflict resolution via pairwise comparison of concessions. european journal of operational research, 40,109–116. doi: https://doi.org/10.1016/0377-2217(89)90278-6 ma, d., & zheng, x. (1991). 9/9–9/1 scale method of ahp. 2nd international symposium on ahp, 1, 197–202. miller, d.w., starr, m.k. (1960). executive decisions and operations research. englewood cliffs, nj, u.s.a.: prentice-hall inc. ossadnik, w., kaspar, r. (2013). evaluation of ahp software from a management accounting perspective. journal of modelling in management, 8(3), 305-319. doi: https://doi.org/10.1108/jm2-01-2011-0007 ijahp article: goepel/ implementation of an online software tool for the analytic hierarchy process (ahp-os) international journal of the analytic hierarchy process 486 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.590 saaty, t. l. (1980). the analytic hierarchy process: planning, priority setting, resource allocation. mcgraw-hill. doi: https://doi.org/10.1016/0377-2217(82)90022-4 saaty, t.l. (2003). decision-making with the ahp: why is the principal eigenvector necessary. european journal of operational research, 145, 85–91. doi: https://doi.org/10.1016/s0377-2217(02)00227-8 salo, a., hämäläinen, r. (1997). on the measurement of preferences in the analytic hierarchy process. journal of multi-criteria decision analysis, 6, 309–319. doi: https://doi.org/10.1002/(sici)1099-1360(199711)6:6%3c309::aidmcda163%3e3.0.co;2-2 siraj, s., mikhailov, l., keane, j. a., (2015). an interactive decision support tool to estimate priorities from pairwise comparison judgments (priest). international transactions in operational research, 22(2), 217–235. doi: https://doi.org/10.1111/itor.12054 triantaphyllou, e., sánchez, a. (1997). sensitivity analysis approach for some deterministic multi-criteria decision making methods. decision sciences, 28(1), 151194. doi: https://doi.org/10.1111/j.1540-5915.1997.tb01306.x wen-hsiang wu, chang-tzu chiang, chin-tsai lin (2008). comparing the aggregation methods in the analytic hierarchy process when uniform distribution. wseas transactions on business and economics, 5(3), 82 – 87. ijahp article: goepel/ implementation of an online software tool for the analytic hierarchy process (ahp-os) international journal of the analytic hierarchy process 487 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.590 appendix 1. open source software packages used for ahp-os php, version 5.6.31, http://php.net/ mariadb, mariadb foundation, version 5.5.54, https://mariadb.org/ sqlite, version 3.8.10.2, https://sqlite.org/index.html jquery java script library, https://jquery.com/ phpgraphlib graphing library, elliot brueggeman, http://www.ebrueggeman.com/phpgraphlib phpmailer, the classic email-sending library for php, https://github.com/phpmailer/phpmailer 2. other ahp software packages 123ahp, http://123ahp.com/ ahp-os, https://bpmsg.com/academic/ahp.php makeitrational, http://makeitrational.com choiceresults, https://www.filebuzz.com/fileinfo/91865/choiceresults.html decisor, https://github.com/czekster/decisor/ hipre 3+, http://sal.aalto.fi/en/resources/downloadables/hipre3 superdecisions, https://superdecisions.com/ expert choice, https://expertchoice.com/ decision lens, https://decisionlens.com cdp, criterium® decisionplus®, http://www.infoharvest.com/ihroot/index.asp rightchoicedss, https://www.updatestar.com/publisher/tgkconsulting-2369 priest, ahp priority estimation tool, https://sourceforge.net/projects/priority/ ijahp news and events: dr. kahraman is among the 100 prominent turkish scientists contributing to world science international journal of the analytic hierarchy process 299 vol. 9 issue 2 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i2.501 dr. kahraman is among the 100 prominent turkish scientists contributing to world science dr. kahraman (the first on the right), who serves on the ijahp board iof editors, was recognized among the 100 most prominent turkish scientists contributing to world science, and received a plate from the technical university of istanbul for his studies on industrial engineering and fuzzy set theory on september 8, 2017. cengiz kahraman is our guest editor for a special issue on ahp variants based on extensions of ordinary fuzzy sets which will be published in december 2018. ijahp article: raco, ohoitimur, krejci, raton, rottie, paseru, muajan, rachmadi/the dominant factor of lecturers’ research productivity using the ahp: case study of catholic university of de la salle manadoindonesia international journal of the analytic hierarchy process 546 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.819 the dominant factor of lecturers’ research productivity using the ahp: case study of catholic university of de la salle manadoindonesia jozef richard raco* 1 management department, universitas katolik de la salle manado, indonesia kombos, kairagi i, manado 95253, indonesia jraco@unikadelasalle.ac.id johanis ohoitimur philosophy department, sekolah tinggi filsafat seminari pineleng indonesia johoitimur@unikadelasalle.ac.id james v. krejci lewis university, illinois usa college of business one university parkway romeoville, il 60446 krejcija@lewisu.edu yulius raton industrial engineering department, universitas katolik de la salle manado, kombos, kairagi i, manado 95253, indonesia yraton@unikadelasalle.ac.id ronaldo rottie industrial engineering department, universitas katolik de la salle manado, kombos, kairagi i, manado 95253, indonesia rrottie@unikadelasalle.ac.id debby paseru informatics engineering department, universitas katolik de la salle manado, kombos, kairagi i, manado 95253, indonesia dpaseru@unikadelasalle.ac.id octavianus muaja accounting department , universitas katolik de la salle manado, indonesia kombos, kairagi i, manado 95253, indonesia omuaja@unikadelasalle.ac.id ronald rachmadi industrial engineering department, universitas katolik de la salle manado, kombos, kairagi i, manado 95253, indonesia rrachmadi@unikadelasalle.ac.id 1 the authors wish to thank father benansio salombre, the head of yayasan perguruan tinggi universitas de la salle, for his support on this research. *corresponding author mailto:jraco@unikadelasalle.ac.id mailto:johoitimur@unikadelasalle.ac.id mailto:krejcija@lewisu.edu mailto:yraton@unikadelasalle.ac.id mailto:rrottie@unikadelasalle.ac.id mailto:dpaseru@unikadelasalle.ac.id mailto:omuaja@unikadelasalle.ac.id mailto:rrachmadi@unikadelasalle.ac.id ijahp article: raco, ohoitimur, krejci, raton, rottie, paseru, muajan, rachmadi/the dominant factor of lecturers’ research productivity using the ahp: case study of catholic university of de la salle manadoindonesia international journal of the analytic hierarchy process 547 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.819 abstract higher education places a great emphasis on research. in order to remain competitive, catholic universities also place a great focus on research. the quality and quantity of lecturer research is one of the determining factors in the ranking of a tertiary institution. an evaluation of lecturer research productivity requires an instrument to measure lecturer research activities and understand determinants. lecturer productivity involves many complex factors that are both objective and subjective, and is considered part of multi-criteria decision making. this study uses the ahp approach to determine the weights of the criteria and sub-criteria and their dominant factors. this method is used to evaluate the productivity of lecturers at unika de la salle manado and has implications and applications for all universities. keywords: ahp; commitment; higher education; mcdm; productivity; catholic 1. introduction pope francis, in his encyclical letter on ecclesiastical universities and faculties, known as ‘veritatis gaudium’, wrote that research is the primary duty of catholic universities to communicate doctrine to our contemporaries in various countries and cultures. (pope francis, 2017). research is the basis for the development of a civilized society and is used for decision making by most institutions, organizations and companies and is also an integral part of r&d institutions (pal & sarkar, 2020). research has received a great deal of attention and support from the indonesian government in the form of funding which has resulted in a significant number of publications (dirjen dikti, 2019). in higher education, research is one of the key determinates of a university’s ranking. as a result, leaders of higher education institutions are always seeking to motivate their lecturers to conduct and publish research. an evaluation of lecturer research productivity is regularly conducted by higher education institutions to assess a lecturer’s potential for career growth and meeting the challenges of the future. (henry, ghani, hamid, & bakar, 2020). many studies have been conducted to evaluate lecturers' research productivity. however, an evaluation of lecturer productivity is extremely difficult, and involves multiple variables that can be qualitative or quantitative, and can also be complex, sensitive and subject to imprecise measurement (ates, çevik, kahraman, gülbay, & erdogan, 2006). therefore, the quantitative approach is often used (pal & sarkar, 2020). lecturer productivity has been measured by the number of publications, the acquisition of patents, citation of scientific work, or papers presented at the seminars (abramo & d’angelo, 2014). in general, publications are often viewed as the benchmark for assessing lecturers’ research abilities and performance. higher education institutions often assess research capabilities and scientific publications to recruit new lecturers and as criteria for promotion (shin, arimoto, cummings, & teichler, 2014). the multi-criteria decision analysis approach model was used by costa and oliveira (2012) to evaluate lecturer productivity. the following criteria and sub criteria were used: teaching (pedagogical publications, student supervision, teaching materials), research (scientific publications, research projects), transfer of knowledge (patents, ijahp article: raco, ohoitimur, krejci, raton, rottie, paseru, muajan, rachmadi/the dominant factor of lecturers’ research productivity using the ahp: case study of catholic university of de la salle manadoindonesia international journal of the analytic hierarchy process 548 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.819 laws, norms and technical publications; services, consultations and science and technology dissemination), and university (costa & oliveira, 2012). jaramillo et al. also used a multi-criteria decision making approach to rank lecturers at the university of ecuador by evaluating criteria such as administrative management, research, teaching and community service (jaramillo, pico, marquez, & plata, 2017). karmaker et al. applied a combination of multi-criteria decision making models to evaluate the performance of lecturers at the technical university of bangladesh (karmaker et al., 2018). they specifically used the analytical hierarchy process method to determine the weight of performance indicators and the preference preparation technique based on similarity and ideal solutions (topsis). the topsis application was combined with the ahp to determine the composition of lecturers in higher education institutions. they used 15 sub-criteria of the following five criteria: knowledge (teaching ability, understanding student psychology, ability to make decisions), ability to explain (ideas are clear, easy to find, proficient in explaining), discipline (fair, attitude, organized), cooperation (can be an example, dedicated, motivated), creative (encouraging positively, passionately, full of inspiration). do et al. (2019) used a novel multi-criteria hierarchy method in his evaluation of lecturer performance. there were four criteria, 13 sub-criteria and five alternatives assessed as follows: self-evaluation (scientific publication activities, supervising postgraduate students, journal preview process); manager-based evaluation (lecturing activities, language of instruction, lecturing attitude and spirit, evaluation and scoring system); peer-evaluation (cooperation in research projects, teamwork in scientific and lecturing activities, participating in school meetings and events); and student-based evaluation (the content of lessons, lecturer-student interaction, the relevance of the subjects). their findings showed that the highest criterion was the manager's evaluation. the sub-criteria that received high ratings were lecturer activities, lecturer style and content of lessons. tuan et al. used the new integrated mcdm to evaluate five criteria including number of publications, quality of publications, number of books, supervising postgraduate students, and research grants secured as project leaders. the highest ranked criterion was research grants secured as project leaders, followed by the quality of publication and supervising postgraduate students. they also stated that the mcdm approach was appropriate in their study (tuan et al., 2020). additional factors that affect researchers’ productivity are as follows: the terminal degree held by the lecturer (henry et al., 2020); lecturer’s level of commitment (batugal & tindowen, 2019); the number of young lecturers (do et al., 2020); the amount of internal research funding (henry et al., 2020); the degree of researcher motivation; number of teaching hours (abramo & d’angelo, 2014); the extent of infrastructure and support facilities (budiyono, 2019; suhardi, fuad, & rosyidi, 2019); number of research publications (do et al., 2020); degree of cooperation between government and private institutions (suhardi et al., 2019); number of government research grants awarded (suhardi et al., 2019); cooperation with foreign institutions (abramo & d’angelo, 2014); level and capability of network resources available to faculty and study programs (abramo & d’angelo, 2014); degree of regulations that affect lecturers’ productivity (suhardi et al., 2019); and the number of research grants available from international institutions (auranen & nieminen, 2010). ijahp article: raco, ohoitimur, krejci, raton, rottie, paseru, muajan, rachmadi/the dominant factor of lecturers’ research productivity using the ahp: case study of catholic university of de la salle manadoindonesia international journal of the analytic hierarchy process 549 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.819 this study aims to examine what attributes influence the productivity of lecturer research at unika de la salle manado and determine which are the dominant factors using the ahp method. an evaluation of lecturers' research productivity using the ahp method is appropriate precisely because of the complex, subjective and sensitive nature of the analysis. this research, among others, attempts to bridge the gap. the research is organized as follows: the first part is the introduction; the second part discusses the importance of research at catholic universities; the third part states the methods; the fourth part is the results and discussion; and the fifth part provides the conclusions, limitations and recommendations. 2. research at catholic universities catholic universities were born from the heart of the church and gained their identity in close relationship with the church. catholic universities carry out the task of the church to seek and find truth and enhance human dignity and cultural heritage for the good of all people. this is done through scientific research (john paul ii, 1990). catholic universities and institutes serve as a primary source of scientific research (paul vi, 1965). through research, the church continues to reflect the growing treasury of human knowledge in the light of the catholic faith (garrett, 2006). the church exists in contact with the world, and encounters various cultures, new realities and knowledge. the meeting of faith, culture and new knowledge gives rise to new categories. the category accepted by all parties then becomes a means of evangelization. this category is not only limited to new thinking but is further extended by data and scientific findings. this is called creative apologetics (pope francis, 2017). the arguments for the defense of the faith are not only in dogma, but in the data and findings of science. in this reality, the thoughts and teachings of the church are communicated, interpreted, modified, integrated and developed (pope francis, 2017). the blending of thought produces new approaches and arguments on the issue of faith. church doctrine, as well as dogma and tradition are not static, but are developed, strengthened and deepened over time. the teachings of the church are always reflected in the context of a certain period. herein lies the importance of research. research allows the dogma and realm of faith to evolve and be reflected in accordance with the times. research helps the church preach what is relevant for modern times (john paul ii, 1990). research helps the church respond to the problems and needs of the age and discovers the root causes of current serious problems. this is especially relevant in the current time, which is marked by rapid, constant and far-reaching changes in the fields of science and technology. we need a true evangelical hermeneutic to better understand life, the world and humanity, using research based on the truth of reason. (pope francis, 2017). an example of research that helps the church understand modern problems is environmental damages as addressed by pope francis in his encyclical on care of our common home called laudato si. science does not have to conflict with faith (george, 2000). research must pay attention to ethical issues. research activities have to place ethics over the technical, the primacy of the person over things, and the superiority of spirit over matter (john paul ii, 1990). the research activities that need to be addressed by universities are problems currently related to the dignity of human life, the promotion of justice for all, the ijahp article: raco, ohoitimur, krejci, raton, rottie, paseru, muajan, rachmadi/the dominant factor of lecturers’ research productivity using the ahp: case study of catholic university of de la salle manadoindonesia international journal of the analytic hierarchy process 550 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.819 qualities of personal and family life, the protection of nature, research for peace and political stability, the just sharing in the world's resources and investigation of new economic and political orders that will better serve the human community. catholic researchers should develop the social science technical and methodological expertise to be effective in the public sector. simboli (2012) emphasized that a catholic university must have the courage to speak uncomfortable truths, being both critical and constructive, which may not comport with public opinion, but which are necessary to safeguard the authentic good of society. the catholic intellectual tradition has the resources to engage these issues, but what is needed now is the will to do so. the catholic congregation of education invites us to generate new thinking across sciences by using intensive and extensive empirical studies (grace, 2016). collaboration between academic disciplines is needed to solve the complex issues and different dimensions of human life and society. the interdisciplinary approach is positive and promising (pope francis, 2017) and generates the respect due to colleagues by recognizing their expertise (iozzio, 2000). quality research as outlined above will greatly influence all teaching. scientists strive to do research which they consider important, but intrinsic satisfaction and interest are not the only reason to do research. scientists want their work to not only be of personal interest but also important to others and the whole world (warner & caudill, 2013). research also plays a decisive role in economic, social and cultural development (pope francis, 2017). in this way, scientists became apostles among the people of the modern world through productive scientific activities (paul vi, 1965). 3. methods 3.1. research steps this study was carried out using three steps. the first step was to identify the factors that influence the productivity of lecturers’ research as discovered by previous studies and additional attributes or factors proposed by experts. these factors were arranged by experts based on internal influences (strengths and weaknesses) and external influences (opportunities and threats). next, the researchers developed a questionnaire and distributed it to experts online. the experts were asked to add, subtract and determine the factors that they considered to significantly influence lecturers’ productivity at the universitas katolik de la salle. the experts were selected based on their knowledge and involvement in the measurement of lecturers' research productivity. the experts consisted of officials who are directly involved in evaluating the productivity of lecturers, namely the vice rector of research, the head of research and community service, the deans and the head of the study program at unika de la salle. fifty-eight respondents completed the questionnaire. researchers identified the top seven factors from each criterion by computing the results of the surveys and using pareto analysis techniques. according to saaty (2016), seven is a powerful, almost magical number in reaching accurate decisions. he goes on to submit that the number of jurors should be seven, and classification of performance stimuli is nearly perfect when limited to the comparison of up to seven different stimuli. additional examples include maslow's hierarchy of needs which describes seven basic needs and the bible that declares that god worked several days and rested on the seventh day (saaty, 2016). miller (1956) added that a person cannot simultaneously compare more than seven objects (plus or minus two) without becoming confused. ijahp article: raco, ohoitimur, krejci, raton, rottie, paseru, muajan, rachmadi/the dominant factor of lecturers’ research productivity using the ahp: case study of catholic university of de la salle manadoindonesia international journal of the analytic hierarchy process 551 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.819 a pareto chart is a tool that is useful for dividing data into categories and establishing the number of times each category occurs (silungwe & khatleli, 2018). with the pareto chart technique, researchers take 80% of the most chosen attributes (ivancic, 2014). the pareto chart (figure 1) consists of a bar graph combined with a line chart of the factors related to a variable arranged according to the magnitude of the impact of these factors. a pareto chart is a histogram of data that sorts from the largest to the smallest frequencies and is also calculated cumulatively. figure 1 pareto chart the second step was the preparation of the analytic hierarchy process questionnaire in the form of a paired comparison matrix, and the third step was the calculation and analysis of the results of the criteria and sub-criteria and the determination of the global weights. global weight is obtained by multiplying the criteria and sub-criteria. the global weight with the highest value is the dominant or priority factor. 3.2. analytic hierarchy process (ahp) the analytic hierarchy process (ahp) is a decision making method used in multicriteria, multi-objective, and multi-factor environments (do et al., 2020). this method is used to solve complex problems with subjective criteria (gunduz & mohammad, 2020). the ahp is very flexible and useful (almodayan, 2018) in the analysis of qualitative data in a quantitative form (lee, lee, lee, kim, & kim, 2020). this method is also easy to use and only asks respondents to choose from pairwise comparisons (yang, vargas, jin, & zhou, 2020). numata et al. (2020) also state that this method is flexible. also, the mathematical procedure used is simple (ramírezrivera et al., 2020). the ahp method has been inspired by several previous discoveries and has been widely used by various interests in numerous fields (ishizaka & labib, 2011). the ahp begins by setting up the goals, and then determines the criteria, sub-criteria and alternatives which are then arranged in the form of a hierarchy (figure 1). ijahp article: raco, ohoitimur, krejci, raton, rottie, paseru, muajan, rachmadi/the dominant factor of lecturers’ research productivity using the ahp: case study of catholic university of de la salle manadoindonesia international journal of the analytic hierarchy process 552 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.819 figure 2 structure of the ahp by breaking down the problem in the form of a hierarchy, the researcher is able to unravel the complexity of the problem so that it is more easily solved. next, researchers compile a questionnaire in the form of a pairwise comparison. respondents only chose the criterion that is considered the most important of the two choices using a nominal scale between 1 to 9 determined by thomas saaty (table 1), so that human thoughts and feelings and their choices can be quantified. table 1 scale of preferences of ahp linguistic variables ahp scale equal importance 1 intermediate 2 moderately more important 3 intermediate 4 strongly more important 5 intermediate 6 very strongly more important 7 intermediate 8 extremely more important 9 ijahp article: raco, ohoitimur, krejci, raton, rottie, paseru, muajan, rachmadi/the dominant factor of lecturers’ research productivity using the ahp: case study of catholic university of de la salle manadoindonesia international journal of the analytic hierarchy process 553 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.819 making a choice from only two options is usually easier and more accurate (ishizaka & labib, 2011). the results obtained are arranged by rank which also determines the priority of choice. because humans are limited in their ability to make consistent choices, the ahp prepares a test of consistency for each choice. the calculation of consistency uses the following formula: 𝐶𝑅 = 𝐶𝐼 𝑅𝐼 (1) where: cr stand for consistency ratio; ci = consistency index; ri = random consistency index. table 2 random consistency index matrix size random consistency index (ri) 1 0.00 2 0.00 3 0.58 4 0.90 5 1.12 6 1.24 7 1.32 8 1.41 9 1.45 10 1.49 the results are considered consistent if the cr does not exceed 0.1. the ahp produces weights of each criterion according to the pairwise comparison of the decision makers. the greater the weight, the more important the relationship with the criteria is. comparisons are made between the criteria and each criterion with subcriteria. multiplication of the criteria and sub-criteria produces a global weight value. the ahp is a flexible and powerful tool because the scores are based on a pairwise matrix of preferences for both criteria and sub-criteria. researchers choose the questionnaire respondents by considering their experiences and comprehension of the issue. 4. results and discussion the results of the internal and external analysis preformed by the experts are shown in a hierarchy structure in figure 3. ijahp article: raco, ohoitimur, krejci, raton, rottie, paseru, muajan, rachmadi/the dominant factor of lecturers’ research productivity using the ahp: case study of catholic university of de la salle manadoindonesia international journal of the analytic hierarchy process 554 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.819 figure 3 hierarchical structure of ahp the factors were acceptable because they fulfilled the validity and reliability calculations. the results of the validity test (table 3) were carried out using spss 2.0 by comparing the r-test with the r-table. in this study, n = 90 since there were 90 questions in the ahp questionnaire. to obtain the r-table, n = df-2 or n = 90-2 = 88. (r-table = 0.2039 with 95% confidence level). if r-test > r-table, the research instrument or questionnaire is valid. the validity test for the questions was as follows: strength 0.624> 0.2039, weakness 0.647> 0.2039, opportunities 0.685> 0.2039 and threats 0.683> 0.2039. we concluded that all of the items for each of the sub-criteria swot analysis t : threats o : opportunities w : weakness s : strength s1 : having internal quality standard s2 : having lectures with master and doctoral qualifications s3 : lecturers’ commitment s4 : having accredited study programs s5 : has many young lectures s6 : involved in national and internasional scientific activities s7 : having lasallian spirituality w1 : limited internal research funding w2 : lack of research motivation qualifications w3 : too many teaching hours w4 : too many administrative duties w5 : inadequate infrastructure and supporting facilities w6 : few number of lecturers' publications w7 : lack of mastery of methodology and writing skills o1 : cooperation with government and private institutions o2 : networking with association of catholic higher institution o3 : government research grant is increasingly diverse o4 : cooperation with foreign institutions o5 :network of faculties and study programs with professional associations o6 : government scholarship requirements are high and competition increases o7 : support from churches and university foundation t1 : competition with other universities is getting higher t2 : the requirements for submitting a research proposal are too complex t3 : lack of public interest in reading research results t4 : lecturer creativity is hampered by regulations t5 : the high interest of lecturers to work in state university t6 : government scholarship requirements are high and competition increases t7 : research grants from international institutions are few goal critera subcritera ijahp article: raco, ohoitimur, krejci, raton, rottie, paseru, muajan, rachmadi/the dominant factor of lecturers’ research productivity using the ahp: case study of catholic university of de la salle manadoindonesia international journal of the analytic hierarchy process 555 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.819 were valid. the pairwise comparison criteria was 0.549> 0.2039, therefore they were valid. questionnaire reliability testing was conducted to determine how reliable the instrument was at measuring the objectives. the test was carried out using spss 2.0 with the calculation of the cronbach alpha coefficient. the results of the cronbach alpha for the sub-criteria strength, weaknesses, opportunities and threats were 0.939, 0.945, 0.959, 0.974, respectively. the swot criteria comparison questionnaire was 0.785. these results show that all of the questionnaires were reliable or can consistently be used as a good measuring tool. the results of the consistency calculation showed that both of the the criteria and the sub-criteria are consistent because cr ˂ 0.1 (tables 3-7). table 3 pairwise comparison matrix, priority weight and consistency ratio of criteria s w o t priority weight s 1.000 5.623 1.323 3.557 0.459 w 0.178 1.000 0.416 1.270 0.113 o 0.756 2.406 1.000 3.767 0.328 t 0.281 0.788 0.265 1.000 0.100 𝜆 max = 4,061, ci = 0,020, cr = 0,023 table 4 pairwise comparison matrix, priority weight and consistency ratio of strength subcriterion s1 s2 s3 s4 s5 s6 s7 local priority weight s1 1.000 1.082 1.178 1.764 4.207 1.622 0.912 0.188 s2 0.924 1.000 0.499 1.689 2.928 0.601 0.673 0.128 s3 0.849 2.005 1.000 2.922 4.491 1.513 1.443 0.223 s4 0.567 0.592 0.342 1.000 4.870 0.866 0.944 0.120 s5 0.238 0.342 0.223 0.205 1.000 0.318 0.337 0.043 s6 0.617 1.665 0.661 1.154 3.150 1.000 1.074 0.147 s7 1.096 1.486 0.693 1.059 2.967 0.931 1.000 0.152 𝜆 max = 7,204, ci = 0,034, cr = 0,026 explanation: s1: having internal quality standard, s2: having lecturers with master and doctoral qualifications, s3: lecturer’s commitment, s4: having accredited study programs, s5: has many young lectures, s6: involved in national and international scientific activities, s7: having lasallian spirituality; ijahp article: raco, ohoitimur, krejci, raton, rottie, paseru, muajan, rachmadi/the dominant factor of lecturers’ research productivity using the ahp: case study of catholic university of de la salle manadoindonesia international journal of the analytic hierarchy process 556 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.819 table 5 pairwise comparison matrix, priority weight and consistency ratio of weakness subcriterion w1 w2 w3 w4 w5 w6 w7 local priority weight w1 1.000 2.434 1.344 1.131 1.400 1.949 0.995 0.194 w2 0.411 1.000 0.713 0.607 1.174 1.075 1.374 0.123 w3 0.744 1.403 1.000 1.099 0.746 1.430 0.967 0.142 w4 0.884 1.649 0.910 1.000 0.916 1.052 0.634 0.136 w5 0.714 0.852 1.340 1.091 1.000 1.037 0.669 0.131 w6 0.513 0.930 0.699 0.950 0.964 1.000 0.774 0.112 w7 1.005 0.728 1.034 1.578 1.496 1.292 1.000 0.161 𝜆 max = 7,179, ci = 0,030, cr = 0,023 explanation: w1: limited internal research funding, w2: lack of research motivation, w3: too many teaching hours, w4: too many administrative duties, w5: inadequate infrastructure and supporting facilities, w6: few number of lecturers' publications, w7: lack of mastery of methodology and writing skills table 6 pairwise comparison matrix, priority weight and consistency ratio of opportunity subcriterion o1 o2 o3 o4 o5 o6 o7 local priority weight o1 1.000 0.933 0.797 1.167 1.831 0.720 1.026 0.143 o2 1.072 1.000 2.920 2.497 1.663 1.991 1.185 0.231 o3 1.254 0.342 1.000 1.196 1.911 1.321 1.149 0.148 o4 0.857 0.400 0.836 1.000 1.000 1.365 1.347 0.126 o5 0.546 0.601 0.523 1.000 1.000 0.978 1.000 0.107 o6 1.389 0.502 0.757 0.733 1.023 1.000 1.108 0.123 o7 0.975 0.844 0.870 0.743 1.000 0.902 1.000 0.124 𝜆 max = 7,242, ci = 0,040, cr = 0,031 explanation: o1: cooperation with government and private institutions, o2: networking with association of catholic higher institution, o3: government research grant is increasingly diverse, o4: cooperation with foreign institutions, o5: network of faculties and study programs with professional associations, o6: research grant from international institutions, o7: support from churches and university foundation ijahp article: raco, ohoitimur, krejci, raton, rottie, paseru, muajan, rachmadi/the dominant factor of lecturers’ research productivity using the ahp: case study of catholic university of de la salle manadoindonesia international journal of the analytic hierarchy process 557 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.819 table 7 pairwise comparison matrices, priority weights and consistency ratio of threats subcriterion t1 t2 t3 t4 t5 t6 t7 local priority weight t1 1.000 1.801 1.476 1.466 1.340 1.611 1.496 0.197 t2 0.555 1.000 3.132 2.571 2.611 1.880 2.514 0.238 t3 0.678 0.319 1.000 0.985 0.584 0.586 0.550 0.086 t4 0.682 0.389 1.016 1.000 2.126 1.228 1.193 0.132 t5 0.746 0.383 1.711 0.470 1.000 0.701 1.215 0.107 t6 0.621 0.532 1.707 0.815 1.427 1.000 1.750 0.134 t7 0.669 0.398 1.817 0.838 0.823 0.571 1.000 0.105 𝜆 max = 7,304, ci = 0,051, cr = 0,038 explanation: t1: competition with other universities is getting higher, t2: the requirements for submitting a research proposal are too complex, t3: lack of public interest in reading research results, t4: lecturer creativity is hampered by regulations, t5: the high interest of lecturers to work in a state university, t6: government scholarship requirements are high and competition increases, t7: research grants from international institutions are few the results of the global weight calculation (table 8) showed that strength (45.9%) is the most dominant factor. the second is opportunity (32.8%) and the third is weakness (11.3%) and the last is a threat (10%). ijahp article: raco, ohoitimur, krejci, raton, rottie, paseru, muajan, rachmadi/the dominant factor of lecturers’ research productivity using the ahp: case study of catholic university of de la salle manadoindonesia international journal of the analytic hierarchy process 558 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.819 table 8 global weights of sub-criteria weight of swot factors weight of swot_subfactors local global a b c=a*b strengths 0.459 s1 0.188 0.086 s2 0.128 0.059 s3 0.223 0.102 s4 0.120 0.055 s5 0.043 0.020 s6 0.147 0.067 s7 0.152 0.070 weaknesses 0.113 w1 0.194 0.022 w2 0.123 0.014 w3 0.142 0.016 w4 0.136 0.015 w5 0.131 0.015 w6 0.112 0.013 w7 0.161 0.018 a b c=a*b opportunities 0.328 o1 0.143 0.047 o2 0.231 0.076 o3 0.148 0.048 o4 0.126 0.041 o5 0.107 0.035 o6 0.123 0.040 o7 0.124 0.041 threats 0.100 t1 0.197 0.020 t2 0.238 0.024 t3 0.086 0.009 t4 0.132 0.013 t5 0.107 0.011 t6 0.134 0.013 t7 0.105 0.010 sum 1 1 ijahp article: raco, ohoitimur, krejci, raton, rottie, paseru, muajan, rachmadi/the dominant factor of lecturers’ research productivity using the ahp: case study of catholic university of de la salle manadoindonesia international journal of the analytic hierarchy process 559 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.819 figure 4 global weight the data above showed that commitment (s3) is the dominant factor of lecturers’ research productivity (10.2%), followed by s1 (8.6%), o2 (7.6%), s7 (7%), s6 (6.7%). although they have a small value, the analysis also shows that there are weaknesses at the university, namely w1 (2.2%), w7 (1.8%), w3 (1.6%). the commitment of lecturers to their universities is in accordance with research conducted by batugal et al. (2019).their study showed that the lecturer’s commitment was related to their belief in the values, vision and mission of the university. this commitment helps them survive and not desire to leave the university. this commitment also shows their willingness to continue pursuing their careers at their university. madrigal et al. (2018) also confirmed the role of the lecturer’s commitment to continue working at the catholic university. the lecturer’s compatibility with the vision and mission of the university keeps them committed. also, they may think that it would be very expensive to leave the catholic university and that there are no other institutions that are more suitable (madrigal, iracion, & temporosa, 2018). selvanathan et al. (2019) confirmed the normative commitment factor which explains why lecturers continue to work and be productive for an organization because it is viewed as the right thing to do and as a moral norm. they distinguished between affective commitment, continuance and normative commitment (selvanathan, supramaniam, shern, suppramaniam, & muhammad, 2019). budiyono (2019) also confirmed the role of organizational commitment to lecturer productivity. these results highlight the importance of university management paying attention to human resources to maintain and increase the commitment of the lecturers (budiyono, 2019). the results of this study have an impact on management as they determine future programs. in order to maintain or increase the commitment of lecturers, management must focus on their commitment factor. the second goal should be the strengthening of the application of research quality standards. this internal strength will help the institution fortify cooperation with the association of catholic higher education and ijahp article: raco, ohoitimur, krejci, raton, rottie, paseru, muajan, rachmadi/the dominant factor of lecturers’ research productivity using the ahp: case study of catholic university of de la salle manadoindonesia international journal of the analytic hierarchy process 560 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.819 other la salle universities which could be the third goal. the fourth goal is to intensify the lasallian spirituality. the dominant internal strength and opportuntities possessed by the university become a strong basis for the development of the institution going forward. 5. conclusions, limitations and recommendations research has received important attention at the catholic university because through research, scientists have helped the church find the truth, communicate it within various cultural realities and discover new ways of evangelizing in the modern world. research also plays an important role in determining the rank and performance of the university. evaluating the research productivity of lecturers is important for the future development of the university and will improve the quality of teaching. the purpose of this study was to find and determine the dominant factor of lecturer research productivity. many objective and subjective factors were involved in determining the productivity of lecturers. lecturer productivity is a sensitive issue in universities and includes multi-complex factors. identifying internal and external factors is the first step to understanding the productivity of lecturers’ research. the use of the analytic hierarchy process method is very suitable for determining dominant factors in multi-complex decision making. the results showed that lecturer commitment is the dominant factor, followed by research quality standards. management must allocate internal funds to develop lecturer research productivity. this research reinforced the accuracy of the the ahp method, and succeeded in bridging the gap in determining the dominant factors and providing a framework for studying lecturers’ research productivity. this research successfully solved the complex issue of lecturers' research productivity and will serve as a new reference in analyzing its multi-complex attributes. a limitation of this study is that the respondents were limited to a single university, universitas katolik de la salle. the researchers recommend further research involving respondents from other de la salle universities in several countries and also expanding the research to public universities. ijahp article: raco, ohoitimur, krejci, raton, rottie, paseru, muajan, rachmadi/the dominant factor of lecturers’ research productivity using the ahp: case study of catholic university of de la salle manadoindonesia international journal of the analytic hierarchy process 561 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.819 references abramo, g., & d’angelo, c. (2014). how do you define and measure research productivity? scientometrics. scientometrics, 101, 1130–1144. doi: https://doi.org/10.1007/s11192-014-1269-8 almodayan, a. (2018). analytical hierarchy ( ahp ) process method for environmental hazard mapping for jeddah city , saudi arabia. journal of geoscience and environment protection, 6, 143–159. doi: https://doi.org/10.4236/gep.2018.66011 ates, n., çevik, s., kahraman, c., gülbay, m., & erdogan, s. (2006). multi attribute performance evaluation using a hierarchical fuzzy topsis method, studfuzz, 201, 537-572. auranen, o., & nieminen, m. (2010). university research funding and publication performance—an international comparison. research policy, 39(6), 822–834. doi: https://doi.org/10.1016/j.respol.2010.03.003 bahurmoz, a. m. a. (2006). the analytic hierarchy process : a methodology for win-win management. journal of king abdulaziz university : economics and administration, 20(1), 3–16. doi: https://doi.org/10.4197/eco.20-1.1 batugal, m. l. c., & tindowen, d. j. c. (2019). influence of organizational culture on teachers ’ organizational commitment and job satisfaction : the case of catholic higher education institutions in the philippines. universal journal of educational research, 7(11), 2432–2443. doi: https://doi.org/10.13189/ujer.2019.071121 bozanic, d., pamucar, d., & bojanic, d. (2015). modification of the analytic hierarachy process (ahp) method using fuzzy logic: fuzzy ahp approach as a support to the decision making process concerning engagement of the group for additional hindering. serbian journal of management, 10(2), 151–171. doi: https://doi.org/10.5937/sjm10-7223 budiyono. (2019). motivation of work, organizational culture, and organizational commitment towards work produtivity lecturers and employees of economics college aas surakarta ( stie aas surakarta ). humanities & social sciences reviews, 7(5), 226–236. doi: https://doi.org/10.18510/hssr.2019.7528 costa, c. a., & oliveira, m. (2012). a multicriteria decision analysis model for faculty evaluation. omega, 40, 424–436. doi: https://doi.org/10.1016/j.omega.2011.08.006 dirjen dikti. (2019). guidlines of research and community services. (direktorat riset dan pengabdian masyarakat, ed.) (xii). direktorat riset dan pengabdian masyarakat. do, a. d., pham, m. t., dinh, t. h., ngo, t. c., luu, q. d., pham, n. t., … vuong, h. n. (2020). evaluation of lecturers’ performance using a novel hierarchical multicriteria model based on an interval complex neutrosophic set. decision science letters, 9, 119–144. doi: https://doi.org/10.5267/j.dsl.2020.1.003 garrett, m. (2006). the identity of american catholic higher education: a historical ijahp article: raco, ohoitimur, krejci, raton, rottie, paseru, muajan, rachmadi/the dominant factor of lecturers’ research productivity using the ahp: case study of catholic university of de la salle manadoindonesia international journal of the analytic hierarchy process 562 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.819 overview. catholic education: a journal of inquiry and practice, 10(2), 229–247. doi: https://doi.org/10.15365/joce.1002082013 george, f. (2000). ex corde ecclesiae : promises and challenges, journal of catholic education, 4(2), 239-243. doi: 10.15365/joce.0402082013 grace, g. (2016). vatican ii and new thinking about catholic education: aggiornamento thinking and principles into practice. london: centre for research and development in catholic education. gunduz, m., & mohammad, k. o. (2020). assessment of change order impact factors on construction project performance using analytic hierarcy process (ahp), 26(1), 71–85. doi: https://doi.org/10.3846/tede.2019.11262 henry, c., ghani, n. a., hamid, u., & bakar, a. n. (2020). factors contributing towards research productivity in higher education. international journal of evaluation and research in education, 9(1), 203-211. doi: https://doi.org/10.11591/ijere.v9i1.20420 iozzio, m. j. (2000). university as church: fides et ratio as a source for ecclesiology. cathlolic education: a journal of inquiry and practice, 4(2), 218–227. doi: https://doi.org/10.15365/joce.0402062013 ishizaka, a., & labib, a. (2011). review of the main developments in the analytic hierarchy process. expert systems with applications, 38(11), 14336–14345. doi: https://doi.org/10.1016/j.eswa.2011.04.143 ivancic, v. (2014). improving the decision making process through the pareto principle application, economic thought and practice, 2, 633–656. jaramillo, i. f., pico, r. b., marquez, c. v, & plata, d. la. (2017). a model for faculty evaluation in higher education ecuadorian through multi-criteria decision analysis. indian journal of science and technology, 10(18), 1–8. doil https://doi.org/10.17485/ijst/2017/v10i18/110443 john paul ii, p. apostolic constitution of the supreme pontiff john paul ii on catholic universities (1990). vatican. karmaker, c., rahman, s., tahiduzzaman, m., biswas, t., rahman, m., & biiswas, s. (2018). a framework of faculty performance evaluation : a case study in bangladesh. international journal of research in advanced engineering and technology, 5(3), 18–24. lee, d., lee, d., lee, m., kim, m., & kim, t. (2020). analytic hierarchy processbased construction material selection for performance improvement of building construction : the case of a concrete system form. materials, 13, 1–22. doi: https://doi.org/10.3390/ma13071738 madrigal, d., iracion, e., & temporosa, m. a. (2018). workplace core values and organizational commitment of basic education teachers in a philippine catholic university. journal of institutional research south east asia, 16(1), 26–45. ijahp article: raco, ohoitimur, krejci, raton, rottie, paseru, muajan, rachmadi/the dominant factor of lecturers’ research productivity using the ahp: case study of catholic university of de la salle manadoindonesia international journal of the analytic hierarchy process 563 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.819 miller, g. a. (1956). the magical number seven, plus or minus two: some limits on our capacity for processing information. psychological review, 63(2), 81–97. doi: https://doi.org/10.1037/h0043158 numata, m., sugiyama, m., & mogi, g. (2020). barrier analysis for the deployment of renewable based mini-grids in myanmar using the analytical hierarchy process. energies, 13(6), 1400. doi: https://doi.org/10.3390/en13061400 pal, j. k., & sarkar, s. (2020). evaluation of institutional research productivity. journal of library & information technology, 40(1), 58–69. doi: https://doi.org/10.14429/djlit.40.1.14804 paul vi, p. declaration on christian education gravissimum educationis prclaimed by his holiness pope paul vi (1965). vatican. pope francis. apostolic constitution veritatis gaudium on ecclesiastical universities and faculties (2017). vatican. ramírez-rivera, e. d. j., morales-carrera, u. a., castillo-martínez, s., hernandezcarredo, g., ramirez-sucre, m. o., & herrera-corredor, j. a. (2020). analytic hierarchy process as an alternative for the selection of vocabularies for sensory characterization and consumer preference. journal of sensory study, 35(1), 1–11. doi: https://doi.org/10.1111/joss.12547 saaty, t. (2016). seven is the magic number in nature.proceedings of the american philosophical society, 160(4), 335-360. selvanathan, m., supramaniam, m., shern, y. ter, suppramaniam, s., & muhammad, a. m. (2019). the impact of resistance to change among lecturers in private university. university journal of educational research, 7(12), 2568–2572. doi: https://doi.org/10.13189/ujer.2019.071203 shin, j., arimoto, a., cummings, w., & teichler, u. (2014). teaching and research in contemporary higher education. new york: springer. doi: https://doi.org/10.1007/978-94-007-6830-7 silungwe, c., & khatleli, n. (2018). an analysis of the allocation of pertinent risks in the zambian building sector using pareto analysis. international journal of construction management, 20(4), 321–334. doi: https://doi.org/10.1080/15623599.2018.1484853 simboli, b. (2012). ex corde ecclesiae , social science, and the public square. the catholic social science review, 17, 323–336. suhardi, m., fuad, n., & rosyidi, u. (2019). strategy in improving research productivity of the lecturers. advances in social science, education and humanities research, 295, 123–126. doi: https://doi.org/10.2991/icetep-18.2019.30\ tuan, n. a., hue, t. t., lien, l. t., thao, t. d., quyet, n. d., van, l. h., & anh, l. t. (2020). a new integrated mcdm approach for lecturers’ research productivity evaluation. decision science letters, 9, 355–364. doi: https://doi.org/10.5267/j.dsl.2020.5.001 https://psycnet.apa.org/doi/10.1037/h0043158 ijahp article: raco, ohoitimur, krejci, raton, rottie, paseru, muajan, rachmadi/the dominant factor of lecturers’ research productivity using the ahp: case study of catholic university of de la salle manadoindonesia international journal of the analytic hierarchy process 564 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.819 warner, k., & caudill, d. (2013). science, technology, and catholic identity in the education of professionals. catholic education: a journal of inquiry and practice, 16(2), 237–263. doi: https://doi.org/10.15365/joce.1602022013 yang, j., vargas, l., jin, x., & zhou, l. (2020). quality credit evaluation in the internet company: a system based on the analytic hierarchy process. journal of business economics and management, 21(2), 344–372. doi: https://doi.org/10.3846/jbem.2020.11409 ijahp article: kozak, aksoz, özel/ an analytic hierarchy process model for understanding convention planners’ prior factors of convention hotel selection international journal of the analytic hierarchy process 256 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.283 an analytic hierarchy process (ahp) model for understanding convention planners’ prior factors of convention hotel selection * meryem akoğlan kozak faculty of business administration anadolu university eskişehir, turkey e-mail: mkozak@anadolu.edu.tr emre ozan aksoz faculty of tourism anadolu university eskişehir, turkey e-mail: ozana@anadolu.edu.tr çağıl hale özel ** faculty of tourism anadolu university eskişehir, turkey e-mail: chkayar@anadolu.edu.tr abstract in the process of organizing a convention, owners of conventions forward their requests to convention planners and leave the full organization of the event to convention planners. therefore, convention planners perform all the purchases for the organization of a convention. selecting the suitable convention hotel is a critical step for a successful convention, and for that reason, convention hotels should understand convention planners’ preferences in order to have success in convention tourism. the aim of this study is therefore to determine the primary convention hotel selection criteria of convention planners operating in ankara. the delphi technique was used for the development of a data collection form. data were gathered by conducting face to face interviews with managers from 13 travel agents in april, 2014. the analytic hierarchy process (ahp) was employed in the analysis of data. findings indicate that facilities of the convention hotel, price, accessibility, site environment, and local support were clarified as the primary factors, respectively. managerial implications for convention hotel managements are also discussed. keywords: convention planners; travel agencies; convention hotel selection criteria; analytic hierarchy process; ankara-turkey. * this study was supported by anadolu university scientific research projects commission under the grant no:1401e004 ** corresponding author. three authors contributed an equal amount. ijahp article: kozak, aksoz, özel/ an analytic hierarchy process model for understanding convention planners’ prior factors of convention hotel selection international journal of the analytic hierarchy process 257 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.283 1. introduction “convention” is defined as a "meeting of an invited group of domestic or nondomestic people with the same or different professions, who come together to exchange ideas and discuss (eryılmaz, 2012; emeksiz, 2010; crouch and ritchie, 1998). the word “convention” is also used for regular meetings (baloğlu & love, 2001). convention tourism helps promote tourism throughout the whole year at a destination, and for that reason it has become one of the most effective types of tourism in increasing the efficiency and productivity of both tourism establishments and the national economy (crouch & richie, 1998). the participants of convention tourism usually prefer to visit different convention destinations and stay at totally new convention hotels and centers (crouch and ritchie, 1998; crouch and weber, 2002). this means that the market of convention centers and hotels, together with the convention destinations is growing more day by day. the term “convention”, used in the tourism sector, can be divided into three groups. these are called, national conferences, international conferences and international congresses (icca 2011). international congresses take place in the host country and the convention owner or a representative of the owner is present throughout the congress. the participants could be from the host country or from outside the country. at international congresses, both the owner and participants are from outside the host country. frequently, this type of convention is the general assembly of professional associations, which take place in the host country (ünsever, 2008). generally, international conventions are organized in larger conference centers and the preparation requires multiple stages. at national conventions the owner of the convention is from the host country and all the participants are from the same country as well. aksu, yılmaz and gümüş (2013) put forward another type of convention called virtual conventions. according to the authors, virtual conventions can be defined as a convention which is supported by a software program and allows participants to exchange information without any visa, or location and time limitations. some other small-sized meetings which do not technically have the nature of conventions but carry the same features as meetings are also classified under the heading of convention. these kinds of small-sized conventions also expand the volume of convention tourism market. according to international congress and convention organization’s (icca) figures, there has been 11,556 conventions organized around the world in 2012 (icca, 2011). the united states of america, germany and spain took the first three places in the number of conventions organized. turkey was 21st with 179 international conventions organized in 2012. city-based evaluation demonstrated that istanbul took the 9 th place with 128 conventions, antalya was the 109 th with 21 conventions, and ankara was 365 th with five conventions. other leading cities in terms of number of conventions organized are as follows: vienna (195), paris (181) and berlin (172) in 2012. the organization of a convention follows some specific steps. this process starts with the decision to organize a convention. secondly, the owner of the convention charges the convention planner or cooperates with a local convention agent at the convention site. after that, the convention planner that has been charged to organize the convention performs the required purchases, makes the contracts with sponsors and offers the convention as a package product to the participants. based on this explanation, the owner and the planner of the convention are the two main actors in organizing a convention. these two actor’s positions differ in terms of their ijahp article: kozak, aksoz, özel/ an analytic hierarchy process model for understanding convention planners’ prior factors of convention hotel selection international journal of the analytic hierarchy process 258 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.283 qualifications and responsibilities. the convention owner decides to organize the convention. this is a bit different in international conventions where the agent of the same organization in another country shares this title. the convention planner provides the “convention organization service” in the technical sense to the owner (icca, 2011). another critical actor playing a role in convention organization is the convention hotel. convention hotels are special purpose hotel establishments which provide meeting rooms equipped with the necessary equipment for different types of meetings (seminars, workshops, etc..) as well as accommodation services (aksu, yılmaz and gümüş, 2013). as mentioned above, owners of conventions forward their requests to convention planners and leave the full organization of the event to convention planners. convention planners perform all the purchases for the organization of a convention. for that reason, there is tough competition among convention hotels, convention centers and convention destinations to be preferred by the convention planners. that is why being successful in convention tourism requires understanding convention planners’ preferences and doing regular research on the subject. the aim of this study is therefore to determine the primary criteria of convention hotel selection of convention planners operating in ankara. with the help of the ahp model used in this study, it will be possible to understand which features of convention hotels impress convention planners more than others and what other criteria affect a convention planners’ decision about convention hotel selection. besides, the findings of this study will also shed light on how to make the core product of convention hotels compatible with the demands of convention planners. in other words, the findings of this study are expected to contribute to the alignment of supply and demand. the study is arranged as a descriptive study which compiled previous studies in the literature and determined the opinions of convention planners in terms of predefined factors and attributes. 2. literature review there has been much research done about the hotel and site selection criteria of conventions parallel to the recent developments in convention tourism. crouch and ritchie (1997) performed one of the pioneering studies on determining the site selection criteria of conventions. in this study, various parties in convention organization (convention owners, convention planners, business firms and participants etc.) were targeted and were asked about the factors to consider in convention site selection. analysis of the collected data identified eight important criteria. these criteria were listed as accessibility, local support, extra-conference opportunities, accommodation facilities, meeting facilities, information, location of the environment and other criteria. this study was later referred to by other scientific research that was done to determine convention site selection criteria. opperman and chon (1997) divided the site selection criteria of convention planners into two groups. these are organizational criteria and site-specific criteria. among the former is the scope of the association (i.e. regional, national or international) which may predetermine the range of convention locations available to the organizers. another factor is the influence of associations’ ceos or presidents, convention planners and other association members. on the other hand, site-specific criteria include conference and accommodation facilities, accessibility, price levels, ijahp article: kozak, aksoz, özel/ an analytic hierarchy process model for understanding convention planners’ prior factors of convention hotel selection international journal of the analytic hierarchy process 259 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.283 service quality, destination image, safety/security, recreational and entertainment facilities, climate, and dining facilities. in the beginning of the 2000s, some studies focused on the evaluation and scoring of selected destinations according to some specific criteria of convention organization. for instance, chacko and fenich (2000) tried to score selected destinations in terms of availability for convention tourism. in this study, seven states of the usa were evaluated in terms of convention site selection criteria. tweleve criteria were determined by analyzing the content of previous studies in the literature. these criteria are hotel room availability, hotel room rates, meeting space availability, the cost of food and beverage, availability of air service, the cost of air service, convenience of local transportation, promotional appeal of the city, destination services, and safety of attendees, friendliness of local people and helpfulness of service people. seven states were scored according to these criteria and three of them were found to be superior to others in terms of all criteria. also, the promotional appeal of the city was found to be the best determiner of being an attractive convention destination. baloğlu and love (2001) focused on 23 criteria for association meeting planners’ perceptions and tried to determine the performance of five major convention cities in the usa with regard to these criteria. some of the criteria are as follows: capacity of meeting facilities, quality of meeting facilities, safety and security of destination, accessibility of destination, hotel room capacity, city’s reputation and climate. as a result of interviews with 16 planners, more and less significant criteria were determined together with the best performing city. moser (2003) examined the change in convention planners’ and participants’ expectations of site selection. moser tried to determine the leading criteria used in convention planners’ and participants’ decision making process and also tried to specify the methods used in ordering these criteria. criteria used in the studies of crouch and weber (2002), crouch and ritchie (1998), gess (1975) and oppermann and chon (1997) provided guidance in gathering the site selection criteria. some of these criteria are accessibility, local support, extra conference opportunities, and accommodation facilities, meeting facilities, information, site environment and other criteria. findings of the study showed that accessibility was found to be the most significant site selection criteria and a face-to-face interview was found to be the most effective method for determining the relative significance of various criteria. in another study getz (2004) classified the convention site selection criteria mentioned in previous studies under 10 headings. these headings are accessibility, supply of event venues, accommodations, amenities, image as a desirable place to visit, reputation for hosting successful events, safety and comfort of visitors, support services to events, number and type of local organizations and business and cost. chen (2006) conducted a study in which the aim was to propose an analytical hierarchy process (ahp) model to construct an evaluation structure with criteria and associated weights of convention site selection for meeting planners. as a result of a detailed literature review, five main criteria and 17 sub-criteria were identified for convention site selection and a data collection form was developed. the main criteria were meeting and accommodation facilities, costs, site environment, local support and extra conference opportunities. a three-level hierarchical structure was performed according to the responses of 35 academic-related association directors who hosted international conventions in 2003 in taiwan. according to the findings of ijahp article: kozak, aksoz, özel/ an analytic hierarchy process model for understanding convention planners’ prior factors of convention hotel selection international journal of the analytic hierarchy process 260 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.283 the study, meeting and accommodation facilities and site environment were found to be much more significant compared to other criteria. in a study conducted by amiri, zandieh, vahdani, yazdani and soltani (2008), the authors tried to prove the benefits of using a hybrid model in decision-making and classified the convention site selection criteria under five main headings which are meeting and accommodation facilities, cost, site environment, local support and extra conference opportunities. various sub-criteria were specified for each group of main criteria. while these previous studies identified site selection criteria, some other studies tried to analyze the differences between convention planners’ perceptions of convention site selection criteria in terms of the significance they attributed to these criteria. dipietro, breiter, rompf and godlewska (2008) examined whether there was a significant difference between perceptions of members of three event planning associations regarding destination selection criteria. the criteria used in this research were selected based on past research done on the topic of destination selection criteria. differences between the responses of 209 planners were analyzed. while some criteria were rated at different significance by convention planners; perceived value for money, overall cost, support services for events, reputation for hosting successful events, and desirable destination image were found to be important by all respondent groups. there have also been some studies carried out in turkey. for instance, dölalan (2008) aimed to evaluate ankara as a convention destination and determine the position of ankara in terms of convention tourism. perceptions of 98 officials from several travel agencies were measured in this study. findings showed that development of convention tourism in ankara is not at an adequate level. besides, ankara does not have the required amount of convention and accommodation facilities, urban transportation facilities and sufficient coordination between related foundations. in a dissertation study, boz (2010) tried to determine which hotel selection criteria travel agencies in istanbul take into account when they plan conventions. 44 officials were surveyed via questionnaires and were asked to rate 28 criteria with regard to the importance they attributed to each. results from this study revealed that price, star classification, food and beverage quality, convention center, housekeeping services, on time food and beverage services, small and adaptable meeting rooms, and qualified hotel staff were found to be among the most important criteria. while several studies highlighted the broad groups of selection criteria, subsequent studies have focused on more detailed sub-criteria. for instance, in a study conducted by nelson and rys (2008), convention site selection preferences of 1272 association executives were surveyed. 31 detailed convention site selection criteria were listed in terms of their importance, some of which are as follows: cooperative convention staff, competitive rates for exhibit space, hotel rooms within walking distance, affordability of city, convenient ground transport, friendliness of locals, airport-direct flights, reputation of restaurants, good shopping opportunities, availability of quality golf course, quality/variety of night-life, quality museums, availability of water sports and availability of casino gambling. likewise, chiappa (2012) focused on australian convention planners and asked the respondents to assess the importance they give to each of the 22 site-specific attributes when selecting a site. for example, chiappa addressed accommodation under six different sub-criteria, which include accommodation range, accommodation rates, accommodation facilities, accommodation location relative to airport, on-site/off site accommodations and ijahp article: kozak, aksoz, özel/ an analytic hierarchy process model for understanding convention planners’ prior factors of convention hotel selection international journal of the analytic hierarchy process 261 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.283 number of hotels within walking distance to convention center. expected weather, convention and visitor bureau’s assistance and quality and efficiency of industry personnel were some other noticeable criteria used in this study. recent studies have also tried to make a collective assessment of studies about convention site selection criteria. this may be taken as an indicator of satisfaction of literature in terms of knowledge on convention site selection. for instance, elston and draper (2012) aimed to review empirical studies, conducted between the years 19902012, regarding the site selection process of meeting planners. the study also aimed to find out the relatively more and less important criteria of convention site selection. the results revealed that since 1990, cost of hotel rooms, meeting space, food and beverage, and other costs have consistently been important criteria. on the other hand, the most common least important criteria in the series of articles appeared to be the provision of entertainment (e.g., resort activities, nightlife, museums). likewise, mair (2012) made a content analysis of studies conducted between the years of 20002009 on the topics of meetings, conventions, incentive travel, trade fairs and exhibitions. one of the themes determined in this study is site selection. findings of the study revealed that a considerable amount of research in the period from 2000 to 2009 focused on the evaluation of satisfaction by meeting planners, the role of destination image in convention attendance, and the decision-making process of convention participants. also, mair (2012) emphasized that the site selection process of conference and convention organizers emerged as important research topics among the literature of business events. in summary, literature on the site-selection process demonstrated that a large amount of research has been conducted on event planners and site selection crieria. convention and meeting planners have become the key actors of the convention tourism market and their demands and requirements should be listened to carefully. there is nevertheless a relatively neglected area of tourism research as no scientific study has encompassed the hotel selection criteria of specialist convention planners operating in ankara, turkey, who are organizing both national and international conventions all over the world. however, this information may be of importance in terms of shedding light on convention hotel selection criteria that convention planners take into account while planning international conventions. the aim of this present study is therefore to determine the importance rank order of convention hotel selection criteria of national and international convention planners operating in ankara, turkey. the reasons for conducting this study in ankara, the capital city of turkey, are twofold. first, ankara is the center of national and international political traffic in turkey. secondly, ankara has a huge potential for convention tourism. the presence of the turkish grand national assembly in ankara, and the presence of the ministries, public institutions, banks, educational and medical institutions, foundations, rooms, associations, and political parties all contribute to increase the demand for convention organization. also, national and international conventions, seminars, conferences, workshops, plenaries and scientific meetings organized by these institutions provide a potential demand for convention tourism (dölalan, 2008). on the supply side, it is also noticeable that the total capacity of meeting rooms found in certified tourism businesses and public and civil society organizations in ankara exceeds 60,000. ankara is also convenient for sufficiently meeting the accommodation requirements of convention participants. as of the end of 2013, the total number of ministry and municipality certified hotels in ankara had reached 328. ijahp article: kozak, aksoz, özel/ an analytic hierarchy process model for understanding convention planners’ prior factors of convention hotel selection international journal of the analytic hierarchy process 262 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.283 the total bed capacity of those hotels is up to 30,870 (provincial culture and tourism directorate in ankara, 2014). 3. research design the aim of this study is to determine the importance rank order of convention hotel selection criteria of national and international convention planners operating in ankara, turkey. thus, it will be easier to understand which criteria affect convention planners’ final decision about convention hotel selection more than others. besides, ranking these criteria with regard to their importance level will give convention hotels some clues on how to align their product presentation with the demands of convention planners. 3.1. development of data collection form the delphi technique was used for the development of a data collection form. the delphi technique is a widely used and accepted method for gathering data from respondents within their domain of expertise. the technique is designed as a group communication process which aims to achieve a convergence of opinion on a specific real-world issue. the delphi process has been used in various fields of study such as program planning, needs assessment, policy determination, and resource utilization to develop a full range of alternatives, explore or expose underlying assumptions, as well as correlate judgments on a topic spanning a wide range of disciplines. the delphi technique is well suited as a method for consensus-building by using a series of questionnaires delivered using multiple iterations to collect data from a panel of selected subjects. subject selection, time frames for conducting and completing a study, the possibility of low response rates, and unintentionally guiding feedback from the respondent group are areas which should be considered when designing and implementing a delphi study (hsu and sandford, 2007). first, factors and criteria used in previous studies about convention hotel selection criteria of convention planners were derived after a thorough examination of the literature. then, experts were asked to associate those predefined criteria for inclusion in each of the factors. the factors and criteria used in this study were derived from the studies of crouch and ritchie (1997), opperman and chon (1997), clark, evans and knutson (1998), chacko and fenich (2000), upchurch, jeong, clements and jung (2000), baloğlu and love (2001), crouch and louviere (2004), getz (2004), chen (2006), fawzy and samra (2008), amiri, zandieh, vahdani, yazdani and soltani (2008), dipietro, breiter, rompf, godlewska (2008), nelson and rys (2000), tsai and ho (2009), jeon and kim (2011) and chiappa (2012). the delphi technique was performed in three stages and in each of these stages 15 experts were employed. the experts consulted in this study included five academicians, five hotel managers and a total of five other experts from the fields of public relations, food and beverage, travel agency and event management. the first round of the delphi technique was started with eight factors and 41 criteria. after the completion of this first round, some criteria on which experts could not reach consensus and those which were regarded as unimportant or were not fully understood were excluded from the list. besides, it was noticed that some factors were associated with none of the criteria and thus the number of the factors was also reduced. at the beginning of the second round of the delphi technique, five factors and 34 criteria were present in the data collection form. at the end of this round, 12 ijahp article: kozak, aksoz, özel/ an analytic hierarchy process model for understanding convention planners’ prior factors of convention hotel selection international journal of the analytic hierarchy process 263 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.283 more criteria which were associated with none of the factors, because they are too comprehensive, were excluded from the list. in the third and last round, experts came to an agreement on the factors and criteria. more specifically, 22 criteria and five factors associated with these criteria were determined and a data collection form took its final shape. the convention hotel selection hierarchy can be seen in table 1. as seen in table 1, the three level hierarchy of convention hotel selection consists of five factors, which are facilities of convention hotel, accessibility, site environment, price and local support. the main goal of the hierarchy is to determine the primary criteria of convention hotel selection of convention planners operating in ankara. the first factor, facilities of convention hotel, has four criteria. these are listed as technological infrastructure, exhibit space wihout pillars, safety and security, and number of meeting rooms. accessibility, the second factor, can be explained with four criteria, which are distance, airline transportation, frequency of transportation to the region, and roper highways to the region. site environment is another factor in the hierarchy. this factor has six criteria which include climate, destination image, sightseeing opportunities in the region, service quality of hotels in the region, service quality of restaurants in the region, and room capacity of hotels in the region. the fourth factor is price. four criteria are employed to explain the price factor. these are transportation cost, food and beverage prices in the region, accommodation prices in the region, and prices of meeting rooms of hotels in the region. finally, the fifth factor in the hierarchy is local support. this factor consists of support of nongovernmental institutions, support of local government, attitude of citizens towards tourism, and tourist infos in the region. ijahp article: kozak, aksoz, özel/ an analytic hierarchy process model for understanding convention planners’ prior factors of convention hotel selection international journal of the analytic hierarchy process 264 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.283 table 1 the hierarchy of convention hotel selection convention hotel selection facilities of convention hotel accessibility site environment price local support technological infrastructure distance climate transportation cost support of nongovernmental institutions exhibit space without pillars airline transportation destination image food and beverage prices in the region support of local government safety and security frequency of transportation to the region sightseeing opportunities in the region accommodation prices in the region attitude of citizens towards tourism number of meeting rooms proper highways to the region service quality of hotels in the region prices of meeting rooms of hotels in the region tourist infos in the region service quality of restaurants in the region room capacity of hotels in the region 3.2. sample and data collection turkish regulation of travel agents became influential in the determination of the target population of this study. more specifically, convention planners in turkey have to be travel agents in accordance with article 4/paragraph e of turkish regulation of travel agents. a travel agent may undertake a stand-alone organization or more than one travel agency may come together with cooperation agreements and operate under the title of convention planner. for that reason, managers of a class travel agencies operating in ankara formed the accessible sample of this study. according to the 2013 statistics obtained from the association of turkish travel agencies, the total number of a class travel agents in ankara is 76. the authors conducted preliminary telephone interviews with officials from those travel agents and discovered that 40 of these travel agents dealt with the operation of convention organization. ijahp article: kozak, aksoz, özel/ an analytic hierarchy process model for understanding convention planners’ prior factors of convention hotel selection international journal of the analytic hierarchy process 265 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.283 data were collected with face-to-face interviews. travel agencies were telephoned in advance and appoinments were made for interviews. then, face-to-face interviews were conducted with the managers of 13 travel agents in april, 2014. respondents were asked to make pairwise comparisons about the relative importance of convention hotel selection factors and criteria, respectively. the 9-point intensity of saaty’s relative importance scale (1980) was used while making this comparison (table 2). the interviewers explained the procedure for filling out the questionnaire to respondents and clarified points that were not understood, and this contributed to gathering complete and consistent data from the respondents. table 2 9-point intensity of relative importance scale intensity of relative importance definition explanation 1 equal importance two activities contribute equally to objective 1. 3 moderate importance of one over another experience and judgment slightly favor one activity over another 5 essential or strong importance experience and judgment strongly favor one activity over another 7 demonstrated importance an activity is strongly favored, and its dominance is demonstrated in practice 9 extreme importance the evidence favoring one activity over another is of the highest possible order of affirmation 2, 4, 6, 8 intermediate values between the two adjacent judgments when a compromise is needed source: adapted from saaty (1980) 3.3. data analysis one of the questionnaires was excluded from the data analysis since it was perfunctorily filled out and included so many missing values. for that reason, 12 questionnaires were used for further analysis. the analytic hierarchy process (ahp) was employed in the analysis of the data. the analytic hierarchy process is a general theory of measurement. it is used to derive ratio scales from both discrete and continuous paired comparisons. these comparisons may be taken from actual measurements or from a fundamental scale which reflects the relative strength of preferences and feelings. the ahp has a special concern with departure from consistency, its measurement on and dependence within and between the groups of elements of its structure. it has found its widest applications in multicriteria decision making, planning and resource allocation and in conflict resolution (saaty, 1987). in its general form, the ahp is a nonlinear framework for carrying out both deductive and inductive thinking without use of the syllogism by taking several factors into consideration simultaneously and allowing for dependence and for feedback, and making numerical tradeoffs to arrive at a synthesis or conclusion. t. l. saaty developed the ahp in 1971-1975 while at the wharton school (university of pennsylvania, philadelphia, pa). ku and fan (2009) also explain the ahp method as ijahp article: kozak, aksoz, özel/ an analytic hierarchy process model for understanding convention planners’ prior factors of convention hotel selection international journal of the analytic hierarchy process 266 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.283 using the pairwise comparison method to rank order the criteria and alternatives of a problem which are formulated in hierarchical structure (ku and fan, 2009). before the analysis of the data, the consistency of each respondent was analyzed with the help of expert choice software. the consistency ratios for all the answers of the 12 respondents resulted in acceptable values below 0.10 (saaty, 1980). so, data analysis was carried out on 12 consistent questionaires. to obtain an aggregate measure of the pairwise comparisons of all managers involved in this study, the geometric mean of the individual assessments was used (chen, 2006). geometric means of the answers of 12 respondents were calculated with the help of excel software. 4. findings table 3 summarizes the consistency tests for convention hotel selection factors. as suggested by saaty (1980), if the consistency ratio is smaller than 0.10, then the comparisons are acceptable. as shown in the table, the group consistency ratio was found to be 0.01, which indicates an acceptable group judgement. additionally, calculation of consistency ratios of five factors resulted in acceptable values below 0.10 (table 3). table 3 consistency tests for convention hotel selection factors level consistency ratio consistency test goal 0.01 accepted factors 0.01 accepted facilities of convention hotel 0.00 accepted accessibility 0.01 accepted site environment 0.00 accepted price 0.04 accepted local support 0.02 accepted the data obtained as a result of the binary comparison of the factors and criteria are given in table 4. as seen in this table, according to respondents, the most important selection criteria is meeting facilities of convention hotel (0,365). other criteria are as follows; price (0,333), accesibility (0,164), site environment (0,088) and local support (0,050) based on their perceived importance. the importance rankings of each criterion under the factors have also been calculated separately with the help of a software package. for instance, when we examine the criteria of facilities of convention hotels, the most important criteria is the technological infrastucture of the convention hotel (0,318). exhibit space without pillars (0.295) and number of meeting rooms (0.295) have emerged as two other criteria which are of equal importance. within this group, the least important criterion was found to be the safety and security of the convention hotel (0,034). according to the respondents, the most important criterion within the price factor is accomodation prices in the region (0.433). this was followed by the prices of the meeting rooms of convention hotels (0.350), transportation cost (0.125) and food and beverage prices in the region (0.093). in terms of accessibility, airline transportation (0.526) was seen as much more important than the rest of criteria. frequency of ijahp article: kozak, aksoz, özel/ an analytic hierarchy process model for understanding convention planners’ prior factors of convention hotel selection international journal of the analytic hierarchy process 267 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.283 transportation to the region (0.204), distance (0.183) and proper highways to the region (0.087) shared the second, third and fourth places, respectively. table 4 local weight and global weight for each criteria in convention hotel selection factors local weights † criteria local weights global weights ‡ ranking facilities of convention hotel 0.365 technological infrastructure 0.318 0.116 1 exhibit space without pillars 0.295 0.108 2-3 safety and security 0.092 0.034 7 number of meeting rooms 0.295 0.108 2-3 accessibility 0.164 distance 0.183 0.030 12 airline transportation 0.526 0.086 6 frequency of transportation to the region 0.204 0.034 11 proper highways to the region 0.087 0.014 17 site environment 0.088 climate 0.073 0.006 20 destination image 0.157 0.014 16 sightseeing opportunities in the region 0.075 0.007 18-19 service quality of hotels in the region 0.413 0.036 9 service quality of restaurants in the region 0.075 0.007 18-19 room capacity of hotels in the region 0.206 0.018 14 local support 0.050 support of nongovernmental institutions 0.464 0.023 13 support of local government 0.304 0.015 15 attitude of citizens towards tourism 0.121 0.006 21 tourist infos in the region 0.111 0.006 22 price 0.333 transportation cost 0.125 0.042 8 food and beverage prices in the region 0.093 0.031 10 accomodation prices in the region 0.433 0.144 4 prices of meeting rooms of hotels in the region 0.350 0.116 5 when evaluated in terms of the convention hotels’ site environment, service quality of hotels in the region (0.413) was determined to be the most important criteria. room capacity of hotels in the region (0.206) is the second most important criteria, which is followed by destination image (0.157) in third place. sightseeing † local weight is derived from judgment with respect to a single criterion ‡ global weight is derived from multiplication by the weight of the criteria ijahp article: kozak, aksoz, özel/ an analytic hierarchy process model for understanding convention planners’ prior factors of convention hotel selection international journal of the analytic hierarchy process 268 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.283 opportunities in the region (0.075) and service quality of restaurants in the region (0.075) are of equal importance and share the fourth place. in this group the criterion, climate (0.073) was regarded as the least important. as determined by respondents of this study, local support is the least important factor for convention hotel selection. within this factor, support of nongovernmental institutions to the convention (0.464) and support of local government (0.304) were seen as slightly more important whereas attitude of citizens towards tourism (0.121) and tourist infos in the region (0.111) were regarded as relatively unimportant. 5. conclusion and managerial implications the analysis of the data compiled from convention planners operating in ankara demonstrated the primary factors and criteria these planners take into account while deciding on which convention hotel to choose. according to the results obtained with the help of the ahp, facilities of the convention hotel, price, accessibility, site environment, and local support were clarified as the primary factors, respectively. the results indicate that convention planners in ankara primarily pay attention to the unique facilities of convention hotels whereas site environment of hotels and local support in the region stay in the background. based on these results, it should be remarked that convention planners primarily focus on selection criteria which are particular to the characteristics of convention hotels while planning a convention. namely, technological infrastructure of convention hotels, exhibit space without pillars, number of meeting rooms of convention hotels and issues of safety and security stand out among other criteria. the result is compatible with the results of previous studies in the literature, as well. chen (2006) identified the meeting and accommodation facilities as the second most important criteria among convention site selection criteria. admittedly, this approach of convention planners puts a big burden on convention hotel management since convention planners’ expectations from convention hotels will also be very high. for this reason, convention hotels should do their best to meet the needs and wants of convention planners in terms of convention facilities. results of this study indicate that, price is the second most important factor taken into consideration while planning a convention. this supports the results obtained by boz (2010) and elston and draper (2012). the fact that convention planners attach such a degree of importance to price can be interpreted as they are preoccupied with keeping the costs low and they consider the potential sensitivity of convention participants to price. consequently, it may be inferred that large capacity hotels, which make bulk purchases and keep the costs low, will benefit from this situation since they have the opportunity to offer lower prices for convention planners. however, convention hotels should never compromise on quality in order to keep prices low. accessibility has been identified as the third most important factor by convention planners operating in ankara. thus, it can be said that, accessibility is also taken into account by the convention planners while planning a convention. as a matter of fact, accessibility was indentified in many of the previous studies on convention tourism and in some of these studies (moser, 2003), this criteria was seen as a priority by respondents. the present study, in which accessibility was found the be the third criteria in importance after meeting facilities and price, shows that convention planners prefer hotels which are located relatively nearby, can be reached by air ijahp article: kozak, aksoz, özel/ an analytic hierarchy process model for understanding convention planners’ prior factors of convention hotel selection international journal of the analytic hierarchy process 269 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.283 transportation and where the time cost of transportation is low. for example, convention hotels should be located in destinations such as istanbul, ankara, izmir and antalya since they have the advantage of having proper highways and convenient airline transportation. it is also interesting that site environment of the convention hotel is less important according to convention planners, when compared to the first three factors. in other words, the exterior features of the convention hotels such as sight-seeing opportunities in the region, image, climate and service qualities of the hotels in the region almost do not affect the hotel selection of convention planners. this finding contradicts the fact that, nowadays sightseeing tours are highly demanded by convention participants and these tours accompany the convention programs at the expiry date of the convention. nevertheless, conclusions reached by previous studies in the literature support the conclusions of this present study. for instance, elston and draper (2012) examined the results of the empirical studies dealing with site selection process of convention planners, published between the years of 1990 and 2012. the authors noted that recreational facilities and opportunities of the convention site were often seen as the most trivial common criteria. on the other hand, convention planners might have been involved in making this decision because convention participants do not frequently need to get out of hotels that have all inclusive systems, or participants have very limited time to spend with specific convention activities only. future research conducted on convention participants may be helpful in order to understand whether the relative unimportance of site environment is derived from a convention planners’ nature of performing their business or preferences of convention participants. still, the authors of this study are not exaggerating when they claim that “tourism due to congress” is a more popular market trend in turkey rather than “congress with tourism purposes”. another relatively less important criterion was determined as local support in the region. in contrast, an examination of the studies in the literature showed that local support is an important criterion. as an example, wan (2011) expressed that the support of local people and associations to the convention and meeting sector is a strong aspect in macao which is located on the southern coast of china. similarly, chacko and fenich (2000) determined that hospitality of local people is one of the leading conditions of being a convention destination for san antonio, usa. in this study conducted in ankara, turkey, it can be inferred that convention planners see local support as one of the least important factors just because convention planners in ankara operate in an introverted structure and they do not fully need the local support in the region. it should be stated that results obtained in this study are limited to the opinions of convention planners operating in ankara. however, future research may be carried out on all the convention planners operating in turkey so that it may be feasible to provide a map of convention tourism in turkey from the point of view of convention planners. in addition, an assessment of these results with those obtained from other capital cities in the world will enable a comparative interpretation. also, an examination of whether the importance rank order of convention hotel selection factors and criteria vary according to the convention’s being national or international may provide useful tips for identification of priorities for conventions in different spheres. ijahp article: kozak, aksoz, özel/ an analytic hierarchy process model for understanding convention planners’ prior factors of convention hotel selection international journal of the analytic hierarchy process 270 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.283 references aksu, a., yılmaz, g., & gümüş, f. (2013). örneklerle kongre ve fuar yönetimi. ankara: detay publishing. amiri, m., zandieh, m., vahdani, b., yazdani, m., & soltani, r. (2008). a hybrid mcdm model with interval weights and data for convention site selection, journal of applied sciences, 8 (15), 2678-2686. baloğlu, ş., & love, c. (2001). association meeting planners' perceptions of five major convention cities, journal of convention & exhibition management, 3(1), 2130. boz, f. (2010). kongre ve toplantı düzenleyicilerinin otel seçimini etkileyen faktörler üzerine bir alan çalışması: i̇stanbul örneği, unpublished master’s thesis, istanbul university institute of social sciences department of tourism management, istanbul. chacko, h. e., & fenich, g. g. (2000). determining the importance of us convention destination attributes, journal of vacation marketing, 6(3), 211-220. chen, c. (2006). applying the analytical hierarchy process (ahp) approach to convention site selection, journal of travel research, 45, 167-174. chiappa, g. d. (2012). how do meeting organizers choose convention sites based on different types of meetings? an empirical analysis of the italian meetings industry, event management, 16, 157-170. clark, j. d., evans, m. r., & knutson, b. j. (1998). selecting a site for an association convention, journal of hospitality & leisure marketing, 5(1), 81-93. crouch, g. i., & louviere, j. j. (2004). the determinants of convention site selection: a logistic choice model from experimental data, journal of travel research, 43, 118130. crouch, g. i., & richie, j. r. (1998). convention site selection research: a review, conceptual model, and propositional framework, journal of convention & exhibition management, 1(1), 49-69. crouch, g. i., & ritchie, b. (1997). convention site selection research, journal of convention & exhibition management, 1(1), 49-69. crouch, g. i., & weber, k., 2002. marketing of convention tourism. in: weber, k. & chon, k-s., ed. convention tourism-international research and industry perspectives. binghamton: the haworth press inc., 57-78. dipietro, r. b., breiter, d., rompf, p., & godlewska, m. (2008). an exploratory study of differences among meeting and exhibition planners in their destination selection criteria, journal of convention & event tourism, 9(4), 258-276. ijahp article: kozak, aksoz, özel/ an analytic hierarchy process model for understanding convention planners’ prior factors of convention hotel selection international journal of the analytic hierarchy process 271 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.283 dölalan, g. (2008). ankara’da kongre turizmi: a grubu seyahat acentalarının bakış açılarına yönelik bir araştırma, unpublished master’s thesis, sakarya university institute of social sciences department of tourism management, sakarya. elston, k., & draper, j. (2012). a review of meeting planner site selection criteria research, journal of convention & event tourism, 13(3), 203-220. eryılmaz, b. (2012). alternatif turizm çeşidi olarak samsun’da kongre turizmi olanaklarının değerlendirilmesi, 13-16 october, samsun symposium. fawzy, a., & samra, y. a. (2008). a conceptual model for understanding associations' site selection processes: an organizational buyer behavior perspective, journal of convention & event tourism, 9(2), 119-136. getz, d. (2004). bidding on events, journal of convention & exhibition management, 5(2), 1-24. hsu, c. c., and sandford, b. a. (2007). the delphi technique: making sense of consensus, practical assessment, research & evaluation, 12(10),1-8. icca (2011). country and city rankings report, international congress and convention association, http://www.iccaworld.com/dcps/doc.cfm?docid=1003, accessed: 05.07.2013. icca (2012). country and city rankings, http://www.iccaworld.com/dcps/doc.cfm?docid=1604, accessed: 03.03.2014. jeon, y., & kim, j. s. (2011). an application of swot-ahp to develop a strategic planning for a tourist destination, 16. graduate students research conference, 6-8 january, houston, texas. ku, e. c. s., & fan, y. w. (2009). the decision making in selecting online travel agencies: an application of analytic hierarchy process, journal of travel and tourism marketing, 26(5-6), 482-493. mair, j. (2012). a review of business events literature, event management, 16, 133141. moser, s. (2003). convention and congress tourism to nice-how to keep track with changes in demand, unpublished master’s thesis, ma european tourism management bournemouth university, uk. nelson, r., & rys, s. (2000). convention site selection criteria relevant to secondary convention destinations, journal of convention & exhibition management, 2(2-3), 71-82. opperman, m., & chon, k. (1997). convention participation decision-making process, annals of tourism research, 24(1), 178-191. provincial culture and tourism directorate in ankara (2014). provincial culture and tourism directorate in ankara, 2013 (supply of culture and tourism), http://www.ankarakulturturizm.gov.tr/servisler/istatistikler, accessed: 10.04.2014. http://www.iccaworld.com/dcps/doc.cfm?docid=1003 http://www.iccaworld.com/dcps/doc.cfm?docid=1604 ijahp article: kozak, aksoz, özel/ an analytic hierarchy process model for understanding convention planners’ prior factors of convention hotel selection international journal of the analytic hierarchy process 272 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.283 saaty, t. l. (1980). the analytical hierarchy process: planning, priority setting, resource allocation. new york: mcgraw-hill book co. saaty, r. w. (1987). the analytic hierarchy process-what it is and how it is used, math modelling, 9(3) 5, 161-176. tsai, h., & ho, p. (2009). assessing site selection criteria for taiwan lantern festival by analytic hierarchy process, event management, 12, 187–197. upchurch, r. s., jeong, g., clements, c., & jung, i. (2000). meeting planners' perceptions of site selection characteristics, journal of convention & exhibition management, 2(1), 15-35. ünsever, i̇. (2008). kongre turizminin hukuksal analizi, lecture notes, boğaziçi university. wan, y. k. p. (2011). assessing the strengths and weaknesses of macao as an attractive meeting and convention destination: perspectives of key informants, journal of convention and event tourism, 12(2), 129-151. ijahp article: shang/reflections on prioritizing divergent intangible humane acts international journal of the analytic hierarchy process 390 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.536 reflections on prioritizing divergent intangible humane acts jennifer s shang university of pittsburgh pittsburgh, pa shang@katz.pitt.edu “tom saaty will be remembered for being clever, original, and inventive. to his chinese friends, tom was a man of humanity, while seeking to establish and attain for himself, he helped them find a foothold and thrive.” abstract ahp is a versatile tool for prioritizing and making decisions. yet, when facing numerous alternatives with significantly different scale, decision makers often found it impossible to put them together in one matrix and compare them simultaneously. to address the issue, thomas saaty and i (2011) proposed a new analytic hierarchy process-based structure to capture the complex relationship between various levels of activities. without the proposed model, we may not be able to tackle alternatives that are not comparable or it may require a very large number of comparisons, as only comparable items can be compared and used to calculate meaningful priorities. in this paper, we show how to use clustering and pivots to handle this difficulty. through the proposed method, we expand the comparison scale, build a near-consistent matrix, and allow the use of incomparable alternatives. the proposed method helps us effectively derive priorities for alternatives with orders-of-magnitude differences like those in divergent intangible humane acts. keywords: ahp; clustering; pivots; ranking alternatives; decision-making 1. introduction society is shaped by numerous contributions and activities people make to enrich it, but currently there is a lack of an established means to recognize and include these contributions in a compensation system. often, acts of helpfulness are rewarded with words of praise and gestures of appreciation, certificates and mementos, and sometimes with acclamation in the media. at times, the individuals performing kind acts need no monetary reward, but often people are not so fortunate economically. an act of kindness may present an opportunity for monetary compensation. valuing good acts with money allows society to take care of individual’s need, as many benevolent acts can be done by individuals who are unemployed yet eager to contribute. this is an area worth study in the context of multi-criteria decision-making. compensating people for contributions that are not strictly economic but are humane and intangible would benefit society. our model can formally prioritize all criteria and activities systematically, and determine the worth of the service offered and the potential for exchange to enrich ordinary people’s life. mailto:shang@katz.pitt.edu http://izquotes.com/quote/316301 http://izquotes.com/quote/316301 ijahp article: shang/reflections on prioritizing divergent intangible humane acts international journal of the analytic hierarchy process 391 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.536 2. quantifying the relative value of altruistic acts different services to society have different values. the amount of “value” exchanged in any transaction should not be arbitrary or negotiable but carefully planned and evaluated. conventional ahp models are versatile for prioritizing and assessing alternatives. however, when facing numerous alternatives of significantly different scale, decision makers find it hard to put all alternatives together in one matrix and compare them simultaneously. in order to address the issue, we proposed a new analytic hierarchy process-based structure to capture the complex relationship between various levels of activities. the proposed method can not only help establish exchange standards for benevolent acts in communities where products or services may encompass a variety of skills, experience, training, equipment, or risk; but also reduce uncertainty in the evaluation process and outcome. without the proposed model, we may not be able to tackle incomparable alternatives or many alternatives that require a very large number of comparisons. only comparable items can be compared and used to calculate meaningful priorities. we show how to use clustering and pivots to handle the difficulties of having too many alternatives. through the proposed method, we expand the comparison scale, build a near-consistent matrix, and show how to manage incomparable alternatives. the proposed method makes it possible to effectively derive priorities for alternatives with orders-of-magnitude differences (saaty & shang, 2011). 3. ahp for prioritizing intangible humane acts: contributions to society to differentiate divergent intangible humane acts, we develop a cluster and pivots method to capture the relationship among numerous activities contributed by people to society. it avoids the enormous number of pairwise evaluations required for completing large matrices. the proposed model has three distinct steps: 1) for each criterion, group alternatives into clusters. decision makers must decide which alternatives should be in the same cluster, as grouping is a subjective task. heuristically, we can compare the best ordered alternative sequentially with the next ones, from the second best to the worst, until facing the comparison value of 9 or the cluster comprising seven elements. the highest ranked-alternative in this cluster is the pivot of the adjacent clusters, and is the smallest among the remaining alternatives. the same process is repeated until all clusters are linked. 2) all alternatives of the same cluster are compared and prioritized. 3) priorities of all clusters are linked with the pivot (common element belonging to two adjacent clusters) being used to merge adjacent clusters. 4. model results of the benevolent acts we examine the benevolent activities from the perspective of human needs. in figure 1, we modify maslow’s hierarchy of needs (1943) to derive criteria for evaluating acts through which people contribute (see figure 1 for the criteria). they are broken down into 12 sub-criteria. following the three steps outlined in section 3, we obtain the final score of each act (see figure 2), e.g. the final synthesized value of global disease cure is http://www.sciencedirect.com.pitt.idm.oclc.org/science/article/pii/s0377221711004449#b0050 ijahp article: shang/reflections on prioritizing divergent intangible humane acts international journal of the analytic hierarchy process 392 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.536 1,140,230,056 times more important than complimenting people in terms of meeting human needs. figure 1. the criteria, sub-criteria and corresponding weights figure 2. the final priorities of all studied benevolent acts the clustering approach adopted here is critical, as decisions may involve several groups that differ by orders of magnitude from one another. the proposed model allows comparing alternatives if we cluster objects into groups and pivot the largest element in one group as the smallest element in the next group. the priorities in two adjacent groups ijahp article: shang/reflections on prioritizing divergent intangible humane acts international journal of the analytic hierarchy process 393 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.536 should be sufficiently different such that the ratings of the smaller set have impact on the judgment of the larger set. this rearrangement of the alternatives has to be done several times, once for each of the criteria. 5. conclusion kostigen (2009) believes health, welfare, prosperity, leisure, family, and social connections are reasons for happiness. these intangibles are at the heart of all pursuit, especially socio-economic endeavors. hubbard (2007) writes that it is important to develop a method to explain, measure, value, and manage intangibles. in this paper, we follow their advice and look at humane acts from various angles concentrating on existing acts that increase society’s wellbeing. we have proposed an ahp model to capture relationships between altruistic activities, which may be used to better society. the approach is coherent; it structures complex decisions and enables one to transcend the complexity of dealing with problems of orders of magnitude scientifically. when the magnitudes are very small or very large, rating alternatives one at a time involves a lot of guessing, leading to questionable outcomes. instead, comparing alternatives in pairs seems to be necessary for the measurement of intangibles and may result in greater accuracy. through comparisons, we arrange alternatives into homogeneous clusters. we then pivot from one homogeneous cluster to the adjacent one to relate humane acts with diverse orders-of-magnitude. we are facing a divided society where people have to choose between accepting sacrifices for the common good and focusing on protecting one’s own interests. appeals to the common good are often heard when discussing corporate social responsibility, health care systems, environmental pollution, education, crime, and poverty (velasquez et al., 1992). we believe that a community where people reach out and help each other would be a happier one. by exploring a society’s valuation system where people can enumerate and acquire credits for their services, we create “social units” to help society recognize the contributions for all the meaningful efforts people exert. the proposed ahp model offers an objective worth and a common medium of exchange to store values. when benevolent activities are acknowledged, dormant social potential is awakened for further development. our model serves to raise the awareness of how to live an “affluent” life http://www.sciencedirect.com.pitt.idm.oclc.org/science/article/pii/s0377221711004449#b0035 http://www.sciencedirect.com.pitt.idm.oclc.org/science/article/pii/s0377221711004449#b0025 http://www.sciencedirect.com.pitt.idm.oclc.org/science/article/pii/s0377221711004449#b0145 http://www.sciencedirect.com.pitt.idm.oclc.org/science/article/pii/s0377221711004449#b0145 ijahp article: shang/reflections on prioritizing divergent intangible humane acts international journal of the analytic hierarchy process 394 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.536 references hubbard, d.w. (2007). how to measure anything: finding the value of ‘‘intangibles’’ in business. hoboken, nj: john wiley & sons. doi: 10.1002/9781118983836 kostigen, t. (2009). the happiest taxes on earth. retrieved from http://www.marketwatch.com/story/the-happiest-places-on-earth-are-heavily-taxed maslow, a.h. (1943). a theory of human motivation. psychological review 50(4), 370– 396. doi: http://dx.doi.org/10.1037/h0054346 saaty, t.l. and shang, j.s. (2011). an innovative orders-of-magnitude approach to ahp-based mutli-criteria decision making: prioritizing divergent intangible humane acts, european journal of operational research, 214(3), 703-715. doi: https://doi.org/10.1016/j.ejor.2011.05.019 author, a. a., & author, b. b. (date of publication). title of document. retrieved from http://web address velasquez, m., andre, c., shanks, t., meyer, s.j., meyer, m.j. (1992). the common good. retrieved from https://www.scu.edu/ethics/ethics-resources/ethical-decisionmaking/the-common-good/ http://psycnet.apa.org/doi/10.1037/h0054346 https://doi.org/10.1016/j.ejor.2011.05.019 ijahp article: guiora, minutolo, l. vargas, m. vargas, zoffer/enhancing police-community relations via a conflict resolution approach international journal of the analytic hierarchy process 1 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1079 enhancing police-community relations via a conflict resolution approach amos guiora s.j. quinney college of law university of utah amos.guiora@law.utah.edu marcel minutolo rockwell school of business robert morris university minutolo@rmu.edu luis vargas katz graduate school of business university of pittsburgh lgvargas@pitt.edu margaret vargas katz graduate school of business university of pittsburgh mao74@pitt.edu h.j. zoffer katz graduate school of business university of pittsburgh zoffer@katz.pitt.edu abstract despite calls for community-oriented policing and the recognition that it results in improved relationships between the police and community as well as improved public security outcomes, police-community relations are arguably at an all-time low. part of the challenge in achieving enhanced police-community relations is a disparate understanding of what each party wants and what each party can provide as well as prioritizing these factors. we present a project that worked to improve police-community relations through a conflict resolution process using the analytic hierarchy process. the project was conducted with a group of police officers serving an urban, predominately african american community and representatives of the community over a period of several workshops. the workshops identified the goals, criteria, and objectives of each party as well as the perception of the other party’s goals, criteria, and objectives. the results of mailto:amos.guiora@law.utah.edu mailto:minutolo@rmu.edu mailto:lgvargas@pitt.edu mailto:mao74@pitt.edu mailto:zoffer@katz.pitt.edu ijahp article: guiora, minutolo, l. vargas, m. vargas, zoffer/enhancing police-community relations via a conflict resolution approach international journal of the analytic hierarchy process 2 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1079 the meetings, the priorities generated, and similarities/divergences between them are presented. keywords: ahp; conflict resolution; police–community relations negotiations 1. introduction the relationship between the police and the community has gone through difficult times and been strained in the last few years making it difficult for both parties to reach their goals. in fact, some have claimed that police are “suffering a crisis of legitimacy” (ali & pirog, 2019, p. 411). positive police-community relationships are essential to maintaining public safety and order and ensuring that communities thrive. these relationships help reduce fear and biases and build mutual understanding and trust between the police and the communities they serve. the importance of police-community relations to modern policing was first articulated with the “nine principles of policing,” made famous by sir robert peel and the london metropolitan police department in 1829 (lentz & chaires, 2007). these principles are still widely accepted as the foundation for professional policing. peel’s principles (doj, n.d.) emphasize that the police and the public (or community) are one and the same, and hence, they should cooperate at all levels. in fact, robinson and ramsey (2017) note that in order to resolve any tension between the police and the community, both parties need to co-produce safety through community-oriented policing. further, mccarty (2015) states that community policing “encourages partnerships, problem solving, and prevention of crime” (p. 443). however, community policing could not be further from the current reality. despite many calls by all branches of the criminal justice system and community representatives (see for example, bja, 1994; rohe et al., 1997; schanzer et al., 2016) the relationship between police departments and some sectors of the community are strained. ali and pirog (2019) evaluated the efficacy of citizen oversight agencies as a social accountability mechanism; this is important, but oversight does not bridge the divide nor does it help one party understand the needs, perceptions, or priorities of the other. more than 20 years ago, culberston (2000) noted the need for the public to understand the social complexity and divisive issues that plague the relationship between the police and community, yet the challenges have persisted if not grown. this article presents the results of a project that used a unique process of conflict resolution to help find out how police agencies (“police”) and communities (“community”) can better understand each other’s point of view, identify the barriers that prevent understanding, prioritize the reported factors of each party’s point of view as well as their perception of the point of view of the other party, and identify the areas of conflict that would be most promising to allocate resources to close the gap between their respective perceptions. 2. literature review the initial goal of this project was to help the police and community more fully understand each other, so that the second stage of the project could use this understanding to contemplate measurable improvements in the basic relationship between the two ijahp article: guiora, minutolo, l. vargas, m. vargas, zoffer/enhancing police-community relations via a conflict resolution approach international journal of the analytic hierarchy process 3 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1079 parties. this is consistent with the idea of conflict sensitivity (hussein et al., 2019; popovych, 2021; robinson, 2021). conflict sensitivity is the ability of an organization to: 1. understand more completely the context in which it is operating, intergroup tensions and the “divisive” issues with a potential for conflict, and the “connecting” issues with the potential to mitigate conflict and strengthen social cohesion; 2. understand the interaction between its intervention and that context; and 3. act upon that understanding, in order to avoid unintentionally feeding into further division, and maximize the potential contribution to strengthen social cohesion and commitment to shared values. in order for both parties, the police and the community, to achieve conflict sensitivity they need to be open to “double-loop learning” and enable communication practices that allow them to evaluate both the suitability and efficacy of the politics and norms that guide them (argyris & schon, 1978; cartwright, 2002). in the context of police and community relationships where social conflict is particularly high, double-loop learning is necessary to promote adaptiveness on both sides in order to shift the focus from zerosum solution spaces to win-win spaces by optimizing the gains to both parties for the concessions made. to achieve such an end-state, there needs to be a process that facilitates “mutual responsiveness, reciprocity, and invitation to a new relationship” (kelman, 2009, p. 182). the negotiators in this process must employ problem-solving tools that take both parties through a process that ensures double-loop learning and makes sure that both parties are committed to the process (johnson & johnson, 2005). dissipating the tension and reaching a resolution requires that the mediator understands the complexity of the environment. to accomplish this, the mediator needs to gain a depth of understanding of the relationship between and within the parties. they must know the way that “material, attitudinal, cognitive, and perceptual factors contribute to the development of a cooperative and competitive relationship between the parties” (hussein et al., 2019, pp. 25-26). to this end, this article presents an approach that takes each party through a process where these factors are developed and the intensity of the various attributes are elucidated in order to reach a mutually beneficial set of resolutions. 3. methods the first task that needed to be undertaken was to understand the context in which the police and the community are operating. this task required that each party, separately, identified their goals, objectives, and actions in society and prioritized the importance of each element from their own point of view. the prioritization was based on a process called the analytic hierarchy process (ahp) approach, which has a proven record of providing solutions for complex and difficult problems where each side shares as equally as possible in modifying their initial commitments (e.g., minutolo et al., 2022, vargas et al., 2021). prior to understanding the context, it was necessary to find potential members from each party to participate in focus group sessions wherein the goals, objectives, and actions were developed. the participants needed to be people who were prepared to negotiate all issues and allow the process to seek solutions that are equitable for each side to be ijahp article: guiora, minutolo, l. vargas, m. vargas, zoffer/enhancing police-community relations via a conflict resolution approach international journal of the analytic hierarchy process 4 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1079 developed. the people selected for each group needed to be open minded, progressive thinkers, and open to reconsidering even their most deeply-held values. the participants in this project were selected using personal interviews in which we addressed the following issues:  total commitment: participants must begin with a total commitment to the goal of the enterprise, namely, to recognize the importance of improving the relationship between the people of the community involved and the police force of their region.  no prior expectations or demands: participants must be willing to allow the process they are engaged in to operate without prior expectations or demands or initially ruling out any procedures. they need to understand that the process itself will address and evaluate even deeply held positions.  mutual respect: participants need to understand that the process depends on developing mutual respect between participants from both parties. while there will inevitably be significant differences in opinion, the participants must be prepared to listen to different positions, no matter how unacceptable they seem, and recognize the positions of the other side without expressing strong emotional reactions.  strength of personal commitments: participants will use a 1-9 scale to indicate the strength of their personal commitment to one approach or another and to express their judgment as to their priorities. the scale will be used to measure the intensity of preference of one item over another item.  acceptance of opposing views: participants need to be prepared to offer their judgments on actions proposed by either party, and allow the positions to be considered, even if they are inherently opposed to them, understanding that the process itself will allow them to express their opposition.  the process yields balanced tradeoffs: the process is directed to measure benefits and costs, and allow each party to see that it yields gains for each group as close to equality as possible, so that the concluding recommendations can be mathematically shown to the participants to provide a win-win situation where neither side has a significant advantage.  the process yields fair tradeoffs: participants need to be prepared to recognize that inevitable disagreements are considered in a trade-off process that allows concessions to be measured and yields an outcome which is as fair as possible for each party. we selected seven members of each group, the community and the police, to conduct the study described. each group was brought together separately to elicit goals, objectives, and actions. each group was introduced to the ahp process and conducted a brainstorming session. an approximately hour-long introduction to the analytic hierarchy process was provided to each group wherein we provided examples of uses of the ahp in conflict analysis. ijahp article: guiora, minutolo, l. vargas, m. vargas, zoffer/enhancing police-community relations via a conflict resolution approach international journal of the analytic hierarchy process 5 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1079 in their session, the police group developed the following list of their perceived objectives of policing (what they think they ought to be doing): 1. reduce crime 2. public safety 3. first responder 4. investigate crimes 5. enforce the law 6. problem solve with the public 7. build a relationship with the community 8. be a role model 9. educate the community the objectives were grouped into three broad categories of criteria as follows: protect, educate and serve. together, the criteria are the activities that the police engage in to achieve the strategic criteria of cooperation, public approval, and building trust. note that some of the objectives fall into multiple categories. the objectives first responder and public safety, for instance, were classified into the categories protect and serve. the hierarchical model developed out of this process is represented in figure 1. purpose: understand the role of policing cooperation public approval trust protect educate serve  reduce crime  public safety  first responder  investigate crimes  enforce the law  public safety  problem solve w/ public  build relationships w/ community  be a role model  first responder  educate the community figure 1 hierarchy of police objectives the community group consisted of six individuals from the same neighborhood that the police group serves; one individual withdrew. the group had two points of view with respect to the police objectives. the community group differentiated between policing (figure 2) and service (figure 3). policing for this group represents the perception of the police of their activities. with respect to the community that we focused on, police officers working in the police department are not required to live in the city. the community had two thoughts about the police state of mind as a result of not living in the city. the first thought is that they do not live in the neighborhood where they serve and as ijahp article: guiora, minutolo, l. vargas, m. vargas, zoffer/enhancing police-community relations via a conflict resolution approach international journal of the analytic hierarchy process 6 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1079 such have no connection with the community. the second is that the police believe that low-income communities are considered less worthy than others and as such not as deserving as other communities. these two ideas seemed to shape community’s thoughts about police behavior in those communities. thus, the community group thinks that the police work on the assumption or presupposition that criminal behavior is taking place in those communities. based on these thoughts, the community group believes that the police appear to have the following objectives: 1. catch people assumed to be breaking the law (according to the community, the police call this “enforcing the law”) 2. make arrests under the assumption of safety/security 3. maintain control of the community (community labels this a “power and fear policy”) 4. protect city and commercial property because they are considered more valuable since they produce revenue in terms of taxes 5. police in the strictest meaning of the word – “the activities carried out by police officers in order to preserve law and order” 6. seizures 7. quotas 8. compliance with the law the community hierarchy presented in figure 2 illustrates the perception of policing by the community. according to the perception of the community, the funding criteria captures points 4, 6, and 7 above while the control criteria captures 1, 2, 3, and 8. point 5 above is captured as the goal of the overall hierarchy. policing funding control  protection of city & commercial property  seizures  quotas  enforcing the law  arrests  power and fear  compliance figure 2 policing hierarchy (perception of community) the second point of view is based on the idea that police should serve the community. they call this point of view “service” (everybody should be treated equal – equality). ijahp article: guiora, minutolo, l. vargas, m. vargas, zoffer/enhancing police-community relations via a conflict resolution approach international journal of the analytic hierarchy process 7 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1079 according to the community participants, “service” is represented by the following elements: 1. neighborhood representation 2. communication skills 3. advocacy for the community 4. diversion 5. demilitarization 6. redefine the activity as public safety rather than policing 7. harm reduction 8. funding transparency 9. proper behavioral health training with accountability service community administration care / people  neighborhood representation  advocacy for the community  demilitarization  redefine public safety  funding transparency  communication skills  diversion  harm reduction  behavioral health training figure 3 service hierarchy (perception of community) the service hierarchy is captured in figure 3. the activities presented in the list above are divided into community, administration, and care/people or the social dimension. the objectives fit into these categories as follows: community (objectives 1 and 3), administration (objectives 5, 6 and 8) and care/people (objectives 2, 4, 7 and 9). this approach resembles harris’ (2015) proposal of the “10 characteristics of a well-run police operation.” harris’ (2015) ten characteristics include partnership, identifying with the community, focused deterrence against violence, modern use-of-force policy, biasfree policing, early intervention systems, citizen complaint system, independent external oversight, critical incident review, and policies made public. in fact, much of what both parties pointed to is similar to what others have to say on the subject. according to the american bar association (aba, 2018), the major responsibilities of police are as follows: a. identify criminal offenders and criminal activity and, where appropriate, to apprehend offenders and participate in subsequent court proceedings; b. reduce the opportunities for the commission of some crimes through preventive patrol and other measures; c. aid individuals who are in danger of physical harm; d. protect constitutional guarantees; e. facilitate the movement of people and vehicles; ijahp article: guiora, minutolo, l. vargas, m. vargas, zoffer/enhancing police-community relations via a conflict resolution approach international journal of the analytic hierarchy process 8 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1079 f. assist those who cannot care for themselves; g. resolve conflict; h. identify problems that are potentially serious law enforcement or governmental problems; i. create and maintain a feeling of security in the community; j. promote and preserve civil order; and k. provide other services on an emergency basis. these responsibilities are guided by principles that should be inherent in a democratic society, which are to:  safeguard freedom, preserve life and property, protect the constitutional rights of citizens and maintain respect for the rule of law;  provide maximum opportunity for achieving desired social change by freely available, lawful, and orderly means;  give emphasis to those social and behavioral problems which may require the use of force or the use of special investigative abilities which the police possess, and  firmly establish the principle that the police should be restricted to using the amount of force reasonably necessary in responding to any situation. the next task in the process was the prioritization of the other party’s perceptions. each party prioritized their perception of the importance the other party gives to their goals, objectives, and actions. the result of the two tasks produced priorities for each party’s own goals, objectives and actions and the perceived priorities of the other party’s goals, objectives, and actions. to accomplish this, the following items needed to take place; first, group prioritization and avoidance of groupthink; second, synthesis of group priorities of goals, objectives and actions using the ahp; and, finally, the interpretation of the priorities. the priorities developed in the first two tasks were used to identify “divisive” and “connecting” issues. the priorities were used to identify agreements and disagreements between the groups. finally, we were able to use the priorities to arrive at an agreement. the analytic hierarchy process (ahp) is a theory based on relative measurement implemented using pairwise comparisons. suppose that we want to assign priorities to n activities or objectives, some of which may not be measurable (i.e., they are intangible), and others may be measured in different scales for different criteria. to simplify the exposition, assume that we have a criterion in mind (e.g., “importance”). the aim is to determine how important the alternatives or objectives are toward achieving the desired goal. importance could be a conglomerate of criteria, but to make it simple, assume that it is a single criterion. select a pair of alternatives, a and b, and ask the questions, “which alternative is more important?”, and “how much more important is one than the other?” to express the degree of importance, one could use words, but words cannot be processed mathematically unless they are translated into numbers. hence, thomas l. saaty (1980) developed the scale given in table 1 that assigns numbers 1, 3, 5, 7, and 9 to words such as equal, moderate, strong, very strong, and extreme, respectively. the numbers 2, 4, 6 and 8 are used for compromises. the ahp is particularly useful for group decision making because it helps reach consensus and avoid groupthink (aczél & saaty, ijahp article: guiora, minutolo, l. vargas, m. vargas, zoffer/enhancing police-community relations via a conflict resolution approach international journal of the analytic hierarchy process 9 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1079 1983; saaty & vargas, 2007), and satisfies arrow’s conditions of unrestricted domain, pareto’s principle, independence from irrelevant alternatives, and non-dictatorship (saaty & vargas, 2003). table 1 fundamental scale intensity of importance definition explanation 1 equal importance two activities contribute equally to the objective 3 moderate importance experience and judgment slightly favor one activity over another 5 strong importance experience and judgment strongly favor one activity over another 7 very strong or demonstrated importance an activity is favored very strongly over another; its dominance demonstrated in practice 9 extreme importance the evidence favoring one activity over another is of the highest possible order of affirmation 2, 4, 6, 8 reciprocals of above intermediate values between the two adjacent judgments if activity i has one of the above nonzero numbers assigned to it when compared with activity j, then j has the reciprocal value when compared with i when compromise is needed rationals ratios arising from the scale if consistency were to be forced by obtaining n numerical values to span the matrix to develop a relative scale of priorities, pairwise comparisons are used. the numerical comparisons are arranged in a matrix whose entries satisfy the reciprocal property wherein if alternative ai is preferred to alternative aj with an intensity aij>1, then aj is 1/aij times preferred to alternative ai. arranging the pairwise comparisons in a matrix a such as the one given: 𝐴 = 𝐴1 𝐴2 … 𝐴𝑛 𝐴1 𝐴2 ⋮ 𝐴𝑛 [ 1 1/𝑎12 𝑎12 1 ⋯ 𝑎1𝑛 𝑎2𝑛 ⋮ ⋱ ⋮ 1/𝑎1𝑛 1/𝑎2𝑛 ⋯ 1 ] ijahp article: guiora, minutolo, l. vargas, m. vargas, zoffer/enhancing police-community relations via a conflict resolution approach international journal of the analytic hierarchy process 10 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1079 we have what is known as a positive reciprocal matrix. if the entries of the matrix satisfy the property 𝑎𝑖𝑘 = 𝑎𝑖𝑗𝑎𝑗𝑘 for all i, j and k, the matrix is said to be consistent. for consistent matrices, the underlying priorities of the alternatives 𝑤 = (𝑤1,𝑤2,…, 𝑤𝑛) are given by any column of the matrix a. normalizing them to unity, the sum equals 1, all the columns yield the same vector of priorities, and the entries of the matrix are given by 𝑎𝑖𝑗 = 𝑤𝑖 𝑤𝑗 . in this case, it can be easily shown that 𝐴𝑤 = 𝑛𝑤. when the matrix is not consistent, the principal eigenvalue model is used to derive the priorities. the assumption is that the entries of matrix a are multiplicative perturbations of the ratios of the priorities given by 𝑤𝑖 𝑤𝑗 𝜀𝑖𝑗. in this case, we solve the eigenvalue problem 𝐴𝑤 = 𝜆𝑚𝑎𝑥𝑤, where 𝜆𝑚𝑎𝑥 is the largest real eigenvalue of matrix a. the resulting vector of priorities 𝑤 is unique to within a multiplicative constant. because we normalize to unity (the 𝑙1 − 𝑛𝑜𝑟𝑚), the scale produced is a relative ratio scale. it has no units of measurement, and hence, we can measure the importance of intangibles with relative measurement. the ahp is necessary because human relationships involve intangibles, and decisions are usually made in groups. thus, relative measurement can help us understand perceptions of groups expressed in terms of intangibles. groups usually make decisions by consensus. in the ahp, each member of a group provides judgments for the pairwise comparisons. assume that a group wants to select a location for a company to relocate, then the group needs to agree on the final location. there are multiple criteria required to make the decision, and consensus will need to be attained before deciding. one way of helping to achieve consensus is by combining the paired comparison judgments of the members of the group using the geometric mean. saaty and aczel (1983) proved that the only aggregation procedure f that satisfies:  separability: 𝑓(𝑥1,…,𝑥𝑚) = 𝑔(𝑥1) ∘ ⋯∘ 𝑔(𝑥𝑚),  unanimity: 𝑓(𝑥1,… ,𝑥𝑚) = 𝑥,  homogeneity: 𝑓(𝜆𝑥1,…,𝜆𝑥𝑚) = 𝜆𝑓(𝑥1,…,𝑥𝑚),𝜆 > 0, and  the reciprocal property: 𝑓(1/𝑥1,…,1/𝑥𝑚) = 1/𝑓(𝑥1,…,𝑥𝑚), is the geometric mean. separability assumes that each of the individual judgements of the participants can be separated. unanimity assumes that if all judgements have the same value, then the synthesized judgements should have the same value. homogeneity implies that if each individual judges a second ratio to be x times as large as a first ratio, then the synthesized judgement on the second should be x times as large as that on the first. finally, the reciprocal property assumes that if one were to synthesize the reciprocals of 𝑥1,… ,𝑥𝑚, it would lead to the reciprocal of 𝑓(𝑥1,… ,𝑥𝑚). the geometric mean of the judgments from a group of m decision makers {𝑎𝑖𝑗 𝑘 }, where k represents the kth decision maker, and i and j are the alternatives being compared, is given by: 𝑓(𝑎𝑖𝑗 1 ,𝑎𝑖𝑗 2 ,…,𝑎𝑖𝑗 𝑚) = ∏ 𝑎𝑖𝑗 𝑘𝑚 𝑘=1 (1) the question is “when does the geometric mean represent the judgment of the group?” an issue that needs to be addressed here is the dispersion of the judgments. when does the geometric mean of judgments represent the group judgment? saaty and vargas (2007) ijahp article: guiora, minutolo, l. vargas, m. vargas, zoffer/enhancing police-community relations via a conflict resolution approach international journal of the analytic hierarchy process 11 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1079 showed that group judgments are too dispersed to represent the group if we cannot reject the null hypothesis of a statistical test based on the concept of geometric dispersion. geometric dispersion is measured by the geometric variance, a concept introduced to study the variance of judgments elicited in saaty’s scale (saaty & vargas 2007). this scale has positive values and the reciprocal of each positive value. thus, if we have two judgments, a and 1/a, the average is not the arithmetic mean, but the geometric mean, because a and 1/a are equidistant from 1 in the multiplicative sense. hence, the arithmetic mean could not be used in the traditional definition of the variance which is for interval scales which admit positive and negative numbers. saaty and vargas (2007) showed how to approximate the probability distribution of the geometric variance of a set of judgments to perform a statistical test about the geometric variance. using the geometric mean as a group aggregation procedure provides two benefits (a) avoidance of the groupthink effect; and, (b) compliance with the conditions of arrow’s impossibility theorem (saaty & vargas 2012). the police-community project used group judgments based on the principles explained above. 4. results to measure how the goals and objectives were achieved, we needed to first measure the relative importance of the goals and objectives. in the context of the police-community project, the goal for both parties is to evaluate the effectiveness of policing. as stated earlier, it is not enough for one party or the other to state how effective they think the police are at achieving this goal, but rather it is necessary to capture the perception that each party has with respect to this goal given the context of the strategic criteria they have. 4.1. priorities from the police group the police group consisting of seven individuals had over 100 years of combined experience. following the guidelines provided above, the group developed the hierarchy presented in figure 1. they identified three strategic criteria of the police including cooperation, public approval, and trust. cooperation was defined by the group as working with the community. public approval is the degree to which the public approves of the work that they are doing in the community. finally, trust is a measure of the degree to which the community believes that the police have their best interest in mind and execute their duties in good faith. these strategic criteria were prioritized as follows in table 3: table 3 global priorities for police group goals cooperation public approval trust priorities cooperation 1 7 5 0.7015 public approval 1/7 1 1/7 0.0586 trust 1/5 7 1 0.2399 ijahp article: guiora, minutolo, l. vargas, m. vargas, zoffer/enhancing police-community relations via a conflict resolution approach international journal of the analytic hierarchy process 12 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1079 these objectives were prioritized under the strategic criteria of cooperation, public approval and trust. it is interesting to note that not all of the participants agreed with respect to the judgements and in some cases, disagreed not only with the weighting but also with respect to what was most important. finally, the objectives were prioritized with respect to the goals within each of the categories. table 3 provides the results of the strategic criteria and the composite priorities; table 4 shows the priorities of the objectives’ categories; and, table 5 provides the results of the priorities of the activities that the police engage in with respect to the strategic objectives. as seen in table 4, serving the community is seen as the most important activity when it comes to cooperation (0.43), public approval (0.42), and trust (0.48). table 4 priorities of objectives’ categories cooperation public approval trust protect 0.374 0.369 0.21 educate 0.197 0.208 0.310 serve 0.429 0.422 0.48 table 5 priorities of goals and objectives priorities of goals -> 0.7015 0.0586 0.2399 overall cooperation public approval trust priorities reduce crime 0.0559 0.2058 0.036 0.0599 public safety 0.2414 0.1912 0.1923 0.2267 first responder 0.2525 0.1174 0.1698 0.2247 investigate crimes 0.0901 0.1715 0.0902 0.0949 enforce the law 0.1074 0.0368 0.2202 0.1303 problem solve with public 0.0579 0.0659 0.07 0.0613 build relationship with community 0.0944 0.1168 0.1386 0.1063 be a role model 0.0324 0.0287 0.0257 0.0306 educate the community 0.068 0.0659 0.0572 0.0653 as illustrated by the priorities in table 5, the police group sees itself as providing public safety (0.2267), acting as first responders (0.2247), enforcing the law (0.1303), and trying to build relationships with the community (0.1063). these objectives account for 69 percent of the total priority. we will focus on these objectives when developing programs to attain them. ijahp article: guiora, minutolo, l. vargas, m. vargas, zoffer/enhancing police-community relations via a conflict resolution approach international journal of the analytic hierarchy process 13 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1079 4.2. priorities from the community group the community priorities for the point of view policing are given in table 6 and those of service can be found in table 7. table 6 priorities of policing point of view priorities of goals 0.75 0.25 overall control funding priorities 1 enforcing the law 0.20 0.146 2 arrests 0.19 0.142 3 power and fear 0.39 0.289 4 protection of city and commercial property 0.13 0.033 6 seizures 0.20 0.051 7 quotas 0.67 0.167 8 compliance 0.23 0.173 table 7 global priorities of service point of view priorities of goals 0.315 0.123 0.563 overall community administration care/people priorities 1 neighborhood representation 0.36 0.113 2 communication skills 0.18 0.101 3 advocacy for the community 0.64 0.202 4 diversion 0.3 0.169 5 demilitarization 0.32 0.039 6 redefine public safety vs. policing 0.43 0.053 7 harm reduction 0.3 0.168 8 funding transparency 0.25 0.031 9 proper behavioral health training with accountability 0.22 0.124 note that the community’s desired objectives of the police, from the point of view of service, are not far from those described in harris (2015). their similarity can be observed by putting them side-by-side as in table 8. ijahp article: guiora, minutolo, l. vargas, m. vargas, zoffer/enhancing police-community relations via a conflict resolution approach international journal of the analytic hierarchy process 14 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1079 table 8 side-by-side comparisons of harris’ and the community’s objectives harris' objectives community's objectives community priorities  partnership  neighborhood representation 0.113  identifying with the community  advocacy for the community 0.202 administration  an open and accessible citizen complaint process  demilitarization 0.039  independent external oversight  redefine public safety vs. policing 0.053  critical incident review  funding transparency 0.031  policies made public care/people  focused deterrence against violence  communication skills 0.101  a modern use-of-force policy  diversion 0.169  bias-free policing  harm reduction 0.168  early intervention systems  proper behavioral health training with accountability 0.124 next, each party identified the actions needed to implement their objectives. for example, what do the police do now to reduce crime? from the community perspective, what does the community think that the police should do to enhance advocacy for the community? the actions needed to implement their objectives are called concessions in this context since they represent what each party wants from the other and may be able to provide as a means to reconcile the tensions between the two. 4.3. concessions – demands of one party from the other in 2014, cops-community oriented policing services, department of justice (palladian partners, inc., 2014) held a meeting in which they drafted guidelines for “strengthening the relationship between law enforcement and communities of color.” they divided the action items into the following four types: police operations, building relationships between the police and the community, the role of law enforcement, and the role of the community. under police operations, the police are responsible to: ijahp article: guiora, minutolo, l. vargas, m. vargas, zoffer/enhancing police-community relations via a conflict resolution approach international journal of the analytic hierarchy process 15 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1079 1. hire with a focus on recruitment and promotions on procedural justice; 2. train and promote police legitimacy from recruits to the command staff level; and 3. implement geographic accountability. in building relationships between the police and the community, the police are responsible for: 1. building relationships; 2. improving communication; 3. rethinking training; 4. recognizing the power of race; 5. understanding young people; 6. learning about diverse communities; 7. developing community policing; and 8. offering leadership. in the role of law enforcement, the police are responsible to: 1. inform about truth telling to encourage transparency; 2. divert good people out of the criminal justice system; 3. create constitutional policing methods; eliminate stop and frisk as a tactic; 4. engage the community frequently; actively encourage and develop feedback mechanisms from all segments of the community; and 5. collaborate on public safety issues with city agencies, business, nonprofits, and community organizations. the role of the community in this process includes: 1. identifying leaders within the community; 2. reclaiming ownership of the public space; 3. coming outside of the four walls of the church (including faith-based leaders); 4. acknowledging the historical stereotypes; and 5. learning the cultural differences within the latino community to engage them appropriately. the cops’ results are very similar to the results from our workshops with the two groups. based on the priorities obtained from the community group from the service point of view, the community group developed action items of community advocacy, diversion, harm reduction, and proper behavioral health training with accountability. the action items are suggestions for how the police might achieve the criteria that can then be prioritized. some of the items include actions like wearing plain clothes to community events, open town hall meetings at secular locations, and getting to know the community. in sum, the community wants community advocacy (20%), diversion (17%), harm reduction (17%), and proper behavioral health training and accountability (12%). the perception of the community about the objectives of the police are that the police are concerned with controlling through power and fear (0.29), fulfilling quotas (0.17), ijahp article: guiora, minutolo, l. vargas, m. vargas, zoffer/enhancing police-community relations via a conflict resolution approach international journal of the analytic hierarchy process 16 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1079 ensuring compliance (0.17), enforcing the law (0.15), and making arrests (0.14). whereas the police group says that they provide public safety programs (0.023), first responder programs (0.23), enforce the law (0.13), and build relationships with the community (0.11). 5. discussion while the community leaders articulated opposition to the concept of “defund the police,” they similarly articulated a distinctly different perception of police goals from those voiced by police participants. there is an important subtlety in the distinction. the community leaders understand the need for law enforcement and recognize the threat posed to their community by criminal elements. in that sense, there is – perhaps – an unspoken possibility for an alliance. however, as the data we gathered highlights, the community’s perception of law enforcement’s behavior is distinct from its expectations of police behavior. there is clear symmetry between law enforcement self-perception and the community’s expectations, but a corresponding lack of symmetry between the community’s expectations and its perception of law enforcement. closing that gap must come from within the groups. the easiest step could be in how police officers are trained to interact with the community. uniform training in police academies is easier to implement than changing community’s perceptions without facts at hand. despite the differences between the two groups, a great deal of advancement was made during the process. the use of the analytic hierarchy process as an approach to bridge the divide between the two groups served several purposes. first, the process solicited the expectations for a desired goal and the corresponding criteria against which they evaluate how well some alternative is or is not achieving the desired outcome from the perspective of each group. the ahp approach allowed each party to articulate their perception of what the other party wants and how they think the other party evaluates alternatives. one aspect of the approach that is particularly useful is that it provides each party with a scale to see the intensity of priorities of the other party as well as articulate each other’s perception of the intensity with which elements are valued. this process gave both the police and community participants a sense of the perceptions of the other which they did not have prior to the meeting. 6. future work and conclusions while the community developed a set of action items for the police to take to achieve the community’s goals, steps still need to be developed for the police to enhance relations with the community, as well as determining what the community should do to make sure the police are doing what they say they are doing, in other words, to enhance mutual trust. one natural extension of this project is to conduct similar workshops in other communities. not all communities or police organizations will prioritize the goals and objectives in the same way, or even have the same goals and objectives. communities and the police that serve them are very distinct and any attempt to set priorities ought to be localized. however, we have presented an approach to open a dialogue between police and their communities to develop an understanding of how each party perceives what it ijahp article: guiora, minutolo, l. vargas, m. vargas, zoffer/enhancing police-community relations via a conflict resolution approach international journal of the analytic hierarchy process 17 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1079 needs and wants along with the corresponding priorities there is a need to validate these findings in other communities to gather more data to convince the police of the need to change the existing training models. the lack of symmetry in perceptions highlighted in this study demonstrates that law enforcement and the community, although believing they speak the same language, do not. the data compellingly demonstrates an unbridgeable dissonance, a lack of recognition of this dissonance, and the need to proactively change the existing police training model. ijahp article: guiora, minutolo, l. vargas, m. vargas, zoffer/enhancing police-community relations via a conflict resolution approach international journal of the analytic hierarchy process 18 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1079 references aczél, j., and saaty, t.l. 1983. procedures for synthesizing ratio judgements. journal of mathematical psychology, 27(1), 93-102. https://doi.org/10.1016/0022-2496(83)90028-7 ali, m.u., and pirog, m. 2019. social accountability and institutional change: the case of citizen oversight of police. public administration review, 79(3): 411-426. american bar association (aba). 2018. standards on urban police function. cjs publications. argyris, c., and schön, d. 1978. organizational learning: a theory of action perspective. the journal of applied behavioral science, 15(4), 542-548. bureau of justice assistance (bja), u.s. department of justice. 1994. understanding community policing: a framework for action. ncj monograph 148457. cartwright, s. 2002. double-loop learning: a concept and process for leadership educators. journal of leadership education, 1(1): 68-71. culbertson, h.m. 2000. a key step in police-community relations: identify the divisive issues. public relations quarterly, 45(1), 13. eck, k., conrad, c.r., and crabtree, c. 2021. policing and political violence. journal of conflict resolution, 65(10), 1641-1656. eterno, j.a., barrow, c.s., and silverman, e.b. 2017. forcible stops: police and citizens speak out. public administration review, 77(2), 181-192. department of justice (doj). policing 101. community relations service toolkit for policing. https://www.justice.gov/file/1376626/download dixon, t.l., schell, t.l., giles, h., and drogos, k.l. 2008. the influence of race in police–civilian interactions: a content analysis of videotaped interactions taken during cincinnati police traffic stops. journal of communication, 58(3), 530-549. harris, d.a. (2015, may 10). what does good policing look like? hereare 10 policies that every police department should adopt. pittsburgh post-gazette, editorial, d-1. hussein, j.w., bedasa, n.a., and mengistu, b.t. 2019. integrating reflective problemsolving and mindfulness to promote organizational adaptation toward conflict sensitivity. journal of transformative education, 17(1), 24-50. johnson, d.w., and johnson, r.t. 2013. essential components of peace education. in harris, ian m., and mary lee morrison (eds.), peace education (pp. 280-292). mcfarland, routledge. https://doi.org/10.1016/0022-2496(83)90028-7 https://www.justice.gov/file/1376626/download ijahp article: guiora, minutolo, l. vargas, m. vargas, zoffer/enhancing police-community relations via a conflict resolution approach international journal of the analytic hierarchy process 19 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1079 kelman, herbert c. 2009. a social-psychological approach to conflict analysis and resolution. in d. j. sandole, s. byrne, i. sandole-staroste, & j. senehi (eds.), handbook of conflict analysis and resolution (pp. 170–183). new york, ny: routledge. lentz, s.a., and chaires, r.h. 2007. the invention of peel's principles: a study of policing ‘textbook’ history. journal of criminal justice, 35(1), 69-79. mccarty, r. 2015. commentary: community policing: a mechanism for successful assimilation of immigrant cultures into our communities. public administration review 75(3), 443-444. minutolo, m.c., vargas, l.g., guiora, a.n., and ray, m. 2022. applying the ahp to conflict resolution: a russia—nato case study. group decision and negotiation, 32, 147-136. doi: https://doi.org/10.1007/s10726-022-09803-z palladian partners, inc. 2014. strengthening the relationship between law enforcement and communities of color: developing an agenda for action. washington, d.c. office of community oriented policing series. u.s. department of justice. popovych, s. 2021. the concept of "conflict sensitivity" and its application by country and field office staff during implementation of humanitarian programmes: case study of save the children canada. [doctoral dissertation, fordham university]. price, m., and blitman, b.a. 2016. insight policing: a creative approach to building bridges between police and the communities they serve. dispute resolution journal, 71(3), 77. robinson, a. 2021. localisation and conflict sensitivity: lessons on good practice from south sudan. conflict sensitivity resource facility (csrf) better aid forum briefing paper. available at https://www.csrf-southsudan.org/wp-content/uploads/2021/12/csrfbaf-briefing-paper-conflict-sensitivity-and-localisation-in-south-sudan-14dec2021final.pdf robinson, l.o., and ramsey, c.h. (2017). the future of policing reform: the way forward? public administration review, 77, 162-163. doi: https://doi.org/10.1111/puar.12736 rohe, w.m., adams, r.e., and arcury, t.a. 1997. community oriented policing: what it is, why it works, how to get started. center for urban and regional studies, university of north carolina at chapel hill. ncjrs #169113. saaty, t.l., and vargas, l.g. 2007. dispersion of group judgments. mathematical and computer modelling, 46(7-8), 918-925. saaty, t.l., and vargas, l.g. 2012. the possibility of group choice: pairwise comparisons and merging functions. social choice and welfare, 38(3), 481-496. schanzer, d.h., kurzman, c., toliver, j., and miller, e. 2016. the challenge and promise of using community policing strategies to prevent violent extremism: a call for https://doi.org/10.1007/s10726-022-09803-z https://www.csrf-southsudan.org/wp-content/uploads/2021/12/csrf-baf-briefing-paper-conflict-sensitivity-and-localisation-in-south-sudan-14dec2021-final.pdf https://www.csrf-southsudan.org/wp-content/uploads/2021/12/csrf-baf-briefing-paper-conflict-sensitivity-and-localisation-in-south-sudan-14dec2021-final.pdf https://www.csrf-southsudan.org/wp-content/uploads/2021/12/csrf-baf-briefing-paper-conflict-sensitivity-and-localisation-in-south-sudan-14dec2021-final.pdf https://doi.org/10.1111/puar.12736 ijahp article: guiora, minutolo, l. vargas, m. vargas, zoffer/enhancing police-community relations via a conflict resolution approach international journal of the analytic hierarchy process 20 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1079 community partnerships with law enforcement to enhance public safety. durham: triangle center on terrorism and homeland security. vargas, l.g., moreno‐loscertales, c. and moreno‐jiménez, j.m. 2021. conflict resolution in the era of cognitive multicriteria decision‐making: an ahp‐retributive approach. international transactions in operational research,30(3), 1453-1478. doi: https://doi.org/10.1111/itor.13088 ijahp article: ibrar, kim, lee, rho /analytic hierarchy process model for the selection of optimal internet access technologies in rural pakistan international journal of the analytic hierarchy process 254 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.712 analytic hierarchy process model for the selection of optimal internet access technologies in rural pakistan ibrar ahmad global information & telecommunication technology program korea advanced institute of science and technology (kaist) daejeon, republic of korea ibrar@kaist.ac.kr suk kyoung kim professor of the global information & telecommunication technology program korea advanced institute of science and technology (kaist) daejeon, republic of korea bigcandy@kaist.ac.kr jongsu lee professor of the technology management, economics and policy program college of engineering, seoul national university (snu) seoul, republic of korea jxlee@snu.ac.kr jae jeung rho professor of the school of business and technology management korea advanced institute of science and technology (kaist) daejeon, republic of korea jjrho111@kaist.ac.kr abstract information and telecommunication technologies are considered significant for the economic and social development of isolated and remote areas. internet technologies create links between urban and rural areas that can overcome the barrier of distance that typically hinders development. in rural areas of pakistan, internet penetration is low due to high cost, lack of manageable infrastructure, public issues and many other factors that disturb sustainable internet provision; however, the choice of access to technology in rural areas is a complex process. in this paper, we used the analytic hierarchy process (ahp) method, which is a powerful multi-criteria decision-making (mcdm) approach and is a structured procedure for organizing and analyzing complex decisions. furthermore, we proposed an ahp model for the selection of an ideal internet access technology, which would facilitate access to internet services to the 61% of the population who live in pakistan’s rural areas. data were collected from 38 respondents, from both academia and the telecom sector it professionals. the relative weights of each factor and technology alternative were synthesized with expert choice 2000. this paper mailto:ibrar@kaist.ac.kr mailto:bigcandy@kaist.ac.kr mailto:jxlee@snu.ac.kr mailto:jjrho111@kaist.ac.kr ijahp article: ibrar, kim, lee, rho /analytic hierarchy process model for the selection of optimal internet access technologies in rural pakistan international journal of the analytic hierarchy process 255 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.712 can provide comprehensive recommendations to telecommunication policymakers for internet deployment and the selection of optimal internet access technology in rural pakistan. keywords: access technology selection; multi-criteria decision-making (mcdm); analytical hierarchy process (ahp); internet penetration; rural pakistan 1. introduction in the modern information technological revolution, the internet plays a vital role. telecommunications that enable the internet are a link between urban and rural areas that can overcome the hurdle of distance, which interrupts development. despite the rapid worldwide diffusion of the internet, there has been a big gap between developing and developed countries – rich and poor regions – in terms of the number of internet users because of a limitation of telecommunications infrastructure. the same telecommunication services should be offered to both rural and urban areas such as telephone, data transmission, video transmission, and other services, both for individuals (private subscribers) and for the public communities (itu group 7, 2000). in a developing country, telecommunications planning in rural areas is a difficult procedure and characterized by a multitude of difficult problems because these rural areas are technologically poor and need special consideration (andrew et al., 2005). therefore, the planning and provision of internet services to rural areas requires more time and involves a substantial use of manpower from many internet service providers that would be responsible for providing transmission bandwidth to rural inhabitants, including wireless, fiber cables, undersea cables, and satellite, when compared to urban centers. this makes choosing the optimal internet access technology in rural areas more complicated from the perspective of internet service providers. it also requires other decision-making processes and attributes that can take into account the environment of rural areas that has different characteristics than urban areas (andrew et al., 2003). according to the pakistan census (2017) report, the total population of pakistan is 21 million, out of which 61% live in rural areas. to ensure a digital balance between urban and rural areas, the universal service fund (usf) of the government of pakistan is playing an important role in providing for universal access to broadband services and promoting telecommunication services in un-served and under-served areas throughout the country (telecommunications policy, 2015). currently, communication satellite systems facilitate access to telecommunication services in rural pakistan. after the government of pakistan issued 3g/4g licenses in 2014, the usf redesigned the next generation broadband for sustainable development (ng-bsd) program to provide telecom (internet) services to the unserved areas across pakistan. vakataki et al. (2017) highlighted that only 7% of households in rural areas had internet access compared to 18% of households in the urban areas of pakistan in 2016. the gap between urban and rural areas has been reduced in terms of household internet access between 2012 and 2016, as shown in figure 1. yasir (2017) stated that in rural pakistan internet penetration is limited because of many factors such as high cost to the consumers from internet service providers, cultural hurdles, lack of sustainable infrastructure, and the low literacy rate in the country. ijahp article: ibrar, kim, lee, rho /analytic hierarchy process model for the selection of optimal internet access technologies in rural pakistan international journal of the analytic hierarchy process 256 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.712 figure 1 household internet access in pakistan (vakataki et al. 2017) a new approach is needed to increase the low internet access rate that is based on the existing satellite system. considering the environment and constraints in rural pakistan, it is necessary to re-select a suitable internet access technology in order to reach a higher internet access rate. so far, however, there has been no suitable decision model. therefore, this study suggests the ahp model for the selection of an optimal internet access technology within the various constraints to provide access to the central telecommunications network and enlarge connectivity to rural pakistan. in this paper, we developed an analytic hierarchy process (ahp) model, using the multicriteria decision-making (mcdm) approach. the goal of the decision problem is the selection of the optimum internet access technology to provide quality internet services to rural areas of pakistan. chemane et al. (2005), malladi and min (2005), and andrew et al. (2005) have used the mcdm based model and the ahp model to select internet access technologies that are suitable for a specific region. the ahp model provides priority weights for the internet access technologies, and the internet access technology with the highest priority weight is then selected for the rural areas of pakistan from among the different available technologies (such as dsl, cable modem, and wireless). this study is organized into five sections; section one gives an overview of the telecommunication services in pakistan’s rural areas. section two reviews related work by exploring the attributes that affect the selection and deployment of internet access technology for rural areas of pakistan. section three introduces the different internet access technologies that are available in pakistan and develops an ahp model for the selection of access technologies. data collection and the analytical hierarchy process (ahp) analysis will be covered in section four, and section five discusses the conclusions, study limitations, and suggestions for future work. 2. attributes for access technology selection in rural areas proper decisions need to be made in order to ensure the provision of internet services with the most helpful system within numerous constraints (itu-d, 1997). according to saaty (1990), the most important and creative issue in making a decision is the adoption of the factors that are relevant to that decision. saaty (2005) stated that a cluster gives ijahp article: ibrar, kim, lee, rho /analytic hierarchy process model for the selection of optimal internet access technologies in rural pakistan international journal of the analytic hierarchy process 257 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.712 room for one to think about a criteria grouping that has a common set of attributes. the ahp model involves a cluster of elements that are connected to one another by their need. previous studies have shown that each telecommunication service provider has its own criteria, sub-criteria, and alternatives for the particular problem. malladi and min (2005) and douligeris and pereira (1994) investigated cost, quality, and speed attributes and used multi-criteria decision making (mcdm) methods, particularly the analytic hierarchy process (ahp), for the selection of access technology in rural areas. chemane et al. (2005) and andrew et al. (2005) explored several criteria concerning financial and technical aspects and stated that the selection of the most appropriate internet access technology is a challenging and complex process. pakistan's rural areas need a list of possible criteria when selecting access technologies. to combine the final list of the selection criteria the following activities were used: this study conducted a literature survey of previous studies on similar problems, including douligeris and pereira (1994), chemane et al. (2005), malladi and min (2005), chasia (1976), and andrew et al. (2005) that were used as secondary sources to strengthen the list of criteria that includes the most important factors for the problem. interactions with it professionals both from the telecom sector and from academia, who were contacted through e-mail to give their feedback on the preliminary list of attributes, using a five-point likert-type scale. the preliminary list of attributes can be observed in table 1. table 1 preliminary list of attributes attributes infrastructure contents & development fairness public issues technical aspect community of interest coverage government support climatic conditions spectrum licensing reliability supporting policies flexibility cost security fixed cost speed & services (bandwidth) return on investment geographical coverage variable cost response time terrain factors maintenance subsidy using the above-mentioned activities and the results obtained from the respondents on the preliminary list, the weakest attributes (fairness, climate conditions, flexibility, geographical coverage, response time, return on investment, and terrain factors) were dropped from the list for further analysis using the cut-off value method, and a consolidated list of 17 attributes (table 2) that have straightforward effects on the objective of the selection of optimum internet access technologies for rural pakistan was ijahp article: ibrar, kim, lee, rho /analytic hierarchy process model for the selection of optimal internet access technologies in rural pakistan international journal of the analytic hierarchy process 258 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.712 finalized. the ahp model was grouped into three strategic criteria, namely technical aspects, public issues, and cost. each criteria cluster only includes those attributes that are comparable by orders of magnitude. further, 14 sub-criteria attributes (infrastructure, coverage, reliability, security, bandwidth, maintenance, contents & development, community of interest, government support, spectrum licensing, supporting policies, fixed cost, variable cost, and subsidy by government) were grouped according to their respective criteria (technical aspects, public issues, and cost). 2.1 attributes for technical aspects rural areas in developing countries have a lack of technical assistance and equipment repair facilities due to a lack of experienced manpower. from a technical point of view (itu-d, 1997), it is recommended that rural telecommunications systems satisfy certain conditions and proper decisions need to be made to ensure the provision of an effective system and efficient network. gasmi and recuero (2005) stated that in developing countries rural transportation systems are not often well managed, and most are not reachable at all by roads, and the presence of on-site technical staff is rare. therefore, the sub-classification of the technical aspects attributes includes infrastructure, coverage, reliability, security, bandwidth (speed and service) and maintenance. 2.2 attributes for public issues sattar (2007) stated that through telecommunications development, rural communities can obtain the ability to improve their education, health, knowledge, agricultural skills, living standard, and earning. andrew et al. (2005) highlighted that rural communities have a strong interest in their geographical and administrative setups; therefore, their needs have to be taken into account when planning rural telecommunications networks. hudson (2013) highlighted that in the interest of the local community, there should be more and more innovations for the development of public and social issues. the attributes for public issues include contents and development, community of interest, government support, spectrum licensing, and supporting policies. 2.3 attributes for cost chemane et al. (2005) stated that rural areas of developing countries have some economic aspects that make it hard to provide a suitable telecommunication infrastructure. ict and especially the internet is considered a key driver for social and economic development. andrew et al. (2005) analyzed that in developing countries internet penetration is low due to lack of infrastructure, skills, and low-income communities in rural areas. internet services can reduce isolation and would eliminate the hurdles of rural living at affordable prices. according to a report from international monetary fund (imf, 2018), pakistan is among the top eight countries where the majority of the population cannot access or afford the internet. the attributes under the cost criteria are rather self-explanatory. these are fixed cost which covers the expenses for purchasing, deployment, and recovery cost, etc. variable cost refers to maintenance, administration, training, testing, and upgradation, etc. the government might wish to subsidize the rural consumers who have been deprived. new methods (reforms) for neutral financing mechanisms, like subsidy auctions and universal service funds must continuously be revisited so that they can truly benefit rural consumers. (clark & wallstern, 2002). table 2 summarizes the final list of attributes (criteria and sub-criteria) with descriptions for the ahp model. ijahp article: ibrar, kim, lee, rho /analytic hierarchy process model for the selection of optimal internet access technologies in rural pakistan international journal of the analytic hierarchy process 259 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.712 table 2 list of criteria and sub-criteria for ahp model criteria sub-criteria description related literature t e c h n ic a l a sp e c t infrastructure the combination of technologies and systems to make efficient transmissions possible. it contributes to the evaluation of backbone technologies. douligeris (1994); gasmi (2005); clark (2002) coverage how the proposed system can cover wider rural areas. end-users located within the area can have access to the internet. chemane et al. (2005) reliability unreliable service will not encourage the rural population. the networks need safeguards and security against breakdown. provide consistent speed and service. chasia (1976); malladi & min (2005) security protecting telecommunications networks from vandalism/theft the security of physical infrastructure refers to the protection of the equipment and cables installed. douligeris (1994); chemane (2005); malladi & min (2005) speed & services (bandwidth) bandwidth is relevant for both voice and data communication. bandwidth is directly related to the effective performance of the link (speed & services) itu (1997); malladi & min (2005); chemane et al. (2005) maintenance in most rural areas the main supply of electricity does not exist which lags behind telecommunication development and maintenance (low fault liability, no-site repair work, etc.). the system must be capable of maintaining better services. hudson (1989); chasia (1976) p u b li c i ss u e s contents & development (by government) through ict development, rural communities can get maximum benefits to improve their education, knowledge, health, agricultural skills, earnings, and living standard. sattar (2007); herselman, (2003) community of interest rural communities have a strong interest in their immediate geographical and administrative area. this needs to be considered when planning rural telecommunications networks. hudson (2006); andrew et al. (2005) government support modest (limited) support from the government will continue the creative use of the existing services and facilities to give new life to the rural communities. sm nazem (1996) spectrum licensing service delivery was a concern with the license restriction on telecommunication companies. in many countries spectrum is licensed through auctions and there is a high price to pay for some frequencies. andrew et al. (2005) supporting policies most countries have an explicit policy goal of promoting universal access to certain infrastructure utilities including telecommunications, electricity, etc. at affordable prices. clark & wallstern (2002) c o st fixed cost investment required for deploying the access technology (purchase, deployment and central office, etc.) andrew et al. (2005); chemane et al. (2005); variable cost cost of maintenance, administration, training, and up-gradation, etc. (depends on the number of users) andrew et al. (2005); chemane et al. (2005); subsidy the government supports poor or rural consumers for political reasons or as part of a development strategy. new methods (reforms) are necessary to raise subsidies and to ensure access by poor people. clark & wallstern (2002) ijahp article: ibrar, kim, lee, rho /analytic hierarchy process model for the selection of optimal internet access technologies in rural pakistan international journal of the analytic hierarchy process 260 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.712 3. access technologies and ahp model 3.1 selection of access technology for rural areas of pakistan according to the latest telecom indicators (november 2019) from pakistan telecommunication authority (pta)'s website, the total number of broadband subscribers by technologies is 71 million. currently, wired and wireless technologies are accessible as a backbone connection for internet connectivity in remote areas of pakistan. pta’s (november 2019) updates further stated that total broadband penetration is 36.18%, of which 35.02% is mobile broadband penetration and 1.16% is fixed broadband penetration. in pakistan, broadband technologies are dsl, evdo (enhanced voice-data optimized), wimax (worldwide interoperability for microwave access), hfc (hybrid fiber-coaxial), ftth (fiber to the home), mobile broadband, and 3g/4g/lte (fixed). for this study, three relevant access technologies (dsl, fiber optic, and wireless) have been chosen as the decision alternatives. from pta’s (november 2019) updates, figure 2 shows annual broadband subscribers growth by technology in pakistan. figure 2 annual broadband subscribers by technology in pakistan (pta, nov 2019) ptcl (pakistan telecommunication corporation limited) offers and delivers both evdo and dsl technologies to the majority of broadband subscribers in the country with roughly a 65% market share in terms of subscribers (umer & harris, 2017). however, figure 2 shows that the fixed broadband penetration in pakistan remains very low over the past five years due to the dominance of the mobile platform. table 3 briefly compares the advantages and disadvantages of the selected backbone internet access technologies for rural pakistan. ijahp article: ibrar, kim, lee, rho /analytic hierarchy process model for the selection of optimal internet access technologies in rural pakistan international journal of the analytic hierarchy process 261 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.712 table 3 selected technology alternatives for pakistan’s rural areas technology advantages disadvantages dsl dsl (digital subscriber line) the way a computer connects to the internet at high speeds using telephone lines. coverage high cost ease of deployment limited bandwidth efficient latency fiber optic ftth (fiber to the home) the installation and use of optical fiber from a central point. high flexibility most difficult to deploy high reliability long rollout time high speed high cost wireless wimax, mobile bb, and ev-do (enhanced voice-data optimized) wireless transmission of data through radio signals. low-cost equipment less bandwidth high reliability low reach/line of sight fast deployment licensing constraints 3.2 analytical hierarchy process (ahp) model for this study, the multi-criteria (mcdm) approach plays an important role in the development of the ahp model for the selection of internet access technology in rural areas. saaty’s (2008) analytical hierarchy process (ahp) is a mathematical method that is widely used to solve multi-criteria decision problems. ishizaka and labib (2011) explained that the ahp methodology includes four steps to solve decision problems (1) constructing a hierarchy describing the problem, (2) constructing matrices for pair-wise comparisons between successive levels, (3) producing priorities, or relative weights, of the elements at each level of the hierarchy, by judgment scale of pair-wise comparisons, and (4) synthesizing the relative weights of the various levels obtained from the third step to produce an overall score of decision alternatives. a linear hierarchy of ahp methodology can be seen in figure 3. figure 3 linear hierarchy a consistency ratio (cr) ≤ 0.1 (10%) indicates that there is sufficient consistency for the decision. gasiea et al. (2010) stated that the hierarchical tree of the ahp methodology should be designed accordingly, and contain four levels descending from the general to the more specific. the ahp model of this study is shown in figure 4, the top level-1 is the overall goal of the decision, which is the selection of an optimum backbone internet http://en.wikipedia.org/wiki/radio ijahp article: ibrar, kim, lee, rho /analytic hierarchy process model for the selection of optimal internet access technologies in rural pakistan international journal of the analytic hierarchy process 262 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.712 access technology to provide quality internet services to rural pakistan, followed by the decision criteria, which affect the goal directly in level-2. the sub-criteria are in level-3, and the alternatives to be calculated are at the lowest level-4. andrew et al. (2005) explained that the alternative with the maximum weighted value is to be treated as the preferred alternative. figure 4 ahp model for the selection of optimal access technology 3.3 analytical hierarchy process (ahp) survey and data collection the same process that was used to select the attributes selection in section 2 was repeated, and the it experts were contacted again. the ahp survey questionnaire was sent via email to various it experts to collect the data and the respondents were from different sectors. the questionnaire was designed in three parts. the first part (part-a) contained pair-wise comparisons for three main criteria and 14 sub-criteria. the second part (part-b) contained the pair-wise comparisons of 12 attributes with three alternatives (dsl, fiber ijahp article: ibrar, kim, lee, rho /analytic hierarchy process model for the selection of optimal internet access technologies in rural pakistan international journal of the analytic hierarchy process 263 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.712 optic, and wireless). the last part (part c) asked about the general demographics of the respondents. according to saaty and vargas (1994), the experts were asked how important criterion a is relative to criterion b, and then asked to assign a weight between 1 and 9 to represent their judgments: 1 = equal important, 3= moderately important, 5 = strongly important, 7 = very strongly important and 9 = extremely important. 2, 4, 6 and 8 are intermediate values (when compromise is needed). an example of the technical aspects and public issues attributes can be seen in figure 5. figure 5 example of comparison of technical aspects and public issues attributes part a: pair-wise comparison questions with respect to criteria the pair-wise comparison matrices were constructed from the hierarchy, which describes each criterion and sub-criteria in the hierarchical tree of this study in order to assist the respondent’s understanding of the meaning of each factor comparison. part b: pair-wise comparison questions with respect to sub-criteria and alternatives every sub-criteria and alternative that are involved in the hierarchical tree of this study were described to ensure that the respondents understood the meaning of each factor comparison, the meaning of the alternatives, different criteria, and their levels before answering the pair-wise comparison questions. part c: demographics and general information the last part of the survey questionnaire was about the demographic and professional information of respondents. 3.4 data collection after the ahp hierarchical structure was built, the next phase was to perform the pairwise comparison which is one of the major strengths of the ahp. we collected survey data from 43 respondents. out of the 43 respondents, five were invalid due to incomplete answers or an inconsistency rate that was too high. these five samples were discarded. finally, 38 responses were deemed eligible for analysis. the respondents were it experts from different academia and telecom sectors, and they were government employees, private employees, and semi-government employees. the dataset was restricted to it experts who had advanced experience and higher qualifications. ijahp article: ibrar, kim, lee, rho /analytic hierarchy process model for the selection of optimal internet access technologies in rural pakistan international journal of the analytic hierarchy process 264 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.712 4. results and analysis this section gives an in-depth description about how this study followed the analytical hierarchy process (ahp) procedure to obtain comprehensive results. the last step of the ahp synthesizes the results to find a solution regarding the selection of an optimal internet access technology for rural pakistan. the collected results were then entered in a reciprocal matrix in order to form the corresponding pair-wise comparison judgment matrices. furthermore, based on the suggestion of saaty (2001) and saaty and vargas (2007) all of the judgments obtained from individual it experts were aggregated into a representative group judgment by calculating the geometric mean for each pairwise question. the aggregated group judgments were then arranged in corresponding consensus pairwise judgment matrices and finally entered into expert choice 2000 to perform the necessary computations and synthesize the results. the ahp hierarchy reveals the relationship among attributes of one level with the attributes of the level directly below (saaty, 2008). the ahp uses the priorities score achieved from the comparison to weight the priorities in the level and continues this process for every element from the goal level down to the alternative level. the connecting lines of the goal to every criterion mean that the criteria should be pair-wise compared for their significance with reference to the goal. likewise, the connecting lines of every criterion to the sub-criteria mean the following are pair-wise compared for their significance with reference to the criterion itself. lastly, the connecting lines of each sub-criterion to the alternative mean the alternative technologies are pair-wise compared to determine which is significant for that criterion. this pair-wise comparison is utilized to determine the relative significance of the elements that are involved. subsequently, the ahp derives local priorities directly from pair-wise comparisons of the sub-criteria with respect to the criteria, whereas global priorities are derived from the multiplication of the local priority weight of criteria and sub-criteria priorities. once the global priorities of all of the sub-criteria were acquired, they were multiplied by the local priority of each alternative with respect to each sub-criterion to obtain the evaluation score (weight) of each alternative. lastly, in the grand total row an overall score (weight) for each alternative was achieved by summing each evaluation score (weight) in the columns as shown in table 4. according to the judgments of all (38) of the participants, the global and local priorities of the alternatives with respect to the criteria and sub-criteria are shown in table 4. ijahp article: ibrar, kim, lee, rho /analytic hierarchy process model for the selection of optimal internet access technologies in rural pakistan international journal of the analytic hierarchy process 265 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.712 table 4 composite preferences for the criteria & sub-criteria, ir = 0.01 goal: selection of optimal internet access technology for rural areas of pakistan c r it e r ia sub-criteria sub-criteria alternatives global priorities dsl fiber optic wireless t e c h n ic a l a sp e c t (l : 0 .3 7 0 g : 0 .3 7 0 ) coverage (l: 0.506 g: 0.187) 0.051 0.02 0.116 0.187 infrastructure (l: 0.129 g: 0.047) 0.013 0.01 0.024 0.047 reliability (l: 0.365 g: 0.135) bandwidth (l: 0.64 g: 0.09) 0.025 0.021 0.04 0.086 maintenance (l: 0.223 g: 0.03) 0.013 0.004 0.013 0.03 security (l: 0.140 g: 0.02) 0.009 0.008 0.003 0.02 0.111 0.063 0.196 0.370 p u b li c i ss u e s (l : .2 4 0 g : .2 4 0 ) community of interest (l: 0.142 g: 0.034) 0.013 0.005 0.017 0.035 contents & development (l: 0.283 g: 0.068) 0.021 0.012 0.035 0.068 govt. support (l: 0.574 g: 0.138) spectrum licensing (l: 0.48 g: 0.07) 0.017 0.01 0.039 0.066 support policies (l: 0.52 g: 0.07) 0.025 0.011 0.036 0.072 0.076 0.038 0.127 0.241 c o st (l : .3 9 0 g : .3 9 0 ) fixed cost (l: 0.277 g: 0.108) 0.049 0.026 0.033 0.108 subsidy (govt) (l: 0.459 g: 0.18) 0.056 0.036 0.086 0.178 variable cost (l: 0.264 g: 0.103) 0.034 0.017 0.052 0.103 0.139 0.079 0.171 0.389 grand total 0.326 0.180 0.494 1 ijahp article: ibrar, kim, lee, rho /analytic hierarchy process model for the selection of optimal internet access technologies in rural pakistan international journal of the analytic hierarchy process 266 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.712 furthermore, a sensitivity analysis was conducted to show the robustness of the obtained results. according to the sensitivity analysis of the ranking produced by this ahp model, wireless technology with the highest priority score of 0.494 (49%) is the most preferred alternative. dsl is next with a priority of 0.326 (33%), and then fiber optic technology with 0.180 (18%). the sum of the priorities in the grand total row is one (100%). this complies with the ahp procedure and demonstrates that all of the steps of the ahp were applied properly. to verify the consistency of the experts’ judgments comparison, saaty (1980) suggested the adoption of a consistency ratio (cr) to measure how the judgments have been relative to large samples of purely random judgments. in our study, the overall inconsistency ratio (ir) is 0.01 by the ahp for the three access technologies for rural areas in pakistan, and the consistency ratio (cr) ≤ 0.1 indicates that the ahp properly performed all of the steps to achieve the goal – an acceptable decision. moreover, table 4 shows that the comparison of the criteria with respect to the goal yields that the cost criterion has the highest priority of 39% (g, l= 0.390), which indicates it is more important in comparison to the other criteria, technical aspects and public issues with 37% (g, l= 0.370) and 24 % (g, l= 0.240), respectively. furthermore, the sub-criteria are arranged for ranking in descending order according to their relative importance of the global priorities; it is often good to disclose which of them is the most favored. for example, under the global priorities column (table 4), the most important attribute is ‘coverage’ with a priority of 18.70% followed by ’subsidy’ with 17.80% and ‘fixed cost’ with 10.80 %. the least important sub-criteria with priorities of less than 5.0% are ‘infrastructure’, ‘community of interest’, ‘maintenance’ and ‘security’ with 4.70%, 3.50%, 3.0%, and 2.0%, respectively. in the ahp, a hierarchy considers the distribution of a goal between the attribute being compared and judges which attribute has a higher influence on that goal. the overall ranking showed that some attributes have low priority weights. however, these low priorities do not mean that these decision-making standards are not important in the deployment of a telecommunication infrastructure for the rural areas of pakistan. the above analysis demonstrates that the ahp is capable of structuring the problem and giving a systematic method for decision making. the ahp provides decision-makers the opportunity to visualize the weaknesses and the strengths of each technology alternative by comparing their scores against each factor. the obtained weights give information about the alternatives and the way they are used to satisfy the selected attributes. in this regard, a result where one can affirm which alternative technology is more preferable from the it experts’ point of view is made. 5. conclusion the issues surrounding internet access technology selection for rural areas are not only technological, but a complex system of other interrelated factors that cut across various aspects of rural areas and their inhabitants. in pakistan, there is no clear telecom policy for the connectivity of rural areas to date. there are several key factors to be considered that have a big impact on the selection process, related to social, environmental, infrastructure and maintenance concerns. based on the information and knowledge obtained from 38 it experts (from different sectors), this study discussed how to achieve ijahp article: ibrar, kim, lee, rho /analytic hierarchy process model for the selection of optimal internet access technologies in rural pakistan international journal of the analytic hierarchy process 267 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.712 the goal and how to promote internet services to the 61% of the population who live in rural pakistan. for the selection of an optimal internet access technology, a mcdm (multi-criteria decision-making) approach played an important role in the development of the ahp (analytical hierarchy process) model. the ahp hierarchy was structured based on the attributes from previous literature and other identified related attributes. moreover, in pakistan, there are different wired and wireless access technologies available that deliver the connectivity of the internet. this study showed that dsl dominates the fixed broadband market, but its market share has been falling over the past five years due to the dominance of the mobile platform. from the total weights of the overall results, wireless access technology was the most preferred alternative and has the highest score in the ahp model for the rural areas of pakistan. there are some important limitations that have been identified by this study. in pakistan’s rural areas, internet services cost and coverage are the most important factors. the inhabitants of pakistan’s rural communities prefer to use internet services with a more suitable bandwidth, affordable prices, and specifically in their own local language. currently, rural pakistan has access to telephone and internet services, but they are unreliable. this is due to the frequent breaks in the long-haul signal broadcast from the country’s main telecommunication point to the remote access network. information on the coverage area is insufficient; hence, there is a need to improve coverage and reliability. additionally, governments should cooperate with the private sector to promote internet penetration, especially for remote and unserved areas. another limitation is the lack of skilled people in rural pakistan. the government of pakistan and internet service providers should train and educate rural inhabitants about sustainability and maintenance. the respondent's gave a low priority weight in the overall ranking list of the decisionmaking standard to security. however, rural areas of pakistan need to increase security in response to an increased level of theft of copper wires, other internet deployment equipment and damage to property. these low priorities do not mean that these decisionmaking standards are not important in the internet deployment for rural areas. lastly, data were obtained from a small number of participants in the survey, which limits the scope and coverage of the goal. if responses were obtained from a larger number of participants, then there would be a greater chance of selecting a suitable access technology. this larger sample size should involve participants from many different organizations who are aware of the problems of pakistan’s rural areas. from this study, many prospective areas for further research can be acknowledged. in rural pakistan, there is a big issue with power supply, and therefore a possible research initiative could be to study the distribution of the electricity supply to rural communities. the second most important concern for rural communities is the need for education and training about the sustainable use, installation, maintenance, and protection of the telecommunications infrastructure. ijahp article: ibrar, kim, lee, rho /analytic hierarchy process model for the selection of optimal internet access technologies in rural pakistan international journal of the analytic hierarchy process 268 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.712 references andrew, t. n. & petkov, d. (2003). the need for a systems thinking approach to the planning of rural telecommunications infrastructure. telecommunications policy, 27(1-2), 75-93. doi: https://doi.org/10.1016/s0308-5961(02)00095-2 andrew, t.n., rahoo, p. & nepal, t. (2005). enhancing the selection of communication technology for rural telecommunications: an analytic hierarchy process model. international journal of computers, systems, and signals, 6(2), 26-34. chasia, h. (1976). choice of technology for rural telecommunication in developing countries. ieee transactions on communications, 24(7), 732-736. doi: https://doi.org/10.1109/tcom.1976.1093366 chemane, l., ekenberg, l., popov, o. & cossa, t. (2005). mcdm model for selecting internet access technologies-a case study in mozambique. eurocon 2005-the international conference on “computer as a tool”, 2, 1738-1741. doi: https://doi.org/10.1109/eurcon.2005.1630311 clarke, g.r.. & wallsten, s.j. (2002). universal(ly bad) service: providing infrastructure services to rural and poor urban consumers. policy research working paper. doi: https://doi.org/10.1596/1813-9450-2868 digital pakistan policy (2017). available: http://moit.gov.pk/policies/dgp08aug.pdf douligeris, c. & pereira, i.j. (1994). a telecommunications quality study using the analytic hierarchy process. ieee journal on selected areas in communications, 12(2), 241-250. doi: https://doi.org/10.1109/49.272873 field, m.j. (ed.) (1996). telemedicine: a guide to assessing telecommunications for health care. national academies press. gasiea, y., emsley, m. & mikhailov, l. (2010). rural telecommunications infrastructure selection using the analytic network process. journal of telecommunications and information technology, 28-42. gasmi, f. & recuero virto, l. (2005). telecommunications technologies deployment in developing countries: role of markets and institutions. available at ssrn 977441. hudson, h. e. (1989). overcoming the barriers of distance: telecommunications and rural development. ieee technology and society magazine, 8(4), 7-10. doi: https://doi.org/10.1109/44.44526 hudson, h. e. (199) universal access. rural telecommunications 18.5, 22-28. ijahp article: ibrar, kim, lee, rho /analytic hierarchy process model for the selection of optimal internet access technologies in rural pakistan international journal of the analytic hierarchy process 269 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.712 hudson, h. e. (2013). from rural village to global village: telecommunications for development in the information age. routledge. doi: https://doi.org/10.4324/9780203933138 international monetary fund (2018). chapter 2: digital government. fiscal monitor, april 2018: capitalizing on good times. https://www.imf.org/en/publications/fm/ issues/2018/04/06/fiscal-monitor-april-2018 doi: https://doi.org/10.5089/9781616352486.089 internet in pakistan. (n.d.). retrieved september 15, 2013, from audiences capes: https://dhsprogram.com/pubs/pdf/fr290/fr290.pdf ishizaka, a. & labib, a. (2011). review of the main developments in the analytic hierarchy process. expert systems with applications, 38(11), 14336-14345. doi: https://doi.org/10.1016/j.eswa.2011.04.143 itu (1989). rural telecommunication supplements 2. geneva: international telecommunications union. itu-d (1997). new developments in rural telecommunications, study group 2. geneva: international telecommunications union. itu-d, group 7 (2000). new technologies for rural applications, [online]. available: http://www.itu.int/pub/d-stg-rur-2001 itu-d, (2006). analysis of case studies on successful practices in telecommunications for rural and remote areas, study group 2. geneva: itu. itu, (2010). world telecommunication/ict development report 2010: monitoring the wsis targets. geneva: international telecommunications union. kawasumi, k. (2000). new technologies and solutions for rural accessibility. liao, z. (1998). a systematic integration model to support complex decision‐making in a dynamic environment. systems research and behavioral science: the official journal of the international federation for systems research, 15(1), 33-45. doi: https://doi.org/10.1002/(sici)1099-1743(199801/02)15:1%3c33::aidsres189%3e3.0.co;2-6 malladi, s. & min, k. j. (2005). decision support models for the selection of internet access technologies in rural communities. telematics and informatics, 22(3), 201-219. doi: https://doi.org/10.1016/j.tele.2004.10.001 mervana, s. & le, c. (2002). design and implementation of dsl-based access solutions. cisco press. https://dhsprogram.com/pubs/pdf/fr290/fr290.pdf http://www.itu.int/pub/d-stg-rur-2001 ijahp article: ibrar, kim, lee, rho /analytic hierarchy process model for the selection of optimal internet access technologies in rural pakistan international journal of the analytic hierarchy process 270 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.712 nazem, s. m., liu, y. h., lee, h. & shi, y. (1996). implementing telecommunications infrastructure: a rural america case. telematics and informatics, 13(1), 23-31. doi: https://doi.org/10.1016/0736-5853(95)00012-7 pakistan census (2017) available: http://www.pbs.gov.pk/sites/default/files/district_wise_census_results_cen sus_2017.pdf pirdashti, m., ghadi, a., mohammadi, m. & shojatalab, g. (2009). multi-criteria decision-making selection model with application to chemical engineering management decisions. world academy of science, engineering and technology, 49, 54-59. pta (pakistan telecommunication authority) 2017 annual report. available: https://www.pta.gov.pk/assets/media/ann_rep_2017.pdf pta (pakistan telecommunication authority) november 2019 updates. available: https://www.pta.gov.pk//en/telecom-indicators pta (chapter no 6, annual report, 2009). pakistan telecommunication authority, broadband, and value-added services [online], available: http://121.52.153.178:8080/xmlui/bitstream/handle/123456789/11577/annual%20report %202009.pdf?sequence=1&isallowed=y saaty, t. l. (1980). the analytic hierarchy process. new york: mcgraw-hill. saaty, t. l. (1990). how to make a decision: the analytic hierarchy process. european journal of operational research, 48(1), 9-26. doi: https://doi.org/10.1016/03772217(90)90057-i saaty, t. l. & vargas, l. g. (1994). decision making in economic, political, social, and technological environments with the analytic hierarchy process (vol. 7). pittsburgh, pa: rws publications. saaty, t. l. (2005). theory and applications of the analytic network process: decision making with benefits, opportunities, costs, and risks. pittsburgh, pa: rws publications. saaty, t. l. & vargas, l. g. (2007). dispersion of group judgments. mathematical and computer modelling, 46(7-8), 918-925. doi: https://doi.org/10.1016/j.mcm.2007.03.004 saaty, t. l. (2008). decision making with the analytic hierarchy process. international journal of services sciences, 1(1), 83-98. saaty & cillo, b. (2008). the encyclicon: a dictionary of complex decisions using the analytical network process (vol. 2). pittsburgh. pa: rws publications. sattar, k. (2007). a sustainable model for use of icts in rural pakistan. international journal of education and development using ict, 3(2). http://121.52.153.178:8080/xmlui/bitstream/handle/123456789/11577/annual%20report%202009.pdf?sequence=1&isallowed=y http://121.52.153.178:8080/xmlui/bitstream/handle/123456789/11577/annual%20report%202009.pdf?sequence=1&isallowed=y ijahp article: ibrar, kim, lee, rho /analytic hierarchy process model for the selection of optimal internet access technologies in rural pakistan international journal of the analytic hierarchy process 271 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.712 tam, m. c. & tummala, v. r. (2001). an application of the ahp in vendor selection of a telecommunications system. omega, 29(2), 171-182. doi: https://doi.org/10.1016/s0305-0483(00)00039-6 telecommunications policy (2015). https://usf.org.pk/userfiles/file/telecommunications%20policy%20-2015%20 approved.pdf umer, a. & harris (2017). internet landscape of pakistan. available: https://bytesforall.pk/sites/default/files/internet_landscape_pakistan_2017.pdf vakataki, o, s. okuda, a. & roeder, e. (2017). the impact of universal service funds on fixed-broadband deployment and internet adoption in asia and the pacific. asia-pacific superhighway working paper series, united nations economic and social commission for asia and the pacific, 1-52. doi: https://doi.org/10.18356/c6039be4-en yasir, a. (2017). challenges for rural pakistan and the way forward. available: https://ignite.org.pk/blog/2017/12/06/challenges-for-rural-pakistan-and-the-way-forward/ ijahp news and events: ahp/anp applications in economic policy, business, and management with the t. l. saaty decision making hackathon at icefm 2022 international journal of the analytic hierarchy process 1 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.1071 ahp/anp applications in economic policy, business, and management with the t. l. saaty decision making hackathon at icefm 2022 anna ujwary-gil institute of economics, polish academy of sciences, poland anna florek-paszkowska centrum pucp, pontificia universidad católica del perú, lima, perú artur kozioł the mayor of the city and commune of wieliczka, poland organizers of the polish-peruvian 21 st international conference economy finance management (icefm2022) entitled "new challenges in economic policy, business and management",wieliczka, poland, october 22 –25, 2022 anna ujwary-gil, anna florek-paszkowska, and artur kozioł under the patronage of the embassy of the republic of poland in lima, peru, and the embassy of peru in poland, the city of wieliczka hosted the 21 st polish-peruvian conference in the series economy-finance-management, entitled "contemporary challenge in economic policy, business and management" on october 22–25, 2022 the conference was also under the honorary patronage of the honorary consul of the republic of peru and the honorary consulate of romania in krakow. the main organizers of this conference were the institute of economics, polish academy of sciences, ie pas (dr hab. anna ujwary-gil, conference chair and head of scientific committee), the mayor of the city and commune of wieliczka (dr. artur kozioł, conference chair wieliczka), and the centrum pucp, pontificia universidad católica del perú, lima, perú (dr. anna florek-paszkowska, conference chair perú). ijahp news and events: ahp/anp applications in economic policy, business, and management with the t. l. saaty decision making hackathon at icefm 2022 international journal of the analytic hierarchy process 2 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.1071 the polish-peruvian conference was the most prominent scientific and business event of 2022. we hosted participants from 15 countries, including 68 panelists from poland, peru, the usa, spain, the czech republic, japan, iran, brazil, romania, ukraine, india, chile, and malaysia, as part of an interdisciplinary program combining two disciplines: economics and finance, as well as, management and quality science. the keynote of this conference was to create a platform for sharing knowledge and experience between representatives of the following three environments: science, business, and local governments represented by the mayors of the cities of łódź, wrocław, and sopot. the purpose of undertaking a dialogue between representatives of the worlds of science, business and local governments, as part of the conference, was to achieve common goals, such as increasing the flow of information between science and business practice, increasing the level of economic knowledge in the field of management, and deepening and disseminating models of good and proven scientific and business practices. our strategic goal during this conference was to expand the scope of the exchange of experiences between the world of science and business practice. the icefm2022 international scientific advisory committee consisted of anna ujwary-gil, institute of economics, polish academy of sciences, warsaw, poland (conference chair and head of scientific committee) and adam nalepka, university of economics, cracow, poland (honorary chair). the scientific committee consisted of wiktor adamus, jagiellonian university, cracow, poland; wiktor askanas, member of canadian competition tribunal, professor emeritus university of new brunswick, canada; tor helge aas, university of agder, norway; beatrice avolio, head of the department of graduate studies in business academic department, centrum pucp; pontificia universidad católica del peru, lima, perú; małgorzata baran, collegium civitas, warsaw, poland; krzysztof bartosik, institute of economics, polish academy of sciences, warsaw, poland; barbara błaszczyk; institute of economics, polish academy of sciences, warsaw, poland; ryszard borowiecki, cracow university of economics, poland; wojciech czakon, jagiellonian university, cracow, poland; ivano dileo, university of bari "aldo moro", italy; ondřej dvouletý, university of economics in prague, czech republic; jorge a. dulan-encalada, universidad de las américas puebla, mexico; józefa famielec, cracow university of economics, poland; anna florekpaszkowska, centrum pucp, pontificia universidad católica del perú, lima, perú; anna fornalczyk, comper, lodz, poland; jörg freiling, university of bremen, germany; marta gancarczyk, jagiellonian university, poland; bianka godlewska-dzioboń, cracow university of economics, poland; piotr jedynak, jagiellonian university, poland; beata jones, honors faculty fellow, texas christian university, usa; krzysztof klincewicz, university of warsaw, poland; artur kozioł, mayor of the commune and the city of wieliczka, poland; piotr krajewski, prof. uł, university of lodz, poland; christian lehmann, university of applied sciences and arts, hannover, germany; chaoyun liang, national taiwan university, taiwan; anna maria lis, gdansk university of technology, gdansk, poland; marek lisiński cracow university of economics, poland; grażyna leśniak-łebkowska, warsaw school of economics (sgh), poland; manuel gonzález lópez, university of santiago de compostela, spain; csaba makó,szent istván university, centre for social sciences hungarian academyof sciences, hungary; ijahp news and events: ahp/anp applications in economic policy, business, and management with the t. l. saaty decision making hackathon at icefm 2022 international journal of the analytic hierarchy process 3 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.1071 zbigniew makieła, jagiellonian university, poland; carlos mariño, research director, centrum pucp; pontificia universidad católica del peru, lima, perú; percy marquina, director general of centrum pucp; pontificia universidad católica del peru, lima, perú; mieczysław morawski, warsaw university of technology, poland; enrique mu, business management, accounting & ethics; carlow university, usa; wiktor patena, higher colleges of technology, dubai, united arab emirates; halina piekarz, frycz modrzewski academy, cracow, poland; yuji sato, chukyo university, japan; aldís g. sigurðardóttir, university of twente, the netherlands; roberto solano-mendez, universidad de las américas puebla, mexico; włodzimierz sroka, wsb university, dąbrowa górnicza, poland; kazimierz r. śliwa, frycz modrzewski academy, cracow, poland; andrzej szablewski, institute of economics, polish academy of sciences, warsaw, poland; and jon mikel zabala-iturriagagoitia, deusto business school, deusto university, spain. the ceremonial opening of the conference was performed by: conference chairs: artur kozioł (the mayor of the city and commune of wieliczka, poland) anna ujwary-gil (institute of economics, polish academy of sciences, poland) anna florek-paszkowska (centrum pucp, pontificia universidad católica del perú, lima, perú) ambassadors: hubert wieland conroy (embassy of peru in poland, warsaw) antonina magdalena śniadecka – kotarska (embassy of poland in lima, perú) tomasz grodzicki (vice-rector for medical college, jagiellonian university, poland) institutional directors: martyna kobus (institute of economics, polish academy of sciences, poland) percy marquina (centrum pucp, pontificia universidad católica del perú, lima, perú) representative of the ministry: dariusz latoszek (ministry of economic development and technology, poland) ijahp news and events: ahp/anp applications in economic policy, business, and management with the t. l. saaty decision making hackathon at icefm 2022 international journal of the analytic hierarchy process 4 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.1071 ambassador hubert wieland conroy (embassy of peru in poland, warsaw) during the official opening of icefm 2022 as part of the two-day deliberations, twelve sessions were held, including a scientific panel of honorary guests opening the conference led by anna ujwary-gil and anna florek-paszkowska. the scientific panel was attended by professor marian gorynia (president of the polish economic society, poznań university of economics), dr. percy markina (director of the centrum pucp, pontificia universidad católica del perú, lima, perú), professor of jagiellonian university wiktor adamus and professor mieczysław morawski from the warsaw university of technology. during the panel, we discussed contemporary topics related to the post-pandemic economy, challenges related to social responsibility, a smart ("compact") city and its management, and the digital transformation of smart enterprises considering recommendations for urban development. ijahp news and events: ahp/anp applications in economic policy, business, and management with the t. l. saaty decision making hackathon at icefm 2022 international journal of the analytic hierarchy process 5 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.1071 honorary guests panel chaired by anna ujwary-gil and anna florek-paszkowska the second panel was dedicated to cooperation between science and local government, which was led by dr. artur kozioł (president of the city and commune of wieliczka) and professor zbigniew makieła from the jagiellonian university and wsb university in dąbrowa górnicza. the panel was opened by a professor of cracow university of economics, stanisław mazur (rector of the cracow university of economics), presenting cooperation projects between the university and the local government, conditioning their further development. the mayor of łódź, hanna zdanowska, raised the subject of climate protection and economic benefits. jacek sutryk, mayor of wrocław, presented the subject of social innovation as a factor in the development of modern cities, and dr. jacek karnowski, mayor of sopot discussed civic self-government in the context of the well-being of citizens. the first day was focused on scientific panels and presentations of contemporary topics and problems faced by local governments. ijahp news and events: ahp/anp applications in economic policy, business, and management with the t. l. saaty decision making hackathon at icefm 2022 international journal of the analytic hierarchy process 6 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.1071 academic and city mayors panel chaired by artur kozioł and zbigniew makieła on october 23, 2022, three parallel sessions were held. parallel session 1 was chaired by enrique mu (carlow university, usa) and hannia gonzalez urango (ingenio, universitat politècnica de valència, csic, spain; latin american studies association, usa). hannia gonzalez urango presented “hard context indicators for assessing and monitoring sustainability in the private sector.” jan brzozowski and marcin stonawski (cracow university of economics, poland) presented “comparative analysis of adaptation strategies of ukrainian refugees in poland and venezuelan refugees in peru.” carlos arturo hoyos vallejo (centrum pucp, pontificia universidad católica del perú, lima, perú) presented “deep involvement of multi-stakeholders to build long-term sustainable competitive advantages: a foresight study with a public university.” jarmila duháček šebestová and šárka čemerková (silesian university in opava, school of business administration in karviná, czech republic) presented “plan or die: entrepreneurs' response to crisis situation.” zbigniew makieła (jagiellonian university, poland; wsb university, dąbrowa górnicza, poland) presented “the smart city concept and 4t capitals in the city development policies of the metropolis gz.” parallel session 2 was chaired by rafael fernández concha (centrum pucp, pontificia universidad católica del perú, lima, perú). here, małgorzata kutera (jagiellonian university, poland) presented “cryptocurrencies as a subject of financial fraud.” julianna ramirez lozano (centrum pucp, pontificia universidad católica del perú, lima, perú) presented “latin america social entrepreneur profile: leadership, culture, good communication and commitment to social/environmental issues.” vitaliy tsyganok (institute for information ijahp news and events: ahp/anp applications in economic policy, business, and management with the t. l. saaty decision making hackathon at icefm 2022 international journal of the analytic hierarchy process 7 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.1071 recording of national academy of sciences of ukraine) presented “strategic business planning technology,” and rafael fernández concha (centrum pucp, pontificia universidad católica del perú, lima, perú) presented “an analysis of the communicated identity of the online program managers (opm).” finally, parallel session 3 was chaired by aleksander jakimowicz (institute of economics, polish academy of sciences, poland). during that session, agnieszka domańska (warsaw school of economics, poland) presented “quality of life as a keynote between city smartness and sustainability. perspectives of indexes and indicators.” renata wojciechowka (warsaw school of economics, poland) “scio ergo me” (“i know therefore i am”). dominika bąkgrabowska and katarzyna grzesik (wroclaw university of economics, poland) presented “satisfying employee interests depending on the age and form of employees’ employment.” guests and participants of icefm2022, october 24, 2022 on october 24, 2022, three online parallel sessions were held. parallel online session 4 was chaired by alvaro talavera lopez (universidad del pacífico, perú). during this session, risley rengifo tello (pontificia universidad católica del perú) and alvaro talavera lopez (universidad del pacífico, perú) presented “management model for the evaluation of university performance using ahp and fuzzy inference systems.” ivan luis infanzon gutierrez (pontifical catholic university of peru) and alvaro talavera lopez (university of the pacific, peru) presented “anp model to determine a vulnerability index and earthquake mitigation policies.” govind seepersad (the university of the west indies, st. augustine, trinidad and tobago, west indies), shivani d.r. seepersad (the university of the west indies, st. augustine, trinidad and tobago, west indies), ijahp news and events: ahp/anp applications in economic policy, business, and management with the t. l. saaty decision making hackathon at icefm 2022 international journal of the analytic hierarchy process 8 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.1071 and dave goorahoo (california state university, fresno, ca, usa) presented “averting sanitary disasters in the greater caribbean area – prioritizing threats to agribusinesses in antigua and barbuda using a multi-criteria decision making approach.” vigneshkumar chellappa (indian institute of technology guwahati, india) and grzegorz ginda (agh university of science and technology, poland) presented “usefulness of crisp ahp/anp judgment scales to differentiate risk assessment.” parallel online session 5 was chaired by rafikul islam (international islamic university malaysia, kuala lumpur). here, leticia galleguillos peralta (universidad del bío-bío, chile) presented “critical factors for female entrepreneurship success through the ahp method, arauco province, chile case.” nuno torre (sao paulo state university, brasil) and valério salomon (sao paulo state university, brazil) presented “multi-criteria decision making approaches for hydraulic supplier selection – ahp application.” mohammad khalilzadeh (centrum pucp, pontificia universidad católica del perú, lima, perú); ali heidarib (pontificia universidad católica del perú, lima, peru) and sayyid ali banihashemic (iran university of science and technology, tehran, iran) presented “prioritization of the infrastructural construction projects of industrial parks with the hybrid anp-bwm method.” finally, rafikul islam presented “employee performance evaluation using the analytic hierarchy process.” parallel online session 6 was chaired by beatrice avolio (centrum pucp, pontificia universidad católica del perú, lima, perú) who presented “factors that contribute to the underrepresentation of women academics worldwide: a literature review.” maría del carmen arrieta quispe (centrum pucp, pontificia universidad católica del perú, lima, perú) presented “business school´s student satisfaction toward emergency remote teaching.” carlos vílchez-román (centrum pucp, pontificia universidad católica del perú, lima, perú) presented “the societal relevance of research output of three latin american business schools.” finally, anna kowalczyk-kroenke presented “chasing an employee. how to build cooperation with employees of the it sector?” the second day featured three business panels, where we invited business representatives from the usa, romania, and peru. the plenary business panel was opened by john saaty, son of professor thomas saaty (1926-2017), one of the most famous mathematicians in the usa, doctor honoris causa of the jagiellonian university, creator of the analytic hierarchy process (ahp) and analytic network process (anp) decisionmaking methods and their applications in solving problems of the public sector and business. the plenary business panel was chaired by rozann w. saaty (creative decision foundation, usa); john saaty (ceo and co-founder decision lens, usa); enrique mu (carlow university, usa) and anna florek-paszkowska (centrum pucp, pontificia universidad católica del perú, lima, perú) and was dedicated to “what is ahp? origins, state of the art, applications.” john saaty presented “decision making in the public sector: the inside story.” ijahp news and events: ahp/anp applications in economic policy, business, and management with the t. l. saaty decision making hackathon at icefm 2022 international journal of the analytic hierarchy process 9 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.1071 chairs of plenary business panel: rozann w. saaty (on screen), enrique mu, anna florek-paszkowska, and john saaty chairs of plenary business panel: enrique mu, anna florek-paszkowska, and john saaty (left to right) at the end of the conference, a final business session in the form of a discussion panel was led by dariusz latoszek from the ministry of economic development and technology, poland. he is chief specialist for cooperation between the countries of latin america and the caribbean and former commercial counselor at the embassy of the ijahp news and events: ahp/anp applications in economic policy, business, and management with the t. l. saaty decision making hackathon at icefm 2022 international journal of the analytic hierarchy process 10 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.1071 republic of poland in chile and former consul in peru and nigeria. a total of eleven panelists took part in the discussion including: john saaty (ceo and co-founder decision lens, usa) tomasz lubowiecki (ceo 7r sa logistics and development sector) bogusław kwiatkowski (ceo answear.com clothing sector) ștefan porojan (counsellor to the minister, economic and trade promotion office, romanian embassy in poland) juan risi (agribusiness director, prodigio company, peru) sergio hoyos (leader in sustainability in marketing and clothing in alpaca and vicuña, peru) césar ramírez (ceo eureka latam innovation & sustainability, peru) kazimierz murzyn (ceo, klaster lifescience, kraków) irena łobocka (ceo, cluster "sustainable infrastructure") marius gălbenușă (president of the romanian association for sustainable local development, romania) the discussion chaired by dariusz latoszek focused on topics such as:  why poland? why peru? reasons for establishing/developing relations with poland and peru. coincidence, conscious choice, or simple extension after dealing with other countries?  successes and challenges in business relations with the other country.  proposals to improve the relationship: the role of public institutions (including government and regional administration) and the private sector proposals for support from these institutions.  personal impressions from this conference (business aspect), expectations for the future, and suggestions for further steps in business cooperation. business session chaired by dariusz latoszek and panelists ijahp news and events: ahp/anp applications in economic policy, business, and management with the t. l. saaty decision making hackathon at icefm 2022 international journal of the analytic hierarchy process 11 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.1071 simultaneous with this conference, organizers held the tl saaty decision making hackathon, a workshop dedicated to students from all over the world. this is the second time this event has been organized and the first time in poland and europe (the first hackathon was held in pittsburgh, usa). after the ahp hackathon training conducted by the creative decision foundation, the mayor of wieliczka introduced three issues for the thirteen registered groups to solve: 1) management of social services through modern communication systems. creating an information exchange model. 2) managing a system of integrated public transport connections (railway, bus) with the assumption of increasing the participation of people using organized connections, in favor of resigning from individual transport. 3) management of activities aimed at strengthening the area of environmental protection. creation of new ecological projects with a monitoring system. while working on the business cases proposed by the mayor of wieliczka, the students analyzed real cases and proposed specific solutions. student presentations were assessed based on the teams’ ability to analyze the information, clearly define the problem, effectively use decision-making methods (ahp/anp), show creativity, and communicate the recommended course of action. the winners were selected by a panel of judges experienced in ahp/anp. among the judges were members of the international ahp/anp community composed of scientists and practitioners, as well as professionals who work with both the public sector and business partners. hackathon session with students ijahp news and events: ahp/anp applications in economic policy, business, and management with the t. l. saaty decision making hackathon at icefm 2022 international journal of the analytic hierarchy process 12 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.1071 the second ahp-hackathon training, available to students from all over the world, is an event organized in poland and europe (the first was held in pittsburgh, usa). the opening ceremony of the international competition between groups of students was performed by the mayor of wieliczka, artur kozioł, and rozann w. saaty, president of creative decision foundation, providers of prizes for the winning teams. the ahphackathon lasted three days, during which the students learned methods and techniques for solving complex problems in a non-standard way. they used hard data, personal perspectives, and experience to approach the most difficult decisions without fear by changing the method of analysis for problematic issues. in the first part of the hackathon, students learned about the methods developed by the outstanding professor thomas l. saaty (1926-2017), one of the most famous mathematicians in the usa, creator of the analytic hierarchy process (ahp) and analytic network process (anp) decisionmaking methods. these methods revolutionized decision making in the public sector and business arena. after the training conducted by the creative decision foundation (usa), thirteen registered groups of students from poland and peru got acquainted with the issues presented by the mayor of wieliczka. working on the “business case” proposed by the mayor of wieliczka, the students analyzed real cases and proposed specific solutions. the winning team and the best solved problem were announced on october 24 during icefm 2022. the judges panel included members of the international ahp/anp community and was composed of scientists and practitioners as well as professionals who work with both the public sector and business partners. in addition to the academic presentations, we also had the opportunity to visit the worldunique wieliczka salt mine, unesco. underground chapel of st. kinga in the wieliczka salt mine this conference was a unique event in poland. we hope that the next one will attract equally excellent guests, panelists, and participants. we plan to organize the 22 nd edition of icefm 2024 in peru, for which we cordially invite you! ijahp article: goyal p., kumar d., kumar v./application of multi-criteria decision analysis in the area of sustainability: a literature review international journal of the analytic hierarchy process 512 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.720 application of multi-criteria decision analysis in the area of sustainability: a literature review praveen goyal birla institute of technology and science, pilani, india praveeng23@gmail.com divesh kumar malaviya national institute of technology, jaipur, india divesh.dms@mnit.ac.in vinod kumar* indian institute of information technology, lucknow (india) vkmehta.iitr@gmail.com *corresponding author abstract multi-criteria decision analysis is a decision-making technique that is used by practitioners and frequently discussed and applied by researchers. this present study attempts to review the literature of applications of mcda techniques in the area of sustainability using citation analysis. for the data collection, the scopus database was used. with the help of related keywords, 701 research papers were identified and found suitable for analysis. these papers were then analyzed based on country, author, citation, year, and other criteria to demonstrate the pattern of research applying mcda in sustainability. the citespace tool was also used to present the clusters, citation analysis, and keywords as bursts. the findings of the present study will help future researchers better explore the field. keywords: citation analysis; citespace; multi-criteria decision analysis; sustainable development; and sustainability. 1. introduction if the phrase ‘citation analysis’ is searched on google scholar, it yields about 4,850,000 results (sheoran et al., 2018). after browsing a few pages, it becomes evident that this technique has been applied in many disciplines. the field of sustainability is also not untouched as far as the application of citation analysis is concerned. various studies have been conducted using citation analysis to gain insights from the field of sustainability (schoolman et al., 2012; tang et al., 2018). however, the authors were unable to find a single study on the application of this technique in the combined field of mcda and sustainability. mailto:praveeng23@gmail.com mailto:divesh.dms@mnit.ac.in mailto:vkmehta.iitr@gmail.com ijahp article: goyal p., kumar d., kumar v./application of multi-criteria decision analysis in the area of sustainability: a literature review international journal of the analytic hierarchy process 513 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.720 mcda has been used very frequently to solve complex decision-making issues in the shortest time possible (alitaneh, 2019). most decision-making situations involve many factors, and therefore, mcda techniques are becoming very important. mcda includes many analytical techniques such as ahp, demetal, electre, vikor, value analysis, promethee, and rough set (gonzalez & pradenas, 2019). several studies have used fuzzy extensions and hybrids to solve mcda-based problems (ghorabaee et al., 2017). it is worth mentioning here that previous studies have found a growing trend of research in sustainability (mura et al., 2018; tang et al., 2018). it has become increasingly important to take the field of sustainability deeper to provide direction to future researchers. moreover, the area of sustainability has yet to be explored to the core. therefore, the present study explores and investigates the application of mcda and sustainability through citation analysis. the paper begins with a brief review of the literature and then discusses the methodology of the study. the chosen studies are segregated based on several criteria to gain insights. the paper concludes with a discussion, implications, and directions for future research. 2. literature review past studies have explored different functional areas of sustainability using a citation analysis-based approach. recently, tang et al. (2018) broadly reviewed the field of sustainability with the help of 6,459 papers published from 2009-2018 using citation analysis. schoolman et al. (2012) also examined the interdisciplinary nature of the field of sustainability using citation analysis by segregating the literature based on three dimensions of sustainability, i.e., environmental, economic, and social. chabowski et al. (2011) explored the sustainability marketing literature from 1958-2010. liu et al. (2017) and ruiz-real et al. (2019) took the field of sustainability marketing further using a citation analysis approach and studied sustainability from the consumption and retail perspectives, respectively. fu et al. (2016) performed a citation analysis to dig deeper into sustainability operations and allied areas. all of these studies are limited because they only discover the link between sustainability and different management streams such as marketing, operations, and general management, and none of them were able to establish the connection of sustainability with any of the methods. as far as the application of citation analysis in the area of sustainability and mcda techniques is concerned, taticchi et al. (2015) studied the application of mcda to the area of the sustainable supply chain management. similarly, kumar et al. (2017) also explored the use of mcda techniques in the area of sustainable and renewable energy management. pohekar and ramachandran (2004) restricted the scope of their study to a single mcda technique, i.e., the analytical hierarchy process (ahp), to investigate the field of sustainable energy management. these studies also made some focused efforts to review the literature on sustainability and mcda. this review identified some gaps in the limited literature available on the subject. the past studies were either too broad or too narrow to study the sustainability literature. the ijahp article: goyal p., kumar d., kumar v./application of multi-criteria decision analysis in the area of sustainability: a literature review international journal of the analytic hierarchy process 514 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.720 field of sustainability has been explored in some specific directions. moreover, previous studies had a very limited scope from the perspective of mcda techniques. therefore, it is evident from the above discussion that there is a need to perform a comprehensive analysis of the literature, taking into account the application of mcda techniques in the field of sustainability. 3. methodology in this paper, we reviewed a total of 701 scientific articles published in refereed journals focusing on mcda techniques. for simplicity, this review paper focuses on “multicriteria decision analysis” as a single search item because using every technique as a search item would have increased the complexity of the results. this paper intends to classify articles based on country context, year of publication, citations, journal name, etc. the methodology for the present research has been adopted from sheoran et al. (2018). data were sought through the scopus database using keywords like “multicriteria decision analysis" or "mcda” and “sustainability” or “sustainable”. the papers were selected using the keywords sustainability or sustainable in title of document and multi criteria decision analysis or mcda in title, abstract, and keywords in the scopus database. 3.1 citespace methodology this present study sought to develop a comprehensive list of mcda articles published from 1996 to 2018 in the area of sustainability. elsevier's scopus database was used to extract the data for this study, which is “the largest abstract and citation database of peerreviewed literature: scientific journals, books, and conference proceedings. delivering a comprehensive overview of the world's research output in the fields of science, technology, medicine, social sciences, and arts and humanities, scopus features smart tools to track, analyze and visualize research.” (elsevier, 2018) the number of journals covered under the scopus database is one of the motivating factors to use it. the scopus database provides information such as author, title, abstract, affiliating institute, country, keywords, funding agency, and so on. this present study used the citespace tool to visualize research patterns and trends in mcda research. “these data can be used to conduct temporal and spatial analysis, analysis of word cooccurrence, co-authorship analysis, and co-citation analysis.” (jiang et. al, 2019, p. 1931). the articles related to mcda from 1996 to 2018 were retrieved from the scopus database. the search was based on the keywords "multi-criteria decision analysis" or "mcda". 4. analysis and findings 4.1 major subject area coverage the present research shows a significant number of papers that cover various domains. the concept of sustainability or sustainable development has implications in various subject areas including engineering, physical sciences, social sciences, medical, ijahp article: goyal p., kumar d., kumar v./application of multi-criteria decision analysis in the area of sustainability: a literature review international journal of the analytic hierarchy process 515 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.720 agricultural, etc. the highest numbers of papers have been published in the area of environmental science followed by social sciences and engineering, respectively (see table 1). the domain areas of business, management and accounting, and energy also have a significant number of publications. it is worth mentioning that some of the studies may have been cross-listed in multiple disciplines resulting in the number of individual studies exceeding the total number of studies. table 1 subject area coverage with the number of publications area (related to sustainability and mcda) number of papers environmental science 400 social sciences 213 engineering 209 energy 192 business, management and accounting 157 agricultural and biological sciences 70 computer science 63 decision sciences 57 economics, econometrics and finance 46 earth and planetary sciences 30 mathematics 20 chemical engineering 15 materials science 13 ijahp article: goyal p., kumar d., kumar v./application of multi-criteria decision analysis in the area of sustainability: a literature review international journal of the analytic hierarchy process 516 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.720 area (related to sustainability and mcda) number of papers multidisciplinary 11 biochemistry, genetics and molecular biology 9 medicine 7 chemistry 8 arts and humanities 5 psychology 5 physics and astronomy 3 immunology and microbiology 1 pharmacology, toxicology and pharmaceutics 1 4.2 publications by year as far as the evolution of sustainability (and/or sustainable development) is concerned, researchers began publishing papers in this area in 1996 (kumar et al., 2013). initially, there were very few publications (figure 1). from 1996 to 2012, an up and down trend can be observed; however, this area saw stable growth from 2011 to 2014. exponential growth in the number of publications can be seen after 2014 and surprisingly, a drop in number publications was seen in 2018. this may be due to a delay in updating publications on the scopus database. it can be safely said that there has been significant growth in the number of publications in this field in the current decade. ijahp article: goyal p., kumar d., kumar v./application of multi-criteria decision analysis in the area of sustainability: a literature review international journal of the analytic hierarchy process 517 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.720 figure 1 yearly publication distribution 4.3 publications by country this section provides a list of the number of publications by country and as many as 951 countries are represented as can be seen in figure 2. seventy-nine nationalities are represented (plus 7 undefined) in the 701 publications. as far as the affiliation of authors is concerned, india is the country that has published the most on sustainability using mcda techniques with 95 publications, followed by china (89), the united states (83), and iran (51). for example, the first article in this area (2005) was published in energy by two authors, ranjan k. bose and g. anandalingam, from india and the usa, respectively. this paper was on the topic of “sustainable urban energy-environment management with multiple objectives.” this is an example of cross-country research that is very common in the area of sustainability since it is a global issue. ijahp article: goyal p., kumar d., kumar v./application of multi-criteria decision analysis in the area of sustainability: a literature review international journal of the analytic hierarchy process 518 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.720 figure 2 publications by country 4.4 journal and citation trends appendix 1 provides information about the various journals where sustainability-related work has been published using mcda techniques. these journals belong to very diverse areas such as forestry, agriculture, energy, environment, ecology, transportation, operations research, waste management, wastewater, and many others. the journals, sustainability (switzerland), journal of cleaner production, journal of environmental management, energy, energy policy, and international journal of sustainable development and world ecology have a large number of publications in the area of sustainability combined with the mcda technique. however, the majority of the work has been published in core sustainability journals and researchers can target some core field journals. this is also evidence that the area under study is well accepted among journals from different subjects. figure 3 shows the prominence of different journals based on the number of citations. the size of the circle indicates the number of citations each journal received from other journals. similarly, the nodes represent the citation pattern among the journals. european journal of operation research, journal of cleaner production, and expert system with application are the journals that have been frequently cited by their counterpart journals. the journals energy policy, energy, omega, and sustainability also look promising in terms of the number of citations. ijahp article: goyal p., kumar d., kumar v./application of multi-criteria decision analysis in the area of sustainability: a literature review international journal of the analytic hierarchy process 519 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.720 figure 3 journal citation trends 4.5 author with the most publications thomas l. saaty is the most cited author in this field of the present area under the investigation. this is not surprising since saaty developed the concept of the ahp. figure 4 shows other authors with significant contributions in this field of research. recently, govindan, k. has become a prominent author in this area. as seen in figure 4, other authors who have contributed significantly to the area under study include sarkis, j., zahed, l.a., hwang, c.l., zhu, q., kahraman, c., seuring, s., luthra, s., singh, r.k., mathiiyazhagan, k., diabat, a., awasthi, a., etc. ijahp article: goyal p., kumar d., kumar v./application of multi-criteria decision analysis in the area of sustainability: a literature review international journal of the analytic hierarchy process 520 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.720 figure 4 authors with the most publications 4.6 most cited articles among the 701 documents under consideration, the paper titled, “quantitative models for sustainable supply chain management: developments and directions” authored by brandenburg et al. (2009) in the european journal of operational research is the most cited document with a total of 307 citations. this is a review paper that focuses on the various quantitative models in the sustainable supply chain. other papers that received more than 100 citations are shown in table 2. ijahp article: goyal p., kumar d., kumar v./application of multi-criteria decision analysis in the area of sustainability: a literature review international journal of the analytic hierarchy process 521 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.720 table 2 most cited articles publication year document title journal no. of citations 2014 quantitative models for sustainable supply chain management: developments and directions european journal of operational research 307 2007 environmental principles applicable to green supplier evaluation by using multi-objective decision analysis international journal of production research 193 2007 development of composite sustainability performance index for steel industry ecological indicators 177 2007 safe-a hierarchical framework for assessing the sustainability of agricultural systems agriculture, ecosystems and environment 157 2007 key performance indicators and assessment methods for infrastructure sustainability-a south african construction industry perspective building and environment 141 2009 technological, economic and sustainability evaluation of power plants using the analytic hierarchy process energy policy 126 2008 analysis of interactions among the barriers to energy saving in china energy policy 105 2011 advancing integrated systems modelling framework for life cycle sustainability assessment sustainability 104 ijahp article: goyal p., kumar d., kumar v./application of multi-criteria decision analysis in the area of sustainability: a literature review international journal of the analytic hierarchy process 522 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.720 publication year document title journal no. of citations 2007 developing a sustainability framework for the assessment of bioenergy systems energy policy 104 2000 multicriterion analysis for sustainable water resources planning: a case study in spain water resources management 103 4.7 most discussed keywords the keywords of an article represent the focus area of the particular research. this section discusses the top keywords used most frequently in articles in the area of sustainability using mcda techniques. other than sustainability and sustainable development, “decision making” was the most discussed keyword from 2003 to 2005 as depicted in figure 5. further, the term “mathematical model” was another most discussed keyword from 2006 to 2007. most of the keywords are related to decision making or mcda techniques. the citation burst revealed key words that increased significantly over a period of time (figure 5). keywords in the article in the combined file of articles were analyzed for their burstiness. the time interval is shown in blue and the timeline in which the keyword observed burstiness is shown in red. the red line shows the beginning and end of the burst. for example, the keyword ‘decision making’ started in 2003 and ended in 2005 with a burst strength of 4.7058. hot keywords before 2007 belong to decision sciences, conservation of natural resources, and the geographical area of asia and eurasia. the term multi-criteria decision analysis first burst in 2009. similarly, it can be observed that time-specific mcdm techniques, such as topsis, started appearing in citation bursts in 2016. the five most used keywords in the time zone view are shown in figure 6. the keywords that were used the most in each year can be easily identified. for example, “multiple criteria decision support system” was used most in the year 2007 and it was again used prominently to search papers between 2007 and 2009. moreover, this keyword was used quite considerably in 2010. in later years, this particular keyword has been used quite less than other keywords. ijahp article: goyal p., kumar d., kumar v./application of multi-criteria decision analysis in the area of sustainability: a literature review international journal of the analytic hierarchy process 523 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.720 figure 6 top five keywords during different time zones 4.8 clusters the present study identified five thematic clusters with the help of the citespace tool. the labeling of clusters and assignment of a particular number to clusters was done by citespace only, which is not always in exact chronological order. according to chen (2014), eligible clusters should have ten or more members in each cluster with a silhouette value > 0.70. silhouette values range from -1 to 1 and measure the quality of the configuration in clusters and the maximum value represents a complete solution (jiang et. al, 2019). in the present study, four eligible clusters were identified according to the above-mentioned criteria. the authors have attempted to present and explain the summary of the four thematic clusters created by citespace. ijahp article: goyal p., kumar d., kumar v./application of multi-criteria decision analysis in the area of sustainability: a literature review international journal of the analytic hierarchy process 524 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.720 figure 7 cluster analysis using citespace cluster 1: the biggest cluster (#0) has 49 members with a silhouette value of 0.89 and is categorized as the 'sustainable wireless sensor network'. the main keywords appearing in this cluster are composite sustainability performance index, fuzzy analytic hierarchy process, fuzzy multi-objective linear programming model, sustainable supply chain, and sustainable wireless sensor network. the most active citer to the cluster is srie (2015) who contributed 0.5111 in the total silhouette value of this cluster. cluster 2: the second largest cluster (#1) has 41 members with a silhouette value of 0.923. and is categorized as the 'sustainability driver'. the main keywords that appear in this cluster are interpretive structural modeling technique, composite sustainability performance index, sustainability framework, sustainable manufacturing, and sustainable supplier selection. the most active citer to the cluster is (0.4441) mehregan (2014). cluster 3: the third largest cluster (#2) has 38 members with a silhouette value of 0.945 and is categorized as a life cycle sustainability assessment. the main keywords that appeared in this cluster are sustainable development planning, disposal scenario, resources approach, intelligent transport system, and fuzzy ahp approach. the most active citer to the cluster is (0.13171) foolmaun (2013). cluster 4: the fourth largest cluster (#3) has 26 members and a silhouette value of 0.949. it is labeled as supplier selection. the main keywords that appeared in this cluster are sustainability view, order allocation, construction method selection, environmental strategic orientation, and sustainability supplier selection criteria. the most active citer to the cluster is (0.3871) badri (2017). ijahp article: goyal p., kumar d., kumar v./application of multi-criteria decision analysis in the area of sustainability: a literature review international journal of the analytic hierarchy process 525 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.720 5. discussion and conclusion the present literature review considered 701 documents for software-based analysis. this review paper first identified the trends of publication in the area of ‘sustainability with mcda’. several forms of analysis were used to understand the patterns in the area of sustainability with mcda. the findings of the study suggest that there has been substantial growth in the number of publications in this area. however, some areas like arts and humanities, chemical engineering, chemistry, physics and astronomy, psychology, mathematics, immunology, and microbiology still need to be explored in terms of the application of mcda in the field of sustainability. further, areas like management science, earth and planetary science, ecology, climate change, economics, and accounting and finance are also underexplored and can be considered for future research. future studies may also look at interdisciplinary and cross-country based studies. the area of mcda and sustainability is well regarded by journals and has seen a wide variety of publications. as far as journals related to the area under study are concerned, sustainability (switzerland) and journal of cleaner production are the top two journals with 69 and 45 articles, respectively. journal publishing trends will help readers identify journals where similar material can be found or published. moreover, the list of the top 10 most cited articles is provided to understand the paper’s contribution. the prominence of journals and authors in terms of their citations and publications will help budding researchers understand the importance of mcda in sustainability and craft their research direction in a better way. the segregation of sustainability and mcda into different clusters helps beginning researchers develop a better understanding of the field. this research paper is a bibliographic source for the area of sustainability using mcda techniques. though search results provided publications from diverse fields, it also demonstrates the applicability of mcda techniques in several areas of decision-making. this study has contributed to the literature by identifying research papers in the area of sustainability and the mcda technique. this present study has a few limitations. the authors only used two keywords, “mcda” and “multi-criteria decision analysis”. many papers that had the acronym mcda in their abstract and full text may have been omitted from the search result because the titlebased query was run to search articles for the present study. similarly, this review paper only focused on “sustainability” or “sustainable”, and many papers with related keywords like “environment” may have been missed. future researchers may address these issues and can use more keywords to refine the search results. ijahp article: goyal p., kumar d., kumar v./application of multi-criteria decision analysis in the area of sustainability: a literature review international journal of the analytic hierarchy process 526 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.720 references achillas, c., moussiopoulos, n., karagiannidis, a., banias, g., & perkoulidis, g. (2013). the use of multi-criteria decision analysis to tackle waste management problems: a literature review. waste management and research, 31(2), 115-129. doi: https://doi.org/10.1177/0734242x12470203 adiat, k. a. n., nawawi, m. n. m., & abdullah, k. (2012). assessing the accuracy of gis-based elementary multi criteria decision analysis as a spatial prediction tool a case of predicting potential zones of sustainable groundwater resources. journal of hydrology, 440-441, 75-89. doi: https://doi.org/10.1016/j.jhydrol.2012.03.028 ahmadi sani, n., babaie kafaky, s., pukkala, t., & mataji, a. (2016). integrated use of gis, remote sensing and multi-criteria decision analysis to assess ecological land suitability in multi-functional forestry. journal of forestry research, 27(5), 11271135. doi: https://doi.org/10.1007/s11676-016-0242-1 aksoy, e., & san, b. t. (2017). geographical information systems (gis) and multicriteria decision analysis (mcda) integration for sustainable landfill site selection considering dynamic data source. bulletin of engineering geology and the environment, 1-13. doi: https://doi.org/10.1007/s10064-017-1135-z aliewi, a., el-sayed, e., akbar, a., hadi, k., & al-rashed, m. (2017). evaluation of desalination and other strategic management options using multi-criteria decision analysis in kuwait. desalination, 413, 40-51. doi: https://doi.org/10.1016/j.desal.2017.03.006 alitaneh, s. (2019). theories on coefficient of variation scales triangle and normalization of different variables: a new model in development of multiple criteria decision analysis. international journal of the analytic hierarchy process, 11(2), 283-295. doi: https://doi.org/10.13033/ijahp.v11i2.565 almasri, m. n., & kaluarachchi, j. j. (2005). multi-criteria decision analysis for the optimal management of nitrate contamination of aquifers. journal of environmental management, 74(4), 365-381.doi: https://doi.org/10.1016/j.jenvman.2004.10.006 arce, m. e., saavedra, á., míguez, j. l., & granada, e. (2015). the use of greybased methods in multi-criteria decision analysis for the evaluation of sustainable energy systems: a review. renewable and sustainable energy reviews, 47, 924-932. doi: https://doi.org/10.1016/j.rser.2015.03.010 awad-núñez, s., gonzález-cancelas, n., soler-flores, f., & camarero-orive, a. (2015). how should the sustainability of the location of dry ports be measured? a proposed methodology using bayesian networks and multi-criteria decision analysis. transport, 30(3), 312-319. doi: https://doi.org/10.3846/16484142.2015.1081618 https://doi.org/10.1177/0734242x12470203 ijahp article: goyal p., kumar d., kumar v./application of multi-criteria decision analysis in the area of sustainability: a literature review international journal of the analytic hierarchy process 527 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.720 azarnivand, a., & chitsaz, n. (2015). adaptive policy responses to water shortage mitigation in the arid regions--a systematic approach based on edpsir, dematel, and mcda. environmental monitoring and assessment, 187(2): 23, 1-15. doi: https://doi.org/10.1007/s10661-014-4225-4 bachmann, t. m. (2013). towards life cycle sustainability assessment: drawing on the needs project's total cost and multi-criteria decision analysis ranking methods. international journal of life cycle assessment, 18(9), 1698-1709. doi: https://doi.org/10.1007/s11367-012-0535-3 balana, b. b., mathijs, e., & muys, b. (2010). assessing the sustainability of forest management: an application of multi-criteria decision analysis to community forests in northern ethiopia. journal of environmental management, 91(6), 1294-1304. doi: https://doi.org/10.1016/j.jenvman.2010.02.005 bhagat, s., bhardawaj, a., mittal, p., chandak, p., akhtar, m., & sharma, p. (2016). evaluating plastic waste disposal options in delhi using multi criteria decision analysis. iioab journal, 7(11specialissue), 25-35. blanquart, s. (2009). role of multicriteria decision-aid (mcda) to promote sustainable agriculture: heterogeneous data and different kinds of actors in a decision process. international journal of agricultural resources, governance and ecology, 8(2-4), 258-281. doi: https://doi.org/10.1504/ijarge.2009.026229 blattert, c., lemm, r., thees, o., lexer, m. j., & hanewinkel, m. (2017). management of ecosystem services in mountain forests: review of indicators and value functions for model based multi-criteria decision analysis. ecological indicators, 79, 391-409. doi: https://doi.org/10.1016/j.ecolind.2017.04.025 bojórquez-tapia, l. a., luna-gonzález, l., cruz-bello, g. m., gómez-priego, p., juárez-marusich, l., & rosas-pérez, i. (2011). regional environmental assessment for multiagency policy making: implementing an environmental ontology through gis-mcda. environment and planning b: urban analytics and city science, 38(3), 539-563. doi: https://doi.org/10.1068/b36129 bryce, j. m., flintsch, g., & hall, r. p. (2014). a multi criteria decision analysis technique for including environmental impacts in sustainable infrastructure management business practices. transportation research part d: transport and environment, 32, 435-445. doi: https://doi.org/10.1016/j.trd.2014.08.019 cai, y., applegate, s., yue, w., cai, j., wang, x., liu, g., & li, c. (2017). a hybrid life cycle and multi-criteria decision analysis approach for identifying sustainable development strategies of beijing's taxi fleet. energy policy, 100, 314-325. doi: https://doi.org/10.1016/j.enpol.2016.09.047 https://doi.org/10.1504/ijarge.2009.026229 ijahp article: goyal p., kumar d., kumar v./application of multi-criteria decision analysis in the area of sustainability: a literature review international journal of the analytic hierarchy process 528 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.720 calizaya, a., meixner, o., bengtsson, l., & berndtsson, r. (2010). multi-criteria decision analysis (mcda) for integrated water resources management (iwrm) in the lake poopo basin, bolivia. water resources management, 24(10), 2267-2289. doi: https://doi.org/10.1007/s11269-009-9551-x cegan, j. c., filion, a. m., keisler, j. m., & linkov, i. (2017). trends and applications of multi-criteria decision analysis in environmental sciences: literature review. environment systems and decisions, 37(2), 123-133. doi: https://doi.org/10.1007/s10669-017-9642-9 chabowski, b. r., mena, j. a., & gonzalez-padron, t. l. (2011). the structure of sustainability research in marketing, 1958–2008: a basis for future research opportunities. journal of the academy of marketing science, 39(1), 55-70. doi: https://doi.org/10.1007/s11747-010-0212-7 chen, c. (2014). the citespace manual. retrieved: january, 12, 2019. choi, s., kim, j. h., & lee, d. (2012). decision of the water shortage mitigation policy using multi-criteria decision analysis. ksce journal of civil engineering, 16(2), 247-253. doi: https://doi.org/10.1007/s12205-012-0008-z chunye, w., & delu, p. (2017). zoning of hangzhou bay ecological red line using gis-based multi-criteria decision analysis. ocean and coastal management, 139, 4250. doi: https://doi.org/10.1016/j.ocecoaman.2017.01.013 cinelli, m., coles, s. r., & kirwan, k. (2014). analysis of the potentials of multi criteria decision analysis methods to conduct sustainability assessment. ecological indicators, 46, 138-148. doi: https://doi.org/10.1016/j.ecolind.2014.06.011 cobuloglu, h. i., & büyüktahtakin, i. e. (2015). a stochastic multi-criteria decision analysis for sustainable biomass crop selection. expert systems with applications, 42(15-16), 6065-6074. doi: https://doi.org/10.1016/j.eswa.2015.04.006 convertino, m., baker, k. m., vogel, j. t., lu, c., suedel, b., & linkov, i. (2013). multi-criteria decision analysis to select metrics for design and monitoring of sustainable ecosystem restorations. ecological indicators, 26, 76-86. doi: https://doi.org/10.1016/j.ecolind.2012.10.005 datta, d., & kumar ghosh, p. (2015). evaluating sustainability of community endeavours in an indian floodplain wetland using multi-criteria decision analysis. singapore journal of tropical geography, 36(1), 38-56. doi: https://doi.org/10.1111/sjtg.12092 dimopoulou, e., tolidis, k., orfanoudakis, y., & adam, k. (2013). spatial multicriteria decision analysis for site selection of sustainable stone waste disposal. fresenius environmental bulletin, 22(7 a), 2022-2026. https://doi.org/10.1016/j.ecolind.2014.06.011 ijahp article: goyal p., kumar d., kumar v./application of multi-criteria decision analysis in the area of sustainability: a literature review international journal of the analytic hierarchy process 529 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.720 eggers, j., holmgren, s., nordström, e. lämås, t., lind, t., & öhman, k. (2017). balancing different forest values: evaluation of forest management scenarios in a multi-criteria decision analysis framework. forest policy and economics, in press. https://doi.org/10.1016/j.forpol.2017.07.002 esteves, a. m. (2008). evaluating community investments in the mining sector using multi-criteria decision analysis to integrate sia with business planning. environmental impact assessment review, 28(4-5), 338-348. doi: https://doi.org/10.1016/j.eiar.2007.09.003 esteves, a. m. (2008). mining and social development: refocusing community investment using multi-criteria decision analysis. resources policy, 33(1), 39-47. doi: https://doi.org/10.1016/j.resourpol.2008.01.002 estévez, r. a., & gelcich, s. (2015). participative multi-criteria decision analysis in marine management and conservation: research progress and the challenge of integrating value judgments and uncertainty. marine policy, 61, 1-7. doi: https://doi.org/10.1016/j.marpol.2015.06.022 favretto, n., stringer, l. c., dougill, a. j., dallimer, m., perkins, j. s., reed, m. s., mulale, k. (2016). multi-criteria decision analysis to identify dryland ecosystem service trade-offs under different rangeland land uses. ecosystem services, 17, 142151. doi: https://doi.org/10.1016/j.ecoser.2015.12.005 feng, c., & mai, y. (2016). sustainability assessment of products based on fuzzy multi-criteria decision analysis. international journal of advanced manufacturing technology, 85(1-4), 695-710. doi: https://doi.org/10.1007/s00170-015-7978-1 fernandes, i. d. s., ferreira, f. a. f., bento, p., jalali, m. s., & antónio, n. j. s. (2017). assessing sustainable development in urban areas using cognitive mapping and mcda. international journal of sustainable development and world ecology, 1-11. doi: https://doi.org/10.1080/13504509.2017.1358221 fu, x., niu, z., & yeh, m. k. (2016). research trends in sustainable operation: a bibliographic coupling clustering analysis from 1988 to 2016. cluster computing, 19(4), 2211-2223. doi: https://doi.org/10.1007/s10586-016-0624-3 ghajari, y., alesheikh, a., modiri, m., hosnavi, r., & abbasi, m. (2017). spatial modelling of urban physical vulnerability to explosion hazards using gis and fuzzy mcda. sustainability, 9(7), 1274. doi: https://doi.org/10.3390/su9071274 gigović, l., pamučar, d., lukić, d., & marković, s. (2016). gis-fuzzy dematel mcda model for the evaluation of the sites for ecotourism development: a case study of “dunavski ključ” region, serbia. land use policy, 58, 348-365. doi: https://doi.org/10.1016/j.landusepol.2016.07.030 https://doi.org/10.1016/j.forpol.2017.07.002 https://doi.org/10.3390/su9071274 ijahp article: goyal p., kumar d., kumar v./application of multi-criteria decision analysis in the area of sustainability: a literature review international journal of the analytic hierarchy process 530 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.720 gonzalez, f., & pradenas, l. (2019). multi-criteria analysis to improve the service in gas stations. international journal of the analytic hierarchy process, 11(1), 67-90. doi: https://doi.org/10.13033/ijahp.v10i3.601 hatefi, s. m., & torabi, s. a. (2010). a common weight mcda-dea approach to construct composite indicators. ecological economics, 70(1), 114-120. doi: https://doi.org/10.1016/j.ecolecon.2010.08.014 huang, i. b., keisler, j., & linkov, i. (2011). multi-criteria decision analysis in environmental sciences: ten years of applications and trends. science of the total environment, 409(19), 3578-3594. doi: https://doi.org/10.1016/j.scitotenv.2011.06.022 hyde, k. m., & maier, h. r. (2006). distance-based and stochastic uncertainty analysis for multi-criteria decision analysis in excel using visual basic for applications. environmental modelling and software, 21(12), 1695-1710. doi: https://doi.org/10.1016/j.envsoft.2005.08.004 hyde, k. m., maier, h. r., & colby, c. b. (2005). a distance-based uncertainty analysis approach to multi-criteria decision analysis for water resource decision making. journal of environmental management, 77(4), 278-290. doi: https://doi.org/10.1016/j.jenvman.2005.06.011 jeong, j. s., garcía-moruno, l., hernández-blanco, j., & sánchez-ríos, a. (2016). planning of rural housings in reservoir areas under (mass) tourism based on a fuzzy dematel-gis/mcda hybrid and participatory method for alange, spain. habitat international, 57, 143-153. doi: https://doi.org/10.1016/j.habitatint.2016.07.008 jhariya, d. c., kumar, t., gobinath, m., diwan, p., & kishore, n. (2016). assessment of groundwater potential zone using remote sensing, gis and multi criteria decision analysis techniques. journal of the geological society of india, 88(4), 481-492. doi: https://doi.org/10.1007/s12594-016-0511-9 jiang, y., ritchie, b. w., & benckendorff, p. (2019). bibliometric visualisation: an application in tourism crisis and disaster management research. current issues in tourism, 22(16), 1925-1957. doi: https://doi.org/10.1080/13683500.2017.1408574 khalili, n. r., & duecker, s. (2013). application of multi-criteria decision analysis in design of sustainable environmental management system framework. journal of cleaner production, 47, 188-198. doi: https://doi.org/10.1016/j.jclepro.2012.10.044 klein, s. j. w., & whalley, s. (2015). comparing the sustainability of u.s. electricity options through multi-criteria decision analysis. energy policy, 79, 127149. doi: https://doi.org/10.1016/j.enpol.2015.01.007 https://doi.org/10.13033/ijahp.v10i3.601 https://doi.org/10.1016/j.ecolecon.2010.08.014 https://doi.org/10.1016/j.jenvman.2005.06.011 https://doi.org/10.1016/j.jclepro.2012.10.044 https://doi.org/10.1016/j.enpol.2015.01.007 ijahp article: goyal p., kumar d., kumar v./application of multi-criteria decision analysis in the area of sustainability: a literature review international journal of the analytic hierarchy process 531 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.720 kulkarni, s. h., jirage, b. j., & anil, t. r. (2017). alternative energy options for india—a multi-criteria decision analysis to rank energy alternatives using analytic hierarchy process and fuzzy logic with an emphasis to distributed generation. distributed generation and alternative energy journal, 32(2), 29-55. doi: https://doi.org/10.1080/21563306.2017.11869108 kumar, a., sah, b., singh, a. r., deng, y., he, x., kumar, p., & bansal, r. c. (2017). a review of multi criteria decision making (mcdm) towards sustainable renewable energy development. renewable and sustainable energy reviews, 69, 596-609. doi: https://doi.org/10.1016/j.rser.2016.11.191 kumar, v., rahman, z., & kazmi, a. a. (2013). sustainability marketing strategy: an analysis of recent literature. global business review, 14(4), 601-625. doi: https://doi.org/10.1177/0972150913501598 liu, y., qu, y., lei, z., & jia, h. (2017). understanding the evolution of sustainable consumption research. sustainable development, 25(5), 414-430. doi: https://doi.org/10.1002/sd.1671 martín-gamboa, m., iribarren, d., garcía-gusano, d., & dufour, j. (2017). a review of life-cycle approaches coupled with data envelopment analysis within multi-criteria decision analysis for sustainability assessment of energy systems. journal of cleaner production, 150, 164-174. doi: https://doi.org/10.1016/j.jclepro.2017.03.017 maxim, a. (2014). sustainability assessment of electricity generation technologies using weighted multi-criteria decision analysis. energy policy, 65, 284-297. doi: https://doi.org/10.1016/j.enpol.2013.09.059 mendoza, g. a., & martins, h. (2006). new modelling paradigms in using multicriteria decision analysis for sustainable forest management. usda forest service general technical report pnw, (688). doi: https://doi.org/10.1016/j.foreco.2006.03.023 modica, g., laudari, l., barreca, f., & fichera, c. r. (2014). a gis-mcda based model for the suitability evaluation of traditional grape varieties: the case-study of 'mantonico' grape (calabria, italy). international journal of agricultural and environmental information systems, 5(3), 1-16. doi: https://doi.org/10.4018/ijaeis.2014070101 monprapussorn, s., thaitakoo, d., watts, d. j., & banomyong, r. (2009). multi criteria decision analysis and geographic information system framework for hazardous waste transport sustainability. journal of applied sciences, 9(2), 268-277. doi: https://doi.org/10.3923/jas.2009.268.277 https://doi.org/10.1080/21563306.2017.11869108 https://doi.org/10.1016/j.rser.2016.11.191 https://doi.org/10.1177/0972150913501598 https://doi.org/10.1002/sd.1671 https://doi.org/10.1016/j.enpol.2013.09.059 https://doi.org/10.1016/j.foreco.2006.03.023 https://doi.org/10.4018/ijaeis.2014070101 ijahp article: goyal p., kumar d., kumar v./application of multi-criteria decision analysis in the area of sustainability: a literature review international journal of the analytic hierarchy process 532 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.720 mura, m., longo, m., micheli, p., & bolzani, d. (2018). the evolution of sustainability measurement research. international journal of management reviews, 20(3), 661-695. doi: https://doi.org/10.1111/ijmr.12179 musakwa, w. (2018). identifying land suitable for agricultural land reform using gis-mcda in south africa. environment, development and sustainability, 20(5), 2281-2299. doi: https://doi.org/10.1007/s10668-017-9989-6 onu, p. u., quan, x., xu, l., orji, j., & onu, e. (2017). evaluation of sustainable acid rain control options utilizing a fuzzy topsis multi-criteria decision analysis model frame work. journal of cleaner production, 141, 612-625. doi: https://doi.org/10.1016/j.jclepro.2016.09.065 ouma, y. o., kipkorir, e. c., & tateishi, r. (2011). mcda-gis integrated approach for optimized landfill site selection for growing urban regions: an application of neighborhood-proximity analysis. annals of gis, 17(1), 43-62. doi: https://doi.org/10.1080/19475683.2011.558021 pietersen, k. (2006). multiple criteria decision analysis (mcda): a tool to support sustainable management of groundwater resources in south africa. water sa, 32(2), 119-128. doi: https://doi.org/10.4314/wsa.v32i2.5242 pohekar, s. d., & ramachandran, m. (2004). application of multi-criteria decision making to sustainable energy planning—a review. renewable and sustainable energy reviews, 8(4), 365-381. doi: https://doi.org/10.1016/j.rser.2003.12.007 rezaee, a., mortazavi, s. a., gholamreza, p., & sadegh, k. (2014). evaluation and comparison of the sustainability level of agronomy crops under drought condition by using mcda in the east of zayandeh-rud river basin. journal of environmental studies, 40(2), 529-540. romano, g., dal sasso, p., trisorio liuzzi, g., & gentile, f. (2015). multi-criteria decision analysis for land suitability mapping in a rural area of southern italy. land use policy, 48, 131-143. doi: https://doi.org/10.1016/j.landusepol.2015.05.013 rosén, l., back, p., söderqvist, t., norrman, j., brinkhoff, p., norberg, t., & döberl, g. (2015). score: a novel multi-criteria decision analysis approach to assessing the sustainability of contaminated land remediation. science of the total environment, 511, 621-638. doi: https://doi.org/10.1016/j.scitotenv.2014.12.058 ruiz-real, j., uribe-toril, j., gázquez-abad, j., & de pablo valenciano, j. (2019). sustainability and retail: analysis of global research. sustainability, 11(1), 14. doi: https://doi.org/10.3390/su11010014 https://doi.org/10.1111/ijmr.12179 https://doi.org/10.1007/s10668-017-9989-6 https://doi.org/10.1016/j.landusepol.2015.05.013 ijahp article: goyal p., kumar d., kumar v./application of multi-criteria decision analysis in the area of sustainability: a literature review international journal of the analytic hierarchy process 533 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.720 rybarczyk, g., & wu, c. (2010). bicycle facility planning using gis and multicriteria decision analysis. applied geography, 30(2), 282-293. doi: https://doi.org/10.1016/j.apgeog.2009.08.005 santos, j., pagsuyoin, s. a., & latayan, j. (2016). a multi-criteria decision analysis framework for evaluating point-of-use water treatment alternatives. clean technologies and environmental policy, 18(5), 1263-1279. doi: https://doi.org/10.1007/s10098-015-1066-y scholten, l., schuwirth, n., reichert, p., & lienert, j. (2015). tackling uncertainty in multi-criteria decision analysis an application to water supply infrastructure planning. european journal of operational research, 242(1), 243-260. doi: https://doi.org/10.1016/j.ejor.2014.09.044 schoolman, e. d., guest, j. s., bush, k. f., & bell, a. r. (2012). how interdisciplinary is sustainability research? analyzing the structure of an emerging scientific field. sustainability science, 7(1), 67-80. doi: https://doi.org/10.1007/s11625-011-0139-z schwenk, w. s., donovan, t. m., keeton, w. s., & nunery, j. s. (2012). carbon storage, timber production, and biodiversity: comparing ecosystem services with multi-criteria decision analysis. ecological applications, 22(5), 1612-1627. doi: https://doi.org/10.1890/11-0864.1 sen, p., roy, m., & pal, p. (2017). evaluation of environmentally conscious manufacturing programs using a three-hybrid multi-criteria decision analysis method. ecological indicators, 73, 264-273. doi: https://doi.org/10.1016/j.ecolind.2016.09.049 seo, j. p., cho, w., & cheong, t. s. (2015). development of priority setting process for the small stream restoration projects using multi criteria decision analysis. journal of hydroinformatics, 17(2), 211-225. doi: https://doi.org/10.2166/hydro.2014.058 serna, j., díaz martinez, e. n., narváez rincón, p. c., camargo, m., gálvez, d., & orjuela, á. (2016). multi-criteria decision analysis for the selection of sustainable chemical process routes during early design stages. chemical engineering research and design, 113, 28-49. doi: https://doi.org/10.1016/j.cherd.2016.07.001 sheoran, m., kumar, d., kumar, v., & verma, d. (2018). understanding the trends of marketing research and its future directions: a citation analysis. the bottom line, 31(3/4), 191-207. doi: https://doi.org/10.1108/bl-04-2018-0022 sinan erzurumlu, s., & erzurumlu, y. o. (2015). sustainable mining development with community using design thinking and multi-criteria decision analysis. resources policy, 46, 6-14. doi: https://doi.org/10.1016/j.resourpol.2014.10.001 https://doi.org/10.1016/j.apgeog.2009.08.005 ijahp article: goyal p., kumar d., kumar v./application of multi-criteria decision analysis in the area of sustainability: a literature review international journal of the analytic hierarchy process 534 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.720 sparrevik, m., barton, d. n., bates, m. e., & linkov, i. (2012). use of stochastic multi-criteria decision analysis to support sustainable management of contaminated sediments. environmental science and technology, 46(3), 1326-1334. doi: https://doi.org/10.1021/es202225x talukder, b., blay-palmer, a., hipel, k. w., & vanloon, g. w. (2017). elimination method of multi-criteria decision analysis (mcda): a simple methodological approach for assessing agricultural sustainability. sustainability (switzerland), 9(2), 287. doi: https://doi.org/10.3390/su9020287 tang, m., liao, h., wan, z., herrera-viedma, e., & rosen, m. (2018). ten years of sustainability (2009 to 2018): a bibliometric overview. sustainability, 10(5), 1655. doi: https://doi.org/10.3390/su10051655 taticchi, p., garengo, p., nudurupati, s. s., tonelli, f., & pasqualino, r. (2015). a review of decision-support tools and performance measurement and sustainable supply chain management. international journal of production research, 53(21), 6473-6494. doi: https://doi.org/10.1080/00207543.2014.939239 uhde, b., hahn, a., griess, v. c., & knoke, t. (2015). hybrid mcda methods to integrate multiple ecosystem services in forest management planning: a critical review. environmental management, 56(2), 373-388. doi: https://doi.org/10.1007/s00267-015-0503-3 vo, t. t. q., xia, a., rogan, f., wall, d. m., & murphy, j. d. (2017). sustainability assessment of large-scale storage technologies for surplus electricity using group multi-criteria decision analysis. clean technologies and environmental policy, 19(3), 689-703. doi: https://doi.org/10.1007/s10098-016-1250-8 volchko, y., norrman, j., rosén, l., bergknut, m., josefsson, s., söderqvist, t., tysklind, m. (2014). using soil function evaluation in multi-criteria decision analysis for sustainability appraisal of remediation alternatives. science of the total environment, 485-486(1), 785-791. doi: https://doi.org/10.1016/j.scitotenv.2014.01.087 volkart, k., weidmann, n., bauer, c., & hirschberg, s. (2017). multi-criteria decision analysis of energy system transformation pathways: a case study for switzerland. energy policy, 106, 155-168. doi: https://doi.org/10.1016/j.enpol.2017.03.026 wang, h. (2015). a generalized mcda-dea (multi-criterion decision analysis-data envelopment analysis) approach to construct slacks-based composite indicator. energy, 80, 114-122. doi: https://doi.org/10.1016/j.energy.2014.11.051 wang, j. -., jing, y. -., zhang, c. -., & zhao, j. -. (2009). review on multi-criteria decision analysis aid in sustainable energy decision-making. renewable and https://doi.org/10.1016/j.scitotenv.2014.01.087 https://doi.org/10.1016/j.enpol.2017.03.026 ijahp article: goyal p., kumar d., kumar v./application of multi-criteria decision analysis in the area of sustainability: a literature review international journal of the analytic hierarchy process 535 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.720 sustainable energy reviews, 13(9), 2263-2278. doi: https://doi.org/10.1016/j.energy.2014.11.051 wilkens, i., & schmuck, p. (2012). transdisciplinary evaluation of energy scenarios for a german village using multi-criteria decision analysis. sustainability, 4(4), 604629. doi: https://doi.org/10.3390/su4040604 wolfslehner, b., & seidl, r. (2010). harnessing ecosystem models and multi-criteria decision analysis for the support of forest management. environmental management, 46(6), 850-861. doi: https://doi.org/10.1007/s00267-009-9414-5 zheng, j., egger, c., & lienert, j. (2016). a scenario-based mcda framework for wastewater infrastructure planning under uncertainty. journal of environmental management, 183, 895-908. doi: https://doi.org/10.1016/j.jenvman.2016.09.027 ijahp article: goyal p., kumar d., kumar v./application of multi-criteria decision analysis in the area of sustainability: a literature review international journal of the analytic hierarchy process 536 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.720 appendix list of journals with number of publications journal name number of publications sustainability (switzerland) 69 journal of cleaner production 45 journal of environmental management 18 energy 12 energy policy 12 international journal of sustainable development and world ecology 10 ecological indicators 8 resources conservation and recycling 8 building and environment 7 clean technologies and environmental policy 7 environmental modelling and software 7 renewable energy 7 resources policy 7 science of the total environment 7 international journal of life cycle assessment 6 ijahp article: goyal p., kumar d., kumar v./application of multi-criteria decision analysis in the area of sustainability: a literature review international journal of the analytic hierarchy process 537 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.720 journal name number of publications international journal of production research 6 international journal of sustainable engineering 6 marine policy 6 sustainable cities and society 6 water resources management 6 benchmarking 5 business strategy and the environment 5 energies 5 environmental management 5 global journal of flexible systems management 5 land use policy 5 biomass and bioenergy 4 cities 4 ecological economics 4 environment development and sustainability 4 environmental monitoring and assessment 4 environmental science and policy 4 ijahp article: goyal p., kumar d., kumar v./application of multi-criteria decision analysis in the area of sustainability: a literature review international journal of the analytic hierarchy process 538 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.720 journal name number of publications european journal of operational research 4 expert systems with applications 4 international journal of advanced manufacturing technology 4 international journal of business innovation and research 4 journal of civil engineering and management 4 journal of modelling in management 4 management of environmental quality an international journal 4 transport 4 transportation research (part d)transport and environment 4 applied energy 3 applied mathematical modelling 3 applied soft computing journal 3 automation in construction 3 computers and electronics in agriculture 3 industrial management and data systems 3 international journal of environmental research and public health 3 international journal of environmental science and technology 3 ijahp article: goyal p., kumar d., kumar v./application of multi-criteria decision analysis in the area of sustainability: a literature review international journal of the analytic hierarchy process 539 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.720 journal name number of publications journal of environmental planning and management 3 journal of infrastructure systems 3 journal of the operational research society 3 mathematical problems in engineering 3 ocean and coastal management 3 production planning and control 3 progress in industrial ecology 3 sustainability 3 sustainable production and consumption 3 technological forecasting and social change 3 applied sciences switzerland 2 aquaculture international 2 arabian journal for science and engineering 2 archives of civil and mechanical engineering 2 business process management journal 2 chemical engineering transactions 2 competitiveness review 2 ijahp article: goyal p., kumar d., kumar v./application of multi-criteria decision analysis in the area of sustainability: a literature review international journal of the analytic hierarchy process 540 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.720 journal name number of publications computers and industrial engineering 2 construction and building materials 2 desalination 2 energy and buildings 2 environmental earth sciences 2 environmental science and pollution research 2 forest ecology and management 2 integrated environmental assessment and management 2 international journal of applied decision sciences 2 international journal of applied engineering research 2 international journal of logistics systems and management 2 international journal of process management and benchmarking 2 international journal of procurement management 2 international journal of strategic property management 2 international journal of sustainable society 2 international journal of sustainable transportation 2 journal of air transport management 2 ijahp article: goyal p., kumar d., kumar v./application of multi-criteria decision analysis in the area of sustainability: a literature review international journal of the analytic hierarchy process 541 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.720 journal name number of publications journal of environmental studies 2 journal of performance of constructed facilities 2 journal of sustainable tourism 2 journal of the american water resources association 2 journal of urban planning and development 2 journal of water supply research and technology aqua 2 management research review 2 nature environment and pollution technology 2 omega united kingdom 2 procedia manufacturing 2 renewable and sustainable energy reviews 2 shengtai xuebao acta ecologica sinica 2 stochastic environmental research and risk assessment 2 structure and infrastructure engineering 2 technological and economic development of economy 2 uncertain supply chain management 2 waste management and research 2 ijahp article: goyal p., kumar d., kumar v./application of multi-criteria decision analysis in the area of sustainability: a literature review international journal of the analytic hierarchy process 542 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.720 journal name number of publications water international 2 water policy 2 water switzerland 2 wit transactions on ecology and the environment 2 acs sustainable chemistry and engineering 1 advanced science letters 1 advances in natural and applied sciences 1 african journal of agricultural research 1 agricultural economics 1 agriculture ecosystems and environment 1 agroforestry systems 1 agronomy for sustainable development 1 alexandria engineering journal 1 american journal of applied sciences 1 american journal of environmental sciences 1 annals of nuclear energy 1 annals of operations research 1 ijahp article: goyal p., kumar d., kumar v./application of multi-criteria decision analysis in the area of sustainability: a literature review international journal of the analytic hierarchy process 543 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.720 journal name number of publications applied ecology and environmental research 1 applied geography 1 aquaculture economics and management 1 arabian journal of geosciences 1 archives of environmental protection 1 archives of mining sciences 1 arpn journal of engineering and applied sciences 1 asia pacific journal of tourism research 1 asian journal of scientific research 1 australian journal of civil engineering 1 biofuels bioproducts and biorefining 1 bulletin of marine science 1 cirp journal of manufacturing science and technology 1 carpathian journal of earth and environmental sciences 1 case studies on transport policy 1 chemical engineering research and design 1 chinese geographical science 1 ijahp article: goyal p., kumar d., kumar v./application of multi-criteria decision analysis in the area of sustainability: a literature review international journal of the analytic hierarchy process 544 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.720 journal name number of publications chinese journal of population resources and environment 1 civil engineering and environmental systems 1 coastal management 1 cogent business and management 1 computational economics 1 computers and operations research 1 computers and structures 1 computers environment and urban systems 1 corporate ownership and control 1 croatian journal of forest engineering 1 decision support systems 1 distributed generation and alternative energy journal 1 e a m ekonomie a management 1 ecological applications 1 ecological modelling 1 ecology environment and conservation 1 ecosystem services 1 ijahp article: goyal p., kumar d., kumar v./application of multi-criteria decision analysis in the area of sustainability: a literature review international journal of the analytic hierarchy process 545 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.720 journal name number of publications ecosystems 1 educational technology and society 1 energy conversion and management 1 energy education science and technology (part a)energy science and research 1 engineering economics 1 environment international 1 environment systems and decisions 1 environmental and engineering geoscience 1 environmental impact assessment review 1 ijahp article: otay, kahraman/solar pv power plant location selection using a z-fuzzy number based ahp international journal of the analytic hierarchy process 409 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.540 solar pv power plant location selection using a zfuzzy number based ahp i̇rem otay istanbul okan university, faculty of engineering industrial engineering department, 34959 akfirat-tuzla istanbul, turkey irem.otay@okan.edu.tr cengiz kahraman istanbul technical university, faculty of management industrial engineering department, 34367 maçka istanbul, turkey kahramanc@itu.edu.tr abstract one of the most used renewable energy systems to produce clean and sustainable energy are solar energy photovoltaic (pv) plants. the selection among solar energy pv plant location alternatives requires a multi-criteria decision making approach with several conflicting and linguistic criteria. the assessment process is generally done in a vague and imprecise environment. fuzzy set theory is often very beneficial for evaluating the subjective judgments of decision makers. the analytic hierarchy process is the most used multi-criteria decision making method in the world because of its simplicity and efficiency. in this paper, we select a location for a solar energy pv plant using a 4-level hierarchy. we consider several criteria and sub-criteria including initial cost, maintenance cost, slope and distance to highways. a z-fuzzy number is a relatively new concept in fuzzy set theory that enables one to circumvent the limitations of ordinary fuzzy numbers. z-fuzzy numbers can be viewed as a combination of crisp numbers, intervals, fuzzy numbers and random numbers because of their generality. they give a better representation than ordinary fuzzy numbers. this study solves the multi-criteria solar pv power plant location selection problem with a z-fuzzy based ahp method. to check the applicability of the method proposed here, a real-life case study from turkey is presented and solved. keywords: solar pv power plant; location selection; fuzzy ahp; z-fuzzy number; multicriteria; uncertainty 1. introduction the location selection for a solar photovoltaic (pv) power plant is a common problem in a sustainable and renewable energy supply chain. a pv power plant is a pv system composed of a lot of pv cells that convert energy from light directly to electricity. the main steps in a solar pv project are as follows: i. concept development and plant site mailto:irem.otay@okan.edu.tr mailto:kahramanc@itu.edu.tr https://www.sciencedirect.com/science/article/pii/s1364032117312844 http://www.wiki-zero.co/index.php?q=ahr0chm6ly9lbi53awtpcgvkaweub3jnl3dpa2kvughvdg92b2x0ywljx3n5c3rlbq ijahp article: otay, kahraman/solar pv power plant location selection using a z-fuzzy number based ahp international journal of the analytic hierarchy process 410 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.540 identification, ii. pre-feasibility study, iii. feasibility study, iv. legal and financial issues and contracts. v. engineering, construction and commercial operations. the location of a photovoltaic (pv) power plant is a critical issue because of its direct impact on the performance, economic, environmental and social aspects of the plant. in the literature, there are many valuable studies about the problem of power plant location. for instance, khan and rathi (2014) focused on the problem of site selection for a solar pv power plant in india that employed a geographical information system (gis) to solve the problem. samanlioglu and ayağ (2017) used an integrated fuzzy approach by integrating the analytic hierarchy process (ahp) and promethee ii to locate a power plant in turkey. garni and awasthi (2018) conducted a literature review of 50 papers on selecting locations for solar pv power plants. the results show that the analytical hierarchy process and its extensions are the most commonly used methodology, followed by overlay tool analysis in a gis environment. ozdemir and sahin (2018) studied the location selection for a solar pv power plant location using a photovoltaic geographical information system (pvgis) and ahp. as mentioned above, the first step in a solar pv project is identifying the site for a plant. this requires several criteria and alternatives to be considered under uncertainty. the uncertainty arises in assessing the alternatives with respect to the various criteria because linguistic assessments are preferred by decision makers (dm) to exact numerical evaluations. linguistic assessments can best be treated by fuzzy set theory according to the literature. each linguistic assessment is represented by a fuzzy number having a membership function that may be linear or nonlinear. fuzzy set theory has been extended to several types since the original fuzzy set theory that emerged in 1965. figure 1 summarizes the progress of fuzzy set theory from 1965 to today. the main aim of all the extensions is to provide a new point of view when defining the membership function. membership functions of ordinary fuzzy sets are defined by a membership degree and a non-membership degree whose sum is exactly equal to 1. membership functions of type-2 fuzzy sets are represented by three dimensional graphs with an upper function and a lower function. membership functions of intuitionistic fuzzy sets are represented by a membership degree and a non-membership degree whose sum is at most equal to 1. if the sum is less than 1, the difference is called the hesitancy of the dm. membership functions of pythagorean fuzzy sets offer a larger domain to the dm since their squared sum is at most equal to 1. membership functions of neutrosophic fuzzy sets are composed of truthiness, indeterminacy and falsity degrees whose sum might be between 0 and 3. hesitant fuzzy sets are different from other fuzzy sets since they offer techniques for treating more than one possible membership degree for a certain x value. https://content.iospress.com/search?q=author%3a%28%22aya%c4%9f,%20zeki%22%29 ijahp article: otay, kahraman/solar pv power plant location selection using a z-fuzzy number based ahp international journal of the analytic hierarchy process 411 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.540 figure 1 extensions of fuzzy sets within the framework of ordinary fuzzy sets, zadeh (2011) proposed the z-fuzzy number which is an ordered pair of fuzzy numbers (a, b). the first component a represents the fuzzy restriction while the second component b is the reliability of the first component. researchers claim z-numbers perform better when describing human judgments and dealing with uncertainty than traditional fuzzy numbers since they can handle restraint and reliability functions (deng & chan, 2011). in this paper, we convert the linguistic assessments to z-fuzzy numbers and evaluate the pv power plant location alternatives based on the ahp. to the best of our knowledge, this is the first study employing a zfuzzy number-based ahp method for solving the solar pv power plant location problem. the rest of the paper is organized as follows: section 2 gives a literature review of fuzzy ahp; section 3 explains z-fuzzy numbers, including their arithmetic operations; section 4 gives the proposed method z-fuzzy ahp step-by-step; section 5 is an application of the proposed method to a pv plant location problem; section 6 presents the conclusions and directions for further research. 2. fuzzy ahp in the literature fuzzy extensions of ahp have been obtained by using fuzzy numbers. recently, ordinary fuzzy numbers have been extended to several different types of fuzzy numbers such as intuitionistic fuzzy numbers, pythagorean fuzzy numbers and type-2 fuzzy numbers. these extensions have allowed new fuzzy ahp extensions to be developed in the literature that are briefly summarized as follows. ijahp article: otay, kahraman/solar pv power plant location selection using a z-fuzzy number based ahp international journal of the analytic hierarchy process 412 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.540 2.1 ordinary fuzzy ahp ordinary fuzzy ahp methods have been proposed by various authors. the first one was introduced by van laarhoven and pedrycz (1983). they used triangular fuzzy numbers and lootsma’s logarithmic least squares method for deriving fuzzy weights and fuzzy performance scores of alternatives. buckley (1985) used trapezoidal fuzzy numbers and derived fuzzy weights and fuzzy performance scores by using a geometric mean method. boender et al. (1989) modified van laarhoven and pedrycz’s (1983) method and proposed a more robust approach to the normalization of the local priorities. chang (1996) proposed an extent analysis method for deriving priorities from comparison matrices whose elements are defined as triangular fuzzy numbers. the drawback of this method is that it is possible to obtain the value of zero for initial weights or local priorities for some elements of the decision structure (wang et al., 2008). such a computed zero local priority may cause some of the information to not be considered in the calculations (li et al., 2008). cheng (1997) proposed a fuzzy ahp method based on both probability and possibility measures in which performance scores are represented by membership functions and the aggregate weights are calculated by using entropy concepts. mikhailov (2003) proposed a fuzzy extension of ahp which obtains crisp priorities based on an α-cut of fuzzy numbers by using a technique called fuzzy preference programming (fpp). the main drawback of this approach is that each comparison matrix must be constructed as an individual fpp model which increases the complexity of the solution (yu & cheng, 2007). 2.2 type-2 fuzzy ahp the concept of a type-2 fuzzy set was introduced by zadeh (1975) as an extension to an ordinary fuzzy set called a type-1 fuzzy set. kahraman et al. (2014) developed an interval type-2 (it2) fuzzy ahp method together with a new ranking method for type-2 fuzzy sets and then applied the method to supplier selection. kilic and kaya (2015) proposed a new evaluation model for investment projects for development agencies operating in turkey to address the ambiguities and relativities in real world scenarios by using type-2 fuzzy sets and crisp sets simultaneously. abdullah and najib (2014a) proposed a new fuzzy ahp characterized by it2 fs for linguistic variables. it was different from the typical fahp which directly utilized trapezoidal type-1 fuzzy numbers. this method introduced it2 fs to enhance judgments in the fuzzy decision-making environment. this new model included linguistic variables in it2 fs and a rank-value method for normalizing upper and lower memberships of it2 fs. 2.3 intuitionistic fuzzy ahp atanassov (1986)’s intuitionistic fuzzy sets (ifss) include the membership value as well as the non-membership value for describing any x in x such that the sum of membership and non-membership is at most equal to 1. sadiq and tesfamariam (2009) applied the concept of ifs to ahp which is called if-ahp to handle both vagueness and ambiguity related uncertainties in the environmental decision making process. abdullah et al. (2009) applied ifs to the ahp method called if-ahp to quantify vagueness uncertainties in ahp using ifs for the decision-making problem. the authors constructed several linear programming models to generate optimal weights for attributes. wang et al. (2011) proposed an if-ahp method in which the decision information was represented by intuitionistic fuzzy values. the method synthesizes the eigenvectors of the intuitionistic fuzzy comparison matrix in which all the decision information is intuitionistic fuzzy. ijahp article: otay, kahraman/solar pv power plant location selection using a z-fuzzy number based ahp international journal of the analytic hierarchy process 413 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.540 zhang et al. (2011) discussed the approximation of ifs to fuzzy sets based on the relationships between ifs and fuzzy sets. they also showed that the intuitionistic fuzzy complementary judgement matrix approximates the fuzzy complementary judgement matrix and proved that the consistency adjustments of both are equal. based on these results, the authors proposed an ahp method based on an intuitionistic judgment matrix. feng et al. (2012) proposed a dempster shafer – ahp (ds-ahp) method combined with intuitionistic fuzzy information. the expected utility was used to transform a decision matrix having intuitionistic fuzzy information obtained from the assessments and then a non-linear optimization model was used to combine the attributes. wu et al. (2013) proposed an interval valued intuitionistic fuzzy ahp (ivif-ahp) method for mcdm problems developing a score function for interval valued intuitionistic fuzzy numbers (ivifns) and introducing some new concepts such as antisymmetric interval matrix, the transfer interval matrix and the approximate optimal transfer matrix. kaur (2014) proposed a triangular intuitionistic fuzzy number based (tifn) approach for the vendor selection problem using ahp. xu and liao (2014) proposed a new way to check the consistency of the intuitionistic preference relation and then introduced an automatic procedure to repair the most inconsistent one. abdullah and najib (2014a) proposed a new preference scale in the framework of the interval-valued intuitionistic fuzzy ahp (ivif-ahp). the comparison matrix judgment was expressed in ivifns with degree of hesitation. abdullah and najib (2014b) proposed a new method of intuitionistic fuzzy ahp (if-ahp) to deal with uncertainty in decision-making. the new if-ahp was applied to establish a preference in the sustainable energy planning decision-making problem. dutta and guha (2015) proposed a preference programming based weight determination method of if-ahp in which decision makers expressed their pair-wise comparisons by using generalized triangular intuitionistic fuzzy numbers. keshavarzfard and makui (2015) presented an intuitionistic fuzzy multiple attribute decision making (madm) approach for modelling and solving ahp problems with a small amount of relationships among various criteria. if-ahp was used to evaluate the weighting for each criterion and then intuitionistic fuzzy dematel method was applied to establish contextual relationships among the criteria. onar et al. (2015) proposed an interval valued intuitionistic fuzzy ahp (ivif-ahp) approach to measure the overall performance of wind energy alternatives. 2.4 hesitant fuzzy ahp hesitant fuzzy sets (hfs) are useful for dealing with situations where decision makers are hesitant to provide their preferences about objects, preferring to offer a margin of error. hfss permit the membership degree of an element to be a set which is represented as several possible values between 0 and 1. tuysuz and simsek (2017) developed a hesitant fuzzy linguistic term sets based ahp approach and applied it to the performance comparison of cargo firms. kahraman et al. (2017) developed a hesitant fuzzy linguistic ahp and applied it to business-to-customer marketplace prioritization. oztaysi et al. (2015) developed a hesitant fuzzy ahp method involving multi-expert’s linguistic evaluations aggregated by ordered weighted averaging (owa) operator. 2.5 pythagorean fuzzy ahp yager (2013) introduced a pythagorean fuzzy set (pfs) characterized by a membership degree and a non-membership degree satisfying the condition that the square sum of its membership degree and non-membership degree is equal to or less than 1 which is a http://0-www.scopus.com.divit.library.itu.edu.tr/authid/detail.url?authorid=55580964700&amp;eid=2-s2.0-84893082409 ijahp article: otay, kahraman/solar pv power plant location selection using a z-fuzzy number based ahp international journal of the analytic hierarchy process 414 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.540 generalization of ifs. ilbahar et al. (2018) developed a novel approach to risk assessment for occupational health and safety using pythagorean fuzzy ahp and fuzzy inference system. 2.6 neutrosophic ahp the neutrosophic sets developed by smarandache (1998) extend the concept of intuitionistic fuzzy sets (ifss) that were introduced by atanassov (1983) for a new point of view to uncertainty, impreciseness, inconsistency and vagueness. smarandache (1998) introduced the degree of indeterminacy/neutrality as a new and independent component of fuzzy sets and defined a neutrosophic set composed of three components: truth membership, indeterminacy membership, and falsity membership. radwan et al. (2016) developed a novel hybrid neutrosophic ahp approach in learning management systems in decision making to handle indeterminacy of information. abdel-basset et al. (2017) integrated ahp into a delphi framework under a neutrosophic environment and introduced a new technique for checking consistency and calculating the consensus degree of an expert’s opinions. 3. preliminaries: z-fuzzy numbers zadeh (1975) defined a z-number associated with an uncertain variable z as an ordered pair of fuzzy numbers (a, b) where a is a fuzzy subset of the domain x of the variable z and b is a fuzzy subset of the unit interval. the concept of a z-number is intended to provide a basis for computation with numbers which are not totally reliable. a z-number can be used to represent information about an uncertain variable of the type where a represents a value of the variable and b represents an idea of certainty or probability. there are a limited number of studies on z-fuzzy numbers. biswas (2012) observed the drawback of the existing fuzzy numbers, studied z-fuzzy numbers and presented fundamental arithmetic operations for z-fuzzy numbers. abu bakar and gegov (2014) conducted a study ranking z-numbers by proposing a multi–layer decision making methodology. biswas (2016) discussed whether or not the fuzzy set theory was appropriate for large size problems with a number of universes and a lot of elements in these universes. in the study, the researcher also focused on z-fuzzy numbers and their mathematical operations. definition 1. a z-number is an ordered pair of fuzzy numbers denoted as z = (ã, r̃). the first component ã, a restriction on the values, is a real-valued uncertain variable 𝑋. the second component �̃� is a measure of reliability for the first component, described in figure 2. when 𝑎2𝑒𝑞𝑢𝑎𝑙𝑠𝑎3, a trapezoidal fuzzy number becomes a triangular fuzzy number. http://www.sciencedirect.com/science/article/pii/s0925753517312316 http://www.sciencedirect.com/science/article/pii/s0925753517312316 http://www.sciencedirect.com/science/article/pii/s0925753517312316 ijahp article: otay, kahraman/solar pv power plant location selection using a z-fuzzy number based ahp international journal of the analytic hierarchy process 415 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.540 figure 2 a simple z-fuzzy number definition 2. let a fuzzy set a be defined on a universe x be given as: a = {〈x, μa(x)〉 |xϵx} where μa: x → [0,1] is the membership function a. the membership value μa(x) describes the degree of belonging of x ∈ x in 𝐴. the fuzzy expectation of a fuzzy set is denoted as: 𝐸𝐴(𝑥) = ∫ 𝑥𝜇𝐴(𝑥) 𝑑𝑥𝑥 (1) which is not the same as the meaning of the expectation of probability space. it can be considered to be the information strength supporting the fuzzy set 𝐴. definition 3: converting a z-number to a regular fuzzy number consider a z-number 𝑍 = (𝐴,̃ �̃�) described by figure 2. the left is the part of restriction, and the right is the part of reliability. let �̃� = {〈𝑥, 𝜇�̃�(𝑥)〉|𝜇(𝑥) ∈ [0,1]} and �̃� = {〈𝑥, 𝜇�̃�(𝑥)〉|𝜇(𝑥) ∈ [0,1]}, 𝜇�̃�(𝑥) is a trapezoidal membership function, and 𝜇�̃�(𝑥) is a triangular membership function. 1. convert the second part (reliability) into a crisp number using equation 2. 𝛼 = ∫ 𝑥𝜇�̃�(𝑥) 𝑑𝑥 ∫ 𝜇�̃�(𝑥) 𝑑𝑥 (2) where ∫ denotes an algebraic integration. alternatively, equation 3 can be used for this defuzzification: 𝛼 = 𝑎1+2∗(𝑎2+𝑎3)+𝑎4 6 (3) 2. add the weight of the second part (reliability) to the first part (restriction). the weighted z-number can be denoted as �̃�𝛼 = {〈𝑥, 𝜇�̃�𝛼(𝑥)〉|𝜇�̃�𝛼(𝑥) = 𝛼𝜇�̃�(𝑥), 𝜇(𝑥) ∈ [0,1]} 𝑎4 𝑎3 𝑎2 𝑎1 �̃� 𝑍 = �̃�, �̃� 1 0 𝜇�̃�(𝑥) 𝑥 𝑏3 𝑏2 𝑏1 �̃� 1 0 𝜇�̃�(𝑥) 𝑥 ijahp article: otay, kahraman/solar pv power plant location selection using a z-fuzzy number based ahp international journal of the analytic hierarchy process 416 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.540 3. convert the irregular fuzzy number (weighted restriction) to a regular fuzzy number. the ordinary fuzzy set can be denoted as �̃�′ = {〈𝑥, 𝜇�̃�′(𝑥)〉|𝜇�̃�′(𝑥) = 𝜇�̃� ( 𝑥 √𝛼 ) , 𝜇(𝑥) ∈ [0,1]}. �̃�′ has the same fuzzy expectation with �̃�𝛼, and they are equal with respect to fuzzy expectation, which can be denoted by figure 3. figure 3 ordinary fuzzy number transformed from z-number 4. if the restriction function and reliability function are defined as in figure 4 (their heights may be any value between 0 and 1), the calculations are modified as follows: let �̃�𝛿 = {〈𝑥, (𝜇�̃�(𝑥); 𝛿)〉|𝜇(𝑥) ∈ [0,1]} and �̃�𝛽 = {〈𝑥, (𝜇�̃�(𝑥); 𝛽)〉|𝜇(𝑥) ∈ [0,1]}; 𝝁 �̃� 𝜹(𝒙) is a trapezoidal membership function and 𝝁 �̃� 𝜷 (𝒙) is a triangular membership function. when 𝑎2 = 𝑎3, a trapezoidal membership function becomes a triangular membership function. figure 4 a simple z̃δ,β number, �̃�𝛿,𝛽 = �̃�𝛿, �̃�𝛽 in this case, restriction and reliability functions are defined as in equations 4-5, respectively. the reliability membership function in equation 4 is substituted into the defuzzification formula (equation 2 or equation 3) so that equation 6 is obtained. 𝑎4 𝑎3 𝑎2 𝑎1 �̃�𝛿 δ 0 𝜇�̃�(𝑥) 𝑥 √𝛼𝑎4 √𝛼𝑎3 √𝛼𝑎2 √𝛼𝑎1 �̃�′ 1 0 𝜇(𝑥) 𝑥 𝑏3 𝑏2 𝑏1 �̃�𝛽 β 0 𝜇�̃�(𝑥) 𝑥 ijahp article: otay, kahraman/solar pv power plant location selection using a z-fuzzy number based ahp international journal of the analytic hierarchy process 417 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.540 𝜇�̃� 𝛿(𝑥) = { 𝑥−𝑎1 𝑎2−𝑎1 𝛿 , 𝑖𝑓𝑎1 ≤ 𝑥 ≤ 𝑎2 𝛿, 𝑖𝑓𝑎2 ≤ 𝑥 ≤ 𝑎3 𝑎4−𝑥 𝑎4−𝑎3 𝛿, 𝑖𝑓𝑎3 ≤ 𝑥 ≤ 𝑎4 0, 𝑂𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 (4) 𝜇 �̃� 𝛽 (𝑥) = { 𝑥−𝑏1 𝑏2−𝑏1 𝛽 , 𝑖𝑓𝑏1 ≤ 𝑥 ≤ 𝑏2 𝑏3−𝑥 𝑏3−𝑏2 𝛽 , 𝑖𝑓𝑏2 ≤ 𝑥 ≤ 𝑏3 0, 𝑂𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 (5) thus, we have √𝛼 = √ ∫ 𝑥𝝁 �̃� 𝜷 (𝒙) 𝑑𝑥 ∫ 𝝁 �̃� 𝜷 (𝒙) 𝑑𝑥 (6) then, the weighted �̃�𝛿,𝛽 number can be denoted as in equation 6. �̃�𝛿,𝛽 𝛼 = {〈𝑥, 𝝁�̃�𝜶 𝜹 (𝒙)〉|𝝁 �̃�𝜶 𝜹 (𝒙) = ∫𝑥𝝁 �̃� 𝜷 (𝒙)𝑑𝑥 ∫ 𝝁 �̃� 𝜷 (𝒙)𝑑𝑥 𝝁 �̃� 𝜹(𝒙), 𝜇(𝑥) ∈ [0,1]} (7) the ordinary fuzzy number converted from z-fuzzy number can be given as in equation 8. �̃�𝛿,𝛽 ′ = {〈𝑥, 𝝁�̃�′ 𝜹 (𝒙)〉|𝝁�̃�′ 𝜹 (𝒙) = 𝝁 �̃� 𝜹 (𝒙 ∫ 𝝁 �̃� 𝜷 (𝒙) 𝑑𝑥 ∫ 𝑥𝝁 �̃� 𝜷 (𝒙) 𝑑𝑥 ) , 𝜇(𝑥) ∈ [0,1]} (8) 4. zfuzzy number based ahp in this method we integrate z-fuzzy numbers with ahp. the advantage of this integration is to incorporate vagueness in the evaluations and reliabilities to these evaluations into the ahp. the steps of the proposed z-fuzzy number based-ahp are presented in the following: step 1. define the multi-criteria decision making problem and design a hierarchical structure of the problem. step 2. use the scale of linguistic restriction function given in table 1 and the scale of reliability function presented in table 2. these are the scales that have been proposed by the authors. ijahp article: otay, kahraman/solar pv power plant location selection using a z-fuzzy number based ahp international journal of the analytic hierarchy process 418 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.540 table 1 triangular z-fuzzy restriction scale linguistic terms restriction function equally better (e) (1,1,1;1) slightly better (slb) (1,1,3;1) moderately better (mb) (1,3,5;1) strongly better (stb) (3,5,7;1) very strongly better (vstb) (5,7,9;1) certainly better (cb) (7,9,10;1) absolutely better (ab) (9,10,10;1) table 2 reliability scale with its corresponding triangular z-fuzzy numbers linguistic reliability triangular z-fuzzy reliability function absolutely reliable (ar) (0.8,0.9,1;1) strongly reliable (sr) (0.7,0.8,0.9;1) very highly reliable (vhr) (0.6,0.7,0.8;1) highly reliable (hr) (0.5,0.6,0.7;1) fairly reliable (fr) (0.4,0.5,0.6;1) weakly reliable (wr) (0.3,0.4,0.5;1) very weakly reliable (vwr) (0.2,0.3,0.4;1) strongly unreliable (su) (0.1,0.2,0.3;1) absolutely unreliable (au) (0,0.1,0.2;1) decision makers may assign different values for the given linguistic terms and correspondingly different fuzzy restriction functions in table 1 if s/he wants to assign intermediate values. step 3. construct the pairwise comparison matrices and fill them in with their corresponding z-fuzzy numbers using the linguistic scales in tables 1 and 2. step 4. transform z-fuzzy numbers to their corresponding equivalent ordinary fuzzy numbers. step 5. check the consistency of each fuzzy pairwise comparison matrix. assume  ijaa ~ ~  is a fuzzy positive pairwise comparison matrix and  ijaa  is its defuzzified positive pairwise comparison matrix. if the result of the comparisons of  ijaa  is consistent; then, it can imply that the result of the comparisons of  ijaa ~ ~  is also consistent. in the ijahp article: otay, kahraman/solar pv power plant location selection using a z-fuzzy number based ahp international journal of the analytic hierarchy process 419 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.540 consistency measurement, reliability functions are ignored since they cause a consistent pairwise comparison matrix to become an inconsistent one when they are converted to regular fuzzy numbers. step 6. apply buckley’s ordinary fuzzy ahp method (buckley, 1985). the steps of this method are summarized as follows: step 6.1. calculate the geometric mean for each parameter of ija ~ in the n dimensional pairwise comparison matrix of criteria. thus, 𝑛 × 𝑛 matrix is converted to an 𝑛 × 1 matrix. this is the step that converts z-fuzzy numbers to regular fuzzy numbers. step 6.2. sum the values of each parameter in the column then normalize the values in the 𝑛 × 1 matrix. step 6.3. apply the fuzzy division operation to get the normalized weights vector. step 6.4. defuzzify the normalized weights vector using the center of gravity method given by equation 2. step 6.5. normalize the weights so their sum is equal to 1. step 6.6. apply steps (6.1-6.5) for the rest of the pairwise comparison matrices of sub-criteria and alternatives. step 6.7. combine all the weight vectors, to obtain the global weights and determine the best alternative as in the classical ahp. 5. application in this section, a real life problem of a newly founded facility is solved using the proposed z-ahp method. a brief problem definition is presented, followed by the results of the proposed method as illustrated with a figure and tables. 5.1 problem definition in 2015, the renewable energy general directorate reported by the solar energy map (sem) of turkey highlighted that “the total annual insolation time is 2.737 hours, and the total solar energy derived per year is 1.527 kwh/m2 per year” (https://www.rvo.nl/sites/default/files/2015/10/renewable%20energy %20turkey.pdf). the research has shown that capacity additions for renewable energy sources in turkey have increased remarkably especially in recent years. the report of shura (2018) noted that in 2017 there was a net renewable energy capacity of 3.2 gw versus 1.5 gw for non-renewables. the report also said that the pv capacity of 1.79 gw added in 2017 was more than three times what was added in 2016 (shura, 2018). according to the republic of turkey ministry of energy and natural resources, by the end of june 2018 the total pv solar power plant installed capacity will be 4,723 mw comprised of a total of 4,703 mw unlicensed and 23 mw licensed plants (http://www.enerji.gov.tr/en-us/pages/solar). thus, turkey will have one of the largest solar pv markets in europe (shura, 2018). this study aims to solve the location selection problem of a private company for one of its newly installed pv energy production facilities in the southwest part of turkey. based ijahp article: otay, kahraman/solar pv power plant location selection using a z-fuzzy number based ahp international journal of the analytic hierarchy process 420 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.540 on the meetings held with investors and managers, the possible alternative locations are marked in figure 5 (http://www.pv-financing.eu/wp-content/uploads/2017/06/pvfinancing-webinar-it-es-tr-business-models-slides-170517.pdf), are antalya, i̇çel and muğla. figure 5 solar map of turkey showing the alternative locations for the facility (http://www.pv-financing.eu/wp-content/uploads/2017/06/pv-financingwebinar-it-es-tr-business-models-slides-170517.pdf) after a detailed literature review and a meeting with the management, a 4-level hierarchy was designed that is illustrated in figure 6. as seen from the figure, there are seven main criteria: geological & topographic factors, political & legal factors, technical factors, economical factors, environmental factors, location-related factors, and social factors with a total of 18 sub-criteria such as slope, legislation and social support. ijahp article: otay, kahraman/solar pv power plant location selection using a z-fuzzy number based ahp international journal of the analytic hierarchy process 421 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.540 figure 6 hierarchical structure of the problem once the hierarchical structure of the problem was ready, linguistic judgments were obtained from the decision makers, who are three experts from the energy institute of istanbul technical university. table 3 presents the pairwise comparison matrix of the criteria based on the linguistic restriction and reliability values on which the decision makers arrived at consensus, while table 4 illustrates the pairwise comparison matrix of the criteria with corresponding values from tables 1 and 2. table 3 pairwise comparison matrix of the criteria criteria c1 c2 c3 c4 c5 c6 c7 c1 e, e slb, vhr 1/cb, hr 1/vstb, fr 1/mb, sr 1/stb, vhr mb, sr c2 1/slb, vhr e, e 1/ab, sr 1/cb, ar 1/mb, sr 1/stb, fr slb, hr c3 cb, hr ab, sr e, e 1/slb, vhr stb, sr slb, sr cb, sr c4 vstb, fr cb, ar slb, vhr e, e vstb, sr mb, vhr ab, ar c5 mb, sr mb, sr 1/stb, sr 1/vstb, sr e, e 1/mb, sr stb, vhr c6 stb, vhr stb, fr 1/slb, sr 1/mb, vhr mb, sr e, e vstb, ar c7 1/mb, sr 1/slb, hr 1/cb, sr 1/ab, ar 1/stb, vhr 1/vstb, ar e, e c3:technical factors goal: selecting the best appropriate location for solar pv power plant c1:geological & topographic factors c2:political & legal factors c4:economical factors c5:environmental factors c6:location related factors c7:social factors c11: slope c12: aspect c13: longitutelatitude c21: legislation s c22: political stability c31: technological development c32: technological flexibility c41: initial cost (land& material cost ) c42: construction cost c43: operation cost c53: fauna & flora c51: land use c52: agrological capacity c61: distance to urban areas c62: distance to main roads c63: distance to historical sites c71: social support c72: job availability antalya i̇çel muğla ijahp article: otay, kahraman/solar pv power plant location selection using a z-fuzzy number based ahp international journal of the analytic hierarchy process 422 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.540 table 4 pairwise comparison matrix with corresponding triangular z-fuzzy restriction and reliability scales criteria c1 c2 c3 c4 c5 c6 c7 c1 ((1,1,1;1), (1,1,1;1)) ((1,1,3;1), (0.6,0.7,0.8;1)) ((0.1,0.11,0.14;1), (0.5,0.6,0.7;1)) ((0.11,0.14,0.2;1), (0.4,0.5,0.6;1)) ((0.2,0.33,1;1), (0.7,0.8,0.9;1)) ((0.14,0.2,0.33;1), (0.6,0.7,0.8;1)) ((1,3,5;1), (0.7,0.8,0.9;1)) c2 ((0.33,1,1;1), (0.6,0.7,0.8;1)) ((1,1,1;1), (1,1,1;1)) ((0.1,0.1,0.11;1), (0.7,0.8,0.9;1)) ((0.1,0.11,0.14;1), (0.8,0.9,1;1)) ((0.2,0.33,1;1), (0.7,0.8,0.9;1)) ((0.14,0.2,0.33;1), (0.4,0.5,0.6;1)) ((1,1,3;1), (0.5,0.6,0.7;1)) c3 ((7,9,10;1), (0.5,0.6,0.7;1)) ((9,10,10;1), (0.7,0.8,0.9;1)) ((1,1,1;1), (1,1,1;1)) ((0.33,1,1;1), (0.6,0.7,0.8;1)) ((3,5,7;1), (0.7,0.8,0.9;1)) ((1,1,3;1), (0.7,0.8,0.9;1)) ((7,9,10;1), (0.7,0.8,0.9;1)) c4 ((5,7,9;1), (0.4,0.5,0.6;1)) ((7,9,10;1), (0.8,0.9,1;1)) ((1,1,3;1), (0.6,0.7,0.8;1)) ((1,1,1;1), (1,1,1;1)) ((5,7,9;1), (0.7,0.8,0.9;1)) ((1,3,5;1), (0.6,0.7,0.8;1)) ((9,10,10;1), (0.8,0.9,1;1)) c5 ((1,3,5;1), (0.7,0.8,0.9;1)) ((1,3,5;1), (0.7,0.8,0.9;1)) ((0.14,0.2,0.33;1), (0.7,0.8,0.9;1)) ((0.11,0.14,0.2;1), (0.7,0.8,0.9;1)) ((1,1,1;1), (1,1,1;1)) ((0.2,0.33,1;1), (0.7,0.8,0.9;1)) ((3,5,7;1), (0.6,0.7,0.8;1)) c6 ((3,5,7;1), (0.6,0.7,0.8;1)) ((3,5,7;1), (0.4,0.5,0.6;1)) ((0.33,1,1;1), (0.7,0.8,0.9;1)) ((0.2,0.33,1;1), (0.6,0.7,0.8;1)) ((1,3,5;1), (0.7,0.8,0.9;1)) ((1,1,1;1), (1,1,1;1)) ((5,7,9;1), (0.8,0.9,1;1)) c7 ((0.2,0.33,1;1), (0.7,0.8,0.9;1)) ((0.33,1,1;1), (0.5,0.6,0.7;1)) ((0.1,0.11,0.14;1), (0.7,0.8,0.9;1)) ((0.1,0.1,0.11;1), (0.8,0.9,1;1)) ((0.14,0.2,0.33;1), (0.6,0.7,0.8;1)) ((0.11,0.14,0.2;1), (0.8,0.9,1;1)) ((1,1,1;1), (1,1,1;1)) when only the restriction functions are considered in the calculation of the consistency ratio for the matrix, we obtained 0.083 which is lower than the desirable 0.1. after the zfuzzy numbers have been converted to their corresponding equivalent ordinary fuzzy numbers, table 4 is transformed into table 5. table 5 now represents the pairwise comparison matrix of the criteria using ordinary fuzzy numbers. table 5 z-fuzzy numbers converted to ordinary fuzzy numbers c1 c2 c3 c4 c5 c6 c7 c1 (1,1,1;1) (0.84,0.84,2.51;1) (0.08,0.09,0.11;1) (0.08,0.10,0.14;1) (0.18,0.30,0.89;1) (0.12,0.17,0.28;1) (0.89,2.68,4.47;1) c2 (0.28,0.84,0.84;1) (1,1,1;1) (0.09,0.09,0.10;1) (0.09,0.11,0.14;1) (0.18,0.30,0.89;1) (0.10,0.14,0.24;1) (0.77,0.77,2.32;1) c3 (5.42,6.97,7.75;1) (8.05,8.94,8.94;1) (1,1,1;1) (0.28,0.84,0.84;1) (2.68,4.47,6.26;1) (0.89,0.89,2.68;1) (6.26,8.05,8.94;1) c4 (3.54,4.95,6.36;1) (6.64,8.54,9.49;1) (0.84,0.84,2.51;1) (1,1,1;1) (4.47,6.26,8.05;1) (0.84,2.51,4.18;1) (8.54,9.49,9.49;1) c5 (0.89,2.68,4.47;1) (0.89,2.68,4.47;1) (0.13,0.18,0.30;1) (0.10,0.13,0.18;1) (1,1,1;1) (0.18,0.30,0.89;1) (2.51,4.18,5.86;1) c6 (2.51,4.18,5.86;1) (2.12,3.54,4.95;1) (0.30,0.89,0.89;1) (0.17,0.28,0.84;1) (0.89,2.68,4.47;1) (1,1,1;1) (4.74,6.64,8.54;1) c7 (0.18,0.30,0.89;1) (0.26,0.77,0.77;1) (0.09,0.10,0.13;1) (0.09,0.09,0.11;1) (0.12,0.17,0.28;1) (0.11,0.14,0.19;1) (1,1,1;1) after that, buckley’s ordinary fuzzy ahp method is applied to find the weights of the criteria. by following the steps of the methodology in section 4, the fuzzy weights of the criteria are calculated. the values are defuzzified using the center of gravity method and the defuzzified values are normalized. in this study, because of the space constraints, only the calculations for the main criteria are shown. ijahp article: otay, kahraman/solar pv power plant location selection using a z-fuzzy number based ahp international journal of the analytic hierarchy process 423 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.540 table 6 weights of main criteria criteria fuzzy weights normalized fuzzy weights defuzified weights normalized crisp weights c1 (0.27, 0.37, 0.64) (0.02, 0.04, 0.09) 0.05 0.04 c2 (0.23, 0.31, 0.48) (0.02, 0.03, 0.07) 0.04 0.03 c3 (2.10, 2.89, 3.66) (0.15, 0.29, 0.54) 0.32 0.28 c4 (2.51, 3.40, 4.67) (0.18, 0.35, 0.69) 0.39 0.35 c5 (0.46, 0.80, 1.28) (0.03, 0.08, 0.19) 0.10 0.09 c6 (1.02, 1.82, 2.61) (0.07, 0.18, 0.39) 0.21 0.18 c7 (0.18, 0.24, 0.34) (0.01, 0.02, 0.05) 0.03 0.02 table 6 demonstrates the results of the fuzzy, defuzzified and normalized crisp weights of the criteria. according to the results, the most important criterion is c4 economical factors with a weight of 0.35. it is followed by the criterion c3 technical factors with a weight of 0.28. the least important criteria are c7 social factors and c2 political and legal factors. after repeating the steps of the proposed procedure, the calculations for the pairwise comparison matrices of sub-criteria with respect to each main criterion are obtained and the results are illustrated in table 7. table 7 pairwise comparison matrices of sub-criteria wrt c1 c11 c12 c1 3 weight s c11 e.e e.sr 0.145 c12 e.e 0.142 c13 stb.a r stb.s r e.e 0.713 cr=0.05 wrt c2 c21 c22 weights c21 e. e 0.453 c22 slb. hr e. e 0.547 cr=0.00 wrt c3 c31 c32 weights c31 e.e e. vhr 0.50 c32 e. e 0.50 cr=.00 wrt c4 c41 c42 c43 weights c41 e. e e. sr 0.321 c42 e. e 0.312 c43 e. vhr slb. ar e. e 0.367 cr=0.08 wrt c5 c51 c52 c53 weights c51 e. e stb. ar stb. sr 0.713 c52 e. e e. sr 0.145 c53 e. e 0.142 c=0.05 wrt c6 c61 c62 c63 weights c61 e. e e. ar stb. sr 0.458 c62 e. e stb. vhr 0.448 c63 e. e 0.094 cr=0.05 wrt c7 c71 c72 weights c71 e. e mb. sr 0.652 c72 e. e 0.348 cr=0.00 table 8 contains the linguistic pairwise comparison matrices of the alternatives with respect to the sub-criteria. ijahp article: otay, kahraman/solar pv power plant location selection using a z-fuzzy number based ahp international journal of the analytic hierarchy process 424 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.540 table 8 linguistic pairwise comparison matrices of the alternatives wrt c11 a1 a2 a3 weights a1 e ab, ar cb, ar 0.820 a2 e 0.080 a3 slb, sr e 0.101 cr=0.08 wrt c12 a1 a2 a3 weights a1 e e, sr 0.087 a2 e 0.083 a3 ab, ar ab, vhr e 0.829 cr=0.00 wrt c13 a1 a2 a3 weights a1 e ab, vhr ab, hr 0.828 a2 e e, vhr 0.087 a3 e 0.085 cr=0.00 wrt c21 a1 a2 a3 weights a1 e cb, sr cb, hr 0.807 a2 e e, ar 0.099 a3 e 0.094 cr=0.01 wrt c22 a1 a2 a3 weights a1 e ab, hr vstb, vhr 0.788 a2 e 0.088 a3 slb, ar e 0.125 cr=0.08 wrt c31 a1 a2 a3 weights a1 e slb, fr 0.096 a2 e 0.077 a3 cb, hr ab, vhr e 0.827 cr=0.08 wrt c32 a1 a2 a3 weights a1 e slb. ar 0.104 a2 e 0.082 a3 cb. vhr ab. sr e 0.814 cr=0.08 wrt c41 a1 a2 a3 weights a1 e slb. ar 0.099 a2 ab. sr e ab. vhr 0.822 a3 e 0.080 cr=0.08 wrt c42 a1 a2 a3 weights a1 e e. hr 0.085 a2 ab. fr e ab. hr 0.828 a3 e 0.088 cr=0.00 wrt c43 a1 a2 a3 weights a1 e slb. sr 0.123 a2 vstb. vhr e cb. vhr 0.784 a3 e 0.093 cr=0.09 wrt c51 a1 a2 a3 weights a1 e ab. ar slb. sr 0.531 a2 e 0.050 a3 cb. sr e 0.418 cr=0.08 wrt c52 a1 a2 a3 weights a1 e vstb. hr vstb. ar 0.788 a2 e 0.103 a3 e. fr e 0.110 cr=0.03 wrt c53 a1 a2 a3 weights a1 e 0.097 a2 cb. hr e cb. hr 0.806 a3 e. sr e 0.097 cr=0.01 wrt c61 a1 a2 a3 weights a1 e ab. sr ab. ar 0.838 a2 e 0.080 a3 e. hr e 0.082 cr=0.00 wrt c62 a1 a2 a3 weights a1 e cb. vhr vstb. hr 0.796 a2 e 0.089 a3 slb. fr e 0.115 cr=0.00 wrt c63 a1 a2 a3 weights a1 e 0.094 a2 cb. hr e cb. sr 0.807 a3 e. ar e 0.099 cr=0.01 wrt c71 a1 a2 a3 w a1 e e. sr e. ar 0.331 a2 e 0.331 a3 e. ar e 0.338 cr=0.00 wrt c72 a1 a2 a3 w a1 e ab. hr cb. ar 0.821 a2 e 0.076 a3 slb. hr e 0.103 cr=0.08 when we combine these weights in table 9, we obtain the alternative locations rank as follows: i̇çel > mugla > antalya. the best location is i̇çel with a weight of 0.359. it is followed by mugla (0.324) and antalya (0.307), respectively. ijahp article: otay, kahraman/solar pv power plant location selection using a z-fuzzy number based ahp international journal of the analytic hierarchy process 425 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.540 table 9 combination of the weights alternatives 𝑤𝐶11=0.145 𝑤𝐶12=0.142 𝑤𝐶13=0.713 local weights a1 0.820 0.087 0.828 0.722 a2 0.080 0.083 0.087 0.085 a3 0.101 0.829 0.085 0.193 alternatives 𝑤𝐶21=0.453 𝑤𝐶22=0.547 local weights a1 0.807 0.788 0.797 a2 0.099 0.088 0.093 a3 0.094 0.125 0.111 alternatives 𝑤𝐶31=0.50 𝑤𝐶32=0.50 local weights a1 0.096 0.104 0.100 a2 0.077 0.082 0.080 a3 0.827 0.814 0.821 alternatives 𝑤𝐶41=0.321 𝑤𝐶42=0.312 𝑤𝐶43=0.367 local weights a1 0.099 0.085 0.123 0.103 a2 0.822 0.828 0.784 0.810 a3 0.080 0.088 0.093 0.087 alternatives 𝑤𝐶51=0.713 𝑤𝐶52=0.145 𝑤𝐶53=0.142 local weights a1 0.531 0.788 0.097 0.507 a2 0.050 0.103 0.806 0.165 a3 0.418 0.110 0.097 0.328 alternatives 𝑤𝐶61=0.458 𝑤𝐶62=0.448 𝑤𝐶63==0.094 local weights a1 0.838 0.796 0.094 0.749 a2 0.080 0.089 0.807 0.152 a3 0.082 0.115 0.099 0.098 alternatives 𝑤𝐶71=0.652 𝑤𝐶72=0.348 local weights a1 0.331 0.821 0.502 a2 0.331 0.076 0.242 a3 0.338 0.103 0.256 alternatives 𝑤𝐶1=0.04 𝑤𝐶2=0.03 𝑤𝐶3=0.28 𝑤𝐶4=0.35 𝑤𝐶5=0.09 𝑤𝐶6=0.18 𝑤𝐶7=0.02 global weights a1 0.722 0.797 0.100 0.103 0.507 0.749 0.502 0.307 a2 0.085 0.093 0.080 0.810 0.165 0.152 0.242 0.359 a3 0.193 0.111 0.821 0.087 0.328 0.098 0.256 0.324 in table 10, we present the comparison of the proposed method with classical ahp. table10 comparison with classical ahp alternatives method antalya icel mugla z-fuzzy number based ahp 0.307 0.359 0.324 classical ahp 0.312 0.362 0.326 the rankings in both methods are the same, and the priorities are close as well. however, z-fuzzy ahp considers a different uncertainty environment. this may well cause a different ranking in another problem. 6. conclusions and future research directions solar pv power plant location selection is a typical multi-criteria decision making problem based on many tangible and intangible criteria that are appropriate for assessing linguistically. z-fuzzy ahp is a relatively new approach to classical ahp with a fuzzy environment. z-fuzzy numbers are represented by restriction and reliability membership ijahp article: otay, kahraman/solar pv power plant location selection using a z-fuzzy number based ahp international journal of the analytic hierarchy process 426 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.540 functions. the study fills the gap in the literature by proposing a z-fuzzy number based ahp method to determine the best location for a solar pv power plant. in this study, zfuzzy numbers are converted to regular fuzzy numbers and then ordinary fuzzy operations are applied. we applied buckley’s fuzzy ahp approach after the conversion in our z-fuzzy ahp approach. the comparative analysis showed that the proposed extension of fuzzy ahp produced the same rank of the pv power plant location alternatives as the classical ahp did. the consistency of each fuzzy pairwise comparison matrix was calculated after they were defuzzified. the proposed method should be used under vagueness and when reliability to this vagueness is not equal to one. it has been observed that reliability levels have a significant effect on the weights that are obtained. as the reliability values are largely different from one another in a pairwise comparison matrix, the weights can be quite different from each other. hence, reliability levels should be carefully determined when using this method. for further research, there are many possible ways to extend this work. for instance, intuitionistic z-fuzzy numbers can be replaced with ordinary z-fuzzy numbers. another possibility is to replace intuitionistic z-fuzzy numbers with pythagorean z-fuzzy numbers, neutrosophic z-fuzzy numbers or hesitant z-fuzzy numbers. the results can then be compared with the ones obtained in this study. future studies can also perform a sensitivity analysis to see if there is a change when the parameters change. ijahp article: otay, kahraman/solar pv power plant location selection using a z-fuzzy number based ahp international journal of the analytic hierarchy process 427 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.540 references abdel-basset, m., mohamed. m., & sangaiah a.k. (2017). neutrosophic ahp-delphi group decision making model based on trapezoidal neutrosophic numbers. journal of ambient intelligence and humanized computing, 1-17. doi: https://doi.org/10.1007/s12652-017-0548-7 abdullah, l., & najib. l. (2014a). a new preference scale mcdm method based on interval-valued intuitionistic fuzzy sets and the analytic hierarchy process, soft computing. 1-13. doi: https://doi.org/10.1007/s00500-014-1519-y abdullah, l., & najib. l. (2014b). sustainable energy planning decision using the intuitionistic fuzzy analytic hierarchy process: choosing energy technology in malaysia. international journal of sustainable energy, 35(4), 360-377. doi: https://doi.org/10.1080/14786451.2014.907292 abdullah, l., & najib. l. (2016). integration of interval type-2 fuzzy sets and analytic hierarchy process: implication to computational procedures. aip conference proceedings, 1750, 020019. doi: https://doi.org/10.1063/1.4954532 abdullah, l., jaafar. s., & taib. i. (2009). a new analytic hierarchy process in multiattribute group decision making. international journal of soft computing, 4(5), 208-214. abu bakar, a.s., gegov, a. (2015). multi-layer decision methodology for ranking znumbers. international journal of computational intelligence systems, 8(2), 395-406. doi: http://dx.doi.org/10.1080/18756891.2015.1017371 atanassov, k.t. (1986). intuitionistic fuzzy sets. fuzzy sets and systems, 20(1), 87-96. doi: https://doi.org/10.1016/s0165-0114(86)80034-3 biswas, r. (2012). fuzzy numbers redefined. information, 15(4), 1369-1380. biswas, r. (2016). is ‘fuzzy theory’ an appropriate tool for large size decision problems? studies in fuzziness and soft computing, 332, 93-118. doi: https://doi.org/10.1007/9783-319-26302-1_8 boender, c.g.e., de graan. j.g., & lootsma. f.a. (1989). multicriteria decision analysis with fuzzy pairwise comparisons. fuzzy sets and systems, 29(2), 133-143. doi: https://doi.org/10.1016/0165-0114(89)90187-5 buckley, j.j. (1985). fuzzy hierarchical analysis. fuzzy sets and systems, 17(3), 233247. doi: https://doi.org/10.1016/0165-0114(85)90090-9 chang, d.y. (1996). applications of the extent analysis method on fuzzy ahp. european journal of operational research, 95(3), 649-655. doi: https://doi.org/10.1016/03772217(95)00300-2 https://doi.org/10.1063/1.4954532 https://www.tandfonline.com/author/bakar%2c+ahmad+syafadhli+abu https://www.tandfonline.com/author/gegov%2c+alexander https://www.tandfonline.com/toc/tcis20/current ijahp article: otay, kahraman/solar pv power plant location selection using a z-fuzzy number based ahp international journal of the analytic hierarchy process 428 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.540 cheng, c.h. (1997). evaluating naval tactical missile systems by fuzzy ahp based on the grade value of membership function. european journal of operational research, 96(2), 343-350. doi: https://doi.org/10.1016/s0377-2217(96)00026-4 deng, y., chan, f. t. s. (2011). a new fuzzy dempster mcdm method and its application in supplier selection. expert systems with applications, 38, 9854-986. doi: https://doi.org/10.1016/j.eswa.2011.02.017 dutta. b., & guha. d. (2015). preference programming approach for solving intuitionistic fuzzy ahp. international journal of computational intelligence systems, 8(5), 977-991. doi: http://dx.doi.org/10.1080/18756891.2015.1099904 feng, x., qian. x., & wu. q. (2012). a ds-ahp approach for multi-attribute decision making problem with intuitionistic fuzzy information. information technology journal, 11(12), 1764-1769. doi: http://dx.doi.org/10.3923/itj.2012.1764.1769 garni, h.z.a., & awasthi. a. (2018). solar pv power plants site selection. in i. yahyaoui (ed.) a review advances in renewable energies and power technologies, volume 1: solar and wind energies (57-75). elsevier science. doi: https://doi.org/10.1016/b978-0-12-812959-3.00002-2 ilbahar, e., karaşan, a., cebi, s., & kahraman, c. (2018). a novel approach to risk assessment for occupational health and safety using pythagorean fuzzy ahp & fuzzy inference system. safety science, 103, 124-136. doi: https://doi.org/10.1016/j.ssci.2017.10.025 kahraman, c., çevik onar, s., & öztayşi, b. (2017). b2c marketplace prioritization using hesitant fuzzy linguistic ahp. international journal of fuzzy systems, 20(7), 22022215. doi: https://doi.org/10.1007/s40815-017-0429-4 kahraman, c., öztayşi, b., sarı, i̇.u. & turanoğlu, e. (2014). fuzzy analytic hierarchy process with interval type-2 fuzzy sets. knowledge-based systems, 59, 48-57. doi: https://doi.org/10.1016/j.knosys.2014.02.001 kaur, p. (2014). selection of vendor based on intuitionistic fuzzy analytical hierarchy process. advances in operations research, 2014, 1-10. doi: http://dx.doi.org/10.1155/2014/987690. keshavarzfard, r.. & makui, a. (2015). an if-dematel-ahp based on triangular intuitionistic fuzzy numbers (tifns). decision science letters, 4(2), 237-246. doi: 10.5267/j.dsl.2014.11.002 khan, g., rathi, s. (2014). optimal site selection for solar pv power plant in an indian state using geographical information system (gis). international journal of emerging engineering research and technology, 2(7), 260-266. https://www.sciencedirect.com/science/article/pii/b9780128129593000022#! https://www.sciencedirect.com/science/article/pii/b9780128129593000022#! https://www.sciencedirect.com/science/book/9780128129593 http://www.sciencedirect.com/science/article/pii/s0925753517312316 http://www.sciencedirect.com/science/article/pii/s0925753517312316 http://www.sciencedirect.com/science/article/pii/s0925753517312316 http://dx.doi.org/10.1155/2014/987690 ijahp article: otay, kahraman/solar pv power plant location selection using a z-fuzzy number based ahp international journal of the analytic hierarchy process 429 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.540 kilic, m., & kaya, i. (2015). investment project evaluation by a decision making methodology based on type-2 fuzzy sets. applied soft computing, 27, 399-410. doi: https://doi.org/10.1016/j.asoc.2014.11.028 li, c., sun, y.. & du, y. (2008). selection of 3pl service suppliers using a fuzzy analytic network process. control and decision conference, 2008, ccdc 2008, china, 2174-2179. doi: https://doi.org/10.1109/ccdc.2008.4597709 mikhailov, l. (2003). deriving priorities from fuzzy pairwise comparison judgements. fuzzy sets and systems, 134(3), 365-385. doi: https://doi.org/10.1016/s01650114(02)00383-4 onar, s. c., oztaysi, b., otay, i̇., & kahraman, c. (2015). multi-expert wind energy technology selection using interval-valued intuitionistic fuzzy sets. energy, 90, 274-285. doi: https://doi.org/10.1016/j.energy.2015.06.086 ozdemir, s., sahin, g. (2018). multi-criteria decision-making in the location selection for a solar pv power plant using ahp. measurement, 129, 218-226. doi: https://doi.org/10.1016/j.measurement.2018.07.020 oztaysi, b., cevik onar, s., bolturk, e.. & kahraman, c. (2015). hesitant fuzzy analytic hierarchy process, ieee international conference on fuzzy systems, 1-7. doi: https://doi.org/10.1109/fuzz-ieee.2015.7337948 radwan, n.m., senousy, m.b., & riad, a.e.d.m. (2016). neutrosophic ahp multi criteria decision making method applied on the selection of learning management system. international journal of advancements in computing technology (ijact), 8(5), 95-105. sadiq, r., & tesfamariam, s. (2009). environmental decision-making under uncertainty using intuitionistic fuzzy analytic hierarchy process (if-ahp). stochastic environmental research and risk assessment, 23(1), 75-91. doi: https://doi.org/10.1007/s00477-007-0197-z samanlioglu, f., ayağ, z. (2017). a fuzzy ahp-promethee ii approach for evaluation of solar power plant location alternatives in turkey. journal of intelligent & fuzzy systems, 33(2), 859-871. doi: 10.3233/jifs-162122 shura (2018). increasing the share of renewables in turkey’s power system: options for transmission expansion and flexibility (https://www.shura.org.tr/wpcontent/uploads/2018/05/grid-study-eng.pdf visited on 01.09.2018). smarandache, f. (1998). neutrosophy, neutrosophic probability, set, and logic. rehoboth: american research press. tüysüz, f. & şimşek, b. (2017). a hesitant fuzzy linguistic term sets-based ahp approach for analyzing the performance evaluation factors: an application to cargo sector. complex & intelligent systems, 3(3), 167-175. doi: https://doi.org/10.1007/s40747-017-0044-x https://www.sciencedirect.com/science/article/pii/s1568494614005900#! https://www.sciencedirect.com/science/article/pii/s1568494614005900#! https://www.sciencedirect.com/science/journal/15684946 https://www.sciencedirect.com/science/journal/15684946/27/supp/c https://www.sciencedirect.com/science/article/pii/s0263224118306134#! https://www.sciencedirect.com/science/article/pii/s0263224118306134#! https://www.sciencedirect.com/science/journal/02632241/129/supp/c https://content.iospress.com/search?q=author%3a%28%22samanlioglu,%20funda%22%29 https://content.iospress.com/search?q=author%3a%28%22aya%c4%9f,%20zeki%22%29 https://content.iospress.com/journals/journal-of-intelligent-and-fuzzy-systems https://content.iospress.com/journals/journal-of-intelligent-and-fuzzy-systems ijahp article: otay, kahraman/solar pv power plant location selection using a z-fuzzy number based ahp international journal of the analytic hierarchy process 430 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.540 van laarhoven, p.j.m., & pedrycz, w. (1983). a fuzzy extension of saaty's priority theory. fuzzy sets and systems, 11(1), 199-227. doi: https://doi.org/10.1016/s01650114(83)80082-7 wang, h., qian, g., & feng, x. (2011). an intuitionistic fuzzy ahp based on synthesis of eigenvectors and its application. information technology journal, 10(10), 1850-1866. doi: http://dx.doi.org/10.3923/itj.2011.1850.1866 wang, y.m., luo, y., & hua, z. (2008). on the extent analysis method for fuzzy ahp and its applications. european journal of operational research, 186(2), 735-747. doi: https://doi.org/10.1016/j.ejor.2007.01.050 wu, j., huang, h.b., & cao, q.w. (2013). research on ahp with interval-valued intuitionistic fuzzy sets and its application in multicriteria decision making problems. applied mathematical modelling, 37(24), 9898-9906. doi: https://doi.org/10.1016/j.apm.2013.05.035 xu, z., & liao, h. (2014). intuitionistic fuzzy analytic hierarchy process. ieee transactions on fuzzy systems, 22(4), 749-761. doi: https://doi.org/10.1109/tfuzz.2013.2272585 yager, r.r. (2013). pythagorean fuzzy subsets. proceedings of joint ifsa world congress and nafips annual meeting, edmonton, canada, 57-61. doi: https://doi.org/10.1109/ifsa-nafips.2013.6608375 yu, j.r., & cheng, s.j. (2007). an integrated approach for deriving priorities in analytic network process. european journal of operational research, 180(3), 1427-1432. doi: https://doi.org/10.1016/j.ejor.2006.06.005 zadeh, l.a. (1975). the concept of a linguistic variable and its application to approximate reasoning. information sciences, 8(3), 199-249. doi: https://doi.org/10.1016/0020-0255(75)90036-5 zadeh, l.a. (2011). a note on z-numbers. information sciences, 181, 2923-2932. zhang, c., li. w., & wang, l. (2011). ahp under the intuitionistic fuzzy environment, eighth international conference on fuzzy systems and knowledge discovery (fskd2011), 1, 583-587. doi: https://doi.org/10.1109/fskd.2011.6019593 web references 1. https://www.rvo.nl/sites/default/files/2015/10/renewable%20energy%20turkey.pdf (visited on 01.09.2018) 2. http://www.pv-financing.eu/wp-content/uploads/2017/06/pv-financing-webinar-ites-tr-business-models-slides-170517.pdf (visited on 10.09.2018) 3. http://www.enerji.gov.tr/en-us/pages/solar (visited on 10.09.2018) https://ieeexplore.ieee.org/xpl/mostrecentissue.jsp?punumber=6012258 https://ieeexplore.ieee.org/xpl/mostrecentissue.jsp?punumber=6012258 ijahp article: bellahcene, benamar, mekidiche /ahp and wafgp hybrid model for information system project selection international journal of the analytic hierarchy process 228 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.761 ahp and wafgp hybrid model for information system project selection mohammed bellahcene 1 leppese laboratory, department of economics university centre of maghnia, algeria bellahcene_mohammed@yahoo.fr fatima-zohara benamar faculty of economics and management sciences university of tlemcen, algeria benamar_fatimazohra@yahoo.fr mohammad mekidiche leppese laboratory, department of economics university centre of maghnia, algeria mkidiche@yahoo.fr abstract the aim of this study is to propose an integrated analytic hierarchy process (ahp) and weighted additive fuzzy goal programming (wafgp) method for the selection of information system projects that can use all types of linear membership functions and offer more flexibility. the proposed methodology includes three steps. first, an expert team was formed to identify the decision criteria and build a hierarchical model for the information system project selection. then, the ahp was used to estimate the relative weights of the criteria. finally, a wafgp model was formulated and used to select the projects. a hypothetical example is given to show how to use this methodology and its advantages. in comparison to other approaches, the ahp-wafgp hybrid model gives better support for information system project selection by selecting projects that make the best use of available resources and better satisfy the decision goals. furthermore, the sensitivity analysis reveals that the proposed model is robust, adaptable, and not sensitive to small changes. nevertheless, the proposed methodology does not include interdependencies among criteria and alternatives. keywords: project selection; information system; ahp; wafgp 1 this research was supported by the directorate general of scientific research and technological development, algerian ministry of higher education and scientific research. mailto:bellahcene_mohammed@yahoo.fr mailto:benamar_fatimazohra@yahoo.fr mailto:mkidiche@yahoo.fr ijahp article: bellahcene, benamar, mekidiche /ahp and wafgp hybrid model for information system project selection international journal of the analytic hierarchy process 229 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.761 1. introduction today’s contemporary knowledge-intensive environment that is characterized by increased competition and rapid, unpredictable and relentless changes has led organizations to explore new possibilities for detecting and avoiding emerging threats, exploiting market shifts (mikalef & pateli, 2017), and improving decision making (chiang et al., 2018) to increase their competitive advantage and performance. among the different solutions which can be adopted by top management to address these challenges, the selection of information systems (is) is particularly important. indeed, iss support operational tasks, streamlining and progress of organizational processes, decision making, and mechanization (burkland & zachariassen, 2014; almajali, masa’deh, & tarhini, 2016). iss can also improve a firm’s performance and increase its competitive advantage by lowering costs, achieving economies of scale, or enhancing innovation and differentiation (weber & pliskin, 1996; porter & millar, 1985). furthermore, iss improve communication, participation, and collaboration (chang & wong, 2010, deng et al., 2008; lu, huang, & heng, 2006). despite its potential, is project development is not a simple task. the success of this kind of project is influenced by different organizational, technological and environmental factors (okumus et al., 2017; tajudeen, jaafar, & ainin, 2018) that include the fit of business strategy and ict projects (strassmann, 1990; strassmann, 1997; chuang & lin, 2017), software flexibility (wang et al., 2008) or information technology capabilities (ray, muhanna, & barney, 2005, tarafdar & qrunfleh, 2017; ouyang, 2017). because of these variables and others, is projects have a high risk of failure (pan, pan, & devadoss, 2008). according to several studies and reports, more than 60% of is projects do not meet their performance objectives (clegg et al., 1997; the standish group, 2010; brownsell, blackburn, & hawley, 2012; kang, o’brien, & mulva, 2013). wang et al. (2008) also emphasized that “software projects continue to be plagued by budget overruns and a failure to produce software that meets expectations. failure to meet cost budgets may adversely impact future resource allocation and failure to meet time considerations may hamper the firm’s competitive posture.” selecting is projects that are in accordance with corporate objectives and then allocating resources to complete these projects gives rise to critical and complicated business activities (samvedi et al., 2018). indeed, these kinds of decisions engage organizations in substantial long-term commitments that require large investments of resources in skills and capabilities, computer software and hardware, operational procedure adjustments, and so on (chena & chengb, 2009). is project selection is a multi-criteria decision making (mcdm) process because of the large number of alternatives and the need to integrate the views of multiple decision makers (abdel-basset, atef & smarandache, 2019). there are also multiple and often conflicting quantitative and qualitative attributes that are fuzzy and imprecise such as environmental conditions, corporate objectives, benefits, project risks, users’ and decision makers’ preferences, and the limited availability of is resources. finally, interdependencies among these alternatives and criteria should be taken into account. during the last decades, several methodologies have been developed to overcome the is project selection difficulties. for example, schniederjans and wilson (1991) have ijahp article: bellahcene, benamar, mekidiche /ahp and wafgp hybrid model for information system project selection international journal of the analytic hierarchy process 230 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.761 proposed an ahp zero-one linear programming methodology for budgetary and resource constraints. after that, lee and kim (2001) developed an integrated delphi-anp-zogp method to consider the degree of interdependency among is projects. finally, to deal with the imprecise data in is projects and uncertain judgment of decision makers, bolat et al. (2014) developed a hybrid model for fuzzy ahp using a fuzzy multi-objective linear programming model. in spite of this progress, some weaknesses still characterize is project selection methodologies. for example, preemptive/lexicographic goal programming (with priority), used in many studies, is not flexible when dealing with integer problems with many goals (kim et al., 2009). therefore, if the experiences of experts are collected to determine decision criteria weights, then using weighted goal programming can be more flexible and give more credible results. furthermore, the fuzzy goal programming (fgp) methods used in previous studies cannot use all types of linear membership functions (mfs). to overcome these limitations, the present study proposes an easier and more flexible method for is project selection. a hybrid method in which the analytic hierarchy process (ahp) proposed by saaty (1980) is used to formulate the problem in a hierarchical structure and estimate the relative weights of the criteria from the subjective judgments of decision-makers is combined with the weighted additive fuzzy goal programming (wafgp) proposed by yaghoobi et al. (2008) to incorporate the dm’s preferences and trade-off aspiration levels between objectives. the remainder of this research paper is structured as follows: section two presents the literature review, section three describes the proposed ahp-wafgp model, section four provides data and results of an illustrative application for verification, section five explores the sensitivity, and section six provides the conclusions. 2. literature review during the last decades, several studies have been conducted that propose a methodology to overcome the difficulties of is project selection and help organizations, companies and it managers with their choices. initially, a single criterion cost/benefit analysis was suggested by king and schrems (1978). despite its advantages, this method only considers tangible or monetary criteria and skips intangible effects and attributes like risk, business process improvement, or user and decision-maker satisfaction (liang & li, 2008). after that, different models using ranking (buss, 1983), scoring (lucas & moore, 1976; lootsma, mensch, & vos, 1990) and ahp (muralidhar, santhnanm, & wilson, 1990) have been proposed as alternative approaches for is project selection. in practice, these models have been used often to solve real problems because they are uncomplicated and easy to understand so decision-makers feel comfortable with them (lee & kim, 2001). scoring models and ahp models incorporate all of the crucial factors to select projects and provide a quantitative measure that can be used directly to compare alternatives and choose those with the highest scores (keeney & raiffa, 1978). however, even if these models seem more effective than cost/benefit analysis models, they still have three major limitations. first, ranking, scoring and ahp methods do not apply to problems that have ijahp article: bellahcene, benamar, mekidiche /ahp and wafgp hybrid model for information system project selection international journal of the analytic hierarchy process 231 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.761 resource feasibility and optimization requirements. second, these methods ignore the interdependencies (carried costs, additional benefits, etc.) that can exist between is projects and selection criteria. third, is project selection takes place in an incomplete, vague, and uncertain information environment (chena & chengb, 2009). therefore, almost all of the selection objectives and constraints take on a fuzzy and imprecise character. in order to surmount these limitations, studies have proposed more effective methods. in two of the fundamental studies in this area santhanam et al. (1989) and schniederjans and wilson (1991) proposed a zero-one linear programming (zogp) method and an ahp-zogp integrated approach to consider budgetary and resource constraints. in a second approach, several studies proposed methodologies which consider interdependencies among is projects and integrate both qualitative and quantitative factors. for example, santhanam and kyparisis (1996) proposed a nonlinear zogp model and lee and kim (2001) developed a hybrid method using delphi-anp-gp to estimate the degree of interdependency among is projects. kim et al. (2009) proposed an integrated anp-fuzzy logic methodology, and zandi and tavana (2010) developed a hybrid model that combines the technique for order preference by similarity to ideal solution (topsis) with multi-objective decision-making (modm) in order to consider both quantitative and qualitative attributes as well as the interdependencies among candidate projects. a third set of studies tried to deal with the fuzzy and imprecise character of is project selection objectives and constraints. chena and chengb (2009) built a multiple-criteria decision-making method (mcdm) for selecting an information system project based on fuzzy measures and the fuzzy integral. to consider the vague, imprecise, and subjective judgments of decision makers, users and assessors, gerogiannis et al. (2013), presented a selection/evaluation approach that applies a hybrid group decision making method based on topsis and intuitionist fuzzy sets (ifs). bolat et al. (2014) built a systematic and comprehensive model using the fuzzy analytical hierarchical process (fahp) and fuzzy multi-objective linear programming (fmolp). elahi, shamsi, and ghatari (2016) proposed a hybrid method integrating: fuzzy evaluation, which merged a fuzzy expert system with topsis and ahp. toloo, nalchigar, and sohrabi (2018) and toloo and mirbolouki, (2019) developed a data envelopment analysis method. finally, taking into account the indeterminacy and the imprecise nature of linguistic assessments, yepez (2017) developed a model based on the single valued neutrosophic number (svn-numbers); alava et al. (2019) proposed an integrated method of svnnumbers with ahp; and leyva-vazquez, et al. (2020) proposed a hybrid approach of a balanced scorecard model, neutrosophic, ahp and zero-one linear programming. in spite of this progress, some weaknesses still exist in is project selection methodologies. according to kim et al. (2009), preemptive /lexicographic goal programming (with priority) that is used in almost all of the studies is not flexible when dealing with integer problems with many goals. therefore, if the experiences of experts ijahp article: bellahcene, benamar, mekidiche /ahp and wafgp hybrid model for information system project selection international journal of the analytic hierarchy process 232 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.761 are collected to determine decision criteria weights, then using weighted goal programming can be more flexible and give more credible results. furthermore, the fgp methods used in previous studies cannot use all types of linear mfs to determine the fuzzy objective values. to overcome these limitations, the present study proposes an easier and more flexible method for selecting is projects. this method is a hybrid method in which the ahp proposed by saaty (1980) is used to formulate the problem in a hierarchical structure and estimate the relative weights of the criteria from the subjective judgments of decision-makers, combined with the wafgp proposed by yaghoobi et al. (2008) to incorporate the dm’s preferences and trade-off aspirations between objectives to complete the project selection decision. 3. methodology this paper proposes an integrated methodology for is project selection that combines the two following multi-criteria decision-making methods: ahp and wafgp. introduced by saaty (1980), the ahp is a decision-making method used in physical and social fields (saaty & vargas, 2012), and in almost all applications related to decisionmaking (vaidya & kumar, 2006) to address the following three fundamental problems: group decision making, conflict resolution, and pairwise comparisons and neural activity (moreno-jiménez & vargas, 2018). the ahp integrates and compares qualitative and quantitative factors, and objective and subjective factors based on the subjective judgments of the decision maker (bahurmoz, 2019). this method breaks down a complex and unstructured situation into its different components (islam & anis, 2015) and reorganizes them into a hierarchical structure (saaty, 1990) to determine the priority of a set of attributes and alternatives and their relative importance in a decision-making problem (rouyendegh & erkan, 2011). in this study, we used the ahp to synthesize the decision goals, criteria, and alternatives in a hierarchical model. we used judgments in a pairwise comparison matrix to estimate the relative weights (relative importance) of criteria used in is project selection. proposed by yaghoobi et al. (2008), the wafgp is a lp model that can use all types of linear membership functions (figure 1) to determine the degree of mfs for every variable. after finding the relative weights of attributes, we used the wafgp to complete the decision by optimizing project selection. 3.1 select an expert team and build the decision hierarchical structure in the first step, an expert team should be selected that includes a governing board member, the is manager, a quality manager, is external specialists (consultants), and other managers involved with the projects. this team identifies the decision criteria and constructs a hierarchical model for is project selection. according to ahp principles, in the first stage, the is project selection problem is decomposed into a multilevel hierarchical structure of objective, criteria and alternatives (sharma, moon, & bae, 2008) with the goal at the top, followed by objectives with a broad perspective, then criteria, through intermediate levels of criteria (on which subsequent elements depend) to the lowest level which contains the alternatives (saaty, 2008). ijahp article: bellahcene, benamar, mekidiche /ahp and wafgp hybrid model for information system project selection international journal of the analytic hierarchy process 233 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.761 3.2 use ahp to calculate the relative weights of each criterion in the second step, the criteria are presented to an expert team to fill out the pairwise comparison judgment matrices. team members gave their consensual verbal judgement from the fundamental ahp scales of equally important, moderately more important, strongly more important, very strongly more important, and extremely more important. these descriptive preferences are then respectively translated into the numerical values 1, 3, 5, 7, 9 (with 2, 4, 6, and 8 as intermediate values). then, the reciprocals of these values are used for the corresponding transposed judgments (atkinson, bayazit, & karpak, 2015). insofar as the is selection is a strategic decision, the governing board member has the last word and gives the final judgment when significant differences persist between the expert team members. once the criteria comparison matrices are constructed, the largest eigenvalue and the corresponding principle eigenvector of this matrix are calculated and normalized so that its entries sum to one. the normalized eigenvector represents the relative weights of the criteria. at the end of filling out each ahp pairwise comparison matrix with judgments, the inconsistency test should be applied as the inconsistency measure is useful for identifying and correcting possible errors with the entering of the judgments. actual inconsistencies are also identified that may have to be ironed out with the participants (cabala, 2010). to measure the inconsistency level of the pairwise comparison matrix, the consistency index (𝐶𝐼) and consistency ratio (𝐶𝑅) are calculated using the following formula (saaty, 1980; 1996; li, wang, & tong, 2016): 𝐶𝐼 = 𝜆max − 𝑛 𝑛−1 (1) 𝐶𝑅 = 𝐶𝐼 𝑅𝐼 (2) where 𝜆𝑚𝑎𝑥 is the largest eigenvalue of the matrix, 𝑛 is the number of items being compared in the matrix, and 𝑅𝐼 is the random index (ebrahimnejad et al., 2012; ohta, salomon, & silva, 2018). according to saaty and vargas (2012), a 𝐶𝑅 of 0.10 or less is acceptable to continue the ahp analysis. if this indicator is greater than 0.10, judgments should be revisited to discover and correct the causes of the inconsistency (mu & pereyra-rojas, 2017). 3.3 use the wafgp to complete the project selection decision at the final stage, the relative weights of the criteria (given by the ahp) are used to formulate the wafgp model to complete the project selection decision. as an extension of the kim and whang (1998) model, the wafgp formulates fgp programming unequal weights as a single linear programing problem with the concept of tolerance. in this step, the expert team should specify the fuzzy objective value interval and subjectively choose one of the four mfs used in the fawgp for each objective or constraint (torabi & hassini, 2008; liang & cheng, 2009; tamiz & yaghoobi, 2010; díaz-madroñero et al., 2018) (figure 1). ijahp article: bellahcene, benamar, mekidiche /ahp and wafgp hybrid model for information system project selection international journal of the analytic hierarchy process 234 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.761 3.3.1. left membership function the mfs, also called the “larger is better” mfs, are used to maximize the result (chen et al., 2018). for example, in is project selection, mfs can be used for the benefits if decision makers believe that the higher the benefits, the better the optimal solutions. the mathematical formulation of these mfs is as follows: 𝜇𝑖 (𝐴𝑋)𝑖 = { 1 𝑖𝑓 (𝐴𝑋)𝑖 ≥ 𝑏𝑖 1 − 𝑏𝑖 −(𝐴𝑋)𝑖 ∆𝑖𝐿 𝑖𝑓 0 𝑖𝑓 (𝐴𝑋)𝑖 ≤ 𝑏𝑖 − ∆𝑖𝐿 𝑏𝑖 − ∆𝑖𝐿 ≤ (𝐴𝑋)𝑖 ≤ 𝑏𝑖 , 𝑖 = 𝑖0 + 1,…,𝑗0 (3) where:  (𝐴𝑋)𝑖 = ∑ 𝑎𝑖𝑗𝑥𝑗 , 𝑖 = 1,…,𝑘 𝑛 𝑗=1 .  (𝐴𝑋)𝑖 is the mfs.  𝜇𝑖 determines the degree of mfs for the i th fuzzy goal.  for 𝑖 = 𝑖0 + 1,…,𝑗0 , 𝑏𝑖 is the imprecise aspiration level for the i th fuzzy goal.  ∆𝑖𝐿 is the lower admissible violation from the imprecise aspiration levels. 3.3.2. right membership function in the case where results need to be minimized, “the smaller, the better” type of mfs can also be used (chen et al., 2018). for example, such mfs can be used for costs where the smaller the costs, the better the optimal result. these mfs are formulated as: 𝜇𝑖 (𝐴𝑋)𝑖 = { 1 𝑖𝑓 (𝐴𝑋)𝑖 ≤ 𝑏𝑖 1 − (𝐴𝑋)𝑖−𝑏𝑖 ∆𝑖𝑅 𝑖𝑓 0 𝑖𝑓 (𝐴𝑋)𝑖 ≥ 𝑏𝑖 + ∆𝑖𝑅 𝑏𝑖 ≤ (𝐴𝑋)𝑖 ≤ 𝑏𝑖 + ∆𝑖𝑅 , 𝑖 = 1,…,𝑖0 (4) where: ∆ir is the upper admissible violation from the imprecise aspiration level. 3.3.3. triangular membership function also known as the nominal-the-best mfs, this mf is selected when the decision makers want to exactly reach the target value (al-rafaie, 2015). in this case, the positive or negative deviations of the goal from a target value should be as small as possible. the mathematical formulation for this mf is as follows: 𝜇𝑖 (𝐴𝑋)𝑖 = { 0 𝑖𝑓 (𝐴𝑋)𝑖 ≤ 𝑏𝑖 − ∆𝑖𝐿 1 − 𝑏𝑖 –(𝐴𝑋)𝑖 ∆𝑖𝐿 𝑖𝑓 𝑏𝑖 − ∆𝑖𝐿 ≤ (𝐴𝑋)𝑖 ≤ 𝑏𝑖 , 𝑖 = 𝑗0 + 1,…,𝑘0 1 − (𝐴𝑋)𝑖−𝑏𝑖 ∆𝑖𝑅 𝑖𝑓 𝑏𝑖 ≤ (𝐴𝑋)𝑖 ≤ 𝑏𝑖 + ∆𝑖𝑅 0 𝑖𝑓 (𝐴𝑋)𝑖 ≥ 𝑏𝑖 + ∆𝑖𝑅 (5) 3.3.4. trapezoidal membership function trapezoidal mfs are selected when the decision makers prefer setting the target value within a continuous interval (al-rafaie, 2015). this is defined as: ijahp article: bellahcene, benamar, mekidiche /ahp and wafgp hybrid model for information system project selection international journal of the analytic hierarchy process 235 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.761 μi (𝐴𝑋)𝑖 = { 0 𝑖𝑓 (𝐴𝑋)𝑖 ≤ 𝑏 𝑖 𝑙 − ∆𝑖𝐿 1 − 𝑏 𝑖 𝑙−(𝐴𝑋)𝑖 ∆𝑖𝐿 𝑖𝑓 𝑏 𝑖 𝑙 − ∆𝑖𝐿 ≤ (𝐴𝑋)𝑖 ≤ 𝑏 𝑖 𝑙 1 𝑖𝑓 𝑏 𝑖 𝑙 ≤ (𝐴𝑋)𝑖 ≤ 𝑏 𝑖 𝑢 , 𝑖 = 𝑘0 + 1,…,𝑘 1 − (𝐴𝑋)𝑖−𝑏 𝑖 𝑢 ∆𝑖𝑅 𝑖𝑓 𝑏 𝑖 𝑢 ≤ (𝐴𝑋)𝑖 ≤ 𝑏 𝑖 𝑢 + ∆𝑖𝑅 0 𝑖𝑓 (𝐴𝑋)𝑖 ≥ 𝑏 𝑖 𝑢 + ∆𝑖𝑅 (6) where: for = 𝑘0 + 1,…,𝑘 , 𝑏 𝑖 𝑙 and 𝑏 𝑖 𝑢 denote the imprecise lower and upper bounds for the fuzzy goal, respectively. figure 1 linear membership function types after selecting mfs to specify the fuzzy objective value for each objective or constraint, the data can be used in the wafgp model to complete the project selection. the general formulation of this model is as follows: ijahp article: bellahcene, benamar, mekidiche /ahp and wafgp hybrid model for information system project selection international journal of the analytic hierarchy process 236 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.761 𝑀𝑖𝑛𝑧 = ∑ 𝑤𝑖 𝛿𝑖 + ∆𝑖𝑅 𝑖0 𝑖=1 +∑ 𝑤𝑖 𝛿𝑖 − ∆𝑖𝐿 𝑗0 𝑖=𝑖0+1 +∑ 𝑤𝑖( 𝛿𝑖 − ∆𝑖𝐿 𝐾 𝑖=𝑗0+1 + 𝛿𝑖 + ∆𝑖𝑅 ) (7) subject to: (𝐴𝑋)𝑖 − 𝛿𝑖 + ≤ 𝑏𝑖 , 𝑖 = 1,2,…,𝑖0 (8) (𝐴𝑋)𝑖 + 𝛿𝑖 − ≥ 𝑏𝑖 , 𝑖 = 𝑖0 + 1,…,𝑗0 (9) (𝐴𝑋)𝑖 + 𝛿𝑖 − − 𝛿𝑖 + = 𝑏𝑖 , 𝑖 = 𝑗0 + 1,…,𝐾 (10) (𝐴𝑋)𝑖 + 𝛿𝑖 − ≥ 𝑏𝑖 𝑙 , 𝑖 = 𝐾0 + 1,…,𝐾 (11) (𝐴𝑋)𝑖 − 𝛿𝑖 + ≤ 𝑏𝑖 𝑢 , 𝑖 = 𝐾0 + 1,…,𝐾 (12) μi + 𝛿𝑖 + ∆𝑖𝑅 = 1 , 𝑖 = 1,2,…,𝑖0 (13) μi + 𝛿𝑖 − ∆𝑖𝐿 = 1 , 𝑖 = 𝑖0 + 1,…,𝑗0 (14) μi + 𝛿𝑖 − ∆𝑖𝐿 + 𝛿𝑖 + ∆𝑖𝑅 = 1 , 𝑖 = 𝑗0 + 1,…,𝐾 (15) μi ,𝛿𝑖 − ,𝛿𝑖 + ≥ 0 , 𝑖 = 1,…,𝐾 (16) 𝑥 ∈ 𝐶𝑆 (17) where 𝑀𝑖𝑛𝑧 is the objective function,𝐶𝑆 is an optional set of hard constraints such as are found in lp; 𝑥 is the decision variable; 𝑤𝑖 denotes the weight of the fuzzy goal; 𝛿𝑖 − ,𝛿𝑖 + represent the negative and positive deviations. 4. an illustrative application to demonstrate the uses and advantages of the ahp and wafgp hybrid model, a hypothetical example follows. suppose that a firm has to choose five information systems projects from among ten alternatives, under the following criteria: mandated projects, benefit (b), hardware cost (c1), software cost (c2), annual cost of additional manpower (c4), other cost (c3), user preferences (s1), decision-maker preferences (s2), risk factor (r), completion time required (t1), and training time required (t2). adapted from badri et al. (2001), the basic data of goals achievement for each project and the maximum available resources are presented in table 1. 4.1 form the expert team and build a hierarchy model for is project selection to set priorities among selection criteria and to specify the membership functions and the fuzzy values for each objective, an expert team was formed that included a governing board member, the is manager, quality manager, financial manager, network managers, database and software manager of an algerian energy company, and three is specialists (the researchers). next, the hierarchical model was constructed with the following levels: level 0 presents the decision goal (select five is projects), level 1 includes the is project selection criteria, and level 2 contains the ten project alternatives (figure 2). ijahp article: bellahcene, benamar, mekidiche /ahp and wafgp hybrid model for information system project selection international journal of the analytic hierarchy process 237 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.761 figure 2 ahp hierarchy 4.2 use ahp to calculate the relative weights of each criterion (attribute) in the next step, decision attributes and alternatives were presented to the project team to reach a group consensus about each value judgement and arrive at a unique judgment matrix for prioritizing the criteria. then, the superdecisions software was used to determine the normalized weights and synthesize the results. table 2 lists the pairwise comparison judgment matrices, the relative weights of attributes and the consistency ratio. the consistency ratio was under the upper limit of 0.10 (0.06063); which means that decision makers were adequately consistent in ranking the attributes. 4.3 formulate the wafgp and complete the project selection decision once the criteria weights were calculated, the team members specified the type and the data for the membership functions to be used for each objective. as seen in table 3, the expert team wanted to maximize the benefits, user preferences and decision-makers preferences. they assigned a left membership for these objectives and estimated their lower admissible violation from the imprecise aspiration levels respectively at 10000, 10 and 10. the experts also wanted to minimize hardware cost, software cost, other cost, risk, completion time, training time, and cost of additional manpower. they assigned a right membership for these objectives and estimated their upper admissible violation from the imprecise aspiration levels respectively at 20000, 10000, 300, 200000, 450, 500, and 100. after that, and based on the collected information (exposed in the tables 1, 2 and 3), the wafgp model for is project selection was formulated as follows: 𝑀𝑖𝑛 𝑍 = 0.13747[ 𝑛1 10000 ]+ 0.18798[ 𝑝2 20000 ] + 0.18798[ 𝑝3 10000 ] + 0.09023[ 𝑝4 300 ]+ 0.13392[ 𝑝5 200000 ] + 0.03521[ 𝑛6 10 ] + 0.03074[ 𝑛7 10 ] + 0.06653[ 𝑝8 450 ] + 0.05591[ 𝑝9 500 ] + 0.07404[ 𝑝10 100 ] (18) subject to: ijahp article: bellahcene, benamar, mekidiche /ahp and wafgp hybrid model for information system project selection international journal of the analytic hierarchy process 238 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.761 ∑ 𝐵𝑖 𝑥𝑖 + 𝑛1 ≥ 48000 10 𝑖=1 (19) ∑ 𝐻𝑖 𝑥𝑖 − 𝑝2 ≤ 65000 10 𝑖=1 (20) ∑ 𝑆𝑖 𝑥𝑖 − 𝑝3 ≤ 28000 10 𝑖=1 (21) ∑ 𝑂𝑖 𝑥𝑖 − 𝑝4 ≤ 360 10 𝑖=1 (22) ∑ 𝑟𝑖 𝑥𝑖 − 𝑝5 ≤ 0 10 𝑖=1 (23) ∑ 𝑃𝑅𝐷𝑖 𝑥𝑖 + 𝑛6 ≥ 47 10 𝑖=1 (24) ∑ 𝑃𝑅𝑈𝑖 𝑥𝑖 + 𝑛7 ≥ 49 10 𝑖=1 (25) ∑ 𝑡𝑖 𝑥𝑖 − 𝑝8 ≤ 0 10 𝑖=1 (26) ∑ 𝑡𝑡𝑖 𝑥𝑖 − 𝑝9 ≤ 0 10 𝑖=1 (27) ∑ 𝑚𝑖 𝑥𝑖 − 𝑝10 ≤ 1100 10 𝑖=1 (28) μ1 + 1 10000 = 1 (29) μ2 + 1 20000 = 1 (30) μ3 + 1 10000 = 1 (31) μ4 + 1 300 = 1 (32) μ5 + 1 200000 = 1 (33) μ6 + 1 10 = 1 (34) μ7 + 1 10 = 1 (35) μ8 + 1 450 = 1 (36) μ9 + 1 500 = 1 (37) μ10 + 1 100 = 1 (38) 𝑥1 = 1 (39) ∑ 𝑥𝑖 10 1=1 = 5 (40) 𝑥𝑖 = 0 𝑜𝑟 1;𝑖 = 1,2,…,10. ijahp article: mohammed, benamar, mekidiche /ahp and wafgp hybrid model for information system project selection international journal of the analytic hierarchy process 239 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.761 table 1 model data inputs project mandated benefit* hardware cost* software cost* other cost* decision makers preferences users preferences *** risk factor **** completion time** training time** annual cost for additional manpower* 1 yes 1774 1900 3800 00 9.336 9.762 3 50 90 500 2 no 1349 11500 2254 160 9.305 9.638 3 43 18 286 3 no 40600 29500 16020 00 9.349 9.773 4 90 19 545 4 no 1200 21000 7800 18 7.727 8.008 3 60 66 29 5 no 5000 20000 750 190 9.272 9.505 2 83 84 294 6 no 3000 14000 44 20 8.661 9.517 2 67 136 100 7 no 2090 320 16000 00 9.206 9.377 3 91 69 00 8 no 1300 500 1000 30 8.604 9.286 3 97 119 00 9 no 1320 1200 3300 08 7.552 8.193 2 28 61 39 10 no 1720 00 2500 10 7.481 8.002 2 36 24 23 max available ……. 48000 65000 28000 360 47 49 00 00 00 1100 * in 1000$. ** required in days ***means of scores estimated on a scale of 0-10 ****scored on a scale of 0-10 table 2 pairwise comparison judgment matrices and relative weights of criteria c1 c2 c3 c4 b r t1 t2 s1 s2 c1 1 1 3 3 2 2 3 4 3 3 c2 1 1 3 3 2 2 3 4 3 3 c3 1/3 1/3 1 2 1/2 1/3 2 3 3 3 c4 1/3 1/3 ½ 1 1/2 1/3 3 2 3 3 b 1/2 1/2 2 2 1 2 2 3 4 4 r 1/2 1/2 3 3 1/2 1 2 3 4 4 t1 1/3 1/3 ½ 1/3 1/2 1/2 1 2 3 3 t2 1/4 1/4 1/3 1/2 1/3 1/3 1/2 1 4 4 s1 1/3 1/3 1/3 1/3 1/4 1/4 1/3 1/4 1 1/2 s2 1/3 1/3 1/3 1/3 1/4 1/4 1/3 1/4 2 1 𝑤𝑖* 0.18798 0.18798 0.09023 0.07404 0.13747 0.13392 0.06653 0.05591 0.03074 0.03521 * relative weights of criteria. inconsistency: 0.06063 ijahp article: bellahcene, benamar, mekidiche /ahp and wafgp hybrid model for information system project selection international journal of the analytic hierarchy process 240 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.761 table 3 type and data of membership function for every goal objective type of membership functions data of membership functions benefit left membership (10000, 48000) hardware cost right membership (65000, 20000) software cost right membership (28000, 10000) other cost right membership (360, 300) risk right membership (0, 200000) decision-maker’s preference left membership (10, 47) users preference left membership (10, 49) completion time required right membership (0, 450) training time required right membership (0, 500) additional manpower required right membership (1100, 100) table 4 different methods result comparison method projects selected benefit* hardware cost* software cost* other cost* decision-makers preferences users preferences risk factor completion time** training time** annual cost for additional manpower* ahp-wafgp 1,3,8,9, 10 46714 33100 26620 48 42.322 45.016 14 301 313 1107 ahp 1,5,6,7, 10 13584 36220 23094 220 43.956 46.163 12 327 403 917 zogp 1,3,4,6, 10 48294 66400 30164 48 42.554 45.062 14 303 335 1197 ahp-zogp 1,7,8,9, 10 8204 3920 26600 48 42.179 44.62 13 302 363 562 targeted values ---48000 65000 28000 360 47 49 00 00 00 1100 * in 1000$. ** required in days ),( iil b ),( iri b  ),( iri b  ),( iri b  ),( iri b  ),( iil b ),( iil b ),( iri b  ),( iri b  ),( iri b  ijahp article: mohammed, benamar, mekidiche /ahp and wafgp hybrid model for information system project selection international journal of the analytic hierarchy process 241 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.761 in table 4, 𝐵𝑖 𝑖𝑠 the benefit derived from implementing project 𝑖, 𝐻𝑖 is the hardware cost associated with implementing project 𝑖, 𝑆𝑖 is the software cost associated with implementing project 𝑖, 𝑂𝑖 is the other costs associated with implementing project 𝑖, 𝑟𝑖 is the likelihood of failure of project 𝑖; 𝑃𝑅𝐷𝑖 is the decision-maker’s preference for project 𝑖, 𝑃𝑅𝑈𝑖 is the user’s preference for project 𝑖, 𝑡𝑖 is the estimated completion time for project 𝑖, 𝑡𝑡𝑖 is the estimated training time required for project 𝑖, 𝑚𝑖 is the cost of additional manpower for project 𝑖, 𝑝𝑗 and 𝑛𝑗 are the positive and negative deviation variables for the goals 𝑗, 𝜇𝑗 is the degree of membership functions for the goal 𝑗, 𝑖 = 1,2,…,𝑛 is projects, 𝑗 = 1,2,…,𝑚 is project goals, 𝑥𝑖 is a binary variable so it takes on the value of 1 if the project 𝑖 is selected, and it takes on the value 0 otherwise. finally, the lingo 15.0 package was used to solve this model. the optimal solution was: 𝑥1 = 1,𝑥2 = 0,𝑥3 = 1,𝑥4 = 0,𝑥5 = 0,𝑥6 = 0,𝑥7 = 0,𝑥8 = 1,𝑥9 = 1,𝑥10 = 1; and the degrees of membership functions for each goal were estimated as: (𝜇1 ,𝜇2,𝜇3,𝜇4,𝜇5,𝜇6,𝜇7,𝜇8,𝜇9,𝜇10) = (0.87,1,1,1,0.11,0.53,0.60,0.84,0.89,0.93) (42) according to these results, projects 1, 3, 8, 9 and 10 should be selected as the best solutions. in order to illustrate the quality of support provided by an ahp-awfgp integrated model for is project selection, we compared its outputs with those that we obtained separately using ahp, zogp and combined ahp-zogp approaches (ahp data and zogp programs are presented in appendix 1 and appendix 2). as shown in table 4, the benefits provided by the combined ahp-awfgp solution are better than those with the ahp and ahp-zogp methods, and with less unused resources. in comparison with the zogp results, even if its benefits are 3.38% superior to those generated by the ahpwafgp solution, we have shown that these results require fewer financial resources, training, and completion time. while the ahp-wafgp solution respects the budget constraints and requires a budget of $60,857,000, the zogp solution exceeds the hardware cost, software cost, and annual cost of additional manpower limitations, and requires $97,809,000 ($33,490,000 additional budgeting when compared to the target values). in the end, we can say that the integrated ahp-wafgp model gives better support for is project selection decisions by choosing those projects that better satisfy the different decision goals and makes better use of the available resources. 5. sensitivity analysis to determine the adaptability and robustness of the proposed model, a sensitivity analysis was performed. on one hand, since the is implementation is usually plagued by budget and time overruns, the three scenarios were built to accommodate this. the first and second scenarios suppose an initial progressive decline and then a gradual rise in hardware and software budgets. the third presumes a progressive reduction of quantity of tolerance (∆𝑖𝑅) for completion and training time. ijahp article: bellahcene, benamar, mekidiche /ahp and wafgp hybrid model for information system project selection international journal of the analytic hierarchy process 242 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.761 on the other hand, two expert team members wanted to give more importance to the benefits and the users and decision-makers preferences, so the last scenario investigates the effect of varying these relative weights on the final decision. 5.1 scenario 1: decreasing hardware and software budgets the first scenario suggests successive reductions of 10%, 12.5% then 15% of hardware and software available budgets. as seen in table 5, running our model with a 10% reduction yielded the same selected projects of the initial scenario. in the one exception, a reduction of 12.5% or 15% results in the selection of project 6 instead of project 9. therefore, the selected portfolio for these last two includes projects 1, 3, 6, 8 and 10. compared to the initial scenario, the second scenario better satisfies users and decisionmakers and has a greater benefit ($48,394,000). however, this solution has higher costs for hardware ($45,900,000), software ($23,364,000) and additional manpower ($11,680,000). moreover, this solution needs more training time (388 days) and requires a longer completion time (344 days). it should be noted that except for the annual cost for additional manpower, all of the costs respect the initial estimated budgets in this solution. 5.2 scenario 2: increasing hardware and software budgets the second scenario supposes a successive increase of 10%, 12.5% then 15% of the hardware and software available budgets. as seen in table 5, all three yield the same selected projects as the initial scenario. 5.3 scenario 3: decreasing the quantity of tolerance for completion and training time the third scenario supposes successive decreases of 10%, 12.5% then 15% of the quantity of tolerance (∆𝑖𝑅) for completion and training time. as shown in table 5, these three yielded the same selected projects of the initial scenario. 5.4 scenario 4: changing criteria weights more importance was given to the benefits, users preferences, and decision-makers preferences in this scenario and the inputs of the pairwise comparison judgment matrices of the criteria varied with a recalculated output. three versions were calculated using a one point increase for each in the pairwise comparison weights of benefit, user preferences and decision-maker (dm) preferences. table 6 shows the resulting relative weights for the criteria. all three versions resulted in the selection of the same project portfolio as the initial scenario (table 5). therefore, we can conclude that the ahp-wafgp hybrid model proposed here for is selection is adaptable, robust, and not sensitive to small changes in criteria weights, hardware and software cost, and quantity of tolerance for completion and training time. ijahp article: bellahcene, benamar, mekidiche /ahp and wafgp hybrid model for information system project selection international journal of the analytic hierarchy process 243 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.761 table 5 results comparison for different scenarios is project x01 x02 x03 x04 x05 x06 x07 x08 x09 x10 initial scenario      scenario 1: cost increase run1: -10%      run2: -12.5%      run3: -15%      scenario 2: cost decrease run1: +10%      run2: +12.5%      run3: +15%      scenario 3: ∆𝑖𝑅 reduction run1: -10%      run2: -12.5%      run3: -15%      scenario 4: changing weights run1      run2      run3      table 6 criteria relative weights estimated for scenario 4 criteria initial scenario run 1 benefit +1 run 2 users preference +1 run 3 dm preference +1 benefit 0.13747 0.18888 0.13598 0.13624 hardware cost 0.18798 0.17213 0.18463 0.1844 software cost 0.18798 0.17213 0.18463 0.1844 other cost 0.09023 0.08602 0.08859 0.08834 risk 0.13392 0.12777 0.13236 0.13256 decision makers preference 0.03521 0.03375 0.03284 0.05039 users preference 0.03074 0.02946 0.04401 0.05604 completion time 0.06653 0.06305 0.06511 0.06511 training time 0.05591 0.05361 0.05946 0.05604 cost of additional manpower 0.07404 0.07321 0.07239 0.07242 𝐶𝑅 0.05992 0.06555 0.06281 6. conclusions as a strategic decision, is project selection is very important for companies because this kind of choice is usually a risky and costly long-term commitment. furthermore, as it is an mcdm problem, any is selection method that is used must accommodate different alternatives, and multiple and often conflicting fuzzy and imprecise attributes. to deal with these constraints, different models have been developed. however, even if the latest methods reported in the literature consider the diversity and fuzzy nature of the attributes, preemptive/lexicographic goal programming that is often used is not flexible when dealing with an integer problem that has many goals. the aim of this study was to present a simpler, easier, and more flexible methodology for is project selection that can use all types of linear membership functions to specify the fuzzy objective values. the proposed approach combines the ahp and the wafgp. ijahp article: bellahcene, benamar, mekidiche /ahp and wafgp hybrid model for information system project selection international journal of the analytic hierarchy process 244 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.761 first, the ahp was used to estimate the decision criteria weights, and then these weights were used to formulate a wafgp model and complete the project selection decision. this model was demonstrated with a hypothetical example. the results show that the integrated ahp-wafgp approach seems to be easier and simpler than the previous methods (ahp, zogp, and combined ahp-zogp) and gives better support for is project selection decisions by choosing those projects that better satisfy the different decision goals and also makes better use of the available resources. a sensitivity analysis demonstrated that the proposed model is robust, adaptable, and not sensitive to small changes in the model parameters. despite its advantages, the integrated ahp-wafgp approach neglects the uncertain nature of the decision makers’ judgments and the interdependencies that can exist among criteria and is project alternatives. authors can use fuzzy anp or fuzzy non-linear mathematical programming to overcome these limits. furthermore, we have supposed, in the second step of the model that the expert team members should arrive at a consensual judgment. in practice, this consensual judgement was not easily reached. future studies can therefore be improved by using an appropriate method to solicit and aggregate expert judgments. finally, even as it takes into account the fuzzy nature of objectives (goals), the proposed model neglects the fuzzy nature of the relative weight of criteria and the constraints. future research could integrate fuzzy ahp with a fuzzy parameters model having fuzzy constraints. ijahp article: bellahcene, benamar, mekidiche /ahp and wafgp hybrid model for information system project selection international journal of the analytic hierarchy process 245 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.761 references abdel-basset, m., atef, a., & smarandache, f. (2019). a hybrid neutrosophic multiple criteria group decision making approach for project selection. cognitive systems research, 57, 216-227. doi: https://doi.org/10.1016/j.cogsys.2018.10.023 alava, m. v., figueroa, s. d., alcivar, h., & vazquez, m. (2019). single valued neutrosophic numbers and analytic hierarchy process for project selection. neutrosophic sets and systems, 21(1), 122-130. ddi: https://digitalrepository.unm.edu/nss_journal/vol21/iss1/13 almajali, d.a., masa’deh, r., & tarhini, a. (2016). antecedents of erp systems implementation success: a study on jordanian healthcare sector. journal of enterprise information management, 29(4), 549-565. doi: https://doi.org/10.1108/jeim-03-20150024 al-rafaie, a. (2015). a proposed weighted additive model to optimize multiple quality responses in the taguchi method with applications. journal of process mechanical engineering, 229(3), 168-178. doi: https://doi.org/10.1177/0954408913513757 atkinson, m., bayazit, o., & karpak, b. (2015). a case study using the analytic hierarchy process for it outsourcing decision making. international journal of information systems and supply chain management, 8(1), 60-84. doi: https://doi.org/10.4018/ijisscm.2015010104 badri, m. a., davis, d., & davis, d. (2001). a comprehensive 0-1 goal programming model for project selection. international journal of project management, 19, 243-252. doi: https://doi.org/10.1016/s0263-7863(99)00078-2 bahurmoz, a.m. (2019). measuring corporate social responsibility performance: a comprehensive ahp based index. international journal of the analytic hierarchy process, 11(1), 20-41. doi: https://doi.org/10.13033/ijahp.v11i1.608 bolat, b., çebi, f., temur, g. t., & otay, i̇. (2014). a fuzzy integrated approach for project selection. journal of enterprise information management, 27(3), 247-260. doi: https://doi.org/10.1108/jeim-12-2013-0091 brownsell, s., blackburn, s., & hawley, m. (2012). user requirements for an ict-based system to provide care, support and information access for older people in the community. journal of assistive technologies, 6(1), 5-23. doi: http://dx.doi.org/10.1108/17549451211214328 burkland, s. & zachariassen, f. (2014). developing an erp technology: handling incompleteness of the system. scandinavian journal of management, 30(4), 409-426. doi: https://doi.org/10.1016/j.scaman.2014.08.009 buss, m. j. (1983). how to rank computer projects. harvard business review, 61(1), 118–125. https://doi.org/10.1016/j.cogsys.2018.10.023 https://digitalrepository.unm.edu/nss_journal/vol21/iss1/13 https://doi.org/10.1108/jeim-03-2015-0024 https://doi.org/10.1108/jeim-03-2015-0024 https://doi.org/10.1177/0954408913513757 https://doi.org/10.4018/ijisscm.2015010104 https://doi.org/10.1016/s0263-7863(99)00078-2 https://doi.org/10.13033/ijahp.v11i1.608 https://doi.org/10.1108/jeim-12-2013-0091 http://dx.doi.org/10.1108/17549451211214328 https://doi.org/10.1016/j.scaman.2014.08.009 ijahp article: bellahcene, benamar, mekidiche /ahp and wafgp hybrid model for information system project selection international journal of the analytic hierarchy process 246 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.761 cabala, p. (2010). using the analytic hierarchy process in evaluating decisions alternative. operational research and decision, 1, 5-23. chang, h. h. & wong, k. h. (2010). adoption of e-procurement and participation of emarketplace on firm performance: trust as a moderator. information & management, 47, 262–270. doi : https://doi.org/10.1016/j.im.2010.05.002 chen, j., zhang, w., li, j., zhang, w., liu, y., zhao, b., & zhang, y. (2018). optimal sizing for grid-tied microgrids with consideration of joint optimization of planning and operation. ieee transactions on sustainable energy, 9(1), 237-248. doi: https://doi.org/10.1109/tste.2017.2724583 chena, c. t. & chengb, h. l. (2009). a comprehensive model for selecting information system project under fuzzy environment. international journal of project management, 27(04), 389–399. doi: https://doi.org/10.1016/j.ijproman.2008.04.001 chiang, r. h., grover, v., liang, t.-p., & guest, d. z. (2018). strategic value of big data and business analytics. journal of management information systems, 35(2), 383387. doi: http://dx.doi.org/10.1080/07421222.2018.1451950 chuang, s.-h. & lin, h.-n. (2017). performance implications of information-value offering in e-service systems: examining the resource-based perspective and innovation strategy. journal of strategic information systems, 26(1), 22-38. doi: https://doi.org/10.1016/j.jsis.2016.09.001 clegg, c., axtell, c., damodaran, l., & farbey, b. (1997). information technology: a study of performance and the role of human and organizational factor. ergonomics, 40(9), 851-872. doi: https://psycnet.apa.org/doi/10.1080/001401397187694 deng, x., doll, w. j., al-gahtani, s., larsen, t. j., pearson, j. m., & raghunathan, t. s. (2008). a cross-cultural analysis of the end-user computing satisfaction instrument: a multi-group invariance analysis. information & management, 45, 211–220. doi : https://doi.org/10.1016/j.im.2008.02.002 díaz-madroñero, m., pérez-sánchez, m., satorre-aznar, j.r., mula, j., & lópezjiménez, a. (2018). analysis of a wastewater treatment plant using fuzzy goal programming as a management tool: a case study. journal of cleaner production, 180, 20-33. doi: https://doi.org/10.1016/j.jclepro.2018.01.129 ebrahimnejad, s., mousavi, s.m., tavakkoli-moghad, r., hashemi, h., & vahdani, b. (2012). a novel two-phase group decision making approach for construction project selection in a fuzzy environment. applied mathematical modelling, 36, 4197–4217. doi: https://doi.org/10.1016/j.apm.2011.11.050 elahi, s., shamsi, z., & ghatari, a. (2016). a hybrid selection method on information system development projects. international journal of business information systems, 22(4), 495-515. doi: https://doi.org/10.1504/ijbis.2016.077840 https://doi.org/10.1016/j.im.2010.05.002 https://doi.org/10.1109/tste.2017.2724583 https://doi.org/10.1016/j.ijproman.2008.04.001 http://dx.doi.org/10.1080/07421222.2018.1451950 https://doi.org/10.1016/j.jsis.2016.09.001 https://psycnet.apa.org/doi/10.1080/001401397187694 https://doi.org/10.1016/j.im.2008.02.002 https://doi.org/10.1016/j.jclepro.2018.01.129 https://doi.org/10.1016/j.apm.2011.11.050 https://doi.org/10.1504/ijbis.2016.077840 ijahp article: bellahcene, benamar, mekidiche /ahp and wafgp hybrid model for information system project selection international journal of the analytic hierarchy process 247 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.761 gerogiannis, v. c., fitsilis, p., & kameas, a. d. (2013). evaluation of project and portfolio management information systems with the use of a hybrid ifs-topsis method. intelligent decision technologies, 7, 91-105. doi: https://doi.org/10.3233/idt-120153 islam, r. & anis, a. (2015). the application of analytic hierarchy process in higherlearning institutions: a literature review. journal of international business and entrepreneurship development, 8(2), 166-182. doi: https://doi.org/10.1504/jibed.2015.070446 kang, y., o’brien, w. j., & mulva, s.p. (2013). value of it: indirect impact of it on construction project performance via best practices. automation in construction, 35, 383–396. doi: http://dx.doi.org/10.1016/j.autcon.2013.05.011 keeney, r. l. & raiffa, h. (1978). decisions with multiple objectives: preferences and value trade-offs. new york: wiley. kim, i., shin, s., choi, y., thang, n. m., ramos, e.r., & hwang, w.j. (2009). development of a project selection method on information system using anp and fuzzy logic. international journal of computer, electrical, automation, control and information engineering, 3(5), 1286-1291. kim, j. s. & whang, k. s. (1998). a tolerance approach to the fuzzy goal programming problems with unbalanced triangular membership function. european journal of operational research, 107(3), 614-624. doi: https://doi.org/10.1016/s03772217(96)00363-3 king, j. l. & schrems, e. l. (1978). cost-benefit analysis in information systems development and operation. acm computing surveys, 10(1), 19–34. doi: https://doi.org/10.1145/356715.356718 lee, j. w. & kim, s. h. (2001). an integrated approach for interdependent information system project selection. international journal of project management, 19, 111-118. doi: https://doi.org/10.1016/s0263-7863(99)00053-8 leyva-vazquez, m., quiroz-martinez, m. a., portilla-castell, y., hechavarríahernández, j. r., & gonzález-caballero, e. (2020). a new model for the selection of information technology project in a neutrosophic environment. neutrosophic sets and systems, 32(1), 344-360. doi: https://digitalrepository.unm.edu/nss_journal/vol32/iss1/22 li, k. w., wang, z. j., & tong, x. (2016). acceptability analysis and priority weight elicitation for interval multiplicative comparison matrices. european journal of operational research, 250(2), 628-638. doi: https://doi.org/10.1016/j.ejor.2015.09.010 liang, c. & li, q. (2008). enterprise information system project selection with regard to bocr. international journal of project management, 26(8), 810-820. doi: https://doi.org/10.1016/j.ijproman.2007.11.001 https://doi.org/10.3233/idt-120153 https://doi.org/10.1504/jibed.2015.070446 http://dx.doi.org/10.1016/j.autcon.2013.05.011 https://doi.org/10.1016/s0377-2217(96)00363-3 https://doi.org/10.1016/s0377-2217(96)00363-3 https://doi.org/10.1145/356715.356718 https://doi.org/10.1016/s0263-7863(99)00053-8 https://digitalrepository.unm.edu/nss_journal/vol32/iss1/22 https://doi.org/10.1016/j.ejor.2015.09.010 https://doi.org/10.1016/j.ijproman.2007.11.001 ijahp article: bellahcene, benamar, mekidiche /ahp and wafgp hybrid model for information system project selection international journal of the analytic hierarchy process 248 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.761 liang, t.f. & cheng, h.f. (2009). application of fuzzy sets to manufacturing/distribution planning decisions with multi-product and multi-time period in supply chains. expert systems with applications, 36, 3367–3377. doi: https://doi.org/10.1016/j.eswa.2008.01.002 lootsma, f. a., mensch, t. a., & vos, f. d. (1990). multi-criteria analysis and budget reallocation in long-term research planning. european journal of operational research. 47(3), 293-305. doi: https://doi.org/10.1016/0377-2217(90)90216-x lu, x. h., huang, l. h., & heng, m. s. (2006). critical success factors of interorganizational information systems; a case study of cisco and xiao tong in china. information & management (43), 395–408. doi : https://doi.org/10.1016/j.im.2005.06.007 lu, y. & ramamurthy, k. (2011). understanding the link between information technology capability and organizational agility: an empirical examination. mis quarterly, 35(4), 931–954. doi: https://doi.org/10.2307/41409967 lucas, h. c. & moore, j. r. (1976). a multiple criterion scoring approach to reformation system project selection. infor, 14(1), 1-12. doi: https://doi.org/10.1080/03155986.1976.11731622 mikalef, p. & pateli, a. (2017). information technology-enabled dynamic capabilities and their indirect effect on competitive performance: findings from pls-sem and fsqca, journal of business research, 70, 1-16. doi: http://dx.doi.org/10.1016/j.jbusres.2016.09.004 moreno-jiménez, j. & vargas, l. (2018). cognitive multiple criteria decisions making and the legacy of the analytic hierarchy process. estudios de economía aplicada, 36(1), 67-80. doi: https://doi.org/10.25115/eea.v36i1.2516 muralidhar, k., santhnanm, r., & wilson, r. l. (1990). using the analytic hierarchy process for information system project selection. information & management, 18(1), 8795. doi : https://doi.org/10.1016/0378-7206(90)90055-m ohta, r., salomon, v., & silva, m.b. (2018). selection of industrial maintenance strategy: classical ahp and fuzzy ahp applications. international journal of the analytic hierarchy process, 10(2), 254-265. doi: https://doi.org/10.13033/ijahp.v10i2.551 okumus, f., bilgihan, a., ozturk, a.b., & zhao, x. (2017). identifying and overcoming barriers to deployment of information technology projects in hotels. journal of organizational change management, 30(5), 744-766. doi: https://doi.org/10.1108/jocm-12-2015-0239 ouyang, y.c. (2017). information system capabilities and organizational performance: comparing three models. pacific asia journal of the association for information systems, 9(1), 1-28. doi: https://doi.org/10.17705/1pais.09101 https://doi.org/10.1016/j.eswa.2008.01.002 https://doi.org/10.1016/0377-2217(90)90216-x https://doi.org/10.1016/j.im.2005.06.007 https://doi.org/10.2307/41409967 https://doi.org/10.1080/03155986.1976.11731622 http://dx.doi.org/10.1016/j.jbusres.2016.09.004 https://doi.org/10.25115/eea.v36i1.2516 https://doi.org/10.1016/0378-7206(90)90055-m https://doi.org/10.13033/ijahp.v10i2.551 https://doi.org/10.1108/jocm-12-2015-0239 https://doi.org/10.17705/1pais.09101 ijahp article: bellahcene, benamar, mekidiche /ahp and wafgp hybrid model for information system project selection international journal of the analytic hierarchy process 249 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.761 pan, s. l., pan, g., & devadoss, p. r. (2008). managing emerging technology and organizational transformation: an acculturative analysis. information & management, 45, 153–163. doi: https://doi.org/10.1016/j.im.2007.11.003 porter, m. & millar, v. e. (1985). how information technology gives you competitive advantage. harvard business review, 63(4), 149-161. doi: https://hbr.org/1985/07/howinformation-gives-you-competitive-advantage[28/11/2016 ray, g., muhanna, w. a., & barney, j. (2005). information, technology and the performance of the customer service process: a resource-based analysis. mis quarterly, 29(4), 625–651. doi: https://www.jstor.org/stable/25148703. rouyendegh, b.d. & erkan t.e. (2011). erp system selection by ahp method: case study from turkey international. journal of business and management studies, 3(1), 3948. saaty, t. (1990). how to make a decision: the analytic hierarchy process. european journal of operational research, 48(1), 9-26. doi: https://doi.org/10.1016/03772217(90)90057-i saaty, t. l. (1980). the analytic hierarchy process. new york: mcgraw-hill saaty, t. l. (1991). some mathematical concepts of the analytic hierarchy process. behaviormetrica, 18(29), 01-09. doi: https://doi.org/10.2333/bhmk.18.29_1 saaty, t. l. (1996). decision making with dependence and feedback: the analytic network process. pittsburgh: rws publications. saaty, t. l. (2008). decision making with the analytic hierarchy process. international journal of services sciences, 1(1), 83–98. doi: https://doi.org/10.1504/ijssci.2008.017590 saaty, t. & vargas, l. (2012). models, methods, concepts & applications of the analytic hierarchy process. springer science and business media. samvedi, a., jain., v., chan, f.t.s., & chung, s.h. (2018). information system selection for a supply chain based on current trends: the brigs approach. neural computing & applications, 30, 1619–1633. doi: https://link.springer.com/article/10.1007%2fs00521016-2776-8 santhanam, r., muralidhar, k., & shniederjans, m. (1989). a zero-one goal programming approach for information system project selection. omega international journal of mgmt, 17(6), 583-593. doi: https://doi.org/10.1016/0305-0483(89)90062-5 santhanam, r. & kyparisis, g. j. (1995). a multiple criteria decision model for information system project selection. computers and operations research, 22(8), 807825. doi: https://doi.org/10.1016/0305-0548(94)00069-k https://doi.org/10.1016/j.im.2007.11.003 https://hbr.org/1985/07/how-information-gives-you-competitive-advantage%5b28/11/2016 https://hbr.org/1985/07/how-information-gives-you-competitive-advantage%5b28/11/2016 https://www.jstor.org/stable/25148703 https://doi.org/10.1016/0377-2217(90)90057-i https://doi.org/10.1016/0377-2217(90)90057-i https://doi.org/10.2333/bhmk.18.29_1 https://doi.org/10.1504/ijssci.2008.017590 https://link.springer.com/article/10.1007%2fs00521-016-2776-8 https://link.springer.com/article/10.1007%2fs00521-016-2776-8 https://doi.org/10.1016/0305-0483(89)90062-5 https://doi.org/10.1016/0305-0548(94)00069-k ijahp article: bellahcene, benamar, mekidiche /ahp and wafgp hybrid model for information system project selection international journal of the analytic hierarchy process 250 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.761 santhanam, r. & kyparisis, g. j. (1996). a decision model for interdependent information system project selection. european journal of operational research, 89, 380-399. doi: https://doi.org/10.1016/0377-2217(94)00257-6 schniederjans, m. j. & santhanam, r. (1993). a multi-objective constrained resource information system project selection method. european journal of operational research, 70, 244-297. doi: https://doi.org/10.1016/0377-2217(93)90042-l schniederjans, m. j. & wilson, r. l. (1991). using the analytic hierarchy process and goal programming for information system project selection. information & management north-holland, 20, 333-342. doi: https://doi.org/10.1016/0378-7206(91)90032-w sharma, m. j., moon, i., & bae, h. (2008). analytic hierarchy process to assess and optimize distribution network. applied mathematics and computation, 202, 256-265. doi: https://doi.org/10.1016/j.amc.2008.02.008 strassmann, p. a. (1990). the business value of computer. new canaan, connecticut: information economics press. strassmann, p. a. (1997). the squandered computer. new canaan, connecticut: the information economic press. tajudeen, f.p., jaafar, n.i., & ainin, s. (2018). understanding the impact of social media usage among organizations. information & management, 55(3), 308-321. doi: https://doi.org/10.1016/j.im.2017.08.004 tamiz, m. & yaghoobi, m. (2010). nurse scheduling by fuzzy goal programming, new developments in multiple objective and goal programming. lecture notes in economics and mathematical systems, 638, 151-163. doi: https://doi.org/10.1007/978-3-642-103544_11 tarafdar, m. & qrunfleh, s. (2017). agile supply chain strategy and supply chain performance: complementary roles of supply chain practices and information systems capability for agility. international journal of production research, 55(4), 925-938. doi: https://doi.org/10.1080/00207543.2016.1203079 the standish group. (2010). chaos summary 2009. boston: the standish group international. https://www.classes.cs.uchicago.edu/archive/2014/fall/512101/required.reading/standish.group.chaos.2009.pdf toloo, m. & mirbolouki, m. (2019). a new project selection method using data envelopment analysis. computers & industrial engineering, 138, 1-28. doi: https://doi.org/10.1016/j.cie.2019.106119 toloo, m., nalchigar, s., & sohrabi, b. (2018). selecting most efficient information system projects in presence of user subjective opinions: a dea approach. central european journal of operations research, 26, 1027–1051. doi: https://doi.org/10.1007/s10100-018-0549-4 https://doi.org/10.1016/0377-2217(94)00257-6 https://doi.org/10.1016/0377-2217(93)90042-l https://doi.org/10.1016/0378-7206(91)90032-w https://doi.org/10.1016/j.amc.2008.02.008 https://doi.org/10.1016/j.im.2017.08.004 https://doi.org/10.1080/00207543.2016.1203079 https://www.classes.cs.uchicago.edu/archive/2014/fall/51210-1/required.reading/standish.group.chaos.2009.pdf https://www.classes.cs.uchicago.edu/archive/2014/fall/51210-1/required.reading/standish.group.chaos.2009.pdf https://doi.org/10.1016/j.cie.2019.106119 https://doi.org/10.1007/s10100-018-0549-4 ijahp article: bellahcene, benamar, mekidiche /ahp and wafgp hybrid model for information system project selection international journal of the analytic hierarchy process 251 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.761 torabi, s.a. & hassini, e. (2008). an interactive possibilistic programming approach for multiple objective supply chain master planning. fuzzy sets and systems, 159, 193–214. doi: https://doi.org/10.1016/j.fss.2007.08.010 vaidya, o.s. & kumar, s. (2006). analytic hierarchy process: an overview of applications. european journal of operational research, 169(1), 1–29. doi: https://doi.org/10.1016/j.ejor.2004.04.028 wang, e. g., ju, p. h., jiang, j. j., & klein, g. (2008). the effects of change control and management review on software flexibility and project performance. information & management, 45, 438–443. doi : https://doi.org/10.1016/j.im.2008.05.003 weber, y. & pliskin, n. (1996). the effects of information systems integration and organizational culture on a firm's effectiveness. information & management, 30, 81-90. doi : https://doi.org/10.1016/0378-7206(95)00046-1 yaghoobi, m. a., jones, d. f., & tamiz, m. (2008). weighted additive models for solving fuzzy goal programming problems. asia-pacific journal of operational research, 25, 715-733. doi: https://doi.org/10.1142/s0217595908001973 yepez, s.l.t. (2017). decision support based on single valued neutrosophic number for information system project selection. neutrosophic sets and systems, 17(1), 37-41. doi: https://digitalrepository.unm.edu/nss_journal/vol17/iss1/7 zandi, f. & tavana, m. (2010). a multi-attribute group decision support system for information technology project selection. international journal of business information systems, 6(2), 179–199. doi: https://doi.org/10.1504/ijbis.2010.034353 https://doi.org/10.1016/j.fss.2007.08.010 https://doi.org/10.1016/j.ejor.2004.04.028 https://doi.org/10.1016/j.im.2008.05.003 https://doi.org/10.1016/0378-7206(95)00046-1 https://doi.org/10.1142/s0217595908001973 https://digitalrepository.unm.edu/nss_journal/vol17/iss1/7 https://doi.org/10.1504/ijbis.2010.034353 ijahp article: mohammed, benamar, mekidiche /ahp and wafgp hybrid model for information system project selection international journal of the analytic hierarchy process 252 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.761 appendix 1 ahp data table 6 pairwise comparison matrix of alternatives with respect of benefit (b) x10 x9 x8 x7 x6 x5 x4 x3 x2 x1 2 5 5 1/2 1/3 1/4 5 1/5 3 1 x1 ½ 2 3 1/5 1/7 1/7 4 1/7 1 1/3 x2 5 7 7 4 4 3 7 1 7 5 x3 1/5 1/3 ½ 1/5 1/7 1/7 1 1/9 1/4 1/5 x4 5 7 7 2 2 1 7 1/3 7 4 x5 5 7 7 2 1 1/2 7 ¼ 7 3 x6 3 5 5 1 1/2 1/2 5 ¼ 5 2 x7 ¼ 1/2 1 1/5 1/7 1/7 2 1/7 1/3 1/5 x8 1/3 1 2 1/5 1/7 1/7 3 1/7 1/2 1/5 x9 1 3 4 1/3 1/5 1/5 5 1/5 2 1/2 x10 table 7 relative weights of alternatives with respect to each criterion s2 s1 t2 t1 r b c4 c3 c2 c1 0.21740 0.20401 0.15206 0.10935 0.07420 0.07652 0.02005 0.19273 0.03907 0.07972 x1 0.14337 0.15165 0.01773 0.15463 0.07420 0.03578 0.03854 0.02352 0.10943 0.05784 x2 0.28473 0.28487 0.02175 0.03129 0.04200 0.30572 0.01467 0.19273 0.01669 0.01697 x3 0.01614 0.02094 0.05683 0.07789 0.07027 0.01728 0.11331 0.07114 0.02913 0.02149 x4 0.11111 0.08620 0.10817 0.04129 0.14424 0.19441 0.02789 0.01564 0.21664 0.02993 x5 0.05669 0.11092 0.27837 0.05599 0.15821 0.16121 0.05546 0.04829 0.28148 0.04190 x6 0.07943 0.05624 0.08043 0.02234 0.07420 0.10844 0.24624 0.19273 0.01993 0.21161 x7 0.04038 0.04070 0.21295 0.01574 0.07420 0.02075 0.24624 0.03418 0.15536 0.15102 x8 0.02882 0.02856 0.04148 0.28033 0.14424 0.02643 0.07846 0.11452 0.05768 0.10960 x9 0.02194 0.01590 0.03021 0.21112 0.14424 0.05346 0.15916 0.11452 0.07459 0.27992 x10 0.03755 0.04200 0.04280 0.05809 0.00452 0.06387 0.04100 0.01553 0.05400 0.04675 𝐶𝑅 table 8 alternatives final weights and rankings x10 x9 x8 x7 x6 x5 x4 x3 x2 x1 alternatives 0.1324 0.0934 0.1073 0.1145 0.1373 0.1112 0.0457 0.0945 0.0723 0.0914 final weights 2 7 5 3 1 4 10 6 9 8 ranking ijahp article: mohammed, benamar, mekidiche /ahp and wafgp hybrid model for information system project selection international journal of the analytic hierarchy process 253 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.761 appendix ii pre-emptive zogp mathematic formulation 𝑀𝑖𝑛 𝑧 = 𝑃1 (𝑛1),𝑃2 (𝑝2),𝑃3 (𝑝3),𝑃4 (𝑝4),𝑃5 (𝑝5),𝑃6 (𝑛6),𝑃7 (𝑛7),𝑃8 (𝑝8),𝑃9 (𝑝9),𝑃10 (𝑛10) ; (43) subject to: ∑ bixi 10 i=1 + n1 − p1 = 48000; (44) ∑ hixi 10 i=1 + n2 − p2 = 65000; (45) ∑ sixi 10 i=1 + n3 − p3 = 28000; (46) ∑ oixi 10 i=1 + n4 − p4 = 360; (47) ∑ ribixi 10 i=1 + n5 − p5 = 00; (48) ∑ prdixi 10 i=1 + n6 − p6 = 47; (49) ∑ pruixi 10 i=1 + n7 − p7 = 49; (50) ∑ tixi 10 i=1 + n8 − p8 = 00; (51) ∑ ttixi 10 i=1 + n9 − p9 = 00; (52) ∑ mixi 10 i=1 + n10 − p10 = 1100; (53) ∑ xi 10 i=1 = 5; (54) x1 = 1; (55) xi = 0 or 1; i = 1,2, . . ,10; (56) for zogp-ahp, the same mathematic formulation was used with the following objective function: 𝑀𝑖𝑛𝑍 = 𝑃2(𝑝2),𝑃3(𝑝3),𝑃1(𝑛1),𝑃5(𝑝5),𝑃4(𝑝4),𝑃10(𝑛10),𝑃8(𝑝8),𝑃9(𝑝9),𝑃6(𝑛6),𝑃7(𝑛7), 𝑃11(0.0914𝑛11 + 0.0723𝑛12 + 0.0945𝑛13 + 0.0457𝑛14 + 0.1112𝑛15 + 0.1373𝑛16 + 0.1145𝑛17 + 0.1073𝑛18 + 0.0934𝑛19 + 0.1324𝑛20); (56) ijahp article: islam, anis, abdullah/ critical success factors of the challenges in providing quality education: a study on malaysian private higher learning institutions international journal of the analytic hierarchy process 83 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.273 critical success factors of the challenges in providing quality education: a study on malaysian private higher learning institutions rafikul islam department of business administration kulliyah of management sciences and economics international islamic university malaysia (iium) jalan gombak, 53100 kuala lumpur e-mail: rislam@iium.edu.my azilah anis centre of transport and operations management faculty of business management universiti teknologi mara (uitm) 42300 bandar puncak alam, selangor, malaysia e-mail: azilahanis@salam.uitm.edu.my anisah abdullah department of business administration kulliyah of management sciences and economics international islamic university malaysia (iium) jalan gombak, 53100 kuala lumpur e-mail: anisah@iium.edu.my abstract the issue of quality education in higher learning institutions is timely and crucial due to the malaysian government’s aspiration to turn the country into a centre of educational excellence in the asian region. quality education acts as an indicator of the institution’s ability to provide tertiary education to the society as well as an instrument for the nation’s economic growth. to date, numerous studies have been conducted to measure the quality of education in higher learning institutions in malaysia. however, the task of identifying the challenges faced by these institutions in providing quality education and the critical success factors to address the challenges has largely been ignored by previous researchers. it is within this overall context that this study employs the analytic hierarchy process, with the aim of identifying and ranking the challenges and also their critical success factors. data was collected from the stakeholders of malaysian private hlis via semistructured interviews and a questionnaire survey. results indicate that ‘establishing financial capabilities for the institution’s self-sustainability’, ‘complying with the rules and regulations of regulatory agencies and relevant professional bodies’ and ‘providing facilities to ensure the delivery of quality education’ are the challenges that need to be carefully handled by the management of these malaysian private hlis. critical success factors that act as practical solutions to address each challenge were also identified and ranked in this study. ijahp article: islam, anis, abdullah/ critical success factors of the challenges in providing quality education: a study on malaysian private higher learning institutions international journal of the analytic hierarchy process 84 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.273 keywords: quality education, challenges, critical success factors, higher learning institution, analytic hierarchy process 1. introduction the malaysian government’s initiative in launching three educational acts in 1996 has resulted in increasing publicity and interest in malaysia’s educational sector development, specifically in its public and private higher learning institutions (hlis). the national council on higher education act, the private higher educational institutions act, and the national accreditation board have also impacted liberalisation and internationalisation of the higher educational sector in malaysia, enabling the transformation of malaysia into a centre of educational excellence in the asian region. the task of the 20 public universities, 27 polytechnics and 59 community colleges and more than 450 private hlis comprising universities, university colleges, foreign universities and private colleges (www.moe.gov.my) is not only to accommodate the explosive growth of student enrolments in malaysia, but also to collaborate with the ministry of education (moe) in realising the government’s aspirations (arokiasamy, 2011). developing malaysia into a centre of educational excellence and internationalising malaysia’s higher education are major priorities for the moe. as contended by muhamad et al. (2006), private hlis play important roles in fulfilling the government’s aspiration in transforming the nation into a centre of educational excellence in the region. the tenth malaysia plan (2011-2015) also emphasises the significance of private hlis as one of the instruments for driving the nation’s economic growth. in particular, the tenth malaysia plan aims to increase the gdp contribution from private hlis by 2% and attract 150,000 international students by 2015. the vehicle to achieve this objective is through maintaining high standards of quality education, specifically that provided by the private hlis (muhamad et al., 2006). however, negative reports and complaints pertaining to the quality of education in malaysian private hlis as emphasised by muhamad et al. (2006), morshidi (2006), the star online (2007), utusan malaysia (2008), the new straits times (2010), the star (2011), the new straits times (2012), and the star (2013) cast doubts on the quality measures undertaken by these institutions. the high number of unemployed graduates resulting from their less than desirable skills (woo, 2006), as well as the low numbers of qualified lecturers with phd qualifications (muhamad et al., 2006), are some aspects which point to the weaknesses of quality assurance systems in malaysian private hlis. moreover, there are different parties (fion, 2009) that are interested in how education should be run and a general lack of consensus as to the components that constitute quality education in hlis (mcnaught, 2003). fundamentally, the owners and administrators of private hlis not only face stiff competition within the industry, but are also struggling with limited resources to achieve the quality standards that have been established by the moe and malaysian quality assurance (mqa) (yaakob et al., 2009). these malaysian practitioners concur with the views of belle (2009), eric (2007) and donald (2003) that ensuring high quality standards in higher education constitutes one of the major challenges faced in order to remain sustainable in this highly competitive, global era. yet, despite being ijahp article: islam, anis, abdullah/ critical success factors of the challenges in providing quality education: a study on malaysian private higher learning institutions international journal of the analytic hierarchy process 85 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.273 acknowledged as one of the major challenges faced by hlis, few researchers have attempted to explore the issue in depth. hence, this study attempts to identify the most critical areas (challenges) faced by malaysian private hlis in providing quality education and how such challenges should be solved (critical success factors). by using the analytic hierarchy process (ahp), these challenges and critical success factors are then assessed to determine their ranking and contribution in advancing quality in malaysian private hlis. 2. literature review 2.1 quality education in higher learning institutions defining quality in hlis has proven to be a challenging task since researchers and practitioners hold differing views on quality (bornman, 2004). furthermore, as contended by harvey and green (1993), quality education is a term that is highly contested, considerably vague and highly contextual. nevertheless, three concepts have been identified in explaining quality education in hlis. firstly, it refers to the three elements of the educational system namely quality of input, quality of process and quality of output (sahney et al., 2008). input includes factors relating to students, teachers, administrative staff, physical facilities and infrastructure. the processes include teaching, learning and administrative activities while the outputs include examination results, employment, earnings and satisfaction. secondly, it relates to functions and activities of hlis such as its curriculum, teaching faculty’s qualifications, government, facilities, students’ characteristics, management and administration as well as interactive networking (the world declaration on higher education, 1998). the third approach which has gained prominence (watty, 2005) is the concept of quality education in hlis that is related to the stakeholders’ approach. as affirmed by vroeijenstijn (1991), it is advisable to define as clearly as possible the criteria that each stakeholder uses when judging quality education so that all these competing views are taken into account when assessing quality. recognising the importance of stakeholders in hlis as asserted by waaty (2005) and vroeijenstijn (1991), this study is framed by using the stakeholder’s approach to identify the challenges faced by malaysian private hlis in providing quality education and subsequently, the csfs to address each challenge. 2.2 challenges in higher learning institutions belle (2009), eric (2007) and donald (2003) agreed that ensuring high quality standards in hlis constitutes one of the major challenges faced by the hlis to remain sustainable in this highly competitive, global era. specifically, the challenges faced by the hlis identified from early studies include leadership (laurie, 2004; sirvanci, 2004; terry & stanley, 2002), cultural and organizational transformation (ahmad et al., 2007; sirvanci, 2004; srikanthan & dalrymple, 2003), program and curriculum (philip & danial, 2005), customer identification (sirvanci, 2004), accreditation (belle, 2009), faculty and other staff (muhamad et al., 2006), financial (philip, 2007; sirvanci, 2004) and technological advancement (sirvanci, 2004). the studies mentioned above have made conclusions largely based on conceptual standpoints (timothy, 2008; philip, 2007; eric, 2007; philip & danial, 2005; sirvanci, ijahp article: islam, anis, abdullah/ critical success factors of the challenges in providing quality education: a study on malaysian private higher learning institutions international journal of the analytic hierarchy process 86 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.273 2004; laurie, 2004; donald, 2003). hence, the present study extends existing works by examining the challenges faced by hlis, specifically in providing quality education and consequently, providing empirical conclusions by applying the ahp in ranking the challenges. 2.2 critical success factors and quality practices in hlis the principle of csf was proposed by ronald daniel in 1960 and achieved popularity in 1979 through the efforts of john rockart. according to rockart (1979), critical success factors (csfs) have a limited number of areas which, if satisfactory, will ensure competitive performance in organisations. however, in the context of the present study, csfs are the actionable solutions executed by the management of the institutions to address the challenges in providing quality education. the operationalising concept of csf was taken from owlia and aspinwall (1997) who investigated the csfs for tqm in hlis in the united stated and united kingdom, as well as islam (2010) who ranked csfs for the challenges in achieving malaysia’s vision 2020. these researchers conceptualised csfs as either the solutions to the problems encountered (owlia & apinwall, 1997) or the factors that must be implemented to successfully address the challenges (islam, 2010). 2.3 analytical hierarchy process in higher learning institutions in the education industry, ahp methodology has become an increasingly useful tool in different decision making situations (sipahi & timor, 2010). even though previous research has proven the acceptance of ahp as an effective tool in the educational sector, only a few studies touched on the ahp application in education quality management (henny & jan, 2006). some of the recent studies that incorporated the elements of education quality management with ahp application were conducted by aly et al. (2014), kiarazmi (2013), ao et al. (2012), pourhasomi et al. (2012), yeşim yayla and ortaburun (2011), anis and islam (2011), tsinidou et al. (2010), lam et al.(2008) as well as raharjo et al. (2007). 3. objectives overall, the study intends to enhance quality education provided by malaysian private hlis through the following specific objectives: 1. to determine various challenges faced by malaysian private hlis in providing quality education. 2. to ascertain the critical success factors for each challenge faced by malaysian private hlis in delivering quality education. 3. to rank the challenges and critical success factors for each challenge faced by malaysian private hlis in the course of providing quality education by using the analytic hierarch process (ahp). 4. research design/methodology the present study adopted the mixed methods approach to achieve its research objectives. the first stage involved the qualitative approach to identify the challenges faced by malaysian private hlis in providing quality education as well as to ascertain the practical ijahp article: islam, anis, abdullah/ critical success factors of the challenges in providing quality education: a study on malaysian private higher learning institutions international journal of the analytic hierarchy process 87 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.273 solutions (csfs) in addressing the challenges. the second stage of data collection utilised the quantitative approach to rank the identified challenges and critical success factors previously identified. the stakeholders of malaysian private hlis served as the sample for the first stage of data collection for the study. the sample was comprised of 26 respondents, ranging from the authorised personnel in tertiary education regulatory agencies including the ministry of education (moe) and malaysian qualification assurance (mqa), quality directors of the malaysian private hlis, employees of the institutions such as the academics and administrative staff, students, prospective employers of the graduates, parents as well as a member of the national association of private educational institutions (napei) (tang & hussin, 2011; fion, 2009; fion, 2008, harvey & green, 1993). as active participants in hlis, these stakeholders were selected since they are directly involved in deciding how quality is measured and/or in measuring and controlling quality (fion, 2009). in the context of private hlis in malaysia, three major groups of stakeholders were identified by fion (2008). the first group is the government, specifically the moe. the second group, the intermediaries, represent the authority that deals with the auditing or accrediting practices in the malaysian education industry, for example, the mqa and professional bodies. the third group of stakeholders range from the institution itself to the students, parents, employees and potential employers of the graduates. the institution involves the senior level management, the academics and administrators that run and operate the private hlis in malaysia. input from these stakeholders was analysed by applying thematic analysis, a method to analyse qualitative data, as suggested by cresswell, (2009). thematic analysis was applied to analyse the data for the challenges and csfs in order to specifically address each challenge. the list of identified challenges and csfs was modified thrice before the final list was obtained. the list was then validated via a content-construct method (kiarazm & koohkan, 2013) by three professors who have substantial experience in private and public hlis in malaysia. again, the list was amended according to these experts’ hindsight as the top management in the malaysian private hlis. the complete list of challenges and csfs for each challenge is shown in table 5. the ahp questionnaire design for the present study was then constructed by referring to the ahp questionnaires of hayrapetyan and kuruvilla (2012); kim et al. (2005); grandzol (2005) and strasser (2002). refer to appendix 1 for sample of the ahp questionnaire. thirty-five respondents representing stakeholders of malaysian private hlis were carefully selected for the second stage of data collection to enable the ranking of the challenges and the corresponding csfs for each challenge (refer to table 1). the details of the 35 selected respondents are as follows:  for the malaysian private hlis, respondents were selected from three categories. these range from the top management constituting the quality director of the institution, administrators from the managerial level and various academics including professors, phd holders and senior lecturers. the majority have more than 7 years of working experience in their current institutions. the participants from this category ijahp article: islam, anis, abdullah/ critical success factors of the challenges in providing quality education: a study on malaysian private higher learning institutions international journal of the analytic hierarchy process 88 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.273 and the institutions they represent (arokiasamy et al., 2009; mohamad et al., 2006) are listed in table 2.  for the tertiary education regulatory agencies, ahp responses were taken only from the authorised personnel who have substantial experience in handling quality issues of malaysian private hlis.  student views were obtained from those pursuing phd, masters and bachelors programmes at several malaysian private hlis. most were in the midst or end stage of their studies.  the ahp responses were also obtained from the parents of children in malaysian private hlis. their feedback is vital as they expect their children to be employed by public or private organisations as they have invested large amounts of money for their children’s education.  prospective employers representing the external customers of the education industry are also included in the present study. responses were acquired from the owner, general manager and human resource manager of the companies who have direct involvement in recruiting personnel in their organisations, particularly from the malaysian private hlis. demographic profiles of these 35 respondents are illustrated in table 3. the ahp questionnaires in the present study were collected from the respondents via the drop and collect survey method (brown, 1987). in this case, appointments were made to ensure that the questionnaires were personally handed to the respondents. a short briefing was then conducted to explain how the ahp questionnaire should be answered. the respondents were given 3 days to complete the questionnaire as the questionnaire consisted of 156 pairwise comparison statements. table 1 types of stakeholders types of stakeholders frequency percentage  malaysian private hlis 15 42.9  regulatory agencies 10 28.6  students 5 14.3  parents 3 8.6  prospective employer 2 5.7 total 35 100 table 2 number of respondents (from the hlis) and the institution’s ownership types of institution number of respondents percentage  supported by government linked companies 4 26.7  supported by state government/other influential group 5 33.3  owned by ‘entrepreneurs’ 2 13.3  owned by large companies 4 26.7 total 15 100 ijahp article: islam, anis, abdullah/ critical success factors of the challenges in providing quality education: a study on malaysian private higher learning institutions international journal of the analytic hierarchy process 89 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.273 table 3 stakeholder’s demographics profiles demographic profile frequency percentage gender  male 24 68.4  female 11 31.4 race  malay 25 71.4  chinese 5 14.3  indian 2 5.7  others 3 8.6 age group  21 – 30 years 4 11.4  31 – 40 years 12 34.3  41 – 50 years 9 25.7  51 year and above 10 28.6 educational level  certificate 1 2.9  diploma 1 2.9  professional 1 2.9  bachelors 9 25.7  masters 18 51.4  phd 4 11.4  others 1 2.9 5. data analysis for the present study, 35 respondents representing stakeholders of malaysian private hlis were chosen. all questionnaires were completed and answered as required, making them usable for analysis. a sample of the completed questionnaire in a pairwise comparison matrix (pcm) format is provided in exhibit 1. each respondent had different views in response to the significance of the challenges and the relevant csf for each challenge. exhibit 2 exemplifies the interval pcms where the interval of a specific comparison is determined by taking the minimum and the maximum for all responses pertaining to a specific judgement, an example of the interval being (1/9, 9). the wider the length of the interval, the more the respondents differed on the corresponding judgements. it is noted that the differences came from only two individuals, where the majority of the respondents revolved around some particular value within the interval (islam, 2010). in addition, as emphasised by dong et al. (2010), views from different groups of decision makers may differ substantially from each other. therefore, to aggregate different judgements from these 35 respondents, geometric means method was utilised (basak & saaty, 1993). the geometric means of group judgements is the mathematical equivalent of consensus if all the members are considered equal (islam, 2010). for that purpose, expert choice was used to calculate the priorities of the challenges and csfs for each challenge from the pcms using the geometric means. the pcms that are obtained using the geometric means with an acceptable value of consistency ratio (cr<0.1) are illustrated in exhibit 3. it is noted that though the consistency ratios for a number of individual pairwise comparison matrices are more than ijahp article: islam, anis, abdullah/ critical success factors of the challenges in providing quality education: a study on malaysian private higher learning institutions international journal of the analytic hierarchy process 90 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.273 0.10, the ratios for all the combined pairwise comparison matrices (after taking the geometric means) are less than 0.10. exhibit 1 a sample of completed ahp questionnaire in pcms cr = 0.42 csfs for competition csfs for lecturers c1 c11 c12 c13 c14 c15 c16 c2 c21 c22 c23 c24 c25 c26 c27 c11 9 8 1 7 7 c21 1 1 2 1/6 1/7 1 c12 1 1 1 5 c22 1 6 1 1 5 c13 1/5 1/5 1/3 c23 5 1 1 5 c14 1 7 c24 1/5 1/7 1/2 c15 1 c25 1 4 c16 c26 6 cr = 0.17 cr = 0.07 csfs for programme csfs for students c3 c31 c32 c33 c34 c35 c36 c37 c4 c41 c42 c43 c44 c45 c46 c31 1/7 1 ¼ 1/6 1 1/2 c41 1/7 6 7 1 5 c32 7 1 1 7 1 c42 7 7 1 6 c33 1/6 1/7 1/2 1 c43 1/6 1/7 1 c34 1 7 7 c44 1/6 1 c35 7 7 c45 7 c36 1 c46 c37 cr = 0.07 cr = 0.14 csfs for financial csfs for facilities c5 c51 c52 c53 c54 c55 c56 c57 c58 c6 c61 c62 c63 c51 1/8 1/8 1 1/3 1/5 1/5 1/6 c61 1 1 c52 1/8 8 8 7 1 7 c62 1 c53 9 9 9 9 9 c63 c54 1 1 1 7 cr = 0.00 c55 1 1 6 c56 1 2 c57 1 cr = 0.19 csfs for research csfs for accreditation c7 c71 c72 c73 c74 c75 c76 c8 c81 c82 c83 c84 c85 c71 1 1 1/8 1/4 1 c81 8 8 1 1 c72 1 1/8 1/8 1 c82 1 1/6 1 c73 1/8 1/6 1 c83 1/7 1 c74 8 8 c84 6 c75 1 c85 c76 cr = 0.12 cr = 0.10 challenges to provide quality education by malaysian private hlis c1 c2 c3 c4 c5 c6 c7 c8 c1 1/7 1/7 5 1 5 6 1/9 c2 7 7 6 1 7 1 c3 7 6 1 7 1/9 c4 7 1 5 1/6 c5 1/7 8 1 c6 8 1 c7 1/8 c8 ijahp article: islam, anis, abdullah/ critical success factors of the challenges in providing quality education: a study on malaysian private higher learning institutions international journal of the analytic hierarchy process 91 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.273 exhibit 2 interval pcms csfs for competition csfs for lecturers c1 c11 c12 c13 c14 c15 c16 c2 c21 c22 c23 c24 c25 c26 c27 c11 1/7,9 1/8,8 1/8,8 1/9,8 1/7,8 c21 1/9,8 1/9,8 1/9,8 1/7,8 1/8,8 1/8,8 c12 1,9 1/8,8 1/9,8 1/8,8 c22 1,9 3,9 1,9 1/8,9 1/8,9 c13 1/8,8 1/9,8 1/7,8 c23 1/7,8 1/7,8 1/8,9 1/8,8 c14 1/9,8 7,9 c24 1/7,9 1/9,7 1/9,1 c15 1/7,7 c25 1/8,7 1/8,7 c16 c26 1,8 csfs for programme csfs for students c3 c31 c32 c33 c34 c35 c36 c37 c4 c41 c42 c43 c44 c45 c46 c31 1/9,9 1/9,8 1/9,8 1/9,8 1/8,8 1/8,8 c41 1/8,8 1/7,8 1/8,8 1/8,9 1/8,8 c32 1/9,9 1/8,8 1/8,8 1/7,9 1/8,8 c42 5,9 1/8,9 1/9,8 1/8,8 c33 1/8,9 1/9,7 1/9,7 1/9,7 c43 1/3,1/8 1/9,1 1/8,8 c34 1/9,8 1/8,7 1/8,7 c44 1/9,8 1/8,8 c35 1/7,8 1/8,7 c45 1,8 c36 1/8.8 c46 c37 csfs for financial csfs for facilities c5 c51 c52 c53 c54 c55 c56 c57 c58 c6 c61 c62 c63 c51 1/9,8 1/9,1 1/9,9 1/9,9 1/9,5 1/9,8 1/9,9 c61 1/8,9 1/9,8 c52 1/9,1 1/8,9 1/9,9 1/9,7 1/9,5 1/8,9 c62 1/9,7 c53 1/8,9 1/9,9 1/9,9 1/9,9 1,9 c63 c54 1/9,8 1/9,8 1/9,8 1/7,9 c55 1/8,1 1/9,8 1/8,8 c56 1/8,8 1/8,8 c57 1/8,8 csfs for research csfs for accreditation c7 c71 c72 c73 c74 c75 c76 c8 c81 c82 c83 c84 c85 c71 1/8,9 1/8,9 1/9,1 1/8,8 1/9,9 c81 1/9,9 1/8,9 9,1/7 1/8,9 c72 1/8,9 1/8,1/3 1/8,8 1/9,8 c82 1/8,9 1/7,9 1/8,9 c73 1/8,1 1/9,7 1/8,6 c83 1/8,8 1/8,7 c74 1/8,8 1/8,9 c84 1/7,8 c75 1/8,7 c85 c76 challenges to provide quality education by malaysian private hlis c1 c2 c3 c4 c5 c6 c7 c8 c1 1/9,9 1/9,1 1/9,5 1/9,1 1/8,5 1/7,7 1/9,8 c2 1/7,8 1/3,8 1/9,8 1/8,8 1/2,8 1/9,8 c3 1/8,8 1/9,8 1/8,8 1/5,8 1/9,7 c4 1/9,7 1/7,8 1/7,8 1/9,6 c5 1/7,9 5,9 1/9,7 c6 1/7,8 1/9,3 c7 1/9,7 c8 ijahp article: islam, anis, abdullah/ critical success factors of the challenges in providing quality education: a study on malaysian private higher learning institutions international journal of the analytic hierarchy process 92 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.273 exhibit 3 pcms comprising the geometric means of individual judgements cr = 0.02 csfs for competition csfs for lecturers c1 c11 c12 c13 c14 c15 c16 c2 c21 c22 c23 c24 c25 c26 c27 c11 0.879 1.065 0.768 0.636 1.154 c21 0.592 0.406 1.196 1.326 0.796 0.565 c12 1.699 1.264 1.161 1.486 c22 1.300 1.661 2.263 1.029 1.736 c13 1.493 1.222 1.527 c23 2.423 1.957 0.369 1.139 c14 1.354 1.708 c24 1.412 0.369 0.725 c15 0.990 c25 0.337 0.458 c16 c26 0.934 cr = 0.02 cr = 0.01 csfs for programme csfs for students c3 c31 c32 c33 c34 c35 c36 c37 c4 c41 c42 c43 c44 c45 c46 c31 0.972 1.570 0.731 0.635 0.988 1.024 c41 0.761 1.704 0.894 0.281 0.760 c32 1.732 0.900 0.873 1.317 0.882 c42 3.983 1.708 0.675 1.623 c33 0.404 0.439 1.116 0.861 c43 0.617 0.278 0.658 c34 0.669 1.601 1.013 c44 0.394 1.101 c35 2.281 1.274 c45 2.866 c36 0.274 c46 c37 cr = 0.02 cr = 0.01 csfs for financial csfs for facilities c5 c51 c52 c53 c54 c55 c56 c57 c58 c6 c61 c62 c63 c51 0.561 0.716 1.179 0.569 0.568 0.605 0.438 c61 1.891 1.175 c52 1.018 1.389 0.873 0.797 0.984 0.650 c62 0.614 c53 1.622 1.602 1.278 2.008 0.744 c63 c54 1.080 0.587 0.657 0.357 cr = 0.00 c55 0.967 1.612 0.637 c56 1.336 0.509 c57 0.452 cr = 0.01 csfs for research csfs for accreditation c7 c71 c72 c73 c74 c75 c76 c8 c81 c82 c83 c84 c85 c71 1.129 0.399 0.528 0.528 0.461 c81 0.573 1.396 0.569 0.657 c72 0.584 0.333 0.685 0.443 c82 1.453 1.043 1.249 c73 0.465 0.975 0.805 c83 0.614 0.972 c74 1.550 0.984 c84 1.006 c75 0.557 c85 c76 cr = 0.01 cr = 0.01 challenges to provide quality education by malaysian private hlis c1 c2 c3 c4 c5 c6 c7 c8 c1 0.311 0.412 0.529 0.404 0.285 0.626 0.321 c2 1.018 1.657 0.497 1.155 1.199 0.527 c3 1.617 0.667 0.864 1.235 0.564 c4 0.817 0.604 1.551 0.850 c5 2.127 2.087 0.886 c6 1.288 0.622 c7 0.4165 c8 ijahp article: islam, anis, abdullah/ critical success factors of the challenges in providing quality education: a study on malaysian private higher learning institutions international journal of the analytic hierarchy process 93 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.273 6. findings 6.1 challenges in providing quality education by malaysian private hlis the challenges in providing quality education by malaysian private hlis that were identified in the first stage of data collection for the present study, their priorities and corresponding ranks are provided in table 4. the expert choice screen of priorities for the challenges is also illustrated in figure 1 below. table 4 challenges, their priorities and ranks challenges priorities rank gaining competitive advantage in a highly competitive environment (c1) 0.052 7 employing and retaining dedicated academics with phd qualifications, industrial experience and strong research backgrounds (c2) 0.128 4 offering programmes that are continuously relevant to the needs of industry and the nation (c3) 0.128 4 moulding and transforming students from poor academic backgrounds and with low soft skills (c4) 0.108 5 establishing financial capabilities for the institution’s self-sustainability (c5) 0.185 1 providing facilities to ensure the delivery of quality education (c6) 0.130 3 cultivating a research culture among academics (c7) 0.087 6 complying with the rules and regulations of regulatory agencies and relevant professional bodies (c8) 0.182 2 figure 1. expert choice screen of priorities for challenges data in table 4 and figure 1 indicate that malaysian private hlis should focus on the challenges that are ranked top most to ensure that quality education is provided to their stakeholders. the first top three challenges are ‘establishing financial capabilities for the institution’s self-sustainability, ‘complying with the rules and regulations of regulatory agencies and relevant professional bodies’ and ‘providing facilities to ensure the delivery of quality education’. the priority values of the challenges are 0.185, 0.182 and 0.130 respectively. the challenges for ‘employing and retaining dedicated academics with phd qualifications, industrial experience and strong research backgrounds’ and ‘‘offering programmes that are continuously relevant to the needs of industry and the nation’ are both placed at the fourth rank with priority value of 0.128. the least three importance challenges are ‘molding and transforming students from poor academic backgrounds and with low soft skills’, ‘cultivating a research culture among ijahp article: islam, anis, abdullah/ critical success factors of the challenges in providing quality education: a study on malaysian private higher learning institutions international journal of the analytic hierarchy process 94 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.273 academics’ and ‘gaining competitive advantage in a highly competitive environment’ with priority values of 0.108, 0.087 and 0.052 respectively. 6.2 critical success factors for each challenge besides the above findings, csfs for each challenge previously identified in the first stage of data collection were also ranked. details of the csfs for each challenge, their priorities and ranks are shown in table 5. explanations on the challenges and their csfs according to the ranking are elaborated on in the following paragraphs. data in table 4 confirms that ‘establishing financial capabilities for the institution’s selfsustainability’ is the challenge ranked first. therefore, efforts should be focused by the management of malaysian private hlis on ‘high competency in managing the institution’s finance’, ‘obtain the right number of students’ and ‘collaborate with the industries by commercialising and innovating their products as well as improving their processes’. this is because these are the csfs ranked first, second and third with priority values of 0.208, 0.153 and 0.130. ‘complying with the rules and regulations of regulatory agencies and relevant professional bodies’ is the challenge ranked second. out of eight csfs, it is noted that ‘top management commitment and support’, ‘all necessary actions should be taken with regards to the full audit report provided by the mqa’ and ‘establish precise structure of processes and standard operational procedures for the institution’ are the strategies that should be considered by the malaysian private hlis to address the issue of compliance. the priority values of these csfs are 0.247, 0.239 and 0.206 respectively the third ranked challenge is ‘providing facilities to ensure the delivery of quality education’. the findings of this study reveal that ‘allocate certain percentage of the institution’s annual budget to build and improve facilities’ (0.420), ‘comply with the facilities requirements as prescribed by the regulatory agencies (moe and mqa) and relevant professional bodies’ (0.359) and ‘establish an efficient facility/maintenance department’ (0.221) are the csfs that can be applied by the malaysian private hlis to address the issue on facilities. ‘employing and retaining dedicated academics with phd qualifications, industrial experience and strong research backgrounds’ is ranked fourth with the priority value 0.128. three csfs to address the challenge are identified with priority values of 0.208, 0.199 and 0.179 respectively. the first, second and third ranked csfs are ‘establish clear career pathways so that academics can plan the direction of their careers’, ‘offer attractive salary packages’ as well as ‘provide attractive benefits’. ‘offering programmes that are continuously relevant to the needs of industry and the nation’ is another challenge ranked fourth with the priority value of 0.128. to address this issue, several csfs can be utilised by the malaysian private hlis namely; ‘comply with the requirements of mqa and relevant professional bodies’, ‘grow experts to develop the institutions’ curricula’ and ‘continually review the curriculum due to the constant feedback from industry advisors, external examiners, professional bodies and mqa’. the priority values are 0.204, 0.171 and 0.166 respectively. ijahp article: islam, anis, abdullah/ critical success factors of the challenges in providing quality education: a study on malaysian private higher learning institutions international journal of the analytic hierarchy process 95 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.273 six csfs identified the challenge of ‘moulding and transforming students from poor academic backgrounds and with low soft skills’. this challenge is ranked fifth with the priority value of 0.108. focus should be given to the csfs of ‘have dedicated lecturers to deliver knowledge within the students’ area of studies’ (0.336), ‘enforce the teaching and practice of soft skills’ (0.218) and ‘provide relevant services (i.e., remedial classes and advisory system) to improve the performance of academically poor students’ (0.128) as they are ranked first, second and third. ‘cultivating a research culture among academics’ is the challenge that is ranked sixth with the priority value of 0.087. to address the challenge, six csfs are identified. however, ‘provide internal grants and facilitate applications for external grants’ (0.255), ‘provide research facilities for academics to involve actively in research such as financial support, equipment and reduction of teaching workload’ (0.230) and ‘establish a research management centre in order to plan, manage and increase research activities and publications’ (0.170) are the csfs that are placed top most to address the issue of research activities among academics for malaysian private hlis. the final challenge ‘gaining competitive advantage in a highly competitive environment’ (0.127) was ranked seventh in importance. ranking of the csfs by the respondents to address the issue of competition among malaysian private hlis are identified. the first, second and third csfs are ‘offer programmes that are in high demand in the industry/market’ (0.209), and ‘establish comprehensive excellence in every strata of governing the institution’ (0.179) and ‘offer competitive and affordable tuition fees’ (0.178). ijahp article: islam, anis, abdullah/ critical success factors of the challenges in providing quality education: a study on malaysian private higher learning institutions international journal of the analytic hierarchy process 96 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.273 table 5 critical success factors for each challenge, their priorities and ranks challenges critical success factors priorities rank gaining competitive advantage in a highly competitive environment (c1) develop and utilise relevant marketing strategies which help to differentiate the institutions from their competitors (c11) 0.149 5 offer programmes are in high demand in the industry/market (c12) 0.209 1 establish comprehensive excellence in every strata of governing the institution (c13) 0.179 2 offer competitive and affordable tuition fees (c14) 0.178 3 engage efforts to attain full-fledged university status (c15) 0.159 4 venture into programmes that are few or yet to be offered by the other private hlis, provided there is a good demand for the programme (c16) 0.127 6 employing and retaining dedicated academics with phd qualification, industrial experience and strong research background (c2) provide continuous training to enhance academics’ teaching skills, knowledge and motivation (c21) 0.092 6 offer attractive salary packages (c22) 0.199 2 provide attractive benefits (c23) 0.179 3 establish the young lecturers’ scheme (c24) 0.093 5 bring in foreign lecturers, particularly for critical programmes (c25) 0.075 7 establish clear career pathways so that academics can plan the direction of their career (c26) 0.208 1 establish avenues for academics to be prolific in research (c27) 0.155 4 offering programmes that are continuously relevant to the needs of industry and the nation (c3) establish strong linkages between the university and industries (c31) 0.130 5 embed soft skill components in the programmes and modules to enhance student development and employability (c32) 0.145 4 adapt best practices from collaborations with local and foreign universities (c33) 0.093 6 continually review the curriculum due to the constant feedback from industry advisors, external examiners, professional bodies and mqa (c34) 0.166 3 comply with the requirements of mqa and relevant professional bodies (c35) 0.204 1 invite industry experts to give seminars and conduct co-teaching to expose students to real business scenarios (c36) 0.091 7 grow experts to develop the institutions’ curricula (c37) 0.171 2 moulding and transforming students from poor academic backgrounds and with low soft skills (c4) offer bridging programme to enhance language skills and re-engineer thinking skills (c41) 0.118 5 enforce the teaching and practice of soft skills (c42) 0.218 2 hold continuous meetings and dialogues with the student representative councils (c43) 0.075 6 provide relevant services (i.e., remedial classes and advisory system) to improve the performance of academically poor students (c44) 0.128 3 have dedicated lecturers to deliver knowledge within the students’ area of studies (c45) 0.336 1 implement counselling systems to improve poor mind-sets and attitudes the students (c46) 0.126 4 establishing financial capabilities for the institution’s self-sustainability (c5) practice prudence and transparency in budgeting and spending (c51) 0.084 7 fully utilise the institution’s physical assets and multi-tasking of man power (c52) 0.112 5 ijahp article: islam, anis, abdullah/ critical success factors of the challenges in providing quality education: a study on malaysian private higher learning institutions international journal of the analytic hierarchy process 97 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.273 obtain the right number of students (c53) 0.153 2 establish consultancy and training centres to generate income (c54) 0.084 8 obtain continuous support from the state government or parent company (c55) 0.127 4 collaborate with industries by commercialising and innovating their products as well as improving their processes (c56) 0.130 3 establish a good relationship with the government to gain possible government benefits (c57) 0.102 6 high competency in managing the institution’s finance (c58) 0.208 1 providing facilities to ensure a delivery of quality education (c6) allocate certain percentage of the institution’s annual budget to build and improve facilities (c61) 0.420 1 establish an efficient facility/maintenance department (c62) 0.221 3 comply with the facilities’ requirements as prescribed by the regulatory agencies (moe and mqa) and relevant professional bodies (c63) 0.359 2 cultivating a research culture among academics (c7) develop post graduate programmes for the institution (c71) 0.099 5 establish a consultancy centre and collaborate actively with public and private sectors (c72) 0.094 6 establish a research management centre in order to plan, manage and increase research activities and publications (c73) 0.170 3 provide internal grants and facilitate applications for external grants (c74) 0.255 1 impose publication as one of the essential elements for promotion of academics (c75) 0.152 4 provide research facilities for academics to involve actively in research such as financial support, equipment and reduction of teaching workload (c76) 0. 230 2 complying with the rules and regulations of regulatory agencies and relevant professional bodies (c8) establish a quality assurance unit with strong professional links with the mqa and professional bodies (c81) 0.154 5 top management commitment and support (c82) 0.247 1 provide continuous internal and external training to ensure that the requirements of the mqa and professional bodies are compiled and can be executed by all levels of management (c83) 0.155 4 all necessary actions should be taken with regards to the full audit report provided by the mqa (c84) 0.239 2 establish precise structure of processes and standard operational procedures for the institution (c85) 0.206 3 7. conclusions this study provides some valuable insights into the challenges faced by the malaysian private hlis, particularly in providing quality education and the corresponding csfs that act as practical solutions to address the challenges. furthermore, the present study also provides some empirical conclusions by applying the ahp in ranking the challenges and csfs. by obtaining the responses from 35 participants who represent the stakeholders of malaysian private hlis; the ranking of the challenges and csfs for each challenge was established. geometric means method was used to aggregate different judgements from these 35 respondents. the geometric means, priorities and ranks were calculated with the ijahp article: islam, anis, abdullah/ critical success factors of the challenges in providing quality education: a study on malaysian private higher learning institutions international journal of the analytic hierarchy process 98 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.273 help of the software programme, expert choice. from the analysis, two important conclusions can be drawn: 1) from the eight challenges identified in the first stage of data collection, it is proven that ‘establishing financial capabilities for the institution’s selfsustainability’ is the challenge that requires urgent attention by the management of malaysian private hlis. the finding is in agreement with philip (2007) who stated that hlis need adequate funding if they are to provide quality education for the public. furthermore, sufficient funds are needed to achieve the efficiency and effectiveness aspired to by the top management of hlis, particularly the malaysian private hlis (sirvanci, 2004). the other two important challenges that should be carefully managed by the malaysian private hlis are ‘complying with the rules and regulations of regulatory agencies and relevant professional bodies’ as well as ‘providing facilities to ensure the delivery of quality education’. 2) the csfs that have been determined at the first stage of data collection are also ranked. these csfs act as practical solutions to address each challenge. the respondents observed that ‘high competency in managing the institution’s finance’, ‘obtain the right number of students’ and ‘collaborate with the industries by commercialising and innovating their products as well as improving their processes’ are the three significant csfs that should be considered by the management of malaysian private hlis to address the first ranked challenge of financial capability. to address the second ranked challenge of compliance, the survey reports that ‘top management commitment and support’, ‘all necessary actions should be taken with regards to the full audit report provided by the mqa’ and ‘establish a precise structure of processes and standard operational procedures for all the activities of the institution’ are the three relevant csfs that the malaysian private hlis should focus on to address the issue of complying with rules and regulation of regulatory agencies and relevant professional bodies. for the challenge of facilities, the study advises the management of malaysian private hlis to ‘allocate certain percentage of the institution’s annual budget to build and improve facilities’, ‘comply with the facilities’ requirements as prescribed by the regulatory agencies (moe and mqa) and relevant professional bodies’ and ‘establish an efficient facility/maintenance department’ as the functional solutions to address the issue. the findings of this study support the application of the ahp as a viable technique in higher education decision making. however, with a small sample size, caution must be taken, as it limits generalizability of the result. in future studies, it might be possible to proceed with a larger sample size particularly for each type of stakeholder. it is also suggested to examine whether the ranking for the challenges and their corresponding csfs differs according to the different groups of stakeholders for malaysian private hlis. the relationship between the challenges and the csfs can also be determined by applying the anp as a tool in future investigations. ijahp article: islam, anis, abdullah/ critical success factors of the challenges in providing quality education: a study on malaysian private higher learning institutions international journal of the analytic hierarchy process 99 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.273 references ahmad, h., francis, a, & zairi, m. (2007). business process reengineering: critical success factors in higher education. business process management journal, 13(3), 451469. aly, m.f., attia, h.a., & mohammed, a.m. (2014). prioritizing faculty of engineering education performance by using ahp-topsis and balanced scorecard approach. international journal of engineering science and innovative technology, 3(1), 11-23. anis. a. & islam. r. (2011). multiple criteria decision making ’10-’11. poland: the university of economics katowice. ao, y., yang b., yi f., & zou g. (2012). the application of modified delphi-ahp method in the college students’ comprehensive quality evaluation system. international journal of information and education technology, 2(4), 389-393. arokiasamy, l., ismail, m., ahmad, a., & othman, j. (2009). background of malaysia private institutions of higher learning and challenges faced by academics. the journal of international social research, 2(8), 60-6. arokiasamy, l. (2011). an analysis of globalization and higher education in malaysia. australian journal of business and management research, 1(9), 73-81. basak, i., & t.l. saaty (1993). group decision making using the analytic hierarch process. mathematical and computer modelling, 17, 101-109. belle, s.w. (2009, jun 11). colleges and universities must not rest on their laurels. diverse issues in higher education, 26(9), 66. retrieved from http://www.questia.com. accessed on november 20, 2010. bornman, m.g. (2004). program review guidelines for quality assurance in higher education: a south african perspective. international journal of sustainability in higher education, 5(4), 372-383. brown, s. (1987). drop and collect surveys. marketing intelligence & planning, 5(1), 1923. creswell, j. w. (2009). research design: qualitative, quantitative & mixed method approaches (3 rd ed.). thousand oaks, ca: sage publication inc. donald, e.h. (2003). building a leadership vision: eleven strategic challenges for higher education. educause review, 38(4), 24. dong, y., zhang, g., hong, w.c., xu, y., xu, y. (2010). consensus models for ahp group decision making under row geometric mean prioritization method. decision support systems, 49, 281–289. http://www.questia.com/ ijahp article: islam, anis, abdullah/ critical success factors of the challenges in providing quality education: a study on malaysian private higher learning institutions international journal of the analytic hierarchy process 100 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.273 eric, c. (2007). challenges facing business schools in the future. journal of management development, 26(1), 87-92. fion c.b.l. (2009). education hub at crossroads; the development of quality in assurance a competitive tool for singapore’s private tertiary education. quality assurance in education, 17(1), 79-94. fion, c.b.l. (2008). understanding quality assurance: a cross country case study. quality assurance in education, 16(2), 126 – 140. grandzol, j.r. (2005). improving the faculty selection process in higher education: a case for the analytic hierarchy process. association for institutional research, 6, 1-13. harvey, l., & green, d. (1993). defining quality. assessment and evaluation in higher education, 18(1), 9-34. hayrapetyan, r.l., and mohan kuruvilla (2012). a decision support system for college major selection. international journal of business, marketing, and decision sciences, 5(1), 80-93. henny, v.w., & jan, v. (2006). choosing a quality improvement project using the analytic hierarchy process. international journal of quality & reliability management, 23(4), 409-425. islam, r. (2010). critical success factors of the nine challenges in malaysia’s vision 2020. journal of socio-economic planning sciences, 44 (4), 199-211. kiarazm, a., & koohkan f. (2013). performance evaluation in higher education institutes with the use of combinative model ahp and bsc. journal of basic and applied scientific research, 3(4), 940-944. kim, hyunwoo;han, younggoo;kim, sehun;choi, myeonggil (2005). a curriculum design for e-commerce security. journal of information systems education, 16 (1), 55. lam, m.y., poon, k.k., & chin, k.s. (2008). an organisational learning model for vocational education in the context of tqm culture. international journal of quality and reliability management, 25(3), 238-255. laurie, l. (2004). embedding quality; the challenges for higher education. quality assurance in education, 12(4), 157-165. mcnaught, c. (2003). innovation and chance in higher education: managing multiple polarities. perspectives, 7(3), 76-82. morshidi, s. (2006). malaysia, in unesco-apeid & seameo-rihed. higher education in south east asia, bangkok: unesco-apeid. ijahp article: islam, anis, abdullah/ critical success factors of the challenges in providing quality education: a study on malaysian private higher learning institutions international journal of the analytic hierarchy process 101 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.273 muhamad j., chan h.c., shahnon, s., & sibly, s.s. (2006). enhancing quality of faculty in private higher education institutions in malaysia. higher education research monograph, national higher education research institute, monograph 9. owlia, m.s., & aspinwall, e.m. (1997). tqm in higher education – a review. international journal of quality & reliability management, 14(5), 527-543. philip, g.a. (2007). intelligent asia: fostering asia’s brightest. far eastern economic, 170(1), 53-57. philip, g.a., & daniel, c.l. (2005), private higher education: a global revolution. rotterdam: sense publisher. pourhasomi, m.h., khamseh, a.a., & hosseini, s.s.m. (2012). integrating kano’s model into quality function deployment to optimally identify and prioritise the needs of higher education: case study: engineering faculty of tarbiat moallem university. interdisciplinary journal of contemporary research in business, 4(4), 233246. rahorjo, h., xie, m., goh, t.n., & brombacher, a. c. (2007). a methodology to improve higher education quality using the quality function deployment and analytic hierarchy process. total quality management, 18(10), 1097-1115. rockart, j. (1979). chief executive’s define their own data needs. harvard business review, 57(2), 81-93. sahney, s., banwet, d.k., & karunes s. (2008). an integrated framework of indices for quality management in education: a faculty perspective. a tqm journal, 20(5), 502-519. sipahi, s., & timor, m. (2010). the analytic hierarchy process and analytic network process: an overview of applications. management decision, 48(5), 775-808. sirvanci, m. (2004). tqm implementation; critical issues for tqm implementation in higher education. the tqm magazine, 16(6), 382-386. srikanthan, g., & dalrymple, j. (2003). developing alternative perspective for quality in higher education. international journal of educational management, 17(3), 126-136. strasser, s.e., ceyhun ozgur, david l. schroeder, (2002). selecting a business college major: an analysis of criteria and choice using the analytical hierarchy process. american journal of business, 17(2), 47 – 56. tang, s.w., & hussin, s. (2011). quality in higher education: a variety of stakeholders perspectives. international journal of social science and humanity, 1(2), 126-131. terry, n.l., & stanley, e.f. (2002). benchmarking the challenge to quality program implementation. benchmarking international journal, 9(4), 374-387. the tenth malaysia plan (2011 – 2015). ijahp article: islam, anis, abdullah/ critical success factors of the challenges in providing quality education: a study on malaysian private higher learning institutions international journal of the analytic hierarchy process 102 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.273 the world declaration on higher education (1998). higher education in the twenty-first century: vision and action. world conference on higher education, paris. timothy, n. (2008). standards to assure quality in tertiary education: the case of tanzania. quality assurance in education, 16(2), 164 – 180. tsinidou, m., gerogiannis, v., & fitsilis, p. (2010). evaluation of the factors that determine quality in higher education: an empirical study. quality assurance in education, 18(3), 227-244. umayal karpagam, p.l., & suganthi, l. (2010). a strategic framework for managing higher educational institutions. advance in management, 3(10), 15-21. vroeijenstijn, a.i. (1991). external quality assessment: servant of two masters? conference on quality assurance in higher education, hong kong. watty, k. (2005). quality in accounting education: what say the academics? quality assurance in education, 13(2), 120-131. woo, k.y. (2006). malaysian private higher education: a need to study the different interpretations of quality. jasa review papers, 17-21. www.mohe.gov.my yaakob, m., ahmad tarmize, m.a., yunus, b., abdul ghani, z., & mokhtar, m.z. (2009). directions and challenges of private institutions of higher learning in malaysia: a holistic approach from the perspective of universiti tenaga nasional (uniten), asaihl conference, international university cambodia. yeşim yayla, a., & ortaburun, y. (2011). redesigning curriculum in higher education by using analytical hierarchy process and spearman rank correlation test. european journal of scientific research, 53(2), 271-279. http://www.mohe.gov.my/ ijahp article: islam, anis, abdullah/ critical success factors of the challenges in providing quality education: a study on malaysian private higher learning institutions international journal of the analytic hierarchy process 103 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.273 appendix appendix 1: sample for the ahp questionnaire for each statement below, please compare the relative importance with respect to: goal which is to provide quality education by malaysian private higher learning institutions. choose and circle only one number per row by using the following scale: 1= equal 3=moderate 5=strong 7=very strong 9=extreme challenges in providing quality education by malaysian private hlis 1. competition 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 lecturers 2. competition 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 programme 3. competition 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 students 4. competition 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 financial 5. competition 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 facilities 6. competition 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 research 7. competition 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 accreditation 8. lecturers 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 programme 9. lecturers 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 students 10. lecturers 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 financial 11. lecturers 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 facilities 12. lecturers 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 research 13. lecturers 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 accreditation 14. programme 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 students 15. programme 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 financial 16. programme 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 facilities 17. programme 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 research 18. programme 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 accreditation 19. students 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 financial 20. students 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 facilities 21. students 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 research 22. students 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 accreditation 23. financial 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 facilities 24. financial 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 research 25. financial 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 accreditation 26. facilities 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 research 27. facilities 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 accreditation 28. research 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 accreditation summary detail competition gaining competitive advantage in a highly competitive environment lecturers employing and retaining dedicated academics with phd qualification, industrial experience and strong research background programmes offering programmes that are continuously relevant to the needs of industry and the nation students moulding and transforming students from poor academic background and low soft skills financial establishing financial capabilities for the institution’s self-sustainability facilities providing facilities to ensure a delivery of quality education research cultivating a research culture among academics accreditation complying with the rules and regulations of regulatory agencies and relevant professional bodies microsoft word 6b_piratelli_performance_vol2_issue1.docx ijahp article: piratelli, belderrain / supporting the design of a performance measurement system with the analytic network process international journal of the analytic hierarchy process 44 vol. 2, issue 1, 2010 issn 1936-6744 supporting the design of a performance measurement system with the analytic network process claudio luis piratelli university center of araraquara (uniara) araraquara, sp, brazil email: clpiratelli@gmail.com mischel carmen n. belderrain aeronautics institute of technology (ita) são josé dos campos, sp, brazil email: carmen@ita.br abstract the development process for a performance measurement system (pms) can be split into four phases: (1) design; (2) planning and construction; (3) implementation, and; (4) operation and updating. the design phase focuses on the choice of performance indicators and is crucial to the success both of the pms and the organization. this paper deals with the design phase for a pms based on the performance prism using the analytic network process (anp) for modeling and ranking of the performance indicators. the application of the anp as support for the pms design was executed in the higher education sector with a view to the management of an undergraduate course in production engineering. the model and its results assured the representation of the various stakeholders´ objectives – in a significant and balanced manner – through 58 performance indicators distributed in four clusters: satisfaction, processes, capabilities and contribution. keywords: performance measurement system, the performance prism, anp, undergraduate course 1. introduction according to fernandes (2004), organizational performance measurement systems (pms) have been used for more than fifty years, when the tableau de board came about in france. currently, the balanced scorecard (bsc) from kaplan and norton (1990) is the most commonly used pms in corporations and its creators have been the most referenced in the literature over the last two decades – akkermans and oorschot, 2005). following kaplan and norton in the ranking of references in performance measurement are the proposers of the performance prism (neely et al., 2002). there are different approaches to the subdivision of the construction process into phases. the construction process for a pms (bsc, the performance prism, or other) can be subdivided into three large phases (bourne et al., 2000): design (construction), implementation and use of the performance measurements. neely et al. (2002) proposed another subdivision for the process in four phases: design, plan & build, implement & operate and refresh (update). the first, design, focuses on the choice of measurements and their metrics. the second, plan & build, plans the construction of the pms (type of system, form of data access, data distribution configurations and manipulation, etc.), in addition to communicating its goals to the organization. the third, implement & operate, is concerned with the operation of the pms (use of data for management). finally, the fourth phase, refresh, revises the pms and refines it. rob typewritten text http://dx.doi.org/10.13033/ijahp.v2i1.72 rob typewritten text rob typewritten text ijahp article: piratelli, belderrain / supporting the design of a performance measurement system with the analytic network process international journal of the analytic hierarchy process 45 vol. 2, issue 1, 2010 issn 1936-6744 in any of the two classifications, the design phase is crucial to the success of the pms and of the organization – unfeasible strategies and visions and badly-planned performance indicators (pi) are the leading factors causing the failure of pmss (bourne et al., 2002). neglecting this stage can result in the construction of a set of inappropriate measurements and metrics and lead to more serious consequences for an organization. according to neely et al. (2002), organizations usually choose measurements that are easily obtained – with the focus on alternatives instead of studying appropriate measurements for the fundamental goals — value-focused thinking (keeney, 1992). according to suwignjo et al. (2000), organizations do not dedicate time to structuring their performance measurements and understanding their interconnections in a logical manner. this could be decisive in the success of a performance measurement system because: (1) measurements must relate to the organization’s strategy; (2) performance measurements vary from organization to organization, and; (3) performance measurements are dynamic (changing with time). bourne et al. (2002) and smith (2005) corroborate the paragraphs above, stating that between 40% and 60% of large companies in the usa tried to implement the bsc at the end of the last century, and 70% failed, mainly due to: ● the wrong decision about what the measure. many companies identified their performance criteria through diverse techniques (such as brainstorming) without critical analysis of what really is important. in failing to identify a causal relationship between the performance indicators, it is not possible to establish a strategic map and, therefore, the measurements make no sense and are unfocused. ● failing during implementation for diverse reasons, chief among them: internal difficulties, such as boycotts by people who feel threatened, inadequate infrastructure (especially in information technology, which demands heavy investment), and a loss of focus (mainly due to the implementation time that takes from 18 to 24 months on average). the previous discussion highlights the critical importance of the design or construction phase. for this reason, this article focuses on the use of the analytic network process (anp) as a support method for the design phase of a pms. to this end, a multi-criteria decision model will be conceived of in the form of a network, based on the framework for the performance prism in order to ordinate the performance indicators identified as important by the stakeholders on an undergraduate course in production engineering. experimental performance evaluation will be used to validate the model. the article is structured in the following way: section 2 introduces the performance prism concepts; section 3 describes the anp steps; section 4 introduces the justification for choosing the application in the education sector; section 5 introduces the model and discusses the results achieved; finally, section 6 introduces the paper conclusion. 2. the performance prism model in the transition from the 20th to the 21st century, the performance prism model came about as a more flexible proposal in regard to the bsc, capable of being applied to any kind of organization/business. the result of various workshops on performance measurement run by researchers at andersen consulting and the british universities of cranfield and cambridge, the new model is based on three premises: (1) organizations must not center their efforts on satisfying only the expectations and needs of their shareholders and clients, but rather on all the stakeholders involved; ijahp article: piratelli, belderrain / supporting the design of a performance measurement system with the analytic network process international journal of the analytic hierarchy process 46 vol. 2, issue 1, 2010 issn 1936-6744 (2) an organization’s strategies, processes and capabilities must be well-integrated and aligned with the aim of delivering value to its stakeholders, and; (3) organizations and their stakeholders must understand their reciprocal relationships – stakeholders must contribute to organizations in order get value out of them. such premises can be represented in the five faces of a prism, as in figure 1. handy (2002) defines the performance prism (figure 2) as a model that helps identify the critical components of the strategies, processes and capabilities that need to be developed – from a managerial and performance control standpoint – as prerequisites for the satisfaction of stakeholders’ needs and expectations, as well as those of the organization itself. the analogy is to a prism, which, in refracting white light, illustrates the complexity of an apparently simple phenomenon (the same happens when thinking about an organization from the multifaceted standpoint of performance and management). the main difference between the performance prism and the bsc is the premise that, in the former, the strategies are not defined, but must be constructed by the identification of stakeholders’ needs and expectations. this affirmation is corroborated by handy (2002), who points out the main advantage of the performance prism in regard to the bsc: through application of the model in an organization, following the five perspectives in figure 2, the elements that must be approached by the managers become evident. figure 1. the five facets of the performance prism (source: neely, 2005). figure 2. the performance prism model (source: handy, 2002). 2.1 the first face of the prism: stakeholders the first face of the prism aims to reflect on who the fundamental stakeholders in the organization are (investors, employees, consumers, intermediaries, suppliers, regulators, and the community) and what their needs and expectations are. according to handy (2002), the concept “derive its measurements from strategy” is an error committed by nine out of ten citations related to the theme of performance measurement. performance measurements must help the managers to move in the direction desired and the strategy represents only one among many routes to achieving these goals, and may therefore be wrong. hence, instead of identifying the strategies of an organization, its stakeholders and their needs and expectations must be defined so that consistent strategies can be decided on. ijahp article: piratelli, belderrain / supporting the design of a performance measurement system with the analytic network process international journal of the analytic hierarchy process 47 vol. 2, issue 1, 2010 issn 1936-6744 an organization’s strategy must transmit its goals and a plan to achieve them. any and every action plan seeks to create value for its multiple stakeholders. so, a performance measurement system must begin from the perspective of all the stakeholders involved (neely et al., 2002). 2.2 the second face of the prism: contribution by stakeholders the second face of the prism aims to understand what the organization needs and wants from its stakeholders, for example: capital and credit from investors, loyalty and profitability from its clients, ideas and competencies from employees, quality materials and services from suppliers, and so on. this perspective is based on the premise that the organization needs contributions from its stakeholders to better play its role, just as they want to have their needs and expectations satisfied by it. for handy (2002), organizations need loyal and profitable consumers, good suppliers, loyal and satisfied employees, to in return deliver valuable products and services to clients, pay promptly for supplies and reward their employees, respectively. 2.3 the third face of the prism: strategies based on the previous faces, the third face of the prism seeks to reflect on which of the organization's strategies it must conceive of to satisfy the stakeholders. in other words, having defined the main stakeholders, their needs and expectations, and their contributions to the organization, strategies that will be adopted so that the organization can satisfy them must be defined. in this perspective, measurements must be established, the roles of which are: (1) to identify whether the strategies defined are being implemented; (2) to make communication of the strategy within the organization clear; (3) to encourage and incentivize the implementation of the strategies, and (4) to identify whether the strategies are working as planned. different authors have stated that, within an organization, people perform their functions better when they are evaluated by measurements. handy, in neely et al. (2002), says that when measurements are coherent with strategies, human behavior consistent with the strategies is achieved. according to neely et al. (2002), 90% of managers fail in implementing their strategies, because: (1) they assume hypotheses about the organization’s performance drivers – if such hypotheses are not true, the goals will not be achieved; (2) they do not develop “capabilities” for the internal processes and/or they plan processes that are not designed to execute the strategies in practice. in this regard, the authors corroborate kaplan and norton (1992), ratifying that correct measurement of indicators is crucial to the development of capabilities and processes. 2.4 the fourth face of the prism: processes once the strategies have been defined, the fourth face of the prism aims to identify which processes the organization needs to perfect to put the strategies into practice. a process should be understood as a set of operations, stages, and events, which are necessary to the execution of a certain job. within an organization it must be described where, when, and how the work will be done. conceptually, these are easier to understand through representation of the system: inputsactions-outputs-results. according to the authors of the performance prism, the entire process needs macro and micro measurements in order to provide an overview and identify critical details, such as the existence of bottlenecks. the whole process, then, must have someone in charge of identifying what performance measurements and metrics must be taken and by whom. such aspects can be classified, in turn, as measurements of efficiency and measurements of effectiveness. in general, measurements of efficiency are more closely related to process inputs and actions, and ijahp article: piratelli, belderrain / supporting the design of a performance measurement system with the analytic network process international journal of the analytic hierarchy process 48 vol. 2, issue 1, 2010 issn 1936-6744 measurements of effectiveness to outputs and results. measurements of inefficiency and variability are also important performance indicators, such as: defects, waiting time, time without adding value, overproduction, unnecessary movements, excessive stock, space wasting, pollution generated, oversizing, excessive complexity, etc. a compilation of various performance criteria common to a wide range of industrial processes can be found in the appendix of neely et al. (2002) and in the article by neely et al. (2005). 2.5 the fifth face of the prism – capabilities finally, the fifth face of the prism seeks to reflect on what capabilities need to be developed to conduct such processes. behind an efficient and effective process there must be capabilities. handy (2002) defines organizational capabilities as those formed by competent people, practices, technology and infrastructure capable of creating value for the stakeholders through distinct processes and operations. according to the authors of the performance prism, even the well-known capabilities of an organization – those that support the differentiated processes – must be constantly measured to guarantee their sustainability. this section succinctly presented the performance prism, a framework for organizational performance measurement that is based on performance indicators according to the various stakeholders. for additional information on the subject, reading of the authors referred to herein is recommended. 3. the analytic network process the section introduces the anp and its operation steps. the analytic network process (anp) is a multi-criteria decision-making support method from the american school, originating in graph theory, which allows the modeling of a decision-making problem in the form of a network, in order to achieve priorities as regards its elements (criteria and alternatives) (saaty, 2005). in the context of the anp, a network can be defined as a set of clusters, each one with its nodes, which can present dependency relations between each other (intraand inter-clusters) in any direction (including feedback). if the elements of a certain network only present dependency relations in one single direction there is a hierarchical structure. in other words, a hierarchical or tree structure may be understood as a particular network case (silva et. al., 2009). figure 3 illustrates the representation of a decision-making problem in network form. in figure 3, the clusters are represented by an ellipse, and the nodes belonging to a cluster are represented by full circles. the arrows indicate the relations of influence (dependency) between the elements. figure 3. network structure saaty (2005) classifies the nodes in a cluster as: (1) source component: that which exercises an influence on the other elements and is not influenced; (2) intermediary component: that which exercises an goal criteria alternatives ijahp article: piratelli, belderrain / supporting the design of a performance measurement system with the analytic network process international journal of the analytic hierarchy process 49 vol. 2, issue 1, 2010 issn 1936-6744 influence and is influenced by other elements, and (3) absorbing component: that which is only influenced by others. all three types of nodes are included in the example of a network structure portrayed in figure 3. in modeling a decision-making problem in a network, the hypothesis of independence between its elements (criteria and/or alternatives), necessary for the use of the analytic hierarchy process (ahp) by saaty (1980) – one of the widest-used methods in dealing with multi-criteria problems – is left aside. according to saaty (2005), the main advantage of the anp over the ahp is the possibility of working with problems whose criteria, sub-criteria and/or alternatives have interdependencies, which is very common in practice. hence, the results tend to be more effective than for the ahp as a cost to efficiency – greater analytical effort as the number of pairwise comparisons increases between the elements (paula and salomon, 2008). silva et al. (2009) present three stages for the application of the anp to a decision-making problem: 1) formulation of the problem, 2) judgments, and 3) algebraic development. the procedures contained in these steps are presented in brief in figure 4. figure 4. stages for the application of the anp to a decision-making problem (source: silva et al., 2009). stage 1 models the decision-making problem in two steps: the structuring of the problem and the construction of the network. ijahp article: piratelli, belderrain / supporting the design of a performance measurement system with the analytic network process international journal of the analytic hierarchy process 50 vol. 2, issue 1, 2010 issn 1936-6744 step 1.1: in structuring of the problem, the objective of the decision-making context is defined, the clusters, elements and alternatives for decision-making. step 1.2: the construction of the network indicates the dependency relations between the elements of the clusters. according to silva et al. (2009), the dependency relations are described in the matrixes of global and local reach, both of which are binary. the first indicates whether there are dependency relations (1. relation or 0. no relation) between intraor inter-clusters (any elements). the second describes the relation of dependency for each element of the network with the rest (1 or 0) elements. stage 2 can be summed up by the key word judgment. according to silva et al. (2009) ,it is executed in one step and a verification sub-step. step 2.1: pairwise comparisons. for all the connections established in step 1.2, pairwise comparisons must be made according to saaty’s fundamental scale (1980), chart 1. according to silva et al. (2009), two kinds of comparison are made in the anp: (1) between two or more elements when they influence another element in conjunction, and; (2) between two or more clusters (whenever there is at least one relation of dependency between any of its elements). chart 1. saaty’s fundamental scale (source: saaty, 1980). intensity of importance definition description 1 equal importance the two elements contribute equally to the goals 2 intermediate value 3 moderate importance experience and judgment favors one element in relation to the other 4 intermediate value 5 great importance experience and judgment strongly favors one element in relation to the other 6 intermediate value 7 very great importance one element is very strongly favored in relation to the other 8 intermediate value 9 absolute importance one element is absolutely prioritized in relation to the other figure 5 illustrates the use of saaty’s fundamental scale (chart 2) for pairwise comparison between two elements (x and y) – cluster or node – in a network. importance of x in regard to y x=y importance of y in regard to x 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 figure 5. illustration of saaty’s fundamental scale to compare two elements (source: silva et al., 2009). the judgments made in comparisons (1) and (2) described above are computed in decision matrixes of order n, reciprocal and positive (where n is the number of elements compared). for each decision matrix a, the eigenvector and maximum eigenvalue are calculated ( which express the priority value (w) of the elements compared. ijahp article: piratelli, belderrain / supporting the design of a performance measurement system with the analytic network process international journal of the analytic hierarchy process 51 vol. 2, issue 1, 2010 issn 1936-6744 according to gomes (2004), w e  can be obtained, respectively, by (1) and (4) or (5). ninawaw m j jii ,...,1/)()( 1    , (1) where: )2( ,...,1)( 1 nj a a aw m i ij ij ji    such that: 1       1,…,                                                    3                                                                          4   or  1                                                               5   another way to get the priority vector is by calculating the normalized geometric average for each line of matrix a (saaty, 2001). silva et al. (2009) also present the theoretical foundation of the numerical power method to get w and  , used by saaty (2005). this method is relatively simpler for matrixes of large dimensions and seeks convergence for the eigenvalue and eigenvector through the iteration of vectors (oliveira and belderrain, 2008). as the number of comparisons to be made in the anp depends on the number of judgment matrixes between related nodes and between clusters that present inter-related elements, saaty proposes the use of superdecisions software to make comparisons and the respective algebraic calculations in stage 3. equation (6) presents the number of comparisons necessary for the n judgment matrixes ni a decisionmaking problem, where ni is the order of the i-th matrix. ∙ 1 2                                                                          6 step 2.1.1 verifies the consistency of the comparison judgments made in 2.1. the decision matrix a is said to be consistent when all the value judgments are perfect, which means to say that aij x ajk = aik, for any i, j, k (gomes, 2004). in other words, the eigenvector for a ( must be the closest to n. nevertheless, saaty (1980) admits a certain degree of inconsistency in human judgments, above all in quadratic matrixes with n>3, through the indicator ic defined in (7):  1                                                                    7    ijahp article: piratelli, belderrain / supporting the design of a performance measurement system with the analytic network process international journal of the analytic hierarchy process 52 vol. 2, issue 1, 2010 issn 1936-6744 hence, he proposes the calculation consistency ratio (cr), obtained by (8), where ir (index random) are randomly tabled values in function of n, presented in chart 2. according to gomes (2004), when n=2, cr must be zero; when n=3, cr must be less than 0.05; when n=4, cr must be less than 0.09 and; for n>4 cr must be less than or equal to 0.10.                                                                                   8 saaty (1994) observes that the inconsistency indicator must be used to alert the decision maker to the need for a possible revision of their judgments. in other words, the rectification of judgments is not compulsory. chart 2. ir values for squared matrixes of order n, according to the oak ridge national laboratory, usa (source: gomes, 2004). n 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 ir 0 0 0.58 0.9 1.12 1.24 1.32 1.41 1.45 1.49 1.51 1.48 1.56 1.57 1.59 stage 3, according to silva et al. (2009), comprises the construction of the supermatrices and getting results. the authors subdivide it into 5 steps: step 3.1: construction of the supermatrix without weight w (generically represented in figure 6). in w, the network clusters are defined by ch (h=1, 2, ..., n) and the respective nodes by hnn, in the following form: nhnhh eee ,,, 21  . the components whh of the supermatrix represent the matrixes obtained by aggregating the eigenvectors obtained in the pairwised comparisons between the elements through step 2.1 (silva et al., 2009). figure 6. standard structure of a supermatrix (source: saaty, 2005). step 3.2: obtaining the weighted supermatrix through the multiplication of each matrix whh by the corresponding weight of the cluster ch. step 3.3: verification of the weighted supermatrix. according to saaty (2005), the weighted supermatrix obtained in step 3.2 must be stochastic in regard to the columns. otherwise, it must be normalized by the sum in regard to the columns. step 3.4: calculation of the limit matrix through the power method described by oliveira and belderrain (2008). the limit matrix must also be stochastic in regard to the columns. step 3.5: obtaining the results for ranking of the alternatives and criteria, according to the limit matrix obtained in step 3.4. ijahp article: piratelli, belderrain / supporting the design of a performance measurement system with the analytic network process international journal of the analytic hierarchy process 53 vol. 2, issue 1, 2010 issn 1936-6744 for a better illustration of the anp method, reading of saaty (2005), figueira et al. (2005), saaty and vargas (2006) and silva et al. (2009) is recommended. for an understanding of the functioning of the superdecisions software, reading of saaty (2003) is recommended 4. application in higher education this section briefly explains why a pms should be constructed for educational institutions, more specifically, for undergraduate courses. in brazil, most educational institutions still work without a control and management system using indicator measurement (bressiani et al.,2001). the managers of higher educational institutes (hei) and programs, for the most part, do not have a management system that includes performance indicators for their business units (courses) with the level of detail and scope necessary for effective management. course and program coordinators are generally aware of the performance indicators used by the brazilian ministry of education (mec) – government – for course accreditation processes, but often do not have access to other important indicators such as the financial impact of their courses and the satisfaction of those benefited directly and indirectly by the service provided. many private hei managers have financial control through indicators that often do not describe the true cost/benefit relations of the programs/courses in their departments/institutions. a review of the literature shows the increase over time of work proposing the use of the pms as a strategic management system for heis. nevertheless, most of them suggest performance indicators for the institutions, but none deals with the application of the bsc in its full conception, according to the proposal by kaplan and norton (1992). higher educational institutions may have dozens of separate business units, focused on diverse areas of knowledge with their own goals, targets and operational strategies. their corporate strategies and missions, however, tend to be generic. porter (1998) suggests that competition in a given sector is at the level of business units and not between corporations. it makes sense, then, that pmss be molded to the business units, as their strategies must support the corporation’s strategy. 4.1 evaluation by ministry of education (mec) currently, brazilian higher education is evaluated by two agents: one internal to the hei itself, called the self-evaluation commission, whose main instrument is institutional self-evaluation; the other is carried out by external agencies linked to the ministry of education (mec), which carry out inspections: registration and re-registration of institutions, authorization, accreditation and renewal of accreditation for courses, and examining student performance. these agents and their instruments comprise sinaes – the national higher education evaluation system – created by law n° 10,861 (2004). the three main instruments used by sinaes are:  institutional evaluation that aims identify the profile, vocation and operation of the hei, through its activities, courses, programs, projects and sectors, respecting the diversity and specifications of the different academic organizations;  evaluation of the undergraduate courses, with a view to conceptualizing the teaching conditions offered through three main categories: didactic-pedagogical organization, the academic and technical-administrative staff, and the physical installations;  evaluation of student performance on undergraduate courses, via the national student performance examination (enade), to verify student performance in terms of general and specific knowledge acquired, besides skills and competencies required of the career chosen. ijahp article: piratelli, belderrain / supporting the design of a performance measurement system with the analytic network process international journal of the analytic hierarchy process 54 vol. 2, issue 1, 2010 issn 1936-6744 however, what is noted in practice is that the mechanisms of evaluation created by mec are not being carried out as planned by sinaes. the very expansion of higher education in the face of the limited capacity of the ministry to evaluate the universe of higher courses has become one of the major stumbling blocks in the system. the focus in evaluation now on diagnosing the quality of higher education is centered on enade and its indicators. according to the concepts obtained from students on a course that was assessed by this exam, mec calculates a preliminary course concept (cpc), which can dispense it from a renewal of accreditation process (cpc ≥ 3) (normative ordinance n. 4, 2008). for macedo et al. (2005), even if the full range of sinaes evaluation were fully implemented, it would not be enough to contemplate the size and heterogeneity of current higher education. in the scope of evaluation, other authors have suggested that an effective project to reform higher education must conceive of an evaluation system that can handle the different educational segments and institutions that comprise the educational system. in other words, it must be able to identify the strengths and weaknesses of each hei in order to be able to improve them. this makes it necessary for the instruments of evaluation to be able to describe the trajectory followed by the institution and, mainly, compliance with its mission, through careful measurement of pertinent performance indicators. piratelli et al. (2009) present the current evaluation instrument that supports the government accreditation processes for undergraduate courses and some of their potential deficiencies. the results of evaluation process simulations have identified that potential injustices could occur when using the instrument without the subjective intervention of an evaluation commission. the authors particularly show that it is easily possible to approve a course that is failing to comply with the demands of the labor market and pedagogically poorly structured because it has a good physical structure and good people. they also show that a course committed to pedagogical and professional aspects may not have a minimum concept of accreditation because of some deficiencies in the way it hires academic staff and promotes them. in addition, from the balanced scorecard or the performance prism standpoint, mec’s diagnostic and evaluation instruments focus on internal process and learning/growth indicators, and ignore performance from the point of view of the other stakeholders – and mainly for the direct and indirect clients, who chiefly benefit from their processes and products. the arguments presented in this section justify the application of the performance prism in higher education, more specifically for the management of university courses. ijahp article: piratelli, belderrain / supporting the design of a performance measurement system with the analytic network process international journal of the analytic hierarchy process 55 vol. 2, issue 1, 2010 issn 1936-6744 figure 7. the pms modeled with anp in the superdecisions software. 5. modeling of the performance prism in the anp to introduce the model and its results, this section follows the same stages described in section 3. the results are subdivided in sections 5.1 (model and results), 5.2 (evaluation) and 5.3 (model sensitivity analysis). 5. the model and its results stage 1: formulation of the problem. step 1.1: the objective of the problem is to order the performance indicators (pi) for an undergraduate course in production engineering. the pi were identified as relevant by its stakeholders (students, academic staff, educational institute (manager), organizations, and society). the ranking will serve to evaluate the course performance from the standpoint of the various stakeholders, allowing better strategic management (focusing on critical points). step 1.2: the pms was modeled on 4 of the 5 faces of the prism: satisfaction, value delivery processes, capabilities and stakeholder contributions. each face of the prism is represented by a cluster in the anp. so the clusters are satisfaction, processes, capabilities and contribution. the course performance is measured through satisfaction indicators for the stakeholders: students, academic staff, educational institution, society and organizations. satisfaction indicators for each stakeholder, in turn, depend on the nodes belonging to each of the other clusters. as in section 2, the face “strategy” is not measurable, and so is not incorporated in the model. the decision-makers (collegiate board) understand that strategic direction can only be conceived after knowing the importance of each indicator from the stakeholders’ standpoint. figure 7 presents the pms model. chart 3 has the key for the clusters and nodes in the pms network. chart 3: model key code indicator or cluster name code indicator or cluster name ab1 laboratories basic scope cu3 activities content integration ab2 laboratories specific scope cu4 teaching multi-methodology ab3 professional training laboratories scope d global performance bb1 collection update (library) dc1 focused complementary activities bb2 number of copies (library) dc2 guideline percentages ijahp article: piratelli, belderrain / supporting the design of a performance measurement system with the analytic network process international journal of the analytic hierarchy process 56 vol. 2, issue 1, 2010 issn 1936-6744 code indicator or cluster name code indicator or cluster name c1 economic competencies to engineering dc3 students intern supervision c2 general competencies to engineering en1 assisting pupils c3 human competencies to engineering en2 technical visits c4 socio-political competencies to engineering eq1 laboratories basic equipment c5 technical competencies to engineering eq2 laboratories specific equipment cap cluster: capabilities eq3 laboratories professional equipment cd1 teaching didactics fi1 classroom climate cd2 teaching experience fi2 classroom space cd3 professional experience (teacher) fi3 classroom furniture cd4 degree (teacher) gp cluster: global performance co1 collegiate board performance i cluster: main indicator for each subnet co2 structuring core faculty performance ie1 library co3 time to coordinate ie1 cluster: library cp1 level of entering students ie2 laboratories cp2 secretarial ie2ab cluster: laboratories : scope cp2 cluster: secretarial ie2eq cluster: laboratories : equipment/material cp3 working infrastructure ie3 classrooms cp3 cluster: working infrastructure ie3fi cluster: classrooms – physical aspects cp4 coordination ie3rd cluster: classrooms – didactic resources cp4 cluster: coordination if1 percentage of occupation cp5 capacity of academic staff if2 percentage revenue /revenue potential cp5 cluster: capacity of academic staff io1 enade concept cp6 teaching infrastructure it1 scientific bases cp6 cluster: teaching infrastructure it2 work rooms cp7 pedagogical policy project p1 publications cp7co cluster: project consistency p2 service provision (to society) cp7cu cluster: curriculum indicators p3 social projects cp7dc cluster: compliance by dcs p4 solving organizations problems cs1 curriculum-goal p4 cluster: resolution of problems cs2 curriculum-intended egress profile p5 competencies to engineering ct1 contribution academic staff p5 cluster: competencies ct1en cluster: teaching commitment pe cluster: processes ct1re cluster: rule compliance rd1 internet access ct2 employability rd2 multimedia (availability) ct3 contribution hei-staff re1 staff punctuality ct3 cluster: contribution hei-staff re2 staff deadlines ct4 contribution students and society rp1 quality of interns ct4fi cluster: financial rp2 research ct4io cluster: others (enade) rp3 monographies applied ct5 contribution hei –students (scholarships p-e) s1 students (satisfaction) cti1 research incentive s2 academic staff (satisfaction) cti2 working hours s3 hei (satisfaction) cti3 career plan s4 organizations (satisfaction) cti4 remuneration s5 society (satisfaction) ijahp article: piratelli, belderrain / supporting the design of a performance measurement system with the analytic network process international journal of the analytic hierarchy process 57 vol. 2, issue 1, 2010 issn 1936-6744 code indicator or cluster name code indicator or cluster name ctrb cluster: contribution sa1 number of terminals cu1 professional attributions sa2 employee qualification cu2 regional focus sat cluster: satisfaction the dependency relations between the elements of the network in figure 7 are presented in the global reach matrixes (charts 4 and 6) and in the local reach matrixes (charts 5 and 7). stage 2: judgment matrixes and verification of consistencies step 2.1: the judgment matrixes were filled in, mostly, by members of the collegiate board (5 academic staff and 2 students). each group of stakeholders judged, in a consensual manner, the pertinent indicators and clusters. the indicators referring to the hei stakeholders (director board) and organizations were judged by members external to the collegiate board in identical questionnaires shown in figure 5. in these cases the geometrical average was used to get the priority vectors (saaty and peniwati, 2007). the judgments of the indicators for the satisfaction cluster were made in a consensual manner by all members of the collegiate board, resulting in the priority vector in table 1. chart 4. global reach matrix. gp sat pe cap ctrb gp 0 1 0 0 0 sat 0 0 1 1 1 pe 0 0 1 1 1 cap 0 0 0 0 0 ctrb 0 1 0 0 0 chart 5. main network local reach matrix. cap ctrb g p pe sat cp1 cp2 cp3 cp4 cp5 cp6 cp7 ct1 ct2 ct3 ct4 ct5 d p1 p2 p3 p4 p5 s1 s2 s3 s4 s5 c a p cp1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 cp2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 cp3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 cp4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 cp5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 cp6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 cp7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 c t r b ct1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 ct2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 ct3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 ct4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 ct5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 g p d 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 p e p1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 p2 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 p3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 p4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 p5 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 s a t s1 1 1 0 0 1 1 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 s2 0 0 1 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 ijahp article: piratelli, belderrain / supporting the design of a performance measurement system with the analytic network process international journal of the analytic hierarchy process 58 vol. 2, issue 1, 2010 issn 1936-6744 s3 0 0 0 1 0 0 0 0 0 0 1 0 0 1 1 1 0 0 0 0 0 0 0 s4 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 s5 0 0 0 0 1 1 1 0 0 0 0 0 0 1 1 1 0 1 0 0 0 0 0 chart 6. subnet global reach matrices. chart 6. subnet global reach matrices (continuance) chart 7. subnet local reach matrices. i p4 i p5 i cp2 i 0 1 i 0 1 i 0 1 p4 0 0 p5 0 0 cp2 0 0 i cp3 i cp4 i cp5 i 0 1 i 0 1 i 0 1 cp3 0 0 cp4 0 0 cp5 0 0 i cp6 i ct3 i ie1 i 0 1 i 0 1 i 0 1 cp6 0 0 ct3 0 0 ie1 0 0 i ct4if ct4io i ct1en ct1re i ie2ab ie2eq i 0 1 1 i 0 1 1 i 0 1 1 ct4if 0 1 0 ct1en 0 0 0 ie2ab 0 0 0 ct4io 0 0 0 ct1re 0 0 0 ie2eq 0 0 0 i cp7co cp7cu cp7dc i ie3fi iefrd i 0 1 1 1 i 0 1 1 cp7co 0 0 0 0 ie3fi 0 0 0 cp7cu 0 0 0 0 iefrd 0 0 0 cp7dc 0 0 0 0 ijahp article: piratelli, belderrain / supporting the design of a performance measurement system with the analytic network process international journal of the analytic hierarchy process 59 vol. 2, issue 1, 2010 issn 1936-6744 table 1. priority vector for the elements of the satisfaction cluster (stakeholders). stakeholder priority stakeholder priority s1 23.81% s4 19.05% s2 19.05% s5 19.05% s3 19.05% step 2.2. the consistency of the judgments was guaranteed in a satisfactory manner in all the comparison matrixes. stage 3 corresponds to the construction of the supermatrices and obtaining results from the model. steps 3.1 to 3.4 have been omitted for reasons of space. step 3.5: results. the results for the model (weight of indicators in the evaluation) are in table 2. i i p4 rp1 rp2 rp3 p5 c1 c2 c3 c4 c5 i p4 0 1 1 1 i p5 0 1 1 1 1 1 rp1 0 0 0 0 c1 0 0 0 0 0 0 rp2 0 0 0 0 c2 0 0 0 0 0 0 rp3 0 0 0 0 c3 0 0 0 0 0 0 c4 0 0 0 0 0 0 c5 0 0 0 0 0 0 i i i s1 s2 cp2 it1 it2 cp3 co1 co2 co3 cp4 s1 0 0 0 it1 0 0 0 co1 0 0 0 0 s2 0 0 0 it2 0 0 0 co2 0 0 0 0 i cp2 1 1 0 i cp3 1 1 0 co3 0 0 0 0 i cp4 1 1 1 0 i i cd1 cd2 cd3 cd4 cp5 ie1 ie2 ie3 cp6 cd1 0 1 0 0 0 ie1 0 0 0 0 cd2 0 0 0 0 0 ie2 0 0 0 0 cd3 0 0 0 0 0 ie3 0 0 0 0 cd4 0 0 0 0 0 i cp6 1 1 1 0 i cp5 1 1 1 1 0 cp7 i cp7 cs1 cs2 cu1 cu2 cu3 cu4 dc1 dc2 dc3 en1 en2 re1 re2 ct1 cp7 cp7 0 1 1 1 1 1 1 1 1 1 en1 0 0 0 0 0 cs1 0 0 0 0 0 0 0 0 0 0 en2 0 0 0 0 0 cs2 0 0 0 0 0 0 0 0 0 0 re1 0 0 0 0 0 cu1 0 0 0 0 0 0 0 0 0 0 re2 0 0 0 0 0 cu2 0 0 0 0 0 0 0 0 0 0 i ct1 1 1 1 1 0 cu3 0 0 0 0 0 0 0 0 0 0 cu4 0 0 0 0 0 0 0 0 0 0 i dc1 0 0 0 0 0 0 0 0 0 0 ie1 bb1 bb2 dc2 0 0 0 0 0 0 0 0 0 0 i ie1 0 1 1 dc3 0 0 0 0 0 0 0 0 0 0 bb1 0 0 0 bb2 0 0 0 i i ie2 ab1 ab2 ab3 eq1 eq2 eq3 ie3 fi1 fi2 fi3 rd1 rd2 i ie2 0 1 1 1 1 1 1 i ie3 0 1 1 1 1 1 ab1 0 0 0 0 0 0 0 fi1 0 0 0 0 0 0 ab2 0 0 0 0 0 0 0 fi2 0 0 0 0 0 0 ab3 0 0 0 0 0 0 0 fi3 0 0 0 0 0 0 eq1 0 0 0 0 0 0 0 rd1 0 0 0 0 0 0 eq2 0 0 0 0 0 0 0 rd2 0 0 0 0 0 0 eq3 0 0 0 0 0 0 0 i ct4if ct4io i cti1 cti2 cti3 cti4 ct3 if1 if2 io1 ct4 cti1 0 0 0 0 0 ct4if if1 0 0 0 0 cti2 0 0 0 0 0 if2 1 0 0 0 cti3 0 0 0 0 0 ct4io io1 0 0 0 0 cti4 0 0 0 0 0 i ct4 1 1 1 0 i ct3 1 1 1 1 0 subnet ct3 ct3 ct3 subnet ct4 ie3rd subnet ie2 ie2ab ie2eq ie2ab ie2eq ie1 subnet ie3 ie3fi ie3rd ie3fi ct1re ct1en ct1re subnet ie1 ie1 cp6 cp6 cp4 subnet ct1 ct1encp7dccp7cucp7co cp7co cp7cu cp7dc p4 p5 cp2 cp3 cp5 subnet cp4 cp4 subnet cp6 p4 p5 cp2 cp3 cp5 subnet p4 subnet p5 subnet cp2 subnet cp3 subnet cp5 subnet cp7 ijahp article: piratelli, belderrain / supporting the design of a performance measurement system with the analytic network process international journal of the analytic hierarchy process 60 vol. 2, issue 1, 2010 issn 1936-6744 table 2. weight of indicators in the evaluation of course performance. indicators weight in evaluation indicators weight in evaluation indicators weight in evaluation indicators weight in evaluation cd1 3.58% eq2 2.15% c4 1.44% sa1 1.08% cd2 3.58% eq3 2.15% c5 1.44% ab1 1.07% p2 3.31% p1 2.01% it1 1.39% fi1 1.06% ab2 3.22% cu1 1.98% rp2 1.38% fi3 1.06% ab3 3.22% cu2 1.98% sa2 1.33% bb1 1.03% p3 3.14% cu3 1.98% if1 1.31% it2 1.00% co2 2.92% cu4 1.98% io1 1.31% dc1 0.99% co3 2.92% rp3 1.70% cp1 1.27% dc2 0.99% cd3 2.79% cd4 1.64% bb2 1.25% dc3 0.99% fi2 2.66% ct5 1.61% cti4 1.25% rd1 0.98% rp1 2.37% ct2 1.52% cti2 1.25% eq1 0.72% cs1 2.34% co1 1.46% cti3 1.25% re2 0.58% cs2 2.34% c1 1.44% en2 1.16% if2 0.44% en1 2.31% c2 1.44% re1 1.16% cti1 0.42% rd2 2.21% c3 1.44% 5.2 evaluation of course performance through the ranking of indicator priorities obtained (table 2), it was possible to measure course performance individually, as well as globally. for each indicator the course collegiate board described five levels of impact. for each descriptor, a function value was constructed, varying between 0 and 100 points with a corresponding one for each level of impact. course evaluation was done by the collegiate board. table 3 presents the satisfaction percentage for each stakeholder (performance). the global course indicator was 58.62%. figure 8 details the performance indicators pertinent to each stakeholder, illustrating chromatically what must be prioritized by the manager: red-orange: worst performance (urgent action required); yellowgreen: average performance (attention: improvement is required) and; green-dark green: good performance – no action required, keep monitoring). ijahp article: piratelli, belderrain / supporting the design of a performance measurement system with the analytic network process international journal of the analytic hierarchy process 61 vol. 2, issue 1, 2010 issn 1936-6744 figure 8. the pms: its indicators and evaluation of the course for management. table 3. course performance by satisfaction of the stakeholders. final performance % s1 57.10 s2 38.48 s3 54.74 s4 65.81 s5 59.07 d 58.62 table 4. course performance by stakeholder satisfaction (with equal weights). final performance % s1 57.89 s2 41.25 s3 54.88 s4 64.58 s5 58.99 d 58.50 5.3 analysis of the model’s sensitivity as each judgment was made consensually by the stakeholder groups, analysis of the model’s sensitivity was carried out by varying the priority vector in table 1. considering all the stakeholders to have equal importance in judging the indicators, there were no significant changes in the evaluation of the course (despite small variations in the priority vectors). table 4 presents the satisfaction percentage for each stakeholder (performance) for this configuration. table 5 presents the new ranking of the indicators. the new global performance indicator was 58.50%. two other sensitivity analyses were carried out: (1) considering internal stakeholders to be twice as important as the external ones, and (2) considering the external stakeholders to be twice as important as the internal ones. the results of the model are in tables 6 and 7, respectively. indicadores % indicadores % indicadores % indicador % indicador % cd1 50.00% fi2 0.00% p2 0.00% rp1 20.00% cd1 50.00% cd2 100.00% rd2 55.00% p3 37.50% cs1 100.00% cd2 100.00% ab2 75.00% it1 50.00% co2 80.00% cs2 100.00% p2 0.00% ab3 65.00% sa2 50.00% co3 100.00% cu1 75.00% ab2 75.00% cd3 95.00% cti4 100.00% p1 100.00% cu2 75.00% ab3 65.00% fi2 0.00% cti2 50.00% ct5 0.00% cu3 10.00% p3 37.50% en1 35.00% cti3 25.00% co1 80.00% cu4 30.00% cd3 95.00% rd2 55.00% fi1 50.00% if1 50.00% rp3 100.00% fi2 0.00% eq2 90.00% it2 30.00% io1 50.00% c1 75.00% cs1 100.00% eq3 20.00% rd1 0.00% if2 60.00% c2 80.00% cs2 100.00% cd4 100.00% cti1 25.00% 54.74% c3 90.00% rd2 55.00% ct2 70.00% 38.48% c4 90.00% eq2 90.00% sa2 50.00% c5 80.00% eq3 20.00% cp1 25.00% rp2 0.00% cu1 75.00% bb2 50.00% dc1 50.00% cu2 75.00% en2 60.00% dc2 100.00% cu3 10.00% re1 60.00% dc3 50.00% cu4 30.00% sa1 0.00% 65.81% cd4 100.00% ab1 50.00% ct5 0.00% fi1 50.00% c1 75.00% fi3 70.00% c2 80.00% bb1 25.00% c3 90.00% rd1 0.00% c4 90.00% eq1 55.00% c5 80.00% re2 85.00% bb2 50.00% 57.10% ab1 50.00% fi3 70.00% bb1 25.00% dc1 50.00% dc2 100.00% dc3 50.00% rd1 0.00% eq1 55.00% 59.07% s1 s5s2 s3 s4 ijahp article: piratelli, belderrain / supporting the design of a performance measurement system with the analytic network process international journal of the analytic hierarchy process 62 vol. 2, issue 1, 2010 issn 1936-6744 table 5. weights for performance indicators when all the stakeholders have equal weights. indicators weight in evaluation indicators weight in evaluation indicators weight in evaluation indicators weight in evaluation cd1 3.49% rp3 2.08% cd4 1.60% dc2 1.01% cd2 3.49% cu1 2.02% cp1 1.52% dc3 1.01% rp1 2.91% cu2 2.02% co1 1.45% sa1 0.98% co2 2.89% cu3 2.02% it2 1.42% en2 0.90% co3 2.89% cu4 2.02% c1 1.39% re1 0.90% p2 2.80% it1 1.97% c2 1.39% ab1 0.90% ct5 2.80% rd2 1.85% c3 1.39% fi1 0.89% p1 2.76% ct2 1.81% c4 1.39% fi3 0.89% cd3 2.72% en1 1.81% c5 1.39% bb1 0.86% ab2 2.70% eq2 1.80% if1 1.29% rd1 0.82% ab3 2.70% eq3 1.80% io1 1.29% eq1 0.60% p3 2.64% cti4 1.76% sa2 1.20% cti1 0.59% cs1 2.39% cti2 1.76% bb2 1.05% re2 0.45% cs2 2.39% cti3 1.76% dc1 1.01% if2 0.43% fi2 2.23% rp2 1.70% through sensitivity analysis, no significant changes were observed in the course evaluation results. 6. conclusion the design phase, in focusing on the choice of measurements and their metrics, is crucial to the success of the pms and the organization. this article approached the design phase for a pms bases on the performance prism using the analytic network process (anp) for modeling and ranking of the performance indicators. application of the anp as support for the pms design was carried out in the higher education sector to aid management for an undergraduate production engineering course. the model and its results assured the representation of diverse stakeholders “voices” – in a significant and balanced manner – through 58 performance indicators distributed in 4 clusters: satisfaction, processes, capabilities and contribution. the results of the model – ranking of the indicators and measurement of performance – were useful for the course’s collegiate board to be able to reflect on issues that were lacking in information and to discuss action plans make improvements. the group was satisfied with the representative nature and robustness of the model, as sensitivity analysis did not significantly impact the performance evaluation results. it can be stated that the model is legitimate in accurately reflecting strengths and weaknesses of the course. confidence in the model was guaranteed not only by the participation of diverse stakeholders (structuring of indicators and judgments), but also because the collegiate board was willing to construct value table 6. course performance by stakeholder satisfaction (1). table 7. course performance by stakeholder satisfaction (2). final performance % final performance % s1 54.88 s1 60.52 s2 39.55 s2 37.94 s3 56.58 s3 51.89 s4 64.49 s4 66.59 s5 54.67 s5 62.63 d 56.68 d 61.13 ijahp article: piratelli, belderrain / supporting the design of a performance measurement system with the analytic network process international journal of the analytic hierarchy process 63 vol. 2, issue 1, 2010 issn 1936-6744 descriptors and functions for each performance indicator which were useful in operating a subjective evaluation by the group. although this model does not stipulate the mec stakeholder (government), many performance indicators include this agent’s interest. the next step in this work will be to guarantee, through monte carlo simulation, that if the course performance is rated as good by the pms constructed, the government stakeholder (mec) will be satisfied (with a score sufficient for accreditation). references akkermans, h.a. & van oorschot, k.e. (2005). relevance assumed: a case study of balanced scorecard development using system dynamics. journal of the operational research society, 56(8), 931-941. bourne, m., mills, j., wilcox, m., neely, a. & platts, k. (2000). designing, implementing and updating performance measurement systems. international journal of operations & production management, 20(7), 754-771. bourne, m., neely, a., platts, k. & mills, j. (2002). the success and failure of performance measurement initiatives. international journal of operations & production management, 22(11), 1288-1310. bressiani, f., alt, p.r.c. & massote, a.a. (2001). using the balanced scorecard as an instrument of improving performance of an institution of higher education. proceedings of the brazilian congress on engineering education (cobenge), porto alegre, rs, brazil, 29. [in portuguese] fernandes, b.h. (2004). competencies and organizational performance: an empirical study. doctoral dissertation, faculdade de economia e administração, usp, brazil. [in portuguese] figueira, j., salvatore, g. & ehrgott, m. (2005). multiple criteria decision analysis. state of the art surveys. boston: springer science+business media, llc. gomes, l.f.a.m., araya, m.c.g. & carignano, c. (2004). decision making in complex scenarios: introduction to discrete methods of multicriteria decision support. são paulo: pioneira thomson learning. [in portuguese] handy, c. (2002). the performance prism: measuring more is easy, measuring better is hard. in: neely, a., adams, c. & kennerley, m (2002). the performance prism: the scorecard for measuring and managing business success. great britain: prentice hall financial times. kaplan, r. & norton, d. (1992). the balanced scorecard – measures that drive performance. harvard business review, 70(1), 71-79. keeney, r.l. (1992). value-focused thinking: a path to creative decisionmaking, cambridge (usa): harvard university. kennerley, m.p. & neely, a.d. (2000). performance measurement frameworks – a review. proceedings of the 2nd international conference on performance measurement, 291-298. law n. 10,861 of april 14th, 2004. (2004). brasília, federal district, brazil. retrieved jun 26, 2005, from http://www.planalto.gov.br. [in portuguese] ijahp article: piratelli, belderrain / supporting the design of a performance measurement system with the analytic network process international journal of the analytic hierarchy process 64 vol. 2, issue 1, 2010 issn 1936-6744 lee, m.c. (2007). a method of performance evaluation by using the analytic network process and balanced scorecard. proceedings of the international conference on methods and applications of multicriteria decision making, gyeongju, republic of korea. macedo, a.r., trevisan, l.m.v., trevisan, p & macedo, c.s. (2005). higher education in the xxi century and the brazilian university reform. revista ensaio: avaliação e políticas públicas em educação, rio de janeiro, 13(47), 127-148. [in portuguese] neely, a., adams, c & kennerley, m. (2002). the performance prism: the scorecard for measuring and managing business success. great britain: prentice hall financial times. neely, a. & bourne, m. (2000). why measurement initiatives fail. measuring business excellence, 4(4), 3-6. neely, a., gregory, m. & platts, k. (2005). performance measurement system design: developing a literature review and research agenda. international journal of operations & production management, 25(12), 1228-1263. normative ordinance n. 4, of august 5th, 2008. (2008). brasília, federal district, brazil. retrieved august 7, 2008 from http://www.anaceu.org.br/conteudo/legislacao/portarias. [in portuguese] oliveira, c.a. & belderrain, m.c.n. (2008). considerations on finding the ahp priority vectors. in: encuentro nacional de docentes de investigación operativa. posadas, argentina. [in portuguese] paula, d.c. & salomon, v.a.p. (2008). using indicators to analyze the application of methods for decision making with multiple criteria. proceedings of xl simpósio brasileiro de pesquisa operacional “a pesquisa operacional e o uso racional dos recursos hídricos”, joão pessoa, pe. [in portuguese] piratelli, c.l., belderrain, m.c.n & azzolini jr., w.a. (2009). using monte carlo simulation for analyze potentials shortcomings of the evaluation instrument for course accreditation. proceedings of 16nd simpósio de engenharia de produção: “ensino de engenharia de produção”, bauru, sp, 16. [in portuguese] porter, m.e. (1998). competitive advantage to corporate strategy. in: montgomery, c. a. & porter, m. e. (1998). strategy: searching for competitive advantage. rio de janeiro-rj: campus. [in portuguese] saaty, r.w. (2003). decision making in complex. the analytic hierarchy process for decision making and the analytic network process for decision making with dependence and feedback [superdecisions tutorial]. retrieved jun 01, 2008, from http://www.superdecisions.com. saaty, t.l. (1980). the analytic hierarchy process. new york (usa): mcgraw-hill. saaty, t.l. (2001). decision making with dependence and feedback: the analytic network process, pittsburgh, pa: rws publisher. saaty, t.l. (1994). fundamentals of decision making and priority theory – with the analytic hierarchy process. pittsburgh, pa: rws publications. saaty, t.l. (2005). theory and applications of the analytic network process: decision making with benefits, opportunities, costs, and risks, pittsburgh, pa: rws publications. ijahp article: piratelli, belderrain / supporting the design of a performance measurement system with the analytic network process international journal of the analytic hierarchy process 65 vol. 2, issue 1, 2010 issn 1936-6744 saaty, t.l. & peniwati, k. (2007). group decision-making: drawing out and reconciling differences. pittsburgh, pa: rws publications. saaty, t. & vargas, l.g. (2006). decision making with the analytic network process. economic, political, social and technological applications with benefits, opportunities, costs and risks, new york: ny: springer science+business media, llc. silva, a.c., nascimento, l.p., ribeiro, j.r & belderrain, c. n. (2009). anp and ratings model applied to ssp. proceedings of the international symposium on the analytic hierarchy process, pittsburgh, pa, , 10, 1-11. smith, m. (2005). the balanced scorecard. financial management, feb 2005, 27-28. suwignjo, p, bitici, u.s. & carrie, a.s. (2000). quantitative models for performance measurement system. international journal of production economics, 64, 231-241. microsoft word 1preface_mu_p63_vol_1_issue_2_2009.docx                                         ijahp preface: mu/words from the editor‐in‐chief of the ijahp       international journal of the                                                                                            vol. 1 issue 2 2009 issn 1936‐6744                 analytic hierarchy process  words from the editor‐in‐chief    welcome to the second issue of the international journal of the analytic hierarchy process (ijahp). the  mission of the ijahp is to advance the use, education, and development of the analytic  hierarchy/network process (ahp/anp) for decision making. the first article in this issue discusses the  use of ahp‐based questionnaires to evaluate social issues alternatives. this study finds that using ahp to  survey social issues constitutes a more effective method to assess human perceptions on the problems.  this article is followed by an anp application on a practical and important problem: when shall poland  enter the euro one?  this study performs a benefit‐cost‐risk‐opportunity (bocr) of three possible  timeframe scenarios and concludes that poland would be better off entering the euro zone in 2011.  latest news on this topic indicates that this is precisely the strategy that poland is currently pursuing in  relation to the euro zone.  the final article in this issue discusses the use of anp to study combinatorial  auctions. combinatorial auctions are those auctions in which bidders can place bids on combinations of  items, called “packages,” rather than just individual items. anp is particularly suitable to study  combinatorial auctions due to the interdependence of sellers, buyers, bundles, and the related auction  criteria and proposes a model to study this type of auctions.  to close this issue, we have a treat in the  essay section. saaty, vargas, and whitaker provide what may constitute the cliff notes of ahp criticism.  they summarize all the criticism made to date about ahp/anp and address, one by one, each of the  different categories. so, should any of the readers bump into any of these arguments, the response is  more likely to be found in this essay. in a nutshell, this issue presents three research articles from three  different countries, and a review of ahp criticism, enough material to keep ijahp readers busy during  the academic mid‐year recess.    happy holidays,      enrique mu  editor‐in‐chief     rob typewritten text rob typewritten text http://dx.doi.org/10.13033/ijahp.v1i2.56 ijahp article: pegetti, de souza/cognitive maps and ahp for supplier selection in a private higher education institution international journal of the analytic hierarchy process 224 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.303 cognitive maps and ahp for supplier selection in a private higher education institution ana lucia pegetti university of são paulo department of naval architecture and ocean engineering são paulo, sp, brazil e-mail: anapegetti@usp.br jesse d´assunção rebello de souza jr são paulo, sp, brazil e-mail: jsouza@usp.br abstract supplier selection is a key decision in the procurement and purchasing processes. both the choice of criteria and the evaluation of possible alternatives are critical steps in this decision-making process. one of the great challenges of private higher education institutions (phei) in brazil in recent decades has been the attempt to institutionalize administrative practices applied in the business market. the primary goal of this is to optimize their business processes and achieve reduced risks and operational costs, thereby increasing their productivity and the quality of services. these initiatives aim to maintain self-sustaining and competitive institutions in an aggressive market that is constantly expanding. therefore, a critical and professionalized look at their business processes has been one of the solutions for the phei to achieve their organizational goals. in this context, this paper proposes to formalize the decision-making process for the selection of suppliers through their systematization using cognitive maps to structure and identify the criteria that effectively present value during the partner selection of the decision-maker's procurement and purchasing department. the paper also proposes the subsequent prioritization of these criteria for evaluation and selection of potential suppliers by using the ahp multi-criteria method. keywords: supplier selection; cognitive maps; ahp 1. introduction in the mid-1970’s, with the advent of the economic crisis, competition increased between societies in search of technological innovation and a highly qualified workforce. this workforce was, based on technical and scientific knowledge provided by the universities that have become dupes of an international enforcement of the neoliberal economic model, characterized, among other things, by a considerable diminution in the application of government resources in priority social policies (education, wellness, welfare). this opened the public and private universities to commercial exploitation (santos, 2011). ijahp article: pegetti, de souza/cognitive maps and ahp for supplier selection in a private higher education institution international journal of the analytic hierarchy process 225 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.303 it was in the 1990’s, with the phenomenon of globalization and attempts to meet the need for more flexible qualifications rather than the rigid trainings offered by the universities, that there was an increase in the creation of non-university training systems that were more compact and focused on specific assignments. within this backdrop of support for the growing demand for skilled labor from the productive sector, there is the university based on financial results, formed by the expansion of a private network of higher education and university-business partnerships (bergamo, 2008). in brazil, the attempt to turn teaching into a business sector dates back to the period of the military dictatorship, when the law forbade teaching institutions to profit. this reality was altered with the promulgation of the constitution of 1988, which explained the possibility of the existence of for-profit schools, which were effectively regulated with the dissemination of guidelines and bases of 1996. the consequence of this policy was the cutting of funds sent to public education institutions, reducing the number of places in public universities and increasing vacancies in private institutions. these events have changed the type of relationship between students and educational institutions, demonstrating the customer and service provider relationship where scholarships are replaced by loans. in this scenario, students became customers, universities became a market and education became a service to which anyone can achieve. martins (2007) presents a study directed by financial analysts who have compared the current education sector to the health sector in the 1970’s. the similarities noted were an unproductive, fragmented, low-level technology with little professional management, and a giant market that creates a large potential for economic exploitation and thus must be handled with a global business vision rather than a traditional institutional vision in order to optimize results. according to santos (2011), this system favors private universities since they have greater administrative flexibility and can more easily adapt to market conditions. this causes a gradual power shift at the university from the faculty to administrators trained in carrying out agreements with private agents. this corroborates with the findings of finger (1997) who says that there have always been conflicts between academics and administrators since they differ in their vision. the latter have a vision aimed at the exercise of procedures often considered secondary or unnecessary by the first, whose focus is directed to the academic services area of teaching, research and extension. it is in this context of demanding new standards of competitiveness and intense change, that universities are seeking the continuous improvement of their products, services and processes. they seek this improvement through the adoption of administrative practices that aim to improve the implementation of scarce resources (equipment, human and financial) so that they can support appropriate decision-making processes and ensure organizational survival through the adaptation of organizational structure to the reality of uncertainty constants (kobs, 2011). ijahp article: pegetti, de souza/cognitive maps and ahp for supplier selection in a private higher education institution international journal of the analytic hierarchy process 226 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.303 since this change in student university relationship, which began in the 90s, the brazilian educational segment has witnessed a significant increase in the number of higher education institutions (mainly private). this has increased competitiveness in the industry and forced these institutions to seek mechanisms to improve their management practices through the implementation of policies to improve processes in order to achieve excellence at all organizational levels. in the 1990’s, because of the stabilization of the economy and the diversification of the productive sector, there was an increase in demand for qualified professionals and training institutions for specialized labor. this was a great milestone in brazilian higher education because education is regarded as a business. in order to meet this increased demand and due to the expansion of access to higher education promoted by dissemination of the law 9394/96 – guidelines and bases for national education, the brazilian educational sector, especially private higher education (phe), experienced an accelerated expansion characterized by opening new higher education institutions (hei). the demand for higher and graduate level courses has increased which has led to an increase in the number of higher education institutions, the amount of new face-to-face and distance courses, and especially in the number of vacancies. this has led the heis to face inevitable competition, due to a glut of vacancies in various courses. according to data from the census of higher education, the number of higher education institutions has grown in brazil by 152% in the period from 1997 to 2006, being 193% the rate of growth of private higher education institutions (teixeira, 2012). this competitiveness has forced the heis to develop innovative, competitive strategies to attract, win and keep customers to ensure or increase their market share (lee & tai, 2008). this competitive environment requires enterprises to look beyond their organizational boundaries in order to optimize their value chain and coordinate their efforts with the others. in this scenario, looking at the phei as a company that has finite resources, it is necessary to invest in the professionalism of its management so that it can apply its resources optimally, in order to achieve a competitive advantage in the market and ensure the full implementation of its productive force. when allocating resources, managers should be aware that any type of resource retrieved from a cost that does not also provide a return, are decapitalizing the company. the main challenge of institutional managers is to maintain the balance between revenue and expenditure, since the educational activity has become a commodity and input costs to maintain productivity are always on the rise. 2. justification due to the increase in the number of pheis in the past decades, which has unleashed a great opportunity for undergraduate and graduate studies, the institutions have sought the differentials that are required for their survival through the adoption of administrative professionalized practices. ijahp article: pegetti, de souza/cognitive maps and ahp for supplier selection in a private higher education institution international journal of the analytic hierarchy process 227 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.303 this study was conducted in a private institution of higher education in order to assist it in improving its decision-making process concerning the selection of suppliers for cleaning and conservation, which today is the largest financial contract of the university. the vendor selection process is an extremely important activity for most organizations and its realization is one of the first steps towards management of the supply chain. this process is described in the literature as a typical complex problem of decision analysis based on the identification, analysis, assessment and definition of the causal relationships of various quantitative and qualitative criteria raised by specialists. it is characterized as a highly complex issue, especially when it comes to an unstructured problem, where scenarios and decision criteria are not fixed or known a priori, presenting a high risk to the organization. the supplier selection problem (ssp) can be structured by applying a decision-making method that exists in the literature, in particular, a method that can handle situations where there is a need for comparing multiple criteria and later choose the best alternative which is based on the weighting of criteria selected by the decision maker. these criteria, in turn, can be identified in the existing literature, or be elicited directly from decisionmakers through the application of cognitive mapping techniques that take into account the values and subjectivity of a decision-maker in a specific business context. in this context, the prior application of cognitive maps for the identification of the problem´s criteria, valuation and definition of the causal relationships between these criteria is justified based on several studies that prove the effectiveness of this technique for modeling of complex systems. after the definition and evaluation of the criteria for decision-making, it is necessary to apply a multi-criteria decision support method that is capable of dealing with the complexity and subjectivity of the proposed system allowing the repeatability of the method. for this purpose, we will use ahp. 3. objectives the main purpose of this study is to define a systematic procedure based on the cognitive map's concept and the analytic hierarchy process (ahp) multi-criteria method for decision-making, in order to assist the procurement and asset management departments of a phei on ssp. this is due to the lack of studies involving private higher education institutions concerning decision issues triggered by supply selection. therefore, multi-criteria decision-making, using specifically the ahp (analytic hierarchy process) assisted by cognitive maps (cm) (both widely used for solving problems that present multiple objectives or multiple criteria), is an ideal alternative when building an analytic structure to systematize the decision process of this kind of problem (supplier selection). this method has been successfully applied in situations like priority definition, cost-benefit analysis, resource allocation, performance measurement, ijahp article: pegetti, de souza/cognitive maps and ahp for supplier selection in a private higher education institution international journal of the analytic hierarchy process 228 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.303 market research or assessment, requirements determination, strategic decisions, activity planning and sequencing, scenario analysis, negotiation and conflict resolution, decisions and political or social forecasting and analysis of decisions under risk (shimizu, 2010). in order to achieve the proposed objective, a case study on the procurement and asset management departments of an educational institution was performed following these stages:  utilization of a constructivist approach to identify and structure a decision problem using cognitive maps to build a dresser cognitive map in order to identify all the criteria that are subjectively important in the supplier selection process  comparison of the criteria raised in the research with the criteria currently used by the dresser, with the objective being to choose which selection criteria are important to effectively structure the supplier selection decision process  identification of strategic suppliers whose performance will be assessed according to identified criteria  prioritization of the identified suppliers (alternatives) using the ahp (analytic hierarchy process) method  overview and analysis of results this study can be classified as applied research, since a practical application of the data obtained in the literature will be performed, and targeted at the resolution of the specific problem. as for the approach, the work done classifies as qualitative, since it does not require the application of statistical methods and techniques (silva & menezes, 2005). as for its objectives, this is an exploratory research, and its technical procedures are based on bibliographical research and case study, with the purpose to better comprehend the presented problem. 4. literature review after the identification of the study problem, the needs faced by heis to improve their administrative processes and survive market demands and the initiatives implemented in order to achieve this goal were found in the study of bergamo (2008). the next step consisted in evaluating which existing approaches for solving supplier selection problems are available and these were mainly found in the work of wu and barnes (2012). the next step was to seek specific publications on approaches for structuring decision problems and, in the work of eden (2004) and costa (1992) cognitive maps appeared as a method for working on complex problems. using interviews and cognitive mapping to capture individual views of an issue, maps were constructed through the aggregation of individual cognitive maps which were used to facilitate negotiation about value/goal systems, key strategic issues, and option portfolios. this identified the necessary criteria for decision-making and increased knowledge in the system (process) modeling. cognitive maps are largely used as a support tool for structuring application problems, starting from an analysis of cause and effect between the listed criteria and elucidating the relationship between their variables. ijahp article: pegetti, de souza/cognitive maps and ahp for supplier selection in a private higher education institution international journal of the analytic hierarchy process 229 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.303 in the final stage of supplier selection we chose to use the ahp method once the method of structuring was defined and the content published on the work of saaty and vargas (2001) was researched along with several other papers published in isahp. we chose this method because it has been successfully employed in decision-making that involve prioritization, costs and benefits evaluation, resource allocation and strategic decisions, among others. 5. relevance of work nowadays, there are few studies in the field of higher education institutions that discuss decision problems relating to the selection of suppliers. this is due to the fact that only in recent years have the pheis come to adopt successful consolidated administrative tools in order to improve their processes and achieve excellence in their services. in addition, the modeling of the decision-making problems of the nature referred to does not follow a specific theoretical framework. furtado (2005) has conducted an exploratory study in order to determine the degree to which companies conduct formal processes for selecting suppliers. the author figured out, depending on the level of replies received in the survey, that only 15% of companies had formal processes for the structuring of the vendor selection problem, which means that there´s a lack of formalization (structure) of the selection process of suppliers. in the scientific field, the present proposal can reveal more areas of application of cognitive maps associated with ahp in decision-making processes related to vendor selection to compose the supply chain of an institution of higher education. no studies using the two approaches (cognitive map and ahp) for the solution of the vendor selection problem in hies, specifically have been found. 6. theoretical reference 6.1 supplier selection problem (ssp) due to strong competitive pressure, companies have been forced to improve their strategies if they want to overtake their competitors. in light of that, managers have realized the importance of having commercial network suppliers that fulfill their business needs and that are aligned with their market strategy, adequately meeting their demands and sharing ideas, expectations and concerns. christopher (2007) comments that the supply chain is the “organization network involved, through links northbound and southbound, on the different processes and activities that produce value manifested as products and services targeted to the end consumer”. he also noted that its management has proven to be a powerful tool when it comes to obtaining competitive advantage, since it sets out to meet the interests of the company’s productive chain. the adequate management of the network allows optimized production, decrease in costs and ability to meet the expected quality standards. the ijahp article: pegetti, de souza/cognitive maps and ahp for supplier selection in a private higher education institution international journal of the analytic hierarchy process 230 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.303 purchasing department is responsible for price negotiation with suppliers, which will determine, in a way, the company’s competitiveness. therefore, given the importance of decisions related to the supplier selection problem, organizations have been rethinking their business processes, making this topic a highlight in literature about the acquisition of products and services. a common finding in these studies is that supplier selection is a complex problem, which includes several trade-offs between the criteria involved in decision-making. it qualifies as a typical case of multiobjectives and multi-criteria (qualitative and quantitative), with several possible alternatives. clemen and reilly (2001) state that, “the essential problem of decisionmaking with multiple objectives is to decide how to balance the biggest value of an objective with the smallest value of another one.” according to furtado (2005), the main characteristic found in this process is the presence of criteria that aim to identify aspects that help create a vendor profile, and then move forward to the decision of the supply source. the criteria can be quantitative (such as price, productive capacity) which can be easily assessed in order to compare alternatives, or qualitative (such as trust, administrative compatibility) which carry a large subjective factor in the assessment and strongly rely on the personal judgment of the responsible for the process. several studies have been dedicated to the analysis of methods for supplier selection. wu and barnes (2011) perform a systematic review of 140 international articles from 2001 to 2011, classifying them per the criteria and selection approaches used in accordance with a framework based on the work of de boer et al. (2001) and luo et al. (2009), which serves as a reference when defining the stages of supplier selection. the authors define four stages of supplier selection: criteria formulation, qualification, final selection and feedback application. de boer et al. (2001) states that the usage of a reference model for supplier selection increases the effectiveness of the process, since it helps the buyer:  on the resolution of the right problem;  to identify and take into consideration most relevant criteria for the decisionmaking;  to model more precisely the decision problem, considering tangible and intangible factors. ijahp article: pegetti, de souza/cognitive maps and ahp for supplier selection in a private higher education institution international journal of the analytic hierarchy process 231 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.303 figure 1. adapted from wu e barnes (2011) in the first phase of the framework, criteria formulation, the criteria that will be used in the decision making process are determined. according to wu and barnes (2012), the price criterion is still the most widely used, especially in an era of globalized competition. in the authoress’s point of view, to consider only the price criterion is a tactical approach and not a strategic one. this occurs because decision-makers make better decisions based on other important quantitative and qualitative criteria, making the selection of suppliers a complex decision-making process. in the second phase, qualification, a pre-selection is carried out in order to reduce a set of potential suppliers to a smaller set of acceptable suppliers. this phase is seen as a process of pre-selection or classification rather than a prioritization process itself. in the third stage, final selection, we found most of the models developed for the selection of suppliers. in this phase, after the qualification of suppliers, according to some basic criteria, selected companies are contracted to establish partnership with the contractor. the main objective of the fourth phase, application feedback, according to wu and barnes (2011), is providing a mechanism that allows decision makers to evaluate the performance of existing suppliers, in order to continuously improve the selection process ensuring that the best suppliers will always be hired. furtado (2005) states that there are two perspectives to the selection process of suppliers: select new suppliers for the product or service in question, or work with a vendor that already provides service. since the current supplier has recurring problems related to the misconduct of its employees, this work considers the first perspective of selecting new suppliers for the service in question. many few low high criteria formulation qualification final selection feedback application information available to the buyer p o te n ti a l c o m b in a ti o n s f e e d b a c k ijahp article: pegetti, de souza/cognitive maps and ahp for supplier selection in a private higher education institution international journal of the analytic hierarchy process 232 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.303 in this study, the current provider of the cleaning service was excluded from the process because the goal was to rescind the current contract based on recurring problems related to theft of belongings of employees, property damage and improper conduct of employees of the service provider that have been reported and filmed by the institution. thus, the fourth step was not performed as the existing supplier was not being considered. 6.2 supplier selection criteria the hiring company is responsible for defining the attributes or criteria that will be used in the process of supplier selection, notwithstanding the tools that are going to be used to do so. according to furtado (2005), this definition is performed according to an existing derisory context. there are few examples of methods which assist in the identification of the best criteria for supplier selection in the literature (wu and barnes, 2011), but the main criteria among the several existing academic publications are still quality, delivery, price and service. the conclusion of several studies indicates that, even belief that it is important to use multiple criteria in vendor assessment, in reality, the supplier selection which presents the lowest cost is prioritized. which criteria to use and which method can be used to compare suppliers are the two most important questions of an ssp problem. there are several studies that present literature reviews on criteria and methods applied to ssp, but in one of the first articles published on the theme, dickson (1966) identified twenty-three criteria qualitatively classified and sorted by importance as shown in table 1. ijahp article: pegetti, de souza/cognitive maps and ahp for supplier selection in a private higher education institution international journal of the analytic hierarchy process 233 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.303 table 1 criteria for selecting providers classification criteria evaluation 1 quality most importance 2 delivery considerable importance 3 historical performance 4 collateral policies 5 productive capacity 6 price 7 technical competence 8 financial position 9 compliance with procedures media importance 10 communication systems 11 reputation in the industry 12 desire in business 13 management and organization 14 operational controls 15 repair services 16 attitude 17 feeling 18 skill in packs 19 labor relations 20 geographic location 21 number of trades 22 support for training 23 reciprocity in the provision minor importance 6.3 structuring decisory problems for several authors, structuring the problem is the most important as well as the most difficult stage of the entire decisory process. einstein stated “the formulation of a problem is more important than its solution, since the solution requires mathematical or empiric skills”. yu et. al. (2011) argues that the definition of a problem (framing) is subjective, since each individual defines a problem, according to past experiences, knowledge and preferences, which makes the structuring process subjective. the author also states that, “when a problem is inadequately defined, all the effort of its solution can be wasted”. in light of that, other authors, like montibeller (1996), agree on the importance of building a structure which allows the criteria and factors considered important during the decision making process to be represented in an organized manner. this emphasizes the importance of structuring a decisory problem so that the correct ijahp article: pegetti, de souza/cognitive maps and ahp for supplier selection in a private higher education institution international journal of the analytic hierarchy process 234 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.303 problem is solved, through the correct identification of the objectives, exploration of alternatives and better understanding of the situation. the decisory processes are basically divided into two stages: firstly, the structuring of the problem and then, the assessment of potential actions. during the structuring stage, a model is created that schematically represents the decisor’s problem. this is a fundamentally important stage which precedes the decision-making. the problem structuring methods (psms) are approaches that are not based on quantitative methods and cannot be mathematically represented, and their creators are rooted in the traditions of operational research, or po hard (rosenhead & mingers, 2001). in this study, cognitive maps for structuring the proposed problem will be applied, through building the criteria that must be used by the decisions, and afterwards, the ahp will be used to assess the alternatives, in light of the identified criteria. the adopted approach is constructivist, since it allows the modelling of the decision problem, absorbing in a participative way values and interests of the people involved in the process. 6.4 cognitive maps according to eden (2004), cognitive mapping is a task performed in order to describe someone’s thoughts in regards to a specific problem. the graphical representation of these thoughts is presented via a cognitive map. it is grounded on kelly’s personal construct theory (1955), which has three main presuppositional statements (eden, 1988):  man is constantly trying to explain his world.  man establishes the sense in his world through relativism.  man organizes his system of ideas when organizing his world. this last presupposition states that someone’s ideas are organized and interconnected in a hierarchy which is created according to the experience and understanding of each person, forming a private system of constructs. colin eden’s cognitive mapping can be seen as an attempt to isolate and represent someone’s constructs, laying them out in a hierarchical manner. the maps are a network of knots and arrows as links where the arrow direction represents a perceived causality, originated from interviews with the purpose to represent the interviewee’s subjectivity. it is considered a formal modelling technique, with specific rules which makes it recommended for structuring decisory problems. it can be defined as a quadruple cognitive representation, lagged in time (ensslin et al., 2001), according to figure 2. ijahp article: pegetti, de souza/cognitive maps and ahp for supplier selection in a private higher education institution international journal of the analytic hierarchy process 235 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.303 figure 2. model construction of cognitive map ensslin et al. (2001) explains that the representation is quadruple and lagged in time because in t1 there are mental representations of the decisor projected on the t2 instant, in discursive representations (oral or written) transferred to the facilitator through discourse. the facilitator interprets this discourse in t3, generating their mental representations, so that in t4 they transform them into graphical representations through the cognitive map. this process is re-triggered by arrow l2, reducing the distance between the mental representation of the decisor and the construction of the cognitive map. on arrows l1 and l2, the constructivism paradigm and the problem comprehensively through building the cognitive map and the problem description by the decisor through discourse are present. therefore, the actor builds mental representations according to the problem in question. by way of these representations, the decisor develops discursive representations which are transmitted to the facilitator through a discourse from which mental representations are made, and then transformed into graphical representations, i.e., the cognitive map. generally, the cognitive map is built based on interviews made by facilitators and decisions. building a cognitive map depends on the approach by the facilitator, as well as establishing a negotiation process. the facilitator should have a neutral stand, without expressing preferences or judgments, leaving the interviewer comfortable when they present any kind of hesitation or doubt. because this is a process that demands a lot of mental effort and it can become unproductive due to tiredness, each interview should last from 60 to 90 minutes, and should be held either in the interviewee’s environment or in a neutral location. facilitator mental representations (t3) graphical representations (t4) representations (t3) map decisor discursive representations (t2) mental representations (t1) representations (t3) map l2 l1 ijahp article: pegetti, de souza/cognitive maps and ahp for supplier selection in a private higher education institution international journal of the analytic hierarchy process 236 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.303 the interviewee should, as often as possible, position themselves to the left of the facilitator (when right-handed). this way, the interviewee feels engaged in building the map and the facilitator will have them as allies in the process. in order to build a cognitive map, one goes through the following stages, according to ensslin and montibeller (1998):  define a label with the problem: in this stage, the name of the problem to be solved is defined with the decisor;  define the primary assessment elements (paes): these represent the objectives, values, targets regarding the problem, according to the decisions view;  build the concepts (cm), based on paes: the primary element of assessment is action-oriented, supplying the first pole to the concept. the sense of the concept is partially based on the action it suggests. such dynamism can be obtained by putting the verb before the concept (for instance, “to ensure”, “to supply”, “to increment”, etc.) (ackermann, eden and cropper, 1992). therefore, the map must have an action-oriented perspective;  build the concept hierarchy (definition of end connections, means connections and influence connections). the map structure is formed by means and ends concepts, tied by influential connections;  generate the cognitive map: oin this stage, the map is validated with each decisor. after its creation, the cognitive map is analyzed in order to identify the essential aspects to the decisor in the selection process, transitioning to the multi-criteria model. in order to perform a cognitive map analysis, one should (ensslin et al., 2001):  identify the heady concepts (the ones which do not send arrows, meaning the desired objectives) and tail concepts (the ones which send arrows, meaning the number of prepared alternatives in order to achieve the objective of the problem).  identify clusters (which are common concepts that establish a representative connection, separating common concepts and allowing a macro view of the relationships between the branches in a global map).  identify re-triggering knots (which act like vicious cycles).  identify argumentation lines and branches (which consist of the path used by the connected concepts to reach the goal).  identify the descriptors that represent the possible states of the potential actions. according to ensslin et. al (2001), after identifying the branches of a cognitive map, one can migrate to the multi-criteria model. this is because the assessment axes are established over the branches expressing the relevant aspects to the decisor in regards to the analysis of their problem. branch identification allows the facilitator to identify an upcoming stage, the decisor’s fundamental points of view (fpsv). costa (1992) defines fpsv as the values that help a decisor reach their goals, which reflect their fundamental values. these fundamental objectives limit the choice of existing alternatives in a ijahp article: pegetti, de souza/cognitive maps and ahp for supplier selection in a private higher education institution international journal of the analytic hierarchy process 237 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.303 decisory context. this definition is quite similar to the concepts about cognitive maps proposed by eden (2004). however, before stating that something is a fpv, the information (concept) must fulfill the axioms proposed by costa (1992):  isolable: the fpv should be independent from the impact aspects when it comes to the other fpsv of the model;  essential: this value should be considered fundamental. it cannot be a means to achieve a value considered fundamental.  controllable: taking into consideration only those consequences that are influenced exclusively by the set of alternatives. the concepts validated in this stage of the cognitive map analysis are seen as the criteria and sub-criteria in the supplier selection process, and serve as a base to the hierarchical structure of the model. the definition of fpv candidates, according to ensslin et al (2001), is done through the cognitive map analysis, observing the concepts that are neither overly ends (since they would be strategic objectives) nor means (since they would constitute potential actions). following the ends  means direction of a branch, the facilitator initially identifies the first concept that exists between the decisor’s strategic objectives and the potential actions. then, they validate the concept using costa’s axiom, and if it does present these properties the concept becomes an fpv candidate. if it does not present such properties, the concept is rejected and the facilitator moves forward with the branch analysis. it is important to note that, although grouped in clusters or branches, the fpsv are not subordinated amongst themselves, since the tests performed on the path ends (means have the purpose to identify fundamental and independent aspects (lima, 2008). subordination is due to a need to decompose an fpv in order to make it operational. to address the characteristics of potential actions more effectively, decomposing an fpv is often necessary. this decomposition increases the understanding of the performance of the fpv potential actions. by nature, epvs are subordinated to the fpsv, and their identification happens in a similar way to that of the fpsv: the facilitator, through the analysis of the cognitive map on the path ends (means, instead of validating with the decisor the concepts in light of the axioms defined by costa (1992), observe if there are declared means with the purpose to reach the concepts validated as fpsv. keeney (1992) notes the detail that an fpv will only be decomposed if there are, at least, two epvs to the validated fpv. after identifying the fpsv and their possible decomposition into epsv, it is possible to build a point of view hierarchical tree. 6.5 ahp the ahp method was developed in the 1970s by thomas l. saaty, and has been used in complex decisory processes across several knowledge areas including health, business, government, industry, etc. it is a simple and clear method that is easy to explain to ijahp article: pegetti, de souza/cognitive maps and ahp for supplier selection in a private higher education institution international journal of the analytic hierarchy process 238 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.303 decisors and allows an interaction between analyst and decisor. its application depends on a hierarchical structure of the problem that demonstrates the relationship between the problem objective, assessment areas, criteria, sub-criteria and alternatives. therefore, a complex problem can be divided into smaller problems on different levels and uncertainties and other influencing factors can be included in the assessment. the fundamental elements of the ahp method are as follows:  attributes and properties: a set of finite alternatives is compared according to a finite set of criteria.  binary correlation: when comparing two elements in light of a criterion, an element can be preferable or indifferent in regards to another one.  fundamental scale: a priority value is given to each element, based on a numerical scale from 1 to 9.  hierarchy: set of elements sorted by preference and homogeneous in their respective hierarchical levels. the method is grounded in psychological and mathematical principles that are based on the capacity of the human mind to perform comparisons of elements in pairs and matrices. when facing a comparison problem among several elements of a set, the human mind creates a hierarchical process on which the ahp is based. the main objective of the study rests at the highest level, then on the following levels rest the criteria (properties through which the alternatives will be assessed) and on the lowest level rest the alternatives to be selected. in order to apply the ahp, the following stages must be performed:  definition of the decisory problem  decomposition of the decisory problem into a hierarchy of easily understandable problems  identification of meaningful criteria  identification of meaningful alternatives  designation of the relative significance of the attributes by the decisor, through passing parity judgments to assess both the importance of each criterion and the performance of each alternative in light of those criteria  designation of their preference in regards to each attribute and pair of alternatives  registration of the comparisons between attributes and alternatives on matrices of 1/9 and 9 fractions, where each matrix is assessed by its eigenvalue to check the coherence of the judgments, generating a “coherence ratio”, which will be equal to 1 if all judgments were coherent amongst themselves  calculation of the global preference values to each alternative  alternative selection  sensitivity analysis ijahp article: pegetti, de souza/cognitive maps and ahp for supplier selection in a private higher education institution international journal of the analytic hierarchy process 239 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.303 7. case study with the goal of continually improving their processes to meet the needs triggered by the changes in the organizational context and translating strategic objectives into measurable targets, pheis have been adopting modern administrative practices and tools to reduce operational costs, to elevate productivity and increase agility and efficiency, given the dynamic and complex nature of the market. this case study was done in a renowned private higher education institution (phei), located in the southeast region of brazil, specifically in the city of santos, on the southern coast of the state of sao paulo. this organization has as one of their institutional policies the pursuit of better business processes as an important ally for cost reduction and focus on activities that effectively add value to the business and to the client. the business process chosen for this study is part of a macro process related to the acquisition of products and services for the university, a responsibility currently shared by two departments: psd and amd. the research problem of this study will be solved following these stages:  structuring the problem using cognitive maps to build the criteria which are to be used by the decisors. the adopted approach is constructivist, since it allows for modelling of the decision problem and incorporates values and interests of the people involved in the process.  development of the decisor’s cognitive maps to structure the decisory supplier selection problem according to the construction methodology described by ensslin and montibeller (1998). the goal is to contextualize the institution process within the framework proposed by wu and barnes (2011) with the series of criteria discussed on section 6.2 supplier selection criteria as a foundation;  analysis of cognitive maps to identify and validate the fundamental points of view (fpsv) obtained on each decisor’s cognitive map, according to costa’s axioms (1992);  the hierarchical structure based on the listed fpsv and epsv (criteria and subcriteria);  the passing of peer-to-peer judgments using the ahp method, along with the decisors;  sensitivity analysis on the weight of the listed criteria to assess the impacts associated with the value alterations of the decisor’s judgments (entry variables) on the exit variables (alternatives, i.e., priority of the selected suppliers). 7.1 amd –asset management department amd is a sector associated with the administrative pro-vice-chancellor, whose mission is to manage an infrastructure of over nine hundred thousand square feet; as well as services and assets intended to support the end activity of the university by transforming them into a differentiated value perceived by the client. it is composed of thirty-eight employees who perform various functions, including the amd manager, who will be referenced as the decisor of the conducted case study. the activities performed by this sector include maintenance, a variety of services, infrastructure and asset. ijahp article: pegetti, de souza/cognitive maps and ahp for supplier selection in a private higher education institution international journal of the analytic hierarchy process 240 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.303 nowadays, amd is responsible for 77% of institutional costs related to the acquisition of products and services, and electric power, cleaning and conservation, computer maintenance, repairs and maintenance and security services are the most expensive to the institution. at the university, the cleaning and conservation services are currently the biggest consumers of financial resources, representing approximately 3.1% of the total expenses paid by the institution. therefore, it is a service of great importance and visibility to the organization. the main purpose of the cleaning and conservation services hired by the university and made possible by their suppliers is to prepare the environment and keep the order. they are to allow for greater safety and a better performance of the actions that need to be implemented by the institution, therefore improving the client’s perception regarding the quality of the services provided. the main objectives of the cleaning and conservation services are:  to keep the environment clean and pleasant for clients and employees;  to conserve equipment and installations;  to prevent accidents in general;  to preserve the image of the institution;  to ensure safety and confidentiality. for these reasons, the cleaning as well as the asset conservation services are very important to all organizations since, if not performed or if performed without meeting minimal quality standards, they can directly and negatively affect their users, something that can damage the image of the service provider, in this case the institution, in the eyes of the customers. 7.2 supplier selection process in the phei the supplier selection process for the cleaning and conservation services currently performed by the phei is based on the amd manager’s knowledge and experience, consisting of an empirical process, one that is not structured or adapted to the institution. the amd manager has autonomy to make decisions concerning the processes under his scope, which include cleaning and conservation. he must however provide a justification to the administrative pro-vice-chancellor (to which he is a subordinate) about the problems with the current supplier because it is the most expensive service to the phei, having a dedicated ledger account. once the need of a change in suppliers is identified, the same supplier will not take part in the selection process due to its troubled history. after the endorsement is received, the manager, supported by their team, performs a benchmarking with local companies with similar qualifications in order to identify possible supplier candidates. this helps reduce the list of alternative from the beginning. during this stage, the two criteria used in the pre-selection of possible candidates are: ijahp article: pegetti, de souza/cognitive maps and ahp for supplier selection in a private higher education institution international journal of the analytic hierarchy process 241 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.303 references obtained on the contracts and experience with support for other same-size companies. these two criteria are, in fact, pre-requirements for the company in order to be added to the list of alternatives for upcoming suppliers. from this stage, an initial list with the names of companies that provide services is generated, and a proposal request with a statement of work and a few pre-established rules is prepared. this request is then submitted to all these companies and a deadline of thirty days is designated for them to present their proposals. after these proposals are received, the companies are assessed in accordance with the following criteria: service price, number of labor claims, financial capacity, conformity with the category employees’ convention, and number of workers available to provide the service, contract permanence time, references obtained during the benchmarking, area manager’s empathy and service plan. for this last criterion, the manager schedules a meeting with the pre-selected companies and asks for the detailed work plan, based on the specific needs established by the phei. once the plan is received, the companies are also assessed according to administrative organization criteria, technical support capacity and the quality of the materials, described in the work proposal. as soon as this assessment is completed, the supplier is hired. in this particular study, after the completion of the pre-selection stage (based on an initial list of eliminatory pre-requisites), only two of the seven companies contacted were approved to participate in the final selection stage: cm and inservice. 7.3 case study development this case study was initiated after the definition of the finalists (two final alternatives, c.m. and inservice) which were going to be assessed with ahp. the first step was to hold meetings with the decisor in order to build the cognitive map, applying the stages described on section 6.4 cognitive maps. the creation of the decisor’s cognitive map was initiated with the definition of a label to the problem, in order to represent the main objective, still observing the organizational purposes regarding resource optimization. therefore, the label “cleaning services supplier selection” was chosen and validated with the decisor. next, we proceeded to build the primary assessment elements. in order to do that, the we asked the decisor to present which aspects they considered important when it comes to selecting a cleaning services supplier. a list with the main paes can be found below: ijahp article: pegetti, de souza/cognitive maps and ahp for supplier selection in a private higher education institution international journal of the analytic hierarchy process 242 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.303 table 2 primary assessment elements of the amd manager label: selection of cleaning service supplier 1. history of labor processes amd manager 2. officials replacement 3. references benchmarking 4. residence time in contracts 5. qualified supervisor 6. best value service 7. supplier with financial capacity 8. compliance with the collective bargaining agreement 9. work plan after obtaining the paes stated by the decisor, the next stage consisted of transforming these paes into concepts. the goal is to build the concept hierarchy. for each listed concept, through questions like “why is this concept important?” (heading towards the end concepts that are located at the top of the hierarchy), and “what are the means to achieve this concept?” (heading towards the means concepts), new concepts which assisted with the comprehension of the aspects stated by the decisor were generated. from this dynamic, after a few iterations and refinements, we managed to finalize the decisor’s cognitive map. after the creation of the decisor’s cognitive map, the next step was to analyze the cognitive map, starting by the cluster identification. for the amd manager, these clusters were identified: cluster i – resources optimization, cluster ii – service quality, cluster iii – administrative structure and cluster iv – credibility, presented in table 3: table 3 clusters of cognitive map of the amd manager cluster i resource optimization c7, c8, c9, c16, c31, c32, c33, c34 cluster ii quality of service c3, c6, cc19, c20, c21, c28, c29, c30, c35 cluster iii administrative structure c10, c12, c13, c36, c39, c41, c42, c43 cluster iv credibility c4, c5, c25, c26, c27 with the clusters identified in figure 3, the next stage of the methodology consisted in identifying the branches on the maps. this stage is important because clusters are very generic and they make it hard to establish the fpsv, which are the aspects that the decisors wish to measure when choosing a supplier. regarding the map of each decisor, the facilitator of the process (this study’s author) identified the branches which are candidates to fpsv. eight branches were identified to the amd manager: branch 1 – service value, branch 2 – risk reduction, branch 3 – service continuity, branch 4 – team qualification, branch 5 – internal organization, branch 6 – operational capacity, branch 7 – quality of the materials and branch 8 – references. ijahp article: pegetti, de souza/cognitive maps and ahp for supplier selection in a private higher education institution international journal of the analytic hierarchy process 243 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.303 figure 3. cognitive map with identified branches legend: label: ---------- primary assessment elements: --------- concepts: ----------- ijahp article: pegetti, de souza/cognitive maps and ahp for supplier selection in a private higher education institution international journal of the analytic hierarchy process 244 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.303 the branches identified on the map of the amd manager are presented on figure 3. table 4 below shows the branches grouped by cluster: table 4 branches on the cognitive map of the amd manager clusters branches name concepts cluster i resource optimization branch 1 value of the service c32, c31, c7, c33, c34 branch 2 risk reduction c32, c16, c15, c8, c2, c9 cluster ii quality of service branch 3 continuity of service c20, c19, c3, c21, c35 branch 4 training of staff c20, c30, c6, c28, c29 cluster iii administrative structure branch 5 internal organization c36, c12, c41, c13 branch 6 operating capacity c36, c39, c10, c42, c43 branch 7 quality of material c36, c38, c40 cluster iv credibility branch 8 credibility c27, c4, c5, c25, c26, on the facilitator’s analysis, the identified branches represent the following decisor’s concerns:  branch 1: does the supplier have a service value compatible with the market and with the budget of the department, in a way of optimizing the institutional resources?  branch 2: does the supplier comply with the labor laws, so that it minimizes risks of institutional losses due to labor claims?  branch 3: does the supplier have a contingent in order to replace service providers in the event of absences, so that the continuity of the services is maintained?  branch 4: does the supplier have a trained team and a skilled supervisor in order to develop a relationship based on partnership, rather than merely the provision of services?  branch 5: does the supplier have an internal organization in order to deal effectively with hr, law-related and operational issues?  branch 6: does the supplier have a technical team able to produce an adequate work plan to the institution?  branch 7: does the supplier use high-quality materials when performing its operational activities?  branch 8: does the supplier have good references from the market in which it operates? having identified the branches, the next stage of the method is the identification of the fpsv through validating the concepts contained on the map, according to bana and costa’s axioms (1992). regarding each of the branches on the decisor’s map, the facilitator started the process of identifying fvp candidates, analyzing and later checking ijahp article: pegetti, de souza/cognitive maps and ahp for supplier selection in a private higher education institution international journal of the analytic hierarchy process 245 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.303 with the decisor elements which qualify as being isolable, controllable and essential to the decisor’s judgement. the fvp candidates presented by the amd decisor:  fpv1: support capacity, with origins on r3.  fpv2: administrative organization, with origins on r5.  fpv3: institutional risk, with origins on r2.  fpv4: price, with origins on r1.  fpv5: credibility, with origins on r8.  fpv6: quality of the materials, with origins on r7. after decomposing the fpsv into epsv, the next step was building the points of view hierarchical tree from the decisor’s perspective, which finalized the structuring of the multi-criteria model. the family tree of pvs that was built, presented in figure 4, qualifies the structuring of the decisor’s multi-criteria model, in which the criteria are represented by fpsv and the sub-criteria, by epsv. figure 4. multicriteria model based on the cognitive map of the decisor having built the multi-criteria model with the usage of the cognitive map of the amd decisor, the next step was to create a hierarchical model on the software tool, establishing the hierarchical relationship between the elements. the hierarchy displayed in the figure represents a situation in which the decisor wishes to select a new supplier for the cleaning services. the criteria that have an impact on the general objective, displayed on the second level of hierarchy, are: support capacity, administrative organization, institutional risk, price, credibility and quality of the materials. the criteria support capacity, administrative organization, institutional risk and credibility are subdivided into third level subcriteria: teams technical qualification, other contracts, backup employees, operational organization, legal advice, hr policies, labor claims, financial capacity, contract permanence time and references from hiring companies. on the last level, the alternatives are: company c.m. and company inservice. ijahp article: pegetti, de souza/cognitive maps and ahp for supplier selection in a private higher education institution international journal of the analytic hierarchy process 246 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.303 after creating the hierarchical model, it was necessary to fulfill the dominant matrices, where the alternatives in relation to each criterion and the criteria of a specific level in relation to the criterion on the immediately superior level were compared. saaty’s fundamental scale was used, establishing the preference of the decisor. in this manner, the verbal judgments were transformed into a numerical scale. the decisor’s judgments are presented in table 5: table 5 comparison matrix of the criteria of the problem of supplier selection service capacity administrative organization institutional risk price credibility material quality relative priority service capacity 1 1/4 1/4 1/2 1/2 1/3 0,06 administrative organization 4 1 1/4 1/3 1/2 1/3 0,09 institutional risk 4 4 1 3 3 1/2 0,27 price 2 3 1/3 1 1/2 1/3 0,12 credibility 2 2 1/3 2 1 1/3 0,13 material quality 3 3 2 3 3 1 0,32 therefore, the criteria prioritization order is: quality of the materials, institutional risk, credibility, price, administrative organization and support capacity. the same calculations for sub criteria support capacity, administrative organization, institutional risk and credibility were performed, which resulted on the following sub criteria prioritization order. afterwards, the rc of the decisor’s judgments was calculated. using the equation aw = λmax w in order to obtain the eigenvector, we have: 1 1/4 1/4 1/2 1/2 1/3 0,061 0,38 4 1 1/4 1/3 1/2 1/3 0,099 0,62 4 4 1 3 3 1/2 x 0,271 = 1,82 2 3 1/3 1 1/2 1/3 0,118 0,80 2 2 1/3 2 1 1/3 0,132 0,88 3 3 2 3 3 1 0,317 2,09 figure 5. matrix multiplication by its normalized value applying formula λmax = average of vector aw / w, we obtain the eigenvector: and applying equation (λmax – n )/ (n-1), the ic will be ic = (6,54 – 6) /(6-1) = 0,108. ijahp article: pegetti, de souza/cognitive maps and ahp for supplier selection in a private higher education institution international journal of the analytic hierarchy process 247 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.303 given that n = 6 and the random index (ir) associated with the dominant matrix is 1,24, we calculate rc = ic/ir = 0,108 / 1,24 = 0,0873 < 0,10. in this manner, the inconsistency emitted by the decisor towards the criteria is acceptable. performing the same calculations on sub criteria of criteria support capacity, administrative organization, institutional risk and credibility, we achieved, respectively, the following values: rc: 0,0512 < 0,10, rc: 0,0512 < 0,10, rc: 0,0 < 0,10 e rc: 0,0 < 0,10. therefore, the inconsistency emitted by the decisor towards the sub criteria was also acceptable. next, the alternative priority vectors were calculated, according to each criterion, by way of normalization and the priority vector calculation, which resulted on the values shown in table 6. ijahp article: pegetti, de souza/cognitive maps and ahp for supplier selection in a private higher education institution international journal of the analytic hierarchy process 248 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.303 figure 6. family tree of the fundamental point of view of the amd manager ijahp article: pegetti, de souza/cognitive maps and ahp for supplier selection in a private higher education institution international journal of the analytic hierarchy process 249 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.303 pri ce q u al i ty of th e mate ri al ran k i mg 0,11955 0,31899 labor c l ai ms fi n an ci al c apaci ty te ch n i cal capaci ty o th e r con tracts back u p empl oye e s o pe raci on al o rgan i z ati on le gal advi ce hr pol i ci e s c on tract pe rman e n t ti me re fe re n ce s c .m. 0,33330 0,25000 0,25000 0,20000 0,25000 0,25000 0,16667 0,16667 0,14286 0,16667 0,25000 0,16667 0,22095 in s e rvi ce 0,66667 0,75000 0,75000 0,80000 0,75000 0,75000 0,83000 0,83000 0,85714 0,83333 0,75000 0,83000 0,77883 al te rn ati ve s c re di bi l i ty 0,27569 0,05870 0,09328 0,13379 c ri te ri a in sti tu ci on al ri sk s u pport c apaci ty admi n i strati ve o rgan i z ati on s u bcri te ri a 0,66667 0,33333 0,37640 0,14937 0,47423 0,24931 0,15706 0,59363 0,80000 0,20000 figure 7. weights obtained for the supplier selection problem table 6 final calculation alternative calculation final ranking c.m. (0,11955*0,33330)+(0,27569*0,66667*0,25)+(0,27569*0,33333*0,25)+(0,0587*0,3764*0,2)+(0,0587*0,14 937*0,25)+(0,0587*0,47423*0,25)+(0,09328*0,24931*0,16667)+(0,09328*0,15706*0,166667)+(0,09328* 0,59363*0,14286)+(0,31899*0,16667)+(0,13379*0,8*0,25)+(0,13379*0,2*0,16667) 0,22095 inservice (0,11955*0,66667)+(0,27569*0,33333*0,75)+(0,27569*0,33333*0,75)+(0,0587*0,3764*0,8)+(0,0587*0,14 937*0,75)+(0,0587*0,47423*0,75)+(0,09328*0,24931*0,83)+(0,09328*0,15706*0,83)+(0,09328*0,59363* 0,85714)+(0,31899*0,83333)+(0,13379*0,8*0,75)+(0,13379*0,2*0,83) 0,77883 ijahp article: pegetti, de souza/cognitive maps and ahp for supplier selection in a private higher education institution international journal of the analytic hierarchy process 250 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.303 with the purpose of demonstrating the robustness of the method, we moved forward to the sensitivity analysis of the model generated for each criterion judged by the decisor. the assessment process on the price criterion was initiated, with a variation of 10% (for more and for less) on the criterion weight value, without having any alteration in the order of the alternatives. figure 8. sensitivity analysis of the model this procedure was repeated on each model criterion and no alteration was identified, which demonstrates that the proposed method satisfactorily fulfilled the criteria modelling objective, and that company inservice fulfilled in a satisfactorily superior way the criterion designated by the decisor. therefore, with the conclusion of the sensitivity analysis, the case study according to the objective defined for this paper was concluded. 8. limitations of the model the major limitation of this study is that it is a case study within the education industry; therefore, the results may not be applied to the supplier selection process in other industries. another limitation was the sample size of suppliers: though the project has successfully implemented the ahp approach in the supplier selection process and proposed the final best choice, there still might be better choices outside the candidate pool. ijahp article: pegetti, de souza/cognitive maps and ahp for supplier selection in a private higher education institution international journal of the analytic hierarchy process 251 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.303 due to the institutional need to immediately hire another company to replace the current provider, only two companies were able to present their proposals. this case study does not have the objective to determine a method for selecting suppliers, nor does it intend to define the criteria which should be considered in problems of supplier prioritization. it limits itself to presenting a systematic procedure that allows structuring the decision criteria in a recursive and replicable way. through the analysis of the proposals and without applying the model, we noted a dominance of one company over the other. one of the characteristics of the created model is that it does not consider evaluating the current vendor, therefore excluding them from the supplier selection process. the exclusion of the current vendor from this selection process was intentional. 9. conclusions after the decisor’s assessment, results were transferred to the superdecisions tool, with the purpose of performing and displaying the calculations regarding the method. the identified criteria were prioritized according to the following importance order, defined by the decisor: quality of the materials, institutional risk, credibility, price, administrative organization and support capacity. table 7 weights obtained for the supplier selection problem total quality of the material institucional risk credibility price administrative organization support capacity 100% 31,9¨% 27,6% 13,4% 12% 9,3% 5,9% the fact that the quality of the materials criterion came on top of the list of relevant criteria was a revelation to the amd manager, since, to him, the most relevant criterion were organization and support capacity, which ended up at the bottom of the list. the evaluation of alternatives by the ahp method was performed twice by the decision maker, and in these assessments the criterion quality of materials had a small advantage over institutional risk. from a business point of view, the quality of materials criteria is more important than the institutional risk criteria since the latter is a condition for the pre-selection of suppliers. before the final selection by the ahp method, an initial list of potential suppliers was generated and prior research on the financial situation and ongoing lawsuits for each participating supplier of the process was carried out. those who had many judicial events or low financial performance were excluded from the final list, so the institutional risk was reduced from the beginning of the process. ijahp article: pegetti, de souza/cognitive maps and ahp for supplier selection in a private higher education institution international journal of the analytic hierarchy process 252 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.303 although in most of the supplier selection problems the price criterion is a key point for the decisor´s final decision, for phei it was not because bid prices submitted by companies were very similar. the true differentiation was determined by the quality of the material used. this criterion strongly impacts the perceived final value of the service by phei´s manager. it´s important to review table 2 – criteria for selecting providers which refers to a study by dickson (1966) who has identified twenty-three criteria qualitatively classified and sorted by importance, and according to the table, the quality criterion is considered the most relevant in the ssp. therefore, the ahp method applied on this model validated itself and confirmed the robustness of the multi-criteria model that was built, since after performing the sensitivity analysis to assess the coherence of the decisor’s judgement, the order of the alternatives was unchanged. this qualifies the selected alternative as an absolute dominant. regarding the selected alternative, the decisor, before structuring the problem and considering the criteria, had already manifested their preference for the inservice company based on the proposals presented by the two companies. after the completion of this case study, the analyzed phei showed interest in applying the method on more strategic business decisions, i.e., to develop other models for the most important and complex decisions. the decision-maker that took part on this study also showed interest in reviewing the model considering the stage of evaluation of the service providers. regarding the scientific field, the importance of this work is revealing yet another area for application of the cognitive maps associated with the multi-criteria decision-aiding approach (ahp). this can be applied specifically to decision-making processes linked to supplier selection, with the goal of creating the supply chain for a higher education institution, since few studies of this nature have been found, none that use the aforementioned approaches (cognitive maps and ahp) to problem-solving supplier selection on heis, specifically. in short, the ahp method offered an effective and efficient way to select suppliers, incorporating a consistence check to reduce human discrepancy, providing a method that combined objective factors and subjective expert judgments and taking into account both qualitative and qualitative information. in the future, the problem of selecting suppliers covering all stages of selection described in the wu and barnes (2011) framework, mainly to phase 4 – feedback application, for the monitoring of suppliers selected, which relates to the ongoing assessment of the suppliers selected by means of a performance analysis could be developed. this step is important and has been raised by the decision maker of this work since, in many cases, one can use the performance evaluation of a service provider to determine whether or not to include it in a new supplier selection process. in this context, there is a dependency between the possible criteria used for decision-making which restricts the use of ahp method, this being one of its limitations as it has the premise of independence among the criteria. in this case, the anp method is indicated which is a generalization of the ahp, and considers the decision problem as dependent on a network of criteria. ijahp article: pegetti, de souza/cognitive maps and ahp for supplier selection in a private higher education institution international journal of the analytic hierarchy process 253 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.303 references ackermann, f., eden, c., & cropper, s. (1992). getting started with cognitive mapping. banxia software. bergamo, f., farah, o. e., & giuliani, a. c. (2008). a lealdade e a educação superior: ferramenta estratégica para a retenção de clientes. revista ibero-americana de estratégia, 6(1), 55-62. costa, c. a. b. (1992). structuration, construction et exploitation d'un modèle multicritère d'aide à la décision. thèse de doctorat, universidade tecnica de lisboa. clemens, r. t., & reilly, t. (2001). making hard decisions with decision tools®. christopher, m. (2007). logística e gerenciamento da cadeia de suprimentos: criando redes que agregam valor. thomson learning. de boer, l., labro, e., & morlacchi, p. (2001). a review of methods supporting supplier selection. european journal of purchasing & supply management, 7(2), 75-89. doi:10.1016/s0969-7012(00)00028-9 dickson, g. w. (1966). an analysis of vendor selection systems and decisions. journal of purchasing, 2(1), 5-17. eden, c. (1988). cognitive mapping. european journal of operational research, 36(1), 1-13. doi:10.1016/0377-2217(88)90002-1 eden, c. (2004). analyzing cognitive maps to help structure issues or problems. european journal of operational research, 159(3), 673-686. doi:10.1016/s0377-2217(03)00431-4 ensslin, l., neto, g. m., & noronha, s. m. (2001). apoio à decisão: metodologias para estruturação de problemas e avaliação multicritério de alternativas. insular. furtado, g. a. p. (2005). critérios de seleção de fornecedores para relacionamentos de parceria: um estudo em empresas de grande porte. doctoral dissertation, universidade de são paulo. keeney, r. l. (1992). value-focused thinking: a path to creative decision making. cambridge, ma:harvard university. doi: 10.1002/mcda.4020020108 lima, a. s. proposta de método para modelagem de critérios de priorização de projetos de pesquisa e desenvolvimento aeroespaciais. 189f. 2008. doctoral dissertation, dissertação (mestrado-engenharia aeronáutica e mecânica–área de produção)–instituto tecnológico de aeronáutica, são josé dos campos, sp. http://dx.doi.org/10.1016/s0969-7012(00)00028-9 http://dx.doi.org/10.1016/0377-2217(88)90002-1 http://dx.doi.org/10.1016/s0377-2217(03)00431-4 ijahp article: pegetti, de souza/cognitive maps and ahp for supplier selection in a private higher education institution international journal of the analytic hierarchy process 254 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.303 luo, x., wu, c., rosenberg, d., & barnes, d. (2009). supplier selection in agile supply chains: an information-processing model and an illustration. journal of purchasing and supply management, 15(4), 249-262. doi:10.1016/j.pursup.2009.05.004 martins, g. m. (2007). um recurso estratégico para a gestão das organizações universitárias. estudos: revista da associação brasileira de mantenedoras de ensino superior. ano, 25. montibeller neto, g. (1996). mapas cognitivos: uma ferramenta de apoio à estruturação de problemas. dissertação de mestrado da universidade federal de santa catarina, santa catarina-brasil. rosenhead, j., & mingers, j. (2001). rational analysis for a problematic world revisited. john wiley and sons. doi: 10.1023/b:spaa.0000019133.40573.e4 saaty, t. l., & vargas, l. g. (2001). models, methods, concepts & applications of the analytic hierarchy process. boston: kluwer academic publishers. doi: 10.1007/978-14615-1665-1 shimizu, t. (2010). decisão nas organizações. são paulo: atlas. silva, e. l., & menezes, e. m. (2001). metodologia da pesquisa e elaboração de dissertação. florianópolis, ufsc. teixeira, s. (2012). censo da educação superior 2010. wu, c., & barnes, d. (2011). a literature review of decision-making models and approaches for partner selection in agile supply chains. journal of purchasing and supply management, 17(4), 256-274. doi:10.1016/j.pursup.2011.09.002 wu, c., & barnes, d. (2012). a dynamic feedback model for partner selection in agile supply chains. international journal of operations & production management, 32(1), 79-103. doi:http://dx.doi.org/10.1108/01443571211195745 yu, a.s.o. (2011). tomada de decisão nas organizações. são paulo: saraiva. http://dx.doi.org/10.1016/j.pursup.2009.05.004 http://dx.doi.org/10.1016/j.pursup.2011.09.002 ijahp article: pegetti, de souza/cognitive maps and ahp for supplier selection in a private higher education institution international journal of the analytic hierarchy process 255 vol. 8 issue 2 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i2.303 appendices appendix a – alternatives ranking alternatives ranking appendix c – alternative comparison appendix d – criteria and sub criteria weights appendix b – criteria weights ijahp article: mantogiannis, katsigiannis/assessing real estate investment alternatives: a multicriteria and multi-stakeholder decision aid tool international journal of the analytic hierarchy process 136 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.702 assessing real estate investment alternatives: a multi-criteria and multi-stakeholder decision aid tool vasileios mantogiannis lancaster university management school v.mantogiannis@gmail.com fotios katsigiannis lancaster university management school f.katsigiannis@lancaster.ac.uk abstract investment decisions in private real-estate demand the consideration of several qualitative and quantitative criteria, as well as the different or even conflicting interests of the participating stakeholders. meanwhile, certain indicators are subject to severe uncertainty, which will eventually alter the expected outcome of the investment decision. even though multi-criteria decision making (mcdm) techniques have been extensively used in real-estate investment appraisals, there is limited evidence from the private rented sector, which constitutes a large part of the existing real estate assets. the existing approaches are not designed to capture the inherent variability of the decision environment, and they do not always achieve a consensus among the participating actors. in this work, through a rigorous literature review, we were able to identify a comprehensive list of assessment criteria, which were further validated through an iterative delphi-based consensus-making process. the selected criteria were then used to construct an analytical hierarchy process (ahp) model evaluating four real world, real estate investment alternatives from the uk private rented market. the volatility of the financial performance indicators was grasped through several monte carlo simulation runs. we tested the described solution approach with preference data obtained by seven senior real estate decision-makers. our computational results suggest that financial performance is the main group of selection criteria. however, the sensitivity of the outcome indicates that location and property characteristics may greatly affect real estate investment decisions. keywords: multiple criteria; decision support; mcdm; ahp; delphi; real estate; investment appraisal 1. introduction real estate has always been a popular field for investment. the private rented sector (prs) used to be a minority investment market, mainly due to low demand and the perception of low returns compared to other investment assets. the prs deals with the construction and modification or purchase of properties, with the goal of renting them privately. it plays a vital socio-economic role, specifically in the uk market, by serving mailto:v.mantogiannis@gmail.com mailto:f.katsigiannis@lancaster.ac.uk ijahp article: mantogiannis, katsigiannis/assessing real estate investment alternatives: a multicriteria and multi-stakeholder decision aid tool international journal of the analytic hierarchy process 137 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.702 major demographic segments (e.g. professionals, students) who ask for housing variety and flexibility, either for the short or long term (treasury, 2010). in recent years, the prs sector has faced rapid growth, and accounts for 20.3% of all uk households. interestingly, in london the corresponding proportion of households that are prs is 30% (office for national statistics, 2017). since 2002, the prs has doubled in size and knight frank (2017) estimates that the figure will rise to 24% by 2021. in addition, the continuous rise of housing prices and rents has led individuals and large institutional investors, as well as companies outside the field of real estate, to turn their attention to investment in the prs. the sector has become popular since it offers a lowrisk investment vehicle with long-term growing returns, as well as a continuous increase of the asset value due to the steady rise of house prices. multiple criteria decision analysis (mcda), or multiple criteria decision-making (mcdm), has been widely used as a support tool for complex investment decisions where quantitative and qualitative assessment criteria exist (zopounidis & doumpos, 2002). quantitative factors, such as financial indicators (e.g. npv), can be easily understood during the decision-making process, i.e., a project with higher returns is preferred over another with less. however, decision-making that involves qualitative criteria faces significant complications. for instance, the choice among alternative locations (e.g. a and b) is a matter that has no evident solution since it requires the consideration of the stakeholders’ preferences and experience. mcdm may provide the methods and tools that are needed to quantify comparisons with qualitative criteria. furthermore, in the presence of multiple criteria, there will be trade-offs among the criteria by the decision-maker, since the final selection will take into account a list of different or even conflicting objectives. for instance, the decision-maker might prefer larger investments in prestigious locations over low-valued assets in infamous or remote locations. similarly, other interested parties may express alternative preferences regarding the value and the location of the asset. consequently, mcdm methods are suitable for investment appraisal in the prs since they compare the existing alternatives for selection according to the identified list of criteria and their importance, and finally provide a rating or ranking of the alternatives. researchers have mostly focused on the risks associated with real estate investments and suggest that an understanding and quantification of the risks can lead to better decisionmaking (atherton et al., 2008; chen and khumpaisal, 2009; d’alpaos and canesi, 2014). despite the value of risk assessment, other factors can drive a decision in real estate, and are usually omitted from the process, such as the financial performance of the investment or other qualitative criteria that the asset might have. another stream of research focuses on real estate investments from the perspective of portfolio selection and management (kallberg et al., 1996; andrew and glenn, 2003). the research focuses on portfolio diversification, and how each investment asset can create overall value for a particular portfolio. there have been few attempts to address the investment appraisal process itself, yet, to the best of our knowledge, instead have focused on the individual characteristics of the process such as the fairness of the market value and transaction price, the profit maximization or the technical characteristics of the project (kaklauskas et al., 2007; maliene, 2011). however, only a small part of the existing literature addresses investment appraisal of real estate projects from a micro-perspective, while there is no mention of the private rented sector. whether the investor is an experienced individual in ijahp article: mantogiannis, katsigiannis/assessing real estate investment alternatives: a multicriteria and multi-stakeholder decision aid tool international journal of the analytic hierarchy process 138 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.702 real estate, a large institution, or someone from outside of the sector who is trying to diversify their portfolio and take advantage of the situation, the fact is that expertise and research in this sector are limited, and the investigation of these investment decisions will provide useful insights for all of the stakeholders. investment decisions in real-estate assets stem from a balanced consideration of factors generated by the behavior or preferences of multiple stakeholders. on the one hand, investors usually look to optimize profit in the short and long term, while financial institutions alter their financial support models according to the growth and risk trends of each market and the quality of the asset (investor aspect). on the other hand, sellers look for the right moment to maximize the return on their old investment and tenants’ behavior regarding price, quality or even flat-sharing can set the price of a property in a specific area (market aspect). the above factors are the reason why the present study used a multi-stakeholder approach to take into account criteria from different points of view. the behavior of the investor’s side is captured by the opinions of actual investors in the area and the market aspect by the opinions of experienced consultants. the objective of the present study is to model investment decisions in prs properties with an mcdm method, and to support the selection from among the existing alternatives in the market. in order to achieve this goal, the work follows a sequential workflow, beginning with a literature review of the existing knowledge and frameworks on real estate investment appraisal, as well as the selection of the appropriate mcdm method to structure our problem. subsequently, a list of criteria that investors and consultants take into consideration in the investment process is provided and a consensus reaching method is used to determine the final list of criteria and quantify their importance. finally, with the use of the selected mcdm method, a preference model for the selection of the best prs alternative for investment is created. taking all of the previous discussion into consideration, we can conclude that this research should address particular aspects of the investment appraisal of private rented properties. in summary, the current study contributes to the literature by:  providing a comprehensive list of criteria that decision-makers take into account before investing in a prs property or project. the composition of this list was based on the existing literature and was further validated and extended by investors and consultants of the sector.  quantifying the significance of each criterion with the use of opinions from experts from the sector, through a consensus-reaching technique.  analyzing the problem using a hierarchical structure and constructing a decisionmaking tool to support investors’ decisions in the private rented sector. 2. literature review 2.1 multiple criteria decision-making multiple criteria decision-making has grown as an important part of modern decision science and operations research, and supported by computational and mathematical tools allows the subjective evaluation of performance criteria and alternatives by decisionmakers (zavadskas et al., 2014). vast technological advancements and rapid economic growth have changed the nature and complexity of modern society’s problems, requiring ijahp article: mantogiannis, katsigiannis/assessing real estate investment alternatives: a multicriteria and multi-stakeholder decision aid tool international journal of the analytic hierarchy process 139 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.702 decision-makers to deal with problems that have multiple criteria and multiple decision alternatives. the popularity of and the need for mcdm applications over the last 15-20 years has been evidenced by the increased number of studies in this field. from 1999 to 2000, there were 628 research studies about mcdm (toloie-eshlaghy, 2011), while from 2000 to 2014, there have been 393 papers on the application of such methods in various decision problems (mardani et al., 2015). the areas of application of the methods vary, with the most popular areas being transportation and logistics, business and financial management, managerial and strategic planning, project management and evaluation and energy, environment and sustainability. other significant fields of application are supply chain management, manufacturing systems, information technology management and military operations and strategy. one of the most eminent mcdm methods, in terms of application in decision models and publications, is the analytic hierarchy process (ahp), which was developed by saaty (1980). almost one-third of mcdm applications have used ahp, and this is because it is simple to use and able to break down the components of the problem in a hierarchical structure. popular applications of mcdm methods have also been developed with the use of the elimination and choice expressing reality method (electre), the analytic network process (anp), the technique for order of preference by similarity to ideal solution (topsis), the preference ranking organization method for enrichment evaluations (promethee), other aggregation decision-making methods and with the hybrid use of the existing methods (mardani et al., 2015). the modeling of financial problems is more complex and often follows a different logic that considers the complexity and ill-structured nature of the problems, the existence of multiple criteria, which are sometimes conflicting, and the subjectivity of the decisionmakers in the evaluation process (roy, 1988; zopounidis & doumpos, 2002). not only the complex and multidimensional nature of financial decisions, but also the increased importance of making effective financial decisions, makes mcdm a well-suited methodology to address these kinds of problems. consequently, mcdm methods have been used systematically over the last decades as a tool to aid in financial decision making (zopounidis & doumpos, 2002; spronk et al., 2005). especially in the last ten years, the application of mcdm in financial decisions has increased dramatically with the number of publications since 2002 tripling those from before 2002 (zopounidis et al., 2015). portfolio optimization, credit risk and bankruptcy, corporate performance, asset evaluation and investment appraisal are the principal areas in which mcdm methods have been used in financial decisions, with the ahp method used in 124 papers out of 273 studies (zopounidis et al., 2015). decisions in the real estate sector and real estate investments usually require a mixed knowledge of asset evaluation, investment appraisal, and technical diligence. the required diversity of the factors and the existence of multiple conflicting criteria (e.g. selecting low cost over high quality) increase the complexity of the decision-making process, and therefore the necessity of multiple criteria analysis. mcdm methods have been applied in the real estate sector, either prior to the decision point to assist the evaluation and/or risk assessment of a real estate investment or after the investment decision to evaluate the performance of the asset and the investment efficiency. ijahp article: mantogiannis, katsigiannis/assessing real estate investment alternatives: a multicriteria and multi-stakeholder decision aid tool international journal of the analytic hierarchy process 140 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.702 the following section presents all of the relevant literature in real estate decision making and investment appraisal, especially the proposed decision frameworks with mcdm methods. 2.2 decision-making in real estate the existing mcdm applications on real estate can be classified into two main research streams as follows: studies that propose frameworks prior to the decision point in order to assist the process (pre-investment/decision studies) and studies that assess the outcome of the decision to provide useful insights and proposals for future decisions (postinvestment/decision studies). decision models in real estate have been developed to assist either the investment appraisal of development projects or the valuation of investment on existing properties. zavadskas et al. (2004) analyzed the market for commercial property development in eastern europe by proposing a model to choose among alternative projects with the principal objective being the maximization of profit. another model in commercial development was proposed by chen and khumpaisal (2009), who addressed risk assessment in commercial real estate development with the use of anp, a more complex version of the ahp method. they developed a multiple criteria model to assess social, economic, environmental and technological risks. rocha et al. (2007) attempted to address the uncertainties that real estate investments present in emerging economies, which are usually related to demand, price, and costs. they proposed a model for the housing development market that determines the optimum investment strategy to cover demand, the right timing for construction and the maximum expected income. jiang et al. (2007) also address the risks in real estate development with a decision model based on the ahp technique and simulation of the factors that influence the calculation of npv. the study hierarchically breaks down the uncertain variables that can affect the net inflow and outflow of the project, which are mainly related to market competition and the construction process. monte carlo simulations are also conducted to provide the range of npv values in each year of the project, which allow the investor to decide whether or not to invest in the project. atherton et al. (2008) investigated the risk factors that affect the profit calculation in real estate development appraisal. the proposed model approaches profit calculation with residual valuation and traditional cash flow analysis, followed by simulation or forecasting of the uncertain variables and a sensitivity analysis of the npv and irr values. a fund divestiture situation in the indonesian real estate market was the reason to develop another decision model in real estate development. ronyastra et al. (2015) appraised four real estate projects under a given budget constraint and constructed a ranking of the alternatives based on promethee, an mcdm method suitable for building outranking relations. for investment decisions on existing properties, uysal and tosun (2004) attempted to break down the valuation of a household residency in the turkish market, and suggested mcdm as the best tool for this analysis. kaklauskas et al. (2007) emphasized that and proposed a methodology to define the utility and market value of a real estate property. in their work, they analyzed the theoretical framework of the proposed model using the method of complex criteria assessment (copras), an mcdm method that evaluates the alternatives in terms of the degree of utility and significance. lutzkendorf and lorenz (2007) identified a list of key sustainability performance indicators and highlighted the need for simultaneous consideration of ijahp article: mantogiannis, katsigiannis/assessing real estate investment alternatives: a multicriteria and multi-stakeholder decision aid tool international journal of the analytic hierarchy process 141 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.702 economic, environmental and social criteria before planning to invest in a property. a combination of the above was attempted by lopez et al. (2010), who designed a software tool to aid decisions about selling or buying real estate. tiesmeier (2016) worked the valuation models of existing properties by introducing a model for real estate decision support, focusing on luxury residencies in spain. the author highlights the need for structuring the problem on an mcdm method using the ahp technique. in most of the pre-decision models, the authors addressed the importance of risk assessment during the investment appraisal stage, believing that a better understanding of the risks will lead to a better decision. even though this is a major issue in investment appraisal, only a few researchers succeeded in imprinting the complexity of real estate decisions and incorporating all of the relevant criteria (financial, technical, etc.) to construct a decision model. nevertheless, most of them agreed that mcdm frameworks were important and suitable in real estate decisions. next, we discuss the post-decision models, which analyze investment decisions in real estate after the time that these decisions have been made. even though these models cannot help the decision-maker make a particular decision, they offer valuable insights on investment efficiency and help them make better decisions in the future. kettani et al. (1998), in the canadian real estate market, first investigated the outcome of an investment and its effect on the market. they proposed a model that estimates the behavior of the market, with respect to the objectives that different stakeholders (property buyers/sellers, institutional investors, real estate brokers) might consider. wang (2005) presented another model, which analyzed real estate investment decisions and measured investment performance. the framework evaluates government real estate investments in china, according to a range of technical and non-technical factors, and is built upon the knowledge of previous decisions and aims to discover possible projects for investment. it intends to reform the investment strategy in real estate and provide an online-based decision-making tool for the chinese government. the selection of real estate investment projects and their influence in the regional economy were analyzed by ginevicius and zubrecovas (2009). by providing an extended list of economic efficiency and the projects’ efficiency criteria, they included every stakeholder in the real estate environment and attempted to analyze the impact of a particular selection on each one of them. the study gives valuable stimuli on the criteria that should be included in the investment appraisal of real estate projects to achieve better investment efficiency. risk and uncertainty in real estate decisions has also been addressed with regards to the global financial crisis. d’alpaos and canesi (2014) attempted to correlate risk factors and uncertainties due to the global financial crisis with decision variables. this is another model that emphasizes risk assessment, but is welltailored in the current era of fluid economy and uncertainty. table 1 provides a synopsis of the analyzed decision models, categorized by the point where the decision is made in relation to the assistance point of the developed model. overall, the investigation of investment decisions in real estate has been limited. to the best of our knowledge, there are no studies aiding investment decisions in the prs. this study attempts to address this gap by constructing a decision support tool for the investment appraisal of prs properties in the uk or other similar real estate markets. for its development, we used the ahp since its hierarchical structure encompasses the ijahp article: mantogiannis, katsigiannis/assessing real estate investment alternatives: a multicriteria and multi-stakeholder decision aid tool international journal of the analytic hierarchy process 142 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.702 problem specifications; it is simple and transparent and can be enriched through the consideration of multiple stakeholders’ preferences. ijahp article: mantogiannis, katsigiannis/assessing real estate investment alternatives: a multi-criteria and multi-stakeholder decision aid tool international journal of the analytic hierarchy process 143 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.702 table 1 list of decision models in real estate investment appraisal study decision point focus real estate sector region mcdm method adair and hutchinson, 2005 pre risk assessment land properties uk atherton et al., 2008 pre profit calculation, risk assessment property development chen and khumpaisal, 2009 pre risk assessment commercial uk anp d’alpaos and canesi, 2014 post risk assessment all relevant ginevicius and zubrecovas, 2009 post investment efficiency all relevant electre jiang et al., 2007 pre npv calculation, risk assessment property development ahp kaklauskas et al., 2007 pre utility and market value definition all relevant copras kettani et al., 1998 post market behaviour prediction all relevant canada lopez et al., 2011 pre decision supporting tool all relevant lutzkendorf and lorenz, 2007 pre market value fairness all relevant maliene, 2011 pre evaluation of transaction price specialised properties electre rocha et al., 2007 pre risk assessment, strategy guidance residency development ronyastra et al., 2015 pre optimum portfolio selection commercial, retail, office indonesia promethee tiesmeier, 2016 pre decision structuring luxury residencies spain ahp uysal and tosun, 2004 pre residency selection household residencies turkey wang, 2005 post investment performance government china zavadskas et al., 2004 pre profit maximisation commercial properties eastern europe electre current study pre alternatives evaluation private rented sector uk ahp ijahp article: mantogiannis, katsigiannis/assessing real estate investment alternatives: a multicriteria and multi-stakeholder decision aid tool international journal of the analytic hierarchy process 144 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.702 3. methodology 3.1 research design the research methodology of the present study aims to address the gap identified in the literature review by developing a multi-criteria mechanism that can incorporate the preferences of multiple stakeholders and aid in the assessment of real estate alternatives in the prs under financial performance uncertainty. curating an appropriate mcdm method is of vital importance in order to construct an effective decision model and reach the desired quality of results. guarini et al. (2018) investigated the selection of mcdm methods in real estate decisions and proposed a methodology to select the best-suited method for the specific needs of the evaluation. according to their model, the ahp seems to be the most appropriate method for the investment appraisal of prs properties since it is a full aggregation, low input approach that can analyze quantitative, qualitative or mixed type indicators. in addition, the ahp is the most suitable method to structure a decision problem with a large number of criteria and sub-criteria and a relatively small number of alternatives. taking into consideration the framework of guarini et al., as well as the fact that ahp is the dominant mcdm method of application in the literature, and especially in the field of complex financial decisions, we selected it to approach the examined decision problem. this study aims to provide a decision model that is generally acceptable and incorporates a balanced opinion of experts. therefore, we used the delphi technique to obtain a high quality of input, and this method also helps structure the problem and identifies the appropriate list of decision criteria. the delphi technique is one of the best techniques to reach consensus in a decision-making problem (sekhar et al., 2015), especially in problems where there are no clear criteria for evaluation (taleai and mansourian, 2008). in addition to the assessment and weighting of each criterion, the ahp requires the evaluation of the alternatives based on the significance attributed to each of the assessment criteria. the qualitative criteria in the evaluation process are assessed with the experts’ opinion, while the quantitative ones are based on secondary data. quantitative criteria, such as financial indicators (i.e. future cash flows), often contain growth factors which are either empirical estimates or assumed based on historical data. therefore, it is important to incorporate a method in the financial modeling stage to model the uncertainty occurring from these factors and provide more robust input estimates. loizou and french (2012) proposed the use of a monte carlo simulation as the best method to address risk and uncertainty in real estate decisions. in addition to that, the effectiveness of monte carlo in three of the decision models described in the literature review renders it a suitable method to address this issue in the financial modeling stage (jiang et al. 2007; atherton et al., 2008; ginevicius & zubrecovas, 2009). the list of the experts that participated in the study, along with their role and experience is provided in table 2. all of the participants were approached due to their extensive expertise in the uk prs real estate market. ijahp article: mantogiannis, katsigiannis/assessing real estate investment alternatives: a multicriteria and multi-stakeholder decision aid tool international journal of the analytic hierarchy process 145 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.702 table 2 list of experts expert name role description years of experience participant 1 investor owner of property management company 12 participant 2 investor private real estate investor 23 participant 3 investor private real estate investor 15 participant 4 investor owner of real estate investment company 44 participant 5 consultant senior real estate consultant in major consultancy 10 participant 6 consultant ceo of major real estate consultancy 30 participant 7 consultant head of capital markets in major real estate consultancy 18 participant 8 consultant professor of real estate 27 figure 1 represents the research workflow and demonstrates the construction of the decision model. the research process starts with a literature review of relevant papers, followed by data collection through a 3-round delphi, then financial modeling of the alternatives and finally, an analysis of the data with the analytic hierarchy process (ahp). the literature review is essential to generate an initial list of criteria to input into the delphi method, and to position our study among the existing studies. the delphi method is used to determine the final list of criteria and quantify their weights, which is an essential step for ahp analysis. the ahp analysis also requires an assessment of the alternatives based on the final list of criteria. the input of qualitative criteria requires experts’ opinions, while the financial criteria are determined in the financial modeling stage using the information about the alternatives that is provided by the investors. ijahp article: mantogiannis, katsigiannis/assessing real estate investment alternatives: a multicriteria and multi-stakeholder decision aid tool international journal of the analytic hierarchy process 146 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.702 figure 1 research workflow 3.2 analytic hierarchy process (ahp) the ahp models the decision-making problem as a hierarchical tree by breaking down the objective of the problem, the criteria, the sub-criteria and the alternatives to reach a decision point (saaty, 1986). figure 2 illustrates the main components of the ahp, i.e., upper level (objective) and lower levels (criteria, sub-criteria, and alternatives). the principle of ahp is the pairwise comparison of criteria and of groups of sub-criteria using the fundamental scale of absolute numbers as a scaling method (saaty 1977, 2008). figure 2 generalized representation of an ahp hierarchy the ahp starts with the definition of the decision problem and its objectives, and ends with the evaluation and ranking of the alternatives. it follows the algorithm of the following basic steps (saaty, 1986, 1990, 2008, 2013; saaty & vargas 1987): ijahp article: mantogiannis, katsigiannis/assessing real estate investment alternatives: a multicriteria and multi-stakeholder decision aid tool international journal of the analytic hierarchy process 147 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.702 step 1: define the problematic situation and the decision objective. step 2: structure the hierarchy of the decision, starting with the broad objective and continuing with the decomposition of the situation into criteria, sub-criteria, and alternatives. step 3: collect data by conducting pairwise comparisons within each level of the criteria or sub-criteria, according to the fundamental scale of absolute numbers. step 4: construct pairwise comparison matrices of each level of criteria and of the alternatives under each criterion (square matrix of size n for the criteria themselves and n×m matrix for the comparison of alternatives – n represents the number of criteria and m the number of alternatives). the diagonal elements of the square matrix are equal to one, since it represents a comparison of a criterion with itself, while the value of the rest of the cells (i,j) determines the importance of one criterion over another with regards to the objective of the problem. if the value of the cell (i,j) is greater than one, it indicates that the criterion in the i th is more important than the criterion in the j th column, while if the value is less than one it indicates the opposite. step 5: data normalization and priority extraction. compute the division of each element towards the sum of the corresponding column. then, the sum of the i th row in the normalized matrix determines the weight of the i th criterion (eigenvector wi) as a percentage. finally, the weights of sub-criteria are calculated with regards to the weights of the main criteria by multiplying the percentages of a group of sub-criteria with the weight percentage of the main criterion. step 6: calculate the consistency ratio (cr) as in formula (1): cr=(consistency index (ci))/(random index (ri)) (1) where: 𝐶𝐼=(𝑀𝑎𝑥.𝑒𝑖𝑔𝑒𝑛 𝑣𝑎𝑙𝑢𝑒−𝑛)/(𝑛−1) and 𝑅𝐼 is a 𝑟𝑎𝑛𝑑𝑜𝑚𝑙𝑦 𝑔𝑒𝑛𝑒𝑟𝑎𝑡𝑒𝑑 𝑚𝑎𝑡𝑟𝑖𝑥. saaty (2005) proposes that the 𝐶r should be no more than 0.1 in order to have consistent judgement. step 7: evaluate the alternatives for each sub-criterion. find the normalized values pj(i.j) of the alternative j under the criterion i, by dividing the value of (i,j) in the n×m matrix towards the sum of the i th row. step 8: rate and rank the alternatives. the rating of the alternative j is determined by the algebraic sum of pj(i.j) for all the criteria i. according to this rating, the alternatives are being ranked. step 9: report the final scores for each criterion, sub-criterion, and alternative. 3.3 the delphi technique the delphi method is considered one of the most well-known methodologies that allow a consensus to be reached in a decision-making problem (sekhar et al., 2015). the use of ijahp article: mantogiannis, katsigiannis/assessing real estate investment alternatives: a multicriteria and multi-stakeholder decision aid tool international journal of the analytic hierarchy process 148 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.702 the method originated in the 1950s, when it was first used in a number of surveys conducted by rand corporation to develop a technique that would make it possible to gain the most reliable consensus from a group of people with specific expertise (dalkey & helmer, 1963; okoli & pawlowski, 2004; arof, 2015). it is the most effective method in terms of collecting experts’ opinions about problems, in which there are no clear criteria for evaluation (taleai & mansourian, 2008), and it lies between qualitative and quantitative research techniques (steward, 2001). the delphi method is an iterative process of questionnaires with controlled feedback, given to a panel of experts, who are called panelists and are anonymous (thangaratinam & redman, 2005). an “expert” is considered an individual with relevant knowledge and experience in a particular area of research that is determined by the objectives of the study (cantrill et al., 1996). the research is conducted in multiple rounds of questionnaires. the first round usually involves brainstorming and seeks to obtain an open response about the topic. the panelists are asked to complete an open-ended questionnaire, describe their particular point of view and identify key factors that affect the decision-making process of the problem (arof, 2015). the subsequent rounds are much more specific to the situation and the research question, and seek to quantify the identified factors from the first round. in the second round, panelists rate or rank the importance of each factor according to their expertise, while in the subsequent rounds they review and confirm their responses in order to effectively reach the desired consensus (powell, 2003). the characteristics of the method vary with regards to each study. the first round assesses the qualitative part of the situation, especially in projects with limited literature, while the subsequent rounds quantify the factors identified through the qualitative process. a two or three-round delphi (depending on the use or not of the first round) is usually considered suitable and effective for reaching a consensus (iqbal & pipon-young, 2009), while repeated rounds may lead to fatigue of the participants (walker & selfe, 1996). the size of the panel varies and there are no strict rules on its composition. the number of participants depends on the availability of experts on the particular topic, as well as on the time available to conduct the study. while studies have been conducted with as few as two and as many as three thousand participants, turoff and linstone (2002) suggest between ten to fifty panelists. in contrast to other consensus-reaching methods (e.g. focus groups) in which people interact with each other, the delphi method maintains anonymity among the panelists (hartman, 1981). this is to avoid the disadvantage of a dominant individual or of collective group thinking, which can affect the individual opinions of the experts (kim et al., 2013). in the present study, the delphi technique is used in the preliminary stage of research to identify, shortlist and quantify the criteria that real estate experts take into consideration during the investment appraisal of a project (steps one to three of the ahp). the delphi technique is conducted in three rounds, and the panel of experts consists of seven experts, including four real estate investors and three senior real estate consultants. figure 3 displays the process of the delphi technique in the present study. ijahp article: mantogiannis, katsigiannis/assessing real estate investment alternatives: a multicriteria and multi-stakeholder decision aid tool international journal of the analytic hierarchy process 149 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.702 in the first round, the participants were asked to answer an open-ended questionnaire to explain their views and the criteria that they take into consideration when they appraise a real estate investment. they were also asked to comment on the criteria identified in the relevant literature (contribution to steps 1 & 2 of the ahp). table 3 contains the questions that the participants were asked during the first round of the delphi technique. figure 3 applied delphi process in the study table 3 open-ended questions for the first round of delphi questions 1. can you describe your approach in real estate investment projects? 2. what kind of project do you prefer/suggest for real estate investment? 3. which method of financing do you usually prefer/suggest? 4. name all of the possible financial aspects that could affect your decision about a real estate investment (e.g. npv, irr, profitability index). 5. do you consider location a major criterion when you invest? in your opinion, what are the criteria that affect the location factor? 6. which real estate criteria do you take into consideration (e.g. total area, property condition)? 7. what other criteria come to mind? table 4 shows the initial list of criteria for property investment according to the literature and the opinions of the investors and consultants who participated in the study. the last column identifies which of these criteria have been shortlisted according to the ijahp article: mantogiannis, katsigiannis/assessing real estate investment alternatives: a multicriteria and multi-stakeholder decision aid tool international journal of the analytic hierarchy process 150 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.702 participants regarding their suitability for the present assessment and in an effort to avoid overlapping criteria. table 4 initial list of criteria main criteria sub-criteria source shortlisted financial initial investment outlay ginevičius and zubrecovas, 2009; tiesmeier, 2016 yes total selling price kaklauskas et al., 2007; ginevičius and zubrecovas, 2009; maliene 2011 yes net cash flow ginevičius and zubrecovas, 2009; no net present value (npv) ginevičius and zubrecovas, 2009; ronyastra et al., 2015 yes profitability index ginevičius and zubrecovas, 2009; ronyastra et al., 2015 yes payback period ginevičius and zubrecovas, 2009; ronyastra et al., 2015 yes internal rate of return ginevičius and zubrecovas, 2009; yes mortgage structure and eligibility ginevičius and zubrecovas, 2009; recognised by the investor no maintenance running costs tiesmeier, 2016; recognised by the investor no location prestige of locality maliene, 2011; uysal and tosun, 2014; tiesmeier, 2016 yes market potential zavadskas et al., 2010; maliene, 2011; ronyastra et al., 2015 yes distance from places of interest (city center, universities, businesses etc.) ginevičius and zubrecovas, 2009; maliene, 2011; uysal and tosun, 2014; tiesmeier, 2016 yes public transportation ginevičius and zubrecovas, 2009; zavadskas et al., 2010; maliene, 2011; tiesmeier, 2016 yes existence of car parking ginevičius and zubrecovas, 2009; zavadskas et al., 2010; maliene, 2011; tiesmeier, 2016 yes real estate (property assessment) presence of competitors ginevičius and zubrecovas, 2009; zavadskas et al., 2010; no year of built recognised by the consultant no property condition maliene, 2011; uysal and tosun, 2014; tiesmeier, 2016 yes design / layout maliene, 2011; tiesmeier, 2016 yes total area kettani et al., 1998; kaklauskas et al., 2007; maliene, 2011; uysal and tosun, 2014; tiesmeier, 2016 yes number of bedrooms maliene, 2011; tiesmeier, 2016 yes type of bedrooms recognised by the investor no energy efficiency tiesmeier, 2016; recognised by the consultant yes other amenities (parking kaklauskas et al., 2007; uysal and yes ijahp article: mantogiannis, katsigiannis/assessing real estate investment alternatives: a multicriteria and multi-stakeholder decision aid tool international journal of the analytic hierarchy process 151 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.702 garage, garden etc.) tosun, 2014; tiesmeier, 2016 other risk profile of the project ronyastra et al., 2015 no social benefit ginevičius and zubrecovas, 2009; ronyastra et al., 2015 no the rationale behind the eliminated criteria was explained by the participants. in the financial criteria group, there was no need to include the net cash flow criterion since it is represented by the net present value (npv) criterion. mortgage structure and running costs were eliminated because they are included in the calculation of cash flows. the location criteria group remained the same because the decision makers considered all of the sub-criteria important. in the real estate criteria group, the experts suggested that the presence of competitors is irrelevant to the present problem, the year built is represented through the property condition criterion and the type of bedrooms criteria is represented in the design/layout criteria, while they suggested the title property assessment for the group. although the other criteria groups contain two very important criteria for real estate projects, they are not considered relevant in the present decision-making problem. once the final list of criteria, sub-criteria and the four alternatives to be considered for investment were identified, the present decision-making problem can be structured according to the ahp. figure 4 represents the decision tree that supports the decision of selecting the best real estate project for investment. ijahp article: mantogiannis, katsigiannis/assessing real estate investment alternatives: a multi-criteria and multi-stakeholder decision aid tool international journal of the analytic hierarchy process 152 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.702 figure 4 decision tree ijahp article: mantogiannis, katsigiannis/assessing real estate investment alternatives: a multicriteria and multi-stakeholder decision aid tool international journal of the analytic hierarchy process 153 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.702 during the second round, the participants conducted a pairwise comparison of the final criteria and sub-criteria, according to the fundamental scale of absolute numbers. in the third round, participants were asked to compare the criteria again, using the same process as in round two. this time, the participants were provided with the average panel answer from round two and their individual answer in each particular question, and were asked to confirm or change their responses. this is the point in the process where a consensus should be reached. the mean scores from round three were used as input in step 3 of the ahp. 3.4 financial modeling financial indicators such as net present value, internal rate of return, profitability index and payback period are key criteria in real estate investment projects. it is necessary to know the value of each indicator to evaluate the alternatives in order to apply the ahp. in the present study, the investors identified eight properties as investment opportunities, from which four were used in the assessment. all of the properties are located in the northwest region of england, which is the main area of interest for the participating investors. the properties are located less than 40 miles away from each other so that it was possible to compare them. the alternative projects are being financed through special mortgages provided by uk banks for the private rented sector. the investors give a downpayment of an initial amount, which is a proportion of the total price of the property, and thereafter repay the rest of the amount in monthly installments for a predetermined period of years. in this paper, installments are considered fixed for the total period of the mortgage. in addition, for each alternative, the decision-makers have provided data on monthly rental income and running costs (table 5). the period of the project and the repayment of the mortgage was considered 20 years. the discount rate is 8%, which was the rate suggested by lasalle (2015) and the desired rate of the decisionmakers for the private rented sector. table 5 data for alternatives assessment list of criteria penchwintan, (a1) hicks road, (a2) crosby, (a3) acomb, (a4) location north wales liverpool liverpool manchester initial outlay (£ k) 45 40.5 35 55 total area (m 2 ) 117.5 120 150 180 number of bedrooms 5 5 4 8 monthly income (£) 2180 2300 2000 3500 monthly costs (£) 400 420 350 530 monthly mortgage installment (£) 631 568 450 871 rental income growth rate (std. deviation) 2.46% (2.51%) 3.23% (2.09%) 3.23% (2.09%) 3.23% (2.09%) cost growth rate (std. deviation) 2.22% (1.07%) discount rate r 8% period t (years) 20 ijahp article: mantogiannis, katsigiannis/assessing real estate investment alternatives: a multicriteria and multi-stakeholder decision aid tool international journal of the analytic hierarchy process 154 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.702 rental income and running costs are two variables which are not fixed and are expected to change throughout the total period of the investment. to address this issue in the financial modeling, we used estimators of growth for these two major data. for the rental income growth, we used the annual growth rate provided by the uk housing observatory of the lancaster university management school (yusupova et al., 2015) for each particular region of the alternatives, and for the cost growth, we used the inflation rate in the uk market as the estimator (office for national statistics, 2018). the financial indicators for the ahp analysis were net present value (npv), internal rate of return (irr), profitability index or profit to investment ratio (pi) and payback period (pp). the calculation of npv contains some estimated values (i.e., income and cost growth estimators) which also affects the outcome of irr, pi and pp. therefore, to provide a more precise and valid input for the ahp analysis, we run monte carlo simulations to absorb the uncertainty of these factors. 4. results 4.1 delphi results the mean scores and standard deviations were collected from the two quantitative rounds of delphi. a mean score greater than one indicates that criterion 1 is superior to criterion 2. the closer the value to nine the higher the degree of importance of one criterion over another, while the closer the value to one the more equally the two criteria contribute to the objective. a mean score less than one indicates that criterion 2 is superior to criterion 1 with a degree of importance equal to 1/(mean score). for example, in the main group of criteria, the financial criterion is expected to be the most important since it scores above one when compared with both location and property assessment (3.43 and 3.14, respectively). standard deviation figures are provided to highlight if the experts changed their individual opinions to comport with the rest of the panel or not. the reason why the delphi technique is used in these kinds of knowledge-seeking situations is because it provides the experts an opportunity to re-evaluate and reflect on their initial views, take into consideration the answers that other experts gave, and then modify or hold fast to their ratings. in fact, in all of the pairwise comparison cases, the standard deviation figures were decreased from round 2 to round 3 (sd2=2.45, sd3=1.18), indicating that the experts did change their initial ratings to compromise with each other. this not only verifies the successful application of the delphi method with the experts’ opinions, but also provides the ahp with more reliable input. apart from reaching a consensus in general, there were some disagreements in certain pairwise comparisons between the investors and the consultants. in the pairwise comparison of location with property assessment criteria, the investors rated them as having almost equal importance (1.34), while the consultants’ opinion indicated a moderate importance of location over property assessment. in their feedback, the latter group stated that market potential is a factor that can define the market value of the property in the future and should be taken into consideration more than the property assessment criteria. other points of disagreement were the comparisons of property condition, design/layout and energy efficiency factors with the number of bedrooms. the consultants suggested that the first factors were far more important because they affect running and maintenance costs, and tenant satisfaction, while investors insisted that the ijahp article: mantogiannis, katsigiannis/assessing real estate investment alternatives: a multicriteria and multi-stakeholder decision aid tool international journal of the analytic hierarchy process 155 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.702 number of bedrooms, which affects the rental income of the property was more important. the mean scores from round 3 were used to construct the pairwise comparison matrices for each group of criteria, which were then used as the starting point of the ahp analysis. 4.2 financial modeling results the simulated results of the financial indicators after 5000 iterations are shown in table 6. the mean score of each indicator was used in the ahp analysis, assuming that the number of iterations was enough to provide a high accuracy of input. in addition, the standard deviation, the minimum, and the maximum values were also provided to better understand the behavior of the financial indicators due to the use of estimated values. standard deviation figures are provided to define the lower and upper boundary of each indicator, demonstrating the worstand best-case scenarios. table 6 monte carlo simulation results of the financial indicators financial indicators a1 a2 a3 a4 npv mean 148.54 196.28 178.24 321.20 (£ k) sd 17.11 16.68 14.56 25.17 min 101.12 144.28 129.55 242.12 max 221.05 257.10 226.01 410.81 irr mean 35.72% 45.64% 47.78% 52.79% (%) sd 1.56% 1.41% 1.43% 1.50% min 31.01% 41.65% 43.66% 48.05% max 41.73% 49.78% 53.52% 58.07% pi mean 4.29 5.87 6.06 6.83 (times) sd 0.39 0.41 0.40 0.44 min 3.19 4.64 4.85 5.38 max 5.73 7.36 7.30 8.46 pp mean 3.62 2.58 2.39 2.05 (years) sd 0.25 0.19 0.18 0.19 min 2.92 2.00 1.83 1.49 max 4.63 3.20 3.12 2.69 as table 4 demonstrates, even the financial indicators simulation results show the complexity of the investment decision. the table shows only four out of the seventeen criteria that each alternative evaluated, and even though they are very similar the rankings of the alternatives are different when each individual indicator is considered. the npv results indicate that the a4 alternative is the best to invest in, followed by a2, a3, and a1. the irr, pi and pp results also indicate that a4 is the best investment; however, the following order is different than with npv (a3, a2, and a1). therefore, even within the same group of financial sub-criteria, a different alternative can be preferred for each subcriterion, and the complexity of the decision increases even more when the criteria from the other groups are incorporated in the analysis. having analyzed all of the components of the ahp model proposed in this paper, in the next section we present the results of our study. ijahp article: mantogiannis, katsigiannis/assessing real estate investment alternatives: a multicriteria and multi-stakeholder decision aid tool international journal of the analytic hierarchy process 156 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.702 4.3 ahp results table 7 presents the normalized matrix of the main group of criteria after the weight analysis in the ahp. among the three main criteria, the financial criteria are the most important to consider in the investment appraisal of prs properties. they have three times the weight of location and four times the weight of the property assessment criteria. table 7 normalized matrix of main criteria criteria financial location real estate weights financial 0.621 0.694 0.516 61.0% location 0.181 0.202 0.320 23.5% real estate 0.198 0.104 0.164 15.5% tables 8, 9 and 10 present the normalized matrices and the corresponding weights within each group of sub-criteria. among the financial sub-criteria, npv of future cash flows is the dominant criterion, and accounts for one-third of the group, followed by pi which accounts for about one-fifth. initial investment outlay, irr, and pp contribute almost equally to the decision, while the selling price is the least considered criterion since financing options are available to overcome the obstacle of a high selling price. within the location sub-criteria, the prestige of locality and the market potential are the most important criteria, and account for one-third each. the first determines the attractiveness of the location and often defines the present market value of the property. the latter can define the market value of the property in the future and is very important in the prs because investors in the sector do not only invest because of the rental income from the property, but also to own a valuable asset in the long-term. in the group of property assessment sub-criteria, the property condition and the total area are the most important decision factors since they are usually related to the price fairness of the property. other amenities is the least considered factor, since the amenities increase the price of the property and the running costs without creating any extra value for the owner. however, these weights only correspond inside their group and need to be multiplied by the weight of their corresponding main criterion to determine the final weight on the investment decision. in addition, the importance of the criteria is only one part of the decision process; the other part is the assessment of the alternatives on each criterion, and the final selection and ranking synthesizes these two parts. table 8 normalized matrix of financial sub-criteria sub-criteria initial outlay selling price npv pi pp irr weight initial outlay 0.093 0.248 0.088 0.102 0.074 0.052 11.0% selling price 0.024 0.064 0.158 0.041 0.070 0.052 6.8% npv 0.375 0.145 0.357 0.458 0.521 0.222 34.6% pi 0.183 0.312 0.156 0.200 0.171 0.321 22.4% pp 0.148 0.109 0.081 0.137 0.118 0.253 14.1% irr 0.177 0.122 0.159 0.062 0.046 0.099 11.1% ijahp article: mantogiannis, katsigiannis/assessing real estate investment alternatives: a multicriteria and multi-stakeholder decision aid tool international journal of the analytic hierarchy process 157 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.702 table 9 normalized matrix of location sub-criteria sub-criteria prestige of locality market potential distance from places of interest public transportation car parking weight prestige of locality 0.425 0.551 0.328 0.288 0.224 36.3% market potential 0.212 0.275 0.500 0.329 0.312 32.6% distance from places of interest 0.151 0.064 0.117 0.288 0.240 17.2% public transportation 0.106 0.060 0.029 0.072 0.168 8.7% car parking 0.106 0.049 0.027 0.024 0.056 5.3% table 10 normalized matrix of property assessment sub-criteria subcriteria property condition design/ layout total area no. of bedrooms energy efficiency other amenities weight property condition 0.333 0.399 0.351 0.249 0.284 0.228 30.7% design / layout 0.123 0.147 0.198 0.256 0.081 0.157 16.0% total area 0.245 0.192 0.258 0.332 0.319 0.257 26.7% no. of bedrooms 0.119 0.051 0.070 0.089 0.199 0.137 11.1% energy efficiency 0.105 0.162 0.072 0.040 0.089 0.169 10.6% other amenities 0.075 0.048 0.052 0.034 0.027 0.051 4.8% table 11 provides the final weights of all of the sub-criteria, and therefore their accountability on the decision model, as well as the evaluation of the alternatives according to each criterion. it also includes the objective function, which determines what the decision-maker looks for in the evaluation of an alternative according to the particular criterion. min indicates that the lowest value of the alternatives is desired and the criterion accounts negatively in the ahp score, while max indicates that the maximum value is being sought and the criterion accounts positively in the ahp score. ijahp article: mantogiannis, katsigiannis/assessing real estate investment alternatives: a multicriteria and multi-stakeholder decision aid tool international journal of the analytic hierarchy process 158 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.702 table 11 data to obtain ahp rating and ranking list of criteria objective function weight evaluation of alternatives units a1 a2 a3 a4 initial outlay min 6.68% 45 40.5 35 55 £ k selling price max 4.17% 150 135 109.9 199.9 £ k npv max 21.13% 148.2 195.0 177.5 319.0 £ k pi max 13.67% 4.32 5.83 6.10 6.82 times pp min 8.60% 3.62 2.56 2.38 2.05 years irr max 6.77% 35.80 45.65 47.84 52.88 % prestige of locality max 8.52% 6.2 4.5 3.4 8.5 rating market potential max 7.64% 8.4 4.3 5 3 rating distance from places of interest min 4.03% 0.9 2 2.3 0.5 miles public transportation min 2.04% 0.1 0.1 0.5 1 miles car parking max 1.23% 8 4 5 6 rating property condition max 4.77% 5.5 1 4 6 rating design / layout max 2.48% 3 2 3 6 rating total area max 4.15% 117.5 120 150 180 m 2 no. of bedrooms max 1.72% 5 5 4 8 no. energy efficiency max 1.65% 66 1 64 70 rating other amenities max 0.74% 5 5 3 7 rating from the delphi input and the ahp analysis, it is evident that the decision-makers agree that the financial indicators have higher importance than the qualitative criteria of the decision, such as location and property assessment. among the seventeen total criteria, npv is considered the most important with 21.13% of the total weight, followed by pi which accounts for 13.67%. these two criteria are the only ones with a weight higher than 10%. with all of the financial criteria accounting for 61% of the total weight, the monte carlo was more than essential to absorb uncertainty from the financial figures and to provide significant ahp inputs. in addition, the qualitative criteria (location and property assessment) accounted for 39% of the total weight, and will obviously affect the ijahp article: mantogiannis, katsigiannis/assessing real estate investment alternatives: a multicriteria and multi-stakeholder decision aid tool international journal of the analytic hierarchy process 159 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.702 final selection, especially in cases where an alternative might score well in quantitative factors and fail to score well in qualitative ones when compared to the others. therefore, these criteria should not be ignored, in order to consider the realistic and complex nature of the decision analysis and to reach a quality decision that will include every important factor. based on the objective function, the weight of each criterion and the evaluation of each alternative, the final ahp rating was calculated and the alternatives were ranked accordingly in table 12. table 12 rating and ranking of the alternatives alternative ahp rating rank penchwintan (a1) 2.55 2 hicks road (a2) 1.62 4 crosby (a3) 1.77 3 acomb (a4) 3.06 1 from the case study of the four properties, the acomb (a4) property achieved the highest ahp rating and was the number one choice for investment. acomb had the best scores in five out of six of the financial criteria, which account for the majority of the total weights, and was also the best property in eight out of the eleven qualitative criteria. therefore, this is a natural and expected outcome. however, the complex nature of the investment decision and the effectiveness of the present decision model was highlighted when we excluded the a4 alternative. penchwintan (a1) ranked second in the overall evaluation with an ahp rating higher than the a3 and a2 alternatives. penchwintan (a1) achieved the worst scores in financial evaluation, but good scores in the qualitative criteria when compared to a3 and a2, making it the second-best option for investment. in the initial discussion with the experts when the four alternatives for investment were handed out, one of the investors stated, “with a first glimpse and without having done any kind of analysis, the a1 option seems a very good opportunity”. this might be a coincidence or an exception, but to a certain degree, it indicates the appropriateness of a mcdm application in investment decisions and the capability of the method to model decision instincts and rules of thumb. crosby (a3) managed to rate higher than hicks road (a2), despite the fact that its npv score (the most important criterion) was lower. the two projects had close scores in the rest of the financial criteria, but the high performance of crosby when compared to hicks road, ranked the first one-third in the overall ranking of the alternatives, making the property with the second highest npv the worst investment alternative. the final ranking of the alternatives not only indicate the complexity of the investment decision, but also the impact of the multiple criteria approach in the investment appraisal of a property. ijahp article: mantogiannis, katsigiannis/assessing real estate investment alternatives: a multicriteria and multi-stakeholder decision aid tool international journal of the analytic hierarchy process 160 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.702 5. conclusion a decision-making model for investment in prs properties was developed in the present study, which included financial performance indicators and other qualitative criteria. with the proposed model, interested parties can evaluate real estate alternatives in this sector. the decision model was delivered to the participating experts to assist them in their future selection of properties in the area or in other relevant markets. the case study of the four alternatives was done not only to assist the investors with this specific decision-problem, but also to illustrate the use of the developed multiple criteria procedure. the methodology was developed in order to structure the problem according to the decision objective, include all the relevant criteria, capture high-quality input data and absorb uncertainty to the greatest possible extent. the ahp was selected as the best mcdm method to achieve the desired decision outcome, which is to provide a rate and rank for each alternative. the delphi technique and monte carlo simulations were used along with the ahp method, in order to enhance the quality of the data and create a robust decision model. it is important to highlight the effectiveness of the three-round delphi technique, which helped structure the problem hierarchically with the relevant criteria, and reached the desired consensus in the third round of the pairwise comparisons, which was the primary reason of its use. the findings from the case study indicated the need for multiple criteria frameworks in real estate investment appraisal. even though financial performance is the most important driver for investment decisions in the sector, other qualitative factors cannot be excluded from the decision analysis. according to the experts’ opinions, the financial criteria accounted for the 61% of the total weight, with the rest being the qualitative criteria (i.e., location and property assessment). in a corresponding case study presented by ronyastra et al. (2015), the financial criteria accounted for 73% of the total, which demonstrates the significant importance of this type of criteria in a real estate problem, however, the qualitative aspects of the decision should not be ignored. in the present study, despite the fact that the property which scored the best in the financial criteria was the best investment selection, the qualitative criteria determined the order of ranking of the rest of the alternatives, leaving the alternative with the worst financial performance ranking second. similarly, in ronyastra et al. (2015), the alternative with the best npv was positioned second after an alternative with a much lower risk profile. both studies highlight the importance of qualitative criteria in real estate decision-making. from a managerial perspective, our model helps investors make more informed decisions, with tangible evidence, on the selection of one alternative over another. in addition, it constitutes a tool for property managers to determine the potential of each property in the market. the practical implications of the study are the identification of the criteria and their weights, and the quality of the input and the application of the decision model. this research identified all of the relevant criteria and has categorized them into three groups of sub-criteria, including financial indicators, location criteria, and property assessment. the location and property assessment criteria are both qualitative criteria but are categorized into discrete groups because, according to the experts, location plays a major role in this sector. a high quality of input regarding the weights of the criteria and the evaluation of the alternatives was achieved and increased the robustness of the model. the decision model was implemented by the investors and can be applied to other cases of prs investments. moreover, the model can also be used in other real estate markets (e.g. commercial, office or retail development) with little or no modifications. ijahp article: mantogiannis, katsigiannis/assessing real estate investment alternatives: a multicriteria and multi-stakeholder decision aid tool international journal of the analytic hierarchy process 161 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.702 this research also has some implications for the academic community. the decision model was constructed using both the ahp and delphi methods, and the link with the financial modeling stage provides a higher degree of objectivity in real estate decisions when compared to models that only use experts’ judgments. the monte carlo simulation addresses the uncertainty of the financial indicators by incorporating exogenous factors which are beyond the control of the decision makers. future studies should provide a sensitivity analysis on the impact of cost and income in order to assess their effect on the final decision. another interesting pathway for future research would be to incorporate the operational constraints of the investors. as a future step, we aim to develop a graphical user interface embedding the proposed decision model, therefore facilitating its broader use in the industry. ultimately, the proposition of multi-criteria methods should always consider the trade-offs between inclusivity and complexity. in detail, validating the importance of each criterion with multiple stakeholders may reduce or increase the number of decision nodes depictured in the focal hierarchy. however, increased numbers of criteria (n), despite improving the resolution of the model, result in quadratic increases in the pairwise preference data required [n(n1)/2)]. therefore, the proposed method can serve as a decision aid mechanism rather than a decision-making tool. ultimately, we may conclude that the final decision will be based on the participating stakeholder’s opinions, experience and managerial insights. ijahp article: mantogiannis, katsigiannis/assessing real estate investment alternatives: a multicriteria and multi-stakeholder decision aid tool international journal of the analytic hierarchy process 162 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.702 references adair, a., & hutchison, n. (2005). the reporting of risk in real estate appraisal property risk scoring. journal of property investment & finance, 23(3), 254-268. doi: https://doi.org/10.1108/14635780510599467 andrew, m., & glenn, m. (2003). public and private real estate in a mixed-asset portfolio. journal of real estate portfolio management, 9(3), 193-203. doi: https://doi.org/10.1111/j.1540-6229.2011.00306.x arof, a.m. (2015). the application of a combined delphi-ahp method in maritime transport research-a review. asian social science, 11(23), 73. doi: https://doi.org/10.5539/ass.v11n23p73 atherton, e., french, n., & gabrielli, l. (2008). decision theory and real estate development: a note on uncertainty. journal of european real estate research, 1(2), 162-182. doi: https://doi.org/10.1108/17539260810918730 ball, m. (2010). the uk private rented sector as a source of affordable accommodation. york: joseph rowntree foundation. cantrill, j.a., sibbald, b., & buetow, s. (1996). the delphi and nominal group techniques in health services research. international journal of pharmacy practice, 4(2), 67-74. doi: https://doi.org/10.1111/j.2042-7174.1996.tb00844.x chen, z., & khumpaisal, s. (2009). an analytic network process for risks assessment in commercial real estate development. journal of property investment & finance, 27(3), 238-258. doi: https://doi.org/10.1108/14635780910951957 d’alpaos, c., & canesi, r. (2014). risks assessment in real estate investments in times of global crisis. wseas transactions on business and economics, 11(1), 369-379. dalkey, n., & helmer, o. (1963). an experimental application of the delphi method to the use of experts. management science, 9(3), 458-467. doi: https://doi.org/10.1287/mnsc.9.3.458 geltner, d., & kluger, b. (1998). reit‐based pure‐play portfolios: the case of property types. real estate economics, 26(4), 581-612. doi: https://doi.org/10.1111/15406229.00758 ginevičius, r., & zubrecovas, v. (2009). selection of the optimal real estate investment project basing on multiple criteria evaluation using stochastic dimensions. journal of business economics and management, 10(3), 261-270. doi: https://doi.org/10.3846/16111699.2009.10.261-270 guarini, m.r., battisti, f., & chiovitti, a. (2018). a methodology for the selection of multi-criteria decision analysis methods in real estate and land management mrocesses. sustainability, 10(2), 507. doi: https://doi.org/10.3390/su10020507 ijahp article: mantogiannis, katsigiannis/assessing real estate investment alternatives: a multicriteria and multi-stakeholder decision aid tool international journal of the analytic hierarchy process 163 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.702 hartman, a. (1981). reaching consensus using the delphi technique. educational leadership, 38(6), 495-97. hui, e.c.m., lau, o.m.f., & lo, k.k. (2009). a fuzzy decision-making approach for portfolio management with direct real estate investment. international journal of strategic property management, 13(2), 191. doi: https://doi.org/10.3846/1648715x.2009.13.191-204 jiang, g. m., hu, z. p., & jin, j. y. (2007). quantitative evaluation of real estate's risk based on ahp and simulation. systems engineering-theory & practice, 27(9), 77-81. doi: https://doi.org/10.1016/s1874-8651(08)60054-x kaklauskas, a., zavadskas, e.k., banaitis, a., & šatkauskas, g. (2007). defining the utility and market value of a real estate: a multiple criteria approach. international journal of strategic property management, 11(2), 107-120. doi: https://doi.org/10.3846/1648715x.2007.9637564 kallberg, j.g., liu, c.h., & greig, d.w. (1996). the role of real estate in the portfolio allocation process. real estate economics, 24(3), 359-377. doi: https://doi.org/10.1111/1540-6229.00695 kettani, o., oral, m., & siskos, y. (1998). a multiple criteria analysis model for real estate evaluation. journal of global optimization, 12(2), 197-214. kim, m., jang, y.c., & lee, s. (2013). application of delphi-ahp methods to select the priorities of weee for recycling in a waste management decision-making tool. journal of environmental management, 128, 941-948. doi: https://doi.org/10.1016/j.jenvman.2013.06.049 knight frank research (2017). the uk tenant survey. multi-housing 2017. prs research, uk. lasalle investment management (2015). the case for uk private rented sector,.european research & strategy. loizou, p., & french, n. (2012). risk and uncertainty in development: a critical evaluation of using the monte carlo simulation method as a decision tool in real estate development projects. journal of property investment & finance, 30(2), 198-210. doi: https://doi.org/10.1108/14635781211206922 lópez, v., santos, m., & montero, j. (2010). fuzzy specification in real estate market decision making. international journal of computational intelligence systems, 3(1), 820. doi: https://doi.org/10.2991/ijcis.2010.3.1.2 maliene, v. (2011). specialised property valuation: multiple criteria decision analysis. journal of retail & leisure property, 9(5), 443-450. doi: https://doi.org/10.1057/rlp.2011.7 ijahp article: mantogiannis, katsigiannis/assessing real estate investment alternatives: a multicriteria and multi-stakeholder decision aid tool international journal of the analytic hierarchy process 164 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.702 office for national statistics (2017). english housing survey. headline report 2016-17. ministry of housing communities & local government, uk: 2017. office for national statistics (2018). inflation and price indices, uk. okoli, c., & pawlowski, s.d. (2004). the delphi method as a research tool: an example, design considerations and applications. information & management, 42(1), 15-29. doi: https://doi.org/10.1016/j.im.2003.11.002 powell, c. (2003). the delphi technique: myths and realities. journal of advanced nursing, 41(4), 376-382. doi: https://doi.org/10.1046/j.1365-2648.2003.02537.x rocha, k., salles, l., garcia, f.a.a., sardinha, j.a., & teixeira, j.p. (2007). real estate and real options a case study. emerging markets review, 8(1), 67-79. doi: https://doi.org/10.1016/j.ememar.2006.09.008 ronyastra, i.m., gunarta, i.k., & ciptomulyono, u. (2015). a multi criteria decision analysis for reinvestment action portfolio selection problem in an indonesian real estate company. procedia manufacturing, 4, 558-567. doi: https://doi.org/10.1016/j.promfg.2015.11.076 saaty, t. l. (1977). a scaling method for priorities in hierarchical structures. journal of mathematical psychology, 15(3), 234-281. doi: https://doi.org/10.1016/00222496(77)90033-5 saaty, t. l., & vargas, l. g. (1987). uncertainty and rank order in the analytic hierarchy process. european journal of operational research, 32(1), 107-117. doi: https://doi.org/10.1016/0377-2217(87)90275-x saaty, t. l. (1986). axiomatic foundation of the analytic hierarchy process. management science, 32(7), 841-855. doi: https://doi.org/10.1287/mnsc.32.7.841 saaty, t. l. (1990). how to make a decision: the analytic hierarchy process. european journal of operational research, 48(1), 9-26. doi: https://doi.org/10.1016/03772217(90)90057-i saaty, t. l. (2008). decision making with the analytic hierarchy process. international journal of services sciences, 1(1), 83-98. saaty, t. l. (2013). analytic hierarchy process. in encyclopedia of operations research and management science, 52-64. boston, ma: springer. doi: https://doi.org/10.1007/978-1-4419-1153-7_31 sekhar, c., patwardhan, m., & vyas, v. (2015). a delphi-ahp-topsis based framework for the prioritization of intellectual capital indicators: a smes perspective. procedia-social and behavioral sciences, 189, 275-284. doi: https://doi.org/10.1016/j.sbspro.2015.03.223 ijahp article: mantogiannis, katsigiannis/assessing real estate investment alternatives: a multicriteria and multi-stakeholder decision aid tool international journal of the analytic hierarchy process 165 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.702 soaita, a.m., searle, b.a., mckee, k., & moore, t. (2017). becoming a landlord: strategies of property-based welfare in the private rental sector in great britain. housing studies, 32(5), 613-637. doi: https://doi.org/10.1080/02673037.2016.1228855 stewart, j. (2001). is the delphi technique a qualitative method?. medical education, 35(10), 922. taleai, m., & mansourian, a. (2008). using delphi-ahp method to survey major factors causing urban plan implementation failure. journal of applied sciences, 8(15), 27462751. doi: https://doi.org/10.3923/jas.2008.2746.2751 tiesmeier, d. (2016). mcdm problem-structuring framework and a real estate decision support model, doctoral dissertation, manchester business school. treasury, h.m. (2010). investment in the uk private rented sector. london: hm government. turoff, m. and linstone, h. (2002). the delphi method: techniques and applications. uysal, f., & tosun, ö. (2014). multi criteria analysis of the residential properties in antalya using todim method. procedia-social and behavioral sciences, 109, 322-326. doi: https://doi.org/10.1016/j.sbspro.2013.12.465 walker, a.m., & selfe, j. (1996). the delphi method: a useful tool for the allied health researcher. british journal of therapy and rehabilitation, 3(12), 677-681. doi: https://doi.org/10.12968/bjtr.1996.3.12.14731 wang, w.k. (2005). a knowledge-based decision support system for measuring the performance of government real estate investment. expert systems with applications, 29(4), 901-912. doi: https://doi.org/10.1016/j.eswa.2005.06.017 worzala, e., & bajtelsmit, v. (1997). real estate asset allocation and the decisionmaking framework used by pension fund managers. journal of real estate portfolio management, 3(1), 47-56. yusupova, a., pavlidis, e. g., paya, i., & peel, d. a. (2015). uk housing observatory, department of economics, lancaster university management school. zavadskas, e. k., turskis, z., & kildienė, s. (2014). state of art surveys of overviews on mcdm/madm methods. technological and economic development of economy, 20(1), 165-179. doi: https://doi.org/10.3846/20294913.2014.892037 zavadskas, e.k., ustinovichius, l. & stasiulionis, a. (2004). multicriteria valuation of commercial construction projects for investment purposes. journal of civil engineering and management, 10(2), 151-166. doi: https://doi.org/10.1080/13923730.2004.9636299 zopounidis, c. and doumpos, m. (2002). multi‐criteria decision aid in financial decision making: methodologies and literature review. journal of multi‐criteria decision analysis, 11(4‐5), 167-186. doi: https://doi.org/10.1002/mcda.333 ijahp article: karpak/reflections: mathematical principles of decision making international journal of the analytic hierarchy process 341 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.521 reflections: mathematical principles of decision making birsen karpak distinguished professor of management department of management youngstown state university youngstown, ohio, 44555 bkarpak@ysu.edu abstract this article discusses my reflections on mathematical principles of decision making by thomas saaty (saaty t. l., 2010). in this book, saaty very clearly explains his analytic hierarchy process (ahp) theory for measuring both tangible and intangible factors. experts judgments are elicited about the dominance of a factor over another one via pairwise comparisons using an absolute scale and priorities of the factors are derived. the important concepts of the ahp such as compatibility index, validation, sensitivity analysis for testing the robustness of the priorities derived, and its generalization to structures with dependence and feedback, and the analytic network process (anp) are given. extensions of the theory to complex decisions involving benefits, opportunities, costs and risks and applications to resource allocation and conflict resolution are included, as well as the generalization to continuous and time dependent judgments is also covered 1 . keywords: ahp; anp, bocr; dynamic priorities; continuous judgments; resource allocation; conflict resolution 1. introduction saaty’s ‘mathematical principles of decision making’ is one of my favorites of his work (saaty t. l., 2010). initially, i thought this book would be too difficult and mathematical to understand, yet when i started reading it i was pleasantly surprised that what i had already read in other books and articles was far better and more clearly explained here. when ijahp editors decided to allocate the third issue of the journal solely to thomas saaty’s work, i thought that i should reflect upon this book, and share my views with the audience and hopefully encourage more people to read it. 1 acknowledgments: i would like to thank my friend and colleague thomas saaty who inspired me to work on ahp/anp. because of him i was able to implement a cutting edge methodology in my research and even more important i was able to share this knowledge with my students. mailto:bkarpak@ysu.edu ijahp article: karpak/reflections: mathematical principles of decision making international journal of the analytic hierarchy process 342 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.521 the book consists of eighteen chapters which i will try to reflect upon briefly, but i cannot omit the explanation of the analytic hierarchy process (ahp) in the preface. saaty (2010) writes, ahp is the thinking man’s rational way to combine logic to identify connection among attributes and judgments to derive priorities from causal explanation. its questions revolve around what dominates what on the average or on the whole and how strongly it is expressed verbally and translated numerically with the use of the absolute fundamental scale. (p. xiii) i like to emphasize expressing judgments “verbally.” whenever i use ahp or anp in my research, i always guide the experts to express their judgments in a verbal mode. though this is anecdotal evidence, in almost all cases where my mba students (quite a few of them are managers in organizations) give their judgments in a verbal mode they come up with consistent judgments if they have a choice to make about a real life multi-criteria case they have expertise about. i will write my reflections on the book to try to emphasize the new concepts he introduced and parts i found interesting. i would especially like to communicate to our future generations. as a person who is more interested in solid applications, implementations of the theory and methodology, i want to encourage people who apply ahp/anp to read the book so that they understand the measurement scales, how to construct logical and complete models, how to elicit pairwise comparisons without causing expert fatigue, how to test robustness of the results and how to validate it. structuring is the most creative and challenging part of model building. one should not neglect a sensitivity analysis to test the robustness of the priorities derived for real life decisions. 2. reflections on the book “judgments, as well as the priorities derived from them use absolute numbers,” saaty (2010, p. 24) writes in chapter 1. “one can see from the literature of scales that scales derived as in the ahp are a new paradigm in measurement,” he continues, and asserts that “many people do not understand well” (saaty, 2010, p. 24). i concur with this statement considering the variety of articles i have read and reviewed for journals that were written by people, who use the ahp/anp method but do not work directly in ahp/anp field. i think this book would be very useful for them if they can be convinced to not be afraid of reading it like i was years ago. i like the comments about intuition in this chapter that follow; “…..intuition is very reliable when a knowledgeable person provides judgments….” (saaty, 2010, p. 25). i completely agree; i have observed so many experts comparing criteria and/or alternatives pairwise and giving their judgments who were very pleased with the priorities developed ijahp article: karpak/reflections: mathematical principles of decision making international journal of the analytic hierarchy process 343 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.521 as a result of these comparisons. the market share models of mcdonald’s, arby’s and burger king are another good example. as many readers of this book will know, market share derived using expert’s pairwise comparisons of criteria and alternatives give results very close to the actual market share. i had not thought about the relationship between how the brain works, “how it is characterized as a continuous response mechanism that operates through the electrical firings of neurons,” and ahp/anp until i read chapter two (saaty, 2010, p. 29). he writes, “…there is a law of firing and synthesis of signals in the brain that we can derive from generalizing our representation of the process of creating priorities from pairwise judgments in decision making.” (saaty, 2010, p. 35) there is similarity and difference as well. in my understanding, if the total stimulus is not strong enough (is not over a threshold) the firing of the neuron may not occur and this signal may be lost. in our ahp/anp synthesizing, however, all signals, pairwise judgments contribute to creating priorities. it is fascinating to read this chapter; i hope some researchers will explore this relationship further and discover new information in this area. in chapter 3, saaty explains structures in decision making. in the field of decision making a structure is the first step in organizing, representing and solving a problem. this is the creative part of decision making. as he explains it, structure must be logical and it must relate the items accurately and be complete, including everything that has important influence. i think everybody will concur with this statement, yet it is difficult to test if the structure has all these traits. besides, if the problem is structured as a network with dependence and feedback we may not omit any judgment even if it has very little influence since locally unimportant judgments may become quite important globally because of feedback and dependence. i like the title of chapter 3, “on the subjective geometry of hierarchies and networks.” here, saaty relates the geometry we know with subjective geometry, vertices and edges that connect our subjective thoughts with paths and cycles. i had never related the two until i read this chapter though i understand that saaty and h.s. shih (saaty & shih, 2009) published an article on this subject in the european journal of operations research. i would like to write about the absolute scale, one of the scales of measurement discussed in chapter 4, since i think this is generally one that people are the least familiar with it. i will share comments saaty made on the likert scale, since it is widely used in statistical analysis, with regard to rating versus making comparisons, since some people believe that rating-based ranking is a more objective approach. i will also introduce the concept of compatibility since it is used much less in the applied articles published in journals. saaty writes, “an absolute scale is a special instance of a ratio scale with the constant multiplier equal to one…it is dimensionless.” (saaty, 2010, p.101) authors generally refer to ratio scale any time they introduce ahp and/or anp in a publication, yet i have ijahp article: karpak/reflections: mathematical principles of decision making international journal of the analytic hierarchy process 344 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.521 not read any article other than that belonging to thomas saaty that mentions the absolute scale. the likert scale is a psychometric scale commonly used in questionnaires. “in a rather haphazard way people have the habit of using ordinal numbers to create scales,” saaty writes (saaty, 2010, p. 109). “in addition they perform meaningless arithmetic operation on such numbers that makes no sense,” he continues. on the technical side, it is simply wrong to treat evaluations on a likert scale, such as a 1 or a 5 as real numbers that can be multiplied and summed. if ‘5’ stands for ‘excellent’ and ‘4’ stands for ‘very good,’ we know that ‘5’ is better than ‘4, but we don’t know by how much. we can’t even assume that the distance between ‘5’ and ‘4’ is equal to the distance between ‘4’ and ‘3.’ many users of likert scale are simply unaware of this technical problem. we can use a rating approach for ranking alternatives. “rating is a quick but not accurate way of ranking,” saaty asserts (saaty, 2010, p. 111). “if we have many alternatives, it is convenient and efficient although often misleading,” he continues (saaty, 2010, p.111). i agree with him. i would only use the rating approach for ranking many alternatives, eliminate the worst ones, then continue with pairwise comparisons for the remaining few manageable number, for example, the best ten alternatives. if i use the rating approach i would, at least, determine intensity of scales with pairwise comparisons. checking the consistency of judgments is a well-known concept for ahp/anp applications. authors always illustrate the consistency ratio (c.r) of pairwise comparisons. what we should do if the c.r. is beyond the desired 0.10 for practical applications, is well explained in chapter 4 of the book. though almost all authors denote the consistency of the judgments, very few of them refer to compatibility. the compatibility index illustrates how close two vectors are. if we have two experts from whom we are eliciting judgments we can compare the closeness of their priorities for the alternatives via compatibility analysis. one practical concern while implementing anp is the number of judgments required and their validity. the anp requires quite a few judgments, and experts may not have the patience to provide all of them. chapter 5 addresses the expert fatigue problem and how to expedite making the decision without loss of validity. the best approach in my judgment is to construct as small of a model as possible, yet it should be a valid representation of the reality. once you have the most compact model, with the smallest number of components, and the smallest number of elements in each component, you can reduce number of judgments elicited from the experts by comparing, for example, all the elements in the top row as that set of n-1 judgments forms a spanning tree. of course, the most scientific approach would be to apply harker’s algorithm explained in this chapter. harker suggests that decision maker should provide at least one judgment in each column. chapter 6 covers the axioms of ahp/anp, whereas chapter 7 deals with how to construct a group judgment from the judgments of individuals comprising this group. a group ijahp article: karpak/reflections: mathematical principles of decision making international journal of the analytic hierarchy process 345 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.521 judgment also has to satisfy the reciprocal property. it is very difficult to summarize and comment on them in this reflection. these two chapters are quite mathematical; they are for those who want to learn mathematical principals of decision making. validation is relevant in social as well as physical sciences, as saaty writes in chapter 8 (saaty, 2010). he gives examples for validation for a single matrix, for hierarchies, as well as for networks. he and a colleague find the relative wealth of seven nations as a result of pairwise comparisons with respect to relative power and standing with no direct consideration of the economic data. the judgements were based on their perception of “soft-power”. they illustrate how close the values are to values obtained by normalizing the gnp of the nations. he does not do any compatibility analysis of these two vectors. i did not either; i believe he did this exercise before he came up with the concept of compatibility; yet it would be interesting to calculate one. my gut feeling, looking at the numbers, says that it is much less than 1.1. as i was writing about this, i realized that relative power needs to be defined. i think gnp is one factor of relative power, but it has other components too. i would not think that priorities developed comparing different nations with respect to relative power and standing illustrates priorities of the gnp. i would have identified factors influencing gnp and compared different nations with respect to these factors. i wished i could have discussed it with him because i hope i am not missing something. as an example, for hierarchical validation he analyzes the decision of the u.s. congress towards china in 2000 to grant the country permanent normal trade relations (pntr) status. he then illustrates that with a benefits, opportunities, costs and risks (bocr) model pntr is a dominant alternative. i wished that congress had actually made a decision based on the result of a bocr analysis that they had participated as experts in giving pairwise comparisons in the bocr model. that would have been a far more convincing validation example for me. one anp validation example that is given predicted the market share of airlines and compared it with the actual market share. validation is far more challenging, in my judgment, when we don’t know the results, when experts compare qualitative factors pairwise. saaty concludes this chapter by asserting that group decision making “tends to give better results because of the broader knowledge and because of the possibility of debate” (saaty, 2010, p. 282). we can use priorities of the alternatives as objective function coefficients of a linear programming (lp) problem and determine the best resource allocation. an example of resource allocation with ahp is given in chapter 9, concerning human resource recruiting at biological detection systems (bds). since there are 31 applicants, saaty used the rating approach to rank them. then these priorities are used as objective function coefficients and solved by lp subject to salary, and some other constraints. such constraints are, for example, there should be just one marketing assistant, at most one shipping clerk, at most one quality specialist and so on. he concludes the chapter saying, “many real-world problems cannot be readily solved by lp since they usually contain ijahp article: karpak/reflections: mathematical principles of decision making international journal of the analytic hierarchy process 346 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.521 intangible variables” (saaty, 2010, p.300). combining lp with ahp and or anp “makes it possible to deal with all optimization problems whether the variables are tangible or intangible.” (saaty, 2010, p. 300) i hope i created some interest that will cause you to want to read this book because of what i commented on so far. i will be far briefer for the following eight chapters since my expertise is more limited on conflict resolution, time dependent decisions, and continuous judgments. chapter 10 mostly reports saaty’s conflict resolution endeavors for the middle east israeli-palestinian conflict. this project, mainly funded in its later years by a well-known foundation, “involved a mixed group of palestinians, knowledgeable pro-israeli experts, and others from the outside, like china, saudi arabia, turkey and the us.” (saaty, 2010, p.317) over a three-day period, the team structured the problem and developed several potential alternatives. the purpose was to formulate packages of tradeoffs, comprised of both tangible and intangible factors, so that both parties would feel they were fairly treated. i think it would be interesting to many ahp scholars to read the entire process from the book as i cannot summarize it in a couple of paragraphs. at the end, the bocr synthesis suggested that “the best policy to resolve the middle east conflict is to establish a two state solution.” (saaty, 2010, p.340) sensitivity analysis suggests that results were robust. saaty reports that “the most significant results come from the efforts and road-map generated.” (p.340) saaty asserts, in chapter 11 that our political bodies need a formal approach for prioritization. he explains how different decision making is in politics from decision making in business. he prioritized most of the bills that were before the u.s. congress in 2005. first, he prioritized major areas of government. each major criterion has subcriteria with priorities. the alternatives were rated with respect to the sub-criteria using an intensity scale determined by making pairwise comparisons. of course, he is well aware that he needs “to seek further agreement from many more people in government to fine-tune the results he obtained.” (p.365) he concludes that, “voting on bills in government is a very complicated process.” (p. 365) an official who has to vote needs to “consider national objectives, constituency interest and commitments to party and to other senators or representatives.” (p. 368) chapters 12, 13, and 14 deal with uncertainty and time dependence of judgments and answer criticisms of the ahp. he introduces the “other” criterion if the expert is unsure of his/her structure, and illustrates with a sensitivity analysis how priorities of the alternatives might change with the introduction of “other” criterion. i have not seen the “other” criterion or “other” alternative so far among the research i have read on ahp/anp applications and/or theory. i have seen relatively little research, again so far, on time-dependent decision making, or dynamic priorities in the ahp/anp. chapter 13 could be a good start on these topics. there is a lot to be done on generalizing from discrete to continuous judgments, as response to stimuli. ijahp article: karpak/reflections: mathematical principles of decision making international journal of the analytic hierarchy process 347 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.521 3. conclusion as saaty writes in the conclusion of the book, it takes little time to learn the mechanics of ahp/anp, yet structuring the problem is far more challenging, requiring expertise, creativity and good knowledge about the problem. one should not ignore checking how robust the priorities are to changes in expert judgments by performing a sensitivity analysis of the problem. readers will need to explore some more recent publications to understand sensitivity analysis in anp models. as i wrote above, there is a lot of work to be done on continuous as well as time-dependent judgments. ijahp article: karpak/reflections: mathematical principles of decision making international journal of the analytic hierarchy process 348 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.521 references saaty, t. l. (2010). mathematical principles of decisions making. pittsburgh: rws publications. saaty, t. l., & shih, h. s. (2009). structures in decision making: on the subjective geometry of hierarchies and networks. european journal of operations research, 199(3), 867-872. doi: https://doi.org/10.1016/j.ejor.2009.01.064 https://doi.org/10.1016/j.ejor.2009.01.064 ijahp article: barić, starčević/implementation of analytic hierarchy process in solving transport problems international journal of the analytic hierarchy process 295 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.251 implementation of analytic hierarchy process in solving transport problems danijela barić faculty of transport and traffic sciences university of zagreb, croatia dbaric@fpz.hr martin starčević faculty of transport and traffic sciences university of zagreb, croatia martin.starcevic@fpz.hr abstract this paper presents the analysis of the implementation of the analytic hierarchy process (ahp) in solving problems in the area of transport. the transport sector is currently one of the most important sectors, and an efficient transport system requires continuous planning. investments in transport infrastructure are projects that have specific characteristics. these projects have an extremely long lifetime (some thirty years), are not profit-oriented, are usually financed by the state of the local administration, and are primarily for social use. the decision-making process, which includes making decisions on investments, is extremely complex. the decision-maker has to envisage the future, and consequently make decisions in a modern and flexible manner. the methods used in the process of evaluating the investment projects in transport are numerous and usually distinguished by the optimization criteria. the projects can be evaluated through the prism of a single criterion or several criteria. the most common optimization criterion is the economic system; however, long-term and high-quality solutions require the introduction of other optimization criteria such as technological, technical, ecological and many others. therefore, in such situations it is essential to implement multi-criteria decision methods which will eventually result in making the optimal decision. it will be determined in the paper to what extent the ahp method is implemented regarding transport problems and for which categories of transport problems. key words: multi-criteria decision-making; analytic hierarchy process; transport; investments mailto:dbaric@fpz.hr mailto:martin.starcevic@fpz.hr ijahp article: barić, starčević/implementation of analytic hierarchy process in solving transport problems international journal of the analytic hierarchy process 296 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.251 1. introduction traffic is like the bloodstream of every country, and therefore has deep effects on the economic and other activities within the country. the level of development of the transport system is correlated to the level of economic development and quality of the living conditions, and thus the entire economy of a country, region or area depends on transport. therefore, it is vital to the interests of every community to continuously upgrade the level of quality of their transport system. transport problems are not isolated and independent, and their solutions require an interdisciplinary approach. project planning in transport can be characterized as a process of creative optimization using available resources. the analyses have shown that in practice (i.e. in pre-investment and investment studies) projects whose aim is financial efficiency most often rely on methods where a single criterion is used. one such method is the cost-benefit analysis, and it is used in the perspective of financial decision-making. the projects evaluated in this manner are acceptable if they have been positively assessed according to certain criteria of financial decision-making, i.e. expressed in monetary values. on the other hand, in scientific papers, analysis to select an optimal project use other criteria which are not based exclusively on the criteria of financial categories, monetary flows, etc. it is, namely, because of the complexity of the traffic system, that the approach of assessing a project with several criteria is important. the implementation of several criteria is used in assessing the projects using multi-criteria decision-making methods. one of the most frequently applied methods to assess projects in the transport sector is the analytic hierarchy process. it was developed by thomas saaty in the 1970s in order to solve the complex problems of decision making. it is one of the best known, most verified and most used decision making methods, i.e. method for multi-criteria analysis. its basic advantage is reflected in the decision maker’s ability to adjust the number of attributes. attributes are criteria and alternatives about the decision being made simultaneously, and can be described both quantitatively and qualitatively. therefore, the ahp method allows flexibility of the decision making process and helps the decision makers to set the priorities, and make the best decision while taking into consideration both qualitative and quantitative aspects of decisions. the implementation of the analytic hierarchy process is significant in major investment projects that require substantial capital engagement, and have high social significance (e.g. transport infrastructure investment projects). 2. objectives various methods are used for solving transport problems including both singleor multicriteria ones. the objective of the paper is to determine to what extent the ahp method is used for decision-making in transport problems and for which categories of transport problems. the analysis of the papers in which the ahp method is implemented for solving the problems in the area of traffic primarily meant searching through single databases according to the key words in the title of the paper. however, numerous papers in which ahp has been implemented have ahp in their title. therefore, it may be expected that the research results will deviate to a certain extent from the actual situation. ijahp article: barić, starčević/implementation of analytic hierarchy process in solving transport problems international journal of the analytic hierarchy process 297 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.251 3. methodology the areas of transport will be classified in which ahp is usually implemented, the category of papers will be determined in which ahp is usually used and the frequency of implementing ahp will be defined by reviewing the available literature, scientific papers, technical papers, diploma theses, doctoral dissertations and projects in the area of transport. relevant databases will be analyzed. the scientific papers that primarily deal with the application of the ahp method have been analyzed by searching the journals of the leading publishers of academic papers (elsevier, taylor & francis, springer, hrčak, etc.), selected scientific journals, selected conferences, doctoral dissertations, and diploma theses. 3.1 analysis of the number of published papers in which analytic hierarchy process has been implemented in solving the transport issues in academic journals of selected publishers journals from the available databases have been analyzed using the following publishers: elsevier, taylor & francis, springer, hrčak and hrvatska znanstvena bibliografija (e. croatian scientific bibiliography). the databases have been searched using the selected key words: ahp & analytic hierarchy process in the titles, key words and summaries. only those papers in which the ahp method was implemented for solving traffic and transport problems have been selected. by searching the databases, the journals in which published papers that have been primarily oriented to solving transport problems using the analytic hierarchy process were determined. according to isi web of knowledge, relevant scientific journals from the category transportation science & technology are as follows: ieee transactions on intelligent transportation systems journal of transportation engineering-asce proceedings of the institution of civil engineers-transport transportation science iet intelligent transport systems ite journal-institute of transportation engineers transportation letters-the international journal of transportation research international journal of automotive technology transportation research record transportation research part a, b, c, d, e proceedings of the institution of mechanical engineers part d, part f networks & spatial economics transportation promet-traffic & transportation journal of intelligent transportation systems transportmetrica transportation planning and technology transport journal of advanced transportation 3.2 analysis of published papers in which the analytic hierarchy process has been implemented for solving transport issues in selected academic journals academic papers between 2009-2013 from three journals were analyzed to find articles in which the ahp method has been implemented for solving traffic problems. the selected journals were the international journal of the analytic hierarchy process ijahp article: barić, starčević/implementation of analytic hierarchy process in solving transport problems international journal of the analytic hierarchy process 298 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.251 (issn: 1936-6744), promet – traffic & transportation (issn: 1848-4069), and transport (1648-4142 (print), 1648-3480 (online)). the international journal of the analytic hierarchy process (ijahp) was selected because of its primary purpose of publishing articles about multi-criteria decision-making using the analytic hierarchy process (ahp) and the analytic network process (anp). ijahp has been continuously published since 2009. there have been 10 issues with a total of 45 papers published, out of which six can be identified as those dealing with the implementation of the ahp method for solving problems in the area of transport (table 1). table 1 selected papers dealing with the implementation of the ahp method for solving transport problems, published in the ijahp from 2009 to 2013 vol./no./y. ijahp authors title of the paper vol. 2, no. 1 (2010) leila paula alves da silva nascimento, mischel carmen neyra belderrain purchase of a new air superiority fighter using the analytic network hierarchy process vol. 2, no. 2 (2010) sukulpat khumpaisal, mohd noor nazali, zairul musa nisham, andrew david ross an application of the analytic network process to assess risks in a megaconstruction project vol. 3, no. 2 (2011) eugene kopytov, vasilijs demidovs, natalia petukhova application of the analytic hierarchy process in development of train schedule information systems vol. 4, no. 1 (2012) patrizia lombardi, silvia giordano evaluating the european smart cities visions of the future vol. 4, no. 1 (2012) marta bottero, valentina ferretti, silvia pomarico assessing the sustainability of alternative transport infrastructures vol. 4, no. 1 (2012) francesca abastante, marta bottero, isabella maria lami using the analytic network process for addressing a transport decision problem the international journal promet-traffic & transportation was selected as the national journal from the author’s country that is primarily oriented to papers in the field of transport. this journal has been continually publishing scientific papers for as many as 26 years. the journal is published bimonthly, and up to now there have been 143 issues in which 1,065 papers have been published, out of which seven can be singled out with the primary topic of the implementation of the ahp method for solving transport problems. in the period from 2009 to 2013 there were 30 issues of this journal in which a total of 259 academic papers were published, out of which 4 can be singled out in which the ahp method was implemented for solving transport problems (see table 2). although we chose to analyze the period 2009-2013, it is important to mention that this journal published scientific papers with the implementation of the ahp method earlier (bošnjak et al., 2005, barić et al., 2007, jugović et al., 2007). ijahp article: barić, starčević/implementation of analytic hierarchy process in solving transport problems international journal of the analytic hierarchy process 299 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.251 table 2 selected papers in which the ahp method has been implemented for solving transport problems, published in promet-traffic & transportation from 2009 to 2013 vol./no./y. promet authors title of the paper vol. 22, no. 3 (2010) deda đelović, dijana medenica mitrović an approach to the selection of optimal transformation strategy in seaports vol. 23, no. 4 (2011) aleksandra deluka-tibljaš, barbara karleuša, čedomir benac ahp methodology application in garageparking facility location selection vol. 23, no. 5 (2011) fatemeh haghighat application of a multi-criteria approach to road safety evaluation in the bushehr province, iran vol. 23, no. 6 (2011) andrija vidović, sanja steiner, igor štimac development potentials of low cost aviation in the republic of croatia transport has been selected as an international journal primarily oriented towards papers in the area of transport. this journal has been published since 2002 and since then there have been 57 issues with a total of 606 scientific papers out of which 10 focused on the implementation of the ahp method to solve transportation problems. the period from 2009 to 2013 saw the publication of 20 issues of this journal with a total of 251 academic papers published, out of which 9 papers could be singled out which dealt with the implementation of the ahp method (see table 3). ijahp article: barić, starčević/implementation of analytic hierarchy process in solving transport problems international journal of the analytic hierarchy process 300 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.251 table 3 selected papers in which the ahp method was implemented to solve transport problems, published in transport from 2009 to 2013 vol./no./y. transport authors title of the paper volume 24, issue 2 (2009) lijana maskeliūnaite, henrikas sivilevičius & valentinas podvezko research on the quality of passenger transportation by railway volume 25, issue 2 (2010) nebojsa bojovic, branislav boskovic, milos milenkovic & aleksandar sunjic a two‐level approach to the problem of rail freight car fleet composition volume 25, issue 4 (2010) henrikas sivilevičius & lijana maskeliūnaite the criteria for identifying the quality of passengers’ transportation by railway and their ranking using ahp method volume 25, issue 4 (2010) zenonas turskis & edmundas kazimieras zavadskas a new fuzzy additive ratio assessment method (aras‐f). case study: the analysis of fuzzy multiple criteria in order to select the logistic centers location volume 26, issue 1 (2011) henrikas sivilevičius modelling the interaction of transport system elements volume 27, issue 3 (2012) szabolcs duleba, tsutomu mishina & yoshiaki shimazaki a dynamic analysis on public bus transport's supply quality by using ahp volume 28, issue 4 (2013) jinbao zhao & wei deng fuzzy multiobjective decision support model for urban rail transit projects in china volume 28, issue 4 (2013) szabolcs duleba, yoshiaki shimazaki & tsutomu mishina an analysis on the connections of factors in a public transport system by ahp-ism volume 28, issue 4 (2013) valentinas podvezko & henrikas sivilevičius the use of ahp and rank correlation methods for determining the significance of the interaction between the elements of a transport system having a strong influence on traffic safety figure 1 shows the share of the published papers in which the analytic hierarchy process was implemented to solve transport problems in the total number of published papers in the ijahp, promet-traffic&transportation and transport in the last five years. ijahp article: barić, starčević/implementation of analytic hierarchy process in solving transport problems international journal of the analytic hierarchy process 301 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.251 figure 1. share of papers in which the analytic hierarchy process has been implemented for solving transport problems in ijahp, promet – traffic & transportation and transport figure 2 shows the comparison of the three selected journals regarding the total number of the published papers in the last five years and the number of the published papers in which the analytic hierarchy process was implemented. during the period 2009–2013, a larger number of papers related to ahp-transport issues were published in transport (9 papers) than in ijahp (6 papers) and promet – traffic & transportation (4 papers). figure 2. comparison of the total number of published papers and papers in which the analytic hierarchy process has been implemented in this paper, we selected papers from three journals; however, we analyzed several other journals which published articles discussing ahp, for example transportation (gercek et. al, 2004), the baltic journal of road and bridge engineering (aghdaie et al., 2012), automation in construction (lin et al., 2008), journal of business economics and management (podvezko et al., 2009), tehnički vjesnik – technical gazette (rouyendegh ijahp article: barić, starčević/implementation of analytic hierarchy process in solving transport problems international journal of the analytic hierarchy process 302 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.251 et al., 2012), journal of transportation engineering (sun et al., 2011), transportation research part d: transport and environment (vreeker et al., 2002), etc. 3.3 analysis of the published papers at the international symposium on the analytic hierarchy process (isahp) in which the analytic hierarchy process has been used for solving transport issues the international symposium on the analytic hierarchy process (isahp) for decision making is a conference on multi-criteria decision analysis, particularly the analytic hierarchy process (ahp) and its extension, the analytic network process (anp), both developed by thomas l. saaty. the symposium brings together researchers, teachers, students and users of ahp/anp to share their research and experiences in decision making. table 4 presents the selected papers from isahp conferences (2009 to 2013) in which the analytic hierarchy process has been primarily implemented to solve transport issues. in four previous isahp conferences before 2009 (2001, 2003, 2005 and 2007) some authors published papers focusing on the implementation of ahp method for solving transport issues (anagnostopoulos et al., 2001, ding et al., 2001, anagnostopoulos et al., 2003, galvão jr., 2003, gerdsri et al., 2005, ko, 2005, piantanakulchai, 2005, jung et. al., 2007). ijahp article: barić, starčević/implementation of analytic hierarchy process in solving transport problems international journal of the analytic hierarchy process 303 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.251 table 4 selected papers focusing on the implementation of the ahp method for solving transport issues, published at isahp from 2009 to 2013 isahp authors title of the paper isahp 2009 denise lindstrom bandeira, joão luiz becker, amanda kruse rocha applying ahp to the prioritization of maritime booking confirmation thomas michael lintner the aerospace performance factor: utilization of the analytic hierarchy process to develop a balanced performance and safety indicator of the national airspace system for the federal aviation administration berna ulutas, burak ulutas an analytic network process combined data envelopment analysis methodology to evaluate the performance of airports in turkey isahp 2011 pablo aragonás-beltrén, j.p. pastor-ferrando, j. montesinos-valera selection of rail improvement projects using the analytic network process (anp) elio padoano, fabio lamanna, giovanni longo, giorgio medeossi, stefano strami the ahp approach for railway project appraisal: proposal for a specific problem structure isabella m. lami, francesca abastante, marta bottero using the analytic network process (anp) for ranking the critical issues of a transport decision problem: the italian section of the corridor 24 marta bottero, valentina ferretti, silvia pomarico an application of the analytic network process for assessing the sustainability of different transport infrastructures waclaw przybylo, józef moryl choice of the best public transportation system for the city of cracow and cracow district eugene kopytov, vasilijs demidovs, natalia petukhova choice of the model of presenting temporal data in the system of the train traffic schedule isahp 2013 giuseppe bruno, emilio esposito, andrea genovese a hybrid ahp/fst model for regional aircraft evaluation christian tabi amponsah an integrated approach for prioritizing projects for implementation using ahp richard cimler analytic hierarchy process and agent-based simulation for traffic modeling mohd uzair mohd rosli car’s dashboard improvement design concepts through integration of ahp and triz insannul kamil, buang alias, abdul hakim mohammed, nilda tri putri, cresti kalani design of performance evaluation tools for drainage of roads system in developing country (case study: drainage system for city roads in padang indonesia) ijahp article: barić, starčević/implementation of analytic hierarchy process in solving transport problems international journal of the analytic hierarchy process 304 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.251 the share of papers in which the ahp method has been implemented to solve transport issues is presented in figure 3. in all isahp conferences (2001-2013) a total of 610 papers were published, of which 22 were related to transport issues. during the analyzed period 2009-2013, 346 papers were published of which 14 related to transport issues. the highest number of total papers at a single conference was at isahp 2011 (162 papers), and this was also the conference with the highest number of papers related to ahptransport issues (6 papers). figure 3. share of papers in which the ahp method has been implemented, published at all isahp conferences according to data about the topic of the papers in table 4, the share of papers has been determined with regard to the transport branch (figure 4). figure 4 shows that the analytic hierarchy process is most used in solving road traffic problems (27%), followed by rail (18%) and air transport (14%). also, the implementation of the ahp method is important in the selection of the priorities in the realization of the traffic projects (18%) and the selection of the terminal locations (9%). the analytic hierarchy process was also used in the papers for the selection of the best public transport system (5%), selection of the sustainable transport infrastructure (5%) and the maritime traffic problem (4%). ijahp article: barić, starčević/implementation of analytic hierarchy process in solving transport problems international journal of the analytic hierarchy process 305 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.251 figure 4. share of the papers in which ahp method has been implemented with regard to the transport branch and purpose 3.4 analysis of students’ papers at the university of zagreb, faculty of traffic and transport sciences implementing the analytic hierarchy process for solving transportation issues at the university of zagreb, faculty of transport and traffic sciences undergraduate, graduate, specialist and doctoral study programmes are performed. the students’ papers have been analyzed including diploma thesis, master thesis and doctoral dissertations, in which primarily the ahp method was used to solve the transport problems from all transport branches (road, rail, air, urban, telecommunications, maritime). the selected academic papers are presented in table 5. table 5 overview of students’ papers (diploma thesis, master thesis, doctoral dissertation) dealing with the implementation of ahp method in solving transportation issues paper category/year author title of the paper doctoral dissertation (2013) grgurević, i. determination of starting locations of carpooling trips in the cities doctoral dissertation (2013) petrović, m planning of locations of intermodal passenger terminals in urban-suburban rail transportation doctoral dissertation (2010) barić, d. model of planning traffic and technological projects in railways development doctoral dissertation (2010) vidović, a. model of low-tariff aircraft operations in croatia master thesis (2010) ćosić, p. segmentation of users of telecommunication services by means of multi-criteria decision-making ijahp article: barić, starčević/implementation of analytic hierarchy process in solving transport problems international journal of the analytic hierarchy process 306 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.251 paper category/year author title of the paper diploma thesis (2013) ćorluka, i. selection of the optimal version of the project of reconstructing the intersection of bistrička street and trakošćanska street in the city of zagreb by applying analytic hierarchy process diploma thesis (2013) divić, k. assessment of possible solutions of reconstructing the intersection of state road 60 and county road 6157 by applying ahp method diploma thesis (2013) glasnović, m. selection of optimal version of the project in reconstructing the intersection of križni put street and blaža jurišića avenue in zagreb using multi-criteria decision-making diploma thesis (2013) karoglan, a. assessment of possible solutions of reconstructing the intersection of kaštelanska street and exit from jadranska magistrala in the city of solin by applying ahp method diploma thesis (2013) kljajić, m. assessment of possible solutions of reconstructing the intersection of the streets petra svačića josipa rimca hanibala lucića in slavonski brod by applying multicriteria analysis diploma thesis (2012) babić, s. analysis of assessment methods of road traffic projects diploma thesis (2012) jurić, i. assessment of versions in solving the reconstruction of the intersection of čavićeva street and borongajska street in the city of zagreb by implementing analytic hierarchy process diploma thesis (2012) mikić, v. implementation of ahp method in assessing the variant solutions in reconstructing the intersection of dubrovačka street – industry road in the city of požega diploma thesis (2011) bareša, i. assessment of traffic and technological projects of the city of velika gorica and proposal for design priority diploma thesis (2011) kostić, i. multi-criteria decision-making in selection of optimal project of intersection reconstruction diploma thesis (2011) sirovec, s. assessment of traffic and technological projects of the city of krapina and proposal of design priorities diploma thesis (2011) jozić, v. selection of mobility control measures in the cities using multi-criteria analysis diploma thesis (2010) radoš, i. multi-criteria analysis of possible scenarios of the zagreb airport development source: croatian scientific bibliography (crosbi), http://bib.irb.hr/ figure 5 indicates an increase of academic papers about the use of the ahp method for solving transport issues, especially in diploma theses, reflecting the growing popularity of the ahp method among students. http://bib.irb.hr/ ijahp article: barić, starčević/implementation of analytic hierarchy process in solving transport problems international journal of the analytic hierarchy process 307 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.251 figure 5. share of the academic theses in which ahp method has been implemented with regard to the transport issues 4. discussion and conclusions based on the previous research it may be concluded that the implementation of ahp in solving problems in the area of transport is extremely high. the analysis of relevant databases has shown that the ahp method has been implemented in scientific papers, scientific projects, diploma theses and doctoral dissertations. although the cost-benefit analysis is currently the only method implemented in practice, the number of scientific papers based on the ahp method is an indicator for the justification of implementing this method in practice as well, i.e. in pre-investment and investment studies. the methods used in the process of assessing transport and technological projects, including making investment decisions, are numerous and usually differ regarding criteria of optimization. the projects can be assessed through the prism of a single criterion, but also based on several criteria. the usual optimization criterion is the economic characteristic. however, for the purpose of long-term and higher quality solutions it is necessary to introduce several criteria of optimization. apart from the economic criteria, the technological, technical, environmental and numerous other criteria are significant. this is where the ahp method plays a very important role. by analyzing the literature it can be concluded that there are numerous papers in which the ahp method has been implemented in order to solve transport problems. all of the transport branches have been represented including road, rail, air, maritime, telecommunications, and postal traffic. within the context of ahp method implementation in order to solve transport problems, it may be concluded that the ahp method is usually implemented for the selection of the optimal road or rail route, selection of the optimal transport means, establishing priorities in project realization, ijahp article: barić, starčević/implementation of analytic hierarchy process in solving transport problems international journal of the analytic hierarchy process 308 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.251 selection of the optimal telecommunication operator, selection of the airline, urban traffic management, determining of locations of terminals, definition of schedules, selection of the optimal traffic infrastructure, determining of road and rail corridors, etc. generally, these issues can be divided into the following three groups: transport planning, transport infrastructure and transport management. ijahp article: barić, starčević/implementation of analytic hierarchy process in solving transport problems international journal of the analytic hierarchy process 309 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.251 references aghdaie, m. h., hashemkhani zolfani, s., zavadskas, e. k (2012). prioritizing constructing projects of municipalities based on ahp and copras-g: a case study about footbridges in iran. the baltic journal of road and bridge engineering, 7(2), 145– 153. http://dx.doi.org/10.3846/bjrbe.2012.20 amponsah, c.t. (2013). an integrated approach for prioritizing projects for implementation using ahp, proceedings of the 12th international symposium on the analytic hierarchy process, kuala lumpur, malaysia, 23 rd – 26 th june 2013. anagnostopoulos, k., giannopoulou, m., roukounis, y., vavatsikos, a. (2001). using ahp to select projects in urban planning, proceedings of the 6th international symposium on the analytic hierarchy process, berne, switzerland, 2 nd – 4 th august 2001, 1-10. anagnostopoulos, k.p., giannopoulou, m., roukounis, y. (2003). a two-steps procedure for selecting projects at regional level, proceedings of the 7th international symposium on the analytic hierarchy process, bali, indonesia, 7 th – 9 th august 2003, 109-118 aragonás-beltrén, p., pastor-ferrando, j.p., montesinos-valera, j. (2011). selection of rail improvement projects using the analytic network process (anp), proceedings of the 11th international symposium on the analytic hierarchy process, sorrento, naples, italy, 15 th – 18 th june 2011. bandeira, d.l., becker, j.l., rocha, a.k. (2009). applying ahp to the prioritization of maritime booking confirmation, proceedings of the 10th international symposium on the analytic hierarchy process, pittsburgh, pennsylvania, usa, 29 th july – 1 st august 2009. barić, d., čurepić, d., & radačić, ž. (2007). implementation of relevant methods in assessing traffic-technological projects, promet – traffic & transportation, 19(5), 329336. bošnjak, i., kavran, z. & matijević, d. (2005). design of delivery areas of public postal operator by applying ahp model. promet – traffic & transportation, 17(2), 77-75. bottero, m., ferretti, v., pomarico, s. (2011). an application of the analytic network process for assessing the sustainability of different transport infrastructures, proceedings of the 11th international symposium on the analytic hierarchy process, sorrento, naples, italy, 15 th – 18 th june 2011. bruno, g., esposito, e., genovese, a. (2013). a hybrid ahp/fst model for regional aircraft evaluation, proceedings of the 12th international symposium on the analytic hierarchy process, kuala lumpur, malaysia, 23 rd – 26 th june 2013. http://dx.doi.org/10.3846/bjrbe.2012.20 ijahp article: barić, starčević/implementation of analytic hierarchy process in solving transport problems international journal of the analytic hierarchy process 310 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.251 cimler, r. analytic hierarchy process and agent-based simulation for traffic modeling, proceedings of the 12th international symposium on the analytic hierarchy process, kuala lumpur, malaysia, 23 rd – 26 th june 2013. deluka-tibljaš, a., karleuša, b. & benac, č. (2011). ahp methodology application in garage-parking facility location selection. promet – traffic & transportation, 23(4), 303-313. đelović, d. & mitrović, d.m. (2010). an approach to the selection of optimal transformation strategy in seaports. promet – traffic & transportation, 22(3), 219-228. galvão jr., f. de a., cunha, c.b., gualda, n.d.f. an application of the analytic hierarchy process (ahp) for locating a distribution center, proceedings of the 7th international symposium on the analytic hierarchy process, bali, indonesia, 7 th – 9 th august 2003, 243-252. gercek, h., karpak, b., kilincaslan, t.(2004). a multiple criteria approach for the evaluation of the rail transit networks in istanbul, transportation, 31(2), 203-228. gerdsri, n., kocaoglu, d.f. (2005). applying the analytic hierarchy process (ahp) to build a strategic framework for technology roadmapping, proceedings of the 8th international symposium on the analytic hierarchy process, honolulu, hawaii, usa, 8 th – 10 th july 2005. haghighat, f. (2011). application of a multi-criteria approach to road safety evaluation in the bushehr province, iran. promet – traffic & transportation, 23(5), 341-352. jung, b.d., kwon, y., kim, h. (2007). the analysis of priorities of its using analytic hierarchy processes, proceedings of the 9th international symposium on the analytic hierarchy process, valparaíso, chile, 2 nd – 6 th august 2007. kamil, i., alias, b., mohammed, a.h., putri, n.t., kalani, c. (2013). design of performance evaluation tools for drainage of roads system in developing country (case study: drainage system for city roads in padang indonesia), proceedings of the 12th international symposium on the analytic hierarchy process, kuala lumpur, malaysia, 23 rd – 26 th june 2013. ko, j. (2005). solving a distribution facility location problem using an analytic hierarchy process approach, proceedings of the 8th international symposium on the analytic hierarchy process, honolulu, hawaii, usa, 8 th – 10 th july 2005. kopytov, e., demidovs, v., petukhova, n. (2011). choice of the model of presenting temporal data in the system of the train traffic schedule, proceedings of the 11th international symposium on the analytic hierarchy process, sorrento, naples, italy, 15 th – 18 th june 2011. lami, i.m., abastante, f., bottero, m. (2011). using the analytic network process (anp) for ranking the critical issues of a transport decision problem: the italian section ijahp article: barić, starčević/implementation of analytic hierarchy process in solving transport problems international journal of the analytic hierarchy process 311 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.251 of the corridor 24, proceedings of the 11th international symposium on the analytic hierarchy process, sorrento, naples, italy, 15 th – 18 th june 2011. lin, c. c., wang, w. c., yu, w. der. (2008). improving ahp for construction with an adaptive ahp approach (a3). automation in construction, 17(2), 80–187. lintner, t.m. (2009). the aerospace performance factor: utilization of the analytic hierarchy process to develop a balanced performance and safety indicator of the national airspace system for the federal aviation administration, proceedings of the 10th international symposium on the analytic hierarchy process, pittsburgh, pennsylvania, usa, 29 th july – 1 st august 2009. padoano, e., lamanna, f., longo, g., medeossi, g., strami, s. (2011). the ahp approach for railway project appraisal: proposal for a specific problem structure, proceedings of the 11th international symposium on the analytic hierarchy process, sorrento, naples, italy, 15 th – 18 th june 2011. piantanakulchai, m. (2005). analytic network process model for highway corridor planning, proceedings of the 8th international symposium on the analytic hierarchy process, honolulu, hawaii, usa, 8th – 10th july 2005. podvezko, v. (2009). application of ahp technique. journal of business economics and management, 10(2), 181–189. http://dx.doi.org/10.3846/1611-1699.2009.10.181189 przybylo, w., moryl, j. (2011). choice of the best public transportation system for the city of cracow and cracow district, proceedings of the 11th international symposium on the analytic hierarchy process, sorrento, naples, italy, 15 th – 18 th june 2011. rosli, m.u.m., (2013). car’s dashboard improvement design concepts through integration of ahp and triz, proceedings of the 12th international symposium on the analytic hierarchy process, kuala lumpur, malaysia, 23 rd – 26 th june 2013. rouyendegh, b. d., erkan, t. e. (2012). selection of academic staff using the fuzzy analytic hierarchy process (fahp): a pilot study. tehnički vjesnik – technical gazette, 19(4), 923–929. sun, l., gu, w. (2011). pavement condition assessment using fuzzy logic theory and analytic hierarchy process. journal of transportation engineering, 137(9), 648–655. http://dx.doi.org/10.1061/(asce)te.1943-5436.0000239 ulutas, berna, ulutas, burak (2009). an analytic network process combined data envelopment analysis methodology to evaluate the performance of airports in turkey, proceedings of the 10th international symposium on the analytic hierarchy process, pittsburgh, pennsylvania, usa, 29 th july – 1 st august 2009. vidović, a., steiner, s. & štimac, i. (2011). development potentials of low cost aviation in the republic of croatia. promet – traffic & transportation, 23(6), 519-527. http://dx.doi.org/10.3846/1611-1699.2009.10.181-189 http://dx.doi.org/10.3846/1611-1699.2009.10.181-189 http://dx.doi.org/10.1061/(asce)te.1943-5436.0000239 ijahp article: barić, starčević/implementation of analytic hierarchy process in solving transport problems international journal of the analytic hierarchy process 312 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.251 vreeker, r., nijkamp, p., ter welle, c. (2002). a multicriteria decision support methodology for evaluating airport expansion plans. transportation research part d: transport and environment, 7(1), 27–47. ijahp article: demirtas, buruk, sağır/a multi-criteria job evaluation method for a state bank international journal of the analytic hierarchy process 188 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.264 a multi-criteria job evaluation method for a state bank ezgi aktar demirtas eskisehir osmangazi university industrial engineering department, meselik campus, 26480 eskisehir/turkey e-mail:eaktar@ogu.edu.tr yeliz buruk eskisehir osmangazi university industrial engineering department, meselik campus, 26480 eskisehir/turkey e-mail:yelizburuk@ogu.edu.tr müjgan sağır eskisehir osmangazi university industrial engineering department, meselik campus, 26480 eskisehir/turkey e-mail: mujgan.sagir@gmail.com abstract wage management is an important task which affects a firm’s productivity in the short term and the consistency of the firm’s activities in long term. if an organization fails to establish a fair wage policy among the personnel, conflict is inevitable in the organization. a fair wage policy can be achieved by job evaluation, which is a technique used to determine the relative importance of all jobs in an organization. jobs are evaluated with respect to ability, responsibility, effort, and job condition factors etc. which make it a multi-criteria problem for organizations. in this study, a job evaluation methodology is developed for a state bank in turkey. the relative importance of the evaluation criteria which is then used to grade jobs with respect to one another by a liberatore scale is determined by an analytic network process (anp) model. this new methodology has a positive effect on competence and performance management systems. keywords: job evaluation; analytic network process; multi-criteria decision making; banking. 1. introduction the most important characteristic of the job evaluation problem is the existence of multiple factors that influence the evaluation. the evaluation process is often the duty of a committee; and the data that are available are fuzzy while the description, responsibilities and requirements of the jobs are not usually precisely determined. nevertheless, for many organizations job evaluation is a crucial activity that enables the rationalization of the links between the importance of a job and the corresponding wage ijahp article: demirtas, buruk, sağır/a multi-criteria job evaluation method for a state bank international journal of the analytic hierarchy process 189 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.264 (chen & lee, 2007; erarslan et. al., 2013; spyridakos et. al., 2001). it is a systematic process that enables the design and establishment of human resources improvement procedures and fair wage systems. actually, job evaluation is concerned with the assessment of a value system that encapsulates the importance of the parameters of and reflects the global responsibility and duties of a job. it is remarkable that job evaluation is not concerned with those holding the job, but instead with how much responsibility the job has and its share in the production of the desired results. the job evaluation process results in a positive influence on competence and performance management. this is because it helps establish a reward system that links the importance of the job to the payment offered, and supports the designation of human resources development requirements in order to improve the effectiveness of the job’s operation. a considerable number of approaches have been developed and used for the job evaluation process. the simplest ranks or classifies the jobs based on a comparative process or a points factor rating system (armstrong and murlis, 1994; neathley, 1994). another approach utilizes a scale for the classification of the jobs. different levels of grades are assigned to a number of characteristics such as “decision making”, “knowledge required” and “equipment used”. every job is placed in a position on the total scale by its evaluation of the characteristics. this approach can easily be applied in cases where there are a small number of jobs to be evaluated, the jobs are not too complex and can be described by the characteristics used, and it is quite easy to determine the lines between two neighboring positions on the scales for every characteristic. another commonly used approach is based on the comparison of the job with an internal benchmark. this approach cannot be applied in cases where there are a small number of jobs and not a high degree of differentiation among the jobs. the most common and perhaps the most reliable method in job evaluation is the point method (erarslan & arıkan, 2004; xing, 2008). according to this method the evaluation of the jobs are derived from a multi-attribute value system. the principles of this value system are based on the essentials of the multi-attribute utility theory (keeney and raifa, 1976; keeney, 1992). this approach is widely used by management consultants and usually provides reasonable results, but is lacking when it comes to estimation of the weights of the attributes and the evaluation of the jobs on the criteria. actually, the weights of factors are estimated through a survey analysis or are directly expressed by an expert or a management consultant. it is obvious that in this case the determination of the components of the value system operates like a “black box” for the organization. also, the individual circumstances of the enterprise or organization are not taken into account to the extent that is required (spyridakos et al., 2001). recently, the most widely used methods in job evaluation are the ahp and anp methods, developed and introduced to the literature by thomas l. saaty. they are mainly used to solve multi-criteria decision-making problems (saaty, 2001; saaty, 2000). one of the most important assumptions of ahp is that criteria at the same level are independent from one another and that their effects on each other are not taken into consideration. in fact, many criteria affecting decision-making problems interact with each other, and it is vital to pay attention to these relationships among criteria in order to make the best decisions possible. ijahp article: demirtas, buruk, sağır/a multi-criteria job evaluation method for a state bank international journal of the analytic hierarchy process 190 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.264 on the other hand, the anp eliminates the necessity of modeling by sticking to a hierarchical structure (saaty, 2001). the decision-making problem using anp is modeled as a network and all internal dependencies (feedbacks) and interactions among the criteria are taken into consideration during the modeling process. moreover, it is not costly to apply. as evaluation is done using a scale appropriate for the jobs in the enterprise, it enables the enterprise to develop its own evaluation system. the possibility of making mistakes is enormously low as long as the phases of anp are performed consistently and regularly. absolute and understandable results are obtained, and structural changes in jobs and enterprise can easily be reflected on the evaluation plans. in recent studies, spyridakos (2001) worked on the multi-criteria job evaluation for large organizations, dağdeviren (2004a) used ahp to evaluate different jobs in an electricity enterprise, dağdeviren also (2004b) used a goal programming model to determine factor degree points, chen & lee (2007) used a performance evaluation model based on anp for project managers using managerial practices, and erarslan et. al. (2013) used the fuzzy ahp method for the job evaluation procedures in a private steel company. although, seçme et. al. (2009) used fuzzy performance evaluation in the turkish banking sector using ahp and topsis, a study that evaluates banking jobs by multicriteria methods has not been conducted to the best of our knowledge. in the present study, anp was used because it can exactly respond to the subjective needs of the enterprise and partially eliminate the inconveniences of the conventional methods. characteristics such as its flexibility, its ability to find solutions to problems in a short time, its capacity to incorporate all kinds of interactions, dependency and feedback in the model and the opportunities it provides to evaluate all relationships systematically make anp superior over the other methods. the interaction and feedback among the criteria also required the author to use anp rather than ahp in the study. anp is based upon pairwise comparison just like ahp, and saaty’s 1-9 ratio scale is used in pairwise comparisons. the super decisions software package was used to determine criteria weights. 2. a feedback model to evaluate jobs for a state bank an anp model was proposed in order to determine the weights of the criteria and subcriteria that were used to evaluate the jobs in a state bank. criteria and sub-criteria were adapted from the “job grouping system-jgs” which began to be used in 1982, and was expanded in the turkish metal industrialists union publication in 1996 (metal grouping system, 1996). the jgs is used to determine and evaluate the circumstances, difficulties and commitments required of the job in daily life. this system is not a tool designed for determining the performance of workers, but for evaluating the jobs. the jgs not only contributes to the determination of the payment system, but also to the planning of training for workers and their professional development and promotion. it evaluates a job using the following four main criteria: ability, responsibility, effort and working conditions. sixteen different banking jobs in a branch of a state bank were evaluated by four main jgs criteria and 17 sub-criteria. one thousand total points were distributed among the criteria and sub-criteria in accordance with the weights obtained by anp. during the distribution of sub-criteria points among the criteria degrees, a liberatore scale ijahp article: demirtas, buruk, sağır/a multi-criteria job evaluation method for a state bank international journal of the analytic hierarchy process 191 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.264 (liberatore, 1992) was used. the higher the total points obtained from the sub-criteria, the higher the level of required qualities, responsibilities and abilities the person who will do that job must have. the number of criteria and sub-criteria also meet the standard of including a maximum of seven sub-criteria within each cluster in the network (saaty, 2001). this flexible method is easy to understand and apply. if new criteria are introduced in accordance with different job systems, these changes can easily be incorporated into the model. the proposed method enables related people to take part in the decision-making process (group decision making). it also ensures a significant decrease in the decision making time when compared to other job evaluation methods. one of the most important characteristics which distinguish this method from others is the consistency ratio acquired from pairwise comparisons. this ratio provides people within and outside the enterprise with information about the reliability of the results of the study and prevents possible disagreements. the structure of the anp makes it applicable in the evaluation of the jobs done in different enterprises by using similar and/or different criteria. the feedback network is illustrated in figure 1 where criteria are illustrated by clusters and sub-criteria are illustrated by nodes. arrows are used to show the feedbacks within the clusters and the interactions among the clusters. figure 1. the feedback network for the state bank anp is used to determine the alternative weights and ranking in problems in which there are a finite number of alternatives. however, there are no clusters of alternative networks in this study since our aim is to design a system which serves a more general purpose and will be used in the evaluation of all the jobs in the enterprise. the cluster of alternatives is made up of all the jobs in the enterprise. when all kind of jobs are evaluated in big ijahp article: demirtas, buruk, sağır/a multi-criteria job evaluation method for a state bank international journal of the analytic hierarchy process 192 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.264 enterprises, it is certain that pairwise comparisons are unreliable and consistency is not available due to the fact that job characteristics could somehow be different. for this reason, only the criteria and sub-criteria weights were determined through anp. all interactions and feedbacks in the networks summarized in table 1 were determined, and the required connections were made with the help of the human resources department and general director of the state bank. the definitions for all criteria are shown in the appendix. table 1 the relationships of the criteria affected criteria effector criteria skills 1.education 2, 3 2.craft knowledge 1, 3, 4, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17 3.experience 1, 2 4.complexity of job 1, 2, 5, 7, 8, 9, 10, 11, 12, 13, 14 5.judgement 1, 2, 3, 4, 7, 13, 14 6.language skills 1, 2 7.hardware and software skills 1, 2, 3, 4, 10, 11, 13, 14 responsibility 8.job relationship responsibility 2, 3, 5, 13, 14 9.legal and financial responsibility 2, 3, 5, 13, 14 10.job follow-up 2, 3, 5, 13, 14 11.tracking career developments 1, 2, 6, 7 12.administrative responsibility 1, 2, 3, 5, 13, 14 effort 13.job concentration 8, 9,10, 12, 14, 15, 16, 17 14.mental effort 1, 2, 4, 5, 6, 7, 8, 9, 10, 11, 12 15.physical effort 16, 17 conditions 16.physical conditions 13, 15 17.personel safety 2, 3, 13, 15 3. criteria weighting pairwise comparisons of the criteria and sub-criteria were done using saaty’s 1-9 ratio scale. the inconsistency ratio is required to be less than 0.1. sub-criteria weights were obtained from the pairwise comparisons and are shown in table 2. ijahp article: demirtas, buruk, sağır/a multi-criteria job evaluation method for a state bank international journal of the analytic hierarchy process 193 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.264 table 2 the relative importance of sub-criteria sub-criteria relative importance education 0.074959 craft knowledge 0.229686 experience 0.115071 judgment 0.049841 complexity of job 0.052167 language skills 0.014348 hardware and software skills 0.063694 job relationship responsibility 0.025682 legal and financial responsibility 0.031823 job follow-up 0.060793 tracking career developments 0.040720 administrative responsibility 0.042092 job concentration 0.05169 mental effort 0.097818 physical effort 0.014426 physical conditions 0.026392 personnel safety 0.008397 in this study, the five point rating scale developed by liberatore (1992) was used in the distribution of sub-criteria points among the criteria levels. the weight of each level was determined by the analytic hierarchy process. in a five point scale, the highest level is always considered as nine times more important than the lowest level. moreover, the inconsistency ratios are at the level of 0.05 or lower. the level weights found for the five point scale are shown in table 3. table 3 the relative importance of levels level 1 2 3 4 5 0.033 0.063 0.129 0.261 0.513 in the distribution of the sub-criteria scores among the criteria levels, the sub-criterion score was considered to be the score of the highest level. the scores of the other levels were found by comparing the highest level with the each individual level. for instance, level 4 point for ‘experience’ is calculated as 59 by using equation 1. level 4 point for ‘experience’ = (0.261*115)/(0.513) = 59 (1) in the same way, the other sub-criteria points are distributed among the criteria levels and shown in table 4 for the banking jobs. ijahp article: demirtas, buruk, sağır/a multi-criteria job evaluation method for a state bank international journal of the analytic hierarchy process 194 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.264 table 4 criteria, sub-criteria and level scores criteria/sub-criteria total scores level scores skills 599 1 2 3 4 5 education 75 5 9 19 38 75 craft knowledge 229 15 28 58 117 229 experience 115 7 14 29 59 115 complexity of job 52 3 6 13 26 52 judgment 50 3 6 13 25 50 language skills 14 1 2 4 7 14 hardware and software skills 64 4 8 16 33 64 responsibility 202 job relationship resp. 26 2 3 7 13 26 legal and financial resp. 32 2 4 8 16 32 job follow-up 61 4 7 15 31 61 tracking career develop. 41 3 5 10 21 41 administrative resp. 42 3 5 11 21 42 effort 164 job concentration 52 3 6 13 26 52 mental effort 98 6 12 25 50 98 physical effort 14 1 2 4 7 14 conditions 35 physical conditions 26 2 3 7 13 26 personnel safety 9 1 1 2 5 9 a questionnaire was prepared to determine sub-criteria levels for 16 different banking jobs. according to the questionnaire, sub-criteria levels and total scores of the banking jobs are summarized in table 5 and table 6. ijahp article: demirtas, buruk, sağır/a multi-criteria job evaluation method for a state bank international journal of the analytic hierarchy process 195 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.264 table 5 sub criteria levels for the banking jobs b a n k in g j o b s b ra n c h d ir e c to r b a n k in g e x p e rt a ss is ta n t b a n k in g e x p e rt m a rk e ti n g d e p a rt m e n t d ir e c to r m a rk e ti n g d e p a rt m e n t c o -d ir e c to r m a rk e ti n g d e p a rt m e n t s e rv ic e e x e c u ti v e m a rk e ti n g d e p a rt m e n t s e rv ic e p e rs o n n e l c re d it d e p a rt m e n t d ir e c to r c re d it d e p a rt m e n t c o -d ir e c to r c re d it d e p a rt m e n t s e rv ic e e x e c u ti v e c re d it d e p a rt m e n t s e rv ic e p e rs o n n e l o p e ra ti o n a l jo b s d e p a rt m e n t d ir e c to r o p e ra ti o n a l jo b s d e p a rt m e n t c o -d ir e c to r o p e ra ti o n a l jo b s s e rv ic e e x e c u ti v e o p e ra ti o n a l jo b s s e rv ic e p e rs o n n e l s e c u ri ty p e rs o n n e l skills education 5 4 4 4 4 3 3 5 4 4 3 5 4 3 3 3 craft knowledge 5 4 4 4 4 3 3 5 4 4 3 5 4 3 3 4 experience 5 4 4 5 4 3 3 5 4 4 3 5 4 3 3 4 complexity of job 5 4 4 4 4 3 3 4 4 4 3 5 4 3 3 2 judgment 5 4 4 5 4 3 3 5 4 4 3 4 4 3 3 4 language skills 4 4 4 4 4 3 3 4 4 4 3 4 4 3 3 2 hardware and software skills 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 responsibility job relationship resp. 5 4 4 4 4 3 3 5 4 4 3 5 4 3 3 4 legal and financial resp. 4 4 4 4 4 3 3 4 4 4 3 4 4 3 3 3 job follow-up 4 4 4 5 4 4 4 4 4 4 4 4 4 4 4 3 tracking career develop. 4 4 4 4 4 4 3 4 4 4 4 4 4 4 3 3 administrative resp. 4 4 3 4 4 3 3 4 4 3 3 4 4 3 3 3 effort job concentration 4 4 4 4 4 4 3 4 4 4 4 4 4 4 3 4 mental effort 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 physical effort 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 5 conditions physical conditions 5 4 4 4 4 4 4 4 4 4 4 5 4 4 4 5 personnel safety 3 3 3 4 4 3 3 3 3 3 3 3 4 3 3 5 ijahp article: demirtas, buruk, sağır/a multi-criteria job evaluation method for a state bank international journal of the analytic hierarchy process 196 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.264 table 6 total scores of the banking jobs banking jobs total score branch director 787 operational jobs department director 762 credit department director 748 marketing department director 619 marketing department co-director 508 operational jobs department co-director 508 banking expert 505 credit department co-director 505 assistant banking expert 495 credit department service executive 496 security personnel 418 marketing department service executive 345 credit department service personnel 345 operational jobs service executive 345 operational jobs service personnel 321 marketing department service personnel 321 4. conclusions an anp model was proposed to evaluate the jobs in a branch of a state bank in turkey. the weights of the criteria and sub-criteria were determined by using the anp model. one thousand points in total were distributed among the criteria and sub-criteria in accordance with the weights obtained through anp. a liberatore scale was used for the distribution of the sub-criteria points among the criteria degrees. finally, total scores were calculated for each job. the anp is a flexible method which is easy to use. when new criteria need to be introduced for different job systems, these changes can easily be incorporated into the model. besides, the method enables related people to take part in the decision-making process and facilitates group decision-making. this guarantees a significant decrease in decision making time in comparison with other job evaluation methods. moreover, the consistency ratios obtained through anp prevent possible disagreements by providing people within and outside the enterprise with the necessary information about the reliability of the results of the study. the software package ‘super decisions’ doesn’t pave the way for making a sensitivity analysis without an alternative cluster. this cluster consists of all the works in the enterprise. when all jobs in a big enterprise are evaluated using anp, it is certain that pairwise comparisons are unreliable and inconsistency occurs. for this reason, in enterprises in which there are few jobs (less than 7), a sensitivity analysis can be done by including alternative clusters in the anp model. under these circumstances, how the ijahp article: demirtas, buruk, sağır/a multi-criteria job evaluation method for a state bank international journal of the analytic hierarchy process 197 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.264 weight of the criteria and sub-criteria as well as the significance levels of jobs will change in case of a change in the weight of any criterion can easily be seen. ijahp article: demirtas, buruk, sağır/a multi-criteria job evaluation method for a state bank international journal of the analytic hierarchy process 198 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.264 references armstrong, m. & murlis, h., (1994). reward management: a handbook of renumeration strategy and practice. london: kogan page. chen, s. h., lee, h. t. (2007). performance evaluation model for project managers using managerial practices. international journal of project management, 25(6), 543–551. dağdeviren, m., akay, d. and kurt, m. (2004a). analytical hierarchy process for job evaluation and application. journal of the faculty of engineering and architecture of gazi university, 19, 131-138. dağdeviren, m., akay, d. and kurt, m. (2004b). job evaluation, use of goal programming technique at determine factor degree points. journal of the faculty of engineering and architecture of gazi university, 19, 89-95. eraslan, e. and arıkan, b. (2004). determination of wages using the point method, seniority and success: an application in internal production department of a manufacturing facility. journal of the faculty of engineering and architecture of gazi university, 19, 139-150. eraslan, e., atalay, k. d., dagdeviren, m., aksakal, e. (2013). using fuzzy wage management system in heavy industry. procedia social and behavioral sciences, 73, 7 – 13. keeney, r. (1992). value-focused thinking: a path to creative decision making. london: harvard university press. keeney, r.l. & raifa, h. (1976). decision with multiple objectives: preferences and value tradeoffs. new york: wiley. liberatore, m.j., nydick, r.l., sanchez, p.m. (1992). the evaluation of research papers. interfaces, 22(2), 92-100. neathley, f. (1994). job evaluation in the 1990s. london: industrial relations services. saaty, t. (2000). fundamentals of decision making and priority theory with the analytic hierarchy process. pittsburgh, usa: rws publications. saaty, t.l. (2001). decision making with dependence and feedback: the analytic network process. pittsburgh, usa: rws publications. secme n. y., bayrakdaroglu, a., kahraman, c., (2009). fuzzy performance evaluation in turkish banking sector using analytic hierarchy process and topsis. expert systems with applications, 36(9), 11699–11709. spyridakos, a., siskos, y., yannacopoulos, d., skouris, a. (2001). multicriteria job evaluation for large organizations. european journal of operational research, 130(2), 375-387. ijahp article: demirtas, buruk, sağır/a multi-criteria job evaluation method for a state bank international journal of the analytic hierarchy process 199 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.264 metal industry job grouping system. (1996). i̇stanbul: turkish metal industrialists union publication, şahinkaya press. xing, l., li, y. and ma, l. (2008). study on the application of point method in job evaluation. proceedings of the 7th wuhan international conference on e-business, china, 2835-2839. ijahp article: demirtas, buruk, sağır/a multi-criteria job evaluation method for a state bank international journal of the analytic hierarchy process 200 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.264 appendix criteria definition skills 1.education required education level and degree for performing job with a satisfactory level. 2.craft knowledge all technical, procedural and organizational knowledge such as knowledge of equipment and machinery, knowledge of concepts, ideas, other cultures or languages required for the job. 3.experience after the acquisition of the necessary basic information and education, the required experience period for doing the job accurately and with the desired quality. 4.complexity of job complexity of jobs created by job characteristics such as variability, diversity, and uncertainty etc. 5.judgement the ability to act independently and accurately and have fast decision making skills and use initiative. 6.language skills the required level of foreign language related to the job. 7.hardware and software skills the ability of using office equipments such as typewriters, computers, copiers, etc. and/or the level of knowledge of the software features related to the job. responsibility 8.job relationship responsibility the direct responsibility for the supervision, coordination or management of employees, or others in an equivalent position. 9.legal and financial responsibility responsibility for legal issues and financial resources such as cash, vouchers, cheques, debits and credits, invoices, budgets and income 10.job follow-up once a job starts, it is usually required to follow-up the activities related to it in order to achieve the organizational goals in terms of cost and quality. 11.tracking career developments it is required to expand job-related knowledge to improve ability and earnings potential. 12.administrative responsibility responsibility of supervising and directing others, the number and type (qualification) of people supervised. effort 13.job concentration the level of required attention of the senses such as seeing, hearing, feeling, touch. 14.mental effort analytical, problem solving and judgmental skills related to design, handling of people, development of policies and procedures, and planning and strategic skills are required. 15.physical effort the degree of physical effort required to do the job with a normal tempo. conditions 16.physical conditions the degree of unpleasantness or discomfort caused. 17.personel safety despite the implementation of all safety requirements, consideration of the possible accidents and sanitary drawbacks which can arise from the nature of the job. ijahp article: degtiarev, borisov/prioritization of alternatives based on analytic hierarchy process using interval type-2 fuzzy sets and probability-theoretical interval comparison international journal of the analytic hierarchy process 447 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.586 prioritization of alternatives based on analytic hierarchy process using interval type-2 fuzzy sets and probability-theoretical interval comparison konstantin y. degtiarev national research university higher school of economics (hse) faculty of computer science school of software engineering kdegtiarev@hse.ru mikhail y. borisov rwth aachen university, media informatics program michaelborisovha@gmail.com abstract the analytic hierarchy process (ahp) enables decision-makers to prioritize alternatives. however, when an expert expresses judgments using natural language statements (e.g. words or phrases) inherent vagueness of language constructs can cause the interpretation to be imprecise. the fuzzy analytic hierarchy process (fahp) can be viewed in the context of the classical ahp expansion. while performing pairwise comparisons domain experts are accustomed to operating with verbal terms in their judgments. most existing fahp approaches do not consider a human’s confidence in the estimates provided. this paper presents a model that gives weight to the constraints on domains of expert assessments as they are almost always supplied with certain degrees of confidence. interval type-2 membership functions (it2mf) along with the probability-theoretical procedure for comparison of intervals can be applied here as suitable modeling options. empirical comparison of fahp that makes use of triangular fuzzy numbers and it2mfbased fahp is also presented. keywords: analytic hierarchy process; expert assessment; degree of confidence; linguistic label; type-1 membership function; fuzzy number; interval type-2 membership function; fuzzy synthetic extents; interval calculations; prioritizing alternatives 1. introduction people constantly make decisions starting from utterly trivial cases based on unconscious instantaneous choices and ending with important ones that require a deep grasp of the matter. as a rule, human decision-framing abilities allow for specifying a potential set of domain specific alternatives (options). the rough (shallow) approach to thinking and deciding often relies on basic perceptions prompting that alternative ia is generally ‘better’ than alternative ja , i, j 1, n , i j , n 1 , without conducting comparisons with regard to specific criteria or some additional details. this kind of approach does not always lead to a good choice. mailto:kdegtiarev@hse.ru mailto:michaelborisovha@gmail.com ijahp article: degtiarev, borisov/prioritization of alternatives based on analytic hierarchy process using interval type-2 fuzzy sets and probability-theoretical interval comparison international journal of the analytic hierarchy process 448 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.586 the analytic hierarchy process (ahp) proposed by thomas l. saaty enables decisionmakers to prioritize alternatives and make choices in favor of alternatives that correspond to understanding the problem and requirements imposed on its solution (saaty, 1980). this goal is achieved by defining a hierarchical model that reflects characteristic properties of the problem, determining criteria and alternatives in reference to the goal as the top element of the hierarchy (model), and prioritizing alternatives and criteria using pairwise comparisons. with reference to the “classical” 3-level hierarchy (goal [l0] – criteria [l1] – alternatives [l2]), the method allows the use of intuitive expert estimates that express the superiority of alternative ia over alternative ja , i, j 1, n , concerning the determined criteria. results of such comparisons are not always expressed in terms of quantitative characteristics; verbal constructs can be used instead. usually the intuitive evaluation is transformed by the expert to a corresponding number on saaty’s fundamental scale with integers from 1-9. the ahp calls for a series of pairwise comparisons of alternatives against a specified criteria set, and it provides for the construction of several positive reciprocal matrices m (matrices of overall priorities, mop). namely, they are criteria comparison matrices relative to a system’s objective and alternatives comparison matrices regarding each criterion on the list. the number of such matrices is determined by the number of constituents at levels l1 and l2 as well as the links between layers. for the sake of simplicity, any extra indices in matrix notations are omitted, if possible, in the following text. the maximum eigenvalue of the corresponding matrix, its order, the values of consistency index and tabulated random index are used to check the consistency of every constructed square matrix (saaty, 1980; saaty, 1987). once these potentially lengthy steps of ahp realization are over, we switch our attention to calculating weights of alternatives giving a proper account to the weights of criteria obtained earlier. weights of alternatives make it possible to rank them, and the alternative with the highest priority (rank) value is proclaimed as the most feasible decision option for the problem at hand. an extra valuable step, the sensitivity analysis, allows the slight changes of criteria weights to ranking of alternatives to be considered more fully. this step serves as a base to formulate comments on those details of the model that are worthy of notice in making the ultimate decision when considering results of calculations (saaty, 1994a, 1994b). the expert usually follows a complex system of reasoning while making his/her assessment. it conforms to kahneman’s system 2 of thinking that requires mental efforts, deep analysis and attention (kahneman, 2013). as a rule, estimates are formulated in terms of natural language statements, i.e. words, phrases or short sentences. this outcome can be mapped to some form that is reminiscent of numeric values of the fundamental scale, i.e. numbers ranging from 1 to 9 as an expression of the assessment done. however, when a person expresses judgments in the context of ‘ ia versus ja ’ comparisons using natural language statements, the latter can often be interpreted imprecisely due to inherent vagueness of the language constructs and specifics of information pieces captured from diverse sources. vagueness of lingual forms stands for the deficiency of meaning. its interpretation is usually multiplied by a human’s mental capacity, embracing beliefs, thoughts and sensations in the mind or collaborative environment of decision-making problems; it then becomes a matter of degree and leads to variations (fine, 1975; keefe, 2007). hence, fuzzy logic methodology as a modeling base can be suitable in such case(s). ijahp article: degtiarev, borisov/prioritization of alternatives based on analytic hierarchy process using interval type-2 fuzzy sets and probability-theoretical interval comparison international journal of the analytic hierarchy process 449 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.586 fuzzy analytic hierarchy process (fahp) can be considered as a development of the classical ahp that allows experts to operate with customary linguistic terms (e.g. strong predominance, highly preferred etc.) while performing pairwise comparisons which are not immediately converted into plain numeric representation form (kabir & hasin, 2011). to provide such opportunities to experts, calculations within fahp can be done using t1mf (zadeh, 1965). according to zadeh (1965), the concept of a fuzzy set is bound up with a way to “treat fuzziness in a quantitative manner”. let u (universal set) be a set of elements denoted generically as x, i.e. { }u x ; fuzzy set a u is a set of ordered pairs   a (x,μ (x)) , and the mapping a μ : x [0,1] is a (type-1) membership function (abbr. t1mf) of a fuzzy set a . we can simply write a : u [0,1] identifying the notion of the set with its representing function a μ . fuzzy set is represented by its membership function, and this fact makes two commonly employed abbreviations fs and mf interchangeable (zadeh, 1965; klir & yuan, 1995; wierman, 1997; mendel, 2017). value a μ (x) is a degree (or grade) of membership of x in the set a . convex normal (height equals to 1) fuzzy set defined on the real line is called a fuzzy number (abbr. fn). it is a generalization of the usual number commonly called crisp (wierman, 1997). type-1 fuzzy sets serve as a basic framework to handle vagueness that is typical of natural language statements (linguistic values) and the context of their use. thus, the saaty scale can be modernized in a way that expert(s) choose a linguistic term associated with a corresponding t1mf that best characterizes his/her assessment. as an example (figure 1), five type-1 symmetrical triangular membership functions that cover domain [1,9] u can be used (kabir & hasin, 2011). figure 1 possible form of modified fundamental (saaty) scale however, the fuzzy approach to ahp has its drawbacks (e.g. fuzzy numbers ranking that lacks prevailing view about its realization, disparity between fahp arithmetic operations and basic principles of ahp, checking validity of results obtained) and still cannot be considered an ideal approach (zhü, 2014). it’s worth noting that thomas l. saaty questioned unjustified fuzzification of ahp. in his reasoned opinion, he states that the numeric representation of pairwise comparison judgments embraces the “fuzziness” as it is a subject to uncertainty and posits that the fundamental scale in use is already fuzzy (saaty & tran, 2007). it is rather difficult to quarrel with such a stance, since the fuzzification of judgements blindly for the sake of improving consistency or effectiveness of decision-making results, “without giving good reasons for doing it”, does not seem ijahp article: degtiarev, borisov/prioritization of alternatives based on analytic hierarchy process using interval type-2 fuzzy sets and probability-theoretical interval comparison international journal of the analytic hierarchy process 450 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.586 sound. nevertheless, the idea of using a fuzzy logic approach as applied to ahp preserves the consideration of important practical nuances of expert judgements and forms of their expression used while filling in matrices m. in the author’s opinion, an important serious deficiency of fahp lies in the fact that the approach does not take into account the expert's confidence in the estimate he/she provides. it means that if one expert is confident about a specified k-th assessment in pairwise comparison (e.g. the opinion may look like a 2-tuple {“moderate importance”, “surely”}), while another one expresses certain doubt in the same case, the calculations within the fahp are made without taking such assurance levels into account. a vital question must be asked in view of this dilemma. what could be a way to give a proper weight to an expert's confidence (reliability) degree provided along with the main estimate (restriction on values of variable interest) in fahp? the main objective of the work is to develop the approach, which would make it possible to integrate an expert's confidence degree into the fahp model. consequently, the paper is organized as follows: analytic hierarchy process (ahp) and its fuzzy counterparts (transition from ahp to fahp) are considered in section 2. for each of those methods, drawbacks are identified and the need to develop a new approach is justified. section 3 discusses the potential of interval type-2 membership functions (it2mf) usage in fahp, corresponding calculations based on fuzzy synthetic extents (fse) and the probabilitytheoretical interval comparison approach following the ideas of chang (1996) and sevastyanov, et al. (2002). comparison of fahp utilizing t1mf and fahp using it2mf along with interval comparison constitutes the foundations of section 4. the example based on the numerical data from kabir & hasim (2011) is discussed here. necessary computational steps are performed both manually and with the assistance of a developed program (working prototype) ensuring the correctness of results obtained. final remarks concerning the approach and potential for its further development are drawn in the conclusion section. 2. transition from ahp to fahp – brief overview of approaches if experts express their opinions verbally (words or phrases in natural language), then these judgments are inherently inaccurate (vague); such a situation complicates the conversion of verbal estimates to specific integers ranging from 1 to 9. if we consider odd elements 1,3,5,7 and 9 as core values on the fundamental scale, the intermediate even values 2,4,6 and 8 can be considered by way of auxiliary ‘anchors’ for experts to adjust the mapping of verbal statement(s) to proper numeric label(s) (saaty & tran, 2007). such an opportunity seems practical, but another challenging problem emerges. calculations with values that fill in matrices m do not consider degrees of confidence that an expert may, and usually does, attach to judgments provided. a specialist normally supplements their opinion with remarks, which can be reduced to short linguistic labels like “almost sure”, “beyond a doubt”, “unsure at large”, etc. two expert opinions 1o and 2o can be identical ( 1 2 eq.o o o  ) in classical ahp, but the appearance of confidence (reliability) degrees 1c and 2c , 1 2c c , as constituents of expert judgments results in different estimates as 2-tuples  eq. 1,o c and  eq. 2,o c that must be addressed properly. ijahp article: degtiarev, borisov/prioritization of alternatives based on analytic hierarchy process using interval type-2 fuzzy sets and probability-theoretical interval comparison international journal of the analytic hierarchy process 451 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.586 2.1 fahp –use of type-1 membership functions if mop is filled in with fuzzy expert estimates (e.g. triangular t1mf), the basic question is closely connected with processing of fuzzy pairwise comparison matrices. working out approaches to potential computational schemes of fahp was not limited to isolated instances in the last few decades. for example, the fuzzy variant of saaty’s pairwise comparison extended by the graan-loostma method was proposed by van laarhoven and pedrycz (1983) to obtain fuzzy performance scores (and weights) from the matrix of fuzzy ratios. fuzzy opinions/comparisons were expected in the form of triangular fuzzy numbers (de graan, 1980; lootsma, 1980; van laarhoven & pedrycz, 1983). it was the first paper to address the use of fahp and it still attracts the attention of researchers today. according to google scholar, it was cited more than 2,660 times as of december 2018. the same approach to pairwise comparisons of triangular fuzzy numbers (tfn) was used by chang (1992, 1996). based on a fuzzy comparison matrix being filled with tfns, he used an extent analysis method to calculate crisp weights (priority vector was obtained on the base of arithmetic means). the rule to compare fuzzy numbers was considered in the paper providing a way to estimate the degree of possibility of 1 2a a ( 2 1a a ), where 1a and 2a are two tfns. within the framework of this approach, a set of sums of elements for each i-th row of matrix ij n n m m      ( n n n n i ij ij ij ij j 1 j 1 j 1 j 1 r , ,s m l c r                   , i 1, n ; ijl , ijc and ijr are left, center and right points defining (i,j)-th matrix element as piecewise linear function, correspondingly) is obtained first. after that, all calculated 3tuple values irs are normalized, i.e. fuzzy synthetic extents (fse) n i i i j 1 r rs s s    , i 1, n , as fuzzy variant of arithmetic means are calculated for each i-th row of m matrix (chang, 1996; wang et al., 2008; shapiro & koissi, 2017). normalized values are then used to determine degrees of possibility that one fse is greater than another one as follows (both is and js are in the form of tfn, indices i and j allow to distinguish between them in the formula, i, j 1, n ):   i j i j i j i i j j j i 1, if v ( ) (( ) ( )), if 0, otherwise c c s s r l r c c l l r            then, these degrees are compared with the purpose of obtaining for each is the smallest value  i jimv v j i, j 1, ns s |     utilized in calculating components of priority vector w (crisp weights n i i k k 1 mv mvw    ) for criteria (alternatives) under consideration. further material is built on the ideas presented by chang (1996). this paper also attracts a lot of attention (according to google scholar, it has been cited more than 3,390 times). however, certain issues concerning the application of extent analysis on fuzzy ahp were highlighted by wang et al. (2008) he stresses weights determined by the extent analysis method. he showed that in some cases calculated weights do not convey relative ijahp article: degtiarev, borisov/prioritization of alternatives based on analytic hierarchy process using interval type-2 fuzzy sets and probability-theoretical interval comparison international journal of the analytic hierarchy process 452 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.586 importance of criteria (alternatives). besides, the potentially possible appearance of zero weights associated with some criteria or alternatives leads to the exclusion of the latter from consideration in decision analysis. these aspects must definitely be placed among the limitations of chang’s model (shapiro & koissi, 2017). 2.2 fahp – gradual switch to interval type-2 membership functions the application potential of it2mf in both the ahp and anp have received the active attention of researchers during the last decade both in theory and in relation to solving applied problems (transportation, healthcare, vendor selection, infrastructure management, etc.). on the one hand, publications touch upon vital issues of uncertainty measures – “each of them is an interval, and the length of the interval is an indicator of uncertainty” – for it2fs, transformation of linguistic perceptions (words, phrases, etc.) into it2fs, and schemes of aggregation of subjective judgments (pre-ranked linguistic measures) in decision-making problems to name a few (wu & mendel, 2007b; mendel & wu, 2006; wu & mendel, 2007a). all those topics are of current importance when the matter concerns data preparation to fill in matrices m. we mean here elicitation of verbal opinions from n experts, their representation by way of it2mf and further aggregation to obtain the elements for processing under fahp. on the other hand, other authors carried out research in the broad field of it2mf-based fuzzy decision-making approaches that extend the classical bellman-zadeh (1970) model and take into account risk attitude, and utilize various ranking methods of it2fs in the decision making process (runkler et al., 2017; chen & lee, 2010; chen et al., 2012; abdullah & najib, 2014; chiao, 2016). it makes it clear that it2fs as an object of research gives ample scope for their use in different computational methods and their modifications aimed at producing decision priorities. for stakeholders involved in a given problem, the latter serves as a reference point and a topic for discussion when the ultimate decision is made. one important milestone in the development of fahp is the extension of hierarchical analysis to the use of fuzzy ratios in preference matrices that was considered by buckley (1985). the proposed model made it possible to capture the vague (imprecise) responses of experts in the form of fuzzy numbers, and it broadens the scope of saaty’s ahp while comparing pairs of alternatives. fuzzy weights of those alternatives are calculated based on the geometric mean method followed by a regular combination procedure that takes the hierarchical structure under consideration. it was the first time that burkley’s fuzzy ahp model was subsequently used and further extended by kahraman, öztayşi et al. (2014) to the case of it2fs filling in mop. their approach makes provision for the transition to a linguistic scale and the use of new defuzzification methods (dtrit and dtrat) for triangular and trapezoidal it2fs at certain computational steps. these methods relate to calculating defuzzified values (dvs) of fuzzy sets followed by ranking of the latter with respect to the dvs obtained. the ranking approach yields results that may differ from those ranks procured, for example, by means of likelihood method and arithmetic operations on it2fs (kahraman et al., 2014; chen & lee, 2010). it’s normal, since the ranking outcome depends on particularities of the approach in use. some of the approaches and their brief comparison can be found in chen & lee (2010) and chen and yang at al. (2012). the computational scheme proposed by kahraman et al. (2014) relates to direct manipulations with it2fs (formal representation of linguistic terms) that are elements of pairwise comparison matrices, use of defuzzified values to check the consistency of the ijahp article: degtiarev, borisov/prioritization of alternatives based on analytic hierarchy process using interval type-2 fuzzy sets and probability-theoretical interval comparison international journal of the analytic hierarchy process 453 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.586 matrix constructed and calculation of geometric means for each row of the comparison matrix with obtaining priority weights of criteria (alternatives). the global weights of alternatives are first calculated in the form of it2mf, then they are defuzzified to get normalized weights (crisp numbers) determining the final ranking of the alternatives (best option). the core word as applied to manipulations with it2fs under this method is “direct”. in the present paper, it2fs as aggregated expert estimates determined by lower and upper type-1 mfs are used in chang’s model (1992, 1996) which is expanded with the computationally efficient probability-theoretical approach to compare calculated fuzzy synthetic extents (fse). the results of comparison come to interval-based priorities (ranking) of alternatives. 2.3 fahp – most influential papers and the choice of chang’s model several models of fuzzy logic versions of ahp that underlie the previously mentioned approaches are discussed by shapiro and koissi (2017). these authors emphasize lootsma (1980), van laarhoven & pedrycz (1983), burkley (1985) and chang (1992, 1996) as the most influential fahp papers that laid the basis for publications that appeared from the 1990s until now. they advance arguments for the appropriateness of fuzzy versions of ahp incorporating the conclusions and observations drawn by several cited authors. in particular, they touch on issues of natural language statements used by decision makers, vagueness of human thoughts that cannot be modeled adequately by crisp numbers, inherent imprecision of pairwise comparisons, casting doubt on precise estimate values. moreover, each of publications by van laarhoven & pedrycz (1983), burkley (1985) and chang (1996) received practically 200 or more citations per year from 2012 2017 (ahmed & kılıç, 2018). as a result of the previous discussion, it is justified to ask whether or not it is advisable to choose the chang model as a basis for computations in the proposed approach. the choice can be defended for several reasons. first, the fuzzy extent analysis is one of the most commonly used and cited models in the last 5-6 years when considering the most influential fahp papers (ahmed & kılıç, 2015). in spite of its existing weak points, the citations of chang’s work totals 300-350 per year for this period of time which perceptibly exceeds indices of “competitors” (ahmed & kılıç, 2018). the fuzzy extent analysis is attractive because of its possible modifications that affect the effectiveness of decision making. ranking (pairwise comparison) of fuzzy sets can be done differently, alternative approaches to defuzzification as a significant constituent of chang’s method may also be studied in detail to contribute to such changes (ahmed & kılıç, 2015; chang, 1996). these reasons support the use of the given method now as well as support it as a subject of further consideration and improvement. secondly, the computational steps of the model are simple, and this fact can support its use by way of illustration of the method in the paper. also, chang’s method as any other fahp models, can be extended by type-2 fuzzy sets in different ways (here we mean processing steps as such). thus, it makes space for potentially extensive studies in the field of comparing results of various fahp methods enriched with interesting and wholesome t2fs formalism (kahraman et al., 2014). 2.4 fahp – z-numbers as a new trend in cww another attempt to add flexibility and an approach to language-oriented representation of expert estimates to the original ahp is the z-ahp approach. it is grounded on the concept of z-number proposed by zadeh as a new theoretic trend in the field of ijahp article: degtiarev, borisov/prioritization of alternatives based on analytic hierarchy process using interval type-2 fuzzy sets and probability-theoretical interval comparison international journal of the analytic hierarchy process 454 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.586 computing with words (cww) (zadeh, 2011). z-ahp is worth mentioning here as a topic of independent arresting research in the field of fuzzy logic; however, it is not covered in the present paper. the z-number concept is based on the important aspect of information’s (human assessment) reliability coupled with constraint on values of uncertain variables. a reliable summary (fuzzy granule) formalizes inherent uncertainty of information human’s deal with routinely (zadeh, 2011). z-number is viewed as a pair  a, b , where the first component a in the form of t1mf is a constraint on values of uncertain variable x u ( ). the second component ( b ) expresses reliability of a , i.e. the degree of confidence that a is exactly as specified. thus, z-number “absorbs” the ideological foundations of fuzzy logic, basics of natural language modeling and reliability of composed information granule(s). publications in the field of multicriteria decision making (mcdm) responded to the potential of z-number application in well-known methods (e.g. ahp, topsis, vikor, todim); however, they basically provide for conversion of z-numbers to classical fuzzy numbers (azadeh et al., 2013; kang et al., 2012; zadeh, 2011; yaakob & gegov, 2016; kang et al., 2018; krohling et al., 2018). for example, azadeh et al. (2013) considered the z-ahp approach built upon initial conversion of z-numbers to regular fn characterized by t1mf. such conversion was originally proposed by kang and wei, et al. (2012) under preserving fuzzy expectation of fuzzy set in view. the conversion simplifies the processing phases, but it also alters fuzzy granules summarized in initial expert estimates. currently, the name z-ahp virtually implies fahp because of the conversion stage that ‘adds’ the weight of the second part ( b ) of the z-number to its first part ( a ). in that sense, z-ahp as it is presented in many publications endeavors to escape at the early stages from its z- prefix in name and aims at being considered as one of the fahp approaches that makes use of t1fs. 3. it2mf in fahp – how do we come to interval comparison? we pursue an object to develop a model that implicitly takes into account constraints on an expert's assessments as they can be supplied with certain degrees of confidence. type2 membership functions (t2mf) can be considered an option here; however, the lack of available data needed to construct them can be a problem. despite the conceptual appeal, t2mf requires more lengthy computations in comparison with type-1 fuzzy sets. therefore, it2mf does not have the processing complexity associated with generalized t2mf and still provides a sufficient degree of flexibility. they are widely used in practical problems and make it possible to reflect (model) linguistic uncertainty adequately (mendel et al., 2006; gimaletdinova & degtiarev, 2017). it2mf are characterized by constant secondary membership functions for every parameter x u (x denotes a primary variable). the matter concerns type-2 function μ(x,u) , where x u , xu j [0,1]  (u as a secondary variable has domain xj as a primary membership of x), 0 μ(x,u) 1  , and all μ(x,u) , secondary grades of it2mf, are equal to 1 (mendel & john, 2002). more details and comments on it2mf can be found in mendel, john & liu (2006), wu & mendel (2007b), mendel (2017), and mendel et al. (n.d.). ijahp article: degtiarev, borisov/prioritization of alternatives based on analytic hierarchy process using interval type-2 fuzzy sets and probability-theoretical interval comparison international journal of the analytic hierarchy process 455 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.586 the statement xj [0,1] merits special attention as it is considered at length in bustince, fernandez, hagras et al. (2015) in the context of four special interpretative cases, i.e. multisets, combination of numbers and intervals, interval-valued fuzzy set (ivfs) and multi-ivfs. the theory of it2fs and their formal processing (operational base) proceed from the interpretation of xj under identity it2fs = ivfs (mendel, hagras et al., 2016). the intrinsic potential of the relationship between ivfs and it2fs requires close attention and further study. in the present paper, the definition of xj as ivfs, i.e.  (x, u) | u [μ(x),μ(x)] , where μ(x) and μ(x) are lower and upper membership functions of fs, correspondingly, is used without any alterations. therefore, the abbreviation ‘it2fs’ in the sense of ivfs prevails throughout the text. 3.1 it2mf can we link lmf and umf with confidence of estimates? lower and upper type-1 membership functions (lmf and umf) of some it2mf a play a role of delimiters of the footprint of uncertainty (fou) area as a union of all vertical slices of type-2 function representing memberships (intervals) for all x u , x x u fou(a) j   is the region enclosed by lmf and umf of a . these intervals, or more specifically, their alterable widths, may represent degrees of an expert’s confidence in the estimates provided. the wider the corresponding interval is, the lesser degree of confidence it manifests. more narrow intervals correspond to a greater (proportionally) degree of confidence. based on these degrees, the prototype of the original fahp approach can be developed. at each point x u , values of lmf and umf functions serve in the capacity of expert's estimate. the width of the vertical interval bounded by upper and lower mf values can be used to express the expert’s confidence level. we accept the premise that mop m is filled in with expert estimates in the form of it2mfs (results of aggregation of individual measures obtained from expert group). these functions must be compared, and the possibility degree that one of the functions is bigger than the other one steps aside from ordinary number towards an interval owing to the presence of lmf and umf. to ensure computational efficiency, the preference is given to piecewise linear functions (triangular shape) as upper and lower mfs of it2mf. following chang’s model (1992, 1996) and performing calculations of fuzzy synthetic extents (fse) for matrix m (generalized notation introduced earlier) filled in with it2mfs with further comparison of the latter, we should realize that degrees of possibility that is is greater than js expressed by ordinary numbers will turn into intervals due to the fact that it2mf is bounded by upper and lower t1mfs. we must consider the intersections of upper and lower membership functions (umf and lmf) separately to obtain resultant intervals. operations of addition, subtraction, multiplication and division as applied to two intervals denoted as  1 2a a , a and  1 2b b , b can be expressed as follows: 1.      1 2 1 2 1 1 2 2a b a ,a b , b a b ,a b      2.      1 2 1 2 1 2 2 1a b a ,a b , b a b ,a b      3.    1 1 1 2 2 1 2 2 1 1 1 2 2 1 2 2a * b min a * b , a * b , a * b , a * b , max a * b , a * b , a * b , a * b    4.    1 1 1 2 2 1 2 2 1 1 1 2 2 1 2 2a / b min a / b , a / b , a / b , a / b , max a / b , a / b , a / b , a / b    , ijahp article: degtiarev, borisov/prioritization of alternatives based on analytic hierarchy process using interval type-2 fuzzy sets and probability-theoretical interval comparison international journal of the analytic hierarchy process 456 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.586 where 1a and 2a are left and right boundaries of the first interval, 1b and 2b – left and right boundaries of the 2 nd interval at hand. 3.2 can comparison of intervals be used in it2mf-based ahp? to derive the weight vector by the algorithm proposed by chang (1996), we need to find the minimal possibility degree reflecting the fact that one fse is greater than another one for each row of matrix m. since these degrees are represented by intervals, it requires some effort to find the smallest of them. in the case of two intervals, the smallest one can easily be uncovered only if the right boundary of one interval is less than the left boundary of the second interval. in all other cases, different approaches to comparison can be utilized; in the framework of the present paper, the algorithm proposed by sevastyanov et al. (2002) is considered (sevastianov, 2007). to summarize the ideas that can be put into practice, assume that there are two independent intervals  1 2a a , a ,  1 2b b , b , and independent random variables aa  and bb having uniform distribution over intervals a and b. if corresponding intervals intersect, it leads to the appearance of subintervals being a subject of further study. let the event kh be such that aa  , bb , where a and b are subintervals of a and b, correspondingly. all a constitutes interval a, and the same can be said about the totality of subintervals b with reference to b. probability  kp h can be calculated based on the reasoning from geometric considerations due to the aforesaid distribution of a and b . the following formula was proposed by sevastyanov et al. (2002) to compare intervals, i.e.       n k k k 1 p b a p h p b a h      , where a and b are two independent intervals, kh is the event related to variables a and b falling into corresponding subintervals, n (upper parameter in summation) is the number of such events. for example, consider the following events kh , k 1, 4 , for intersecting intervals a and b (figure 2):    1 1 1 1 2h a , b b ,a a b      2 1 1 2 2h a , b a , b a b      3 1 2 1 2h b , a b , a a b      4 1 2 2 2h b , a a , b a b   thus, p() values can be expressed as follows:      1 1 1 2 1 2 1 2 1p h (b a ) (a a ) * (a b ) (b b )     ,      2 1 1 2 1 2 2 2 1p h (b a ) (a a ) * (b a ) (b b )     ,      3 2 1 2 1 2 1 2 1p h (a b ) (a a ) * (a b ) (b b )     and      4 2 1 2 1 2 2 2 1p h (a b ) (a a ) * (b a ) (b b )     , i.e.  kp b a h 1   , k 3 ; if k 3 , corresponding p( ) 0.5  . ijahp article: degtiarev, borisov/prioritization of alternatives based on analytic hierarchy process using interval type-2 fuzzy sets and probability-theoretical interval comparison international journal of the analytic hierarchy process 457 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.586 figure 2 schematic representation of intersection of intervals a and b the result of calculations is the probability value to be interpreted so that if it is greater than a certain threshold value θ, one interval is considered as being bigger than the other one (sevastyanov et al., 2002). the choice of this threshold value depends on the problem being solved, and in certain situations it makes sense to take a value close to 0.95 (as an example of a rather big value that is approaching 1). if no special restrictions are imposed, it is suggested to take a value greater than 0.5 (sevastyanov et al., 2002). is it possible to unequivocally accept this advice while dealing with modeling fuzziness in fahp? most likely, the value of 0.5 can be accepted as a primary general reference point, but the answer to the question raised relates, in the first place, to peculiarities of natural language statements in use and the task (problem) per se. responses obtained by way of such constructs (words, phrases, short sentences) bear the tinge of vagueness and inaccuracy. it is almost impossible to specify exactly the “optimal” deviation from the 0.5 landmark (apparently towards bigger values) to derive the threshold value applicable to the case at hand. it means that such a value can be uncovered based on empirical studies only. results of several conducted experiments (attainment of identical outcomes by means of fahp based on using t1mf and the proposed approach covered by the paper) allowed the threshold to be fixed to a value θ on average at the level of appr. 0.83; several tests ensured values in the range [0.64,0.96]θ . thus, it meets the previously mentioned suggestion (the value exceeding 0.5) and does not look like an excessively big one. naturally, we may consider such an estimate as tentative (rough) and a purely empirical trial that may require subsequent refinement, but now it entirely suits us as a landmark to use in the following steps of the approach. on the other hand, the value of 0.83 corresponds to the case of coincidence of results assured by basic fahp utilizing t1mf and the proposed approach. coincidence is not just idee fixe here, it simply shows that under a certain value of threshold we may get the same results (in terms of final ranking) when different approaches are used. the choice of threshold at intermediate computational steps adds extra flexibility in terms of perception of attained results allowing the human factor to show itself not only in fuzzification of linguistic data. having at our disposal the method for comparing fse, calculations over matrix m can be performed in the context of fahp with recourse to it2mf. however, one important question remains. how do we fill in cells of matrix m ? in the case of t1mf-based fahp, a fuzzy scale based on t1mf (extension of the fundamental scale) is used, and it does not consider the degree of confidence of assessments experts provide. the latter seems applicable in our case, although it requires some changes of the scale subject to expressed degrees of confidence. for instance, as was mentioned above, depending on the degree of confidence expressed by experts, we may transform the upper and lower boundaries into intervals proceeding from the fact that the higher the degree of confidence is, the narrower the interval becomes. thus, we may introduce a fuzzy ijahp article: degtiarev, borisov/prioritization of alternatives based on analytic hierarchy process using interval type-2 fuzzy sets and probability-theoretical interval comparison international journal of the analytic hierarchy process 458 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.586 expert’s confidence scale that bears a stark analogy with the scale shown in figure 1. membership functions 1 5a a shown can be associated from left to right with linguistic labels “not sure at all” ( 1a ), “not very sure” ( 2a ), “surely” (confident/certain, 3a ), “very sure” ( 4a ) and “absolutely sure” ( 5a ). a symmetrical case of membership functions is easier in the view of processing. however, it is not a mandatory requirement, and it can be tempered at any time. the starting point for t1mf-to-it2mf conversion can be presented as follows: as a natural conviction, the estimate from the base scale corresponds to the average degree of confidence (linguistic term “surely”). to do this, we also need to choose the rule to convert t1mf to it2mf. for this purpose, a series of experiments was carried out to recognize that it is appropriate to convert upper and lower boundaries to interval 0.03  units long being symmetrically located relative to original upper and lower bounds. for example, consider the estimate (2.02, 3.08, 4.64) representing t1mf in the context of basic fahp. to convert it to the expert's estimate (opinion) with a confidence degree “surely” (“i’m sure”), we perform the transformation described above to obtain realistic enough albeit empirically specified boundaries ((2.005, 2.035), 3.08, (4.625, 4.655)) of it2mf. on the condition that the threshold value θ equals 0.83, experimental studies reveal almost identical results obtained by it2mf-based fahp and ‘original’ fahp that uses t1mf. however, it should be emphasized that the actual choice of “conversion” value(s) is a topic for rapt attention and further study because it is closely bound up with distinctive features of the problem under consideration, preferred methods to aggregate expert opinions to obtain it2mf and other substantial factors. essentially, such value(s) must be determined, but not just chosen to minimize the impact of the human factor and possible biases in favor of various inexplicable reasons. to use other linguistic terms as degrees of expert’s confidence (e.g. 1 pa a , where 'magic' number p equals 7 2 ), defuzzification of fuzzy numbers representing linguistic terms on a fuzzy confidence scale can be done to calculate the ratio of these numbers. depending on the degree of confidence, the interval covered by it2mf should be either expanded or grow narrower in accord with the ratio of defuzzified values of fns on the confidence scale. 4. comparison of fahp based on t1mf and fahp using it2mf to compare the proposed modification of fahp (pmfahp) with the already existing version(s), we consider the example used by kabir and hasim (2011). the authors compare results of ahp and fahp. by the same example, we will in turn examine the results provided by basic fahp (the one that uses t1mf (tfn) as expert estimates) and pmfahp. the following data table (matrix ij[ ]m m that is filled in with triangular fns represents aggregated results as provided by fourteen domain experts and corresponding reciprocals) is used in the example (kabir & hasim, 2011). ijahp article: degtiarev, borisov/prioritization of alternatives based on analytic hierarchy process using interval type-2 fuzzy sets and probability-theoretical interval comparison international journal of the analytic hierarchy process 459 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.586 table 1 fuzzy matrix m of attribute comparison attributes unit price annual demand criticality last use date durability unit price 1.0, 1.0, 1.0 0.89, 1.6, 2.25 0.65, 1.07, 1.88 0.82, 1.47, 2.76 0.8, 1.37, 3.19 annual demand 0.44, 0.62, 1.12 1.0, 1.0, 1.0 2.02, 3.08, 4.64 0.8, 1, 1.47 1.17, 2.36, 4.53 criticality 0.53, 0.93, 1.53 0.22, 0.34, 0.5 1.0, 1.0, 1.0 0.68, 1.11, 1.66 0.8, 1, 1.72 last use date 0.36, 0.68, 1.21 0.68, 1, 1.26 0.6, 0.9, 1.47 1.0, 1.0, 1.0 0.76, 0.93, 1.25 durability 0.31, 0.73, 1.26 0.22, 0.42, 0.86 0.58, 1, 1.26 0.8, 1.08, 1.32 1.0, 1.0, 1.0 fuzzy numbers ijm summarized in table 1 can be represented as ij ij ij ij(α β ,β δ ) , where ij ij ij0 α β δ   (buckley, 1985). for example, tfn 12 (0.89,1.6, 2.25)m  may be written by way of pair (0.89 1.6,1.6 2.25) , and 1 12m  becomes approximately 1 1 1 1 12 12 12 12(δ β ,β α )     , i.e. 21 (0.44 0.62, 0.62 1.12)m  . the fuzzy matrix m can be expressed as a totality of three crisp matrices lm (left), cm (center) and rm (right) by the number of parameters defining each tfn. it is shown by buckley (1985) that a fuzzy positive reciprocal matrix m is consistent if and only if cm is consistent (reduction to classical hierarchical analysis). with reference to cm matrix, the principal eigenvalue maxλ 5.27 , consistency index ci 0.068 and consistency ratio cr ci ri 0.06  (under (n 5)ri 1.12  ) point to the conclusion that matrix cm (the same as matrix m ) is acceptably consistent (saaty, 1980; saaty, 1987). based on the matrix m shown above, fse are calculated first, i.e.  u (4.16, 6.51,11.08) (1 42.14,1 27.68,1 19.13) 0.09,0.235,0.58s    ,  a (5.43,8.06,12.76) (1 42.14,1 27.68,1 19.13) 0.13,0.291,0.67s    ,  c (3.23, 4.38, 6.41) (1 42.14,1 27.68,1 19.13) 0.077,0.158,0.34s    ,  l (3.4, 4.51,6.19) (1 42.14,1 27.68,1 19.13) 0.08,0.163,0.32s    ,  d (2.91, 4.23,5.7) (1 42.14,1 27.68,1 19.13) 0.07,0.153,0.3s    . thereafter, probabilities that one fse is greater than another one for each fse concerned are as follows: ijahp article: degtiarev, borisov/prioritization of alternatives based on analytic hierarchy process using interval type-2 fuzzy sets and probability-theoretical interval comparison international journal of the analytic hierarchy process 460 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.586         u a u c u l u d p 0.9 p 1 p 1 p 1 s s s s s s s s                 a u a c a l a d p 1 p 1 p 1 p 1 s s s s s s s s                 c u c a c l c d p 0.75 p 0.61 p 0.98 p 1 s s s s s s s s                 l u l a l c l d p 0.75 p 0.6 p 1.0 p 1 s s s s s s s s         (1)         d u d a d c d l p 0.7 p 0.55 p 0.98 p 0.96 s s s s s s s s         for each ( )s in the results of comparisons (1), we choose the smallest value and placed it into the vector of weights w. for the example under view, a 5-component (by the number of attributes used in matrix m) vector     t t 1 5,..., 0.9,1.0, 0.61, 0.6, 0.55w w w  is obtained. after normalization it takes the form of     t tнорм н н 1 5,..., 0.246, 0.273, 0.167, 0.164, 0.15w w w  (2) vector (2) expresses preferences of experts summarized in the assessment provided. in this case, attributes can be enumerated in descending order of preference (priorities). annual demand (a), unit price (u), criticality (c), last use date (l) and durability (d) constitute a chain a u c l d with a having the highest preference (27.3%). weights of alternatives c and l are barely discernible, and this issue should be addressed by analysts separately. the names of attributes used here are the same as those used by kabir and hasim (2011). what can be said about results of the proposed approach? as we use the same reference table 1 and since we have no information on the degree of certainty (confidence), to which he evaluations are associated with, we may naturally assume that these estimates imply an average degree of certainty. it seems that it is a natural conviction, i.e. a group of experts confirm the assessment (aggregated it2mf) with the label/degree “surely”. thus, matrix mmod takes the following form: ijahp article: degtiarev, borisov/prioritization of alternatives based on analytic hierarchy process using interval type-2 fuzzy sets and probability-theoretical interval comparison international journal of the analytic hierarchy process 461 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.586 table 2 modified fuzzy matrix mmod of attribute comparison attributes unit price annual demand criticality last use date durability unit price (1.0, 1.0), 1.0, (1.0, 1.0) (0.875, 0.905), 1.6, (2.235, 2.265) (0.635, 0.665), 1.07, (1.865, 1.895) (0.805, 0.835), 1.47, (2.745, 2.775) (0.785, 0.815), 1.37, (3.175, 3.205) annual demand (0.441, 0.447), 0.625, (1.104, 1.142) (1.0, 1.0), 1.0, (1.0, 1.0) (2.005, 2.035), 3.08, (4.625, 4.655) (0.785, 0.815), 1.0, (1.455, 1.485) (1.155, 1.185), 2.36, (4.515, 4.545) criticality (0.527, 0.536), 0.934, (1.503, 1.574) (0.214, 0.216), 0.324, (0.491, 0.498) (1.0, 1.0), 1.0, (1.0, 1.0) (0.665, 0.695), 1.11, (1.645, 1.675) (0.785, 0.815), 1.0, (1.705, 1.735) last use date (0.36, 0.364), 0.68, (1.197, 1.242) (0.673, 0.687), 1.0, (1.226, 1.273) (0.597, 0.607), 0.9, (1.438, 1.503) (1.0, 1.0), 1.0, (1.0, 1.0) (0.745, 0.775), 0.93, (1.235, 1.265) durability (0.307, 0.314), 0.729, (1.226, 1.273) (0.22, 0.221), 0.423, (0.843, 0.865) (0.576, 0.586), 1.0, (1.226, 1.273) (0.79, 0.803), 1.075, (1.29, 1.342) (1.0, 1.0), 1.0, (1.0, 1.0) to check the consistency of the matrix mmod, the defuzzified triangular type-2 fuzzy set (dtrit) approach can be applied without concern for the threshold value θ (kahraman et al., 2014). for all elements of the modified matrix the maximum membership degree α of lmf equals one, and, for example, the defuzzified value (1.58) of 12m is calculated as     2.265 0.875 1.6 0.875 3 0.875 2.235 0.905 1.6 0.905 3 0.905 2         . the principal eigenvalue maxλ with reference to the crisp matrix with values defuzzified in such a way equals 5.57, ci 0.142 , thus bringing cr to exceed the 0.1 guiding line (appr. 0.127). type-2 defuzzification methods lead to obtaining non-fuzzy (crisp) values, and alternatively interval type-2 defuzzification approaches can be used to convert lmf and umf of it2mf to a type-1 membership function. such transformation is called a type reduction technique (mendel, 2017; runkler et al., 2018). in particular, computationally efficient defuzzication that uses the vertical slice representation of it2mf can be suggested for use (nie & tan, 2008). for instance, the element 21m of mmod matrix results in t1mf (0.444, 0.625, 1.123), i.e. for each point x u the method ijahp article: degtiarev, borisov/prioritization of alternatives based on analytic hierarchy process using interval type-2 fuzzy sets and probability-theoretical interval comparison international journal of the analytic hierarchy process 462 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.586 involves calculation of the mean of lmf and umf. the subsequent computational steps are related to consistency checking on the basis of derived tfn. proceeding from the close resemblance between defuzzified tfn from it2mf (table 2) and t1mf of the first model (fuzzy matrix m, table 1) as well as the use of θ threshold, matrix mmod is rated as acceptably consistent ( cr 0.061 ). based on values summarized in table 2, fse are calculated on the assumption that in this case it2mf constitute elements of the modified matrix: u (0.096, 0.101), 0.235 (0.57, 0.59)s ,  a (0.126, 0.131), 0.291 (0.656, 0.676)s ,  c (0.075, 0.078), 0.157 (0.328, 0.342)s ,  l (0.079, 0.082), 0.162 (0.315, 0.331)s ,  d (0.068, 0.07), 0.152 (0.289, 0.303)s ,  also, probabilities that one fse is greater than another one for each fse become as                 u a u c u l u d p 0.886, 0.891 p 1.0,1.0 p 1.0,1.0 p 1.0,1.0 s s s s s s s s                         a u a c a l a d p 1.0,1.0 p 1.0,1.0 p 1.0,1.0 p 1.0,1.0 s s s s s s s s                         c u c a c l c d p 0.746, 0.76 p 0.595, 0.617 p 0.979, 0.98 p 1.0,1.0 s s s s s s s s         (3)                 l u l a l c l d p 0.748, 0.765 p 0.589, 0.614 p 1.0,1.0 p 1.0,1.0 s s s s s s s s                         d u d a d c d l p 0.695, 0.715 p 0.532, 0.561 p 0.976, 0.978 p 0.953, 0.956 s s s s s s s s         in each case of intervals (3) we choose the lowest possible calculated values which define the bounds of intervals (sevastyanov, rog & venberg, 2002; sevastianov, 2007). thus, the procedure results in the interval-based vector              tt 1 5,..., 0.886, 0.891 , 1.0,1.0 , 0.595, 0.617 , 0.589, 0.614 , 0.532, 0.561w w w = . after normalization of upper and lower bounds separately it leads to obtaining vector with five interval-based elements, i.e.             t t норм н н 1 5 0.242, 0, 245 , 0.271, 0.277 , 0.165, 0.167 , ,..., 0.163, 0.166 , 0.147, 0.152 w w w           (4) as it is seen from equation 4, the attributes form a descending order preference list a u c l d that under the threshold value of 0.83 is completely equal to the same list (chain) obtained earlier. the weights-intervals of c and l intersect, but the one for alternative c is located slightly to the right of н 4w (it corresponds to l). it can be noted that the number of operations needed to perform the later version of the computational ijahp article: degtiarev, borisov/prioritization of alternatives based on analytic hierarchy process using interval type-2 fuzzy sets and probability-theoretical interval comparison international journal of the analytic hierarchy process 463 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.586 procedure as well as complexity of realization are not strongly affected. however, the flexibility of the approach becomes more apparent due to taking the degree of confidence of the expert(s) into account. besides, ranking in the form of intervals gives an extra degree of freedom to decision-makers. discussions with stakeholders can be rather tough in many practical cases, many arguments that are disregarded in the model for objective reasons (complexity of situation, information deficiency, diverse views, etc.) can be considered during discussions and consultations in the context of possible deviations from crisp values of priorities. the width of intervals makes it possible to add more conscious flexibility to the process of final decision’s elaboration. the prototype of the program implemented in java 1.8 using javafx graphics packages and jfoenix (javafx material design library) gives an opportunity to verify the computational aspects of the proposed approach and to perform several experiments on both different and similar, although slightly altered, sets of data. when the program runs, the user as a decision-maker (expert) can choose his/her assessments and degrees of confidence from pre-defined lists (the latter can be modified in case of need) while filling in cells of matrices m at both levels l1 and l2 (criteria and alternatives). besides, parameters of t1mfs associated with corresponding verbal labels can also be tuned as needed. the program converts given t1mfs to it2mfs, performs calculations described above to generate final ranking of alternatives and displays results in a handy form of a bar graph. 5. conclusion the analytic hierarchy process (ahp) is a widespread approach in decision making based on pairwise comparisons that implicitly consider both quantitative and qualitative characteristics of compared elements of the system’s hierarchical model. in general, fuzzification of ahp makes it possible to deal with expert’s imprecise estimates; however, in practice degrees of confidence that are normally attached to almost all judgments expressed by humans are those features that differentiate results of comparisons in matrices m. in other words, the same hypothetical estimate (opinion) 1o with different degrees of confidence 1c and 2c must be perceived as two different information granules 1 1( , )o c and 1 2( , )o c to be considered and processed accordingly. such a view gives an impetus to the development of the fahp approach (its prototype), which allows the mentioned trait to be taken into account. in the framework of the present paper, the original approach to fahp is discussed. it handles not only fuzzy expert estimates, but also fuzzy degrees of confidence associated with them. fuzzy logic creates grounds for modeling estimates expressed by vague words and statements in natural language. ideas summarized in the paper are also implemented in a java program whose interface (in russian) allows one to choose a working project, specify/modify criteria and alternatives used in the project and graphically display the results (ranking of alternatives) of the algorithm. the relative transparency of primordial estimates specified in natural language constructs, ease of their use, cognitive clearness of possible changes in matrices m as well as graphical representation form of prioritization results become highly imperative during group discussions with stakeholders held while solving “soft” problems, i.e. problems, in which the human factor and verbal descriptive gestalts are the core dominants. ijahp article: degtiarev, borisov/prioritization of alternatives based on analytic hierarchy process using interval type-2 fuzzy sets and probability-theoretical interval comparison international journal of the analytic hierarchy process 464 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.586 the proposed approach also allows expert assessments to be handled i j( , )o c represented by it2mfs under different threshold values θ. on the one hand, there is a possibility to tweak it neatly to the peculiarities of the task at hand. on the other hand, the choice of varying θ values permits the iterative realization of fahp for the purpose of thorough understanding of the resultant prioritization of alternatives (tendencies observed, agility of interval’s numerical bounds, etc.) within the all-around system analysis required in any sound decision-making problem. further research into the role of θ and δ parameters as well as fine tuning of their values also merits a detailed consideration. fahp based on using it2mf and a probability-theoretical approach to interval comparison does not seem like an unduly overburdened variant of ahp with no practical purport. finally, experts get an opportunity to express their estimates (opinions) in customary and handy verbal forms that are transparently modeled for use at further computational steps. we are fully aware of the fact that this fuzzy approach to ahp in the present state can be seen only as a prelude to its onward development. it refers mainly to formal representation of individual expert assessments by way of it2mf (judgements as such jointly with confidence degrees) and computations that operate on such fuzzy information granules. aggregation of individual it2mf as expert’s estimates (opinions), the use of different defuzzification approaches, modifications of chang’s model or the choice in favor of alternative computational schemes in the capacity of the core of it2mf-based fahp are those aspects and topics that bring specificity to further development of the method covered by the paper. in our opinion, interval-based computations also have a good expansion potential here. along with the use of threshold values they can both be elaborated to arrive at a fahp computational method that is comprehensible by all stakeholders and customizable. ijahp article: degtiarev, borisov/prioritization of alternatives based on analytic hierarchy process using interval type-2 fuzzy sets and probability-theoretical interval comparison international journal of the analytic hierarchy process 465 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.586 references abdullah, l., & najib, l. (2014). a new type-2 fuzzy set of linguistic variables for the fuzzy analytic hierarchy process. expert systems with applications, 41(7), 3297 – 3305. doi: https://doi.org/10.1016/j.eswa.2013.11.028 ahmed, f., & kılıç, k. (2015). modification to fuzzy extent analysis method and its performance analysis. proceedings of the 6 th international conference on industrial engineering and systems management (iesm), 435 – 438. doi: https://doi.org/10.1109/iesm.2015.7380193 ahmed, f., & kılıç, k. (2018). fuzzy analytic hierarchy process: a performance analysis of different algorithms. fuzzy sets and systems, doi: https://doi.org/10.1016/j.fss.2018.08.009. azadeh, a., saberi, m., atashbar, n.z., chang, e., & pazhoheshfar, p. (2013). z-ahp: a z-number extension of fuzzy analytical hierarchy process. proceedings of the 7 th ieee international conference on digital ecosystems and technologies, 141–147. doi: https://doi.org/10.1109/dest.2013.6611344 bellman, r., & zadeh, l.a. (1970). decision-making in fuzzy environment. management science, 17(4), b141 – b164. doi: https://doi.org/10.1287/mnsc.17.4.b141 buckley, j.j. (1985). fuzzy hierarchical analysis, fuzzy sets and systems, 17, 233 – 247. doi: https://doi.org/10.1016/0165-0114(85)90090-9 bustince, h., fernandez, j., hagras, h., herrera, f., pagola, m., & barrenechea, e. (2015). interval type-2 fuzzy sets are generalization of interval-valued fuzzy sets: towards a wider view on their relationship. ieee trans. on fuzzy systems, 23(5), 1876 – 1882. doi: https://doi.org/10.1109/tfuzz.2014.2362149 chang, d.y. (1992). extent analysis and synthetic decision. optimization techn. and applications, 1, 352 – 355. chang, d.y. (1996). applications of the extent analysis method on fuzzy ahp. european journal of operational research, 95(3), 649 – 655. doi: https://doi.org/10.1016/0377-2217(95)00300-2 chen, s.m., & lee, l.-w. (2010). fuzzy multiple attributes group decision-making based on the ranking values and the arithmetic operations of interval type-2 fuzzy sets. expert systems with applications, 37, 824 – 833. doi: https://doi.org/10.1016/j.eswa.2009.06.094 chen, s.-m., yang, m.-w., lee, l.-w., & yang, s.-w. (2012). fuzzy multiple attributes group decision-making based on ranking interval type-2 fuzzy sets. expert systems with applications, 39, 5295 – 5308. doi: https://doi.org/10.1016/j.eswa.2011.11.008 chiao, k.-p. (2016). the multi-criteria group decision making methodology using type-2 fuzzy linguistic judgments. applied soft computing, 49, 189 – 211. doi: https://doi.org/10.1016/j.asoc.2016.07.050 ijahp article: degtiarev, borisov/prioritization of alternatives based on analytic hierarchy process using interval type-2 fuzzy sets and probability-theoretical interval comparison international journal of the analytic hierarchy process 466 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.586 fine, k. (1975). vagueness, truth and logic. synthese, 30(3-4), 265 – 300. doi: https://doi.org/10.1007/bf00485047 gimaletdinova, a.r., & degtiarev, k.y. (2017). type-2 fuzzy rule-based model of urban metro positioning service. proceedings of the institute for system programming (isp ras), 29(4), 87 – 106. doi: 10.15514/ispras-2017-29(4)-6 de graan, j.g. (1980). extensions of the multiple criteria analysis method of t. l. saaty. tech. report 80-3. leidschendam, the netherlands : national institute for water supply,. kabir, g., & hasin, m.a.a. (2011). comparative analysis of ahp and fuzzy ahp models for multicriteria inventory classification. international journal of fuzzy logic systems (ijfls), 1(1), 1 – 16. kahneman, d. (2013). thinking, fast and slow. new york: farrar, straus and giroux. kahraman, c., öztayşi, b., sarı, i.u., & turanoğlu, e. (2014). fuzzy analytic hierarchy process with interval type-2 fuzzy sets. knowledge-based systems, 59, 48 – 57. doi: https://doi.org/10.1016/j.knosys.2014.02.001 kang, b., deng y., & sadiq r. (2018). total utility of z-number. applied intelligence, 48(3), 703 – 729. doi: https://doi.org/10.1007/s10489-017-1001-5 kang, b., wei, d., li, y., & deng, y. (2012). a method of converting z-number to classical fuzzy number. journal of information and computational science, 9(3), 703 – 709. keefe, r. (2007). theories of vagueness. cambridge, uk: cambridge university press. klir, g.j., & yuan, b. (1995). fuzzy sets and fuzzy logic: theory and applications. upper saddle river, new jersey: prentice hall. doi: https://doi.org/10.1080/03081079708945184 krohling, r., pacheco a.g.c., & dos santos, g.a. (2017). todim and topsis with znumbers. in pan, y. lu, x. (eds). frontiers of information technology & electronic engineering. zhejiang university press & springer. van laarhoven, p.j.m., & pedrycz, w. (1983). a fuzzy extension of saaty’s priority theory. fuzzy sets and systems, 11(1-3), 229 – 241. doi: https://doi.org/10.1016/s01650114(83)80082-7 lootsma, f.a. (1980). saaty’s priority theory and the nomination of a senior professor in operations research. european journal of operational research, 4(6), 380 – 388. doi: https://doi.org/10.1016/0377-2217(80)90189-7 mendel, j.m. (2017). uncertain rule-based fuzzy systems: introduction and new directions, 2 nd ed. springer international publishing. doi: https://doi.org/10.1007/978-3319-51370-6_1 ijahp article: degtiarev, borisov/prioritization of alternatives based on analytic hierarchy process using interval type-2 fuzzy sets and probability-theoretical interval comparison international journal of the analytic hierarchy process 467 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.586 mendel, j.m., & john, r.i. (2002). type-2 fuzzy sets made simple, ieee transactions on fuzzy systems, 10(2), 117 – 127. doi: https://doi.org/10.1109/91.995115 mendel, j.m., john, r.i. & liu, f. (2006). interval type-2 fuzzy logic systems made simple. ieee transactions on fuzzy systems, 14(6), 808 – 821. doi: https://doi.org/10.1109/tfuzz.2006.879986 mendel, j.m., hagras, h., bustince, h., & herrera, f. (2016). comments on “interval type-2 fuzzy sets are generalization of interval-valued fuzzy sets: towards a wide view on their relationship”. ieee transactions on fuzzy systems, 24(1), 249 – 250. doi: https://doi.org/10.1109/tfuzz.2015.2446508 mendel, j.m., hagras, h., & john, r.i. (n.d.). standard background material about interval type-2 fuzzy logic systems that can be used by all authors. retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.125.4195&rep=rep1&type=pdf mendel, j.m., & wu, h. (2006). type-2 fuzzistics for symmetric interval type-2 fuzzy sets: part 1, forward problems. ieee transactions on fuzzy systems, 14(6), 781 – 792. doi: https://doi.org/10.1109/tfuzz.2006.881441 nie, m., & tan, w.w. (2008). towards an efficient type-reduction method for interval type-2 fuzzy logic systems. proceedings of ieee international conference on fuzzy systems (ieee world congress on computational intelligence), 1425 – 1432. doi: https://doi.org/10.1109/fuzzy.2008.4630559 runkler, t., chen, c., & john, r. (2018). type reduction operators for interval type-2 defuzzification. information sciences, 467, 464 – 476. doi: https://doi.org/10.1016/j.ins.2018.08.023 runkler, t., coupland, s., & john, r. (2017). interval type-2 fuzzy decision making. international journal of approximate reasoning, 80, 217 – 224. doi: https://doi.org/10.1016/j.ijar.2016.09.007 saaty, r.w. (1987). the analytic hierarchy process – what it is and how it is used. mathematical modelling, 9(3-5), 161 – 176. doi: https://doi.org/10.1016/02700255(87)90473-8 saaty, t.l. (1980). the analytic hierarchy process: planning, priority setting, resource allocation (decision-making series). mcgraw-hill publ. (russian translation: саати т. (1993). принятие решений. метод анализа иерархий. м.: радио и связь). saaty, t.l. (1994a). fundamentals of decision making and priority theory with the analytic hierarchy process, 1 st ed. pittsburgh, pa: rws publications. saaty, t.l. (1994b). highlights and critical points in the theory and application of the analytic hierarchy process. european journal of operational research, 74(3), 426 – 447. doi: https://doi.org/10.1016/0377-2217(94)90222-4 ijahp article: degtiarev, borisov/prioritization of alternatives based on analytic hierarchy process using interval type-2 fuzzy sets and probability-theoretical interval comparison international journal of the analytic hierarchy process 468 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.586 saaty, t.l., & tran, l.t. (2007). on the invalidity of fuzzifying numerical judgments in the analytic hierarchy process. mathematical and computer modeling, 46(7-8), 962 – 975. doi: https://doi.org/10.1016/j.mcm.2007.03.022 sevastianov, p. (2007). numerical methods for interval and fuzzy number comparison based on probabilistic approach and dempster-shafer theory. information sciences, 177, 4645 – 4661. doi: https://doi.org/10.1016/j.ins.2007.05.001 sevastyanov, p.v., rog, p., & venberg a.v. (2002). a constructive numerical method for the comparison of intervals. proceedings of the 4 th international conference on parallel processing and applied mathematics (lncs, 2328, eds, wyrzykowski, r., et al.), 756 – 761. doi: https://doi.org/10.1007/3-540-48086-2_84 shapiro, a.f., & koissi, m.-c. (2017). fuzzy logic modifications of the analytic hierarchy process. insurance: mathematics and economics, 75, 189 – 202. doi: https://doi.org/10.1016/j.insmatheco.2017.05.003 wang, y.-m., luo, y., & hua, z. (2008). on the extent analysis method for fuzzy ahp and its applications. european journal of operational research, 186, 735 – 747. doi: https://doi.org/10.1016/j.ejor.2007.01.050 wierman, m.j. (1997). central values of fuzzy numbers – defuzzification. information sciences, 100 (1-4), 207 – 215. doi: https://doi.org/10.1016/s0020-0255(96)00278-2 wu, d., & mendel, j.m. (2007a). uncertainty measures for interval type-2 fzzy sets. information sciences, 177, 5378 – 5393. doi: https://doi.org/10.1016/j.ins.2007.07.012 yaakob, a.m., & gegov, a. (2016). interactive topsis based group decision making methodology using z-numbers. international journal of computational intelligence systems, 9(2), 311 – 324. doi: http://dx.doi.org/10.1080/18756891.2016.1150003 zadeh, l.a. (1965). fuzzy sets. information and control, 8(3), 338 – 353. doi: https://doi.org/10.1016/s0019-9958(65)90241-x zadeh, l.a. (2011). a note on z-numbers. information sciences, 181(14), 2923 – 2932. doi: https://doi.org/10.1016/j.ins.2011.02.022 zhü, k. (2014). fuzzy analytic hierarchy process: fallacy of the popular methods. european journal of operational research, 236, 209 – 217. doi: https://doi.org/10.1016/j.ejor.2013.10.034 ijahp article: lohan, ganguly, kumar/“what’s foreign is better”: a fuzzy ahp analysis to evaluate factors that influence foreign product choice among indian consumers international journal of the analytic hierarchy process 460 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.743 “what’s foreign is better”: a fuzzy ahp analysis to evaluate factors that influence foreign product choice among indian consumers amanish lohan o. p. jindal global university alohan@jgu.edu.in anirban ganguly o. p. jindal global university aganguly@jgu.edu.in chitresh kumar o. p. jindal global university ckumar@jgu.edu.in abstract emerging economies across the globe have observed drastic changes in consumer purchasing behavior in the last three decades. this has been attributed to rapid globalization, coupled with technological advancement that has led to consumer affinity towards foreign brands. in this paper an attempt has been made to understand this affinity towards foreign brands across product categories. we used the fuzzy analytic hierarchy process to evaluate the factors that influence foreign product purchasing behavior among indian consumers. a set of five important influencing factors were identified and responses from over 100 indian consumers were collected across five product categories (automobiles, home appliances, apparel, cosmetics and watches). it was found that for apparel, watches and cosmetics, perceived quality of the brand/product was the most valued attribute. whereas, for home appliances the country of origin mattered the most, and for watches perceived quality as well as country of manufacture were equally important. the findings of this study will help shape marketing strategies for foreign brands trying to make inroads or gain market share in emerging markets like india keywords: foreign products; foreign product purchasing behavior; foreign brands; fuzzy analytic hierarchy process (f-ahp); multi-criteria decision-making (mcdm); analytic hierarchy process (ahp) 1. introduction globalization and technological advancement, along with the ever-competitive business environment has resulted in developing countries (emerging markets) becoming important for multinational corporations (mncs). in addition, a rising standard of living in these locations, coupled with competitive product pricing and information accessibility through advanced technologies has exposed consumers to a wider range of competitively priced foreign products and brands (kaynak and kara, 2002; lee and nguyen, 2017) and transformed their purchasing behavior. prior studies in the area of consumer behavior have documented this changing preference mailto:alohan@jgu.edu.in mailto:aganguly@jgu.edu.in mailto:ckumar@jgu.edu.in ijahp article: lohan, ganguly, kumar/“what’s foreign is better”: a fuzzy ahp analysis to evaluate factors that influence foreign product choice among indian consumers international journal of the analytic hierarchy process 461 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.743 for foreign products as opposed to their domestic counterparts (balabanis and diamantopoulos, 2016; batra et al., 2000; diamantopoulos et al., 2018; ger and belk, 1996; winit et al., 2014; zhou and hui, 2003). this preference can be attributed to perceived higher quality and the prestige attached to global/foreign brands that leads to high self-esteem (özsomer, 2012). balabanis and diamontopoulos (2016) studied consumer preference for foreign products over domestic products and found that multiple factors form the basis of such preferential purchase behavior such as perceived inferiority of domestic products and social aggrandizement, among others. perceived inferiority demonstrates an individual’s need to differentiate oneself from their in-group (kent and burnight, 1951), while social aggrandizement stems from consumers’ preference for a foreign group, which can also include foreign products (arora et al., 2019; balabanis and diamontopoulos, 2016). further, consumer product preference depends on multiple other factors like country of origin, product typologies, quality, symbolic value, brand image, self-identity, and self-congruency, among others (balabanis and diamontopoulos, 2004; carpenter and nakamoto, 1989; ismail et al., 2012; yasin, 2007). additionally, factors like the image of the country the brand belongs to (halkias et al., 2016) and purchasing power of the consumer also affect overall purchase behavior (mockaitis et al., 2013). although prior studies have identified these factors and their effect on foreign brand purchases, these studies have used different methods and different factors across different product categories, which did not allow for common comparison benchmarks. therefore, this paper attempts to compare a common set of factors that influence foreign product purchasing across multiple product categories using a common method. to achieve the intended objective, the study identified five factors that influence foreign product purchases based on a review of the extant literature and discussions with subject matter experts. these factors are as follows: perceived quality, country of origin, the country image of the brand, country of manufacture and foreign sounding name as major determinants affecting the choice of foreign products among indian customers. to assess the relative importance of the identified factors that influence foreign product preference, the authors decided to use the fuzzy analytic hierarchy process (f-ahp), a multi-criteria decision-making (mcdm) model. prior studies involving the ahp have observed that representing the linguistic term through an exact, crisp number often fails to exhibit a true reflection of a decision maker’s judgment (kahraman, 2018), and therefore might reflect inaccurate judgements on the part of the evaluators. on the contrary, the f-ahp allows the respondents to provide relatively imprecise (or vague) responses while undergoing pair wise comparison of alternatives in a hierarchical analysis (buckley, 1985). keeping this in mind, the authors chose to use the f-ahp, instead of the traditional ahp, as the preferred method for their study. the study encompassed five product categories namely, apparel, automobiles, watches, cosmetics, and home appliances. india was chosen as the target audience due to its expanding economy and the stiff market competition between domestic and foreign brands in all of the aforementioned categories. beginning with an overview of the literature on foreign product preference and fahp, the paper then identifies a set of critical factors that might influence foreign product purchases among indian consumers. next, the multi-criteria decision-making (mcdm) technique f-ahp is used to prioritize the identified factors. subsequently, ijahp article: lohan, ganguly, kumar/“what’s foreign is better”: a fuzzy ahp analysis to evaluate factors that influence foreign product choice among indian consumers international journal of the analytic hierarchy process 462 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.743 the findings are discussed in relation to the extant literature. limitations and future research paths are also discussed. the findings of this study will help foreign mncs that are attempting to make inroads into emerging economies like india understand the relative importance of factors that influence foreign product purchasing behavior, enabling them to design their business and marketing strategies more effectively. 2. review of the extant literature this section discusses consumers’ foreign product preference in general and then foreign product preference among indian consumers in particular. further, it provides an overview of the f-ahp and reasons why it was the chosen technique, providing the much-needed foundation for discussing the methodology and the shortlisted factors for analyzing the foreign product purchase behavior. 2.1 foreign product preference the system justification theory (sjt), developed by jost and banaji (1994), is often considered the foundation of consumers’ foreign product preference (balabanis and diamantopoulos, 2016). further, cosmopolitanism, which is defined as consumers’ preference for products outside their own culture/country, can lead to an affinity for foreign products over domestic products (rojas-méndez and chapa, 2019). this is especially true for consumers in emerging markets who choose foreign products over domestic ones, and often represents a consumer’s social status and self-esteem (khan et al., 2012; kinra, 2006; rodrigo et al., 2019). one of the major factors behind the preference of foreign products lies in the image of the country where the product is manufactured, the country of origin (coo). comparable products with similar features and functionalities are often chosen based on their coo since there is an unwritten norm associating coo to superior (or inferior) quality of products (elliott and cameron, 1994; rodrigo et al., 2019; wilcox, 2015). the country of origin develops an association for a certain product type and varies across individuals in different countries. for example, japanese electronics are considered reliable and of superior quality globally (pappu et al., 2006). this is due to the perception of quality ingrained because of the coo, i.e., the quality of electronics produced in japan which has been built over a period of time. furthermore, the image of a foreign country can enhance consumers’ beliefs about product quality and can influence its evaluation (yasin, 2007), in turn influencing the consumers’ decision-making process. another critical determinant of foreign product preference is the foreign sounding name of the product. this stems from the brand association literature and states that consumers often associate a certain product with specific characteristics based on name, type, and image (james, 2005). foreign sounding brands are those that use certain attributes associated with a specific country or region to tap into consumers’ evaluations of the product (leclerc et al., 1994; kinra, 2006). examples include american jeans, milano bags, french perfumes, and so on. since indian consumers perceive domestic brands to be inferior in terms of technology and quality in comparison to foreign brands, foreign sounding names have a positive effect on product image (kinra, 2006). this makes it crucial to investigate the effect of foreign sounding brands separate from other factors like coo, country image or product quality. ijahp article: lohan, ganguly, kumar/“what’s foreign is better”: a fuzzy ahp analysis to evaluate factors that influence foreign product choice among indian consumers international journal of the analytic hierarchy process 463 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.743 another critical determinant of foreign product preference is the country of manufacture which is different from the country of origin (coo) of the brand. in a globalized world, a product’s brand name is often associated with a certain country even though the manufacturing takes place in another country. such products are known as bi-national products (zhou and hui, 2003). the perception sometimes changes if the ‘made in’ tags display a different country (hui and zhou, 2003). although for brands emanating from developing countries and manufactured in the same country, the effect is negligible; research shows that country of manufacture influences the relationship between the consumer and perceived product quality (iyer and kalita, 1997). 2.2 indian consumers and foreign product preference the demand for foreign brands in india has grown exponentially in the last three decades. this is primarily due to liberalization and removal of trade barriers in the early 1990s. automobile, electronics, apparel, home appliance, and toiletry companies from multiple countries ranging from germany, france, the usa, korea, japan, china, and the uk among others have made inroads (kinra, 2006). kinra (2006) further observed that in india foreign brands were preferred over domestic ones due to superior technology, quality, status, and esteem. this is in line with the findings of nes and bilkey (1993) that showed products from less developed countries were regarded as low quality. however, research exhibits mixed findings concerning indian consumers. some research shows that indian consumers are moderately ethnocentric (bawa, 2004) while some shows that they are highly ethnocentric without any confounding effect from age, gender or educational level (upadhyay, 2006). post-2010 research shows that indian consumers tend to be moderately ethnocentric and their behavior is driven by certain socio-psychological factors (jain and jain, 2013). a study by kumar et al. (2011) found that consumers who scored high on an ethnocentric scale prefer domestic products; however, a large proportion of the population prefers foreign products. it could be stated that foreign product preferences of indian consumers tend to change over time, due to factors such as changing demographics and social identity. the contemporary indian consumer has access to a multitude of foreign brands with more than 50 entering the market post-2015 (economic times, 2017). as mentioned earlier, limited research has been done in the indian context that encompasses all of the factors that influence consumers’ purchase of foreign goods across different product categories. this study attempts to fill this research gap using the f-ahp as a single method adopted across all product categories. further sections provide an overview of the method. 2.3 overview of fuzzy ahp the analytical hierarchy process (ahp) is a multi-criteria decision-making technique used for prioritizing attributes that affect decision-making and identifying alternatives based on the relative importance of the attributes (saaty, 1980; macharis et al., 2004; ganguly and merino, 2015). the ahp requires constructing hierarchies to describe the problem across attributes and alternatives (yadav and sharma, 2016). then, the ahp attempts to conclusively reach pairwise judgments using reciprocal matrices ranking the actions, criteria or any attribute relevant to the decision across the alternatives (hugher, 2009). the ahp (and f-ahp) can also accommodate both quantitative as well as qualitative factors (goyal et al., 2015). there can be multiple levels within the hierarchical structure as part of the problem. the primary problem statement is placed at the top of the hierarchical structure, with attributes and subattributes in subsequently lower levels (figure 1). the ahp model is elaborated once the hierarchy is constructed (figure 1). through pairwise comparison of attributes, a ijahp article: lohan, ganguly, kumar/“what’s foreign is better”: a fuzzy ahp analysis to evaluate factors that influence foreign product choice among indian consumers international journal of the analytic hierarchy process 464 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.743 matrix containing relative importance (weights) is formulated. this relative importance is developed on the basis of a nine-point scale with attributes on each side (figure 2). by selecting any attribute, the respondent can weigh the pair and provide a relative importance for one of the attributes. the respondent can also choose to stay neutral (figure 2). figure 1 hierarchal structure of the alternatives and attributes as mentioned, the final level of the ahp hierarchical tree is comprised of the final alternatives for selection using the pairwise comparison of the attributes (and the subattributes). for example, if the same consumer was to choose between three foreign brands of apparel based on the same set of attributes (perceived quality, country of origin, country image, country of manufacture, and foreign sounding names), the last level of the hierarchy would be the three foreign brands of apparel (the three alternatives). however, since the current study focuses on identifying a set of attributes and assessing their relative criticality on the choice of five different types of products, the final level of alternatives was beyond the scope of this study, and therefore was not considered. figure 2 ahp nine-point pairwise comparison scale (presented for two attributes) ijahp article: lohan, ganguly, kumar/“what’s foreign is better”: a fuzzy ahp analysis to evaluate factors that influence foreign product choice among indian consumers international journal of the analytic hierarchy process 465 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.743 the ahp has been widely used in decision-making processes, but it does have shortcomings. the nine-point scale used for this process is considered ambiguous to an extent as it does not fully encompass the uncertainty of the decision-making process (kilinci and onal, 2011). also, the ahp does not allow decision-making in certain situations where determining a preference is not the objective due to information asymmetry of other confounding factors (nieto-morote and ruz-vila, 2011). to overcome this challenge, fuzzy numbers and scales are developed and have been extended to methods like type-2 fuzzy sets, hesitant fuzzy sets and pythagorean fuzzy sets (kahraman, 2018). in the case of the f-ahp, the selection of the alternatives is conducted using the fuzzy set theory in conventional ahp (kumar and rahman, 2017). the f-ahp allows the use of fuzzy ratios which makes it possible for the respondents to provide imprecise or vague responses when comparing two attributes (buckley, 1985; buckley et al., 2001; saaty, 1990). this allows the researcher to consider the uncertainty in the respondent’s judgment (ayhan, 2013; ganguly and kumar, 2019). the f-ahp technique was used effectively by buckley (1985, 1990) and buckley et al. (2001) provided a three-step process of decision-making using the f-ahp. the fahp uses triangular fuzzy numbers as a replacement for the nine-point scale used in the ahp. therefore, due to these advantages of the f-ahp, this method was used for the current study. once the weights are finalized, normalized weights are calculated for all the attributes/factors. finally, the alternatives are ranked through the use of these varied weights of attributes/factors. the alternative with the highest value is the preferred option. 3. research model and methodology this research work intended to explore the following research questions: rq1: what factors dominate consumer decision-making while purchasing products of foreign brands across different product categories? rq2: what are the different ways these identified factors influence consumer decision-making while purchasing products of foreign brands across different product categories? based on these research questions, the objective of this study was to identify and prioritize factors that influence consumer decision-making for foreign brands. the first step entailed the identification of a set of relevant and critical factors that might influence foreign product choice among indian customers. these initial factors were identified based on an in-depth review of the extant literature, along with discussions with experts in the areas of consumer behavior and marketing. after identifying the factors, the next phase was to refine (and condense) the identified factors in order to determine the few important factors from the entire set. this was done through a second round of iteration with the experts. once the set of shortlisted (five) factors was identified, they were used in the structured ahp questionnaire for the final ranking (figures 1 and 2). further, the current study also investigated the product choice factors for five different categories of products. these categories are automobiles, home appliances, ijahp article: lohan, ganguly, kumar/“what’s foreign is better”: a fuzzy ahp analysis to evaluate factors that influence foreign product choice among indian consumers international journal of the analytic hierarchy process 466 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.743 watches, apparel, and cosmetics. all of these categories have considerable representation in both domestic and foreign brands in the indian market. the authors also visited industry and retail experts in all of the product categories to reconfirm the relevance of the shortlisted factors. for each product category, consumers from the dominant demography for that category were chosen. this was done according to the recommendations of the industry retail experts. the respondents were briefed about the format of the ahp questionnaire and the pairwise comparison (appendix 1). all of the data was gathered through survey questionnaires individually administered to respondents to eliminate any potential response bias. the following sections of the paper discuss the identified factors. table 1 provides the number of responses collected for each of the product categories. table 1 number of respondents from each of the identified sectors product categories number of respondents automobile 21 home appliances 20 apparel 21 watches 23 cosmetics 22 total 107 4. identifying attributes that affect consumer preference for foreign products the final set of factors was obtained after reiterations and relevance based on discussions held with experts with research and industry experience in the area of consumer choice, including industry retail experts. these final set of five factors are discussed in detail below. 4.1 perceived quality the intention of product quality is to provide a competitive advantage for any particular product or brand within the market (jacobsen and aaker, 1987). high product quality improves market share as well as lowers costs to the firm (crosby, 1979). ouchi (1981) identified product quality as the primary factor for the high demand for japanese products. the foreign product preference research has also concluded that brands strive to attain the highest level of product quality (thakor and lavack, 2003; koh et al., 2010). recently, researchers have focused on what attributes are responsible for foreign product purchase and perceived quality was found to be intrinsically associated with most of these attributes (rodrigo, et al., 2019). as kinra (2006) observed, consumers in developing markets perceive foreign products as superior in terms of quality. therefore, this kind of brand association is driven by the fact that consumers perceive certain countries of origin to be synonymous with higher quality. for these reasons, perceived product quality is considered as a separate factor while assessing foreign product preference. ijahp article: lohan, ganguly, kumar/“what’s foreign is better”: a fuzzy ahp analysis to evaluate factors that influence foreign product choice among indian consumers international journal of the analytic hierarchy process 467 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.743 4.2 country image the image of a country is assessed from an outsider’s perspective for a foreign brand and leads to a perception about a brand or product from that country. until 2010, there was limited understanding about this construct and its conceptual development (fan, 2010). country image is similar to coo, yet distinct, as the former is based on perceptions and the latter on associations (han and terpstra, 1988). if the image of a country is negatively perceived by a consumer, it is bound to have a negative effect on product preference. the relevance of a country’s image was initially researched in the area of tourism (tasci and gartner, 2007), but its scope has expanded to foreign products in cases where the perceived image of a country was negative (maher and carter, 2011). this has led to further research in the field of international marketing on the possible effects of a country's image on foreign product purchases as an independent construct, although most researchers still combine it with coo literature (papadopoulos and heslop, 2014). therefore, the authors thought it was critical to consider country image as one of the factors for foreign product preference. 4.3 country of origin the effect of coo was first explored by bilkey and nes (1982) who researched how buyer evaluations are influenced by the source of the product. another critical addition to this effect was made by roth and romeo (1992) who examined how a specific product fits the perception of the country it comes from. prior research has shown that coo significantly influences consumers’ evaluations (costa et al., 2016; koubaa et al., 2015). there are several products in the market that have established dominance using their country-based association with the brand name (kinra, 2006). these studies on the effect of coo have been conducted in various contexts with a focus on distinct customer segments (bhaskaran and sukumaran, 2007). other aspects like culture and symbolic meanings (guo, 2013) can affect coo in emerging markets. these associations can impact strategies for any firm looking to make inroads into developing markets or looking to expand their existing market share. therefore, the country of origin is one of the factors being used to assess foreign product preference for consumers. here, it should be noted that we have segregated country of origin (coo) from the country of manufacture since there are certain products whose brand, design, production, and consumption are spread across multiple countries which causes consumers’ perceptions to change accordingly (roth and romeo, 1992). 4.4 country of manufacture every product originating from one country and being manufactured or assembled in another is considered a “bi-national product” (iyer and kalita, 1997). because of an ease in government regulations and low wages in certain countries, western brands have the option of shifting their manufacturing base to developing countries (kurtulus and bozbay, 2015). these products, when sold in the market, usually contain a ‘made in’ tag providing the name of the country in which it was manufactured (ulgado and loee, 1993). for example, at one point in time, chinese firms from the fashion industry started manufacturing in italy simply to procure the ‘made in italy’ tag (donadio, 2010). this can create a secondary perception in the consumer's mind regarding the product quality, as the coo is different than the manufacturing country. if the image of the manufacturing country is perceived negatively by the consumer, it can create discontent and negative perceptions about the product as well. this product bias (schooler, 1965) is not caused by the perceived quality of the product, but because of the source of its manufacturing. although, ijahp article: lohan, ganguly, kumar/“what’s foreign is better”: a fuzzy ahp analysis to evaluate factors that influence foreign product choice among indian consumers international journal of the analytic hierarchy process 468 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.743 research also shows that manufacturing in a country with a favorable image does not always lead to positive brand image evaluation (allman et al., 2016). hence, it is essential to observe consumer perceptions towards foreign products based on the country of manufacture and its importance across all product categories. for this reason, the country of manufacture has been considered as one of the factors for analysis. 4.5 foreign sounding name since most brands market themselves domestically and internationally to position themselves as a global brand (brown and hagel, 2005), the purpose of having a foreign sounding name for a brand is to create a brand image that enhances the overall brand equity for the product (leclerc et al., 1994). kinra (2006) established the consumer’s product association based on the name of the brand. these names are derived from certain geographical regions or languages which make them foreign for non-native speakers of that language. there is also a tendency to use foreign names to induce ‘perception of higher quality and greater social status’ for a brand (melnyk et al., 2012; zhou et al., 2010). if the country has a positive image in the consumer’s mind about certain products, then using a foreign sounding name may enhance that effect. on the contrary, if the consumer has a negative perception of the country, then the foreign sounding name can also have a negative effect. in developing economies like india, there are several domestic brands like allen solly, da milano, and louis philippe who use this strategy to attract consumers based on the foreign sounding name of the brand (business today, 2017). this makes it imperative to consider the foreign sounding name as one of the factors influencing foreign product preference. 5. using fuzzy ahp to prioritize foreign product preference factors the current study adapted and modified studies conducted by badri and abdulla (2004) and ayhan (2013) and developed a five-step process to conduct the f-ahp analysis. this section details the analysis step-by-step. step 1: the first step of the f-ahp analysis involved developing the objective and the ahp hierarchical structure (figure 1). the basic objective of the study was to identify and prioritize a set of important factors that might influence foreign product purchasing behavior among indian consumers. the final set of factors was identified based on the review of the literature and discussions with subject matter experts (research and industry). step 2: once the basic hierarchy was developed, the next stage comprised of constructing the pairwise matrix and putting the choices in a structured ahp questionnaire. the responses collected through a traditional ahp scale were converted to the f-ahp scale and further analysis was done. the identified factors were compared to the overall objective of the study and table 2 provides the fuzzy pairwise comparison matrix for the identified factors for one of the responses for the automobile product category 1 . additionally, as there were multiple responses for each 1 tables 2, 3 and 4 provide the responses of one of the respondents out of the 21 respondents surveyed for the automobile product category. other individual’s responses are not provided as the ahp response matrix for all of the respondents surveyed has the same structure with different values. table 4 illustrates the final weights calculated as an average of all 21 responses. ijahp article: lohan, ganguly, kumar/“what’s foreign is better”: a fuzzy ahp analysis to evaluate factors that influence foreign product choice among indian consumers international journal of the analytic hierarchy process 469 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.743 of the product categories, the preferences were averaged to calculate the final prioritized values of each of the factors for all product categories. table 2 factors that influence foreign product purchase (automobile): pairwise comparison matrix perceived quality country of origin country image country of manufacture foreign sounding names perceived quality (1,1,1) (7,8,9) (9,9,9) (7,8,9) (9,9,9) country of origin (1/9,1/8,1/7) (1,1,1) (9,9,9) (1,1,1) (7,8,9) country image (1/9,1/9,1/9) (1/9,1/9,1/9) (1,1,1) (1/9,1/9,1/9) (1,1,1) country of manufacture (1/9,1/8,1/7) (1,1,1) (9,9,9) (1,1,1) (9,9,9) foreign sounding names (1/9,1/9,1/9) (1/9,1/8,1/7) (1,1,1) (1/9,1/9,1/9) (1,1,1) step 3: after prioritization of the factors by the respondents, the next step comprised of determining the geometric mean of the fuzzy comparison value for each of the factors using equation (1) below: rĩ = [∏ x̃ij n j=1 ] 1 n (1) where, rĩ = geometric mean x̃ij = weights of the attributes and or sub − attributes n = number of attributes / sub-attributes therefore, following equation (1), the geometric mean of fuzzy comparison values of ‘perceived quality’ is calculated as follows: rĩ = [∏ x̃ij n j=1 ] 1 n = [(1 ∗ 7 ∗ 9 ∗ 7 ∗ 9) 1 5; (1 ∗ 8 ∗ 9 ∗ 8 ∗ 9) 1 5; (1 ∗ 9 ∗ 9 ∗ 9 ∗ 9) 1 5] = (5.24; 5.83; 5.80) the geometric means of the fuzzy comparison values of all of the criteria are subsequently exhibited in table 3 including their total and reversed values. additionally, in the last row of the table, as the fuzzy triangular numbers have to be arranged in increasing order, the order of the numbers is changed. ijahp article: lohan, ganguly, kumar/“what’s foreign is better”: a fuzzy ahp analysis to evaluate factors that influence foreign product choice among indian consumers international journal of the analytic hierarchy process 470 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.743 table 3 the geometric means of the fuzzy comparison values geometric mean (rĩ) perceived quality 5.24 5.53 5.80 country of origin 1.48 1.55 1.63 country image 0.27 0.27 0.27 country of manufacture 1.55 1.59 1.63 foreign sounding names 0.27 0.27 0.28 total 8.81 9.21 9.61 inverse (power of -1) 0.114 0.109 0.104 increasing order 0.104 0.109 0.114 step 4: after determining the geometric means, fuzzy weights for each of the factors were calculated. this involved multiplying the geometric mean (rĩ) by the reverse fuzzy triangular numbers (ftns) that were arranged in ascending order as shown in equation (2). wi = r̃i ⊗ (r̃1 ⊕ r̃2 ⊕ … … . .⊕ r̃n) −1 = (aw, bw, cw) (2) step 5: this step comprised of de-fuzzing the fuzzy weights for the factors, normalizing them, and arriving at the final weights of the identified factors 2 as illustrated in table 4. table 4 factors that influence foreign product purchase (automobile): geometric mean and normalized weights geometric mean (rĩ) fuzzy weights (wi) mi ni perceived quality 5.24 5.53 5.80 0.255 0.325 0.331 0.304 0.613 country of origin 1.48 1.55 1.63 0.318 0.421 0.601 0.447 0.231 country image 0.27 0.27 0.27 0.143 0.182 0.262 0.196 0.021 country of manufacture 1.55 1.59 1.63 0.055 0.071 0.103 0.077 0.070 foreign sounding names 0.27 0.27 0.28 0.030 0.037 0.055 0.041 0.065 total 8.81 9.21 9.61 inverse (power of -1) 0.114 0.109 0.104 increasing order 0.104 0.109 0.114 mi = non-fuzzy weights and ni = normalized weights 2 for a more in-depth discussion of the f-ahp process and the fuzzy triangular distribution, kindly refer to ayhan (2013) and sing and prasher (2017). ijahp article: lohan, ganguly, kumar/“what’s foreign is better”: a fuzzy ahp analysis to evaluate factors that influence foreign product choice among indian consumers international journal of the analytic hierarchy process 471 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.743 the final normalized weights for the identified factors that influence foreign product purchase (automobile) for one of the 21 responses are shown in table 4. table 5 displays the average of all of the experts surveyed for the study along with the rankings of the factors. the discussion regarding the results has been provided in the results and findings section. table 5 ranking the factors that influence foreign product purchase and their normalized weights for 21 respondents (n=21) for the automobile sector factors influencing foreign product purchase mean normalized weights rank perceived quality 0.512 1 country of origin 0.270 2 country image 0.039 5 country of manufacture 0.134 3 foreign sounding name 0.044 4 as mentioned earlier, the information exhibited in tables 2, 3 and 4 deals with one of the five product categories identified as a part of this study. tables 6 and 7 and figure 3 provide the average prioritized weights and ranking of the factors for all five product categories. table 6 final weights of the factors for each of the five categories product category identified factors perceived quality country of origin country image country of manufacture foreign sounding names automobile 0.512 0.270 0.039 0.134 0.044 home appliances 0.167 0.237 0.040 0.495 0.061 apparel 0.426 0.039 0.082 0.083 0.370 cosmetics 0.521 0.164 0.044 0.035 0.236 watches 0.233 0.083 0.326 0.033 0.324 ijahp article: lohan, ganguly, kumar/“what’s foreign is better”: a fuzzy ahp analysis to evaluate factors that influence foreign product choice among indian consumers international journal of the analytic hierarchy process 472 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.743 figure 3 final weights of the factors along for each of the five categories table 7 ranking of the factors for each of the five categories product category identified factors perceived quality country of origin country image country of manufacture foreign sounding name automobile 1 2 5 3 4 home appliances 3 2 5 1 4 apparel 1 5 4 3 2 cosmetics 1 3 4 5 2 watches 3 4 1 5 2 finally, it should also be mentioned that the results of the analysis were aggregated using the method of aggregating individual judgements (aij), which aggregates the pairwise comparison results (forman and peniwati, 1998; wang and chen, 2019). the aij, which can also be called an ‘anterior aggregation’, significantly simplifies the computation since the fuzzy weights are assessed only once, and is therefore considered a more widespread popular methodology for aggregation (wang and chen, 2019; wang et al., 2019). additionally, following the methodology suggested by leung and cao (2000), a consistency test for the pairwise judgments was performed, and the consistencies were within the permissible limits. section 6 of this paper discusses the findings in the context of the extant literature. 0.512 0.167 0.426 0.521 0.233 0.27 0.237 0.039 0.164 0.083 0.039 0.04 0.082 0.044 0.326 0.134 0.495 0.083 0.035 0.033 0.044 0.061 0.37 0.236 0.324 0 0.1 0.2 0.3 0.4 0.5 0.6 automobile home appliances apparel cosmetics watches foreign sounding names country of manufacture country image country of origin perceived quality ijahp article: lohan, ganguly, kumar/“what’s foreign is better”: a fuzzy ahp analysis to evaluate factors that influence foreign product choice among indian consumers international journal of the analytic hierarchy process 473 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.743 6. results and findings 6.1 descriptive statistics data were collected from 107 respondents and distributed equally across all five categories (table 1). overall both males and females were equally represented in the sample except in the automobile (male 56%) and cosmetics (female 71%) categories. in all of the categories, nearly 80% of the respondents belonged to the 25-35 years old age group. similarly, nearly 80% of the respondents in all of the categories had a graduate or postgraduate education. in all of the categories, nearly 70% of the respondents were employed in services or had their own business. work experience was distributed similarly, with a majority of respondents working at the mid or senior level. table 8 provides the demographic details of the data collected across product categories. the homogenous nature of the respondents across all of the categories allowed us to compare and contrast the factors with higher confidence. table 8 demographic characteristics of the respondents automobile (n= 21) cosmetics (n=22) apparel (n=21) home appliances (n=20) watches (n=23) total (n=107) gender (%) male 64.7 29.4 52.9 55.0 40.0 48.8 female 35.3 70.6 47.1 45.0 60.0 51.2 age (%) < 25 years 11.8 41.2 35.3 10.0 40.0 26.7 25-30 years 23.5 29.4 35.3 20.0 33.3 27.9 31-35 years 35.3 11.8 23.5 40.0 26.7 27.9 36-40 years 11.8 11.8 5.9 15.0 0.0 9.3 > 40 years 17.6 5.9 0.0 15.0 0.0 8.1 education level (%) high school 0.0 11.8 5.9 5.0 13.3 7.0 graduation 41.2 41.2 41.2 40.0 40.0 40.7 post-graduate 52.9 47.1 47.1 35.0 40.0 44.2 doctorate 5.9 0.0 5.9 20.0 6.7 8.1 post doctorate 0.0 0.0 0.0 0.0 0.0 0.0 occupation (%) service 47.1 47.1 35.3 40.0 40.0 41.9 business 23.5 11.8 23.5 15.0 26.7 19.8 self 11.8 23.5 23.5 30.0 26.7 23.3 freelancing 17.6 17.6 17.6 10.0 0.0 12.8 between jobs 0.0 0.0 0.0 5.0 6.7 2.3 work experience (%) director 11.8 0.0 5.9 15.0 6.7 8.1 senior level 41.2 41.2 35.3 25.0 20.0 32.6 ijahp article: lohan, ganguly, kumar/“what’s foreign is better”: a fuzzy ahp analysis to evaluate factors that influence foreign product choice among indian consumers international journal of the analytic hierarchy process 474 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.743 automobile (n= 21) cosmetics (n=22) apparel (n=21) home appliances (n=20) watches (n=23) total (n=107) middle level 29.4 29.4 23.5 30.0 40.0 30.2 junior level 17.6 17.6 23.5 20.0 26.7 20.9 consultant 0.0 11.8 11.8 10.0 6.7 8.1 further sections discuss the factors that affect foreign product preference across five product categories namely, automobiles, apparel, cosmetics, home appliances, and watches. the following discussion references tables 5 and 6. 6.2 automobiles for the purchase of automobiles, product quality is the most important factor that influences nearly half (0.512) of the decision-making process (table 5). product quality was followed by country of origin (0.27) and country of manufacture (0.134). this is in line with the findings of balabanis and siamagka (2017) who found that the country of origin has a significant impact on consumer purchase behavior for expensive products. further, considering that automobile usage has a higher risk factor compared with other categories and therefore consumers would emphasize the quality of the vehicle they buy, the results are consistent with other similar studies, i.e., wu et al. (2019). this study showed that the quality of a product was a factor that was further influenced by if the brand name was chinese or european, highlighting the quality of chinese cars in relation to european cars in both the luxury and nonluxury segment. even among males, 78% gave the highest preference to product quality, while among females the figure was around 55%. on the contrary, the country image of the brand was the least preferred factor. this is interesting as consumers, although worried about the origin of the brand, do not care about the country image of the brand. individuals under the age of 25 gave the highest weight to country image (32%). this shows that an automobile brand from a country with even a negative image would be preferred by indian consumers due to its quality and reputation; however, such cases are rare. one such example is the dacia car, sold as duster by nissan in india, which originated in romania. since romania is an eastern european country, it has no image in indian consumers' mind. however, the product has been a success in india, mostly due to the product quality. these findings support wu et al. (2019) whose study on car purchase behavior by chinese consumers based on names of the company/cars found that country of origin and english sounding names are a signal of quality. however, these findings are in contrast to the findings of triandis (1995) who found that favorable perception of a country would lead to a higher preference for that country's brands. our study shows that consumers of automobiles in india prefer certain companies that hold a favorable reputation in terms of their quality and not the country image. also, these findings were consistent among all age groups, education levels and occupations with a few exceptions discussed above. therefore, automobile makers should focus on building their brands through product quality rather than through the creation of appeal for a country or any other dimension. 6.3 apparel for apparel, the most influential factor was product quality (0.426) which is similar to the automobile sector (0.426). after this, a foreign sounding name was the most preferred factor with a weight of 0.370. the other three factors combined for the ijahp article: lohan, ganguly, kumar/“what’s foreign is better”: a fuzzy ahp analysis to evaluate factors that influence foreign product choice among indian consumers international journal of the analytic hierarchy process 475 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.743 remaining weight (0.204). in the case of apparel, country of origin was the least preferred factor (0.039). these results are very much consistent with the outcomes of giridhar et al. (2017), whose study of indian consumer preferences for foreign apparel found that perceived quality was a significant factor, whereas, need for uniqueness was not. these results are also somewhat similar to the study done by kinra (2006) which established that foreign sounding names have a positive effect on consumers. the preference for a foreign sounding name strongly resonated with individuals below the age of 30 (78%) and those working at a middle-level or lower (69%). further, the need for indian consumers to be a part of the out-group is showcased by several indian apparel brands selling under the aegis of a foreign sounding brand (e.g., peter england, louis philippe and allen solly, etc.). this can be attributed to the probable perceived in-group (indian sounding brands) based inferiority of indian consumers and a favorable preference for out-groups (foreign sounding brands) (balabanis and diamontopoulos, 2004). interestingly, the factor with the least weight, country of origin, was most dominant for individuals who were running their own businesses (65%) compared to salaried individuals (24%). 6.4 home appliances while the automobile and apparel product categories were most highly influenced by the perceived quality of foreign brands, for home appliances the highest level of importance was given to the country of manufacture (0.495) followed by country of origin (0.237) and perceived quality (0.167). consumers seem to be willing to judge the quality of a home appliance based on its country of manufacture. however, among the female respondents (45%), a majority preferred quality over the country of manufacture (61%). also, for individuals above the age of 30 (70%), country of origin was almost as important (56%) as country of manufacture. hui and zhou (2003) showed that the 'made in' tag on a product tends to shift the perception of the consumer, although, that shift was in comparison to the country of origin or country image. a probable explanation for such consumer preference could be that the consumer perceives certain countries as superior in terms of manufacturing prowess when compared to others for home appliances. this can only be measured by its origin of manufacture. 6.5 cosmetics for cosmetics, the highest weight was given to perceived quality (0.521) followed by a foreign sounding name (0.236). this was followed by country of origin (0.164). the country of manufacture, in this case, was least important for consumers. once again, this shows the variance in consumer preference across different product categories. although perceived quality is important for consumers of cosmetics, the name of the brand has a massive influence on their decision-making. interestingly, among the female respondents (71%), the majority who were under the age of 25 gave the highest preference to a foreign sounding name (65%). this is also corroborated by indian cosmetics brands like lakmé (an indian brand conceived by the tata group, now owned by hindustan unilever; it is a take on the french word for the hindi word ‘lakshmi, the goddess of wealth who was very beautiful’) or nykaa (indian brand conceived in 2012) who both use foreign sounding names. this also shows that millennials have a higher sense of names and the status associated with them as well as with the actual usability of the product. although, in this case, even though the name is important, the country image is the second least important factor. the foreign sounding names that consumers prefer are associated with certain countries even though the consumer might not be aware of that association. for them, the non-indianness of the name itself is enough for it to be perceived as superior to its ijahp article: lohan, ganguly, kumar/“what’s foreign is better”: a fuzzy ahp analysis to evaluate factors that influence foreign product choice among indian consumers international journal of the analytic hierarchy process 476 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.743 indian counterparts. this is proven by the dominance of the lakmé brand which holds almost 18% of the indian cosmetics market (economic times, 2020). for an indian consumer, this name may have a sense of superiority since it sounds western. the results could differ if names from asian or east asian countries are compared with western names as shown by wu et al. (2019) for the chinese automobile market. 6.6 watches for watches, the country image of the brand had the highest relevance (0.326) followed very closely by a foreign sounding name (0.324) and perceived quality (0.233). it is difficult to comprehend the relationship between perceived quality and a country's image without considering the country of manufacture. these findings are similar to the findings of diamantopoulos et al. (2019) for russian consumers, who had low acceptance for domestic watches, but considered similar products from foreign brands the ‘real thing.’ interestingly, the country of manufacture was least important with a weight of 0.033. in a completely different context, this higher association of brand names and foreign sounding names and a low association with perceived quality and country of manufacture can be justification for counterfeit products or watches, where utility at a lower price is considered more important than having nothing at all (tang et al., 2014). for individuals above the age of 25 (60%), almost half preferred product quality (48%) above any other factor. the majority of foreign watches in india are swiss or american followed by countries like japan and italy. the factor of foreign sounding names is almost as important for consumers. this once again confirms past research on the perceived superiority of out-groups (kent and burnight, 1951; kinra, 2006). even indian watch brands are named after greek mythology (titan) or musical instruments of italian lineage (sonata). the preference for watches is in stark contrast to automobiles and home appliances where the country of origin and country of manufacture, respectively were most important after perceived quality. here, the perceived quality was the third most important factor. one explanation could be the country of origin or country image associated with superior quality. in this case, the country's image and the brand's foreign sounding name are crucial for consumers’ decision making. it should also be noted that the majority of respondents in this category worked at a middle or junior level in a company (70%). for these individuals, the foreign sounding name had the highest preference (78%), especially for those below the age of 25 (70%). this corroborates the fact that millennials are looking for a superior outgroup which exemplifies a sense of superiority in terms of name and image of the country. further, for watches, near equal importance was given to the three following factors: country image, foreign sounding name and perceived quality. this also depicts the interrelationship among these three factors as perceived quality over a period of time by certain brands of the same country will lead to increased country image. then, the adoption of foreign sounding names by domestic brands will be a strategy to compete with the foreign brands from those countries e.g., swiss watches known for their quality and precision, german cars known for their engineering or japanese cars known for their build quality and durability. ijahp article: lohan, ganguly, kumar/“what’s foreign is better”: a fuzzy ahp analysis to evaluate factors that influence foreign product choice among indian consumers international journal of the analytic hierarchy process 477 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.743 7. limitations of the current study one of the limitations of the study comes from the choice of method. even though the f-ahp is a robust tool, the consistency ratio tends to rise for larger data groups, grouped data or any attribute matrix of more than four (saaty, 1990). our study only used data points with a consistency ratio of less than 0.20, which is within the acceptable limits considering our data size of 105 for a five-attribute comparison matrix across five different product categories. we followed wedley (1993) who recommended 0.2 or less as an acceptable limit for large attribute matrix sizes. the consistency ratios (crs) were from 4% to 20% for the selected set of responses. responses that had ratios greater than 20% were not used for the analysis. this was in line with the recommendations of wedley (1993) where a cr of 20% or less is accepted for aggregated large sample sizes. except for cosmetics, most of the average crs were within the permissible limit of 10%. the average cr for foreign apparel was 5.5% (21 survey responses), cosmetics was 10.2% (22 survey responses), automobiles was 5.8% (21 survey responses) and home appliances was 9.4% (20 survey responses). further, we collected data based on the number of respondents considered adequate for performing the f-ahp analysis. adding data for more product categories as well as looking at multiple locations will provide a more robust analytical framework for extending the results outside the product categories and in the field of consumer preference. further, we have only looked at foreign product preference and not domestic products. there could be variations in the results if domestic brands are also considered. additionally, the study does not differentiate between foreign brands from different locations, i.e., from north america, europe or south-east asia. the addition of this layer in the analysis might give more in-depth insights. finally, the sample size for this study was based on convenience sampling over a certain geographical region. these results cannot be generalized for other countries at the moment. finally, a basic disadvantage of using the ahp is the fact that rank reversals as well as the subjective nature of the technique are constraints of the technique (oguzitimur, 2011). further, the transitive nature of the technique often plays a role in increasing the value of the consistency ratio, which was also partially reflected in this study. 8. conclusions and further research the current study used the f-ahp to analyze the identified factors that influence foreign brand product purchase among indian consumers across five product categories. the factors were identified through an extant literature review and shortlisted using expert opinions. further, the study found that the relative importance of the factors varied across product categories. perceived quality is important for the purchase of automobiles, apparel, and cosmetics; the country of origin is important for home appliances and country image is important for watches. the relative importance of various foreign product preference factors can help managers and companies create a more efficient strategy to gain market share. foreign companies looking to make inroads into the indian market can use this study to design and develop their product according to consumers’ preferences, whereas, indian ijahp article: lohan, ganguly, kumar/“what’s foreign is better”: a fuzzy ahp analysis to evaluate factors that influence foreign product choice among indian consumers international journal of the analytic hierarchy process 478 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.743 companies can strategize ways to be more globally appealing among the domestic consumers. the results of the study, which can be seen from the evolutionary perspective, were consistent with research prior to 2010 that found that indian consumers attach value to qualities that are not closely related to the product but with the larger image of the country (godey et al., 2012). these qualities vary significantly across countries and the differences are more related to cultural identities and economic development. further, our findings are also aligned with the research done by hsu et al. (2017) that found that cosmetics product purchase was moderated by country of origin and price sensitivity. further, we also established that the identity of the product, i.e., its ethnographic dimension is also valuable and sometimes matters more than other factors which is similar to the findings of fischer and zeugner-roth (2017) who found that ethnicity and country of origin of the product are related to the consumer market country and their interrelationships. studies done by hsu et al. (2017), godey et al. (2012) and fischer and zeugner-roth (2017) etc. are not product specific, whereas studies done by wu et al. (2019) and diamantopoulos et al. (2019) were limited to one or two products. in this regard we have provided a study of five different products and found that the attributes and their relative significance vary across different product categories. this shows academics as well as managers that it is necessary to study different product choice evaluations separately. for future research, this study can be extended across continents in multiple locations among developed and developing countries to understand foreign brand preference across multiple cultures with varied socio-economic backgrounds. further, the study can also be extended to more product categories so that wide generalizations can be made for the field of consumer behavior. the study can be replicated for the product categories using different decision-making techniques to identify the relative importance of the same factors. a comparison of such studies will allow the development of more robust theories in the field of consumer product preference. ijahp article: lohan, ganguly, kumar/“what’s foreign is better”: a fuzzy ahp analysis to evaluate factors that influence foreign product choice among indian consumers international journal of the analytic hierarchy process 479 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.743 references allman, h. f., fenik, a. p., hewett, k., & morgan, f. n. (2016). brand image evaluations: the interactive roles of country of manufacture, brand concept, and vertical line extension type. journal of international marketing, 24(2), 40-61. doi: https://doi.org/10.1509/jim.15.0055 arora, a. s., arora, a., & taras, v. (2019). the moderating role of culture in social media-based spatial imagery, consumer xenocentrism, and word-of-mouth for global virtual teams. international journal of cross cultural management, 19(2), 160-193. doi: https://doi.org/10.1177/1470595819856379 ayhan, m. b. (2013). a fuzzy ahp approach for supplier selection problem: a sase study in a gear motor company. international journal of managing value and supply chains (ijmvsc), 4(3), 11-23. doi: https://doi.org/10.5121/ijmvsc.2013.4302 badri, m. a., & abdulla, m. h. (2004). awards of excellence in institutions of higher education: an ahp approach. international journal of educational management, 18(4), 224-242. doi: https://doi.org/10.1108/09513540410538813 balabanis, g., & diamantopoulos, a. (2004). domestic country bias, country-oforigin effects, and consumer ethnocentrism: a multidimensional unfolding approach. journal of the academy of marketing science, 32(1), 80-95. doi: https://doi.org/10.1509/jim.15.0138 balabanis, g., & diamantopoulos, a. (2016). consumer xenocentrism as determinant of foreign product preference: a system justification perspective. journal of international marketing, 24(3), 58-77. doi: https://doi.org/10.1509/jim.15.0138 balabanis, g., & siamagka, n. t. (2017). inconsistencies in the behavioural effects of consumer ethnocentrism: the role of brand, product category and country of origin. international marketing review, 34(2), 166-182. doi: https://doi.org/10.1108/imr-03-2015-0057 batra, r., ramaswamy, v., alden, d. l., steenkamp, j. b. e., & ramachander, s. (2000). effects of brand local and nonlocal origin on consumer attitudes in developing countries. journal of consumer psychology, 9(2), 83-95. doi: https://doi.org/10.1207/s15327663jcp0902_3 bawa, a. (2004). consumer ethnocentrism: cetscale validation and measurement of extent. vikalpa, 29(3), 43-58. doi: https://doi.org/10.1177/0256090920040304 bilkey, w. j., & nes, e. (1982). country-of-origin effects on product evaluations. journal of international business studies, 13(1), 89-100. doi: https://doi.org/10.1057/palgrave.jibs.8490539 bhaskaran, s., & sukumaran, n. (2007). contextual and methodological issues in coo studies. marketing intelligence and planning, 25(1), 66-81. doi: https://doi.org/10.1108/02634500710722407 brown, j. s., & hagel iii, j. (2005). disruptive management. the mckinsey quarterly, 1, 34. ijahp article: lohan, ganguly, kumar/“what’s foreign is better”: a fuzzy ahp analysis to evaluate factors that influence foreign product choice among indian consumers international journal of the analytic hierarchy process 480 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.743 buckley, j.j. (1985). fuzzy hierarchical analysis. fuzzy sets and systems, 17(1), 233– 247. doi: https://doi.org/10.1016/0165-0114(85)90090-9 buckley, j.j. (1990). fuzzy eigenvalues and input-output analysis. fuzzy sets and systems, 34(2), 187-195. doi: https://doi.org/10.1016/0165-0114(90)90158-3 buckley, j.j., feuring, t., & hayashi, y. (2001). fuzzy hierarchical analysis revisited. european journal of operational research, 129(1), 48-64. doi: https://doi.org/10.1016/s0377-2217(99)00405-1 business today (2017). 10 brands that sound international but are actually indian photos-1. retrieved 9 january 2020, from https://www.businesstoday.in/photos/inthe-news/10-brands-that-sound-international-but-are-actually-indian/1203.html costa, c., carneiro, j., & goldszmidt, r. (2016). a contingent approach to countryof-origin effects on foreign products evaluation: interaction of facets of country image with product classes. international business review, 25(5), 1066-1075. doi: https://doi.org/10.1016/j.ibusrev.2016.01.003 diamantopoulos, a., davydova, o., & arslanagic-kalajdzic, m. (2019). modeling the role of consumer xenocentrism in impacting preferences for domestic and foreign brands: a mediation analysis. journal of business research, 104, 587-596. doi: https://doi.org/10.1016/j.jbusres.2018.12.007 donadio, r. (2010). chinese remake the ‘made in italy’ fashion label. new york times, 12. economic times (2017). 50 global retailers to enter india in 6 months retail wave. [online] the economic times. available at: https://economictimes.indiatimes.com/industry/services/retail/50-global-retailers-toenter-india-in-6-months/india-the-most-promising-retailmarket/slideshow/59233635.cms [accessed 5 jan. 2020]. economic times. cosmetics market to grow by 25% to $20 billion by 2025. (2020). retrieved 8 january 2020, from https://economictimes.indiatimes.com/industry/consproducts/fmcg/cosmetics-market-to-grow-by-25-to-20-billion-by2025/articleshow/62838360.cms elliott, g. r., & cameron, r. c. (1994). consumer perception of product quality and the country-of-origin effect. journal of international marketing, 2(2), 49-62. fan, y. (2010). branding the nation: towards a better understanding. place branding and public diplomacy, 6(2), 97-103. doi: https://doi.org/10.1057/pb.2010.16 fischer, p. m., & zeugner-roth, k. p. (2017). disentangling country-of-origin effects: the interplay of product ethnicity, national identity, and consumer ethnocentrism. marketing letters, 28(2), 189-204. doi: https://doi.org/10.1007/s11002-016-9400-7 https://www.businesstoday.in/photos/in-the-news/10-brands-that-sound-international-but-are-actually-indian/1203.html https://www.businesstoday.in/photos/in-the-news/10-brands-that-sound-international-but-are-actually-indian/1203.html https://economictimes.indiatimes.com/industry/services/retail/50-global-retailers-to-enter-india-in-6-months/india-the-most-promising-retail-market/slideshow/59233635.cms https://economictimes.indiatimes.com/industry/services/retail/50-global-retailers-to-enter-india-in-6-months/india-the-most-promising-retail-market/slideshow/59233635.cms https://economictimes.indiatimes.com/industry/services/retail/50-global-retailers-to-enter-india-in-6-months/india-the-most-promising-retail-market/slideshow/59233635.cms https://economictimes.indiatimes.com/industry/cons-products/fmcg/cosmetics-market-to-grow-by-25-to-20-billion-by-2025/articleshow/62838360.cms https://economictimes.indiatimes.com/industry/cons-products/fmcg/cosmetics-market-to-grow-by-25-to-20-billion-by-2025/articleshow/62838360.cms https://economictimes.indiatimes.com/industry/cons-products/fmcg/cosmetics-market-to-grow-by-25-to-20-billion-by-2025/articleshow/62838360.cms ijahp article: lohan, ganguly, kumar/“what’s foreign is better”: a fuzzy ahp analysis to evaluate factors that influence foreign product choice among indian consumers international journal of the analytic hierarchy process 481 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.743 forman, e., & peniwati, k. (1998). aggregating individual judgments and priorities with the analytic hierarchy process. european journal of operational research, 108(1), 165-169. doi: https://doi.org/10.1016/s0377-2217(97)00244-0 ganguly, a., & merino, d. n. (2015). an integrated ahp-qfd approach for evaluating competing technological processes. international journal of the analytic hierarchy process, 7(3), 539-559. doi: https://doi.org/10.13033/ijahp.v7i3.315 ganguly, a., & kumar, c. (2019). evaluating supply chain resiliency strategies in the indian pharmaceutical sector: a fuzzy analytic hierarchy process (f-ahp) approach. international journal of the analytic hierarchy process, 11(2), 153-180. doi: https://doi.org/10.13033/ijahp.v11i2.620 ger, g., & belk, r. w. (1996). cross-cultural differences in materialism. journal of economic psychology, 17(1), 55-77. doi: https://doi.org/10.1016/01674870(95)00035-6 giridhar, v., joshi, r., & sadarangani, p. h. (2016). purchase intentions of consumer towards foreign brand apparel: evidences from india. in designing and implementing global supply chain management (pp. 185-209). igi global. doi: https://doi.org/10.4018/978-1-4666-9720-1.ch010 godey, b., pederzoli, d., aiello, g., donvito, r., chan, p., oh, h., ... & weitz, b. (2012). brand and country-of-origin effect on consumers' decision to purchase luxury products. journal of business research, 65(10), 1461-1470. doi: https://doi.org/10.1016/j.jbusres.2011.10.012 goyal, p., rahman, z., & kazmi, a. a. (2015). identification and prioritization of corporate sustainability practices using analytical hierarchy process. journal of modelling in management, 10(1), 23-49. doi: https://doi.org/10.1108/jm2-09-20120030 guo, x. (2013). living in a global world: influence of consumer global orientation on attitudes toward global brands from developed versus emerging countries. journal of international marketing, 21(1), 1-22. doi: https://doi.org/10.1509/jim.12.0065 halkias, g., davvetas, v., & diamantopoulos, a. (2016). the interplay between country stereotypes and perceived brand globalness/localness as drivers of brand preference. journal of business research, 69(9), 3621-3628. doi: https://doi.org/10.1016/j.jbusres.2016.03.022 han, c. m., & terpstra, v. (1988). country-of-origin effects for uni-national and binational products. journal of international business studies, 19(2), 235-255. doi: https://doi.org/10.1057/palgrave.jibs.8490379 hsu, c. l., chang, c. y., & yansritakul, c. (2017). exploring purchase intention of green skincare products using the theory of planned behavior: testing the moderating effects of country of origin and price sensitivity. journal of retailing and consumer services, 34, 145-152. doi: https://doi.org/10.1016/j.jretconser.2016.10.006 ijahp article: lohan, ganguly, kumar/“what’s foreign is better”: a fuzzy ahp analysis to evaluate factors that influence foreign product choice among indian consumers international journal of the analytic hierarchy process 482 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.743 ismail, z., masood, s., & tawab, z. m. (2012). factors affecting consumer preference of international brands over local brands. 2nd international conference on social science and humanity, chennai, india, march 10. iyer, g. r., & kalita, j. k. (1997). the impact of country-of-origin and country-ofmanufacture cues on consumer perceptions of quality and value. journal of global marketing, 11(1), 7-28. doi: https://doi.org/10.1300/j042v11n01_02 jain, s. k., & jain, r. (2013). consumer ethnocentrism and its antecedents: an exploratory study of consumers in india. asian journal of business research, 3(1), 118. doi: https://doi.org/10.14707/ajbr.130001 james, d. (2005). guilty through association: brand association transfer to brand alliances. journal of consumer marketing, 22(1), 14-24. doi: https://doi.org/10.1108/07363760510576518 jost, j. t., & banaji, m. r. (1994). the role of stereotyping in system‐justification and the production of false consciousness. british journal of social psychology, 33(1), 1-27. doi: https://doi.org/10.1111/j.2044-8309.1994.tb01008.x kahraman, c. (2018). a brief literature review for fuzzy ahp. international journal of the analytic hierarchy process, 10(2), 293-297. kaynak, e., & kara, a. (2002). consumer perceptions of foreign products: an analysis of product-country images and ethnocentrism. european journal of marketing, 36(7/8), 928-949. doi: https://doi.org/10.1108/03090560210430881 kent, d. p., & burnight, r. g. (1951). group centrism in complex societies. american journal of sociology, 57(3), 256-259. doi: https://doi.org/10.1086/220943 khan, h., bamber, d., & quazi, a. (2012). relevant or redundant: elite consumers' perception of foreign-made products in an emerging market. journal of marketing management, 28(9-10), 1190-1216. doi: https://doi.org/10.1080/0267257x.2011.635153 kinra, n. (2006). the effect of country-of-origin on foreign brand names in the indian market. marketing intelligence and planning, 24(1), 15-30. doi: https://doi.org/10.1108/02634500610641534 koh, n. s., hu, n., & clemons, e. k. (2010). do online reviews reflect a product’s true perceived quality? an investigation of online movie reviews across cultures. electronic commerce research and applications, 9(5), 374-385. doi: https://doi.org/10.1016/j.elerap.2010.04.001 koubaa, y., methamem, r. b., & fort, f. (2015). multidimensional structures of brand and country images, and their effects on product evaluation. international journal of market research, 57(1), 95-124. doi: https://doi.org/10.2501/ijmr-2015007 kumar, a., fairhurst, a., & kim, y. k. (2011). ethnocentric tendencies of indian consumers: impact on preference for domestic versus foreign products. the ijahp article: lohan, ganguly, kumar/“what’s foreign is better”: a fuzzy ahp analysis to evaluate factors that influence foreign product choice among indian consumers international journal of the analytic hierarchy process 483 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.743 international review of retail, distribution and consumer research, 21(4), 323-341. doi: https://doi.org/10.1080/09593969.2011.595496 kumar, d., & rahman, z. (2017). analyzing enablers of sustainable supply chain: ism and fuzzy ahp approach. journal of modelling in management, 12(3), 498-524. doi: https://doi.org/10.1108/jm2-02-2016-0013 kurtulus, k., & bozbay, z. (2015). exploring country of manufacture effect on strong and weak brands. in marketing in transition: scarcity, globalism, and sustainability (pp. 407-411). springer, cham. doi: https://doi.org/10.1007/978-3319-18687-0_151 leclerc, f., schmitt, b. h., & dubé, l. (1994). foreign branding and its effects on product perceptions and attitudes. journal of marketing research, 31(2), 263-270. doi: https://doi.org/10.1177/002224379403100209 lee, j., & nguyen, m. j. (2017). product attributes and preference for foreign brands among vietnamese consumers. journal of retailing and consumer services, 35, 7683. doi: https://doi.org/10.1016/j.jretconser.2016.12.001 leung, l. c., & cao, d. (2000). on consistency and ranking of alternatives in fuzzy ahp. european journal of operational research, 124(1), 102–113. doi: https://doi.org/10.1016/s0377-2217(99)00118-6 macharis, c., springael, j., de brucker, k., & verbeke, a. (2004). promethee and ahp: the design of operational synergies in multicriteria analysis: strengthening promethee with ideas of ahp. european journal of operational research, 153(2), 307-317. doi: https://doi.org/10.1016/s0377-2217(03)00153-x maher, a. a., & carter, l. l. (2011). the affective and cognitive components of country image: perceptions of american products in kuwait. international marketing review, 28(6), 559-580. doi: https://doi.org/10.1108/02651331111181411 melnyk, v., klein, k., & völckner, f. (2012). the double-edged sword of foreign brand names for companies from emerging countries. journal of marketing, 76(6), 21-37. doi: https://doi.org/10.1509/jm.11.0349 mockaitis, a. i., salciuviene, l., & ghauri, p. n. (2013). on what do consumer product preferences depend? determining domestic versus foreign product preferences in an emerging economy market. journal of international consumer marketing, 25(3), 166-180. doi: https://doi.org/10.1080/08961530.2013.780456 nes, e., & bilkey, w. j. (1993). a multi-cue test of country-of-origin theory. in n. papadopoulos, l. heslop (eds), product-country images: impact and role in international marketing (179-195). routledge. oguzitimur, s., (2011). why fuzzy analytic hierarchy process approach for transport problems? european regional science association ersa conference, vienna, austria. ijahp article: lohan, ganguly, kumar/“what’s foreign is better”: a fuzzy ahp analysis to evaluate factors that influence foreign product choice among indian consumers international journal of the analytic hierarchy process 484 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.743 özsomer, a. (2012). the interplay between global and local brands: a closer look at perceived brand globalness and local iconness. journal of international marketing, 20(2), 72-95. doi: https://doi.org/10.1509/jim.11.0105 papadopoulos, n., & heslop, l. a. (2014). product-country images: impact and role in international marketing. routledge, usa pappu, r., quester, p. g., & cooksey, r. w. (2006). consumer-based brand equity and country-of-origin relationships: some empirical evidence. european journal of marketing, 40(5/6), 696-717. doi: https://doi.org/10.1108/03090560610657903 rodrigo, p., khan, h., & ekinci, y. (2019). the determinants of foreign product preference amongst elite consumers in an emerging market. journal of retailing and consumer services, 46, 139-148. doi: https://doi.org/10.1016/j.jretconser.2018.04.012 rojas-méndez, j. i., & chapa, s. (2019). x-scale: a new scale to measure consumer xenocentrism. marketing intelligence and planning, (forthcoming, ahead-of-print), doi: https://doi.org/10.1108/mip-01-2019-0062 roth, m. s., & romeo, j. b. (1992). matching product category and country image perceptions: a framework for managing country-of-origin effects. journal of international business studies, 23(3), 477-497. doi: https://doi.org/10.1057/palgrave.jibs.8490276 saaty, t. l. (1980). the analytical hierarchy process: planning, priority, resource allocation. pittsburgh, usa: rws publications. saaty, t. l. (1990). how to make a decision: the analytic hierarchy process. european journal of operational research, 48(1), 9-26. doi: https://doi.org/10.1016/0377-2217(90)90057-i tang, f., tian, v. i., & zaichkowsky, j. (2014). understanding counterfeit consumption. asia pacific journal of marketing and logistics, 26(1), 4-20. doi: https://doi.org/10.1108/apjml-11-2012-0121 tasci, a. d., & gartner, w. c. (2007). destination image and its functional relationships. journal of travel research, 45(4), 413-425. doi: https://doi.org/10.1177/0047287507299569 thakor, m. v., & lavack, a. m. (2003). effect of perceived brand origin associations on consumer perceptions of quality. journal of product and brand management, 12(6), 394-407. doi: https://doi.org/10.1108/10610420310498821 triandis, h. c., chan, d. k. s., bhawuk, d. p., iwao, s., & sinha, j. b. (1995). multimethod probes of allocentrism and idiocentrism. international journal of psychology, 30(4), 461-480. doi: https://doi.org/10.1080/00207599508246580 upadhyay, y. and singh, s. k. (2006). preference for domestic goods: a study of consumer ethnocentrism. vision, 10(3), 59-68. doi: https://doi.org/10.1177/097226290601000306 https://doi.org/10.1108/mip-01-2019-0062 ijahp article: lohan, ganguly, kumar/“what’s foreign is better”: a fuzzy ahp analysis to evaluate factors that influence foreign product choice among indian consumers international journal of the analytic hierarchy process 485 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.743 wang, y. c., chen, t., & yeh, y. l. (2019). advanced 3d printing technologies for the aircraft industry: a fuzzy systematic approach for assessing the critical factors. the international journal of advanced manufacturing technology, 105(10), 4059-4069. doi: https://doi.org/10.1007/s00170-018-1927-8 wang, y. c., & chen, t. c. t. (2019). a partial-consensus posterior-aggregation fahp method—supplier selection problem as an example. mathematics, 7(2), 179. doi: https://doi.org/10.3390/math7020179 wedley, w. c. (1993). consistency prediction for incomplete ahp matrices. mathematical and computer modelling, 17(4-5), 151-161. doi: https://doi.org/10.1016/0895-7177(93)90183-y wilcox d. (2015). country-of-origin bias: a literature review and prescription for the global world. in: spotts h. (eds) marketing, technology and customer commitment in the new economy. developments in marketing science. springer. doi: https://doi.org/10.1007/978-3-319-11779-9_35 winit, w., gregory, g., cleveland, m., and verlegh, p. (2014). global vs local brands: how home country bias and price differences impact brand evaluations. international marketing review, 31(2), 102-128. wu, f., sun, q., grewal, r., & li, s. (2019). brand name types and consumer demand: evidence from china’s automobile market. journal of marketing research, 56(1), 158-175. doi: https://doi.org/10.1177/0022243718820571 yadav, v., & sharma, m. k. (2016). multi-criteria supplier selection model using the analytic hierarchy process approach. journal of modelling in management, 11(1), 326-354. doi: https://doi.org/10.1108/jm2-06-2014-0052 yasin, n. m., nasser noor, m., & mohamad, o. (2007). does image of country-oforigin matter to brand equity? journal of product and brand management, 16(1), 3848. doi: https://doi.org/10.1108/10610420710731142 zhou, l., & hui, m. k. (2003). symbolic value of foreign products in the people's republic of china. journal of international marketing, 11(2), 36-58. zhou, l., yang, z., & hui, m. k. (2010). non-local or local brands? a multi-level investigation into confidence in brand origin identification and its strategic implications. journal of the academy of marketing science, 38(2), 202-218. doi: https://doi.org/10.1007/s11747-009-0153-1 ijahp article: lohan, ganguly, kumar/“what’s foreign is better”: a fuzzy ahp analysis to evaluate factors that influence foreign product choice among indian consumers international journal of the analytic hierarchy process 486 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.743 appendix 1 questionnaire using analytical hierarchy process (ahp) to determine factors for choosing foreign apparel brands parti – basic information please share some information about yourself to put your answers in context. please tick the appropriate box. (please highlight / circle the preferred option) gender male female age ( in years) < 25 years 25 – 30 years 31 – 35 years 36 – 40 years > 40 years qualification high school graduation post – graduate doctorate post doctorate occupation service business self freelancing between jobs position director senior level middle level junior level consultant partii – ranking the foreign product selection attributes given below is a set of factors we have carefully chosen which might be critical for a consumer when choosing a foreign brand. please provide us with the pair-wise comparison between the attributes to determine their relative importance. the details of the attributes and the ahp scale are provided below: the ahp scale: relative intensity definition explanation 1 equally preferred the two attributes in question (i and j) are of equal importance 3 a little more preferred one variable is a little more important than the other 5 moderately preferred one variable is much more important than the other 7 highly preferred one variable is very much more important than the other 9 extremely preferred one variable is extremely more important than the other reciprocal if attribute i has one of the above numbers assigned to it when compared with attribute j, then j has the value 1/number assigned to it when compared with i. more ijahp article: lohan, ganguly, kumar/“what’s foreign is better”: a fuzzy ahp analysis to evaluate factors that influence foreign product choice among indian consumers international journal of the analytic hierarchy process 487 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.743 relative intensity definition explanation formally if nij = x then nji = 1/x. partiii – pairwise comparison pairwise comparison of attributes attribute i attribute j foreign apparel selection criteria perceived quality 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 country of origin perceived quality 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 country image of brand perceived quality 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 country of manufactu re perceived quality 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 foreign sounding name country of origin 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 country image of brand country of origin 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 country of manufactu re country of origin 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 foreign sounding name country image of brand 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 country of manufactu re country image of brand 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 foreign sounding name country of manufactu re 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 foreign sounding name ijahp: mu/welcome to our first issue in 2020! international journal of the analytic hierarchy process 1 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.760 welcome to our first issue in 2020! dear ijahp readers: welcome to our first issue in 2020! this volume marks our twelfth consecutive year of publication. while changes in the world were expected in this time, it has only been during the past few weeks that we have experienced social disruptive change on a global scale. the health crisis caused by covid-19 is unprecedented and caught all of us by surprise. ijahp expresses its solidarity with all of the people whose lives are being affected by the pandemic and express our wish for all to emerge from this crisis wiser and with a renewed faith in humanity. while this drama has unfolded, this issue was being prepared for production, thanks to the hard work of our ijahp team. our regular articles deal with important multi-criteria decision-making applications in different disciplines and countries. however, there is also a special topics section in this issue that is based on studies presented at mcdm 2019 in istanbul, turkey. dr. ilker topcu, istanbul technical university, and dr. ozay ozaydin, dogus university, graciously accepted the role of guest editors for this section, and they will introduce these papers in our guest preface. we have also had the opportunity to talk to dr. leandro pecchia, a decision making expert from the university of warwick, about his thoughts on the role of the decisionmaking community during the current pandemic. you can read his thought-provoking opinions in the news and events section. also, in this issue, dr. kirti peniwati discusses, in our essay section, the role of ahp/anp in striving for agility in volatile, uncertain, complex and ambiguous (vuca) environments. this is certainly an important topic of discussion for the times in which we live. in conclusion, this is an interesting issue to read and here at ijahp we wish for all of our readers and the ahp/anp community at large to remain safe and healthy! enrique mu, phd ijahp, editor-in-chief ijahp: mu/ ijahp is now indexed in scopus international journal of the analytic hierarchy process 143 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.596 ijahp is now indexed in scopus september 15, 2018 ijahp editor-in-chief at isahp 2018 workshop we have good news for our readers and authors! ijahp has been officially approved for indexing in scopus beginning with all articles published in 2018. this means that our journal constitutes not only a useful source for ahp/anp theory and applications, but is also an accepted academic outlet for our scholarly readers and authors. also, beginning in this issue, we will have a section called “special topic articles.” this section will undergo the same standard peer review process as the rest of the articles. however, we will invite a guest editor to coordinate the publication process for the articles in this section based on the specific choice of topic. we consider this approach to be more practical than the current “special issue” practice. for example, rather than waiting to have enough articles to publish a special issue on fuzzy ahp, we have invited dr. cengiz kahraman, to guest edit our inaugural “special topic articles” section focusing on fuzzy ahp and are including three articles on this topic. finally, we are all back from the international symposium of the analytic hierarchy process isahp 2018 that took place in hong kong from july 12-15. a large number of the old timers were there and we spent countless moments remembering tom saaty, our founder, and the many anecdotes about him and good times we all had with him. in this issue, rozann saaty has prepared a summary of the symposium that will be of interest to both those who could not attend the symposium and also those who were there. enjoy this issue, enrique mu, phd editor-in-chief ijahp news and events: mu/isahp2020 virtual conference international journal of the analytic hierarchy process 598 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.843 isahp2020 virtual conference rozann saaty participants from more than 30 countries attended the first virtual conference of the international symposium of the ahp (isahp) on december 3-6, 2020, sponsored by the creative decisions foundation. it was the 16 th isahp conference since the first that was held in 1988 at tianjin university, tianjin, china. the isahp2020 is a biennial event that takes place during even years, but it was cancelled in january because of covid-19 fears. the conference was brought back to life in late summer when the board of the creative decisions foundation agreed to sponsor a virtual conference in december. planning a conference usually takes much more than the few months that remained until the end of the year, so expectations were modest, but the ahp community rose to support it in a major way. the organizing committee dr. enrique mu, professor and university faculty research officer at carlow university, pittsburgh, pa, usa, served as the chair of isahp2020 and was responsible for the event overall. he also served as the chair of the isahp2014 conference in washington, d.c. dr. elena rokou, executive research director of the creative decisions foundation, planned the meeting. dr. antonella petrillo, professor at the university of naples “parthenope” (department of engineering) where she conducts research activities on multi-criteria decision analysis (mcda), was the program co-chair. dr. lirong wei, new york, ny, was the manager, and dr. birsen karpak, distinguished professor at youngstown state university, ohio, headed the scientific committee. rozann saaty, as president of the creative decisions foundation, provided the funds for the event and shared her isahp know-how. it was a real team effort with a participatory decisionmaking process. thanks to this team for the excellent conference that they organized in the record time of four months! the theme of the conference was “the next generation” and many young people and first-timers attended. special sessions were scheduled so students at the masters and phd level could make presentations. ijahp news and events: mu/isahp2020 virtual conference international journal of the analytic hierarchy process 599 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.843 zoom, the cloud platform, provided visual and audio communication for the conference. the participants handled everything better than anyone could have expected. still, margarita delgado, phd, and gabriel conzuelo, phd, from the university of pittsburgh, provided technical support to the session chairs and presenters. the workshops six pre-conference workshops were given on december 3 rd . the workshops were a rare opportunity for attendees to meet, engage with, and learn with some of the most prominent people in the ahp community, see below. here is the link to the videos of the workshops on youtube: https://youtube.com/playlist?list=pl_j2c3ikvypxx0zlo1v4j4da_ucaljwwy the workshop presenters orrin cooper sensitivity analysis luis vargas negotiation enrique mu getting your papers published william adams anp row sensitivity elena rokou benefits, opportunities, costs and risks calculations demystified. daniel saaty facilitating group decisions https://youtube.com/playlist?list=pl_j2c3ikvypxx0zlo1v4j4da_ucaljwwy ijahp news and events: mu/isahp2020 virtual conference international journal of the analytic hierarchy process 600 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.843 orrin cooper gave the workshop on “sensitivity analysis.” orrin is assistant professor in the department of marketing and supply chain management at the university of memphis in tennessee. he obtained his phd in decision science and operations management at the university of pittsburgh with thomas l. saaty as his advisor. luis vargas gave the workshop on “how to conduct a negotiation using ahp.” luis vargas is a professor at the katz school of business at the university of pittsburgh. he holds a phd in mathematics from the university of sevilla (spain) and received the juan march foundation scholarship in spain, which enabled him to move to the university of pennsylvania in 1976-78 where he studied under thomas saaty and obtained a second phd. he worked with tom throughout his life, co-authoring numerous papers and books william adams, currently the director of algorithms at the software company decision lens, inc., gave a workshop titled “the how and whys of anp row sensitivity.” dr. adams holds a phd in algebraic geometry, with more than 20 years of experience designing mathematical software. bill designed and implemented the superdecisions anp software for the creative decisions foundation. enrique mu’s workshop was titled “how to improve your chance of getting your ahp/anp paper published.” dr. mu is the university faculty research officer at carlow university, pittsburgh, pennsylvania. he obtained his phd from the university of pittsburgh where he was a close colleague of tom saaty. enrique is the editor-in-chief and founder of the international journal for the analytic hierarchy process (ijahp), which means he is well-qualified to offer advice on publishing papers on the ahp. elena rokou, chief research officer at the creative decisions foundation, the ahp/anp research institute funded by dr. thomas saaty, offered a workshop titled “demystifying benefits-opportunities-costs-risks in anp calculations.” dr. rokou holds a phd degree in the field of operational research from the national technical university of athens. she specializes in project scheduling, optimization algorithms, and anp and group decision making. daniel saaty, thomas l. saaty’s son, is the chief scientist and co-founder of the decision lens software company along with his brother john. dan’s workshop was on “facilitating group decision-making.” dan is an expert in the implementation of the analytic hierarchy process (ahp) and analytic network process (anp) as well as having over 20 years of experience in developing and implementing advanced decision methodologies. ijahp news and events: mu/isahp2020 virtual conference international journal of the analytic hierarchy process 601 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.843 the speakers the conference hosted an invited keynote speaker and two plenary speakers as follows: dr. blaženka divjak of the university of zagreb in varaždin, croatia, gave the keynote address “leadership in complex and chaotic situations.” she was the croatian minister for science and education from june 2017 to july 2020 and is the author of over 150 scientific and professional papers, including articles in multi-criteria decision-making (ahp/anp). dr. divjak gave a fascinating talk about the rapid decisions the government in croatia had to make during the complex and chaotic upheaval of the covid-19 virus in 2020. her previous experience in education was invaluable as croatia planned and successfully shifted from classroom teaching to a mix of online and hybrid courses. the video of her talk is available on the creative decisions foundation youtube channel. alessio ishizaka gave a plenary address titled “three developments of the analytic hierarchy process: calibrated fuzzy ahp, ahpsort and group analytic hierarchy process ordering (gahpo).” professor ishizaka is head of the information systems, supply chain and decision-making department and full professor in decision analysis at neoma business school, ecole de commerce & management, france. his research is in the area of decision analysis, where he has published more than 100 papers and is the author of a key textbook, multicriteria decision analysis. professor ishizaka is one of the most qualified people in the ahp community to talk about approaches that combine ahp with other decision-making methods. blaženka divjak keynote speaker alessio ishizaka plenary address https://www.youtube.com/watch?v=cxsj84l9s08&list=pl_j2c3ikvypw7ykrs_aamyu8pno1ifygj&index=7&t=2438s https://neoma-bs.com/ https://neoma-bs.com/ https://neoma-bs.com/ https://neoma-bs.com/ https://neoma-bs.com/ https://neoma-bs.com/ https://neoma-bs.com/ https://neoma-bs.com/ https://neoma-bs.com/ https://neoma-bs.com/ https://neoma-bs.com/ https://neoma-bs.com/ https://neoma-bs.com/ https://neoma-bs.com/ ijahp news and events: mu/isahp2020 virtual conference international journal of the analytic hierarchy process 602 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.843 william c. wedley gave a plenary address titled “ahp/anp: issues and insights for the future.” dr. wedley is professor emeritus of the beedie school of business, simon fraser university, vancouver, canada, where he specializes in operations management and decision making. he attended the first isahp conference in tianjin, china, in 1988, and was also the chair of the 1996 isahp conference in vancouver, canada. he spoke about the ahp/anp providing a comprehensive procedure for capturing the intensities of different properties on a ratio scale and then combining multiple such scales into a meaningful multi-criteria decision. the ratio feature, ease of use, and flexibility to measure many phenomena, has allowed the ahp/anp to become the dominant mcdm procedure. during the last five minutes there was an interesting scholarly discussion between bill and luis vargas. the video is posted on creative decisions foundation’s youtube channel: https://youtu.be/t-5wdijxaug. the sessions the conference included 130 attendees, 72 presentations, including 9 posters that were organized into 19 sessions with two parallel sessions at each time. the conference schedule was limited to between 8am – 2pm (us eastern time) in order to offer the best possible availability to participants from other time zones. the online conference had positive aspects which included that the powerpoint presentations were easier to see on one’s own computer, the audio was better than in the usual meeting room at a conference, and it was less costly for both the attendees and their organizations with no transportation, lodging or food costs. universities will save a significant amount of money in the future by having their people attend virtual conferences instead of live ones. the less positive side was that participants missed the social interaction afforded by coffee breaks, lunches, and gala dinners, and the chance to visit a foreign country. in future meetings, virtual hangout rooms offered by zoom might partially suffice for socializing, though individuals would need to bring their own beverages. all of the videos from the conference are available on creative decisions foundation’s youtube channel: https://www.youtube.com/channel/ucqheyejzvjjxsewvg06v2yg. select the playlist tab to find the videos well-organized into categories of speeches, workshops and presentations. poster sessions were held during the last hour on friday and saturday of the conference. these, again, surpassed expectations. young people had been invited and encouraged to make poster presentations and we are proud of the polished way they performed in public. new ahp software apps were demonstrated during one poster session. a new addition to the isahp events was a student roundtable in which undergraduate and graduate students discussed the benefits and challenges of learning the ahp during bill wedley plenary address https://youtu.be/t-5wdijxaug https://www.youtube.com/channel/ucqheyejzvjjxsewvg06v2yg ijahp news and events: mu/isahp2020 virtual conference international journal of the analytic hierarchy process 603 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.843 their coursework. this roundtable was co-chaired by dr. enrique mu (carlow university), dr. elena rokou (university of pittsburgh) and dr. anna florek (centrum, pontificia universidad católica del perú) who invited their students to join in this lively discussion. the success of this experience may lead to similar roundtables becoming part of the program in meetings in the future. the closing ceremony during the last session on sunday, best paper awards were presented and awards were given to the track chairs for their service. each isahp awardee was able to say, “hi,” while being on the screen as they received their award isahp2020 was an academic success, but just as importantly, it was a great ahp/anp community success! isahp2020 participants saying, “hi and farewell until 2022” 1 1 this screenshot shows the international nature of isahp2020. the 24 participants in the picture are from the usa, italy, chile, china, perú, malaysia, mexico, spain, croatia, turkey and others. ijahp article: tsyganok, kadenko, andriichuk/usage of scales with different number of grades for pair comparisons in decision support systems international journal of the analytic hierarchy process 112 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.259 usage of scales with different number of grades for pair comparisons in decision support systems v.v.tsyganok e-mail: vitaliy.tsyganok@gmail.com s.v.kadenko  e-mail: seriga2009@gmail.com o.v.andriichuk e-mail: andreychuck@ukr.net national academy of sciences of ukraine the institute for information recording kyiv, ukraine abstract in this paper we suggest an original approach to conducting individual pair comparisons during individual and group multi-criteria decision-making (including ahp/anp-based decisions). with this approach every expert is given an opportunity to use the scale, in the degree of detail (number of points/grades) that most adequately reflects his/her competence in the issue under consideration for every single pair comparison. before aggregation all separate expert estimates (judgments) are brought to a unified scale, and scales in which these judgments were built are assigned respective weights. a respective instrument for pair comparison conduction has been developed, and an experiment has been organized. the experiment statistically proves that as a result of suggested technology usage, there is an increase in the degree of correspondence between estimates, input by an expert, and his (her) own notions on examination objects. keywords: group decision making; decision support system; expert judgments; pairwise comparisons; different scales 1. introduction multi-criteria decision making facilitates solutions for a broad spectrum of problems. a vast amount of research, both theoretical and practical is being conducted to facilitate multi-criteria decision-making in management, environmental protection, production (defelice & petrillo, 2013a,b), logistics (noorul haq & kannan, 2007), (kannan, noorul haq & sasikumar, 2008), project selection (zandi & tavana, 2010) and other areas  corresponding author mailto:vitaliy.tsyganok@gmail.com mailto:serh@gmail.com mailto:andreychuck@ukr.net ijahp article: tsyganok, kadenko, andriichuk/usage of scales with different number of grades for pair comparisons in decision support systems international journal of the analytic hierarchy process 113 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.259 (kalika & rossinsky, 2003, vaidogas & zavadskas, 2007). pair comparisons are widely used for multi-criteria decision-making in various weakly-structured domains (i.e., domains, where no benchmarks exist and expert data is the only credible information source). the ahp has a special place among pair comparison-based approaches and related methods, which are utilized in various spheres of human activity. the practice of expert examination conduction (including ahp-based examinations) indicates that there are certain difficulties that arise when verbal scales are used for expert examination. the expert/decision-maker is often allowed to use only one scale for pair comparisons. in order to get thorough and undistorted data from an expert, (s)he must be offered the opportunity to input estimates in a scale, which most adequately corresponds to his/her competence (awareness) level of the issue under consideration. the suggested research resolves the issue of using verbal scales with a different degree of detail for each particular pair comparison, in order to ensure maximal credibility of knowledge obtained from an expert (expert information must be thorough and undistorted). to ensure the information obtained from an expert is thorough, we suggest using verbal scales with a sufficient degree of detail: the more points the scale includes, the more information an expert can, potentially, input into a dss using this particular scale. to avoid information distortion (if an expert is unsure of the degree of dominance between objects in a pair, i.e. (s)he is not competent enough), we suggest giving experts the opportunity to use scales with a low degree of detail, or even allowing them to refuse to estimate preferences in certain object pairs. in our research we also consider an important factor that influences the level of expert information distortion – quantitative equivalent, corresponding to a respective value from a verbal scale. correspondence between preference value input by an expert and this expert’s notions about the ratio of object weights on a pair is an issue of great importance, as it influences the credibility of expert data-based decision-making recommendations. 2. literature review a key study in the described area is the recent research by elliott (2010) addressing the influence of a chosen quantitative scale upon correspondence between estimation results and an expert’s own notions. it was demonstrated that scale selection has considerable impact upon the resulting decision variant estimate. three quantitative scale types were analyzed, whose point values were assigned to fundamental scale points of two kinds, i.e. scales with 5 and 9 grades (saaty, 2006). quantitative scales under consideration included integer, balanced and power scales (salo & hamalainen, 1997, stevens, 1957). besides that we should mention research done in the context of ahp by ma & zheng (1991) and dodd, donegan & mcmaster (1995). the most popular and probably the simplest scale is an integer scale where standard linguistic (verbal) values correspond to respective numeric equivalents (from 1 to 9) (see table 1). ijahp article: tsyganok, kadenko, andriichuk/usage of scales with different number of grades for pair comparisons in decision support systems international journal of the analytic hierarchy process 114 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.259 table 1 verbal expressions used by experts to determine the preference degrees in alternative pairs in an integer scale verbal expression numeric equivalent equal 1 weakly or slightly preferred 2 moderately preferred 3 moderately plus preferred 4 strongly preferred 5 “strongly plus” preferred 6 very strongly preferred 7 very, very strongly preferred 8 extremely preferred 9 when a scale with 5 grades (instead of 9) is used, only 5 verbal expressions corresponding to odd values (1, 3, 5, 7, 9) are utilized. we should note that phrases presented in table 1, introduced by tom saaty (1980), are often referred to as fundamental scale (or saaty scale) values. originally the scale consisted of only 5 grades, so even values (2, 4, 6, 8) and respective verbal expressions correspond to intermediary (transitional) preference degrees. although it may seem that more elaborate verbal expressions should facilitate more laconic and exact descriptions of preference degrees, the trouble is that these expressions must provide a clear and exact description of the relationship between different preference degrees. for example, a person may clearly imagine a ‘weak’ dominance being weaker than ‘moderate’ and ‘strong’ dominance, but the relationship (ratio) between ‘extreme’ and ‘absolute’ dominance degrees is unclear. integer values, corresponding to linguistic phrases show “how many times” one alternative exceeds the other according to a given criterion, i.e. reflect multiplicative preferences. some researchers noticed that usage of integer scales leads to uneven distribution of alternative weights calculated based on these scales. for instance, a change of preference from ‘weak’ (2) to ‘moderate’ (3) has a larger effect on respective alternative weight, than the change from ‘very very strong’ (8) to ‘extreme’ (9). in order to overcome this drawback, salo and hamalainen (1997) suggested a balanced scale where the change of weights remains constant when preferences change. in the so-called balanced scale the alternative weights, which are calculated based on pair comparisons, are evenly distributed depending on initial pair comparison data. numeric values, corresponding to verbal expressions are calculated according to the formula: w w a   1 , where w is the weight of alternative, which dominates in the respective pair, as presented in table 2. ijahp article: tsyganok, kadenko, andriichuk/usage of scales with different number of grades for pair comparisons in decision support systems international journal of the analytic hierarchy process 115 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.259 table 2 numeric equivalents for balanced scale verbal expression numeric equivalent no dominance (equal) 0.5/0.5 = 1 weak or insignificant dominance 0.55/0.45 = 11/9 moderate dominance 0.6/0.4 = 3/2 more than moderate dominance 0.65/0.35 = 13/7 strong dominance 0.7/0.3 = 7/3 more than strong dominance 0.75/0.25 = 3 very strong dominance 0.8/0.2 = 4 very-very strong dominance 0.85/0.15 = 17/3 extreme dominance 0.9/0.1 = 9 it should be stressed that the scale presented in table 2 is fully balanced only in the case of 2 alternatives. another attempt to make a scale whose values more clearly represent the estimator’s preferences is a power scale suggested, among other authors, by stevens (1957) and lootsmaa (1980, 1991). numeric values, corresponding to linguistic phrases for power scale, are calculated based on the expression: 1 1 9    y x a , where x is an integer value from table 1, corresponding to the same verbal expression, while y is the number of scale grades. for 9 grades the numeric equivalents for power scale are presented in table 3. table 3 numeric equivalents for power scale verbal expression corresponding numeric value no dominance (equal) 19 8 0  weak or insignificant dominance 316.19 8 1  moderate dominance 732.19 8 2  more than moderate dominance 280.29 8 3  strong dominance 39 8 4  more than strong dominance 948.39 8 5  very strong dominance 196.59 8 6  very-very strong dominance 839.69 8 7  extreme dominance 99 8 8  in contrast to a ‘balanced’ scale, weights obtained based on pair comparisons in a power scale are evenly distributed under any number of alternatives. ijahp article: tsyganok, kadenko, andriichuk/usage of scales with different number of grades for pair comparisons in decision support systems international journal of the analytic hierarchy process 116 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.259 speaking about ‘convenience’ of different scales, we should mention the results obtained by elliot (2010), particularly an analysis of data on expert’s attitudes toward the proposed estimation scale obtained from 64 experts. the question asked the experts if they thought that the number of preference values to choose from was: a) too large; b) just fine; or c) too small. the expert’s opinions were distributed as shown in table 4. table 4 percentages of expert answers were as follows too many grades just fine number of grades is too small 5 grades 43,8% 53,1% 3,1% 9 grades 84,4% 15,6% 0% we tend to feel that the conclusion that was made based on data from table 4 about the advantages of a 5-grade scale can only be relevant for the given group of experts and for a specific expert examination on a specific subject. another set of conclusions to be made from the research is as follows: 1) the choice of an adequate number of grades in a scale to be used for expert estimation is a topical issue; 2) the fact that opinions of experts concerning the most “comfortable” number of grades in a scale, varies, indicates that offering every single expert a separate scale is better than selecting one scale for all experts to estimate alternatives; 3) for every aspect of examination the expert should be able to choose some scale, which is optimal for this particular issue in terms of the number of grades; 4) the optimal number of grades is not always 5 or 9. a review and comparison of five scales is provided in ji & jiang (2003). besides the already listed scales, the review also features the scale of ma & zheng (1991) and the scale of donegan, dodd & mcmaster (1995). numeric values, corresponding to verbal expressions in the scale of ma & zheng (1991) are calculated according to the expression: xy y a   1 , where x is a respective integer value from table 1, while y is the number of grades in the scale. numeric values are presented in table 5. table 5 numeric equivalents for the scale of ma & zheng (1991) verbal expression corresponding numeric value no dominance (equal) 9/9 = 1 weak or insignificant dominance 9/8 moderate dominance 9/7 more than moderate dominance 9/6 = 3/2 strong dominance 9/5 more than strong dominance 9/4 very strong dominance 9/3 = 3 very-very strong dominance 9/2 extreme dominance 9/1 = 9 ijahp article: tsyganok, kadenko, andriichuk/usage of scales with different number of grades for pair comparisons in decision support systems international journal of the analytic hierarchy process 117 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.259 the scale, suggested by donegan, dodd & mcmaster (1995) is a bit more difficult to understand. numeric values, corresponding to verbal expressions in the scale, suggested are calculated according to the expression                 1 1 tanhexp 1 h x a , where x is a respective integer value from table 1, while h is a parameter, calculated based on the concept of horizons (ranges). calculation of an 8-based horizon ( 3141h ) is based on an assumption that for alternatives a, b and c the following transitive relation holds: a dominates over c with the degree of dominance 9 ( 9aca ), if 8 bcab aa , i.e. „8 • 8 = 9”. calculation of the 7-based horizon: 261h is based on a similar assumption that 9aca if 7 bcab aa , i.e. „7 • 7 = 9”. numeric values for the scale of dodd, donegan & mcmaster (1995) are set forth in table 6. table 6 numeric equivalents for the scale of dodd, donegan & mcmaster (1995) verbal expression corresponding numeric value no dominance (equal) 1 weak or insignificant dominance 1,132 moderate dominance 1,287 more than moderate dominance 1,477 strong dominance 1,720 more than strong dominance 2,060 very strong dominance 2,600 very-very strong dominance 3,732 extreme dominance 9 a comparative study of the above-mentioned scales and optimization model for selection of scales are set forth in dong (2008). a constructive original approach to classification of ratio scales and linking them to each other was recently suggested by william wedley (2010). in contrast to the research described in the listed publications (containing useful ideas to arm ourselves with), we suggest choosing a different scale for each single pair comparison and not for all pair comparisons. in the experimental part of our research we will focus on three particular scales: integer-value fundamental scale with 5 grades, 9 grades, and on a “mixed” scale, where an expert can chose the type of scale and the number of grades (from 2 to 9) for every single pair comparison. 3. hypotheses/objectives the purpose of the present study is to prove that in order to ensure that thorough and undistorted expert information on the relation between objects (on estimates provided during pair comparisons) is obtained, an expert should be given an opportunity to use scales with different degrees of detail (accuracy). this hypothesis is based on a presumption that in every issue under consideration (and in every pair comparison) an ijahp article: tsyganok, kadenko, andriichuk/usage of scales with different number of grades for pair comparisons in decision support systems international journal of the analytic hierarchy process 118 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.259 expert has a different level of knowledge/competency/awareness. each expert’s competence level can correspond to a respective estimation scale: the higher the expert’s competence, the more detailed scale (s)he can use to adequately present his(her) knowledge. according to the same principle, an uninformed/incompetent expert should have an opportunity to use a scale with a smaller number of grades (including ordinal scale with only two values – “more” or “less”) for pair comparisons, or even refuse to compare objects in a pair because of incompetence. it is understandable that an expert judgment provided in a more detailed scale should be considered more significant than that same judgment provided in a less detailed scale because in the first case the expert is more confident, and his self-estimated competence in the issue under consideration is higher. consequently, if during pair comparisons an expert considers objects equal, this judgment can be considered the same as a refusal to conduct this particular comparison (inability to evaluate preference of objects in a pair due to doubts/low competency in the issue under consideration). as we see, in verbal scales there is no real need for a grade “equal”/“no preference”, because if an expert chooses this value, (s)he might as well “skip” (refuse to estimate) the respective preference. anyway, the choice of “equal” preference value does not introduce any additional information about the relation between objects. proof (confirmation) of any hypothesis in a weakly structured domain (in which we are conducting our research) is problematic, as there are absolutely no benchmarks to compare results with. that is why the only way to confirm the hypothesis is an experiment using estimates provided by experts. such an experiment is described in section 5 of this paper. 4. research design/methodology during the research a methodology and respective software tools were developed to conduct expert estimation based on the abovementioned approach. in group estimation every expert is offered the opportunity to provide pair comparisons in verbal scales with different degrees of detail. each particular pair comparison starts with the scale including only two values («less» and «more») with an opportunity to refuse to provide the judgment – «no idea» (figure 1). figure 1. initial estimation in ordinal scale ijahp article: tsyganok, kadenko, andriichuk/usage of scales with different number of grades for pair comparisons in decision support systems international journal of the analytic hierarchy process 119 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.259 figure 2. procedure of gradual estimate precision increase if ordinal comparison is provided (one of the values «less» or «more» is selected) the expert is given the opportunity to gradually make the estimate more precise, and stop estimation at any stage («confirm» button on figure 2). in the process of this iterative procedure the final estimate is conducted in the scale which most adequately corresponds to the expert’s competence about the issue of defining the preference relation between two particular objects. the final estimate may be provided in a scale including 2 to 8 grades. it should be noted that the developed tool allows an expert to be sure that the quantitative equivalent really corresponds to this or that verbal expression from the estimation scale. such confidence is achieved through providing the user (expert) with interactive graphic tips (hints) which allow him to imagine the approximate relation between objects and thus improve the degree of correspondence between the expert’s personal notions and the information (s)he inputs during pair comparisons. for aggregation of incomplete comparison matrices provided by a group of experts, when different comparisons can be conducted in scales with different accuracy, we suggest using the method based on enumeration of all spanning trees with further averaging of priority vectors calculated for every tree (tsyganok, 2010). before calculation of priority vectors, all pair comparison matrix elements (judgements) are brought to a unified (most detailed) scale. during this process, weights of particular judgements (pair comparisons) are taken into consideration. the weights depend on the degree of detail of scales the comparisons were provided in. 4.1 problem statement the formal statement of the alternative weight calculation problem in our case can be shown as follows. what is given: ai , i[1..m] – expert pair comparison matrices (pcm) with dimensionality of nхn, which have the following properties: 1) matrices are reciprocally-symmetrical, that is ijahp article: tsyganok, kadenko, andriichuk/usage of scales with different number of grades for pair comparisons in decision support systems international journal of the analytic hierarchy process 120 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.259 why we are going to use only the elements above the principal diagonal; 2) matrices are multiplicative, i.e. each element aij shows how many times an object with index i is better than object with index j according to some criterion; 3) in the general case, matrices are incomplete, because an expert can, for some reasons, abstain from providing some pair comparisons; 4) every single element of a pcm is obtained in some scale, which is assigned a weight coefficient sj, j[0..8]) cl, l [1..m] – relative competence of experts in the group. we should find: the resulting object (alternative) weight vector (priority vector) wk, k[1..n]. 4.2 unification of estimates the problem of aggregating individual expert estimates includes bringing estimates provided in different scales to a unified form. at this stage, we consider it appropriate to bring estimates provided by different experts in different scales to a single scale, the most informative (detailed) one. bringing the estimates to less informative scales is irrelevant, because in this case the information given in scales with a larger number of grades will be lost. one of the ways to solve the problem of unification of estimates is to define clear correspondence between each of the grades of the less informative scale and some sub-set of grades in the more informative scale, and subsequently to bring the estimates to the more informative scale through selecting respective grades from these scales. in order to establish this clear correspondence, we should keep in mind that in the case when the grade of a less informative scale covers some range of grades in a more informative scale (one grade corresponds to a range of grades), when a respective value is selected on the more informative scale, it should be the value which is equally distant from the limits of this range. in this case a certain value, most likely provided by the expert, will be selected. if we consider all possible estimate values corresponding to some scale grade, random values, distributed according to some law, which is close to normal, then when the information is unified and the estimates are brought to the more informative scale the estimate given in the less informative scale should be replaced by the mathematical expectation of the range of grades in the more informative one. for symmetric distribution laws (which include normal distribution), it is appropriate to take the simple average of lower and upper border (limit) values of the range of grades of the more informative scale, lying within the limits of the grade of the less informative scale. ijahp article: tsyganok, kadenko, andriichuk/usage of scales with different number of grades for pair comparisons in decision support systems international journal of the analytic hierarchy process 121 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.259 figure 3. numeric equivalents for grades in scales of: a) 8; b) 4; c) 3 grades figure 3a displays the standard integer-value scale of 9 grades (maximal number of grades among the scales under consideration), which includes numeric values from 2 to 9 (elliot, 2010; saaty 2006). based on the abovementioned considerations, during pair comparisons in scales, which include from 2 to 9 grades of dominance of one alternative over another, a numeric value corresponding to a certain grade of a scale with a smaller number of grades (the less informative one) is calculated as a simple average of respective values of “range limiting” grades of the more informative scale. for instance, the value, corresponding to the 2 nd grade from the left of the scale of 4 grades equals 4.5 (figure 3b) in a scale of 9 grades. the step-by-step explanation is as follows. due to psycho-physiological constraints, the largest possible number of grades in a scale equals 9. each grade in a 9-grade scale corresponds to an integer value from the set {2..9}, while the limits of each of these grades are as follows: for the 2 nd grade – 1.5 on the left, 2.5 on the right, for the 3 rd grade –2.5 on the left, 3.5 on the right, etc. (figure 3a). consequently, for the 2 nd grade of the scale of 4 grades (which corresponds to combined 4 th and 5 th grades of the scale of 9 grades), the average value is 5.4 2 5.55.3   , where 3.5 is the left border value of the 4 th grade of the scale with maximal number of grades, while 5.5 is the right border value of the 5 th grade. calculation of numeric equivalents for the scale of 3 grades is displayed on figure 3c. the general formula for calculating these values for any given number of grades looks as follows:   n lp ilm n i   2 1 , (1) where n i m is the numeric equivalent of the i-th of n scale grades; l is the left border of the scale (l = 1.5); p is the right border of the scale (p = 9.5). 2 3 4 5 6 7 8 9 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5 9.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 5.5 8.1(6) 2.8(3) 2 3 4 5 6 7 8 9 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5 a) b) c) ijahp article: tsyganok, kadenko, andriichuk/usage of scales with different number of grades for pair comparisons in decision support systems international journal of the analytic hierarchy process 122 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.259 consequently, the numeric equivalent for the rightmost (3 rd ) grade of the scale of 3 grades, is calculated as follows:   )6(1.8 3 5.15.9 35.1 2 13 3   m , as shown on figure 3c. 4.3 weighting of estimates besides the problem of unification of expert estimates provided in different scales, processing of unified estimates is also an important task within the process of estimate aggregation. we suggest an approach to processing of unified estimates based on the idea of assigning different weights to estimates provided in different scales. an estimate provided in the more informative scale should weigh more than the estimate provided by an expert using the less detailed scale. the presumption is based on our belief that usage of a more detailed scale requires more competence in the issues under consideration from the expert. in fact, an expert using the scale with larger number of grades is in a way using a “more precise device” for measurement (estimation) of objects than an expert using a less detailed scale. the specific form of dependence between weight (significance) of expert estimates provided in some scale and the number of grades in the scale needs to be addressed in a separate study. we can state however, that the dependence of the weight of an estimate provided in some scale on the number of grades in this scale is a monotonously increasing function. besides that, it can be seen that the significance of adding more grades to a scale decreases with the increase of the number of grades. based on these considerations, we suggest linking the scale weight coefficient (indicating the scale’s degree of detail or informative content) to the quantity of information which can potentially be obtained from an expert providing an estimate in the given scale. in fact, such an indicator shows to what extent the usage of a certain scale for expert estimation decreases the general entropy of the system (subject domain description model). one of the options for a simplified calculation of such informative content indicator, based on the assumption that selection of any scale grade by an expert is equally probable (probability is evenly distributed), is calculation of the quantity of information according to hartley’s formula (hartley, 1928): ni 2log , where n is the number of expert estimation scale grades. according to this formula, the following weight coefficients are “assigned” to the scales used: 0 (for n=1) – the expert cannot define the preferences among alternatives; 1 (for n=2) – the expert defined only ordinal preferences among alternatives (“better” or “worse”); 3log2 – after defining ordinal preference the expert defined the degree of dominance using 2 additional preference grades (for instance, “strong” or “weak” preference); 4log2 .. 9log2 – after defining ordinal preference of one alternative from a pair over the other, the expert specified the degree of dominance using the scale with respective number of grades (3 to 8). the advantages of the given approach to scale weight calculation are its simplicity and gradual decrease of significance of adding new grades to the scale. as we see, the estimates provided by experts during pair ijahp article: tsyganok, kadenko, andriichuk/usage of scales with different number of grades for pair comparisons in decision support systems international journal of the analytic hierarchy process 123 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.259 comparisons of alternatives are assigned weight coefficients depending on the scales they used for estimation. 4.4 aggregation of estimates after the estimates are unified and weighted, the pcm obtained from different experts can be aggregated. in order to fully utilize the redundancy of expert information, we suggest aggregating individual pcm using the so-called combinatorial method (tsyganok, 2010). before aggregation of estimates, completeness and consistency of individual pcms should be checked. completeness and consistency are ensured as follows. in the combinatorial method, priority vectors are calculated based on some basic sets of pair comparisons. if such basic sets are not complete, the priority vector cannot be calculated, so in such a case the expert should be re-addressed with a request to provide the missing basic pair comparisons, and thus ensure the completeness of the set. if the matrices are not consistent enough, the results of the whole expert examination, even if they can be obtained, will be less credible. in order to check consistency (and, if necessary, improvement) we suggest using a spectral consistency coefficient (zgurovsky, totsenko & tsyganok, 2004). after the aggregation is done, alternative weights can be calculated based on the aggregate pcm using one of the numerous approaches available (tsyganok 2010). at this point the problem posed in the beginning of this section is solved. 5. data/model analysis to confirm the hypothesis set forth in section 3 of this paper, experimental research has been conducted with real experts. the description of the experiment itself is provided below. the purpose of the experiment is to compare the suggested technology with existing approaches for obtaining relative factor weights particularly with ahp-based ones (saaty, 2008). we suggest comparing the degrees of correspondence between subject domain objects (factors) and relative factor weights, calculated using this or that expert estimation technology. in fact, we are comparing the results obtained using expert estimation technologies with the “model” values formed in the expert’s mind. results (relative factor weights), obtained based on the suggested technology we are testing, are compared with weights calculated based on pair comparisons provided in integer-value scales with 5 and 9 grades. weights of criteria (factors) for pair comparisons in the fundamental scale are calculated using saaty’s eigenvector method, based on pcm. we suggest conducting the experiment in the following 4 stages: 1. formulation of the goal (problem) and factors which influence it. at this stage the expert formulates the problem he or she considers him or herself competent in, and lists 5 to 7 mutually independent factors which in his\her opinion are most significant for solving this problem. ijahp article: tsyganok, kadenko, andriichuk/usage of scales with different number of grades for pair comparisons in decision support systems international journal of the analytic hierarchy process 124 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.259 2. input of pair comparisons in three different scales. all pair comparisons, which must be input throughout the experiment (for all three technologies, respectively, using scales with 5, 9 grades and a “mixed” scale) are offered to the expert for input in random order. 3. calculation of factor weights. relative weights of objects (factors) are calculated based on pcms obtained at the previous stage which are processed using the respective methods. 3 priority vectors are calculated. the eigenvector method is used to process matrices built using the first two approaches, while the so-called combinatorial (or spanning tree enumeration) method is used to define a priority vector based on a matrix including comparisons provided in different scales, (tsyganok, 2010) (particularly, its modification allowing for usage of different weights for different estimation scales). 4. ranking of calculated weight vectors. at this stage the expert ranks the three previously calculated factor weight vectors in their decreasing order of relevance, i.e. their correspondence to the expert’s actual notions. vectors in the form of unnamed bar diagrams are displayed on the screen in random order without technology specification. each respondent (experiment participant) is offered the opportunity to rank the vectors according to their correspondence to his/her perceptions of quantitative relations between impacts of the formulated factors. a quantitative indicator, formed as a result of experiment, is the degree of preference of this or that expert estimation technology (i.e., frequency of this technology being rated as number one in the ranking of all technologies under consideration). 5.1 experiment integrity aspect in order to ensure experiment integrity certain steps were taken. 1) subject domain chosen by the respondent. at the first of the listed stages we suggest that an expert select a subject domain, in which he or she considers him or herself competent in. the expert is offered the opportunity to formulate a problem which, in his or her opinion, is most understandable. independent subject domain choice guarantees that expert examination organizers are not biased in any way (as the subject domain is chosen by the expert). the expert examination organizer (knowledge organizer) does not “impose” a subject domain upon the expert, so there can be no situation in which the expert is not sufficiently competent to conduct estimation in the subject domain. because of the described feature, the experiment becomes more universal than the experiment conducted by elliot in 2010. 2) same factors, different technologies. once the problem is formulated, the expert is offered the opportunity to list a set of factors which describe it. this same set of factors is used to evaluate different expert estimation technologies in the process of the experiment, thus ensuring the relevance and credibility of comparison of results obtained using different technologies. 3) independence of factors. when criteria (factors) are formulated, it is required that they must describe the problem most thoroughly, and at the same time be mutually independent (no “intersections” or ijahp article: tsyganok, kadenko, andriichuk/usage of scales with different number of grades for pair comparisons in decision support systems international journal of the analytic hierarchy process 125 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.259 mutual impacts between criteria are allowed). this is the necessary condition for obtaining credible results with pcm processing methods. 4) random order of factor enumeration. another critical feature embedded in the experiment is the opportunity to name the factors in random order: in the first place the expert, intuitively, recalls (and, consequently, names) the factors he or she considers most important. this order of factors remains the same for all methods. this condition is also important for ensuring equal credibility of results obtained based on pair comparisons (as in pair comparison method the number of comparisons a given object is featured in depends on the order of objects). 5) limitation on the quantity of factors. the number of factors must not exceed 72. this condition is determined by psychophysiological constraints of an average individual (expert) (miller, 1956). besides, the condition also plays an important role when ranges and numbers of grades for used scales are defined. 6) random order of comparisons. random choice of pairs of objects (factors) to be presented to the expert for comparison, as well as random choice of estimation technology, at the second stage, allow for a decrease in correlation between estimates of ratios obtained using different methods for the same pairs of objects during one experiment session (instance). in this way we can ensure mutual independence of specific pair comparisons. 7) “blind” ranking of estimation technologies. at the fourth stage, the expert is presented a bar diagram, on which he or she can chose one of three factor weight vectors. the type of the scale in which the estimation has been conducted is not specified. in this way, we can guarantee that the rankings of expert estimation technologies according to their correspondence to expert’s own notions of the problem are unbiased. 5.2 interpretation of experiment results the experiment was conducted using a specially designed software application. the result of each experiment represented a file including the following information: 1. expert’s name 2. problem title 3. list of factors 4. matrices of paired comparisons of factors, obtained using different technologies of expert estimation 5. time, spent by the expert on every question (comparison) 6. ranking of expert estimation technologies (criterion weight vectors) 7. expert’s own explanation of the ranking every expert can participate in the experiment several times as long as he(she) formulates different problems. once a particular examination (expertise) is done, the relevance of its results is checked because further statistical processing of results obtained from different ijahp article: tsyganok, kadenko, andriichuk/usage of scales with different number of grades for pair comparisons in decision support systems international journal of the analytic hierarchy process 126 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.259 experts can be done only under the condition that the set of expert estimation technology rankings is adequate. a relevance (adequacy) check includes two stages: 1. during the analysis, results which can be interpreted as “careless” are filtered off. these include a) results which took the expert too little time to obtain (any expert requires at least several seconds to formulate a well-considered answer or estimate); b) pcms where most estimates coincide; c) files with resulting data which were corrected manually; d) incomplete pcms, based on which priority vectors cannot be calculated due to lack of basic pair comparisons. 2. factor weight vectors are filtered off if pcms, based on which these weights were calculated, contain significant contradictions (inconsistency level is too high). for example, if one of the three resulting weight vectors is significantly different from the other two, the expert might ignore this vector and the respective technology, and not include it into the ranking. even if the vector and the respective technology are included in the ranking as the lowest-ranking (most inadequate) one, the whole ranking will not bear information on correspondence between weight vectors and expert’s notions of the problem. 5.3 statistical relevance of experiment’s results the minimal number of experiment instances necessary for achievement of desired statistical relevance of results has been calculated as follows. the estimate of statistical relevance was based on central-limit theorem. under a confidence probability level p =0.9 (probability that the random value in question falls into the confidence interval β), and confidence interval size β=0.1, selected for the experiment, the necessary number of experiment instances is calculated from the inequality:   21 2 )1(   pf pp n   , (2) where 1 f is a inverse laplace function; p is the frequency of repetition of the resulting random characteristic. we select the values of p based on the data from the table of experimental results already obtained (see table 7) as the worst (closest to 0.5) probability (frequency). in our case, “worst” frequency means that the characteristic assumes some value as frequently as it does not, so that probabilities p and (1 – p) are equal. if we look at the second column of the table {10/630.16; 12/630.19; 41/630.65} we can see that frequency p =0.65 is the “worst” one in a sense. this value should be put into the formula above. if we replace the variables with the actual values, we get the following results:   65.19.01 f ,    72.29.0 21 f , 72.2 1.0 )65.01(65.0 2  n , 9.61n . this means that in order for the experiment results (i.e. conclusions regarding the preference of this or that decision support technology) to be relevant it is sufficient to analyze data from at least 62 instances of the experiment. ijahp article: tsyganok, kadenko, andriichuk/usage of scales with different number of grades for pair comparisons in decision support systems international journal of the analytic hierarchy process 127 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.259 5.4 numeric results of the experiment the experts/respondents in the experiment were represented mostly by graduate it students at one of the universities in kyiv, ukraine. so far, around 100 respondents participated in the experiment. after some results were filtered off, as described above, the set was limited to 63 rankings of factor weight vectors (and respective expert estimation technologies), and 2 rankings turned out to be incomplete (included two, but not three vectors). the results are presented in table 7. table 7 comparative experimental research results name of pair comparison technology number of participants, who assigned the specified rank to respective technology “1” “2” “3” fundamental preference scale with 5 grades 10 15 37 fundamental preference scale with 9 grades 12 33 17 technology suggested in the paper 41 15 7 as we can see, most respondents preferred the suggested expert estimation technology based on aggregation of results of pair comparisons provided in different scales. based on the results of the experimental research, we can conclude that in most of the analyzed cases expert estimates obtained using the suggested technology are more consistent with an expert’s individual perceptions of the examination subject, in comparison to estimates, based on traditional estimation techniques (where fixed number of verbal scale grades is used). consequently, wide implementation of the suggested pair comparison instrument in decision support technologies (including those using ahp/anp) seems adequate. 6. limitations usage of the suggested tool for pair comparisons may require a longer time during expert estimation and as a result more resources than traditional methods. this may result from the fact that more actions are required from experts during each pair comparison. however, this also results in higher credibility of expert estimates and recommendations to decision makers. 7. conclusions as a result of the research, we have suggested an expert estimation mechanism which allows experts to use scales of different accuracy for each pair comparison. relevance of the suggested approach is experimentally proven. it has been demonstrated that usage of the respective tool for pair comparisons allows us to improve the degree of correspondence between an expert’s estimates and his notions of the examination subject. this improvement results from the fact that experts use scales whose accuracy is most consistent with their competency in every issue under consideration. ijahp article: tsyganok, kadenko, andriichuk/usage of scales with different number of grades for pair comparisons in decision support systems international journal of the analytic hierarchy process 128 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.259 the suggested approach can be considered an extension of (and not an alternative to) ahp/anp approaches. implementation of the suggested expert estimation technology in combination with pair comparison matrix aggregation methods (including group methods) improves the credibility of ahp/anp-based recommendations given to decision makers. the technology’s basic advantages are universality and flexibility. these features allow it to be utilized in existing and new decision support tools, particularly in those areas where multiple criteria (factors, attributes) of different natures, both qualitative and quantitative, should be taken into account and where expert data can be the only credible source of information. such areas include managerial decisions, project selection, personnel evaluation, logistics, strategic planning and others. ijahp article: tsyganok, kadenko, andriichuk/usage of scales with different number of grades for pair comparisons in decision support systems international journal of the analytic hierarchy process 129 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.259 references de felice, f., & petrillo, a. (2013a). multicriteria approach for process modelling in strategic environmental management planning. international journal of simulation and process modelling, 8(1), 6–16. doi: http://dx.doi.org/10.1504/ijspm.2013.055190 de felice, f., & petrillo, a. (2013b). decision-making analysis to improve public participation in strategic energy production management. studies in fuzziness and soft computing 305, 129–142. doi: 10.1007/978-3-642-35635-3_11 dodd, f.j., donegan, h.a., & mcmaster, t.b.m. (1995). scale horizons in analytic hierarchies. journal of multi-criteria decision analysis, 4, 177–188. doi: 10.1002/mcda.4020040304 yucheng dong, yinfeng xu, hongyi li, & min dai (2008). a comparative study of the numerical scales and the prioritization methods in ahp. european journal of operational research, 186, 229–242. doi:10.1016/j.ejor.2007.01.044 elliott, m.a. (2010). selecting numerical scales for pairwise comparisons. reliability engineering and system safety, 95, 750–763. doi:10.1016/j.ress.2010.02.013 hartley, r.v.l. (1928). transmission of information. bell system technical journal, 7, 535–63. doi: 10.1002/j.1538-7305.1928.tb01236.x ji, p., & jiang, r. (2003). scale transitivity in the ahp. journal of the operational research society, 54(8), 896–905. doi:10.1057/palgrave.jors.2601557 kalika, v.i., & rossinsky, g. (2003). methodology of multi-criteria decision making accounting for uncertainty and some applications. international journal of management and decision making, 4(2/3), 240 – 271. doi: http://dx.doi.org/10.1504/ijmdm.2003.003507 kannan, g., noorul haq, a., & sasikumar, p. (2008) an application of the analytical hierarchy process and fuzzy analytical hierarchy process in the selection of collecting centre location for the reverse logistics. multicriteria decision-making supply chain model. international journal of management and decision making, 9(4), 350 – 365. doi: http://dx.doi.org/10.1504/ijmdm.2008.019360 lootsma f.a. (1989). conflict resolution via pairwise comparisons of concessions. european journal of operational research, 40, 109–116. doi:10.1016/03772217(89)90278-6 lootsma f.a. (1991). scale sensitivity and rank preservation in a multiplicative variant of the ahp and smart. report 91–67. faculty twi. delft university of technology. delft. the netherlands. ma, d., & zheng, x. (1991). scale method of ahp. proceedings of the second international symposium on the ahp, 1, 197–202. http://dx.doi.org/10.1016/j.ejor.2007.01.044 http://dx.doi.org/10.1016/j.ress.2010.02.013 http://dx.doi.org/10.1002/j.1538-7305.1928.tb01236.x http://dx.doi.org/10.1016/0377-2217(89)90278-6 http://dx.doi.org/10.1016/0377-2217(89)90278-6 ijahp article: tsyganok, kadenko, andriichuk/usage of scales with different number of grades for pair comparisons in decision support systems international journal of the analytic hierarchy process 130 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.259 miller, g.a. (1956). the magical number seven, plus or minus two: some limits on our capacity for processing information. the psychological review, 63(2), 81–97. http://dx.doi.org/10.1037/h0043158 noorul haq, a., & kannan, g. (2007). a hybrid normalised multi criteria decision making for the vendor selection in a supply chain model. international journal of management and decision making, 8 (5/6), 601–622. doi: http://dx.doi.org/10.1504/ijmdm.2007.013421 saaty, t.l. (2006). fundamentals of decision making and priority theory with the analytic hierarchy process. pittsburgh, pa: rws publications. saaty t.l. (2008). relative measurement and its generalization in decision making. why pairwise comparisons are central in mathematics for the measurement of intangible factors. the analytic hierarchy/network process. statistics and operations research, 102(2), 251–318. doi:10.1007/bf03191825 salo, a.a., & hamalainen, r.p. (1997). on the measurement of preferences in the analytic hierarchy process. journal of multi-criteria decision analysis, 6, 309–319. doi: 10.1002/(sici)1099-1360(199711)6:6<309::aid-mcda163>3.0.co;2-2 stevens, s.s. (1957). on the psychophysical law. psychology review, 64, 153–181. doi: http://dx.doi.org/10.1037/h0046162 tsyganok, v.v. (2010). investigation of the aggregation effectiveness of expert estimates obtained by the pairwise comparison method. mathematical and computer modelling, 52(3-4), 538–544. doi:10.1016/j.mcm.2010.03.052 vaidogas, e.r., & zavadskas, e.k. (2007). introducing reliability measures into multicriteria decision-making. international journal of management and decision making, 8(5/6), 475 – 496. doi: http://dx.doi.org/10.1504/ijmdm.2007.013413 wedley, w.c., & eng ung choo (2010). a taxonomy of ratio scales. or-52 keynotes and extended abstracts, operational research society ltd., royal holloway university of london, uk 7-9/09/2010, 199–203. zandi, f. & tavana, m. (2010). a multi-attribute group decision support system for information technology project selection. international journal of business information systems, 6(2), 179 – 199. doi: http://dx.doi.org/10.1504/ijbis.2010.034353 zgurovsky, m.z., totsenko, v.v., & tsyganok, v.v. (2004). group incomplete paired comparisons with account of expert competence. mathematical and computer modelling, 39(4), 349–361. doi:10.1016/s0895-7177(04)90511-0 http://psycnet.apa.org/doi/10.1037/h0043158 http://psycnet.apa.org/doi/10.1037/h0046162 http://dx.doi.org/10.1016/j.mcm.2010.03.052 http://dx.doi.org/10.1016/s0895-7177(04)90511-0 ijahp article: naseh/space mission definition based on analytical hierarchy process (ahp) method international journal of the analytic hierarchy process 145 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.583 space mission definition based on analytical hierarchy process (ahp) method hassan naseh aerospace research institute ministry of science, research and technology tehran, iran, p.o. box: 14665-834 hnaseh@ari.ac.ir abstract generally, missions of remote sensing satellites are divided into three types: monitoring missions, recognition missions and surveillance missions. these missions need the sun synchronous orbits (ssos) or multi-sun synchronous orbits (mssos) in order to perform the operations. these ssos and mssos have many requirements that make deciding the best type a multi-criteria decision problem. to this end, the analytical hierarchy process (ahp) methodology is utilized to decide the orbit type in the remote sensing satellites missions. therefore, the objective in the above methodology is to reach the orbit in the minimum amount of time and with the minimum cost (ssos or mssos). the criterions are remote sensing missions, and the alternatives in the ahp methodology are the ssos and mssos. in the conclusion, the results of methodology are presented and evaluated. keywords: ahp; space mission; remote sensing; sso; msso 1. introduction the requirements of a planetary observation system are strictly related to the orbit design and, in particular, to its shape and inclination. the periodic sun synchronous orbits (pssos) are often considered the most suitable option with regards to the observation of the earth, the ground resolution, the coverage (both in latitude and in longitude), the repetition of observation, and the same condition of solar illumination, (ortore et al., 2012). in fact, these orbits allow observation of a given region of the planet at regular time intervals (periodic orbits) with approximately the same solar illumination conditions (sun-synchronism). unlike the strict sun synchronous orbits (ssos), which are characterized by a typical relationship between orbit altitude and inclination and that can be modified using solar sail systems, the introduction of the periodicity (repetition of observation) limits the possible altitudes to a finite number (represented by the points belonging to the previously mentioned altitude–inclination curve) (bolle & circi, 2011). the choice of the solution results was developed from a compromise among the several requirements since it is not possible to have the best solution for all of them (e.g. the coverage and the repetition of observation are in contrast) (ortore et al., 2012). the multi-sun synchronous orbits (mssos), which the classical pssos are a particular solution of, represent an important alternative to the use of the pssos. the mssos form a general category of orbits that are characterized by two periodicity conditions. the first ijahp article: naseh/space mission definition based on analytical hierarchy process (ahp) method international journal of the analytic hierarchy process 146 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.583 condition is related to the observation of a given area (periodic orbit), and the second one refers to the solar illumination which repeats itself at regular time intervals (multi-sun synchronism). the two periodicities are properly linked so as to obtain cycles of observation of the same area in which the solar illumination gradually varies along with the choice of the orbit elements. it returns to the initial condition after a number of nodal days which is a multiple of the revisit time. a significant advantage of the mssos lies in the great flexibility of choice of the orbit inclination concerning the ssos that are retrograde and usually quasi-polar. for that matter a limitation of the exploitable launching sites is determined. (ulivieri & anselmo, 1991). the critical factors for decision making (criteria/sub-criteria) are as follows: 1. remote sensing satellites (capderou m. 2005; larson wily j. et. al. 1999; fortescue p. et. al. 2011; griffin m. d. et. al. 1991): one of the most important key parameters in the space mission definition is orbital parameters (e.g. orbital inclination, eccentricity, argument of perigee, ascending node, etc.). the orbital parameters are strongly affected by performance of sub-systems in remote sensing satellites (larson wily j. et. al. 1999). hence, sub-criteria of remote sensing satellites are technology complexity, power supply subsystem, thermal control subsystem, remote sensing payload (camera), number and location of ground segment, satellite operation and satellite security. 2. mission application: there are many applications from the remote sensing satellites, but the most important applications are environment and resources management, target surveillance, change detection, cartography, and meteorological missions (larson wily j. et. al. 1999; fortescue p. et. al. 2011). furthermore, these sub-criterions are influenced from the orbital specifications (capderou m. 2005). hence, sub-criterion of level two related to the orbit type are revisit time, local time of imaging in the each day, sensor calibration, overlap criteria, and regional coverage. recently, some research has focused on the application of the analytic hierarchy process (ahp) methodology. for example, the selection of scholarship recipients has been studied. this research involves the analysis of multi-criteria decision making by prioritizing which criterion was the most important for the student selection process (kim et. al., 2017). the scholarship distribution process for the students was conducted by the selection team as the expert and with the scholarship budget which has been determined by universal university. furthermore, the ahp process is executed and implemented in detail in a way that can show the ahp procedure and the methodology expressions (taherdoost, 2017). the ahp has gained increasing attention in the construction management (cm) domain as a technique to analyze complex situations and make sound decisions (darko a. et. al., 2018). also, it is performed as the combination of insights from a scientometric mapping technique and social network analysis (sna) to study collaboration networks. scientometric mapping technique overlay mapping was applied to obtain a cognitive map of the ahp field, and sna was used to study co-authorship networks (emrouznejad et. al., 2016). classical ahp was extended by use of the d-ahp to model various types of uncertainty, and represents an extension of the dempster– shafer theory (deng et al., 2014; fan et al., 2016). the d-ahp allows determination of the weights of the alternatives and has proved effective in addressing supplier selection ijahp article: naseh/space mission definition based on analytical hierarchy process (ahp) method international journal of the analytic hierarchy process 147 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.583 problems, to represent the decision matrix of pairwise comparisons given by experts and to deal with problems of grouting efficiency evaluation (deng et al., 2014; fan et al., 2016).this branch of the literature includes a group of works combining insights from dea with ahp (deahp) and in some cases with fahp (ramanathan, 2006; che, wang, & chuang, 2010). following ramanathan (2006), sevkli et al. (2007) apply this hybrid approach to a real industry case and show that deahp outperforms the ahp method for supplier selection. on another hand, this paper has been criticized by wang, chin, and leung (2009) and shows the weaknesses of the deahp. there has been some known effort functions mentioned in the literature. in the study, in order to look for a suitable effort function for the allocating purpose, we first made a comparison among them. as a result of the lack of data for creating the parameters, an effort function cannot always be depended on. the ahp was considered as a possible method to compensate for this deficiency. a common example used in the literature was also illustrated for an elementary test and verification. to accomplish the mission definition procedure, a revised effort minimization method was used for an integral calculation (naseh, 2011). 2. the ahp – step by step the ahp is based on the experience gained by its developer, t. l. saaty (1980), while directing research projects in the u.s. arms control and disarmament agency. it was developed as a reaction to the finding that there is a miserable lack of common, easily understood and easy-to-implement methodologies to enable complex decisions to be made. since then, the simplicity and power of the ahp has led to its widespread use across multiple domains in every part of the world. the ahp has been used in business, government, social studies, r & d, defense and other domains involving decisions in which choice, prioritization or forecasting is needed. the ahp provides a means of decomposing the problem into a hierarchy of sub problems which can more easily be comprehended and subjectively evaluated. the subjective evaluations are converted into numerical values and processed to rank each alternative on a numerical scale. the methodology of the ahp can be explained in the following steps: step 1: the problem is decomposed into a hierarchy of goal, criteria, sub-criteria and alternatives. this is the most creative and important part of decision-making. structuring the decision problem as a hierarchy is fundamental to the process of ahp. a hierarchy indicates a relationship between elements of one level with those of the level immediately below. figure 1 shows a generic hierarchic structure. at the root of the hierarchy is the goal or objective of the problem being studied and analyzed. ijahp article: naseh/space mission definition based on analytical hierarchy process (ahp) method international journal of the analytic hierarchy process 148 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.583 figure 1 generic hierarchic structure step 2: data are collected from experts or decision-makers corresponding to the hierarchic structure in the pairwise comparison of alternatives on a qualitative scale as described below. experts can rate the comparison as equal, marginally strong, strong, very strong, and extremely strong. the opinion can be collected in a specially designed format as shown in figure 2. figure 2 format for pair wise comparisons step 3: the pairwise comparisons of various criteria generated in step 2 are organized into a square matrix. the diagonal elements of the matrix equal 1. the criterion in the i th row is better than the criterion in the j th column if the value of the element (i, j) is more than 1; otherwise, the criterion in the j th column is better than that in the i th row. the (j, i) element of the matrix is the reciprocal of the (i, j) element. step 4: the principal eigenvalue and the corresponding normalized right eigenvector of the comparison matrix give the relative importance of the various criteria being compared. the elements of the normalized eigenvector are termed weights with respect to the criteria or sub-criteria and ratings with respect to the alternatives. step 5: the consistency of the matrix of order n is evaluated. comparisons made by this method are subjective and the ahp tolerates inconsistency through the amount of redundancy in the approach. if this consistency index fails to reach a required level, then answers to comparisons may be re-examined. the consistency index, ci, is calculated as shown in equation 1. . . 1 ma x n c i n     (1) where max is the maximum eigenvalue of the judgment matrix. this ci can be compared with that of a random matrix, ri. the ratio derived, ci/ri, is termed the consistency ratio (cr). saaty suggests the value of cr should be less than 0.1. ijahp article: naseh/space mission definition based on analytical hierarchy process (ahp) method international journal of the analytic hierarchy process 149 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.583 step 6: the rating of each alternative is multiplied by the weights of the sub-criteria and aggregated to get local ratings with respect to each criterion. the local ratings are then multiplied by the weights of the criteria and aggregated to get global ratings. the ahp produces weight values for each alternative based on the judged importance of one alternative over another with respect to a common criterion. 3. ahp applied to the decision making problem in this section, we applied ahp as a systematic approach to develop a decision making method for determining the favorite alternative (sun synchronous prioritization) and requirements to meet in space mission design. the schematic of ahp applied to the decision making problem is shown in the figure 3. figure 3 shows the objective function in level 1, the decision making criterion in level 2, and the alternative of the problem in level 3. in this case, the alternatives are repeated with each application sub-criteria. next, the criteria and sub-criteria for our decision making model will be introduced. see the appendix for the decision model definitions. the remote sensing satellite is influenced from the technology complexity (figure 4), thermal control subsystem (figure 5), power supply subsystem (figure 6), payload as an active or passive sensor (figure 7), number and location of the ground segment (figure 8), and satellite operation (figure 9). the second level of sub-criterion in remote sensing application includes revisit time (figure 10), local time (figure 9), sensor calibration (figure 11), overlap criteria (figure 9), and regional coverage (figures 9 and 12). figure 3 schematic of ahp modeling applied to the decision making problem ijahp article: naseh/space mission definition based on analytical hierarchy process (ahp) method international journal of the analytic hierarchy process 150 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.583 figure 4 space based laser concept (technology complexity) of jaxa and osaka university (maini a. k. et. al., 2011) figure 5 thermal control subsystem and types of received energy (maini a. k. et. al., 2011) ijahp article: naseh/space mission definition based on analytical hierarchy process (ahp) method international journal of the analytic hierarchy process 151 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.583 figure 6 power supply subsystem performance in eclipse (maini a. k. et. al., 2011) figure 7 payload (type of sensors) (maini a. k. et. al., 2011) ijahp article: naseh/space mission definition based on analytical hierarchy process (ahp) method international journal of the analytic hierarchy process 152 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.583 figure 8 number and location of ground segment (maini a. k. et. al., 2011) figure 9 earth observation and coverage (maini a. k. et. al., 2011) figure 10 revisit time versus ground sampling distance or gsd (naseh h. et. al., 2016) ijahp article: naseh/space mission definition based on analytical hierarchy process (ahp) method international journal of the analytic hierarchy process 153 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.583 figure 11 active sensor calibrations (maini a. k. et. al., 2011) figure 12 spectral resolution versus ground sampling distance or gsd (naseh h. et. al., 2016) the alternative capability with respect to the criterion is listed in table 1. thus, the pairwise comparison matrix of criterion on a qualitative scale is obtained based on expert knowledge and system configuration. in this decision making problem the knowledge of ten experts with more than 10 years related academic experience was utilized and the average achieved scores were used. next, it was determined that the ahp analysis was achieved between the acceptable ranges of inconsistency. to this end, the pairwise comparison matrix of alternatives is presented in table 2. the decision making problem model in the expert choice software is presented in the figure 13, and samples of pairwise comparison tables are shown in the figure 14. ijahp article: naseh/space mission definition based on analytical hierarchy process (ahp) method international journal of the analytic hierarchy process 154 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.583 table 1 alternative capability with respect to the criterion pmss ss criteria row good very good commercial applications 1 local global region coverage 2 higher lower revisit time 3 non periodic periodic revisit time in specific local time 4 no required required upper stage 5 limited variety ground segment for guidance, navigation and control 6 complexity in power supply and thermal control no complexity satellite design complexity 7 non suitable suitable monitoring, recognition and surveillance 8 table 2 pair wise comparison matrix of criterion on a qualitative scale row commercial application region coverage revisit time revisit time in specific local time ground segment for guidance, navigation and control satellite design complexity monitoring, recognition and surveillance commercial application 0.2 0.2 0.2 0.2 1 0.2 0.14 region coverage 0.2 0.2 0.14 5 5 0.2 0.14 revisit time 0.2 0.2 0.14 0.2 5 5 0.14 revisit time in specific local time 0.2 0.2 0.2 1 5 5 0.14 ground segment for guidance, navigation and control 1 0.2 0.2 0.2 0.2 0.2 0.14 satellite design complexity 7 7 7 7 7 7 1 monitoring, recognition and surveillance 5 0.14 0.14 0.2 5 1 0.14 ijahp article: naseh/space mission definition based on analytical hierarchy process (ahp) method international journal of the analytic hierarchy process 155 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.583 figure 13 decision making problem model in the expert choice software ijahp article: naseh/space mission definition based on analytical hierarchy process (ahp) method international journal of the analytic hierarchy process 156 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.583 figure 14 samples of pairwise comparison tables 4. result presentation and validation the application of the ahp to the decision problem was presented in the previous section, and this section presents the results of the decision making process. the methodology is modeled in the expert choice software. the results are shown in figures 15 to 22 from any criteria aspect that was presented in the figure 3. ijahp article: naseh/space mission definition based on analytical hierarchy process (ahp) method international journal of the analytic hierarchy process 157 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.583 figure 15 sun synchronous orbit prioritization (overall in space missions) figure 16 sun synchronous orbit prioritization (from application type criteria aspect) figure 17 sun synchronous orbit prioritization (from optical remote sensing criteria aspect) figure 18 sun synchronous orbit prioritization (from environment management criteria aspect) figure 19 sun synchronous orbit prioritization (from target surveillance criteria aspect) ijahp article: naseh/space mission definition based on analytical hierarchy process (ahp) method international journal of the analytic hierarchy process 158 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.583 figure 20 sun synchronous orbit prioritization (from change detection criteria aspect) figure 21 sun synchronous orbit prioritization (from cartography criteria aspect) figure 22 sun synchronous orbit prioritization (from meteorological criteria aspect) 5. conclusions in this paper, an application of ahp for space mission definitions (from orbit and application aspects) is presented. the schematic of ahp applied to the decision making problem is shown in figure 3. based on this and table 2, which shows the pairwise comparison matrix of criterion on a qualitative scale, the methodology was implemented and the results are shown in figures 15 to 22 (using expert choice software). the figures 15 to 22 show that ss and mss orbits each have a higher priority for the various missions. for example, figures 16, 17, 18 and 19 illustrate that ss orbits have higher priority for the application type, optical remote sensing, environment management and target surveillance, respectively. furthermore, figure 15 shows the higher priority of the mss orbits in the overall decision making (overall aspects) with the ahp’s objective (when time and cost affect the mission analysis). the methodology verification is performed based on the overall inconsistency in the results (figures 15 to 22) which is lower than 0.2. finally, sso in the meteorological missions has the most priority in the applications. ijahp article: naseh/space mission definition based on analytical hierarchy process (ahp) method international journal of the analytic hierarchy process 159 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.583 references bolle, a., circi, c. (2011). modified sun-synchronous orbits by means of solar sails. journal of recent patents on space technology, 1 (1), 72–79. doi: 10.2174/1877611611101010072 capderou m., translated by stephen lyle (2005). satellites orbits and missions. france: springer verlag. che, z. h. (2010). using fuzzy analytic hierarchy process and particle swarm optimization for balanced and defective supply chain problems considering weee/rohs directives. international journal of production research 48, 3355-3381. darko a., chang a. p. c., ameyaw e. e., owusu e. k., edwards d. j. (2018). review of application of analytic hierarchy process (ahp) in construction. international journal of construction management.. doi: https://doi.org/10.1080/15623599.2018.1452098 deng, x., y. hu, y. deng, and s. mahadevan (2014). supplier selection using ahp methodology extended by d numbers. expert systems with applications, 41, 156–167. doi: https://doi.org/10.1016/j.eswa.2013.07.018 emrouznejad a., marra m. (2016). the state of the art development of ahp (1979– 2017): a literature review with a social network analysis. international journal of production research, 55(22), 6653-6675. doi: https://doi.org/10.1080/00207543.2017.1334976 fan, g., d. zhong, f. yan, and p. yue (2016). a hybrid fuzzy evaluation method for curtain grouting efficiency assessment based on an ahp method extended by d numbers. expert systems with applications, 44, 289–303. doi: https://doi.org/10.1016/j.eswa.2015.09.006 fortescue p., swinerd g., stark j. (2011). spacecraft systems engineering, 4 th edition, john wiley & sons, ltd. doi: 10.1002/9781119971009 griffin michael d., french james r. (1991). space vehicle design, aiaa education series, american institute of aeronautics & astronautics. kim n., joungkoo p., jeong-ja c. (2017). perceptual differences in core competencies between tourism industry practitioners and students using analytic hierarchy process (ahp). journal of hospitality, leisure, sport & tourism education, 20, 76-86. doi: https://doi.org/10.1016/j.jhlste.2017.04.003 larson wily j. and wertz james r. (1999), space mission analysis and design. california, u.s.a.: microcosm press. maini a. k., agrawal v. (2011). satellite technology: principles and applications. second edition. john wiley & sons. ijahp article: naseh/space mission definition based on analytical hierarchy process (ahp) method international journal of the analytic hierarchy process 160 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.583 marfuah, widiantoro s. (2017). the implementation of analytical hierarchy process method for outstanding achievement scholarship reception selection at universal university of batam, international conference on environment and technology (ictech). doi: https://doi.org/10.1088/1755-1315/97/1/012003 naseh h., akhlaghi, a. m., mirshams, m., irani, s. (2011). an analytic hierarchy process based reliability allocation method (aram). international symposium on the analytic hierarchy process. sorrento, italy. naseh h., meibudy n. m., hamedian h. (2016), statistical model development for estimating the mass, dimension and power specifications of optical remote sensing satellite with the specified mission and orbital disturbance analysis. 15 th international conference of iranian society of aerospace engineering. tehran, iran. (in persian). ortore e., c. circi , f. bunkheila, c. ulivieri (2012). earth and mars observation using periodic orbits. journal of advances in space research, 49, 185–195. doi: https://doi.org/10.1016/j.asr.2011.09.016 ramanathan, r. (2006). data envelopment analysis for weight derivation and aggregation in the analytic hierarchy process. computers & operations research, 33, 1289–1307. doi: https://doi.org/10.1016/j.cor.2004.09.020 saaty, t.l. (1980). the analytic hierarchy process. new york, ny: mcgraw-hill. sevkli, m., lenny koh, s. c., zaim, s., demirbag, m., and tatoglu, e. (2007). an application of data envelopment analytic hierarchy process for supplier selection: a case study of beko in turkey. international journal of production research, 45, 1973– 2003. doi: https://doi.org/10.1080/00207540600957399 taherdoost h. (2017). decision making using the analytic hierarchy process (ahp) a step by step approach. international journal of economics and management systems, 2, 244-246. ulivieri, c., anselmo, l. (1991). multi-sun-synchronous orbits for earth observation. advances in the astronautical sciences, 76, 123–133. wang, y. m., k. s. chin, and j. p. f. leung (2009). a note on the application of the data envelopment analytic hierarchy process for supplier selection. international journal of production research, 47, 3121–3138. doi: https://doi.org/10.1080/00207540701805653 ijahp article: naseh/space mission definition based on analytical hierarchy process (ahp) method international journal of the analytic hierarchy process 161 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.583 appendix decision model definitions remote sensing satellites: the satellites are used for performing three types of space missions: monitoring missions, recognition missions and surveillance missions. technology complexity: remote sensing satellites have a variety of technologies in payloads, such as high resolution optical camera, synthetic aperture radar (sar), laser optic imaging and etc. power supply subsystem: the power supply subsystem has a very important role in the decision making because the remote sensing satellite will not be able to perform any mission without power. thermal control subsystem: the thermal control subsystem has a very important role in the remote sensing satellite because the performance of payload in the remote sensing satellite is influenced by the temperature and the thermal control subsystem and must provide a suitable range of temperature. remote sensing payload: there are a variety of remote sensing payloads in the space mission that can be selected based on the needs. for example, the sar payload selected for the remote sensing mission independent of local time (every time can be imaging from the earth) and optical camera selected for the remote sensing mission depending on local time (the earth imaging just performed in the day, not night). number and location of ground segment: the number and location of the ground segment is very important for communication aspects. communication approaches can be selected online or in the store and used in the remote sensing satellite. in the store and forward must be considered the hardware limitations for saving data. satellite operation, the type of scenarios for covering the earth for decision making regarding the overlap/without overlap of imaging in each repeat. application: there are various types of applications in the remote sensing satellite that will be able to service humans. these applications are environment and resources management, target surveillance, change detection, cartography, and meteorological missions. furthermore, these applications are selected based on human needs. revisit time: the revisit time is the elapsed time between the first and second tracking from the target region. local time: the local time is used for optical imaging. sensor calibration: sensor calibration is used for clearing the ephemerids drifts. overlap criteria: the overlap criteria is important for repeat ground tracking. regional coverage: the sso can be earth global coverage, and mss can be regional coverage. sun synchronous (ss): these orbits allow the observation of a given region of the planet at regular time intervals (periodic orbits) with approximately the same solar illumination conditions (sun-synchronism). multi-sun synchronous (mss): the mss orbits form a general category of orbits that are characterized by two periodicity conditions. the first condition is related to the observation of a given area (periodic orbit), and the second refers to the solar illumination, which repeats itself at regular time intervals (multi-sun synchronism). ijahp news and events: karpak/multiple criteria decision making (mcdm) 2019 was in turkey a second time international journal of the analytic hierarchy process 300 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.689 multiple criteria decision making (mcdm) 2019 was in turkey a second time the 25 th mcdm meeting was held in istanbul, turkey at istanbul technical university, june 16 june 21, 2019. it has been nearly twenty years since the last international conference on mcdm was held in turkey in ankara in 2000. the mcdm 2019 organizing committee is aware of the carbon footprint caused by the participants for their attendance to mcdm 2019, and to partially compensate for this they made donations on behalf of all participants to the tema foundation the turkish foundation for combatting soil erosion, for reforestation, and the protection of natural habitats (www.tema.org.tr). they also prepared environmentally friendly conference bags. environmentally friendly conference bag there were 261 participants from 5 continents and 39 countries at the conference. among all of the participants, 82 were students, 19 were from industries, and 6 were accompanying persons. the majority was from turkey (86) followed by the united states (23), the united kingdom and germany (12) and brazil and algeria each had 11 participants. there would have been even more participants from turkey since there are so many mcdm researchers there, but universities don’t pay registration fees if an international conference is in turkey! i found this conference very successful academically, socially and organizationally. it was amazing how excellent the organization was! i was even more astonished when i found out what kind of obstacles they faced during the process. for example, they even had to consider an alternative meeting place when most of the participants had already reserved their hotels around the current venue because their school was going to be demolished. even as recently as july 13, 2019 i found out that ilker topcu, the conference organizer, was transferring some office collections to his home since the http://www.tema.org.tr/ ijahp news and events: karpak/multiple criteria decision making (mcdm) 2019 was in turkey a second time international journal of the analytic hierarchy process 301 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.689 building was going to be demolished soon. how lucky we were that it did not happen before june 21 st ! yet, conference organizers were always smiling overcoming obstacles to prepare an impeccable conference for us. number of participants from different continents ijahp news and events: karpak/multiple criteria decision making (mcdm) 2019 was in turkey a second time international journal of the analytic hierarchy process 302 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.689 261 participants from 5 different continents conference chair, co-chairs, and organizing committee chair. from left to right, y. i̇lker topcu (istanbul technical university), özgür kabak (istanbul technical university), şule önsel ekici (dogus university, dean of engineering), and özay özaydın (dogus university). ijahp news and events: karpak/multiple criteria decision making (mcdm) 2019 was in turkey a second time international journal of the analytic hierarchy process 303 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.689 one of the unique features of this conference was that it has been organized by five universities in turkey; istanbul technical university (host), dogus university, koc university, sabanci university and beykent university. conference organizers managed to follow the traditions of the international society on mcdm and made the conference affordable to all and provided support to those who were short of funding. besides the organizers, the international society on mcdm also offered a budget for providing discounted fees to 26 participants, and creative decisions foundation (cdf) supported 22 participants by awarding grants that covered their whole conference fee. the support of the mcdm society and cdf facilitated so many students and young scholars’ participation in the conference. ilker topcu, conference general chair, presented a plaque of appreciation for both organizations for their support to the participants. murat köksalan, president, international society on mcdm, receives the plaque from ilker topcu (on the left), murat köksalan (on the right) https://www.mcdmsociety.org/ ijahp news and events: karpak/multiple criteria decision making (mcdm) 2019 was in turkey a second time international journal of the analytic hierarchy process 304 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.689 rozann w. saaty, president and executive director, creative decision foundation, shows the plaque she received from mcdm 2019 organizing committee tom saaty (1926-2017) was there in sprit. the creative decisions foundation was established in 1996 by thomas l. saaty and his wife rozann whitaker saaty to promote the cause of rational decision making and spread awareness of the analytic hierarchy process (ahp) theory of decision making created by dr. saaty. https://www.creativedecisions.net/ ijahp news and events: karpak/multiple criteria decision making (mcdm) 2019 was in turkey a second time international journal of the analytic hierarchy process 305 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.689 scientific program the conference consisted of 222 presentations and in 62 sessions. since it is quite challenging to summarize all of them, i will summarize the ahp/anp related sessions since that is what our audience is interested in. there were 12 sessions in ahp/anp stream, 3 special sessions, 2 tutorial sessions, 1 invited parallel session, and 6 contributed parallel sessions. the three special sessions were: ahp in practice as a ‘’decision conference’’ in turkey sponsored by arama; ahp/anp applications in production and manufacturing – sponsored by borcelik; ahp/anp applications in industry 4.0 sponsored by çi̇mtaş. the one invited session was on theory and applications of anp; the six contributed sessions were consistency issues in ahp, ahp/anp applications in academia and education, ahp/anp applications in urban and regional development, practical applications of ahp/anp, ahp/anp applications in aviation and business applications of ahp/anp. i usually attend presentations in different areas of mcdm, not only ahp/anp related to sessions in order to fertilize different ideas from different areas of mcdm, yet this time there were so many very interesting presentations on ahp/anp i had to confine myself to the ahp/anp presentations; i even missed some cdf awardees presentations. i was especially saddened that i missed ahp in practice as a ‘’decision conference’’ in turkey sponsored by arama, since i knew from my past experience that this decision conference was solving practical problems together with the decision makers and i am very much interested in this. invited plenary talks the 4 plenary talks were delivered by;  pınar keskinocak, georgia institute of technology, usa: “quantitative models for decision-support in healthcare applications”  kathrin klamroth, university of wuppertal, germany: “multi-objective combinatorial optimization – beyond the bi-objective case”  tuomas sandholm, carnegie mellon, usa: “new results for solving imperfectinformation games”  luis vargas, university of pittsburgh, usa: “voting with intensity of preferences” ijahp news and events: karpak/multiple criteria decision making (mcdm) 2019 was in turkey a second time international journal of the analytic hierarchy process 306 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.689 keskinocak asserts that health systems focus on a broad set of activities including prevention, screening, diagnosis, and treatment. most health systems face multiple objectives and limited resources while striving to provide high quality care and promote health and well-being. data-driven quantitative models, embedded into decision-support systems, have significant potential for improving patient and population outcomes, efficiency, and effectiveness in healthcare. in this presentation, she discussed a few applications from organ transplant, vaccination, screening, and workforce allocation decisions. klamroth stated that multi-objective combinatorial optimization (moco) is a quickly growing field that is highly relevant for a multitude of application areas and at the same time highly challenging due to its inherent complexity. typical examples of moco problems include multiobjective knapsack and assignment problems, the multi-objective travelling salesman problem, and network problems like multi-objective minimum spanning tree, shortest path, and minimum cost flow problems. starting with a review of recent developments in the field of moco, she analyzed the main drivers of the complexity of moco problems and identified cases where moco problems are actually easy. she discussed general solution concepts such as generic scalarization based algorithms, branch and bound methods, and upper and lower bound sets. ijahp news and events: karpak/multiple criteria decision making (mcdm) 2019 was in turkey a second time international journal of the analytic hierarchy process 307 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.689 sandholm contends that most real-world settings are imperfect-information games. they present challenges beyond those in perfect-information games. he discussed some the more recent work on imperfect-information games. topics include a unified framework for abstracting games with bounds on solution quality, a sound depth-limited search framework, the fastest equilibrium-finding algorithms, deep learning as an alternative to abstraction, a general framework for online convex optimization for sequential decision processes and extensive-form games, the first scalable algorithm for tremblinghand equilibrium refinements, and trembling-hand refinement of stackelberg equilibria. last but not least was the talk from vargas. he explained the method he developed based on the idea of pairwise voting to rank projects or candidates and incorporate them into the ranking process. this focused on how strongly the referees/voters feel about the comparisons they make. voting is a modified form of ranking and all the votes are equally important. however, there are situations in which the votes are not just ordinal, but each voter expresses an intensity of preference for the different candidates, e.g., ranking projects for funding. vargas showed that this method yields the same results as ordinal voting in large populations when the intensity of preferences becomes extreme. voting with intensity of preferences does not violate democracy, but softens the stand of voters and allows for consideration of the diversity of issues involved in voting. ijahp news and events: karpak/multiple criteria decision making (mcdm) 2019 was in turkey a second time international journal of the analytic hierarchy process 308 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.689 tutorials the 3 tutorial sessions consisted of 2 tutorials delivered by orrin cooper and pekka j. korhonen. orrin cooper, university of memphis, usa: “improve your decisions by learning and experiencing the anp best practices.” special session sponsored by creative decision foundation, a tribute to thomas l. saaty, and it was a great opportunity to learn anp. pekka j korhonen, aalto university, finland: “molp vs. dea – relatives or friends” doctoral dissertation award session the doctoral dissertation award session took place on june 19 th at mustafa kemal conference hall. the finalists this year were:  andreia guerreiro, university of coimbra, portugal  fritz bökler, osnabrück university, germany  mohammad ghaderi, pompeu fabra university, spain (via skype) as announced during the conference banquet, the doctoral dissertation award was given to fritz bökler. ijahp news and events: karpak/multiple criteria decision making (mcdm) 2019 was in turkey a second time international journal of the analytic hierarchy process 309 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.689 business meeting the business meeting was held on june 20 th at mustafa kemal conference hall. murat köksalan chaired the meeting. the highlight of the business meeting was the announcement of the next conference location. the mcdm2021 conference will take place in portsmouth, uk (tentatively) from june 27 july 2, and it will be chaired by alessio ishizaka and co-chaired by banu lokman. society awardees during the conference banquet, international society on mcdm awardees and the mcdm doctoral dissertation awardee were announced. the awardees were:  the mcdm gold medals: o margareth wiecek, clemson university, usa o michalis doumpos, technical university of crete, greece  the mcdm edgeworth-pareto award: o ahti salo, system analysis laboratory, finland  the georg cantor award o kathrin klamroth, university of wuppertal, germany  the mcdm doctoral dissertation award o fritz bökler, osnabrück university, germany society awardees gave their talks on june 21 th at itu management faculty building. publications selected abstracts that were presented at the conference were invited to be considered as chapters in the book titled “multiple criteria decision making beyond the information age: selected works from mcdm 2019 istanbul” within the springer‘s book series “contributions to management science”. participants also have the opportunity to submit ijahp news and events: karpak/multiple criteria decision making (mcdm) 2019 was in turkey a second time international journal of the analytic hierarchy process 310 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.689 their manuscripts in the special issues of journal of enterprise information management, journal of global optimization, international journal of the analytic hierarchy process, journal of multi-criteria decision analysis, annals of operations research, and optimization and engineering. i urge everyone not to miss this opportunity. closing session the closing session took place on june 21 st at the itu management faculty building. murat köksalan (top left), matthias ehrgott (top right), and ilker topcu (bottom) at the closing session social events in addition to the high-quality scientific program based on the contributions of the presenters, organizers made the conference even more memorable by organizing attractive social events. on sunday evening, the “welcome reception” took place at “havuzbasi” on the istanbul technical university macka campus. participants were offered plentiful meze and finger foods along with unlimited drinks. the creative decision foundation’s award reception, held monday evening, was also at this location. ijahp news and events: karpak/multiple criteria decision making (mcdm) 2019 was in turkey a second time international journal of the analytic hierarchy process 311 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.689 creative decision foundation award reception, june 17, 2019 rozann saaty addressing cdf awardees for the “outing on wednesday”, visits to sakıp sabanci museum and emirgan park were organized in the afternoon. sabancı university’s sakıp sabancı museum, a private fine arts museum, is located at one of istanbul’s oldest settlements on the bosphorus river. the museum presents a versatile museological environment with its rich permanent collection, the comprehensive temporary exhibitions, and its conservation units. emirgan park, one of the biggest parks in istanbul, is the homeland of tulips. the park includes approximately 90 species of trees and 3,000 varieties of tulips. it is said to be the most beautiful park in istanbul. ijahp news and events: karpak/multiple criteria decision making (mcdm) 2019 was in turkey a second time international journal of the analytic hierarchy process 312 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.689 sabanci museum birsen karpak and rozann saaty ijahp news and events: karpak/multiple criteria decision making (mcdm) 2019 was in turkey a second time international journal of the analytic hierarchy process 313 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.689 after visiting the museum and the park, a dinner cruise on the bosphorus was organized. the guided boat ride on the bosphorus was a unique experience for seeing the beauties of istanbul including palaces, monuments, and mansions accompanied by trees and flowers along the european and asian sides of the bosphorus. during the cruise, a buffet dinner was also served and participants enjoyed traditional turkish street foods, unlimited local beer and wine, and dj music. participants were chatting with each other exchanging ideas and dancing! . dinner cruise birsen karpak, serpil sayin and pinar keskinocak chatting and enjoying the beauty of bosphorus at night ijahp news and events: karpak/multiple criteria decision making (mcdm) 2019 was in turkey a second time international journal of the analytic hierarchy process 314 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.689 the conference banquet was held at the historical sait halim paşa mansion on thursday evening. the mansion, located at the bosphorus and overlooking the mesmerizing seafront, was built in 1878. at this mansion 19 th -century architectural elegance meets 21 st -century service quality. the banquet started with a reception, and was followed by a four-course dinner executed by an award-winning executive chef. participants enjoyed live lounge, jazz, latin, and oldies music by yosun and aid band, and then live local music by a balkan band. a view from the banquet at sait halimpasha mansion ijahp news and events: karpak/multiple criteria decision making (mcdm) 2019 was in turkey a second time international journal of the analytic hierarchy process 315 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.689 yes, this was mcdm 2019 beyond the information age! first and second generation mcdm people at the banquet. fusun ulengin, murat koksalan, pekka korhonen, jyrki wallenius, ralph steuer, and birsen karpak are pioneers of mcdm; ilker topcu, sule onsel ekici, serpil sayin, ozgur kabak are the second generation. ijahp news and events: karpak/multiple criteria decision making (mcdm) 2019 was in turkey a second time international journal of the analytic hierarchy process 316 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.689 more second generation mcdm at the banquet that night, jyrki wallenius, chairman of the awards committee, announced the awards of the society. jyrki wallenius and murat köksalan (top left); society awardees (top right); doctoral dissertation finalists (bottom row) ijahp news and events: karpak/multiple criteria decision making (mcdm) 2019 was in turkey a second time international journal of the analytic hierarchy process 317 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.689 scientific program committee members twenty-seven scientific program committee members came from every continent except australia! they were emel aktaş (cranfield university), umut asan (istanbul technical university), adiel teixeira de almeida (federal university of pernambuco), sarah ben amor (university of ottawa), matthias ehrgott (lancaster university), gabriele eichfelder (tu ilmenau), josé rui figueira (technical university of lisbon), salvatore greco (university of catania), zahir irani (university of bradford), özgür kabak (istanbul technical university), birsen karpak (youngstown state university), kathrin klamroth (bergische universität wuppertal), murat köksalan (middle east technical university), banu lokman (middle east technical university), kaisa miettinen (university of jyväskylä), enrique mu (carlow university), özay özaydın (doğuş university), serpil sayın (koç university), hsu-shih shih (tamkang university), johannes siebert (university of bayreuth), seyhan sipahi nişel (istanbul university), theo stewart (university of cape town), i̇lker topcu (istanbul technical university), füsun ülengin (sabancı university), jyrki wallenius (aalto university), margaret wiecek (clemson university),constantin zopounidis (technical university of crete). i would like to express my gratitude to my friends, y. i̇lker topcu, mcdm 2019 conference general chair, general co-chairs özgür kabak and özay özaydın, organizing committee chair şule önsel ekici and program committee chair serpil sayın, conference secretary çigdem kadaifçi, and organizing committee members sezi çevik onar, bilal ervural, sait gül, gül i̇mamoğlu, berk küçükaltan, başar öztayşi, seda yanık, özgür yanmaz, and umut asan. special recognition is due hardworking volunteer students dilek hafızoğlu, nihat can sinayiş, i̇brahim küçükçiçek, musa m. almasri, atakan burak birbir, fatime i̇rem yağan, yeliz çotoy, esma nur acungil, damla erden, and kerim uygur kızıl. conference chair, co-chairs, organizing and program committee chairs from left to right, özay özaydın (dogus university), serpil sayın (koc university), y. i̇lker topcu (istanbul technical university), şule önsel ekici (dogus university), and özgür kabak (istanbul technical university). ijahp news and events: karpak/multiple criteria decision making (mcdm) 2019 was in turkey a second time international journal of the analytic hierarchy process 318 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.689 i also would like to thank the conference organizing committee for selecting a competent professional conference organizer, cormep; event planner, beyaz event; securing the sponsorship of the international society on mcdm, creative decisions foundation, and tübi̇tak (the scientific and technological research council of turkey); and obtaining so many session sponsors (çimtaş pipe fabrication and trading, borcelik steel industry trade, arama participatory management consulting; food, beverage, product, and service sponsors sakıp sabancı museum, mavi, springer, anadolu efes, coca cola i̇çecek, dimes, muratbey). all contributed to the success of this conference. it is such a success considering the current economic conditions of turkey. i also wish to thank to the plenary speakers, the tutorial speakers, the invited session organizers, the session chairs, and all the presenters and participants who contributed to the organization of such a wonderful conference! last but not least, i would like to thank my friend ilker topcu who provided the basic information for this news article. anything you appreciate belongs to him, and any mistakes or inadvertent missing information belongs to me. birsen karpak, ph.d. distinguished professor of management, http://ysu.edu/bkarpak associate editor, international journal of the analytic hierarchy process scientific program committee member, mcdm2019 www.mcdm2019.org; instagram.com/mcdm2019; twitter.com/mcdm2019 http://ysu.edu/bkarpak http://www.ijahp.org/ http://www.ijahp.org/ http://www.mcdm2019.org/ http://www.mcdm2019.org/ http://www.instagram.com/mcdm2019 http://www.twitter.com/mcdm2019 ijahp article: sari, wan mohamed, ayu jalil/conceptual model for the implementation of electric ground support equipment in ground handling activities in indonesia international journal of the analytic hierarchy process 170 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.825 conceptual model for the implementation of electric ground support equipment in ground handling activities in indonesia mustika sari malaysia institute of transport, universiti teknologi mara, malaysia 40450 shah alam, selangor darul ehsan, malaysia institute transportation and logistic trisakti, jakarta, indonesia mustika0017@gmail.com wan mazlina wan mohamed malaysia institute of transport, universiti teknologi mara, malaysia 40450 shah alam, selangor darul ehsan, malaysia wmazlina@uitm.edu.my siti ayu jalil malaysia institute of transport, universiti teknologi mara, malaysia 40450 shah alam, selangor darul ehsan, malaysia ayu090@uitm.edu.my abstract this research aims to develop a model for the implementation of electric ground support equipment (e-gse) in indonesia. it employed the qualitative and quantitative method. data were collected through interviews with airport stakeholders including ground handling, airline and airport experts. the data analysis used the analytical network process (anp) with super decisions software tools. the anp is a mathematical theory that can be used to analyze the influence of approaches and assumptions used to solve related problems. the key stakeholders from the airlines, ground handling, and airport authority agreed that the critical criteria to implement electric ground support equipment (e-gse) are emissions standards, pollution sources, limit value, employee exposure, and stationary measurements. the alternative solutions are regulation, company finance, and infrastructure. those criteria and alternative solutions are in line with the indonesian government’s regulation, supported by provision of the governor, concerning vehicle tax incentives for electric battery vehicles to support ground handling companies investment in electric ground support equipment (e-gse) to reduce emissions in airports. keywords: emission; regulations; electric vehicle; ground operation 1. introduction an airport is a place for runways and takeoffs, landings and aircraft maintenance. the aviation industry is strongly influenced by air traffic conditions, service disruptions, weather, and mechanical disruptions (wei & gosling, 2013). more than 2 billion mailto:mustika0017@gmail.com mailto:ayu090@uitm.edu.my ijahp article: sari, wan mohamed, ayu jalil/conceptual model for the implementation of electric ground support equipment in ground handling activities in indonesia international journal of the analytic hierarchy process 171 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.825 passengers travel around the world on airplanes and airports are part of a country's development that has socio-economic and environmental impacts. due to the increasing demand for air transportation, the aviation industry is predicted to continue to grow. higher funding, development, repair and maintenance of airports need to be maintained (sameh & scavuzzi, 2016). airport operators are responsible for providing the facilities and services of the air transportation industry. these responsibilities include service terminals, land transportation facilities, the provision of train terminals and cargo transportation between flight terminals and passenger terminals. they are also responsible for managing commercial airports, land or airport transportation, parking infrastructure, airport entry access, and other related factors outside the airport terminal boundaries (graham, 2009). ground handling helps the fleet of aircraft on the ground. accuracy, speed, safety, security and cost-effectiveness are crucial to minimize turnaround time. ground handling also coordinates the movement of aircraft at the terminal gate during the time of departure. equipment on land such as a ground power unit, pushback car, cargo and passenger boarding stair are known as ground support equipment (gse). all these ground handling vehicles use diesel in their operations to cut costs; however, they create more pollution in the area during operation. consequently, emissions planning for equipment at airports is very important. multiple companies have attempted to switch their focus to fuel operation or be updated for ground operating vehicles (aerospace, 2012). the main environmental problems associated with airport transfers are emissions, noise, land use, and energy consumption (sameh & scavuzzi, 2016). air transportation contributes greatly to air pollution, accounting for 12 percent of total emissions, while human-caused carbon emissions account for 26.4 percent (icao, 2010). air pollution comes from diesel fuel and dust when aircraft are on the ground. emissions from diesel fuel occur when an airplane refuels and departs and varies based on the type of aircraft. emissions from dust originate from brakes, tires and asphalt (nadia & mantecchini, 2014). the main sources of air pollution in air transportation are aircraft engines and ground support equipment that use diesel fuel, such as nitrogen oxides (nox), volatile organic compounds (voc), polycyclic aromatic hydrocarbons (pah), sulfur dioxide (so2), and other materials (kristensen et al., 2012). the invention of electric vehicles is an effective method for lowering carbon emissions in the transportation sector. however, completely replacing fossil fuels with electricity or battery power is a challenge for the transportation industry. by employing the vehicle to grid (v2g) concept, electric transportation can deliver stored energy to the electric grid. electric vehicles can be classified into the following categories: battery electric vehicles (bev), hybrid electric vehicles (hev), plug-in-hybrid electric vehicles (phev), and extended range-electric vehicle (erev) (yuan et al., 2017). electric batteries that are below the speed of 40 miles per hour with zero emissions can be applied to cars. phev are similar to hev with the additional feature of a charging ijahp article: sari, wan mohamed, ayu jalil/conceptual model for the implementation of electric ground support equipment in ground handling activities in indonesia international journal of the analytic hierarchy process 172 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.825 port, which is usually plugged into an electrical outlet. some of the features of phev are improved fuel efficiency and lower emissions. bev are fully electrically-operated, and the electrical outlet charging feature makes it a vehicle without emissions. an erev is a combination of a battery electric vehicle and a plug-in hybrid electric vehicle with better efficiency (shaukat et al., 2018). for many years, because of the need of countries to improve the economy and lives of their inhabitants, fossil fuels such as petroleum, coal and gas have been used on a large scale. however, fossil fuels are one of the main culprits causing ozone depletion and air pollution. one solution to reducing pollution is the use of electric vehicles because electric fuel produces zero emissions. (feng & wang, 2000). emissions are generated from local power plants that use electricity to provide power to vehicle equipment at airports. these emissions are often released when using land support equipment. equipment to support aircraft activities on the ground that use diesel fuel significantly contribute to pollution at airports (airport cooporative research program, 2012). in indonesia, aircraft handling procedures consist of catering, interior handling, refueling, maintenance, passenger handling, transportation, and others. one effort that can be made to achieve high performance is the improvement of maintenance performance (nugroho et al., 2012). despite several policies, mainland implementation companies do not have proper regulations regarding the use of environmentally friendly fuels. in airport activities, diesel ground support equipment can cause pollution, so the invention of electric ground support equipment can reduce pollution at airports (kusnoputranto & kristanto, 2014). in response to climate concerns and a rising interest in electric vehicles, the transportation sectors of southeast asian countries are shifting towards the electric vehicle (ev) industry and ecosystem. the popularity of electric cars has encouraged a transition away from fossil fuels and has spread to indonesia, where electric power adoption is on the rise. the countries have established comprehensive electric vehicle policies that take a holistic approach to developing the entire ev ecosystem by promoting demand and incentivizing private investment. currently, there are 30 electric vehicle charging ports at soekarno hatta airport, around 200,000 aircraft passengers and 50,000 workers and visitors (non-passengers) who arrive every day, making this airport a strategic location for a campaign for the use of electric cars. the airport authority and indonesia power will provide the infrastructure for charging electric car batteries known as public electric vehicle charging stations. the transportation minister has released a statement restricting the use of ground support equipment to support airline activities on the ground. the challenge of this regulation arises from the limit on the age of the gse because it costs a large amount of money for ground handling companies to rejuvenate their equipment (sucipto, 2017). electric vehicles have a bright future in indonesia. in recent years, the indonesian government has made a concerted effort to increase the use of electric vehicles in the country. electric vehicles are used in some airports, such as taxis, buses, skytrains, and ijahp article: sari, wan mohamed, ayu jalil/conceptual model for the implementation of electric ground support equipment in ground handling activities in indonesia international journal of the analytic hierarchy process 173 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.825 ground support equipment. despite the numerous benefits of using electric vehicles in airports, there is no plan in place in indonesia to use electric ground support equipment. 2. research method first, an in-depth interview was conducted to gain a comprehensive understanding of the problems, and data were collected from the ground handling company. next, comprehensive pairwise questionnaires were conducted before coming to conclusions (rusdiyana & devi, 2013). the population was chosen based on non-probability as well as a purposive sampling technique. the total population consisted of ten respondents from the airlines, airport authority and a ground handling company. the interview utilized a scale that is commonly applied to measure statements, and the selected value represents the intensity of the judgment. the range has been chosen based on the stimulus-response theory and validated for effectiveness. table 1 ultimate measure of the complete amount strength of importance description justification 1 likely significant two activities equally influence the objective 3 relatively significant experience and judgment prefer one activity more than the other 5 solidly significant ability and assessment strongly support one activity over the other 7 precisely significant very strong activity, proven by practice 9 excessively significant evidence favors one activity over the other 2,4,6,8 agreement between the above values need to add a numerical assessment because there is no right word to describe it source : (saaty, 2016) the analytical network process (anp) is the tool for analysis, and the process utilizes super decisions tools. the anp is a mathematical theory that can analyze the influence of approaches and assumptions to solve related problems. the anp method thoroughly explains the representation of related factors and responses. the method involves three steps. first, questionnaires are distributed and in-depth interviews with experts are conducted to identify essential significant factors. second, the results are collected and an anp network is created to develop a questionnaire. third, the analysis is applied to create alternative solutions, priorities and strategic policies (haura et al., 2016). ijahp article: sari, wan mohamed, ayu jalil/conceptual model for the implementation of electric ground support equipment in ground handling activities in indonesia international journal of the analytic hierarchy process 174 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.825 figure 1 analytic network process (ascarya, 2011) for the research process, the criteria and alternatives are the most important for the feasibility of using e-gse in the aviation industry, and the research stages will be carried out as shown in figure 1, such as cluster and criteria (node). there are three phases or stages of research to be conducted, namely; construction of the model, quantification of the model, synthesizing and analyzing the data. the geometric mean is a type of average calculation that shows a certain tendency or value, and has the following formula: (∏ =𝑛𝑖 1𝑎𝑖 ) 1/𝑛 = √𝑎1𝑎2,,𝑎𝑛 𝑛 n = respondent 1.…….. n i = pairwise 1……..i to make a pairwise comparison, super decisions software version 2.10.0 was used. this software is commonly used in anp research. examples of questionnaires for conducting pairwise comparisons are shown in figure 2. figure 2 sample questionnaire for pairwise comparison ijahp article: sari, wan mohamed, ayu jalil/conceptual model for the implementation of electric ground support equipment in ground handling activities in indonesia international journal of the analytic hierarchy process 175 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.825 3. results and discussion the survey was processed for each informant using the anp super decisions software, and produced three super-matrices that prioritozed the order of significant problems and their respective solutions. furthermore, the results were divided into groups of practitioners, academics, and regulators to produce the order of priority. the average value and pattern were used to determine the priority order. clusters were divided into three categories of goal, criteria, and alternatives. the goal was the use of electric vehicle equipment in the airline industry, while criteria and nodes were the limit values, emissions standards, employee exposure, stationary measurements, and pollution sources. the alternatives referred to company finances, regulations, and infrastructures. figure 3 structure model anp the results from the survey were processed for each respondent using the anp framework to decipher the important criteria and alternatives for the use of electric vehicle equipment in the airline industry. the framework model also formed the basis of the questionnaires, and the data that were processed produced two matrices that gave priority order and appropriate alternatives from the perspective of each respondent. in this study, there were ten respondents with airline, ground handling, and airport expertise who had different ideas and perspectives. therefore, the anp in the super decisions software presented the results that determined the overall priority order, which was then divided into three groups (clusters) of experts, namely the aviation industry, a ground handling company, and the airport authority. 3.1 expert perspective from the aviation industry in this discussion, the first research question will be answered regarding the most important criteria for the feasibility of using the e-gse in the aviation industry from the perspective of the airlines. the results show that the obtained criteria are divided into five categories, namely, the limit values, emissions standards, employee exposure, stationary measurements, and pollution sources. figure 4 shows the result of the interviews with criteria 1: limit values the goal: the conceptual model in the implementation of electric ground support equipment criteria 2: emission standard criteria 4: stationary measurement criteria 3: employee exposure criteria 5: pollution sources alternative 1 : company financial alternative 2 : regulation alternative 3 : infrastructure ijahp article: sari, wan mohamed, ayu jalil/conceptual model for the implementation of electric ground support equipment in ground handling activities in indonesia international journal of the analytic hierarchy process 176 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.825 airlines experts with the positions of pilot, corporate analyst, and operations manager. figure 4 criteria from airlines experts the results showed that the essential geometric mean (gmk) value is 0.262 for the pollution sources category, which becomes an important issue for the feasibility of using the e-gse from the perspective of an airlines expert. the second rank for the problem of e-gse use is the limit value with a geometric mean score of 0.248, while that in the third place is emissions standards at 0.208. furthermore, employee exposure is in fourth place with a value of 0.174, and in last place is stationary measurements with a geometric mean value of 0.106. figure 5 alternative solution from airlines experts meanwhile, the alternative solutions related to the feasibility of using e-gse in the aviation industry from the perspective of the airlines are divided into three alternatives, namely, company finance, regulation, and infrastructure. the results showed that the crucial geometric mean (gmk) value is 0.459 in regulation for the feasibility of using the e-gse from the perspective of the airlines. the second ranked alternative is infrastructure with a value of 0.280, and the ultimate alternative solution is company finance with a geometric mean value of 0.259. 3.2 expert perspective from the ground handling perspective after discussing the related criteria and alternative solutions, the next important criteria will be addressed regarding the feasibility of using e-gse in the aviation industry from the combined perspective of four ground handling experts. 0.248 0.208 0.174 0.106 0.262 0 0.05 0.1 0.15 0.2 0.25 0.3 1limit value 2emission standard 3employee exposure 4stationary measurements 5pollution sources criteria 0.259 0.459 0.280 0 0.1 0.2 0.3 0.4 0.5 1company financial 2regulation 3infrastructure alternative solutions ijahp article: sari, wan mohamed, ayu jalil/conceptual model for the implementation of electric ground support equipment in ground handling activities in indonesia international journal of the analytic hierarchy process 177 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.825 figure 6 criteria from ground handling experts the results showed that the most critical geometric mean (gmk) value is 0.256 in the emissions standards category. the second ranked criterion is pollution sources with a mean geometric value of 0.230, while the third priority criterion is stationary measurement with a mean geometric value of 0.212. the fourth ranked criterion for the e-gse feasibility problem from the perspective of ground handling experts is the limit value with a geometric mean value of 0.161, which is similar to employee exposure with a value of 0.138. figure 7 alternative solutions from ground handling experts meanwhile, the alternative solutions from the perspective of ground handling experts show the essential geometric mean (gmk) value as 0.459 in the regulation category. this is similar to the result from the perspective of the airlines. the second ranked alternative is company finance with a mean geometric value of 0.310, and the last alternative solution is the infrastructure with a geometric mean value of 0.229. however, this result is essential and ranks second when compared to the element of company finance. 3.3 expert perspective from the airport authority the next criteria will be addressed regarding the feasibility of using the e-gse in the aviation industry from the perspective of the airport authority. the results showed that the priority of each criterion is divided into five categories, namely, limit values, emissions standards, employee exposure, stationary measurements, and pollution sources. 0.161 0.256 0.138 0.212 0.230 0 0.05 0.1 0.15 0.2 0.25 0.3 1limit value 2emission standard 3employee exposure 4stationay measurements 5pollution sources criteria 0.310 0.459 0.229 0 0.1 0.2 0.3 0.4 0.5 1company financial 2regulation 3infrastructure alternative solutions ijahp article: sari, wan mohamed, ayu jalil/conceptual model for the implementation of electric ground support equipment in ground handling activities in indonesia international journal of the analytic hierarchy process 178 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.825 figure 8 criteria from airport authority experts the results showed that the most important geometric mean (gmk) value is 0.337 in the emissions standards category, which is therefore an important issue in the feasibility of using the e-gse in the aviation industry and is similar to the ground handling perspective. the second ranked criterion is stationary measurement with a geometric mean value of 0.195, while the third priority criterion is the pollution source with a value of 0.189. the fourth ranked crtierion from the perspective of airports is employee exposure with a geometric mean value of 0.166, and the last is the limit value of 0.111. figure 9 alternative solutions from airport authority expert meanwhile, for the alternative solutions related to the feasibility of using e-gse from the airport authority perspective, the most important geometric mean (gmk) value is 0.402 in the regulation category. these results are similar to all the experts opinions. the second ranked e-gse feasibility solution is infrastructure with a geometric mean value of 0.372, and company finance has a geometric mean value of 0.224. these are relatively similar to the results from the airlines perspective, whereby infrastructure is essential and ranks second when compared to company finance. 0.111 0.337 0.166 0.195 0.189 0 0.1 0.2 0.3 0.4 1limit value 2emission standard 3employee exposure 4stationary… 5pollution sorces criteria 0.224 0.402 0.372 0 0.1 0.2 0.3 0.4 0.5 1company financial 2regulation 3infrastructure alternative solutions ijahp article: sari, wan mohamed, ayu jalil/conceptual model for the implementation of electric ground support equipment in ground handling activities in indonesia international journal of the analytic hierarchy process 179 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.825 figure 10 summary of criteria from the airlines, ground handling, and airport authority perspectives figures 10 and 11 give a summary of the three stakeholder’s perspectives for the criteria. the highest rank is for emissions standards with a value of 0.57 and the alternative solution, regulation, has a value of 1.05. the emissions standards and regulation criteria are the most important for the conceptual model for the implementation of the use of electrical ground support equipment at soekarno hatta airport, jakarta, indonesia. figure 11 summary of alternative solutions from the airlines, ground handling, and airport authority perspectives 3.4 discussion according to scheelhaase (2018), it is essential to review the sources of emissions from aircraft operations. these emissions are carbon dioxide (co2), nitrogen oxides (nox), sulfur oxides (sox), water vapor (h2o), and aerosols. currently, international and domestic market measures only regulate co2 emissions in aviation. the intention of carbon offsetting and reduction scheme for international aviation (corsia) and european emissions trading scheme (eu ets) for air transportation is to reduce co2 emissions in aviation. however, there are other criteria, such as emissions measuring instruments at airports that companies need to invest in for the long-term according to corsia and iata standards (scheelhaase, 2018). from an economic perspective, the costs are very high. fuel costs for an internal combustion engine (ice) gse are an important variable. for large airports, the use of land support equipment is significant and a large amount of fuel is used; therefore, the emissions released are substantial. the investment in land supporting equipment is growing for large airports (morrow et al., 2007). the ground handling experts suggest that to solve pollution-related problems, control and 0 0.1 0.2 0.3 0.4 0.5 0.6 1 limit value 2 emission standard 3 employee exposure 4 stationay measurements 5 pollution sources criteria 0 0.2 0.4 0.6 0.8 1 1.2 1 company financial 2 regulation 3 infrastructure alternative solution ijahp article: sari, wan mohamed, ayu jalil/conceptual model for the implementation of electric ground support equipment in ground handling activities in indonesia international journal of the analytic hierarchy process 180 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.825 management should be in line with presidential law no.55.this law states that indonesia is committed to encouraging the acceleration of the battery-based motor vehicle program to support the reduction of greenhouse gas emissions; increase energy efficiency, energy security, and energy conservation in the transportation sector; realize clean energy and clean air; and encourage acceleration of use. airlines are undergoing some challenges such as cost pressures, compliance, and operational issues caused by competition and growth in the industry. ground support equipment (gse) are directly affected by these challenges, including the rising cost of fuel and pressure to reduce air pollutants. many airlines, power utilities, and other gse industry stakeholders are examining the cost-effectiveness of using e-gse versus the alternatives of gasoline and diesel-fueled internal combustion engine (ice) (morrow et al., 2007). research on airports shows that they are responsible for the pollution generated at the local level which contributes to global warming. local air pollution is determined by the amount of gases emitted by aircraft during their landing and takeoff period, according to icao annex 16, volume 2. the latter is divided into four stages, according to icao standards: take–off (0.7 minutes), climb (2.2 minutes to 3,000 feet above ground), approach (four minutes from 3,000 feet to landing), and idle (when the aircraft is taxiing or standing on the ground with engines on) (grampella et al., 2017). the analytic hierarchy process and analytic network process can be applied to the study of transportation even though the method is not specific to the feasibility of using electric ground support equipment. in this study, the analytical network process (anp) method was used to develop the conceptual model for the implementation of using electric fuel for ground support equipment by interviewing several people who are experts in the airlines, ground handling, and the airport authority. this method is considered appropriate for the subject under study because it creates a concept model that explains the global idea of involvement of individuals or groups and its development. the most important parameters in this conceptual model are pollution requirements, and the most important alternative approach is regulation. regulation is based on presidential regulation no. 55 from 2019, and emissions standards are based on the international air transport association (iata) and carbon offsetting and reduction scheme for international aviation (corsia) standards. 4. conclusion the three stakeholders from the airlines, ground handling, and airport authority concluded that the most critical criteria for the implementation of electric ground support equipment are emissions standards, followed by pollution sources, limit value, stationary measurements, and employee exposure. the ranked alternative solutions are regulation, company finance, and infrastructure. to support airports, the airlines and ground handling agents in the implementation of the use of electric hevicles in ground support equipment, the government provided presidential regulation no.55 in 2019 outlining the fast track program for battery-powered electric vehicles for road transportation. the provision of the regulation from the governor discusses vehicle tax duty incentives for motorized battery-based vehicles (bev) since their usage will decrease pollution. ijahp article: sari, wan mohamed, ayu jalil/conceptual model for the implementation of electric ground support equipment in ground handling activities in indonesia international journal of the analytic hierarchy process 181 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.825 references aerospace. (2012). reducing the environmental impacts of ground operations and departing aircraft: an industry code of practice. practice working group. airport cooperative research program (acrp) (2012). airport ground support equipment (gse): emission reduction strategies,inventory, and tutorial. washington, dc: the national academies press. https://doi.org/10.17226/22681. ascarya. (2011). the persistence of low profit and loss sharing financing in islamic banking: the case of indonesia. the 4th bank indonesia’s islamic banking reserch forum. doi: https://doi.org/10.13033/isahp.y2009.087 feng & wang. (2000). performance evaluation for airlines including the consideration of financial ratios. journal of air transport management, 6(3), 133–142. doi: https://doi.org/10.1016/s0969-6997(00)00003-x graham, a. (2009). managing the airports, an international perspective (third ed). butterworth-heinemann. grampella, m., martini,g., scotti, d.,tassan, f., zambon, g. (2017). determinants of airports’ environmental effects. transportation research, 50, 327–344. doi: https://doi.org/10.1016/j.trd.2016.11.007 haura, a., baga, l.,tanjung, h. (2016). analysis of cash waqf management on sharia financial service cooperation. analytical network process approach, 3(1), 89–105. icao. (2010). icao environment report. kristensen, k.,brogaard, l.,jacobsen, j., ellerman, t. (2012). air pollution in airports. the danish eco counci, 1(1). kusnoputranto & kristanto. (2014). capacity evaluation of environmental noise for airport sustainability through eco-airport. research journal of science & it management, 03(03), 20–27. morrow, k., hochard, d., franchort, j. (2007). cost benefit analysis modeling tool for electric vs . ice airport ground support equipment — development and results. doi: https://doi.org/10.2172/911917 nadia, m., & mantecchini, l. (2014). a transport carbon footprint methodology to assess airport carbon emissions. journal of air transport management, 37, 76–86. doi: https://doi.org/10.1016/j.jairtraman.2014.03.001 nugroho, i. a., riastuti, u. h., & iridiastadi, h. (2012). performance improvement suggestions for ground handling using lean solutions approach. procedia social and behavioral sciences, 65(icibsos), 462-467. doi: https://doi.org/10.1016/j.sbspro.2012.11.149 ijahp article: sari, wan mohamed, ayu jalil/conceptual model for the implementation of electric ground support equipment in ground handling activities in indonesia international journal of the analytic hierarchy process 182 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.825 rusdiyana & devi. (2013). challenges in developing baitul maal wat tamwil (bmt) in indonesia using analytic network process (anp). business & management, 4(2), 51–62. doi: https://doi.org/10.29244/jam.1.2.107-118 saaty, r. (2016). decision making in complex environments. pittsburgh, pa, usa: katz graduate school of business. sameh & scavuzzi. (2016). environmental sustainability measures for airports. centre of research in air and space law, 7, 1–12. scheelhaase, j. d. (2018). how to regulate aviation’s full climate impact as intended by the eu council from 2020 onwards. journal of air transport management, 75, 68–74. doi: https://doi.org/10.1016/j.jairtraman.2018.11.007 shaukat, n., khan, b., ali, s. m., mehmood, c. a., khan, j., farid, u.,ullah, z., anwar, m. (2018). a survey on electric vehicle transportation within smart grid system. renewable and sustainable energy reviews, 81, 1329–1349. doi: https://doi.org/10.1016/j.rser.2017.05.092 sucipto. (2017). indonesia aviation outlook 2017. signature, 219. wei, w., & gosling, g. d. (2013). strategies for collaborative funding of intermodal airport ground access projects. journal of air transport management, 32, 78–86. doi: https://doi.org/10.1016/j.jairtraman.2013.07.006 yuan, k., song, y., mu, y., andri, i., pina, a., ferrão, p., fournier, j., lacarrière, b., & corre, o. le. (2017). a load forecast method for fast charging stations of electric a load forecast method for fast on charging stations of electric vehicles on the freeway considering the information interaction. 9th international conference on applied energy, icae2017. doi: https://doi.org/10.1016/j.egypro.2017.12.584 ijahp: mu/ the ahp/anp conference community international journal of the analytic hierarchy process 300 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.730 the ahp/anp conference community as ijahp editor-in-chief, i have the unique opportunity of getting to know ahp/anp scholars and practitioners from all over the world. while my initial acquaintance is usually through this journal, it is often followed by personal interactions at diverse academic meetings in our discipline. i was privileged to attend the multi-criteria decision-making (mcdm) conference in istanbul this past summer, and the central european conference of information and intelligent systems (ceciis) in varazdin, croatia, this fall. we reported about our mcdm experience in our previous issue, and are reporting about the ceciis in this current one. in case you are wondering what the connection is between these conferences and the ijahp, it is worth mentioning that many of our most distinguished ijahp authors and board members actively participate in these conferences. some of them polish their studies by presenting them at these conferences so that they can later be submitted to and published in our journal. for this reason, this journal has always made a point of publicizing and reporting on conferences related to our discipline. in this particular issue, in addition to our customary portfolio of ahp/anp articles, we report the promotion of professor nina begičević, member of the ceciis program committee and ijahp board member since its founding, to the position of dean of the faculty of informatics (foi) of the university of zagreb. it is worth mentioning that dr. begičević is the first female and youngest faculty member appointed to the position of dean of foi. we conclude this issue with a reflection on the role of conferences in the academic experience given by dr. blazenka divjak, minister of science and education of the republic of croatia, in her opening remarks to the 30 th ceciis meeting. enjoy this issue and have a happy holiday season! enrique mu ijahp editor-in-chief ijahp: special issues, guest editors’ reflections international journal of the analytic hierarchy process 1 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.1072 special issues, guest editors’ reflections birsen karpak distinguished professor emeritus youngstown state university, usa bkarpak@ysu.edu enrique mu professor, university faculty research officer carlow university, usa emu@carlow.edu emel aktas chair of supply chain analytics cranfield school of management, cranfield, beds mk43 0al, uk emel.aktas@cranfield.ac.uk ilker topcu professor of decision sciences industrial engineering department istanbul technical university, turkey ilker.topcu@itu.edu.tr isahp 2022 was an excellent meeting! we secured quite a few special issues for conference participants. yet, these reflections are confined to the journal of enterprise information management (jeim) and the journal of multiple criteria decision analysis (jmcda). jeim will be guest edited by birsen karpak, emel aktas and ilker topcu; we are pleased with this international cooperation among the usa, the uk, and turkey. the guest editors for jmcda will be birsen karpak and enrique mu. even though we are reflecting upon submissions for the two journals (jeim and jmcda), quite a few of the following suggestions are useful for any special issue of ijahp. first, do not submit to these journals before receiving our friendly reviews. of course, you will receive a friendly review for jeim from birsen karpak, emel aktas, and ilker topcu and from birsen karpak and enrique mu for jmcda. the process is simple to receive a friendly review for your submission; just send your manuscript to dr. karpak, using the above e-mail, stating if you would like to submit it for review in jmcda or jeim. she will coordinate the friendly review with the other guest editors. a friendly review involves receiving friends’ suggestions about your manuscript, to the best of their knowledge, to position your paper in the best possible light to the reviewers. it may include suggestions on how to organize the paper and what to include in each mailto:bkarpak@ysu.edu mailto:emu@carlow.edu mailto:emel.aktas@cranfield.ac.uk mailto:ilker.topcu@itu.edu.tr ijahp: special issues, guest editors’ reflections international journal of the analytic hierarchy process 2 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.1072 section. it is not a proofreading exercise, and a friendly review does not guarantee publication. the paper will still go under the blind review process that the journal follows. independent reviewers will provide their feedback on its suitability to the journal with revision recommendations. after receiving the friendly review, you should improve your manuscript as much as you can and submit it to the appropriate journal. your manuscript will be processed as soon as you submit it to the journal and a decision on whether it should be published will be made as soon as the reviewers’ reports are received. so, don’t wait until the submission deadline; submit your manuscript as soon as you can. we will try to accomplish a quick review. if a particular reviewer cannot do it, we might extend the time needed for the review; however, if we receive enough reviews, we will continue with the process and possibly cancel reviews not done on time. there are some journals that successfully implement this policy. we believe that, ideally, publications and reviews should be free services to the community for ethical reasons. submitted manuscripts should not have been published previously nor be under consideration for publication elsewhere. the submitted manuscripts will be sent to reviewers as they arrive, but the editors will have the last word in accepting or rejecting any manuscript. when the page proofs are finalized, the fully typeset and proofed version of the article will be published online. this is referred to as the earlycite version. while an earlycite article has yet to be assigned to a volume or issue, it does have a digital object identifier (doi) and is fully citable. it will be compiled into an issue according to the journal’s issue schedule, with papers being added by chronological date of publication. read the journal you are planning to submit to as well as the call for papers very carefully. (see call for papers jmcda, call for papers jeim). for this special issue of jmcda, we invite original research contributions about the theory, computation, and practice of the analytic hierarchy/network process (ahp/anp) and cross fertilization with other multicriteria decision analysis (mcda) methodologies. we welcome high quality papers that address any type of decision problem, proposes methods and algorithms for solving them and describes important applications of the ahp/anp by itself or with other mcda methodologies in practice. applications that have been implemented and affect managerial decisions are highly sought after. the deadline for special issue submissions to jmcda is june 30, 2023. for jeim, the un’s sustainable consumption and production patterns, circular supply chain governance, role of risk analysis and disaster management in resilient supply chains, machine learning, artificial intelligence and digitalization, blockchain, and industry 4.0 are hot topics. of course, we look forward to articles on any topic listed in the call for papers. the deadline for the special issue submissions for this journal is also june 30, 2023. https://onlinelibrary.wiley.com/pb-assets/assets/10991360/callfpapersjmcda_nov2022-1667494091497.pdf https://www.emeraldgrouppublishing.com/calls-for-papers/ahpanp-supply-chain-resilience-era-digital-enterprise ijahp: special issues, guest editors’ reflections international journal of the analytic hierarchy process 3 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.1072 please read the scope of the journal carefully. in one of the quite high-quality special issues we arranged for our conference in the past, we had an excellent paper that we could not publish because it was out of the scope of the journal. because of this experience, we will try to avoid out-of-scope submissions. it is to our advantage that we have more than one good quality special issue; one of the journals accepts any application and theory of multiple criteria decision-making methods. have a reliable writer copy edit your manuscript prior to submitting it to a journal. most of us have our own manuscripts copy edited, and quite a few of our colleagues, even some editors-in-chief, use professional proofreading services. typos or english grammar construction errors that are present affect the editor as well as the reviewers’ thoughts either consciously or unconsciously about the quality of the whole submission, including the quality of the underpinning research. we cannot recommend any copy editor for ethical reasons, but you can receive recommendations from your colleagues and friends. one of the authors of this news article has recently published a paper without any minor revisions; it was copy edited by a professional copy editor. remember that submitting a manuscript to a journal is, in a broader sense, inserting yourself into an existing conversation about your topic between the journal and its readers. so, be mindful of what has been said before about your topic in the target journal and cite it as part of your discussion. one of the authors of this news article attended a conference with several journal editors. one of them, an editor-in-chief of a reputable journal, said that they initially check the manuscript references, and if there are no references from the journal to which the author is submitting, they desk-reject the paper since that leads them to believe that no one from the journal is interested in this area of research. in any case, reviewing the journal to which you intend to submit your research is a good practice. you should look for any papers that are relevant to your submission that have been published in that particular journal. be mindful that your manuscript for the special issue must be different from the conference proceedings on which it is based. suppose you published just two pages of extended abstract to this conference. in that case, you don’t need to worry about publishing full papers in these special issues and in any other special issues though you should still be careful not to plagiarize it from your own paper. a common practice is using a different, rather extended title in the journal submission to state that this piece and the conference submission are two different, albeit related, research manuscripts. more importantly, if you wrote a final paper of five or more pages, you should write an entirely new article for these special issues. otherwise, it might be rejected by the reviewers or by the editors. in any case, around five pages are sufficient for proceedings; we assume that everyone knows the proper article length for journals. finally, follow the author guidelines for each journal and check your manuscript’s similarity to other works before submission. good luck! ijahp article: yoon/the properties of human being’s decision making international journal of the analytic hierarchy process 380 vol. 9 issue 3 2017 issn1936-6744 https://doi.org/10.13033/ijahp.v9i3.528 the properties of human being’s decision making commemorating the late thomas l. saaty min-suk yoon professor of electronic commerce chonnam national univerisy yeosu, s. korea msyoon@chonnam.ac.kr abstract in order to honor thomas saaty, the developer of analytic hierarchy process (ahp)/ analytic network process (anp), and focus on the paper he wrote just before he died, this essay mentions the rank order problem due to the addition/deletion of irrelevant alternatives over which there were controversies in the ahp. saaty ended the controversies with the explanation that both rank preservation and rank reversal are possible depending on the decision making circumstance. in this paper, we will summarize the three properties that fundamentally act on how to prioritize and aggregate the subjective judgments of human beings in ranking alternatives: comparisons, judgmental dependency and rank order. this essay also mentions the rank order in relation to the cases of addition/deletion of irrelevant criteria from previous studies. this essay closes by expressing gratitude to thomas saaty for his work. keywords: decision making; rank preservation; rank reversal; subjective judgment; comparisons 1. introduction this essay is written in honor of the distinguished professor thomas l. saaty, who created the analytic hierarchy process (ahp)/analytic network process (anp). the main paper that will be discussed is ‘rank preservation and reversal in decision making’ (saaty, 2015). it deals with ‘rank order’, which has been a major subject of controversy in relation to the ahp. we will also briefly mention his work in which his thoughts and philosophy are considered to have been especially well-organized. saaty essentially explains human decision making in terms of how to prioritize and aggregate judgements beyond the understanding of the technical methodology. this is recognized as a more philosophical approach to subjective judgment. mailto:msyoon@chonnam.ac.kr ijahp article: yoon/the properties of human being’s decision making international journal of the analytic hierarchy process 381 vol. 9 issue 3 2017 issn1936-6744 https://doi.org/10.13033/ijahp.v9i3.528 2. the rank reversal issue rank reversal in ahp dealt with what were perceived to be illegitimate changes in the ranks of the alternatives upon changing the structure of the decision in one of two ways: 1) adding or deleting irrelevant alternatives—copies of an alternative; and 2) adding or deleting an irrelevant criterion under which the priorities of the alternatives would be tied (saaty et al. 2009). the beginning of such thought was revealed from the controversy of rank reversal caused by irrelevant alternatives in the early 1980’s when ahp was popularly known as an innovative methodology of decision making. rank reversal by irrelevant alternatives triggered numerous scholars to be interested in the controversy (saaty et al., 2009). while some critics regarded rank reversal as a shortcoming of ahp, some advocates suggested modified judgement methods of the priorities of alternatives to avoid rank reversal (yoon & kinoshita, 2009). saaty, however, ended the controversy of rank reversal by describing the two decision circumstances as to whether rank reversal is legitimate or not. saaty posited that, ahp with its pairwise comparisons of the alternatives is concerned with those cases where rank can and should change, and ahp also has a procedure that preserves rank absolutely as needed. the former circumstance requires relative measurement among alternatives, and the latter requires absolute measurement corresponding to the ideal alternative that is obtained from implicit understanding and comparison of many alternatives (millet & saaty, 2000). using various examples, saaty also shows whether the attributes of assessment due to the addition or removal of alternatives are intrinsically independent of, or dependent on the given alternatives assessment. there are two lessons from the examples. one is that it is we who must decide in a particular decision problem whether, for that problem, rank needs to be preserved or not. it is not automatically written in the abstract structure of the real-life problem itself. the other lesson is that we cannot use one and only one procedure for aggregating preferences in a multiple criteria decision process once and for all. we need one procedure to preserve rank and another to allow rank to change (saaty, 2015, p. 36). therefore, decision-makers experiences and knowledge are important when deciding whether or not rank reversal is legitimate or justifiable. 3. properties of subjective judgment finally, in the paper, we can see saaty’s beliefs about 3 properties of a human being’s subjective judgment: comparison, judgmental dependency and rank order (saaty, 2015). ijahp article: yoon/the properties of human being’s decision making international journal of the analytic hierarchy process 382 vol. 9 issue 3 2017 issn1936-6744 https://doi.org/10.13033/ijahp.v9i3.528 3.1 property of comparisons one of the outstanding contributions of ahp is its pairwise comparisons method to elicit objectively clear priorities from subjectively fuzzy judgments. at this point, it can be found that saaty believes human being’s subjective judgment intrinsically arises from the comparison of one thing to another, sometimes an ideal value. thus, “the ahp is a decision-making theory based on relative measurement. it derives cardinal scales from paired comparisons” (saaty 2015, p. 36), and “all methods that do not compare alternatives and the answers they give must and should be suspect” (saaty 2015, p. 37). there are two ways to compare alternatives, pairwise comparisons among alternatives and comparisons to an ideal alternative. the procedure that preserves rank absolutely as needed is done by comparisons too. for the priorities of criteria, pairwise comparisons have been obviously used. 3.2 property of judgmental dependency comparisons among alternatives implies, how good an alternative is depends on what it is compared with. so, implicitly, comparison with respect to an ideal implies dependence on other alternatives. therefore, the priorities of an alternative depend on a given set of alternatives. if the set is changed by adding or deleting alternatives, the result of priorities can change with the changed ideal. hence, alternatives cannot be evaluated independently one at a time. additionally, “rank may also be allowed to reverse when the weights of the criteria depend on the alternatives so that a criterion becomes more or less important depending on what alternatives there are” (saaty 2015, p. 37). we know that such dependence of criteria on alternatives can be analytically synthesized by the anp. 3.3 property of rank order in addition to examples of rank order circumstances, saaty (2015 p. 37) adds that “rank needs to be preserved in established systematic operations and allowed to change (rank reverse) in exploratory and tentative kinds of decisions.” in reality, keeping rank order is concerned with whether the established order is to be maintained or allowed to change with a change of circumstances. we add here a more specific description about irrelevant criteria. one of the criticisms of ahp was rank reversal by adding or deleting irrelevant criteria under which the alternatives are equal. the former were called “indifferent criteria” and the latter “wash criteria.” the correct approach to deal with wash and indifferent criteria is as follows; do not delete them or add them but simply, in the former case, assign zero priorities to the alternatives and keep that criterion, and in the latter case not to add them or if added to consider this a new decision respecting the influence of added criteria on the final outcome which could lead to different priorities and ranks (saaty et. al. 2009). 4. additional remarks on rank reversal the hierarchic structure of criteria can show how to adjust the current priorities of criteria when irrelevant criteria are added or deleted. logically, the priority of a criterion is ijahp article: yoon/the properties of human being’s decision making international journal of the analytic hierarchy process 383 vol. 9 issue 3 2017 issn1936-6744 https://doi.org/10.13033/ijahp.v9i3.528 distributed hierarchically downward to criteria belonging to the same criterion in the upper level. therefore, addition or deletion of a criterion in a level requires upward changes to the priority of the directly related upper criterion, which can prevent rank reversal. however, it is common that adding or deleting criteria makes it a different decision-making problem. 5. acknowledgment: honoring thomas l. saaty the ahp was first introduced to korea in the early 1980s, but it was not until around 1990 that it became widely known in the field of decision making. i received a master's degree using the ahp in those days, thereby starting a relationship with the process which continued when i earned a doctoral degree with a more systematic approach to the ahp. at the same time, i studied the anp by myself, and began research activities to solve problems by utilizing the ahp/anp. i did not even think that the ahp/anp would be my life partner until then. i had the unforgettable opportunity of meeting thomas saaty for the first time on the day just before the international symposium on the ahp (isahp) in 2003 in indonesia. since then, i have attended every isahp meeting since 2005 in hawaii to the 2016 meeting in london, while taking time to become friendly with so many international scholars and researchers. during my sabbatical leave as a visiting scholar hosted by saaty at the university of pittsburgh in 2012, i had a deeper exchange with him and recognized, even at the age of 86, his enthusiastic research and humanity. recalling this time with him reminds me of his passionate life and my utmost respect for him. although there are many things that i still vividly remember, some of them that stick out are as follows. first of all, there were always numerous scholars and researchers from other countries studying and doing research with him. second, it was also very impressive that many researchers from abroad visited the university even for a while to look in on his classes at the katz graduate school of business and hold discussions. third, his graduate classes had many students who came from other countries and enthusiastically participated to learn the ahp/anp. fourth, he provided groups from overseas with short-term courses during summer vacation outside regular classes. fifth, his occasional tele-seminars with foreign researchers over the internet were also surprising. there is so much more to say. even during the few times i visited his home, i was impressed by his humane features thinking about his wife, rozann. in fact, one thing i should point out is her devoted assistance. as a lifelong partner of saaty, she not only helped her husband's academic activities, but also demonstrated her ability as a researcher. ijahp article: yoon/the properties of human being’s decision making international journal of the analytic hierarchy process 384 vol. 9 issue 3 2017 issn1936-6744 https://doi.org/10.13033/ijahp.v9i3.528 references millet, i. & saaty, t.l. (2000). on the relativity of relative measures—accommodating both rank preservation and rank reversals in the ahp. european journal of operational research, 121, 205-212. doi: https://doi.org/10.1016/s0377-2217(99)00040-5 saaty t. l. (2015). rank preservation and reversal in decision making. journal of advances in management sciences & information systems, 1, 34-37. saaty t. l., vargas, l.g. & whitaker, r. (2009). addressing with brevity criticisms of the analytic hierarchy process, international journal of the analytic hierarchy process, 1(2), 121-134. doi: http://dx.doi.org/10.13033/ijahp.v1i2.53 yoon, m.s. & kinoshita, e. (2009). classification of the analytic hierarchy process approaches by application circumstances, international journal of management science, 16(1), 17-47. https://doi.org/10.1016/s0377-2217(99)00040-5 microsoft word 541-3086-6-ceeg12-16.docx ijahp article: vargas/the legacy of the analytic hierarchy/network process international journal of the analytic hierarchy process 320 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.541 the legacy of the analytic hierarchy/network process luis g. vargas the joseph m. katz graduate school of business university of pittsburgh lgvargas@pitt.edu “in memory of my mentor and friend tom saaty” abstract there is no doubt that the analytic hierarchy process is in itself a remarkable contribution to the field of decision making. in this paper, i have tried to point out what i consider to be tom saaty’s three most important contributions within his analytic hierarchy process: group decision making, conflict resolution and the fundamental equation of pairwise comparisons. the third contribution occupied him most of his life in an attempt to connect pairwise comparisons with brain activity related to cognitive processes. keywords: analytic hierarchy process; group decision making; conflict resolution; fundamental equation of pairwise comparisons 1. introduction one of the most difficult tasks modelers face is the incorporation of human behavior into decision-making. it is known that human behavior is not always rational in the way it is assumed by the rational choice school with the axiom of transitivity. in recent years, a new way of thinking has evolved using psychology and economics that is trying to show that transitivity need not always be satisfied to be a rational decision maker. kahneman and tversky (1979) showed the many problems that expected utility theory has as a descriptive theory of behavior leading to preference reversals; and tversky and thaler (1990) gave some plausible explanations as to how preference reversals may occur when people make decisions. richard thaler, the 2017 nobel prize in economics, has demonstrated that mankind is afflicted by emotion and irrationality, which influences their decision making about everything from retirement savings, to health-care policy, to professional sports. this is in complete agreement with what thomas l. saaty has been saying for years. saaty’s theory (1977, 1980, 1986), the analytic hierarchy process (ahp), is based on the idea that making decisions need not assume transitivity. one could go one step further and imply that lack of transitivity in preferences may lead to rank reversals. it is one of the reasons why saaty’s theory has been criticized. however, a theory of decisionmaking should allow for intransitivity if we expect to capture what thaler (2017) calls “predictably irrational” behavior. thaler does not believe that human beings are ijahp article: vargas/the legacy of the analytic hierarchy/network process international journal of the analytic hierarchy process 321 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.541 randomly irrational. he is not the only one who believes this to be the case. ariely (2008) also challenges the assumptions about making decisions based on rational thought. we believe we make decisions by comparing alternatives in pairs according to different criteria, but for every pairwise comparison, we have one and only one criterion in mind. we perform all the comparisons according to all the criteria, and somehow, we synthesize all the comparisons in our brain to arrive at the final decision. saaty created his theory to help model this process and to incorporate the experience, talent and knowledge of the actors involved in the resolution process. however, as a theory of decision-making, the ahp and its extension to networks in the analytic network process (anp) approximate how we really make decisions. the methodology developed by thomas l. saaty provides the flexibility of accepting or rejecting transitivity in the modelling process. we know that a necessary and sufficient condition for rank preservation in a single matrix of pairwise comparisons is that row dominance be satisfied (saaty & vargas, 2012). nonetheless, people are not always transitive, and hence, they violate the fundamental premise of the rational choice school and thaler’s “predictable irrational” behavior follows. when tom saaty conceived the ahp, he envisioned three fundamental problems that needed to be addressed: (1) group decision-making, (2) conflict resolution, and (3) the role of pairwise comparisons, in particular in neural and brain activity. 2. three important areas 2.1 group decision-making now more than ever, group decision-making is critical at all societal levels. problems are becoming more complex requiring multiple experts to understand all the dimensions of the problem, and the implications of decisions are multidimensional. we need to be able to make decisions in groups without the fear of having a decision being imposed on us. this would be the case, if all we do is rank alternatives, because then we could fall under the umbrella of arrow’s impossibility theorem (arrow, 1951). the starting point of arrow’s theory is a set of at least three alternatives and at least two individuals in a group. the individuals rank order the alternatives according to their preferences. the objective of group decision making is to find a procedure, known as a social welfare function (swf), which combines the individual preferences into a group preference. arrow (1951) imposed the following conditions to swf: unrestricted domain (i.e., all orderings are possible), unanimity (i.e., if all the individual members prefer an alternative over another, the group should also maintain that preference), independence from irrelevant alternatives (i.e., the addition or deletion of an alternative should not alter the group ordering of the remaining alternatives) and non-dictatorship (i.e., there is no individual in the group whose preferences coincide with the group preference). under these conditions arrow proved that no swf exists. we need to ensure that a decision by a group is not a dictatorial decision. saaty and vargas (2012) showed that it is possible to make decisions in groups without being dictatorial so long as the intensity of preference given by the individual judgments is represented with an absolute scale, and the social welfare function is the geometric mean ijahp article: vargas/the legacy of the analytic hierarchy/network process international journal of the analytic hierarchy process 322 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.541 of the individual judgments. this seminal paper in group decision making on which saaty and i collaborated was published in the journal of social choice and welfare in 2012. 2.2 conflict resolution in 1981, the book “getting to yes” revolutionized the way conflicts were looked at (fisher & ury 1981). fisher and ury introduced the concept of principled negotiation in which the participants are problem solvers. the approach is based on four principles: (1) separate the people from the problem, (2) focus on interests not positions, (3) invent options for mutual gain, and (4) insist on using objective criteria. in this approach, the parties do not see each other as adversaries, but as collaborators in search of a fair solution. however, the approach does not measure gains and losses of the parties for different options. thus, the parties may not be able to perceive how fair the proposed solution is. what is needed is the development of scales that represent the preferences of the parties. it is not enough to assign numbers to preferences without any mathematical assumptions because we want to ensure that the results belong to a measurement scale. this is a difficult problem if the dimensions of the conflict involve intangibles, which by definition are considered not to have a scale of measurement. pairwise comparisons from saaty’s absolute scale can be used to build such relative measurement scales. in saaty et al. (2017), this approach was used to show that a fair solution (developed by those involved in the process) exists. this is just one of many examples that show that to deal with conflicts the negotiation approach needs to be measurement based. since intangibles are always involved, we need pairwise comparisons to build measurement scales, which we can use to compute gain/loss ratios of tradeoffs from each party’s perspective. gain/loss ratios are not symmetric, and the tradeoffs are non-zero sums. hence, measurement allows for the selection of tradeoffs for which both parties benefit equally through a maxmin optimization model. 2.3 pairwise comparisons and neural activity the nervous system uses its own kind of mathematical function patterns to deal with both external and internal realities. the conscious part of the nervous system is there to respond to what happens outside by regulating externally received information signals from the senses and the skin and muscles of the body itself. to do that, it needs to communicate with its subconscious using the familiar language of neural firing. saaty and vargas (2017) show that because reciprocal pairwise comparisons are performed at the neural level, the division algebra of octonions, in which commutativity and associativity are not satisfied, provides the structure needed to represent mental processes (baez, 2001). tom showed, by extending the discrete pairwise comparisons to continuous spaces, that the response of a neuron in spontaneous activity, w(s), is an eigenfunction solution of a fredholm’s integral equation of the second kind if and only if it satisfies the functional equation w(as) = bw(s), where s represents stimuli (saaty, 2015; 2017a,b). he called this equation the fundamental equation of pairwise comparisons. its solution in the space of ijahp article: vargas/the legacy of the analytic hierarchy/network process international journal of the analytic hierarchy process 323 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.541 octonions is given by !w(u) = a ln b ln a( )u p u( )⊕ p u( )a ln bln a( )u , where p(u) is a periodic function of period 1. it satisfies the condition !w(uv) = !w(u)!w(v) if and only p(u) satisfies the semigroup condition p(u+v) = p(u)p(v), and it can generate the group of automorphisms, g2. in g2, these functions are given by !w(u) = b ue2 nπu , and they are dense in the space of continuous functions defined on the octonions. thus, all continuous functions could be expressed as linear combinations of the solution of the equation, and they could generate the group of automorphisms. in sum, any representation of brain activity with octonions could be expressed with the solution of the equation ( ) ( )as b s=w w . according to this result, the firing of neurons through the continuous paired comparison process generates a smooth g2-manifold in which cognition and the representations of our thoughts could take place (saaty and vargas, 2017). ijahp article: vargas/the legacy of the analytic hierarchy/network process international journal of the analytic hierarchy process 324 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.541 references ariely, d. (2008). predictably irrational: the hidden forces that shape our decisions. new york: harper collins publishers. arrow, k. j. (1951). social choice and individual values. new york: john wiley & sons. doi: https://doi.org/10.1007/978-3-531-90400-9_6 baez, j.c. (2001). the octonions, bulletin of the american mathematical society, 39(2) 145-205. doi: https://doi.org/10.1090/s0273-0979-01-00934-x fisher, r. & ury, w. (1981). getting to yes: negotiating agreement without giving in. new york: penguin books. kahneman, d. & tversky, a. (1979). prospect theory: an analysis of decision under risk, econometrica 47(2), 263-291. doi: http://www.jstor.org/stable/1914185 saaty, t.l. (1977). a scaling method for priorities in hierarchical structures. journal of mathematical psychology 15, 234–281. doi: https://doi.org/10.1016/00222496(77)90033-5 saaty, t.l. (1980). the analytic hierarchy process. new york: mcgraw hill. saaty, t.l. (1986). axiomatic foundation of the analytic hierarchy process. management science 32(7), 841–855. doi: https://doi.org/10.1287/mnsc.32.7.841 saaty, t.l. (2015). the neural network process (nnp): generalization of the ahp and anp to the continuous case of neural firing. pittsburgh, pa: rws publications. saaty, t.l. (2017a). neurons the decision makers, part i: the firing function of a single neuron, neural networks, 86, 102-114. doi: https://doi.org/10.1016/j.neunet.2016.04.003 saaty, t.l. (2017b). neurons the decision makers, part ii: the firings of many neurons and their density; the neural network its connections and field of firings, neural networks 86, 115-122. doi: https://doi.org/10.1016/j.neunet.2016.04.004 saaty, t.l. and l.g. vargas (2012). the possibility of group choice: pairwise comparisons and merging functions, social choice and welfare, 38(3), 481-496. doi: https://doi.org/10.1007/s00355-011-0541-6 saaty, t.l. & vargas, l.g. (2017). origin of neural firing and synthesis in making comparisons, european journal of pure and applied mathematics, 10(4), 602-613. saaty, t.l., vargas, l.g. & zoffer, h.j. (2017). a structured scientific solution to the israeli–palestinian conflict: the analytic hierarch process approach, decision analytics 2(7), 2-53. doi: https://doi.org/10.1186/s40165-015-0017-3 ijahp article: vargas/the legacy of the analytic hierarchy/network process international journal of the analytic hierarchy process 325 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.541 thaler, r. h. (2017). misbehaving: the making of behavioral economics. w.w. norton – via google books. doi: 10.1111/risa.12871 tversky, a. &thaler, r.h. (1990). anomalies: preference reversals, the journal of economic perspectives , 4(2), 201-211. doi: 10.1257/jep.4.2.201 ijahp: mu/isahp2020: lessons learned and the future of conferences international journal of the analytic hierarchy process 366 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.844 isahp2020: lessons learned and the future of conferences more than 128 participants from 30 countries attended the 16 th isahp meeting of the international symposium of the ahp (isahp2020) that took place for the first time in cyberspace rather than in a physical place. most academic conferences have moved to virtual formats with mixed success; however, for the most part the attitude has been that of “it would be worse not to have it” rather than “what a great opportunity.” i must confess that my initial reaction was that a virtual isahp would not be the “real” conference that we are all accustomed to, so for most of the year i had resigned myself to the idea of not having our biannual meeting this year. however, the proliferation of virtual meetings this year, and my teammate’s enthusiasm, moved me to think, by august, that we should give it a try, although i must confess i was still somewhat skeptical. my doubts did not come from the technical feasibility, although the timeframe was challenging, but in terms of conference success. what do i mean by conference success? many metrics have been created for this purpose with indicators ranging in number from a few to 20 or more. these indicators cover all sorts of conference aspects such as overall satisfaction, content assessment, networking opportunities, perceived value for the money and others. for the ijahp team who are also the isahp conference organizers, however, our measure of success is simply the satisfaction of our ahp/anp community in terms of academic content and social networking. i qualify the networking aspect, which usually refers to establishing and maintaining professional contacts, with the additional word “social” because our community values the friendships and social aspect as much as the professional one. since they began in 1988, our isahp meetings have traditionally resembled extended family gatherings. personally, my concern was that after such a rough year of virtual work and pandemic stress, there would be little interest and energy and perhaps not enough research material available to submit for a conference announced only four months in advance. true, this timeframe was also challenging from an organizational point of view, but even if everything was done on time, i feared that the conference would still be a failure without community participation. but, i am glad we decided to do it! the response was far more enthusiastic than anticipated and the level of participation, in terms of submissions and attendees, was similar or perhaps even greater than several of our previous meetings. our organizing team, which is recognized individually in another section of this issue, rose to the occasion and had a professional online meeting ready on time. however, what is more important is that our ahp/anp colleagues and friends also responded enthusiastically by collaborating as track chairs and reviewers as well as by participating in the event. ijahp: mu/isahp2020: lessons learned and the future of conferences international journal of the analytic hierarchy process 367 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.844 what have we learned from the experience? let’s start with the pros. first, in virtual presentations all members of the audience have a good view and audio of each presentation since it is shown on their own computer screen (naturally, a technical issue may occur, but fortunately these were limited to only a few incidents during the symposium). second, while the responsibility of the academic content lies with the academic committee composed of the program and track chairs, one important advantage is that the sessions are recorded and can be viewed over and over again as needed. this is particularly useful for conferences —like our event—in which there are parallel sessions. by having the sessions recorded, we can re-visit each and every presentation and virtually attend every session that took place at isahp2020 for years to come. i found this particularly useful as i reviewed the workshops and plenary sessions to write about sessions of particular interest. finally, the most important benefit we found is that participants can be part of the event no matter how far apart they are (there is no “here” or “there” in cyberspace). because of this, attendees that would not normally have participated due to time, distance and financial issues (not everybody can afford the time and money to go to kuala lumpur or hong kong) can now do so. this was an opportunity we identified early on and for this reason the conference theme was “ahp/anp: the next generation.” we saw the isahp2020 virtual format as a way to integrate first-timers and young attendees that would not have otherwise participated. we believe we had partial success on this front. there are also some cons to a virtual format meeting. first, the social presence is very weak. social presence has been identified as a key factor in the success of virtual instruction, and it can be loosely defined as the extent to which the participant feels part of a social circle. clicking links and watching presentations can be devoid of social presence (as i have experienced in the extreme at some conferences where the presentations were pre-recorded). at isahp2020, the zoom meeting format for each of the sessions allowed the participants to browse, see who else was in the session, and who was not, and exchange warm greetings via the chat. we do however feel that we fell short in providing spaces for social interaction during our conference. second, time differences do offer challenges for the participants. having the conference run from 8 am – 2 pm us eastern time meant that the european attendees would be attending from 2pm – 8pm, while the asian participants would be participating between 8pm – 2am in their local time zones. other associations with large numbers of participants in different time zones have addressed this problem by dividing the conference into three days in which each day schedule is centered around one of the time zones. i think there are still other alternatives to be explored to address this inconvenience. fortunately, the ahp/anp participants were very gracious about the timeframe and participated enthusiastically even in the closing ceremony which was the last event to take place on sunday, december 6 th at 2pm. you will find in this ijahp issue a news article and reflections on the isahp2020 event, and an essay by me that is an academic summary and reflection on the addresses by the keynote and plenary speakers. my personal conclusion about the future of conferences is that virtual presentations are here to stay and that all conferences will be hybrid in the future. even when we are able to return to meeting in physical spaces, an online component will be needed so as not to ignore the large number of participants that would like to participate but cannot be physically present. once they have enjoyed being part (albeit virtually) of a conference, ijahp: mu/isahp2020: lessons learned and the future of conferences international journal of the analytic hierarchy process 368 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.844 they will not want to be left out in the future and the conference organizers should not ignore this important part of participation. still, one thing that became clear at isahp2020 is that we long to see and meet each other with a big hug and an even bigger smile. we want to raise a glass of wine to toast to each other’s health, or to the conference success, or simply to life; we long for the human touch! enrique mu, phd ijahp editor-in-chief ijahp article: singh, suthar/development of a risk assessment method for small-sized hospitals using ahp: a case in northern india international journal of the analytic hierarchy process 306 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 development of a risk assessment method for small-sized hospitals using ahp: a case in northern india lakhwinder pal singh 1 associate professor, department of industrial and production engineering, dr b r ambedkar national institute of technology jalandhar, punjab (india)-144011 singhl@nitj.ac.in harish suthar dr b r ambedkar national institute of technology jalandhar, punjab (india)-144011 harishs.ip.16@nitj.ac.in abstract several studies suggest that a major factor causing acute low back pain in nursing staff is the manual handling of disabled patients. in india, especially the northern part that consists of non-metropolitan cities, most of the hospitals still lack advanced patient handling methods and techniques. therefore, it is necessary to devise a method of assessment for testing the effectiveness of an ergonomic intervention or training for safe patient handling. a proper quantification of risk involved in manual handling is required so that the severity of injuries caused by this handling can be reduced by an ergonomic intervention, which in turn helps redesign the task of manual handling. this study aimed to develop a qualitative method of risk assessment using the ahp for manual patient handling and to evaluate the validity and reliability of the risk assessment. the method is validated using the concepts of construct validity and content validity. the reliability was estimated through stability (test-retest) and homogeneity (internal consistency). the tests for validity and reliability were conducted in 130 units of 7 small-sized hospitals. the results of the current study reveal that the method was reliable and valid for risk assessment of patient handling. keywords: analytic hierarchy process (ahp); patient handling; risk assessment method for hospitals 1. introduction the job of patient handling is physically demanding, and has a high risk for causing musculoskeletal disorders among caregivers (putz-anderson et al., 1997). nurses are acknowledgments: the authors would like to give their gratitude to the panel of experts, management and workers of the small-sized hospitals in the region, for their sincere cooperation in the collection of data. the authors are also thankful to staff of the government department for their co-operation and support. mailto:singhl@nitj.ac.in mailto:harishs.ip.16@nitj.ac.in ijahp article: singh, suthar/development of a risk assessment method for small-sized hospitals using ahp: a case in northern india international journal of the analytic hierarchy process 307 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 exposed to different risk factors including lifting and moving patients, pushing and pulling heavy equipment, working at forced postures or standing for long periods without taking adequate rest periods (national research council, 2001). the exceeding amount of work demands are present irrespective of the physical and mental capability of the workers, which also increases the risk of musculoskeletal injuries (waters, 2010). a number of methods have been developed to evaluate posture during work that look at the handling of patients or any task. the ovako working posture analysing system (owas) (karhu et al., 1981) is a simple and well-documented method. however, it does not differentiate between the right and left upper limbs; also, it does not evaluate the parts of the body such as the neck, elbows, and wrists. additionally, the method does not consider repetition or duration of the sequential postures. on the other hand, another method, the rapid upper limb assessment (rula), mainly focuses on disorders in the upper part of the body (mcatamney & corlett, 1993), and provides a quick assessment of the upper body posture. the method for loading on the upper body (luba) evaluates the stress of working postures on the upper body (kee & karwowski, 2001). at the same time, the rapid entire body assessment (reba) is applicable to the entire field where posture analysis is required. it provides a scoring system, which considers both static and dynamic activities (hignett & mcatamney, 2004); however, it lacks detail and precision and only covers the force, repetition and posture risk factors. the duration and frequency of items are not considered, and the lack of time-based measures in reba leads to the most common postures and the high duty cycle postures being ranked the same. 2. literature review researchers from different countries have reported different levels for the prevalence of back pain in one year; 47% in the united states (trinkoff, 2002), 75% in greece (alexopoulos, 2003), 64% in sweden (johansson, 1995; josephson, 1997), 66.8% in the netherlands (knibbe, 1996) and 68% in switzerland (maul, 2003). some nurses suffering disabling back injuries have even had to abandon their jobs (stubbs et al., 1986). moreover, hospitals with higher incidence of such disorders among the workers have a higher staff turnover rate that further increases the costs of health care (osha, 2009). about 78% of the nurses who experienced back pain in the past 6 months did not report it to their management (cato et al., 1989). additionally, the work environment of public nurses has an increased risk for back pain in the lumbar region (colombini et al., 1990). many studies across the world have recognized the handling of patients as a high-risk activity and recommended a redesign of the practices (de castro et al., 2006). therefore, appropriate ergonomic intervention programs offer a great opportunity to reduce physical stress and the risk of injury in the lower backs of nurses (garg et al., 1991). there are number of tools to assess risk in patient handling including the method of observation of risk, which describes and assesses the working technique used for the transfer of patients with respect to the safety and health of nurses. the method consists of 24 items grouped into three phases: preparatory phase, initial position and actual execution. after quantifying the valuations, the general score of the working technique with respect to the level of musculoskeletal risk and safety, the validity and reliability of the method was satisfactorily calculated (johnsson, kjellberg, kjellberg, & lagerstrom, 2004). another method of observation contains 23 items and was tested with five different tasks of transferring patients, including a weighted score to assess ijahp article: singh, suthar/development of a risk assessment method for small-sized hospitals using ahp: a case in northern india international journal of the analytic hierarchy process 308 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 the mobilization technique used (warming, juul-kristensen, et al., 2004). on the other hand, a system for comprehensive evaluation and a theoretical model to assess the risk of low back pain has also been used for analysis of the risk of low back pain and load on the spine during lifting of patients using different techniques (marmaras et al., 1999b). however, despite all of these methods, currently, the mapo (health care mobilization of patients hospitalized) is the most prominently used method. this method examines the analysis of risks in order to establish preventive measures, including the relocation of workers with limitations to the mobilization of patients (battevi et al., 2006). to identify the major factors for making decisions or developing an evaluation method and their weights, there are a number of multiple criteria decision methods available. the analytic hierarchy process (ahp), developed by saaty (1990), is a combination of mathematics and interaction of the intended work (viswanadhan, 2005; wang & wang, 2010). the ahp is one of the most successful techniques for solving decision making problems involving the goals, the alternatives for reaching the goals and the criteria for evaluating the alternatives (harker & vargas, 1987). a number of studies that applied the ahp include andersson and menckel, (1995), arbel and orgler, (1990), armacost et.al. (1994), badri (2001) and bayazit (2005). moreover, ahp has been successfully implemented in various organizations such as integrated manufacturing, layout design (al-harbi, 2001), assessment of technology asset decisions (boucher & macstravic, 1991), flexible industrialized systems and in many other engineering related fields (arbel & orgler, 1990; armacost et al., 1994; cambron & evans, 1991; das et al., 2012; saaty, 1990; shikdar and al-araimi, 2001). 2.1 need for study the job of patient handling is physically demanding and has a high risk of musculoskeletal disorders among caregivers (putz-anderson et al., 1997). the exceeding amount work demands are present irrespective of the physical and mental capability of the workers and increase the risk of musculoskeletal injuries (waters, 2010). therefore, appropriate ergonomic intervention programs can offer a great opportunity to reduce physical stress and the risk of injury in the lower backs of nurses (garg et al., 1991). currently, there are several methods to evaluate risks, with different criteria that have a greater or lesser applicability depending on the working environment in which they are used. these methods are mainly developed and used for hospitals with advanced operational settings in developed countries. however, in developing countries like india, especially the northern state of punjab with non-metropolitan cities like jalandhar, kapurthala, and ludhiana, the majority of the hospitals are small-sized and lack advanced facilities, including adequate patient handling methods and techniques. therefore, it is necessary to devise a method of assessment for testing the effectiveness of an ergonomic intervention or training for safe patient handling. in this study, an approach (ahp) based on the ability of mathematical structure of consistent matrices and the associated eigenvectors to generate true or approximate weights was used. the ahp works on an eigenvalue which is based on pair-wise comparisons (bayazit, 2005; boucher & mac stravic, 1991; saaty, 1990). qualitative and quantitative analyses can be performed simultaneously and calibration can be done using a suitable numeric scale (saaty, 1985). in the current study, a method for assessing risk involved in patient handling is proposed for small-sized indian hospitals. the current method is based on the method established in the past and includes some important customized and contemporary ijahp article: singh, suthar/development of a risk assessment method for small-sized hospitals using ahp: a case in northern india international journal of the analytic hierarchy process 309 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 parameters for the specific workplace setting. it also includes a validation and reliability analysis of the method by conducting observation in the case hospitals. 3. methodology 3.1 selection of aspects the methodology of the study mainly focuses on developing a method for the assessment of risk involved in patient handling activities in small-scale hospitals; therefore, to quantify the level of risk, the critical inherent aspects are considered. these aspects (items) are based on the three-observation method (kjellberg et al., 2000; johnsson et al., 2004; villarroya et al., 2017). the eventual selection included eight items. i. level of dependency (weight= 3 points) ii. climate conditions (weight= 2 points) iii. work place conditions (weight= 5 points) iv. primary aids (weight= 6 points) v. secondary aids (weight= 6 points) vi. work culture (weight= 4 points) vii. training (weight= 2 points) viii. risk perception (weight= 2 points) level of dependency of patients on the caregiver (j. knibbe & waaijer, 2005; hignett, fray, et al., 2014) has a maximum score of 3 points and depends on two factors including mobility level and cooperation of the patient. the mobility of patients is further divided into five levels (see table 1) and cooperation level is divided into three levels as shown in table 2. table1 mobility level and type of handling sr. no mobility level type of handling 1 level a: moving patients and independent persons who dress and clean themselves. safe handling: patients do not depend on the caregiver in any situation. 2 level b: patients able to stand and use a walker. practically safe handling: patients depend on the caregiver in a few situations. 3 level c: patients who are partially supported, but require a wheelchair. partially safe handling: patients depend on the caregiver in many situations. 4 level d: patients unable to stand on their legs. practically unsafe handling: patients depend on the caregiver in most cases. 5 level e: patients who are completely immovable unsafe handling: patients always depend on the caregiver. ijahp article: singh, suthar/development of a risk assessment method for small-sized hospitals using ahp: a case in northern india international journal of the analytic hierarchy process 310 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 table 2 cooperation level and type of handling sr. no cooperation level type of handling 1 level 1: fully cooperative safe handling: when patients are cooperative (patients who cooperate with the caretaker during handling) 2 level 2: partially cooperative partially safe handling: when the patients are partially cooperative (patients who have residual motor capacity) 3 level 3: non cooperative unsafe handling: when the patients are not cooperative (patients who cannot use the upper and lower parts of their body, and must be completely lifted in transfer operations). the score between the 15 combinations of mobility levels and levels of cooperation is distributed linearly in figure 1. figure 1 linear distribution of score between various combinations the final score of the factor is obtained by multiplying the number of patients in each cell by the corresponding value, and dividing the sum of the number of patients analyzed as shown in table 3. table 3 combined score of the different levels of mobility and cooperation level (cooperation v/s mobility) level a level b level c level d level f level 1 3 2.7 2.4 2.1 1.8 level 2 2.5 2.2 1.9 1.6 1.3 level 3 2 1.7 1.4 1.1 0.8 ijahp article: singh, suthar/development of a risk assessment method for small-sized hospitals using ahp: a case in northern india international journal of the analytic hierarchy process 311 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 the climate conditions factor (weight = 2 points) checks whether the climate conditions are appropriate or not with scores given if the specific condition is fulfilled. under the climate conditions, the following sub-factors are considered (see table 4). table 4 point distribution for climate conditions sr. no sub-factor condition points 1 temperature the temperature is between 14 and 27◦c. if fulfilled, 0.50 points; if not fulfilled, 0 points. 2 humidity the relative humidity is 30% 70%. if fulfilled, 0.50 points; if not fulfilled, 0 points. 3 lighting 500 lux (equivalent to 70% of sun light) lightening. if fulfilled. 0.50 points; if not fulfilled, 0 points. 4 noise the environment is less noisy if fulfilled, 0.50 points; if not fulfilled, 0 points. the workspace conditions factor (weight = 5 points) considers the condition of the workspace. it considers conditions such as access to bathrooms, condition of toilets, possibility of regulating beds for and the space of the rooms to safely perform patient handling. it has a maximum score of 5 points (see table 5.). table 5 point distribution for workspace conditions sub-factor condition point (i) bathrooms access to the bathroom without obstacles if fulfilled, 0.625 points if not fulfilled, 0 points door width of at least 85 cm, and space for mechanical aids. if fulfilled, 0.625 points if not fulfilled, 0 points (ii) water closet commode is at least 50 cm high if fulfilled, 0.625 points if not fulfilled, 0 points working space for handling a wheelchair if fulfilled, 0.625 points if not fulfilled, 0 points (iii) adjustable beds both the height and inclination can be changed if fulfilled, 1.25 points if not fulfilled 0 points (iv) rooms space between beds at least 90 cm if fulfilled 0.625 points if not fulfilled 0 points free space of at least 120 cm between bed and wall if fulfilled 0.625 points if not fulfilled 0 points ijahp article: singh, suthar/development of a risk assessment method for small-sized hospitals using ahp: a case in northern india international journal of the analytic hierarchy process 312 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 the primary aids are those aids which directly help in handling and reduce the major portion of the load on the nurse. this factor has a maximum score of 6 points and considers the equipment available to perform the lifting or transfer of patients by means of primary aids (mobilization cranes, etc.). there are 1.5 points assigned for each type of existing aid that fulfills the prerequisites shown below and aids that were available in the unit in sufficient numbers. the primary aids include lifting equipment such as a handling crane and aids that help nurses remain in the correct posture such as wheelchairs, adjustable electric beds, adjustable stretchers etc. the prerequisites for the primary and secondary aids include being in the right state, appropriate training for use been given, easy to use for the nurses, and actually helps in handling. the secondary aids assist in handling and reduce the effort required from the nurse for handling. this factor has a maximum score of 6 points and includes access to the available equipment to perform the lifting, and transfer of patients, as well as the existence of minor aids (transfer, sliding sheets, etc.). a score of 1.5 points was assigned for each type of existing aid that fulfilled the prerequisites and the aids that were available in sufficient numbers in the unit. these aids should receive adequate maintenance so that they can be used for safe handling. the secondary aids include sliding sheets, transfer sheets, rotating discs, walkers and standing cranes. these secondary aids help nurses change the position of the patient and help in transportation and turning. the work culture factor has a maximum score of 4 points and considers the breaks given between work, ratio of patients per nurse, night work and support from co-workers in handling of patients (see table 6). table 6 point distribution of work culture sr. no sub factor condition score 1 patient to nurse ratio an overall ratio of 10:1 if fulfilled, 1 point; if not fulfilled, 0 points. 2 night shift no night shift if fulfilled, 1 point; if not fulfilled, 0 points. 3 colleague support if there is support from colleagues if fulfilled, 1 point; if not fulfilled, 0 points. 4 rest no time pressures and rest is given between tasks if fulfilled, 1 point; if not fulfilled, 0 points. this training factor has a maximum score of 2 points and assesses the validity and effectiveness of the training given for patient handling (see table 7). ijahp article: singh, suthar/development of a risk assessment method for small-sized hospitals using ahp: a case in northern india international journal of the analytic hierarchy process 313 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 table 7 point distribution of training sr. no sub-factor condition points 1 information information about the risks related to manual handling is given in training. if fulfilled, 0.5 points; if not fulfilled 0 points. 2 theoretical training theoretical and practical training is given to at least 75% of the workers. if fulfilled, 0.50 points; if not fulfilled 0 points. 3 training period training gap should not be more than 2 years. if fulfilled, 0.50 points; if not fulfilled, 0 points. 4 evaluation of training evaluation of training after 3 or 4 month of training. if fulfilled, 0.50 points; if not fulfilled, 0 points. this risk perception factor has a maximum score of 2 points. the perception of nurses was assessed by asking them some questions. the main objective was to determine the physical or mental load of patient handling. the details of the points assigned for risk perception are given in the table 8. table 8 points assigned for risk perception sr. no sub factor question points i danger do you consider that the posture adopted during the handling of patients is dangerous to your health? if yes, 0.5 points; if no 0 points. ii planning is there any plan for the handling of the patients? if yes, 0.5 points; if no 0 points. iii load is patient handling a light activity? if yes, 0.5 points; if no 0 points. iv rest is there any need for rest in patient handling? if yes, 0.5 points; if no 0 points. 3.2 the ahp approach the ahp approach is based on the ability of the mathematical structure of consistent matrices and the associated eigenvectors to generate true or approximate weights. the ahp works with an eigenvalue which is based on pairwise comparisons (bayazit, 2005; boucher & mac stravic, 1991; saaty, 1990). qualitative and quantitative analyses can be performed simultaneously and calibration can be done using a suitable numeric scale (saaty, 1985). the detailed procedure to carry out the ahp analysis consists of the following steps (saaty, 1985; saaty, 1990): ijahp article: singh, suthar/development of a risk assessment method for small-sized hospitals using ahp: a case in northern india international journal of the analytic hierarchy process 314 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 step-1: hierarchical structuring of a decision problem and selection of criteria. at the topmost level is the goal or focus. at the intermediate and lower levels are the criteria or sub-criteria and the available alternatives, respectively. step-2: construction of a pair-wise comparison matrix for each level with respect to the higher levels. in this step, the relative importance of different alternatives with respect to the immediately above sub-criteria is determined. this is followed by rating the relative priority of the criteria by assigning a weight between 1 (equal importance) and 9 (extreme importance) to the most important criterion. in contrast, the reciprocal of this value is assigned to the other criterion in the pair. step-3: application of eigenvector methods to calculate the relative weight for the pairwise comparison of options on each criterion. step-4: check the consistency associated with the comparison matrix, using the consistency ratio (cr) of the consistency index (ci) with the appropriate value of the random index (ri). step-5: repeat the above steps for all levels in the hierarchy. step-6: evaluate the overall relative value by linear addition function. each factor has a different weight based on their contribution to musculoskeletal disorders. the weight must be determined in terms of the relative importance of the item for scoring. the pairwise comparison, which is part of the ahp, is used for defining the weight of each factor. the weight of each factor shows the contribution of that factor in reducing risk exposure. for the pairwise comparison, two factors were evaluated at a time in terms of their relative importance. index values from 1 to 9 and from 1 to 1/9 were used for the numerical rating. first, a comparison matrix was created. the weight of each factor comes as an eigenvector also known as a priority matrix. next, the consistency ratio (cr) was calculated to measure how consistent the judgments were relative to large samples of purely random judgments. if the cr is greater than 0.1 (10%), the judgments are untrustworthy because they are too close to randomness and the exercise is valueless or must be repeated. to find the consistency ratio (cr), first we must find the consistency index (ci) with the help of the following equation: consistency index (ci) = ( 𝜆𝑚𝑎𝑥− 1 𝑛 − 1 ) the consistency ratio (cr) was calculated using the method given by saaty. in this case, there were 8 factors to be compared. thus, the consistency ratio (cr) was calculated using the following expression: consistency ratio (cr) = ( 𝐶𝐼 1.41 ) a consistency ratio of less than 0.1 implies that the pairwise comparison is consistent. ijahp article: singh, suthar/development of a risk assessment method for small-sized hospitals using ahp: a case in northern india international journal of the analytic hierarchy process 315 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 consistency of pairwise comparison (𝐴𝑥 = 𝜆𝑚𝑎𝑥 𝑥) 𝜆𝑚𝑎𝑥 = 𝑀𝑎𝑥(10.73, 10.37, 9.32, 8.98, 8.98, 10.36, 10.42, 10.42) 𝜆𝑚𝑎𝑥 = 10.73 𝐶𝐼 = ( 10.73 − 8 8 − 1 ) = 0.14 𝐶𝐼 = 0.14 1.41 = 0.09 cr is less than 0.1, so the evaluations are consistent. the obtained weights are shown above in the list of the items. these items were further divided into sub-items and a relevant score was assigned to each sub-item. the study focused on the validity and reliability analysis of the method; therefore, the details of the score distribution were included in this paper. the score distribution is given in figure 2. 3.3 observation at hospitals the study included small-scale hospitals; the national accreditation board for hospitals and healthcare providers (nabh) defines a small-scale hospital as a hospital with less than 50 units. the data was collected from seven hospitals located in and around jalandhar in punjab (india). a total of 130 different units were observed during a period of 60 working days. these hospitals differ in their organizational structure and medical facilities (see tables 9 and 12). the units were divided into three categories (i, ii and iii; see table 12) depending on the type of wards or facilities. since the cities under consideration are non-metropolitan in nature, the range of facilities available in each hospital is similar in kind and status. an agreement was made with the hospital management to not disclose their names, so an alphabetic-numeric code was assigned for each unit in the different hospitals.the sample was selected in the study based on non-probability convenience sampling. the willingness of the hospital management to participate in the survey study was a major consideration. the method proposed in the study also considered only those factors that are critical and for which the patient would not be disturbed during their acquisition. ijahp article: singh, suthar/development of a risk assessment method for small-sized hospitals using ahp: a case in northern india international journal of the analytic hierarchy process 316 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 figure 2 score distribution in factors (level a. & level 1.) 3.0 points (level a. & level 2.) 2.5 points (level a. & level 3.) 2.0 points (level b. & level 1.) 2.7 points (level b. & level 2.) 2.2 points (level b. & level 3.) 1.7 points (level c. & level 1.) 2.4 points (level c. & level 2.) 1.9 points (level c. & level 3.) 1.4 points (level d. & level 1.) 2.1 points (level d. & level 2.) 1.6 points (level d. & level 3.) 1.1 points (level e. & level 1.) 1.8 points (level e. & level 2.) 1.3 points (level e. & level 3.) 0.8 points temperature 0.5 or 0 points pintspoints humidity 0.5 or 0 points lighting 0.5 or 0 points points noise 0.5 or 0 points bathrooms 1.25 or 0 points water closet 1.25 or 0 points adjustable beds 1.25 or 0 points rooms 1.25 or 0 points handling crane 1.5 or 0 points wheelchair 1.5 or 0 points adjustable electric bed 1.5 or 0 points adjustable stretcher 1.5 or 0 points dependency level max score 3 points climate condition max score 2 points workspace condition max score 5 points primary aid max score 6 points sliding sheet 1.5 or 0 points transfer sheet 1.5 or 0 points rotating disc 1.5 or 0 points walker 1.5 or 0 points patient to nurse ratio 1 or 0 points night shift 1 or 0 points colleague support 1 or 0 points rest 1 or 0 points information 0.5 or 0 points theoretical training 0.5 or 0 points training period 0.5 or 0 points evaluation of training 0.5 or 0 points danger 0.5 or 0 points planning 0.5 or 0 points load 0.5 or 0 points rest 0.5 or 0 points secondary aids max score 6 points work culture max score 4 points training max score 2 points risk perception max score 2 points final score max score 30 points ijahp article: singh, suthar/development of a risk assessment method for small-sized hospitals using ahp: a case in northern india international journal of the analytic hierarchy process 317 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 table 9 type and number of units in hospitals hospital code type no of units hospital -h1 government hospital 6 hospital -h2 psychiatric hospital 9 hospital -h3 multi-specialist 25 hospital -h4 self-financing hospital 4 hospital -h5 multi-specialist 19 hospital -h6 heart specialist 25 hospital -h7 multi-specialist 42 total units observed 130 3.3.1 risk levels obtained by the final score to obtain the final value of the risk level of the unit or service evaluated, the score obtained by all the items was added to a maximum of 30 points, which was subdivided into three risk levels (table 10) with their respective color codes. table 10 categories for level of risk risk level score range significance green from 20.01-30 points. the risk of nurses suffering musculoskeletal disorders during the handling of the patient is acceptable. yellow from 10.01-20 points. the risk of nurses suffering musculoskeletal disorders during the handling of the patient is moderate. red from 0.8-10 points. the risk of nurses suffering musculoskeletal disorders during the handling of the patient is unacceptable. 3.4 reliability of the method a method is considered highly reliable if it produces similar results under the same conditions. various kinds of reliability coefficients, with values ranging between 0.00 (much error) and 1.00 (no error), can be used to indicate the amount of error in the scores. there are several general classes of reliability estimates; however, in the current study, there was a need to measure the test-retest reliability and internal consistency to test the reliability of the method. 3.4.1 test-retest reliability there was only one observer to collect the data; therefore, consistency of measure was ijahp article: singh, suthar/development of a risk assessment method for small-sized hospitals using ahp: a case in northern india international journal of the analytic hierarchy process 318 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 assessed by calculating the test-retest reliability. there are various coefficients for measuring the test-retest reliability. for the data collected in the current study, the interclass correlation coefficient (icc) is the appropriate measure for reliability because the quantitative measurements made on the units were organized into groups. consistency was required between the different groups of the same type of unit in a hospital. the selection of the appropriate type of icc was done by using the guidelines given by koo and li (2016). two parameters need to be defined for a type of icc, (a) model and (b) type. in this study, the rater (observer) was fixed and the subject was selected randomly, so the two-way mixed model was used (model=3) with a single measure type (type=1). the calculation of the icc (3, 1) was done using the spss (landers, 2015). cicchetti (1994) gives the following (table 11) guidelines for interpretation for icc test-retest agreement measures: table 11 interpretation for icc test measures 3.4.2 internal consistency the classical measure for the internal consistency of an assessment method is cronbach’s alpha (α). cronbach’s alpha was calculated using the suitable statistical software package. 3.5 validation of the method the construct validity is relevant in this study. construct validity refers to the correctness of the conclusion about the relationship between two variables. in the current study, the data was collected using one method; therefore, it cannot be compared with the previous established method and accident rate in the hospital. therefore, it is more important to consider the construct validity. 3.5.1 construct validity the data was divided into three groups (group i, group ii and group iii) to establish the construct validity (see table 12). to establish the construct validity, three null hypotheses were proposed and tested for acceptance/rejection of the alternative hypotheses; therefore, a student’s t-test was used to test the same at a 95% confidence level. sr no. agreement level measure 1 less than 0.40 poor 2 between 0.40 and 0.59 fair 3 between 0.60 and 0.74 good 4 between 0.75 and 1.00 excellent ijahp article: singh, suthar/development of a risk assessment method for small-sized hospitals using ahp: a case in northern india international journal of the analytic hierarchy process 319 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 table 12 groups of units and their description groups description group i this group consists of the operation theaters, recovery units, psychiatric units, critical care units (ccu), surgical wards and intensive care units (icu) because of the availability of the handling equipment in these units. group ii this group consists of all types of private rooms, which have more probability of risk exposure than group i, but have less exposure then group iii. group iii this group includes all types of general wards. this group will have the lowest score which means higher risk exposure. 3.5.2 content validity content validity refers to how well the method measures what it intended to measure. in evaluation, the content validity index (cvi) is the most widely used index. many studies suggest that factors with an i-cvi of 0.78 or higher for three or more experts could be considered evidence of good content validity. also, a s-cvi (cvi) value greater than 0.9 shows good agreement between the experts. therefore, in the current study the experts were from hospitals or were scholars who were already working in human safety. there were six experts who rated the factors considered in the method. 3.5.3 calculation of most occurring factor in the observed hospital simple arithmetic was used to find the most occurring factor that increased the risk exposure. the total score obtained for a factor was divided by the maximum score obtained; therefore, the conjugate of this fraction gave the contribution. further, the most contributing factor can be found for each hospital and for all combined hospital units by using the following formula. % occurance = (1 − score obtained for a factor in a hospital maximum score which can be obtained ) × 100 4. results to test the reliability of the assessment method, the interclass correlation coefficient (icc) was calculated using spss; icc values greater than 0.60 represent good test-retest reliability and a value less than 0.4 is considered poor reliability. the results of this phase may be observed in table 13. ijahp article: singh, suthar/development of a risk assessment method for small-sized hospitals using ahp: a case in northern india international journal of the analytic hierarchy process 320 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 table 13 groups of units and their description hospitals h1 and h4 were excluded from this test of reliability because the data from both hospitals were less as far as number of units to test reliability. the data showed two types of icc, single measure and average measure. in this study, only one rater (observer) was present; therefore, the single measure icc will give the correct measure of reliability. the results show the values of icc (single measure) around 0.6, so the assessment is considered good if the reliability of the method is considered. thus, the method tested in this section has good reliability. 4.1 internal consistency the cronbach’s alpha (α) is a special type of icc and is equivalent to the icc (3, k). therefore, to confirm the above results about the reliability of the method, it was obtained from spss as shown in table 14. the value of alpha (α) should remain greater than 0.7 for consistency in the assessment. the values range from 0.7 to 0.8, which reflects the consistency of the tested method. therefore, with the results of test-retest reliability and the internal consistency it can be concluded that the method used to assess risk exposure in the job of patient handling gives consistent results and is a stable procedure to assess the risk involved. table 14 result of internal consistency, the cronbach’s alpha (α) is a special type of icc hospital no. of units tested cronbach’s alpha (α ) h2 8 0.746 h3 18 0.756 h5 16 0.741 h6 14 0.716 h7 32 0.801 hospital no of unit tested interclass correlational coefficient (icc) single measure average measure h2 8 0.595 0.746 h3 18 0.607 0.756 h5 16 0.588 0.741 h6 14 0.558 0.716 h7 32 0.668 0.801 ijahp article: singh, suthar/development of a risk assessment method for small-sized hospitals using ahp: a case in northern india international journal of the analytic hierarchy process 321 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 4.2 validity 4.2.1 construct validity the method is validated if group (i) gets a higher score than groups (ii) & (iii) and the score of group (ii) is greater than the score of group (iii). therefore, the hypotheses assumed for equality of means has to be rejected with higher significance. the difference between the mean of the data of each group is shown in the table 15. table 15 mean and variance of different group of units group mean n std. deviation variance group i 18.06 37 1.64 2.711 group ii 14.95 82 1.17 1.39 group iii 11.57 11 0.51 0.27 a mere observation of the data shown above confirms that the means of the data are decreasing continuously from group (i) to group (iii). it was previously established that group (i) should get a higher score because of the availability of handling equipment and other facilities in these units. this statement is supported by the higher mean of group (i), also statistically confirmed for rejecting the hypothesis that the units receive an equal score. the t-test performed on the data of the three groups of units using spss gives the results exhibited in table 16. table 16 results of construct validity levene’s test equality of means t-test for equality of variances hypothesis p value mean difference t value significance 1. between group i & ii 0.03 3.11 15.55 a 0.001 a 2. between group ii & iii 0.009 3.38503 16.618 a 0.001 a 3. between group i & iii 0.018 6.49509 20.767 a 0.001 a the spss package first performed a test for equality of variance known as levene’s test, where the value of ‘p’ is less than 0.05. the t-test performed on the condition of inequality of variance confirmed the statistically significant difference between the means of each group. hence, the alternative hypothesis was accepted with a very high significance (95% level of significance). the value of ‘t’ was a high positive number, which means a large difference between the means of the scores. group (i), group (ii) and group (iii) of the units and the positive value confirms that the score of group i is higher than the score of group ii and so on. therefore, the validation analysis of the method shows that this method is useful for assessing the risk exposure in patient handling as well as for finding the factor that requires immediate attention or is the main cause of inappropriate handling. additionally, this method can be used to assess the effectiveness of the ergonomic intervention or the effect of employ new handling equipment. ijahp article: singh, suthar/development of a risk assessment method for small-sized hospitals using ahp: a case in northern india international journal of the analytic hierarchy process 322 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 4.2.1 content validity content validity refers to the extent to which the factors of a test or method can measure or assess what the test intended (wynd et al., 2003). one of the most widely used measures of content validity is the content validity index (cvi). a panel of content experts was asked to rate each item on a scale in terms of its relevance to the underlying construct. the experts for this study included research supervisors and doctors from different hospitals. there were six experts used to determine the content validity. it is advisable to have a minimum of three experts to determine the cvi. the value of i-cvi should be greater than 0.75, and the value of s-cvi should be greater than 0.92 for the factor to be relevant for assessing the risk in patient handling. 4.2.2 most occurring factor in observed hospital figure 3 shows that factor 5 has the highest percentage occurrence (74.8%). thus, based on the above results, it is concluded that factor 5 has the highest occurrence of contributing to an increased risk exposure among all the seven hospitals. therefore, the inclusion of the secondary type of mechanical aids (such as handling crane, adjustable stretcher, adjustable electric bed, transfer sheets, standing crane, etc.) will reduce the risk exposure in the seven hospitals. figure 3 results of the most contributing factor when combining all hospitals ijahp article: singh, suthar/development of a risk assessment method for small-sized hospitals using ahp: a case in northern india international journal of the analytic hierarchy process 323 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 description of the factors: factor 1level of dependency factor 2climate conditions factor 3work places conditions factor 4primary aids factor 5secondary aids factor 6work culture factor 7training factor 8risk perception the radar diagram exhibited in figure 3 plots the overall contributing factors for all hospitals covered under this study. in six out of seven hospitals, factor 5 (secondary aids) emerges as the factor with the highest percentage contribution whereas only one hospital was found with factor 7 (training) as the most dominating factor. however, the overall radar diagram shows factor 5 as a most contributing factor confirming the results from previous diagrams. figure 4 percentage contribution of the each factor. figure 4 above shows the highest percentage contribution (74.8%) for factor 5. therefore, from all the above results, it can be concluded that factor 5 has the highest contribution to increasing the risk exposure in the sampled hospitals. the inclusion of the secondary type of mechanical aids (such as handling crane, adjustable stretcher, adjustable electric bed, transfer sheets, standing crane, etc.) will reduce the risk exposure in the hospitals. 5. discussion 5.1 the observation method the observation method was developed to meet the need for a simple and practical tool to assess handling techniques during patient transfer tasks. the main application would be in evaluation of intervention effects. the method includes all required aspects of ijahp article: singh, suthar/development of a risk assessment method for small-sized hospitals using ahp: a case in northern india international journal of the analytic hierarchy process 324 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 patient handling, which various researchers previously used. these items are well tested in various parts of the world under different conditions. the new feature of the presented method is that it includes almost all the critical aspects of patient handling. the scoring system is based on documented risk factors for musculoskeletal disorders and on aspects of work technique related to generally accepted ergonomic principles. a similar system used in this method is called hempa (kjellberg et.al, 2000). however, this method is partially different so that it is applicable in hospitals in non-metropolitan cities in developing countries like india. here, some factors were excluded from the method to make it easier to implement. therefore, it considers fewer factors than hempa. the excluded factors make the method more accessible for the nursing staff in the indian health care industry and institutions. 5.2 reliability of the method the reliability of the method was assessed using two similar aspects of reliability. the first was the test-retest reliability, which checks the agreement between observations taken on same subject at a different time points. the test-retest reliability was assessed using the interclass correlation coefficient (icc). the value of icc was approximately 0.6 for different hospitals, which reflects a good relationship between observations. the other aspect considered was internal consistency, which was assessed using the cronbach’s alpha. both the interclass correlation coefficient (icc) and the cronbach’s alpha showed similar results. the analysis of the internal consistency evaluation showed that the method presents homogeneity among its items. it was not necessary to remove any item from the instrument, as the data revealed an alpha ranging from 0.746 to 0.801 for different hospitals, demonstrating that the current proposed method is reliable. hence, both the measures of reliability concluded that the method is reliable in its observation and produces consistent results. 4.3 validity of the method validity refers to the extent to which the method/instrument assesses the factors that it is supposed to. validation of the current method was done by evaluating the construct validity. for the validation, the units were divided in three groups in terms of their facility for patient handling. the nursing staff working at group ii type of units was shown to be at greater risk than the nursing staff working at group i units. the results of the method at group i and group ii type of units confirmed the above observations. therefore, the t-test was performed to statistically verify that group i has a higher score than group ii, group ii has a higher score than group iii and eventually group i has a higher score than group iii. the results of the t-test reject hypotheses with an equal score between the groups with a high significance level. hence, the t-test confirmed that the score of group i type of units is higher than group ii type of units, which eventually validates the method for assessing the risk exposure in patient handling. in addition, the content validity index shows that the factors chosen for the method are relevant to assessment of the risk exposure. 5.4 applicability the method proposed and developed in this study is applicable in small-scale hospitals in non-metropolitan cities. this method is able to determine the prevalence of musculoskeletal disorder risk, low back injury, and other associated risks among the patient attendants. the individuals who have awareness and training with knowledge in transfer methods and ergonomics can easily adopt and use this method. there is a need to provide initial training to learn the items, definitions, and scoring system. the ijahp article: singh, suthar/development of a risk assessment method for small-sized hospitals using ahp: a case in northern india international journal of the analytic hierarchy process 325 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 method is applicable in wards, in patient’s homes or in other places where transfers occur, as no special equipment is needed. therefore, it has significant applications in assessing risk exposure in patient handling. 6. conclusion the handling of bedridden patients with mobility deficiencies constitutes one of the main risk factors for the health of their caregivers. in the health sector, there is a high incidence of musculoskeletal disorders. many times, injuries are associated with the mobilization of patients, but also linked to the adoption of forced or static postures maintained over time, performing repetitive movements throughout the workday or manipulation of objects, such as medication carts or stretchers. many epidemiological studies of personnel involved in the care of patients show the presence of acute low back pain with a higher prevalence with respect to other populations of workers exposed to physical risks. therefore, the incorporation of mechanical equipment to help with mobilization is a remarkable preventive measure, since it promotes work postures and minimum manipulation effort by the worker. the current study developed a method to evaluate the risk exposure in patient handling, based on the previously established method. construct validity presented a significant difference between all three groups of units, demonstrating that the method was able to detect these differences. the internal consistency analyzed by icc and cronbach’s alpha coefficient demonstrated that the method is consistent in observation. the results indicated that the method is viable and may contribute to better planning for nursing assistance, together with other preventive ergonomic strategies in small-scale hospitals in non-metropolitan cities such as jalandhar and areas around punjab in india and in other developing countries. therefore, the method is significant for health care workers to systematically plan the level of assistance needed in hospitals with settings and facilities similar to india. ijahp article: singh, suthar/development of a risk assessment method for small-sized hospitals using ahp: a case in northern india international journal of the analytic hierarchy process 326 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 references akarte, m., surendra, n., ravi, b., and rangaraj, n. (2001). web based casting supplier evaluation using analytical hierarchy process. journal of the operational research society, 52(5), 511-522. doi: https://doi.org/10.1038/sj.jors.2601124 al-harbi, k. m. a. (2001). application of the ahp in project management. international journal of project management, 19(1), 19-27. doi: https://doi.org/10.1016/s0263-7863(99)00038-1 andersson, r., and menckel, e. (1995). on the prevention of accidents and injuries: a comparative analysis of conceptual frameworks. accident analysis & prevention, 27(6), 757-768. doi: https://doi.org/10.1016/0001-4575(95)00031-3 arbel, a., and orgler, y. e. (1990). an application of the ahp to bank strategic planning: the mergers and acquisitions process. european journal of operational research, 48(1), 27-37. doi: https://doi.org/10.1016/0377-2217(90)90058-j armacost, r. l., componation, p. j., mullens, m. a., and swart, w. w. (1994). an ahp framework for prioritizing customer requirements in qfd: an industrialized housing application. iie transactions, 26(4), 72-79. doi: https://doi.org/10.1080/07408179408966620 battevi, n., menoni, o., ricci, m. g., & cairoli, s. (2006). mapo index for risk assessment of patient manual handling in hospital wards: a validation study. ergonomics, 49(7), 671-687. doi: https://doi.org/10.1080/00140130600581041 bordini, l., de vito, g., molteni, g., & boccardi, s. (1999). epidemiologia delle alterazioni muscolo-scheletriche da sovraccarico biomeccanico del rachide nella movimentazione manuale di pazienti. medicina del lavoro, 90(2), 103-116. cato, c., olson, d. k., & studer, m. (1989). incidence, prevalence, and variables associated with low back pain in staff nurses. aaohn journal, 37(8), 321-327. doi: https://doi.org/10.1177/216507998903700804 colombini, d., riva, f., lue, d., nava, c., petri, a., basilico, s. & menoni, o. (1999). initial epidemiological data on the clinical effects in health workers employed in the manual lifting of patients in wards. la medicina del lavoro, 90(2), 201-228. de castro, a. b., hagan, p., & nelson, a. (2006). prioritizing safe patient handling: the american nurses association’s handle with care campaign. journal of nursing adinistration, 36(7-8), 363-369. doi: https://doi.org/10.1097/00005110-200607000-00009 engkvist, i. l., hagberg, m., hjelm, e. w., menckel, e., ekenvall, l., & prosa study group. (1998). the accident process preceding overexertion back injuries in https://doi.org/10.1080/00140130600581041 ijahp article: singh, suthar/development of a risk assessment method for small-sized hospitals using ahp: a case in northern india international journal of the analytic hierarchy process 327 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 nursing personnel. scandinavian journal of work, environment & health, 367-375. doi: https://doi.org/10.5271/sjweh.357 garg, a., owen, b., beller, d., & banaag, j. (1991). a biomechanical and ergonomic evaluation of patient transferring tasks: wheelchair to shower chair and shower chair to wheelchair. ergonomics, 34(4), 407-419. doi: https://doi.org/10.1080/00140139108967325 goldman, r. h., jarrard, m. r., kim, r., loomis, s., & atkins, e. h. (2000). prioritizing back injury risk in hospital employees: application and comparison of different injury rates. journal of occupational and environmental medicine, 42(6), 645-652. doi: https://doi.org/10.1097/00043764-200006000-00016 hignett, s., & mcatamney, l. (2004). rapid entire body assessment. handbook of human factors and ergonomics methods (pp. 97-108). crc press. doi: https://doi.org/10.1201/9780203489925.ch8 johnsson, c., kjellberg, k., kjellberg, a., lagerstr¨om, m (2004). a direct observation instrument for assessment of nurses’patient transfer technique (dino). in: applied ergonomics, 35(6), 591–601. doi: https://doi.org/10.1016/j.apergo.2004.06.004 karhu, o., härkönen, r., sorvali, p., & vepsäläinen, p. (1981). observing working postures in industry: examples of owas application. applied ergonomics, 12(1), 13-17. doi: https://doi.org/10.1016/0003-6870(81)90088-0 kee, d., & karwowski, w. (2001). luba: an assessment technique for postural loading on the upper body based on joint motion discomfort and maximum holding time. applied ergonomics, 32(4), 357-366. doi: https://doi.org/10.1016/s0003-6870(01)00006-0 kjellberg, k., johnsson, c., proper, k., olsson, e., & hagberg, m. (2000). an observation instrument for assessment of work technique in patient transfer tasks. applied ergonomics, 31(2), 139-150. doi: https://doi.org/10.1016/s0003-6870(99)00046-0 koo, t. k., & li, m. y. (2016). a guideline of selecting and reporting intraclass correlation coefficients for reliability research. journal of chiropractic medicine, 15(2), 155-163. doi: https://doi.org/10.1016/j.jcm.2016.02.012 landers, r. (2015). computing intraclass correlations (icc) as estimates of interrater reliability in spss, the winnower 2. doi: https://doi.org/10.15200/winn.143518.81744 national research council (2001). musculoskeletal disorders and the workplace: low back and upper extremities. marmaras, n., poulakakis, g., & papakostopoulos, v. (1999). ergonomic design in ancient greece. applied ergonomics, 30(4), 361-368. doi: https://doi.org/10.1016/s0003-6870(98)00050-7 ijahp article: singh, suthar/development of a risk assessment method for small-sized hospitals using ahp: a case in northern india international journal of the analytic hierarchy process 328 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.771 mcatamney, l., & corlett, e. n. (1993). rula: a survey method for the investigation of work-related upper limb disorders. applied ergonomics, 24(2), 91-99. doi: https://doi.org/10.1016/0003-6870(93)90080-s osha (2009). ergonomics for the prevention of musculoskeletal disorders: guidelines for nursing homes. department of labor, occupational safety and health administration. doi: https://doi.org/10.1037/e372172004-001 putz-anderson, v., bernard, b. p., burt, s. e., cole, l. l., fairfield-estill, c., fine, l. j. & nelson, n. (1997). musculoskeletal disorders and workplace factors. national institute for occupational safety and health (niosh), 104. saaty, t.l. (1990). how to make a decision: the analytic hierarchy process. european journal of operational research, 48, 9−26. doi: https://doi.org/10.1016/0377-2217(90)90057-i warming, s., juul-kristensen, b., et al. (2004). an observation instrument for the description and evaluation of patient transfer technique”. in: applied ergonomics 35.6, pp. 603–614. doi: https://doi.org/10.1016/j.apergo.2004.06.007 shrout, p. e., & fleiss, j. l. (1979). intraclass correlations: uses in assessing rater reliability. psychological bulletin, 86(2), 420. doi: https://doi.org/10.1037/0033-2909.86.2.420 stubbs, d. a., buckle, p. w., hudson, m. p., rivers, p. m., & baty, d. (1986). backing out: nurse wastage associated with back pain. international journal of nursing studies, 23(4), 325-336. doi: https://doi.org/10.1016/0020-7489(86)90055-6 villarroya, a., arezes, p., de freijo, s. d., & fraga, f. (2017). validity and reliability of the hempa method for patient handling assessment. applied ergonomics, 65, 209-222. doi: https://doi.org/10.1016/j.apergo.2017.06.018 waters tr., (2010). introduction to ergonomics for healthcare workers. rehabilitation nursing. ijahp article: milkova, andreichikova, andreichikov/at the junction of mathematics and psychology: cognitive orientation of the ahp/anp and new perspectives of structuring complexity international journal of the analytic hierarchy process 110 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.611 at the junction of mathematics and psychology: cognitive orientation of the ahp/anp and new perspectives of structuring complexity 1 maria milkova central institute of economics and mathematics of the russian academy of science m.a.milkova@gmail.com olga andreichikova central institute of economics and mathematics of the russian academy of science alexandrol@mail.ru alexander andreichikov central institute of economics and mathematics of the russian academy of science andreichickov@mail.ru abstract this paper aims to draw attention to the interdisciplinary research of the ahp/anp methodology by emphasizing how it can be studied from a cognitive perspective. we provide an overview of the main cognitive approaches in decision-making, and consider different heuristics that lie at the basis of pairwise comparisons. we emphasize that the ahp/anp must be considered at the junction of mathematics and psychology, and for further development of the methodology, we should examine the ahp/anp from the cognitive point of view. we review the recent experimental studies of the ahp/anp that test human behavior in real decision problems. we also discuss the future applicability of the ahp/anp methodology in the experience age the age of not only digital information and knowledge, but also behavior. this article is just a small step on the way to discovering the cognitive aspects and future extensions of decision making with the ahp/anp. keywords: cognitive decision making; ahp; anp; heuristics; heuristic decision making; cognitive psychology; experience age 1 the reported study was funded by rfbr according to the research project № 19-010-00293 "development of methodology, economic and mathematical models, methods and decision support systems for search studies to identify opportunities for import substitution of high-tech products based on the world patent and financial information resources" mailto:m.a.milkova@gmail.com mailto:alexandrol@mail.ru ijahp article: milkova, andreichikova, andreichikov/at the junction of mathematics and psychology: cognitive orientation of the ahp/anp and new perspectives of structuring complexity international journal of the analytic hierarchy process 111 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.611 1. introduction decision making is one of the fundamental cognitive processes of human beings. how are decisions made? gigerenzer and gaissmaier (2011) proposed three major answers to this question. they proposed that the mind applies logic, statistics, or heuristics. each of these is suited to a particular kind of problem. while rational thinking is common in studying complex problems, analytical approaches lose their relevance when the probabilities of uncertain events are not adequately computed in formal models. in that case, heuristics is often the only practical method for decision making under uncertainty. the term “heuristics 2 ” implies an efficient cognitive process, conscious or unconscious, that ignores part of the information (gigerenzer & gaissmaier, 2011). the classical view of heuristics states that heuristic decisions imply greater errors than do “rational” decisions as defined by logic or statistical models. on the other hand, many studies show that a good heuristic can be better than a complex strategy when used in the proper environment (mousavi & gigerenzer, 2014). in spite of different views on the nature of heuristics, the uncontestable fact is that in a world of uncertainty heuristics is an indispensable tool. continuing the idea that our decisions are governed more by heuristics than by logic, how we use heuristics conforms to the natural human practice of making comparisons. regarding saaty’s concept of decision processes, the ahp/anp are psychophysical theories of measurement where comparisons are made based on feelings and judgments (saaty, 2008). thus, heuristics lies at the basis of the ahp/anp pairwise comparisons. for all that has been written about the ahp/anp, much misunderstanding still exists (whitaker, 2007). moreover, the ahp/anp concepts have been both highly praised and strongly criticized. this dichotomy is largely due to the difficulty of testing the methods because the ahp/anp incorporates both quantitative and qualitative criteria (ishizaka, et. al., 2011). from our point of view, in order to give convincing support of the applicability of the ahp/anp in the decision making process, we should follow the new way of cognitive orientation of the methodology and use the methods of experimental economics that help understand human behavior while making decisions with the ahp/anp. the ahp/anp is not just a mathematical theory, but an interdisciplinary approach that lies at the junction of mathematics and psychology, conscious and unconscious, tangible and intangible, subjective and objective truth. it is this interdisciplinary approach that we need to adopt in the new reality, which is the post-digital 3 age. increasingly, we are no longer in a world where digital technology and media are separate from a ‘natural’ human and social life (jandrić, et al., 2018). according to simon jenkins (the guardian, 2017), “we are now heading for “post-digital”, the age of experience”. the experience age is 2 heuristics a way of solving problems by discovering things yourself and learning from your own experiences (cambridge dictionary). 3 the term postdigital has in recent years been applied across a broad range of disciplines, often with contradictory meanings. the term postdigital is also entering the academic discourse. to map the various definitions, deployments and discussions see taffel (2015), jandrić, et.al. (2018). ijahp article: milkova, andreichikova, andreichikov/at the junction of mathematics and psychology: cognitive orientation of the ahp/anp and new perspectives of structuring complexity international journal of the analytic hierarchy process 112 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.611 the age of not only digital information and knowledge, but also behavior. today, information about our activities, feelings, movements and thoughts is often collected without our consent, and our perception of reality can be manipulated by nudging 4 . however, applying the same measure to the entire population (even to clusters of the population) would not be good, but far too little is known to take appropriate individual measures (helbing, 2019). the aim of this paper is to draw attention to the interdisciplinary research of the ahp/anp methodology by emphasizing how it can be studied from a cognitive perspective. this paper is organized as follows: section 2 provides an overview of studies that aimed to verify the practicality of ahp/anp using laboratory experiments. section 3 describes common heuristic approaches in decision-making. section 4 summarizes the cognitive aspects of decision making with the ahp/anp. section 5 provides the discussion of future applicability of ahp/anp in the experience age. section 6 presents our conclusions. 2. experimental validation of the ahp/anp despite the fact that the ahp/anp has been applied in a diverse range of areas, there is no clear evidence that the methods provide its users with their ‘best’ choice and not an arbitrary one (ishizaka et. al., 2011). it seems difficult to assess whether a satisfying choice has been made by the decision makers because feedback on the decision may be very slow. the reason that the ahp/anp methodology is underestimated is probably because there are not enough controlled laboratory experiments testing the approach with nonmeasurable decision criteria and analyzing participants’ level of satisfaction after the experiment. the authors believe that accumulating controlled laboratory tests on basic everyday decisions would lead to growth in the popularity of the ahp/anp in more important problems. experimental validation with subjective results is more convincing than the techniques with verifiable objective results because they deal with problems where the ahp/anp is more likely to be applied (ishizaka et al., 2011). as a first step, we provide an overview of experimental studies that test the ahp on elementary decision problems. huizingh and vrolijk (1997) asked 180 participants to solve the hypothetical problem of choosing a room to rent. the authors observed that participants were more satisfied with the ahp result than with a random selection. in brugha (2000), two groups of 10 4 nudging – a concept that proposes indirect suggestions as ways to influence the behavior and decision making of groups or individuals. examples of nudging are personalized prices, special offers, different advertisements, and etc. ijahp article: milkova, andreichikova, andreichikov/at the junction of mathematics and psychology: cognitive orientation of the ahp/anp and new perspectives of structuring complexity international journal of the analytic hierarchy process 113 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.611 students were asked to solve the hypothetical problem of a career and a car selection. it was observed that the participants preferred to use scoring with intervals (scoring with respect to a reference) over relative measurement (as in the ahp), but relative measurement was preferred when intervals were difficult to identify. the results calculated by the methods were not compared, probably because it was a fictitious problem. korhonen and topdagi (2003) empirically investigated the performance of the ahp, when the utility of the objects cannot be evaluated on the same ratio scale. this kind of problem occurs when a decision maker is asked to compare, for instance, the objects (s)he likes to the objects (s)he hates. in that case, the authors expected very poor performance of the ahp because each object the decision maker likes is presumably “absolutely better” than any object (s)he hates. to test the hypothesis, four vegans and four non-vegans used the ahp to rank meals described on paper. however, the results demonstrated that the ahp is able to estimate the reasonable utility values for objects very well. in another experiment, brugha (2004) asked 53 students to choose what they would do next year. it was observed that they preferred to use simple methods for screening and more elaborate methods for ranking (smart, maut and ahp). the results calculated by the methods were not analyzed, probably because it was a fictitious problem. ishizaka et al. (2011) tested how well the ahp fares as a choice support system in a real decision problem. the authors tested the problem of selecting a box of five chocolates. the ranking provided by the ahp was statistically compared with three additional rankings given by the subjects in the experiment, one at the beginning, one after providing the ahp with the necessary pairwise comparisons and one after learning the ranking provided by the ahp. while the rankings varied widely across subjects, the authors observed that for each individual all four rankings were similar. hence, subjects were consistent and the ahp was, for the most part, able to replicate their rankings. furthermore, while the rankings were similar, the authors found that the ahp ranking helped the decision makers reformulate their choices by taking into account suggestions made by the ahp. thus, by reviewing actual laboratory experiments it is shown that the ahp is useful in assisting the decision making process. however, to give convincing support of the applicability the ahp/anp in the decision-making process, we should provide more laboratory experiments that stress the decision-making process with the ahp/anp (note that we could not find any experimental studies that tested the applicability of the anp). 3. heuristics overview as we move from the laboratory to reality, it is obvious that our life is full of choices that are too unique to lend any useful data for analysis except heuristics. therefore, we will provide an overview of common heuristic approaches for decision making under uncertainty. simon (1957) was the first to propose an alternative basis for the mathematical modeling of decision making by introducing a theory of “bounded rationality” that states when ijahp article: milkova, andreichikova, andreichikov/at the junction of mathematics and psychology: cognitive orientation of the ahp/anp and new perspectives of structuring complexity international journal of the analytic hierarchy process 114 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.611 individuals make decisions, their rationality is limited by the tractability of the decision problem, the cognitive limitations of their minds, and the time available to make the decision. simon suggests that people use heuristics to make decisions rather than a strict rigid rule of optimization. in this view, decision makers seek a satisfactory solution rather than an optimal one (simon, 1956). exploring the territory of bounded rationality that simon (1957) had defined, amos tversky and kahneman (2003) developed their own perspective on heuristics in human decision-making. a judgment is said to be mediated by a heuristic when the individual assesses a specified target attribute of a judgment object by substituting a related heuristic attribute that comes more readily to mind. in an earlier work, tversky and kahneman (1974) described three judgmental heuristics: (1) representativeness, (2) availability, and (3) anchoring and adjustment. these heuristics underlie many intuitive judgments. we will briefly describe each judgmental heuristic. representativeness is usually employed when people are asked to judge the probability that object a belongs to class or process b. an individual thing has a high representativeness for a category if it is very similar to a prototype of that category. consider the following problem: bob is an opera fan who enjoys touring art museums when on holiday. growing up, he enjoyed playing chess with family members and friends. which situation is more likely? a. bob plays trumpet for a major symphony orchestra b. bob is a manager a large proportion of people will choose a in the above problem because bob’s description matches the stereotype we may hold about classical musicians rather than managers. in reality, the likelihood of b being true is far greater because managers make up a much larger proportion of the population. availability is employed when people are asked to assess the frequency of a class or the probability of an event by the ease with which instances or occurrences can be brought to mind. when an infrequent event can easily be brought to mind, people tend to overestimate its likelihood. for example, investors may judge the quality of an investment based on information that was recently in the news, ignoring other relevant facts (tversky & kahneman, 1974). adjustment from an anchor is usually employed in numerical prediction when people make estimates by starting from an initial value that is adjusted to yield the final answer. different starting points yield different estimates, which are biased toward the initial values. for example, participants received comprehensive information regarding a property, including either a high or low list price, before touring the property. finally, participants were asked to estimate the actual value of the property. when the list price was high, the final estimates also tended to be elevated. ijahp article: milkova, andreichikova, andreichikov/at the junction of mathematics and psychology: cognitive orientation of the ahp/anp and new perspectives of structuring complexity international journal of the analytic hierarchy process 115 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.611 in their experiments, tversky and kahneman (1974, 2003) showed that people acting under uncertainty rely on a limited number of heuristic principles which reduce the complex tasks of assessing probabilities and predicting values to simpler judgmental operations. nevertheless, kahneman and tversky (1996) argue that cognitive biases have efficient practical implications for areas including clinical judgment, entrepreneurship, finance, and management. thus, according to kahneman and colleagues, the usage of heuristics saves effort but at the cost of accuracy. in this view, humans and other animals rely on heuristics because searching for information and making computations costs time and effort; the trade off with heuristics is some loss in accuracy but with faster and more frugal cognition (gigerenzer & gaissmaier, 2011). in contrast, gigerenzer and goldstein (1996) argue that heuristics are "fast and frugal" and refer to simple, task-specific decision strategies that can be used to make judgments that are accurate rather than biased. unlike statistical optimization procedures, heuristics do not try to optimize (i.e., find the best solution), but rather satisfy (i.e., find a goodenough solution). gigerenzer and gaissmaier (2011) review four classes of fast and frugal heuristics. the first class exploits recognition memory, the second relies on one good reason only (and ignores all other reasons), the third weights all cues or alternatives equally, and the fourth relies on social information. the studies of fast and frugal heuristics have shown that less effort can lead to judgments that are more accurate. for example, heuristics from the one-reason heuristics group (e.g. take-the-best 5 , hiatus 6 ) often predict more accurately than multiple regression, linear and bayesian models, neural networks, and decision-tree algorithms, and show the same or better performance than pareto/nbd model (gigerenzer & goldstein, 1996; czerlinski et al., 1999; martignon & hoffrage, 2002; brighton, 2006; wübben & wangenheim, 2008). as another example, tallying heuristics to estimate a criterion does not estimate weights but simply counts the number of positive cues. it has been shown that tallying often predicts with equal or greater accuracy than multiple regression (czerlinski et al., 1999). nevertheless, a criticism of the “less is more effect” also occurs in the literature, possibly because the effect is predicted to be small (pachur & biele, 2007; pohl, 2006; katsikopoulos et al., 2010). in any case, the results of all research put heuristics on par with the standard statistical models of “rational” cognition (gigerenzer & gaissmaier, 2011). 5 take-the-best to infer which of two alternatives has the higher value, (a) search through cues in order of validity; (b) stop search as soon as a cue discriminates; (c) choose the alternative this cue favors (for more details see for example gigerenzer & goldstein, 1996) 6 hiatus set an interval (the hiatus) and infer whether one selected criterion is active or not during the hiatus (for more details see for example wübben & wangenheim, 2008). ijahp article: milkova, andreichikova, andreichikov/at the junction of mathematics and psychology: cognitive orientation of the ahp/anp and new perspectives of structuring complexity international journal of the analytic hierarchy process 116 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.611 4. ahp/anp as a cognitive process many comparative studies of heuristics and analytical approaches have been done, but we could not find any research about the comparative evaluation of heuristics and psychophysical methods such as the ahp/anp. could we apply heuristics without sufficiently reducing the complexity of the problem? could we combine logic and heuristics? saaty (2015) emphasizes that the anp mathematical approach aims at capturing the grand design and modularity of the brain and its workings. as the main functions of the ahp/anp are to (1) structure complexity, (2) measure preferences, and (3) synthesize the results, we will consider them as three parts of the cognitive process that underlie decision making. 4.1 structuring complexity structuring the complexity is the basis of decision-making. when facing a choice between options that differ on several decision-relevant attributes, one could rely on the most relevant attribute only to make the decision, or one could integrate information from several attributes and base the choice on that combined information. understanding the cognitive processes underlying decision-making is one of the most important subjects of psychological research. as the brain works through the functioning of its networks of neurons, where neurons are decision makers deciding to fire or not to fire, interactions of the neurons are similar to the flow of influences in decision-making. thus, our perception of reality is better represented through a network that enables us to more accurately understand, control, and predict happenings in the world around us (saaty, 2015) 7 . however, if the considered problem is intricate, a designed network structure may be very large, which can lead to reduced concentration from the decision maker on the procedure of pairwise comparisons. thus, it is the decision makers’ responsibility to build a hierarchy/network structure that represents the reality and is not overburdened with unnecessary details. in order to reduce the problem of a high number of judgments, the traditional ahp/anp procedure may be implemented using clusters and pivots, as presented by ishizaka (2012). objects are divided into several ordered clusters such that two adjacent clusters have one common object, the pivot. then, pairwise comparisons are performed for each cluster and priorities are calculated. final priorities are derived by using the pivot to link priorities of each cluster. on the other hand, psychological experiments show that in order to simplify the choice between alternatives, people often disregard components that the alternatives share, and focus on the components that distinguish them (tversky, 1972). this approach to choice 7 in this paper, we do not separate the ahp and anp approaches because they imply the same methodology, but we should remember that anp is recommended for studying complex decision problems. ijahp article: milkova, andreichikova, andreichikov/at the junction of mathematics and psychology: cognitive orientation of the ahp/anp and new perspectives of structuring complexity international journal of the analytic hierarchy process 117 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.611 problems may produce inconsistent preferences because a pair of prospects can be decomposed into common and distinctive components in more than one way, and different decompositions sometimes lead to different preferences (kahneman & tversky, 1979). to take into account these findings, in some decision problems we may assess criteria of possible alternatives before the alternatives are exactly defined (weiss, 1987). thus, the main step of decision making, which is constructing a network (or hierarchy) structure of the problem, is a cognitive process where we should find the right balance between perception and reality. although there are no standard recipes for building the “right” network (or hierarchy) structure, we should remember that in structuring complexity there is an upper limit on our capacity to process information on simultaneously interacting elements with reliable accuracy and validity. this limit is seven plus or minus two elements, as published by miller (1956), as well as in the context of the ahp by saaty and ozdemir (2003). 4.2 measuring preferences according to natural human practice of making comparisons, saaty suggested a scientific way of constructing a subjective scale that can be applied to a diverse range of issues being assessed (saaty, 2008). saaty (2015) also suggests that if there is adequate knowledge, one can compare anything with anything else that shares a common attribute or criterion, but priorities always depend on what other things are compared with. continuing the idea that our decisions are governed more by heuristics than by logic, we emphasize that the procedure of pairwise comparisons involves an inference in which one could rely on judgmental heuristics (such as availability or anchoring and adjustment). reasoning this way, judgments in pairwise comparisons may be skewed due to attribute substitution. when an infrequent event can easily be brought to mind, we tend to overestimate its likelihood (tversky & kahneman, 1974). for example, a driver may estimate the safety of car a “strongly better” than car b because yesterday he saw car b crash on the way to work. or, an investor may judge the quality of an investment to startup b “better” than to startup a because he saw information about startup b in the news more often, ignoring other relevant facts. moreover, due to the endowment effect, we can give higher judgements to things that we already own than to things that we do not own. it is also well documented that humans behave differently depending on their emotional states (loewenstein & lerner, 2003). for example, anger increases risk taking, fear is, in general, correlated with higher risk aversion, and it is not clear how sadness affects risky choices (campos-vazquez & cuilty, 2013). measurement of judgment consistency plays a key role in the procedure of pairwise comparisons. while several authors still suggest that a consistency test is needed for adequate pairwise comparisons, psychological experiments show strong evidence of the irrationality in human decision-making (zhang et al., 2018; tversky & kahneman, 1981). moreover, in recent years a new way of thinking has evolved using psychology and economics that is trying to show that transitivity need not always be satisfied in order to make a rational decision (moreno-jiménez & vargas, 2018). saaty (2013) argues that if humans were always perfectly consistent, they would not be able to learn new things that modify or change the relations among what they knew ijahp article: milkova, andreichikova, andreichikov/at the junction of mathematics and psychology: cognitive orientation of the ahp/anp and new perspectives of structuring complexity international journal of the analytic hierarchy process 118 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.611 before. but, there is a level of tolerable inconsistency that we must allow beyond which the judgments would appear to be uninformed, random, or arbitrary. thus, while people rely on a limited amount of heuristic to make biased judgments, the ahp/anp offers a judgmental structure for evaluation in a consistent manner without simplification of the problem (tversky & kahneman, 1974). while the validity of using the fundamental scale in making comparisons has been validated by physical and decision problem experiments, critiques have been expressed about the distance of the ahp/anp from the axioms of classical utility theory (saaty, 2008; dyer, 1990). saaty (1990) and forman and gass (2001) have always rejected this criticism, arguing that the normative foundations of the methodology are not in utility theory, but in the theory of measurement. nevertheless, bernasconi et al. (2010) reexamined the descriptive and normative foundations of the ahp in light of the modern theory of psychological measurement and gave empirical evidence to suggest that the method of ratio scaling mimics the cognitive process involved in decision-making. 4.3 synthesizing in saaty (2015) it is noticed that our brain has a way of meshing together its parts and subparts using the matrix theory that arises from decision-making that has analogous complexity.). saaty (2015) considers the brain to be a synthesizer of the firings of individual neurons into clusters of information, and these in turn into larger clusters and so on, leading to an integrated whole. thus, saaty’s anp methodology provides a mathematical way to synthesize the signals in the brain. any decision has several favorable and unfavorable aspects to consider. to make complex decisions we need to synthesize not only clusters of information but also different sides of the problem, e.g. benefits (and opportunities), costs and risks (bocr). each of these four concerns utilizes a separate structure for the decision (saaty & vargas, 2006). psychological experiments show that when making a choice under both risk and uncertainty, the subjective value of a specific loss is larger than the subjective value of an equivalent gain (kahneman & tversky, 1979). this phenomenon is called “loss aversion” and is supported by huge amounts of examples. some examples include the endowment effect, the status-quo bias, and under-investment in the stock market (morewedge & giblin, 2015; dean et.al., 2017; benartzi & thaler, 1995). on the other hand, there are studies that show that in some situations people are not loss averse, e.g. when the decision involves exchanging goods, like money, that are given up as intended (novemsky & kahneman, 2005). similarly, the ownership of multiple units attenuates the endowment effect and the implied loss aversion (rottenstreich et al., 2013). in light of this discussion, we suggest that applying saaty’s bocr-concept of structuring a problem serves to make the decision process transparent and understandable. as the last level of synthesizing, the ahp/anp can be applied in the process of group decision making and in the negotiation process (saaty & vargas, 2012; moreno-jiménez & ijahp article: milkova, andreichikova, andreichikov/at the junction of mathematics and psychology: cognitive orientation of the ahp/anp and new perspectives of structuring complexity international journal of the analytic hierarchy process 119 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.611 vargas, 2018; vargas, 2017). even though group decision making is critical at all societal levels, the main question today is the opportunity of applying the ahp/anp methodology in the experience age. today, people generate and store more data than ever before as they interact with both real and virtual environments. mobile devices, internet everywhere, micro-computers, mobile sensors and high-speed connectivity are all native to the new reality. thus, information is moving toward experience, driven by the changing context of our online interactions. digital traces of behavior and cognition offer an unprecedented opportunity to test theories outside the laboratory. one of the most promising areas of future growth in the experience age may be achieved by analyzing naturally occurring real-world data sets that affect and reveal human behavior (goldstone & lupyan, 2016; paxton & griffiths, 2017). the question in this direction is the opportunity of converting digital traces of behavior to the ratio scale to measure social preferences. another direction of ahp/anp applicability is the opportunity of building internet ratings of goods/services based on comparative assessment. according to neuroeconomic results that show people compare choices within a set rather than assigning separate utilities, we could use the system of online pairwise comparisons to collect and store individuals’ judgments and then build appropriate ratings of choices (camerer et al., 2004). 5. discussion: ahp/anp in the experience age while the ahp/anp methodology is intended to reflect an individual's decision process, it is not complete without accounting for social dynamics in the decision. this idea is supported by a wealth of evidence showing that people routinely base decisions on the choices of other people (abrahamson & rosenkopf, 1997; delre et al., 2007). moreover, living in the experience age, an increasing amount of information about us is exploited to manipulate our choices. in helbing (2019), gigerenzer described an example about parliamentary elections in india in 2014. in a study, undecided voters could find out more information about the candidates using an internet search engine. however, web pages had been manipulated. for one group, more positive items about candidate 1 popped up on the first page and negative ones were shown later on. the other groups experienced the same for the other candidates. this and similar manipulative procedures are common practice on the internet. it is estimated that for candidates who appear on the first page of a search (thanks to such manipulation), the number of votes they receive from undecided voters increases by 20%. helbing (2019) uses the term big nudging to describe when the increasing amount of personal information about us, which is often collected without our consent, reveals what we think, how we feel and how we can be manipulated. this insider information is exploited to manipulate us to make choices that we would otherwise not make, for example, buying some overpriced products or those that we do not need, or perhaps giving our vote to a certain political party. ijahp article: milkova, andreichikova, andreichikov/at the junction of mathematics and psychology: cognitive orientation of the ahp/anp and new perspectives of structuring complexity international journal of the analytic hierarchy process 120 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.611 thus, living in the experience age, we need a new dynamic system of assessing the changing reality based on objective information, perception and subjective judgments. for the functioning of society, it is essential that people fill different roles, which are fitting to the respective situations they are in. from our point of view, as ahp/anp has the instruments for measuring preferences and synthesizing different aspects of cognition, it could have a big future in the experience age. for example, since natural language is used to indicate a preference between different criteria, we can test the opportunity of using natural language processing for collecting human preferences from the web and translating them into numerical values for further integration into the ahp/anp. although the ahp/anp is aimed at measuring individual preferences, in the era of huge amounts of data we should think about the opportunity of aggregation of human judgments for identifying global trends, changes and risks. 6. conclusions considering the ahp/anp in light of the modern theory of psychological measurement, it is suggested that the methods mimic the cognitive process involved in decision-making. as ahp/anp incorporates both quantitative and qualitative criteria, it is difficult to assess the final decision. in some areas, slow feedback on the quality of the decision makes it impossible to adjust the ahp/anp approach for experts in their decision processes. from our point of view, increasing the amount of controlled laboratory tests on a basic everyday decision would lead to growth in the popularity of the ahp/anp in more important problems. we think that the experimental validation with subjective results is more convincing than the techniques with verifiable objective results because they deal with problems where the ahp/anp is more likely to be applied (ishizaka et al., 2011). thus, for further development of the ahp/anp we should follow the new way of cognitive orientation of the methodology and use laboratory experiments to understand human behavior. a huge amount of studies concerning the role of heuristics in decision-making have been done. the “heuristics and biases” program, introduced by kahneman and tversky, showed that simple heuristics are efficient because they piggyback on basic computations that the mind has evolved to make. heuristics of the “fast and frugal heuristics” program, developed by gigerenzer and colleagues, ignore information to make decisions faster, more frugally, and/or more accurately than more complex methods. however, heuristics as an efficient cognitive process may be used not only in fast decisions, but also in deliberate strategies. the ahp/anp offers a judgmental structure for comparisons in a consistent manner without sufficiently reducing the complexity of the problem. whether applying heuristics may help answer questions like “which of the two elements has more influence”, pairwise comparisons give a more accurate answer on “how strong is this influence”. moreover, when people make decisions across the boundaries of different areas of information they need a way to synthesize priorities in addition to applying heuristics. living in the experience age, decision making is inseparable from social dynamics. this is especially relevant in an age where the internet and mobile devices have transformed how people look for and communicate about different things. people generate and store ijahp article: milkova, andreichikova, andreichikov/at the junction of mathematics and psychology: cognitive orientation of the ahp/anp and new perspectives of structuring complexity international journal of the analytic hierarchy process 121 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.611 huge amounts of data as they interact with both real and virtual environments. thus, one of the most promising areas of future development of the applicability of the ahp/anp may be achieved by analyzing naturally occurring real-world data sets that affect and reveal human behavior. ijahp article: milkova, andreichikova, andreichikov/at the junction of mathematics and psychology: cognitive orientation of the ahp/anp and new perspectives of structuring complexity international journal of the analytic hierarchy process 122 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.611 references abrahamson, e., & rosenkopf, l. (1997). social network effects on the extent of innovation diffusion: a computer simulation. organization science, 8(3), 289-309. doi: https://doi.org/10.1287/orsc.8.3.289 benartzi, s., & thaler, r. (1995). myopic loss aversion and the equity premium puzzle. the quarterly journal of economics, 110(1), 73-92. doi: 10.2307/2118511 bernasconi, m., choirat, c., & seri, r. (2010). the analytic hierarchy process and the theory of measurement. management science, 56(4), 699-711. doi: https://doi.org/10.1287/mnsc.1090.1123 brugha, c. (2000). relative measurement and the power function. european journal of operational research, 121, 627–640. doi: https://doi.org/10.1016/s03772217(99)00057-0 brugha, c. (2004). phased multicriteria preference finding. european journal of operational research, 158, 308–316. doi: https://doi.org/10.1016/j.ejor.2003.06.006 brighton, h. (2006). robust inference with simple cognitive models. in c. lebiere, & b. wray (eds.), between a rock and a hard place: cognitive science principles meet aihard problems: papers from the aaai spring symposium (17-22). menlo park, california: aaai press. camerer c. f., loewenstein, g., & prelec, d. (2004). neuroeconomics: why economics needs brains. scandinavian journal of economics, 106 (3), 555-579. doi: 10.1111/j.0347-0520.2004.00377.x campos-vazquez, & r.m., cuilty, e. (2013). the role of emotions on risk aversion: a prospect theory experiment. journal of behavioral and experimental economics, 50, 19. doi: https://doi.org/10.1016/j.socec.2014.01.001 czerlinski, j., gigerenzer, g., & goldstein, d.g. (1999). how good are simple heuristics? in g. gigerenzer, p. m. todd, & the abc research group, simple heuristics that make us smart (97–118). new york: oxford university press. doi: https://doi.org/10.5751/es-00277-050204 dean, m., kıbrıs, o., & masatlioglu, y. (2017). limited attention and status quo bias. journal of economic theory, 169, 93-127. doi: https://doi.org/10.1016/j.jet.2017.01.009 delre, s. a., jager, w., & janssen, m.a. (2007). diffusion dynamics in small-world networks with heterogeneous consumers. computational and mathematical organization theory, 13(2), 185-202. doi: https://doi.org/10.1007/s10588-006-9007-2 dyer, j. (1990). remarks on the analytic hierarchy process. management science, 36(3), 249-258. https://doi.org/10.1016/j.jet.2017.01.009 ijahp article: milkova, andreichikova, andreichikov/at the junction of mathematics and psychology: cognitive orientation of the ahp/anp and new perspectives of structuring complexity international journal of the analytic hierarchy process 123 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.611 forman, e.h., & gass, s. i. (2001). the analytic hierarchy process: an exposition. operations research, 49(4), 469-486. doi: https://doi.org/10.1287/opre.49.4.469.11231 gigerenzer, g., & goldstein, d. g. (1996). reasoning the fast and frugal way: models of bounded rationality. psychological review, 103 (4), 650-669. doi: https://doi.org/10.1037//0033-295x.103.4.650 gigerenzer, g., & gaissmaier, w. (2011). heuristic decision making. annual review of psychology, 62, 451-482. doi: https://doi.org/10.1146/annurev-psych-120709-145346 goldstone, r.l., & lupyan, g. (2016). discovering psychological principles by mining naturally occurring data sets. topics in cognitive science, 8, 548–568. doi: 10.1111/tops.12212 helbing, d. (2019). towards digital enlightenment: essays on the dark and light sides of the digital revolution. springer. doi: https://doi.org/10.1007/978-3-319-90869-4 huizingh e., & vrolijk h (1997). extending the applicability of the analytic hierarchy process. socio-economic planning sciences, 31(1), 29–39. doi: https://doi.org/10.1016/s0038-0121(96)00025-0 ishizaka, a., balkenborg, d., & kaplan, t. (2011). does ahp help us make a choice? an experimental evaluation. journal of the operational research society, 62, 1801–1812. doi: https://doi.org/10.1057/jors.2010.158 ishizaka, a. (2012). clusters and pivots for evaluating a large number of alternatives in ahp. pesquisa operacional, 32(1), 87-101. doi: https://doi.org/10.1590/s010174382012005000002 jandrić, p., knox, j., besley, t., ryberg, t., suoranta, j., & hayes, s. (2018). postdigital science and education. educational philosophy and theory, 50(10), 893-899. doi: 10.1080/00131857.2018.1454000 kahneman, d. & tversky, a. (1979). prospect theory: an analysis of decision under risk. econometrica, 47(2), 263-291. doi: https://doi.org/10.2307/1914185 kahneman, d. & tversky, a. (1996). on the reality of cognitive illusions. psychological review, 103(3), 582–591. doi: https://doi.org/10.1037//0033-295x.103.3.582 kahneman, d. (2003). a perspective on judgment and choice: mapping bounded rationality. american psychologist, 58(9), 697-720. doi: https://doi.org/10.1037/0003066x.58.9.697 katsikopoulos, k. v., schooler, l.j., & hertwig, r. (2010). the robust beauty of ordinary information. psychological review, 117 (4), 1259–1266. doi: https://doi.org/10.1037/a0020418 ijahp article: milkova, andreichikova, andreichikov/at the junction of mathematics and psychology: cognitive orientation of the ahp/anp and new perspectives of structuring complexity international journal of the analytic hierarchy process 124 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.611 korhonen, p. & topdagi, h. (2003). performance of the ahp in comparison of gains and losses. mathematical and computer modelling, 37, 757–766. doi: https://doi.org/10.1016/s0895-7177(03)00083-9 loewenstein, g., & lerner, j. s. (2003). the role of affect in decision making. in r. j. davidson, k. r. scherer, & h. h. goldsmith (eds.), series in affective science. handbook of affective sciences (619-642). new york, ny: oxford university press. martignon, l., & hoffrage, u. (2002). fast, frugal, and fit: simple heuristics for paired comparisons. theory and decision, 52 (1), 29–71. doi: https://doi.org/10.1093/acprof:oso/9780199744282.003.0012 miller, g.a. (1956). the magical number seven, plus or minus two: some limits on our capacity for processing information. psychological review, 63(2), 81-97. doi: https://doi.org/10.1037/h0043158 moreno-jiménez, j.m., & vargas, l.g. (2018). cognitive ahp-multifactor decision making. international symposium on the analytic hierarchy process, hong kong, hk. morewedge, c.k., & giblin, c.e. (2015). explanations of the endowment effect: an integrative review. trends in cognitive science, 19(6), 339-348. doi: https://doi.org/10.1016/j.tics.2015.04.004 mousavi, s., & gigerenzer, g. (2014). risk, uncertainty, and heuristics. journal of business research, 67, 1671-1678. doi: https://doi.org/10.1016/j.jbusres.2014.02.013 novemsky, n., & kahneman, d. (2005). the boundaries of loss aversion. journal of marketing research, 42, 119–128. doi: https://doi.org/10.1509/jmkr.42.2.119.62292 pachur, t., & biele, g. (2007). forecasting from ignorance: the use and usefulness of recognition in lay predictions of sports events. acta psychologica, 125(1), 99-116. doi: https://doi.org/10.1016/j.actpsy.2006.07.002 paxton, a., & griffiths, t.l. (2017). finding the traces of behavioral and cognitive processes in big data and naturally occurring datasets. behavior research methods, 49(5), 1630-1638. doi: 10.3758/s13428-017-0874-x pohl, r. f. (2006). empirical tests of the recognition heuristic. journal of behavioral decision making, 19(3), 251-271. doi: https://doi.org/10.1002/bdm.522 rottenstreich, y., burson, k., & faro, d. (2013). multiple unit holdings yield attenuated endowment effect. management science, 59, 545-555. doi: https://doi.org/10.1287/mnsc.1120.1562 saaty, t. l. (1990). how to make a decision: the analytic hierarchy process. european journal of operational research, 48, 9-26. doi: https://doi.org/10.1016/03772217(90)90057-i https://doi.org/10.1016/j.tics.2015.04.004 ijahp article: milkova, andreichikova, andreichikov/at the junction of mathematics and psychology: cognitive orientation of the ahp/anp and new perspectives of structuring complexity international journal of the analytic hierarchy process 125 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.611 saaty, t. l. (2008). relative measurement and its generalization in decision making: why pairwise comparisons are central in mathematics for the measurement of intangible factors. racsam, 102(2), 251-318. doi: https://doi.org/10.1007/bf03191825 saaty, t. l., & ozdemir, m. s. (2003). why the magic number seven plus or minus two. mathematical and computer modelling, 38, 233-244. doi: https://doi.org/10.1016/s08957177(03)90083-5 saaty, t. l., & vargas luis g. (2006). decision making with the analytic network process: economic, political, social and technological applications with benefits, opportunities, costs and risks. new york: springer. saaty, t.l., & vargas, l.g. (2012). the possibility of group choice: pairwise comparisons and merging functions, social choice and welfare, 38(3), 481-496. doi: https://doi.org/10.1007/s00355-011-0541-6 saaty, t. l. (2013). on the measurement of intangibles. a principal eigenvector approach to relative measurement derived from paired comparisons. notes of the ams, 60(2).doi: https://doi.org/10.1090/noti944 saaty, t.l. (2015). the neural network process (nnp): generalization of the ahp and anp to the continuous case of neural firing. pittsburgh, pa: rws publications. simon, h. a. (1956). rational choice and the structure of the environment. psychological review, 63(2), 129–138. simon, h. a. (1957). models of man. new york: wiley & sons. taffel, s. (2015). perspectives on the postdigital: beyond rhetorics of progress and novelty. convergence, 22(3), 324-338. doi: https://doi.org/10.1177/1354856514567827 the guardian (2017). https://www.theguardian.com/commentisfree/2017/feb/02/digitalrevolution-age-of-experience-books-vinyl (23.10.2018) tversky, a. (1972). elimination by aspects: a theory of choice. psychological review, 79, 281-299. doi: https://psycnet.apa.org/doi/10.1037/h0032955 tversky, a., & kahneman, d. (1981). the framing of decisions and the psychology of choice, science, 211(4481), 453-458. doi: 10.1126/science.7455683 tversky, a., & kahneman, d. (1974). judgment under uncertainty: heuristics and biases. science, new series, 185, 4157. doi: https://doi.org/10.1037/h0032955 vargas, l.g. (2017). how to write a contract with the ahp. international journal of the analytic hierarchy process, 9(2), 274-284. doi: https://doi.org/10.13033/ijahp.v9i2.490 https://doi.org/10.1177/1354856514567827 https://www.theguardian.com/commentisfree/2017/feb/02/digital-revolution-age-of-experience-books-vinyl https://www.theguardian.com/commentisfree/2017/feb/02/digital-revolution-age-of-experience-books-vinyl https://doi.org/10.13033/ijahp.v9i2.490 ijahp article: milkova, andreichikova, andreichikov/at the junction of mathematics and psychology: cognitive orientation of the ahp/anp and new perspectives of structuring complexity international journal of the analytic hierarchy process 126 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.611 weiss, e. n., 1987. using the analytic hierarchy process in a dynamic environment, mathematical modelling, 9/3-5, 211 -218. doi: http://dx.doi.org/10.1016/02700255(87)90478-7 whitaker, r. (2007). criticisms of the analytic hierarchy process: why they often make no sense. mathematical and computer modelling, 46, 948–961. doi: https://doi.org/10.1016/j.mcm.2007.03.016 wübben, m., & wangenheim, f. (2008). instant customer base analysis: managerial heuristics often “get it right”, journal of marketing, 72, 82–93. doi: https://doi.org/10.1093/acprof:oso/9780199744282.003.0036 zhang, h., chen, x., dong, y. et al. (2018) . analyzing saaty’s consistency test in pairwise comparison method: a perspective based on linguistic and numerical scale. soft computing, 22(6), 1933–1943. doi: https://doi.org/10.1007/s00500-016-2454-x https://doi.org/10.1007/s00500-016-2454-x ijahp essay: peniwati/striving for agility in vuca environment: the analytic hierarchy/network process international journal of the analytic hierarchy process 166 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.753 striving for agility in a vuca environment: the analytic hierarchy/network process kirti peniwati ppm graduate school of management jakarta, indonesia kirti@indo.net.id introduction the business world is currently operating in an increasingly challenging environment characterized as volatile, uncertain, complex, and ambiguous (vuca). if anything, the current covid-19 crisis is a demonstration of this. what began as a health crisis has become a social and business crisis of large proportions. the current world is dynamic, which is fundamentally different from the past which was much more stable. the success that resulted in the past from simply offering excellent products and services does not sustain anymore. agile transformation appears to be the right response that is needed from business organizations for this new environment. many argue that the claims that agile transformation does not work as expected are a result of an inadequate understanding of the fact that agile is about mindset change. the agile movement was born out of the belief that amazing experiences can happen if people come together and seek common ground. it is a breakthrough approach that needs to be understood in order to implement it because it is a total mindset change. it is not business as usual, and therefore the typical practice of continuous improvement will not work anymore. academics have been supporting business since the industrial revolution began by contributing relevant knowledge and methods. this support gets more sophisticated as businesses need to respond to the increasingly complex environment. this increased sophistication includes breaking down boundaries between disciplines, and facilitating implementation by designing the right cross-discipline approach for any given situation. presentations at the 2018 drucker forum indicated that a major transformation of management is already under way. models of “the organization of the future” have emerged in a number of large post-bureaucratic organizations. these organizations are all thriving in a vuca marketplace in their unique ways, but also think that they still have much to learn and improve upon. they are in the continuous learning mode. the purpose of this article is to create awareness that the ahp/anp is capable of supporting an organization’s continuous learning process, but it needs to be complemented by appreciative inquiry (ai) that is applied by a competent facilitator. ai is the groundbreaking approach for organizational development that uses narrative and story-telling to capture different knowledge, perceptions, and aspirations from members of the organization. the potential for amazing results strongly depends on how well the facilitator internalizes agile mindsets, his facilitating competence to synergize ai and ahp/anp applications, and his ability to implement it in group and organization settings. mailto:kirti@indo.net.id ijahp essay: peniwati/striving for agility in vuca environment: the analytic hierarchy/network process international journal of the analytic hierarchy process 167 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.753 academic supports follow business journey early in the era of the industrial revolution, academics played a crucial role in designing techniques and systems to help businesses be more effective and efficient in their operations. at that time, workers were considered as simply one of the business’ resources to exploit. total quality control (tqc) was initiated in japan when workers were involved in using statistical process control to solve quality and productivity problems in operations. they realized that long term process improvement is more important than short term financial gains. as participation spread to the whole organization, and the concept of quality and productivity emerged as something that could be improved, the total quality management (tqm) movement began to flourish. the initiative of applying the new approaches of unending continuous improvements and participative management led to an awareness that well-defined organizational boundaries were crumbling with the need to align different initiatives. academics supported this development as they applied their concepts and methods to participative management in general and group decision-making in particular. problem solving and creative thinking also emerged as needed skills. the tqm movement challenged academics to cross boundaries and integrate their disciplines by designing effective supports. tqm focuses on continuous process improvements by looking inward, assuming that the higher the product quality, the higher the customer satisfaction. there were times when product quality was considered as a competitive advantage. this was when tqm had not emphasized enough the total shift that was needed from the deep rooted ‘product out’ thinking to the new ‘market in’ mindset. this mindset requires producers to really listen and respond to what their consumers want rather than blindly follow internal policies, standards and procedures. it is about commitment, not compliance. striving for agility in a vuca environment requires a total mindset shift, of which the ‘market in’ becomes the most critical. the “market in” concept moves the attention from internal, which assumes that the organization knows what their customers want, to external, which looks out and directly asks the customers what would delight them. it is a mindset that requires agility, not blindly copying approaches that work in other organizations. agile is a mindset of people that continuously focuses on delivering value to customers as the primary goal of their work. appreciative inquiry (ai) is a breakthrough approach for organizational development. it involves a guided conversation about what matters using stories. according to stephen denning, stories help people make sense of organizations, generating fresh depth and breadth of perception as meaning emerges internally from narratives they hear from external sources. when we believe in stories, they resonate and generate creativity, interaction and transformation. an open-ended question would naturally prompt story telling that could lead to learning and inspired actions. therefore, formulating the right question could lead to desirable changes. ijahp essay: peniwati/striving for agility in vuca environment: the analytic hierarchy/network process international journal of the analytic hierarchy process 168 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.753 agile management and leadership stephen denning is the world’s greatest promoter of the importance of story-telling in management and leadership. he maintains that striving for agile means building organizations that are as innovative as they are efficient, and as passion-filled as they are pragmatic. it is a journey of continuously growing agile mindsets in organizations; it is not about implementing a new practice, process, or structure. denning defines radical management with his seven principles of continuous innovations as seen below: 1. delighting clients 2. self-organizing teams 3. client-driven iterations 4. delivering values to clients in each iteration 5. radical transparency 6. continuous self-improvement 7. interactive communication these changes require commitment and leadership from management as the distinction between leadership and management dissolves. it is a feedback process where managers are committed to change and leaders also manage. how the ahp supports continuous learning in organizations agile decision making is a very complex process of setting priorities among diverse important dimensions. it requires eliciting and integrating different judgments of relative importance from the understanding, perceptions and preferences of many people to come up with a more-or-less acceptable collective sense of priorities. below, we discuss how the ahp/anp can support the adoption of denning’s four vital management principles for the agile journey.  transparency an ahp model is a hierarchy or network representation of a thinking process. it shows what key elements are being considered, how they influence each other and the relative strengths of their influence. the use of the ahp fundamental scale, deriving relative priorities in 0 to 1 ratio scales, and synthesizing the priorities across a structured model, enables the ahp/anp to model a system of any complexity that is being analyzed without theoretical constraints. the hierarchy/network structure could be single and simple, single and complex, or a set of interconnected models. when the modeling is done well, the purpose of the thinking is clear and well defined. an ahp/anp model is an effective means for sharing complex information in a concise way. tangible models help people understand, and perceive the situation even if they are not involved in the process.  competence ahp/anp techniques help a facilitator to integrate the knowledge and aspirations of individuals into a coherent group outlook. as a means for group collaboration, it develops group alignment and competencies which play a significant role in group and organizational learning. the ahp/anp is equipped with a variety of techniques that enable its facilitator to allow ijahp essay: peniwati/striving for agility in vuca environment: the analytic hierarchy/network process international journal of the analytic hierarchy process 169 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.753 different input from actors or group participants that have different roles and authority. actors or institutions that pursue their own objectives that would affect the organization’s achievement could be considered as elements in a level of a model. judgments of relative importance can be assigned to the decision makers, whereas judgments on likelihood or preference are left to the experts. relative priority can also be assigned to the different people providing judgments before aggregating in order to obtain a final result of synthesized priorities.  localization an ahp facilitator can design a series of ahp/anp processes that ensure alignment in the case where a big organizational problem needs to be decomposed into smaller ones to be dealt with by any number of small groups.  upside when the modeling process is done well, it opens up the possibility for personal growth by the individual facilitating the group process, participating in an ahp/anp modeling process or interpreting its outcome. two philosophies representing the ahp/anp axioms ahp/anp users need to make sure that its axioms are satisfied so that a valid model is created that produces the expected output. internalization of the two important philosophies underlying the ahp helps apply the ahp/anp mathematical axioms in practice.  it is better to be approximately right than precisely wrong. it is essential that the important elements in an ahp/anp model be elicited from the group involved in the process. this is where the ai approach could play a role. redundancy in eliciting judgments of relative importance, inconsistency measures, and sensitivity analyses are the means for ensuring acceptance of the relative priority obtained from the anp/ahp process. a facilitator has room to judge the extent of their use to meet the expectations of the participants as a group and keep them satisfied with the process. the facilitator is responsible for striking the balance between striving for accuracy and keeping excitement in the group high.  objectivity is agreed upon subjectivity. a sense of priority is inherently subjective, therefore absolute objectivity cannot be accomplished in group decision making. a certain level of objectivity can only be accomplished through discussion that aims to align perceptions while respecting each individual’s sense of priorities. the facilitator needs to be able to ensure that only homogeneous judgments are being aggregated. this result can be obtained either through group discussion or by revising the ahp/anp structure. the ahp’s technique of measuring the distance of two ratio scales is useful to assess the level of compatibility between the judgments of one particular member to those of group aggregation. high incompatibility indicates new knowledge or a different way of thinking, and then further exploration would be useful. different perceptions and judgments of individuals in a group are assets, not problems. ijahp essay: peniwati/striving for agility in vuca environment: the analytic hierarchy/network process international journal of the analytic hierarchy process 170 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.753 conclusion the vuca environment requires agile responses that are obtained by continuously learning organizations that practice management and leadership with a new mindset. the ahp/anp is the means to facilitate the changes in such organizations, strengthened by the ai approach to elicit the important and relevant factors or ideas for the organization from members of the organization, experts, and its stakeholders. the ahp/anp facilitates systematically structuring, eliciting judgments of relative dominance, synthesizing, and analyzing the result in a rational way. its iterative and dynamic capabilities enable organizations to review and improve a developed model to their satisfaction over time. supporting organizations to continuously learn to run as organic entities with networks of teams is a challenging, yet exciting, learning and growing opportunity for ahp/anp practitioners. the ahp/anp is holistically applicable to organizations, in their collaboration with customers and other stakeholders, in breaking down internal and external boundaries, and in shifting the understanding of management and leadership. it has been shown during the decades of its development and its many applications that the ahp/anp is capable of supporting the process of becoming agile. ai, as a breakthrough approach in organizational development, is indispensable for supporting ahp/anp facilitators. ai is helpful for inquiring about and eliciting organizational values and valuable assets, and ahp is helpful for prioritizing those values and valuables. together, they can be a powerful support system for the business world to continually innovate in its agile transformation. enhanced decision-making and learning are two of the 15 un global challenges for humanity, therefore the integration of the two groundbreaking approaches of ahp/anp and ai can help with this challenge. this integration requires solid collaboration between academics and practitioners. ijahp essay: peniwati/striving for agility in vuca environment: the analytic hierarchy/network process international journal of the analytic hierarchy process 171 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.753 references cooperrider d., whitney, d. and stavros, j.m. (2008). appreciative inquiry handbook, premium 2 nd edition. brunswick, oh: crown custom publishing, inc. denning, s. (2010). leader’s guide to radical management: reinventing the workplace for the 21 st century. san fransisco, ca: jossey bass. denning, s. (2018). leader’s guide to storytelling: mastering the art and discipline of business narrative, revised and updated. san fransisco, ca: jossey bass. denning, s. (2018). the age of agile: how smart companies are transforming the way work gets done. amacom. denning, s. (2018). drucker forum 2018: a major transformation of management is already under way. forbes.com/sites/stevedenning/. doerr, john (2019). measure what matters: how google, bono, and the gates foundation rock the world with okrs. new york: portfolio/penguin. dos santos p.h. et al. (2019). the analytic hierarchy process supporting decision making for sustainable development: an overview of applications. journal of cleaner production, 212(1). dweck, c. (2017). mindset: changing the way you think to fulfil your potential, revised edition. london: robinson. reed, j. and stoltz, p.g. (2011). put your mindset to work: the one asset you need to win and keep the job you love. great britain and usa: portfolio penguin. doi: https://doi.org/10.1177/1066480712449800 saaty, thomas l. and peniwati, k. (2008). group decision making: drawing out and reconciling differences. pittsburgh, pa: rws publication. timms, h. and heimans, j. (2018). new power: how it’s changing the 21 st century – and why you need to know. new york: doubleday. ijahp essay: divjak/reflections on the importance of academic conferences international journal of the analytic hierarchy process 435 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.732 reflections on the importance of academic conferences note from the editor – the following opening remarks were given by dr. blazenka divjak, founder of the central european conference of information and intelligent systems (ceciis), university of zagreb’s faculty of informatics professor, and current minister of science and education of croatia. they constitute an interesting reflection on the value of attending academic conferences. dr. blazenka divjak addressing the 30 th ceciis meeting in varazdin, croatia ijahp essay: divjak/reflections on the importance of academic conferences international journal of the analytic hierarchy process 436 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.732 dear ladies and gentlemen, dear colleagues, dear friends, i greet you all at the beginning of 30 th conference, and wish you successful work and socializing here. since it is 30 years in a row that this conference has existed and it is many years for a scientific conference, i will try to answer why it is important to have a conference in such a digitalized world. as always, it is a good idea to look at what people wiser than me have said about it. the great american author f. scott fitzgerald once said, “no grand idea was ever born in a conference, but a lot of foolish ideas have died there.” this is half cynical, but also half true; you can choose which one is which. there is so much new knowledge, research and theories around and the quantity is rising exponentially. therefore, i believe it is important to have the opportunity to discuss ideas, consider new knowledge and hopefully extract some wisdom from it. as the great isaac asimov noted, “the saddest aspect of life right now is that science gathers knowledge faster than society gathers wisdom.” finally, the economist mark skousen has a very simple idea, “since the 1970s, i've been a big fan of attending conferences as a great way to learn, network, socialize and enjoy a new environment. it's always refreshing to get out and see a whole new world.” i would say that many of you that have come to this conference year after year can probably confirm the strength of our networking and friendship. in that context, i recognize the friends of foi (prof. wolf rauch and prof. guenter mueller) that are keynote speakers today, but also a newly established friend and the third keynote speaker sirje virkus. i would also like to welcome prof. enrique mu, a longstanding friend of foi and current program committee member, who has come from the united states to participate in this event. so to conclude, the role of conferences today may be to gather wisdom and put aside foolish ideas, but most of all it is to socialize and enjoy each other. and this is a recipe for a successful conference as well. with that, i would like to thank the organizing and program committee for having successfully prepared this year’s conference, and i wish all of you a fruitful conference. thank you, blazenka divjak ijahp article: periaiah, islam, abdullah/environmental impact assessment for malyasian bauxte mining industry international journal of the analytic hierarchy process 2 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.851 environmental impact assessment for malyasian bauxite mining industry nagendran periaiah graduate school of management international islamic university malaysia jalan gombak, 53100 kuala laumpur malaysia nagen@envichem.com.my rafikul islam department of business administration international islamic university malaysia jalan gombak, 53100 kuala lumpur malaysia rislam@iium.edu.my muhammad faris abdullah department of urban and regional planning international islamic university malaysia jalan gombak, 53100 kuala lumpur mfaris@iium.edu.my abstract the mining industry plays an important role in the economic development of malaysia. however, uncontrolled mining activities have caused serious environmental impacts. recently, bauxite mining in kuantan, in the state of pahang, stained fifteen kilometers of pahang’s coastline with red arsenic particles and heavy metal pollution washed from open-pit bauxite mines into the nearby sea. this has caused potentially catastrophic damages to the ecosystem off the coast of pahang. this triggered the government of malaysia to issue a temporary ban on bauxite mining while the state government engaged in expensive clean-up. mining activities require an environmental impact assessment (eia). environmental decisions are complex and multidisciplinary including knowledge bases which incorporate natural, physical, and social sciences, politics, and ethics. this research proposes a decision support framework that uses the analytic network process (anp) to help decision makers in eia pertaining to the bauxite mining industry. keywords: bauxite mining; environmental pollution; environmental impact assessment; decision making; analytic network process 1. introduction the mining industry plays a significant role in the development of a country. it assures an adequate and continuous supply of raw materials for the construction and manufacturing sectors. over 33 different types of minerals comprising metallic, nonmetallic, and energy minerals are available in malaysia (mcom, 2017). the metallic mineral mining subsector commonly produces minerals such as tin, gold, bauxite, and mailto:nagen@envichem.com.my mailto:rislam@iium.edu.my mailto:mfaris@iium.edu.my ijahp article: periaiah, islam, abdullah/environmental impact assessment for malyasian bauxte mining industry international journal of the analytic hierarchy process 3 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.851 iron ore. additionally, by-products such as zircon, monazite, rutile, struvite, and silver are also produced from tin and gold mining. the non-metallic or commonly known as industrial mineral subsector produces limestone, clay, kaolin, silica, feldspar, and mica. in 2017, malaysia’s economy accelerated with 5.9% growth in the gross domestic product (gdp), at a current value of $329.85 billion. mining and quarrying alone contributed $30.05 billion, which made up 9.11% of the country’s gdp (mcom, 2017). overall, the total production value of major minerals in malaysia increased by 55%, from $1 billion in 2010 to $1.5 billion in 2015 (mcom, 2017). the increasing demand for minerals, especially from china and india, has opened up the path for companies to explore various types of deposits from iron ore to gold. in the 1980s, the collapse of the tin market caused a decline in the malaysian mining industry. however, that incident did not dissuade experts from believing that there were still untapped deposits of minerals worth roughly $81.9 billion that could transform landowners into billionaires (tmr, 2017). since 2008, chinese companies have reportedly invested nearly half a billion dollars in the extraction of iron ore from malaysia (tmr, 2017). according to the minerals and geoscience department, in 2016, a total of 34 iron ore mines, 32 tin mines, and eight gold mines operated in peninsular malaysia (jmg, 2018). before 1980, malaysia’s metallic mining industries were mainly dominated by tin, iron, and gold mines. much research including that of sarupria et al. (2019), lodhia (2018), bond and morrison (2018), louw (2018), carvalho (2017), skuta et al. (2017), lee et al. (2017), jain et al. (2016), garcia et al. (2016), faradiella et al. (2016), venkateswarlu et al. (2016), and jamal et al. (2015) revealed mineral mining as one of the major causes of heavy metal contamination in the environment. residue containing heavy metals from tin mines and metallurgical operation sites is often further dispersed into the environment by wind and/or water. lately, the demand for bauxite mining has increased due to intensifying progress in the industrial sector. bauxite is usually regarded as the best material for making aluminum. bauxite is also widely used in the production of paper, water purification, petroleum refining, the electric power industry, the aircraft industry, machinery, and the civil tool-making industry. due to its broad applications, bauxite mining activities have been escalating. in 2015, malaysia was the world’s top producer of bauxite (cmo, 2016). the advancement of malaysia from a minor bauxite producer to the world’s top producer has resulted in consequences in the form of increasing detrimental environmental impacts. figure 1 shows the impact of bauxite mining on surrounding areas in pahang state, malaysia. ijahp article: periaiah, islam, abdullah/environmental impact assessment for malyasian bauxte mining industry international journal of the analytic hierarchy process 4 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.851 figure 1 impact of bauxite mining on the surrounding areas in pahang (source: cmo, 2016) clean malaysia organization (cmo), an independent online news site reported that the introduction of bauxite mining has transformed the port town of kuantan from quiet byways to heavy traffic of ore-hauling trucks (cmo, 2016). the surrounding environment, vehicles, homes, and trees have accumulated a thick layer of red dust due to emissions released from the movements of bauxite loading and unloading trucks. locals have also complained that bauxite emissions have caused skin irritation (abdullah et al., 2016). environmental experts have warned that ingestion of bauxite emissions increase the risk of developing cancer. many fruit orchards and small-scale oil palm planters abandoned their agriculture businesses and leased their land to mining contractors for short-term cash benefits. due to uncontrolled licensing and the presence of illegal mining contractors, the areas surrounding kuantan port have become heavily contaminated, turning it into a red colored zone. during rainy seasons, the surface washout from these contaminated areas flows into the nearby river and turns the water red (malaysiakini, 2019). environmental experts have also warned about the occurrence of arsenic and heavy metal in water bodies washed down from the open-pit bauxite mines (usgs, 2016). mining activities damage the environment and ecosystem. many environmental experts hope for an indefinite extension of the ban imposed by the government (daim, 2019). the state government has carried out vigorous clean-up efforts. meanwhile, experts from various government agencies and consultants are working together to develop more environmentally friendly bauxite mining standard operating procedures (povera, 2019). ijahp article: periaiah, islam, abdullah/environmental impact assessment for malyasian bauxte mining industry international journal of the analytic hierarchy process 5 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.851 environmental and socio-economic protection from mining operations depends entirely on eia and its enforcement (solbar & keskitalo, 2017). however, an eia is an intrinsically complex multi-dimensional procedure. due to its complexity, eia implementation is often not entirely satisfactory (bond & morrison, 2018). an eia often deals with attributes that are difficult to define and components that may involve both quantitative and qualitative factors (hamida et al., 2021; kaya & kahraman, 2011). during eia projects, decision makers often receive four general types of technical inputs including the results of modeling and monitoring studies, risk assessment, cost or cost-benefit analysis, and stakeholder preferences (mahmud, 2016). while choosing to present modeling and monitoring results as quantitative estimates, the project team can incorporate a higher degree of qualitative judgments for risk assessment and cost-benefit analysis (bond & morrison, 2018). structured information about stakeholder preferences may not be presented to the decision maker and conducted in an ad hoc or subjective manner that intensifies the difficulty of defending the decision process as reliable and unbiased (asadabadi et al., 2019). moreover, the application of structured approaches often relates to the lack of flexibility in adapting to local concerns or an inaccurate representation of marginal views. the development of a systematic methodology to rank projects based on combined qualitative and quantitative inputs from scientific and engineering studies of risks, costs, and benefits together with the views and values of stakeholders has not yet been achieved (aminu et al., 2017). this makes it difficult for decision makers to identify all plausible alternatives and make full use of all available and necessary information when choosing between available project alternatives (beltran et al., 2017). multi-criteria decision analysis (mcda) methods have been found to be useful in dealing with complex and ill-defined environmental decision-making problems (toth & vacik, 2018; saffari et al., 2017; harun & samat, 2016; dell'anna et al., 2020). further, there is a need to develop systematic mcda tools to support the decisionmaking process pertaining to eia, especially in the context of malaysia. beltran et al. (2017) stressed that innovative methods might be required to achieve progress in the eia process. the analytic network process (anp) has been found to be a useful mcda tool for systematically analyzing the views of several groups of experts belonging to diverse fields in an eia study (kadoic et al., 2019). beltran et al. (2017) also proposed the use of the anp to address the needs of multiple criteria and multiple stakeholders in eia. in order to simultaneously consider subjective issues and inter-influence among the criteria, this study develops a decision support framework that uses the anp to help decision makers with eia pertaining to the bauxite mining industry in malaysia. 2. literature review developing countries have yielded billions of tons of bauxite and aluminum, but malpractice by the industry has resulted in environmental damage and social unrest. thorpe and watve (2015) stressed that bauxite mining and processing had varied and robust impacts on the environment because they not only modify the landscape, but also generate severe pollution by discharging wastes into the biosphere (soil, atmosphere, and water). bauxite mining is a major open cast mining activity that has a significant negative impact on the local environment (skuta et al., 2017). the major threats from this activity are dust pollution, vegetation loss, forest fragmentation and ijahp article: periaiah, islam, abdullah/environmental impact assessment for malyasian bauxte mining industry international journal of the analytic hierarchy process 6 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.851 biodiversity loss, negative impact on water resources, generation of wastelands, and social impact (rohan & samant, 2012). leena et al. (2017) reported the socioeconomic and environmental impacts of mining in northern europe which include land degradation, damage to water quality, pollution, harm to livestock and wildlife biodiversity. huang et al. (2011) pointed out that the application of conventional eia tools is becoming increasingly difficult for three reasons. first, there are many emerging risks like climate change and nanotechnology for which information is not available, and decision-making often deals with significant uncertainty. second, for many traditional stressors and situations, multiple lines of evidence regarding the same measure (e.g., risk) are available, but they may point to different management alternatives. finally, stakeholders, who may have a vested interest in specific courses of action are gaining increased access to all the available information and given the data uncertainty, can often justify opposing courses of actions. bauxite mining is not new in malaysia. the malaysian minerals and geoscience department (jmg) reported that bauxite mining has taken place in the state of johor since early 2000 (jmg, 2018). bauxite mining operations in teluk ramunia, johor have been operating for more than 17 years without any controversy (abdullah et al., 2016). the gross output of bauxite from johor was valued at $20.02 million in 2015 with seven establishments. in contrast, pahang’s gross output of bauxite was valued at $613.17 million in 2015 with 88 establishments (dosm, 2017). these statistics show that the magnitude of bauxite mining in pahang is almost 30 times more than in johor. the number of establishments conducting bauxite mining in johor is also less when compared to pahang. therefore, bauxite mining in johor is under control and does not pose serious environmental impacts to the surrounding areas. meanwhile, the recent bauxite mining activities in kuantan have created different circumstances within a short period of time. bauxite mining activities in kuantan offer some exciting economic opportunities for various parties including individual land-owners. however, extensive and aggressive mining activities including transporting and stockpiling of bauxite in huge quantities have taken place. this has ultimately caused environmental problems leading to community outrage (abdullah et al., 2016). malaysia incorporates the environmental impact assessment into a mandatory requirement within the planning-permission process (mahmud, 2016). the eia procedure is built into the integrated project-planning concept and conducted in tandem with a pre-feasibility study. in malaysia, an eia is obligatory under section 34a, environmental quality act 1974. section 34a empowers the minister of natural resources and the environment to prescribe any activity which may have significant environmental impacts as a prescribed activity. the legislation empowers the director general of the department of environment (doe) to: “…protect and enhance the quality of the environment through licensing, setting of standards, coordination of research, and dissemination of information to the public” (mahmud, 2016). section 34a (amendment 2012) further requires the project proponent of a prescribed activity to appoint a qualified person to submit an eia report to the director general of doe for approval. the eia report must be in accordance with the guidelines issued by the doe and contain an assessment of the prescribed ijahp article: periaiah, islam, abdullah/environmental impact assessment for malyasian bauxte mining industry international journal of the analytic hierarchy process 7 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.851 activities’ impact on the environment, and details on the proposed measures to be executed (doe, 2016). activities subject to an eia are prescribed under the environmental quality (prescribed activities) (environmental impact assessment) order, 2015. in terms of implementation, two types of eia reports that are comprised of a first schedule and second schedule are implemented. the activities specified in the first schedule do not require a public display and public comment, while activities under the second schedule require a public display and public comments. a total of 21 categories of projects are listed as prescribed activities under the first schedule and a total of 17 categories of projects are listed as prescribed activities under the second schedule. generally, the requirement for the second schedule is more stringent and the size or significance of the project is larger compared to the first schedule. the analytic hierarchy process (ahp) has been broadly utilized in the area of eia. the ahp method widely functions to support the complex system related to decisionmaking from several alternatives, or for weight estimation in many fields (saaty, 2005). in fact, the ahp helps establish the logical analysis of the problem by dividing it into its component parts. the analysis then aids the decision makers who, by means of several pairwise comparisons, can appreciate the influence of the considered elements in a hierarchical structure (saaty, 2005). however, for many problems a hierarchy cannot be formed because of dependencies (inner/outer) and influences between and within clusters (criteria, alternatives). one of the most advanced and complex multi-criteria decision-making methods is the analytic network process (anp) (kadoic et al., 2019). the anp is a generalization of the analytic hierarchy process (saaty, 2005). the basic structure in the anp is an influence network of clusters. this method supports modeling dependencies and feedback between elements in the network. for this reason, the anp is one of the most appropriate methods for making decisions in the fields characterized by dependencies among decision making elements such as in the area of environmental science (kadoic et al., 2019). the availability of literature concerning the application of mcda in eia specific to bauxite mining operations is still minimal. to date, mcda tools have been applied in the areas of environmental management (kheybari et al., 2020), eia for mining (ataei et al., 2016), sustainable tourism (hadiwijaya et al., 2018; aminu et al., 2017), waste management (abba et al., 2013; samah et al., 2010), project management (beltran et al., 2017), a wind power project (tian et al., 2013), urban industrial planning (kaya & kahraman, 2011), and construction (liu & lai, 2009). an eia is a possible conflict resolution tool available for use in environmentally sensitive projects such as bauxite mining. the literature has emphasized the need for an eia to manage various environmental impacts such as dust pollution, water pollution, ecology, land use impact, socio-economic impact, and health impact which have arisen due to mining activities. although malaysia has a comprehensive eia procedure (doe, 2016) specific for mining industries, there are some weaknesses in the guidelines provided in the eia handbook, which have resulted in criticism because of its vulnerability to abuse through the submission of multiple mini-projects (mahmud, 2016). ijahp article: periaiah, islam, abdullah/environmental impact assessment for malyasian bauxte mining industry international journal of the analytic hierarchy process 8 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.851 for those mining projects that were prescribed and subjected to an eia, the decisionmaking process faced the risk of high subjectivity. this subjectivity arises from the determination of relevant criteria, evaluation of the criteria, and the incorporation of the decision makers’ opinions/judgments (sarupria et al., 2019). the literature reveals that this problem appears to be under-researched. although mahmud (2016) observed that the government and the public understood the eia process in malaysia, there were also noticeable deficiencies. moreover, only a limited number of studies have been conducted in malaysia pertaining to eia and mining industries. motivated by these observations, this research aims to identify the criteria relevant to mining industries, as well as design an eia decision-making model using the anp, with special reference to the bauxite mining projects in kuantan, malaysia. 3. research methodology in the present research, the anp framework consists of three clusters (environmental, economy and ecological) that serve as a foundation for the development of the decision-making framework. the anp demonstrated a compound of two essential parts. the first consisted of a control hierarchy or network of criteria that controlled the interactions in the system. the second component of the anp was the network of influences among the criteria and clusters. the network depended on the criteria, as the network of influence was different for each criterion. thus, a supermatrix of limiting influence was computed for each control criterion. then, each of these supermatrices was weighed by the priority of its control criteria, and the results were synthesized through the addition of all the control criteria (aminu et al., 2017). this study also adopted and modified the decision-making framework for eia from ataei et al. (2016), liu and lai (2009), samah et al. (2010), kaya and kahraman (2011), tian et al. (2013), abba et al. (2013), and younes et al. (2015). in the first stage of the research, an exploratory study was conducted to identify and understand the criteria and the influence network. the second stage involved the use of questionnaires to perform the pairwise comparisons. during the third stage, the collected data were synthesized using the anp superdecisions software. finally, the synthesized results from the anp superdecisions software were used to develop the decision support framework. the research process is shown in figure 2. ijahp article: periaiah, islam, abdullah/environmental impact assessment for malyasian bauxte mining industry international journal of the analytic hierarchy process 9 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.851 stage 1: initial phase  research idea  problem statement  research objective  literature review stage 2: research design phase  research framework  research strategies  research instrument stage 3: data collection phase  data collection stage 4: data analysis phase  data analysis using superdecisions software  development of decision-making framework figure 2 research process of the study several methods often applied during the exploratory stage are interviews with experts in the area (sarupria et al., 2019; spiegel, 2017) and a review of secondary sources (louw, 2018; leonard, 2017). the exploratory stage in this study involved several semi-structured interviews with people who have knowledge and insight about eia in mining industries. the main purpose of this exploratory stage was to determine relevant criteria for the eia in bauxite mining industries. quantitative data collection activities commenced after the survey instrument was finalized. research data were collected through the distribution of the survey instrument to the regulators in the department of environment (doe), malaysia. the selected regulators were those who oversee eia approvals and subject experts, known as doe registered eia consultants in malaysia. several academicians and the public were also included in this study to analyze different opinions on criteria weightage and ranking. respondents in this research were chosen by considering their understanding of eia for mining industries. the eia consultants selected to identify the clusters of the anp influence network were from the fields of general environment, ecology, and socio-economic affairs. as for the regulators, the position of director or assistant director for the eia department in doe putrajaya headquarters and doe pahang state were selected for the interviews and questionnaire survey. table 1 provides a summary of the eia experts from relevant subject areas, regulators, academicians and the public who were selected as respondents. ijahp article: periaiah, islam, abdullah/environmental impact assessment for malyasian bauxte mining industry international journal of the analytic hierarchy process 10 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.851 table 1 list of respondents for the study category subject area/ location number of respondents expert – eia consultants general environment (air, noise, and water) socio-economic affairs ecology 5 3 3 academics environment, socioeconomic and ecology 3 public local mp representative, business owner and the village head 3 regulator – doe officer doe putrajaya and pahang 5 total respondents 22 as shown in table 1, a total of 22 respondents participated in this study to make pairwise comparisons among the clusters and their constituent elements. the research questionnaire dealt with ten criteria divided into three clusters as shown in figure 3. these ten criteria were selected based on the interviews conducted and also based on the literature review. figure 3 research conceptual model – influence network ijahp article: periaiah, islam, abdullah/environmental impact assessment for malyasian bauxte mining industry international journal of the analytic hierarchy process 11 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.851 for each criterion, the components were compared according to their relative impact or absence of impact on each component at the top of the supermatrix. this enabled the development of priorities to weigh the block matrices of eigenvector columns under that component in the supermatrix. weighing the components of the unweighted supermatrix resulted in a stochastic matrix called the weighted supermatrix. saaty (2005) emphasized that the supermatrix needs to be stochastic to obtain significant limiting priorities. the supermatrix was reduced to a matrix before taking the limit, whereby each of its columns sums to unity, thus, resulting in a matrix called a column stochastic matrix. each column in the supermatrix sums to the number of its non-zero eigenvectors. that is why clusters were compared before conversion into a stochastic matrix. the clusters were compared according to their impact on each other with respect to the general control criterion previously considered. in the case of several control criteria, the process was repeated several times for a decision problem, for each control criterion. for each control criterion, several comparison matrices were needed. each matrix was used to compare the influence of all the clusters on a given cluster to which they were connected. this resulted in an eigenvector that influenced all the clusters. a vector had zero components when there was no influence. the priority of a component was used to weigh all the elements in the block of the supermatrix that corresponded to the elements of both the influencing and the influenced cluster. the outcome was a stochastic supermatrix. once the priority weight for each criterion was determined using a pairwise comparison matrix, the priority results were used to develop a decision-making framework. this framework assisted the authorities in their decision-making process pertaining to the bauxite mining project. the determination of priorities is a one-time process, but once established, it may be generalized and used for all subsequent bauxite mining evaluations. the next step is to develop a simplified assessment form for decision makers to evaluate the alternatives. as this research focused on eia for the bauxite mining industry, the goal of this research is an eia report approval. the decision is whether to reject, approve or approve with a condition. baseline monitoring data and assessment results from eia consultants can be incorporated into the model. an absolute measurement technique was used to rate each criterion. each criterion was evaluated by measuring the significance of the criteria intensities. finally, the overall acceptability of the bauxite mining project was calculated. 4. results and discussion 4.1 demographic profile of the respondents as mentioned in the previous section, the present study involved 22 respondents belonging to six demographic categories including eia consultants specializing in the general environment, eia consultants specializing in the socio-economic sector, eia consultants specializing in ecology, doe enforcement officers, academicians, and members of the public residing close to the bauxite mining sites. after continuous follow-up for six months, only 22 respondents, or 44% of the 50 e-mails sent out responded by returning the survey questionnaires. many declined on account of their tight schedule, traveling program or refinery turn-around. table 2 provides detailed information on the respondents’ demographic profile. twelve of the respondents had more than 10 years of work experience, while the rest had six to 10 years of work ijahp article: periaiah, islam, abdullah/environmental impact assessment for malyasian bauxte mining industry international journal of the analytic hierarchy process 12 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.851 experience. as for the respondents’ qualifications, 10 individuals were subject specialists with doctorate degrees (phd), eight respondents held a master’s degree, and the remaining four held bachelor’s degrees. table 2 detailed information on respondents’ profiles demographic profile no. of respondents percentage category of respondents: doe officer 5 22.7 eia consultant – environment eia consultant – socioeconomic 5 3 22.7 13.6 eia consultant ecology academic public 3 3 3 13.6 13.6 13.6 education level: bachelors 4 18.2 masters 8 36.4 phd 10 45.5 position: general manager & above 11 50.0 senior manager 5 22.7 manager others 4 2 18.2 9.1 type of employment: public sector private sector self – employment others 5 11 3 3 22.7 50.0 13.6 13.6 work experience: 6 to 10 years more than 10 years 10 12 45.5 54.5 total 22 100 4.2 model construction ten criteria were identified and grouped into three clusters according to their common properties. the definitions of these ten criteria are provided in table 3 and the three clusters are environmental, economic, and ecological. through the identification of dependencies among all the components, the inter-relationships structure appears as an influential network. figure 4 depicts the anp influence network. ijahp article: periaiah, islam, abdullah/environmental impact assessment for malyasian bauxte mining industry international journal of the analytic hierarchy process 13 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.851 table 3 definition of the ten criteria criteria definition air air quality in malaysia is reported as the api (air pollutant index). four of the index's pollutant components are carbon monoxide, ozone, nitrogen dioxide, and sulphur dioxide. water water quality in malaysia is reported as the wqi (water quality index). it is calculated based on six of the index's pollutant components, namely dissolve oxygen (do), biological oxygen demand (bod), chemical oxygen demand (cod), ammoniacal nitrogen (nh3-n), suspended solid (ss), and ph. soil chemical residue noise decibel of noise waste rubbish and construction waste culture cultural heritage destruction and landscape demolition society public facility/transportation inaccessibility and community disconnection economy economic activity disturbance terrestrial threat to animals, plants, and endangered species aquatic threat to animals living in water ijahp article: periaiah, islam, abdullah/environmental impact assessment for malyasian bauxte mining industry international journal of the analytic hierarchy process 14 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.851 figure 4 anp influence network in figure 4, the arrow that points from water to soil denotes that soil pollution is influenced by water pollution. this is true especially during heavy rainfall and the monsoon season. surface water runoff often carries many pollutants and dirt across the soil which causes soil pollution. meanwhile, the presence of a double-headed arrow between water and soil further shows a two-way influence between soil and water. this indicates that soil pollution may also influence water pollution. soil from mining sites may enter the water and cause water pollution. note that no arrow appears between soil and noise. this shows that soil pollution does not influence noise pollution, and vice versa. all the remaining arrows in the network were established following the same procedure. experts’ opinions were sought to determine and confirm the dependence relationships among the criteria. 4.3 determination of criteria weights the next stage of the anp process is to form pairwise comparison matrices for the clusters and criteria. a pairwise comparison matrix was constructed for each cluster and all the respective items it influenced. each criterion was compared with another criterion as per its relative influence on the main criterion. the relative importance was then arranged into a matrix, and the respondents were asked to choose the relative influence among the criteria, with respect to the chosen criterion. next, respondents judged the relative importance with respect to the prioritized key criteria, such as air, water, soil, noise, waste, culture, society, economic, terrestrial, and aquatic. in pairwise comparison matrices, saaty’s 1-9 scale was used to express the extent to which one element is dominant over another according to the criterion/property to which they were compared (saaty, 2005). ijahp article: periaiah, islam, abdullah/environmental impact assessment for malyasian bauxte mining industry international journal of the analytic hierarchy process 15 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.851 once the comparison of clusters was completed, each criterion that interlinked was further compared with respect to the selected key criteria. table 4 presents a summary of 10 completed pairwise comparison matrices for all the key criteria by 22 respondents (geometric mean value of 22 respondents). each matrix represents one of the ten key criteria in this study. some rows and columns were left blank as there is no relationship between the criteria. in the air pollution matrix (top left of table 4), the first comparison between water pollution and soil pollution showed water pollution as little more than equally important as soil pollution, with an intensity of importance of 1.43. this was followed by water and waste at 2.19, and soil and waste at 2.00. the air pollution matrix is a combination of results from three clusters (environment, economy, and ecology). the next key criterion, water pollution (top right corner of table 4), has a geometric mean influence with respect to water pollution of 1.64 for air over soil, 2.46 for air over waste, 1.52 for soil over waste, 2.13 for economy over culture, and 1.00 for terrestrial over aquatic. all the remaining eight pairwise comparison matrices in table 4 were established following the same procedure. all pairwise comparison matrices in table 4 showed a cr value of less than 0.1, thus, confirming that the pairwise comparisons or judgements given by all the respondents were consistent and acceptable (asadabadi et al., 2019). after formation of pairwise comparison matrices, the priority values and ranks of the bauxite mining impacts were established using the superdecisions software. the determination of the overall priority values or weightages of the criteria is not straight forward in the anp as some criteria have more than two dependencies or interlink effects. therefore, in this anp application, two more matrices were established, namely a limit matrix (table 5) and the weighted supermatrix (table 6). cells that show a mean value of 0.0 denote the absence of a relationship or dependences among the criteria. ijahp article: periaiah, islam, abdullah/environmental impact assessment for malyasian bauxte mining industry international journal of the analytic hierarchy process 16 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.851 table 4 pairwise comparison matrices for the ten criteria legend: air = air, wtr = water, sol = soil, nos = noise, wst = waste, clt = culture, scy = society, ecm = economic, trl = terrestrial and aqu = aquatic air wtr sol wst clt ecm trl aqu wtr air sol wst ecm clt trl aqu wtr 1 1.43 2.19 air 1 1.64 2.46 sol 1 2.00 sol 1 1.52 wst 1 wst 1 clt 1 0.69 ecm 1 2.13 ecm 1 clt 1 trl 1 0.54 trl 1 1.00 aqu 1 aqu 1 sol air wtr wst clt ecm trl aqu wst air wtr sol clt ecm trl aqu air 1 1.57 2.89 air 1 1.06 0.71 wtr 1 2.03 wtr 1 0.59 wst 1 sol 1 clt 1 0.61 clt 1 0.40 ecm 1 ecm 1 trl 1 1.33 trl 1 1.29 aqu 1 aqu 1 clt air wtr sol nos wst scy ecm trl aqu scy air wtr sol nos wst clt ecm trl aqu air 1 0.69 1.08 1.43 1.04 air 1 0.96 1.87 2.12 1 wtr 1 1.67 1.84 1.38 wtr 1 1.87 2.52 1.01 sol 1 1.29 0.88 sol 1 1.14 0.60 nos 1 0.81 nos 1 0.59 wst 1 wst 1 scy 1 0.83 clt 1 0.45 ecm 1 ecm 1 trl 1 1.29 trl 1 1.76 aqu 1 aqu 1 ecm air wtr sol nos wst clt scy trl aqu trl air wtr sol nos wst scy clt ecm air 1 0.96 1.41 2.12 1.41 air 1 1.20 1.20 2.52 1.24 wtr 1 1.26 1.93 1.33 wtr 1 1.44 2.56 1.30 sol 1 1.85 1.38 sol 1 2.82 1.27 nos 1 0.74 nos 1 0.75 wst 1 wst 1 clt 1 0.64 scy 1 1.05 scy 1 clt 1 1 0.85 trl 1 2.95 ecm 1 aqu 1 aqu air wtr sol nos wst clt scy ecm nos ecm clt air 1 0.90 1.44 2.41 0.99 ecm 1 1.01 wtr 1 2.04 2.97 1.43 clt 1 sol 1 2.61 1.17 nos 1 0.73 wst 1 clt 1 0.80 0.99 scy 1 1.40 ecm 1 ijahp article: periaiah, islam, abdullah/environmental impact assessment for malyasian bauxte mining industry international journal of the analytic hierarchy process 17 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.851 table 5 limit matrix for the clusters and criteria air water soil noise waste culture society economic terrestrial aquatic air 0.10313 0.10313 0.10313 0.10313 0.10313 0.10313 0.10313 0.10313 0.10313 0.10313 water 0.12064 0.12064 0.12064 0.12064 0.12064 0.12064 0.12064 0.12064 0.12064 0.12064 soil 0.11515 0.11515 0.11515 0.11515 0.11515 0.11515 0.11515 0.11515 0.11515 0.11515 noise 0.03499 0.03499 0.03499 0.03499 0.03499 0.03499 0.03499 0.03499 0.03499 0.03499 waste 0.11963 0.11963 0.11963 0.11963 0.11963 0.11963 0.11963 0.11963 0.11963 0.11963 culture 0.05868 0.05868 0.05868 0.05868 0.05868 0.05868 0.05868 0.05868 0.05868 0.05868 society 0.02292 0.02292 0.02292 0.02292 0.02292 0.02292 0.02292 0.02292 0.02292 0.02292 economic 0.03877 0.03877 0.03877 0.03877 0.03877 0.03877 0.03877 0.03877 0.03877 0.03877 terrestrial 0.18304 0.18304 0.18304 0.18304 0.18304 0.18304 0.18304 0.18304 0.18304 0.18304 aquatic 0.20304 0.20304 0.20304 0.20304 0.20304 0.20304 0.20304 0.20304 0.20304 0.20304 table 6 weighted supermatrix for the clusters and criteria air water soil noise waste culture society economic terrestrial aquatic air 0.00000 0.06735 0.05971 0.00000 0.09513 0.06355 0.07006 0.10379 0.18641 0.16389 water 0.16069 0.00000 0.14719 0.00000 0.20123 0.17970 0.12037 0.10251 0.11310 0.12346 soil 0.15843 0.26541 0.00000 0.00000 0.18582 0.08550 0.07929 0.13283 0.08711 0.08193 noise 0.00000 0.00000 0.00000 0.00000 0.00000 0.06742 0.08669 0.08628 0.06219 0.07055 waste 0.16305 0.14941 0.27526 0.80380 0.00000 0.08627 0.12602 0.05703 0.03363 0.04261 culture 0.06270 0.06562 0.06562 0.10939 0.07225 0.00000 0.05601 0.05727 0.04739 0.05958 society 0.00000 0.00000 0.00000 0.00000 0.00000 0.08726 0.00000 0.06028 0.04739 0.03343 economic 0.05500 0.05208 0.05208 0.08681 0.04544 0.03030 0.06155 0.00000 0.02278 0.02454 terrestrial 0.15465 0.18616 0.20007 0.00000 0.14198 0.14194 0.20001 0.27014 0.00000 0.40001 aquatic 0.24548 0.21398 0.20007 0.00000 0.25815 0.25807 0.20001 0.12987 0.40001 0.00000 ijahp article: periaiah, islam, abdullah/environmental impact assessment for malyasian bauxte mining industry international journal of the analytic hierarchy process 18 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.851 4.4 ranking based on all the respondents this section presents the priorities and corresponding ranks of the ten criteria in the eia for the bauxite mining industry (table 7). to generate the priorities, all the respondents were considered together using the geometric means of the individual judgements provided by them. air pollution ranked first with a normalized priority value of 0.1659 or 16.6%, followed by water pollution at 15.5%, and soil pollution at 14.0%. in combination, these three criteria accounted for 46.1%, and easily outranked the economic potential which was prioritized at only 12.0%. this finding was similar to ataei et al. (2016) where the researcher used the matrix method for an eia for coal mining in iran. the findings of ataei et al. (2016) also show that the environmental criteria have a high impact at 49.1 %. table 7 priorities and ranks of the ten criteria criteria priority value rank air 0.1659 1 water 0.1549 2 soil 0.1397 3 economic 0.1197 4 waste 0.1157 5 terrestrial 0.0882 6 culture 0.0766 7 aquatic 0.0719 8 society 0.0394 9 noise 0.0279 10 4.5 development of decision-making framework a decision-making framework was developed using the priority weights determined by the anp. three respondents completed the environmental impact assessment form for the kuantan bauxite mining project and the geometric means were used to obtain the intensity of impact/pollution. table 8 summarizes the results for the intensity of impact/pollution for each criterion as rated by the respondents. the intensities of impact/pollution were rated as very low, low, medium, high, and very high. each of these ratings corresponds to a score of 50%, 26.2%, 13.3%, 6.7%, and 3.6%, respectively. the higher the intensity of impact/pollution, the lower the corresponding score. if one gives the best rating of very low impact/pollution for all the criteria, then the highest total score for the project would be 50 points. the respondents rated air pollution as very high significance and water pollution as high significance. soil pollution and negative economic impact were rated as medium significance, while waste generation, aquatic impact, and noise pollution were rated as low significance. finally, the impact on terrestrial, culture, and society categories was rated as very low significance. the results showed that the overall bauxite mining project at kuantan scored 21.48 points out of the highest possible 50 points. hence, this score represents an overall project score of 41.92%. ijahp article: periaiah, islam, abdullah/environmental impact assessment for malyasian bauxte mining industry international journal of the analytic hierarchy process 19 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.851 table 8 framework of intensity of impact/pollution for each criterion criteria priority impact/ pollution level score (point) overall project score (%) individual criteria score (%) very low low medium slightly high high 0.501 0.262 0.133 0.067 0.036 air 0.1659  0.60 1.19 7.2 water 0.1549  1.04 2.08 13.4 soil 0.1397  1.86 3.72 26.5 economic 0.1197  1.59 3.18 26.5 waste 0.1157  3.03 6.06 52.3 terrestrial 0.0882  4.42 8.84 100.0 culture 0.0766  3.84 7.68 100.0 aquatic 0.0719  1.88 3.77 52.3 society 0.0394  1.97 3.95 100.0 noise 0.0279  0.73 1.46 52.3 score 1.00 20.96/ 50 41.92 ijahp article: periaiah, islam, abdullah/environmental impact assessment for malyasian bauxte mining industry international journal of the analytic hierarchy process 20 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.851 5. limitations and recommendations for future research the study would have been more reliable if higher level government officials such as the director of the department of environment, director of minerals and geoscience department malaysia, and minister of water, land and natural resources of malaysia were involved. efforts were made to interview these high-ranking officials; however, due to their tight schedules, the interview sessions were conducted with their juniors. this study would have provided a more balanced and complete view on bauxite mining if only high level officials had been interviewed. their opinions and feedback would have been more relevant as they are the actual decision makers for bauxite mining issues. the anp can be used as a final decision-making model by including some alternatives using baseline data or the estimated value for each criterion. the decision-making problem in eia is whether to approve or reject the proposed project. since bauxite mining has huge economic potential, one will choose to approve the project. therefore, most approvals are given together with special requirements to manage issues concerning significant criteria such as air, water, and soil pollution. however, baseline data are not available for bauxite mining in malaysia as no eia had been done for bauxite mining. hence, this study was unable to include alternatives and baseline data. after receiving the responses from the respondents, it is evident that pollution is a major problem even though actions have been taken to alleviate it. this research only contributes to the prioritization of the significant environmental criteria specific to bauxite mining. as for future research, attention should be directed towards the development of environmental management plans and standard operating procedures (sop) to reduce the identified significant environmental impact of bauxite mining activities. with air, water, and soil pollution being major concerns, a mechanism to reduce the impacts of these factors should be studied and documented. the enforcement agency such as doe officers should be more active in managing the impacts. future research should also focus on the qualitative part of the research, whereby, high-ranking government officials who oversee decision-making pertaining to eia should be interviewed to better understand government policy and planning. further, the anp analysis should be expanded to include sub-criteria relevant to the environmental and socio-economic criteria. final decisions such as to approve, reject, or approve with conditions should be incorporated as alternatives in the anp model. scientific data such as actual air, water, and soil monitoring results should also be included in the anp analysis. 6. conclusion this study highlighted the repercussions of bauxite mining on the environment through the impacts of the environment, ecology, and economy. ten relevant criteria belonging to three clusters were identified through the literature survey and further confirmations were obtained from subject experts. the environmental cluster consisted of the following five criteria: air, water, soil, noise, and waste. the economic cluster contained three criteria which are the economy, society, and culture. meanwhile, the ecological cluster was made up of the terrestrial and aquatic criteria. ijahp article: periaiah, islam, abdullah/environmental impact assessment for malyasian bauxte mining industry international journal of the analytic hierarchy process 21 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.851 the priorities of these criteria are important for decision-making in eia. the anp analysis ranked air pollution as the first priority at 16.6%, followed by water pollution at 15.5%, soil pollution at 14.0%, economic impact at 12.0%, waste generation at 11.6%, terrestrial impact at 8.8%, cultural impact at 7.7%, aquatic impact at 7.2%, society at 3.9 %, and finally noise at 2.8%. indirectly, these values show the amount of effort needed to manage each component to meet the economic benefit of the bauxite mining project. air, water, and soil pollution are the three highest ranking criteria that require standard operating procedures or a management plan to control and manage the possible detrimental effects that may result from bauxite mining activities. the priority values for the ten significant criteria derived from the anp were used to develop a framework that is suitable to support the overall decision-making process concerning the eia of the bauxite mining industry. the framework supports both the qualitative and quantitative data. thus, it allows decision-makers to approve, reject, or approve with conditions different bauxite mining projects. based on the testing done for decision-making framework developed in this study, the bauxite mining projects in kuantan only scored 41.92%. this result suggests that the decision on bauxite mining in kuantan should be rejected. however, considering the positive economic benefit to the pahang state government and the residents, the decision maker may choose to approve the project with conditions. finally, this study pioneers the use of the anp to rank the weightage of the environmental impacts caused by bauxite mining in malaysia. the primary contribution of this study is the development of a network model that prioritizes bauxite mining impacts, criteria, and clusters using the anp. this study sets a benchmark for prioritizing the significance of environmental impacts in bauxite mining operations. ijahp article: periaiah, islam, abdullah/environmental impact assessment for malyasian bauxte mining industry international journal of the analytic hierarchy process 22 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.851 references abba, a. h., noor, z. z., yusuf, r. o., din, m. f. m. d., & hassan, m. a. a. (2013). assessing environmental impacts of municipal solid waste of johor by analytical hierarchy process. resources, conservation and recycling, 73(april), 188 196. doi: https://doi.org/10.1016/j.resconrec.2013.01.003 abdullah, n. h., mohamed, n., sulaiman, l. h., zakaria, t. a., & rahim, d. a. (2016). potential health impacts of bauxite mining in kuantan. malays journal of medical science, 23(3), 1-8. aminu, m., matori, a. n., yusof, k. w., malakahmad, a., & zainol, r. b. (2017). analytic network process (anp)-based spatial decision support system (sdss) for sustainable tourism planning in cameron highlands, malaysia. arab journal of geoscience, 10(286), 1-20. doi: https://doi.org/10.1007/s12517-017-3067-0 asadabadi, m. r., chang, e., & saberi, m. (2019). are mcdm methods useful? a critical review of analytic hierarchy process (ahp) and analytic network process (anp). cogent engineering, 6(1), 1-11. doi: https://doi.org/10.1080/23311916.2019.1623153 ataei, m., asr, e. t., khalokakaie, r., ghanbari, k., & mohammadi, m. r. t. (2016). semi-quantitative environmental impact assessment and sustainability level determination of coal mining using a mathematical model. journal of mining & environment, 7(2), 185-193. beltran, p. a., melon, m. g., & valera, j. m. (2017). how to assess stakeholders' influence in project management? a proposal based on the analytic network process international journal of project management, 35(3), 451-462. doi: https://doi.org/10.1016/j.ijproman.2017.01.001 bond, a., & morrison, a. s. (2018). environmental impact assessment and the quest for sustainable mining (vol. part 2). new york: routledge. carvalho, f. p. (2017). mining industry and sustainable development: time for change. food and energy security, 6(2), 61-77. cmo. (2016). bauxite in malaysia: will the ban bring relief? by clean malaysia organization (cmo). retrieved on: december 11, 2020, from http://cleanmalaysia.com/2016/01/26 daim, n. (2019). 6-month trial period for new bauxite mining sop. new straits times. retrieved on: january 17, 2021, from https://www.nst.com.my/news/nation/2019/11/535940/6-month-trial-periodnew-bauxite-mining-sop dell'anna, f., bottero, m., becchio, c., corgnati, s.p., & mondini, g. (2020). designing a decision support system to evaluate the environmental and extraeconomic performances of a nearly zero-energy building. smart and sustainable built environment, 9(4), 413-442. doi: https://doi.org/10.1108/sasbe-09-2019-0121 doe. (2016). eia order 2015 by department of environment (doe) malaysia. retrieved on: august 26, 2020, from https://www.doe.gov.my/portalv1/en https://doi.org/10.1016/j.resconrec.2013.01.003 https://doi.org/10.1007/s12517-017-3067-0 https://doi.org/10.1080/23311916.2019.1623153 https://doi.org/10.1016/j.ijproman.2017.01.001 http://cleanmalaysia.com/2016/01/26 https://www.nst.com.my/news/nation/2019/11/535940/6-month-trial-period-new-bauxite-mining-sop https://www.nst.com.my/news/nation/2019/11/535940/6-month-trial-period-new-bauxite-mining-sop https://www.emerald.com/insight/search?q=federico%20dell%27anna https://www.emerald.com/insight/search?q=marta%20bottero https://www.emerald.com/insight/search?q=cristina%20becchio https://www.emerald.com/insight/search?q=stefano%20paolo%20corgnati https://www.emerald.com/insight/search?q=giulio%20mondini https://www.emerald.com/insight/publication/issn/2046-6099 https://www.emerald.com/insight/publication/issn/2046-6099 https://doi.org/10.1108/sasbe-09-2019-0121 https://www.doe.gov.my/portalv1/en ijahp article: periaiah, islam, abdullah/environmental impact assessment for malyasian bauxte mining industry international journal of the analytic hierarchy process 23 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.851 dosm. (2017). department of statistics malaysia (dosm). retrieved on: april 5, 2020, from http://www.dosm.gov.my faradiella, m. k., rahman, m. s. a., madzin, z., jusop, s., yusuff, f. m., & ariffin, m. (2016). the occurrence and potential ecological risk assessment of bauxite mineimpacted water and sediments in kuantan, pahang, malaysia. environmental science pollution research, 24(1), 1-16. doi: https://doi.org/10.1007/s11356-016-7814-7 garcia, l. c., ribeiro, d. b., roque, f. d. o., & quintero, j. m. o. (2016). brazil's worst mining disaster: corporations must be compelled to pay the actual environmental costs. ecological applications, 27(1), 5-9. doi: https://doi.org/10.1002/eap.1461 hadiwijaya, n. a., hamdani, h., syafrianto, a., & tanjung, z. (2018). the decision model for selection of tourism site using analytic network process method. international journal of intelligent systems and applications, 9(september), 23-31. doi: https://doi.org/10.5815/ijisa.2018.09.03 hamida, a., alsudairi, a., alshaibani, k., & alshamrani, o. (2020). environmental impacts cost assessment model of residential building using an artificial neural network. engineering, construction and architectural management, vol. ahead-ofprint no. doi: https://doi.org/10.1108/ecam-06-2020-0450 harun, n., & samat, n. (2016). gis-based multicriteria evaluation approach in planning tourism development sites in environmentally sensitive areas. shs web of conferences, 23(02001), 1-16. doi: https://doi.org/10.1051/shsconf/20162302001 huang, i. b., keisler, j., & linkov, i. (2011). multi-criteria decision analysis in environmental sciences: ten years of applications and trends. science of the total environment, 409(1), 3578–3594. doi: https://doi.org/10.1016/j.scitotenv.2011.06.022 jain, r. k., cui, z. c., & domen, j. k. (2016). environmental impact of mining and mineral processing. oxford, uk butterworth-heinemann. jamal, a., yadav, h. l., & pandey, s. s. (2015). heavy metals from acid mine drainage in coal mines-a case study. european journal of advances in engineering and technology, 2(8), 16-20. jmg. (2018). malaysian minerals production for 2016 by minerals and geoscience department (jmg) malaysia. retrieved on: october 22, 2019, from http://www.data.gov.my/data/ms_my/dataset/malaysia-minerals-production doi: https://doi.org/10.7186/bgsm51200501 kadoic, n., divjak, b., & redep, n. b. (2019). integrating the dematel with the analytic network process for effective decision-making. central european journal of operations research, 27(3), 653–678. doi: https://doi.org/10.1007/s10100-0180601-4 kaya, t., & kahraman, c. (2011). an integrated fuzzy ahp–electre methodology for environmental impact assessment. expert systems with applications, 38(7), 8553-8562. doi: https://doi.org/10.1016/j.eswa.2011.01.057 http://www.dosm.gov.my/ https://doi.org/10.1007/s11356-016-7814-7 https://doi.org/10.1002/eap.1461 https://doi.org/10.5815/ijisa.2018.09.03 https://www.emerald.com/insight/search?q=amneh%20hamida https://www.emerald.com/insight/search?q=abdulsalam%20alsudairi https://www.emerald.com/insight/search?q=khalid%20alshaibani https://www.emerald.com/insight/search?q=othman%20alshamrani https://www.emerald.com/insight/publication/issn/0969-9988 https://doi.org/10.1108/ecam-06-2020-0450 https://doi.org/10.1016/j.scitotenv.2011.06.022 http://www.data.gov.my/data/ms_my/dataset/malaysia-minerals-production https://doi.org/10.7186/bgsm51200501 https://doi.org/10.1007/s10100-018-0601-4 https://doi.org/10.1007/s10100-018-0601-4 https://doi.org/10.1016/j.eswa.2011.01.057 ijahp article: periaiah, islam, abdullah/environmental impact assessment for malyasian bauxte mining industry international journal of the analytic hierarchy process 24 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.851 kheybari, s., rezaie, f. m., & farazmand, h. (2020). analytic network process: an overview of applications. applied mathematics and computation, 367(february), 124780. doi: https://doi.org/10.1016/j.amc.2019.124780 lee, k. y., ho, l. y., tan, k. h., tham, y. y., ling, s. p., qureshi, a. m., . . . nordin, r. (2017). environmental and occupational health impact of bauxite mining in malaysia: a review. international medical journal malaysia, 16(2), 137-150. leena, s., thomas, e., elena, k., korchak, e., vigdis, n., & gregory, a. p. (2017). social impacts of the “glocal” mining business: case studies from northern europe. mineral economics, 30(1), 31-39. doi: https://doi.org/10.1007/s13563-0160092-5 leonard, l. (2017). examining environmental impact assessments and participation: the case of mining development in dullstroom, mpumalanga, south africa. journal of environmental assessment policy and management, 19(1), 1750002(1-25). doi: https://doi.org/10.1142/s1464333217500028 liu, k. f. r., & lai, j.-h. (2009). decision-support for environmental impact assessment: a hybrid approach using fuzzy logic and fuzzy analytic network process. expert systems with applications, 36(3), 5119-5136. doi: https://doi.org/10.1016/j.eswa.2008.06.045 lodhia, s. k. (ed.) (2018). mining and sustainable development. new york: routledge. louw, l. (2018). addressing critical water and waste issues in environmental impact assessment (eia): the case of coal mining in south africa. (masters in environmental management). north-west university, potchefstroom campus. (22142762). mahmud, a. r. (2016). guideline on environmental impact assessment in malaysia. kuala lumpur: department of environment (doe), malaysia malaysiakini. (2019). kuantan mp wants ministry to disclose sop on bauxite mining. malaysiakini. retrieved on: january 2020, from https://www.malaysiakini.com/news/464783 mcom. (2017). malaysian chamber of mines (mcom). retrieved on: april 5, 2018 from http://malaysianminerals.com povera, a. (2019). bauxite sop finalised, enforcement starts today. new straits times. retrieved on: january 17, 2020, from https://www.nst.com.my/news/nation/2019/09/518862/bauxite-sop-finalisedenforcement-starts-today-nsttv rohan, j. l., & samant, j. s. (2012). studies on the impact of bauxite mining activities on environment in kolhapur district. paper presented at the proceeding of international conference swrdm-2012, department of environmental science, shivaji university, kolhapur. doi: https://doi.org/10.26634/jfet.14.4.16069 https://doi.org/10.1016/j.amc.2019.124780 https://doi.org/10.1007/s13563-016-0092-5 https://doi.org/10.1007/s13563-016-0092-5 https://doi.org/10.1142/s1464333217500028 https://doi.org/10.1016/j.eswa.2008.06.045 https://www.malaysiakini.com/news/464783 http://malaysianminerals.com/ https://www.nst.com.my/news/nation/2019/09/518862/bauxite-sop-finalised-enforcement-starts-today-nsttv https://www.nst.com.my/news/nation/2019/09/518862/bauxite-sop-finalised-enforcement-starts-today-nsttv https://doi.org/10.26634/jfet.14.4.16069 ijahp article: periaiah, islam, abdullah/environmental impact assessment for malyasian bauxte mining industry international journal of the analytic hierarchy process 25 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.851 saaty, t. l. (2005). theory and applications of the analytic network process: decision making with benefits, opportunities, costs, and risks. pittsburgh: rws publications. saffari, a., ataei, m., sereshki, f., & naderi, m. (2019). environmental impact assessment (eia) by using the fuzzy delphi folchi (fdf) method (case study: shahrood cement plant, iran). environment, development and sustainability, 21(2), 817-860. doi: https://doi.org/10.1007/s10668-017-0063-1 samah, m. a. a., manaf, l. a., & zukki, n. l. m. (2010). application of ahp model for evaluation of solid waste treatment technology. international journal of engineering techsci, 1(1), 35-40. sarupria, m., manjare, s. d., & girap, m. (2019). environmental impact assessment studies for mining area in goa, india, using the new approach. environmental monitoring and assessment, 191(1), 1-17. doi: https://doi.org/10.1007/s10661-0187135-z skuta, r., kucerova, r., pavelek, z., & dirner, v. (2017). assessment of mining activities with respect to the environmental protection. acta montanistica slovaca, 22(1), 79-93. solbar, l., & keskitalo, e. c. h. (2017). a role for authority supervision in impact assessment? examples from finnish eia reviews. arctic review on law and politics, 8(1), 52-72. doi: https://doi.org/10.23865/arctic.v8.661 spiegel, s. j. (2017). eias, power and political ecology: situating resource struggles and the techno-politics of small-scale mining. geoforum, 87(december), 95-107. doi: https://doi.org/10.1016/j.geoforum.2017.10.010 thorpe, c. (2015). lateritic plateaus in the northern western ghats, india; a review of bauxite mining restoration practices. journal of the ecological society, special section: rocky plateaus, 25-44. tian, w., bai, j., sun, h., & zhao, y. (2013). application of the analytic hierarchy process to a sustainability assessment of coastal beach exploitation: a case study of the wind power projects on the coastal beaches of yancheng, china. journal of environmental management, 115(1), 251-256. doi: https://doi.org/10.1016/j.jenvman.2012.11.015 tmr. (2017). mining sector could see silver lining ahead. the malaysian reserve (tmr). retrieved on: january 17, 2020, from https://themalaysianreserve.com/2017/03/31/mining-sector-could-see-silverlining-ahead/ doi: https://doi.org/10.1108/oxan-db251218 toth, w., & vacik, h. (2018). a comprehensive uncertainty analysis of the analytic hierarchy process methodology applied in the context of environmental decision making. journal of multi-criteria decision analysis, 25(6), 142-161. doi: https://doi.org/10.1002/mcda.1648 https://doi.org/10.1007/s10668-017-0063-1 https://doi.org/10.1007/s10661-018-7135-z https://doi.org/10.1007/s10661-018-7135-z https://doi.org/10.23865/arctic.v8.661 https://doi.org/10.1016/j.geoforum.2017.10.010 https://doi.org/10.1016/j.jenvman.2012.11.015 https://themalaysianreserve.com/2017/03/31/mining-sector-could-see-silver-lining-ahead/ https://themalaysianreserve.com/2017/03/31/mining-sector-could-see-silver-lining-ahead/ https://doi.org/10.1108/oxan-db251218 https://doi.org/10.1002/mcda.1648 ijahp article: periaiah, islam, abdullah/environmental impact assessment for malyasian bauxte mining industry international journal of the analytic hierarchy process 26 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.851 usgs. (2016). world alumina refinery and bauxite mine production and bauxite reserves by united states geological survey (usgs). retrieved on: april 5, 2018, from https://www.usgs.gov/ doi: https://doi.org/10.4135/9781412939591.n1177 venkateswarlu, k., nirola, r., kuppusamy, s., thavamani, p., naidu, r., & megharaj, m. (2016). abandoned metalliferous mines: ecological impacts and potential approaches for reclamation. reviews in environmental science and bio technology, 15(2), 327–354. doi: https://doi.org/10.1007/s11157-016-9398-6 younes, m. k., basri, n. e. a., nopiah, z. m., basri, h., & abushammala, m. f. m. (2015). use of a combination of mrss-anp for making an innovative landfill siting decision model. mathematical problems in engineering, 501, 381926 (online). doi: https://doi.org/10.1155/2015/381926 https://www.usgs.gov/ https://doi.org/10.4135/9781412939591.n1177 https://doi.org/10.1007/s11157-016-9398-6 https://doi.org/10.1155/2015/381926 ijahp article: noshad, asgharipour/application of the analytic hierarchy method to optimize the choice of a gluten-free cookie formula fortified with banana peel powder international journal of the analytic hierarchy process 355 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.618 application of the analytic hierarchy method to optimize the choice of a gluten-free cookie formula fortified with banana peel powder mohammad noshad * department of food science & technology faculty of animal science and food technology, agricultural sciences and natural resources university of khuzestan, mollasani, iran noshad@asnrukh.ac.ir sahar asgharipour asghari.pur@gmail.com abstract when considering several factors affecting the quality of gluten-free cookies, choosing the optimal formula based on multi-criteria decision making is complex and not easy. therefore, in this study the analytic hierarchy process (ahp) method was used to optimize the best formula for a gluten-free cookie containing banana peel powder. for this purpose, physicochemical aspects such as texture, aroma, taste, color and porosity, fiber content, antioxidant activity, phenolic compounds, moisture content and ash content were evaluated to determine the optimal formula for a gluten-free cookie containing banana peel powder. the results showed that the optimal cookie contained 4% banana peel powder with a weight of 0.178. moreover, the results showed that the rate of inconsistency was 0.77, and since it was less than 0.1 the results are reliable. keywords: ahp; glutenfree cookie; banana peel powder 1. introduction celiac disease is an autoimmune disorder in which the patient has a permanent wheat gluten intolerance. when these individuals' intake gluten, the small intestine becomes inflamed causing damage to the intestinal mucosa and problems absorbing nutrients. the only effective treatment for celiac disease is adherence to a gluten-free diet throughout the patient's life (giuberti, marti, fortunati, & gallo, 2017; lazaridou, duta, papageorgiou, belc, & biliaderis, 2007; molinari et al., 2017). therefore, paying attention to the production of gluten-free foods is very important for these patients. currently, in many countries, extensive studies are being conducted in various fields related to this disease. the production of gluten-free products including fiber can play an * acknowledgement: the authors would like to express their sincere thanks to the research deputy of agricultural sciences and natural resources university of khuzestan for their financial support. mailto:noshad@asnrukh.ac.ir mailto:asghari.pur@gmail.com ijahp article: noshad, asgharipour/application of the analytic hierarchy method to optimize the choice of a gluten-free cookie formula fortified with banana peel powder international journal of the analytic hierarchy process 356 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.618 important role in achieving products for celiac patients (simons & hall iii, 2018; turabi, sumnu, & sahin, 2010). dietary fibers are carbohydrate polymers with 10 or more monomeric units that are not hydrolyzed by human intestinal enzymes. the fibers have many functions such as reducing cholesterol, blood glucose, and constipation (lebesi & tzia, 2011; o'shea, arendt, & gallagher, 2012). bananas are one of the most widely used fruits in the world and are grown in almost all tropical countries (choo & aziz, 2010; zhang, whistler, bemiller, & hamaker, 2005). the banana peel makes up about 40% of the weight of a fresh banana and is considered agricultural waste and therefore discarded. the banana skin includes high levels of fiber, starch, pectin and effective chemical compounds. most of the carotenoids found in the banana skin are xanthophyll (yellow matter), with a small amount of laurate, palmitate or caprate. the banana peel is rich in polymers such as lignin, hemicellulose, and pectin (gonzález-montelongo, lobo, & gonzález, 2010; ortiz et al., 2017; tibolla, pelissari, martins, vicente, & menegalli, 2018). cookies are a wheat-based food and are an important product of the bakery industry, and are very popular in the daily diet. cookies have a very long shelf life and are available at almost any time (jan, panesar, & singh, 2018; popov-raljić, mastilović, laličićpetronijević, kevrešan, & demin, 2013; simons & hall iii, 2018). therefore, changes in the composition of cookies to enhance their nutritional or functional properties have attracted the attention of researchers (molinari et al., 2017). when considering several factors affecting cookie quality, choosing the optimal formula based on multi-criteria decision making is complex and not easy, especially because most of the considered criteria are conflicting so that an increase in the desirability of one can reduce the utility of another. for this reason, multi-criteria decision-making methods have been developed to help solve these problems. in these methods, several options are compared based on several criteria, and the best option or arrangement of the appropriate options is selected. among the multi-criteria decision-making methods, the analytic hierarchy process (ahp) method is considered effective for considering quantitative and qualitative conditions and variables simultaneously (ho & ma, 2017; saaty, 2008; sardh et al., 2017). however, no research has been conducted to date on the use of the ahp method to optimize food formulations. this study aimed to use the ahp method to optimize the best formula for gluten-free cookies containing banana peel powder. 2. materials and methods the ingredients that were used in the production of the cookie formulations include rice flour (shomal powderineh co, iran), sodium bicarbonate (shahriar, iran), shortening, sugar powder, dried milk powder (pegah co., iran), xanthan gum (rhodia food company, france), lecithin, sodium chloride and water and these were collected from the local market. 2.1. preparation of the banana peel powder after preparing the banana skin and washing it with water, it was placed in a 1% citric acid solution for 10 minutes, then placed in an oven at 40°c for 24 hours. next, the ijahp article: noshad, asgharipour/application of the analytic hierarchy method to optimize the choice of a gluten-free cookie formula fortified with banana peel powder international journal of the analytic hierarchy process 357 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.618 samples were dried until the moisture content was 20%. the dried samples were then milled, and a 35 size mesh sieve was used to make the particle size uniform. 2.2. cookie preparation jacob and leelavathi (2007) used methods based on the aacc 10-52 standard with some modifications to prepare the cookie dough. the formula of the cookie dough contained 40g rice flour, 24g sugar powder, 12g shortening, 1.2g dried skim milk, 0.32g sodium bicarbonate, 0.18g sodium chloride, 0.18g lecithin, 0.2g xanthan gum, banana peel powder (0-12 g/100g samples of rice flour) and water in the necessary amount. the cookie dough was sectioned into parts with a thickness of 0.5cm × 4cm. the cookies were baked in the oven (sgc5-ud2311n, snowa, iran) at 180°c. 2.3. moisture and ash content the cookie moisture on a wet weight was measured with the approved method of aacc 44-16, and the cookie ashes were measured with aacc08-01. 2.4. determination of color the color of the cakes was determined using the konica minolta colorimeter device (cr400 model, japan). in this experiment, l* was determined as the criterion of brightness, a* as the criterion of redness, and b* as the criterion of yellowness. 2.5. texture the texture of the samples was analyzed using a ta xt2i (stable micro systems, goldalming, uk) equipped with a 500kg load cell. a full cookie was pressed at a speed of 5 mm/s with a 6 mm cylindrical probe to 50% of the cookie thickness. through compression, a curve was created with the force greater than the distance. the highest maximum value was recorded because this value showed the first rupture of the cookie at a particular point. this amount of force was considered as the hardness (cevoli, balestra, ragni, & fabbri, 2013). 2.6. dietary and non-dietary fiber the fiber content was calculated according to the standard aoac-991.43 method. in this test, acidic digestion and then basic digestion were performed, then the samples were dried in the ovens (garcia-amezquita, tejada-ortigoza, heredia-olea, serna-saldívar & welti-chanes, 2018) . 2.7. total phenolic content (tpc) the total phenolic content of the compounds was determined according to singleton et al. (1999). the sample was defatted with a ratio of v/v 1:1 chlorophyram/petroleum ether. it was then dried in an oven at 40°c. about 1g of the defatted dry extract was mixed with 10ml of water or methanol. the mixture was processed in a centrifuge with 2000g for 15 minutes. then, 70ml of the solution was separated and mixed with 900ml distilled water, 1ml of folin ciocalteous reagent, and 2ml of 10% sodium carbonate. the absorption was recorded after 60 minutes at a wavelength of 765nm. the total phenolic content as the equivalent of the gallic acid (gae)/g dry weight sample was calculated from the calibration curve using the standard gallic acid solution (singleton et al., 1999). ijahp article: noshad, asgharipour/application of the analytic hierarchy method to optimize the choice of a gluten-free cookie formula fortified with banana peel powder international journal of the analytic hierarchy process 358 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.618 2.8. dpph assay this method is based on the dpph free-radical recovery by antioxidants in the absence of other free radicals in the environment. this results in coloration in an environment where its intensity can be measured by a spectrophotometer at a wavelength of 517nm. the experiment was carried out according to benvenuti, pellati, melegari and bertelli (2004). 2.9. best choice of produced sample in this research, a hierarchical analysis method was used to prioritize and select the best produced sample. first, the analyzed solutions and criteria obtained through the initial reviews were developed in a hierarchical analysis based on a hierarchical structure (including 4 options and 9 criteria). the data were analyzed using the expert choice software. the ahp method used in this study is effective when quantitative and qualitative criteria are present simultaneously, and when the multiple decision-making criteria make selection conditions problematic. the hierarchy determines the importance and priority of the various criteria. the ahp process was carried out as follows (saaty, 2008). the analytic hierarchy process begins by identifying and prioritizing decision elements. these elements are comprised of four levels including goal, criteria, sub-criteria and possible options that are used in prioritization. the first level includes determining the goal. in this research, the goal is to achieve the best gluten-free cookie formula. the second level includes determining the criteria. the nine criteria used in this study are texture, flavor, color, moisture, porosity, total fiber content, antioxidant activity, total phenolic content and ash content. the third level includes the solutions, and the four solutions were analyzed, including the produced products to achieve the goal of optimizing the formula of the gluten-free cookie. figure 1 shows the construction of the analytic hierarchy process (ahp) model. ijahp article: noshad, asgharipour/application of the analytic hierarchy method to optimize the choice of a gluten-free cookie formula fortified with banana peel powder international journal of the analytic hierarchy process 359 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.618 figure 1 analytic hierarchy process (ahp) model for optimizing the formula of glutenfree cookies best choice of a gluten-free cookie formulation that contains banana peel powder c o lo r f la v o r t e x tu re p o ro si ty m o is tu re c o n te n t t o ta l fi b e r a n ti o x id a n t a c ti v it y t f c a sh c o n tr o l c o o k ie 4 % c o o k ie – 8 % c o o k ie – 1 2 % ijahp article: noshad, asgharipour/application of the analytic hierarchy method to optimize the choice of a gluten-free cookie formula fortified with banana peel powder international journal of the analytic hierarchy process 360 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.618 2.10. statistical analysis the data reported in this study are the mean of a minimum of three replicates. all of the data were subjected to an analysis of variance (anova), and later to the duncan’s multiple range test to evaluate the significant differences among the treatments at p < 0.05. the statistical analyses were performed using spss 23.0 (spss science, chicago, il, usa). 3. results and discussion 3.1. moisture content moisture content is an important factor for evaluating the shelf-life and microbial stability of food products (lebesi & tzia, 2011). the results of the analysis of variance of the moisture content showed that there was a significant difference between the control and other samples (p< 0.05), however, there was no significant difference between the 4% and 8% samples (table 1). the results indicate that by increasing the amount of banana peel powder, which is the fiber source, the moisture content of the samples increases significantly (table 1). the use of dietary fiber increases the absorption and the ability to keep water in the samples, which leads to a delay in the samples becoming stale. this feature (increase in moisture content) is related to the porous structure of the fiber, which is formed by polysaccharide chains, and makes it possible to retain a large amount of water through a hydrogen bond (kethireddipalli, hung, phillips, & mcwatters, 2002; lebesi & tzia, 2011). 3.2. color the results of the analysis of variance showed that the addition of the banana peel powder had a significant effect on the color values (table 1). the addition of the banana peel powder caused a decrease in l * and b * values. the lowest l* and b *values were for the control sample, and the 12% sample contained the highest values. the addition of the banana peel powder increased the a * value in the samples (table 1). the reason for the change in the color value during the addition of the banana peel powder to the samples is probably due to the color and pigmentation of the banana that is created during the drying process as a result of the breakdown of pigments and enzymatic and non enzymatic browning reactions in the sample (chua, mujumdar, hawlader, chou & ho, 2001; prachayawarakorn, tia, plyto & soponronnarit, 2008). ijahp article: noshad, asgharipour/application of the analytic hierarchy method to optimize the choice of a gluten-free cookie formula fortified with banana peel powder international journal of the analytic hierarchy process 361 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.618 table 1 influence of the banana peel powder on moisture content and color values of the samples the different letters in each column indicate a significant difference (p<0.05) between the samples. 3.3. texture the hardness is the maximum force required to squeeze the food material between the teeth to achieve a certain deformity (chaiya & pongsawatmanit, 2011). the analysis of variance showed that the addition of the 4% banana peel powder did not have a significant effect on the hardness of the samples, while the increase in the amount of banana peel powder (4%-12%) increased the hardness of the samples significantly (table 2). the increased hardness as a result of the increased banana peel powder is likely due to the connections between the banana peel powder and the corn starch granules, which prevent gelatinization of starch and cell wall fragmentation and thickening of cell walls during bubble formation. this is because the fiber interferes with the formation of air bubbles and increases the thickness of the cell wall. the results of this study were consistent with the results of other researchers regarding the rheological properties of the cake dough with the addition of apple pomace (sudha, baskaran & leelavathi, 2007). 3.4. ash content the results of the analysis of variance showed that the addition of the banana peel powder caused a significant increase in the ash content of the samples (table 2). since increasing the amount of the banana peel powder, the minerals in the samples also increased, indicating the presence of minerals in the banana peel powder. the results were consistent with the results of other researchers (choo & aziz, 2010). 3.5. total phenolic content (tpc) phenolic compounds are one of the most important antioxidant compounds in fruits and vegetables. epidemiologic studies show that natural phenolic compounds play a major role in human health through antioxidant properties (baliga, baliga, kandathil, bhat, & vayalil, 2011). the results (table 2) showed that adding the banana peel powder had a significant effect (p< 0.05) on the tpc of the samples. the sample containing the 12% banana peel powder had the highest tpc among the samples. the increase in tpc in the sample due to the addition of the banana peel powder shows the presence of phenolic compounds in the banana peel powder (gonzález-montelongo et al., 2010). l* value a* value b* value moisture content (%) treatment 71.58±0.13 a 1.51±0.05 c 26.78±0.31 a 2.60±0.12 c control 47.42±0.62 b 3.17±0.48 b 24.85±0.47 b 3.50±0.11 b cookie4% 43.49±0.22 c 3.49±0.19 b 22.01±0.53 c 3.86±0.3 b cookie8% 41.48±0.16 d 4.41±0.28 a 20.42±0.13 d 4.95±0. 2 a cookie 12% ijahp article: noshad, asgharipour/application of the analytic hierarchy method to optimize the choice of a gluten-free cookie formula fortified with banana peel powder international journal of the analytic hierarchy process 362 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.618 3.6. antioxidant activity the results of the dpph radical inhibitory investigation are shown in table 2. the results of the study showed that cookies containing the banana peel powder had more radical inhibitory capability when compared to the control sample, and the addition of the banana peel powder had a significant effect (p < 0.05) on the amount of radical inhibitory effect of the samples. as the amount of banana peel powder increased, the amount of the radical inhibitory strength of the samples also increased, as a result of the presence of phenolic compounds in the banana powder (someya, yoshiki, & okubo, 2002). table 2 influence of the banana peel powder on texture, ash, tpc, and antioxidant properties of the samples ash (%) tpc (mg/g dw) dpph (%) hardness (n) treatment 1.75±0.16 b 22.55±1.62 d 13.4±0. 2 c 2.41±1.41 c control 2.02±0.06 a 74.65±1.06 c 19.9±0. 9 b 5.16±0.04 cb cookie4% 2.04±0.06 a 102.90±1.27 b 22.7±1.1 b 8.91±2.30 b cookie8% 2.16±0.01 a 139.60±4.8 a 40.8±1. 2 a 21.34±0.92 a cookie – 12% the different letters in each column indicate a significant difference (p<0.05) between the samples. 3.7. fiber (dietary and nondietary fiber) content according to the definition of aacc, dietary fiber is a nutrition unit of the plant, similar to carbohydrates, that is resistant to digestion and absorption in the small intestine of humans, but it can be fermented completely or partially in the colon. fiber composed of a large amount of cellulose, hemicellulose, and lignin is insoluble (non-dietary), while fibers containing high levels of polyfructose and pectin, are soluble (dietary) (elleuch et al., 2008; o'shea et al., 2012). the results of the analysis of the dietary and non-dietary fiber content of the samples are shown in table 3. moreover, based on the results of the analysis of variance, the addition of the banana peel powder has a significant effect on the amount of dietary and non-dietary fiber samples, so that the sample containing 12% banana peel powder has the highest amount of dietary and non-dietary fiber. ijahp article: noshad, asgharipour/application of the analytic hierarchy method to optimize the choice of a gluten-free cookie formula fortified with banana peel powder international journal of the analytic hierarchy process 363 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.618 table 3 influence of the banana peel powder on dietary and non-dietary fiber of samples crude fiber (g/100g sample) dietary fiber (g/100g sample) non-dietary fiber (g/100g sample) treatment 0.51±0.05 d 0.29±0.03 d 0.56±0.02 d control 0.74±0.01 c 0.47±0.02 c 0.78±0.02 c cookie4% 1.06±0.01 b 0.76±0.04 b 1.15±0.04 b cookie8% 1.42±0.02 a 0.99±0.04 a 1.50±0.04 a cookie12% the different letters in each column indicate a significant difference (p<0.05) between the samples 3.8. best choice of produced sample in the hierarchy analysis, there is a possibility of an inconsistency in the judgments. therefore, a measure called the "inconsistency rate" is used to determine the degree of inconsistency of the judgments. these coefficients determine the magnitude of the probability that the pairwise matrix is completely randomly filled. an incompatibility percentage of 0.1 is the acceptable maximum (prevc, šegatin, ulrih, & cigić, 2013; saaty, 2008; sardh et al., 2017). therefore, in order to evaluate and prioritize the four products, according to the hierarchical analysis based on the two physical criteria (with sub-criteria texture, aroma, taste, color and porosity) and a chemical measure (with the sub-criteria fiber content, antioxidant activity, tpc, moisture content and ash content), the results showed that the cookie that contained 4% banana peel powder with the weight of 0.178 was optimal when compared to the other products. moreover, the results showed that the rate of inconsistency was 0.77, and since it was less than 0.1, the results are highly reliable (figure 2). ijahp article: noshad, asgharipour/application of the analytic hierarchy method to optimize the choice of a gluten-free cookie formula fortified with banana peel powder international journal of the analytic hierarchy process 364 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.618 figure 2 prioritization of the formulation of gluten-free cookies 4. conclusion when considering the several factors affecting the quality of gluten-free cookies, choosing the optimal formula based on multi-criteria decision making is complex and not easy. therefore, in this study the analytic hierarchy process (ahp) method was used to optimize the best formula for gluten-free cookies containing banana peel powder. for this purpose, physicochemical aspects such as texture, aroma, taste, color and porosity, fiber content, antioxidant activity, phenolic compounds, moisture content and ash content were evaluated to determine the optimal formula of gluten-free cookies containing banana peel powder. the results showed that the cookie that contained 4% banana peel powder with the weight of 0.178 was optimal when compared to the other products. moreover, the results showed that the rate of inconsistency was 0.77, and since it was less than 0.1 the results are highly reliable. ijahp article: noshad, asgharipour/application of the analytic hierarchy method to optimize the choice of a gluten-free cookie formula fortified with banana peel powder international journal of the analytic hierarchy process 365 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.618 references baliga, m. s., baliga, b. r. v., kandathil, s. m., bhat, h. p. & vayalil, p. k. (2011). a review of the chemistry and pharmacology of the date fruits (phoenix dactylifera l.). food research international, 44(7), 1812-1822. doi: https://doi.org/10.1016/j.foodres.2010.07.004 benvenuti, s., pellati, f., melegari, m. & bertelli, d. (2004). polyphenols, anthocyanins, ascorbic acid, and radical scavenging activity of rubus, ribes, and aronia. journal of food science, 69(3), 164-169. doi: https://doi.org/10.1111/j.1365-2621.2004.tb13352.x cevoli, c., balestra, f., ragni, l. & fabbri, a. (2013). rheological characterisation of selected food hydrocolloids by traditional and simplified techniques. food hydrocolloids, 33(1), 142-150. doi: https://doi.org/10.1016/j.foodhyd.2013.02.022 chaiya, b. & pongsawatmanit, r. (2011). quality of batter and sponge cake prepared from wheat-tapioca flour blends. kasetsart journal (natural sciences), 45(45), 305-313. choo, c. l. & aziz, n. a. a. (2010). effects of banana flour and β-glucan on the nutritional and sensory evaluation of noodles. food chemistry, 119(1), 34-40. doi: https://doi.org/10.1016/j.foodchem.2009.05.004 chua, k., mujumdar, a., hawlader, m., chou, s. & ho, j. (2001). batch drying of banana pieces—effect of stepwise change in drying air temperature on drying kinetics and product colour. food research international, 34(8), 721-731. doi: https://doi.org/10.1016/s0963-9969(01)00094-1 elleuch, m., besbes, s., roiseux, o., blecker, c., deroanne, c., drira, n.-e. & attia, h. (2008). date flesh: chemical composition and characteristics of the dietary fibre. food cchemistry, 111(3), 676-682. doi: https://doi.org/10.1016/j.foodchem.2008.04.036 garcia-amezquita, l. e., tejada-ortigoza, v., heredia-olea, e., serna-saldívar, s. o., & welti-chanes, j. (2018). differences in the dietary fiber content of fruits and their byproducts quantified by conventional and integrated aoac official methodologies. journal of food composition and analysis, 67, 77-85. doi: https://doi.org/10.1016/j.jfca.2018.01.004 giuberti, g., marti, a., fortunati, p. & gallo, a. (2017). gluten free rice cookies with resistant starch ingredients from modified waxy rice starches: nutritional aspects and textural characteristics. journal of cereal science, 76, 157-164. doi: https://doi.org/10.1016/j.jcs.2017.06.008 gonzález-montelongo, r., lobo, m. g. & gonzález, m. (2010). the effect of extraction temperature, time and number of steps on the antioxidant capacity of methanolic banana peel extracts. separation and purification technology, 71(3), 347-355. doi: https://doi.org/10.1016/j.seppur.2009.12.022 ijahp article: noshad, asgharipour/application of the analytic hierarchy method to optimize the choice of a gluten-free cookie formula fortified with banana peel powder international journal of the analytic hierarchy process 366 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.618 ho, w. & ma, x. (2017). the state-of-the-art integrations and applications of the analytic hierarchy process. european journal of operational research, 267(2), 399-414. doi: https://doi.org/10.1016/j.ejor.2017.09.007 jacob, j. & leelavathi, k. (2007). effect of fat-type on cookie dough and cookie quality. journal of food engineering, 79(1), 299-305. doi: https://doi.org/10.1016/j.jfoodeng.2006.01.058 jan, k. n., panesar, p. & singh, s. (2018). optimization of antioxidant activity, textural and sensory characteristics of gluten-free cookies made from whole indian quinoa flour. lwt, 93, 573-582. doi: https://doi.org/10.1016/j.lwt.2018.04.013 kethireddipalli, p., hung, y. c., phillips, r. & mcwatters, k. (2002). evaluating the role of cell wall material and soluble protein in the functionality of cowpea (vigna unguiculata) pastes. journal of food science, 67(1), 53-59. doi: https://doi.org/10.1111/j.1365-2621.2002.tb11358.x lazaridou, a., duta, d., papageorgiou, m., belc, n. & biliaderis, c. g. (2007). effects of hydrocolloids on dough rheology and bread quality parameters in gluten-free formulations. journal of food engineering, 79(3), 1033-1047. doi: https://doi.org/10.1016/j.jfoodeng.2006.03.032 lebesi, d. m. & tzia, c. (2011). effect of the addition of different dietary fiber and edible cereal bran sources on the baking and sensory characteristics of cupcakes. food and bioprocess technology, 4(5), 710-722. doi: https://doi.org/10.1007/s11947-0090181-3 molinari, r., costantini, l., timperio, a. m., lelli, v., bonafaccia, f., bonafaccia, g. & merendino, n. (2017). tartary buckwheat malt as ingredient of gluten-free cookies. journal of cereal science, 165, 232-240. doi: https://doi.org/10.1016/j.jcs.2017.11.011 o'shea, n., arendt, e. k. & gallagher, e. (2012). dietary fibre and phytochemical characteristics of fruit and vegetable by-products and their recent applications as novel ingredients in food products. innovative food science & emerging technologies, 16, 110. doi: https://doi.org/10.1016/j.ifset.2012.06.002 ortiz, l., dorta, e., lobo, m. g., gonzález-mendoza, l. a., díaz, c. & gonzález, m. (2017). use of banana (musa acuminata colla aaa) peel extract as an antioxidant source in orange juices. plant foods for human nutrition, 72(1), 60-66. doi: https://doi.org/10.1007/s11130-016-0591-0 popov-raljić, j. v., mastilović, j. s., laličić-petronijević, j. g., kevrešan, ž. s. & demin, m. a. (2013). sensory and color properties of dietary cookies with different fiber sources during 180 days of storage. hemijska industrija, 67(1), 123. doi: https://doi.org/10.2298/hemind120327047p ijahp article: noshad, asgharipour/application of the analytic hierarchy method to optimize the choice of a gluten-free cookie formula fortified with banana peel powder international journal of the analytic hierarchy process 367 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.618 prachayawarakorn, s., tia, w., plyto, n. & soponronnarit, s. (2008). drying kinetics and quality attributes of low-fat banana slices dried at high temperature. journal of food engineering, 85(4), 509-517. doi: https://doi.org/10.1016/j.jfoodeng.2007.08.011 prevc, t., šegatin, n., ulrih, n. p. & cigić, b. (2013). dpph assay of vegetable oils and model antioxidants in protic and aprotic solvents. talanta, 109, 13-19. doi: https://doi.org/10.1016/j.talanta.2013.03.046 saaty, t. l. (2008). decision making with the analytic hierarchy process. international journal of services sciences, 1(1), 83-98. sardh, e., gouya, l., bloomer, j., balwani, m., bissell, d., rees, d. c. & langendonk, j. (2017). explore: a prospective, multinational natural history study of patients with acute hepatic porphyrias (ahp) with recurrent attacks: american society of hematology. doi: https://doi.org/10.1016/j.jval.2018.04.856 simons, c. w. & hall iii, c. (2018). consumer acceptability of gluten‐free cookies containing raw cooked and germinated pinto bean flours. food science & nutrition, 6(1), 77-84. doi: https://doi.org/10.1002/fsn3.531 singleton, v. l., orthofer, r. & lamuela-raventós, r. m. (1999). analysis of total phenols and other oxidation substrates and antioxidants by means of folin-ciocalteu reagent methods in enzymology, 299, 152-178. doi: https://doi.org/10.1016/s00766879(99)99017-1 someya, s., yoshiki, y. & okubo, k. (2002). antioxidant compounds from bananas (musa cavendish). food chemistry, 79(3), 351-354. doi: https://doi.org/10.1016/s03088146(02)00186-3 sudha, m., baskaran, v. & leelavathi, k. (2007). apple pomace as a source of dietary fiber and polyphenols and its effect on the rheological characteristics and cake making. food chemistry, 104(2), 686-692. doi: https://doi.org/10.1016/j.foodchem.2006.12.016 tibolla, h., pelissari, f. m., martins, j. t., vicente, a. & menegalli, f. c. (2018). cellulose nanofibers produced from banana peel by chemical and mechanical treatments: characterization and cytotoxicity assessment. food hydrocolloids, 75, 192-201. doi: https://doi.org/10.1016/j.foodhyd.2017.08.027 turabi, e., sumnu, g. & sahin, s. (2010). quantitative analysis of macro and microstructure of gluten-free rice cakes containing different types of gums baked in different ovens. food hydrocolloids, 24(8), 755-762. doi: https://doi.org/10.1016/j.foodhyd.2010.04.001 zhang, p., whistler, r. l., bemiller, j. n. & hamaker, b. r. (2005). banana starch: production, physicochemical properties, and digestibility—a review. carbohydrate polymers, 59(4), 443-458. doi: https://doi.org/10.1016/j.carbpol.2004.10.014 ijahp article : aragonés-beltrán, garcía-melón/ analysis of the participation of stakeholders in environmental management based on anp: application to a spanish natural park international journal of the analytic hierarchy process 2 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.276 analysis of the participation of stakeholders in environmental management based on anp: application to a spanish natural park pablo aragonés-beltrán ingenio (csic-upv) universitat politècnica de valència camino de vera s/n, 46022 valencia, spain. email: aragones@dpi.upv.es mónica garcía-melón ingenio (csic-upv) universitat politècnica de valència camino de vera s/n, 46022 valencia, spain. e-mail: mgarciam@dpi.upv.es vicent estruch-guitart department of economy and social sciences universitat politècnica de valència camino de vera s/n, 46022, valencia, spain. vestruch@esp.upv.es abstract the influence of the participation of stakeholders in environmental planning is a real problem that has not been fully tackled in the existing literature. the influences exerted in the decision making process by different stakeholders, who want to satisfy their own interests, need to be analyzed in order to make a model of the problem that is closer to reality. the aim of this paper is to analyze these influences in a specific environmental problem, namely rice straw management in the natural park of la albufera, valencia (spain), by means of the anp methodology. the main question we explored was how to measure the influence among stakeholders. this a complex question because it is difficult for a stakeholder to answer the direct question, “who do you think exerts more influence on you when you have to solve a problem?” in this work, we assumed that information exchange is the tool to measure the influence among the individuals in the network. these data are used to solve the anp model. the final aim was to prove the utility of anp to measure the influences among stakeholders in a social network. keywords: analytic network process; stakeholder analysis, participatory decision making. mailto:mgarciam@dpi.upv.es ijahp article : aragonés-beltrán, garcía-melón/ analysis of the participation of stakeholders in environmental management based on anp: application to a spanish natural park international journal of the analytic hierarchy process 3 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.276 1. introduction due to the complexity and interrelations of the problems caused by global society (economic development, natural resource management, global warming etc.) public policy managers must conduct stakeholder analyses in order to identify and take into account the individuals, groups and organizations involved in or affected by policies (bryson, 2004). the present work analyzed the influences among stakeholders involved in a specific environmental problem, namely rice straw management in the natural park of la albufera, valencia (spain). the case study included different entities that have been identified as being stakeholders for many years. the identification of stakeholders was not within the scope of our study, but rather analyzing (and quantifying) which stakeholders have more influence on the decision-making process. this case study was presented at the isahp 2013 in which we used social network analysis (sna) to identify and quantify the influence between the stakeholders combined with ahp to assign different weights to the stakeholders when using group ahp (garcía-melón, estruch-guitart, aragonés-beltrán, & monterde-roca, 2013). in this work, we used the anp, instead of sna, to measure the influences among the stakeholders of this network. this was not an easy task because it is difficult to directly ask the following question to each stakeholder: “who do you think exerts more influence on you when you have to solve a problem?” therefore, we needed to use indirect questions and chose to use the information exchange flow as a way to measure the influence among individuals of the network. this model is a first approach to prove the utility of anp to measure the influences between stakeholders in a social network. our case study focused on solving a participatory decision making problem in the management of the natural park la albufera (valencia, spain). the albufera natural park, with a surface area of 21,000 hectares, is located 9 km to the south of the city of valencia (figure1). figure 1. map of la albufera ijahp article : aragonés-beltrán, garcía-melón/ analysis of the participation of stakeholders in environmental management based on anp: application to a spanish natural park international journal of the analytic hierarchy process 4 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.276 in this park, like in most mediterranean wetlands, agriculture and particularly rice farming has altered (for more than two centuries) the aquatic ecosystems. the current ecosystem cannot be understood without rice farming because it is the basis of the food chain and in summer, the flooding of its 15,000 hectares has become an alternative habitat for the living fauna. at present, rice straw, which was formerly used as a crop product, has no economic value. straw burning is the most economical disposal practice, though it affects the surrounding population. nowadays, agri-environmental regulations prohibit this practice, except under special conditions. different stakeholders are involved in the search for solutions which would satisfy their own interests. 2. literature review agricultural activity, as any other human activity, generates an environmental impact. agricultural techniques that can be adopted to minimize impacts depend on how the proposed solutions affect the interests of the stakeholders and on their power or influence over other actors. the interactions between the different actors and their flow of information are critical for the adoption of sustainable and innovative practices (isaac, 2012). due to the complexity and interrelations of the problems caused by global society (economic development, natural resource management, global warming, and so on) public policy managers must conduct a stakeholder analysis to identify and take account of the individuals , groups and organizations involved in or affected by policies (bryson, 2004). the purpose of a stakeholder analysis is to understand stakeholder behavior, expectations, relationships and influences or resources they can bring to the decisionmaking process (brugha & varvasovszky, 2000). prell et al. (2009) suggest that the increasing use of stakeholder analysis in natural resources management reflects the recognition that stakeholders can and should influence environmental decision-making processes (prell, hubacek, & reed, 2009). this approach is being promoted by the eu (directive 2003/35/ec2)(european community, 2003); one of its objectives is to promote the participation of different actors (mainly at the level of citizenship) in the development of certain environmental action plans and programs. for the netherlands environmental assessment agency (pbl), stakeholders' participation is important in generating different forms of knowledge that support political decisionmaking processes (hage, leroy, & petersen, 2010). in the u.s. the environmental protection agency and other agencies promote citizen participation in environmental decision-making processes, and have programs that evaluate their participation (charnley & engelbert, 2005). this is especially important for the implementation of environmental conservation action plans in sensitive areas with conflicting land use (mushove & vogel, 2005). however, some studies show the difficulties and challenges of enhancing the participation of stakeholders in the management and conservation of the sites (apostolopoulou, drakou, & pediaditi, 2012), (larson, de freitas, & hicks, 2013), (campo, bousquet, & villanueva, 2010), (davies & white, 2012). a review of different participatory approaches for environmental management can be found in reed (2008), and a review of the origins and justification for stakeholder analysis can be found in reed et al. (2009). ijahp article : aragonés-beltrán, garcía-melón/ analysis of the participation of stakeholders in environmental management based on anp: application to a spanish natural park international journal of the analytic hierarchy process 5 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.276 the participation of stakeholders in environmental management planning and decision making is a real problem that has not been fully resolved, although there are different approaches and techniques which address specific problems (janssen, goosen, & omtzigt, 2006) (goosen, janssen, & vermaat, 2007) (elgin & weible, 2013), (o’toole, keneley, & coffey, 2013). several approaches have been proposed to investigate the relationships among stakeholders, like the basic stakeholder analysis technique, power versus interest grids, stakeholder influence diagrams, the participation planning matrix, interrelationship diagrams (bryson, 2004), or actor-linkage matrices (biggs & matsaert, 1999). however, these techniques do not allow for determining an individual value of the influence of each actor in a decision-making process. 3. hypotheses/objectives the main objective of this work was “to apply anp to analyze and measure the influences among stakeholders in a social network”. as far as we know, anp has never been applied to analyze the influence between stakeholders in a social network. this is not an easy task because different ways to model the network can be established. the main problem we faced was how to define the concept “influence” in a specific network. in this research we developed an easy anp model in which 8 individuals of a social network were the elements of the network. as there are not many stakeholders, there are not any clusters. 4. research design/methodology in this paper we used the anp methodology to analyze a particular group of stakeholders within a case study that are institutions and organizations. the relationships between the stakeholders are analyzed using the information flow among them. the steps followed in the methodology are shown in figure 2: figure 2. methodology 1.selection of relevant stakeholders 2.definition of the influence concept among stakeholders on the specific network 3.design the questionnaire to obtain the influence matrix 4.network modeling with anp 5.results analysis ijahp article : aragonés-beltrán, garcía-melón/ analysis of the participation of stakeholders in environmental management based on anp: application to a spanish natural park international journal of the analytic hierarchy process 6 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.276 5. data/model analysis 5.1 step 1: selection of relevant stakeholders the institutions and organizations involved in the problem are identified as having responsibilities in managing the park or as interest groups. they are: 1. technical management office of the park (tmo) 2. environment department (regional ministry) (denv) 3. department of agriculture (regional ministry) (dagr) 4. technical staff of valencia city council (vlc) 5. la unio farmers’ union (unio) 6. seo birdlife (seo) 7. la albufera fishermen's association (fish) 8. hunters society (hunt) the different stakeholders involved in the problem have conflicting interests. valencia city council is the owner of the lake and is therefore interested in keeping it in the best possible environmental condition. two other stakeholders are regional ministries: the department of agriculture, which is responsible for the development and management of environmental policies (and decides on which farming techniques are subsidized or banned), and the department of environment, who is the actual politician responsible for the park. the politician is ultimately responsible for environmental policies. the management office of the park is responsible for monitoring the conditions and activities developed in the park. the interest of the farmers' union is to reduce farming costs as much as possible. the last two stakeholders are one representative of the fishermen's association, which is affected by the increasing mortality of fish due to rice-straw contamination, and a representative of seo birdlife, which is an association for environmental defense that is very popular in the area. 5.2 step 2: definition of the influence concept among stakeholders on the specific network the definition of the influence concept among stakeholders on the specific network is a critical point of the method. the main question that arises is how to "measure" the influence that some individuals have on others in the social network when solving a particular problem. anp is a method designed to solve complex decision making problems. it is a theory of relative measurement of decision maker's preferences (saaty, 2005). in this work we tried to extend this idea to measure the influence between individuals in a network. there is usually an individual or a group acting as decision maker in any given decision-making problem. however, in a social network, the first problem is usually the identification of the most influential individuals or groups so that a group of experts or a decision making group can be created. therefore, in our point of view, the main difficulty we found when applying anp to this problem was what questions to ask the members of the network in order to determine their influence in solving a particular problem. following the social network analysis (sna) performed in the above mentioned study, we decided to measure influences by using the information flow among the stakeholders. the traditional question used in sna is: who requests information from you regarding ijahp article : aragonés-beltrán, garcía-melón/ analysis of the participation of stakeholders in environmental management based on anp: application to a spanish natural park international journal of the analytic hierarchy process 7 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.276 this problem? for a more robust analysis, we decided to obtain this information from the sender and the receiver of the information. we analyzed two kinds of information flows, asking the following questions: 1. question q1: regarding the consequences of the different rice straw management methods, who do you ask for information and how often? (active). in this case we assumed that if the individual “a” requires information from individual “b” it is because “b” will influence “a”. the main idea is that if i ask someone for information, then presumably this information will influence me. we assumed that the one asking for information is making an effort and thus he will better remember whom he asked. 2. question q2: regarding the consequences of the different rice straw management methods, who asks you for information and how often? (passive). in this case, we assume that if individual “a” provides information to individual “b” it is because “a” influences “b”. in this case, the receiver of this demand for information has to make less effort than in the previous case. therefore, he may not remember exactly who has asked for the information. when these two questions were designed we assumed that the answers to the questions would match, because if a says he has asked b for information, it is logical to think that b will say that a has asked him for information. however, in our case study, we were surprised that in many cases the answers to both questions did not match. namely, a said he had asked for information from b, but b did not recognize that a asked him. this was because we worked with institutions and not with individuals, and therefore the sender and receiver of information within an organization was not always the same person and it is impossible for all members of an organization to know the contacts of all other staff members. for this reason we studied the influence in both cases. 5.3 step 3: design the questionnaire to obtain the influence matrix each stakeholder answered the two questions presented in section 5.2. table 1 information questionnaire active search: question 1 q1. regarding the consequences of the different rice straw management methods, who do you ask for information? how often? o once a year o monthly o once every six months o weekly o once every three months o daily table 2 information questionnaire passive search: question 2 q2. regarding the consequences of the different rice straw management methods, who asks you for information? how often? o once a year o monthly o once every six months o weekly o once every three months o daily ijahp article : aragonés-beltrán, garcía-melón/ analysis of the participation of stakeholders in environmental management based on anp: application to a spanish natural park international journal of the analytic hierarchy process 8 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.276 5.4 steps 4 & 5: network modeling with anp and analysis of results with the information collected from the stakeholders, we designed two similar anp models, one for each question. each model is a network with a single cluster whose elements are each of the stakeholders. 5.4.1 first question model the relations between the elements of this network were established from the answers given by the stakeholders to the first question. question 1 asked, “who do you ask for information (active)”. table 3 is built by columns. for example, vlc asked tmo, dagr, unio, fish and hunter for information, so we consider these stakeholders exert influence on vlc. in the column vlc we show the frequency (weekly, monthly etc.) this stakeholder reported that he has requested information from the other stakeholders. in order to quantify the intensity of the influence on a particular stakeholder, for example vlc, a pairwise comparison matrix was constructed among the stakeholders that influence him (tmo, dagr, unio, fish and hunter), analyzing the flow of information (yearly, every six months, monthly, and so on) that each stakeholder provides regarding the other. figure 3 shows the judgments and influences calculated for the vlc node using superdecisions software. table 3 shows the unweighted supermatrix that was obtained. table 3 influence matrix question 1, “who do you ask for information?” figure 3. influence of stakeholders on vlc (model q1) 1. tmo 2. denv 3. dagr 4.vlc 5. unio 6. seo 7. fish 8. hunt 1. tmo 0 monthly year monthly year monthly monthly 6 month 2. denv weekly 0 year 0 6 month monthly year 0 3. dagr weekly monthly 0 year 4 month monthly year 0 4.vlc weekly monthly 0 0 0 year monthly monthly 5. unio weekly weekly monthly 6 month 0 monthly 0 0 6. seo monthly 0 0 monthly monthly 0 year 0 7. fish monthly year year year 0 year 0 6 month 8. hunt 0 year year 0 year 0 year 0 ijahp article : aragonés-beltrán, garcía-melón/ analysis of the participation of stakeholders in environmental management based on anp: application to a spanish natural park international journal of the analytic hierarchy process 9 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.276 table 4 weighted supermatrix for question 1 from the weighted matrix the results shown in figure 4 were obtained. figure 4. results question 1 these results show that the most influential stakeholders in this model are: unio (19,8%), tmo (16,5), dagr (16,3%) and seo (16,0 %). these stakeholders were reported to be the most frequently consulted about the problem. 5.4.2 second question model the relations between the elements of this network were established from the answers of the stakeholders to the second question. question 2 asked, “who asks information from you and how often? (passive)”. table 5 is built by rows being aij = (frequency) if the element in the column has requested information from the element in the row, and aij =0 otherwise. for example, vlc is asked for information from tmo, dagr, unio, seo and fish, so we consider vlc exerts influence on these stakeholders. 1. tmo 2. denv 3. dagr 4.vlc 5. unio 6. seo 7. fish 8. hunt 1. tmo 0,000 0,284 0,091 0,397 0,055 0,233 0,389 0,143 2. denv 0,214 0,000 0,091 0,000 0,130 0,233 0,056 0,000 3. dagr 0,214 0,284 0,000 0,048 0,246 0,233 0,056 0,000 4.vlc 0,214 0,284 0,000 0,000 0,000 0,033 0,389 0,714 5. unio 0,214 0,076 0,636 0,110 0,000 0,233 0,000 0,000 6. seo 0,071 0,000 0,000 0,397 0,502 0,000 0,056 0,000 7. fish 0,071 0,036 0,091 0,048 0,000 0,033 0,000 0,143 8. hunt 0,000 0,036 0,091 0,000 0,067 0,000 0,056 0,000 0,165 0,116 0,163 0,117 0,198 0,160 0,047 0,035 0,000 0,050 0,100 0,150 0,200 0,250 1. tmo 2. denv 3. dagr 4.vlc 5. unio 6. seo 7. fish 8. hunt results q1 ijahp article : aragonés-beltrán, garcía-melón/ analysis of the participation of stakeholders in environmental management based on anp: application to a spanish natural park international journal of the analytic hierarchy process 10 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.276 table 5 influence matrix question 2, “who asks information from you? figure 5 shows the influence of stakeholders on vlc with this model q2. the weighted supermatrix was obtained in the same way as with the question 1 (table 6). figure 5. influence of stakeholders on vlc (model q2) table 6 weighted supermatrix for question 2 from the weighted matrix the results shown in figure 6 were obtained. 1. tmo 2. denv 3. dagr 4.vlc 5. unio 6. seo 7. fish 8. hunt 1. tmo 0 weekly monthly monthly weekly monthly weekly 2 weeks 2. denv monthly 0 monthly monthly weekly weekly year year 3. dagr year year 0 0 monthly 0 year year 4.vlc 6 month 0 year 0 6 month 6 month year 0 5. unio 0 year 4 month 0 0 weekly 0 year 6. seo monthly monthly monthly year monthly 0 year 0 7. fish 4 month year year 2 month 0 year 0 year 8. hunt 0 0 0 year 0 0 year 0 1. tmo 2. denv 3. dagr 4.vlc 5. unio 6. seo 7. fish 8. hunt 1. tmo 0,000 0,550 0,271 0,332 0,273 0,124 0,643 0,667 2. denv 0,362 0,000 0,271 0,352 0,273 0,281 0,071 0,083 3. dagr 0,039 0,050 0,000 0,000 0,091 0,000 0,071 0,083 4.vlc 0,076 0,000 0,034 0,000 0,273 0,281 0,071 0,000 5. unio 0,000 0,050 0,120 0,000 0,000 0,281 0,000 0,083 6. seo 0,362 0,301 0,271 0,045 0,091 0,000 0,071 0,000 7. fish 0,161 0,050 0,034 0,222 0,000 0,033 0,000 0,083 8. hunt 0,000 0,000 0,000 0,049 0,000 0,000 0,071 0,000 ijahp article : aragonés-beltrán, garcía-melón/ analysis of the participation of stakeholders in environmental management based on anp: application to a spanish natural park international journal of the analytic hierarchy process 11 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.276 figure 6. results from question 2 these results show that the most influential stakeholders in this model are: tmo (27,4%), denv (22,6%) and seo (19,4%). these stakeholders are the most frequently consulted about the problem. figure 7. comparison of results 6. limitations the most important limitation in this study was the need to use indirect questions to measure the influences among stakeholders 0,274 0,226 0,035 0,102 0,071 0,194 0,087 0,011 0,000 0,050 0,100 0,150 0,200 0,250 0,300 1. tmo 2. denv 3. dagr 4.vlc 5. unio 6. seo 7. fish 8. hunt results q2 0 .1 6 5 0 .1 1 6 0 .1 6 3 0 .1 1 7 0 .1 9 8 0 .1 6 0 0 .0 4 7 0 .0 3 5 0 .2 7 4 0 .2 2 6 0 .0 3 5 0 .1 0 2 0 .0 7 1 0 .1 9 4 0 .0 8 7 0 .0 1 1 0.000 0.050 0.100 0.150 0.200 0.250 0.300 1. tmo 2. denv 3. dagr 4.vlc 5. unio 6. seo 7. fish 8. hunt comparison of results results q1 results q2 ijahp article : aragonés-beltrán, garcía-melón/ analysis of the participation of stakeholders in environmental management based on anp: application to a spanish natural park international journal of the analytic hierarchy process 12 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.276 7. conclusions the main conclusion obtained in this work is that it is possible to use anp to analyze the influence among stakeholders in a social network. nevertheless, this is not an easy task. the main problem is defining the concept of “influence” when the stakeholders face a problem. social network analysis is based on the study of flows of information. for this reason and as a first approach to the problem, we considered the information exchange as a way to measure the influence among the individuals of a social network. the social network that was studied is small, and the problem we addressed is one of many that exist in the management of the park. this particular problem arises once a year and not every year, however it is source of many conflicts among stakeholders. the results show different influence intensities depending on how the questions about the exchange of information are formulated. in our opinion, the main reasons for these differences are the following:  in this case we worked with institutions, and even though the questionnaire was passed out to certain people, it is very possible that when an individual has requested information from an institution, the person who replied to him was not the same person who filled out the questionnaire.  for an individual, it is easier to remember whom he has asked for information, than who has asked him. asking for information requires effort, and is therefore more memorable, because usually such information is needed to solve the problem. these conclusions were reached after discussions with the stakeholders. this work is a first step for the use of anp. however, in order to effectively use this model, it is essential to analyze the concept of “influence” and to carefully prepare the proper question(s) presented to the stakeholders ijahp article : aragonés-beltrán, garcía-melón/ analysis of the participation of stakeholders in environmental management based on anp: application to a spanish natural park international journal of the analytic hierarchy process 13 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.276 references apostolopoulou, e., drakou, e. g., & pediaditi, k. (2012). participation in the management of greek natura 2000 sites: evidence from a cross-level analysis. journal of environmental management, 113, 308–18. biggs, s., & matsaert, h. (1999). an actor-oriented approach for strengthening research and development capabilities in natural resource systems. public administration and development, 19(3), 231–262. brugha, r., & varvasovszky, z. (2000). stakeholder analysis : a review. health policy and planning, 15(3), 239–246. bryson, j. m. (2004). what to do when stakeholders matter. public management review, 6(1), 21–53. campo, p. c., bousquet, f., & villanueva, t. r. (2010). modelling with stakeholders within a development project. environmental modelling & software, 25(11), 1302–1321. charnley, s., & engelbert, b. (2005). evaluating public participation in environmental decision-making: epa’s superfund community involvement program. journal of environmental management, 77(3), 165–82. davies, a. l., & white, r. m. (2012). collaboration in natural resource governance: reconciling stakeholder expectations in deer management in scotland. journal of environmental management, 112, 160–9. elgin, d., & weible, c. (2013). a stakeholder analysis of colorado climate and energy issues using policy snalytical vapacity and the advocacy coalition framework. review of policy research, 30(1), 114–133. european community (2003). directive 2003/35/ec of the european parliament and of the council of 26 may 2003 providing for public participation in respect of the drawing up of certain plans and programmes relating to the environment and amending with regard to public participation . official journal of the european union, l156, 17–24. garcía-melón, m., estruch-guitart, v., aragonés-beltrán, p., & monterde-roca, b. (2013). social network analysis in participatory environmental decision making. the case of spanish wetland la albufera. in proceedings of the 12th international symposium on the analytic hierarchy process for multicriteria decision making. goosen, h., janssen, r., & vermaat, j. e. (2007). decision support for participatory wetland decision-making. ecological engineering, 30(2), 187–199. hage, m., leroy, p., & petersen, a. c. (2010). stakeholder participation in environmental knowledge production. futures, 42(3), 254–264. ijahp article : aragonés-beltrán, garcía-melón/ analysis of the participation of stakeholders in environmental management based on anp: application to a spanish natural park international journal of the analytic hierarchy process 14 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.276 isaac, m. e. (2012). agricultural information exchange and organizational ties: the effect of network topology on managing agrodiversity. agricultural systems, 109, 9–15. janssen, m. a., goosen, h., & omtzigt, n. (2006). a simple mediation and negotiation support tool for water management in the netherlands. landscape and urban planning, 78(1-2), 71–84. larson, s., de freitas, d. m., & hicks, c. c. (2013). sense of place as a determinant of people’s attitudes towards the environment: implications for natural resources management and planning in the great barrier reef, australia. journal of environmental management, 117, 226–34. mushove, p., & vogel, c. (2005). heads or tails? stakeholder analysis as a tool for conservation area management. global environmental change, 15(3), 184–198. o’toole, k., keneley, m., & coffey, b. (2013). the participatory logic of coastal management under the project state: insights from the estuary entrance management support system (eemss) in victoria, australia. environmental science & policy, 27, 206–214. prell, c., hubacek, k., & reed, m. (2009). stakeholder analysis and social network analysis in natural resource management. society & natural resources, 22(6), 501–518. reed, m. s. (2008). stakeholder participation for environmental management: a literature review. biological conservation, 141(10), 2417–2431. reed, m. s., graves, a., dandy, n., posthumus, h., hubacek, k., morris, j., stringer, l. (2009). who’s in and why? a typology of stakeholder analysis methods for natural resource management. journal of environmental management, 90(5), 1933–1949. saaty, t. l. (2005). the analytic hierarchy and analytic network processes for the measurement of intangible criteria and for decision-making. in j. figueira, s. greco, & m. ehrgott (eds.), multiple criteria decision analysis: state of the art surveys. international series in operations research & management science, (345–405), springer. ijahp news and events: karpak/upcoming conferences of interest to the ahp/anp community international journal of the analytic hierarchy process 1 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.1073 upcoming conferences of interest to the ahp/anp community birsen karpak distinguished professor emeritus youngstown state university, usa bkarpak@ysu.edu there are three upcoming conferences that would be beneficial for the ahp/anp community. i will present them in chronological order. the first conference is “the 23 rd conference of the international federation of operational research societies”, ifors2023 stgo to be held in santiago, chile, july 10-14, 2023. important dates & general information: december 1, 2022: opening of abstract submissions march 15, 2023: due date for abstracts submissions april 25, 2023: early registration deadline march 1, 2023: registration system is open may 10, 2023: final registration deadline for authors in the scientific program, there is a cluster for multiple criteria decision analysis, in which there are already three active sessions: ahp/anp and extensions (birsen karpak) evolutionary multiobjective optimization (ricardo takahashi and elizabeth wanner) new frontiers in multiple criteria decision aiding (josé figueira, salvatore greco, roman slowinski) mailto:bkarpak@ysu.edu ijahp news and events: karpak/upcoming conferences of interest to the ahp/anp community international journal of the analytic hierarchy process 2 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.1073 if you would like to propose any sessions in the mcdm area, contact salvatore greco (salgreco@unict.it). we are especially eager for submissions from our south american ahp/anp friends. second, the informs annual meeting will be held in phoenix, arizona, usa, october 15-18, 2023. phoenix is a popular location with informs members. we are expecting some ahp/anp and extensions scholars to participate in this conference. i am planning to invite our ahp/anp colleagues from all over the world. in the scientific program, there is a cluster for multiple criteria decision analysis, in which there will be at least one ahp/anp invited session: ahp/anp and extensions (organized by birsen karpak and ilker topcu) last, but not least, is the mcdm 2024 conference. the tunisian decision aid society is organizing this international conference on multiple criteria decision making which will be held in hammamet, tunisia, june 2-7, 2024. mailto:salgreco@unict.it https://mcdm2024.org/?page_id=53 ijahp news and events: karpak/upcoming conferences of interest to the ahp/anp community international journal of the analytic hierarchy process 3 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.1073 the conference is an interdisciplinary forum for the presentation of recent developments and applications in the field of decision aid sciences. this conference aims to disseminate recent models and techniques related to multiple criteria decision making through researchers and practitioners from all over the world. there will be rigorous plenary talks by invited speakers as well as contributed talks. a doctoral dissertation session for doctoral students in the early stages of their dissertation research will be organized during the conference, as well as a meet-the-editors session. important dates & general information: september 15, 2023: submission of session proposals february 15, 2024: submission of abstracts march 15, 2024: acceptance notice april 30, 2024: early registration may 10, 2024: preliminary program june 2-7, 2024: conference dates conference preparation is well underway even though the conference is not until june 2024. there will be quite a few ahp/anp invited sessions at this conference. there has always been a large amount of ahp/anp community participation in mcdm bi-annual meetings. we expect similar participation in 2024 as well. please visit the website mcdm 2024 for further details. https://mcdm2024.org/?page_id=53 ijahp article: garuti/reflections on scales from measurements, not measurements from scales international journal of the analytic hierarchy process 349 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.522 reflections on scales from measurements, not measurements from scales claudio garuti fulcrum ingenieria, chile claudiogaruti@fulcrum.cl abstract this essay will discuss the different kinds of scales that are frequently used, and what can and cannot be done with them. we will also explore what absolute numbers are and how to work with them to deal with quantitative and qualitative data. from my point of view, this is the starting point for everything else in the ahp/anp; this knowledge is crucial to understanding why and how ahp was built. indeed, it gives the key for knowing how to deal with the data that we have to face in real-life problems. keywords: scales; measurement; ahp 1. introduction first of all, i would like to interpret what dr. tom saaty meant with the title of his paper, “scales from measurement, not measurement from scales.” (saaty, 2004) with this title, tom pointed out the importance of being careful how scales are used when measuring. there are many different kinds of scales and they present different properties or qualities that one must be aware of. for instance, let us explore the following sentence (holder, 1990): if a is weakly more important than b (3, on saaty’s scale), and b is weakly more important than c (again 3, on saaty’s scale), it implies that a is absolutely more important that c (9, in saaty’s scale)…indeed, the above logic does violence to the normal usage of the english language. (p. 1074) as the reader can see, holder’s question is; how do two weak relations (3) make an extreme one (9)? of course, if we do not realize that saaty’s fundamental scale is an absolute-ratio scale instead of an ordinal scale (or just a semantic and non-continuous scale in holder’s words), and assuming we don’t have a good understanding of what a ratio ijahp article: garuti/reflections on scales from measurements, not measurements from scales international journal of the analytic hierarchy process 350 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.522 scale, an absolute ratio scale and an ordinal scale are, then this confusion of “how is it possible that two weak pair-comparisons make an extreme” can easily arise. if we think that saaty’s fundamental scale is an ordinal scale, then we are in trouble and this sentence is significant. on the other hand, if we know that saaty´s scale is an absolute ratio scale, and we know what that means, then we can easily understand why two “weak” comparisons (moderate in saaty’s scale) can (and must) make an extreme comparison. i think this is one of the things tom was thinking about when he wrote this paper. so, in the end, if we build scales from measurement, we will get scales that have properties that allow them to be used rationally. for instance, a scale where 3 times 3 is 9, not 3 or 4 or 5 as holder would expect (he considered saaty’s scale too short, or too easy to get the extreme or last value of the scale). this issue can be explained using apples too (as tom liked to do). if we have a melon that weighs as much as 3 apples, and a watermelon that weighs as much as 3 melons, then we have a watermelon that weighs as much as 9 apples. by the way, using a very wide scale (an exponential scale for instance) may produce a big loss of precision in the pair comparison process; this happens because the human being is able to compare only homogeneous objects. (homogeneous refers to objects that belong to the same order of magnitude, that is, in a ratio of 1 to 9). one question has always intrigued me about this issue of scales and numbers. why in school (or university), in mathematics classes, does nobody teach about this fundamental issue? do we know what kind of numbers we are using? where do they come from? what is it that you can or cannot do with them? in brief, what are those "absolute" things that we call numbers, and are continuously working with in such familiar ways? saaty said (2004), human beings have been genetically endowed with the talent to compare things, and it does not seem that that talent will leave us soon. we need to make comparisons to survive, since there are no absolutes. the ahp is the way to make comparisons scientifically. with this method we deduce a scale of priorities by comparing objects in terms of their relative dominance with respect to some attribute. what is established here, even if it seems surprising, first comes the metric to perform the measurements then (and only then), comes a unit and a convenient zero to put in the scale. (p. 1) therefore, first we need to build the metric, and then we may build the scale (related to this metric) to be able to use it to make measurements. so, first comes the measurement from which we can later build the scale, and then (and only then), comes the scale of measurement (not inversely). again, in saaty’s words (2004): ijahp article: garuti/reflections on scales from measurements, not measurements from scales international journal of the analytic hierarchy process 351 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.522 traditionally, measurements are associated with an existing physical scale, from which we can assign to any object a number. measurement is a transformation from objects to numbers in a one-to-one correspondence. however, this is not always the only way, particularly when the property that is being measured is an intangible and there are no scales that we can use to make the measurement. in this case, the objects to be measured can be used to establish the different values of the scale that they would receive as a group, not individually. (p. 2) saaty concluded by saying (2004), the relative scales are much more general, since a measurement performed in a physical scale with a unit can be reduced to a relative measurement, but not inversely. when passing from a physical to relative scale we simple discard the information related to the concept of its dimensional unit and its zero, and this process does not affect the resulting relative values or priorities. it is relative measurement that allows us to measure intangibles. if they were tangibles, a scale would probably already exist and it would not be necessary to create a special one to measure them. sometimes relative measurement would “seem” to work against personal system values and understanding, because they are not based on an arbitrary unit to make measurements as happens with physical scales. however, they might be used to give a completely different interpretation about what the measurement means, through a scheme or formula based on understanding in a decision. finally, in the decision making process, the criteria need to be measured in relative terms with respect to superior objectives to obtain their weights, and use them ijahp article: garuti/reflections on scales from measurements, not measurements from scales international journal of the analytic hierarchy process 352 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.522 to weight the priorities of the alternatives. therefore, the construction of scales is an inescapable issue. (p. 2) this essay will show the difference between having a scale before making a measurement and building a scale as part of the measurement process. tom presented a paper about this at the 17 th international mcdm conference held at whistler, british columbia, canada and the contents of that paper is the focus of this article. 2. measurements the measurement, or the result of the act of measuring, has been defined in many different ways: measuring is the process of discovering an attribute’s dimensions, extension, quantity, degree or capacity in an object under observation and then representing these elements in a data language based on qualitative or quantitative terms,” or “measure is the process of assigning to every element from a list of n elements or individuals one and just one from k categories.” or “measure is to know or give a value through a comparison with a known pattern, that is, to apply a metric,” or “a measurement is the result of an objective process the purpose of which is to determine a characteristic number for a specific physical situation,” or ”measurement is any set of rules for assigning numbers to attributes of objects. (saaty 2004, p.2) in spite of this variety of definitions, there is in all of them the notion (perhaps limited and narrow) of what a measure is, for example, measure is a function which assigns a number to an object. in general, measurements are values that belong to a certain kind of scale that characterizes all that measures. frequently, the measurements are represented by numbers, but sometimes they are represented by words. underlying any measure is the idea of “greater than” or “less than”, which in combination with some ordering of the objects gives the possibility of selecting one of the measured objects according to some merit. for instance, “the bigger the number, the better the object will be considered”, or vice versa. when an object is measured using a physical instrument, the assigned number is a multiple or a fraction of the arbitrarily chosen unit. hence, that measure is arbitrary itself and has to be interpreted in terms of its meaning or contribution to the goal in mind. the interpretation of a measurement using a physical scale depends on our physical senses (or our knowledge or experience with that scale) to be interpreted and then correctly used. for a blind person, a centimeter has no more sense than a color, ijahp article: garuti/reflections on scales from measurements, not measurements from scales international journal of the analytic hierarchy process 353 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.522 unless he or she uses their fingers or some other part of the body to internalize it. this shows clearly the magnitude of the dependence of measurement on our body and not just on our rational minds, as our mechanism for interpreting the world. 3. scales there is a clear distinction between reading the numbers of a scale, which is what we usually mean when we say we are measuring something, and the mathematical properties behind the scale. there is confusion in the way we use the concept of scale. on the one hand, we have a clear mathematical definition of a scale, invariant under a transformation, (by the way, this invariant concept which is the core of mathematical definition of scale, is not present in the current definition of scale in google or in any “scales for dummies”), or what we are doing when we measure using a scale (assigning numbers from the scale to the objects). on the other hand, we have the lay use of the word scale, by which we mean one of the many existing scales that human beings have built that are frequently related to some dispositive or physical instrument. they are more properly called physical or objective scales. mathematical definition of a scale: mathematically, a scale is a triplet composed of a set of numbers, a set of objects, and a transformation from the numbers to the objects. the scale also has a more abstract interpretation that only refers to the nature of the numbers and not to the objects, or how the numbers are assigned to the objects. the kind of transformation or the forms to create the numbers admissible for a particular measurement define what is called a scale of measurement for a measurement operation. we have different kinds of transformation and scales described as follows: nominal scale: invariant under a one-to-one correspondence; for example, when a name, or a telephone number is assigned to an object, there is one and only one name and telephone number assigned to the objects in the set. ordinal scale: invariant under monotone transformation, where the numbers order the objects, but the magnitude of those numbers are useful only for defining whether the order is increasing or decreasing; for example, when assigning numbers 1 and 2 to two people to indicate that one is taller than the other, without including more information about their real height. the minor number can be assigned to the taller person or vice versa. interval scale: invariant under a positive linear transformation. for instance, the linear transformation f = (9/5) + 32, which transform readings of temperature from celsius to fahrenheit. notice that it is not possible to add two measures x1 and x2 in an interval scale, because then y1 + y2 = (ax1+b) + (ax2 + b) = a(x1+x2) + 2b, which take the form of (ax3 + 2b) which is not in the form of (ax + b) anymore. however, we can take the average of both readings, because after dividing by 2 we are back to the original form. this is why 10 degrees of temperature plus 15 degrees of temperature does not produce 25 degrees of temperature (at most, its sum can make 15 degrees). proportional scale: invariant under homogenous transformation, y = ax, a>0. an example is a transformation from pounds to kilograms, with the transformation k = 2.2p. the proportion of the weights of two objects is the same and does not depend on whether the measurement was in pounds or kilos. the zero has no correspondence with any measurement of a real object, it is applied only to objects that do not present ijahp article: garuti/reflections on scales from measurements, not measurements from scales international journal of the analytic hierarchy process 354 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.522 the property also, it is not possible to divide by zero, and get back a result we can interpret. we also may note that we can add two measures of the same scale a(x1+x2) = a(x3) which have the same form of a. we can also multiply and divide different readings from the same proportional scales. when dividing two measures of the same proportional scale, the ratio of any two measures within it belongs to a proportional absolute scale. for example, 6kg /3kg = 2. the number “2” belongs to a proportional absolute scale, showing that the object weighing 6kg is double the weight of the object weighing 3kg. the number 2 is an absolute number because it cannot be transformed to any different number. the idea of n “absolute number” gives us the entry to present the following scale. absolute scale: invariant under the identity transformation y = x, (with a = 1, coming from the ratio: ax1/ax2 = 1x3). examples of this scale are the numbers used to count people in a room, and the natural and real numbers (that is, those used to resolve equations). these absolute numbers are defined in terms of correspondence and equivalence classes of one-to-one correspondence (the peano postulates), not in terms of some unit of measure starting from an origin at zero. there are other well-known scales too, like the logarithmic or semi-log scales. the complexity of the scale defines what kind of arithmetic operations can be carried out over the set of numbers in a valid way. none of the absolute scales have a need for a zero or a unit. in spite of the fact that it is said that a proportional scale has an absolute zero, this is only a supposition that makes it easy to work with the scale. nowhere in the mathematical definition of a scale is it stated that it should have a unit and an origin with a zero value. in general, people use the term objective scale when referring to scales that use some type of dispositive measurement (that is, it resolves a situation once and for all), then the measurement made on it is said to be an objective measurement. however, it is significant that not any measure made with a dispositive measurement is an objective measurement. for instance, the likert scale is used when the intention is to measure subjective intensity-of-preference judgments using an objective scale (it is an absolute scale), measuring all the objects one by one. such a scale lacks the definition of a unit and a point of reference for the precise representation of the state of the mind of the person giving the judgments; it is measuring his real understanding about something. but even in the best case, the resulting scale would only be an ordinal scale. to allow the correct manipulation of the measurements and capture the underlying order is a complex problem; cardinal mental measurements are required, not ordinal. this is the origin of a second kind of scale. besides the physical scales, relative scales emerge. relative scales also possess the properties of a mathematical scale, particularly the properties of the absolute mathematical scale. relative scales do not exist objectively out of the time-line and are not valid for any object of the set; they only serve for a given interval of time and for a given set of objects. there are two ways to obtain a relative scale. the first is to normalize the elements from an objective scale by dividing each by their sums. the resulting scale may be described as an absolute relative scale. the other way to obtain a relative absolute scale is very different. the origin of this relative scale is not from an objective scale, though its readings belong to a mathematical scale and, further, are an absolute mathematical scale with relative values. the relative measures are obtained from the ijahp article: garuti/reflections on scales from measurements, not measurements from scales international journal of the analytic hierarchy process 355 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.522 real world too by comparing a pair of objects that share some property and assuming that the smaller member defines the unit value for the property. every pair of objects of some particular set have to be related to each other with a number from the fundamental 1-9 scale of the ahp that indicates how many times larger the dominant member is than the smaller member. then the measures obtained have to be related with real numbers, in such a way as to build or create a relative scale of absolute numbers. so, there are these two ways to make measurements; one through an instrument and making the correspondence in a direct way. and the other through judgments that are then processed to make a correspondence from the members of the set to numbers in an indirect way. the next figure represents the two kinds of scale construction: ijahp article: garuti/reflections on scales from measurements, not measurements from scales international journal of the analytic hierarchy process 356 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.522 figure 1. classification of scales (saaty 2004, p.6) mathematical scale invariant under the identity transformation. there is no specification of a unit or a zero. physical scale built physically before any measurement. determine an objective understanding (its meaning is based on a general agreement). absolute measures can be adjusted to any mathematical scale and have an arbitrary unit that is determined before the measurement and an arbitrary zero, absolute or relative, that is determined before the measurement. the physical measures can always be transformed to relative measures. mental scale built mathematically only for some objects of the set. come from subjective knowledge (its meaning come from a tacit understanding). relative measurements applicable to particular objects only -the relative unit is generated from the measure using the smaller object as the unit. the relative zero, is applied to all the objects not belonging to the set and can be generated after the scale of measurement. to transform this scale to a physical scale requires the knowledge of the dimensional multiplicand “a” that corresponds to a unique physical measurement. special cases always possible ijahp article: garuti/reflections on scales from measurements, not measurements from scales international journal of the analytic hierarchy process 357 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.522 relative deduced scales: from my perspective, the most important part of tom’s paper is relative deduced scales. here is where you may find the power of the paired comparison process and its application in ahp/anp in relative and absolute measurement mode. let´s start from the beginning. there are two ways to obtain relative deduced scales. the first and most well-known is the typical scale deduced by the combination of two or more basic or primitive relative scales. examples of these deduced scales are velocity that comes from the relation of distance over time, pressure as force per square inch, the quantity of alcohol in the blood, the density of some liquids, etc. there is an alternative way to define relative deduced scales. in spite of being widely known in the literature, this approach causes suffering to the critic coming from the old school, who is addicted to the way of measurement used in engineering or affine areas, and reluctant to see the differences between how people think and measure and how a machine of low understanding makes measurements through a mechanism. let me explain through an example related to the height of a person. when comparing two persons, both considered to be very tall, one of them will have a relative value less than the other. the relative value the person would receive in this comparison is one of many possible values; it depends on the height of the other person involved in the comparison. the relative value of the tallest person is not unique, as it would be if we made a direct measurement. as many people know, the ahp uses a fundamental scale of absolute numbers for making comparisons between pairs of objects. using the scale, one assigns a number to answer the question of how many times one object dominates the other with respect to a certain property. the fundamental scale has the following intensities: 1 equal (neither object dominates the other under the property), 3 moderate, 5 strong, 7 very strong, 9 extreme domination (objects should be grouped for pairwise comparison so that only one order of magnitude is required). the fundamental scale is for judgments, in the absence of measurements from an existing scale, in which case the values themselves can be used (by taking their ratio) to make the comparison. these numbers are entered in a reciprocal positive matrix. of course, all these numbers are absolute numbers, whether from judgment, or by taking the ratio of existing measures from some pre-defined scale. moreover, when an absolute scale already exists, the meaning of a measurement using it, its intrinsic value, may or may not reflect our value system (which, by the way, is the basis of, and can be captured by the paired comparison process). let me show an example about the difference between direct data and one possible representation of our value system; the way prices in stores are measured in ratio scales (they have a zero and a unit). anyone can say how many times more expensive one object is than the other by dividing one price by the other. nevertheless, this process does not necessarily represent our perception. imagine that you are buying a cell phone of middle price, about us$500. if you find a phone of similar characteristics for $1,500 you probably will not say it is 3 times (moderate) more expensive, but extremely more expensive and you will assign a 9 (extreme) to that comparison instead of 3. but if you find another cell-phone for us$600, you will say that is a little more expensive and you will probably assign a value between 1 and 2 when comparing with the initial phone. this example shows that our perception or system value is the one that has to be used to interpret numbers, even when it is about ijahp article: garuti/reflections on scales from measurements, not measurements from scales international journal of the analytic hierarchy process 358 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.522 money, to reflect our perceptions. hard data (like the price), in general does not have to be translated directly by a normalization process to relative numbers, unless we explicitly accept that the price or market value exactly reflects our value system. thus, even hard data should first be interpreted through our preferences using the fundamental scale of ahp, and then the relative numbers deduced that more faithfully reflect our value scale for the measured property. the next example considers a case of temperature reading. a temperature scale is a monotone function with unique values that are linearly related to the temperature being measured. there are the absolute scales of temperature kelvin (k), and rankine (ra) where k = 1.8ra, with an absolute zero that represents the total ceasing of molecular activity. both define a proportional scale. there are 3 more temperature scales: the familiar celsius scale (c), the fahrenheit scale (f) and reamur (r), where f = 1.8c+32, and r = 0.8c. all are interval scales. since c = k 273.15, then both scales have the same unit. if we work with kelvin values, then it is possible to produce ratios as we did in the price example. but, if we use the celsius or fahrenheit scale, one cannot simply divide the values since they belong to an interval scale. if ratios are required, we first have to form differences of temperature and divide it by some other difference of temperature. this is because (ax1+b) / (ax2+b) has no numerical interpretation, as happens also with (ax1+b) + (ax2+b) = a(x1+ x2) + 2b, which does not have the form of the original scale (ax+b). however, (ax1+b) (ax2+b) = a(x1x2) = ax3, does have the form of a proportional scale and can be used to make ratios. imagine a typical temperature reading from a cold day of february is 5 degrees. as the temperature rises, it is legitimate to ask how much preferable the difference between 15 and 5 degrees is than the difference between 10 and 5 degrees. the numerical proportion of 2 (10/5) is mathematically correct. but here, the exactness of mathematics does not reflect our real sensation about the warmer situation. one physically feels the difference to be “strongly warmer” so using the number 5 from saaty’s scale represents our judgment better than the “slightly warmer” number 2. the output of a set of paired comparisons is a priority vector. saaty would say it is comprised of relative absolute numbers. its properties correspond to a mathematical scale where it is possible to perform arithmetic operations but it has no unit. the relative scales deduced in this manner always define an absolute proportional scale, since this is the only scale where we can talk about the relationship of one value to another through division. the priority vector is calculated with mathematical rigor. this is done using the concept of order of dominance (from order topology). thus, the value of one paired comparison itself is not relevant, but the set of comparisons as a whole, the relative absolute priority vector, is relevant. its derivation from the set of paired comparisons is based on graph and system theory. this concept of dominance is also strongly related to the consistency/inconsistency measure, and how new information can be linked with old information in a dynamic system without corrupting the data. once it is available for one set of objects, the deduced measure might be used as the value scale for those objects and for other objects that are going to be pairwise compared with the first set. sometimes the measure relating the old objects and the new ones is kept unaltered, and in some other cases the measure changes as new objects come into the original set. the scale is revised and expanded any time it is necessary in order to include the new values that come from recently performed paired comparisons. unlike physical scales, the values of the relative scale (the ijahp article: garuti/reflections on scales from measurements, not measurements from scales international journal of the analytic hierarchy process 359 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.522 priority vector) do not exist before the objects. they appear at the same time the paired comparisons of the objects are finished and transformed into a priority vector using the ahp process of finding the eigenvector associated to the maximum eigenvalue of the reciprocal pairwise comparison matrix. once the priority vector is finished, any object outside of the set of objects has the value zero. if it is ever considered as a candidate to be measured, then it has to be moved into the set of objects and all the previous measures of the priority vector have to be revised. in other words, the deduction of relative measurement is an iterative process, and it depends on the particular objects under consideration for the measurement. the relative mode of measurement: relative measurements are useful for providing priorities, for instance, for issues like beauty or quality of life. these values give information about the relative intensity of beauty or quality of life, not its absolute value taken from some origin or zero value (what is the zero value for beauty?). this is because for intangible properties we don’t have a scale of measurement with a zero and unit before knowing what objects are to be measured. by definition, relative priorities do not relate the measured objects to an absolute pattern (which most of the times doesn’t even exist), but only in relation to the other objects of the set. this is precisely the goal of deducing or building a relative metric. the relative measures obtained were deduced within a specific context and they only have sense inside that context, in the same way that decisions have sense within a given context. the absolute mode of measurement: there is a second way to obtain a priority vector; this is by using the absolute mode of the ahp. in this mode, the alternatives are considered one at the time, though the criteria and sub-criteria are prioritized as usual with paired comparisons. each terminal criterion has its own scale of intensities or gradation levels, these levels have to be pairwise compared to establish their priorities; that is, we deduce the scale of measurement for each specific criterion. then a level of intensity is assigned for each alternative under every terminal criterion. each intensity priority scale is weighted by the global terminal criterion weight and summed for all the terminal criteria. thus, a total priority value for each alternative is determined independently of the rest of the alternatives. this is the normative scheme of the ahp since it requires expert knowledge to define and prioritize the intensities. the final synthesis may or may not differ from the relative measurement mode, this will depend on the degree of relation among the alternatives, the criteria and eventually the scales and the way we represent those relations. the paired comparisons and their respective final priorities (deduced without the use of some tangible measurement tool), give rise to a relative value scale, just like percentages (deduced from tangible measurements), or probabilities (deduced from the frequency of occurrence of some event). relative value scales do not depend on some unique arbitrary unit applied repetitively over the whole set or in some part of it in a linear way (indeed, these deduced scales rarely are linear). the ideal unit of the scale is produced by dividing each value of the scale by its greatest value (maximum or infinite norm). potentially, the measurement of any element could serve as a unit of measure, dividing by its value, thus the unit does not need to be unique and is not used to measure anything else, as happens in the case of measurements with physical scales. however, it is possible to compare any new element with the ideal and assign the proper value, greater or lower than that unit. ijahp article: garuti/reflections on scales from measurements, not measurements from scales international journal of the analytic hierarchy process 360 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.522 in summary, the usual way to make absolute measurements is that we first have to create the scale of measurement, often involving a pattern or arbitrary unit with respect to which all the rest of the measures are related linearly, and then assign a value of the scale to each of the elements that are being measured. on the other hand, with the relative way of measuring presented here, that allows measuring both tangibles and intangibles, we first specify the objects that will be measured, and then we pairwise compare the objects according to some property they have in common, through assigning values from the fundamental scale, and only then do we deduce a common scale of relative values. these are two opposite ways to make a measurement. a relative scale is a triplet of objects that correspond; first to a fundamental scale, second a transformation that assigns numbers from the fundamental scale to a pair of objects, and third a transformation that assigns real numbers to these numbers which come from the fundamental scale. it means an indirect transformation from the initial numbers to the real numbers. these ideas are the very basis on which the ahp has been developed. 4. epilogue i was so lucky to have learned these things from tom in a simple and clear way. in my 25 years spreading, teaching (and applying) ahp/anp in many different universities, in undergraduate and graduate courses, in chile and also in other countries, i have never seen anybody taking care to explain about this fundamental issue, not even in the curricula of scientific careers like engineering, mathematics, or physics. this "hole" in the curricula has always left me intrigued. as a matter of fact, i have a real story about this issue. once while teaching ahp, for an mba course in the federico santa maria university in chile, one student came to me at the end of the lesson to ask a question. i was sitting at my desk, and she leaned over to me and asked me (a little bit perplexed), if i was sure about what i had just said in class. since (in her words), “this is my third graduate program and i have never heard anything like that, and this could change my way of looking at and understanding real life problems.” from my perspective, i do believe that the problem is the knowledge behind the numbers. by the way, i remember having told her, “as far as i know, this is true and moreover is something that we must never forget.” ijahp article: garuti/reflections on scales from measurements, not measurements from scales international journal of the analytic hierarchy process 361 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.522 references holder, r. d. (1990). some comments on the analytic hierarchy process. the journal of the operational research society, 41(11), 1073-1076. doi: https://doi.org/10.1057/jors.1990.167 saaty, t. l. (2004). scales from measurements not measurement from scales! proceedings of the 17 th international mcdm conference, whistler, british columbia, canada. ijahp article: poveda/using multi-criteria decision-making to assess the importance of human capital in meeting the goals and objectives of sustainable development: an application of the analytic hierarchy process international journal of the analytic hierarchy process 1 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1067 using multi-criteria decision-making to assess the importance of human capital in meeting the goals and objectives of sustainable development: an application of the analytic hierarchy process cesar a. poveda, pmp, cpem, p.eng., ph.d. assistant professor school of engineering, mathematics and science robert morris university, pennsylvania poveda@ualberta.ca; poveda@rmu.edu the concept of human capital continues to evolve and it has gained momentum in the process. besides its impact on the economic growth of business and entire societies, human capital has emerged as a critical area of performance in achieving the goals and objectives of sustainable development. following a brief historical background and a description of its evolution, the role of human capital in meeting the goals and objectives of sustainability is examined. a simulated case using a multi-criteria decision-making (mcdm) method known as the analytic hierarchy process (ahp) further explores the importance or relevance of human capital in financial institutions. research demonstrates the evolution of human capital as well as its increasing coordinating and supporting role in achieving social progress, economic growth, and environmental protection. to better manage and assess the role of human capital, we present a set of resources for the identification and selection of criteria and indicators as well as structure and scientificbased options that allow the effective engagement and participation of stakeholders to assess the relative and absolute importance or relevance of human capital within the concept of sustainable development. while challenges remain to fully understand the role of human capital in the sustainability environment, research outcomes demonstrate the progress already made. keywords: sustainability assessment; criteria and indicators; multi-criteria decisionmaking 1. brief overview of human capital evolution: embedding human capital into the concept of sustainability the evolution of the term ‘human capital’ can be discussed using three major eras: early beginnings, mid-twentieth century, and contemporary or modern. the early beginnings era was led by the first economist to theorize about capitalism, adam smith. the midtwentieth century era brought widespread popularity to the term human capital as macroeconomics and nobel prize winners theodore schultz and gary becker became widely known and influential. more contemporary or modern conceptions of human capital make a distinction between different categories (e.g., general human capital, specific skills). ijahp article: poveda/using multi-criteria decision-making to assess the importance of human capital in meeting the goals and objectives of sustainable development: an application of the analytic hierarchy process international journal of the analytic hierarchy process 2 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1067 many economists and non-economists have considered human beings or their skills as capital throughout the evolution of the idea of human capital. most of the well-known names in the history of economic thought neither attempted an evaluation of human capital nor employed the concept for any specific purpose (kiker, 1966). however, the connection between humans and their skills and economic growth and productivity has been recognized since the first attempts to define the term capital. adam smith did not specifically define the term capital, but included the skills and useful abilities of human beings in his category of fixed capital (kiker, 1966). smith’s writings about capital discussed in part ‘the acquired and useful abilities of all the inhabitants or members of society’. according to smith, the concept of “fixed capital” includes machines, land improvements, and buildings but should also consider the acquired and useful abilities of all the inhabitants or members of society. “the acquisition of such talents, by the maintenance of the acquirer during his education, study, or apprenticeship, always costs a real expense, which is a capital fixed and realized, as it were, in his person” (smith, 1776, p. 282). although adam smith laid the foundation for the concept of human capital back in 1776, by including human capacities in the notion of capital stock, the idea of human capital only started to gain significance in the 1950s and 1960s. to highlight the importance of human capital, some have linked education and training with healthy national economies and economic growth (akintoye & adidu, 2008) whereas others have emphasized the critical role of human resources in the wealth of nations and the active role of human beings in building economic, social, and political organizations, and promoting national development (harbison, 1973). friedrich list, johann von thünen, irving fisher, and others recognized the role of incorporating human beings in the concept of human capital; however, the concept was not universally accepted at the time (savvides & stengos, 2008). for instance, alfred marshall considered the idea of including human beings in the concept of capital ‘unrealistic’ because human beings are not marketable. the realization of the economic significance of human capital in the late 1950s and early 1960s was led by gary becker and theodore schultz. schultz argued that the best form of investing in human capital is by educating human beings which results in “increases in national income and the ‘rate of return’ to the decision to invest in education” (savvides & stengos, 2008, p.15). weisbrod (1961) built on schultz’s findings to further the concept of human capital. public health, highway construction and flood control policies, population and immigration policies, and educational and vocational rehabilitation policies were the main reasons that weisbrod formulated a calculation for a capital value for people as productive assets in monetary value (savvides & stengos, 2008). the interest in human capital during the mid-twentieth century era concluded with the publication of a special issue of the journal of political economy. the special issue drew attention to how investment in human capital can provide answers in the following three areas: personal distribution of income, economic growth, and structure of earning. the earliest interpretation of capital given centuries ago included the following four types of fixed capital: useful machinery, profitable buildings, improvement of land, and acquired and useful abilities of all the inhabitants or members of the society (smith, 1776). however, “the rediscovery of human capital in the modern literature is associated with the efforts of analysts to understand the growth over time in labor productivity in the ijahp article: poveda/using multi-criteria decision-making to assess the importance of human capital in meeting the goals and objectives of sustainable development: an application of the analytic hierarchy process international journal of the analytic hierarchy process 3 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1067 u.s. economy” (schuh, 2002, p. 76). nelson and phelps (1966) acknowledged human capital as a growth-generating factor. mankiw et al. (1992) included human capital in neo-classical economic growth models and benhabib and spiegel (1994) considered the concept as part of the productivity growth nexus. the importance of human resource capital (particularly education) as a source of economic growth started to gain prominence in the second half of the twentieth century (tisdell, 2000). previous neoclassical models for economic growth emphasized quantities of human-made capital and labor whereas classical models considered factors such as natural resources and technological progress. instead of attributing economic growth to increases in the quantity of capital and labor, an analysis of the sources of economic growth found that technological progress and education have a central role in improvements in the productive quality of human-made capital and labor (denison, 1962). while education continued to be part of the set of contributors to economic growth, new theories of growth developed years later included other factors such as technical knowledge, economies of agglomeration and scale. the latest evolution of the concept of human capital departs from the relationship between human capital and economic growth and addresses the nexus of economic growth and sustainability. the relationship between human capital and sustainability has only gathered attention and been addressed by scholars in the last few years. more recent studies use the term capital to include all forms of assets and capabilities natural, biological, financial and human that can be connected to and used for sustainable development (slaus & jacobs, 2011). natural, biological, social, technological, financial, and cultural are all forms of capital included in sustainability studies (slaus & jacobs, 2011). furthermore, the complex interconnection and interdependent characteristics of sustainability require a close analysis of the interaction among the various types of capital. farčnick & istenič (2020) identified two lines of research in the area of sustainable production or sustainable economic growth and human capital; the first wellresearched area focuses on human capital and economic growth whereas economic growth and sustainability have been addressed only recently. the problem of sustainability has now evolved to the stage where it endangers not only human life but threatens to undermine the natural capital on which human civilization is based. “the sustainability of human capital is interwoven with the sustainability of all other forms of capital” (slaus & jacobs, 2011, p. 99). the often-cited model of strong and weak sustainability has a distinctive role in human capital. “strong sustainability requires that both natural and human-made capital have to be maintained, while weak sustainability holds that utility of the sum of all capitals has to be maintained for future generations” (slaus & jacobs, 2011, p. 106). ugnich et. al. (2021) indicate that “sustainable development implies a constant growth in the well-being of people in a quality environment.” but, the transition to sustainable development poses serious challenges for all components of the socio-economic structure of society, and a person here appears as a key subject of sustainable development. people must have the knowledge, skills, and tools, and be well-informed as well as feel supported to achieve the expected sustainability vision, goals and objectives. ijahp article: poveda/using multi-criteria decision-making to assess the importance of human capital in meeting the goals and objectives of sustainable development: an application of the analytic hierarchy process international journal of the analytic hierarchy process 4 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1067 2. defining human capital: the challenge of quantifying an evolving concept the first known use of the term human capital can be traced back to 1799 and it is defined as “the skills, knowledge, and qualifications of a person, group, or workforce considered as economic assets” (merriam-webster, n.d.). another formal definition given by the oxford english dictionary indicates that human capital refers to “the skills the labor force possesses and is regarded as a resource or asset.” goldin (2016) adds “it encompasses the notion that there are investments in people (e.g., education, training, health) and that these investments increase an individual’s productivity” (p. 56). although scientific literature often goes back to adam smith to refer to the origins of the term human capital, work in the area of economics points to a formal use of the term by irving fisher in 1897 (goldin, 2016). adam smith identified four types of fixed capital: useful machines and instruments of trade; building as the means of procuring revenue; improvements of land; and the acquired and useful abilities of all the inhabitants or members of the society. the fourth fixed capital refers to smith’s (1776) interpretation of human capital, his writings state: the acquisition of such talents, by the maintenance of the acquirer during his education, study, or apprenticeship, always costs a real expense, which is a capital fixed and realized, as it were, in his person. those talents, as they make a part of his fortune, so do they likewise of that of the society to which he belongs. the improved dexterity of a workman may be considered in the same light as a machine or instrument of trade which facilitates and abridges labour, and which, though it costs a certain expense, repays that expense with a profit. (p. 282) by the second era in the evolution of the term, the mid-twentieth century, a recognizable formal discussion on human capital was provided by arthur cecil pigou. referring to the term human capital, he wrote (1928): there is such a thing as investment in human capital as well as investment in material capital. so soon as this is recognised, the distinction between economy in consumption and economy in investment becomes blurred. for, up to a point, consumption is investment in personal productive capacity. this is especially important in connection with children: to reduce unduly expenditure on their consumption may greatly lower their efficiency in after-life. even for adults, after we have descended a certain distance along the scale of wealth, so that we are beyond the region of luxuries and "unnecessary" comforts, a check to personal consumption is also a check to investment. (p. 29) years later, schultz (1962) provided a historical overview of the term human capital focusing on two major components: education and training. human capital is then “the stock of productive skills, talents, health, and expertise of the labor force, just as physical capital is the stock of plant, equipment, machines, and tools” (goldin, 2016, p. 83). more contemporary definitions of human capital can be found in reputable, reliable and worldwide organizations such as the organization for economic co-operation and development (oecd), the world bank, and the world economic forum: ijahp article: poveda/using multi-criteria decision-making to assess the importance of human capital in meeting the goals and objectives of sustainable development: an application of the analytic hierarchy process international journal of the analytic hierarchy process 5 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1067 human capital can be broadly defined as the stock of knowledge, skills and other personal characteristics embodied in people that helps them to be productive. pursuing formal education (early childhood, formal school system, adult training programmes) but also informal and on-the-job learning and work experience all represent investment in human capital. (oecd, n.d.). human capital consists of the knowledge, skills, and health that people invest in and accumulate throughout their lives, enabling them to realize their potential as productive members of society. investing in people through nutrition, health care, quality education, jobs and skills helps develop human capital, and this is key to ending extreme poverty and creating more inclusive societies (the world bank, 2022). human capital—the capabilities and skills of individuals and populations—is a key driver of economic prosperity and productivity. it can be developed by ensuring individuals are able to sustain good health, and they are in possession of in-demand skills and capabilities. the value of human capital is realized in the labour market through productive employment, and it is developed through education during the first two decades of an individual’s life as well as through mid-career training investments (world economic forum, 2020). the definition of human capital can be adapted to fit narrower views of a project or business. organizations often focus on employee experience and educational achievement to measure human capital and the economic value of their workforce. furthermore, the term human capital is either used in a narrow sense with reference to the innate talents, abilities, skills and acquired knowledge of individual human beings or is broadened to include the entire spectrum of an individual‘s intellectual, physical and psychological abilities (slaus & jacobs, 2011). other definitions of human capital link the assets (i.e., employees) to the economy and economic growth. porreca (2020) refers to human capital as the contribution of the individual worker to the output of an economy, whereas slaus and jacobs (2011) use the term human capital to refer to human beings as one of the means and contributing factors in economic growth and social development. the evolution of the human capital concept has been influenced by contemporary developments such as sustainability. porreca (2020) indicates that economic growth and improvements in social well-being, are tied to the concept of human capital, whereas farčnick and istenič (2020) link human capital to sustainable consumption of production inputs, especially energy, as well as to a decreasing carbon footprint of a specific economic unit/country. moreover, the connection between population and the development of human capital has become evident to the extent that the central determinant of resource productivity and sustainability is human capital including social capital (slaus & jacobs, 2011). nonetheless, “the human factor and its influence on development in general and more particularly on sustainable development can be analyzed from many points of view” (ciuhu, 2016, p. 43). similar to sustainability, the human capital concept is characterized by the constant evolution of its definition (some areas of assessment are highly subjective) and the need for adaptation is embedded in it. all three characteristics are determining factors to measure its relevance (i.e., weight) ijahp article: poveda/using multi-criteria decision-making to assess the importance of human capital in meeting the goals and objectives of sustainable development: an application of the analytic hierarchy process international journal of the analytic hierarchy process 6 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1067 within the concept of sustainable development. “you can’t manage what you can’t measure” or “if you can’t measure it, you can’t improve it” are quotes often used to highlight the criticality of setting measurable and tangible goals and objectives. for sustainability and human capital, the successful design and implementation of goals and objectives is an expansion of the ability to define the concepts. 3. identifying and selecting indicators of human capital to measure progress toward sustainability the use of indicators and composite indices are some of the most widely used instruments to measure, track, manage, and improve sustainability performance. nevertheless, indicators and composite indices must be used with caution because there is an increasing focus on the various factors included in index scores to measure sustainability instead of examining the underlying factors that provide a better understanding of these index scores (porreca, 2020). the starting point of the identification and selection processes of indicators is the definition of the term human capital. the concept of human capital not only continues to evolve but is also directly influenced by the principles of sustainability. based on the principles of sustainable development created by the un commission in 1987, the concept of sustainable development includes the following three main areas of performance also known as pillars or dimensions: natural environment, economy and society. while the definition of sustainable development provided in our common future report also known as the brundtland report has been widely criticized, it still provides a convenient conceptual departing point. throughout the years, countless frameworks along with graphic representations of sustainability have been proposed. figures 1 includes some of the most common graphic representations of sustainability. either by conceptual frameworks or graphic representations of sustainability, the integration of two or more areas of performance (i.e., pillars or dimensions) to capture the different facets of sustainable development is widely recognized and accepted. because human capital is embedded in the social dimension of sustainability, the dimensions included in the concept of sustainable development have an essential role in determining the importance (i.e., weight) of human capital in achieving the desired overall performance (i.e., goal and objectives). farčnick and istenič (2020) concluded that human capital and awareness of its importance play a critical role in achieving the goal and objectives of sustainable development. from its very early beginning, sustainability has included two or more areas of performance (i.e., dimensions or pillars). figure 1(h) illustrates ‘the egg of well-being’ in which people are represented as immersed in the white of ecosystems whereas figure 1(d) illustrates the most common representation of sustainability in a venn diagram in which the social, economic, and environmental pillars are interconnected. while culture and policy are other pillars often found in frameworks and definitions of sustainability, others propose that the concept of sustainable development must include a broader range of economic, ecological, political, technological, and social areas of performance (slaus & jacobs, 2011). human capital has become an asset for organizations to achieve their sustainability goals, objectives and vision. therefore, human capital must be associated with metrics in order to measure and manage progress toward the desired target(s). in the five capitals model ijahp article: poveda/using multi-criteria decision-making to assess the importance of human capital in meeting the goals and objectives of sustainable development: an application of the analytic hierarchy process international journal of the analytic hierarchy process 7 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1067 (fcm) which is graphically represented in figure 1(l), porritt (2005) includes the following five types of sustainable capital: manufactured, financial, social, human, and natural. the fcm considers people’s knowledge, motivation, health, and skills as primary factors for productive work; by nurturing and developing those primary factors, the economy is set to flourish through training and education (porritt, 2005). slaus and jacobs (2011) include areas such as knowledge, skills, attitudes, capacities of individual, social and cultural endowments of the collectives, including the capacity for discovery, invention, innovation and resourcefulness within the concept of human capital. from the assessment standpoint, human capital is difficult to quantify precisely and there is not a fixed set of criteria to analyze human capital; however, some of the most common factors included in human capital are education, knowledge, specific skills, and health. from the environmental, social and governance (esg) standpoint, human capital can be evaluated across areas including well-being, diversity, equity and inclusion (dei), employee experience, and operational excellence. bremen et al. (2021) write: human capital metrics include workforce profile, pay, benefits, careers, hiring, retention, productivity, wellbeing and culture. governance and ethical metrics related to human capital include whistle-blower policies, unethical behavior tied to monetary losses, dismissal and incentives against excessive risk-taking. there are also several human capital management-related esg metrics, including employee productivity, pay gaps, high-performance employee experience, and equitable access to reskilling and upskilling programs. there are quantitative metrics, including pay-equity ratios, diversity and representation targets, the retention rate of top talent, investment in employee upskilling, return on work, and the total cost of work. and there are metrics that cut across categories, such as benefit claims ratio and total workforce value. human capital can be viewed as a stock of knowledge and skills that have a direct impact on how we produce and consume goods. knowledge and skills can be used as instruments for optimizing management, operational, and production processes and can result in the reduction of negative environmental impacts in the form of ghg emissions, exploitation of natural resources, energy and water consumption levels, and waste. porreca (2020) indicates that human capital is the sum of factors related to educational, health, and societal outcomes that individuals experience and bring into the workplace. salim et al. (2017) include various factors to measure human capital such as average years of formal education, the portion of people who hold at least a secondary qualification or a tertiary degree, present value of human capital stock per capita, and the present value of human capital stock per capita in the labor force. qualities within the concept of human capital most commonly found in the literature are education, technical or on-the-job training, health, mental and emotional well-being, punctuality, problemsolving, people management, and communication skills. ijahp article: poveda/using multi-criteria decision-making to assess the importance of human capital in meeting the goals and objectives of sustainable development: an application of the analytic hierarchy process international journal of the analytic hierarchy process 8 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1067 (f) (e) (b) (a) (c) human systems ecological systems society ecology economy culture society ecology economy economy ecology culture polity society environment economy society (d) society ecology economy human capital ijahp article: poveda/using multi-criteria decision-making to assess the importance of human capital in meeting the goals and objectives of sustainable development: an application of the analytic hierarchy process international journal of the analytic hierarchy process 9 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1067 figure 1 graphic representations of sustainability (j) (l) (k) (j) (g) (i) (h) manufactured capital financial capital social capital human capital natural capital ecosystem people e n v ir o n m e n ta l s o c ie ty e c o n o m y ecology society economy culture ecology society economy environment (l) ijahp article: poveda/using multi-criteria decision-making to assess the importance of human capital in meeting the goals and objectives of sustainable development: an application of the analytic hierarchy process international journal of the analytic hierarchy process 10 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1067 sustainability performance is often strongly linked to its environmental dimension but the connection with other evolving and emerging areas of performance is transforming the traditional approaches to sustainable development. for instance, kim and go (2020) found that environmental performance is directly linked to political institutions and social capital, both of which are correlated to human capital. the development of human capital over time is also a function of the quantity and quality of human capital (which includes all forms of social capital), natural capital (e.g., ecosystem, air, water) and human-made capital (e.g., money, infrastructure, buildings, roads) and their evolution (salus & jacobs, 2011). to that extent, the individual’s well-being is a determinant factor in the development and implementation of sustainable development strategies and achieving sustainability goals and objectives. with an individual-focused perspective, becker (1962) made a distinction between specific and general human capital; the former includes training and qualities that benefit the individual at an organization whereas the latter can be associated with education and training that benefits a specific company. similarly, giarini (1980) identified four types of resources within the wide range of human capabilities of human capital: 1) social or organizational resources for governance, commerce, production, and education; 2) mental-intellectual resources such as ideas, knowledge, science, technology, and information; 3) cultural and psychological resources including values, customs, ways of life, character formation, personality development and individuality; and 4) productive resources such as skills and tool. although the scientific literature provides a plethora of definitions and factors, known henceforth as criteria or indicators, to include within the concept of human capital, other resources can provide further support in the early stages of the identification and selection processes of criteria and indicators. 4. resources for the identification and selection of human capital criteria and indicators definitions of sustainability and human capital have merit in their attempt to encapsulate the essence of the concepts but both terms are embedded in vagueness. barlett (2012) determined that the vagueness found in the definitions of sustainability allow it to mean whatever the user wants. furthermore, poveda (2016) indicates that the definition of the term has been subject to interpretation based on the specific interests of something or someone, which has generated confusion and increased the vagueness already embedded in the terminology. while the terminology is widely known, an agreed upon definition has not necessarily been accepted. as a result, the identification and selection of criteria and indicators become a determining factor for scientists and practitioners to properly capture the true meaning of the concept of sustainable development and its elements (e.g., human capital). based on their origin, various resources for the identification and selection of human capital criteria and indicators can be clustered into the following four groups: commercially and research-based developed appraisal instruments; wellestablished, reliable, and reputable organizations, individuals or groups of academics and practitioners; and agreement reached through consensus. while the seven resources included in figure 2 are the departing point for the identification and selection of human capital criteria and indicators, the use of scientific-based processes (e.g., multi-criteria decision-making [mcdm]) assist in refining the selection, validating the final set of ijahp article: poveda/using multi-criteria decision-making to assess the importance of human capital in meeting the goals and objectives of sustainable development: an application of the analytic hierarchy process international journal of the analytic hierarchy process 11 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1067 criteria and indicators, and evaluating their importance (i.e., weight) within the concept of sustainable development. figure 2 resources for the identification and selection of human capital criteria and indicators a. indices, ranking, surveys and others: public and private organizations develop commercial or research-based appraisal instruments using the composite index approach. instruments included in this group of resources typically group criteria under three subprinciples (i.e., dimensions, pillars) to then combine them to calculate the score or value of the principle (i.e., sustainability). although it is rare for these instruments to include a criterion called human capital, their social dimension or pillar typically includes a set of indicators to address the human capital facet of sustainability. for instance, the quality of living, liveability or sustainability performance of cities and communities are commonly assessed, compared, and ranked using instruments such as the sustainable city index, quality of living survey and ranking, and most livable cities index developed by arcadis, mercer, and the martin prosperity institute, respectively. a widely known and implemented set of appraisal instruments using the composite index approach are used in the construction environment. leed, breeam, green globes, casbee, and green star are some of the rating systems used to assess environmental factors and sustainability of a wide range of projects within the construction industry. other instruments have been designed to particularly measure human capital including the world economic forum global human capital index and the world bank human capital index. human capital group originator resources for identification and selection of criterial and indicators available resources for criteria and indicators commercially and research-based developed appraisal instruments well-established, reliable, and reputable organizations individual or groups of academics and practitioners agreement reached through consensus indices, rankings, surveys, and others local, regional, national, and international organizations industry, organization or project specific standards, plans, policies, strategies, initiatives, and programs academically and scientifically authored resources committees and organizations for standardization governmental laws, regulations, policies, and mandates management and processes best practices ijahp article: poveda/using multi-criteria decision-making to assess the importance of human capital in meeting the goals and objectives of sustainable development: an application of the analytic hierarchy process international journal of the analytic hierarchy process 12 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1067 b. local, regional, national, and international organizations: public, private, nongovernmental (ngos), intergovernmental (igos), and many other types of organizations work towards achieving sustainability through the development, implementation, monitoring, and assessment of sustainability indicators frameworks. the united nations (un) is the most important igo supporting sustainable development and climate action through its sustainable development agenda launched in 2015. the sustainable development goals (sdgs) or global goals are part of the 2030 agenda and they emphasize the interconnection and interdependent nature of the various aspects within the concept of sustainable development. on the other hand, sustainable development indicators (sdis) “are to be developed at the appropriate level of detail to ensure proper assessment of the situation with regard to each particular challenge” (european commission, 2009, p. 4). as a result, geographic-specific organizations focus on developing frameworks to meet local, regional, or national sustainability needs instead of using global frameworks. c. industry, organization, or project-specific standards, plans, policies, strategies, initiatives, and programs: the assessment of sustainability and human capital varies based on the type of industry, organization or project because the set of criteria and indicators are a direct reflection of the needs and vision of internal and external stakeholders. the potential of this category as a resource to help in the identification and selection of criteria and indicators can be illustrated in the following examples: 1) sustainability standards developed by the tourism industry for sustainable tour operations, sustainable destinations, sustainable hotels, and sustainable events and conferences among many others; 2) sustainability, green, and climate change plans developed by cities around the world, the greenest city action plan (city of vancouver, canada) and onenyc 2050: new york city’s strategic plan are two examples of these type of plans; 3) common practices include the development of environmental and sustainability policies, esg (environmental, social, and governance) policies, and csr (corporate social responsibility) policies; and 4) environmental and sustainability strategies, initiatives, and programs with elements of the facet of human capital embedded are designed and implemented by industries, organizations, and projects. d. academically and scientifically authored resources: the concept of sustainability has been around for centuries but gained traction in 1987 with the release of our common future. with developments in every facet of sustainable development including human capital, academics and researchers with a wide range of backgrounds are motivated to study the implications of those developments and propose science-based solutions to achieve the vision of stakeholders in relation to sustainability goals and objectives. academics and researchers are constantly proposing frameworks, instruments, methodologies, and many other appraisal approaches to assess and manage sustainability. using theoretical or application-based approaches, academics and researchers provide credible and reliable resources for the identification and selection of sustainability and human capital criteria and indicators. e. management and processes best practices: industries, organizations, or projects aim to implement a set of practices that have proven an acceptable level of success over time. best management practices (bmps) are typically used to set a measurable desired level of operational or management performance. “best practices in management and processes ijahp article: poveda/using multi-criteria decision-making to assess the importance of human capital in meeting the goals and objectives of sustainable development: an application of the analytic hierarchy process international journal of the analytic hierarchy process 13 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1067 may not have deep impacts on the functionality of the organization, as standards or management systems; though this enhances the flexibility characteristic of bmps” (poveda, 2014, p. 522). the best ways of doing something (i.e., performing a task) are captured in processes and procedures that serve as guidelines in the delivery of a product or service. while laws, regulations, policies, and mandates may direct the minimum set of requirements, bmps are the ‘go-to’ set of guidelines for organizations, practitioners, stakeholders, and decision-makers to improve operational and management performance. bmps can be found embedded in other types of resources (e.g., standards) or developed by governmental agencies, private organizations, industry associations, temporary partnerships, focus groups, f. governmental laws, regulations, policies, and mandates: either from self-motivation or as a result of public pressure, local, regional, national, and international governmental bodies have faced the reality of incorporating the need for achieving a more sustainable future into laws, regulations, policies, and mandates. as the concept of sustainability and sustainable development continues to evolve, the interconnection and dependent nature of its dimensions or pillars have put sustainability at the center of political agendas. while all dimensions or pillars must work in harmony, society and social development have become the engine to achieve inclusive and resilient societies where citizens have a voice and governments respond to their sustainability needs. instead of being the finish line, governmental laws, regulations, policies, and mandates are the starting point in the identification and selection of criteria and indicators. to avoid becoming a bureaucratic tool, any set of criteria and indicators proposed by governmental agencies must be carefully analyzed and subsequently complemented with those identified and selected from the other six resources. g. committees and organizations for standardization: standards are repeatable, consistent, agreed upon, and documented processes that are developed by an authority which typically uses general consent as a basis of comparison. standards set a rule for following or measuring a specific process with a level of expected quality that is generally accepted as normal. different committees and organizations around the world are dedicated to identifying and developing the standards to better make products, manage a process, deliver a service, or supply products and materials. because of the need to incorporate sustainability into their standards, committees and organizations for standardization have reviewed and updated standards already published or developed brand new standards to address specific market or industry needs. while the international organization for standardization (iso) and its 167 national standards bodies are the most widely known organization enacting standards, many other committees and organizations for standardization can serve as resources for identifying and selecting criteria and indicators for sustainability and human capital. 5. evaluating the importance of human capital within the concept of sustainable development the venn diagram illustrated in figure 1(d) is the most widely known graphic representation of sustainability. the intersecting circles imply an equal weight (i.e., relevance or importance) for each of the dimensions or pillars. although the equal weight is a mere assumption and matter of interpretation, the relevance or importance of the ijahp article: poveda/using multi-criteria decision-making to assess the importance of human capital in meeting the goals and objectives of sustainable development: an application of the analytic hierarchy process international journal of the analytic hierarchy process 14 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1067 various facets included in other graphic representations and frameworks has a determinant factor in the outcomes of the sustainability assessment process. because capturing the various facets of sustainable development is the main objective in sustainability assessment studies, holistic approaches, strategies, methodologies, models, and appraisals usually include a number of areas of performance within each dimension or pillar. these areas of performance are grouped into the three most common pillars or dimensions: social, economic, and environmental. human capital is an area of performance within the social pillar and its relevance can be determined using the composite index approach with the assistance of mcdm or multi-criteria decision analysis (mcda) methods. 5.1 hierarchical structural organization (hso) in sustainability assessment sustainability assessment studies use a wide range of tools which can be integrated into the following three groups: indicators and indices, product-related assessments, and integrated assessments (ness et al., 2007). the use of indicators has gained popularity because of their simplicity in representing the state of a specific area of performance. indicators can be integrated or non-integrated depending on the need to integrate the different areas of performance. an index, formally known as a composite index, results from the integration of various indicators representing the respective areas of performance. to organize the elements within a composite index and facilitate the assessment and interpretation of results, sustainability assessment studies often use the hso approach. the elements within composite indices follow the structure illustrated in figure 3. the highest level of the hso is the principle. the principle represents a statement of the fundamental desired outcome. the principle of sustainability is often formulated around the core concept of sustainable development. sustainability as a principle is shaped by stakeholders’ vision, values, traditions, needs, and scientific knowledge among other factors. the principle can include a number of sub-principles (composite sub-indices from the sustainability assessment standpoint). the second level of the hso often includes sub-principles aligned with the three dimensions or pillars of sustainability. social, economic, and environmental performances can be measured and presented as composite sub-indices. criteria can be found in the third level of the hso. “criteria are the intermediate points to which the information provided by indicators can be integrated and where an interpretable assessment crystallizes” (pokorny & adams, 2003, p. 20). therefore, criteria add meaning and functionality to the sub-principle without themselves becoming a direct measure of performance. energy, water, air quality, climate, and education are some examples of criteria often included in a sustainability assessment. to evaluate the performance of each criterion, indicators are designed to capture and deliver the corresponding information. therefore, indicators determine whether or not a particular criterion has the desired performance. verifiers are the lowest level of the hso. verifiers collect data, information, or observations used to demonstrate the desired state of performance or reflect the current condition of an indicator. ijahp article: poveda/using multi-criteria decision-making to assess the importance of human capital in meeting the goals and objectives of sustainable development: an application of the analytic hierarchy process international journal of the analytic hierarchy process 15 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1067 hierarchical structural level hierarchical structural organization applied to sustainability with emphasis on the social dimension and human capital principle (composite index) sub-principle (composite sub-index) criterion indicator verifier acronyms c1, c2, … cn = criteria within each sub-principle (i.e., pillar, dimension) i1, i2, ... in = indicators within each criterion (e.g., human capital) v1, v2, … vn = verifiers within each indicator figure 3 hierarchical structural organization (hso) commonly used in sustainability assessment studies ijahp article: poveda/using multi-criteria decision-making to assess the importance of human capital in meeting the goals and objectives of sustainable development: an application of the analytic hierarchy process international journal of the analytic hierarchy process 16 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1067 5.2 using mcdm and mcda to weight the elements within the hso with an absolute weight of 100%, the principle (i.e., sustainability) is the only element within the hso with a fixed weight. although other elements can be equally weighted to facilitate or expedite the assessment process, assigning equal weights to sub-principles, criteria, and indicators most likely does not reflect the accurate importance of the element(s) within the concept of sustainable development. as the sum of the weights of the elements in each level of the hso must add to 100%, the importance or relevance of an element is determined by its weight in relation to the others within the same level of the hso. mcdm and mcda can assist in the selecting, ranking and weighting indicators, criteria, and sub-principles. the main focus of mcdm and mcda is structuring and solving decision and planning problems with multiple criteria. mcdm and mcda have rapidly evolved and been applied to solve problems in a wide range of areas including but not limited to finance, education, transportation, ecology, supply chain, urban sanitation, and economy. mcdm and mcda problems can be grouped into the following three categories: multi-criteria choice, multi-criteria ranking, and multi-criteria sorting (vassilev et al., 2005). independent of the type of problem, the decision maker (dm) is an influential factor in the success of the decisionmaking process. the dm gives additional information to select the preferred alternative(s) and also provides input based on his/her preferences according to the goals (poveda & lipsett, 2013). the several methods developed to solve multi-criteria problems can be grouped into the following three classes: multi-attribute utility theory (maut) methods, outranking methods, and non-classical mcda approaches. first, the maut methods provide the dm with the opportunity to quantify the appeal (i.e., desire to select) of a number of alternatives in which the dm handles a level of uncertainty, risk, and trade-offs among the multiple alternatives. second, the outranking methods depart from the assumption that there is limited comparability among the alternatives. it is assumed in most outranking methods that the dm “is unable to differentiate among the four binary relations (i.e., the indifference i [reflexive and symmetric], the weak preference q [irreflexive and antisymmetric], the strict preference p [irreflexive and antisymmetric], and the incomparability r [irreflexive and symmetric]) used to compare two alternatives” (poveda & lipsett, 2013, p. 207). third, the nonclassical mcda approaches require making a distinction between internal and external uncertainties. while external uncertainties are associated with imperfect knowledge related to the consequences of actions, internal uncertainties refer to the dm’s values and judgements (figueira et al., 2005). table 1 includes some of the most popular mcdm and mcda methods used to select, rank, and weight alternatives through decision makers’ (i.e., stakeholders’) participation. ijahp article: poveda/using multi-criteria decision-making to assess the importance of human capital in meeting the goals and objectives of sustainable development: an application of the analytic hierarchy process international journal of the analytic hierarchy process 17 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1067 table 1 mcdm and mcda methods multi-attribute utility theory (maut) methods outranking methods non-classical mcda approaches analytic hierarchy process (ahp) analytic network process (anp) utilites additives (uta) methods value tradeoff method direct weighting method measuring attractiveness by a categorical based evaluation technique (macbeth) method preference ranking organization method for enrichment of evaluations (promethee) methods includes: promethee i, promethee ii, promethee iii, promethee iv, promethee v, and promethee vi. élimination et choix traduisant la realité (electre) methods includes: electre i, electre-iv electre-is, electre ii, electre iii, electre iv, electre tri, and electre-a tactic method fuzzy-promethee promethee-gaia promethee-gdss note:the table is not comprehensive; it only includes some of the most popular mcdm and mcda methods. 6. simulated case study: importance of human capital in the sustainability of financial institutions organizations from a wide range of industries release an annual esg, corporate social responsibility (csr) or sustainability report. financial institutions include a set of indicators under a number of criteria to evaluate performance in areas such as social, economic, governance, and environment among others in their reports. while financial institutions are not mandated to follow a standardized template to report esg, csr or sustainability performance, some commonalities in sub-principles (i.e., dimensions or pillars), criteria, and indicators can be found. financial institutions may use other key performance indicators (kpis) to measure, track and manage performance in various areas; however, annual reports released to the public include those with a critical role in achieving the organization’s sustainability goals, objectives, and vision and are considered important to stakeholders, shareholders, and stockholders. the canadian ‘big six’ financial institutions include the royal bank of canada (rbc), toronto-dominion bank (td canada trust), bank of nova scotia (scotiabank), national bank of canada, bank of montreal (bmo), and canadian imperial bank of commerce (cibc). the latter two release an annual sustainability report whereas the other four financial institutions share their performance information in an esg report. scotiabank, a leading financial institution in the americas, has a team of over 90,000 employees and assets of over $1.3 trillion (as of january 31, 2023) (scotiabank, 2023a). scotiabank received excellence awards in human capital management from brandon hall group for best unique/innovative leadership development programs for its data and analytics skill building and ilead people manager essentials programs (scotiabank, 2021). scotiabank was also named one of canada’s most admired corporate cultures for 2021 by waterstone human capital (scotiabank 2021). scotiabank’s performance in human capital can be linked directly to the multiple awards over the years in canada and ijahp article: poveda/using multi-criteria decision-making to assess the importance of human capital in meeting the goals and objectives of sustainable development: an application of the analytic hierarchy process international journal of the analytic hierarchy process 18 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1067 throughout the americas as top employer, best workplace, corporate governance, or specific areas of performance such as diversity, gender equality, inclusion, leadership, and social responsibility among many others (scotiabank, 2023b). furthermore, some of the latest recognitions received by scotiabank include six wins in the 2022 global finance sustainable finance awards including best bank in canada for leadership in sustainable finance (scotiabank, 2023c), north america’s best bank for sustainable finance at the 2022 euromoney awards for excellence (scotiabank, 2023d), and the 2022 corporate equality index: best places to work for lgbtq+ quality with a perfect score of 100% (scotiabank, 2023e). scotiabank’s size (i.e., number of employees), solid status in the financial market, and esg performance indicate a successful approach by the institution to attract, develop and preserve its human capital. therefore, scotiabank’s esg report and performance provide the right conditions to apply mcdm or mcda methods, in particular the ahp, to evaluate the importance or relevance of human capital in achieving the overall sustainability vision, goals, and objectives. 6.1 analytical hierarchy process (ahp): brief theoretical background and assigning weights developed in the 1970s by thomas l. saaty, the ahp, one of the most widely used and easily implemented maut methods, structures the decision problem in a hierarchy and applies a measurement scale to obtain vectors of normalized weights or priorities using pairwise comparisons (saaty, 1977). furthermore, bouyssou et al. (2006) describe three characteristics for building an evaluation model using the ahp method: 1) the evaluation model is structured in a hierarchical way; 2) the same assessment technique is used at each node of the hierarchy; and 3) the assessment of the “children” nodes of a common “parent” node is based on pairwise comparisons. the node at the top of the hierarchy represents the main objective of the decision problem to be resolved by the decision maker. based on the number of alternatives, nodes at each level of the hierarchy can split as many times as there are alternatives. the end result for each node is the aggregation of the analysis of the alternatives in the level node immediately below. saaty (2008) structures the decision problem in the following four steps: define the problem and knowledge sought; structure the decision hierarchy; build the pairwise comparison matrices; and weight the priorities. to assign weights or scores to each ‘child’ node (i.e., alternative) of a ‘parent’ node, the decision maker follows a three steps process: 1) the decision maker is asked to compare the alternatives (e.g., sub-principles, criteria, indicators) in a pairwise comparison in terms of their relative importance using a conventional semantic scale; 2) the qualitative assessments given by the participants are quantified (i.e., quantitative interpretation), resulting in an n x n pairwise comparison matrix; and 3) the pairwise comparison matrix is used to determine a score or weight wi then the eigenvector corresponding to the maximum eigenvalue of the matrix is computed, and normalized to add up to 1. although slight modifications related to its interpretation and conceptual additions have been made, the measurement scale used by the ahp method has not been fundamentally changed over the years. table 2 illustrates the adaptation of poveda and lipsett (2013) to the measurement scale used by decision-makers in the ahp method. the measurement scale assists in the construction of pairwise matrices. the values within the pairwise matrices show the degree of importance or relevance that an element (e.g., sub-principles, criteria, indicators) within the same hierarchical level has over the others when they are compared pairwise. ijahp article: poveda/using multi-criteria decision-making to assess the importance of human capital in meeting the goals and objectives of sustainable development: an application of the analytic hierarchy process international journal of the analytic hierarchy process 19 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1067 table 2 measurement scale used in the ahp method intensity of importance definition explanation 1 equal importance two activities contribute equally to the objective 2 weak or slight importance 3 moderate importance of one over another experience and judgment slightly favor one activity over another 4 moderate plus importance 5 essential or strong importance experience and judgment strongly favor one activity over another 6 strong plus importance 7 very strong or demonstrated importance an activity is favored very strongly over another; its dominance is demonstrated in practice 8 very, very strong importance 9 extreme importance the evidence favoring one activity over another is of the highest possible order of affirmation where; 2, 4, 6, 8 intermediate values between the two adjacent judgments when compromise is needed 1.1–1.9 if the activities are very close may be difficult to assign the best value but when compared with other contrasting activities the size of the small numbers would not be too noticeable, yet they can still indicate the relative importance of the activities reciprocals if activity i has one of the above non-zero numbers assigned to it when compared with activity j, then j has the reciprocal value when compared with i a reasonable assumption rationals ratios arising from the scale if consistency were to be forced by obtaining n numerical values to span the matrix in the ahp method, the relevance or importance of a series of elements (e.g., subprinciples, criteria, or indicators) is determined using pairwise comparison matrices. an element compared with itself has a weight valued as 1. the structure of pairwise comparison matrices consists of a number of elements, m, and a series of criteria, n. criteria n are the same elements m. the formation of a pairwise comparison matrix results in a certain element m becoming an n criterion (e.g., m1 = n1). because elements can be assessed in terms of every criterion, the relevance or importance (i.e., weight) of each element can be calculated as well. furthermore, aij represents the weight of an element over a criterion where, i=1,2, 3, ……m and j=1,2,3, …...n. the pairwise comparison matrix below represents the basic decision problem to be solved using the ahp method to weight the alternatives in each hierarchical level: ijahp article: poveda/using multi-criteria decision-making to assess the importance of human capital in meeting the goals and objectives of sustainable development: an application of the analytic hierarchy process international journal of the analytic hierarchy process 20 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1067 alternative/criteria alternative absolute weights criteria m1 m2 m3 m4 …….. mm n1 a11 a12 a13 a14 …….. a1m w1 n2 a21 a22 a23 a24 …….. a2m w2 n3 a31 a32 a33 a34 …….. a3m w3 n4 a41 a42 a43 a44 …….. a4m w4 . . . . . . . . . . . . . . . . . . . . . nn an1 an2 an3 an4 anm wnm . . . . . . . . . . . . . . . . . . . . . mm = nn a21 = wn2/wm1, etc wnm = wnn = wmm 6.2 weighting the importance of human capital within the context of sustainable development the latest esg report released by scotiabank includes the following four sub-principles (i.e., pillars or dimensions): governance, sustainable finance, environment, and social. table 4 organizes the criteria and indicators under each sub-principle. additionally, subprinciples, criteria, and indicators have been codified to facilitate the description of the elements within the text and their graphic representation. a total of 32 indicators were found in the esg report with the social sub-principle being the only one using four criteria to group the selected set of indicators. two out of the four criteria in the social sub-principle group include the indicators linked to human capital. criteria c-1 and c-2 include four and seven indicators, respectively. establishing the hierarchy is the first critical step to assessing the importance (i.e., weight) of the different elements included in scotiabank’s esg report using the ahp method. figure 4 illustrates the various levels in the hierarchical structure and how they are linked. for this simulated case, the weights of each sub-principle, criteria under the social sub-principles and indicators under criteria c-1 and c-2 are needed to assess the importance of human capital within the context of sustainable development. the four clusters of indicators (i.e., criteria) under the social sub-principle do not include an equal number of indicators. indicators reflect a specific facet of esg or sustainable development or can be a combination of two more facets; these indicators are often called where; m1 = n1, m2 = n2, m3 = n3, m4 = n4, a11 = wn1/wm1 a21 = wn2/wm1 a31 = wn3/wm1 a41 = wn4/wm1 w1 = wn1 = wm1 w2 = wn2 = wm2 w3 = wn3 = wm3 w4 = wn4 = wm4 ijahp article: poveda/using multi-criteria decision-making to assess the importance of human capital in meeting the goals and objectives of sustainable development: an application of the analytic hierarchy process international journal of the analytic hierarchy process 21 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1067 multi-facet or multi-attribute indicators. each element, but in particular indicators, is designed to capture, track, and manage the organization’s performance in order to achieve the desired sustainability vision, goals, and objectives. establishing and understanding the hierarchical structure is the preliminary step to forming the pairwise comparisons using the measurement scale described in section 5.1 and illustrated in table 2. the pairwise comparison process consists of a free evaluation of the relative importance of each sub-principle over others by the decision-maker(s). the first pairwise comparison computes the relative priorities of the sub-principles in a 4 x 4 matrix which includes the following elements: governance (sp-1), sustainable finance (sp-2), environment (sp-3), and social (sp-4). the second set of pairwise comparison matrices includes the elements within each sub-principle. because sp-1, sp-2, and sp-3 do not use criteria to group the set of indicators included in each one of them, the only pairwise comparison is a 4 x 4 matrix formed by criteria c-1, c-2, c-3, and c-4. the fourth level of the hierarchical structure includes a larger number of matrices. these pairwise comparisons evaluate the relative importance of each indicator over others included in each specific criterion. furthermore, sub-principle sp-4 (i.e., social) includes four criteria which result in individual pairwise comparison matrices. the two criteria including indicators linked to human capital are c-1 and c-2 which form 4 x 4 and 7 x 7 pairwise comparison matrices, respectively. ijahp article: poveda/using multi-criteria decision-making to assess the importance of human capital in meeting the goals and objectives of sustainable development: an application of the analytic hierarchy process international journal of the analytic hierarchy process 22 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1067 table 4 sub-principles, criteria, indicators included in scotiabank’s esg report sub-principles criteria indicators name code name code name code governance sp-1 indicators are not grouped using criteria n/a board of directors i-1 employees that attested to the scotiabank code of conduct i-2 dow jones sustainability index (djsi) north america i-3 cdp climate change score i-4 sustainable finance sp-2 indicators are not grouped using criteria n/a green, social, sustainable and sustainability-linked bonds underwritten i-5 green, sustainable and sustainability-linked loans i-6 sustainability and green bonds purchased by scotiabank i-7 sustainability and green bonds issued by scotiabank i-8 environment sp-3 indicators are not grouped using criteria n/a capital mobilized for climate-related finance i-9 % of electricity from non-emitting sources i-10 % decrease in scope 1 and 2 greenhouse gas (ghg) emission from 2016 levels i-11 internal carbon price per tonne co2e i-12 total ghg emissions (global, tonnes co2e) i-13 social sp-4 investing in our employees c-1 employee engagement score i-14 employees who believe scotiabank is committed to being socially responsible i-15 voluntary employee turnover rate i-16 total investment in employee training and career development i-17 leadership and employee diversity c-2 executive management team % women i-18 women i-19 people of color i-20 persons with disabilities i-21 indigenous peoples i-22 diverse gender identity i-23 employees that identify their sexual orientation as being lesbian, gay, bisexual or another diverse sexual orientation i-24 empowering our customers c-3 customer experience – number of follow-up calls made to retail customers i-25 customer case reviews completed by the office of the ombudsman i-26 small business loans i-27 access to banking for indigenous communities, businesses and peoples i-28 access to banking. total scotiabank colpatria zerofee accounts i-29 building resilience in our communities c-4 economic value distributed i-30 total value of community investment i-31 total number of hours volunteered by employees i-32 note: scotiabank doesn’t assign codes to sub-principles (e.g., governance) and criteria (e.g., investing in our employees). codes are use to facilitate the description of the elements within the text and their graphic representation in figure 4. ijahp article: poveda/using multi-criteria decision-making to assess the importance of human capital in meeting the goals and objectives of sustainable development: an application of the analytic hierarchy process international journal of the analytic hierarchy process 23 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1067 description environment, social & governance (esg) 100% • sp-1 25% sp-2 15% sp-3 25% sp-4 35% n/a n/a n/a c-1 35% c-2 15% c-3 30% c-4 20% • i-1 10% • i-2 25% • i-3 35% • i-4 30% • i-5 25% • i-6 25% • i-7 25% • i-8 25% • i-9 15% • i-10 10% • i-11 30% • i-12 18% • i-13 27% • i-14 25% • i-15 20% • i-16 35% • i-17 20% • i-18 12% • i-19 9% • i-20 22% • i-21 16% • i-22 18% • i-23 8% • i-24 15% • i-25 20% • i-26 20% • i-27 20% • i-28 20% • i-29 20% • i-30 20% • i-31 20% • i-32 20% 100% 100% 100% 100% 100% 100% 100% 2 .5 0 % 6 .2 5 % 8 .7 5 % 7 .5 0 % 3 .7 5 % 3 .7 5 % 3 .7 5 % 3 .7 5 % 3 .7 5 % 2 .5 0 % 7 .5 0 % 4 .5 0 % 6 .7 5 % 8 .7 5 % 7 .0 0 % 1 2 .2 5 % 7 .0 0 % 1 .8 0 % 1 .3 5 % 3 .3 0 % 2 .4 0 % 2 .7 0 % 1 .2 0 % 2 .2 5 % 6 .0 0 % 6 .0 0 % 6 .0 0 % 6 .0 0 % 6 .0 0 % 7 .0 0 % 6 .0 0 % 7 .0 0 % 25% 15% 25% 35% 15% 30% 20% 2 .5 0 % 6 .2 5 % 8 .7 5 % 7 .5 0 % 3 .7 5 % 3 .7 5 % 3 .7 5 % 3 .7 5 % 3 .7 5 % 2 .5 0 % 7 .5 0 % 4 .5 0 % 6 .7 5 % 3 .0 6 2 5 % 2 .4 5 % 4 .2 8 7 5 % 2 .4 5 % 0 .6 3 % 0 .4 7 2 5 % 1 .1 5 5 % 0 .8 4 % 0 .9 4 5 % 0 .4 2 % 0 .7 8 7 5 % 2 .1 0 % 2 .1 0 % 2 .1 0 % 2 .1 0 % 2 .1 0 % 2 .4 5 % 2 .1 0 2 .4 5 % 25% 15% 25% 12.25% 5.25% 10.50% 7.00% 17.50% 100% principle esg has an absolute weight of 100% sub-principles weight principle is distributed among the four sub-principles criteria relative weight of 100% is distributed among the criteria within each sub-principle indicators relative weight of 100% is distributed among the indicators within each criterion checking point relative weight of indicators within each criterion must add to 100% relative weight of indicators calculated multiplying relative weight of indicator by relative weight of criterion checking point relative weight of indicators within each criterion must add to the relative weight of the criterion absolute weight of indicators calculated multiplying the relative weight of the indicator by the relative weight of the criterion and sub-principle checking point addition of the absolute weight of indicator within a criterion must be equal to multiplying the relative weights of the criterion and sub-principle adding human capital weight absolute weight of human capital results from adding absolute weights of indicators within criteria c-1 and c-2 final checking point addition of absolute weights of all indicators must be 100% figure 4 weight distribution for sub-principles, criteria, and indicators included in the scotiabank’s esg report criteria and indicators linked to human capital ijahp article: poveda/using multi-criteria decision-making to assess the importance of human capital in meeting the goals and objectives of sustainable development: an application of the analytic hierarchy process international journal of the analytic hierarchy process 24 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1067 the outcomes of the pairwise comparison matrices allow the calculation of relative and absolute weights of each element within the hierarchical structure. relative weights can be calculated to indicate the importance or relevance of an element in relation to others within a set or group (e.g., criteria) at the same level of the hso. relative weights can also be calculated to understand the importance or relevance of an element in relation to the hierarchical levels above of it. the weight of an element in the fourth hierarchical level that considers the relative weights of the nodes in each hierarchical level above becomes the absolute weight of the element; absolute weight refers to the importance or relevance of an element in the whole hierarchical structure. the ahp method results in allocating relative and absolute weights to the four criteria (c-1, c-2, c-3, c-4) included in the social sub-principle. to illustrate the difference between relative and absolute weights, consider criterion c-1 (investing in our employees). the relative weights for criterion c-1 resulted in 35%; the remaining 65% is distributed among the other three criteria included in sp-4 (social sub-principle). the absolute weight of a criterion can be calculated as follow: 𝑊𝑒𝑖𝑔ℎ𝑡𝑎𝑏𝑠𝑜𝑙𝑢𝑡𝑒 = (𝑊𝑒𝑖𝑔ℎ𝑡𝑟𝑒𝑙𝑎𝑡𝑖𝑣𝑒 𝐶𝑟𝑖𝑡𝑒𝑟𝑖𝑜𝑛 ) 𝑥 ( 𝑊𝑒𝑖𝑔ℎ𝑡𝑟𝑒𝑙𝑎𝑡𝑖𝑣𝑒 𝑆𝑢𝑏 − 𝑝𝑟𝑖𝑛𝑐𝑖𝑝𝑙𝑒) calculation for criterion c-1, 𝑊𝑒𝑖𝑔ℎ𝑡𝑎𝑏𝑠𝑜𝑙𝑢𝑡𝑒 = (0.35 𝑥 0.35) = 0.1225 𝑜𝑟 12.25% similarly, the relative and absolute weight of an indicator can be calculated. the relative weight of indicator i-14 resulted in 25%; the remaining 75% is distributed among the other three indicators included in criterion c-1. the absolute weight of an indicator can be calculated as follow: 𝑊𝑒𝑖𝑔ℎ𝑡𝑎𝑏𝑠𝑜𝑙𝑢𝑡𝑒 = (𝑊𝑒𝑖𝑔ℎ𝑡𝑟𝑒𝑙𝑎𝑡𝑖𝑣𝑒 𝐼𝑛𝑑𝑖𝑐𝑎𝑡𝑜𝑟) 𝑥 (𝑊𝑒𝑖𝑔ℎ𝑡𝑟𝑒𝑙𝑎𝑡𝑖𝑣𝑒 𝐶𝑟𝑖𝑡𝑒𝑟𝑖𝑜𝑛) 𝑥 (𝑊𝑒𝑖𝑔ℎ𝑡𝑟𝑒𝑙𝑎𝑡𝑖𝑣𝑒 𝑆𝑢𝑏 − 𝑝𝑟𝑖𝑛𝑐𝑖𝑝𝑙𝑒) calculation for indicator i-14, 𝑊𝑒𝑖𝑔ℎ𝑡𝑎𝑏𝑠𝑜𝑙𝑢𝑡𝑒 = (0.25) 𝑥 (0.35) 𝑥 (0.35) = 0.030625 𝑜𝑟 3.0625% because criteria c-1 and c-2 include the indicators linked to human capital, the ahp method allowed the identification of a relative weight of human capital of 50% within the social sub-principle. similarly, the absolute weight of human capital was 17.5% which represents the importance or relevance of human capital in meeting the esg goals, objectives and vision of the organization. 6.3 sensitivity analysis: evaluating the impact of human capital on the overall esg performance the sensitivity analysis to determine how the different weight values of the independent variables (i.e., indicators linked to human capital) affect the weight of other dependent variables (e.g., indicators, criteria, sub-principles) must consider the hso and the elements (i.e., sub-principles, criteria, indicators) within each of its levels. furthermore, while the absolute weight of the principle remains at 100%, the weights of the other ijahp article: poveda/using multi-criteria decision-making to assess the importance of human capital in meeting the goals and objectives of sustainable development: an application of the analytic hierarchy process international journal of the analytic hierarchy process 25 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1067 elements in the hso depend on the approach used. four approaches can be used to determine the weights of the elements in the hso: 1) equal weight approach; 2) topdown approach; 3) mixed approach; and 4) bottom-up approach. the equal weight approach considers the equal distribution of weight among the elements within the same level of hso; therefore, mcdm or mdma methods such as the ahp methodology are not needed to assign the weights based on the relevance of an element in relation to the others. the top-down approach allocates equal weights to the elements within the sub-principle level (i.e., sp-1, sp-2, sp-3, and sp-4 have the same weight), but the weights of the elements within the criterion and indicator levels are the result of applying the ahp methodology. therefore, the top-down approach presents several potential outcomes to the impact of human capital indicators. table 5 presents each scenario of the potential impact of human capital on the overall esg performance. the weights of indicators linked to human capital can vary, increase or decrease. because the weights of the subprinciples are already set after their weights have been equally distributed, the impact of the weights of indicators linked to human capital can only have an effect on the weights of the criteria c-1 (investing in our employees) and c-2 (leadership and employee diversity). the weights of the criteria can increase even if the weights of the indicators linked to human capital decrease because the human capital indicators are mixed with other types of indicators under the same criterion. similarly, the weights of the criteria can decrease even if the weights of the indicators linked to human capital increase. ijahp article: poveda/using multi-criteria decision-making to assess the importance of human capital in meeting the goals and objectives of sustainable development: an application of the analytic hierarchy process international journal of the analytic hierarchy process 26 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1067 table 5 potential impact of human capital indicators to the overall esg performance approach to weighting elements within each level of the hso principle esg sub-principle (social [sp-4]) criterion c-1 (investing in our employees) criterion c-2 (leadership and employee diversity) indicators linked to human capital equal approach 100% ↔ ↔ ↔ ↔ top-down approach 100% ↔ ↔ ↔ ↕ 100% ↔ ↑ ↑ ↑ 100% ↔ ↑ ↓ ↑ 100% ↔ ↓ ↑ ↑ 100% ↔ ↓ ↓ ↑ 100% ↔ ↑ ↓ ↓ 100% ↔ ↓ ↑ ↓ 100% ↔ ↓ ↓ ↓ 100% ↔ ↑ ↑ ↓ mixed approach 100% ↔ ↕ ↕ ↔ 100% ↔ ↑ ↓ ↔ 100% ↔ ↓ ↑ ↔ bottom-up approach 100% ↑ ↑ ↑ ↑ 100% ↑ ↑ ↓ ↑ 100% ↑ ↓ ↑ ↑ 100% ↑ ↓ ↓ ↑ 100% ↑ ↑ ↓ ↓ 100% ↑ ↓ ↑ ↓ 100% ↑ ↓ ↓ ↓ 100% ↑ ↑ ↑ ↓ 100% ↓ ↑ ↑ ↑ 100% ↓ ↑ ↓ ↑ 100% ↓ ↓ ↑ ↑ 100% ↓ ↓ ↓ ↑ 100% ↓ ↑ ↓ ↓ 100% ↓ ↓ ↑ ↓ 100% ↓ ↓ ↓ ↓ 100% ↓ ↑ ↑ ↓ symbols: ↔ = the weights of the elements within the same level of the hso are equally distributed ↕ = the weights of the elements within the same level of the hso are variable ↑ = the weight of the elements within the same level of the hso increases ↓ = the weight of the elements within the same level of the hso decreases the mixed approach allocates equal weights to the elements within the sub-principle and indicator levels; therefore, the ahp methodology is not implemented in the weight allocation process. the ahp methodology is an instrument to allocate the weights of the elements within the criterion level. the weights of criteria c-1 (investing in our employees) and c-2 (leadership and employee diversity) vary. since criteria c-1 and c-2 mix human capital with indicators that measure other facets of the social sub-principles (sp-4) and the indicators are equally weighted, indicators linked to human capital have limited impact on the weight of criteria c-1 and c-2. nevertheless, few potential outcomes can be identified after applying the ahp methodology. table 5 includes three scenarios: 1) the weights of both c-1 and c-2 increase or decrease simultaneously; 2) the weight of c-1 increases and the weight of c-2 decreases; and 3) the weight of c-1 decreases and the weight of c-2 increases. the bottom-up approach uses the mcdm or mcda methods such as the ahp methodology to allocate weights to the elements within the indicator level. then, the ijahp article: poveda/using multi-criteria decision-making to assess the importance of human capital in meeting the goals and objectives of sustainable development: an application of the analytic hierarchy process international journal of the analytic hierarchy process 27 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1067 values of the weight of the criteria are the result of adding the weights of the indicators within each specific criterion. similarly, the values of the weights of the sub-principles are the result of adding the weights of the criteria within each specific sub-principle. adding the weights of the sub-principles results in the absolute weight of the principle. the principle is at the top of the hso and its absolute weight is set at 100%. using the bottom-up approach, the impact of indicators linked to human capital on the weights of criteria c-1 (investing in our employees) and c-2 (leadership and employee diversity) and sub-principles sp-4 (social) increases. table 5 includes the different potential scenarios of the impact of the indicators linked to human capital. stakeholder and decision-maker groups could use the bottom-up approach to allocate higher weights to indicators linked to human capital and influence the esg performance of the organization (i.e., scotiabank). 7. conclusions sustainability and human capital are continuously evolving concepts. a concept with a modest origin linked to productivity and economic growth has become one of the engines to achieve the vision, goals, and objectives of sustainability. human capital has a supporting and coordinating role among the various pillars or dimensions included in the concept of sustainable development. furthermore, the development of human capital has a determinant and direct impact on reducing environmental degradation and improving environmental protection while advancing social progress and economic growth. although the impact of human capital has become noticeable, the degree of its importance or relevance in meeting the goals and objectives of sustainable development and achieving the stakeholders’ vision of sustainably and esg is still uncertain. evaluating the impact of human capital has a number of challenges. first, the constant evolution of the concepts makes a widely accepted and useful definition in sustainability assessment studies evasive. second, human capital is not and cannot be a static concept. the characteristics of human capital vary based on the specific needs of a project, organization, or industry. therefore, the concept of human capital has temporal and spatial features which indicate its adaptable nature in time and context. third, the two previous challenges contribute to the already difficult task of assessing human capital. either identifying and selecting the proper set of criteria and indicators or evaluating the relevance or importance of human capital within the sustainable development context, practitioners and scientists face the need to better understand the concept itself and its role in broader contexts. the assessment of human capital can be assisted by mcdm and mcda methods. structured and scientific-based methodologies using an effective engagement and participation of stakeholders presents a valid and reliable option for understanding the importance or relevance of human capital in meeting the goals and objectives of sustainable development. among a wider range of options, the ahp method captures preestablished goals and objectives through the development of pairwise comparison matrices that include elements identified and selected by stakeholders. furthermore, the assessment process allows stakeholders to embed their vision of a sustainable future and the priority that should be given to the various facets of the concept of sustainable development and each element within it. ijahp article: poveda/using multi-criteria decision-making to assess the importance of human capital in meeting the goals and objectives of sustainable development: an application of the analytic hierarchy process international journal of the analytic hierarchy process 28 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1067 references akintoye, i. r., & adidu, f.a. (2008). optimising national growth through human resources investments. european journal of scientific research, 22, 433-443. barlett, a. a. (2012). meaning of sustainability. teachers clearinghouse for science and society education newslatter, 31(1), 1-17 becker, g. s. (1962). investment in human capital: a theoretical analysis. journal of political economy, 70(5, part 2), 9-49. doi: https://doi.org/10.1086/258724 benhabib, j., & spiegel, m. m. (1994). the role of human capital in economic development evidence from aggregate cross-country data. journal of monetary economics, 34(2), 143–173. doi: https://doi.org/10.1016/0304-3932(94)90047-7 bouyssou, d., marchant, t., pirlot, m., tsoukias, a., & vincke, p. (2006). evaluation and decision models with multiple criteria: stepping stones for the analyst. in p. bernus, j. blazewicz, g. schmidt, m. shaw (eds.), international handbooks on information systems (pp 89-113). springer. doi: https://doi.org/10.1007/978-3-662-46816-6_4 bremen, j. m., ganu, s., sung, a., & wurtzel, m. (2021, sept 2). human capital is the key to a successful esg strategy. world economic forum. https://www.weforum.org/agenda/2021/09/human-capital-is-the-key-to-a-successful-esgstrategy. ciuhu, a. m. (2016). human capital and sustainability: current challenges. ecoeconomics review, 2(1), 43-48. denison, e. f. (1962). sources of economic growth and the alternative before us. committee for economic development. european commission. (2009). sustainable development indicators: overview of relevant fp-funded research and identification of further needs. european communities: luxembourg. farčnick, d., & istenič, t. (2020), the importance of human capital for sustainability. in v. žabkar, & t. redek (eds.), challenges on the path toward sustainability in europe (pp. 237-252). emerald publishing limited. doi: https://doi.org/10.1108/978-1-80043972-620201002 figueira, j.r., greco, s., & ehrgott, m. (2005). multiple criteria decision analysis: state of the art surveys. kluwer academic publishers. giarini, o. (1980). dialogue on wealth and welfare: an alternative view of world capital formation. elsevier. goldin, c. (2016). human capital. in c. diebolt, & m. haupert (eds). handbook of cliometrics (pp.55-86). springer. https://www.weforum.org/agenda/2021/09/human-capital-is-the-key-to-a-successful-esg-strategy https://www.weforum.org/agenda/2021/09/human-capital-is-the-key-to-a-successful-esg-strategy ijahp article: poveda/using multi-criteria decision-making to assess the importance of human capital in meeting the goals and objectives of sustainable development: an application of the analytic hierarchy process international journal of the analytic hierarchy process 29 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1067 harbison, f.h. (1973). human resources as wealth of nations. oxford: oxford university press. kiker, b. f. (1966). the historical roots of the concept of human capital. journal of political economy, 75(5), 481-499. doi: https://doi.org/10.1086/259201 kim, d., & go, s. (2020). human capital and environmental sustainability. sustainability, 12(11), 4736. doi: https://doi.org/10.3390/su12114736 mankiw, n. g., romer, d., & weil, d. n. (1992). a contribution to the empirics of economic growth. the quarterly journal of economics, 107(2), 407–437. doi: https://doi.org/10.2307/2118477 merriam-webster. (n.d.). human capital. in merriam-webster.com dictionary. retrieved september 8, 2022, from https://www.merriamwebster.com/dictionary/human%20capital ness, b., urbel-pirrsalu, e., anderberg, s., & olsson, l. (2007). categorising tools for sustainability assessment. ecological economics, 60(3), 498-508. doi: https://doi.org/10.1016/j.ecolecon.2006.07.023 nelson, r. r., & phelps, e. s. (1966). investment in humans, technological diffusion, and economic growth. the american economic review, 56(1/2), 69–75. oecd. (n.d.). productivity, human capital and educational policies. https://www.oecd.org/economy/human-capital/ pigou, a. c. (1928). a study in public finance. macmillan and co. pokorny, b., & adams, m. (2003). what do criteria and indicators assess? an analysis of five c&i sets relevant for forest management in the brazilian amazon. international forestry review, 5(1), 202-208. doi: https://doi.org/10.1505/ifor.5.1.20.17434 porreca, z. (2020). environmental sustainability and human capital development. consilience. the journal of sustainable development, 22, 48–57. porritt, j. (2005). capitalism as if the world matters. earthscan. poveda, c. (2014). a methodology for pre-selecting sustainable development indicators (sdis) with application to surface mining operations. 2nd international conference on environmental and economic impact on sustainable development: incorporating environmental economics, toxicology and brownfields, ancona, italy. poveda, c. a. (2016). the theory of dimensional balance of needs. international journal of sustainable development & world ecology, 24(2), 97-119. doi: https://doi.org/10.1080/13504509.2016.1201019 https://www.merriam-webster.com/dictionary/human%20capital https://www.merriam-webster.com/dictionary/human%20capital https://www.oecd.org/economy/human-capital/ ijahp article: poveda/using multi-criteria decision-making to assess the importance of human capital in meeting the goals and objectives of sustainable development: an application of the analytic hierarchy process international journal of the analytic hierarchy process 30 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1067 poveda, c. a. & lipsett, m. (2013). weighting sustainable development indicators (sdis) for surface mining operations using the analytical hierarchy process (ahp). international journal of the analytic hierarchy process, 5(2), 200-222. doi: https://doi.org/10.13033/ijahp.v5i2.199 saaty, t. (1977). a scaling method for priorities in hierarchical structures. journal of mathematical psychology, 15, 234-81. doi: https://doi.org/10.1016/00222496(77)90033-5 saaty, t. (2008). decision making with the analytic hierarchy process. international journal of services science, 1(1), 83-98. salim, r., yao, y., & chen, g. s. (2017). does human capital matter for energy consumption in china? energy economics, 67, 49–59. doi: https://doi.org/10.1016/j.eneco.2017.05.016 savvides, a., & stengos, t. (2008). human capital and economic growth. stanford: stanford university press. schuh, g.e. (2002), human capital for sustainable economic development. in k.s. bawa, & r. seidler (eds). dimensions of sustainable development – volume i. unesco publishing-eolss publishers. schultz, t. (1962). reflections on investment in man. journal of political economy, 70(1). doi: 10.1086/258723 scotiabank. (2021). 2021 scotiabank environment, social & governance (esg) report. https://www.scotiabank.com/content/dam/scotiabank/corporate/documents/scotiabank_2 021_esg_report_final.pdf scotiabank. (2023a). about scotiabank. https://www.scotiabank.com/ca/en/about/ourcompany/corporate-profile.html scotiabank. (2023b). our awards. https://www.scotiabank.com/careers/en/careers/whyscotiabank/our-awards.html scotiabank. (2023c). awards. https://www.scotiabank.com/ca/en/about/perspectives.articles.impact.2022-05scotiabank-recognized-global-leader.html scotiabank. (2023d). scotiabank’s sustainable finance work recognized by 2022 euromoney awards. https://www.scotiabank.com/ca/en/about/perspectives.articles.impact.2022-07euromoney-awards.html scotiabank. (2023e). markets insights. https://www.gbm.scotiabank.com/en/marketinsights/article.diversity-equity-and-inclusion.scotia-named-one-of-best-places-to-workfor-lgbtq-equality.html https://econpapers.repec.org/scripts/redir.pf?u=https%3a%2f%2fdoi.org%2f10.1086%252f258723;h=repec:ucp:jpolec:v:70:y:1962:p:1 https://www.scotiabank.com/content/dam/scotiabank/corporate/documents/scotiabank_2021_esg_report_final.pdf https://www.scotiabank.com/content/dam/scotiabank/corporate/documents/scotiabank_2021_esg_report_final.pdf https://www.scotiabank.com/ca/en/about/our-company/corporate-profile.html https://www.scotiabank.com/ca/en/about/our-company/corporate-profile.html https://www.scotiabank.com/careers/en/careers/why-scotiabank/our-awards.html https://www.scotiabank.com/careers/en/careers/why-scotiabank/our-awards.html https://www.scotiabank.com/ca/en/about/perspectives.articles.impact.2022-05-scotiabank-recognized-global-leader.html https://www.scotiabank.com/ca/en/about/perspectives.articles.impact.2022-05-scotiabank-recognized-global-leader.html https://www.scotiabank.com/ca/en/about/perspectives.articles.impact.2022-07-euromoney-awards.html https://www.scotiabank.com/ca/en/about/perspectives.articles.impact.2022-07-euromoney-awards.html https://www.gbm.scotiabank.com/en/market-insights/article.diversity-equity-and-inclusion.scotia-named-one-of-best-places-to-work-for-lgbtq-equality.html https://www.gbm.scotiabank.com/en/market-insights/article.diversity-equity-and-inclusion.scotia-named-one-of-best-places-to-work-for-lgbtq-equality.html https://www.gbm.scotiabank.com/en/market-insights/article.diversity-equity-and-inclusion.scotia-named-one-of-best-places-to-work-for-lgbtq-equality.html ijahp article: poveda/using multi-criteria decision-making to assess the importance of human capital in meeting the goals and objectives of sustainable development: an application of the analytic hierarchy process international journal of the analytic hierarchy process 31 vol 15 issue 1 2023 issn 1936-6744 https://doi.org/10.13033/ijahp.v15i1.1067 slaus, i., & jacobs, g. (2011). human capital and sustainability. sustainability, 3(1), 97154. doi: https://doi.org/10.3390/su3010097 smith, a. (1776). an inquiry into the nature and causes of the wealth of nations. random house. the world bank. (n.d.). the human capital project: frequently asked questions. https://www.worldbank.org/en/publication/human-capital/brief/the-human-capitalproject-frequently-asked-questions#hcp2 tisdell, c. (2000): sustainable development of human resource capital. principles of sustainable development, vol. 1. encyclopedia of life support systems (eolss), developed under the auspices of the unesco, eolss publishers, paris, france, accessible at http://www.eolss.net/sample-chapters/c13/e1-46a-02-02.pdf ugnich, c. a., chemokozv, a., & ugnich, m. (2021). human capital in the system of sustainable development goals: significance and prospects. e3s web of conferences, 258, 7053. vassilev, v., genova, k., & vassileva, m. (2005). a brief survey of multicriteria decision making methods and software systems. cybernetics and information technologies, 5(1). weisbrod, b. a. (1961). the valuation of human capital. journal of political economy, 69(5), 425. doi: https://doi.org/10.1086/258535 world economic forum. (2020, dec 16). global competitiveness report special edition 2020: how countries are performing on the road to recovery. https://www.weforum.org/reports/the-global-competitiveness-report-2020/in-full https://www.worldbank.org/en/publication/human-capital/brief/the-human-capital-project-frequently-asked-questions#hcp2 https://www.worldbank.org/en/publication/human-capital/brief/the-human-capital-project-frequently-asked-questions#hcp2 http://www.eolss.net/sample-chapters/c13/e1-46a-02-02.pdf ijahp: garuti/reflections on common misunderstandings when using ahp and a response to criticism of saaty’s consistency index international journal of the analytic hierarchy process 488 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.573 reflections on common misunderstanding when using ahp and a response to criticism of saaty’s consistency index claudio garuti fulcrum ingenieria claudiogaruti@fulcrum.cl abstract this essay deals with misunderstandings which, in this author’s opinion, seem to prevail in the minds of some ahp critics. the main apparent problem is the lack of necessary knowledge about the scale concept including what a scale is, how it is mathematically defined, and what one can or cannot do with a scale. this knowledge is critical for a good understanding of ahp. this study will review the concept of scales, discuss common misconceptions, introduce the compatibility index g and explain how it addresses one of the previous fallacies perpetuated about ahp. keywords: ahp misconceptions; scales; compatibility index g 1. introduction there are different kinds of scales and what one can do with each one will depend on the properties of the scale. for instance, it is clear that with an ordinal scale you cannot make arithmetic operations (they are not allowed), but with a ratio scale you can. a scale is mathematically defined by its invariant function; the core of the scales and its properties come from this mathematical function. mathematical definition of a scale: mathematically, a scale is a triplet composed of a set of numbers, a set of objects, and a transformation from the numbers to the objects. the scale also has a more abstract interpretation that only refers to the nature of the numbers and not to the objects, or how the numbers are assigned to the objects. the kind of transformation or the forms to create the numbers admissible for a particular measurement define what is called a scale of measurement for a measurement operation. we have different kinds of transformation (invariant) and scales as follows: the description of these scales can be found in saaty, l.t. (2001), the analytic network process: decision making with dependence and feedback and in garuti, c. and escudey m. (2005), toma de decisiones en escenarios complejos. nominal scale: invariant under a one-to-one correspondence; for example, when a name or telephone number is assigned to an object, there is one and only one name and telephone number assigned to each object in the set. ordinal scale: invariant under monotone transformation, where the numbers order the objects, but the magnitude of those numbers are only useful for defining whether the mailto:claudiogaruti@fulcrum.cl ijahp: garuti/reflections on common misunderstandings when using ahp and a response to criticism of saaty’s consistency index international journal of the analytic hierarchy process 489 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.573 order is increasing or decreasing; for example, when assigning numbers 1 and 2 to two people to indicate that one is taller than the other, without including more information about their real height. the minor number can be assigned to the taller person or vice versa. interval scale: invariant under a positive linear transformation. for instance, the linear transformation f = (9/5) + 32, which transforms readings of temperature from celsius to fahrenheit. notice that it is not possible to add two measures x1 and x2 in an interval scale, because then y1 + y2 = (ax1+b) + (ax2 + b) = a(x1+x2) + 2b, which takes the form of (ax3 + 2b) which is not in the form of (ax + b) anymore. however, we can take the average of both readings because after dividing by 2 we are back to the original form. this is why 10 degrees of temperature plus 15 degrees of temperature does not produce 25 degrees of temperature (at most, its sum can make 15 degree). proportional scale: invariant under homogenous transformation, y = ax, a>0. an example is a transformation from pounds to kilograms with the transformation k = 2.2p. the proportion of the weights of two objects is the same and does not depend on whether the measurement was in pounds or kilos. the zero has no correspondence with any measurement of a real object; it is only applied to objects that do not present the property. it is not possible to divide by zero and get back a result we can interpret. we also may note that we can add two measures of the same scale a(x1+x2) = a(x3) which have the same form of ax. we can also multiply and divide different readings from the same proportional scales. when dividing two measures of the same proportional scale, the ratio of any two measures within it belongs to a proportional absolute scale. for example, 6kg /3kg = 2. the number “2” belongs to a proportional absolute scale, showing that the object weighing 6kg is double the weight of the object weighing 3kg. the number 2 is an absolute number because it cannot be transformed to any different number. the idea of an “absolute number” gives us the entry to present the following scale. absolute scale: invariant under the identity transformation y = x, (with a = 1), coming from the ratio: y1/y2 = ax1/ax2 = 1x3. examples of this scale are the numbers used to count people in a room, and the natural and real numbers (that is, those used to resolve equations). these absolute numbers are defined in terms of correspondence and equivalence classes of one-to-one correspondence following the postulates of the great italian mathematician peano, not in terms of some unit of measure starting from an origin at zero. for the last 3 scales (interval, proportional and absolute), it is important to not confuse or mix the concept of the invariant of transformation (the concept that defines the scale) with the linear equation within the invariant of transformation. in doing so, we may fall into different conceptual errors. for instance, to believe that the absolute scale is just another proportional scale no different from any other ratio scale using as argument that the invariant of transformation of an absolute ratio scale (the identity function y=x) is just a particular case of the ratio scale (y=ax with a=1). although, this is true from the mathematical function point of view, when talking about scale and invariant of transformation it has a different meaning. an absolute scale is a different scale from a proportional scale; it has different properties and a different way of being built. for ijahp: garuti/reflections on common misunderstandings when using ahp and a response to criticism of saaty’s consistency index international journal of the analytic hierarchy process 490 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.573 instance, an absolute scale is dimensionless and its numbers are absolute numbers which means they cannot be transformed into another (number 3 is number 3 no matter what). another possible error (a kind of extrapolation of the last exposed error) is to believe that a proportional scale is just a particular case of the interval ratio scale, thinking that the transformation y=ax is just a particular case of the transformation   y=ax. but, we know that interval and proportional ratio scales are different scales since they have a different invariant, and thus different properties. if we fail to keep these differences in mind when working with the scales, then we may have several misconceptions, such as believing that the core of a ratio and absolute ratio scale is defined by the existence of a natural or absolute zero, but the zero (when existing) is just a consequence of the invariant transformation not it’s cause. for example, in thermodynamics, the absolute zero for temperature is defined as the detention of movement at the molecular level and it is represented as 0 degrees kelvin or 0 k. this is the absolute (and also natural) zero for temperature; there is no temperature below 0 k. however, the presence of this natural zero doesn’t make the kelvin scale of temperature a proportional scale, it is still an interval ratio scale like any temperature scale (3k degree + 5k degree do not make 8k degree) and this is due to the characteristics of the temperature function. on the other hand, suppose that we have built a scale for beauty. does that mean that i know the zero for beauty? no, it doesn't mean that necessarily. it just means that i know the ratio between the elements of the set and nothing else. we have to note that this scale was built from singular elements (probably using a fundamental scale) thus is not a continuous function, but a discrete one. in this way, the zero value does not need to exist (it could exist but it doesn’t need to). an important corollary is that the scale of ugly is not necessarily the inverse of the scale of beauty (they probably are 2 different scales). there is no need to reach one from the other passing through zero. simply put, when i say that a is 3 times more beautiful than b, it doesn’t mean (necessarily) that b is 3 times as ugly as a. none of the absolute scales have a need for a unit or zero. in spite of the fact that it is said that a proportional scale has an absolute zero, this is only a supposition that makes it easier to work with the scale. nowhere in the mathematical definition of a scale is it stated that it should have a unit and an origin with a zero value. (saaty, 2001; (garuti & escudey, 2005). therefore, the definitions that come from sources like “scales for dummies” are just an incomplete list of properties of scales and do not help to really understand what a scale is. in order for a fuller understanding, we need to know the invariant of the scale. there is the core of the scale. 2. common errors in ahp use with the scales definition clear, let’s start discussing the first common error. the first error or confusion i have found was thinking of saaty’s fundamental scale as an ordinal scale. this initial confusion leads to the following errors: 1  stand by difference or interval ijahp: garuti/reflections on common misunderstandings when using ahp and a response to criticism of saaty’s consistency index international journal of the analytic hierarchy process 491 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.573 error 1apply the arrow’s impossibility theorem to ahp numbers. arrow’s theorem says that it is not possible to combine 3 different ordinal rankings (a, b, c and b, c, a and c, b, a) in a way that the 5 properties of the ranking are obeyed. of course if we are working on a ratio scale (as in the case of ahp), then the impossibility becomes possible. indeed, it is easy to combine different ratio scales into just one. moreover, it is easy to demonstrate that a total inverse ordinal ranking (a, b, c and c, b, a for instance) can be more compatible (close) than another that has the same order (a, b, c and a, b, c) when a, b and c are numbers based in a ratio scale. this shows that trying to apply arrow’s theorem on cardinal numbers is a fallacy. arrow’s theorem is based on an ordinal scale (as arrow correctly pointed out in his demonstration) and ahp is based on a ratio-absolute scale. thus, arrow´s theorem is not applicable for ahp operations and results. error 2how is it possible that 2 moderates (or weak) comparisons can make an extreme one? this is very common mistake which is presented as: if a=3b and b=3c, then a=9c (two moderate or weak make an extreme). first, we have to remember that number 3 means moderate or weak and 9 means extreme in the verbal mode of saaty’s fundamental scale. but, saaty’s fundamental scale is an absolute ratio scale. keeping this in mind, it becomes obvious that two moderate comparisons (3) must make an extreme (9), if you want to be totally consistent (or close to 9 if you want to be close to fully consistent). this is because in an absolute scale (as in any ratio scale as well) 3 times 3 is 9, not 4, not 5, not 6. again, the problem here seems to be in the belief that saaty’s fundamental scale is an ordinal kind of scale. i think this confusion may come from the verbal mode of saaty’s scale. some people seem to believe that the verbal mode of the scale makes the scale itself an ordinal one. that is not true; the verbal mode is just a way to make the numeric scale more operative over qualitative criteria. (by the way, this can also be made through a graphical mode and this mode or appearance does not take away any cardinality from the original scale). error 3why does saaty’s fundamental scale not have a zero? this misinterpretation revolves around the claim about the lack of zero in saaty’s fundamental scale. saaty’s fundamental scale is based on an absolute ratio scale, and no absolute ratio scale has a zero (it can exist, but is not a necessity) because the absolute ratio scale is a scale built from the ratio of two ratio scales and zero is not a proportion to anything. also, the neutral of the scale is the number 1 not zero. thus, when i say that a=b, i’m really saying that a/b=1, and not necessarily that a-b=0. for instance, suppose that helen and betty are equally beautiful (or smart). when i say that helen is as beautiful as betty, i’m not defining an absolute zero for beauty; i’m just saying that the ratio of beauty between helen and betty is one, or that helen is as beautiful as betty. by the way, saaty’s scale is intended only for pair-comparison purposes. it is not made for evaluation purposes (i have seen different applications of ahp where the author uses the fundamental scale to evaluate the alternatives). the fundamental scale is a scale to build scales of measurement not to evaluate alternatives. thomas saaty himself said, “build scales from measurement, not measurement from scales” (isahp2009, pittsburgh 2009). ijahp: garuti/reflections on common misunderstandings when using ahp and a response to criticism of saaty’s consistency index international journal of the analytic hierarchy process 492 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.573 error 4what happens beyond 9 in saaty’s fundamental scale? the next misinterpretation is a little bit trickier. it says: if a=3b and b=4c, then a=12c (due to the absolute-ratio scale consideration). but saaty’s fundamental scale only goes until 9; so, what happens beyond 9? to answer this objection, it must first be noted that saaty’s fundamental scale was built to be used by humans, and humans have a problem when trying to compare two very different objects. a person’s precision decays exponentially as the ratio of comparison grows. indeed, if anyone tries to guess the number of baseballs that fit into a one meter cubic box, they will have serious trouble getting the right number. however, their precision will increase rapidly if the size of the box is reduced to 1/3. this is because now the size of the balls are relatively comparable with the box (relatively comparable means one order of magnitude, no more). therefore, saaty’s 1-9 scale is a human capacity issue, not an ahp issue. if someone wants to use larger numbers for a quantitative criteria it is possible, but they must be warned about the difficulties. this last misinterpretation is also related to the second axiom of ahp, the homogeneity axiom. this axiom states that you have to compare homogenous objects, which means objects that are within one order of magnitude. when two objects do not belong to the same order of magnitude in some criterion, then the ahp model must be adjusted in a way that does not break the second axiom. this last error can produce many different “new errors”, even wrong demonstrations that ahp is incorrect or even that the saaty’s consistency index diverges. one such criticism is described in “on the convergence of the pairwise comparisons inconsistency reduction process” (koczkodaj & szybowski, 2015). for instance, a classic error puts criteria that are very important to the problem with others that have low importance or even irrelevant to the problem in the same level or cluster. this is a very typical “design error” in the ahp/anp models. when this happens, it makes it highly probable that the output results (the priority ranking) will be incorrect. in fact, there was just such a case where a professor wrote a paper trying to show that the ahp was wrong. however, the professor was not aware of axiom 2, and built a wrong model with heterogeneous elements in the same cluster. of course, with a wrong model anything can happen (garbage in garbage out). by the way, in this example when the error (heterogeneous criteria) in the model is corrected the right results for the alternatives emerge. it is important to mention, that this list of errors should be seen not only as errors produced by misunderstandings on ahp, but also a list of good practices for creating a model. this list of errors can be used to understand how to use saaty’s fundamental scale and also to know what kind of scale is being created when creating new scales of measurement so we can know what to do with that scale. these errors (and others) can be found at www.researchgate.net, under ahp issues and different papers published. 3. consistency index misinterpretation and compatibility index g another error that can be produced due to the misuse of axiom 2 involves saaty’s consistency index. the same basic example is used as in the criticism made by w.w. koczkodaj and j. szybowski (2015). this criticism says that saaty’s consistency index is too wide (infinitely wide indeed). the mathematical and logical errors of this argument against saaty’s consistency index will be discussed. ijahp: garuti/reflections on common misunderstandings when using ahp and a response to criticism of saaty’s consistency index international journal of the analytic hierarchy process 493 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.573 the logical argument is related to axiom 2 of ahp that is using numbers beyond 9 in the pc-matrix. the mathematical argument is related with the reduction to the absurdum, using the compatibility index g for measuring in weighted environments combined with the logical fact that a zero vector is not a valid point of comparison with any vector in weighted environments. first, the compatibility index g must be introduced. this index is useful in determining closeness (similarity) between vectors in a weighted environment (where the euclidean measurement is not effective). g is mathematically defined as: g = ½ i (ai + bi) mini (a, b) / maxi (a, b) g is a continuous real function that returns values within (0 – 1) range, with 1 representing total compatibility (a = b, parallel vectors) and 0 total incompatibility (a ┴ b, perpendicular vectors). a, b are normalized priority vectors that belong to an absolute ratio scale within a weighted environment. more details about the compatibility index g can be found in garuti (2012), measuring in weighted environments: moving from metric to order topology, garuti (2014) compatibility of ahp/anp vectors with known results and saaty, l.t. (2010), group decision making: drawing out and reconciling differences. it is also interesting to read about the jaccard index in jaccard, p. (1901) distribution de la flore alpine dans le basin des dranses et dans quelques regions voisines) since the reader may find that the g index seems to be a mathematical generalization of the j index (point to point). table 1 shows the meaning of ranges of compatibility in terms of index g and its description. table 1 ranges of compatibilities and its meaning degree of compatibility compatibility value range (g%) description compatible very high ≥ 90% very high compatibility compatibility at cardinal level (compatible vectors) yes high 85 – 89.9 high compatibility (almost compatible vectors) moderate 75 – 84.9 moderate compatibility (not compatible vectors) no low 65 – 74.9 low level of compatibility (not compatible vectors) very low 60 – 64.9 very low compatibility (almost incompatible vectors) null (random) < 60% random level of compatibility (totally incompatible vectors) ijahp: garuti/reflections on common misunderstandings when using ahp and a response to criticism of saaty’s consistency index international journal of the analytic hierarchy process 494 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.573 the basic example (the criticism): the critic says, “the consistency index of ahp (saaty’s index) is wrong since it may let some values (comparisons) pass that are not acceptable by common sense”. description of the example: suppose there are three equal bars of the same length like the ones in figure 1. a b c figure 1 bar length of course, the correct pair-comparison matrix (pc matrix) for this situation is the following (consistent) pcmatrix: figure 2 bar comparisons 1 the obvious (correct) priority vector “w” is given by w= {1/3, 1/3, 1/3} with 100% consistency (cr=0). this is because the bars are all equally long. suppose now that (due to some visualization mistake) the new appreciation about the bars is as shown in figure 3. figure 3 bar comparisons 2 the new perturbed priority vector is: w* = {0.4126, 0.3275, 0.2599}, with cr = 0.05 (95% of consistency) which according to the theory is the maximum acceptable cr for a 3x3 pcmatrix. also, 2 is the maximum possible value if we want to stay within 95% consistency. the critic claims that the a-c bar comparison has a 100% difference (100% of error) which is not an acceptable or tolerable error (easy to see even with the naked eye). also, the global error (deviation) in the priority vectors is 15.85%, calculated with the common formula: e=abs(w*w), for each coordinate and then added over the ijahp: garuti/reflections on common misunderstandings when using ahp and a response to criticism of saaty’s consistency index international journal of the analytic hierarchy process 495 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.573 coordinates. but, saaty’s consistency index says that cr=95% (or 5% of inconsistency), is a tolerable limit for a 3x3 pc matrix. hence, the critic claims that saaty’s consistency index is wrong (useless and mathematically unsound to precisely quote the critic). the response: the critic misunderstands two important things. first, the cr=ci/ri (the saaty’s index of consistency) comes from the eigenvalue-eigenvector problem, so it is a systemic approach. thus, it is not concerned with any particular comparison, like comparison in the (1, 3) matrix position in this case. second, the possible error should be measured by its final result (the resulting metric) not in the prior or any middle step. the first misunderstanding is self-explanatory (systemic approach). for the second one, before any calculation, we need to understand what kind of numbers we are dealing with (in what environment we are working) because it is not the same for errors and deviations to be closer to a big priority than to a little one. this is a weighted environment and the measure of closeness (proximity) and thus possible errors has to be considered in this situation. we must work in the order topology domain to correctly measure the closeness of two vectors or rule of measurement on this environment. to do this task correctly, two aspects of the information have to be considered, the intensity (the weight or priority) and the degree of deviation between the two priority vectors (geometrically it can be seen as the projection between the vectors). the only index that takes good care of these two factors simultaneously is the compatibility index g. the explanation: graphically we are trying to make the following reasoning by demonstrating the reduction to the absurdum. figure 4 compatible rule of measurements if saaty’s consistency index is wrong, then the reference and perturbed metrics (w and w*) cannot be compatible (cannot measure the same), since we know one is correct (w) and the other is supposed to be wrong (w*). ijahp: garuti/reflections on common misunderstandings when using ahp and a response to criticism of saaty’s consistency index international journal of the analytic hierarchy process 496 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.573 summarizing the vectors of the correct and perturbed metric: correct metric (priority vector w) : 0.3333 0.3333 0.3333 perturbed or approximated metric (priority vector w*) : 0.4126 0.3275 0.2599 thus, the basic question is, “how close is the perturbed metric to the correct metric?” to respond to that question it is necessary to evaluate the similarity (or closeness) of the two priority vectors, and this job is correctly performed by the index of compatibility (g). assessing g (correct vs perturbed), the value obtained is: g=85.72% which in numerical terms represents almost compatible metrics. (see table 1). g=90% is a threshold to consider two priority vectors as compatible vectors. also, g=85% is an acceptable lower limit value. hence, the two metrics are relatively close (close enough considering that they are not physical measures). we have shown that both metrics are compatible that is they are measured almost with the same rule (as shown in figure 4). one rule cannot be correct and the other wrong if they are compatible rules. thus, the second rule (w*) is an acceptable rule of measurement, and by reduction to the absurdum, the criticism about saaty’s consistency index is incorrect. it is important to note that the same exercise was performed with 4x4 to 9x9 pc matrices. the value (n-1) was put in the position cell (1, n), (n= matrix dimension) and even better results were obtained for the compatibility index g than in the 3x3 dimension matrix. this is shown in the cells in bold in table 2. table 2 compatibility indices for perturbed matrices from range 3x3 to 9x9 the outcome of table 2 is not a surprise since it comes from a matrix built within an intrinsic systemic behavior. the pair comparison process in the matrix produces highly inconsistency 5% 3x3 0,333333333 0,333333333 0,333333333 1 (1,3)==>2 0,4126 0,3275 0,2599 1 0,30131416 0,324634375 0,231272015 85,7% 6% 4x4 0,25 0,25 0,25 0,25 1 (1,4)==>3 0,331 0,2407 0,2407 0,1888 1,0012 0,219410876 0,23622298 0,23622298 0,16569088 85,8% 6% 5x5 0,2 0,2 0,2 0,2 0,2 1 (1,5)==>4 0,277 0,1906 0,1906 0,1906 0,151 0,9998 0,172202166 0,1861209 0,1861209 0,1861209 0,1325025 86,3% 5% 6x6 0,1667 0,1667 0,1667 0,1667 0,1667 0,1667 1 (1,6)==>5 0,2392 0,1582 0,1582 0,1582 0,1582 0,1279 0,9999 0,14139725 0,15418172 0,15418172 0,15418172 0,15418172 0,11302523 87,1% 4% 9x9 0,111111111 0,111111111 0,111111111 0,111111111 0,111111111 0,111111111 0,11111111 0,11111111 0,11111111 1 (1,9)==>7 0,1717 0,1055 0,1055 0,1055 0,1055 0,1055 0,1055 0,1055 0,0897 0,9999 0,091506863 0,102836125 0,102836125 0,102836125 0,102836125 0,102836125 0,10283613 0,10283613 0,08105741 0,89241714 89,2% ijahp: garuti/reflections on common misunderstandings when using ahp and a response to criticism of saaty’s consistency index international journal of the analytic hierarchy process 497 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.573 related elements among the pairs. when searching for the equilibrium point of the matrix (the eigenvector that represents the metric of the matrix) this process of relations and interconnections can be perceived as a growing complex system (as the graph theory shows). thus, the analysis of the quality of the consistency index must be done considering this relevant fact (complex system) with many connections and redundancies (garuti, c., salomon, v. & spencer, i., 2008). redundancies (redundant judgments) are necessary and very important because they give more reliability to the system (any system without redundancy is a vulnerable or fragile system). in addition, these redundancies give more stability to the system because they allow for an acceptable result even with a cell in the matrix having a very bad pair comparison value. when a system allows for redundancies it has the capacity to receive new information that may or may not be consistent with the old one. this characteristic allows the system to evolve by connecting old and new data in a peaceful way. for instance, in table 2 for the case of the 9x9 matrix in position (1, 9), there is a 8 instead of 1; that is a “very large error” of 800%, (following the bar example, the bar would be visualized as 8 times bigger on this specific comparison) and in spite of that we still obtained a healthy outcome of 89.2% for compatibility index g (even better than the rest of the g values). even more, when performing the hypothetic case for n=15 (15x15 matrix) with a value of 14 in the cell position (1, 15), (a huge 1400% error), the outcome for compatibility index g is 99.96% (almost 100% of compatibility). this means that both vectors (correct and perturbed) are almost identical in terms of measurement. thus, the trend of compatibility clearly shows that the divergence in the value of a cell (1, n), (or any other cell by the way) does not produce any decay in the quality of the generated metric. by the way, the saaty´s consistency index for this case was 3%, a very good consistency index for a 15x15 matrix (a very large and inadvisable matrix). it is interesting to know what happens if we leave “n” (the size of the matrix and the value for the pair comparison position (1, n)) free to grow beyond 9. in this case, the acceptable error in saaty’s consistency index may diverge (go to infinity). this means that in a very large pc matrix you can put a pair comparison number as big as you like and still get an acceptable ratio of consistency. this abnormal behavior for saaty’s consistency index is also revealed by the compatibility index g. as “n” the size of the matrix and the pair comparison value in the (1, n) position increase, g decreases making the priority vector (the final or perturbed metric) more and more incompatible with the reference vector. for instance, in a 15x15 pc matrix the value for cell (1, 15) can be as large as 50 (5000% “error”), a very large value (beyond one order of magnitude) and still have an acceptable consistency (10%). for this case, the output value for g is 79.7%, which according to table 1 is not an acceptable value for the quality test (not compatible vector). nevertheless, it is interesting to understand what it means to leave “n” free to grow in a weighted environment. we have to remember that our reference vector or metric is defined by: {1/n, 1/n… 1/n}, then, as n grows the reference vector tends toward the vector {0, 0... 0}, the null vector. the null vector (or zero vector) is not a point of reference for anything in a weighted environment (using this vector as a reference point is like dividing by zero in a mathematical demonstration). thus, the useful mathematical concept of limit analysis ijahp: garuti/reflections on common misunderstandings when using ahp and a response to criticism of saaty’s consistency index international journal of the analytic hierarchy process 498 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.573 behavior is not applicable for this situation. in conclusion, we cannot leave “n” free to grow as presented in the demonstration that saaty’s consistency index is wrong. it is also interesting to reflect here about the great relevance of axiom 2 of ahp (the homogeneity axiom) which states that one must not make comparisons beyond one order of magnitude. this problem or inconsistency is possible in very large matrices (that is, when we leave “n” free to grow). there are several important conclusions that can be made from this example. from the first misunderstanding, that the cr=ci/ri (the saaty’s index of consistency) comes from the eigenvalue-eigenvector problem and thus is a systemic kind of approach we are not concerned with any particular comparison but the full pc-matrix (the whole set of comparisons). also, redundancies are important and need to be captured (as in any systemic approach, redundancies are fundamental in the system behavior). for the second misunderstanding that says the possible error should be measured by its final result (the resulting metric) not in the prior or any middle step, it is important to note that we need to understand what kind of numbers we are dealing with (what kind of environment are we working in before any calculations. this is because (and just as an example) for errors and deviations it is not the same to be close to a big priority than to a little one. this is a weighted environment and the measurement of the closeness (proximity) and thus possible errors has to be considered in this situation. so, we cannot just determine the difference of coordinates between two vectors to find their closeness. moreover, in general, mcdm belongs to the order topology domain, and we must work in this domain to adequately measure the closeness in this environment. to do this task correctly, two aspects of the information have to be considered, the intensity (the weight or priority) and the degree of deviation between the two priority vectors (the projection between the vectors). the only index that considers these two factors simultaneously is the compatibility index g. additional caveats: 1. when possible, all pair comparisons in the matrix have to be done, and all of the pair comparisons have to take into account the rights and the wrongs, which by the way are indistinguishable, to correctly assess consistency and priority (the weighted metric of the matrix). this is where the eigenvector operator (and its principal eigenvalue) perform the best. 2. the analysis of the behavior of an isolated element to characterize the whole system (this would be a kind of basic mechanical analysis) is not valid. the pc-matrix represents a highly related system. thus, it is not possible to evaluate a complex system of behavior (the pc matrix) by the behavior of only one of its elements; there are redundancies that are not well captured in the one isolated element analysis. 3. if you want to have a representative consistency index (without a very large bad comparison), you should never go beyond a matrix size of 9 (9x9 matrices) in any pc matrix. this is because it is very probable that you may have comparisons beyond 9 inside the matrix as the size of the matrix increases. by the way, this is aligned with axiom 2 of ahp which is in keeping with the homogeneity factor (not going beyond one order of magnitude between the elements to be compared). breaking axiom 2 ijahp: garuti/reflections on common misunderstandings when using ahp and a response to criticism of saaty’s consistency index international journal of the analytic hierarchy process 499 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.573 may produce a loss of consistency as well which is another source of error that normally comes from a poor modelling process. 4. it is not possible to apply the common formula for error measurement as: e = abs(w*w) within a weighted environment since it is not the same to be close in a high weighted element as in a low one. (this common formula works ok in the euclidean or flat space, but not in a weighted one). 5. finally, it seems that the threshold of 10% of saaty’s consistency index could be too lax for some cases. indeed, this threshold is 5% for 3x3 matrices and if we want to keep an acceptable level of compatibility the threshold should not go beyond 6 or 7% in matrices of superior order (instead of the current 10%). nevertheless, this also depends on the kind of problem being solved. however, we think that more investigation and numerical tests should be carried out in this line of research. a final important comment about consistency: of course, better consistency (100% for instance) can always be achieved. the question is, “do we really obtain a better result when being totally consistent?” the answer is probably no. because, in real problems we never have the “real” answer (the true metric “w” to use as reference). experience shows that pursuing consistent metrics per se may provide less sustained results. for instance, in the presented problem, one could answer that: a/b=2, a/c=2, and b/c=1, as shown in figure 5, and he/she would be totally consistent (but consistently wrong). figure 5 total consistent pairwise matrix in figure 5, the new priority vector is w**= (0.5, 0.25, 0.25), with cr=0 (totally consistent), and g= 71.5%, which means not compatible vectors (low compatibility). thus, a totally consistent metric is incompatible with the correct result. hence, in the end it is better to be approximately correct than consistently wrong. the consistency index is just a thermometer not a goal. it is also important to note the fact that the quality of the metric of any pc-matrix is not found in some specific pc judgment of the matrix. it should be found in its final interaction, which means after all the relations and redundancies have played their part in the search of an equilibrium point of the system (the principal eigenvector). therefore, defining the quality of the metric that comes from a pc-matrix directly from the matrix values (the judgments), as presented in this criticism example, is not a good idea. ijahp: garuti/reflections on common misunderstandings when using ahp and a response to criticism of saaty’s consistency index international journal of the analytic hierarchy process 500 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.573 4. conclusions the list of errors that have been presented have to be seen not only as errors produced by misunderstandings, but also as a list of good practices for creating a model. they can help show how to correctly use saaty’s fundamental scale and also show how to know what kind of scale is being created and thus what can and cannot be done with that scale. the bar example presented in the paper shows that systemic behavior cannot be analyzed by its elements in a separate way. the pc matrix is a system and the pair comparison judgments are its elements. the g index shows that one poor comparison (very poor indeed) can be present and an acceptable quality metric can still result (an acceptable priority vector). the relevant conclusions from this example were presented earlier in the paper and are a clear rebuttal for the criticism that claims saaty’s consistency index is useless and mathematically unsound. ijahp: garuti/reflections on common misunderstandings when using ahp and a response to criticism of saaty’s consistency index international journal of the analytic hierarchy process 501 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.573 references garuti, c. (2012). measuring in weighted environments: moving from metric to order topology. in f. de felice, a. petrillo, t. saaty (ed.), applications and theory of analytic hierarchy process decision making for strategic decisions (pp. 247-275) santiago, chile: universidad federico santa maria. doi: 10.5772/63670 garuti, c. (2014b). compatibility of ahp/anp vectors with known results. isahp2014 garuti, c., salomon, v., spencer, i. (2008). a systemic rebuttal to the criticism of using the eigenvector for priority assessment in the analytic hierarchy process for decision making. computación y sistemas (review), 12(2), 192-207. garuti, c escudey m. (2005). toma de decisiones en escenarios complejos. santiagochile: editorial universidad de santiago publications. jaccard, p. (1901). distribution de la flore alpine dans le basin des dranses et dans quelques regions voisines. bulletin de la société vaudoise des sciences naturelles 37, 241-272. doi: http://dx.doi.org/10.5169/seals-266440 mahalanobis, p.c., (1936). on the generalized distance in statistics. proceedings of the national institute of science of india, 12, 49-55. saaty, l.t. (2001). the analytic network process: decision making with dependence and feedback. pittsburgh, pa: rws publications. saaty, l.t. (2010). group decision making: drawing out and reconciling differences. pittsburgh, pa: rws publications. doi: https://doi.org/10.13033/ijahp.v9i3.533 koczkodaj, w.w., szybowski, j. (2015). on the convergence of the pairwise comparisons inconsistency reduction process. ijahp news and events: mu/ijahp team presents at informs 2017 international journal of the analytic hierarchy process 409 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.531 ijahp team presents at informs 2017 submitted by valentina ferretti enrique mu ijahp news editor, dr. valentina ferretti, wins award during october 2017 at the informs annual meeting, which took place in houston, our ijahp news and events section editor, dr. valentina ferretti, won the 2017 informs decision analysis practice award. the das/sdp (decision analysis society/society of decision professionals) practice award is a joint award given annually to the best decision analysis application, as judged by a panel of society members. the intent of this award is to recognize, promote, and publicize good decision analysis practice. after obtaining the runner up position for the informs 2016 decision analysis practice award with a work titled, “from spatial swot analysis to mcda and choice experiments: an integrated approach for historical heritage management in a new world heritage site” (authors: valentina ferretti and elisa gandino), valentina decided to try again and applied for the 2017 edition of the award with a new decision analysis intervention. this second project titled, “how to improve educational programs for underprivileged children? the impacts of value-focused decision analysis” was authored by valentina and gabriella csányi, one of her students from the master’s in decision science program in the department of management of the london school of economics and political science. with this project, valentina and gabriella supported a foundation in hungary in developing a selection model for underprivileged children to be welcomed in the foundation’s educational program. the developed framework allowed the board of the foundation to define a set of shared objectives to be achieved in the selection process, and is expected to decrease dropout rates of children from the program. this project will also be featured at the 2018 annual meeting of the decision analysis affinity group (sdp’s annual conference). valentina (left) and gabriella (right) together with dr. greg hamm, the co-chair of the 2017 decision analysis practice award committee ijahp news and events: mu/ijahp team presents at informs 2017 international journal of the analytic hierarchy process 410 vol. 9 issue 3 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i3.531 gabriella (left) and valentina (right) at the informs meeting ahp/anp topics at informs 2017 on a different note, dr. enrique mu, ijahp editor-in-chief, presented two studies along with co-authors orrin cooper and michael peasley about anp best practices and the implications of coherence (this last study led by dr. cooper). this session was dedicated to anp, and also featured dr. gabriela sava who discussed whether a shared-decision making model for colorectal cancer screening could lead to better medical outcomes. from left to right, enrique mu, gabriela sava and orrin cooper ijahp article: jantscher, schwarz, zinser/ decision support in it service management: applying ahp methodology to the itil incident management process international journal of the analytic hierarchy process 104 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.284 decision support in it service management: applying ahp methodology to the itil incident management process martin jantscher fh joanneum university of applied sciences graz, austria e-mail: martin.jantscher@edu.fh-joanneum.at christopher schwarz fh joanneum university of applied sciences graz, austria e-mail: christopher.schwarz@fh-joanneum.at erwin zinser 1 fh joanneum university of applied sciences graz, austria e-mail: erwin.zinser@fh-joanneum.at abstract the order in which service engineers in the field of it service management decide to resolve incidents is crucial considering the impact it has on the business performance of the it service provider. a possible solution to reduce negative impacts is to support service engineers by means of a decision support system that calculates priorities for the incidents based on their business impact. considering these priorities in their decisions, service engineers can help reduce such negative impacts on the it service provider’s business. the aim of this study was to incorporate the analytical hierarchy process (ahp) method into a software tool to prioritize incidents according to the severity of their business impact. as a result, a decision support system called incident prioritizer (ip) was developed. it uses the ahp method to calculate priorities for incidents based on typical criteria that are relevant in the field of it service management to assess the business impact of incidents. these criteria are commonly defined by the management of an it service provider organization, and reflect the management’s understanding of the business impact that is caused by incidents. the results suggest that the ahp method can be successfully applied for the given problem of incident prioritization. the prioritization is facilitated by an ahp decision model that consists of only one level of criteria and considers judgments from human beings as well as data from external information systems. 1 corresponding author ijahp article: jantscher, schwarz, zinser/ decision support in it service management: applying ahp methodology to the itil incident management process international journal of the analytic hierarchy process 105 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.284 keywords: analytic hierarchy process, it service management, decision support system, incident prioritizer, itil. 1. introduction and objectives in the era of cloud computing the management of information technology (it) infrastructure becomes an ever increasing challenge. large it service providers (isp) in particular have to find ways to manage complex, centralized and large-scale it infrastructure. the service providers are supported in their endeavor by consulting best practice guidelines on managing it infrastructure effectively such as the it infrastructure library (itil). it is a common practice for it service providers to issue contracts to customers that specify the it service offered as well as numerous service level targets that the it service provider must meet. furthermore, the contract specifies penalties that are due in case the it service provider fails to meet those service level targets. moreover, it is common for it service providers to employ service engineers that are responsible for resolving incidents. incidents are basically interruptions of it services which have a negative impact on the business performance of affected customers. these interruptions can have various causes such as hardware failure or software crashes. large it service providers face dozens of incidents each day that often occur at the same time. typically, service engineers decide in which order they resolve those incidents. these decisions are made without any information on the penalties that are caused by the incidents, even though the order in which service engineers decide to resolve incidents can have a severe negative impact on the business of it service providers (beims, 2010). a possible solution to close this information gap is to provide the service engineers with a decision support system that prioritizes incidents based on the severity of their business impact. thus, the overall objective of this study is to incorporate the analytical hierarchy process (ahp) method into itil-compliant software to prioritize incidents according to the severity of their business impact. the ahp is a widely accepted approach for decision making in different domains. consequently, it is also an adequate approach for the purposeful prioritization of incidents. the objective of this study was achieved by developing a software tool called incident prioritizer (ip). the ip uses the ahp method to calculate priorities for incidents based on well-defined decision criteria. it is a decision support system that supports the aforementioned decision process of service engineers by prioritizing all open incidents according to the severity of their business impact. to assess the business impact of incidents multiple criteria are taken into account. these criteria are commonly defined by the management of an it service provider. the output of the ip is prioritized incidents. the calculated priorities represent the incidents’ business impact. when service engineers consider these priorities in the decision process, it can help reduce negative impact on the business of the it service provider. this paper describes how exactly the ahp method is applied in the context of the incident prioritizer software. moreover, this paper will unveil implementation details of ijahp article: jantscher, schwarz, zinser/ decision support in it service management: applying ahp methodology to the itil incident management process international journal of the analytic hierarchy process 106 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.284 the incident prioritizer with emphasis on the technical implementation of the ahp method. 2. literature review it is a well-known fact in the field of it service management (itsm) that incidents have a negative impact on the business performance of it service providers. the itil guideline thus recommends that it service providers prioritize their incidents based on their business impact (ogc, 2007). business impacts are factors like financial losses or the number of customers that are affected by an incident. in addition, beims (2010) states that the order in which service engineers decide to resolve incidents can also contribute to a negative impact on the business of the it service provider. simulation modeling provides an efficient means in an itsm context to support decisionmaking (orta et al., 2014). numerous simulation techniques exist to address different fields of application aiming at prominent modules of the itil itsm framework, i.e. service strategy, service design, service transition, and service operation. among them, state-based process models, discrete event simulation, petri-net models as well as reliable mathematical simulation techniques are commonly applied (power, 2002). the common denominator of these approaches turns out to be performing prediction with respect to system behavior in order to improve the overall performance of itil-related processes. however, many of these simulation experiments rely on synthetic data that is generated by simulation software and, thus, they do not necessarily reflect real-world scenarios. furthermore, until recently, ahp was not yet extensively used to support decisionmaking regarding itsm. repschlaeger et al. (2014) successfully applied ahp for provider selection of a cloud-based it service management system. wan et al. (2011) determined various indicators of it service management process where weights of each indicator were calculated by the ahp methodology. moreover, wan and wan (2012) made use of ahp in order to improve the overall performance of global companies with respect to their it service management processes. incident management as part of the service operation module of the itil framework deals with managing and restoring normal service operation after an interruption to minimize severe impacts on the business (office of government commerce, 2011a). hence, for a systems engineer, making the correct decision is crucial in order to decide which incident has to be dealt with first in order to keep the impact on the respective business as minimal as possible. therefore, we applied ahp methodology to support the prioritization process regarding incident management based on well-defined criteria. furthermore, we developed a software tool relying on innovative cloud technology which delivers decision-support based on the ahp prioritization algorithm. there is a lot of itil-compliant software commercially available that is used by it service providers to support their business (itil, 2014). however, to the best of our knowledge, no itilcompliant software exists that relies on a robust mathematical foundation for prioritizing it-related incidents. ijahp article: jantscher, schwarz, zinser/ decision support in it service management: applying ahp methodology to the itil incident management process international journal of the analytic hierarchy process 107 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.284 3. methodology the incident prioritizer (ip) software calculates priorities for incidents on the basis of an ahp decision model. this model has the goal of finding the incident with the greatest business impact. thus, the decision model considers incidents as its alternatives. it is important to understand that the decision model used in the ip is not static. it varies depending on the management’s preferences regarding criteria as well as on the current open incidents of an it service provider organization. the criteria that can be used in the decision model have been derived from expert knowledge. the expert is working at a large it service provider organization in austria that serves customers all over the globe. based on interviews with this expert the following four criteria were developed:  noofcustaffected – the number of customers affected by an incident.  isimpcustaffected – is a customer marked as important affected by an incident?  penalty – the total penalty caused by an incident.  remainingtime – the time remaining until a service level target is violated. the incident prioritizer software allows the management of the it service provider to choose from the aforementioned four criteria the ones they think influence the business impact of incidents most. the chosen criteria are then considered in the ahp decision model for the calculation of priorities. the judgments of the criteria with respect to the goal have to be carried out on an individual basis by the management of the it service provider that is using this decision support system. this step is carried out via the so called frontend module of the ip software (see section 5.1). the judgments made there will reflect the management’s understanding of what contributes most to the business impact caused by incidents. the judgments of the alternatives (incidents) with respect to the criteria are carried out automatically by the ip software based on so-called criteria data. the term criteria data refers to numerical data that is processed by the ip software to obtain judgments for alternatives with respect to upper level criteria. 4. decision model analysis as already mentioned in the previous section, the ip software uses an ahp decision model to calculate priorities for incidents. figure 1 shows a representative example of an ahp decision model that can be successfully processed by the ip software to calculate priorities. ijahp article: jantscher, schwarz, zinser/ decision support in it service management: applying ahp methodology to the itil incident management process international journal of the analytic hierarchy process 108 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.284 figure 1. representative example of an incident prioritizer compliant ahp decision model the decision model shown in figure 1 with its criteria, their weightings and the alternatives was defined by the authors of this paper. the weightings, the alternatives and their related data are all fictional and arbitrary. thus, the example decision model does not claim to be an accurate model to provide a real world it service provider with optimal incident priorities, but to be a valid model that helps explain how the incident prioritizer software functions. however, as already mentioned, in a real world application of the incident prioritizer the management of an it service provider organization will define the criteria, their weightings and the alternatives according to their business goals. the example decision model in figure 1 assumes that a service provider’s management decided to consider all of the four well-defined criteria in the prioritization process. the criteria weightings that the management made in this example are shown in brackets underneath the names of the criteria (e.g. 0.76 is the weight for the criterion “penalty”). the criteria weightings were obtained using saaty’s concept of pairwise comparison. the ip software provides a graphical user interface called “frontend module” that enables its users to pairwise compare criteria and thus enables them to define criteria weightings (see section 5.1). in this example, the management puts the highest weight on the criterion “penalty” when it comes to assess the business impact of incidents. figure 1 shows three incidents as alternatives that have to be prioritized (i.e. “inc 1”, “inc 2” and “inc 3”). the white boxes that are connected to the upper level criteria represent the so-called criteria data of the incidents (e.g. 3, 5, and 2 for the criterion “noofcustaffected”). the content of the top left white box in figure 1 has the following meaning: incident 1 (“inc 1”) affects 3 customers. as compared to the other two incidents and how many customers they affect, the priority of incident 1 with respect to the criterion “noofcustaffected” is 0.3. the final priorities of incidents are also shown in brackets underneath the name of the incidents (e.g. 0.57 is the final priority for incident “inc 3”). all calculations of priorities are carried out by the incident prioritizer software. as already mentioned in section 0, the judgments of the alternatives with respect to the criteria do not come from human beings. instead, the judgments are based on the aforementioned criteria data, and are obtained by putting the criteria data of incidents in relation to each other. for instance, let’s compare inc1 to inc2 with respect to the criterion “noofcustaffected” from figure 1. the criteria data for incident 1 with respect ijahp article: jantscher, schwarz, zinser/ decision support in it service management: applying ahp methodology to the itil incident management process international journal of the analytic hierarchy process 109 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.284 to “noofcustaffected” is 3. the criteria data for incident 2 is 5. the judgment is obtained by dividing the criteria data of incident 1 by criteria data of incident 2 (3/5). this step is carried out by the incident prioritizer software. the following section will provide details on the technical implementation of the incident prioritizer software. 5. implementation details the incident prioritizer is part of a software architecture that involves state-of-the-art cloud computing technology. figure 2 illustrates the complete software architecture of which the incident prioritizer is a part. figure 2. overall architecture of the incident prioritizer software figure 2 exhibits a hybrid cloud architecture that consists of a private and a public cloud. both clouds incorporate a microsoft service bus component. the private cloud additionally contains the modules “frontend”, “incident prioritizer”, “sharepoint” and “ontology”. the modules “frontend” and “incident prioritizer” are connected to the service bus in the private cloud. the module “incident prioritizer” is also connected to the service bus in the public cloud as is the “ontology” module. the service bus exhibits a publish/subscribe messaging behavior. it orchestrates the communication between modules that are connected to the service bus. the connected modules communicate via so called “topics”. in figure 2 the topics are indicated with the “t” letter. an extract of the incident prioritizer data flow illustrated in figure 3 will help explain how the service bus works in the context of the incident prioritizer. ijahp article: jantscher, schwarz, zinser/ decision support in it service management: applying ahp methodology to the itil incident management process international journal of the analytic hierarchy process 110 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.284 figure 3. extract of the data flow within the ip software architecture in step 1 of figure 3 the frontend publishes a calculatepriorities() message on the incident prioritizer topic of the service bus located in the private cloud. in step 2, the ip module receives this message due to its subscription on the incident prioritizer topic. upon receiving the calculatepriorities() message, two actions are triggered within the ip module. the first action is a response message (step 3) that is published back to the topic to inform the frontend (step 4) that the calculation is in progress. the second action spawns a process thread within the ip module that carries out all the further steps of the calculation. in figure 3 those calculation steps start with step 5 by querying the ontology topic in the public cloud for incidents. since the ontology module located in the private cloud is subscribed to the ontology topic in the public cloud, it receives the incident query in step 6. in step 7 it responds by publishing the requested list of incidents back on the ontology topic where the ip module is also subscribed to. in step 8 the ip module receives the list of incidents and continues its calculation process by querying the sharepoint module directly via its interface for criteria in step 9. in step 10 the sharepoint module responds back to the ip module with a list of criteria that the ip module will consider later in its calculation process. the results of the priority calculations of the ip have to be requested by the frontend module. this step is not shown in figure 3. this centralized communication via a service bus has certain benefits such as a unified communication interface for each module connected to the service bus. figure 2 and figure 3 exhibit a service bus in the public cloud. it shows that both the ip and the ontology module are located in a private cloud and are interacting with the ontology topic in the public cloud. while the given problem of prioritizing incidents does not necessarily demand such a sophisticated hybrid cloud architecture it could be valuable for other business scenarios, e.g. collaboration between two companies, or outsourcing of noncritical business information into the cloud for cost cutting reasons. ijahp article: jantscher, schwarz, zinser/ decision support in it service management: applying ahp methodology to the itil incident management process international journal of the analytic hierarchy process 111 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.284 the architecture shown in figure 2 and figure 3 features the following technologies for its components:  service bus (public cloud) – windows azure cloud 2  service bus (private cloud) – windows azure pack for windows server  incident prioritizer module – windows service  sharepoint module – microsoft sharepoint 2013  frontend module – windows 8.1 app  ontology module – java-based web service while the ip module in this architecture interacts with the ontology via the public cloudbased service bus and sharepoint, the description of those two modules is beyond the scope of this paper. upcoming publications will deal with these topics in detail as well as a description of the frontend module. details on the service bus in the private and public cloud will also be published separately. after describing the overall architecture with its data flow mainly coordinated via the service bus, the following sections will outline the modules of the architecture shown in figure 2 in more detail. emphasis will be placed on the incident prioritizer module since it is the software that incorporates the ahp method that is the focus of this paper. 5.1 frontend module the frontend module provides a graphical user interface for using the functionality of the incident prioritizer. technically, it is a windows 8.1 application that can be used to trigger the calculation process of the incident prioritizer and to display the resulting incidents and their priorities. figure 4 shows a screenshot of the view on the incidents. figure 4. frontend – incident list 2 microsoft, sharepoint, windows, windows server, and windows azure are either registered trademarks or trademarks of microsoft corporation in the united states and/or other countries. ijahp article: jantscher, schwarz, zinser/ decision support in it service management: applying ahp methodology to the itil incident management process international journal of the analytic hierarchy process 112 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.284 figure 4 illustrates a list of prioritized incidents. it shows that the incident with the id 73 has the name “inc1-antispam” and a priority of 5. the priority scheme used here ranges from 1 being the highest and 5 being the lowest priority. this information screen is highly relevant for the service engineer’s decision about which incident needs to be resolved first. the frontend module also allows forecasting priorities of incidents up to 18 hours. the forecast screen is shown in figure 5. figure 5. frontend – forecast priorities the frontend module also provides the user interface to customize calculation parameters such as criteria and their priorities that should be taken into consideration in the calculation process. this interface is used by the management of the it service provider to express their understanding of the business impact of incidents. figure 6 shows a screenshot of this interface. ijahp article: jantscher, schwarz, zinser/ decision support in it service management: applying ahp methodology to the itil incident management process international journal of the analytic hierarchy process 113 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.284 figure 6. frontend – criteria customization the screen in figure 6 shows that four criteria are active and are to be considered in the prioritization process. the middle part of the screen shows the pairwise comparisons of the active criteria. the pairwise comparisons are based on saaty’s scale of absolute numbers (saaty, 2008), and result in criteria priorities. these criteria priorities are visualized in a pie chart in the right part of the screen. the calculation parameters defined here translate into the criteria level of the ahp decision model (as shown in figure 1) that is used later for the calculation of incident priorities. the frontend module communicates with the incident prioritizer module via the service bus component to trigger the calculation process, fetch data like the results of the calculation and change calculation parameters such as criteria. 5.2 incident prioritizer module this section describes the incident prioritizer module that was developed for this study with the goal of prioritizing incidents according to the severity of their business impact by using the ahp method. the ip module was developed in the programming language c# and its internal structure follows the idea of separation of duties. thus, the overarching task of applying the ahp method for prioritizing incidents based on calculations that require data from different sources was broken down into four distinct components. figure 7 shows these components. ijahp article: jantscher, schwarz, zinser/ decision support in it service management: applying ahp methodology to the itil incident management process international journal of the analytic hierarchy process 114 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.284 figure 7. components of the incident prioritizer in figure 7, the controller component defines the messages that can be used to control the ip module from outside by the frontend module (e.g. to calculate priorities). also it coordinates the data flow between the other three components. the datafetcher component is responsible for the communication between the ip module and the other modules that act as data sources for the ip, i.e. sharepoint and ontology. despite the name datafetcher, it does not exclusively fetch data from the named sources but also pushes back, for example, result data and priorities to the sharepoint module. furthermore, it acts as a provider for criteria data for the criteria “noofcustaffected” and “isimpcustaffected” that is fed into the ahpengine component. the sltcalculations component implements algorithms necessary to calculate criteria data based on service level targets information. to accomplish this, the sltcalculations component communicates with the datafetcher component to get service level target data from the ontology. like the datafetcher, the sltcalculations component acts as a provider for criteria data for the criteria “remainingtime” and “penalty”. the ahpengine component is responsible for calculating priorities based on data about incidents, criteria with their priorities and criteria data, and it uses the ahp method to obtain priorities. therefore, all the data available for an incident and its respective service are mapped into an incident prioritizer compliant ahp decision model that is described in section 4 before carrying out the calculation. since the focus of this paper is the application of the ahp method, the implementation of the ahpengine component of the incident prioritizer module will now be explained in more detail. a part of the ahpengine component is based on an open-source implementation of the ahp method. the open decision maker (odm) is an implementation of ahp that also includes a graphical user interface to design and calculate an ahp decision model. it was written in the programming language java. the authors of odm are bender, blocherer, rossmehl, rotter (odm, 2010). for the ahpengine component parts from the java source code of odm 1.0.1 were taken and translated into the programming language c# since this is the language in which the incident prioritizer module was developed. the ijahp article: jantscher, schwarz, zinser/ decision support in it service management: applying ahp methodology to the itil incident management process international journal of the analytic hierarchy process 115 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.284 free edition of the software tool “java to c# converter” was used to support this translation process. the odm implementation of ahp imposes limits regarding the maximal possible number of criteria that can be used in the decision model. according to the documentation included in the source code of odm 1.0.1, the maximum possible number of criteria is 24. the ahpengine component is represented by the c# class ahpengine. this class offers the single point of access to the ahp functionality provided by the odm ahp implementation within the incident prioritizer module. the ahpengine class imposes the additional limit that only one level of criteria can be used to determine priorities for incidents. thus, sub-criteria that are normally possible in the ahp method are not possible in the application of ahp in the incident prioritizer. the ahpengine class provides all methods necessary to set up the required ahp decision model for the task of incident prioritization. these methods are: (1) addcriterion() – to add a criterion to the decision model. (2) addalternative() – to add alternatives to the ahp decision model. in the case of the incident prioritizer implementation the alternatives are incidents. (3) setcomparisons() – to add the pairwise comparisons for criteria that are necessary to calculate priorities of the criteria. (4) setaltcomparisonsbasedondirectdata() – to add the criteria data (e.g. number of customers). (5) calculatepriorities() – to determine the final priorities of the alternatives (incidents) from the set up ahp decision model. figure 8 shows what these methods are used for in the context of the ahp decision model introduced in section 4. figure 8. ahpengine methods to set up an ahp decision model figure 8 shows the effect of the ahpengine methods that are used to set up an ahp decision model within the incident prioritizer software module. the (1) addcriterion() method is called for each criterion that should be considered in the prioritization process. the criteria to be used here are restricted to the four well-defined criteria that were ijahp article: jantscher, schwarz, zinser/ decision support in it service management: applying ahp methodology to the itil incident management process international journal of the analytic hierarchy process 116 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.284 introduced in section 0. the (2) addalternative() method is called for each alternative to be considered. in the context of the incident prioritizer software the alternatives to be considered are incidents. the example in figure 8 shows three incidents to be considered for prioritization that are abbreviated as “inc 1” to “inc 3”. the (3) setcomparisons() method is called once in each prioritization process. as already explained in section 4, pairwise comparisons of the criteria are carried out by the management of an it service provider. by calling (3) setcomparisons() the results of pairwise comparing the criteria are transformed into the criteria weightings (e.g. weight 0.08 for criterion noofcustaffected). the (4) setaltcomparisonsbasedondirectdata() is also called only once in each prioritization process. similar to the setcomparisons() method the setaltcomparisonsbasedondirect-data() method transforms the criteria data (e.g. 3, 5, 2 for noofcustaffected) after pairwise comparing them into priorities with respect to the upper level criterion (e.g. priorities 0.3, 0.5, 0.2 for upper level criterion noofcustaffected). this process of pairwise comparing the criteria data was already described in section 4 of this paper. once the decision model is fully built with criteria, alternatives and criteria data, the (5) calculatepriorities() method is called at last to obtain the final priorities of incidents (e.g. incident “inc 1” has the final priority 0.32). the final priorities obtained from the ahpengine are later translated into absolute numbers ranging from 1 (highest priority) to 5 (lowest priority). the translation to absolute numbers is performed in order to make the priority information easier to understand for the people that are using the incident prioritizer decision support system. furthermore, integers ranging from 1 to 5 perfectly match priority codes of the incident priority matrix as described elsewhere (office of government commerce, 2011a). the prioritized incidents can finally be viewed in the frontend module that was already introduced in figure 4. 5.3 sharepoint module in the context of the incident prioritizer, this module is used like a database. it stores the data for criteria and criteria weightings that are considered by the ip module in its ahpbased calculation of priorities. also the sharepoint module stores the results of the priority calculation process carried out by the ip module. figure 9 shows a screenshot of the sharepoint list “criterion”. it shows the four criteria that can be taken into consideration by the incident prioritizer. ijahp article: jantscher, schwarz, zinser/ decision support in it service management: applying ahp methodology to the itil incident management process international journal of the analytic hierarchy process 117 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.284 figure 9. sharepoint list “criterion” 5.4 ontology module the ontology module provides the incident prioritizer with so-called incident-related data. the term incident-related data is used within the context of the incident prioritizer and includes for example data about customers, contracts, services and service level targets that are affected by an incident. after querying this incident-related data from the ontology the incident prioritizer module processes it internally into criteria data. as already explained in section 4, criteria data is crucial to obtain judgments for the alternatives with respect to the criteria. the ontology module hosts an itil-compliant service catalog based on semantic technologies (office of government commerce, 2011b). in the discipline of it service management and it’s most popular framework itil, the service catalog is a central component that brings together the different perspectives on the it service that is provided by an it service provider. the perspectives include among others: customers, customer contracts with agreed service level targets, the it services offered to potential customers, the it services consumed by actual customers, the technical dependencies of the it services offered (e.g. hardware, software), etc. the concept and the implementation of this module were introduced by wagner (2012). figure 10 shows an extract of the service catalog model that is implemented by the ontology module. ijahp article: jantscher, schwarz, zinser/ decision support in it service management: applying ahp methodology to the itil incident management process international journal of the analytic hierarchy process 118 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.284 figure 10. extract of service catalog of ontology module the service catalog shown in figure 10 consists of entities of certain types. the entity “yac” for example is of type “customer” and the entity “antispam” is of type “service module”. each entity type has a defined set of data attributes. entities of type “customer” for example have the following set of data attributes: name, address, zip code, and country. the service catalog also models relationships between these entities. the entity “antispam” for example is related to the entity “sla”. by establishing relationships between the entities in the service catalog valuable business information is generated that can be further processed by other information systems such as the incident prioritizer module. in the case of the incident prioritizer, the ontology ijahp article: jantscher, schwarz, zinser/ decision support in it service management: applying ahp methodology to the itil incident management process international journal of the analytic hierarchy process 119 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.284 offers the highly valuable incident-related data that is necessary for generating criteria data. as described in section 4, criteria data is crucial for obtaining the judgments for alternatives (incidents) with respect to the criteria. technically, the incident prioritizer module queries the ontology for this information by using the sparql language that is designed to retrieve information from ontologies. 6. limitations so far, the developed decision support system has been tested with only three synthetic incidents. however, in a real business scenario the system needs to be able to handle dozens or even hundreds of incidents which might lead to a non-deterministic behavior of the software module. another limitation concerns the consistency of judgments made within the ip software. by design of the ip judgments are made on the criterion level by human beings as well as by the software itself on the alternatives level. hitherto, the ip does not check the consistency of the judgments on any of those two levels. 7. conclusions in this study a decision support system called incident prioritizer was developed. it implements the ahp method for the purpose of prioritization. in particular, the incident prioritizer is an itil-compliant software tool that can be used by it service providers to prioritize incidents based on the severity of their business impact. the incident prioritizer was presented as part of a software architecture that features state-of-the-art cloud computing technology. the incident prioritizer was developed in the programming language c# and requires three other software modules (frontend, ontology, sharepoint) for proper functioning. this paper introduced the ahp decision model that is used by the incident prioritizer software to calculate priorities for incidents. commonly, incident prioritizer compliant decision models are restricted to only one level of criteria. furthermore, the judgments for the criteria have to be carried out by the management of the it service provider whereas the judgments for the alternatives with respect to the criteria are carried out programmatically by the incident prioritizer on the basis of criteria data. future research should test the developed decision support system in a real business scenario to find out whether the software provides meaningful results when facing the challenge of prioritizing dozens of incidents. another suggestion for further research is to check for the consistency of judgments that are considered in the prioritization process. ijahp article: jantscher, schwarz, zinser/ decision support in it service management: applying ahp methodology to the itil incident management process international journal of the analytic hierarchy process 120 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.284 references beims, m. (2010). it-service management in der praxis mit itil 3. munich: hanser verlag. itil. (2014). endorsed software tools, http://www.itilofficialsite.com/softwarescheme/endorsedsoftwaretools/endorsedsoftwaretools.aspx (accessed march 31, 2014) office of government commerce (ogc). (2011a). itil® service design. office of government commerce (ogc). (2011b). itil® service operation. ogc. (2007). itilv3 service operation. norwich: the stationary office. odm. (2010). open decision maker user manual [pdf-file], http://sourceforge.net/projects/opendecisionmak/files/latest/download?source=files (accessed may 20, 2014) orta, e., ruiz, m., hurtado, n. and gawn, d. (2014). decision-making in it service management: a simulation based approach. decision support systems, 66, 36-51. power, d.j. (2002). decision support systems: concepts and resources for managers. westport, ct: quorum books. repschlaeger, r., proehl, r. and zarnekow, r. (2014). cloud service management decision support: an application of ahp for provider selection of a cloud-based it service management system. intelligent decision technologies, 8, 95–110. saaty, t.l. (2008). decision making with the analytic hierarchy process, international journal of services sciences, 1(1), 83–98. wagner, g. (2012). semantik itsm (master thesis). graz: fh joanneum university of applied sciences. wan, j. and wan, x. (2012). case study on m company best practice with global it management. technology and investment, 3, 143-148. wan, j., zhang, h. and wan, d. (2011). evaluation on information technology service management process with ahp. technology and investment, 2, 38-46. http://www.itil-officialsite.com/softwarescheme/endorsedsoftwaretools/endorsedsoftwaretools.aspx http://www.itil-officialsite.com/softwarescheme/endorsedsoftwaretools/endorsedsoftwaretools.aspx http://sourceforge.net/projects/opendecisionmak/files/latest/download?source=files ijahp article: kamps, jassemi-zargani/decision making in dynamic environments: an application of machine learning to the analytical hierarchy process international journal of the analytic hierarchy process 27 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.766 decision making in dynamic environments: an application of machine learning to the analytical hierarchy process caelum kamps drdc ottawa research centre canada caelum.kamps@drdc-rddc.gc.ca rahim jassemi-zargani drdc ottawa research centre canada rahim.jassemi@drdc-rddc.gc.ca abstract the purpose of this work is to propose a method of algorithmic decision making that builds on the analytical hierarchy process by applying reinforcement learning. decision making in dynamic environments requires adaptability as new information becomes available. the analytical hierarchy process (ahp) provides a method for comparative decision making but is insufficient to handle information that becomes available over time. using the opinions of one or many subject matter experts and the ahp, the relative importance of evidence can be quantified. however, the ability to explicitly measure the interdependencies is more challenging. the interdependency between the different evidence can be exploited to improve the model accuracy, particularly when information is missing or uncertain. to establish this ability within a decision-making tool, the ahp method can be optimized through a stochastic gradient descent algorithm. to illustrate the effectiveness of the proposed method, an experiment was conducted on air target threat classification in time series developing scenarios. keywords: threat assessments; ahp; machine learning; decision making 1. introduction during complex decision making for classification, the ability of a subject matter expert (sme) to reach a conclusion depends on the availability of information. in dynamic situations, where information is developing over time, the decision maker must be flexible as information becomes available or is lacking. air target threat assessment is such a situation. the available systems dictate the rate and amount of target information that accumulates. additionally, with the increasing amount of air traffic, it is not feasible for an individual or even a team to be able to evaluate and classify each target that is detected. the need for machine-aided classification is evident and increasing. it is desirable that the method of algorithmic classification be as follows: mailto:rahim.jassemi@drdc-rddc.gc.ca ijahp article: kamps, jassemi-zargani/decision making in dynamic environments: an application of machine learning to the analytical hierarchy process international journal of the analytic hierarchy process 28 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.766 1. adaptive to updated information 2. predictive of unknown information 3. reflective of the classification of a sme the analytic hierarchy process (ahp) is a general theory of measurement. the method facilitates deductive and inductive algorithmic decision making by comparing the relative importance of evidence (saaty, 1987). threat assessment (ta) is the fusion of many sources of information for the purpose of classifying and predicting the intent, capability, and proximity of detected entities. (johansson & falkman, 2008) this fusion of information is used to infer an associated target threat value (tv) for each entity at any given point in time. this work is particularly concerned with the application of algorithmic target threat assessment to airborne targets during states of uncertainty and deficient information. the objective of this paper is to present methods to improve the accuracy of the ahp network evaluations when not all the cues are known. the assumption is that under the complete information state (cis), the state where all the cues are known, the optimal decision or reference decision is the complete ahp network. the problem that is being addressed is what to do when not all the cues are known. how can the network be adapted to best predict the reference decision? given a state of some information, what are the optimal weights and best techniques to improve the accuracy of prediction for the correct threat classification? the proposed decision structure is a hierarchal decision tree. through the ahp, the nodes are each assigned a weight corresponding to their importance. machine learning techniques can then be used to optimize the node weights when not all the information is known. this structure can then be further improved by introducing special learning nodes called sigmoid nodes. this will be shown in the following sections covering the ahp, the scenario description, the proposed methodology, and finally, the experiments and results. 2. analytical hierarchy process the analytical hierarchy process (ahp) is a well-known algorithm for decision making support (saaty, 1987). it allows a sme to break the problem down into criteria groups to more easily compare evidences, called cues. the structure of the decision tree is shown in figure 1. ijahp article: kamps, jassemi-zargani/decision making in dynamic environments: an application of machine learning to the analytical hierarchy process international journal of the analytic hierarchy process 29 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.766 a sme is given a list of cues to partition into criteria. within each criteria, the cues are compared pairwise with one another. this information is stored in comparison matrices. assuming the sme is relatively consistent in their comparisons, a principal eigenvector analysis on the comparison matrices can be used to obtain normalized weights for each of the cues. once this is done, the same process is applied to the criteria to generate normalized criterion weightings. the structure then takes input from each of the targets or alternatives and makes a decision based on the previously defined weights. it is important to understand that since the weights are normalized, the sum of all the cue weights within a criterion is unity. similarly, the sum of the weights of each of the criterion is unity (saaty, 1987). currently, the ahp assumes that all the inputs are known. it relies on the ability of a sme to rank each of the cues with respect to others within the category. one of the limitations of the ahp is that it does not consider the relative importance of cues with respect to multiple criteria. for example, if one cue is used when considering two criteria, then its relative importance might not be the same in both cases. the analytical network process (anp) offers the ability to do these multiple comparisons. in both methods, all of the cues must be known to make an accurate decision. this is not what is being addressed here. in the anp and ahp community, dependency refers to the influence of the criteria under consideration on the importance of the cue (mu, 2006). the type of dependency that is being addressed by this paper is different. here we consider dependency in the calculation of the problem at hand. this means that cues might hold information about each other, either explicitly in their formulae for calculation or implicitly in correlation. the method proposed in this paper deals with accounting for the values of unknown cues that might have these dependency features with some of the known cues. 3. cues the calculation of some cues is dependent on others. for example, to calculate the estimated closest point of approach of a target to a defended asset, the orientation, range, and track history of the target must be known. the introduction of dependent cues creates potentially complex and unknown relationships between cues that are hypothetically accounted for during states of complete information but are not necessarily understood when not all the inputs are known. this is one of the challenges that will be addressed. here, the difference between dependent and independent cues can be figure 1 general ahp structure ijahp article: kamps, jassemi-zargani/decision making in dynamic environments: an application of machine learning to the analytical hierarchy process international journal of the analytic hierarchy process 30 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.766 established. cues that require that other information be known before a measurement can be made are dependent. conversely, cues that can be measured on their own are independent. table 1 shows the dependency and criteria for each of the cues that will be considered for the target threat assessment. table 1 target threat assessment cues cue criteria dependent/independent dependencies speed (sp) capability independent n/a altitude (al) proximity independent n/a range (r) proximity independent n/a orientation (o) intent independent n/a closest point of approach (cpa) capability dependent orientation & range angle of attack (aoa) intent dependent aoa & altitude time before cpa (tbcpa) proximity dependent cpa & speed using the cues from table 1, a hierarchal decision tree is built and shown in figure 2. figure 2 hierarchal decision tree to initialize the weights for each of the nodes in the decision tree using the ahp, four comparison matrices were generated and are shown in table 2. the rankings provided by a study of cues and information order are used to create generalized pairwise comparisons for each cue (liebhaber, kobus & feher, 2002). ijahp article: kamps, jassemi-zargani/decision making in dynamic environments: an application of machine learning to the analytical hierarchy process international journal of the analytic hierarchy process 31 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.766 table 2 comparison matrices for the hierarchy shown in figure 2 threat value (tv) intent proximity capability intent aoa orientation proximity 1 2 2 aoa 1 2 capability 1 1 orientation 1 intent 1 proximity capability alt tbcpa range speed cpa alt 1 2 3 speed 1 2 tbcpa 1 2 cpa 1 range 1 the weights generated through principal eigenvector analysis of the above comparison matrices are presented in table 3. table 3 weights of each criteria and cues criteria weight sub-criteria (cues) weight intent 0.25 orientation 0.33 aoa 0.66 proximity 0.5 range 0.54 altitude 0.30 tbcpa 0.16 capability 0.25 speed 0.66 cpa 0.33 4. information domain partitioning in real situations, information accumulates as a time series. not all information is known right away because it takes time to run tests and acquire data. to represent each of the combinations of known information, the information domain is partitioned into an information state space (iss). this is a list of every possible scenario that can occur with the information available. each of the scenarios is referred to as an information state (is). the number of information states is directly dependent on the number of independent cues. each is can be encoded as a binary string. if there are four independent cues, denoted 1, 2, 3, and 4, then the state in which cue 1 and 4 are known is 1001 or information state 9. clearly, any combination of cues can be represented as a 4-bit string. thus, there are 2 4 information states. the iss can now be explicitly defined. in table 4, the dependent cues receive a designation of 1 (known) if their dependencies are satisfied. ijahp article: kamps, jassemi-zargani/decision making in dynamic environments: an application of machine learning to the analytical hierarchy process international journal of the analytic hierarchy process 32 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.766 each is can be referenced either by the string that is the first four binary digits in its row of the iss table or by its state identification (id). table 4 iss table independent dependent state id speed altitude orientation range aoa cpa tbcpa 1 0 0 0 1 0 0 0 2 0 0 1 0 0 0 0 3 0 0 1 1 1 0 0 4 0 1 0 0 0 0 0 5 0 1 0 1 0 0 0 6 0 1 1 0 0 0 0 7 0 1 1 1 1 1 0 8 1 0 0 0 0 0 0 9 1 0 0 1 0 0 0 10 1 0 1 0 0 0 0 11 1 0 1 1 1 0 0 12 1 1 0 0 0 0 0 13 1 1 0 1 0 0 0 14 1 1 1 0 0 0 0 15 1 1 1 1 1 1 1 5. methodology 5.1 initialization and base cases to prove the validity of the method proposed in later sections, a base case must be generated in order to define some reasonable minimum accuracy of classification. a decision structure for each is used to evaluate relative improvement is created using the ahp method. this ensures that the base case still reflects the relative weights of each of the known cues and is the best estimate available for the optimal classification given the information provided by the sme. information state 11 or 1011 can be considered as an example to illustrate how a base case decision structure for that state can be generated. in this example, the cis decision structure is recreated, however, all the cues that are not available and each criterion that becomes empty as a result of this process are removed. for is 11, the decision structure is shown in figure 3. ijahp article: kamps, jassemi-zargani/decision making in dynamic environments: an application of machine learning to the analytical hierarchy process international journal of the analytic hierarchy process 33 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.766 figure 3 decision structure to produce weights for this decision structure, the ahp method must be applied again, and this time each comparison matrix only contains elements that appear in the decision structure. since all the comparisons are pairwise, they still reflect the relative weights of each cue and criteria within their respective category. thus, the ahp method is still a valid procedure for calculating the decision of this structure. this process is then applied to every other is to produce a different base case for each. this base case initialization process is outlined in the following information flow chart shown in figure 4. figure 4 initialization process 5.2 mathematical representation of decision trees when considering the state of complete information, the decision structure needs to be explicitly defined to be able to apply machine learning for optimization purposes. taking the inputs from a specific target, a method to easily describe the propagation of the inputs to the outputs is described below.  calculate the threat value of target x1 𝑇(𝑥) = ∑ 𝜃𝑖 ∙ ( ∑ 𝜔𝑖𝑗 𝜇𝑖𝑗 (𝑥) 𝑗 𝜖 𝐾 ) 𝑖 ∈ 𝑁 where: ijahp article: kamps, jassemi-zargani/decision making in dynamic environments: an application of machine learning to the analytical hierarchy process international journal of the analytic hierarchy process 34 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.766  𝜃 = [𝜃1, … , 𝜃𝑁 ] is the vector of criteria weights  n is the number of criteria  𝜔 = [ 𝜔11 ⋯ 𝜔1𝐾 ⋮ ⋱ ⋮ 𝜔𝑁1 ⋯ 𝜔𝑁𝐾 ] is a matrix of cue weights  the rows of 𝜔 represent the cues within each criteria.  k is the size of the criteria with the most cues  smaller criteria have trailing zero entries  𝜇(𝑥) = [ 𝜇11(𝑥) ⋯ 𝜇1𝐾 (𝑥) ⋮ ⋱ ⋮ 𝜇𝑁1(𝑥) ⋯ 𝜇𝑁𝐾 (𝑥) ] is the measurement of each of the cues from target x and has the same shape as omega with the same zero entries it is important to note that because of the properties of the ahp method, the sum of θ or any row in ω is always unity. additionally, it is desirable for the threat value to be normalized, which also normalizes the measurements 𝜇𝑖𝑗 (𝑥). with these properties, the threat value t, is always in the interval [0, 1]. for every other information state, different theta and omega values are generated using the ahp method. to create scenarios in which only some of the values of 𝜇(𝑥) are known, each information state is associated with a variable denoted as an input shape parameter. these are binary matrices with entries of either 0 or 1. they have the exact shape of μ, but if the cue exists in that information state then the entry in the position of the cue is 1, otherwise it is 0. this allows an easy partitioning of μ to create scenarios into a lesser is for training purposes. 5.3 weight optimization when considering an incomplete is, any is that is not the cis, the weights are calculated using the ahp method and are therefore intelligent in the sense that they take into consideration the relative importance of one another. in this situation, the problem is the uncertainty of the information that is lacking. using weight redistribution through mini batch stochastic gradient descent (ruder, 2017), it is possible to improve the accuracy of a correct threat classification of any information state that considers a minimum of two cues. to accomplish this, a cost function needs to be implemented to measure the error between the predicted threat value of an information state and the reference threat value. ijahp article: kamps, jassemi-zargani/decision making in dynamic environments: an application of machine learning to the analytical hierarchy process international journal of the analytic hierarchy process 35 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.766  quadratic cost: 𝐶1 = 1 2𝐵 ∑ ( 𝑌(𝑥) − 𝑇(𝑥) )2 𝑥 ∈ 𝐵  entropic cost function: 𝐶 = − 1 𝐵 ∑ [𝑌(𝑥) ln(𝑇(𝑥)) + ( 1 − 𝑌(𝑥) ) ln( 1 − 𝑇(𝑥) )] 𝑥 ∈ 𝐵 where:  c is the cost to be minimized  b is the batch of training targets  x is a target  y(x) is the reference threat for target x  t(x) is the threat calculated in the information state being trained both cost functions have their advantages; however, the entropic cost has a much steeper derivative and provides improved optimization when learning slowdown occurs. the method of stochastic gradient descent takes the gradient of the cost with respect to the weights θ and ω for that information state and shifts them in the opposite direction of the gradient with respect to some learning rate, α. 𝜃𝑖 = −𝛼𝜃 𝜕𝐶 𝜕𝜃𝑖 𝜔𝑖𝑗 = −𝛼𝜔 𝜕𝐶 𝜕𝜔𝑖𝑗 where:  c is entropic cost  𝛼𝜃 is the learning rate for the criteria weights  𝛼𝜔 is the learning rate for the cue weights  𝜕𝐶 𝜕𝜃𝑖 = − 1 |𝑁| ∑ ( 𝑌(𝑥)−𝑇(𝑥) 𝑇(𝑥)(1−𝑇(𝑥)) ) 𝜕𝑇(𝑥) 𝜕𝜃𝑖 𝑥 ∈ |𝑵|  𝜕𝐶 𝜕𝜔𝑖𝑗 = − 1 |𝑁| ∑ (𝑌(𝑥) − 𝑇(𝑥))𝑥 ∈ |𝑵| 𝜕𝑇(𝑥) 𝜕𝜔𝑖𝑗  𝜕𝑇 𝜕𝜃𝑖 = ∑ 𝜔𝑖𝑗 𝜇𝑖𝑗𝑗 ∈ 𝐾 (𝑥)  𝜕𝑇 𝜕𝜔𝑖𝑗 = 𝜃𝑖 𝜇𝑖𝑗 (𝑥) ijahp article: kamps, jassemi-zargani/decision making in dynamic environments: an application of machine learning to the analytical hierarchy process international journal of the analytic hierarchy process 36 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.766 this process is repeated until some end condition is met to produce optimized weights for that information state. these weights are then saved and are the new weights used to calculate threat for that information state. the reason the cost function is averaged over a batch of training examples is to reduce the noise of any single scenario which helps avoid overfitting. it smooths the learning process and under some conditions results in better weight optimization. the weights of the cis do not change. they are the weights which are used to calculate the reference threat value. the weights that do change are those of each information state. 5.4 adapted decision method one of the largest limitations of the above process is the inability to improve the classification accuracy unless many cues exist in the information state that can be reweighted. next, a new structure of the decision tree is proposed that allows for much greater improvement in accuracy after optimization. 5.4.1 sigma node the sigma node is a function that is frequently used in machine learning. specifically, its properties are used for training neural networks. the sigmoid node is a special case of the logistic function. it is a simple, non-linear function that provides a balance between linear and non-linear behavior (menon, 1996). sigma functions are of the following form.  sigma function 𝑆(𝜌, 𝛽, 𝑥) = 1 1 + 𝑒 −𝜌(𝜇− 𝛽) where:  ρ is a shape parameter  μ is the input  β is a shift parameter for these purposes, it is useful to consider the standard values for the shape and shift parameter as:  ρ=4.5  β= 0.5 the sigma function with these parameters is denoted the standard sigma function (ssf). this is because of the similarity of this function to the identity function on the interval [0,1], as seen in the figure 5. this is desirable because it means that the use of ssf nodes within the decision structure will result in a largely unchanged threat value from the original method given the same cue measurements. ijahp article: kamps, jassemi-zargani/decision making in dynamic environments: an application of machine learning to the analytical hierarchy process international journal of the analytic hierarchy process 37 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.766 figure 5 sigma function 5.4.2 updated decision structure to improve the ability of a learning function to optimize the threat value prediction in any is, an update to the decision structure needs to be made. when considering the cis, the decision shape takes the same form as in the previous case but with an added step. before input to any of the weight nodes, the input passes through an ssf node. this structure is called a sigmoid decision tree as seen in figure 6. when sigmoid operations are being considered (i.e., in the adapted decision method), the reference threat value becomes the output of the decision tree as seen in figure 6. to create the sigmoid decision trees for each is, the process described in section 5.1 is applied. the difference is the addition of ssf nodes before input to any weight node. these decision structures become the new base cases for each of the iss used as comparisons for how much the learning algorithm that is described in the following section can improve the prediction. ijahp article: kamps, jassemi-zargani/decision making in dynamic environments: an application of machine learning to the analytical hierarchy process international journal of the analytic hierarchy process 38 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.766 figure 6 sigmoid decision tree 5.5 sigma node justification the sigmoid decision structure offers a method to quantify sensitivity to dependency within the cues. input from cues with more influence can be adjusted to reflect their impact on future calculations. using the same learning process described in section 5.3, the value of the sigma parameters can be optimized to produce further improved threat value predictions. using the shape and shift parameters, a broad spectrum of i/o relations can be achieved through sigma optimization. to illustrate the flexibility of the sigma node to adapt to countless circumstances, figure 7 offers some examples of different i/o relationships. threat value (tv) capability proximity intent tbcpa al r cpa sp aoa o ssf ssf ssf ssf ssf ssf ssf ssf ssf ssf target ijahp article: kamps, jassemi-zargani/decision making in dynamic environments: an application of machine learning to the analytical hierarchy process international journal of the analytic hierarchy process 39 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.766 (a) (b) figure 7 (a) varying shape parameters, (b) varying shift parameter an added improvement of including sigma nodes in the threat calculation lies in the ability of the sigma function to easily adjust the threat value output even when few cues are available. this means the structure can adapt to reflect a more accurate prediction even when only one cue is known. 5.6 sigma calculations and optimization given the new decision structure in figure 6, the threat value calculation must be established.  expanded sigma threat calculation 𝑇(𝑥) = ∑ [𝜃𝑖 ∗ 𝑆 (𝜑𝑖 , 𝛿𝑖 ∑ (𝜔𝑖𝑗 𝑆(𝜌𝑖𝑗 , 𝛽𝑖𝑗 , 𝜇𝑖𝑗 (𝑥))) 𝑗 ∈ 𝐾 )] 𝑖 ∈ 𝑁  substitute 𝛾𝑖 (𝑥) = ∑ (𝜔𝑖𝑗 𝑆(𝜌𝑖𝑗 , 𝛽𝑖𝑗 , 𝜇𝑖𝑗 (𝑥))) 𝑗 ∈ 𝐾  simplified version 𝑇(𝑥) = ∑ 𝜃𝑖 ∗ 𝑆(𝜑𝑖 , 𝛿𝑖 , 𝛾𝑖 (𝑥)) 𝑖 ∈ 𝑁 where:  θ, ω, and μ are as defined in 2.2  s is a sigma function ijahp article: kamps, jassemi-zargani/decision making in dynamic environments: an application of machine learning to the analytical hierarchy process international journal of the analytic hierarchy process 40 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.766  𝛽 = [ 𝛽11 ⋯ 𝛽1𝐾 ⋮ ⋱ ⋮ 𝛽𝑁1 ⋯ 𝛽𝑁𝐾 ] and 𝜌 = [ 𝜌11 ⋯ 𝜌1𝐾 ⋮ ⋱ ⋮ 𝜌𝑁1 ⋯ 𝜌𝑁𝐾 ] are the matrices of cue shift and shape parameters, respectively.  𝛿 = [𝛿1, … , 𝛿𝑁 ] and 𝜑 = [𝜑1, … , 𝜑𝑁 ] are the vectors of criteria shift and shape parameters, respectively. the relationship between the sigma threat structure and the original structure is highly correlated. the following plot (figure 8) outlines the correlation between threat value calculations of one versus the other. figure 8 correlation of sigmoid and standard decision structures the correlation shows that in the intermediate threat value section the output is functionally the same. at the extremes, there is a slight deviation from the perfect correlation. the pearson correlation coefficient for the two outputs is 0.971. this is a strong indication that the sigma threat calculation very strongly supports the results from the original decision structure method. therefore, the argument that the validity of the ssf supplemented decision tree is representative of the original ahp method is very strong. 5.7 sigmoid (sgd) optimization learning through sgd, the ability of any is to predict the correct classification can be greatly improved. the entropic cost function is used for its ability to improve the speed at which the algorithm converges to optimization. consider the following progression of ijahp article: kamps, jassemi-zargani/decision making in dynamic environments: an application of machine learning to the analytical hierarchy process international journal of the analytic hierarchy process 41 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.766 calculations that will lead to the gradient of the cost function with respect to the sigma parameters.  partials of sigma function with respect to generic placeholder variables, x, y, and z.    x zyxs ),,( [𝑆(𝑥, 𝑦, 𝑧)]2(𝑧 − 𝑦)𝑒 −𝑥(𝑧−𝑦)    z zyxs ),,( [𝑆(𝑥, 𝑦, 𝑧)]2𝑥𝑒 −𝑥(𝑧−𝑦)    y zyxs ),,( z zyxs    ),,( 1. partial of sigma threat with respect to a criteria shape parameter i t   = 𝜃𝑖   i iii s     ,, where:  γ_i is defined above in section 2.6  α_i is the criteria shape parameter  φ_i is the criteria shift parameter 2. partial of sigma threat with respect to cue shape parameter                        ij i iij tt    where: i iii i i st          ),,( and ij ijiij ij ij i s           ),,( 3. partial of sigma threat with respect to a criteria shift parameter 𝜕𝑇 𝜕𝛿𝑖 = 𝜃𝑖 𝜕𝑆(𝜑𝑖 , 𝛿𝑖 , 𝛾𝑖 ) 𝜕𝛿𝑖 4. partial of sigma threat with respect to a cue shift parameter ijahp article: kamps, jassemi-zargani/decision making in dynamic environments: an application of machine learning to the analytical hierarchy process international journal of the analytic hierarchy process 42 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.766                        ij i iij tt    where: i iii i i st          ),,( and ij i     = 𝜔𝑖𝑗 ij ijijij s     ),,( 5.8 input generation all the independent cues are randomly generated based on their own probability density functions so that there are an infinite number of scenarios. 1. orientation: the angle at which the target is oriented with respect to some defended asset. 2. range: the horizontal distance of the target from the defended asset 3. speed: the magnitude of the velocity of the target with respect to the ground 4. altitude: the vertical distance of the target from the ground each cue is generated using a realistic interval for random value selection. depending on the cue, the probability of the value is either based on a uniform or normal distribution. speed and altitude are based on a normal distribution since most targets will exhibit standard speed and altitude values. range and orientation are generated with a uniform distribution since there is no bias towards a particular position of the target. using this generation method for the independent cues, cpa, aoa and tbcpa can be calculated when the other required cues are available. figure 9 visually represents the geometric cues. figure 9 graphical representation of cues cpa target defended asset heading distance to cpa range at cpa aoa range orientation ijahp article: kamps, jassemi-zargani/decision making in dynamic environments: an application of machine learning to the analytical hierarchy process international journal of the analytic hierarchy process 43 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.766 the following formulae are used to calculate the dependent cues. 1. 𝐴𝑂𝐴 = |𝐻𝑒𝑎𝑑𝑖𝑛𝑔 − 𝑂𝑟𝑖𝑒𝑛𝑡𝑎𝑡𝑖𝑜𝑛| 2. 𝐶𝑃𝐴 = (||𝑥𝑐𝑝𝑎, 𝑦𝑐𝑝𝑎 || 2 + 𝐴𝑙𝑡𝑖𝑡𝑢𝑑𝑒 2) 1 2 where: (𝑥𝑐𝑝𝑎 , 𝑦𝑐𝑝𝑎 ) = min t > 0 ||𝑅 ∗ cos(𝑂) + 𝑡 ∗ cos(𝐻) , 𝑅 ∗ sin(𝑂) + 𝑡 ∗ sin(𝑅)||  r = range  o = orientation  h = heading 3. 𝑇𝐵𝐶𝑃𝐴 = 𝐶𝑃𝐴 𝑆𝑝𝑒𝑒𝑑 heading is not used in threat value structure, but is used in the intermediate calculation of cues. it is also generated randomly and is independent of every other cue. once the cues have been generated or calculated, they need to be normalized with respect to the severity of their threat value. recall that all the inputs need to be normalized to a [0, 1] scale so that the decision structure can work properly. the general process is to assume some average case and relate the severity of the cue to how improbable the value. for this example, the severity normalizations are very simple. for example, when considering altitude, there is some pdf for the random generation of altitude. the severity of the altitude is directly related to the deviation from the average value. if a target is much higher or much lower than generally expected, then it is considered suspicious. 𝑃(𝐴𝑙𝑡𝑖𝑡𝑢𝑑𝑒 = 𝑥) = 𝑁𝑜𝑟𝑚𝑎𝑙. 𝑃𝐷𝐹𝑎𝑙𝑡𝑖𝑡𝑢𝑑𝑒 (𝑚𝑒𝑎𝑛 = 𝑎, 𝑆𝑇𝐷 = 𝑏, 𝑥) 𝑆𝑒𝑣𝑒𝑟𝑒𝑖𝑡𝑦 𝑜𝑓 𝐴𝑙𝑡𝑖𝑡𝑢𝑑𝑒 = 𝐶𝐷𝐹𝑎𝑙𝑡𝑖𝑡𝑢𝑑𝑒 (𝑚𝑒𝑎𝑛 = 𝑎, 𝑆𝑇𝐷 = 𝑏, (𝑏 + |𝑏 − 𝑥|)) where:  severity of altitude is the normalized threat value for altitude  𝑁𝑜𝑟𝑚𝑎𝑙. 𝑃𝐷𝐹𝑎𝑙𝑡𝑖𝑡𝑢𝑑𝑒 is a normal probability distribution function for the generation of altitude values  𝐶𝐷𝐹𝑎𝑙𝑡𝑖𝑡𝑢𝑑𝑒 is the cumulative density function of altitude  a is the mean of the pdf  b is the standard deviation of the pdf in a similar way, each of the cues can be assigned a threat value given their randomized measurements. as stated above, this is a very simple method which is not to say that it is a good method to evaluate a target, but more to illustrate the ability of the method to predict the outcome using cue dependencies and conditional probabilities. ijahp article: kamps, jassemi-zargani/decision making in dynamic environments: an application of machine learning to the analytical hierarchy process international journal of the analytic hierarchy process 44 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.766 5.9 optimization algorithm sgd is applied to both the sigma parameters and the weight nodes so that an optimal structure for each is can be generated. first, the cues are segmented into their base case information states as described in section 5.1. the structures are built and the ssf nodes are inserted to prepare each is for training. these base cases are saved in a database as the reference decision structures for later comparison with the optimized versions. the second step is the optimization process. this process occurs in the following three phases: training data pre preparation, training, and evaluation. 5.9.1 training data preparation sgd attempts to optimize a cost function with respect to some training parameters. if there is some underlying tendency of the function under evaluation to have a specific distribution, then sgd will recognize this and optimize with respect to that distribution. given a large sample of target input cases, the target threat value calculation converges to the probability distribution shown in figure 10. figure 10 probability distributions of target threat value the threat value output approximately takes the form of a normal distribution with a mean of 0.43 and standard deviation of 0.14. if sgd were applied to the information states with a training batch that was randomly sampled, the optimization might learn to predict the distribution as opposed to the dependencies and relationships within the cues themselves since the probability that the correct classification will be within the standard deviation is so high. in other words, any underlying interdependencies of the threat calculation might get overpowered by the predisposition of the output to reflect a normal distribution. to address this problem, a partitioning algorithm was implemented to create a uniform set of training targets from which to sample from during training. a random sample of 1000 reference targets were selected to mimic a uniform distribution of threat value. this is known as a training set. using this set during training will not reveal the underlying ijahp article: kamps, jassemi-zargani/decision making in dynamic environments: an application of machine learning to the analytical hierarchy process international journal of the analytic hierarchy process 45 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.766 probability of the outcome to the decision structure and consequently requires that the prediction be based on cue interdependency and cue conditional probabilities. 5.9.2 training method the training process is when the optimization occurs. a pseudo code for learning optimization is outlined below. the training set is described in section 5.9.1 and all the greek variables have been defined in the sections about weight and sigma parameter optimization. the length of the batch is the number of cost gradients that are averaged per iteration. as mentioned before, this averaging promotes smoother learning due to reduced noise. while (not stop condition): for each (information state in information state space): average cost gradient(𝜃, 𝜔, 𝜑, 𝛿, 𝜌, 𝛽) = [0, 0, 0, 0, 0, 0] for (length (batch)): from uniform training set select 𝜇(𝑥) reference threat = cis threat value ( 𝜇(𝑥) ) partition 𝜇(𝑥) into is shape from database select 𝜃, 𝜔, 𝜑, 𝛿, 𝜌, 𝛽 for is calculate is threat value: 𝑇(𝜃, 𝜔, 𝜑, 𝛿, 𝜌, 𝛽, 𝜇(𝑥) ) calculate ∇𝐶𝑜𝑠𝑡(𝜃, 𝜔, 𝜑, 𝛿, 𝜌, 𝛽) average cost gradient = average cost gradient + ∇𝐶𝑜𝑠𝑡(𝜃,𝜔,𝜑,𝛿,𝜌,𝛽) 𝐿𝑒𝑛𝑔𝑡ℎ 𝐵𝑎𝑡𝑐ℎ [𝜃, 𝜔, 𝜑, 𝛿, 𝜌, 𝛽] = −[𝛼𝜃 , 𝛼𝜔, 𝛼𝜑 , 𝛼𝛿 , 𝛼𝜌, 𝛼𝛽 ] − ∇(𝜃, 𝜔, 𝜑, 𝛿, 𝜌, 𝛽) update database with new 𝜃, 𝜔, 𝜑, 𝛿, 𝜌, 𝛽 for current is a skeleton outline of the pseudo code is depicted in the information flow graphic in figure 11. below, the optimization of the weights is shown; however, the process for the sigma parameters is much the same. ijahp article: kamps, jassemi-zargani/decision making in dynamic environments: an application of machine learning to the analytical hierarchy process international journal of the analytic hierarchy process 46 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.766 figure 11 graphic representation of information flow for the example scenario in the following sections, the following hyper parameters were used. these might not be the optimal values; however, they are enough to illustrate the results in the following sections. there is a set of selected hyper parameters which were found using a grid search and a set of random parameters that were found using random generation shown in table 5. both sets produce similar results. table 5 hyper parameters values learning rate best selected values best random values 𝜃 0.005 0.00661 𝜔 0.01 0.000795 𝛿 0.05 0.11 𝛽 0.05 0.00661 𝜑 0.5 1.645 𝜌 1.5 9.03 batch 3 3 iterations 800 1000 6. experiment and results 6.1 measures of performance three measures of performances were used to compare the base case to the trained structures. the first is range classification. a target threat prediction is considered correct if the difference between the reference threat value and the predicated threat value are within 0.1 of each other. = |𝑅𝑒𝑓𝑒𝑟𝑒𝑛𝑐𝑒 − 𝑇ℎ𝑟𝑒𝑎𝑡| < 0.1 ijahp article: kamps, jassemi-zargani/decision making in dynamic environments: an application of machine learning to the analytical hierarchy process international journal of the analytic hierarchy process 47 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.766 the second measure of performance is the bin classification method. the target threat values are partitioned into five bins of length 0.2. 𝐹𝑟𝑖𝑒𝑛𝑑𝑙𝑦 → 𝑁𝑒𝑢𝑡𝑟𝑎𝑙 → 𝑈𝑛𝑘𝑛𝑜𝑤𝑛 → 𝑆𝑢𝑠𝑝𝑖𝑐𝑖𝑜𝑢𝑠 → 𝑇ℎ𝑟𝑒𝑎𝑡𝑒𝑛𝑖𝑛𝑔 if the reference threat value and the predicted threat value fall within the same bin, the prediction is considered correct. finally, the mean error of prediction and the standard deviation of that error is calculated. for each of these measures of performance, the result will be considered as an average over all of the information states and for each individual state. 6.2 results using the hyper parameters described in section 5.9, an experiment was run in which targets from the training set were randomly sampled to apply sgd optimization over the learning parameters to the information states. the progression of the average cost at every 10 cycles of the 800 iteration learning progression is depicted in figure 12. learning occurs very quickly at the start, followed by a gradual reduction in cost as learning continues. the cost used to create this plot is the average quadratic cost. the yellow line represents a moving average. it can also be seen that as learning progresses, the cost becomes smoother with fewer large upward spikes, indicating a smaller standard deviation on error. figure 12 learning progression in 800 iterations with 10 per cycle the final optimized structures were saved in a database. based on a trial of 1000 targets sampled randomly to form a validation set, the following results were generated for each measure of performance as shown in table 6. ijahp article: kamps, jassemi-zargani/decision making in dynamic environments: an application of machine learning to the analytical hierarchy process international journal of the analytic hierarchy process 48 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.766 table 6 performance results metric untrained trained information state 1 mean 6 1 mean 6 mean absolute error 0.285 0.152 0.073 0.121 0.096 0.066 error standard deviation 0.161 0.120 0.053 0.082 0.069 0.044 range classification accuracy 12 % 37 % 62 % 46 % 55 % 79 % bin classification accuracy 13 % 33 % 72 % 45 % 48% 69 % using range classification, there is an average improvement of 18%. bin classification showed an improvement of 15%. the range classifications are higher which is expected since when using bin classification, there are cases when even if the prediction is very accurate, the deviation causes the prediction to fall within the wrong bin. the mean error and standard deviation also give merit to the performance of the algorithm. the mean error is reduced by 37% from 0.152 to 0.096. additionally, the average standard deviation is almost halved. this is an indication that the learned structure is not only more accurate, it also attests to the fact that when it is wrong, it is still more precise. the error is much more highly concentrated meaning that even an incorrect classification is likely not far from a correct one. the distribution of target threat values that are output during the validation set have the following distributions: green is the base case, red is the trained case and blue is the reference threat output. figure 13 target threat value output distributions ijahp article: kamps, jassemi-zargani/decision making in dynamic environments: an application of machine learning to the analytical hierarchy process international journal of the analytic hierarchy process 49 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.766 figure 13 graphically shows the reduction of not only the error of classification but also the standard deviation. it is important to remember that the trained case was trained on a training set that had a uniform distribution. it was not given any information about what the probability distribution might look like. 6.3 analysis of the information states one of the other goals of this study was to prove the ability to predict dependent cues. as shown in figure 14, the base case accuracy for each information state is represented in red and the optimized value is in green. organizing the information states by the number of known independent cues produces the results shown in figure 14. figure 14 independent cues accuracy 7. conclusions the ahp is a useful tool for synthesizing a decision based on the opinions of a sme. supplementation of ssf nodes into the standard decision structure preserves the decisions that would have been made before the supplementation. the goal is to be able to predict the decision of a complete ssf supplemented decision structure with weights generated through the ahp when not all the information is available. this paper has shown that by applying the ahp to derive the weights and using machine learning to adjust the parameters of the sigmoid nodes in the deficient decision structures, the predictive capabilities can be vastly improved. this method is useful for all states of information availability and shows promise for use in practice. ijahp article: kamps, jassemi-zargani/decision making in dynamic environments: an application of machine learning to the analytical hierarchy process international journal of the analytic hierarchy process 50 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.766 references johansson, f. & falkman,g. (2008). a bayesian network approach to threat evaluation with application to an air defense scenario. 11th international conference on information fusion, 1-7. cologne. liebhaber, m.j., kobus, d.a., & feher, b.a. (2002). studies of u.s. navy cues, information order, and impact of conflicting data. technical report 1888. san diego: spawar systems center. doi: https://doi.org/10.21236/ada406334 menon, a., mehrotra, k., mohan, c.k., & ranka, s. (1996). characterization of a class of sigmoid functions with applications to neural networks. neural networks, 9(5), 819835. doi: https://doi.org/10.1016/0893-6080(95)00107-7 mu, e. (2006). a unified framework for site selection and business forecasting using anp. journal of systems science and systems engineering, 15(2), 178–188. doi: https://doi.org/10.1007/s11518-006-5006-6 ruder, s. (2017). an overview of gradient descent optimization algorithms. computer research repository (corr), 1609.04747. saaty, t.l. (1987). the analytical hierarchy process – what it is and how it is used. pittsburgh, pa: pergamon journals ltd. ijahp article: ocampo, clark/an analytic hierarchy process (ahp) approach in the selection of sustainable manufacturing initiatives: a case in a semiconductor manufacturing firm in the philippines international journal of the analytic hierarchy process 32 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.223 an analytic hierarchy process (ahp) approach in the selection of sustainable manufacturing initiatives: a case in a semiconductor manufacturing firm in the philippines lanndon ocampo department of mechanical engineering university of san carlos, philippines don_leafriser@yahoo.com eppie clark department of industrial engineering de la salle university-manila, philippines eppie.clark@dlsu.edu.ph abstract this paper proposes an evaluation framework of sustainable manufacturing (sm) initiatives using the hierarchical structure of sustainability indicators set adopted by the us national institute of standards and technology (us nist) in the context of the analytic hierarchy process (ahp). evaluating sm initiatives developed by manufacturing firms is crucial for resource allocation, and ensuring that investments enhance the sustainability performance of the firm. this evaluation is a challenge because of the multi-criteria nature of the problem and the presence of subjective criteria for which little or no information on their measurement systems is available. thus, this study is appropriate due to the following reasons: (1) us nist provides a comprehensive evaluation model of sustainability with its four-level hierarchy that provides evidence of depth and details of sustainability evaluation, and (2) ahp has the capability to handle multi-level decision-making structure with the use of expert judgments in a pairwise comparison process. a case study of a semiconductor manufacturing firm is presented to illustrate the proposed evaluation framework. results show that firms must strengthen their financial base through programs that improve efficiency, quality and productivity before carrying out initiatives that address the environment and the immediate community. this work presents a framework that could guide decision-makers, in a way that is simple and comprehensive in their attempt to promote sustainability. keywords: sustainable manufacturing, evaluation, analytic hierarchy process, multicriteria decision-making 1. introduction the current global focus on sustainability compels manufacturing firms to structure their decisions on manufactured products and manufacturing processes beyond traditional profit-based approaches, and utilize a more holistic view that incorporates environmental mailto:don_leafriser@yahoo.com mailto:eppie.clark@dlsu.edu.ph ijahp article: ocampo, clark/an analytic hierarchy process (ahp) approach in the selection of sustainable manufacturing initiatives: a case in a semiconductor manufacturing firm in the philippines international journal of the analytic hierarchy process 33 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.223 concerns and social well-being. this discussion has been the focal point of research following the un report that formally declared the need to adopt sustainable development (brundtland, 1987). the manufacturing industry, as a key sector in the sustainability focus (rosen and kishawy, 2012; joung et al., 2013), has a role in translating sustainability from a profound concept into manageable pieces worthy of attention in research (mani et al., 2012). furthermore, the explicit declaration of the u.s. department of commerce on sustainable manufacturing (sm) which promotes “the creation of manufactured products that use processes that minimize negative environmental impacts, conserve energy and natural resources, are safe for employees, communities and consumers and are economically sound” (joung et al., 2013) has formally linked manufacturing to the mainstream sustainability research. interest in sm has developed in both industry and academia and has inspired leading economies (kovac, 2012). the evidences from these efforts include well-established approaches such as cleaner production (engelhardt et al., 1994), industrial ecology (frosch and gallopoulus, 1989; baas and boons, 2004), 5r approach (reduce, reuse, remanufacture, recycle and recovery) (ageron et al., 2012), green production (lai, 1993), and design for recyclability (despeisse et al., 2012). approaches to sm are derived primarily from addressing sustainability in three widely accepted dimensions, i.e. environmental, economic and social, most notably known as the triple-bottom line (elkington, 1997). as firms adopt sm initiatives, sustainability indicators are designed to measure and monitor performance of a given approach (ragas et al., 1995). a number of indicators have been published by renowned institutions, international agencies and bodies, universities, and government and industries. a review of such indicator sets is carried out in several works, e.g. joung et al. (2013), singh et al. (2012), böhringer and jochem (2007), mayer (2008). these indicator sets and a hybrid of these sets have been used to assess and evaluate sustainability in different domains (chen et al., 2012, jawahir et al., 2007, de silva et al., 2009 and mani et al., 2012). at present, the most critical and comprehensive framework of sm indicators has been developed by joung et al. (2013) with results adopted by the us national institute of standards and technology (us nist). ocampo and clark (2015) adopted this framework in identifying input elements in developing sustainable manufacturing initiatives. the framework is a critical integration of eleven internationally-accepted sustainability indicator sets in different levels of the economy. the framework is plausible as it is hierarchically structured so that a great level of detail is achieved. the evaluation of sm initiatives is a multi-criteria decision problem because a number of criteria must be taken into consideration when assessing the degree to which these initiatives conform to the dimensions of sustainability. the complexity increases due to the presence of subjective criteria with little or no information available on their corresponding measurement systems. one important consideration in selecting a particular method in strategy selection problems is the ability of the method to address assessment involving value judgments, assumption and scenarios (heijungs et al., 2010) which are characteristics of mcdm methods (herva and roca, 2013). a survey of literature in mcdm carried out by herva and roca (2013) indicates that analytic hierarchy process/analytic network process (ahp/anp) and outranking methods are commonly used in industry-related applications. due to its logical and simple structure in handling comprehensive evaluation of multi-layer decision problems, ahp is used in this ijahp article: ocampo, clark/an analytic hierarchy process (ahp) approach in the selection of sustainable manufacturing initiatives: a case in a semiconductor manufacturing firm in the philippines international journal of the analytic hierarchy process 34 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.223 paper to select sm initiatives. in ahp, a decision problem is expressed as a hierarchy of decision components and priorities are derived from pairwise comparisons and judgments (saaty, 1980). numerous applications of ahp in sustainability assessment were explored in the literature (krajnc and glavic, 2005, gupta et al., 2011, garbie, 2011; ocampo and clark, 2015). this leaves ahp as the most prominent mcda method in sustainability assessment (seuring, 2013). a critical review of ahp and its applications is carried out by vaidya and kumar (2006) and subramanian and ramanathan (2012). although several published works on sustainability assessment have already been done, selection of sm initiatives based on a comprehensive framework is rare. this paper attempts to present a selection process of sm initiatives which has an sm evaluation framework based from us nist and where the decomposition and prioritization process is done using ahp. this extends the work of ocampo and clark (2015) where the main departure of this work lies on the evaluation of sm initiatives carried out by a manufacturing firm. this area is an important focus in research as it provides better insights for managers and decision-makers at the firm level on the selection of initiatives that advance sustainability. this aids in the decision-making process of problems that comprise both tangible and intangible components with multi-dimensional scales. a case study of a multi-national semiconductor firm with a manufacturing site located in the philippines is presented to describe the selection process. this paper is organized as follows: section 2 provides the general methodology of the study. section 3 presents a case study in a semiconductor manufacturing firm. section 4 and section 5 present the results and discussion of the selection process using ahp and relevance of the results in sustainability research. section 6 concludes the study with a discussion of future possible work. 2. proposed method 2.1 analytic hierarchy process ahp is a powerful tool in multi-criteria decision analysis (mcda) particularly in hierarchical decision-making. ahp decomposes a decision problem hierarchically into components of different levels. decision-makers elicit pairwise comparisons, based from their value judgments, of the elements in the same level with respect to an element in a higher immediate level using the famous saaty fundamental 9-point scale (saaty, 1980). priority vectors (w) are obtained from the pairwise comparison matrix (a) by solving an eigenvalue problem in the following relation: aw = λmaxw (1) where λmax is the maximum eigenvalue of the positive reciprocal square matrix (a). when decision-making in the pairwise comparisons matrix is consistent λmax = n; otherwise, λmax > n where n is the number of elements being compared. the consistency index (ci), as a measure of degree of consistency, is calculated using the formula ci = λmax−n n−1 (2) ijahp article: ocampo, clark/an analytic hierarchy process (ahp) approach in the selection of sustainable manufacturing initiatives: a case in a semiconductor manufacturing firm in the philippines international journal of the analytic hierarchy process 35 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.223 the consistency ratio (cr) is computed as cr = ci ri (3) where ri is the mean random consistency index. it is suggested that a cr ≤ 0.10 be obtained; otherwise, decision-makers are asked to revise the pairwise comparisons matrix. synthesizing priorities across the hierarchy is done using the distributive mode of the ahp which can be represented in the form: wj = ∑ cixij n i=1 (4) where wj is the global weight of alternative j, ci is the weight of criteria i with respect to the goal, and xij is the local weight of alternative j with respect to criteria i. this produces a global priority vector of alternatives. see saaty (1980) for the comprehensive discussion of the ahp. 2.2 proposed model in general, the procedure of selecting sm initiatives using ahp is as follows: 1. adopt the hierarchical evaluation structure described by joung et al. (2013) which later became the standard sm indicators used by us nist. the structure and its details can be accessed through the sustainable manufacturing indicators repository (smir) website (smir, 2011). it is composed of three levels which are (from top to bottom) the sm dimension component, the criteria component and the sub-criteria component. the application of this structure and its formation into an evaluation framework comprises the general hierarchical evaluation framework adopted in this study as shown in figure 1. ijahp article: ocampo, clark/an analytic hierarchy process (ahp) approach in the selection of sustainable manufacturing initiatives: a case in a semiconductor manufacturing firm in the philippines international journal of the analytic hierarchy process 36 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.223 figure 1. general hierarchical evaluation framework 2. elicit pairwise comparisons based on the framework developed in step 1 using saaty’s fundamental scale. in eliciting pairwise comparison, generally we ask this question: “given a parent element and given a pair of elements, how much more does a given member of the pair dominate the other member of the pair with respect to a parent element?” (promentilla et al., 2006). this forms a positive reciprocal pairwise comparisons matrix. local priority vectors are computed using equation 1. consistency is checked using equations 2 and 3. note that it is suggested that c.r. be less than 0.10 (saaty, 1980). 3. after obtaining all local priority vectors, a judgment is synthesized using equation 4 to obtain global priorities of alternatives. note that this vector is used to rank alternatives with their degree of impact or contribution to the goal. 3. case study an actual case study was carried out from the work published by ocampo and clark (2014). the case study focused on identifying sm initiatives of a semiconductor manufacturing firm in the philippines. fc semiconductor is a multi-national firm which is one of the prime players in the semiconductor industry (ocampo and clark, 2014). fc has sites strategically located around the world with manufacturing sites mostly located in developing countries such as the philippines. as part of corporate directives, fc is committed to adopting sustainability practices in decision-making especially on critical areas in its manufacturing sites in asia. despite being a key manufacturing industry in technological advancement, the semiconductor industry has serious sustainability issues goal (a) sm dimension component (b) criteria component (c) sub-criteria component (d) sm initiatives (e) ijahp article: ocampo, clark/an analytic hierarchy process (ahp) approach in the selection of sustainable manufacturing initiatives: a case in a semiconductor manufacturing firm in the philippines international journal of the analytic hierarchy process 37 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.223 which have been brought about by the unprecedented growth of natural resource consumption as inputs to production, a large number of new chemicals introduced annually, and the health and safety concerns related to the use of these chemicals (ocampo and clark, 2014). fc’s cebu, philippines manufacturing site strives to address these issues and conform to the corporate drive toward sustainability by complying with international standards such as the directive of the european union on the restricted use of hazardous substances (rohs), and the waste electrical and electronic equipment directive (wee directive), and with international certifications such as the iso 9000, iso 14000 and iso 18000 series. furthermore, the site has developed initiatives that promote sustainability, the details of which can be found in the work of ocampo and clark (2014). in this paper, we evaluated sm initiatives of fc semiconductor with their degree of relevance to sustainability using the hierarchical framework of joung et al. (2013). five sm initiatives were presented for evaluation: health and wellness programs (e1), competitive employee compensation and career development (e2), sound occupational health and safety program (e3), elimination of lead (pb) in the plating process (e4) and lean six sigma programs (e5). details of each initiative are found in the work of ocampo and clark (2014). table 1, obtained from the work of joung et al (2013), shows the decision components with their corresponding codes, and figure 2 presents the operational framework of this case study. the coding system for this study assigned alphabetical letters to each level of the hierarchical framework and numbers were assigned to the arrangement of elements in each level. this is consistent with the coding system adopted by ocampo and clark (2015) except for the sm initiatives component. the goal is coded as a; sm dimensions are coded as b; criteria components are coded as c; sub-criteria components are coded as d; and lastly, sm initiatives are coded as e. for instance, employee health and safety, a sub-criterion, is coded as d25. ijahp article: ocampo, clark/an analytic hierarchy process (ahp) approach in the selection of sustainable manufacturing initiatives: a case in a semiconductor manufacturing firm in the philippines international journal of the analytic hierarchy process 38 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.223 table 1 decision components and their codes decision components and elements code decision components and elements code decision components and elements code evaluation of sustainable manufacturing initiatives a acidification substance d5 end-of-service-life product handling d22 environmental stewardship b1 effluent d6 research and development d23 economic growth b2 air emissions d7 community development d24 social well-being b3 solid waste emissions d8 employees health and safety d25 pollution c1 waste energy emissions d9 employees career development d26 emissions c2 water consumption d10 employee satisfaction d27 resource consumption c3 material consumption d11 health and safety impacts from manufacturing and product use d28 natural habitat conservation c4 energy/electrical consumption d12 customer satisfaction from operations and products d29 profit c5 land use d13 inclusion of specific rights to customer d30 cost c6 biodiversity management d14 product responsibility d31 investment c7 natural habitat quality d15 justice/equity d32 employee c8 habitat management d16 community development programs d33 customer c9 revenue d17 health and wellness program e1 community c10 profit d18 competitive employee compensation and career development e2 toxic substance d1 materials acquisition d19 sound occupational health and safety e3 greenhouse gas emissions d2 production d20 elimination of lead in plating process e4 ozone depletion gas emissions d3 product transfer to customer d21 lean six sigma programs e5 noise d4 the operational framework in figure 2 shows how this selection process implements the inputs obtained from the us nist sm indicators repository, from the work of ocampo and clark (2014) on sm initiatives in union with the ahp. a detailed form of the hierarchy is shown in figure 3. a group of experts composed of four sustainability researchers, two manufacturing managers from the case firm, and three consultants were invited to a focus group discussion (fgd) in order to elicit judgments through pairwise comparisons. these groups of experts were selected to promote a balance between conceptual and applicable approaches for the case firm. sustainability researchers and consultants were able to provide comprehensive knowledge on the current state of sustainability manufacturing initiatives both locally and globally. on the other hand, manufacturing managers from the case firm provided information on the firm’s internal configurations that were applicable. although these groups have different perspectives, these differences were not investigated in this work. selection of domain experts was based on their involvement in manufacturing industries with a threshold set of at least 10 years managerial experience. this qualification ensured that they had the capacity and previous knowledge in carrying out vital manufacturing decisions. these managers have sufficient background in sustainability which includes technical knowledge in quality, environmental and social responsibility management systems. these individuals were made known to the researchers through peer referrals or respected scholars and practitioners in the field. the expert group was informed of the purpose of the group ijahp article: ocampo, clark/an analytic hierarchy process (ahp) approach in the selection of sustainable manufacturing initiatives: a case in a semiconductor manufacturing firm in the philippines international journal of the analytic hierarchy process 39 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.223 discussion in advance, and their role in eliciting judgments. the fgd required the group to come up with a consensus judgment for each pairwise comparison, thus the use of group aggregation approaches in the framework of the ahp was not applicable. the results of the pairwise comparisons are presented in the next section. figure 2. operational framework of the case study goal (a) sm dimension component (b) criteria component (c) sub-criteria component (d) sm initiatives (e) us nist sustainable manufacturing indicators repository joung et al (2013) sustainable manufacturing initiatives of the case firm ocampo and clark (2014) analytic hierarchy process (ahp) ijahp article: ocampo, clark/an analytic hierarchy process (ahp) approach in the selection of sustainable manufacturing initiatives: a case in a semiconductor manufacturing firm in the philippines international journal of the analytic hierarchy process 40 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.223 figure 3. decision problem of the evaluation of sm initiatives 4. results and discussion based on figure 3, there are four types of pairwise comparisons in this study. the first type describes pairwise comparisons of elements of the sm dimensions component with respect to the goal. the second type describes pairwise comparisons of the elements in the criteria component with respect to their parent sm dimension element. the third type refers to the pairwise comparisons of the elements in the sub-criteria component with respect to their parent criterion. lastly, the fourth type of pairwise comparisons describes comparing pairwise the sm initiative with respect to each sub-criterion. a total of 47 pairwise comparisons were required in this study. a b1 b2 b3 c1 c2 c3 c4 c7 c5 c6 c8 c10 c9 e2 e3 e4 e5 e1 d5 d4 d3 d2 d1 d9 d8 d7 d6 d16 d15 d14 d13 d12 d11 d10 d18 d17 d22 d21 d20 d19 d24 d23 d27 d26 d25 d30 d29 d28 d33 d32 d31 ijahp article: ocampo, clark/an analytic hierarchy process (ahp) approach in the selection of sustainable manufacturing initiatives: a case in a semiconductor manufacturing firm in the philippines international journal of the analytic hierarchy process 41 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.223 we could not present all 47 pairwise comparisons in this paper due to the large amount of space that would be required to do so. readers may request the full set of tables from the corresponding author. nevertheless, we provide sample pairwise comparison matrices in the following discussions. a sample pairwise comparison of the first type which is comparing sm dimensions on their degree of impact to sustainability is shown in table 2. the question being asked in table 2 is this: “comparing environmental stewardship (b1) and economic growth (b2), which one more dominates the goal (g) and by how much?” the resulting priority vector, the maximum eigenvalue (λmax) and the consistency ratio (c.r.) are shown in table 2. table 3 shows a sample of the pairwise comparisons of the second type. the question being asked in table 3 is this: “comparing pollution (c1) and emissions (c2), which one more dominates environmental stewardship (b1), and by how much?” table 4 shows a sample of the pairwise comparisons of the third type. the question being asked in table 4 is this: “comparing toxic substance (d1) and greenhouse gas emissions (d2), which one more dominates pollution (c1), and by how much?” finally, table 5 shows a sample of pairwise comparisons of comparing sm initiatives with respect to each sub-criterion. the question being asked in table 5 is this: “comparing health and wellness programs (e1) and competitive employee compensation and career development (e2), which one more dominates employee health and safety (d25), and by how much?”. the following tables present the local priority vectors of each pairwise comparisons matrix with their corresponding maximum eigenvalues and consistency ratio (c.r.). c.r. values range from 0.0 to 0.0732 which satisfy the 0.10 threshold of saaty (1980). table 2 comparing environmental stewardship (b1) and economic growth (b2), which one more dominates the goal (g) and by how much? a b1 b2 b3 local priority vector b1 1 1/2 1/2 0.200 b2 2 1 1 0.400 b3 2 1 1 0.400 λmax = 3, c. r. = 0.0 table 3 comparing pollution (c1) and emissions (c2), which one more dominates environmental stewardship (b1), and by how much? b1 c1 c2 c3 c4 local priority vector c1 1 1 3 2 0.351 c2 1 1 3 2 0.351 c3 1/3 1/3 1 2 0.161 c4 1/2 1/2 1/2 1 0.137 λmax = 4.155, c. r. = 0.058 ijahp article: ocampo, clark/an analytic hierarchy process (ahp) approach in the selection of sustainable manufacturing initiatives: a case in a semiconductor manufacturing firm in the philippines international journal of the analytic hierarchy process 42 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.223 table 4 comparing toxic substance (d1) and greenhouse gas emissions (d2), which one more dominates pollution (c1), and by how much? c1 d1 d2 d3 d4 d5 local priority vector d1 1 1 3 5 3 0.348 d2 1 1 3 5 3 0.348 d3 1/3 1/3 1 2 1 0.120 d4 1/5 1/5 1/2 1 1/2 0.065 d5 1/3 1/3 1 2 1 0.120 λmax = 5.005, c. r. = 0.001 table 5 comparing health and wellness program (e1) and competitive employee compensation and career development (e2), which one more dominates employee health and safety (d25), and by how much? d25 e1 e2 e3 e4 e5 local priority vector e1 1 4 1 2 6 0.342 e2 1/4 1 1/4 1/3 3 0.084 e3 1 4 1 2 6 0.342 e4 1/2 3 1/2 1 5 0.182 e5 1/6 1/3 1/6 1/5 1 0.050 λmax = 5.091, c. r. = 0.021 table 6 shows all the priority vectors of sm initiatives with respect to each sub-criterion. such vectors are obtained from the last type of pairwise comparisons process. table 7, on the other hand, presents the normalized priority vectors of the elements in the subcriterion component. this normalization process follows the distributive mode of the ahp. the sum of all these weights is equal to unity. ijahp article: ocampo, clark/an analytic hierarchy process (ahp) approach in the selection of sustainable manufacturing initiatives: a case in a semiconductor manufacturing firm in the philippines international journal of the analytic hierarchy process 43 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.223 table 6 priority vectors of sm initiatives d1 d2 d3 d4 d5 d6 d7 d8 d9 d10 d11 d12 d13 d14 d15 d16 d17 e1 0.251 0.170 0.198 0.201 0.263 0.151 0.146 0.125 0.125 0.112 0.112 0.125 0.167 0.143 0.143 0.143 0.119 e2 0.074 0.084 0.082 0.067 0.079 0.070 0.075 0.125 0.125 0.112 0.112 0.125 0.167 0.143 0.143 0.143 0.119 e3 0.153 0.116 0.116 0.512 0.114 0.108 0.103 0.125 0.125 0.112 0.112 0.224 0.167 0.143 0.143 0.143 0.191 e4 0.448 0.566 0.541 0.090 0.465 0.607 0.602 0.500 0.224 0.480 0.185 0.125 0.333 0.429 0.429 0.429 0.080 e5 0.074 0.064 0.063 0.130 0.079 0.064 0.075 0.125 0.401 0.185 0.480 0.401 0.167 0.143 0.143 0.143 0.492 d18 d19 d20 d21 d22 d23 d24 d25 d26 d27 d28 d29 d30 d31 d32 d33 e1 0.060 0.125 0.075 0.167 0.200 0.086 0.338 0.342 0.134 0.134 0.143 0.125 0.167 0.125 0.200 0.250 e2 0.076 0.125 0.075 0.167 0.200 0.086 0.338 0.084 0.414 0.414 0.143 0.125 0.167 0.125 0.200 0.250 e3 0.179 0.125 0.213 0.167 0.200 0.127 0.181 0.342 0.134 0.134 0.143 0.125 0.167 0.125 0.200 0.250 e4 0.137 0.224 0.104 0.167 0.200 0.504 0.082 0.182 0.086 0.086 0.429 0.224 0.167 0.401 0.200 0.125 e5 0.549 0.401 0.534 0.333 0.200 0.198 0.061 0.050 0.232 0.232 0.143 0.401 0.333 0.224 0.200 0.125 ijahp article: ocampo, clark/an analytic hierarchy process (ahp) approach in the selection of sustainable manufacturing initiatives: a case in a semiconductor manufacturing firm in the philippines international journal of the analytic hierarchy process 44 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.223 table 7 normalized priority vectors of sub-criteria elements sub-criterion normalized vector sub-criterion normalized vector d1 0.024 d18 0.080 d2 0.024 d19 0.053 d3 0.008 d20 0.053 d4 0.005 d21 0.027 d5 0.008 d22 0.027 d6 0.016 d23 0.027 d7 0.032 d24 0.053 d8 0.016 d25 0.060 d9 0.005 d26 0.020 d10 0.009 d27 0.020 d11 0.003 d28 0.040 d12 0.009 d29 0.080 d13 0.009 d30 0.080 d14 0.016 d31 0.033 d15 0.008 d32 0.033 d16 0.008 d33 0.033 d17 0.080 multiplying tables 6 and table 7 in matrix form, the product is the global priority vector or final weights of the sm initiatives. this process is described in equation 4. technically, through matrix multiplication, the contributions of a particular sm initiative for all sub-criteria are added up and the sum is thus the global weight of that initiative. results are reflected in table 8. table 8 ranking of sm initiative sm initiative global priority vector e5 0.276 e4 0.243 e3 0.172 e1 0.162 e2 0.146 final ranking of sm initiatives shows that lean six sigma programs initiative ranks first followed by elimination of lead in the plating process, sound occupational health and safety, health and wellness programs and competitive employee compensation and career ijahp article: ocampo, clark/an analytic hierarchy process (ahp) approach in the selection of sustainable manufacturing initiatives: a case in a semiconductor manufacturing firm in the philippines international journal of the analytic hierarchy process 45 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.223 development. these results could provide substantial insights to decision-makers and sustainability researchers. first, the dominant priority of lean six sigma programs could be explained in two ways. although the current discussions on sustainability are motivated toward socio-environmental well-being, one cannot deny that at the firm level, the basic premise is the existence of the firm into the future. thus, firms create initiatives in an attempt to promote financial stability along with stiff market competition. in that sense, firms seldom promote relevant environmental and social drives if the financial base of the firm is weak. furthermore, growing discussions in the current literature also argue that these environmental and social programs may require a relatively high investment in the short-run and quantifying the return on such investments has not been well established (ageron et al., 2012; law and gunasekaran, 2012). therefore, the requisite of a firm being economically stable must be satisfied first before moving on to other significant environmental and social initiatives. second, central also to current research, is the exploration of positive relationships of lean manufacturing strategies on environmental programs such as cleaner production. next, while lean manufacturing is widely known to enhance market and financial performance, it also improves environmental practices (yang et al., 2011) because manufacturing organizations advocating lean manufacturing have the established infrastructures for identifying and eliminating wastes (bergmiller and mccright, 2009). lastly, sound occupational health and safety, health and wellness programs and competitive employee compensation and career development initiatives are part of firms’ social responsibility to their employees and to the immediate community as well. aside from being part of the social responsibility of firms, maintaining the health and safety of a workforce and creating programs that enhance job security and career development are fundamental to strengthening the human resource base of an organization which is essential to achieving corporate goals (zhang and liu, 2011). 5. conclusion evaluating sustainable manufacturing initiatives of manufacturing firms is an essential step in assisting decision-makers in identification of priority initiatives that have a higher degree of impact on sustainability. this will eventually result in the efficient allocation of a firm’s resources and faster advancement of a firm to the demands of sustainability. this kind of an evaluation is a challenge due to the number of criteria that must be considered, notwithstanding the subjectivity and difficulty of measurement on these criteria. this problem is addressed in the current literature through mcdm methods of which ahp is a popular one because of its simplicity and comprehensiveness in dealing with multi-level and multi-criteria decision problems involving intangibles. the framework which includes the number and depth of the evaluation process is also crucial in developing a comprehensive solution to the problem. thus, this paper presents an evaluation framework of sm initiatives by using the hierarchical sustainability indicators structure of us nist combined with the methodology of ahp. a case study is conducted in a semiconductor manufacturing firm in order to demonstrate the proposed framework. results show that the lean six sigma programs have the largest priority followed by the elimination of lead in the plating process, sound occupational health and safety, health and wellness programs and competitive employee compensation and career development. these results are related in a way to show how firms can improve their investment allocations, relationships with customers and suppliers, and the pattern of technological ijahp article: ocampo, clark/an analytic hierarchy process (ahp) approach in the selection of sustainable manufacturing initiatives: a case in a semiconductor manufacturing firm in the philippines international journal of the analytic hierarchy process 46 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.223 development that the firm or the industry directs. this supports some claims which state that financial viability of the firm serves as the basis for further implementation of initiatives which may include environmental and social programs. firms may resist these forms of investments if the financial base of the firm is weak and the competitiveness of the firm is not established or fully developed. note that the results of this study may be representative only for large manufacturing firms, not small and medium-sized firms. future theories and empirical studies for smaller firms must be undertaken to support this claim. furthermore, this work assumes that the composition of the group is homogeneous, which may not be so in a general case. investigating differences among group members could be an extension of this work. nevertheless, this application of ahp advances our knowledge about selecting sm initiatives and has resulting implications in decision-making. ijahp article: ocampo, clark/an analytic hierarchy process (ahp) approach in the selection of sustainable manufacturing initiatives: a case in a semiconductor manufacturing firm in the philippines international journal of the analytic hierarchy process 47 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.223 references ageron, b., gunasekaran, a. and spalanzani, a. (2012). sustainable supply chain management: an empirical study. international journal of production economics, 140(1), 168-182. baas, l.w. and boons, f.a. (2004). an industrial ecology project in practice: exploring the boundaries of decision-making levels in regional industrial systems. journal of cleaner production, 12(8-10), 1073-1085. bergmiller, g.g. and mccright, p.r. (2009). lean manufacturers’ transcendence to green manufacturing. proceedings of the 2009 industrial engineering research conference, miami, florida. böhringer, c. and jochem, p.e.p. (2007). measuring the immeasurable – a survey of sustainability indices. ecological indicators, 63(1), 1-8. brundtland, g.h. (1987). report of the world commission on environment and development: our common future. oxford, uk: oxford university press. chen, c.c., shih, h.s., shyur, h.j. and wu, k.s. (2012). a business strategy selection of green supply chain management via an analytic network process. computers and mathematics with applications, 64(8), 2544-2557. de silva, n., jawahir, i.s., dillon, jr. o. and russell, m. (2009). a new comprehensive methodology for the evaluation of product sustainability at the design and development stage of consumer electronic products. international journal of sustainable manufacturing, 1(3), 251-264. despeisse, m., ball, p.d., evans, s. and levers, a. (2012). industrial ecology at factory level-a conceptual model. journal of cleaner production, 31, 30-39. elkington, j. (1997). cannibals with forks: the triple bottom line of 21st century business. oxford: capstone. engelhardt, m.s., huisingh, d. and schnitzer, h. (1994). european roundtable on cleaner production programs. journal of cleaner production, 2(3-4), 131. frosch, r.a. and gallopoulus, n.e. (1989). strategies for manufacturing. scientific american, 189(3), 144-152. garbie, i.h. (2011). framework of manufacturing enterprises sustainability incorporating globalization issues. proceedings of the 41st international conference on computers and industrial engineering, los angeles, ca usa. gupta, a., vangari, r., jayal, a. d. and jawahir, i. s. (2011). priority evaluation of product metrics for sustainable manufacturing. in: a. bernard (ed.) global product development, (631-641). berlin, heidelberg: springer-verlag. ijahp article: ocampo, clark/an analytic hierarchy process (ahp) approach in the selection of sustainable manufacturing initiatives: a case in a semiconductor manufacturing firm in the philippines international journal of the analytic hierarchy process 48 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.223 heijungs, r., huppes, g. and guinee, j.b. (2010). life cycle assessment and sustainability analysis of products, materials and technologies: toward a scientific framework for sustainability life cycle analysis. polymer degradation and stability, 95(3), 422-428. herva, m. and roca, e. (2013). review of combined approaches and multi-criteria analysis for corporate environmental evaluation. journal of cleaner production, 39, 355371. jawahir, i.s., rouch, k.e., dillon, o.w., holloway, l. and hall, a. (2007). design for sustainability (dfs): new challenges in developing and implementing a curriculum for next generation design and manufacturing engineers. international journal of engineering education, 23(6), 1053-1064. joung, c.b, carrell, j., sarkar, p. and feng, s.c. (2013). categorization of indicators for sustainable manufacturing. ecological indicators, 24, 148-157. kovac, m. (2012). comparison of foresights in the manufacturing research. transfer inovacii, 23, 284-288. krajnc, d. and glavic, p. (2005). a model for integrated assessment of sustainable development. resources, conservation and recycling, 43(2), 189-208. lai, o.k. (1993). making sense of the greening of consumption and production. journal of cleaner production, 1(1), 43-47. law, k.m.y. and gunasekaran, a. (2012). sustainability development in high-tech manufacturing firms in hong kong: motivators and readiness. international journal of production research, 137(1), 116-125. mani, m., madan, j., lee, j.h., lyons, k. and gupta, s.k. (2012). characterizing sustainability for manufacturing performance assessment. proceedings of the asme 2012 international design engineering technical conferences and computers and information in engineering conference, chicago, il, usa. mayer, a.l. (2008). strengths and weaknesses of common sustainability indices for multidimensional systems. environment international, 34(2), 277-291. ocampo, l.a. and clark, e.e. (2014). developing a framework for manufacturing strategies selection. dlsu business & economics review, 23(2), 115-131. ocampo, l.a. and clark, e.e. (2015). a proposed framework in developing sustainable manufacturing initiatives using analytic hierarchy process. industrial and systems engineering review, 3(1), 7-16. ijahp article: ocampo, clark/an analytic hierarchy process (ahp) approach in the selection of sustainable manufacturing initiatives: a case in a semiconductor manufacturing firm in the philippines international journal of the analytic hierarchy process 49 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.223 promentilla, m.a.b., furuichi, t., ishii, k. and tanikawa, n. (2006). evaluation of remedial countermeasures using the analytic network process. waste management, 26(12), 1410-1421. ragas, a.m.j., knapen, m.j., van de heuvel, p.j.m., eijkenboom, r.g.f.t.m., buise, c.l. and van de laar, b.j. (1995). towards a sustainability indicator for production systems. journal of cleaner production, 3(1-2), 123-129. rosen, m.a., and kishawy, h.a. (2012). sustainable manufacturing and design: concepts, practices and needs. sustainability, 4(2), 154-174. saaty, t. l. (1980). the analytic hierarchy process. mcgraw-hill, new york. seuring, s. (2013). a review of modeling approaches for sustainable supply chain management. decision support systems, 54(4), 1513-1520. singh, r.k., murty, h.r., gupta, s.k. and dikshit, a.k. (2012). an overview of sustainability assessment methodologies. ecological indicators, 15(1), 281-299. smir. (2011). sustainable manufacturing indicator repository, http://www.nist.gov/el/msid/smir.cfm subramanian, n. and ramanathan, r. (2012). a review of applications of analytic hierarchy process in operations management. international journal of production economics, 138(2), 215-241. vaidya, o.s. and kumar, s. (2006). analytic hierarchy process: an overview of applications. european journal of operational research, 169(1), 1-29. yang, m.g., hong, p. and modi, s. (2011). impact of lean manufacturing and environmental management on business performance: an empirical study of manufacturing firms. international journal of production economics, 129(2), 251-261. zhang, s.f. and liu, s.y. (2011). a gra-based intuitionistic fuzzy multi-criteria group decision making method for personnel selection. expert systems with application, 38(9), 11401-11405. ijahp: topcu, ozaydin/mcdm2019 special topics articles international journal of the analytic hierarchy process 2 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.754 mcdm2019 special topics articles after the 25th international conference on multiple-criteria decision making (mcdm2019) in istanbul, one of the most successful mcdm conferences to date, authors that presented their works there were invited to submit these studies to the special topics section of the international journal of the analytic hierarchy process. the conference attracted 261 participants from five continents and 39 countries, and encompassed a wide variety of topics. among the sessions at the conference, 12 were in the ahp/anp stream. in addition, three special sessions, two tutorials, one invited parallel session, and six contributed parallel sessions were all ahp/anp related. the creative decisions foundation (cdf) generously provided 21 grants to participants from 13 countries who had either recently received their phds or were phd students who had prepared works on the ahp/anp. the wide variety of topics is reflected in the papers selected for the ijahp special topics section. these papers went through a set of reviews and three were selected for publication. the first study in this special issue is an application for soil erosion management, while the second is in the healthcare field, and the final one is an assessment of real-estate investments. in summary, these papers represent the outstanding work from mcdm2019, and we are certain they will constitute good ahp/anp references for ijahp readers. ilker topcu professor istanbul technical university department of industrial engineering ilker.topcu@itu.edu.tr ozay ozaydin assistant professor dogus university department of industrial engineering oozaydin@dogus.edu.tr 1_editor_words_vol2_issue2_p77 ijahp preface: mu/words from the editor-in-chief international journal of the 77 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 words from the editor-in-chief dear ijahp readers: this is the second and last ijahp issue for this year and with this we are proud to announce we are completing our second year of existence. we open this issue with an application by islam and ismail showing the use of ahp for the prioritization of longterm objectives in malaysia to fulfill their vision for 2020, the year in which malaysia expects to join the group of developed countries. this article is followed by a case study by khumpaisal et al of how anp can be used in the management of mega-projects such as terminal 5 in the london heathrow international airport. this article is useful because it provides a framework of analysis that includes social, economic, ecological, technical, and political factors for the management of mega-projects. this article is followed by the pecchia et al article about the use of an ahp web-based questionnaire to assess fall risks factors among elderly people. the last paper in our peer-reviewed section is written by stan lipovetsky and is an interpretation of the ahp eigenvector solution for the lay person. our essay section also offers an article about the same topic written by the ahp creator, thomas saaty. rozann saaty, ijahp managing editor has graciously written an introduction to the two articles. enjoy this issue! enrique mu, phd ijahp editor-in-chief rob typewritten text rob typewritten text rob typewritten text rob typewritten text rob typewritten text http://dx.doi.org/10.13033/ijahp.v2i2.85 rob typewritten text rob typewritten text dynamic vendor selection using fuzzy ahp ijahp article: koul, verma/dynamic vendor selection: a fuzzy ahp approach international journal of the analytic hierarchy process 118 vol. 4 issue 2 2012 issn 1936-6744 dynamic vendor selection: a fuzzy ahp approach saroj koul* jindal global business school, op jindal global university, ncr of delhi, 131001 (india) skoul@jgu.edu.in rakesh verma operations management group, national institute of industrial engineering (nitie), mumbai, 400087 (india) rakeshverma@nitie.edu abstract in the ever-changing business world, appropriate vendor selection can be crucial in supply chain management. dynamic models supporting vendors over time are not always crisp; rather they involve a high degree of fuzziness and uncertainty in real life situations. this paper proposes a dynamic model with uncertainty based on fuzzy ahp for longterm strategic vendor selection problems. the selection of partnership suppliers is illustrated by this methodology. keywords: analytic hierarchy process, ahp, dynamic model, fuzzy sets, supply chain management 1. introduction in today’s highly competitive and interrelated manufacturing environment, materials represent a substantial part of the value of products. because material cost accounts for such a high percentage of the total cost, the key objective of the purchasing department ought to be obtaining the right quality of a product in the right quantity from the right source at the right time. the right source can provide the right quality of material on time at a reasonable price (heizer and render, 2001). supplier selection and evaluation are very important to the success of a manufacturing firm because the cost and quality of goods and services sold are directly related to the cost and quality of goods and services purchased. therefore, purchasing and supplier selection have an important role in the supply chain process (hartley and choi, 1996; degraeve, labro, and roodhooft, 2000). traditionally, vendors are selected based on their ability to meet the quality requirements, their delivery schedule, and the price * corresponding author rob typewritten text http://dx.doi.org/10.13033/ijahp.v4i2.25 rob typewritten text ijahp article: koul, verma/dynamic vendor selection: a fuzzy ahp approach international journal of the analytic hierarchy process 119 vol. 4 issue 2 2012 issn 1936-6744 offered. vendor selection and evaluation is a common problem in the process of acquiring the necessary materials to support the outputs of an organization. the problem involves finding and periodically evaluating the most suitable vendor(s) based on various vendors’ capabilities. this is especially difficult when a purchase is complex, of highdollar value and perhaps critical. a process of formal vendor evaluation and ranking is also necessary. the process of vendor selection is a problem-solving process which involves the work of problem definition, formulation of criteria, qualification, and choice. the supplier selection process is a multi-objective decision encompassing many tangible and intangible factors in a hierarchical manner. traditional methodologies of the supplier selection process in research literature include the cost-ratio method, the categorical method, weighted-point evaluations, mathematical programming models and statistical or probabilistic approaches (yan, yu, and cheng, 2003; oliveria and vadi, 2002). dickson has identified 23 important criteria in the study of supplier decision-making (dickson, 1966). weber et al. (2000) has compiled many articles in this area, and has used a linear weighting model for supplier selection. linear weighting models place a weight on each criterion and provide a total score for each supplier by summing up the supplier’s performance on the criteria multiplied by these weights. hokey min (1994) used a multiattribute utility approach in international supplier selection. using interpretative structural modeling in their study, anukal mandal and s. g. desmukh (1994) developed an analytical framework which combines qualitative and quantitative factors. motwani et al. (1999) developed a model for the supplier selection process in developing countries. youssef et al. (1996) developed a simple model for supplier evaluation and selection in an advanced manufacturing technology environment. mathematical programming models use the techniques of linear programming, mixed integer programming, and goal programming to determine vendor selection and order quantity decisions simultaneously. the purpose of mathematical programming models is to select several vendors in order to minimize or maximize an objective function subject to both vendor and buyer constraints. competitive advantage is often determined by the effectiveness of an organization's supply chain, and as a result the evaluation and selection of suppliers has become an increasingly important management activity. but the evaluation process is complex as much of the data are difficult to obtain and ambiguous or vague to interpret. in addition, the dynamic global environment of changing exchange rates, economic conditions, and technical infrastructure, demand that the pool of potential suppliers be re-evaluated periodically. drawbacks of the above approaches include neglect of multi-period planning horizons for vendor selection, and selection of vendors based on experience and intuition. these approaches are obviously subjective and their weaknesses are clear. to overcome these problems, we use the fuzzy analytic hierarchy process to propose the decision model. 2. dynamic fuzzy ahp method the analytic hierarchy process is a powerful and flexible decision-making process to help people set priorities and make the best decision when both qualitative and quantitative aspects need to be considered. by reducing complex decisions to a series of ijahp article: koul, verma/dynamic vendor selection: a fuzzy ahp approach international journal of the analytic hierarchy process 120 vol. 4 issue 2 2012 issn 1936-6744 one-on-one comparisons and then synthesizing the results, many researchers have concluded that ahp is a useful, practical and systematic method for vendor rating (barbarosoglu and yazgac, 1997); it has certainly been applied successfully. however, in many practical cases the human preference model is uncertain and decision-makers might be reluctant or unable to assign exact numerical values to the comparison judgments. for instance, when evaluating different suppliers, the decision-makers are usually unsure about their level of preference due to incomplete and uncertain information about possible suppliers and their performances. since some of the supplier evaluation criteria are subjective and qualitative, it is very difficult for the decision-maker to express the strength of his preferences and to provide exact pair-wise comparison judgments. for this reason, a methodology based on fuzzy ahp can help to reach an effective decision. in this way, we can deal with the uncertainty and vagueness in the decision process. fuzzy ahp consists of deriving the local priorities from these fuzzy preference ratios, which are subsequently aggregated to form the global priorities. the fuzzy ahp computes fuzzy priorities based on arithmetic operations for fuzzy triangular (or trapezoidal) numbers. to be able to use the fuzzy arithmetic operations, specific assumptions about the forms of membership functions are required. however, the most important criticism directed at fuzzy arithmetic operations is their failure to address the issue of consistency. there is no explicit articulation on what would constitute an inconsistent comparison matrix within the fuzzy ahp context, and equally important on how inconsistent information should be handled. lacking a mechanism to exclude inconsistent data, fuzzy priorities so obtained are likely to be flawed (zimmerman, 1991; buckley, feuring, and hayashi, 2001). in addition to combining the ahp approach with other methods, zaim et al. (2003) has discussed a fuzzy analytic hierarchy based approach for supplier selection in the area of marketing. chan and kumar (2007) extended it by including risk factors involved in global supplier selection to handle the fuzziness of the data involved in deciding the preferences of different decision variables. chen et al. (2007) also employed a hierarchical model using triangular fuzzy numbers to deal with supplier selection problems. benyoucef and mustafa (2007) validated the design of the supplier selection system for a hospital and its underlying fuzzy ahp model. saaty and tran (2007) invalidate the fuzzifying numerical judgments in the analytic hierarchy process. in fuzzy ahp, preferences between alternatives are determined by making fuzzy pairwise comparisons. if appropriate with the fuzzy ahp method, vendor selection criteria are determined and compared to the sub criteria and then the importance level for each criterion is found with the calculation of the process according to the given hierarchy structure. a decision making process arises to select the vendors. according to the purchasing department of any manufacturing company the following criteria set is constructed for supplier selection in subsection 3.1. in a fuzzy pairwise comparison, the decision maker examines two alternatives by considering one criterion and indicates a preference. these comparisons are made using a preference scale, which assigns numerical values to different levels of preference. the standard preference scale used for ahp is a 1-9 scale which lies between “equal importance” to “extreme importance”. however, a 1 to 5 ratio scale is applied for fuzzy ahp. ijahp article: koul, verma/dynamic vendor selection: a fuzzy ahp approach international journal of the analytic hierarchy process 121 vol. 4 issue 2 2012 issn 1936-6744 table 1 fuzzy ahp scale statement tfn absolute (row to column) (7/2, 4, 9/2) very strong (row to column) (5/2, 3, 7/2) fairly strong (row to column) (3/2, 2, 5/2) weak (row to column) (2/3, 1, 3/2) equal (1, 1, 1) weak (column to row) (2/3, 1, 3/2) fairly strong (column to row) (2/5, 1/2, 2,/3) very strong (column to row) (2/7, 1/3, 2/5) absolute (column to row) (2/9, 1/4, 2/7) source: tolga et al, 2005 in this study, the framework of a feasible region of relative weights was adopted. first, allowing the feasible region to include tolerance deviations of the fuzzy ratios, we define fuzzy consistency as the existence of relative weights within the region. second, we devise a maximum/minimum set ranking method to derive a crisp ranking from the global fuzzy weights (noci and toletti, 2000; leung and cao, 2000). the following steps of fuzzy ahp proposed by chang (1996) have been utilized in selecting vendors during a multi-period. according to the method of chang’s extent analysis, each object is taken and an analysis for each goal is performed respectively. therefore, m extent analysis values for each object can be obtained with the following signs: .,...,2,1,,...,, 21 nimmm m ggg iii = where .),...,2,1( mjm j g i = all are tfns. the steps of chang’s extent analysis (chang, 1996) are as follows: step 1: the value of fuzzy synthetic extent with respect to the ith object is defined as: 1 1 11 − = ==       ⊗= ∑∑∑ n i m j j g m j j gi ii mms (1) to obtain ∑ = m j j g i m 1 , the fuzzy addition operation of m extent analysis values for a particular matrix is performed such as: ijahp article: koul, verma/dynamic vendor selection: a fuzzy ahp approach international journal of the analytic hierarchy process 122 vol. 4 issue 2 2012 issn 1936-6744       = ∑∑∑∑ ==== m j j m j j m j j m j j g umlm i 1111 ,, (2) and the following is obtained       ∑∑ = = n i m j j g i m 1 1 , by performing the fuzzy addition operation of .),...,2,1( mjm j g i = such that: =      ∑∑ = = n i m j j g i m 1 1       ∑∑∑ === n i i n i i n i i uml 111 ,, (3) and 1 1 1 − = =       ∑∑ n i m j j g i m can be calculated by the inverse of eq. (3), as follows:             =      ∑∑∑ ∑∑ === − = = n i i n i i n i i n i m j j g lmu m i 111 1 1 1 1 , 1 , 1 (4) step 2: as ),,( 1111 umlm = and ),,( 2222 umlm = are two triangular fuzzy numbers, the degree of possibility of ),,( 2222 umlm = ≥ ),,( 1111 umlm = is defined as: [ ] xy mm yxmmv ≥ =≥ ))(),(min(sup)( 2112 µµ (5) and can be expressed as follows: )()()( 22112 dmmhgtmmv mµ=∩=≥ (6)         −−− − ≥ ≥ = otherwise lmum ul ulif mmif )()( )( 0 1 1122 21 21 12 (7) figure 1 (kahraman et al., 2004) illustrates eq. (6) where d is the ordinate of the highest intersection point d between 1m µ and 2m µ . to compare ),,( 1111 umlm = and ),,( 2222 umlm = , we need both the values of )( 21 mmv ≥ and )( 12 mmv ≥ . ijahp article: koul, verma/dynamic vendor selection: a fuzzy ahp approach international journal of the analytic hierarchy process 123 vol. 4 issue 2 2012 issn 1936-6744 figure 1 intersection between m1 and m2 (kahraman et al., 2004) step 3: the degree possibility for a convex fuzzy number to be greater than k convex fuzzy ),...,2,1( kim i = numbers can be defined by: [ ])(...)()(),...,,( 2121 kk mmandandmmandmmvmmmmv ≥≥≥=≥ kimmv i ,...,2,1),(min =≥= (8) assume that )(min)( kii ssvad ≥= for .;,...,2,1 iknk ≠= (9) then the weight vector is given by t nadadadw ))(),...,(),(( ' 2 ' 1 '' = (10) where ),...,2,1( niai = are the n elements. step 4: via normalization, the normalized weight vectors are: t nadadadw ))(),...,(),(( 21= (11) where w is a non-fuzzy number. 3. an illustrative example this numerical example presents a mathematical model to select suppliers in a multiperiod environment. 3.1. define the criteria for vendor selection the main objective is the selection of the best supplier for a firm in a dynamic environment. the problem has three levels of hierarchy in kth period where (k=1, 2, 3). l2 m2 l1 d u2 m1 u1 ( )12 m~m~v ≥ m2 m1 ijahp article: koul, verma/dynamic vendor selection: a fuzzy ahp approach international journal of the analytic hierarchy process 124 vol. 4 issue 2 2012 issn 1936-6744 the analytic time periods are the past (k=1), present (k=2), and near future (k=3). thus, the decision makers can estimate the relative weights ratios for each pair of alternatives under every attribute as well as the relative weights ratios for the attributes. application of common criteria to all suppliers makes objective comparisons possible. the criteria which are considered here in the selection of the best supplier in dynamic environment are: • quality of the product • delivery • overall cost of the product • flexibility in service the hierarchy of the selection criteria and decision alternatives (i.e., suppliers) in a dynamic environment can be seen in figure 2. in the hierarchy, the overall objective (i.e., the best supplier) is placed at level 1, criteria at level 2, and the suppliers alternatives at level 3 in kth period where k=1,2,3. the above mentioned criterion helps decide the best supplier for an organization in each period. the preference of one over another has been decided by the decision makers. human judgment may not always be crisp and therefore the evaluation scale used by decision makers is illustrated table 1. figure 2 dynamic hierarchy for supplier selection the fuzzy pair wise reciprocal judgments matrix of the decision maker for each criterion with respect to the overall objective (i.e. selecting the best supplier) (see table 2), and for best supplier selection delivery cost quality supplier 1 supplier 2 supplier 3 supplier 4 k=1, 2, 3 k=1, 2, 3 flexibility in service ijahp article: koul, verma/dynamic vendor selection: a fuzzy ahp approach international journal of the analytic hierarchy process 125 vol. 4 issue 2 2012 issn 1936-6744 each supplier with each of the criterion (see table 3-6) in each of the three period are determined with the help of fahp scale defined in table 1. table 2 the criterion fuzzy pairwise comparison matrix in k period (k=1, 2, 3) k quality delivery cost flexibility in service quality 1 2 3 (1,1,1) (1,1,1) (1,1,1) (2/3,1,3/2) (5/2,3,7/2) (3/2,2,5/2) (3/2,2,5/2) (3/2,2,5/2) (5/2,3,7/2) (3/2,2,5/2) (3/2,2,5/2) (7/2,4,9/2) delivery 1 2 3 (2/3,1,3/2) (2/7,1/3,2/5) (2/5,1/2,2/3) (1,1,1) (1,1,1) (1,1,1) (2/5,1/2,2/3) (2/5,1/2,2/3) (5/2,3,7/2) (2/3,1,3/2) (2/3,1,3/2) (5/2,3,7/2) cost 1 2 3 (2/5,1/2,2/3) (2/5,1/2,2/3) (2/7,1/3,2/5) (3/2,2,5/2) (3/2,2,5/2) (2/7,1/3,2/5) (1,1,1) (1,1,1) (1,1,1) (1,1,1) (3/2,2,5/2) (3/2,2,5/2) flexibility in service 1 2 3 (2/5,1/2,2/3) (2/5,1/2,2/3) (2/9,1/4,2/7) (2/3,1,3/2) (2/3,1,3/2) (2/7,1/3,2/5) (1,1,1) (2/5,1/2,2/3) (2/5,1/2,2/3) (1,1,1) (1,1,1) (1,1,1) table 3 fuzzy judgments for alternatives with respect to quality in k period (k=1, 2, 3) quality k supplier 1 supplier 2 supplier 3 supplier 4 supplier 1 1 2 3 (1,1,1) (1,1,1) (1,1,1) (3/2,2,5/2) (2/5,1/2,2/3) (2/3,1,3/2) (3/2,2,5/2) (5/2,3,7/2) (3/2,2,5/2) (7/2,4,9/2) (3/2,2,5/2) (3/2,2,5/2) supplier 2 1 2 3 (2/5,1/2,2/3) (3/2,2,5/2) (2/3,1,3/2) (1,1,1) (1,1,1) (1,1,1) (2/5,1/2,2/3) (2/5,1/2,2/3) (2/5,1/2,2/3) (3/2,2,5/2) (2/3,1,3/2) (2/3,1,3/2) supplier 3 1 2 3 (2/5,1/2,2/3) (2/7,1/3,2/5) (2/5,1/2,2/3) (2/7,1/3,2/5) (3/2,2,5/2) (3/2,2,5/2) (1,1,1) (1,1,1) (1,1,1) (3/2,2,5/2) (3/2,2,5/2) (2/5,1/2,2/3) supplier 4 1 2 3 (2/9,1/4,2/7) (2/5,1/2,2/3) (2/5,1/2,2/3) (2/5,1/2,2/3) (2/3,1,3/2) (2/3,1,3/2) (2/5,1/2,2/3) (2/5,1/2,2/3) (3/2,2,5/2) (1,1,1) (1,1,1) (1,1,1) ijahp article: koul, verma/dynamic vendor selection: a fuzzy ahp approach international journal of the analytic hierarchy process 126 vol. 4 issue 2 2012 issn 1936-6744 table 4 the alternatives fuzzy pair-wise comparison matrix with respect to criteria delivery in period (k=1, 2, 3) delivery k supplier 1 supplier 2 supplier 3 supplier 4 supplier 1 1 2 3 (1,1,1) (1,1,1) (1,1,1) (2/3,1,3/2) (2/9,1/4,2/7) (2/3,1,3/2) (2/3,1,3/2) (2/3,1,3/2) (1,1,1) (2/3,1,3/2) (2/5,1/2,2/3) (2/5,1/2,2/3) supplier 2 1 2 3 (2/3,1,3/2) (7/2,4,9/2) (2/3,1,3/2) (1,1,1) (1,1,1) (1,1,1) (3/2,2,5/2) (3/2,2,5/2) (2/5,1/2,2/3) (3/2,2,5/2) (3/2,2,5/2) (2/3,1,3/2) supplier 3 1 2 3 (2/3,1,3/2) (2/3,1,3/2) (1,1,1) (2/5,1/2,2/3) (2/5,1/2,2/3) (3/2,2,5/2) (1,1,1) (1,1,1) (1,1,1) (3/2,2,5/2) (2/3,1,3/2) (2/5,1/2,2/3) supplier 4 1 2 3 (2/3,1,3/2) (3/2,2,5/2) (3/2,2,5/2) (2/5,1/2,2/3) (2/5,1/2,3/2) (2/3,1,3/2) (2/5,1/2,2/3) (2/3,1,3/2) (3/2,2,5/2) (1,1,1) (1,1,1) (1,1,1) table 5 the alternatives fuzzy pair-wise comparison matrix with respect to criteria cost in k period (k=1, 2, 3) cost k supplier 1 supplier 2 supplier 3 supplier 4 supplier 1 1 2 3 (1,1,1) (1,1,1) (1,1,1) (3/2,2,5/2) (3/2,2,5/2) (3/2,2,5/2) (2/3,1,3/2) (3/2,2,5/2) (3/2,2,5/2) (5/2,3,7/2) (2/3,1,3/2) (2/3,1,3/2) supplier 2 1 2 3 (2/5,1/2,2/3) (2/5,1/2,2/3) (2/5,1/2,2/3) (1,1,1) (1,1,1) (1,1,1) (2/3,1,3/2) (1,1,1) (1,1,1) (3/2,2,5/2) (3/2,2,5/2) (3/2,2,5/2) supplier 3 1 2 3 (2/3,1,3/2) (2/5,1/2,2/3) (2/5,1/2,2/3) (2/3,1,3/2) (1,1,1) (1,1,1) (1,1,1) (1,1,1) (1,1,1) (3/2,2,5/2) (2/5,1/2,2/3) (2/5,1/2,2/3) supplier 4 1 2 3 (2/7,1/3,2/5) (2/3,1,3/2) (2/3,1,3/2) (2/5,1/2,2/3) (2/5,1/2,2/3) (2/5,1/2,2/3) (2/5,1/2,2/3) (2/5,1/2,2/3) (3/2,2,5/2) (1,1,1) (1,1,1) (1,1,1) ijahp article: koul, verma/dynamic vendor selection: a fuzzy ahp approach international journal of the analytic hierarchy process 127 vol. 4 issue 2 2012 issn 1936-6744 table 6 the alternatives fuzzy pair-wise comparison matrix with respect to criteria flexibility in service in k period (k=1, 2, 3) flexibility in service k supplier 1 supplier 2 supplier 3 supplier 4 supplier 1 1 2 3 (1,1,1) (1,1,1) (1,1,1) (3/2,2,5/2) (2/9,1/4,2/7) (2/3,1,3/2) (3/2,2,5/2) (2/3,1,3/2) (2/3,1,3/2) (3/2,2,5/2) (2/5,1/2,2/3) (2/5,1/2,2/3) supplier 2 1 2 3 (2/5,1/2,2/3) (7/2,4,9/2) (2/3,1,3/2) (1,1,1) (1,1,1) (1,1,1) (3/2,2,5/2) (3/2,2,5/2) (2/3,1,3/2) (3/2,2,5/2) (3/2,2,5/2) (2/5,1/2,2/3) supplier 3 1 2 3 (2/5,1/2,2/3) (2/3,1,3/2) (3/2,1,3/2) (2/5,1/2,2/3) (2/5,1/2,2/3) (2/3,1,3/2) (1,1,1) (1,1,1) (1,1,1) (2/3,1,3/2) (2/3,1,3/2) (2/5,1/2,2/3) supplier 4 1 2 3 (2/5,1/2,2/3) (3/2,2,5/2) (3/2,2,5/2) (2/5,1/2,2/3) (2/5,1/2,2/3) (3/2,2,5/2) (2/3,1,3/2) (2/3,1,3/2) (3/2,2,5/2) (1,1,1) (1,1,1) (1,1,1) in order to identify the computation procedures the pair-wise judgments from table 3 for period 3 are evaluated as follows: quality supplier 1 supplier 2 supplier 3 supplier 4 supplier 1 (1,1,1) (2/3,1,3/2) (3/2,2,5/2) (3/2,2,5/2) supplier 2 (2/3,1,3/2) (1,1,1) (2/5,1/2,2/3) (2/3,1,3/2) supplier 3 (2/5,1/2,2/3) (3/2,2,5/2) (1,1,1) (2/5,1/2,2/3) supplier 4 (2/5,1/2,2/3) (2/3,1,3/2) (3/2,2,5/2) (1,1,1) 1 1 1 1(4.667, 6, 7.5) ( , , )22.667 18.00 14.267 (0.206, 0.333, 0.526) qsupplier = ⊗ = 2 1 1 1(2.733, 3.5, 4.667) ( , , )22.667 18.00 14.267 (0.121, 0.194, 0.327) qsupplier = ⊗ = ijahp article: koul, verma/dynamic vendor selection: a fuzzy ahp approach international journal of the analytic hierarchy process 128 vol. 4 issue 2 2012 issn 1936-6744 3 1 1 1(3.30, 4.0, 4.833) ( , , )22.667 18.00 14.267 (0.146, 0.222, 0.339) qsupplier = ⊗ = 4 1 1 1(3.567, 4.5, 5.667) ( , , )22.667 18.00 14.267 (0.157, 0.250, 0.327) qsupplier = ⊗ = after determining these results, these fuzzy values are compared by using eq. (7): 1 2 1 3 1 4 ( ) 1, ( ) 1, ( ) 1 q q q q q q v supplier supplier v supplier supplier v supplier supplier ≥ = ≥ = ≥ = 2 1 2 3 2 4 ( ) 0.466, ( ) 0.867, ( ) 0.753 q q q q q q v supplier supplier v supplier supplier v supplier supplier ≥ = ≥ = ≥ = 3 1 3 2 3 4 ( ) 0.545, ( ) 1, ( ) 0.867 q q q q q q v supplier supplier v supplier supplier v supplier supplier ≥ = ≥ = ≥ = 4 1 4 2 4 3 ( ) 0.697, ( ) 1, ( ) 1 q q q q q q v supplier supplier v supplier supplier v supplier supplier ≥ = ≥ = ≥ = then priority weights are calculated by using eq. (8): 1)1,1,1min()( 1 ' ==qsupplierd 466.0)753.0,867.0,466.0min()( 2 ' ==qsupplierd 545.0)867.0,1,545.0min()( 3 ' ==qsupplierd 697.0)1,1,697.0min()( 4 ' ==qsupplierd therefore, the weight vector from table 3 for period 3 is calculated as: )697.0,545.0,466.0,1(=supplierqw after the normalization of these values priority weights with respect to criteria quality are calculated as: )154.0,261.0,187.0,397.0(=supplierqw ijahp article: koul, verma/dynamic vendor selection: a fuzzy ahp approach international journal of the analytic hierarchy process 129 vol. 4 issue 2 2012 issn 1936-6744 the same systematic approach is considered for the other evaluations, and priority weights are expressed correspondingly in tables 7, 8, 9 and 10 as follows. table 11 represents the priority weights of the four suppliers in all the three periods, and is obtained by multiplying the priority weights of criteria to the suppliers’ weights with respect to all criteria in each period respectively. 4. discussion of results as we can see in figure 3, the priority of quality improvement is important for every supplier for all of the periods. we can also observe from table 7 that cost is more important than delivery in the first two periods. figure 4 shows the trend of supplier’s priority; supplier 1 is the most preferred in the first two periods, but supplier 4 over performed supplier 1 by a small margin in third period. similarly supplier 2 is the most preferred over supplier 3 in the first two periods, but in the third period supplier 3 over performed supplier 2 with by a large margin. hence, we can conclude that if we have to select only one supplier then supplier 1 will be chosen for the first two periods, and supplier 4 for the third period. however, when we have selected two or more suppliers then there will be an option as shown in table 12. table 7 the priority weights of criteria in period 1-3 criteria ▼ period ▼ 1 2 3 quality 0.40 0.65 0.66 delivery 0.19 0.00 0.34 cost 0.26 0.35 0.00 flexibility in service 0.15 0.00 0.00 table 8 the priority weights of suppliers with respect to all criteria in period 1 suppliers ▼ criteria quality delivery cost flexibility in service 1 1.00 0.24 0.47 0.53 2 0.00 0.37 0.23 0.39 3 0.00 0.27 0.30 0.04 4 0.00 0.12 0.00 0.04 ijahp article: koul, verma/dynamic vendor selection: a fuzzy ahp approach international journal of the analytic hierarchy process 130 vol. 4 issue 2 2012 issn 1936-6744 table 9 the priority weights of suppliers with respect to all criteria in period 2 suppliers ▼ criteria quality delivery cost flexibility in service 1 0.42 0.00 0.53 0.00 2 0.23 0.76 0.33 0.86 3 0.31 0.00 0.03 0.00 4 0.03 0.24 0.11 0.14 table 10 the priority weights of suppliers with respect to all criteria in period 3 suppliers ▼ criteria quality delivery cost flexibility in service 1 0.37 0.14 0.43 0.15 2 0.17 0.18 0.26 0.15 3 0.20 0.26 0.02 0.15 4 0.26 0.42 0.29 0.55 table 11 the priority weights of suppliers in periods 1-3 suppliers ▼ period 1 2 3 1 0.65 0.46 0.29 2 0.19 0.27 0.17 3 0.13 0.22 0.22 4 0.03 0.06 0.31 ijahp article: koul, verma/dynamic vendor selection: a fuzzy ahp approach international journal of the analytic hierarchy process 131 vol. 4 issue 2 2012 issn 1936-6744 figure 3 priorities of criteria in three periods figure 4 priority weights of suppliers in three periods ijahp article: koul, verma/dynamic vendor selection: a fuzzy ahp approach international journal of the analytic hierarchy process 132 vol. 4 issue 2 2012 issn 1936-6744 table 12 order of supplier selection in three periods no. of suppliers ▼ period 1 2 3 1 s1 s1 s4 2 s1, s2 s1, s2 s1, s4 3 s1, s2, s3 s1, s2, s3 s1, s3, s4 4 s1, s2, s3, s4 s1, s2, s3, s4 s1, s2, s3, s4 fuzzy ahp uses intersection operation while evaluating comparison results. the result of the fuzzy intersection can be obtained as zero (e.g. the priority weights of criteria in period 2 is (0.65, 0.00, 0.35, 0.00)t which means that the corresponding criterion has no importance. this finding raises the question “if this criterion is of concern to the decision, then how can it have a zero importance?” in fact, it is an ordinary consequence of fuzzy logic. fuzzy pair wise comparisons show that if a criterion is less important than all of the others, then relatively this criterion has no importance and the weight is zero. even if it is declared that a criterion is relevant for the decision making process, it has no importance when compared with the others. in the classic ahp method, deterministic values and operations do not permit the situation “having zero weight”. but, if a criterion is evaluated as “less than all of the others”, then the numerical result of this situation, the weight of this criterion, would be near to zero. furthermore, the weight can descend to 0.01 which means that this criterion is not so important to the final decision. fuzzy ahp totally neglects the criterion which is less important than the others whereas classical ahp uses this criterion with a small weight. the presentation of this additional information for the decision maker, that there is no difference between the existence or nonexistence of such a criterion, can be an advantage for fuzzy ahp. therefore, the decision maker can focus on the more important criteria. linguistic and subjective evaluations take place in questionnaire form. each linguistic variable has its own numerical value in the predefined scale. in classical ahp these numerical values are exact numbers, whereas in the fuzzy ahp method they are intervals between two numbers with a most likely value. as is the nature of the human beings, linguistic values can change from person to person. in these circumstances, taking the fuzziness into account will result in less risky decisions. here, classical and fuzzy methods are not competitors when the same conditions are present. the important point is that if the information/evaluations are certain, the classical method should be preferred. if the information/evaluations are not certain, the fuzzy method should be preferred. in recent years, because of the characteristics of information and decision makers, probable deviation should be integrated into the decision making processes. because of this a fuzzy version has been developed for each decision making method. the fuzzy ahp method is a natural result of this. ijahp article: koul, verma/dynamic vendor selection: a fuzzy ahp approach international journal of the analytic hierarchy process 133 vol. 4 issue 2 2012 issn 1936-6744 5. conclusions customarily in global supply chain management, companies must select suitable suppliers over a long period of time. in dynamic business environments, attributes and weights may change over time and the actual decision may not be crisp, but rather fuzzy in nature. traditional multiple attribute decision-making methods may not solve the longterm performance measurement problems in a fuzzy environment. in this methodology, one cannot find a consistent process for fuzzy inputs and crisp weights and the consistency index method is not appropriate because of the fuzziness. in fact, chang’s fuzzy ahp comprises such a mechanism during the pairwise calculations when the membership values or possibilities are compared and the intersections are obtained. furthermore, the fuzziness concept has some bias such as the inconsistency of the decision maker. this paper proposes a dynamic approach based on fuzzy ahp for supplier selection problems that can help reach an effective decision. this makes it possible to deal with the uncertainty and vagueness in the decision process. therefore, for each decision making method, a fuzzy version has been developed. fuzzy ahp method is a natural result of this. for future work, we will try to extend the fuzzy ahp method to an intuitionistic fuzzy ahp method to select the suppliers. acknowledgements the authors are grateful to the reviewers for their valuable suggestions that have made the paper more systematic and instructive. ijahp article: koul, verma/dynamic vendor selection: a fuzzy ahp approach international journal of the analytic hierarchy process 134 vol. 4 issue 2 2012 issn 1936-6744 references barbarosoglu, g., and yazgac, t. (1997). an application of the analytic hierarchy process to the supplier selection problem. production and inventory management journal, 38(1), 14-21. bellman, r.e., and zadeh, l.a. (1970). decision making in a fuzzy environment. management sciences, 17, 141-164. benyoucef, m., and canbolat, m. (2007). fuzzy ahp-based supplier selection in eprocurement. international journal of services and operations management, 3(2), 172 – 192. buckley j.j., feuring t., and hayashi y., (2001). fuzzy hierarchical analysis revisited. european journal of operational research, 129, 48-64. chan, f.t.s. and kumar, n. (2007) global supplier development considering risk factors using fuzzy extended ahp-based approach, omega, 35( 4), 417-431. chang, d. (1996). application of the extent analysis method on fuzzy ahp. european journal of operational research, 95, 649-655. chen, c.t., ching-t.l., and huang, s.f. (2006). a fuzzy approach for supplier evaluation and selection in supply chain management. international journal of production economics, 102(2), 289-301. degraeve, z., labro, e., & roodhooft, f., (2000). an evaluation of vendor selection models from a total cost of ownership perspective. european journal of operational research, 125, 34-58. dickson, g., (1966). an analysis of vendor selection systems and decisions. journal of purchasing, 2(1), 5-17. hartley, j.l., & choi, t.y., (1996). supplier development: customer as a catalyst of process change. business horizons, 39(4), 37-40. heizer, j.& render, b. (2001). operations management. upper saddle river: prentice hall, 431-457. kahraman c., cebeci, u., and ruan, d. (2004) multi-attribute comparison of catering service companies using fuzzy ahp: the case of turkey. international journal of production economics, 87, 171–184. leung, l.c., and cao, d. (2000). theory and methodology, on consistency and ranking of alternatives in fuzzy ahp. european journal of operational research, 124, 102-113. ijahp article: koul, verma/dynamic vendor selection: a fuzzy ahp approach international journal of the analytic hierarchy process 135 vol. 4 issue 2 2012 issn 1936-6744 mandal, a., and deshmukh, s.g. (1994). vendor selection using interpretive structural modeling. international journal of operations & production management, 14 (6), 52-59. motwani, j., youssef, m., kathawala, y., and futch, e. (1999). supplier selection in developing countries: a model development. integrated manufacturing system, 10(3), 154-161. muralidharan, c., anantharaman, n., and deshmukh, s.g. (2002). a multi-criteria group decision-making model for supplier rating. the journal of supply chain management, fall, 22-33. min, h., (1994). international supplier selection. international journal of physical distribution & logistics management, 24(5), 24-33. noci, g., & toletti, g., (2000). selecting quality-based programmes in small firms: a comparison between the fuzzy linguistic approach and the analytic hierarchy process. international journal of production economics, 67, 113-133. oliveira, r.c., & lourenço, j.c., (2002). a multi-criteria model for assigning new orders to service suppliers. european journal of operational research, 139(2), 390-399. saaty, t.l. (1980). the analytical hierarchy process. new york, ny: mcgraw-hill. saaty, t.l and tran, l. t., (2007). on the invalidity of fuzzifying numerical judgments in the analytic hierarchy process. mathematical and computer modelling, 46(7-8), 962 – 975. tolga, e., demircan, m. l. & kahraman, c. (2005) operating system selection using fuzzy replacement analysis and analytic hierarchy process. international journal of production economics, 97, 89-117. weber, c. a., current, j. r., & desai, a. (2000). an optimization approach to determining the number of vendors to employ. supply chain management: an international journal, 2(5), 90-98. yan, h., yu, z., & cheng, t.c.e. (2003). a strategic model for supply chain design with logical constraints: formulation and solution. computers & operations research, 30(14), 2135-2155. youssef, m.a., zairi, m., & mohanty, b. (1996). supplier selection in an advanced manufacturing technology environment: an optimization model. benchmarking for quality management & technology, 3(4), 60-72. zadeh, l.a., (1965). fuzzy sets. information and control, 8, 338-353. ijahp article: koul, verma/dynamic vendor selection: a fuzzy ahp approach international journal of the analytic hierarchy process 136 vol. 4 issue 2 2012 issn 1936-6744 zaim, s., sevkli,m. and tarim,m. (2003). fuzzy analytic hierarchy based approach for supplier selection, journal of euromarketing, 12(3/4), 147-176. zimmermann, h.j. (1991). fuzzy set theory and its applications. dordrecht: kluwer academic publishers. ijahp article: rajguru, shah, ramkrishnan/ change dynamics in maharashtra state small farming systemsfield survey and analysis through analytic hierarchy process international journal of the analytic hierarchy process 228 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.266 change dynamics in maharashtra state small farming systems field survey and analysis through analytical hierarchy process anand j. rajguru centre for technology alternatives for rural areas (ctara) indian institute of technology bombay (iitb) mumbai, maharashtra, india e-mail: rajgurutoo@gmail.com narendra g. shah centre for technology alternatives for rural areas (ctara) indian institute of technology bombay (iitb) mumbai, maharashtra, india e-mail: nshah@iitb.ac.in ganesh ramkrishnan department of computer science and engineering indian institute of technology bombay (iitb) mumbai, maharashtra, india e-mail: ganesh@cse.iitb.ac.in abstract several issues threaten the sustainability of small farms. during the 1970-80’s, the green revolution helped the indian food supplies become more self-sufficient; however, the post green revolution situation is not certain. farmers prefer and accept farming practices that provide good economic returns as well as proper care for the environment. the scientific community, policy makers and farmers themselves are confronted with difficult decisions in selecting the optimal agricultural practices. this paper reports results from a study that uses a multi-criteria analysis tool called ahp (analytical hierarchical process) to assess options such as chemical, organic and partial organic farming. farmers practicing these three different farming systems were interviewed and asked to fill out a questionnaire, and both quantitative and qualitative attributes was synthesized. three different crops (i.e. rice, sugarcane and mango) were considered for ahp analysis. the ahp methodology is useful in explaining multi-criteria analysis to the extension agents and opinion makers, and may help convince these people to support the use of best farming practices while keeping sustainability in mind (kiyotada h., 2000). keywords: farming systems, multi-criteria analysis, ahp ranking 1. introduction india has 329 million hectares of agricultural land, which amounts to 0.28 ha/ca (hectare per capita) as compared to the world’s average of 0.76 ha/ca. in order to maintain the current food consumption levels per capita, assuming current rates of population growth and no further agricultural intensification, agriculture would need an area equivalent to one half of the current terrestrial land area by 2030 and twothird by 2070 (chaudhary et al., 2005). at present, india's agricultural growth rate is exceeding the population growth rate. however, indian farming systems also need to address the ‘sustainability’ question. indian farmers are facing serious problems such mailto:rajgurutoo@gmail.com ijahp article: rajguru, shah, ramkrishnan/ change dynamics in maharashtra state small farming systemsfield survey and analysis through analytic hierarchy process international journal of the analytic hierarchy process 229 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.266 as declining ground water levels, electricity shortages, marketing risk, and climate variability. traditional culture and social values are changing as the attraction of migrating to cities is increasing. following a monotonic crop pattern for many years, farmers sell the surplus of their production after fulfilling their own requirements. most of the farmers have small areas of land, and the income from these small farms is not enough to sustain their livelihood. therefore, these farmers are dependent on other livelihood options like small jobs in towns or nearby larger farms. the younger generation is not interested in farm related work due to the availability of less laborious job opportunities in towns and cities that seemingly provide higher cash returns. during the course of our surveys, we observed that 30-40 % of the available farmland has been kept fallow for a stretch of several years. one of the reasons for this is that even though the yield of the crop has not drastically reduced (physical/energy ratios more than 1 as explained in section 2.1), the economic efficiency of agriculture is much lower than other occupations in the studied areas. future cropping systems and farm management options need to address these issues in order to reduce sustainability risks. such farm practices would potentially drive cost reduction in agriculture inputs, advanced holistic farm training, transport cost reduction of farm produce, knowledge about local market, value addition, etc. with all this in mind, it is important that farmers are provided inputs for decision making based on environmental, social, technical and economic perspectives. in this article, the analytical hierarchy process (ahp) is used for policy planning at the village level, and its application in choosing the best farming practices between organic, chemical and partial organic farming, is explained from each of the four perspectives mentioned above. 1.1 chemical, organic and partial organic farming systems in india, a variety of farming practices have been prevalent since ancient times. this can be partially attributed to the fact that india has different climatic zones. this is an advantage as multiple crops can be grown and different cultivation patterns can be adopted in different regions. however, regardless of the zones and crop patterns, three different farming practices exist in india in general, namely, organic farming, partial organic farming and chemical farming. organic farming is characterized by the absence of external chemical inputs. partial organic farming practices involve minimal external chemical input along with organic input from the farm itself, while chemical farming relies completely on specified external chemical input. even within organic farming, a variety of farming practices exist in india. there are several organic fertilizers and pesticides, each having its own effect on crops and human health. many of the organic farming practices are diminishing because of industrialized farming and market driven external inputs. the usage aspect of fertilizer that originated with the green revolution has been overdone, leading to soil degradation and subsequent reduction in yield, irrespective of increase in the external inputs of fertilizers, seeds and pesticides. organic farming is based on the foundational principle of maintaining soil nutrition. however, introduction of organic practices does not seem to be increasing the yield in the proportion desired to satisfy the ever growing needs. reddy (2010) provides an accepted definition of organic farming as: “organic agriculture is holistic production management systems which promote and enhance agro-ecosystem health, including biodiversity, biological cycles and soil biological activity”. reddy also cites a number of studies which gives yield/productivities at higher as well as lower levels. recently, the idea of an evergreen revolution, promoted by several agriculture planning bodies in india, stresses that the crop productivities can be enhanced by increasing the soil nutrition ijahp article: rajguru, shah, ramkrishnan/ change dynamics in maharashtra state small farming systemsfield survey and analysis through analytic hierarchy process international journal of the analytic hierarchy process 230 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.266 and by using a high yielding variety of seeds, as well as biotic and abiotic stress resistance variety of seeds. the evergreen revolution has its roots in biotechnology. while the profit and market oriented green revolution served its purpose in the initial years, the indiscriminate use of fertilizers and pesticides has led to undesirable consequences two and half decades later, especially in states such as punjab and haryana. the yield and underground water levels have decreased drastically. water pollution, soil salinity, pesticide residue, indebtedness and suicides are common problems faced by the present day indian farmer. there is an increasing awareness amongst farmers about the economic effects of the green revolution on soil and yield. partial organic farming practices are increasingly turning out to be a socially, technically, environmentally as well as economically viable farming practice in several parts of india. several ngos have been motivating farmers to mix organic biomass and fertilizers with the soil. research has shown that application of a small percent of chemical fertilizer and organic biomass/fertilizers gives more yield than the exclusive use of organic or chemical farming practices associated with environmental benefits (schneider et al., 2011). the agriculture marketing system is one of the most complex issues due to governance and control by both local and central government bodies. after selling agriculture produce to a commission agent at an apmc (agricultural produce market committee) market, farmers only get a 30% return by the time the produce reaches the consumer. this results in the consumer getting costly food and the farmers getting limited rewards. 2. agriculture-food-health connection (consumer concerns) food obtained by organic farming has been found to be richer in nutrients like vitamin c, iron, magnesium, and phosphorus and contain significantly fewer nitrates as compared to chemically grown food (worthington et al., 2001). during the course of experiments performed in our field surveys, we found that 1 kg of chemically grown sugarcane contains a sweetness equivalent to 0.65 kg of its organic counterpart. additionally, the final dry mass of chemical sugarcane is 10% less than those grown organically. similar results have been observed with other vegetables (woese et al., 1997). chemically grown vegetables contain a lower percentage of antioxidants in comparison to organic produce (ren et al., 2001; carbonaro et al., 2002). in summary, chemically grown food, while apparently solving the problem of quantity of production, potentially compromises on quality, measured in terms of nutrient content. moreover, pesticide residues can cause long-term chronic diseases like cancer (dich et al., 1997; zahm & blair, 1992), liver and kidney damage (amr, 1999), disorders of the nervous system (dich et al., 1997), birth defects (garry, schreinemachers & harkins, 1996), and disruption of the immune system (zahm & blair, 1992). pesticides can cause irreparable health disorders as they accumulate in the body fat. a report (mathur, 2005) found nearly six to thirteen pesticides in virtually all blood samples, some of them were: hch, aldrin, ddt, monocrotophos, endosulfan, phosphamidon, chlorpyrifos and malathion. past studies in punjab have revealed pesticide residues in breast milk (kalra, singh & battu, 1994), milk from cattle (kalra et al., 1999), and fruits and vegetables. pesticide use in india has increased by two orders of magnitude from 154 million tons in 1954 to 88,000 million tons in 200l. punjab is one of the largest consumers of pesticides at 6,972 million tons a year (menon ramesh, 2005). figure 1 (rao, c. h. s. et al., 2005) shows the increasing number of cases of acute pesticide poisoning in a hospital in andhra pradesh (ap), and this is representative of other parts of india. chronic diseases account for the largest number of deaths in india with communicable diseases, maternal and prenatal conditions, and nutritional deficiencies following behind (world health organization). these signs indicate that while india’s food security may be addressed in coming years, food safety concerns still need to be ijahp article: rajguru, shah, ramkrishnan/ change dynamics in maharashtra state small farming systemsfield survey and analysis through analytic hierarchy process international journal of the analytic hierarchy process 231 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.266 addressed. we found in our survey that 85% of the farmers growing vegetables and fruits with the aid of pesticides and fertilizers do not eat the vegetables and fruits from their own farms. however, they do consciously isolate some areas in their farm that are kept free from chemical pesticides and fertilizers, and personally use produce only from such areas. figure 1. yearly admission and deaths related to pesticide toxicity at the mahatma gandhi memorial hospital warangal (ap) during (1997-20002) 2.1 physical /energy conversion ratios and financial efficiencies energy conversion ratio in paddy cultivation was computed by considering energy equivalents of seed, fertilizer (organic and inorganic) human and animal labour, tractor and the paddy crop as output (ramchndra, & nagarathna, 2001). energy efficiency of a rice crop in the surveyed areas was found to be 3.16. costs of various farming operations for the rice crop in the nane and vadap villages in the konkan area of maharshtra (india), and the revenue generated in monetary units were used in computing economic efficiencies. it was found that the economic efficiency for the paddy crop is 0.3. these two efficiency numbers indicate that while the energy efficiencies are adequate the economic efficiencies need improvement. 3. call for technological changes during our survey, it became apparent that the undesired effects of the green revolution are well understood by indian farmers. this realization over the past few years has prompted the use of compost and biomass in many parts of india. however, this method needs larger acceptance and practice. in this paper, we used a multicriteria decision making technique called the analytical hierarchy process (ahp) to analyze and rank the alternative technologies for farming, in order to enhance the practice of doing alternate farming practice amongst a wider mass that includes the development workers. while we have used ahp for ranking three high level alternative farming practices based on a combination of qualitative and quantitative attributes, the same procedure can be used to select amongst more specific alternatives, such as the choice of certain crops in certain seasons, the cropping patterns, the choice of seed quality, the type of tillage to be employed, the appropriate methods for water conservation, the alternatives for water pumping, the selection of markets and so on. the ahp process was applied from each of 4 different perspectives, namely, economic, environmental, technological and social. ijahp article: rajguru, shah, ramkrishnan/ change dynamics in maharashtra state small farming systemsfield survey and analysis through analytic hierarchy process international journal of the analytic hierarchy process 232 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.266 4. analytical hierarchy process (ahp) the analytic hierarchy process (ahp) is a structured technique for multi-criteria decision making. ranking of solution alternatives is based on mathematics and human psychology. rather than prescribing a “correct” decision, the ahp helps people to prioritize from available alternatives. inputs to the ahp are: a set of technology alternatives a set of qualitative and quantitative attributes which potentially influence the preference of one technology over the other (these could be optimally arranged in hierarchy) values of the quantitative attributes for each alternative could be cost/benefit depending on the perspective of the decision maker. pair-wise preference values of the technology alternatives with respect to each qualitative attribute. pair wise preference values of each attribute with respect to every other attribute. the output of the ahp procedure is a consolidated score for each technology alternative based on how the alternatives are ranked. internally, ahp uses eigen decomposition analysis of the pair wise comparison matrices to compute relative weights of the attributes and weights of the alternatives with respect to qualitative attributes. the strength of the ahp that is leveraged in this work is its ability to analyze such qualitative/subjective inputs by converting them into quantitative/objective values. additionally, the ahp provides indicators reflecting the consistency of the qualitative inputs. for example, if a is twice as preferred as b, and b is twice as preferred as c, then a being only half as preferred as c could lead to inconsistency that needs to be dealt with. our analysis in the work presented here is heavily driven by qualitative as well as quantitative inputs from farmers which makes the ahp a suitable ranking tool. rezaei-moghaddam & karami (2007) present the use of ahp for the selection of a sustainable agriculture development model between the two alternatives of ecological modernization and de-modernization. they employ nine criteria in the selection process. four different groups, namely, elite farmers, lapuei’s women committee, lapuei’s sustainable development cooperative, and the green development society were interviewed with questions based on these nine different criteria. alphonce (1997) presented possible uses of ahp for decision making in agriculture. the following problems were explored: farm portions to allocate to each of the food crops, resource allocation to agricultural activities, best location for a village store, choice between subsistence and cash crops production, and determination of the crop production technology mawapanga and debertin (1996) explain the use of ahp in multi-criteria decision making in agriculture using a participatory approach. three alternative farming practices i.e., conventional farming, biodynamic farming and organic farming are analyzed using four perspectives, namely, sustainability, profit maximization, health concerns and environmental concerns. karami (2006) presents analysis of alternative water irrigation systems with three groups of farmers using nine objective criteria. in parra-lopez, calatrava-requena and de-haro-gimenez (2008) they use ahp for choosing a best alternative amongst different farming systems (that is, organic, conventional and integrated) for the production of olives. the author studied four ijahp article: rajguru, shah, ramkrishnan/ change dynamics in maharashtra state small farming systemsfield survey and analysis through analytic hierarchy process international journal of the analytic hierarchy process 233 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.266 perspectives which were social, technological, environmental and economic. mahajan, ramkrishnan & date (2008) present web based java software for ahp. 5. objectives the objectives of our research are as follows: a. to capture the changes in agricultural practice in specific regions in western india through a sample survey of farmers for three crops: rice, mango and sugarcane. b. to examine the process of decision making by using a multi-criteria analysis tool called analytical hierarchy process (ahp). 6. research methodology the attributes to be used in the ahp model were chosen based on discussions with agriculture specialists, farmers, agriculture university experts and relevant literature. the attributes chosen focused on those things which were in line with the interests of the farmers. farmers farm in order to have sufficient food for their families, and then they sell the surplus to increase their income. however, when the objective of farming is monetary reward, the farmer employs all necessary means keeping in mind yield enhancement. after facing the side effects of chemical farming, some farmers have realized the importance of sustainable agricultural practices and are beginning to be interested in environmental, social and technological perspectives. three different crop systems i.e. rice, sugarcane and, mango were studied. figure 2 depicts a schematic representation of the different perspectives and attributes (both qualitative and quantitative) considered while making decisions on choice of the farming practice. following saaty (1994), we used numerical ratings (in the range of 1-9) for qualitative attributes, as prescribed in table 1. in this study, the regions that were selected for the application of ahp over three different crop systems are from thane, kolhapur, solapur, and jalgaon districts in maharashtra (western state in india). the choice of criteria, perspectives, and alternatives were decided upon after consulting the farmers and agricultural experts. farmers were selected based on the field survey and those who showed interest in our study. detailed questionnaires were designed and revised based on experiences from the field survey. three different farming practices, namely, organic farming, chemical farming and partial organic farming were selected as technological alternatives. these alternatives are selected based on 32 sub criteria under four perspectives (i.e., economic, environmental, social and technological). the four perspectives are described in the next section. in this survey, we collected data from 63 farmers (17 organic, 23 chemical and 23 partial organic) who agreed to respond to our questionnaires. thus, our sampling method is not random, but it is purposive sampling. the socio-economic and educational backgrounds of the farmers who were surveyed are provided in tables 3, 5, and 6 for sugarcane, mango and rice respectively. ijahp article: rajguru, shah, ramkrishnan/ change dynamics in maharashtra state small farming systemsfield survey and analysis through analytic hierarchy process international journal of the analytic hierarchy process 232 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.266 figure 2. ahp model for choosing sustainable agriculture practice for sugarcane, rice and mango quantitative attributes qualitative attributes income(a1) yield(a2) returns from livestock(a3) total operations cost(a4) income from other sources(a5) stability of income over time(a16) independence to external agriculture inputs(a17) trade and sale opportunities(a18) productivity(a6) percentage of successful harvest(a7) no of crops per year(a8) farmers work health conditions (a19) quality of products(a20) orientation to crop rotation(a21) training facility(a22) water requirement(a12) organic carbon(a13) soil ph(a14) water ph(a15) soil fertility(a28) biodiversity(a29) soil erosion(a30) direct local employment(a9) indirect local employment(a10) highest education in family (a11) social justice in rural areas(a23) recreational use(a24) compatibility with local sociocultural values(a25) migration opportunity(a26) success to grow on waste land(a27) organic farming partial organic farming chemical farming economic perspective technological perspective social perspective environmental perspective sustainable farming practice ijahp article: rajguru, shah, ramkrishnan/ change dynamics in maharashtra state small farming systemsfield survey and analysis through analytic hierarchy process international journal of the analytic hierarchy process 233 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.266 table 1 judgment scores for the importance/preference of criteria using ahp (saaty, 1994) verbal judgment numerical rating extremely important / preferred 9 very strongly to extremely important / preferred 8 very strongly important / preferred 7 strongly to very strongly important / preferred 6 strongly important / preferred 5 moderately to strongly important / preferred 4 moderately important / preferred 3 equally to moderately important / preferred 2 equally important / preferred 1 6.1 economic perspective  farm income is calculated based on the total sales and expenses throughout the year. it includes the interest on capital, external farm inputs, salaries to workers, and other expenses. these items are restricted only for the crop under study.  income from other sources includes income from animals, selling vermincompost, income from intercrop. income other than farming activity is not considered.  total operations cost is the cost of land preparation, fertilizer, salaries of workers, pesticides, water, fuel etc.  value of major livestock owned is the value of animals which farmers own on their farm.  yield is the output from the crops after harvesting (kg/acre)  stability of income over time is the measure, which illustrates whether income coming from the farm is long periodic or short periodic. it is from the intercrops like vegetables, flowers etc. that give regular income.  independence from external agriculture inputs like chemical fertilizers, seed, pesticides, farm mechanization machinery etc.  trade and sale opportunities are the opportunities available for selling the agriculture products. discussions with farmers indicate that they feel more comfortable if their products get sold on their doorstep. 6.2 technological perspective  productivity is measured as the ratio of agricultural outputs to agricultural inputs. percentage of successful harvest depends on the risk associated with a contingent loss of production usually related to biological and agronomic conditions of the cultivation.  number of crops per year is the total number of crops on the same farm throughout the year.  farmers work health conditions relates to health problems due to chemical pesticide, water pollution, and new pest varieties etc. ijahp article: rajguru, shah, ramkrishnan/ change dynamics in maharashtra state small farming systemsfield survey and analysis through analytic hierarchy process international journal of the analytic hierarchy process 234 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.266  quality of products is ascertained from criteria like nutritional value, taste, smell, shape, size etc.  orientation to crop rotation is the pattern of crops on the same farm.  training facility is the availability of getting training from different sources like ngos, government training centers, agriculture specialists etc. 6.3 social perspective  direct local employment is the opportunity that is made available due to the particular farming system on the farm and outside the farm.  indirect local employment is the employment created indirectly like the rural tourism, organic centers, local farm input industries etc.  the level of education in the family is an important parameter to compare the different farming practices, and its relevance with the cultural aspects.  social justice in a rural area is about integrity in the community because of a changed style of farming.  recreational use is related to the concept of tourism for attracting the city population to get the feel of the rural environment. it is also important in terms of spreading traditional knowledge and history.  wasteland includes regions which do not produce the regular crop due to the bad soil health. there are holistic methods to grow crops in such areas.  compatibility with local socio-cultural values includes whether a farmer faces difficulties from other farmers after changing their farming practice.  migration opportunity is the opportunity the farmers in a given region consider when deciding whether to go to city to obtain a higher income. 6.4 environmental perspective  water requirement is the periodic frequency of watering the crops.  water ph recommended for irrigation water is from 6.5 to 8.4. this technological information is verified by the ngos and the public health centre.  soil ph recommended for different crop is from 5.5.to 8.  organic carbon is a major criteria considered by farmers to evaluate soil fertility.  soil fertility was related to the agronomic quality of soil, which depends on soil structure and low levels of pollution/contamination.  biodiversity which can be measured as the number and variety of different living beings present, including diversity of the sugarcane and rice and mango crop, wildlife, micro fauna, beneficial fauna, domestic animals and wild flora.  soil erosion is measured based on the slope of field and maximum rainfall. table 2 provides details about the farmers who were surveyed giving average farm size, number of farmers surveyed, irrigation method, marketing method and type of seeds used. it is observed that a large number of organic farmers are using local variety seeds and use direct to customer marketing method. tables 3, 5, 6 contain the data obtained from the various surveys. this quantitative data is for three different crops with three different farm practices. table 4 gives data about the nature of the attributes. the nature of the attribute is judged as a cost or a benefit. an attribute is considered a cost if it increases with time causing a negative effect. it is considered a benefit if it has a positive effect. for example, if the water requirement for any crop increases over time, it increases the cost of pumping for the same crop. however, if it decreases over a period of time it is beneficial. in chemical sugarcane farming the ijahp article: rajguru, shah, ramkrishnan/ change dynamics in maharashtra state small farming systemsfield survey and analysis through analytic hierarchy process international journal of the analytic hierarchy process 235 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.266 water retaining capacity of soil decreases, and therefore more water is required. in the case of organic sugarcane farming, the water requirement decreases as soil organic carbon content increases. table 2 details from the farmers’ fields c r o p t y p e o f fa r m in g n o . o f fa r m e r s in te r v ie w e d a v e r a g e fa r m s iz e (a c r e ) ir r ig a ti o n m e th o d m a r k e ti n g m e th o d s e e d (h y b r id / lo c a l) rice organic farming 7 3.5 (4 well water + 1 rainfed) 4 direct customer + 3 middle man) (2 hybrid + 3local) chemical farming 7 3 (5 well water + 2 rainfed) 4 direct customer + 3 middle man) all hybrid partial farming 7 2.5 (4 well water + 3 rainfed) 4 direct customer + 3 middle man) all hybrid sugarcane organic farming 5 2 well and canal direct to sugar factory all hybrid chemical farming 8 3 well and canal direct to sugar factory all hybrid partial farming 8 3 well and canal direct to sugar factory all hybrid mango organic farming 5 1.5 (4 well water + 1 rainfed) 4 direct customer + 1 middle man) local chemical farming 8 1 (5 well water + 3 rainfed) 1 direct customer + 7 middle man) local partial farming 8 1.5 (6 well water + 2 rainfed) 2 direct customer + 6 middle man) local ijahp article: rajguru, shah, ramkrishnan/ change dynamics in maharashtra state small farming systemsfield survey and analysis through analytic hierarchy process international journal of the analytic hierarchy process 236 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.266 table 3 quantitative data for the sugarcane crop no. quantitative attributes organic farming chemical farming partial organic farming economic perspective a1 income (rs/acre) 63765 55789 78947 a2 yield (tons/acre) 42 42 52 a3 value of major livestock owned (rs) 78000 5000 30000 a4 total operations cost (rs/acre) 19120 30040 42105 a5 income from other source (rs) 60000 30000 40000 technological perspective a6 productivity 4.1 2.5 3.5 a7 percentage of successful harvest (%) 90 60 70 a8 no of crops/year 6 2 3 social perspective a9 direct local employment (%) 30 -10 24 a10 indirect local employment (%) 30 -12 20 a11 highest education in family (standard) 15 10 12 environmental perspective a12 water requirement (inches/year) 20 46.8 34 a13 water ph 7.5 8 7.5 a14 soil ph 6.5 7.4 7 a15 organic carbon 1.5 0.6 0.8 ijahp article: rajguru, shah, ramkrishnan/ change dynamics in maharashtra state small farming systemsfield survey and analysis through analytic hierarchy process international journal of the analytic hierarchy process 237 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.266 table 4 nature of quantitative attributes (cost/benefit) for all crops criteria cost/benefit a1 income benefit a2 yield benefit a3 value of major livestock owned cost a4 total operations cost cost a5 income from other source benefit a6 productivity benefit a7 harvest growing success prospects benefit a8 no of crops per year benefit a9 direct local employment benefit a10 indirect local employment benefit a11 highest education in family benefit a12 water requirement cost a13 water ph cost a14 soil ph cost a15 organic carbon benefit ijahp article: rajguru, shah, ramkrishnan/ change dynamics in maharashtra state small farming systemsfield survey and analysis through analytic hierarchy process international journal of the analytic hierarchy process 238 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.266 table 5 quantitative data for the mango crop no. quantitative attributes organic farming chemical farming partial organic farming economic perspective a1 income (rs/acre) 190000 150000 170000 a2 yield (kg/acre) 2500 2700 2600 a3 value of major livestock owned (rs) 150000 75000 100000 a4 total operations cost (rs) 35000 30000 33000 a5 income from other source (rs) 80000 50000 60000 technological perspective a6 productivity 9 7.8 8.1 a7 percentage of successful harvest (%) 90 60 70 a8 no of crops per year 6 2 3 social perspective a9 direct local employment (%) 30 -10 24 a10 indirect local employment (%) 30 -12 20 a11 highest education in family (standard) 15 10 12 environmental perspective a12 water requirement (inches / year) - - - a13 water ph 7.5 7 7.3 a14 soil ph 7.5 6.3 6.5 a15 organic carbon (%) 2.5 1.8 2 ijahp article: rajguru, shah, ramkrishnan/ change dynamics in maharashtra state small farming systemsfield survey and analysis through analytic hierarchy process international journal of the analytic hierarchy process 239 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.266 table 6 quantitative data for the rice crop no. quantitative attributes organic farming chemical farming partial organic farming economic perspective a1 income (rs/acre) 9150 -4200 2750 a2 yield (kg per acre) 560 600 750 a3 value of major livestock owned (rs) 88000 5000 50000 a4 total operations cost (rs/acre) 4860 10200 8500 a5 income from other source (rs) 50000 15000 30000 technological perspective a6 productivity 10.8 5 4.3 a7 harvest growing success prospects 90 60 70 a8 no of crops per year 6 2 3 sociocultural perspective a9 direct local employment (%) 30 -10 24 a10 indirect local employment (%) 30 -12 20 a11 highest education in family 15 10 12 environmental perspective a12 water requirement (inches / year) 19 31 23 a13 water ph 7 7.5 7.5 a14 soil ph 6.7 7.3 7 a15 organic carbon 1.5 0.8 1.2 7. stepwise calculation of technology ranking in ahp to understand how the final rankings were obtained, the stepwise calculation of ahp methodology will be explained. for each separate perspective the ahp is used throughout the hierarchy. for example, if the goal is to determine a sustainable farming practice for mango cultivation among organic, partial organic and chemical farming with an economic perspective, the attributes taken into consideration are as shown figure 2. these are quantitative attributes (a1, a2, a3, a4, a5) and qualitative attributes (a16, a17, a18). similarly, from the technological perspective, the attributes taken into consideration are a6, a7, a8, a19, a20, a11, a22. likewise, for the social and environmental perspectives, attributes are shown in figure 2. the same method is used for the rice and sugarcane crops when implementing the ahp method. as discussed in section 6, this work considered three crops and four perspectives for evaluating the performance the ahp method. therefore, is it implemented (3×4) 12 separate times to obtain the final rankings. a detailed procedure for ahp is discussed below for the sugarcane crop from the economic perspective. the same method was implemented for ranking all of the hierarchies for each crop in order to calculate the final ranking. table 7 extracts the ijahp article: rajguru, shah, ramkrishnan/ change dynamics in maharashtra state small farming systemsfield survey and analysis through analytic hierarchy process international journal of the analytic hierarchy process 240 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.266 specific quantitative attributes computed for sugarcane. ahp is implemented using java web based software developed by mahajan, ramkrishnan, and date (2008). table 7 quantitative attributes for sugarcane crop no. quantitative attributes organic farming (t1) chemical farming (t2) partial organic farming (t3) economic perspective a1 income (rs/acre) 63765 55789 78947 a2 yield (tons/acre) 42 42 52 a3 value of major livestock owned (rs) 78000 5000 30000 a4 total operations cost (rs/acre) 19120 30040 42105 a5 income from other source (rs) 60000 30000 40000 the above data was inputted into the software, and normalization of the attributes was done by using equations 1 and 2 in this software. for benefit attribute minmax min ii iij ij tt tt p    (1) for cost attribute minmax max ii iji ij tt tt p    (2) where ‘t’ is the actual value, ‘p’ is the normalized value, ‘i’ is the number of a row and ‘j’ is the number of a column. therefore, the normalized matrix of the quantitative data in table 7 is obtained from the software and shown in table 8. table 8 normalized quantitative matrix attribute t1 t2 t3 a1 0.34 0 1 a2 0 0 1 a3 0 1 0.66 a4 1 0.52 0 a5 1 0 0.33 next, priority vectors (weights) are calculated by pair wise comparison of attribute to attribute data as shown in table 9. here the eigenvector was calculated and the summation of all the eigenvectors (-0.2565, -0.1309, -0.1133, -0.2745, -0.1678, ijahp article: rajguru, shah, ramkrishnan/ change dynamics in maharashtra state small farming systemsfield survey and analysis through analytic hierarchy process international journal of the analytic hierarchy process 241 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.266 0.8411, -0.1314, -0.2757) is -2.1912. the value of this summation is used to divide each eigenvector to get the priority vector. the consistency of the pair wise comparison was checked by the consistency ratio (cr). the consistency ratio was calculated by equation 3 given below. the value of the consistency ratio should be less than 0.1. the consistency index (ci) is given by equation 4, and the ri is the random consistency index which depends upon the number of attributes in the matrix. consistency ratio = ri ci < 0.1 (3) ci (consistency index) = 1 n eeigen valu maximum n (4) where ‘n’ is number of row or column in the pair wise matrix. table 9 priority vector (weights) by pair wise comparison for economic perspective in sugarcane a1 a2 a3 a4 a5 a16 a17 a18 eigen vector priority vector a1 1.00 2.00 2.10 2.00 1.80 0.17 1.00 1.50 0.117 0.117 a2 0.50 1.00 2.00 0.25 1.30 0.25 1.00 0.20 0.060 0.060 a3 0.48 0.50 1.00 0.33 1.00 0.30 1.00 0.25 0.052 0.052 a4 0.50 4.00 3.00 1.00 2.00 0.20 1.00 2.00 0.125 0.125 a5 0.56 0.77 1.00 0.50 1.00 0.20 3.00 1.00 0.077 0.077 a16 6.00 4.00 3.33 5.00 5.00 1.00 7.00 4.00 0.384 0.384 a17 1.00 1.00 1.00 1.00 0.33 0.14 1.00 0.33 0.060 0.060 a18 0.67 5.00 4.00 0.50 1.00 0.25 3.00 1.00 0.126 0.126 max eigen value = 8.9367 cr = 0.09 <0.1 acceptable qualitative attributes were converted into quantitative data using pair wise comparison for each of the farming practices, and its corresponding priority vector (weight) was calculated as explained above. the priority vector (weight) for the qualitative attribute is shown in table 10. ijahp article: rajguru, shah, ramkrishnan/ change dynamics in maharashtra state small farming systemsfield survey and analysis through analytic hierarchy process international journal of the analytic hierarchy process 242 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.266 table 10 pair wise comparison of qualitative attributes (a) a16, (b) a17, (c) a18 for economic perspective with sugarcane (a) stability of income over time (a16) t1 t2 t3 eigen vector priority vector t1 1 4 3 0.9214 0.630103 t2 0.25 1 2 0.3194 0.218423 t3 0.33 0.5 1 0.2215 0.151473 cr =0.09 acceptable (b) independence to external agricultural inputs (a17) t1 t2 t3 eigen vector priority vector t1 1 7 3.2 0.9201 0.657825 t2 0.14 1 0.2 0.0998 0.071351 t3 0.31 5 1 0.3788 0.270822 cr =0.06 acceptable (c) trade and sale opportunities (a18) t1 t2 t3 eigen vector priority vector t1 1 8 5 0.9759 0.7597 t2 0.12 1 1.2 0.1516 0.118 t3 0.2 0.83 1 0.157 0.1222 cr =0.08 acceptable in the last step, data obtained from tables 8, 9 and 10 were used to calculate the ranking of technology as shown in table 11. the software uses the following equation 5 to calculate the final ranking.  n ij iijj wr  (5) here ρij is the normalized value taken from tables 8 and 10, while wi is the weight calculated from table 9. rj is the final rank value, so the maximum value in the row will be ranked first as shown in table 11. final rankings of sugarcane farming from technological, environmental and social perspectives are given in tables 12, 13 and 14. the same methodology was followed for the calculation of the final ranking of mango and rice cropping systems, and the results obtained are presented in figure 3. ijahp article: rajguru, shah, ramkrishnan/ change dynamics in maharashtra state small farming systemsfield survey and analysis through analytic hierarchy process international journal of the analytic hierarchy process 243 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.266 table 11 ranking of all the three farming practices for sugarcane farming with the economic perspective wi t1 (ρij) t2 (ρij) t3(ρij) a1 0.12 0.34 0.00 1.00 a2 0.06 0.00 0.00 1.00 a3 0.05 0.00 1.00 0.66 a4 0.13 1.00 0.52 0.00 a5 0.08 1.00 0.00 0.33 a16 0.38 0.63 0.22 0.15 a17 0.06 0.66 0.07 0.27 a18 0.13 0.76 0.12 0.12 rj 0.62 0.22 0.33 rank 1 3 2 table 12 ranking of all the three farming practices for sugarcane farming with the technological perspective wi t1 (ρij) t2 (ρij) t3(ρij) a6 0.45 1.00 0.00 0.63 a7 0.16 1.00 0.00 0.33 a8 0.07 1.00 0.00 0.25 a19 0.07 0.68 0.10 0.22 a20 0.07 0.73 0.13 0.14 a21 0.10 0.72 0.10 0.19 a22 0.09 0.72 0.10 0.19 rj 0.91 0.03 0.41 rank 1 3 2 ijahp article: rajguru, shah, ramkrishnan/ change dynamics in maharashtra state small farming systemsfield survey and analysis through analytic hierarchy process international journal of the analytic hierarchy process 244 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.266 table 13 ranking of all the three farming practices for sugarcane farming with the social perspective wi t1 (ρij) t2 (ρij) t3(ρij) a9 0.33 1.00 0.00 0.85 a10 0.10 1.00 0.00 0.76 a11 0.14 1.00 0.00 0.40 a23 0.09 0.70 0.11 0.19 a24 0.10 0.70 0.11 0.19 a25 0.09 0.27 0.50 0.24 a26 0.08 0.19 0.56 0.25 a27 0.08 0.76 0.10 0.14 rj 0.80 0.12 0.50 rank 1 3 2 table 14 ranking of all the three farming practices for sugarcane farming with the environmental perspective wi t1 (ρij) t2 (ρij) t3(ρij) a12 0.19 1.00 0.00 0.48 a13 0.16 1.00 0.00 1.00 a14 0.15 0.00 1.00 0.55 a15 0.11 1.00 0.00 0.22 a28 0.12 0.75 0.12 0.13 a29 0.12 0.75 0.12 0.13 a30 0.16 0.75 0.12 0.13 rj 0.75 0.20 0.41 rank 1 3 2 ijahp article: rajguru, shah, ramkrishnan/ change dynamics in maharashtra state small farming systemsfield survey and analysis through analytic hierarchy process international journal of the analytic hierarchy process 245 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.266 (a) (b) (c) (d) figure 3. technology ranking for all the crops (a) economical perspective, (b) technological perspective, (c) social perspective and (d) environmental perspective 8. discussion marketing forces, government subsidies and many other available alternatives complicates the decision making process for farmers in india. farmers are faced with a difficult decision when selecting an appropriate resource and management practice for farming. this study illustrates how the ahp can be used to demonstrate to farmers how to reach the most appropriate solution for allocation in a particular activity. the importance of ahp as a decision support tool has been demonstrated. ahp will be helpful in policy making for various issues at the village level because of its simplicity, effectiveness and ability to deal with qualitative as well as quantitative criteria. even though the ahp is a data intensive model, its approach is bottom up which is necessary for sustainable participatory development. it was observed that data collected from a survey like the one in this study is likely to reflect large inconsistencies. the ahp analysis demonstrated its capability of handling decisionmaking situations with some uncertainties and inconsistencies. in this study, ahp was applied to structure a multi-criteria prioritization (ranking) problem with the overall objective of selecting sustainable farming practices in a region with farmers as stakeholders. through the ahp structure, participants evaluated and ranked (prioritized) three alternatives via 30 criteria. the alternatives (influencing factors) were organic, chemical and partial organic farming practices, while the major criteria 0 0.2 0.4 0.6 0.8 1 sugarcane rice mango 0 0.2 0.4 0.6 0.8 1 sugarcane rice mango 0 0.2 0.4 0.6 0.8 1 sugarcane rice mango organic farming chemical farming partial organic farming 0 0.2 0.4 0.6 0.8 1 sugarcane rice mango ijahp article: rajguru, shah, ramkrishnan/ change dynamics in maharashtra state small farming systemsfield survey and analysis through analytic hierarchy process international journal of the analytic hierarchy process 246 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.266 were classified as qualitative and quantitative attributes. both the qualitative and quantitative attributes were considered under different perspectives. the qualitative attributes included soil health, local employment, social justice, etc. the quantitative attributes considered were water requirement, soil ph, productivity etc. the options were ranked from four different points of views (called perspectives), namely economic, technological, social and environmental. the ranking of alternatives as shown in figure 3 indicated that from the economic, technological, social and environmental perspectives, organic farming is the most suitable option for sugarcane. in rice farming, it was observed that from the economic, environmental and technological perspectives organic farming is best option, while from a social perspective partial organic was mostly preferred. in the case of the mango crop from all the perspectives, organic farming is most favorable. in all the crops, the economic perspective was most important followed by the environmental perspective. we infer that the alternatives used in farming practice should give decent returns and must be environmentally friendly. it may be difficult and impractical to shift all the farms to organic for reasons of inertia as well as anxiety expressed by farmers, etc. it was however evident from the study that such a goal may be progressively attained. prior experience (bhattacharyya & chakraborty, 2005) suggests that with clear scientific evidence, strong political will and institutional support it is possible to achieve the ‘fully organic’ goal towards sustainable agriculture to meet the long term food and nutritional requirements in india. it was clearly evident that appropriate resource allocation in agriculture is only possible through a bottom up and participatory approach. this study also made it clear that in order for the small farming system to sustain farmers, they must resort to additional sources of income than the revenue obtained from the principal crop. the results also show that partial organic and organic farming are favorable farming practices, and the policy makers should take note. ijahp article: rajguru, shah, ramkrishnan/ change dynamics in maharashtra state small farming systemsfield survey and analysis through analytic hierarchy process international journal of the analytic hierarchy process 247 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.266 references alphonce, c. b. (1997). application of the analytic hierarchy process in agriculture in developing countries, agricultural systems, 53, 97-112. amr, m. m. (1999). pesticide monitoring and its health problems in egypt, a third world country, toxicology letters, 107, 1–13. bhattacharyya, p., & chakraborty, g. (2005). current status of organic farming in india and other countries, indian journal of fertilizers, l (19), 111-123. carbonaro, m., mattera, m., nicoli, s., bergamo, p., cappelloni, m. (2002). modulation of antioxidant compounds in organic vs conventional fruit peach, prunus persica l., and pear, pyrus communis l., journal of agricultural food chemicals, 50(19), 5458-5462. chaudhury, j., mandal, u. k., sharma, k.l., ghosh, h., mandal, b. (2005). assessing soil quality under long-term rice-based cropping system, communications in soil science and plant analysis, 36, 1141–1161. dich, j., zahm, s. h., hanberg, a., adami, h.o. (1997). pesticides and cancer, cancer causes control, 8(3), 420-443. garry, v. f., schreinemachers, harkins m.e. (1996). pesticide appliers, biocides and birth defects in rural minnesota, environmental health perspectives, 1044, 394-399. kalra, r. l., singh, b., battu, r. s. (1994). organochlorine pesticide residues in human milk in punjab, india, environmental pollution, 85, 147-151. kalra, r. l., kaur, h., sharma, s., et al. (1999). ddt and hch residues in dairy milk samples collected from different geographical regions of india: a multicentre study, food additives and contaminants, 16(10), 411-417. karami, e. (2006). appropriateness of farmer’s adoption of irrigation methods: the application of the ahp model, agricultural systems, 87, 101–119. kiyotada, h., (2000). multicriteria analysis for agricultural resource management: a critical survey and future perspectives, european journal of operational research, 122, 486-500 mahajan, s., ramkrishnan, g., date, a. w. (2008). user friendly package for ranking of technology alternatives using analytical hierarchy process, btp report iit bombay. mathur, h. b., agarwal, h. c., johnson, s., saikia, n. (2005). analysis of pesticide residues in blood samples from villages of punjab, centre for science and environment new delhi, http://www.cseindia.org/userfiles/punjab_blood_report.pdf , accessed nov. 10. 2011. mawapanga, m. n., & debertin, d. l. (1996). choosing between alternative farming systems: an application of the analytic hierarchy process, review of agricultural economics, 8, 385-401. ijahp article: rajguru, shah, ramkrishnan/ change dynamics in maharashtra state small farming systemsfield survey and analysis through analytic hierarchy process international journal of the analytic hierarchy process 248 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.266 menon ramesh, (2005). poison in their veins, http://indiatogether.org/bloodcide agriculture, accessed nov. 12. 2013. parra-lopez, c., calatrava-requena, j., de-haro-gimenez, t. (2008). a systemic comparative assessment of the multifunctional performance of alternative olive systems in spain within an ahp-extended framework, ecological economics, 64, 820-834. ramchndra, t. v., & nagarathna, a. v. (2001). energetics in paddy cultivation in uttara kannada district, journal of energy conversion and management, 42, 131155. rao, c. h. s., venkateswarlu, v., surender, t., eddleston, m., buckley, n. a. (2005). pesticide poisoning in south india: opportunities for prevention and improved medical management, tropical medicine and international health, 10, 581–588. reddy, b. s. (2010). organic farming: status, issues and prospects a review, agricultural economics research review, 23, 343-358. ren, h., bao, h., endo, h., hayashi t. (2001). antioxidative and antimicrobial activities and flavonoid contents of organically cultivated vegetables. nippon shokuhin kagaku kogaku kaishi, 48(4), 246-252. rezaei-moghaddam, k., & karami, e. (2007). a multiple criteria evaluation of sustainable agricultural development models using ahp, environment, development and sustainability, 10(4), 407-426. saaty, t. l. (1994). how to make a decision: the analytic hierarchy process, interfaces, 24, 19-43. schneider, u.a., havlik, p., schmid, e., et al. (2011). impacts of population growth, economic development, and technical change on global food production and consumption, agricultural systems, 104, 204–215. woese, x., lange, d., boess, c., bogl, k.w. (1997). a comparison of organically and conventionally grown foods: results of a review of the relevant literature, journal of the science of food and agriculture, 74, 281-293. world health organization, who global report, preventing chronic diseases a vital investment, http://www.who.int/chp/chronic_disease_report/full_report.pdf, accessed jan. 10. 2012. worthington,w. (2001). nutritional quality of organic versus conventional fruits, vegetables, and grains, the journal of alternative and complementary medicine, 72, 161– 173. zahm, s. h., & blair, a. (1992). pesticides and non-hodgkins lymphoma, cancer research, 52(19), 5485-5488. http://indiatogether.org/bloodcide-%20agriculture http://indiatogether.org/bloodcide-%20agriculture ijahp article: marcarelli, rossi, ferrarro, lucadamo/an investment choice problem and calendar anomalies: a group ahp model for investors international journal of the analytic hierarchy process 176 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.726 an investment choice problem and calendar anomalies: a group ahp model for investors gabriella marcarelli university of sannio gabriella.marcarelli@gmail.com matteo rossi university of sannio matteo.rossi@unisannio.it antonella ferraro antonellaferraro88@live.it antonio lucadamo antonio.lucadamo@unisannio.it abstract an investment choice can be influenced by numerous qualitative and quantitative factors that often conflict with one other. therefore, portfolio management choice is a multicriteria decision problem that requires flexible and analytic decision tools for investors. for this task, the analytic hierarchy process (ahp) is suitable. we propose an ahp group-based model to analyze an investment choice problem looking at two financial markets including spain and france. the evaluation criteria that we used in our model are the return of the stock market, performance of government bonds and calendar effects in the financial markets. the 2017 french and spanish equity market returns and the government bond performances for each country are available in public databases. mean tests were performed in order to analyze calendar anomalies for both of the markets from 2007-2017. the aim of our study is to propose a model that allows simultaneous evaluation of the impact of the previously mentioned factors on investment choice. our analysis involves 69 students from the department demm of the university of sannio (italy) who have worked on financial market simulators. the data were obtained using questionnaires. the common priority vector procedure (cpvp) was used to determine the individual priorities (derived by individual judgments matrices) and aggregate the individual priorities (derived by individual judgments matrices) to obtain the group preferences. the results show that the decision makers prefer to invest in diversified portfolios. keywords: ahp; calendar anomalies; government bonds; stock market; investment choice; consistency mailto:gabriella.marcarelli@gmail.com mailto:matteo.rossi@unisannio.it mailto:antonellaferraro88@live.it mailto:antonio.lucadamo@unisannio.it ijahp article: marcarelli, rossi, ferrarro, lucadamo/an investment choice problem and calendar anomalies: a group ahp model for investors international journal of the analytic hierarchy process 177 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.726 1. introduction in recent years, the financial markets have changed because of globalization. today, a wide range of investment opportunities is available to investors. investors can select different products (stocks, bonds, currencies, options) in the diverse financial markets. in this way, the financial markets are related but each one of them has specific characteristics with particular opportunities for investors. financial decision makers differ in their aims and restrictions, which makes portfolio management choice more complex and dynamic (khaksari et al., 1989). in addition, a financial problem could be constructed because it is classified as a strategic choice (zopounidis, 1999). the selection of an investment depends on several qualitative and quantitative criteria that can be conflicting. therefore, an investment decision is a multi-criteria problem that requires some flexible and analytic tools for the financial agents. the analytic hierarchy process (ahp) can be applied in this type of problem (saaty, 1980; ülengin & ülengin, 1994). in fact, the ahp method can assist investors throughout the entire process, from the formulation of the problem to the evaluation and all of the choices in between (zopounidis, 1999). based on this, we analyzed financial decisions by applying the ahp to investment choice problems. we prefer the ahp model to other multicriteria methods (such as anp) because the analysis is simplified, assuming that criteria are independent (marcarelli, 2018). to define the ranking of investment choices, our model considers the following evaluation criteria: return of the stock market, performance of government bonds, and the presence of calendar effects (ce). roy, kolte, sangvikar, and pawar (2019) introduced two models to study the stock market volatility. jang and park (2019) suggested the return of the stock market and the performance of government bonds as the main criteria to consider. they affirmed that the previous research has shown that the bond and stock markets influence investments by decision makers (christiano et al., 1996; kontonikas & zekaite, 2018). in addition to these two criteria that are connected to market efficiency, we propose considering a factor that detects the irrationality of financial agents, that is, the presence of calendar anomalies. the aim of our study is to propose a model that allows simultaneous evaluation of the impact of the previously mentioned mixed factors for the investment choice. ce refer to the cycling irregularity in a financial market (latif et al., 2011), and in order to analyze these anomalies we perform some statistical tests. we constructed a hierarchical model that is composed of the following three levels. on the top is the selection of the investment choice, the intermediate level is represented by the return of the stock market, the performance of government bonds, and the presence of calendar effects, and on the bottom level, there are four financial investments, two of which are diversified in two different markets and two are entirely in one market. we considered these four choices in order to avoid the neutral option (50% 50%). the neutral option could lead investors to choose a response that did not reflect their true behavior (krosnick, 1991). the structure of the paper is as follows: section 2 offers literature reviews on the ahp method and calendar anomalies; section 3 presents some statistical tests used to check for the presence of calendar anomalies and the ahp method; section 4 discusses the results of our procedure; finally, section 5 provides some conclusions and future developments. ijahp article: marcarelli, rossi, ferrarro, lucadamo/an investment choice problem and calendar anomalies: a group ahp model for investors international journal of the analytic hierarchy process 178 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.726 2. literature review saaty introduced the ahp as a multicriteria method in the 1970s. this method represents a problem by using a hierarchical structure and derives relative and global weights for the hierarchical elements that are based on expert judgements (saaty, 1980). the ahp analyzes complex decision problems with both qualitative and quantitative elements and provides priorities using pairwise comparison matrices. the ahp has been applied in several fields, such as portfolio selection, resource allocation, environmental impact evaluation, risk and performance management (naji, mousrij, cillo & chierici, 2019; mital, del giudice & papa, 2018). saaty and vargas (1982) illustrated some applications in business, energy, health and transportation. in finance, ahp has been used for portfolio comparisons (martel et al., 1988), the evaluation of the exchange rate (ülengin & ülengin, 1994), and financial decision problems (zopounidis, 1999). spronk et al. (2005) analyzed the contributions of multicriteria decision methods in finance, and zouponidis et al. (2015) published a bibliographic survey of multicriteria analysis contributions in financial decision making. furthermore, the ahp method has been used to select target markets and distribution channels, and direct resource allocation among portfolio elements (saaty & vargas, 1982). calendar effects are among the evaluation criteria used to reach the goal of the study, and represent an important topic in the financial field. calendar effects describe when a change in stock prices is influenced by specific periods of the calendar year. various studies have documented unexpected and abnormal regularities in relation to certain moments of a day, days of the week, periods of a month or months of the year (wachtel, 1942; rozeff & kinney, 1976; french, 1980; barone, 1990; agrawal & tandon, 1994). essentially, the academic world and professional operators have, in various ways, analyzed stock returns by looking for a link between price changes and what times these changes occur. therefore, this study explores the principal calendar effects. weekend effect the weekend calendar effect describes how stock prices tend to fall on mondays. the first two studies that verify the speed of the generative process of stock prices are fama (1965) and granger and morgenstern (1970). these studies show that when the market is closed, the stochastic process followed by the share price (random walk) continues to operate, but at a lower speed. this means that the closing price on a monday is less than the closing price on the previous friday (latif et al., 2011). these results have been substantiated by other studies (french, 1980; gibbons & hess, 1981; jaffe & westerfield, 1985; schwert, 2003; chen & singal, 2003; miller et al., 2006). january effect “as goes january, so goes the year” is a famous law in the stock market. this is also called the “turn of the year” effect. in other words, there are abnormal returns in january in most countries (gultekin & gultekin, 1983). the first evidence of abnormal stock returns in january for the u.s. stock markets was observed by wachtel (1942). this effect was confirmed by many other scholars (rozeff & kinney 1976; barone, 1990; wong et al., 2006; rossi & fattoruso, 2017). ijahp article: marcarelli, rossi, ferrarro, lucadamo/an investment choice problem and calendar anomalies: a group ahp model for investors international journal of the analytic hierarchy process 179 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.726 holiday effect the holiday effect shows a significant return on days before public holidays (pettengill, 1989; ariel, 1990). this effect influences the performance of daily share returns. ariel (1990) verifies a significant growth on christmas eve and may day eve when compared with other holidays. this abnormality is present in different markets. different observations (lakonishok & smidt, 1988; barone, 1990; kim & park, 1994; meneu & pardo, 2004; cao et al., 2009) confirm the presence of abnormal post-holiday returns. turn of the month (tom) anomaly in 1987, ariel first identified the tom effect for the u.s. stock market. he discovered that mean returns are higher at the end of a month and at the beginning of the next month. this result is confirmed by many scholars (pettengill & jordan, 1988; agrawal & tandon, 1994). some recent research has confirmed that this effect is also present in other stock markets (hensel & ziemba, 1996; mcconnell & xu, 2008). 3. methodology 3.1 analysis of calendar effects this paper proposes an application of the group-ahp model to an investment choice problem regarding the french and spanish financial markets. for this reason, we considered government bond performance and the stock market return as evaluation criteria. these data were obtained from the sella sgr 1 report (2018) and yahoo finance. the third criterion was the presence of calendar anomalies (ca) for both markets. to analyze ca, we introduced some statistical tests to evaluate if the difference between the number of changes between 2007-2017 were significant. first, we analyzed the calendar effects described above. we computed the averages of the rates of change for both indices and used some figures to evaluate the behavior as the first descriptive analysis of each calendar effect. then, the following statistical test on the proportion of differences was performed to verify if there was a significant effect on the markets: 𝐻0: 𝑞1 − 𝑞2 = 0 𝐻1: 𝑞1 − 𝑞2 > 0 or 𝐻1: 𝑞1 − 𝑞2 < 0 indicating with 𝑞1 the population ratio of positive (or negative) changes in the analyzed period (for example, during the weekend), whereas 𝑞2 is the proportion of the population in the other periods. to verify if the null hypothesis (𝐻0) can be rejected or not: 1 sella sgr is the asset management firm of sella group which operates since 1983. ijahp article: marcarelli, rossi, ferrarro, lucadamo/an investment choice problem and calendar anomalies: a group ahp model for investors international journal of the analytic hierarchy process 180 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.726 𝑍 = 𝑝1 − 𝑝2 √𝑝𝑐 (1 − 𝑝𝑐 ) ( 1 𝑛1 + 1 𝑛2 ) ~𝑁(0,1) where:  𝑝1 is the relative frequency in the sample obtained from the first population  𝑝2 is the relative frequency in the sample obtained from the second population  𝑝𝑐 = 𝑛1𝑝1+𝑛2𝑝2 𝑛1+𝑛2  𝑛1 and 𝑛2 are the sizes of the two samples if 𝑍 > 𝑧𝛼 (or 𝑍 < −𝑧𝛼) then 𝐻0 has to be rejected. 𝑧𝛼 is the quantile of the normal standardized distribution and 𝛼 is the selected significance level (piccolo, 2010). weekend effect (we) to verify the presence of the we, we considered the correlation between changes in stock values that occurred on monday and friday. we checked to see if the percentage of positive changes on monday was lower than the percentage of positive changes on the weekends. january effect (je) to test if the je was present in our data, we analyzed if the ratio of positive changes in stock values seen in january was greater than the percentage of positive changes in stock values in the other months of the year. holiday effect (he) to verify the existence of the he, we examined if the ratio of positive changes that occurred during the days before a holiday was greater than the percentage of positive changes on the other days of the year. turn-of-the-month effect (tome) to test for the presence of the tome, we verified if the ratio of the positive changes in the stock prices during the last day of the month and the first three days of the following month was greater than the ratio of the positive changes on the other days of the month. 3.2 group ahp to prioritize criteria and evaluate alternatives the ahp procedure involves the following steps (saaty, 1980; saaty & vargas, 1982; saaty, 1994): 1. decomposition of the problem into a hierarchy; 2. data collection using pairwise comparisons; 3. deriving relative priorities; 4. synthesizing relative priorities to obtain the global ranking. step 1 the problem was decomposed into three levels. on the bottom level are the alternatives; the intermediate level contains the criteria used to compare the investment choices; on the top level is the objective of the problem. ijahp article: marcarelli, rossi, ferrarro, lucadamo/an investment choice problem and calendar anomalies: a group ahp model for investors international journal of the analytic hierarchy process 181 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.726 step 2 this step involves the use of pairwise comparisons to establish the relative priorities, which represent the importance of the components of a level in relation to the components in the level immediately above. we assigned a judgment, aij, to couples of elements (xi,xj) in a level as they related to a given component in an upper level. this judgment is a value greater than 1 if the component xi is preferred to the component xj, whereas the opposite preference relationship is indicated by a value lower than 1, and the difference between the two components is indicated by 1. a positive reciprocal square matrix is obtained when comparing n components of a level. all of the values on the main diagonal are equal to 1. we used a nine-point scale proposed by saaty to assign the judgements (saaty & vargas, 1982). the determination of a priority vector is always accompanied by a check of logical consistency in the allocation of judgements. the matrix is perfectly consistent if the following condition exists: aij ∙ ajk = aik for each i, j, k = 1, 2, … . . , n (1) unfortunately, this matrix may not be consistent. this can happen because of inaccuracies, errors or simply a violation of transitivity and/or proportionality. a preference relation is transitive, assuming that xi is preferred to xj and xj is preferred to xk, then xi is preferred to xk. furthermore, the inconsistency may be caused by a violation of the proportionality between the elements even if the transitive property is satisfied. the preference relationship is proportional if aij = 4 and ajk = 2 , then aik = 8. consistency indices to check the inconsistency level in a set of pairwise judgements have been proposed in the literature (saaty, 1980; koczkodaj, 1993; salo-hamalainen, 1997; crawford & williams, 1985). each index represents the degree of inconsistency in the judgements expressed as a real number. to check the inconsistency of judgments, saaty suggested the consistency ratio (cr): cr = ci ri (2) where 𝐶𝐼 = (𝜆𝑚𝑎𝑥 − 𝑛) ∕ (𝑛 − 1) (3) is the consistency index and ri is the average value of ci derived from a sample of 50,000 randomly generated reciprocal matrices (saaty, 1980). the cr value increases as inconsistency increases. the matrix has a tolerable inconsistency if cr < 0.1. the consistency of judgements is important because it is strictly linked to the accuracy of the preferences. when the judgements are inconsistent, the priority estimates are not reliable because each prioritization method may provide a different priority vector (grzybowski, 2016). instead, if the matrix is fully consistent, then all of the prioritization methods should give the same result. the consistency indices and thresholds that are proposed in the literature may be useful to address cardinal consistency, but they do not take into account ordinal consistency or transitivity (siraj et al., 2015). to overcome this kind of a problem, amenta et al. (2018, ijahp article: marcarelli, rossi, ferrarro, lucadamo/an investment choice problem and calendar anomalies: a group ahp model for investors international journal of the analytic hierarchy process 182 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.726 2020) proposed some approximated transitivity thresholds for some consistency indices. these thresholds are useful because they allow judgement revision to be avoided if a qualitative ranking of preferences is the goal. if the consistency index ranges between the consistency and transitivity threshold values, then we may be sure of the accuracy of the preferences. step 3 the relative relevance of the components can be obtained as the eigenvector associated with the maximum eigenvalue of a: a ⋅ w = λmax ⋅ w for i, j = 1, 2, … , n (4) by using the eigenvector method (em) proposed by saaty. other methods can be used to estimate the priority vector (saaty, 1980, aguaron & moreno-jimenez, 2003; pelaez & lamata, 2003, gass & rapsak, 2004) and include the arithmetic mean method (amm), the row geometric mean method (rgmm), the logarithmic least squares (lls) method, the singular value decomposition (svd). step 4 in the final ahp step, the global priorities are derived, which express the relevance of the investment choices. the relative weights of the components are aggregated by the principle of hierarchical composition. the global weights provide the global ranking of the alternatives. once the final ranking is obtained, a sensitivity analysis is used to verify the stability of the resulting solution. 4. analysis of results 4.1 test to check for the presence of anomalies before applying the ahp method, the calendar anomalies in the two markets were analyzed. an empirical analysis was performed to verify the presence of the anomalies taking into account the price data from two different indices as follows: cac 40 (french stock market) and ibex 35 (spanish stock market). this research studied the ce for ten years between 2007-2017. the data were obtained from “yahoo finance”. the number of observations for the two indices are 2811 and 2807 from the cac and ibex 35, respectively. a significance level of 𝛼 =0.05 (𝑧𝛼 = 1.645) was used to determine the presence of ce in the two different markets. weekend effect to evaluate the weekend effect, the proportion of positive changes on monday and friday was considered. the data are described in the following tables and figures. ijahp article: marcarelli, rossi, ferrarro, lucadamo/an investment choice problem and calendar anomalies: a group ahp model for investors international journal of the analytic hierarchy process 183 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.726 table 2 weekend effect days cac 40 ibex 35 # of positive changes # of changes # of positive changes # of changes monday 264 556 235 555 friday 291 553 301 553 source: authors’ calculations figure 2 weekend effect for the cac 40 and ibex 35 indices source: authors’ calculations from figure 2, we note that the proportion of positive changes in both markets is higher on friday. therefore, it is necessary to verify if the differences are statistically significant using the following test: h0: q1 − q2 = 0 and h1: q1 − q2 < 0 considering the data in the previous tables, we have zcac 40 = −1.7117 and zibex 35 = −4.0258. in both markets, the difference between the positive changes on monday and friday is significant; therefore, there is a weekend effect. 0 0.1 0.2 0.3 0.4 0.5 0.6 monday friday proportion of positive changes ibex 35 cac 40 ijahp article: marcarelli, rossi, ferrarro, lucadamo/an investment choice problem and calendar anomalies: a group ahp model for investors international journal of the analytic hierarchy process 184 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.726 january effect table 3 january effect months cac 40 ibex 35 # of positive changes # of changes # of positive changes # of changes january 109 234 106 233 other months 1325 2576 1334 2573 source: authors’ calculations figure 3 january effect for the cac 40 and ibex 35 indices source: authors’ calculations it is clear from the data that in both france and spain the proportion of positive changes in january is smaller than in other months. for this reason, there is no january effect, as confirmed by the following test: h0: q1 − q2 = 0 and h1: q1 − q2 > 0 zcac 40 = −1.4225 and zibex 35 = −1.8577. it is not possible to reject h0. 0.42 0.44 0.46 0.48 0.5 0.52 january other months proportion of positive changes ibex 35 cac 40 ijahp article: marcarelli, rossi, ferrarro, lucadamo/an investment choice problem and calendar anomalies: a group ahp model for investors international journal of the analytic hierarchy process 185 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.726 holiday effect table 4 holiday effect cac 40 ibex 35 # of positive changes # of changes # of positive changes # of changes holiday 24 44 28 44 no holiday 1410 2766 1395 2762 source: authors’ calculations figure 4 holiday effect for the cac 40 and ibex 35 indices source: authors’ calculations the relative frequency of the positive changes seems to be higher for both indices in the days before a holiday when compared with other days. to evaluate if the differences are statistically significant, we consider the classical test: h0: q1 − q2 = 0 and h1: q1 − q2 > 0 zcac 40 = 0.4699 and zibex 35 = 1.7283 allows us to affirm that the he only happens in the spanish market. 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 holiday no holiday proportion of positive changes ibex 35 cac 40 ijahp article: marcarelli, rossi, ferrarro, lucadamo/an investment choice problem and calendar anomalies: a group ahp model for investors international journal of the analytic hierarchy process 186 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.726 turn-of-the-month effect table 5 turn-of-the-month effect days cac 40 ibex 35 # of positive changes # of changes # of positive changes # of changes days (–1, +3) 271 526 285 527 other days 1163 2284 1140 2279 source: authors’ calculations figure 5 tome for the cac 40 and ibex 35 indices source: authors’ calculations the number of positive changes is greater at the tome when compared with other days. zcac 40 = 0.2488 and zibex 35 = 1.6792 lead us to reject h0 for the spanish market, whereas in the french market, there are no significant differences. h0: q1 − q2 = 0 and h1: q1 − q2 > 0 table 6 highlights the existence of the ces. 0.47 0.48 0.49 0.5 0.51 0.52 0.53 0.54 0.55 days (–1, +3) other days proportion of positive changes ibex 35 cac 40 ijahp article: marcarelli, rossi, ferrarro, lucadamo/an investment choice problem and calendar anomalies: a group ahp model for investors international journal of the analytic hierarchy process 187 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.726 table 6 synthesis of effects france spain weekend effect sey sey january effect no no holiday effect no sey turn-of-the month effect no sey source: authors’ calculations in conclusion, our study has shown that the french stock market is only affected by the weekend effect, while in spain all three effects (weekend, holiday and turn-of-the-month) are present. 4.2 analysis of the ahp model results after the analysis of the ce, we structured the financial choice problem (fig. 9) and implemented the ahp using three levels that included the objective, the three criteria, and the four financial choices. figure 6 hierarchical structure of the problem source: authors’ calculations our analysis took place between april-june 2019, and included 69 students from the department demm of the university of sannio (italy) who have worked on financial market simulators. to help the experts express their judgements using pairwise comparisons, a simple questionnaire was constructed with guidelines for the completion (see appendix 1). many of the matrices have cr > 0.1. in-depth analysis of the these matrices shows that they are transitive because their cr is lower than the transitivity threshold (amenta et al., 2018, 2019). we know that transitivity ensures the reliability of investment choice problem presence of calendar anomalies stock market return 100% in spain 25% in france and 75% in spain 100% in france 75% in france and 25% in spain government bond performance criteria alternatives goal ijahp article: marcarelli, rossi, ferrarro, lucadamo/an investment choice problem and calendar anomalies: a group ahp model for investors international journal of the analytic hierarchy process 188 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.726 the priority vector (sjrai et al., 2015). if the consistency ratio ranges between the consistency and transitivity threshold values, then we can be confident of the accuracy of the preferences. we may consider the following matrix: ( 1 1/4 1/7 3 4 1 1/4 1/5 7 4 1 7 1/3 5 1/7 1 ) the cr is higher than the consistency threshold suggested by saaty (0.1), but lower than the transitivity threshold suggested by amenta et al. (2019), so we may consider the judgements acceptable. the ranking of preferences is the same when calculating the priority vector using different methods. table 7 ranking of the alternatives for a transitive matrix investment choice aggregation methods ranking amm gmm em france 0.124376 0. 100707 0.130 4 spain 0.154308 0. 117714 0.143 3 25% in france and 75% in spain 0.537953 0. 658619 0.554 1 75% in france and 25% in spain 0.183363 0. 122958 0.172 2 source: authors’ calculations some matrices have a consistency index that is greater than the transitivity threshold; in these cases, since it is not possible to revise the judgements (the experts are anonymous), the corresponding questionnaires were removed from our analysis. the analysis of cardinal and ordinal consistency eliminated 24 questionnaires. the aggregation of the relative weights provides a global priority vector that represents the classification of the investment choices. the ahp was applied for each decision maker, and in this way we obtained 45 preference vectors. to aggregate the overall priorities that were calculated for each of the experts, we considered the common priority vector procedure (cpvp) proposed by amenta et al. 2019 (see appendix 2). this procedure considers the majority group preference and diminishes the influence of extreme individual opinions when deriving the common vector. figure 10 exhibits the global order of the investment choices. ijahp article: marcarelli, rossi, ferrarro, lucadamo/an investment choice problem and calendar anomalies: a group ahp model for investors international journal of the analytic hierarchy process 189 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.726 figure 7 global ranking of the investment choices source: authors’ calculations figure 8 global ranking of criteria source: authors’ calculations the analysis of 45 students’ preferences shows that they prefer a diversified investment. in particular, the best choice is a diversified investment with three-quarters in france and a quarter in spain (preferred by approximately 40% of the investors); the worst choice is investing only in spain (approximately 19%). regarding the importance of the criteria, figure 11 shows that the presence of ce (0.27) is less important than the equity market return (0.39) and government bond performance (0.34). tables 8 and 9 show normalized priority vectors. 0 0.05 0.1 0.15 0.2 0.25 0.3 france spain 25% in france and 75% in spain 75% in france and 25% in spain france spain 25% in france and 75% in spain 75% in france and 25% in spain gmm 0.176938844 0.157945227 0.17863068 0.298792379 0 0.1 0.2 0.3 presence of calendar anomalies return on the stock market performance of the government bonds presence of calendar anomalies return on the stock market performance of the government bonds gmm 0.183622996 0.264109191 0.232702601 ijahp article: marcarelli, rossi, ferrarro, lucadamo/an investment choice problem and calendar anomalies: a group ahp model for investors international journal of the analytic hierarchy process 190 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.726 table 8 normalized priority vectors for alternatives investment choice normalized priority vector ranking total in france: 100% france 0,217822591 3 total in spain: 100% spain 0,194440281 4 diversified favoring spain: 25% in france and 75% in spain 0,219905345 2 diversified favoring france: 75% in france and 25% in spain 0,367831782 1 source: authors’ calculations table 9 normalized priority vectors for criteria criteria normalized priority vector ranking presence of calendar anomalies 0,26986127 3 stock market return 0,38814769 1 government bond performance 0,34199104 2 source: authors’ calculations 5. concluding remarks this paper applied the ahp multi-criteria method to investigate an investment choice problem in two european countries. the ahp model required an in-depth analysis of calendar anomalies. to check for the presence of anomalies, we performed some statistical tests. the analysis showed that the spanish stock market exhibits three effects, while the french market exhibits only one anomaly. our model shows that the investors chose a diversified portfolio. in particular, a “diversified favoring france” was preferred to a “diversified favoring spain” portfolio. the presence of calendar anomalies was the least used criterion as determined by the weights. the procedure has strengths and weaknesses. first, the ahp methodology allows the phenomenon to be broken down into a hierarchical structure. then, it assigns personal judgements using pairwise matrices. in addition, the ahp is adaptable and makes it possible to analyze changes in the ranking based on varying the weights of criteria. humans are involved in the decision-making process, and for this reason, it is important to consider both individual preferences and knowledge (zopounidis & doumpos, 2002). from this point of view, the model is obviously correct according to the calculations and the procedure, but could not always be entirely predictable and controllable, which affects its reliability and authenticity. our proposal has theoretical and practical implications. the model allows us to analyze the influence of rational and irrational criteria, and highlights the poor financial knowledge of investors. the model considers a combination of different elements that have never been considered together in the existing literature. the investor's poor ijahp article: marcarelli, rossi, ferrarro, lucadamo/an investment choice problem and calendar anomalies: a group ahp model for investors international journal of the analytic hierarchy process 191 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.726 financial knowledge is demonstrated by the fact that the calendar anomalies are considered the least important criterion, although it is the only criterion that detects the irrationality of the financial agents. furthermore, the choice of a diversified investment had the highest priority. therefore, we think that if we had considered a fifth alternative (the 50% 50% investment), this would have most likely been the top choice. this leads us to believe that there is a lack of financial literacy. financial literacy is more than just knowledge or information, but the ability to use the information and resources to achieve and maintain financial wellbeing (huston, 2010). according to west (2012), our study confirms that financial literacy activities should seek to educate investors about the stock market, government bonds, other financial products and individual behavior. in fact, psychological biases and the limitations of the decision maker can influence financial behavior and, as a result, the investment choice (figure 9). figure 9 impact of psychological factors and financial literacy on investment choice source: authors’ calculations in this paper, we presented preliminary results from our study. our aim was to extend the analysis in three different ways in the future: to involve a greater number of investors, to consider investors from different countries, to apply some other methods and compare the results, to study the impact of financial literacy on the investment choice. financial skills financial behaviour investment choice core competences threats opportunities psycological factors financial literacy ijahp article: marcarelli, rossi, ferrarro, lucadamo/an investment choice problem and calendar anomalies: a group ahp model for investors international journal of the analytic hierarchy process 192 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.726 references agrawal, a. and tandon, k. (1994). anomalies or illusions? evidence from stock markets in eighteen countries. journal of international money and finance, 13(1), 83– 106. doi: https://doi.org/10.1016/0261-5606(94)90026-4 aguaron, j., moreno-jimenez, j. (2003). the geometric consistency index: approximated threshold. european journal of operational research, 147, 137–145. doi: https://doi.org/10.1016/s0377-2217(02)00255-2 al-loughani, n.e., al-saad, k.m. and ali, m.m. (2005). the holiday effect and stock return in the kuwait stock exchange. global competitiveness, 13(1), 81–91. amenta, p., lucadamo, a., & marcarelli, g. (2020). on the transitivity and consistency approximated thresholds of some consistency indices for pairwise comparison matrices. information sciences, 507, 274-287. doi: https://doi.org/10.1016/j.ins.2019.08.042. amenta, p., ishizaka, a., lucadamo, a., marcarelli, g. and vyas, v. (2019). computing a common preference vector in a complex multi-actor and multi-group decision system in analytic hierarchy process context, annals of operations research (anor), 284, 33– 62. doi: https://doi.org/10.1007/s10479-019-03258-3 amenta, p., lucadamo, a. and marcarelli, g. (2018). approximate thresholds for salohamalainen index, ifac-papersonline, 11(11), 1655-1659. doi: https://doi.org/10.1016/j.ifacol.2018.08.219 ariel, r.a. (1987). a monthly effect in stock returns. journal of financial economics, 18(1), 161–174. doi: https://doi.org/10.1016/0304-405x(87)90066-3 ariel, r.a. (1990). high stock returns before holidays: existence and evidence on possible causes. the journal of finance, 45(5), 1611–1626. doi: https://doi.org/10.1111/j.1540-6261.1990.tb03731.x barone, e. (1990). the italian stock market: efficiency and calendar anomalies. journal of banking and finance, 14(2–3), 483–510. doi: https://doi.org/10.1016/03784266(90)90061-6 cao, x., premachandra, i. m., bhabra, g. s., & pin tang, y. (2009). firm size and the pre-holiday effect in new zealand. international research journal of finance and economics. chen, h. and singal, v. (2003). role of speculative short sales in price formation: the case of the weekend effect. the journal of finance, 58(2), 685–706. doi: https://doi.org/10.1111/1540-6261.00541 crawford, g., williams, c. (1985). a note on the analysis of subjective judgment matrices. journal of mathematical psychology, 29, 387–405. doi: https://doi.org/10.1016/0022-2496(85)90002-1 https://doi.org/10.1016/j.ins.2019.08.042 https://doi.org/10.1007/s10479-019-03258-3 https://www.sciencedirect.com/science/journal/24058963 ijahp article: marcarelli, rossi, ferrarro, lucadamo/an investment choice problem and calendar anomalies: a group ahp model for investors international journal of the analytic hierarchy process 193 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.726 christiano, l.j., eichenbaum, m. and evans, c.l. (1996). the effects of monetary policy shocks: evidence from the flow of funds. review of economics and statistics, 78, 16–34. doi: https://doi.org/10.2307/2109845 dodd, o. and gakhovich, a. (2011). the holiday effect in central and eastern european financial markets. investment management and financial innovations, 8(4), 29–35. expert choice software, expert choice, inc. (1994) 4922 ellsworth ave., pittsburgh 15213. fama, e.f. (1965). the behavior of stock-market prices. the journal of business, 38(1), 34–105. forman, e., & peniwati, k. (1998). aggregating individual judgements and priorities with the analytic hierarchy process. european journal of operational research, 108, 165–169. doi: https://doi.org/10.1016/s0377-2217(97)00244-0 french, k.r. (1980). stock returns and the weekend effect. journal of financial economics, 8(1), 55–69. gass, s., rapcsak, t. (2004). singular value decomposition in ahp. european journal of operational research, 154, 573–584. doi: https://doi.org/10.1016/s0377-2217(02)007555 gibbons, m. and hess, p. (1981). day of the week effects and asset returns. journal of business, 54, 579–596. doi: https://doi.org/10.1086/296147 granger, c.w.j., morgenstern, o. (1970) predictability of stock market prices. lexington, ma: heath lexington books. grzybowski, a. (2016). new result on inconsistency indices and their relationship with the quality of priority vector estimation. expert systems with applications, 43, 197–212. doi: https://doi.org/10.1016/j.eswa.2015.08.049 gultekin, m. n. & gultekin, n. b. (1983). stock market seasonality. international evidence. journal of financial economics, 12(4), 469-481. doi: https://doi.org/10.1016/0304-405x(83)90044-2 hensel, c. r. & ziemba, w. t. (1996). investment results from exploiting turn-of-themonth effects. (digest summary). journal of portfolio management, 22(3), 17-23. doi: https://doi.org/10.3905/jpm.1996.409556 hogan, k.m., oolson, g. and rahmlow, h. (2000). a model for the prediction of corporate bankruptcy using the analytic hierarchy process. multi-criteria applications, 10, 85-102. hurson, c. and zouponidis, c. (1995). on the use of multi-criteria decision aid methods to portfolio selection. journal of eruo-asian management, 1(2), 69-94. ijahp article: marcarelli, rossi, ferrarro, lucadamo/an investment choice problem and calendar anomalies: a group ahp model for investors international journal of the analytic hierarchy process 194 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.726 huston s.j. (2010). measuring f literacy. the journal of consumer affairs, 44(2), 296316. jaffe, j. & westerfield, r. (1985). the weekend effect in common stock returns: the international evidence. journal of finance, 40, 237-44. jang, j.y. and park, m.j. (2019). a study on global investors’ criteria for investment in the local currency bond markets using ahp methods: the case of the republic of korea. risk,s 7(4), 1-20. doi: https://doi.org/10.3390/risks7040101 khaksari, s., kamath r. & grieves, r. (1989). a new approach to determining optimum portfolio mix. the journal of portfolio management, 15(3), 43-49. doi: https://doi.org/10.3905/jpm.1989.409201 kim, c.w. and park, j. (1994). holiday effect and stock returns: further evidence. journal of financial and quantitative analysis, 29(1), 145–157. doi: https://doi.org/10.2307/2331196 koczkodaj, w. (1993). a new definition of consistency of pairwise comparisons. mathematical and computer modelling, 18, 79–84. doi: https://doi.org/10.1016/08957177(93)90059-8 kok, k.l. and wong, y.c. (2004). seasonal anomalies of stocks in asean equity markets. sunway academic journal, 1, 1–11. kontonikas, a. and zivile, z. (2018). monetary policy and stock valuation: structural var identification and size effects. quantitative finance, 18, 837–48. doi: https://doi.org/10.1080/14697688.2017.1414516 krosnick, j.a. (1991). response strategies for coping with the cognitive demands of attitude measures in surveys. applied cognitive psychology, 5, 213-236. doi: 10.1002/acp.2350050305. doi: https://doi.org/10.1002/acp.2350050305 lakonishok, j. and smidt, s. (1988). are seasonal anomalies real? a ninety-year perspective. review of financial studies, 1(4), 403–425. doi: https://doi.org/10.1093/rfs/1.4.403 latif, m., arshad, s., fatima, m., & farooq, s. (2011). market efficiency, market anomalies, causes, evidences, and some behavioral aspects of market anomalies. research journal of finance and accounting institute of management sciences, 2(9/10) bahauddin zakaria university, multan, pakistan. marcarelli, g. (2018) an integrated network model for performance management: a focus on healthcare organizations. international journal of managerial and financial accounting, 10(2), 163 – 180. doi: https://doi.org/10.1504/ijmfa.2018.10012812 marrett, g.k. and worthington, a.c. (2009). an empirical note on the holiday effect in the australian stock market. applied financial letters, 16, 1769–1772. doi: https://doi.org/10.1080/13504850701675474 ijahp article: marcarelli, rossi, ferrarro, lucadamo/an investment choice problem and calendar anomalies: a group ahp model for investors international journal of the analytic hierarchy process 195 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.726 martel, j.m, khoury, n.t. and bergeron, m. (1988). an application of a multicriteria approach to portfolio comparisons., journal of the operational research society, 39(7), 617-628. doi: https://doi.org/10.1057/jors.1988.107 mcconnell, j.j. and xu, w. (2008). equity returns at the turn of the month. financial analysts journal, 64(2), 49–64. doi: https://doi.org/10.2469/faj.v64.n2.11 naji, m.a., mousrij, a., cillo, v. and chierici, r. (2019). measuring the maintenance performance through fuzzy logic and analytical hierarchy process. international journal of managerial and financial accounting, 11(3/4), 290-319. doi: https://doi.org/10.1504/ijmfa.2019.10025665 meneu, v. and pardo, a. (2004). pre-holiday effect, large trades and small investor behaviour. journal of empirical finance, 11, 231–246. doi: https://doi.org/10.1016/j.jempfin.2003.01.002 miller, j.a., atkinson-palombo, c.m. and balling, r.c. (2006). quantifying the ozone ‘weekend effect’ at various locations in phoenix, arizona. atmospheric environment, 40(39), 7644–7658. doi: https://doi.org/10.1016/j.atmosenv.2006.05.023 mital, m., del giudice, m., and papa, a. (2018). comparing supply chain risks for multiple product categories with cognitive mapping and analytic hierarchy process. technological forecasting and social change, 131, 159-170. doi: https://doi.org/10.1016/j.techfore.2017.05.036 mylonakis, j. and tserkezos, d. (2008). the january effect results in the athens stock exchange (ase). global journal of finance and banking issues, 2(2), 44–55. pelaez, j., lamata, m. (2003). a new measure of consistency for positive reciprocal matrices. computer and mathematics with applications, 46, 1839– 1845. doi: https://doi.org/10.1016/s0898-1221(03)90240-9 pettengill, g.n. (1989). holiday closings and security returns. the journal of financial research, 12(1), 57–67. doi: https://doi.org/10.1111/j.1475-6803.1989.tb00101.x pettengill, g.n. and jordan, b.d. (1988). a comprehensive examination of volume effects and seasonality in daily security returns. journal of financial research, 11(1), 57–70. doi: https://doi.org/10.1111/j.1475-6803.1988.tb00066.x piccolo, d. (2010). statistica. il mulino. rossi, m. (2015). the efficient market hypothesis and calendar anomalies: a literature review. international journal of managerial and financial accounting, 7(3-4). rossi, m. and fattoruso, g. (2017). the emh and the market anomalies: an empirical analysis on italian stock market. international journal of managerial and financial accounting, 9(3), 222 – 241. doi: https://doi.org/10.1504/ijmfa.2017.086689 http://www.inderscience.com/filter.php?aid=86689 http://www.inderscience.com/filter.php?aid=86689 https://www.inderscience.com/jhome.php?jcode=ijmfa https://www.inderscience.com/jhome.php?jcode=ijmfa ijahp article: marcarelli, rossi, ferrarro, lucadamo/an investment choice problem and calendar anomalies: a group ahp model for investors international journal of the analytic hierarchy process 196 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.726 roy, j. k., kolte, a., sangvikar, b., & pawar, a. (2019). accessing the equity return volatility effect of east and south asian nations: the econometrics modelling method. international journal of recent technology and engineering, 8(3 special issue), 594– 603. doi: https://doi.org/10.35940/ijrte.c1120.1083s19. rozeff, m. s., & kinney, w. r. (1976). capital market seasonality: the case of stock returns. journal of financial economics, 3(4), 379-402. doi: https://doi.org/10.1016/0304-405x(76)90028-3 saaty, t.l. (1980). the analytic hierarchy process. new york: mcgraw-hill. saaty, t.l. (1994). how to make a decision: the analytic hierarchy process. interfaces, 24(6), 19–43. doi: https://doi.org/10.1287/inte.24.6.19 saaty, t.l. and vargas, l.g. (1982) the logic of priorities; applications in business, energy, health, and transportation. boston: kluwer-nijhoff, reprinted in paperback (1991), pittsburgh: rws publications. salo, a., hamalainen, r. (1997). on the measurement of preference in the analytic hierarchy process. journal of multi-criteria decision analysis 6, 309– 319. doi: https://doi.org/10.1002/(sici)1099-1360(199711)6:6%3c309::aid-mcda163%3e3.0.co;22 schwert, g.w. (2003). anomalies and market efficiency. handbook of the economics of finance, 1, 939–974. doi: https://doi.org/10.1016/s1574-0102(03)01024-0 siraj, s., mikhailov, l., keane, j. (2015). contribution of individual judgments toward inconsistency in pairwise comparisons. european journal of operational research, 242, 557–567. doi: https://doi.org/10.1016/j.ejor.2014.10.024 spronk, j., steuer, r.e. and zoupounidis, c. (2005). multicriteria decision analysis/aid in finance. in figuiera, j., greco, s. and ehrgott, m (eds), multiple criteria decision analysis: state of the art surveys. new york: springer sciences, 799-857. doi: https://doi.org/10.1007/0-387-23081-5_20 ülengin, f., & ülengin, b. (1994). forecasting foreign exchange rates: a comparative evaluation of ahp. international journal of management science, 22(5), 505-519. doi: https://doi.org/10.1016/0305-0483(94)90031-0 wachtel, s. b. (1942). certain observations on seasonal movement in stock prices. journal of business, 15(2), 184–193. west j. (2012). financial literacy education and behaviour unhinged: combating bias and poor product design. international journal of consumer studies, 36, 523–530. doi: https://doi.org/10.1111/j.1470-6431.2012.01118.x wong w. k., agarwal a. and wonf, n. t. (2006). the disappearing calendar anomalies in the singapore stock market. the lahore journal of economics, 11(2), 123-139. doi: https://doi.org/10.35536/lje.2006.v11.i2.a7 https://doi.org/10.35940/ijrte.c1120.1083s19 ijahp article: marcarelli, rossi, ferrarro, lucadamo/an investment choice problem and calendar anomalies: a group ahp model for investors international journal of the analytic hierarchy process 197 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.726 zopounidis, c. (1999). multicriteria decision aid in financial management. european journal of operational research, 119(2), 404-415. doi: https://doi.org/10.1016/s03772217(99)00142-3 zopounidis, c., & doumpos, m. (2002). multi-criteria decision aid in financial decision making: methodologies and literature review. journal of multi-criteria decision analysis, 11, 167–186. doi: https://doi.org/10.1002/mcda.333 zouponidis c., galariotis, e., doumpos, m., sarri, s., & andriosopoulos, k. (2015). multi criteria decision aiding for finance: an updated bibliographic survey. european journal of operational research, 247, 339-348. doi: https://doi.org/10.1016/j.ejor.2015.05.032 ijahp article: marcarelli, rossi, ferrarro, lucadamo/an investment choice problem and calendar anomalies: a group ahp model for investors international journal of the analytic hierarchy process 198 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.726 appendix 1 questionnaire for each pair of criteria, indicate which is better than the "investment choice" objective with a cross. (in case of indifference do not indicate any choice) indicate the degree of preference for the selected criterion on the left. (in the case of indifference, indicate 1.) the values from 2 to 9 indicate progressively increasing degrees of importance, from weak to absolute. criterion 1: “presence of calendar anomalies” 2 criterion 2: “stock market return in 2017” 1 2 3 4 5 6 7 8 9 criterion 1: “presence of calendar anomalies” criterion 3: “government bond performance in 2017” 1 2 3 4 5 6 7 8 9 criterion 2: “stock market return in 2017” criterion 3: “government bond performance in 2017” 1 2 3 4 5 6 7 8 9 2 “presence of calendar anomalies”: we speak of calendar anomalies when precise dates or periods of the calendar year affect the variation in share prices. four calendar effects were considered: 1. weekend effect: negative equity returns on monday and positive on friday; 2. holiday effect: higher equity returns in the days preceding the holidays; 3. turn of the month effect: higher bond yields on the last trading day of the month and the first three of the next; 4. january effect: higher yields in january compared to other months. ijahp article: marcarelli, rossi, ferrarro, lucadamo/an investment choice problem and calendar anomalies: a group ahp model for investors international journal of the analytic hierarchy process 199 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.726 for each pair of alternatives, indicate which is better than the criterion “presence of calendar anomalies” with a cross, considering that there were 1 out of 4 anomalies in france and 3 out of 4 anomalies in spain. (in the case of indifference, do not indicate any choice) indicate the degree of preference for the alternative selected on the left. (in the case of indifference. indicate 1.) the values from 2 to 9 indicate progressively increasing degrees of importance, from weak to absolute. alternative 1: “investment in france” alternative 2: “investment in spain” 1 2 3 4 5 6 7 8 9 alternative 1: “investment in france” alternative 3: “investment 25% in france and 75% in spain” 1 2 3 4 5 6 7 8 9 alternative 1: “investment in france” alternative 4: “investment 75% in france and 25% in spain” 1 2 3 4 5 6 7 8 9 alternative 2: “investment in spain” alternative 3: “investment 25% in france and 75% in spain” 1 2 3 4 5 6 7 8 9 alternative 2: “investment in spain” alternative 4: “investment 75% in france and 25% in spain” 1 2 3 4 5 6 7 8 9 alternative 3: “investment 25% in france and 75% in spain” alternative 4: “investment 75% in france and 25% in spain” 1 2 3 4 5 6 7 8 9 ijahp article: marcarelli, rossi, ferrarro, lucadamo/an investment choice problem and calendar anomalies: a group ahp model for investors international journal of the analytic hierarchy process 200 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.726 for each pair of alternatives, indicate which is better than the criterion "stock market return in 2017" with a cross, considering that the french cac 40 gained 9.3%, and the spanish ibex 35 gained 7.4%. (in the case of indifference, do not indicate any choice) indicate the degree of preference for the alternative selected on the left. (in the case of indifference, indicate 1.) the values from 2 to 9 indicate progressively increasing degrees of importance, from weak to absolute. alternative 1: “investment in france” alternative 2: “investment in spain” 1 2 3 4 5 6 7 8 9 alternative 1: “investment in france” alternative 3: “investment 25% in france and 75% in spain” 1 2 3 4 5 6 7 8 9 alternative 1: “investment in france” alternative 4: “investment 75% in france and 25% in spain” 1 2 3 4 5 6 7 8 9 alternative 2: “investment in spain” alternative 3: “investment 25% in france and 75% in spain” 1 2 3 4 5 6 7 8 9 alternative 2: “investment in spain” alternative 4: “investment 75% in france and 25% in spain” 1 2 3 4 5 6 7 8 9 alternative 3: “investment 25% in france and 75% in spain” alternative 4: “investment 75% in france and 25% in spain” 1 2 3 4 5 6 7 8 9 ijahp article: marcarelli, rossi, ferrarro, lucadamo/an investment choice problem and calendar anomalies: a group ahp model for investors international journal of the analytic hierarchy process 201 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.726 for each pair of alternatives, indicate which is better than the criterion “government bond performance in 2017” with a cross, considering that there has been a growth in the ten-year yields with levels of 0.78% for the french oat and 1.57% for spanish bonos. (in the case of indifference, do not indicate any choice.) indicate the degree of preference for the alternative selected on the left. (in the case of indifference, indicate 1.) the values from 2 to 9 indicate progressively increasing degrees of importance, from weak to absolute. alternative 1: “investment in france” alternative 2: “investment in spain” 1 2 3 4 5 6 7 8 9 alternative 1: “investment in france” alternative 3: “investment 25% in france and 75% in spain” 1 2 3 4 5 6 7 8 9 alternative 1: “investment in france” alternative 4: “investment 75% in france and 25% in spain” 1 2 3 4 5 6 7 8 9 alternative 2: “investment in spain” alternative 3: “investment 25% in france and 75% in spain” 1 2 3 4 5 6 7 8 9 alternative 2: “investment in spain” alternative 4: “investment 75% in france and 25% in spain” 1 2 3 4 5 6 7 8 9 alternative 3: “investment 25% in france and 75% in spain” alternative 4: “investment 75% in france and 25% in spain” 1 2 3 4 5 6 7 8 9 ijahp article: marcarelli, rossi, ferrarro, lucadamo/an investment choice problem and calendar anomalies: a group ahp model for investors international journal of the analytic hierarchy process 202 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.726 appendix ii common priority vector procedure the common priority vector procedure is an aggregation method recently introduced by amenta et al. (2019). the aim of this procedure is to aggregate the dms judgments with the hypothesis that they can have a common priority vector. some of them agree in a higher way with this common solution, whereas others have a smaller concordance with it. the goal is to find this common vector, indicated by q, which synthesizes the values of the alternatives as seen by the majority of dms. the solution is given by: wk = w̃k + ek = λ̃kqq t + ek where wk = λkqq t is a positive matrix of unitary rank providing the same information of judgment matrix xk; w̃k = λ̃kqq t; λ̃k is the salience of order k associated to common vector q, ek is the residual matrix, with k = 1, … , k (k is the number of dms) and q t is the transpose of q vector. the final common vector and the saliences can be found by minimizing the following loss function: l(q, λ̃k, αk) = ∑ αk k k=1 ‖wk − λ̃kqq t‖ f 2 = ∑ αk‖λkqkqk t − λ̃kqq t‖ f 2 k k=1 = ∑ αk‖wk − w̃k‖f 2 k k=1 the weights αk are generally fixed a priori according to previous information. the common vector and the saliences then minimize the distance between the individual priority vectors and the common vector q. the solution of the previous function can be obtained by implementing an iterative algorithm, based on an alternating least squares procedure. all of the salience λ̃k are initialized to fixed values and the other quantities are computed iteratively until a convergence criterion is reached. ijahp article: kahraman, öztayşi, onar, doğan / intuitionistic fuzzy originated interval type-2 fuzzy ahp: an application to damless hydroelectric power plants international journal of the analytic hierarchy process 266 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.538 intuitionistic fuzzy originated interval type-2 fuzzy ahp: an application to damless hydroelectric power plants cengiz kahraman istanbul technical university industrial engineering department i̇stanbul kahramanc@itu.edu.tr başar öztayşi istanbul technical university industrial engineering department i̇stanbul oztaysib@itu.edu.tr sezi çevik onar istanbul technical university industrial engineering department i̇stanbul cevikse@itu.edu.tr onur doğan istanbul technical university industrial engineering department i̇stanbul onurdoganmail@gmail.com abstract intuitionistic fuzzy sets (ifs) proposed by atanassov (1983, 1986) are a generalization of ordinary fuzzy sets. they incorporate the degree of hesitation which is defined as 1 minus the sum of membership and non-membership degrees. type-2 fuzzy sets were first introduced by zadeh (1975) as an extension of the concept of an ordinary fuzzy set. type-2 fuzzy sets have grades of membership that are themselves fuzzy. the membership function of a type-2 fuzzy set is three-dimensional, and it is the new third dimension that provides additional degrees of freedom for handling uncertainties. an intuitionistic fuzzy set can be converted to a type-2 fuzzy set by subtracting its nonmembership function from 1. thus, an intuitionistic fuzzy multi-criteria decision making problem can be solved by using type-2 fuzzy multi-criteria decision making techniques. in this paper, an intuitionistic fuzzy originated interval type-2 fuzzy ahp method is developed and applied to the technology selection problem of a damless hydroelectric power plant. damless hydroelectric power plants are environmentally friendly and sustainable energy production systems. several criteria and damless technology alternatives along the mailto:kahramanc@itu.edu.tr mailto:oztaysib@itu.edu.tr mailto:cevikse@itu.edu.tr mailto:onurdoganmail@gmail.com ijahp article: kahraman, öztayşi, onar, doğan / intuitionistic fuzzy originated interval type-2 fuzzy ahp: an application to damless hydroelectric power plants international journal of the analytic hierarchy process 267 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.538 sakarya river in turkey are considered. linguistic evaluations are considered in this multicriteria damless technology selection problem. keywords: intuitionistic fuzzy sets; type-2 fuzzy sets; ahp; multi-criteria decision making; damless hydroelectric power 1. introduction multi-criteria decision making approaches model the decision problem by considering all related characteristics, perspectives and views of the stakeholders. in the traditional multi-criteria decision making approaches (mcdm) the evaluations are done with exact numbers. fuzzy mcdm approaches are helpful tools in order to deal with the problems that involve uncertainty and imprecision. type-1 fuzzy sets introduced by zadeh (1965) where each element is defined with a membership degree in the interval [0, 1] are important for dealing with vagueness. yet, type-1 fuzzy sets have limitations in defining the uncertainties. therefore, there are several extensions of fuzzy sets that deal with the shortcomings. intuitionistic fuzzy sets introduced by atanassov (1983) enable defining both the membership and non-membership degrees. type-2 fuzzy sets as fuzzy sets define a three-dimensional fuzzy set where the new third dimension represents the degrees of freedom (zadeh, 1975). both of these two extensions have various advantages and transitivity among these sets is possible. energy consumption has been rising due to population growth and exponential technology improvements. carbon based energy resources such as petroleum or gas are not enough to fulfill the demand by the population on earth. carbon based energy resources release greenhouse gases and increase carbon emissions which create severe environmental and health problems. renewable energy can be the major solution to the increased energy need. renewable resources like solar, wind, geothermal, and hydro energy are clean and limitless. renewable energy investments are crucial for supplying the energy need. hydroelectric power is the leading renewable energy resource that accounts for approximately 16% of total electricity production and 85% of renewable electricity production (özcan et al., 2017). it is one of the most convenient energy resources that can balance the fluctuations between energy demand and supply. damless hydroelectric power plants are eco-friendly and sustainable energy systems. selecting the appropriate damless hydroelectric power is important for the success of renewable energy investments. yet, the criteria in damless hydroelectric power selection includes uncertainties and imprecision. classical ahp as a selection method uses a linguistic scale involving some degree of vagueness. in this linguistic scale, every linguistic term has a corresponding numerical value. however, an expert may want to use a continuous interval rather than an exact discrete number such as “between 2 and 3” or “larger than 7”. in this case, “between 2 and 3” and “larger than 7” can be represented by triangular fuzzy numbers. there are several ordinary fuzzy ahp methods developed by some researchers in the literature which include buckley (1985), laarhoven and pedrycz (1983), chang (1996), etc. emerging extensions of ordinary fuzzy sets caused new extensions of the fuzzy ahp method such as intuitionistic fuzzy ahp, hesitant fuzzy ahp, type-2 fuzzy ahp and pythagorean fuzzy ahp (otay et al., 2017; büyüközkan et al., 2017; zhu & xu, 2014; cevik onar et al., 2014; kahraman et al., 2014; ilbahar et al., 2018). ijahp article: kahraman, öztayşi, onar, doğan / intuitionistic fuzzy originated interval type-2 fuzzy ahp: an application to damless hydroelectric power plants international journal of the analytic hierarchy process 268 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.538 the aim of this study is to show the potential applicability of type-2 fuzzy conversion of intuitionistic fuzzy sets in multi-criteria selection problems. an intuitionistic fuzzy set can be converted to a type-2 fuzzy set by subtracting its non-membership function from 1. thus, an intuitionistic fuzzy multi-criteria decision making problem can be solved by using type-2 fuzzy multi-criteria decision making techniques. this is the advantage of our approach since type-2 fuzzy ahp methods have been already well-developed in the literature. the originality of the paper is the development of an intuitionistic fuzzy originated interval type-2 fuzzy ahp and its application to the technology selection problem of a damless hydroelectric power plant. the rest of the study is organized as follows: section 2 explains the preliminaries on intuitionistic and type-2 fuzzy sets. the proposed intuitionistic fuzzy originated interval type-2 fuzzy ahp is given in section 3. application of the proposed methodology and the sensitivity analysis for a damless hydroelectric power plant selection problem is given in section 4 and in the final section, conclusions are given. 2. intuitionistic and type-2 fuzzy sets 2.1 triangular intuitionistic fuzzy sets the intuitionistic fuzzy sets introduced by atanassov (1986, 1999) are expressed by a membership value and a non-membership value for any x in x so that their sum is at most 1. definition 1: let 𝑋 ≠ ∅ be a given set. an intuitionistic fuzzy set in x is an object a given by �̃� = {〈𝑥,𝜇�̃�(𝑥),𝑣�̃�(𝑥)〉;𝑥𝜖𝑋}, (1) where 𝜇�̃�:𝑋 → [0,1] and 𝑣�̃�:𝑋 → [0,1] satisfy the condition 0 ≤ 𝜇�̃�(𝑥)+ 𝑣�̃�(𝑥) ≤ 1, (2) for every 𝑥𝜖𝑋. hesitancy is equal to “1-(𝜇�̃�(𝑥)+ 𝑣�̃�(𝑥))” definition 2: a triangular intuitionistic fuzzy number (tifn) �̃� is an intuitionistic fuzzy subset in ℝ with the following membership function and non-membership function: 𝜇�̃�(𝑥) = { 𝑥−𝑎1 𝑎2−𝑎1 ,𝑓𝑜𝑟 𝑎1 ≤ 𝑥 ≤ 𝑎2 𝑎3−𝑥 𝑎3−𝑎2 ,𝑓𝑜𝑟 𝑎2 ≤ 𝑥 ≤ 𝑎3 0 , 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 (3) and 𝑣�̃�(𝑥) = { 𝑎2−𝑥 𝑎2−𝑎1 ′ ,𝑓𝑜𝑟 𝑎1 ′ ≤ 𝑥 ≤ 𝑎2 𝑥−𝑎2 𝑎3 ′−𝑎2 ,𝑓𝑜𝑟 𝑎2 ≤ 𝑥 ≤ 𝑎3 ′ 1 , 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 (4) ijahp article: kahraman, öztayşi, onar, doğan / intuitionistic fuzzy originated interval type-2 fuzzy ahp: an application to damless hydroelectric power plants international journal of the analytic hierarchy process 269 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.538 where 𝑎1 ′ ≤ 𝑎1 ≤ 𝑎2 ≤ 𝑎3 ≤ 𝑎3 ′ , 0 ≤ 𝜇�̃�(𝑥) + 𝑣�̃�(𝑥) ≤ 1 and tifn is denoted by �̃�𝑇𝐼𝐹𝑁 = (𝑎1,𝑎2,𝑎3;𝑎1 ′ ,𝑎2,𝑎3 ′ ) . definition 3: arithmetic operations for tifns are as follows: let �̃�𝑇𝑟𝐼𝐹𝑁 = (𝑎1,𝑎2,𝑎3;𝑎1 ′ ,𝑎2,𝑎3 ′ ) and �̃�𝑇𝑟𝐼𝐹𝑁 = (𝑏1,𝑏2,𝑏3;𝑏1 ′ ,𝑏2,𝑏3 ′) be two tifns. then, addition: �̃� = �̃� + �̃� is also a tifn: �̃� = (𝑎1 + 𝑏1,𝑎2 + 𝑏2,𝑎3 + 𝑏3;𝑎1 ′ + 𝑏1 ′ ,𝑎2 + 𝑏2, 𝑎3 ′ + 𝑏3 ′). (5) multiplication: �̃� = �̃� ⊗ �̃� is also a tifn: �̃� ≅ (𝑎1𝑏1,𝑎2𝑏2,𝑎3𝑏3;𝑎1 ′𝑏1 ′ ,𝑎2𝑏2,𝑎3 ′ 𝑏3 ′). (6) subtruction: �̃� = �̃� ⊖ �̃� is also a tifn: �̃� = (𝑎1 − 𝑏3,𝑎2 − 𝑏2,𝑎3 − 𝑏1;𝑎1 ′ − 𝑏3 ′ ,𝑎2 − 𝑏2,𝑎3 ′ − 𝑏1 ′). (7) division: �̃� = �̃� ⊘ �̃� is also a tifn: �̃� ≅ (𝑎1/𝑏3,𝑎2/𝑏2,𝑎3/𝑏1;𝑎1 ′/𝑏3 ′ ,𝑎2/𝑏2,𝑎3 ′ /𝑏1 ′). (8) definition 4: let 𝐼𝑖 = (𝑎𝑖 𝐿,𝑎𝑖 𝑀,𝑎𝑖 𝑈;𝑎𝑖 ′𝐿,𝑎𝑖 𝑀,𝑎𝑖 ′𝑈) is a tifn. then, the defuzzification is realized by using function defined in eq. (9). 𝑑𝑓 = 𝑎𝑖 𝐿+𝑎𝑖 𝑀+𝑎𝑖 𝑈 3 + 𝑎𝑖 ′𝐿+𝑎𝑖 𝑀+𝑎𝑖 ′𝑈 𝜏 (9) where τ is a very large number. aggregation operator for triangular intuitionistic fuzzy sets is defined as in definition 5. definition 5: let �̃�𝑗 = ((𝑎𝑗 𝐿,𝑎𝑗 𝑀,𝑎𝑗 𝑈),(𝑎𝑗 ′𝐿,𝑎𝑗 𝑀,𝑎𝑗 ′𝑈)), j=1, 2, …, n be a set of triangular fuzzy number intuitionistic fuzzy numbers (𝑇𝐹𝑁𝐼𝐹𝑁) where 𝑎𝑗 𝐿,𝑎𝑗 𝑀,𝑎𝑗 𝑈, 𝑎𝑗 ′𝐿,𝑎𝑗 𝑀,𝑎𝑗 ′𝑈 ≥ 1 . the aggregation operator in equation 10 for these numbers is proposed by us.                                            n j n j n j wu j wm j wl j n j n j wu j wm j n j wl j nm jjjjjj aaaaaa iiif 1 1 1 '' 1 11 21 ,,,,,, ,,,  (10) where  tnwwww ,,, 21  is the weight vector of     1,1,0,,,2,1 1    n j jjj wwnji  . ijahp article: kahraman, öztayşi, onar, doğan / intuitionistic fuzzy originated interval type-2 fuzzy ahp: an application to damless hydroelectric power plants international journal of the analytic hierarchy process 270 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.538 2.2 preliminaries: type-2 fuzzy sets in type-1 (ordinary) fuzzy sets each element has a degree of membership which is given by a membership function valued in the interval [0, 1] (zadeh, 1965). the concept of a type-2 fuzzy set was introduced by zadeh (1975) as an extension of the concept of an ordinary fuzzy set called a type-1 fuzzy set. such sets are fuzzy sets whose membership grades themselves are type-1 fuzzy sets. they are very useful in circumstances where it is difficult to determine an exact membership function for a fuzzy set; hence, they are useful for incorporating linguistic uncertainties, e.g., the words that are used in linguistic knowledge can mean different things to different people (karnik & mendel, 2001). while the membership functions of ordinary fuzzy sets are two-dimensional, the membership functions of type-2 fuzzy sets are three-dimensional. it is the new third-dimension that provides additional degrees of freedom that make it possible to directly model uncertainties. figure 1 illustrates an interval type-2 fuzzy set. figure 1 interval type-2 fuzzy set where jx denotes an interval [0,1]. the type-2 fuzzy set ã̃ also can be represented by equation 11 (mendel et al., 2006): ã̃ = ∫ ∫ μ ã̃ (x,u) (x,u)⁄ u∈jxx∈x (11) where jx ⊆ [0,1] and ∬ denote union over all admissible 𝑥 and 𝑢. let ã̃ be a type-2 fuzzy set in the universe of discourse x represented by the type-2 membership function μ ã̃ . if all μ ã̃ (x,u) = 1, then ã̃ is called an interval type-2 fuzzy set (buckley, 1985). an interval type-2 fuzzy set ã̃ can be regarded as a special case of a type-2 fuzzy set, represented by eq. (12) (mendel et al., 2006): ã̃ = ∫ ∫ 1 (x,u)⁄ u∈jxx∈x (12) http://en.wikipedia.org/wiki/membership_function_%28mathematics%29 ijahp article: kahraman, öztayşi, onar, doğan / intuitionistic fuzzy originated interval type-2 fuzzy ahp: an application to damless hydroelectric power plants international journal of the analytic hierarchy process 271 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.538 where jx ⊆ [0,1]. arithmetic operations with triangular interval type-2 fuzzy sets are given in the following. definition 6: the upper and lower membership functions of an interval type-2 fuzzy set are type-1 membership functions. a triangular interval type-2 fuzzy set is illustrated as �̃̃�𝑖 = (�̃�𝑖 𝑈; �̃�𝑖 𝐿) = ((𝑎𝑖1 𝑈 ,𝑎𝑖2 𝑈 ,𝑎𝑖3 𝑈 ;𝐻(�̃�𝑖 𝑈)), (𝑎𝑖1 𝐿 ,𝑎𝑖2 𝐿 ,𝑎𝑖3 𝐿 , ;𝐻(�̃�𝑖 𝐿))) where �̃�𝑖 𝑈 and �̃�𝑖 𝐿 are type-1 fuzzy sets; 𝑎𝑖1 𝑈 , 𝑎𝑖2 𝑈 , 𝑎𝑖3 𝑈 , 𝑎𝑖1 𝐿 , 𝑎𝑖2 𝐿 and 𝑎𝑖3 𝐿 are the reference points of the interval type-2 fuzzy set �̃̃�𝑖. 𝐻(�̃�𝑖 𝑈) denotes the membership value of the element 𝑎𝑖2 𝑈 in the upper triangular membership function. 𝐻(�̃�𝑖 𝐿) denotes the membership value of the element 𝑎𝑎2 𝐿 in the lower triangular membership function. 𝐻(�̃�𝑖 𝑈) ∈ [0,1], 𝐻(�̃�𝑖 𝐿) ∈ [0,1] and 1 ≤ 𝑖 ≤ 𝑛 (chen & lee, 2010). definition 7: the addition operation between the triangular interval type-2 fuzzy sets �̃̃�1 = ((𝑎11 𝑈 ,𝑎12 𝑈 ,𝑎13 𝑈 ;𝐻(�̃�1 𝑈)), (𝑎11 𝐿 ,𝑎12 𝐿 ,𝑎13 𝐿 , ;𝐻(�̃�1 𝐿))) and �̃̃�2 = ((𝑎21 𝑈 ,𝑎22 𝑈 ,𝑎23 𝑈 ;𝐻(�̃�2 𝑈)), (𝑎21 𝐿 ,𝑎22 𝐿 ,𝑎23 𝐿 , ;𝐻(�̃�2 𝐿))) is defined by equation 13 (chen & lee, 2010): �̃̃�1 ⊕ �̃̃�2 = ((𝑎11 𝑈 + 𝑎21 𝑈 ,𝑎12 𝑈 + 𝑎22 𝑈 ,𝑎13 𝑈 + 𝑎23 𝑈 ; 𝑚𝑖𝑛(𝐻(�̃�1 𝑈);𝐻(�̃�2 𝑈)),𝑚𝑖𝑛(𝐻(�̃�1 𝑈);𝐻(�̃�2 𝑈))), (𝑎11 𝐿 + 𝑎21 𝐿 ,𝑎12 𝐿 + 𝑎22 𝐿 ,𝑎13 𝐿 + 𝑎23 𝐿 , ; 𝑚𝑖𝑛(𝐻(�̃�1 𝐿);𝐻(�̃�2 𝐿)),𝑚𝑖𝑛(𝐻(�̃�1 𝐿);𝐻(�̃�2 𝐿)))) (13) definition 8: the subtraction operation between the triangular interval type-2 fuzzy sets �̃̃�1 = ((𝑎11 𝑈 ,𝑎12 𝑈 ,𝑎13 𝑈 ;𝐻(�̃�1 𝑈)), (𝑎11 𝐿 ,𝑎12 𝐿 ,𝑎13 𝐿 , ;𝐻(�̃�1 𝐿))) and �̃̃�2 = ((𝑎21 𝑈 ,𝑎22 𝑈 ,𝑎23 𝑈 ;𝐻(�̃�2 𝑈)), (𝑎21 𝐿 ,𝑎22 𝐿 ,𝑎23 𝐿 , ;𝐻(�̃�2 𝐿))) is defined by equation 14 (chen & lee, 2010): �̃̃�1 ⊖ �̃̃�2 = ((𝑎11 𝑈 − 𝑎23 𝑈 ,𝑎12 𝑈 − 𝑎22 𝑈 ,𝑎13 𝑈 − 𝑎21 𝑈 ; 𝑚𝑖𝑛(𝐻(�̃�1 𝑈);𝐻(�̃�2 𝑈)),𝑚𝑖𝑛(𝐻(�̃�1 𝑈);𝐻(�̃�2 𝑈))), ((𝑎11 𝐿 − 𝑎23 𝐿 ,𝑎12 𝐿 − 𝑎22 𝐿 ,𝑎13 𝐿 − 𝑎21 𝐿 ; 𝑚𝑖𝑛(𝐻(�̃�1 𝑈);𝐻(�̃�2 𝑈)),𝑚𝑖𝑛(𝐻(�̃�1 𝑈);𝐻(�̃�2 𝑈))) (14) ijahp article: kahraman, öztayşi, onar, doğan / intuitionistic fuzzy originated interval type-2 fuzzy ahp: an application to damless hydroelectric power plants international journal of the analytic hierarchy process 272 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.538 definition 9: the multiplication operation between the triangular interval type-2 fuzzy sets �̃̃�1 = ((𝑎11 𝑈 ,𝑎12 𝑈 ,𝑎13 𝑈 ;𝐻(�̃�1 𝑈)), (𝑎11 𝐿 ,𝑎12 𝐿 ,𝑎13 𝐿 , ;𝐻(�̃�1 𝐿))) and �̃̃�2 = ((𝑎21 𝑈 ,𝑎22 𝑈 ,𝑎23 𝑈 ;𝐻(�̃�2 𝑈)), (𝑎21 𝐿 ,𝑎22 𝐿 ,𝑎23 𝐿 , ;𝐻(�̃�2 𝐿)))is defined by eq. (15) (chen & lee, 2010): �̃̃�1 ⊗ �̃̃�2 ≅ ((𝑎11 𝑈 × 𝑎21 𝑈 ,𝑎12 𝑈 × 𝑎22 𝑈 ,𝑎13 𝑈 × 𝑎23 𝑈 ; 𝑚𝑖𝑛(𝐻(�̃�1 𝑈);𝐻(�̃�2 𝑈)),𝑚𝑖𝑛(𝐻(�̃�1 𝑈);𝐻(�̃�2 𝑈))), (𝑎11 𝐿 × 𝑎21 𝐿 ,𝑎12 𝐿 × 𝑎22 𝐿 ,𝑎13 𝐿 × 𝑎23 𝐿 ; 𝑚𝑖𝑛(𝐻(�̃�1 𝑈);𝐻(�̃�2 𝑈)),𝑚𝑖𝑛(𝐻(�̃�1 𝑈);𝐻(�̃�2 𝑈))) (15) definition 10: the arithmetic operations between the triangular interval type-2 fuzzy sets �̃̃�1 = ((𝑎11 𝑈 ,𝑎12 𝑈 ,𝑎13 𝑈 ;𝐻(�̃�1 𝑈)), (𝑎11 𝐿 ,𝑎12 𝐿 ,𝑎13 𝐿 , ;𝐻(�̃�1 𝐿))) and �̃̃�2 = ((𝑎21 𝑈 ,𝑎22 𝑈 ,𝑎23 𝑈 ;𝐻(�̃�2 𝑈)), (𝑎21 𝐿 ,𝑎22 𝐿 ,𝑎23 𝐿 , ;𝐻(�̃�2 𝐿))) and the crisp value k is defined by equations 16 and 17 (chen & lee, 2010): 𝑘�̃̃�1 = ((𝑘 × 𝑎11 𝑈 ,𝑘 × 𝑎12 𝑈 ,𝑘 × 𝑎13 𝑈 );𝐻(�̃�1 𝑈), (𝑘 × 𝑎11 𝐿 ,𝑘 × 𝑎12 𝐿 ,𝑘 × 𝑎13 𝐿 ;𝐻(�̃�1 𝐿))) (16) �̃̃�1 𝑘 = (( 1 𝑘 × 𝑎11 𝑈 , 1 𝑘 × 𝑎12 𝑈 , 1 𝑘 × 𝑎13 𝑈 );𝐻(�̃�1 𝑈), ( 1 𝑘 × 𝑎11 𝐿 , 1 𝑘 × 𝑎12 𝐿 , 1 𝑘 × 𝑎13 𝐿 ;𝐻(�̃�1 𝐿))) (17) where k > 0. 3. intuitionistic fuzzy originated interval type-2 fuzzy ahp the steps of the methodology are given in the following: step 1: determine the evaluation criteria and alternatives and define the hierarchy of the problem. step 2: construct the pairwise comparison matrices using a triangular intuitionistic fuzzy linguistic evaluation scale for each of the experts based on our proposed scale given in table 1. ijahp article: kahraman, öztayşi, onar, doğan / intuitionistic fuzzy originated interval type-2 fuzzy ahp: an application to damless hydroelectric power plants international journal of the analytic hierarchy process 273 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.538 table 1 triangular intuitionistic fuzzy scale linguistic terms intuitionistic fuzzy numbers absolutely strong (as) {(7.5, 9, 10.5), (7, 9, 11)} very strong (vs) {(5.5, 7, 8.5), (5, 7, 9)} fairly strong (fs) {(3.5, 5, 6.5), (3, 5, 7)} slightly strong (ss) {(1.5, 3, 4.5), (1, 3, 5)} exactly equal (e) {(1, 1, 1), (1, 1, 1)} if factor i has one of the above linguistic variables assigned to it when compared with factor j, then j has the reciprocal value when compared with i: 1/�̃�𝑗 = ((1/𝑎𝑗 𝑈,1/𝑎𝑗 𝑀,1/ 𝑎𝑗 𝐿),(1/𝑎𝑗 ′𝑈,1/𝑎𝑗 𝑀,1/ 𝑎𝑗 ′𝐿)) reciprocals of above step 3: check the consistency of each fuzzy pairwise comparison matrix. assume  ijaa ~~ ~~  is a fuzzy positive reciprocal matrix and  ijaa  is its defuzzified positive reciprocal matrix. if the result of the comparisons of  ijaa  is consistent, then it can imply that the result of the comparisons of  ijaa ~~ ~~  is also consistent. in order to check the consistencies of the fuzzy pairwise comparison matrices, the proposed defuzzification method given in equation 9 is used. step 4: aggregate the intuitionistic fuzzy pairwise comparison matrices using equation 10. step5: transform the aggregated triangular intuitionistic fuzzy pairwise comparison matrix into triangular type-2 fuzzy (tt2f) pairwise comparison matrix. an example of this transformation is illustrated in figure 2. ijahp article: kahraman, öztayşi, onar, doğan / intuitionistic fuzzy originated interval type-2 fuzzy ahp: an application to damless hydroelectric power plants international journal of the analytic hierarchy process 274 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.538 (a) (b) figure 2 transformation from tifn to tt2f number a type-2 fuzzy pairwise comparison matrix is represented by equation 18.                                 1~~ 1~~ 1 ~~1~~ 1 ~~~~1 1 ~~~~ ~~1 ~~ ~~~~1 ~~ 21 2 12 112 21 2 112 21         nn n n nn n n aa a a aa aa aa aa a (18) where 1 �̃̃�⁄ = {( 1 𝑎13 𝑈 , 1 𝑎12 𝑈 , 1 𝑎11 𝑈 ;1),( 1 𝑎23 𝐿 , 1 𝑎22 𝐿 , 1 𝑎21 𝐿 ;1)} step 6: calculate the geometric mean of each row and then compute the fuzzy weights by normalization. the geometric mean of each row jr ~~ is calculated by equation 19. �̃̃�𝑗 = [�̃̃�𝑗1 ⊗ … ⊗ �̃̃�𝑗𝑛] 1 𝑛⁄ (19) where √�̃̃�𝑗𝑛 𝑛 = {((√∏ 𝑎𝑗1 𝑈𝑛 𝑗=1 𝑛 , √∏ 𝑎𝑗2 𝑈𝑛 𝑗=1 𝑛 , √∏ 𝑎𝑗3 𝑈𝑛 𝑗=1 𝑛 ;1)),((√∏ 𝑎𝑗1 𝐿𝑛 𝑗=1 𝑛 , √∏ 𝑎𝑗2 𝐿𝑛 𝑗=1 𝑛 , √∏ 𝑎𝑗3 𝐿𝑛 𝑗=1 𝑛 ;1))} the fuzzy weight i w ~~ of the i th criterion is calculated by equation 20. ijahp article: kahraman, öztayşi, onar, doğan / intuitionistic fuzzy originated interval type-2 fuzzy ahp: an application to damless hydroelectric power plants international journal of the analytic hierarchy process 275 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.538 �̃̃�𝑖 = �̃̃�𝑖 ⊗ [�̃̃�1 ⊕ …⊕ �̃̃�𝑖 ⊕ …⊕ �̃̃�𝑛] −1 (20) where �̃̃�𝑖𝑗 �̃̃�𝑖𝑗 = {( 𝑎1 𝑢 𝑏3 𝑢 , 𝑎2 𝑢 𝑏2 𝑢 , 𝑎3 𝑢 𝑏1 𝑢 ,1) ,( 𝑎1 𝐿 𝑏3 𝐿 , 𝑎2 𝐿 𝑏2 𝐿 , 𝑎3 𝐿 𝑏1 𝐿 ,1)} step 7: aggregate the fuzzy weights and fuzzy performance scores by equation 21: �̃̃�𝑖 = ∑ �̃̃�𝑗�̃̃�𝑖𝑗,∀𝑖. 𝑛 𝑗=1 (21) where �̃̃�𝑖 is the fuzzy utility of alternative i; �̃̃�𝑗 is the weight of the criterion j, and �̃̃�𝑖𝑗 is the performance score of alternative i with respect to criterion j. step 8: apply the classical ahp method’s procedure to determine the best alternative. in order to find the crisp weights, the defuzzification formula given in equation 22 is applied. for an interval type-2 triangular fuzzy number ã̃1 = ((a11 u ,a12 u ,a13 u ;𝐻(ã1 u),(a11 l ,a12 l ,a13 l ;𝐻(ã1 l))) the defuzzification formula is as follows (kahraman et al., 2014): dttrt= (a13 u -a11 u )+(a12 u -a11 u ) 3 +a11 u +h(ã1 l )[ (a13 l -a11 l ,)+( a12 l -a11 l ) 3 +a11 l ] 2 (22) 4. an application to damless hydroelectric power plants existing energy structures that seriously pollute the environment because of the use of fossil fuels must be gradually replaced by clean and renewable energy structures (li et al., 2018). all over the world, efforts to reduce carbon emissions have increased interest in renewable energy production. with the global acceptance of the effort to reduce the use of fossil fuels, renewable energy will gain importance in the next three to four decades (sarasúa et al., 2014; sternberg, 2010). hydroelectric power generation is of special interest to generate clean energy (ioannidou & o’hanley, 2018). one of the hydroelectric power plant types is damless hydroelectric power. it can be referred to as flow-of-the-river power plants. the transition of the flow energy of a river into electricity by a damless or free-flow hydroelectric power plant is called damless hydroelectric transition (volshanik, 1999). damless hydropower plants are characterized by the storage capacity of the energy (sarasúa et al., 2014). the energy thus produced varies greatly depending on the current flow in the stream. damless hydroelectric plants have two main advantages which are high social sustainability and low environmental cost in rural areas (jager et al., 2015). damless plants have shown promise in rural areas of the u.s., europe, africa and asia with the potential of generating hydroelectricity (tilmant et al., 2012; kuby et al., 2005; szabó et al., 2013; reddy et al., 2006). irrigation systems and waste water streams can provide opportunities for harmless hydropower generation (adhau et al., 2012). although ijahp article: kahraman, öztayşi, onar, doğan / intuitionistic fuzzy originated interval type-2 fuzzy ahp: an application to damless hydroelectric power plants international journal of the analytic hierarchy process 276 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.538 damless hydroelectric plants have advantages, there are also some limitations. generating energy with damless technologies is water dependent and depends on the speed of streamflow (sternberg, 2010). however, new damless technologies try to overcome these limitations to generate energy permanently. there are two alternatives that will be compared in this study. these alternatives are the cascaded damless hydroelectric turbine (a1) and the helical turbine (a2). the cascades of hydro turbines are installed on pontoons or fixed in the water stream by means of cantilever suspension. they work without building a dam on the river. figure 3 presents an illustration of cascaded hydro turbines. figure 3 cascades of hydro turbines helical unique generation (hug) is a new alternate source of hydroelectric energy without a dam (figure 4). the helical turbine is a cross flow unidirectional rotation machine that makes it valuable for underwater currents generated by wave fluctuations. figure 4 helical turbine the following evaluation criteria have been considered for this multi-criteria decision making problem (https://www.turbulent.be/damless-hydro-power-plants-and-itsimplications/). https://www.turbulent.be/damless-hydro-power-plants-and-its-implications/ https://www.turbulent.be/damless-hydro-power-plants-and-its-implications/ ijahp article: kahraman, öztayşi, onar, doğan / intuitionistic fuzzy originated interval type-2 fuzzy ahp: an application to damless hydroelectric power plants international journal of the analytic hierarchy process 277 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.538 cost (c1): in the damless hydroelectric power plant projects, several factors affect the cost such as plant size, local economy, resources to be used, site-specific conditions (river flow, water amount etc.). the costs of the turbine technology are also taken into account as cost criterion, which refers to the sum of equipment, electrical connection, infrastructure, planning and other installation costs. maintenance frequency (c2): operating and maintenance frequency refers to the frequency of activities required to maximize the energy produced and to avoid any costly breakdowns. corrosion resistance (c3): submerged metals corrode in time. the duration of corrosion exposure is an important factor in evaluating turbine alternatives as it directly affects cost. production capacity (c4): the amount of electricity that a turbine can generate is another consideration in damless hydroelectric power plant projects. technological upgrade (c5): at a particular point, it will often be economical to upgrade the current technology in order to increase production capacity and reduce increased operating and maintenance costs. the refurbishment may require additional construction work depending on the degree of corrosion and wear. 4.1 application of the method step 1: the hierarchy of the problem is presented in figure 5. goal: selection of the best technology for damless energy production cost maintenance frequency corrosion resistance production capacity technological upgrade cascades of hydro turbines helical turbine figure 5 hierarchy of the damless technology selection problem ijahp article: kahraman, öztayşi, onar, doğan / intuitionistic fuzzy originated interval type-2 fuzzy ahp: an application to damless hydroelectric power plants international journal of the analytic hierarchy process 278 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.538 step 2: the weights of three experts are determined as 0.25, 0.45 and 0.30 based on their experiences. their pairwise comparison matrices are given in table 2. table 2 pairwise comparison matrices of the main criteria with respect to the goal expert 1 c1 c2 c3 c4 c5 c1 e fs vs 1/ss ss c2 1/fs e ss 1/vs 1/ss c3 1/vs 1/ss e 1/as 1/vs c4 ss vs as e vs c5 1/ss ss vs 1/vs e expert 2 c1 c2 c3 c4 c5 c1 e ss fs 1/fs e c2 1/ss e e 1/as 1/fs c3 1/fs e e 1/as 1/fs c4 fs as as e as c5 e fs fs 1/as e expert 3 c1 c2 c3 c4 c5 c1 e fs ss 1/vs e c2 1/vs e e 1/fs 1/ss c3 1/ss e e 1/as 1/vs c4 vs fs vs e vs c5 e ss vs 1/as e the pairwise comparisons of the alternatives with respect to the criteria have been made by three experts by using the linguistic scale given in table 1. these comparisons are presented in table 3. table 3 pairwise comparison matrices of the alternatives with respect to the criteria expert 1 c1 c2 c3 c4 c5 a1 a2 a1 a2 a1 a2 a1 a2 a1 a2 a1 e vs a1 e ss a1 e 1/ss a1 e fs a1 e fs a2 1/vs e a2 1/ss e a2 ss e a2 1/fs e a2 1/fs e expert 2 c1 c2 c3 c4 c5 a1 a2 a1 a2 a1 a2 a1 a2 a1 a2 a1 e vs a1 e fs a1 e e a1 e ss a1 e ss a2 1/vs e a2 1/fs e a2 e e a2 1/ss e a2 1/ss e expert 3 c1 c2 c3 c4 c5 a1 a2 a1 a2 a1 a2 a1 a2 a1 a2 a1 e fs a1 e ss a1 e 1/fs a1 e fs a1 e e a2 1/fs e a2 1/ss e a2 fs e a2 1/fs e a2 e e step 3: in this step, the consistencies of the intuitionistic fuzzy pairwise comparison matrices are checked. first of all, the values in the pairwise comparison matrices of the main criteria are defuzzified by using equation 9 and then the crisp consistency measurement method is applied. the consistency of each pairwise comparison matrix has been provided to be under 0.1. the value of τ has been assigned as 100 in our ijahp article: kahraman, öztayşi, onar, doğan / intuitionistic fuzzy originated interval type-2 fuzzy ahp: an application to damless hydroelectric power plants international journal of the analytic hierarchy process 279 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.538 calculations. the defuzzified pairwise comparison matrices for the main criteria are given in table 4. table 4 defuzzified pairwise comparison matrices expert 1 expert 2 expert 3 c 1 c 2 c 3 c 4 c 5 c 1 c 2 c 3 c 4 c 5 c 1 c 2 c 3 c 4 c 5 c 1 1. 0 5. 2 7. 2 0. 3 3. 1 c 1 1. 0 3. 1 5. 2 0. 2 1. 0 c 1 1. 0 3. 1 3. 1 0. 1 1. 0 c 2 0. 2 1. 0 3. 1 0. 1 0. 3 c 2 0. 3 1. 0 1. 0 0. 1 0. 2 c 2 0. 3 1. 0 1. 0 0. 2 0. 3 c 3 0. 1 0. 3 1. 0 0. 1 0. 1 c 3 0. 2 1. 0 1. 0 0. 1 0. 2 c 3 0. 3 1. 0 1. 0 0. 1 0. 1 c 4 3. 1 7. 2 9. 3 1. 0 7. 2 c 4 5. 2 9. 3 9. 3 1. 0 9. 3 c 4 7. 2 5. 2 9. 3 1. 0 7. 2 c 5 0. 3 3. 1 7. 2 0. 1 1. 0 c 5 1. 0 5. 2 5. 2 0. 1 1. 0 c 5 1. 0 3. 1 7. 2 0. 1 1. 0 cr = 0.085 cr = 0.073 cr = 0.097 since the pairwise comparisons of alternatives are based on 2 × 2 matrices, it is not necessary to check the consistencies of these matrices. step 4: the aggregated triangular intuitionistic fuzzy pairwise comparison matrices for the main criteria and alternatives are given in table 5 and table 6, respectively. ijahp article: kahraman, öztayşi, onar, doğan / intuitionistic fuzzy originated interval type-2 fuzzy ahp: an application to damless hydroelectric power plants international journal of the analytic hierarchy process 280 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.538 table 5 aggregated triangular intuitionistic fuzzy pairwise comparison matrix for the criteria c1 c2 c3 c 1 {(1, 1, 1), (1, 1, 1)} {(2.39, 3.97, 5.51), (1.83, 3.97, 6.02)} {(3.04, 4.67, 6.22), (2.45, 4.67, 6.74)} c 2 {(0.17, 0.23, 0.37), (0.15, 0.23, 0.47)} {(1, 1, 1), (1, 1, 1)} {(1.11, 1.32, 1.46), (1, 1.32, 1.5)} c 3 {(0.16, 0.21, 0.33), (0.15, 0.21, 0.41)} {(0.69, 0.76, 0.9), (0.67, 0.76, 1)} {(1, 1, 1), (1, 1, 1)} c 4 {(3.24, 4.87, 6.43), (2.66, 4.87, 6.94)} {(5.52, 7.09, 8.63), (4.99, 7.09, 9.14)} {(6.83, 8.35, 9.86), (6.33, 8.35, 10.36)} c 5 {(0.69, 0.76, 0.9), (0.67, 0.76, 1)} {(2.2, 3.78, 5.31), (1.64, 3.78, 5.82)} {(4.49, 6.02, 7.53), (3.97, 6.02, 8.04)} c4 c5 c 1 {(0.16, 0.21, 0.31), (0.14, 0.21, 0.38)} {(1.11, 1.32, 1.46), (1, 1.32, 1.5)} c 2 {(0.12, 0.14, 0.18), (0.11, 0.14, 0.2)} {(0.19, 0.26, 0.46), (0.17, 0.26, 0.61)} c 3 {(0.1, 0.11, 0.13), (0.09, 0.11, 0.14)} {(0.13, 0.17, 0.22), (0.12, 0.17, 0.25)} c 4 {(1, 1, 1), (1, 1, 1)} {(6.32, 7.84, 9.35), (5.82, 7.84, 9.85)} c 5 {(0.1, 0.12, 0.14), (0.1, 0.12, 0.16)} {(1, 1, 1), (1, 1, 1)} table 6 aggregated triangular intuitionistic fuzzy pairwise comparison matrices for the alternatives a1 a2 c1 a1 {(1, 1, 1), (1, 1, 1)} {(4.8, 6.33, 7.84), (4.29, 6.33, 8.35)} a2 {(0.13, 0.16, 0.21), (0.12, 0.16, 0.23)} {(1, 1, 1), (1, 1, 1)} c2 a1 {(1, 1, 1), (1, 1, 1)} {(2.2, 3.78, 5.31), (1.64, 3.78, 5.82)} a2 {(0.19, 0.26, 0.46), (0.17, 0.26, 0.61)} {(1, 1, 1), (1, 1, 1)} c3 a1 {(1, 1, 1), (1, 1, 1)} {(0.39, 0.47, 0.62), (0.37, 0.47, 0.72)} a2 {(1.61, 2.13, 2.55), (1.39, 2.13, 2.68)} {(1, 1, 1), (1, 1, 1)} c4 a1 {(1, 1, 1), (1, 1, 1)} {(2.39, 3.97, 5.51), (1.83, 3.97, 6.02)} a2 {(0.18, 0.25, 0.42), (0.17, 0.25, 0.55)} {(1, 1, 1), (1, 1, 1)} c5 a1 {(1, 1, 1), (1, 1, 1)} {(1.64, 2.45, 3.14), (1.32, 2.45, 3.36)} a2 {(0.32, 0.41, 0.61), (0.3, 0.41, 0.76)} {(1, 1, 1), (1, 1, 1)} ijahp article: kahraman, öztayşi, onar, doğan / intuitionistic fuzzy originated interval type-2 fuzzy ahp: an application to damless hydroelectric power plants international journal of the analytic hierarchy process 281 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.538 step 5: now, the aggregated triangular intuitionistic fuzzy pairwise comparison matrix is transformed into tt2f pairwise comparison matrix for the main criteria (table 7) and the alternatives (table 8). table 7 transformation to tt2f pairwise comparison matrix for the main criteria c1 c2 c1 {(1, 1, 1; 1), (1, 1, 1 ;1)} {(1.83, 3.97, 6.02; 1), (2.39, 3.97, 5.51 ;1)} c2 {(0.15, 0.23, 0.47; 1), (0.17, 0.23, 0.37 ;1)} {(1, 1, 1; 1), (1, 1, 1 ;1)} c3 {(0.15, 0.21, 0.41; 1), (0.16, 0.21, 0.33 ;1)} {(0.67, 0.76, 1; 1), (0.69, 0.76, 0.9 ;1)} c4 {(2.66, 4.87, 6.94; 1), (3.24, 4.87, 6.43 ;1)} {(4.99, 7.09, 9.14; 1), (5.52, 7.09, 8.63 ;1)} c5 {(0.67, 0.76, 1; 1), (0.69, 0.76, 0.9 ;1)} {(1.64, 3.78, 5.82; 1), (2.2, 3.78, 5.31 ;1)} c3 c4 c1 {(2.45, 4.67, 6.74; 1), (3.04, 4.67, 6.22 ;1)} {(0.14, 0.21, 0.38; 1), (0.16, 0.21, 0.31 ;1)} c2 {(1, 1.32, 1.5; 1), (1.11, 1.32, 1.46 ;1)} {(0.11, 0.14, 0.2; 1), (0.12, 0.14, 0.18 ;1)} c3 {(1, 1, 1; 1), (1, 1, 1 ;1)} {(0.09, 0.11, 0.14; 1), (0.1, 0.11, 0.13 ;1)} c4 {(6.33, 8.35, 10.36; 1), (6.83, 8.35, 9.86 ;1)} {(1, 1, 1; 1), (1, 1, 1 ;1)} c5 {(3.97, 6.02, 8.04; 1), (4.49, 6.02, 7.53 ;1)} {(0.1, 0.12, 0.16; 1), (0.1, 0.12, 0.14 ;1)} c5 c1 {(1, 1.32, 1.5; 1), (1.11, 1.32, 1.46 ;1)} c2 {(0.17, 0.26, 0.61; 1), (0.19, 0.26, 0.46 ;1)} c3 {(0.12, 0.17, 0.25; 1), (0.13, 0.17, 0.22 ;1)} c4 {(5.82, 7.84, 9.85; 1), (6.32, 7.84, 9.35 ;1)} c5 {(1, 1, 1; 1), (1, 1, 1 ;1)} ijahp article: kahraman, öztayşi, onar, doğan / intuitionistic fuzzy originated interval type-2 fuzzy ahp: an application to damless hydroelectric power plants international journal of the analytic hierarchy process 282 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.538 table 8 transformation to tt2f pairwise comparison matrices for the alternatives a1 a2 c 1 a 1 {(1, 1, 1; 1), (1, 1, 1 ;1)} {(4.29, 6.33, 8.35; 1), (4.8, 6.33, 7.84 ;1)} a 2 {(0.12, 0.16, 0.23; 1), (0.13, 0.16, 0.21 ;1)} {(1, 1, 1; 1), (1, 1, 1 ;1)} c 2 a 1 {(1, 1, 1; 1), (1, 1, 1 ;1)} {(1.64, 3.78, 5.82; 1), (2.2, 3.78, 5.31 ;1)} a 2 {(0.17, 0.26, 0.61; 1), (0.19, 0.26, 0.46 ;1)} {(1, 1, 1; 1), (1, 1, 1 ;1)} c 3 a 1 {(1, 1, 1; 1), (1, 1, 1 ;1)} {(0.37, 0.47, 0.72; 1), (0.39, 0.47, 0.62 ;1)} a 2 {(1.39, 2.13, 2.68; 1), (1.61, 2.13, 2.55 ;1)} {(1, 1, 1; 1), (1, 1, 1 ;1)} c 4 a 1 {(1, 1, 1; 1), (1, 1, 1 ;1)} {(1.83, 3.97, 6.02; 1), (2.39, 3.97, 5.51 ;1)} a 2 {(0.17, 0.25, 0.55; 1), (0.18, 0.25, 0.42 ;1)} {(1, 1, 1; 1), (1, 1, 1 ;1)} c 5 a 1 {(1, 1, 1; 1), (1, 1, 1 ;1)} {(1.32, 2.45, 3.36; 1), (1.64, 2.45, 3.14 ;1)} a 2 {(0.3, 0.41, 0.76; 1), (0.32, 0.41, 0.61 ;1)} {(1, 1, 1; 1), (1, 1, 1 ;1)} step 6: the geometric means of each row for alternatives are calculated and given in table 9 and table 10, respectively. table 9 geometric means of tt2f sets for the main criteria the main criteria and c1 {(0.92, 1.38, 1.87; 1), (1.05, 1.38, 1.73 ;1)} c2 {(0.31, 0.41, 0.61; 1), (0.33, 0.41, 0.54 ;1)} c3 {(0.26, 0.31, 0.43; 1), (0.27, 0.31, 0.39 ;1)} c4 {(3.45, 4.68, 5.78; 1), (3.78, 4.68, 5.52 ;1)} c5 {(0.84, 1.15, 1.49; 1), (0.93, 1.15, 1.39 ;1)} ijahp article: kahraman, öztayşi, onar, doğan / intuitionistic fuzzy originated interval type-2 fuzzy ahp: an application to damless hydroelectric power plants international journal of the analytic hierarchy process 283 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.538 table 10 geometric means of tt2f sets for the alternatives c1 a1 {(2.07, 2.52, 2.89; 1), (2.19, 2.52, 2.8 ;1)} a2 {(0.35, 0.4, 0.48; 1), (0.36, 0.4, 0.46 ;1)} c2 a1 {(1.28, 1.94, 2.41; 1), (1.48, 1.94, 2.3 ;1)} a2 {(0.41, 0.51, 0.78; 1), (0.43, 0.51, 0.67 ;1)} c3 a1 {(0.61, 0.68, 0.85; 1), (0.63, 0.68, 0.79 ;1)} a2 {(1.18, 1.46, 1.64; 1), (1.27, 1.46, 1.6 ;1)} c4 a1 {(1.35, 1.99, 2.45; 1), (1.55, 1.99, 2.35 ;1)} a2 {(0.41, 0.5, 0.74; 1), (0.43, 0.5, 0.65 ;1)} c5 a1 {(1.15, 1.57, 1.83; 1), (1.28, 1.57, 1.77 ;1)} a2 {(0.55, 0.64, 0.87; 1), (0.56, 0.64, 0.78 ;1)} tables 11 and 12 give the normalized type-2 fuzzy weights of each criterion and alternative, respectively. table 11 normalized type-2 fuzzy weights of the main criteria with respect to goal c1 {(0.09, 0.17, 0.32; 1), (0.11, 0.17, 0.27 ;1)} c2 {(0.03, 0.05, 0.11; 1), (0.03, 0.05, 0.08 ;1)} c3 {(0.03, 0.04, 0.07; 1), (0.03, 0.04, 0.06 ;1)} c4 {(0.34, 0.59, 1; 1), (0.4, 0.59, 0.87 ;1)} c5 {(0.08, 0.15, 0.26; 1), (0.1, 0.15, 0.22 ;1)} table 12 normalized type-2 fuzzy weights of the alternatives with respect to criteria c1 a1 {(0.14, 0.21, 0.31; 1), (0.15, 0.21, 0.28 ;1)} a2 {(0.02, 0.03, 0.05; 1), (0.03, 0.03, 0.04 ;1)} c2 a1 {(0.09, 0.16, 0.26; 1), (0.1, 0.16, 0.23 ;1)} a2 {(0.03, 0.04, 0.08; 1), (0.03, 0.04, 0.07 ;1)} c3 a1 {(0.04, 0.06, 0.09; 1), (0.04, 0.06, 0.08 ;1)} a2 {(0.08, 0.12, 0.18; 1), (0.09, 0.12, 0.16 ;1)} c4 a1 {(0.09, 0.16, 0.26; 1), (0.11, 0.16, 0.23 ;1)} a2 {(0.03, 0.04, 0.08; 1), (0.03, 0.04, 0.06 ;1)} c5 a1 {(0.08, 0.13, 0.2; 1), (0.09, 0.13, 0.17 ;1)} a2 {(0.04, 0.05, 0.09; 1), (0.04, 0.05, 0.08 ;1)} step 7: the fuzzy performance scores of alternatives with respect to the main criteria are calculated using equation 21 and given in table 13. ijahp article: kahraman, öztayşi, onar, doğan / intuitionistic fuzzy originated interval type-2 fuzzy ahp: an application to damless hydroelectric power plants international journal of the analytic hierarchy process 284 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.538 table 13 fuzzy performance scores of the alternatives a1 {(0.05, 0.16, 0.45; 1), (0.07, 0.16, 0.34 ;1)} a2 {(0.02, 0.04, 0.14; 1), (0.02, 0.04, 0.1 ;1)} in figure 6, the tt2f performance scores of the alternatives are illustrated. figure 6 type-2 fuzzy performance scores step 8: applying the defuzzification formula in equation 22, the following results are obtained: dttrt(𝐴1) = 0.21 dttrt(𝐴2) = 0.06 based on these results, alternative a1 is by far superior to alternative a2. 4.2 validity and sensitivity to check the validity of the proposed methodology, the classical ahp is applied. table 14 shows the crisp numerical evaluations which are the mid-points in the linguistic scale for each expert. ijahp article: kahraman, öztayşi, onar, doğan / intuitionistic fuzzy originated interval type-2 fuzzy ahp: an application to damless hydroelectric power plants international journal of the analytic hierarchy process 285 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.538 table 14 crisp numerical evaluations for criteria by three experts expert 1 expert 2 expert 3 c 1 c 2 c 3 c 4 c 5 c 1 c 2 c 3 c 4 c 5 c 1 c 2 c 3 c 4 c 5 c 1 1 5 7 0.3 3 c 1 1 3 5 0.2 1 c 1 1 5 3 0.1 1 c 2 0.2 1 3 0.1 0.3 c 2 0.3 1 1 0.1 0.2 c 2 0.1 1 1 0.2 0.3 c 3 0.1 0.3 1 0.1 0.1 c 3 0.2 1 1 0.1 0.2 c 3 0.3 1 1 0.1 0.1 c 4 3 7 9 1 7 c 4 5 9 9 1 9 c 4 7 5 7 1 7 c 5 0.3 3 7 0.1 1 c 5 1 5 5 0.1 1 c 5 1 3 7 0.1 1 cr = 0.007 cr = 0.049 cr = 0.074 table 15 presents the crisp numerical evaluations for alternatives by three experts. ijahp article: kahraman, öztayşi, onar, doğan / intuitionistic fuzzy originated interval type-2 fuzzy ahp: an application to damless hydroelectric power plants international journal of the analytic hierarchy process 286 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.538 table 15 crisp numerical evaluations for alternatives by three experts expert 1 c1 c2 c3 c4 c5 a1 a2 a1 a2 a1 a2 a1 a2 a1 a2 a 1 1.00 7.00 a 1 1.00 3.00 a 1 1.00 0.33 a 1 1.00 5.00 a 1 1.00 5.00 a 2 0.14 1.00 a 2 0.33 1.00 a 2 3.00 1.00 a 2 0.20 1.00 a 2 0.20 1.00 expert 2 c1 c2 c3 c4 c5 a1 a2 a1 a2 a1 a2 a1 a2 a1 a2 a 1 1.00 7.00 a 1 1.00 5.00 a 1 1.00 1.00 a 1 1.00 3.00 a 1 1.00 3.00 a 2 0.14 1.00 a 2 0.20 1.00 a 2 1.00 1.00 a 2 0.33 1.00 a 2 0.33 1.00 expert 3 c1 c2 c3 c4 c5 a1 a2 a1 a2 a1 a2 a1 a2 a1 a2 a 1 1.00 5.00 a 1 1.00 3.00 a 1 1.00 0.20 a 1 1.00 5.00 a 1 1.00 1.00 a 2 0.20 1.00 a 2 0.33 1.00 a 2 5.00 1.00 a 2 0.20 1.00 a 2 1.00 1.00 the aggregated pairwise comparison matrix is shown in table 16. table 16 aggregated pairwise comparison matrix with respect to the goal c1 c2 c3 c4 c5 c1 1.00 4.22 4.72 0.21 1.44 c2 0.24 1.00 1.44 0.15 0.28 c3 0.21 0.69 1.00 0.11 0.16 c4 4.72 6.80 9.00 1.00 7.61 c5 0.69 3.56 6.26 0.13 1.00 from the matrix in table 17, the weights are obtained as 𝑤𝐶1 = 0.17, 𝑤𝐶2 = 0.05, 𝑤𝐶3 = 0.04, 𝑤𝐶4 = 0.58, 𝑤𝐶5 = 0.16. ijahp article: kahraman, öztayşi, onar, doğan / intuitionistic fuzzy originated interval type-2 fuzzy ahp: an application to damless hydroelectric power plants international journal of the analytic hierarchy process 287 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.538 table 17 gives the scores of alternatives with respect to the criteria together with their weights. table 17 alternative scores and criteria weights c1 c2 c3 c4 c5 weights 0.17 0.05 0.04 0.58 0.16 a1 0.86 0.79 0.32 0.80 0.71 a2 0.14 0.21 0.68 0.20 0.29 from table 17, the overall scores of alternatives are obtained as 𝐴1 = 0.8, 𝐴2 = 0.2. this shows that crisp and fuzzy approaches yield the same ranking result. however, they may produce different ranking results when asymmetric fuzzy numbers are used with large supports in pairwise comparison matrices. in order to check the robustness of the given decisions one-at-a-time, a sensitivity analysis is applied in the following. in this sensitivity analysis, the weight of each criterion between 0-1 is changed and the ranking of the alternatives is observed. when the weight of a criterion is increased or decreased, the weights of other criteria are allocated proportional to their initial weights. the sum of the weights equals 1 in each case. the sensitivity analysis results are given in figure 7. ijahp article: kahraman, öztayşi, onar, doğan / intuitionistic fuzzy originated interval type-2 fuzzy ahp: an application to damless hydroelectric power plants international journal of the analytic hierarchy process 288 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.538 figure 7 sensitivity analysis results the selection of a1 is a robust decision since in almost all sensitivity analyses a1 is superior to a2. for the changes in the weights of criteria cost (c1), maintenance frequency (c2), production capacity (c4) and technological upgrade (c5), a1 is never overtaken by a2. however, for changes in the weights of the criterion corrosion resistance (c3), a2 is selected when its weight is greater than approximately 0.7. unless the criterion corrosion resistance (c3) is highly important, a1 is certainly better than a2. 0.000 0.100 0.200 0.300 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 c1 a1 a2 0.000 0.100 0.200 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 c2 a1 a2 0.000 0.100 0.200 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 c3 a1 a2 0.000 0.100 0.200 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 c4 a1 a2 0.000 0.100 0.200 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 c5 a1 a2 ijahp article: kahraman, öztayşi, onar, doğan / intuitionistic fuzzy originated interval type-2 fuzzy ahp: an application to damless hydroelectric power plants international journal of the analytic hierarchy process 289 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.538 5. conclusion ahp is the most used multi-criteria decision making method all over the world. it has been extended to several fuzzy versions such as hesitant fuzzy ahp and intuitionistic fuzzy ahp. we have proposed a fuzzy ahp method which is originally formulated by intuitionistic fuzzy numbers and then converted to type-2 fuzzy ahp. although modeling with intuitionistic fuzzy sets is popular and easier with respect to the other fuzzy extensions, the solution of intuitionistic fuzzy models is somewhat harder than type-2 fuzzy modeling. hence, a transformation from intuitionistic fuzzy sets to type-2 fuzzy sets has been preferred. the pairwise comparative evaluations of multiple experts has also been considered and these evaluations through intuitionistic fuzzy aggregation operators have been aggregated. another important advantage of our study is the usage of fuzzy sets along the whole process and the usage of defuzzification in the final step. the damless energy production technology selection problem proved the validity and consistency of our proposed methodology. the sensitivity analysis showed that the decision given by the methodology is quite robust. for further research, other possible transformations between extensions of fuzzy sets such as from pythagorean fuzzy sets to type-2 fuzzy sets are suggested. in addition, interval valued intuitionistic fuzzy numbers or trapezoidal intuitionistic fuzzy numbers can be used instead of tifns in the proposed methodology. ijahp article: kahraman, öztayşi, onar, doğan / intuitionistic fuzzy originated interval type-2 fuzzy ahp: an application to damless hydroelectric power plants international journal of the analytic hierarchy process 290 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.538 references adhau, s. p., moharil, r. m., & adhau, p. g. (2012). mini-hydro power generation on existing irrigation projects: case study of indian sites. renewable and sustainable energy reviews, 16(7), 4785-4795. doi: https://doi.org/10.1016/j.rser.2012.03.066 atanassov, k. (1986). intuitionistic fuzzy sets. fuzzy sets and systems, 20(1), 87–96. doi: https://doi.org/10.1016/s0165-0114(86)80034-3 atanassov, k. (1999). intuitionistic fuzzy sets. heidelberg, germany: springer. buckley, j.j. (1985). fuzzy hierarchical analysis. fuzzy sets and systems, 34, 187-195. büyüközkan g., güleryüz s., karpak b. (2017). a new combined if-dematel and ifanp approach for crm partner evaluation. international journal of production economics, 191, 194-206. doi: https://doi.org/10.1016/j.ijpe.2017.05.012 cevik onar, s., oztaysi, b., kahraman, c. (2014). strategic decision selection using hesitant fuzzy topsis and interval type-2 fuzzy ahp: a case study. international journal of computational intelligence systems, 7(5), 1002-1021. doi: https://doi.org/10.1016/s0165-0114(86)80034-3 chang, d. y. (1996). applications of the extent analysis method on fuzzy-ahp. european journal of operational research, 95(3), 649-655. doi: https://doi.org/10.1016/0377-2217(95)00300-2 chen, x., & tan, c. (2010). intuitionistic fuzzy choquet integral operator for multicriteria decision making. expert systems with applications, 37(1), 149-157. doi: https://doi.org/10.1016/j.eswa.2009.05.005 chen, d., zhang, l., jiao, j. (2010). triangle fuzzy number intuitionistic fuzzy aggregation operators and their application to group decision making. in f.l. wang, h. deng, y. gao, j. lei (eds.), artificial intelligence and computational intelligence (350357). berlin/heidelberg: springer-verlag. ilbahar, e., karaşan, a. cebi, s. & kahraman, c. (2018). a novel approach to risk assessment for occupational health and safety using pythagorean fuzzy ahp & fuzzy inference system. safety science, 103, 124-136. doi: https://doi.org/10.1016/j.ssci.2017.10.025 ioannidou, c. & o’hanley, j. r. (2018). eco-friendly location of small hydropower. european journal of operational research, 264(3), 907-918. doi: https://doi.org/10.1016/j.ejor.2016.06.067 jager, h. i., efroymson, r. a., opperman, j. j., & kelly, m. r. (2015). spatial design principles for sustainable hydropower development. renewable and sustainable energy reviews, 45, 808-816. doi: https://doi.org/10.1016/j.rser.2015.01.067 https://doi.org/10.1016/j.rser.2012.03.066 https://doi.org/10.1016/s0165-0114(86)80034-3 https://doi.org/10.1016/j.ijpe.2017.05.012 https://www.tandfonline.com/doi/abs/10.1080/18756891.2014.964011 https://www.tandfonline.com/doi/abs/10.1080/18756891.2014.964011 https://doi.org/10.1016/s0165-0114(86)80034-3 https://doi.org/10.1016/0377-2217(95)00300-2 https://doi.org/10.1016/j.eswa.2009.05.005 https://doi.org/10.1016/j.ssci.2017.10.025 https://doi.org/10.1016/j.ejor.2016.06.067 ijahp article: kahraman, öztayşi, onar, doğan / intuitionistic fuzzy originated interval type-2 fuzzy ahp: an application to damless hydroelectric power plants international journal of the analytic hierarchy process 291 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.538 kahraman, c. öztayşi, b., uçal sarı, i̇., turanoğlu, e. (2014), fuzzy analytic hierarchy process with interval type-2 fuzzy sets. knowledge-based systems, 59, 48-57. doi: https://doi.org/10.1016/j.knosys.2014.02.001 karnik n. n. & mendel, j. m. (2001). centroid of a type-2 fuzzy set. information science, 132, 195–220. doi: https://doi.org/10.1016/s0020-0255(01)00069-x kuby, m. j., fagan, w. f., revelle, c. s., & graf, w. l. (2005). a multiobjective optimization model for dam removal: an example trading off salmon passage with hydropower and water storage in the willamette basin. advances in water resources, 28(8), 845-855. doi: https://doi.org/10.1016/j.advwatres.2004.12.015 laarhoven, p. j. m., and pedrycz, w. (1983). a fuzzy extension of saaty’s priority theory. fuzzy sets and systems, 11, 229-241. doi: https://doi.org/10.1016/s01650114(83)80082-7 li, x.z., chen, z.j., fan, x.c., & cheng, z.j. (2018). hydropower development situation and prospects in china. renewable and sustainable energy reviews, 82(1), 232-239. doi: https://doi.org/10.1016/j.rser.2017.08.090 mendel, j. m., john, r. i. & liu, f. (2006). interval type-2 fuzzy logic systems made simple. ieee transaction fuzzy systems, 14(6), 808–821. doi: https://doi.org/10.1109/tfuzz.2006.879986 otay i̇., öztayşi b., çevik onar s., kahraman c. (2017). multi-expert performance evaluation of healthcare institutions using an integrated intuitionistic fuzzy ahp&dea methodology. knowledge-based systems, 1, 123-124. doi: https://doi.org/10.1016/j.knosys.2017.06.028 özcan, e. c., ünlüsoy, s. & eren, t. (2017). a combined goal programming – ahp approach supported with topsis for maintenance strategy selection in hydroelectric power plants. renewable and sustainable energy reviews, 78, 1410-1423. doi: https://doi.org/10.1016/j.rser.2017.04.039 sternberg, r. (2010). hydropower's future, the environment, and global electricity systems. renewable and sustainable energy reviews, 14(2), 713-723. doi: https://doi.org/10.1016/j.rser.2009.08.016 reddy, v. r., uitto, j. i., frans, d. r., & matin, n. (2006). achieving global environmental benefits through local development of clean energy? the case of small hilly hydel in india. energy policy, 34(18), 4069-4080. doi: https://doi.org/10.1016/j.enpol.2005.09.026 sarasúa, j. i., elías, p., martínez-lucas, g., pérez-díaz, j. i., wilhelmi, j. r., & sánchez, j. á. (2014). stability analysis of a run-of-river diversion hydropower plant with surge tank and spillway in the head pond. the scientific world journal, 2014, 1-13. doi: https://doi.org/10.1155/2014/874060 https://doi.org/10.1016/j.knosys.2014.02.001 https://doi.org/10.1016/s0020-0255(01)00069-x https://doi.org/10.1016/j.advwatres.2004.12.015 https://doi.org/10.1016/s0165-0114(83)80082-7 https://doi.org/10.1016/s0165-0114(83)80082-7 https://doi.org/10.1016/j.rser.2017.08.090 https://doi.org/10.1109/tfuzz.2006.879986 https://doi.org/10.1016/j.knosys.2017.06.028 https://doi.org/10.1016/j.rser.2017.04.039 https://doi.org/10.1016/j.rser.2009.08.016 https://doi.org/10.1016/j.enpol.2005.09.026 ijahp article: kahraman, öztayşi, onar, doğan / intuitionistic fuzzy originated interval type-2 fuzzy ahp: an application to damless hydroelectric power plants international journal of the analytic hierarchy process 292 vol. 10 issue 2 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i2.538 szabó, s., bódis, k., huld, t., & moner-girona, m. (2013). sustainable energy planning: leapfrogging the energy poverty gap in africa. renewable and sustainable energy reviews, 28, 500-509. doi: https://doi.org/10.1016/j.rser.2013.08.044 tilmant, a., kinzelbach, w., juizo, d., beevers, l., senn, d., & casarotto, c. (2012). economic valuation of benefits and costs associated with the coordinated development and management of the zambezi river basin. water policy, 14(3), 490-508. doi: https://doi.org/10.2166/wp.2011.189 volshanik, v. v. (1999). calculation of the energy potential of flows realized by damless (free-flowing) hydropower plants. hydrotechnical construction, 33(4), 227–229. doi: https://doi.org/10.1007/bf02764513 zadeh, l.a. (1965). fuzzy sets. information and control. 8(3), 338-353. doi: https://doi.org/10.1016/s0019-9958(65)90241-x zadeh, l. a. (1975). the concept of a linguistic variable and its application to approximate reasoning. information sciences, 8, 199-249. doi: https://doi.org/10.1016/0020-0255(75)90036-5 zhu, b. & xu, z. (2014). analytic hierarchy process-hesitant group decision making. european journal of operational research, 239(3), 794-801. doi: https://doi.org/10.1016/j.ejor.2014.06.019 web sources: https://www.turbulent.be/damless-hydro-power-plants-and-its-implications/ https://doi.org/10.1016/j.rser.2013.08.044 https://doi.org/10.2166/wp.2011.189 https://doi.org/10.1007/bf02764513 https://doi.org/10.1016/s0019-9958(65)90241-x https://doi.org/10.1016/0020-0255(75)90036-5 https://doi.org/10.1016/j.ejor.2014.06.019 ijahp article: santos, costa, gomes/multicriteria decision-making in the selection of warships: a new approach to the ahp method international journal of the analytic hierarchy process 147 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.833 multicriteria decision-making in the selection of warships: a new approach to the ahp method marcos dos santos instituto militar de engenharia (ime) brazil marcosdossantos_doutorado_uff@yahoo.com.br igor pinheiro de araújo costa universidade federal fluminense (uff) brazil costa_igor@id.uff.br carlos francisco simões gomes universidade federal fluminense (uff) brazil cfsg1@bol.com.br abstract the budgetary constraints for the brazilian navy (bn) have caused several negative effects, resulting in an undersized fleet, decreasing the capacity to protect marine oil and natural gas fields, combat marine pollution from ships, and monitor other illegal activities at sea and inland waters. this paper aims to choose a medium-sized warship to be built by the bn, through the application of the analytic hierarchy process (ahp) method. after a bibliometric study on multiple-criteria decision-making (mcdm), the ahp was chosen as the most appropriate method for the proposed case study. we analyzed three ship projects with regard to nine operational and economic criteria, taking into account the evaluations of bn officers with recognized experience and knowledge in military operations. we also introduced a sensitivity analysis based on the relationship between standard deviation and mean scores in order to verify and increase the reliability of the ranking. as a result, the methodology suggested that the best option is to build a brandnew ship with more significant modernizations to provide for the operational needs of the bn. keywords: analytic hierarchy process; multi-criteria; warship 1. introduction the national defense strategy (nds) that was approved by decree no. 6,703 on december 18, 2008 was updated in 2012, and aims to ensure the security of brazil in both peacetime and crises by establishing guidelines for the proper preparation and training of the brazilian armed forces (af). even in peace times, brazil must have a modern af, equipped and trained to guarantee its sovereignty and strategic interests, supporting its foreign policy and positions in mailto:marcosdossantos_doutorado_uff@yahoo.com.br mailto:costa_igor@id.uff.br mailto:cfsg1@bol.com.br ijahp article: santos, costa, gomes/multicriteria decision-making in the selection of warships: a new approach to the ahp method international journal of the analytic hierarchy process 148 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.833 international forums. the brazilian navy (bn) must have 18 escort ships (including frigates, corvettes and destroyers) to allow the formation of two task groups including one escort close to the main body, and another for the surface action groups’ remote defense. escort ships must have anti-submarine, anti-surface and anti-aircraft (missile availability) defense capabilities. they must be capable of transporting, supplying, operating and maintaining helicopters being prepared to attack surface and submarine targets, as well as carrying out enlightenment operations. however, since the last decade of the twentieth century, the budget provided by the federal government has fallen short of the amount needed to meet the bn needs, making it impossible to allocate the necessary and sufficient funds for operation, maintenance and re-equipment. according to tenório et al. (2020), due to successive budget restrictions, the bn currently has an undersized fleet of only eleven escort ships, many of which are nearing the end of their useful lives, resulting in non-compliance with the minimum requirements of the naval force, decreasing the operational capacity and the accumulation of demands of various orders. the budgetary context has caused several negative effects including maintenance reduction, spare parts stocks reduction, and training and labor loss. the natural consequence is the degradation of the naval assets’ preparation. thus, the bn's capacity to protect marine oil and natural gas fields, combat marine pollution by ships, support the prevention of irregular fishing and the monitoring of other illegal activities at sea and inland waters is reduced. thus, according to tenório et al. (2020), it is expected that the bn will make “opportunity purchases” or build new vessels, in order to maintain the readiness of the fleet, as well as the training of its entire operating structure. this paper aims to choose a new vessel to be built for the bn by applying the analytic hierarchy process (ahp) technique. the ahp was chosen because it is a compensatory and hierarchic method that is indicated mainly for problems with a medium number of alternatives and criteria, considering the discrimination of results and cognitive effort in the pairwise comparisons. the concepts of hierarchy and compensatory decision rules are in accord with military culture, which facilitates the analysis by the experts. the paper is structured into six sections. the introduction describes the objectives of the research. section 2 presents the literature review. section 3 provides the theoretical background, while section 4 presents the methodology applied in the study. section 5 describes the problem, analyzes the results and introduces the sensitivity analysis through the mean scores and standard deviation. section 6 concludes the research. 2. literature review the academic literature contains many examples of the application of multi-criteria decision-making (mcdm) methods in the military field (hamurcu & eren, 2020). bisdikian et al. (2013) proposed a framework for scoring and classification of network sensors, which are relevant for military applications, based on their attributes, using the ahp method. gigović et al. (2016) presented a new mcdm technique, mairca (multitax ideal-real comparative analysis), based on the combined use of geographic information systems (gis) and multicriteria techniques. the authors applied the model dematel-anp for the selection of suitable locations for the installation of ammunition ijahp article: santos, costa, gomes/multicriteria decision-making in the selection of warships: a new approach to the ahp method international journal of the analytic hierarchy process 149 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.833 deposits. sánchez-lozano et al. (2015) conducted a study for the selection of military training aircraft for the spanish air force through hybrid modeling composed of ahp, topsis and fuzzy logic. according to the authors, this hybrid modeling, mcdm and fuzzy logic, can be applied efficiently to solve decision problems with criteria of a different nature. di bona et al. (2016) proposed an approach based on the integrated factors method (ifm), whose values are adjusted using the ahp method, depending on the importance of each factor and each unit of the system. the reasons that led to the development of ifm-based ahp are the result of careful analysis of current military and commercial approaches. according to the authors, the result is a dynamic model, which combines the advantages of the allocation method and the multicriteria decision-making technique. zhang et al. (2005) proposed a hybrid model composed of fuzzy trapezoidal ahp and fuzzy integral for ordering and evaluating weapon systems. the performance classifications of the criteria are described by linguistic terms expressed in diffuse trapezoidal numbers, while the weights of the criteria are obtained by fuzzy trapezoidal ahp. suharyo et al. (2017) selected the best location for the installation of a military naval base by applying a model that compiles an application of the theory and method of the coverage technique integrated into the fuzzy-ahp model. in order to meet the need for military and commercial approaches, di bona and forcina (2017) implemented the reliability allocation method called analytic critical flow method (acfm), a reliability allocation model for parallel configurations in series based on the failure analysis of each unit of the system. the new approach is based on the critical flow method, the results of which are combined with the ahp method. wang et al. (2008) combined the fuzzy ahp and topsis (technique for order performance by similarity to ideal solution) techniques to evaluate the effectiveness of air combat of military aircraft. in the study, the fuzzy ahp method was used to determine the relative weights of multiple evaluation criteria and synthesize the classifications of candidate aircraft. topsis was employed to get a crisp overall performance value for each alternative to make a final decision. altunok et al. (2010) compared the performance of the ahp, weighted product (wp) and topsis methods to select graduate students from the defense science institute of the turkish military academy. according to the study, the ahp presented the best performance in the proposed analysis. sánchez-lozano and rodríguez (2020) applied the fuzzy reference ideal method (frim) and the ahp method to select the best advanced military training aircraft for the spanish air force. çarman and şakar (2019) conducted a study on the positioning of the surveillance system within a national security project in turkey using the ahp method. sánchez-lozano et al. (2020) conducted a study to prioritize obsolete military coastal batteries, to transform them into places of tourist interest in spain, through the application of the gis, ahp and topsis methods. kiracı and akan (2020) used the interval type-2 fuzzy ahp (it2fahp) and interval type-2 fuzzy (it2ftopsis) methods to choose the most suitable aircraft. şenol (2020) applied the ahp and anp methods to evaluate airworthiness criteria for military aircraft. hamurcu and eren (2020) applied an integrated methodology based on ahp and topsis methods to evaluate unmanned aerial vehicle (uav) alternatives in the selection process. first, the ahp was used to determine the weights of the criteria, while topsis was applied to classify vehicle alternatives in the decision problem. starčević et al. (2019) ijahp article: santos, costa, gomes/multicriteria decision-making in the selection of warships: a new approach to the ahp method international journal of the analytic hierarchy process 150 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.833 selected ground vehicles for the provision of military units intended for multinational operations using the ahp and data envelopment analysis (dea). costa et al. (2021) proposed and applied the electre-mor method to select the most suitable hospital aircraft for the brazilian government to acquire in the fight against the covid-19 pandemic. zhang et al. (2012) applied the topsis method in conjunction with basic probability assignment (bpa) to obtain the classification of the threat of military targets. lu and wang (2011) provided an alternative, integral fuzzy non-additive approach to dealing with fuzzy mcda problems, especially when there is dependence on the criteria considered. the main objective of the article was to discuss how the optimal compensation transaction policy of the industrial cooperation program (icp), a military trade agreement between taiwan and the usa works, through the proposed fuzzy model. gazibey et al. (2015) applied the dematel method to understand the cause-and-effect relationships between the criteria for the selection of main battle tanks. the method was applied in the primary and secondary criteria separately. adetunji et al. (2018) applied the topsis method and monte carlo simulations for risk management for obsolescence in the u.s. armed forces. genc (2015) conducted a study to support decision-making in the acquisition of military tanks through the application of the electre iii and promethee ii methods. costa et al. (2020) applied the thor 2 method to select the brazilian navy's most suitable hospital care vessel (nash) to support the fight against the covid-19 pandemic. tenório et al. (2020) selected a ship for purchase by the bn, from eight ships used by navies around the world through the thor method. in the study, the authors considered "opportunity purchases" of frigates, while in this paper we analyzed options of corvettes to be built by bn. 2.1 bibliometric study on “multicriteria” and “warship” the bibliometric research was carried out through the scopus database using the capes (coordenação de aperfeiçoamento de pessoal de nível superior – brazil) portal during 2020. the terms used were selected by article title, abstract, keywords, document type article, conference paper and article in press, according to table 1. table 1 technical and operational characteristics headings results "multicriteria" and "warship" 37 figure 1 shows the journal articles, conference papers and book chapters published by year. the articles began in 1997; however, a temporary gap is observed between 1998 and 2001, 2004, 2006 and 2017. the years with the highest numbers of published papers were 2010, 2011 and 2020. ijahp article: santos, costa, gomes/multicriteria decision-making in the selection of warships: a new approach to the ahp method international journal of the analytic hierarchy process 151 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.833 figure 1 articles per year source: scopus, 2020 table 2 shows the institutions that published 2 or more articles. the harbin engineering university ranks first with 5 papers. there are 2 institutions that published 3 articles each, 3 institutions with two articles each, and 54 institutions with only one article each, for a total of 60 articles. table 2 most representative institutions institutions articles harbin engineering university 5 naval group 3 aix marseille université 3 university of michigan, ann arbor 2 syracuse university 2 naval university of engineering 2 table 3 presents the distribution of articles by country or territory. the search found articles from 14 countries. china ranks first with 12 articles, followed by the united states with 8 articles. europe is represented by 8 countries with 15 published articles (41% of the total). the american continent, with 2 countries, has 9 articles (24% of the total). asia, with 3 countries, has 15 articles (41% of the total). oceania, represented by australia, has 1 article (3% of the total). ijahp article: santos, costa, gomes/multicriteria decision-making in the selection of warships: a new approach to the ahp method international journal of the analytic hierarchy process 152 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.833 table 3 number of articles by country countries articles china 12 united states 8 france 3 italy 3 united kingdom 3 netherlands 2 taiwan 2 australia 1 brazil 1 bulgaria 1 russia 1 spain 1 sweden 1 turkey 1 table 4 shows that 3 authors stand out with 3 publications, while 5 authors have 2 articles. the others appear with only one publication each. there is no great preference in the area. table 4 number of articles by author authors articles published li, x. 3 siegel, p. 3 toulgoat, i. 3 chowdhury, s. 2 lacroix, y. 2 li, p. 2 messac, a. 2 tong, w. 2 the journal of quality and reliability engineering international stands out with two publications. there are also 35 journals/conferences with 1 article each, for a total of 37 articles. in general, there is no preference for one journal over another, as shown in table 5. ijahp article: santos, costa, gomes/multicriteria decision-making in the selection of warships: a new approach to the ahp method international journal of the analytic hierarchy process 153 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.833 table 5 articles by journals journals articles quality and reliability engineering international 2 european journal of operational research 1 global journal of flexible systems management 1 international journal of applied decision sciences 1 journal of global optimization 1 journal of multi-criteria decision analysis 1 journal of ship production 1 marine technology and sname news 1 naval research logistics 1 smart innovation systems and technologies 1 software quality journal 1 structural and multidisciplinary optimization 1 systems engineering and electronics 1 we emphasize that there have not been any articles on the subject published in the international journal on analytic hierarchy process (ijahp). therefore, this paper represents an opportunity and a real contribution to academia and society, as it represents the employment of the most applied mcdm method to support the decision-making process in a relevant military problem. table 6 shows that the area of engineering stands out with 38.5% of the published articles, followed by computer science (23.1%), decision science (12.3%) and mathematics (12.3%). this analysis identifies research for solving warships’ problems by methods application, not only for demonstrating mathematical schematization theories. table 6 highlighted knowledge areas knowledge area percentage engineering 38.5% computer science 23.1% decision science 12.3% mathematics 12.3% business, management 4.6% materials science 3.1% physics and astronomics 3.1% economics 1.5% environmental sciences 1.5% table 7 shows the applications according to the methods used in decision processes. the ahp is the most used method. ijahp article: santos, costa, gomes/multicriteria decision-making in the selection of warships: a new approach to the ahp method international journal of the analytic hierarchy process 154 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.833 table 7 methods and theories applied in the database method documents percentage accumulated sum analytic hierarchy process (ahp) 2.034 20.00% 20.00% fuzzy logic 1.361 13.38% 33.38% electre family 1.304 12.82% 46.20% gis 1.137 11.18% 57.37% promethee family 1.090 10.72% 68.09% multi-objective optimization 928 9.12% 77.21% topsis 862 8.47% 85.69% simple multi-attribute rating technique (smart) 405 3.98% 89.67% rough set 311 3.06% 92.73% macbeth 182 1.79% 94.51% regime 153 1.50% 96.02% multi-attribute utility theory (maut) 104 1.02% 97.04% borda 60 0.59% 97.63% condorcet 44 0.43% 98.06% todim 43 0.42% 98.49% copeland 36 0.35% 98.84% vip 35 0.34% 99.18% analytic network processes (anp) 32 0.31% 99.50% zapros 28 0.28% 99.77% thor 15 0.15% 99.92% tomaso 8 0.08% 100.00% total 10.172 100.00% 3. analytic hierarchy process (ahp) a problem can be solved in several ways, even if there is only one solution because it is possible to decide whether to carry out the proposed action or not. decision-making may involve simple everyday situations or complex issues that require the use of quantitative and qualitative parameters. a good solution implies a multidimensional vision (gomes et al., 2017; moreira et al., 2021). in this context, the mcdm methods are significant tools for public or private organization managers. mcdm consists of a set of techniques to assist a decision-maker (dm). whether the decision maker is an individual, a group of people, or a technicians’ committee or managers, mcdm aids in making decisions about a complex problem by evaluating and choosing alternatives to solve it using different criteria and points of view (kadziński & tervonen, 2013). the importance of the criteria is defined by the dm in an interactive process with other technical-political actors (almeida-dias et al., 2012). in multiple criteria decisions, problem alternatives are compared pairwise, and the results express the preferences of the dm with the use of comparative notions. ranking, choice ijahp article: santos, costa, gomes/multicriteria decision-making in the selection of warships: a new approach to the ahp method international journal of the analytic hierarchy process 155 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.833 or sorting decisions concerning a finite set of alternatives evaluated on a finite set of criteria are important kinds of problems in many real-world areas of decision-making. the multicriteria decision analysis (mcda) methods have been widely used because they are scientific and subjective, and are able to aggregate all the characteristics that are considered important, including non-quantitative ones, with the enabling transparency purpose and process systematization related to decision-making problems (pinto junior & soares de mello, 2013). multicriteria methods are used for decision-making in several areas. for warships, creating a "hierarchy of alternatives" is no simple task. natalizi (2015) considers this problem because there is a wide variety of technical solutions to carry out the typical features of military ships that cover both platform and combat systems. mcda is highly multidisciplinary and based on a set of matrices or models that will aid the decision process (individual or joint) by considering value judgments and not only technical issues to evaluate alternatives in order to solve real problems (oliveira et al., 2021; santos et al., 2015). these methodologies operate as a basis for discussion, especially in cases where there are conflicts among the dm, or when the problem perception by the various actors involved is not yet fully consolidated regarding the analysis. the ahp, proposed by saaty (1980), is a multicriteria methodology that aims to select or choose the best alternatives through a process that considers different evaluation criteria. according to costa et al. (2016), the ahp method allows the comparison of both quantitative and qualitative criteria. vaidya and kumar (2006) state that the method is considered one of the most well-known and widely disseminated decision-making tools, having the greatest number of applications reported in the literature. the ahp is a comprehensive tool developed for constructing decision models and establishing decision priorities concerning a finite set of alternatives (dong & cooper, 2016). comparisons are made using a scale of absolute judgments (1, 3, 5, 7, 9), as well as intermediate values between the two judgments that represent the relative measure of one alternative over another with respect to a given criterion (dožić & kalić, 2014). according to ali et al. (2017), the main ahp steps include: 1. statement of the goal, decision criteria and alternatives. 2. development of a pairwise comparison matrix. when making criteria and alternative judgments, the expert compares pairwise the elements in the level of hierarchy to each of the elements in the superior level of the hierarchy (saaty, 2008). 3. development of a standardized/normalized matrix. 4. development of a priority vector. 5. computation of the consistency ratio. according to serra costa (2011), even when judgments are obtained from experts, some inconsistency may occur. one way of measuring the intensity or degree of inconsistency in a matrix of pairwise judgments is to ijahp article: santos, costa, gomes/multicriteria decision-making in the selection of warships: a new approach to the ahp method international journal of the analytic hierarchy process 156 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.833 evaluate how the highest eigenvalue of this matrix deviates from the order of the matrix (saaty, 1980). the consistency index (c.i.) can be calculated as shown in (1). 𝑪. 𝑰. = |𝝀𝒎𝒂𝒙−𝒏| 𝒏−𝟏 (1) the gravity of the occurrence of inconsistency can be evaluated by considering its ratio to the average c.i. obtained from a large number of matrices of the same order generated by entering random judgments. this is the consistency ratio (c.r.), and it is used as a parameter to evaluate the inconsistency obtained from the judgment matrix order (saaty, 1980) as shown in (2). 𝐂. 𝐑. = 𝑪.𝑰. 𝐑.𝐈. (2) where r.i.= the consistency index obtained from a large number of randomly generated reciprocal matrices with non-negative elements (brunelli & fedrizzi, 2011). according to vargas (1982), if the calculated c.r. is lower than 0.1, the judgment matrix is considered consistent. 6. development of a priority matrix. after steps 2 through 5 have been performed for each criterion, the results of step 4 are summarized in a priority matrix by listing the decision alternatives vertically and the criteria horizontally. the column entries are the priority vectors for each criterion. 7. development of a criteria pairwise development matrix. 8. development of an overall priority vector. multiplying the criteria priority vector (from step 7) by the priority matrix (from step 6), which may then be used to determine the overall ranking of alternatives (step 8). 9. choosing the alternative with the highest rank. 4. methodology for the development of modeling a problem, it should be noted that many possibilities lead to several applicable models according to saaty and vargas (2013). in this context, considering the hierarchical organizational culture of the bn, the compensatory nature of the problem and that the study presents well-defined characteristics of the alpha type problem, the ahp method was selected from among numerous other multiple-criteria models presented in the bibliometric study by bhutia and phipon (2012). based on vogt (2013), this research followed four steps: 1. interviews with 10 bn officers, with more than 20 years of experience, to obtain the alternatives of ships and criteria to be considered in the proposed analysis, for the choice of the most suitable vessel to be built by the bn, as well as to establish the weights between criteria; ijahp article: santos, costa, gomes/multicriteria decision-making in the selection of warships: a new approach to the ahp method international journal of the analytic hierarchy process 157 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.833 2. creating the decision matrix with all cells filled, where each cell corresponds to a ship alternative and its respective criterion; 3. obtaining the ranking of ships as a result; 4. sensitivity analysis with standard deviation and mean. 4.1 selection of alternatives in 1994, the bn began the construction of the corvette barroso at the navy arsenal of rio de janeiro (narj), a medium-sized vessel weighing 2.500 tons, which was finally launched in 2008, 14 years after the start of construction. this long timespan makes it a new ship, but not a modern ship. today, with the need to build new ships for the brazilian fleet, the bn faces a new challenge, the choice of three project ships, which make up the set of alternatives to be analyzed in this research. model 1. replicate the current corvette barroso; model 2. build a slightly modernized ship (2.600 ton corvette); or model 3. build a model with more significant modernizations (3.000 ton corvette). 4.2 selection of criteria table 8 summarizes the main technical and operational characteristics of the 3 vessels. ijahp article: santos, costa, gomes/multicriteria decision-making in the selection of warships: a new approach to the ahp method international journal of the analytic hierarchy process 158 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.833 table 8 technical and operational characteristics characteristics barroso configuration 1 cv -2600 configuration 2 cv-3000 configuration 3 l over-all(m) 103.4 115.00 118.00 l water-line(m) 96.30 105.00 108.00 b water-line(m) 11.40 13.00 13.50 b max(m) flare 7.5° :12.70 15.00 15.50 t(m) 4.00 4.00 4.30 d(m) 6.75 8.70 9.00 l/b(wl) 8.45 8.08 8.00 lwl/d 14.27 12.07 12.00 t/d 0.59 0.49 0.48 cb 0.53 0.48 0.48 cp long. 0.67 0.622 0.622 lightweight 1710 1.815 2.030 lightweight + res.proj(ton) 1.813 1.924 2.152 dwt(tons) 418 589 728 maximum displacement(tons) 2.231 2.513 2.880 max. displacement + sla(tons) 2.388 2.690 3.085 b/d 1.69 1.494 1.500 s water plane(m2) 1.029 1.057 1.133 master section(m2) 36.12 40.14 44.82 gm(m) 1.70 1.68 1.68 t roll(sec) 6.97 7.99 8.3 t pich(sec) 5.19 5.40 5.48 s wet area(m2) 1.215 1.348 1.463 propulsion(mode) codog/codad codad/codoe codad/codoe pb max(mw) speed 15kts: 1.994 22.03 23.79 action radius(n.m.)/15 4.000 9.330 10.660 action radius(n.m.)/18 xxx 7.070 8.011 fuel endurance(days)/15 11 26 30 fuel endurance(days)/18 xxx 16 19 autonomy(days) 30 25 35 crew(people) 150 100+20 100+20 maximum speed(knot) 27 28 28 slrvnax 1.415 1.405 1.386 froude number 0.452 0.449 0.442 propeller(day/rmp/28kts) xxx 3.50 m/5 blades/285 rpm 3.50 m/ 5 blades/ 292 rpm electrical generation(kw) 2.600 3.240kw + 408 kw 3.240 kw + 408 kw primary cannon ba e 114 mm mk oto melara76mmsp oto melara76mmsp secondary cannon bofors40mm mk3 2x bofors40mm mk-4 2x bofors40mm mk4 ijahp article: santos, costa, gomes/multicriteria decision-making in the selection of warships: a new approach to the ahp method international journal of the analytic hierarchy process 159 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.833 characteristics barroso configuration 1 cv -2600 configuration 2 cv-3000 configuration 3 machine guns asuw missiles 8x exocet ssm40 bi 3 8x exocet ssm40 bi 3 8x exocet ssm40 bi 3 aaw missiles asw torpedoes 2x iii raytheon mk-46 2x iii raytheon mk46 2x iii raytheon mk46 helicopter 1x lynx / aw 159 1x lynx / aw 159 1x lynx / aw 159 initial cost to obtain us$ 290 million us$ 310 million us$ 310 million life cycle 35 years us$ 592 million us$ 633 million us$ 633 million first class commissioning time 6 years 8 years 8 years source: vogt, 2013 from the interviews with the experts (bn officers) and analysis of the operational and technical data of the vessels (table 8), the following criteria were chosen for the choice of a vessel: • action radius (c1): greatest distance (in nautical miles) the ship can travel from its base and return without refueling. • fuel endurance (c2): time interval (in days) that a ship can navigate without refueling with speed at 15 knots. • autonomy (c3): maximum interval of time (in days) that a ship can operate without any type of supplies (fuel, drinking water, foodstuff, etc.). • primary cannon (c4): a weapon with a high rate of fire that functions to warn or neutralize possible threats, such as ships, aircraft or missiles. it is called "primary" when the ship has other alternative guns, usually of smaller caliber. • secondary cannon (c5): an alternative cannon to the "primary cannon", usually of smaller caliber. • aaw missiles (c6): anti-aircraft warfare missiles. • initial cost (c7): cost of obtaining or building a ship. • life cycle cost (c8): life cycle cost of a ship, includes the purchase (or construction), operation and modernization. the purchase price represents about 25%, the expenses for crew and operations account for 67%, and the possible modernization corresponds to 5% to 8%. • construction time (c9): criterion is self-explanatory, considering from the start of the project to the actual delivery of the ship to the operating sector. ijahp article: santos, costa, gomes/multicriteria decision-making in the selection of warships: a new approach to the ahp method international journal of the analytic hierarchy process 160 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.833 5. application of the ahp method 5.1 decision matrix table 9 presents the decision matrix. table 9 decision matrix model 1 model 2 model 3 action radius (c1) 4000 9330 10660 fuel endurance (c2) 11 26 30 autonomy (c3) 30 25 35 primary cannon (c4) 25 25 120 secondary cannon (c5) 1 2 2 aaw missiles (c6) 0 1 1 initial cost (c7) r$ 290,000,000 r$ 310,000,000 r$ 310,000,000 life cycle cost (c8) r$ 592,000,000 r$ 633,000,000 r$ 633,000,000 construction time (c9) 6 years 8 years 8 years for the criteria to be compared in parity, all the values in the decision matrix were normalized within each criterion. in this paper, we used a weighted average to normalize the values (table 10). table 10 normalized decision matrix model 1 model 2 model 3 sum action radius (c1) 0.1667 0.3889 0.4443 1 fuel endurance (c2) 0.1641 0.388 0.4477 1 autonomy (c3) 0.3333 0.2777 0.3888 1 primary cannon (c4) 0.147 0.147 0.7058 1 secondary cannon (c5) 0.2 0.4 0.4 1 aaw missiles (c6) 0 0.5 0.5 1 initial cost (c7) 0.3483 0.3258 0.3258 1 life cycle cost (c8) 0.3483 0.3258 0.3258 1 construction time (c9) 0.4 0.3 0.3 1 5.2 parity comparison between criteria and weighting matrix the nine criteria were compared two by two, through an interview with the specialists. the purpose of the interview was to list the pertinent criteria for the choice of the most appropriate vessel for the brazilian navy's needs, as well as to establish the inter-criteria weights considering saaty’s fundamental scale (saaty, 1980). from the weights listed above, the following matrix was created (table 11). ijahp article: santos, costa, gomes/multicriteria decision-making in the selection of warships: a new approach to the ahp method international journal of the analytic hierarchy process 161 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.833 table 11 weight matrix c1 c2 c3 c4 c5 c6 c7 c8 c9 c1 1 1 1 0.33 5 1 0.33 0.33 0.25 c2 1 1 1 0.33 5 1 0.33 0.33 0.25 c3 1 1 1 0.33 5 1 0.33 0.33 0.25 c4 3 3 3 1 3 1 0.33 0.33 0.33 c5 0.20 0.2 0.2 0.33 1 0.2 0.2 0.2 0.14 c6 1 1 1 1 5 1 0.33 0.33 0.33 c7 3 3 3 3 5 3 1 1 1 c8 3 3 3 3 5 3 1 1 1 c9 4 4 4 3 7 3 1 1 1 by applying the same normalization procedure in table 10, the normalized weight matrix is obtained (table 12). table 12 normalized weight matrix c1 c2 c3 c4 c5 c6 c7 c8 c9 c1 0.06 0.06 0.06 0.03 0.12 0.07 0.07 0.07 0.05 c2 0.06 0.06 0.06 0.03 0.12 0.07 0.07 0.07 0.05 c3 0.06 0.06 0.06 0.03 0.12 0.07 0.07 0.07 0.05 c4 0.17 0.17 0.17 0.08 0.07 0.07 0.07 0.07 0.07 c5 0.01 0.01 0.01 0.03 0.02 0.01 0.04 0.04 0.03 c6 0.06 0.06 0.06 0.08 0.12 0.07 0.07 0.07 0.07 c7 0.17 0.17 0.17 0.24 0.12 0.21 0.20 0.20 0.22 c8 0.17 0.17 0.17 0.24 0.12 0.21 0.20 0.20 0.22 c9 0.23 0.23 0.23 0.24 0.17 0.21 0.20 0.20 0.22 sum 1 1 1 1 1 1 1 1 1 table 13 shows the priority vector of the criteria. the construction time had greater weight, followed by the initial cost and the life cycle cost. initial cost and life cycle cost had the same values. ijahp article: santos, costa, gomes/multicriteria decision-making in the selection of warships: a new approach to the ahp method international journal of the analytic hierarchy process 162 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.833 table 13 priority vector of criteria 1º construction time 0.217021892 2º initial cost 0.192221993 3º life cycle cost 0.192221993 4º primary cannon 0.106447057 5º aaw missiles 0.073107421 6º action radius 0.065070664 7º fuel endurance 0.065070664 8º autonomy 0.065070664 9º secondary cannon 0.023767652 5.3 results the multiplication of the normalized decision matrix by the respective priority vector of the criteria gives the following ranking (table 14). table 14 ranking of alternatives classification model priorities 1 st option model 3 (totally new model) 0.3949 2 nd option model 2 0.3207 3 rd option model 1 (actual model) 0.2843 the application of the ahp method indicates that the best alternative is the construction of a totally new vessel (model 3), even considering the risks involved. the worst alternative (model 1) would be to replicate the current corvette barroso. this is corroborated by the "score" of the distance of model 3 from model 2 being greater than the distance of model 2 from model 1. 5.4 sensitivity analysis the traditional ahp method did generate a small discrimination in the ranking of alternatives, which may indicate the need for a more careful sensitivity analysis. to increase the discriminatory power, we applied the framework presented in figure 2. ijahp article: santos, costa, gomes/multicriteria decision-making in the selection of warships: a new approach to the ahp method international journal of the analytic hierarchy process 163 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.833 figure 2 steps for sensitivity analysis step 1 has been fulfilled because the decision matrix has already been established (table 8). table 15 shows the values of the mean scores (step 2), standard deviations (step 3) and the gaussian factor (step 4), given by standard deviation/mean ratio. table 15 mean score and standard deviation of each criterion criterion mean scores standard deviation gaussian factor action radius 0.3333 0.1469 0.441 fuel endurance 0.3333 0.1495 0.448 autonomy 0.3333 0.0555 0.167 primary cannon 0.3333 0.3226 0.968 secondary cannon 0.3333 0.1154 0.346 aaw missiles 0.3333 0.2886 0.866 initial cost 0.3333 0.0129 0.039 life cycle cost 0.3333 0.013 0.038 construction time 0.3333 0.0577 0.173 the higher the gaussian factor value, the harder it is to obtain a high score on the criterion. after this calculation, table 16 provides the new weights (gaussian factor values) already normalized from each criterion (steps 5 and 6). ijahp article: santos, costa, gomes/multicriteria decision-making in the selection of warships: a new approach to the ahp method international journal of the analytic hierarchy process 164 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.833 table 16 new normalized weights of each criterion criterion new weights action radius 0.126 fuel endurance 0.129 autonomy 0.048 primary cannon 0.278 secondary cannon 0.099 aaw missiles 0.248 initial cost 0.011 life cycle cost 0.011 construction time 0.050 finally, we arrive at the same ranking (step 7); however, with a more marked trade-off between the alternatives, corroborating that model 3 (totally new) is the most appropriate decision for the bn (table 17). in this approach, model 2 was in an intermediate position between model 3 and model 1. table 17 sensitivity analysis of the ranking of alternatives classification model priorities 1 st option model 3 (totally new) 0.5144 2 nd option model 2 0.3390 3 rd option model 1 (actual model) 0.1465 6. conclusion this paper aimed to choose the best choice for a new vessel to be built for the bn through the application of the ahp method. the definition of the method was made through a bibliometric survey of publications on the multiple-criteria and decision theme, in which we verified that the ahp method is the most used in all areas of knowledge based on scopus database. the method proved to be efficient in the proposed analysis, both in obtaining the weights of the criteria, as well as in the ordering of the models evaluated. it was concluded that the best alternative would be the construction of a new vessel, and this option was endorsed through the sensitivity analysis that followed. the use of weights to classify the variables considered all the possibilities involved, providing more reliability to the decision-making process. in summary, the sensitivity analysis increased the accuracy of the decision. evaluating the reasons that led to the final ranking of the alternatives, we note that model 3, although it did not exhibit the best performances on the criteria with higher weights assigned by the specialists, obtained the best results due to its highest performance in the other six criteria, of operational nature, which makes this ship the most suitable to be built by the bn. ijahp article: santos, costa, gomes/multicriteria decision-making in the selection of warships: a new approach to the ahp method international journal of the analytic hierarchy process 165 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.833 moreover, we emphasize the ease, flexibility, reliability and speed of application of the sensitivity analysis, which allows the weights to be obtained and alternatives to be ordered through the gaussian factor, without the need to apply pairwise comparisons of alternatives and criteria. finally, we suggest this model of ordering alternatives using the ahp and gaussian factor to be further applied in tactical, operational and strategic applications in the military area, given that this type of problem greatly affects the sovereignty and public safety of nations around the world. ijahp article: santos, costa, gomes/multicriteria decision-making in the selection of warships: a new approach to the ahp method international journal of the analytic hierarchy process 166 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.833 references adetunji, o., bischoff, j., & willy, c. j. (2018). managing system obsolescence via multicriteria decision making. systems engineering, 21(4), 307–321. doi: https://doi.org/10.1002/sys.21436 ali, y., asghar, a., muhammad, n., & salman, a. (2017). selection of a fighter aircraft to improve the effectiveness of air combat in the war on terror: pakistan air force-a case in point. international journal of the analytic hierarchy process, 9(2), 244-273. doi: https://doi.org/10.13033/ijahp.v9i2.489 almeida-dias, j., figueira, j. r., & roy, b. (2012). a multiple criteria sorting method where each category is characterized by several reference actions: the electre tri-nc method. european journal of operational research, 217(3), 567–579. doi: https://doi.org/10.1016/j.ejor.2011.09.047 altunok, t., özpeynirci, ö., kazançoǧlu, y., & yilmaz, r. (2010). comparative analysis of multi-criteria decision making methods for postgraduate student selection. egitim arastirmalari eurasian journal of educational research, 40, 1–15. https://www.scopus.com/inward/record.uri?eid=2-s2.077957593455&partnerid=40&md5=4b52454f0e8341c8cba15c87e408dc49 bhutia, p. w., & phipon, r. (2012). application of ahp and topsis method for supplier selection problem. iosr journal of engineering, 2(10), 43–50. doi: https://doi.org/10.9790/3021-021034350 bisdikian, c., kaplan, l. m., & srivastava, m. b. (2013). on the quality and value of information in sensor networks. acm transactions on sensor networks, 9(4). doi: https://doi.org/10.1145/2489253.2489265 brunelli, m., & fedrizzi, m. (2011). characterizing properties for inconsistency indices in the ahp. proceedings of the international symposium on the analytic hierarchy process, sorrento, italy. doi: https://doi.org/10.13033/isahp.y2011.049 çarman, f., & tuncer şakar, c. (2019). an mcdm-integrated maximum coverage approach for positioning of military surveillance systems. journal of the operational research society, 70(1), 162–176. doi: https://doi.org/10.1080/01605682.2018.1442651 costa, i. p. de a., maêda, s. m. do n., teixeira, l. f. h. de s. de b., gomes, c. f. s., & santos, m. dos. (2020). choosing a hospital assistance ship to fight the covid-19 pandemic. revista de saude publica, 54. doi: https://doi.org/10.11606/s15188787.2020054002792 costa, i. p. de a., sanseverino, a. m., barcelos, m. r. dos s., belderrain, m. c. n., gomes, c. f. s., & santos, m. dos. (2020). choosing flying hospitals in the fight against the covid-19 pandemic: structuring and modeling a complex problem using the vft and electre-mor methods. ieee latin america transactions, 100(1e). costa, j. f. s., borges, a. r., & dos santos machado, t. (2016). analytic hierarchy process applied to industrial location: a brazilian perspective on jeans manufacturing. ijahp article: santos, costa, gomes/multicriteria decision-making in the selection of warships: a new approach to the ahp method international journal of the analytic hierarchy process 167 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.833 international journal of the analytic hierarchy process, 8(1), 77-91. doi: https://doi.org/10.13033/ijahp.v8i1.210 da serra costa, j. f. (2011). a genetic algorithm to obtain consistency in analytic hierarchy process. brazilian journal of operations & production management, 8(1), 55– 64. doi: https://doi.org/10.4322/bjopm.2011.003 di bona, g., & forcina, a. (2017). analytic critical flow method (acfm): a reliability allocation method based on analytic hierarchy process. journal of failure analysis and prevention, 17(6), 1149–1163. doi: https://doi.org/10.1007/s11668-017-0353-9 di bona, g., forcina, a., petrillo, a., de felice, f., & silvestri, a. (2016). a-ifm reliability allocation model based on multicriteria approach. international journal of quality and reliability management, 33(5), 676–698. doi: https://doi.org/10.1108/ijqrm-05-2015-0082 dong, q., & cooper, o. (2016). a peer-to-peer dynamic adaptive consensus reaching model for the group ahp decision making. european journal of operational research, 250(2), 521–530. doi: https://doi.org/10.1016/j.ejor.2015.09.016 dožić, s., & kalić, m. (2014). an ahp approach to aircraft selection process. transportation research procedia, 3, 165–174. doi: https://doi.org/10.1016/j.trpro.2014.10.102 gazibey, y., kantemir, o., & demirel, a. (2015). interaction among the criteria affecting main battle tank selection: an analysis with dematel method. defence science journal, 65(5), 345–355. doi: https://doi.org/10.14429/dsj.65.8924 genc, t. (2015). application of electre iii and promethee ii in evaluating the military tanks. international journal of procurement management, 8(4), 457–475. doi: https://doi.org/10.1504/ijpm.2015.070743 gigović, l., pamuĉar, d., bajić, z., & milićević, m. (2016). the combination of expert judgment and gis-mairca analysis for the selection of sites for ammunition depots. sustainability (switzerland), 8(4). doi: https://doi.org/10.3390/su8040372 gomes, c. f. s., costa, h. g., & de barros, a. p. (2017). sensibility analysis of mcda using prospective in brazilian energy sector. journal of modelling in management, 12(3), 475-497. doi: https://doi.org/10.1108/jm2-01-2016-0005 hamurcu, m., & eren, t. (2020). selection of unmanned aerial vehicles by using multicriteria decision-making for defence. journal of mathematics, 2020. doi: https://doi.org/10.1155/2020/4308756 kadziński, m., & tervonen, t. (2013). robust multi-criteria ranking with additive value models and holistic pair-wise preference statements. european journal of operational research, 228(1), 169–180. doi: https://doi.org/10.1016/j.ejor.2013.01.022 kiracı, k., & akan, e. (2020). aircraft selection by applying ahp and topsis in interval type-2 fuzzy sets. journal of air transport management, 89. doi: ijahp article: santos, costa, gomes/multicriteria decision-making in the selection of warships: a new approach to the ahp method international journal of the analytic hierarchy process 168 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.833 https://doi.org/10.1016/j.jairtraman.2020.101924 lu, w.-m., & wang, t.-c. (2011). a fuzzy multi-criteria model for the industrial cooperation program transaction strategies: a case in taiwan. expert systems with applications, 38(3), 1490–1500. https://doi.org/10.1016/j.eswa.2010.07.060 natalizi, r. a. (2015). análise de custos e benefícios na escolha da configuração de navios. revista da escola de guerra naval, 353–380. pinto junior, r. p. da s., & soares de mello, j. c. c. b. (2013). identification of the best selection of employees for the accomplishment of inspection services in the state electric sector. production, 23(1), 135–143. saaty, t l. (1980). the analytic hierarchy process: planning, priority setting, resource allocation. new york, ny: mcgraw-hill. saaty, thomas l. (2008). relative measurement and its generalization in decision making why pairwise comparisons are central in mathematics for the measurement of intangible factors the analytic hierarchy/network process. racsam-revista de la real academia de ciencias exactas, fisicas y naturales. serie a. matematicas, 102(2), 251– 318. doi: https://doi.org/10.1007/bf03191825 saaty, thomas l, & vargas, l. g. (2013). the logic of priorities: applications of business, energy, health and transportation. springer science & business media. sánchez-lozano, j. m., & rodríguez, o. n. (2020). application of fuzzy reference ideal method (frim) to the military advanced training aircraft selection. applied soft computing journal, 88. doi: https://doi.org/10.1016/j.asoc.2020.106061 sánchez-lozano, j. m., salmerón-vera, f. j., & ros-casajús, c. (2020). prioritization of cartagena coastal military batteries to transform them into scientific, tourist and cultural places of interest: a gis-mcdm approach. sustainability (switzerland), 12(23), 1–16. doi: https://doi.org/10.3390/su12239908 sánchez-lozano, j. m., serna, j., & dolón-payán, a. (2015). evaluating military training aircrafts through the combination of multi-criteria decision making processes with fuzzy logic. a case study in the spanish air force academy. aerospace science and technology, 42, 58–65. doi: https://doi.org/10.1016/j.ast.2014.12.028 santos, m., quintal, r. s., paixão, a. c. da, & gomes, c. f. s. (2015). simulation of operation of an integrated information for emergency pre-hospital care in rio de janeiro municipality. procedia computer science, 55, 931–938. doi: https://doi.org/10.1016/j.procs.2015.07.111 şenol, m. b. (2020). evaluation and prioritization of technical and operational airworthiness factors for flight safety. aircraft engineering and aerospace technology, 92(7), 1049–1061. doi: https://doi.org/10.1108/aeat-03-2020-0058 starčević, s., bojović, n., junevičius, r., & skrickij, v. (2019). analytical hierarchy process method and data envelopment analysis application in terrain vehicle selection. ijahp article: santos, costa, gomes/multicriteria decision-making in the selection of warships: a new approach to the ahp method international journal of the analytic hierarchy process 169 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.833 transport, 34(5), 600–616. doi: https://doi.org/10.3846/transport.2019.11710 suharyo, o. s., manfaat, d., & armono, h. d. (2017). establishing the location of naval base using fuzzy mcdm and covering technique methods: a case study. international journal of operations and quantitative management, 23(1), 61–87. https://www.scopus.com/inward/record.uri?eid=2-s2.085014439577&partnerid=40&md5=f46baba391bd10544ce5f43d4b1f7750 tenório, f. m., dos santos, m., gomes, c. f. s., & araujo, j. de c. (2020). navy warship selection and multicriteria analysis: the thor method supporting decision making. international joint conference on industrial engineering and operations management, 27–39. springer. doi: https://doi.org/10.1007/978-3-030-56920-4_3 vaidya, o. s., & kumar, s. (2006). analytic hierarchy process: an overview of applications. european journal of operational research, 169(1), 1–29. vargas, l. g. (1982). reciprocal matrices with random coefficients. mathematical modelling, 3(1), 69–81. vogt, r. (2013). corvetas sucessoras da barroso: comparação de dois tipos de obtenção. revista marítima brasileira, rio de janeiro, 133(04/06), 108–129. wang, j., fan, k., su, y., liang, s., & wang, w. (2008). air combat effectiveness assessment of military aircraft using a fuzzy ahp and topsis methodology, 655–662. doi: https://doi.org/10.1109/asc-icsc.2008.4675442 zhang, c., ma, c.-b., & xu, j.-d. (2005). a new fuzzy mcdm method based on trapezoidal fuzzy ahp and hierarchical fuzzy integral (w. l. & j. y. (eds.); vol. 3614, issue part ii, pp. 466–474). doi: https://www.scopus.com/inward/record.uri?eid=2s2.0-26944467567&partnerid=40&md5=42c5462050765996ac66dd621ccbf823 zhang, h., kang, b., li, y., zhang, y., & deng, y. (2012). target threat assessment based on interval data fusion. journal of computational information systems, 8(6), 2609– 2616. doi: https://www.scopus.com/inward/record.uri?eid=2-s2.084861421837&partnerid=40&md5=b3bda65a8b9244eff9a6bc3e81f78941 ijahp news and events: slowinski/2022 mcdm junior researcher best paper award international journal of the analytic hierarchy process 1 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.1068 2022 mcdm junior researcher best paper award roman słowiński 2022 jury chair poznań university of technology, poland mohammad ghaderi, pompeu fabra university, spain (right) with roman słowiński, the 2022 jury chair, poznań university of technology, poland (left) was selected as the winner by the jury the mcdm junior research best paper award is bestowed by the mcdm section of informs. it recognizes outstanding papers in the field of mcdm written by a junior researcher or a team of junior researchers. the judging criteria for this award are:  novelty in theoretical, methodological, or applied research in mcdm (50%),  relevance to real-life applications (30%),  potential impact of the obtained results on society (20%). ijahp news and events: slowinski/2022 mcdm junior researcher best paper award international journal of the analytic hierarchy process 2 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.1068 this award was first given in 2018. the rules in the 2022 edition state (https://connect.informs.org/multiple-criteria-decision-making/awards/new-item2):  a candidate must be the single author or the leading coauthor of an article. a team of candidates must consist of the leading coauthors of an article.  the article must be written in the english language and published electronically in a peer-reviewed journal between january 1, 2019 and january 1, 2022.  all candidates must be within seven years of graduation from their ph.d. program by january 1, 2022.  all candidates must be members of the mcdm section of informs.  all candidates must be nominated by a mcdm section member.  executive board members of informs mcdm will serve as the judges.  up to five finalists will present their papers in the award session.  the winner will receive a cash award of $500, while all finalists who are not selected as the winner will receive an honorable mention. the following ten researchers were award candidates in 2022: leading junior author nominating mcdm member merve bodur serpil sayin mohammad dehghanimohammadabadi violeta cvetkoska sune lauth gadegaard serpil sayin mohammad ghaderi núria agell marc juanpera mohammad dehghani jiapeng liu sarah ben amor seyedabdolhossein (abdi) mehvar lisa scholten lucia roselli danielle costa morais eleftherios siskos salvatore corrente michal tomczyk jyrki wallenius https://connect.informs.org/multiple-criteria-decision-making/awards/new-item2 ijahp news and events: slowinski/2022 mcdm junior researcher best paper award international journal of the analytic hierarchy process 3 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.1068 the 2022 jury was composed of the following members of the executive board: cristiano cavalcante, universidade federal de pernambuco marco cinelli, leiden university salvatore greco, university of catania ozlem karsu, bilkent university caroline mota, universidade federal de pernambuco firdevs ulus, bilkent university roman słowiński, poznań university of technology (chair) the jury carefully evaluated all ten papers and decided to invite the following five finalists to a special session of the mcdm section at the 2022 informs meeting in indianapolis: leading junior author article reference merve bodur university of toronto canada david bergman, merve bodur, carlos cardonha, andre a. cire (2021): network models for multiobjective discrete optimization. informs journal on computing 34(2): 990-1005. sune lauth gadegaard aarhus university denmark sune lauth gadegaard, lars relund nielsen, matthias ehrgott (2019): bi-objective branch-and-cut algorithms based on lp relaxation and bound sets. informs journal on computing 31(4): 790-804. mohammad ghaderi pompeu fabra university, spain mohammad ghaderi, miłosz kadziński (2021): incorporating uncovered structural patterns in value functions construction. omega 99: 102203. jiapeng liu xi’an jiaotong university, china jiapeng liu, miłosz kadziński, xiuwu liao, xiaoxin mao (2021): data-driven preference learning methods for valuedriven multiple criteria sorting with interacting criteria. informs journal on computing 33(2): 586-606. eleftherios siskos paul scherrer institut switzerland eleftherios siskos, peter burgherr (2022): multicriteria decision support for the evaluation of electricity supply resilience: exploration of interacting criteria. european journal of operational research 298: 611-626. ijahp news and events: slowinski/2022 mcdm junior researcher best paper award international journal of the analytic hierarchy process 4 vol. 14 issue 3 2022 issn 1936-6744 https://doi.org/10.13033/ijahp.v14i3.1068 finalists with roman słowiński, the 2022 jury chair, poznań university of technology (on left), from left to right, sune lauth gadegaard, aarhus university, denmark; eleftherios siskos, paul scherrer institut switzerland; merve bodur, university of toronto, canada; and mohammad ghaderi pompeu fabra university, spain (the winner). jiapeng liu, xi’an jiaotong university, china could not attend the meeting. all five finalists received a diploma at the business meeting of the mcdm section on october 16, 2022. the winner selected by the jury was mohammad ghaderi, pompeu fabra university, spain. this was a great success for the mcdm/a community to receive ten nominations and to have five finalists for this award! international journal of analytic hierarchy process ijahp article: dali, abdullah, islam/prioritization of the indicators and sub-indicators of maqasid al-shariah in measuring liveability of cities international journal of the analytic hierarchy process 348 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.597 prioritization of the indicators and subindicators of maqasid al-shariah in measuring liveability of cities norimah md dali 1 and alias abdullah 2 department of urban and regional planning kulliyyah of architecture and environmental design international islamic university malaysia jalan gombak, 53100 kuala lumpur 1 nmddali@gmail.com, 2 dralias@iium.edu.my rafikul islam 1 department of business administration kulliyyah of economics and management sciences international islamic university malaysia jalan gombak, 53100 kuala lumpur rislam@iium.edu.my abstract opinions were sought from a panel of two groups of malaysian experts, i.e., the urban planners and the maqasid al-shariah scholars with the aim of developing an evaluation model via identifying and ranking the maqasid indicators and subindicators for liveability and quality of life in cities. the measurement utilizes the dharuriyyat (essentials or necessities) dimension of the maqasid al-shariah principles based on al-shatibi’s school of maslahah which targets public interests and benefits of living in cities. this is supported by ibn ashur and contemporarily by yusuf al-qaradhawi who emphasize harmony, justice and global peace. the analytic hierarchy process (ahp) has been used as the main method to prioritize the indicators and sub-indicators. the ahp results indicate that religion, life, intellect, lineage and wealth are in descending order of importance, similar to the priorities of the classic maqasid al-shariah doctrine. however, the sub-indicators are ranked in terms of priorities based on the consensus of the urban planners and maqasid practitioners which ultimately form the islamic liveability measurement for cities. keywords: maqasid al-shari’ah; liveability of cities; malaysia; ahp 1. introduction a review of the existing human well-being measurement vis-a-vis liveability indicators, at the world level and the malaysian level reveals five common themes, namely (i) politics and governance, (ii) economics, (iii) social and culture, (iv) 1 acknowledgement: the research was financially supported by the myra incentive research grant scheme (mirgs) 2013 (project id: mirgs 13-02-001-0005) ministry of education, malaysia. we would like to thank the government of malaysia, particularly, the ministry of education for supporting this research. the authors are also thankful to the reviewers for their insightful comments. mailto:nmddali@gmail.com mailto:rislam@iium.edu.my ijahp article: dali, abdullah, islam/prioritization of the indicators and sub-indicators of maqasid al-shariah in measuring liveability of cities international journal of the analytic hierarchy process 349 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.597 environment and (v) infrastructure (sarkawi et al., 2015). this is evident in the global liveability index by the economist intelligence unit (eiu), the quality of living index by mercer (global human resources consulting firm) and the most liveable city index by monocle (culture and lifestyle magazine). the review also revealed the missing indicators of religion and lineage within the western indicators, while the malaysian indicators appear to show inconsistency and lack of detail about religious indicators in the measurement. however, in the former, terms like religious restrictions and freedom of opinion were used as if connoting religion. for the malaysian indicators, sporadic religious indicators are used, but not specifically with the idea of fulfilling the objectives of islamic law. because of these gaps, this paper aims to streamline the human well-being indicators and sub-indicators in the context of maqasid al-shariah (objectives of islamic law) especially in preserving and safeguarding the five essentials of religion (faith), self (life), education (intellect), social (lineage) and economy (wealth) as highlighted by auda (2008). these five prerequisites of human well-being should be safeguarded in order to render cities liveable. 1.1 conventional liveability indicators and sub-indicators at the world level, conventional or western liveability indicators and sub-indicators that are currently used to rank cities as a ‘world’s most liveable city’ are represented by eiu, mercer and monocle. at the local level, in this case the malaysian level, there are some liveability, quality of life and sustainability indices that have been formulated by several government agencies. a review of the indicators and sub-indicators at both of these levels serve as a useful precursor to more in-depth study on the missing indicators or gaps of measuring liveability of cities. this study, therefore, points to the need to expand the scope beyond the conventional sphere by examining islamic perspectives of living which are guided by the safeguarding of indicators and sub-indicators that fulfill the maqasid al-shariah. in other words, liveability is the ability to protect one’s faith, life, intellect, lineage and wealth. 1.1.1 three world organization’s liveability indicators the conventional liveability branding championed by three world organizations, i.e., eiu (economist intelligence unit), mercer and monocle is typically concerned with western values as evidenced by the indicators and sub-indicators used by them (steep, 2013). they seem to be inclusive and have similar categories and themes, namely political, social, economic, culture, environment, education and infrastructure. these indicators are being used to rate the liveability of cities and rank the cities of the world as “most liveable cities” for expatriates, businessmen and managers. since the indicators are broad, they can also be applicable to the urban residents living in those cities. this study emphasizes that liveability should be concerned with the very people living in the cities not with what the conventional studies have focused on. broadly, the indicators and categories of the three organizations clearly reflect a certain degree of similarity. despite their minor differences, the indicators could fall under the main categories of stability, health, culture and environment, education and infrastructure (see table 1). ijahp article: dali, abdullah, islam/prioritization of the indicators and sub-indicators of maqasid al-shariah in measuring liveability of cities international journal of the analytic hierarchy process 350 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.597 table 1 similarities of the three world organization’s indicators sources: 1) https://www.imercer.com/content/mobility/quality-of-living-city-rankings.html; 2) https://monocle.com/film/affairs/top-25-cities-201; 3)http://www.news.com.au/lifestyle/reallife/news-life/monocles-2015-quality-oflife-survey hence, looking through the indicators of the three organizations, as far as political stability, economic environment, socio-cultural environment, health, education, housing, public transportation and infrastructure are concerned, they are all important and applicable to local residents as well as expatriates. perhaps indicators such as climate/sunshine, natural environment, urban design, quality of architecture and tolerance are criteria meant more for consideration by expatriates and managers who are to be transferred to cities. these are criteria they can consider and decide whether it is worthwhile for them to take up a post in those cities. if they consider the conditions tolerable enough and the compensation fees given by their companies commensurate enough for their affected quality of life, then they might consider living in those cities. on the other hand, for the people who are part of the native city population these criteria are secondary and not of prime importance. in fact, what really matters for them is the fulfillment of their basic needs and livelihood like availability of social facilities, political stability, housing, employment, infrastructure and good public transportation. 1.1.2 malaysian liveability indicators at the malaysian level, there are several sets of indicators that measure quality of life vis-à-vis liveability of cities. “cities” in the malaysian context are cities that have official city status as declared by the ministry of well-being, housing and local government. compared to world standards, cities in malaysia are relatively small in size since the top three cities and towns in malaysia by population based on the 2010 population and housing census were just barely above the one million mark (department of statistics malaysia, 2014). they are as follows: kuala lumpur (kl) 1,700,750 petaling jaya (pj) 1,812,633 johor bahru (jb) 1,386,569 however, because it is projected that by 2025 the urban population will increase from eiu’s global liveability index (30 indicators 5 categories) mercer’s quality of living index (39 indicators 10 categories) monocle’s most liveable city index (11 indicators) 1. stability 1.political and social environment 1. safety / crime 2. healthcare 2. medical and health considerations 2.medical care 3. culture and environment 3. socio-cultural environment 3.environmental issues and access to nature 4. tolerance 5. urban design 6. quality of architecture 4. education 4. schools and education not mentioned 5. infrastructure not mentioned 7. international connectivity 5.public services & transport 8. public transportation not categorized 6.economic environment 9. business conditions not categorized 7.consumer goods 10. pro-active policy development not categorized 8.recreation 11. climate / sunshine 9.natural environment not categorized 10.housing not mentioned https://www.imercer.com/content/mobility/quality-of-living-city-rankings.html https://monocle.com/film/affairs/top-25-cities-201 http://www.news.com.au/lifestyle/real-life/news-life/monocles-2015-quality-of-%20life-survey http://www.news.com.au/lifestyle/real-life/news-life/monocles-2015-quality-of-%20life-survey ijahp article: dali, abdullah, islam/prioritization of the indicators and sub-indicators of maqasid al-shariah in measuring liveability of cities international journal of the analytic hierarchy process 351 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.597 20.29 million to 27.30 million, cities like the above and other major towns are being flocked to by the urban populations, hence questions on liveability have become the central agenda for the government and the local authorities (national urbanisation policy 2, 2016-2025). in fact, since 1999, six government agencies have focused their attention on monitoring quality of life and sustainability of malaysian cities. they have produced their respective well-being studies as follows: 1. malaysian wellbeing index (mwi report, 2013) by the economic planning unit (epu). http://www.epu.gov.my/ 2. malaysian urban-rural national indicators network for sustainable development (murninets, 2012) by the federal town and country planning department (officially known as jpbd or ftcpd, now rebranded as malaysiaplan) https://murninet.townplan.gov.my/ 3. malaysian ummah development index (mudi report, 2014) institute of understanding islam (officially, ikim) http://www.ikim.gov.my/index.php/en/ 4. malaysian family wellbeing index (mfwi, 2011) by the national family and population development board (officially, lppkn) http://www.lppkn.gov.my/index.php/en/population-services/110-kajian-indekskesejahteraan-keluarga-malaysia-2011.html 5. muslim religiosity and personality indexing: implications for nation building, 2006. (mrpi), by institut pengajian komuniti dan keamanan (pekka), now institute of social studies, universiti putra malaysia (upm). 6. malaysian syari’ah index (msi, 2015) by the department of islamic development malaysia (officially, jakim) http://www.islam.gov.my/en for comparison purposes with the western indicators, three malaysian studies qualify to be examined, and they are the mwi, mfwi and murninets. like their western counterparts, there are five common themes that these three studies have emphasized namely politics and governance, economics, social, environment and infrastructure. this is shown in table 2. those sub-indicators in bold highlight the emphasis of the respective agencies which are missing in the western indicators. for example, the mwi emphasizes public safety and social participation in addition to the other standard of social facilities that cities ought to provide. the mfwi is concerned with family safety, and the role of religion and spiritual practice for family well-being. lastly, the murninets, which is a measuring tool to measure sustainability level of local authorities as represented by their respective cities or towns, stresses the overall planning objective of ensuring quality of life. http://www.epu.gov.my/ https://murninet.townplan.gov.my/ http://www.ikim.gov.my/index.php/en/ http://www.lppkn.gov.my/index.php/en/population-services/110-kajian-indeks-kesejahteraan-keluarga-malaysia-2011.html http://www.lppkn.gov.my/index.php/en/population-services/110-kajian-indeks-kesejahteraan-keluarga-malaysia-2011.html http://www.islam.gov.my/en ijahp article: dali, abdullah, islam/prioritization of the indicators and sub-indicators of maqasid al-shariah in measuring liveability of cities international journal of the analytic hierarchy process 352 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.597 table 2 five common themes (indicators) and their sub-indicators categorically, based on the five general themes, the examination of the indicators and sub-indicators of the respective agencies, three salient features emerge. i) each agency’s indicators are very organizational-biased there are three main purposes for the mwi 2013 of epu. firstly, it aims to complement the measurement of economic development which is normally based on income per capita. secondly, it aims to measure the impact of the government’s socio-economic policies on the quality of life and well-being of the people. lastly, it aims to identify socio-economic issues in order to formulate appropriate policies and strategies for the country’s development. meanwhile, the objective of murninets is predominantly to provide a diagnostic tool for urban managers and local governments to undertake regular performance reviews of the urban sub-sectors and to prepare for the five common themes the malaysian wellbeing index, (mwi), epu the malaysian family wellbeing index (mfwi), lppkn the malaysian urban rural national indicators network on sustainable development (murninets), ftcpd 1.politics and governance -governance -public safety -safety at home -family safety -emergency response knowledge -delivery system -strengthening institutions -enforcement and monitoring -security and safety -municipal development 2.economics -income and distribution -working life -family living standards -family economic situation -future savings -debt burden -economic growth -poverty -private investment 3.social -housing -education -leisure -social participation -culture -family -community cooperation/ relationship/involvement -role of religion -spiritual practice -parental involvement -family resilience -family functioning -time with family -work-family balance -husband/wife relationship -parental relationship -family health practice -stress management -residential -quality of life -demography 4.environment -health -environment -pollution level -family health level -changes in land use -heritage preservation, agriculture and tourism -environmental quality -risk management -environmental management 5.infrastructure -transport -communica tions -basic amenities -utility efficiency -solid waste& sewerage management -transportation -community facilities ijahp article: dali, abdullah, islam/prioritization of the indicators and sub-indicators of maqasid al-shariah in measuring liveability of cities international journal of the analytic hierarchy process 353 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.597 budgeting for urban service purposes. the objectives of the mfwi are solely to measure family well-being, to describe the state of family well-being based on a set of indicators developed and to propose recommendations to improve family wellbeing. to update the family well-being situation, lppkn has conducted another fresh survey to review the 2011 mfwi study in 2016. ii) each agency has different objectives epu is concerned about socio-economic well-being, thus data sought to satisfy the 14 components that support the various socio-economic indicators. on the other hand, for the ftcpd, which is a department that is responsible for preparing the national physical plan, structure plans, local plans and special area plans data pertaining to planning for human well-being, liveability of cities and quality of life for both the urban as well as the rural areas need to be collected for the database for planning and development (town and country planning act, act 172, 1976). in devising the murninets, up-to-date data for the respective dimensions, themes and indicators needs to be keyed-in to come up with the sustainability level of cities i.e. 80 % and above is considered as sustainable, 50% to 80% moderately sustainable and scores below 50% are less sustainable (http://murninet.townplan.gov.my/). meanwhile, the lppkn is seen to update its 7 dimensions and 24 indicators of the mfwi through its recent survey exercise (2016). however, while the data collection process seems to overlap, it is found that the three sets of indicators obtained complement each other. this is because each agency’s scope and function though quite distinct are useful when coordinated with other agencies. notwithstanding, epu looks at the population at the macro scale; the ftcpd focuses on physical planning as stipulated by the town and country planning act of 1976 and the lppkn zooms into the family context. however, upon examining all of the three agencies indicators and sub-indicators, five common themes emerged as far as quality of life is concerned. they are politics, economics, society, environment and infrastructure. noticeably, some liveability indicators that are supposed to specifically relate to the maqasid al-shariah indicators are not extensively detailed in this list. for instance, referring back to table 2 (see bold text), epu mentions public safety in passing as well as social participation; in the mfwi, there are things like family safety, role of religion and spiritual practice; and in the murninets, there is an indicator tagged as quality of life. in fact, all these indicators imply protection of religion, life, family and community harmony which the maqasid al-shariah espouses. iii) well-being policy making based on different premise epu’s overall malaysian well-being index is based on macro and secondary data sources. the murninets is based on secondary data from various agencies at the local government level while the mfwi is based on sample surveys of families in kuala lumpur. hence, all three agencies are basing their well-being policies on a different premise; therefore, their findings cannot be generalized and are noncomparable to represent the urban population living in malaysian cities. however, the results of these three human well-being studies provide useful information to policy makers. for example, the epu’s study (figure 1) clearly shows that the components of family, environment and working life need further action by the relevant government agencies. this is because the social well-being of malaysians is lagging behind the economic well-being where the latter improved by 31 points from 2000 to 2014 whereas the former achieved an improvement of only 22.6 points. the family index showed only a minimal improvement of 0.1 point. therefore, this unhealthy ijahp article: dali, abdullah, islam/prioritization of the indicators and sub-indicators of maqasid al-shariah in measuring liveability of cities international journal of the analytic hierarchy process 354 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.597 imbalance should be rectified in order to create a truly happy and desired quality of life for all malaysians. also income and distribution increased to 136.5 points at the expense of environment (103.4 pts.) and working life (114.4 pts.). this depicts that even though the overall or composite index showed an improvement of 25.6 points, what matters is that the social well-being component has not satisfactorily increased. therefore, it has not translated to an increase in the desired quality of life and liveability for the average malaysians. source: the malaysian economy in figures (2016), economic planning unit (epu). figure 1 malaysian quality of life 2000-2014 by epu on the contrary, the results of the mfwi’s study shows that religion and spirituality gained the highest score in the family well-being context, that is 8.25 as compared to family economy which is only at 6.90 (the lowest score). table 3 lists the scores of all the seven domains based on the mfwi’s study. the agency monitors the situation by conducting a fresh survey (2016) and the mfwi will be updated accordingly. table 3 malaysian family well-being index, 2011 the seven domains scores out of 10.0 1. family & religion/spirituality 8.25 2. family& community 7.83 3. family relationships 7.82 4. family safety 7.39 5. family health 7.38 6. housing & environment 7.28 7. family economy 6.90 overall family wellbeing index 7.55 source: the malaysian family wellbeing index report, 2011 ijahp article: dali, abdullah, islam/prioritization of the indicators and sub-indicators of maqasid al-shariah in measuring liveability of cities international journal of the analytic hierarchy process 355 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.597 overall, the studies conducted by the three malaysian agencies help policy makers effectively make decisions in their sphere of authority and responsibility. however, they need to collaborate and coordinate their efforts towards achieving malaysian well-being as a whole. nevertheless, the results help move towards a more pragmatic measurement of quality of life in malaysia, hence streamlining more strategic policies and development for the achievement of an overall quality and well-being of malaysians. the effort to streamline these studies has been monitored by the malaysian syariah index (msi) by jakim (2015) launched by the government (razak, 2015). msi is an effort to reflect the compliance to the maqasid al-shariah indicators by the respective government departments in fulfilling the objectives of islamic principles. the aim of the index is to measure and evaluate malaysia’s level of maqasid al-shariah compliance in the government’s administration system. with this, malaysia is said to be the first country in the world that introduces and applies the five main elements of dharuriyyāt al-khams of the maqasid al-shariah (kamali, 2012). the msi is a measurement method that may also be termed as key performance indicator (kpi). hence, the kpi determines whether the particular sector is performing its functions better or worse and is monitored annually. performance by average scores by sectors for the two years can be seen in table 4. table 4 sectoral scores of the malaysian syariah index (msi), 2015 and 2016 sector average score performance description year 2015 2016 2015 2016 law 87.19 84.91 excellent excellent politics 79.19 74.70 good good economics 65.27 65.46 good good education 82.49 80.12 excellent excellent health 73.92 77.79 good good culture 66.47 64.67 good good infra. and environment 62.31 72.89 good good social 68.52 72.28 good good overall score 75.42% 76.06% good good source: jab. kemajuan islam malaysia (jakim, 2016) and the new straits times 12 th august, 2017. there is a slight improvement of the syriah index from 75.42% in 2015 to 76.6% in 2016. even though the scores have not yet met the target of 80% as set by the former prime minister, the syariah index is proof that efforts and initiatives taken and implemented by the government have impacted the people in terms of improvements especially in three sectors, namely health, infrastructure and society which include well-being of the community, religious care, and religious activities. hence, the vision as embedded in the maqasid al-shariah that seeks to protect human welfare, regardless of race, language and religion has been given priority. this means that not only the basic, physical and material needs have to be adequately provided, but ethical values and spiritual needs of human beings must also be protected. ijahp article: dali, abdullah, islam/prioritization of the indicators and sub-indicators of maqasid al-shariah in measuring liveability of cities international journal of the analytic hierarchy process 356 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.597 maqasid al-shariah is seen as capable in playing its part, for instance, in balancing work and play and balancing thinking globally and acting locally and practicing religious values and enjoying a good neighborhood, solidarity and community bonding especially in the context of malaysia’s diversity in race and religion. in essence, these factors when given equal weight and consideration would provide an overall goal of development and in achieving the desired quality of life for malaysians in all aspects of life socially, economically, environmentally and religiously and as a package for holistic liveable malaysian cities. when comparing western indicators against the malaysian ones, some features need to be highlighted. the literature regarding quality of life and liveable cities indicators from the world organizations show an outstanding gap, i.e., the religion factor is not considered. however, the analysis of the indicators and the results of city ranking based on the five specific domains reveal some commonalities. in other words, the indicators are seen to be exhaustively streamlined. but, at the same time they also serve as evidence that the indicators, interpretation and scope, organizational aim and objectives and methodology of the survey conducted influenced the different results on the cities ranking. the literature also exposes that, in fact, there are no ‘one-size-fits-all’ indicators as far as liveability and quality of life of cities is concerned (sarkawi et al., 2015). nevertheless, the world indicators are not targeting the quality of life for native residents or city populations per se but more for expatriates, businessmen, investors, corporations and to a lesser extent visitors and tourists. in fact, they are very much applicable to local residents and useful for town planners and city managers so that investments could be made to improve the facilities for the people. hence, in the long run, ranking of these cities will be on par with those of other cities in the world. similarly, for the malaysian models, even though there are attempts to include religious or islamic indicators in some of the studies, the emphasis is not deep enough to incorporate maqasid al-shariah. therefore, it is proposed that the religious factor especially emphasizing the maqasid al-shariah fundamentals be the main islamic liveability indicators and sub-indicators that this study aims to formulate. 1.2 universal importance of religion for liveability the outstanding finding from the western liveability indicators reveals that the only thing lacking is that religious indicators are not included when religion is generally and historically important (albright & ashbrook, 2001). a survey was done by the pew research centre to determine whether religion is indeed important to human lives (see figure 2). conclusively, all the population from the twenty three countries, west and east, developed and undeveloped countries, agreed that religion is important to them. universally, religion here includes christianity, islam, hinduism, buddhism, etc. ijahp article: dali, abdullah, islam/prioritization of the indicators and sub-indicators of maqasid al-shariah in measuring liveability of cities international journal of the analytic hierarchy process 357 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.597 figure 2 importance of religion in one’s life source: pew research centre, a non-partisan american "fact tank" based in washington, d.c. 2. the maqasid al-shariah fundamentals in defining the core dimension of dharuriyyat (necessities) of human lives this study adopts the maqasid views of renowned scholars like al-shatibi, ibn ashur, auda jasser, and yusuf al-qardhawi. generally, since the collective concern of urban liveability by these scholars is maslahah (public benefits), this study approaches human liveability and quality of life via the islamic framework of maqasid alshariah (as listed in table 5). table 5 the core dimension of maqasid al-shariah – dharuriyyat (necessities) dharuriyyat dimension 1. faith/religion (al-din) 2. life (al-nafs) 3. intellect/mind (al-aql) 4. lineage/progeny/dignity (al-nasl/al-nas/al-ird) 5. wealth/property (al-mal) islam as a way of life embraces the five essentials of its law, namely the dimension of necessities (dharuriyyat) and its five indicators. this study focuses on the dharuriyyat not the hajiyyat (complementary) and tahsiniyyat (embellishment). since the overriding aim of this study is to develop a model to measure liveability of cities by using maqasid al-shariah indicators and sub-indicators based on the dharuriyyāt dimension, the study design embraces a mixed-method approach where interviews of experts of two related fields i.e. the maqasid al-shariah scholars and the professional urban planners were carried out in two phases. the first phase which is qualitative in nature was based on semi-structured face-to-face interviews, and the second phase utilized the expert opinion survey via structured questionnaires which solicited both qualitative and quantitative data. the second phase was carried out in ijahp article: dali, abdullah, islam/prioritization of the indicators and sub-indicators of maqasid al-shariah in measuring liveability of cities international journal of the analytic hierarchy process 358 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.597 three stages of analytic hierarchy process (ahp) application, the verification or validation stage (qualitative) via the face validity technique and finally the actual conducting of the opinion survey stage (quantitative) via questionnaires. this is a crucial stage because the ahp could gather quantitative judgements in prioritizing the dimension, indicators and sub-indicators of maqasid al-shariah via the pairwise comparison matrices (pcm). prioritization is important to determine which indicators and sub-indicators are relatively more important in measuring liveability of cities within the islamic realm. during the validation stage of the ahp, the maqasid al-shariah experts endorsed and verified the indicators and sub-indicators of the maqasid principle as comprehensive and meaningful to be expanded into the maqasidic model of liveability of cities. those sub-indicators are based on literature reviews and expert opinions from surveys undertaken by the researchers. each indicator is given an initial for instance (f) for religion or faith and is further broken down into sub-indicators initialled as f1, f2, f3, and f4 and so on as listed in table 6. initials are useful when doing pairwise comparisons and their respective weights to determine the ranking or prioritization or importance level during the analysis stages. ijahp article: dali, abdullah, islam/prioritization of the indicators and sub-indicators of maqasid al-shariah in measuring liveability of cities international journal of the analytic hierarchy process 359 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.597 table 6 five indicators of the maqasid al-shariah and their respective sub-indicators indicators sub-indicators 1. faith/religion (al-din) (f)  safeguarding religious facilities, schools, mosques, suraus, islamic land use planning (f1)  religious activities/ programs at all levels of cityneighbourhoods (f2)  investments/budget on religious facilities and activities e.g. funding mosques, religious schools/ madrasahs/ “pondok” (f3)  constitution/legislation/state enactment (f4) 2. life (al-nafs)-(l)  fulfilling basic needs-food, shelter, housing, transportation, jobs etc. through zakat, job-matching, poverty eradication programs and charity. equitable baitulmal distribution for the 8 asnafs, waqaf etc.(l1)  public healthcare and recreational facilities/green environment (l2)  safety – security and protection against social crimes (l3)  charity/welfare centres/social facilities (l4)  constitution/legislation/state enactment (l5) 3. intellect (al-aql)-(a)  integrated aqli and naqli education system and facilities. allocations on r & d, nation building programs for future leaders, lifelong learning, freedom of speech/views/ijtihad etc. (a1  no drugs, no alcohol (a2)  ict infrastructure (a3)  constitution/legislation/state enactment (a4) 4. lineage/progeny/dignity (al-nasl/al-nasb/al-ird)-(p)  protecting family units and upholding the marriage institution (p1)  individual privacy/social rights/dignity-protecting families and neighbourhoods-guarded and gated facilities (p2)  prevention of immoral behaviours/adultery/vice/crime (p3)  prevention of discrimination/women’s dignity/ ‘awrah’, minority, racial, religion etc. (p4)  constitution/legislation/state enactment (p5) 5. wealth/property (al-mal)  wealth generation/economic opportunities. halal job generation, crime against wealth, halal investment business opportunities etc. (w1)  security of property/wealth. crime against wealth, bribery, ‘riba’, cheating (w2)  islamic wealth and financial management services(w3)  constitution/legislation/state enactment (w4) ijahp article: dali, abdullah, islam/prioritization of the indicators and sub-indicators of maqasid al-shariah in measuring liveability of cities international journal of the analytic hierarchy process 360 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.597 the next section provides a brief review of literature on applications of ahp in urban and regional planning and urban liveability. 3. literature review 3.1 ahp in urban and regional planning since its introduction, ahp has been applied extensively and effectively in many disciplines especially in complex decision and evaluation problems involving a multitude of objectives and stakeholders (saaty, 1977). this is because ahp is flexible, explicit and easily traceable (contreras et al., 2008; anis & islam, 2015). in fact, ahp has been applied in more than 30 diverse fields ranging from medicine, logistics, petroleum pipeline, hospitality, fast food restaurants, accounting, urban and regional planning and so on (saaty & islam, 2015). urban planning involves the “arts” and “sciences”, combining qualitative and quantitative criteria in its plan making. ahp’s flexibility, simplicity and capability form a powerful tool for urban planning applications. furthermore, urban planning handles voluminous and heterogeneous data; therefore, ahp performs effective analysis in deriving priorities, insights and options for planning scenarios or alternatives. meanwhile, the wills and decisions of political masters are real when it comes to the execution of planning policies and strategies of the plans prepared by the urban planners. ahp is able to simply define the extent of land use zones, the preferred development area and the ideal urban planning scenarios. these qualities of ahp will guide the politicians in decision-making resulting in systematic solutions of problems. there are numerous applications of ahp in urban and regional planning (zebardast, 2002). zebardast concludes that ahp is a suitable method in urban and regional planning due to its inherent simplicity and ability to incorporate both qualitative as well as quantitative factors. he discussed how ahp can be used in a site selection problem for urban development. on the other hand, lee and chan (2008) have used ahp for urban renewal in hong kong. according to the authors, multiple parties are involved in the renewal process that comprises citizens, professionals, policy makers and so on and each group has its own concerns. the authors recognize the difficulty in fulfilling everybody’s objectives, nevertheless through the ahp prioritization process an amicable trade-off can be reached. ameen and mourshed (2018) have used ahp to develop an urban assessment framework for iraqi cities. due to decades-long conflicts and war, the infrastructure of iraqi cities has been severely damaged. the authors have made commendable efforts to develop the afore-mentioned model by identifying and assigning priorities to the prime indicators. the authors found that water, safety and transportation and infrastructure indicators were rated highly by the respondents. the authors conclude that their developed methodology would play a key role in the promotion of built environment and ensuring sustainable iraqi cities. 3.2 ahp in liveability measures in one of his pioneering and early works, saaty (1986) used his own developed absolute measurement process of ahp to rank 329 cities in the united states. he used nine criteria, namely, climate, housing, healthcare, crime, transportation, education, arts, recreation, economics and a set of intensities for each criterion. the 10 best cities were found to be: nassau, ny; san francisco, ca; los angeles, ca; ijahp article: dali, abdullah, islam/prioritization of the indicators and sub-indicators of maqasid al-shariah in measuring liveability of cities international journal of the analytic hierarchy process 361 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.597 boston, ma; burlington, vt; portsmouth, ma; albany, ny; philadelphia, pa; seattle, wa; and pittsburgh, pa. in another work, saaty (2013) used the bocr framework of ahp to rank six cities in the world, namely: dubai, istanbul, la paz, new york, riyadh and tokyo. measuring liveability of cities is an important task as it provides useful information to the people who would like to choose the best place to live. it also provides information to local councils on which aspects of the city life need improvement. a number of research works are available regarding liveability measures of cities in china. huang et al. (2018) embarked on a project to evaluate liveability of living spaces of a number of provinces in china. the authors integrated ahp with the data collected from remote sensing and a statistical survey. the results showed a significant difference in liveability measures between villages and towns. the liveability index for towns was found to be higher compared to villages. further, spatial difference of the functional elements of land use was ascribed as the main reason for the difference in liveability measures. lee and chi (2010) developed an integrated method using anp and delphi to evaluate liveability of a selected number of places in china using five criteria: health, safety, comfort, convenience and socio-economics. the findings, as the authors claim, provide the local authorities with information to help combat natural disasters, but the paper did not provide the details on how these disasters could be averted. in another related work on city liveability measurement in china, yan et al. (2011) used the multiplicative model of ahp. the authors claim that for evaluation of urban habitability, their method is more scientific when compared with the existing methods used for a similar purpose. this is the first time that ahp is being applied in the planning field (liveability) explicitly relating to maqāṣid al-sharīah. this research attempts to spearhead the application of the ahp technique in the liveability planning sphere in the context of islamic values and ethics since none of the previous studies in urban planning or built environment-based relate to the objectives of islamic law. 4. theoretical framework a theoretical framework forms the structure and components towards developing the islamic liveability evaluation model. generally ahp involves four levels in a hierarchy, namely the goal (level 1), dimension (level 2), sub-dimensions (level 3) and indicators (level 4). applying this hierarchy (figure 3) in the context of liveability, the goal is to prioritize the indicators and sub-indicators of maqasid alshariah in measuring liveability of cities. necessities (dharuriyyat) form the focus or dimension of which they are to safeguard the five maqasid al-shariah essentials or indicators of faith, life, intellect, lineage and wealth. each indicator is further broken down into sub-indicators. for example, under faith there are four salient subindicators to be measured like availability of religious facilities, activities, funds and enforcement of religious legislations to nurture and protect faith of the ummah or muslim communities in cities. ijahp article: dali, abdullah, islam/prioritization of the indicators and sub-indicators of maqasid al-shariah in measuring liveability of cities international journal of the analytic hierarchy process 362 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.597 figure 3 theoretical framework for the islamic liveability measurement since the indicators and the sub-indicators are involved with multiple choices, according to anis and islam (2011), a multiple criteria decision making process such as the ahp questionnaire survey is an appropriate approach. therefore, a questionnaire survey involving 25 senior malaysian urban planners and 25 religious department officials and maqasid al-shariah practitioners was conducted to prioritize the indicators and the sub-indicators. this is to satisfy the objectives of the study, which are as follows: 1. to ascertain the indicators and sub-indicators that measure liveability, quality of life and sustainability of urban living and well-being of urban residents; 2. to rank the indicators and sub-indicators as precursors to measure urban liveability 5. methodology the application of the mixed-method of qualitative and quantitative methodologies used for this study help to strengthen the findings in terms of expanding the qualitative cardinal ranking into quantitative ordinal ranking (creswell, 2009). this research combines five qualitative surveys (interview surveys) and one quantitative (questionnaire survey) data collection method. the qualitative surveys furnished subjective opinions and were analyzed by the atlas.ti software while the quantitative survey translated them into the degree of relative importance via weighting of the ahp’s superdecisions software. for example, traditionally the five pertinent indicators of the maqasid al-shariah are considered of equal importance and subjectively ranked in a descending order of significance. this order however may differ on a case to case and urgency basis. however, the quantitative method is more definitive as it ranks religion first because it was given the highest weight by all the respondents; in this case, it was judged as having extreme importance according to the numerical rating (saaty, 2008). the scores also show the degree of importance among the respective indicators in relation to another indicator. the superdecisions software was employed to extract the weight of priorities of the maqasid indicators to prioritize the sub-dimensions and indicators of maqasid shari’ah neccessities (dharuriyyat) level 1-goal level 2dimension faith 1. facilities 2. activities 3. funds 4. legislation life 1. basic needs 2. healthcare 3. security 4. charity 5. legislation intellect 1. integrated education system 2. no drug, no alcohol 3. legislation lineage 1. family & marriage institution 2. neighbourhood privacy 3. prevention of immoral behaviour 4. discrimination 5. legislation wealth 1. wealth generation 2. security of property/ wealth 3. financial management 4. legislation level 4 – indicators level 3sub dimensions ijahp article: dali, abdullah, islam/prioritization of the indicators and sub-indicators of maqasid al-shariah in measuring liveability of cities international journal of the analytic hierarchy process 363 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.597 and sub-indicators. besides ranking the priorities, the consistency ratio (cr) was also determined. the cr reveals reliability and consistency levels of the feedback. overall, it was found that the cr is almost significantly identical for all the indicators and subindicators. each cr showed that they are all well below the threshold value of 0.1 with the highest at only 0.067 and the lowest at 0.0149. this means that the consistencies of their judgments are high. the respondents comprise two groups of 50 experts in their respective fields i.e. 25 urban planners and 25 maqasid al-shariah scholars and practitioners. the researchers set several criteria of selection with the objective of ensuring data quality. for the urban planners, three main criteria that meet the purposeful sampling technique for the selection of respondents are as follows: i) very senior (more than 20 years working experience) malaysian public sector officials and practicing urban planners at the ministry, federal, state, putrajaya, local authorities, and private practice levels. ii) have been involved directly with the promotion of sustainability planning iii) have professional experience in the ministry of urban well-being, housing and local government, federal town and country planning department, state town and country planning department, consulting firms and putrajaya corporation. for the maqasid group, the criteria for selection are their involvement in religious affairs like: i) being officials in the state government who are directly related to shariah matters. ii) holding religious positions and practice like nazir, imams, bilal and ustaz iii) university chairman and lecturers who are involved in maqasid al-shariah portfolios. iv) head of islamic project of an economic entity, perbadanan nasional berhad (pnb). v) senior fellows of the islamic understanding institute malaysia (ikim). 6. data analysis for the data analysis, data was gathered from the two ahp steps as follows: step 1 – validation stage that involved feedback from four selected maqasid experts via the face validity interviews. step 2 questionnaire survey that involved 25 maqasid practitioners and 25 urban planners (n = 50) the ahp process started with the validation stage of the questionnaire. this pertains to feedback on the draft questionnaire devised by the researcher based on literature reviews and data gathered from three preliminary interview surveys. advice and additions from the experts were incorporated and the ahp questionnaire was finalized before distribution to the 50 respondents to answer via the drop and collect survey method (brown, 1987). respondents were required to do the pairwise comparison matrices (pcm) on the indicators and their respective sub-indicators as shown in table 6. consequently, this study produced altogether 18 results in the form of pcm tables and 18 histograms. ijahp article: dali, abdullah, islam/prioritization of the indicators and sub-indicators of maqasid al-shariah in measuring liveability of cities international journal of the analytic hierarchy process 364 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.597 as an example, a sample of the combined (n = 50) completed questionnaire responses are shown in table 7. figure 4 shows the weights in a graphical form. table 7 shows the respective relative importance of the five indicators as reflected by their weights and ranks. the overall cr is also provided. figure 4 clearly compares the weights of the individual indicator which signifies its priority. table 7 the pcm combined (n = 50) dharuriyyat indicators (n = 50) indicators religion life intellect lineage wealth weights r a n k religion 1 2.79 3.11 3.97 3.73 0.432 1 life 1 2.24 2.28 3.25 0.231 2 intellect 1 1.50 3.73 0.155 3 lineage 1 3.05 0.119 4 wealth 1 0.063 5 cr = 0.054 figure 4 weights of indicators to form aggregated pcm, the geometric means method was used (basak & saaty, 1993). this is a mathematical equivalent of the consensus of the group judgment obtained from the feedback of the 50 respondents (islam, 2010). the superdecisions software was used to calculate the priorities of the indicators and sub-indicators from the pcms using those geometric means. the consistency ratio (cr) of less than 0.1 is considered acceptable and overall the cr was 0.054 which is less than 0.1; hence it is regarded as consistent and reliable. 7. findings results of the responses from the 50 respondents to the ahp questionnaire survey are in the form of pairwise comparison matrix (pcm) tables and were done individually. each pcm is accompanied by a histogram that shows the weights and consistency ratios (cr). altogether, there are 18 results in pcm tables and 18 histograms. hence, the findings are divided into individual groups of maqasid experts and urban planners. however, for the purpose of this paper, the combined findings are deemed representative of the overall findings since the weights are not significantly contrasting. 0.432 0.231 0.155 0.119 0.063 0.000 0.100 0.200 0.300 0.400 0.500 faith life intellect lineage wealth weights ijahp article: dali, abdullah, islam/prioritization of the indicators and sub-indicators of maqasid al-shariah in measuring liveability of cities international journal of the analytic hierarchy process 365 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.597 one significant finding is that all 50 respondents ranked the dharuriyyat indicators in the exact order of the classic maqasid al-shariah discipline. hence, the priorities are assigned in the descending order of religion, life, intellect, lineage and wealth accordingly. but, in terms of their cardinal values, figure 4 shows their respective weight. faith or religion scored the highest at 0.432 followed by life and so on. meanwhile, figure 5 reveals the priorities for the sub-indicators where the 50 experts have confirmed the five dharuriyyat of the maqasid al-shariah classic fundamentals; the respondents have assigned priorities to the sub-indicators according to their point of view. the respective sub-indicators that scored the first rank are as follows: f2-religious activities/ programs at all levels of city-neighborhoods. l1-fulfilling basic needs-food, shelter, housing, transportation, jobs etc. through zakat, job-matching, poverty eradication programs and charity. equitable baitulmal distribution for the 8 asnafs, waqaf etc. a1-integrated aqli and naqli education system and facilities. allocations on r & d, nation building programs for future leaders, lifelong learning, freedom of speech/views/ijtihad etc. p1protecting family units and upholding the marriage institution w1-wealth generation/economic opportunities, halal job generation, crime against wealth, halal investment business opportunities, etc. ijahp article: dali, abdullah, islam/prioritization of the indicators and sub-indicators of maqasid al-shariah in measuring liveability of cities international journal of the analytic hierarchy process 366 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.597 figure 5 hierarchic model of maqasid al-shariah with weights of the sub-indicators (note: detailed breakdown of the sub-indicator category as denoted with the initials can be seen in table 6) the sub-indicators, f2, l1, a1, p1 and w1 (shown bold in figure 5) are the most important within their respective sub-indicator category. hence, for a city to qualify as liveable, it should be measured against these five sub-indicators i.e. it should have religious activities/programs at all levels of city-neighborhoods, fulfill basic needs, provide integrated education system, protect family and provide halal economic activities and wealth generation institutions. conclusively, table 8 highlights the ranking of importance of indicators and subindicators by both of the expert groups. in terms of indicators, the ranking tallies with the classical maqasid al-shariah priorities. however, in terms of the sub-indicators, the expert groups ranked those according to their own judgments based on their experience and contemporary issues and values. indeed, maqasid al-shariah should be suited to the modern times and context. the combination of theological ethics as outlined by maqasid al-shariah and its modern day human-centric applications will finally propagate liveability and human well-being. level 2: indicators level 1: goal l3 (0.208) l5 (0.103) l2 (0.191) l4 (0.069) l1 (0.429) a4 (0.146) a3 (0.129) a2 (0.265) p2 (0.142) p4 (0.122) p3 (0.172) p5 (0.156) p1 (0.408) w1 (0.319) w4 (0.194) w3 (0.179) w2 (0.307) a1 (0.460) level 3: sub indicators f2 (0.319) f3 (0.273) f4 (0.148) f1 (0.260) selection of the best dharuriyyat dimension, indicators and sub-indicators faith (0.432) life (0.231) lineage (0.119) wealth (0.063) intellect (0.155) ijahp article: dali, abdullah, islam/prioritization of the indicators and sub-indicators of maqasid al-shariah in measuring liveability of cities international journal of the analytic hierarchy process 367 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.597 table 8 ranking of indicators and sub-indicators by the combined respondents 8. conclusions this study provides insights regarding the confirmation of the five priorities of the dharuriyyat dimension of the maqasid al-shariah as far as liveability, quality of life and sustainability of human living in cities are concerned. results from the ahp exercise which is objective and scientific in nature establishes that the various maqasid al-shariah elements, namely religion, life, intellect, lineage and wealth are in the descending order of importance. modern day urban planners and maqasid alshariah scholars and religious officials unwaveringly concur with this order in their judgments as evidenced by their responses in the questionnaire survey. this study therefore provides a strong empirical conclusion on the importance of maqasid alshariah for human well-being and the findings are in agreement with the set priorities laid out by the ‘old school’ doctrine. however, this study further extended these set priorities by elaborating on them in terms of rank and weight. also this study provides details on sub-indicators in each category and ranks the priorities from the calculated weights of the superdecisions software. interestingly, from the ahp analysis two other striking conclusions could be drawn. 1) overall, both groups came up with exactly the same ranking despite their very different religious background. the urban planners are western and have a modern background while the maqasid officials have an islamic educational and practice background. however, their outlook remains intact where they exude strong ‘religious’ perspectives and consensus. 2) the consistency ratio (cr) is very high throughout which shows that they are consistent with their opinions and judgments. it reflects that they gave honest and well-thought opinions and therefore the overall conclusions of the study are very reliable. finally, the findings of this study support the application of the ahp as an appropriate and viable technique because of the multiplicity of variables or indicators and sub-indicators involved. even though the sample size is relatively small, i.e., 25 respondents for each group, the respondents are experts and high profile personnel in their respective fields. their invaluable insights gave credit to the quality of responses that they gave, and therefore form a wealth of information and judgments. in conclusion, to render a city liveable, it has to protect religion, life, intellect, lineage and wealth. also, it has to ensure that the sub-indicators pertaining to religion and religious activities persists, the education system is inclusive and integrates both worldly and the religious knowledge, the family institution and intellect are protected, indicators rank subindicators rank subindicators rank subindicators rank subindicators rank subindicators rank religion 1 life 2 f1 3 l1 1 a1 1 p1 1 w1 1 intellect 3 f2 1 l2 3 a2 2 p2 4 w2 2 lineage 4 f3 2 l3 2 a3 4 p3 2 w3 4 wealth 5 f4 4 l4 5 a4 3 p4 5 w4 3 l5 4 p5 3 ijahp article: dali, abdullah, islam/prioritization of the indicators and sub-indicators of maqasid al-shariah in measuring liveability of cities international journal of the analytic hierarchy process 368 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.597 and lastly, the halal wealth generation and distribution are preserved and safeguarded for the sake of the city inhabitants. ijahp article: dali, abdullah, islam/prioritization of the indicators and sub-indicators of maqasid al-shariah in measuring liveability of cities international journal of the analytic hierarchy process 369 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.597 references albright, c. r., & ashbrook, j. b. (2001). where god lives in the human brain. a resurgence of interest in the study of religiosity in different disciplines. naperville, il: sourcebooks. ameen, f.m., & mourshed, m. (2018). urban sustainability assessment framework development: the ranking and weighting of iraqi indicators using analytic hierarchy process. sustainable cities and society, in press. anis, a., & islam, r. (2015). the application of analytic hierarchy process in higher-learning institutions: a literature review. journal international business and entrepreneurship development, 8(2), 166-182. doi: http://dx.doi.org/10.1504/jibed.2015.070446 anis, a., & islam. r. (2011). multiple criteria decision making. poland: the university of economics katowice. auda, j. (2008). maqasid al-shari’ah as philosophy of islamic law, a system approach. london: international institute of islamic law. basak, i., & t.l. saaty (1993). group decision making using the analytic hierarchy process. mathematical and computer modelling, 17, 101-109. doi: https://doi.org/10.1016/0895-7177(93)90179-3 brown, s. (1987). drop and collect surveys: a neglected research technique? marketing intelligence & planning, 5(1), 19-23. doi: https://doi.org/10.1108/eb045742 contreras, f., hanaki, k., aramaki, t., & connors, s. (2008). application of analytical hierarchy process to analyse stakeholder’s preferences for municipal solid waste management plans, boston, usa. resources, conservation and recycling, 52(7), 979-991. doi: https://doi.org/10.1016/j.resconrec.2008.03.003 creswell, j. w. (2009). research design: qualitative, quantitative & mixed method approaches (3rd ed.). thousand oaks, ca: sage publication inc. department of statistics malaysia (august, 2014). population and housing census of malaysia 2010. huang, a., xu, y., liu, c., hao, j., sun, p., zheng, w., & lu, l. (2018). evaluation on liveability of living space based on multiple functions of land use at county level. transactions of the chinese society of agricultural engineering, 34(8), 252-261. islam, r. (2010). critical success factors of the nine challenges in malaysia’s vision 2020. socio-economic planning sciences, 44(4), 199-211. doi: https://doi.org/10.1016/j.seps.2010.07.002 jab. kemajuan islam malaysia (jakim, 2016) and the new straits times 12th august, 2017. ijahp article: dali, abdullah, islam/prioritization of the indicators and sub-indicators of maqasid al-shariah in measuring liveability of cities international journal of the analytic hierarchy process 370 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.597 jakim (2015) indeks syariah malaysia, model tadbir urus berteraskan maqasid syariah. kamali, m. h. (2012). maqasid al-shariah made simple. washington d.c.: international institute of islamic thought. lee, g.k.l., & chan, e.h.w. (2008). the analytic hierarchy process (ahp) approach for assessment of urban renewal proposals. social indicators research, 89(1), 155-168. doi: https://doi.org/10.1007/s11205-007-9228-x lee, y.f., & chi, y.y. (2010). using the analytic network process to establish a new evaluation model of environment liveability. 40 th international conference on computers and industrial engineering, awaji, japan. doi: https://doi.org/10.1109/iccie.2010.5668365 malaysian family wellbeing index report, 2011. malaysian syari’ah index (msi, 2015). http://www.islam.gov.my/en malaysian town and country planning act, act 172, 1976. malaysian ummah development index (mudi report (2014) http://www.ikim.gov.my/index.php/en/ malaysian urban-rural national indicators network for sustainable development (murninets, 2012. https://murninet.townplan.gov.my/ malaysian wellbeing index report, epu, 2013 http://www.epu.gov.my/ muslim religiosity and personality indexing implications for nation building. upm, (2006) national urbanisation policy 2 (2016-2025). https://townplan.gov.my razak, d.s.n. (2015) memartabatkan syiar dan syariat islam, teks ucapan perdana di majlis perdana ulama-umara 2015 dan pelancaran indeks syariah malaysia, di dewan perdana, pusat konvensyen antarabangsa putrajaya, utusan malaysia. saaty, t.l. (1977). a scaling method for priorities in hierarchical structures. journal of mathematical psychology, 15(3), 234-281. doi: https://doi.org/10.1016/00222496(77)90033-5 saaty, t.l. (1986). absolute and relative measurement with the ahp: the most liveable cities in the united states. socio-economic planning sciences, 20(6), 327331. doi: https://doi.org/10.1016/0038-0121(86)90043-1 saaty, t.l. (2008). decision making with the analytic hierarchy process. international journal of services sciences, 1(1), 83–98. doi: http://dx.doi.org/10.1504/ijssci.2008.017590 saaty, t.l. (2013). compact city: the next urban evolution in response to climate change. pittsburgh, pa: rws publications. http://www.islam.gov.my/en http://www.ikim.gov.my/index.php/en/ https://murninet.townplan.gov.my/ http://www.epu.gov.my/ https://townplan.gov.my/ ijahp article: dali, abdullah, islam/prioritization of the indicators and sub-indicators of maqasid al-shariah in measuring liveability of cities international journal of the analytic hierarchy process 371 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.597 saaty, t.l., & islam, r. (2015) hierarchon vol. ii: a dictionary of hierarchies. pittsburgh, pa: rws publications. sarkawi, a.a., abdullah, a, & dali, n.m. (2015). contextualising the islamic fundamentals in the contemporary concepts of sustainability, liveability and quality of life in the built environment. middle-east journal of scientific research, 23(6), 1249-1256. doi: 10.5829/idosi.mejsr.2015.23.06.22287 steep (2013). how to become the world’s best city. retrieved from http://www.steep.fi/cases/research/2013/11/how-to-become-the-worlds-best-city/ on 13th september 2014. yan, f.y., & yuan, g.d., ge, l., & li, h.g. (2011). the evaluation research on livable city based on multiplication model and ahp. proceedings of the 18 th international conference on industrial engineering and engineering management, changchun, china. doi: https://doi.org/10.1109/icieem.2011.6035149 zebardast, e. (2002). application of analytic hierarchy process in urban and regional planning. honar-ha-ye-ziba, 10, 13-21. ijahp article: nachtnebel, singh/ prioritizing hydropower development using analytical hierarchy process (ahp) a case study of nepal international journal of the analytic hierarchy process 313 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.253 prioritizing hydropower development using analytical hierarchy process (ahp) – a case study of nepal hans peter nachtnebel inst. of water management, hydrology and hydraulic engineering department of water-atmosphere-environment university of boku, a-1190 vienna, muthgasse 18 email: hans_peter.nachtnebel@boku.ac.at *rana pratap singh 1 inst. of water management, hydrology and hydraulic engineering department of water-atmosphere-environment university of boku, a-1190 vienna, muthgasse 18 email: rnprtpsngh@yahoo.com abstract nepal possesses huge hydropower potential but lacks experience, funding and political stability which are critical to development. different national strategies have been proposed in the past ranging from promoting small scale hydro that provides local to regional economic incentives to the recommendation of large schemes to enhance national objectives. this paper analyses the last few decades of hydropower development in nepal and proposes a multi-criterion approach, the analytic hierarchy process (ahp), to identify projects considering social and environmental concerns in addition to economic objectives. a multi-perspective look at prioritizing hydropower schemes namely micro (below 1 mw), small (from 1-25 mw), medium (from 25-100 mw), big (from 100-1000 mw) and large (greater than 1000 mw) is important in order for hydropower development to proceed in the best possible way. this perspective could be of use in strengthening hydropower related strategy and policy in nepal. the prioritization procedure is embedded into a multi-objective framework including six goals, namely a technical goal measured by four criteria, a social goal with five criteria, an economic goal with six criteria, an environmental goal with 4 criteria, a political goal characterised by 4 criteria and an additional goal referring to the various uncertainties, expressed by five criteria. evidence based subjective value judgment based on secondary sources, mainly related documents and experts consultations, is used for the prioritization approach. this research could help policy makers to maximize the benefit to the country by adopting appropriate policies and strategies. 1 acknowledgement authors would like to thank the anonymous reviewer whose comments and suggestions improved the quality of the manuscript. mailto:rnprtpsngh@yahoo.com ijahp article: nachtnebel, singh/ prioritizing hydropower development using analytical hierarchy process (ahp) a case study of nepal international journal of the analytic hierarchy process 314 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.253 the effectiveness of multi criteria evaluation techniques and the analytic hierarchy process (ahp) for hydropower prioritization is the focus of this research. the outcome of the paper is the secondary information based on the ahp application for hydropower scale of schemes prioritization. this paper is presented in six sections. the first section is the introduction focusing on the country context and mcda in hydropower. the second section discusses the problem definition, and the third section describes objectives and tasks. similarly, the fourth section details the applied methodology, and the fifth section describes the results and discussion. finally, the sixth section presents conclusions and recommendations of the research work. keywords: hydropower; alternatives; ahp, prioritize; decision making 1. introduction nepal has a rich potential of hydropower with 83,000 mw available, but currently only 727 mw of that hydroelectricity has been tapped. nepal, engraved with extreme energy poverty, has per capita energy and electricity consumption of 16 gigajoule and 67 units respectively, which is far below the asian average (water and energy commission secretariat, 2010a). electricity consumption is growing at an annual rate of more than 10% (90 mw), and industrialization is halted due to the ever widening energy gap (1000 mw at present) in the country (wecs, 2010b). to overcome this problem, water resources and hydropower as a panacea for transforming the country’s economy are at the top of the development agenda in nepal. 1.1 hydropower development in nepal so far, less than 1% percent of the hydropower potential has been tapped in the last 100 years. though there could be many reasons for the underdevelopment of hydropower, the most important is a lack of understanding and poor planning. in the nepalese context, the business approach, and the social factors need more awareness and capacity building (sovacool, dhakal et al., 2010). most often the hydropower sector has been understood in a very fragmented way, dominated by techno-economics and ignoring many important aspects. since the late 1970s, foreign aid completely dominated the power sector and until recently hydropower development was on an ad hoc basis under various models insisted upon by donors (pun, 2008). after restoration of democracy in 1990, hydropower took on a new look. the tenth plan specifies eight criteria applicable for infrastructure including hydropower which demands the use of multi criteria decision analysis (mcda) (npc, 2002). the government of nepal has committed its energy sector development for poverty reduction and economic development through hydropower (water and energy commission secretariat, 2010c). it classifies hydropower schemes as micro up to 1 mw, small from 1 to 25 mw, medium from 25 to 100 mw, big from 100 to 1000 mw, and large from 1000 mw upwards (water and energy commission secretariat 2010a). hydropower must be viewed in multidimensional ways including irrigation, water supply, navigation, tourism, hydropower etc. in order to be used most effectively (rees, holmes, et al., 2006; world bank, 2003). a decision framework should consider a net ijahp article: nachtnebel, singh/ prioritizing hydropower development using analytical hierarchy process (ahp) a case study of nepal international journal of the analytic hierarchy process 315 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.253 improvement of income of the lower income groups in society and avoid the traditional approach of unfair decision making (gunawardena, 2010). hence, mcdm could certainly be very much appropriate in decision making. 1.2 multi criteria decision analysis (mcda) in hydropower hydropower development is a long term investment with high up-front costs and risks. it has a long gestation period and is full of social and environmental challenges that make decision making more complex. the hydropower decision making process extends beyond the classical model of an optimal solution and shifts the conception of a problem to a satisfactory solution (guitouni & martel, 1998). throughout the past decades, the hydropower project planning paradigm shifted from techno economics to socio-environmental (iha, 2014; unep, 2007). mcda is gaining popularity worldwide, and differs according to project nature, site and country policy (foran, 2010). also in nepal, a multi criteria analysis for hydropower scheme selection is gaining attention by translating quick and tangible economic benefits under consideration of socio-economic consequences and environmental impacts. this requires an approach which encourages local participation and use of affordable, reliable and maintainable technologies. limited natural resources, socio-ecological constraints and conflicting interest among diverse stakeholders like the community, government authorities, investors, developers, professionals, civil society, economists, sociologists, environmentalists etc. demands evaluations and prioritizing among available alternatives for the best fit. it is important during the planning of a hydropower project to identify through a social impact study who will benefit from the project and especially who will be exposed to negative impacts (mathur, 2008). over the years, several mcda methods have been proposed (loken, 2007). some widely applied methods are the pair wise comparison and outranking methods (hajkowicz & collins, 2007). mcdm is used in different subjects and ahp, electre, and promethee are some very popular tools frequently applied for natural resources, water management and energy planning including hydropower (toloie-eshlaghy & homayonfar 2011; balali et al. 2014; mendoza & martins 2006). one of the most popular and trusted tools, called the analytical hierarchy process (ahp), is very appropriate for decision making on hydropower (akash et al. 1999; ertay et al. 2013; supriyasilp et al. 2009). in the nepalese context, ahp is applicable in decision analysis for many infrastructure developments including hydropower (bhattarai, 1997). 2. problem definition 2.1 different scales and broad impacts in nepal, two major strategies are debated as solutions to meet the energy need. the first is the development of numerous small hydropower plants with minimal external dependency, and another is the development of a more considerable size plants that offers the benefit of economies of scale (bergner, 2013). though each scale of hydropower development is important, it is necessary to prioritize them within the existing country context to maximize the national interest and benefit. this is an important but missing step in the nepalese context. ijahp article: nachtnebel, singh/ prioritizing hydropower development using analytical hierarchy process (ahp) a case study of nepal international journal of the analytic hierarchy process 316 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.253 due to specific interests and preferences of diverse stakeholders, the hydropower sector is full of controversies and conflict. there are several instances where a particular scale of hydropower scheme would fit best, but in another instance it falls far behind. there are complex and conflicting preferences related to technical, social, economic, environmental, political and associated uncertainties for different scales of schemes. though a high potential exists in nepal, only 10,000 to 12,000 mw equivalents of hydropower plants are within the cost of us$1,000 -1,500 per kw and easy to implement (shrestha, 2012). this should be a priority. also, several attractive hydropower sites totalling 2,110 mw that are close to the middle hill road project that is under construction are attractive at a lower construction cost (kuwar, 2013.). further, existing hydropower plants that are integrated into the national power grid are regionally unbalanced and the transmission is inefficient with a record highest loss of 36% in 1984 (bhattarai, 2012; nepal & jamash, 2012). the national power transmission system has a strong correlation with the size of the scheme that is prioritized, and so is applicable with other infrastructure like roads and bridges. the existing grid could accommodate only a small to medium plant; a big or large scale power plant integration into the grid would require enhancement of the existing grid capacity which is a time consuming and costly affair. therefore, the problem is to choose which scale of power plant scheme should have priority. a strong pool of more than 10,000 human resources in the hydropower sector have experience with small and medium scale plants (nea, 2012.) . while bigger scale schemes like upper tamakoshi (456 mw), kali gandaki (144 mw), mistri kola are excellent examples of road access, safety enhancement, business expansion etc., micro schemes are still the preferred solution for rural energy access (rajauriya, 2012). medium power plant development is crucial for the nation’s social capital formation in order to take the next steps to big or large schemes in the future. employment opportunities increase with increased scale of generation except when large size plants are export oriented. unfortunately, nepal has very limited experience and manpower both in quantity and quality for big or large scale plants. also, dam-related resettlement practices have failed to restore livelihoods in nepal (fast & hansson, 2013). socially, large size projects have many hurdles to overcome before they can progress. within this context, it is critical to prioritize scale of schemes in order to maximize the national benefit. with the availability of local construction materials and support infrastructure like bridges, roads, and grids the economic viability of hydropower has been enhanced in recent years. although bigger schemes may benefit economies of scale in general, the cost can remain high due to the majority of materials being imported and the need for additional investment in support infrastructure. bigger size projects can make more contribution to the national energy need if they are not export oriented. further, foreign aid and external financing makes the projects costlier because of the mode of financing and the impact of contracting on project economies (panday, 2003). presently, the financing sectors are hesitant to take any huge investment risks, and prefer to test the business reliability in steps. available financing, and the 1520% of the national budget allocated for the hydropower sector for coming 5 years, could bridge the energy gap (nrb, 2013; jha, 2069 b.s.). however, several of the licenses for large size projects that were issued earlier have been cancelled because of capacity and finance constraints. ijahp article: nachtnebel, singh/ prioritizing hydropower development using analytical hierarchy process (ahp) a case study of nepal international journal of the analytic hierarchy process 317 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.253 within the financing scenario discussed, it is challenging to maximize the economic benefits from the development of the most appropriate schemes of hydropower. the environment is one of the important issues to be examined in all major infrastructure projects including hydropower, which is subjective to the site, type and size of the schemes (bhatta & khanal, 2009). in general, the bigger the plant, the bigger the environmental concerns and negative consequences will be. in this respect, the smaller plants are more environmentally friendly. reservoir inundation and sediments are major challenges in the nepalese context whose mitigation may demand further investment (thapa, shrestha, et al., 2005). hence, prioritization with this in mind can assist in identifying an environmentally friendly hydropower scheme to develop. 2.2 policies and strategies in hydropower development to make best use of available resources and maximize the national interest, required policy and strategies are very important. in the absence of multi-dimensional analysis of these alternatives, efforts on hydropower development in nepal are diffused. government propositions about hydropower development frequently change and sometime contradict each another, specifically concerning their priority on alternative (scale of) schemes. the country is missing a clear vision with respect to hydropower development as well a profound planning procedure for best use of available resources in the country. the government of nepal (gon) emphasizes energy access to the majority of the people as quickly as possible. some of the ambitious plans put forwards include an electricity crisis mitigation work plan of 25,000 mw in 20 years and 10,000 mw in 10 years. while the government is continually supporting decentralized energy programs for access, recently their interest has been for bigger size projects like upper tamakoshi (456mw), budhigandaki (600 mw), nausyalgad (400 mw) etc. there are also plans and policies favouring storage type and multipurpose medium to large hydropower plants (jica 2013). as mentioned above, big to large single or multipurpose schemes demand a huge long term investment and careful preparation (thut et al. 2011). hence, the government opts for hydropower project development based on particular objectives while the country needs prioritizing the scale of hydropower development in order to maximize the national interest. 2.3 risks related to hydropower development in hydropower, different types of risks are applicable at different stages and are of different magnitude (world bank, 2005). technical risks are high, specifically in developing countries engaging in large projects. political (policy) risks due to flaws in policy and political stability could also be damaging to hydropower development. environmental risks arising from climate change, glaciers outbursts and seismic risk are serious in the nepalese context (icimod, 2011; agrawala, raksakulthai, aalst, larsen, smith & reynolds, 2003; nsc, 2012). economic risk and mitigation strategies relating to capital investment, local currency fluctuation and market failures must be worked out in advance (shrestha, 2014). social risk is also very important specifically concerning resettlement and land acquisition (messerschmidt, 2008). delay and adverse impact on the project economy must be minimized to ensure smooth implementation of the schemes (panthi, 2007). with society’s increased public awareness about rights concerning natural resources and possible consequences of adverse environmental impacts, hydropower development suffers from controversies, conflicts, longer gestation periods, cost overrun ijahp article: nachtnebel, singh/ prioritizing hydropower development using analytical hierarchy process (ahp) a case study of nepal international journal of the analytic hierarchy process 318 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.253 and slow progress. hence, an evaluation of the uncertainties associated with hydropower schemes is another important goal in prioritization research. identifying and assessing all these risk elements is a challenging task. due to limited data availability and the complexity of the problem, the application of more qualitatively expressed indicators is recommended. several non-tangible factors need subjective judgement which is critical, and they also need proper comparison and evaluation. to make the best use of available resources and maximize the national interest, coherent policy and strategies are very important. in the absence of multi-dimensional analysis of these scale dependent alternatives efforts on hydropower development in nepal are diffused. government propositions about hydropower development frequently changes and sometime they contradict to each other and specifically the priority of scale changes. 3. objective and tasks in the previous section, the contribution of different sized hydropower plants to economic development, the social implications and the environmental consequences were discussed together with the associated uncertainties. it is obvious that for a sound assessment and subsequent prioritization of schemes a set of tasks together with descriptive indicators from which some are qualitatively expressed is needed the final evaluation or project prioritization must be based on the overall performance obtained by ranking the accumulated and weighted project outputs. the primary objective of this study is to recommend the most appropriate scale (micro, small, medium, big or large) hydropower for nepal in the present context and immediate future using ahp. as mentioned in section 1.1, hydropower is classified into five groups depending upon its generation capacity.  micro hydro: up to 1 mw  small hydro: 1 to 25 mw  medium hydro: 25 to 100 mw  big hydro: 100 to 1000 mw  large hydro: 1000 mw upwards in other words, this set of development options, constituting our alternatives has to be evaluated with respect to a set of goals, each of them described by several criteria. thus, a multi-objective framework with these goals together with their respective criteria will be identified. several similar studies found from scientific publications are used in drafting a framework (nachtnebel et al. 1994; ganoulis 2008). though each scheme, independent from its scale, is contributing to development prioritization is very important to make a decision for the best outcome. therefore, for the prioritization of hydropower among the five available alternatives several goals and criterion contributing to prioritization must be taken into account simultaneously. the involvement of numerous stakeholders with different priorities and criteria with differing weights that are able to reach a compromise is important (haralambopoulos & polatidis, 2003). ijahp article: nachtnebel, singh/ prioritizing hydropower development using analytical hierarchy process (ahp) a case study of nepal international journal of the analytic hierarchy process 319 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.253 since the 1980’s, there have been several methods invented to solve decision problems. the mca methods used most often that focus on hydropower are fuzzy set analysis, distance to ideal point, pair wise comparison and outranking methods, multi criteria value function, distance to ideal point and out ranking method, weighted summation / multiplication (strin & groselj, 2010). though more than one method is recommended to verify the decision results, some studies show that the various methods used for decision analysis mostly produce similar results (shajari, bakhshoode et al., 2008). therefore, in order to minimize the complexity of some of the methods based on nonlinear forms and complex mathematics, the multi-criteria analysis (mca) based on pair wise comparison was chosen for this research. there are several advanced tools applied in pairwise comparison for alternatives ranking like ahp, electre, promethee etc. one of the most widely applied tools called the analytical hierarchy process (ahp) is used in the present study. the ahp methodology has been accepted by the international scientific community as a robust and flexible multi-criteria decision making method that deals with complex decision problems (strin & groselj, 2010). ahp is widely used in decision making, particularly for the hydropower context (subramanian & ramanathan, 2012; rosso, 2014; vucijak, 2013 and ahmed, 2014).the ahp is effective as a decision aid that can assist decisions makers in choosing the best alternative or ranking a set of alternatives specifically in a sector like hydropower where group conflict among different stakeholders exists. with the increased application of ahp in the nepalese context, some of the earlier hydropower related studies based on ahp and multi criteria endorse the suitability of the ahp application in the present research (bodin & gass, 2004;. bhattarai, 2003; bhattarai and fujiwara, 1995; usaid-sari, 2002). hydropower prioritization and planning requires appropriate data input, and in the absence of the required data available literature and second hand information could be organized to understand the sector. hence, this study needs secondary information gathered from several published water resources and hydropower related studies, peer reviewed scientific and professional journals, academic researches, news articles, project reports and publications. 4. methodology this study focuses on analysing hydropower development in nepal and contributing to a sound strategy for hydropower development. in order to achieve this task a framework is developed containing goals and criteria, and then a method is proposed to achieve a ranking of the alternatives with respect to their size. the present research approach uses evidence based secondary sources of information which gives better insights than what could be available from other approach like opinion survey, workshop, expert views, group or actors opinion etc. secondary data sources could be cross checked from various sources to minimize error and fill information gaps which enhance the reliability of the research. in the present study, information pertaining to different aspects of hydropower, associated with several stakeholders, belonging to different groups is collected. different stakeholders that belong to different groups may have different preferences and could be analysed with respect to their particular group such as economist, sociologist, and environmentalist and so on, but in order to make the ijahp article: nachtnebel, singh/ prioritizing hydropower development using analytical hierarchy process (ahp) a case study of nepal international journal of the analytic hierarchy process 320 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.253 research simple, the present study treats all responses as a single group of respondents for analysis. it is a research problem dealing with several alternatives, one decision maker (main author of the paper as researcher), and numerous outcomes and thus using mcdm. 4.1 identifying goals and criteria to meet the overall objectives set for the present research, keeping in mind the problem described earlier, the hydropower analysis in the present study should be viewed with respect to certain goals and measured on specific criteria. while economic, social, environmental and political goals are very common in mcdm applications, two additional goals are included. due to the fact that hydropower requires a sound technical education and training at several levels, this being especially relevant for nepalese conditions, technology has been integrated as a specific goal that is relevant for decision making. similarly, hydropower is binding over a long time and a huge investment, while at the same time is full of uncertainties throughout its life. hence, uncertainties have to be evaluated in the decision making. finally, the following six goals with their characterising criteria and desired directions (expectations) are considered in the present study:  technical: maximize self-reliance  social: maximize social benefits  economic: enhance country economy  environmental: minimize adverse impact  political: maximize national progress  uncertainties: increased safety against risks following the problem definition discussed in section 2 and information gathered so far from various sources, criterion are identified and placed with corresponding goals to achieve the overall objective set in this research. in table 1 the final grouping of criteria with respect to goal, along with the symbol and expected objective is presented. ijahp article: nachtnebel, singh/ prioritizing hydropower development using analytical hierarchy process (ahp) a case study of nepal international journal of the analytic hierarchy process 321 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.253 table 1 goals and criterion with their objective goals criterion description symbol objective technical experience and expertise to study, design, implement, operate and maintain hydropower projects t1 maximize project development dependency on outsider’s technical assistance to design, implement and operate t2 minimize grid readiness for transmission and distribution to accommodate new projects t3 maximize system and business handling capability to manage contracts, negotiations, agreements and logistics t4 maximize social equity and benefits distribution, induced safety and services to participating communities s1 maximize inclusiveness of beneficiaries specially poor and women and good governance of project in terms of transparency in decision and information sharing s2 maximize social capital formation enhancing people capability to replicate and participate in new projects development s3 maximize energy access to beneficiaries and enhancing reliable energy availability in the country s4 maximize heritage and culture preservation against adverse influence or loss s5 conserve economic generation capacity to avail more energy and combat energy import from outside e1 maximize investment and operation cost of power generation e2 minimize enterprise strengthening by number and capacity and flourishing many economic activity e3 maximize use of locally available construction materials, human resources and existing infrastructure like road, bridge etc. e4 maximize finance resources availability and conditionality on investment e5 maximize developers interest and readiness to develop the project e6 maximize environmenta l river morphology conservation to preserve ecosystem and riparian ecology e1 conserve terrestrial (land, forest) environment protecting from encroachment and inundation due to project e2 conserve water quality, availability and connectivity ensuring peoples need in the vicinity e3 maximize waste and pollution management during project construction and afterwards e4 minimize political policy & strategy support in country to attract more projects implementations p1 maximize time plan to meet power deficit and also ensure power development target plan for future p2 minimize contribution to overall national development agenda like industrialization, transport, new cities, participation to the international grid p3 maximize regional balance of power system to avail power in every parts (region) of the country p4 maximize uncertainties technological risk due imported technology and scale of schemes u1 minimize political risk to instability in the country and fluidal hydropower policy or strategy u2 minimize environmental risk due to climate change and geophysics of the region u3 minimize social (implementation) risk arising due to peoples dissatisfaction and revolts against project development u4 minimize financing ( market) risks due to weaker financing capability of national financing institutions and harsh conditionality posed by international financers u4 minimize ijahp article: nachtnebel, singh/ prioritizing hydropower development using analytical hierarchy process (ahp) a case study of nepal international journal of the analytic hierarchy process 322 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.253 4.2 assessment and ranking procedure in this paper there are five alternatives (capacity range of hydropower schemes) to be ranked by one decision maker with respect to a set of six goals and 28 criteria. numerous multi-criteria techniques are at hand (toloie-eshlaghy & homayonfar, 2011). the selection procedure to identify an appropriate technique is again an mcdm approach (ozernoy, 1997). here, the ability to handle qualitatively expressed criteria, to analyse the sensitivity of ranking, the visual support of the method and the proven applicability to hydropower project assessment were decisive. the analytical hierarchy process satisfies these conditions using a software called expert choice. subsequently, the fundamentals of ahp are discussed briefly to facilitate its understanding and application (saaty, 1986). the mca model is represented by an evaluation matrix x of n decision options and m criteria. the raw performance score for decision option i with respect to criterion j is denoted by xi,j. the importance of each criterion is usually given in a one dimensional weights vector w containing m weights, where wj denotes the weight assigned to the jth criterion. it is possible for x and w to contain a mix of qualitative and quantitative data. a great variety of mca algorithms can be used to either rank or score the decision options. the mca algorithms will define, by some means, one or both of these functions: ri = f 1 (w, x) and ui = f 2 (w, x) here ri is an ordinal number representing the rank position of decision option i and ui is the overall performance score of option i. the solution of ri and ui occurs within a broader mca decision making process. the mca process generally contains the following stages: choosing decision options and evaluation criteria, obtaining performance measures (xi,j) for the evaluation matrix, transforming them into commensurate units, weighting the criteria, ranking or scoring the options, performing sensitivity analysis and finally making a decision (rac, 1992). one of the most widely applied techniques is the analytic hierarchy process is pairwise comparison (saaty 1987). this approach involves comparing criteria and alternatives in every unique pair giving n (n-1) / 2 comparisons. the comparisons can be made to attain criteria weights and decision option performance scores. various scaling systems can be used. ahp decision makers are asked to express a preference for one criteria/option over another in each pair on a nine point scale. the ahp is based on the axiomatic foundation as follows (saaty, 1986): 1. the reciprocal property that is basic in making paired comparisons 2. homogeneity that is characteristic of people’s ability for making paired comparisons among things that are not too dissimilar with respect to a common property and, hence, need for arranging them within an order preserving hierarchy 3. dependence of a lower level on the adjacent higher level 4. the idea that an outcome can only reflect expectations when the latter are well represented in the hierarchy. ijahp article: nachtnebel, singh/ prioritizing hydropower development using analytical hierarchy process (ahp) a case study of nepal international journal of the analytic hierarchy process 323 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.253 the work on the ahp involves the estimation of priority weights of a set of criteria or alternatives from a square matrix of pair-wise comparison a = [aij], which is positive and if the paired comparison judgment is perfectly consistent it is reciprocal, i.e. aij = 1/aji for all ij = 1, 2, 3,.., n. the final normalized weight of its i-th factor, wi, is given by w a ai ij kj k n           / 1  i = 1, 2, ...., n. in real life judgments an error on the judgment is unavoidable. the suggested eigen value method computes w as the principal right eigen value of the matrix a or w satisfies the following system of n linear equations: a w =  max w, where  max is the maximum eigen value of a. this is to say that w a w i ij j j n   1  max i = 1, 2, ..., n. the natural measure of inconsistency or deviation from consistency, called consistency index (ci) is defined as ci   maxn n1 . the consistency index of a randomly generated reciprocal matrix from scale 1 to 9, with reciprocals forced, for each size of matrix called random index (ri) is presented in table 2. table 2 random index (ri) source: saaty, (2007) then the consistency ratio is (cr) = ci / ri, where ri value applied corresponding to the matrix size. the value of cr < 0.01 is typically considered an acceptable limit if this limit is not reached one should reduce the inconsistencies by revising his judgments. the other task in the hierarchy is the synthesis of the judgments throughout the hierarchy in order to compute the overall priorities of the alternatives with respect to the goal or objectives. the weights are created by summing the priority of each element according to a given criterion by the weights of that criterion. a pair-wise comparison scale for an evaluation of the relative importance of factors used in the ahp subjective judgment in matrix order 1 2 3 4 5 6 7 8 9 10 ri 0.00 0.00 0.58 0.90 1.12 1.24 1.32 1.41 1.45 1.49 ijahp article: nachtnebel, singh/ prioritizing hydropower development using analytical hierarchy process (ahp) a case study of nepal international journal of the analytic hierarchy process 324 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.253 accordance is given in table 2 (saaty, 2007). table 3 scale of pair-wise comparison intensity of importance definition explanation 1 equal importance two activities contribute the equally to the objective. 2 weak 3 moderate importance experience and judgment slightly favour one activity over another. 4 moderate plus 5 strong importance experience and judgment strongly favour one activity over another. 6 strong plus 7 very strong or demonstrated importance an activity is favoured very strongly over another; its dominance demonstrated in practice 8 very very strong 9 (absolute) extreme importance the evidence favouring one activity over another is of the highest possible order of affirmation. source: saaty, 2007. 4.3 research work flow the research work flow followed in the present study is described below in figure 1. figure 1. study work flow ijahp article: nachtnebel, singh/ prioritizing hydropower development using analytical hierarchy process (ahp) a case study of nepal international journal of the analytic hierarchy process 325 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.253 first, information was collected from various reliable secondary sources like government institutions, published reports, electronic media etc. the information obtained was organized by criteria, and further reviewed against five alternatives of hydropower schemes. secondary information was entered into a table with the goals and criteria against the alternatives which ranged from excellent to worst. this was used later to make pair comparisons. in the third stage, the organized data was used for pair comparisons and weight allocations which are required in ahp using software called expert choice for further processing. the ahp used in this research is presented in following figure 2. figure 2. the ahp model at this stage, consistency in data entry at each step was checked and maintained within an acceptable level. at the end, the software provided results. these results are reviewed and further discussed in detail with the experts working in hydropower sector of nepal for cross verification of the data entered and results obtained. this will enhance reliability of the results obtained by the ahp application in the hydropower study. finally, corresponding scores for alternatives were obtained which were then used for their ranking or comparison. the highest score has the most priority, whereas the lowest score is the least prioritized scale of hydropower in nepal. similarly, a sensitivity analysis was carried out for obtaining delegates decisive elements in hydropower prioritization in nepal. 5. results and discussion a large amount of data was required in this research which may be of qualitative or quantities type. lack of sufficient data and subjectivity in data entry was a critical challenge in this research. since reliability varies considerably among various sources of information, this may have had an impact in final results obtained in this research. further, weights assigned to goals and criteria are logged from reviewed documents and experts’ suggestion which is again subjective and dynamic within the country context. this could have also influenced the results of the research. hence, in these situations, as much as possible, several secondary sources of information that were collected were ijahp article: nachtnebel, singh/ prioritizing hydropower development using analytical hierarchy process (ahp) a case study of nepal international journal of the analytic hierarchy process 326 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.253 reviewed carefully and discussed frequently with professionals working in this sector to minimize error and enhance the reliability of the results. every type of hydropower developed in the country could contribute in different ways. for example micro scale plants work well for remote energy access, but could not contribute much to the national energy need. big or large schemes certainly need a huge amount of funding and a long gestation time while small or medium power plants managed domestically could result better benefit for the country. thus each scale of scheme could best fit in some context and benefit the country in different ways. the trends indicate a shift of interest from an earlier preference for small and medium scale towards medium and big scale hydropower development. the government focuses on big size projects and already initiated super 10 projects as the nation’s priority. economies of scale in power generation are attracting more medium and big size plants. while large size projects are under consideration and discussion, several micro to medium schemes are being implemented. considering all of these complicated aspects of hydropower in nepal, an ahp application to prioritize scale of schemes is important. the results of this ahp based study take into consideration the individual preference or importance of each goal, and criteria and alternatives with respect to the overall objective of the problem. the researcher as the decision maker in the present study uses his best understanding about the goals and criteria contribution to realize the main objectives of the research. in this analysis, the bottom level pairwise comparisons among criteria with respect to the main objectives are carried out by the decision maker. the same approach is followed while making mid-level comparisons. at the top level the weight assigned to the goals was followed based on reviewed papers and practice applicable to hydropower in lcds including nepal. 5.1 goals, weight and alternatives rankings if each goal in the present study was weighted equally it would weigh in at 17%. the strong importance of economy in project selection is found in several project reports and scholarly articles (bhattarai, 1997; marttunen et al., 2010). accordingly, economy is weighted at 25%. in same manner, the importance of the political goal is estimated at 20%, slightly above average. this is the same for the social goal. the technical goal is weighted at 15%, which is close but slightly below average weight. the environmental goal and its associated risks are weighted at 10%. overall the weight assigned to each goal and its corresponding prioritization can be viewed in figure 3. at present, the majority of factors found medium and big scale schemes as the priority, while environmental factors and uncertainties favoured micro and small scale hydropower plants. one can note that big scale hydropower development was in the interest of social, economic and political factors, whereas the technical factor preferred medium scale plants. both for environmental and uncertainties, the preference order was from micro towards large scale hydropower development in nepal. among the factors, in terms of importance, economic factors remain the highest (25% weightage) and were found to be the most sensitive. sensitivity with respect to change in economic factor weightage is further discussed in section 5.3. ijahp article: nachtnebel, singh/ prioritizing hydropower development using analytical hierarchy process (ahp) a case study of nepal international journal of the analytic hierarchy process 327 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.253 figure 3. prioritization with respect to various goals and weight analysis found that a change in economic weightage of 20% could change the priority order; this is a big jump and may happen in long run but not in immediate future. all other factors were found to be stable within a foreseeable change in weightage. hence, this analysis was found to be sound and could be adopted for policy formulation and strategy. nepal could definitely maximize the benefit from hydropower development in the country by adopting the appropriate backup for the top scoring scale of schemes. further, in the long run, with the strengthening of nepal’s economy big schemes (100 to 1000 mw) will take priority surpassing medium schemes which are currently the highest priority. maybe one day large schemes (>1000 mw) will be highly beneficial to the country with the changing country context and preferences of schemes. therefore, the most beneficial policy and strategies must be able to be tuned according to whatever the current situation in the country is. 5.2 overall and goal wise prioritization of alternatives each scale of schemes could best fit in some context and benefit the country in different ways. in the present context, considering all goals and respective criterion, ranking of alternatives are shown in figure 4. it was found that medium scale schemes in nepal ranked first, followed in decreasing priority by big, small, micro and large scale schemes. the two extreme scales of schemes fell at the lowest priority namely micro and large scale hydropower. ijahp article: nachtnebel, singh/ prioritizing hydropower development using analytical hierarchy process (ahp) a case study of nepal international journal of the analytic hierarchy process 328 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.253 figure 4. overall syntheses on prioritization of hydropower in nepal micro schemes, though excellent on the environmental goal and uncertainties measures, was poor on the economic and social expectations because of very low generation capacity. in the case of large schemes, in spite of excellent economies of scale, due to its energy export and external financing requirement, its overall contribution to the country and ranking is lowest. although big and small schemes are ranked second and third, they are in competition with the highest ranking medium schemes. these schemes could be managed and implemented domestically to meet energy needs and benefit the country. the ahp results help in understanding the relative prominence of available alternatives with respect to goals. the overall synthesis on prioritization with respect to various goals prioritizing alternatives is presented in figure 5. it is interesting to note that large scale schemes are least preferred with respect to every factor except economics. future changes in priority ranking are very much dependent on changing economics in the country. if such large schemes are developed within the need and the resources originate in-country, the priority of this scale will change. unfortunately, this scenario is very unlikely because large projects, in view of nepal’s limited financial resources, would require as a pre-requisite power exports contracts with neighbouring countries. micro schemes are excellent with respect to technical, environmental and uncertainties, but fall behind on economic and social goals. small schemes perform best on technical, but poor on economic goals and yield an average performance on the rest of the goals. similarly, big schemes are excellent on economic, social and political goals, whereas medium schemes are the best performer on technical, social and political goals. this analysis also indicates that the country’s technical capacity is possibly capable of handling up to big scale schemes. in days to come, if funding is organized and modality of project development ensures more resources and benefits to circulate within the country, big schemes could become a priority very soon. ijahp article: nachtnebel, singh/ prioritizing hydropower development using analytical hierarchy process (ahp) a case study of nepal international journal of the analytic hierarchy process 329 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.253 technical goal environmental goal socio goal political goal economic goal uncertainties figure 5. prioritization with respect to various goals 5.3 sensitivity analysis in general, there are uncertainties in the data, models (e.g. transferring data into impacts) and in preferences. further, some of the information and data used may not be correctly understood, expressed or reviewed. allocations of weight to the goals and corresponding criteria might have errors because of subjectivity in its assessment. all these factors could be a major source of error and cause the end result to be different than it would otherwise be. it is thus important to check that slight variations of the parameters (weight allocated) don't have a large influence on the analysis results (rankings). thus, the objective of the sensitivity analysis is to determine the change in the alternative ranking with the change in the weight allocated to the goals and criteria. the sensitivity could be analysed only with the factor of one level below the objective. hence, the sensitivity analysis, another important feature of ahp application, will verify the trustworthiness of ranking obtained. as shown in figure 3 with respect to the economic goal whose weightage changed from the allocated 25% by +/5% to observe a change in ranking. although the ranking values are slightly changing it is not enough to make a change in the overall ranking obtained earlier. hence, the economic weight allocation is stable. if the weight is increased (changed) to 35%, then the ranking order is changed and big schemes would be the top priority as shown in the figure 6. this sensitivity could be performed directly by varying ijahp article: nachtnebel, singh/ prioritizing hydropower development using analytical hierarchy process (ahp) a case study of nepal international journal of the analytic hierarchy process 330 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.253 the weight allocated (in figure 3) to the goals one at a time to observe the overall impact on the ranking of alternatives. alternatively, the gradient sensitivity with respect to each goal could be analyzed separately to observe the influence on ranking order of alternatives. alternatively, the gradient sensitivity with respect to each goal could be analyzed separately to observe the influence on ranking order of alternatives. a gradient sensitivity of each and every criteria was reviewed with a change in weightage to determine the sensitive factors as shown in the case of the economic goal in adjacent figure 6. here, the economic goal seems sensitive to weight change if it exceeds 35% weightage and the decision or ranking of alternatives could be influenced. however, the presently allocated weight of 25% to economy is unlikely to change in the near future, but could receive higher weightage with a strengthened economy in the long run. similarly, we can vary the weightage allocated to other goals. it was found that varying the weightage with +/5% over the allocated one does not change the ranking except by varying the alternatives value slightly. hence, those weight allocations for the goals are also stable and the ranking obtained is trusted. figure 6. sensitivity with respect to economic factor 6. conclusions and recommendations the main objective of the study was to assist in the development of a sound strategy for hydropower development in nepal. ahp based on secondary information is an easy and reliable approach to prioritize hydropower planning. it is far more informative than a workshop or survey based approach. additionally, this approach helps to understand the relative prominence of alternatives with respect to set criteria in more detail with evidence and references. within a framework of six goals including 28 criteria applied for the five alternative scales of hydropower schemes medium scale power generation is the best option followed by large hydropower in nepal’s present context and immediate future. small hydropower is the third priority followed by mini and micro in the fourth ranked position. ijahp article: nachtnebel, singh/ prioritizing hydropower development using analytical hierarchy process (ahp) a case study of nepal international journal of the analytic hierarchy process 331 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.253 large hydropower of more than 1000 mw is at the least preferred in the nepalese context at the moment. among the considered goals, the economic goal is of the most important and likely to remain highly weighted. it may even increase further with a strengthened economy and neighbouring market expansion. as such change takes place slowly the prioritization order will remain constant for the near future (say the next fifteen years). though this factor is currently stable for a wide variation (15%-35%) it could change the priority in the long run which would put big hydropower on top and even enhance large hydropower in the priority order. this research methodology of secondary information based ahp application in hydropower prioritization is a new approach. it produces reliable results in the nepalese context and could be applied in a similar context. it could be of use for the researchers, professionals, planners and other hydropower stakeholders in nepal and similar countries. specifically, this analysis could help the hydropower policy maker in the country to opt for the appropriate policy and strategy to maximize the benefits from hydro resources. this exercise is important to conduct from time to time in order to assess the prioritization which may slowly change within the evolving country context. though the ahp applied in this study helped the prioritization of available hydropower alternatives in nepal, it is recommended to further crosscheck the results by using other scientific tools like promethee, electre etc. to validate the conclusions of the study. ijahp article: nachtnebel, singh/ prioritizing hydropower development using analytical hierarchy process (ahp) a case study of nepal international journal of the analytic hierarchy process 332 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.253 references agrawala, s., raksakulthai, v., aalst, m.v., larsen, p., smith, j. & reynolds, j. (2003). development and climate change in nepal: focus on water resources and hydropower. paris, france: organisation for economic co-operation and development. ahmad, s. & tahar, r.m., 2014. selection of renewable energy sources for sustainable development of electricity generation system using analytic hierarchy process: a case of malaysia. renewable energy, 63, 458–466. akash, b.a., mamlook, r. & mohsen, m.s. (1999). multi – criteria selection of electric power plants using analytical hierarchy process. electric power system research, 1999, 29-35. doi: 10.1016/s0378-7796(99)00004-8 balali, v., zahraie, b. & roozbahani, a. (2014). a comparison of ahp and promethee family decision making methods for selection of building structural system. american journal of civil engineering and architecture, 2(5), 149–159. available at: http://pubs.sciepub.com/ajcea/2/5/1/index.html [accessed october 1, 2014]. bergner, m. (2013). developing nepal’s hydroelectric resources: policy alternatives. university of virginia. bhatta, r.p. & khanal, s.n. (2010). environmental impact assessment system in nepal a study on policy and legal instruments in nepal. african journal of environmental science and technology, 4(9), 586–594. bhattarai, b. (2012). hydroelectricitybase for economic development. vidhyut, a biannual publication of nea. bhattarai, s. (2003). analytical hierarchy process for developing decisions: prospects and progress in nepal. in: proceedings of international symposium on the analytical hierarchy process, isahp, 7-9 august 2003, bali, indonesia. bhattarai, s. (1997). appropriate scale of hydropower development for nepal: an analytical hierarchy process approach. asian institute of technology (ait), master’s thesis no. ip-2-97. bhattarai, s. & fujiwara, o. (1995). evaluation of appropriate scale of hydropower development for nepal: analytical hierarchy process approach, infrastructure planning and management program, asian institute of technology, bangkok. bodin, l. & gass, s. (2004). exercise for teaching the analytical hierarchy process. informs transaction on education, 4(2), 1-13. ertay, t., kahraman, c. & kaya, i. (2013). evaluation of renewable energy alternatives using macbeth and fuzzy ahp multicriteria methods: the case of turkey. technological and economic development of economy, 19(1), 38-62. ijahp article: nachtnebel, singh/ prioritizing hydropower development using analytical hierarchy process (ahp) a case study of nepal international journal of the analytic hierarchy process 333 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.253 fast, t. & hansson s. (2013). another dam development project? development-forced displacement and resettlement in nepal. master’s thesis, lund university, center for east and south-east asian studies. foran, t. 2009b. making hydropower more sustainable? a sustainability measurement approach led by the hydropower sustainability assessment forum. policy brief. draft july 2009. user working paper wp-2009-14, chiang mai, thailand: unit for social and environmental research (user) chiang mai university. ganoulis, j. (2006). multi criteria decision analysis (mcda) for conflict resolution in sharing ground water resources. retrieved on 27 th feb 2014, http://www.inweb.gr guitouni, a. & martel, j.m. (1998). tentative guidelines to help choosing an appropriate mcda method. european journal of operational research, 109, 501-521. gunawardena, u.a.d.p. (2010). inequalities and externalities of power sector: a case of broadlands hydropower project in sri lanka. energy policy, 38, 726-734. hajkowicz, s, & collins, k. (2007). a review of multiple criteria analysis for water resources planning and management. water resource management, 21,1553-1566. haralambopoulos, d.a. & polatidis, h. (2003). renewable energy projects: structuring a multi-criteria group decision-making framework. renewable energy, 28, 961-973. icimod (2011). glacial lakes and glacial lakes outburst floods in nepal. international center for integrated mountain development, kathmandu, nepal. iha (2014). hydropower sustainability assessment protocol 2014. international hydropower association, retrieved 2/26/14, http://www.hydrosustainability.org/protocol.aspx jha, a.k. (2069b.s.). nepal’s hydropower now or never. vidhyut, a bi-annual publication of nea, 23(1), 86-87. jica (2013). nationwide master plan study on storage-type hydroelectric power development in nepal. nepal electricity authority, kathmandu, nepal. kuwar, s. (2070 b.s.) lokmarg connecting mid hill (in nepali). retrieved on 31 st jan 2013, http://kantipuronline.com/np/2070/10/17/full-story/383413.html ledec, g. & quintero, j.d. (2003). good dams and bad dams: environmental criteria for site selection of hydroelectric projects. the world bank. latin america and caribbean region sustainable development working paper, 16, 21. loken, e. (2007). use of multicriteria decision analysis methods for energy planning problems. renewable and sustainable energy reviews, 11, 1584-1595. http://www.inweb.gr/ http://www.hydrosustainability.org/protocol.aspx http://kantipuronline.com/np/2070/10/17/full-story/383413.html ijahp article: nachtnebel, singh/ prioritizing hydropower development using analytical hierarchy process (ahp) a case study of nepal international journal of the analytic hierarchy process 334 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.253 londono, a., (2005). ida guarantee paves renewed interest in private hydropower : the nam theun 2 project largest cross border project financing in east asia, washington dc.: world bank marttunen, m., sarkkula, j. & hämäläinen, r.p. (2010). special session on multi-criteria decision analysis (mcda) and its application opportunities in environmental planning and decision making. in regional workshop for coordination of research on hydropower development in the lower mekong basin. vientiane, 33. mathur, h.m. (2008). the future of large dams: dealing with social, environmental, institutional and political costs. hydro nepal: journal of water, energy and environment, 2, 1–3. mendoza, g.a. & martins, h. (2006). multi-criteria decision analysis in natural resource management: a critical review of methods and new modelling paradigms. forest ecology and management, 230,1–22. messerschmidt, d. (2008). an opportunities based approach to mitigating risks associated with infrastructure development projects. hydro nepal: journal of water, energy and environment, 3, 9-15. doi: 10.3126/hn.v3i0.1912 nachtnebel, h.p., eder, g. & bogardi, i. (1994). evaluation of criteria in hydropower utilization in the context of sustainable development, unesco symposium “water resource planning in changing world” karlsruhe. iv/13iv/24. nea (2012). janshakti bibaran 2068/2069. vidyut, bi annual publication of nea, 23(1), 88. nepal r. & jamash t. (2012). reforming small electricity under political instability: the case of nepal. energy policy, 40, 242–251. npc (2002). the tenth five year plan (2002-2007). national planning commission, government of nepal, kathmandu, nepal. nrb (2013). recent macro economic situation (english)—2069. nepal rastra bank, kathmandu, nepal. nsc (2012). seismic hazard map of nepalearthquake hazards programusgs, national seismological center, department of mines and geology. ozernoy, v.m. (1997). proceedings of the xith international conference on mcdm, 1–6 august 1994, coimbra, portugal. in joao climaco, ed. multicriteria analysis. verlin berline: springer berlin heidelberg, 103. panday,b. (2003). big hydro, big hanky-panky. nepali time. retrieved on 15th jan 2014. http://nepalitimes.com/news.php?id=4860 http://nepalitimes.com/news.php?id=4860 ijahp article: nachtnebel, singh/ prioritizing hydropower development using analytical hierarchy process (ahp) a case study of nepal international journal of the analytic hierarchy process 335 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.253 panthi, k. (2007). prioritizing and estimating hydropower construction risks: a case study of nyadi hydropower project. university of new mexico: nepal study center, 16. retrieved on 15 th april 2014, http://hdl.handle.net/1928/3298 pun, s.b. (2008). hydropower development in nepal: lessons from past models. hydro nepal journal of water, energy and environment, 1(2), 5-8. rac (1992). multi-criteria analysis as a resource assessment tool, research paper no. 6, march, resource assessment commission, canberra, australia. rajauriya, a. (2012). development of small hydropower projects in nepal, business development forum – hydropower special, 1, 6-12. rees, h.g., holmes, m.g.r., fry, m.j., young, a.r., pitson & d.g. (2006). environmental modelling & software, 21(7), 1001-1012. rosso, m., bottero, m., pomarico, s., la ferlita, s. & comino, e. (2014). integrating multicriteria evaluation and stakeholders analysis for assessing hydropower projects. energy policy, 67, 870-881. saaty, r.w. (1987). the analytic hierarchy process – what it is and how it is used. mathematical modelling, 9, 161–176. saaty, t.l. (1986). axiomatic foundation of the analytic hierarchy process. management science, 32(7), 841-855. saaty, t.l. (2007). fundamentals of decision making and priority theory with the analytic hierarchy process vol. vi of the ahp series (2 nd ed), pittsburgh: rws publications. shajari, s., bakhshoode, m. & soltani, g.r. (2008). suitability of multiple-criteria decision making simulations to study irrigation water demand: a case study in the doroudzan river basin, iran. american-eurasian journal of agricultural & environmental sciences, 2, 25-35. shrestha, r.b. (2012). what kind of projects are appropriate (in nepali). retrieved on 4 th may 2012, http://gorkhapatra.org.np/gopa.detail.php?article_id=65356cat_id=18. shrestha, r.s. (2014). hydropower financing & risk mitigation, downloaded on 10 th feb 2014, http://www.ratnasansar.com/2010/07/hydropower-financing-risk-mitigation.html. sovacool, b.k., dhakal, s., gippner, o. & barnbawale m.j. (2010). halting hydro: a review of the socio-technical barriers to hydroelectric power plants in nepal. energy, 36 (2011), 3468 – 3476. strin, l.z., & groselj, p. (2010). multiple criteria methods with focus on analytic hierarchy process and group decision making, croatian operational research review (crorr), 1, 1 -11. http://hdl.handle.net/1928/3298 http://gorkhapatra.org.np/gopa.detail.php?article_id=65356cat_id=18 http://www.ratnasansar.com/2010/07/hydropower-financing-risk-mitigation.html ijahp article: nachtnebel, singh/ prioritizing hydropower development using analytical hierarchy process (ahp) a case study of nepal international journal of the analytic hierarchy process 336 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.253 subramanian, n. & ramanathan, r. (2012). a review of application of analytical hierarchy process in operation management. international journal of production economics, 138(2), 215-241. supriyasilp, t., pongput, k. & boonyasirikul, t., 2009. hydropower development priority using mcdm method. energy policy, 37, 1866–1875. thapa, b., shrestha,r., dhakal,p. & thapa,b.s. (2005). problems of nepalese hydropower projects due to suspended sediments. journal of aquatic ecosystem health and management, 8(3), 251-257. thut, w., shrestha, r.s., dafflon, b. & aschwanden, h. (2011). water and hydropower in federal nepal: development and decision making from a comparative perspective. forum of federations canada. isbn: 978-0-9877517-1-3. toloie-eshlaghy, a. & homayonfar, m., 2011. mcdm methodologies and applications: a literature review from 1999 to 2009. research journal of international studies, (21), 86–137. unep (2007). dams and development project, a compendium of relevant practices for improved decisionmaking, planning and management of dams and their alternatives. nairobi, kenya: united nation environment programme. isbn: 978-92-807-2816-3 vučijak, b. et al., 2013. applicability of multicriteria decision aid to sustainable hydropower. applied energy, 101, 261–267. available at: http://www.sciencedirect.com/science/article/pii/s0306261912003960 [accessed october 21, 2014]. water and energy commission secretariat ( 2010). twenty-year-task-force-report 2010, government of nepal, kathmandu, nepal. ijahp news and events: mu/decision making in the time of corona virus: an interview with dr. leandro pecchia international journal of the analytic hierarchy process 172 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.755 decision making in the time of corona virus: an interview with dr. leandro pecchia enrique mu editor-in-chief, ijahp leandro pecchia at work at ijahp we are concerned for the well-being of our friends and colleagues around the world. in our concern, we have systematically contacted many of them to know how they are riding out this crisis. we recently contacted our colleague and long-time friend leandro pecchia. dr. pecchia, originally from italy, is an associate professor of biomedical engineering at the university of warwick, u. k., and he has been part of the ahp/anp community for a long time. he was program chair for the international symposium of the analytic hierarchy process (isahp) that took place in 2016 in london, u.k, and has used his decision-making expertise in the biomedical field, where he is considered a world expert. we were thrilled to find out that he is not only “fine and in a great mood”, but that he and his team at uw are actually very active despite the catastrophic circumstances engulfing the world. dr. pecchia is an expert in medical devices, artificial intelligence and decision making. he is also the elected president of the european society of biomedical engineering (i.e., eambes), treasurer of the clinical engineering division of the ifmbe, and secretary general of the iupesm, the global union society of medical physicists and biomedical engineers. in this role, dr. pecchia kindly shared with us his thoughts about the current crisis: ijahp news and events: mu/decision making in the time of corona virus: an interview with dr. leandro pecchia international journal of the analytic hierarchy process 173 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.755 “the first need during an emergency is getting appropriate knowledge and clear guidance. it has been more than 60 years since many of our countries have faced such an emergency in their mainland. in response to this need, eambes, ifmbe and iupesm members are supporting the world health organization (who) to define minimum requirements for essential medical devices, in-vitro diagnostics and personal protection equipment (ppe) to fight the covid-19 pandemic. for a list of essential medical devices being reviewed weekly you can see this: https://www.who.int/publications-detail/diseasecommodity-package---novel-coronavirus-(ncov). this emergency has revealed the frailty of our production and supply chain for medical devices and ppe. so many years of a profit-focused market has left our countries without endemic productions of critical devices and ppe. once this crisis is over, we will have to rethink our manufacturing models for critical devices and equipment, following principles of circular economy and local manufacturing. to respond now, during this pandemic, we are assisting manufacturers in the uk midlands and in italy, who are converting their production lines to start manufacturing ppes (e.g. masks, visor) and medical devices (ventilators). in particular, we are helping with the regulatory and design aspects, which in this period are critical. not only do we need to ensure the usual standard of quality, but we also need to consider making these devices even more resilient because if hospitals fail, we will need to safely operationalize these devices in field hospitals in tents. therefore, military and ragged standards become relevant too. meanwhile, we have to help our regulators define fast-track certification procedures. this is not easy, but all are responding very well including the fda, european commission and national institutes. moreover, this virus spreads as we breathe. therefore, we need to remotely monitor patients, suspected infected people, and their relatives. we need to control their health trajectories and react promptly and appropriately. in order to respond to this need, we have supported telemedicine companies adapting their platforms for covid monitoring of patients and self-isolated citizens. this is now serving hundreds of self-isolated patients and citizens in the rome region in italy for three local national health service (nhs) trusts, and discussion is going on about opening the same platform to uk nhs trusts for high-risk populations (e.g., cancer patients). finally, we need to be ready for next year. the scientific community has been firmly clear in this regard. sars-cov-2 is not a transient problem. we will need to coexist with this virus forever, hoping that new cost-effective vaccines and therapies will become available that will mitigate the risks. while vaccines and therapies are under development, we need to be ready to early-detect covid-19 from symptoms. this is actually the challenge for next winter. in response to this need, we are working to develop an ai system to detect pneumonia from symptoms (e.g., coughing, pain etc.). i hope we will have something like an app, which will enable people to self-assess their health condition and distinguish covid-19 from regular flu or other seasonal influences. we are working on a database that is not covid-specific because data on covid are not available yet. we are using a dataset from bosnia-herzegovinian and learning how to distinguish pneumonia from bronchitis, asthma and normal influences, https://www.who.int/publications-detail/disease-commodity-package---novel-coronavirus-(ncov) https://www.who.int/publications-detail/disease-commodity-package---novel-coronavirus-(ncov) ijahp news and events: mu/decision making in the time of corona virus: an interview with dr. leandro pecchia international journal of the analytic hierarchy process 174 vol. 12 issue 1 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i1.755 but i believe the work we will produce can then be adapted for covid if/when data becomes available. this is not sufficient, but this is all we can do in this moment. in conclusion, this pandemic is not to be taken lightly. it requires us all to respond with all our strength. it requires responsible thinking, responsible action and also responsible silence, when we do not have specific competences. we have seen do-it-yourself approaches to ppe and medical devices that are potentially harmful, apart from being useless, and talk shows full of useless and contradictory opinions. this is not the time for opinions. this is the time for responsible thinking, responsible action, and responsible silence!” further information on the telemedicine application for covid-19 can be found here: https://www.adilife.net/covid-19/ leandro pecchia and colleagues https://www.adilife.net/covid-19/ ijahp article: bolturk, kahraman/interval-valued neutrosophic ahp with possibility degree method international journal of the analytic hierarchy process 431 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.545 interval-valued neutrosophic ahp with possibility degree method eda bolturk cengiz kahraman istanbul technical university department of industrial engineering, 34367 macka, besiktas, istanbul, turkey abstract the analytic hierarchy process (ahp) is one of the most widely used methods in multi criteria decision making (mcdm) in many areas. the method has been extended with hesitant fuzzy sets, intuitionistic fuzzy sets, neutrosophic sets, and type -2 fuzzy sets etc. these extended methods can consider the vagueness in decision making problems through different definitions of membership functions. each of them tries to increase the effectiveness of ahp under uncertainty. decision makers can fully express their judgments through neutrosophic sets (ns) since ns are based on three independent parameters, truthiness (t), indeterminancy (i) and falsity (f), providing a distinction between a ‘relative truth’ and an ‘absolute truth’. in this paper, we employ the possibility degree method for ranking interval numbers in our neutrosophic ahp approach by utilizing ns’ representation power. besides, we employ interval-valued ns since a larger domain for the definition of t, i, and f is provided. pairwise comparison matrices can be filled in by using linguistic terms such as weakly more important, moderately more important or extremely important. then, we obtain the relative importance degrees of criteria by using the possibility degree method. in order to show the effectiveness of our method, a mcdm application is given in energy planning. comparative and sensitivity analyses are also presented in the paper. keywords: multi criteria decision making; energy; interval-valued neutrosophic sets; possibility degree method; ahp 1. introduction neutrosophic sets were introduced by smarandache (1995) to avoid the incapabilities of ordinary fuzzy sets and their extensions. neutrosophic sets are defined as the sets where each element of the universe has a degree of truthiness, indeterminacy and falsity which are between ]-0,1+[ the non-standard unit interval (rivieccio, 2008) . in neutrosophic sets, uncertainty is represented as truth (t: degrees of belongingness) and falsity (f: nonbelongingness) values; inconsistency is represented as indeterminacy (i: degree of hesitancy) value. since indeterminacy value is assigned for distinguishing relativity and absoluteness of a decision maker’s preferences in neutrosophic sets, it is said to be superior to ordinary fuzzy sets and its extensions. ijahp article: bolturk, kahraman/interval-valued neutrosophic ahp with possibility degree method international journal of the analytic hierarchy process 432 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.545 there are some neutrosophic ahp methods developed and published in the literature. abdel-basset et al. (2018a) used a neutrosophic ahp with swot analysis in strategic planning. bolturk and kahraman (2018) introduced an interval-valued neutrosophic ahp method and proposed the interval-valued neutrosophic ahp with cosine similarity measure. abdel-basset et al. (2018b) proposed an integrated model consisting of a neutrosophic decision making trial and evaluation laboratory (dematel) technique with ahp in supply chain management. abdel-basset et al. (2017a) used the ahp method with neutrosophic sets in multi criteria group decision making for candidate selection. abdel-basset et al. (2017b) conducted the integration of ahp into a delphi framework under a neutrosophic environment and introduced a new technique for checking consistency and calculating the degree of the expert’s opinions. radwan et al. (2016) developed a novel hybrid neutrosophic ahp approach in learning management systems in decision making to handle indeterminacy of information. in this paper, we present an interval-valued neutrosophic ahp method with the possibility degree (pd) method for energy planning. possibility degree provides an objective scoring procedure in pairwise comparisons instead of subjective scoring. to the best of our knowledge, the ahp method with interval-valued neutrosophic sets has not yet been proposed. neutrosophic ahp enables decision makers to take into account their hesitancy in defining a membership function. neutrosophic logic is a generalization of all other logics. its definition needs more parameters and it presents more information about the considered problem with its t, i, and f elements. the proposed neutrosophic ahp method can consider the optimistic (o), pessimistic (p) and neutral (n) points of views of decision makers (dms). the rest of the paper is constructed as follows. in section 2, fuzzy extensions of ahp are summarized. the preliminaries of neutrosophic sets are detailed in section 3. the concept of possibility degree method is given in section 4. the proposed interval-valued neutrosophic ahp with possibility degree (ivnahp-pd) method is presented in section 5. in order to show their effectiveness, these methods are used in the selection of the best renewable energy alternative. finally, the conclusions are given in section 7. 2. fuzzy extensions of ahp the ahp method, which is the most-used multi-criteria decision making method, has been extended with different fuzzy sets such as intuitionistic fuzzy sets, hesitant fuzzy sets, type-2 fuzzy sets, pythagorean fuzzy sets etc. the fuzzy extensions of ahp can be summarized as follows. atanassov’s (1983) intuitionistic fuzzy sets which incorporate the degree of hesitation to the definition of a membership function have been frequently used in mcdm problems. the earliest work in fuzzy ahp was introduced by van laarhoven & pedrycz (1983) using triangular membership functions for pairwise comparisons. buckley (1985) determined the fuzzy priorities of pairwise comparison ratios whose membership functions are trapezoidal fuzzy numbers. chang (1996) introduced a new approach for handling fuzzy ahp with the use of triangular fuzzy numbers and the synthetic extent values of pairwise comparisons. lee et al. (2005) proposed a fuzzy ahp model in order to evaluate performance. zeng et al. (2007) used fuzzy ahp in order to structure risk factors. hesitant fuzzy sets proposed by torra ijahp article: bolturk, kahraman/interval-valued neutrosophic ahp with possibility degree method international journal of the analytic hierarchy process 433 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.545 (2010) have been used in the literature. kahraman et al. (2014) introduced an interval type-2 fuzzy ahp method together with a new ranking method for type-2 fuzzy sets and applied it to a supplier selection problem. oztaysi et al. (2015) developed a hesitant fuzzy ahp method involving multi-expert’s linguistic evaluations aggregated by an ordered weighted averaging operator. the developed method is applied to a multi-criteria supplier selection problem. buyukozkan et al. (2016) developed a framework that integrates the intuitionistic fuzzy ahp and intuitionistic fuzzy vikor. deepika and kannan (2016) proposed an intuitionistic fuzzy ahp method in order to check the consistency for an automatic repairing procedure. kahraman et al. (2016) presented a new hesitant fuzzy ahp method in order to solve a warehouse location selection problem for a turkish humanitarian relief organization by using hesitant fuzzy preference information. the aim of this study is to eliminate the decision maker’s hesitancy in the evaluation. zhu et al. (2016) proposed hesitant ahp which can consider the hesitancy experienced by the decision makers. type-2 fuzzy sets are the extension of type-1 fuzzy sets. abdullah and najib (2016) studied a version of interval type-2 fuzzy ahp and realized its implication to the computational procedure. erdogan and kaya (2016) proposed a mcdm methodology consisting of three techniques which are delphi methodology, type-2 fuzzy ahp and type-2 fuzzy topsis and applied it to real case work in order to switch the focus to exhaust gases from the increasing number of motor vehicles as the major factor of air pollution in istanbul. these are just a sampling of papers using ahp with type-2 fuzzy sets. the pythagorean fuzzy ahp method was developed by ilbahar et al. (2018) in the literature. it presents a novel approach to risk assessment for occupational health and safety using pythagorean fuzzy ahp and a fuzzy inference system. senvar (2018) proposed a systematic approach based on hesitant fuzzy ahp to deal with incomplete information in complex customer oriented mcdm problems. bolturk and kahraman (2018) developed an interval-valued neutrosophic ahp method using a cosine similarity measure. in figure 1, we graphically present the neutrosophic ahp papers published in the literature with respect to the types of the documents. 78% of the documents are articles, while the rest of them are conference papers. figure 1 document type of neutrosophic ahp papers 78% 22% article conference paper ijahp article: bolturk, kahraman/interval-valued neutrosophic ahp with possibility degree method international journal of the analytic hierarchy process 434 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.545 figure 2 presents the subject areas of neutrosophic ahp papers and notes that computer science is the area most commonly discussed representing 38% of papers. there are 9 papers related to neutrosophic ahp published in the scopus database. figure 2 document types of neutrosophic ahp papers. 3. preliminaries we give the basic notions and operations on interval-valued neutrosophic sets in this section. 3.1 neutrosophic sets in neutrosophic sets literature, the first specific symbol for a neutrosophic set was used by bolturk and kahraman (2018). the symbol �̃� represents a neutrosophic fuzzy set. the three dots represent the elements of a neutrosophic set; t, i, f and tilde represents that it is also a fuzzy set. definition 1. (smarandache, 1998) let e be a universe. a neutrosophic set �̃� in e is characterized by a truth-membership function ta, an indeterminacy-membership function ia, and a falsity-membership function fa. ta (𝑥), ia (𝑥) and fa (𝑥) are real standard elements of [0, 1]. a neutrosophic set �̃� can be given by equation 1: �̃� = {< 𝑥, (𝑇𝐴 (𝑥), 𝐼𝐴 (𝑥), 𝐹𝐴 (𝑥)) >: 𝑥 ∈ 𝐸, (𝑇𝐴 (𝑥), 𝐼𝐴 (𝑥), 𝐹𝐴 (𝑥) ∈] −0,1[+)}. (1) there is no restriction on the sum of ta (𝑥), ia (𝑥) and fa (𝑥), so that 0 − ≤ 𝑇𝐴 (𝑥) + 𝐼𝐴 (𝑥) + 𝐹𝐴 (𝑥) ≤ 3 +. definition 2. (biswas et al., 2014) let e be a universe. a single valued neutrosophic set a in e is characterized by a truth-membership function 𝑇𝐴, an indeterminacy-membership function 𝐼𝐴 and a falsity-membership function 𝐹𝐴. 𝑇𝐴, 𝐼𝐴, and 𝐹𝐴 are real standard elements of [0,1]. it can be written as ijahp article: bolturk, kahraman/interval-valued neutrosophic ahp with possibility degree method international journal of the analytic hierarchy process 435 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.545 �̃� = {< 𝑥, (𝑇𝐴 (𝑥), 𝐼𝐴 (𝑥), 𝐹𝐴 (𝑥)) >: 𝑥 ∈ 𝐸, 𝑇𝐴 (𝑥), 𝐼𝐴 (𝑥), 𝐹𝐴 (𝑥) ∈ [0,1]}. (2) there is no restriction on the sum of ta (𝑥); ia (𝑥) and fa (𝑥), so 0 − ≤ 𝑇𝐴 (𝑥) + 𝐼𝐴 (𝑥) + 𝐹𝐴 (𝑥) ≤ 3 +. definition 3. (wang et al., 2010) let x be a space of points (objects) with a generic element in x denoted by x. a neutrosophic set �̃� in x characterized by a truth – membership function 𝑇𝐴, an indeterminacy-membership function 𝐼𝐴 and a falsitymembership function 𝐹𝐴. 𝑇𝐴, 𝐼𝐴 and, 𝐹𝐴are real standard or non-standard subsets of ]0−, 1+[. these are shown by equations 3-5: 𝑇𝐴: x → ]0 −, 1+[. (3) 𝐼𝐴: x →]0 −, 1+[. (4) 𝐹𝐴: x →]0 −, 1+[. (5) 3.2 interval-valued neutrosophic sets definition 4. x̃⃛j = 〈[tj l, tj u], [ ij l, ij u], [fj l, fj u]〉 is a collection of interval-valued neutrosophic numbers where 𝑗 = 1,2, … , 𝑛 and n is the number of decision makers. definition 5. (li et al., 2016) let x be a universe of discourse. an interval-valued neutrosophic set �̃� in x is independently defined by a truth-membership function 𝑇𝑁(𝑥), an indeterminacy-membership function 𝐼𝑁(𝑥), and a falsity-membership function 𝐹𝑁(𝑥) for each 𝑥 ∈ 𝑋, where 𝑇𝑁(𝑥) = [𝑇𝑁(𝑥) 𝐿 , 𝑇𝑁(𝑥) 𝑈 ] ⊆ [0,1], 𝐼𝑁(𝑥) = [𝐼𝑁(𝑥) 𝐿 , 𝐼𝑁(𝑥) 𝑈 ] ⊆ [0,1], and 𝐹𝑁(𝑥) = [𝐹𝑁(𝑥) 𝐿 , 𝐹𝑁(𝑥) 𝑈 ] ⊆ [0,1]. they also meet the condition0 ≤ 𝑇𝑁 𝐿(𝑥) + 𝐼𝑁 𝐿 (𝑥) + 𝐹𝑁 𝐿(𝑥) ≤ 3. so, the interval-valued neutrosophic set ñ⃛ can be given by equation 6: ñ⃛ = {〈x, [ tn l(x), tn u(x)], [ in l (x), in u(x)], [ fn l (x), fn u(x)]〉|x ∈ x}. (6) definition 6. (zhang et al. 2014) let �̃� = 〈[ 𝑇𝑎 𝐿, 𝑇𝑎 𝑈], [ 𝐼𝑎 𝐿, 𝐼𝑎 𝑈], [ 𝐹𝑎 𝐿, 𝐹𝑎 𝑈]〉 and �̃� = 〈[ 𝑇𝑏 𝐿, 𝑇𝑏 𝑈], [ 𝐼𝑏 𝐿, 𝐼𝑏 𝑈], [ 𝐹𝑏 𝐿, 𝐹𝑏 𝑈]〉 be two interval-valued neutrosophic numbers. their relations and arithmetic operations are given by equations 7-10: 1. �̃�c = 〈[fa l, fa u], [1 − ia u, 1 − ia l], [ta l, ta u]〉 (7) 2. �̃� ⊆ �̃� if and only if ta l ≤ tb l; ta u ≤ tb u; ia l ≥ ib l, ia u ≥ ib u; fa l ≥ fb l, fa u ≥ fb u (8) 3. �̃� = �̃� if and only if �̃� ⊆ �̃� and �̃� ⊆ �̃�. (9) 4. �̃�⨁�̃� = 〈[ta l + tb l − ta ltb l, ta u + tb u − ta utb u], [ia lib l, ia uib u], [fa lfb l, fa ufb u]〉 (10) definition 7. the deneutrosophication function for an interval-valued neutrosophic number is given in equation 1 (bolturk & kahraman, 2018): ijahp article: bolturk, kahraman/interval-valued neutrosophic ahp with possibility degree method international journal of the analytic hierarchy process 436 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.545 𝔇(x̃⃛j) = ( (tj 𝐿+tj u) 2 + (1 − (ij l+ij u) 2 ) ∗ (ij u) − ( fj l+fj u 2 ) ∗ (1 − fj u)) (11) where x̃⃛j = 〈[tj l, tj u], [ ij l, ij u], [fj l, fj u]〉. 3.3 possibility degree method wu et al. (2013) proposed the possibility degree method to show the ranking between two numbers and the formulization is summarized as follows: let 𝑤𝑖and 𝑤𝑗be interval numbers that are given as 𝑤𝑖 = [𝑤𝑖 −, 𝑤𝑖 +] and 𝑤𝑗 = [𝑤𝑗 −, 𝑤𝑗 +] , respectively. the possibility degree of 𝑤𝑖 ≥ 𝑤𝑗 is given in equation 12. 𝑝(𝑤𝑖 ≥ 𝑤𝑗) = 𝑚𝑖𝑛{𝐿𝑤𝑖 +𝐿𝑤𝑗 ,𝑚𝑎𝑥(𝑤𝑖 +−𝑤𝑗 −,0)} 𝐿𝑤𝑖 +𝐿𝑤𝑗 (12) where 𝐿𝑤𝑖 = 𝑤𝑖 + − 𝑤𝑖 − and 𝐿𝑤𝑗 = 𝑤𝑗 + − 𝑤𝑗 − and 𝑝𝑖𝑗 ≥ 0, 𝑝𝑖𝑗 + 𝑝𝑗𝑖 = 1, 𝑝𝑖𝑖 = 1/2. 4. ivnahp-pd method the steps of the proposed ivnahp-pd method are given in the following: step 1: define the hierarchy of the problem. in figure 3, the three level hierarchy of the considered problem in this paper is illustrated. figure 3 a three level hierarchy step 2: assign the interval-valued neutrosophic preferences for pairwise comparison matrices in the hierarchy. equation 13 shows an interval-valued neutrosophic pairwise comparison matrix. the scale given in table 1 is used to assign the interval-valued neutrosophic preferences. �̃� = [ [0.5,0.5], [0.5,0.5], [0.5,0.5] [ t12 l (x), t12 u (x)], [ in l (x), in u(x)], [ fn l (x), fn u(x)] [ t13 l (x), t13 u (x)], [ i13 l (x), i13 u (x)], [ f13 l (x), f13 u (x)] [ t21 l (x), t21 u (x)], [ i21 l (x), i21 u (x)], [ f21 l (x), f21 u (x)] [0.5,0.5], [0.5,0.5], [0.5,0.5] [ t23 l (x), t23 u (x)], [ i23 l (x), i23 u (x)], [ f23 l (x), f23 u (x)] [ t31 l (x), t31 u (x)], [ i31 l (x), i31 u (x)], [ f31 l (x), fnm u (x)] [ t32 l (x), t32 u (x)], [ i32 l (x), i32 u (x)], [ f32 l (x), f32 u (x)] [0.5,0.5], [0.5,0.5], [0.5,0.5] ] (13) ijahp article: bolturk, kahraman/interval-valued neutrosophic ahp with possibility degree method international journal of the analytic hierarchy process 437 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.545 table 1 linguistic terms and their neutrosophic numerical values linguistic term neutrosophic sets equally important ([0.5,0.5], [0.5,0.5], [0.5,0.5]) weakly more important ([0.50,0.60], [0.35,0.45], [0.40,0.50]) moderate importance ([0.55,0.65], [0.30,0.40], [0.35,0.45]) moderately more important ([0.60,0.70], [0.25,0.35], [0.30,0.40]) strong importance ([0.65,0.75], [0.20,0.30], [0.25,0.35]) strongly more important ([0.70,0.80], [0.15,0.25], [0.20,0.30]) very strong importance ([0.75,0.85], [0.10,0.20], [0.15,0.25]) very strongly more important ([0.80,0.90], [0.05,0.10], [0.10,0.20]) extreme importance ([0.90,0.95], [0,0.05], [0.05,0.15]) extremely high importance ([0.95,1.0], [0.0,0.0], [0.0,0.10]) absolutely more important ([1.0,1.0], [0.0,0.0], [0.0,0.0]) step 3: construct the interval-valued neutrosophic preference relation matrix �̃� = (�̃�𝑖𝑗)𝑛×𝑛 . step 4: obtain the score judgment matrix 𝑆𝑛×𝑛. consider the neutrosophic set �̃� = 〈[ 𝑇𝑎 𝐿, 𝑇𝑎 𝑈], [ 𝐼𝑎 𝐿, 𝐼𝑎 𝑈], [ 𝐹𝑎 𝐿, 𝐹𝑎 𝑈]〉. for a neutral point of view: the lower limit of the closed interval: 𝑆𝑎,𝑁 𝐿 = 𝑇𝑎 𝐿 − 𝐹𝑎 𝑈 − (𝐼𝑎 𝑈 − 𝐼𝑎 𝐿) 2⁄ (14) the upper limit of the closed interval: 𝑆𝑎,𝑁 𝑈 = 𝑇𝑎 𝑈 − 𝐹𝑎 𝐿 + (𝐼𝑎 𝑈 − 𝐼𝑎 𝐿) 2⁄ (15) for an optimistic point of view: the lower limit of the closed interval: 𝑆𝑎.𝑂 𝐿 = 𝑇𝑎 𝐿 − 𝐹𝑎 𝑈 + (𝐼𝑎 𝑈 − 𝐼𝑎 𝐿) (16) the upper limit of the closed interval: 𝑆𝑎,𝑂 𝑈 = 𝑇𝑎 𝑈 − 𝐹𝑎 𝐿 + (𝐼𝑎 𝑈 − 𝐼𝑎 𝐿) (17) for a pessimistic point of view: the lower limit of the closed interval: 𝑆𝑎,𝑃 𝐿 = 𝑇𝑎 𝐿 − 𝐹𝑎 𝑈 − (𝐼𝑎 𝑈 − 𝐼𝑎 𝐿) (18) the upper limit of the closed interval: 𝑆𝑎,𝑃 𝑈 = 𝑇𝑎 𝑈 − 𝐹𝑎 𝐿 − (𝐼𝑎 𝑈 − 𝐼𝑎 𝐿) (19) ijahp article: bolturk, kahraman/interval-valued neutrosophic ahp with possibility degree method international journal of the analytic hierarchy process 438 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.545 step 5: obtain the interval neutrosophic multiplicative matrices �̃� = [�̃�𝑖𝑗]𝑛×𝑛 where �̃�𝑖𝑗 = 10 �̃�𝑖𝑗 for each type of points of view. then, we have �̃�𝑁 = [�̃�𝑖𝑗,𝑅𝑁]𝑛×𝑛, �̃�𝑂 = [�̃�𝑖𝑗,𝑅𝑇]𝑛×𝑛, and �̃�𝑃 = [�̃�𝑖𝑗,𝑅𝐴]𝑛×𝑛. step 6: determine the priority vectors of the matrices �̃� = [�̃�𝑖𝑗]𝑛×𝑛by using equation 20 (wu et al., 2013). then, we have �̃�𝑖 = [ ∑ �̃�𝒊𝒋 𝑳𝒏 𝒋=𝟏 ∑ ∑ �̃�𝒊𝒋 𝑼𝒏 𝒋=𝟏 𝒏 𝒊=𝟏 , ∑ �̃�𝒊𝒋 𝑼𝒏 𝒋=𝟏 ∑ ∑ �̃�𝒊𝒋 𝑳𝒏 𝒋=𝟏 𝒏 𝒊=𝟏 ] (20) then, we have �̃�𝑖,𝑁 = [ ∑ �̃�𝑖𝑗,𝑅𝑁 𝐿𝑛 𝑗=1 ∑ ∑ �̃�𝑖𝑗,𝑅𝑁 𝑈𝑛 𝑗=1 𝑛 𝑖=1 , ∑ �̃�𝑖𝑗,𝑅𝑁 𝑈𝑛 𝑗=1 ∑ ∑ �̃�𝑖𝑗,𝑅𝑁 𝐿𝑛 𝑗=1 𝑛 𝑖=1 ] (21) �̃�𝑖,𝑂 = [ ∑ �̃�𝑖𝑗,𝑅𝑇 𝐿𝑛 𝑗=1 ∑ ∑ �̃�𝑖𝑗,𝑅𝑇 𝑈𝑛 𝑗=1 𝑛 𝑖=1 , ∑ �̃�𝑖𝑗,𝑅𝑇 𝑈𝑛 𝑗=1 ∑ ∑ �̃�𝑖𝑗,𝑅𝑇 𝐿𝑛 𝑗=1 𝑛 𝑖=1 ] (22) �̃�𝑖,𝑃 = [ ∑ �̃�𝑖𝑗,𝑅𝐴 𝐿𝑛 𝑗=1 ∑ ∑ �̃�𝑖𝑗,𝑅𝐴 𝑈𝑛 𝑗=1 𝑛 𝑖=1 , ∑ �̃�𝑖𝑗,𝑅𝐴 𝑈𝑛 𝑗=1 ∑ ∑ �̃�𝑖𝑗,𝑅𝐴 𝐿𝑛 𝑗=1 𝑛 𝑖=1 ] (23) step 7: construct the possibility degree matrix 𝑃 = (𝑝𝑖𝑗)𝑚×𝑛 by comparing the intervals in step 6. to do this, use equation 11. then, we have 𝑃𝑁 = (𝑝𝑖𝑗,𝑁)𝑛×𝑛 , 𝑃𝑂 = (𝑝𝑖𝑗,𝑂)𝑛×𝑛 , and 𝑃𝑃 = (𝑝𝑖𝑗,𝑃)𝑛×𝑛 . step 8: calculate the set of weights for each possibility degree matrix by using equation 24. 𝑤𝑖 = 1 𝑛 (∑ 𝑝(𝑤𝑖 ≥ 𝑤𝑗) + 𝑛 2 − 1𝑛𝑗=1 ) (24) where 𝑝(𝑤𝑖 ≥ 𝑤𝑗) ≥ 0 and 𝑝(𝑤𝑖 ≥ 𝑤𝑖) = 1 2 . then, we have 𝑤𝑖,𝑁 = 1 𝑛 (∑ 𝑝𝑁(𝑤𝑖 ≥ 𝑤𝑗) + 𝑛 2 − 1𝑛𝑗=1 ) (25) 𝑤𝑖,𝑇 = 1 𝑛 (∑ 𝑝𝑇(𝑤𝑖 ≥ 𝑤𝑗) + 𝑛 2 − 1𝑛𝑗=1 ) (26) 𝑤𝑖,𝐴 = 1 𝑛 (∑ 𝑝𝐴(𝑤𝑖 ≥ 𝑤𝑗) + 𝑛 2 − 1𝑛𝑗=1 ) (27) step 9: apply steps 1-8 for each pairwise comparison matrix in the hierarchy. then, calculate the overall weights of the alternatives based on the sets of weights obtained in step 8. step 10: select the alternative with the highest value with respect to each case and then make your decision. equation 11 can be used for this purpose. ijahp article: bolturk, kahraman/interval-valued neutrosophic ahp with possibility degree method international journal of the analytic hierarchy process 439 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.545 5. application: selection among alternative renewable energy sources under a vague environment, evaluation of renewable energy alternatives requires different criteria and alternatives to be considered. this paper presents a hypothetical problem about selecting a renewable energy production system in turkey. the investor wants to invest in one of three renewable energy alternatives namely biomass, solar power and ocean energy. these three options are based on the geographical conditions in which the investor lives. in the considered renewable energy investment decision making problem, there are three alternatives and four evaluation criteria. the decision makers (dms) are composed of an executive (dm1) from an energy company, and two academicians (dm2, dm3) from the energy institute of a university. the alternatives are biomass energy (be), solar power energy (spe) and ocean energy (oe). based on the literature review, four evaluation criteria are determined as c1land cost, c2sustainability, c3environment, c4operating cost (lee & chang, 2018). table 2 presents the compromised pairwise comparison matrix of criteria filled with linguistic terms. step 1: the hierarchy of the renewable energy selection problem is illustrated in figure 4 figure 4 hierarchical structure of application step 2: the compromised pairwise comparison matrix for the main criteria is given in table 2. table 2 compromised pairwise comparison matrix of the criteria land cost sustainability environment operating cost land cost equal importance sustainability extremely high importance equal importance moderately more importance strong importance environment strongly more importance equal importance moderate importance best renewable energy selection land cost biomass energy solar power energy ocean energy sustainability biomass energy solar power energy ocean energy environment biomass energy solar power energy ocean energy operating cost biomass energy solar power energy ocean energy ijahp article: bolturk, kahraman/interval-valued neutrosophic ahp with possibility degree method international journal of the analytic hierarchy process 440 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.545 operating cost weakly more importance equal importance step 3: assign the interval-valued neutrosophic preferences for the pairwise comparison matrix of the criteria as given in table 3. table 3 interval-valued neutrosophic preferences land cost sustainability environment operating cost land cost [0.5,0.5], [0.5,0.5], [0.5,0.5] [0.0,0.10], [0.0,0.0], [0.95,1.0] [0.20,0.30], [0.15,0.25], [0.70,0.80] [0.40,0.50], [0.35,0.45], [0.50,0.60] sustainab ility [0.95,1.0],[0.0,0.0], [0.0,0.10] [0.5,0.5], [0.5,0.5], [0.5,0.5] [0.60,0.70], [0.25,0.35], [0.30,0.40] [0.65,0.75], [0.20,0.30], [0.25,0.35] environm ent [0.70,0.80], [0.15,0.25], [0.20,0.30] [0.30,0.40], [0.25,0.35], [0.60,0.70] [0.5,0.5], [0.5,0.5], [0.5,0.5] [0.55,0.65], [0.30,0.40], [0.35,0.45] operating cost [0.50,0.60], [0.35,0.45], [0.40,0.50] [0.25,0.35], [0.20,0.30], [0.65,0.75] [0.35,0.45], [0.30,0.40], [0.55,0.65] [0.5,0.5], [0.5,0.5], [0.5,0.5] step 4: calculate the score judgement matrix �̃� = (�̃�𝑖𝑗)𝑛×𝑛 and the interval multiplicative matrix �̃� = (�̃�𝑖𝑗)𝑛×𝑛, where �̃�𝑖𝑗 = 10 �̃�𝑖𝑗 . n, o, and p represent the neutral case, optimistic case, and pessimistic case, respectively. for the neutral point of view, three experts construct the following pairwise comparison matrices: �̃�𝑁 = [ [0,0] [−1, −0.85] [−0.675, −0.325] [−0.275,0.075] [0.85, 1] [0,0] [0.15, 0.45] [0.25,0.55] [0.35, 0.65] [−0.05, 0.25] [−0.45, −0.15] [−0.55, −0.25] [0,0] [−0.35, −0.05] [0.05, 0.35] [0,0] ] �̃�𝑂 = [ [0,0] [−1, −0.85] [−0.45, −0.25] [−0.05,0.15] [0.85, 1] [0,0] [0.3, 0.5] [0.4,0.6] [0.5, 0.7] [0.1, 0.3] [−0.3, −0.1] [−0.4, −0.2] [0,0] [−0.2,0] [0.2, 0.4] [0,0] ] �̃�𝑃 = [ [0,0] [−1, −0.85] [−0.75, −0.55] [−0.35, −0.15] [0.85, 1] [0,0] [0.25, 0.3] [0.35,0.4] [0.46, 0.5] [0.04, 0.1] [−0.35, −0.3] [−0.45, −0.4] [0,0] [−0.26, −0.2] [0.14, 0.2] [0,0] ] ijahp article: bolturk, kahraman/interval-valued neutrosophic ahp with possibility degree method international journal of the analytic hierarchy process 441 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.545 step 5: obtain the interval neutrosophic multiplicative matrices for the main criteria. �̃�𝑁 = [ [1,1] [0.1,0.141] [0.211,0.473] [0.531, 1.189] [7.08, 10] [1, 1] [1.413,2.818] [1.778, 3.548] [2.239, 4.467] [0.891, 1.778] [0.355, 0.708] [0.282, 0.562] [1, 1] [0.447, 0.891] [1.122,2.239] [1,1] ] �̃�𝑂 = [ [1,1] [0.1,0.141] [0.355, 0.562] [0.891,1.413] [7.08, 10] [1,1] [1.995, 3.162] [2.521,3.981] [3.16, 5.01] [1.26, 1.99] [0.501, 0.794] [0.398, 0.631] [1,1] [0.631,1] [1.585, 2.512] [1,1] ] �̃�𝑃 = [ [1, 1] [0.1,0.141] [0.178, 0.282] [0.447,0.708] [7.08, 10] [1, 1] [1.761, 1.995] [2.24,2.512] [2.86, 3.16] [1.1, 1.26. ] [0.442, 0.501] [0.355, 0.398] [1,1] [0.553,0.631] [1.388, 1.585] [1,1] ] step 6: obtain the priority vectors of the interval multiplicative matrix and normalize them. �̃�𝑁 = (0.148, 0.368, 0.286,0.199) 𝑇 �̃�𝑂 = (0.143,0.372,0.292,0.193) 𝑇 �̃�𝑃 = (0.125,0.419,0.247,0.208) 𝑇 step 7: we ranked the criteria based on priority vector values: sustainability > environment > operating cost > land cost. the sustainability criterion is the most effective one. step 8: the pairwise comparison matrices of alternatives with respect to each main criterion are given in table 4. ijahp article: bolturk, kahraman/interval-valued neutrosophic ahp with possibility degree method international journal of the analytic hierarchy process 442 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.545 table 4 pairwise comparison matrices for alternatives biomass energy solar power energy ocean energy s u st a in a b il it y ( s ) biomass energy [0.5,0.5], [0.5,0.5], [0.5,0.5] [0.05,0.15], [0,0.05], [0.90,0.95] [0.0,0.10], [0.0,0.0], [0.95,1.0] solar power energy [0.90,0.95], [0,0.05], [0.05,0.15] [0.5,0.5], [0.5,0.5], [0.5,0.5] [0.30,0.40], [0.25,0.35],[0.60,0.70] ocean energy [0.95,1.0], [0.0,0.0], [0.0,0.10] [0.60,0.70], [0.25,0.35], [0.30,0.40] [0.5,0.5], [0.5,0.5], [0.5,0.5] e n v ir o n m e n ta l (e ) biomass energy [0.5,0.5], [0.5,0.5], [0.5,0.5] [0.25,0.35], [0.20,0.30], [0.65,0.75] [0.10,0.20], [0.05,0.10],[0.80,0.90] solar power energy [0.65,0.75], [0.20,0.30], [0.25,0.35] [0.5,0.5], [0.5,0.5], [0.5,0.5] [0.15,0.25], [0.10,0.20],[0.75,0.85] ocean energy [0.80,0.90], [0.05,0.10], [0.10,0.20] [0.75,0.85], [0.10,0.20], [0.15,0.25] [0.5,0.5], [0.5,0.5], [0.5,0.5] o p e ra ti n g c o st ( l c ) biomass energy [0.5,0.5], [0.5,0.5], [0.5,0.5] [0.25,0.35], [0.20,0.30], [0.65,0.75] [0.15,0.25], [0.10,0.20],[0.75,0.85] solar power energy [0.65,0.75], [0.20,0.30], [0.25,0.35] [0.5,0.5], [0.5,0.5], [0.5,0.5] [0.25,0.35], [0.20,0.30], [0.65,0.75] ocean energy [0.75,0.85], [0.10,0.20], [0.15,0.25] [0.65,0.75], [0.20,0.30], [0.25,0.35] [0.5,0.5], [0.5,0.5], [0.5,0.5] l a n d c o st ( l c ) biomass energy [0.5,0.5], [0.5,0.5], [0.5,0.5] [0.25,0.35],[0.20,0.30], [0.65,0.75] [0.35,0.45], [0.30,0.40],[0.55,0.65] solar power energy [0.65,0.75], [0.20,0.30], [0.25,0.35] [0.5,0.5], [0.5,0.5], [0.5,0.5] [0.70,0.80], [0.15,0.25], [0.20,0.30] ocean energy [0.55,0.65], [0.30,0.40], [0.35,0.45] [0.20,0.30], [0.15,0.25], [0.70,0.80] [0.5,0.5], [0.5,0.5], [0.5,0.5] ijahp article: bolturk, kahraman/interval-valued neutrosophic ahp with possibility degree method international journal of the analytic hierarchy process 443 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.545 step 9: following steps 1-7, we obtained the priority vectors of the interval multiplicative matrix and normalized them. because of the space constraints, we only give the priority vectors for each of neutral, optimistic, and pessimistic cases. �̃�𝑁 = (0.238, 0.371, 0.391) 𝑇; �̃�𝑂 = (0.360,0.290,0.350) 𝑇; �̃�𝑃 = (0.304,0.332,0.363) 𝑇; �̃�𝑁 = (0.308, 0.446, 0.246) 𝑇; �̃�𝑂 = (0.325,0.470,0.205) 𝑇; �̃�𝑃 = (0.305,0.567,0.128)𝑇; 𝑂�̃�𝑁 = (0.241, 0.348, 0.412) 𝑇; 𝑂�̃�𝑂 = (0.238,0.338,0.424) 𝑇; 𝑂�̃�𝑃 = (0.238,0.206,0.556) 𝑇; 𝐿�̃�𝑁 = (0.265, 0.421, 0.314) 𝑇; 𝐿�̃�𝑂 = (0.256,0.437,0.307) 𝑇; 𝐿�̃�𝑃 = (0.380,0.573,0.188) 𝑇 step 10: from the priority vectors of alternatives and criteria obtained in the previous steps, we ranked the alternatives based on the neutral, optimistic, and pessimistic cases. 𝑊�̃� = (0.270, 0.402, 0.329) 𝑇; �̃�𝑂 = (0.292,0.399,0.303) 𝑇; �̃�𝑃 = (0.301,0.431,0.297) 𝑇 step 11: we select the alternative with the highest priority value. under neutral conditions, the ranking is spe>oe>be. under optimistic conditions, the ranking is the same. however, the ranking becomes spe>be>oe under pessimistic conditions. thus, the decision is to invest in solar power energy. 6. conclusion under uncertainty, neutrosophic sets help decision makers to fully reflect the expert’s judgments in their minds. neutrosophic sets provide a distinction between a ‘relative truth’ and an ‘absolute truth’. besides, interval-valued sets have been preferred since they make it possible for decision makers to use a larger domain to assign the degrees of truthiness (t), indeterminacy (i) and falsity (f). in this study, the fuzzy extensions of the ahp method in the literature are summarized. there are several ahp methods with fuzzy sets such as type 2 fuzzy sets, intuitionistic fuzzy sets, hesitant fuzzy sets, neutrosophic sets and pythagorean fuzzy sets. to the best of our knowledge, the paper represents the first time the interval-valued neutrosophic ahp method has been used with the possibility degree method. the steps of the ivnahp-pd have been given step by step. an application in energy planning has been presented with the ivnahp-pd method. in conclusion, the criterion of sustainability has the highest priority in this selection problem and solar power energy is the best alternative. for further research, another type of measure like cosine similarity can be used with ivnahp or other extensions of ahp such as neutrosophic ahp or pythagorean fuzzy ahp can be used together with any other measure. ijahp article: bolturk, kahraman/interval-valued neutrosophic ahp with possibility degree method international journal of the analytic hierarchy process 444 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.545 references abdel-basset m, mohamed m, sangaiah ak (2017a) neutrosophic ahp-delphi group decision making model based on trapezoidal neutrosophic numbers. journal of ambient intelligence and humanized computing, 9(5), 1-17. doi: https://doi.org/10.1007/s12652017-0548-7 abdel-basset, m., mohamed, m., & smarandache, f. (2018a). an extension of neutrosophic ahp-swot analysis for strategic planning and decisionmaking. symmetry, 10(4.) doi:10.3390/sym10040116. abdel-basset, m., manogaran, g., & mohamed, m. (2018b). internet of things (iot) and its impact on supply chain: a framework for building smart, secure and efficient systems. future generation computer systems, 86, 614-628. doi:10.1016/j.future.2018.04.051 abdel-basset, m., mohamed, m., zhou, y., & hezam, i. (2017b). multi-criteria group decision making based on neutrosophic analytic hierarchy process. journal of intelligent and fuzzy systems, 33(6), 4055-4066. doi:10.3233/jifs-17981. abdullah l, najib l (2016). integration of interval type-2 fuzzy sets and analytic hierarchy process: implication to computational procedures. aip conference proceedings 1750(1), 020019 doi: 10.1063/1.4954532. atanassov kt (1983) intuitionistic fuzzy sets. vii itkr’s session, sofia. biswas p, pramanik s, giri bc (2014). a new methodology for neutrosophic multiattribute decision making with unknown weight information. neutrosophic sets and systems, 3, 42-50. bolturk, e., kahraman, c. (2018) a novel interval-valued neutrosophic ahp with cosine similarity measure. soft computing, 22(15), 4941-4958. doi: https://doi.org/10.1007/s00500-018-3140-y buckley, j.j. (1985). fuzzy hierarchical analysis. fuzzy sets and systems, 34, 187-195. buyukozkan g, feyzioglu o, gocer f (2016). evaluation of hospital web services using intuitionistic fuzzy ahp and intuitionistic fuzzy vikor. ieee international conference on industrial engineering and engineering management 2016, 607-611. doi: 10.1109/ieem.2016.7797947 chang, d. y. (1996). applications of the extent analysis method on fuzzy ahp. european journal of operational research, 95(3), 649-655. doi: https://doi.org/10.1016/0377-2217(95)00300-2 deepika m, kannan ask (2016). global supplier selection using intuitionistic fuzzy analytic hierarchy process. international conference on electrical, electronics, and optimization techniques, iceeot 2016, 2390. doi: 10.1109/iceeot.2016.7755122 javascript:void(0) javascript:void(0) ijahp article: bolturk, kahraman/interval-valued neutrosophic ahp with possibility degree method international journal of the analytic hierarchy process 445 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.545 erdogan m, kaya i (2016) evaluating alternative-fuel busses for public transportation in istanbul using interval type-2 fuzzy ahp and topsis. journal of multiple-valued logic and soft computing, 26(6), 625-642. ilbahar, e., karaşan, a., cebi, s., & kahraman, c. (2018). a novel approach to risk assessment for occupational health and safety using pythagorean fuzzy ahp & fuzzy inference system. safety science, 103, 124-136. doi:10.1016/j.ssci.2017.10.025. kahraman c, bolturk e, onar sc, oztaysi b, goztepe k (2016). multi-attribute warehouse location selection in humanitarian logistics using hesitant fuzzy ahp. international journal of the analytic hierarchy process, 8(2), 271-298. doi: https://doi.org/10.13033/ijahp.v8i2.387 kahraman, c., öztayşi, b., sarı, i̇.u., turanoğlu, e. (2014) fuzzy analytic hierarchy process with interval type-2 fuzzy sets. knowledge-based systems, 59, 48-57. doi: https://doi.org/10.1016/j.knosys.2014.02.001 lee, h-c., chang, c-t. (2018). comparative analysis of mcdm methods for ranking renewable energy sources in taiwan,. renewable and sustainable energy reviews, 92, 883-896. doi: https://doi.org/10.1016/j.rser.2018.05.007 li y, wang y, liu p (2016). multiple attribute group decision-making methods based on trapezoidal fuzzy two-dimension linguistic power generalized aggregation operators. soft computing, 20(7), 2689-2704. doi: https://doi.org/10.1007/s00500-015-1668-7 oztaysi b, onar sc, bolturk e, kahraman c (2015). hesitant fuzzy analytic hierarchy process. ieee international conference on fuzzy systems,1-7. doi: https://doi.org/10.1109/fuzz-ieee.2015.7337948 radwan nm, senousy mb, riad aedm (2016). neutrosophic ahp multi criteria decision making method applied on the selection of learning management system. international journal of advancements in computing technology (ijact), 8(5), 95-105. doi: https://doi.org/10.1111/exsy.12170 rivieccio u., (2008). neutrosophic logics: prospects and problems. fuzzy sets systems 159(14), 1860–1868. doi: https://doi.org/10.1016/j.fss.2007.11.011 senvar oa (2018). systematic customer oriented approach based on hesitant fuzzy ahp for performance assessments of service departments. advances in intelligent systems and computing, 643,289-300. doi: https://doi.org/10.1007/978-3319-66827-7_26 smarandache f (1998) neutrosophy neutrosophic probability. set, and logic. rehoboth: american research press. smarandache, f. (1995). neutrosophic logic and set, mss., http://fs.gallup.unm.edu/neutrosophy.htm https://dl.acm.org/author_page.cfm?id=81100008814&coll=dl&dl=acm&trk=0 https://dl.acm.org/author_page.cfm?id=81484655812&coll=dl&dl=acm&trk=0 https://dl.acm.org/author_page.cfm?id=87958990357&coll=dl&dl=acm&trk=0 https://dl.acm.org/author_page.cfm?id=87958753657&coll=dl&dl=acm&trk=0 ijahp article: bolturk, kahraman/interval-valued neutrosophic ahp with possibility degree method international journal of the analytic hierarchy process 446 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.545 torra v (2010). hesitant fuzzy sets. international journal of intelligent systems 25,529– 539. doi: https://doi.org/10.1002/int.20418 wang h, smarandache f, zhang yq, sunderraman r (2010). single valued neutrosophic sets. multispace multistruct, 4, 410–413. wu, j., huang. h., & cao, q. (2013). research on ahp with interval-valued intuitionistic fuzzy sets and its application in multi-criteria decision making problems. applied mathematical modelling. 37(24), 9898-9906. doi: https://doi.org/10.1016/j.apm.2013.05.035 van laarhoven, p. j. m., & pedrycz, w. (1983). a fuzzy extension of saaty's priority theory. fuzzy sets and systems, 11(1-3), 229-241. doi: https://doi.org/10.1016/s01650114(83)80082-7 zeng, j., an, m., & smith, n. j. (2007). application of a fuzzy based decision making methodology to construction project risk assessment. international journal of project management, 25(6), 589-600. doi: https://doi.org/10.1016/j.ijproman.2007.02.006 zhang hy, wang jq, chen, xh (2014). interval neutrosophic sets and their application in multicriteria decision-making problems. the scientific world journal, 2014, 15. doi: http://dx.doi.org/10.1155/2014/645953 zhu b, xu z, zhang r, hong m (2016). hesitant analytic hierarchy process. european journal of operational research, 250(2),602-614. doi: https://doi.org/10.1016/j.ejor.2015.09.063 https://doi.org/10.1016/j.apm.2013.05.035 ijahp article: wilford, bodin, gordon/using the analytic hierarchy process to assess the impact of internal control weaknesses on firm performance international journal of the analytic hierarchy process 203 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.709 using the analytic hierarchy process to assess the impact of internal control weaknesses on firm performance amanda lyn wilford dixie l. leavitt school of business southern utah university amandawilford@suu.edu lawrence d. bodin robert h. smith school of business university of maryland lbodin@rhsmith.umd.edu lawrence a. gordon robert h. smith school of business university of maryland lgordon@rhsmith.umd.edu abstract since the passage of the sarbanes-oxley act (sox) of 2002, many studies have examined the impact of material weaknesses in internal control systems (mwics) on firm performance. overall, these studies indicate that a negative association exists between poor internal control and firm performance. prior research suggests that the above noted association between internal control and firm performance should be affected by both the actual number and the different types of mwics. however, this stream of research has focused on using a binary measure for internal control and has not considered the combined impact that the different types of mwics may have on firm performance. in this study, we create and introduce a new internal control index, derived from the analytic hierarchy process (ahp). we then show that more information regarding the impact of mwics can be obtained through our ahp index measure as opposed to the binary measure that is commonly used. these findings have important implications for a firm’s stakeholders (e.g., managers, stockholders, creditors, financial analysts, employees, and auditors). keywords: analytic hierarchy process; internal control; performance measurement; material weaknesses; internal control measures. 1. introduction the sarbanes-oxley act of 2002 (sox) was passed in response to the perceived lack of internal controls within the scandal-ridden companies of the early 2000s (e.g., enron, tyco, worldcom). to comply with section 404 of sox, large public companies are required to fulfill two main internal control requirements. first, the firm’s management must prepare a report that states its responsibility for establishing, maintaining and mailto:amanda.wilford@ie.edu mailto:lbodin@rhsmith.umd.edu mailto:lgordon@rhsmith.umd.edu ijahp article: wilford, bodin, gordon/using the analytic hierarchy process to assess the impact of internal control weaknesses on firm performance international journal of the analytic hierarchy process 204 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.709 assessing its firm’s internal control system. second, the firm’s auditors must attest to and report on the management’s assessment of the firm’s internal control system. a key aspect of compliance with the sox requirements is that firms must report their material weaknesses in internal control systems (mwics) within their 10-k (annual) and 10-q (quarterly) reports that are filed with the securities and exchange commission (sec). with the increased attention on internal controls, audit analytics has created a database that aggregates the mwics from all of the 10-k and 10-q reports that are filed with the securities and exchange commission (sec). in addition, audit analytics classifies the number of sox-related mwics reported by firms and auditors into 21 different categories that represent different types of mwics. this database has been widely used to empirically examine issues surrounding the association between a firm’s internal control system and its performance. generally speaking, studies have provided strong evidence of a negative association between the presence of mwics and firm performance (e.g., beneish, billings & hodder, 2008; gordon & wilford, 2012; cheng, dhaliwal & zhang, 2013; feng, li, mcvay & skaife, 2015; gao & jia, 2016; mao & ettredge, 2016; d’mello, gao & jia, 2017; ge, koester & mcvay, 2017; cheng, goh & kim, 2018). a cursory look at the different mwics reported by firms under sox and prior research suggests that some types of mwics can have a greater impact on firm performance than others (beneish et al., 2008; gordon & wilford, 2012; weiss, 2014; balsam, jiang & lu, 2014; cheng et al., 2018). additionally, a few empirical studies found that the number of mwics can negatively impact firm performance (gordon & wilford, 2012; keane, elder & albring, 2012; darrough, huang & zur, 2018). most research, starting with the seminal mwics work by ge and mcvay (2005), presents mwics as a binary variable (bv) that is set equal to one if a firm reports at least one mwics and/or at least one type of mwics and set at zero otherwise. nevertheless, the bv does not capture the fact that reporting multiple mwics and multiple types of mwics may impact a firm’s performance differently than just acknowledging the presence of a mwics. the use of the bv implicitly assumes that firm performance is associated with the mere reporting of mwics, irrespective of the aggregation of mwics types that are reported. it is also worth noting that the number of mwics reported does not always directly correspond to the number of mwics types. a firm could report only one mwics in its 10-k report, but that mwics could be included in multiple mwics types in the audit analytics database. given that most firms that report mwics report more than one mwics, as well as multiple types of mwics, our objective in this paper is to examine how the analytic hierarchy process (ahp) can be used to aid the stakeholder’s examination of the severity of control issues in their organizations. to do so, we examined the issue through two different, but related, analyses. first, we developed an ahp model to determine weights that could be applied to the different mwics that are reported by companies. we used these weights to develop an internal control index (ici) as a measure of mwics. our ici is derived from an experiment with auditors and an experiment with managers. (i.e., what we refer to as ici-a and ici-m, respectively). second, we assessed whether there was a relationship between our ici and firm performance (where firm performance is measured ijahp article: wilford, bodin, gordon/using the analytic hierarchy process to assess the impact of internal control weaknesses on firm performance international journal of the analytic hierarchy process 205 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.709 as return on assets [roa]). this second analysis helped us determine the utility of our ici measures compared to the bv measure that is commonly used. our findings indicate that auditors and managers assign different levels of significance to different types of weaknesses. these differences support the use of an ahp-based index to analyze mwics rather than, or as a supplement to, a bv measure. furthermore, our findings show that using an ahp-derived index to examine the association between firm performance and mwics is more informative than using a bv measure. the current study contributes to the growing ahp and internal control literature in the following ways. first, this study provides a practical application for the use of the ahp in determining the impact of mwics on firm performance. second, using an aggregate measure of internal control, this study provides empirical evidence that different weaknesses will have differing impacts on firm performance. third, based on a larger and more contemporary sample of firms, this study confirms and expands upon the prior literature that shows a negative association between firm performance and mwics. this latter issue highlights the incentive that firms have to not report any internal control weaknesses within their 10-ks that are filed with the sec (securities and exchange commission). it also helps explain why many firms claim to have effective internal controls when in reality they have mwics (rice & weber, 2012; rice, weber & wu, 2015). furthermore, only a small percentage of firms actually report mwics in their 10k reports and most of them that are reported were detected by the firm’s auditors rather than the firm’s management (bedard & graham, 2011). fourth, the findings from the current study help explain why firms have an incentive to delay the reporting of mwics until the time of reporting restatements rather than in their original 10-k reports (rice & weber, 2012). the remainder of this paper proceeds as follows. in section 2, we review the relevant literature leading up to the development of the basic hypotheses underlying this study. in section 3, we discuss the research design that forms the basis for our study. in section 4, we examine the results related to our hypotheses. finally, in section 5, we discuss the implications and conclusions of our analysis. 2. background and hypotheses 2.1 the use of the ahp in accounting research not surprisingly, the ahp has been used within accounting research as a tool to examine accounting evaluation and planning decisions. using the ahp to model auditing procedure preferences, arrington et al. (1984) provided one of the first applications of the ahp within accounting research. additionally, arrington et al. (1984) suggested the ahp as an applicable tool to make “judgments such as materiality, internal control evaluation, opinion qualifications, and strategic planning” (arrington et al. 1984). other applications of the ahp in accounting include examining auditors’ evaluations of internal audit functions (messier & schneider, 1988; campbell, 1994), tests of controls (spires, 1991), audit planning (bedard et al., 1991), performance evaluation (emby & etherington, 1996), accounting treatment (boyle, 1985), and the evaluation of information security investments (bodin et al., 2005; 2008). similarly, amponsah (2011) used the ahp to determine the critical success factors of public-private-partnerships. ijahp article: wilford, bodin, gordon/using the analytic hierarchy process to assess the impact of internal control weaknesses on firm performance international journal of the analytic hierarchy process 206 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.709 and more recently, chen et al. (2017) applied the ahp to an internal control framework using chinese firms. internal controls, by nature, are qualitative and when internal control system issues arise, there is no general process in place to evaluate and compare these issues. first, we sought to determine whether users of financial statements perceive that there are differences within the types of weaknesses that are reported within their organizations. to do so, we used the structure of the ahp rating model to test the following hypothesis, stated in the null form. h1: there is no difference in the perception of severity about the types of reported mwics between the different users of financial statements (auditors vs. managers). to test hypothesis h1, we developed and ran an ahp ratings model that uses pairwise comparisons determined by an experienced set of auditors from various firms. further, we developed a second ahp ratings model based on the pairwise comparisons generated by a set of managers from various firms. this process is described in section 3 below. as will be seen in section 3, the ahp ratings models developed in this study are variants of the traditional ahp ratings model. in a traditional ratings model, each alternative is given a score between 0 and 1 and the alternatives with the largest scores (or smallest scores) are identified as being the best alternatives. in contrast, the ratings models studied in this paper determine weights to assign to each mwics or aggregated mwics (alternative) and these weights are used to set up an ordinary least squares (ols) regression model similar to feng et al. (2015). 2.2 internal control measurement in current accounting research internal control research in accounting literature has examined the link between the reliability of financial reporting, through mwics, and firm performance. for example, cheng et al. (2013) found that firms that report mwics are less efficient with their capital investments. using frontier analysis, cheng et al. (2018) found that operational efficiency is significantly lower when mwics are reported. weiss (2016) showed that the negative impact on performance, from reporting mwics, is more severe for familyowned firms than for non-family-owned firms. additionally, feng et al. (2015) found that if a firm reports mwics, they are more likely to have a lower return on assets. furthermore, darrough et al. (2018) showed that acquirers that reported mwics have a larger negative stock market reaction to acquisitions and lower future performance than acquirers without such mwics. there are numerous additional empirical studies that document an association between mwics that are reported under sox and firm performance. for example, research indicates that there is a negative association between firms reporting mwics and stock market returns (e.g., beneish et al., 2008; hammersley, myers & shakespeare, 2008; rezee, r. espahbodi, p. espahbodi & h. espahbodi, 2012; chen et al., 2017). this negative relationship also holds between firms that report mwics and return on assets (ge & mcvay, 2005; feng et al., 2015). firms that report mwics also tend to exhibit lower accrual quality (e.g., see doyle et al., 2007b; asbaugh-skaife et al., 2008; bedard, r. hoitash, u. hoitash & westermann, 2012), and lower accrual quality has been tied to ijahp article: wilford, bodin, gordon/using the analytic hierarchy process to assess the impact of internal control weaknesses on firm performance international journal of the analytic hierarchy process 207 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.709 lower firm performance (e.g. dechow & dichev, 2002; francis, lafond, olsson & schipper, 2005). as noted earlier, the initial post-sox internal control research measures mwics as a binary variable (bv) set equal to one if a firm reports at least one mwics in their financial reports and set to zero otherwise (ge & mcvay, 2005; doyle, ge & mcvay, 2007a; doyle et al., 2007b; ogneva, subramanyam & raghunandan, 2007; li, peters, richardson & watson, 2012). as research has developed in this area, mwics have also been classified into account-level types and entity-level types, established as individual bvs that are set equal to one if the mwics reported was related to one or the other and set to zero otherwise. using this classification, bedard et al. (2012) demonstrated that remediation of entity-level mwics results in significant changes in abnormal accruals (i.e., amounts in the financial statements that have been incurred during the period, but not yet realized through a cash transaction). additionally, asare and wright (2013) found that analysts are more skeptical of audit reports that are associated with an internal control report that lists entity-level mwics, and balsam et al. (2014) showed a relationship between equity incentives and the likelihood of reporting entity-level mwics. more recent internal control research further refines the mwics classification and examines the audit analytics specified types of mwics, reported in isolation, as bvs that are set equal to one if the firm has an mwics that corresponds to an mwics type that is under consideration, and set to zero otherwise. focusing on the efficiencies in inventory management, feng et al. (2015) found that inventory management declines when inventory account-related mwics are reported. li et al. (2012) indicated that mwics types related to a firm’s information technology environment influence the accuracy of management decisions differently than other mwics types. koester, lim, and vigeland (2015) investigated the relationship between tax account-related mwics and unrecognized tax benefits. in addition to the differences related to the mwics types, gordon and wilford (2012) and darrough et al. (2018) showed that the number of reported mwics has an impact on firm performance. in their examination of mwics and audit fees, keane et al., (2012) found that audit fees increase in relation to the number of mwics reported. despite the suggestion in recent research that different mwics types can have differing effects on performance, the common practice of measuring mwics as a single binary variable has ignored the aggregate impact of the number of mwics reported and the impact of reporting multiple types of mwics. accordingly, in this paper we tested the following basic hypothesis (stated in its null form): h2: there is no additional information provided about the relationship between mwics and firm performance when a firm’s mwics is measured based on an ahp internal control index (ici) versus a binary variable (bv). 3. research design to test the above hypotheses, we first created and analyzed the results from an analytic hierarchy process evaluation of internal control that we call the internal control index ijahp article: wilford, bodin, gordon/using the analytic hierarchy process to assess the impact of internal control weaknesses on firm performance international journal of the analytic hierarchy process 208 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.709 (ici). the elements ascribed to the creation of the ici are detailed in section 3.1. after this initial evaluation, we compared the ici to the bv to examine whether the ici would provide more information about the impact of internal control on firm performance than the bv. the details of this analysis are presented in section 3.2. 3.1 ahp experiment the ici was developed through the following five major steps. step 1: design of the ahp tree. our ahp tree has three levels. level 1 of our ahp tree is the goal node. the overall goal of this ahp is the development of an ici and the hierarchy was designed around this goal. level 2 of our ahp tree consists of the five primary criteria that include personnel weaknesses (pw), financing/accounting reporting issues (fri), policy issues (pi), restatements/adjustments (r/a), and regulatory issues (ri). we developed the primary criteria as aggregate categories of the 21 sub-criteria discussed below to reduce the number of required pairwise comparisons. if each of these categories were to be classified as a primary criterion, decision makers would have to analyze (21*20)/2 = 210 pairwise comparisons. two accounting professionals, each with over eight years of experience, examined our classification of the 21 mwics into these five criteria and their review resulted in no significant modifications to the classification. level 3 of our ahp tree consists of 21 sub-criteria that we derived from the categories used by audit analytics to classify firm reported mwics. a sideways view of our ahp tree is shown in figure 1. each criterion and subcriterion, along with its abbreviation in parentheses is documented in figure 1. ijahp article: wilford, bodin, gordon/using the analytic hierarchy process to assess the impact of internal control weaknesses on firm performance international journal of the analytic hierarchy process 209 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.709 figure 1 ahp tree for internal control comparisons step 2: pairwise comparison experiment. we conducted an experiment on specific users of financial statements, with practicing corporate auditors and managers evaluating the criteria and sub-criteria in the ahp tree hierarchy using pairwise comparisons. these pairwise comparisons are a key component of the ahp and are used to supply the data to carry out the analysis. the pairwise comparisons represent a ratio of the weights assigned to the two criteria that are being compared. the experts selected for this ahp experiment were managers of financial reporting and senior auditors responsible for auditing the financial reports of public companies. auditors and managers of financial reporting are two similar groups of stakeholders with one important distinguishing characteristic, independence. this independence may influence the auditors’ ability to be more objective internal control weaknesses personnel weaknesses (pw) accounting personnel resources, competency/training (act) ethical compliance issues with personnel (eci) senior management competency, tone, reliability issues (smi) insufficient or non-existent internal audit function (ini) segregations of duties/design of control (personnel) (sd) financing/accounting reporting issues (fri) treasury control issues (tc) journal entry control issues (jec) non-routine transaction control issues (ntc) inadequate disclosure controls (id) material and/or numerous auditor/ye adjustments (ma) policy issues (pi) untimely or inadequate account reconciliations (uar) accounting documentation, policy and/or procedure (adp) information technology, software, security and access issues (it) restatements/adjustment s (r/a) restatement or nonreliance of company filings (rnf) restatement of previous 404 disclosures (rpd) sab 108 adjustments noted (sab) regulatory issues (ri) ineffective or understaffed audit committee (iua) ineffective regulatory compliance issues (irc) management/board/audit committee investigations (mi) sec or other regulatory investigations and/or inquiries (si) scope/disclaimer of opinion or other limitations (sdl) ijahp article: wilford, bodin, gordon/using the analytic hierarchy process to assess the impact of internal control weaknesses on firm performance international journal of the analytic hierarchy process 210 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.709 in their assessment of the different types of mwics. experts were randomly selected from the linkedin social network of one of the authors. an online questionnaire was sent to ten auditors and ten managers. seven auditors and seven managers responded to our invitation to participate in this study. the auditor experts have achieved at least an audit manager position within a top ten public accounting firm and each have at least seven years of auditing experience. the manager experts are corporate managers at sec registered firms and large (over 1,000 employees) private firms who oversee their company’s financial reporting function. even though the managers at private firms are not required to comply with sox, managers of financial reporting in all organizations must have a strong understanding of internal controls for financial reporting. the experts performed pairwise comparisons of the categories and types of mwics that are identified in the ahp tree, presented in figure 1. these pairwise comparisons were used to supply the data to calculate the weights assigned to each of the five categories and each of the 21 mwics. step 3: determining the ahp weights. the expert responses were evaluated to determine the weights of the various criteria and sub-criteria. to determine these weights, all of the pairwise comparisons (i.e., judgments) that were made by the experts were stored in pairwise comparison matrices. pairwise comparison matrix a = (aij) is a positive reciprocal matrix (i.e. aji = 1/aij for each element (i,j) in a positive reciprocal matrix). thus, the comparison of criterion i to criterion j will always be the reciprocal of the comparison of criterion j to criterion i. these pairwise comparison matrices were then used to determine the weights assigned to each criterion and sub-criterion. then, the eigenvector associated with the maximum eigenvalue of a pairwise comparison matrix was used to determine the weights assigned to all of the criteria or sub-criteria associated with the pairwise comparison matrix. to aggregate each of the seven experts’ pairwise comparison matrices, we took the geometric mean to determine the overall weights for the manager group. this same process was performed for the auditor group. different pairwise comparison matrices were developed for the auditor group and the manager group. step 4: computing the overall weight assigned to each mwics. in the fourth step, we determined the overall weight assigned to each mwics. in the solution found in step 3, the sum of the weights computed for each of the criteria values must sum to one. furthermore, the sum of the weights assigned to a mwics where the mwics is assigned to one of the five criteria is equal to the product of the ahp weight for the mwics, given that the mwics is assigned to a particular criterion, and multiplied by the weight assigned to the criterion. as a result, the overall weight assigned to each of the 21 mwics must sum to one. table 1 presents the tabular solution to the determination of these weights for both the auditors and managers. step 5: weight to assign to an entry in the audit analytic database. to calculate the ici for any given entry i in the audit analytic database, we applied the weights, determined in the ahp experiment in step 4 to the data contained for each entry i in the audit analytics database. the result of this application is an ici for each firm-specific observation. the audit analytics data does not specifically define which types of ijahp article: wilford, bodin, gordon/using the analytic hierarchy process to assess the impact of internal control weaknesses on firm performance international journal of the analytic hierarchy process 211 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.709 mwics are associated with which weakness and as such we were unable to construct indices that combined both the number and type of mwics reported (see appendix a for detailed definitions of each of the 21 mwics types). the number of mwics reported in audit analytics is derived directly from firm reported information in sec documents. mathematically, the ici for each firm-year observation i derived from the audit analytics database is defined as follows: icii = ∑ 21 k=1 (wk * mwik) where icii = internal control index value for firm-year observation i; wk = ahp weight assigned to mwics type k, where type k is one of the 21 different types of mwics categorized by audit analytics where ∑ 21k=1 (wk)=1; mwik = variable assigned a value of one if firm-year observation i reports mwics type k, and set to zero otherwise. the value of each icii ranges between 0 and 1. we interpreted the icii for firm-year observation i as an indication of the firm’s level of internal control. a high (low) value of the icii indicates weaker (stronger) internal control for firm-year observation i. in the procedures outlined above, we calculated two alternative values for the ici. ici-a corresponds to the ahp results from the auditors’ responses whereas ici-m corresponds to the ahp results from the managers’ responses. bv, ici-a and ici-m, our mwics measures, were calculated based on the presence/absence of mwics. firms that reported mwics have positive index values and firms that do not report mwics have index values of 0. it is unlikely that the weights produced from the ahp will be the optimal set of weights. however, if the ici provides a more complete explanation of the relationship between firm performance and mwics than the bv, then we have shown that the ici is capable of providing additional insights into prior research results. 3.2 research design for ols regression analysis of firm performance the establishment of the ici is shown above through the ahp experiment analysis explanation. additionally, our examination of firm performance requires that we also use a bv for comparison purposes. the bv, like the ici, was also calculated for all audit analytics observations and becomes a key independent variable in our firm performance analysis. the bv is set equal to one if a firm-year observation from audit analytics reports any mwics and set to zero otherwise. to examine and compare the bv and ici measures of mwics to firm performance, we established an ordinary least squares (ols) model whose specifications are similar to feng et al. (2015). as noted above, our chosen measure of firm performance for this analysis is roa. equation (1) defines the mwics-roa model that was used to test the relationship between roa and mwics. ijahp article: wilford, bodin, gordon/using the analytic hierarchy process to assess the impact of internal control weaknesses on firm performance international journal of the analytic hierarchy process 212 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.709 we utilized two different measures for mwics: bv and ici. when the ici is included in equation (1), it can be either ici-a or ici-m. also, the coefficients produced through an ols regression of equation (1) are identified as  within equation (1). ols regression equation for the mwics-roa model: roai = 0 + 1*mwicsi +2*sizei + 3*segi + 4*foreigni + 5*growthi + 6*lossi + 7*cap_inti + 8*vol_salei + 9*agei + 10*aud_bigi + 11*pr_roai +  (1) the variables in equation (1) for firm-year observation i are shown below. we used data from the audit analytics, crsp, and compustat databases to calculate the dependent and independent variables in our analyses (with the raw database used to gather the information included in parentheses): roai = measured as earnings before extraordinary items divided by average total assets for firm-year observation i (compustat); mwicsi = internal control measure for firm-year observation i in year t. this measure is interchangeable as bv, ici-a, or ici-m (audit analytics); sizei = log of the market value of equity for firm-year observation i (compustat); segi = measured as the number of business segments reported for firm year observation i (computat segments); foreigni = indication of foreign sales that is set equal to one, zero otherwise, if a firm-year observation i reports a foreign currency translation adjustment (compustat); growthi = measured as the sales growth rate for firm-year observation i over the time period t-2 through t-1 (compustat); lossi = measured as the proportion of years (t-2 through t) during which a firm reports a loss in earnings (compustat); cap_inti = capital intensity that is measured as the log of pp&e for year t (compustat); vol_salei = standard deviation of sales to average assets from years t-6 through t, at least three years of data are required (compustat); agei = log of years firm is included in the crsp database t (crsp header file); aud_bigi = dummy variable set to 1 if firm utilizes one of the 6 biggest auditors (audit analytics); ijahp article: wilford, bodin, gordon/using the analytic hierarchy process to assess the impact of internal control weaknesses on firm performance international journal of the analytic hierarchy process 213 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.709 pr_roai = measured as earnings before extraordinary items divided by average total assets in year t-1 (compustat); the dependent variable in equation (1) described above is return on assets (roa). to provide comparability with prior research (feng et al., 2015), we utilized the same type of ols model specification and similar dependent and independent variables. we clustered by firm and controlled for year and industry fixed effects. in the analysis described above, we expected that the coefficient values of the independent variables would be similar in sign and significance to those of prior studies. additionally, we expected the coefficient values of the mwics measures generated by our analysis, our key independent variables, would be significant and negative (as suggested by previous research). through the above analyses, we wanted to determine whether the ici measures could provide significant additional information with respect to the impact of mwics on performance. more specifically, we tested h2 by comparing the coefficients obtained for our mwics measures in equation (1). if the models that utilized the ici measures produced more information than the information produced through the utilization of the bv measure, we would have evidence to reject h2. as noted above, to calculate the dependent and independent variables in our performance analyses, we used data from the audit analytics, crsp, and compustat databases. the audit analytics data serves as the base for our dataset and includes all yearly firmspecific internal control data for fiscal years 2004 through 2018. to construct a sample for our ahp application/performance analysis, we began by including all observations from audit analytics with audit internal control reports in fiscal years 2004 through 2018. the raw initial sample is comprised of 60,544 firm-year observations. then, we applied two screens to the data to arrive at our final sample that was used in the analyses. first, we eliminated 1,135 duplicate observations (i.e., in cases where internal control reports are restated, we retained the most recent observation). in our second screen, we eliminated an additional 21,484 observations that did not have the necessary data available to calculate the variables required for our analysis. our final sample consisted of 37,925 observations. this final total included 35,326 observations with no mwics (what we refer to in the next section as the control group) reported and 2,599 observations with mwics reported. 4. results 4.1 results of ahp analysis in table 1, we present the results of our ahp analysis that are derived from the responses of both auditors and managers of financial reporting. these results were obtained through our application of the procedures described above. as stated above, we aggregated our expert responses utilizing the geometric mean because of homogeneity in the group structure and there were no signs of conflicts of interest among the experts (ossadnik, schinke & kaspar, 2016). ijahp article: wilford, bodin, gordon/using the analytic hierarchy process to assess the impact of internal control weaknesses on firm performance international journal of the analytic hierarchy process 214 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.709 to examine the consistency of our results, we applied a formula, denoted as equation (2), developed by saaty (1980). equation (2) calculates the internal consistency of the experts’ judgments: (2) where: = average (aω/ω); a = matrix of pairwise comparisons; ω = vector of weights; c.r. = consistency ratio, determined by calculating the consistency index from a large sample of purely random judgments. the values associated with this variable are derived from saaty (1980). for purposes of our analysis, c.r. is set equal to 1.12 for the 5x5 matrices and set equal to 0.58 for the 3x3 matrices; c.i. = consistency index. saaty (1980) argued that reliable judgments had consistency indices of less than 0.10. after the weights for all of the criteria and sub-criteria were calculated, we applied the above consistency check to each pairwise comparison matrix. the results of this first pass suggested that there was a lack of consistency within some of the expert’s judgments. consistency indices for all of the individual judgments ranged between 0.00 (completely consistent) and 0.57 (highly inconsistent). given the online nature of the ahp questionnaire, we examined each of the pairwise judgments and attempted to emulate the results that would have been the product of a group discussion that is generally associated with the ahp. this additional step ensured consistency within each expert’s pairwise comparison matrices (i.e., ci was less than 0.10 for all of the matrices). inconsistencies among some of the judgments could be due to a couple of different factors. first, the auditors and managers that carried out this analysis had a limited understanding of the ahp environment. if the pairwise comparisons had been made within a controlled group environment (all of the auditors or managers being in the same room at the same time with an experienced ahp moderator), this problem would have been eliminated. second, limited ahp training can lead to difficulty visualizing the relationship between all of the judgments. to ensure that the integrity of the weights were maintained, any adjustment that was made to a pairwise comparison was based on the judgment made with respect to the first variable judgment and the weights associated with each matrix before and after the judgments were revised. we found that the differences in weights before and after the pairwise comparisons were adjusted were insignificant and the c.i. after the adjustments were made was always less than 0.10 (unadjusted weights are available from the authors upon request). table 1 and figures 2 7 present a summary of the results of our ahp analysis. the criteria weights in table 1 and the associated figures indicate the level of impact that each criterion has on the financial statements when compared to the other criteria and sub-criteria. within the ahp derived weights, we see similarities among the weights derived by both the auditors (table 1, panel a) and the managers (table 1, panel b). it is c.i. = l max -n n-1 æ è ç ö ø ÷ /c.r. l max ijahp article: wilford, bodin, gordon/using the analytic hierarchy process to assess the impact of internal control weaknesses on firm performance international journal of the analytic hierarchy process 215 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.709 important to note that the pw criterion (personnel weaknesses) has the highest criterion weight among both the auditors and the managers. this result is in line with internal control guidance that states that the control environment is foundational to a strong internal control system (coso, 2013). these criteria weights indicate that managers feel that the pw criterion has the greatest impact on the financial statements, with the criterion having 51% of the weight. auditors also feel that the pw criterion has the greatest impact (29.2%) on the financial statements. however, the auditors also feel that the fri and pi criteria have a significant impact on the financial statements (27.6% and 19.7%, respectively). the sub-criteria weights indicate the level of impact that each of the sub-criteria (types of mwics) has on the financial statements when compared to the other sub-criteria. the auditor experts indicated that with respect to the pw criterion, sub-criterion smi has the greatest impact on the financial statements (36.4%). additionally, they clearly indicated that sub-criteria jec and ma have the greatest impact on the financial statements (28.4% and 25.0%, respectively), within the fri criterion category. auditor experts indicated that within the pi criterion, sub-criterion adp has the greatest impact on the financial statements (45.4%). they also indicated that sub-criterion rnf has a significantly greater impact (58.3%) on the financial statements than the other types of mwics in the r/a category. finally, the auditor experts concluded that within the ri criteria category, sub-criteria sdl, mi, and si have the greatest impact (27.9%, 27.4%, and 24.5%, respectively) on the financial statements. in contrast to the auditor results, the managers found that with respect to the pw criterion, sub-criteria smi and eci have the greatest impact on the financial statements (36.7% and 26.9%, respectively). managers found that criterion fri dominating subcriteria are not as clear-cut and three types of mwics or sub-criteria have about an equal impact on the financial statements with sub-criterion, ma at 27.9%, sub-criterion ntc at 24.6%, and sub-criterion jec at 22.4%. managers view sub-criterion it as having the greatest financial statement impact (41.4%) within the pi criteria category. the manager experts indicated that sub-criteria rnf and sab, within the r/a criterion, have an equally significant impact on the financial statements (39.5% and 38.2%, respectively). they also indicated that within the ri criterion category, si has the greatest impact (31.9%) on the financial statements. ijahp article: wilford, bodin, gordon/using the analytic hierarchy process to assess the impact of internal control weaknesses on firm performance international journal of the analytic hierarchy process 216 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.709 table 1 weights for the ici with auditors and managers* panel a: weights based on auditor assessments criteria pw fri pi r/a ri 0.292 0.276 0.197 0.084 0.150 subcriteria act eci smi ini sd tc jec ntc id ma uar adp it rnf rpd sab iua irc mi si sdl 0.106 0.207 0.364 0.100 0.223 0.182 0.284 0.181 0.103 0.250 0.295 0.454 0.251 0.583 0.188 0.229 0.122 0.080 0.274 0.245 0.279 weight 0.031 0.061 0.106 0.029 0.065 0.050 0.078 0.050 0.028 0.069 0.058 0.090 0.049 0.049 0.016 0.019 0.018 0.012 0.041 0.037 0.042 panel b: weights based on manager assessments criteria pw fri pi r/a ri 0.510 0.113 0.122 0.108 0.147 subcriteria act eci smi ini sd tc jec ntc id ma uar adp it rnf rpd sab iua irc mi si sdl 0.165 0.269 0.367 0.064 0.134 0.147 0.224 0.246 0.104 0.279 0.281 0.305 0.414 0.395 0.222 0.382 0.164 0.172 0.119 0.319 0.225 weight 0.084 0.137 0.187 0.033 0.068 0.017 0.025 0.028 0.012 0.032 0.034 0.037 0.051 0.043 0.024 0.041 0.024 0.025 0.018 0.047 0.033 *panel a presents the weights from the ahp analysis, based on the auditor sample assessments. panel b presents the weights from the ahp analysis, based on the manager sample assessments. the abbreviations provided for the criteria represent the following categorizations: pw – personnel weaknesses, fri – financing/accounting reporting issues, pi – policy issues, r/a – restatements/adjustments, and ri – regulatory issues. the sub-criteria are listed in figure 1 and are defined and described in appendix a. the values tied to the criteria and sub-criteria were obtained through an ahp experiment. the weight associated with each sub-criterion, mwics type, was calculated as the product of the criteria value multiplied by the sub-criteria value. ijahp article: wilford, bodin, gordon/using the analytic hierarchy process to assess the impact of internal control weaknesses on firm performance international journal of the analytic hierarchy process 217 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.709 *figures 2 – 7 (f2-f7) graphically present and provide a comparison of the weights from the ahp analysis, based on the auditor and manager sample assessments. the abbreviations provided for the criteria represent the following categorizations: pw – personnel weaknesses, fri – financing/accounting reporting issues, pi – policy issues, r/a – restatements/adjustments, and ri – regulatory issues. the sub-criteria are listed in figure 1 and are defined and described in appendix a. the values tied to the criteria and sub-criteria were obtained through an ahp experiment. the weight associated with each subcriterion, mwics type, was calculated as the product of the criteria value multiplied by the sub-criteria value. 29.2% 27.6% 19.7% 8.4% 15.0% 51.0% 11.3% 12.2% 10.8% 14.7% pw fri pi r/a ri f2. criteria weights by expert category auditor manager 3.1% 6.1% 10.6% 2.9% 6.5% 8.4% 13.7% 18.7% 3.3% 6.8% act eci smi ini sd f3. pw: overall subcriteria weights by expert category auditor pw manager pw 5.0% 7.8% 5.0% 2.8% 6.9% 1.7% 2.5% 2.8% 1.2% 3.2% tc jec ntc id ma f4. fri: overall subcriteria weights by expert category auditor fri manager fri 5.8% 9.0% 4.9% 3.4% 3.7% 5.1% uar adp it f5. pi: overall sub-criteria weights by expert category auditor pi manager pi 4.9% 1.6% 1.9% 4.3% 2.4% 4.1% rnf rpd sab f6. r/a: overall subcriteria weights by expert category auditor r/a manager r/a 1.8% 1.2% 4.1% 3.7% 4.2% 2.4% 2.5% 1.8% 4.7% 3.3% iua irc mi si sdl f7. ri: overall sub-criteria weights by expert category auditor ri manager ri ijahp article: wilford, bodin, gordon/using the analytic hierarchy process to assess the impact of internal control weaknesses on firm performance international journal of the analytic hierarchy process 218 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.709 as a final analysis, we performed t-tests on each of the criteria to determine if there were significant differences between the perceptions of the auditors and the perceptions of the managers as they relate to mwics. our results indicate that there are significant differences between the auditor and manager expert judgements for all of the criteria, with the exception of the ri criterion. more specifically, our two-sided t-test results show differences in the pw criterion that are significant at the 0.01 level, differences in the fri criterion that are significant at the 0.05 level, differences in the pi criterion that are significant at the 0.10 level, and differences in the r/a criterion that are significant at the 0.10 level. overall, the results of our ahp analysis indicate that differences do exist in the perceptions of financial statement users and that certain types of mwics are viewed as more severe by managers and auditors than other types of mwics. as such, this analysis goes against the null hypothesis presented as h1 and therefore h1 is rejected. 4.2 results for ols regression analysis of firm performance to examine our second hypothesis, we ran an ols regression based on equation (1) and outlined and discussed in section 3. the results of this regression are displayed in table 2. column 1 presents the regression results when bv was used as the measure for mwics. column 2 presents the regression results when ici-a was used as the measure for mwics. finally, column 3 presents the regression results when ici-m was used as the measure for mwics. using the results in table 2, we found that the coefficients and signs related to the independent variables align closely with the results in prior research (feng et al., 2015). additionally, we found that, as expected, the coefficients associated with the mwics measures based on bv, ici-a and ici-m have a significant and negative relationship with our measure of roa, indicating that firms that report mwics experience lower profitability. further, the mwics coefficients for each of the models (see columns 1, 2, and 3 in table 2) are significant at the 0.01 level, and there is only a marginal difference among the t-values of the different measures. next, we analyzed the three roa models, bv, ici-a and ici-m. the variables for these models are similar with the exception of the variable associated with the mwics metric. first, we examined the overall model results of table 2 through the r-square measure and found that there was essentially no difference between the fit provided by the three roa models when the mwics measures were interchanged. to compare the mwics measures, we compared the t-values (shown in parentheses below the coefficients in table 2) and found that there was only a marginally significant difference among the tvalues of these three measures. these results were expected. next, we compared the differences between the results generated using the bv model and the results generated using the ici-a and ici-m models. first, we focused on the coefficients attached to the measures bv, ici-a and ici-m (given in table 2) and the difference in firm performance when we used the ici measures versus the bv measure. these results show that using the ici-a and the ici-m metrics provide additional information that has not been investigated in prior research. this information was derived from the weights in table 1 that were generated by our use of the ahp. ijahp article: wilford, bodin, gordon/using the analytic hierarchy process to assess the impact of internal control weaknesses on firm performance international journal of the analytic hierarchy process 219 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.709 the raw data for our sample consisted of 2,599 firms that reported at least one mwics and 35,326 firms that reported no mwics. the average roa for our sample is 5.93%. this value indicates that on average every dollar of assets reported by a firm will generate a 5.93% profit. we applied the above information to the data contained in table 2 and found the following. if a firm reports one or more mwics and the bv measure is utilized, our regression estimates indicate that the roa decreases by 2%, since the bv coefficient in column 1 of table 2 is equal to -0.02. more specifically, the roa decreases from 5.93% to 3.93% (the roas for the raw initial sample are available from the first author upon request). in other words, since the coefficient that is tied to bv is -0.02, then -0.02 multiplied by the value of 1 for the bv indicates a flat decrease of 2% regardless of the type and/or aggregation of mwics that are reported. this important observation occurs since the only information that we used to determine the impact on roa, when using the bv measure, was whether the firm reported an mwics (value is set to 1) or did not report an mwics (value is set to 0). the actual mwics that the firm reports were not used to determine the impact on roa. given the base information related to our analysis with the bv, we turned our attention to examining how the results indicate that the roa will be impacted when we use either the ici-a metric or the ici-m metric. the weights that we found for the 21 mwics and considered in this analysis are displayed in table 1. as described earlier, the ahp was used to compute these weights. then, we summed the weights (from table 1) for all of the mwics that were reported with the 2,599 observations that report mwics in our sample and divided this sum by 2,599. the average ici-a weight for firms in our sample is 0.2379 and the average ici-m weight for firms in our sample is 0.1816. to examine the economic implications of the ici-a and the ici-m, we applied the same practice used in the bv analysis above. however, when we utilized ici-a as our mwics metric, we were able to estimate the impact that different combinations of mwics would have on performance. this is not possible with the bv. the average ici-a weight for firms in our sample is 0.2379 and the coefficient associated with ici-a (from table 2) is -0.082. applying this to the coefficient in table 2, column 2, we see that the impact on the average roa would be approximately 2% (.2379 multiplied by -0.082 equals 1.95%). therefore, the roa for an average firm that reports weaknesses when the ici-a is used would be equal to 5.93% 1.95% = 3.98%, similar to the results obtained using the bv. however, the values of ici-a do not consist of a sole measure for all firms and the variation provides additional insight regarding the impact of mwics on the roa. for example, the maximum value of the ici-a is 0.876. applying this value to the coefficient indicates that mwics firms reporting the maximum value will have an roa that is 7.2% (.876 multiplied by -0.082) lower than the average. with an average roa of 5.93%, the roa of this mwics firm is estimated to be -1.27% (5.93% less 7.2 %). as discussed above, the impact on the roa depends on the mwics that are reported. ijahp article: wilford, bodin, gordon/using the analytic hierarchy process to assess the impact of internal control weaknesses on firm performance international journal of the analytic hierarchy process 220 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.709 table 2 ols regression results: return on assets analysis dependent variable: roa (1) (2) (3) variables bv ici-a ici-m intercept -0.002 -0.002 -0.003 (-0.26) (-0.31) (-0.45) bv -0.020*** (-8.10) ici-a -0.082*** (-8.66) ici-m -0.093*** (-7.94) size 0.001 0.001 0.001* (1.61) (1.62) (1.69) seg 0.001 0.001 0.001 (1.17) (1.20) (1.18) foreign -0.002 -0.002 -0.002 (-1.39) (-1.38) (-1.41) growth 0.000 0.000 0.000 (-.86) (-0.83) (-0.83) loss -0.010 -0.010 -0.010 (-1.60) (-1.59) (-1.61) cap_int 0.003*** 0.003*** 0.003*** (3.74) (3.76) (3.75) vol_sale -0.002 -0.002 -0.002 (-0.79) (-0.79) (-0.82) age 0.004*** 0.004*** 0.004*** (5.28) (5.28) (5.30) aud_big 0.004 0.004 0.004 (1.26) (1.21) (1.22) pr_roa 0.740*** 0.740*** 0.740*** (22.36) (22.37) (22.40) industry controls yes yes yes year controls yes yes yes firm cluster yes yes yes observations 37,925 37,925 37,925 r-squared 66.07% 66.08% 66.08% *table 2 reports results for the following ols regression using the feng et al. (2015) model with differing mwics index variables that represent bv, ici-a and ici-m, respectively: roa = 0 + 1*mwics +2*size + 3*seg + 4*foreign + 5*growth + 6*loss + 7*cap_int + 8*vol_sale + 9*age + 10*aud_big + 11*pr_roa +  (1) column 1 presents the regression results when bv is used as the measure for mwics. column 2 presents the regression results when ici-a is used as the measure for mwics. finally, column 3 presents the regression results when ici-m is used as the measure for mwics. statistical significance at the 0.01, 0.05, and 0.10 level is indicated by *** , ** , and * , respectively. t-statistics are listed in parentheses and based on two-way standard errors that are clustered at the firm level. all independent variables are winsorized at the 1% and 99% levels. all variables are defined above in the research design section. ijahp article: wilford, bodin, gordon/using the analytic hierarchy process to assess the impact of internal control weaknesses on firm performance international journal of the analytic hierarchy process 221 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.709 to examine the results using our ici-m measure, we applied the same procedures that were applied to the ici-a. as shown in table 1, the ahp weights for the ici-m differ from those of the ici-a. the average ici-m weight for firms in our sample is 0.1816. applying the average ici-m value to the coefficient in table 2, column 3, we see that the average impact on the roa would be approximately -1.7% (.1816 multiplied by 0.093). this percentage of -1.7% is similar to the corresponding values generated by applying the bv and the ici-a. the maximum value of the ici-m is 0.857. applying this value to the coefficient would result in an estimated decrease of 7.97% (0.857 multiplied by -0.093). for a firm that reports the maximum ici-m, their corresponding roa would be -2.04% (5.93% less 7.97%). as with the ici-a, the impact on the roa is dependent upon the types of mwics that the firm reports. these three examples illustrate the utility of the ahp approach presented in this paper. using the ahp to identify and apply weights to the mwics allows stakeholders to gain more detailed and granular information than can be obtained with a binary measure such as the bv. therefore, the above analysis leads us to reject hypothesis h2. 5. conclusions the above analysis generates several important implications for researchers, the stakeholders of firms, and regulators. the most obvious of these implications are as follows. first, the negative association between mwics and firm performance continues to exist. therefore, the emphasis on strong internal control systems by management, investors and regulators continues to be well justified. second, management has a strong incentive to avoid reporting any mwics, regardless of type, because of the impact on performance. the fact that external auditors identify a much larger share of mwics than management clearly shows that management is responding to this incentive. therefore, external auditors need to continue to be vigilant in their efforts to identify mwics not identified by management. furthermore, in their role as advisors, external auditors need to emphasize the importance of preventing mwics, as well as the importance of quickly remediating any mwics that do arise to their clients. third, regulators (and in particular the sec) need to re-evaluate the incentives, or lack thereof, for firms to report mwics in their initial internal control reports rather than as part of a restatement filing. in this latter regard, it may be that regulators need to strengthen the penalties associated with identifying mwics at the time of restatements, rather than on the 10-k report. fourth, using the ahp within the internal control context can provide more granularity for a firm’s management to make decisions on how to proceed with corrective action. more specifically, a firm’s management can use the ahp to determine which types of mwics are more detrimental to different aspects of a firm’s performance and then remediate the most serious types of weaknesses that the company is reporting. research examining the association between internal control systems and firm performance has expanded significantly since the passage of sox in 2002 because of the disclosure requirements related to material weaknesses in internal control systems (mwics). the research contained in the current study confirms the findings of previous researchers concerning a negative association between mwics and firm performance. ijahp article: wilford, bodin, gordon/using the analytic hierarchy process to assess the impact of internal control weaknesses on firm performance international journal of the analytic hierarchy process 222 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.709 however, based on a large and contemporary sample, it also extends the prior research by showing that the weighted aggregation of mwics types that are reported will have an impact on performance that is not clearly defined through the use of a binary variable. the fact that a very small percentage of firms report mwics in their internal control reports filed with the sec (rice et al., 2015), coupled with the findings that most mwics are identified by external auditors (bedard & graham, 2011), suggest that management recognizes the fact that disclosing any mwics is likely to be detrimental to a firm’s performance. the findings from the current study also provide insight into why many firms report mwics in a restatement filing with the sec rather than in their initial 10-k (rice et al., 2015). as with all empirical studies, our study has limitations, of which three seem most notable. first, since all mwics measures are based on firms that have mwics, our sample is limited to assessing internal control strength based on firms that report mwics. second, we are limited to examining the time period from 2004 forward because of the sox-related data availability. third, and as noted in the introduction, internal control systems include more than just controls over financial reporting. they also include controls that promote the efficiency and effectiveness of operations and controls that ensure compliance with laws and regulations (coso, 2013). although beyond the scope of the current study, links between operations and financial reporting have been made via measures of financial performance (cheng et al., 2013; feng et al., 2015) and future research could examine whether an internal control system index could be developed that includes operational aspects of internal control. another interesting avenue for future research could examine how the different categories of criteria impact firm performance. while prior research has looked at individual types of mwics, it has not as yet categorized them as we did within the current study and this type of categorization could add additional depth to the growing research in this area. additionally, further analysis on differences between auditor and manager understanding of the relationship between controls and performance could provide insights into this area of research. as a final mention, future research could examine how the indices are related to the risk factors identified by companies within their 10-ks. ijahp article: wilford, bodin, gordon/using the analytic hierarchy process to assess the impact of internal control weaknesses on firm performance international journal of the analytic hierarchy process 223 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.709 references amponsah, c.t. (2011). application of multi-criteria decision making process to determine critical success factors for procurement of capital projects under public-private partnerships. international journal of the analytic hierarchy process, 3(2), 107-129. ddoi: https://doi.org/10.13033/ijahp.v3i2.121 asare, s. k. & wright a. (2013). the effect of type of internal control report on users’ confidence in the accompanying financial statement report. contemporary accounting review, 29(1), 152–175.ddoi: https://doi.org/10.1111/j.1911-3846.2011.01080.x ashbaugh-skaife, h., collins, d.w. & kinney, w.r. (2007). the discovery and reporting of internal control deficiencies prior to sox-mandated audits. journal of accounting and economics, 44(1-2), 166–192. doi: https://doi.org/10.1016/j.jacceco.2006.10.001 ashbaugh-skaife, h., collins, d.w., kinney, w.r. & lafond, r. (2008). the effect of sox 404 internal control deficiencies and their remediation on accrual quality. the accounting review, 83(1), 217–250. doi: https://doi.org/10.2308/accr.2008.83.1.217 balsam, s., jiang, w. & lu, b. (2014). equity incentives and internal control weaknesses. contemporary accounting review, 31(1), 178–201. doi: https://doi.org/10.1111/1911-3846.12018 bedard, j.c. & graham l. (2011). detection and severity classifications of sarbanesoxley section 404 internal control deficiencies. the accounting review, 86(3), 825–855. doi: https://doi.org/10.2308/accr.00000036 bedard, j.c., hoitash, r., hoitash, u. & westermann, k. (2012). material weakness remediation and earnings quality: a detailed examination by type of control deficiency. auditing: a journal of practice & theory, 31(1), 57–78. doi: https://doi.org/10.2308/ajpt-10190 beneish, m.d., billings, m. & hodder, l. (2008). internal control weaknesses and information uncertainty. the accounting review, 83(3), 665–703. doi: https://doi.org/10.2308/accr.2008.83.3.665 carhart, m.m. (1997). on persistence in mutual fund performance. journal of finance, 52(1), 57–82. chen, j., chan, k.c., dong, w. & zhang, f. (2017). internal control and stock price crash risk: evidence from china. european accounting review, 26(1), 125–152. doi: https://doi.org/10.1080/09638180.2015.1117008 cheng, m., dhaliwal, d. & zhang, y. (2013). does investment efficiency improve after the disclosure of material weaknesses in internal control over financial reporting. journal of accounting and economics, 56(1), 1–18. doi: https://doi.org/10.1016/j.jacceco.2013.03.001 ijahp article: wilford, bodin, gordon/using the analytic hierarchy process to assess the impact of internal control weaknesses on firm performance international journal of the analytic hierarchy process 224 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.709 cheng, q., goh, b.w., & kim, j.b. (2018). internal control and operational efficiency. contemporary accounting research, 35(2), 1102–1139. doi: https://doi.org/10.1111/1911-3846.12409 committee of sponsoring organizations of the treadway commission (coso) (2013). internal control – integrated framework. american institute of certified public accountants. darrough, m., huang, r. & zur, e. (2018). acquirer internal control weaknesses in the market for corporate control. contemporary accounting research, 35(1), 211–244. doi: https://doi.org/10.1111/1911-3846.12366 d’mello, r., gao, x. & jia, y. (2017). internal control and internal capital allocation: evidence from internal capital markets of multi-segment firms. review of accounting studies, 22(1), 251–287. doi: https://doi.org/10.1007/s11142-016-9377-8 daniel, k. & titman s. (1997). evidence on the characteristics of cross sectional variation in stock returns. the journal of finance, 52(1), 1–33. doi: https://doi.org/10.3386/w5604 dechow, p.m. & dichev, i. d. (2002). the quality of accruals and earnings: the role of accrual estimation errors. the accounting review, 77(s-1), 35–59. doi: https://doi.org/10.2308/accr.2002.77.s-1.35 doyle, j. t., ge, w., & mcvay s. e. (2007a). determinants of weaknesses in internal control over financial reporting. journal of accounting and economics, 44(1-2), 193– 223. doi: https://doi.org/10.1016/j.jacceco.2006.10.003 doyle, j.t., ge, w. & mcvay s. (2007b). accruals quality and internal control over financial reporting. the accounting review, 82(5), 1141–1170. doi: https://doi.org/10.2308/accr.2007.82.5.1141 fama, e.f. & french k.r. (1992). the cross-section of expected stock returns. journal of finance, 47(2), 427–465. doi: https://doi.org/10.1111/j.1540-6261.1992.tb04398.x fama, e.f. & french k.r. (2008). average return, b/m, and share issues. the journal of finance, 63(6), 2971–2995. doi: https://doi.org/10.1111/j.1540-6261.2008.01418.x fama, e.f. & macbeth j.d. (1973). risk, return, and equilibrium: empirical tests. the journal of political economy, 81(3), 607–636. doi: https://doi.org/10.1086/260061 feng, m., li, c., mcvay, s.e. & skaife, h. (2015). does ineffective internal control over financial reporting affect a firm’s operations? evidence form firms’ inventory management. the accounting review, 90(2), 529–557. doi: https://doi.org/10.2308/accr50909 francis, j., lafond, r., olsson, p. & schipper, k. (2005). the market pricing of accruals quality. journal of accounting and economics, 39(2), 295–327. doi: https://doi.org/10.1016/j.jacceco.2004.06.003 https://doi.org/10.2308/accr-50909 https://doi.org/10.2308/accr-50909 ijahp article: wilford, bodin, gordon/using the analytic hierarchy process to assess the impact of internal control weaknesses on firm performance international journal of the analytic hierarchy process 225 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.709 gao, x. & jia, y. (2016). internal control over financial reporting and safeguarding of corporate resources: evidence from the value of cash holdings. contemporary accounting research, 33(2), 783–814. doi: https://doi.org/10.1111/1911-3846.12164 ge, w. & mcvay, s. (2005). the disclosure of material weaknesses in internal control after the sarbanes-oxley act. accounting horizons, 19(3), 137–158. doi: https://doi.org/10.2308/acch.2005.19.3.137 ge, w., koester, a. & mcvay, s. (2017). benefits and costs of sarbanes-oxley section 404(b) exemption: evidence from small firms’ internal control disclosures. journal of accounting and economics, 63(2), 358–384. doi: https://doi.org/10.1016/j.jacceco.2017.01.001 gordon, l.a. & wilford, a. (2012). an analysis of multiple consecutive years of material weaknesses in internal control. the accounting review, 87(6), 2027–2060. doi: https://doi.org/10.2308/accr-50211 hammersley, j.s., myers, l.a. & shakespeare, c. (2008). market reactions to the disclosure of internal control weaknesses and to the characteristics of those weaknesses under section 302 of the sarbanes oxley act of 2002. review of accounting studies, 13(1), 141–165. doi: https://doi.org/10.1007/s11142-007-9046-z keane, m.j., elder, r.j. & albring, s.m. (2012). the effect of the type and number of internal control weaknesses and their remediation on audit fees. review of accounting and finance, 11(4), 377–399. doi: https://doi.org/10.1108/14757701211279178 koester, a., lim, s.c. & vigeland, r.l. (2015). the effect of tax-related material weakness in internal controls on the market valuation of unrecognized tax benefits. the journal of the american taxation association, 37(1), 129–155. doi: https://doi.org/10.2308/atax-50952 li, c., peters, g., richardson, v.j. & watson, m.w. (2012). the consequences of information technology control weaknesses on management information system: the case of sarbanes-oxley internal control reports. mis quarterly, 36(1), 179–204. doi: https://doi.org/10.2307/41410413 loughran, t. & wellman, j.w. (2011). new evidence on the relation between the enterprise multiple and average stock returns. journal of financial and quantitative analysis, 46(6), 1629–1650. doi: https://doi.org/10.1017/s0022109011000445 mao, j. & ettredge, m. (2016). internal control deficiency disclosures among chinese reverse merger firms. abacus, 52(3), 441–472. doi: https://doi.org/10.1111/abac.12087 mcnichols, m.f. (2002). discussion of the quality of accruals and earnings: the role of accrual estimation errors. the accounting review, 77(s-1), 61–69. doi: https://doi.org/10.2308/accr.2002.77.s-1.61 ijahp article: wilford, bodin, gordon/using the analytic hierarchy process to assess the impact of internal control weaknesses on firm performance international journal of the analytic hierarchy process 226 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.709 ogneva, m., subramanyam, k.r. & raghunandan, k. (2007). internal control weakness and cost of equity: evidence form sox section 404 disclosures. the accounting review, 82(5), 1255–1297. doi: https://doi.org/10.2308/accr.2007.82.5.1255 ossadnik, w., schinke, s. & kaspar, r.h. (2016). group aggregation techniques for analytic hierarchy process and analytic network process: a comparative analysis. group decision and negotiation, 25(2), 421-457. doi: https://doi.org/10.1007/s10726-015-9448-4 rezee, z., espahbodi, r., espahbodi, p. & espahbodi, h. (2012). firm characteristics and stock price reaction to sox 404 compliance. abacus, 48(4), 473–501. doi: https://doi.org/10.1111/j.1467-6281.2012.00377.x rice, s.c. & weber, d.p. (2012). how effective is internal control reporting under sox 404? determinants of the (non-) disclosure of existing material weaknesses. journal of accounting research, 50(3), 811–843. doi: https://doi.org/10.1111/j.1475679x.2011.00434.x rice, s.c., weber, d.p. & wu, b. (2015). does sox 404 have teeth? consequences of the failure to report existing internal control weaknesses. the accounting review, 90(3), 1169–1200. doi: https://doi.org/10.2308/accr-50974 saaty, t.l. (1980). the analytic hierarchy process. new york: mcgraw-hill. securities and exchange commission (sec) (2003). final rule: management's report on internal control over financial reporting and certification of disclosure in exchange act periodic reports, 33-8238. washington, d.c. http://www.sec.gov/rules/final/33-8238.htm weiss, d. (2014). internal controls in family-owned firms. european accounting review, 23(3), 463–482. doi: https://doi.org/10.1080/09638180.2013.821814 https://doi.org/10.1007/s10726-015-9448-4 ijahp article: wilford, bodin, gordon/using the analytic hierarchy process to assess the impact of internal control weaknesses on firm performance international journal of the analytic hierarchy process 227 vol. 12 issue 2 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i2.709 appendix a internal control weakness classifications from audit analytics accounting personnel resources, competency/training (act) accounting personnel resources, competency and training mwics that result from a lack of required skills or experience will be included in this category. ethical or compliance issues with personnel (eci) this category of mwics will include any deficiencies related to personnel complying with policies or ethical standards, committing fraudulent acts or intentionally misrepresenting financial reports. senior management competency, tone, reliability issues (smi) this category of mwics is reserved for issues that are related to senior management conduct. insufficient or non-existent internal audit function (ini) this category of mwics will include any issues where the company states that the mwics was related to an inadequate internal or nonexistent internal audit function. segregations of duties/design of controls (personnel) (sd) mwics issues related to the segregation of duties (i.e., separating duties between different individuals) will be included in this category. treasury control issues (tc) treasury-related mwics (i.e., cash receipts and cash disbursements) will be included in this category. journal entry control issues (jec) if an mwics states that the issue is the result of deficiencies in the journal entry process, it will be included in this category. non-routine transaction control issues (ntc) if the mwics is tagged as the result of a non-routine process (i.e., acquisition, etc.), then it will be included in this category. inadequate disclosure controls (id) if there is an issue related to the disclosure of financial reporting information, the mwics will be included in this category. material and/or numerous auditor/ye adjustments (ma) if the mwics is included because of a high number of auditor/manager proposed adjustments at year end, the mwics will be included in this category. untimely or inadequate account reconciliations (uar) if untimely or inadequate account reconciliations are identified as the reason for the mwics, it will be included in this category. accounting documentation, policy and/or procedures (adp) any mwics that is the result of inadequate documentation, policies or procedures should be included in this category. this is a category that will generally be checked whenever an mwics is reported. information technology, software, security & access issues (it) if an mwics is related to information technology issues associated with accounting and financial reporting, it will be included in this category. restatement or nonreliance of company filings (rnf) this category is used for mwics that result in the restatement of financial information. restatement of previous 404 disclosures (rpd) this category is used for mwics that result from a restatement of a prior 404 opinion. sab 108 adjustments noted (sab) this category is used for mwics where a sab 108 is used to correct financial balances related to accounting errors. ineffective or understaffed audit committee (iua) this is an mwics category used when an audit committee does not exist, have the experience, or have the independence required through sox. ineffective regulatory compliance issues (irc) this is an mwics category that is used when regulatory requirements are not met. management/board/audit committee investigations (mi) if there is an internal investigation in progress that is related to accounting or financial reporting, this category is used. sec or other regulatory investigations and/or inquiries (si) this mwics category is used when there is an sec or other regulatory investigation underway. scope/disclaimer of opinion or other limitations (sdl) this mwics category is used when a company or auditor indicates that they could not audit the internal controls. *source: data dictionary – internal controls, audit analytics. microsoft word 7b_silva_militarydecision_vol2_issue1.docx ijahp article: nascimento, da silva, belderrain / purchase of a new air superiority fighter using the analytic hierarchy process international journal of the analytic hierarchy process 66 vol. 2, issue 1, 2010 issn 1936-6744 purchase of a new air superiority fighter using the analytic network hierarchy process leila paula alves da silva nascimento aeronautics institute of technology são josé dos campos, sp, brazil leilapasn@gmail.com amanda cecília simões da silva* aeronautics institute of technology são josé dos campos, sp, brazil amanda@ita.br mischel carmen neyra belderrain aeronautics institute of technology são josé dos campos, sp, brazil carmen@ita.br abstract the f-x2 project intends to define the purchase of new air superiority fighter for the brazilian air force (baf). the purpose of this work is to present a military-based application of the analytic network process (anp) method, adapting the f-x2 project case to the use of fictitious data. the anp stands out due to the fact that its characteristics provide realistic solutions. specific literature has approached its theoretical aspects and its applications in real case and empirical studies. keywords: multiple criteria decision making, anp, f-x2 project. 1. introduction the multiple criteria decision making methods (mcdm) help the evaluation of a set of alternatives in relation to a certain set of criteria. the mcdm has many effective methods to organize a complex problem into a structure that makes it possible to be analyzed such as, the analytic network process (anp) method. the anp is characterized by means of the decomposition of a decision-making issue in a network structure, with no hierarchical relationship among its elements. actually, the anp allows relationships of dependence and feedback among the elements. taking into account that often in real world there is dependence among those criteria, the anp provides a more realistic representation of the problem to the decision-maker. the purpose of this work is to perform an application of the anp method in a military decision environment, simulating the choice of a new air superiority jet fighter for the brazilian air force (baf). it is the so-called f-x2 project, confidentially managed by baf and by the ministry of defense. the project data included in this work were collected from media publications; therefore, it does not represent at all the completion of the process.  corresponding authors rob typewritten text http://dx.doi.org/10.13033/ijahp.v2i1.73 rob typewritten text ijahp article: nascimento, da silva, belderrain / purchase of a new air superiority fighter using the analytic hierarchy process international journal of the analytic hierarchy process 67 vol. 2, issue 1, 2010 issn 1936-6744 the f-x2 project is a complex decision-making issue that involves alternatives and multiple criteria and, at first, there might be dependent relationships among them. for this reason, a method which involves quantitative and qualitative criteria and that also allows dependent relationships becomes necessary to better approach the problem. the anp complies with such requirements. thus, the present work is structured as follows: section 2 is comprised of the anp method. section 3 outlines the application of the method to the addressed problem. section 4 deals with final observations and last, the references used. 2. theoretical reference “decision–making issues may be difficult to solve, due to the complexity, the uncertainty inherent to the situation, the presence of multiple objectives and different perspectives, which might lead to different conclusions” (clemen and reilly, 2001). a decision problem is considered multicriteria when it has, at least, one decision context, two conflicting evaluation criteria, a finite set of alternatives, a measure scale, decision matrixes and an accrual method of preferences. the mcdm methods are meant to clarify the decision-making process above all, so they can help decision-makers evaluate and choose the alternatives to the problem. it is important to point out that the objective of mcdm methods is not to present to decision makers the optimum solution of the problem but instead to help them, bearing their preferences in mind. 2.1 analytic network process (anp) first published by thomas l. saaty in 1996, the analytic network process (anp) is a discrete method of the mcdm american school. it is regarded as a generalization of the analytic hierarchy process (ahp) and it utilizes a network (instead of hierarchy) with no need to specify levels, also allowing dependent relationships between elements. anp overcomes the limitation of the linear hierarchical structure while contradicting the axiom of independence (saaty, 2005). according to saaty (1999), anp “synthesizes the dependency effect and feedback inside and among sets (clusters) of elements”. in figure 1, it is observed that a network is a non-linear structure expanding in all directions. it has nonorganized clusters in a pre-defined order and presents influence (or dependence) relationships, which are transmitted in the same set of elements (inner dependence) and also among sets (outer dependence). the dependence relationships among the elements of clusters c4-c2, c4-c3, c1-c4 e c1-c2 of figure 1 represents some examples of outer dependence. the dependence relationship among the elements of clusters c2-c3 characterizes feedback. a loop in cluster c1 indicates an inner dependence of its elements (saaty, 2005). ijahp article: nascimento, da silva, belderrain / purchase of a new air superiority fighter using the analytic hierarchy process international journal of the analytic hierarchy process 68 vol. 2, issue 1, 2010 issn 1936-6744 figure 1. network structure of the anp (source: saaty, 2005) as for the outer dependence relationships among clusters in network structure, they can be classified as shown in figure 2, as: 1) source component, whose elements influence any elements of any other cluster, although they are not under any influence; 2) intermediate component, which is influenced and influences an element of another cluster; and, 3) sink component, which is influenced by the elements of another cluster (saaty, 2005). figure 2. connections in a network (source: saaty, 2005) aiming to facilitate the understanding of the concept of anp, a flow chart of the method is presented in figure 3. the suggested flow chart identifies the following stages and the necessary steps to the application of the anp: stage 1: formulation of the decision problem ● step 1 – problem structuring; ● step 2 network construction. ijahp article: nascimento, da silva, belderrain / purchase of a new air superiority fighter using the analytic hierarchy process international journal of the analytic hierarchy process 69 vol. 2, issue 1, 2010 issn 1936-6744 stage 2: judgments ● step 1 construction of global and local reachability matrices; ● step 2 pairwise comparison of the elements and clusters; ● step 3 verification of the consistency of judgment; ● step 4 acquisition of eigenvectors and clusters weights matrix. stage 3: algebraic development ● step 1 construction of unweighted supermatrix; ● step 2 acquisition of the weighted supermatrix; ● step 3 verification of the weighted supermatrix stochasticity; ● step 4 acquisition of the limit matrix; ● step 5 final result. more details about the anp method may be found in saaty (2005). figure 3. anp method application flow chart: stages and steps followed in the anp method (source: authors) ijahp article: nascimento, da silva, belderrain / purchase of a new air superiority fighter using the analytic hierarchy process international journal of the analytic hierarchy process 70 vol. 2, issue 1, 2010 issn 1936-6744 3. application of anp: the f-x2 project case the f-x2, a project worth billions of brazilian reals, aims at re-equipping the baf jet fighter fleet with fifth-generation supersonic aircraft. they will replace the current mirage 2000 fighter planes initially, and afterwards, the f-5 fighters. it is an old baf project, started in 1998, during president fernando henrique cardoso’s term. due to administrative reasons, the fx project (or phoenix project) was not concluded. the f-x2, reactivated by the current president, has been dealt with on a confidential basis by both baf and the ministry of defense, since it handles confidential warfare data of the countries involved. in 2007, the re-edition of the baf directive dca 400-6 (aeronautical systems and materials life cycle) was approved by the aeronautical command, in view of “setting up the planning and execution of the necessary steps and main events of aeronautical systems and materials life cycle” (dca 400-6, 2007). the directive governs all of the aeronautical command systems and materials life cycle. it includes: the detection of an operational or logistics deficiency, technological or economic opportunity; the phase of creation; viability, definition, development/purchase, production, implementation, utilization, revival, modernization or improvement through its deactivation, when the life cycle is finished (dca 400-6, 2007). thus, the dca 400-6 sets the guidelines for the process of acquisition of the baf fighter planes. the acquisition process will be “direct choice” which, in other words, means exemption of a bidding process. the brazilian requirement is the commercial offset agreement, described as “one of the main focal points of the program, so as to provide qualification to the national industrial depot and also to allow an effective participation in future improvements and updates throughout the equipment life cycle” (o estado de são paulo, 2008). six international manufacturers of fighter planes models – jas-39a gripen, su-35 e, rafale-c, f-18 e/f, f-35 lightning and typhoon – replied to the request for information (rfi) forwarded by brazil, which was the first formal contact with the companies or interested governments. “the alternatives are analyzed upon evaluation of risks, timeline and cost-benefit relation and the strategical execution of various activities which the systems and materials life cicle is comprised of” (dca 400-6, 2007). the purpose is to collect data to set forth technical, logistical and industrial requirements (tlir) and to draw detailed technical specifications, which will be part of the agreements of development or acquisition. after analyzing the manufacturers’ replies to the rfi, the selection of three of the six initial fighter jets – rafale-c, f-18 e/f and jas-39a gripen – was announced. a request for proposal (rfp) was forwarded to these parties, having as its parameters the requirements set forth in tlir. the f-x2 project is currently in this step and, the brazilian government will soon announce the chosen manufacturer. to illustrate an application of anp method, fictitious data regarding the f-x2 project were adopted. the intention of this work is to demonstrate that the anp method can be applied in case of decisionmaking issues in many different areas, including the military environment. the application will simulate the two phases (rfi and rfp) of the selection process, using anp in both of them. first, the anp will be applied to the six initial alternatives of fighter planes, selecting three of them. upon a new application of the anp, the final selection will be obtained. in both phases, the anp was reproduced in the free software superdecisions, available at www.superdecisions.com, considering the stages and steps included in the flow chart of figure 3. ijahp article: nascimento, da silva, belderrain / purchase of a new air superiority fighter using the analytic hierarchy process international journal of the analytic hierarchy process 71 vol. 2, issue 1, 2010 issn 1936-6744 3.2 illustrative example of f-x2 project the problem is contextualized as follows: “selection of a supersonic aircraft manufacturer for the reequipment of the baf jet fighter fleet”. due to the fact that it is an illustrative example, only the criteria relevant to the authors were considered. the existing dependency relationships, as well as the pairwise comparisons were also determined by the authors (decision-makers, in this case), without any knowledge of military issues. nevertheless, some military personnel, who were not involved with the f-x2 project were consulted and they were able to provide important data, such as elements, dependency and comparisons. as previously explained, the selection of fighter planes occurred in two phases. in phase 1, referred to as “pre-selection”, resulting from rfi, macro criteria were used, that is, with no specific details. and, in phase 2, named “final selection”, resulting from rfp, the criteria included in the previous stage were further discussed, leading to more information and details than in the pre-selection phase. phase 1: pre-selection – rfi stage 1: formulation of the decision problem ● step 1 – problem structuring: the objective, clusters, elements and alternatives must be defined. in this case, the objective is to select three manufacturers of fighter planes. there are two clusters to be considered: general aspects and alternatives, each one including its respective elements. the cluster general aspects consists of the following elements: costs, offset, performance, weaponry, airborne systems (avionics and sensors) and logistics. the cluster alternatives consists of the following elements: jas-39a gripen, su-35 e, rafale-c, f-18 e/f, f-35 lightning and typhoon. ● step 2 network construction: the dependency relationships and feedback among the elements of clusters must be set forth. figure 4 presents phase 1 network. it is observed the inner dependence among the elements of the cluster general aspects. besides, its elements influence the ones of the cluster alternatives and vice-versa, which characterizes a feedback. figure 4. network structure to phase 1 ijahp article: nascimento, da silva, belderrain / purchase of a new air superiority fighter using the analytic hierarchy process international journal of the analytic hierarchy process 72 vol. 2, issue 1, 2010 issn 1936-6744 stage 2: judgment ● step 1construction of global and local reachability matrices: to better view the existence of network dependence relationships, there must be constructed matrices of global and local range. table 1 illustrates the global reachability matrix. due to the fact that the elements of the cluster alternatives were considered independently from each other, it is possible to observe the global reachability matrix at zero value, concerning the interaction among its elements. the other interactions receive a value 1, indicating that there is dependence among the elements. table 1. global reachability matrix for the aircraft selection example clusters alternatives general aspects alternatives 0 1 general aspects 1 1 table 2 illustrates the local reachability matrix, which specifies the dependence relationship among the elements of the clusters. value 1 is assigned whenever dependence occurs. if there is none, a zero-value is assigned. table 2. local reachability matrix for the aircraft selection example elements f-18 f-35 gripen rafale su-35 typhoon w o c p l as f-18 e/f 0 0 0 0 0 0 1 1 1 1 1 1 f-35 lightning 0 0 0 0 0 0 1 1 1 1 1 1 jas-39a gripen 0 0 0 0 0 0 1 1 1 1 1 1 rafale-c 0 0 0 0 0 0 1 1 1 1 1 1 su-35 e 0 0 0 0 0 0 1 1 1 1 1 1 typhoon 0 0 0 0 0 0 1 1 1 1 1 1 weaponry (w) 1 1 1 1 1 1 0 0 0 0 0 1 offset (o) 1 1 1 1 1 1 1 0 0 0 0 1 costs (c) 1 1 1 1 1 1 1 0 0 1 1 1 performance (p) 1 1 1 1 1 1 1 0 0 0 0 1 logistics (l) 1 1 1 1 1 1 1 0 0 0 0 1 airborne systems (as) 1 1 1 1 1 1 1 0 0 0 0 0 the construction of binary matrices is followed by paired comparisons, required to every connection existing on the network. in order to do that, saaty’s fundamental scale is used. ● step 2 pairwise comparisons of the elements and clusters: such comparisons aim at obtaining eigenvectors of these elements and the weight of each cluster, respectively. the comparisons to be performed are the ones in which a single element of a cluster has a dependence relationship with a minimum of two elements of another cluster. ● steps 3 and 4 verification of the consistency of judgment and acquisition of eigenvectors and clusters weights matrix: the verification consists of observing if the values of the consistency ratio (cr), obtained from the decision matrices are within the maximum limit of acceptance of 0.10. in this work, all of the decision matrices are consistent. thus, both eigenvectors and clusters weights are registered. the registered eigenvectors are now called relative priority vectors of the judged elements. once all the comparisons are performed and all of the consistencies are verified, the next stage is started. ijahp article: nascimento, da silva, belderrain / purchase of a new air superiority fighter using the analytic hierarchy process international journal of the analytic hierarchy process 73 vol. 2, issue 1, 2010 issn 1936-6744 stage 3: algebraic development ● step 1 construction of unweighted supermatrix: from the accrual of the eigenvectors obtained at the paired comparisons among the elements of the previous stage. ● steps 2 and 3 acquisition of the weighted supermatrix and verification of its stochasticity: it is the result from multiplying the cluster weight matrix (which consists of the eigenvectors of the comparisons among clusters) with the unweighted supermatrix. this one shall be stochastic in relation to the columns (the total amount of the elements of the column shall be 1). if the weighted supermatrix is not stochastic, related to the columns, it should be normalized to make it stochastic. ● step 4 acquisition of the limit matrix: obtained raising the weighted supermatrix to the powers until its convergence. the limit matrix shall also be stochastic in relation to the columns. the final result can be observed in it. ● step 5 final result: the final result is obtained with the aid of the alternatives priority ranking. in this case, the three fighter planes presenting the major ranking proportions (figure 5) are: rafale-c > f-18 e/f > jas-39a gripen. followed by su-35 e > typhoon > f-35 lightning. regarding the elements of the cluster general aspects, the elements obtaining the major priorities were costs and offset, with priority levels of 0.26249 and 0.22276, respectively. figure 5. final result of anp to phase 1. regarding the elements of the cluster general aspects, the elements obtaining the major priorities were costs and offset, with priority levels of 0.26249 and 0.22276, respectively (table 3). table 3. elements of the cluster general aspects priorities for the aircraft selection example elements normalized by cluster limiting weaponry 0.029050 0.04809 offset 0.134575 0.22276 costs 0.158576 0.26249 performance 0.134113 0.22200 logistics 0.055435 0.09176 airborne systems 0.092377 0.15291 thus, phase 2 begins with the three first fighter planes of the ranking (figure 5) as alternatives to the decision-making issue. ijahp article: nascimento, da silva, belderrain / purchase of a new air superiority fighter using the analytic hierarchy process international journal of the analytic hierarchy process 74 vol. 2, issue 1, 2010 issn 1936-6744 phase 2: final selection rfp aiming to improve the model and attempting to imitate the reality, each element of the cluster general aspects of the pre-selection phase is now considered a cluster. the elements are associated to each one. therefore, the following clusters and elements are created: performance cluster (p): range ratio, maximum speed, operational altitude, load capacity, take-off, landing, maneuver and climbing performances; offset cluster (o): industrial, commercial and technological; costs cluster (c): aircraft purchasing, maintenance and operation costs (flight hour); airborne systems cluster (as): radar, self-defense, data-link, helmet mounted display (hmd) and flight refueling; logistics cluster (l): maintenance and support equipment, technical publications, spare parts, staff training and contracted logistic support; weaponry cluster (w): cannons, missiles and intelligent bombs. it is worth highlighting that phase 2 followed the same procedure presented in the flow chart of figure 3. however, the steps and stages are not shown here. phase 2 network structure is shown in figure 6. ijahp article: nascimento, da silva, belderrain / purchase of a new air superiority fighter using the analytic hierarchy process international journal of the analytic hierarchy process 75 vol. 2, issue 1, 2010 issn 1936-6744 figure 6. network structure to phase 2 table 4 presents the priorities of the each element of the clusters. among them, the ones with major priority are: technological (from offset cluster) with a priority of 0.68296; and aircraft purchasing cost (from costs cluster) with priority of 0.53766. such result confirms the data obtained in phase 1 and it shows that the technological offset and the aircraft purchasing cost are critical to the selection of the fighter planes. table 4. elements priorities for the aircraft selection example clusters elements normalized by cluster limiting weaponry intelligent bombs 0.034823 0.32662 cannons 0.034831 0.32670 missiles 0.036961 0.34668 offset commercial 0.005849 0.09409 industrial 0.013860 0.22295 technological 0.042457 0.68296 costs aircraft purchasing cost 0.101740 0.53766 maintenance cost 0.054339 0.28716 operation cost 0.033147 0.17517 performance load capacity 0.034256 0.23557 maneuver performance 0.012385 0.08517 climbing performance 0.015361 0.10563 take-off performance 0.010039 0.06904 landing performance 0.009913 0.06817 range ratio 0.029416 0.20229 operational altitude 0.011115 0.07643 maximum speed 0.022933 0.15770 logistics maintenance and support equipment 0.014596 0.19169 spare parts 0.010063 0.13216 technical publications 0.013230 0.17375 contracted logistic support 0.014957 0.19643 staff training 0.023299 0.30598 airborne systems self-defense 0.014900 0.20790 data-link 0.012167 0.16976 hmd 0.007288 0.10169 radar 0.012274 0.17126 flight refueling 0.025041 0.34939 the final result of this phase of the illustrative example is shown in figure 7, with the following alternatives priority ranking: rafale-c > f-18 e/f > jas-39a gripen. figure 7. final result of anp although it is only an illustrative example of the anp application, the results obtained in both phases were satisfactory and coherent with what is published in the media. ijahp article: nascimento, da silva, belderrain / purchase of a new air superiority fighter using the analytic hierarchy process international journal of the analytic hierarchy process 76 vol. 2, issue 1, 2010 issn 1936-6744 4. final observations the multiple criteria decision making method (mcdm) has as its objective to evaluate and to provide a final ranking of the alternatives dealt with, including complex decision-making environments, involving external factors, such as, multiple decision-makers, conflicting judgment, cost criteria, among others. various methods of support to the decision-making process are discussed in the specific publications. in this work, the analytic network process (anp) method was used in this work. the anp method has as its characteristics the inclusion of qualitative and quantitative criteria, structured in networks, in which the dependence relationships and feedback among the elements are allowed. this work’s intention was to present an application of anp, adapting the f-x2 project case, baf's project of purchase of a new air superiority fighter to re-equip the brazilian fleet. the criteria used as clusters and elements, as well as pairwise comparisons, were assigned by the authors and do not correspond to the real project. moreover, it is observed that the anp method can be applied in case of decision-making issues involving military environment. the f-x2 project could benefit from the use of ratings and bocr merits (benefits, opportunities, costs and risks). there are many ways of evaluating and selecting fighter aircraft alternatives to the problem at hand. so, the involved parties should be the ones to decide and adapt the best method to the decision-making process, according to their specific demands. references clemen, r.t. & reilly, t. (2001). making hard decisions. califórnia: duxbury. ministry of defense, aeronautical command, logistic, baf directive dca 400-6, (2007). aeronautical systems and materials life cycle. [in portuguese] o estado de são paulo, july 2, 2008. fab dá início a processo para definir empresa que venderá caças ao país. [in portuguese] saaty, t.l., “fundamentals of the analytic network process”. proceedings of the isahp v, kobe, japan, 1999, august 12-14. saaty, t.l. (2005). theory and applications of the analytic network process: decision making with benefits, opportunities, costs and risks. pittsburgh, pa: rws publications. saaty, r. w. (2003). decision making in complex. the analytic hierarchy process for decision making and the analytic network process for decision making with dependence and feedback [superdecisions tutorial]. retrieved jun 01, 2008, from http://www.superdecisions.com. ijahp article: minzoni, mounoud, zahraei/ using anp to design a living system like balanced operating model for intangible services international journal of the analytic hierarchy process 148 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.267 using ahp to design a living system like balanced operating model for intagible services angela minzoni * ecole centrale paris / laboratoire génie industriel chatenay-malabry france chair. operational efficiency and management systems corporate chair bnp paribas / ecp angela.minzoni@ecp.fr eléonore mounoud * ecole centrale paris / laboratoire génie industriel chatenay-malabry france chair. operational efficiency and management systems corporate chair bnp paribas / ecp eleonore.mounoud@ecp.fr majid fathi zahraei post doctoral fellow. operational efficiency and management systems corporate chair bnp paribas / ecp ecole centrale paris / laboratoire génie industriel chatenay-malabry france majid.zahraei@ecp.fr 1 abstract there is no need to present another anp-based approach for weighting criteria within decision making contexts. academic literature in disciplines like economics, engineering, political sciences, statistics or mathematics testify to the broadness of topics, situations and cultures where the method’s value has been proven. from supply planning or road mapping to monitoring, from crisis management or banking crime to rural water supply, decision makers –both in governments or firmshave implemented this method on the five continents. however, the method has been used less in areas beyond decision making. for example, in the field of innovation or organisational design, it can be expected that anp facilitates the formulation of a suitable dynamic to combine the complex links of experts’ 1 the authors are thankful to jean guy caputo and antonmaria minzoni alessio for their contribution in the understanding of mathematical concepts applied to the study of living and dynamic systems. mailto:angela.minzoni@ecp.fr mailto:eleonore.mounoud@ecp.fr mailto:majid.zahraei@ecp.fr ijahp article: minzoni, mounoud, zahraei/ using anp to design a living system like balanced operating model for intangible services international journal of the analytic hierarchy process 149 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.267 representations of the system under study. we advance the hypothesis that the anp method’s friendliness to rough data, and ability to combine tangible and intangible information can be useful for providing the relevant interactions for living systems like modelling which is at the centre of the cross disciplinary creativity needed to design innovative and balanced operating models. in this context “operating models” shall be understood as the descriptions of how organizations operate across processes, people and technology in order to accomplish their functions. keywords: innovation; design; strategy; operations; intangible services 1. introduction this study was motivated by one basic assumption and two main issues stemming from the academic literature and an organization’s observation. the basic assumption states that operating models for the tangible, manufacturing industry are less and less adequate for intangible knowledge and information based firms. excessive process optimisation has lessened the global vision of the system in which processes take place affecting the ways indicators are calculated, data are gathered, supervision is carried out and ultimately decisions are made or alternatives identified. there is a need to find new ways of linking the global vision and in particular, understanding their dynamic interactions among the macro and the micro levels other than the well-known traditional subordinate alignment. organizations delivering intangible services have not yet developed any specific designs regarding operating models. they have mostly applied and adapted ready to think models issued from the manufacturing industry. as a result, organizations have a structure oriented, mechanistic approach of operations that is less and less consistent within today’s changing environment that calls for organic-like organizations (burns and stalker, 1961; bellomo, 2008; collinson, 2012). mechanistic shall be understood here as the fundamental conviction that operations need to be homogenised, standardized, overparcelled and over-optimized until tasks are reduced to their more elementary content, lose value and become ideal candidates for off shoring. we feel this philosophy has reached its limit because of the enormous bureaucracy required and the mere fact that it doesn’t make sense for workers, managers and decision makers. operating models in this study are understood under a broad perspective focusing not only on the way processes are engineered and operated but also including strategy, governance and resources issues. thus we have defined operation model as “from the governance to the tools”. this basic assumption underlines the search for a new operating model design focused on balance, this being understood through the metaphor of the living systems long lasting homeostatic regulations. the two main goals of this paper are as follows: 1) the elaboration of a cross-disciplinary integrated research methodology (including organizational sciences, anthropology and operations research) able to help the organization design an innovative operating model and thus contributing to academic research by innovating in terms of methodology. ijahp article: minzoni, mounoud, zahraei/ using anp to design a living system like balanced operating model for intangible services international journal of the analytic hierarchy process 150 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.267 2) the use of the analytic network process (anp) method for a design purpose, that is, beyond the traditional decision making context. this new kind of application leads to in-depth examination of the organizational features at stake in short or medium term decision making contexts and long term prospective design contexts. this approach also leads to the examination of the role of experts in these two different contexts even if this study focuses on organizational design. we postulate that the anp approach can be central to such a methodology as it is able to cope with very diverse parameters, including multiple sloppy factors and to challenge data gathering, expert’s status and legitimacy, and the role and understanding of subjectivity. we notice that anp literature until now has not developed an in-depth approach on human factors related to experts, to the numeral expression of ratings during pairwise comparisons nor to the expert’s learning dynamics of their own system as much as it has developed the algorithmic reflection. in the following pages we will present in detail a cross-disciplinary action-research initiative in progress. this work is undertaken in a highly knowledge-intensive itsupported industry. it involves mid and top-level managers in the task of designing a new overarching organizational system, encompassing both operational and strategic issues. the anp method, in combination with organization sciences and anthropology methods, is used first to assess the criteria involved in the current model, and second to design a new model. a key challenge is to be able to go beyond a mere structural description of an operating model in terms of units and linkages. operating models are detailed descriptions of how organizations operate across processes, people and technology in order to accomplish their functions. operating model design should not be confused with detailed design of processes, systems and organization structures. for example, an operating model might specify which business processes are needed and whether those processes should be outsourced, centralized as shared services or handled by business units and where they should be ‘shored’. it would not however define the details of the processes to be executed. in particular, operating models indicate the level of process automation and standardization. current practices such as accumulating processes for standardized products as a way to achieve efficiency has led to high process redundancies and a high degree of complication in operating models often referred to as “bad complexity”. thus a new call for simplicity is currently being made. we believe that a radically new approach to operating models is needed. we believe this can be done by looking at the field of design, and borrowing proven practices of co-creation to foster innovative operating models based upon multi-disciplinary insights in the concept of equilibrium. 2. literature review references have been chosen according to their potential to inspire innovation in the field of operating models under the broad focus of “from the governance to the tools”. this requires an exploration of the literature, under a cross disciplinary focus, on topics such as governance, strategy, business models, operating models, resilience, design, neural networks, human factor, narratology, mapping, creativity, cybernetics, cognitive sciences, systemic approaches, subjectivity, etc. a special focus has thus been given to literature highlighting complexity, networks and links. this vast topic overview is necessary in ijahp article: minzoni, mounoud, zahraei/ using anp to design a living system like balanced operating model for intangible services international journal of the analytic hierarchy process 151 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.267 order to carry out accurate criteria identification for the multi-criteria modelling of operations. it is also necessary to animate the working meetings with experts, who are, at the same time, empowered by the organization to be the members of the design team for the future operating model prototype. existing or literature about multi-criteria issues in general and about operating models in particular is almost exclusively focused on decision making models. according to da silva et al. (2009), before 2003 these methods, and in particular those concerning supply chain issues, used almost exclusively quantitative criteria (for ex: linear weighting models, total cost ownership, principal component analysis). after 2003, more attention began to be given to qualitative criteria and, consequently, actual methods tend to use both. the isahp 2014 was innovative in its suggestion of the use of anp for exploring situations beyond decision making. organizational sciences as well as anthropology will look at operating model multi-criteria issues with a qualitative approach. these disciplines will also pay attention to the ways that quantitative data are gathered and interpreted. actually, these disciplines are developed in specific and often separate research departments and institutions. consequently, the present research-inprogress is innovative because of the simultaneous integration of three disciplines in a single research department and methodological approach. the anp approach has proven to be well adapted to the context of the present study because of its capacity to deal with heterogeneous and sloppy data through pairwise comparisons and consistency index calculations. 2.1 cross-disciplinary approach we detail the various issues that these three disciplines focus on when they deal with the differences in the ways organizations operate and the consequences it has on their equilibrium. organizational research focuses on issues like resilience and management of high-reliability organisations. operational research focuses on issues like costs, benefits, opportunities and risks of operating models’ characteristics such as outsourcing or off-shoring. anthropology focuses on cross-disciplinary methods, group dynamics, creativity and cultural and cognitive issues. going beyond decision making needs to develop an integrated research methodology where the issues mentioned above are put to use simultaneously to address n. bellomo’s (2003) fundamental statement, “the main problem consists in understanding how the qualitative interpretation of social reality, delivered by research in the social sciences, can be transferred into quantitative description produced by mathematical equations”. we can add that this is even more true in domains like governance, strategy or management where data are essentially qualitative and related to notions like power, leadership, followership, influence, intuition, information gaps, experience, and trust. next, we will discuss the relevant streams of organizational research pertinent to our cross disciplinary approach. 2.2 high-reliability organizations organizational research has been scrutinizing high-reliability organizations (hro, organizations in which accidents rarely occur despite the error-prone nature of the work such as nuclear plants, aircraft carriers etc.) for more than 20 years to understand how they are able to operate safely despite challenging operating conditions. as work in many organizations had become increasingly fast-paced and the margin for error has become smaller, the notion of ‘high reliability’ has received growing attention in many industries. at the core of high reliability organizing is a set of principles embodied in processes and ijahp article: minzoni, mounoud, zahraei/ using anp to design a living system like balanced operating model for intangible services international journal of the analytic hierarchy process 152 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.267 practices that enable organizations to focus attention on emergent problems and to deploy the right set of resources to address those problems. noticing and responding to small disturbances and vulnerabilities allow people to take corrective action before escalation into crisis. the issue is that small problems are very difficult to detect. weick and suttclife (1999) have characterized hro by five key principles that facilitate both problem detection and problem management. problem detection involves three of the principles. firstly, using failure and near failure as ways to gain insight into the strengths and weaknesses of the system; secondly, avoiding the tendency to minimize or explain away problems; and thirdly, being aware of operations interconnectedness and how problems in one area can spread to other areas. problem management is based upon the last two principles: developing the capability to cope with un-expected events (that is resilience), and understanding where the expertise is in the organization and ensuring that decisions about how to deal with problems are made by those experts (that is expertise recognition or deference) . the processes and practices that enact those principles might differ depending on the organization’s context and the set of resources and constraints that it faces. reliability and safety are difficult to account for because what is not happening (catastrophic error) is more visible than what is happening (timely human adjustments). consequently, reliability and safety have been described as ‘dynamic non-events’ (weick, 1987 & 2011). ‘dynamic’ means that reliability and safety result from managing continuous change, and ‘non-events’ means that we recognize reliability and safety by the absence of errors and accidents. as dynamic non-events, reliability and safety must be recurrently accomplished every day. so reliability can be defined as a dynamic set of properties, activities, and responses and thus open the question of the balance or inner equilibrium of the system. although the high-reliability organization (hro) literature identifies several attributes that differentiate hros from other types of organizations, these studies do not explain how an hro comes into being (there are a few exceptions). o'neil and krane (2012) built a 97-year longitudinal case study tracking the emergence and continuation of hro characteristics in the federal aviation administration’s air traffic control operations to tackle that issue. their analysis shows that hro characteristics emerged incrementally over an extremely long period of time, and policy changes preceded organizational changes early in the process of hro development, but the relationship of policy change to organizational change decreased in later stages. these findings should be related to the most recent work on financial markets by martinez-moyano et al. (2014) dedicated to rule development, compliance, and organizational change. this study used causal-loop modeling from system dynamics to articulate the tension between production goals that focus on short-term, certain, salient benefits and required adherence to productionconstraining rules that attempt to mitigate long-term, uncertain, non-salient risks. they describe how organizations attend to rules depending on the nature of the benefits of production compared with those of rule compliance. their model identifies the operative mechanisms responsible for the development of pressures for production and for rule compliance in organizations, providing an explanation both for problem-prone organizations characterized by erosion of standards and increased violations and for organizations following rules more reliably. these two pieces of research encourage us to articulate both organizational and institutional features in our modeling. ijahp article: minzoni, mounoud, zahraei/ using anp to design a living system like balanced operating model for intangible services international journal of the analytic hierarchy process 153 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.267 the anp method (saaty, 1999, 2005, 2008) is a discrete, multiple criteria method that is characterized by the decomposition of a problem into a network structure without hierarchical relations among its elements and is a suitable approach within the context of the current study. this method allows the establishment of relations of dependence and feedback among criteria and alternatives. the way the problem or the situation or the system is decomposed is thus of paramount importance. this decomposition will be a specific and iterative one where the problem will be stated and decomposed and restated and recomposed in a parallel movement. the goal of the current research is to take advantage of this method to imagine a new system and not simply choose between solution alternatives for a well delimited problem. organization sciences and anthropology have the adequate tools and methodologies to address this question and to facilitate expert’s identification of the suitable criteria needed for the anp, as well as to put to use anp results towards the final objective. 2.3 mathematical modeling of living systems mathematical sciences applied to biology will help to address the dynamic equilibrium and stability issues. we stress that the particular focus on biology is key to the research presented in this paper because of its particular non-reciprocal and non-linear network specificities, moreover the behavior of the operating system under study may depend on more than just the present state and the number of states might not be finite. a convenient definition of equilibrium for our purposes is taken from mathematical modelling in biology (j.d. murray, 2008). it is said that the system is in equilibrium provided that its structure and relations are such that its state does not change in time. moreover, if the system is perturbed and recovers the original state we say that the equilibrium is stable. this notion is appropriated for a fuzzy cognitive map (fcm) formulation of the system. literature about fcm has also been useful within the context of the present study. first introduced by political scientist r. axelrod (1976), these are causal maps linking goals, values and trends. they allow systemic influence propagation, in particular forward and backward chaining and are well adapted to soft knowledge domains (kosko, 1986, stylos & groumpos, 1999). according to aguilar (2005), fcm are the fusion of the advances of the fuzzy logic and cognitive maps theories. this approach offers the advantage of quantified results adapted to the understanding of dynamic systems, especially interesting when the systems involve significant nonlinear feedbacks and hard to find or uncertain numerical data. mapping is thus recognized in disciplines like politics, anthropology, economics, management sciences, cognitive, educational or environmental sciences to explore similarities and differences between stakeholders’ understanding of an issue and to improve communication between stakeholders. mapping integrates different perspectives for improving the overall understanding of a multi-functional system.in the case of this study the broad overview of operating models. this technique identifies and overcomes stakeholders’ knowledge limitations and misconceptions associated with a given resource (jones et al., 2011) ijahp article: minzoni, mounoud, zahraei/ using anp to design a living system like balanced operating model for intangible services international journal of the analytic hierarchy process 154 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.267 we can state that unless operating models are studied as living systems they will not reflect the complexity and dynamism of operating models in knowledge-based organizations. we shall also remember here the attested weakness of a fast running, over optimized, highly fragmented system left without error margins or resilient capacity. in the complicated mechanistic environment that has been raised up, operational efficiency seems to be supported by increasing multiple level procedures and rules, more regulatory mechanisms and long top down delegation cascades. this has lessened the innovative capacity, the long term view and quality of professional relationships among operators and also among the hierarchy. 2.4 anthropological approach there are still very few anthropological discussions on operating models. these will particularly highlight cultural issues among onshore and offshore dynamics (eaton, 2011) such as the tools of globalization, the round-the-clock style, the primary and secondary communication channels, and workplace privacy, but will also take into account power outages per day, number of laptops needed for virtual meetings, the number of emails and the sending practices linked to this communication. these studies call attention to the fact that predominant theories about offshore outsourcing focus mainly on cost incentives and short-term efficiency gains which can lead to an important percentage of failures for offshore outsourcing activities because of the neglect of people related cultural specificities and the obscuring of the longer term potential risks and costs. these theories would also evaluate offshore outsourcing practices only from the perspective of the firm who is looking to outsource. it can also be noticed that operational research papers rarely take into account the impact of their optimization calculations on people competencies, lack of control over the processes, irregular quality, power relations or hidden costs let alone country policies or regulations. the anthropological perspective will also draw the attention to the local perception of the global and suggest the development of a perspective from the local to the global analyzing how global aspects are put into local frames and how to find locality within globalization (korff, 2003). anthropologists will also focus on the offshore outsourcing discussions and jokes in the media, television talk shows, cartoons or movies. media conveys stereotypes about lowcost, low-value activities and people performing them, the “yes men”, dependent syndromes, and electronic shake hands, etc. as highlighted by the pioneering chicago school study by thomas & znaniecki between 1918 and 1920 about the crossed impacts of immigration both in the left country and in the newly reached one, the use of narratives is singularly adequate to the expression of complex interwoven intercultural situations like the one studied in this paper. cognitivists have also stressed the important role of stories (not to be confused with anecdotes) in the learning process. this is particularly important when the learning degree needed is as high as is the need for thinking up the cycle of the object to come, in this case a new and broad spectrum operating model design. narratology techniques in design provide a way of getting a feel for new terrains, tying objectives together, expressing deep basic assumptions and identifying challenges and risks. they advocate taking into account the verbal expression of subjectivity as a ijahp article: minzoni, mounoud, zahraei/ using anp to design a living system like balanced operating model for intangible services international journal of the analytic hierarchy process 155 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.267 powerful mean for users and designers to express and address human needs, in particular at the starting point of a design process. stories are important cognitive events because they encapsulate information, knowledge, context, and emotion into one compact package. they provide a context for anchoring understanding and for assessing and integrating new knowledge and are much more memorable than facts and figures. this is particularly true at the starting point of a design process where the primary goal is not to produce a functional specification, but get a high-level vision of the system components and their dynamic relations. 3. proposed cross-disciplinary approach outlining operations in highly knowledge-intensive it-supported intangible industries involves operators, mid and top-level managers as well as researchers in the task of designing a new overarching organizational system, encompassing both operational and strategic issues. the anp approach and tools were tested first to assess the current model’s fragmented criteria, mainly cost and performance oriented, and second to create a new model where the main drivers will be the system’s balance and stability. a key challenge is to be able to go beyond a mere structural description of the multiple units and linkages of each unit cluster of the organizational model. we develop a methodology that allows us to provide an eco-system like functional description of the dependences and interdependences within the system and their impact on the robustness and resilience of the new system. the anp is central to this methodology as it is able to cope with very diverse parameters, including multiple sloppy factors and to challenge data gathering, experts’ status and legitimacy as well as the role and understanding of subjectivity. the methodology design here does not refer to the empirical procedures employed for obtaining or eliciting data, but rather to the conditions that must be satisfied if the data are to be judged evidentially relevant for the acceptance or rejection of an explanation or interpretation. companies are continuously working to improve their day-to-day operational processes. however, most of them prefer not to meddle with their operating model unless they have to. in most cases, companies only decide to change their operating model when facing major events such as a merger, business crisis, regulatory shift or new competitor entrance. event-driven changes are easier to conduct because they are less questioned and can be conducted quickly based on top-down directives. communication can then be delivered on a need-to-know basis and limited to people who are directly affected. unfortunately, beyond the quick wins, too many companies falter in their efforts to redefine their operating models while facing financial pressure thus limiting investment in new capabilities. existing approaches to the development of operating models are based on traditional, linear problem solving techniques. this type of approach can be very effective when uncertainty and complexity are low and when the new design doesn’t need to be radically different than the current one. however, they cannot address the needs of the current highly fluid, uncertain environment, where it-supported services need to radically change their operating models in order to survive. all the traditional levers, such as economies of scale, single platforms and shared services must be cut back, or at least can’t be exploited as before. therefore, high value services are paradoxically always looking for new ways of reducing costs while coping with a complex and disintegrated ijahp article: minzoni, mounoud, zahraei/ using anp to design a living system like balanced operating model for intangible services international journal of the analytic hierarchy process 156 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.267 value chain while at the same time aiming at delivering excellent customer service. however, traditional operating model improvements based upon continuous improvements rather than design are unlikely to be enough to deliver the cost base reductions that are being sought. that’s why alternative cost reduction measures, such as straightthrough processing, first-time resolution and self-service channels are sought. straight through processing (stp) is a good example of the willingness to minimize human input to make staff savings, which usually represent more than half of intangible services’ costs. thus, complex it architectures are needed to accommodate new operating models with greater demands for data. cost reduction is only part of the challenge. if design is limited to process improvements, the new operating model will never fully achieve any change of logic or even if it includes new structures, accountability principles and key metrics (such as kpi). in addition, equal importance should be given to the interfaces and behavioral assumptions embedded in governance rules (that is defining by whom and how decisions are supposed to be made) and management principles (that is defining with what resources and nature of autonomy people are supposed to work with). to create alternative operating models calls for new design criteria to be identified. for example, companies may have strategies but they will need to make them more granular for setting design criteria along which building operating models. the same reasoning applies to governance rules and management principles that have to be translated into design criteria in order to be put to work with the usual suspects, such as cost reduction levers, to build an over-all view of the balance of the operating model. 4. research design/methodology the current study, which is still ongoing, aims to develop a tentative operating model for the banking sector that is for intangible, it high knowledge-based sectors. this model will be inspired by models devoted to the understanding of living and natural systems, that is global but with rough granularity rather than ultra-detailed over-parceled models adapted to over-parceled ways to describe and organize reality. an intensive cross-disciplinary qualitative phase allowed us to identify the relevant criteria to be used for modelling with anp. this phase was conducted by applying methods such as narratology and mapping, both methods are familiar to anthropologists and organizational scientists and recognized as particularly adapted to complex situations. three groups of 3 to 5 managers in three different business units were empowered by the organization to be members of the design team for the so called ‘prototyping’ of alternative operating models. asking these managers to sit in a room as design experts put them in a very different posture compared to being involved as an expert in decision making. making decisions is a normal experience for managers whether they are operational managers in charge of running business units or functional ones involved in organisational choices. decision making is one of the central activities of management. good decision making is regarded as an essential skill to become an effective manager and in order to have a successful career. operation research (or) is a discipline explicitly devoted to aiding decision makers. decision making is supposed to be about identifying and choosing alternatives based on values and preferences. however, managers do not have prior experience generating alternatives or choosing the one that ijahp article: minzoni, mounoud, zahraei/ using anp to design a living system like balanced operating model for intangible services international journal of the analytic hierarchy process 157 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.267 has the highest probability of success or effectiveness and best fits with the company objectives. on the contrary the engineering design process focuses on generating alternatives and involves a number of steps and parts of the process may need to be repeated many times before the process ends. the engineering design process includes research, conceptualization, feasibility assessment, establishing design requirements, preliminary design, detailed design, production planning and tool design, and finally production. design is often referred to as design science, but the distinction between scientific method and design method was introduced as early as the 1960s. since h.simon (1969) popularized the term in his argument for the scientific study of the artificial (as opposed to the natural), design science has come to have the meaning of scientific study of design. furthermore, designing a new operating model requires not only rigor but also creativity and a capacity to step away from the present way of thinking, representing and organising operations. thus, being an expert involved in a group during a design process is much more challenging than being involved in a decision-making exercise. firstly, it involves energy dedicated to the project over a long period that we estimate will be up to a minimum of two years. secondly, it means participating in a working group with peers, a group that possesses expertise over the different areas from governance to the tools. thirdly, as excessive process optimisation has lessened the global vision of the system in which processes take place, it can be lengthy and difficult for the group to encapsulate their detailed knowledge of daily operation and decision processes into a coherent model. in the face of that difficulty, it is necessary for the experts to allow themselves (and feel allowed) to spend the time to overcome that collective pitfall is not straightforward. lastly, once a more global view of the current system is reached, more energy is needed to imagine alternatives with a prospective stance. these two last points can cause experts to feel puzzled during some working sessions, as they are mainly involved with the current model and its related ways of thinking about operations in their daily life with the limited exceptions of weekly working sessions. recent approaches (ashkenas, 2012) have pointed out the need to innovate in terms of operational efficiency representations and measures within the organizations especially for intangible service providers. over-optimization of parcelled up processes has lessened the global vision of the system in which the processes takes place. in fact, these firms have taken on, almost as is, the same ways of thinking familiar to the mechanistic manufacturing industries. until now, academic research has focused on these kinds of industries and the same mechanistic approach will prevail when dealing with services. we can observe, for example, that academic literature (web of science survey, 19952014) about the onshore/offshore issues (with a leap up between 2007 and 2010 and mainly originated in the us) focuses principally on criteria such as infrastructure and salary costs. the main scientific reviews addressing this issue are published in the following domains: computer science information systems, and management or operations research. these publications will rarely be cross-disciplinary, which probably contributes to the spread of the narrow vision echoed in firms. key words in these contributions will be aligning, control, flexibility, provider’s selection, capability loss, security, conflict of interest, transportation, shortest path, maximum flow, pure minimum cost flow, etc. ijahp article: minzoni, mounoud, zahraei/ using anp to design a living system like balanced operating model for intangible services international journal of the analytic hierarchy process 158 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.267 it is important to note that the lean management (womack, 1991), once deprived of the original japanese cultural seeds embedded in the main principles conveyed by this approach, has contributed to the atomization of processes and the complication of their management. indeed, the lean approach arose almost 25 years ago within a social and business context where database marketing was coming forth, where gaps between the executives and employees salaries were growing deep, where the competition was based on it systems, where lbo came out as a financial technique, and where leaders were seen as negotiators. today, the world has changed in its political, financial, managerial and social aspirations, thus the need to renew the operational efficiency fundamentals. even if the operational research scientific community (two examples among many: padillo & diaby, 1999; cardoen et al, 2010) has mainly focused on production planning or supply chains in manufacturing industries and much less on the intangible, high-speed, knowledge-based industries like the banking sector some ideas are insightful. for example: the need to include other important criteria besides the monetary element in the analysis of the operating models alternatives, such as the importance and maturity of technology, the risks at stake, the workforce stability, the complexity level in planning, control and supervise, the quality standards, the complexity of end products, the expected volume fluctuations and uncertain volume estimates. the operating model issue can be approached from an operational point of view or from a technology management perspective depending on what triggers the decision. when the focus of managerial planning is on operations the operating model analysis is has a short time frame, is tactical and performed in the context of an efficient allocation of resources among different activities. conversely, triggers such as new product development yield operating model analysis with a predominance of technologically-oriented issues that require a more strategic approach, thus framing the problem within a high level of managerial planning. the transfer of manufacturing approaches to the service delivery has led to a loss of view of the entire context of service operations. this loss is even more important when transactions are intangible, electronic and often automated. according to a recent study (collinson, 2011) the complexity in the banking sector can be described as an accumulation of procedures in order to improve performance. however, this so called “bad complexity” can reach a tipping point where costs outweigh benefits. collison’s study points out the negative impact of the geographical diversity on banks not to mention directly offshoring activities. it also points out the harm of internal forms of bad complexity which are even more harmful than external forms. an over focus on the issue of time especially when the time for tasks becomes shorter and shorter with nor margins, reduces the scope of the activity, the process and the responsibilities thus congesting the system which is contrary to the original purpose of smoothing the system. it seems clear today that the banking operating models must be carefully designed before they are implemented in order to take into account the multiple parameters influencing operational efficiency or being influenced by it. the external environment, such as regulators, clients or standards, the institution’s corporate governance, the business ijahp article: minzoni, mounoud, zahraei/ using anp to design a living system like balanced operating model for intangible services international journal of the analytic hierarchy process 159 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.267 strategy, the processes, the procedures and the tools are included in these parameters. they are heterogeneous in nature and are not all measurable. this challenges traditional or methodologies which rely upon detailed flow measures, and makes methods like anp very useful since it is a discrete multiple criteria method characterized by the decomposition of a problem into a network structure. the method allows relations of dependence and feedback among criteria and alternatives (saaty, 2005). even though the anp method has been mostly used in decision making contexts characterized by an organizational specific dynamic between experts and decision makers, this article makes the hypothesis that the anp can help invent an innovative understanding of operational efficiency during the design process. on the other hand, anp can also benefit from a particular methodology common in anthropology which mixes the power of maps in visualising the whole system at a glance and the power of narratology, letting experts express their knowledge in a situated and embodied way. in the case of this study, experts are members of the design team and, as such, they are not only data providers for researchers but they contribute to the data processing and the outlining of the future system. the contribution of the anp methodology within the actual cross-disciplinary design process is shown in figure 1. once the criteria have been identified by experts through organizational prototyping meetings methodologically based on narratology and mapping techniques, we proceeded to the first pairwise comparison round in order to assess consistency cluster per cluster. this also allowed experts to review, when and if needed, the initial criteria and their distribution on the map. on the basis of the anp methodology and pareto law, some criteria were identified as outstanding. the limiting values will be considered the “pillars” of each cluster. inner and outer dependency links as well as feedback links will be explored on the basis of these “pillars”. in this step the intention was to reach an elementary consistent model. the project is currently almost at the end of stage 4 (table 1). 5. data/model analysis the new research and academic approaches presented here aim to help banks and knowledge-based services think about their own operational efficiency system differently than the usual mechanical terms such as an automated car assembling factory. this new way is inspired by the methods used for understanding living ecosystems. the living system metaphor is also more consistent with renewed approaches not only for organizational design but also for decision making, governance or collaboration as well encouraging new responses to the need for coordination, control, swarm intelligence and collective wisdom. this methodology was carefully designed to allow from the beginning an end to end vision of the research project while leaving the necessary place for step by step hazards to be taken into account. the research project design focuses attention on the following: the coordination of a cross-disciplinary integrated, conceptual and methodological approach including anthropology, organization sciences and operations research. these three disciplines contribute to the understanding of operations under a living system focus. ijahp article: minzoni, mounoud, zahraei/ using anp to design a living system like balanced operating model for intangible services international journal of the analytic hierarchy process 160 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.267 the fostering of a high level of trust amongst all the members of the design team, operations experts as well as researchers. a first step towards this aim was done by the mental mapping constructed on the basis of experts experience, perceptions and understandings of their system. this was a multi-dimensional, multi-level and multicriteria tool for recognizing the plurality of expert’s perceptions and values as well as for filtering, organizing and storing heterogeneous data. this method also allowed the use of analogies in the cognitive process which prove to be a powerful means to represent the process under a new and large focus and to generate predictions on what should happen if some elements were modified (collins and gentner, 1987). a frequently used metaphor was the one of electric grids so that the system under study could be analysed in terms of electric impulses arousing activity from cluster to cluster thus importing relational structures from one domain to another. the identification of design-oriented useful data. the collection of data has been done on a situated basis, that is data was carefully chosen not to cover all the system’s reality but most importantly, just enough data was provided to be coherent and scalable. two citations inspired the data selection. weinberg (1993) stated, “before we can assign numbers to our observations, we must understand the process by which we obtained them in the first place”. also, wiener (1921), said, “now, things do not in general run around with their measures stamped on them like the capacity of a freight car: it requires a certain amount of investigation to discover what their measure are”. the identification of suitable criteria for the application of the anp methodology: these criteria were identified by experts through the linking of the maps (macro-level information) and the data (micro-level information). the data was envisaged as possible captors of the system’s grid. the elaboration of scenarios issued from anp analysis in order to equip the team with the information and knowledge necessary for the conceptualization of a new system, that is mainly through the establishment of new connections between the grid clusters and criteria. these new connections are aiming at the operating model equilibrium, thus making a distinction between organization and structure. the organization determines the line that cannot be crossed for the system keeping its identity (maturana &varela, 1972). ijahp article: minzoni, mounoud, zahraei/ using anp to design a living system like balanced operating model for intangible services international journal of the analytic hierarchy process 161 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.267 table 1 research process yes 5 . c a u sa lt ie s & p r io r it ie s yes yes 7 . im p le m e n ta ti o n design definition expert co-design model validation pairwise comparison criteria are comparable consistency ratio(c.r) model construction ; narratology, mapping consistency analysis consistency ratio(c.r) criteria priorities in each cluster respondents geomean super matrix limit matrix merit model construction by pillars pareto law eliminate network model construction based on pillars network ; weighting causality final model designed model testing valid balance merit criteria stability criteria priorities in entire model network model construction based on pillars in fcm limit matrix pareto law insufficient criteria fcm network error anp network error yes yes yes 80% 80% 20% yes 1 . m o d e l co n st ru ct io n 2 . j u d g e m e n t 3 . c a lc u la ti o n 4 . d e li m it a ti o n 6 . v a li d a ti o n yes anp network ; pairwise comparison ijahp article: minzoni, mounoud, zahraei/ using anp to design a living system like balanced operating model for intangible services international journal of the analytic hierarchy process 162 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.267 yes 5 . c a u sa lt ie s & p r io r it ie s yes yes 7 . im p le m e n ta ti o n design definition expert co-design model validation pairwise comparison criteria are comparable consistency ratio(c.r) model construction ; narratology, mapping consistency analysis consistency ratio(c.r) criteria priorities in each cluster respondents geomean super matrix limit matrix merit model construction by pillars pareto law eliminate network model construction based on pillars network ; weighting causality final model designed model testing valid balance merit criteria stability criteria priorities in entire model network model construction based on pillars in fcm limit matrix pareto law insufficient criteria fcm network error anp network error yes yes yes 80% 80% 20% yes 1 . m o d e l c o n st r u c ti o n 2 . j u d g e m e n t 3 . c a lc u la ti o n 4 . d e li m it a ti o n 6 . v a li d a ti o n yes anp network ; pairwise comparison ijahp article: minzoni, mounoud, zahraei/ using anp to design a living system like balanced operating model for intangible services international journal of the analytic hierarchy process 163 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.267 the cross-disciplinary research design, methodology and interaction modalities are familiar in the academic literature on simplexity (berthoz, 2009) which proposes not to master nor reduce complexity but to cope with it. simplexity as a concept can be considered an heir of newton’s statement about nature, in turn closely related to the principle of parsimony put forward by father william of ockham in the 14th century. ockham is the father of efficient reasoning, and of the idea called ockam’s razor that one should not multiply entities beyond necessity has been attributed to him. father ockham’s thought was far from thinking that seven centuries later the idea of adding more and more entities, products, information systems, rules, guidelines, etc. would become the core of business and that managers and leaders would be disarmed by not knowing how to define when a given situation is beyond necessity or not yet. in the complicated environment that has developed over at least fifty years, operational efficiency seems to be supported by multiple level procedures and rules, more and more regulatory mechanisms and long top down delegation cascades. this has lessened the innovative capacity, the long term view and quality of professional relationships among operators but also among the hierarchy. coping with complex environments and organisations is essentially a matter of relations. this matter of relations refers to the ways people put themselves together to act, how these ways are designed by the internal explicit and tacit rules and guidelines, which attitudes are expected, and what can be said to whom and when. it is amazing to observe how much sapience is embedded in operators, how much experience is accumulated in experts’ minds and how little of this is put to work in decision making processes and organizational design initiatives. to understand complexity we can learn from disciplines like physics or biology when they define, for example, the operational efficiency of a membrane, the elasticity of a movement or the intercellular transport of micro fluids. some attempts to understand complexity in other sectors have explored, for example, the models which may describe financial complexity as if it was similar to the movement of molecules in a gas or the dynamics of stars in a galaxy. banks now recognise that there is a real practical use, for instance, in pricing financial deals, in modelling the markets as 'almost random' processes and then calculating probabilities of future market movements. operational efficiency can also take advantage of this kind of research, especially to imagine new, simple (simplex) ways of working. we can hypothesize that mapping techniques allowing the visualization of the information as well as its context and dynamics can be put to work to the benefit of simplicity. the way the experts identified criteria, their reaction towards pair-wise comparisons, consistencies and inconsistencies as well as how aggregations took place are presented here. to start with, we found it necessary to ask the experts in the three different groups to describe their organisational system “from the governance to the tools’ in their own words without referring only to the official documents such as procedures, process charts, flow charts or strategy presentations. all these documents have no link to one another. they describe different realities inside the company and are not supposed to communicate. first, they were asked to tell the whole story and draw the global map of their world by putting together information and knowledge about what we defined as ijahp article: minzoni, mounoud, zahraei/ using anp to design a living system like balanced operating model for intangible services international journal of the analytic hierarchy process 164 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.267 being the journey ‘from the governance to the tools’ across ‘the 5 continents’ or clusters (governance, strategy, operations, procedures and tools). we asked them to identify key components of these five clusters. we called these first maps structural maps as they were mostly composed of units, departments, committees, rules, etc., and said more about what things are than how things actually get done. this first step made it clear (for them and for us) that there was no whole spectrum of data that was easily available that we could use to build upon, but also that we would have to use some measurable and many sloppy factors to describe the system. the anp appeared to be the most adapted method because of its ability to associate heterogeneous factors and its friendliness to rough data. we then had to introduce the method to the experts, especially the idea that we would deal with their subjectivity rather than try to avoid it. we also introduced the idea that not only the assessment of the current operating model but also the design of alternative ones would be based on their judgments through the pair-wise comparison of the relative weight of the criteria. prior to using anp to assess the balance of the operating models we had to redesign the map from a structural map of clusters to a functional map of clusters and criteria. this change of focus in the way of talking about and representing their system was a turning point in the knowledge the experts had about their system. the functional map was not so much about what doesn’t work since improving the model was not our goal. it built on the knowledge of how things work to describe what is produced (work, knowledge, learning, errors, opportunities, behaviors, values) throughout the daily life of the organization. thanks to the lay out provided by the map representing the clusters of criteria, and despite two important difficulties (the prospective nature of the goal, the relatively abstract or polysemic nature of the criteria), experts were able to engage in the pair-wise comparisons. when inconsistencies appeared and differences in experts’ judgments were obtained, they were used as opportunities to elicit the reasoning behind weighting and to discuss and clarify the meaning and relevance of the criteria. even more interesting were the discussions about the aggregate weighting, and the goal of getting away from a limited view on the performance of operating models defined in terms of costs reduction and moving towards the definition of a balanced operating model. this process took over a year to complete with regular four-hour meetings every 3 weeks. that amount of time was needed to go through the different phases and the changes implied in terms of tasks and relationships. drawing the first structural map (about “what” are the structural elements of the system) across the so-called ‘5 continents’ was unusual but less challenging than when the experts were asked to talk about “how” the system functions. here is where subjectivity arose and different postures emerged, from those pretending that the system functions the way it is supposed to, to more critical voices identifying the difficulties, drawbacks and dysfunctional consequences of past decisions. it was important not to spend time criticizing the current operating model for two reasons. firstly, it could make some group members engaged in the current organizing process and projects feeling uncomfortable, and secondly, it could limit the discussion to complaining and thus prevent a prospective stance to emerge. the introduction of the anp and the practice of pair-wise comparisons provided the occasion to go one step further and recognize the legitimacy of subjectivity in the design process. ijahp article: minzoni, mounoud, zahraei/ using anp to design a living system like balanced operating model for intangible services international journal of the analytic hierarchy process 165 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.267 a high level of trust is a necessary condition when asking people to put subjectivity to work and engage in different tasks and relationships. castelfranchi and falcone (2000) stress the importance of trust, particularly beyond a mere quantitative, probabilistic and opaque view of trust as supported by the economics and game theory. the authors speak in favor of a cognitive view of trust as a complex structure of beliefs and goals encompassing rational and irrational components. the question of trust seems central to the co-designers teams in ways such as trust in the team’s judgment as a whole beyond personal expert judgments, trust in the collective eliciting of the gaps between the prescribed and the real which inevitably puts actors at stake, trust in individual and team capacity to imagine a totally new operating model issued from and not just improving or optimizing actual dysfunctions. we thus make the proposition that subjectivity is an element of trust within design approaches and the anp has proven a useful tool to situate subjectivity and create value from it. different disciplines today show a marked revival of interest in and legitimation of the deliverances of subjective reflection, subjects being at once products and agents of history, agents of knowing as much as of action. subjective techniques and methods of enquiry are thus critical in the study of complex systems functioning when faced with a new operating model’s design in a complex intangible services system, it shall be kept in mind that there is no precise method to measure criteria such as power, influence, political effectiveness, and cultural gaps (saaty, 2008). the issue of subjective judgments shall be carefully addressed. the ahp & anp methods use a scale from 1 to 9 for relative weighting (saaty, 2008). in this scale, nine shows the significant difference between the two objects and shows the dominance of one object over another, just as numbers represent equal importance between two compared objects. it is important to note that the anp matrices and super matrices are reciprocal and always positive. a capital merit in this study is that anp outputs give consistent data. the cross-disciplinary method of the current study, which integrates anthropology, organization sciences and operations research focuses on swarm intelligence and collective wisdom. in this context, researchers can explain in more detail the concept of pair-wise comparison to the respondent by visualization of the model’s map and fundamental scale of the ahp and anp approach. we believe this approach provides a bi-lateral benefit for the researchers and the organization’s experts as respondents and codesigners of the new model learn about the system and focus on what is happening and might happen in the future. as presented in figure 1, testing the consistency ratio (c.r) of the matrix is a key step. owing to unavoidable inconsistency between judgments and the need for the matching of priorities by the invariance, it is essential to obtain the priorities in the form of eigenvectors of the matrix of the paired comparisons. the consistency ratio is calculated and examined by the researchers and experts to guide their design process as the objects are being compared. figure 1 presents the geomean calculated for a part of the model. ijahp article: minzoni, mounoud, zahraei/ using anp to design a living system like balanced operating model for intangible services international journal of the analytic hierarchy process 166 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.267 figure 1. a portion of the model's geomean to analyze the priorities of the criteria and the c.r of the respondents the current study used the latest version 2.2.6 of the super decision software. to quantify the collective wisdom of the experts, the geometric mean has been computed by microsoft office excel. the results of the geometric means are used to create a “master” super decision model. the c.r for the geometric mean was computed and followed the rule, c.r must be less than 0.1. figure 2 presents a portion of the results of the merit and cluster matrix. the same process is undertaken for the whole model. if we take the “environment” cluster example, the two main pillars would be regulation and clients. figure 2. priorities of the criteria cluster criteria limit normalized 1 environment 1.1 market infrastructure 0,018219 0,1856 1.2 clients 0,031599 0,3219 1.3 business provider 0,007968 0,08117 1.4 regulations 0,033509 0,34135 1.5 competitors 0,00687 0,06998 business strategy c.r : 0.0003 investment cost reduction business ambitions product develop ment client develop ment lobby staff reduction homogeneity tom cross selling group governance c.r : 0.0157 strategy macro risk management of risks compliance funding goi environment c.r : 0.02535 market infrastructure client s business provider regulation competitor ijahp article: minzoni, mounoud, zahraei/ using anp to design a living system like balanced operating model for intangible services international journal of the analytic hierarchy process 167 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.267 6. limitations at this stage of the project, the main limitations can be stated as follows: the heterogeneous kind of data gathered within the system under study is composed of data from clusters as different as governance, strategy and operations the lack of literature about the concept of balanced systems applied to organizations looked at under a living system/ecosystem metaphor. as usual with metaphors the main limitation is the over parametrization of the model (with many free parameters one can fit any desired behaviour). in the present approach we are not fitting a linear model behaviour but rather estimating the parameters of an “independent experiment” and then comparing qualitative features of different possible behaviours as possible functions of the determined parameters. the lack of an appropriate benchmark as a stable model designed to compare the existing one. classical “decision making” oriented studies will use known ratios, for example, carbon/nitrogen in the soil to estimate deviations and assess judgements. in the case of the study presented here, the pertinent ratios have to be discovered. the relatively abstract or polysemic criteria used in defining the “how” an organization works and the prospective nature of the goal we feel these limitations are a result of using a recognized decision method within a different context, that is, the design of a new system. this question challenges key issues like data gathering, expert’s participation or consistency and sharpens creativity. 7. conclusion combining the use of anp with methods and approaches from organization and anthropological sciences has confirmed our conviction of the anp methods crossdisciplinary friendliness. the anp approach has helped experts express their views of the system in a multi-level way, adapted to the broad operating model study’s spectrum “from the governance to the tools”. it has also allowed them to engage in an in-depth discussion about the role played by subjectivity and intuition in design processes. the anp provides a substitute to experiments in the determinaion of the model, and will again assess relevance structure of the fcm to come. given the trivalent dynamic characteristics of fcm matrices, these will provide an approximation to the posible equilibria and how they control the dynamics of the system. the next step will be to provide a proof of concept of the possibility of modelling an organization’s complex behaviour using the living system metaphor with differential equations. we believe this issue is at the centre of the modelling of any socialorganizational process. finally, we note that only a number of studies in different situations which prove predictive will validate the present approach. ijahp article: minzoni, mounoud, zahraei/ using anp to design a living system like balanced operating model for intangible services international journal of the analytic hierarchy process 168 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.267 references aguilar, j (2005). a survey about fuzzy cognitive maps papers. international journal of computational cognition, 3(2), 2733. ashkenas, r. (2012). it's time to rethink continuous improvement, http://blogs.hbr.org/2012/05/its-time-to-rethink-continuous/ axelrod, r. (1976). structure of decision. the cognitive maps of political elites. princeton, nj: princeton university press. bellomo, n. ed (2008). modeling complex living systems: a kinetic theory and stochastic game approach. modeling and simulation in science, engineering and technology. boston, basel, berlin: birkhäuser publications. berthoz, a (2009). la simplexité. odile jacob. burns, t. & stalker, g. m. (1961). the management of innovation. london: tavistock. cardoen b. et al (2010). operating room planning and scheduling: a literature review, european journal of operational research, 201(3), 921-932. castelfranchi, c & falcone, r (2000).trust is much more than subjective probability: mental components and sources of trust, proceedings of the 33rd hawaii international conference on system sciences, 1-10. collins, a., and d. gentner. (1987). how people construct mental models. in d. holland and n. quinn (eds), cultural models in language and thought (243-268). cambridge: cambridge university press. collinson s. (2012). true cost of complexity in the banking sector. simplicity partnership. da silva, a.s et al. (2009). anp and ratings model applied to supplier selection problem, proceedings on the international symposium on the ahp, 1-14. eaton t. (2011). a cultural analysis of information technology offshore outsourcing : an exercise in multi sited ethnography of virtual work, wayne state university dissertations, paper 370. jones, n et al. (2011). mental models: an interdisciplinary synthesis of theory and methods, ecology and society, 16(1), 46. korff, r. (2003). local enclosures of globalization. the power of locality. dialectical anthropology, 27(1), 1-18. kosko, b (1986). fuzzy cognitive maps, international journal of man-machine studies, 24, 65-75. http://blogs.hbr.org/2012/05/its-time-to-rethink-continuous/ ijahp article: minzoni, mounoud, zahraei/ using anp to design a living system like balanced operating model for intangible services international journal of the analytic hierarchy process 169 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.267 martinez-moyano, i et al. (2014). drift and adjustment in organizational rule compliance: explaining the regulatory pendulum in financial markets, organization science, 25(2), 321. maturana, h. & f. varela. (1980). autopoiesis and cognition: the realization of the living. boston studies in the philosophy of sciences, 42. boston, ma, dordrecht, holland: d. reidel publications. murray, jd (2008). mathematical biology. berlin heidelberg: springer verlag. o'neil, p. d. and krane, d. (2012). policy and organizational change in the federal aviation administration: the ontogenesis of a high-reliability organization. public administration review, 72, 98–111. padillo j. m. & m diaby (1999). a multiple-criteria decision methodology for the makeor-buy problem. international journal of production research, 37 (14), 3203-3229. santiago medina hurtado. (2010). modeling of operative risk using fuzzy expert systems in michael glykas (ed.), fuzzy cognitive maps advances in theory, methodologies, tools and applications (135-159). berlin heidelberg: springer. saaty, t.l., (1999). fundamentals of the analytic network process. isahp 1999, kobe, japan. saaty, t.l. (2005). theory and applications of the analytic network process: decision making with benefits, opportunities, costs, and risks. pittsburgh, pa: rws publications. saaty, t. l. (2008). the analytic hierarchy and analytic network measurement processes: applications to decisions under risk, european journal of pure and applied mathematics, 1(1), 122-196. simon, h.a. (1969). the sciences of the artificial. cambridge, ma: m.i.t. press. staats, b. r., & david m. upton. (2007). lean principles, learning, and software production: evidence from indian software services, harvard business school working paper. no. 08-001. stylos, c & groumpos, p. (1999). mathematical formulation of fuzzy cognitive maps. proceedings of the 7th mediterranean conference on control and automation (med99) haifa, israel, 22512261. thomas wi & f znaniecki (1918-1920) the polish peasant in europe and america. monograph of an immigrant group. chicago: university of chicago press. web of science survey (1995-1997), unpublished survey data weick ke (1987). organizational culture as a source of high reliability. california management review, 29, 112-127. ijahp article: minzoni, mounoud, zahraei/ using anp to design a living system like balanced operating model for intangible services international journal of the analytic hierarchy process 170 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.267 weick ke (2011). organizing for transient reliability: the production of dynamic nonevents. journal of contingencies and crisis management, 19, 21-27. weick ke, sutcliffe km, obstfeld d (1999). organizing for high reliability: processes of collective mindfulness. in sutton ri, straw bm (eds). research in organizational behavior, vol. 21. (81-124), greenwich, ct: jai press. weinberg, g. (1993). quality software management. new york, n y: dorset house. wiener, n. (1921) a new theory of measurement: a study in the logic of mathematics, proceedings of the london mathematical society, 2(19), 181-205. womack, j et al (1991) the machine that changed the world. new york, harper collins. ijahp article: forbes, hebb, mu/ethical decision making in action: evaluating hospital care attendance approaches international journal of the analytic hierarchy process 313 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.592 ethical decision making in action: evaluating hospital care attendance approaches julie forbes jeforbes@live.carlow.edu abigail hebb amhebb@live.carlow.edu enrique mu emu@carlow.edu abstract patient safety is a priority in the hospital. hospitals are always considering cost effective ways to keep patients safe and free from harm. every year an average of 340,000 hospitalized patients are injured due to falls. providing the best possible care attendance to prevent these incidents is very important. it is demonstrated here that, beyond medical and financial considerations, the proper selection of care attendance is an ethical decision. this decision requires considering the needs of, as well as getting input from, all the parties involved (hospitals, nurses, and patients). unfortunately, until now, the care attendance discussion has mainly considered the hospital’s perspective and rarely that of the patient. using a stakeholder theoretical approach taken from ethical decision making literature and the analytic hierarchy process which allows the integration of multiple stakeholder perspectives and the inclusion of intangible variables (such as patient’s perceived value), we developed an evaluation framework to enable the prioritization and allocation of resources to the different care attendance approaches: care attendant (ca), continuous video monitoring (cvm), normal rounding (nr) and family visitor sitters (fvs). the decision criteria have been identified from the extant medical evidence-based literature, and expert opinions from three decision-makers (each representing a particular stakeholder’s perspective) were used to assess the criteria weights and rate the alternatives. keywords: ethical care; patient care; care attendance; continuous video monitoring 1. introduction patient safety is the prevention of adverse events and errors in healthcare to patients (agency for healthcare research and quality, 2018). nurses play an essential role in patient safety. patients who are at risk for adverse outcomes including falls, falls with injury, harm to self and others may need increased supervision. an interdisciplinary team including nurses determines the need for increased supervision at the bedside. bed alarms, chair alarms, low beds, and fall mats are standard interventions to prevent falls and keep patients safe while hospitalized (cournan, fusco-gessick, & wright, 2018). however, there are circumstances when patients require increased measures such as hourly rounding, care attendants, continuous video monitoring, and family visitor sitters. determining the best increased measure to provide for each patient in care is circumstantial. the agency for healthcare research and quality (2018) reports between ijahp article: forbes, hebb, mu/ethical decision making in action: evaluating hospital care attendance approaches international journal of the analytic hierarchy process 314 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.592 700,000 and 1,000,000 people fall in the hospital each year. the number of falls that occur each year is equivalent to the population of dallas, texas (u.s. population city and town population, n.d.). more than one-third of in-hospital falls result in injury, including serious injuries such as fractures, head trauma, and even death. reports estimate 30 50% of falls result in injury, costing on average $14,000 per fall (the joint commission, 2015). these costs are covered between hospitals, insurance coverage, and patients resulting in serious financial hazard particularly for patients. the center for disease control (cdc) reports “medical costs of fall injuries for u.s. patients ages 65 or older are $34 billion annually, hospital costs account for only two-thirds of the total cost of fall injuries” (cdc, 2016). in summary, fall injuries involve important costs for both hospitals and patients. while there is no question of the importance of addressing the best way to provide care to patients, it is proposed here that the evaluation of a care attendance approach should be addressed as an ethical decision and a suitable moral decision-making framework should be used for this purpose. in this study, we will first argue that care attendance is an ethical healthcare issue and second, we will use an ethical decision-making approach, rooted in stakeholder theory and using the analytic hierarchy process (ahp) as the methodology for care attendance evaluation (freeman 1984). 2. literature review the literature review comprises two domains. the first domain is the literature discussion that supports the proposal of evaluation as an ethical decision. the second seeks to identify current care attendance approaches, their characteristics, and the elaboration of criteria to evaluate them from different perspectives, more specifically hospital, nurse, and patient’s perspectives. the complete literature review is shown in appendix a. 2.1. care attendance as an ethical decision frameworks for ethical decision making are helpful in examining a clinical situation or action to determine if the situation involves ethical issues. curtin’s 6-step model of ethical decision making recommends the following steps: 1) perception of the problem, 2) identification of ethical components, 3) clarification of persons involved, 4) exploration of options, 5) application of ethical theory, and 6) resolution/evaluation (curtin, 1979; stuart & sundeen, 1987). 2.1.1 perception of the problem this step is aimed at identifying if an ethical dilemma exists, and if so the context of the dilemma. our review of care attendance approaches literature shows the presence of a moral conflict. hospital management would prefer a solution that reduces cost of the care attendance approach; nurses are more highly concerned with their patient’s safety and patients assess care attendance preference based on personal values and perceptions (e.g., a patient may find more value in having a family member providing care than a trained care attendant) (jeffers et al., 2013; neville, dibona, & mahler, 2016; tzeng & yin, 2007). ijahp article: forbes, hebb, mu/ethical decision making in action: evaluating hospital care attendance approaches international journal of the analytic hierarchy process 315 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.592 2.1.2 identification of the ethical components the second step in the ethical analysis is to identify ethical components by answering questions like, what is the underlying issue/problem? and who is affected by this dilemma? these three potentially conflicting perspectives of the hospital, nurse, and patient constitute the key idea for our discussion of care attendance as an ethical decision. our review of the literature shows the majority of the care attendance approaches discussion has been done from the hospital’s perspective, even less from the nurse’s perspective and rarely if ever from the patient’s perspective (babine et al., 2018; torkelson & dobal, 1999; laws & crawford, 2013; solimine et al., 2018; neville, dibona, & mahler, 2016; tzeng & yin, 2007). 2.1.3 clarification of the people involved in the third step of the process of ethical analysis, the relevant questions to be asked at this stage are: what are the rights of people/person involved?, who should be included in decision making?, and for whom is the decision being made? the three parties previously identified (hospitals, nurses, and patients) all have clearly defined rights to participate in care decisions. hospitals are responsible for providing quality healthcare while maintaining the financial viability of their services. nurses are professionally committed to the well-being of the patients under their care. finally, patients will be directly affected by the outcome of the care attendance selection. therefore, it is felt that all parties should participate in care attendance approaches evaluation decisions. furthermore, care attendance evaluations should be made to address the needs of all three parties and not only those of the patients. 2.1.4 exploration of the options the fourth step in the ethical analysis is the exploration of the options, and at this step the relevant questions to be asked include: what alternatives may exist? and what is the purpose and potential consequences of each alternative. based on the extant evidence-based medical literature, the following care attendance approach options have been identified: care attendant (ca), continuous video monitoring (cvm), normal rounding (nr) and family/visitor/friend (fvs). this study will examine the purpose and potential consequences of each of these alternatives and will provide an evaluation framework for this purpose. 2.1.5 application of ethical theories application of ethical theories is the fifth step of the ethical analysis. the application of ethical theories in situational analysis strengthens the final decision. a relevant question at this stage is, which ethical or theoretical framework should we use? our previous discussion, in particular the necessity of addressing the needs of the different parties (hospitals, nurses, and patients), suggests the application of stakeholder theory which has become relevant for social responsibility and ethical management in general (harrison & freeman, 1999; freeman et al., 2010). a stakeholder in an organization denotes “any group or individual who can affect the achievement or is ijahp article: forbes, hebb, mu/ethical decision making in action: evaluating hospital care attendance approaches international journal of the analytic hierarchy process 316 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.592 affected by the achievement of the organization’s objectives” (thompson 1967; freeman 1984). this theory implies that it is a fundamental ethical principle that those who will be affected by a decision will be informed, and will preferably participate in the ultimate decision. 2.1.6 resolution into action the sixth and final step is the requirement to decide on a resolution or resolution into action. relevant questions to be addressed here are the following: what is the goal of one’s decision?, how can we ensure the decision is the best for all concerned?, how can the resulting choice be implemented? and how can the resulting ethical choice be evaluated? the development of this stage is the next step in our paper. our decision goal is to evaluate the existing care attendance approaches concerning criteria developed from the extant literature and expert opinion, including criteria from the key stakeholders to ensure it is the best for hospitals, nurses, and patients. suggestions about how to implement the choice(s) and its subsequent evaluation will also be included. 2.2. care attendance approaches in the evidence-based medical literature based on medical evidence-based extant literature (appendix a) and expert opinions of three decision makers (co-authors), a nursing administrator, a registered nurse in active service, and a layperson with extensive patient experience, we have identified criteria and alternatives for our care attendance approaches evaluation model. given the nature of the proposed evaluation, a benefit/cost approach was used for the analysis. the benefit criteria include safety and customer value including the sub-criteria patient and hospital perceived value respectively as described in appendix c. the cost criteria includes fixed costs including two sub-criteria, acquisition and setting up costs, and variable costs (operational) as shown in appendix d. alternatives include (nurse-dedicated) care attendants, continuous video monitoring, normal rounding, and family visitor sitters as described in appendix b. 2.2.1 care attendance approaches considerations patient safety is essential in the acute care setting and healthcare professionals are continually looking for ways to improve patient safety and reduce falls (votruba, graham, wisinski, & syed, 2016). the center for medicare and medicaid implemented the inpatient prospective payment system that went into effect on october 1 st, 2008 resulting in lower reimbursement for hospital-acquired conditions from the center for medicare and medicaid. many other third-party payers have begun to follow these guidelines regarding reimbursement. one major healthcare acquired condition is a patient fall that could result in fractures, joint dislocation, head injury, and/or crushing injury. falls can contribute to an increased length of stay, increase in the cost of patient care, patient fear of falling, and emotional distress to the patient (burtson & vento, 2015). patients are presenting to the acute care setting more and more with delirium, confusion, and other high-risk behaviors that can lead to harm or falls. hospitals are continually trying to improve the safety of patients. nursing units are requesting staff that are ijahp article: forbes, hebb, mu/ethical decision making in action: evaluating hospital care attendance approaches international journal of the analytic hierarchy process 317 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.592 beyond budget due to sitter needs. the use of sitters is an uncontrolled and unanticipated dilemma for many hospital managers (laws & crawford, 2013). implementation of programs, such as sitters, can be expensive for hospital systems. the use of sitters can be a challenge for staffing resources. the average cost for a sitter is $240/sitter/day. hospitals in the united states have reported spending between $500,000 $2,000,000 each year on sitter costs. hospitals are continually looking for ways to improve safety and decrease these costs in the acute care setting (davis, kutash, & whyte iv, 2017). patient and staff satisfaction are significant factors in healthcare today. patient satisfaction includes pain management, response to requests for help (call bell) and the attentiveness of the staff to meet needs during a hospital stay. staff satisfaction can create a positive work environment, increase morale, and improve staff engagement (flowers et al., 2016). patient satisfaction has been associated with adequate staffing of inpatient units and positive environments (danaf et al., 2017). the hospital consumer assessment of healthcare providers and system (hcahps) is a national standardized survey consisting of 32 questions about a hospital stay from the patient’s perspective that are reported publically. the hcahps survey allows for data to be produced about the perception of patients that allow for objective and meaningful comparisons of hospitals. the survey also provides incentives for hospitals to improve the quality of care provided to patients and enhance accountability by increasing transparency (centers for medicaid and medicare services, 2017). the national quality nurse quality indicators (ndnqi) started in 2000 and 95% of magnet hospitals participate annually. the nqndi surveys measure nursing quality, improve nurse engagement, strengthen the nurse’s work environment, assess nurse staffing levels, and improve reimbursement under the current pay for performance policies. about 300,000 nurses provide insight annually and help create improvements for those who are directly responsible for the patient experience. the results enable hospitals to develop more effective, finely targeted improvements, understand the relationship between nursing-sensitive indicators, staffing, and registered nurse survey data (national database for nursing quality indicators, 2018). a summary of our major findings in the medical evidenced-based extant literature is shown in appendix a. 2.2.2 care attendance approaches care attendants (ca) are staff fully dedicated to the patient and seem to be an ideal solution although they may have limited capability regarding fall prevention. improved care attendant training and constant work practices should be considered by hospitals (jong, kitchen, & hill, 2017). practices should be nurse-led for initiation and discontinuation of patient sitters for management of safety risk behaviors associated with delirium, dementia, and fall risk patients (colella et al., 2017). continuous video monitoring (cmv) can help in the reduction of falls in inpatient rehabilitation settings, especially patients who may be cognitively impaired (cournan, fusco-gessick, & wright, 2016). one study reported a cost savings of $2.02 million in certified nursing assistants (jeffers et al., 2013). cvm can be utilized as an additional ijahp article: forbes, hebb, mu/ethical decision making in action: evaluating hospital care attendance approaches international journal of the analytic hierarchy process 318 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.592 component of a fall prevention program (sandjecklin, johnson, & tylka, 2016). cmv can be used as a safe alternative for patient companions and does not show an increase in the risk of falling for patients (votruba, graham, winsinski, & syed, 2016). nursing culture can shift and trust in the effectiveness of new technology such as video monitoring to address patient safety can increase (burtson & vento, 2015). normal rounding (nr) is an essential aspect of patient safety and quality of care. protocols for hourly rounding should be based on individual patient needs and staff preferences (fabry, 2015). throughout the implementation of normal hourly rounding education, feedback, use of staff champions and the presence of leaders are needed for success (kessler, claude-gutekunst, donchez, dries, & snyder, 2012). the use of hourly rounding protocols can reduce call light usage, misuse of call lights and improve overall patient satisfaction by increasing the time spent at the bedside. it is essential to keep staff informed and have continued support from leaders for successful implementation of hourly rounding (dearmon et al., 2013). normal rounding of patients can improve communication, teamwork and coordination with patients, family members and inter-professional colleagues (kessler et al., 2012). normal rounding can increase staff and patient satisfaction through increased visibility of the nurses by the patients (flowers et al., 2016). rounding on patients allows the nurse to be present, address the immediate concerns of the patient, and be proactive in their care. effective use of rounding on patients can impact how patients perceive nursing care, improving overall hcahps for patient satisfaction (neville, lake, lemunyon, paul, & whitmore, 2012); (danaf et al., 2017). families and friend visitors can act as sitters (fvs) since they like to be actively involved in fall prevention of their loved ones. patient family members may assist when needed. it is essential to remember that families are informal caregivers and most of the time not professionally trained individuals. nurses need to assess and evaluate family members understanding and ability to help with fall prevention (schoberer, breimaier, mandl, halfens, & lohrmann, 2016). collaborative relationships built with staff, patients and companions (family members, private aides, and sitters) about a patient’s risk for falling are essential in preventing fall-related injuries (tzeng & yin, 2007). however, inappropriate delegation to family members for fall prevention can increase the risk of injury to family members. healthcare team members should not expect families to provide professional care to their loved ones and frequent assessment and reassessment are needed (tzeng & yin, 2009). a summary of the care attendance approaches found in the medical extant literature is shown in appendix b. 3. objective the purpose of the proposed current study is to develop an ethical-oriented care attendance approach evaluation framework by taking into consideration the conflicting needs and perspectives of hospitals, nurses and patients. 4. research design/methodology according to curtin’s 6-step model and philosophy, a nurse has a unique relationship with a patient which enables him/her to provide close observation and allows them to ijahp article: forbes, hebb, mu/ethical decision making in action: evaluating hospital care attendance approaches international journal of the analytic hierarchy process 319 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.592 identify ethical dilemmas and clinical crises during which the management of these issues requires clinical expertise and moral judgement. the constant change in patient acuity, limited resources, and advanced technology creates ethical dilemmas along the way (wood, 2001). not only is it essential to gather adequate relevant data in decision making, it is also important to understand the emotional impact of situations on humans that can offer valuable insight into personal responsibility and accountability (curtin, 1978). to address these conflicting perspectives, a stakeholder analysis is proposed. the present study provides an evaluation, using the analytic hierarchy process (ahp) methodology, and curtin’s 6-step model of ethical decision making of four different care attendant approaches identified from the current literature: care attendant (ca), continuous video monitoring (cvm), normal rounding (nr) and family/visitor/friend (fvs) (saaty, 2001; curtin, 1978). these approaches constitute the alternatives in our ahp model. the evaluation criteria have been obtained from a review of evidence-based medical literature (shown in appendix a) as discussed in the literature review section. their importance is assessed by three decision-makers (study co-authors) which include a nursing hospital administrator, a registered nurse in active patient care service and a layperson with extensive patient experience due to family circumstances. each of the experts represents the hospital, nursing and patient’s point of view respectively when addressing the ethical implications. the pairwise comparisons for the criteria weights were done via group discussion while the ratings of the alternatives were distributed among the stakeholder expert representatives. for example, whenever the comparison question involved a hospital-related criterion (e.g. costs) the team member who is a hospital administrator would perform the assessment; when the comparison question involved a nurse-related criterion (e.g. safety) the active licensed train nurse would intervene; and finally, for the patient-related perspective (e.g. patient’s perceived value), the patient stakeholder expert representative would issue the judgment. the team met on different occasions and the intensity judgments were agreed upon through discussion of the different views and negotiating consensus to address this as well as any potential inconsistency. a three-person team makes it much easier than one with a larger number to reach consensus and negotiate inconsistent judgments when they arise. following standard practice, the consistency ratio, called the inconsistency index in super decisions (2018) software, was always kept at less than or equal to .1 for the agreed comparison. the analytic hierarchy process (ahp) developed by saaty (2001) is one of the most widely used multi-criteria decision-making frameworks. one of the reasons for its popularity is that it is easy to understand and use by decision-makers. most often, in selection and evaluation applications, the ahp hierarchy frames a decision as a hierarchical model where the top level captures the goal of the model, the intermediate levels contain the criteria and sub-criteria, and the bottom level consists of the alternatives. in the ratings model approach, the decision-maker can derive a local weight for each of the criteria by pairwise comparison of their relative importance, using saaty’s (2001) intensity scale with ranges from 1 (equally important) to 9 (extremely more important) with respect to the decision goal. these judgments are tabulated in a pairwise comparison matrix (pcm) and the relative weights are calculated by raising the pcm to powers until the limit matrix is obtained. once the criteria weights are obtained, the alternatives can be evaluated against each of the criterion using a ratings scale (e.g. from “poor” to “excellent”) and the final rating for each alternative is obtained as a weighted ijahp article: forbes, hebb, mu/ethical decision making in action: evaluating hospital care attendance approaches international journal of the analytic hierarchy process 320 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.592 sum of each criterion rating times its relative weight. while a thorough discussion of ahp is beyond the scope of the present study, the reader is referred to the extensive literature on the topic such as saaty (2001) and mu and pereyra-rojas (2017). an ahp benefit/cost (b/c) approach and super decisions (2018) software for the ahp stakeholder was used following the procedural recommendations of mu and pereyrarojas (2018) for this type of analysis. one important caveat is that rather than using a traditional financial b/c approach, we took advantage of the ahp methodology to model intangible criteria such as “perceived value” as well as the inclusion of benefit criteria corresponding to the different stakeholders such as hospital’s and patient’s perceived value and nurse’s concern (safety) as seen in figure 1. a benefit is a gain that can be either financial or intangible. a specific benefit hierarchical model was developed for our study as shown in figure 1. the cost criteria are more attuned with objective financial considerations as seen in figure 2. figure 1 benefits model figure 2 cost model ijahp article: forbes, hebb, mu/ethical decision making in action: evaluating hospital care attendance approaches international journal of the analytic hierarchy process 321 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.592 5. benefit and cost analysis criteria/sub-criteria and alternatives the summary of alternatives for this study is shown in appendix b. the benefit model criteria/sub-criteria are shown in appendix c. a subjective rating scale was developed for each criterion/sub-criterion and is shown below in tables 1-3. the alternatives were rated accordingly by the decision-making team as shown in table 4 and the priorities based on this are shown in table 5. table 1 benefit model ratings scale explanation of customer value-patient perceived rating scale benefit model ratings scale explanation of customer valuepatient perceived rating scale excellent if patient considers the care attendant approach is the best they have ever received. extremely high praise for care provided, and met every need of the patient. very good very professionally attended, to although no moderate degree of empathy. good professionally well done although no empathy developed between patient and care attendant approach. regular patient feels that approach covered the very minimum of what was expected by patient expectations. deficient the patient considers that the care attendant approach did not meet his/ her care needs and expectations. table 2 benefit model ratings scale explanation of customer value-hospital perceived rating scale benefit model ratings scale explanation of customer valuehospital perceived rating scale excellent patient is very satisfied with care received, gives praise openly and recommends hospital care to all friends and relatives. very good patient is very satisfied with care and considers it to be above average expectations. will return to have care and recommend to close friends and family. good the patient is satisfied with care received and considers that needs were met, but could have been provided better overall care. will return as customer, but will not actively recommend to other customers. regular patient will consider that needs were barely met. while no complaints will be made, he/ she may not return and will not recommend care attendant approach to other customers. deficient needs were not met. care was poor and may issue formal voice complaint or grievance. unlikely to return if have the opportunity will express negative opinion to family and friends. ijahp article: forbes, hebb, mu/ethical decision making in action: evaluating hospital care attendance approaches international journal of the analytic hierarchy process 322 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.592 table 3 benefit model ratings scale explanation of safety rating scale benefit model ratings scale explanation of safety rating scale excellent risk of harm and unsafe care to patient based on care approach is very minimal. there is a high degree of a trusting relationship between patient and care providers. very good risk of harm and unsafe care to patient based on the care attendant approach is minor. trusting relationships were developed between patient and care providers, but could have been enhanced. good risk of harm and unsafe care on care attendant approach is moderately possible. patient may experience unsafe care resulting in no injury. regular risk of harm and unsafe care on care attendant approach is moderately possible. patient may experience unsafe care resulting in minor injury. injury may result in increased length of hospital stay deficient risk of harm and unsafe care on care attendant approach is moderately possible. patient may experience unsafe care resulting in major injury. injury may result in increased length of hospital stay or death. ijahp article: forbes, hebb, mu/ethical decision making in action: evaluating hospital care attendance approaches international journal of the analytic hierarchy process 323 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.592 table 4 benefits criteria model with alternatives rating scale benefits criteria model with alternatives rating scale b1 customer valve (0.167) b2 safety (0.833) b1-1 patient perceived value (0.667) b1-2 hospital perceived value (0.333) measure ment type subjective subjective subjective rating scale 1 excellent 0.4 03 very good 0.1 55 good 0.0 65 regular deficient 1 excellent 0.4 03 very good 0.1 55 good 0.0 65 regular deficient 1 excellent 0.4 03 very good 0.1 55 good 0.0 65 regular deficient a1 care attendant s very good good very good a2 continuo us video monitorin g good very good good a3 normal rounding regular good regular a4 family visitor sitter (fvs) very good good regular ijahp article: forbes, hebb, mu/ethical decision making in action: evaluating hospital care attendance approaches international journal of the analytic hierarchy process 324 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.592 5.1 benefit model pairwise comparison 5.1.1 with respect to benefits, what is more important safety or customer value? figure 3 criteria pairwise comparison with respect to the goal: safety vs customer value safety is important because both nurses and hospital administrators want the patient to remain safe during their care. also, length of stay could increase or injury could result if the patient were to fall or stop receiving care by removing an iv or drain. patients, nurses and hospital customers also want to feel safe during their hospital stay and they value the attention given to patient safety. also, customer value is significant because it is a determinant factor in whether the customer will come back to receive care at the facility or choose to go elsewhere. however, patient safety takes priority over where they will choose to receive care in the future. even if the patient doesn’t perceive it to be so, it is vital for the patient to remain safe during their hospital stay. for this reason, safety is considered to be more important than customer value leading to safety (0.833) having a much higher priority than customer value (0.167). 5.1.2 with respect to customer value, what is more important patient perceived value or hospital perceived value? figure 4 sub-criteria pairwise comparison with respect to the “customer value” criterion: patient’s perceived value vs hospital’s perceived value it is the expected that the patient will come back to receive care, if needed, at the hospital based on the previous care they received and the patient’s perceived value of that care. these are subjective perceptions and the hospital and patient may even disagree on their perceptions. therefore, whose perception should be more important? this is a hot topic for discussion, but it was agreed that while the patient’s perception may be independent from the actual safety situation, the patient’s satisfaction should still be given greater consideration. for this reason, patient perceived value is from equally to moderately more important (3) than hospital perceived value, and hence their relative priorities are 0.667 and 0.333 respectively. ijahp article: forbes, hebb, mu/ethical decision making in action: evaluating hospital care attendance approaches international journal of the analytic hierarchy process 325 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.592 table 5 benefits model results benefits model ratings alternatives b1 customer value b2 safety total normal rank 0.167 0.833 b1-1 patient perceived b1-2 hospital perceived local weights 0.667 0.333 global weights 0.111 0.056 0.833 1.000 a1 care attendant (ca) 0.403 0.155 0.403 0.383 0.521 1 a2 continuous video monitoring (cvm) 0.155 0.403 0.155 0.155 0.210 2 a3 normal rounding (nr) 0.065 0.155 0.065 0.084 0.114 4 a4 family visitor sitter (fvs) 0.403 0.155 0.065 0.108 0.146 3 0.736 1.000 the above results show that using a care attendant to provide patient care is by far (0.521) the most beneficial approach taking into account the importance given to each of the benefits. this approach addresses the hospital’s concern for patient safety as well as the patient’s own safety perception. 5.2 benefit model sensitivity analysis while the superiority of the care attendant (ca) alternative (0.521) is clear, it is necessary to perform a sensitivity analysis to assess the robustness of this decision. table 6 shows the results for the case of both criteria (b1-customer value and b2-safety) having the same weight, while table 7 considers the same scenario but with the caveat of the patient and hospital’s perceived value having the same importance. ijahp article: forbes, hebb, mu/ethical decision making in action: evaluating hospital care attendance approaches international journal of the analytic hierarchy process 326 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.592 table 6 sensitivity scenario 1: benefits model with equally important criteria benefits model ratings alternatives b1 customer value b2 safety total normal rank 0.500 0.500 b1-1 patient perceived b1-2 hospital perceived local weights 0.667 0.333 global weights 0.334 0.167 0.5 1.000 a1 care attendant (ca) 0.403 0.155 0.403 0.363 0.494 1 a2 continuous video monitoring (cvm) 0.155 0.403 0.155 0.155 0.210 3 a3 normal rounding (nr) 0.065 0.155 0.065 0.121 0.165 4 a4 family visitor sitter (fvs) 0.403 0.155 0.065 0.193 0.262 2 0.736 1.000 table 7 sensitivity scenario 2: benefits model with equally important criteria plus equally important sub-criteria benefits model ratings alternatives b1 customer value b2 safety total normal rank 0.500 b1-1 patient perceived b1-2 hospital perceived local weights 0.5 0.5 global weights 0.250 0.250 0.500 1.000 a1 care attendant (ca) 0.403 0.155 0.403 0.364 0.495 1 a2 continuous video monitoring (cvm) 0.155 0.403 0.155 0.155 0.210 3 a3 normal rounding (nr) 0.065 0.155 0.065 0.149 0.203 4 a4 family visitor sitter (fvs) 0.403 0.155 0.065 0.172 0.234 2 0.736 1.000 as can be concluded from these sensitivity scenarios, the rank of the alternatives remains the same. this means that the original results from table 2 are quite robust. indeed, the use of a care attendant (ca) is by far the most beneficial alternative. ijahp article: forbes, hebb, mu/ethical decision making in action: evaluating hospital care attendance approaches international journal of the analytic hierarchy process 327 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.592 5.3 cost model analysis criteria/ sub-criteria a summary of cost criteria/sub-criteria is shown in appendix d. an objective rating scale, based on us dollars, was developed for each cost criteria/sub-criteria and the alternatives were rated accordingly as shown in table 8. the priority results based on this ahp cost model analysis are shown in table 9. (see table 8 on next page) ijahp article: forbes, hebb, mu/ethical decision making in action: evaluating hospital care attendance approaches international journal of the analytic hierarchy process 328 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.592 table 8 costs criteria model with alternatives rating scale costs criteria model with alternatives rating scale c1 fixed costs (0.125) c2 variable costs (0.875) c1-1 acquisition costs (0.125) c1-2 setting up costs (0.875) n/a measurement type objective objective objective rating scale rating amount 1 $20,000 0.75 $15,000 0.5 $10,000 0.25 $5,000 0 $0 rating amount 1 $9,360 0.75 $7,500 0.5 $5,000 0.25 $2,500 0 $0 rating amount 1 $312.00 0.75 $234.00 0.5 $156.00 0.25 $72.00 0 $0 a1 care attendants $0 (0) average $13 hourly wage per care attendant total value for 6 week training period $3,210 (before benefits and taxes) in a 24 hour period a patient would have 3 care attendants taking care of them. therefore, 3 attendants need to be trained. total cost: $9,360 (1) $13 per hour $312/day/patient (1) a2 continuous video monitoring $20,000/monitor per patient (1) cost for three care attendants (ca) with training divided by the number of patients that the ca can watch in the hub at one time is eight patients, value for the cost of virtually monitoring the one patient $9,360 /8total cost $1,170 (0.125) $39/day/patient (0.125) a3 normal rounding $0.00 (0) $0.00 (0) $52/day/patient (10 minutes per patient per hour) (0.166) a4 family visitor sitter (fvs) $0.00 (0) $0.00 (0) $26/day/patient (5 minutes per patient per day) (0.083) ijahp article: forbes, hebb, mu/ethical decision making in action: evaluating hospital care attendance approaches international journal of the analytic hierarchy process 329 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.592 5.4. pairwise comparison of costs criteria/ sub-criteria 5.4.1. with respect to the ca cost, which is more important fixed costs or variable costs? figure 5 pairwise comparison with respect to the goal: fixed costs vs variable costs the variable costs are more important because it is important to be able to operate the solution. in the economic analysis, variable costs are more important than fixed costs, in particular for the breakeven analysis. for this reason, we concluded that the variable costs (0.875) are very strongly more important than fixed costs (0.125). also, none of the alternatives have expensive fixed costs associated with them or will depreciate over time. 5.4.2. with respect to fixed cost, which is more important acquisition or the setting up costs? figure 6 pairwise comparison of sub-criteria with respect to “fixed costs” criterion: acquisition costs vs setting up costs the setting up costs are more important than the acquisition costs because the setting up costs involve multiple interdisciplinary teams and departments as well as initial staff education and buy-in. setting up costs (0 .875) were estimated to be very strongly more important than the acquisition costs (0.125). ijahp article: forbes, hebb, mu/ethical decision making in action: evaluating hospital care attendance approaches international journal of the analytic hierarchy process 330 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.592 table 9 cost model results costs model ratings alternatives c1 fixed costs c2 variable costs total normal rank criteria weights 0.130 0.875 c1-1 acquisition cost c1-2 setting up cost local weights 0.125 0.875 global weights 0.016 0.114 0.875 1 a1 care attendant (ca) 0.000 1.000 1.000 0.989 0.631 1 a2 continuous video monitoring (cvm) 1.000 0.125 0.125 0.359 0.229 2 a3 normal rounding (nr) 0.000 0.000 0.166 0.145 0.093 3 a4 family visitor sitter (fvs) 0.000 0.000 0.083 0.073 0.046 4 1.566 1.000 5.5 sensitivity analysis for the cost model data from table 9 (normal priorities) shows that the costliest solution to increase patient safety is the use of a care attendant (0.631) followed by the engagement of a family visitor sitter (0.046). to evaluate how much these results are affected by the importance of the criteria a sensitivity analysis was performed. in the first scenario (table 10), the same weight was given to both fixed and variable costs (although this is not the standard financial practice which generally assigns more importance to the variable costs). still, the most expensive alternative is care attendant followed by family visitor sitter. when adding the consideration of the sub-criteria acquisition and setting up costs having the same importance, the rank of the alternative costs does not change as shown in table 11. this sensitivity analysis indicates that the results of table 9 are pretty robust. ijahp article: forbes, hebb, mu/ethical decision making in action: evaluating hospital care attendance approaches international journal of the analytic hierarchy process 331 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.592 table 10 sensitivity scenario 1: cost model with equally important criteria costs model ratings alternatives c1 fixed costs c2 variable costs total normal rank criteria weights 0.500 c1-1 acquisition cost c1-2 setting up cost local weights 0.125 0.875 global weights 0.063 0.438 0.500 1.000 a1 care attendant (ca) 0.000 1.000 1.000 0.938 0.715 1 a2 continuous video monitoring (cvm) 1.000 0.125 0.125 0.250 0.191 2 a3 normal rounding (nr) 0.000 0.000 0.166 0.083 0.063 3 a4 family visitor sitter (fvs) 0.000 0.000 0.083 0.042 0.032 4 1.312 1.000 table 11 sensitivity scenario 2: same as scenario 1 with equally important sub-criteria costs model ratings alternatives c1 fixed costs c2 variable costs total normal rank criteria weights 0.500 c1-1 acquisition cost c1-2 setting up cost local weights 0.500 0.500 global weights 0.250 0.250 0.500 1.000 a1 care attendant (ca) 0.000 1.000 1.000 0.750 0.572 1 a2 continuous video monitoring (cvm) 1.000 0.125 0.125 0.437 0.333 2 a3 normal rounding (nr) 0.000 0.000 0.166 0.083 0.063 3 a4 family visitor sitter (fvs) 0.000 0.000 0.083 0.041 0.032 4 1.312 1.000 5.6 benefit cost analysis in benefit cost analysis, the alternatives are prioritized with respect to benefits (table 5) and costs (table 9) separately. the priorities from the benefits table reflect how beneficial the care attendance approach alternatives are. the table can be interpreted as the higher the priority the higher the benefit. on the other hand, the priorities from the costs table reflect how costly the alternatives are; that is, the higher the priority the more costly the alternative. for this reason, it is necessary to calculate the b/c ratio for each of ijahp article: forbes, hebb, mu/ethical decision making in action: evaluating hospital care attendance approaches international journal of the analytic hierarchy process 332 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.592 the alternatives. the alternative with the highest b/c ratio will be the most benefit cost effective as shown in table 12. table 12 benefit cost b/c analysis benefit cost analysis alternatives benefit cost benefit/cost ratio normal rank a1 care attendant (ca) 0.521 0.631 0.826 0.134 3 a2 continuous video monitoring (cvm) 0.21 0.229 0.917 0.149 4 a3 normal rounding (nr) 0.114 0.093 1.226 0.200 2 a4 family visitor sitter (fvs) 0.146 0.046 3.174 0.517 1 6.142 1.000 from the table above we find that the most benefit cost effective alternative is a4 family visitor sitter (fvs) with a normalized b/c ratio of 0.517, simply because it has the least cost priority (0.046) than any other alternative. however, a1 care attendant (ca) is the one with the highest benefit priority (0.521) more than 3 times that of a4 fvs (0.146), but also with the highest cost priority (0.631). this leads us to discuss whether the benefits should weigh more than the costs when making the final decision. this analysis will be made next. 5.6.1 benefit cost sensitivity analysis using strategic criteria the consideration of whether the benefits should outweigh costs is related to the strategic criteria for making the overall decision. this is important because the final evaluation priority of the alternatives may be very sensitive to the importance given to either benefits or costs. in our decision analysis, our model criteria are based on the ethical need to incorporate all the stakeholder’s considerations; that is, those corresponding to the hospital, nurses and patients as shown in the strategic criteria row in table 13. following best practices, the benefits and costs merits were rated according to their importance to address the proposed strategic criteria (saaty & ozdemir, 2005). the results are shown in table 13. as can be seen for our decision, the overarching importance is given to the patient satisfaction (0.731) followed by nurses considerations (0.188) and hospital concerns (0.08) respectively. based on the importance of these strategic criteria, it is concluded that benefits should have a greater weight (0.642) than costs (0.359). therefore, the next step would be to re-calculate our benefit cost analysis from table 12 taking into consideration the greater importance of benefits. the results, using the additive subtractive analysis commonly used for this type of analysis, are shown in table 13. ijahp article: forbes, hebb, mu/ethical decision making in action: evaluating hospital care attendance approaches international journal of the analytic hierarchy process 333 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.592 table 13 strategic benefit cost analysis strategic benefit cost analysis strategic criteria hospital concerns (0.080) nursing considerations (0.188) patient satisfaction (0.731) measurement subjective subjective subjective rating scale 5 high importance 4 3 medium importance 2 1 low importance 5 high importance 4 3 medium importance 2 1 low importance 5 high importance 4 3 medium importance 2 1 low importance total normalized benefits 3 5 5 4.835 0.642 costs 4 1 3 2.701 0.359 7.536 1 table 14 allows a comparison of the original results (b and c unweighted) with the new ones using strategic criteria. lines 3 and 4 show the multiplicative (b/c) original results and line 5 shows the rank of the alternatives as taken from table 9 which we previously discussed. to perform a weighted benefit cost analysis it is recommended that the additive subtractive synthesis analysis of the form b – c (unweighted) or bb – cc (weighted) as shown in table 14 be used. in this table, line 7 provides the normalized results for the unweighted case and line 8 provides the rank of the alternatives. notice that the rank of alternatives is the same whether we use the multiplicative approach (line 5) or the additive synthesis approach (line 7). our next step is to use the benefits and costs weight (lines 9 and 10) as factors in the calculation of the weighted synthesis shown in line 13. as can be seen, this drastically changes the order of our preferences (line 15). when using our strategic criteria to weight the benefits and costs, the original worst alternative (a1 care attendant) becomes the very best. what used to be the best alternative (a4 family visitor sitter) now becomes the second best alternative. the worst alternative is now a3 normal rounding while a2 continuous video monitoring remains as the third best alternative. these results, based on strategic weighting of the merits b and c, as shown in lines 14 (normalized priorities) and 15 (rank), are the ones that must be considered in strategic decision-making concerning care attendance approach evaluation. ijahp article: forbes, hebb, mu/ethical decision making in action: evaluating hospital care attendance approaches international journal of the analytic hierarchy process 334 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.592 table 14 strategic benefit cost criteria analysis results a1 a2 a3 a4 line description formula value care attendance continuous video monitoring normal rounding family visitor sitter 1 benefits b 0.521 0.21 0.114 0.146 2 costs c 0.631 0.229 0.093 0.046 3 b/c multiplicative b/c 0.826 0.917 1.226 3.174 4 b/c normalized 0.134 0.149 0.200 0.517 5 rank 4 3 2 1 6 bc additive synthesis b c -0.110 -0.019 0.021 0.100 7 additive synthesis normalized -0.440 -0.076 0.084 0.400 8 rank 4 3 2 1 9 benefits weight b 0.642 10 costs weight c 0.359 11 weighted benefits b*b 0.334 0.135 0.073 0.094 12 weighted costs c*c 0.227 0.082 0.033 0.017 13 bc weighted additive synthesis b*b c*c 0.108 0.053 0.040 0.077 14 weighted additive synthesis normalized 0.389 0.190 0.143 0.278 15 rank 1 3 4 2 figures 7a and 7b show in a graphical way how sensitive our bc weighted additive synthesis results (line 13 in table 15) are with respect to the weights given to the benefits (b) and costs (c) in lines 9 and 10 respectively from table 15. the addition of b + c must always add to 1. ijahp article: forbes, hebb, mu/ethical decision making in action: evaluating hospital care attendance approaches international journal of the analytic hierarchy process 335 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.592 figure 7a strategic sensitivity respect to benefits figure 7b strategic sensitivity respect to costs notice, in figure 7a, that when benefits are as important as costs (0.5 each), a1 (care attendant) is the worst alternative, as found in our original scenario, because it has the lowest priority. however, as the benefits begin having more importance than the costs, a1 starts to quickly outperform the other alternatives. therefore, since our care attendance approach evaluation is based on the need to provide benefits (0.642) rather than diminish costs (0.359), it is clear that the use of a1 (care attendant) constitutes the best alternative. -0.400 -0.300 -0.200 -0.100 0.000 0.100 0.200 0.300 0.400 0.500 0.600 0 0.25 0.5 0.75 1 bc synthesis b*b c*c benefits weight b a1 ca a2 cvm a3 nr a4 fvs -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 0 0.25 0.5 0.75 1 bc synthesis b*b c*c costs weight c a1 ca a2 cvm a3 nr a4 fvs ijahp article: forbes, hebb, mu/ethical decision making in action: evaluating hospital care attendance approaches international journal of the analytic hierarchy process 336 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.592 this conclusion can also be derived from figure 7b where our bc weighted additive synthesis results show that a1 (care attendant) becomes the worse solution as the importance of costs increases while a1 (care attendant) constitutes the best solution when costs weights are lower than 0.5 approximately; that is, costs are less important than benefits. 6. conclusion in conclusion, this study shows a basic model, derived from the extant medical literature that can be used to prioritize the different care attendance approaches used in the acute care hospital setting. suitable format for care attendance approaches is an important aspect to keeping patients safe while in the hospital. furthermore, this study addresses the choice of care attendance as an ethical decision, and for this reason the three different stakeholder opinions are included namely, hospital administration, nurses and patients. our study shows that the best overall approach is the use of a dedicated care attendant (0.389, table 14) followed by family/visitor sitters (0.278). this is highly influenced by the fact that the close presence of a person gives the patient a strong sense of security and patient’s perception has a large importance as a benefit (0.667, table 5). still, most hospitals will need to use mixed patient care approaches to balance both benefits and costs. the findings of this strategic benefit-cost strategic analysis sensitivity allowed us to conclude that in a mixed approach situation, hospitals could distribute resources as suggested by this study prioritization; that is, ca could be allocated 38.9% (table 14) of the resources to keep patients safe and maximize patient’s perception of safety. as each patient’s ethical implications and decisions come into play, resources for the other alternatives could be distributed as follows: fvs (27.8%), cvm (19%) and finally nr (14.3%). this model can be used as an evaluation format for hospitals and serve as a reference for care attendance approaches resource allocation acute care settings. 7. limitations and future research expert representation for each stakeholder perspective may be somewhat limited. ahp cannot fully take away the subjectivity of a single participant and for this reason, extending the number of qualified expert judgments in all stakeholder groups may be convenient. this can be done by having focus groups of experts for each stakeholder perspective or surveying a large number of stakeholders. another area of exploration is to develop different decision hierarchies for each stakeholder perspective, namely, for the hospital, nurse and patient’s perspective. in this case, the criteria of each perspective could be specifically tailored in a different hierarchy for each stakeholder. for example, while patients may not think much of the overall costs of each care attendance approach, they may be more mindful of out-of-pocket costs. while there are several possible areas of further development, this study constitutes an important first step toward a quantitative evaluation framework taking into account the essential factors, based on extant literature, to assess current care attendance approaches. also, this evaluation framework can serve as a practical reference for decision-makers in the acute care setting. ijahp article: forbes, hebb, mu/ethical decision making in action: evaluating hospital care attendance approaches international journal of the analytic hierarchy process 337 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.592 8. contributions one of the most important and unique characteristics of the present study is that it shows that care attendance selection is an ethical decision based on curtin’s (1978) ethical model criteria. to address the ethical dilemma of cost-effectiveness (hospital view) versus safety and value (patients and nurse), this study integrates all these different perspectives in a care attendant evaluation framework following a stakeholder’s approach (freeman, 1984). another important characteristic is that rather than using a traditional financial b/c analysis, we have used the analytic hierarchy process which allows the inclusion of intangible considerations such as “patient’s perceived value” (saaty, 2001). the final priorities obtained for each approach can be used to allocate resources proportionately. also, this study allows understanding of the key factors, based on current medical literature, involved in the evaluation of the different care attendance approaches. finally, this report follows recent best practices proposed for this type of study (mu, cooper & peasley, 2018). ijahp article: forbes, hebb, mu/ethical decision making in action: evaluating hospital care attendance approaches international journal of the analytic hierarchy process 338 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.592 references babine, r. l., hyrkas, k. e., hallen, s., wierman, h., bachand, d., chapman, j., & fuller, v. (2018). falls and delirium in an acute care setting: a retrospective chart review before and after an organisation-wide interprofessional education. journal of clinical nursing, 27, 1429-1441. doi: https://doi.org/10.1111/jocn.14259 boswell, d. j., ramsey, j., smith, m. a., & wagers, b. (2001). the cost-effectiveness of a patient-sitter program in an acute care hospital: a test of the impact of sitters on the incidence of falls and patient satisfaction. quality management in health care, 10(1), 10-16. burtson, p. l., & vento, l. (2015). sitter reduction through mobile video monitoring a nurse-driven sitter protocol and administrative oversight. the journal of nursing administration, 45(7/8), 363-368. doi: 10.1097/nna.0000000000000216 centers for disease control. (2017). important facts about falls. retrieved from http://www.cdc.gov/homeandrecreationalsafety/falls/adultfalls.html colella, j., aroh, d., douglas, c., vanbuitenen, n., galesi, j., castro, a., menacker, m. (2017). managing delirium behaviors with one-to-one sitters. nursing management, 47(8), 1-5. doi: 10.1097/01.numa.0000522172.15640.6b centers for medicaid and medicare services. (2017, december 21). retrieved august 14, 2018, from https://www.cms.gov/medicare/quality-initiatives-patient-assessmentinstruments/hospitalqualityinits/hospitalhcahps.html cournan, m., fusco-gessick, b., & wright, l. (2018). improving patient safety through video monitoring. rehabilitation nursing, 43(2), 111-115. doi:10.1002/rnj.308 curtin, l. (1978). a proposed critical model for ethical analysis. nursing forum, xvii(1). doi:10.1111/j.1744-6198.1978.tb01254.x danaf, j., chang, b., shaear, m., johnson, k., miller, s., nester, l., aboumater, h. (2017). surfacing and addressing hospitalized patient’s needs: proactive nursing rounding as a tool. journal of nursing management, 00, 1-8. doi: https://doi.org/10.1111/jonm.12580 davis, j., kutash, m., & whyte iv, j. (2017). a comparative study of patient sitters with video monitoring versus in-room sitters. journal of nursing education and practice, 7(3), 137-142. doi: https://doi.org/10.5430/jnep.v7n3p137 dearmon, v., roussel, l., buckner, e., m., pomrenke, b., salas, s., brown, a. (2013). transforming care at the bedside (tab): enhancing direct care and value-added care. journal of nursing management, 21, 668-678. doi: https://doi.org/10.1111/j.13652834.2012.01412.x https://www.cms.gov/medicare/quality-initiatives-patient-assessment-instruments/hospitalqualityinits/hospitalhcahps.html https://www.cms.gov/medicare/quality-initiatives-patient-assessment-instruments/hospitalqualityinits/hospitalhcahps.html ijahp article: forbes, hebb, mu/ethical decision making in action: evaluating hospital care attendance approaches international journal of the analytic hierarchy process 339 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.592 fabry, d. (2015). hourly rounding: perspectives and perceptions of the frontline nursing staff. journal of nursing management, 23, 200-210. doi: https://doi.org/10.1111/jonm.12114 freeman, r. e. (1984). strategic management: a stakeholder approach. cambridge, uk: cambridge university press. doi: https://doi.org/10.1017/cbo9781139192675.003 flowers, k., wright, k., langdon, r., mcllwrath, m., wainwright, c., & johnson, m. (2016). intentional rounding: facilitators, benefits, and barriers. journal of clinical nursing, 25, 1346-1355. doi: https://doi.org/10.1111/jocn.13217 goldsack, j., bergey, m., mascioli, s., & cunningham, j. (2015). hourly rounding and patient falls: what factors boost success? nursing2015research corner, 25-30. doi: 10.1097/01.nurse.0000459798.79840.95 jeffers, s., searcey, p., herring, k., lester, k., goetz-smith, h., & nelson, p. (2013). centralized video monitoring for patient safety: a denver health lean journal. nursing economics, 31(6), 298-306. the joint commission. (2017). retrieved april 12, 2018, from https://www.jointcommission.org/standards_information/npsgs.aspx jong, l., kitchen, s., & hill, a. (2017). exploring falls prevention capabilities, barriers and training needs among patient sitters in a hospital setting: a pilot survey. geriatric nursing, 1-8. doi: https://doi.org/10.1016/j.gerinurse.2017.09.006 kessler, b., claude-gutekunst, m., donchez, a. m., dries, r. f., & snyder, m. m. (2012). the merry-go-round of patient rounding: assure your patients get the brass ring. medsurg nursing, 21(4), 240-245. laws, d., & crawford, c. (2013). alternative strategies to constant patient observation and sitters. the journal of nursing administration, 43(10), 497-501. doi: 10.1097/nna.0b013e3182a3e83e morgan, l., flynn, l., robertson, e., new, s., forde-johnston, c., & mcculloch, p. (2016). intentional rounding: a staff-led quality improvement intervention in the prevention of patient falls. journal of clinical nursing, 26, 115-124. doi: https://doi.org/10.1111/jocn.13401 mu, e., cooper, o. and peasley, a. (2018). improving analytic network process reporting. international symposium of the analytic hierarchy process, hong kong. mu, e., & pereyra-rojas, m. (2017). practical decision making: an introduction to the analytical hierarchy process (ahp) using superdecisons v2. springer briefs in operations research:springer international publishing. national database for nursing quality indicators. (2018). retrieved august 14, 2018, from http://www.pressganey.com/solutions/clinical-quality/nursing-quality ijahp article: forbes, hebb, mu/ethical decision making in action: evaluating hospital care attendance approaches international journal of the analytic hierarchy process 340 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.592 neville, k., lake, k., lemunyon, d., paul, d., & whitmore, k. (2012). nurses; perceptions of patient rounding. the journal of nursing administration, 42(2), 83-88. doi: 10.1097/nna.0b013e318243365e neville, k., dibona, c., & mahler, m. (2016). validation of the nurse’s perception of patient rounding scale: an exploratory study of the influence of shift work on nurses' perception of patient rounding. orthopedic nursing, 35(2), 84-912. doi: 10.1097/nor.0000000000000223 rochefort, c., buckeridge, d., & abrahamowicz, m. (2015). improving patient safety by optimizing the use of nursing human resources. implementation science, 10(89), 1-11. saaty, t. l. (2001). decision making for leaders: the analytic hierarchy process for decisions in a complex world. pittsburgh, pa: rws publications. saaty, t. l. and ozdemir, m. s. (2005). the encyclicon: a dictionary of decisions with dependence and feedback based on the analytic network process. pittsburgh, pa: rws publications. sand-jecklin, k., johnson, j. r., & tylka, s. (2016). protecting patient safety: can video monitoring prevent falls in high-risk patient populations. journal of nursing care quality, 31, 131-138. doi: 10.1097/ncq.0000000000000163 schoberer, d., breimaier, h., mandl, m., halfens, r., & lohrmann, c. (2016). involving the consumers: a exploration of user’s' and caregiver’s needs and expectations on a fall prevention brochure: a qualitative study. geriatric nursing, 37, 207-214. doi: https://doi.org/10.1016/j.gerinurse.2016.02.011 solimine, s., takeshita, j., goebert, d., lee, j., schultz, b., guerrero, m., lawyer, l. (2018). characteristics of patients with constant observers. psychosomatics, 59(1), 67-74. doi: https://doi.org/10.1016/j.psym.2017.08.003 super decisions (2018). super decisions. retrieved from https://www.superdecisions.com/ torkelson, d., & dobal, m. (1999). constant observation in medical-surgical settings: a multi hospital study. nursing economics, 17(3), 149-155. tzeng, h., & yin, c. (2007). using family visitors, sitters, or volunteers to prevent falls. journal of nursing administration, 37(78), 329-334. doi: 10.1097/01.nna.0000285116.68313.c1 tzeng, h., & yin, c. (2009). inpatient falls: the impact of family and personal caregivers. applied nursing research, 22, 159-165. doi: https://doi.org/10.1016/j.apnr.2007.09.002 tzeng, h., yin, c., & grunawalt, j. (2008). effective assessment of use of sitters by nurses in inpatient care setting. journal of advanced nursing, 64(2), 176-184. doi: https://doi.org/10.1111/j.1365-2648.2008.04779.x ijahp article: forbes, hebb, mu/ethical decision making in action: evaluating hospital care attendance approaches international journal of the analytic hierarchy process 341 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.592 votruba, l., graham, b., wisinski, j., & syed, a. (2016). video monitoring to reduce falls and patient companion costs for adult inpatients. nursing economics, 34, 185-189. wood, j. (2001). ethical decision making. journal of perianesthesia nursing, 16(1), 610. doi: https://doi.org/10.1053/jpan.2001.18202 ijahp article: forbes, hebb, mu/ethical decision making in action: evaluating hospital care attendance approaches international journal of the analytic hierarchy process 342 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.592 appendices appendix a review summary of extant literature on care attendance item citation topic pertinent findings 1 (sand-jecklin, johnson, & tylka, 2016) hospital, patient perceived, customer value, safety, continuous video monitoring cvm can also aide in reducing cost of sitter hours and reduction in patient falls. 2 (votruba, graham, wisinski, & syed, 2016) customer value, hospital perceived, patient perceived, safety, fixed cost-installation costs, variable costs, continuous video monitoring cmv can be used for elopement, protect patients from interfering with medical devices, and monitor seizure activity as well. 3 (kessler, claudegutekunst, donchez, dries, & snyder, 2012) normal rounding, patient and hospital perceived customer value, safety increased press ganey scores usually in the 80 th to 90 th percentile in how well patients pain was controlled, promptness in response to call bell, and how well staff cared for patients. fall rate of patients decreased from 5.46% to 2.19% after implementation of hourly rounding. 4 (flowers et al., 2016) patient and hospital perceived, customer value, normal rounding intentional rounding includes common elements of assessing and managing pain, assistance with toileting, repositioning and comfort, and ensuring essential items including call bell, telephone, and bedside table easily accessible to patient. staff engagement is key for successful implementation. 5 (solimine et al., 2018) care attendants, safety, patient perceived, hospital perceived, care attendants a multi-disciplinary team approach to safety and sitters can be effective. screening was performed on patients to assess ability to make basic needs known, respond to diversion, and ambulate independently. this approach showed success in prevention of delirium, prevent functional and cognitive decline, and decrease patient falls. 6 (tzeng & yin, 2007) fvs, customer value, patient perceived the involvement of family visitor sitters can provide psychological support to patients, but cannot replace rn’s in effectively reducing patient falls. this is because family visitor sitters lack professional training. 7 (neville, lake, lemunyon, paul, & whitmore, 2012) hospital perceived and patient perceived customer value, normal rounding it is important to have staff engagement with the use of hourly rounding. effective rounding can impact how patients perceive nursing care, improving hchaps. 8 (morgan et al., 2016) customer value, hospital perceived, patient intentional rounding can aide in significantly reducing the ijahp article: forbes, hebb, mu/ethical decision making in action: evaluating hospital care attendance approaches international journal of the analytic hierarchy process 343 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.592 perceived, safety, normal rounding number of patient falls. 9 (tzeng, yin, & grunawalt, 2008) customer value, hospital perceived, safety, care attendants the use of a patient attendant assessment tool (paat) should be used in the acute care setting related to the provision of constant observation sitters. it may be helpful in the rn assessment of patient needs for sitter to better judge requests for scare nursing resources. 10 (davis, kutash, & whyte iv, 2017) care attendants, continuous video monitoring, operational costs, safety, customer value, patient perceived, hospital perceived there was no significant difference in patient falls for cvm and in room sitters. the study revealed a decrease in costs per patient sitter day without a significant increase in patient falls or harms when video monitoring was used on a unit for majority of patients who required constant observers. 11 (boswell, ramsey, smith, & wagers, 2001) customer value, patient perceived, safety, implementation costs, operational costssetting up, fixed costs, care attendants sitters can have a marginal impact of variables selected in the model (patient falls, dissatisfaction, and quality care). this study showed a better overall understanding of costs associated with a patient sitter program. 12 (laws & crawford, 2013) safety, customer value, hospital perceived, patient perceived, variable costs, care attendants implementation a program that focused on preventing delirium can aid in decreasing sitter use. 13 (burtson & vento, 2015) operational costssetting up customer value, hospital perceived, continuous video monitoring, care attendants, safety the implementation of a nursing driven sitter protocol requires that the change agents address widespread nursing beliefs in the effectiveness of sitters through standardized reasons for sitter use. 14 (cournan, fuscogessick, & wright, 2018) safety, operational costssetting up, continuous video monitoring, fixed costsacquisition cost continuous video monitoring can aide in reduction of costs associated with sitter usage. 15 (babine et al., 2018) safety, customer value, hospital and patient perceived, variable cost, setting up costs the study results indicate that improving delirium recognition and treatment through interprofessional education can reduce falls and length of stay. 16 (neville, dibona, & mahler, 2016) customer valuehospital perception safety, normal rounding nursing leadership is essential for the success of patient rounding. leaders and nurses should work collaboratively to design optimal tool for patient rounding practices, safety for improved quality care. 17 (torkelson & customer value, patient it is essential to have shared decision making and ijahp article: forbes, hebb, mu/ethical decision making in action: evaluating hospital care attendance approaches international journal of the analytic hierarchy process 344 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.592 dobal, 1999) perceived & hospital perceived, safety, care attendants collaboration between nurses, physicians, patients, families and case managers to continue or discontinue constant observation of patients. 18 (rochefort, buckeridge, & abrahamowicz, 2015) customer value, hospital perceived, safety, care attendants this study would aide in helping leaders in making the most effective use of scare nursing resources. 19 (jong, kitchen, & hill, 2017) customer value, hospital perceived & hospital perceived, safety, fixed cost setting up costs, care attendants there is a gap between patient related risk factors and environmental related. improved training of sitters is needed. 20 (colella et al., 2017) customer value, hospital perceived, care attendants inter-professional partnerships with a common goal of providing safe quality outcomes is needed. 21 (goldsack, bergey, mascioli, & cunningham, 2015) customer value, patient perceived, hospital perceived, safety, normal rounding proactive hourly rounding is essential to keep patients safe and reduce number of falls. it is critical that leadership and frontline staff are involved in the program design. 22 (danaf et al., 2017) customer value, patient perceived, hospital perceived, normal rounding proactive rounding helps address hospitalized patients’ immediate patient needs. nursing teams need to be engaged and informed on essential proactive rounding tasks. 23 (schoberer, breimaier, mandl, halfens, & lohrmann, 2016) customer value, patient perceived, safety, family visitor sitter brochures should be developed regarding information about risk factors, preventative strategies for falls, and additional support to allow family to be involved in patient care. 24 (tzeng & yin, 2009) customer value, patient perceived, hospital perceived, safety, family visitor sitter, normal rounding assessment and reassessment of family members understanding and abilities are key to fall prevention and promotion of safety. family members in taiwan tend to depend on family members to provide care. however, frequent visits from the bedside nurse are still needed. 25 (jeffers et al., 2013) fixedacquisition costs, setting up costs, variable cost safety, customer value, hospital perceived, continuous video monitoring the study showed that cvm had a positive impact on patient falls. ijahp article: forbes, hebb, mu/ethical decision making in action: evaluating hospital care attendance approaches international journal of the analytic hierarchy process 345 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.592 appendix b summary of care attendance alternatives and literature source id definition discussion source from the literature review a1care attendants (ca) the staff members who help patients who are unable to be on their own due to increased risk factors for injuries or accidents. care attendants that are monitoring a patient 1:1 can be measured through the use of the use of the nurse’s completion of the need for a care attendant form algorithm available in the electronic healthcare record system. the cost/time of care attendants can be measured to the number of fte’s used, fall injury data, and cost associated. having a care attendant in the room takes resources away from the unit. it provides the patient with a person that is able to watch for their safety constantly. however, the care attendant must remain attentive at all times or it could still put the patient at risk for safety issues including falling and discontinuing therapies. these issues still occur currently when care attendants are in the room. (solimine et al., 2018); (davis, kutash, & whyte iv, 2017); (boswell, ramsey, smith, & wagers, 2001); (laws & crawford, 2013); (neville, dibona, & mahler, 2016); (torkelson & dobal, 1999); (rochefort, buckeridge, & abrahamowicz, 2015); (jong, kitchen, & hill, 2017); (colella et al., 2017) a2 continuous video monitoring (cmv) the 24/7 use of video technology to watch multiple at-risk patients to prevent fall injuries and accidents. allows the care attendant to be used in resources in other areas of need. (cross trained as a patient care technicians). (sand-jecklin, johnson, & tylka, 2016); (votruba, graham, wisinski, & syed, 2016); (davis, kutash, & whyte iv, 2017); (burtson & vento, 2015); (cournan, fuscogessick, & wright, 2018); (jeffers et al., 2013) a3normal rounding (nr) the checking of all patients on each unit by staff for immediate needs including; toileting, pain, positioning, infusion rates and placement, and overall safety of environment should occur hourly. normal rounding prevents accidents but can be a brief check on the patient and we only see what is going on in the moment. patients are often only able to be rounded on every hour, thus putting the patient at an increased risk for falls due to lack of unsafe attempts at toileting, pain, or items not within reach when a staff member may not present in the room (kessler, claude-gutekunst, donchez, dries, & snyder, 2012); (flowers et al., 2016); (neville, lake, lemunyon, paul, & whitmore, 2012); (danaf et al., 2017); (tzeng & yin, 2009) a4family visitor sitter (fvs) having a family member or visitor help patients who are unable to be on their own due to increased risk factors for injuries or accidents. (tzeng & yin, 2007); (schoberer, breimaier, mandl, halfens, & lohrmann, 2016); (tzeng & yin, 2009) ijahp article: forbes, hebb, mu/ethical decision making in action: evaluating hospital care attendance approaches international journal of the analytic hierarchy process 346 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.592 appendix c benefits criteria/sub-criteria summary and literature source criteria sub-criteria definition measurement and use source from literature review b1 customer value the value convened by the solution, the extent of an environment where high quality care is the overall principle measured through patient satisfaction scores (nationally through hcahps and press ganey. the alternative that provides greater value to the patient, the better. (sand-jecklin, johnson, & tylka, 2016); (votruba, graham, wisinski, & syed, 2016); (kessler, claude-gutekunst, donchez, dries, & snyder, 2012); (flowers et al., 2016); (tzeng & yin, 2007); (neville, lake, lemunyon, paul, & whitmore, 2012); (neville, dibona, & mahler, 2016); (torkelson & dobal, 1999); (rochefort, buckeridge, & abrahamowicz, 2015); (jong, kitchen, & hill, 2017); (colella et al., 2017); (goldsack, bergey, mascioli, & cunningham, 2015); (schoberer, breimaier, mandl, halfens, & lohrmann, 2016); (tzeng & yin, 2009) b1-1 patient perceived is the value of care attendants approach seen by the patient. the patient’s perception of care provided during hospital stay. measured subjectively through point of view of each patient from hchaps and press ganey scores (national measures for satisfaction) the alternative that is perceived higher to the patient, the better the alternative. (sand-jecklin, johnson, & tylka, 2016; (votruba, graham, wisinski, & syed, 2016); (kessler, claude-gutekunst, donchez, dries, & snyder, 2012); (flowers et al., 2016); (tzeng & yin, 2007); (neville, lake, lemunyon, paul, & whitmore, 2012); (boswell, ramsey, smith, & wagers, 2001); (laws & crawford, 2013); (babine et al., 2018); (neville, dibona, & mahler, 2016); (torkelson & dobal, 1999); (jong, kitchen, & hill, 2017); (goldsack, bergey, mascioli, & cunningham, 2015); (schoberer, breimaier, mandl, halfens, & lohrmann, 2016); (tzeng & yin, 2009) b1-2 hospital perceived is the value of care attendants approach seen by the hospital. the hospital’s perception of providing quality care to each particular patient measured subjectively through hospital perception of care provided to patients. (ndnqi national safety measures completed by rn every 2 years). the alternative that is perceived higher to the hospital, the better the alternative. (sand-jecklin, johnson, & tylka, 2016); (votruba, graham, wisinski, & syed, 2016); (kessler, claude-gutekunst, donchez, dries, & snyder, 2012); (solimine et al., 2018); (neville, lake, lemunyon, paul, & whitmore, 2012); (morgan et al., 2016); (tzeng, yin, & grunawalt, 2008); (laws & crawford, 2013); (burtson & vento, 2015); (babine et al., 2018); (neville, dibona, & mahler, 2016); (torkelson & dobal, 1999); (rochefort, buckeridge, & abrahamowicz, 2015); (jong, kitchen, & hill, 2017); (colella et al., 2017); (goldsack, bergey, mascioli, & cunningham, 2015); (tzeng & yin, 2009); (jeffers et al., 2013) b2 safety the extent to which a patient is protected from harm while in the hospital. harm could include falls, injuries, and other infections. monitored and measured subjectively on expert opinion and literature. the higher the safety provided to the patient the better the alternative. (sand-jecklin, johnson, & tylka, 2016); (votruba, graham, wisinski, & syed, 2016); (kessler, claude-gutekunst, donchez, dries, & snyder, 2012); (solimine et al., 2018); (morgan et al., 2016); (tzeng, yin, & grunawalt, 2008); (boswell, ramsey, smith, & wagers, 2001); (laws & crawford, 2013); (burtson & vento, 2015); (cournan, fusco-gessick, & wright, 2018); (cournan, fusco-gessick, & wright, 2018); (babine et al., 2018); (neville et al, 2016); (torkelson & dobal, 1999); (jong, kitchen, & hill, 2017); (rochefort, buckeridge, & abrahamowicz, 2015); (schoberer, breimaier, mandl, halfens, & lohrmann, 2016) ijahp article: forbes, hebb, mu/ethical decision making in action: evaluating hospital care attendance approaches international journal of the analytic hierarchy process 347 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.592 appendix d cost criteria/ sub-criteria summary and literature source criteria sub-criteria definition measurement and use source from literature review c1 fixed costs (installation) the cost of installing the solution. measured as amount of money that is needed to install as specified by vendor and hospital staff the alternative that costs the most, the better the alternative. (boswell, ramsey, smith, & wagers, 2001); (cournan, fusco-gessick, & wright, 2018); (jong, kitchen, & hill, 2017); (jeffers et al., 2013) c1-1 acquisition costs the amount of money it costs to acquire, pay the vendor for each solution. measured as the amount of money that needs to be invested for implementation purposes as specified by the vendor and hospital staff. the alternative that costs the most, the better the alternative. (boswell, ramsey, smith, & wagers, 2001); (cournan, fusco-gessick, & wright, 2018); (jong, kitchen, & hill, 2017); (jeffers et al., 2013) c1-2 setting up costs the cost of operating the total solution. measured as the amount of money that is needed to upgrade the system such as wireless capability, panels, and sprinkler systems. the alternative that costs the most, the better the alternative. (votruba, graham, wisinski, & syed, 2016); (boswell, ramsey, smith, & wagers, 2001); (burtson & vento, 2015); (cournan, fusco-gessick, & wright, 2018); (jong, kitchen, & hill, 2017); (jeffers et al., 2013) c2 variable costs (operational) the cost of operating the total solution. measured as the amount of money that is needed to keep the operation running. can be measured in terms of fte ratio/used. the alternative that costs the most, the better the alternative. (votruba, graham, wisinski, & syed, 2016); (davis, kutash, & whyte iv, 2017); (boswell, ramsey, smith, & wagers, 2001); (laws & crawford, 2013); (jong, kitchen, & hill, 2017); (jeffers et al., 2013) ijahp article: poveda, lipsett/weighting sustainable development indicators (sdis) for surface mining operations using the analytical hierarchy process international journal of the analytic hierarchy process 200 vol. 5 issue 2 2013 issn 1936-6744 weighting sustainable develoment indicators (sdis) for surface mining operations using the analytical hierarchy process (ahp) cesar a. poveda department of mechanical engineering university of alberta edmonton, ab, canada email: poveda@ualberta.ca michael g. lipsett department of mechanical engineering university of alberta edmonton, ab, canada email: mlipsett@ualberta.ca abstract current multi-criteria decision-making methods (mcdm) present valid alternatives for weighting the various criteria while allowing for the participation of different stakeholders. among those, the analytical hierarchy process (ahp) structures the decision problem in a manner that is easy for the stakeholders to comprehend and allows them to analyze independent sub-problems by structuring the problem in a hierarchy and using pairwise comparisons. this paper presents the application of the analytical hierarchy process to weight the different criteria to measure the sustainability of surface mining operations. prior to the application of the ahp method, the various criteria were preselected using a preliminary selection method consisting of the identification of criteria from six different sources: governmental regulations; committees and organizations for standardization; management and processes best practices; academicallyand scientifically-authored resources; local, regional, national, and international organizations; and industry sector standards and programs. criteria with different common sources of origin, as well as discretionary project and stakeholder relevance were chosen for the preselected list. the different social, economic, and environmental criteria were classified in ten different areas of excellence to facilitate the application of the weighting method. therefore, each criterion’s final weight is impacted by the criterion’s weight itself and the area of excellence’s weight obtained in the application of the ahp method. the results of the weighting process assist scientists and practitioners by not only identifying those criteria that stakeholders consider relevant in the sustainability assessment process, but also by expressing the degree to which the criteria should be addressed in order to accomplish the project’s and/or organization’s sustainability goals. mailto:poveda@ualberta.ca ijahp article: poveda, lipsett/weighting sustainable development indicators (sdis) for surface mining operations using the analytical hierarchy process international journal of the analytic hierarchy process 201 vol. 5 issue 2 2013 issn 1936-6744 keywords: sustainability, sustainable development indicators (sdis), analytical hierarchy process (ahp), multi-criteria decision-making methods (mcdm), surface mining operations 1. introduction with the emergence of sustainability not only in practice but also as a solid area of research, the assessment of sustainable development integrating its three pillars advances into using scientific and mathematical approaches with the ultimate goal of meeting and balancing the different stakeholder needs. this manuscript presents a framework for utilizing the analytical hierarchy process (ahp) to weight sustainable development indicators (e.g., criteria) for surface mining operations. this multi-criteria decisionmaking method is part of an integrated approach for sustainability assessment encountered in the wa-pa-su project sustainability rating system (poveda & lipsett, 2011a; 2011b). the basic components of a sustainability assessment methodology involve three distinct stages: (1) identification of sustainable development indicators (sdis), which answers the question of what to measure; (2) development of metrics, which addresses the challenge of how to measure the sdis; and (3) application of assessment models (i.e., assessment methodology), which typically uses a scientific approach to deliver a comprehensive valuation that includes an assessment of the diverse impacts, input of stakeholders’ views, and application of mathematical models. instead of abstract and complex assessment tools, the users and stakeholders favor simplistic, flexible, and practical approaches with an expected numeric value as the result. a numeric result of the assessment facilitates not only an understanding of the methodology, but also the internal and external performance benchmarking process. depending on the methodology, tool, instrument, or process used, the results of the assessment are given in comparative parameters (e.g., time, cost) or simply a value in a numeric scale; however, munda (2006) states “from the point of view of the management what is really important is the benchmarking exercise and not the ranking.” previous assessments have mainly focused on the environmental criteria instead of integrating the three pillars of sustainability (social, economic and environmental). however, as sustainability is becoming better understood diverse tools, methodologies, processes, and instruments are developing to integrate the social and economic facets, with the aim of achieving a balanced approach to sustainability assessment. furthermore, while a notorious transition has occurred from environmental regulations to environmental assessment, demonstrating substantial levels of maturation in practice and theory, other pillars of sustainability (i.e., social and economic) face challenges in advancing at the same rate. nevertheless, sustainability is still in its infant stage, and progress made in the environmental area, which is better understood by stakeholders and the public in general, demonstrates the need for the social and economic pillars to improve. the economics of sustainability tend to be interpreted as how well an organization is doing financially, instead of measuring the economic impacts of its rob typewritten text http://dx.doi.org/10.13033/ijahp.v5i2.199 ijahp article: poveda, lipsett/weighting sustainable development indicators (sdis) for surface mining operations using the analytical hierarchy process international journal of the analytic hierarchy process 202 vol. 5 issue 2 2013 issn 1936-6744 performance, and the social pillar is faced with the major challenge of measuring impacts that are intrinsically subjective. several environmental and sustainability assessment tools, instruments, processes, and methodologies have been developed and are continuously evolving to address the stakeholders’ needs. the outcome of scientific research in areas of sustainability is poorly understood. rating systems stand out and have gained attention and credibility, as demonstrated by the vast number of certified projects around the world and by the widely-known advantages of using them (yudelson, 2008; issa, rankin, & christian, 2009). green and sustainability rating systems inherently possess a developed scale in which the users are requested to achieve a certain level with the aim of guaranteeing the sustainability of the project and/or organization. rating systems are developed to meet the needs of specific characteristics, with the aim of categorizing, certifying or acknowledging the project and/or organization as sustainable. therefore, the sdis included in the assessment process are selected to reflect the diverse impacts and/or expected performance of projects and/or organizations during their life cycles. the use of rating systems has rapidly spread in certain industries (e.g., buildings), which has required the development of a number of rating systems for specific projects (e.g., schools, healthcare, homes, commercial, neighborhoods) within the building industry. however, other projects and industries do not possess such rating systems to demonstrate their performance in sustainable development. among others, leadership in energy and environmental design (leed), comprehensive assessment system for built environment efficacy (casbee), building research establishment environmental assessment method (breeam), gbtool, and green start lead their local markets and are working to rapidly penetrate markets abroad. areas of performance (e.g., categories) and criteria are part of most rating systems. a comparison of the performance against a criterion or number of criteria is typically used in the assessment process. however, the distribution of points and weights across the different areas and criteria of the rating system becomes a critical issue in the development process (trusty, 2008). criteria take the sdi concept (in a rating system context) a step further by allocating weight through a quantitative multi-criteria analysis (mca). each rating system allocates weight to each criteria and category using specific methodology to then obtain a weighted summation (e.g., final score) by the addition of every criterion’s weight if the project or task has met a pre-established requirement. a company and/or project is categorized, certified, or acknowledged as sustainable based on the number of points or parameters accomplished in a pre-determined rating scale. therefore, stakeholder engagement and participation is essential not only during the implementation of the rating system, but also in the development phase of the assessment tool (i.e., the criteria weighting process), as it translates into efficient decision-making and sustainability assessment processes. stakeholder participation increases the credibility factor and facilitates implementation and penetration into the market. weighting the categories and criteria requires considering the application of multi-criteria decision methods (mcdm). stakeholders are faced with the challenge of evaluating the relevance of distinctive categories (e.g., management, water, materials, and air) and social, economic, and environmental criteria. ijahp article: poveda, lipsett/weighting sustainable development indicators (sdis) for surface mining operations using the analytical hierarchy process international journal of the analytic hierarchy process 203 vol. 5 issue 2 2013 issn 1936-6744 2. development, usage and weighting of sustainable development indicators (sdis) in 1987, the brundtland commission—formally known as the world commission on environment and development (wced)—changed the way industry does business by introducing a formal definition of sustainability. since then, the international community—including governments, scientists, politicians, sociologists, engineers, and economists—has come together in an effort to link the plans, policies, and programs (ppp) of sustainable development at a macro level, with the goals and objectives at the organizational and project levels. the development and implementation of sdis have contributed to close the gap; however, the identification and measurement of sdis is in permanent evolution. the united nations (un) describes the functions of sdis as leading to better decisions and more effective actions by simplifying, clarifying, and making aggregated information available to policy-makers (united nations [un], 2007). key performance indicators (kpis) for sustainability known as sdis have largely been used to demonstrate the performance of implemented ppp in a diverse range of organizations and industries. the sdis or kpis for sustainability evaluate social, economic, and environmental performance of projects and/or organizations. in 1992, agenda 21 was adopted after the united nations conference on sustainable development and environment to guide programs and actions designed to achieve environmentally sound and sustainable development (essd) at global, regional, and local levels (harger & meyer, 1996). therefore, measuring and assessing the results of implementing essd indicators (e.g., sdis or kpis for sustainability) has become relevant to define the effectiveness of the ppp. moreover, benchmarking performance requires the development of metrics and the definition of a scale against which results can be measured, verified, compared, and correlated. however, no benchmarking process can take place unless a common set of sdis are used to measure the sustainability of similar projects and/or organizations within an industry sector. consequently, the design and development of sdis, including the definition of the final assessment set of indicators and their metrics, are activities in which success is measured by the effective engagement and participation of the different stakeholders. an sdi measures the performance of a specific subject, and is not to be used in isolation when assessing sustainability as a whole due to its multi-disciplinary nature. therefore, different sdis are developed not only representing the different facets of sustainability, (e.g., social, economic, and environmental) but also addressing the different stakeholders’ needs of an explicit organization, project, or industry sector. at the macro level, benchmarking performance and progress of developing and developed countries, and comparing the status of whole countries in terms of a specific aspect, are two areas of proven usefulness of sdis’ implementation in addition to measuring the effectiveness of ppp. at the organizational and project level, the linkage with macro-level goals and objectives represents a major obstacle. additionally, sdis development faces two major hurdles that are still under international debate among scientists: which indicators should be included in the assessment of sustainability (i.e., what should be measured?), and how those indicators should be measured (i.e., which metrics are to be used?). since the set of ijahp article: poveda, lipsett/weighting sustainable development indicators (sdis) for surface mining operations using the analytical hierarchy process international journal of the analytic hierarchy process 204 vol. 5 issue 2 2013 issn 1936-6744 sdis to be included in the assessment is proven to define the success of the process, guidelines and considerations for assisting with the design of sdis have been developed (gibson, hassan, holtz, tansey, & whitelaw, 2010; harger & meyer, 1996; hart, 1999; international institute of sustainable development [iisd], 2012; taylor, 2006; united nations [un], 2007). furthermore, simplification and practicability are the main reasons behind the appeal for the design and use of a sole indicator (i.e., a composite indicator [ci]) to assess sustainability (gasparatos, el-haram, & horner, 2008). however, data aggregation into a sole indicator implies compensability and substitutability between criteria (munda & nardo, 2005). even though these disadvantages are hardly compatible with the vision of sustainability (gasparatos et al. 2008; neumayer, 2003), multi-criteria decision methods (mcdm) allow an alternative and viable perspective to aggregate the criteria into a ci using techniques such as electre (figueira, greco, roy, & slowinski, 2010). 3. the wa-pa-su rating system: structure and sdis for surface mining operations the applicability of the ahp, a multi-criteria decision-making method, is demonstrated in the development of the wa-pa-su project sustainability rating system—a verification process to assist demonstrating compliance in sustainable development performance during project life cycle through the implementation of enhanced strategies to mitigate environmental, social, health, and economic impacts (poveda & lipsett, 2011a,b). the ahp is a fundamental pillar in an integrated approach for a new methodology for sustainability assessment for long-term projects. the wa-pa-su project sustainability rating system assessment methodology is integrated for three distinct areas of knowledge. these areas are (1) sustainable development theory and fundamentals are the basis for the development of the rating system, since the aim is to find a balanced path to the social, economic, and environmental needs; (2) the multi-criteria decision-making analysis (mdma) allows for the engagement and participation of stakeholders during the decision-making process of the design and implementation of the criteria weighting system; and (3) the continual performance improvement immersed in the assessment methodology assists organizations and/or projects in improving performance over time. poveda and lipsett (2011a) describe the necessity for developing a methodology for the assessment of sustainability, which fills the existing gaps in industrial projects with an emphasis in the oil sands developments. the integrated assessment methodology, initially conceived with oil sands projects in mind, evolved into a methodology with characteristics of applicability to other long-term projects. the wa-pa-su project sustainability rating system with application to oil sands operations consists of ten (10) subdivisions, ten (10) areas of excellence within each subdivision, and a number of criteria within each area of excellence (poveda & lipsett, 2011b). aligned with the project’s life cycle, the wa-pa-su project sustainability ratings system contains the subdivisions of project integration; provisional housing/buildings; permanent housing/buildings; roads; oil transportation and storage; mining process; in-situ process; upgrading and refining; shutdown and reclamation; and co2, sox and other greenhouse gas mitigation, capture, and storage. the applicability of the ahp methodology described ijahp article: poveda, lipsett/weighting sustainable development indicators (sdis) for surface mining operations using the analytical hierarchy process international journal of the analytic hierarchy process 205 vol. 5 issue 2 2013 issn 1936-6744 in this manuscript focuses on the surface mining process which, in the case of canadian oil sands projects, occurs for bitumen located within 75 m of the surface. the surface mining sub-division includes the mining itself and other related processes to recover the bitumen by removal of overburden from an oil sands deposit (poveda & lipsett, 2011b). the pre-selected sdis for the surface mining operations in oil sands projects are identified in six (6) potential sources, and grouped in three (3) areas known as group originators of sdis. the group of indicators agreed upon through consensus by public or governmental representatives includes governmental regulations as well as committees and organizations for standardization; academicallyand scientifically-authored resources as well as management and processes best practices grouped into the academic and practitioners identified indicators group; and the organizationally-established indicators group including local, regional, national, and international organizations and surface mining industry standards and programs. table 1 illustrates the pre-selected sdis or kpis for sustainable development in each area of excellence for the surface mining operation in the oil sands projects. the design of the different areas of excellence is based on three distinctive facets of the projects: the resources involved in project development, stakeholder expectations, and potential environmental, economic, and social impact. table 1 pre-selected sdis for surface mining operation in oil sands projects project & environmental management excellence peme site & soil resource excellence ssre strategic environmental assessment (sea) environmental impact assessment (eia) cumulative environmental impact assessment (as per cumulative impact threshold requirements for alberta oil sands) social impact assessment (sia) economic impact assessment (eia) biophysical impact assessment (bia) project lifecycle assessment (pla) environmental protection management plan environmental risk management plan emergency response management plan water management plan solid waste management plan erosion and sediment control plan hazard management plan (includes assessments, inspections and procedures) safety management plan (includes safety training, reporting and prevention of incidents) environmental management systems sustainable public procurement strategies regulatory compliance (approvals, licenses, and permits) independent verified auditing and reporting plans mining effluents 1 : monitoring, control & reduction biological monitoring studies and reports overburden management implementation and monitoring of structures to prevent erosion and soil runoff re-used excavation material proportion of non-previously developed land used proportion of protected land used total waste extracted (non-saleable, including overburden) percentage of resource extracted relative to the total amount of the permitted reserves of that resource tree harvest management deforestation water resource excellence wre atmosphere & air resource excellence aare mining effluents 1 : monitoring, control, & reduction water supply & consumption usage of recycled water & wastewater management ground water resources: protection & monitoring muskeg drainage: monitoring & control control of formation dewatering seepage prevention (from ponds, pits and landfills) construction of water management systems and structures acid drainage: monitoring & control aquatic life protection & monitoring ghgs 2 : monitoring, control, & reduction fugitive emissions: monitoring, control, & reduction dust control noise & vibration management natural & artificial lighting excellence nale energy resource excellence ere luminosity control and regulatory compliance internal production of energy consumed (renewable energy use) consumption of primary energy (natural gas, lpg, petrol, and other fuels) consumption of secondary energy (electricity and heat) ijahp article: poveda, lipsett/weighting sustainable development indicators (sdis) for surface mining operations using the analytical hierarchy process international journal of the analytic hierarchy process 206 vol. 5 issue 2 2013 issn 1936-6744 resources & materials excellence rme innovation in design & operations excellence idoe usage of chemical substances hazardous material management, storage and disposal improvement in machine application efficiency machines material re-use waste management (reduce, reuse and recycle of non-renewable resources) distance of materials suppliers investment in innovation clean technology innovations: testing and implementation of new technologies infrastructure & buildings excellence ibe education, research, & community excellence erce ecological footprint mining location within or proximal to water bodies proximity of mining operations and mining material processing and tailing ponds monitoring and protection of wildlife monitoring and protection of vegetation area of habitat created/destroyed (area disturbed by oil sands development) affected animal and vegetal species monitoring and protection of biodiversity and habitat (includes biological studies and reports) tailings ponds location and impacts study reduction of land area used for tailings ponds operations total area of permitted developments total land area newly opened for extraction activities (including area for overburden storage and tailings) transportation distance of customers, business travel, workforce, and community for fly-in and fly-out operations communication & transportation facilities investment in research workforce awareness training programs (safety, and environmental, social, economic, and health impacts) community awareness programs community and stakeholder consultation and involvement poverty alleviation of affected areas wealth distribution contribution to social development of communities & participation in regional co-operative efforts contribution to economic and institutional development of communities employment, unemployment and underemployment rates contribution to gdp expenditure on environmental protection ethical investment percentage of employees that are stakeholders in the company ratio of lowest wage to national legal minimum health, pension and other benefits and redundancy packages provided to employees as percentage of total employment cost expenditure on health and safety inflation rate internal return ratio environmental liabilities return of investment payback period infrastructure & buildings excellence ibe education, research, & community excellence erce investment in employee training and education lost-time injuries lost-time injuries frequency women/men employment ratio percentage of ethnic minorities employed relative to the total number of employees work satisfaction housing provision for workforce housing development for local communities projects acceptability female-to-male wage ratio net migration rate to projects areas number of direct and indirect employees net employment creation percentage of hours of training employee turnover fatalities at work total number of health and safety complaints from local communities percentage of employees sourced from local communities relative to the total number of employees on-going health monitoring (workers and local communities) health care management/first aid facilities number of local suppliers relative to the total number of suppliers number of local contractors relative to the total number of contractors 1 mining effluents include: arsenic, cooper, cyanide, lead, nickel, zinc, total suspended solids, radium, ph 2 ghgs include: sulphur dioxide (so2), ozone (o3), nitrogen dioxide (no2), particular matter (pm2.5), carbon monoxide (co), oxides of nitrogen (nox), volatile organic compounds (vocs), hydrogen sulphide (h2s). 4. multi-criteria decision-making methods and the analytical hierarchy process the multi-criteria decision analysis and methods (mcda, mcdm) have evolved rapidly, and their applicability has been proven in a variety of areas, including education, transport, economy and finance, supply chain, wastewater and urban sanitation, and ijahp article: poveda, lipsett/weighting sustainable development indicators (sdis) for surface mining operations using the analytical hierarchy process international journal of the analytic hierarchy process 207 vol. 5 issue 2 2013 issn 1936-6744 ecology. today, thousands of manuscripts and dozens of books have been devoted to this area of knowledge, and this section presents a brief description of the existing mcdm and the context for the ahp in the mcda environment. structuring and solving decision and planning problems with multiple criteria is the focus of mcdm and mcda studies and research. mcdm problems can be divided into three categories: problems of multi-criteria choice, problems of multi-criteria ranking, and problems of multi-criteria sorting (vassilev, genova, & vassileva, 2005). independent of the problem or set of problems to solve, there is an additional component that defines the success of the decision-making process. the decision maker (dm) provides additional information in order to select the preferred alternative(s), and provides input based on his/her preferences based on the goals sought to accomplish. with the aim of providing the most feasible solution, several methods have been developed to solve multi-criteria problems and these can be grouped in three distinctive classes. the first class is the multiattribute utility theory (maut) method which gives the decision-maker (dm) the ability to quantify the desirability of a series of alternatives in which a certain level of uncertainty and risk are considered. the ahp weighting method (saaty, 1994) and its most recent extension, the anp (analytic network process); the uta method (beuthe & scannella, 2001); the value tradeoff method (keeney & raiffa, 1993); the direct weighting method (von winterfeldt & edwards, 1986); and the macbeth (measuring attractiveness by a categorical based evaluation technique) method (banaecosta & chagas, 2004) are among the most common in the multi-attribute utility theory methods group. the second class is the outranking methods and these have been developed based on the assumption that there is limited comparability among the alternatives, and in most of the outranking methods, it is assumed that the dm is unable to differentiate among the four binary relations (i.e., the indifference i [reflexive and symmetric], the weak preference q [irreflexive and antisymmetric], the strict preference p [irreflexive and antisymmetric], and the incomparability r [irreflexive and symmetric]) used to compare two alternatives. the main examples of this second group are the promethee methods (brans & mareschal, 1994), the electre methods (roy, 1996), and the tactic methods (vansnick, 1986). the promethee methods include promethee i (partial ranking), ii (complete ranking), iii (ranking based on intervals), iv (continuous case), v (mcda including segmentation constrains or mcda under constraints), and vi (representation of the human brain). the electre methods (roy, 1996) include electre i (choice, crisp s relation), is (choice, valued s relation), ii (ranking, crisp s relation), iii (ranking, valued s relation), iv (ranking, valued s relation and no weights on criteria), and tri (sorting, value s relation), in which crisp s means a yes/no relation (either outranks or not) and value s means that a credibility degree for the outranking is computed in the interval [0,1]. the tactic method (vansnick, 1986) is similar to electre i, but yields a global preference relation instead of a choice set. like electre i, the tactic method consists of three main steps: preference modeling, aggregation, and exploitation. the tactic method is fairly close to the (weighted) condorcet method. the third class of the non-classical mcda approaches requires distinguishing between internal and external uncertainties. internal uncertainties relate to dm values and judgments, while external uncertainties refer to imperfect knowledge concerning consequences of actions (figueira, greco, & ehrgott, 2005). figueira et al. ijahp article: poveda, lipsett/weighting sustainable development indicators (sdis) for surface mining operations using the analytical hierarchy process international journal of the analytic hierarchy process 208 vol. 5 issue 2 2013 issn 1936-6744 (2005) describes four broad approaches for dealing with external uncertainties: “multiattribute utility theory and some extensions; stochastic dominance concepts, primarily in the context of pairwise comparisons of alternatives; the use of surrogate risk measures such as additional decision criteria; and the integration of mcda and scenario planning.” additionally, some hybrid methods have been developed. the fuzzy set theory has been used for choice, ranking, and sorting problems in the mcda, taking several different approaches (e.g., fuzzy-promethee). promethee-gaia uses the visual interactive module gaia to provide graphical representation support to the promethee methodology, and procedures such as promethee-gdss (group decision support system) have been developed based on the promethee-gaia to provide additional decision aid to a group of decision-makers. in addition to these three classes of mcdm, another area of consideration in decision-making is the use of systems support or software systems which provide support to researchers and/or practitioners (e.g., dm) in different areas/steps of the decision-making process. vassilev et al. (2005) classified the developed systems supporting the solution of multi-criteria analysis and multi-criteria optimization problems into three groups: commercial, research or teaching, and experimental. the authors also divide the software systems supporting the solution of multi-criteria analysis problems into two classes: software systems with a general purpose and problem-oriented software systems. the ahp was originally developed by saaty (1977, 1980, 1982, 1990), and it is not only flexible, but also one of the most easily-implemented multi-attribute utility theory (maut) methods (anselin, meire, & anselin 1989). the ahp technique describes a problem using a hierarchy, which in its simplest case has three levels, and applies a measurement scale to obtain vectors of normalized weights or priorities using pairwise comparisons. bouyssou, marchant, pirlot, tsoukias, and vincke (2006) describe the main characteristic of the ahp method; “the evaluation model is structured in a hierarchical way, the same assessment technique is used at each node of the hierarchy, and the assessment of the “children” nodes of a common “parent” node is based on pairwise comparisons”. the top-level node in the hierarchy represents the main objective of the dm, and is the result of the aggregation of the analysis of the alternatives in the second level node. as there are alternatives in each node and nodes can split as many times as there are alternatives, the number of levels in the hierarchy depends on the initial analysis of the problem and how the decision problem has been structured. saaty (2008) describes the organized way for generating priorities in four steps as follows: (1) problem definition and knowledge sought, (2) structure the decision hierarchy in which the top is the goal of the decision then intermediate and lowest levels, (3) build the set of pairwise comparison matrices, then use each element in the upper level to compare the element in the level immediately below with respect to it, and (4) use the priorities from the comparisons to weight the priorities in the level immediately below. do this for every element. then for each element in the level below add its weighted values and obtain its overall or global priority. continue this process of weighting and adding until the final priorities of the alternatives in the bottom-most level are obtained. furthermore, the process of assigning weights or scores to each of the “children” (i.e., alternative) nodes of a “parent” node (except for the bottom nodes) can be summarized as follows: (1) the participants (e.g., dm, client, stakeholders) are asked to compare the alternatives (e.g., ijahp article: poveda, lipsett/weighting sustainable development indicators (sdis) for surface mining operations using the analytical hierarchy process international journal of the analytic hierarchy process 209 vol. 5 issue 2 2013 issn 1936-6744 criteria, indicators) in a pairwise comparison in terms of their relative importance and using a conventional semantic scale; (2) the qualitative assessments given by the participants are quantified (i.e., quantitative interpretation), resulting in an n x n pairwise comparison matrix; and (3) using the pairwise comparison matrix, a score or weight wi is obtained to then be computed as the eigenvector corresponding to the maximum eigenvalue of the matrix, and they are normalized to add up to 1. 5. setting the weighting process, sdis ranking, and the decisionmakers the ahp methodology assists scientists and practitioners in the decision-making process of weighting a series of criteria that are, for the most part, implicitly subjective. the assessment of sustainability implies the involvement of social, economic, and environmental aspects as minimum requirements mandated by the triple bottom line. however, other scholars include additional areas such as policy, culture, and values, while others combine two or more pillars of sustainability using multi-facet or multiattribute indicators (e.g., socio-economic indicators). although some areas of sustainability are fairly well-developed and understood (e.g., environmental), others are still in the infant stage (e.g., social) and, at this point, involve a great degree of subjectivity (poveda & lipsett 2013a). even though the graphic representation of sustainability in which three equally-sized circles intersect each other implies the balance and equality of the pillars, the indicators within each pillar are to be proportionally weighted. since the number of indicators and the areas of assessment vary, a preliminary classification of the indicators is recommended. the process can group the indicators using the different pillars of sustainability, areas of a project, pre-determined areas of excellence, or any other classification, with the condition that stakeholders are preliminarily debriefed, as they need to understand what brings those indicators together (i.e., characteristics commonality). to demonstrate the applicability of the ahp methodology in the weighting of sdis for surface mining operations, the sdis have been classified in ten (10) different areas of excellence. these areas address the different aspects of surface mining operations that not only concern the various stakeholders but also align with the fundamentals and theory of sustainability. additionally, the weighting process mandates the prompt and effective engagement and involvement of the stakeholders that are directly impacted or impact the functionality and/or development of an organization or project. the number of indicators in each pillar and the identification and classification of stakeholders are two areas in which scholars, scientists, and practitioners have not reached common ground. however, stakeholders are recognized as critical components in the success of the decision-making and sustainability assessment processes. surface mining projects are unique in many ways. not only are impacts on the environment rapidly noted by local communities, but economic benefits are also tangible on local and national levels. therefore, stakeholders become rapidly knowledgeable regarding how the projects directly affect them. even though the identification and classification of stakeholders is still an area for development, experience and the ijahp article: poveda, lipsett/weighting sustainable development indicators (sdis) for surface mining operations using the analytical hierarchy process international journal of the analytic hierarchy process 210 vol. 5 issue 2 2013 issn 1936-6744 “learning-as-you-go” process have resulted in the identification of a number of stakeholders for the surface mining projects for the canadian oil sands operations/projects. owner companies, epc (engineering, procurement, and construction) companies, contractors, suppliers, logistics providers, government/regulators, local communities, local business, aboriginal communities, ngos (non-governmental organizations), scientists and researchers, media (television, press, radio), industry and community associations, and financiers are some of those interested parties that may be actively or passively engaged in the development of the projects. development does not imply the approval of the projects or giving the social license to operate. the canadian oil sands are a good example of surface mining operations due to (1) the large reserves or resources exploited, (2) the comparatively stringent set of regulations, and (3) the large number of stakeholders engaged in the process, among other valid reasons. 6. the hierarchy in the ahp, the relative value of surface mining operations’ sustainability is viewed as the main objective, which is obtained by way of a combination of a number of criteria (i.e., areas of excellence), each with their own relative importance, relevance, weight, or priority with respect to their influence to the overall objective. these three levels are linked together in a hierarchical structure, as shown in figure 1, where the top level is the objective and the next level consists of the different criteria (i.e., areas of excellence). in our application of the ahp methodology, we consider ten (10) areas of excellence: project & environmental management excellence (peme); site & soil resource excellence (ssre); water resource excellence (wre); atmosphere & air resource excellence (aare); natural & artificial lighting excellence (nale); energy resource excellence (ere); resources & materials excellence (rme); innovation in design & operations excellence (idoe); infrastructure & buildings excellence (ibe); and education, research, & community excellence (erce). additional criteria can be considered in other sustainability assessment rating systems, which must be conceptualized during the development phase of the assessment tool, with the aim of having a level of consistency in order to benchmark performance between projects and/or organizations. poveda and lipsett (2011b) explain each criterion (i.e., area of excellence), and that the main objective for each of them is to apply fundamentals and principles, as well as the latest advances and technologies, with the aim of targeting a level of excellence in performance. additionally, the criteria (i.e., areas of excellence) take three aspects into consideration: resources involved in project development; stakeholders’ expectations; and potential environmental, economic, social, health, and other impacts. the next level in the hierarchy materializes once each criterion (i.e., area of excellence) is considered as a cluster, to which a certain number of indicators contribute. the number of indicators may vary in each criterion, and each one of the indicators has its own weight, relevance, importance, or priority with respect to the particular criterion (i.e., area of excellence). in our application, the number of indicators in each criterion varies. those indicators reflect the different pillars of sustainability (i.e., social, economic, and environmental) or can be the combination of two or three of the pillars, which are being ijahp article: poveda, lipsett/weighting sustainable development indicators (sdis) for surface mining operations using the analytical hierarchy process international journal of the analytic hierarchy process 211 vol. 5 issue 2 2013 issn 1936-6744 called multi-facet or multi-attribute indicators. additionally, the classification of indicators considers when and where a set of activities occurs within the surface mining operations (poveda & lipsett, 2011b). 7. measurement scale the fundamentals of the measurement scale utilized in the ahp method have not changed since the methodology was introduced by thomas l. saaty in the 1970s (saaty, 1977). however, a comparison of the different tables presenting the measurement scale notes slight modifications of how the scale is interpreted, and/or conceptual additions that have been introduced and observed in different publications throughout the years (saaty 1977, 1980, 1982, 1990, 1994, 2008). though those differences may be semantic interpretations, the stakeholders must be presented with a consistent and clear measurement scale with the aim of obtaining optimum results. in the application of the ahp methodology in the weighting process of sustainability indicators for surface mining operations, the measurement scale used is represented in table 2. while the measurement scale adopted for this application considers the principles of the ahp methodology, the information presented considers the different measurement scales introduced throughout the years. furthermore, the measurement scale illustrates a descriptive and detailed compilation of how the information must be presented to the decision-makers (i.e., stakeholders) during the process of weighting the indicators. the measurement scale developed and detailed by saaty throughout the years addresses the hierarchical structure of the problem by assisting decision-makers in setting the weights or priorities for each criteria and indicators. it reflects the relative strength of each element at a level in the hierarchy with respect to other elements considered in the weighting process at different levels and between each other. in our application, the weights or priorities of criteria (i.e., areas of excellence) and indicators (i.e., sustainable development indicators [social, economic, environmental, and multi-attribute/facet]) are calculated to then be integrated in the calculations for sustainability assessment developed in the wa-pa-su project sustainability rating system (poveda & lipsett, 2011a,b). this serves as an integrated approach for sustainable development of long-term projects (i.e., projects having a life cycle that exceeds a 2-year period [which includes only the execution phase] from start to finish [e.g., mining, industrial, oil & gas, energy]). the measurement scale consists of nine points. anselin et al. (1989) indicate that nine points are chosen because psychologists have concluded that nine objects are the most that an individual can simultaneously compare and consistently rank. the scale ranges from 1, which indicates an equal importance between elements to 9, which refers to an absolute importance of one element over another. additionally, the pair values of 2, 4, 6, and 8 indicate intermediate values between two adjacent judgments, and some compromise is needed. the construction of pairwise matrices and their values within are assisted by the measurement scale which indicates the level of strength or dominance that an indicator or criterion has over others when they are compared pairwise. consequently, sets of section b section a objective: surface mining operations partial assessment: weighting of criteria of sdis a component of an integrated assessment of sustainability (e.g., surface mining operations). p r o je c t & e n v ir o n m e n ta l m a n a g e m e n t e x c e ll e n c e p e m e s it e & s o il r e so u r c e e x c e ll e n c e s s r e w a te r r e so u r c e e x c e ll e n c e w r e a tm o sp h e r e & a ir r e so u r c e e x c e ll e n c e a a r e n a tu r a l & a r ti fi c ia l l ig h ti n g e x c e ll e n c e n a l e e n e r g y r e so u r c e e x c e ll e n c e e r e r e so u r c e s & m a te r ia ls e x c e ll e n c e – r m e in n o v a ti o n i n d e si g n & o p e r a ti o n s e x c e ll e n c e – id o e in fr a st r u c tu r e & b u il d in g s e x c e ll e n c e i b e e d u c a ti o n , r e se a r c h & c o m m u n it y e x c e ll e n c e e r c e criteria: areas of excellence a1 a2 a3 c1 c2 c3 p1 p2 p3 z1 z2 z3 m1 m2 m3 o1 o2 o3 s1 s2 s3 f1 f2 f3 x1 x2 x3 l1 l2 l3 indicators: social, economic, environmental, and multi-attribute/facet figure 1. hierarchy structure of the evaluation for sustainability of surface mining operations (section a) or the overall oil sands projects (section b) which may include two or more sub-divisions. the ahp method is used as partial assessment in the weighting of criteria of sdis as a component of an integrated assessment of sustainability in the wa-pa-su project sustainability rating system (poveda and lipsett 2011a, b). the wa-pa-su project sustainability rating system 2 or more sub-divisions of the wa-pa-su project sustainability rating system are included in the weighting process. ijahp article: poveda, lipsett/weighting sustainable development indicators (sdis) for surface mining operations using the analytical hierarchy process international journal of the analytic hierarchy process 213 vol. 5 issue 2 2013 issn 1936-6744 pairwise comparisons are the result of simultaneous rankings broken down. consistency in the use of the measurement scale is required within the same pairwise comparison matrix and among different matrices in the event the study requires more than one matrix. however, the construction of a matrix of pairwise comparisons does not impose strong requirements of consistency (anselin et al., 1989). table 2 the fundamental scale according to saaty (1977, 1980, 1982, 1990, 1994, 2008) intensity of importance definition explanation 1 equal importance two activities contribute equally to the objective7 2 weak or slight 3 moderate importance of one over another experience and judgment slightly favour one activity over another 4 moderate plus 5 essential or strong importance experience and judgment strongly favour one activity over another 6 strong plus 7 very strong or demonstrated importance an activity is favoured very strongly over another; its dominance is demonstrated in practice 8 very, very strong 9 extreme importance the evidence favouring one activity over another is of the highest possible order of affirmation where; 2, 4, 6, 8 intermediate values between the two adjacent judgments when compromise is needed 1.1–1.9 if the activities are very close may be difficult to assign the best value but when compared with other contrasting activities the size of the small numbers would not be too noticeable, yet they can still indicate the relative importance of the activities reciprocals if activity i has one of the above non-zero numbers assigned to it when compared with activity j, then j has the reciprocal value when compared with i a reasonable assumption rationals ratios arising from the scale if consistency were to be forced by obtaining n numerical values to span the matrix 8. pairwise comparison matrices pairwise comparison matrices are used to determine the relative importance of a series of elements in terms of each criterion. when an element is compared with itself, the value of the weight becomes 1. the structure followed in this paper consists of a number of elements, m, and a series of criteria, n. n criteria are the same elements m, when forming the pairwise comparison matrix certain element m becomes a n criteria (e.g. m1 = n1). since elements can be evaluated in terms of the different criteria, the relative importance or weight of each element can be calculated as well. in the pairwise comparison matrices, aij represents the relative importance or weight of an element over a criteria where, i=1,2,3,……m and j=1,2,3,…..n. therefore, the core of the typical problem to be solved using the ahp methodology to weight the alternatives (criteria) can be represented by the following pairwise comparison matrix: ijahp article: poveda, lipsett/weighting sustainable development indicators (sdis) for surface mining operations using the analytical hierarchy process international journal of the analytic hierarchy process 214 vol. 5 issue 2 2013 issn 1936-6744 alternative/criteria alternative absolute weights criteria m1 m2 m3 m4 …….. mm n1 a11 a12 a13 a14 …….. a1m w1 n2 a21 a22 a23 a24 …….. a2m w2 n3 a31 a32 a33 a34 …….. a3m w3 n4 a41 a42 a43 a44 …….. a4m w4 . . . . . . . . . . . . . . . . . . . . . nn an1 an2 an3 an4 anm wnm where; m1 = n1, a11 = wn1/wm1 w1 = wn1 = wm1 m2 = n2, a21 = wn2/wm1 w2 = wn2 = wm2 m3 = n3, a31 = wn3/wm1 w3 = wn3 = wm3 m4 = n4, a41 = wn4/wm1 w4 = wn4 = wm4 . . . . . . . . . . . . . . . . . . . . . mm = nn a21 = wn2/wm1, etc wnm = wnn = wmm the first pairwise comparison compares the different criteria (i.e., areas of excellence) in a 10 x 10 matrix which includes the following elements: project & environmental management excellence (peme); site & soil resource excellence (ssre); water resource excellence (wre); atmosphere & air resource excellence (aare); natural & artificial lighting excellence (nale); energy resource excellence (ere); resources & materials excellence (rme); innovation in design & operations excellence (idoe); infrastructure & buildings excellence (ibe); and education, research, & community excellence (erce). in the assessment process (pairwise comparison), the decision-maker is free to evaluate the relative importance of each alternative/criterion over others. finding the largest eigenvalue and associated eigenvector, the absolute value of each weight can be calculated from the relative pairwise weights. in detail, if n criteria have known relative weights/importance of w1, w2, …wn, then the comparison of the relative importance of criterion i to criterion j gives a value of n(i, j) = m (i, j) = wi / wj for the element (i, j) in the pairwise comparison matrix n or m (m = n but m is called alternative and n criteria when forming the pairwise comparison matrices). additionally, alternative/criteria n(j, i) = m (j, i) = wj / wi which justified the use of reciprocals in table 2. to build the matrix, the alternative and criteria are compared pairwise to then estimate the weight attached to each alternative/criteria using the eigenvector associated with the largest eigenvalue. in this application of the ahp method, there is no pre-established consistency or mathematical sense in implying that n(i, j) x n (j, k) = n(i, k) or that an alternative/criterion follows a semantic relationship with its degree of importance. therefore, alternative/criterion i is not more important than j, and neither is alternative/criteria j higher than k, or i ranked lower than k. as the value for inconsistency increases, it is expected to find a greater eigenvalue (above n). therefore, the pairwise comparisons have a poorer representation by the eigenvector. finally, the values for w1, w2, …wn, can be found by calculating the geometric mean of each matrix row and then ijahp article: poveda, lipsett/weighting sustainable development indicators (sdis) for surface mining operations using the analytical hierarchy process international journal of the analytic hierarchy process 215 vol. 5 issue 2 2013 issn 1936-6744 normalizing by dividing each number by its total. these represent the corresponding value of importance given to each alternative/criterion. the second set of pairwise comparison is integrated with alternatives/criteria at the 3 rd level. in figure 1, the third level consists of the indicators in each criterion (i.e., area of excellence). the number of alternatives/criteria in each pairwise comparison matrix varies as follows: peme with 19 sdis, ssre with 11 sdis, wre with 11 sdis, aare with 4 sdis, nale with 1 sdi, ere with 3 sdis, rme with 6 sdis, idoe with 2 sdis, ibe with 14 sdis, and erce with 44 sdis. the identification, pre-selection, and classification methodology of sdis for surface mining operations was assisted by six different sources grouped in three areas: indicators agreed upon by public or governmental representatives through consensus, indicators identified by academics and practitioners, and indicators established by organizations. although the assessment of sustainability and sdis are still areas in an infant stage, the measurement methodology of criteria for surface mining operations was developed based on the continual performance improvement (cpi) methodology (poveda & lipsett, 2013b). the weighting of sdis can be assisted by using a variety of approaches including the ahp methodology used in this application. therefore, the weighting of the alternative/criteria in each pairwise comparison matrix follows the same parameters used in the 10 x 10 matrix to weight the criteria (i.e., areas of excellence) at level two (node two) in the hierarchy, with the aim of consistency in the weighting process of each alternative/criteria in each level (node) of the system (hierarchy). each pairwise comparison matrix at level three (indicators [i.e., social, economic, environmental, and multi-attribute/facet]) is an independent sub-system. the final weight of the each indicator is impacted by the results of what integrates the 10 x 10 pairwise comparison matrix at level two (node two) in which the criteria (i.e., areas of excellence) have been weighted; therefore, the level of relevance or importance to each sub-system (pairwise comparison in level three [indicators level]) must be calculated considering the weight of each criteria (area of excellence). 9. expected results and contributions the expected results can be presented in the two scenarios represented in figure 1: (1) partial assessment for the overall sustainability performance of the oil sands projects, in which the weighting of criteria of sdis for surface mining operations is a component for the assessment of the project (section a); and (2) overall assessment for sustainability performance of the oil sands projects in which ten (10) sub-divisions represented a component for the assessment of the projects (section b). in figure 2, the same hierarchy structure as in figure 1 sections a & b is presented, but with the respective criteria and indicators showing the priority weights. to obtain the resulting overall weight for each indicator (sdis) in case a of figure 2 (surface mining operations as an isolated system in the overall sustainability assessment of the oil sands projects), the priority weights have to be multiplied by the weight of the respective criterion (i.e., area of excellence). for example, sdi1’= sdi1 x sdi = 0.175 x 0.325 = 0.056 ijahp article: poveda, lipsett/weighting sustainable development indicators (sdis) for surface mining operations using the analytical hierarchy process international journal of the analytic hierarchy process 216 vol. 5 issue 2 2013 issn 1936-6744 the overall weights must sum to the respective weight of each indicator as noted in figure 2 for the examples illustrated (e.g., sdi = 0.056 + 0.080 + 0.142 + 0.047 = 0.325), while the sum of weights of all indicators must sum to the unit (one [1]) (e.g., objective [surface mining operations] = 0.115 + 0.051 + 0.145 + 0.325 + 0.055 +0.105 + 0.085 + 0.025 + 0.038 + 0.056 = 1). similarly, the calculations can be done in case b of figure 2 (surface mining operations as one of the ten (10) sub-divisions included in the wa-pa-su project sustainability rating system to measure the sustainability of the oil sands projects) (poveda & lipsett 2011a,b). the priority weights also have to be multiplied by the weight of the respective criterion (i.e., area of excellence). however, since the surface mining operation is another sub-division in the system (objectives), an additional step must be included to calculate the weight of each sub-division to then be multiplied by the weight of the respective criterion (i.e., area of excellence). therefore, the weight of a particular indicator with reference to the overall system can be calculated as: sdi1’ = sdi1 x sdi x sdi objective = 0.175 x 0.325 x 0.345 = 0.020 in case b of figure 2, the overall weight of the objective must sum to the unit (one [1]), while the overall weights of the criteria must add to the weight of a particular objective, and the overall weight of the indicators must sum to the respective total of the multiplication of the weight of the objective by the weight of the indicators (e.g., sdi = 0.020 + 0.027 + 0.049 + 0.016 = 0.112 in which 0.112 = 0.345 x 0.325). previously, in order to submit the different sdis to a weighting process supported by a multi-criteria decision-making (mcdm) methodology, the critical task in sustainability assessment has referred to the identification and design of metrics, which assists decision-makers in addressing the questions of what to measure and how to measure the sdis, respectively. moreover, decision-makers (stakeholders) are faced with a costbenefit paradigm of implementing a series of sdis to demonstrate a certain level of sustainability performance while addressing the stakeholders’ needs. despite the fact that there is a series of beneficial factors behind the applicability and usefulness of sdis, there are also certain costs to be considered (poveda & lipsett, 2013a). the next query(s) in the decision-maker’s mind involves the level of relevance or importance of each sdi. the application of the ahp method assists in addressing questions such as (1) should all the sdis be weighted equally? (2) should the sdis user expend the same level of resources for each indicator to address the impact (social, economic, environmental) that they represent?, and (3) is each indicator equally important for each group of stakeholders? although finding universally-accepted responses is not the main aim in the application of the ahp method, the different groups of stakeholders have an opportunity to be heard and express their individual needs through an effective engagement and participatory process that leads to the assessment of the weight of each of the indicators (sdis), criteria (i.e., areas of excellence), and/or objectives (i.e., sub-divisions). furthermore, the ahp method, like other multi-criteria decision-making methodologies, helps the decision-makers face the complex problem of evaluating multiple conflicting and subjective sdis. ijahp article: poveda, lipsett/weighting sustainable development indicators (sdis) for surface mining operations using the analytical hierarchy process international journal of the analytic hierarchy process 217 vol. 5 issue 2 2013 issn 1936-6744 a b figure 2. hierarchical structure of the evaluation process of the two hypothetical applications of the ahp methodology to weight sdis to measure the surface mining operation and oil sands projects sustainability decision-makers face the challenge of multiple choices in their routine operations or among their list of activities. therefore, they usually prefer simplistic, rapid, and ijahp article: poveda, lipsett/weighting sustainable development indicators (sdis) for surface mining operations using the analytical hierarchy process international journal of the analytic hierarchy process 218 vol. 5 issue 2 2013 issn 1936-6744 applicable methodologies to find answers to their queries. the ahp methodology is similar to using the common sense decision-making approach. consequently, decisionmakers easily understand the approach applied in this methodology. simplicity in the sdis assessment becomes a strategic element from the stakeholder’s standpoint. in surface mining operations, the different stakeholders vary with respect to their level of education, experience, and seniority level (management position), among other impacting factors in the assessment process. additionally, the results (weights) of applying the ahp method can be easily communicated and understood by the different decision-making groups. bahurmoz (2003) noted that using ahp in group settings leads to better communication, clearer understanding, and consensus among members of the decisionmaking group. therefore, a greater commitment to choosing the alternative is expected. 10. discussion and future research in addition to the various challenges decision-makers encounter during the projects conception, planning, execution, and closing phases, the different stakeholders—who often become decision-makers—are facing the pressure of obtaining the “social license” to operate with the aim of smoothly executing and delivering their projects. different industries are exchanging the well-known mentality of “business as usual” for proactive approaches to address the stakeholders’ needs. implementing more environmentallyfriendly practices has been not enough. therefore, organizations are including social and economic performance indicators to demonstrate their commitment to the triple bottom line often addressed in the fundamentals of sustainable development. the number or selection of sdis for a determined kind of project or industry is still under debate, not only among stakeholders but also within the international scientific community. while selecting specific sdis for a project, organization, or industry seems to be the preference, the main challenge in applying such criteria lies in the area of benchmarking sustainable development performance. surface mining operations and the mining industry encounter similar difficulties when determining how to answer not only questions such as [1] what to measure and [2] how to measure the selected set of sdis, but also in finding the level of importance (weight) of each sdi. the application of the ahp method, the design of its hierarchy, and the development of the pairwise comparisons required in the methodology assume that questions 1 and 2 have been satisfactorily answered and universally accepted. nevertheless, multi-criteria decision-making (mcdm) methodologies may offer new perspectives, not only in the weighting, but also in the selection and design of metrics for sids. while applying the ahp methodology offers a clear representation of the different groups of decision-makers regarding the level of importance of the various sdis, criteria (i.e., areas of excellence), and objectives (i.e., sub-divisions), future research must address the validation of the findings (overall indicators’ weights) and areas such as the level of importance or relevance of the different decision-makers (stakeholders), independency of pairwise comparison matrices, and the influence of sdis among each other. the validation of the findings refers to comparing the values (weights) obtained after applying the ahp methodology with scientific evidence. the weight of an indicator measuring main environmental impacts is expected to be higher than other indicators reflecting have lesser impact, which can be measured through various scientific ijahp article: poveda, lipsett/weighting sustainable development indicators (sdis) for surface mining operations using the analytical hierarchy process international journal of the analytic hierarchy process 219 vol. 5 issue 2 2013 issn 1936-6744 parameters (e.g., ghg emissions, energy consumption). the weight of each sdi is not only determined by decision-makers (stakeholders) based on the fact that they represent the three pillars of sustainability (social, economic, and environmental), other factors should be investigated to calculate the final overall indicators’ weights; the sdis’ weighting should include the weights of each stakeholder group (e.g., is the input of a politician and a small business representative equality weighted?); the decision-maker’s seniority level (e.g., is the input of a ceo and a junior manager equally weighted?); and the decision-maker’s relevance represented in a combination of years of experience, position, and seniority in a determined position (e.g., is the input of a junior project manager with 10 years of experience and a senior superintendent with 30 years of experience equally weighted?). finally, pairwise comparison matrices and sdis have been treated as independent bodies and the outcomes have been read as such. future research should question such independency and/or find the interconnection between the different matrices and among the various sdis in each matrix. for example, an indicator representing the water resources excellence (wre) area of excellence may be closely linked to another indicator representing the energy resource excellence (ere) area of excellence. understanding such dynamism may result in addressing the subjectivity often encountered among sdis and the metrics used to measure them. ijahp article: poveda, lipsett/weighting sustainable development indicators (sdis) for surface mining operations using the analytical hierarchy process international journal of the analytic hierarchy process 220 vol. 5 issue 2 2013 issn 1936-6744 references anselin, a., meire, p.m., & anselin, l. (1989). multicriteria techniques in ecological evaluation: an example using the analytical hierarchy process. biological conservation, 49(3), 215-229. bahurmoz, a. (2003). the analytic hierarchy process at dar al-hekmah, saudi arabia. interfaces, 33(4), 70-78. banaecosta, c.a, & chagas, m.p. (2004). career choice problem: an example of how to use macbeth to build a quantitative value model based on qualitative value judgments. european journal of operational research, 153(2), 323-331. beuthe, m., & scannella, g. (2001). comparative analysis of uta multicriteria methods. european journal of operational research, 130(2), 246-262. bouyssou, d., marchant, t., pirlot, m., tsoukias, a., & vincke, p. (2006). evaluation and decision models with multiple criteria: stepping stones for the analyst. boston: springer. brans, j., & mareschal, b. (1994). the promcalc & gaia decision support system for multicriteria decision aid. decision support system, 12, 297-310. figueira, j.r., greco, s., & ehrgott, m. (2005). multiple criteria decision analysis: state of the art surveys. boston/dordrecht/london: kluwer academic publishers. figueira, j.r., greco, s., roy, b., & slowinski, r. (2010). electre methods: main features and recent developments. in: zopounidis, c. & parpalos, p. (eds). handbook of multicriteria analysis. chapter 3, 51-89. new york: springer. gasparatos, a., el-haram, m., & horner, m. (2008). a critical review of reductionist approaches for assessing the progress towards sustainability. env. imp. asses, 28(4/5), 286-311. gibson, r. b., hassan, s., holtz, s., tansey, j., & whitelaw, g. (2010). sustainability assessment: criteria and processes. london, uk: earthscan. harger, j.r.e., & meyer, f.m. (1996). definition of indicators for environmentally sustainable development. chemosphere, 33(9), 1749-1175. hart, m. (1999). guide to sustainable community indicators. north andover, ma: hart environmental data. iisd (international institute of sustainable development). (2012). standards-csr guidance. retrieved march 23, 2012, from http://iisd.org/standards/csr.asp. ijahp article: poveda, lipsett/weighting sustainable development indicators (sdis) for surface mining operations using the analytical hierarchy process international journal of the analytic hierarchy process 221 vol. 5 issue 2 2013 issn 1936-6744 issa, m.h., rankin, j.h., & christian, a.j. (2009). a methodology to assess the costs and financial benefits of green buildings from an industry perspective. proceedings of the canadian society for civil engineering, montreal, qc, 2, 1111-1120. keeney, r.l., & raiffa, h. (1993) decisions with multiple objectives, preferences and value tradeoffs. cambridge, uk: cambridge university press. munda, g. (2006). a naiade based approach for sustainability benchmarking. int. j. of environmental technology and management, 6(1/2), 65-78. munda, g., & nardo, m. (2005). constructing consistent composite indicators: the issue of weights. luxembourg: european communities. neumayer, e. (2003). weak versus strong sustainability: exploring the limits of two opposing paradigms. cheltenham, uk: edward elgar. poveda, c., & lipsett, m. (2011a). a review of sustainability assessment and sustainability/environmental rating system and credit weighting tools. journal of sustainable development, 4(6), 36-55. poveda, c., & lipsett, m. (2011b). a rating system for sustainability of industrial projects with application in oil sands and heavy oil projects: areas of excellence, subdivisions, and management interaction. journal of sustainable development, 4(4), 3-21. poveda, c., & lipsett, m. (2013a). using sustainable development indicators (sdis) for sustainability assessment of surface mining operations in the oil sands projects: applicability, usefulness and costs. energy and sustainability 2013 conference proceedings, bucharest, romania, 55-67. poveda, c., & lipsett, m. (2013b). design of performance improvement factors (pifs) for sustainable development indicators (sdis) metrics for oil sands projects with application to surface mining operations based on continual performance improvement (cpi). journal of sustainable development, 6(8), 52-70. roy, b. (1996). multicriteria methodology for decision aiding. dordrecht, the netherlands: kluwer academic publishers. saaty, t. (1977). a scaling method for priorities in hierarchical structures. journal of mathematical psychology, 15, 234-81. saaty, t. (1980). the analytical hierarchy process. new york: mcgraw hill. saaty, t. (1982). decision making for leaders. belmont: lifetime learning publications. saaty, t. (1990). how to make a decision: the analytic hierarchy process. european journal of operations research, 48, 9-26. ijahp article: poveda, lipsett/weighting sustainable development indicators (sdis) for surface mining operations using the analytical hierarchy process international journal of the analytic hierarchy process 222 vol. 5 issue 2 2013 issn 1936-6744 saaty, t. (1994). highlights and critical points in the theory and application of the analytic hierarchy process. european journal of operational research, 74, 426-447. saaty, t. (2008). decision making with the analytic hierarchy process. int. j. services science, 1(1), 83-98. taylor, a. (2006). sustainability indicator frameworks in alberta: setting the context and identifying opportunities. drayton valley, canada: the pembina institute. trusty, w. (2008). standards versus recommended practice: separating process and prescriptive measures from building performance. journal of astm international, 5(2). united nations (un). (2007). indicators of sustainable development: guidelines and methodologies. retrieved april 12, 2012, from http://www.un.org/esa/sustdev/natlinfo/indicators/guidelines.pdf. vansnick, j. (1986). on the problem of weights in multiple criteria decision making (the noncompensatory approach). european journal of operational research, 24, 288-294. vassilev, v., genova, k., & vassileva, m. (2005). a brief survey of multicriteria decision making methods and software systems. cybernetics and information technologies, 5(1). vonwinterfeldt, d., & edwards, w. (1986). decision analysis and behavioral research. london: cambridge university press. yudelson, j. (2008). the green building revolution. washington, d.c: island press. ijahp article: entani/group interval weights based on conjunction approximation of individual interval weights international journal of the analytic hierarchy process 427 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.277 group interval weights based on conjunction approximation of individual interval weights tomoe entani graduate school of applied informatics university of hyogo kobe, japan e-mail:entani@ai.u-hyogo.ac.jp abstract the individual and group decisions in this study are denoted as the normalized interval weights of alternatives as in interval ahp. it assumes that a decision maker uses crisp values in the interval weights in giving comparisons. the interval weights reflect uncertainty in a decision maker’s mind. then, the group interval weight is obtained as a conjunction approximation of the individual interval weights. for a consensus, the group interval weight is obtained so as to intersect with all the individual interval weights. in other words, the group interval weight has something in common with each individual interval weight. the group decision depends on how much the decision makers are satisfied or dissatisfied with it. the satisfaction of a decision maker is measured by the ranges of the group interval weights which s/he can support. similarly, the decision maker’s dissatisfaction is defined by the ranges which are out of his/her decision. it is better to maximize the satisfaction and simultaneously to minimize the dissatisfaction. however, there is a trade-off between these two objectives. in the proposed model, the importance of the satisfaction or dissatisfaction is given. then, the decision makers find not only the group decision but also their satisfaction and dissatisfaction with it. keywords: group decision making; interval analysis; analytic hierarchy process 1. introduction in the analytic hierarchy process (ahp), the crisp priority weights of alternatives are obtained from the pairwise comparison matrix given by a decision maker (saaty, 1980). the elements of a matrix are crisp values such as 1/3, and 5. the well-known techniques to obtain the weights from the given crisp comparisons are geometric mean and eigenvector methods. the obtained weights are also crisp. the other technique is interval ahp, where the weights are obtained as an interval to reflect the inconsistency among the given crisp comparisons (sugihara & tanaka, 2001; sugihara, ishii, & tanaka, 2004). the interval weights are obtained so as to include the given comparisons as close as possible. the crisp comparisons are extended into interval ones to reflect our uncertain judgments (saaty & vargas, 1987; arbel, 1989). in group ahp, some works handle the interval or fuzzy comparisons, instead of crisp comparisons (dopazo, chouinard & guisse, 2014; xu, 2013). however, this study handles the crisp comparisons and the proposed approach with crisp comparisons can be extended into the interval comparisons on the same principle. ijahp article: entani/group interval weights based on conjunction approximation of individual interval weights international journal of the analytic hierarchy process 428 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.277 group decision making is discussed from the viewpoint of ahp (dyer & forman, 1992; basak & saaty, 1993). there are two approaches to aggregate the individuals into a group by geometric mean and so on (aczel & saaty, 1983; altuzarra, moreno-jimenez, & salvador, 2007; entani & inuiguchi, 2010; forman, peniwati, 1998; yeh & chang, 2009). one approach is to aggregate the individual judgments first and then the group decision is obtained from the aggregated judgments. the other approach is to aggregate the individual decisions which are independently obtained from the individually given judgments. since the latter aggregation can show a decision maker his/her decision, it helps him/her to understand the relationship between his/her decision and the group decision. this study also assumes that the group decision is obtained as the aggregation of individual decisions and discusses how to aggregate them. for a consensus, the group weight is obtained so as to have something in common with each individual weight. the quality of the group decision is measured by the satisfaction or dissatisfaction of the decision makers. some parts of the group interval weights are supported by a decision maker but the other parts are not. therefore, the satisfaction and dissatisfaction of all decision makers are maximized and minimized, respectively. however, they have a tradeoff relationship so that the importance of the satisfaction or dissatisfaction is introduced and the group decision depends on it. 2. individual decisions by interval ahp in ahp, decision maker 𝑘 gives the following pairwise comparison matrix 𝐴𝑘 = [ 1 ⋯ 𝑎𝑘1𝑛 ⋮ 𝑎𝑘𝑖𝑗 ⋮ 𝑎𝑘𝑛1 ⋯ 1 ] , ∀𝑘, (1) where 𝑎𝑘𝑖𝑗 is his/her intuitive judgment on the importance ratio of alternative 𝑖 to that of alternative 𝑗. the comparisons satisfy 𝑎𝑘𝑖𝑖 = 1 and 𝑎𝑘𝑖𝑗 = 1/𝑎𝑘𝑗𝑖 for all 𝑘, 𝑖 and 𝑗. the comparisons are consistent if and only if 𝑎𝑘𝑖𝑗 = 𝑎𝑘𝑖𝑙 𝑎𝑘𝑙𝑗 , ∀𝑖, 𝑗, 𝑙. (2) by the eigenvector method, the weights of alternatives are obtained as the eigenvector corresponding to principal eigenvalue, 𝐴𝑘 𝒘𝑘 = 𝜆𝒘𝑘 , where 𝒘𝑘 = (𝑤𝑘1, … , 𝑤𝑘𝑛) 𝑇 and ∑ 𝑤𝑘𝑖𝑖 = 1. it is noted that the weights, 𝑤𝑘𝑖 , ∀𝑖 are crisp. in interval ahp, it is assumed that the given comparisons are inconsistent since the weights of alternatives are uncertain (sugihara & tanaka, 2001; sugihara et al., 2004). an alternative is compared to the other 𝑛 − 1 alternatives. for example, denoting the weights of alternative 1 in giving comparisons 𝑎𝑘12 and 𝑎𝑘13 as 𝑤𝑘1 2 and 𝑤𝑘1 3 , respectively, they are not always equal, 𝑤𝑘1 2 ≠ 𝑤𝑘1 3 . these weights of alternative 1 depend on which alterative it is compared to. then, the interval weight of alternative 1, 𝑊𝑘1 = [𝑤𝑘1, �̅�𝑘1], includes these weights as 𝑤𝑘1 ≤ 𝑤𝑘1 2 ≤ 𝑤𝑘1 and 𝑤𝑘1 ≤ 𝑤𝑘1 3 ≤ 𝑤𝑘1. in this way, the uncertain weight of an alternative is denoted as an interval. the weight of alternative 𝑖 is denoted as interval 𝑊𝑘𝑖 = [𝑤𝑘𝑖 , 𝑤𝑘𝑖 ], whose width 𝑤𝑘𝑖 − 𝑤𝑘𝑖 represents its uncertainty. in other words, the decision maker 𝑘 uses a real value in the interval weight 𝑊𝑘𝑖 in the given comparison 𝑎𝑘𝑖𝑗, where 𝑗 ≠ 𝑖. the problem to obtain the interval weights is formulated as the following linear programming (lp) problem. ijahp article: entani/group interval weights based on conjunction approximation of individual interval weights international journal of the analytic hierarchy process 429 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.277 min      ∑ ( 𝑖 𝑤𝑘𝑖 − 𝑤𝑘𝑖 ), s. t. ∑ 𝑤𝑘𝑖 𝑖≠𝑗 + 𝑤𝑘𝑗 ≥ 1, ∀𝑗, ∑ 𝑤𝑘𝑖 𝑖≠𝑗 + 𝑤𝑘𝑗 ≤ 1, ∀𝑗, 𝑤𝑘𝑖 𝑤𝑘𝑗 ≤ 𝑎𝑘𝑖𝑗 ≤ 𝑤𝑘𝑖 𝑤𝑘𝑗 , ∀𝑖, 𝑗, 𝑤𝑘𝑖 ≥ 𝜖, ∀𝑖. (3) the first two constraints are for the normalization of intervals based on interval probability (de campos, huete, & moral, 1994; tanaka, sugihara, & maeda, 2004). the redundancy of the intervals to make their sum 1 is excluded. for instance, the 1st inequality for 𝑗 requires 𝑤𝑘𝑗 to not be too small. when the weights are crisp, 𝑤𝑘𝑖 = 𝑤𝑘𝑖 = 𝑤𝑘𝑖 , ∀𝑖, two inequalities are replaced into ∑ 𝑤𝑘𝑖𝑖 = 1. the next inequalities are the inclusion constraints. these inequalities require the obtained interval weights to include the given comparisons as 𝑎𝑘𝑖𝑗 ∈ 𝑊𝑘𝑖 𝑊𝑘𝑗 = [𝑤𝑘𝑖 , 𝑤𝑘𝑖 ] [𝑤𝑘𝑗 , 𝑤𝑘𝑗 ] = [ 𝑤𝑘𝑖 𝑤𝑘𝑗 , 𝑤𝑘𝑖 𝑤𝑘𝑗 ], (4) where the fraction of intervals is defined as its maximum range. in the case of interval comparison, [𝑎𝑘𝑖𝑗 , �̅�𝑘𝑖𝑗 ], it replaces crisp comparison 𝑎𝑘𝑖𝑗 in equation 4. by minimizing the widths of the interval weights, the obtained weights are as close as possible to the given comparisons because of the inclusion constraints equation 4. it is, in other words, minimizing uncertainty of the decision. if the comparisons are consistent as in equation 2, the upper and lower bounds of the interval weights are equal, 𝑤𝑘𝑖 = 𝑤𝑘𝑖 , so that the weights are crisp as 𝑤𝑘𝑖 = 𝑤𝑘𝑖 = 𝑤𝑘𝑖 , ∀𝑖. they are equal to those by the geometric mean and eigenvector methods. the more inconsistent the given comparisons are, the wider the obtained interval weights become. the inconsistency of the given comparisons is measured by the optimal objective function value of equation 3. 3. group decision from individual decisions 3.1 relation between group and individual decisions in the case of a group of 𝑚 decision makers, there are 𝑚 sets of individual interval weights. they are independently obtained from their comparison matrices by equation 3. the group decision is also denoted as interval weights 𝑊𝑖 = [𝑤𝑖 , 𝑤𝑖 ] and they are normalized as ∑ 𝑤𝑖 𝑖≠𝑗 + 𝑤𝑗 ≥ 1, ∀𝑗, ∑ 𝑤𝑖 𝑖≠𝑗 + 𝑤𝑗 ≤ 1, ∀𝑗. (5) ijahp article: entani/group interval weights based on conjunction approximation of individual interval weights international journal of the analytic hierarchy process 430 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.277 then, based on the idea that a decision maker accepts the group decision if it has something in common with his/her decision, the group interval weight should satisfy the following conditions. 𝑤𝑖 ≤ 𝑤𝑘𝑖 , 𝑤𝑘𝑖 ≤ 𝑤𝑖 ∀𝑘 ↔ 𝑤𝑖 ≤ min 𝑘 𝑤𝑘𝑖 , max 𝑘 𝑤𝑘𝑖 ≤ 𝑤𝑖 , (6) where the group decision intersects to all the individual decisions as 𝑊𝑖 ∩ 𝑊𝑘𝑖 ≠ ∅ ∀𝑘. in equation 6, the upper and lower bounds of the group interval weight are approximated by the lower and upper bounds of the individual interval weights, respectively. it is based on the conjunction approximation. it is noted that the group interval weight complements the individual interval weights from the possibility viewpoint. the group decision includes and is included in the individual decisions as the extreme cases of equation 6. assuming that the group decision includes all the individual decisions as 𝑊𝑖 ⊇ 𝑊𝑘𝑖 ∀𝑘, equation 6 is rewritten as 𝑤𝑖 ≤ 𝑤𝑘𝑖 ≤ 𝑤𝑘𝑖 , 𝑤𝑘𝑖 ≤ 𝑤𝑘𝑖 ≤ 𝑤𝑖 , ∀𝑘 ↔ 𝑤𝑖 ≤ 𝑤𝑘𝑖 ≤ 𝑤𝑘𝑖 ≤ 𝑤𝑖 , ∀𝑘 ↔ 𝑤𝑖 = min 𝑘 𝑤𝑘𝑖 , max 𝑘 𝑤𝑘𝑖 = 𝑤𝑖 . (7) such a group decision is based on the upper approximation of the individual decisions. all the possible evaluations in the individual decisions are included in the group decision. the group interval weight whose bounds are 𝑤𝑖 < min𝑘 𝑤𝑘𝑖 and max𝑘 𝑤𝑘𝑖 < 𝑤𝑖 is redundant, since it includes the weight which is out of any decision maker’s interval weight. by equation 6, some evaluations in an individual decision may be ignored compared to the others’ decisions, but the more precise group decision than that is obtained by equation 7. while, based on their lower approximation, the group decision is included in all the individual decisions as 𝑊𝑖 ⊆ 𝑊𝑘𝑖 , ∀𝑘, equation 6 is rewritten as 𝑤𝑖 ≤ 𝑤𝑖 ≤ 𝑤𝑘𝑖 , 𝑤𝑘𝑖 ≤ 𝑤𝑖 ≤ 𝑤𝑖 , ∀𝑘 ↔ 𝑤𝑘𝑖 ≤ 𝑤𝑖 ≤ 𝑤𝑖 ≤ 𝑤𝑘𝑖 , ∀𝑘 ↔ max 𝑘 𝑤𝑘𝑖 = 𝑤𝑖 , 𝑤𝑖 = min 𝑘 𝑤𝑘𝑖 , if  max 𝑘 𝑤𝑘𝑖 ≤ min 𝑘 𝑤𝑘𝑖 . (8) if max𝑘 𝑤𝑘𝑖 > min𝑘 𝑤𝑘𝑖, there is no group weight included in all the individual weights. equations 6 or 7 result in a group decision for any individual decisions, but equation 8 requires the individual decisions have something in common with each other. this study defines the general condition of the group decision by equation 6, and uses equations 7 and 8 as its extreme cases. 3.2 satisfaction and dissatisfaction of decision maker by equation 6 the individual and group decisions intersect each other and the group interval weight has something in common with each individual interval weight. a decision maker is satisfied with the group decision if s/he can support most of it. then, the similarity of the group decision to his/her decision measures his/her satisfaction. while, s/he is not satisfied with the group decision if s/he cannot support some of it. then, the difference of the group decision from his/her decision measures his/her dissatisfaction. in figure 1, an individual interval weight is illustrated as the top line and the possible group weights which satisfy equation 6 are illustrated as the following four lines. his/her ijahp article: entani/group interval weights based on conjunction approximation of individual interval weights international journal of the analytic hierarchy process 431 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.277 satisfaction and dissatisfaction of an alternative are dented as 𝛼 and 𝛽, respectively. the lower two lines show the group weights based on the lower and upper approximations of the individual weights, respectively. figure 1. individual and group decisions decision maker 𝑘 is satisfied with the group weight of alternative 𝑖 with degree 𝛼𝑘𝑖 , which is defined as follows. 𝛼𝑘𝑖 = min { (𝑤𝑖 − 𝑤𝑘𝑖 ), (𝑤𝑘𝑖 − 𝑤𝑖 ), (𝑤𝑖 − 𝑤𝑖 ), (𝑤𝑘𝑖 − 𝑤𝑘𝑖 )}. (9) this range in the group interval weight is included in the individual one. since s/he supports this range, his/her satisfaction is measured as the sum of these ranges of all alternatives, 𝛼𝑘 = ∑ 𝛼𝑘𝑖𝑖 . then, the sum of the satisfaction of all decision makers should be maximized for the better group decision as max ∑ 𝛼𝑘 𝑘 = max ∑ 𝛼𝑘𝑖 𝑘𝑖 . (10) on the other hand, the dissatisfaction degree 𝛽𝑘𝑖 is defined as follows. 𝛽𝑘𝑖 = max { (𝑤𝑘𝑖 − 𝑤𝑖 ), (𝑤𝑖 − 𝑤𝑘𝑖 ),0, (𝑤𝑘𝑖 − 𝑤𝑖 + 𝑤𝑖 − 𝑤𝑘𝑖 )}. (11) this range in the group interval weight is not included in the individual one so that it is not supported by him/her. then, the sum of the dissatisfaction with all alternatives should be minimized as min ∑ 𝛽𝑘 𝑘 = min ∑ 𝛽𝑘𝑖 𝑘𝑖 . (12) as shown in figure 1, the group interval weight of alternative 𝑖 is divided into the ranges satisfying and dissatisfying decision maker 𝑘 as 𝑤𝑖 − 𝑤𝑖 = 𝛼𝑘𝑖 + 𝛽𝑘𝑖 . (13) it is a trade-off between the increase of satisfaction and the decrease of dissatisfaction. when the group interval weight of an alternative is uncertain so that its width is wide, both the satisfaction and dissatisfaction tend to be large. ijahp article: entani/group interval weights based on conjunction approximation of individual interval weights international journal of the analytic hierarchy process 432 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.277 3.3 model to obtain group decision the group decision is denoted as the normalized interval weights as equation 5 and should have something in common with each individual decision as equation 6 for a consensus. under these conditions, the satisfaction and dissatisfaction of the decision makers are maximized and minimized, respectively, as equation 10 with equation 9 and equation 12 with equation 11. then, the problem to obtain a group decision is formulated as follows. max   ∑ 𝛼𝑘𝑖 𝑘𝑖 , min   ∑ 𝛽𝑘𝑖 𝑘𝑖 , s. t. 𝑤𝑖 ≤ 𝑤𝑘𝑖 , ∀𝑖, 𝑘, 𝑤𝑘𝑖 ≤ 𝑤𝑖 , ∀𝑖, 𝑘, 𝑤𝑖 − 𝑤𝑘𝑖 ≥ 𝛼𝑘𝑖 , ∀𝑖, 𝑘, 𝑤𝑘𝑖 − 𝑤𝑖 ≥ 𝛼𝑘𝑖 , ∀𝑖, 𝑘, 𝑤𝑖 − 𝑤𝑖 ≥ 𝛼𝑘𝑖 , ∀𝑖, 𝑘, 𝑤𝑘𝑖 − 𝑤𝑘𝑖 ≥ 𝛼𝑘𝑖 , ∀𝑖, 𝑘, 𝑤𝑘𝑖 − 𝑤𝑖 ≤ 𝛽𝑘𝑖 , ∀𝑖, 𝑘, 𝑤𝑖 − 𝑤𝑘𝑖 ≤ 𝛽𝑘𝑖 , ∀𝑖, 𝑘, 0 ≤ 𝛽𝑘𝑖 , ∀𝑖, 𝑘, 𝑤𝑘𝑖 − 𝑤𝑖 + 𝑤𝑖 − 𝑤𝑘𝑖 ≤ 𝛽𝑘𝑖 , ∀𝑖, 𝑘, ∑ 𝑤𝑖 𝑖≠𝑗 + 𝑤𝑗 ≥ 1, ∀𝑗, ∑ 𝑤𝑖 𝑖≠𝑗 + 𝑤𝑗 ≤ 1, ∀𝑗, 𝜖 ≤ 𝑤𝑖 ≤ 𝑤𝑖 , ∀𝑖, (14) where the variables are the upper and lower bounds of the group interval weight, 𝑤𝑖 and 𝑤𝑖, and the individual satisfaction and dissatisfaction, 𝛼𝑘𝑖 and 𝛽𝑘𝑖. the upper and lower bounds of a group interval weight are approximated by the lower and upper bounds of all the individual interval weights by equation 3. as mentioned in section 2, the conjunction approximation as the general condition of the group decision by equation 6 includes the extreme conditions as the upper and lower approximations by equations 7 and 8, respectively. based on the upper approximation as 𝑊𝑘𝑖 ⊆ 𝑊𝑖 = [min𝑘 𝑤𝑘𝑖 , max𝑘 𝑤𝑘𝑖 ] by equation 7 illustrated as the lowest line in figure 1, decision maker 𝑘 is completely satisfied with the group decision. in this case, his/her satisfaction becomes the maximum, 𝛼𝑘𝑖 = 𝑤𝑘𝑖 − 𝑤𝑘𝑖 which is the width of his/her interval weight. in order for a group interval weight to include all the individual interval weights, the width of the group interval weight tends to be wide, especially when the individual weights are not very similar. a part of the group weight supported by an individual may not be supported by the other individuals. therefore, as a result of maximizing the satisfaction of all individuals, the dissatisfaction is increased. on the other hand, decision maker 𝑘 is not dissatisfied with the group decision when the group ijahp article: entani/group interval weights based on conjunction approximation of individual interval weights international journal of the analytic hierarchy process 433 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.277 decision is based on the lower approximation as 𝑊𝑘𝑖 ⊇ 𝑊𝑖 = [max𝑘 𝑤𝑘𝑖 , min𝑘 𝑤𝑘𝑖 ] by equation 8 illustrated as the 2nd line from the lowest in figure 1. in this case, his/her dissatisfaction becomes the minimum, 𝛽𝑘𝑖 = 0. when all the individual interval weights are common with each other as max𝑘 𝑤𝑘𝑖 ≤ min𝑘 𝑤𝑖𝑘 as in equation 8, the dissatisfaction of all decision makers is 0. though, this is seldom satisfied, since the individual decisions are often different. if the individual weight is crisp as 𝑤𝑘𝑖 = 𝑤𝑘𝑖 = 𝑤𝑘𝑖 , it can support a point of the group interval weight. his/her satisfaction and dissatisfaction are 𝛼𝑘𝑖 = 0 and 𝛽𝑘𝑖 = 𝑤𝑖 − 𝑤𝑖, respectively. therefore, the ranges of the satisfaction and dissatisfaction are 0 ≤ 𝛼𝑘𝑖 ≤ 𝑤𝑘𝑖 − 𝑤𝑘𝑖 and 0 ≤ 𝛽𝑘𝑖 ≤ 𝑤𝑖 − 𝑤𝑖 , respectively. for calculation, two objective functions in equation 14 are aggregated by the weighting approach as max   𝜆 ∑ 𝛼𝑘𝑖 𝑘𝑖 − (1 − 𝜆) ∑ 𝛽𝑘𝑖 𝑘𝑖 , (15) where 𝜆 and (1 − 𝜆) are the weights for satisfaction and dissatisfaction, respectively. focusing on the satisfaction with 𝜆 = 1, the group decision is obtained as the upper approximation of the individual decisions by equation 7. while focusing on the dissatisfaction with 𝜆 = 0, the group decision approaches the lower approximation of the individual decisions by equation 8. the inconsistency among the given comparisons by a decision maker is reflected in his/her interval weights given by equation 3. the individual interval weights include all the possibilities in his/her judgments. it may be uncomfortable for a decision maker to accept the weights out of his/her possible decision. s/he tends to disagree with the group decision if it includes the weight which s/he cannot support by his/her possible decision. from this viewpoint, it is reasonable to primarily minimize the dissatisfaction and secondarily maximize the satisfaction in order to be 1 − 𝜆 > 𝜆. in equation 14, the sums of the satisfaction and dissatisfaction with all alternatives of all decision makers are maximized and minimized, respectively. since the satisfaction and dissatisfaction of each decision maker are not considered, it happens that the group decision which satisfies the specific decision maker may be obtained as the optimal solution of lp problem (equation 14). for the fairness of the decision makers, it is useful to determine the appropriate thresholds of the satisfaction and dissatisfaction of a decision maker. however, it is not easy, since the satisfaction and dissatisfaction depend on the inconsistency in the given comparisons. instead, it can be done by minimizing the deviation of the most and the least satisfaction and dissatisfaction of all the decision makers as follows. ijahp article: entani/group interval weights based on conjunction approximation of individual interval weights international journal of the analytic hierarchy process 434 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.277 min   𝜆(𝛼 − 𝛼) + (1 − 𝜆)(𝛽 − 𝛽) 𝑠. 𝑡. constraints in (14), 𝛽 ≤ ∑ 𝛽𝑘𝑖 𝑖 ≤ 𝛽, ∀𝑘, 𝛼 ≤ ∑ 𝛼𝑘𝑖 𝑖 ≤ 𝛼, ∀𝑘, ∑ 𝛽𝑘𝑖 𝑖 ≤ ∑ 𝛽𝑘𝑖 ∗ 𝑖 , ∀𝑘, ∑ 𝛼𝑘𝑖 𝑖 ≥ ∑ 𝛼𝑘𝑖 ∗ 𝑖 , ∀𝑘, (16) where 𝛼𝑘𝑖 ∗ , ∀𝑘, 𝑖 and 𝛽𝑘𝑖 ∗ , ∀𝑘, 𝑖 are the optimal solutions of equation 14. the sums of the satisfaction and dissatisfaction are maximized and minimized primarily in equation 14 and under the condition the maximum deviations among the satisfaction and dissatisfaction of all decision makers are minimized in equation 16. as a result, the group decision considers the fairness of the individual decision makers when some decision maker gives more inconsistent comparisons than the others. 4. numerical example three decision makers give the following crisp comparison matrices of 4 alternatives independently. 𝐴1 = [ 1 2 4 8 1/2 1 2 4 1/4 1/2 1 2 1/8 1/4 1/2 1 ] , 𝐴2 = [ 1 3 3 4 1/3 1 3 3 1/3 1/3 1 4 1/4 1/3 1/4 1 ] , 𝐴3 = [ 1 1 4 6 1 1 1 2 1/4 1 1 3 1/6 1/2 1/3 1 ] . (17) by equation 3, each decision is obtained as the following interval weights of alternatives. 𝑊1 = [ 0.533 0.267 0.133 0.067 ] , 𝑊2 = [ 0.571 [0.190,0.214] [0.071,0.190] [0.048,0.143] ] , 𝑊3 = [ 0.390 [0.244,0.390] [0.098,0.244] [0.065,0.122] ] . (18) all the decision makers evaluate alternative 1 the best and agree with the weights of alternatives 3 and 4 as 0.133 and 0.067, respectively. they are the common weights of all individual weights. ijahp article: entani/group interval weights based on conjunction approximation of individual interval weights international journal of the analytic hierarchy process 435 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.277 in comparison, the crisp weights are obtained by geometric mean method, where the weight is obtained as a geometric mean of the comparisons, 𝑤𝑘𝑖 = (∏ 𝑎𝑘𝑖𝑗𝑗 ) 1/𝑛 / ∑ (∏ 𝑎𝑘𝑖𝑗𝑗 ) 1/𝑛 𝑖 . 𝑤1 = [ 0.533 0.267 0.133 0.067 ] , 𝑤2 = [ 0.494 0.265 0.165 0.048 ] , 𝑤3 = [ 0.467 0.251 0.196 0.086 ]. (19) the comparisons by decision maker 1, 𝐴1, are consistent as in equation 2 so that his/her decision, 𝑊1, is denoted as crisp weights. they are equal to those by geometric mean and eigenvector methods. as for the other decision makers, 𝐴2 and 𝐴3, the rough rankings by equations 18 and 19 are not contradicted. when compared to the plausible evaluations by the crisp weights in equation 19, the interval weights by interval ahp in equation 18 show us the possible evaluations. it is possible that alternative 3 in a2 is evaluated as less important by the interval weights than by the crisp weights. similarly in a3, we find that alternative 2 may be evaluated as equal to alternative 1. the given comparisons are condensed into the crisp weights for the plausible decision. the interval weights give the possible decision including the given comparisons. the interval weights are useful enough to aid in decision making, since a decision is often at the decision maker’s discretion. since the individual decision reflects all the possibilities in the given comparisons, it is reasonable that the weight for the dissatisfaction, 1 − 𝜆 , is more than that of the satisfaction, 𝜆. then, 𝜆 is supposed to be 0.1 and 0.4, where 1 − 𝜆 > 𝜆, and in addition 0.9. first, equation 14 is solved and the satisfaction and dissatisfaction of each decision maker are obtained. then, equation 16 is solved within them and the group interval weights are obtained. 𝑊(𝜆 = 0.1) = [ [0.390,0.571] [0.214,0.267] [0.133,0.200] [0.067,0.143] ] (width:  0.377), 𝑊(𝜆 = 0.4) = [ [0.390,0.571] [0.214,0.267] [0.098,0.200] [0.066,0.143] ] (width:  0.413), 𝑊(𝜆 = 0.9) = [ [0.390,0.571] [0.190,0.390] [0.071,0.244] [0.048,0.143] ] (width:  0.649). (20) in comparison, the group crisp weights as the geometric means of the corresponding three individual crisp weights in equation 19 are as follows. ijahp article: entani/group interval weights based on conjunction approximation of individual interval weights international journal of the analytic hierarchy process 436 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.277 𝑤 = [ 0.497 0.261 0.163 0.076 ] (21) since the given comparisons are the same, the group weights by two methods are similar and the crisp group weights in equation 21 are included in the interval group weights in equation 20. the interval group weights show us the possibilities of evaluations from the individual decisions. we find that the evaluation of alternative 1 is the most uncertain because of its wide width. as for the relationship between alternatives 3 and 4, there is a possibility that alternative 4 is evaluated better than alternative 3. the individual weights of alternative 1 by all decision makers are crisp in equation 18 so that his/her satisfaction cannot be more than 0 and his/her dissatisfaction is equal to the width of its group interval weight, 0.181, which does not depend on 𝜆 . when the satisfaction is primarily maximized with 𝜆 = 0.9, all the individual interval weights are included in the group interval weight with the minimum width. the group decision is from their minimum to their maximum, 𝑊𝑖 = [min { 𝑤1𝑖 , 𝑤2𝑖 , 𝑤3𝑖 }, max { �̅�1𝑖 , �̅�2𝑖 , �̅�3𝑖 }]. when the importance of the satisfaction, 𝜆, increases from 0.1 to 0.9, the widths of the group interval weights become wide from 0.377 to 0.649. the group decision becomes uncertain so as to satisfy the decision makers by increasing the supported ranges. from the opposite viewpoint of the dissatisfaction, when its importance, 1 − 𝜆, increases from 0.1 to 0.9, the width of the group interval weight decreases from 0.649 to 0.377 by reducing the unsupported ranges. the group interval weights with various importance of satisfaction 𝜆s are not very different. in cases where there are two kinds of the importance of satisfaction and dissatisfaction (𝜆, 1 − 𝜆) as (0.1,0.9) and (0.4,0.6) , the satisfaction and dissatisfaction of decision maker 𝑘, 𝛼𝑘 and 𝛽𝑘, respectively, are compared in equation 22. (0.1,0.9): (𝛼1, 𝛼2, 𝛼3) = (0,0.133,0.145), (𝛽1, 𝛽2, 𝛽3) = (0.377,0.244,0.232), (0.4,0.6): (𝛼1, 𝛼2, 𝛼3) = (0,0.169,0.181), (𝛽1, 𝛽2, 𝛽3) = (0.413,0.244,0.232). (22) with (𝜆, 1 − 𝜆) = (0.1,0.9) , the sum of the satisfaction and dissatisfaction of each decision maker is 0.377 and that with (𝜆, 1 − 𝜆) = (0.4,0.6) is 0.413. they are equal to the sums of the widths of the group interval weights as in equation 13 and figure 1. the importance of the satisfaction, 𝜆, increases from 0.1 to 0.4, the satisfaction of decision makers 2 and 3 also increase by 0.036. correspondingly, the dissatisfaction of decision maker 1, whose satisfaction cannot go over 0, increases by 0.036 since the importance of the dissatisfaction, 1 − 𝜆, decreases from 0.9 to 0.6. there is a trade-off between the increase of the satisfaction and the decrease of the dissatisfaction. 5. conclusion the individual and group decisions in this study are both denoted as the normalized interval weights of alternatives. at first, the individual decisions are independently obtained from the crisp comparisons given by the corresponding decision makers. then, the group decision is obtained so as to have something in common with each individual decision. the proposed model minimizes the deviations of the upper and lower bounds of the group interval weight from those of each individual interval weight. therefore, the ijahp article: entani/group interval weights based on conjunction approximation of individual interval weights international journal of the analytic hierarchy process 437 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.277 obtained group decision is the conjunction approximation of the individual decisions. a decision maker supports a part of the group interval weight but s/he does not support the other part of it. in other words, the group interval weight is divided into the ranges satisfying and dissatisfying a decision maker. for a better group decision, the satisfaction and dissatisfaction of all decision makers should be maximized and minimized, respectively. however, since these two objectives have a trade-off relationship, they are aggregated by the weighting approaches. as the extreme case of the conjunction approximation, the group decision can be obtained as the upper or lower approximation of the individual decisions. the former group decision includes all the individual decisions by primarily maximizing the satisfaction. the latter group decision is included in all the individual decisions by primarily minimizing the dissatisfaction. the group interval weights reflect the possibilities in the given crisp comparisons and the relationship among the individuals. since they show us the possible evaluations of alternatives with the satisfaction or dissatisfaction degree, they are useful in decision making. in this paper, the group interval weights are obtained as the approximation of the individual interval weights from the crisp comparison matrices. in future work, the group weight approximation process can be combined into determining the individual weights from the comparisons. moreover, the comparisons can be extended into interval and fuzzy to be more suitable for our intuitive judgments. ijahp article: entani/group interval weights based on conjunction approximation of individual interval weights international journal of the analytic hierarchy process 438 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.277 references aczel, j., & saaty, t. l. (1983). procedure for synthesizing ratio judgments. journal of mathematical psychology, 27, 93-102. doi:10.1016/0022-2496(83)90028-7 arbel, a. (1989). approximate articulation of preference and priority derivation. european journal of operational research, 43, 317-326. doi: 10.1016/03772217(89)90231-2 altuzarra, a., moreno-jimenez, j. m., & salvador, m. (2007). a bayesian priorization procedure for ahp-group decision making. european journal of operational research, 182(1), 364-382. doi:10.1016/j.ejor.2006.07.025 basak, i., & saaty, t. l. (1993). group decision making using analytic hierarchy process. mathematical and computer modelling, 17 (4/5), 101-109. doi:10.1016/08957177(93)90179-3 de campos, l. m., huete, j. f., & moral, s. (1994). probability intervals: a tool for uncertain reasoning. international journal of uncertainty, 2 (2), 167-196. doi: 10.1142/s0218488594000146 e. dopazo, k. l., chouinard, s., & guisse, j. (2014). a parametric model for determining consensus priority vectors from fuzzy comparison matrices. fuzzy sets and systems, 246, 49-61. doi:10.1016/j.fss.2013.07.022 dyer, r. f., & forman, e. h. (1992). group decision support with the analytic hierarchy process. decision support systems, 8, 94-124. entani, t., & inuiguchi, m. (2010). lower approximations of group of intervals. proceedings of scis and isis, 9-14. forman, e., & peniwati, k. (1998). aggregating individual judgments and priorities with the analytic hierarchy process. european journal of operational research, 108, 165169. doi: http://dx.doi.org/10.1016/s0377-2217(97)00244-0 saaty, t. l. (1980). the analytic hierarchy process. new york: mcgraw-hill. saaty, t. l., & vargas l. g. (1987). uncertainty and rank order in the analytic hierarchy process. european journal of operational research, 32, 107-117. doi:10.1016/03772217(87)90275-x sugihara, k., ishii, h., & tanaka, h. (2004). interval priorities in ahp by interval regression analysis. european journal of operational research, 158 (3), 745-754. sugihara, k., & tanaka, h. (2001). interval evaluations in the analytic hierarchy process by possibilistic analysis. computational intelligence, 17 (3), 567-579. tanaka, h., sugihara, k., & maeda, y. (2004). non-additive measures by interval probability functions. information sciences, 164, 209-227. doi:10.1016/j.ins.2003.06.001 http://dx.doi.org/10.1016/0022-2496(83)90028-7 http://dx.doi.org/10.1016/j.ejor.2006.07.025 http://dx.doi.org/10.1016/0895-7177(93)90179-3 http://dx.doi.org/10.1016/0895-7177(93)90179-3 http://dx.doi.org/10.1016/j.fss.2013.07.022 http://dx.doi.org/10.1016/s0377-2217(97)00244-0 http://dx.doi.org/10.1016/0377-2217(87)90275-x http://dx.doi.org/10.1016/0377-2217(87)90275-x http://dx.doi.org/10.1016/j.ins.2003.06.001 ijahp article: entani/group interval weights based on conjunction approximation of individual interval weights international journal of the analytic hierarchy process 439 vol. 7 issue 3 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i3.277 xu, z. (2013). group decision making model and approach based on interval preference ordering. computers & industrial engineering, 64(3), 797–803. yeh, c.-h., & chang, y.-h. (2009). modeling subjective evaluation for fuzzy group multicriteria decision making. european journal of operational research, 194(2), 464473 doi:10.1016/j.ejor.2007.12.029 http://dx.doi.org/10.1016/j.ejor.2007.12.029 ijahp news and events: mu/ ceciis turns 30! international journal of the analytic hierarchy process 439 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.727 central european conference of information and intelligent systems (ceciis) turns 30! this past october, ceciis celebrated 30 years since its founding at the university of zagreb’s faculty of organization and informatics (foi) in varazdin, croatia. the aim of this conference is to promote the interface of researchers involved in the development and application of methods and techniques in the field of information and intelligent systems. the foi is highly focused on decision-making and their faculty and students use ahp/anp widely in their academic endeavors. the event was sponsored by the president of croatia, mrs. kolinda grabar kitarović, and the minister of science and education, dr. blazenka divjak, who was also the speaker at the opening ceremony. her talk was about why we need to meet at conferences, and she quoted several thinkers’ opinions on the topic. one of the most interesting quotes was the statement that “although great ideas may not be born in these conferences, very bad ones have perished there”. dr. slavko vidović, previous professor of foi, founder and manager of infodom company a gold sponsor of ceciis talks to dr. blazenka divjak, foi professor and current minister of science and education professor nina begicevic, newly elected foi dean, addresses ceciis participants ceciis is the most important ict conference in middle and southeast europe. this year’s main topic was related to digital transformation and its influence on the future. the conference lasted for three days. the invited lectures were delivered by dr. wolf rauch (karl-franzens university, wien, austria), dr. sirje virkus (university in tallin, estonia), and dr. günter müller (university in freiburg, germany). at the conference, 43 papers were presented that were evaluated by 78 reviewers. in addition, several other events were organized such as a workshop on digitrans methodology, a computer games conference, an international doctoral seminar and a student section with 19 research papers. dr. enrique mu, ijahp editor-in-chief and a member of the organizing committee, participated as a panelist for the students’ presentations. ijahp news and events: mu/ ceciis turns 30! international journal of the analytic hierarchy process 440 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.727 dr. violeta vidaček-hainš, foi faculty, dr. enrique mu, official friend of foi and ceciis student participants ceciis online international student presenters during the different social events, participants continued to add other reasons why these conferences are important such as the opportunity to share a glass of wine with an esteemed colleague or simply to give each other a warm friendly embrace, all things that cannot currently be done via skype. we wish for ceciis to keep going strong and send a big thank you to our colleagues for the productive and wonderful experience. looking forward to ceciis 2020! ijahp article: ohoitimur, krejci, raco, raton, taroreh/ pastoral strategic planning priorities for the church: case study of the vicariate episcopal of tonsea of the diocese of manado international journal of the analytic hierarchy process 415 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.674 pastoral strategic planning priorities for the church: case study of the vicariate episcopal of tonsea of the diocese of manado johanis ohoitimur philosophy department, sekolah tinggi filsafat seminari pineleng, indonesia pineleng, manado 95662, indonesia james v. krejci lewis university, illinois usa college of business one university parkway romeoville, il 60446 jozef richard raco* management department , universitas katolik de la salle manado, indonesia kombos, kairagi i, manado 95253, indonesia *corresponding author yulius christian raton industrial engineering department, universitas katolik de la salle manado, indonesia kombos, kairagi i, manado 95253, indonesia frankie j. h., taroreh accounting department, universitas katolik de la salle manado, indonesia kombos, kairagi i, manado 95253, indonesia abstract strategic planning is a commonly used tool in profit-oriented institutions. however, it is rarely applied to non-profit organizations such as churches. the vicariate episcopal of tonsea of the diocese of manado, a local catholic church community, served as a test case for the implementation of strategic planning methods to further the pastoral mission. the study used the analytic hierarchy process to determine the ranking of key elements of the pastoral mission, which provided a methodology upon which to formulate strategic plans and best utilize the available, but limited, resources. this study demonstrates the importance of the analytic hierarchy process in determining the priority of the programs identified by the church. the researchers recommend further and deeper research on other vicariates of the diocese of manado. keywords: analytic hierarchy process; strategic planning; management; catholic; vicariate episcopal of tonsea; diocese of manado ijahp article: ohoitimur, krejci, raco, raton, taroreh/ pastoral strategic planning priorities for the church: case study of the vicariate episcopal of tonsea of the diocese of manado international journal of the analytic hierarchy process 416 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.674 1. introduction management science methodologies and strategic planning processes are routinely applied and relied upon in military and profit-oriented business institutions (golden, wasil, harker, 1989). such methodologies and processes are embraced because they ensure better performance of the institution and survival in today's world of fierce competition. however, such methodologies and processes are rather uncommon in faithbased organizations. recently, non-profit institutions, such as the church, have begun to adopt the principles of management science and strategic planning in an effort to make their programs more structured, directed, contextual and efficient. in an effort to fulfill its pastoral missions, the vicariate episcopal of tonsea of the diocese of manado, a catholic church community, concluded that it was in need of a strategic methodology to determine the priorities of each of its programs in order to fulfill the current needs of the parishioners in the most efficient and effective manner. historically, formal strategic planning techniques have not been applied to the activities of the vicariate episcopal of tonsea. as such, this study aims to determine the most appropriate priority program to assist the vicariate episcopal of tonsea in ranking its large number of varied objectives. simply stated, the study seeks to determine what the best method is to determine the priority of the dictates of the vicariate. the application of analytic hierarchy process (ahp) to rank priorities was proposed to establish priorities. however, a review of the literature showed no academic papers regarding the use of the ahp in the strategic planning process of the church. this study is considered pioneer research for strategic planning and the determination of priority programs within the church environment using the ahp method. the vicariate implemented the ahp and the results were analyzed. the ahp proved to be a valuable method to assess the relative importance of the different priorities of the vicariate. the ahp format also provided several benefits beyond establishing priorities. the ahp clarified the communication of goals and tracking of objectives and increased the prospect of meeting the short timeline dictated by the bishop. in addition, the ahp process was relatively simple for participants to understand and easy to implement. it also provided a quantitative method of ranking, which reduced the amount of variation and arguments typically associated with qualitative analysis. however, while the ahp was suitable and useful for the vicariate’s stated goals, it was not without limitation and required additional research and analysis. the primary limitation was that the ahp technique was not known by the user community. even more, since there has never been an empirical study of church programs that used strategic planning and ahp methods, researchers are encouraged to further investigate and utilize ahp as a valuable tool to determine priorities in unique and non-traditional applications. this paper is organized into seven sections. first, the background, problems, objectives, and limitations of the research are explained. second, a literature review is provided, which identifies previous studies on strategic planning in the church context. the third ijahp article: ohoitimur, krejci, raco, raton, taroreh/ pastoral strategic planning priorities for the church: case study of the vicariate episcopal of tonsea of the diocese of manado international journal of the analytic hierarchy process 417 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.674 section discusses the vicariate of episcopal of tonsea case study. this is followed by a discussion of the methodology, the analysis, and results of the case study. lastly, the conclusions and recommendations and proposals for additional research are detailed. 2. literature review 2.1. strategic planning overview management science is the broad interdisciplinary study of problem solving and decision making in human organizations and is focused on the development of techniques that enable decision makers to cope with the complexity of our world. strategic planning is an organization’s process of defining its strategy, or direction, and making decisions about allocating its resources to pursue this strategy. the word 'strategy' comes from the greek word ‘strategos’ which means the art of managing troops and equipment to face and defeat enemies (ahmed, bwisa, & otieno, 2014). the word was originally used to describe the planning process a military leader used to develop tactics and strategies to conquer their enemies. the term was adopted by the business sector to define the actions that its managers take to outperform their competitors and achieve superior profitability (thompson, peteraf, gamble, strickland, 2016). strategic management can be thought of in several ways. first, strategic management is a continuous effort by an organization or company to adjust and update itself to meet the market demand. second, it is a process of overcoming problems that can disrupt future business objectives through continuous improvement of policies and plans (wanyoike, j & orwa, b, 2016). third, strategic management develops a roadmap for how the organization can continue to exist in a highly competitive environment. a critical part of business strategy is the planning process. the purpose of strategic planning is to set overall goals for the organization and develop a plan to achieve the organization’s objectives. strategic planning requires identifying potential alternative actions, evaluating the risks and benefits of each alternative and then selecting the best alternative actions. (arasa & k’obonyo, 2012). athapaththu (2016) added that strategic planning assists managers in recognizing priorities and determining the actions that are needed. business organizations and non-profit institutions have long espoused the strategic planning process. indeed, it has been shown that having a formal planning process is a key factor in focusing and motivating people to achieve their organizational goals and objectives (bakhmutsky, 2013). 2.2. the church’s potential espousal of strategic planning processes not all of the historical evidence demonstrates the church’s reluctance to implement strategic planning processes. there is evidence in the holy scriptures, both in the old and the new testament, of the importance of strategic planning. it is mentioned in genesis that god worked with a master plan, creating the world, the day and the night, the horizon, water and plants – and that all of these priorities were completed in only six days. after resting, god then built the nation of israel and sent jesus christ. ijahp article: ohoitimur, krejci, raco, raton, taroreh/ pastoral strategic planning priorities for the church: case study of the vicariate episcopal of tonsea of the diocese of manado international journal of the analytic hierarchy process 418 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.674 the text of the bible repeatedly references “god’s plan” as well. a non-exhaustive list of such references include: god has a plan since the early ages (2 kings 19:25); god’s plan stays forever (psalm 33:10-11); jesus was sent according to god’s plan (acts 2:23); and through christ everything has been set up (ephesians 1:11) (bakhmutsky, 2013). in addition, the societal structure of the church seemingly exemplifies principles of organizational management. the church has a structure, communities, rules, and employees. it regulates money, operates schools, and hospitals – all of which need to be managed and controlled using applied sciences, such as management or accounting. the church, as do all successful organizations, requires an effective management plan in order to function (kagumu, j. & njuguna, 2016). furthermore, the current sexual issues that have negatively affected the catholic church have forced church officials and the hierarchy to look for a comprehensive solution from the administrative, education, management and disciplinary system of the holy church. it requires that the church have a comprehensive strategic plan to organize itself more effectively and efficiently in order to be reestablished as a trusted institution to preach the salvation of human beings both on earth and in heaven. church leadership has an important role in developing a strategic management process and a framework to achieve objectives of the church (oosthuizen & lategan, 2016). the strategic planning process can provide an effective tool for the church to undergo the changes that are needed in pastoral service and to regain the respect of the people. the church, as a social entity and a non-profit organization, was required to start considering strategic planning techniques in the planning of pastoral services. by utilizing strategic planning methodologies, it is anticipated that the church’s programs will be more focused, efficient, have specific targets, and can be evaluated and improved. 2.3. the potential benefits of strategic planning for the church there are several important benefits for the church when implementing strategic planning. first, the church is able to go on the offensive in carrying out its mission rather than remaining on the defensive and reacting to the culture. second, the church members will be able to focus on the mission and purpose of the church, setting aside their personal agendas. third, it encourages good stewardship as members are more likely to give talents, time and finances to the church that exhibits a clear purpose and direction. fourth, it allows the church the freedom to cull unproductive ministries and programs that drain limited resources. fifth, a strategic plan gives the church a means to effectively evaluate progress and efficiency. sixth, it reveals the church's strengths and weaknesses so that it may build on the strengths and overcome the weaknesses. seventh, it helps the church embrace change and accept new members and ideas. a strategic plan allows the church to control and reduce its operational cost, and at the same time improve the quality of services (wanyoike, j & orwa, b, 2016). strategic planning helps the church have a clear direction so that it can build a systematic framework to sharpen its competitive focus, design program models and priorities and ensure customer-focused growth strategies. for this reason, all stakeholders must be actively involved in determining the strategic plan of the church (mutia, k’aol, & ijahp article: ohoitimur, krejci, raco, raton, taroreh/ pastoral strategic planning priorities for the church: case study of the vicariate episcopal of tonsea of the diocese of manado international journal of the analytic hierarchy process 419 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.674 katuse, 2016). the church needs an effective strategic plan to foster growth in the future (oosthuizen & lategan, 2016). the church's strategic plan entails three key elements: 1) the mission statement of the church – its very reason for existence; 2) the activities, strategies, acceptable outcomes and fundamental calling; and 3) a vision of the ideal and unique 'image' of the future church including guiding values principles (mutia et al., 2016). 3. case study: the vicariate episcopal of tonsea the vicariate episcopal of tonsea is located in north sulawesi indonesia (appendix, figure 1). covers the government area of north minahasa regency and bitung city and has a total of 21,996 catholics out of 182,619 local residents in that area. this vicariate has 9 parishes and is served by 12 priests. the ratio of a priest to his flock is 1:1,833. under the leadership of bishop benedictus untu, the diocese of manado conducted a diocesan synod in 2018 to commemorate the 150 th year of the catholic mission in north sulawesi. the synod produced a strategic plan for 2019-2023, which consisted of 6 strategic programs with 38 elements to be implemented in the pastoral service of the diocese. the six strategic programs are: 1) ministry of the word and sacrament; 2) preserving the treasury of faith; 3) fellowship and leadership; 4) promoting the dignity of the laity; 5) catholic education and; 6) church property (untu, 2018). the vicariate episcopal of tonsea, which is part of the diocese of manado, needed to determine the priority of these programs for the short-term, medium-term and long-term periods. it was necessary to determine the priorities of the programs for the vicariate to ensure that the programs were contextual and able to answer the current needs of catholics in that area. the priorities must be specific, measurable, attainable, relevant and time-bound. the application of the analytic hierarchy process (ahp) to rank priorities was proposed as a method to establish priorities. however, a review of the literature showed no academic papers regarding the use of the ahp in the strategic planning process of the church. therefore, this study aims to determine the effectiveness of the use of the ahp to determine the program priorities of the vicariate apostolic of tonsea. a disciplined method of establishing priorities was important because of the complexity and quantity of the criteria and sub-criteria. 4. methodology the basic direction of the diocese of manado was promulgated on september 14, 2018 and established the strategic plan for 2019-2023, which consisted of 6 strategic programs with 38 elements that were to be implemented by the various parishes. due to the limited resources (time, money, people) and the vast area covered by each priest, the vicariate needed to identify a methodology to determine the priorities of each program in order to fulfill the current needs of the parishioners in the most efficient and effective manner. since the criteria and sub-criteria were quite complex, the researcher determined that the analytic hierarchy process (ahp) was the appropriate tool for data gathering and analysis. the ahp method is commonly used to determin program priorities and to ijahp article: ohoitimur, krejci, raco, raton, taroreh/ pastoral strategic planning priorities for the church: case study of the vicariate episcopal of tonsea of the diocese of manado international journal of the analytic hierarchy process 420 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.674 simplify the complexity of problems. this method provides an excellent combination of quantitative and qualitative analysis. the data, which was descriptive and qualitative in nature, can be easily quantified using the ahp method (saaty, 2008). the ahp is a typical systems engineering method that transforms qualitative analysis into quantitative analysis and is widely used to solve decisions with complicated structures and a large numbers of decision criteria and factors which are difficult to quantify (li, ni, dong, & zhu, 2018; fabjanowicz, bystrzanowska, namieśnik, tobiszewski, & płotka-wasylka, 2018). as a result, many researchers use this method because of its simplicity and adaptability (promentilla, aviso, lucas, razon, & tan, 2018). it is very often used by decision makers to find and determine policies and program priorities (duleba & moslem, 2019). the first step in this method is to determine the purpose of the study, then proceed to find the criteria and sub-criteria (nguyen, fong, & ho, 2010). the purpose of the study was to determine the implementation priorities for the vicariate apostolic of tonsea based on the strategic plan developed by the diocese of manado for 2019–2023. the six strategic programs became the criteria and after focus group discussions, it was determined that 26 of the elements qualified as the sub-criteria that must be implemented in the vicariate episcopal of tonsea. twelve of the criteria were eliminated because it was determined that they needed to be implemented at the diocese level. table 5 shows the criteria and sub-criteria of this research. after determining the purpose of the study, and setting up the criteria and sub-criteria and alternatives, the hierarchy was arranged as shown in figure 2 below. ijahp article: ohoitimur, krejci, raco, raton, taroreh/ pastoral strategic planning priorities for the church: case study of the vicariate episcopal of tonsea of the diocese of manado international journal of the analytic hierarchy process 421 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.674 figure 2 hierarchy of vicariate episcopal of tonsea priority program goal: strategic program priorities of the vicariate apostolic of tonsea based on the strategic planning of the diocese of manado for 2019 2023 fellowship and leadership organize pastoral structure animate the spirituality of church based groups promote kinship and fraternity strengthen the system & the role of catechesis commission preserving the treasury of faith upgrade the catechetic program upgrade the catechetic facilities arrange the territory and categorical unit reinforce the commission of liturgy role model and competency of liturgy leadership ministry of the word and sacraments complete the liturgy facilities education & comprehension of liturgical music education & comprehension of enculturation empower lay apostolate unity and solidarity between the lay and the ordained servants upgrade the socio-economic quality of the people reinforce the partisanship for the poor foster integral family development promote the quality of relationship with people of other religious beliefs take sides to the preservation of the environment promoting dignity of the laity catholic education catholic education to those studying at non-catholic school attention to early age education develop non-formal education and training skills managing church property expand & synergizing the competency system build up a transparent financial information system build up the asset management system make good use of the church property ijahp article: ohoitimur, krejci, raco, raton, taroreh/ pastoral strategic planning priorities for the church: case study of the vicariate episcopal of tonsea of the diocese of manado international journal of the analytic hierarchy process 422 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.674 once the goal, criteria and sub-criteria were identified and modeled, then a questionnaire was compiled using the analytic hierarchy process technique, which is in the form of pairwise comparisons. the parish priests of the vicariate episcopal of tonsea, who were the policy makers in each of their respective parishes, completed the questionnaire. it was determined that the parish priests had the proper knowledge, skills, and experience to complete this task. all of them have worked in the parishes for over three years and know the people, culture and living environment. they were considered the experts, the people who understood the true problems, felt the consequences of a problem or had an interest in the problem (raco & tanod, 2014). there were 10 respondents. the ahp does not have a formula for a required number of respondents, but a minimum of 2 people must be involved. in the ahp, the number of responses is not critical, the quality of the data and the quality of the respondent is most important (duleba & moslem, 2019). the respondents must have lived-experience and be very knowledgeable about the issues being studied. they should be the decision makers, and be willing and able to share their knowledge regarding the issues, which were complex and qualitative in nature, based on their perception, experience and intuition. the purpose of the questionnaire in the form of pairwise comparisons, as shown in table 3, was to calculate the weight of each criteria and sub-criteria based on the hierarchical structure. the respondents determined the weight of each of the criteria and sub-criteria. pairwise comparisons were made in the form of absolute determinants that represent how many criteria or sub-criteria are compared to one another (budimčević, 2018). the paired comparison uses a scale that has been determined by saaty (2008) where the respondents choose numbers between 1 and 9, which are listed in the scale as shown in table 1. ijahp article: ohoitimur, krejci, raco, raton, taroreh/ pastoral strategic planning priorities for the church: case study of the vicariate episcopal of tonsea of the diocese of manado international journal of the analytic hierarchy process 423 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.674 table 1 saaty’s comparative scale intensity of importance on an absolute scale definition explanation 1 equal importance two activities contribute equally to the objective 3 moderate importance of one over another experience and judgment strongly favor one activity over another 5 essential or strong importance experience and judgment strongly favor one activity over another 7 very strong importance an activity is strongly favored and its dominance is demonstrated in practice 9 extreme importance the evidence favoring one activity over another is of the highest possible order of affirmation 2, 4, 6, 8 intermediate values between the two adjacent judgments when compromise is needed the numbers on the scale of comparison allow respondents to determine the ratings, that were previously expressed verbally and qualitatively, in a quantitative numerical form that actually represents values (fabjonowics, namiesnik, tobiszewski, plotka-wasylka, 2018). this ability to quantify qualitative data is one of the advantages of the ahp method. the results of the data obtained through the questionnaire were then aggregated using the geometric average equation as appears in the equation 1 below. 𝐺𝑀 = √(𝑥1)(𝑥2) … (𝑥𝑛 ) 𝑛 (1) after averaging the questionnaire data, a pairwise comparison matrix was compiled starting with the criteria and then the sub-criteria as shown in equation 2 below. 𝐴 = [𝑎𝑖𝑗 ], 𝑎𝑖𝑗 = 𝑤𝑖 𝑤𝑗⁄ , 𝑎𝑗𝑖 = 1 𝑎𝑖𝑗 ⁄ 𝑎𝑖𝑖 = 1 (2) to calculate the priority weights, the paired comparison matrix was normalized using equation 3 below. 𝑏𝑖𝑗 = 𝑎𝑖𝑗 ∑ 𝑎𝑖𝑗 𝑛 𝑖=1 (3) this was followed by calculating the average row in the pairing matrix, which is the priority weight, using equation 4 below. ijahp article: ohoitimur, krejci, raco, raton, taroreh/ pastoral strategic planning priorities for the church: case study of the vicariate episcopal of tonsea of the diocese of manado international journal of the analytic hierarchy process 424 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.674 𝑤𝑖 = ∑ 𝑏𝑖𝑗 𝑛 𝑗=1 𝑛 (4) a consistency test was performed to determine the level of consistency in the paired comparison matrix with the following steps: calculate the maximum (principal) eigenvalue using equation 5. 𝜆𝑚𝑎𝑥 = ∑ (𝐴𝑤)𝑖 𝑛𝑤𝑖 (5) 𝑛 𝑖=1 calculate the consistency of the index with equation 6 below. 𝐶𝐼 = 𝜆𝑚𝑎𝑥 − 𝑛 𝑛 − 1 (6) a boundary test, determined by saaty, was conducted after obtaining a consistency index. the test was measured using the consistency ratio, which is a comparison between the consistency index and the ratio index (ri). the equation for calculating the consistency ratio is shown in equation 7 as follows: 𝐶𝑅 = 𝐶𝐼 𝑅𝐼 (7) table 2 shows the ri value for each n object. table 2 ratio index n 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 ri 0.00 0.00 0.58 0.90 1.12 1.24 1.32 1.41 1.45 1.49 1.51 1.48 1.56 1.57 1.59 if cr <= 0.10 (10%), this means that the user's answer is consistent and the resulting solution is optimal and acceptable. conversely, if the result was more than 0.1, the judgment is not correct and the study needs to be repeated (saaty, 2008). since this methodology has roots in mathematics and psychology, there is no need to ensure statistical generalization. it is only required that the sample size be sufficient to implement the ahp, which is one of the key reasons this methodology has been used effectively with small samples of less than 50 key representatives or stakeholders. an additional advantage of the ahp method lies in the use of pairwise comparisons that reduce the burden of prioritizing in decision-making. the application of a numeric preference scale enables consistent comparison of quantitative and/or qualitative criteria. the relative simplicity of this method means that the interviewer does not need to be present to guide the respondent through the process, which may be required when using ijahp article: ohoitimur, krejci, raco, raton, taroreh/ pastoral strategic planning priorities for the church: case study of the vicariate episcopal of tonsea of the diocese of manado international journal of the analytic hierarchy process 425 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.674 complex techniques such as contingent valuation. therefore, the ahp does not require probability sampling or large samples in order to generate valid and reliable results. however, it is necessary to ensure that respondents are selected on the basis of being representative of the experience under study (morgan, 2017). 5. results determining program priorities for the vicariate episcopal of tonsea is very important before implementing the strategic plan. the analysis started with the construction of a hierarchical model. in the first layer, the main goal was formulated, in the second layer the criteria were determined, and finally in the third layer the sub-criteria were located. the goal and criteria, and the criteria and sub-criteria were connected. in this study, both criteria and sub-criteria were taken from the manado diocesan strategic plan for 2019-2023. the strategic plan of the diocese of manado contained six strategic programs and 38 sub-programs. the six strategic programs of the manado diocese were the criteria, and the 38 sub-programs were used as sub-criteria. only 26 out of 38 sub-criteria were used in the study because they were directly related initiatives that are managed at the parish level. after the goals, criteria and sub-criteria were determined, a questionnaire was given to the parish priests of the vicariate episcopal of tonsea. they were asked to choose the relative importance of each criterion. their preferences in pairwise comparisons are defined by saaty’s 1-9 scale as shown in table 1. the results of the calculations of the pairwise comparison and the priority weight for the criteria are presented in table 3 below. table 3 pairwise comparison and priority weight of the strategic program priorities fl tf mws dl ce mcp total priority weight fl 0.197 0.225 0.169 0.210 0.214 0.169 1.183 0.197 tf 0.173 0.198 0.261 0.152 0.162 0.257 1.203 0.201 mws 0.272 0.176 0.232 0.312 0.224 0.211 1.427 0.238 dl 0.128 0.177 0.102 0.136 0.178 0.142 0.863 0.144 ce 0.111 0.146 0.125 0.092 0.120 0.120 0.714 0.119 mcp 0.119 0.078 0.112 0.098 0.102 0.102 0.610 0.102 𝜆𝑚𝑎𝑥 = 6.082, 𝐶𝐼 = 0.016, 𝐶𝑅 = 0.013 information: symbol criteria: fl: fellowship and leadership; tf: treasury of faith; mws: ministry of word and sacrament; dl: dignity of laity; ce: catholic education; mcp: managing church property the results of the pairwise comparison and the priority weight of the sub-criteria with respect to the criteria appear in table 4 below. ijahp article: ohoitimur, krejci, raco, raton, taroreh/ pastoral strategic planning priorities for the church: case study of the vicariate episcopal of tonsea of the diocese of manado international journal of the analytic hierarchy process 426 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.674 table 4 pair wise comparison and priority weight of the element of the strategic program fellowship & leadership fl 1 fl2 fl3 priority weight fl1 1.000 1.558 1.032 0.387 fl2 0.642 1.000 1.379 0.318 fl3 0.969 0.725 1.000 0.295 𝜆𝑚𝑎𝑥 = 3.060, 𝐶𝐼 = 0.030, 𝐶𝑅 = 0.052 preserving the treasury of faith tf 1 tf 2 tf 3 tf 4 priority weight tf 1 1.000 0.783 1.627 1.629 0.285 tf 2 1.277 1.000 2.273 1.781 0.359 tf 3 0.615 0.440 1.000 1.182 0.180 tf 4 0.614 0.562 0.846 1.000 0.176 𝜆𝑚𝑎𝑥 = 4.014, 𝐶𝐼 = 0.005, 𝐶𝑅 = 0.0053 ministry of the word and sacrament mws 1 mws 2 mws 3 mws 4 mws 5 priority weight mws 1 1.000 0.926 1.167 1.032 1.196 0.205 mws 2 1.080 1.000 2.113 2.217 1.558 0.295 mws 3 0.857 0.473 1.000 1.012 1.235 0.168 mws 4 0.969 0.451 0.988 1.000 1.649 0.183 mws 5 0.836 0.642 0.810 0.607 1.000 0.149 𝜆𝑚𝑎𝑥 = 5.082, 𝐶𝐼 = 0,021, 𝐶𝑅 = 0,0184 promoting the dignity of the laity dl 1 dl 2 dl 3 dl 4 dl 5 dl 6 dl 7 priority weight dl 1 1.000 1.849 2.148 1.134 1.077 1.702 1.395 0.197 dl 1 0.541 1.000 1.629 0.907 0.926 1.351 1.608 0.147 dl 1 0.466 0.614 1.000 0.919 1.182 1.629 1.187 0.130 dl 1 0.882 1.102 1.088 1.000 1.403 1.901 2.148 0.176 dl 1 0.929 1.080 0.846 0.713 1.000 1.934 2.395 0.160 dl 1 0.587 0.740 0.614 0.526 0.517 1.000 1.395 0.099 dl 1 0.717 0.622 0.843 0.466 0.418 0.717 1.000 0.091 𝜆𝑚𝑎𝑥 = 7.16, 𝐶𝐼 = 0.027, 𝐶𝑅 = 0.0208 catholic education ce 1 ce 2 ce 3 priority weight ce 1 1.000 1.220 1.318 0.379 ce 2 0.819 1.000 2.120 0.389 ce 3 0.759 0.472 1.000 0.232 𝜆𝑚𝑎𝑥 = 3.051, 𝐶𝐼 = 0.025, 𝐶𝑅 = 0.044 managing the church property mcp 1 mcp 2 mcp 3 mcp 4 priority weight mcp 1 1.000 0.987 1.241 0.885 0.243 mcp 2 1.013 1.000 2.217 0.744 0.284 mcp 3 0.806 0.451 1.000 1.403 0.216 mcp 4 1.129 1.344 0.712 1.000 0.257 𝜆𝑚𝑎𝑥 = 4.241, 𝐶𝐼 = 0.065, 𝐶𝑅 = 0.072 ijahp article: ohoitimur, krejci, raco, raton, taroreh/ pastoral strategic planning priorities for the church: case study of the vicariate episcopal of tonsea of the diocese of manado international journal of the analytic hierarchy process 427 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.674 the results of the study are presented in table 5. the most important factors in the selection of the program priorities are: 1) the ministry of word and the sacrament, 23.8%; 2) preserving the treasury of faith, 20.1%; 3) fellowship and leadership, 19.7%; 4) promoting the dignity of the laity, 14.4%; 5) catholic education, 11.9% and; 6) managing the church property, 10.2%. the sub-criteria ‘organizing pastoral structure’ was the highest (7.6%), followed by ‘upgrading the catechetic program’ (7.2%) and ‘the role model and competency of liturgy leadership’ (7.0%). the cr calculation uncovered that the inconsistency values of all of the criteria and subcriteria were less than 0.1 (tables 3 and 4). this means that the consistency of the level of satisfaction and evaluation within the matrix of the criteria and sub-criteria is acceptable, the judgments are trustworthy, and the results obtained are considered reliable. ijahp article: ohoitimur, krejci, raco, raton, taroreh/ pastoral strategic planning priorities for the church: case study of the vicariate episcopal of tonsea of the diocese of manado international journal of the analytic hierarchy process 428 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.674 table 5 value of the criteria and sub-criteria of the vicariate episcopal of tonsea priority program value of priority of criteria ( wc) value of priority of sub-criteria (wsc) overall priorities of the subcriteria (wc * wsc) fellowship & leadership 0.197 organize pastoral structure (fl1) 0.387 0.0762 animate the spirituality of church based groups (fl2) 0.318 0.0626 promoting kinship and fraternity (fl3) 0.295 0.0581 sum 1.000 0.1970 preserving the treasury of faith 0.201 strengthening the system and the role of catechetic commission (tf1) 0.285 0.0571 upgrade the catechetic program (tf2) 0.359 0.0720 arrange the territory and categorical unit (tf3) 0.180 0.0361 upgrade the catechetic facilities (tf1) 0.176 0.0353 sum 1.000 0.201 ministry of the word and sacrament 0.238 reinforce the commission of liturgy (mws1) 0.205 0.0488 role model and competency of liturgy leadership (mws1) 0.295 0.0702 complete the liturgy facilities (mws1) 0.168 0.0400 education & comprehension of liturgical music (mws1) 0.183 0.0435 education & comprehension of enculturation (mws1) 0.149 0.0354 sum 1.000 0.238 promoting the dignity of the laity 0.144 fostering integral family development (dl1) 0.197 0.0283 empowering lay apostolate (dl2) 0.147 0.0211 unity and solidarity between the lay and the ordained-servant (dl3) 0.130 0.0187 upgrade the socio-economic quality of the people (dl4) 0.176 0.0253 reinforce the partisanship for the poor (dl5) 0.160 0.0230 promoting the quality of relationship with people of other religious beliefs (dl6) 0.099 0.0142 taking sides in the preservation of the environment (dl7) 0.091 0.0131 sum 1.000 0.144 ijahp article: ohoitimur, krejci, raco, raton, taroreh/ pastoral strategic planning priorities for the church: case study of the vicariate episcopal of tonsea of the diocese of manado international journal of the analytic hierarchy process 429 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.674 catholic education 0.119 catholic education to those studying at non-catholic schools (ce 1) 0.379 0.0451 attention to early age education (ce 1) 0.389 0.0463 develop non-formal education and training skills (ce 1) 0.232 0.0276 sum 1.000 0.1190 managing the church property 0.102 expand & synergize the competency system (mcp 1) 0.243 0.0247 build up a transparent financial information system (mcp 2) 0.284 0.0289 build up the asset management system (mcp 3) 0.216 0.0219 make good use of church property (mcp 4) 0.257 0.0261 sum 1.000 0.102 6. discussion the primary goal of the researchers was to determine the program priorities in order to implement the strategic plan of the diocese. the need to set priorities was paramount due to the limited resources in the parishes. this study offered a decision hierarchy that provided a robust and valid quantitative methodology that was reliable and workable. the use of the ahp provided researchers with a valuable method to assess the relative importance of the different variables. additionally, the ahp format provided several benefits beyond establishing the priorities. first, the use of the parish priests as the experts in the study offered them an opportunity to be involved in setting priorities that they would eventually be responsible for implementing. second, an unanticipated benefit was that the various programs were communicated to the priests as part of the ranked pairings, which gave them a greater insight into the projects that they would be implementing. third, by serving as experts the priests assumed a greater level of ownership of the various programs since they were involved in the implementation process rather than having the program being dictated to them from above. there was a sense of urgency for the priests because the strategic program of the vicariate and the budgets needed to be submitted and validated by the bishop in october 2019. the determination of the program priorities by the priests was critical because each of the vicariates had unique problems and challenges. fourth, the ahp methodology provided a simple means of prioritizing the strategic plan using a proven method that was easy to understand and required minimal time, travel, or statistical skills. the results in table 4 provide an implementation map for the pastoral strategic program by establishing the priorities in a clear and understandable list. the ministry of the word and sacrament (23.8%) was the leading factor. the pastoral strategic program should implement the ministry of the word and sacrament plan as the first priority since this factor was identified as the most important need. this factor should be viewed by all as the most critical element for a successful and efficient pastoral program. based on the qualitative analysis of the need by the community, each criterion and sub-criterion discussed previously could then be evaluated and the priorities set for their implementation. ijahp article: ohoitimur, krejci, raco, raton, taroreh/ pastoral strategic planning priorities for the church: case study of the vicariate episcopal of tonsea of the diocese of manado international journal of the analytic hierarchy process 430 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.674 in exploring the priorities of the programs of the vicariate episcopal of tonsea, the application of the ahp had two principal strengths. 1. due to the limited resources available and travel restrictions for doing extensive fieldwork, it was determined that it was not possible to conduct a statistically representative survey. to this end, the ahp provided an approach where the opinions of key informants (parish priests) could be successfully explored in a rigorous and robust manner within these restrictions. 2. the second benefit of using ahp relates the design structure associated with this method rather than its application. through the development of the hierarchy, it is possible to group all of the criteria and sub-criteria into a single tier or key categories. this proved to be an effective communication tool as the implementation plan was developed. the ahp was suitable and useful for this study; however, it also has limitations. grouping constraints into a manageable number of key categories resulted in the loss of some detail. inconsistency in the data could potentially be an issue if the participants or respondents lacked comprehension regarding the method. in addition, some participants struggled with differentiating priorities in the pairwise comparison, which resulted in all of the criteria and sub-criteria being equally important. several respondents had not encountered the ahp method prior to being interviewed, but were provided with a work example and the opportunity to ask questions prior to completing the questionnaire. while the findings remain valid, these issues identified some potential limitations of the ahp and highlighted the need to consider the suitability of its application, particularly with respondents who were unfamiliar with this method. 7. conclusion this study aimed to apply the ahp method as a proven methodology to determine the strategic planning priorities of the vicariate of tonsea pastoral program. to achieve the objective, the goals, criteria and sub-criteria of the strategic plan were set up in an ahp matrix and pairwise comparisons were completed by the experts. the results of this study suggested that the ahp method functioned as a significant tool in determining the strategic planning priorities of the vicariate of tonsea pastoral program. a review of the literature showed that this was the first time the ahp had been used in a study related to church affairs. it proved to be very helpful to the church leaders and decision makers, and catholic parish priests of the vicariate. the ahp provided a methodology to prioritize the strategic plan criteria and sub-criteria that would otherwise have been defined based on the feelings of the parish priests. using the ahp in this study eliminated subjectivity and biases in evaluating the strategic planning priority. the concept of the scale of assessment was also introduced to provide more objective and reliable decision criteria. the findings from this study suggest several management implications. ijahp article: ohoitimur, krejci, raco, raton, taroreh/ pastoral strategic planning priorities for the church: case study of the vicariate episcopal of tonsea of the diocese of manado international journal of the analytic hierarchy process 431 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.674 1. criteria weights, which were derived from the study, could be the guidelines on which management decisions are based. the criteria with the highest weights should receive the greatest attention from the management. 2. the attention given to the criteria and sub-criteria with the higher weights does not mean that the criteria and sub-criteria with lower weights should be neglected. this may suggest that the study should be repeated on an annual basis to determine any changes in the needs of the parishes. 3. interpretation of the findings may differ from person to person, so caution is needed when applying the study’s findings to practice. a narrative that supports the analysis could provide background into the thought processes, and ranking methodologies could be useful in the implementation phase. however, this research has identified two potential limitations of the ahp including the loss of detail that results from undertaking pairwise comparisons with broad categories, and the issue of inconsistent responses. in the case of the former, the solution is to adopt the use of a semi-structured interviewing plan. the researchers should conduct a combination of the ahp with semi-structured interviewing. in conclusion, it is recommended that these issues be given full consideration in the design and application of similar studies in the future. the researchers also recommend conducting future studies on other vicariates of the diocese of manado to determine whether the results would be similar to or different from this study. it is recommended that a similar study using different methods such as fuzzy ahp be conducted in the future. ijahp article: ohoitimur, krejci, raco, raton, taroreh/ pastoral strategic planning priorities for the church: case study of the vicariate episcopal of tonsea of the diocese of manado international journal of the analytic hierarchy process 432 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.674 references ahmed, a., bwisa, h. m., & otieno, r. o. (2014). strategic decision making : process, models, and theories. business management and strategy, 5(1), 78-104. doi: https://doi.org/10.5296/bms.v5i1.5267. arasa, r., & k’obonyo, p. (2012). the relationship between strategic planning and firm performance. international journal of humanities and social science, 2(22), 201–213. athapaththu, h. k. s. h. (2016). an overview of strategic management : an analysis of the concepts and the importance of strategic management. international journal of scientific and research publications, 6(2), 124–127. bakhmutsky, e. (2013). strategic planning for a local church. paper for baptist bible seminary. clarks summit, pennsylvania. budimčević, k. (2018). measuring performance of middle east airlinesahp approach. international journal of the analytic hierarchy process, 10(3), 391–408. doi: https://doi.org/10.13033/ijahp.v10i3.595 duleba, s., & moslem, s. (2019). examining pareto optimality in analytic hierarchy process on real data: an application in public transport service development. expert systems with applications, 116, 21–30. doi: https://doi.org/10.1016/j.eswa.2018.08.049. fabjanowicz, m., bystrzanowska, m., namieśnik, j., tobiszewski, m., & płotkawasylka, j. (2018). an analytical hierarchy process for selection of the optimal procedure for resveratrol determination in wine samples. microchemical journal, 142, 126–134. doi: https://doi.org/10.1016/j.microc.2018.06.028. golden, b. l., wasil, a., harker, p. t. (1989). the analytic hierarchy process: applications and studies. springer. kagumu, j, g., & njuguna, r. (2016). organizational factors influencing strategy implemenation. international journal of current business and social sciences, 1(5), 350–385. li, h., ni, f., dong, q., & zhu, y. (2018). application of analytic hierarchy process in network level pavement maintenance decision-making. international journal of pavement research and technology, 11(4), 345–354. doi: https://doi.org/10.1016/j.ijprt.2017.09.015. morgan, r. (2017). an investigation of constraints upon fisheries diversification using the analytic hierarchy process (ahp). marine policy, 86, 24–30. doi: https://doi.org/10.1016/j.marpol.2017.05.037. mutia, p. m., k’aol, g. o. k., & katuse, p. (2016). setting the strategic direction and its influence on church growth in kenya. international journal of humanities and social sciences, 8(1), 32–41. https://doi.org/10.5296/bms.v5i1.5267 https://doi.org/10.1016/j.eswa.2018.08.049 https://doi.org/10.1016/j.microc.2018.06.028 https://doi.org/10.1016/j.ijprt.2017.09.015 https://doi.org/10.1016/j.marpol.2017.05.037 ijahp article: ohoitimur, krejci, raco, raton, taroreh/ pastoral strategic planning priorities for the church: case study of the vicariate episcopal of tonsea of the diocese of manado international journal of the analytic hierarchy process 433 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.674 nguyen, h. l., fong, c., & ho, c. (2010). using analytical hierarchy process in decision analysis the case of vietnam state securities commission. ibusiness, 2, 139– 144. doi: https://doi.org/10.4236/ib.2010.22017. oosthuizen, j., & lategan, l. (2016). strategic management of the church as challenge and reality : journal for christian scholarship, 4, 277–297. promentilla, m. a. b., aviso, k. b., lucas, r. i. g., razon, l. f., & tan, r. r. (2018). teaching analytic hierarchy process (ahp) in undergraduate chemical engineering courses. education for chemical engineers, 23, 34–41. doi: https://doi.org/10.1016/j.ece.2018.05.002. raco, j. r., & tanod, r. h. m. (2014). the phenomenological method in entrepreneurship. international journal of entrepreneurship and small business, 22(3), 276–285. doi: https://doi.org/10.1504/ijesb.2014.063776 saaty, t. l. (2008). decision making with the analytic hierarchy process. international journal of services sciences, 1(1), 83–98. thompson, a., peteraf, m.,gamble, j., strickland iii, a. (2016). crafting and executing strategy: the quest for competitive advantage, 21 ed. mcgraw hill education. untu, b. e. r. (2018). the strategic planning of the diocese of manado (original text in indonesia). manado, indonesia: manado press. wanyoike, j, k., & orwa, b, h. (2016). strategic management practices and sustainability of faith-based organisations: an empirical study of anglican, diocese of thika joseph kuria wanyoike. international journal of education and research, 4(1), 169–186. https://doi.org/10.4236/ib.2010.22017 https://doi.org/10.1016/j.ece.2018.05.002 https://doi.org/10.1504/ijesb.2014.063776 ijahp article: ohoitimur, krejci, raco, raton, taroreh/ pastoral strategic planning priorities for the church: case study of the vicariate episcopal of tonsea of the diocese of manado international journal of the analytic hierarchy process 434 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.674 appendix figure 1 working area of the vicariate episcopal of tonsea ijahp article: mimović, krstić, savic/serbia joining the european union: an ahp model for forecasting the accessing date international journal of the analytic hierarchy process 2 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.616 serbia joining the european union: an anp model for forecasting the accessing date predrag miroslav mimović university of kragujevac, faculty of economics mimovicp@kg.ac.rs ana krstić university of kragujevac, faculty of economics anakrstic@kg.ac.rs jovana savic university of kragujevac, faculty of economics jokasavic92@gmail.com abstract for the republic of serbia to acquire full membership in the european union depends on a large number of factors, and it is very difficult to predict whether the country will attain membership and when it will happen. this paper first describes the analytic network process (anp), which is then applied to predict the most probable date of the republic of serbia's entry into the european union taking into account the economic, political and legal conditions that the republic must fulfill in the stabilization and association process. the model is based on negotiating chapters that are considered crucial in the process of eurointegration and their interactions and relationships. the work is an extensive and improved version of the paper titled "forecasting the accession of the republic of serbia to the european union by using the analytic network process" (krstić et al., 2018). it is based on the complex network model of the analytic network process, which encompasses not only multiple clusters and their elements, as well as their external and internal networks of influence, but in particular the impact of alternatives on relevant factors. all calculations were performed using the superdecisions software package. keywords: eurointegration; forecasting; accession date; analytic network process, republic of serbia 1. introduction modern business conditions are characterized by high uncertainty and complexity, which is why the decision making process is rather difficult and focused not on the best, but the optimal solution. the choice of an optimal solution depends on a large number of heterogeneous criteria and their interdependencies. in the conditions of multicenter decision making, many methods can be used, and one of the most famous is the analytic hierarchy process (ahp). this method is based on a clear hierarchical structure and division of goals, criteria and decision making alternatives, as well as neglecting the impacts and relationships that can exist between criteria, which can lead to decisions that are not optimal. in order to expand the scope of application of the analytic hierarchy process on the decision problems an extension called the analytic network process (anp) which deals with conditions of interdependence and mailto:mimovicp@kg.ac.rs mailto:anakrstic@kg.ac.rs mailto:jokasavic92@gmail.com ijahp article: mimović, krstić, savic/serbia joining the european union: an ahp model for forecasting the accessing date international journal of the analytic hierarchy process 3 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.616 feedback between and within clusters was developed. many decision problems cannot be structured in the form of a hierarchy precisely because of the existence of interactions and dependencies between the elements at the higher and lower levels of the hierarchy. the anp can accommodate these interactions because the elements are structured in the form of a network (saaty, 2001a). this approach allows us to more fully comprehend and encompass complex relationships, especially in situations characterized by risk and uncertainty with two-way pairwise comparisons. in light of this, this paper examines the possibility of applying the analytic network process to forecast the date that the republic of serbia could acquire full membership into the european union (eu). the european union is one of the most powerful regional communities today and is characterized by numerous political, economic and other complex problems stemming from the relationships between member states and candidates which affect the survival of the european union. therefore, the acquisition of the status of full member depends not only on how quickly the reforms are harmonized between the regulations in the republic of serbia and the legal requirements and heritage of the european union, but also how satisfied the other countries are with the proposals of the republic of serbia as a candidate for membership. the aim of this research is to use the analytic network process to predict the most probable date that the republic of serbia can enter the european union by examining the economic, political and legal conditions which the republic must fulfill in the process of stabilization and association. in accordance with the defined object and goal, the paper starts with the assumption that the implementation of the analytic network process, respecting the economic, political and legal conditions that must be included in the negotiating chapters, will enable us to accurately forecast the date of the republic of serbia's entry into the european union. the paper is organized as follows: the first section of the paper includes a review of the literature in which decision and prediction problems are presented that use multicriteria prediction methods or the analytic hierarchy process, or its extensions, the analytic network process. in the second section of the paper, the research methodology is presented. first, we describe the analytic network process, and then the prediction problem and the construction of the model itself. the results of the model are presented in the third section, while the last section contains the concluding observations, work restrictions and guidelines for the future research. 2. literature review the anp has a wide application for solving decision making problems that involve risk, uncertainty, dynamics and complexity. the anp is widely applied as a predictive instrument based on the distribution of the relative probabilities of future outcomes. mimović (2012) studied the prediction of sales of a new car model, and the fiat 500l showed that the anp can be used very successfully to structure the influence of various factors on the final outcome of the prediction process. sales forecasting using the anp was carried out by voulgaridou et al. (2009) who envisaged the volume of sales of a new product (the new edition of the book), as well as shih et al.(2012) who researched the scope of sales of printers in taiwan. ozorhon et al. (2007) applied the anp to predict the performance of international construction joint ventures. using the anp model, the authors examined the determinants of the performance of these joint ventures and came to the conclusion that the most important determinants of their success are the relationships between partners, ijahp article: mimović, krstić, savic/serbia joining the european union: an ahp model for forecasting the accessing date international journal of the analytic hierarchy process 4 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.616 structural factors and partner compliance, primarily cultural. the anp proved to be a good model for predicting the probability of failure in the business of construction companies in turkey, including internal and external factors that determine the current business situation of the company and the links between them (ozorhon et al. 2007; dikmen et al. 2010). in the field of economic forecasting, the anp method is often used to predict the likelihood of a financial crisis or predict the recovery of the economy (niemira & saaty, 2004; saaty & vargas, 2006, 2013; blair et al. 2010; azis, 2010). interesting research by adamus (2010) and saaty and vargas (2013) was carried out in connection with forecasting poland's entry into the eurozone. the results showed that the optimal alternative for poland was late entry (after 2011). political problems are also common in the application of the anp method. saaty and vargas (2006, 2013) used this method to deal with forecasting possible ways to overcome the conflict between china and taiwan and america's response to north korea's nuclear threats. further, wu et al. (2014) combined wireless sensor networks (wsn) and the anp to reliably predict and monitor disasters, and to build the hill slope prediction model. historical disaster data is analyzed using the anp model, enabling users to predict and prevent disasters. wey (2014) used the anp in environment planning support systems and showed that when historical data are lacking and a broad spectrum of social impact was involved, the anp, with the capacity to manage dependence and feedback among the factors, could serve as a tool to forecast outcomes by using expert judgment. these are obtained from the limit super matrix of the anp that represents forecasts for the next period. chung et al. (2016) applied a two-stage anp to develop a predictive model for strategy execution problems. this study sought to help the case corporation predict possible types of problems encountered in the strategy execution stage, establishing corresponding executive methods to the types of problems that could help the case corporation prepare for strategy execution, and focusing on the strategy execution point that elevates corporate executive ability corresponding directly to the corporation’s international competitiveness. they showed that the anp provides a systematic structure and clear network system, affording different element indicators but related weights, by allowing decision makers to make better decisions in the application of strategic planning and resource allocation. popa et al. (2018) applied the anp as an alternative to econometric tools for predicting the market share in the farming industry, in which the market share is described as a network of nodes and clusters. this showed that the anp model is a powerful instrument which can provide a deep understanding of the research problem which was how different factors affect the market share in the plant protection products industry. 3. methodology 3.1 analytic network process due to the limited use of the ahp method for decision making in situations of risk and uncertainty because of its inability to encompass complex relationships, thomas l. saaty (2001c) developed the anp method as an extension of the ahp method. the essential concept of the anp method is the impacts i.e. the interactions that exist between elements and clusters, which can be inner (between cluster elements) and outer (between clusters). in fact, this indicates that in this method it is not necessary to clearly define the levels of hierarchy, but there is a network consisting of elements and influences. therefore, the anp model consists of two parts. the first part consists of a control hierarchy or a network of criteria and sub-criteria, which control the interactions in the system being studied, while the other part represents a network of influences between elements and clusters. it is important to note that one anp ijahp article: mimović, krstić, savic/serbia joining the european union: an ahp model for forecasting the accessing date international journal of the analytic hierarchy process 5 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.616 model can have more than one network of influences. the use of the anp method often involves reliance on the control hierarchy, i.e. the control system of profit, costs, chances and risks. the synthesis of results obtained from the control systems is first performed by calculating the profit margin and the chances, costs and risks for each individual alternative, and then the result is normalized at the level of all alternatives in order to achieve the best outcome (saaty, 1999). the procedure for applying the anp method involves the following five steps: 1. the decomposition of the problem – a decision problem is decomposed into its main components. 2. the cluster formation for the evaluation – after defining the decision making objectives, it is also necessary to generate clusters for the purpose of evaluation by a criterion, sub-criterion (if it is possible) and cluster alternative. 3. the structuring of the anp model – the anp is applied to different decision making problems in the field of marketing, health, politics, military issues, society, predictions, etc. their accuracy of forecasting has been proven in impressive applications in the field of economic trends, sports events etc. 4. a paired comparison and prioritization – in this step, it is necessary to compare the pairs of elements of decision making as well as the synthesis of priorities for all of the alternatives. the estimations are made with a fundamental scale of 1-9 (table 1), which the comparative study showed simulates human thinking most adequately. 5. the sensitivity analysis of the solution – it is finally possible to make a decision and carry out a sensitivity analysis in terms of the impact which, according to the importance of some criteria or sub-criteria, a final outcome has on a given solution; it is also possible to determine how big or small these indicators are through an analysis. ijahp article: mimović, krstić, savic/serbia joining the european union: an ahp model for forecasting the accessing date international journal of the analytic hierarchy process 6 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.616 table 1 scale of comparison 1–9 intensity of importance definition explanation 1 equal importance two activities contribute equally to the objective 3 moderate importance experience and judgment slightly favor one activity over another 5 strong importance experience and judgment strongly favor one activity over another 7 very strong or demonstrated importance an activity is favored very strongly over another; its dominance is demonstrated in practice 9 extreme importance the evidence favoring one activity over another is of the highest possible order of affirmation 2, 4, 6, 8 mean value of two estimates a compromise is needed reciprocals of the above if activity i has one of the above nonzero numbers assigned to it when compared with activity j, then j has the reciprocal value when compared with i source: saaty, t. l. & kearns p. k. (1985) in relation to the ahp, the anp method allows each decision to appear in the form of a network rather than a strict hierarchy. hierarchical decisions can be subjective and predetermined due to the imposed structure, while matrix decisions, whose structure includes dependencies and feedback, represent the world more realistically. this makes the anp method a more effective tool in relation to ahp for deciding in practice (saaty, 2001b). additionally, the anp has been successfully applied in many areas of forecasting since it allows for the quick incorporation of feedback and simple comparison with actual results. however, the deficiencies of this model must be mentioned, and include its complexity, the fact that comparison of clusters can often be unclear and confusing, the fact that the process takes longer than in the ahp method, and the imprecise results that can be obtained. 3.2 problem description and the construction of anp model in this paper, we use the anp to predict the date of the republic of serbia's entry into the european union, whose model is based on negotiating chapters that are grouped into three clusters including economic, political and legal conditions. fulfilling these conditions is considered crucial for the european integration process for many reasons. according to the national convention (2016) on the accession of the republic of serbia to the european union, the harmonization of the legislation of the candidate country with the regulations of the union implies fulfillment of the first, political criteria. the economic criteria involves the democratic organization of the state, stability of institutions, rule of law, protection of human and minority rights and existence of a functional market economy able to withstand the pressure of competition in the unified market of the european union. the political issue of special importance is the normalization of relations with kosovo, which, in the ijahp article: mimović, krstić, savic/serbia joining the european union: an ahp model for forecasting the accessing date international journal of the analytic hierarchy process 7 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.616 opinion of many analysts including the council of the european union, together with the chapter on the rule of law, is considered fundamental to the negotiation process. also, this chapter is one of the key aspects of foreign, security and defense policy. as a candidate, the republic of serbia is obliged to adopt the legal acquis of the european union, which is why the most important issue is the harmonization of domestic legislation with this law and the construction of institutions that will ensure its implementation. consequently, the most complex chapters of accession are chapters 23 and 24 which are dedicated to justice, human rights, freedom and security and are collectively called "the rule of law" (eu info center, 2014). regarding legal requirements, the competition chapter is one of the most demanding and complex in the negotiation process, and this is one of the chapters for which transitional deadlines are often sought, mostly due to the state aid policy (eu info center, 2015). related to this chapter is the chapter on the right of companies that defines the conditions that will allow economic entities equal treatment in the market of the european union. from the economic perspective, the republic of serbia has to take certain measures regarding the improvement of the investment climate and liberalization in foreign trade policy. according to the european commission, the economic problems of the republic of serbia are represented by a high budget deficit and the absence of more stringent financial controls, as well as poor conditions in the labor market and high unemployment. because trade integration with the european union has remained high, a smooth implementation of the transitional trade agreement has continued. an economic chapter of vital importance for the integration process relates to the principle of the free movement of goods since this is the principle on which the european union is based (eu info center, 2014). in the context of accession to the eurozone, an important chapter relates to the regulation of economic and monetary policy and the adoption of the euro as the national currency. another important economic aspect is the functioning of a customs union. without this the eu's common trade policy and development, its common agricultural market and the effective coordination of economic and monetary policies, would not be possible (eu info center, 2015). as for the definition of the accession date, jean-claude juncker, president of the european commission said in a 2017 mandate that there would be no enlargement of the european union until 2019. this indicates that there is no room for the entry of the republic of serbia before that period. the european union budget that is being prepared for the period 2021-2026 is an important indicator for entry, and the european union plans for expansion will be based on it. according to jadranka joksimović, the minister for eurointegration, the definition of a more precise time of entry is important for the planning of activities and budgets related to certain areas such as environmental protection, agriculture or food safety. bearing in mind this statement, the fourth cluster is defined, which includes the alternative periods of time of the republic of serbia's entry into the european union. based on the white paper on the future of europe: five scenarios, the description of the problem of forecasting and the republic of serbia’s process of eurointegration, an anp model was formed that included the following clusters and their elements (ec, 2017). a,b,c,d a https://ec.europa.eu/commission/future-europe/white-paper-future-europe/white-paper-futureeurope-five-scenarios_en b http://www.parlament.gov.rs/upload/documents/seio/nac_strat_undp_2006_gledanje.p dfhttp://cep.org.rs/images/cepgizkonf/policy_brief_pm_and_eu_neg_srb.pdf https://ec.europa.eu/commission/future-europe/white-paper-future-europe/white-paper-future-europe-five-scenarios_en https://ec.europa.eu/commission/future-europe/white-paper-future-europe/white-paper-future-europe-five-scenarios_en http://www.parlament.gov.rs/upload/documents/seio/nac_strat_undp_2006_gledanje.pdf http://www.parlament.gov.rs/upload/documents/seio/nac_strat_undp_2006_gledanje.pdf http://cep.org.rs/images/cepgizkonf/policy_brief_pm_and_eu_neg_srb.pdf ijahp article: mimović, krstić, savic/serbia joining the european union: an ahp model for forecasting the accessing date international journal of the analytic hierarchy process 8 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.616  a cluster of criteria, which includes groups of relevant factors that need to be taken into account when evaluating alternative future outcomes entry or non-eu entry dates, which includes: internal factors, external factors, costs factors and risk factors (european comission accesion criteria) e ;  cluster of internal factors, which includes sub-criterion of the internal factors criteria: compatibility of economic, legal-political and social system, internal political conflicts and relations with the former autonomous province of kosovo (with metohija). according to the publication published at the end of the first phase of negotiations, the so-called screening process in 2015, the conditions defined by negotiation chapters taken in this paper within this cluster are (eu info center, 2015): chapters that include the free movement of goods from one end of the european union to another; coordination of economic policies through joint planning of integration of the economy, stimulation of economic growth, securing jobs and competitiveness of the economy, independence of the national bank of serbia and adoption of the euro as the national currency; the abolition of duties at the national borders of the member states of the european union, the establishment of a uniform system of taxation of imports, the adoption of rules for combating all forms of smuggling, organized crime, terrorism, money laundering, the adoption of the "e-duties" system; improving the social protection system, increasing the educational level of the workforce, achieving high employment rates and social inclusion; rules on the establishment, registration, performance of activities, domestic and cross-border mergers and divisions of companies; rules for suppressing agreements to negotiate prices, limit production, development and investment or sharing markets and sources of supply, rules for preventing the abuse of a dominant position and mergers that may have an impact on competition in the market; reform the judiciary, fight against corruption, fundamental rights and citizens' rights; rules for border control, visa, migration, asylum, police cooperation, fight against organized crime, terrorism and human trafficking, cooperation in the field of drugs, customs cooperation and judicial cooperation in criminal and civil matters; rules for the prevention of arms trafficking, participation in civilian and military missions, the expansion of the diplomatic-consular network and conditions that the republic of serbia has to fulfill in relation to kosovo. regarding the regulation of relations with pristina, the european council (2015) defined the following conditions: the establishment of the community of serbian municipalities, the establishment of an electricity supply company in kosovo, the establishment of a telecommunications company, the termination of financing serbian structures and participation in elections, the payment of pensions to members c european council conclusions (2015). madrid european council, reproduced from the bulletin of the european communities, no. 12/1995. d european security strategy (2004). a secure europe in a better world, internet, 11/06/2004, http://www.iue.eu.int/cms3_fo/showpage.asp?id=391&lang=en e https://ec.europa.eu/neighbourhood-enlargement/policy/glossary/terms/accession-criteria_en http://www.iue.eu.int/cms3_fo/showpage.asp?id=391&lang=en https://ec.europa.eu/neighbourhood-enlargement/policy/glossary/terms/accession-criteria_en ijahp article: mimović, krstić, savic/serbia joining the european union: an ahp model for forecasting the accessing date international journal of the analytic hierarchy process 9 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.616 of the police, judiciary and civil protection regulation, respecting the provisions on official visits, regulating customs and controlling passageways, ensuring free entry of people from kosovo to the republic of serbia, ensuring kosovo's participation in regional initiatives, recognition of university diplomas and support to eulex-led trials;  external factors cluster consisting of geo-political factors, fatigue from eu enlargement, the security challenges facing the eu and the strengthening of conservative right forces in the eu;  the costs cluster, which consists of the costs of alignment with eu standards, norms and regulations, the reduction of budget revenues due to the loss of existing customs revenues, the negative current balance due to the liberalization of imports and opportunity costs in case of non-eu accession;  the risk cluster, which, in the case of joining the eu constitutes the risk of unequal exchange and exploitation in relations to developed eu member states, the risk of alienation and exploitation of natural resources, the risk of potential disproportions in economic development, the risk of increased competition and, in this regard, the risk of potential choking on certain branches of industry, as well as the risk of an increase in unemployment and, implicitly, as a consequence, the risk of strengthening social problems.  the alternative cluster containing four alternative outcomes which include entering the eu by 2025 at the latest, entering the eu between 2025 and 2030, joining the eu between 2030 and 2035, as well as the possibility that serbia will not become a member of the eu. obviously, there are generally two possible outcomes regarding the perspective of serbia's membership in the eu which are to become a member of the eu or not. differentiating the possible deadlines for joining the eu is not uniquely determined, but is the result of the interaction of several different factors, variables in time and space. the mentioned deadlines were those that most often appeared in the media, in the statements of eu officials and members of the government of the republic of serbia, and, of course, could be changed depending on many circumstances. interactions and relations that exist between and within clusters, which are important for the comparison of couples, can be determined based on insight into the description of the problem. these are the following relations:  the criteria are compared with the main objective of the model;  internal factors cluster elements are, as sub-criteria of the internal factors criteria, compared to each other on a scale of 1-9;  external factors cluster elements are, as sub-criteria of the external factors criteria, compared to each other on a scale of 1-9;  costs cluster elements are, as sub-criteria of the costs criteria, compared to each other on a scale of 1-9; ijahp article: mimović, krstić, savic/serbia joining the european union: an ahp model for forecasting the accessing date international journal of the analytic hierarchy process 10 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.616  risk cluster elements are, as sub-criteria of the risk criteria, compared to each other on a scale of 1-9;  internal factors cluster elements influence alternative outcomes in the sense that the dynamics and degree of their realization determine the outcome and dynamics of the eu accession process;  external factors cluster elements also affect alternative outcomes. geopolitical factors, as well as movements and trends within the eu itself, can significantly affect the final outcome of the eu accession process not only for serbia, but for other countries in the region. they can also have a dynamic determinant in terms of potential accession dates. in addition, some of these factors also have an impact on the elements of other clusters, as is the case with geo-political factors that can significantly affect not only the internal political life in serbia, depending on the movement of current global geopolitical relations, but also affect relations with kosovo;  elements of cluster costs affect alternative outcomes because the process of accession of any eu country has its own cost, whereby this price is reflected not only in different categories of costs associated with systemic alignments and potential losses in case of eu entry, but also in lost revenue in the case of not entering the eu. therefore, the costs cluster and the internal factors cluster also have one-way dependency;  risk cluster elements definitely influence alternative outcomes because the nature and degree of possible risks associated with entering the eu largely define the dynamics of association, but also the consciousness and mood of the citizens about to the idea of association. in addition to these dependencies, there is also feedback from the alternatives cluster to other clusters of the sub-criteria. namely, it is obvious that the decision to enter or not enter the eu, as well as the decision about the dynamics and date of accession can affect priorities of the internal factors cluster elements, costs factor and risk factor. the ability to join the eu earlier will certainly affect the acceleration of the dynamics and degree of systemic harmonization with eu standards, as well as the rising costs of alignment, and the speeding up of the process of resolving the problems of kosovo, etc., the accession to the eu will increase numerous potential risks from the integration of the economic system of the eu, on the other hand, giving up membership in the eu reduces the mentioned risks, but also carries a big opportunity cost. finally, the model also has a number of internal interdependencies. thus, in a external factors cluster, geo-political factors definitely affect the internal political opportunities in the eu. these factors include the fatigue of eu enlargement, security challenges and the strengthening of the conservative right. enhanced competition of european manufacturers can put pressure on domestic producers and extinguish production in certain branches of industry, and thus lead to an increase in unemployment and the aggravation of social problems. based on the factors discussed above, the anp model was constructed and is shown in figure 1. ijahp article: mimović, krstić, savic/serbia joining the european union: an ahp model for forecasting the accessing date international journal of the analytic hierarchy process 11 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.616 figure 1 anp model for forecasting the entry of serbia into the eu. source: authors the results of the model are shown in the tables below. table 2 matrix of criteria comparison costs factor risk factor internal factors external factors 7 3 5 costs factor ¼ 1/3 risk factor 3 inconsistency 0.04435 source: authors table 3 relative importance of the criteria in relation to the main goal of the model criteria relative importance of the criteria external factors 0.56829 costs factor 0.05912 risk factor 0.25230 internal factors 0.12029 ijahp article: mimović, krstić, savic/serbia joining the european union: an ahp model for forecasting the accessing date international journal of the analytic hierarchy process 12 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.616 table 4 relative importance subcriteria of the internal factors criteria name normalized by cluster relations with kosovo 0.31503 internal political conflicts 0.10009 compliance of legal-political system 0.21425 compliance of economic system 0.19582 compliance of social system 0.17481 table 5 relative importance subcriteria of the external factors criteria name normalized by cluster security challenges of the eu 0.23015 geo-political factors 0.61264 strengthening of the eu conservative right 0.09393 fatigue of eu enlargement 0.06329 table 6 relative importance subcriteria of the costs factors criteria name normalized by cluster negative current balance due to the liberalization of imports 0.07916 opportunity costs in case of non-eu accession 0.69699 reduction of budget revenues due to the loss of customs revenues 0.07827 costs of alignment with eu standards 0.14558 table 7 relative importance subcriteria of the risk factors criteria name normalized by cluster risk of alienation and exploitation of natural resources 0.14916 risk of unequal exchange and exploitation 0.06836 risk of potential disproportions in economic development 0.14282 risk of potential choking on certain branches of industry 0.08153 risk of increased competition 0.11403 risk of unemployment increase 0.20677 risk of aggravation social problems 0.23734 ijahp article: mimović, krstić, savic/serbia joining the european union: an ahp model for forecasting the accessing date international journal of the analytic hierarchy process 13 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.616 table 8 the likelihood of achievement and alternatives ranking alternatives raw normal ideal ranking before 2025 0.203559 0.425020 1.000000 1 between 2025 and 2030 0.091804 0.191682 0.450995 3 between 2030 and 2035 0.095800 0.200024 0.470624 2 not entering the european union 0.087777 0.183274 0.431213 4 in figure 1, the anp model of the prediction of the indicative date of serbia's entry into the eu is presented. its structure is made up of clusters, elements and connections between them. these links, represented by arrows, indicate the direction of influence between the model elements established when structuring and modeling the problem. comparison of pairs of model elements, clusters, or elements within the same cluster, or between different clusters, is normally done using the 1-9 scale (table 1). pair comparisons are basic to the ahp/anp methodology. when comparing a few factors, one can determine the rationale of the relative importance, preference, or probability of these factors depending on the need. this ratio is the ratio of two factors to be compared. comparison of factors, or sub-criterion within certain groups of conditions is carried out according to the 1-9 scale in relation to the factor that influences them from either the observed group of conditions (inner dependence) or from another group of conditions (outer dependence). this determines the relative importance of these factors. comparison of alternatives is also done on the 1-9 scale, but in terms of what is more likely to be achieved than in terms of the observed factor, which determines the priority of the alternative. in this case, it can be interpreted as its likelihood of achievement. once inputs are made for each segment of the model, the information is synthesized in order to achieve the general preference of alternative outcomes. this synthesis gives a report that ranks alternatives (outcomes) with respect to the general goal. the report may include a detailed ranking showing how each alternative is evaluated against each criterion. the results are shown in tables 3-8 after all the necessary comparisons have been made in accordance with theoretically based principles, taking into account the assumed inner and outer interdependencies within the cluster and between them, and after executing calculations using the superdecisions software package which was developed as software support for anp application. table 3 shows the relative importance of the criteria in the model. it is estimated that the highest relative importance in relation to the main objective of the model is an external factors criteria (the weight coefficient is 0.56829), then the risk factors (0.25230), the internal factors (0.12029) and the costs factors (0.05912). this result could be interpreted such that the final outcome of serbia’s eu accession process is highly influenced by the factors on which the republic of serbia has the lowest influence, and as such have the greatest significance. tables 4-7 show the relative importance of the sub-criterion within the criteria to which they belong. thus, among the internal factors cluster elements, it is estimated that the greatest relative importance, in the context of european integrations, is the sub-criteria ‘relations with kosovo’ (0.31503). among the external factors cluster elements, the greatest relative importance is the sub-criteria ‘geopolitical factors’ (0.61264), among the costs criteria the sub-criteria with the greatest importance is ‘opportunity costs of non-entry into the eu’ (0.69699), and among the elements of the risk factors sub-criteria the ijahp article: mimović, krstić, savic/serbia joining the european union: an ahp model for forecasting the accessing date international journal of the analytic hierarchy process 14 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.616 ‘risk of aggravation of social problems’ (0.23734) as a result of the impact of other risk factors. the obtained priorities of the alternatives, shown in table 8, are interpreted in terms of the probability of achieving the chosen alternatives. we see that the highest estimated probability of achievement is the accession of serbia to the eu by 2025 (42.5%), then the entry of serbia between 2030 and 2035 (20%), then between 2025 and 2030 (19.17%), and the probability that serbia will not enter the eu is 18.32%. the obtained results show a clear commitment to joining the eu, with the view that if the accession does not come to 2025, it will not be until at least 2030. the ideal column shows the results divided by the highest value, so that the highest ranking has a priority of 1, while the others are in the same proportion as in the normalized value column. when there are multiple decision makers, in order to find the final weights, i.e. priorities of the criteria and the ranks of the alternatives, the following formula is used to calculate the geometric mean (saaty & peniwati, 2008): k ik kk ki ww    1   i where wi represents the final weight of the i criterion, аnd ikw the relative weight of the i element calculated using k evaluator. in this case, instead of engaging external experts in the field of politics, law and economics, as well as individual comparisons to evaluate and calculate the final priorities of alternatives using a geometric environment, the authors, agreed on common assessments. the authors consider the approach to the construction and implementation of an anp model in this article to be measured and unpretentious. this is because the goal is not to answer questions that society as a whole is concerned with, but to point out the need and possibilities of the anp to analyze and solve such complex social problems. the application and methodology of decision making methods like the anp can coordinate and synthesize a multitude of information and deal with complexities. for these reasons, the observed model is structured as a general, formal framework for forecasting, which is subject to changes in individual elements of the problem of the so-called predetermination (prediction), and their relatively easy incorporation into the model. the goal of the forecasting and the obtained results indicate the high level of uncertainty present in the forecasting process, which only confirms the need for additional analysis of the sensitivity of the results to changes in the values of the key parameters of the model. this will allow the authors to obtain a more complete and comprehensive estimation of the target value. a sensitivity analysis of the results performed by changing the level of significance of the higher level elements, i.e. basic elements, may more or less significantly affect the order of importance and the assessment of the observed options. this shows the performance of the alternative in relation to each criterion, and how these alternatives are sensitive to the changes in the importance of the criteria. thus, from table 8, it can be seen that the growth of the relative importance of the criterion ‘relations with kosovo’ from 0.0001 to 0.9999 affects, to a certain extent, the likelihood of alternative outcomes. it increases the likelihood that there will be no entry into the eu (from 13.43% to 24.53%), and slightly reduces the probability of expected entry by 2025 (from 43.19% to 41.63%). there is a greater sensitivity to changing the relative importance of the sub-criteria ‘relates with kosovo’ shows the remaining alternative outcomes, whose likelihood of achievement decreases to a greater extent. it decreases the likelihood of entry into the eu between 2025 and 2030 from 20.46% to 17.53%, while the probability of joining the eu in the period from 2030 to 2035 decreases from 22.92% to 16.31%. ijahp article: mimović, krstić, savic/serbia joining the european union: an ahp model for forecasting the accessing date international journal of the analytic hierarchy process 15 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.616 the obtained results are fundamentally different from those derived from the previous simple model of forecasting serbia's entry into the eu (krstić, et al., 2018). in this model, the sub-criteria ‘relations with kosovo’ essentially influences the entry of serbia into the eu only in terms of deadlines, but will not stop the process, if the remaining presumed interdependencies of the factors are unchanged. how can this difference be explained? first of all, the observed model is much more complex than the previous simple model because it includes multiple clusters, nodes and direct feedback network interactions between them, which are the result of existing interdependencies. in addition, in the meantime, there were several events and circumstances that radically influenced the importance of relations with kosovo in the process of serbia's accession to the eu. first of all, the us administration has a more flexible attitude towards resolving the problem of kosovo's unrecognized independence, and brussels also has a more relaxed attitude, in which no deadlines exist for resolving the kosovo node. this, as well as other complicated political situations in the world, gave serbia more maneuvering space for political calculations and bargaining. this left the option for serbia to decide not to enter the eu in case of a lose-lose outcome, that is nothing is gained and everything is lost in terms of kosovo. in any case, when it comes to relations with kosovo, the dissolution depends very much on serbia itself, which is not the case with some other factors and circumstances. thus, if the influence of changing the relative importance of the subcriteria ‘unemployment increase’ is viewed, the probability of achieving alternative outcomes do not show a pronounced sensitivity to these changes. the growth of the relative importance of this sub-criteria leads to a slight increase in the likelihood of serbia entering the eu after 2030 from 18% to 22%, decreases the likelihood of entry between 2025 and 2030 from 19% to 18%, reduces the probability of entry by 2025 from 43% to 40%, while the probability of entry between 2030 and 2035 remains unchanged at 20% (table 9). similar sensitivity analyses can be performed for other sub-criteria in the model. table 9 the impact of changing the relative importance of the sub-criteria ‘relations with kosovo’ on the movement of achieving alternatives probability relations with kosovo not entering into the eu entering in the eu by 2025 entering the eu between 2025 and 2030 entering the eu between 2030 and 2035 0.0000 0.1343 0.4319 0.2046 0.2292 0.1667 0.1515 0.4295 0.2001 0.2190 0.3333 0.1691 0.4270 0.1954 0.2084 0.5000 0.1873 0.4245 0.1906 0.1976 0.6667 0.2061 0.4218 0.1857 0.1864 0.8333 0.2254 0.4191 0.1806 0.1749 1.0000 0.2453 0.4163 0.1753 0.1631 4. conclusion this paper presents the possibility of using the analytic network process methodology in the process of predicting the probability of alternative future outcomes. the specific example is of serbia’s entrance into the european union. bearing in mind that a large number of factors and their influence networks have to be taken into account for a serious analysis, its theoretical as well as methodological and practical implications can be considered. theoretical implications are reflected in ijahp article: mimović, krstić, savic/serbia joining the european union: an ahp model for forecasting the accessing date international journal of the analytic hierarchy process 16 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.616 the fact that the effective ability of the analytic network process to conceptually encompass all relevant factors from the prediction/decision-making context is confirmed. despite the usual limitations encountered when applying qualitative predictive methods (precision, lack of information, bias, price, etc.) the anp can serve as a satisfactory basis for the creative solution of decision-making issues in situations of growing complexity and uncertainty when a decision needs to be made quickly. presenting the use of an anp forecasting model in the field of eurointegration, which is extremely complex and uncertain, confirms its ability to be a good support in the forecasting process. in situations characterized by the pronounced dynamics and unpredictability of the environment and many factors that determine it, as is the case with the european union and serbia, prediction methods based on econometric models do not yield satisfactory results. social justification is reflected in the possibility of using the results of the research by the competent institutions of the republic of serbia for redefining or making new and better strategic decisions in the process of stabilization and association. the limitations of this research relate primarily to the fact that within certain sub-criteria it is possible to identify and evaluate more complex influence networks which would greatly increase the complexity of the model and the number of assessments that should be carried out. given the limited cognitive capacities of the decision makers, this would certainly influence the increase of subjectivity in the assessment. significant objectivity could be achieved by including a group of experts in the areas relevant to the eu integration process in the comparison process. then after obtaining the individual ranking lists of alternatives and their probability of achievement, one could use a geometric mean to obtain a unique and objective ranking list of alternative outcomes and a better assessment of their probability. also, clusters cover only some negotiating chapters which may have an impact on the obtained results. the proposed decision support system, based on the anp, forms a flexible and systematic framework for forecasting. the anp qualitatively improves formal prediction models by allowing more consistent and systematic generation of additional factors and alignment of exogenous variables. the anp-based approach to forecasting allows decision makers to take into account both quantitative and qualitative variables. by structuring the network, the relationships between factors can be defined and analyzed effectively. in the priority setting process, qualitative and subjective assessments may be included by several people, as well as quantitative data from different sources. by using the presented approach, decision makers are not limited to past data, as assumptions and predictions about the future development of factors involved, and future actions of actors involved in the hierarchy can be made. the anp proved to be a methodology capable of producing results that match the perceptions and expectations. of course, this does not imply uncritical acceptance of the anp. auto correction is an essential component of scientific thinking and only through continuous practical testing in terms of endpoints and outcomes of the prediction process will it be possible to see the true predictive abilities of the anp. ijahp article: mimović, krstić, savic/serbia joining the european union: an ahp model for forecasting the accessing date international journal of the analytic hierarchy process 17 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.616 references adamus, w. (2010). when shall poland enter the euro zone? decision making with the anp. the school of business, 1, 31-52. azis, i. j. (2010). predicting a recovery date from the economic crisis of 2008. socioeconomic planning sciences, 44(3), 122-129. doi: https://doi.org/10.1016/j.seps.2010.03.003 blair, a. r., mandelker, g. n., saaty, t. l., & whitaker, r. (2010). forecasting the resurgence of the us economy in 2010: an expert judgment approach. socioeconomic planning sciences, 44(3), 114-121. doi: https://doi.org/10.1016/j.seps.2010.03.002 chung, c-c., chao, l-c., lou, s-j. (2016). constructing the predictive model for strategy execution problems -the application of the two-stage analytic network process. journal of information hiding and multimedia signal processing, 7(1), 184197. dikmen, i., talat birgonul, m., ozorhon, b., & egilmezer sapci, n. (2010). using analytic network process to assess business failure risks of construction firms. engineering, construction and architectural management, 17(4), 369-386. doi: https://doi.org/10.1108/09699981011056574 eu info centar. (2014). negotiation chapters 35 steps towards the european union, http://euinfo.rs/srbija-i-eu/ eu info centar. (2015). negotiation chapters 35 steps towards the european union. drugo, revised edition: http://euinfo.rs/srbija-i-eu/ european council. (2015). normalisation of relations between serbia and kosovo, chapter 35: other issues. conference on accession to the european union–serbia– brussels, 1-8. european council conclusions (2015). madrid european council, reproduced from the bulletin of the european communities, no. 12/1995. european security strategy (2004). a secure europe in a better world, internet, 11/06/2004, http://www.iue.eu.int/cms3_fo/showpage.asp?id=391&lang=en krstić a., savić, j., mimović, p. (2018). forecasting the accession of the republic serbia to the european union by using analytic network process, xiii balkan conference on operational research, balcor 2018, 25-28.may, belgrade, proceedings, mladenović, n., martić, m., vidović, m., urošević, d. 243-250, isbn: 978-86-80593-64-7 mimović, p. (2012). application of anp in forecasting automobile sales of fiat 500l, economic horizons, 14(3), 169-179. doi: https://doi.org/10.5937/ekonhor1203165m http://euinfo.rs/srbija-i-eu/ http://euinfo.rs/srbija-i-eu/ http://www.iue.eu.int/cms3_fo/showpage.asp?id=391&lang=en ijahp article: mimović, krstić, savic/serbia joining the european union: an ahp model for forecasting the accessing date international journal of the analytic hierarchy process 18 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.616 niemira, m. p., & saaty, t. l. (2004). an analytic network process model for financial-crisis forecasting. international journal of forecasting, 20(4), 573-587. doi: https://doi.org/10.1016/j.ijforecast.2003.09.013 ozorhon, b., dikmen, i., birgonul, m. t. (2007). using analytic network process to predict the performance of international construction joint ventures. journal of management in engineering, 23(3), 156-163. popa, a., hadad, s., paiusan, r. & nastase, m. (2018). a new method for agricultural market share assessment. sustainability, 11(88), 1-13. doi: https://doi.org/10.3390/su11010088 saaty, t.l., kearns, k. p. (1985). analytical planning: the organization of systems, the analytic hierarchy process series. pittsburgh: rws publications. saaty, t. l. (1999). fundamentals of the analytic network process. proceedings of the 5th international symposium on the analytic hierarchy process, 12-14. saaty, t. l. (2001a). analytic network process. encyclopedia of operations research and management science, 28-35. doi: https://doi.org/10.1007/1-4020-0611-x_32 saaty, t. l. (2001b). decision making with the analytic network process (anp) and its “super-decisions” software the national missile defense (nmd) exanple. isahp 2001, berne, switzerland, 365-382. saaty, t. l. (2001c). decision making with dependence and feedback: the anp. pittsburgh: rws publications. saaty, t. l., & vargas, l. g. (2006). decision making with the analytic network process. economic, political, social and technological applications with benefits, opportunities, costs and risks. new york: springer. saaty, t.l., peniwati, k. (2008). group decision making: drawing out and reconciling differences. pittsburgh, pa: rws publications. saaty, t. l., & vargas, l. g. (2013). decision making with the analytic network process. economic, political, social and technological applications with benefits, opportunities, costs and risks. 2nd edition. new york: springer. shih, h. s., lee, e. s., chuang, s. h., & chen, c. c. (2012). a forecasting decision on the sales volume of printers in taiwan: an exploitation of the analytic network process. computers & mathematics with applications, 64(6), 1545-1556. doi: https://doi.org/10.1016/j.camwa.2011.12.082 vargas, l. g. (1990). an overview of the analytic hierarchy process and its applications. european journal of operational research, 48(1), 2-8. voulgaridou, d., kirytopoulos, k., & leopoulos, v. (2009). an analytic network process approach for sales forecasting. operational research, 9(1), 35-53. doi: https://doi.org/10.1007/s12351-008-0026-2 ijahp article: mimović, krstić, savic/serbia joining the european union: an ahp model for forecasting the accessing date international journal of the analytic hierarchy process 19 vol. 11 issue 1 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i1.616 wey, wm. (2014). the application of dynamic network process to environment planning support systems. international journal of environmental and ecological engineering, 8(8), 914-917. wu, c-i., kung, h-y., chen, c-h., kuo, l-c. (2014). an intelligent slope disaster prediction and monitoring system based on wsn and anp. expert systems with applications, 41, 4554-4562. doi: https://doi.org/10.1016/j.eswa.2013.12.049 internet sources http://www.parlament.gov.rs/upload/documents/seio/nac_strat_undp_2006_gl edanje.df http://cep.org.rs/images/cepgizkonf/policy_brief_pm_and_eu_neg_srb.pdf https://ec.europa.eu/neighbourhood-enlargement/policy/glossary/terms/accessioncriteria_en https://ec.europa.eu/commission/future-europe/white-paper-future-europe/whitepaper-future-europe-five-scenarios_en http://www.parlament.gov.rs/upload/documents/seio/nac_strat_undp_2006_gledanje.df http://www.parlament.gov.rs/upload/documents/seio/nac_strat_undp_2006_gledanje.df http://cep.org.rs/images/cepgizkonf/policy_brief_pm_and_eu_neg_srb.pdf https://ec.europa.eu/neighbourhood-enlargement/policy/glossary/terms/accession-criteria_en https://ec.europa.eu/neighbourhood-enlargement/policy/glossary/terms/accession-criteria_en https://ec.europa.eu/commission/future-europe/white-paper-future-europe/white-paper-future-europe-five-scenarios_en https://ec.europa.eu/commission/future-europe/white-paper-future-europe/white-paper-future-europe-five-scenarios_en ijahp article: huang/ management strategies for taiwan shimen reservoir catchment area: perspectives of collaborative planning international journal of the analytic hierarchy process 121 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.244 management strategies for taiwan shimen reservoir catchment area: perspectives of collaborative planning hun-feng huang 1 * department of public administration and policy national taipei university new taipei 23741,taiwan, r.o.c. & water resources management and policy research center tamkang university new taipei 25137, taiwan, r.o.c. e-mail: hhf@water.tku.edu.tw hen-chin chen department of public administration and policy national taipei university new taipei 23741,taiwan, r.o.c. e-mail: hcc31699@ms41.hinet.net sean liu water resources management and policy research center tamkang university new taipei 25137,taiwan, r.o.c. e-mail:sliu@water.tku.edu.tw abstract in taiwan there are 113 reservoir catchment areas delineated as public water source protection areas covering 25% of the country’s land area. many important reservoir areas are vulnerable to global climate change, and water resources are already under increasing ecological, societal, hydrological and economic pressures. in this research, a case study of shimen reservoir catchment area management was constructed. the research utilizes the collaborative planning model. thirteen factors under the categories of “land use”, “water quality protection”, “quantity of water supply and demand”, “water and soil conservation”, and “laws and institutions” were identified and corresponding response strategies are proposed. in order to optimize the use of limited government budgets and resources, the factors and corresponding response strategies were assessed and ranked in terms of their comparative impacts. the analytical network process (anp) was 1 *corresponding author mailto:hhf@water.tku.edu.tw mailto:hcc31699@ms41.hinet.net ijahp article: huang/ management strategies for taiwan shimen reservoir catchment area: perspectives of collaborative planning international journal of the analytic hierarchy process 122 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.244 employed in this research for quantifying multiple stakeholder's’ perceptions of the above mentioned factors and response strategies. the anp results show that “national land planning” is the most important issue, under which “the extent of land use” is the strongest factor. “public participation” is shown to be the strategy of the highest priority, under which land development’s environmental impacts need to be reviewed, and an information management platform needs to be established so that value added analysis can be performed in support of catchment management strategic planning. keywords: reservoir catchment area, collaborative planning, analytical network process 1. introduction in taiwan there are 113 reservoir catchment areas delineated as public water source protection areas covering 25% of the country’s land area or a total of 8,972 square kilometers. the water source preservation sites cover a massive area and as a result multiple jurisdictions and the involvement of multiple management units, and therefore factors that influence water quality and quantity control are already under increasing ecological, societal, hydrological and economic pressures (huang, chen & liu, 2014). in this research, a case study of shimen reservoir catchment area management is constructed. the catchment area of shimen reservoir is about 763.4 square kilometers. the shimen reservoir is located on the mid-stream reach of the dahan river which is taiwan's third largest reservoir and artificial lake. completed in 1964, the dam and reservoir now supply water to more than three million people in northern taiwan. the reservoir has been operated for over 50 years to serve the 5 originally planned functions; i.e., irrigation, hydropower generation, public water supply, flood mitigation and tourism. over the years, the reservoir has significantly contributed to agricultural production, industrial development, enhancement of living standards, increase of employment opportunities, and alleviation of flood and drought disasters in the northern taiwan region. shimen in chinese means stone gate, and the reservoir site has been so named because at the mouth of the river a pair of huge rocks on either side of the bank look like a stone gate. this study utilized collaborative planning, through in-depth interviews with stakeholders, to construct key issues that the catchment management in the shimen reservoir now faces. through team brainstorming and expert consultation, thirteen factors under the categories of “land use”, “water quality protection”, “quantity of water supply and demand”, “water and soil conservation”, and “laws and institutions” were identified and corresponding response strategies are proposed. in order to optimize the use of limited government budgets and resources, the factors and corresponding response strategies were assessed and ranked in terms of their comparative impacts. lastly, the analytical network process (anp) (saaty & saaty, 2003) was employed in this research for quantifying multiple stakeholders’ perceptions of the above mentioned factors and response strategies. additionally, investigation of recommended research and analysis development was discussed in order to provide reference for future implementation and research tactics for the current government. in order to know more about the management problems of the shimen reservoir catchment area, this study conducted site visits by using qualitative approach stakeholder interviews. we conducted 18 interviews with http://ndltd.ncl.edu.tw/cgi-bin/gs32/gsweb.cgi/ccd=p4hwhe/search?q=kwe=%22collaborative%20planning%22.&searchmode=basic http://en.wikipedia.org/wiki/reservoir ijahp article: huang/ management strategies for taiwan shimen reservoir catchment area: perspectives of collaborative planning international journal of the analytic hierarchy process 123 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.244 stakeholders representing different authorities and policy communities related to our topic. a summary of the interview participants is shown in table 1. table 1 interview participants general classification interview units relevant authorities （7 events） northern region water resources office, wra new taipei city government national land planning team, ministry of the interior forestry bureau watershed management division ,coa land management department, council of indigenous peoples taipei branch soil and water conservation bureau, coa water quality protection division, epa local representative society （3 events） flood control oversight union green formosa front lo-fu elementary school local business （2 events） lung chu holiday village xiao wu lai vacation village township offices or representatives （3 events） da-xi township, tao-yuan county jianshih township, shin-chu county fu-hsing township, tao-yuan county relevant scholars and experts （3 events） water environment research center, national taipei university of technology te-chi reservoir watershed management committee disaster prevention center, national chiao tung university we constructed four key discussion points that are of concern to the management of the preservation sites at shimen reservoir by utilizing content of local interviews with stakeholders as well as taking account of the compiled results of related documents. the discussion points are “land management”, “rules and regulations”, “execution”, and “environmental justice” (water resources agency, 2009a). this includes discussion articles as follows: (1) land management issues a. illegal usage of national forests b. over-utilization of land c. compensation to areas near silt dams and preservation sites near the reservoir d. concurrence of preservation sites near reservoir and urban planning (2) issues regarding rules and regulations a. lack of overall planning of preservation sites and mechanism of integration b. lack of a platform to coordinate all levels of ministry ijahp article: huang/ management strategies for taiwan shimen reservoir catchment area: perspectives of collaborative planning international journal of the analytic hierarchy process 124 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.244 (3) execution issues a. lack of manpower in the executive branch b. unable to implement laws to suppress illegal jobs due to public opinions (4) issues of environmental justice a. source of income for local residents b. publication of information and participation of people these issues that water preservation sites near the shimen reservoir now face, and the analytically compiled results of interview records and related documents regarding taiwanese water preservation sites (water resources agency, 2009b) were reviewed in hopes of constructing five main aspects that the management of taiwanese catchments and similar environments now face. key points for discussion under these five issues include “land use”, “water quality protection”, “quantity of water supply and demand”, “water and soil conservation” as well as “laws and institutions”. 2. hypotheses/objectives the so-called ’collaborative planning’ means people who live close together construct new ways of solving problems, thinking and acting through being embedded in multiple joint relationships. (healey, 2006) it looks at how to form a cooperation culture, integrating the local residents, the relevant authorities and stakeholders in order to create policy learning through dialogue, and seeking consensus on issues of common concern. according to this definition we know that collaborative planning attaches importance to the social relationship system and social learning communication. the former emphasizes the ability to build relationships, expecting to cross the system fragmentation caused by cultural barriers, organizational division of labor and power, while the latter is conducive to social mobilization, and reorganizing people who are socially marginalized. this inclusive plan has a multicultural nature. it helps to develop cooperation, communication and to build relationships of mutual understanding and trust. in addition, collaborative planning activities are conscious policy-driven and try to reach three goals by embedding the views of strategic, long-term, and mutual relationships into the governance process. these three goals are: (1) to help build relationship capacity by informing the political community and stakeholder’s views on related issues, (2) to form a field for the stakeholders to exchange views, and (3) by forming the means of solving problems to help form the cooperation of new ways of thinking and actions or reconstructing the procedures of dealing with the problems. in order to reach these goals, we need to design a system with systematic processes (healey, 2006). the so-called ‘system’ refers to different types of entities, including the norms, actors expectations, and organizational rules. it links the interactive organization and the behavior of the members acting in accordance with the system (ostrom, 2007). in essence, the system has some common characteristics: (1) the system is a product designed by humans which means the system needs to change with the environment's affection or human needs, and (2) the system can not only regulate individual's behavior patterns but also construct serious ways of environmental, economic and social interaction to reduce the uncertainty of human society because of its feedback mechanisms. ijahp article: huang/ management strategies for taiwan shimen reservoir catchment area: perspectives of collaborative planning international journal of the analytic hierarchy process 125 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.244 as e. ostrom (1999) said, we will be able to understand the meaning of the system more when we actually use the rules. a cooperation plan is a conscious policy-oriented plan which tries to embed the views of strategic, long-term, and mutual relationships into the governance process. its goal is to look for a win-win result and avoid the participant’s different perspectives which could cause a win-lose result. therefore, it forms an inclusive discussion through the system design in order to solve the dilemma of the coexistence of advantaged and disadvantaged groups. the links between the rules and the system features can establish different types of system arrangements between the actors and the rules. the emphasis of collaborative planning is based on formal resources. it receives stakeholder’s attention and makes them be active participants in discussing and cooperating on risk issues, and thus shapes the essence of actions. more specifically, the factors we need to consider during the whole policy process include the actors, results and the policy itself. (ostrom, 2007) when everyone holds different views with regards to a problem, frame conflict exists because of different explanations of the issues. therefore, both sides will have difficulty appealing to reason and developing solutions when there is a lack of standard protocol specification. that is the reason why policy issues happen (schön and rein, 1994). schön and rein (1994) call the policy stance built by belief, perception and appreciation ‘frame’. in fact, ‘frame’ is deeply rooted in the system and related to the interests that people desire and fight for closely. interests shape the ‘frame’ and make the stakeholders enter the subject areas and move to defend their interest and cause the frame conflict. stakeholder’s different levels of awareness of policy issues affect the strategy of problem solving. the catchment area this study talks about is very large, crossing different administrative areas and involving many management units which causes a structural problem that makes it hard for stakeholders to come to a consensus about ways to solve the issues. in order to practice the cooperation and collaboration that a cooperation plan emphasizes, the government needs to take the stakeholders’ opinion into consideration when they are in the planning process. generally speaking, the collaborative planning process includes six stages as illustrated in figure 1. the stages are: (1) defining the problem, (2) establishing an assessment standard, (3) developing alternative programs, (4) comparison of alternative programs, (5) selecting an alternative program, and (6) conducting evaluation. ijahp article: huang/ management strategies for taiwan shimen reservoir catchment area: perspectives of collaborative planning international journal of the analytic hierarchy process 126 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.244 figure 1. diagram of collaborative planning (source: patton & sawicki, 1986, p. 26) this study uses the theory of collaborative planning as well as the analytical network process (anp). through interviews with stakeholders affected by the policy, we recompiled influencing factors regarding the management of the research on water preservation sites as well as alternative program strategies. by gaining a better grasp of key aspects and influencing factors, we can provide consultation for future government administrations and develop important strategies. 3. research design/methodology in order to put cooperation into practice, the opinions of those whose interests are affected by the policy must be included during the planning phase of the process. 3.1 stakeholder classification the earliest method of stakeholder classification started with r. e. freeman (1984) by using the strategic management point to define ‘stakeholder’ as “any group or individual who is affected by the organization goals or who can affect to reach the goals”. in regard to public policy, this means anyone who is affected by some problem directly or indirectly or who is accepting positive or negative affection directly or indirectly after the government institute takes any step (wu d., 2008). chang-tay chiou (2013) further divided stakeholders into three categories: (1) policy makers: the individual or group who makes, uses and executes the policy; (2) policy beneficiary: the individual or group who gets the benefit directly or indirectly; and (3) policy victims: the individual or group who loses because of misconduct of policy design and side effects. following r. mitchell’s way, this study divides stakeholders into seven types according to power, legitimacy and urgency as shown in figure 2. power refers to the ability the stakeholders have to affect the agenda or decisions. legitimacy means the appropriate relationship between the stakeholders and the issues. in other words, is it reasonable or appropriate for the stakeholders to have an interest? urgency means the stakeholders need to raise the issues immediately and need of the government to respond right away. it is worth noting that the position of the stakeholders is not absolutely constant, and depends on their power, legitimacy and urgency. this study attempted to classify the stakeholders using these features according to the specific case, and used this as a basis for choosing interviewers. ijahp article: huang/ management strategies for taiwan shimen reservoir catchment area: perspectives of collaborative planning international journal of the analytic hierarchy process 127 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.244 figure 2. stakeholder typology: one, two, or three attributes present source: mitchell, agle & wood (1997, p. 874) 3.1.1 definitive stakeholders: this type needs to possess all three characteristics. once their request is made, it should be the first priority for the government to consider and develop clear laws to fit their need. 3.1.2 expectant stakeholders: this type needs to possess at least two characteristics. they usually take a proactive approach, so the pressure of the government response will relatively increase. we can break it down into three categories according to their own characteristics. 3.1.3 latent stakeholders: this type only has a single characteristic, therefore the government does not fully respond to their request. we can break it down into three categories according to their own characteristics. 3.1.4 non-stakeholders: this type possesses no characteristics. this study conducted an anp questionnaire survey which included 9 experts who are very knowledgeable about management of catchment areas. they represent different institutes and professional backgrounds which are shown in table 2. table 2 also includes the classification principle of policy stakeholders discussed by mitchell (1997). ijahp article: huang/ management strategies for taiwan shimen reservoir catchment area: perspectives of collaborative planning international journal of the analytic hierarchy process 128 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.244 table2 anp questionnaire survey interviewers list interview unit covers property stakes interviewer’s title taipei water management office, water resources agency, ministry of economic affairs 2，4，7 (legitimacy) director conservation division, water resources agency, ministry of economic affairs 2，4，7 (legitimacy) leader northern region water resources office, water resource agency, ministry of economic affairs. 3，6，7 (urgency) deputy director deji reservoir watershed management committee, ministry of economic affairs 3，6，7 (urgency) executive secretary water resources agency, ministry of economic affairs 1，4，5 (power) consultant institute of environmental engineering, national taiwan university 1，5 (power) assistant professor department of civil engineering, taipei university of technology 1，5 (power) professor tamkang university 1，4 (power) professor taiwan typhoon and flood research center 1，4 (power) deputy director 3.2 analytic network process (anp) application in order to understand the priorities of those whose interests are affected, the opinions of each related group were analyzed and categorized by the influencing factor of their judgments through the analytic network process (anp). our research of the procedure of anp’s application is shown in figure 3. anp’s basic structure can be classified into four main points: evaluation and comparison of the systematization of complex comparisons and assessments and setting the scale, establishment of pairwise comparison of matrices, prioritization of vectors and maximizing the eigenvalue, and testing for consistency (saaty and saaty, 2003). figure 3. procedure diagram of research application ijahp article: huang/ management strategies for taiwan shimen reservoir catchment area: perspectives of collaborative planning international journal of the analytic hierarchy process 129 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.244 3.2.1 anp hypothesis the anp method is an extension of the ahp method, and therefore there is a great deal of similarity between the two. the ahp method holds the following principles (seldtani & takahashi, 2001）: (1) a system can be decomposed into many types, and forms a straight line shown in the hierarchy; (2) it is assumed that the factor of each level is independent; (3) the factor of each level can be evaluated based on the factor of the last level; (4) while conducting comparison evaluation value scale can be converted into ratio scale; (5) after conducting pairwise comparison, pairwise comparison of matrices to process; (6) preference relations satisfy transitivity and not only the advantages and disadvantages but also the strength satisfies transitivity at the same time; (7) it is not easy to have full transitivity, therefore it is permissible to have non-full transitivity existence; (8) the advantage of the factor can be obtained by weighted law; (9) as long as any factor shows up in the hierarchy, it doesn’t matter what level the advantage is, it is still considered to be related to the whole evaluation structure and not the independence of the hierarchy checklist. all except the first two basic assumptions of the ahp method above are applicable to the anp method. 3.2.2 anp’s steps the analytic network process (anp) can be divided into eight steps: step 1define the problem: according to the nature of decision problem, can list the factors which might affect the problem, and collect related information in order to summarize the decision problem. step 2 decide the decision-making group: based on the domain of the problem, bring in scholars and experts from the related fields to form a decision-making group. step 3create the problem’s network hierarchy assessment model architecture diagram: after collating and summarizing the information of decision problems, determine the factors that affect the decision problems including the goal, evaluation standard and backup plan. in the problem structure every level has a dependent and feedback relationship. this study consulted the in-depth interviews and related documents in order to determine five aspects of the aforementioned main issues. through team brainstorming and expert consultation, five main criteria were created to evaluate each aspect and to list possible deciding factors of each aspect (thirteen in total). furthermore, eight alternative methods were proposed in accordance to management issues, thus creating the layered network structure of anp shown in table 3. a map of the relationship network of the anp was constructed and is shown in figure 4. the analysis was conducted using superdecisions tm software (creative decisions foundation, pittsburgh, pa, usa). ijahp article: huang/ management strategies for taiwan shimen reservoir catchment area: perspectives of collaborative planning international journal of the analytic hierarchy process 130 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.244 table3 evaluation aspects, influencing factors and alternatives criteria factors a. quantity of water supply and demand a-1 changes in surface runoff a-2 decrease in reservoir capacity a-3 downstream water requirements b. water quality protection b-1 waste water emission b-2 source water turbidity b-3 reservoir eutrophication c. water and soil conservation c-1 collapse of sloping fields c-2 ground vegetation d. national land planning d-1 environmental and geological properties d-2 intensity of land usage e. coordination and communication e-1 obtainability of basic environmental data e-2 information sharing and applicable benefits e-3 public knowledge and participation alternatives 1. post-exploitation environment impact analysis 2. ground surface monitoring system 3. water environment monitoring 4. reduction of pollution and improvement measures 5. water supply potential and demand assessment 6. landside control 7. information platform settlement and value-added data analysis 8. public participation figure 4. anp relationship network map (using superdecisions tm software) ijahp article: huang/ management strategies for taiwan shimen reservoir catchment area: perspectives of collaborative planning international journal of the analytic hierarchy process 131 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.244 step 4 pairwise comparison evaluation: after a network hierarchy structure is built, evaluate the same level’s relevant importance of different evaluation factors. the appraisal method is based on the last level’s factor to pairwise compare the importance from this level’s factors to the last level’s factors. this can decrease the burden on the decision-makers thinking, and can also show the relativity of decision factors more clearly. the analytic hierarchy process uses nominal scales as the evaluation standard which is divided into a nine-point scale as shown in table 4. table 4 anp’s evaluation comparison scales rating scale definition explanation 1 equal importance the comparison of two programs are of equal importance 3 moderate importance experience and judgment shows slightly in favor of one program 5 essential importance experience and judgment shows strongly in favor of one program 7 strong importance the facts show strongly in favor of one program 9 extreme importance full evidence shows absolutely in favor of one program 2,4,6,8 intermediate values between the explanations above source: revised from saaty, 2001, p. 26. step 5 create a pairwise comparison matrix: after creating the assessment model architecture diagram, start to calculate the relevant importance of different level’s evaluation factors as shown in figure 5. figure 5. establishment of pairwise comparison of matrices (using superdecisions tm software) step 6obtain a pairwise comparison matrix feature vector and the maximize eigenvalue: after obtaining the pairwise comparison matrix , use the eigenvalue method normally used in data analysis to obtain the feature vector and the maximum eigenvalue. in the procedure, obtain the last level’s feature vector and then turn down, and finally obtain the maximum eigenvalue overall which means the priority of the lowest level’s backup plan to the highest goal. step 7 consistence test: the pairwise comparison matrix needs to pass the consistence test. the consistence test is based on the consistence ratio of the pairwise comparison matrix to proceed. cr=ci/ri which ci means consistence index and ri means random index. consistence index ci=λmax/n-1 and ri is based on the level number n of the ijahp article: huang/ management strategies for taiwan shimen reservoir catchment area: perspectives of collaborative planning international journal of the analytic hierarchy process 132 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.244 pairwise comparison matrix. when cr≤0.1 it means the evaluation number has consistence and is acceptable. step 8 choose the fittest plan: after performing the steps above, obtain different evaluation standards and the priority number of the plans. the larger the numbers, the higher the priority for the acceptance of the plan. in summary, the fittest plan chosen through the analytic network process not only considers the dependent problems existing between the plans and standards, but also fits the determined goals. 4. data/model analysis in the anp method, each aspect of evaluation, influencing factors and alternative programs are independent but have the property of mutual influence (saaty, 2001). this study took the mutually influencing relationship between every criteria (a-e), 13 factors (a1, a2…e2, e3), and program (alternative 1-8) to construct a map of the relationship network of the anp as shown in table 3 & figure 6 . when considering the example of management of preservation sites, based on investigative results (table 5), respondents believe that “national land planning” is most important in comparison with other aspects with an estimated importance ratio of 28.33%. the second most important aspect is “soil and water preservation”, at a ratio of 23.56%, while having “supply and demand of water” and “maintenance of water quality” at 22.39% and 17.76% respectively. the anp survey results all show that of 13 reservoir catchment preservation management influencing factors, the top five relatively important factors are: “intensity of land usage”, “collapse of sloping fields”, “decline of reservoir capacity”, “environment and geological properties” and “downstream water requirements”. furthermore, respondents believe that the impact of “land usage intensity” and “collapse of sloping fields” are most significant, at ratios of 19.03% and 16.72% respectively (figure 6). statistics and analyzed results of the 8 alternative programs show that respondents believe “participation of people” to be the primary concern in future management strategies, at 27.40%. regarding land usage and development, the second concern was to conduct a complete “review of the impact of development of soil and water resources on the environment”, at a ratio of 18.20%. the third was “the construction of information management platforms and benefit analysis”, at a ratio of 13.92%; the fourth was “water environment monitoring”, at 13.76%; and the fifth was “surface environment monitoring system”, at 10.29% (figure7). ijahp article: huang/ management strategies for taiwan shimen reservoir catchment area: perspectives of collaborative planning international journal of the analytic hierarchy process 133 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.244 table 5 anp final investigative results (by 9 policy stakeholders) criteria normalized ratio factors normalized ratio alternatives normalized ratio a. 0.223850667 a-1 0.039009292 1. post-exploitation environment impact analysis 0.1819701 a-2 0.095333226 a-3 0.089508141 2. ground surface monitoring system 0.102935 b. 0.177465 b-1 0.066628759 b-2 0.082405215 3. water environment monitoring 0.1376381 b-3 0.028431028 4. reduction of pollution and improvement measures 0.0926468 c. 0.235567444 c-1 0.167162434 c-2 0.06840501 5. water supply potential and demand assessment 0.0260503 d. 0.283278667 d-1 0.092976636 d-2 0.190302031 6. landside control 0.0455706 e. 0.079838111 e-1 0.018315917 7. information platform settlement and value-added data analysis 0.1391923 e-2 0.024730024 e-3 0.036792118 8. public participation 0.2739964 figure 6. anp investigative results of the 13 factors figure 7. anp investigative results of the 8 alternative programs 3.68% 2.47% 1.83% 19.03% 9.30% 6.84% 16.72% 2.84% 8.24% 6.66% 8.95% 9.53% 3.90% 0% 5% 10% 15% 20% e-3 public knowledge and participation e-1 obtainability of basic environmental… d-1 environmental and geological… c-1 collapse of sloping fields b-2 source water turbidity a-3 downstream water requirements a-1 changes in surface runoff 27.40% 13.92% 4.56% 2.61% 9.26% 13.76% 10.29% 18.20% 0% 5% 10% 15% 20% 25% 30% 8. public participation 7. information platform settlement and… 6. landside control 5. water supply potential and demand… 4. reduction of pollution and… 3. water environment monitoring 2. ground surface monitoring system 1. post-exploitation environment impact… ijahp article: huang/ management strategies for taiwan shimen reservoir catchment area: perspectives of collaborative planning international journal of the analytic hierarchy process 134 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.244 5. limitations this research used the shimen reservoir’s water preservation area as the research site. however, due to limited interview subjects, compilation of documents and related arrangement of conditions, it might not be suitable for other taiwanese catchments, but can be of general reference and utility, collaborative planning, and multiple criteria strategizing for reservoir catchments of similar environments to implement in the future. 6. conclusions this paper utilized the perspective of collaborative planning of stakeholders for investigation. the anp survey results showed that “national land planning” is the primary problem that needs to be resolved for the shimen reservoir, and its main influencing factor originates from “the extent of land use”. secondly, “water and soil conservation” is an issue that needs to be of increasing concern, and management strategies should be developed for its main influence factor, “collapse of sloping fields”. in regards to alternative programs for the management of catchment preservations, respondents believe “public participation” should be of main concern for the management strategies of future water catchment preservation sites. they also believe that promoting application of management strategies such as “post-exploitation environment impact analysis” and “information platform settlement and value-added data analysis” would strengthen the result of future planning of water source preservation areas, environmental impact analysis and further follow-up of investigations and management. based on the above, we know “land planning” is the primary processing task of the shimen catchment area. in fact, the most efficient fundamental control method is land planning for unpredictable weather disasters. land planning is not only reasonable development, distribution and use of the land, but is also a system including values, law regime, practices and executive management. although the taiwanese interior department proposed the idea, the execution schedule of the land planning law still got postponed. based on this action, the management of shimen catchment area will not be acted upon completely, but will slowly change due to the environmental need for survival and weather change. thus, this study promotes three suggestions according to the primary strategy of future management which is ‘public participation’. 6.1 realize the importance of local people participating in catchment area protection in recent years, the conservation experience of catchment areas inside the country makes us slowly realize that ‘people’ are the most important lesson. the prerequisite to conserving the catchment area based on people is to fully communicate with local people and attempt to combine the awareness of the risk of disaster prevention with local people’s power. with the evolution of time, group trust starts to be a potential expectation asset and helps the group to solve the common problems with cooperation and coordination. (kramer, brewer and hanna, 1996) 6.2 catchment protection area should establish autonomous public participation mechanisms any system’s change involves social, ecological and economic factors and becomes a complex and dependent process. therefore, it is hard to use the linear model to explain its causality, not to mention to predict the results. facing this unchangeable fact, the first ijahp article: huang/ management strategies for taiwan shimen reservoir catchment area: perspectives of collaborative planning international journal of the analytic hierarchy process 135 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.244 priority is to enhance the adapting ability of the system with the help of policy tools. we can only overcome the unpredictable effect from trial and error by increasing adapting ability. in other words, ‘collect and archive the environmental basic information’ should be the basis of managing catchment conservation areas. they cannot be efficiently managed without a full information data base. this suggests that we should keep promoting the development of environmental survey and monitoring technology, enhancing the information collecting ability, increasing real-time monitoring effectiveness and establishing the early warning system to continue to reinforce incomplete basic monitoring information. however, technology has always been peopleoriented so incorporating autonomous public participation mechanisms, such as water conservation volunteers, will become the main development of in-situ information built and real-time early warning reporting function. 6.3 the catchment area’s governance should continue to communicate with local people commitment is the most basic requirement for joint action. learning how to set up the interacting platform, combine participants different views, establish the basis of consensus, coordinate the policy tools, and execute is the way to solve differences and produce complementary roles. this could include (1) willingness to compromise and have a cooperative attitude for the policy results, (2) collecting related knowledge to fit an uncertain environment, and overcoming the dilemma together, (3) combining all the necessary information in such a way to reach the policy goals (giacchino and kakabasdse, 2003). once participants have adequate policy commitment, the ‘supportive context 2 ’ will help the decisions to combine economy, society and environment and have adequate ability to adapt in order to efficiently respond to unpredictable impacts (meadowcroft, 2005). thus, we suggest that the catchment area should maintain the operation of a cross-border coordination platform and include the stakeholders, especially the local people, in the policy consultation process so that the best solution to solve the hard problems of the catchment area can be developed. in summary, since in the same situation everyone’s frame view varies, it is obvious that their way of handling problem would vary as well. therefore, based on the feasibility of the policy, this study suggests that the government should expand the level of participation and combine local people’s opinions into the policy-making process even when the value and goals are in conflict. if the policy process is inclusive, it will be easier to obtain policy legitimacy and to combine the local information more efficiently to manage the local catchment area together. 2 the so-called supportive context means all kinds of information from the environment, including the assistance of people, tools, system or devices offered by the work place or the group. because it is considered to be proper in our culture, it can help to accomplish the individual work result. ijahp article: huang/ management strategies for taiwan shimen reservoir catchment area: perspectives of collaborative planning international journal of the analytic hierarchy process 136 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.244 references chiou, c. t. (2013). public policy. 4th ed. taipei: guliu. freeman, r. e. (1984). strategic management: a stakeholder approach. new york: cambridge university press. giacchino, s., and s. kakabadse. (2003). successful policy implementation: the route to building self-confident government. international review of administrative science 69(2), 139-160. healey, p. (2006). collaborative planning, second edition: shaping places in fragmented societies. london: palgrave macmillan. huang, h. f., chen, h. c., and s. liu. (2014). management strategies for taiwan reservoir catchment areas: a case study in shih-men reservoir catchment area, proceedings of the international symposium on the analytic hierarchy process:13th isahp conference, washington dc, 254-255. koppenjan, j. f. m., and e. h. klijn. (2004). managing uncertainties in networks. new york: routledge. kramer, r. m., m. b. brewer, and b. a. hanna. (1996). collective trust and collective action: the decision to trust as a social decision. in r. m. kramer and t. r. tyler (eds.), trust in organizations: frontiers of theory and research, 357-389. thousand oaks, ca: sage publications. lin, s. p. (2011). an analysis of the public management. taipei: wunan. meadowcroft, j. (1997). planning, democracy and sustainable development. international political science review 18(2), 167-189. meadowcroft, j., k. n. farrell, and j. spangenberg. (2005). developing a framework for sustainability governance in the european union. international journal of sustainable development, 8(1/2), 3-11. mitchell, r. k., agle, b. r. & wood, d. j. (1997). toward a theory of stakeholder identification and salience: the principle of who and what really counts. academy of management review, 22(4), 853–886. ostrom, e. (2007). institutional rational choice: an assessment of the institutional analysis and the development framework. in p. a. sabatier (ed). theories of the policy process, 21-64. boulder, co: westview. ostrom, e. (1999). coping with tragedies of the commons. annual review of political science 2, 493-535. patton, c. & d. sawwicki, (1986). basic models of policy analysis. englewood cliffs, nj: prentice-hall. http://www.google.com.tw/search?hl=zh-tw&tbo=p&tbm=bks&q=inauthor:%22patsy+healey%22 ijahp article: huang/ management strategies for taiwan shimen reservoir catchment area: perspectives of collaborative planning international journal of the analytic hierarchy process 137 vol. 7 issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.244 saaty, t. l. & saaty, r. w. (2003). decision making in complex environments (ahp & anp), super decisions. seldtani, k., & i. takahashi (2001). a unified model and analysis for ahp and anp. journal of operations research society of japan, 44(1), 67-89. stone, d. a. (2002). policy paradox: the art of political decision making. new york: w. w. norton & co. thomas l. saaty, (2001). decision making with dependence and feedback: the analytic network process. pittsburgh: rws publications. water resources agency, ministry of economic affairs, r.o.c (2009a). strategic planning of water quality management within shihmen reservoir catchment. water resources agency, ministry of economic affairs, r.o.c (2009b). strategic planning of management techniques for water quality and quantity protection areas (1/2). wu, d. (2008). a dictionary of public policy. taipei: wunan. http://books.google.com.tw/books?id=9gkjbipyrz4c&printsec=frontcover&dq=a+dictionary+of+public+policy&hl=zh-tw&sa=x&ei=suatvlhyo8un8awqwihqcg&ved=0cb0q6aewaa ijahp news and events: mu/nina begičević, ijahp editorial board member, elected as university of zagreb’s new dean of the faculty of organization and informatics international journal of the analytic hierarchy process 437 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.728 nina begičević, ijahp editorial board member, elected as university of zagreb’s new dean of the faculty of organization and informatics with a total of 58 votes (83%), associate professor nina begičević ređep, phd has been elected the new dean of the faculty of organization and informatics (foi) and has a mandate to serve in the position for two years. her position begins in october 2019, with the beginning of the new 2019/2020 academic year. nina begičević ređep was born in 1980, in varaždin. she graduated from the faculty of organization and informatics (foi), university of zagreb, in 2003 and two years later she completed her postgraduate master's degree in business management systems. she was also an u.s. government scholar at the university of pittsburgh, katz graduate school of business, through the junior faculty development program (jfdp) from december 2007 to june 2008. since 2009, she has been teaching at the faculty of organization and informatics, first as an assistant professor, and then as an associate professor in business decision making, strategic planning, education technology and the application of digital technologies in education and e-learning. she has been a member of the faculty of organization and informatics board for the last seven years, and is currently the vice dean for scientific research and international relations. in october 2019, she will take over the duty of dean as the youngest candidate and the first female dean in the history of foi. ijahp news and events: mu/nina begičević, ijahp editorial board member, elected as university of zagreb’s new dean of the faculty of organization and informatics international journal of the analytic hierarchy process 438 vol. 11 issue 3 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i3.728 she is a co-author and author of a number of books and monographs, as well as more than 60 scientific articles. she cooperated with some of the most respected scientists and professors in the u.s., such as thomas l. saaty, phd, the inventor, architect and primary theoretician of the analytic hierarchy process (ahp), as well as enrique mu, phd, an expert in the management of information systems and current ijahp editor-in-chief, at the university of pittsburgh. nina begičević ređep is also a member of 6 international conference program boards, the editorial board of the international scientific journal ijahp and is a reviewer for a dozen other international scientific journals. she has received 9 awards for her work in science and education in croatia and abroad, including an award for scientific achievement and creativity in ahp (analytic hierarchy process), that she received in chile in 2007. we wish nina great success in her new position and we are very proud that she is part of our ahp/anp community. source: https://www.foi.unizg.hr/en/news/new-foi-dean-elected-assoc-prof-ninabegicevic-redep-phd https://www.foi.unizg.hr/en/news/new-foi-dean-elected-assoc-prof-nina-begicevic-redep-phd https://www.foi.unizg.hr/en/news/new-foi-dean-elected-assoc-prof-nina-begicevic-redep-phd a style guide for papers presented to the international journal of the analytic hierarchy process ijahp book review: salomon, zinoviev, zinovieva/ alessio ishizaka and philippe nemery’s multi-criteria decision analysis international journal of the analytic hierarchy process 284 vol. 5 issue 2 2013 issn 1936-6744 book review: alessio ishizaka and philippe nemery’s multi-criteria decision analysis valerio a. p. salomon sao paulo state university (unesp) guaratingueta, sp, brazil e-mail: salomon@feg.unesp.br aleksandr zinoviev institute of strength physics and materials science of the siberian branch of the russian academy of sciences (ispms sb ras) tomsk, russia e-mail: kelast90@gmail.com olga zinovieva national research tomsk state university (tsu) tomsk, russia e-mail: emelyanova@ispms.tsc.ru in august 2013 the british office of john wiley & sons published alessio ishizaka and philippe nemery’s “multi-criteria decision analysis: methods and software”. this book has many similarities to “multi-criteria decision analysis: state of the art surveys” edited by jose figueira, salvatore greco and matthias ehrgott and published in 2005 by springer. the main similarity is that it discusses concepts and procedures for several multi-criteria decision analysis (mcda) methods, such as the analytic hierarchy process (ahp), elimination et choix traduisant la realité (electre), and the multiattribute utility theory (maut), to name a few. the greatest difference between these two works lies in the method of authorship. alessio ishizaka and philippe nemery wrote their whole book (with the exception of the dea chapter which was written by jeanmarc huguenin). jose figueira, salvatore greco and mathias ehrgott simply edited contributions from several coauthors. for example, jose figueira, vincent mossou and bernard roy wrote the chapter on electre methods, while the chapter on ahp was written by thomas saaty. this main difference results in two consequences which are a lack of uniformity, and the fact that some content in the book is simply based on the individual authors’ opinion. the entirety of ishizaka and nemery’s book hangs together with a uniform thread. from chapter 2, on ahp, to chapter 10, on data envelopment analysis (dea), the chapters have the same main sections: “essential concepts”, “software”, and “in the black box”. therefore, after reading a chapter on one particular method the reader can anticipate what will be found in future chapters about other methods. this setup can be very useful for systematic readers. on the other hand perhaps this style of writing can be boring, or monotonous, for other kinds of readers.  corresponding author thanks the international association for the exchange of students for technical experience (iaeste). ijahp book review: salomon, zinoviev, zinovieva/ alessio ishizaka and philippe nemery’s multi-criteria decision analysis international journal of the analytic hierarchy process 285 vol. 5 issue 2 2013 issn 1936-6744 this uniformity extends to the book’s division into the following four parts: part i, full aggregation approach; part ii, outranking approach; part iii, goal, aspiration or reference-level approach; and, part iv, integrated systems. part i contains chapters 2 (ahp), 3 (analytic network process), 4 (maut), and 5 (measuring attractiveness by a categorical based evaluation technique). chapter 1 is the introduction and does not belong to any part. this review is limited to chapters 1 to 3, since they are focused on the ahp. chapters 4, 5, 7 (electre) and 11 (multi-method platform) also cite the ahp, but all refer to chapter 2. ishizaka and nemery’s book is a milestone in ahp’s literature. on july 2008, jyrki wallenius, then the president of international society of mcda, published (in inform’s journal, management science) a bibliometric analysis on mcda using the institute for scientific information’s database. that analysis found that the ahp was the mcda method with the highest number of publications. the more than 400 papers published from 2000 to 2004 on ahp have many authors. nevertheless, until now, the books on ahp referred to most often have the same main author, thomas l. saaty. future bibliographical sections on ahp’s works may now have a new book reference. papers usually present more updated information than books, since their publication process is supposed to be faster, and they are reviewed and edited in more detail. however, papers often have page or word limits, and are not able to completely repeat theoretical points since they only refer to original papers or books. nevertheless, a book reference can repeat, or even rewrite the theory, without space limitations. this is another reason why ishizaka and nemery’s book is an important addition to the ahp literature. one shortcoming of the ishizaka and nemery’s book as an ahp reference is the authors’ opinions on mcda generalizations because it is not clear whose opinions they are. for instance, one of the first lines from section 2.2 introduces ahp as a “particularly useful method when the decision maker is unable to construct a utility function, otherwise maut is recommended”. this statement is possibly the personal opinion of alessio ishizaka or philippe nemery, or perhaps both of them. however, it is written as a matter of fact. the statement induces mcda users to use ahp only if they cannot “construct a utility function”. this is a serious statement to consider because “the aim of the book is to make mcda methods even more intelligible to novice users such as students” (page 2). if the statement sounds incorrect to ahp’s practitioners, it is interesting to point out that even jim dyer, an ahp’s criticizer, has already proposed the conjoint use of ahp and maut (management science, 1990). the incorrect statement that ahp should be used only after maut proves inadequate is based on the premise that there is an adequate mcda method for a type of decision problem. conversely, some mcda methods should not be applied to certain types of decision problems. “none of the methods are perfect nor can they be applied to all problems” (page 6). the book cites bernard roy’s paper (journal of operations operational research society, 1981), where four main types of decision problems are identified. they are choice problems (selection of one among the alternative solutions for the problem), sorting problems (alternatives are grouped in categories), ranking problems (alternative are ordered from “best to worst”), and description problems (description of alternatives and consequences). there are also problems from mcda ijahp book review: salomon, zinoviev, zinovieva/ alessio ishizaka and philippe nemery’s multi-criteria decision analysis international journal of the analytic hierarchy process 286 vol. 5 issue 2 2013 issn 1936-6744 theory, such as the design problem or the elicitation problem, but ishizaka and nemery’s book focused only on those four. in ishizaka and nemery’s table 1.2, ahp is listed as an mcda method that is only useful for solving choice problems or ranking problems. “ahpsort” (section 2.5.4), an extension of ahp, was proposed as a method for solving the sorting problems by alessio ishizaka, craig pearman and philippe nemery in an article published in the international journal of production research, in 2012. ahpsort was proposed as “a new variation of ahp”; however, it is nothing more than an unusual way to apply ahp (clustering alternatives first). as a matter of fact, it was already done in ahp’s original paper (page 273). therefore ahpsort is simply a new name for an old thing. nevertheless, ahp can be applied for all mcda problems. description problems have been previously studied by ahp researchers, with the well-known benefits-opportunities-costs-risks (bocr) model researched by diederik wijnmalen, william wedley, mujgan ozdemir, luis vargas, and thomas saaty, himself, to name a few. ishizaka and nemery’s table 1.4, “required inputs for mcda ranking or choice method”, is also controversial. ahp was inserted in the middle of the “mcda method” column. this table’s rows were sorted by “effort input” ranging from “very high” to “very low”. one inference from that table is that maut is the most difficult mcda method used to solve a choice problem or a ranking problem, anp is the second most difficult, followed by macbeth, electre, promethee, gp, topsis, and dea in that order. this is simply a matter of opinion. many people find that dea applications are harder than ahp applications. chapter 2 presents ahp’s essential concepts (section 2.2) including problem structuring, priority calculation, consistency check, and sensitivity analysis. the concepts are presented along with an illustrative example named “case study 2.1”. makeitrational, an ahp’s software package developed in poland, is presented because of its “simplicity and the free trial version available”. the black box of ahp (section 2.4) includes an interesting subsection on judgment scales (section 2.4.2). finally, the extensions of ahp are presented in the final section of chapter 2(section 2.5) chapter 3 starts by discussing the analytic network process’s essential concepts (section 3.2) including inner dependency of alternatives and criteria, outer dependency, and influence matrix. three illustrative examples (case studies 3.1, 3.2 and 3.3; the last two cases are almost the same, with a little variation in the modeling) are presented. next, super decisions, an mcda software package developed for teaching purposes and downloadable free of charge, is presented, and followed by another example (case study 3.4). the black box of analytic network process (section 3.4) includes concepts for markov chains and the supermatrix. curiously, this chapter is the one with the fewest bibliographical references which include two books, one on markov chains (j. norris, cambridge university press) and another on the analytic network process (thomas saaty, rws). taking into account both chapters 2 and 3, around a quarter of the book was dedicated to the ahp. this prioritization emphasizes the importance of ahp for the mcda. the book has many merits, such as uniformity, essential conceptualization of different methods, and proper authorship. it does not present any novel concepts rather it will be http://www.multitran.ru/c/m.exe?t=42901_1_2&s1=%ef%ee%e4%f7%b8%f0%ea%e8%e2%e0%f2%fc ijahp book review: salomon, zinoviev, zinovieva/ alessio ishizaka and philippe nemery’s multi-criteria decision analysis international journal of the analytic hierarchy process 287 vol. 5 issue 2 2013 issn 1936-6744 used as a reference to learn or review the essentials of ahp or other mcda methods and software. if the reader is looking for innovation, then it would be better to search journals or proceedings about ahp, mcda, or correlated areas like operational research or management science. in summary, ishizaka and nemery’s “multi-criteria decision analysis: methods and software” is a good reference book on ahp. it should be read and referred to with careful attention, as with any other book. in our opinion, it will be a milestone in the ahp’s literature, and is a very interesting read presenting the authors’ point of view on essential and practical concepts from the ahp. rob typewritten text http://dx.doi.org/10.13033/ijahp.v5i2.201 ijahp article: dixon-ogbechi, jagun, ighomereho et al./ determination of promotional strategy for organizations in the nigerian insurance industry using the ahp model international journal of the analytic hierarchy process 68 vol. 7issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.261 determination of promotional strategy for organizations in the nigerian insurance industry using the ahp model bolajoko nkemdinim dixon-ogbechi faculty of business administration university of lagos, lagos, nigeria dixonogbechi@yahoo.com sikuade oladimeji jagun sol simon investments lagos, nigeria sjagun@yahoo.com salome ogheneochuko ighomereho department of economics and business studies redeemer’s university ogun state, nigeria ighomerehosalome@gmail.com ganiyu ajao rahim faculty of business administration university of lagos lagos, nigeria abdulrahimajao@yahoo.com elizabeth marie haran bertolon school of business, salem state university salem, ma, usa eharan@salemstate.edu abstract several studies have revealed that insurance companies are experiencing low patronage, and these studies have recommended the use of promotional strategies to create awareness and to boost customer patronage. research has revealed that there are seven basic promotional tools that most companies in the service industry can use in their promotional mix. thus, this study applied the analytic hierarchy process model (ahp) to assist managers in nigerian insurance companies in developing a promotional strategy by determining the best mix of the promotional elements to use given certain criteria. to achieve this goal, a survey approach was used. a multistage sampling technique was used to select a sample of sixteen insurance companies out of a possible 59 companies in the lagos metropolis, and questionnaires were administered to managers of these companies. the study revealed that the major promotional tools best suited for companies in the mailto:dixonogbechi@yahoo.com mailto:sjagun@yahoo.com mailto:ighomerehosalome@gmail.com mailto:abdulrahimajao@yahoo.com mailto:eharan@salemstate.edu ijahp article: dixon-ogbechi, jagun, ighomereho et al./ determination of promotional strategy for organizations in the nigerian insurance industry using the ahp model international journal of the analytic hierarchy process 69 vol. 7issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.261 nigerian insurance industry are advertising, direct marketing, personal selling and publicity. thus, in developing their promotional strategies, these companies should invest more in advertising, direct marketing, personal selling and publicity while investing minimally on public relations, sales promotion and sponsorship. keywords: nigerian insurance companies, promotional strategy, analytic hierarchy process model (ahp), lagos metropolis. 1. introduction in order to effectively carry out marketing activities, organizations need to ensure they have the appropriate mix of marketing elements. the basic elements of the marketing mix of an organization are popularly known by the acronym ‘4ps’ namely: product, promotion, price and place (mccarthy, 2002; dixon-ogbechi, odugbemi and aiyeku 2011). thus, promotion is an important component of the marketing mix of a business organization. it is a critical aspect of an organizations’ overall marketing mix and a major determinant of their success or failure (shimp, 2007). promotion is very essential for organizational survival because organizations have to communicate with their target markets and inform them about the product, its features, utility and availability (kotler, armstrong, wong and saunders, 2009; balaghar, majidazar and niromand, 2012). in order to develop its promotional strategy, organizations have access to numerous forms of promotion elements, the combination of which is collectively referred to as the promotion mix or marketing communication mix (kotler and armstrong, 2010). thus, the promotional mix is a set of elements/tools that a business can use to effectively communicate the benefits of its products or services to its customers (kotler, 2001). there are seven major promotional tools (advertising, personal selling, sales promotion, publicity, public relations, direct marketing, and sponsorship) (kotler, 2000; idris, et al., 2012); these tools are used to develop the promotional strategy of an organization. the selection of promotional elements/tools depends on certain factors which include the target audience/market, competitors’ strategy, types and nature of product; message content, and organization budget (kotler, 2001); the stage in the product life cycle, price and the funds available for promotional activities (lancaster and massingham (1999)); push and pull strategies, buyer readiness, type of distribution, promotion objectives, cost and availability of media (reena, 2013); market size/concentration and customer information needs (riley, 2012); actions of competitors (nickels, mchugh and mchugh, 2002); and characteristics of the organization itself (learn marketing, 2001). therefore, the challenge is to develop a suitable promotional strategy by selecting the right mix of the promotional tools appropriate for a particular organization at a particular time given these criteria, and then to use this strategy effectively to achieve organizational objectives (kotler, 2000). the effective application of this strategy can give an organization competitive advantage. thus, organizations, including insurance companies, that understand the compelling need for effective marketing communication and take advantage of all the tools available to communicate effectively with their target market will have a competitive advantage (alireza, et al. 2011). this is of importance in the insurance industry because research has revealed that financial service providers, a class to which insurance companies belong, are not perceived as highly trusted. therefore, they may have difficulty selling risk-based products using a single promotion tool (cox, ijahp article: dixon-ogbechi, jagun, ighomereho et al./ determination of promotional strategy for organizations in the nigerian insurance industry using the ahp model international journal of the analytic hierarchy process 70 vol. 7issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.261 2007). the effort to promote insurance products and services is therefore quite distinct, just as in some other typical service based organizations. thus, this study applied the analytic hierarchy process model (ahp) to assist managers in nigerian insurance companies to develop a promotional strategy by determining the best mix of the promotional elements to use given certain criteria. 1.1 statement of problem kotler (2000) posited that the critical success factors for firms lie not only in producing good products, competitively priced and distributed through appropriate channels to generate sales and profits, but the benefits of the products must also be communicated to customers through effective promotional tools. promotion is the marketing element which forms the communication link between the organization and its target market (that is, potential and existing buyers, consumers, customers). its main functions are to inform and persuade the target market to act positively towards an organization or its products. effective promotion of insurance products is crucial due to their intangible nature, and the similarity of products offered by most operators in the insurance industry (patt, suarez, and hess, 2010). given the similarities of their product offerings, the target market may not be able to make clear distinctions between different insurance services, and are often not aware of the wide range of insurance services that are available (patt, suarez, and hess, 2010). thus, it is important for organizations in the nigerian insurance industry to develop an appropriate promotional strategy to reach their target market particularly because research has revealed that in the insurance industry, promotion strategies have been widely used to create awareness and to increase customer patronage (saaty and ansari, 2011). also, given that research has shown that the selection of promotional elements/tools depends on certain factors, organizations in the nigerian insurance industry need to decide on the appropriate promotional strategy to use given these factors. this study sought to contribute to extant literature by applying the ahp model to help organizations in the nigerian insurance industry develop their promotional strategy. a study of this nature is also important because review of extant literature reveals that promotion in the financial services industry, a group to which insurance companies belong, is an understudied research area, compared to manufacturing firms (grankvist, carolina, and ann, 2004). as such, this study seeks to add to the body of knowledge in this area. 2. literature review 2.1 promotion tools in service organizations organizations use various tools to promote their products. the major promotional tools used in physical product organizations are advertising, personal selling, sales promotion, publicity and public relations. the combination of these tools results in the promotional mix used for organizational promotional strategy. this is in support of lovelock and wirtz’s (2004) view that the promotion mix elements include: advertising, personal contact, publicity and public relations, sales promotion, instrumental material and corporate design. in addition to these, kotler and armstrong (2010) added direct marketing as a promotional element. in other related studies, sponsorship was suggested as another element of the marketing communication mix (fill, 2005; kotler and keller, 2009, idris, et al., 2012). with regards to service organizations, okyere, et al. (2011) confirmed that service firms use various tools of communication to promote their ijahp article: dixon-ogbechi, jagun, ighomereho et al./ determination of promotional strategy for organizations in the nigerian insurance industry using the ahp model international journal of the analytic hierarchy process 71 vol. 7issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.261 offerings in order to achieve their promotional objectives. donelly and peter (1998) identified advertising, sales promotion and personal selling as the three marketing communication tools used by service organizations, while etzel, et al. (2001) opined that the major promotion tools are advertising, personal selling, sales promotion and public relations. therefore, for this study we assume that the seven major promotion tools used in service firms are: advertising, personal selling, sales promotions, public relation, publicity, direct marketing and sponsorship (kotler, 2000; idris et al., 2012). advertising is viewed as any paid form of non-personal communication about an organization, good, service or an idea by an identified sponsor (kotler and armstrong, 2010). personal selling, unlike other promotional tools, involves direct relationships between the seller and the prospect or customer (kotler, 2001). kotler and armstrong (2010) claimed that personal selling is the most effective promotional mix element in building up buyer’s preferences, conviction and action in certain stages of purchase decision. meidan (1996) is of the opinion that personal selling is probably the most important and frequently used element of the promotion tools in the financial services industry. kurtz and boone (2006) referred to sales promotion as any marketing activities used by manufacturers or producers to induce and provide extra value to consumers and dealers with the goal of stimulating consumer purchases and dealer effectiveness. public relations is viewed by kotler and armstrong (2010) as a conscious and coordinated effort by an organization to maintain good relations with the company’s various stakeholders. kotler and armstrong (2010) further conceptualized publicity as the dissemination of information through personal or non-personal means, which is not directly paid for by the organization, and the organization is not the source. direct marketing is the promotional element which entails using consumer direct channels to communicate with customers without the use of marketing middlemen (kotler and keller, 2006). lagae (2005) defined sponsorship as a form of business agreement between two parties, where the sponsor provides money, goods, services or know-how, whilst the sponsored party (individual, event or organization) offers rights and associations that the sponsor utilizes for commercial purposes. the above notwithstanding, achumba (2004) observed that in developing its promotional strategy, an organization might require some mixture of two or more of these elements given that the various promotional mix elements are not mutually exclusive. 2.2 factors/criteria that determine the choice of promotion tools research has shown that companies that communicate effectively with their customers through the use of the right type of promotional tools are better able to attract, maintain and satisfy their customers (balaghar, et al. 2012). however, the decision of which promotional strategy to use is a complicated one due to the inherent strengths and weaknesses of the various types of promotional tools and the need for companies to select the right combination of the promotional elements given certain factors (kotler and armstrong, 2010). lancaster and massingham (1999) cited by idris et al. (2012) suggested five factors which usually influence the selection of promotion mix elements as the nature of the market, the nature of the product, the stage in the product life cycle, price and the funds available for promotional activities. according to kotler (2000) companies face the task of distributing the promotion budget over the various promotion tools available. reena (2013) opined that the choice of promotion tools is largely influenced by seven factors namely; push and pull strategies, product features, stage of the product life cycle, buyer readiness, type of buyer, type of distribution as well as promotion objectives, budget, cost and availability of media. riley (2012) identified the ijahp article: dixon-ogbechi, jagun, ighomereho et al./ determination of promotional strategy for organizations in the nigerian insurance industry using the ahp model international journal of the analytic hierarchy process 72 vol. 7issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.261 four factors that should be taken into account when deciding which and how much of each tool to use in a promotional marketing campaign as resource availability, the cost of each promotional tool, market size/concentration and customer information needs. on the other hand, in the view of nickels, mchugh and mchugh (2002) the relative emphasis given to each element of the promotion mix depends on the type of product, stage in the product life cycle, target market characteristics, actions of competitors and available funds. according to kotler (2001) target audience, competitor’s strategy, type of product, message content and organization budget are the major criteria for the choice of promotion tools. he further suggested supporting marketing mix strategies and the available promotion budget as two major organizational factors that influence the structure of a company’s promotion mix. meanwhile, learn marketing (2001) identified the characteristics of the product or service, characteristics of the target market, characteristics of the organization itself, competitors' promotion activities and the firm's promotion objectives as relevant factors in promotional mix decisions. based on the synthesis of the factors influencing the selection of promotion tools identified by various authors, we identified nine (9) criteria which include: type of product (tp); stage in product life cycle (splc); promotional budget (pb); type of distribution strategy [push or pull] (tps); target market characteristics (tmc); cost of media (cm); availability of media (am); communication objective (co) and actions of competitors (ac) as determinants of the promotional tools used by organizations and these were used in this study. 3. objectives/hypothesis 3.1 research objectives this study aims at determining: a. the major promotional tools used by companies in the nigerian insurance industry for their promotional strategy. b. the relative importance of the criteria/factors taken into consideration by organizations in the nigerian insurance industry when developing their promotional strategy. c. the promotional strategy for companies in the nigerian insurance industry using analytic hierarchy process (ahp). 3.2 research hypothesis: h0: there is no significant difference among the relative importance ratings of the criteria/factors taken into consideration by organizations in the nigerian insurance industry when developing their promotional strategy. h1: there is a significant difference among the relative importance ratings of the criteria/factors taken into consideration by organizations in the nigerian insurance industry when developing their promotional strategy. 4. research design/methodology 4.1 research design for the purpose of this study, the researchers adopted a survey research design using a quantitative research approach. the justification for choosing a survey method was based ijahp article: dixon-ogbechi, jagun, ighomereho et al./ determination of promotional strategy for organizations in the nigerian insurance industry using the ahp model international journal of the analytic hierarchy process 73 vol. 7issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.261 on the fact that the study is interested in gaining an understanding of the problem being investigated (ezirim, et al. 2004), without attempt to manipulate or control the subjects (asika, 1991). 4.2 population, sample size and sampling technique the population of this study consists of the 59 insurance companies that are licensed and listed with the national insurance commission of nigeria (naicom, 2013). the sampling technique employed in this study was multi-stage in nature. the first stage entailed using the judgmental sampling technique to sample the sixteen (16) most popular and active insurance companies in nigeria. according to onuoha (2012) these include: aiico insurance plc, niger insurance plc, industrial and general insurance (igi), leadway assurance, nicon, lasaco, oasis, mutual benefits assurance plc, royal exchange, crusader, savana insurance, gateway insurance, quality insurance, liberty insurance, cbn agric insurance limited and access insurance. thereafter, a census of the managers in the selected insurance companies was studied. lastly, the stratified sampling technique was used to group the sample of interest into three strata i.e. top management staff, middle level management staff and junior management staff. this sampling approach was adopted to overcome the problem of some members of the sample being over or under represented (collis and hussey, 2003). 4.3 instrumentation the survey instrument used in this study was a questionnaire that was developed so that respondents could complete it by independently. a self-administered questionnaire was chosen due to the fact that this type of survey offers respondents greater anonymity, which encourages them to more readily disclose feelings and attitudes (cooper and schindler, 2008). the questionnaire consisted of three sections (a, b and c). section a was designed based on saaty’s (2001) 9-point scale, ranging from “equally important 1”, fairly moderately more important 2”, moderately more important 3”, “fairly strongly more important 4”, “strongly more important 5”, “fairly very strongly more important 6”, “very strongly more important 7”, “fairly extremely (absolutely) more important 8”, and extremely (absolutely) more important 9” . section b asked questions on the adoption and practices of promotional strategies by the companies, while section c collected general demographic information of the respondents (i.e. gender, age, religion, educational qualification, cadre of employment, year company was established and years of working experience). the questionnaire was subjected to expert opinion for content validity before it was administered. 4.4 the ahp model this study used the ahp model to assist managers in the insurance industry in determining the relative importance of the promotional tools and in turn using this knowledge to develop their promotional mix. the goal being pursued is at the highest level of the hierarchy, the criterion is next at the middle or intermediate level of the hierarchy, and the alternatives to be evaluated are at the lowest level (saaty, 1980; 2000 and 2001). in this study, the goal is to determine the promotional strategy for insurance companies, and the criteria are the nine important factors influencing the choice of the promotional tools. the promotion alternatives to be evaluated are at the lowest level, and are the seven promotional tools/techniques. all the variables were identified from a literature search. this is expressed figure 1. ijahp article: dixon-ogbechi, jagun, ighomereho et al./ determination of promotional strategy for organizations in the nigerian insurance industry using the ahp model international journal of the analytic hierarchy process 74 vol. 7issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.261 figure 1. ahp model of study where: goal= determine promotional strategy for insurance companies criteria = actions of competitors (ac), availability of media (am), communication objective (co), cost of media (cm), promotional budget (pb), stage in product life cycle (splc), target market characteristics (tmc), type of distribution strategy [push or pull] (tps), and type of product (tp). alternatives = advertising (a), direct marketing (dm), personal selling (ps), publicity (pct), public relation (pr), sales promotions (sp) and sponsorship (spn). 4.5 methods of data analysis the data collected from section a was analyzed using expert choice (ec8). ec8 is a software developed to simplify the computations involved in analyzing data obtained from the relative importance scaled statements and the automations of computations involved in analyzing such data using ahp (timor, 2006). while the statistical package for social sciences (spss-20) was used to analyze data gathered in sections b and c of the questionnaire, the data analyses in these sections were descriptive (frequency distributions, percentages, means and standard deviations). the hypothesis was tested using the one-way analysis of variance (anova) at the 5% level of significance while post hoc tests were conducted using the least significant difference (lsd) and duncan tests at alpha=0.05. ad dm ps pct pr sp spn ac am cm spls tmc tps tp determine promotional strategy for insurance companies co pb ijahp article: dixon-ogbechi, jagun, ighomereho et al./ determination of promotional strategy for organizations in the nigerian insurance industry using the ahp model international journal of the analytic hierarchy process 75 vol. 7issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.261 5. data/model analysis and discussion 5.1 descriptive analysis a total of 111 responses were received from the 208 questionnaires distributed for a 53% response rate. the aiico insurance returned 17 of 20 or 85%; nicon insurance returned 10 of 20 or 50%; niger insurance returned 5 of 18 or 28%; gateway insurance returned 6 of 15 or 40%; agric insurance returned 12 of 15 or 80%; lasaco insurance returned 4 of 10 or 40%; oasis insurance returned 4 of 10 or 40%; royal exchange assoc. returned 3 of 10 or 30%; wapic ins returned 7 of 10or 70%; ind. & general insurance returned 7 of 10 or 70%; leadway assoc. returned 9 of 20 or 45%; mutual ben assoc. returned 14 of 20 or 70%; and crusader insurance returned 13 of 20 or 65%. sixty two percent of the respondents were male, and the highest proportion, 40%, was in their forties. none of the respondents were over 60, and only 13% were younger than 30. the religious breakdown was fairly even: 52% christian; 48% muslim. only 3% of the respondents had education less than bachelor’s degree, and 54% had a master’s degree. 41% were in middle management, 36% were in lower management, followed by 21% in top management. seven companies reported that they did not have a specialized division in charge of promotion, and one company’s respondents were mixed in their responses to that question. six companies reported having a specialized department in charge of promotion. two companies appeared to not have budgetary allocations for promotion and of the respondents who said they did have allocations (85%), most rated them high (42%) or medium (37%). 62% felt that the customer’s response to the promotions was encouraging, while 20% of them considered it fairly encouraging. only 14% said they seldom or never used promotional strategies, while the highest proportion reported using them regularly. when asked what benefit they expected from adopting promotional strategies respondents could select more than one response. the most often selected responses were increased market share, noted 64 times; consumer awareness, noted 45 times; and enhanced competitive positioning, noted 31 times. the group was fairly evenly split between those who felt that the adoption of promotional strategies in insurance companies was poor 47%, and those that did not think it was poor, 53%. the reasons given by those who thought it was poor were the inadequate budgetary allocation, (noted 37 times), the fact that it is costly (noted 34 times), and the intangible nature of the insurance products and services (noted 23 times). 5.2 consistency index (ci) the consistency ratios of all the pair-wise comparison matrices were not greater than 0.1, hence the judgments of the respondents were all seen to be consistent and therefore acceptable. 5.3 composite priorities 5.3.1 composite priorities of promotional tools the composite priorities and ranking of the promotional tools used by companies in the nigerian insurance industry for their promotional strategy are as presented in table1. ijahp article: dixon-ogbechi, jagun, ighomereho et al./ determination of promotional strategy for organizations in the nigerian insurance industry using the ahp model international journal of the analytic hierarchy process 76 vol. 7issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.261 table1 average composite priorities & ranking of the promotional tools for the insurance industry promo-tools ad dm ps pct pr sp spn average composite priority 0.316 0.248 0.186 0.112 0.068 0.043 0.027 relative importance ranking 7 6 5 4 3 2 1 source: survey research (2014) table 1 reveals that the most important promotional tool for companies in the nigerian insurance industry is advertising followed by direct marketing, personal selling, publicity, public relations, sales promotion and sponsorship respectively. 5.3.2 average composite priorities of criteria/factors the average composite priorities and ranking of the criteria/factors taken into consideration by organizations in the nigerian insurance industry when developing their promotional strategy are as presented in table 2. table 2 average composite priorities & ranking of the criteria/factors considered for promostrategy by insurance industry promotools ac am co cm pb splc tmc tps tp average composite priority 0.227 0.227 0.173 0.133 0.086 0.063 0.043 .030 .018 relative importance ranking 8.5 8.5 7 6 5 4 3 2 1 source: survey research, 2014 table 2 reveals that the most important factors influencing the selection of promotion tools by companies in the nigerian insurance industry are actions of competitors (ac) and availability of media (am), followed by communication objective (co); cost of media (cm); promotional budget (pb); stage in product life cycle (splc); target market characteristics (tmc); type of distribution strategy [push or pull] (tps) and type of product (tp). 5.4 hypothesis test the one-way analysis of variance (anova) was used to test the hypothesis that there is no significant difference among the relative importance ratings of the promotional tools ijahp article: dixon-ogbechi, jagun, ighomereho et al./ determination of promotional strategy for organizations in the nigerian insurance industry using the ahp model international journal of the analytic hierarchy process 77 vol. 7issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.261 used by organizations in the nigerian insurance industry when developing their promotional strategy; this is as shown in table 3: table 3 anova test of hypothesis h0 ahp ratings sum of squares df mean square f sig. between groups .703 8 .088 65.106 .000 within groups .146 108 .001 total .848 116 source: survey research, 2014 table 3 reveals that there are statistically significant differences among the means, therefore h0 is rejected. given this a post hoc test was conducted using the least significant difference (lsd) and a duncan test to know which specific groups differ from each other. the lsd was used to do pairwise comparisons of these differences and it was discovered that there were significant differences among all groups except for ac and am; pb and spls; spls and tmc; tmc and tps; tmc and tp; and tps and tp. the duncan test was subsequently used to group the variables by how close such ratings are; and it was revealed that tp, tps and tmc; tmc and spls; spls and pb; am and ac were close. this is consistent with the ahp output findings. the duncan test results are presented in table 4. table 4 duncan test of ahp promotional criteria ratings promotional subset for alpha=0.05 criteria n 1 2 3 4 5 6 duncan a tp 13 .0182 tps 13 .0298 tmc 13 .0428 .0428 spls 13 .0630 .0630 pb 13 .0863 cm 13 .1328 co 13 .1731 am 13 .2269 ac 13 .2273 sig. .109 .165 .109 1.000 1.000 .979 source: survey research (2014) 6. limitations though we wanted to study all the sixteen selected insurance companies, we were only able to access thirteen because all efforts to locate the remaining three proved ijahp article: dixon-ogbechi, jagun, ighomereho et al./ determination of promotional strategy for organizations in the nigerian insurance industry using the ahp model international journal of the analytic hierarchy process 78 vol. 7issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.261 unsuccessful. this however, did not affect the outcome of our study since the thirteen we studied were representative of the population. 7. conclusions our findings reveal that the promotional tools used by companies in the nigerian insurance industry for their promotional strategy are advertising, direct marketing, personal selling, publicity, public relations, sales promotion and sponsorship. however, the major promotional tools used are advertising, direct marketing, personal selling, and publicity. this is in support of okyere, et al.’s (2011) view that service firms use various tools of communication to promote their offerings in order to achieve their promotional objectives. the findings are partly in support of the views that advertising, sales promotion, personal selling (donelly and peter, 1998) and public relations (etzel, et al. 2001) are the major promotional tools used by service organizations. they are contrary to meidan’s (1996) view that personal selling is probably the most important and frequently used element of the promotion tools in the financial services industry, a class to which insurance companies also belong. the relative importance ranking of the criteria/factors taken into consideration by organizations in the nigerian insurance industry when developing their promotional strategy reveals that the actions of competitors (ac) and availability of media (am) were the most important followed by communication objective (co), cost of media (cm), promotional budget (pb), stage in product life cycle (splc), target market characteristics (tmc), type of distribution strategy [push or pull] (tps) and type of product (tp).our findings are at variance with kotler’s (2001) view that target audience, competitor’s strategy, type of product, message content and organization budget are the major criteria for the choice of promotion tools, and with riley’s (2012) position that the four factors that should be taken into account in deciding which and how much of each tool to use in a promotional marketing campaign are resource availability, the cost of each promotional tool, market size/concentration and customer information needs. however, the findings are partly in support of the views of lancaster and massingham (1999) and reena (2013). 8. recommendations in developing their promotional strategies, companies in the nigerian insurance industry should invest more on advertising, direct marketing, personal selling and publicity given that these have a pooled composite priority of 86.2%, while investing minimally in public relations, sales promotion and sponsorship. therefore, we recommend that organizations in the nigerian insurance industry should allocate 86.2% of their promotional budget to advertising, direct marketing, personal selling and publicity, and the remaining 13.8% should be allocated to public relations, sales promotion and sponsorship. this would provide an optimal mix for their promotional strategy. further studies can be carried out by applying the analytic hierarchy process (ahp) to investigate the relative importance of promotional tools among selected companies in nigeria. the ahp can also be used to carry out a comparative study of the promotional strategies employed by multinationals and indigenous companies in nigeria. ijahp article: dixon-ogbechi, jagun, ighomereho et al./ determination of promotional strategy for organizations in the nigerian insurance industry using the ahp model international journal of the analytic hierarchy process 79 vol. 7issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.261 ijahp article: dixon-ogbechi, jagun, ighomereho et al./ determination of promotional strategy for organizations in the nigerian insurance industry using the ahp model international journal of the analytic hierarchy process 80 vol. 7issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.261 references alireza m., ramak, r., & abbas, r. (2011). exploring innovative promotional strategies in life insurance companies (case study of iran), 2011 international conference on sociality and economics development, 1-5. achumba, i.c. (2004). strategic marketing management in the 21 st century. lagos: macwilliams and capital publishers incorporation. asika, n. (1991). research methodology in the behavioural sciences. lagos: longman press. balaghar, a.a., majidazar, m. & niromand, m. (2012). evaluation of effectiveness of sales promotional tools on sales volume (case study: iran tractor manufacturing complex (itmc)), middle-east journal of scientific research, 11 (4): 470-480. collis, j. & hussey, r. (2003). business research: a practical guide for undergraduate and postgraduate students, 2 nd edition. basingstoke: palgrave macmillan. cooper, d.r. & schindler, p.s. (2008). business research methods, (10 th ed.). new york: mcgraw-hill international. cox, p. (2007). analyzing the determinants of narrative risk information in uk ftse 100 annual reports. british accounting review, 39(3), 227-248. donelly, (jr.), j.h. & peter, j.p. (1998). marketing management: knowledge and skill. usa: mcgraw-hill companies inc. dixon-ogbechi, bolajoko nkemdinim, odugbemi, tolu & aiyeku, joseph fola (2011). marketing nigerian medicinal plants: a 4ps perspective. international conference of the international academy of african business and development (iaabd), edmonton, canada, 663-669. etzel, m.j., walker, b.j., & stanton, w.j. (2001). marketing (12th ed.). new york: mcgraw-hill companies inc. ezirim, a.c., briggs, b.r. & maclayton, d.w. (2004). research methodology for management and social sciences. nigeria: mccomputer press. fill, c. (2005). marketing communication: engagement, strategies and practices (4th ed.). england: pearson education limited. grankvist, a., carolina, k & ann, p. (2004). promotion strategies for banking services: case study of nordea in estonia, bachelor thesis, lulea university of technology. retrieved october 20, 2013, from: http://www.pure.itu.se/portal/files.pdf. idris, a. a.,asokere, a. s.,ajemunigbohun, s. s.,oreshile, a. s. & olutade, e. o. (2012). an empirical study of the efficacy of marketing communication mix elements in http://www.pure.itu.se/portal/files.pdf ijahp article: dixon-ogbechi, jagun, ighomereho et al./ determination of promotional strategy for organizations in the nigerian insurance industry using the ahp model international journal of the analytic hierarchy process 81 vol. 7issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.261 selected insurance companies in nigeria, australian journal of business and management research, 2(5), 8-18. kotler, p. (2000). marketing management (millenium edition ed.). london: prenticehall. kotler, p. (2001). marketing management (10th ed.). london: prentice hall international editions. kotler, p & armstrong, g. (2010). principles of marketing, 13 th edition. new jersey: pearson educational, incorporation. kotler, p & keller, k. (2006). marketing management, 12 th edition. new jersey: pearson educational inc. kotler, p., g. armstrong, v. wong & j. saunders (2009). advertising in sports centers. in principles of marketing (5th edition), prentice advertising on the internet. (7-17). new jersey: prentice hall. kurtz, d. & boone, l. (2006). principles of marketing, 12 th edition. independence, kentucky: south western college publishing. kotler, p., & keller, k.l. (2009). marketing management (13th ed.). new jersey: pearson education inc. learn marketing (2001). introduction to promotion: determining the composition of the promotion mix, retrieved october 20, 2013, from: http://courses.unt.edu/kt3650_27/sld 002.htm. lancaster, g., & massingham, l. (1999). essentials of marketing. london: mcgraw-hill. lagae, w. (2005). sports sponsorship and marketing communication: a european perspective. harlow: ft prentice hall. lovelock, c., & wirtz, j. (2004). services marketing: people, technology, strategy (5th ed.). new jersey: pearson education international. meidan, a. (1996). marketing financial services. hamsphire and london: macmillan press ltd. mccarthy, e. j. (2002). basic marketing, il: richard d. irwin. naicom (2013). national insurance commission of nigeria -insurance and reinsurance companies in nigeria, retrieved october 20, 2013, from: http://naicom. gov.ng/companies. nickels, w.g., mchuhg, j.m. and mchugh, s.n. (2002). understanding business. new york: mcgraw-hill http://courses.unt.edu/kt3650_27/sld http://naicom/ http://naicom.gov.ng/companies ijahp article: dixon-ogbechi, jagun, ighomereho et al./ determination of promotional strategy for organizations in the nigerian insurance industry using the ahp model international journal of the analytic hierarchy process 82 vol. 7issue 1 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i1.261 okyere, n.y., agypong, g.k., & nyarku, k.m. (2011). the effect of marketing communications on the sales performance of ghana telecom (vodafone, ghana). international journal of marketing studies, 3(4), 50-62. onuoha, r. (2012). 16 insurance companies known to nigerians, retrieved october 20, 2013, from: http//www.vanguardngr.com/2012/10/16-insurance-companies-known-tonigerians-report/ patt, a., suarez, p., & hess, u., (2010). how do small-holders understand insurance, and how much do they want it? evidence from africa. global environmental change, 20, 153–161. reena, j. (2013). 7 important factors that affect “promotion mix”, retrieved october 20, 2013, from: http://www.shareyouressays.com/112830/7-important-factors-that-affectpromotionmix. riley, j. (2012). promotion introducing the promotional mix, retrieved october 20, 2013, from: http://www.tutor2u.net/business/marketing/promotion_factors.asp. shimp, t.a. (2007). integrated marketing communication in advertising and promotion, 7 th edition. mason, oh: thomson southwestern. saaty, t.l. (1980). the analytic hierarchy process, planning, priority setting, resource allocation, new york: mcgraw-hill. saaty, t.l. (2001). decision making for leaders. (3 rd eds.), pittsburgh, pa: rws publications. saaty, t.l. (2000). fundamentals of decision making and priority theory with analytical hierarchy process. pittsburgh, pa: rws publications. saaty, a.s. and ansari, a.z. (2011). factors critical in marketing strategies of insurance companies in saudi arabia, international journal of marketing studies, 3(3), 104-121. http://dx.doi.org/10.5539/ijms.v3n3p104. tanner j.f. jr., in olasz, s.c. (2006). marketing’s role in a relationship age, baylor business review, waco, spring, 24(2), 2-7. http://www.shareyouressays.com/112830/7-important-factors-that-affect-promotionmix http://www.shareyouressays.com/112830/7-important-factors-that-affect-promotionmix http://www.tutor2u.net/business/marketing/promotion_factors.asp http://dx.doi.org/10.5539/ijms.v3n3p104 ijahp article: mizuno, kinoshita/ an algebraic representation via differential equations for pairwise comparisons of ahp international journal of the analytic hierarchy process 136 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.278 an algebraic representation via differential equations for pairwise comparisons of ahp takafumi mizuno faculty of urban science meijo university tmizuno@urban.meijo-u.ac.jp eizo kinoshita faculty of urban science meijo university kinoshit@urban.meijo-u.ac.jp abstract we propose a simple algebraic representation for pairwise comparisons of ahp. the representation is an associative relation between the importances of elements and consists of basic arithmetic operations. first, we define a ratio, which is estimated by decision makers by comparing the importances of elements, as a partial differentiation of importances (section 2). then, we construct systems of differential equations. algebraic representations of the importances are derived as formal solutions of the equations. we analyze pairwise comparisons and the construction of the importances from them with the representations (section 3). the validity of using eigenvectors and c.i. in ahp is illustrated by deriving a particular solution of the equations. https://doi.org/10.13033/ijahp.v9i1.278 keywords: pairwise comparison method; ahp; partial differentiation 1. introduction pairwise comparisons are primitive procedures in ahp (saaty, 1977, 1980). decision makers construct relative importances of elements from ratios of pairs of elements. let 𝑎1, ⋯, 𝑎𝑛 be the elements, and 𝑥𝑖 be an importance of an element 𝑎𝑖. decision makers want to obtain 𝑥𝑖, but they can only estimate ratios 𝑥𝑖/𝑥𝑗 by pairwise comparisons for all pairs (𝑎𝑖, 𝑎𝑗). there are many methods to derive importances from the set of ratios (cogger & yu, 1985). in the actual usage of ahp, relative importances are often obtained by applying the principal eigenvector method (saaty, 1980). in this method, a ratio 𝑟𝑖𝑗 which is an estimation of 𝑥𝑖/𝑥𝑗 is arranged in the 𝑖-th row 𝑗-th column cell in the pairwise comparison matrix 𝑅, which is 𝑛 × 𝑛 square matrix. the importances which decision makers want are obtained as elements of the principal eigenvector of 𝑅 ; a detected relative importance 𝑥�̂� is an element of vector �̂� = [𝑥1̂, ⋯ , 𝑥�̂�] 𝑡 which holds r�̂� = λ𝑚𝑎𝑥�̂�. harker and vargas (1987) discussed why we can regard the vector as the approximation of importances. their illustrations, however, are correct but quite difficult because of https://doi.org/10.13033/ijahp.v9i1.278 https://doi.org/10.13033/ijahp.v9i1.278 ijahp article: mizuno, kinoshita/ an algebraic representation via differential equations for pairwise comparisons of ahp international journal of the analytic hierarchy process 137 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.278 their analyses of eigenvectors. in a decision making process, we have to make decision makers intuitively understand the usefulness of the methods. we also want to construct useful semantics which treat mental measurements and physical models with the same scheme. in this paper, we propose a representation which simply illustrates the validity of calculations for relative importances from pairwise comparison. 2. hypotheses we presume that the importance 𝑥𝑖 of an element 𝑎𝑖 can be represented in a multivariable function whose arguments are 𝑥𝑗, 𝑗 ≠ 𝑖; x𝑖 ≡ 𝑥𝑖(𝑥1, ⋯ , 𝑥𝑖−1, 𝑥𝑖+1, ⋯ , 𝑥𝑛), 𝑖 = 1, ⋯ , 𝑛. (1) in the pairwise comparisons of the ahp, for all pairs (𝑥𝑖, 𝑥𝑗), decision makers give an estimated ratio 𝑟𝑖𝑗 which means that 𝑥𝑖 is 𝑟𝑖𝑗 times as large as 𝑥𝑗. we make the further assumption that the ratio is an estimation of the partial differentiation of these functions; 𝑟𝑖𝑗 ≡ (𝑛 − 1) 𝜕𝑥𝑖 𝜕𝑥𝑗 . (2) it means that if decision makers enlarge the estimate of the 𝑎𝑗, then the 𝑎𝑖 will be larger, and the 𝑎𝑖 growth rate of the estimate will be 𝑟𝑖𝑗 times larger than that of the 𝑎𝑗. there is a term (𝑛 − 1) in equation (2), because decision makers estimate the ratio of 𝑥𝑖 as a single-variable function whose argument is 𝑥𝑗 in spite of the former assumption that the function is an (𝑛 − 1)-variable function. 3. an analysis of the pairwise comparison method with the hypotheses in the previous section, we can write the pairwise comparison matrix 𝑅 as follows: 𝑅 = [ 𝑟11 ⋯ 𝑟1𝑛 ⋮ ⋱ ⋮ 𝑟𝑛1 ⋯ 𝑟𝑛𝑛 ] ≡ (𝑛 − 1) [ 𝜕𝑥1 𝜕𝑥1 ⋯ 𝜕𝑥1 𝜕𝑥𝑛 ⋮ ⋱ ⋮ 𝜕𝑥𝑛 𝜕𝑥1 ⋯ 𝜕𝑥𝑛 𝜕𝑥𝑛] = (𝑛 − 1)𝜕𝒙𝜕𝒙𝑡 (3) where ∂𝒙 = [𝜕𝑥1, ⋯ , 𝜕𝑥𝑛] 𝑡, and ∂𝒙 = [1/𝜕𝑥1, ⋯ ,1/𝜕𝑥𝑛] 𝑡. let 𝑑𝒙 = [𝑑𝑥1, ⋯ , 𝑑𝑥𝑛] 𝑡, and let us consider a product 𝑅𝑑𝒙. combining the formula of total differentiation, we obtain a relation 𝑑𝒙 = (∂𝒙 ∂𝒙𝑡 − 𝐼)𝑑𝒙 = 1 (𝑛 − 1) (𝑅 − 𝐼)𝑑𝒙, (4) 𝑑𝑥𝑖 = 1 (𝑛 − 1) [𝑟𝑖1𝑑𝑥1 + ⋯ + 𝑟𝑖,𝑖−1𝑑𝑥𝑖−1 + 𝑟𝑖,𝑖+1𝑑𝑥𝑖+1 + ⋯ + 𝑟𝑖𝑛𝑑𝑥𝑛]. (5) https://doi.org/10.13033/ijahp.v9i1.278 ijahp article: mizuno, kinoshita/ an algebraic representation via differential equations for pairwise comparisons of ahp international journal of the analytic hierarchy process 138 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.278 where 𝐼 is the identity matrix. notice that the total differentiation of 𝑥𝑖 is 𝑑𝑥𝑖 = ∂𝑥𝑖/𝜕𝑥1𝑑𝑥1 + ⋯ + ∂𝑥𝑖/𝜕𝑥𝑖−1𝑑𝑥𝑖−1 + ∂𝑥𝑖/𝜕𝑥𝑖+1𝑑𝑥𝑖+1 + ⋯ + ∂𝑥𝑖/𝜕𝑥𝑛𝑑𝑥𝑛 . we can represent the importances 𝒙 as a system of total differential equations. we obtain an algebraic representation of 𝑥𝑖 by integrating equation (5). 𝑥𝑖 = ∫ 𝑑𝑥𝑖 = 1 (𝑛 − 1) [∫ 𝑟𝑖1 𝑑𝑥1 + ⋯ + ∫ 𝑟𝑖𝑛 𝑑𝑥𝑛] = 1 (𝑛 − 1) [𝑟𝑖1𝑥1 + ⋯ + 𝑟𝑖,𝑖−1𝑥𝑖−1 + 𝑟𝑖,𝑖+1𝑥𝑖+1 + ⋯ + 𝑟𝑖𝑛𝑥𝑛] − 𝑑𝑖, (6) 𝒙 = 1 𝑛 − 1 (𝑅 − 𝐼)𝒙 − 𝒅 (7) where 𝒅 = [𝑑1, ⋯ , 𝑑𝑛] 𝑡 is a constant of integration. to determine the constant, we reformulate equation (7). 𝑅𝒙 = 𝑛𝒙 + (𝑛 − 1)𝒅. (8) this is an algebraic representation for importances. it has a degree of freedom caused by the constant of integration 𝒅. to find particular solutions by determining the constant of integration, let �̂� be an eigenvector of 𝑅, and λ its corresponding eigenvalue. thus the representation can be transformed to: 𝑅�̂� = λ�̂� = 𝑛�̂� + (𝑛 − 1)𝒅, (9) 𝒅 = λ − 𝑛 (𝑛 − 1) �̂�. (10) we obtain a representation of importances as the system of equations: 𝒙 = 1 𝑛 − 1 (𝑅 − 𝐼)𝒙 − λ − 𝑛 𝑛 − 1 �̂�, (11) 1 𝑛 − 1 (𝑅 − 𝑛𝐼)𝒙 = λ − 𝑛 𝑛 − 1 �̂�. (12) if the null-space of the matrix (𝑅 − 𝑛𝐼) has the same dimensions, then the solution will be 𝒙 = 𝒚 + �̂�. (13) a vector 𝒚 is the solution of the equation (𝑅 − 𝑛𝐼)𝒚=0. we can confirm that �̂� is also the solution of the equation (12). https://doi.org/10.13033/ijahp.v9i1.278 ijahp article: mizuno, kinoshita/ an algebraic representation via differential equations for pairwise comparisons of ahp international journal of the analytic hierarchy process 139 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.278 4. conclusions we propose an algebraic representation for the pairwise comparisons of the ahp. a key idea is that we regard ratios of importances as partial differentiations of them. relations between importances are derived directly from these differentiations. in section 3, we also naturally introduced why eigenvectors are needed and what c.i. the term (λ − 𝑛)/(𝑛 − 1) , is. eigenvectors are particular solutions of the system of differential equations, and c.i. is a coefficient of the nonhomogeneous term of the equations. in this paper, we demonstrate that estimated ratios can be regarded as differentials of importances without any fault. this means that we can include physical models in the pairwise comparisons of the ahp. we expect that mental measurements, which are obtained using ordinary pairwise comparisons, and physical models are treated using the same scheme. and we can apply the semantics to machine learnings, or can retrieve importances of any element automatically to put in the ahp. in real databases of physical models, there are many numeric calculations for extracting differentiations. https://doi.org/10.13033/ijahp.v9i1.278 ijahp article: mizuno, kinoshita/ an algebraic representation via differential equations for pairwise comparisons of ahp international journal of the analytic hierarchy process 140 vol. 9 issue 1 2017 issn 1936-6744 https://doi.org/10.13033/ijahp.v9i1.278 references saaty, t. (1980). the analytic hierarchy process, new york: mcgraw-hill. doi: http://dx.doi.org/10.1080/00137918308956077 saaty, t. (1977). a scaling method for priorities in hierarchical structure. journal of mathematical psychology, 15, 234-281. doi: http://dx.doi.org/10.1016/00222496(77)90033-5 cogger, k. o. & yu., p. l. (1985). eigenweight vectors and least-distance approximation for revealed preference in pairwise weight ratios. journal of optimization theory and applications, 46(4), 483-491. doi: 10.1007/bf00939153 harker, p. and vargas, l. (1987). the theory of ratio scale estimation: saaty's analytic hierarchy process. management science, 33(11), 1383–1403. doi: http://dx.doi.org/10.1287/mnsc.33.11.1383 https://doi.org/10.13033/ijahp.v9i1.278 http://dx.doi.org/10.1080/00137918308956077 http://dx.doi.org/10.1016/0022-2496(77)90033-5 http://dx.doi.org/10.1016/0022-2496(77)90033-5 http://dx.doi.org/10.1287/mnsc.33.11.1383 ijahp article: sawant, sarode/identification of criteria for third party logistics suppliers (3pl) in supply chain management (scm) in indian manufacturing industries international journal of the analytic hierarchy process 278 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.845 identification of criteria for third party logistics suppliers (3pl) in supply chain management (scm) in indian manufacturing industries swapnil sawant mechanical (manufacturing systems) lokmanya tilak college of engineering, affiliated with university of mumbai navi mumbai, maharashtra, india sawantswapnil138@gmail.com avinash sarode professor, lokmanya tilak college of engineering department of mechanical engineering, affiliated with university of mumbai navi mumbai, maharashtra, india avinashsarode@gmail.com abstract the indian manufacturing industry is growing rapidly, and supply chain management (scm) plays the most important role in the industry. in scm, customer satisfaction in terms of quantity, quality and on time delivery is the most important critical factor. to satisfy this requirement, the best third-party logistics (3pl) service provider is required. therefore, the selection of the best third-party logistics provider is one of the basic requirements in scm. logistics services are the backbone of an economy, providing the efficient, cost effective flow of goods and services on which other commercial sectors depend. the logistics companies work as the outsourced or third-party service providers and support the organization's logistics functions. in this study, we identified some important criteria for 3pl implementation in scm in indian manufacturing industries. with the help of this study, supply chain managers from small to medium sized manufacturing industries can simplify the selection process for 3pl vendors. this study will help in the selection of the best vendor from such a competitive group and provide justification for the selection. keywords: third party logistics vendor selection; 3pl; supply chain management; ahp; indian manufacturing; sensitivity analysis 1. introduction the 3pl market in india is under-developed and highly fragmented. however, recent trends show that the indian market has come of age with small family-run businesses giving way/progressing to professional-run corporate enterprises (chaudhari & sarode, 2018). this refreshing change is a welcome sign for growth, as professionalism can help build efficiency and reduce costs. the manufacturing industry's success and growth play mailto:sawantswapnil138@gmail.com mailto:avinashsarode@gmail.com ijahp article: sawant, sarode/identification of criteria for third party logistics suppliers (3pl) in supply chain management (scm) in indian manufacturing industries international journal of the analytic hierarchy process 279 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.845 an important role in the economic development of any country (sarode & kole, 2016). in the current market for each product, there is high competition that creates a challenging situation for each organization. to stay competitive each organization needs to proactively increase and improve its productivity. to stay continuously competitive, choosing the best 3pl service provider plays an important role for manufacturing industries (sarode & khodke, 2011). to maintain a competitive edge, every company needs to adjust and adapt to changes in the market and act accordingly. a company's top management plays an important role in selecting any procedure for successful scm (aguezzoul, 2014). before 20 years ago, logistics was the key factor for only some indian manufacturing industries, but now these industries seem to be a step ahead of others in their use of logistics. logistics costs in india are about 13% of the country's total gdp. for example, in 2012-13, the cost of logistics was around $130 billion (mitra, 2008). our study seeks to identify and prioritize the dominant cf's (critical factors) for adopting the best logistics practices for indian industries. the identification and prioritization of cf's may help practitioners/managers better achieve their goals. in this study, we identify and analyze the various critical factors (cf's) in adopting logistics practices. in this paper, we identify and rank the drivers/enablers for 3pl implementation in supply chains. in order to identify the enablers for 3pl, it is necessary to prepare a method which is capable of collecting appropriate information. therefore, this paper identified 28 enablers to solve the above problem. data was collected from various manufacturing industries in india and thirteen main criteria including cost, relationships, services, quality, it systems, flexibility, government regulations and policy, and others, were categorized into 42 enablers. this paper ranks the enablers in the context of indian manufacturing using the ahp to enhance the supply chains the structure of this paper is organized as follows: the literature review is presented in section 2. section 3 presents the solution methodology with the ahp framework. ranking of enablers/drivers is discussed in the results and discussion presented in section 4. section 5 presents a case study of practical application verified by using a sensitivity analysis. the final section 6 summarizes the conclusions and future scope. 1.1 problem statement our main objective in this research is to analyze the key factors required in 3pl's implementation in scm. then, we will select and prioritize the factors by using an ahp framework. the final task is to validate the work by using a sensitivity analysis in a case study. 1.2 research objective to determine the solutions to the problem, we optimized the objective of our research as follows. we studied the awareness and importance given to the selection of third party logistics members in the industries. also, we identified several drivers to implement successful 3pl in the supply chain. for this, we found key success factors discussed in the literature. then, we conducted a pilot study among top management, academic experts, field experts and others to validate and finalize the key success factors. for this survey, we collected responses from industrial experts and evaluate the responses. based on these responses, we identified the essential enablers for 3pl in sc through mcdm ijahp article: sawant, sarode/identification of criteria for third party logistics suppliers (3pl) in supply chain management (scm) in indian manufacturing industries international journal of the analytic hierarchy process 280 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.845 (multi criteria decision modeling). further, to validate our work, we studied the awareness and the importance given to the selection by using a case study with the help of a sensitivity analysis. 2. literature review this literature review will summarize the literature on logistics related to supply chain management (scm) and the identification of key success factors to implement successful logistics in organizations. the strategy we used to review the literature was to find relevant keywords by defining the project and the key concepts of the project, and generating keywords using synonyms. through this process, we found key success factors through a detailed literature review. sarode and kole (2016) found in their literature search that 15 critical factors are the most important. aguezzoul (2014) found that top management support, cost, quality, operational performance, and customer satisfaction are the major critical success factors. in this paper, 87 articles published between 1994-2013 were reviewed, and based on the analysis, cost, relationship, services, quality, information and equipment systems, flexibility, delivery, professionalism, financial position, location, and reputation were the 11 main factors which were classified and sub divided into 45 sub criteria. the authors categorized articles into five groups according to the technique used. eight papers used the ahp technique for criteria selection. peng (2012) identified and prioritized logistics out-sourcing service suppliers using ahp and presented a case study of a frozen food enterprise. in this case study, based on an evaluation index system, logistics cost, operating efficiency, service quality, and technology level were targeted as critical success factors. according to an article by garg (2018), 16 variables that influence the logistics performance were classified under six levels based on their dependability. investment by government and logistics service providers (lsp) plays an important role in improving the performance of logistics. according to the author, lsp should improve and concentrate on continuous development in information technology and human resources. luthra (2017) stated that logistics initiatives in business have been growing for the past one to two decades, which has resulted in environmental concerns gaining more attention. in this paper, based on a case study of indian auto component manufacturers, regulatory factors, competitiveness factors, financial factors and supply chain factors were determined to be important in 3pl implementation, and were further classified into 13 sub-factors for detailed information. birgul (2015) identified five key strategic factors broken into 20 sub-factors that help satisfy customer needs in terms of quality, quantity and on-time delivery by third party logistics providers. in this article, an ahp based framework is designed for an it distribution company. xu (2017) classified first-order performance evaluation indices as financial performance, operation process, innovation ability and customer relationship, followed by 13 second-order indices. mangla (2016) provided a structural model of logistics implementation in manufacturing industries in india based on ahp in which he defined five main factors including regulatory factors, global competitiveness factors, economic factors, hr and organizational factors and strategic factors followed by 25 detailed sub-factors. ijahp article: sawant, sarode/identification of criteria for third party logistics suppliers (3pl) in supply chain management (scm) in indian manufacturing industries international journal of the analytic hierarchy process 281 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.845 korpela (1996) defined and prioritized performance for achieving superior logistics. according to the international benchmarking clearinghouse, "benchmarking is a process in which companies target key improvement areas within their firms, identify and study best practices of others in these areas and implement new processes and systems to enhance their own productivity and quality." reliability, flexibility, lead-time, cost effectiveness, and value added are prioritized as critical success factors of logistics. yahya (1999) confirms that in every organization, selection of the best 3pl (third party logistics providers) is always difficult for top management. before making any decision, buyers and purchasing managers must analyze the strengths and weaknesses of the 3pl suppliers being considered. in a simple categorized model, eight main criteria are identified, followed by 13 sub-criteria. mothilal (2014) identified the key success factors of 3pl in the indian manufacturing sector using studies from other locations such as the us, north america, hong-kong, uk, china, france, and finland divided into the sectors of 3pl, mnc's, manufacturing, and the automotive industry. when the findings of those studies were compared with those from the indian industrial 3pl sector, six key success factors were identified of which the main three were breadth of services, industrial focus, and investment in it. mitra (2016) conducted a case study in north america about 3pl in manufacturing industries by using statistical data analysis. according to this article, internationalization, industry focus, investment in information systems, availability of skilled professionals and integration of supply chains stand out as the most important success factors. mitra (2009) carried out an in-depth survey of indian express delivery service providers in terms of demographics, success factors, performance metrics, problems faced and opportunities for growth by studying clusters of micro and small industries, and medium and large industries and identified fourteen success factors. globalization, on-time delivery, breadth of services, relationships with customers and investment in it were found to be the main success factors. mitra (2010) compared north american 3pl firms with indian firms and found that 3pl experience and 3pl relationships were the key success factors for north american firms while these are only moderately important factors for indian firms. according to this research, indian firms lag behind north american firms because they invest less in information systems, have a low availability of management talent, and poorly integrate their supply chains. from our in-depth literature review, we summarized the results into 13 main factors and 42 sub-factors as shown in table 1. ijahp article: sawant, sarode/identification of criteria for third party logistics suppliers (3pl) in supply chain management (scm) in indian manufacturing industries international journal of the analytic hierarchy process 282 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.845 table 1 drivers/enablers from the literature sr. no. criteria sr. no. sub criteria 1 cost reduction ✔ ✔ ✔ ✔ 2 expected leasing cost ✔ ✔ ✔ 3 operation cost ✔ ✔ ✔ ✔ ✔ ✔ 4 warehousing cost ✔ ✔ ✔ 5 cost saving ✔ ✔ ✔ ✔ ✔ 6 economic benefits ✔ ✔ ✔ ✔ ✔ ✔ ✔ 1 3pl opportunistic behaviour ✔ ✔ ✔ ✔ ✔ ✔ 2 reliability ✔ ✔ ✔ ✔ ✔ ✔ 3 truth & trust ✔ ✔ ✔ ✔ ✔ 4 integration & co-ordination ✔ ✔ ✔ ✔ ✔ ✔ ✔ 1 breadth of services ✔ ✔ ✔ ✔ ✔ ✔ ✔ 2 pre-sale / post-sale services ✔ ✔ ✔ ✔ 3 value added services ✔ ✔ ✔ ✔ 1 commitment to continuous improvement ✔ ✔ ✔ ✔ 2 environmental issues ✔ ✔ ✔ ✔ 3 risk management ✔ ✔ ✔ 1 information accessability ✔ ✔ ✔ ✔ ✔ ✔ 2 information technology ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ 3 storage technology ✔ ✔ ✔ 4 transportation technology ✔ ✔ ✔ 1 ability to meet future requirement ✔ ✔ ✔ ✔ ✔ ✔ 2 customer satisfaction ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ 1 on time delivery/shipment ✔ ✔ ✔ ✔ ✔ 2 lead time ✔ ✔ ✔ 3 accuaracy of delivery time ✔ ✔ ✔ ✔ ✔ 4 reduction in time & raw material consumption ✔ ✔ ✔ ✔ ✔ 1 punctuality ✔ ✔ ✔ 2 expertise ✔ ✔ ✔ ✔ 3 experience ✔ ✔ ✔ 1 upgrading of equipments & services ✔ ✔ ✔ ✔ ✔ 2 improved process efficiency ✔ ✔ ✔ ✔ ✔ ✔ 1 geographyical specialization & coverage ✔ ✔ ✔ ✔ 2 international scope ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ 1 govt. legislation & support ✔ ✔ ✔ ✔ ✔ 2 tax benefits & related policy ✔ ✔ ✔ 1 competition among firm ✔ ✔ ✔ 2 benchmarking standards ✔ ✔ ✔ ✔ ✔ ✔ 3 adopting sustainability ✔ ✔ ✔ 1 role of stakeholder & supports ✔ ✔ ✔ 2 top management support ✔ ✔ ✔ ✔ ✔ 3 skilled human forces ✔ ✔ ✔ ✔ ✔ ✔ 4 social responsibility ✔ ✔ ✔ 5 customer environmental awareness ✔ ✔ ✔ ✔ ✔ 1 cost (aicha a., 2014) [2] (jian liang peng, 2012) [3] (s. mangla, 2016) [8] (j. korpela, 1996) [9] (s. yahya, 1999) [10] (s. mothi lal, 2014) [11] (s. k. garg, 2018) [4] (s. luthra, 2017) [5] (birgul k. c., 2015) [6] (subo xu, 2017) [7] (tugrul u. d., 2015) [12] author & year ⏩ drivers ⏬ services3 flexibility & reputation 6 10 locations regulatory & policy competitiveness supply chain factor 13 12 11 9 8 7 5 information systems + technology 4 quality delivery professionalism finiancial position 2 relationships ijahp article: sawant, sarode/identification of criteria for third party logistics suppliers (3pl) in supply chain management (scm) in indian manufacturing industries international journal of the analytic hierarchy process 283 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.845 3. methodology from the literature review, we identified 13 main factors and 42 sub-factors that could be used to rank criteria and sub-criteria for selecting 3pl for scm (supply chain management) in indian manufacturing industries. a complex problem can be solved using the techniques of mcdm (multi-criteria decision making). we propose the framework of criteria and sub-criteria shown in figure 1 for prioritizing the criteria and sub-criteria defined in table 2. 3.1 framework for 3pl implementation in supply chain management in this section, we prioritized the key success factors for finding the best 3pl in supply chain management by using the analytic hierarchy process (ahp) approach. an ahpbased framework with four levels is shown in figure 1. the hierarchy level 1 indicates the objective of the research, i.e., to identify key success factors required for selecting the best 3pl in scm. the hierarchy level 2 shows the main key success factors. the hierarchy level 3 shows the sub-factors of drivers/enablers. the hierarchy level 4 shows the bottom level, where the key success factors are prioritized. figure 1 ahp framework for identification of key factors for 3pl implementation in scm ijahp article: sawant, sarode/identification of criteria for third party logistics suppliers (3pl) in supply chain management (scm) in indian manufacturing industries international journal of the analytic hierarchy process 284 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.845 table 2 definitions of symbols in figure 1 sr. no. drivers symbol sr. no. drivers symbol 1. cost [cost] a. on time delivery/shipment [delr 1] a. cost reduction [cost 1] b. lead time [delr 2] b. operation cost [cost 2] c. accuracy of delivery time [delr 3] c. warehousing cost [cost 3] d. reduction in time of raw material consumption [delr 4] d. cost saving [cost 4] 8. professionalism [prof] e. economic benefits [cost 5] a. punctuality [prof 1] 2. 3pl relationships [relt] b. expertise [prof 2] a. 3pl opportunistic behavior [relt 1] c. experience [prof 3] b. reliability [relt 2] 9. finiancial position [finc] c. truth & trust [relt 3] a. upgrading of equipment & services [finc 1] d. integration & co-ordination [relt 4] b. improved process efficiency [finc 2] 3. services [serv] 10. locations [loct] a. breadth of services [serv 1] a. geographical specialization & coverage [loct 1] b. pre-sale / post-sale services [serv 2] b. international scope [loct 2] c. value added services [serv 3] 11. regulatory & policy [regl] 4. quality [qult] a. govt. legislation & support [regl 1] a. commitment to continuous improvement [qult 1] b. tax benefits & related policy [regl 2] b. environmental issues [qult 2] 12. competitiveness [comp] c. risk management [qult 3] a. competition among firm [comp 1] 5. information systems + technology [info] b. benchmarking standards [comp 2] a. information accessibility [info 1] c. adopting sustainability [comp 3] b. information technology [info 2] 13. supply chain factors [scft] c. storage technology [info 3] a. role of stakeholder & supports [scft 1] .d. transportation technology [info 4] b. top management support [scft 2] 6. flexibility & reputation [flex] c. skilled human forces [scft 3] a. customer satisfaction [flex 1] d. social responsibility [scft 4] b. ability to meet future requirement [flex 2] e. customer environmental awareness [scft 5] 7. delivery [delr] 3.2 overview of ahp the ahp is a measurement tool used to determine the relative weights of the respective criteria. this tool mainly works on the prioritization of the key factors. with the help of this tool, we can particularly check inconsistency, the correlation of each criterion with our structure. the root of the ahp method is a non-linear framework multi criteria decision-making tool. from 1971-1975, t.l. saaty developed the ahp at wharton school (pennsylvania university, philadelphia, pennsylvania). for the ahp calculation, we need to conduct a pairwise comparison of our survey results with a one to nine rating. 3.3 key points of ahp methodology 1) identify key factors and structure prioritization hierarchy model. 2) preparation of questionnaire and data collection for further calculations. 3) from the survey data, visualize standardized weights for each key factor and their related sub-factors. 4) then, check the consistency of each key factor and their sub factors by calculating the consistency ratio (cr) and eventually revising comparative matrices by asking experts, if their consistency is too low, to review and revise their judgments. 5) if the cr is less than 0.1, consider that the judgments are consistent. the acceptable cr range varies depending on the size of the matrix. these are the guidelines for the cr.  where the cr value is equal to or less than the recommended value for a specific matrix size, it implies that the matrix evaluation is acceptable or ijahp article: sawant, sarode/identification of criteria for third party logistics suppliers (3pl) in supply chain management (scm) in indian manufacturing industries international journal of the analytic hierarchy process 285 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.845 indicates a good level of consistency in the comparative judgments represented in that matrix.  if the cr is more than an acceptable value, the judgments in that matrix are inconsistent and the evaluation process should be reviewed, reconsidered, and improved. 3.4 survey instrument construction based on the literature review, a list of drivers used in sustainable supply chain management (sscm) was developed. in the pretesting phase of the questionnaire, industry representatives were consulted about their views on the drivers and sub-drivers which were selected. the sub-drivers in the questionnaire were pairwise compared on a five-point likert scale. we performed two surveys; the first survey included an overview of all sub-drivers and the second survey consisted of pairwise comparing the main drivers. both questionnaires were divided into two sections; the first section collected organizational information, and the second section was the body of the survey arranged in tabular format with multiple choice grid variables ranging from not important to very important, and used the likert scale for ease of understanding on the part of the respondent. in the second survey for pairwise comparing the main drivers, saaty’s ninepoint scale was used. 3.5 collection of data collection of data included meeting our focused manufacturing industries in india and sending them questionnaires. academicians with relevant subject expertise and people from industry were asked to examine the questionnaire and brainstorm a list of criteria. the pilot study accepted the first fifteen emailed responses and subsequent follow-ups were done. the suggestions were analyzed and incorporated into questionnaires before executing it thoroughly. further data was collected using 103 respondents from topand mid-level management executives in industrial engineering, operations, and sustainable supply chain management (sscm). the questionnaires were designed to facilitate the ahp pairwise comparison data. the questionnaires were sent to relevant experts in 103 companies selected using the indian industry directory and internet searches. of the 103 questionnaires, 53 questionnaires were received at the end of four months, representing a response rate of 51.46%. due to time constraints, we began analysis on the 53 responses since the response rate was greater than 20% which is considered acceptable for data analysis (malhotra & grover, 1998). 3.6 response from survey we used the likert scale in the questionnaires throughout all sectors including pharmaceuticals, automobile, it, fmcg, manufacturing, etc. we received a 60% response from the manufacturing sector, 27% from the logistics sector, and 13% from other sectors as shown in figure 2. ijahp article: sawant, sarode/identification of criteria for third party logistics suppliers (3pl) in supply chain management (scm) in indian manufacturing industries international journal of the analytic hierarchy process 286 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.845 figure 2 aggregation of survey 4. results and discussion the data was collected with the help of questionnaires and processed using the ahp tool as mentioned above. the results are discussed in this section. 4.1 relative weightage for sscm drivers since we had already performed a pilot study, we decided on a sample size of 103 questionnaires that were sent via email. since the rate of return of questionnaires was low, a strong request was made personally to the individuals that had received them. the 53 responses were used to construct a (13 x 13) matrix for the pairwise comparisons, depicted in table 3. table 3 pairwise comparison matrix for drivers manufact uring 60% logistics services 27% others 13% respondents' sectors 1 2 3 4 5 6 7 8 9 10 11 12 13 1 1 5.019 4.566 4.453 3.849 3.585 4.283 4.226 4.377 3.906 4.075 4.660 4.642 2 0.199 1 3.453 2.981 3.170 2.925 3.170 3.189 3.321 3.264 3.302 3.057 3.377 3 0.219 0.290 1 3.528 3.200 3.075 3.170 3.075 3.264 3.434 3.434 3.528 3.509 4 0.225 0.335 0.283 1 4.208 3.377 4.094 4.057 3.849 3.811 4.132 4.321 4.358 5 0.260 0.315 0.313 0.238 1 5.094 5.642 5.623 5.774 5.585 5.623 5.717 5.585 6 0.279 0.342 0.325 0.296 0.196 1 7.038 6.792 6.547 6.604 6.434 6.434 6.642 7 0.233 0.315 0.315 0.244 0.177 0.142 1 4.981 4.868 4.585 4.528 4.453 4.830 8 0.237 0.314 0.325 0.246 0.178 0.147 0.201 1 3.509 3.340 3.321 3.226 3.245 9 0.228 0.301 0.306 0.260 0.173 0.153 0.205 0.285 1 3.151 3.132 3.208 3.340 10 0.256 0.306 0.291 0.262 0.179 0.151 0.218 0.299 0.317 1 3.245 3.075 3.302 11 0.245 0.303 0.291 0.242 0.178 0.155 0.221 0.301 0.319 0.308 1 3.717 3.283 12 0.215 0.327 0.283 0.231 0.175 0.155 0.225 0.310 0.312 0.325 0.269 1 3.321 13 0.215 0.296 0.285 0.229 0.179 0.151 0.207 0.308 0.299 0.303 0.305 0.301 1 sum 3.812 9.464 12.038 14.212 16.862 20.111 29.674 34.447 37.757 39.616 42.800 46.697 50.434 ijahp article: sawant, sarode/identification of criteria for third party logistics suppliers (3pl) in supply chain management (scm) in indian manufacturing industries international journal of the analytic hierarchy process 287 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.845 table 4 normalized matrix for pair wise comparison of drivers table 5 calculation of λmax the result of the calculations was the prioritization of the key factors in the sustainable supply chain as shown in table 6. the most important drivers are discussed next. 4.2 discussion cost logistics and supply chain costs play an important role in every organization. purchase price, freight insurance, warehousing, custom duties, and other costs account for 550% of a product's cost. in this category, five enablers are listed as follows: cost reduction [cost1] ranked first among all drivers at 0.1058. operation costs [cost2] has a global weightage of 0.0496, warehousing costs [cost3] has a weightage of 0.0291. cost saving [cost4] also plays an important role at 0.0135. the overall company initiative toward economic benefits, [cost5], is 0.0070. relationships relationships also play an important role in supply chain management. there are strategic, tactical, transactional, internal, and possibly more types of relationships among members of the supply chain community. logistic member's opportunistic behavior [relt1] ranked fourth in overall drivers at 0.0733; reliability [relt2] plays an important role with a weightage of 0.0293. relationships between suppliers and customers [relt3], which some like to call 'partnerships', play an especially important 1 2 3 4 5 6 7 8 9 10 11 12 13 sum criteria weight (w) 1 0.262 0.530 0.379 0.313 0.228 0.178 0.144 0.123 0.116 0.099 0.095 0.100 0.092 2.660 0.205 2 0.052 0.106 0.287 0.210 0.188 0.145 0.107 0.093 0.088 0.082 0.077 0.065 0.067 1.567 0.121 3 0.057 0.031 0.083 0.248 0.190 0.153 0.107 0.089 0.086 0.087 0.080 0.076 0.070 1.357 0.104 4 0.059 0.035 0.024 0.070 0.250 0.168 0.138 0.118 0.102 0.096 0.097 0.093 0.086 1.335 0.103 5 0.068 0.033 0.026 0.017 0.059 0.253 0.190 0.163 0.153 0.141 0.131 0.122 0.111 1.469 0.113 6 0.073 0.036 0.027 0.021 0.012 0.050 0.237 0.197 0.173 0.167 0.150 0.138 0.132 1.413 0.109 7 0.061 0.033 0.026 0.017 0.011 0.007 0.034 0.145 0.129 0.116 0.106 0.095 0.096 0.875 0.067 8 0.062 0.033 0.027 0.017 0.011 0.007 0.007 0.029 0.093 0.084 0.078 0.069 0.064 0.582 0.045 9 0.060 0.032 0.025 0.018 0.010 0.008 0.007 0.008 0.026 0.080 0.073 0.069 0.066 0.483 0.037 10 0.067 0.032 0.024 0.018 0.011 0.008 0.007 0.009 0.008 0.025 0.076 0.066 0.065 0.417 0.032 11 0.064 0.032 0.024 0.017 0.011 0.008 0.007 0.009 0.008 0.008 0.023 0.080 0.065 0.356 0.027 12 0.056 0.035 0.024 0.016 0.010 0.008 0.008 0.009 0.008 0.008 0.006 0.021 0.066 0.275 0.021 13 0.057 0.031 0.024 0.016 0.011 0.007 0.007 0.009 0.008 0.008 0.007 0.006 0.020 0.211 0.016 sum 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 3 4 5 6 7 8 9 10 11 12 13 consistency vector 1 1.000 5.019 4.566 4.453 3.849 3.585 4.283 4.226 4.377 3.906 4.075 4.660 4.642 17.836 2 0.199 1.000 3.453 2.981 3.170 2.925 3.170 3.189 3.321 3.264 3.302 3.057 3.377 10.966 3 0.219 0.290 1.000 3.528 3.200 3.075 3.170 3.075 3.264 3.434 3.434 3.528 3.509 11.209 4 0.225 0.335 0.283 1.000 4.208 3.377 4.094 4.057 3.849 3.811 4.132 4.321 4.358 15.904 5 0.260 0.315 0.313 0.238 1.000 5.094 5.642 5.623 5.774 5.585 5.623 5.717 5.585 32.467 6 0.279 0.342 0.325 0.296 0.196 1.000 7.038 6.792 6.547 6.604 6.434 6.434 6.642 42.123 7 0.233 0.315 0.315 0.244 0.177 0.142 1.000 4.981 4.868 4.585 4.528 4.453 4.830 22.042 8 0.237 0.314 0.325 0.246 0.178 0.147 0.201 1.000 3.509 3.340 3.321 3.226 3.245 11.653 9 0.228 0.301 0.306 0.260 0.173 0.153 0.205 0.285 1.000 3.151 3.132 3.208 3.340 3.132 10 0.256 0.306 0.291 0.262 0.179 0.151 0.218 0.299 0.317 1.000 3.245 3.075 3.302 1.030 11 0.245 0.303 0.291 0.242 0.178 0.155 0.221 0.301 0.319 0.308 1.000 3.717 3.283 0.319 12 0.215 0.327 0.283 0.231 0.175 0.155 0.225 0.310 0.312 0.325 0.269 1.000 3.321 0.084 13 0.215 0.296 0.285 0.229 0.179 0.151 0.207 0.308 0.299 0.303 0.305 0.301 1.000 0.091 ijahp article: sawant, sarode/identification of criteria for third party logistics suppliers (3pl) in supply chain management (scm) in indian manufacturing industries international journal of the analytic hierarchy process 288 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.845 role with a weightage of 0.0122. to make supply chains work properly, integration and coordination [relt4] are necessary with a weightage of 0.0063. services in today's competitive world, the customer is the focus of every industry. while focusing on service (it is impossible to be perfect), each company aims to provide the best service to their customers. therefore, breadth of service [serv1] is important at 0.0728, ranking fifth in overall drivers. the customer wants a smooth and easy experience, so companies’ pre-sale/post-sales services [serv2] are important at 0.0231. it is up to a company how good the service can be that they deliver. if a company is starting to build long-term relationships with customers to gain their loyalty, it should consider shifting from product-orientated strategy to a customer-focused strategy, so value-added services [serv3] are important at (0.0081). quality supply chain management plays a key role in product quality and overall profitability. for this reason, quality control in the supply chain is critical for maintaining a competitive edge in the marketplace while reducing operation costs. without quality control, waste becomes more than a tolerable amount, so risk management [qult3] at 0.0075 is necessary. with globalization, the world has literally become a global village, to keep up in a continuously competitive environment, a company must be committed to continuous improvement. [qult1] is required and ranks third among drivers with a global weightage of 0.0781. logistic activities are regarded as a significant source of environmental pollution and greenhouse emission, harmful to both health and ecosystem qualities, so environmental issues [qult2] are important at 0.0174. information systems technology a supply chain's information system plays an important role in achieving its efficiency and effectiveness. it systems can ensure that their logistics functional operations provide customer satisfaction for the lowest total cost. therefore, information accessibility [info1] is ranked sixth at 0.0634 while information technology [info2] is ranked 11th in overall drivers. having the latest technology to manage storage [info3] and transportation [info4] play an important role with a global weightage of 0.0128 and 0.0060, respectively. flexibility and reputation customer satisfaction is a measure that determines how happy customers are with a company's product, services, and capabilities in general. with the help of some reports and surveys about customer satisfaction, a company can improve or change its products and services, which is why customer satisfaction [flex1] is ranked second with a weightage of 0.0792. in this competition, each company must maintain its ability to meet future requirements, so [flex2] is ranked 12th with a weightage 0.0298. delivery delivery requirements and the customer's expectation are two items that need to be a priority for every organization. products should be delivered and arrive on time; therefore, on-time delivery [delr1] is important, ranking eighth among all factors, with 0.0398. lead-time [delr2] is ranked 20th with a weightage of 0.0145. on-time delivery (otd) performance affects more than customers do. failing otd is usually an indicator ijahp article: sawant, sarode/identification of criteria for third party logistics suppliers (3pl) in supply chain management (scm) in indian manufacturing industries international journal of the analytic hierarchy process 289 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.845 of poor production efficiency and materials handling procedures; therefore, accuracy of delivery time [delr 3] is necessary at 0.0088, and reduction in time of raw material consumption [delr4] is important at 0.0038. professionalism punctuality means completing a task within the expected time. being punctual indicates a respect for the value of time. it is an etiquette which motivates a person to perform a task in a timely manner; therefore, punctuality [prof1] is ranked tenth at 0.0317. skills are the expertise or talent needed to do a job or task well. expertise is what makes you confident and dependable at your job, so it has a special place in the key factors with a weightage of 0.0102 [prof2]. business experience is important for gaining the upper hand. there are not many shortcuts that are useful to gain competitive advantages against rivals. knowledge and insights learned from experience are what make a company better than others; therefore, experience [prof3] is required and has a weightage of 0.0031. financial position before choosing any third party, it is important to have a statement of their financial position to determine the state of their liquidity risk, financial risk, credit risk and business risk. therefore, the selected vendor should always be financially able to upgrade their equipment and services. therefore, this is ranked ninth with a weightage 0.0325. in this field, process efficiency can be defined as the amount of effort required to produce a business outcome. therefore, it is very important that a company should always have the ability to improve process efficiency [finc] at 0.0045. location if a business owner can develop the best product, construct the most stunning business space, hire the best employees, and have a rock-solid business plan, but chooses the wrong location then all his efforts will have been in vain. choosing the right location is crucial because it influences many factors such as customer and vendor access, employee safety, protection from natural disasters and building functionality. therefore, geographic specialization and coverage [loct1] ranked 15th in all drivers with a weightage of 0.0272. sourcing products from overseas can often equate to low-cost country sourcing, where products or materials originate because their exceptionally low labor and production costs results in modest purchasing prices. therefore, international scope [loct2] is important with a weightage of 0.0045. regulatory policies the indian government has amended many rules and regulations to protect the environment for future generations. in today's competitive world, each organization looks toward success in any way it can. a government’s role is thus vital to protect the environment and the heath and other benefits for human beings. therefore, government regulations and support [regl1] ranked 16th overall with 0.0236. nevertheless, at the same time, government tax policy directly influences interest rates. if interest rates increase because of government spending, then it leads to decreased consumer spending. if interest rates decrease, it leads to an increase in investment and businesses can increase their productivity. therefore, these policies [regl2] received a weightage of 0.0034. ijahp article: sawant, sarode/identification of criteria for third party logistics suppliers (3pl) in supply chain management (scm) in indian manufacturing industries international journal of the analytic hierarchy process 290 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.845 competitiveness in a market for the same product, there are often many manufacturers, so there is high competition in every field. to compete, each organization moves toward price cutting, increases its advertising, improves its product quality along with many other improvements. therefore, competition among firms [comp1] is ranked 19th with 0.0146. benchmarking standards are a set of standards used as a point of reference for evaluating performance or level of quality of products or services. benchmarks can be developed from an organization's own experience, from the experience of other organizations in the industry, or from legal requirements such as environmental regulations; therefore, benchmarking [comp2] reaches a global weight of 0.0049. sustainability [comp3] receives a global weightage of 0.0015. supply chain factors a small business's success and creditability are related to the actions of numerous stakeholders, as they are the ones that affect the business; therefore, the role of the stakeholder [scft1] gains a weight of 0.0079. similarly, top management support is critical to the success of a manufacturing strategy. top management support is how companies get the necessary resources (facilities, capital, it, & human resources) to achieve benefits from their different strategies. top management receives a global weightage of 0.0041. skilled human resources mean highly trained, educated, or experienced members of the workforce that can complete complex tasks on the job. thus, skilled workers have a weightage 0.0022. social responsibility involves sensitivity towards social, cultural, economic, and environmental issues, so it is weighted at 0.0012. in today's world, the impact of an organization on the environment is always under observation. each organization needs to be aware of the impact of its actions on the environment. therefore, it has 0.0006 global weights. ijahp article: sawant, sarode/identification of criteria for third party logistics suppliers (3pl) in supply chain management (scm) in indian manufacturing industries international journal of the analytic hierarchy process 291 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.845 table 6 drivers ranked for 3pl in scm in indian manufacturing industries sr. no. criteria global weights sr. no. local weights rank global weights rank 1 cost reduction [cost 1] 0.516 12 0.1058 1 2 operation cost [cost 2] 0.242 17 0.0496 7 3 warehousing cost [cost 3] 0.142 25 0.0291 14 4 cost saving [cost 4] 0.066 37 0.0135 21 5 economic benefits [cost 5] 0.034 42 0.0070 29 1 3pl opportunistic behaviour [relt 1] 0.606 9 0.0733 4 reliability [relt 2] 0.242 17 0.0293 13 3 truth & trust [relt 3] 0.101 31 0.0122 23 4 integration & co-ordination [relt 4] 0.052 40 0.0063 30 1 breadth of services [serv 1] 0.700 7 0.0728 5 2 pre-sale / post-sale services [serv 2] 0.222 21 0.0231 17 3 value added services [serv 3] 0.078 32 0.0081 26 1 commitment to continuous improvement [qult 1] 0.758 4 0.0781 3 2 environmental issues [qult 2] 0.169 23 0.0174 18 3 risk management [qult 3] 0.073 33 0.0075 28 1 information accessability [info 1] 0.561 11 0.0634 6 2 information technology [info 2] 0.273 14 0.0308 11 3 storage technology [info 3] 0.113 30 0.0128 22 4 transportation technology [info 4] 0.053 39 0.0060 31 1 customer satisfaction [flex 1] 0.727 5 0.0792 2 2 ability to meet future requirement [flex 2] 0.273 14 0.0298 12 1 on time delivery/shipment [delr 1] 0.594 10 0.0398 8 2 lead time [delr 2] 0.217 22 0.0145 20 3 accuaracy of delivery time [delr 3] 0.132 27 0.0088 25 4 reduction in time & raw material consumption [delr 4] 0.057 38 0.0038 36 1 punctuality [prof 1] 0.705 6 0.0317 10 2 expertise [prof 2] 0.226 20 0.0102 24 3 experience [prof 3] 0.069 36 0.0031 38 1 upgrading of equipments & services [finc 1] 0.879 1 0.0325 9 2 improved process efficiency [finc 2] 0.121 29 0.0045 34 1 geographyical specialization & coverage [loct 1] 0.851 3 0.0272 15 2 international scope [loct 2] 0.149 24 0.0048 33 1 govt. legislation & support [regl 1] 0.875 2 0.0236 16 2 tax benefits & related policy [regl 2] 0.125 28 0.0034 37 1 competition among firm [comp 1] 0.694 8 0.0146 19 2 benchmarking standards [comp 2] 0.234 19 0.0049 32 3 adopting sustainability [comp 3] 0.072 34 0.0015 40 1 role of stakeholder & supports [scft 1] 0.496 13 0.0079 27 2 top management support [scft 2] 0.257 16 0.0041 35 3 skilled human forces [scft 3] 0.137 26 0.0022 39 4 social responsibility [scft 4] 0.072 34 0.0012 41 5 customer environmental awareness [scft 5] 0.038 41 0.0006 42 9 0.032 fleibility & reputation[flex] 6 0.109 4 quality [qult] 5 information systems + technology [info] 0.103 0.113 0.067 0.045 0.037 7 2 relationships [relt] 3 services [serv] 1 cost [cost] sub criteria 0.205 0.121 0.104 10 8 professionalism [prof] locations [loct] 11 regulatory & policy [regl] 12 competitiveness [comp] 0.027 0.021 0.01613 supply chain factor [scft] finiancial position [finc] delivery [delr] ijahp article: sawant, sarode/identification of criteria for third party logistics suppliers (3pl) in supply chain management (scm) in indian manufacturing industries international journal of the analytic hierarchy process 292 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.845 5. case study 5.1 introduction to problem anson’s electro mechanical works is a well-known submersible pump manufacturing company having a pan-india presence. there are more than 350 anson employees currently working in india. the plant provides direct and indirect employment to more than 600 people. anson started their first manufacturing plant in 1967 in jogeshwari, a suburb of mumbai, then in 1989 established another manufacturing unit in palghar, thane district. anson has steadily grown over the years and their brands, anson’s and anco, have become the hallmark of quality, reliability, and endurance. anson’s manufactures horizontal and vertical open-well submersible pumps and monobloc pumps. they also manufacture atex certified flameproof motors that are used in petrol and diesel dispensing units. anco motors manufactures 6” and 4” submersible pumps, monobloc ganga pumps, and self-priming saraswati and triveni models. the main advantages of localization are:  reduction in cost of components, resulting in a lower final price of the pumps. localization is undertaken in the situation where the cost benefit comes to more than 40% compared to imported goods.  better service is expected, as local companies are easily accessible.  reduction in delivery time. the time required to import any material is generally greater than 2 weeks, which also results in an increase in company overhead costs.  better coordination between vendors and anson’s. the purpose of this case study is to select 3pl for anson’s electro mechanical works in palghar for the purchase of cast iron components for manufacturing "6” submersible motor pumps for bore wells xrf b2 series 4-12". the three main vendors of cast iron components are:  v1. consolidated eutectics, 33, m.i.d.c., shiroli, kolhapur416 122.  v2. nsvp industrial casting pvt. ltd., plot no: -b-200, functional estate, udyog nagar, udhana, surat-394 210.  v3. john t. hardaker pvt. ltd., 51, a. b. government industrial estate, charkop, kandivali (w), mumbai – 400 067. this study focuses on cast iron components from three vendors v1, v2 and v3, which have been preselected from a list of approved vendors. the goal is to find the best vendor among the three and then, to place an order for a large quantity among them based on their ranking. 5.2 ahp calculations as mentioned earlier, we must follow the steps for ahp calculations. first, we collected data using questionnaires on the three vendors from personnel at anson’s pvt ltd. twenty questionnaires were mailed to superiors at anson pvt ltd and seven questionnaires were received at the end of four months, representing a response rate of 35%. due to time ijahp article: sawant, sarode/identification of criteria for third party logistics suppliers (3pl) in supply chain management (scm) in indian manufacturing industries international journal of the analytic hierarchy process 293 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.845 constraints, we began analysis of the seven responses. the response rate of 35% is acceptable for analysis; a response rate of more than 20% is considered acceptable for data analysis (malhotra & grover, 1998). since we have already performed the calculations for our key factors and their subfactors, we followed the same ahp steps again. first, we collected the data in the form of the pairwise comparisons, and then normalized the matrix of the data. next, we collected the pairwise comparison data for the sub-factors. to confirm the accuracy and acceptability of our work, we checked each sub-factor's consistency ratio to make sure that (cr) calculated ˂ (cr) recommended. we concluded that if the level of consistency is good, that is, cr ≤ 10%, then it is acceptable. table 7 consistency ratio for sub factors with w.r.t vendors level elements of the matrix with respect to consistency ratio (c.r.) level 3 vendor 1 vendor 2 vendor 3 1 cost reduction 0.0963 2 operation cost 0.0907 3 warehousing cost 0.0888 4 cost saving 0.0821 5 economic benefits 0.0772 6 3pl opportunistic behavior 0.0664 7 reliability 0.0866 8 truth & trust 0.0746 9 integration & co-ordination 0.0775 10 breadth of services 0.0673 11 pre-sale / post-sale services 0.0189 12 value added services 0.0164 13 continuous improvement 0.0447 14 environmental issues 0.0152 15 risk management 0.0838 16 information accessibility 0.0770 17 information technology 0.0943 18 storage technology 0.0683 19 transportation technology 0.0656 20 customer satisfaction 0.0914 21 ability to meet future requirement 0.0883 22 on time delivery/shipment 0.0398 23 lead time 0.0731 24 accuracy of delivery time 0.0723 25 reduction in time of raw material consumption 0.0758 26 punctuality 0.0347 27 expertise 0.0953 28 experience 0.0621 29 upgrading of equipment & services 0.0961 30 improved process efficiency 0.0754 31 geographical specialization & coverage 0.0650 32 international scope 0.0574 33 govt. legislation & support 0.0938 34 tax benefits & related policy 0.0966 35 competition among firms 0.0873 36 benchmarking standards 0.0535 37 adopting sustainability 0.0304 38 role of stakeholder & supports 0.0970 39 top management support 0.0970 40 skilled human forces 0.0740 41 social responsibility 0.0945 42 customer environmental awareness 0.0953 ijahp article: sawant, sarode/identification of criteria for third party logistics suppliers (3pl) in supply chain management (scm) in indian manufacturing industries international journal of the analytic hierarchy process 294 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.845 from table 7, we confirmed that the pairwise comparison matrices in level three used for rating the vendors are consistently good, i.e., cr < 0.1, and the ahp synthesizing process can be used to calculate the priorities of the 3pl vendors to determine an overall outcome. 5.3 calculation of overall weights in the model above, the overall weights of each vendor were calculated by first multiplying each sub-factor by the weight of its respective major factor and the vendor rating associated with it. the values obtained from each of the 42 sub-factors were added to give the final weight for each vendor. for example, for any vendor, say v1, the weights are calculated as follows for the weight of v1: v1 = ∑ (𝑠 ∗ 𝑚 ∗ 𝑟)42𝑆=1 where: s = sub-criteria, of which there are 42 in total in this study. m = number of main criteria. r = rating of vendor v1 associated with that sub-criteria the above formula was applied in microsoft excel 2007 to obtain the results given in table 8. table 8 final weights and rank of the three vendors the results show that the ahp model can handle many tangible and intangible criteria as it has 42 sub-criteria and 13 main criteria that are easily managed. vendor 1 (v1) is the top-ranked choice using the ahp method, and hence v1 is the best choice of vendor in the case study. 5.4 order allocation under capacity constraints in the case study we conducted, the order quantity was relatively large with a 600-unit monthly requirement. the quantity is too large to be satisfied by any single supplier. it is, however, subject to fluctuations, depending on the market demand. keeping the fluctuating demand situation in mind, it was decided to allocate orders to different vendors. the proportion of the order quantity was influenced by the vendor weights. this decision to distribute orders among vendors is advantageous as it also promotes healthy competition and reduces the risk of production stoppage if any one vendor is unable to fulfill the order due to unforeseen circumstances. sr. no. vendor final weights ranking 1 v1 0.5618 1 2 v2 0.2798 2 3 v3 0.1585 3 ijahp article: sawant, sarode/identification of criteria for third party logistics suppliers (3pl) in supply chain management (scm) in indian manufacturing industries international journal of the analytic hierarchy process 295 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.845 an ahp and linear programming approach was used in this study for order allocation. ghodsypour and o'brien (1998) presented the linear programming approach, where the objective function maximizes the total value of purchase (tvp). the data required for the solution are given below: order quantity = 600 units maximum permissible defect rate = 1200 ppm (parts per million) table 9 capacity allotted to each vendor vendor capacity (units) defect rate(ppm) v1 300 40 v2 250 50 v3 250 50 the above data are input into the general equations for a linear programming problem. the order quantities are: x = order quantity for vendor v1 y = order quantity for vendor v2 z = order quantity for vendor v3 the objective function becomes maximize tvp = 0.5586x +0.2830y + 0.1586z subject to the quality constraints, 0.004x + 0.005y + 0.005z ≤ 1200. the capacity constraints are: x + y + z = 700 x ≤ 300 y ≤ 250 z ≤ 250 the above linear programming problem is solved by using the “wolfram alpha widget linear programming calculator”. the inputs and outputs are given below in figures 3 and 4. figure 3 input for linear programming problem ijahp article: sawant, sarode/identification of criteria for third party logistics suppliers (3pl) in supply chain management (scm) in indian manufacturing industries international journal of the analytic hierarchy process 296 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.845 figure 4 output for linear programming problem the results are summarized in table 10. table 10 quantity ordered from each vendor vendor order quantity v1 300 v2 250 v3 150 from the results, it is observed that the vendor with the highest weight, i.e., vendor v1, is given more preference and receives a larger order quantity. the order quantity allocated to the vendor with the highest weight is limited only by the production capacity, although the final decision usually lies with the purchaser. in this case study, the defect rate had no serious impact on the order quantity, as the values were similar for all the vendors. 5.5 sensitivity analysis a sensitivity analysis identified the impact of changes in the priority of the criteria on the suppliers' performance and the order quantities. in this study, a sensitivity analysis was conducted by varying one parameter, cost, to study its impact on order quantity, impact of varying cost parameter on suppliers' performance and order quantity. cost is one of the parameters considered for vendor selection. out of the 13 major criteria considered, cost is one parameter, which fluctuates the most due to the competitive business environment, and hence it was investigated using a sensitivity analysis. variations in the cost priority affect the suppliers' ratings, which in turn are the parameters of the objective function in linear programming, thus variations in cost have an impact on the order quantity. ghodsypour and o'brien (1998) state that a sensitivity analysis cannot be performed on general packages of lp such as lindo or solver (a function of microsoft excel) as they consider the variation of one coefficient while others are constant, while the change in a parameter will change all the coefficients of the objective function. this study proposes the use of microsoft excel 2007, by using various formulae to bypass this ijahp article: sawant, sarode/identification of criteria for third party logistics suppliers (3pl) in supply chain management (scm) in indian manufacturing industries international journal of the analytic hierarchy process 297 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.845 problem. microsoft excel is used to determine the change in the remaining criteria with the changes in the cost parameter. the new criteria weights are input into an excel worksheet shown in table 11 applying the ahp (saaty) to find new vendor ratings. these ratings are then fed into an excel worksheet to find the optimum order quantities. the approach however can only be applied to those models whose weights are utilized in a crisp form during calculation. figure 6 shows the approach adopted for the sensitivity analysis. figure 5 approach adopted for sensitivity analysis table 11 criteria weights by varying cost criteria weights [cost] [relt] [serv] [qult] [info] [flex] [delr] [prof] [finc] [loct] [regl] [comp] [scft] 0 0.1522 0.1308 0.1296 0.1421 0.1371 0.0843 0.0566 0.0465 0.0403 0.0340 0.0264 0.0201 0.1 0.1370 0.1177 0.1166 0.1279 0.1234 0.0758 0.0509 0.0419 0.0362 0.0306 0.0238 0.0181 0.2 0.1218 0.1047 0.1036 0.1137 0.1097 0.0674 0.0453 0.0372 0.0322 0.0272 0.0211 0.0161 0.3 0.1065 0.0916 0.0907 0.0995 0.0960 0.0590 0.0396 0.0326 0.0282 0.0238 0.0185 0.0141 0.4 0.0913 0.0785 0.0777 0.0853 0.0823 0.0506 0.0340 0.0279 0.0242 0.0204 0.0158 0.0121 0.5 0.0761 0.0654 0.0648 0.0711 0.0686 0.0421 0.0283 0.0233 0.0201 0.0170 0.0132 0.0101 0.6 0.0609 0.0523 0.0518 0.0569 0.0548 0.0337 0.0226 0.0186 0.0161 0.0136 0.0106 0.0081 0.7 0.0457 0.0392 0.0389 0.0426 0.0411 0.0253 0.0170 0.0140 0.0121 0.0102 0.0079 0.0060 0.8 0.0304 0.0262 0.0259 0.0284 0.0274 0.0169 0.0113 0.0093 0.0081 0.0068 0.0053 0.0040 0.9 0.0152 0.0131 0.0130 0.0142 0.0137 0.0084 0.0057 0.0047 0.0040 0.0034 0.0026 0.0020 1 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 when the cost is zero, the other criteria are normalized to determine their new weights. for other values of cost, the following formula is applied: n = o − ( (nc − oc) ∗ o 1 − oc ) vary cost weight from 0 to 1 use ms excel to measure changes in remaining criteria use ms excel to find new vendor ratings use " wolfram:alpha widget :linear programming calculator, to find new order quantity ijahp article: sawant, sarode/identification of criteria for third party logistics suppliers (3pl) in supply chain management (scm) in indian manufacturing industries international journal of the analytic hierarchy process 298 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.845 n = new criteria weight o = original criteria weight nc = new cost weight oc = original cost weight after obtaining, the new criteria weights, the new vendor ratings obtained through excel for each of the new scenarios are shown in table 12. for the first step, table 12 shows the microsoft excel results for the variation in the other parameters with the variation in cost weight from zero to one. the weights of the three vendors do not vary much as shown in table 12. individual criteria do not have much influence because of the large number of criteria (42 total parameters). the order quantity in each case remained the same when solved with excel. the only option available for vendors to obtain a greater share of the orders is to improve their performance for many criteria to improve their overall ratings. the model included in the study thus promotes an effort towards improving the overall efficiency of the vendors. table 12 variations in vendor weights with changes in cost weight vendor final weight ranking vendor final weight ranking cost = 0 cost = 0.6 v1 0.5551 1 v1 0.5748 1 v2 0.2854 2 v2 0.2691 2 v3 0.1597 3 v3 0.1563 3 cost = 0.1 cost = 0.7 v1 0.5583 1 v1 0.5781 1 v2 0.2827 2 v2 0.2664 2 v3 0.1591 3 v3 0.1557 3 cost = 0.2 cost = 0.8 v1 0.5617 1 v1 0.5814 1 v2 0.2800 2 v2 0.2637 2 v3 0.1586 3 v3 0.1551 3 cost = 0.3 cost = 0.9 v1 0.5650 1 v1 0.5847 1 v2 0.2773 2 v2 0.2610 2 v3 0.1580 3 v3 0.1545 3 cost = 0.4 cost = 1 v1 0.5683 1 v1 0.5879 1 v2 0.2746 2 v2 0.2583 2 v3 0.1574 3 v3 0.154 3 cost = 0.5 v1 0.5716 1 v2 0.2719 2 v3 0.1568 3 ijahp article: sawant, sarode/identification of criteria for third party logistics suppliers (3pl) in supply chain management (scm) in indian manufacturing industries international journal of the analytic hierarchy process 299 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.845 figure 6 variations in vendor weights with changes in cost weight (graphical representation) 6. conclusion for the research objective of identifying criteria for 3pl in scm, we conducted a literature review. based on the literature review, 42 sub factors were identified and distributed among 13 key factors including cost, relationships, services, quality, information systems and technology, flexibility and reputation, delivery, professionalism, financial position, location, regulatory and policy, competitiveness and supply chain factors. with the help of this research, it was concluded that not all key factors have the same weightages in prioritizing 3pl vendors for scm and not all key factors have the same intensity. from the perspective of indian manufacturing industries, this study focused on finding the key factors for 3pl in scm. this research ranked the key factors using an ahp-based approach. this research has provided a best industry solution for identifying key success factors and provided a benchmark for the implementation of best choice of 3pl in scm. with the help of our case study based on an indian pump manufacturing industry, we used a sensitivity analysis to identify the impact of varying one parameter on order quantity. the sensitivity analysis indicated that due to the large number of criteria, the model is insensitive to changes in the priority of a single criterion. the order allocation in the study does not change with any change in the cost parameter. 0.5551 0.5583 0.5617 0.565 0.5683 0.5716 0.5748 0.5781 0.5814 0.5847 0.5879 0.2854 0.2827 0.28 0.2773 0.2746 0.2719 0.2691 0.2664 0.2637 0.261 0.2583 0.1597 0.1591 0.1586 0.158 0.1574 0.1568 0.1563 0.1557 0.1551 0.1545 0.154 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 c=0 c=0.1 c=0.2 c=0.3 c=0.4 c=0.5 c=0.6 c=0.7 c=0.8 c=0.9 c=1 v e n d o r w e ig h ts variation in cost weights sensitivity analysis v1 v2 v3 ijahp article: sawant, sarode/identification of criteria for third party logistics suppliers (3pl) in supply chain management (scm) in indian manufacturing industries international journal of the analytic hierarchy process 300 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.845 in addition, this research found only 42 enablers, but researchers in future studies can work to find more or different enablers. to validate the enablers/drivers we found, other methods like the analytical network process (anp) and the interpretive ranking process (irp), fuzzy ahp, can be used to rank the factors. the following are the major contributions of this study: • this research recognizes the importance of multi-criteria decision making and identifies a large number of criteria, which play a role in vendor selection. • the study identifies those criteria, which play a major role in the vendor selection process in the indian manufacturing sector. the importance of each criterion is determined through an extensive survey conducted in the indian manufacturing industry. • as noted in the literature review, most papers on vendor selection consider only a few criteria for selecting a vendor. this study shows that many criteria can be successfully incorporated into a vendor selection model so that each parameter which can affect the selection process plays its part. • the study demonstrates that ahp-based techniques can easily handle a large number of both tangible and intangible criteria and get consistent results. • microsoft excel spreadsheets have been utilized at every stage of the vendor selection process. the use of excel spreadsheets saves time, reduces human error during computation, and provides a real-time vendor selection framework. for future research, we can employ various methodologies such as focus groups, in-depth interviews and brainstorming sessions with experts. second, this case study was conducted specifically for the indian manufacturing industry and may not be appropriate for other industries or other parts of the world. third, only the ahp and a sensitivity analysis were considered for this study. other significant methods like fuzzy ahp, anp, etc. can be useful for further work. ijahp article: sawant, sarode/identification of criteria for third party logistics suppliers (3pl) in supply chain management (scm) in indian manufacturing industries international journal of the analytic hierarchy process 301 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.845 references achanga, p., shehab, e., roy, r. & nelder, g. (2006). critical success factors for lean implementations within sme's. journal of manufacturing technology management, 17(4), 460-471. doi: https://doi.org/10.1108/17410380610662889 agrawal, s., singh, r.k., murtaza, q. (2015). prioritizing critical success factors for reverse logistics implementation using fuzzy-topsis methodology. journal of industrial engineering international, 12, 15-27. doi: 10.1007/s40092015-0124-8. aguezzoul, a. (2005). third-party logistics selection problem: a literature review on criteria and methods. omega, 49, 69-78. doi: http: //dx.doi.org/10.1016/j. omega.2014.05.009j. 2014. ali, s.s., kaur, r. (2018), an analysis of satisfaction level of 3pl service users with the help of acsi. benchmarking: an international journal, 25(1), .24-46. doi: https://doi.org/10.1108/bij-10-2016-0163 aktas, e. & ulengin, f. (2005). outsourcing logistics activities in turkey. journal of enterprise information management, 18(3), 316-329. doi: https://doi.org/10.1108/17410390510591996 arroyo, p., gaytan, j., de boer, l. (). a survey of third-party logistics in mexico and a comparison with reports on europe and usa, www.emeraldinsight.com/0144-3577.html. doi: https://doi.org/10.1108/01443570610666984 arvis, j.f., saslavsky, d., ojala, l. (2014). connecting to compete 2014 trade logistics in the global economy, the logistics performance index and its indicators. the international bank for reconstruction and development. doi: https://doi.org/10.1596/20399 asthana, s., bhat, h., singh, r. (2015). a study of important factors for the performance measurement of third-party logistics (3pl) organizations in the indian logistics industry. crisis report. bhatnagar, r., sohal, a.s. (2005). supply chain competitiveness measuring the impact of location factors, uncertainty and manufacturing practices. technovation, 25(2005), 443– 456. doi: https://doi.org/10.1016/s0969-7012(00)00020-4 bruno, g., esposito, e., genovese, a., passaro, r. (2012). ahp-based approaches for supplier evaluation: problems and perspectives. journal of purchasing & supply management, 18(2012) 159–172. doi: https://doi.org/10.1016/j.pursup.2012.05.001 chaudhari, j.s., sarode, a.d. (2018). identification of drivers in sustainable supply chain management: a review. ltcoe conference. cheong, m.l.f. (2003). logistics outsourcing and 3pl challenges. european journal of operational research. http://www.emeraldinsight.com/0144-3577.html https://doi.org/10.1108/01443570610666984 ijahp article: sawant, sarode/identification of criteria for third party logistics suppliers (3pl) in supply chain management (scm) in indian manufacturing industries international journal of the analytic hierarchy process 302 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.845 cirpin, b.k. & kabadayim, n. (2015). analytic hierarchy process in third-party logistics provider selection criteria evaluation: a case study in it distributor company. international journal multidisplinary sciences & engineering, 6(3). daim, t.u., udbye, a., balasubramanian, a. (2012). use of analytic hierarchy process (ahp) for selection of 3pl providers. journal of manufacturing technology management, 24(1), 28 51. doi: https://doi.org/10.1108/17410381311287472 dinter, b. (2012). success factors for information logistics strategy — an empirical investigation. decision support systems, 54(3) 1207–1218. doi: https://doi.org/10.1016/j.dss.2012.09.001 dupuis, m. & prime, n. (1996).business distance and global retailing: a model for analysis of key success/failure factors. international journal of retail & distribution management, 24(11), 30 38. doi: https://doi.org/10.1108/09590559610131709 ellram, l.m. (1991).key success factors and barriers in international purchasing partnerships. management decision, 29(7), 38-44. doi: https://doi.org/10.1108/00251749110007175 fleischmann, m., krikke, h.r., dekker, r., douwqe, s., flapper, p. (2000). a characterization of logistics networks for product recovery. omega, 28(2000), 653 666. doi: https://doi.org/10.1016/s0305-0483 (00)00022-0 gunasekaran, a., ngai, e.w.t. (2003). the successful management of a small logistics company. international journal of physical distribution & logistics management, 33(9), 825 842. doi: https://doi.org/10.1108/09600030310503352 hertz, s. & alfredsson, m. (2016). strategic development of third-party logistics providers. industrial marketing management, 32(2), 139–149. doi: https://doi.org/10.1016/s0019-8501 (02)00228-6 hu, y-c., chiu, y-j., hsu, c-s., & chan, y-y. (2015). identifying key factors for introducing gps-based fleet management systems to the logistics industry. mathematical problems in engineering, 2015, 1-14. doi: http://dx.doi.org/10.1155/2015/413203. huo, b., ye, y., zhao, x. (2015). the impacts of trust and contracts on opportunism in the 3pl industry: the moderating role of demand uncertainty. international. journal of production economics 170 (2015) 160–170. doi: https://doi.org/10.1016/j.ijpe.2015.09.018 jhawar, a., garg, s.k. (2018). modeling of critical factors for improving logistics performance of india using interpretive structural modeling. international journal of applied management sciences and engineering, 5(1), 1-23. doi: https://doi.org/10.4018/ijamse.2018010103 karakadlar, i.s. (2005). key success factors for strategic logistics & supply chain management to enhance competitiveness. http;//www.reserarchgate.net conference paper. https://dl.acm.org/toc/dssy/2013/54/3 https://dl.acm.org/toc/dssy/2013/54/3 https://dl.acm.org/toc/dssy/2013/54/3 https://doi.org/10.1016/j.dss.2012.09.001 http://dx.doi.org/10.1155/2015/413203 ijahp article: sawant, sarode/identification of criteria for third party logistics suppliers (3pl) in supply chain management (scm) in indian manufacturing industries international journal of the analytic hierarchy process 303 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.845 knemeyer, m. & murphy, p.r. (2005). is the glass half full or half empty? international journal of physical distribution & logistics management, 35(10), 708 727. doi: https://doi.org/10.1108/09600030510634571 knemeyer, m. & murphy, p.r. (2004). evaluating the performance of third-party logistics arrangements: a relationship marketing perspective. journal of supply chain management. doi: https://doi.org/10.1111/j.1745-493x.2004.tb00254.x korpela, j. & tuominen, m. (1996). benchmarking logistics performance with an application of the analytic hierarchy process. ieee transactions on engineering management, 43(3), 323-333. doi: https://doi.org/10.1109/17.511842 large, r.o. (2007). the influence of customer-specific adaptations on the performance of third-party-logistics relationships—document studies and propositions. international journal of logistics research and applications: a leading journal of supply chain management, 10(2), 123-133. doi: https://doi.org/10.1080/13675560701427684 leahy, s.e., murphy, p.r., & poist, r.f. (1995). determinants of successful logistical relationships: a third-party provider perspective. transportation journal, 35(2), 5-13. lieb, r. & miller, j. (2000). the use of third-party logistics services by large us manufacturers: the 2000 survey. international journal of logistics research and applications: a leading journal of supply chain management, 5(1), 1-12. doi: https://doi.org/10.1080/13675560110114270 lokhande, s.p., sarode, a.d. (2020). identification & prioritization of agile manufacturing enablers for small & medium scale industries. iop conference series: materials science & engineering, 810(1), 012034. doi: https://doi.org/10.1088/1757899x/810/1/012034 luthra, s., mangla, s.k., kumar, s., garg, d., haleem, a. (2017). identify and prioritise the critical factors in implementing the reverse logistics practices: a case of indian auto component manufacturer. international. journal of business and systems research, 11(1/2), 42 61. doi: https://doi.org/10.1504/ijbsr.2017.10000877 maloni, m.j., carter, c.r. (2006). opportunities for research in third party logistics, transportation journal, 45(2), 23 38. mangla s.k., govindan k., luthra s. (2016). critical success factors for reverse logistics in indian industries, a structural model. journal of cleaner production, 129, 608-621. doi: https://doi.org/10.1016/j.jclepro.2016.03.124 mitra, s. & bagchi, p.k. (2008). key success factors, performance metrics, and globalization issues in the third-party logistics (3pl) industry: a survey of north american service providers. supply chain forum: an international journal, 9(1), 4254. doi: https://doi.org/10.1080/16258312.2008.11517189 ijahp article: sawant, sarode/identification of criteria for third party logistics suppliers (3pl) in supply chain management (scm) in indian manufacturing industries international journal of the analytic hierarchy process 304 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.845 mitra, s., mukherjee, a., pal, p. & dutta, s. a survey of indian express delivery service providers. iim calcutta, wps no. 638/ may 2009. doi: https://doi.org/10.1177/0020881712469470 mitra, s. (2010). a comparative study of north american and indian third-party logistics (3pl) service providers. 2010 ieee. doi: https://doi.org/10.1109/iccae.2010.5451460 mothilal, s., gunasekaran, a., nachiappan s.p. & jayaram, j. (2011). key success factors and their performance implications in the indian third-party logistics (3pl) industry. international journal of production research, 50(9), 2407-2422. doi: https://doi.org/10.1080/00207543.2011.581004 peng, j. (2012). selection of logistics outsourcing service suppliers based on ahp: 2012 international conference on future electrical power and energy systems. energy procedia 17 (2012), 595 – 601. doi: https://doi.org/10.1016/j.egypro.2012.02.141 punakivi, m. & saranen, j. (2001). identifying the success factors in e-grocery home delivery. international journal of retail & distribution management, 29(4). 156 163. doi: https://doi.org/10.1108/09590550110387953 ragatz, g.l., handfield, r.b. & scannell, t.v. (2012). success factors for integrating suppliers into new product development, 14, 190-202. doi: 10.1111/1540-5885.1430190. raut, r., kharat, m., kamble, s., kumar, c.s. (). sustainable evaluation & selection of potential third-party logistics providers (3pl): an integrated mcdm approach. benchmarking: an international journal, 25(1), 76-97. doi: https:// doi.org/10.1108/bij05-2016-0065. raut, r.d., kharat, m.g., kamble, s.s., kamble, s.j., desai, r. (2018). evaluation and selection of third-party logistics providers using an integrated multi-criteria decisionmaking approach. international journal of services and operations management, 29(3), 2018. doi: https://doi.org/10.1504/ijsom.2018.10010642 sahay, b.s., mohan, r. (2006). third party logistics practices: an india perspective. park, y. (2011). analytic hierarchy process for decision making in kinesiology: an application in selecting athletic shoes for walking. university of illinois at urbana champaign. doctoral dissertation. sahin, o.z., mohamed, s. (2016). evaluating sea level rise adaptation options on the gold coast, australia: an mcda approach. emerald group publishing group. sarode, a., kole, s.y. (2016). a literature overview on green supply chain management and critical factor. international journal of advanced engineering innovative technology, 3(1), 1-5. sarode, a.d., khodke, p.m. (2011). a framework for performance measurement of supply chain management. international journal of advanced engineering technology, ii(iv), 182-190. ijahp article: sawant, sarode/identification of criteria for third party logistics suppliers (3pl) in supply chain management (scm) in indian manufacturing industries international journal of the analytic hierarchy process 305 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.845 sarode a. d., sunnapwar v. k., & khodke p. m. (2008). a literature review for identification of performance measures for establishing a framework for performance measurement in supply chains. international journal of applied management and technology, 6(3), 241-273 sharda, k. & chatterjee, l, (2011), configurations of outsourcing firms and organizational performance. strategic outsourcing: an international journal, 4(2), 152 178. doi: https://doi.org/10.1108/17538291111147991 shepherd, b. & hamanaka, s. (2015). overcoming trade logistics challenges: asiapacific experiences. asia pacific journal of marketing and logistics, 27(3), 444 466. doi: https://doi.org/10.1108/apjml-09-2014-0133 schroter, n., schroter, i. (2010). supply chain management and logistics. verlag w. kohlhammer, stuttgart. seite 15. sullivan, f. (2006). 3pl services in india_ challenges, opportunities and recommendations a study. 1 58. taherdoost, h. (2017). decision making using the analytic hierarchy process (ahp): a step by step approach. international journal of economics and management systems, 2, 2017. tan, k.c. (2001). a framework of supply chain management literature. european journal of purchasing & supply management, 7(2001), 39 -48. trent, r.j., & monczka, r.m. (1994). effective cross-functional sourcing teams: critical success factors. national association of purchasing management, inc. doi: https://doi.org/10.1111/j.1745-493x.1994.tb00267.x vaidyanathan, g. (2005). a framework for evaluating third-party logistics. communications of the acm, 48(1), 89-94. doi: https://doi.org/10.1145/1039539.1039544 xu, s. ( ). an optimized performance analysis of closed loop supply chain of third-party logistics based on analytic hierarchy process. revista de la facultad de ingenieraí u.c.v., 32(12), 827-835. yahya, s. & kingsman, b. (1999).vendor rating for an entrepreneur development programme: a case study using the analytic hierarchy process method. journal of the operational research society, 50, 916 930. doi: https://doi.org/10.1057/palgrave.jors.2600797 ijahp article: kravchenko, seredenko/decision-making with modeling of problem situations using the analytic network hierarchy process international journal of the analytic hierarchy process 28 vol. 3 issue 1 2011 issn 1936-6744 decision-making with modeling of problem situations using the analytic network hierarchy process tatiana kravchenko state university – higher school of economics moscow, russia e-mail: tkravchenko@hse.ru natalia seredenko state university – higher school of economics moscow, russia e-mail: alia_nata@mail.ru abstract the modeling of problem situations is a very important issue in decision-making theory. actually, there are no decision support systems which include decision making methods under risk and uncertainty. the main advantage of a proposed approach is the ability to process dependences and feedbacks which may exist between conditions, sub-conditions and their realizations. keywords: modeling of economic decision-making problem situations, analytic hierarchy process (ahp), analytic network process (anp), relevant significance estimates of problem situations. 1. introduction the decision maker (dm) has to consider interrelation of his particular case with other areas of concern during the decision process. thereto dm needs a comprehensive model of an external environment, which is called a scenario. in practice the dm constructs a more limited model focused on his specific problem – the local scenario (figure 1). figure 1 model of the local scenario rob typewritten text http://dx.doi.org/10.13033/ijahp.v3i1.81 ijahp article: kravchenko, seredenko/decision-making with modeling of problem situations using the analytic network hierarchy process international journal of the analytic hierarchy process 29 vol. 3 issue 1 2011 issn 1936-6744 when dm doesn’t model even a local scenario, then, in fact, the decision is accomplished in “ideal conditions” without considering an external environment. quite often the local scenario is represented in the form of variant problem situations, which dm should take into account. in this case the choosing of an efficient alternative becomes more complicated because experts should evaluate alternatives considering all possible problem situations and dm should compare all the estimates obtained. nevertheless, only the modeling of alternative problem situations can increase the efficiency of decision-making. there are two approaches which are mainly used for the modeling of problem situations: simulation modeling and expert forecasting. it is necessary to assign a set of conditions and dependences between them to deal with a simulation model. then we may model a problem situation by combining different results of implementation. if it is difficult to specify all dependences in the formal way, the modeling of problem situations are often based on expert forecast. there are questions which have to be solved for successful forecast: • organizational support of forecasting; • stating demands for forecasting; • organizing an analytical work group; • setting up an expert committee; and • preparing all needed background materials, software and data. precise expert forecast can be stated only if well prepared and competent experts engaged. information source must be reliable and evaluations must be correctly collected and processed. only experienced specialists are invited to join the expert committee. the main advantage of problem situations modeling approach is ability to process dependences and feedbacks which may exist between conditions, sub-conditions and their realizations (most probable results of checking sub-conditions). the methodological basis of proposed approach to the modeling of problem situations and evaluating judgments are the analytical hierarchy process (ahp) and analytical network process (anp) (saaty, 1993; saaty, 2003; saaty, 2008). 2. theoretical background of the decision-making conditions modeling we use ahp and anp for building hierarchy of conditions, which are decomposed into sub-conditions and realizations (probable results of checking sub-conditions). all these items are considered in the problem to be solved. there is a main decision goal “to find priorities of conditions, sub-conditions and their realizations” on the top of the hierarchy structure. the number of levels of conditions depends on the statement of the problem. ijahp article: kravchenko, seredenko/decision-making with modeling of problem situations using the analytic network hierarchy process international journal of the analytic hierarchy process 30 vol. 3 issue 1 2011 issn 1936-6744 we assume that the main goal and levels of conditions, which are located under the main goal, are composed in the control hierarchy of conditions. figure 2 examples of the sub-conditions networks we use analytical hierarchy process for modeling sub-conditions and their realizations. under each condition of the control hierarchy the sub-conditions network should be modeled. type of networks under the control hierarchy depends on the structural complexity of the problem. figure 2 shows possible variants of the subconditions networks. to process the control hierarchy of conditions we may use the analytical hierarchy process (saaty, 1993). to get results (to calculate the relevant significance estimates of sub-conditions and their realizations) we may use the analytical network process (saaty, 2008). 3. calculating the relevant significance estimates of decision-making conditions, sub-conditions and their realizations control hierarchy of conditions goal(to find priorities of conditions, sub-conditions and their realizations) conditions of decision-making examples of the sub-conditions networks under the control hierarchy of conditions : sub-conditions composed of their realizations ijahp article: kravchenko, seredenko/decision-making with modeling of problem situations using the analytic network hierarchy process international journal of the analytic hierarchy process 31 vol. 3 issue 1 2011 issn 1936-6744 3.1 calculating the relevant significance estimates of decision-making conditions notation • mmuuuu mm ...1),...,,...,,( 1 == — decision-making conditions; • pmn — elements of the pairwise comparison matrix for conditions, formed by dm, ;...1, mnm = • )...,,( .1.. meigeigmeig ppp = — principal eigenvector of the pairwise comparison matrix for conditions; • mmaxλ — principal eigenvalue of the pairwise comparison matrix for conditions; and • )...,,( 1 mm ppp = — vector of the relevant significance estimates (weights) of conditions. input data • mu — decision-making conditions, mm ...1= ; • mnp — elements of the pairwise comparison matrix for conditions, ....1 mm = all ratios should be estimates as numbers using the fundamental scale of the ahp, consisting of: {1/9, 1/8, 1/7, 1/6, 1/5, 1/4, 1/3, 1/2, 1, 2, 3, 4, 5, 6, 7, 8, 9}. there is a correlation between evaluations mnp and nmp : .1 mnnm pp = it means that if ,7=nmp then the relevant significance estimates of mcondition is very strong dominate ncondition. note that in this case nmp = 1/7, so it means that the relevant significance estimates of ncondition is very strongly dominated by mcondition. solution algorithm 1. dm forms input data. 2. dm fills up the pairwise comparison matrix with elements ,mnp where ,...1, mnm = to evaluate the relevant significance estimates of mand nconditions. 3. then the principal eigenvector of the pairwise comparison matrix for conditions meigp . has to be obtained (saaty, 2003). there is a general equation for obtaining the principal eigenvector (in compliance with the principal eigenvector definition): (1) meigmmeigmn ppp .max. ×=× λ 4. the elements of obtained eigenvector have to be normalized: (2) ∑ = m meig meig m p p p . . 5. the vector )...,,( 1 mm ppp = is the required vector relevant significance estimates of conditions, which will be used further for calculating the relevant significance estimates of sub-conditions and their realizations. the example of forming and processing the control hierarchy of conditions is described in chapter 5.2. ijahp article: kravchenko, seredenko/decision-making with modeling of problem situations using the analytic network hierarchy process international journal of the analytic hierarchy process 32 vol. 3 issue 1 2011 issn 1936-6744 table 1 the fundamental scale of absolute numbers rating interpretation of the evaluations 1 relevant significance estimate is equal importance 2 relevant significance estimate is weak 3 relevant significance estimate is moderate importance 4 relevant significance estimate is moderate plus 5 relevant significance estimate is strong importance 6 relevant significance estimate is strong plus 7 relevant significance estimate is demonstrated importance 8 relevant significance estimate is very, very strong 9 relevant significance estimate is extreme importance 3.2 calculating relevant significance estimates of decision-making sub-conditions and their realizations after processing the control hierarchy of conditions, dm should form and make calculations over all sub-networks, presenting the sub-conditions structure for each condition. as stated above, the structure of sub-networks under the control hierarchy of conditions depends on the decision-making problem. one of the possible views of decision sub-conditions and their realizations sub-networks is shown in figure 3. ijahp article: kravchenko, seredenko/decision-making with modeling of problem situations using the analytic network hierarchy process international journal of the analytic hierarchy process 33 vol. 3 issue 1 2011 issn 1936-6744 sub-goal to find priorities of subconditions and their realizations sub-condition um1 realization um11 realization um12 realization um1k realization um1ky . . . . . . . . . . . . . . . . . . sub-condition um2 realization um21 realization um22 realization um2k realization um2ky . . . . . . . . . . . . . . . . . . sub-condition umym realization umym1 realization umym2 realization umymk realization umymky . . . . . . . . . . . . . . . . . . ... figure 3 example of the sub-networks under the control hierarchy of conditions such sub-network structures can have the following dependences between the subgoal, sub-conditions and their realizations: • connections between the sub-goal and sub-conditions, influencing the subgoal. this type of outer dependence is shown by the solid arrows outgoing the sub-goal in figure 3 (dependences of type i). • connections between the sub-conditions and their realizations: • connections between two different sub-conditions and their realizations. this type of outer dependence is shown in figure 3 by the dotted arrows, drawn from one sub-condition to another sub-condition whose elements influence it (dependences of type ii.a); • interconnections between two different sub-conditions and their realizations. this type of outer dependence is shown in figure 3 by the two dotted multidirectional arrows between two different sub-conditions (dependences of type ii.b). • connections between realizations within some sub-condition. this type of inner dependence is shown in figure 3 by the dotted arrows, drawn from one sub-condition to the same sub-condition (dependences of type iii). as stated above, it is applicable to use anp for processing such types of subnetworks. in this paper we give the interpretation of anp for modeling subconditions and their realizations. notation ijahp article: kravchenko, seredenko/decision-making with modeling of problem situations using the analytic network hierarchy process international journal of the analytic hierarchy process 34 vol. 3 issue 1 2011 issn 1936-6744 dependences of type i • myu — decision-making sub-conditions of mcondition, where ,...1 myy = for all mm ...1= ; • myku — krealizations, formed within ysub-condition, within mcondition, where ,...1 ykk = ,...1 myy = ....1 mm = • i tymp ),( — elements of the pairwise comparison matrix for yand tsubconditions influencing mcondition. these matrixes reflect the dependences of type i. the number of such type of matrixes equals to the number of conditions, which are considered in the decisionmaking problem. the general view of the pairwise comparison matrix of type i is illustrated in table 2. table 2 general view of the pairwise comparison matrix for the sub-conditions of mcondition 1mu … mtu … mmyu 1mu … … … myu … … i tymp ),( … mmy u • i meigp . — principal eigenvector of the pairwise comparison matrix for umy sub-conditions influencing mcondition; ....1 mm = • i mmaxλ — principal eigenvalue of the pairwise comparison matrix for subconditions influencing mcondition; ....1 mm = • i myp — vector of the relevant significance estimates (weights) of subconditions influencing mcondition; ,...1 myy = ....1 mm = dependences of type iia • t — sub-conditions of m-condition, influencing on the other sub-conditions; • y — sub-conditions of m-condition, being influenced by the other subconditions; • myku — krealizations of ysub-condition (of m-condition), being influenced by the other sub-conditions, ykk ...1= ; • mthu — hrealization of tsub-condition (of m-condition), influencing the realizations of other sub-conditions, tkh ...1= ; and ijahp article: kravchenko, seredenko/decision-making with modeling of problem situations using the analytic network hierarchy process international journal of the analytic hierarchy process 35 vol. 3 issue 1 2011 issn 1936-6744 • iia uuu mtlmthmyk p ),( — elements of the pairwise comparison matrix for hand l realizations of tsub-condition of mcondition, influencing krealizations of ysub-condition. note that in compliance with the notation conventions, as shown in figure 3, ysubconditions are the sub-conditions from which the dotted arrows come and tsubconditions are the sub-conditions in which the dotted arrows are. each generated dependence (dotted arrow) would be defined by such number of pairwise comparison matrixes for hand lrealizations of tsub-conditions, what number of krealizations is contained in ysub-condition, being influenced by tsubcondition. the general view of the pairwise comparison matrixes of type iia is illustrated in table 3. table 3 general view of the pairwise comparison matrix for the realizations of tsubcondition, influencing the realizations of ysub-condition 1mtu … mtlu … tmtku 1mtu … … … mthu … … iia uuu mthmyk p ,( … tmtk u • piiаeig.m(y,t) — principal eigenvectors of the pairwise comparison matrixes in which the realizations of tsub-condition, influencing the realizations of y sub-conditions, are estimated; • λiiаmax.m(y,t) — principal eigenvalues of the pairwise comparison matrixes; and • piiаm(y,t) — vectors of the relevant significance estimates (weights) of decision-making realizations. dependences of type iib in case of interdependence between two different sub-conditions and their realizations (dependences type iib, as shown in figure 3 by the two dotted multidirectional arrows), besides matrixes of type iia, additional matrixes should be completed. iib uuu mylmykmth p ),( — elements of the pairwise comparison matrix for kand lrealizations of ysub-condition of mcondition, being influenced by the hrealizations of tsub-condition. ijahp article: kravchenko, seredenko/decision-making with modeling of problem situations using the analytic network hierarchy process international journal of the analytic hierarchy process 36 vol. 3 issue 1 2011 issn 1936-6744 note that in compliance with the notation conventions, as shown in figure 3, tsubconditions are the sub-conditions, from which the dotted arrows come and ysubconditions are the sub-conditions in which the dotted arrows are. each generated dependence (dotted arrow) would be defined by such number of pairwise comparison matrixes for kand lrealizations of ysub-conditions, what number of hrealizations is contained in tsub-condition, being influenced by ysubcondition. the general view of the pairwise comparison matrixes of type iib is illustrated in table 4. table 4 general view of the pairwise comparison matrix for the realizations of tsubcondition, being influenced by the realizations of ysub-condition 1myu … mylu … ymyku 1myu … … … myku … … ( mumthu iibp … ymyk u thus, tables 3 and 4 show the type ii dependences. • piibeig.m(t,y) — principal eigenvectors of the pairwise comparison matrixes, in which the realizations of tsub-condition, being influenced by the realizations of ysub-conditions of mcondition, are estimated; • λiibmax.m(t,y) — principal eigenvalues of the pairwise comparison matrixes; and • piibm(t,y) — vector of the relevant significance estimates (weights) of decisionmaking realizations. dependences of type iii • t — sub-conditions of m-condition with internal dependences between realizations; • umtk — krealizations of tsub-condition of mcondition, being influenced by the other realizations of the same sub-condition, where tkk ...1= ; • umth , umtl – hand lrealizations of tsub-condition of mcondition, influencing the other realizations of the same sub-condition, where tkk ...1= ; and • ),( mtlmthmtk uuu iiip – elements of the pairwise comparison matrix for hand lrealizations of tsub-condition of mcondition, influencing krealizations of the same tsub-condition. note that each generated dependence (shown as dotted arrow drawn from one subcondition to the same sub-condition) would be defined by such number of pairwise comparison matrixes for kand lrealizations, what number of realizations is contained in tsub-condition. the general view of the pairwise comparison matrixes of type iii is illustrated in table 5. ijahp article: kravchenko, seredenko/decision-making with modeling of problem situations using the analytic network hierarchy process international journal of the analytic hierarchy process 37 vol. 3 issue 1 2011 issn 1936-6744 table 5 general view of the pairwise comparison matrix for the realizations of sub-condition with internal dependences and feedback iii mtu 1 … iii mtlu … iii mtk t u iii mtu 1 … … … iii mthu … … iii uu mthmtk p ( … iii mtk t u • iii ueig mt p . — principal eigenvectors of the pairwise comparison matrixes for the realizations of each tsub-condition of mcondition; • iii u mtmax λ — principal eigenvalues of the pairwise comparison matrixes for the realizations of each tsub-condition of mcondition; • iii u mt p — vector of the relevant significance estimates (weights) of decisionmaking realizations of sub-conditions with internal dependences of mcondition; • pmsupermatr – weighted supermatrix composed of the eigenvectors derived from the pairwise comparison matrixes for the realizations of each sub-condition of mcondition with elements: iia uuu mtlmthmyk p ),( , iib uuu mylmykmth p ),( , ),( mtlmthmtk uuu iiip ; • limmsupermatrp — weighted supermatrix being raised to powers (the limit supermatrix); and • mykp — required vectors of the global relevant significance estimates (weights) of decision-making realizations. input data • myu — decision-making sub-conditions of mcondition, where ,...1 mm = myy ...1= ; • myku — krealizations of tsub-condition of mcondition; and • pim(y,t) , iia uuu mtlmthmyk p ),( , iib uuu mylmykmth p ),( , ),( mtlmthmtk uuu iiip — elements of the pairwise comparison matrixes formed by dm. solution algorithm 1. dm forms input data. ijahp article: kravchenko, seredenko/decision-making with modeling of problem situations using the analytic network hierarchy process international journal of the analytic hierarchy process 38 vol. 3 issue 1 2011 issn 1936-6744 2. dm fills up the pairwise comparison matrixes with elements i tymp ),( to evaluate the relevant significance estimates of sub-conditions. all ratios should be estimates as numbers using the fundamental scale of the ahp. interpretation of the evaluations is illustrated in table 1. 3. then the principal eigenvector of the pairwise comparison matrix for subconditions i meigp . has to be obtained (saaty, 2003). there is a general equation for obtaining the principal eigenvector (in compliance with the principal eigenvector definition): (3) ..max.),( i meig i m i meig i tym ppp ×=× λ 4. the elements of obtained eigenvector have to be normalized: (4) . 1 . . ∑ = = my y i myeig i myeig my i p p p 5. dm forms the pairwise comparison matrixes with elements ),( mtlumthumyku iiap , ),( mylumykumthu iibp and ),( mtlmthmtk uuu iiip . filling the matrixes, the dm has to answer the basic question in all pairwise comparisons: «how many times more dominant is one element than the other with respect to a certain element (sub-condition or condition)? ». all ratios should be estimates as numbers using the fundamental scale of the ahp. this allows to take into account the mutual influences and inner dependences of subconditions. interpretation of the evaluations is illustrated in table 1. 6. then the principal eigenvectors of the pairwise comparison matrixes for realizations iia tymp ),(eig. , iib ytmp ),(eig. and iii mtpeig. have to be obtained (saaty, 2003). there are general equations for obtaining the principal eigenvectors: (5.1) ),(.),(max),(.),( ** tymeigiiatymiiatymeigiiamtlumthumyku iia ppp λ= ; (5.2) ),(.),(max),(.),( ** ytmeigiibytmiibytmeigiibmylumykumthu iib ppp λ= ; (6) mteig iii mt iii mteig iii uuu iii ppp mtlmthmtk .max.),( ** λ= . 7. the elements of obtained eigenvectors have to be normalized: (7.1) ∑ −= = tkk ktymeig iia ktymeig iia ktym iia p p p 1 ),(. ),(. ),( ; (7.2) ∑ −= = tkk kytmeig iib kytmeig iib kytm iib p p p 1 ),(. ),(. ),( ; (8) ∑ −= = tkk mtkeig iii mtkeig iii mtk iii p p p 1 . . . 8. all derived vectors of the relevant significance estimates (weights) of decisionmaking realizations and sub-conditions (pt. 4 and pt. 7) should be placed in the colijahp article: kravchenko, seredenko/decision-making with modeling of problem situations using the analytic network hierarchy process international journal of the analytic hierarchy process 39 vol. 3 issue 1 2011 issn 1936-6744 umns of the supermatrix pmsupermatr (table 6). the blocks of the supermatrix are to be filled according to the following rules: • the vectors pim presenting dependences of type i are used to weigh the elements of the corresponding blocks of the supermatrix. • the vectors piiam(y,t), piibm(t,y) presenting dependences of type iia and type iib are placed within the outside of the main diagonal blocks according to the following rules: • in the case of outer dependence between two different sub-conditions and their realizations (dependence of type iia), only the one block of the supermatrix has to be filled. the symmetry with respect to the main diagonal block is all zero. this case is illustrated in table 6 by blocks 2.1 and 2.2; and • in the case of interdependence between two different sub-conditions and their realizations (dependence of type iib), both blocks symmetry with respect to the main diagonal of the supermatrix have to be filled. this case is illustrated in table 6 by blocks 4.1 and 4.2. • the vectors iiimtp presenting dependences of type iii should be placed on the main diagonal blocks. this case is illustrated in table 6 by block 3. in case there are no inner dependences within the sub-condition then the corresponding block is all zero. this case is illustrated in table 6 by block 1. 9. then, to obtain the limit supermatrix, it is necessary to raise the supermatrix to powers until it is orderly cyclic (saaty, 2008): (9) . 1 lim 1 lim ∑ = ∞→ = n k k supermatrksupermatr p n p 10. in compliance with anp the required vectors of the global relevant significance estimates (weights) of decision-making realizations pmyk may be obtained from the limit supermatrix limmsupermatrp (saaty, 2008). note that all inner and outer dependences between sub-conditions and their realizations are taken into account. the similar calculations should be implemented for each mcondition, where m=1...m. table 6 general view of the weighted supermatrix sub-conditions realizations … realizations … realizations r ea liz atio ns (1) zero entries (2.2) zero entries r ea liz atio ns (2.1) zero entries (3) nonzero entries (4.2) nonzero entries su bco nd iti on s r ea liz atio ns (4.1) nonzero entries the example of processing the sub-conditions and their realizations network structure is described in chapter 5.3. ijahp article: kravchenko, seredenko/decision-making with modeling of problem situations using the analytic network hierarchy process international journal of the analytic hierarchy process 40 vol. 3 issue 1 2011 issn 1936-6744 4. modeling of decision-making problem situations in fact, the problem situation is a set of different realizations of all considered subconditions of all considered decision-making conditions, between which there is a logical conjunction “and”. besides, one problem situation should contain only one realization of each sub-condition. because of the logical connective “and” it is reasonable to add the relevant significance estimates (weights) of decision-making realizations which make a problem situation in aggregate. thus, the indicator of the relevant significance estimate (weight) of decision-making problem situation may be calculated as a sum of weights of all realizations which are made in aggregate this problem situation. if there is a considerable quantity of different sub-conditions and their realizations, then the quantity of decision-making problem situations will be very great. in addition, after forming a set of problem situations every expert should provide all corresponding judgments (evaluating alternatives with respect to the criteria etc.) considering all problem situations. so the labour-intensiveness of decision-making can be unreasonably high. therefore only six-to-eight problem situations with the highest weights are usually considered. the example of forming a set of problem situations is described in chapter 6.4. 5. the example of forming a set of problem situations and their relevant significance estimates to illustrate the stated theoretical basis consider it outsourcing decision making. this decision-making problem has been described by the scientific group of youxu tjader, jennifer shang, luis vargas and jerry may. these researchers made a report at the international symposium isahp2009. in that paper (tjader, shang, vargas and may, 2009) the anp and the balanced scorecard (bsc) are used for it outsourcing decision making. according to the current article, it is offered to use the bsc for revealing a set of decision-making conditions and their sub-conditions. thus the concept of “perspective” is interpreted as a decision-making condition and concept of “key indicator” is interpreted as a decision-making sub-condition. the ahp/anp are used for the modeling of decision-making problem situations and calculating their relevant significance estimates. 5.1 target setting it outsourcing decision making is a really topical problem for any large-scale enterprise. it is necessary to analyze all circumstances around the company which may occur in the future to make an effective decision. so, as shown in figure 4, we consider perspectives (financial, customer, company learning and growth, internal operations) as decision-making conditions and their key indicators as decision-making subconditions (tjader, shang, vargas and may, 2009). ijahp article: kravchenko, seredenko/decision-making with modeling of problem situations using the analytic network hierarchy process international journal of the analytic hierarchy process 41 vol. 3 issue 1 2011 issn 1936-6744 financial cash flow industry leader profitability cost savings customer availability prod/serv customer satisfaction price stability customer database employee competency company learning and growth employee satisfaction technology rd management expertise internal operations core focus quality internal control agility certificates figure 4 framework of perspectives and key indicators 5.2 forming the control hierarchy of conditions figure 5 gives the control hierarchy of conditions for the it outsourcing decision making. the control hierarchy goal (to calculate the relevant significance estimates of conditions, sub-conditions and their realizations) decision-making conditions: 1. financial 2. customer 4. company learning and growth 3. internal operations figure 5 the control hierarchy for the it outsourcing decision making ijahp article: kravchenko, seredenko/decision-making with modeling of problem situations using the analytic network hierarchy process international journal of the analytic hierarchy process 42 vol. 3 issue 1 2011 issn 1936-6744 after forming the set of conditions it is necessary to model network sub-conditions structure for each of the conditions. 5.3 the example of the sub-conditions network structure for the financial condition figure 6 gives the sub-conditions network structure for the financial condition for the it outsourcing decision making. sub-goal to calculate the relevant significance estimates of the it outsourcing decision making subconditions and their realizations for the condition 1. financial 1.1. cash flow ---------------------------------------investing in it infrastructure will be reduced investing in it infrastructure will be remained unchanged investing in it infrastructure will be increased 1.2. industry leader ---------------------------------------------------the company will become an industry leader the company will not become an industry leader 1.4. cost savings -------------------------------------------cost saving will be improved cost saving will remain unchanged cost saving will increase 1.3. profitability ---------------------------------------------------profitability will increase profitability will remain unchanged profitability will increase figure 6 the sub-conditions network structure for the financial condition as shown in figure 6, the financial condition consists of the following subconditions: • 1.1. cash flow. • 1.2. industry leader. • 1.3. profitability. • 1.4. cost savings. in turn, each of the sub-conditions consists of the appropriate set of realizations. for example, the sub-condition “cash flow” may subdivide into the following realizations: • investing in it infrastructure will be reduced. • investing in it infrastructure will remain unchanged. • investing in it infrastructure will be increased. ijahp article: kravchenko, seredenko/decision-making with modeling of problem situations using the analytic network hierarchy process international journal of the analytic hierarchy process 43 vol. 3 issue 1 2011 issn 1936-6744 5.4 the example of modeling the problem situations according to the rules stated above, we may form the required set of problem situations for the it outsourcing decision making. the elements in this set are as follows: • s1: {investing in it infrastructure will be reduced; the company will become an industry leader; profitability will increase; …}. • s2: { investing in it infrastructure will remain unchanged; the company will become an industry leader; profitability will increase; …}. • s3: { investing in it infrastructure will be increased; the company will become an industry leader; profitability will increase; …}. • s4: { investing in it infrastructure will be reduced; the company will not become an industry leader; profitability will increase; …}. • … by this way all possible combinations of realizations must be searched. then all the relevant significance estimates (weights) of decision-making realizations of subconditions for each condition have to be calculated. as an example the relevant significance estimates of decision-making realizations of sub-conditions for the financial condition are resulted in table 7. all calculations are made by the decision support system superdecisions (saaty, 2002). table 7 the relevant significance estimates of decision-making realizations of sub-conditions for the financial condition realizations estimates of subconditions weighted by the estimates of conditions local estimates of realizations (within subconditions) global estimates of realizations 1.1. cash flow: 0,04 investing in it infrastructure will be reduced 0,6 0,024 investing in it infrastructure will remain unchanged 0,2 0,008 investing in it infrastructure will be increased 0,2 0,008 1.2. industry leader: 0,09 the company will become an industry leader 0,3 0,027 the company will not become an industry leader 0,7 0,063 1.3. profitability: 0,19 profitability will increase 0,5 0,095 profitability will remain unchanged 0,4 0,076 profitability will increase 0,1 0,019 … … … … ijahp article: kravchenko, seredenko/decision-making with modeling of problem situations using the analytic network hierarchy process international journal of the analytic hierarchy process 44 vol. 3 issue 1 2011 issn 1936-6744 after calculating such values for every condition, the relevant significant estimates of problem situations may be obtained. as stated above, these coefficients may be calculated as a sum of the relevant significant estimates of realizations of which the problem situations are composed. we may cite as an example the problem situation 1s : investing in it infrastructure will be reduced; the company will become an industry leader; profitability will increase; cost saving will be improved; availability of product will decrease; customer satisfaction will decrease; price stability will remain unchanged; customer database will be reduced by 10 percent; employee satisfaction will remain unchanged; technology rd will be developed; employee competency will remain unchanged; management knowhow will increase; certifications will remain unchanged; core focus will remain unchanged; quality will increase; internal control will be strengthened; agility will reduce. the relevant significant estimate of 1s p may be calculated this way: (10) 0,742 0,0140,016 0,008 0,022 0,087 0,014 0,012 0,009 0,035 0,016 0,055 0,033 0,077 0,054 0,095 0,027 0,024 p 1s =+++++++++ ++++++++= in this sum (10) the first four terms represent the relevant significant estimates of realizations of sub-conditions for the financial condition, the next four are the estimates of realizations of sub-conditions for the condition customer, the next five are the estimates of realizations of sub-conditions for the condition internal operations and the last four are the estimates of realizations of sub-conditions for the condition company learning and growth. by the same way the relevant significant estimates of all the other problem situations should be calculated. then the problem situations have to be sorted by the coefficients and the most significant of them are to be included in the task of the it outsourcing decision making. 6. conclusion this paper provides an approach to the modeling of economic decision-making problem situations using the analytical hierarchy process and the analytical network process. the calculation algorithm of the relevant significant estimates of decision-making conditions, sub-conditions and their realizations is described. the decision support system superdecisions has been used for the auxiliary calculations. the development of an algorithm and software for modeling of problem situations and finding among them the most significant are the directions for future researches. references saaty, t.l. (1993). принятие решений — метод анализа иерархий. м.: радио и связь. [in russian]. ijahp article: kravchenko, seredenko/decision-making with modeling of problem situations using the analytic network hierarchy process international journal of the analytic hierarchy process 45 vol. 3 issue 1 2011 issn 1936-6744 saaty, t.l. (2008). принятие решений при зависимостях и обратных связях. аналитические сети. м.: изд-во лки. [in russian]. saaty, t.l. (2003). decision-making with the ahp: why is the principal eigenvector necessary. european journal of operational research. 145 (1), 85-91. youxu tjader, jennifer shang, luis vargas, jerry may. (2009). integrating the analytic network process and the balanced scorecard for strategic it outsourcing decision making. the joseph m. katz graduate school of business university of pittsburgh. saaty, r.w. (2002). decision making in complex environments: the analytic network process (anp) for dependence and feedback; a manual for the anp software superdecisions. creative decisions foundation, 4922 ellsworth avenue, pittsburgh, pa 15213. ijahp article: raco, krejci, ohoitimur, raton, adrian, rottie, soputan, sumual, ngenget sumual, sumakud/priority sector of small and medium enterprises using ahp: a case study of yamaru enterprise international journal of the analytic hierarchy process 220 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.862 priority sector of small and medium enterprises using ahp: a case study of yamaru enterprise jozef richard raco management department, universitas katolik de la salle manado, indonesia jraco@unikadelasalle.ac.ied james v. krejci lewis university, illinois usa college of business one university parkway romeoville, il 60446 krejcija@lewisu.edu johanis ohoitimur philosophy department, sekolah tinggi filsafat seminari pineleng, indonesia johoitimur@unikadelasalle.ac.id yulius raton industrial engineering department, universitas katolik de la salle manado, yraton@unikadelasalle.ac.id angelia melani adrian informatics engineering department, universitas katolik de la salle manado madrian@unikadelasalle.ac.id ronaldo rottie industrial engineering department, universitas katolik de la salle manado, rrottie@unikadelasalle.ac.id jeanette soputan faculty of animal husbandry, universitas sam ratulangi manado, indonesia jeanette@unsrat.ac.id stevanus ngenget hospitality and tourism department, universitas katolik de la salle manado, indonesia sngenget@unikadelasalle.ac.id loureine sumual management department, universitas katolik de la salle manado, indonesia lsumual@unikadelasalle.ac.id erick sumakud accounting department, universitas katolik de la salle manado, indonesia esumakud@unikadelasalle.ac.id mailto:jraco@unikadelasalle.ac.ied mailto:johoitimur@unikadelasalle.ac.id mailto:yraton@unikadelasalle.ac.id mailto:madrian@unikadelasalle.ac.id mailto:jeanette@unsrat.ac.id mailto:sngenget@unikadelasalle.ac.id mailto:lsumual@unikadelasalle.ac.id mailto:esumakud@unikadelasalle.ac.id ijahp article: raco, krejci, ohoitimur, raton, adrian, rottie, soputan, sumual, ngenget sumual, sumakud/priority sector of small and medium enterprises using ahp: a case study of yamaru enterprise international journal of the analytic hierarchy process 221 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.862 abstract small and medium enterprises (smes) play an important role in the development of a country's economy by providing employment and using local resources; they also have the ability to quickly adapt to external and internal changes and technological developments due to their flexible nature. the development of smes continues to be encouraged, but there are many complex factors that impact their development, primarily focused on the lack of resources. this study seeks to determine priority sectors for developing smes and business units that offer the greatest potential opportunities. this study uses the ahp method, which is widely used in multiple-criteria decision analysis (mcda). out of the five criteria analyzed, human resources was the priority factor (28.1%) that must become the main focus for yamaru enterprise. the results of this study can be adopted by other smes. keywords: smes; yamaru; ahp; manado; strategic management; mcdm 1. introduction several previous studies suggest that small and medium enterprises (smes) are a critical economic engine in a country's development (arthur-aidoo, aigbavboa, & thwala, 2016). according to rojas-lema et al. (2019), the economic growth of developing countries is positively related to the work of smes. the economy of most industrialized countries, previously sustained by large companies, is now dominated by smes. this is partly due to increasingly fierce global competition. the rapid development of technology provides many opportunities for smes to be more specialized in their product and service offerings. technology also allows smes to enter the market with a simple strategy while utilizing all their available resources (suroso, anggraeni, & andriyansah, 2017). smes contribute to solving the unemployment problem experienced by many countries and promote equal opportunity for job seekers by creating a significant number of new jobs. in addition, smes are also vital for a country’s economy because they utilize local natural resources (idar, yusoff, & mahmood, 2012). smes have become instruments of state development because they can mobilize and empower society as a whole (suroso et al., 2017). this is due to several positive characteristics of smes; they are flexible, dynamic, innovative, efficient and small in size so that they are both nimble and flexible (idar et al., 2012). additionally, direct feedback, short decision-making paths, and faster market understanding allow them to quickly respond to consumer needs. however, smes must overcome their lack of resources which include a small workforce and revenue (razak, abdullah, & ersoy, 2018). smes in indonesia are growing rapidly and are very important for several reasons such as their potential to create jobs. smes were able to contend with the indonesian economic crisis of 1997 better than many large companies (sunardi, widyarini, & hidajat, 2012). smes contributed 59.08% of the growth in the gross domestic product of $350,000,000, making smes one of the most important factors driving economic growth in indonesia ijahp article: raco, krejci, ohoitimur, raton, adrian, rottie, soputan, sumual, ngenget sumual, sumakud/priority sector of small and medium enterprises using ahp: a case study of yamaru enterprise international journal of the analytic hierarchy process 222 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.862 (suroso et al., 2017). ninety-nine percent of all businesses in indonesia are smes. they are the backbone of the indonesian economy and contribute significant revenue (irjayanti & aziz, 2012). according to garcía-villagrán et al. (2020), during the covid-19 pandemic, smes were encouraged to innovate by utilizing existing information technologies for marketing, promotion and payment systems, improving communication with consumers, building networks (networking), and partnerships, both with fellow smes, industry and government (garcía-villagrán, cano-olivos, martínez-flores, & sánchez-partida, 2020) hadiyati and hendrasto (2021) added that during the covid-19 pandemic, the indonesian government provided assistance to smes in the form of a policy of delaying loan repayment and encouraging domestic product consumption to stimulate smes to produce substitutes for imported products (hadiyati & hendrasto, 2021). as a result, smes have played an important role in the economic development of communities both during and post-covid-19 pandemic. in indonesia, smes are defined as an enterprise having a net worth of not more than idr 50,000,000.00 (fifty million rupiah), excluding land and buildings, where the business itself has an annual revenue up to a maximum of idr 500,000,000.00 (five hundred million rupiah), not including land and buildings (president of indonesia, 2008). although important, smes have a number of challenges. their biggest problem is the lack of availability of skilled human resources (sunardi et al., 2012), limited access to markets, technology, information and capital. many smes do not understand market needs, market share, and how to deliver their product or service to the market. they lack bargaining power, especially when dealing with big buyers. although they understand market conditions, they have difficulty obtaining raw materials both in terms of availability and delivery. smes face significant competition to obtain access to bank loans or government financial institutions. many are forced to find funding through lending institutions that charge high interest rates. they encounter logistical problems including energy, fuel, and labor costs (irjayanti & aziz, 2012). another distinctive feature of smes is that they are easy to start and easy to close (maulina & fordian, 2018; pawitan, 2012). indonesian smes are generally classified as part of the agricultural sector (pawitan, 2012). the yamaru sme, which is engaged in agriculture as well as animal husbandry and religious figurine production, can be viewed as an example of a sme and the results of this study can serve as a reference for other smes. this research studies the smes managed by the yamaru foundation. this foundation is owned by the brothers of ‘septem dolorem mariae’, a catholic religious order in the diocese of manado. the yamaru sme is located in the village of woloan, north sulawesi, indonesia. the yamaru sme operates three business units including agriculture, chicken-farming and religious figurine production. the goal of this research is to answer the following questions; what is the sector that should be prioritized for the development of the yamaru sme?, and what business units have the most potential to be developed? this research focuses on marketing, financial, human resources, technology and management praxis. ijahp article: raco, krejci, ohoitimur, raton, adrian, rottie, soputan, sumual, ngenget sumual, sumakud/priority sector of small and medium enterprises using ahp: a case study of yamaru enterprise international journal of the analytic hierarchy process 223 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.862 yamaru smes have a business capital of idr 150.000.000 (us$ 10,500.00) excluding land and buildings, and employ 50 employees. the foundation aspires to expand its business and has sought assistance in assessing which businesses should be improved and also what sector should be developed in order to expand their foundation. this study uses ahp as one of the multiple criteria decision making (mcdm) methods which is commonly used by decision makers to determine business priorities. the research will be used by the leadership of the yamaru foundation to determine future program priorities. this study has several limitations. first, the number of respondents was only 10 people. second, the respondents had a limited understanding of the ahp method; therefore, as suggested by mu and pereyra-rojas (2018), the researchers were required to provide initial information about this methodology. all respondents attended a three hour meeting where the ahp methodology was explained and instructions given on how to fill out the questionnaire (mu & pereyra-rojas, 2018). 2. literature review there are several key factors that influence the development of smes such as human resources (voca & havolli, 2019); marketing and technology (ahmad, imm, aziz, & basha, 2020); government support (my, 2020); entrepreneurship; and capital (luo, alshami, & mansor, 2019). researchers have focused on financial, marketing, technology, human resources, and practical management factors as criteria. researchers do not use sub-criteria to avoid clashes between the same elements on other criteria. 2.1 marketing in smes marketing has several definitions. first, marketing is defined as a management process in a business’s effort to identify, anticipate and meet consumer needs. second, marketing is defined as the process of planning and executing business concepts, determining prices and promotions with the aim of meeting consumer needs. third, marketing is defined as a function of the organization and a number of processes for creating, communicating and delivering value to consumers and cultivating relationships with stakeholders that will benefit the business itself and consumers (padmore, taylor, & frecknall-hughes, 2006). the researchers used the definition of marketing as all business activities and held that it is the responsibility of all parties in the business. marketing has a very important position in smes. a business actor or entrepreneur must understand the potential opportunities and market size for the product or service (indrawati, 2012). understanding market opportunities includes the ability to forecast demand and determine position relative to the competition. according to hamdani (2012), smes in indonesia generally do business utilizing traditional marketing techniques (hamdani & wirawan, 2012). problems commonly faced by smes include inadequate advertising and promotion of products or services, and lack of a marketing strategy that results in imprecise sales forecasts and plans. market research in indonesia generally relies on data collected from a few customers having a limited network of clients (rojas-lema et al., 2019) that results in an inadequate understanding of market conditions. many times, this results in smes competing with large companies that can sell the same products at a lower price and/or with better service (maulina & fordian, ijahp article: raco, krejci, ohoitimur, raton, adrian, rottie, soputan, sumual, ngenget sumual, sumakud/priority sector of small and medium enterprises using ahp: a case study of yamaru enterprise international journal of the analytic hierarchy process 224 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.862 2018). additionally, many smes in indonesia do not properly use their time to understand the unique value of their products or services (razak et al., 2018) or have marketers with the skills necessary to provide better customer service and overcome the perception held by many people in the domestic market that smes sell products or services that are low quality. 2.2 management praxis management practices carried out by smes are related to the pattern and effectiveness of coordination, leadership, network structures and institutional relationships. management practice emphasizes efforts to establish a common business vision and understand and implement strategies throughout the organization. coordination and collaboration between business networks helps reduce uncertainty and risk. practical management is also associated with the process of innovation and development. institutional relationships increase interactions with stakeholders and strengthen business networks with suppliers, business partners, government and educational institutions (rojas-lema et al., 2019). problems faced by smes related to practical management are lack of control over products, inefficient product planning (jester & vera, 2012), insufficient task evaluation, including description of the division of tasks among workers, lack of productivity, and a general lack of good management practices. 2.3 finance finance plays an important role relative to business expansion. smes typically rely on very limited personal savings or family funds to become established. lenders are somewhat hesitant to provide assistance to new business units because there is a significant risk of failure. to offset this risk, the lending institution requires a high interest loan. problems faced in acquiring loan funds are often related to poor accounting records and a lack of budget plans (jester & vera, 2012). another problem related to access to loans is that smes are small and they prefer short-term debt (hendrawan, 2012), while lending institutions prefer providing long-term debt. hendrawan (2012) added that the larger the business size, the higher the probability that the smes will be able to acquire a loan. loan obtainment is also related to growth, the higher and faster the growth, the easier it is to access a loan. furthermore, the age of the sme also matters. the older the firm, the stronger the belief that they have the ability to meet their financial obligations on time, which increases the chances of getting a loan. service businesses are usually less attractive to lending institutions than highly capital intensive businesses such as manufacturing because service businesses lack assets that can be used as collateral for loans. larger banks sometimes require less collateral and also charge lower interest compared to small banks; however, the big banks tend to provide loans to smes that have been around for a long time and are larger because they are considered less risky. the educational background of the smes manager is also extremely important. managers with higher education will find it easier to access assistance. many smes experience information asymmetry regarding bank loans (gherghina, botezatu, hosszu, & ijahp article: raco, krejci, ohoitimur, raton, adrian, rottie, soputan, sumual, ngenget sumual, sumakud/priority sector of small and medium enterprises using ahp: a case study of yamaru enterprise international journal of the analytic hierarchy process 225 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.862 simionescu, 2020). financial shortcomings and difficulties negatively impact the production capacity of smes (razak et al., 2018). 2.4 technology technology has always been associated with machines, tools and instruments to improve and accelerate business operations. fast technological developments need to be exploited by smes to seek and find new methods and processes to be able to survive competition (hamdani & wirawan, 2012). in the competitive business environment, the ability to survive is dependent on the ability to innovate using technology. those who adapt and redefine their production processes usually survive. smes have great opportunities with technological advances because of their small size which makes them very flexible in adapting to technological advances (askum, staub, & gonullu, 2017). however, limited funds can also lead to a lack of technology development making it difficult to innovate and employ experts (hamdani & wirawan, 2012). many smes still use traditional technology such as old machines and outdated manual business equipment resulting in slow and inefficient production processes (maulina & fordian, 2018). 2.5 human resources reliable human resources who have expertise and are well educated and have extensive experience in the business world will augment business performance (suroso et al., 2017). smes in indonesia experience serious problems related to their human resources, who generally lack sufficient expertise, experience and education. lack of human resource skills affects business management, entrepreneurship, production engineering, product development, engineering design quality control, business organization, accounting, data processing, marketing techniques and market research (maulina & fordian, 2018). 2.6 analytical hierarchy process we are faced daily with decisions that are both very simple and more complex. one method that is widely used in decision making is the ahp which was introduced by saaty (atanasova-pachemska, lapevski, & timovski, 2014). this method sorts the problem and arranges it in the form of a hierarchy to reduce the complexity of the problem and therefore greatly facilitates decision makers ability to make decisions, determine the criteria to be used and alternatives to be evaluated (mu & pereyra-rojas, 2018). this method is also able to handle intangible criteria such as experience, subjective preferences (ishizaka & labib, 2009) and intuition originating from multi-person respondents with multi-criteria input (vrana, 2008). this method can handle qualitative and quantitative data (vrana, s, 2008) based on individual perceptions. the mathematical formulas used are not difficult, rather easy to understand and use (forbes, hebb, & mu, 2018). the ahp involves the four following steps: first, arrange the problem in the form of a multilevel hierarchy. next, data is collected from respondents using a questionnaire in the form of pairwise comparisons. a consistency test is performed to ensure data validity and then the value of the elements at each level is calculated (criteria, alternatives). next, the aggregation of the related weights (relative) is calculated to obtain the overall priorities which provides the global ranking of alternatives (marcarelli & squillante, 2019). ijahp article: raco, krejci, ohoitimur, raton, adrian, rottie, soputan, sumual, ngenget sumual, sumakud/priority sector of small and medium enterprises using ahp: a case study of yamaru enterprise international journal of the analytic hierarchy process 226 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.862 3. methodology this research follows several steps. the first step is to identify a decision problem and compile a hierarchy. the topmost level is the goal of the research, the second level is the criterion, and the third level is the alternatives as shown in figure 1. figure 1 hierarchy structure of ahp the goal of this research is to determine the priority sectors of yamaru smes. there are five factors that are used as criteria, namely: marketing, technology, human resources, financial and management. marketing in this study is defined as the process of communicating about products and their value to consumers and building networks that are beneficial to both consumers and the yamaru foundation. technology is understood as the use of tools, machines and computers to improve product quality and quantity. human resources are understood as workers or managers who have adequate skills and knowledge and are able to take advantage of available technology or apply appropriate methods as well as innovate and be creative to improve yamaru's performance. financial is defined as the ability to manage, have an effective accounting system and have a good budget plan. management is understood as good coordination, and the leadership and networking ability possessed by yamaru's managers which affords a good working environment for all employees and stakeholders. the alternatives are religious figurines, chicken farming and agriculture. the hierarchical structure consists of goals, criteria and alternatives. sub-criteria have not been included in this study. the second step is data collection through a questionnaire which is arranged in the form of pairwise comparisons using the values set by saaty. ijahp article: raco, krejci, ohoitimur, raton, adrian, rottie, soputan, sumual, ngenget sumual, sumakud/priority sector of small and medium enterprises using ahp: a case study of yamaru enterprise international journal of the analytic hierarchy process 227 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.862 table 1 saaty’s (1987) comparative scale intensity of importance on an absolute scale reciprocal definition explanation 1 1 equal importance two activities contribute equally to the objective 3 1/3 moderate importance of one over another experience and judgment strongly favor one activity over another 5 1/5 essential or strong importance experience and judgment strongly favor one activity over another 7 1/7 very strong importance an activity is strongly favored and its dominance demonstrated in practice 9 1/9 extreme importance the evidence favoring one activity over another is of the highest possible order of affirmation 2,4,6,8 1/2 , 1/4 , 1/6 , 1/8 intermediate values between the two adjacent judgments when compromise is needed the advantage of pairwise comparison is that respondents only compare the two options which makes the evaluation very easy (raco et al., 2020). according to psychologists, it is easier and more accurate to choose between two options than to choose from all the alternatives at once (ishizaka & labib, 2009). when distributing the questionnaire, the researcher considered who were the right people to be involved, who would be involved in making the decision and for whom this decision would be made? (forbes et al., 2018). the researchers identified the ten key leaders of the organization who are directly responsible for operation of the yamaru sme. this questionnaire was distributed and completed by the 10 leaders of the foundation. these leaders were considered to be experts since they have been part of yamaru since its inception, have experience and knowledge in all the various industries of the organization including raising livestock, farming and production of religious figurines. there are three chicken farming managers who have over 20 years of experience managing layer chicken coops. since 2018, the managers have been entrusted as the head of the animal husbandry unit. the ability to manage laying hens is balanced with their skill of keeping egg production above 80% on average. this is achieved by maintaining the cleanliness of the cages, ijahp article: raco, krejci, ohoitimur, raton, adrian, rottie, soputan, sumual, ngenget sumual, sumakud/priority sector of small and medium enterprises using ahp: a case study of yamaru enterprise international journal of the analytic hierarchy process 228 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.862 quality of the feed and water and regular vaccinations. they now manage over 7000 chickens. there are 3 religious figurine-making individuals that are very good at marketing, opening and producing statues. they are sculpture artists that have been working since 2016 and in 2019 succeeded in doubling their production. data collection was carried out in the form of group decision makers to reduce cognitive bias and obtain the group participant's synergy (mu & pereyra-rojas, 2018). saaty emphasized that any rule to combine the judgments of several individuals should also properly satisfy the reciprocal (r. saaty, 1987). however, not all respondents or experts always make the same choice. different levels of knowledge and experience lead to different choices. the researchers asked respondents to avoid using extreme numbers such as 9 when filling out the questionnaires because in reality all of the criteria have approximately the same importance, and the extreme selection would not be helpful. if the authors found extreme values, the researchers recommended adjusting that number. therefore, it is possible to give different weighting and importance to their choices. in real-world decision-making, it often happens that the decision-making situation is so unclear that the actions taken are blurred and not precisely known (vrana, s, 2008). in this study, the researchers used the aggregation of individual judgments and manually calculated them using the geometric mean (equation 1). according to basak & saaty (1993) and mu & pereyra-rojas (2018), the geometric mean is the correct way to synthesize the judgments given by the experts as reciprocal matrices. according to mu & pereyra-rojas (2018), the geometric mean is the correct way to aggregate judgments with the ahp and the geometric mean aggregation for our study. 𝐺𝑀 = √(𝑥1)(𝑥2) … (𝑥𝑛 ) 𝑛 … … … … (1) after obtaining the gm value for the questionnaire data, a pairwise comparison matrix for the criteria and alternatives is compiled using equation 2: 𝐴 = [𝑎𝑖𝑗 ], 𝑎𝑖𝑗 = 𝑤𝑖 𝑤𝑗 , 𝑎𝑗𝑖 = 1 𝑎𝑖𝑗 , 𝑎𝑖𝑖 = 1 (2) the calculation of the priority weight begins by normalizing the pairwise comparison matrix using equation 3, 𝑏𝑖𝑗 = 𝑎𝑖𝑗 ∑ 𝑎𝑖𝑗 𝑛 𝑖=𝑛 (3) the priority weight is obtained using equation 4, 𝑤𝑖 = ∑ 𝑏𝑖𝑗 𝑛 𝑗=1 𝑛 (4) ijahp article: raco, krejci, ohoitimur, raton, adrian, rottie, soputan, sumual, ngenget sumual, sumakud/priority sector of small and medium enterprises using ahp: a case study of yamaru enterprise international journal of the analytic hierarchy process 229 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.862 after the priority weight value is obtained, the consistency test is then determined by using equations 5, 6 and 7 as follows: the third step is to calculate the lambda maximum using the following formula: 𝜆𝑚𝑎𝑥 = ∑ (𝐴𝑤)𝑖 𝑛𝑤𝑖 (5) 𝑛 𝑖=1 the fourth step is to calculate the consistency index with equation 6 𝐶𝐼 = 𝜆𝑚𝑎𝑥 − 𝑛 𝑛 − 1 (6) the fifth step is to calculate the consistency ratio with a formula using the ratio index (table 2). 𝐶𝑅 = 𝐶𝐼 𝑅𝐼 (7) table 2 ratio index (saaty, 1987) n 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 r.i 0.00 0.00 0.58 0.90 1.12 1.24 1.32 1.41 1.45 1.49 1.51 1.48 1.56 1.57 1.59 the calculation of the consistency index and the consistency ratio shows that the answers given by the respondents are open to inconsistency. this is because the numerical values come from individual subjective choices. therefore, inconsistency in making choices is inevitable (mu & pereyra-rojas, 2018), but the inconsistency has limits that can be accepted and tolerated. the inconsistency must be below 10% (˂ 0.1) (saaty, 1987). checking consistency is a crucial step to avoid misleading solutions. the sixth step was calculating the global weight. global weight is obtained by multiplying criteria and alternatives. the seventh step is a sensitivity analysis using super decision version 2.10.0. a sensitivity analysis is known as a 'what if analysis' to see the extent to which the final result will change if there is a change in the criterion weights (mu & pereyra-rojas, 2018). a sensitivity analysis helps researchers understand the strength of the decisions made and what the determining criteria are. this is an important part of the decisionmaking process and no final decision can be made in the absence of a sensitivity analysis (mu & pereyra-rojas, 2018). therefore, to ensure their robustness, the researcher conducted a sensitivity analysis between the criteria. 4. results and discussion the goal of the study was to determine the most prioritized sector of the yamaru smes. based on the literature review, the researcher determined five criteria to be studied, namely, management praxis, financial performance, human resources, technology utilization and marketing. there are three businesses which are among the alternatives ijahp article: raco, krejci, ohoitimur, raton, adrian, rottie, soputan, sumual, ngenget sumual, sumakud/priority sector of small and medium enterprises using ahp: a case study of yamaru enterprise international journal of the analytic hierarchy process 230 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.862 that will be studied, namely, agriculture, chicken-farming and religious figurine production. the results of the data analysis showed that both the criteria and alternatives were consistent, namely cr ˂ 0.1, so the results were valid for use. the data was calculated and synthesized using microsoft office excel version 2019. the data was obtained through a questionnaire filled out by the yamaru administrator and obtained using the 2019 version of microsoft office excel and the results are shown in tables 3-9. the opinion of each respondent, according to saaty, is used as the opinion of the group by combining these opinions using the geometric mean (saaty, t., 2013; saaty, r.,1987). saaty (2008) added that the geometric mean is the best way to combine the opinions of each individual. table 3 pairwise comparison, ʎ-max, priority, ci and cr of criteria mrk mgt fin tec hr priority mrk 1.000 0.424 0.346 1.009 0.397 0.107 mgt 2.357 1.000 0.683 1.738 0.896 0.225 fin 2.891 1.463 1.000 1.421 0.680 0.250 tec 0.992 0.575 0.704 1.000 0.534 0.138 hr 2.517 1.116 1.470 1.872 1.000 0.281 ʎ-max = 5.072 ci = 0.018 cr = 0.016 (cr ˂ 0.1) explanation: mrk = marketing mgt = management praxis fin = financial tec = technology hr = human resources the priority weight was obtained using equation 4. ijahp article: raco, krejci, ohoitimur, raton, adrian, rottie, soputan, sumual, ngenget sumual, sumakud/priority sector of small and medium enterprises using ahp: a case study of yamaru enterprise international journal of the analytic hierarchy process 231 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.862 table 4 pairwise comparison, ʎ-max, priority, ci and cr of marketing toward alternative relfig agri chic-farm priority rel.fig 1.000 1.175 0.636 0.2910 agri 0.851 1.000 0.515 0.2437 chic-farm 1.573 1.940 1.000 0.4652 ʎ-max = 3.0003 ci= 0.0001 cr = 0.0002 (cr ˂ 0.1) explanation: rel.figur = religious figurine agri = agriculture chic-farm = chicken-farming table 5 pairwise comparison, ʎ-max, priority, ci and cr of management praxis toward alternative relfig agri chic-farm priority rel. fig 1.000 1.459 0.393 0.252 agri 0.685 1.000 0.429 0.202 chic-farm 2.544 2.330 1.000 0.546 ʎ-max = 3.024 ci= 0.012 cr = 0.021 (cr ˂ 0.1) table 6 pairwise comparison, ʎ-max, priority, ci and cr of finance toward alternative relfig agri chic-farm priority rel.fig 1.000 2.221 0.416 0.296 agri 0.450 1.000 0.446 0.180 chic-farm 2.405 2.244 1.000 0.524 ʎ-max = 3.085 ci= 0.042 cr = 0.073 (cr ˂ 0.1) table 7 pairwise comparison, ʎ-max, priority, ci and cr of technology toward alternative relfig agri chic-farm priority rel.fig 1.000 1.116 0.584 0.281 agri 0.896 1.000 0.634 0.268 chic-farm 1.712 1.578 1.000 0.451 ʎ-max = 3.004 ci= 0.002 cr = 0.003 (cr ˂ 0.1) ijahp article: raco, krejci, ohoitimur, raton, adrian, rottie, soputan, sumual, ngenget sumual, sumakud/priority sector of small and medium enterprises using ahp: a case study of yamaru enterprise international journal of the analytic hierarchy process 232 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.862 table 8 pairwise comparison, ʎ-max, priority, cr and cr toward human resources relfig agri chic-farm priority rel.fig 1.000 0.823 0.552 0.247 agri 1.215 1.000 0.610 0.291 chic-farm 1.810 1.639 1.000 0.462 ʎ-max = 3.001 ci= 0.001 cr = 0.001 (cr ˂ 0.1) table 9 final result of the alternatives criteria global weight alternatives local weight global weight rel-fig agri chic-farm rel-fig agri chic-farm a b c d e f = c*b g = d*b h = e*b marketing 0.107 0.291 0.244 0.465 0.031 0.026 0.050 management 0.225 0.252 0.202 0.546 0.057 0.045 0.123 finance 0.250 0.296 0.180 0.524 0.074 0.045 0.131 technology 0.138 0.281 0.268 0.451 0.039 0.037 0.062 human resources 0.281 0.247 0.291 0.462 0.069 0.082 0.130 sum 0.270 0.235 0.495 explanation : column c refers to religious figurine the same applies to columns a h in table 9 the final results of the alternatives (table 9) calculation shows that the sector with the greatest weight was human resources (28.1%), followed by finance (25%) and then management praxis (22.5%). the business unit with the most potential to be developed was the chicken-farming business (49.5%), followed by religious figurine (27%) and agriculture (23.5%). the results of the sensitivity test using super decisions version 2.10.0 showed that by increasing the financial criteria by 10%, 30% and even 50% from its previous result, we found no change in the rating results for the business unit which had the highest rating (figure 2). this means that the alternative of chicken-farming has the most potential to be developed. ijahp article: raco, krejci, ohoitimur, raton, adrian, rottie, soputan, sumual, ngenget sumual, sumakud/priority sector of small and medium enterprises using ahp: a case study of yamaru enterprise international journal of the analytic hierarchy process 233 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.862 figure 2 sensitivity analysis by increasing the financial criteria to 10%, 30% and 50% the next sensitivity analysis increased the management praxis criteria by 10%, 30% and 50% from the previous results, and the results show that there is no change in the alternative business units. the chicken-farming alternative remains the business unit with the most potential to be developed (figure 3). figure 3 sensitivity analysis by increasing management praxis criteria to 10%, 30% & 50% ijahp article: raco, krejci, ohoitimur, raton, adrian, rottie, soputan, sumual, ngenget sumual, sumakud/priority sector of small and medium enterprises using ahp: a case study of yamaru enterprise international journal of the analytic hierarchy process 234 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.862 therefore, it can be concluded that increasing the percentage of the financial and praxis management criteria using a sensitivity analysis does not provide a significant change to the existing business unit rating. the results of the sensitivity analysis of the criteria showed that the management praxis criteria are the most sensitive because a change of 10% of those criteria will alter the existing business priorities. the results of the data analysis show that human resources is the most important factor (28.1%) for yamaru's business development. the importance of human resources has been confirmed by hernita et al (2021). they found that human resources were a key factor in the performance and success of smes (hernita, surya, perwira, abubakar, & idris, 2021). hadiyati and hendrasto (2021) added that in this covid-19 pandemic era, reliable, knowledgeable human resources are key because they provide creative and innovative ideas for a small business to survive and grow (hadiyati & hendrasto, 2021). this was also confirmed again by abdulaal and nordin (2020) who found that creativity and innovation can be expected from quality human resources, (abdulaal & nordin, 2020). bilan et al. (2020) found that good human resources are an important and effective factor in the growth of smes, especially in increasing sales revenue, enterprise assets, and expansion volume (bilan, mishchuk, roshchyk, & joshi, 2020). luo et al. (2019) also found that human resources are a very important factor that constitute the core competitiveness of an enterprise (luo et al., 2019). therefore, ahmad and imm (2020) recommend that smes recruit skilled human resources and then invest in them through education and tailored training to increase their knowledge and expertise to enhance innovation (ahmad et al., 2020). my (2020) recommends that reliable human resources in smes are needed to adapt an organization in today's rapidly changing digital age. she also points out that these human resources are critical and cannot simply be replaced with an advanced technology (my, 2020). surya et al. (2021) found that in addition to technical capabilities, human resources need to develop skills associated with work behavior and personal attributes. these findings serve as critical inputs for yamaru sme’s management as it develops a plan to make the necessary key investments in its human resources, including recruitment strategies focused on hiring employees with the necessary skills that are congruent with yamaru's needs, providing employee development through education and training and of course, accompanied by adequate wage and remuneration schemes. 5. conclusion the aim of the study was to find the priority factors and efforts that must be considered in the development of yamaru smes using the ahp application where decisions are made by a group of 10 decision makers. the researchers used the ahp method to make a real decision, which was to determine the priority sectors and the type of business that should be developed. the reliability of the ahp is very high as it detects the highest and least priorities and provides information to yamaru management about the priority sector and types of business that must be developed. the results show that the most important criteria are ijahp article: raco, krejci, ohoitimur, raton, adrian, rottie, soputan, sumual, ngenget sumual, sumakud/priority sector of small and medium enterprises using ahp: a case study of yamaru enterprise international journal of the analytic hierarchy process 235 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.862 human resources (28.1%), followed by finance (25%), then praxis management (22.5%), technology (13.8%) and marketing (10.7%). the business that must be the main concern is chicken-farming (49.5%), followed by the production of religious figurines (27%) and finally agriculture (25.5%). the sensitivity analysis showed that the alternative of chicken farming is robust. an increase of 10%, 30% and even 50% of financial and management praxis criteria did not significantly impact the overall ranking when it comes to choosing the best alternative. this is important information for yamaru's management. in developing its business, yamaru must focus on the development of human resources in the chicken-farming business, for example, good training and education. the ahp's application to the yamaru smes studied the business development by taking into account the views of a group of decision makers. this study will help yamaru's management focus and establish priorities with an emphasis on the importance of human resource management (hrm) and the development of the chicken farming business. the limitations and shortcomings of the conducted research were primarily related to the limited number of respondents. this study was based on the opinions of 10 respondents who are familiar with and know yamaru well. in this study, the researchers did not include a sub-criteria analysis. the results of this study may not be free of bias as the outcome is based on the understanding and judgment of those respondents. additionally, all respondents did not understand how to complete the ahp questionnaire, so the researcher had to provide additional information. future studies will look at the other criteria. ijahp article: raco, krejci, ohoitimur, raton, adrian, rottie, soputan, sumual, ngenget sumual, sumakud/priority sector of small and medium enterprises using ahp: a case study of yamaru enterprise international journal of the analytic hierarchy process 236 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.862 references abdulaal, a. m., & nordin, n. (2020). review paper: the mediating role of creativity on the relationship between knowledge management and human resource management toward innovation performance in the jordanian smes. international journal of industrial management, 8(2018), 12–18. doi: https://doi.org/https://doi.org/10.15282/ijim.8.0.2020.5759 ahmad, n., imm, n. s., aziz, y. a., & basha, n. k. (2020). why not knowledge-based hrm ? its direct and indirect effect on innovative service offerings. sage open, 1–20. doi: https://doi.org/10.1177/2158244020983003 arthur-aidoo, b. m., aigbavboa, c. o., & thwala, w. d. (2016). attitudes of owners which impedes firm’s growth : a case of small and medium-sized construction firms in ghana. procedia engineering, 164, 230–234. doi: https://doi.org/10.1016/j.proeng.2016.11.614 askum, o. ., staub, s., & gonullu, o. (2017). current debates in business studies (1st ed.). london: ijopec publication. atanasova-pachemska, lapevski, m., & timovski, r. (2014). analytical hierarchical process ( ahp ) method. unitech, 2014, 373–380. basak, i., & saaty, t. (1993). group decision making using the analytical hierarchy process. mathematical computer modelling, 17(415), 101-109as. bilan, y., mishchuk, h., roshchyk, i., & joshi, o. (2020). hiring and retiaining skilled employees in smes: problems in human resource practices and links with organizational success. business: theory and practice, 21(2), 780–791. doi: https://doi.org/https://doi.org/10.3846/btp.2020.12750 forbes, j., hebb, a., & mu, e. (2018). ethical decision making in action: evaluating hospital attendance approaches. international journal of analytic hierarchy process, 10(3), 313–347. doi: https://doi.org/10.13033/ijahp.v10i3.592 garcía-villagrán, a., cano-olivos, p., martínez-flores, j. l., & sánchez-partida, d. (2020). the covid-19 effect in mexican smes. advances in science, technology and engineering systems, 5(6), 63–71. doi: https://doi.org/https://dx.doi.org/10.25046/aj050608 gherghina, s. ., botezatu, m. a., hosszu, a., & simionescu, l. . (2020). small and medium-sized enterprises (smes): the engine of economic growth through investments and innovation. sustainability, 12(347), 1–22. doi: https://doi.org/10.3390/su12010347 hadiyati, e., & hendrasto, f. (2021). entrepreneurial marketing strategy of micro , small and medium enterprises in pandemic covid-19 era. international journal of economic and business administration, ix(2), 178–191. hamdani, j., & wirawan, c. (2012). open innovation implementation to sustain https://doi.org/10.13033/ijahp.v10i3.592 ijahp article: raco, krejci, ohoitimur, raton, adrian, rottie, soputan, sumual, ngenget sumual, sumakud/priority sector of small and medium enterprises using ahp: a case study of yamaru enterprise international journal of the analytic hierarchy process 237 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.862 indonesian smes. procedia economics and finance, 4, 223–233. doi: https://doi.org/10.1016/s2212-5671(12)00337-1 hendrawan, b. (2012). the small mediumfree trade zone that able to acquire debt. procedia economics and finance, 4, 76–85. doi: https://doi.org/10.1016/s22125671(12)00323-1 hernita, h., surya, b., perwira, i., abubakar, h., & idris, m. (2021). economic business sustainability and strengthening human resource capacity based on increasing the productivity of small and medium enterprises ( smes ) in makassar city , indonesia. sustainability, 13(3177), 1–36. doi: https://doi.org/https://doi.org/10.3390/su13063177 idar, r., yusoff, y., & mahmood, r. (2012). the effect of market orientation as mediator to strategic planning practices and performance relationship : evidence from malaysian smes. procedia economics and finance, 4, 68–75. doi: https://doi.org/10.1016/s22125671(12)00322-x indrawati, s. (2012). based on local competencies. procedia economics and finance, 4, 59–67. doi: https://doi.org/10.1016/s2212-5671(12)00321-8 irjayanti, m., & aziz, a. . (2012). barrier factors and potential solutions for indonesian smes. procedia economic and finance, 4, 3–12). doi: https://doi.org/10.1016/s22125671(12)00315-2 ishizaka, a., & labib, a. (2009). analytic hierarchy process and expert choice : benefits and limitations. or insight, 22(4), 201–220. doi: https://doi.org/10.1057/ori.2009.10 jester, i., & vera, m. de. (2012). an assessment of micro, small and medium enterprises that underwent up issi’s integrated plant surverys for the periode 2006 to 2011. procedia economics and finance, 4, 350–364). doi: https://doi.org/10.1016/s22125671(12)00349-8 luo, m., al-shami, s. a. h., & mansor, n. b. (2019). innovation: the key factor of competitiveness for chinese smes. opcion, año, 35(20), 2899–2921. marcarelli, g., & squillante, m. (2019). a group-ahp-based approach for selecting the best public tender. soft computing, 1–8. doi: https://doi.org/10.1007/s00500-019-044791 maulina, e., & fordian, d. (2018). analysis of restricting factors of small medium enterprises in bandung, west java indonesia. review of integrative business and economics research, 7(3), 297–307. mu, e., & pereyra-rojas, m. (2018). practical decision making using super decisions v3. an introduction to the analytic hierarchy process. pittsburgh: springer. my, p. t. (2020). factors influencing on human resources development in smes service enterprises in industry 4.0: the case of thai nguyen province. international journal of https://doi.org/10.1016/s2212-5671(12)00323-1 https://doi.org/10.1016/s2212-5671(12)00323-1 https://doi.org/10.1016/s2212-5671(12)00349-8 https://doi.org/10.1016/s2212-5671(12)00349-8 ijahp article: raco, krejci, ohoitimur, raton, adrian, rottie, soputan, sumual, ngenget sumual, sumakud/priority sector of small and medium enterprises using ahp: a case study of yamaru enterprise international journal of the analytic hierarchy process 238 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.862 business, economics and management, 7(3), 166–173. doi: https://doi.org/10.18488/journal.62.2020.73.166.173 padmore, m., taylor, n., & frecknall-hughes, j. (2006). marketing in small and medium sized enterprises. international journal of entreprenerial behaviour, 12(6), 361–387. doi: https://doi.org/10.1108/13552550610710153 pawitan, g. (2012). characteristics of small medium manufacturing industries in the era of acfta : case study from west java. procedia economic and finance, 4, 130–139) doi: https://doi.org/10.1016/s2212-5671(12)00328-0 president of indonesia (2008). law of the republic of indonesia on small and medium enterprises, pub. l. no. 20, indonesia. raco, j., ohoitimur, j., krejci, j. v, raton, y., rottie, r., paseru, d., … rachmadi, r. (2020). the dominant factor of lecturers’ research productivity usingthe ahp: case study of catholic university of de la salle manado. international journal of analytic hierarchy process, 12(3), 546–564. doi: https://doi.org/10.13033/ijahp.v12i3.819 razak, d. a., abdullah, m. ., & ersoy, a. (2018). small medium enterprises (smes) in turkey and malaysia: a comparative discussion on issues and challenges. international journal of business economics and law, 15(3), 1–10. rojas-lema, x., pumisacho, v., alfaro-saiz, j., & garcia, d. (2019). evaluating management practices in horizontal cooperation smes networks : the ecuadorian context. gestao & producao, 26(2), 1–13. doi: https://doi.org/10.1590/0104-530x-408319 saaty, r. . (1987). the analytic hierarcy process: what and how it is used. math modelling, 9(3), 161–176. saaty, t. l. (2013). on the measurement of intangibles. a principal eigenvector approach to relative measurement derived from paired comparisons. notices of the american mathematical society, 60(2), 192–208. saaty, t. l. (2008). relative measurement and its generalization in decision making. why pairwise comparisons are central in mathematics for the measurement of intangible factors. the analytic hierarchy/network process. racsam, 102(2), 251– 318. sunardi, o., widyarini, m., & hidajat, j. (2012). the impact of sales forces training program to employees behaviour styles (a quasi-experimental case study in a medium sized enterprise). procedia economics and finance, 4, 264–273). doi: https://doi.org/10.1016/s2212-5671(12)00341-3 suroso, a., anggraeni, a., & andriyansah. (2017). optimizing smes’ business performance through human capital management. european research studies journal, xx(january), 588–599. https://doi.org/10.1016/s2212-5671(12)00328-0 https://doi.org/10.1590/0104-530x-4083-19 https://doi.org/10.1590/0104-530x-4083-19 ijahp article: raco, krejci, ohoitimur, raton, adrian, rottie, soputan, sumual, ngenget sumual, sumakud/priority sector of small and medium enterprises using ahp: a case study of yamaru enterprise international journal of the analytic hierarchy process 239 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.862 surya, b., menne, f., sabhan, h., suriani, s., abubakar, h., & idris, m. (2021). economic growth, increasing productivity of smes, and open innovation. journal of open innovation: technology, market, and complexity, 7(20), 1–37. doi: https://doi.org/https://doi.org/10.3390/joitmc7010020 voca, z., & havolli, y. (2019). the impact of human resources development on small and medium enterprises (smes) performance: the case of kosovo. journal of economic and management sciences, 2(2), 45–54. doi: https://doi.org/https://doi.org/10.30560/jems.v2n2p45 vrana, s, a. (2008). evaluating the knowledge , relevance and experience of expert decision makers utilizing the fuzzy-ahp. agricultural economics, 54(11), 529–535. doi: https://doi.org/10.17221/264-agricecon ijahp article: tramarico, marins, urbina, salomon/benefits assessment of training on supply chain management international journal of the analytic hierarchy process 240 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.272 benefits assessment of training on supply chain management claudemir tramarico sao paulo state university (unesp) sao paulo, brazil claudemir.leif@terra.com.br fernando marins sao paulo state university (unesp) sao paulo, brazil fmarins@feg.unesp.br ligia urbina technological institute of aeronautics brazil ligia@ita.br valerio salomon * sao paulo state university (unesp) sao paulo, brazil salomon@feg.unesp.br abstract supply chain management (scm) is a critical factor in the current global scenario. this organizational capability has a recent knowledge base, which is being accumulated, validated, and certified by groups like the association for operations management (apics). therefore, training in scm has been growing as one of the most convenient ways of becoming “certified in production and inventory management” (cpim) from apics. companies all over the world have invested in scm training; however, some companies have conditioned the continuity of their training programs to the benefits assessment. this paper contributes by proposing an evaluation model for specific program training on scm. this model was applied in a global chemical company, which allowed capturing its impact on organizational and individual competencies, as well as on the core competencies. the proposed model includes the analytic hierarchy process (ahp) and concepts in the scm literature. the main result revealed by this research is that an scm training based on apics cpim is really perceived as beneficial, in * acknowledgment: for the research presented in this work, authors received financial support from capes foundation (phd grant) and sao paulo state research foundation (fapesp grant no. 2013/035257). mailto:claudemir.leif@terra.com.br mailto:fmarins@feg.unesp.br mailto:ligia@ita.br mailto:salomon@feg.unesp.br ijahp article: tramarico, marins, urbina, salomon/benefits assessment of training on supply chain management international journal of the analytic hierarchy process 241 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.272 individual or organizational terms, for a real-world company. therefore, this company should be confident that its scm training program is improving and strengthening its core competencies. keywords: analytic hierarchy process; supply chain management; training 1. introduction supply chain management (scm) is a critical factor in the current global scenario (hult, ketchen & arrelti, 2007). this organizational capability has a recent knowledge base which is being accumulated, validated, and certified by groups like association for operations management (known as apics, formerly american production and inventory control society) (coriat & dosi, 2002). scm is defined as the “design, planning, execution, control, and monitoring of supply chain activities with the objective of creating net value, building a competitive infrastructure, leveraging worldwide logistics, synchronizing supply with demand and measuring performance globally” (blackstone, 2013). scm capability encompasses multiple functions, which need to be learned over time by organizations. training provides the opportunity to learn and accumulate these scm capabilities that have to be mastered. these capabilities are fundamental to engage and upgrade in global value networks, which coordinate “…what is to be produced, how it is to be produced, and […] how the flow of product along the chain is to be handled” (humphrey & schmitz, 2002). in this context, it becomes relevant to obtain scm certification (tan, 2001). therefore, training in scm has been growing as one of the most convenient ways of becoming “certified in production and inventory management” (cpim) from apics. also scm training and cpim contribute to the development of the field of terminology concepts and strategies related to scm: demand management, master production scheduling, materials planning, capacity management, sales and operations planning, production environments and process, purchasing, physical distribution, performance measures, supplier relationships, lean and just-in-time, quality systems and continuous improvement (apics, 2015). since scm certification is considered a strategic advantage in the global economy most organizations have invested in the development of scm capabilities through training because it provides a broad perspective of the whole supply chain, its functions, its relations, and offers knowledge to understand how the overall supply chain integrates (lummus, 2007; apics, 2015). in fact, investment in training is also expected to enable, for instance the development of the employees’ base of knowledge and skills, which is related to the improvement and mastering of scm functions; the education of team members who interact with or support supply chain activities, helping them to increase efficiency and generate ideas for improvements; and the adoption and effective use of new information technologies; the development and improvement of scm practices that promote effective supply chain management in dynamic environments (apics, 2015; tracey & smith-doerflein, 2001). companies all over the world have invested in scm training in order to obtain certification (cpim) from apics. however, some companies have conditioned the continuity of their training programs to the benefits assessment. ijahp article: tramarico, marins, urbina, salomon/benefits assessment of training on supply chain management international journal of the analytic hierarchy process 242 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.272 most traditional methodologies for assessing the investment returns for training initiatives utilize very aggregated financial and non-financial measures which do not satisfy the company´s need for more detailed information about the impacts of investments on their base of resource, capabilities and competencies (satiman, abu et al., 2015; bukowitz, williams et al., 2004). some related approaches showed the relationship between higher level scm practices, as well as green scm practices that have been learned at some point in time and performance (zhu & sarkis, 2004; li, ragu-nathan, et al., 2006). in this direction, cheung, myers and mentzer (2010) presented evidence that, “for both buyers and suppliers, relationship learning is a critical, strategic component of relationship value in cross-border exchange”. in a closer approach, because they dealt with scm training focusing on the learning dimension, manyathi & niyimbanira (2014) attributed problems detected at the scm level to the need for reinforcing their scm training program, utilizing learning theories as guidance in its design, development and implementation, so as to increase workforce competence. thus, a research opportunity exists to develop an evaluation model with a strong influence of the resource and capabilities view of the firm (barney, 1991; barney, wright & ketchen, 2001; coriat & dosi, 2002, prahalad & hamel, 1990; fleury & fleury, 2001). at the company, evaluation of training on scm was based on the participant’s perception, but the benefits detected do not emerge from any structured model. thus, an evaluation approach was configured, considering the benefits suggested by the association for operation management, which includes a relevant set of individual and organizational benefits (apics, 2015). nonetheless, it was considered that this evaluation tool could not easily account for the main benefits that the company’s decision makers were expecting from scm training. indeed, their expectations were associated with the contribution that scm training could make for the development of the internal resources and capabilities of the company. therefore, companies wanted to see measures of the impacts from a particular training program on individual, organizational, and core competencies. in their view, an scm training program should develop individual competencies, improving the trainee’s ability to apply his knowledge, skills, abilities and other characteristics that are required to perform scm functions, activities and practices (draganidis & mentzas, 2006; jackson & schuler, 2003). also, according to the company, an scm training program should develop organizational competencies or capabilities, which could be seen as “ensembles of skills of individual members of the organization and, at the same time, directly organization-embodied elements of knowledge, routines etc. that represent organizational know how about scm functions and its ability to perform the coordination, integration and orchestration of the set of required tasks, utilizing organizational resources, for the purpose of achieving business results.” (coriat & dosi, 2002; helfat & peteraf, 2003; lebortef, 1998; fleury & fleury, 2004). lastly, decision makers of the company were expecting that investments in scm training would have an impact on the company’s core competencies, which are those competencies that the company leverages in order to achieve a competitive advantage, and which in turn are supported by other organizational competencies, such as scm (prahalad & hamel, 1990; fleury & fleury, 2004). it should be noted that organizational capabilities and core competencies are not easy to assess, because they “involve bundles of routines, which are strongly tacit in nature [… ijahp article: tramarico, marins, urbina, salomon/benefits assessment of training on supply chain management international journal of the analytic hierarchy process 243 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.272 and] a collective dimension, [which is] irreducible to the sum of individual skills” (coriat & dosi, 2002). however, it can be argued that gains and developments in such competencies could be evaluated, because it is possible to perceive them, even when they are tacit, since they reflect the accumulated knowledge. thus, such improvement in the scm knowledge base of the company, arising from training is reflected, mainly, in human resources as gains in individual competencies. also, the knowledge accumulates and gets embedded in organizational systems, which are seen as “management and organizational routines, procedures, instructions, documentation, […] in processes and products and services production flows, and in the ways of performing certain activities in organizations” (figueiredo, 2005). lastly, it can be argued that gains in individual and organizational competencies are expected to be reflected in core competencies (fleury &fleury, 2004). in that context, this paper contributes by proposing an evaluation model for specific program training on scm. this model was applied in a global chemical company, which allowed capturing its impact on organizational and individual competencies, as well as on the core competencies. it should be pointed out that research provides a better understanding of the relationship between scm training and a company’s core competencies. the proposed procedure includes applying the analytic hierarchy process (ahp) in the context of a global chemical corporation that was interested in assessing its scm training program. the interviewed sample contained trained employees and their managers. the research methods adopted in this paper are mathematical modelling and surveying. section 2 introduces a literature review, mainly with concepts on scm, and training assessment. section 3 describes ahp in detail. in section 4, the scm program training assessment of a global chemical company is presented, and section 5 concludes and suggests future research directions. 2. literature review 2.1 supply chain management scm has become strategically important to the business process. the literature review covered 22 relevant papers from science-journals categorized into seven topics discussed in this section:  supplier selection  collaborative supply network  distribution network  green supply chain  purchasing strategy  supply chain integration  value chain papers focused on supplier selection used various criteria to evaluate, including price and delivery performance. chan and kumar (2007) presented the fuzzy and ahp as an analytical approach i.e. the combination of the fuzzy set theory and the ahp for global supplier selection which is the same procedure used by chan et al. (2008). ertay et al. ijahp article: tramarico, marins, urbina, salomon/benefits assessment of training on supply chain management international journal of the analytic hierarchy process 244 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.272 (2011) used electre iii to evaluate and classify performance of suppliers. saen (2007) and sevkli et al. (2007) performed the evaluation process of supplier selection incorporating ahp and data envelopment analysis (dea). ha and krishnan (2008) added the neural network in supplier selection. the review considered 11 papers dealing with supplier selection using ahp or ahp combined with other methods. the collaboration between companies of another supply chain is known as horizontal collaboration. naesens, gelders and pintelon (2009) provided a comprehensive framework for a horizontal collaborative initiative with resource pooling in inventories using ahp. lin and ho (2014) proposed an integration of scm in the hospital industry of taiwan. the distribution network objective is to allocate a number of points of consumption and supply. ho and emrouznejad (2009) proposed to evaluate the performance of warehousing using ahp and goal programming. sharma, moon and bae (2008) illustrated a distribution network using ahp based on product characteristics. the green supply chain is an approach for improving environmental performance of processes including supply base environmental performance management and reverse logistics. hsu and hu (2008) focused on the green supply chain for investigating the consistency and priority approaches on processes implementation in the taiwanese electronic industry. lu, wu and kuo (2007) presented a procedure to evaluate cooperating with green supplies using fuzzy and ahp. wang et al. (2012) proposed a model using the same method to analyze the risks of different alternatives and green practice in the fashion supply chain. the purchasing strategy is based on business strategy alignment. drake and lee (2008) investigated the prioritization of components in the business strategy of a south korean elevator manufacturer using ahp. the supply chain integration is an advantage through scm that operates by customer demand. palma-mendoza (2014) proposed supply chain re-design to support supply chain integration with supply chain operations reference model (scor) and ahp. the value chain provides products and services to customer’s needs. rabelo et al. (2007) presented a framework to integrate scor, ahp and discreteevent simulation in services and manufacturing in a case study manufacturing in southeast asia. the result was effective and practical support for a senior executive’s decisions. the details about the topics and methods of the literature reviewed can be seen in table1. ijahp article: tramarico, marins, urbina, salomon/benefits assessment of training on supply chain management international journal of the analytic hierarchy process 245 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.272 table 1 topic and method no. authorship year s u p p li e r s e le c ti o n c o ll a b o ra ti v e s u p p ly d is tr ib u ti o n n e tw o rk g re e n s u p p ly c h a in p u rc h a si n g s tr a te g y s u p p ly c h a in i n te g ra ti o n v a lu e c h a in a h p d e a d is c re te -e v e n t s im u la ti o n e l e c t r e i ii f u z z y l o g ic g o a l p ro g ra m m in g n e u ra l n e tw o rk 1 chan, f.t.s. and kumar, n. 2007 √ √ √ 2 chan, f.t.s. et al. 2008 √ √ √ 3 chan, f.t.s. and chan, h. k. 2010 √ √ 4 drake, p.r. and lee, d.m. 2009 √ √ 5 erdem, a.s. and goecen, e. 2012 √ √ √ 6 ertay, t. et al. 2011 √ √ √ √ 7 ha, s.h. and krishnan, r. 2008 √ √ √ √ 8 ho, w. and emrouznejad, a. 2009 √ √ √ 9 ho, w. et al. 2011 √ √ 10 hsu, c.w. and hu, a.h. 2008 √ √ √ 11 labib, a.w 2011 √ √ 12 levary, r.r. 2008 √ √ 13 lin, r.h. and ho, p.y. 2014 √ √ 14 lu, l.y.y et al. 2007 √ √ √ 15 naesens, k. et al. 2009 √ √ 16 opasanon, s. and lertsanti, p. 2013 √ √ 17 palma, m. and jaime, a. 2014 √ √ 18 rabelo, l. et al. 2007 √ √ √ 19 saen, r.f. 2007 √ √ √ 20 sevkli, m. et al. 2007 √ √ √ 21 sharma, m.j. et al. 2008 √ √ 22 wang, x. et al. 2012 √ √ √ topic method 2.2 training assessment training in scm could also have a pervasive impact on core competencies. thus, a comprehensive training assessment model should take into account the benefits related to the improvement of core, organizational and individual competencies. prahalad and hamel (1990) defined core competencies as “the collective learning in the organization, especially how to coordinate diverse production skills and integrate multiple streams of technologies”. these competencies do differentiate the company from its competitors creating a competitive differentiation that enchants clients, and propitiates sustainable advantages. ijahp article: tramarico, marins, urbina, salomon/benefits assessment of training on supply chain management international journal of the analytic hierarchy process 246 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.272 apics suggests that scm training could be evaluated using two sets of benefits as individual benefits and organizational benefits (apics, 2015). the literature review provides a description of the benefits (table 2). table 2 individual and organizational benefits benefits description author individual individual recognition recognize the individual by the means of its characteristics, valorizing or praising its attributes, conquers, services or capabilities. oxford dictionaries (2015) improve scm knowledge the adoption and integration of what people know, how well people communicate what they know and how quickly people learn new things, can give a company a sustainable competitive advantage. van zyl (2003) credential recognition the credential allows certified individuals to demonstrate knowledge in the professional activities of scm. lummus (2007) validate of knowledge and abilities to assess a worker's knowledge and abilities must perceive the status characteristic as relevant to an organizational task. in this way, supply chain managers need broad and deep knowledge and abilities. treem (2013), gammelgaard & larson (2001) organizational common understanding of vocabulary and processes common understanding the term for a given concept and that terminology would be consistent in the company. lummus (2007) use of best practices methods that have been found to be effective mean for success in accomplishing goals, and that can be used, or adapted for use gilbert (2014), bulkeley (2006) improve company performance the capability to share explicit and tacit knowledge for the company enables competitive performance. schoenherr, griffith & chandra (2014) proven knowledge and organizational skills focus on support an individual be more effective to work and operate better in groups and in organizational. khadivar et al. (2007) there are organizational competencies which are specific competencies by business area such as design, marketing, sales, production and logistics. some of these are core competencies of the organization since they are reasons for the company’s survival, while others serve as supporters (fleury & fleury, 2001). individual competencies are the skills and capabilities in every person which reflect both the intrinsic and acquired knowledge and personal characteristics (fleury & fleury, 2004). ijahp article: tramarico, marins, urbina, salomon/benefits assessment of training on supply chain management international journal of the analytic hierarchy process 247 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.272 the cpim program is internationally recognized in all branches of industry and organized into five modules (apics, 2015):  basics of supply chain management the basic concepts in managing the complete flow of materials in a supply chain from suppliers to customers are covered in the basics module. covers manufacturing, distribution, service, and retail industries.  master planning of resources explore and be able to apply the principles of demand management, sales and operations planning, master scheduling, and distribution planning, and to identify conditions that require action. evaluates knowledge of both supply and demand planning for midto long-term independent demand.  detailed scheduling and planning acquire a working knowledge of the tools and techniques for planning of inventory, including planning techniques such as mrp, crp, lean, toc, and projects.  execution and control of operations learn to translate plans into operational activities and define and apply techniques in the operations field. comparing actual output to plans and taking appropriate corrective actions  strategic management of resources move your learning to the next level through the smr module that includes higher-level thinking or strategic planning and implementation of operations. 3. method 3.1 analytical hierarchy process training in scm can be assessed using qualitative criteria of various scm processes. the ahp method application uses hierarchy models (saaty, 2010a). the ahp application is often divided into two phases of the decision process: the problem structuring and the elicitation of priorities through pairwise comparisons (ishizaka & nemery, 2013). the characteristics of the ahp are the adoption of the well-known fundamental scale proposed by saaty (2010b) to consult experts about the problem's alternatives and criteria, generating a pairwise comparison matrix a, in the sequence, using linear algebra concepts, as the eigenvector (w), and eigenvalue (λmax), it is possible to get their relative priorities. the ahp priorities are obtained with an application of the perronfrobenius theorem as presented in (1) (saaty, 1977): a w = λmax w (1) the consistency among the comparisons is an important propriety for a. if a has consistent comparisons, then aij = wi/wj, for i, j = 1, 2,… n, where n is the order of a, and this way, aij = aik akj. besides that, if a is a consistent matrix, then λmax = n. the consistency index, µ, calculated by (2), is a measure of the distance between λmax and n: µ = (λmax – n)/(n – 1) (2) as pointed out by saaty (2010a), if µ is lower than 0.10 the matrix a is consistent. otherwise, a review on the comparisons may be necessary. ijahp article: tramarico, marins, urbina, salomon/benefits assessment of training on supply chain management international journal of the analytic hierarchy process 248 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.272 there is also a possibility of using ratings, also known as “absolute measures”, in the ahp application, i.e. each alternative is compared with many other alternatives, while ratings compare each alternative with an ideal one (saaty, 2006). the level of performance corresponding to attributes in linguistic scales varies from “poor” to “good”. table 3 presents five degrees of quality for training assessment. table 3 five-level degrees of quality level degree of quality 1 poor 2 between poor and fair 3 fair 4 between fair and good 5 good 4. evaluation of scm training the research presented in this paper was conducted in a global chemical company. employees and their managers from chemical plants located in the state of sao paulo, brazil, were surveyed. the researched company has around 1,000 employees (including managers) in brazil, and is one of the top 10 companies in the chemical and petrochemical branches. the company’s core competencies were defined as: drive innovation, collaborate for achievement, drive sustainable solutions, act with entrepreneurial drive, and demonstrate customer focus. a training program was implemented one year ago to prepare for apics cpim. more than 100 employees have participated in the training program. frequently asked questions regarding the scm training gains are: “does the scm training program deliver individual benefits?”, “does the training program deliver organizational benefits?”, and “does the training program contribute to the strengthening of the company’s core competencies?” figure 1 presents the company’s core competencies that are an integral part of learning and development programs. ijahp article: tramarico, marins, urbina, salomon/benefits assessment of training on supply chain management international journal of the analytic hierarchy process 249 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.272 core competency explanation drive innovation contribute creative solutions and foster the spirit of innovation collaborate for achievement seek and act upon value-adding opportunities for collaboration within and across units drive sustainable solutions decide and act persistently to achieve economic, environmental and societal benefits for the long term act with entrepreneurial drive take ownership for results, overcome obstacles and drive change demonstrate customer focus act to deliver customer value while achieving profitable business results figure 1. company’s core competencies a main step of the ahp is the decomposition of the problem into a hierarchy, by defining the objectives or task’s goals, criteria and alternatives. the goal is “assess scm training’, and the company’s core competencies (figure 1) are the criteria in the second level. figure 2 presents the hierarchy model applied in the assessment of scm training. figure 2. hierarchy model for training assessment data was collected from january to march 2014. 174 company’s employees and managers were surveyed by filling out forms evaluating the training benefit according the degree of quality they perceived for each one of the company’s core competencies. those individuals questioned included supply chain managers, supply chain consultant, supply planner, demand planner, detailed scheduler, master scheduler, materials planner and customer service representative. the profile of the respondents is consistent with our research objectives. the priorities presented in table 4 were obtained by normalizing components of w. as for “poor” we have zero priority, this degree was not compared. the consistency index for the comparison matrix presented in table 4 is around 0.01. assess scm training drive innovation (di) collaborate for achievement (ca) drive sustainable solutions (ds) act with entrepreneurial drive (ae) demonstrate customer focus (cf) ijahp article: tramarico, marins, urbina, salomon/benefits assessment of training on supply chain management international journal of the analytic hierarchy process 250 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.272 table 4 pairwise comparisons of degrees of quality g bfg f bpf priority good (g) 1 2 3 4 1 between fair and good (bfg) 1/2 1 2 3 0.59 fair (f) 1/3 1/2 1 2 0.34 between poor and fair (bpf) 1/4 1/3 1/2 1 0.20 table 5 presents the survey’s results. for instance, 26 employees or managers judged that scm training was “good” for di. table 5 assessment from employees and managers degree of quality (priority) di ca ds ae cf good (1) 26 30 32 44 56 between fair and good (0.59) 81 65 63 73 69 fair (0.34) 51 56 53 36 41 between poor and fair (0.20) 15 21 22 16 5 poor (0) 1 2 4 5 3 a priority was computed for a core competency by multiplying the score by the degree of quality divided by total number of employees. for instance, 26 people assessed di as good, then (26 × 1)/174 = 0.15; 81 assessed di as bfg, then (81 × 0.59)/174 = 0.27; 51 assessed di as f, then (51 × 0.34)/174 = 0.10; 15 assessed di as bpf, then (15 × 0.29)/174 = 0.02. summing all these assessments, 0.15, 0.27, 0.10 and 0.02 the aggregate priority for di is 0.54. the same procedure was performed to ca, ds, ae and cf. table 6 presents the aggregate priorities from employees and managers assessments. table 6 aggregate priority degree of quality di ca ds ae cf good 0.15 0.17 0.18 0.25 0.32 between fair and good 0.27 0.22 0.21 0.25 0.23 fair 0.10 0.11 0.10 0.07 0.08 between poor and fair 0.02 0.02 0.03 0.02 0.01 poor 0.00 0.00 0.00 0.00 0.00 aggregate priority 0.54 0.53 0.53 0.59 0.64 the assessment was performed by all employees and managers who attended the scm training. the aggregate priority (table 6) for di criterion was around 0.54, representing the degree of quality between fair and good (table 4). di is a core competency to contribute to creative and innovative solutions (figure 1). for ca and ds the results ijahp article: tramarico, marins, urbina, salomon/benefits assessment of training on supply chain management international journal of the analytic hierarchy process 251 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.272 were not qualitatively different being around 0.53, which represents the degree of quality between fair and good. ca and ds are core competencies to seek and act upon valueadding opportunities for collaboration. the ae criterion had an aggregated priority equal to 0.59, representing the degree of quality between fair and good. ae is a core competency to take ownership for results and drive changes. the cf criterion obtained the highest priority of 0.64, representing the degree of quality good. cf is a core competency to contribute to deliver customer value while achieving profitable business results it was observed in the assessment that employees and managers may have made a significant evaluation and agreed that the scm training program delivered individual and organizational benefits. it was also clear that the assessment allowed the evaluation of the gains from the scm training program on the company’s core competencies. 5. conclusions the main result revealed by this research is that scm training is really perceived as beneficial, in individual or organizational terms, to a real-world company. therefore, the company should be confident in the scm training program to improve and strengthen its core competencies. the assessment of scm training and competencies ensures that the company explores the details, and can be used to support the scm company’s strategy mainly linked to customer focus and entrepreneurial drive. this work reveals that apics cpim training enables and prepares employees and managers to make right choices for their organization. the model presented in section 4 can be improved by prioritizing core competencies one against another. a long term assessment based on tangible aspects of training effectiveness and efficiency should be considered as one of the next steps in research. additionally, the proposed assessment presents possibilities for improvement using disaggregate priorities on individual and organizational alternatives as next steps. a new approach based on benefits, opportunities, costs and risks is suggested as future research. the model and results of a benefit assessment of training on scm were primarily developed and obtained in a global chemical corporation. even though only a plant located in the brazilian state of sao paulo was surveyed, the model and results can be adapted to other locations or industry branches, mutatis mutandis. ijahp article: tramarico, marins, urbina, salomon/benefits assessment of training on supply chain management international journal of the analytic hierarchy process 252 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.272 references apics. (2015). certified in production and inventory management. retrieved from http://www.apics.org/cpim (accessed may 27, 2015). barney, j. b. (1991). firm resources and sustained competitive advantage. journal of management, 17(1), 99–120. barney, j. b., wright, m., & ketchen jr, d. j. (2001). the resource-based view of the firm: ten years after 1991. journal of management, 27(6), 625–641. blackstone, j. h. jr. (ed.) (2013). apics dictionary. 14 th edition. chicago: apics. bukowitz, w. r., williams, r. l., & mactas, e. s. (2004). human capital measurement. research technology management, 47(3), 43–49. bulkeley, h. (2006). urban sustainability: learning from best practice? environment and planning a. 38(6), 1029. chan, f. t. s., & kumar, n. (2007). global supplier development considering risk factors using fuzzy extended ahp-based approach. omega, 35(4), 417–431. chan, f. t. s., kumar, n., tiwari, m. k., lau, h. c. w., & choy, k. l. (2008). global supplier selection: a fuzzy-ahp approach. international journal of production research, 46(14), 3825–3857. cheung, m. s., myers, m. b., & mentzer, j. t. (2010). does relationship learning lead to relationship value? a cross-national supply chain investigation. journal of operations management, 28(6), 472–487. coriat, b., & dosi, g. (2002). the nature and accumulation of organizational competences/capabilities. revista brasileira de inovação, 1(2), 275–326. draganidis, f., & mentzas, g. (2006). competency based management: a review of systems and approaches. information management & computer security, 14(1), 51–64. drake, p. r., & lee, d. m. (2008). component prioritisation for strategic purchasing and the case study of a south korean elevator manufacturer. the international journal of advanced manufacturing technology, 43(9-10), 883–895. ertay, t., kahveci, a., & tabanlı, r. m. (2011). an integrated multi-criteria group decision-making approach to efficient supplier selection and clustering using fuzzy preference relations. international journal of computer integrated manufacturing, 24(12), 1152–1167. ha, s., & krishnan, r. (2008). a hybrid approach to supplier selection for the maintenance of a competitive supply chain. expert systems with applications, 34(2), 1303–1311. ijahp article: tramarico, marins, urbina, salomon/benefits assessment of training on supply chain management international journal of the analytic hierarchy process 253 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.272 helfat, c. e., & peteraf, m. a. (2003). the dynamic resource‐based view: capability lifecycles. strategic management journal, 24(10), 997–1010. ho, w., & emrouznejad, a. (2009). multi-criteria logistics distribution network design using sas/or. expert systems with applications, 36(3), 7288–7298. hult, g.t.m., ketchen, d.j.k. , & arrelti, m. (2007). strategic supply chain management: improving performance through a culture of competitiveness and knowledge development. strategic. management. journal, 28, 1035-1052. humphrey, j., & schmitz, h. (2002). how does insertion in global value chains affect upgrading in industrial clusters?. regional studies, 36(9), 1017–1027. figueiredo,p. n. (2005). technological accumulation and industrial innovation. concepts, measurement and evidence in brazil. são paulo em perspectiva, 19(1), 54–69. (in portuguese) fleury, m.t.l., & fleury, a. (2001). building the concept of competence. revista administração contemporânea, 5, 183–196. (in portuguese) fleury, m.t.l., & fleury, a. (2004). aligning strategy and skills. revista de administração de empresas, 44(1), 44–57. (in portuguese) gammelgaard, b. & larson, p. d. (2001). logistics skills and competencies for supply chain management. journal of business logistics, 22(2), 27–50. gilbert, l. (2014). social justice and the" green" city urbe. revista brasileira de gestão urbana, 6(2), 158–169. ishizaka, a., & nemery, p. (2013). multi-criteria decision analysis: methods and software. west sussex: john wiley & sons. jackson, s. e., & schuler, r. s. (2003). managing human resources through strategic partnerships. (8th ed.). mason, oh: south-western pub. khadivar, a., zadeh, a. r., khani, m., & jalali, s. m. j. (2007, december). a conceptual model for knowledge flow in supply chain. in industrial engineering and engineering management, 2007 ieee international conference on (pp. 352-356). ieee. leboterf, g. (1998). evaluer les compe´tences, quels jugements? quels crite`res? quelles instances? la compe´tence au travail, 135(2), 143–51. li, s., ragu-nathan, b., ragu-nathan, t. s., & rao, s. s. (2006). the impact of supply chain management practices on competitive advantage and organizational performance. omega, 34(2), 107–124. ijahp article: tramarico, marins, urbina, salomon/benefits assessment of training on supply chain management international journal of the analytic hierarchy process 254 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.272 lu, l. y. y., wu, c. h., & kuo, t.-c. (2007). environmental principles applicable to green supplier evaluation by using multi-objective decision analysis. international journal of production research, 45(18-19), 4317–4331. lummus, r.r. (2007). the role of apics in professionalizing operations management. journal of operations management, 25(2), 336–345. manyathi, s., & niyimbanira, f. (2014). a theoretical evaluation of the role of training for supply chain management employees of the department of provincial treasury: case of kwazulu-natal, south africa. mediterranean journal of social sciences, 5(7), 231. naesens, k., gelders, l., & pintelon, l. (2009). a swift response framework for measuring the strategic fit for a horizontal collaborative initiative. international journal of production economics, 121(2), 550–561. oxford dictionaries (2015). oxford university press. retrieved from http://www.oxforddictionaries.com (accessed may 27, 2015) palma-mendoza, j. a. (2014). analytical hierarchy process and scor model to support supply chain re-design. international journal of information management, 34(5), 634– 638. prahalad, c., & hamel, g. (1990). the core competence of the corporation. harvard business review, 68, 79–91. rabelo, l., eskandari, h., shaalan, t., & helal, m. (2007). value chain analysis using hybrid simulation and ahp. international journal of production economics, 105(2), 536–547. saaty, t.l. (1977). a scaling method for priorities in hierarchical structures. journal of mathematical psychology, 15, 234–281. saaty, t.l., & ozdemir, m. (2003). negative priorities in the analytic hierarchy process. mathematical and computer modelling, 37, 1063–1075. saaty, t.l. (2006). rank from comparisons and from ratings in the analytic hierarchy/network processes. european journal of operational research, 168, 557–570. saaty, t.l. (2010a). principia mathematica decernendi: mathematical principles of decision-making. pittsburgh: rws publications. saaty, t.l. (2010b). who won the winter 2010 olympics? a quest into priorities. journal of multi-criteria decision analysis, 17 (1–2), 25–36. saen, r. f. (2007). a new mathematical approach for suppliers selection: accounting for non-homogeneity is important. applied mathematics and computation, 185(1), 84–95. ijahp article: tramarico, marins, urbina, salomon/benefits assessment of training on supply chain management international journal of the analytic hierarchy process 255 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.272 satiman, l. h., abu mansor, n. n., & zulkifli, n. (2015). return on investment (roi) training evaluation in malaysian smes: factors influencing the adoption process. development and learning in organizations, 29(2), 18–21. sevkli, m., lenny koh, s. c., zaim, s., demirbag, m., & tatoglu, e. (2007). an application of data envelopment analytic hierarchy process for supplier selection: a case study of beko in turkey. international journal of production research, 45(9), 1973– 2003. sharma, m. j., moon, i., & bae, h. (2008). analytic hierarchy process to assess and optimize distribution network. applied mathematics and computation, 202(1), 256–265. schoenherr, t., griffith, d. a., & chandra, a. (2014). knowledge management in supply chains: the role of explicit and tacit knowledge. journal of business logistics, 35(2), 121–135. tan, k. c. (2001). a framework of supply chain management literature. european journal of purchasing & supply management, 7(1), 39–48. tracey, m., & smith-doerflein, k.a. (2001). supply chain management: what training professionals need to know. industrial and commercial training, 33(3), 99–104. treem, j. w. (2013).technology use as a status cue: the influences of mundane and novel technologies on knowledge assessments in organizations. journal of communication, 63, 1032–1053 van zyl, c. (2003). supply chain knowledge management adoption increases overall efficiency and competitiveness. south africa journal of information management, 5(4). doi: 10.4102/sajim.v5i4.381 wang, x., chan, h. k., yee, r. w. y., & diaz-rainey, i. (2012). a two-stage fuzzyahp model for risk assessment of implementing green initiatives in the fashion supply chain. international journal of production economics, 135(2), 595–606. zhu, q., & sarkis, j. (2004). relationships between operational practices and performance among early adopters of green supply chain management practices in chinese manufacturing enterprises. journal of operations management, 22(3), 265–289. microsoft word 4b_montevechi_ultrassonicsystem_vol2_issue1.docx ijahp article: montevechi, guimarães, de oliveira, friend / selection of ultrasonic scanning system in a private hospital in brazil international journal of the analytic hierarchy process 14 vol. 2, issue 1, 2010 issn 1936-6744 decision-making with multiple criteria in the selection of ultrasonic scanning system in a private hospital in brazil josé arnaldo barra montevechi federal university of itajubá (unifei) itajubá, mg, brazil e-mail: montevechi@unifei.edu.br italo freire guimarães federal university of itajubá (unifei) itajubá, mg, brazil e-mail: italofg@globo.com mona liza moura de oliveira federal university of itajubá (unifei) itajubá, mg, brazil e-mail: monaoli@yahoo.com.br jonathan daniel friend federal university of itajubá (unifei) itajubá, mg, brazil e-mail: daniel.friend1@gmail.com abstract this article presents the analytical hierarchy process (ahp) as an aiding approach in decision-making, applied to the technical evaluation of the acquisition process of ultrasonic device in a private hospital in brazil. some steps and principles of ahp, such as the complexity of hierarchical decomposition, comparative judgments, priorities synthesis, and sensitivity analysis show, themselves to be utilities for the success of the technical evaluation process, going from meeting the hospital’s necessity for transparency to auditable process documentation. the hospital’s management’s expectation of separating the hospital’s medical technical evaluation process from the commercial evaluation process was achieved through the application of ahp. despite the great number of possible variables for the ultra-sound’s technical evaluation, the decision-making team adapted itself easily and quickly to the employed method, contributing to the success of the work. the range of medical equipment installed in brazil presents geographic concentration, with a large percentage of the equipment remaining unused due to lack of maintenance or inadequate staff to operate them, and the quantity of determined image devices per habitant above the average of other developed countries. that reality leads to the waste of resources in the health sector and adverse effects to the population’s health. this confirms the pertinence of the instrument’s application in this work, and corresponding author ijahp article: montevechi, guimarães, de oliveira, friend / selection of ultrasonic scanning system in a private hospital in brazil international journal of the analytic hierarchy process 15 vol. 2, issue 1, 2010 issn 1936-6744 indicates the potential increase in its utilization in the planning and control of public and private resources in the health sector. key-words: health technology assessment, medical equipment, analytical hierarchy process 1. introduction healthcare costs and the increasing cost of healthcare are public interest concerns in many countries (lim et. al, 2009). the health sector suffers the effects of the world phenomenon called “medical inflation”, which consists of medical service’s tendency to increase its prices above general economic price indexes. uncontrolled technological incorporation is one of the principal factors in the phenomenon’s development. this is the consequence of some biomedical technology characteristics, such as their accumulative and non-substitutive character. for example, computed tomography (ct) scanners don’t substitute for conventional radiology (x-rays) devices, but are used in addition to them. furthermore, the intensive nature of the biomedical technology workforce contributes to this phenomenon of medical inflation, as it impedes or diminishes the economy of scale’s gain in work force cost. this problem is also exacerbated by the fact that biomedical technology is disseminated with great velocity and, being incorporated precociously, many times without proving its efficiency (mendes, 2006). developing countries are technologically dependent on developed countries, and import a great deal of medical equipment, especially high tech instruments. in the case of incorporating technologies without criteria, developing countries run the risk of importing systems designed for developed countries’ epidemiological reality, instead of creating solutions for their own countries’ problems. the lack of concrete actions for the medical-hospital equipment incorporation’s control and planning in brazil, as well as the unawareness of this equipment’s quantity and quality in the health network have led to a series of misrepresentations, such as high concentration of image equipment in closely-located sites, generating high complexity processing demands and high cost that are often unnecessary. for example, the city of belo horizonte in brazil has a population similar to paris, france; however, it has 8 times the installed ct scanners, despite dealing with endemic tropical diseases such as dengue, yellow fever and mansonic schistosomiasis, implying the inefficient utilization of the purchased equipment and lack of other, more necessary equipment (nepp, 2000). along with this scenario, there is a growing awareness of the relationship between the medical and the pharmaceutical industries. it can reach the ethical limit of marketing investments, materializing in the form of gifts to physicians and its effects on their professional conduct with patients, including the induction and prescription of a determined medicine or execution of a certain procedure. similar phenomenon occurs in the medical equipment industry (halperin, hutchinson & barrier, 2004; collins, 2006). therefore, the physicians responsible for the technical choice in the acquisition process can define the equipment to be acquired, basing the choice, principally, on their personal gains, in detriment to the interests of the hospital, which is purchasing the device. ramirez (2005) comments that a detailed evaluation in the moment of equipment acquisition is very important in order to avoid needlessly raising the cost of health services. nevertheless, in brazil for cultural reasons, the greatest preoccupation is concentrated in the execution of work and the medical equipment’s purchase, and not to processes such as rational purchase planning, receipt and acceptance protocols, training, and maintenance, among others. rob typewritten text http://dx.doi.org/10.13033/ijahp.v2i1.70 ijahp article: montevechi, guimarães, de oliveira, friend / selection of ultrasonic scanning system in a private hospital in brazil international journal of the analytic hierarchy process 16 vol. 2, issue 1, 2010 issn 1936-6744 in this context, support instruments for decision-making, such as the analytical hierarchy process (ahp), are indicated for the systemization of complex decisions in the health sector, with prioritization of projects and allocation of resources, among other uses, contributing to the decision’s rationalization and reduction of wastes (panerai & mohr, 1990). the present work describes the application of ahp in the technical evaluation process of medical device acquisition in a private brazilian hospital, with the purpose of contributing to the process’s rationalization and systemization and of generating auditable documentation of the criteria utilized to obtain the evaluated equipment’s technical ranking. this will permit a more rational approach in choosing the commercial proposal winner. the target support instrument for decision-making was brought into, unveiled and implanted in the hospital, which previously applied no formal or systemized methodology for the technical purchase evaluation of medical equipment. the absence of such an instrument had caused serious problems for this institution in the past, leading to the acquisition without criteria of very expensive imported medical equipment that contributed to the financial insolvency of the institution, which the public ministry considered closing. the institution’s managers searched for management tools which gave transparency to the medical equipment acquisition process, and permitted its audit, expecting that they could reduce ethical miscues in the evaluation process, and reduce the risk of repeating past errors. the methodology used in the work is an action-research in which an empirically-based qualitative and quantitative focus is conceived and carried out in strict association with an action or with a resolution of a collective problem and in which the problem’s researchers and participants are involved in a cooperative or participative manner (brandão, 1999). 2. diagnostic ultrasonic scanning system presently, the diagnostic ultrasonic exam is the most used diagnostic imaging method in clinical use, making up about 25% of all image studies realized in the world (forsbeg, 2004). it is one of the most rapidly growing means of image obtainment in the number of exams realized in the entire world. such growth, among other factors, is owed to the low cost of the exam, real time interactions and its apparent lack of adverse bioeffects (rumack, wilson & charboneau, 1999) and, also, for being a non-invasive method and not utilizing ionizing radiation, such as conventional radiology and computed tomography (andreassi, venneri and picano, 2007). typical ultrasonic equipment model can be acquired from many manufacturers offering devices that increase modularly in sophistication, resources and price, which can go from us$ 20,000 to more than us$ 300,000, depending on the configuration. in general, the ultrasound equipment possesses a modular design that permits future updates and upgrades, including specialized functions, such as doppler cardiac imaging and harmonic tissue imaging, (ecri, 2005). standard ultrasonic scanning system consists of an electrical pulse generation and transmission unit to excite the transducers, a reception and amplification unit for the captured signals, a control and processing unit to configure the emission and reception parameters, and a unit to visualize the process’s resulting image in one or more monitors. it sums up to this: a set of transducers, an interface with an operator, an energy source and some recording devices (ecri, 2005). ijahp article: montevechi, guimarães, de oliveira, friend / selection of ultrasonic scanning system in a private hospital in brazil international journal of the analytic hierarchy process 17 vol. 2, issue 1, 2010 issn 1936-6744 as a consequence of the device’s large market, the medical equipment’s industries that produce the ultrasonic scanning system have invested in the development of new technologies that increase the image’s diagnostic capacity. such technologies are put on the market under many different commercial names by the industries, as a marketing strategy to show competitive advantage over the competition, in spite of conceptually dealing with the same technical resources. this results in confusion for the purchaser and the final user. a multicriteria decision approach can rationalize this comparison and avoid waste and unnecessary expenditure. 3. the analytical hierarchy process the analytical hierarchy process (ahp) developed by mathematician thomas lorie saaty is the most used multi-criteria decision-making method probably due to its ease of application. saaty (1987) comments that the ahp theory was developed in 1972 to solve a specific contingency planning problem and later, in 1977, an application of greater visibility: the projection of future alternatives for the developing country of sudan. since then, the ideas developed progressively through its applications in various other projects. forman (2001) comments that the theory’s application in 1983 in computer program expert choice, caused a rapid growth of the number, the diversity and the complexity of ahp applications. thousands of articles about ahp application have been published in scientific journals (ho, xu and dey, 2010), many of which are related to the health area (lettieri and masella 2009; liberatore and nydick, 2008; wu, lin and chen, 2007; sloane et. al, 2003). ahp is perhaps the most used decision support method in the literature for selection of suppliers (van der rhee, verma and plaschka, 2009). ahp has been used in applications for governmental organizations and for private businesses, some of which institutionalized its use, such as the united states departments of defense and energy, and the xerox corporation (saaty, 1994). the pairwise judgments phase based in a scale of reason permits the decision-maker to use objective considerations (concrete facts and knowledge) and subjective considerations (intuition, creativity, personal experience and insight) in a logical path. the utilization of a scale of reason makes analysis of tangible and intangible variables possible. on the other hand, this phase incorporates redundancy of judgments of one element in relation to the other and vice-versa, which in turn reduces the errors of measurement or judgment and permits the production of the judgment’s consistency measurement. the method supports a certain level of inconsistency in the judgments elaborated by the decision-maker, as is with human beings. still, it supplies a measure of the judgment’s consistency, and a limit from which they should be reviewed (forman, 2001). 4. case study the mário penna foundation (fmp), headquartered in belo horizonte, is the largest provider of chemotherapy and radiotherapy services in the state of minas gerais, and one of the 10 largest oncologic centers in brazil. fmp maintains two hospital units in belo horizonte the mário penna hospital and the luxemburgo hospital. in 2006, the luxemburgo hospital, site of the application of this work, possessed 190 beds, of which 20 were in the icu – intensive care unit, 6 surgical rooms and the largest radiotherapy service in minas gerais. its annual production consists of 12,000 admittances, 5,300 surgeries, 18,000 chemotherapy sessions, 90,000 radiotherapy sessions and 290,000 laboratory exams. ijahp article: montevechi, guimarães, de oliveira, friend / selection of ultrasonic scanning system in a private hospital in brazil international journal of the analytic hierarchy process 18 vol. 2, issue 1, 2010 issn 1936-6744 luxemburgo hospital is a high complexity general hospital focusing in four areas: oncology, and its medical support specialties: cardiology, hemodynamic and neurosurgery. the hospital needed to purchase an ultrasonic scanner with color doppler and endovaginal and endorectal capabilities so that the institution remains a high complexity center, according to the legislation at the time. as the evaluation for the acquisition of new ultrasonic equipment began, a different purchasing structure was proposed to the fmp superintendent, with ahp application for the technical evaluation and ranking of the equipment under consideration. it was immediately accepted . the purpose was to avoid the institution’s previous experiences of acquisition without criteria of medical equipment, which contributed to the near financial insolvency of the institution, and to give transparency and an audit trail to the process. 4.1 purchase process structure the necessity to purchase a new ultrasonic scanner made the application of ahp possible in the technical evaluation process. such equipment possesses a great number of technical characteristics that must be evaluated and, according to ecri (2004), the brand and model alternatives offered by the principal manufacturers are very similar one to another, in relation to quality and performance. the evaluation technique was structured as one of six subprojects within the ultrasonic scanning system acquisition process. this paper shows the subproject of technical evaluation. the technical evaluation was the ahp application’s target, and was performed by the luxemburgo hospital ultrasonic service medical team. the process of technical evaluation consisted of the following roles and agents: 1. decision-makers: responsible for the ratification of the technical decision and assuming its consequences (gomes, gomes and almeida, 2002). in this work the role was exercised by two physician representatives of the luxemburgo hospital ultrasound service medical team. 2. facilitator: responsible for clarifying and modeling the process of evaluation, focusing the solution of the problem, coordinating the point of view of the decision-makers, keeping them motivated and highlighting the learning in the decision process. this agent must have experience in the decision process and maintain a neutral position in order to not intervene in the judgment of the decision-makers (gomes, gomes and almeida, 2002). that role was exercised by one of the paper’s authors. 3. analyst: responsible for aiding the decision-makers and the facilitator in the problem structure and in the identification of factors that influence the evolution, solution and configuration of the problem. that role was exercised by the person responsible for the purchasing department and the one responsible for the legal department. 4.2 the construction of the hierarchy a sketch of the initial hierarchy was assembled based on the literature’s data, supplied by ecri (2004); kolzer et. al, (2002), and free suggestions from the decision-makers and analysts. the first level of decision hierarchy, after the goal, was extracted from the suggestion of ecri (2004). it purged the cost of the item as not being the target of the technical evaluation, but of the commercial evaluation. still, the initial quantity of criteria was very large, averaging out to 115 items to be evaluated later in a pairwise fashion. ijahp article: montevechi, guimarães, de oliveira, friend / selection of ultrasonic scanning system in a private hospital in brazil international journal of the analytic hierarchy process 19 vol. 2, issue 1, 2010 issn 1936-6744 by using a consensus, an effort was made to reduce the number of criteria to avoid criteria that weren’t very expressive or were redundant, facilitating the elaboration of the weights and judgments, facilitating the elaboration of the purchase requisition order, and facilitating the verification of the equipment criteria. the professional experience of the deciding physicians indicated ecri’s same sentiment (2004): there are not distinct differences in the products’ ultimate function, producing ultrasonic imagery, between the similar manufacturers’ models. on the other hand, it was decided that the analysis would be qualitative, keeping in mind, that the luxemburgo hospital and the medical equipment companies in belo horizonte don’t have devices available, such as phantom, for quantitative comparison and verification of the transducer’s quality parameters. the principal criterion would be the transducer’s performance evaluation, in each image’s modality, in the sense of producing an image with diagnostic quality from the physician’s point of view. concurrently, the criteria that wouldn’t have the possibility of being confirmed were purged, including the time for the problem’s solution and the time of arrival to the client after solicitation of the local manufacturer. this decision was made considering that other hospitals would possibly be visited to have equipment installed, and they didn’t make a systematic or historic control of these maintenance parameters in their clinical engineering. the rationalization effort resulted in a hierarchical structure with 35 criteria and subcriteria under the goal. among these, 27 criteria were the targets of direct comparison between the alternatives, considered most relevant and with the least probability of being redundant or of having the same weight or judgment value with pairwise alternative equipment comparison available on the market. thus, the hierarchical decision structure was concentrated as shown in figure 1. these criteria form the base of the purchase requisition order, and were the evaluated items in the visits to other hospitals and in the devices’ demonstrations by the manufacturers or its distributors in the luxemburgo hospital. next, the deciding physicians made a definition of the local weights for each group and subgroup of criteria, as shown in figure 2. during the judgments the weights were attributed through consensus by all the decision-makers. the overall calculated inconsistency was 0.02, and then the results were considered acceptable. the program expert choice, version 11, was used for the calculations. to illustrate the judgments made by the decision-makers, figure 3 shows the matrix used to calculate the weights of the main criteria. in this case, the inconsistency was 0.03, and also the results were considered acceptable. all the others judgments were made using the same approach. in the first level, the item that obtained by far the most weight was the criterion “functions and features” with 0.568, being considered more important than all the other criteria combined (ease in use, image storage and documentation, and customer support). this is due to three understandings on part of the decision-makers: the ultrasonic equipment, in general, doesn’t incur many maintenance problems; the decision-makers use many equipment models, as they already work in other ultrasonic services, the learning of one more equipment or program’s functionality or interface was not a problem; the archiving is a final process, and does not contribute directly to the diagnostic result. in the “functions and features” node, the subcriterion with the most weight was the “convex probe” with 0.648, followed by the “linear probe” and the “endocavity probe”. the “convex probe” is the most used probe to practice medicine by the physicians in the general clinic, and that is the reason for it having the greater weight. the “linear probe” is most used to examine children and the “endocavity probe” is used in gynecology. ijahp article: montevechi, guimarães, de oliveira, friend / selection of ultrasonic scanning system in a private hospital in brazil international journal of the analytic hierarchy process 20 vol. 2, issue 1, 2010 issn 1936-6744 for each probe, second level criterion, the most important subcriterion was “performance with b mode”, image modality most utilized in practice, followed by subcriterion “multi-frequency image”, indicative of the axial resolution and achievable field depth with the probe. the image resources in the “color and power doppler” mode and the “harmonic image” are less utilized in day-to-day hospital oncology. figure 1. proposed decision hierarchy for the acquisition of ultrasound equipment. in relation to the criterion “ease in use”, the most valued subcriterion was the “ergonomics and keyboard” with 0.385, reflecting the physician’s preoccupation with the command arrangement of the console and its positioning ease, adapting itself to the height of the different physicians that will utilize the equipment during the long work day. the second most valued subcriterion was the “friendly interface” with 0.246, perform. w/ b mode perform. w/ color and power doppler convex probe perform. w/ pulsed doppler perform. w/ harmonic image perform. w/ multi-frequency image functions and features perform. w/ b mode perform. w/ color and power doppler linear probe perform. w/ pulsed doppler perform. w/ harmonic image perform. w/ multi-frequency image perform. w/ b mode endocavity probe perform. w/ color and power doppler perform. w/ pulsed doppler perform. w/ multi-frequency image technical ranking ergonomics and keyboard auto-text ease of use presets and tables of obstetrics and gynecology friendly interface cine review image storage and documentation connection interfaces hd storage removable media belo horizonte local technical team availability of parts guarantee conditionscustomer support training possibility of updates ijahp article: montevechi, guimarães, de oliveira, friend / selection of ultrasonic scanning system in a private hospital in brazil international journal of the analytic hierarchy process 21 vol. 2, issue 1, 2010 issn 1936-6744 referring to the software’s useability. the third subcriterion was the “cine review” with 0.156, as a resource utilized principally for the choice of the best image for documentation. the other subcriteria such as auto-text, a resource that increases the productivity through the filling-out of pre-selected texts, and “presets and tables of gynecology and obstetrics”, defined by the existence of pre-configurations and calculations available for various types of exams were, in this order, less valued. figure 2. subcriteria’s local weight in relation to its node. for the first level group “image storage and documentation”, the most valued subcriterion was the “removable media” with 0.400, representing the means of recording for definitive archiving of the exams, along with the “connection interfaces” with 0.400, specifying the modalities available for image printing, demonstration or recording. the least valued subcriterion was the “storage on hard disk”, keeping in technical functions convex probe perform. w/ b mode 0.495 ranking and features 0.648 perform. w/ color and power doppler 0.14 1 0.568 perform. w/ pulsed doppler 0.054 perform. w/ harmonic imaging 0.109 goal perform. w/ multi-frequency image 0.201 linear probe perform. w/ b mode 0.354 0.23 perform. w/ color and power doppler 0.101 perform. w/ pulsed doppler 0.052 perform. w/ harmonic imaging 0.167 perform. w/ multi-frequency image 0.327 endocavity probe perform. w/ b mode 0.527 0.122 perform. w/ color and power doppler 0.125 perform. w/ pulsed doppler 0.061 perform. w/ harmonic imaging 0.287 ease of use ergonomics and keyboard 0.385 0.244 auto-text 0.12 presets and tables of ob/gyn 0.092 friendly interface 0.246 cine review 0.156 image storage connection interfaces 0.4 and documentation hd storage 0.2 0.112 removable media 0.4 customer support belo horizonte local technical team 0.5 0.075 0.575 availabilities of parts 0.5 guarantee conditions 0.195 training 0.127 possibility of updates 0.103 ijahp article: montevechi, guimarães, de oliveira, friend / selection of ultrasonic scanning system in a private hospital in brazil international journal of the analytic hierarchy process 22 vol. 2, issue 1, 2010 issn 1936-6744 mind that the equipment seen in the initial market research possessed at least 40 gb on the disk, more than sufficient for the daily storage of images at the luxemburgo hospital and even for passing to a definitive archiving in the removable hardware (dvd or recordable cd). functions and features ease in use image storage and documentation customer support weights calculated by the eigenvector functions and features 1 3 5 6 0.568 ease in use 1/3 1 3 3 0.244 image storage and documentation 1/5 1/3 1 2 0.112 customer support 1/6 1/3 1/2 1 0.075 inconsistency: 0.03 figure 3. main criteria weights. the criterion “customer support” had the subcriterion “in belo horizonte” with 0.575, as the most important, reflecting the value of proximity for technical care in the same city as the hospital for rapid problem solutions. the subcriteria “conditions of guarantee” and “training” were in second and third places in importance with 0.195 and 0.127 of the weight, respectively. the least valued subcriterion in the group was “possibility of upgrade” with 0.103, keeping in mind that, maintaining the ultrasonic hospital service’s same level of demand, it wouldn’t be necessary to add additional resources. in relation to the support criterion “in belo horizonte”, there was a tie at 0.500, between the subcriterion “local technical team” and “availability of parts.” the sucbcriterion “local technical team” reflects the given importance to the structure and qualification of the maintenance personnel, in hopes of less time stopped for a more efficient first line maintenance. the subcriterion “availability of parts” was considered due to the inventory of plates and probes in brazil and in belo horizonte, as well as the client-aid policy in case of problems, keeping in mind that the institution is philanthropic and subject to a larger delay in the importation of parts due to the governmental bureaucracy. 4.3 – equipment offered by the competition: the alternatives in response to the purchase requisition order sent to the four selected businesses to participate in the competition, the following equipment was offered, in the same price line and intermediate market share: 1. siemens: model g40; 2. toshiba: model nemio ssa-550 a; 3. ge: model logic 5 pro; 4. philips: model envisor sc. ijahp article: montevechi, guimarães, de oliveira, friend / selection of ultrasonic scanning system in a private hospital in brazil international journal of the analytic hierarchy process 23 vol. 2, issue 1, 2010 issn 1936-6744 the definition of the best equipment, outlined in quantitative measures, is not the objective of this study. thus, we will substitute the brands and models offered in the competition with the letters “a” to “d” in relation to the sequence above. the study’s purpose is to obtain a technical ranking of the evaluated ultrasonic equipment based on a realistic point of view and the necessities of the luxemburgo hospital through qualitative criteria connected to the professional experience of the medical service representatives, being unnecessary to decline a result in itself, but having capital importance to the process’s description of the evaluation technique and the definition of the criteria used for the luxemburgo hospital. 4.4 – alternative judgments by the specialists once the proposals of the selected commercial equipment were received, the decision-making team evaluated each of the presented models within previously defined criteria, either as used in another hospital or ultrasonic service, or for a demonstration period in the institution. after the evaluations, the decision-makers made and expressed their judgments in consensus, putting the data directly into the software expert choice. the alternatives were judged by pairwise comparisons for each criterion, using a fundamental scale with values from 1 to 9, as described in saaty (2004), guaranteeing that the level of inconsistency remained within the permitted range. 4.5 – time consumed in the process as the decision-makers possessed more than one workplace, the quantity of hours used and the necessity of conciliating schedules for meetings were critical points for the decision-making process. the process consumed about 7 hours of each of the two decision-makers’ time in the following ways: 1. 1-hour initial meeting in which the facilitator explained separately to each of the decision-makers, due to scheduling conflicts, the proposed methodology; 2. 1.5-hour meeting, held separately with each of the decision-makers, due to scheduling conflicts, in which the facilitator presented the criteria raised in the literature, an initial decision tree; 3. 2.5-hour meeting held with both decision-makers in which they arrived at a consensus about the decision tree, which served in the logbook for the equipment evaluation and for elaboration of the purchase requisition order; 4. 2-hour final meeting held with both decision-makers, in which the physicians, after having evaluated the equipment, offered their judgment in the form of a consensus on the software, and realized the sensitivity analysis. during the time consumed in the process, the doctors involved became familiar with the equipment and how to operate it. in brazil the doctors themselves do this kind of exam rather than a technician trained in the use of the ultrasonic device, such as in the united states. the process was conducted by an expert in the use of ahp theory. 5 – results analysis after the mathematical synthesis, the priority vectors for each alternative were obtained for each criterion. the global synthesis presented a ranking of the alternatives according to the priorities vector in relation to the goal, as demonstrated in figure 4: a. first choice: device d; b. second choice: device c; c. third choice: device a; d. fourth choice: device b. ijahp article: montevechi, guimarães, de oliveira, friend / selection of ultrasonic scanning system in a private hospital in brazil international journal of the analytic hierarchy process 24 vol. 2, issue 1, 2010 issn 1936-6744 this result can be seen in the first line of figure 4. the other lines show how the eigenvectors calculated the normalization of the best device under each subcriteria having number 1. ijahp article: montevechi, guimarães, de oliveira, friend / selection of ultrasonic scanning system in a private hospital in brazil international journal of the analytic hierarchy process 25 vol. 2, issue 1, 2010 issn 1936-6744 figure 4. results of the pairwise comparison. alternatives device a device b device c device d total 0.222 0.217 0.238 0.323 functions and features / convex probe / perform. w/ b mode (g: .182) 0.239 0.26 0.437 1 functions and features / convex probe / perform. w/ color and power doppler (g: .051) 0.5 0.5 1 1 functions and features / convex probe / perform. w/ pulsed doppler (g: .020) 0.5 0.5 1 1 functions and features / convex probe / perform. w/ harmonic image (g: .040) 0.367 0.367 0.817 1 functions and features / convex probe / perform. w/ multi-frequency image (g: .074) 1 1 1 1 functions and features / linear probe / perform. w/ b mode (g: .046) 1 1 1 1 functions and features / linear probe / perform. w/ color and power doppler (g: .013) 0.413 0.413 0.704 1 functions and features / linear probe / perform. w/ pulsed doppler (g: .007) 1 1 1 1 functions and features / linear probe / perform. w/ harmonic image (g: .022) 1 1 1 1 functions and features / linear probe / perform. w/ multi-frequency image (g: .043) 0.5 0.5 0.25 1 functions and features / endocavity probe / perform. w/ b mode (g: .037) 1 1 1 1 functions and features / endocavity probe / perform. w/ color and power doppler (g: .009) 1 1 1 1 functions and features / endocavity probe / perform. w/ pulsed doppler (g: .004) 1 1 1 1 functions and features / endocavity probe / perform. w/ multi-frequency image (g: .020) 1 0.5 0.5 0.5 ease of use / ergonomics and keyboard (g: .094) 1 1 1 1 ease of use / auto-text (g: .029) 1 1 1 1 ease of use / presets and tables ob/gyn (g: .023) 1 1 1 1 ease of use / friendly interface (g: .060) 1 1 1 0.5 ease of use / cine review (g: .038) 1 0.5 0.5 0.5 image storage and documentation / connection interfaces (g: .045) 1 1 1 1 image storage and documentation / hd storage (g: .022) 1 1 0.539 0.311 image storage and documentation / removable device (g: .045) 0.5 0.5 0.5 1 customer support / in belo horizonte / local technical team (g: .022) 0.309 0.577 0.309 1 customer support / in belo horizonte / availability of parts (g: .022) 0.456 0.647 0.288 1 ijahp article: montevechi, guimarães, de oliveira, friend / selection of ultrasonic scanning system in a private hospital in brazil international journal of the analytic hierarchy process 26 vol. 2, issue 1, 2010 issn 1936-6744 figure 5 shows the result of this valuation in the expert choice screen. the consistency of the decisionmakers’ judgments is evident in that the suggested maximum limit of the degree of inconsistency in each criterion and subcriterion was not surpassed in any case. in simulation by the decision-making team, the sensitivity analysis showed itself to be useful in the following phase of the acquisition process: the commercial closing. thus, in the case that the best commercial condition isn’t relative to the first technical choice, and the institution opts for the second technical purchase option, the sensitivity analysis will permit the purchasing department to verify which criteria that provider will need to reinforce in its equipment in order to arrive at the closest possible priority vector obtained through the first technical choice. this improves the initial configuration of the offered equipment. such an analysis was facilitated by the program’s functionalities for mathematical treatment, expert choice version 11, as it carries out an automatic recalculation and graphic redesign if the decision-makers alter the weight of a criterion. thus, the impact this alteration would bring to the final ranking can be verified. the final results analysis is thus aligned with the decision-making team’s sentiment, expressing the preferences and expectations of the decision-makers in numbers. the result can be expressed and decomposed for each criterion and subcriterion, permitting the data to be audited at any time. figure 5. results in the expert choice screen. 6 – conclusions the application of ahp achieved success in the proposed technical evaluation process’s systemization and in the generation of a hierarchical tree model with criteria suggestion for technical evaluation of general use ultrasonic diagnostic equipment acquisition. the adhesion and understanding of the method by the medical decision-makers and the analysts were very much facilitated by the utilization of the program expert choice, which possesses an easy and intuitive utilization and interface. the medical decision-makers, despite their inexperience with the instrument, were able to manipulate the program, using their judgments and implementing alterations to the weights in the sensitivity analysis functionality. as four alternatives were selected (equipment from different manufacturers) to participate in the purchasing process, the number of pairwise comparisons necessary for the evaluation of each final criterion was 6. as such, a total of 162 pairwise comparisons were necessary (6 x 27), once the 27 technical direct comparison criteria were defined with the construction of the hierarchical tree by the decision-makers. ijahp article: montevechi, guimarães, de oliveira, friend / selection of ultrasonic scanning system in a private hospital in brazil international journal of the analytic hierarchy process 27 vol. 2, issue 1, 2010 issn 1936-6744 although the number of comparisons seems large, the values were inserted in the program expert choice at the same time that the decision-makers arrived at a consensus about the judgment. that process, along with the general checking of the judgments, took just two hours, and was not expressed as wearisome or tiring by the decision-makers. ahp showed itself to be a useful tool in the medical equipment acquisition process evaluation technique utilized due to: a. the decomposition of the problem’s complexity in a hierarchical tree that permitted the criteria’s previous, clear definition to be evaluated and was fundamental as a guide for the elaboration of the competition purchase requisition order, as well as the logbook for the evaluation visits in the purchase process of the participating equipment installed in other hospitals. b. the judgment of the weights and the notes provided an auditable map of the criteria and values used by the decision-making team, which can avoid possible indirect gains of members. the generated documentation functions as an auditable trail that permits the criteria, the criteria’s weight and the judgment’s values to be checked for each alternative ahead of the criterion by the director or an independent auditor. this grants transparency to the equipment’s technical evaluation process. c. the result can be expressed in a clear, systemized and numerical form, despite having been an extremely complex evaluation due to the number of variables and the equipment’s technical attributes. thus, a technical ranking of the evaluated alternatives that was generated can be compared with the ranking generated by the commercial evaluation, and carried out in a parallel process. that clear and defined criteria comparison permits the final decision about equipment purchase to be taken by the principal decision-maker of the institution (director), even if he or she is not from the medical sector, the technical sector (clinical engineering) or from the commercial area. acknowledgements the authors would like to express their gratitude to cnpq and fapemig for the support throughout our study. references andreassi, m.g., venneri, l. & picano, e. (2007). cardiac imaging: the effects of diagnostic cardiac ultrasound. progress in biophysics and molecular biology, 93, 399 410. brandão (1999). c. r. repensando a pesquisa participante. 8 ed. são paulo: editora brasiliense. collins, j. (2006). professionalism and physician interactions with industry. journal of the american college of radiology, 3, (5), 325 332. ecri emergency care research institute. (2004). health care product comparison system (hpcs). scanning systems, ultrasonic, general-purpose. pa, eua: ecri. ecri emergency care research institute. (2005) health care product comparison system (hpcs). scanning systems, ultrasonic, general-purpose. pa, eua: ecri press. forman, e.h. & selly, m.a. (2001). decision by objectives: how to convince others that you are right. new jersey: world scientific. forsbeg, f. (2004). ultrasonic biomedical technology: marketing versus clinical reality. ultrasonics, 42, 17 – 27. ijahp article: montevechi, guimarães, de oliveira, friend / selection of ultrasonic scanning system in a private hospital in brazil international journal of the analytic hierarchy process 28 vol. 2, issue 1, 2010 issn 1936-6744 gomes, l.f.a.m., gomes, c.f.s. & almeida, a.t. de. (2002). tomada de decisão gerencial – enfoque multicritério. são paulo: atlas. halperin, e.c., hutchinson, p. & barrier, r.c. (2004). a population-based study of the prevalence and influence of gifts to radiation oncologists from pharmaceutical companies and medical equipment manufactures. international journal of radiation oncology, biology, physics, 59, (5), 1477 – 1483. ho, wi., xu, x. & dey, p.k. (2010). multi-criteria decision making approaches for supplier evaluation and selection: a literature review. european journal of operational research, 202, 16–24. kolzer, j.f., viegas, f., rocco, e. de & ojeda, r.g. (2002). aquisição de equipamentos de ultrasonografia diagnóstica. in: anais do xviii congresso brasileiro de engenharia biomédica. são josé dos campos, sp. lettieri, e. & masella, c. (2009). priority setting for technology adoption at a hospital level: relevant issues from the literature. health policy, 90, 81–88. liberatore, m.j. & nydick , r.l.(2008). the analytic hierarchy process in medical and health care decision making: a literature review. european journal of operational research, 189, 194–207. lim, m.e., o’reilly, d., tarride, j., burke, n., ferrusi, i.l., campbell, k. & goeree, r. (2009). health technology assessment for radiologists: basic principles and evaluation framework. journal of the american college of radiology, 6, (5). mendes, e.v. (2006). uma agenda para a saúde. 2 ed. são paulo: editora hucitec. nepp – núcleo de estudos de políticas públicas. (2000). o setor saúde e o complexo de saúde no brasil. campinas, sp: unicamp. panerai, r.b. & mohr, j.p. (1990). evaluation de tecnologias en salud. metodologia para paises en desarrollo. washington: organizacion panamericana de la salud. ramirez, e.f.f. (2005). modelo conexionista para avaliação de propostas para aquisição de equipamentos médico-hospitalares. tese (doutorado em engenharia elétrica) – faculdade de engenharia elétrica e de computação da universidade estadual de campinas, campinas, sp. rumack, c. m., wilson, s.r. & charboneau, j.w. (1999). tratado de ultra-sonografia diagnóstica. 2ª ed. rio de janeiro: editora guanabara koogan. saaty, t.l. (1987). método de análise hierárquica. são paulo: mcgraw-hill. saaty, t.l. (1994). highlights and critical points in the theory and application of the analytic hierarchy process. european journal of operational research, 74 (3), 426 447. saaty, t.l. (2004). decision making – the analytic hierarchy and network processes (ahp/anp). journal of systems science and systems engineering, 13(1), 1 34. ijahp article: montevechi, guimarães, de oliveira, friend / selection of ultrasonic scanning system in a private hospital in brazil international journal of the analytic hierarchy process 29 vol. 2, issue 1, 2010 issn 1936-6744 sloane, e.b., liberatore, m. j., nydick, r.l., luo, w. & chung, q.b. (2003). using the analytic hierarchy process as a clinical engineering tool to facilitate an iterative, multidisciplinary, microeconomic health technology assessment. computers & operations research, 30, 1447 – 1465. van der rhee, b., verma, r. & plaschka, g. (2009). understanding trade-offs in the supplier selection process: the role of flexibility, delivery, and value-added services/support. int. j. production economics, 120, 30–41. wu, c., lin, c. & chen, h. (2007). optimal selection of location for taiwanese hospitals to ensure a competitive advantage by using the analytic hierarchy process and sensitivity analysis. building and environment, 42, 1431–1444. ijahp article: iida, koizumi/a framework using the analytic hierarchy process for local governments in japan to evaluate projects based on outcomes international journal of the analytic hierarchy process 372 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.593 a framework using the analytic hierarchy process for local governments in japan to evaluate projects based on outcomes yoichi iida 1 school of general education and management studies, suwa university of science nagano, japan iida@rs.sus.ac.jp ryo koizumi school of general education and management studies, suwa university of science nagano, japan su2.cha.n8.gmail.com abstract currently, in japan, women are required to participate and advance in the workplace because of the decline in the labor population. however, it is very difficult for them to do so because of the patriarchal system within japanese history. in the 1990s, local governments began various projects to encourage women to participate in society. at the same time, local government administrations were introduced to project evaluation to boost the efficiency of their projects. however, project evaluation did not function well because project evaluation needs to be linked to outcomes of the program, and the outcome step was often performed independently. the purpose of this paper is to show how to perform project evaluation to promote women’s active participation in society in japan. using the analytic hierarchy process, we show how to evaluate projects in a way that can connect to the outcomes of the program. this framework can also be used to calculate the degree of contribution of projects in the program from two perspectives. first, the decision makers in a local government evaluate the outcomes of the program and second, the people in charge of the projects that make up the program do the evaluation for their own projects. in this paper, we evaluated actual projects of a city in japan and were able to show the direction they should take, such as reduction or expansion for each project in the next fiscal year, by using numerical values obtained in the process of calculation by this framework. 1 acknowledgements: the first author wishes to express his deepest gratitude to late professor thomas l. saaty for his very useful advice and warm encouragement. the authors wish to thank the anonymous referees for their helpful and useful suggestions. ijahp article: iida, koizumi/a framework using the analytic hierarchy process for local governments in japan to evaluate projects based on outcomes international journal of the analytic hierarchy process 373 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.593 keywords: analytic hierarchy process (ahp); local government; gender equality in japan; project evaluation; contribution degree of projects. 1. introduction currently, the labor population is decreasing in japan so women's participation and advancement in the workplace is required. however, it is very difficult for women to work outside of the home because of the patriarchal history of japan. to overcome these difficulties, japan has passed legislation such as the basic act for gender-equal society established in 1999. also, the local governments in japan have focused on programs and projects to promote gender equality using project evaluation. on the other hand, about twenty years have passed since evaluation was introduced in japan’s government administrations. at first, this evaluation was introduced as part of decentralization reform. it then changed to administrative evaluation for budget cuts to overcome financial crises in local government (yamaya, 2016). after that, evaluation evolved with the plan-do-check-act cycle and key performance indicator (kpi) which is like evaluating a company under the influence of new public management (npm). in the midst of these changes, the original purpose of administrative evaluation by local governments became ambiguous. as a result, local government administration evaluation has begun to be regarded as useless work and officials who had to draw up the evaluation sheets have come to feel “evaluation tiredness”. the work is tedious and complicated because several programs are planned to achieve each policy (and have to be evaluated), and several projects are planned to achieve each program (and have to be evaluated) and actual administrative activities have to be implemented for these projects, and they too have to be evaluated. so, in this mass of activity, it is not easy to find the actual evaluation of projects that should be linked to the program. most japanese researchers in policy evaluation point out that if projects are not evaluated relative to each other with respect to the program, it is impossible to use administrative evaluation budgeting for the following fiscal year or to decide the future direction of the project (like whether or not it should be reduced). this is an ongoing problem of administrative evaluation in local government. the purpose of this paper is to promote women’s active participation in society in japan using the analytic hierarchy process (ahp) to perform evaluation projects that can be connected to the outcomes of the program to which they belong. it is difficult to measure how much each project affects the program if the people who evaluate outcomes must adhere to quantitative evaluation. with the ahp, it is possible to include intangibles in the evaluation. in this framework, outcomes of a program are represented as viewpoints and the projects that make up the program are divided into their functions. eventually, the degree to which projects contribute to the program and their relative value can be calculated. by “contribution degree” we mean the relative value a project contributes to the program. this framework has two other perspectives to calculate contribution degrees of projects. ijahp article: iida, koizumi/a framework using the analytic hierarchy process for local governments in japan to evaluate projects based on outcomes international journal of the analytic hierarchy process 374 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.593 the decision makers in a local government make the judgments to evaluate the outcomes of the program, and the people in charge of the projects make the judgments related to their specific projects. in section 4, we show how to evaluate the projects using the ahp and link these evaluations to the outcomes of the program. in section 5, we give an example of real projects to promote women’s active participation in chino city in japan. this framework differs from the ahp because zero is accepted as an evaluation value. on the other hand, we did not use the analytic network process to calculate the contribution degrees because of the need to keep the process simple so users could perform the calculations themselves. 2. literature review first, we reviewed papers about project evaluation. there are many about policy evaluation and administrative evaluation by japanese researchers (for example, ishihara, 2005; yamaya, 2016). generally, an administrative evaluation in a local government is almost a project evaluation (tanaka, 2014). mie prefecture in japan was the first local government to conduct administrative evaluations in 1996. thereafter, many researchers studied how project evaluations were done in mie, and many local administrative bodies in japan imitated that method; for example, they drew up project evaluation sheets themselves. as is well-known, policies, programs and projects are often represented as the hierarchy shown in figure 1. on the other hand, as mentioned above, an administrative evaluation is often a project evaluation and the hierarchy may be elaborated to include several criteria as shown in figure 2. in many articles, the most famous criteria in figure 2 are the 3e, namely economy, efficiency and effectiveness. other typical criteria that are used are legality, regularity, security, resilience, necessity, urgency, priority, adequacy, satisfaction, transparency, quickness and so on (tanaka, 2014). figure 1 hierarchy of administrative activities in a local government program project 1 project 2 project 3 policy ijahp article: iida, koizumi/a framework using the analytic hierarchy process for local governments in japan to evaluate projects based on outcomes international journal of the analytic hierarchy process 375 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.593 figure 2 simple hierarchy for the relative evaluation of projects in a program thus, the relationship between policies, programs and projects is described as a hierarchy and there are many articles using the ahp that do this in administrative policy evaluation. miyagawa (1994) picked the ahp up in his book about policy science. kinoshita published books collecting practical use examples of the ahp in local governments (kinoshita & taji, 2005; kinoshita & ohya, 2007). however, they did not do project evaluation linked to outcomes of the program. to do that, we need to investigate a logic model for project evaluation in evaluation theory. administrative evaluation in japan initially referred to the way it was done in the united states of america or the united kingdom. stipak pointed out the importance of subjective measures in program evaluation (stipak, 1987). hatry (2007) wrote “these objectives normally should be stated in general, not quantitative, terms and should remain reasonably stable”. his words “not quantitative” are very suggestive, although his performance measurement was actually a part of project evaluation related to efficiency. the ahp can deal with evaluation qualitatively without losing reasonable stability. as is well-known, saaty applied the bocr model (anp) to local governments (saaty & özdemir, 2005). the problems of selection of projects, project evaluation, risk assessment in projects and so on have been studied using the ahp (dey, 2006; mahmoodzadeh et al, 2007; huang et al, 2008; zayed et al, 2008). furthermore, there were often political or social problems to solve using the ahp (saaty, 1982, 1983; vargas, 1990). next, we reviewed papers about gender equality in japan. there are a lot of articles and books about women's social advancement written by japanese researchers. for example, naito, who is a very famous researcher of the japanese gender equality issue, classified viewpoints on the topic in three ways including the division of labor by gender, the wage gap by gender and the job promotion gap by gender (naito, 2015). we used these viewpoints as criteria in our hierarchy in figure 5. in their book, naito & yamaya (2015) showed some functions promoting a society with gender equality. we used their results in the hierarchy in figure 5. 3. objectives of the study most local governments do not use relative project evaluation because it is difficult to evaluate them relatively with respect to a program. in fact, it is difficult to elaborate outcome indicators of the program and the projects, particularly if the evaluators are restricted to quantitative evaluations as is prevalent in japanese administrative evaluation. program project 1 project 2 project 3 criterion 1 criterion 2 criterion 3 criterion 4 criterion 5 ijahp article: iida, koizumi/a framework using the analytic hierarchy process for local governments in japan to evaluate projects based on outcomes international journal of the analytic hierarchy process 376 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.593 as a result of what they perceive to be useless administrative evaluations for the purpose of decision-making, officials begin to feel “evaluation tiredness.” ahp has the following strong points:  it is possible to incorporate qualitative matters like human feelings and experiences into the evaluation, while at the end the results are comprehensive evaluation values similar to those obtained with quantitative evaluations.  it is possible to divide key items of evaluation into several viewpoints using a hierarchy, while different individuals can perform the relative evaluations that directly concern the alternatives.  evaluators can argue the weight of each item during the process of evaluation and confirm the results of the process of evaluation later. this means that the ahp is suitable for solving the above-mentioned problem of administrative evaluation. furthermore, as shown in section 2, we already have the viewpoints related to outcomes of the program to promote women’s active participation in society in japan which are approved by most people (naito & yamaya, 2015). the purpose of this paper is to show how to perform a project evaluation so the evaluation of projects can be connected to the outcomes of the program. we show how to use the ahp to easily perform relative evaluation of projects. 4. a framework of project evaluation linked to the program there are three points in this ahp framework: (1) what is the relative evaluation value of a project? (2) what is a hierarchy to calculate the relative evaluation values? and (3) who does the evaluations by performing the pairwise comparisons of items in the hierarchy? by answering these three questions, we can perform relative evaluation of projects similar to the ahp. we constructed this framework under the following conditions:  the hierarchy in the ahp incorporates outcomes of the program in order to perform relative evaluation of projects.  we use only the existing numerical values on administrative evaluation documents such as project evaluation sheets.  the people in charge of the projects evaluate only their projects regardless of the outcome of the program, while the decision makers in a local government evaluate outcomes of the program and how much each project contributes to the program. first, we defined relative evaluation values of projects as follows: relative evaluation value = contribution degree × individual project value most local governments have already performed administrative evaluations and calculated the numerical values of projects. in many cases, they are calculating a target achievement rate based on numerical targets and actual results as shown on the left side of figure 3. several numerical targets are set for each project. so, in the above ijahp article: iida, koizumi/a framework using the analytic hierarchy process for local governments in japan to evaluate projects based on outcomes international journal of the analytic hierarchy process 377 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.593 formulation we defined an individual project value by averaging those target achievement rates. this allows us to use the already existing numerical values from administrative evaluation documents. figure 3 the relationship between projects and the program furthermore, to calculate the above relative evaluation values we need to decide how much each project contributes to the program (figure 3). we simply called this “contribution degree of a project”. this contribution degree of each project is calculated using the ahp. thus, the relative evaluation value of each project is calculated by multiplying the average of target achievement rates of it, which is the individual project value, by its contribution degree. next, we constructed the hierarchy shown in figure 4 to calculate the relative evaluation values of the projects. from the definition of relative evaluation values, it is sufficient to calculate the contribution degrees of the projects with the ahp. firstly, the program is divided into some viewpoints which are the criteria used to evaluate projects in the ahp. in general, the outcome of the program is evaluated by these numerical indicators obtained from the viewpoints of the program. if adequate indicators are found for these outcomes, we can evaluate the program. second, projects are divided into some functions to distinguish the program from the combined viewpoints. it is difficult for the evaluators to directly compare two projects with respect to each viewpoint because their judgement may include their bias. it is fairer and easier to compare functions of projects instead of projects with respect to each viewpoint. it is better to select viewpoints and functions based on academic results of researchers in the field related to the program. in this way, it is easier to obtain approval from many citizens. thus, we constructed the four-layer hierarchy with a viewpoints level and a function level as in figure 4. this hierarchy has two parts, namely, the strictly ahp upper part and the judgments inserted through pairwise comparisons in the lower part. the upper part is calculated by standard ahp. the pairwise comparison part does not always use the ahp because the evaluation values are often zero. contribution degree individual project value = the average of target achievement rates target achievement rate = actual result / numerical target intermediate outcome final outcome program project n target activity result target activity result target activity result ... ijahp article: iida, koizumi/a framework using the analytic hierarchy process for local governments in japan to evaluate projects based on outcomes international journal of the analytic hierarchy process 378 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.593 figure 4 hierarchy consisting of two parts finally, we decided on the people to perform pairwise comparisons of items in the hierarchy. there are two parts in the hierarchy in figure 4. the decision makers in a local government performed pairwise comparisons of items in the ahp part, while the people in charge of the project performed them for the project performance. these people were chosen because items in the ahp part relate to the future direction and the present state of the city, and items about the projects connect to the actual activity. we will show how to synthesize weights of items which were obtained by these pairwise comparisons using a real project in the next section. 5. actual project evaluation about gender equality in a city we performed a project evaluation for the real projects of chino city in japan using the proposed framework. chino city has been individually evaluating the projects using the original project evaluation sheets for about ten years (inazawa, 2012). these project sheets include many numerical values, but they did not perform a relative evaluation of projects well. we applied the framework to projects belonging to the program to promote gender equality in chino city, whose name is "realizing a society promoting women’s participation and advance in the workplace" and whose number is 06-05. this program began with ten projects, however, by 2016 it only had four active projects whose numbers are 06-05-01, 02, 03 and 08 (chino city, 2017). these projects are as follows: project 01 holding lectures to enlighten gender equality project 02 gender equality promotion for employment and in the workplace project 03 gender equality promotion in subregion project 08 gender equality promotion for family and at school program viewpoint 1 function 1 viewpoint 2 function 2 function 3 function 4 project 1 project 2 project 3 ahp part pairwise comparison part ijahp article: iida, koizumi/a framework using the analytic hierarchy process for local governments in japan to evaluate projects based on outcomes international journal of the analytic hierarchy process 379 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.593 5.1 confirming the individual project values of the projects we need individual project values to calculate relative evaluation values of projects. as mentioned in the previous section, we used the average of these target achievement rates of each project (table 1). for example, it follows from the row of project 01 that this project has four types of target achievement rates and the average of them is 93.750%. if the target achievement rate is larger than 100%, then we replaced it with 100% because it means that the target of the activities was not adequate. for example, we replaced 166.67% with 100% for project 02. table 1 individual project values of projects on project evaluation sheets (chino city, 2017) unit (%) projects\items 1 2 3 4 average project 01 100 100 100 75 93.750 project 02 53.33 166.67 100 140 100 84.443 project 03 90 0 100 77.14 66.785 project 08 300 100 40 70.000 5.2 constructing a hierarchy and evaluators we constructed a four-layer hierarchy (figure 5). first, the name of the program was at the top of the hierarchy and those of the projects were at the bottom. next, we decided on three viewpoints as outcomes of the program which are the division of labor by gender, the wage gap by gender and the job promotion gap by gender. these are the most important viewpoints of gender equality in japan (naito, 2015). the division of labor by gender: viewpoint that women are domestic laborers at home, while men work outside. the wage gap by gender: viewpoint that women’s wages are lower than men’s wages. the job promotion gap by gender: viewpoint that women are excluded from decision making and therefore find it hard to be promoted in the workplace. ijahp article: iida, koizumi/a framework using the analytic hierarchy process for local governments in japan to evaluate projects based on outcomes international journal of the analytic hierarchy process 380 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.593 figure 5 hierarchy for calculating contribution degrees we decided on functions of projects to achieve those viewpoints referring to the book by naito and yamaya (2015). these functions are place for advice, opportunity of learning, environment for access to information and interaction with specialists. furthermore, we added two functions with characteristics of administrative activities and gender equality that are grasp of current situation and support of promotion in the workplace. place for advice: function of consultation such as telephone and individual consultation. opportunity of learning: function of learning issues about gender equality such as courses and workshops. environment for access to information: function of offering information to solve issues related to gender equality such as library and website. interaction with specialists: function of connecting to specialists like psychological counselors and attorneys to solve issues related to gender equality such as a party after a lecture. grasp of current situation: function of surveying the current situation such as questionnaire survey and consciousness investigation. promotion of gender equality society the job promotion gap by gender the wage gap by gender the division of labor by gender project 01 project 02 project 03 project 08 environment for access to information opportunity of learning place of advice support of promotion in the workspace grasp of current situation interaction with specialists ijahp article: iida, koizumi/a framework using the analytic hierarchy process for local governments in japan to evaluate projects based on outcomes international journal of the analytic hierarchy process 381 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.593 support of promotion in the workplace: function of supporting the promotion of women's participation in decisions in the workspace such as certifications for companies promoting women’s participation and advance in the workplace. the director and the assistant director in charge of the program who are decision makers in a local government were chosen to perform pairwise comparisons in the ahp part, and the chief person in charge of these projects was chosen to make those comparisons for the project evaluation part (figure 4). we thought that two or more people should decide on the future direction of the program. 5.3 calculating the contribution degrees of projects first, we calculated the weights of items in the hierarchy in figure 5 and obtained synthesized evaluation values of functions. after that, we calculated the contribution degrees at the end of this section. 5.3.1 calculating the synthesizing evaluation values of functions we interviewed the director and the assistant director in charge of the program separately to perform pairwise comparisons of the ahp. we combined these values using geometric means and calculated weights using the eigenvector method of the ahp (table 2). the question is which viewpoint is more important for realizing a society promoting women’s participation and advance in the workplace? and how much more important? table 2 pairwise comparison table and weights of viewpoints with respect to the program program division wage job weights division of labor 1 4.583 3.873 0.665 wage gap 0.218 1 0.378 0.111 job promotion 0.258 2.646 1 0.224 c.i.= 0.036 similarly, we combined their values using geometric means and calculated weights of six functions with respect to each viewpoint (tables 3 to 5). the question is which function is more important (as functions of activities) to achieve the viewpoint? and how much more important? table 3 evaluation table of functions with respect to the first viewpoint of the division of labor division adv. learn info. spec. current prom. weights advice 1 1.095 1.183 2.449 4.472 1.118 0.237 learning 0.913 1 0.447 0.913 5 1 0.167 information 0.845 2.236 1 0.913 5.292 1 0.219 specialists 0.408 1.095 1.095 1 3.873 1 0.166 current 0.224 0.2 0.189 0.258 1 0.775 0.055 promotion 0.894 1 1 1 1.291 1 0.156 c.i.= 0.069 ijahp article: iida, koizumi/a framework using the analytic hierarchy process for local governments in japan to evaluate projects based on outcomes international journal of the analytic hierarchy process 382 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.593 table 4 evaluation table of functions with respect to the second viewpoint of the wage gap wage gap adv. learn info. spec. current prom. weights advice 1 0.183 0.236 5 0.913 0.236 0.094 learning 5.477 1 1.732 3.873 3.873 0.913 0.290 information 4.243 0.577 1 3.873 3.873 0.775 0.223 specialists 0.2 0.258 0.258 1 2 0.236 0.060 current 1.095 0.258 0.258 0.5 1 0.236 0.059 promotion 4.243 1.095 1.291 4.243 4.243 1 0.272 c.i.= 0.112 table 5 evaluation table of functions with respect to the third viewpoint of the job promotion gap promotion adv. learn info. spec. current prom. weights advice 1 0.2 0.224 0.2 0.267 0.154 0.032 learning 5 1 2 3.873 3.464 0.169 0.192 information 4.472 0.5 1 1.732 1 0.183 0.102 specialists 5 0.258 0.577 1 1.291 0.183 0.085 current 3.742 0.289 1 0.775 1 0.183 0.079 promotion 6.481 5.916 5.477 5.477 5.477 1 0.510 c.i.= 0.100 in summary, the following evaluation table of functions with respect to viewpoints was obtained (table 6). table 6 evaluation table of functions with respect to each viewpoint functions \ viewpoints division wage gap promotion gap advice 0.237 0.094 0.032 learning 0.167 0.290 0.192 information 0.219 0.223 0.102 specialists 0.166 0.060 0.085 current 0.055 0.059 0.079 promotion 0.156 0.272 0.510 finally, we calculated synthesizing evaluation values of functions with respect to the program. we obtained table 7 from tables 2 and 6 by the following calculation: ( 0.237 0.094 0.032 0.167 0.290 0.192 0.219 0.223 0.102 0.166 0.060 0.085 0.055 0.059 0.079 0.156 0.272 0.510) ( 0.665 0.111 0.224 ) = ( 0.175 0.187 0.193 0.136 0.061 0.248) . ijahp article: iida, koizumi/a framework using the analytic hierarchy process for local governments in japan to evaluate projects based on outcomes international journal of the analytic hierarchy process 383 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.593 table 7 synthesizing evaluation values of functions with respect to the program functions advice learn info. special current promotion total valuation 0.175 0.187 0.193 0.136 0.061 0.248 1.000 5.3.2 calculating the weights of projects with respect to functions we interviewed the chief person in charge of these projects and then asked her to perform pairwise comparisons of the ahp using a questionnaire. in this way, because all the projects do not have all the functions, we needed to decide which projects had meaningful functions. table 8 was made with the chief person in charge, the predecessor and the vice-chief person in charge before the chief person in charge performed pairwise comparisons. table 8 relationship between projects and functions advice learn info. special current promotion project 01 na na project 02 na project 03 project 08 na according to table 8, the chief person in charge compared projects with respect to each function by pairwise comparison of the ahp obtaining tables 9 to 14. the question is which project has more function? and how much more important? table 9 weights of projects with respect to function, place for advice advice project 01 project 03 weights project 01 1 1/5 0.167 project 03 5 1 0.833 c.i.= 0 table 10 weights of projects with respect to function, opportunity of learning learn project 01 project 02 project 03 project 08 weights project 01 1 1 5 1/7 0.145 project 02 1 1 3 1/7 0.121 project 03 1/5 1/3 1 1/7 0.051 project 08 7 7 7 1 0.682 c.i.= 0.088 ijahp article: iida, koizumi/a framework using the analytic hierarchy process for local governments in japan to evaluate projects based on outcomes international journal of the analytic hierarchy process 384 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.593 table 11 weights of projects with respect to function, environment for access to information info. project 01 project 02 project 03 project 08 weights project 01 1 5 5 1 0.411 project 02 1/5 1 3 1/5 0.113 project 03 1/5 1 1/5 0.064 project 08 1 5 5 1 0.411 c.i.= 0.052 table 12 weights of projects with respect to function, interaction with specialists special project 01 project 02 project 03 project 08 weights project 01 1 7 3 1 0.368 project 02 1/7 1 1/3 1/7 0.050 project 03 1/3 3 1 1/7 0.109 project 08 1 7 7 1 0.473 c.i.= 0.042 table 13 weights of projects with respect to function, grasp of current situation current project 02 project 03 project 08 weights project 02 1 1 5 0.455 project 03 1 1 5 0.455 project 08 1/5 1/5 1 0.091 c.i.= 0 table 14 weights of projects with respect to function, support of promotion in the workplace promotion project 02 project 03 project 08 weights project 02 1 5 5 0.701 project 03 1/5 1 1/3 0.097 project 08 1/5 3 1 0.202 c.i.= 0.068 ijahp article: iida, koizumi/a framework using the analytic hierarchy process for local governments in japan to evaluate projects based on outcomes international journal of the analytic hierarchy process 385 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.593 in summary, we obtained the following table about weights of functions included in projects. table 15 weight of each function included in projects advice learn info. special current promotion project 01 0.167 0.145 0.411 0.368 0 0 project 02 0 0.121 0.113 0.050 0.455 0.701 project 03 0.833 0.051 0.064 0.109 0.455 0.097 project 08 0 0.682 0.411 0.473 0.091 0.202 total 1.000 1.000 1.000 1.000 1.000 1.000 5.3.3 calculating the contribution degrees of projects here, we calculated the contribution degree of each project to the program by combining table 7 and table 15. in fact, table 16 was obtained by the following calculation: ( 0.167 0.145 0.411 0.368 0 0 0 0.121 0.113 0.050 0.455 0.701 0.833 0.051 0.064 0.109 0.455 0.097 0 0.682 0.411 0.473 0.091 0.202 ) ( 0.175 0.187 0.193 0.136 0.061 0.248) = ( 0.186 0.253 0.235 0.327 ). table 16 contribution degrees of the projects with respect to the program projects project 01 project 02 project 03 project 08 contribution degrees 0.186 0.253 0.235 0.327 5.4 calculating the relative evaluation values of the projects and implication finally, we calculated the relative evaluation values of projects with respect to the programs by the following formula defined in section 4: relative evaluation value = contribution degree × individual project value. as mentioned in section 4, individual project values are the average of target achievement rates of the project and this is shown in table 1. consequently, we obtained relative evaluation values in table 17. this was our goal for the framework. ijahp article: iida, koizumi/a framework using the analytic hierarchy process for local governments in japan to evaluate projects based on outcomes international journal of the analytic hierarchy process 386 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.593 table 17 contribution degrees and relative evaluation values of projects with respect to the program project 01 project 02 project 03 project 08 contribution degrees 0.186 0.253 0.235 0.327 individual project values 93.750 84.443 66.785 70.000 relative evaluation values 17.421 21.356 15.663 22.872 once all of the calculations were completed the following results were obtained from table 17:  project 01 has the lowest contribution degree and a low relative evaluation value, although it has the highest individual project values (table 17). it does not have the functions “support of promotion in the workspace” (table 8) which has high synthesizing evaluation values (table 7). consequently, project 01 should be reduced due to deep cuts in the budget in the next fiscal year.  project 02 has high contribution degree and high relative evaluation value (table 17). it has the function “support of promotion in the workspace” in a high ratio (table 15) which has high synthesizing evaluation values (table 7). consequently, project 02 should be maintained at the status quo in the next fiscal year.  project 03 has the lowest relative evaluation and low contribution degree (table 17). it has fewer functions of “support of promotion in the workspace”, “environment for access to information” and “opportunity of learning” (table 15). consequently, project 03 should be cut from the budget in the next fiscal year.  project 08 has the highest relative evaluation value and the highest contribution degree (table 17). it has almost all functions in the highest ratio (table 15). consequently, project 08 should be expanded in the next fiscal year.  if both projects 01 and 03 are reduced in the next fiscal year, then a new project will need to be started for the function “place of advice” (table 8). 6. implications and limitations the developed framework of project evaluation linked to outcomes of the program in this paper has the following strong points:  it is possible to clarify outcomes of the program which are the same as those of the projects (figure 5 and table 2).  it is possible to clarify the current situation of the local government with regards to the program (or the policy) by giving weights of importance to viewpoints of the program (table 2). for example, chino city is developing gender equality as far can be seen from the results in section 5.  it is possible to clarify the role of each project because it is divided into several functions (tables 8 and 15).  it is easier for evaluators to compare functions of projects than to directly compare projects (tables 3 to 5).  it is possible to separate the evaluation of city officials in charge of a project from the evaluation of the project (tables 1 and 17). for example, if the individual project ijahp article: iida, koizumi/a framework using the analytic hierarchy process for local governments in japan to evaluate projects based on outcomes international journal of the analytic hierarchy process 387 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.593 value, which is the average of target achievement rates of the project, is high, then the officials in charge should be rated highly. on the other hand, if the relative evaluation value and the contribution degree of the project are low, then the project is evaluated low and will be reduced in the next fiscal year.  it is possible to decide on the future direction of each project by obtaining the contribution degree of the project (see section 5.3). this means that this framework includes effectiveness evaluation of the projects to the program. furthermore, we found out the following through the process of evaluation in section 5:  this was the first time that the chief person in charge considered the program related to their projects deeply.  the chief person in charge could understand the relationship between the projects belonging to the program well. for example, they did not need to pursue target achievement rate over 100% while they had projects of low target achievement rate (table 1). as a result, they will be able to improve the quality of their projects.  the director had a strong opinion about the gender equality program, although the assistant director and the chief person in charge did not know it.  the chief person in charge can discuss their projects with the director and the assistant director using this framework. this may solve the previous issue of administrative evaluation like project evaluation being seen as useless for officials. there are a few limitations of our research. first, we supposed that local governments had already evaluated their projects and calculated several kinds of target achievement rates for each project. in fact, target management type evaluation is the mainstream in japan. if the governments do not have adequate numerical values for individual evaluations, they would need to find these individual project values before applying the framework. this means they would need to improve the indicators to evaluate the effectiveness of projects. next, historically, administrative evaluation is a technique used for budget cuts. the framework proposed in this paper does not directly contribute to making a budget cut. however, this framework can contribute to the clarification of the outcome of the project by not using quantitative terms. this is a strong point of the ahp, that qualitative as well as quantitative factors can be included. if evaluators do not know outcomes of the program at all, they cannot evaluate the effectiveness of the projects. finally, it is difficult to decide on viewpoints of criteria in this framework with which a majority of the people agree. for the program with the numerical example in section 5, it was easier to decide on viewpoints of criteria because the field of gender equality has a lot of results from japanese researchers. this difficulty is like that of a logic model in policy evaluation. on the other hand, the five functions in figure 5, except for support of promotion in the workspace, can be used for many other projects in future problems. 7. conclusions we showed a framework of project evaluation to calculate the effectiveness of projects for local governments using the ahp. we called this value the contribution degree of the ijahp article: iida, koizumi/a framework using the analytic hierarchy process for local governments in japan to evaluate projects based on outcomes international journal of the analytic hierarchy process 388 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.593 project. the contribution degree is calculated based on viewpoints of the program and functions of the projects. this framework is a kind of logic model using the ahp because we constructed a hierarchy with viewpoints and functions with it. finally, we used the ahp because of its simplicity for evaluators, but at the same time we assumed that all the items were mutually independent. however, those in the same level have dependence between each other. a future problem to address would be to soak up dependence with the anp to the extent that it does not compromise the results. ijahp article: iida, koizumi/a framework using the analytic hierarchy process for local governments in japan to evaluate projects based on outcomes international journal of the analytic hierarchy process 389 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.593 references chino city. (2016/17). project evaluation sheets of the program: gender equality (in japanese). no. 06-05-01 to 03 and 08. dey, p. k. (2006). integrated project evaluation and selection using multiple-attribute decision-making technique. international journal of production economics, 103(1), 90103. doi: https://doi.org/10.1016/j.ijpe.2004.11.018 hatry, p.h. (2007). performance measurement: getting results, 2nd edition. washington, d.c.: the urban institute. doi: https://doi.org/10.1080/01944360701755709 huang, c. c., chu, p. y., & chiang, y. h. (2008). a fuzzy ahp application in government-sponsored r&d project selection. omega, 36(6), 1038-1052. doi: https://doi.org/10.1016/j.omega.2006.05.003 inazawa, k. (2012). gyosei-hyoka no donyu to katsuyo: yosan/kesan, sogokeikaku (zoho ban) (in japanese) [introduction and practical use of administrative evaluation: budget, settlement of accounts, comprehensive plan (enlarged edition)]. japan: imagine syuppan. ishihara, t. (eds.). (2005). jichitai-gyosei-hyoka case study (in japanese) [performance measurement in japanese local government]. japan: toyo keizai. kinoshita, e., & ohya, t. (eds.). (2007). kigyo & gyosei notameno ahp jireisyu (in japanese) [the collection of ahp practical use examples]. japan: juse press. kinoshita, e., & taji, k. (eds.). (2005). gyosei-keiei notameno ishiketteiho: ahp wo tsukatta nanmon-dakai no shinsyuho (in japanese) [decision making methods for administrative management: a new method by ahp to overcome difficult problems]. japan: gyosei. mahmoodzadeh, s., shahrabi, j., pariazar, m., & zaeri, m. s. (2007). project selection by using fuzzy ahp and topsis technique. world academy of science, engineering and technology, 30, 333-338. miyagawa, k. (1994). seisaku-kagaku no kiso (in japanese) [foundations of policy sciences]. japan: toyo keizai. naito, k. (2015). danjyo-kyodosankaku-syakai-keisei seisaku (in japanese) [gender equality society formation policy]. in naito, k., & yamaya, k. (eds.). danjyokyodosankaku-seisaku: gyosei-hyoka to shisetsu-hyoka (in japanese) (1-12). japan: koyoshobo. naito, k., & yamaya, k. (eds.). (2015). danjyo-kyodosankaku-seisaku: gyosei-hyoka to shisetsu-hyoka (in japanese) [gender equality policy: administrative evaluation and facility evaluation]. japan: koyoshobo. ijahp article: iida, koizumi/a framework using the analytic hierarchy process for local governments in japan to evaluate projects based on outcomes international journal of the analytic hierarchy process 390 vol. 10 issue 3 2018 issn 1936-6744 https://doi.org/10.13033/ijahp.v10i3.593 saaty, t.l. (1980). the analytic hierarchy process: planning, priority setting, resource allocatiom. new york: mcgraw-hill. saaty, t.l. (1982). decision making for leaders. new york: lifetime learning. saaty, t.l. (1983). priority setting in complex problems. ieee transaction on engineering management em, 30, 140–155. doi: https://doi.org/10.1109/tem.1983.6448606 saaty, t.l. (1996). decision making with dependence and feedback: the analytic network process. pittsburgh, pa: rws publications. saaty, t.l. (2001). fundamentals of decision making and priority theory with the analytic hierarchy process, 2nd edition. pittsburgh, pa: rws publications. saaty, t.l., & özdemir, m.s. (2005). the encyclicon: a dictionary of decisions with dependence and feedback based on the analytic network process. pittsburgh, pa: rws publications. stipak, b. (1987). using subjective measures in program evaluation. in busson, t., & coulter, p. (eds.). policy evaluation for local government (45-61). westport, ct: greenwood press. tanaka, k. (2014). jichitai-hyoka no senryaku (in japanese) [strategy of local government evaluation]. japan: toyo keizai. vargas, l.g. (1990). an overview of the analytic hierarchy process and its applications. european journal of operational research 48 (1), 2–8. doi: https://doi.org/10.1016/0377-2217(90)90056-h yamaya, k. (2016). many faces of evaluation in japanese local governments: 20 years of decentralization and financial crisis (in japanese). japanese journal of evaluation studies, 16(1), 31-45. zayed, t., amer, m., & pan, j. (2008). assessing risk and uncertainty inherent in chinese highway projects using ahp. international journal of project management, 26(4), 408-419. doi: https://doi.org/10.1016/j.ijproman.2007.05.012 ijhap article: sabia, salomon, sobreira junior, marins, lima/ determination of the most appropriate policy for river basin management international journal of the analytic hierarchy process 201 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.263 determination of the most appropriate policy for river basin management rodolfo j. sabia cariri regional university (urca) juazeiro do norte, ce, brazil e-mail: rodolfo.sabia@urca.br valerio a. p. salomon sao paulo state university (unesp) guaratingueta, sp, brazil e-mail: salomon@feg.unesp.br francisco a. v. sobreira junior cariri regional university (urca) juazeiro do norte, ce, brazil e-mail: jr2_vilar@hotmail.com fernando a. s. marins sao paulo state university (unesp) guaratingueta, sp, brazil e-mail: fmarins@feg.unesp.br anna f. o. lima cariri regional university (urca) juazeiro do norte, ce, brazil e-mail: annaflavia.lima@gmail.com abstract the parliament of the salgado river provided actions and recommendations for the future of the salgado river basin which is in the south of ceara state, brazil. these recommendations were obtained through a democratic process, with the involvement of around 100 participants from private companies, public organizations and the third sector. the intention was to define a model that is compatible with the aspirations of society regarding the use and quality of the water, as well as define necessary decision actions and strategies. the main goal of this article is to determine, with a multi-criteria decision analysis, an appropriate policy for the river basin management. therefore, the analytic hierarchy process (ahp) was applied. the criteria are a combination of economic, environmental, and social issues; the alternatives were defined as essentially  corresponding author. this research was financially supported by sao paulo research foundation (fapesp grant no. 2013/03525-7). ijhap article: sabia, salomon, sobreira junior, marins, lima/ determination of the most appropriate policy for river basin management international journal of the analytic hierarchy process 202 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.263 preservationist policy, sustainable policy, and essentially economic policy. the ahp application was effective in this decision analysis. keywords: ahp; environmental policies; river basin management. 1. introduction disordered growth associated with the process of water quality degradation has caused serious problems to the salgado river basin in the brazilian state of ceara. river basins in northeast brazil are affected by the intermittence of rivers, rainfall irregularity and the lack of an efficient and participative management policy. multi-criteria decision analysis (mcda) may help the assessment of environmental impacts and in turn improve the management of river basins. the parliament of the salgado river provided actions and recommendations for the future of the salgado river basin, through a democratic process, with around 100 participants from private companies, public organizations and also from the third sector. the objective of this research was to define an mcda model compatible with the aspirations of society regarding the use and quality of the water, as well as to define the necessary actions and strategies. the main goal of this article is to determine, with mcda, the appropriate policy for the river basin management. therefore, the analytic hierarchy process (ahp) was applied. section 2 highlights some works from mcda literature on environmental management. section 3 discusses methodological issues which resulted in the mcda model. in sections 4 and 5, the ahp application and results are presented and discussed. section 6 ends this article with conclusions, limitations and a proposal for future works. 2. literature review in the past, environmental management has been regarded as a constraint for companies and has even been considered an obstacle to competitiveness. nowadays, however, it is treated differently, with increasingly more companies adopting strategies and making decisions towards environmental management (frondel et al., 2007). river basin management is a major issue in environmental management. ahp has been applied to rank alternatives when several criteria and sub-criteria are present in the decision (tahriri et al., 2008). ahp has proved itself a suitable method for pondering judgments in complex decisions (awasthi & chauhan, 2011; bottero et al., 2011; gao & hailu, 2012). with ahp, decision-making problems are structured in hierarchies, and qualitative or quantitative data can be used to derive ratio scales among the decision elements in each hierarchical level (bello-dambatta et al., 2009), which makes it one of the most preferable approaches for mcda. there are several software models (e.g. expert choice and super decisions) already used in environmental management problems (contreras et al., 2008). according to saaty (2008), an mcda method such as ahp balances the interactions between decision criteria and synthesizes information into a vector of priorities for the ijhap article: sabia, salomon, sobreira junior, marins, lima/ determination of the most appropriate policy for river basin management international journal of the analytic hierarchy process 203 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.263 alternatives. ahp is often used to solve complex decision problems, being widely used to determine the weights or priorities (dong et al., 2010; xie & tang, 2010; vidal et al., 2011). ahp has many environmental management applications such as ecotourism (ok et al., 2011), natural resources (schmoldt et al., 2001), forestry (samari et al., 2012) coastal systems (ryu et al., 2012), and disaster risk measurement (carreño et al., 2007; chen, et al. 2009). an expert choice model was developed to prioritize alternatives of water utilization from a brazilian power plant (larrubia, 2010). swot (strengths-weaknesses-opportunitiesthreatens) analysis was integrated with ahp in another watershed application, a turkish lake basin (yavuz & baycan, 2013). swot analysis is a multi-criteria framework for strategic planning (humphrey, 2005). the bocr (benefits-costs-opportunities-risks) which is similar to swot framework is a usual model in ahp applications (wijnmalen, 2007). pest (political-economic-social-technological) analysis is a framework to evaluate environmental factors in business planning (macginty et al., 2013). 3. research methodology this article aims to present the best policy from a group of alternatives that is capable of meeting the aspirations of society regarding the salgado river. there are many environmental problems on this river basin which can be addressed with a participatory application of the ahp. one of the authors has participated in the meetings of the parliament of the salgado river, and has played a dual role in the process. at first, he spoke to the parliament about how mcda and ahp could be useful in the study of policies for the river basin management. then, he heard and collected opinions about the importance of criteria and alternatives to the policies. this article is more than a case research as it presents the mcda modeling with ahp for this important environmental decision problem. for this study, a total of seven meetings were conducted. the first and the last were plenary meetings, in the main city of juazeiro do norte. five meetings were conducted with only the local participants in a single city: barbalha, caririaçu, crato, juazeiro do norte and missao velha. according to the brazilian institution of geography and statistics, the population of these cities sums more than half a million inhabitants, the area is around 5,500 km 2 , and their gross domestic product, in 2011, was 2.6 million dollars (http://www.ibge.gov.br/english/). figure 1 presents the criteria hierarchy obtained from the parliament of the salgado river. this hierarchical structure was validated by the parliament in the final plenary meeting. ijhap article: sabia, salomon, sobreira junior, marins, lima/ determination of the most appropriate policy for river basin management international journal of the analytic hierarchy process 204 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.263 figure 1. criteria and sub-criteria for the salgado river basin management. as can be observed, there are two homonymous sub-criteria named “residential”; one below the “social” criterion and another below the “socio-economic” criterion. this is because the resident’s point-of-view was considered regarding these two criteria. as a matter of fact, this repetition may suggest dependence or influence among the hierarchy’s components. in this case, the analytic network process (anp) may be the proper multicriteria decision method. however, due to innovative application in the local river basin management, ahp application was maintained, since it was considered a simpler method. the hierarchy focuses on two (social and economic) of the four factors from the pest analysis which also include political and technological factors (macginty et al., 2013). political factors are considered in the bottom of the hierarchy with the alternatives. technological factors were not considered in this case. one reason for this is that society was not concerned with this major issue at that moment. that is, social and economic problems were greater than political or technological ones. an expert choice model was developed for ahp application. judgments and priorities are presented in section 4. 4. ahp application table 1 presents the judgments of the relative priorities for the criteria, according to the parliament of salgado river based on the fundamental scale of absolute numbers (saaty, 2014). determination of the most appropriate policy for salgado river basin management economic agriculture and livestock industrial services environmental landscape fauna and flora siltation social governmental recreational residential socioeconomic agricultural industrial residential socioenvironmental waste disposal water supply water treatment ijhap article: sabia, salomon, sobreira junior, marins, lima/ determination of the most appropriate policy for river basin management international journal of the analytic hierarchy process 205 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.263 table 1 priorities for the criteria criterion ec en so sc sn priority economic (ec) 1 2 8 7 7 50% environmental (en) 1/2 1 4 6 6 31% social (so) 1/8 1/4 1 1/2 1/2 5% socio-economic (sc) 1/7 1/6 2 1 1 7% socio-environmental (sn) 1/7 1/6 2 1 1 7% the pairwise comparison matrix presented in table 1 has an inconsistency ratio around 3%. it is assumed that an inconsistency ratio value lower than 10% is an indication of the reliability of information (saaty, 1977; garuti & salomon, 2011). economic factors have the highest priority with 50%. they are followed by environmental factors with 31%. social factors have the lowest priority with only 5%. table 2 presents the judgments on the relative priorities for the economic sub-criteria, also according the parliament of salgado river and based on the fundamental scale of absolute numbers. the inconsistency ratio was around 5%. ‘services’ has the highest priority which indicates that it is the economic sector preferred by society. table 2 priorities for economic sub-criteria sub-criterion a i s priority agriculture and livestock (a) 1 3 1 40% industrial (i) 1/3 1 1/6 10% services (s) 1 6 1 50% similar judgments were conducted for the other four criteria. table 3 summarizes the local and overall priorities for the sub-criteria. no comparison matrix has an inconsistency ratio greater than 10%. sub-criterion ‘services’ has the highest overall priority. ijhap article: sabia, salomon, sobreira junior, marins, lima/ determination of the most appropriate policy for river basin management international journal of the analytic hierarchy process 206 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.263 table 3 priorities for all sub-criteria sub-criterion / criterion local priority overall priority agriculture and livestock / economic 40% 20.0% industrial / economic 10% 5.0% services / economic 50% 25.0% landscape / environmental 40% 12.5% fauna and flora / environmental 40% 12.5% siltation / environmental 20% 6.0% governmental / social 44% 2.2% recreational / social 50% 2.5% residential / social 6% 0.3% agricultural / socio-economic 67% 4.7% industrial / socio-economic 22% 1.5% residential / socio-economic 11% 0.8% waste disposal / socio-environmental 14% 1.0% water supply / socio-environmental 29% 2.0% water treatment / socio-environmental 57% 4.0% three alternatives policies were defined by the parliament of salgado river. these policies are an essentially preservationist policy, a sustainable policy, or an essentially economic policy. table 4 presents the judgments on the relative priorities for the alternatives policies regarding the economic sub-criterion a (agriculture and livestock). the comparison matrix is 100% consistent. table 4 local priorities for alternatives policies regarding agriculture and livestock policy p s e priority essentially preservationist (p) 1 3 3 69% sustainable (s) 1/3 1 1 23% essentially economic (e) 1/3 1 1 8% similar judgments were conducted for the other four criteria. table 5 summarizes the local priorities for alternatives policies regarding economic sub-criteria. table 5 local priorities for alternatives policies regarding economic sub-criteria policy a (40%) i (10%) s (50%) priority essentially preservationist 69% 78% 60% 65% sustainable 23% 14% 20% 21% essentially economic 8% 8% 20% 14% 5. results and discussion table 6 summarizes the local and overall priorities for alternative policies regarding all criteria. essentially preservationist policies have the highest overall priority with 60%. ijhap article: sabia, salomon, sobreira junior, marins, lima/ determination of the most appropriate policy for river basin management international journal of the analytic hierarchy process 207 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.263 table 6 priorities for alternatives policies regarding all criteria policy ec (50%) en (31%) so (5%) sc (7%) sn (7%) overall priority essentially preservationist 65% 60% 67% 36% 42% 60% sustainable 21% 21% 22% 48% 40% 24% essentially economic 14% 19% 11% 16% 18% 16% figure 2 presents the variation of overall priorities for alternatives as a function of priority for socio-economic criterion. this is the only criterion where the essentially preservationist policy has a local priority lower than another policy. however, the priority for the socio-economic factor shall be increased to more than 75%, and then the sustainable policy will have an overall priority higher than essentially preservationist policy. figure 2. sensitivity analysis (expert choice software) 6. conclusions the essentially preservationist policy has the highest priority. after a sensitivity analysis was performed this priority was confirmed. this result was presented to all the members of the parliament of salgado river, and they agreed to pursue essentially preservationist policies. however, the economic criterion was the most relevant and was a result of the need for infrastructure to ensure the achievements of environmental yearnings. a large number of publications applying ahp, or another mcda method in river basin management do not exist. in this sense, environmental management was used as a background theory. it is clear that the study of river basin management linked to brazilian policy management instruments is a challenge that must be faced every day. therefore, the ahp applications are useful in contributing to the carrying out of the purposes and the policies to be adopted. one of the main benefits of the ahp application ijhap article: sabia, salomon, sobreira junior, marins, lima/ determination of the most appropriate policy for river basin management international journal of the analytic hierarchy process 208 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.263 was the lack of political bias when prioritizing alternatives and criteria. the judgments were impartially conducted. the results of this study have implications for the use of a suitable alternative for the estimation of the benefits of river basin management improvements. the ahp with its hierarchy structure was an effective method for studying the problem. the ahp established priorities to alternative policies according to the parliament of salgado river. the essentially preservationist policy must be the policy for river basin management in brazil. this may not be the solution for other countries. thus, the application of this study is preliminarily bound to places with similar culture and laws as brazil. ijhap article: sabia, salomon, sobreira junior, marins, lima/ determination of the most appropriate policy for river basin management international journal of the analytic hierarchy process 209 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.263 references awasthi, a. & chauhan, s.s. (2011). using ahp and dempster-shafer theory for evaluating sustainable transport solutions. environmental modelling software, 26(6), 787–796. bello-dambatta, a., farmani, r., javadi a.a., & evans b.m. (2009). the analytical hierarchy process for contaminated land management. advanced engineering informatics, 23, 433–41. bottero, m., comino, e., & riggio, v. (2011). application of the analytic hierarchy process and the analytic network process for the assessment of different wastewater treatment systems. environmental modelling software, 26(10), 1211–1224. carreño, m.l., cardona, o.d., barbat, a.h. (2007) a disaster risk management performance index. natural hazards, 41(1), 1–20. chen, g; tao, l; zhang, h. study on the methodology for evaluating urban and regional disasters carrying capacity and its application. safety science, 4 (1), 50–58. contreras, f., hanaki, k., aramaki, t., & connors s. (2008). application of analytical hierarchy process to analyze stakeholders preferences for municipal solid waste management plans, boston, usa. resources conservation and recycling, 52, 979–991. dong, y.c., zhang, g.q., hong, w.c., & xu, y.f. (2010). consensus models for ahp group decision making under row geometric mean prioritization method. decision support systems, 49, 281–289. frondel, m., horbach, j., & rennings, k. (2004). what triggers environmental management and innovation? empirical evidence for germany. rwi: discussion papers, 15, 1–29. gao, l. & hailu, a. (2012). ranking management strategies with complex outcomes: an ahp-fuzzy evaluation of recreational fishing using an integrated agent-based model of a coral reef ecosystem. environmental modelling software 31, 3–18. garuti a., c. & salomon, v.a.p. (2011) compatibility indices between priority vectors. international journal of the analytical hierarchy process, 4(2), 153–160. humphrey, a. s. (2005) swot analysis for management consulting. sri alumni association newsletter, 7–8. larrubia, c.b. (2010) ahp as a decision support tool for the maintenance of the water reservoir of ilha solteira. master’s degree dissertation, civil engineering graduate program. ilha solteira: sao paulo state university. [in portuguese]. macginty, r., carrasco, r., oddershede, a. & vargas, m. (2013). strategic foresight using an analytic hierarchy process: environmental impact assessment of the electric ijhap article: sabia, salomon, sobreira junior, marins, lima/ determination of the most appropriate policy for river basin management international journal of the analytic hierarchy process 210 vol. 7 issue 2 2015 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v7i2.263 grid in 2025. international journal of the analytical hierarchy process, 5(2), 186–199. ok, k., okan, t., & yilmaz, e. (2011). a comparative study on activity selection with multi-criteria decision-making techniques in ecotourism planning. scientific research and essays, 6(6), 1417–1427. ryu, j., leschine, t. m., nam, j., chang, w. k., & dyson, k. (2011). a resilience-based approach for comparing expert preferences across two large-scale coastal management programs. journal of environmental management, 92 (1), 92–101. saaty t.l. (1977) a scaling method for priorities in hierarchical structures. journal of mathematical psychology, 15, 234–281. saaty t.l. (2008) decision making with the analytic hierarchy process. international journal of service sciences, 1(1), 83–98. saaty, t.l. (2014) the neural network process (nnp). pittsburgh: rws publications. samari, d., azadi, h., zarafshani, k., hosseininia, g., & witlox, f. (2012). determining appropriate forestry extension model: application of ahp in the zagros area, iran. foreign policy economics, 15, 91–97. schmoldt, d.l., kangas, j., mendoza, g.a., & pesonen, m. (2001, ed.). the analytic hierarchy process in natural resource and environmental decision making. dordrecht: kluwer. tahriri, f., osman, m.r., ali, a., & yusuff, r. m. (2008). a review of supplier selection methods in manufacturing industries. suranaree journal of science technology, 15, 201– 8. vidal, l.a., marle, f., & bocquet, j.c. (2011). using a delphi process and the analytic hierarchy process (ahp) to evaluate the complexity of projects. expert systems of applications, 38, 5388–5405. wijnmalen, d.j.d. (2007) analysis of benefits, opportunities, costs, and risks (bocr) with the ahp–anp: a critical validation. mathematical and computer modelling, 46 (7–8), 892-905. xie, f. & tang, d.s. (2010) the application of ahp and fuzzy comprehensive evaluation in harmonious level measurement between human and water in city. journal of computer information systems, 6, 4647–4656. yavuz, f., & baycan, t. (2014), application of combined analytic hierarchy process (ahp) and swot for integrated watershed management. international journal of the analytical hierarchy process, 6(1), 3–32. the environment effect on vendor selection ijahp article: shih/ environmental impact on the vendor selection problem in electronics firmsa systematic analytic network process with bocr international journal of the analytic hierarchy process 202 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.256 environmental impact on the vendor selection problem in electronics firms a systematic analytic network process with bocr hsu-shih shih* 1 department of management sciences, tamkang university tamsui, new taipei 25137, taiwan, r.o.c. hshih@mail.tku.edu.tw chi-bin cheng department of information management, tamkang university tamsui, new taipei 25137, taiwan, r.o.c. cbcheng@mail.tku.edu.tw chiau-ching chen department of management sciences, tamkang university tamsui, new taipei 25137, taiwan, r.o.c. fermat123456789@gmail.com yi-chun lin department of management sciences, tamkang university tamsui, new taipei 25137, taiwan, r.o.c. jcsiawase@hotmail.com abstract as environmental deterioration and global warming become issues that receive more attention, this study investigates the environmental impact on the vendor or supplier selection problem. after re-examining the existing selection process, the processes for qualification and final selection are highlighted by considering environmental (or green) and traditional criteria. next, the analytic network process (anp) is adopted with the merits of benefits, opportunities, costs, and risks (bocr) to assess all factors for ranking suppliers for the certain and uncertain effects as well as the dependences and feedback. in addition, this study integrates the content validity ratio and factor analysis into the solution procedure for the benefit of anp analysis. lastly, a case of selecting suppliers for an electronics firm in taiwan is illustrated. the results show that the criteria of greatest concern are consistency delivery, flexibility, and cost, whereas environmental criteria are valued less. 1 the authors gratefully acknowledge the support from pegatron corp., lite-on semiconductor corp. and asustek computer inc. in taiwan on this study. ijahp article: shih/ environmental impact on the vendor selection problem in electronics firmsa systematic analytic network process with bocr international journal of the analytic hierarchy process 203 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.256 key words: analytic network process; anp; environmental management; supplier selection; benefits, opportunities, costs, and risks; bocr; qualification. 1. introduction as environmental deterioration and global warming become issues that receive more attention, business enterprises, especially in the electronics industry, have undertaken environmental plans inside and outside their corporate organizations. by way of infusing environmental awareness into corporate goals, firms can propagate the importance of environmental protection from the top down and combine green objectives with corporate functions, such as procurement, design, manufacturing, and marketing, so as to set forth comprehensive environmental activities for the making of true green enterprises (chen et al., 2012; chiou and tzeng, 2002). among various corporate mechanisms, procurement easily takes up more than half of a firm’s total expenditures (leenders et al., 2002), especially when it comes to original design/equipment manufacturing (odm/oem) or electronics manufacturing service (ems) (huang and keskar, 2007). companies with limited resources usually focus on their core business competitiveness, while handling their non-focused peripheral businesses through a variety of outsourcing. as a result, collaboration between buyers and suppliers is becoming greater (green et al., 1996; wadhwa and ravindram, 2007; lee et al., 2009). with cooperative activities of outsourcing and strategic alliances now more common, the relationship between companies and their suppliers has transformed from the traditional antagonistic one into a mutual commitment. therefore, companies must be prudent in the selection of suppliers in order to formulate a win-win situation (sonmez, 2006; tahriri et al., 2008; ting and cho, 2008). in response to environmental issues being taken more seriously, proposals on the selection of green suppliers have involved the use of environmental or green criteria (such as environmental certification and environmental management system) being appended to traditional evaluation criteria such as prices and quality (handfield et al., 2002; humphreys et al., 2003; lee et al., 2009; hsu and hu, 2009; tuzkaya et al., 2009). however, it seems that these studies over-estimate the effects of environmental issues by adding many environmental related criteria on supplier selection. in practice, traditional criteria may still dominate the choice of suppliers as supported by a recent survey (genoverse et al., 2013). therefore, a motivation of this study is to explore when and how to use environmental and traditional criteria in the supplier selection process. upon further investigation we find that research into the process of supplier ijahp article: shih/ environmental impact on the vendor selection problem in electronics firmsa systematic analytic network process with bocr international journal of the analytic hierarchy process 204 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.256 or vendor selection is limited. de boer et al., (2001) classified the selection process as a problem definition, formulation of criteria, qualification, and final choice. in practice, it is very likely that environmental guidelines will be used for qualification, and those firms that cross the threshold can get into the final choice stage. it seems that these studies simplify the process. after reviewing the environmental criteria, these criteria can be further grouped into two types: mandatory and voluntary. the former includes rohs (restriction of hazardous substances directive), weee (waste electrical and electronic equipment directive), etc., which firms are expected to obey through regulations and laws. the latter comprises epeat (electronic product environmental assessment tool), iso 14000 standards, etc., which demonstrate the efforts of suppliers. thus, “deep green” enterprises are requesting more of such voluntary criteria than “light green” enterprises. these two types of environmental criteria  mandatory and voluntary  play a central part in the stages of qualification and final choice of the vendor selection process. due to the multi-dimensional aspects or criteria being evaluated, past studies generally view supplier selection as a mcdm problem (ho et al., 2010), and many techniques can attack this problem, i.e., multi-attribute decision making, multi-objective decision making, statistical approaches, intelligent approaches, and others (bruno et al., 2012; de boer et al., 2001; genoverse et al., 2013; ho et al., 2010; shyur and shih, 2006; wu and barnes, 2011). most recent approaches concentrate on a combination of different techniques for different solution procedures, e.g., anp with data envelopment analysis (kuo and lin, 2012). among these techniques, the analytic hierarchy/network process (ahp/anp) is extensively used, because it is able to handle tangible and intangible factors. moreover, the new concept from the merits of benefits, opportunities, costs, and risks (bocr) assesses all factors and interests involving certain and uncertain effects (saaty, 2005). these advantages make the anp bocr model a suitable tool for evaluation (bottero and ferretti, 2010) and are especially suitable for assessing intangible environmental criteria. in addition, the traditional anp procedure relies on brainstorming to clarify some key issues and appears to be a loose structure to attack the problem. thus, the content validity ratio and factor analysis from statistics are integrated into the procedure for the benefit of anp analysis. we believe this integrated network model provides another option for selecting and ranking suppliers. ijahp article: shih/ environmental impact on the vendor selection problem in electronics firmsa systematic analytic network process with bocr international journal of the analytic hierarchy process 205 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.256 this study is organized as follows. section 2 reviews the related literature. section 3 proposes our methodology. section 4 exhibits a case through the systematic anp bocr model with sensitivity analyses. finally, this study draws forth conclusions. 2. literature review vendor or supplier selection is a common dilemma when trying to obtain the necessary materials for operations. the problem involves a process of decision making along with some analytic methods. we now discuss the procedures, the criteria for supplier selection under an environmental impact, and the main methodology used in the study  the analytical network process with bocr model. 2.1 purchasing procedures most previous research studies have directly analyzed the final choice of suppliers as a supplier selection problem. it is obvious that the purchasing decision should go through some steps prior to the final choice. dobler and burt (1996) listed a typical purchasing cycle as: (i) recognize, define, and describe the need; (ii) transmit the need; (iii) investigate, qualify, and select the supplier; (iv) prepare and issue the purchase order; (v) follow up on the order; (vi) receive and inspect the material; (vii) audit the invoice; and (viii) close the order. de boer et al. (2001) also mentioned that a completed supplier selection should be divided into: problem definition, criteria formulation, supplier qualification and final choice. in the qualification stage, potential suppliers must equal or exceed the basic requirement for screening and become an acceptable supplier. an order winner is then chosen by some characterized selection criteria. thus, supplier selection is not a single work of selection. indeed, there should be various activities in the purchasing process (chopra and meindl, 2007). after environmental issues were considered in selecting suppliers, most studies have concentrated on supplier assessment, e.g., bai and sarkis (2010). from their viewpoint, the evaluation is simultaneously based on traditional and environmental criteria as shown on path (a) in figure 1. however, after examining most purchasing procedures of the electronics industry in taiwan, environmental criteria is used for qualification, and the final choice of suppliers only depends on traditional criteria whose procedure is presented on path (b) of figure 1. humphreys et al. (2003) and nicosia and moore (2006) supported this viewpoint with some complex phases. it seems that the above two viewpoints, (a) and (b), are not the whole story of supplier selection. for instance, some environmental criteria could be utilized for the purpose ijahp article: shih/ environmental impact on the vendor selection problem in electronics firmsa systematic analytic network process with bocr international journal of the analytic hierarchy process 206 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.256 of screening, e.g., rohs with yes/no responses. hence, we extend noci’s argument (1997) and propose a procedure for enhancing the environmental issues and allocating different types of the criteria as shown on path (c) of figure 1. the difference among the mentioned paths is one concern of this study. problem formulation formulation of criteria qualification final selection (a) (b) (c) figure 1. different paths of supplier selection processes note: (a) most recent studies investigate the selection process by considering traditional and environmental criteria simultaneously at the final step. (b) the current process in taiwan’s electronics industry takes environmental criteria for qualification and then uses traditional criteria for the final selection. (c) the proposed process considers both traditional and environmental criteria for the steps of qualification and final selection. 2.2 supplier selection criteria supplier selection criteria can be divided into two types: mandatory and voluntary. for environmentally concerned enterprises, mandatory environmental and traditional criteria are required and generally considered for the purpose of qualification. for instance, firms provide a checklist of specified items to make sure qualified suppliers meet them, and these suppliers are considered as candidate suppliers. these items, which are environment-related, industrial safe, or quality related, are presented in the forms of regulations or laws by the european union and the united states, proving especially vital for asian firms trying to export electronics merchandises to these countries. these environmental criteria, which are expected to be obeyed, include rohs, weee, european union’s ecodesign directive, etc. these works are generally executed in the qualification step of figure 1. voluntary environmental criteria encompass epeat, eu eco-label flower, iecq (international electrotechnical commission quality assessment system for electronic components) qc 080000, iso 14000 standards, etc., which demonstrate efforts put forth by suppliers. these criteria can be seen as an achievement of the suppliers, and the more included the better. however, “deep green” enterprises may consider some of them as mandatory criteria that will be conducted in the qualification step, e.g., iso 14000. in addition, genovese et al. (2013) presented corporate social responsibility (csr) criteria, which ijahp article: shih/ environmental impact on the vendor selection problem in electronics firmsa systematic analytic network process with bocr international journal of the analytic hierarchy process 207 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.256 are generally at the strategic level for large enterprises and can be taken as an intangible factor for assessing suppliers. the criteria of cost/price, delivery, and quality have been traditionally valued the highest in the past, whereas the criteria of service, responsiveness, and environmental considerations are more important now. dickson first suggested 23 criteria for vendor selection (see weber et al., 1991). choi and harley (1996) explored 26 evaluation criteria for the car manufacturing industry and focused on the different positions of suppliers in the supply chain, pointing out that quality and delivery are important. barbarosoglu and yazgac (1997) structured a four-level supplier performance assessment structure and emphasized the importance of quality and different costs. leender et al. (2002) organized vendor performance by quality, price, delivery, and service. gencer and gürpinar (2007) considered business structure, manufacturing capability, and quality system as three clusters in a network for a case study on an electronics firm. yang et al. (2008) selected quality, price and terms, supply chain support, and technology as the major criteria for analyzing vendor selection for the electronics industries in taiwan. environmental criteria for assessing suppliers’ performance have risen in demand. noci (1997) suggested four major environmental criteria with corresponding indicators for supplier evaluation. handfield et al. (2002) identified the top 10 criteria for environmental performance. humphreys et al. (2003) proposed an almost full structure for the supplier selection process with a couple of stages on quantitative and qualitative as well as traditional and environmental criteria. huang and keskar (2007) appended safety and environmental metrics to the supply chain council’s scor (supply chain operations reference) model in practice. kuo and lin (2012) considered four criteria in the environment dimension for evaluation. büyüközkan and çifçi (2012) looked at green suppliers’ evaluation criteria and organizational performance (traditional criteria) as two clusters in a network for evaluating green suppliers as a whole. we know that environment-related criteria are becoming more significant from these studies, and these mandatory and voluntary criteria are presented in the forms of qualified criteria, performance measures, or cooperation image, which can be evaluated during the steps of qualification and selection. 2.3 analytical methods and anp with bocr the anp methodology is a special case of ahp, which deals with dependence and feedback among clusters and their elements in addition to a network structure (saaty ijahp article: shih/ environmental impact on the vendor selection problem in electronics firmsa systematic analytic network process with bocr international journal of the analytic hierarchy process 208 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.256 2005). traditionally, ahp/anp is analyzed by using a single hierarchy/network, but such an approach is weaker at presenting a negative effect. saaty (1980) directly considered a cost-benefit analysis by ahp, representing the analysis with two hierarchies. focusing on the alternatives, we divide ahp’s benefit priorities by ahp’s cost priorities to get the benefit-cost ratios of the alternatives for the final choice. saaty and özdemir (2003) added the positive aspect of opportunities and the negative aspect of risks to shape the prototype of bocr for a comprehensive analysis. through the bocr aspects we are able to understand the items of benefits associated with positive and certain contents, the items of opportunities with positive and uncertain contents, the items of costs with negative and certain contents, and the items of risks with negative and uncertain contents. for ranking and selection, our target is to obtain greater benefits and opportunities, while encountering smaller costs and risks. in addition, different decision makers may give varying degrees of importance to each bocr aspect, depending on strategic issues or goals. therefore, saaty and özdemir (2005) suggested that individuals or enterprises make pairwise comparisons under the goals in order to obtain bocr’s weights (i.e., b, o, c, and r, correspondingly). for synthesizing these four aspects, saaty (2005) further recommended the formula bb+ooccrr for the bocr calculation. here, b, o, c, and r are the priorities of the respective aspects, and b, o, c, and r are their weights, respectively. if the resulting value is positive, then it indicates the positive value is higher than the negative value and the choice turns out to be appropriate. bottero and ferretti (2011) pointed out that the anp bocr model helps decision makers to venture deeper into the uncertainty of the problems. hence, we choose this model for the analysis. after examining the anp bocr model that has been used in the literature since 2003, we found that many studies only provided a simplified model that might not accommodate the complex environment effect. table 1 collects some typical applications of the anp bocr model in the literature, in which full bocr networks are preferred as in bottero and ferretti (2011). the contents of table 1 illustrate the guide for researchers to use the model. in addition, the content validity ratio and factor analysis are adopted to support its solution procedure for a better coherence and rigorous process. ijahp article: shih/ environmental impact on the vendor selection problem in electronics firmsa systematic analytic network process with bocr international journal of the analytic hierarchy process 209 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.256 table 1 some typical bocr configurations in anp application bocr configuration authors (year) remarks selection of a high-tech transaction system four alternativescriteriaactors connections erdoğmuş et al. (2005) four criteria cluster in the bocr aspects, respectively evaluation of alternative fuels for residential heating 1. three alternativescriteriaactors connections 2. six clusters connected network in the benefits aspect erdoğmuş et al. (2006) different criteria in each criteria cluster of the ocr aspects supplier selection for the plastic parts of a refrigerator plant 1. benefits: 3 clusters connected network 2. opportunities: 2 clusters 3. costs: 2 clusters 4. risks: 2 clusters ustun and demirtas (2008) priority determination in strategic energy policies four criteria, participants, and policies connected networks dağdeviren and eraslan (2008) different criteria in each criteria cluster in the bocr aspects evaluation of buyer-supplier relationships in the high-tech industry single hierarchy including bocr four aspects at one level with sub-criteria showing dependence lee et al. (2009) single hierarchy oriented project prioritization in higher education institutions 1. benefits: 4 clusters connected network 2. costs: 3 clusters connected network 3. risks: 4 clusters connected network begičević et al. (2010) in the bcr aspects only selection of dispatching rules in fms sub-criteria connected sub-networks under the criteria of economy, strategy, and customer, respectively, at one level in bocr’s four hierarchies yazgan et al. (2010) four-hierarchy oriented location selection for a waste incinerator plant 1. benefits environment: 2 clusters; socio-economy: 2 clusters 2. opportunities environment: 3 clusters connected network; socio-economy: 2 clusters 3. costs environment: 5 clusters connected network; socio-economy: 4 clusters connected network 4. risks environment: 4 clusters connected network; socio-economy: 3 clusters connected network bottero and ferretti (2011) different criteria in two clusters in the bocr aspects site selection for coastal oil jetties single network including bocr’s four clusters and alternative hasanzadeh et al. (2013) single network oriented ijahp article: shih/ environmental impact on the vendor selection problem in electronics firmsa systematic analytic network process with bocr international journal of the analytic hierarchy process 210 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.256 3. research methodologies we mainly used the anp bocr model to simultaneously consider environmental and traditional elements for supplier selection and focused on the illustration of the last step due to a simple work on the yes/no questions in the qualification step. the suggested research framework involved two phases: obtaining a validated anp with bocr merits with four sub-networks, and designing the questionnaires of pairwise comparisons with calculation. figure 2 illustrates the proposed systematic anp bocr process with seven steps. 3.1 construction of the anp bocr network for the evaluation stage, we checked all organized criteria by the content validity ratio (cvr) to fit the aspects of benefits, opportunities, costs, and risks, respectively. next, we executed factor analysis to determine groups of the criteria and the clusters they belonged to. after identifying the relations between the dependence and feedback relations between clusters and criteria, we established four bocr sub-networks. to determine the selected criteria under the individual aspects of benefits, opportunities, costs, and risks, we adopted cvr as proposed by lawshe (1975) to delete minor criteria in order to enhance the suitability of the concerned criteria in each aspect. the cvr formula quantifies the agreement on the questionnaire content from a professional perspective. the quantified value is between 1 and -1, and the larger the value is, the more important the topics are regarded to be. cvr = (ne  n/2) / (n/2), (1) where ne = number of experts who think the topics are important (or essential), and n = total number of experts who fill out the questionnaire. when there are 5 experts filling out the questionnaire (n=5), the content validity ratio (cvr) derived from the above equation is 0.99; when the number of experts is 30 (n=30), cvr is 0.33. ijahp article: shih/ environmental impact on the vendor selection problem in electronics firmsa systematic analytic network process with bocr international journal of the analytic hierarchy process 211 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.256 we used the five-level scale for the evaluation. in the effectiveness verification, we only picked items with 4 or 5 points for the new calculation. moreover, the cvr value of one question higher than the specified cvr value determined by n (the number of figure 2. proposed systematic anp bocr process (i) clarify the goal of the problem (ii) sort out supplier evaluation criteria step 1 (i) take questionnaire survey to identify effective criteria with the content validity ratio (cvr) (ii) utilize factor analysis (fa) to construct the clusters and their elements of bocr sub-networks step 2 (i) construct supermatrices (ii) compute the limiting priorities (iii) synthesize the priorities of bocr aspects sub-networks step 3 (i) take questionnaire survey to determine the relations of dependence and feedback in each sub-network (ii) perform expanded pair-wise comparisons (iii) obtain relative intensities of clusters and their criteria step 4 sensitivity analysis step 6 step 5 step 7 rank and select the outcomes robust decisions no yes ijahp article: shih/ environmental impact on the vendor selection problem in electronics firmsa systematic analytic network process with bocr international journal of the analytic hierarchy process 212 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.256 experts filling out the questionnaire) was kept. next, we executed factor analysis on the criteria of each aspect to obtain the corresponding clusters or dimensions that these criteria belong to for the anp network. the questionnaires sent by both paper and email asked the respondents to evaluate the level of importance of each criterion in the aspect of benefits, opportunities, costs, and risks, respectively. tabachnick and fidell (2007) suggested that factor analysis should have a minimum number of samples that are five times the number of questions in a questionnaire as shown in the appendices. before factor analysis, we performed two tasks on the suitability for further analysis. first, we implemented the kaiser-meyer-olkin (kmo) measure of sampling adequacy and the bartlett test of sphericity on the questionnaires to make sure the questionnaire items were appropriate for factor analysis. according to kaiser (1970), the value of kmo is between 0 and 1, and the greater the value is, the more appropriate the data are. in addition, if the p-value resulting from the bartlett test is less than the significant level, then the data variables are correlated and suitable for factor analysis. after reducing the criteria data, all the kmo values are greater than 0.6, and the p-values are less than 0.05, which supports the assumption that the data are appropriate for factor analysis. the clusters can then be determined from various criteria by factor analysis. following the clusters and their criteria being chosen in each aspect, the experts then decided upon the dependence and feedback relations between clusters and criteria. the results showed that more than half of the experts agreed that the factors influence one another, supporting the assumption that the factors are dependent. in our study we considered environmental and traditional criteria for the two phases as path (c) in figure 1. a pairwise comparison was performed on these criteria under the merits of benefits, opportunities, costs, and risks. the bocr-anp network obtained from expert opinions is shown in figure 3 for our case. ijahp article: shih/ environmental impact on the vendor selection problem in electronics firmsa systematic analytic network process with bocr international journal of the analytic hierarchy process 213 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.256 figure 3. anp bocr’s network 3.2 expert questionnaires for the anp bocr model the second phase was to design the pairwise comparison questionnaires to form the supermatrices and to execute their calculation. 3.2.1 design of the questionnaires we designed pairwise comparison questionnaires for four aspects of bocr. there were two-stage questionnaires for the dependent relations among clusters and their elements, weights among clusters, and intensity of influence among elements. 3.2.2 pairwise comparisons and group aggregation r1 alternatives r1.1 x r1.2 y r1.3 z b5 product assurance b5.1 probability of customer complaints b5.2 improvement and crisis management of customer complaints b5.3 quality b5.4 design capability b5.5 equipment and technology b1 alternatives b1.1 x b1.2 y b1.3 z b4 finance b4.1 financial performance b3 production responsiveness b3.1 consistent delivery b3.2 flexibility b2 environment b2.1 green image b2.2 environmental plans b2.3 compliance with environmental directive o5 product internal support o5.1 probability of customer complaints o 5.2 improvement and crisis management of customer complaints o 5.3 design capability o 5.4 equipment and technology o1 alternatives o1.1 x o1.2 y o1.3 z o4 customer promise o4.1 quality o4.2 consistent delivery o3 flexibility o3.1 flexibility o2 environment o2.1 green image o2.2 green competitiveness o 2.3 environmental plans o 2.4 compliance with environmental directive c5 service quality c5.1 consistent delivery c5.2 probability of customer complaints c5.3 improvement and crisis management of customer complaints c1 alternatives c1.1 x c1.2 y c1.3 z c4 price c4.1 price c3 production line control c3.1 quality c3.2 flexibility c2 environment c2.1 compliance with environmental directive r4 product reliability r4.1 consistent delivery r 4.2 probability of customer complaints r4.3 quality r4.4 improvement and crisis management of customer complaints r3 flexibility r3.1 flexibility r2 environment r2.1 pollution/waste management r 2.2 compliance with environmental directive ijahp article: shih/ environmental impact on the vendor selection problem in electronics firmsa systematic analytic network process with bocr international journal of the analytic hierarchy process 214 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.256 an analysis on the questionnaires filled out by the experts was first given, and we created a pairwise comparison matrix a by pairwise comparing the n elements of e1, e2,…, en and checking its consistency. after observing that different experts might give different answers to the questionnaire, a geometric mean operation is a commonly used method for the aggregation of experts’ opinions (saaty, 1989). in addition, a satisfaction index proposed by huang et al. (2009) can be used for measuring the group opinion to improve the decision quality. 3.2.3 construction of supermatrices and computation of the limiting priorities the above priorities of each matrix were then arranged, representing the impact of a given set of elements in a cluster on another element in the network, as the sub-columns of the corresponding column of an unweighted supermatrix w. we then normalized and synchronized each column of the unweighted supermatrix to record the overall clusters’ influence by column, and this operation made the supermatrix’s column stochastic, as a weighted supermatrix w a . we next obtained the limiting supermatrix w n = a lim w a . the derived weights were then employed to weight the elements of the corresponding column blocks of the weighted supermatrix (saaty, 2005). 3.2.4 syntheses of the priorities of bocr since there are four aspects to be considered, we synthesized the values from these four. the major concern was the supplier cluster, which meant we wanted to know which supplier (as the elements in the alternative cluster) is a better one. each alternative obtains the priority values under the bocr aspects to represent the importance of that aspect. we know that when bb+ooccrr > 1, it means the target is preferred, and when bb+ooccrr  1, it means the target’s benefits and opportunities are threatened by the costs and risks (saaty, 2005). 4. case study 4.1 background information the company in this case study, company p hereafter, provides professional ems and has already participated in environmental projects and actions for many years. it sets up environmental policies and green standard procedures through clean production, green purchasing, and green manufacturing, which include product designs, components’ procurements, production and manufacturing, and final customer services. having received many international environmental certifications, including green partner, ijahp article: shih/ environmental impact on the vendor selection problem in electronics firmsa systematic analytic network process with bocr international journal of the analytic hierarchy process 215 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.256 ohsas (occupational health and safety advisory services) 18001, iso 14001, and iecq qc 080000, company p has put forth much effort to reach the goal of “zero pollution, zero accidents” via greenhouse gas emissions and the setting up of reduction targets, environmental monitoring, waste, wastewater, hazardous chemicals, energy-efficiency management, and environmental requirements upon its suppliers. adhering to the electronic industry code of conduct (eicc), company p actively attends eicc gatherings in order to understand sustainability-related issues and the latest development trends and to respond to corporate social responsibility (csr) projects, such as supply chain csr management, voluntary carbon reduction action, and so on. for its manufacturing process, company p has instituted suppliers/contractors management, which asks suppliers to conform to its green purchasing polices and environmental requirements’ approval-related information on a supplier relation management website and egreen management platform. furthermore, it executes a quarterly business review to evaluate potential suppliers that qualify under its own environmental standards. the case study involved the purchasing of laptop power adapters. since environmental criteria are rather diverse for different purposes, we organized the criteria from the above works (sections 2.1 and 2.2) after the qualification step on the “deep green” aspect. we then sent these criteria to domain experts in the purchasing departments of electronics firms in taiwan. there are 18 criteria, 6 environmental-related criteria and 12 traditional criteria, for future development. table 2 lists these criteria. moreover, after screening the potential suppliers through traditional and environmental criteria, three suppliers denoted by x, y, and z were left in the qualified vendor list, and further analysis was conducted to choose a better supplier. ijahp article: shih/ environmental impact on the vendor selection problem in electronics firmsa systematic analytic network process with bocr international journal of the analytic hierarchy process 216 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.256 table 2 supplier selection criteria used in the study traditional criteria environmental criteria 1. consistent delivery 1. green image 2. probability of customer complaints 2. green competitiveness 3. improvement and crisis management of customer complaints 3. environmental plans 4. quality 4. pollution/waste management 5. design capability 5. compliance with environmental directives 6. equipment and technology 6. green design capability 7. flexibility 8. price 9. relationship with buyers 10. image 11. organisational management and size 12. financial performance 4.2 construction of the analytical network to make sure the criteria listed in table 2 are valid for our study, we checked their cvr. the questionnaires were sent out to 12 experts in the departments of purchasing, research and development, marketing, and quality assurance in company p. all of them were returned, for a response rate of 100%. when n=12, only questions with cvr values higher than 0.56 were kept, with the rest deleted. table 3 shows the valid criteria under bocr. table 3 validated supplier selection criteria under bocr merits benefits opportunities costs risks traditional criteria 1. consistent delivery 1. consistent delivery 1. consistent delivery 1. consistent delivery 2. probability of customer complaints 2. probability of customer complaints 2. probability of customer complaints 2. probability of customer complaints 3. improvement and crisis management of customer complaints 3. improvement and crisis management of customer complaints 3. improvement and crisis management of customer complaints 3. improvement and crisis management of customer complaints 4. quality 4. quality 4. quality 4. quality 5. design capability 5. design capability 7. flexibility 7. flexibility 6. equipment and technology 6. equipment and technology 8. price 8. price 7. flexibility 7. flexibility 12. financial performance environmental criteria 1. green image 3. environmental plans 1. green image 2. green competitiveness 5. compliance with environmental directives 4. pollution/waste management 5. compliance with environmental directives 5. compliance with environmental directives 5. compliance with environmental directives 6. green design capability next, questionnaires used for factor analysis were sent to employees in the electronics ijahp article: shih/ environmental impact on the vendor selection problem in electronics firmsa systematic analytic network process with bocr international journal of the analytic hierarchy process 217 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.256 industry in taiwan for them to evaluate the level of importance of each criterion in the bocr aspects. this study received 93 copies of the questionnaire back, with 81 effective copies, i.e., an 87.1% effective response rate. the results derived from the analysis encompass several clusters with various criteria in each aspect (figure 2). the dependence and feedback relations between clusters and their elements have been decided by five experts. figure 3 below shows the resulting four sub-networks. 4.3 integration and analysis of the expert questionnaires we chose three experts from purchasing, research and development, and sales and marketing of company p for the study. because the paired comparisons in anp are not as common as the five-level scale questionnaire, we provided an example posited before the pair comparisons in order to clarify any possible misunderstanding in response to the following paired comparison. the experts’ responses were aggregated by geometric mean to obtain the group information with a satisfactory level. the priorities of all compared matrices form the unweighted supermatrices in the bocr aspects. after normalizing and synthesizing to record the overall clusters’ influence by column (in table 4), the weighted supermatrices in the bocr were established respectively. the weighted supermatrix was then multiplied by itself over and over until the elements in the matrix reached a stable convergence condition, which are the limiting supermatrices as shown in table 5 under benefits. table 4 the weight of each cluster under the aspect of benefits alternatives environment production responsiveness finance product assurance alternatives 0.0000 0.5000 0.4000 1.0000 0.5816 environment 0.0884 0.5000 0.0000 0.0000 0.0000 production responsiveness 0.4871 0.0000 0.2000 0.0000 0.3090 finance 0.0912 0.0000 0.4000 0.0000 0.0000 product assurance 0.3333 0.0000 0.0000 0.0000 0.1095 ijahp article: shih/ environmental impact on the vendor selection problem in electronics firmsa systematic analytic network process with bocr international journal of the analytic hierarchy process 218 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.256 table 5 limiting supermatrix under the aspect of benefits alternatives environment production responsiveness finance product assurance x y z b2.1 b2.2 b2.3 b3.1 b3.2 b4.1 b5.1 b5.2 b5.3 b5.4 b5.5 alternatives x 0.0754 0.0754 0.0754 0.0754 0.0754 0.0754 0.0754 0.0754 0.0754 0.0754 0.0754 0.0754 0.0754 0.0754 y 0.2975 0.2975 0.2975 0.2975 0.2975 0.2975 0.2975 0.2975 0.2975 0.2975 0.2975 0.2975 0.2975 0.2975 z 0.0495 0.0495 0.0495 0.0495 0.0495 0.0495 0.0495 0.0495 0.0495 0.0495 0.0495 0.0495 0.0495 0.0495 environment b2.1 green image 0.0213 0.0213 0.0213 0.0213 0.0213 0.0213 0.0213 0.0213 0.0213 0.0213 0.0213 0.0213 0.0213 0.0213 b2.2 environmental plans 0.0065 0.0065 0.0065 0.0065 0.0065 0.0065 0.0065 0.0065 0.0065 0.0065 0.0065 0.0065 0.0065 0.0065 b2.3 compliance with environmental directives 0.0255 0.0255 0.0255 0.0255 0.0255 0.0255 0.0255 0.0255 0.0255 0.0255 0.0255 0.0255 0.0255 0.0255 production responsiveness b3.1 consistent delivery 0.1132 0.1132 0.1132 0.1132 0.1132 0.1132 0.1132 0.1132 0.1132 0.1132 0.1132 0.1132 0.1132 0.1132 b3.2 flexibility 0.1585 0.1585 0.1585 0.1585 0.1585 0.1585 0.1585 0.1585 0.1585 0.1585 0.1585 0.1585 0.1585 0.1585 finance b4.1 financial performance 0.0951 0.0951 0.0951 0.0951 0.0951 0.0951 0.0951 0.0951 0.0951 0.0951 0.0951 0.0951 0.0951 0.0951 product assurance b5.1 probability of customer complaints 0.0220 0.0220 0.0220 0.0220 0.0220 0.0220 0.0220 0.0220 0.0220 0.0220 0.0220 0.0220 0.0220 0.0220 b5.2 improvement and crisis management of customer complaints 0.0170 0.0170 0.0170 0.0170 0.0170 0.0170 0.0170 0.0170 0.0170 0.0170 0.0170 0.0170 0.0170 0.0170 b5.3 quality 0.0227 0.0227 0.0227 0.0227 0.0227 0.0227 0.0227 0.0227 0.0227 0.0227 0.0227 0.0227 0.0227 0.0227 b5.4 design capability 0.0530 0.0530 0.0530 0.0530 0.0530 0.0530 0.0530 0.0530 0.0530 0.0530 0.0530 0.0530 0.0530 0.0530 b5.5 equipment and technology 0.0427 0.0427 0.0427 0.0427 0.0427 0.0427 0.0427 0.0427 0.0427 0.0427 0.0427 0.0427 0.0427 0.0427 4.4 syntheses of the priorities of bocr to supplier selection table 6 presents the final priorities or weights synthesized from bocr. this study adopted the bb+ooccrr equation to synthesize the priorities of the suppliers. the resulting sequence in descending order is supplier x, supplier y, and then supplier z. a radar chart shows the performance of criteria in each aspect in figure 4 so that we can visually check the importance of the specific criterion under all aspects. here, consistent delivery and flexibility were the major criteria in the bor aspects, and price was the major criterion in the cost aspect. 4.5 sensitivity analysis sensitivity analysis is a technique for systematically changing the elements to see if the final selection is affected. the target analyses are the changes in the priorities of the criteria of flexibility in the benefits aspect, the environment cluster in the benefits aspect, and the priority of the benefits aspect, respectively, with 50% of their original values. the results of the analysis showed that the rank of the suppliers is kept the same. figure 5 presents the result from a change in the priority on the selection of suppliers. thus, we believe that the model is rather stable. please refer to ijahp article: shih/ environmental impact on the vendor selection problem in electronics firmsa systematic analytic network process with bocr international journal of the analytic hierarchy process 219 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.256 step 6 in the flowchart of figure 2. table 6 synthesized priorities and the ranks of the suppliers weight suppliers b o c r synthesized priorities rank 0.3629 0.1630 0.3261 0.1480 x 0.1785 0.2195 0.3058 0.2888 -0.0419 2 y 0.7043 0.6431 0.4052 0.1154 0.2112 1 z 0.1172 0.1374 0.2890 0.5959 -0.1175 3 figure 4. radar chart of each criterion in bocr figure 5. sensitivity analysis on the change of the priority ijahp article: shih/ environmental impact on the vendor selection problem in electronics firmsa systematic analytic network process with bocr international journal of the analytic hierarchy process 220 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.256 4.6 discussion we discussed the analysis with the senior managers of the firm and received their responses. three key issues were identified. first, company p’s final decision is to allocate 65% of its procurement of laptop adapters to supplier y and the rest to supplier x. the decision is close to our analysis result, whose priority is suppler y  suppler x  supplier z, with regard to the second source in its procurement process. in addition, based on the different situations of purchasing components for the laptops, the senior managers think the model can be used for their general procedure, but would not fit the procedure of rush purchasing. second, company p with “deep green” orientation has forced higher environmental standards of mandatory and voluntary types on its suppliers at the qualification step, which include environmental regulations, directives, and certifications, and this may cause the analysis results to not tell the whole story when less environmental criteria are the focus in the selection step. as the environmental requirements are attracting more attention, the mandatory or voluntary environmental criteria will become more and more significant in the supplier selection process. the rest of the study tried to simplify the process and shorten the purchasing cycle. thus, the traditional criteria were the chief concerns for the firm’s business operations, but there will be a gap that needs improvement in the future. third, the past literature focuses on the importance of quality, providing a result not quite the same as the one from our study. it is not that company p has given up on product quality, but rather its three suppliers provide products with similar high quality, demonstrating that product quality has become a basic threshold in the selection of suppliers, and thus the quality factor is not manifested in our result. finally, theoretically, anp is more flexible than ahp in structuring our problem, especially concerning the importance on the dependence and feedback relations between cluster and criteria. however, we make a trade-off between these relations and complexity. some statistical tools are integrated into our solution procedure for reducing the complexity in using anp. 5. conclusions this study deals with the environmental effect on the vendor selection problem for electronics firms in taiwan through an anp bocr analysis. first, this study examined the different paths of supplier selection processes, finding that taiwanese electronics firms concentrate on using environmental criteria in the qualification step. the results show that the traditional criteria are still the main concerns, revealing the reality that taiwan’s electronics industry for ems is forced to take the traditional criteria into account for the final decision for market competition. this finding has ijahp article: shih/ environmental impact on the vendor selection problem in electronics firmsa systematic analytic network process with bocr international journal of the analytic hierarchy process 221 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.256 also been confirmed by the recent survey in the aerospace and the railway industries in the uk (genovese et al., 2013). however, this is not the right action for rewarding the environmental endeavors of suppliers. we recommend that business enterprises target more environmentally-relevant criteria in both the qualification step and the final choice step as better actions towards environmental protection. second, the final decision of the firm was close to our result in selecting vendors of laptop adapters. we believe that the proposed model is applicable to many other choices of supply. third and finally, anp is well-known for processing tangible and intangible factors with dependences and feedbacks. with the anp bocr model, we can assess all aspects for ranking suppliers. due to these bocr aspects, we clearly settle upon the criteria and compare them with the criteria of the same constructive aspects without being affected by other different aspects. furthermore, the content validity ratio and factor analysis are integrated into the procedure, making the model more accessible and the process more rigorous. our study illustrates these advantages when looking at supplier selection. ijahp article: shih/ environmental impact on the vendor selection problem in electronics firmsa systematic analytic network process with bocr international journal of the analytic hierarchy process 222 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.256 references bai, c., & sarkis, j. (2010). green supplier development: analytical evaluation using rough set theory. journal of cleaner production, 18, 1200-1210. barbarosoglu, g., & yazgac, t. (1997). an application of the analytic hierarchy process to the supplier selection problem. apics production and inventory management journal, 38(1), 14-21. begičević, n., divjak, b., & hunjak, t. (2010). decision-making on prioritization of projects in higher education institutions using the analytic network process approach. central european journal of operations research, 18(3), 341-364. bottero, m., & ferretti, v. (2011). an analytic network process-based approach for location problems: the case of a new waste incinerator plant in the province of torino (italy). journal of multi-criteria decision analysis, 17, 63-84. bruno, g., esposito, e., genovese, a., & passaro, r. (2012). ahp-based approaches for supplier evaluation: problems and perspectives. journal of purchasing and supply management, 18, 159-172. büyüközkan, g., & çifçi, g. (2012). a novel hybrid mcdm approach based on fuzzy dematel, fuzzy anp and fuzzy topsis to evaluate green suppliers. expert systems with applications, 39, 3000-3011. chen, c.-c., shih, h.-s., shyur, h.-j., & wu, k.-s. (2012). a business strategy selection of green supply chain management via an analytic network process. computers and mathematics with applications, 64(8), 2544-2557. chiou, h.-k., & tzeng, g.-h. (2002). fuzzy multiple-criteria decision-making approach for industrial green engineering. environmental management, 30(6), 816-830. choi, t.y., & hartley, j.l. (1996). an exploration of supplier selection practices across the supply chain. journal of operations management, 14, 333-343. chopra, s., & meindl, p. (2007). supply chain management, 3 rd ed. upper saddle river, nj: pearson. dağdeviren, m., & eraslan, e. (2008). priority determination in strategic energy policies in turkey using analytic network process (anp) with group decision making. international journal of energy research, 32, 1047-1057. de boer, l., labro, e., & morlacchi, p. (2001). a review of methods supporting supplier selection. european journal of purchasing and supply management, 7, 75-89. dobler, d.w., & burt, d.n. (1996). purchasing and supply management, 6th ed. new york: mcgraw-hill. erdoğmuş, ş., muzaffer, k., & koç, e. (2005). evaluating high-tech alternatives by using analytic network process with bocr and multiactors. evaluation and program planning, 28, 391-399. erdoğmuş, ş., aras, h., & koç, e. (2006). evaluation of alternative fuels for residential heating in turkey using analytic network process (anp) with group decision-making. renewable and sustainable energy reviews, 10, 269-279. ijahp article: shih/ environmental impact on the vendor selection problem in electronics firmsa systematic analytic network process with bocr international journal of the analytic hierarchy process 223 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.256 gencer, c., gürpinar, d. (2007). analytic network process in supplier selection: a case study in an electronic firm. applied mathematical modelling, 31, 2475-2486. genoverse, a., koh, s.c.l., bruno, g., & esposito, e. (2013). green supplier selection: state of the art and some empirical evidence. international journal of production research, 51(10), 2868-2886. green, k., morton, b., new, s. (1996). purchasing and environmental management: interactions, policies and opportunities. business strategy and the environment, 5(3), 188-197. handfield, r.b., walton, s.v., sroufe, r., & melnyk, s.a. (2002). applying environment criteria to supplier assessment: a study in the application of the analytic hierarchy process. european journal of operational research, 141, 70-87. hasanzadeh, m., danehkar, a., & azizi, m. (2013). the application of analytical network process to environmental prioritizing criteria for coastal oil jetties site selection in persian gulf coasts (iran). ocean and coastal management, 73, 136-144. huang, s.h., & keskar, h. (2007). comprehensive and configurable metrics for supplier selection. international journal of production economics, 105, 510-523. huang, y.s., liao, j.t., & lin, z.l. (2009). a study on aggregation of group decision. systems research and behavioral science, 26, 445-454. humphreys, p.k., wong, y.k., & chan, f.t.s. (2003). integrating environmental criteria into the supplier selection process. journal of materials processing technology, 138, 349-356. ho, w., xu, x., & dey, p.k. (2010). multi-criteria decision making approaches for supplier evaluation and selection: a literature review. european journal of operational research, 202, 16-24. hsu, c.w., & hu, a.h. (2009). applying hazardous substance management to supplier selection using analytic network process. journal of cleaner production, 17(2), 255-264. kaiser, h.f. (1970). a second generation little jiffy. psychometrika, 35, 401-415. kuo, r.j., & lin, y.j. (2012). supplier selection using analytic network process and data envelopment analysis. international journal of production research, 50(11), 852-2863. lawshe, c.h. (1975). a quantitative approach to content validity. personnel psychology, 28, 563-575. lee, a.h.i., chang, h.j., & lin, c.y. (2009). an evaluation model of buyer-supplier relationships in high-tech industry  the case of an electronic components manufacturer in taiwan. computers and industrial engineering, 57, 1417-1430. leenders, m.r., fearon, h.e., flynn, a.e., & johnson, p.f. (2002). purchasing and supply management. 11 th ed., boston: irwin. nicosia, n., & moore, n.y. (2006). implementing purchasing and supply chain management: best practices in market research. rand project air force, rand corporation, santa monica, ca. noci, g. (1997). designing green vendor rating systems for the assessment of a http://www.sciencedirect.com/science/journal/09596526 http://www.sciencedirect.com/science/journal/09596526/17/2 ijahp article: shih/ environmental impact on the vendor selection problem in electronics firmsa systematic analytic network process with bocr international journal of the analytic hierarchy process 224 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.256 supplier’s environmental performance. european journal of purchasing and supply management, 3(2), 103-114. saaty, t.l. (1980). the analytic hierarchy process. new york: mcgraw-hill. saaty, t.l. (1989). group decision making and the ahp, in golden, b.l. et al. (eds.) the analytic hierarchy process: applications and studies. springer-verlag, new york, pp. 59-67. saaty, t.l. (2005). theory and applications of the analytic network process: decision making with benefits, opportunities, costs, and risks. pittsburgh, pennsylvania: rws publications. saaty, t.l., & özdemir, m.s. (2003). negative priorities in the analytic hierarchy process. mathematical and computer modelling, 37(9-10), 1063-1075. saaty, t.l., & özdemir, m.s. (2005). the encyclicon: a dictionary of decisions with dependence and feedback based on the analytic network process. pittsburgh, pa: rws publications. shyur, h.j., & shih, h.s. (2006). a hybrid mcdm model for strategic vendor selection. mathematical and computer modelling, 44(7-8), 749-761. sonmez, m. (2006). a review and critique of supplier selection process and practice. business school occasional papers series paper 2006:1, loughborough, uk: loughborough university. stevenson, w.j. (2009). operations management, 10 th ed. new york, ny: mcgraw-hill/irwin. tabachnick, b.g., & fidell, l.s. (2007). using multivariate statistics, 5 th ed. boston, ma: pearson. tahriri, f., osman, m.r., & yusuff, r.m. (2008). a review of supplier selection methods in manufacturing industries. suranaree journal science technology, 15(3), 201-208. ting, s.c., & cho, d.i. (2008). an integrated approach for supplier selection and purchasing decisions. supply chain management, 13(2), 116-127. tuzkaya, g., ozgen, a., ozgen, d., & tuzkaya, u.r. (2009). environmental performance evaluation of suppliers: a hybrid fuzzy multi criteria decision approach. international journal of environment science and technology, 6(3), 477-490. ustun, o., & demirtas, e.a. (2008). an integrated multi-objective decision-making process for multi-period lot-sizing with supplier selection. omega, 36, 509-521. wadhwa, v., & ravindran, a.r. (2007). vendor selection in outsourcing. computers and operations research, 34(12), 3725-3737. weber, c.a., current, j.r., & benton, w.c. (1991). vendor selection criteria and methods. european journal of operational research, 50, 2-18. wu, c., & barnes, d. (2011). a literature review of decision-making models and approaches for partner selection in agile supply chains. journal of purchasing and supply management, 17, 256-274. yang, j.l., chiu, h.n., tzeng, g.h., & yeh, r.h. (2008). vendor selection by ijahp article: shih/ environmental impact on the vendor selection problem in electronics firmsa systematic analytic network process with bocr international journal of the analytic hierarchy process 225 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.256 integrated fuzzy mcdm techniques with independent and interdependent relationships. information sciences, 178, 4166-4183. yazgan, h.r., boran, s., & goztepe, k. (2010). selection of dispatching rules in fms: anp model based on bocr with choquet integral. international journal of advanced manufacturing technology, 49, 785-801. ijahp article: shih/ environmental impact on the vendor selection problem in electronics firmsa systematic analytic network process with bocr international journal of the analytic hierarchy process 226 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.256 appendix a questionnaire i benefit merit confirmation relationships between each cluster for vendor selection cluster y n 1-1 products assurance affects suppliers □ □ 1-2 products assurance affects finance □ □ 1-3 products assurance affects environment □ □ 1-4 products assurance affects production responsiveness □ □ 2-1 suppliers affects finance □ □ 2-2 suppliers affects environment □ □ 2-3 suppliers affects production responsiveness □ □ appendix b questionnaire ii benefit merit the importance of each criteria criteria extremely extremely disagree agree 1 2 3 4 5 1. consistent delivery: the ability of meeting delivery deadlines and arriving on time. □ □ □ □ □ 2. probability of customer complaints: the probability of the products providing to buyers didn’t satisfy requirements. □ □ □ □ □ 3. improvement and crisis management of customer complaints: the responsiveness of product improvements and customers complaints. □ □ □ □ □ ijahp article: shih/ environmental impact on the vendor selection problem in electronics firmsa systematic analytic network process with bocr international journal of the analytic hierarchy process 227 vol. 6 issue 2 2014 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v6i2.256 appendix c questionnaire iii benefit merit part a please make pair-comparisons on clusters for choosing an appropriate supplier q1：in product assurance cluster, please make pair-comparisons on each cluster cluster intensity cluster 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 product assurance □ □ □ □ □ □ □ □ □ □ □ □ □ □ □ □ □ suppliers product assurance □ □ □ □ □ □ □ □ □ □ □ □ □ □ □ □ □ production responsivenes s suppliers □ □ □ □ □ □ □ □ □ □ □ □ □ □ □ □ □ production responsivenes s part b please make pair-comparisons on elements for choosing an appropriate supplier q1-1：in probability of customer complaints factor, please make pair-comparisons on each criteria criteria intensity criteria 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 probability of customer complaints □ □ □ □ □ □ □ □ □ □ □ □ □ □ □ □ □ improvement and crisis management of customer complaints probability of customer complaints □ □ □ □ □ □ □ □ □ □ □ □ □ □ □ □ □ quality probability of customer complaints □ □ □ □ □ □ □ □ □ □ □ □ □ □ □ □ □ design capability probability of customer complaints equipment and technology ijahp article: serra costa, borges, machado/analytic hierarchy process applied to industrial location: a brazilian perspective on jeans manufacture international journal of the analytic hierarchy process 77 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.210 analytic hierarchy process applied to industrial location: a brazilian perspective on jeans manufacturing jose fabiano serra costa associated professor, statistics department rio de janeiro state university (uerj) brazil fabiano@ime.uerj.br aline ramos borges statistics department rio de janeiro state university (uerj) brazil alineborges_est@outlook.com thaís dos santos machado statistics department rio de janeiro state university (uerj) brazil thais.smachado@gmail.com abstract the textile sector is one of the most important among all industrial branches. it is important in world commerce because it is a highlight both in the economy of developed countries and in emergent countries. the purpose of this study is to help a brazilian textile company, which specializes in jeans production, to study the conditions for installing a plant in three possible regions where the most important textile companies of the country are located. in the study, the analytic hierarchy process (ahp), a multicriteria methodology for decision support, is used and results show a slight tendency toward the rio de janeiro state when compared to the santa catarina state, with both presenting a significant advantage over the rio grande do norte state. key words: textile industry; industrial location; multicriteria decision; ahp mailto:fabiano@ime.uerj.br mailto:alineborges_est@outlook.com mailto:thais.smachado@gmail.com ijahp article: serra costa, borges, machado/analytic hierarchy process applied to industrial location: a brazilian perspective on jeans manufacture international journal of the analytic hierarchy process 78 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.210 1. introduction the textile segment is one of the most traditional industrial branches, with significant relevance in the economy of both developed and emergent countries. these countries owe in large part the key role they play today in global manufacturing commerce to this sector. the textile production chain, including spinning, weaving, knitting, finishing/processing and manufacturing, has gone through several recent transformations. some of those are worth highlighting, especially those related to technological changes, which allowed for an expansive increase in productivity, and the growing importance of free trading groups, such as: nafta; european union; north africa and south asia, as india and pakistan; southeast asia and far east; mercosur and latin america (gorini, 2000). the brazilian textile industry first appeared at the end of the nineteenth century and made great advances with the increase of local consumption starting after world war i (stein, 1979). it has become very important in the development of the national industrial policy. currently, the sector includes large, small and micro companies, which can be characterized by their technological and managerial heterogeneity (melo et al., 2007). on the other hand, it is noted that due to technological innovations in the textile industrial park, there has been a significant impact on the workforce profile, requiring a higher level of qualification, which is an important aspect of competitiveness (melo et al., 2007). after more than forty years of import quotas, the textile and clothing sector will become subject to the general rules of the general agreement on tariffs and trade from 2005. liberalization has been controversial because both textiles and clothing contribute to employment in developed countries, particularly in regions where alternative jobs may be difficult to find (nordås, 2004). the goal of this study is to help a brazilian textile company, in the jeans segment, to evaluate the conditions for a plant installation using the analytic hierarchy process (ahp). three regions, where the most relevant textile companies in the country are concentrated, were considered in this analysis. there are several criteria that are fundamental in the analysis of possible locations to install a manufacturing plant, such as the availability of workforce and feedstock to be processed into final or intermediate products. the local infrastructure must also be studied in order to determine the possibility of economy of scale and demand availability to assure that the production reaches the determined end destination (sousa, 2004). finally, financing options should be evaluated to guarantee that the activities will be viable through a pre-determined timeline. ijahp article: serra costa, borges, machado/analytic hierarchy process applied to industrial location: a brazilian perspective on jeans manufacture international journal of the analytic hierarchy process 79 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.210 a review of recent literature (after the year 2000) discovered some articles that apply the analytic hierarchy process to the textile/apparel industry, which emphasizes the current importance and the applicability of this model (table 1). table 1 recent literature relating ahp to the textile industry title author(s) fuzzy ahp-based decision support system for selecting erp systems in textile industry by using balanced scorecard cebeci (2008). multicriteria decision making approach to evaluate optimum maintenance strategy in textile industry shyjith, ilangkumaran & kumanan, (2008). manufacturing performance criteria: an ahp application in a textile company kaya, caliskan, fgozlu, (2007). determination of quality value of cotton fibre using hybrid ahp-topsis method of multi-criteria decision-making majumdar, sarkar & majumdar (2005). comparison of fuzzy ahp and fuzzy topsis methods for facility location selection ertuğrul, & karakaşoğlu, (2008). ahp approach in the credit evaluation of the manufacturing firms in turkey yurdakul, & tansel ic (2004). supply chain management in the textile industry: a supplier selection model with the analytical hierarchy process koprulu & albayrakoglu (2007). study on the selection and evaluation system of the textile machine manufacturer's supply enterprises based on ahp theory zhi yu-ping (2008). the application of ahp approach to fabric selections liu zhenzhen, chen chao (2008). optimization of production combination in a textile factory using ahptechnique madhoushi & amirfazli (2002). selection of maintenance policy for textile industry using hybrid multicriteria decision making approach ilangkumaran, & kumanan, (2009). 2. theoretical foundation 2.1 textile industry in brazil according to the brazilian association of textile and clothing industry, brazil has one of the world’s largest industrial parks and is one of the top ten global markets in the segment (abit, 2014). it is the third main jersey manufacturer, one of the top five clothing producers (table 2), the sixth largest textile and clothing industry of the world, ijahp article: serra costa, borges, machado/analytic hierarchy process applied to industrial location: a brazilian perspective on jeans manufacture international journal of the analytic hierarchy process 80 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.210 and is listed among the top eight most relevant markets for threads, fibers and fabrics (abit, 2014). table 2 textile industry ranking country production (mil ton) world % china 38,561 50.7 india 5,793 7.6 usa 4,021 5.3 pakistan 2,820 3.7 brazil 2,249 3.0 indonesia 1,899 2.5 taiwan 1,815 2.4 turkey 1,447 1.9 south korea 1,401 1.8 thailand 902 1.2 source: abit, 2014 the textile sector in brazil includes more than 32,000 companies, 80% of which are smalland medium-sized manufacturers, all over the country. they employ around 1.7 million brazilian workers, and 75% are manufacturing sector employees, mostly women (abit, 2014). during the first period of the industrialization, manufacturing activity in brazil was concentrated mainly in the southeast region (cury, 1999). now this activity is broader, distributed in various regions of the country, following the world tendency for decentralization. these regions offer, besides fiscal incentives, lower labor costs, and better sources of workforce and feedstock (saboia & kubrusly, 2008). the first region for the segment of clothing and other textile articles is located in the northeast region (states of pernambuco, ceará, bahia and rio grande do norte). in this region, some of the main companies are vicunha têxtil s.a. and coteminas s.a. (leão, 1999). the second region is the south, which has significant relevance due to the geographic proximity of the main consumer’s center and mercosul countries (argentina, brazil, paraguay and uruguay). a highlight in this region is itajaí valley, an important textile sector located in state of santa catarina. some of the main companies, among others, situated in this area are: companhia têxtil karsten, marisol s.a., renaux blue label, malwee (cury, 1999). the third region is the southeast (the states of são paulo, minas gerais and rio de janeiro), where small, medium and large size companies are concentrated: companhia têxtil ferreira guimarães, werner fabrica de tecidos s.a. and companhia industrial cataguases (saboia & kubrusly, 2008). ijahp article: serra costa, borges, machado/analytic hierarchy process applied to industrial location: a brazilian perspective on jeans manufacture international journal of the analytic hierarchy process 81 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.210 2.2 cotton and denim (or jeans) cotton is a natural vegetal fiber, presented as a thin, 24 to 38mm long, thread. it can be produced in almost every continent considering the little requirements regarding soil and climate. the most relevant advantages of cotton, when compared to other artificial and synthetic fibers, include the comfort of the manufactured items, especially favorable to hot weather countries, and the ecological aspects since it is biodegradable (oliveira, 1977). currently, this fiber is responsible for 70% of the textile world market, with 20 million tons of production per year. the highly distinctive countries in cotton production are: the us, brazil, argentina, spain, china, russia, india, pakistan, egypt and australia. cotton is native in brazil and has continuously played a major role in the country’s textile industry development, as a main input to the threads and cloths production (kon & coan, 2005). presently, brazil is the world’s fifth cotton producer, and the largest production comes from the midwest and northeast regions (aubert & goldminc, 2001). in the fashion industry cotton is the most important feedstock for the manufacture of denim, which is composed of crude or indigo stained warps interlaced with white woof threads, to create the fabric. the denim market reaches all kinds of social-economic layers as well as all age groups (panucci-filho & garcias, 2010). fighting with china and turkey, year-by-year, for the first position in the world’s denim production rank, brazil produces 25 million meters a month (abit, 2014). 3. ahp – analytic hierarchy process the analytic hierarchy process is a multicriteria methodology that aims to select or choose the best alternatives in a process that considers different evaluation criteria (saaty, 1980). an advantage of the ahp method is that it allows for the comparison of both quantitative and qualitative criteria. it is considered one of the most well-known and widely disseminated decision making tools, having the greatest number of applications reported in the literature (vaidya & kumar, 2006). the method is based on constructing hierarchies and making pairwise judgments. the key elements of a decision problem hierarchy are: the main focus – general purpose, a viable alternatives group – definition of which options will be analyzed in the light of the determined criteria, criteria and sub criteria –attributes or items that will be evaluated in pairs, hierarchy building – after defining the problem, the hierarchy that will represent the problem structure is built (garuti & sandoval, 2006). ijahp article: serra costa, borges, machado/analytic hierarchy process applied to industrial location: a brazilian perspective on jeans manufacture international journal of the analytic hierarchy process 82 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.210 when making criteria and alternative judgments, the expert compares pairwise the elements in the level of the hierarchy to each of the elements in the superior level of the hierarchy (saaty, 2008). if the problem has sub-criteria, those should be submitted to the same evaluation/comparison process. finally, the importance of the criteria is compared to the main focus. there are several criteria (and sub-criteria) judgment scales and alternatives that have been proposed and used in the literature. in this article, the scale proposed by saaty (1980) was used. so, in the ahp, the prioritization is done in the following steps: gather the expert’s judgments, normalize the judgments to obtain the local weights (or the priorities), and calculate the global weights by weighting them by the weight of their respective elements in the level above. several alternatives to conjugate information provided by the experts have been proposed and many of them lead to values that are nearly consistent (forman & peniwati, 1998; escobar et al., 2004; saaty & peniwati, 2007; escobar & moreno-jiménez, 2007; freitas et al., 2008; innes, 2008). in this article, the authors decided to use the geometric means to aggregate the individual judgments (forman & peniwati, 1998; escobar & moreno-jiménez, 2007). even when judgments are obtained from experts, some inconsistency may occur (serra costa, 2011). in the case of ahp, according to saaty (1980), one way of measuring the intensity or degree of inconsistency in a matrix of pairwise judgment is to evaluate how the highest eigenvalue of this matrix deviates from the order of the matrix. so, the consistency index (c.i.) can be calculated as shown in equation 1. max . . 1 n c i n     (1) according to saaty (1980), the gravity of the inconsistency occurrence can be evaluated by considering its ratio to the average consistency index obtained from a large number of matrices of the same order generated by entering random judgments. this is the consistency ratio (c.r.), and it is used as a parameter to evaluate the inconsistency obtained from the judgment matrix order (equation 2). c.r. = c.i./r.i. (2) where: c.i. = consistency index defined in equation (1); r.i. = consistency index obtained from a large number of randomly generated reciprocal matrices with non-negative elements. (brunelli & fedrizzi, 2011). ijahp article: serra costa, borges, machado/analytic hierarchy process applied to industrial location: a brazilian perspective on jeans manufacture international journal of the analytic hierarchy process 83 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.210 according to vargas (1982), if the calculated c.r. is lower than 0.1, the judgment matrix is considered consistent. 4. data analysis and results when selecting the criteria that substantiate the decision model, the research is based on the description of economic conditions that shows how each criterion is subdivided, and that in this subdivision there are other groups, which will be only taken into consideration to guide the expert’s judgment. to create the structure, the opinion of the experts was taken into account. the experts are individuals (or a group of individuals) responsible for the performance analysis of the elements of each layer or level of the hierarchy in relation to those to which they are connected in the superior level. the effectiveness of the results is associated with the competency of the experts in providing value judgments, so these experts must acknowledge and even master the topic about which they are making judgments (malhotra et al., 2007). ten experts were selected, including university professors of textile engineering, and industrial and textile manufacturing managers with at least 10 years of industry experience. after some consideration, the model was established by consensus: • financial fi: labor cost (cost per work hours, journey and unions), energy (natural light, acclimatization costs and energy system stability), feedstock (distance from suppliers and transportation), taxes (tax exemption), building (land costs and construction materials cost); • infrastructure is: consumption market (distance and size), local taxes (land usage taxes and licenses), climate (temperature, humidity, natural light), workforce access (transportation) and products access (transportation); • human resources hr: technical schools (existence of schools, distance, access to schools), regional culture (holidays and religion), population age group (sex and job stability), indirect benefits (feeding support, health insurance and production bonus) and local consumption philosophy (demand, income, fashion influence). as mentioned in section 2, the three regions most favorable and already prepared for the textile industry in brazil are: northeast, south and southeast (saboia & kubrusly, 2008; leão, 1999). from these regions, three candidate states were selected: rio de janeiro (rj), santa catarina (sc) and rio grande do norte (rn). figure 1 depicts the decision problem structure. ijahp article: serra costa, borges, machado/analytic hierarchy process applied to industrial location: a brazilian perspective on jeans manufacture international journal of the analytic hierarchy process 84 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.210 location infrastructure financial human resources santa catarina rio de janeiro rio grande do norte manpower energy feedstock taxes building consumption market local taxes climate workers access products access technical schools regional culture age group indirect benefits local consumption phylosophy figure 1. structure of decision problem data was gathered for the model using questionnaires sent by e-mail to the experts selected to participate in the research. when experts could not access the questionnaire, an interview was arranged to collect the information. all the experts confirmed they are deeply involved in or have specific knowledge of textile manufacture, either applied or academic. 4.1 analysis of the sub-criteria in the light of the criteria the information gathered from the experts was processed according to the methodology. the local weight of each sub-criteria, in the light of the superior criteria, was determined, as shown in table 3. ijahp article: serra costa, borges, machado/analytic hierarchy process applied to industrial location: a brazilian perspective on jeans manufacture international journal of the analytic hierarchy process 85 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.210 table 3 local weights of the sub-criteria in the light of the criteria criteria local weight sub-criteria financial 0.210 manpower 0.180 energy 0.296 feedstock 0.246 taxes 0.068 building infrastructure 0.374 consumption 0.190 local taxes 0.097 climate 0.137 workforce access 0.202 products access human resources 0.358 technical schools 0.154 regional culture 0.094 age group 0.227 indirect benefits 0.167 philosophy the consistency ratio of each matrix of sub-criteria confirmed that the judgments were consistent for the three criteria (financial = 0.0274, infrastructure = 0.0077 and human resources = 0.0134). 4.2 analysis of the alternatives in the light of the sub-criteria next, the local weights of the alternatives in light of the sub-criteria were calculated, as depicted in table 4, which also shows the consistency ratios (c.r.) of the sub-criteria. the sum of the priorities for the alternatives for each sub-criterion is 1, computed by adding across the rows. by examining the rows it is easy to determine which alternative is best for each of the sub-criteria. ijahp article: serra costa, borges, machado/analytic hierarchy process applied to industrial location: a brazilian perspective on jeans manufacture international journal of the analytic hierarchy process 86 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.210 table 4 local weights of the alternatives in the light of the criteria and consistency reasons criteria sub-criteria alternatives cr rj sc rn financial manpower 0.186 0.227 0.587 0.0216 energy 0.362 0.342 0.296 0.0415 feedstock 0.413 0.426 0.161 0.0454 taxes 0.17 0.215 0.615 0.0044 building 0.187 0.195 0.618 0.0005 infrastructure consumption 0.618 0.277 0.105 0.0659 local taxes 0.157 0.233 0.61 0.0378 climate 0.316 0.43 0.254 0.0179 workforce access 0.306 0.437 0.257 0.0149 products access 0.415 0.466 0.119 0.0059 human resources technical schools 0.53 0.377 0.093 0.0027 regional culture 0.237 0.567 0.196 0.0539 age group 0.402 0.471 0.127 0.004 indirect benefits 0.36 0.36 0.28 0.0625 philosophy 0.615 0.273 0.112 0.0314 4.3 analysis of the alternatives in the light of the sub-criteria the local weights of the criteria in the light of the main focus were placed in table 5. table 5 local weights in light of the main focus criteria local weight financial 0.455 infrastructure 0.261 human resources 0.284 ijahp article: serra costa, borges, machado/analytic hierarchy process applied to industrial location: a brazilian perspective on jeans manufacture international journal of the analytic hierarchy process 87 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.210 the consistency ratio of the matrix referring to the pairwise comparisons of the main criteria with respect to the main focus is equal to 0.0189. therefore, the judgments are consistent. 4.4 global weights using the hierarchy previously presented in figure 1, it is possible to determine the global weights (table 6). as presented, in order to calculate the global weights (gw), it is necessary to combine the local weights related to the alternatives, criteria and sub-criteria. the elements of the global weights give the performances (priorities) of the alternatives in light of the main focus. table 6 global weights of the alternatives alternatives global weights rio de janeiro 0.364 santa catarina 0.339 rio grande do norte 0.297 5. conclusions questions related to industrial location are generally strongly connected to several characteristics that need to be analyzed. most of the time, these characteristics are in conflict and difficult to quantify. in the situation analyzed here, fifteen sub-criteria and three criteria were used to characterize the necessary attributes to be considered when installing a textile plant for the denim / jeans segment in brazil. the results of the global weights demonstrated a slight tendency for the state of rio de janeiro (0.364) with little advantage over the state of santa catarina (0.339), and both with significant advantage when compared to rio grande do norte (0.297). the reasons for this could be attributed to the local weights determined by the experts for rio de janeiro and santa catarina in the financial criterion (sub-criterion feedstock) – which had the highest local weights among all the criteria of 0.455, and the infrastructure criterion (sub-criterion consumption market) and the human resources criterion (sub-criterion technical schools). overall, as expected, global weight values for the alternatives were not disparate when compared to each other, especially when considering that these states represent the main ijahp article: serra costa, borges, machado/analytic hierarchy process applied to industrial location: a brazilian perspective on jeans manufacture international journal of the analytic hierarchy process 88 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.210 productive poles in this segment, all of them situated in the most relevant economic regions in the sector in brazil. the ahp method proved to be very effective, efficient, and simple to apply according to the experts who participated. their opinion is reinforced by the values calculated during the judgments consistency analysis, which were considerably lower than the acceptable limits. ijahp article: serra costa, borges, machado/analytic hierarchy process applied to industrial location: a brazilian perspective on jeans manufacture international journal of the analytic hierarchy process 89 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.210 references abit associação brasileira da indústria têxtil e de confecção. relatório de atividades 2013. available at: < http://www.abit.org.br/publicacao.aspx >. accessed: march 2014. aubert, c., goldminc, m. (2001). vétement: la fibre écologique. [s.l.]: terre vivante. brunelli, m., fedrizzi, m. (2011). characterizing properties for inconsistency indices in the ahp, proceedings of the international symposium on the analytic hierarchy process, sorrento, italy. cebeci, u. (2008). fuzzy ahp-based decision support system for selecting erp systems in textile industry by using balanced scorecard. expert systems with applications, 36(5), 8900–8909. doi:10.1016/j.eswa.2008.11.046 cury, v.m. (1999). algodão e proteção: indústria textil no brasil, 1890 – 1930. história revista, 4 (1/2), 79-97. escobar, m.t., aguarón, j., moreno-jiménez, j.m. (2004). a note on ahp group consistency for the row geometric mean prioritization procedure. european journal of operational research, 153(2), 318 -322.doi:10.1016/s0377-2217(03)00154-1 escobar, m.t., moreno-jiménez, j.m. (2007). aggregation of individual preference structures in ahp-group decision making. group decision and negotiation, 16(4), 287-301. doi: 10.1007/s10726-006-9050-x ertuğrul,i. karakaşoğlu, n. (2008). comparison of fuzzy ahp and fuzzy topsis methods for facility location selection. the international journal of advanced manufacturing technology, 39(7-8), 783-795. doi:10.1007/s00170-007-1249-8 forman, e., peniwati, k. (1998). aggregating individual judgments and priorities with the analytic hierarchy process. european journal of operational research, 108(1), 165-169. doi:10.1016/s0377-2217(97)00244-0 freitas, a.l.p., trevizano, w. a., costa, h.g. (2008). uma abordagem multicritério para problemas decisórios com múltiplos grupos de avaliadores. investigação operacional, 28, 133-149. garuti, c., sandoval, m. (2006). the ahp: a multicriteria decision making methodology for shiftwork prioritizing. journal of systems science and systems engineering, 15(2), 189-200. doi:10.1007/s11518-006-5007-5 gorini, a.p.f. (2000). panorama do setor têxtil no brasil e no mundo: reestruturação e perspectivas. bndes setorial, rio de janeiro, 12, 17-50. innes, j. (2008). managing impact reduction in fisheries: a multi-criteria assessment of objective priorities. proceeding of the 37th australian conference of economist (ace08), queensland, australia. http://dx.doi.org/10.1016/j.eswa.2008.11.046 http://dx.doi.org/10.1016/s0377-2217(03)00154-1 http://dx.doi.org/10.1016/s0377-2217(97)00244-0 ijahp article: serra costa, borges, machado/analytic hierarchy process applied to industrial location: a brazilian perspective on jeans manufacture international journal of the analytic hierarchy process 90 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.210 kaya, e., caliskan, f.d., gozlu, s. (2007). manufacturing performance criteria: an ahp application in a textile company. management of engineering and technology, 1186 – 1194. doi: 10.1109/picmet.2007.4349442 kon, a., coan, d.c. (2005). transformações da indústria têxtil brasileira: a transição para a modernização. revista de economia mackenzie, são paulo, 3( 3), 11-34. koprulu, a., albayrakoglu, m.m. (2007). supply chain management in the textile industry: a supplier selection model with the analytical hierarchy process. proceeding of the international symposium on the analytic hierarchy process. viña del mar, chile. leão, m. d. (1999). desenvolvimento tecnológico para controle ambiental na indústria têxtil: malhas no estado de minas gerais. relatório final. belo horizonte, 255. majumdar, a., sarkar, b., majumdar, p. k. (2005). determination of quality value of cotton fibre using hybrid ahp-topsis method of multi-criteria decision-making. journal of the textile institute, 96(5), 303-309. doi:10.1533/joti.2005.0013 malhotra, v. a., lee m. d., khurana, (2007). a domain experts influence decision quality: towards a robust method for their identification. journal of petroleum science and engineering, 57, 181-194. doi:10.1016/j.petrol.2005.09.007 melo, m.o.b.c., cavalcanti, g.a., gonçalves, h.s., duarte, s.t.v.g. (2007). inovações tecnológicas na cadeia produtiva têxtil: análise e estudo de caso em indústria no nordeste do brasil. revista produção on line, 7(2), 99-117. available in: <http://www.producaoonline.org.br/index.php/rpo/article/view/75>. accessed: april 2014. nordås, h.k. (2004). the global textile and clothing industry post the agreement on textiles and clothing. world, 7, 1-000. oliveira, m.h. (1977). principais matérias-primas utilizadas na indústria têxtil. bndes setorial, rio de janeiro, brasil. panucci-filho, l., garcias, p.m. visão geral das alianças no crescimento das organizações: uma análise sob a ótica das indústrias de confecções de jeans. (2010). revista admpg gestão estratégica, pr, 3(1), available at: <http://www.admpg.com.br/revista2010/artigos/resumo%20area%207% 20artigo%207.pdf>. accessed: march 2013. saaty, t.l. (1980). the analytic hierarchy process, new york: mcgraw hill. doi: 10.1080/00137918308956077 saaty, t. l. (2008). relative measurement and its generalization in decision making, why pairwise comparisons are central in mathematics for the measurement of intangible factors, the analytic hierarchy/network process, racsam review of the royal spanish academy of sciences, series a, mathematics, 102 (2), 251-318. doi: 10.1007/bf03191825 saaty, t. l., peniwati, k. (2007). group decision making: drawing out and reconciling differences. pittsburgh: rws publications. http://dx.doi.org/10.1109/picmet.2007.4349442 http://dx.doi.org/10.1016/j.petrol.2005.09.007 ijahp article: serra costa, borges, machado/analytic hierarchy process applied to industrial location: a brazilian perspective on jeans manufacture international journal of the analytic hierarchy process 91 vol. 8 issue 1 2016 issn 1936-6744 http://dx.doi.org/10.13033/ijahp.v8i1.210 saboia, j.; kubrusly, l. (2008). diferenciais regionais e setoriais na indústria brasileira. revista de economia aplicada, 12(1), 125-149. doi: 10.1590/s1413-80502008000100006 serra costa, j.f. (2011). a genetic algorithm to obtain consistency in analytic hierarchy process. brazilian jounal of operations & production management, 8(1), 55-64. doi: 10.4322/bjopm.2011.003 shyjith, k., ilangkumaran, m., kumanan, s. (2008) multicriteria decision making approach to evaluate optimum maintenance strategy in textile industry, journal of quality in maintenance engineering, 14 (4), 375 – 386. doi: 10.1108/13552510810909975 sousa, l. g. (2004). memórias de economia: a realidade brasileira. [s.l.]: eumed.net. 226. available at: <http://www.eumed.net/cursecon/libreria/2004/lgs-mem/index.htm>. access in: april 2011. stein, s. (1979). origens e evolução da indústria textil no brasil – 1850/1950. editora campus, rio de janeiro, brasil. vaidya, o. s., kumar, s. (2006). analytic hierarchy process: an overview of applications. european journal of operational research, 169 (1), 1-29. doi:10.1016/j.ejor.2004.04.028 vargas, l.g. (1982). reciprocal matrices with random coefficients. mathematical modelling, 3, 69-81. doi:10.1016/0270-0255(82)90013-6 yurdakul, m., tansel ic, y. (2004). ahp approach in the credit evaluation of the manufacturing firms in turkey. international journal of production economics, 88(3), 269–289. doi:10.1016/s0925-5273(03)00189-0 http://dx.doi.org/10.1108/13552510810909975 http://dx.doi.org/10.1016/j.ejor.2004.04.028 http://dx.doi.org/10.1016/0270-0255(82)90013-6 http://dx.doi.org/10.1016/s0925-5273(03)00189-0 ijahp article: dhital, pyakurel, bajracharya, shrestha/ framework for sustainability of renewable energy projects in nepal international journal of the analytic hierarchy process issn 1936-6744 76 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.250 framework for sustainability assessment of renewable energy projects in nepal ram prasad dhital institute of engineering tribhuvan university, nepal e-mail: rpdhital@yahoo.com parakram pyakurel alternative energy promotion centre, nepal email: parakram_pyakurel@yahoo.com tri ratna bajracharya center for energy studies, institute of engineering tribhuvan university, nepal e-mail: triratna@ioe.edu.np rajendra shrestha mechanical engineering department, institute of engineering tribhuvan university, nepal e-mail: rsfluid@hotmail.com abstract long term sustainability of renewable energy projects in nepal has been a challenging issue because a certain amount of investment subsidy from the government is used to build the projects. after installation of the energy system, it is the responsibility of the participating community or the users to operate, maintain and manage the system. the sustainability of renewable energy projects largely depends on how much revenue it can generate from its users. revenue from users’ depends upon multiple factors categorized as technical, financial/economic, social, institutional and environmental. as such, sustainability of the projects needs to be evaluated based on these multiple criteria in a holistic manner. this paper focuses on identifying all of the possible factors relating to sustainability of rural and renewable energy projects in nepal in the context of climate change and a green economy. these factors are identified from the perspective of all concerned people ranging from project implementers to end users, as well as all of the stakeholders. a brief literature review is conducted on the utility of multi criteria methods for a sustainability assessment of renewable energy projects followed by an assessment of the relative standing of ahp. an appropriate ahp-based framework for a sustainability assessment of the project is recommended taking into consideration the factors related to sustainability that were identified from the perspective of a wide range of people. keywords: sustainability, renewable energy, multiple criteria analysis, ahp. ijahp article: dhital, pyakurel, bajracharya, shrestha/ framework for sustainability of renewable energy projects in nepal international journal of the analytic hierarchy process issn 1936-6744 77 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.250 1. introduction renewable energy technologies (rets) have been promoted in nepal since the early 1970s, but these technologies were only widely disseminated after the establishment of a dedicated organization called the alternative energy promotion center (aepc) in 1996. with the support of various development partners and the firm commitment of the government, some 2000 micro hydro projects generating 20 mw electricity, 250,000 solar home systems, 250,000 biogas plants and 500,000 improved cooking stoves have been installed in different parts of the country (aepc, 2011). these projects are a means of rural electrification and enhance energy access of rural communities. however, many remote villages in nepal still lack access to electricity. an overwhelming majority of households (96 percent) in urban areas have access to electricity in their dwelling, while the corresponding figure for rural households is only 63 percent (cbs, 2011). the term, sustainable development, was popularized in our common future, a report published by the world commission on environment and development (wced) in 1987. also known as the brundtland report, our common future included the “classic” definition of sustainable development: “development which meets the needs of the present without compromising the ability of future generations to meet their own needs". acceptance of the report by the united nations (un) general assembly gave the term political salience, and in 1992 leaders set out the principles of sustainable development at the un conference on environment and development (unced) in rio de janeiro, brazil, also referred to as the rio summit and the earth summit (drexhage et al., 2010).sustainability is a major concern for the design of rural electrification projects because without such consideration projects can fail and waste resources. securing a sustainable energy provision is one of the central political challenges of the present. ever since the united nation conferences on climate change held in rio de janeiro (1992), kyoto (1997) and bali (2007) and reports published by the intergovernmental panel on climate change sustainable energy provision has become an important topic for political decision makers throughout the world (carrera et al., 2009). in nepal, rural and renewable energy projects are built with a certain amount of investment subsidy from the government. after installation of the energy system, it is the responsibility of the participating community or the users to operate, maintain and manage the system. the sustainability of renewable energy projects largely depends on how much revenue it can generate from its users. revenue from users’ depends upon multiple factors categorized as technical, financial/economic, social, institutional and environmental. as such, sustainability of the projects needs to be evaluated based on these multiple criteria in a holistic manner. the sustainability evaluation of rural and renewable energy projects is of great importance because annually around 2 billion nepalese rupees is spent in the renewable energy sector according to aepc’s annual budget. it can be argued that about 30 billion nepalese rupees have been mobilized from the government of nepal, many external development partners, local governments and the communities in this sector. therefore, many people are concerned about whether or not this huge investment has had an impact on the sustainability of the energy systems installed. ijahp article: dhital, pyakurel, bajracharya, shrestha/ framework for sustainability of renewable energy projects in nepal international journal of the analytic hierarchy process issn 1936-6744 78 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.250 while the monitoring and evaluation processes of energy systems adopted by aepc assesses technical components of energy systems, there is no assessment tool to date which evaluates the overall sustainability of energy systems. in other words, aepc has not developed an integrated sustainability assessment tool which can assess multiple components of sustainability including economic, environment, social and institutional components. therefore, a framework to assess the sustainability while considering multiple components is necessary and this paper develops such a framework by applying the analytical hierarchy process. the following key questions need to be addressed to assess the sustainability of renewable energy projects installed in nepal: 1. how can sustainability of rural and renewable energy projects be measured objectively? 2. are existing renewable energy projects sustainable? 3. what factors promote or hinder the sustainable operation of renewable energy projects? this paper identifies various possible factors relating to sustainability of renewable energy projects in the context of climate change and a green economy. these factors are identified from the perspective of all concerned people ranging from project implementers to end users, as well as all of the stakeholders. the analytical hierarchy process (ahp) is then used to build a framework of sustainability. numerical values for the factors affecting sustainability are assigned so that sustainability can be measured objectively and the projects can be ranked and evaluated based on their degree of sustainability. 2. literature review there have been a number of attempts to define criteria for the assessment of sustainability. in this respect, the working group of united nations environment programme (unep) on sustainable development has developed qualitative criteria for the assessment of the product design (afgan et al., 2008). in the context of rural and renewable energy, sustainability indicators have been suggested for the qualitatitive and quantitative assessments of sustainability. efforts have been made by hak et. al. (2012) to critically review sustainability indicators and contribute to the development of a suitable methodology for sustainability assessment. they (hak et. al., 2012) began with a broad review of the vast body of work, both practical and academic research. both scientists and practitioners have sought the development of methods for assessing the quality of the indicators. according to hak et. al., (2012), both scientists and practitioners have usually defined some criteria for the assessment of the quality of indicators, however, neither have provided major support by developing reliable, practical and operative methods for indicator assessment. therefore, it can be said that there is no general consensus among scientists and practitioners on a specific set of sustainability indicators and the methods of assessments of these indicators. ijahp article: dhital, pyakurel, bajracharya, shrestha/ framework for sustainability of renewable energy projects in nepal international journal of the analytic hierarchy process issn 1936-6744 79 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.250 if sustainability is to be measured objectively, the indicated sustainability must be quantified. multi criteria decision analysis (mcda) is one tool for quantifying sustainability. mcda is a form of integrated sustainability evaluation. it is an operational evaluation and decision support approach that is suitable for addressing complex problems featuring high uncertainty, conflicting objectives, different forms of data and information, and multi interests and perspectives for complex and evolving biophysical and socio-economic systems (wang et al., 2009). since rural and renewable energy systems exhibit features such as conflicting objectives, multi interests and perspectives and different forms of data and information, mcda is an appropriate tool for use with these systems. it is also necessary to link techno-economic analysis with a multi-criteria decisions support tool because of the qualitative and quantitative aspects involved in such systems. figure 1 shows the linkage of techno-economic analysis with multi-criteria decisions support. figure 1. linking techno-economic analysis with multi-criteria decisions support tool (oikonomou, 2011). multi-criteria decision making (mcdm) may be broadly classified as multi objective decision making (modm) and multi attribute decision making (madm). modm is a mathematical technique of optimization and requires all the criteria to be formulated in a mathematical framework. the modm can evaluate infinite and continuous types of problems. whereas problems having finite sets of possible choices and alternatives are described in terms of their attributes, and here the multi attribute decision making (madm) methodology is used. madm problem solving does not require the classical mathematical programming tool (bhattarai, 1997). since finite sets of possible choices and alternatives are components of sustainability evaluation and decision making, madm is the appropriate tool for this study. there are several techniques for madm including the analytical hierarchy process (ahp), the simple multi attribute rating technique (smart), and the preference ranking organization method for enrichment evaluations (promethee) to name few. the fact that ahp can use value judgments, which are obtained from experts' choices and multiple actors' choices, as inputs for analysis when there is a lack of numerical data makes the ahp the most viable alternative for sustainability related decision making. in a less developed country like nepal, sufficient and reliable databases do not exist (bhattarai, 1997), and decision makers, especially political and public interest groups, are ijahp article: dhital, pyakurel, bajracharya, shrestha/ framework for sustainability of renewable energy projects in nepal international journal of the analytic hierarchy process issn 1936-6744 80 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.250 relatively less educated. in this context, value judgments are more applicable than numerical data for decision making. therefore, this is another reason why the ahp is applicable for this study. the ahp has been adopted to prepare a framework for sustainability assessment. ahpbased multiple criteria analysis deals with the relative priority of importance of each factor in comparison to a certain criteria. a hierarchical structure of these factors is formed by grouping them into different levels. the application of the hierarchical structure allows the factors to be broken down into details. ahp-based multiple criteria analysis starts with building a tree-like structure, with criteria at higher level and factors and sub-factors at lower levels. the objective of the evaluation lies at the top, and the options or alternatives to be evaluated are placed at the lowest level of the hierarchy. the ahp simplifies the process of identification and assessment of criteria, factors and subfactors related to a problem (panthi & bhattarai, 2008). the hierarchical structure of the goal, dimensions and factors developed for this study is guided by the research of bhattarai & adhikari (2011) with rural drinking water facilities. the literature review shows that a process similar to the ahp has been used by the world energy council (wec) to produce an "assessment index" (ai) for energy policies and energy securities assessments of different countries. the steps adopted by the wec to calculate ai (which are similar to ahp) are (martchamadol & kumar, 2013):  normalization, data are normalized via homogeneous transformation in the range 0 (low) to 10 (high) within a cluster of countries;  weighting, weighting factor are equal for each indicators of each dimension, and ai is then calculated by using averages and equal weighting of all dimensions;  presentation of results, ai gives the ranking of a country within the cluster of countries. the following was predicted about ahp back in 2006 (vaidya & kumar, 2006), and time has shown that these predictions have come true.  ahp is going to be used widely for decision making.  ahp use is rising in developing countries. that augurs well with the economic development of this block of countries, such as india, china, etc.  lots of research is going on in countries like the united states where they have had a head start using the ahp. focus there seems to be on combining other techniques with ahp. this takes advantage of the versatility of the ahp along with the focused use of the supporting techniques.  software applications will be used to address the issue of complexities arising out of the integrated applications of ahp and other techniques to represent the real life situations. the sustainability framework for rural micro projects with a focus on water projects has already been developed, but there are no frameworks that focus exclusively on rural and renewable energy projects of nepal. ijahp article: dhital, pyakurel, bajracharya, shrestha/ framework for sustainability of renewable energy projects in nepal international journal of the analytic hierarchy process issn 1936-6744 81 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.250 3. hypotheses/objectives the hypothesis of this study is that a general framework can be developed to measure the sustainability of rural and renewable energy systems of nepal with a focus on micro hydro power technology, solar pumping and solar home systems. it is also assumed that the ahp framework is most suited for ranking projects from a sustainability and framework development perspective in the nepalese context. the overall objective of this study is to develop a framework to measure the sustainability of rural and renewable energy projects and eventually assess the sustainability of renewable energy projects that have been installed for a year or more. 4. research design/methodology site surveys at different villages in rural areas of nepal where renewable energy projects have been implemented or are planned for electrification were carried out. the local management committees that are responsible for operation and maintenance of such projects were interviewed. micro hydro power plants, solar photovoltaics and biogas systems were considered because they are the main technologies used in rural areas of nepal. the main aim of these surveys was to identify sustainability criteria in a local/national context. furthermore, consultations with energy experts, professionals working in the field of rural electrification and energy companies that implement rural electrification projects in nepal were carried out. based on surveys, consultations and literature review sustainability criteria were then identified. finally, hierarchical structures for sustainability assessment were developed. the hierarchical structure, as shown in figure 6 of this paper, was constructed in order to identify the most sustainable project in the nepalese context using ahp. the technologies considered for evaluation were micro-hydropower, solar home systems, solar pv pumping, and bio-energy (biomass/biogas). a similar approach using the ahp was adopted by amigum et al. (2011) in south africa to select the most appropriate renewable energy technology. their study revealed that wind energy was most suitable in the context of south africa. in order to select the most sustainable project using a given technology, a method of objective assessment involving quantification of data is necessary. for instance, to identify the most sustainable micro-hydro project among many micro-hydro projects, a method to select the most sustainable project has to be developed. once a method has been developed, the most sustainable project within each of the considered technologies can be identified. ultimately, the projects with different technologies can be evaluated using ahp by adopting the hierarchical structure developed in this paper. 5. data/model analysis the triangular approach, which takes into account the three dimensions of sustainable development -economic, social and environmentalcontinues to be highly influential. this approach forms the basis of the structure of the indicators of sustainable ijahp article: dhital, pyakurel, bajracharya, shrestha/ framework for sustainability of renewable energy projects in nepal international journal of the analytic hierarchy process issn 1936-6744 82 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.250 development collected by key organizations all over the world, including the un, the oecd and the european commission (nieves et al., 2010). figure 2 shows the dimensions of sustainability and their interrelationships based on the triangular approach. figure 2. dimensions of sustainability and their interrelationships (nieves et al., 2010). other dimensions have also been suggested in different studies. the united nations commission on sustainable development (csd) devised a framework for monitoring the various sustainability indicators for assessing the performance of governments towards sustainable development goals. the structure of the framework comprises four dimensions including social, environment, economic and institutional, and is broken down into 38 sub-indicators and 15 main indicators. the institution of chemical engineers (icheme) has also developed sustainability metrics covering three dimensions including environmental, economic and social which are further sub-divided into a set of indicators. these metrics were initiated to assess the sustainability performance of the process industry (singh et al., 2012). the wuppertal institute also developed a framework of sustainability by addressing the four dimensions of sustainable development, as defined by the united nations csd. these four aspects are linked through a set of various indicators (singh et al., 2012). overall, based on the literature review presented above, it can be said that different sets of indicators have been developed by various research organizations to assess sustainability. there is no universally adopted, rigid set of dimensions and indicators, and the indicators can change on a case by case basis. for rural and renewable energy projects, efforts have been made by various researchers to identify the dimensions of sustainability. a total of five dimensions/indicators suggested for rural electrification (ilskog et. al., 2008) are listed below. ijahp article: dhital, pyakurel, bajracharya, shrestha/ framework for sustainability of renewable energy projects in nepal international journal of the analytic hierarchy process issn 1936-6744 83 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.250  technical sustainability focused on maintaining the energy services during the economic lifetime of the initial investment;  economic sustainability focused on survival of the service beyond the economic lifetime of the initial investment;  social sustainability focused on equitable distribution of the benefits offered by electrification;  environmental sustainability focused on the conservation of natural resources, avoiding degradation of the environment and preventing indoor air pollution;  institutional sustainability focused on survival of the organization and its ability to maintain adequate performance with respect to the other dimensions of sustainability. a study to determine qualitative factors that contribute to sustainability of rural and renewable energy projects has been carried out which suggests the following features as part of successful and sustainable renewable energy projects (sovacool, 2013):  selecting appropriate technology through feasibility studies and surveys that, by asking local users what they want, are able to identify community needs and desirable energy services;  coupling renewable energy with income generating activities and partnering with livelihood groups such as farmers and crop processors, small businesses, restaurants, and community cooperatives;  providing access to financing and micro credit to overcome the first cost hurdle with purchasing systems; having political leadership and a requisite alignment of national and local policies;  building capacity and investing in local institutions rather than merely providing technology; being flexible in terms of deadlines and changing circumstances, including the avoidance of promoting technology selected only by donors;  conducting outreach and marketing campaigns and research to ensure that economic, social, and policy issues are addressed alongside traditional engineering and environmental aspects;  encouraging active participation (and feedback) from communities, essentially creating as much interaction among designers, producers, and users as possible;  avoiding giving away systems for free and instead requiring community contributions and cost-sharing;  enforcing technical standards and certifications so units, components, installation practices, and maintenance procedures are all sufficient to ensure reliable system operation. based on work carried out by previous researchers the five dimensions/criteria considered to prepare the sustainability framework in this paper are as follows: technical, economic, environmental, social and institutional. the qualitative factors of renewable energy projects listed above can be incorporated under the five dimensions considered for this study. for instance, capacity building can be included under the institutional sustainability dimension. various sub-criteria have been identified within each of the 5 dimensions to measure the sustainability. the sub-criteria identification was ijahp article: dhital, pyakurel, bajracharya, shrestha/ framework for sustainability of renewable energy projects in nepal international journal of the analytic hierarchy process issn 1936-6744 84 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.250 based on the literature review and findings from field visits in rural areas of nepal. in addition, consultations with local village communities where rural and renewable energy projects operate were carried out to ensure that the sub-criteria identified for this research are strongly relevant in the nepalese context. the typical evaluation criteria of energy supply systems are suggested in figure 3 (wang et al., 2009). aspects criteria technical efficiency energy efficiency primary energy ratio safety reliability maturity others economic investment cost operation and maintenance cost fuel cost electric cost net present value (npv) payback period service life equivalent annual cost (eac) others environmental nox emission co2 emission co emission so2 emission particles emission non-methane volatile organic compounds (nmvocs) land use noise others social social acceptability job creation social benefits others figure 3. evaluation criteria of energy supply systems (adapted from wang et al., 2009) ijahp article: dhital, pyakurel, bajracharya, shrestha/ framework for sustainability of renewable energy projects in nepal international journal of the analytic hierarchy process issn 1936-6744 85 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.250 the sub-criteria in figure 3 have been modified to fit the context of nepal in order to make the sustainability framework highly relevant for rural villages in this country. for example, nox, so2 emission and particles emission is not relevant in context of the technologies used in nepal and has therefore been eliminated. decision making to ensure the sustainability of rural and renewable energy systems can be a difficult task because of the complex interaction of the technical, economic, environmental, social and institutional dimensions. therefore, multi-criteria decision analysis (mcda) can be applied to this kind of decision making, and ahp is a mcda tool used in energy systems (kahraman et al., 2010). the ahp is a multi-criteria decision making (mcdm) method which helps a decision-maker facing a complex problem with multiple conflicting and subjective criteria (e.g. location or investment selection, projects ranking, etc) (ishizaka et al., 2011). the fact that ahp helps in decision making when there are subjective criteria involved makes it very applicable to rural and renewable energy systems' sustainability assessment because such energy systems entail subjective criteria like social and institutional components. besides, ahp has broken through the academic community and is widely used by practitioners. this widespread use is certainly due to its ease of applicability and the structure of the ahp which is intuitive to how managers solve problems. the hierarchical modeling of the problem, the possibility to adopt verbal judgments and the verification of the consistency are its major assets. expert choice, the user-friendly supporting software, has certainly largely contributed to the success of the method (ishizaka et al., 2011). the ahp has been used in the energy sector for energy policy formulation, energy planning, power plant selection, power plant location selection, energy resource allocation, integrated resource planning, energy exploitation, controlling greenhouse gas (ghg) emissions, and developing energy management systems (amer et al., 2011). for pakistan, the following hierarchical structure has been proposed by amer et al. (2011) and is shown in figure 4. ijahp article: dhital, pyakurel, bajracharya, shrestha/ framework for sustainability of renewable energy projects in nepal international journal of the analytic hierarchy process issn 1936-6744 86 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.250 figure 4. proposed decision model for selection, evaluation and ranking of renewable energy technologies for electricity generation (amer et al., 2011). criteria/dimensions and sub-criteria were identified and a hierarchical structure was prepared in order to develop a framework for sustainability assessment for this study. five dimensions, technical, economic, environmental, social and institutional, have been considered and sub-criteria/factors were then identified for each of the dimensions. the guidelines of aepc, consultations with energy experts and project management committees and a literature review were used to identify the factors. for the context of nepal, 2 factors have been identified for the technical as well as the economic dimensions, 5 factors have been identified for environmental, 4 for social and 3 for the institutional dimension. figure 5 shows the ahp hierarchical structure prepared for this study. ijahp article: dhital, pyakurel, bajracharya, shrestha/ framework for sustainability of renewable energy projects in nepal international journal of the analytic hierarchy process issn 1936-6744 87 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.250 figure 5. hierarchical structure of goals, dimensions and factors with their weights for rural and renewable energy sustainability it is necessary to assign a weighting factor to each of the dimensions for ahp analysis. saaty’s 1 to 9 scale (ishizaka et al., 2011) as shown below was used for pairwise comparison. ijahp article: dhital, pyakurel, bajracharya, shrestha/ framework for sustainability of renewable energy projects in nepal international journal of the analytic hierarchy process issn 1936-6744 88 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.250 intensity of importance definition 1 equal importance 2 weak 3 moderate importance 4 moderate plus 5 strong importance 6 strong plus 7 very strong or demonstrated importance 8 very, very strong 9 extreme importance the pairwise comparison among the dimensions can be done through expert's judgments. a 5 by 5 matrix can be prepared based on saaty's 1 to 9 scale, and this weight vector can then be determined by the normalization of the 5 by 5 matrix. similarly, pairwise comparisons between the factors of each dimension can be carried out based on experts' judgments. finally, comparisons among various projects can be performed. four projects developed for use with different technologies were considered for analysis. in nepal, solar pv based home electrification systems, solar pv based water pumping schemes, micro-hydro power plants and bioenergy projects (improved cooking stove or biogas plants) are the most commonly implemented renewable energy systems. these four systems were considered for ahp analysis. the most successful projects under each of the above mentioned systems were then selected and these selected projects were evaluated using the ahp. in this way, the ahp evaluation helped ascertain the most sustainable project among projects constructed using different technologies. the hierarchical structure of the projects is shown in figure 6. ijahp article: dhital, pyakurel, bajracharya, shrestha/ framework for sustainability of renewable energy projects in nepal international journal of the analytic hierarchy process issn 1936-6744 89 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.250 figure 6. hierarchical structure with selected projects an alternative approach to saaty's 1 to 9 scale is to develop a weighting factor based on practical experience and consultations with project users, managers and operators. the following pre-determined weighting factor in terms of percentage has been suggested. technical dimension: 25% economic dimension: 18% environmental dimension: 17% social dimension: 20% institutional dimension: 20% samplings of rural and renewable energy projects throughout nepal are required to fine tune the weighting values suggested above. these weighting factors are based on informal monitoring and evaluations of projects by aepc that suggest that the technical dimension is the most critical; hence, requiring the highest weighted value. the social and institutional dimensions follow the technical dimension in terms of criticality. case studies of successful and failed projects are required to substantiate the weighting values. an ahp analysis can be performed on projects starting with the weighting factors suggested above, and a sensitivity analysis can then be performed to identify the critical dimensions. a total score of 10 can be assigned to each of the dimensions, and then multiplied by the corresponding weighting value to quantify the sustainability assessment. since the factors ijahp article: dhital, pyakurel, bajracharya, shrestha/ framework for sustainability of renewable energy projects in nepal international journal of the analytic hierarchy process issn 1936-6744 90 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.250 or sub-criteria have been identified for all the dimensions, scores out of 10 for any given dimension can be obtained by adding the scores assigned to each factors within the given dimension. 6. limitations the factors/indicators affecting the sustainability have been developed mostly by considering micro hydro power technology, solar pv technology and bioenergyspecifically improved cooking stove and biogastechnology. therefore, the indicators may not be fully applicable for other energy technologies. weighting factors have been given to all the 5 criteria based on experts' judgments, and need corroboration from other stakeholders. 7. conclusions the capability of a rural electrification project to sustain itself is one main criterion to be considered before project development in order to prevent any premature failure of the project. the ahp has been used to develop a sustainability framework for rural and renewable energy systems of nepal in this study. ahp-based hierarchical structure of goals, dimensions and factors with their weights for rural and renewable energy sustainability have been formed, and this structure provides a framework to help in decision making related to project development and also rank various projects based on their sustainability. weighting factors to each dimension of sustainability has also been proposed based on aepc's project evaluation and monitoring. expert choice should be used in order to determine the criticality of each dimension. it is expected that the framework for sustainability developed in this paper will provide input to the integrated rural and renewable energy planning and policy making. ijahp article: dhital, pyakurel, bajracharya, shrestha/ framework for sustainability of renewable energy projects in nepal international journal of the analytic hierarchy process issn 1936-6744 91 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.250 references aepc (2011). annual report for 2010. alternative energy promotion centre (aepc). cbs (2010-2011). nepal living standards survey. central bureau of statistics, volume i, 2010-2011, government of nepal. drexhage, j., & murphy, d. (2010). sustainable development: from brundtland to rio 2012. united nations headquarters, new york. carrera, d.g., & mack, a. (2009). sustainability assessment of energy technologies via social indicators: results of a survey among european energy experts. energy policy, 38 (2) 1030–1039. afgan, n. h. et al. (2008). sustainability assessment of a hybrid energy system. energy policy, 36(8), 2903– 2910. hak, t. et al. (2012). a method to assess the relevance of sustainability indicators: application to the indicator set of the czech republic’s sustainable development strategy. elsevier ecological indicators, 17, 46–57. oikonomou, v. (2011).linking least-cost energy system costs models with mca: an assessment of the eu renewable energy targets and supporting policies. elsevier energy policy, 39, 2786–2799. bhattarai,s. (1997). appropriate scale of hydropower development for nepal: a multicriteria decision making approach. masters of engineering thesis, school of civil engineering, asian institute of technology. bhattarai, s., & adhikari, b.k. (2011). analytic hierarchy process for rural micro project sustainability monitoring in nepal. proceeding of international symposium on analytic hierarchy process 2011 – isahp 2011, pittsburgh, creative decision foundation. panthi, k. & bhattarai, s. (2008). a framework to assess sustainability of communitybased water projects using multi-criteria analysis. first international conference on construction in developing countries (iccidc-i), karachi, pakistan. martchamadol, j. & kumar, s. (2013). an aggregated energy security performance indicator. elsevier applied energy, 103, 653–670. vaidya, o.s. & kumar, s. (2006). analytic hierarchy process: an overview of applications. european journal of operational research, 169(1), 1–29. amigum, b. et al. (2011). multi criteria analysis for sustainability assessments of electricity generation systems in a rural community in south africa. micro ijahp article: dhital, pyakurel, bajracharya, shrestha/ framework for sustainability of renewable energy projects in nepal international journal of the analytic hierarchy process issn 1936-6744 92 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.250 perspectives for decentralized energy supply (mpdes), proceedings of the international conference, technische universität berlin. wang, j.j et al. (2009). review on multi-criteria decision analysis aid in sustainable energy. renewable and sustainable energy reviews, 13(9), 2263–2278. singh, r.k. et al. (2012). an overview of sustainability assessment methodologies. elsevier ecological indicators, 15(1), 281–299. nieves, l.j & rio, p. del. (2010). contribution of renewable energy sources to the sustainable development of islands: an overview of the literature and a research agenda. open access, sustainiability, 2, 783-811. ilskog, e. and kjellstrom, b.(2008). and then they lived sustainably ever after? assessment of rural electrification cases by means of indicators. elsevier energy policy 36(7), 2674-2684. sovacool, b. k. (2013). a qualitative factor analysis of renewable energy and sustainable energy for all (se4all) in the asia-pacific. elsevier energy policy, 59, 393–403. kahraman, c., & kaya, i. (2010). a fuzzy multicriteria methodology for selection among energy alternatives. expert systems with applications, in press, corrected proof, doi: 10.1016/j.eswa.2010.1002.1095. ishizaka a., & labib, a.(2011). review of the main developments in the analytic hierarchy process. expert systems with applications, 38(11), 14336-14345. amer,m., & daim, t.u.(2011). selection of renewable energy technologies for a developing county: a case of pakistan. elsevier energy for sustainable development, 15, 420–435. ijahp article: jihadi, vilantika, sholichah, arifin/best sharia bank in indonesia: an analytical hierarchy process (ahp) approach international journal of the analytic hierarchy process 92 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.824 best sharia bank in indonesia: an analytical hierarchy process (ahp) approach m. jihadi faculty of economics and business, university of muhammadiyah malang, indonesia jihadi@umm.ac.id elok vilantika faculty of economics and business, university of muhammadiyah gresik, indonesia elokvilant@gmail.com fatmawati sholichah stie pemnas indonesia malang, indonesia fsholichah@gmail.com zainal arifin stie indonesia banjarmasin, indonesia zainal@stiei-kayutangi-bjm.ac.id abstract this study aims to determine the performance of islamic commercial banks in indonesia using the following assessment criteria: capital, asset quality, efficiency, liquidity, sharia conformity & profitability (scnp), and the maqasid index. the sample includes the five best sharia banks in indonesia. all the data are obtained based on the annual reports of each bank taken from the bank's websites in 2019. data were analyzed using the analytical hierarchy process (ahp) method. this tool helps make decisions from several alternative choices of sharia bank institutions. the results show that bank muamalat indonesia, bank syariah mandiri, and bri syariah are the three best performing sharia banks in 2019. keywords: sharia bank performance; scnp; maqasid index; analytical hierarchy process 1. introduction banks play an important role in preserving the stability of the economy of a country and are also a societal need. currently, it is very unusual for individuals who do not have a partnership with a bank to act as either a savings institution or a lending institution. the mailto:jihadi@umm.ac.id mailto:elokvilant@gmail.com mailto:fsholichah@gmail.com mailto:zainal@stiei-kayutangi-bjm.ac.id ijahp article: jihadi, vilantika, sholichah, arifin/best sharia bank in indonesia: an analytical hierarchy process (ahp) approach international journal of the analytic hierarchy process 93 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.824 modern banking world uses interest as its key instrument; however, the use of interest is forbidden in islam because it requires usury and does not comply with islamic law. the sharia bank has arisen from this requirement, and provides interest-free services to muslims who are not pleased with the operation of the interest scheme. as a nation with a majority muslim population, indonesia holds a very high potential market share. in 1991, indonesia's first sharia bank was bank muamalat indonesia (bmi). initially, the presence of sharia banks did not receive attention from national banks and sharia banks had no official legal basis, so the presence of sharia banking was limited to being an alternative bank for the citizens of indonesia. however, sharia banks have shown positive growth over the course of time. in 2020, the financial services authority known as (ojk) in indonesia reported that the growth of sharia banking was slowing down, but still higher than traditional banks, namely 10.14% year-to-date (ytd) growth in loans and 9.35% ytd growth in assets. as the organization controlling and overseeing banking in indonesia, bank indonesia is charged with assessing whether a bank is in good health or not based on the bank’s financial statements. table 1 provides a description of sharia banks' output growth in indonesia based on financial ratios. this table shows that the overall average financial ratio, which is a bank performance measurement metric, exhibits a year-on-year rise. currently, there are 14 sharia commercial banks in indonesia. bank syariah mandiri, bank muamalat indonesia, bri syariah, bni syariah, and bank mega syariah (fadilah, 2019) are the five highest performing of those banks in indonesia. table 1 sharia commercial bank financial ratios financial ratio 2016 2017 2018 2019 car 16.63 17.91 20.39 20.59 roa 0.63 0.63 1.28 1.73 npf 4.42 4.76 3.26 3.23 fdr 85.99 79.61 78.53 77.91 oer 96.22 94.91 89.18 84.45 nom 0.68 0.67 1.42 1.92 yield 58.84 51.29 47.69 46.46 invest 34.64 35.22 36.56 39.89 source: financial services authority (2019) sharia banks' efficiency measures in indonesia do not have their own instruments and still use the same measures as traditional banks, which are financial ratio calculation instruments. this is considered an inaccurate reflection of the success of sharia banks because there are differences between traditional banks and sharia banks in the main tasks and methods (bedoui, 2012). in traditional banks, the camels method (capital, quality of assets, management, earning, liquidity, and sensitivity) can be used to measure performance. kouser and saba (2012), ahsan (2016), islami and ashrafuzzaman (2015), suresh and bardastani (2016), bashatweh and ahmed (2020) are previous studies that used camels as a benchmark for bank results. ijahp article: jihadi, vilantika, sholichah, arifin/best sharia bank in indonesia: an analytical hierarchy process (ahp) approach international journal of the analytic hierarchy process 94 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.824 sharia banks have to pay attention to their results in the financial and non-financial aspects of the different characteristics between traditional banks and sharia banks (antonio, sanrego, & taufiq, 2012). to assess efficiency of the principles and practices of sharia banks, some studies have used non-financial metrics. other studies have used sharia conformity and profitability (scnp) from ratnaputri (2013), apriani, kusnendi, and firmansyah (2018), djuwita (2019) and kuppusamy, saleh, and samudhram (2010). the scnp equation calculates the sharia variable with the average ratio of sharia compliance and traditional variables with the average ratio of profitability. the maqasid index is used in other studies such as rusydiana and parisi (2016), hartono and sobari (2017), mega, isni, and taufiq (2019), lesmana and haron (2019), tarique, islam, mohamed, razak, and hamdan (2020) as an indicator of the success of sharia banks. tahdzib alfard (educating people), iqomah al-adl (upholding justice), and jablb maslahah (welfare) are the concepts of the sharia maqasid index that are related to the theory proposed by abu zaharah. kholid and bachtiar (2016) which stated that as a muslim all the activities must be according to islamic law, including economic activities. the three tests, namely the maqasid index, scnp and camels are combined by ramdhoni and fauzi (2020). there are several alternative options of banks in indonesia making customers selective in their choice of which bank is the best. the decision-making process is not simple and there are many aspects that need to be considered so as not to make the wrong choice. the analytical hierarchy process (ahp) methodology is able to help solve decisionmaking problems. previous scholars, including shahhoseini, khassehkhan, and shanyani (2012), abduh and omar (2012), and tabash (2017) have used the ahp approach in sharia banks. using the camels, scnp, and the sharia maqasid index parameters with the analytical hierarchy process to assist decision-making, this study aims to assess the performance of sharia banking in indonesia. 2. literature review 2.1 sharia banks in indonesia, islamic banks are called sharia banks. they are financial institutions that carry out business activities based on sharia principles, namely the rules of an agreement based on islamic law between banks and other parties for depositing funds or financing business activities or other activities declared in accordance with sharia. the legality of islamic banks in indonesia has been protected by law since the issuance of banking law no.7 of 1992 which was later refined according to banking conditions as stipulated in law no. 21 of 2008 (grant et al., 2017). one of the characteristics of islamic banks is that they do not charge interest to customers, but use profit sharing as well as other rewards in accordance with the agreed contracts. the basic concept of islamic banking is based on the al quran and hadiths (razak, 2015). 2.2 sharia bank performance 1. capital adequacy ratio (car) the capital adequacy ratio (car) is the product of a comparison, taking into account market risk, between the capital ratio and the minimum capital adequacy ratio. high car demonstrates that the bank is capable of funding operation activities ijahp article: jihadi, vilantika, sholichah, arifin/best sharia bank in indonesia: an analytical hierarchy process (ahp) approach international journal of the analytic hierarchy process 95 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.824 and contributes profitability. bateni, vakilifard, and asghari (2014) used car as an assessment of bank results, and below is the car formula: car = capital risk weighted assets 2. non-performing financing (npf) non-performing financing (npf) is a ratio that represents the bank management's willingness to deal with problem financing. the higher the ratio, the poorer the quality of the bank lending is, which results in a higher the degree of non-performing financing. to test the efficacy of funding disbursed by sharia banks, the npf ratio (wulandari, suryana, & utami, 2019) can be calculated as follows: npf = non − performing financing total financing 3. operating efficiency ratio (oer) the operating efficiency ratio (oer) is the ratio of operating expenditures and operating revenue. in operational activities, this ratio is used to calculate the efficiency level and bank output. the lower the percentage of the oer ratio, the more productive the operating costs used by the bank are. high oer means that the bank does not operate effectively because a high value reflects the vast amount of operating costs required to gain operating profits. oer = operating expenses operating income 4. financing-to-deposit ratio (fdr) fdr shows a bank's ability to repay withdrawals made by depositors by relying on loans offered as a source of liquidity. the higher the fdr level, the lower the banking liquidity capacity (loen & ericson, 2008). in this case, the bank must be able to monitor the amount of funds channeled and the third party funds that the bank collects in order for the fdr value to remain secure. after the fdr rises, the npf will decrease. good liquidity ensures that, to meet all commitments, the bank has a sufficient source of funds. the risk of loan problems with higher bank liquidity is reduced (akbar, 2016). the fdr value is calculated with the following formula: fdr = total financing total deposits 2.3 sharia conformity and profitability sharia conformity and profitability (scnp) is an approach to assess a sharia bank's financial output by calculating the suitability of the sharia of a bank, but it does not neglect the profitability factor because profit is fundamentally one of the business institution's objectives (kuppusamy et al., 2010). three ratios, namely the islamic expenditure ratio, islamic benefit ratio, and profit sharing ratio, are used to calculate sharia adherence variables. three ratios, namely return on assets (roa), return on equity (roe), and profit margin (pm), are also measured by the profitability variable. ijahp article: jihadi, vilantika, sholichah, arifin/best sharia bank in indonesia: an analytical hierarchy process (ahp) approach international journal of the analytic hierarchy process 96 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.824 each of the compliance ratios and profitability ratios of sharia will be summed resulting in a four quadrant graph where each quadrant is divided by all of the banks' average ratios (see figure 1). figure 1 model sharia conformity and profitability (scnp) source : kuppusamy et al. (2010) 2.4 maqasid index by adapting the definition of sekaran (2000), mohammed (2008) describes maqasid sharia as a measurement instrument to assess the efficiency of sharia banking. the maqasid sharia index is intended to test sharia banking, which not only highlights the financial dimensions, but also integrates the notion of noble islamic principles. as seen in figure 2 below, there are 3 instruments that can be used to calculate maqasid sharia. figure 2 sharia maqasid source : mohammed (2008) ijahp article: jihadi, vilantika, sholichah, arifin/best sharia bank in indonesia: an analytical hierarchy process (ahp) approach international journal of the analytic hierarchy process 97 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.824 2.5 conceptual framework figure 3 shows a framework for the research. figure 3 research framework 3. methodology the analytical hierarchy process (ahp) is used in this study as a technique to prioritize criteria that influence decision-making regarding the performance of indonesia’s best sharia bank. the sample includes five sharia banks that have done well in 2019. the calculation uses six metrics, namely capital, asset quality, production, liquidity, sharia conformity and profitability (scnp) and sharia maqasid index, to assess the performance of sharia banks. the measurement of the consistency index (ci) and consistency ratio (cr) make the ahp approach different from other methods. the steps for using the ahp method are as follows: a. formulate a hierarchical composition to prioritize and review these requirements, a hierarchical structure should be created after deciding the key priorities, parameters, and alternatives. b. use the relative scale measurements shown in table 2 to construct a pairwise comparison matrix (size n x n) for each lower level with one matrix for each factor at the level directly above it. performance evaluation of sharia bank in indonesia capital (car) asset (npf) efficiency (oer) liquidity (fdr) scnp sharia maqasid index bank syariah mandiri bank muamalat indonesia bri syariah bni syariah bank mega syariah scnp ijahp article: jihadi, vilantika, sholichah, arifin/best sharia bank in indonesia: an analytical hierarchy process (ahp) approach international journal of the analytic hierarchy process 98 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.824 table 2 saaty scale source: saaty (2008) c. calculate the consistency index (ci) with the formula: 𝐶𝐼 = 𝜆𝑚𝑎𝑥 − 𝑛 𝑛 − 1 ci: consistency index 𝞴max: eigen value n: matrix size d. calculate the consistency ratio (cr) with the formula 𝐶𝑅 = 𝐶𝐼 𝑅𝐼 cr: consistency ratio ci: consistency index ri: random consistency index the ri table that is the reference for calculating cr in the ahp method can be seen in table 3. table 3 random consistency index size of matrix 1 2 3 4 5 6 7 8 9 10 ri 0 0 0.58 0.9 1.12 1.24 1.32 1.41 1.45 1.49 source: saaty, 2008 the appropriate degree of inconsistency is below 10% for the ahp process. if the cr value is higher than 10%, the evaluation must be updated because a degree of inconsistency that is too high which will lead to an error. ijahp article: jihadi, vilantika, sholichah, arifin/best sharia bank in indonesia: an analytical hierarchy process (ahp) approach international journal of the analytic hierarchy process 99 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.824 4. analysis of results 4.1 performance of sharia banks based on sharia conformity and profitability (scnp) as measured by sharia conformity and profitability (scnp), the results of data processing related to the financial performance of sharia banking in indonesia can be seen in table 4. table 4 sharia bank performance based on scnp in 2019 sharia bank sharia conformity profitability quadrant bank syariah mandiri 79,13% 9,92% urq bank muamalat indonesia 77,52% 0,33% ulq bri syariah 82,13% 1,31% lrq bni syariah 78,41% 8,86% llq bank mega syariah 76,59% 3,45% ulq source: analysis results (2020) table 4 shows that sharia commercial banks in indonesia were divided into four quadrants in 2019, namely the upper right quadrant (urq), the upper left quadrant (ulq), the lower right quadrant (lrq), and left lower quadrant (llq). the results are according to research conducted by ratnaputri (2013) which states that the performance of islamic commercial banks based on scnp is spread over four quadrants. in this study, it is assumed that the sharia conformity indicator is more important than profitability considering the main objective of islamic banking is the benefit of the people by carrying out economic activities in accordance with islamic studies. in general, the performance of islamic commercial banks using sharia conformity and profitability (scnp) can be seen in figure 4. figure 4 graph scnp source: output spss (2020) 4.2 performance of sharia banks is based on the maqasid index table 5 below shows the results of a report on the ranking of sharia banks in indonesia based on the maqasid index. ijahp article: jihadi, vilantika, sholichah, arifin/best sharia bank in indonesia: an analytical hierarchy process (ahp) approach international journal of the analytic hierarchy process 100 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.824 table 5 rating of sharia banks based on the maqasid sharia index in indonesia sharia bank ranked maqasid index bank syariah mandiri 2 bank muamalat indonesia 1 bri syariah 3 bni syariah 4 bank mega syariah 5 source : ramdhoni and fauzi (2020); nugraha, nugroho, lindra, and sukiati (2020) bank muamalat indonesia occupies the first place (see table 5) based on a maqasid index evaluation. these findings are consistent with the research results of antonio et al. (2012), prasetyowati and handoko (2016), ramdhoni and fauzi (2020) and nugraha et al. (2020), which state that bank muamalat is a sharia bank with the best performance based on the maqasid index. 4.3 sharia bank performance based on camels this research used four indicators from camels because it focuses on the financial factors, but did not use the profitability indicator because it has been calculated in the evaluation of the scnp. the results of the camels-based processing of financial performance data at sharia banks in indonesia can be seen in table 6. table 6 performance of sharia banks based on camels in 2019 sharia bank capital (car) asset quality (npf) efficiency (oer) liquidity (fdr) bank syariah mandiri 16,15% 1,00% 82,89% 75,54% bank muamalat indonesia 12,42% 4,30% 99,50% 73,18% bri syariah 25,26% 3,38% 96,80% 80,12% bni syariah 18,88% 1,44% 81,26% 74,31% bank mega syariah 19,96% 1,72% 93,71% 94,53% source: annual report (2019) table 6 shows that all sharia banks in indonesia were categorized as “very healthy” during 2019. 4.4 sharia bank performance assessment using the ahp method following the collection of data about sharia banks using scnp, the maqasid index and camels, the next step was to use the expert choice decision program to pick the best bank using the analytical hierarchy process (ahp). the results are shown in figure 4. ijahp article: jihadi, vilantika, sholichah, arifin/best sharia bank in indonesia: an analytical hierarchy process (ahp) approach international journal of the analytic hierarchy process 101 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.824 figure 4 criteria weights source: output expert choice 11 (2020) based on the weighting of the criteria, the most important sharia bank performance appraisal is the maqasid index (0.467). this is in line with research conducted by al ghifari, handoko, and yani (2015) and (ramdhoni and fauzi (2020). they stated that the use of the maqasid index as a measurement of the performance of sharia banks is considered the most comprehensive because the sharia conformity and profitability (scnp) approach still adopts conventional ratios and therefore does not completely include islamic aspects, while the camels approach only uses conventional ratios. the consistency index (ci) is 0.08, and is accepted because it is lower than 0.10 (10%). the next step is to find the eigenvalues of each bank based on the criteria. the results are shown in table 7. table 7 alternative-criteria eigenvalues capital (0.074) asset (0.039) efficiency (0,024) liquidity (0.140) scnp (0,257) mi (0.467) bank syariah mandiri 0.063 0.485 0.268 0.135 0.325 0.261 bank muamalat indonesia 0.033 0.035 0.034 0.486 0.056 0.513 bri syariah 0.513 0.068 0.070 0.071 0.401 0.129 bni syariah 0.129 0.273 0.496 0.272 0.140 0.063 bank mega syariah 0.261 0.139 0.132 0.037 0.079 0.033 source: analysis results (2020) the final step is weighting to determine which bank has the best performance based on the largest weight. the final weighting resuls can be seen in table 8. model name: ahp syariah priorities with respect to: goal: choosing the best islamic bank capital .074 asset quality .039 efficiency .024 liquidity .140 scnp .257 maqashid .467 inconsistency = 0.08 with 0 missing judgments. page 1 of 110/2/2011 11:16:57 a dapendapen ijahp article: jihadi, vilantika, sholichah, arifin/best sharia bank in indonesia: an analytical hierarchy process (ahp) approach international journal of the analytic hierarchy process 102 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.824 table 8 final assessment results sharia bank final weight bank syariah mandiri 0.254 bank muamalat indonesia 0.326 bri syariah 0.214 bni syariah 0.137 bank mega syariah 0.069 total 1 source: analysis results (2020) table 8 shows that bank muamalat indonesia has the largest weight, namely 0.326. therefore, the performance of bank muamalat indonesia is the best when compared to other islamic banks. bank syariah mandiri ranks second and bri syariah ranks third. these are the three best sharia banks based on their 2019 performance. 5. conclusion the most important aspect in decision making is choosing the right criteria for achieving the stated goals. this research is based on several articles on the performance of sharia banks, and uses several criteria, namely capital, asset quality, efficiency, liquidity, sharia conformity and profitability, and the maqasid index. data were collected based on the annual reports of each bank which were then analyzed using a multi-criteria decision making tool, namely the analytic hierarchy process (ahp). based on the results of data analysis, bank muamalat indonesia is the best performing islamic bank in indonesia. bank muamalat indonesia is superior when using the maqasid index criteria, where in the pairwise comparison matrix for the criteria, the maqasid index has the greatest weight. al ghifari (2015) and ramdhoni & fauzi (2020) state that the use of the maqasid index as a measurement of the performance of islamic banks is considered more comprehensive than other measures because the sharia conformity and profitability (scnp) approach still adopts conventional ratios and does not include islamic aspects as a whole. bedoui and walid (2013) state that maqashid sharia covers all sides of life that intersect with individuals, social, society, economics, and intellectuals. the maqasid index and sharia conformity and profitability (scnp) can complement the camels approach as instruments for measuring the performance of islamic banks. the camels analysis, which is a measurement tool for conventional banks, is considered less reflective of the performance of islamic banks; therefore, the use of the scnp and the maqashid index aim to provide better results when compared to measurements using conventional methods (prasetyowati dan handoko, 2016). 6. practical implications it is time for islamic commercial banks to use performance measurement tools to measure their compliance with sharia law in addition to the use of the conventional ijahp article: jihadi, vilantika, sholichah, arifin/best sharia bank in indonesia: an analytical hierarchy process (ahp) approach international journal of the analytic hierarchy process 103 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.824 performance measurement tools that are commonly used. this will help fully understand the financial achievements of islamic banks and their relationship with the teachings and objectives of sharia. the implementation of islamic banking performance measurement tools based on the sharia maqashid index and sharia conformity and profitability (scnp) methods is expected to be considered by regulators in order to provide a more complete picture of the performance of islamic banking that does not only consider the financial aspects. 7. limitations and future research the camels method used in this study did not include all factors, such as sensitivity to market risk. this is because the data obtained by the researchers was incomplete, especially data related to additional capital and potential exchange rate losses which are used as a formula/ratio to calculate the sensitivity factor. in addition, the number of islamic banks is limited, so hopefully more alternative choices will be available for future research and a longer period of time of data collection can be used to obtain more accurate results. ijahp article: jihadi, vilantika, sholichah, arifin/best sharia bank in indonesia: an analytical hierarchy process (ahp) approach international journal of the analytic hierarchy process 104 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.824 references abduh, m., & omar, m. a. (2012). islamic–bank selection criteria in malaysia: an ahp approach. business intelligence journal, 5(2), 271-281. ahsan, m. k. (2016). measuring financial performance based on camel: a study on selected islamic banks in bangladesh. asian business review, 6(1), 47-56. doi: https://doi.org/10.18034/abr.v6i1.803 al ghifari, m., handoko, l. h., & yani, e. a. (2015). analysis of sharia banking performance in indonesia and malaysia with the maqashid index approach. journal ekonomi dan perbankan syariah, 3(2), 47-66. doi: https://doi.org/10.46899/jeps.v3i2.160 antonio, m. s., sanrego, y. d., & taufiq, m. (2012). an analysis of islamic banking performance: maqashid index implementation in indonesia and jordania. journal of islamic finance, 1(1), 012-029. apriani, n., kusnendi, & firmansyah. (2018). implementation of good governance business sharia (ggbs) and its implications for sharia conformity and profitability (scnp) financial performance in sharia commercial banks in indonesia. review of islamic economics and finance (rief), 1(1), 1-10. doi: https://doi.org/10.17509/rief.v1i1.23741 bashatweh, d. a. d., & ahmed, p. e. y. (2020). financial performance evaluation of the commercial banks in jordan: based on the camels framework. international journal of advanced science and technology, 29(5), 985-994. bateni, l., vakilifard, h., & asghari, f. (2014). the influential factors on capital adequacy ratio in iranian banks. international journal of economics and finance, 6(11), 108-116. doi: 10.5539/ijef.v6n11p108 bedoui, m. h. e. (2012). shari'a-based ethical performance measurement framework. chair for ethics and financial norms. bedoui, h., & walid, m. (2013). islamic bank performance and maqashid al shariah. paper presented at 9th asia-pasific economics association conference, di osaka, jepang, 27-28. djuwita, d. (2019). the influence of sharia compliance and islamic corporate governance on financial performance of sharia commercial bank. al-amwal: jurnal edonomi dan perbankan syari'ah, 11(2), 205-220. doi: 10.24235/amwal.v11i2.4072 grant cr, pradata ne, muljosantoso r. 2017. islamic finance and markets – indonesia’ (gettingthedealthrough.com, september 2017) accessed 28 oct 2019. ijahp article: jihadi, vilantika, sholichah, arifin/best sharia bank in indonesia: an analytical hierarchy process (ahp) approach international journal of the analytic hierarchy process 105 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.824 hartono, s., & sobari, a. (2017). sharia maqashid index as a measuring performance of islamic banking: a more holistic approach. corporate ownership & control, 14(2), 193201. doi: 10.22495/cocv14i2c1p5 islami, t. u., & ashrafuzzaman, m. (2015). a comparative study of islamic and conventional banking in bangladesh: camel analysis. journal of business and technology (dhaka), x(1), 73-91. doi: https://doi.org/10.3329/jbt.v10i1.26907 kouser, r., & saba, i. (2012). gauging the financial performance of banking sector using camel model: comparison of conventional, mixed and pure islamic banks in pakistan. international research journal of finance and economics (82). kuppusamy, m., saleh, a. s., & samudhram, a. (2010). measurement of islamic banks performance using a shariah conformity and profitablity model. review of islamic economics and finance (rief), 12(2), 35-48. lesmana, s., & haron, m. h. (2019). maqasid shariah based performance of islamic banks, islamic corporate governance, and contingency theory: a theoretical framework. international journal of accounting, finance and business (ijafb), 4(24), 70-86. mega, s., isni, a., & taufiq, m. (2019). performance of indonesian islamic banks based on maqasid shariah index: stakeholder theory perspective. eurasia: economics & business, 7(25), 18-27. doi: https://doi.org/10.18551/econeurasia.2019-07 mohammed, m. o., razak, d.a. (2008). the performance measures of islamic banking based on the maqashid framework. paper of iium international accounting conference (intac iv) held at putra jaya marroitt, 1-17. nugraha, e., nugroho, l., lindra, c. n., & sukiati, w. (2020). maqashid sharia implementation in indonesia and bahrain. etikonomi, 19(1), 155-168. doi: https://doi.org/10.15408/etk.v19i1.14655 prasetyowati, l. a., & handoko, l. h. (2016). pengukuran kinerja bank umum syariah dengan maqasid index dan sharia conformity and profitability (scnp). jurnal akuntansi dan keuangan islam, 4(2), 107-130. doi: https://doi.org/10.35836/jakis.v4i2.22 ramdhoni, m. i., & fauzi, f. a. (2020). islamic banks performance: an assessment using sharia maqashid index, sharia conformity and profitability and camels. international journal of applied business research, 2(1), 15-30. doi: https://doi.org/10.35313/ijabr.v2i01.79 ratnaputri, w. (2013). the analysis of islamic bank financial performance by using camel and shariah conformity and profitability (scnp). jurnal dinamika manajemen, 4(2), 215-226. razak, abd hakim abd. 2015. the fundamentals of islamic banking and finance: a prologue. ejif – european journal of islamic finance. 2, 1-11 ijahp article: jihadi, vilantika, sholichah, arifin/best sharia bank in indonesia: an analytical hierarchy process (ahp) approach international journal of the analytic hierarchy process 106 vol. 13 issue 1 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i1.824 rusydiana, a. s., & parisi, s. a. (2016). the measurement of islamic bank performance: a study using maqasid index and profitability. global review of islamic economics and business, 4(1), 001-014. doi: https://doi.org/10.14421/grieb.2016.041-01 sekaran, u. (2000). research methods for business: a skill building approach. new york: john wiley & sons. shahhoseini, m. a., khassehkhan, s., & shanyani, n. (2012). identifying key performance indicators of an iranian islamic bank based on bsc and ahp. journal of american science, 8(1s), 64-73. suresh, d. c., & bardastani, m. m. (2016). financial performance of selected conventional and islamic banks in kingdom of bahrain – a camel ranking based approach. european journal of economic, law and politics (elp), 1(1), 24-59. doi: https://doi.org/10.19044/elp.v3no1a2 tabash, m. i. (2017). critical challenges affecting islamic banking growth in india using analytical hierarchy process (ahp). banks and bank systems, 12(3), 27-34. doi: https://doi.org/10.21511/bbs.12(3).2017.02 tarique, islam, mohamed, razak, & hamdan. (2020). constructing a maqasid (objective) based performance measurement index for islamic banks. international journal of the analytic hierarchy process, 12(2), 328-354. doi: https://doi.org/10.13033/ijahp.v12i2.770 wulandari, m. v., suryana, & utami, s. a. (2019). determinant of non-performing financing in indonesia islamic bank. the 2nd international conference on islamic economics, business, and philanthropy (iciebp) theme: “sustainability and socio economic growth”, kne social sciences, 453-468. doi: 10.18502/kss.v3i13.4223 swot analysis ijahp article: el hefnawy, soliman mohammed/review of different method for deriving weights in the analytic hierarchy process international journal of the analytic hierarchy process issn 1936-6744 92 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.226 review of different methods for deriving weights in the analytic hierarchy process ali el hefnawy faculty of economics and political science cairo university, egypt e-mail: ahefnawy@aucegypt.edu ahmed soliman mohammed institute of national planning, egypt e-mail: ahmed_inp@hotmail.com abstract the analytic hierarchy process (ahp) is a multi-criteria decision making technique developed to solve both single and group decision making problems. the ahp approach that was proposed by saaty to derive priority weights by using the eigenvector method (evm) has its drawbacks, and since its publication various methods have been developed as alternatives. this paper presents an extensive review of various methods for deriving priority weights in ahp, including group decision making, and focuses on comparison of each method with the evm. the results of the comparison reveal that each method has its advantages and disadvantages, and the decision of which method to use should be based on the application. keywords: analytic hierarchy process, review, multicriteria decision, group decision making. 1. introduction the analytic hierarchy process (ahp) is one of the most widely used tools to solve multi-criteria decision making problems. it was developed by thomas saaty in the 1970's while he was directing research projects for the arms control and disarmament agency at the u.s. department of state. during his successive work, saaty realized that there was a lack of common, easily understood, and easily applicable methodology to facilitate making complex decisions (forman and gass, 2001). therefore, he attempted to develop a simple, practical, systematic way to help people make complex decisions, and the result was the ahp. saaty criticized ignoring the human factors, at least in social sciences, by making simplified assumptions to suit the quantitative models. to make realistic models, all important tangible and intangible, quantitatively measurable, and qualitative factors must be included and measured, and this is what the ahp attempts to accomplish (saaty, 1988). the power and simplicity behind the ahp has led to its widespread acceptance and use throughout the world; a literature review revealed that the ahp model has been widely and successfully applied to a wide range of problem situations that include: o the selection of one alternative from a given set of alternatives, where multiple decision criteria are involved. o ranking by ordering a set of alternatives from most to least desirable. mailto:ahefnawy@aucegypt.edu ijahp article: el hefnawy, soliman mohammed/review of different method for deriving weights in the analytic hierarchy process international journal of the analytic hierarchy process issn 1936-6744 93 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.226 o prioritization by determining the relative merit of members of a set of alternatives, as opposed to selecting a single one or merely ranking them. o allocation of scarce resources. o predicting outcomes. o measuring performance. there are also numerous articles and doctoral dissertations that discuss the applications of ahp in many fields. for example, zanakis, et al. (1995) studied over 100 applications of ahp within the service, social/manpower, natural resource/energy, education, and government sectors. vaidya and kumar (2006) reviewed about 32 articles that use ahp in selection problems, 26 articles in evaluation problems, 7 articles in benefit–cost analysis, 10 articles in allocation problems, 18 articles in the field of planning and development, 20 articles in priority and ranking problems, 21 articles in decision making problems, 4 articles in forecasting, and 5 articles in medicine and related fields. in spite of its widespread use and applications, saaty's approach for deriving weights and related consistency measure in ahp using the eigenvector method (evm) has some drawbacks which have been discussed in the literature from different perspectives. therefore, several attempts have been made to suggest different techniques to overcome these drawbacks; some of the proposed methods are deterministic while the others are stochastic ones. however, it is strongly noted that evm is still the most used method to derive weights in the vast majority of the applications of ahp (ishizaka and labib, 2011; mirhedayatian et. al., 2011; grzybowski, 2010). this paper aims to provide a detailed review of the basic, theoretical concepts as well as the methodological developments of the ahp, focusing on the method of deriving weights in both single and group decision making problems. it is hoped that this review will help researchers to be aware of the successive methodological developments of ahp since its appearance, and to take advantage of these developments in the applications. although the main developments of the ahp have been briefly reviewed in ishizaka and labib (2011), the current paper tries to provide a wider and more detailed review about the methodological development of ahp in both single and group decision making problems. it is worth mentioning that the review of the ahp applications is not in the scope of this paper, and it has been discussed in other papers such as subramanian and ramanathan (2012), ho, (2008), and vaidya and kumar (2006). this paper consists of five sections and is organized as follows. in section 2, we review the basics of ahp including structuring the decision problem, pairwise comparison, deriving priority weights, and synthesizing to get the final weights. section 3 discusses the different methods for deriving priority weights. section 4 discusses methods for deriving priority weights in group decision making problems, while section 5 gives some conclusions. 2. basics of ahp ahp is a structured technique for organizing and analyzing complex decision problems. the goal of the ahp is to find a unique vector of priority weights ),...,,( 21 n wwww of n alternatives with respect to a given criterion. the priority weights are chosen to be nonnegative and normalized to unity. the actual priority ijahp article: el hefnawy, soliman mohammed/review of different method for deriving weights in the analytic hierarchy process international journal of the analytic hierarchy process issn 1936-6744 94 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.226 ratios j i w w which represent the pairwise comparison of each two elements can't be given precisely, but they can be translated into verbal expressions and then estimated by a decision maker or an expert who may make small errors in judgment (saaty, 1994). to obtain the weights for the alternatives using ahp, one should follow four primary steps: structuring the complex problem in a hierarchical form, constructing the pairwise comparison judgments, the prioritization procedures, and finally synthesizing through the structure to get the final weights for the alternatives. 2.1 structuring the problem in a hierarchical form structuring a decision is the first step that should be taken to organize and represent any problem, particularly a decision problem. to deal with a problem, one attempts to identify the elements that relate to it, their connections and interactions, the cause(s) that give rise to the problem, and possible ways to a solution (saaty and shih, 2009). structuring the decision problem in ahp is performed by decomposing the problem into its constituent parts and then presenting them in a hierarchy form (figure 1). this structure comprises the main goal at the top level, criteria at the intermediate level, and finally the lowest level contains the options or alternatives. the criteria can be further broken down into subcriteria, sub-subcriteria, and so on, in as many levels as required. the elements at a particular level of the hierarchy are kept, independent of, but comparable to, the elements at the same level, and elements at any level are directly related to elements at the level immediately below them (ramanujam and saaty, 1981). to build the hierarchy structure for the decision problem, saaty (1994) suggested the following main steps: identify the overall goal on the top level of the structure. the main goal can be identified by answering the following questions: what are we trying to accomplish? what is the main question? identify criteria that must be satisfied to fulfill the overall goal. identify sub-criteria under each criterion, if relevant. the options or alternatives are added at bottom of the structure. representing problems in a hierarchal form has two basic advantages; it provides an overall view of the complex system of the situation, and helps the decision maker assess the homogeneity of the issues in each level, so he can that compare such elements accurately. on the other hand, an element in a given level in the hierarchal form does not need to function as an attribute for all the elements in the level below (saaty, 1990). also, the decision maker can add or delete levels and elements to the hierarchy as necessary. ijahp article: el hefnawy, soliman mohammed/review of different method for deriving weights in the analytic hierarchy process international journal of the analytic hierarchy process issn 1936-6744 95 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.226 figure 1. hierarchy structure in ahp 2.2 pairwise comparison judgments using ratio scale the second step in the ahp is the construction of the matrix of comparison judgments given by a decision maker or an expert. the judgment or comparison is a numerical representation of a relationship between two elements that share a common parent in the hierarchy, and it represents the dominance or importance of an element over the other one in the same level. the set of all such judgments can be represented in a square matrix called a pairwise comparison matrix (pcm), in which the set of elements are put in the rows and columns of the matrix where the set of elements is compared with itself. in the pcm, the decision maker is asked to evaluate the various elements of the problem; each element is evaluated with regard to the other one. the pairwise comparison process is strongly recommended by psychologists; they argue that, to express one’s opinion on only two alternatives is easier and more accurate than to do it simultaneously on all the alternatives. moreover, it allows consistency cross checking between the different pairwise comparisons (ishizaka and labib, 2001). to make pairwise comparisons, saaty proposed a ratio scale measurement (called the fundamental scale) to be used by the policy maker or expert to indicate how many times more important or dominant one element is over the other one for each pair of elements (criteria or alternatives). this ratio scale ranges from one (equal importance) to 9 (extreme importance) (table 1). if, for example, element i is assigned 3 in comparison with element j, this means that element i is three times important compared to element j with regard to the overall goal. saaty (1990) stated that the scale (1.1, 1.2 . . . . . 1.9) can be used when the elements being compared are closer together than indicated by the above scale. table 1 the fundamental scale used in ahp dominance of importance definition 1 equal importance 2 weak or slight 3 moderate importance 4 moderate plus 5 strong importance 6 strong plus 7 very strong importance 8 very, very strong 9 extreme importance source: (saaty, 1990). ijahp article: el hefnawy, soliman mohammed/review of different method for deriving weights in the analytic hierarchy process international journal of the analytic hierarchy process issn 1936-6744 96 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.226 saaty and ozdemir (2003) suggested that, for the consideration of consistency of the information derived from relations among the compared elements, the number of compared elements should not be large (not more than seven). on the other hand, saaty (2001) stated that, for the consistency and homogeneity condition required for stability of the eigenvector of priorities, the upper limit of the scale should not be more than 9. he argued that this condition is intrinsic to the way in which our brains actually operate; qualitatively, people can only deal with information involving a few facts simultaneously. the set of all pairwise comparison judgments are presented in a square matrix in which the set of compared elements is put on the rows and columns of the matrix. this matrix is a "reciprocal symmetric matrix" (rsm) and takes the following form: a =                     nnnn n n n 1 n aaa aaa aaa a a a aaa ........ ........ ........ 21 22221 11211 2 21   the rsm satisfies the following conditions:  it is a matrix of non-zero elements, njia ij ,...,1,,0  (1)  comparing an element with itself is always assigned the value 1, so the main diagonal entries of the matrix are all 1, nia ii ,...,1,1  (2)  elements below the diagonal are the reciprocal of the elements above the diagonal. this means that if p represents the dominance of element a over element b, then the dominance of b compared to a is p 1 , njijifor a aoraa ji ijjiij ,...,1,, 1 1  (3) in ahp, a number of comparison matrices are constructed; one matrix for the criteria in terms of their importance to achieve the overall goal, and a number of matrices are constructed for the alternatives with regard to each criterion. in some cases when the number of alternatives is large, we face a problem of missing comparisons judgments, i.e. the comparison matrix is incomplete. in such cases, missing entries must be filled in with values that improve the consistency of the matrix. according to harker (1987), there are three reasons for obtaining incomplete judgments. these reasons are first, the great time and effort needed to complete 2/)1( nn comparisons; second, unwillingness of the decision maker/expert to make a direct comparison between two alternatives; and third, being unsure of some of the comparisons. deriving priorities from incomplete comparison matrices has been discussed in the literature by several studies such as harker (1987a; 1987b), shiraishi et al. (1998), nishizawa (2005), fedrizzi and giove (2007), and gomez-ruiz et al. (2010). ijahp article: el hefnawy, soliman mohammed/review of different method for deriving weights in the analytic hierarchy process international journal of the analytic hierarchy process issn 1936-6744 97 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.226 2.3 prioritization procedures as was mentioned before, the goal of ahp is to find a unique, nonnegative, and normalized natural vector of priorities nwww ,...,, 21 of alternatives with respect to a number of criteria. the actual priority ratios j i w w which represent the pairwise comparison of each two elements cannot be given precisely, but they can be estimated by judgments of a decision-maker or an expert about alternatives and criteria that are used to judge the alternatives. so, in the conventional ahp, a decision-maker estimates these ratios of priorities, which form the pcm   nnij aa   . in reality, we cannot expect that the elements of pcm give exactly priority ratios, as the human mind is not a perfect measurement device. human judgments may depend on personal taste, experience, specific knowledge, the judge’s temporary mood and temper and may vary with time. based upon this, checking consistency is very important for the comparison matrix before using it to derive priority weights. therefore, the values of the pcm ( ija ) are taken as estimates of the actual priority weight ratios nji w w j i ,...,1,,  for the compared elements in the matrix. then, priority weights vector ),...,,( 21 n wwww for these elements are derived based on the values of the pcm (a): a =                     nnnn n n n 1 n aaa aaa aaa a a a aaa ........ ........ ........ 21 22221 11211 2 21   ≈                     n nnn n n n 1 n w w w w w w w w w w w w w w w w w w a a a aaa ........ ........ ........ 21 2 2 2 1 2 1 2 1 1 1 2 21  a necessary and sufficient condition for matrix a to be perfectly consistent is that nji w w a j i ij ,...,1,,  (4) saaty (1990) stated the following conditions for the positive pcm to be consistent: 1) reciprocal: njijifor a a ji ij ,...,1,, 1  (5) 2) transitive: nkjiforaaa ikjkij ,...,1,,,  (6) therefore, measuring the degree of inconsistency of the pcm is a fundamental issue in ahp. to derive priority weights from pairwise comparison matrices as well as to measure the degree of inconsistency of the pcm, saaty proposed using the evm. he stated that the priority vector should satisfy two conditions; the first is that it belongs to a ratio scale which means that it should remain invariant under multiplication by a positive constant, the second is that it must be unique which implies that it should be invariant under hierarchic composition for its own judgment matrix so that one does not keep getting new priority vectors from that matrix (saaty, 2003). according to the evm, the required priority vector w for a consistent matrix a must satisfy the relation waw n , n> 0, where a has been multiplied on the right by the transpose of the vector of weights w as follows: ijahp article: el hefnawy, soliman mohammed/review of different method for deriving weights in the analytic hierarchy process international journal of the analytic hierarchy process issn 1936-6744 98 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.226 nw. = w w n = w w w w w w w w w w a a = aw a a n 1 n 1 n n 1 n n 1 1 1 n 1 n1                                             the result of this multiplication is nw. thus, solving this system of homogeneous linear equations waw n or 0ni) w-(a results in the right eigenvector of the a matrix corresponding to the eigenvalue n. it is clear that matrix a has a rank one because every row is a constant multiple of the first row. thus, all its eigenvalues except one are zero, and the sum of the eigenvalues of the matrix is equal to its trace, the sum of the diagonal elements. in this case, the trace of a is equal to n. therefore, n is the largest, or principal, eigenvalue of a (saaty, 1990). then, the solution of waw n is the principal right eigenvector w corresponding to the principal eigenvalue n of a, and it consists of positive entries and gives the desired unique priority vector. according to saaty (1990, 2003), a necessary and sufficient condition for consistency is that the principal eigenvalue of a is equal to n, the order of that matrix. the inconsistent matrix is the derivation from a consistent one by perturbation of some or all the matrix components. because the eigenvalues and eigenvectors of a matrix depend continuously on components of the matrix, small perturbations in the components will result in small changes in the eigenvalues and eigenvectors. thus, when the perturbations of the components are small, the maximal eigenvalue is close to n, and the corresponding normalized eigenvector is close to the normalized eigenvector of the unperturbed consistent matrix. therefore, saaty argued that the normalized eigenvector is also valid for inconsistent matrices. from the perturbation theory, it is known that a small perturbation around a simple eigenvalue when a is consistent leads to an eigenvalue problem of the form waw max λ , and a may no longer be consistent but is still reciprocal (saaty, 1990). saaty proved that n max  , with the equality if and only if a is consistent. the question now is to what extent the inconsistent comparison matrix can be used to derive priority weights. to answer this question it is necessary to measure the consistency level for comparison matrices. therefore, saaty proposed a measure of consistency, called the consistency ratio (cr). the cr is a ratio of the consistency index (ci) to the random index (ri). ri ci cr  (7) 1   n n ci max  (8) where n is the dimension of the pcm, max is maximal eigenvalue, and ri is calculated as the average value of consistency indices of a 500 randomly generated reciprocal matrices from the scale 1 to 9. the average ri and the order of the matrix are shown in table 2. ijahp article: el hefnawy, soliman mohammed/review of different method for deriving weights in the analytic hierarchy process international journal of the analytic hierarchy process issn 1936-6744 99 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.226 table 2 saaty's random index n 1 2 3 4 5 6 7 8 9 10 ri 0 0 0.52 0.89 1.11 1.25 1.35 1.40 1.45 1.49 source: (saaty, 1990). if the cr is less than or equal to 10%, the matrix can be considered to have an acceptable consistency. if one can't reach this acceptable level, one should gather more information or reexamine the framework of the hierarchy. ishizaka and labib (2011) stated that other researchers have run simulations with different numbers of matrices and their random indices were different but close to saaty’s. grzybowski (2010) argued that even if the comparisons are done very carefully, pcm is inconsistent and we have to express the relation between the pcm elements and the priority weights in the form: j ijij w w ea  , where ije is a perturbation factor which is expected to be near 1, and in the statistical approach this factor is interpreted as a realization of a random variable. therefore, several researchers have attempted to propose new methods to measure consistency other than the cr. for example; crawford and williams (1985) suggested a new measure for consistency based on the sum the difference between the ratio of calculated priorities j i w w and the given comparisons in the geometric consistency index (gci), where: )2()1( )ln(ln2 2      nn w w a gci j i ji ij (9) then, the smaller the value of gci, the smaller the distance between the judgments ij a and the ratios j i w w will be, and the better the fit between judgments and priorities. aguaron and moreno-jimenez (2003) proposed thresholds for the gci which are analogous to that proposed by saaty's cr. dodd, et. al. (1993) proposed a statistical approach based upon the significance level concept (95% or 99% …). the level of confidence is selected by the decision maker(s) and differs according to the decision problems. the ci for a given comparison matrix is compared to the tolerance level at the selected confidence level, and then the decision is made to reject or accept the matrix. moreover, zeshui and cuiping (1999) proposed a convergent algorithm to modify consistency for a given comparison matrix using the following relationship:              1 )( )( )( )( )1( k j k ik ijij w w aa k (10) where ),....,,( 21 n wwww  is the principle eigenvector corresponding to the maximal eigenvalue max  , 10   . this process is repeated until the modified matrix satisfies the consistency condition. ijahp article: el hefnawy, soliman mohammed/review of different method for deriving weights in the analytic hierarchy process international journal of the analytic hierarchy process issn 1936-6744 100 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.226 also, peláez and lamata (2003) proposed a method based on the determinant of the pcm. alonso and lamata (2006) proposed a new criterion for accepting/rejecting pcm in ahp using the average principal eigenvalues max  . stein and mezzi (2007) proposed a new consistency measure called the harmonic consistency index (hci), based on the method of additive normalization (an). ergu et al.( 2011) introduced a new method based on the theorem of matrix multiplication, vectors dot product, and the definition of consistent pairwise comparison matrix. borodin et al. (2011) proposed two methods for correcting the elements of an inconsistent pairwise comparison matrix to get a consistent one using a correction matrix h such that; ‖ ‖ so that ( ) is consistent. the first method uses the lagrange multipliers, while the second solves the correction problem by correcting the inconsistent set of linear equations with the spare structure. 2.4 synthesizing through the structure to get the final weights after computing priority weights for the criteria and alternatives, the final step is to obtain the overall priority weights. this is done for each alternative by multiplying the priority weight obtained from the alternatives' comparisons according to a specific criterion by the corresponding priority weight of this criterion, and then added over all the criteria. the alternative with the highest priority weight value should be taken as the best one. this synthesizing process can be illustrated as follows: ( ) ∑ [ ( ) ( ) ] (11) where: ( ) is the weight value of alternative , ( ) weight value of the i th criterion ( ), ( ) is the weight value of alternative with regard to of the i th criterion ( ), . this process is repeated to all the alternatives to get the final priority weight vector . 3. different methods for deriving priority weights a review of the literature revealed that using evm to derive priority weights in ahp has some drawbacks including the following: a) condition of order preservation condition of order preservation (cop) means that the ranking process should not only indicate an order of preference among the alternatives, but also reflect the intensity or the cardinal preference among these alternatives as it is indicated by the ratios of the numerical values. for example, if there are four alternatives such that strongly dominates , and moderately dominates , then whenever possible, the vector of priorities w be such that 4 3 2 1 w w w w  , which indicates that the intensity of preference of over is higher than the intensity of preference of over . costa and vansnick (2008) argued that using evm to derive weights in ahp does not satisfy this property. b) the right and left eigenvector inconsistency the second drawback is related to the right and left eigenvector inconsistency; where saaty used only right eigenvectors, the use of left eigenvectors should be equally justified. for ranking a set of alternatives in a decision problem, both eigenvector approaches should yield the same result. unfortunately, it is often not the case in ahp, the solution of the right eigenvector waw n which gives the right ijahp article: el hefnawy, soliman mohammed/review of different method for deriving weights in the analytic hierarchy process international journal of the analytic hierarchy process issn 1936-6744 101 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.226 eigenvector w is not necessarily the same as the left eigenvector solution  tt waw  (ishizaka and labib, 2011). this right and left inconsistency arises only for inconsistent matrices with a dimension greater than 3. c) rank reversal rank reversal phenomenon refers to the changes of the relative rankings of the other alternatives after a new alternative is added or a current one is deleted. such a phenomenon was first noticed and pointed out by belton and gear (1983). in order to avoid the rank reversal, they suggested normalizing the eigenvector weights of alternatives using their maximum rather than their sum, which was usually called b– g modified ahp. however, saaty and vargas (1985) provided a counterexample to show that b–g modified ahp was also subject to rank reversal. barzilai and golany (1994) showed that no normalization method could prevent rank reversal. several other studies investigated the problem, and the literature shows that the rank reversal phenomenon has not been perfectly resolved, and debate still exists about the ways of avoiding rank reversals (wang and elhag, 2006). in addition to the problems discussed above, other disadvantages of evm have been discussed regarding its calculation difficulties and the lack of a practical statistical theory behind it. the eigenvalue method is a deterministic (non-stochastic) method to derive priority weights where the errors in judgments are assumed not to exist. therefore, several researchers have attempted to present different methods to derive priority weights in ahp. each method has its advantages and disadvantages where there is no specific standard to measure the superiority of one over the others. some of these methods are deterministic, and others used the stochastic techniques. they are discussed in the following subsections. 3.1 deterministic methods 1) mean of normalized values this is the oldest method introduced by psychologists who used the pairwise comparison matrices before saaty (ishizaka and labib, 2001). this method can be, briefly explained in the following steps: computing the sum of each column of the pcm a: njas n i ijj ,...,1, 1    . normalizing the elements of matrix a by dividing each value by its column sum: nji s a a j ij ij ,...,,1,,~  . computing the mean of each row: ni n a w n i ij i ,...,1, ~ 1    . the vector of mean values calculated in step three is taken as the priority weight vector w. for this method, no mathematical approach is available for dealing with inconsistent matrices. 2) method of least square the proposed least square method (lsm) is based on minimizing the sum square of the differences (distances) between the actual priority weight ratios j i w w and the elements of the comparison matrix ( ij a ) such that j i ij w w a  . for example: ijahp article: el hefnawy, soliman mohammed/review of different method for deriving weights in the analytic hierarchy process international journal of the analytic hierarchy process issn 1936-6744 102 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.226 chu et al. (1979) utilized a weighted least-square method by solving the following minimizing problem: min     n i i n i n j ijij wwwas 11 1 2 2)(  (12) where  is the lagrange multiplier. differentiating the previous equation with respect to m w yields the following set of equations:     n i mmjim n i imim nmwaawwa 11 ,...,2,1,0)()(  (13) these equations in addition to the one of 1 1   n i i w form a set of ( ) inhomogeneous linear equations with ( ) unknowns and can be solved for ,,...,, 21 n www . moreover, they stated that the weighted least-square method is conceptually easier to understand than the evm. however, at the same time, they argued that the evm guarantees that the obtained priority vector is nonnegative, while it is not known whether such a theorem exists for the weighted least-square method. although the numerical results they obtained indicate that the weights obtained by this method are greater than zero and are comparable to those obtainable by the evm. cogger and yu (1985) introduced a new eigen-weight technique to derive priority weights. unlike saaty's approach, they were interested in the upper diagonal elements of the comparison matrix. they defined three matrices t = [tij], u = [uij], and d = [dij] where         o.w,0 ijifij ij a t ,         o.w,0 ijifij ij w u ,         o.w,0 ijif1in dij , and j i ij w w w  . in addition to the column weight vector w with elements niw i ,...,1,  , 0iw , and 1 1   n i i w . it is clear from the above definitions that: dwuw  or wuwd  1 (14) thus, w may be interpreted as the normalized eigenvector of d -1 u corresponding to the unit eigenvalue. since t is an estimate of u, a reasonable estimate of w would be any nonzero solution w  , if it exists, to the equation 0)( 1   witd  . they argued that the new eigen-weight method commands some advantages over the evm that is used by saaty, including less rigid assumptions on the error terms, robustness of solution, in addition to the computational advantage. crawford and williams (1985) proposed the logarithmic least squares method (llsm) to minimize the multiplicative error ij e such that: j i ijij w w ea  , nji ,...,1,  (15) the objective in that model is to find the weight vector ),...,,( 21 n wwww that minimizes the sum of logarithmic values of these errors square, i.e. to minimize: ijahp article: el hefnawy, soliman mohammed/review of different method for deriving weights in the analytic hierarchy process international journal of the analytic hierarchy process issn 1936-6744 103 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.226 2 )(lnln          i j j i ij w w as (16) they proved that the s function is strictly convex and it has a unique minimum at the point i w , where ij n n ji aw 1  . thus, weights are derived by calculating the normalized values of the geometric mean of the comparison values ij a . they stated that, compared with the evm, the geometric mean method is better and easier to calculate. gao et al. (2009) argued the (llsm) function is rather difficult to solve because the objective function s is nonlinear and usually non-convex, moreover, no unique solution exists, and the solutions are not easily computable. therefore, they proposed three methods using the following definition of the error: ijijij wwae  , nji ,...,1,  (17) the three methods take the following forms: new least squares method 2 1 1 )( ij n i n j ij wwamin    (18) minimax method ijij ji wwamaxmin   1,1 (19) absolute deviation method     n i n j ijij wwamin 1 1 (20) in addition, some researchers have proposed another least squares method which aims to minimize sum of squares of additive error ij e such that: ij j i ij e w w a  , nji ,...,1,  (21) i.e. to minimize: 2 )(          i j j i ij w w as (22) 3) method of linear programming chandran et al. (2005) developed a two-stage linear programming approach for generating priority weights. the proposed model aims to minimize the absolute logarithms values of the error term , where nji e ij ,...,1,, 1 ω ij  , and ij e as defined by equation 15. they used three transformed decision variables: ijijijii zandw   ),ω(ln),(ln ij . first-stage: min      1 1 1 n i n ij ijz (23) s.t. ijahp article: el hefnawy, soliman mohammed/review of different method for deriving weights in the analytic hierarchy process international journal of the analytic hierarchy process issn 1936-6744 104 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.226 jinjiaijijji  ;,...,1,),ln( (24) jinjiz jiij  ;,...,1,, jinjiz jiij  ;,...,1,, (25) 11  (26) 1;,...,1,,0  ijji anji (27) qsomeforaakallforaanji jqiqjkikji ,;,;,...,1,,0   (28) njiz ij ,...,1,,0  (29) nji iji ,...,1,,edunrestrict,  (30) by defining * z as the optimal value of the objective function in the first-stage; this objective function is, in some sense, a measure of the inconsistency, that is, the greater the value of the objective function, the more inconsistent is the matrix. in the case of perfect consistency, there is no error in the estimate and * z is equal to zero. therefore, they defined the (ci) within the linear programming framework as follows: )1( 2 *   nn z ci lp (31) lp ci is the average value of ijz for elements above the diagonal in the comparison matrix. they showed that lp ci and saaty's ci are highly correlated. second stage due to the possibility of multiple optimal solutions to the first-stage model, another linear programming model has been solved to select from this set of alternatives optima the priority vector that minimizes the maximum of errors ijω as follows: min max z subject to the same constraints of the first stage model in addition to the following constraints: * 1 1 1 zz n i n ij ij      (32) jinjizz ij  ;,...,1,, max (33) njiz ij ,...,1,,0  (34) njiz ,...,1,,0 max  (35) wang and chin (2011) proposed an approximate solution to the evm using linear programming approach. the proposed model aims to maximize the principal eignvalue of the pcm ( max  ), or equivalently to minimize max 1 1   n i io . they defined the assurance region (ar)  such that:                           n j jij i i n j jij i i ca c ra r 11 1 min, 1 maxmin (36) where and are the row sums and column sums of the comparison matrix a respectively. then, they proved that: ijahp article: el hefnawy, soliman mohammed/review of different method for deriving weights in the analytic hierarchy process international journal of the analytic hierarchy process issn 1936-6744 105 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.226 nj n w o w j j j ,...,2,1,   (37) it is evident that maximizing the eigenvalue max is equivalent to minimizing the sum of ),...,1(, njz j  . so, the linear approximate model to the em can finally be formulated as follows: min   n i io 1 (38) s.t niwoa i n j jij ,...,2,1, 1   (39) niw n o ii ,...,2,1,0 1  (40) niwo ii ,...,2,1,0 1    (41) 1 1   n i iw (42) oow ii , (43) by using a numerical example, they showed that the proposed model turns out to be as good as the evm in producing extremely close priorities to eigenvector weights, but it is easier to solve and more convenient to use. 4) method of goal programming bryson (1995) introduced a goal programming method (gpm) for generating a priority vector and the corresponding measure of consistency. at first, he defined the real numbers 1&1   ijij  , such that )(*)(    ij ij j i ij w w a   where ijij   & cannot both be greater than 1. then, the following relations are defined:                             ij j i ij ij j i ij ij j i ijij a w w a w w a w w )(1 )(1 )(1    therefore, a consistency can be found in case of 1  ijij  . the objective in this model is to minimize the product  i j ijij , which can be translated to the following linear goal programming problem: )log(logminlog ijij i ij     (44) s.t. njiaww ijijijji   1,logloglogloglog  (45) 0log,log   ijij  (46) ijahp article: el hefnawy, soliman mohammed/review of different method for deriving weights in the analytic hierarchy process international journal of the analytic hierarchy process issn 1936-6744 106 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.226 the solution of this problem results in the un-normalized vector ),...,,( 21 nwwww which can be further normalized to give the vector w. lin (2006) proved that, for the bryson's gpm, when the minimal objective value is reached, a maximum of ( ) equations exist, including the objective function, for determining the values of variables. consequently, alternative optimal solutions are likely to exist, and the solution that is obtained is not necessarily the most consistent. he argued that both the gpm and the llsm have their individual drawbacks. on one hand, the gpm performs better than the llsm when outliers exist, but suffers from the problem of alternative optimal solutions. on the other hand, the llsm gives a unique solution but is sensitive to outliers. to overcome these drawbacks, he proposed an enhanced goal programming method (egpm) which combines the principle of the gpm and the (llsm) as follows:  logmin (47) s.t njiaww ijijijji   1logloglogloglog  (48) )log(loglog ijij i ij     (49) ))(log)((log 22    ijij i ij  (50) where denotes a sufficiently small positive number. according to lin, this method gives a unique solution and is not sensitive to outlier. grzybowski (2010) proposed another goal programming approach to minimize the distance to the given pcm with respect to given criterion functions measuring the distance as follows: min     n i ii dd n gpci 1 )( 1 (51) s.t. ij n j ijii nwwadd    1 (52) 1 1   n j j w (53) 0,,   iii ddw (54) in a consistent case the goal programming consistency index (gpci) is always equal to 0. for inconsistent pcms the index takes positive values. 5) method data envelopment analysis some researchers have used the data envelopment analysis (dea) approach to derive priority weights in ahp. for example, ramanathan (2006) used a dea approach to derive priority weights for consistent matrices. his model, which is briefly called data envelopment ahp (deahp), takes the following form: j j j vawmax  00 (55) s.t 1 1 u (56) niuva j j ij ,...,2,1,0 1  (57) njvu j ,...,2,1,0, 1  (58) ijahp article: el hefnawy, soliman mohammed/review of different method for deriving weights in the analytic hierarchy process international journal of the analytic hierarchy process issn 1936-6744 107 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.226 where 0 w refers to the priority weight of the criterion or alternative under evaluation. this model is solved for each criterion and alternative to obtain the priority weights vector. wang et al. (2008) stated that ramanathan's deahp model suffers from some drawbacks. on one hand, it may produce counterintuitive local weights for inconsistent comparison matrices. on the other hand, it may be overly insensitive to some comparisons in a comparison matrix. therefore, they developed a new dea model with assurance region (dea\ar) as follows: 0 wmax (59) s.t. nivaw j n j iji ,...,2,1,1 1   (60) njnwvw jjj ,...,2,1,//  (61) njvw jj ,...,2,1,0,  (62) where subscript zero refers to the decision criterion or alternative under evaluation and  is the assurance region defined in equation 36. they proved that the dea\ar model can produce true weights for perfectly consistent comparison matrices. for inconsistent ones, and due to the role of assurance region, it is able to produce rational, logical, and intuitive weights consistent with the decision makers' subjective judgments. in addition, wang and chin (2009) and mirhedayatian and saen (2011) also developed different dea models to overcome the drawbacks of deahp for deriving priorities from the pairwise comparison matrices. 6) method of interval priority estimation sugihara et al. (2004) suggested a method for estimating priority weights in intervals rather than crisp values. the estimated interval weights are denoted as   iii www , where i w and iw are the lower and upper bounds of the interval weight iw . the estimated interval matrix can be defined as follows: )(,,, jiji w w w w w j i j i ij           (63) assuming that  i w , where  is a very small positive number. to estimate the interval weights, they used a linear programming model called "possibilistic ahp for crisp data (pahpc)", the model takes the following form: iwiw min , )( i i i wwj   (64) s.t. ijij wwajiji  )(, (65) ijij wwajiji  )(, (66) 1  j ji i wwj (67) ijahp article: el hefnawy, soliman mohammed/review of different method for deriving weights in the analytic hierarchy process international journal of the analytic hierarchy process issn 1936-6744 108 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.226 1  j ji i wwj (68) 1 j i wwi (69)  i wi (70) where is a set },...,2,1{ n . they proved that the pahpc gives an optimal solution (interval vector), and that the consistency between the given matrix and the model can be represented as the value of the objective function , that is if ( ) it can be said that the given matrix is perfectly consistent. on the other hand, the estimated interval weights   iii www , is preferred over   jjj www , if and only if ji ww  and ji ww  . entani and tanaka (2007) adopted the idea of interval weights not only to obtain the local priority intervals (weights for each matrix) but also the global priority intervals (overall priority weights). if the local weight of alternative under criterion is denoted as ],[ kikiki www  and the referenced priority weight of criterion is denoted as ],[ kkki ppp  , then the global weight of alternative is obtained as ki k ki wpw  by interval arithmetic. 7) other deterministic methods grass and rapcsak (2004) used the singular value decomposition (svd), wang et al., (2007) proposed the method of correlation coefficient maximization approach (ccma) by maximizing the correlation coefficients between the priorities themselves and each column of the comparison matrix. also, podinovski (2007) proposed an approach to obtain a unique point estimates for the priority weights from interval judgments. the proposed approach transforms the problem to a multi-criterial minimization problem with equally important criteria, and can be solved using the method of symmetrical-lexicographic problem (sl-problem). 3.2 stochastic methods the stochastic methods assume that the evaluation of the priority ratio may come from a range of possibilities, and the judge gives his most likely or best estimate of the required evaluation within a range of possibilities. a number of methods tried to employ probability distributions to describe the pairwise comparison of the alternatives. for example, in their earlier work, lipovetsky and tishler (1997) analyzed the ahp for five types of random variables distributions including: triangle, beta, normal, laplace, and cauchy. they showed that the probability density function (p.d.f) of ij a 1 is not the same as the one of ij a when the distribution of ij a is normal, triangle, laplace, or beta. however, if the density function of ij a is cauchy, the density function of ij a 1 is also cauchy. then, the p.d.f. of these variables is used to estimate the elements of the comparison matrix which can be further used to derive the priorities weights using any of the deterministic methods. on the other hand, several articles in the literature investigated the problem of extracting preference weights from interval judgments rather than point ones. for example, haines (1998) argued that the decision maker may be uncertain about his preferences and therefore reluctant to assign single point scores to the pairwise ijahp article: el hefnawy, soliman mohammed/review of different method for deriving weights in the analytic hierarchy process international journal of the analytic hierarchy process issn 1936-6744 109 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.226 comparisons. instead, the decision maker may assign an interval with upper and lower bounds in an interval judgment matrix ( ],[ ijijnn ul  i ). such a matrix is reciprocal in the sense that: ij ji u l 1  and ij ji l u 1  . the interval pairwise comparison matrix is commonly assembled in a matrix of the form: i =                       1) 1 , 1 () 1 , 1 ( ),.(),(1) 1 , 1 ( ),(),(),(1 ........ ........ ........ 2211 2\22323 1212 1113131212 nnnn nn nn lulu ulul lu ululul  the set of priority weights on the alternatives are defined by:          n i iiij i i ijn niwwu w w lwwws 1 21 ,...,2,1,1,0,:),...,,( (71) haines (1998) argued that the feasible region, s, is a polytope and any weight vector, w, in that region can therefore be expressed as a convex combination of its extreme vertices in the following way: 1. generate (k – 1) points randomly on the interval, [0, 1], i.e. generate a random partition of [0, l] comprising l subintervals with lengths l ddd ,...,, 21 , where lid i ,....,1,0  and    l i i d 1 1. 2. form the random weight vector   l i ii vdw 1 , where the vectors, l vvv ,...,, 21 , denote the extreme vertices of s. various other simulation approaches have been reported for the stochastic interval ahp. for example, an early and much cited work of arbel and vargas (1993) applied a uniform distribution to the decision makers selected intervals. banuelas and antony (2004) used gamma and triangular distributions, and zhang et al. (2003) adopted both normal and uniform distributions. moreover, laininen and hamalainen (2003) used the logarithmic linear regression approach. the model aims to minimize the sum of square of the multiplicative error term ij e defined by equation 15. by taking the logarithm for two sides of equation 15: ijjiij ewwa lnlnlnln  (72) or equivalently, njnijiy ijjiij ,...,1,1,...,1,,   (73) in matrix notation, the proposed linear model takes the following form: εxβy  (74) the vector of error term ε is assumed to be normally distributed with expectation 0)( e and with constant variance ivar 2 )(   , y is the 1)2/)1(( nn vector ijahp article: el hefnawy, soliman mohammed/review of different method for deriving weights in the analytic hierarchy process international journal of the analytic hierarchy process issn 1936-6744 110 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.226 of dependent variable, β is the )1( n vector of parameters, and x is nnn  )1( matrix of the dummy variables taking values (-1, 0,1) according to the values of i and j. to avoid problems of identification, the n th alternative is established as the benchmark alternative )10(  nn w . the priority vector niw i ,...,1,  is obtained by the relationship: niw n i i i i ,...,1, )exp( )exp( 1      (75) in addition to the above, various other approaches have been reported for the stochastic interval ahp. for example, arbel and vargas (1993) and hauser and tadikamalla (1996) applied a uniform distribution to the selected intervals; zhang et al. (2003) adopted both normal and uniform distributions; banuelas and antony (2004) used gamma and triangular distributions; while jalao, et al. (2014) used beta distribution to model the linguistic imprecise pairwise comparisons, then a nonlinear programming model has been used to calculate priority weights from the beta stochastic pairwise comparisons. 4. deriving priority weights in group decision making problems in group decision making (gdm) problems, a number of decision makers/experts express their preferences about the alternatives, and then these preferences are incorporated to obtain a common conclusion about the alternatives to support the final decision. gdm has received considerable interest in recent years, as many decision-making processes in the real world take place in group situations. moreover, it is argued that using the opinions of several decision makers/experts when solving a decision making problem may produce better and more effective solutions to complex problems. the increasing complexity of the current socio-economic environment makes it less possible for a single decision maker/expert to consider all relevant aspects of a problem (vanícek et al., 2009). reviewing the literature in gdm revealed that there are four different approaches to incorporate preferences of the multiple decision makers/experts. three of these approaches deal with the judgments of individuals while the other one deals with the priorities derived from the judgments. these four approaches include: consensus between the actors where group members give a single numeric value for each pair of compared elements resulting in a "consensus" matrix. compromise or voting when consensus cannot be reached. aggregation of individual judgments (aij) which refers to the aggregation of the elements of comparison matrices of the different policy makers/experts to provide a new pairwise comparison matrix for the group. aggregation of individual priorities (aip) which refers to the aggregation of the priority weights derived separately. in consensus and voting options, group members agree upon a value for each pairwise comparison. however, this requires the presence of all group members and a considerable amount of discussions. in addition, these methods are usually strenuous and time consuming, especially if the hierarchy is large with many pairwise comparison matrices. therefore, it is most appropriate to use aggregating approaches in group decision making problems. ijahp article: el hefnawy, soliman mohammed/review of different method for deriving weights in the analytic hierarchy process international journal of the analytic hierarchy process issn 1936-6744 111 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.226 the most widely used methods for aggregation process are the geometric mean method (gmm) and the arithmetic mean method (amm). to show, simply, how these approaches can be used in the aggregation process for both aij and aip, let mknjia ijk ,...,1;,...,1,,  represents the pairwise judgments in comparing alternative i with alternative j expressed by the k th member in the group decision making process, and ik w represents the ith priority weight for the kth member in the group. then, the composite judgment g ij a and the composite priority weight g i w for that group are:  for aij: using the (gmm): ) 1 ( 1 mm k ijk g aa ij          using the (amm): m a a m k ijk g ij         1  for aip: using the (gmm): ) 1 ( 1 mm k ik g ww i          using the (amm): m w w m k ik g i         1 but which of these methods is better? the answer to this question has been a subject of debate between researchers. some have argued that the gmm is appropriate for combining judgments because it preserves the reciprocal property of the comparison matrices; it is showed by aczel and saaty (1983) that the geometric mean is the only mathematically valid way to synthesize reciprocal judgments preserving the reciprocal condition. in addition, when equal importance is given to all the policy makers, gmm is the proper way of synthesizing judgments. escobar and morenojime´nez (2007) argued that using gmm for aggregation of judgments preserves the acceptable inconsistency level, that is, if the individual decision makers have an acceptable inconsistency, then so does the group. on the other hand, ramanathan and ganesh (1994) argued that using gmm for the aggregation of judgments does not satisfy the pareto optimality axiom which means that if all individuals prefer a to b, then so should the group. the group may be homogeneous in some paired comparisons and heterogeneous in others, thus violating pareto optimality. however, van den honert and lootsma (1996) argued that this violation of the pareto optimality is not related to any deficiency in gmm. therefore, and based on the above discussions, gmm is appropriate for both (aij) and (aip). also in gdm situations, when we are dealing with a group of decision makers or experts, it should be noted that their opinions are not necessarily equally important. some of them have higher formal authority, better academic training, better socioeconomic background, are greater experts in the considered field, or have greater influence. such decision makers or experts can have a higher weight, which means that their judgments or priorities will have a larger influence on the group judgments or priorities. determining the relative importance of the group’s members is remarkable when considered in gdm problems. ijahp article: el hefnawy, soliman mohammed/review of different method for deriving weights in the analytic hierarchy process international journal of the analytic hierarchy process issn 1936-6744 112 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.226 in addition to the aggregation process of the individuals' preferences (judgments or priorities) to obtain the group priority weights, several researchers attempted to develop methods to be applied directly to derive priority weights for group decision making situations. some of these methods are generalizations of those which are used in the single case. also, these methods can be divided into deterministic and stochastic methods: 4.1 deterministic methods the lgp model is one of the deterministic methods proposed by bryson and joseph (1999) to derive priority weights in gdm ahp. lgp aims to minimize the product: k ij k ij m k n i n j qp   1 1 1 , where 1&1  ij k ij k qp are real numbers such that )(*)( k ij k ij j i ij q p w w a  , and k ij k ij qp & cannot both be greater than 1, nji ,...,1,  , and mk ,...,1 (where k is the number of decision makers/experts). the (lgp) model takes the following form:   k k m min log) 1 (log (76) s.t. mknjiaqpww ijk k ij k ijji ,...,1;,...,1,logloglogloglog  (77) mkqp s i j kk ij k ij ,...1,0log)loglog) 1 (   (78) 0log,log   ijij  (79) where )1(  nns , and all variables are nonnegative. the solution of this model gives the un-normalized vector ),...,,( 11 n wwww which can be normalized to give the normalized consensus priority point vector ),...,,( 11 n vvvv . entani and inuiguchi (2010) proposed three approaches based on the concept of interval regression analysis to derive the group interval priority weights ],[ iii www  using the aggregation of the individual priority intervals. first, the interval priority weights ],[ ikikik www  of alternatives are obtained from the individually given pairwise comparison matrices. then, these individual interval priority weights are aggregated in order to reach the group priority weights. the three methods are: 1. least upper approximation model 2. greatest lower approximation model 3. least squares model on the other hand, bolloju (2001) argued that the application of the ahp to model the subjective preferences of individuals in large groups requires homogeneity of the group. aggregating decision maker’s preferences into a single model may not represent any single decision maker nor satisfy the majority of the decision makers. therefore, he proposed a method of aggregating individual ahp models in homogeneous subgroups based on similarities in decision makers' preferences. this approach consists of four steps: ijahp article: el hefnawy, soliman mohammed/review of different method for deriving weights in the analytic hierarchy process international journal of the analytic hierarchy process issn 1936-6744 113 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.226 first, individual utility functions are derived from the ahp models that represent preferences of individual decision makers. a utility function, uf i , corresponding to the ahp model of the k th member of a group of m members can be defined as: k j n i k j k fwuf k    1 , where k j w is the weight associated with the factor (criterion) k j f for that member, and k n is the number of factors used by that member. second, the differences among the individual utility functions are resolved and unified utility functions are produced through content analysis. third, the unified utility functions are divided into subgroups based on similarities in preferences. several techniques such as cluster analysis, data envelopment analysis, and discriminate analysis can be used for identifying subgroups or clusters based on similarities in the unified decision models. fourth, the utility functions in each subgroup are aggregated to represent subgroup's aggregated utility functions using arithmetic mean. on the other hand, and similar to the single decision making case, dea has been used in the group decision making problems. for example, wang and chin (2009) extended their dea method for deriving the priorities in the single case, for use with group decision problems (the method is briefly called deaw&c) as follows: j n j m k k jk xawmax            1 1 )( 00  (80) s.t. 1 1 1 1 )(            j n j m k n i k ijk xa (81) ninxxa ij n j m k k ijk ,...,1, 1 1 )(            (82) njx j ,...,2,1,0  (83) where m is the number of decision makers, k is the weight assigned to the k th decision maker. hosseinian et al. (2009) proposed another dea model called data envelopment analysis for weight derivation in group decision "dea-wdgd". they argued that this model enables simpler derivation of weights than does the deaw&c model. the dea-wdgd takes the following form:   n i i wmax 1 (84) s.t. niwva ik m k n j k ij ,...,1,0 1 1 )(         (85)   n i i wmax 1 (86) 1,...,1,0 11   mkvv kkkk  (87) mknivw ki ,...,1;,...,2,1,0,  (88) ijahp article: el hefnawy, soliman mohammed/review of different method for deriving weights in the analytic hierarchy process international journal of the analytic hierarchy process issn 1936-6744 114 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.226 grošelj et al. (2011) proved that the dea-wdgd model violates the reciprocal property which is required in the ahp model. therefore, he proposed a new dea model using the weighted geometric mean instead of the weighted arithmetic mean in the deaw&c. the new model is called weighted geometric mean dea (wgmdea) and takes the following form:    j n j m k kk j xawmax    1 1 )( 00  (89) s.t.   1 1 1 1 )(            j n j n i km k k ij xa  (90)   ninxxa ij n j km k k ij ,...,1, 1 1 )(            (91) njx j ,...,2,1,0  (92) angiz et. al. (2012) proposed another dea model which manipulates both the priority weights and ranking aspect of each decision maker. in addition, the nonlinear programming approach has been used in the gdm. for example, chou et al (2007) proposed a nonlinear programming model to minimize the sum square of distances between the group weights and single weights as follows:       n j k r kcr cwwuwujmin 1 2 , ),( (93) s.t. 1 1   m i i u , 1 1   n j j w (94) where ku is the aggregation weight and ],1( r , 0c are two constants to control the aggregation process. they presented an iterative algorithm to derive the aggregated priority weights and aggregates pairwise comparison matrices into a consensus matrix. 4.2 stochastic methods some researchers proposed stochastic and statistical approaches for the gdm problems; for example, ramanathan (1997) proposed a method of stochastic goal programming to derive the maximum likelihood point estimates for priority weights for gdm. according to ramanathan, the judgments can be interpreted as stochastic when more than one value for the same judgment is considered.; this is the situation of group decision making. he argued that the judgment is captured on a semantic scale and is converted into a numerical integer value ij using a logarithmic scale. then the numerical estimate of the preference ratio j i w w is defined as ija ij j i ea w w   , where , is a scale parameter. the model takes the following form ijahp article: el hefnawy, soliman mohammed/review of different method for deriving weights in the analytic hierarchy process international journal of the analytic hierarchy process issn 1936-6744 115 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.226           n i n ij ij ijd min 1 22 2  (95) s.t ijniad ijijji  ;,...,1,* (96) nii ,...,2,1,0  (97) where ij d are the deviational variables, and 2 ij  is the variance associated with ij  . basak (2011) used a statistical model to derive composite priority weights of a number of decision makers. first, he defined the following relation: mkni a t n jij ijk ik ,...,1;,...,1, 1 1 1 ,1      (98) where ijka denotes the pcm entry denoting the intensity of preference of alternative ai to alternative aj (or of criterion ci to criterion cj) by the k th decision maker. he argued that ikt can be considered as an estimate for i in the following stochastic model: ikiikt   (99) where niww n i iii ,...,1),/( 1     , and ik are random variables distributed as multivariate normal ),0( σn ; then, ikt are then distributed as multivariate normal ),( σπn , where ),...,,( 11 n  . let kt denotes the vector of random variables ),...,,( 21 nkkk ttt for the k th individual in a given group. the model parameters  and σ can be estimated by maximum likelihood method and are given by: ni t t n i im im i ,...,1,ˆ 1     (100) ),...,( 1 1 1 nmm m k km ttt m t    (101) )()( 1 1ˆ 1 mkm m k k tttt m      σ (102) however, basak did not give any logical explanation about the relation in equation 98. also, he did not prove his arguments about the relationship between ikt and i which creates some vagueness with regards to his model. another method proposed by altuzarra et al. (2007), who proposed a new prioritization procedure based on bayesian approach, is as follows: ijahp article: el hefnawy, soliman mohammed/review of different method for deriving weights in the analytic hierarchy process international journal of the analytic hierarchy process issn 1936-6744 116 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.226 for m decision makers, let },...,2,1),({ )( mkaa ijk k  be the reciprocal judgment matrices for group decision makers, 0),,....,,( 21  i g n gg wwww be the group’s un-normalized priorities for the alternatives, and 0),,....,,( 21  i g n gg vvvv be their normalized distribution values:    n j g j g i i g w w v 1 . the proposed approach adopts a multiplicative model with log-normal errors, this model is given by: mknjie w w a ijkg j g i ijk ,...,1;,...,1,,  , (103) with k ij e distributed as ),0( 2)( k ln  . by eliminating reciprocal judgments ),( jia ij  and taking logarithms, a regression model with normal errors is obtained: mknijniy ijk g j g iijk ,...,1;,...,1;,...,1,   , (104) where )log(),log(),log( ijkijk g i g iijkijk eandway   . to avoid problems of identification, the n th alternative is established as the benchmark alternative )10(  nn v . in matrix notation, this model is expressed as: )()( kgk εxβy  (105) they proved that, by taking a constant non-informative distribution as the prior distribution for the vector of log-priorities g β , the posterior distribution of g β for complete and precise information is given by: yβ | g follows )ˆ,ˆ(1 bbnn   , where:     m k k k m k k b 1 )( )( 1 )( ˆ ˆ   β , 1 1 1 )( )(ˆ            xxς m k k b  (106)                       nnn nnn nnn xx /2/1/1 /1./2/1 /1/1/2 )( ........ ........ ........ 1  , and 2)( )( 1 k k    (107) ijahp article: el hefnawy, soliman mohammed/review of different method for deriving weights in the analytic hierarchy process international journal of the analytic hierarchy process issn 1936-6744 117 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.226 for the aggregation of individual judgments (aij), a matrix of aggregated judgments )( g ij g aa  is constructed using the geometric mean. then, the model becomes:    m k ijk g ij g ij y m ay 1 1 )log( (108) ij g j g iijk m k g j g i    1 (109) in this case, the maximum likelihood estimator of the vector of the group’s log priorities, g β , is obtained by the geometric mean (of judgments) estimator given by: 1,...,1, 1ˆ 1 ,    niy n yandyy g j g ij g i g n g iiaij (110) for the aggregation of individual priorities, they used the geometric mean as following:    m k k aip m 1 )(ˆ 1 ˆ ww (111) or;    r k k aip m 1 )(ˆ1ˆ β (112) they proved that the proposed bayesian estimator for complete and precise matrices with known variance )ˆ( b β is more efficient than the usual aij β̂ and aip β̂ applied to aggregate judgments and priorities. they also extended the bayesian approach in case the information provided by any of the decision makers is incomplete, which means that not all of the decision makers express the n(n-1)/2 possible judgments in the reciprocal pairwise comparison. also, gargallo et al. (2007) used the bayesian hierarchical model based on mixtures of normal distributions to describe the distribution of individual priorities and for the identification of existing groups of the decision makers’ preferences. in spite of its advantages as a powerful tool for parameter estimation problems and inferences, using bayesian approach for deriving priority weights in group ahp has some disadvantages. on one hand, it works with a lot of assumptions especially for the prior distribution of the parameters; on the other hand, it often comes with a high computational cost, especially in models with a large number of parameters. researchers may find it difficult to understand and apply this approach for decision making problems with these limitations. 5. conclusion since its real appearance as a multi-criteria decision making technique in the 1980's, ahp has been greatly accepted among researchers as a simple and practical technique that can be applied to different fields for different objectives. in this paper, the main theoretical concepts of ahp were discussed including structuring of a decision problem, judgments, prioritization procedures, consistency indices, and synthesizing to obtain the final weights. the methodological developments of deriving weights in ahp, in both single and group decision making, are reviewed in detail. ijahp article: el hefnawy, soliman mohammed/review of different method for deriving weights in the analytic hierarchy process international journal of the analytic hierarchy process issn 1936-6744 118 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.226 previous literature reviews revealed that using evm for deriving weights in ahp, as proposed by saaty, has some shortcomings. therefore, several attempts have been made to propose new methods using different techniques to be used as alternatives to evm. this paper reviewd several proposed methods, and concluded, first, many of these methods are complicated and not easy to be applied compared to evm, second, the proposed methods did not provide real solutions for the claimed drawbacks of the evm, particularly, the rank reversal problem which is still not fully resolved, and third, there is no measure for the superiority of these methods to tell determine which technique obtains the best results, especially for those who are not specialized. in addition, most of the applicants of ahp are unaware of its successive developments (ishizaka and labib, 2011). these conclusions may explain the limited use of those suggested methods to derive weights in ahp compared to evm. ijahp article: el hefnawy, soliman mohammed/review of different method for deriving weights in the analytic hierarchy process international journal of the analytic hierarchy process issn 1936-6744 119 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.226 references aczel, j. and t.l. saaty (1983). procedures for synthesizing ratio judgments. journal of mathematical psychology, 27, 93–102. aguaron, j. and j. m. t. moreno-jimenez (2003). the geometric consistency index: approximated thresholds. in european journal of operational research, 147, 137−145. alonso, j. a. and m. t. lamata (2006). consistency in the analytic hierarchy process: a new approach. international journal of uncertainty, fuzziness and knowledge-based systems, 14 (4), 445−459. altuzarra, a., j. maría, morenojiménez, and m. salvador (2007). a bayesian prioritization procedure for ahp-group decision making. european journal of operational research, 182, 367–382. angiz, l. m. z., a. mustafa, n. a. ghani, and a. a. kamil (2012). group decision via usage of analytic hierarchy process and preference aggregation method. sainsmalaysiana, 41(3), 361–366. arbel, a. and l., g. vargas (1993). preference simulation and preference programming: robustness issues in priority derivation. european journal of operational research, 69, 200-209. banuelas, r. and j. antony (2004). modified analytic hierarchy process to incorporate uncertainty and managerial aspects. international journal of production research, 42, 3851–3872. basak, i. (2011). an alternate method of deriving priorities and related inferences for group decision making in analytic hierarchy process. journal of multi-criteria decision analysis, 18(5-6), 279-287. belton, v. and t. gear (1983). on a shortcoming of saaty's method of analytic hierarchies. omega, 11 (3), 228-230. bolloju, n. (2001). aggregation of analytic hierarchy process models based on similarities in decision makers preferences. european journal of operations research, (128), 499-508. borodin, d., v. gorelik, w. d. bruyn, b. v. vreckem (2011). two methods for consistency-driven approximation of a pairwise comparison matrix. proceedings of the international symposium on the analytic hierarchy process 2011. bryson, n. (1995). a goal programming method for generating priority vectors. the journal of the operational research society, 46 (5), 641-648. bryson, n. and a. joseph (1999). generating consensus priority point vectors: a logarithmic goal programming approach. computers & operations research, 26, 637-643. chandran, b., b. golden and e. wasil (2005). linear programming models for estimating weights in the analytic hierarchy process. computers & operations research, 32, 2235–2254. chou, m., h. lee, c. chu and c. cheng (2007). synthesizing comparison matrices of ahp under group decision. knowledge-based intelligent information and engineering systems, 1323–1330. http://onlinelibrary.wiley.com/journal/10.1002/(issn)1099-1360 http://onlinelibrary.wiley.com/journal/10.1002/(issn)1099-1360 ijahp article: el hefnawy, soliman mohammed/review of different method for deriving weights in the analytic hierarchy process international journal of the analytic hierarchy process issn 1936-6744 120 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.226 chu, a. t. w., r. e. kalaba and k. npingarn (1979). a comparison of two methods for determining the weights of belonging to fuzzy sets. journal of optimization theory and application, 27((4), 531-538. cogger, k. o. and p. l. yu (1985). eigenweightvectors and least-distance approximation for revealed preference in pairwise weight ratios. journal of optimization theory and applications, 46 (4), 483-491. costa, c.a.b. and j. vansnick (2008). a critical analysis of the eigenvalue method used to derive priorities in ahp. european journal of operations research, 187, 1422–1428. crawford, g. and c. williams (1985). a note on the analysis of subjective judgment matrices. journal of mathematical psychology, 29(4), 387–405. entani, t. and h. tanaka (2007). interval estimations of global weights in ahp by upper approximation. fuzzy sets and systems, 158, 1913 – 1921. entani, t. and m. inuiguchi (2010). group decisions in interval ahp based on interval regression analysis. integrated uncertainty management and applications, 68, 269-280. ergu, d., g. kou, y. peng and y. she (2011). a simple method to improve the consistency ratio of the pairwise comparison matrix in anp. european journal of operations research, 213(1), 246-259. escobar, m. t. and j. m. moreno-jime´nez (2007). aggregation of individual preference structures in ahp-group decision making. group decision and negotiation, 16, 287-301. fedrizzi, m. and s. giove, (2007). incomplete pairwise comparison and consistency optimization. european journal of operational research, 183(1), 303–313. forman, e. h. and s. i. gass, (2001). the analytic hierarchy process: an exposition. operations research, 49(4), 469-486. gao, s., z. zhang, and c. cao (2009). new methods of estimating weights in ahp. international symposium on information processing, 201-204. gargallo, p., j. m. moreno-jime´nez and m. salvador (2007). ahp-group decision making: a bayesian approach based on mixtures for group pattern identification. group decision and negotiation, 16, 485–506. gomez-ruiz, j. a., m. karanik, and j. i. peláez (2010). estimation of missing judgments in ahp pairwise matrices using a neural network-based model. applied mathematics and computation, 216, 2959–2975. grass, s. i. and t. rapcsak (2004). singular value decomposition in ahp. european journal of operational research, 154, 573-584. grošelj, p., š. pezdevšek, and p. malovrh (2011). methods based on data envelopment analysis for deriving group priorities in analytic hierarchy process. central european journal of operations research, 19(3), 267-284. grzybowski a.z. (2010). goal programming approach for deriving priority vectors some new ideas. scientific research of the institute of mathematics and computer science, 1(9), 17-27. haines, l. m. (1998). a statistical approach to the analytic hierarchy process with interval judgments. (i). distributions on feasible regions. european journal of operational research, 110, 112-125. ijahp article: el hefnawy, soliman mohammed/review of different method for deriving weights in the analytic hierarchy process international journal of the analytic hierarchy process issn 1936-6744 121 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.226 harker, p.t. (1987a). alternative modes of questioning in the analytic hierarchy process. mathematical modeling, 9(3–5), 353–360. harker, p.t. (1987b) incomplete pairwise comparisons in the analytic hierarchy process. mathematical modeling, 9(11), 837–848. hauser, d. and p. tadikamalla (1996). the analytic hierarchy process in an uncertain environment: a simulation approach. european journal of operational research, 91, 27-37. ho, w. (2008). integrated analytic hierarchy process and its applicationsa literature review. european journal of operations research, 186, 211-228. hosseinian s.s., h. navidi and a. hajfathaliha (2009). a new method based on data envelopment analysis to derive weight vector in the group analytic hierarchy process. journal of applied sciences, 1(8), 3343-3349. ishizaka, a. and a. labib (2011). review of the main developments in the analytic hierarchy process. expert systems with applications, 38, 14336–14345. jalao, e. r., t. wu and d. shunk (2014). a stochastic ahp decision making methodology for imprecise preferences. information sciences, 270, 192–203. laininen, p. and r. p. hamalainen (2003). analyzing ahp-matrices by regression. european journal of operational research, 148, 514–524. lin, c. –c (2006). an enhanced goal programming method for generating priority vectors. the journal of the operational research society 57 (12), 1491-1496. lipovetsky, s. and a. tishler (1997). interval estimation of priorities in the ahp: methodology and application. israel institute of business research, working paper no. 16/97, faculty of management, tel aviv university. mirhedayatian, s. m. and r. f. saen (2011). a new approach for weight derivation using data envelopment analysis in the analytic hierarchy process. journal of the operational research society, 62, 585-1 595. mirhedayatian, s. m., m. jafarian, and r. f. saen (2011). a multi-objective slack based measure of efficiency model for weight derivation in the analytic hierarchy process. journal of the applied sciences, 11(19), 3338-3350. nishizawa, k. (2005). estimation of unknown comparisons in incomplete ahp and its compensation. report of the research institute of industrial technology, nihon university, no. 77. peláez, j. i. and m. t. lamata (2003). a new measure of consistency for positive reciprocal matrices. computers and mathematics with applications, 46, 1839-1845. podinovsky, v. v. (2007). interval articulation of superiority and precise elicitation of priorities. european journal of operational research, 180, 406-417. ramanathan, r. (1997). stochastic decision making using multiplicative ahp. european journal of operational research, 97, 543-549. ramanathan, r. (2006). data envelopment analysis for weight derivation and aggregation in the analytic hierarchy process. computers & operations research, 33, 1289–1307. ramanathan, r. and l.s., ganesh (1994). group preference aggregation methods employed in ahp: an evaluation and an intrinsic process for deriving members' weightages. european journal of operational research, 79, 249-265. ijahp article: el hefnawy, soliman mohammed/review of different method for deriving weights in the analytic hierarchy process international journal of the analytic hierarchy process issn 1936-6744 122 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.226 ramanujam, v. and t. l. saaty (1981). technological choice in the less developed countries: an analytic hierarchy approach. technological forecasting and social change, 19, 81–98. saaty, t. l. (1988). analytic hierarchy process: planning, priority setting, and resource allocation. pittsburgh, pa: university of pittsburgh press. saaty, t. l. (1990). how to make a decision: analytic hierarchy process. european journal of operations research (48), 9-26. saaty, t. l. (1994). how to make a decision: analytic hierarchy process. interfaces, 24, 19-43. saaty, t. l. (2001). the seven pillars of the analytic hierarchy process: models, methods, concepts & applications of the analytic hierarchy process. international series in operations research & management science, 34, 27-46. saaty, t. l. (2003). decision-making with the ahp: why is the principal eigenvector necessary. european journal of operations research, 145, 85-91. saaty, t. l. and m. s. ozdemir (2003). why the magic number seven plus or minus two. mathematical and computer modeling, 38, 233-244. saaty, t.l. and l.g. vargas (1985). the legitimacy of rank reversal. omega, 12(5), 513-516. saaty, t. l. and h. shih (2009). structures in decision making: on the subjective geometry of hierarchies and networks. european journal of operations research, 199, 867-872. shiraishi, s., t. obata, and m. daigo (1998). properties of a positive reciprocal matrix and their application to ahp. journal of operational research society of japan, 41, 404–414. stein, w. e., and mizzi p. j., (2007). the harmonic consistency index for the analytic hierarchy process. european journal of operations research, 177, 488497. subramanian, n. and r. ramanathan (2012). a review of applications of analytic hierarchy process in operations management. international journal of production economics, 138, 215–241. sugihara, k., h. ishii and h. tanaka (2004). interval priorities in ahp by interval regression analysis. european journal of operational research, 158, 745–754. vaidya, o. s., and kumar, s. (2006). analytic hierarchy process: an overview of applications. european journal of operations research, 169, 1-29. vanícek, j., i. vrana and s. ali (2009). fuzzy aggregation and averaging for group decision making: a generalization and survey. knowledge-based systems, 22, 79-84. van den honert, r. c. and f.a. lootsma (1996). group preference aggregation in the multiplicative ahp ± the model of the group decision process and pareto optimality. european journal of operational research, 96, 363-370. wang, y. and k. chin (2009). new data envelopment analysis method for priority determination and group decision making in the analytic hierarchy process. european journal of operational research (195), 239–250. wang, y. and k. chin (2011). a linear programming approximation to the eigenvector method in the analytic hierarchy process. information sciences, 181, 5240–5248. https://springerlink3.metapress.com/content/978-0-7923-7267-7/ https://springerlink3.metapress.com/content/978-0-7923-7267-7/ https://springerlink3.metapress.com/content/0884-8289/ https://springerlink3.metapress.com/content/0884-8289/ ijahp article: el hefnawy, soliman mohammed/review of different method for deriving weights in the analytic hierarchy process international journal of the analytic hierarchy process issn 1936-6744 123 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.226 wang, y. and t. m. s. elhag (2006). an approach to avoiding rank reversal in ahp. decision support systems, 42(3), 1474-1480. wang, y., c. parkan and y. luo (2007). priority estimation in the ahp through maximization of correlation coefficient. applied mathematical modeling, 31, 27112718. wang, y., k. chin and g. k. k. poon (2008). a data envelopment analysis method with assurance region for weight generation in the analytic hierarchy process. decision support systems, 45, 913–921. zanakis, s., t. mandakovic, s. gupta, s. sahay, and s. hong, (1995). a review of program evaluation and fund allocation within the service and government sectors. socio-economic planning sciences, 29, 59-79. zeshui, x. and w. cuiping (1999). a consistency improving method in the analytic hierarchy process. european journal of operational research, 116, 443-449. zhang, q., j. ma, z.p. fan, and w.c. chiang (2003). a statistical approach to multiple-attribute decision-making with interval numbers. international journal of systems science, 34, 683–692. ijahp article: maludin, syarief, rifin, rochman/reassembling technology transfer in indonesia international journal of the analytic hierarchy process 437 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.846 reassembling technology transfer in indonesia syafrizal maludin 1 syafrizal_maludin@yahoo.com rizal syarief rsyarief@yahoo.com amzul rifin amzul_rifin@yahoo.com nurual taufiqu rochman ufiq2000@yahoo.com abstract this article aims to provide a dynamic picture of the technology transfer process in public research institutions in indonesia that has been updated by establishing the national research and innovation agency in august 2021. this body is directly under the president of the republic of indonesia based on presidential regulation number 78 of 2021. during the research period, there was a change in the landscape of technology transfer actors. some of these changes are in line with the results of the analysis, namely increasing the role of gfri and research and development agencies under the ministry. this change is referred to as technology transfer reassembly. the arrangement of technology transfer leads to a new form. the research was initiated in october 2017 using the ahp to determine the best institutional arrangement for integrating research. the ahp results show the rank of institutional arrangements from highest to lowest as government funded research institutes (gfri) (0.27833), a research division under the ministerial office (0.24890), universities (0.17966), private r&d (0.13589) and foreign agencies (0.07214). government funded research institutes are the top choice of experts for having a significant role in the technology transfer process. the core function of gfri in the technology transfer system is to plan, conduct and develop technology and they have contributed significantly to the policy-making process by providing information and policy recommendations. this research enriched the application of the recommendations by establishing the national research and innovation agency as the most influential actor in building a national technology transfer system. 1 corresponding author acknowledgement: this paper has been developed by the first author in his doctoral dissertation funded by the ministry of research education and higher education of the republic of indonesia decree number 35/m/kp/ix/2014 mailto:syafrizal_maludin@yahoo.com mailto:rsyarief@yahoo.com mailto:amzul_rifin@yahoo.com mailto:ufiq2000@yahoo.com ijahp article: maludin, syarief, rifin, rochman/reassembling technology transfer in indonesia international journal of the analytic hierarchy process 438 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.846 keywords: analytic hierarchy process; government technology transfer; presidential regulation number 78 of 2021; public research institutes; science technology park; technology business incubator; national research and innovation agency (nria). 1. introduction technology innovation is increasingly expected to underpin economic growth (bercovitz & feldmann, 2006; blohmke, 2014; costantini & liberati, 2014), but expectations are far from being met. in addition, the high cost of research, the maintenance costs of patents, and the inability to create new technology-driven companies make technology transfer a necessity. the method of technology transfer occurs through the dynamics of technological developments in certain industrial sectors. technological developments, for example, are growing faster in the information technology, telecommunications and banking sectors than in the mineral and textile technology sectors. on the other hand, the planned changes are made by the most influential parties when it comes to technology transfer. this study focuses on technology transfer to government research institutions so that the reassembly that occurs is initiated by the government through incentives, regulations and reorganization of institutional arrangements. people expect research institutes to provide scientific solutions to a variety of issues, from food to energy to pharmacy and manufacturing. incentives and increased budgets had been set by the government to increase both the quality and quantity of research. the key objective in developing science and technology is strengthening national competitiveness (delgado et al., 2012; slaughter & rhoades, 1996). universities and government r&d institutions have produced many research results that can significantly reduce dependence on food imports, such as research of soybeans and wheat which can be successfully produced on a commercial scale. an application of this research called fortification, which was an effort to eliminate stunting in indonesia led by president joko widodo in 2018, can be achieved by the country; however, other considerations for wheat, soybeans, and enhancement materials are still provided by imports. the number of patents has been increased in line with increasing socialization and training to protect intellectual property through training and workshops on intellectual property (ip) in areas like patent drafting. these have been conducted by ip consultants, universities, research institutes, as well as the ministry of law and legislation of the republic of indonesia. ijahp article: maludin, syarief, rifin, rochman/reassembling technology transfer in indonesia international journal of the analytic hierarchy process 439 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.846 figure 1 patents of the indonesian institute of sciences 1991-2016 published in 2017 by pusat inovasi lipi (modified) it can be inferred that industrial development is still based on their principal invention to be applied to their product. researchers were encouraged to submit their research under patent protection and disseminate it to a potential private sector user. on the other side, in the macro view, the growing number of patents has not increased relative to the number of foreign patents. this could indicate that industrial development still relies on their principal patent from other countries while the objective of local research institutes is to register patents. researchers have been encouraged to submit their research protected by the patent regime and disseminate it to potential users from the private sector. it is worth understanding the difference between tools and objectives in this appropriability regime case, which is similar to the difference between justice and law (horvath & kelsen, 1957). justice should be the objective of the law. the law as the objective of justice is rather inept. therefore, a patent is a tool to protect achievement in the intellectual field. when the patent becomes the objective, it will lead to a maintenance fee that should be paid by the taxpayer. since the main objective of this appropriability regime is to protect the owner of the scholarly outcome and benefit the user, the number of patents achieved should not be the target. when the outcome of the research, as the single apparatus of one product, was protected, most potential users were reluctant to utilize it. the worst cases have occurred in a number of state universities and public research institutions that have paid the maintenance fees for unutilized patents. 1 1 6 0 2 1 1 5 3 6 16 17 11 12 15 18 21 21 24 38 43 34 29 43 53 81 0 10 20 30 40 50 60 70 80 90 1 9 9 1 1 9 9 2 1 9 9 3 1 9 9 4 1 9 9 5 1 9 9 6 1 9 9 7 1 9 9 8 1 9 9 9 2 0 0 0 2 0 0 1 2 0 0 2 2 0 0 3 2 0 0 4 2 0 0 5 2 0 0 6 2 0 0 7 2 0 0 8 2 0 0 9 2 0 1 0 2 0 1 1 2 0 1 2 2 0 1 3 2 0 1 4 2 0 1 5 2 0 1 6 paten ijahp article: maludin, syarief, rifin, rochman/reassembling technology transfer in indonesia international journal of the analytic hierarchy process 440 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.846 figure 2 local and foreign patents in indonesia 2015mid 2017 published in 2017 by direktorat jenderal hak kekayaan intelektual republik indonesia, 2 most international patents relate to products from multinational companies (mnc). however, opportunities for local inventors are still available if mnc are willing to collaborate with a local research institute. the most generic to the most advanced technology could be produced by a local inventor. in an interview with one of the mnc production managers about the innovation process, the manager discussed the possibility of utilizing a patent from a local inventor if it was supported with relevant documents that ensure the safety of the product. another important issue was the price of the license, which should be cheaper than the patent provided by a foreign inventor. the process of utilizing a patent from overseas is not as complicated as contacting the researcher to solve problems within the production process. the quickest way is to buy a cheap invention abroad, which leads to a larger gap between industry (technology-user) and public research (technology-provider). this research investigated the idea of policymakers linking research output to industry using the analytical hierarchy process (ahp). 2. literature review 2.1 technology transfer system at the public research institute modifications to the study of dissemination, transfer and technological innovation are adapted based on the orientation and scale of the research. this linear flow also grows with changes in the nature of the technology and its users. when discussing a technology transfer policy, there are two kinds of technology transfer. the first is called american centrist and refers to technological developments that are carried out primarily on the basis of private r&d that is influenced by the demand of users of technological products. on the other hand, mazzucato (2012) shows the significant role of the federal government's r&d in the development and transfer of technology in areas such as local area network applications and touch screen technology. however, the success of the use of technology in america is tied to the users of the business. the second kind is called 2 directorate general of intellectual property of the republic of indonesia. it is commonly called the indonesian patent office 1,302 1,471 482 7578 7765 2764 2 0 1 5 2 0 1 6 2 0 1 7 indonesia internasional ijahp article: maludin, syarief, rifin, rochman/reassembling technology transfer in indonesia international journal of the analytic hierarchy process 441 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.846 scandinavian centrist where the government plays the predominant role in the development of innovation and technology transfer. hassan and jamaluddin (2015) ) divide the technology transfer processes into ten categories around the world, as follows: 1. calantone model (calantone et al. 1990) this model is composed of 9 elements, namely: environmental considerations that impact all stages of the technology transfer process. technology transfer actors composed of technology providers, suppliers, ngos, and governments. structure, which is a component that describes the relationships, mechanisms and communication between the actors involved in the technology transfer process. the lines of communication are built through political, economic and business relationships. a process, which is an element that involves negotiations between technology suppliers and users impacted by communication channels consisting of technology selection, partners and technology transfer modes. a technology transfer function that involves the implementation, assessment and oversight of technology transfer projects. the background of the two actors of technology transfer that affect the success of the technology transfer process. political factors which include the political system, political structure and the strength of the relationship between the state and ngos. economic factors, namely the level of economic growth and stability that affect the speed of industrialization based on technology transfer. cultural factors that influence the successful implementation of the technology transfer program. 2. simkoko model (simkoko, 1992) an effective technology transfer mechanism is influenced by 7 elements of this model, namely technology transfer, technology users, technology, mechanisms, technology transfer, and technology users' environment, the surrounding environment. 3. kumar model (kumar et al., 1999) the study examined important elements for industry in order to increase its capacity to use developed country technology. the industrial sector in indonesia is the object of research which includes the actors involved in the process, transfer method, size, type, standard activity, the nature of ownership, training and capability of the r&d division (rd). three important factors in technology transfer, which are presented in this study, are dynamic investment, activity and learning capacities. ijahp article: maludin, syarief, rifin, rochman/reassembling technology transfer in indonesia international journal of the analytic hierarchy process 442 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.846 4. lin and berg model ( lin dan berg, 2001) this study highlights the important factors that influence the performance of the technology transfer process abroad in taiwan. the three actors of technology transfer are the technology transfer environment, the transferor and the beneficiary of the technology transfer (transferor/transferee) and the culture of the country of origin. 5. malik model (malik 2002) this model involves elements of knowledge, expertise, actors, processes, modes, barriers and opportunities for technology transfer. broadly speaking, the process of technology transfer is influenced by two factors, namely factors that strengthen and hinder technology transfer. 6. wang model (wang et al., 2004) this model represents the transfer of knowledge and technology from the leaders of multinational companies with their overseas branches. transferee and transferor characteristics affect the efficiency of technology transfer. additionally, a country's economic advantage also influences the characteristics of the technology transfer process. 7. steenhuis and bruijn model (steenhuis & buijn, 2005) this model study was carried out in the civilian aviation industry. this model calls for a balance of needs between the transferor and the transferee. 8. waroonkun and stewart model (waroonkun & stewart, 2008) the model is aimed at strengthening the technology transfer process in developing countries. in this model a business environment consisting of certain techniques, expertise, and capability requirements related to technology transfer needs to be developed to strengthen the technology transfer process in developing countries. the standard methodology is one of the requirements for the effectiveness of the technology transfer process. moreover, a country's maturity in understanding and applying technology transfer influences its success rate in the technology transfer process. 9. mohamed model (mohamed et al., 2012) there are four categories that affect successful results in technology transfer including support for technology transfer, environmental infrastructure, and learning capacity. 10. khabiri model (khabiri et al., 2012) an effective technology transfer mechanism is influenced by seven components of this model as follows: technology transfer, technology users, technology, mechanism, environment of technology transfer, technology users and universal environment. technology transfer under this research focuses on the transfer of technology and knowhow from technology providers to average users or end users. intermediary users play a significant role in technology transfer when research results will be useful when combined with other discoveries. an example of this is the invention of the touchscreen layer that cannot be used alone, but must be used in conjunction with other inventions to produce communication devices or other optical devices. therefore, the invention of an ijahp article: maludin, syarief, rifin, rochman/reassembling technology transfer in indonesia international journal of the analytic hierarchy process 443 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.846 actor technology supplier can be a raw material for comparison of other technology suppliers. 2.2 ahp concept in the field of technology transfer a google scholar search using “ahp” and “technology transfer” as the keywords led to 8 relevant articles. each article has a different research focus and locus. table 1 previous research # author article title 1 chehrehpak et al., 2012 select optimal methods for the technology transfer by using the analytic hierarchy process (ahp). 2 erensal & albayrak, 2007 transferring appropriate manufacturing technologies for developing countries. 3 gerdsri & kocaoglu, 2007 applying the analytic hierarchy process (ahp) to build a strategic framework for technology road mapping 4 kumar et al., 2015b benchmarking supply chains by analyzing technology transfer critical barriers using the ahp approach. 5 lee et al., 2013 using ahp to determine intangible priority factors for technology transfer adoption. 6 wang et al., 2009 changing technology transfer strategies in a non-profit organization an examination of itri. 7 yazdani et al., 2011 ranking of technology transfer barriers in developing countries; a case study of iran’s biotechnology industry. the selection of the articles is based on a similar research cluster. each article is in the cluster of technology transfer of various technologies. this proves that ahp is pertinent in this cluster. kumar et al. (2015) employed the ahp in their research to measure the critical factors in technology transfer, including policymakers in international organizations, government officials and environmentalists. this concept led the authors to consider that the decision-maker and manager should be the ones evaluating the appropriateness of technology transfer. this approach fits the needs of managers to anticipate critical factors to achieve efficiency of the technology transfer process. ijahp article: maludin, syarief, rifin, rochman/reassembling technology transfer in indonesia international journal of the analytic hierarchy process 444 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.846 the ahp was also utilized to develop a strategic framework of technology road mapping (gerdsri & kocaoglu, 2007). this roadmap was called ‘technology development envelope’, which translated to a technology transformation that flexibly and dynamically assisted organizations to effectively conduct technology transformation. the decisions that were made were valued by alternatives developed through change management. bozeman (2000) employed the contingent effectiveness model (cem) to measure the effectiveness of the technology transfer process. the objective of this method is similar to the ahp in that it effectively measures criteria in the technology transfer process that include the role of capacity development and the political system. in conjunction with an indonesian technology transfer system, the political system greatly influences the development of science and technology (st) and includes the theme of research and the position of high-ranking officials in the budget control over research activities. this used to be a two-way communication where scientists briefed parliament about trends as they relate to current issues. therefore, technology transfer from a public research institute to users was unfavorable. however, the promised budget increase for st since 2003 has never happened. budgets for st can be very flexible in contrast to budgets for education and health, where changes in budget allocation for research can fluctuate. however, the expectation of st's contribution to competitiveness is always ranked first, which makes rethinking technology transfer the key question of this research. another problem is the flow of the transfer from the technology provider to the user. experts agree on the egocentric nature of the public research body that has led to unproductive action it is uncommon to see the flow of technology-based-products that were invented in basic research institutes and developed by the applied research agency. therefore, establishing the nria would be better if it exists as a body under the ministry of research and technology before it is applied to a wider scale. however, the classic dilemma occurs when the decision-maker has to make a decision in a limited timeframe. one of the objectives of the nria is to centralize (not synergize) the actor that functions as the technology provider and has new terms such as litbangjirap 3 for research – development – assessment and application (rdaa). this policy aims to increase the role of technology to strengthen national competitiveness. it is also supported by establishing an endowment fund, extending the retirement age, decreasing taxes on research activities and establishing the national repository. 3. public research institutes as agents of technology transfer in indonesia taxonomy of technology transfer was developed to represent the agents of technology transfer in indonesia, which consist of type of technology and transfer, transaction method, classification and characteristics. technology transfer was also divided by long, short or medium range of time and source of finance (reisman, 2005). ideally, technology transfer does not include expenses for the transferor and transferee (zero cost 3 it stands for penelitian, pengembangan, penkajian and penerapan. ijahp article: maludin, syarief, rifin, rochman/reassembling technology transfer in indonesia international journal of the analytic hierarchy process 445 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.846 transaction). however, with changes in the market environment, the technology provider can earn, in the form of a license, while the transferee will earn profit from selling the product in the market. agents involved in technology transfer could be involved in either an external technology transfer and/or internal technology transfer. the modality characteristic determines the mode of transfer, effectiveness, expenses, price, and some agents. in 2021, the nria was established as the only national scale think tank through presidential regulation number 78 of 2021 4 . it was later updated to presidential regulation no 78 of 2021. the nria assists the president in carrying out research, development, assessment, and implementation, as well as nationally integrated inventions and innovations and monitoring, controlling, and evaluating the implementation of tasks. all government research and development institutions, including those under the technical ministry and local governments will be under the nria. the structure consists of 7 deputies, each of which oversees several directorates, namely: 1. deputy for development policy 2. deputy for research and innovation policy 3. deputy for human resources in science and technology 4. deputy for research and innovation infrastructure 5. deputy for research and innovation facilitation 6. deputy for research and innovation utilization 7. deputy for regional research and innovation technical activities of research, development, assessment, application as well as inventions and innovations are carried out by non-structural organizations called research organizations consisting of the following areas: 1. nuclear power 2. aviation & space 3. technology assessment & application 4. life sciences 5. earth science 6. engineering science 7. social sciences, humanities 8. marine & fisheries 9. archeology 10. metrology 11. religion & religion 12. language & literature 13. environment & forestry 14. governance & community welfare 4 the implementation of national research and innovation is regulated by presidential regulation no. 33 of 2021 which was issued on april 28, 2021, and revised by presidential regulation no. 78 of 2021 which was issued on august 24, 2021. ijahp article: maludin, syarief, rifin, rochman/reassembling technology transfer in indonesia international journal of the analytic hierarchy process 446 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.846 15. agriculture and livestock 16. health this research does not distinguish the structure and arrangement of research institutions in the two presidential regulations. in accordance with the limitations of the research, the technology transfer under study is the transfer of technology providers using government budgets. 4. methods this study employed an ahp approach. there were two phases of meetings for preparing questionnaires and interviews. the purpose of the first meeting was to determine the factors that should be included in the ahp. this was an important step to prevent the respondent from misinterpreting the questionnaire. the experts provided their advice in a weighted pair comparison between the factors of the ahp structure. this was followed by data confirmation and questionnaire validation. at our second meeting, we arrived with a questionnaire. while there was deep discussion throughout the process of the ahp survey, most experts agreed that the core problem of technology transfer is the absence of a national research institute. therefore, this was followed by establishing the national research and innovation agency of the republic of indonesia. in this extensive research, the discussion before establishing the agency is provided as it is among the recommendations. the variables presented in the questionnaire were made to facilitate understanding in the assessment of criteria so that the meaning of each of the variables in the questions were easy to understand. references related to variables are found in the research methodology section. figure 3 pairwise structure for competitive-based technology transfer ijahp article: maludin, syarief, rifin, rochman/reassembling technology transfer in indonesia international journal of the analytic hierarchy process 447 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.846 based on the first round of meetings with the experts, we developed a four-level hierarchy structure. the first layer of the structure is the objective of “achieving a competitive-based innovative technology.” strengthening the capacity of technology transfer is the objective of a technology transfer process according to the experts based on their experience and literature research. the experts concluded that the transfer of innovative and appropriate technology is essential. the second layer of the hierarchy structure contains the criteria. the experts agreed on five conditions for achieving the objective, including patent productivity, employment, market creation, the eco-friendly process of manufacturing, and the efficient process of production. in the first interview, the experts suggested that the patent should be viewed as the protection of intellectual production in the commercial field. in other words, the patent must be treated as a tool and not as an objective. most public institutions put the number of patents achieved as their target which leads to high maintenance costs for ip. this has led to the business value of a patent being whether or not it is bankable 5 . however, new technology should also meet ecological safety standards. the third layer includes the agents or actors of technology transfer. the experts mostly agreed on six parties which include a research division under technical ministries, government research institutes, universities, nongovernment organizations, industrial r&d and global agencies. the fourth layer is the most important for the decision-maker and includes four alternative strategies which are building tbi and stp; establishing an overseas representative office for st; strengthening existing public research institutes and increasing budgets for st. in the survey, each factor was compared and connected to the above factors. for example, at level 1, namely criteria, each factor was compared to support a goal. at level 2, the actor factors were compared to support each of the criteria listed at level 1. in discussions and surveys, it was necessary to adjust the description of the comparative factors because of the time pressure resulting from questions that seem to be repeated. returning the focus of the discussion to each question was important in limiting the topic being discussed. the ten ahp questionnaire files were entered into an ahp table created in excel. the measurement of the consistency index of each respondent will help reconfirm the responses with a low level of consistency. this data was also used to combine the surveys from the ten experts using the geometric mean. next, the combined ranking of the geometric mean was entered into the superdecisionss 6 software. 5 the motivation for researchers to protect intellectual property is a measure of the achievements of universities and research organizations. this has led to an increase in the number of patents, which was not followed by an increase in the number of applications. the mistake was in considering the realization of the number of patents as an objective and not as a tool. 6 superdecisions® is a decision support software that implements the ahp and anp. ijahp article: maludin, syarief, rifin, rochman/reassembling technology transfer in indonesia international journal of the analytic hierarchy process 448 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.846 5. results and discussion the selected experts from the field of technology transfer were first contacted by the academic staff via letter followed by contacting either the key person or the secretary of the key person. limited time in their schedules made it difficult to get an interview with the experts, which was a reason for not using a focus group. based on the results of questionnaires that were appropriate in terms of consistency and sensitivity, the results are shown in figure 4. figure 4 weighted pairwise structure for competitive-based technology transfer there were three modes of discussion in the first round of interviews. first, a combination of communication by email and telephone calls was used. during the first contact, we explained the objective of the research and the method of the interview. in this first round of interviews, experts were encouraged to ask, confirm or share their ideas and recommendations relating to the questionnaire. there were twelve government officials involved with technology transfer and innovation technology who agreed to participate in this project. however, two respondents preferred to respond to open-ended questions and via discussion, and another expert, due to very limited time, preferred to provide resources and share his experiences and ideas for technology development in indonesia. this structure was built based on recommendations from and discussion with an expert in the first meeting. the view of each element in the structure was strongly influenced by the background of their work. the experts suggested the key criterion of patent ijahp article: maludin, syarief, rifin, rochman/reassembling technology transfer in indonesia international journal of the analytic hierarchy process 449 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.846 productivity in the success of the technology transfer process (weight=0.34898). when the protection of intellectual production does not lead to commercialization, it is recommended that another appropriability regime be used such as copyright in an academic publication. patent registration with no exit strategy will result in expensive maintenance costs. this criterion was also chosen based on the different treatment of patents by public and private organizations. some public research institutes treat ip as a target of ip management. in the firm, ip was treated as a tool for protecting operations based on new technology. when a patent is treated as a target of the innovation process, patent registration as well as the maintenance cost will increase. however, in this scenario, there will be no return for either the inventor or the office. employment (weight=0.19329) was another criterion that experts endorsed to consider the importance of government expenditure that should be returned to the taxpayer. the growing population of indonesia has experienced increased unemployment. development in science and technology is likely one alternative to absorb unemployment. it relates to establishing tbi and stp where new entrepreneurs are created. there were fourteen agreements 7 on utilizing technology between lipi and businesses that called for job applications to work in the center. market creation was regarded as the second most important criterion (weight=0.31247). the application of new technology can create demand. feasibility studies equip the producer to sell the product. the flow of technology in the market sometimes leads to a market-driven push for research. in this regard, the technology provider considers how their research could solve the problem; however, there is limited consideration of potential customers (di stefano et al., 2012; herstatt & lettl, 2004; moerle & specht, 2014). this is similar to the resource-based view (rbv) which believes that business development should begin with internal capacity. in this view, the researchers satisfy their curiosity and desire to participate in intellectual activities, but there is limited consideration of the potential customer. another concept of marketing innovation-based products is being market-driven and implies that product development should consider the preference of the market (horbach et al., 2012). for example, if the market wants the product painted red, then it is better not to sell a product that is painted black. experts found that the impact of technology transfer should be measured and justified by developing the right policy. eco-friendly innovative products are available in some research centers in indonesia. experts successfully showed the number of research proposals for advanced technology that were developed by local scientists. however, these products do not meet the demands in the market. one expert presented the case of anti-fouling paint that was developed in one research center in indonesia. under the international standard of maritime industry, every vessel that sails over international borders should use special anti-fouling paint that does not destroy the environment. there are many other ecofriendly products developed in research centers that fail to go to market. 7 english version of this information could be accessed in http://inovasi.lipi.go.id/en/profile/agreement ijahp article: maludin, syarief, rifin, rochman/reassembling technology transfer in indonesia international journal of the analytic hierarchy process 450 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.846 learning from industrial countries that meet the efficient process of production by utilization of innovation, technology transfer should support the low-cost manufacturing process (weight=0.07165). however, it seems that when technology comes to an existing industrial cluster, it can lead to the bankruptcy of that industry. experts shared the case of the apparel industry in west java that closed when a technology program was introduced to the cluster. another example was the agri-industry in north sumatra that closed as a collaborative program from indonesia and an international body applied technology and a new business process. therefore, this criterion was worth including. most experts agreed with the key actors capable of strengthening the technology transfer system in indonesia. actors play an important role in establishing key research that meets the demand as a bankable product or the research that assists small and medium entrepreneurs and farmers. four alternative schemes should be considered to establish a better outcome of technology transfer. based on the different backgrounds of their work, several different nomenclatures with similar meanings were used to describe the expert’s position. some experts were called the manager of product development (mcd), and others were called the chief innovation officer (cio). the research scheme was built to consider the differences of terminology. 5.1 evaluation of criteria the rank of the weight of the individual variables in the selected criteria is as follows, from highest to lowest: patent productivity (0.34898), market creation (0.31237), employment (0.19329), eco-product (0.07371) and better cost of product (0.07165). patent productivity was ranked first along with market creation within the criteria cluster. the success of the technology process on a macro scale is a measure of increasing awareness of intellectual property that could be valued by the number of patents created and utilized. the budget allocation for patent development in public research institutes should be treated as a long-term investment which is different from patent success in the private sector. it leads to the valuation of the technology when considering whether one particular outcome of the research has industrial application when the selected technology is submitted to the patent office to determine the novelty of the technology. alternatively, the research and development division in the manufacturing industry gets the ideas through an internal process and external environment that can lead to intellectual protection. therefore, most technologies have been operated internally before the protection process while new technology developed in a public research institute would enter the market trial and licensing procedure. the key criterion of successful technology transfer is patent productivity. market creation is the second most important variable to achieve a competitive-based innovation technology. technological change plays a strategic role in strengthening national competitiveness and requires a migration path from idea creation to prototyping, and a minimum viable product to a commercial product. for example, the migration from ijahp article: maludin, syarief, rifin, rochman/reassembling technology transfer in indonesia international journal of the analytic hierarchy process 451 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.846 a paging communicator to a cellular phone or smartphone required certain migration paths. users do not instantly accept new technology while the product cycle for advanced technology may last for a shorter time. however, when the efficiency of production leads to a lower product price in the market and is supported with better promotion, the business can meet people’s demands. migration paths for individual technologies are created differently (innovation creates its artery). also, targeted consumers are not always equipped with sufficient information to decide whether they need one particular kind of advanced technology or another. people often form their interests and demand based on the outcome of the product (outcome-driven innovation). the group of experts aligned the variable of market creation (0.31237) with the efficiency of the manufacturing process (0.07165). in most cases, the initial phase of developing an applied technology has high risk and uncertainty. in the next stage, when the new technology has achieved its target, the old manufacturing process becomes expensive and unreliable. in informal discussions, experts highlighted the important role of standards as a criterion for competitiveness. innovation-based products should meet product standardization and intellectual property. 5.2 evaluation of actors the actors are weighted from highest to the lowest as follows: government funded research institutes (gfri), (0.27833), research division under ministerial office, (0.24890), universities, (0.17966), private r&d, (0.13589) and foreign agencies, (0.07214). the experts believe that government funded research institutes are the top actors that play a significant role in the technology transfer process. the core function of gfri in the technology transfer system is to plan, conduct and develop technology, and they have contributed significantly to the policymaking process by providing information and policy recommendations. this recommendation was acted on by the government by establishing the nria. however, the transition has experienced delays due to procedural budgeting and human resources allocation. also, the covid-19 pandemic provided the researcher with unlimited working hours from home. the research and development division in the ministerial office provides solutions to technical problems identified by the ministerial office. this division has a close relationship with the core function of the ministerial office where most of the technology created by the researcher was developed and utilized. public-funded research institutes and technical divisions under the ministerial office would be the two groups of actors integrated into the nria. this is expected to open pathways between researchers and make the flow of basic research to application more controllable. however, there are two different opinions regarding the concept of technology transfer under the nria. first, a group of researchers and stakeholders have a pessimistic view that based on the background of transferring technology it is highly unlikely that it will go from basic to development and application at the end of the tunnel. every actor has his own methods and potential users. the second group believes that the problem of technology transfer is not about institutional arrangement. they see the problem as the narrow view of the decision-maker and believe the solution should come ijahp article: maludin, syarief, rifin, rochman/reassembling technology transfer in indonesia international journal of the analytic hierarchy process 452 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.846 from a higher level. parliament should also be a representative of experts at universities and research institutes. this would reduce the unbalanced view of developing science and technology capability and technology transfer. universities contribute significantly to the technology transfer system supported by both the quality and quantity of new students (worker bees). research in the university begins in laboratories or classes with joint work between lecturers and students. youth programs to create entrepreneurs are introduced to students, including financial incentives, business coaching, and the privilege to access research facilities on campus including an expert in a particular field. local area network (lan) and touchscreens are success stories of technologies developed by gfri (mazzucato, 2011). most non-government organizations (ngos) work directly or indirectly in the technology transfer process and are related to eco-products, agribusiness for women, energy, and food processing. funding comes from international actors that represent developed countries or international organizations. ngos are more responsive than government bodies due to their tight connection with the user, especially in the fields of agribusiness and small business. the role of private enterprises in the technology transfer process is still small. most manufacturing sites are not equipped with an r&d division. the principal of a business provides equipment, supplies and production manuals for a manufacturing process that maintains the stability of the supply and standards of the product. however, in some cases, this closes the opportunity for collaboration with domestic research institutes. indonesia is considered to have a growing market that follows the increasing number of middle-class. technology-based-products such as automobiles and smartphones are in sufficient demand, but this demand has not been followed by the development of an internal research division. foreign agencies in this research are the domestic technology provider as representatives of foreign countries, universities, bodies, or research institutes. most collaboration occurs for non-commercial research outcomes, such as joint research, joint publication, student exchange and scholarships. 5.3 evaluation of policy recommendation the important outcome of the ahp decision-making process is the ranking of alternatives for action (strategies) that can be translated into policy recommendations. the highest weight is establishing a technology and business incubator (tbi) and science and technology park (stp) (0.42578), followed by opening an overseas-sciencerepresentatives office (0.32837), strengthening existing public research institutes (0.16062) and increasing government expenditure on research and development (0.19329). experts ranked tbi and stp (0.42578) as the most significant for introducing new technology to the user as well as protection from risk and uncertainty for both researchers and entrepreneurs. new technology is achieved through market validation and appropriate trials for a reliable product. a tbi would be developed inside a university ijahp article: maludin, syarief, rifin, rochman/reassembling technology transfer in indonesia international journal of the analytic hierarchy process 453 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.846 campus and be closely aligned with experts. a stp treats technology in the wider space which is supported by sufficient access to industry as the user of technology and the scientist as the technology provider. the results indicate that policymakers in indonesia should establish science and technology representatives. experts were able to show that international research collaboration was attached to a particular individual scientist. this flagship was very flexible, easy to set up and easy to fail. globalization required a wider and stronger engagement and communication in the field of science and technology. experts also found a growing number of visits of foreign bodies to research installations in indonesia, but these were generally not well represented. indonesia has had education attachés that did not cover the subject of science and technology. education attachés are required to be a certified lecturer or teacher who emphasizes student and youth exchange programs. regulators found strengthening existing science and technology institutions to be more important than increasing gerd. many parties had advised the government to increase the budget allocation for st. however, the experts found that items of st were extremely flexible and that fluctuations in the budget had influenced the results of research activities. one expert suggested that if the budget was not limited, then the results would still be the same, which was indicated by the same research proposal, the same outcome and the same researcher. another view suggested that, a research organization with old equipment and an old building needed to be refurbished. therefore, increasing gerd (0.19329) was more important than strengthening existing research institutes (0.16062). 6. conclusion the highest score in every layer was patent productivity (0.34898); followed by government-funded research institute (0.27833) and establishing the technology incubator and science technology park (0.42578). the number of patents generated by national research institutes is lower than a global player in indonesia; hence, patent productivity has become the greatest target for the public research institute. however, patents should be treated as the pathway for bankable and commercial utilization because setting a patent as the target will create maintenance costs. government-funded research institutes are expected to play a significant role in the technology transfer system. centralization of public research institutes with technical divisions under the ministerial office in the nria would be the right decision if and only if, it is equipped with a comprehensive arrangement that involves stakeholders. while several developed countries have experienced successful tbi and stp, the experts found that it would also increase the technology utilization in indonesia. the private sector should be involved in creating a productive stp and tbi. this research is relevant because of the establishment of the nria which is based on the inefficient government ijahp article: maludin, syarief, rifin, rochman/reassembling technology transfer in indonesia international journal of the analytic hierarchy process 454 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.846 r&d budget, which was the unpopular alternative on the ahp structure. there are three other important recommendations of this research for the nria. 7. further research during a discussion of the ahp questionnaire in december 2017, the need for reconsideration of bicameralism in indonesia was raised. this is related to the status of a publicly funded research body and the indirect interference of a political party. it occurred when integration of the structure of the ministry of research and technology with the ministry of education of the republic of indonesia as part of the formation of the nria was a risk of political compromise in the government's r&d development at that time. ijahp article: maludin, syarief, rifin, rochman/reassembling technology transfer in indonesia international journal of the analytic hierarchy process 455 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.846 references bercovitz, j., & feldmann, m. (2006). entrepreneurial universities and technology transfer: a conceptual framework for understanding knowledge-based economic development. journal of technology transfer, 31(1), 175–188. https://doi.org/10.1007/s10961-005-5029-z blohmke, j. (2014). technology complexity, technology transfer mechanisms and sustainable development. energy for sustainable development, 23, 237–246. doi: https://doi.org/10.1016/j.esd.2014.09.003 bozeman, b. (2000). technology transfer and public policy: a review of research and theory. research policy, 29(4–5), 627–655. doi: https://doi.org/10.1016/s00487333(99)00093-1 chehrehpak, m., alirezaei, a., & farmani, m. (2012). selecting optimal methods for the technology transfer by using the analytic hierarchy process (ahp). indian journal of science and technology, 5(4), 2540–2546. costantini, v., & liberati, p. (2014). technology transfer, institutions and development. technological forecasting and social change, 88, 26–48. doi: https://doi.org/10.1016/j.techfore.2014.06.014 delgado, m., ketels, c., porter, m. e., & stern, s. (2012). the determinants of national competitiveness. zhurnal eksperimental’noi i teoreticheskoi fiziki, 47. doi: https://doi.org/10.3386/w18249 di stefano, g., gambardella, a., & verona, g. (2012). technology push and demandpull perspectives in innovation studies: current findings and future research directions. research policy, 41(8), 1283–1295. doi: https://doi.org/10.1016/j.respol.2012.03.021 direktorat jenderal hak kekayaan intelektual republik indonesia. (2017). pendaftaran paten lokal dan internasional di indonesia 2015 2017. http://statistik.dgip.go.id/statistik/production/paten_negara.php erensal, y. c., & albayrak, y. e. (2007). transferring appropriate manufacturing technologies for developing countries. doi: https://doi.org/10.1108/17410380810847891 gerdsri, n., & kocaoglu, d. f. (2007). applying the analytic hierarchy process (ahp) to build a strategic framework for technology roadmapping. mathematical and computer modelling, 46(7–8), 1071–1080. doi: https://doi.org/10.1016/j.mcm.2007.03.015 herstatt, c., & lettl, c. (2004). management of “technology push” development projects. international journal of technology management, 27(2/3), 155. doi: https://doi.org/10.1504/ijtm.2004.003950 ijahp article: maludin, syarief, rifin, rochman/reassembling technology transfer in indonesia international journal of the analytic hierarchy process 456 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.846 horbach, j., rammer, c., & rennings, k. (2012). determinants of eco-innovations by type of environmental impact the role of regulatory push/pull, technology push and market pull. ecological economics, 78, 112–122. doi: https://doi.org/10.1016/j.ecolecon.2012.04.005 horvath, b., & kelsen, h. (1957). what is justice? justice, law, and politics in the mirror of science. collected essays. the american journal of comparative law. doi: https://doi.org/10.2307/837546 kumar, s., luthra, s., & haleem, a. (2015). benchmarking supply chains by analyzing technology transfer critical barriers using ahp approach. benchmarking: an international journal, 22(4), 538–558. doi: https://doi.org/10.1108/bij-05-2014-0040 kumar, s., luthra, s., haleem, a., mangla, s. k., & garg, d. (2015a). identification and evaluation of critical factors to technology transfer using ahp approach. international strategic management review, 3(1–2), 24–42. doi: https://doi.org/10.1016/j.ism.2015.09.001 kumar, s., luthra, s., haleem, a., mangla, s. k., & garg, d. (2015b). identification and evaluation of critical factors to technology transfer using ahp approach. in international strategic management review (vol. 3, issue 1). holy spirit university of kaslik. doi: https://doi.org/10.1016/j.ism.2015.09.001 lee, s., kim, w., kim, y. m., oh, k. j., ahmadabadi, m. n., najafi, m., gholami, p. p., & gholami, p. p. (2013). using analytic hierarchy process (ahp) to use intangible factors for technology transfer adoption. expert systems with applications. 39(7), 63886395. doi: https://doi.org/10.1016/j.eswa.2011.12.030 mazzucato, m. (2011). the entrepreneurial state: debunking private vs. public sctor myths. publicaffairs. miller, g. a. (1981). trends and debates in cognitive psychology. cognition, 10(1–3), 215–225. doi: https://doi.org/10.1016/0010-0277(81)90049-4 moerle, m., & specht, d. (2014). technology push. gabler wirtschaftslexikon. pusat inovasi lipi. (2017). ip port pusat inovasi. http://inovasi.lipi.go.id/id/hki reisman, a. (2005). transfer of technologies: a cross-disciplinary taxonomy. omega, 33(3), 189–202. doi: https://doi.org/10.1016/j.omega.2004.04.004 saaty, t. l. (2003). saaty t.l. (2003), the analytic hierarchy process (ahp) for decision making and the analytic network process (anp) for decision making with dependence and feedback. creative decisions foundation, 114. http://www.rwspublications.com/books/ ijahp article: maludin, syarief, rifin, rochman/reassembling technology transfer in indonesia international journal of the analytic hierarchy process 457 vol. 13 issue 3 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i3.846 sipahi, s., & timor, m. (2010). the analytic hierarchy process and analytic network process: an overview of applications. management decision, 48(5), 775–808. doi: https://doi.org/10.1108/00251741011043920 slaughter, s., & rhoades, g. (1996). the emergence of a competitiveness research and development policy coalition and the commercialization of academic science and technology. science technology and human values, 21(3), 303–339. doi: https://doi.org/10.1177/016224399602100303 wang, b., shyu, j. z., cheng, c. c., & hsieh, c. h. (2009). changing technology transfer strategies in a non-profit organization an examination of itri. international journal of innovative computing, information and control, 5(6), 1527–1538. yazdani, k., rashvanlouei, k. y., & ismail, k. (2011). ranking of technology transfer barriers in developing countries; a case study of iran’s biotechnology industry. ieee international conference on industrial engineering and engineering management, 1602–1606. doi: https://doi.org/10.1109/ieem.2011.6118187 ijahp: mu/words from the editor in chief international journal of the analytic hierarchy process issn 1936-6744 1 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.257 september 30, 2014 dear ijahp reader: welcome to our first issue in 2014! this year has been very busy for the ahp community due to the international symposium of the analytic hierarchy process (isahp 2014) that took place from june 29 – july 2 in washington, d.c. ijahp sponsored and participated in the organization of the meeting and several selected papers from the symposium will be published in our upcoming issues. the ijahp editorial board was instrumental in the academic organization of isahp 2014 while maestro meetings, led by milagros pereyra, did a great job handling the symposium logistics. it was very hard work for all of us; however, it was worth it, given that the meeting was a success both academically and socially. we will include a more extensive report of the symposium in our next issue, given that some administrative aspects had not been completed by the time the contents for the current issue were scheduled. our first article in this issue, by yavuz and baycan, deals with the problem of implementing a collaborative watershed management strategy and for this purpose, involving relevant stakeholders is a key element for the success of the strategy. next, mehta, verma and seth develop a model to prioritize 13 total quality management criteria and 68 sub-criteria in the engineering education sector. servqual is an accepted framework for the assessment of service quality. miri et al use this framework, in combination with anp to prioritize the service quality of insurance branches. renewable energy resource (rer) projects have become very important for the sustainability of our society. in the fourth paper in this issue, dhital et al address a rer project evaluation in nepal from the perspective of those responsible for implementing the project as well as the users. finally, we conclude the section of peer-reviewed articles with a theoretical paper. hefnawi and soliman review the different methods used to obtain the ahp priority vector, and compare it with the current eigenvector method. besides the customary set of research articles, we have included in this issue a very interesting interview with thomas l. saaty, ahp/anp creator and founder of this journal. don’t miss this interview since tom still has much to teach us about ahp. also, if you are soccer fan you must have watched the fifa 2014 world cup this past summer and are probably aware of the controversy surrounding the fifa golden ball award, the award which recognizes the best player in the tournament. a ijahp: mu/words from the editor in chief international journal of the analytic hierarchy process issn 1936-6744 2 vol. 6 issue 1 2014 http://dx.doi.org/10.13033/ijahp.v6i1.257 large number of fans didn’t agree with fifa’s decision of the award winner. being originally from latin america, where soccer, religion and politics are the three most important activities (although i’m not sure in what order), i have included some reflections on how ahp could contribute to the decision about who, in my humble opinion, actually deserved the award, and how to avoid these award controversies in the future. finally, you may notice that there is a doi identification at the bottom of the first page in each article. the doi format is as follows: http://dx.doi.org/10.13033/ijahp2014.v6.is1.xxx where xxx is the corresponding article identification (e.g. 001, 002, 003…) in the issue this doi identification will give a unique identifier to the electronic article, independent of any change to the journal host, and will allow ijahp to be on par with best practices in electronic journal publishing. at ijahp we are always looking for ways to improve the journal standing, and to ensure that our ahp community can be proud of the quality of their official journal. enjoy our ijahp issue! enrique mu, phd editor-in-chief http://dx.doi.org/10.13033/ijahp2014.v6.is1.xxx ijahp article: garg, sharma, jain/an ahp-based optimal distribution model and its application in covid-19 vaccination international journal of the analytic hierarchy process 207 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.863 an ahp-based optimal distribution model and its application in covid-19 vaccination arpan garg research scholar; dept. of mathematics & scientific computing national institute of technology, hamirpur, india arpan1996garg@gmail.com y d sharma professor & head; dept. of mathematics & scientific computing national institute of technology, hamirpur, india yds@nith.ac.in subit kumar jain assistant professor; dept. of mathematics & scientific computing national institute of technology, hamirpur, india jain.subit@nith.ac.in abstract covid-19 is causing a large number of causalities and producing tedious healthcare management problems at a global level. during a pandemic, resource availability and optimal distribution of the resources may save lives. due to this issue, the authors have proposed an analytical hierarchy process (ahp) based optimal distribution model. the proposed distribution model advances the ahp and enhances real-time model applicability by eliminating judgmental scale errors. the model development is systematically discussed. also, the proposed model is utilized as a state-level optimal covid-19 vaccine distribution model with limited vaccine availability. the covid-19 vaccine distribution model used 28 indian states and 7 union territories as the decision elements for the vaccination problem. the state-wise preference weights were calculated using the geometric mean ahp analysis method. the optimal state-level distribution of the covid-19 vaccine was obtained using preference weights, vaccine availability and the fact that a patient requires exactly 𝑟vaccine doses to complete a vaccination schedule. the optimal covid-19 vaccine distribution along with state and union territory rank, and preference weights were compiled. the obtained results found kerala, maharashtra, uttarakhand, karnataka, and west bengal to be the most covid-19 affected states. in the future, the authors suggest using the proposed model to design an optimal vaccine distribution strategy at the district or country level, and to design a vaccine storage/inventory model to ensure optimal use of a vaccine storage center covering nearby territories. keywords: covid-19; vaccine distribution model; ahp; mcdm; risk assessment mailto:arpan1996garg@gmail.com mailto:yds@nith.ac.in mailto:jain.subit@nith.ac.in ijahp article: garg, sharma, jain/an ahp-based optimal distribution model and its application in covid-19 vaccination international journal of the analytic hierarchy process 208 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.863 1. introduction in early december 2019, the first active human covid-19 (corona virus disease) case was officially identified in wuhan city, hubei province, china (who situation report– 94). on january 30, 2020, the world health organization (who) declared covid-19 with its exponential rate of increase and zoonotic nature, an international public health emergency (who situation report –11). on february 18, 2021, only 1 year and 3 months after first being identified, covid-19 had infected around 110 million people worldwide. a sobering statistic shows that out of 87,769,856 cases, 2,441,043 or almost 3% have died. (worldometer, 2020). currently, this pandemic has affected 219 countries and 2 international conveyances. many government organizations, universities, and independent institutions are investing a very large amount of money, time, and manpower in research to ensure the creation of a scientifically sound and safe treatment or vaccine for covid-19. vaccines can be divided into a number of different types such as live attenuated vaccines (lav), inactivated virus vaccines, sub-unit vaccines, viral vector-based vaccines, dna vaccines, and mrna vaccines (kaur & gupta, 2020); however, they all work on the same principle of stimulating the immune response to recognize a disease-causing organism. according to the who (steps in vaccine development), a vaccine candidate has to clear pre-clinical studies (including studies on animals for efficiency and safety), phase-1, phase-2, phase-3, and post-marketing surveillance studies to ensure complete vaccine development. the covid-19 pandemic has produced a number of healthcare issues including not only finding an efficient vaccine, but determining an optimal distribution strategy as well. a pandemic response using a vaccine requires a systematic solution for vaccine i) development ii) production, and iii) distribution. for the first step, according to who, a number of vaccine candidates from all over the world are in various clinical stages (subdivided into phase 1, ½, 2, and 3), and some have shown promising outcomes in the human trials (who draft-landscape,2021). the second step of the program includes the production of a clinically sound vaccine. since, the most advanced and cutting-edge production strategy is needed to fulfill the global requirements for covid-19 vaccines, many alliances among various biotech companies have been formed to advance vaccine production (lancet, 2020). the final step of the program requires an answer to the question; what is the optimal distribution plan? the ministry of health and family welfare, in india has issued operational guidelines (2020) for phase-1 vaccine roll out, and plans to sequentially vaccinate about 10 million healthcare workers, 20 million frontline workers, and around 270 million people above 50 years of age. the phase-1 roll out (2021) of the covishield (viral vector vaccine) and covaxin (inactivated virus vaccine) vaccines in india started in january 16, 2021. both covishield and covaxin need 2-doses to complete the vaccination schedule. since, an inactivated virus type vaccine requires booster shots to maintain immunity (kaur & gupta, 2020), those receiving covaxin will need an additional booster dose to complete the vaccination schedule. the authors observed that the production of the covid-19 vaccine would definitely fall short of the global requirements hence, the need for an optimal covid-19 vaccine distribution strategy is an expected multicriteria decision-making (mcdm) problem in the near future due to the availability constraints and multiple dose requirement of the covid-19 vaccine. ijahp article: garg, sharma, jain/an ahp-based optimal distribution model and its application in covid-19 vaccination international journal of the analytic hierarchy process 209 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.863 while searching for a solution to the state-level vaccine distribution problem, the authors found a popular mcdm problem-solving tool known as the ahp. the ahp methodology was introduced by t.l saaty (1977, 1990, 2003) to solve real-time complex mcdm problems. the application of the existing ahp model can be defined as a fivestep procedure (rosenbloom, 1997). the easy to apply systematic ahp model has a large domain of applications, including supply chain management, construction, healthcare management, safety sciences, risk assessment and many other non-mathematical fields. recently, we have observed the presence of various mathematical theories (mishra et al., 2020) in covid-19 research. the ahp has been useful in solving various healthcare policy issues including a number of management issues due to the covid-19 pandemic. mohammed et al. (2021) applied ahp methodology to propose a convalescent plasma transfusion intelligent framework for covid-19 patients, and garg et al. (2020) conducted a covid-19 risk assessment considering the ease in lockdown restrictions and population density of various activities. moreover, halder et al. (2020), improt et al. (2019), corvinet et al.(2020),and hezametal et al. (2021) have applied the ahp to deal with various health care management issues. the authors observed that the ahp is an efficient mcdm technique with less computational requirements than most methods that can be utilized to develop a multipurpose distribution model resulting in significant advancements. the ahp uses saaty’s fundamental scale (saaty, 1990) to collect pairwise comparison responses from different respondents, and to construct a pairwise comparison matrix (pcm). therefore, the outputs are highly dependent on the data collection and human responses. moreover, the solution strategy for a real-time distribution problem using the ahp provides no possible way to use human responses. this methodological issue can be solved by replacing the verbal scale with a well-defined mathematical expression that helps produce an unbiased pcm for the vaccination model. therefore, the authors have developed a more dynamic ahp-based distribution model that has some significant improvements over the existing ahp model. in the present article, the authors have proposed an ahp-based multipurpose optimal distribution model. the proposed model is used to determine an optimal distribution strategy for the covid-19 vaccine for 28 indian states and 7 union territories (u.ts). it can be used for limited, but varying, vaccine availability and can be generalized for large or small territorial areas. moreover, the authors have also identified and discussed various healthcare management applications of the proposed distribution model. the work presented in this article is divided into four sections. the first section provides a general introduction to the vaccine distribution problem and the ahp procedure. in the second section, the authors have developed an ahp-based multipurpose distribution model. the third section includes the results and discussion, in which the proposed model is applied with the covid-19 data to provide an optimal state-level covid-19 vaccine distribution strategy in india. in the last section, the conclusions are discussed and the future scope of the proposed model is presented to researchers and various policy makers for the use of ahp to solve heath care management issues. 2. ahp distribution model development an ethical vaccine distribution model must be free from human bias that might favor people based on region or religion, or include possible methodological errors resulting from a human judgmental scale. however, the distribution model must be able to compile ijahp article: garg, sharma, jain/an ahp-based optimal distribution model and its application in covid-19 vaccination international journal of the analytic hierarchy process 210 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.863 the heterogeneous effects of a number of factors to the degree of importance present among the decision elements. the authors have proposed an ahp-based optimal distribution model, along with required advancements, in a systematic manner as given below: 1. hierarchy structure: the first step in this model includes identifying a finite number of decision elements, and then defining the distribution problem into a hierarchy structure of decision elements (saaty, 1987). 2. decision element coefficient(𝑅𝑖 ): the decision element coefficient(𝑅𝑖 )value for the 𝑖𝑡ℎ decision element, compiling the combined effect of various heterogeneous factors, is defined as 𝑅𝑖 = ∑ 𝑤𝑗 𝑗 ∗ 𝑓𝑎𝑐𝑡𝑜𝑟𝑗 where ∑ 𝑤𝑗𝑗 = 1, and 𝑤𝑗 represents a justified weight assigned to the 𝑗 𝑡ℎ factor. the general computation formula 𝑅𝑖 is designed to include the effect of a finitely large number of important factors for the problem into the model. the proposed strategy for addressing a large number of factors, including those accountable for small but significant preference variation, may help improve the real-time applicability of the model. 3. pairwise comparison matrix (pcm): the proposed model uses the mathematical function, 𝑓: 𝑆 → 𝑅+ defined over the set 𝑆 of all possible pairs of decision element coefficients to the set 𝑅+of positive real numbers, such that 𝑓( 𝑅𝑖 , 𝑅𝑗 ) = 𝑓𝑖𝑗 = 𝑅𝑖 𝑅𝑗 ⁄ ∀ 𝑖 𝑎𝑛𝑑 𝑗. for pairwise comparison of decision elements to construct the required pcm [𝑓𝑖𝑗 ]𝑙×𝑙 . mathematically, the pairwise comparison function 𝑓,with 𝑓𝑖𝑗 . 𝑓𝑗𝑘 = 𝑅𝑖 𝑅𝑗 . 𝑅𝑗 𝑅𝑘 = 𝑅𝑖 𝑅𝑘 = 𝑓𝑖𝑘 ensures perfect consistency (saaty, 2003) of the pcm that eliminates any potential conflict over the selection of a particular ahp analysis method. also, replacing the linguistic judgment scale from a well-defined mathematical function 𝑓, leads to the elimination of human bias. 4. preference weight𝑠(𝑣𝑖 ) and ranking: the preference weight values(𝑣𝑖 ) can be calculated using the suitable ahp analysis method. however, the model provides a complete choice for selecting a suitable ahp analysis method; the authors used the geometric mean method (gmm)(crawford & williams, 1985) to calculate the preference weight values(𝑣𝑖 ). the presented model can be utilized as an efficient ahp-based mcdm problem solving tool and as a multipurpose optimal distribution model. further, the model is designed to ijahp article: garg, sharma, jain/an ahp-based optimal distribution model and its application in covid-19 vaccination international journal of the analytic hierarchy process 211 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.863 calculate an optimal state-level distribution strategy for covid-19 vaccines. the additional steps required to develop an optimal state-level distribution model are given below. 5. optimal vaccine distribution: the optimal vaccine distribution for the 𝑖𝑡ℎ state with total vaccine availability 𝑉, and the fact that a patient requires exactly 𝑟 vaccine doses along with ensuring state-wise homogeneity is defined as 𝑟. ⌊ 𝑉.𝑣𝑖 𝑟 ⌋ ; where, ⌊ ⌋is greatest integer function. the vaccine distribution formula uses the greatest integer function to obtain distribution optimality by avoiding statelevel redundant overlapping of vaccine doses (i.e. vaccine doses left that are not sufficient for single patient treatment). 3. results and discussion the proposed vaccine distribution model is explained as a five-step procedure. since covid-19 is transmitted via human contact (chan et al., 2020), it is commonly thought that highly populated countries will suffer the worst from covid-19. therefore, the authors have utilized this distribution model to create an optimal vaccine distribution strategy for the world’s second most populated (1.39 billion) and seventh largest country, india) (population of india, 2021). the proposed model was applied for all states and u.ts, excluding lakshadweep, to determine an optimal vaccine distribution strategy for the covid-19 vaccine using the reported covid-19 data as of january 15, 2021(covid19india, 2021). for the first step, the model used 28 indian states and 7 u.ts as decision elements for the problem. the hierarchy structure of the state-level optimal vaccine distribution problem was defined by placing all 35 decision elements as alternatives of the study at the second hierarchy layer, and the problem to select a state or u.t receiving the largest vaccine lot was placed at the first hierarchy layer (figure1). figure 1 hierarchy structure of the vaccine distribution problem ijahp article: garg, sharma, jain/an ahp-based optimal distribution model and its application in covid-19 vaccination international journal of the analytic hierarchy process 212 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.863 the authors observed that for a country like india, that covers a large territorial area and is sub-divided into a number of states and union territories (u.t) that differ from each other in size, population, density, health care facilities and medical staff availability etc., it is important to consider a measure of healthcare efficiency heterogeneity among the states/u.ts. since the lack of healthcare facilities and medical staff availability during the pandemic resulted in a higher number of deaths, these figures along with the number of active covid-19 cases are considered critical factors for a vaccination model to a degree of significance to calculate the risk coefficient value relating to a state/u.t. for the second step in formulating an optimal covid-19 vaccine distribution model, the authors calculated the state risk coefficient (𝑅𝑖 ), considering the number of active covid-19 cases(𝑎𝑖 ), and the number of deaths(𝑑𝑖 ) reported in the 𝑖 𝑡ℎ state/u.t. the model application utilized two key factors for this problem, but it can also be generalized for future studies using several other factors such as the strength of the healthcare system and front line workers, pregnant women, condition of infected people, and population of senior citizens (individuals age >= 50). the globally reported 3% death rate influenced the authors to assign a weight of 0.03 to the number of deaths and 0.97 to the other factor, i.e., ri = 0.97 ∗ 𝑎𝑖 + 0.03 ∗ 𝑑𝑖 . also, all 28 states and 7 u.ts in india were listed from 1 to 35 in alphabetical order with the number of active covid-19 cases(𝑎𝑖 ), number of deaths(𝑑𝑖 ) reported as of january 15,2021 (covid19india, 2021), and the calculated state risk coefficient (𝑅𝑖 ) in table1. ijahp article: garg, sharma, jain/an ahp-based optimal distribution model and its application in covid-19 vaccination international journal of the analytic hierarchy process 213 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.863 table 1 state-wise reported covid-19 data (covid19india, 2021) and calculated 𝐑𝐢 value s.no (𝑖) state/union territory active (𝑎𝑖 ) death (𝑑𝑖 ) state risk coefficient (𝑅𝑖 ) s.no (𝑖) state/union territory active (𝑎𝑖 ) death (𝑑𝑖 ) state risk coefficient (𝑅𝑖 ) 1 andaman and nicobar islands 22 62 23.2 19 madhya pradesh 6957 3746 6860.67 2 andhra pradesh 2199 7139 2347.2 20 maharashtra 52152 50336 52097.5 3 arunachal pradesh 64 56 63.76 21 manipur 436 365 433.87 4 assam 1616 1066 1599.5 22 meghalaya 161 144 160.49 5 bihar 3981 1449 3905.04 23 mizoram 89 9 86.6 6 chandigarh 266 330 267.92 24 nagaland 104 88 103.52 7 chhattisgar h 6923 3544 6821.63 25 odisha 1910 1951 1911.23 8 dadra and nagar haveli and daman and diu 11 2 10.73 26 puducherry 294 640 304.38 9 delhi 2795 10732 3033.11 27 punjab 2739 5485 2821.38 10 goa 866 753 862.61 28 rajasthan 5608 2744 5522.08 11 gujarat 6750 4360 6678.3 29 sikkim 163 130 162.01 12 haryana 2184 2979 2207.85 30 tamil nadu 6299 12251 6477.56 13 himachal pradesh 767 951 772.52 31 telengana 4442 1574 4355.96 14 jammu and kashmir 1428 1920 1442.76 32 tripura 45 387 55.26 15 jharkhand 1289 1049 1281.8 33 uttarakhand 9581 8558 9550.31 16 karnataka 8790 12158 8891.04 34 uttar pradesh 2406 1602 2381.88 17 kerala 67492 3416 65569.7 35 west bengal 7223 10026 7307.09 18 ladakh 104 128 104.72 ijahp article: garg, sharma, jain/an ahp-based optimal distribution model and its application in covid-19 vaccination international journal of the analytic hierarchy process 214 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.863 table 2 optimal covid-19 vaccine distribution for 𝒓 = 𝟐, 𝟑 𝒂𝒏𝒅 𝑽 = 𝟏. 𝟓, 𝟐, 𝟐. 𝟓 𝒂𝒏𝒅 𝟒. 𝟓 (in lakh) with state preference weight and rank s.no (𝑖) state or union territory preference weight (𝑣𝑖 ) rank optimal vaccine distribution with 𝑟 = 2 and optimal vaccine distribution with 𝑟 = 3 and 𝑉 = 1.5 lakh 𝑉 = 2 lakh 𝑉 = 2.5 lakh 𝑉 = 4.5 lakh 𝑉 = 1.5 lakh 𝑉 = 2 lakh 𝑉 = 2.5 lakh 𝑉 = 4.5 lakh 1 andaman and nicobar islands 0.000112362 34 16 22 28 50 15 21 27 48 2 andhra pradesh 0.01136795 16 1704 2272 2840 5114 1704 2271 2841 5115 3 arunachal pradesh 0.000308802 32 46 60 76 138 45 60 75 138 4 assam 0.007746692 19 1162 1548 1936 3486 1161 1548 1935 3486 5 bihar 0.018912875 12 2836 3782 4728 8510 2835 3780 4728 8508 6 chandigarh 0.001297589 26 194 258 324 582 192 258 324 582 7 chhattisgarh 0.033038492 7 4954 6606 8258 14866 4953 6606 8259 14865 8 dadra and nagar haveli and daman and diu 0.000051967 35 6 10 12 22 6 9 12 21 9 delhi 0.014689947 13 2202 2936 3672 6610 2202 2937 3672 6609 10 goa 0.004177789 22 626 834 1044 1880 624 834 1044 1878 11 gujarat 0.032344317 8 4850 6468 8086 14554 4851 6468 8085 14553 12 haryana 0.010693051 17 1602 2138 2672 4810 1602 2136 2673 4809 13 himachal pradesh 0.003741466 23 560 748 934 1682 561 747 933 1683 14 jammu and kashmir 0.00698757 20 1048 1396 1746 3144 1047 1395 1746 3144 15 jharkhand 0.006208009 21 930 1240 1552 2792 930 1239 1551 2793 16 karnataka 0.043061051 4 6458 8612 10764 19376 6459 8610 10764 19377 17 kerala 0.317567018 1 47634 63512 79390 142904 47634 63513 79389 142905 18 ladakh 0.00050718 29 76 100 126 228 75 99 126 228 19 madhya pradesh 0.033227571 6 4984 6644 8306 14952 4983 6645 8304 14952 20 maharashtra 0.252318511 2 37846 50462 63078 113542 37845 50463 63078 113541 21 manipur 0.002101318 24 314 420 524 944 315 420 525 945 22 meghalaya 0.000777285 28 116 154 194 348 114 153 192 348 23 mizoram 0.000419421 31 62 82 104 188 60 81 102 186 24 nagaland 0.000501368 30 74 100 124 224 75 99 123 225 25 odisha 0.009256462 18 1388 1850 2314 4164 1386 1851 2313 4164 26 puducherry 0.001474172 25 220 294 368 662 219 294 366 663 27 punjab 0.013664497 14 2048 2732 3416 6148 2049 2730 3414 6147 ijahp article: garg, sharma, jain/an ahp-based optimal distribution model and its application in covid-19 vaccination international journal of the analytic hierarchy process 215 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.863 28 rajasthan 0.026744517 10 4010 5348 6686 12034 4011 5346 6684 12033 29 sikkim 0.000784646 27 116 156 196 352 117 156 195 351 30 tamil nadu 0.031372094 9 4704 6274 7842 14116 4704 6273 7842 14115 31 telengana 0.021096769 11 3164 4218 5274 9492 3162 4218 5274 9492 32 tripura 0.000267635 33 40 52 66 120 39 51 66 120 33 uttarakhand 0.046254025 3 6938 9250 11562 20814 6936 9249 11562 20814 34 uttar pradesh 0.011535912 15 1730 2306 2882 5190 1728 2307 2883 5190 35 west bengal 0.03538967 5 5308 7076 8846 15924 5307 7077 8847 15924 for the third step, the authors performed the pair wise comparison by 𝑓𝑖𝑗 = 𝑅𝑖 𝑅𝑗 ⁄ for every possible pair (𝑖, 𝑗); 𝑖, 𝑗 =1,2…35 using the calculated 𝑅𝑖 value. the computational work was done with microsoft excel and statistical software for social sciences (spss) to formulate a 35 order pcm [𝑓𝑖𝑗 ]. for the fourth step, the gmm (crawford & williams, 1985) was applied over the pcm[𝑓𝑖𝑗 ] to calculate the state and u.t preference weights (𝑣𝑖 ). for the fifth step, an optimal covid-19 vaccine distribution was calculated, for 𝑟 = 2 with 𝑉 = 1.5, 2, 2.5 𝑎𝑛𝑑 4.5𝑙𝑎𝑘ℎ, and for 𝑟 = 3 with𝑉 = 1.5, 2, 2.5 𝑎𝑛𝑑 4.5 𝑙𝑎𝑘ℎ, using the mathematical formula 𝑟. ⌊ 𝑉.𝑣𝑖 𝑟 ⌋ as mentioned in step 5. when vaccine availability fell short of the requirement, an optimal distribution model, that was capable of ensuring homogeneity among the states and u.ts for different 𝑉and 𝑟 values was required. it was found that the model saved a total of 34, 40, 30, 38 and 54, 54, 46, 48 vaccine doses from overlapping for 𝑟 = 2 with 𝑉 = 1.5, 2, 2.5 𝑎𝑛𝑑 4.5 lakh and 𝑟 = 3 with 𝑉 = 1.5, 2, 2.5 𝑎𝑛𝑑 4.5 lakh availability respectively, which are able to be reassigned. the integrated model outcomes including the optimal covid-19 vaccine distribution for 𝑟 = 2,3 𝑎𝑛𝑑 𝑉 = 1.5,2,2.5 𝑎𝑛𝑑 4.5 𝑙𝑎𝑘ℎvaccine availability and the state preference weight(𝑣𝑖 ) are represented in table2. 4. conclusions covid-19 is one of the most lethal pandemics ever faced by humankind, and is producing a large number of tedious management problems. because of this, the authors observed that a multipurpose distribution model that is able to compile a large number of critical factors according to the requirements of a problem under study is required to deal with this problem. in this work, the authors developed an ahp-based multi-purpose distribution model to ensure optimal drug/vaccine distribution. the proposed model used a mathematical function for pairwise comparison to construct a pcm and eliminated any possible error caused by using linguistic judgment scales. it ensures perfect consistency to avoid any conflict that results from using different ahp analysis methods. real-time distribution optimality was achieved by mathematically embedding the fact that exactly 𝑟 vaccine doses are required to complete a vaccination schedule for one patient; therefore, avoiding overlapping of vaccine doses in every state or u.t. ijahp article: garg, sharma, jain/an ahp-based optimal distribution model and its application in covid-19 vaccination international journal of the analytic hierarchy process 216 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.863 the proposed model was applied in a systematic manner based on reported covid-19 data for 28 indian states and 7 u.ts to provide an optimal distribution of limited covid19 vaccine doses. the state and u.t preference weight values(𝑣𝑖 )were calculated using the geometric mean ahp analysis method (crawford & williams, 1985) and are ranked from 1 to 35 for a greater to lesser 𝑣𝑖 value. the computational results from the proposed model along with optimal vaccine distribution that was subject to varying vaccine availability(𝑉) along with state-wise preference weights and ranks are shown in table 2. in the future, the proposed distribution model can be used as a multipurpose mcdm tool to solve various management problems due to its methodological flexibility. the model can be applied to design optimal vaccine distribution in smaller territorial areas such as at the district level within a state/u.t by considering districts as the decision elements of the problem, or at the vaccination center level within a city by considering vaccination centers as the decision elements of the problem. the model can also be generalized from the state level to the country level. moreover, the distribution model is flexible enough to use varying vaccine availability or vaccine production capacity, and to provide an optimal vaccine distribution in milligrams or other si units. the authors also suggest using the proposed model to design a vaccine storage or inventory model to provide optimal use of a vaccine storage center to cover nearby territories. ijahp article: garg, sharma, jain/an ahp-based optimal distribution model and its application in covid-19 vaccination international journal of the analytic hierarchy process 217 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.863 references chan, j. f. w., yuan, s., kok, k. h., to, k. k. w., chu, h., yang, j., ... & yuen, k. y. (2020). a familial cluster of pneumonia associated with the 2019 novel coronavirus indicating person-to-person transmission: a study of a family cluster. the lancet, 395(10223), 514-523. doi: https://doi.org/10.1016/s0140-6736(20)30154-9 corvin, j. a., chan, i., aguadoloi, c. x., dollman, i., & gonzales, j. (2020). analytic hierarchy process: an innovative technique for culturally tailoring evidence‐based interventions to reduce health disparities. health expectations, 24(s1),70-81. doi:https://doi.org/10.1111/hex.13022 crawford, g., & williams, c. (1985). a note on the analysis of subjective judgment matrices. journal of mathematical psychology, 29(4), 387-405. doi: https://doi.org/10.1016/0022-2496(85)90002-1 covid19india dashboard.daily update on covid-19 in india. available at https://www.covid19india.org/. garg, a., & ganesh, t. (2020). an analytical hierarchy process approach for covid-19 risk assessment study amid the latest re-open and unlock phase in india. international journal of the analytic hierarchy process, 12(3), 656-576. doi: https://doi.org/10.13033/ijahp.v12i3.814 halder, b., bandyopadhyay, j., &banik, p. (2020). assessment of hospital sites’ suitability by spatial information technologies using ahp and gis-based multi-criteria approach of rajpur–sonarpur municipality. modeling earth systems and environment, 6(4), 2581-2596. doi: https://doi.org/10.1007/s40808-020-00852-4 hezam, i. m., nayeem, m. k., foul, a., &alrasheedi, a. f. (2021). covid-19 vaccine: a neutrosophic mcdm approach for determining the priority groups. results in physics, 20, 103654. doi: https://doi.org/10.1016/j.rinp.2020.103654 improta, g., perrone, a., russo, m. a., & triassi, m. (2019). health technology assessment (hta) of optoelectronic biosensors for oncology by analytic hierarchy process (ahp) and likert scale. bmc medical research methodology, 19(1), 1-14. doi: https://doi.org/10.1186/s12874-019-0775-z kaur, s. p., & gupta, v. (2020). covid-19 vaccine: a comprehensive status report. virus research, 198114. doi:https://doi.org/10.1016/j.virusres.2020.198114 lancet, t. (2020).global governance for covid-19 vaccines. lancet, 395(10241), 1883. doi: 10.1016/s0140-6736(20)31405-7 ministry of health and family welfare (mohfw) government of india. covid-19 vaccine operational guidelines, 61. available at https://www.mohfw.gov.in/pdf/covid19vaccineog111chapter16.pdf https://doi.org/10.1016/s0140-6736(20)30154-9 https://doi.org/10.1111/hex.13022 https://doi.org/10.1016/0022-2496(85)90002-1 https://www.covid19india.org/ https://doi.org/10.13033/ijahp.v12i3.814 https://doi.org/10.1007/s40808-020-00852-4 https://doi.org/10.1016/j.rinp.2020.103654 https://doi.org/10.1186/s12874-019-0775-z https://doi.org/10.1016/j.virusres.2020.198114 https://dx.doi.org/10.1016%2fs0140-6736(20)31405-7 https://www.mohfw.gov.in/pdf/covid19vaccineog111chapter16.pdf ijahp article: garg, sharma, jain/an ahp-based optimal distribution model and its application in covid-19 vaccination international journal of the analytic hierarchy process 218 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.863 ministry of health and family welfare (mohfw) government of india. letter from additional secretary mohfw regarding vaccine roll out. available at https://www.mohfw.gov.in/pdf/letterfromaddlsecymohfwregcontraindicationsandfac tsheetforcovid19vaccines.pdf mishra, a.m., purohit, s.d., owolabi, k.m. & sharma, y.d. (2020). a nonlinear epidemiological model considering asymptotic and quarantine classes for sars cov-2 virus. chaos, solitons & fractals, 138, 109953.doi: https://doi.org/10.1016/j.chaos.2020.109953 mohammed, t. j., albahri, a. s., zaidan, a. a., albahri, o. s., al-obaidi, j. r., zaidan, b. b., ... & hadi, s. m. (2021). convalescent-plasma-transfusion intelligent framework for rescuing covid-19 patients across centralised/decentralised telemedicine hospitals based on ahp-group topsis and matching component. applied intelligence, 1-32. doi:https://doi.org/10.1007/s10489-020-02169-2 population of india (2021). available at https://www.worldometers.info/world-population/india-population/ rosenbloom, e. s. (1997). a probabilistic interpretation of the final rankings in ahp. european journal of operational research, 96(2), 371-378. doi: https://doi.org/10.1016/s0377-2217(96)00049-5 saaty, t. l. (1977). a scaling method for priorities in hierarchical structures. journal of mathematical psychology, 15(3), 234-281. doi: https://doi.org/10.1016/00222496(77)90033-5 saaty, t. l. (1990). how to make a decision: the analytic hierarchy process. european journal of operational research, 48(1), 9-26. doi: https://doi.org/10.1016/03772217(90)90057-i saaty, t. l. (2003). decision-making with the ahp: why is the principal eigenvector necessary. european journal of operational research, 145(1), 85-91. doi: https://doi.org/10.1016/s0377-2217(02)00227-8 saaty, r. w. (1987). the analytic hierarchy process—what it is and how it is used. mathematical modelling, 9(3-5), 161-176.doi:https://doi.org/10.1016/02700255(87)90473-8 worldometers.covid-19 corona virus pandemic. available at https://www.worldometers.info/coronavirus/#countries world health organization (who). steps in vaccine development. available at https://www.who.int/docs/default-source/coronaviruse/risk-comms-updates/update45vaccines-developement.pdf?sfvrsn=13098bfc_5 https://www.mohfw.gov.in/pdf/letterfromaddlsecymohfwregcontraindicationsandfactsheetforcovid19vaccines.pdf https://www.mohfw.gov.in/pdf/letterfromaddlsecymohfwregcontraindicationsandfactsheetforcovid19vaccines.pdf https://doi.org/10.1016/j.chaos.2020.109953 https://doi.org/10.1007/s10489-020-02169-2 https://www.worldometers.info/world-population/india-population/ https://doi.org/10.1016/s0377-2217(96)00049-5 https://doi.org/10.1016/0022-2496(77)90033-5 https://doi.org/10.1016/0022-2496(77)90033-5 https://doi.org/10.1016/0377-2217(90)90057-i https://doi.org/10.1016/0377-2217(90)90057-i https://doi.org/10.1016/s0377-2217(02)00227-8 https://doi.org/10.1016/0270-0255(87)90473-8 https://doi.org/10.1016/0270-0255(87)90473-8 https://www.worldometers.info/coronavirus/#countries https://www.who.int/docs/default-source/coronaviruse/risk-comms-updates/update45-vaccines-developement.pdf?sfvrsn=13098bfc_5 https://www.who.int/docs/default-source/coronaviruse/risk-comms-updates/update45-vaccines-developement.pdf?sfvrsn=13098bfc_5 ijahp article: garg, sharma, jain/an ahp-based optimal distribution model and its application in covid-19 vaccination international journal of the analytic hierarchy process 219 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.863 world health organization (who).coronavirus disease 2019(covid-19) situation report –94. available at https://www.who.int/docs/default-source/coronaviruse/situationreports/20200423-sitrep-94-covid-19.pdf world health organization (who). coronavirus disease 2019 (covid-19) situation report–11. available at https://www.who.int/docs/default-source/coronaviruse/situationreports/20200131-sitrep-11-ncov.pdf?sfvrsn=de7c0f7_4 world health organization (who).draft-landscape-of-covid-19-candidate-vaccines on 16 feb. 2021. available at https://www.who.int/publications/m/item/draft-landscape-ofcovid-19-candidate-vaccines https://www.who.int/docs/default-source/coronaviruse/situation-reports/20200423-sitrep-94-covid-19.pdf https://www.who.int/docs/default-source/coronaviruse/situation-reports/20200423-sitrep-94-covid-19.pdf https://www.who.int/docs/default-source/coronaviruse/situation-reports/20200131-sitrep-11-ncov.pdf?sfvrsn=de7c0f7_4 https://www.who.int/docs/default-source/coronaviruse/situation-reports/20200131-sitrep-11-ncov.pdf?sfvrsn=de7c0f7_4 https://www.who.int/publications/m/item/draft-landscape-of-covid-19-candidate-vaccines https://www.who.int/publications/m/item/draft-landscape-of-covid-19-candidate-vaccines ijahp article: pereira, botter, robles/port terminal in the northern region of brazil: decision upon public port or private use terminal international journal of the analytic hierarchy process 195 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.617 port terminal in the northern region of brazil: decision about public port or private use terminal felipe george gomes pereira university of sao paulo brazil eng.felipegeorge@gmail.com rui carlos botter university of sao paulo rcbotter@usp.br leo tadeau robles federal university of maranhao leotadeurobles@uol.com.br abstract brazil plays an important role in the world’s trade of agricultural grains, notably soybeans. recently, brazilian production of soybeans has moved to the central-western and northern regions, thus creating a demand for new outflow routes for export. this paper discusses the decision factors for the implementation of specialized port terminals located in the northeast and north of the country, specifically in the area known as arco norte. as a methodology, the analytic hierarchy process (ahp) decision tool is used. this involves applying a questionnaire to port experts and managers to identify and classify decision criteria for investments in the possible alternatives available in brazil’s regulatory legislation: leasing of structures in a public port terminal or implementing a private terminal. the results show port location (54%) as the most relevant criteria, followed by economic and financial (27.8%), performance and dimensioning (11.2%), and contract and legislation (6.9%) criteria. the study pointed to the investment priority the mode private use terminals, which is confirmed by the country’s recent investment in port terminals which has been effective. keywords: soybean export logistics; multicriteria analysis; port planning; brazilian ports 1. introduction in brazil, agribusiness has been responsible for increasing the country's participation in global markets and this product group includes the country’s most important export contributor, responsible for 44% of total brazilian exports in 2017 (cepea, 2017). the main agricultural products exported are soybeans, corn, red and white meat, sugarcane, and coffee. the latter are the country’s more traditional exports. mailto:eng.felipegeorge@gmail.com mailto:rcbotter@usp.br ijahp article: pereira, botter, robles/port terminal in the northern region of brazil: decision upon public port or private use terminal international journal of the analytic hierarchy process 196 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.617 this paper focuses on soybean production and exportation, the recent production trend moving to brazil’s central-western and northern regions, and the consequent requirement for new alternatives to the traditional export routes of the national ports, santos in são paulo state and paranaguá, paraná state respectively in the south-eastern and southern regions (conab, 2017). however, the new port linkages have highlighted operational limitations for the export flow, mainly due to inadequate and inefficient land transport and local port infrastructure (cnt, 2015). hibernon et al. (2016) pointed out that, despite the remarkable expansion of soybean production and exportation, the logistical cost is still a problem that must be solved in order to increase brazilian competitiveness in the global market. in brazil, the port sector, as other infrastructure sectors, is regulated by the government and the services are provided by the private players. brazil’s law n. 12815/2013, regulated by the decree n. 9048/2017, constitutes the legal framework reinforcing and stimulating the private participation in ports operation (brasil, 2013; 2017). as galvão et al. (2017) observed, brazilian legislation defines the concepts of organized port (port perimeter under port authority management); port for public use (public port, pp); and terminals for private use (tups), among which the main differences are the type of cargo handled, land ownership (privately owned or publicly rented), government authorization, and conditions of employing the dock workers. in recent years, the tendering process for public ports on leased areas within the public organized ports has faced a lack of interest from private parties. for example, in 2013, a first round of bidding regarding the arco norte logistic corridor was unsuccessful, leading to its cancellation, as well as the subsequent tender processes related to public port areas leasing and operation port areas. this demonstrates private investors' lack of confidence in the present port regulations framework for investing in port facilities. in this sense, this paper analyzes the attributes that support private decisions to become involved in public use port terminals (pp) or private use terminals (tup) by applying the analytic hierarchy process (ahp) method. the ahp is used to identify and evaluate the main criteria adopted by decision makers to choose a port investment modality, regarding the alternatives of an area and infrastructure leasing in an organized public port, as opposed to implementation of a private use terminal. the study approached the information to support investment optimization in order to meet the current and future requirements of the brazilian logistics system by identifying the relevant decisionmaking factors and their differences applied to public and private use terminals. this paper presents a brief analysis of the soybean business in brazil and its logistics requirements, describes the characteristics of the arco norte logistics corridor, presents a review of port competitiveness and performance indicators, describes the use of the ahp method and its application to decision making on investment in public or private ports in brazil, and then analyzes the results and presents the paper’s conclusions, limitations, and contributions. ijahp article: pereira, botter, robles/port terminal in the northern region of brazil: decision upon public port or private use terminal international journal of the analytic hierarchy process 197 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.617 2. soybean business in brazil soybean culture is important in brazilian agribusiness and in the country's economy, especially with regard to exports. however, the country has a weakness related to logistical constraints in grain transport to export ports (cutrim et al., 2015). in 2015, a study presented by a private entity, the cnt (national transportation confederation) focusing on the soybean and corn logistics chains emphasized the requirement of modernization, expansion, and interconnection of transport modes to enhance logistics efficiency and reliability for product flows from their true origin to destination. this will require greater integration, including improved warehousing facilities and a betterbalanced modal transportation split in order to support the country's current grain production and its future expansion (cnt, 2015). historically, until the 1990s, national soybean production was carried out in brazil’s southern region. in more recent years, production has moved to the central-western region, mainly due to the existence of cheaper land, the adoption of new production technologies, and a migration of traditional and skilled producers from the south. the same process has occurred in the direction of the northern and north-eastern regions, with producers searching to fulfill increased demand, further stimulated by reasonable product prices (conab, 2017). soybean production in brazil increased by 10.3% between 2000/01 and 2014/15 due to land expansion and productivity that has increased up to 3.0 ton/ha, the world’s second largest for soybean cultivation. the european and asian markets are the most significant destination of brazilian soybeans, with china, due to its increasing demand, the main trading partner, consuming 62.9% of brazil’s soybean exports (cnt, 2015). in 2017, brazil produced almost 115 million tons of soybeans, with the central-western region, comprising the states of mato grosso, mato grosso do sul, and goiás, accounting for 50.9 million tons (44.3%) and paraná, rio grande do sul, and santa catarina states (southern region) accounting for 40.9 million tons (35.6%) (ibge, 2017). almost half of the country's soybean production is destined for export, highlighting brazil’s position as an important international economic player. brazil, argentina, and the u.s.a. are the main world soybean exporters with a total of 18.5 million tons in 2017/18 (usda, 2018). figure 1 shows the main brazilian ports and a representation of the arco norte (“north bow”) ports corridor, a hypothetical line linking the northern and north-eastern brazilian ports, possible alternatives for soybean exports. ijahp article: pereira, botter, robles/port terminal in the northern region of brazil: decision upon public port or private use terminal international journal of the analytic hierarchy process 198 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.617 figure 1 brazilian main ports source: adapted from mtpa, 2018 when discussing the potential difficulties for soybean exportation from the ports of the northern region, hibernon filho et al. (2016) confirmed the current lack of port capacity to meet the regional export demand, which represents a significant concern considering the forecasted production and export growth. they also pointed out that brazil’s apparent institutional and political impasse could cause institutional instability, reducing investment in public or private ports which requires major investments with long-term returns. thus, institutional and legal security is essential. the brazilian ministry of agriculture (mapa) forecasts that the area planted with soybeans will reach 43.2 million ha in 2026, an increase of 10.0 million ha over the next ten years. the production projections for 2025/2026 indicate a production of 129.2 million tons, a 35.1% increase over 2015/16. domestic consumption is predicted to increase by 22.6%, reaching 53.4 million tons. thus, approximately 75.8 million tons will be available for exportation, and with stocks carried over this figure is predicted to reach a possible 96.8 million tons of soybeans exported in 2025/26 (see table 1). ijahp article: pereira, botter, robles/port terminal in the northern region of brazil: decision upon public port or private use terminal international journal of the analytic hierarchy process 199 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.617 table 1 brazil: soybean production, consumption and export projections (thousand tons) source: mapa, 2017 year production consumption exports projection upper l. projection upper l. projection upper l. 2015/16 95,631 43,600 55,350 2016/17 100,783 110,264 43,001 47,135 57,602 63,550 2017/18 103,228 114,837 44,970 50,815 59,891 68,227 2018/19 106,866 121,169 46,032 52,099 62,161 72,432 2019/20 109,877 126,301 47,094 53,376 64,431 76,291 2020/21 113,162 131,578 48,156 54,645 66,701 79,961 2021/22 116,339 136,517 49,218 55,908 68,972 83,497 2022/23 119,562 141,385 50,280 57,165 71,242 86,931 2023/24 122,975 146,110 51,342 58,417 73,512 90,285 2024/25 125,975 150,755 52,404 59,663 75,512 93,573 2025/26 129,181 155,316 53,466 60,905 78,053 96,805 the soybean export logistics chain is comprised of the collection from rural production sites, an intermediate warehousing phase, and transportation to export ports. this transportation is mainly by highways, and to a lesser extent by railways, and much less by waterways with, of course, dependence on the possibility of multimodal moving. presently, the production has to travel long distances with significant transit time from the pick-up of goods by the carriers (mainly trucks) to their arrival at the destination. brazil’s transportation matrix is mainly based on road transport, which demonstrates a remarkable imbalance. for example, at port of santos (south-eastern region), the main brazilian port, no more than 27% of its movements use the railway mode, with the exception of the agribusiness sector (sugarcane, soybeans, and corn) with 53% (porto de santos, 2018). in general terms, the railways in brazil almost exclusively transport commodities destined for export (mainly iron ore and agricultural products). the logistics infrastructure for agricultural commodities’ outflow must also consider the availability of warehouses for storing and treating the harvest, as well as port structures for transshipment of cargo and loading on ships. the warehousing structure is considered a problematic and critical issue for brazilian agribusiness exports, suffering from inadequate provision and underinvestment to meet the sector’s requirements (caixeta filho, 2006). nevertheless, the transportation link determines the profitability of brazilian agribusiness. the country's transport system imposes a barrier to the exploitation of the farms’ advantages (before the gate) in grain production and, in fact, it is a major issue for competitiveness in the international markets. as previously indicated, the projected grain production growth for brazil will exceed the national infrastructure capacity resulting in consequences such as transport delays, breaches of contracts, and significant losses in the international markets. according to cnt (2015), the only alternative for improving brazil’s logistics infrastructure involves ijahp article: pereira, botter, robles/port terminal in the northern region of brazil: decision upon public port or private use terminal international journal of the analytic hierarchy process 200 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.617 increasing the level and integration of public and private investment in infrastructure. some of these required investments have already been implemented in the arco norte export logistics corridor, covering the northern and northeasters regions, as discussed in the following section. 3. arco norte export logistics corridor the search for new logistics alternatives became a priority with the geographic shift of agribusiness and the movement towards the central-western and northern regions (cnt, 2015). an advantage of the displacement of the agricultural frontier to the north is its greater proximity to the northern hemisphere, with gains possibilities in transport time and even freight volumes due to capacity expansion of the panama canal (hibernon filho et al., 2016). the arco norte logistics export corridor, also known as the arco norte system, comprises multimodal corridors, port support, and operational platforms located at porto velho, rondônia state and miritituba, pará state, both in the country’s northern region. these logistics platforms are used to transport regional grain production to the arco norte ports of itacoatiara, amazonas state; santarém, barcarena; vila do conde, pará state; itaqui, in the city of são luís, maranhão state; and santana, amapá state (movimento pró-logística do mato grosso, 2017). figure 2 shows the main arco norte system corridors; related to madeira, tapajós, and tocatins river basins. the transport infrastructure comprising these logistics corridors is still considered inadequate to link the production zones to the arco norte system ports. regarding roadways, the system suffers from a lack of maintenance and even of asphalt paving in many places. on the waterways, more signaling devices are required for navigation safety, and the rail network is dispersed and not significant. because of these conditions, long-distance cargo transportation occurs by trucks moving directly from production farms or via transshipment terminals (movimento pró-logística do mato grosso, 2017). however, despite all its deficiencies, the arco norte system has increased its participation in the national soybean and corn exports from 17.2% in 2014, to 21% in 2015, and 19% in 2016 (movimento pró-logística do mato grosso, 2017). in the amazon river basin, the main transportation mode is the waterway, supported by trucks production moving between transshipment points. ijahp article: pereira, botter, robles/port terminal in the northern region of brazil: decision upon public port or private use terminal international journal of the analytic hierarchy process 201 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.617 figure 2 arco norte export logistics corridor source: adapted from movimento pró-logística do mato grosso (2017) 4. brazilian port institutional framework (public and private) and port competitiveness issues an efficient logistics system is a basic requirement to face the commercial and competitive pressures of the global agribusiness market, and requires steady modernization and expansion of transport and port capacities. brazil, a coastal country, has been historically developed by its port infrastructure and, in recent years, port institutional regulation has been marked by significant institutional and legal changes (galvão et al., 2017). the brazilian government and its agencies have focused on stimulating private sector investment to develop the logistics and port sector. in this sense, brazilian port reform was successful in adopting a landlord model of port governance with port authorities responsible for port management and the private sector for investment and operations. however, the full implementation of this model requires further development of port business management. law 12815/2013, named the port law, confirmed the option to concede and lease land and public infrastructure in the so-called organized port (a polygon established by the law regarding public ports pp) to the private sector for cargo movement and storage operations. decree n. 9048/2017 aimed to attract more private investment to ports and regulated the terms of concession and lease agreements of terminals for public use, the extension of the term of the contracts, the possibility of expansion of the leased area, and of the period of leasing renewal. 1. madeira corridor 2. tapajós corridor 3 tocantins corridor railway waterway railway grain handling amount subtitle ijahp article: pereira, botter, robles/port terminal in the northern region of brazil: decision upon public port or private use terminal international journal of the analytic hierarchy process 202 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.617 the decree also defined new criteria for the authorization and operation of out-of-port terminals, with an extension of the deadline for operations to begin. it also provided antaq, the national regulatory agency, greater autonomy in the authorization process for private installations, and introduced the first phase of new documentation requirements in applications for authorization. previously, the concession and lease agreements term were 25 years, and only able to be renewed once for the same period. through the decree, this term became 35 years, with a limit of 70 years in total. 1 the concessionaire or lessee was previously required to formally express its interest in extending the agreement to the granting authority at least 24 months before the end of the term; now, it must express its interest at least 60 months in advance. previously, those interested in obtaining a port facility authorization had to make a request to antaq in two steps, at any time, by presenting the required documents. following antaq's approval, the interested parties would present more specific documents within ninety days. next, antaq would send the documentation to the granting authority within a period of 15 days, for analysis and to conclude the process. finally, the binding contracts would be signed. now, once the request is authorized, antaq sends the documents to the granting authority directly for contract finalization in a faster and much less bureaucratic process. previously, any expansion of the port facility into areas located outside the organized port, not exceeding 25% of the original area, would be approved by the granting authority through a new concession contract. now, the new contract does not require a new public announcement and depends only on approval by the granting authority. in addition, private terminals located within the organized port can now expand their facilities. in the sense of port investment as a tool for local, regional, and national development, this study builds on port competitiveness studies such as those conducted by tongzon (1995), malchow and kanafani (2001), nir et al. (2003), khan (2004), and bichou and gray (2004). worthy of note is a 2017 issue of the scholarly journal research in transportation business and management (rtbm), which published 21 port studies addressing port governance, organization and performance. brooks et al. (2017), the journal guest editors, identified some of the drivers of ports’ role in national development. these included private sector participation (devolution) as part of government aims to assure port conditions while making the ports more profitable and efficient, as well as “greener” and more sustainable. rodrigue and notteboom (2017) pointed out that international export and import of grain are important in international trade and have an effective role in the global economy. 1 this issue is currently under legal judgement and the matter of these discussions on government entities is far beyond this paper’s focus. ijahp article: pereira, botter, robles/port terminal in the northern region of brazil: decision upon public port or private use terminal international journal of the analytic hierarchy process 203 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.617 this study is concerned with private port operations and uses the multi-criteria analysis method to support decision making to evaluate and choose between public or private use port terminals to brazilian ports in the arco norte system. the main objective is to identify the most important criteria for the investment decision between pp or tup. 5. ahp method as a decision support tool for investment decision between public and private ports the ahp multi-criteria analysis method is based on an active weighting process in which the various relevant attributes are represented by their relative importance. this method is characterized by the division of the problem into descending hierarchical levels, starting with the global objective, criteria, sub-criteria, and alternatives at successive levels (saaty, 1996). the ahp method results are derived from pair comparison measurements using values from 1 to 9 based on the rating scale for comparative trials (saaty, 2008). pinheiro et al. (2003) observed that one of the main advantages of the application of the ahp method is to avoid subjective or biased weighting factors in multicriteria analysis to support a decision-making process and emphasized one particular issue. as is wellknown, the participants in a process could influence the output decision by interfering in the weighting of the relevant criteria. this does not imply that it is necessary to eliminate the personal factor or human intervention in political, social or other terms, rather it is necessary to establish an agreed upon basis for discussions. according to silva et al. (2007), the ahp supports the decision-making process and can lead decision makers to evaluate and select from among alternative courses of action for certain problems. to achieve this, the ahp method divides the decision problem into hierarchical levels in order to facilitate understanding and evaluation through the construction of the stages of the multi-criteria model and proposes weighting factors for each criterion. 5.1 multi-criteria hierarchical analysis applied to the port sector applications of the ahp method as a tool to support the decision-making process for port choice have been presented by song and yeo (2004), addressing chinese ports; lirn et al. (2004), looking at transshipment ports; and mazza and robles (2004), who looked at brazilian container ports. more recently, du (2014; 2015) used ahp for evaluating the performance of european container ports. nazemzadeh and vanelslander (2015) applied the ahp method to identify factors affecting port selection for northern european ports. the results identified the following port selection criteria, in decreasing order of importance: port costs, geographical location, quality of hinterland connections, productivity, and capacity. in a brazilian port case, mazza and robles (2004) used financial and operational criteria to characterize container port choice. their criteria were port tariffs, service level, operational capacity, and financial stability. the authors identified that port location is the primary factor in costs incurred by shippers, due to the relationship between the distance covered and the weight (tonnage) of the cargo transported. ijahp article: pereira, botter, robles/port terminal in the northern region of brazil: decision upon public port or private use terminal international journal of the analytic hierarchy process 204 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.617 gartner et al. (2012) presented a proposal for multi-criteria modeling through the application of ahp to port problems of regulation, planning, and management in privatized port areas, dealing with the hierarchy of areas destined for leasing investments. in addition, they observed that the method was appropriate to the port sector due to its multidisciplinary nature, which involves socioeconomic, environmental, and political value judgments. the choice of location for a port terminal expansion involves several specific attributes that, from experts’ perspectives, could be analyzed by applying the ahp method. these criteria could cover aspects related to road access, availability of area for expansion, socio-environmental impacts, and local infrastructure conditions (loureiro et al., 2015). magalhães and botter (2015) determined the ahp method to be an adequate decision support tool regarding regulatory issues in privatized port areas. the application of the ahp method to the decision-making processes in the public sector is common, as presented by pinheiro et al. (2003), although it has not been frequently used in brazilian port leasing project analysis. this paper aims to help rectify this oversight. 5.2 modeling the analytical hierarchy process this study used the ahp method as a decision support tool to evaluate and select the most viable type of port facility investment for soybean export operations in the northern region of brazil. the ahp can be justified by its potential to deal with various decision makers’ different perceptions of the relevant criteria. the objective was to identify the main criteria relevant to support decision makers in the choice of investment, considering the alternatives of public port leasing or private use terminal implementation, as allowed and established by current brazilian regulatory instruments. the ahp structure was elaborated based on four criteria and their subsequent sub-criteria (figure 3), some established by a literature review as previously mentioned in section 5.1 and others observed in brazilian port regulation, such as contract conditions, port legislation, and performance and dimensioning required by public and private ports. each criteria was evaluated in terms of its respective subcriteria, and compared with each other on a scale of relative intensity of importance ranging from 1 to 9 (saaty, 2008). the criteria "port location" relates to the availability of transportation infrastructure and superstructure, regional port availability, consideration of land and waterway access, availability of areas for future expansion, and the existence of port structures. the "economic and financial" criteria consider the determining factors for the technical, economic and financial, and environmental feasibility analysis of the terminal. "performance and dimensioning" involves the terminal’s operational efficiency which is directly related to its installed operational capacity. it considers the loading capabilities, land reception, warehousing, and annual handling capacity. the criteria "contracts and legislation" evaluates the requirements and constraints of the public bidding requirements regarding the lease of a terminal in a public port, as well as the legislation that regulates the authorization of terminals for private use, and national development plans. ijahp article: pereira, botter, robles/port terminal in the northern region of brazil: decision upon public port or private use terminal international journal of the analytic hierarchy process 205 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.617 figure 3 analytical hierarchy structure source: excerpted from expert choice software following the construction of the hierarchy, the definition of priorities and comparative judgments was carried out by the sample of decision makers and port sector experts. this group was purposefully selected and contacted by e-mail, telephone, and a social network (linkedin) to verify their willingness and availability to participate in the survey. the final decision-making group included 35 members, comprised of 19 specialists, including engineering consultants and port operators; five academic researchers and 11 professionals linked to the surveillance and regulation of waterway transport services and ports. each decision maker selected compared the criteria in a given hierarchical level as described in an online questionnaire implemented using the application google forms as observed in the appendix. once the values of the decision-makers' priority judgments were obtained, they were evaluated using a demonstration version of the software package expert choice. both demonstration and complete versions are available on the manufacturer's website (http://expertchoice.com/). the expert choice structured decision tool allows users to analyze complex problems in a clear and understandable way, to accurately measure the importance of competing objectives and criteria, to synthesize information, knowledge and judgments to conduct sensitivity analyses, to communicate clearly and to share results, to iterate with participants in the decision process when needed, and to allocate resources as required. the software was used to calculate the local average priorities and the overall priorities, ensuring the matrices normalization and the logical consistency of the judgments. however, it was difficult to obtain complete responses from all 35. despite follow-up requests, only 12 fully and effectively answered the questionnaire. it is necessary that participants be fully engaged and committed because missing data and inconsistent judgments can interfere with the analysis. the commitment of the specialists interviewed for this research was assured by the researchers through further personal contacts with the http://expertchoice.com/ ijahp article: pereira, botter, robles/port terminal in the northern region of brazil: decision upon public port or private use terminal international journal of the analytic hierarchy process 206 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.617 respondents. subsequently, the data from the final 12 participants' judgments were combined, the data normalized and its consistency verified. the pairwise comparison generates square matrices, where the number in line i and column j gives the importance of criteria ci in relation to cj. the number of judgments required to construct a generic judgment matrix a is n (n-1) / 2, where n is the number of elements belonging to this matrix. the elements of a are defined by the conditions: 𝑨 = | 1 𝑎12 ⋯ 𝑎1𝑛 1/𝑎21 1 ⋯ 𝑎2𝑛 ⋮ 1/𝑎 𝑛1 ⋮ 1/𝑎 𝑛2 ⋯ ⋮ 1 | conditions:  𝑎𝑖𝑗 > 0 → 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒  𝑎𝑖𝑗 = 1 ∴ 𝑎𝑗𝑖 = 1  𝑎𝑖𝑗 > 1 𝑎𝑗𝑖 → 𝑅𝑒𝑐𝑖𝑝𝑟𝑜𝑐𝑎𝑙  𝑎𝑖𝑗 = 𝑎𝑗𝑖 . 𝑎𝑗𝑖 → consistent the normalized matrix was calculated by the sum of the elements of each column and dividing each element by the respective column summation. the calculation of priorities is represented by the arithmetic mean of the standard matrix lines which represents the priority of each element. the pairwise comparison of the alternatives related to a criteria of a hierarchical level immediately superior will have a vector of weights w = (w1, w2,.., wn). the priorities can be calculated through the product of all intermediate priorities from the lowest hierarchical level to the highest. the logical consistency is calculated through the consistency index (ci) of subjective character, composed for each alternative based on preferences derived from the comparison matrix. the ci evaluates the degree of inconsistency of the matrix of parity judgments, through the equation ci = (λmax n) / (n 1) where n is the order of the matrix and λmax is the highest eigenvalue of parity judgments matrix. the consistency ratio (cr) allows the inconsistency to be evaluated according to the order of the judgment matrix, through the equation cr = ci / ri, where ri is the random index. the ri is the consistency index obtained for a reciprocal random matrix, with nonnegative elements, for varied matrix sizes n. a cr less than 0.10 is acceptable. for values of cr> 0.10 a revision in the comparison matrix is suggested. table 2 summarizes the results obtained for the priorities for each criteria analyzed, aiming to identify the most relevant criteria for decisions applied to public and private ports investment alternatives. ijahp article: pereira, botter, robles/port terminal in the northern region of brazil: decision upon public port or private use terminal international journal of the analytic hierarchy process 207 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.617 table 2 brazil: soybean production, consumption and export projections (thousand tons) source: excerpted from expert choice software level 1 objective level 2 criteria level 3 – sub-criteria identify most important criteria for choosing investment between public use terminal or private use terminal port location (54,1%) road access (37,3%) rail access (30,1%) waterway access (18,3%) expansion area availability (9,1%) infra and superstructure availability (5,3%) economic and financial (27,8%) capex (52,3%) revenues and costs (28,5%) financing (10,7%) wacc (8,5%) performance and dimensioning (11,2%) berth loading capacity (54,7%) land reception capacity (25,5%) storage capacity (11,4%) annual handling capacity (8,5%) contracts and legislation (6,9%) using time (57,5%) grant amount $ (23,9%) port fees (11,9%) political and social planning (6,7%) in the port location criteria (figure 4), the importance of the sub-criteria road access and rail access were highlighted, with 37.3% (0.373) and 30.1% (0.301), respectively. the other sub-criteria add up to an importance of 32.7% (0.337), with the infra and super structure availability subcategory, at 5.3% (0.053), as the least important. figure 4 criteria "port location" priorities from expert choice the results of the analysis of the economic and financial criteria (figure 5) show the predominance of the capex (capital expenditure) sub-criteria at 52.3% (0.523), followed by revenues and costs 28.5% (0.285), financing 10.7% (0.107), and wacc (weighted average cost of capital) 8.5% (0.085). capex, capital needed to invest for terminal construction and implementation, is an essential variable in the terminal feasibility analysis to be performed in the project phase. other sub-criteria, although important as indicators to verify the terminal viability, were evaluated with low importance in relation to the previous sub-criteria. ijahp article: pereira, botter, robles/port terminal in the northern region of brazil: decision upon public port or private use terminal international journal of the analytic hierarchy process 208 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.617 figure 5 criteria "economic and financial" priorities from expert choice the terminal efficiency analyzed by the performance and scalability criteria (figure 6) had its importance in the loading capacity demonstrated by a 54.7% (0.547) index, higher than the other sub-criteria. the sub-criteria annual movement capacity at 8.5% (0.085) was the least important, although intrinsically related to cargo loading capacity for exporting vessels, it also depends on the efficiency of other land-based and warehousing processes. figure 6 criteria "performance and dimensioning" priorities from expert choice the analysis of the criteria contract and legislation (figure 7), which covers aspects of terminal leasing at the public port and authorization of the private port, demonstrates the use period sub-criteria, at 57.5% (0.575), as the most relevant. note that, following recent changes in the law regulating the operation of public and private terminals, the leasing term of a public terminal, previously 25 years, has been extended to 35 years. for terminals for private use, the operating authorization term remains 25 years. the sub-criteria concession value 23.9% (0.239), port fees 11.9% (0.119), and political and social planning 6.7% (0.067) express lower relevance than the term of use of the terminal. however, these sub-criteria are also significant for terminal economic feasibility and may have different interpretations for each terminal model. figure 7 criteria "contract and legislation" priorities from expert choice the interaction between all criteria judgments determines the final priority of each in relation to the objective. the final priority will be determined by the sum of the products multiplied by the criteria weighting combined with the sub-criteria priority weightings. ijahp article: pereira, botter, robles/port terminal in the northern region of brazil: decision upon public port or private use terminal international journal of the analytic hierarchy process 209 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.617 global priorities can be determined as the product of all intermediate priorities from the lowest hierarchical level to the highest. figure 8 shows that the criteria of greatest importance to the overall objective was port location 54.1% (0.541), followed by the economic and financial criteria at 27.8% (0.278); performance and dimensioning 11.2% (0.112); and contract and legislation 6.9% (0.069). figure 8 global priorities from expert choice the sub-criteria priorities most relevant to the overall objective are road access with 23.4% (0.234), rail access with 18.9% (0.189), capex with 12.1% (0.121), and waterway access with 11.5% (0.115). when combined, these contribute 65% of the total importance for the decision-making process as analyzed. the combined synthesis of priorities with respect to identifying the most important criteria for choosing investments between public use terminals and private use terminals is shown in the figure 9. it is noted that the sub-criteria road access with 20.2%, rail access with 16.3% and capex with 14.6% are the most important in relation to the overall objective. the transport of commodities by roads and highways is predominant in brazil, thus it is expected to have a strong influence in the decisions of investment in terminal ports. figure 9 combined synthesis of priorities from expert choice 54.1% 27.8% 11.2% 6.9% port location economic and financial performance and dimensioning contracts and legislation ijahp article: pereira, botter, robles/port terminal in the northern region of brazil: decision upon public port or private use terminal international journal of the analytic hierarchy process 210 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.617 finally, the decision makers were individually asked their preference for the type of port facility model they thought was most appropriate for investing in to facilitate grain exports in the north and northeast routes of brazil (the arco norte system). this question was not associated with the ahp method, whose objective was to identify the most relevant criteria in the choice of investment in port terminals; however, it was done to verify these experts’ preferences to reflect the current trend for private ports in the region. all of the respondents opted for private use port terminals. this unanimous decision in favor of tups may be associated with brazilian institutional and legal uncertainties due to the political situation presently faced by the country. in port concessions, the bidders are usually provided with a basic terminal infrastructure, ensuring that after the bidding process they can begin operations in a short period of time. however, the terminals bids related to the arco norte system are greenfield projects, that is, they do not have any existing port infrastructure. therefore, the winning bidder would be responsible for all terminal structures, including the environmental liabilities, and should also guarantee to the granting authority the return of all facilities at the end of the concession. on the other hand, once a tup is authorized, the entire port undertaking will be under the control of the investors. another issue identified in interviews with the respondents was the lack of complete information in public notices which is necessary for the full understanding of the object being tendered, as a factor for the decisions in favor of private terminals. this information gap often prevents a precise definition of the project’s unique characteristics which are important for planning and determination of implementation costs and revenues, and thus for the terminal feasibility analysis. at the time of writing, none of the published tenders relating to area and port infrastructure leasing in arco norte that have focused on the movement of grains have received interest from investors, demonstrating their feelings of insecurity regarding tups. however, subsequent to law 12815/2013, only tups have been authorized by antaq. 6. conclusions the literature review and data gathered demonstrate the importance of agribusiness in general and the soybean export trade in particular to the brazilian economy. nevertheless, a remarkable need for investment in logistics and port sector development to allow greater competitiveness for brazilian products on the international market remains. despite deficiencies in the transport infrastructure linking production zones and exporting ports, the arco norte system has increased its share of domestic exports. the question of the best way to maintain private investment continuity in the port sector remains, especially regarding the alternatives of investment in terminals in pp or tups. the application of ahp to assess the decision factors identified the criteria of port localization, at 54%, as most important. the important sub-criteria of port location were identified as the availability of road and rail access. the low density and extent of the railway network and the poor utilization of the waterways in the export routes result in the predominant use of the road modality. in addition, assessment of the economic and ijahp article: pereira, botter, robles/port terminal in the northern region of brazil: decision upon public port or private use terminal international journal of the analytic hierarchy process 211 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.617 financial criteria established the priority of the sub-criteria capex and revenues and costs. the use of the ahp multi-criteria analysis methodology was a study objective and it can be considered accomplished. in addition, the main criteria for choosing an investment model were identified and the researchers concluded that the ahp method offered decision makers and interested parties a tool for analysis and choice between the alternatives of a public or private terminal. moreover, interviews with the respondents found that 100 % of them would prefer to invest in private use terminals. this result is in accordance with the current situation in brazil. thus, the ahp method is recommended for future studies analyzing multiple alternatives of public and private terminals. the tool can be further extended to terminals specializing in the movement of the different types of cargo. in addition, it is suggested that the technique be applied with a larger number of professionals, including interested investors. ijahp article: pereira, botter, robles/port terminal in the northern region of brazil: decision upon public port or private use terminal international journal of the analytic hierarchy process 212 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.617 references bichou, k., & gray, r. (2004). a logistics and supply chain management approach to port performance measurement. maritime policy and management, 1, 47-67. doi: https://doi.org/10.1080/0308883032000174454 brasil (2013). presidência da república. lei n. 12815 from june, 5, 2013. brasil (2017). presidência da república. decreto n. 9048, may, 2017. brooks, m.r., culinane, k.p.b., & pallis, a.a. (2017). revisiting port governance and port reform: a multi-country examination. research in transportation business and management. 22, 1-10. doi: https://doi.org/10.1016/j.rtbm.2017.02.005 caixeta filho, j.v. (2006). novos corredores devem mudar matriz de transporte. visão agrícola, ano, 3(5), esalq/usp. cepea (centro de estudos avançados em economia aplicada – esalq/usp) (2017). índices de exportação do agronegócio. cnt (confederação nacional do transporte) (2015). transporte e desenvolvimento. entraves logísticos ao escoamento de soja e milho. conab (companhia nacional de abastecimento) (2017). estimativa do escoamento das exportações do complexo soja e milho pelos portos nacionais safra 2016/17 (compêndio de estudos, 6). cutrim, s.s., robles, l.t., & pereira, n.n. (eds) (2015). tópicos estratégicos portuários volume i. sao luis: edufma. du, t. (2014/2015). ahp analysis for evaluation of european container port performance. msc dissertation. erasmus university rotterdam. galvão, c.b., robles, l.t., & guerise, l.c. (2017). 20 years of port reform in brazil: insights into the reform process. research in transportation business & management, 22, 153–160. doi: https://doi.org/10.1016/j.rtbm.2017.01.002 gartner, i.r., rocha, c.h., & granemann, s.r. (2012). modelagem multicriterial aplicada a problemas de regulação em áreas portuárias privatizadas (multi-criteria modeling applied to regulatory issues of privatized port areas). rac, 16(4), 493-517. doi: https://doi.org/10.1590/s1415-65552012000400002 hibernon filho, h.m., cutrim, s.s., robles, l.t., & pereira, n. n. (2016). potencial e limitações para o escoamento de soja pelos portos da região norte. in brasil: anais do enegep – encontro nacional de engenharia de produção. joão pessoa, pb, brazil: enegep. doi: https://doi.org/10.14488/enegep2018_tn_sto_265_520_35603 ijahp article: pereira, botter, robles/port terminal in the northern region of brazil: decision upon public port or private use terminal international journal of the analytic hierarchy process 213 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.617 ibge (instituto brasileiro de geografia e estatística) (2017). levantamento sistemático da produção agrícola – tabela 1618. khan, haris m. (2004, may 10-16). impacts of globalization on port competitiveness. pakistan & gulf economist. lirn, t.c., hathanopoulou, m.j., & beresford, a.c.k. (2004). an application of ahp on transshipment port selection: a global perspective. maritime economics & logistics, 6, 70-91. doi: https://doi.org/10.1057/palgrave.mel.9100093 loureiro, j.f., freitas, r.r., & gonzales, w. (2015). proposta de um método de localização para expansão de um terminal portuário por meio do analytic hierarchy process (ahp). revista espacios, 36(10). doi: https://doi.org/10.11606/t.3.2016.tde05102016-091344 magalhães, j.r., & botter, r.c. (2015). modelo de análise multicritério de apoio à decisão para aprovação de novos terminais portuários privativos no brasil. in i congresso internacional de desempenho portuário – cidesport. doi: https://doi.org/10.17648/cidesport-2018-89368 malchow, m., & kanafani, a. (2004). a disaggregate analysis of port selection. transportation research part e, 40, 317-337. doi: https://doi.org/10.1016/j.tre.2003.05.001 mapa (ministério da agricultura, pecuária e abastecimento) (2017). projeções do agronegócio brasil 2016/2017 a 2026/27. projeções de longo prazo. brasília, df, brazil: secretaria de política agrícola. marins, c.s., souza, d.o., & barros, m.s. (2009). o uso do método de análise hierárquica (ahp) na tomada de decisões gerenciais – um estudo de caso. in anais do xli simpósio brasileiro de pesquisa operacional – sobrapo. porto seguro, ba, brazil: universidade estadual de maringá. doi: https://doi.org/10.14488/enegep2017_tn_sto_247_427_33060 mazza, m., & robles, l.t. (2004). análise dos fatores de decisão na escolha de portos brasileiros na logística de exportação de carga conteinerizada na perspectiva do embarcador: uma proposta metodológica. in anais do xi simpep – simpósio de engenharia de produção. bauru, sp, brazil: unesp. doi: https://doi.org/10.5151/marine-spolm2014-127156 movimento pró-logística do mato grosso (2017). a importância do arco norte na competitividade da exportação agropecuária. mtpa (ministério dos transportes, portos e aviação civil) (2018). sistema portuário nacional. ijahp article: pereira, botter, robles/port terminal in the northern region of brazil: decision upon public port or private use terminal international journal of the analytic hierarchy process 214 vol. 11 issue 2 2019 issn 1936-6744 https://doi.org/10.13033/ijahp.v11i2.617 nazemzadeh, m. & vanelslander, t. (2015). the container transport system: selection criteria and business attractiveness for north-european ports. maritime economics & logistics, 17(2), 221-245. doi: https://doi.org/10.1057/mel.2015.1 nir, a-s., lin, k., & liang, g-s. (2003). port choice behaviour – from the perspective of the shipper. maritime policy and management, 2, 165-173. doi: https://doi.org/10.1080/0308883032000069262 pinheiro, p.c., kagan, h., xavier, m., & znamensky, a. (2003). aplicação do método de análise hierárquica ao problema de priorização de investimentos em estradas vicinais no estado de são paulo. in anais do xvii congresso de pesquisa e transporte anpet. rio de janeiro, rj, brazil: ime. doi: https://doi.org/10.11606/t.44.1984.tde-25082015143959 porto de santos (2018). infraestrutura. acesso ferroviário. retrieved from http://www.portodesantos.com.br/infraestrutura/acesso-ferroviario/ rodrigue, j. p., & notteboom, t. (2017). the containerization of commodities. a new growth dynamic for containerization. in rodrigue, j-p. (ed.), the geography of transport systems (4th ed.). new york: routledge. doi: https://doi.org/10.1057/9781137475770.0005 saaty, t.l. (1996). decision making with dependence and feedback the analytic network process. pittsburgh, pa: rws publications. saaty, t.l. (2008). decision making with the analytic hierarchy process. international journal of services sciences, 1(1), 83-98. silva, a.c.s., guedes, e.c.c.; ribeiro, j.r., nascimento, l.p.a.s., belderrain, m.c.n., & correia, a.r. (2007). multiple criteria methods applied to select suppliers of a capital goods company. in proceedings of the 9th international symposium on the analytic hierarchy process for multi-criteria decision making, viña del mar, chile: isahp. song, d.w., & yeo, k-t. (2004). a competitive analysis of chinese container ports using the analytic hierarchy process. maritime economics & logistics, 1, 34-52. doi: https://doi.org/10.1057/palgrave.mel.9100096 tongzon, j.l. (1995). determinants of port performance and efficiency. transportation research, 3, 245-252. usda (united states department of agriculture) (2018). oilseeds world market and trade. ijahp article: akman, karaman/pillars in the making, industry 4.0 on the horizon international journal of the analytic hierarchy process 240 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.839 pillars in the making, industry 4.0 on the horizon engin akman independent researcher mechanicsburg, pa, usa usenakman@gmail.com abdullah karaman college of engineering and technology, american university of the middle east, kuwait abdullah.karaman@aum.edu.kw abstract industry 4.0 (i4.0) marks a new era in manufacturing and has attracted notable attention from practitioners and researchers. current production processes are being transformed towards interconnecting the elements of manufacturing systems as a result of digitization. adopting new technologies is an indispensable practice to compete and sustain business concerns. in this paper, the analytical hierarchy process (ahp), a multi-criteria decision-making methodology, is employed to evaluate and weigh the nine pillars that are the building blocks of an i4.0 system. the assessment model suggests three dimensions, nine pillars, and thirty-four sub-pillars which are evaluated by fourteen i4.0 professionals responding to a pairwise questionnaire. the results are important as they reflect the opinions of the professionals and can help define strategies for companies investing in i4.0 technologies by elucidating the relative impacts of factors in an i4.0 environment. keywords: industry 4.0; nine pillars; cyber-physical systems; industrial internet of things; multi-criteria decision-making; ahp 1. introduction breakthroughs in industry are defined as industrial revolutions and it is accepted that civilization has witnessed four such revolutions. the energy resources and production tools employed in manufacturing, as well as business processes, define the industrial eras. it is widely presumed that the first industrial revolution started in the late 1700s when steam energy was introduced to provide powerful machines. the entry of electrical energy and mass production in manufacturing marked the start of industry 2.0 about a century later. the inclusion of computers, electronics, and automation in the 1970s is considered the start of the third industrial revolution (rüßmann et al., 2015). there is no consensus among researchers about the specific start and end dates of each of the industrial revolutions. the authors think there is only a beginning of an industrial mailto:usenakman@gmail.com mailto:abdullah.karaman@aum.edu.kw ijahp article: akman, karaman/pillars in the making, industry 4.0 on the horizon international journal of the analytic hierarchy process 241 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.839 revolution, as the inventions of the earlier periods are still in use in one form or another. a distinguishing quality of the revolutions is that they are disruptive. industry 4.0 (i4.0) describes a production style in which the internet, connected networks, automation, and digitization rise giving way to a cyber-revolution, causing systems invented during the first three industrial revolutions to become smart. i4.0 was first articulated at the hannover fair in 2011 (vogel-heuser & hess, 2016) and has been changing the pace of manufacturing in the last decade. in this new chapter of the industrial revolution, businesses plan to integrate industrial assets acquired into digital platforms and the internet to make them more intelligent and interactive, thus, creating cyber-physical systems (cps; hermann et al., 2016). i4.0 enables physical systems to learn, think, manage, process, communicate, decide, and optimize as a result of advanced technologies (rüßmann et al., 2015). an operating i4.0 manufacturing system can be achieved by smart and autonomous systems supported by big data and machine learning. i4.0 enables connected and communicating computers to make decisions with limited human participation (marr, 2016). though there are no identifying standards for the elements, processes, and interactions, there are initiatives to define the standards, particularly in the us and germany (rüßmann et al., 2015). a network of cpss and the internet of things (iot) make effective use of accumulated data and turn that data into results creating a principle summarized as “optimize yourself, communicate and help optimize the rest”. the volumes of data are exploding, and more data has been created in two years than in the entire history of humanity. less than 0.5% of the data is used or analyzed indicating the scale of the potential as data is produced at a greater rate than the industries absorb and consume it (marr, 2015). big data is accumulated by about 22 billion connected devices (mercer, 2019). the industrial iot (iiot) is being used in more machines and objects, and processes such as procurement, design, production, and maintenance of companies will be further automated. iiot, cps, and big data are major enablers of the transition to smart manufacturing or i4.0, but the concept is more sophisticated and has numerous components. most researchers agree that the components can be grouped into, though not limited to, the following nine pillars: big data and analytics, autonomous robots, simulation, horizontal and vertical system integration, the industrial internet of things, cybersecurity, the cloud, additive manufacturing and augmented reality (alcácer & cruz-machado, 2019; bahrin et al., 2016; motyl et al., 2017; rüßmann et al., 2015; vaidya et al., 2018; wang & wang, 2016). each pillar requires a thorough understanding and sound infrastructure starting from the conceptual design to the after-sales services. the combination of all of the pillars leads to a complete transformation to i4.0. in fact, each pillar has a great potential impact on the production systems and sophisticated nature that make it harder for practitioners and academics to handle. this study aims to investigate the pillars and their components that can enable a benchmark for professionals performing a gap analysis and help direct investments in building i4.0 processes. ijahp article: akman, karaman/pillars in the making, industry 4.0 on the horizon international journal of the analytic hierarchy process 242 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.839 many of the world's leading industrial nations have been investing in national initiatives to develop advanced manufacturing, innovation, and design to keep up with i4.0 trends in the last decade. for example, the investment projects to adopt i4.0 technologies in 20152020 exceeded 40 billion euros in germany (oztemel & gursev, 2020). the investments have been driven by the vision to achieve intelligent factories and smart manufacturing augmented by internet technologies (zhong et al., 2017). new business processes are mostly aimed at increased flexibility in manufacturing to satisfy the ever-changing and customized customer demands (alcácer & cruz-machado, 2019) with pressure to meet strict quality and delivery time requirements. despite the aspiration to integrate innovative concepts into current practices, most businesses lack strategy, knowledge, and skills. recent research conducted with 2,029 csuite leaders from 19 countries showed that while 70% believe switching to i4.0 technologies is an obligation for long-term success, only 10% of the executives have a clear and robust roadmap paving the way to i4.0. about 60% think they are in the stage of understanding the skills needed for the transition period and only 20% agree their institutions have the necessary skills in the i4.0 era (deloitte development llc, 2020). investments in i4.0 technologies are only possible after the correct strategy is defined by competent corporate executives (luthra & mangla, 2018). notwithstanding the fast pace of technological innovation and the increasing amount of research carried out by practitioners and academics, the i4.0 paradigm is far from maturity. academic studies focusing on the concepts, definitions, components, skills, and relationships among them can help build a roadmap and assist in implementation of i4.0. our title "pillars in the making, industry 4.0 is on the horizon” addresses the gap in the literature and suggests that there is a long way to go. this research contributes building the concept of i4.0 from the viewpoint of the industrial professional focusing on the criteria that ensure the design, realization, and development of i4.0. defining a comprehensive configuration for i4.0 that is adaptable across various industries is crucial since the components of the new manufacturing systems will be highly independent and incorporated. in addition, there are very diverse application alternatives introduced by technology developing companies (contreras-masse et al., 2020). a conceptual structure showing the hierarchy of the elements can help both practitioners and academics in the making of i4.0. we adopted a taxonomic method, which has contributed to the development of science since early times, to define i4.0. the study is aimed at disentangling the concept and helping design a roadmap for the new industrial era for practitioners, academics, and policymakers. in this paper, the analytical hierarchy process (ahp), a multi-criteria decision-making (mcdm) methodology, is used to evaluate and rank the nine pillars that are the building blocks of i4.0. the assessment model considers three dimensions, nine pillars, and thirtyfour sub-pillars which are evaluated by fourteen i4.0 professionals replying to a pairwise questionnaire. the results of the study are valuable in putting forth the ranking of the pillars and the alignment of projects in transforming to i4.0. ijahp article: akman, karaman/pillars in the making, industry 4.0 on the horizon international journal of the analytic hierarchy process 243 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.839 the article is structured as follows. the literature review focuses on the literature defining i4.0 and the nine pillars. the two-phase research methodology is explained in the following section. results are explained in the findings and discussion section. the conclusion explains the contribution of the article, the limitations, and further research topics motivated by the results. 2. literature review academia as well as industries focus on i4.0 as it is a decisive topic for the future of manufacturing. it is shaping many different kinds of industries, supply chains, human resources, investments, education, training, and society (bai et al., 2020). however, there is a substantial lack of comprehension about i4.0 technologies and their interactions (kamble et al., 2018). the companies adopting new technologies enjoy increasing productivity and quality, higher customer satisfaction, and competitiveness (rüßmann et al., 2015). better results are expected due to the improvements recorded in production, financial performance, market share, supply chain management, product lifecycle management, talent management, and business models (prause, 2015; rosa et al., 2020). in addition to i4.0 being an industrial revolution, the gap in knowledge and the advantages of the i4.0 innovations are major reasons both researchers and practitioners are attracted to this topic (liao et al., 2017; rosa et al., 2020). müller et al. (2018) suggested that the current status of a company regarding i4.0 adoption falls into one of the four following groups: craft manufacturers (do not have any plan to switch to i4.0), introductory phase planners (trying to embed new systems), i4.0 users (implemented some of the pillars to a certain degree), and full-scale adopters. the overwhelming majority of corporations, excluding the first group, are consumers of i4.0 technologies which show the importance of literature that disentangles the concept into absorbable and adaptable elements. i4.0 solutions can help solve problems in different spheres as well as industries. for example, the applications adopted in the healthcare sector have enabled effective methods for frontline workers in the covid-19 pandemic (javaid et al., 2020). da silva et al. (2019) focused on technology transfer (tt) in importing and integrating advanced technologies into the functional members of manufacturing processes within the framework of i4.0. collaboration instead of competition and “real-time visibility” in the supply chain will be the major characteristics of tt. to contribute to the understanding and delivery of technology in building i4.0 infrastructure, more research will be needed. our study has the potential to guide professionals engaged in the tt process. attempts to classify the technologies to build a taxonomy, identifying the challenges and solutions, defining standards and scales to explain the outcomes for both digital and physical assets are the main areas of research in the i4.0 literature (dalenogare et al., 2018; luthra & mangla, 2018; oztemel & gursev, 2020). mcdm methods have been used in the i4.0 literature to evaluate the concept from different points of view. mcdm, ijahp article: akman, karaman/pillars in the making, industry 4.0 on the horizon international journal of the analytic hierarchy process 244 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.839 particularly the ahp, which can bring conflicting opinions of experts together is a suitable method for the i4.0 paradigm as maturity level, utilization and understanding of each pillar is at a different stage and the adoption rates of i4.0 technologies vary significantly across industrial sectors and sizes of corporations (calabrese et al., 2020; frank et al., 2019). frank et al. (2019) addressed the gaps in understanding of i4.0 technologies and adoption patterns of manufacturing companies. luthra and mangla (2018) focused on the criteria of i4.0 implementation in supply chain sustainability employing ahp and rank organizational challenges ahead of technological challenges for success. erdogan et al. (2018) used ahp-vikor to support the selection of useful strategies in the i4.0 transformation employing factors like human resources, information systems, business models, and standardization. bai et al. (2020) developed hybrid mcdm utilizing a hesitant fuzzy set, cumulative prospect theory, and vikor assessing the priority of i4.0 technologies from a sustainability perspective. contreras-masse et al. (2020) proposed promethee-ii as an mcdm to select iiot platforms employed in i4.0 manufacturing. singh et al. (2018) employed the ahp methodology to define and rank the impact of success factors in i4.0 realization. adapting the technology infrastructure and each pillar according to company requirements combined with training of users enables transforming the processes into an operating i4.0 environment (rüßmann et al., 2015), which encompasses high digitalization, smart manufacturing, and inter and intra-company connectivity (müller et al., 2018). 2.1 big data and analytics of the nine components, "big data" is the engine of i4.0 as the development is realized by processing this data. the huge volume of data is not crucial as conceived, but the feasibility and quality of the data are crucial since the results depend on the processing of the relevant data. the term “big data” was first used by michael cox and david ellsworth at the proceedings of the ieee 8 th conference on visualization in 1997 (press, 2013): “visualization provides an interesting challenge for computer systems: data sets are generally quite large, taxing the capacities of main memory, local disk, and even remote disk. we call this the problem of big data. when data sets do not fit in main memory (in core), or when they do not fit even on local disk, the most common solution is to acquire more resources.” the concept might have been expressed in other reports or scholarly research even years ago, but we want to highlight that the very first users of the concept perceived it as a “problem” and the “implied” need for cloud computing as “local disk” is incompetent. high volume, variety, and velocity are typical features of big data (kaur & singh, 2018), and technologies akin to optical image recognition which can retrieve a certain word in millions of scanned pages enable the usage of big data (vrochidis et al., 2010). similarly, data is produced in production facilities, workstations, machinery, maintenance units, vehicles, robots, power plants, supply chains, marketing, and after-sales efforts, etc. in ijahp article: akman, karaman/pillars in the making, industry 4.0 on the horizon international journal of the analytic hierarchy process 245 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.839 millions of lines as a result of i4.0 technology, and the retrieval and utilization of the right data play an important role in big data. this pillar encompasses 3 main sub-pillars that are elaborated in table 1 as data management (bordeleau et al., 2018; kamble et al., 2018; lu, 2017; tao et al., 2018; vaidya et al., 2018), data mining (kamble et al., 2018; lee et al., 2014a; lee et al., 2014b; lu, 2017; tao et al., 2018; vaidya et al., 2018) and self-organized manufacturing (lee et al., 2014a; lee et al., 2014b; tao et al., 2018; wang et al., 2016; wang & wang, 2016). the sub-pillars or factors that make these features work need to be taken into consideration when designing an effective big data and analytics infrastructure. 2.2 the cloud cloud computing technology was created to store, protect, and process constantly growing data, that is, to produce new information from existing data, to reveal complex relationships between data, and to make it available when needed (oztemel & gursev, 2020). this big data can be transmitted to people, smart objects, and machines quickly achieving reaction times in milliseconds (rüssmann et al., 2015). in a cloud, the hardware, software, and real-time data that are used are separated and stored away from the central facilities and are brought together when necessary, often automatically. obviously, the sine qua non of this new order is the iiot. everything is considered as service in cloud computing and the services include infrastructure, platform, and software (alcácer & cruz-machado, 2019). the infrastructure consists of storage, networking, and virtualization; platforms include operating systems to run the software and are used to develop applications; software applications enable processing data retrieved via interfaces. access to a cloud can be grouped into four types according to the following access protocols (alcácer & cruzmachado, 2019): public, private, hybrid, and community, which is a specialized part of the public cloud. 2.3 the industrial internet of things the iiot is the communication network that enables the communication of physical objects integrated with each other using traditional internet protocols. “industrial’ and “things” are crucial concepts in building the iiot pillar of an i4.0 system. things can be whatever is used in industrial manufacturing including objects and humans (alcácer & cruz-machado, 2019), and the number of active objects exceeded 22 billion (mercer, 2019). the iiot infrastructure enables storing, transmitting, analyzing real-time data, and communicating with the rest of the system elements to optimize the output in terms of lead times, effectiveness, quality, etc. navigation systems used by billions of people every day are a good example of the importance of obtaining real-time data. these systems allow one to review the route and reach the destination using less time and energy. utilization of the iiot is increasing in numerous fields like energy management, marketing, transportation, smart homes, transportation, agriculture, health, and education (alcácer & cruz-machado, 2019; rüßmann et al., 2015). the success of the iiot is dependent on a clear definition of configuration, architectural models, standards, projects, performance indicators, and industrial activities. four ijahp article: akman, karaman/pillars in the making, industry 4.0 on the horizon international journal of the analytic hierarchy process 246 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.839 architectural levels are proposed to establish and evaluate the iiot infrastructure (alcácer & cruz-machado, 2019) starting from the “object” to the end-user. the physical or sensing layer detects and translates the data tags obtained by touching the original source of the data. the network layer performs data transmission from the “object” to the service layer. this level is decisive in the required data to be shared and the destination stations of that data. the third level stores the big data for optimization and decision-making, and the interface level serves as an interface to display data and provide interaction with the system and users. the connections between the four layers can be wired or wireless and additional monitoring provided by a human or other smart device can increase the value of analysis and the level of optimization. four level automation that is synonymous with the four layers of the iiot has been used in traditional industries such as steel manufacturing for decades (gauvreau, 2011), but i4.0 technologies and the internet provided individual stations or objects to store, analyze, decide using “real-time” data and perform actions to reach pre-defined solutions autonomously. the connection and interaction of elements in the system support optimizing the whole value chain. 2.4 cybersecurity the connected systems communicating via the iiot network consisting of embedded electronic systems like rfid, chips, and sensors and physical assets such as objects, machinery, hardware, etc. are called cpss in the i4.0 literature (alcácer & cruzmachado, 2019). cpss are central to automation in smart factories as they can plan and operate all production processes with the retrieved data. the traditional automation hierarchy forms a triangle starting with the programmable logic controller (plc) level to the user-level interface, whereas cps-based i4.0 automation forms a spider-web with distributed service (hozdic, 2015). cpss establish communication between mechanical and electronic components through i4.0 technologies within a network system to create smart manufacturing. cpss increase connectivity with global networks using standard internet protocols that make sensitive corporate data, crucial industrial components, and production lines vulnerable to hostile cyber intervention (rüssmann et al., 2015). the attacks can be originated from an internal or external source (alcácer & cruz-machado, 2019). corporations are trying to overcome cybersecurity flaws by utilizing secure communications, sophisticated access management of systems, machines, and users as well as cooperating with professional cybersecurity companies. cybersecurity control points can be categorized into three groups as physical controls regulating access to physical assets, access controls to data resources and communication controls securing the data transmission via networks (flatt et al., 2016; kobara, 2016). 2.5 horizontal and vertical system integration horizontal and vertical integration of real-time data flows between all elements of the manufacturing process and is a key concept in a smart factory (hozdic, 2015). horizontal integration relates to inter-organizational data transmission beyond the organizational boundaries through all value chains including partners, suppliers, and customers. vertical integration includes data sharing across intra-organizational processes from conceptual design to manufacturing and after-sales services. end-to-end integration is integration ijahp article: akman, karaman/pillars in the making, industry 4.0 on the horizon international journal of the analytic hierarchy process 247 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.839 across the organization and different companies incorporating customer demands through the entire product life cycle (liao et al., 2017). integration combines the virtual and the real world due to sophisticated it infrastructure including sensors, actuators, plcs, manufacturing execution systems, enterprise resource planning systems, machine-tomachine communication (hozdic, 2015). the nature of complete integration within the framework of i4.0 generates coopetition, which brings cooperation and competition together. 2.6 autonomous robots autonomous robots can make decisions and act on their own in dynamic environments without intervention. they are equipped to collect data and analyze, communicate with connected robots, objects, and operators, move in the workstation and ensure the safety of surroundings (rüssmann et al., 2015). embedded systems providing artificial intelligence make these robots smarter. autonomous robots are used in numerous fields, manufacturing, and services as well as mining and agriculture (alcácer & cruz-machado, 2019). the critical factors of successful operations of autonomous robots are interacting with one another and humans. 2.7 simulation simulation, modeling a concept in the virtual world, has been used in the research and development of products and materials. innovative i4.0 technologies made it possible to obtain and analyze real-time data from production, processes, machinery, and human behavior (rüssmann et al., 2015). engineering simulation helps build optimum design while process simulation helps build optimum processes with fast decision making in manufacturing, maintenance, and customer service, etc. (alcácer & cruz-machado, 2019). engineering evaluation is defined as off-line simulation and real-time data using modeling for process performance is called on-line simulation (negahban & smith, 2014). simulation using the current data of an operating system contributes to smart manufacturing considerably as the results such as performance and failures with certain parameters conveyed to similar workstations can help optimize the results of further stations and the next operations of the current station. obviously, on-line simulation requires sophisticated it infrastructure with computational efficiency covering all processes of an enterprise. sub-criteria of simulation infrastructure are virtual prototyping, virtual production, and virtual maintenance, repair, and operations (kamble et al., 2018; mourtzis et al., 2014; qi & tao, 2018). 2.8 additive manufacturing (am) producing physical parts from digital data, also known as rapid prototyping, began in the late 1980s (manfredi et al., 2014). this method used to produce prototypes or sample parts was the first type of 3d printing or additive manufacturing. today am technology is used in manufacturing processes with sophisticated methods beyond rapid prototyping technology. am is defined in the american society for testing and materials (astm) 2792-10 standard as (manfredi et al., 2014): ijahp article: akman, karaman/pillars in the making, industry 4.0 on the horizon international journal of the analytic hierarchy process 248 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.839 “the process of joining materials to make objects from 3d model data, usually layer upon layer, as opposed to subtractive manufacturing technologies.” am is a smart technology that revolutionized conventional manufacturing methods as customized parts can be manufactured in shorter times with high precision and no waste, molds, or assembly. aerospace companies were the first to adopt am technology as it enables complex lightweight designs and decentralized am systems to reduce shipping costs and stocks (rüssmann et al., 2015). a conventional am process includes the geometry design, it tools and interface development, final design, process forming, and control tools (alcácer & cruz-machado, 2019; manfredi et al., 2014). astm standard f2792 groups am processes into seven groups as binder jetting, directed energy deposition, material extrusion, material jetting, powder bed fusion, sheet lamination, and vat polymerization. 2.9 augmented reality (ar) in ar technology, the real images are enriched by combining supporting elements such as real-time audio, images, and graphics developed with different digital technologies in their own environment (oztemel & gursev, 2020). ar glasses, smart devices or monitors, or other types of imaging tools are often needed. ar has a wide field of applications ranging from education and health to military and tourism. ar is an indispensable part of smart manufacturing and is used in operations such as design, assembly, maintenance, logistics, quality and safety management, and warehousing. ar professionals can perform all necessary operations remotely which speeds up the maintenance, training, and assembly activities in different parts of the world with lowskilled workers (rüssmann et al., 2015). ar technology has the potential to be significantly developed and used in many different applications from various walks of life (oztemel & gursev, 2020). one of the key advantages of ar is the storage and retrieval of high-quality and user-friendly data on company processes like manufacturing, maintenance, safety, and quality which can eliminate paperwork and usage of outdated information (blanco-novoa et al., 2018). ar has five sub-pillars including training, design, manufacturing, operations, and service which are elaborated in table 1 (alcácer & cruz-machado, 2019; fraga-lamas et al., 2018; kamble et al., 2018; mourtzis et al., 2017). 3. research methodology we followed a two-phase research methodology in this study to determine technology priorities of i4.0. in phase i, we identified the dimensions, pillars, and sub-pillars of the i4.0 technologies with the help of a comprehensive literature review. then, we formed an experts’ panel (ep). the dimensions, pillars, and sub-pillars of the i4.0 technologies were determined through discussions and the support of the ep. in phase ii, the priorities of the i4.0 technologies including the dimensions, pillars, and sub-pillars were evaluated via the widely recognized ahp methodology by surveying industry professionals. figure 1 illustrates the step-by-step research methodology. ijahp article: akman, karaman/pillars in the making, industry 4.0 on the horizon international journal of the analytic hierarchy process 249 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.839 figure 1 two-phase research methodology for industry 4.0 technologies 3.1 phase i criteria selection the criteria included in the study are a result of combining information from the existing literature and industry experience. the study adopted the nine pillars of i4.0 based on alcácer and cruz-machado (2019), bahrin et al. (2016), motyl et al. (2017), rüßmann et al. (2015), vaidya et al. (2018), and wang and wang (2016). the nine pillars were first grouped into dimensions following the recommendation of the ep as base technologies, smart operations, and smart technologies to make rational and consistent comparisons between similar concepts. the base technologies include the industrial internet of things, horizontal and vertical system integration, big data and analytics, cybersecurity, and the cloud, which provides connectivity and intelligence (frank, 2019). smart operations includes simulation and augmented reality. finally, smart technologies includes autonomous robots and additive manufacturing. then, sub-pillars were determined based on the extant literature and the suggestions from the ep. the ep’s suggestions helped finalized all the criteria and hierarchy included in the study. the ep was composed of two academics and two industry professionals with more than twenty years of experience. the two researchers authored this study; one of them has substantial expertise in the manufacturing industry, whereas the other one has identifying the i4.0 technologies, dimensions, pillars and sub-pillars forming an experts’ panel (ep) finalizing the dimensions, pillars and subpillars of the i4.0 technologies with the support of ep implementation of the questionnaire by the industry professionals ahp method determining the weights of the dimensions, pillars and sub-pillars phase i criteria selection phase ii ahp ijahp article: akman, karaman/pillars in the making, industry 4.0 on the horizon international journal of the analytic hierarchy process 250 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.839 significant experience in multi-criteria decision-making approaches. as a result, the dimensions, pillars, and sub-pillars were obtained from prior studies and up-to-date industry practices. final i4.0 dimensions, pillars, sub-pillars, and their explanations are summarized in table 1 and the structured hierarchy is given in figure 2 (excluding the sub-pillar level due to the space constraint). ijahp article: akman, karaman/pillars in the making, industry 4.0 on the horizon international journal of the analytic hierarchy process 251 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.839 table 1 industry 4.0 dimensions, pillars, and sub-pillars dimension pillar sub-pillar explanation references base technologies the industrial internet of things (iiot) physical layer sensors, machines, and products are connected using standard technologies (includes the physical infrastructure) alcácer and cruz-machado (2019); gilchrist (2016); kamble et al. (2018); lu (2017); posada et al. (2015); rüßmann et al. (2015); sadeghi et al. (2015); thames and schaefer (2016); vaidya et al. (2018); wan et al. (2016); wollschlaeger et al. (2017) network layer industrial wireless networks transmitting data, commands, etc. alcácer and cruz-machado (2019); gilchrist (2016); lu (2017); wan et al. (2016); wollschlaeger et al. (2017) service layer stores the big data for optimization and decisionmaking alcácer and cruz-machado (2019); gilchrist (2016); kamble et al. (2018); lu (2017); posada et al. (2015); thames and schaefer (2016); wan et al. (2016); wollschlaeger et al. (2017) interface layer display data and provide interaction with the system and users alcácer and cruz-machado (2019); gilchrist (2016); thames and schaefer (2016); wan et al. (2016) horizontal and vertical system integration horizontal integration horizontal integration across the entire value creation network (from the material flow to the logistics of product life cycle) brettel et al. (2014); dalenogare et al. (2018); kagermann et al. (2013); lu (2017); posada et al. (2015); rüßmann et al. (2015); stock and seliger (2016); vaidya et al. (2018) ijahp article: akman, karaman/pillars in the making, industry 4.0 on the horizon international journal of the analytic hierarchy process 252 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.839 dimension pillar sub-pillar explanation references vertical integration vertical integration and networked manufacturing systems (include sensors, actuators, programmable logic controllers, manufacturing execution systems, enterprise resource planning systems, machineto-machine communication) integrates product, equipment, and human needs. dalenogare et al. (2018); frank et al. (2019); kagermann et al. (2013); lu (2017); posada et al. (2015); rüßmann et al. (2015); stock and seliger (2016); vaidya et al. (2018); weyer et al. (2015) end-to-end integration end-to-end integration across the entire product life cycle. brettel et al. (2014); dalenogare et al. (2018); kagermann et al. (2013); posada et al. (2015); stock and seliger (2016); vaidya et al. (2018) big data and analytics data management collection (data tagging tools), architecture, integration, classification, warehousing (etl) bordeleau et al. (2018); kamble et al. (2018); lu (2017); tao et al. (2018); vaidya et al. (2018) data mining data visualization real-time machine learning applications include control and monitoring, informationsharing, collaborative decision-making creating operational value. predictive machine learning applications include prognostics and health management. kamble et al. (2018); lee et al. (2014a); lee et al. (2014b); lu (2017); tao et al. (2018); vaidya et al. (2018) ijahp article: akman, karaman/pillars in the making, industry 4.0 on the horizon international journal of the analytic hierarchy process 253 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.839 dimension pillar sub-pillar explanation references self-organized manufacturing negotiation mechanisms for components, machines, and systems to become selfaware, self-predict, selfcompare, self-configure, selfmaintain, self-organize, and self-adaptive lee et al. (2014a); lee et al. (2014b); tao et al. (2018); wang et al. (2016); wang and wang (2016) cybersecurity physical controls prevents unauthorized access to physical assets flatt et al. (2016); kobara (2016); rüßmann et al. (2015); waidner and kasper (2016); access controls restricts unauthorized access to information resources (includes authentication and authorization) flatt et al. (2016); he et al. (2016); kobara (2016); waidner and kasper (2016) communication controls secures the movement of data across networks flatt et al. (2016); he et al. (2016); kobara (2016); rüßmann et al. (2015); waidner and kasper (2016) the cloud infrastructure as a service (iaas) infrastructure to run software and store data alcácer and cruz-machado (2019); almada-lobo (2015); kamble et al. (2018); lee et al. (2014b); rüßmann et al. (2015); vaidya et al. (2018); wan et al. (2016); zhou et al. (2015) platform as a service (paas) platforms to develop applications alcácer and cruz-machado (2019); almada-lobo (2015); lee et al. (2014b); rüßmann et al. (2015); vaidya et al. (2018); wan et al. (2016); zhou et al. (2015) software as a service (saas) software applications to process data alcácer and cruz-machado (2019); almada-lobo (2015); lee et al. (2014b); rüßmann et al. (2015); vaidya et al. (2018); wan et al. (2016); zhou et al. (2015) ijahp article: akman, karaman/pillars in the making, industry 4.0 on the horizon international journal of the analytic hierarchy process 254 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.839 dimension pillar sub-pillar explanation references smart operations simulation virtual prototyping product, material, process development brettel et al. (2014); kamble et al. (2018); mohammad et al. (2021); mourtzis et al. (2014); qi and tao (2018); rüßmann et al. (2015); zawadzki and żywicki (2016) virtual production digital twin: design and optimization of cyberphysical production systems brettel et al. (2014); kamble et al. (2018); lasi et al. (2014); moreno et al. (2017); mourtzis et al. (2014); negri et al. (2017); qi and tao (2018); rodič (2017); rüßmann et al. (2015); schleich et al. (2017); uhlemann et al. (2017); weyer et al. (2016); xu et al. (2016) virtual maintenance, repair, and operations (mro) prediction of proactive maintenance qi and tao (2018) augmented reality training includes job-specific tasks, safety, and security procedures, etc. alcácer and cruz-machado (2019); fraga-lamas et al. (2018); mourtzis et al. (2017); posada et al. (2015); rüßmann et al. (2015) design collaborative engineering, error diagnosis, etc. alcácer and cruz-machado (2019); fraga-lamas et al. (2018); mourtzis et al. (2017); zhong et al. (2017) manufacturing quality assurance, monitoring performance, issuing assembly and maintenance work instructions, tracking, monitoring alcácer and cruz-machado (2019); fraga-lamas et al. (2018); mourtzis et al. (2017) operations augmented interface and operator manuals, heads-up displays, digital product alcácer and cruz-machado (2019); fraga-lamas et al. (2018); kamble et al. (2018); rüßmann et al. (2015) ijahp article: akman, karaman/pillars in the making, industry 4.0 on the horizon international journal of the analytic hierarchy process 255 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.839 dimension pillar sub-pillar explanation references controls, etc. service remote maintenance/repair guidance, manual and service instructions, service inspections, self-service, etc. alcácer and cruz-machado (2019); fraga-lamas et al. (2018); kamble et al. (2018); mourtzis et al. (2017); rüßmann et al. (2015); vaidya et al. (2018) sales and marketing product display and demos, augmented marketing, etc. alcácer and cruz-machado (2019); fraga-lamas et al. (2018); mourtzis et al. (2017) smart technologies autonomous robots interaction with one another use vision sensors, artificial intelligence, and self-learning to work flexibly with high performance bahrin et al. (2016); dopico et al. (2016); kamble et al. (2018); rüßmann et al. (2015); vaidya et al. (2018); interaction with humans work safely hand in hand with humans bahrin et al. (2016); gonzalez et al. (2018); kamble et al. (2018); rüßmann et al. (2015); vaidya et al. (2018); additive manufacturing binder jetting two materials, the binder (usually in liquid form) and the build material (in powder form) are used along with a print head. the print head moves and deposits alternating layers of the build material and the binding material. additive manufacturing research group (2019); alcácer and cruz-machado (2019); international organization for standardization (2015); tofail et al. (2018) ijahp article: akman, karaman/pillars in the making, industry 4.0 on the horizon international journal of the analytic hierarchy process 256 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.839 dimension pillar sub-pillar explanation references directed energy deposition the laser/electron beam is used to deposit materials by melting. additive manufacturing research group (2019); alcácer and cruz-machado (2019); chong et al. (2018); international organization for standardization (2015); tofail et al. (2018) material extrusion a nozzle/orifice is used to fuse materials selectively. additive manufacturing research group (2019); alcácer and cruz-machado (2019); chong et al. (2018); international organization for standardization (2015); tofail et al. (2018); vaidya et al. (2018) material jetting material is jetted onto a build platform using either a continuous or drop-ondemand approach. additive manufacturing research group (2019); alcácer and cruz-machado (2019); international organization for standardization (2015); tofail et al. (2018) powder bed fusion the laser/electron beam is used to melt and fuse the material powder together. additive manufacturing research group (2019); alcácer and cruz-machado (2019); chong et al. (2018); international organization for standardization (2015); tofail et al. (2018); vaidya et al. (2018) sheet lamination sheets/ribbons of metals are used to form objects using ultrasonic welding. additive manufacturing research group (2019); alcácer and cruz-machado (2019); chong et al. (2018); international organization for standardization (2015); tofail et al. (2018) vat polymerisation uses a vat of liquid photopolymer resin to construct the model layer by layer. additive manufacturing research group (2019); alcácer and cruz-machado (2019); chong et al. (2018); international organization for standardization (2015); tofail et al. (2018) ijahp article: akman, karaman/pillars in the making, industry 4.0 on the horizon international journal of the analytic hierarchy process 257 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.839 3.2 phase ii – ahp the assessment of i4.0 technologies is qualitative in its essence and requires a multicriteria context. yet, the perception of the experts is even contradictory. from this perspective, the ahp, suggested by saaty (1980), is a broadly accepted and convenient approach to deal with mcdm (akman & dagdeviren, 2018; karaman & akman, 2018). the ahp methodology handles a mcdm by structuring a decision hierarchy starting with the goal, criteria, and sub-criteria. an important design constraint for the ahp is the assumption that all the criteria and sub-criteria are independent of each other. in fact, the ahp forms a group decision by assessing the criteria and sub-criteria and finding their relative importance. it combines both qualitative and quantitative factors analytically. in this context, the ahp hierarchy for i4.0 (the goals, dimensions, and pillars) is given in figure 2 (excluding the sub-pillar level due to the space constraints). figure 2 hierarchy of the ahp model the next step in the ahp is to determine the priority weights for the determined hierarchy. this is achieved through a pairwise comparison of the dimensions, pillars, and sub-pillars. a standardized scale of the nine levels (as shown in table 2) is used in the pairwise comparisons. h ie r a r c h ic a l g r o u p in g o f in d u st r y 4 .0 t e c h n o lo g ie s base technologies smart operations smart technologies the industrial internet of things (iiot) horizontal and vertical system integration cybersecurity big data and analytics the cloud simulation augmented reality autonomous robots level 1: the goal level 2: the dimension level 3: the pillar additive manufacturing ijahp article: akman, karaman/pillars in the making, industry 4.0 on the horizon international journal of the analytic hierarchy process 258 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.839 table 2 ahp scale of the relative importance degree of importance definition 1 equal importance 3 moderate importance 5 strong importance 7 very strong importance 9 absolute importance 2, 4, 6, 8 are used to articulate intermediate values. the steps for prioritizing the criteria include summarizing the pairwise comparisons in a pairwise evaluation matrix. the results of n criteria comparisons are stored in a 𝑛𝑥𝑛 matrix. let 𝐶 = {𝐶𝑗 \𝑗 = 1,2, ⋯ , 𝑛} represent the set of criteria for i4.0 technologies. the 𝑛𝑥𝑛 assessment matrix, 𝐴, stores the pairwise contrast of the criteria belong to 𝐶. the matrix 𝐴 is written as below in equation (1): 𝑨 = [ 𝒂𝟏𝟏 𝒂𝟐𝟏 𝒂𝟏𝟐 𝒂𝟐𝟐 ⋯ 𝒂𝟏𝒏 𝒂𝟐𝒏 ⋮ ⋱ ⋮ 𝒂𝒏𝟏 𝒂𝒏𝟐 ⋯ 𝒂𝒏𝒏 ] (1) under these circumstances, 𝑎𝑖𝑗 depicts the numerical value of the pairwise evaluation of criteria 𝑖 and 𝑗 for the ahp hierarchy illustrated in figure 2. 𝑎𝑖𝑗 ’s could assume any value between 1 and 9 as defined in table 2 and their reciprocals. in particular, values 1 to 9 are used whenever criteria 𝑖 is more important than criteria 𝑗 and their reciprocals are used whenever criteria 𝑗 is more important than criteria 𝑖. other entries of the matrix 𝐴 are determined by 𝑎𝑖𝑖 = 1, 𝑎𝑗𝑖 = 1 𝑎𝑖𝑗 assuming 𝑎𝑖𝑗 ≠ 0. next, the ahp normalizes and calculates the relative weights of each matrix 𝐴 dividing the entries into the columns to the respective column sums. the priority of the elements is determined by letting 𝑨𝒘 = 𝝀𝒎𝒂𝒙𝒘 (2) where 𝑤 is the principal eigenvector corresponding to the largest eigenvalue 𝜆𝑚𝑎𝑥. a final analysis in the ahp is the consistency check since the pairwise comparison of the criteria of the experts often conflicts and depends on subjective judgments. the consistency check is being conducted in two steps. first, the consistency index (ci) is calculated via 𝑪𝑰 = 𝝀𝒎𝒂𝒙−𝒏 𝒏−𝟏 . (3) ijahp article: akman, karaman/pillars in the making, industry 4.0 on the horizon international journal of the analytic hierarchy process 259 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.839 second, the consistency ratio (cr) is determined by 𝑪𝑹 = 𝑪𝑰 𝑹𝑰 . (4) certain values of the consistency ratio show acceptable pairwise evaluations. in the current literature, a recognized upper value on cr is 0.1. cr values beyond the upper bound require the process to be repeated or more analysis to be done to improve the consistency of the results (karaman & akman, 2018). 3.2.1 application of ahp based on the hierarchy, the authors prepared a questionnaire to assess the relative weights of the dimensions, pillars, and sub-pillars. the questionnaire is shown in the appendix. the standard ahp scale (provided in table 2) was used to signify the corresponding importance, 1 representing equal, and 9 symbolizing absolute importance. the authors adopted a purposive sampling methodology in the selection of fourteen i4.0 professionals. the reliability of the results is highly dependent on the purposive sampling approach (karaman & akman, 2018; öberseder et al., 2013). the professionals experienced in i4.0 in the ep suggested known global decision-makers and opinion leaders. we also used the linkedin platform to search for leading practitioners and contacted them. the decision-makers were intentionally chosen based on their expertise and engagement in i4.0 technologies. the professionals had at least ten years of experience in manufacturing, information technology, research and design, quality, and similar domains, while the majority of them had more than twenty-five years of consulting experience in these domains. the survey was sent to seventeen professionals; twelve of them responded and contributed to our research. two of the ep members also assessed the pairwise comparisons. the authors analyzed the responses received from the i4.0 professionals using microsoft excel. in this step, the ahp weights were obtained by combining the questionnaire answers and computing the geometric average of the pairwise assessments since this creates a group agreement. all the weights of the dimensions, pillars, and sub-pillars were obtained from the authors' own calculations. the cr value of all the assessments was within the acceptable level (less than 0.1). 4. findings and discussion the organizational strategy aligned with i4.0 initiatives is key to success in implementation that can be achieved with the support of executive management (sony & naik, 2020). this obviously requires the inclusion of technical management to build a straightforward strategy and implementation framework where our results can potentially provide a guide. the nine pillars of i4.0 have gained wide acceptance in the literature and many of the sub-pillars are highlighted in numerous papers as seen in table 1. we focused on the studies dealing with similar concepts or criteria to explain i4.0 and compare with our study; however, it is hard to completely collate the findings with previous studies as our study adopts a taxonomic method employing the ahp as a ijahp article: akman, karaman/pillars in the making, industry 4.0 on the horizon international journal of the analytic hierarchy process 260 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.839 mcdm to rank the factors building an i4.0 environment. to the best of our knowledge, this is the first study to handle i4.0 pillars using our methodology. the first level of comparison of the research (shown in table 3) includes base technologies, smart operations, and smart technologies. frank et al. (2019) suggest that the pattern of i4.0 adoption integrates the methodical implementation of smart (front-end) technologies and base technologies. we separated smart operations and proposed them as a third pillar. i4.0 is a fusion of pillar technologies and the path to success requires a sequence of projects ranging from small and gradual to very large and innovative (prause, 2015; sony & naik, 2020). in this manner, a shift of a corporation to i4.0 can be considered as project and change management (sony & naik, 2020). the current study asserts that there are three crucial pillars (phases) to achieve the project of a fully operating i4.0 system. the industry professionals view base technologies as significantly important followed by smart operations and smart technologies. similar to our findings, frank et al. (2019) report that the implementation of base technologies is challenging for companies. the results indicate that practitioners first focus on the status/maturity of factors given in table 4 before fully implementing the factors indicated in tables 5 and 6, respectively. table 3 ahp weights and rank of the dimensions dimensions weights rank base technologies 0.547 1 smart operations 0.287 2 smart technologies 0.166 3 adopting i4.0 will lead to colossal cybersecurity challenges as the confident data engendered by i4.0 is streamed through all value chains (he et al., 2016; sony & naik, 2020). smart manufacturing is dependent on cybersecurity maturity as well as operation technologies (ghobakhloo, 2020). our study reveals that the protection of data (cybersecurity) is a top priority for industry experts followed by big data and analytics, the cloud, iiot, and horizontal and vertical system integration, respectively. frank et al. (2019) adopted the four base technology pillars of cloud, iot, big data, and analytics. their findings propose that as companies have more mature i4.0 smart manufacturing technologies, they employ more advanced levels of base technologies. the implementation rates of the factors are as follows: iot (67%), cloud (60%), analytics (60%), and big data (60%) among advanced adopters. the results of frank et al. (2019) do not resemble our results as they did not include cybersecurity. furthermore, big data and analytics is given priority over the enablers the cloud and iiot by our experts. the logic behind this result may be the emphasis on the quality of the data collected and the value of analytics for the operations and sustainability of the corporations. one of the experts stated that they got “drowned” in one of their projects as millions of lines of data obtained through tags made it hard to tackle and analyze the data. ijahp article: akman, karaman/pillars in the making, industry 4.0 on the horizon international journal of the analytic hierarchy process 261 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.839 table 4 ahp weights and rank of base technologies pillars for base technologies weights rank iiot 0.150 4 horizontal and vertical system integration 0.144 5 big data and analytics 0.158 2 cybersecurity 0.397 1 the cloud 0.152 3 augmented reality, which integrates real operations and simulation, is a core technology for building a smart manufacturing environment and a crucial innovation of i4.0 (oztemel & gursev, 2020). augmented reality is successfully utilized in fields like maintenance, operations, training, and quality control. though simulation is a more traditional operation employed in industry for decades, our results show that professionals give priority to simulation. table 5 ahp weights and rank of the smart operations pillars for smart operations weights rank simulation 0.670 1 augmented reality 0.330 2 autonomous robots and additive manufacturing are state-of-the-art technologies and the most distinguishing innovations of i4.0. though they have similar significance, autonomous robots are seen as a more important element of i4.0. autonomous robots represent more conventional technologies enriched with i4.0 technologies while additive manufacturing is new in mass or end-user destined production and is still perceived as a tool in the prototyping of a designed part or product. this implies that the experts give more importance to conventional manufacturing processes with autonomous robots over additive manufacturing. table 6 ahp weights and rank of the smart technologies pillars for smart technologies weights rank autonomous robots 0.560 1 additive manufacturing 0.440 2 evaluations of dimensions and pillars shed light on the conceptual design and architecture of an i4.0 manufacturing environment. the assessments made so far reflect ijahp article: akman, karaman/pillars in the making, industry 4.0 on the horizon international journal of the analytic hierarchy process 262 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.839 the approaches of experts to i4.0 in general in defining a roadmap to i4.0. the following parts assessing the sub-pillars focus more on tangible assets in forging the operating processes of i4.0. hardware, software, equipment, machines, processes, or technologies pertaining to each sub-pillar can be observed at various maturity levels on shop floors of companies that plan to keep up with the pace of i4.0 technologies. the expert views can enable practitioners or researchers to assess the current infrastructure, decide on the gaps, and plan for the next stages in projects and investments. the iiot consists of various sets of software, hardware, and technologies used in four layers which are connected and integrated via the internet or intranet to support smart production processes. the iiot layers can combine technologies with different maturity levels, but interoperability is the key to successful operation (alcácer & cruz-machado, 2019). the physical layer, the first element to gather raw data from the source where it is produced, is viewed as a sub-pillar with the most significance for the successful operation of iiot followed by the service layer where data is stored, interface layer which serves as the interface for users and network layer which transmits data between the terminals (table 7). table 7 ahp weights and rank of the iiot sub-pillars for iiot weights rank physical layer 0.325 1 network layer 0.218 4 service layer 0.230 2 interface layer 0.227 3 liao et al. (2017) concluded in their research on i4.0 literature that 20.5% of 479 papers included integration factors, while 45 papers paid attention to vertical, 39 papers to horizontal, and 23 papers to end-to-end integration. our results showed that the i4.0 professionals view end-to-end integration as important which shows more focus on this factor is needed in further studies. vertical integration, which mostly includes it infrastructure within a company, is seen as the second factor which implies that corporations need to integrate internal processes before they join external value chains (table 8). achieving this is particularly important in switching to i4.0 as most companies lack successful internal process integration and some even lack functional units like in the engineering department (rüssmann et al., 2015). ijahp article: akman, karaman/pillars in the making, industry 4.0 on the horizon international journal of the analytic hierarchy process 263 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.839 table 8 ahp weights and rank of the horizontal and vertical system integration sub-pillars for horizontal and vertical system integration weights rank horizontal integration 0.224 3 vertical integration 0.275 2 end-to-end integration 0.501 1 the hierarchical importance of the big data and analytics’ sub-pillars is viewed as data management, data mining, and self-organized manufacturing. most commonly, data is produced in the physical systems, and stored and conveyed within cyber systems. large volumes of diverse data are produced, and this data is frequently unstructured (qi & tao, 2018). for example, data collected in different workstations of a manufacturing facility may contain numerous parameters like temperature, pressure, velocity, thickness, etc. that changes in seconds which makes collecting useful data sophisticated. the data management sub-pillar is seen as significantly important which includes data classification, architecture, and collection (table 9). table 9 ahp weights and rank of big data and analytics sub-pillars for big data and analytics weights rank data management 0.415 1 data mining 0.297 2 self-organized manufacturing 0.288 3 the results reveal that experts view access controls to data resources as the most important factor of cybersecurity. preventing malicious intervention as well as being user-friendly are key factors of access controls. communication controls which include data transmission through networks are followed by physical controls setting access standards to tangible assets (table 10). table 10 ahp weights and rank of cybersecurity sub-pillars for cybersecurity weights rank physical controls 0.271 3 access controls 0.398 1 communication controls 0.331 2 ijahp article: akman, karaman/pillars in the making, industry 4.0 on the horizon international journal of the analytic hierarchy process 264 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.839 saas, employed for processing data, is considered the first sub-pillar of the cloud, followed by iaas which stores data, and paas which runs applications (table 11). processing data gathered in the cloud or transmitted to the cloud is important as it is the main role of a cloud. table 11 ahp weights and rank of the cloud sub-pillars for the cloud weights rank infrastructure as a service (iaas) 0.321 2 platform as a service (paas) 0.309 3 software as a service (saas) 0.370 1 the significance of the simulation sub-pillars is revealed as virtual mro, virtual production, and virtual prototyping, respectively (table 12). the first expectation from simulation is the prediction of proactive maintenance followed by the design and optimization of cps and process development. table 12 ahp weights and rank of simulation sub-pillars for simulation weights rank virtual prototyping 0.256 3 virtual production 0.328 2 virtual mro 0.416 1 when we look at the sub-pillars of augmented reality, operations is the most significant factor followed by training, service, and manufacturing which have almost equal importance (table 13). the outcome is not surprising as operations describes the immediate interface between the users and augmented reality systems that play a crucial role in the success of the results. ijahp article: akman, karaman/pillars in the making, industry 4.0 on the horizon international journal of the analytic hierarchy process 265 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.839 table 13 ahp weights and rank of augmented reality sub-pillars for augmented reality weights rank training 0.177 2 design 0.131 5 manufacturing 0.173 4 operations 0.227 1 service 0.176 3 sales and marketing 0.115 6 interaction with humans is viewed as more important than interaction with one another in assessing autonomous robots (table 14). this finding should be considered when designing the new generation robots compatible with i4.0. table 14 ahp weights and rank of autonomous robots sub-pillars for autonomous robots weights rank interaction with one another 0.353 2 interaction with humans 0.647 1 mass manufacturing employing additive manufacturing is a technology unique to the i4.0 environment. the most important factors for building additive manufacturing are vat polymerization, sheet lamination, material jetting, and powder bed fusion (table 15), respectively. table 15 ahp weights and rank of additive manufacturing sub-pillars for additive manufacturing weights rank binder jetting 0.133 5 directed energy deposition 0.125 7 material extrusion 0.128 6 material jetting 0.152 2 powder bed fusion 0.145 4 sheet lamination 0.152 2 vat polymerization 0.165 1 ijahp article: akman, karaman/pillars in the making, industry 4.0 on the horizon international journal of the analytic hierarchy process 266 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.839 the interdependence between the dimensions, the pillars, and the sub-pillars has been analyzed by the ep as an initial stage of model setting. the paper aims to evaluate the importance of factors in building an i4.0 manufacturing environment, which combines three dimensions, nine pillars, and sub-pillars with distinct technological patterns and maturity levels. obviously, the experts each have distinctive perceptions about the pillars, and evaluating their significance ranks entails a thorough disparity. the interdependence across different i4.0 pillars and evaluations of experts displays diversified behavior. the ep put forth that there is no significant interdependence between the dimensions, the criteria, and the subcriteria. the conclusion of the initial evaluation demonstrated that the ahp method is the most appropriate for the problem, and it was chosen in model and solution. 5. conclusions i4.0, also considered the fourth industrial revolution, is a new era of connecting people, equipment, processes, supply chains, and other participants of value creation as a result of the advances in digital technologies. this research proposes a practical and robust mcdm model to support the evaluation of factors in building or assessing the i4.0 system. the study utilized a pairwise survey based on dimensions, pillars, and sub-pillars of i4.0 technologies. the overwhelming majority of literature that taxonomically defines the i4.0 paradigm agrees that i4.0 integrates the nine basic pillars elaborated in this study. the study is the first to focus on the nine pillars of i4.0 in a mcdm context to the best of our knowledge. hence, the findings of the study are valuable in setting forth the ranks of the pillars and the alignment of projects in transforming to i4.0. the results revealed that base technologies are the most significant in configuring i4.0 followed by smart operations and smart technologies. cybersecurity, simulation, and autonomous robots are the leading pillars of base technologies, smart operations, and smart technologies, respectively. the sub-pillars are elements such as technologies, equipment, plcs, sensors, software, or hardware building each pillar. the model also used the evaluation to disentangle the sub-pillars’ comparative significance in the system. the criteria of i4.0 were assessed by the ahp methodology. the findings reflect the perceptions of i4.0 professionals in evaluating the components and can provide a practical roadmap for the business community as they switch to i4.0. the results of the i4.0 system generated are dependent on the configuration of the pillars. it is complicated to design an optimal i4.0 manufacturing process as each pillar has a varied maturity level and the outcomes of different combinations can be hard to predict. the model we propose is applicable at the corporate level among the i4.0 professionals that make decisions about investments and the study has the potential to trigger new research to elucidate the paradigm of i4.0. although the ahp model is comprehensive, the results showed the opinions of experts in various industries which can be considered a limitation as each sector has different features necessitating a different i4.0 architecture. further studies can research the comparative impact of each pillar to support i4.0 technology selection decisions in sectoral clusters to ameliorate this limitation. most i4.0 professionals are experts on just one specific or a few pillars which made it complicated to find ijahp article: akman, karaman/pillars in the making, industry 4.0 on the horizon international journal of the analytic hierarchy process 267 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.839 respondents with a complete understanding of all of the pillars. the combining nature of the ahp model helped alleviate this limitation. a benchmarking study between corporations engaged in i4.0 to compare the systems in operation would be a valuable addition to the literature. as the ahp enables quantitative results from the perceptions of experts, such a study can support the realization of an optimal configuration and highlight areas of improvement. as each pillar has an indispensable impact on the output of i4.0, further studies can focus on the elements of the individual pillars in an mcdm context. another extension can be employing different applicable methodologies to assess i4.0 technologies. ijahp article: akman, karaman/pillars in the making, industry 4.0 on the horizon international journal of the analytic hierarchy process 268 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.839 references additive manufacturing research group (2019). the 7 categories of additive manufacturing. retrieved from https://www.lboro.ac.uk/research/amrg/about/the7categoriesofadditivemanufacturing/ akman, e., & dagdeviren, m. (2018). discovering what makes a sme website good for international trade. technological and economic development of economy, 24(3), 10631079. doi: https://doi.org/10.3846/20294913.2016.1266709 alcácer, v., & cruz-machado, v. (2019). scanning the industry 4.0: a literature review on technologies for manufacturing systems. engineering science and technology, an international journal, 22(3), 899-919. doi: https://doi.org/10.1016/j.jestch.2019.01.006 almada-lobo, f. (2015). the industry 4.0 revolution and the future of manufacturing execution systems (mes). journal of innovation management, 3(4), 16-21. doi: https://doi.org/10.24840/2183-0606_003.004_0003 bahrin, m. a. k., othman, m. f., azli, n. n., & talib, m. f. (2016). industry 4.0: a review on industrial automation and robotic. jurnal teknologi, 78(6-13), 137-143. doi: https://doi.org/10.11113/jt.v78.9285 bai, c., dallasega, p., orzes, g., & sarkis, j. (2020). industry 4.0 technologies assessment: a sustainability perspective. international journal of production economics, 229, 107776. doi: https://doi.org/10.1016/j.ijpe.2020.107776 blanco-novoa, o., fernandez-carames, t. m., fraga-lamas, p., & vilar-montesinos, m. a. (2018). a practical evaluation of commercial industrial augmented reality systems in an industry 4.0 shipyard. ieee access, 6, 8201-8218. doi: https://doi.org/10.1109/access.2018.2802699 bordeleau, f. e., mosconi, e., & santa-eulalia, l. a. (2018, january). business intelligence in industry 4.0: state of the art and research opportunities. in proceedings of the 51st hawaii international conference on system sciences. doi: https://doi.org/10.24251/hicss.2018.495 brettel, m., friederichsen, n., keller, m., & rosenberg, m. (2014). how virtualization, decentralization and network building change the manufacturing landscape: an industry 4.0 perspective. international journal of mechanical, industrial science and engineering, 8(1), 37-44. doi: https://doi.org/10.5281/zenodo.1336426 calabrese, a., levialdi ghiron, n., & tiburzi, l. (2020). ‘evolutions’ and ‘revolutions’ in manufacturers’ implementation of industry 4.0: a literature review, a multiple case study, and a conceptual framework. production planning & control, 1-15. doi: https://doi.org/10.1080/09537287.2020.1719715 ijahp article: akman, karaman/pillars in the making, industry 4.0 on the horizon international journal of the analytic hierarchy process 269 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.839 chong, l., ramakrishna, s., & singh, s. (2018). a review of digital manufacturing-based hybrid additive manufacturing processes. the international journal of advanced manufacturing technology, 95(5-8), 2281-2300. doi: https://doi.org/10.1007/s00170017-1345-3 contreras-masse, r., ochoa-zezzatti, a., garcía, v., & pérez-dominguez, l. (2020). implementing a novel use of multicriteria decision analysis to select iiot platforms for smart manufacturing. symmetry, 12(3), 368. doi: https://doi.org/10.3390/sym12030368 da silva, v. l., kovaleski, j. l., & pagani, r. n. (2019). technology transfer in the supply chain oriented to industry 4.0: a literature review. technology analysis & strategic management, 31(5), 546-562. doi: https://doi.org/10.1080/09537325.2018.1524135 dalenogare, l. s., benitez, g. b., ayala, n. f., & frank, a. g. (2018). the expected contribution of industry 4.0 technologies for industrial performance. international journal of production economics, 204, 383-394. doi: https://doi.org/10.1016/j.ijpe.2018.08.019 deloitte development llc (firm). (2020). the fourth industrial revolution: at the intersection of readiness and responsibility. dopico, m., gomez, a., de la fuente, d., garcía, n., rosillo, r., & puche, j. (2016). a vision of industry 4.0 from an artificial intelligence point of view. in proceedings on the international conference on artificial intelligence (icai) (p. 407). the steering committee of the world congress in computer science, computer engineering and applied computing (worldcomp). erdogan, m., ozkan, b., karasan, a., & kaya, i. (2018). selecting the best strategy for industry 4.0 applications with a case study. in industrial engineering in the industry 4.0 era (pp. 109-119). springer, cham. flatt, h., schriegel, s., jasperneite, j., trsek, h., & adamczyk, h. (2016). analysis of the cyber-security of industry 4.0 technologies based on rami 4.0 and identification of requirements. in 2016 ieee 21st international conference on emerging technologies and factory automation (etfa) (pp. 1-4). ieee. doi: https://doi.org/10.1109/etfa.2016.7733634 fraga-lamas, p., fernández-caramés, t. m., blanco-novoa, ó., & vilar-montesinos, m. a. (2018). a review on industrial augmented reality systems for the industry 4.0 shipyard. ieee access, 6, 13358-13375. doi: https://doi.org/10.1109/access.2018.2808326 frank, a. g., dalenogare, l. s., & ayala, n. f. (2019). industry 4.0 technologies: implementation patterns in manufacturing companies. international journal of production economics, 210, 15-26. doi: https://doi.org/10.1016/j.ijpe.2019.01.004 ijahp article: akman, karaman/pillars in the making, industry 4.0 on the horizon international journal of the analytic hierarchy process 270 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.839 gauvreau, r. p. (2011). easy access to everything-or how the web can bring it all together. iron & steel technology, 8(1), 63-69. ghobakhloo, m. (2020). determinants of information and digital technology implementation for smart manufacturing. international journal of production research, 58(8), 2384-2405. doi: https://doi.org/10.1080/00207543.2019.1630775 gilchrist, a. (2016). industry 4.0: the industrial internet of things. apress. gonzalez, a. g., alves, m. v., viana, g. s., carvalho, l. k., & basilio, j. c. (2018). supervisory control-based navigation architecture: a new framework for autonomous robots in industry 4.0 environments. ieee transactions on industrial informatics, 14(4), 1732-1743. doi: https://doi.org/10.1109/tii.2017.2788079 he, h., maple, c., watson, t., tiwari, a., mehnen, j., jin, y., & gabrys, b. (2016). the security challenges in the iot enabled cyber-physical systems and opportunities for evolutionary computing & other computational intelligence. in 2016 ieee congress on evolutionary computation (cec) (pp. 1015-1021). ieee. doi: https://doi.org/10.1109/cec.2016.7743900 hermann, m., pentek, t., & otto, b. (2016, january). design principles for industry 4.0 scenarios. in 2016 49th hawaii international conference on system sciences (hicss) (pp. 3928-3937). ieee. doi: https://doi.org/10.1109/hicss.2016.488 hozdic, e. (2015). smart factory for industry 4.0: a review. international journal of modern manufacturing technologies, 7(1), 28-35. international organization for standardization (2015). additive manufacturing -general principles -terminology (iso/astm 52900:2015 (astm f2792)). retrieved from https://www.iso.org/standard/69669.html javaid, m., haleem, a., vaishya, r., bahl, s., suman, r., & vaish, a. (2020). industry 4.0 technologies and their applications in fighting covid-19 pandemic. diabetes & metabolic syndrome: clinical research & reviews, 14(4), 419-422. doi: https://doi.org/10.1016/j.dsx.2020.04.032 kagermann, h., wahlster, w., & helbig, j. (2013). recommendations for implementing the strategic initiative industrie 4.0: securing the future of german manufacturing industry; final report of the industrie 4.0 working group. forschungsunion. kamble, s. s., gunasekaran, a., & gawankar, s. a. (2018). sustainable industry 4.0 framework: a systematic literature review identifying the current trends and future perspectives. process safety and environmental protection, 117, 408-425. doi: https://doi.org/10.1016/j.psep.2018.05.009 ijahp article: akman, karaman/pillars in the making, industry 4.0 on the horizon international journal of the analytic hierarchy process 271 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.839 karaman, a. s., & akman, e. (2018). taking-off corporate social responsibility programs: an ahp application in airline industry. journal of air transport management, 68, 187-197. doi: https://doi.org/10.1016/j.jairtraman.2017.06.012 kaur, h., & singh, s. p. (2018). heuristic modeling for sustainable procurement and logistics in a supply chain using big data. computers & operations research, 98, 301321. doi: https://doi.org/10.1016/j.cor.2017.05.008 kobara, k. (2016). cyber physical security for industrial control systems and iot. ieice transactions on information and systems, 99(4), 787-795. doi: https://doi.org/10.1587/transinf.2015ici0001 lasi, h., fettke, p., kemper, h. g., feld, t., & hoffmann, m. (2014). industry 4.0. business & information systems engineering, 6(4), 239-242. doi: https://doi.org/10.1007/s11576-014-0424-4 lee, j., bagheri, b., & kao, h. a. (2014a, july). recent advances and trends of cyberphysical systems and big data analytics in industrial informatics. in proceeding of international conference on industrial informatics (indin) (pp. 1-6). lee, j., kao, h. a., & yang, s. (2014b). service innovation and smart analytics for industry 4.0 and big data environment. procedia cirp, 16, 3-8. doi: https://doi.org/10.1016/j.procir.2014.02.001 liao, y., deschamps, f., loures, e. d. f. r., & ramos, l. f. p. (2017). past, present and future of industry 4.0-a systematic literature review and research agenda proposal. international journal of production research, 55(12), 3609-3629. doi: https://doi.org/10.1080/00207543.2017.1308576 lu, y. (2017). industry 4.0: a survey on technologies, applications and open research issues. journal of industrial information integration, 6, 1-10. doi: https://doi.org/10.1016/j.jii.2017.04.005 luthra, s., & mangla, s. k. (2018). evaluating challenges to industry 4.0 initiatives for supply chain sustainability in emerging economies. process safety and environmental protection, 117, 168-179. doi: https://doi.org/10.1016/j.psep.2018.04.018 manfredi, d., calignano, f., krishnan, m., canali, r., ambrosio, e. p., biamino, s., ... & fino, p. (2014). additive manufacturing of al alloys and aluminium matrix composites (amcs). light metal alloys applications, 11, 3-34. doi: https://doi.org/10.5772/58534 marr, b. (2015). big data: 20 mind-boggling facts everyone must read. forbes magazine. marr, b. (2016). what everyone must know about industry 4.0. forbes tech. ijahp article: akman, karaman/pillars in the making, industry 4.0 on the horizon international journal of the analytic hierarchy process 272 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.839 mercer, d., (2019). global connected and iot device forecast update report, strategy analytics. mohammad, e., albarakah, l., kudair, s., & karaman, a. s. (2021, march). evaluating the industry 4.0 readiness of manufacturing companies: a case study in kuwait. in 11th international conference on industrial engineering and operations management, ieom 2021, singapore (pp. 6625-6636). ieom society. moreno, a., velez, g., ardanza, a., barandiaran, i., de infante, á. r., & chopitea, r. (2017). virtualisation process of a sheet metal punching machine within the industry 4.0 vision. international journal on interactive design and manufacturing (ijidem), 11(2), 365-373. doi:10.1007/s12008-016-0319-2. motyl, b., baronio, g., uberti, s., speranza, d., & filippi, s. (2017). how will change the future engineers’ skills in the industry 4.0 framework? a questionnaire survey. procedia manufacturing, 11, 1501-1509. doi: https://doi.org/10.1016/j.promfg.2017.07.282 mourtzis, d., doukas, m., & bernidaki, d. (2014). simulation in manufacturing: review and challenges. procedia cirp, 25, 213-229. doi: https://doi.org/10.1016/j.procir.2014.10.032 mourtzis, d., zogopoulos, v., & vlachou, e. (2017). augmented reality application to support remote maintenance as a service in the robotics industry. procedia cirp, 63, 4651. doi: https://doi.org/10.1016/j.procir.2017.03.154 müller, j. m., buliga, o., & voigt, k. i. (2018). fortune favors the prepared: how smes approach business model innovations in industry 4.0. technological forecasting and social change, 132, 2-17. doi: https://doi.org/10.1016/j.techfore.2017.12.019 negahban, a., & smith, j. s. (2014). simulation for manufacturing system design and operation: literature review and analysis. journal of manufacturing systems, 33(2), 241261. doi: https://doi.org/10.1016/j.jmsy.2013.12.007 negri, e., fumagalli, l., & macchi, m. (2017). a review of the roles of digital twin in cps-based production systems. procedia manufacturing, 11, 939-948. doi: https://doi.org/10.1016/j.promfg.2017.07.198 öberseder, m., schlegelmilch, b. b. and murphy, p. e. (2013). csr practices and consumer perceptions. journal of business research, 66(10), 1839-1851. doi: https://doi.org/10.1016/j.jbusres.2013.02.005 oztemel, e., & gursev, s. (2020). literature review of industry 4.0 and related technologies. journal of intelligent manufacturing, 31(1), 127-182. doi: https://doi.org/10.1007/s10845-018-1433-8 ijahp article: akman, karaman/pillars in the making, industry 4.0 on the horizon international journal of the analytic hierarchy process 273 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.839 posada, j., toro, c., barandiaran, i., oyarzun, d., stricker, d., de amicis, r., ... & vallarino, i. (2015). visual computing as a key enabling technology for industrie 4.0 and industrial internet. ieee computer graphics and applications, 35(2), 26-40. doi: https://doi.org/10.1109/mcg.2015.45 prause, g. (2015). sustainable business models and structures for industry 4.0. journal of security & sustainability issues, 5(2), 159-169. doi: https://doi.org/10.9770/jssi.2015.5.2(3) press, g. (2013). a very short history of big data. forbes tech magazine. qi, q., & tao, f. (2018). digital twin and big data towards smart manufacturing and industry 4.0: 360 degree comparison. ieee access, 6, 3585-3593. doi: https://doi.org/0.1109/access.2018.2793265 rodič, b. (2017). industry 4.0 and the new simulation modelling paradigm. organizacija, 50(3), 193-207. doi: https://doi.org/10.1515/orga-2017-0017 rosa, p., sassanelli, c., urbinati, a., chiaroni, d., & terzi, s. (2020). assessing relations between circular economy and industry 4.0: a systematic literature review. international journal of production research, 58(6), 1662-1687. doi: https://doi.org/10.1080/00207543.2019.1680896 rüßmann, m., lorenz, m., gerbert, p., waldner, m., justus, j., engel, p., & harnisch, m. (2015). industry 4.0: the future of productivity and growth in manufacturing industries. boston consulting group, 9(1), 54-89. saaty, t. (1980). the analytic hierarchy process: planning, priority setting, resource allocation. pittsburgh pa: university of pittsburgh. sadeghi, a. r., wachsmann, c., & waidner, m. (2015, june). security and privacy challenges in industrial internet of things. in 2015 52nd acm/edac/ieee design automation conference (dac) (pp. 1-6). ieee. doi: https://doi.org/10.1145/2744769.2747942 schleich, b., anwer, n., mathieu, l., & wartzack, s. (2017). shaping the digital twin for design and production engineering. cirp annals, 66(1), 141-144. doi: https://doi.org/10.1016/j.cirp.2017.04.040 singh, j., garg, d., & luthra, s. (2018). an analysis of critical success factors for industry 4.0: an application of analytical hierarchy process. industrial engineering journal, 11(9), 1-15. sony, m., & naik, s. (2020). critical factors for the successful implementation of industry 4.0: a review and future research direction. production planning & control, 31(10), 799-815. doi: https://doi.org/10.1080/09537287.2019.1691278 ijahp article: akman, karaman/pillars in the making, industry 4.0 on the horizon international journal of the analytic hierarchy process 274 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.839 stock, t., & seliger, g. (2016). opportunities of sustainable manufacturing in industry 4.0. procedia cirp, 40, 536-541. doi: https://doi.org/10.1016/j.procir.2016.01.129 tao, f., qi, q., liu, a., & kusiak, a. (2018). data-driven smart manufacturing. journal of manufacturing systems, 48, 157-169. doi: https://doi.org/10.1016/j.jmsy.2018.01.006 thames, l., & schaefer, d. (2016). software-defined cloud manufacturing for industry 4.0. procedia cirp, 52, 12-17. doi: https://doi.org/10.1016/j.procir.2016.07.041 tofail, s. a., koumoulos, e. p., bandyopadhyay, a., bose, s., o’donoghue, l., & charitidis, c. (2018). additive manufacturing: scientific and technological challenges, market uptake and opportunities. materials today, 21(1), 22-37. doi: https://doi.org/10.1016/j.mattod.2017.07.001 uhlemann, t. h. j., lehmann, c., & steinhilper, r. (2017). the digital twin: realizing the cyber-physical production system for industry 4.0. procedia cirp, 61, 335-340. doi: https://doi.org/10.1016/j.procir.2016.11.152 vaidya, s., ambad, p., & bhosle, s. (2018). industry 4.0–a glimpse. procedia manufacturing, 20, 233-238. doi: https://doi.org/10.1016/j.promfg.2018.02.034 vogel-heuser, b., & hess, d. (2016). guest editorial industry 4.0–prerequisites and visions. ieee transactions on automation science and engineering, 13(2), 411-413. doi: https://doi.org/10.1109/tase.2016.2523639 vrochidis, s., papadopoulos, s., moumtzidou, a., sidiropoulos, p., pianta, e., & kompatsiaris, i. (2010). towards content-based patent image retrieval: a framework perspective. world patent information, 32(2), 94-106. doi: https://doi.org/10.1016/j.wpi.2009.05.010 waidner, m., & kasper, m. (2016, march). security in industrie 4.0-challenges and solutions for the fourth industrial revolution. in 2016 design, automation & test in europe conference & exhibition (date) (pp. 1303-1308). ieee. wan, j., tang, s., shu, z., li, d., wang, s., imran, m., & vasilakos, a. v. (2016). software-defined industrial internet of things in the context of industry 4.0. ieee sensors journal, 16(20), 7373-7380. doi: https://doi.org/10.1109/jsen.2016.2565621 wang, s., wan, j., zhang, d., li, d., & zhang, c. (2016). towards smart factory for industry 4.0: a self-organized multi-agent system with big data-based feedback and coordination. computer networks, 101, 158-168. doi: https://doi.org/10.1016/j.comnet.2015.12.017 wang, l., & wang, g. (2016). big data in cyber-physical systems, digital manufacturing and industry 4.0. international journal of engineering and manufacturing (ijem), 6(4), 1-8. doi: https://doi.org/10.5815/ijem.2016.04.01 ijahp article: akman, karaman/pillars in the making, industry 4.0 on the horizon international journal of the analytic hierarchy process 275 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.839 weyer, s., meyer, t., ohmer, m., gorecky, d., & zühlke, d. (2016). future modeling and simulation of cps-based factories: an example from the automotive industry. ifacpapersonline, 49(31), 97-102. doi: https://doi.org/10.1016/j.ifacol.2016.12.168 weyer, s., schmitt, m., ohmer, m., & gorecky, d. (2015). towards industry 4.0standardization as the crucial challenge for highly modular, multi-vendor production systems. ifac-papersonline, 48(3), 579-584. doi: https://doi.org/10.1016/j.ifacol.2015.06.143 wollschlaeger, m., sauter, t., & jasperneite, j. (2017). the future of industrial communication: automation networks in the era of the internet of things and industry 4.0. ieee industrial electronics magazine, 11(1), 17-27. doi: https://doi.org/10.1109/mie.2017.2649104 xu, j., huang, e., hsieh, l., lee, l. h., jia, q. s., & chen, c. h. (2016). simulation optimization in the era of industrial 4.0 and the industrial internet. journal of simulation, 10(4), 310-320. doi: https://doi.org/10.1057/s41273-016-0037-6 zawadzki, p., & żywicki, k. (2016). smart product design and production control for effective mass customization in the industry 4.0 concept. management and production engineering review, 7(3), 105-112. doi: https://doi.org/10.1515/mper-2016-0030 zhong, r. y., xu, x., klotz, e., & newman, s. t. (2017). intelligent manufacturing in the context of industry 4.0: a review. engineering, 3(5), 616-630. doi: https://doi.org/10.1016/j.eng.2017.05.015 zhou, k., liu, t., & zhou, l. (2015, august). industry 4.0: towards future industrial opportunities and challenges. in 2015 12th international conference on fuzzy systems and knowledge discovery (fskd) (pp. 2147-2152). ieee. doi: https://doi.org/10.1109/fskd.2015.7382284 ijahp article: akman, karaman/pillars in the making, industry 4.0 on the horizon international journal of the analytic hierarchy process 276 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.839 appendix pillars in the making, industrie 4.0 on the horizon instructions: please compare the below dimension / pillar / sub-pillar couplet based on their relative importance to each other. only one box is going to be checked in the comparisons. if the item on the left is more important than the item on the right, please use the scale on the left of "1" and indicate its relative importance. if the item on the right is more important than the item on the left, please use the scale on the right of "1" and indicate its relative importance. if the items have equal importance, than check the box for "1". intensity of importance definition 1 equal importance 3 moderate importance 5 strong importance 7 very strong importance 9 absolute importance 2, 4, 6, 8 can be used to express intermediate values. 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 base technologies smart operations base technologies smart technologies smart operations smart technologies 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 the industrial internet of things (iiot) horizontal and vertical system integration the industrial internet of things (iiot) big data and analytics the industrial internet of things (iiot) cybersecurity the industrial internet of things (iiot) the cloud horizontal and vertical system integration big data and analytics horizontal and vertical system integration cybersecurity horizontal and vertical system integration the cloud big data and analytics cybersecurity big data and analytics the cloud cybersecurity the cloud smart operations 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 simulation augmented reality smart technologies 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 autonomous robots additive manufacturing base technologies the industrial internet of things (iiot) 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 physical layer network layer physical layer service layer physical layer interface layer network layer service layer network layer interface layer service layer interface layer horizontal and vertical system integration 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 horizontal integration vertical integration horizontal integration end-to-end integration vertical integration end-to-end integration big data and analytics 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 data management data mining data management self-organized manufacturing data mining self-organized manufacturing cybersecurity 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 physical controls access controls physical controls communication controls access controls communication controls the cloud 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 infrastructure as a service (iaas) platform as a service (paas) infrastructure as a service (iaas) software as a service (saas) platform as a service (paas) software as a service (saas) comparing the dimensions comparing the pillars base technologies comparing the sub-pillars comparing the pillars comparing the pillars ijahp article: akman, karaman/pillars in the making, industry 4.0 on the horizon international journal of the analytic hierarchy process 277 vol. 13 issue 2 2021 issn 1936-6744 https://doi.org/10.13033/ijahp.v13i2.839 smart operations simulation 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 virtual prototyping virtual production virtual prototyping virtual maintenance, repair and operations (mro) virtual production virtual maintenance, repair and operations (mro) augmented reality 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 training design training manufacturing training operations training service training sales and marketing design manufacturing design operations design service design sales and marketing manufacturing operations manufacturing service manufacturing sales and marketing operations service operations sales and marketing service sales and marketing smart technologies autonomous robots 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 interaction with one another interaction with humans additive manufacturing 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 binder jetting directed energy deposition binder jetting material extrusion binder jetting material jetting binder jetting powder bed fusion binder jetting sheet lamination binder jetting vat polymerisation directed energy deposition material extrusion directed energy deposition material jetting directed energy deposition powder bed fusion directed energy deposition sheet lamination directed energy deposition vat polymerisation material extrusion material jetting material extrusion powder bed fusion material extrusion sheet lamination material extrusion vat polymerisation material jetting powder bed fusion material jetting sheet lamination material jetting vat polymerisation powder bed fusion sheet lamination powder bed fusion vat polymerisation sheet lamination vat polymerisation comparing the sub-pillars comparing the sub-pillars microsoft word 5b_costa_post-occupancy_vol2_issue1.docx ijahp article: costa, correa / construction of an ahp-based model to catch criteria weights in post-occupancy evaluation international journal of the analytic hierarchy process 30 vol. 2, issue 1, 2010 issn 1936-6744 construction of an ahp-based model to catch criteria weights in post-occupancy evaluation helder gomes costa fluminense federal university niteroi, rj, brazil e-mail: hgc@latec.uff.br pedro de seixas correa fluminense federal university niteroi, rj, brazil e-mail: pseixas@seicor.com.br abstract the comprehension of customer’s perceptions is a necessity in all types of business. in the context of architecture and civil engineer, post-occupancy evaluation is the main strategy adopted to catch customer’s perception about building performance. usually, questionnaires adopted in poe, take in account subjective criteria whose importance degree are estimated in an arbitrary way. the present work proposes an original variation of poe which makes it able to evaluate criteria degree importance, taking into account the analysis of coherency degree of the arbitrary evaluations. the proposal was applied in a case which results showed the applicability of the proposal and, also, the main difficulties found in its implementation. keywords: ahp, apo, epo, evaluation, multicriteria, mcda. 1. introduction a building is a multifunctional and complex environment, since it has to attend several human necessities such as protection from atmospheric phenomena, patrimonial security, comfort and other subjective needs. post-occupancy evaluation (poe) contributes to improve future buildings by identifying failures and no expectative attendance. it deals with subjective evaluation of the environment during its occupation or using phase. there are several researches about poe in a large variety of fields. those works measure evaluation as perceived by the building occupants; that is in a subjective context. despite this, no previous work was found that links poe with any multicriteria approach, which has been developed since the later ’60s to deal with problems under subjective evaluations. 1.1 objective the present work intends to fulfill the lack of linking highlighted above presenting a model that uses ahp (analytic hierarchy process) to help defining weights of criteria used in an application of poe to a residential building, under the viewer of multiple evaluators. human madden corresponding author rob typewritten text rob typewritten text rob typewritten text rob typewritten text http://dx.doi.org/10.13033/ijahp.v2i1.71 rob typewritten text rob typewritten text rob typewritten text ijahp article: costa, correa / construction of an ahp-based model to catch criteria weights in post-occupancy evaluation international journal of the analytic hierarchy process 31 vol. 2, issue 1, 2010 issn 1936-6744 1.2 why ahp the choice of ahp is due to the fact that this is the unique method that allows measuring the consistency degree of judgments issued by evaluators. 1.3 relevance one of the challenges faced by the construction industry is to get a closer relationship between customers and builders. thus, this article is justified by the relevance of poe in the context of construction. also, it responds to the calling for models that integrate poe to operational research methods able to map inconsistencies in value judgments. 1.4 structure of this paper in the following section, it is presented a short description of poe main central concepts, and then, in the next one, a brief description of ahp, followed by the model construction explanation and application. finally, the conclusion of the research highlights the main contribution and constraints accomplished by the modeling discussed. 2. post-occupancy evaluation it is presented here a brief summary of the main concepts on poe. for detailed discussion about this theme, the reading of zimring (1987), preiser et al (1988), bordass and leaman (2005), zhao (2007), hadjri and crozier (2009) is recommended. it is worth noting that council (2001) presented an important survey on the state of the art of poe. in addtion to presenting the different streams of poe development, this study highlights the fact that poe focuses on the needs of the building occupants under multiple aspects such as health, safety, functionality, efficiency, psychological comfort, aesthetics, quality and satisfaction. therefore, poe meets both behavioral assessments (under the occupant view) and technical performance of the built environment. the possibility of linking poe with psychology is surveyed in ornstein et al. (2009), who highlighted the advantages of this joint. the poe can be accomplished in several ways: interviews, photographic records, films, technical surveys or questionnaires filled by users or professionals involved in the construction industry. in brazil, the center of studies and innovation in building at the federal university of rio grande do sul (nori / ufrgs) in partnership with the office for support of small enterprises (sebrae), also in rio grande do sul, built a system of quality and productivity indicators for construction. the main features of this system are reported in some texts such as formoso & lantrelme (2000) and navarro (2006) this system developed a questionnaire that takes into account the brazilian behavioral assessment, covering aspects regarding the care and quality of the building, focusing on residential use. in the present work, the criteria adopted for modeling are based on that questionnaire. 3. ahp: method and core concepts since ahp is a well known method, a deeper description is not necessary. for the reader who wants to know more about this method, it is suggested the reading of saaty (1980), barba-romero & pomero, (1997) (in spanish) or costa (2006) (in portuguese). 3.1 main principles ahp is a method developed to solve problems of choice (selection of one alternative among a set of alternatives). one of the main advantages of using this method is the possibility to assess the degree of consistency of value judgments. it is based on 3 principles of analytical thinking: ijahp article: costa, correa / construction of an ahp-based model to catch criteria weights in post-occupancy evaluation international journal of the analytic hierarchy process 32 vol. 2, issue 1, 2010 issn 1936-6744 construction of hierarchies: to decompose the complexity of the problem to make it more comprehensible and reasonable. setting priorities: to get the perceptions about the relationship among the elements in the hierarchy, through pair wise comparison. logical consistency: to check the consistency of the pair wise comparison. based on these principles, ahp attempts to get and treat (which differ from eliminate) the subjectivity inherent to qualitative evaluations. 3.2 construction of hierarchies the first step on an ahp-based modeling is to construct the hierarchy. to do this it is necessary to define: the focus or the overall objective; the set of alternatives or the feasible options; and the criteria set. depending on the complexity of the problem, multiple layers of criteria or sub-criteria can be used in the hierarchy. 3.3 judgments of value on the ahp, the evaluator makes pairwise comparison in each node of the hierarchy, comparing the elements in the same level of the hierarchy under the perspective of each element connected to them in a higher layer of hierarchy. the judgments are supported by the scale showed in table 1, which was primarily reported by saaty (1980). table 1. scale for pair wise assessment of preference and importance (saaty, 1980) numerical scale verbal scale 1 equal importance (preference) 3 moderate importance 5 strong importance 7 very strong importance 9 absolute importance 2, 4, 6 and 8 intermediate values it is valuable to say that this scale agrees with the main principles stated for scales done in order to collect subjective judgments, such as those reported in: likert (1932), miller (1954) and parducci (1965), kline (1996), mann et al. (1997), zagreus et al. (2004), spicer and sadler-smith (2005), simon et al. (2006), rosen and olsen (2006), creed and yin (2006) and evans et al. (2007). 3.4 prioritization in this step, ahp calculates priorities taking into account the judgments obtained in the previous step, described above. the procedures to calculate the priorities can be organized into four stages:  structure the matrices of judgments: construct, for each node in the hierarchy, a matrix that stores the judgments on that node. all of these matrices are squared, positive and reciprocal;  obtain normalized matrices: over each one of the matrices of judgments, apply a normalization procedure, obtaining the normalized matrices;  calculate the arrays of local average priorities (lap): find the averages of the columns of each normalized matrices, providing an array of priorities for each node in the hierarchy. it is relevant to highlight that for the node which corresponds to the hierarchy’s focus, the array retains the weights or priorities of the criteria set. ijahp article: costa, correa / construction of an ahp-based model to catch criteria weights in post-occupancy evaluation international journal of the analytic hierarchy process 33 vol. 2, issue 1, 2010 issn 1936-6744  obtain an array with the global average priorities (gap): combine the laps as like in a decision tree, obtaining the gap. this array (gap) keeps the overall priorities of the options. the choice must be the option which has the bigger gap. it is important to highlight that the text above is a short description of ahp, and to do it, the author used an unusual nomenclature. so, terms like lap and gap, with the meaning described above, are primarily creations reported in the present text to describe, in a short way, the procedures that take part in the ahp. therefore, they are not present in other texts which describe ahp in a more detailed way, although the concepts are the same. 3.5 analysis of consistency human judgments can be inconsistent or incoherent. ahp recognizes this fact and provides a procedure for assessing consistency degree of judgments. saaty (1980) proposed the consistency index (ci) as a measure of the degree of inconsistency in the matrix of joint membership judgments: 1 max    n n ci  where: n is the order of the matrix of the judgments; max is the largest eigenvalue of the matrix of the judgments. complementing the consistency analysis, saaty (1980) proposes a comparison between ci and a random consistency index (ri). this comparison provides a consistency ratio (cr), which is calculated according to the equation: ri ci cr  saaty (1980) proposes the acceptance of judgments with consistency ratio smaller than 0.1 (rc < 0.1). 4. modeling and case of application this section presents a modeling to integrate ahp to poe an also describes an application of this proposal. specifically, it describes the use of ahp to identify weights to poe evaluation. 4.1 object of study the experiment focuses the importance degree assigned to poe aspects by the users of residential units in a condominium sited in the district of barra da tijuca, located in the city of rio de janeiro, brazil. this area has been under a strong tendency of growth since the south region of rio de janeiro (copacabana, ipanema, leblon and their neighborhoods) has been saturated. this condominium is a building of 21 (twenty one) floors, containing 8 (eight) apartments per floor, making a total of 168 (one hundred and sixty-eight) residential units. on the ground floor, there are dispersed areas for administration and leisure (swimming pool, sauna, recreation area, ballroom). parking for residents and visitors isdistributed in the underground and ground floors. the survey was conducted 14 (fourteen) months after the building units had been delivered to their owners. ijahp article: costa, correa / construction of an ahp-based model to catch criteria weights in post-occupancy evaluation international journal of the analytic hierarchy process 34 vol. 2, issue 1, 2010 issn 1936-6744 4.2 defining the criteria and structure of the hierarchy based both on the literature review about poe and in the fact that the case was located in brazil, it was decided to adopt the aspects reported by romero and ornstein (2003) as the basis to start the definition of the set of criteria. this was due to the proximity of this work to brazilian construction and habitation particularities. the work of romero and ornestein (2003) recommended taking into account criteria reated to the following aspects when doing a poe evaluation: social areas, layout, construction, finishing, electrical, hydraulic and sanitary features and materials, maintenance, sound, thermal and visual comfort, and economic. in the process of defining the criteria set, it was also took into account the quality of the service by the construction firm in supporting and helping the users of the habitation units at the posoccupancy period. the ahp hierarchical structure of criteria was structured into twenty-five sub-criteria, grouped into five criteria. figures 1 shows the linkage structure among focus, criteria and sub-criteria in hierarchy. 4.3 data collection a questionnaire was used in this study to collect the data. the appendix shows a translation of this questionnaire to english (as it was applied in rio de janeiro, brazil, it was originally written in portuguese). as one can see, the questions presented included in the questionnaire tried to get the pairwise comparison of relative importance of the criteria using saaty’s scale. the questionnaires were distributed throughout the mailboxes of all the habitation units, one questionnaire per residence (168 questionnaires as total) accompanied by a cover letter and guidelines to make it easier to fulfill them. because of economic reasons, data collection was carried out without the presence of an interviewer. initially, it was given a deadline of 10 days for returning the questionnaires fulfilled. only 16 (sixteen) responses were received within this deadline. this number was found insufficient to achieve the objectives of the research. to avoid this difficulty, the data collection was changed to direct approach throughout interviews, which raised the number of responses to 24 (twenty four). it is worth to say that all the residents were asked to participate in the interview, although only 24 agreed to take part in the research, which has resulted in a rate response equal to 14.3%. as reported in zagreus et al. (2004), the usual response rates in poe surveys typically stays between 27% and 88%, using questions about users satisfaction under aspects like: layout, office furnishings, thermal comfort, indoor air quality, lighting, acoustics, and building cleanliness and maintenance. ijahp article: costa, correa / construction of an ahp-based model to catch criteria weights in post-occupancy evaluation international journal of the analytic hierarchy process 35 vol. 2, issue 1, 2010 issn 1936-6744 figure 1. hierarchic structure of criteria and sub-criteria. c1 – service by construction company c1s1 responsiveness c1s2 meeting deadlines c1s3 technical assistance c1s4 prestigious of the construction company c1s5 documentation and user guide c2 – financial aspects c2s1 cost of purchasing c2s2 cost of maintenance the condominium c2s3 cost of maintenance the residential unit c3 quality of the construction c3s1 areas for common use c3s2 localization of the building c3s3 external appearance of the building c3s4 performance of facade c3s5 security of the building c3s6 conformity with specifications c4 – quality of the residential unit project c4s1 layout c4s1s1 social areas c4s1s2 service areas c4s1s3 bedrooms c4s1s4 bathrooms c4s2 environmental comfort c4s2s1 indoor temperature on winter c4s2s2 indoor temperature on summer c4s2s3 lighting c4s2s4 sound insulation c4s3 quantitative and localization of electrical features c4s4 quantitative and localization of hydraulic and sanitary features c4s5 quality of indoor materials c4s5s1 tiles c4s5s2 floorings c4s5s3 – sanitary ware c4s5s4 – frames, fixtures and locks c4s5s5 paint c4s6 project flexibility c5 – quality of finish c5s1 quality and finish of electrical features c5s2 quality and finish of hydraulic and sanitary features c5s2s1 – materials finish c5s2s2 performance of hydraulic features c5s2s3 performance of sanitary features c5s3 indoor features putting c5s3s1 tiles c5s3s2 flooring c5s3s3 – frames and fixtures (doors and windows) c5s3s4 paint c5s4 waterproofing c5s5 achieving specifications for finishing ijahp article: costa, correa / construction of an ahp-based model to catch criteria weights in post-occupancy evaluation international journal of the analytic hierarchy process 36 vol. 2, issue 1, 2010 issn 1936-6744 4.4 calculation of priorities (weights of criteria and sub-criteria) the calculation of the priorities has been done in four steps: first, the relative importance of the criteria for each one of the evaluators (respondents of the questionnaire) was calculated using the algorithm of the ahp prioritization proposed by saaty (1980). after this, the consistency analysis proposed by saaty (1980), was applied over the data to avoid inconsistent judgments. adopting this procedure, the judgments of four respondents were eliminated, since their consistency ratio were bigger than 0.10. this resulted on the elimination of approximately 16.7% of the initial judgments. in a third step, taking into account only the consistent judgments, the individual judgments were aggregated through the use of the geometric mean (as reported in aczel & saaty (1983)) to obtaining a set of pairwise judgment matrixes. finally, the priorities (weights) were calculated through the applications of saaty’s algorithm over the data in the matrixes. figure 2 shows the tree with the average weight distribution of the weights obtained for each of the criteria and sub-criteria in the hierarchy. 4.5 analyzing the results as one can note while analyzing the results shown in figure 2: the criterion with minor importance was criteria c1 (service by the company), this is possible because the data were collected after a large period after the occupancy had began. as mentioned in section 4.1, the survey was conducted 14 (fourteen) months after the building units had been delivered to their owners. the criteria with major importance are s4 and s5, both related to quality aspects. s4 was related to the quality of the project and s5 related to quality of the finish. into the most important criterion (s5), the most important subcriteria is related to the waterproofing (s5s4) and the second more important is related to the finish of hydraulic and sanitary features. these results are coherent with the classification adopted by the builder firm which has classified the building project as “high-luxurious”. 5. conclusion the original integration of the analytic hierarchy process (ahp) with post-occupancy evaluation (poe) was able to identify and avoid inconsistencies in assessments, contributing to reduce uncertainty in subjective evaluations. this is the main contribution of this work. although some works, such as the recent one by sato (2009), deal with application of ahp to customer’s satisfaction evaluation, it is worth noting that the features which have been proposed in the present work were not found in any previous research related to poe or to ahp. the essay developed in this research points toward the proposal of helping companies in construction industry with a strategic tool, since it allows identifying the worthiest aspects under customers’ point of view, highlighting opportunities for improvement and innovation. ijahp article: costa, correa / construction of an ahp-based model to catch criteria weights in post-occupancy evaluation international journal of the analytic hierarchy process 37 vol. 2, issue 1, 2010 issn 1936-6744 figure 2. weights of criteria and sub-criteria. weight of the  criteria c1 (0.,099) c1s1 (0.,094) c1s2 (0.,281) c1s3 (0.,339) c1s4 (0.,099) c1s5 (0.,187) c2 (0,.217) c2s1 (0.,463) c2s22 (0.,275) c2s3 (0.,262) c3 (0.,197) c3s1 (0.,108) c3s2 (0.,230) c3s32 (0,089) c3s4 (0.,084) c3s5 (0.,370) c3s6 (0.,120) c4 (0.,238) c4s1 (0.,138) c4s1s1 (0.,1308) c4s1s2 (0.,11038)) c4s1s3 (0.,573138) c4s1s4 (0.,18738) c4s21 (0,159) c4s23s1 (0.,10938) c4s32s2 (0.,202138) c4s32s3 (0.,315138) c4s23s4 (0.374,138) c4s31 (0.,162) c4s41 (0.,181) c4s51 (0.,145) c4s5s1 (0.130) c4s5s2 (0.188) c4s5s3 (0.185) c4s5s4 (0.383) c4s5s5 (0.115) c4s61 (0.,215) c5 (0.,249) c5s1 (0.,159) c5s2 (0.,270) c5s2s1 (0.,240) c5s2s2 (0.,370) c5s2s3 (0.,390) c5s3 (0.,147) c5s3s1 (0.,179) c5s3s2 (0.,195) c5s3s3 (0,.521) c5s3s4 (0.,105) c5s4 (0.,274) c5s5 (0.,149) ijahp article: costa, correa / construction of an ahp-based model to catch criteria weights in post-occupancy evaluation international journal of the analytic hierarchy process 38 vol. 2, issue 1, 2010 issn 1936-6744 the main difficulties identified while implementing the proposal were the complexity and the size of the questionnaire. to avoid those difficulties, data collection must be done through interviews supported by a facilitator who is an expert in pairwise comparisons. it is also worth highlighting that the research was neither supported by the construction company nor the condominium administration. these facts may also have contributed to the small number of answers, since, if the respondents had perceived the research as an initiative of those actors as a signal of partnership, they might have felt more encouraged to express their views by identifying an opportunity to express their judgments directly to the building board. it is interesting to state that, as usual in case studies, the weights found for the criteria and sub-criteria sets are valid only for the sample investigated, which has limitations imposed by the context and temporal aspects. it is, therefore, recommended no extrapolation of the results to other situations. it is not a strong limitation to the research, since the focus was not to map the criteria weights, but to produce, apply and evaluate the limitations of an original model to make the link between ahp and poe. for future and further developments, it could be interesting:  to review the set of criteria, including variables linked explicitly with sustainability and social responsibility as well as green construction;  to extend this research, covering a more significant sample. to achieve this goal, it would be interesting to get an explicit support from both: the condominium administration and the construction company. acknowledgment the authors would like to state their thanks to the brazilian research council (cnpq), and also to prof. dr. enrique mu and prof. dr. valerio salomon whose comments have raised the quality of this paper. references aczel, t.l. & saaty,t. (1983). procedures for synthesizing ratio judgment. journal of mathematical psychology, 27, 93–102. barba-romero, s., & pomerol, j.-c. (1997). decisiones multicritério: fundamentos teóricos y utilizacion prática. madrid: universidad de alcalá. bordass, b., & leaman, a. (2005). making feedback and post-occupancy evaluation routine 1: a portfolio of feedback techniques. building research and information, 33(4), 347-352. costa, h.g. (2006). auxílio multicritério à decisão: método ahp. rio de janeiro, brasil: associação brasileira de engenharia de produção (abepro). council, f.f. (2001). learning from our buildings: a state-of-the-practice summary of post-occupancy evaluation. washington, dc: national academy press. formoso, c.t., & lantrelme, e.m.v. (2000). a performance measurement system for construction companies in brazil. international project management journal, 6(3), 54-60. ijahp article: costa, correa / construction of an ahp-based model to catch criteria weights in post-occupancy evaluation international journal of the analytic hierarchy process 39 vol. 2, issue 1, 2010 issn 1936-6744 hadjri, k., & crozier, c. (2009). post-occupancy evaluation: purpose, benefits and barriers. facilities, 27(1-2), 21-33. navarro, g.p. (2006). proposta de sistema de indicadores de desempenho para a gestão da produção em empreendimentos de edificações residenciais. m.sc. thesis. porto alegre, brasil: universidade federal do rio grande do sul. ornstein, s.w., ono, r., lopes, p.a., frança, a.j. g. l., kawakita, c.y., machado, m.d., et al. (2009). performance evaluation of a psychiatric facility in são paulo, brasil. facilities, 27(3-4), 152-167. romero, m. de a.; ornstein, s.w. (2003). avaliação pós-ocupação: métodos e técnicas aplicados à habitação social. porto alegre, brasil: coleção habitare antac, 294. preiser, w.f.e., rabinowitz, h.z., & white, e.t. (1988). post-occupancy evaluation. new york: van nostrand reinhold. saaty, t.l. (1980). the analytic hierarchy process. (w. d. s. e. silva, trans.). pittsburgh, pa: rws publications. sato, y. (2009). how to measure human perception in survey questionnaires. international journal of the analytic hierarchy process, 1(2), 64-82. zagreus, l., huizenga, c., & arens, e. (2004). a web-based poe tool for measuring indoor environmental quality. uc berkeley: center for the built environment. retrieved from: http://escholarship.org/uc/item/56s462z4 zhao, d. (2007). the development of post occupancy evaluation abroad and its feasibility in china. beijing acta scientiarum naturalium universitatis pekinensis, 43(6), 797-802. zimring, c.m. (1987). evaluation of designed environments. new york: van nostrand reinhold. ijahp article: costa, correa / construction of an ahp-based model to catch criteria weights in post-occupancy evaluation international journal of the analytic hierarchy process 40 vol. 2, issue 1, 2010 issn 1936-6744 appendix – questionnaire applied in the survey1 general instructions why this form the main goal of this form is to evaluate the satisfaction with the performance of habitation unit, and also to gauge the importance of same aspects to over the satisfaction and dissatisfaction of the habitants of residential units. the knowledge of your opinion is essential to achieve our goal: offer to our customers a portfolio of products and services that exceed their expectative. instructions to fulfill this questionnaire in this questionnaire you are going to evaluate the importance of a good performance of your residential unit under viewpoints. please use the following scale to make pair wise comparisons about the importance of the aspects over your satisfaction with the residential unit. in the case of two aspects have the same importance to you, please check (i – from “equal importance”). otherwise, express the importance degree by checking it in the column which is closer to the more important criterion under your opinion scale for pairwise comparisons code meaning a absolute more important v very strongly more important s strongly more important m moderately more importance i indifference or equal importance using the scale above, make pair wise comparisons regarding the relative importance of a good performance of your residential unit under the criteria set 0 – overall importance in your opinion, what is the relative importance the following criteria to a good performance of your residential unit, under an overall perspective? criterion a v s m i m s v a criterion service financial service construction service project service finishing financial construction financial project financial finishing construction project construction finishing project finishing i – service by the company in your opinion, regarding to company's service, what are the relative importance of the following criteria to a good performance of your residential unit? criterion a v s m i m s v a criterion responsiveness meeting deadlines responsiveness technical assistance responsiveness prestigious of the construction company responsiveness documentation and user guide meeting deadlines technical assistance meeting deadlines prestigious of the construction company meeting deadlines documentation and user guide technical assistance prestigious of the construction company technical assistance documentation and user guide prestigious of the construction company documentation and user guide 1 the survey was distributed in portuguese. it is translated into english here for the benefit of the readers. ijahp article: costa, correa / construction of an ahp-based model to catch criteria weights in post-occupancy evaluation international journal of the analytic hierarchy process 41 vol. 2, issue 1, 2010 issn 1936-6744 ii – financial aspects in your opinion, regarding to financial aspects, what are the relative importance of the following criteria to a good performance of your residential unit? criterion a v s m i m s v a criterion cost of purchasing cost of maintenance the condominium cost of purchasing cost of maintenance the residential unit cost of maintenance the condominium cost of maintenance the residential unit iii – quality of the construction in your opinion, regarding to the quality of the construction viewpoint, what are the relative importance of the following criteria to a good performance of your residential unit? criterion a v s m i m s v a criterion areas for common use localization of the building areas for common use external appearance of the building areas for common use performance of façade areas for common use security of the building areas for common use conformity with specifications (6) localization of the building external appearance of the building localization of the building performance of façade localization of the building security of the building localization of the building conformity with specifications (6) external appearance of the building performance of façade external appearance of the building security of the building external appearance of the building conformity with specifications (6) performance of façade security of the building performance of façade conformity with specifications (6) security of the building conformity with specifications (6) iv – quality of the residential unit project in your opinion, regarding to the quality of the residential unit’s project, what are the relative importance of the following criteria to a good performance of your residential unit? criterion a v s m i m s v a criterion layout environmental comfort layout quantitative and localization of electrical features layout quantitative and localization of hydraulic and sanitary features layout quality of indoor materials layout project flexibility (6) environmental comfort quantitative and localization of electrical features environmental comfort quantitative and localization of hydraulic and sanitary features environmental comfort quality of indoor materials environmental comfort project flexibility (6) quantitative and localization of electrical features quantitative and localization of hydraulic and sanitary features quantitative and localization of electrical features quality of indoor materials quantitative and localization of electrical features project flexibility (6) quantitative and localization of hydraulic and sanitary features quality of indoor materials quantitative and localization of hydraulic and sanitary features project flexibility (6) quality of indoor materials project flexibility (6) ijahp article: costa, correa / construction of an ahp-based model to catch criteria weights in post-occupancy evaluation international journal of the analytic hierarchy process 42 vol. 2, issue 1, 2010 issn 1936-6744 iv – quality of the residential unit project layout in your opinion, regarding to the layout aspects related in the residential unit’s project, what are the relative importances of the following criteria to a good performance of your residential unit? criterion a v s m i m s v a criterion social areas service areas social areas bedrooms social areas bathrooms service areas bedrooms service areas bathrooms bedrooms bathrooms (4 iv – quality of the residential unit project environmental comfort in your opinion, regarding to the environmental comfort related in the residential unit’s project, what are the relative importances of the following criteria to a good performance of your residential unit? criterion a v s m i m s v a criterion indoor temperature on winter indoor temperature on summer indoor temperature on winter lighting indoor temperature on winter sound insulation $indoor temperature on summer lighting indoor temperature on summer sound insulation lighting sound insulation iv – quality of the residential unit project quality of indoor materials in your opinion, regarding to the quality of indoor materials related in the residential unit’s project, what are the relative importances of the following criteria to a good performance of your residential unit? criterion a v s m i m s v a criterion tiles floorings tiles sanitary ware tiles frames, fixtures and locks tiles paint floorings sanitary ware floorings frames, fixtures and locks floorings paint sanitary ware frames, fixtures and locks sanitary ware paint frames, fixtures and locks paint ijahp article: costa, correa / construction of an ahp-based model to catch criteria weights in post-occupancy evaluation international journal of the analytic hierarchy process 43 vol. 2, issue 1, 2010 issn 1936-6744 v – quality of finish in your opinion, regarding to the quality of finish, what are the relative importance of the following criteria to a good performance of your residential unit criterion a v s m i m s v a criterion quality and finish of electrical features quality and finish of hydraulic and sanitary features quality and finish of electrical features indoor features positioning quality and finish of electrical features waterproofing quality and finish of electrical features achieving specifications for finishing quality and finish of hydraulic and sanitary features indoor features putting quality and finish of hydraulic and sanitary features waterproofing quality and finish of hydraulic and sanitary features achieving specifications for finishing indoor features putting waterproofing indoor features putting achieving specifications for finishing waterproofing achieving specifications for finishing v – quality of finish quality and finish of hydraulic and sanitary features in your opinion, regarding to the quality and finish of hydraulic and sanitary features related to the quality of finish, what are the relative importances of the following criteria to a good performance of your residential unit? criterion a v s m i m s v a criterion materials finish performance of hydraulic features materials finish performance of sanitary features performance of hydraulic features performance of sanitary features v – quality of finish indoor features installation in your opinion, regarding to the indoor features installation related to the quality of finish, what are the relative importances of the following criteria to a good performance of your residential unit? criterion a v s m i m s v a criterion tiles flooring tiles frames and fixtures (doors and windows) tiles paint flooring frames and fixtures (doors and windows) flooring paint frames and fixtures (doors and windows) paint ijahp essay: mu/the past and future of ahp/anp: an isahp2020 perspective international journal of the analytic hierarchy process 592 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.842 the past and future of ahp/anp: an isahp2020 perspective enrique mu the ahp/anp future seems bright for the next generation of scholars and practitioners judging from the themes emerging from the isahp2020 virtual conference in december. in this essay, i will visit the academic themes and messages derived from the keynote and plenary speakers who presented their ideas at the conference. their presentations are available at isahp events/keynote/plenary/ahp roundtable (2020). leadership in complex situations our keynote speaker for isahp2020 was dr. blaženka divjak, her topic was “leadership in complex and chaotic situations.” she was, until recently, the croatian minister for science and education (june 2017 july 2020) and led four major reform processes during her mandate. she has also chaired the eu council of ministers for education and council of ministers for research and space during the croatian presidency (january june 2020). in that period, she led the process of adopting several council conclusions related to future teachers, future jobs and brain circulation as well as coordinating the eu response to the covid-19 crises in education and research. what was remarkable about dr. divjak’s presentation was the way she mapped her journey from decision-making researcher (as university professor) to practitioner (as minister of science and education in croatia) and back to being a researcher. true to this approach, she used the cynefin framework that sorts the issues leaders are facing into five contexts (simple, complicated, complex, chaotic and disorder) defined by the nature of the relationship between cause and effect (snowden & boone, 2007). this is a sensemaking model (snowden, 2002) that simultaneously addresses knowledge and decisionmaking (french, 2013). using this framework, she shared with the audience her experience and leadership challenges as minister of education and later as chair of the eu council of ministers of 27 countries prior to and during the pandemic, which she described as moving from a complicated to a complex situation. dr. divjak recommended that leaders in complex situations must be open to new “out-of-the-box ideas” as well as learning by doing, among other suggestions. she ended by explaining how the ahp/anp (saaty, 1980; saaty, 2005) can help decision-making in complex situations, for example by exploring the decision maker’s values (french, 2013). dr. divjak’s address is a good example of how to combine rigor with relevance, and mcdm analysis with important decisions. dr. divjak proposed a sense-making model as the theoretical framework for the decisional context, along with the ahp/anp as the multicriteria decision making analysis method to revise complex decisions in the context of the pandemic. it was an excellent presentation in strategic leadership and decisionmaking that i am certain will be revisited for years to come. three developments of the ahp the second day of the conference, dr. alessio ishizaka, full professor in decision analysis and head of the supply chain, information systems and decision aid department https://www.youtube.com/playlist?list=pl_j2c3ikvypw7ykrs_aamyu8pno1ifygj ijahp essay: mu/the past and future of ahp/anp: an isahp2020 perspective international journal of the analytic hierarchy process 593 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.842 at neoma business school in france, talked about three important developments of the ahp: calibrated fuzzy ahp, ahpsort and gahpo. he showed that mcda research and publications have been systematically expanding and that the ahp/anp is leading the pack (wallenius, dyer et al. 2008). dr. ishizaka started with the development of calibrated ahp, using as an example of bank current account selection (ishizaka et al. 2013). this approach is based on fuzzy ahp (van laarhoven and pedrycz, 1983) which is a combination of fuzzy set theory (zadeh, 1965) and ahp (saaty, 1980) and was developed, in the view of its creators, to take into account uncertainty and imprecision in evaluations. fuzzy set theory requires the definition of a membership function and ishizaka et al. (2013) proposed a way to calibrate the membership functions with comparisons given by the decision-maker on alternatives with known measures. he explained this technique based on a published case measuring the most important factors in selecting a student current account. while dr. ishizaka addressed the issue of how to construct the membership function in fuzzy ahp, it must be noted that fuzzy ahp is not without detractors. for example, saaty and tran (2007) argued that when judgments are allowed to vary in choice over the values of a fundamental scale, as in the ahp, these judgments are themselves already fuzzy. still, the reality is that fuzzy ahp has firmly established itself as a hybrid methodology that is widely used in many situations. the second development highlighted by dr. ishizaka was ahpsort (ishizaka et al., 2011). when the number of alternatives increases, the number of pairwise comparisons also increases drastically. ahpsort, a variant of the ahp, sorts the alternatives into predefined categories. ahpsort facilitates decision-making within large-scale problems. dr. ishizaka explained the two-stage method using a real case study for supplier selection. first, the large number of candidates is sorted into two classes (accepted or rejected) by comparing pairwise each alternative to the limiting profile established for each screening criterion. this generates a shorter list of adequate suppliers. then, a single supplier is selected with the ahp from among the accepted suppliers using suitable evaluation criteria. the screening criteria ensure the minimum adequacy of the supplier while the evaluation criteria are used to rank the alternatives and require in-depth research into the candidates. some members of the audience argued that the ahp has a solution for the case of a large number of alternatives, the use of an absolute scale based on ratings for the alternatives. however, in my opinion, ahpsort offers the opportunity to keep the cognitive advantages of pairwise comparisons by using it at the following two levels: first, for sorting and next, for evaluation. also, separation of screening criteria from evaluation criteria (at least in the given example) allows the use of the ahp for both the qualification and evaluation stages. dr. ishizaka finished his exposition by discussing the group analytic hierarchy process ordering (gahpo) method (ishizaka and labib, 2011). unfortunately, technical issues did not allow him to present this third ahp development properly during his plenary session. issues and insights for the future the third day of the conference, dr. bill wedley was the plenary speaker. he began by stating the “magic or black box” nature of the ahp/anp. i had never thought of the ijahp essay: mu/the past and future of ahp/anp: an isahp2020 perspective international journal of the analytic hierarchy process 594 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.842 ahp/anp this way, but he explained his rationale. the method is easy to use and intuitive and has a way of fascinating all who see it performed. however, as the models get complex they become a black box for many. dr. wedley proposed opening this black box by discussing nine themes related to the ahp/anp. central to all these themes is the following quote, “and therein lie both the advantages and dilemmas of ahp [and anp]. we do not need explicit knowledge of the underlying unit of measure to derive a ratio scale, yet the derived scale has a unit.” (wedley & choo, 2011; emphasis added). dr. wedley continued by stating that by “advantage” he means that we can get scales from measurements (the pairwise comparisons) without having any previous standard unit. although each pairwise comparison has a unit, we do not know the ultimate unit of the scale that evolves from the comparisons. the “dilemma” is that the sum-to-unity priorities have an obscure unit. the priorities are used, too frequently, without recognition of the unit. accordingly, we often aggregate units incorrectly. dr. wedley argued that the ahp/anp community has not paid enough attention to the units of their scales. next, he discussed the different proposed themes. dr. wedley’s presentation was really an exciting tour over the major themes in the ahp/anp. it would not be possible to cover his discussion in enough detail and fortunately, a video recording of his presentation (as well as those of the other presenters) is available for the interested reader. still, i will summarize two themes that were central to his presentation. first theme: do ahp priorities have a unit? dr. wedley considers this theme to be the most important in his presentation. he started using an example from saaty and shang (2011) and then proceeded to explain the units in the case of ideal and relative priorities. in the case of ideal priorities, if we have priorities (pct , psgt) representing the size of two objects such as a cherry tomato ( pct=0.25) and a small green tomato (psgt = 1), it is clear that the last object is the unit. however, if we express their priorities in a distributive mode (pct=0.2, psgt = 0.8), the unit is an undefined abstract object (the combination of both the cherry tomato and the small green tomato). if we extend the example by adding a third object such as a lime, we can have a new set of priorities for the three objects; however, the unit will be different than before because it will now be a new undefined abstract object (which can be imagined as the morphing of the cherry tomato, small green tomato, and lime into one). we have changed the units! furthermore, when aggregating, things get complicated because the unit of the priority vectors changes when we multiply by a criterion weight or a rescaling factor. to nail this idea in the minds of the audience he quoted zahir (2007), “although the sums are all equal to one, what is often ignored is that “one [1] here is not necessarily equal to one [1] there.” dr. wedley concluded by stating that we should not ignore the unit, nor take it for granted. he admonished us to, “follow what happens to the units.” second theme: use the full intensity of ratios. dr. wedley emphasized that the strength of the ahp/anp is precisely their use of ratios and rather than converting ratio scales to interval scales with false zeros, it is better to think of the decision in terms of ratios. for example, vargas (2016) proposed voting with intensities. dr. wedley also mentioned mu’s (2017) work on eyewitness identification where the questions to the eyewitness are shifted from the likes of “is this the person you saw at the crime scene?” to “how many times is suspect a more likely to be the person you saw at the crime scene than suspect ijahp essay: mu/the past and future of ahp/anp: an isahp2020 perspective international journal of the analytic hierarchy process 595 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.842 b?” he also offered polling, which has recently had very poor performance, as a plausible area to improve by applying the ahp ratio-based approach, as well as in corporate strategies where often those adopted by consensus subsequently fail. he suggests asking for strength of conviction to assess whether people are between lukewarm or extremely committed to a strategy. dr. wedley concluded his presentation with a call to action. he stated that to continue to ignore the priority units is dysfunctional and he emphasized the need to apply the ahp using the full power of its ratio scale rather than trying to devalue it by converting it to a lesser interval scale. he believes that this recognition of the power of the full intensity of ratios could catapult the ahp/anp to new heights. dr. wedley feels positive that the power of ratios has not been fully exploited and because of this there is a bright future for the ahp/anp. after having listened to these wonderful speakers, as well as participating in several sessions for the three days of the conference, i certainly agree with him! ijahp essay: mu/the past and future of ahp/anp: an isahp2020 perspective international journal of the analytic hierarchy process 596 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.842 references bernasconi, m., choirat, c. and seri, r. (2010). the analytic hierarchy process and theory of measurement, management science, 56, 4, 699-711. doi: https://doi.org/10.1287/mnsc.1090.1123 french, s. (2013). cynefin, statistics and decision analysis, journal of the operational research society, 64, 547-561. isahp2020 events/keynote/plenary/ahp roundtable (2020). retrieved from https://www.youtube.com/playlist?list=pl_j2c3ikvypw7ykrs_aamyu8pno1ifygj ishizaka, a., labib, a. (2011). selection of new production facilities with the group analytic hierarchy process ordering method, expert systems with applications, 38, 7317-7325. doi: https://doi.org/10.1016/j.eswa.2010.12.004 ishizaka, a., pearman, c., nemery, p. (2012). ahpsort: an ahp based method for sorting problems, international journal of production research, 50(17), 4767-4784. doi: https://doi.org/10.1080/00207543.2012.657966 ishizaka, a., nguyen, n., h. (2013). calibrated fuzzy ahp for current bank account selection, expert systems with applications, 40(9), 3775-3783. doi: https://doi.org/10.1016/j.eswa.2012.12.089 mu, e., chung, t. r., reed, l. i. (2017). paradigm shift in criminal police lineups: eyewitness identification as multicriteria decision making, international journal of production economics, 184, 95-106. doi: https://doi.org/10.1016/j.ijpe.2016.11.019 saaty, t. l. (1980). decision making for leaders: the analytic hierarchy process. new york: mcgraw-hill. saaty, t., l., shang, j. (2011). an innovative orders-of-magnitude approach to ahpbased multi-criteria decision making: prioritizing divergent intangible humane acts, european journal of operational research, 214, 703-715. doi: https://doi.org/10.1016/j.ejor.2011.05.019 snowden, d. (2002). complex acts of knowingparadox and descriptive self-awareness, journal of knowledge management 6(2), 100-111. snowden, d., boone, m. e. (2007). a leader’s framework for decision making, harvard business review, 69-76. doi: https://doi.org/10.1108/13673270210424639 van laarhoven, p., & pedrycz, w. (1983). a fuzzy extension of saaty’s priority theory. fuzzy sets and systems, 11, 199–227. doi: https://doi.org/10.1016/s01650114(83)80082-7 vargas, l. g. (2016). voting with intensity of preferences, international journal of information technology and decision making, 15(4), 839-859. https://www.youtube.com/playlist?list=pl_j2c3ikvypw7ykrs_aamyu8pno1ifygj ijahp essay: mu/the past and future of ahp/anp: an isahp2020 perspective international journal of the analytic hierarchy process 597 vol. 12 issue 3 2020 issn 1936-6744 https://doi.org/10.13033/ijahp.v12i3.842 wallenius, j., dyer, j., fishburn, p., steuer, r., zionts, s. and deb, k. (2008). multiple criteria decision making, multiattribute utility theory: recent accomplishments and what lies ahead. management science 54(7), 1336 – 1349. doi: https://doi.org/10.1287/mnsc.1070.0838 wedley, w. and choo, (2011). multi-criteria ratios: what is the unit? journal of multicriteria decision analysis, 18(3-4), 161-171. doi: https://doi.org/10.1002/mcda.463 zadeh, l. (1965). fuzzy sets. information and control, 8, 338–353. web-based system for assessing risk factors for falls in community-dwelling elderly people using the analytic hierarchy process ijahp article: pecchia, bath, pendleton, bracale/web-based system for assessing risk factors for falls in elderly people international journal of the 135 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 web-based system for assessing risk factors for falls in community-dwelling elderly people using the analytic hierarchy process leandro pecchia∗ department of biomedical, electronic and telecommunication engineering, university federico ii of naples naples, italy e-mail: leandro.pecchia@unina.it peter a. bath health informatics research group and centre for health information management research (chimr), department of information studies university of sheffield sheffield, uk e-mail: p.a.bath@sheffield.ac.uk neil pendleton school of translational medicine university of manchester manchester, uk e-mail: neil.pendleton@manchester.ac.uk marcello bracale department of biomedical, electronic and telecommunication engineering, university federico ii of naples napoli, italy e-mail: bracale@unina.it abstract falls occur frequently among older people and represent the most common cause of injury-related morbidity and mortality in later life. preventing falls is an important way to reduce injuries, hospitalizations, and injury-related morbidity and mortality among older ∗ corresponding author this project of research, and so far the collaboration between the university of sheffield and napoli, was in part supported by academic award for mobility of ph.d. students from university federico ii. an earlier version of this paper was given at the isahp 2009 conference (pecchia et al., 2009). ijahp article: pecchia, bath, pendleton, bracale/web-based system for assessing risk factors for falls in elderly people international journal of the 136 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 people. the research literature has identified hundreds of risk factors for falls among elderly people. prioritizing risk factors for falls is useful for designing effective and efficacious prevention programs. the aim of this study was to use the analytic hierarchy process to develop a hierarchy of risk factors for falls based on the knowledge and experience of experts working in this field. we designed and developed a web portal for participants to submit responses to electronic questionnaires in order to reach the highest number of respondents quickly and to reduce errors in responding. we contacted the person responsible for the falls sections of four scientific societies. finally, we propose a correction method to modify respondents’ relative importance on based on the coherence of their responses, in order not to exclude experts who had submitted the questionnaire twice. keywords: ahp falls in elderly people, web service 1. introduction evidence-based medicine and health care is becoming increasingly important for assessing the effectiveness of a new health care program. nonetheless, in order to maximize the effectiveness, health care professionals (hcps) need to incorporate empirical and experiential evidence with physiological and biomedical principles, considering also the individual patient’s condition — whom they may know personally — their professional values, and features of the system (tonelli, 2001). the relative weight given to each of these areas is not predetermined, but varies according to specific patient’s condition and the hcps’ experience. therefore, it is important to understand the perspectives of hcps on how they balance factors affecting decision-making and care, and whether there is any consistency among them. everyday practice, in fact, often differs from evidence-based procedures reported through guidelines. our study aims to investigate the opinion of a wide number of professional caregivers, with different specializations and experiences, in facing the complex and multi-factorial problem of preventing falls in home-dwelling elderly people. we used the analytic hierarchy process (ahp) (saaty, 1980), to develop a hierarchy of risk factors for falls based on the knowledge and experience of experts working in this field. another reason leading us to investigate experts’ opinion was that not all falls cause injuries, and for this reason, since we were interested in prioritizing risk factors for any kind of falls, we chose not to base our hierarchy on hospital reports. falls occur frequently among older people and represent the most common cause of injury-related morbidity and mortality in later life (king and tinetti, 1995; nevitt and cummings, 1989). the annual incidence of falls among older people is estimated to be between 15% and 40% in community-dwelling people aged 65 and over. the consequences of falls range from psychological harm (parry and steen, 1982), physical rob typewritten text http://dx.doi.org/10.13033/ijahp.v2i2.61 ijahp article: pecchia, bath, pendleton, bracale/web-based system for assessing risk factors for falls in elderly people international journal of the 137 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 injuries (lord and sherrington, 2007), and hospitalization, to death. falls can also negatively affect well-being, mobility, autonomy, and overall quality of life. preventing falls is therefore an important way to reduce injuries, hospitalizations, and injury-related morbidity and mortality among older people (oakley, 1996). identifying and prioritizing risk factors for falls is useful for designing effective and efficacious prevention programs. the research literature has identified hundreds of risk factors for falls among elderly people (gillespie 2000), and it is well established that falls risk in old age is complex and multi-factorial. moreover, it has been shown that reducing risk factors (tinetti, 1996) and developing multi-factorial interventions (tinetti, 1994) are important strategies for reducing the frequency of falls in home-dwelling elderly people. however, we did not find any reports in the research literature of the views of health care professionals on the relative importance of these risk factors. the aim of our study was to examine the views of health care professionals on the relative importance of risk factors for falls in community-dwelling older people. in order to reach a wider number of experts, we designed and developed a web-based system for respondents to submit questionnaires and to analyze the results. the respondents could not discuss their opinion among themselves to reach a consensus, as we were interested in investigating differences in the opinions of professionals. moreover, we chose not to submit the same questionnaire to the same respondents twice, since this was time consuming for them, as clinicians, and would have reduced the responses we received. this led us to adapt the ahp methods to our specific needs, as is discussed further in this paper. although this represents a limitation of the study, this is nonetheless only the first step in our research and further research could utilize a focus group approach with fewer numbers of experts to reach consensus using a more iterative approach. the aim of this paper is to describe the system we developed and, because we modified the ahp method slightly to adapt it to our specific needs, we present and discuss the effects of these adaptations. in order to demonstrate the effectiveness of the web application and the effect of the proposed adaptations, preliminary results from this research are presented. 2. method the method used to prioritize the risk factors is an application of the analytic hierarchy process (ahp), implemented via the world wide web. we adopted ahp as an analytic decision-making method to understand the risk of falls in older people, a problem that is complex, multi-factorial, and multi-dimensional, and therefore suited to this approach (saaty, 2005). ijahp article: pecchia, bath, pendleton, bracale/web-based system for assessing risk factors for falls in elderly people international journal of the 138 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 2.1 hierarchy of risk factors using the research literature, we identified a range of risk factors for falls, and designed a tree by grouping them into categories (that is, general and clinical) and sub categories (that is, physical, mental, socio-environmental, physical health, drugs and medical conditions). we then developed and piloted a questionnaire asking experts to compare pairs of categories, sub-categories, and factors. the tree of factors with their relative priority weights lead us to the hierarchy. this hierarchic approach allows the construction of a consistent step-by-step framework for decision-making. 2.2 questionnaires in order to reach the highest number of respondents, we designed an electronic questionnaire, located at: http://hosting.vaisuinternet.it/, and a web-based service (pecchia, 2008), to analyze the answers remotely. for each pair of category of risk factors (ri, rj) the respondent was asked the following question: “in your opinion is ri, compared to rj: much more important, moderately more important, equally important, moderately less important, much less important.” we posed similar questions to compare the categories of risk factors. several numerical scales have been proposed, apart from the saaty fundamental scale, such as the geometrical scale (finan, 1999, ji, 2003, lootsma, 1993) and the salo-hamalainen scale (salo, 1997). the saaty scale and the geometrical scale are the most commonly used ones. the saaty scale has been supported by saaty’s empirical evidence, but, as described above, it is not a transitive scale. as demonstrated by dong (2008) et al., the geometrical scale is thought to be transitive; however, as saaty (1994) points out, it is difficult to determine the parameters of the geometrical scale. therefore, in accordance with the natural scale of saaty (table 1), we gave a numerical value to each judgment: table 1 saaty fundamental scale judgments score much more important 5 moderately more important 3 equally important 1 moderately less important -3 much less important -5 piloting the questionnaire, as described below, we decided to use 5 (“much more important”) as the maximum magnitude, since the first group of respondents in the pilot appeared to be unclear about using the larger scale, because respondents reported that, in their opinion, there were not factors dominating the others greater than “much more”. the negative signs were used in order to incorporate the responses into a single question, which factor was dominant and by how much. the negative numbers ( ija ) were then ijahp article: pecchia, bath, pendleton, bracale/web-based system for assessing risk factors for falls in elderly people international journal of the 139 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 transformed, via a web service, in its positive reciprocal ( *ija ), using the transformation in equation 1: ij ij a a 1* −= (1) this relation means that, mathematically, if ri is “moderately less important” than rj ( 5−=ija ), then rj should be much more important than ri ( 5=jia ), therefore the reciprocity of the judgments matrix implies 51=ija . moreover, respondents were permitted to use intermediate judgments (as shown in figure 4), scored with even numbers (positive and negative), to express further insights, or if they could not decide between adjacent categories. 2.3 judgments matrixes with the scores provided by each respondent, the web service automatically evaluated the judgment matrix a. it has been shown that, if the judgments are consistent (see next section for details), the normalized eigenvector of this matrix expresses the relative importance of each risk factor. we iterated this step for each category of risk factor. finally, the same algorithm allows the relative importance of each category of risk factors to be assessed. as explained in the following paragraphs, we chose not to compute the average judgment matrix by calculating the geometrical mean between each element. instead, we evaluated a matrix, and so far a vector of global priorities, for each respondent. this allowed us to present the spread of the opinions of the respondents, reporting a range of global priority weights for each factor. then we computed the algebraic mean of these weightings with the relative importance of each respondent, as stated below. 2.4 consistency from each matrix it is possible to estimate the consistency of the responses from each respondent, in order to test the transitivity of judgments, which is a fundamental hypothesis in each decision making method. obviously, a perfect coherence is difficult to achieve for complex judgments. nevertheless, it is always important to assess the degree of coherence of respondents because a high level of inconsistency by individuals can lead to a low level of consistency in the decision framework. the ahp allows the consistency of any questionnaire responses to be measured by posing some redundant questions (saaty, 2005b, saaty 1996). for instance, to compare the three elements, a, b, and c, the respondent is asked to perform the comparison of b with c ( bca ), which could be deduced by the pair comparisons a-b ( aba ) and a-c ( aca ).the answer is then compared with the deduced judgment and the difference ijahp article: pecchia, bath, pendleton, bracale/web-based system for assessing risk factors for falls in elderly people international journal of the 140 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 represents the degree of inconsistency. if the two judgments are not perfectly consistent, there will be an error, which can be quantified as: ab ac bcbc a a aerror −= (2) this error is zero if the judgments are perfectly consistent. mathematically, the consistency of each response is measured with the error generalized defined as: 11 * ijijij aaaerror −= (3) nevertheless, inconsistency is often due to distractions or loss of interest by the respondent and not to a global inconsistency in the respondent’s opinion. this is particularly true in a web based response system. for this reason, when responses are inconsistent, the questionnaire should be re-submitted to the respondent. this strategy is possible when the respondent can meet with the researchers to discuss and settle inconsistencies in responses; however, this was not possible in this study, the exception to this being the piloting. at this point, we chose a threshold error and excluded respondents with a higher level of inconsistency. then we used the inverse of the consistency index to correct the relative importance of each respondent before pooling the data. this method of pooling techniques is that used in meta-analysis (sutton et al., 2000) finally, we modeled the error as an accuracy-error, which is zero when the framework is completely consistent. an increasing error means a progressive loss of consistency individually and overall. this error is then propagated until the final index of relative importance affecting its precision is estimated. for instance, if a respondent judges a>b and b>c, she/he should judge a>>c. a direct answer of a>c or a>>>c, is not perfectly consistent, but it is certainly more consistent than a<c or a=c. the ahp defines a method to assess the degree of inconsistency. we adopted this technique to correct the relative importance of respondents in order to pool the final data as described above. moreover, we used this error to define the interval of precision of each relative weight. 2.5 assessment of relative importance as stated above, after excluding inconsistent respondents and comparing the risk factors, we calculated the relative importance index (ri) within each sub-category (termed “intra-categorical weights” or “local weights”, lw). from the pair-wise comparisons of categories and sub-categories, we estimated the relative importance of each of them (“inter-categorical weights”, icw). finally, by using both weights, we estimated the global relative importance of each risk factor (“global weights”, gw). all those weights ijahp article: pecchia, bath, pendleton, bracale/web-based system for assessing risk factors for falls in elderly people international journal of the 141 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 are affected by the error described above, and have to be reported with an imprecision of less than, or equal to, the threshold-error. 2.6 the web system to reach a wider range of respondents, we designed and developed a web portal for completion of the questionnaires, and a web service for processing the data, analyzing the results and pooling. the whole system, follows the so-called “three tier layer” architecture, and is organized in three areas (see figure 1): the “client area” that acts as presentation tier, the “server area” as business tier and data tier and the “web service area” as a pure business tier level. each of these areas is independent of the others and can operate with other information systems using common standards. in summary, the “client area” aims to collect data and presents an elaboration of the results; the “server area” to manage, to store and to retrieve data; the “web service area” to process raw data and transform it from the judgment matrix to relative importance weights. moreover, in this area, a web service analyzes data regarding respondent experience and, following the system described above, executes the data pooling. figure 1 three “areas” of the web-based system: client, server and web services 2.7 data pooling we pooled global priorities using a weighted average of the judgment of each respondent. we weighted each respondent ( rw ) based on her/his experience, taking into account the following information about respondents: years since specialization, level of education, area of work. these three features were considered equally important, while table 2 presents the weights assigned to each element of each feature. this assignment was chosen according to the hierarchy proposed by saaty (saaty, 2010). the “relative importance weight” is defined as wr/min(wr), so that it is clear that a ph.d. is ijahp article: pecchia, bath, pendleton, bracale/web-based system for assessing risk factors for falls in elderly people international journal of the 142 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 considered about 12 times more important than a professional qualification. although this normalization is more sensible to variation in case new features are inserted, it is much easier to communicate to respondents and decision-makers not skilled in mathematical methods, which was the priority in this study. table 2 weighting assigned according to time since qualifying, highest educational qualification and area of work feature weight ( rw ) relative importance weight years since qualification >15 0.61 15.3 6-15 0.25 6.3 3-6 0.10 2.5 1-2 0.04 1.0 education ph.d., md, or equivalent 0.59 11.8 msc or equivalent 0.25 5.0 bcs or equivalent 0.11 2.2 professional qualification 0.05 1.0 area of work falls health services/studies 0.57 4.1 elderly health services/studies 0.29 2.1 other 0.14 1.0 2.8 correction of respondents relative importance in case of not perfect coherence as described above, we submitted the questionnaires to the respondents, who were not in a single location. at this point, we could not ask them to resolve divergent judgments in order to achieve a consensus. moreover, we preferred not to exclude respondents neither to ask them to complete the questions again, because of the limited time they had available. for these reasons we introduced a correction method in order to accept respondents with a cr over the suggested threshold of 0.1. then we used the exceeding δcr to correct the relative importance weight ( *rw ) of respondents using the formula (3). ( ) ] ]    ∈∆∆∗−∗= ≤∆= −=∆ 1.0;0101 0 1.0 * * crifcrww crifww crcr rr rr (4) we adopted a threshold of 0.1 because the maximum size of elements compared in one category is seven (saaty, 2003, saaty, 2010). nonetheless, this threshold could in future be related to the size of the matrix. ijahp article: pecchia, bath, pendleton, bracale/web-based system for assessing risk factors for falls in elderly people international journal of the 143 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 3. results and discussion 3.1 hierarchy of risk factors from the research literature, we individuated a set of 39 risk factors (tinetti, 1996, tinetti. 1994, gillespie, 2000), which was reduced to 35 during the pilot study, based on feedback from the respondents. five factors were excluded (“falls in the previous 12 months,” “capacity to describe causes of previous fall’s,” “loss of control,” “structural diseases,” “cardiovascular medicine”) because they were considered confusing or repetitive with respect to other factors and a two additional factors were introduced (“poor self rated health” and “previous syncope”) at the suggestion of those participating in the pilot study. we organized the 35 factors into categories and sub-categories by developing a tree. in the research literature, various studies have investigated risk factors for falls. however, few authors have proposed classifications to categorize risk factors (panel of falls prevention, 2001) and none is based on the opinion of experts from different specializations. the tree we designed is shown in figure 2. figure 2 tree of risk factors we considered general risk factors (table 3) to be those that might concern any older person (including healthy older people), and which include environmental factors. table 3 general risk factors organized according to each sub-category physical mental socio-environmental high body mass index (bmi) fear of falls need help toileting loss of weight depression low family support poor joint flexibility early stage dementia low social engagement low walking speed loss of balance low social service support low muscular strength low cognitive perception need to use stairs/steps in home low level of physical activity perceived risk of falls poor self rated health this category was further split into three sub-categories: physical, which includes those factors associated with an individual’s stature and capacity; mental, which includes psychological factors associated with aging; socio-environmental, which includes factors related to the living arrangements of the person. general physical physical health mental drugs socio environmental medical condition clinical risk factors for falls ijahp article: pecchia, bath, pendleton, bracale/web-based system for assessing risk factors for falls in elderly people international journal of the 144 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 clinical risk factors concern frail older people with various health problems (table 4). this group includes three sub-categories called: physical health factors, which are, in different proportions present in healthy, as well as in pathological subjects; drugs, which embrace different medications; medical conditions, which include diseases typically affecting elderly people. table 4 clinical risk factors organized according to each sub-category physical health drugs medical condition visual problems anti-depressives postural hypotension continence problems anti-psychotics nervous system disease dizziness beta-blockers musculoskeletal disease mobility aids diuretics stroke sleeping problems sedatives polypharmacy previous syncope taking any prescribed drugs 3.2 questionnaires as described above, we organized the 35 risk factors into 2 categories and six subcategories. at this point, we needed seven questionnaires, six of which assessed local weights and one to assess category weights. we first used a paper-based questionnaire in order to pilot it in-lab. we then designed and implemented a web page for each questionnaire to reach the highest number of respondents as described below. the time taken to complete the paper version was almost double that for the web-based version: the mean time to complete the electronic questionnaires was 20 (±12) minutes for the respondents involved in the technical pilot, to 27(±14) minutes for the recruited experts, compared to a mean time of 56(±16) minutes for the paper version. additionally, we encountered several errors in the completed paper versions, due perhaps to the limited experience of the respondents with such a specific layout, even though we provided a detailed explanation at the start of each questionnaire, including full examples as shown in figure 3. figure 3 example provided during the piloting of the paper-version of the questionnaire example for example, if you think that in public health initiatives to reduce the spread of hiv/aids, “promoting safe sex” is much more important than “having a needle bank” then mark the two boxes as follows: promoting safe sex > = < having a needle bank 1 2 3 4 5 6 7 8 9 instead if you think that “having a needle bank” is between much more important and very much more important than “promoting safe sex” then please mark the two boxes as follows: promoting safe sex > = < having a needle bank 1 2 3 4 5 6 7 8 9 or, if you think that “having a needle bank” is equally important to “promoting safe sex” then please mark the two boxes as follows: promoting safe sex > = < having a needle bank 1 2 3 4 5 6 7 8 9 ijahp article: pecchia, bath, pendleton, bracale/web-based system for assessing risk factors for falls in elderly people international journal of the 145 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 in the electronic version no compilation errors were possible due to the use of “combobox” objects (figure 4), which allows only a natural scale to be used instead of “sign and magnitude” to express pairwise judgments. figure 4 combobox used to demonstrate responses. in addition, we decided to randomize the order of pairwise comparisons to avoid automatic responses from respondents who might get tired of completing a long questionnaire. moreover, we ensured that we had the same number of recurrences of each risk factor to the left and to the right of the comparison. we did this in order to reduce possible biases due to the tendency of inattentive respondents to correlate the importance of factors to their usual reading. figure 5 and figure 6 show the same questions implemented in the paper version and in the electronic one respectively. figure 5 paper version of questionnaire one set of questions from the paper version is shown in figure 5. we randomized the order of pair wise comparisons and ensured that each factor was twice on the left and twice on the right. ijahp article: pecchia, bath, pendleton, bracale/web-based system for assessing risk factors for falls in elderly people international journal of the 146 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 figure 6 electronic version of questionnaire the same set of questions in figure 5 is shown as they are implemented in the electronic version in figure 6. the drop-down boxes provide options for the respondents to select the appropriate response category. finally, the elaboration of responses performed via the web services, as described further in the paper, allowed the post-elaboration time and the risk of error due to manual transcription to be reduced; this was especially important with this number of respondents. it is well known that the wording of the questions, the order in which they are asked, and the number and form of alternative answers offered can influence the results of surveys. this is particularly important when the questionnaire is submitted via the web with no direct feedback from the responders. for this reason, we tried to reduce wording bias by adopting some forethought. as is evident in figures 5 and 6, in each section of the questionnaire all the rfs appear the same number of times on the right and on the left, so that the respondent could be not influenced by the impression of comparing the rf with the “other.” also the order of the questions is random and neither the same rf appears in two consecutive questions, so that the respondent could be not influenced because of comparing a rf in sequence with all the remaining rfs. all the words used in the questionnaire are technical and selected from literature, so that their meanings and connotations could be clear and familiar to respondents. in order to minimize nonresponse bias, we recruited experts through the “falls” section of national scientific societies of geriatricians (british geriatrics society) and physiotherapists (agile). we have no reason to believe that those specialists who did not respond are any different from the ones who did respond. therefore, we feel it is reasonable to assume that the ijahp article: pecchia, bath, pendleton, bracale/web-based system for assessing risk factors for falls in elderly people international journal of the 147 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 final results are minimally affected by non-response bias related to the respondents’ professional background. 3.3 respondents and execution of the study research ethics approval was obtained for this study from the university of sheffield. a technical pilot study involving 32 respondents was first performed in our labs to define the editorial model while trying to minimize the risk of errors. a scientific pilot study involving a group of nine experts, with different backgrounds and specializations, then completed the questionnaires independently. all nine respondents had working experience in the field of falls in the care of elderly people. four physicians (comprising a consultant geriatrician with 11 years’ experience, a general practitioners/family doctors with 28 years’ experience, a md who specialized as a gerontologist with 28 years’ experience, and a geriatrician with 22), four physiotherapists (with 10, 12, 13 and 13 years’ experience) and one professor of physiotherapy comprised the group. an invitation letter, containing a link to the questionnaire, was sent by email via the moderator of the email distribution list for the british geriatrics society (bgs) falls special interest group, the agila chartered society of physiotherapy working with older people, and to a program of work focused on falls preventions called the “preventing falls program.” the link contained a “get” variable to track the association of the respondent, so we know that, from the 196 final respondents who visited the web questionnaire, 163 were experts from these groups. of these, 68 (41.7%) completed the questionnaires. table 5 gives details about all respondent background. table 5 number of respondents during piloting and the final group of “consistent” respondents. technical piloting scientific piloting final respondent experts university 31 1 2 physicians 1 4 12 physiotherapists 4 44 nurses 10 total 32 9 68 from the group of “final respondent experts” and the “scientific piloting group”, 29% worked in “falls services” and 42% worked within “general elderly care services”. 16% reported having an msc and 8% reported having a ph.d. or md. the mean number of “years since qualification” was 18.9 (±10.56). of the 68 final respondents, 10 (14.7%) completed the questionnaire consistently with a cr <0.1, as suggested by saaty, in all subcategories. moreover 24 (35.3%) exceeded the recommended threshold level in just one section out of seven, with a cr mean of 0.14(±0.03). as explained above, because we preferred neither to exclude respondents ijahp article: pecchia, bath, pendleton, bracale/web-based system for assessing risk factors for falls in elderly people international journal of the 148 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 nor to ask them to answer again, we used a correction method for the respondents who answered with a ( )2.0;1.0∈cr . we increased the cr level, using the δcr to correct the relative importance of respondents as described in formula (3). we also included respondents with a cr≤0.2, (n=37; 54.5%) with such a cr in all seven sections of the questionnaires, and 62 (91.2%) if we include the ones that achieved cr in six of the seven sections of the questionnaires. 3.4 the web system the aim of the web-based system was to reach as wide a range of respondents as possible. we designed the complete system using .net web technologies, particularly so that we could easily implement controls to avoid compilation errors. for example, the system did not allow a respondent to proceed to the following section of the questionnaire if they had not answered all of the questions in the current section. figure 7 summarizes the architecture of the implemented platform and the logic of the system, which follows the three-tier level model outlined in figure 1. first, information about the respondent is requested. the system uses this information to calculate the weighting for the respondent. this weighting is further corrected (reduced) if the respondent’s cr is between 0.1 and 0.2, using formula 3 presented above. the respondent was then asked to prioritize risk factors within each sub-class (first the general risk factors followed by the clinical ones). finally, she/he is asked to prioritize each sub-class and class. the system uses the responses to compose the priority matrices, from which it calculates relative priority weights and cr. if the cr is greater than the threshold of 0.2 in at least one matrix, the respondent is excluded from the final pooling. if the cr is between 0.1 and 0.2 in at least one matrix, the respondent’s weighting is lowered according to formula 4. by weighting the priority of risk factors according to the relative importance of the respondent, the system computes and stores the global weights of each factor. ijahp article: pecchia, bath, pendleton, bracale/web-based system for assessing risk factors for falls in elderly people international journal of the 149 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 figure 7 web system architecture and service logic 3.5 data pooling as discussed in the previous paragraphs, to pool the results obtained from different experts it is important to weight each feature of the respondent. the adopted weighting system is summarized in table 2. we assumed that the three features could be considered to be of equally importance. 3.6 fully coherent respondents the results from the 10 respondents with a cr <0.1 in each matrix are presented in table 6. in this table for each sub-category we reported: its parent category (see figure 2); the mean among respondents of the relative importance normalized using the distributive mode; the range among respondents of the relative importance normalized using the distributive mode; its global weight (gw), which is the mean value among respondents of the relative importance of the sub-category, after weighting each respondent with the scoring system introduced in section 2.7; its relative importance weight (riw), which is the gw normalized with the minimum the gws among all the sub-categories [gw/min(gw)]. physical mental social general risk factors physical health medical drugs clinical risk factors db * sub categories respondent info global weights global weights categories home.aspx q1.aspx q2.aspx q3.aspx q4.aspx q5.aspx q6.aspx q7.aspx errorerror weigh correctionweigh correction respondent weightingrespondent weighting >0.2 matrix evaluationmatrix evaluation eigenevaluation clusters weightsclusters weights error error estimation matrix evaluationmatrix evaluation eigenevaluation local weightslocal weights error error estimation ( )2.0;1.0 ),2.0( +∞ p re se nt at io n la ye r b u si ne ss la ye r d at a la ye r ( )2.0;1.0 nornor and or ),2.0( +∞ ( )2.0;1.0 ijahp article: pecchia, bath, pendleton, bracale/web-based system for assessing risk factors for falls in elderly people international journal of the 150 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 table 6 global weights of sub-categories of risk factors sub-category category mean (%) range (%) gw (%) riw mental general 19.1 12-26 20.2 1.5 physical general 19.1 12-26 20.2 1.5 physical health clinical 16.4 11-22 15.5 1.2 drugs clinical 16.2 10-23 15.3 1.2 medical clinical 13.7 7-21 13.1 1.0 socio-environmental general 15.6 10-21 15.6 1.2 gw: global weight; riw: relative importance weight. table 6 shows that, for all the sub-categories, a score (the “inter-categorical weights”) was evaluated for each respondent and then mediated (mean) among all respondents. these values were then weighted with the relative importance of each respondent to obtain the global weight (gw) of each sub-category. the relative importance weight (riw) was estimated by dividing the gws for the minimum sub-category, which is “clinical-medical subcategories” in this case. the riw reflects how much, in the opinion of respondents, each sub-category is more important than the less important one. although this normalization is less stable than is ideal in case one element in the hierarchy is changed, its results are easier to communicate to physicians. finally, the range gives an indication of the data dispersion, which reflects the differences in the opinions of the respondents. these differences are also due to the general nature of the questions asked in this study; for example, if the questions were more focused on a particular case (for example, a patient suffering from specific condition, or using a more specific definition of a fall), it is possible that the respondents’ opinions may have been less diverse. in fact, different groups of community-dwelling older people could have relatively different risk factors for falls. nonetheless, the design of the questionnaire was done in such way because we were interested in any kind of falls. furthermore, measure divergence among the respondents’ opinions using ahp is a primary outcome of the study. ijahp article: pecchia, bath, pendleton, bracale/web-based system for assessing risk factors for falls in elderly people international journal of the 151 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 table 7 risk factors risk factor category sub-category mean (%) range (%) gw (%) riw low muscular strength general physical 1.77 1.00-2.54 1.89 5.9 low level of physical activity general physical 1.61 0.93-2.29 1.72 5.3 loss of balance general mental 1.53 0.70-2.37 1.62 5.0 need help toileting general socio-environmental 1.56 0.67-2.46 1.59 4.9 fear of falls general mental 1.34 0.55-2.13 1.40 4.3 low walking speed general physical 1.03 0.27-1.78 1.13 3.5 poor joint flexibility general physical 1.12 0.75-1.49 1.12 3.5 use of sedatives clinical drugs 1.11 0.57-1.66 1.10 3.4 use of antipsychotics clinical drugs 1.07 0.58-1.57 1.08 3.3 early stage dementia general mental 0.96 0.38-1.53 1.07 3.3 use of anti-depressives clinical drugs 1.07 0.50-1.64 1.06 3.3 low cognitive perception general mental 0.97 0.62-1.33 1.03 3.2 dizziness clinical health physical 1.09 0.57-1.61 1.01 3.1 perceived risk of falls general mental 0.96 0.32-1.60 1.01 3.1 visual problems clinical health physical 1.06 0.59-1.53 1.00 3.1 table 7 presents the relative importance of the first 15 individual risk factors. we evaluated the same synthetic parameters described above. riw expresses the relative importance of each risk factor normalized to the least important factor (“low social services support”). the range shown in the previous tables is too high to identify clearly the classification between each consecutive risk factor. this may also be due to the high number of risk factors. nonetheless, it is clearly discernible that there is consensus that some factors are much more important than others. an additional investigation could focus on these factors in order to provide a classification that is more selective. 3.7 correction of respondents’ relative importance in case of not perfect coherence by applying the correction introduced above, we included 62 respondents obtaining the following results for the sub categories, as shown in tables 8 and 9. ijahp article: pecchia, bath, pendleton, bracale/web-based system for assessing risk factors for falls in elderly people international journal of the 152 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 table 8 global weights of sub-categories of risk factors sub category category mean (%) range (%) gw (%) riw mental general 22.6 13-32 23.1 1.8 physical general 18.0 11-25 18.3 1.4 physical health clinical 18.0 11-26 16.7 1.3 drugs clinical 16.1 11-22 15.5 1.2 medical clinical 13.2 7-20 13.6 1.1 socio-environmental general 12.0 7-17 12.8 1.0 comparing table 9 with table 7 demonstrates that the ranks of classified categories of rfs do not change much with the exception of “socio-environmental”. in fact, the first five categories are significantly correlated with a spearman’s rank correlation coefficient of 97% (p<0.05). nevertheless, the gws of consecutive categories are too close, in comparison to the wide ranges observed (third columns in table 7 and 9), which reflect divergence in the opinion of the respondents. divergence among the respondents’ opinions is a primary outcome of the study, which was revealed by the ahp. table 9 first 15 risk factors with error corrections risk factor category sub-category mean (%) range (%) gw (%) riw loss of balance general mental 1.90 0.61-3.20 2.10 5.9 fear of falls general mental 1.79 0.82-2.76 1.75 4.9 low muscular strength general physical 1.65 0.86-2.44 1.68 4.7 low level of physical activity general physical 1.33 0.68-1.98 1.40 3.9 need help toileting general socio-environmental 1.17 0.53-1.82 1.28 3.6 sedatives clinical drugs 1.32 0.84-1.80 1.23 3.4 perceived risk of falls general mental 1.22 0.56-1.88 1.19 3.3 dizziness clinical health physical 1.31 0.43-2.20 1.16 3.2 visual problems clinical health physical 1.31 0.56-2.06 1.15 3.2 antipsychotics clinical drugs 1.06 0.61-1.51 1.09 3.0 poor joint flexibility general physical 1.14 0.44-1.83 1.06 3.0 early stage dementia general mental 1.07 0.24-1.90 1.06 3.0 low cognitive perception general mental 0.99 0.55-1.43 1.02 2.8 low walking speed general physical 0.94 0.30-1.58 1.00 2.8 postural hypotension clinical medical 0.98 0.39-1.57 0.99 2.8 in addition, the results on risk factors prioritization do not change substantially, when comparing table 9 with table 7. among the first 15 risk factors, 14 are the same in both groups of respondents. the first five are almost in the same order. nonetheless, the difference between the riws of consecutive factors is minor, and comparable, to the range. this leads us to conclude that a detailed classification of all the factors could be not fully statistically significant, due to the divergence in the opinions of respondents. ijahp article: pecchia, bath, pendleton, bracale/web-based system for assessing risk factors for falls in elderly people international journal of the 153 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 moreover, both groups of respondent were in agreement in classifying drugs “high bmi” and “taking any prescribed drugs” as the two factors with the lowest priority for falls risk in elderly people, and the final ten factors are the same. 3.8 comparison with previous studies a limited number of studies (rubenstein, 1996, rubenstein, 2006, masud 2001) have proposed a classification of the most likely causes of falls in elderly people, based on hospital records. they based their classification on the mean percentage of a cause’s prevalence, calculated from the 3,628 falls reported across 12 studies. these studies considered only few of the rfs that we included in our hierarchy. among these, particularly important causes of falls reported were (in decreasing order of importance): (1) gait/balance disorders or weakness (prevalence 17%, range among studies 4-39%); (2) dizziness/vertigo (13%, 0-30%); (3) confusion (5%, 0-14%); (4) postural hypotension (3%, 0-24%); (5) visual disorders (2%, 0-5%). comparison of our results with this classification is not straightforward. one of the reasons for this is that causes of falls can also be the dependent on other risk factors. an example is drug consumption, which is not considered in these studies, but which may cause vertigo, and which is itself a risk factor. nonetheless, it is possible to estimate the gws of these five causes using tables 7 and 9. in fact, “balance disorders or weakness” are related to other risk factors such as “low muscular strength,” and “loss of balance.” therefore, the gw of these would be 3.51 or 4.00, by summing the gws of these two risk factors from tables 7 and 9, respectively. similarly, “dizziness/vertigo” is related to the risk factor “dizziness” and can be caused by “use of sedatives.” therefore, the gw of this would be 2.11 or 2.39, by summing the gws of these two risk factors from tables 7 and 9, respectively. moreover, “confusion” is related to “low cognitive perception” and can be caused by “early stage dementia.” therefore, the gw of would be 2.10 or 2.08, by summing the gws of these two risk factors from tables 7 and 9 respectively. we considered “postural hypotension” and “visual problems” as independent rf, but in this case there is an inversion between these two rfs between our classifications and the one proposed in previous studies. in fact, postural hypotension is not in the first fifteen rfs of table 7, and is considered less important than visual problems in table 9. although it is not possible to compare our results with previous classifications directly, this proposed prioritization, shows empirically that our results are broadly consistent with ones obtained from previous studies, at least regarding the most important causes of falls. moreover, our research provides further insights regarding the hierarchy of risk factors and produces a measure of the divergence of opinion among experts. 3.9 limitations of the study as noted in section 3.2, the wording of the questions can influence the results of surveys. this is of particular importance when the questionnaire is submitted via the web with no direct feedback or contact with the respondents. for this reason, we tried to reduce any ijahp article: pecchia, bath, pendleton, bracale/web-based system for assessing risk factors for falls in elderly people international journal of the 154 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 bias in the wording by editing the questionnaire as described in section 3.2. nonetheless, possible bias in the wording, the sample size and not re-submitting the questionnaires to respondents with inconsistent responses, are the main limitations of this study. further research could use a focus group with smaller number of experts to reach consensus using a more iterative approach. 4. conclusions in the study described in this paper, we investigated an adaptation of the ahp and how it could contribute to research assessing the priority of risk factors for falls in homedwelling elderly people. our conclusions are that the ahp can contribute to assessing the hierarchy of these risk factors. moreover, it is useful to investigate a relatively broad spectrum of opinions across respondents. this could be due to the different backgrounds of the respondents, and we are investigating this hypothesis in an attempt to quantify any differences. the high number of risk factor individuated, the low difference between consecutive riws, and the relative high spread between the respondents’ opinions do not allow us to identify a statistically significant punctual-scale of risk factors. nonetheless, this study allows us clearly to individuate the factors considered the more relevant for falls compared with the less relevant ones. further research could ask respondents to prioritize the most important risk factors. nonetheless, the difference of opinion between respondents is in itself a primary outcome of this study. further studies should try to investigate the reasons for this. the implementation of the web system to submit blind questionnaires, allows a wide number of respondents to be reached. in addition, the time to complete the questionnaire was almost halved when the electronic version was used. finally, no errors of compilation were registered, and users who completed both versions reported that the electronic version was more user-friendly and easier to use than the paper version. the method suggested to correct the weights attributed to each respondent on the basis of its consistency (cr), allowed us to include a wider number of respondent without dramatic loss of significance in the final scale. the next version of the web system will support a control of coherence, which in real time will suggest and underline respondents’ inconsistencies. this control will ask respondents to review the comparison which presents the highest error, by comparing the ratio of the two corresponding elements of the eigenvectors to the corresponding value of the judgments matrix, as suggested by saaty (1980, 1996). references ags/bgs/aaos panel on falls prevention (2001). guideline for the prevention of falls in older persons. journal of the american geriatrics society, 49: 664–72. ijahp article: pecchia, bath, pendleton, bracale/web-based system for assessing risk factors for falls in elderly people international journal of the 155 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 dong y.c., xu y.f., li, h.y., et al. (2008). a comparative study of the numerical scales and the prioritization methods in ahp. european journal of operational research, 186(1): 229-242. gillespie, l.d., gillespie, w.j., cumming, r., lamb, s.e., & rowe, b.h. (2000). interventions for preventing falls in the elderly. cochrane database syst rev., issue: 2 pages: cd000340 published: 2000. ji p., & jiang, r. (2003). scale transitivity in the ahp, journal of the operational research society, 54 896–905. finan, j.s., & hurley, w.j. (1999). transitive calibration of the ahp verbal scale. european journal of operational research, 112 367–372. king, m.b., & tinetti, m.e. (1995). falls in community-dwelling older persons. journal of the american geriatrisc society, 43:1146-1154. lootsma, f.a. (1993). scale sensitivity in the multiplicative ahp and smart, journal of multi-criteria decision analysis, 2 87–110. lord, s.r., sherrington, c., menz, h.b., & close, j.c.t. (2007). falls in older people: risk factors and strategies for prevention. 2nd edition. cambridge university press. masud, t., & morris r.o. (2001). epidemiology of falls, age and ageing; 30-s4: 3–7. nevitt, m.c., cummings, s.r., kidd, s., & black, d. (1989). risk factors for recurrent non-syncopal falls. a prospective study. journal of the american medical association, 261: 2663–8. oakley, a; dawson, mf; holland, j, et al. (1996). preventing falls and subsequent injury in older people. quality in health care. 5(4): 243-249 parry, s.w., steen, n., galloway, s. r., kenny, r.a., & bond, j. (2001). falls and confidence related quality of life outcome measures in an older british cohort, postgraduate medical journal, 77:103–108. pecchia, l., mirarchi, l., torino s., & bracale m. (2008). assessment of quality control in us: a web service for preventive maintenance, international journal of cars 3 (suppl 1):s9-s10. ijahp article: pecchia, bath, pendleton, bracale/web-based system for assessing risk factors for falls in elderly people international journal of the 156 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 pecchia, l., bath, p.a., pendleton, n., bracale, m. (2009). ahp and risk management: a case study for assessing risk factors for falls in community-dwelling older patients. proceedings isahp 2009 symposium, july 29–august 1, 2009, pittsburgh, pa. rubenstein, l.z. (2006). falls in older people: epidemiology, risk factors and strategies for prevention. age and ageing, 35-s2: ii37–ii41. rubenstein, l.z., & josephson, k.r. (1996). falls. in kenny r.a. ed. syncope in the older patient. london: chapman and hall. saaty, t.l., & özdemir m.s. (2005). the encyclicon, a dictionary of decisions with dependence and feedback based on the analytic network process. pittsburgh, pa: rws publications. saaty, t.l. (1980). the analytic hierarchy process: planning, priority setting, resource allocation. new york: mcgraw-hill. saaty, t.l. (1996). the fundamentals of decision making and priority theory with the analytic hierarchy process, vol. iv of the analytic hierarchy process series. pittsburgh, pa: rws publications. saaty, t.l. (2005). theory and applications of the analytic network process: decision making with benefits, opportunities, costs and risks. pittsburgh, pa: rws publications. saaty, t.l. & özdemir, m.s. (2003). why the magic number seven plus or minus two. mathematical and computer modelling, 38:233-244. saaty ,t.l. (1994). highlights and critical points in the theory and application of the analytic hierarchy process. european journal of operational research, 74 426– 447. saaty, t.l. (2010). principia mathematica decernendi: mathematical principles of decision making. pittsburgh, pa: rws publications. salo, a.a., & hamalainen r.p. (1997). on the measurement of preferences in the analytic hierarchy process. journal of multi-criteria decision analysis, 6, 309–319. sutton, a.j., abrams, k.r., jones, d.r., sheldon, t.a., & song, f. (2000). methods for meta-analysis in medical research. chichester, uk: john wiley. ijahp article: pecchia, bath, pendleton, bracale/web-based system for assessing risk factors for falls in elderly people international journal of the 157 vol. 2, issue 2, 2010 analytic hierarchy process issn 1936-6744 tinetti, m.e., baker, d.i., mcavay, g., et al. (1994). a multifactorial intervention to reduce the risk of falling among elderly people living in the community. new england journal of medicine, 331:821–7. tinetti, m.e., mcavay, g., & claus, e. (1996). does multiple risk factor reduction explain the reduction in fall rate in the yale ficsit trial. american journal of epidemiology,144:389–99. tonelli, m.r. (2001). the limits of evidence-based medicine. respiratory care, dec; 46(12):1435-40.